2024-05-26T22:34:48.472 INFO:root:teuthology version: 0.0.1.dev291+ge7ae758 2024-05-26T22:34:48.472 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/exports} --verbose --owner scheduled_teuthology@teuthology --archive /home/teuthworker/archive/teuthology-2024-05-26_21:24:02-fs-squid-distro-default-smithi/7727073 --name teuthology-2024-05-26_21:24:02-fs-squid-distro-default-smithi -- /home/teuthworker/archive/teuthology-2024-05-26_21:24:02-fs-squid-distro-default-smithi/7727073/orig.config.yaml 2024-05-26T22:34:48.518 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-26T22:34:48.611 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-05-26_21:24:02-fs-squid-distro-default-smithi/7727073 branch: squid description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/exports} email: ceph-qa@ceph.com first_in_suite: false job_id: '7727073' kernel: client: branch: testing kdb: 1 sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: teuthology-2024-05-26_21:24:02-fs-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Replacing daemon mds sha1: 0ef46a246e1118e5adafa1857b4189e7d780c816 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 0ef46a246e1118e5adafa1857b4189e7d780c816 kclient: syntax: v2 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 subvolume_version: 2 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: 7d7cb32d1d130135a98121a564d709025c2154b6 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 1555 sha1: 0ef46a246e1118e5adafa1857b4189e7d780c816 sleep_before_teardown: 0 subset: 10/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_7d7cb32d1d130135a98121a564d709025c2154b6/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 7d7cb32d1d130135a98121a564d709025c2154b6 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi131.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBM+HHdqoC8qRnkJTvrEdj3BbvQXlGnopC9WZsb4L3ON/a+jU2V1bp17hNvc2hmkmrRCpcdsegdRK2GiGTBht86E= smithi187.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBau4rbBJGj2tQGP8Jid05zFpRKpZrP02Gdl97eTNpSIdAVmKwCB5s42zI05KYj+KQSZgVOqEBuYykhgUBFGmvY= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - kclient: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_exports teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: e7ae7580c417d6ab27fda98fecadca5d9f85d63e timestamp: 2024-05-26_21:24:02 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2698841 2024-05-26T22:34:48.611 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_7d7cb32d1d130135a98121a564d709025c2154b6/qa; will attempt to use it 2024-05-26T22:34:48.612 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_7d7cb32d1d130135a98121a564d709025c2154b6/qa/tasks 2024-05-26T22:34:48.612 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-05-26T22:34:48.613 INFO:teuthology.task.internal:Checking packages... 2024-05-26T22:34:48.635 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '0ef46a246e1118e5adafa1857b4189e7d780c816' 2024-05-26T22:34:48.635 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-26T22:34:48.635 INFO:teuthology.packaging:ref: None 2024-05-26T22:34:48.635 INFO:teuthology.packaging:tag: None 2024-05-26T22:34:48.636 INFO:teuthology.packaging:branch: squid 2024-05-26T22:34:48.636 INFO:teuthology.packaging:sha1: 0ef46a246e1118e5adafa1857b4189e7d780c816 2024-05-26T22:34:48.636 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-05-26T22:34:48.947 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2859.g7d7cb32d 2024-05-26T22:34:48.949 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-05-26T22:34:48.955 INFO:teuthology.task.internal:no buildpackages task found 2024-05-26T22:34:48.955 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-05-26T22:34:48.982 INFO:teuthology.task.internal:Saving configuration 2024-05-26T22:34:48.998 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-05-26T22:34:49.015 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-26T22:34:49.041 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi131.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-05-26_21:24:02-fs-squid-distro-default-smithi/7727073', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-26 22:29:50.774959', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBM+HHdqoC8qRnkJTvrEdj3BbvQXlGnopC9WZsb4L3ON/a+jU2V1bp17hNvc2hmkmrRCpcdsegdRK2GiGTBht86E='} 2024-05-26T22:34:49.062 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi187.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-05-26_21:24:02-fs-squid-distro-default-smithi/7727073', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-26 22:29:50.773434', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBau4rbBJGj2tQGP8Jid05zFpRKpZrP02Gdl97eTNpSIdAVmKwCB5s42zI05KYj+KQSZgVOqEBuYykhgUBFGmvY='} 2024-05-26T22:34:49.062 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-05-26T22:34:49.068 INFO:teuthology.task.internal:roles: ubuntu@smithi131.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-05-26T22:34:49.069 INFO:teuthology.task.internal:roles: ubuntu@smithi187.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-05-26T22:34:49.069 INFO:teuthology.run_tasks:Running task console_log... 2024-05-26T22:34:49.153 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fee3a35fbe0>, signals=[15]) 2024-05-26T22:34:49.154 INFO:teuthology.run_tasks:Running task internal.connect... 2024-05-26T22:34:49.160 INFO:teuthology.task.internal:Opening connections... 2024-05-26T22:34:49.160 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi131.front.sepia.ceph.com 2024-05-26T22:34:49.161 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-26T22:34:49.234 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi187.front.sepia.ceph.com 2024-05-26T22:34:49.235 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi187.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-26T22:34:49.307 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-05-26T22:34:49.314 DEBUG:teuthology.orchestra.run.smithi131:> uname -m 2024-05-26T22:34:49.332 INFO:teuthology.orchestra.run.smithi131.stdout:x86_64 2024-05-26T22:34:49.332 DEBUG:teuthology.orchestra.run.smithi131:> cat /etc/os-release 2024-05-26T22:34:49.389 INFO:teuthology.orchestra.run.smithi131.stdout:NAME="CentOS Stream" 2024-05-26T22:34:49.390 INFO:teuthology.orchestra.run.smithi131.stdout:VERSION="9" 2024-05-26T22:34:49.390 INFO:teuthology.orchestra.run.smithi131.stdout:ID="centos" 2024-05-26T22:34:49.390 INFO:teuthology.orchestra.run.smithi131.stdout:ID_LIKE="rhel fedora" 2024-05-26T22:34:49.390 INFO:teuthology.orchestra.run.smithi131.stdout:VERSION_ID="9" 2024-05-26T22:34:49.390 INFO:teuthology.orchestra.run.smithi131.stdout:PLATFORM_ID="platform:el9" 2024-05-26T22:34:49.390 INFO:teuthology.orchestra.run.smithi131.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-26T22:34:49.390 INFO:teuthology.orchestra.run.smithi131.stdout:ANSI_COLOR="0;31" 2024-05-26T22:34:49.390 INFO:teuthology.orchestra.run.smithi131.stdout:LOGO="fedora-logo-icon" 2024-05-26T22:34:49.390 INFO:teuthology.orchestra.run.smithi131.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-26T22:34:49.390 INFO:teuthology.orchestra.run.smithi131.stdout:HOME_URL="https://centos.org/" 2024-05-26T22:34:49.390 INFO:teuthology.orchestra.run.smithi131.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-26T22:34:49.391 INFO:teuthology.orchestra.run.smithi131.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-26T22:34:49.391 INFO:teuthology.orchestra.run.smithi131.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-26T22:34:49.391 INFO:teuthology.lock.ops:Updating smithi131.front.sepia.ceph.com on lock server 2024-05-26T22:34:49.415 DEBUG:teuthology.orchestra.run.smithi187:> uname -m 2024-05-26T22:34:49.434 INFO:teuthology.orchestra.run.smithi187.stdout:x86_64 2024-05-26T22:34:49.434 DEBUG:teuthology.orchestra.run.smithi187:> cat /etc/os-release 2024-05-26T22:34:49.491 INFO:teuthology.orchestra.run.smithi187.stdout:NAME="CentOS Stream" 2024-05-26T22:34:49.492 INFO:teuthology.orchestra.run.smithi187.stdout:VERSION="9" 2024-05-26T22:34:49.492 INFO:teuthology.orchestra.run.smithi187.stdout:ID="centos" 2024-05-26T22:34:49.492 INFO:teuthology.orchestra.run.smithi187.stdout:ID_LIKE="rhel fedora" 2024-05-26T22:34:49.492 INFO:teuthology.orchestra.run.smithi187.stdout:VERSION_ID="9" 2024-05-26T22:34:49.492 INFO:teuthology.orchestra.run.smithi187.stdout:PLATFORM_ID="platform:el9" 2024-05-26T22:34:49.493 INFO:teuthology.orchestra.run.smithi187.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-26T22:34:49.493 INFO:teuthology.orchestra.run.smithi187.stdout:ANSI_COLOR="0;31" 2024-05-26T22:34:49.493 INFO:teuthology.orchestra.run.smithi187.stdout:LOGO="fedora-logo-icon" 2024-05-26T22:34:49.493 INFO:teuthology.orchestra.run.smithi187.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-26T22:34:49.493 INFO:teuthology.orchestra.run.smithi187.stdout:HOME_URL="https://centos.org/" 2024-05-26T22:34:49.493 INFO:teuthology.orchestra.run.smithi187.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-26T22:34:49.494 INFO:teuthology.orchestra.run.smithi187.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-26T22:34:49.494 INFO:teuthology.orchestra.run.smithi187.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-26T22:34:49.494 INFO:teuthology.lock.ops:Updating smithi187.front.sepia.ceph.com on lock server 2024-05-26T22:34:49.515 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-05-26T22:34:49.524 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-05-26T22:34:49.557 INFO:teuthology.task.internal:Checking for old test directory... 2024-05-26T22:34:49.557 DEBUG:teuthology.orchestra.run.smithi131:> test '!' -e /home/ubuntu/cephtest 2024-05-26T22:34:49.560 DEBUG:teuthology.orchestra.run.smithi187:> test '!' -e /home/ubuntu/cephtest 2024-05-26T22:34:49.576 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-05-26T22:34:49.592 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-05-26T22:34:49.592 DEBUG:teuthology.orchestra.run.smithi131:> test -z $(ls -A /var/lib/ceph) 2024-05-26T22:34:49.617 DEBUG:teuthology.orchestra.run.smithi187:> test -z $(ls -A /var/lib/ceph) 2024-05-26T22:34:49.728 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-05-26T22:34:49.780 INFO:teuthology.run_tasks:Running task kernel... 2024-05-26T22:34:49.794 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'} 2024-05-26T22:34:49.794 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}, 'client.2': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}, 'client.3': {'branch': 'testing'}} 2024-05-26T22:34:49.795 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}}, timeout 300 2024-05-26T22:34:49.795 DEBUG:teuthology.orchestra.run.smithi131:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-26T22:34:49.795 DEBUG:teuthology.orchestra.run.smithi187:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-26T22:34:49.811 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-26T22:34:49.812 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-05-26T22:34:49.815 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-26T22:34:49.816 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-05-26T22:34:50.023 DEBUG:teuthology.task.kernel:sha1 for client.0 is b9459a26b0888785d05c47deb35abc9d0354ab93 2024-05-26T22:34:50.023 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "b9459a26b0888785d05c47deb35abc9d0354ab93"... 2024-05-26T22:34:50.023 DEBUG:teuthology.orchestra.run.smithi131:> uname -r 2024-05-26T22:34:50.040 INFO:teuthology.orchestra.run.smithi131.stdout:5.14.0-437.el9.x86_64 2024-05-26T22:34:50.041 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs b9459a26b0888785d05c47deb35abc9d0354ab93 2024-05-26T22:34:50.041 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-05-26T22:34:50.041 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum upgrade -y linux-firmware 2024-05-26T22:34:50.045 DEBUG:teuthology.task.kernel:sha1 for client.1 is b9459a26b0888785d05c47deb35abc9d0354ab93 2024-05-26T22:34:50.045 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "b9459a26b0888785d05c47deb35abc9d0354ab93"... 2024-05-26T22:34:50.046 DEBUG:teuthology.orchestra.run.smithi187:> uname -r 2024-05-26T22:34:50.063 INFO:teuthology.orchestra.run.smithi187.stdout:5.14.0-437.el9.x86_64 2024-05-26T22:34:50.063 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs b9459a26b0888785d05c47deb35abc9d0354ab93 2024-05-26T22:34:50.063 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-05-26T22:34:50.064 DEBUG:teuthology.orchestra.run.smithi187:> sudo yum upgrade -y linux-firmware 2024-05-26T22:34:52.442 INFO:teuthology.orchestra.run.smithi187.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.1 MB 00:00 2024-05-26T22:34:52.449 INFO:teuthology.orchestra.run.smithi131.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.1 MB 00:00 2024-05-26T22:34:54.117 INFO:teuthology.orchestra.run.smithi187.stdout:CentOS Stream 9 - AppStream 32 MB/s | 19 MB 00:00 2024-05-26T22:34:58.118 INFO:teuthology.orchestra.run.smithi187.stdout:CentOS Stream 9 - CRB 15 MB/s | 6.2 MB 00:00 2024-05-26T22:34:59.910 INFO:teuthology.orchestra.run.smithi187.stdout:CentOS Stream 9 - Extras packages 36 kB/s | 17 kB 00:00 2024-05-26T22:35:05.564 INFO:teuthology.orchestra.run.smithi187.stdout:Extra Packages for Enterprise Linux 3.8 MB/s | 21 MB 00:05 2024-05-26T22:35:08.508 INFO:teuthology.orchestra.run.smithi131.stdout:CentOS Stream 9 - AppStream 1.3 MB/s | 19 MB 00:14 2024-05-26T22:35:10.744 INFO:teuthology.orchestra.run.smithi187.stdout:lab-extras 36 kB/s | 1.7 kB 00:00 2024-05-26T22:35:12.679 INFO:teuthology.orchestra.run.smithi187.stdout:Dependencies resolved. 2024-05-26T22:35:12.681 INFO:teuthology.orchestra.run.smithi187.stdout:Nothing to do. 2024-05-26T22:35:12.681 INFO:teuthology.orchestra.run.smithi187.stdout:Complete! 2024-05-26T22:35:12.793 INFO:teuthology.task.kernel:Downloading kernel b9459a26b0888785d05c47deb35abc9d0354ab93 on client.1... 2024-05-26T22:35:12.793 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b9459a26b0888785d05c47deb35abc9d0354ab93 2024-05-26T22:35:12.944 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/b9459a26b0888785d05c47deb35abc9d0354ab93/centos/9/flavors/default/x86_64/ 2024-05-26T22:35:12.945 DEBUG:teuthology.orchestra.run.smithi187:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_gb9459a26b088-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/b9459a26b0888785d05c47deb35abc9d0354ab93/centos/9/flavors/default/x86_64/ --input-file=- 2024-05-26T22:35:12.947 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi187.front.sepia.ceph.com to complete... 2024-05-26T22:35:13.757 INFO:teuthology.orchestra.run.smithi131.stdout:CentOS Stream 9 - CRB 4.0 MB/s | 6.2 MB 00:01 2024-05-26T22:35:15.074 INFO:teuthology.orchestra.run.smithi131.stdout:CentOS Stream 9 - Extras packages 113 kB/s | 17 kB 00:00 2024-05-26T22:35:23.917 INFO:teuthology.orchestra.run.smithi131.stdout:Extra Packages for Enterprise Linux 2.4 MB/s | 21 MB 00:08 2024-05-26T22:35:25.914 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26 22:35:25 URL:https://3.chacra.ceph.com/r/kernel/testing/b9459a26b0888785d05c47deb35abc9d0354ab93/centos/9/flavors/default/x86_64/kernel-6.9.0_gb9459a26b088-1.x86_64.rpm [356863406/356863406] -> "/tmp/kernel.x86_64.rpm" [1] 2024-05-26T22:35:25.914 INFO:teuthology.orchestra.run.smithi187.stderr:FINISHED --2024-05-26 22:35:25-- 2024-05-26T22:35:25.914 INFO:teuthology.orchestra.run.smithi187.stderr:Total wall clock time: 13s 2024-05-26T22:35:25.914 INFO:teuthology.orchestra.run.smithi187.stderr:Downloaded: 1 files, 340M in 13s (26.9 MB/s) 2024-05-26T22:35:25.918 INFO:teuthology.task.kernel:Installing kernel b9459a26b0888785d05c47deb35abc9d0354ab93 on client.1... 2024-05-26T22:35:25.918 DEBUG:teuthology.orchestra.run.smithi187:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-05-26T22:35:28.051 INFO:teuthology.orchestra.run.smithi187.stdout:Verifying... ######################################## 2024-05-26T22:35:28.195 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing... ######################################## 2024-05-26T22:35:28.309 INFO:teuthology.orchestra.run.smithi187.stdout:Updating / installing... 2024-05-26T22:35:29.058 INFO:teuthology.orchestra.run.smithi131.stdout:lab-extras 20 kB/s | 1.7 kB 00:00 2024-05-26T22:35:31.077 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-05-26T22:35:31.078 INFO:teuthology.orchestra.run.smithi131.stdout:Nothing to do. 2024-05-26T22:35:31.078 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-05-26T22:35:31.196 INFO:teuthology.task.kernel:Downloading kernel b9459a26b0888785d05c47deb35abc9d0354ab93 on client.0... 2024-05-26T22:35:31.196 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b9459a26b0888785d05c47deb35abc9d0354ab93 2024-05-26T22:35:31.420 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/b9459a26b0888785d05c47deb35abc9d0354ab93/centos/9/flavors/default/x86_64/ 2024-05-26T22:35:31.420 DEBUG:teuthology.orchestra.run.smithi131:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_gb9459a26b088-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/b9459a26b0888785d05c47deb35abc9d0354ab93/centos/9/flavors/default/x86_64/ --input-file=- 2024-05-26T22:35:31.422 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi131.front.sepia.ceph.com to complete... 2024-05-26T22:35:36.305 INFO:teuthology.orchestra.run.smithi187.stdout:kernel-6.9.0_gb9459a26b088-1 ######################################## 2024-05-26T22:35:43.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26 22:35:43 URL:https://3.chacra.ceph.com/r/kernel/testing/b9459a26b0888785d05c47deb35abc9d0354ab93/centos/9/flavors/default/x86_64/kernel-6.9.0_gb9459a26b088-1.x86_64.rpm [356863406/356863406] -> "/tmp/kernel.x86_64.rpm" [1] 2024-05-26T22:35:43.815 INFO:teuthology.orchestra.run.smithi131.stderr:FINISHED --2024-05-26 22:35:43-- 2024-05-26T22:35:43.815 INFO:teuthology.orchestra.run.smithi131.stderr:Total wall clock time: 12s 2024-05-26T22:35:43.816 INFO:teuthology.orchestra.run.smithi131.stderr:Downloaded: 1 files, 340M in 12s (28.3 MB/s) 2024-05-26T22:35:43.818 INFO:teuthology.task.kernel:Installing kernel b9459a26b0888785d05c47deb35abc9d0354ab93 on client.0... 2024-05-26T22:35:43.818 DEBUG:teuthology.orchestra.run.smithi131:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-05-26T22:35:45.909 INFO:teuthology.orchestra.run.smithi131.stdout:Verifying... ######################################## 2024-05-26T22:35:46.023 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing... ######################################## 2024-05-26T22:35:46.130 INFO:teuthology.orchestra.run.smithi131.stdout:Updating / installing... 2024-05-26T22:35:54.098 INFO:teuthology.orchestra.run.smithi131.stdout:kernel-6.9.0_gb9459a26b088-1 ######################################## 2024-05-26T22:36:15.670 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi187.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-05-26T22:36:15.670 DEBUG:teuthology.orchestra.run.smithi187:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-05-26T22:36:15.783 INFO:teuthology.orchestra.run.smithi187.stdout:/boot/System.map-6.9.0-gb9459a26b088 2024-05-26T22:36:15.784 INFO:teuthology.orchestra.run.smithi187.stdout:/boot/config-6.9.0-gb9459a26b088 2024-05-26T22:36:15.784 INFO:teuthology.orchestra.run.smithi187.stdout:/boot/vmlinuz-6.9.0-gb9459a26b088 2024-05-26T22:36:15.784 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/System.map 2024-05-26T22:36:15.784 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/config 2024-05-26T22:36:15.784 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel 2024-05-26T22:36:15.784 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch 2024-05-26T22:36:15.784 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86 2024-05-26T22:36:15.784 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto 2024-05-26T22:36:15.784 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-05-26T22:36:15.784 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-05-26T22:36:15.784 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-05-26T22:36:15.785 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-05-26T22:36:15.785 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-05-26T22:36:15.785 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-05-26T22:36:15.785 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-05-26T22:36:15.785 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/crc32c-intel.ko 2024-05-26T22:36:15.785 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-05-26T22:36:15.785 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-05-26T22:36:15.785 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-05-26T22:36:15.785 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-05-26T22:36:15.785 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/serpent-avx2.ko 2024-05-26T22:36:15.785 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-05-26T22:36:15.786 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-05-26T22:36:15.786 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-05-26T22:36:15.786 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-05-26T22:36:15.786 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-05-26T22:36:15.786 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-05-26T22:36:15.786 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-05-26T22:36:15.786 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel 2024-05-26T22:36:15.786 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel/cpu 2024-05-26T22:36:15.786 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel/cpu/mce 2024-05-26T22:36:15.786 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-05-26T22:36:15.786 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm 2024-05-26T22:36:15.786 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm/kvm-amd.ko 2024-05-26T22:36:15.787 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm/kvm-intel.ko 2024-05-26T22:36:15.787 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm/kvm.ko 2024-05-26T22:36:15.787 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto 2024-05-26T22:36:15.787 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/ansi_cprng.ko 2024-05-26T22:36:15.787 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/anubis.ko 2024-05-26T22:36:15.787 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/arc4.ko 2024-05-26T22:36:15.787 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx 2024-05-26T22:36:15.787 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_memcpy.ko 2024-05-26T22:36:15.787 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_pq.ko 2024-05-26T22:36:15.787 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_raid6_recov.ko 2024-05-26T22:36:15.787 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_tx.ko 2024-05-26T22:36:15.787 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_xor.ko 2024-05-26T22:36:15.787 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/raid6test.ko 2024-05-26T22:36:15.788 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/authenc.ko 2024-05-26T22:36:15.788 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/authencesn.ko 2024-05-26T22:36:15.788 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/blake2b_generic.ko 2024-05-26T22:36:15.788 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/blowfish_common.ko 2024-05-26T22:36:15.788 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/blowfish_generic.ko 2024-05-26T22:36:15.788 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/camellia_generic.ko 2024-05-26T22:36:15.788 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cast5_generic.ko 2024-05-26T22:36:15.788 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cast6_generic.ko 2024-05-26T22:36:15.788 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cast_common.ko 2024-05-26T22:36:15.788 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/ccm.ko 2024-05-26T22:36:15.788 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cmac.ko 2024-05-26T22:36:15.788 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/crc32_generic.ko 2024-05-26T22:36:15.789 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/crypto_engine.ko 2024-05-26T22:36:15.789 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/crypto_user.ko 2024-05-26T22:36:15.789 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cts.ko 2024-05-26T22:36:15.789 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/des_generic.ko 2024-05-26T22:36:15.789 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/dh_generic.ko 2024-05-26T22:36:15.789 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/echainiv.ko 2024-05-26T22:36:15.789 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/essiv.ko 2024-05-26T22:36:15.789 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/fcrypt.ko 2024-05-26T22:36:15.789 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/khazad.ko 2024-05-26T22:36:15.789 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/lz4.ko 2024-05-26T22:36:15.789 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/lz4hc.ko 2024-05-26T22:36:15.789 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/md4.ko 2024-05-26T22:36:15.789 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/michael_mic.ko 2024-05-26T22:36:15.790 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/pcbc.ko 2024-05-26T22:36:15.790 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/pcrypt.ko 2024-05-26T22:36:15.790 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/rmd160.ko 2024-05-26T22:36:15.790 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/seed.ko 2024-05-26T22:36:15.790 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/serpent_generic.ko 2024-05-26T22:36:15.790 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/tcrypt.ko 2024-05-26T22:36:15.790 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/tea.ko 2024-05-26T22:36:15.790 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/twofish_common.ko 2024-05-26T22:36:15.790 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/twofish_generic.ko 2024-05-26T22:36:15.790 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/vmac.ko 2024-05-26T22:36:15.790 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/wp512.ko 2024-05-26T22:36:15.790 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/xcbc.ko 2024-05-26T22:36:15.790 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/xor.ko 2024-05-26T22:36:15.791 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/xxhash_generic.ko 2024-05-26T22:36:15.791 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers 2024-05-26T22:36:15.791 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi 2024-05-26T22:36:15.791 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/acpi_ipmi.ko 2024-05-26T22:36:15.791 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/acpi_pad.ko 2024-05-26T22:36:15.791 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/ec_sys.ko 2024-05-26T22:36:15.791 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/platform_profile.ko 2024-05-26T22:36:15.791 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/sbs.ko 2024-05-26T22:36:15.791 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/sbshc.ko 2024-05-26T22:36:15.791 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/video.ko 2024-05-26T22:36:15.791 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata 2024-05-26T22:36:15.791 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/acard-ahci.ko 2024-05-26T22:36:15.792 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/ahci_platform.ko 2024-05-26T22:36:15.792 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/ata_generic.ko 2024-05-26T22:36:15.792 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/libahci_platform.ko 2024-05-26T22:36:15.792 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_acpi.ko 2024-05-26T22:36:15.792 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ali.ko 2024-05-26T22:36:15.792 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_amd.ko 2024-05-26T22:36:15.792 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_artop.ko 2024-05-26T22:36:15.792 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_atiixp.ko 2024-05-26T22:36:15.792 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_atp867x.ko 2024-05-26T22:36:15.792 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_cmd640.ko 2024-05-26T22:36:15.792 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_cmd64x.ko 2024-05-26T22:36:15.792 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_cypress.ko 2024-05-26T22:36:15.792 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_efar.ko 2024-05-26T22:36:15.793 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt366.ko 2024-05-26T22:36:15.793 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt37x.ko 2024-05-26T22:36:15.793 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt3x2n.ko 2024-05-26T22:36:15.793 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt3x3.ko 2024-05-26T22:36:15.793 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_it8213.ko 2024-05-26T22:36:15.793 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_it821x.ko 2024-05-26T22:36:15.793 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_jmicron.ko 2024-05-26T22:36:15.793 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_marvell.ko 2024-05-26T22:36:15.793 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_mpiix.ko 2024-05-26T22:36:15.793 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_netcell.ko 2024-05-26T22:36:15.793 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ninja32.ko 2024-05-26T22:36:15.794 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ns87410.ko 2024-05-26T22:36:15.794 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ns87415.ko 2024-05-26T22:36:15.794 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_oldpiix.ko 2024-05-26T22:36:15.794 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_opti.ko 2024-05-26T22:36:15.794 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_optidma.ko 2024-05-26T22:36:15.794 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_pcmcia.ko 2024-05-26T22:36:15.794 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_pdc2027x.ko 2024-05-26T22:36:15.794 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-05-26T22:36:15.794 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_piccolo.ko 2024-05-26T22:36:15.794 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_rdc.ko 2024-05-26T22:36:15.795 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_sch.ko 2024-05-26T22:36:15.795 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_serverworks.ko 2024-05-26T22:36:15.795 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_sil680.ko 2024-05-26T22:36:15.795 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_sis.ko 2024-05-26T22:36:15.795 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_sl82c105.ko 2024-05-26T22:36:15.795 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_triflex.ko 2024-05-26T22:36:15.795 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_via.ko 2024-05-26T22:36:15.795 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pdc_adma.ko 2024-05-26T22:36:15.795 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_inic162x.ko 2024-05-26T22:36:15.795 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_mv.ko 2024-05-26T22:36:15.795 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_nv.ko 2024-05-26T22:36:15.795 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_promise.ko 2024-05-26T22:36:15.795 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_qstor.ko 2024-05-26T22:36:15.796 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sil.ko 2024-05-26T22:36:15.796 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sil24.ko 2024-05-26T22:36:15.796 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sis.ko 2024-05-26T22:36:15.796 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_svw.ko 2024-05-26T22:36:15.796 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sx4.ko 2024-05-26T22:36:15.796 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_uli.ko 2024-05-26T22:36:15.796 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_via.ko 2024-05-26T22:36:15.796 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_vsc.ko 2024-05-26T22:36:15.796 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/auxdisplay 2024-05-26T22:36:15.796 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/auxdisplay/ks0108.ko 2024-05-26T22:36:15.796 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/base 2024-05-26T22:36:15.796 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/base/regmap 2024-05-26T22:36:15.796 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/base/regmap/regmap-i2c.ko 2024-05-26T22:36:15.797 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/base/regmap/regmap-mmio.ko 2024-05-26T22:36:15.797 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/bcma 2024-05-26T22:36:15.797 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/bcma/bcma.ko 2024-05-26T22:36:15.797 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block 2024-05-26T22:36:15.797 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/aoe 2024-05-26T22:36:15.797 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/aoe/aoe.ko 2024-05-26T22:36:15.797 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/brd.ko 2024-05-26T22:36:15.797 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/drbd 2024-05-26T22:36:15.797 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/drbd/drbd.ko 2024-05-26T22:36:15.797 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/floppy.ko 2024-05-26T22:36:15.797 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/loop.ko 2024-05-26T22:36:15.797 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/mtip32xx 2024-05-26T22:36:15.797 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-05-26T22:36:15.798 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/nbd.ko 2024-05-26T22:36:15.798 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/null_blk 2024-05-26T22:36:15.798 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/null_blk/null_blk.ko 2024-05-26T22:36:15.798 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/pktcdvd.ko 2024-05-26T22:36:15.798 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/rbd.ko 2024-05-26T22:36:15.798 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/virtio_blk.ko 2024-05-26T22:36:15.798 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/xen-blkback 2024-05-26T22:36:15.798 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-05-26T22:36:15.798 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/xen-blkfront.ko 2024-05-26T22:36:15.798 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/zram 2024-05-26T22:36:15.798 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/zram/zram.ko 2024-05-26T22:36:15.798 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char 2024-05-26T22:36:15.798 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hangcheck-timer.ko 2024-05-26T22:36:15.799 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random 2024-05-26T22:36:15.799 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/amd-rng.ko 2024-05-26T22:36:15.799 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/intel-rng.ko 2024-05-26T22:36:15.799 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-05-26T22:36:15.799 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/via-rng.ko 2024-05-26T22:36:15.799 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/virtio-rng.ko 2024-05-26T22:36:15.799 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi 2024-05-26T22:36:15.799 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-05-26T22:36:15.799 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-05-26T22:36:15.799 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-05-26T22:36:15.799 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_si.ko 2024-05-26T22:36:15.799 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-05-26T22:36:15.799 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-05-26T22:36:15.800 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/lp.ko 2024-05-26T22:36:15.800 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/mwave 2024-05-26T22:36:15.800 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/mwave/mwave.ko 2024-05-26T22:36:15.800 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ppdev.ko 2024-05-26T22:36:15.800 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tlclk.ko 2024-05-26T22:36:15.800 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm 2024-05-26T22:36:15.800 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm.ko 2024-05-26T22:36:15.800 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_atmel.ko 2024-05-26T22:36:15.800 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_infineon.ko 2024-05-26T22:36:15.800 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_nsc.ko 2024-05-26T22:36:15.800 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_tis.ko 2024-05-26T22:36:15.800 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-05-26T22:36:15.800 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/uv_mmtimer.ko 2024-05-26T22:36:15.800 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/virtio_console.ko 2024-05-26T22:36:15.801 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq 2024-05-26T22:36:15.801 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-05-26T22:36:15.801 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-05-26T22:36:15.801 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/p4-clockmod.ko 2024-05-26T22:36:15.801 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-05-26T22:36:15.801 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/powernow-k8.ko 2024-05-26T22:36:15.801 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/speedstep-lib.ko 2024-05-26T22:36:15.801 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto 2024-05-26T22:36:15.801 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/ccp 2024-05-26T22:36:15.801 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-05-26T22:36:15.801 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/ccp/ccp.ko 2024-05-26T22:36:15.801 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel 2024-05-26T22:36:15.801 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat 2024-05-26T22:36:15.801 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_common 2024-05-26T22:36:15.802 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-05-26T22:36:15.802 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-05-26T22:36:15.802 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-05-26T22:36:15.802 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/padlock-aes.ko 2024-05-26T22:36:15.802 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/padlock-sha.ko 2024-05-26T22:36:15.802 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/virtio 2024-05-26T22:36:15.802 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-05-26T22:36:15.802 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dca 2024-05-26T22:36:15.802 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dca/dca.ko 2024-05-26T22:36:15.802 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/devfreq 2024-05-26T22:36:15.802 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-05-26T22:36:15.802 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma 2024-05-26T22:36:15.803 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/dw 2024-05-26T22:36:15.803 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/dw/dw_dmac.ko 2024-05-26T22:36:15.803 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/ioat 2024-05-26T22:36:15.803 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/ioat/ioatdma.ko 2024-05-26T22:36:15.803 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac 2024-05-26T22:36:15.803 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/amd64_edac.ko 2024-05-26T22:36:15.803 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/e752x_edac.ko 2024-05-26T22:36:15.803 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/edac_mce_amd.ko 2024-05-26T22:36:15.803 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i3000_edac.ko 2024-05-26T22:36:15.803 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i3200_edac.ko 2024-05-26T22:36:15.803 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i5100_edac.ko 2024-05-26T22:36:15.803 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i5400_edac.ko 2024-05-26T22:36:15.803 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i7300_edac.ko 2024-05-26T22:36:15.803 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i7core_edac.ko 2024-05-26T22:36:15.804 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i82975x_edac.ko 2024-05-26T22:36:15.804 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/ie31200_edac.ko 2024-05-26T22:36:15.804 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/sb_edac.ko 2024-05-26T22:36:15.804 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/x38_edac.ko 2024-05-26T22:36:15.804 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/firmware 2024-05-26T22:36:15.804 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/firmware/edd.ko 2024-05-26T22:36:15.804 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/firmware/iscsi_ibft.ko 2024-05-26T22:36:15.804 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio 2024-05-26T22:36:15.804 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio/gpio-ich.ko 2024-05-26T22:36:15.804 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio/gpio-viperboard.ko 2024-05-26T22:36:15.804 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid 2024-05-26T22:36:15.804 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-appleir.ko 2024-05-26T22:36:15.804 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-aureal.ko 2024-05-26T22:36:15.805 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-axff.ko 2024-05-26T22:36:15.805 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-dr.ko 2024-05-26T22:36:15.805 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-elecom.ko 2024-05-26T22:36:15.805 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-elo.ko 2024-05-26T22:36:15.805 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-emsff.ko 2024-05-26T22:36:15.805 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-gaff.ko 2024-05-26T22:36:15.805 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-gt683r.ko 2024-05-26T22:36:15.805 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-gyration.ko 2024-05-26T22:36:15.805 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-holtek-kbd.ko 2024-05-26T22:36:15.805 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-holtek-mouse.ko 2024-05-26T22:36:15.805 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-holtekff.ko 2024-05-26T22:36:15.805 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-hyperv.ko 2024-05-26T22:36:15.805 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-icade.ko 2024-05-26T22:36:15.805 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-keytouch.ko 2024-05-26T22:36:15.806 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-kye.ko 2024-05-26T22:36:15.806 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-lcpower.ko 2024-05-26T22:36:15.806 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-led.ko 2024-05-26T22:36:15.806 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-lenovo.ko 2024-05-26T22:36:15.806 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-logitech-dj.ko 2024-05-26T22:36:15.806 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-05-26T22:36:15.806 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-multitouch.ko 2024-05-26T22:36:15.806 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-ortek.ko 2024-05-26T22:36:15.806 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-penmount.ko 2024-05-26T22:36:15.806 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-petalynx.ko 2024-05-26T22:36:15.806 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-picolcd.ko 2024-05-26T22:36:15.806 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-pl.ko 2024-05-26T22:36:15.806 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-plantronics.ko 2024-05-26T22:36:15.807 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-primax.ko 2024-05-26T22:36:15.807 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-rmi.ko 2024-05-26T22:36:15.807 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-arvo.ko 2024-05-26T22:36:15.807 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-common.ko 2024-05-26T22:36:15.807 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-isku.ko 2024-05-26T22:36:15.807 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-kone.ko 2024-05-26T22:36:15.807 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-05-26T22:36:15.807 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-konepure.ko 2024-05-26T22:36:15.807 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-05-26T22:36:15.807 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-lua.ko 2024-05-26T22:36:15.808 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-pyra.ko 2024-05-26T22:36:15.808 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-ryos.ko 2024-05-26T22:36:15.808 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-savu.ko 2024-05-26T22:36:15.808 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat.ko 2024-05-26T22:36:15.808 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-saitek.ko 2024-05-26T22:36:15.808 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-samsung.ko 2024-05-26T22:36:15.808 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sensor-hub.ko 2024-05-26T22:36:15.809 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sjoy.ko 2024-05-26T22:36:15.809 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sony.ko 2024-05-26T22:36:15.809 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-speedlink.ko 2024-05-26T22:36:15.809 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-steelseries.ko 2024-05-26T22:36:15.809 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sunplus.ko 2024-05-26T22:36:15.809 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-thrustmaster.ko 2024-05-26T22:36:15.810 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-tivo.ko 2024-05-26T22:36:15.810 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-tmff.ko 2024-05-26T22:36:15.810 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-topseed.ko 2024-05-26T22:36:15.810 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-twinhan.ko 2024-05-26T22:36:15.810 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-uclogic.ko 2024-05-26T22:36:15.810 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-waltop.ko 2024-05-26T22:36:15.811 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-wiimote.ko 2024-05-26T22:36:15.811 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-xinmo.ko 2024-05-26T22:36:15.811 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-zpff.ko 2024-05-26T22:36:15.811 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-zydacron.ko 2024-05-26T22:36:15.811 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/uhid.ko 2024-05-26T22:36:15.811 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/wacom.ko 2024-05-26T22:36:15.812 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv 2024-05-26T22:36:15.812 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv/hv_balloon.ko 2024-05-26T22:36:15.812 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv/hv_utils.ko 2024-05-26T22:36:15.812 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv/hv_vmbus.ko 2024-05-26T22:36:15.812 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon 2024-05-26T22:36:15.813 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/abituguru.ko 2024-05-26T22:36:15.813 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/abituguru3.ko 2024-05-26T22:36:15.813 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/acpi_power_meter.ko 2024-05-26T22:36:15.813 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ad7414.ko 2024-05-26T22:36:15.813 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ad7418.ko 2024-05-26T22:36:15.813 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adc128d818.ko 2024-05-26T22:36:15.814 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1025.ko 2024-05-26T22:36:15.814 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1026.ko 2024-05-26T22:36:15.814 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1029.ko 2024-05-26T22:36:15.814 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1031.ko 2024-05-26T22:36:15.814 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm9240.ko 2024-05-26T22:36:15.814 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ads7828.ko 2024-05-26T22:36:15.815 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7410.ko 2024-05-26T22:36:15.815 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7411.ko 2024-05-26T22:36:15.815 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7462.ko 2024-05-26T22:36:15.815 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7470.ko 2024-05-26T22:36:15.815 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7475.ko 2024-05-26T22:36:15.815 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7x10.ko 2024-05-26T22:36:15.816 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/amc6821.ko 2024-05-26T22:36:15.816 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/applesmc.ko 2024-05-26T22:36:15.816 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/asb100.ko 2024-05-26T22:36:15.816 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/asc7621.ko 2024-05-26T22:36:15.816 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/asus_atk0110.ko 2024-05-26T22:36:15.817 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/atxp1.ko 2024-05-26T22:36:15.817 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/coretemp.ko 2024-05-26T22:36:15.817 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-05-26T22:36:15.817 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/dme1737.ko 2024-05-26T22:36:15.817 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ds1621.ko 2024-05-26T22:36:15.817 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ds620.ko 2024-05-26T22:36:15.818 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/emc1403.ko 2024-05-26T22:36:15.818 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/emc6w201.ko 2024-05-26T22:36:15.818 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/f71805f.ko 2024-05-26T22:36:15.818 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/f71882fg.ko 2024-05-26T22:36:15.818 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/f75375s.ko 2024-05-26T22:36:15.819 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/fam15h_power.ko 2024-05-26T22:36:15.819 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/fschmd.ko 2024-05-26T22:36:15.819 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/g760a.ko 2024-05-26T22:36:15.819 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/g762.ko 2024-05-26T22:36:15.819 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/gl518sm.ko 2024-05-26T22:36:15.819 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/gl520sm.ko 2024-05-26T22:36:15.820 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/hwmon-vid.ko 2024-05-26T22:36:15.820 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/i5500_temp.ko 2024-05-26T22:36:15.820 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/i5k_amb.ko 2024-05-26T22:36:15.820 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ibmaem.ko 2024-05-26T22:36:15.820 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ibmpex.ko 2024-05-26T22:36:15.820 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ina209.ko 2024-05-26T22:36:15.821 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ina2xx.ko 2024-05-26T22:36:15.821 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/it87.ko 2024-05-26T22:36:15.821 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/k10temp.ko 2024-05-26T22:36:15.821 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/k8temp.ko 2024-05-26T22:36:15.821 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lineage-pem.ko 2024-05-26T22:36:15.821 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm63.ko 2024-05-26T22:36:15.822 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm73.ko 2024-05-26T22:36:15.822 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm75.ko 2024-05-26T22:36:15.822 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm77.ko 2024-05-26T22:36:15.822 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm78.ko 2024-05-26T22:36:15.822 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm80.ko 2024-05-26T22:36:15.822 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm83.ko 2024-05-26T22:36:15.823 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm85.ko 2024-05-26T22:36:15.823 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm87.ko 2024-05-26T22:36:15.823 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm90.ko 2024-05-26T22:36:15.823 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm92.ko 2024-05-26T22:36:15.823 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm93.ko 2024-05-26T22:36:15.824 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm95234.ko 2024-05-26T22:36:15.824 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm95241.ko 2024-05-26T22:36:15.824 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm95245.ko 2024-05-26T22:36:15.824 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc2945.ko 2024-05-26T22:36:15.824 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4151.ko 2024-05-26T22:36:15.824 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4215.ko 2024-05-26T22:36:15.825 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4222.ko 2024-05-26T22:36:15.825 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4245.ko 2024-05-26T22:36:15.825 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4260.ko 2024-05-26T22:36:15.825 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4261.ko 2024-05-26T22:36:15.825 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max16065.ko 2024-05-26T22:36:15.825 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max1619.ko 2024-05-26T22:36:15.826 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max1668.ko 2024-05-26T22:36:15.826 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max197.ko 2024-05-26T22:36:15.826 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max6639.ko 2024-05-26T22:36:15.826 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max6650.ko 2024-05-26T22:36:15.826 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max6697.ko 2024-05-26T22:36:15.826 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/mcp3021.ko 2024-05-26T22:36:15.827 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct6683.ko 2024-05-26T22:36:15.827 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct6775-core.ko 2024-05-26T22:36:15.827 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct6775.ko 2024-05-26T22:36:15.827 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct7802.ko 2024-05-26T22:36:15.827 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pc87360.ko 2024-05-26T22:36:15.827 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pc87427.ko 2024-05-26T22:36:15.828 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pcf8591.ko 2024-05-26T22:36:15.828 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus 2024-05-26T22:36:15.828 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-05-26T22:36:15.828 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-05-26T22:36:15.828 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-05-26T22:36:15.828 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/max16064.ko 2024-05-26T22:36:15.829 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/max34440.ko 2024-05-26T22:36:15.829 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/max8688.ko 2024-05-26T22:36:15.829 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-05-26T22:36:15.829 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-05-26T22:36:15.829 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-05-26T22:36:15.830 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-05-26T22:36:15.830 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-05-26T22:36:15.830 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-05-26T22:36:15.830 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/powr1220.ko 2024-05-26T22:36:15.830 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sch5627.ko 2024-05-26T22:36:15.830 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sch5636.ko 2024-05-26T22:36:15.831 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sch56xx-common.ko 2024-05-26T22:36:15.831 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sht15.ko 2024-05-26T22:36:15.831 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sht21.ko 2024-05-26T22:36:15.831 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/shtc1.ko 2024-05-26T22:36:15.831 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sis5595.ko 2024-05-26T22:36:15.831 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/smsc47b397.ko 2024-05-26T22:36:15.832 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/smsc47m1.ko 2024-05-26T22:36:15.832 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/smsc47m192.ko 2024-05-26T22:36:15.832 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/thmc50.ko 2024-05-26T22:36:15.832 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp102.ko 2024-05-26T22:36:15.832 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp103.ko 2024-05-26T22:36:15.832 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp401.ko 2024-05-26T22:36:15.833 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp421.ko 2024-05-26T22:36:15.833 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/via-cputemp.ko 2024-05-26T22:36:15.833 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/via686a.ko 2024-05-26T22:36:15.833 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/vt1211.ko 2024-05-26T22:36:15.833 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/vt8231.ko 2024-05-26T22:36:15.834 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83627ehf.ko 2024-05-26T22:36:15.834 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83627hf.ko 2024-05-26T22:36:15.834 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83781d.ko 2024-05-26T22:36:15.834 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83791d.ko 2024-05-26T22:36:15.834 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83792d.ko 2024-05-26T22:36:15.834 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83793.ko 2024-05-26T22:36:15.835 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83795.ko 2024-05-26T22:36:15.835 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83l785ts.ko 2024-05-26T22:36:15.835 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83l786ng.ko 2024-05-26T22:36:15.835 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c 2024-05-26T22:36:15.835 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/algos 2024-05-26T22:36:15.835 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-05-26T22:36:15.836 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-05-26T22:36:15.836 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses 2024-05-26T22:36:15.836 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-05-26T22:36:15.836 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-05-26T22:36:15.836 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-05-26T22:36:15.836 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-05-26T22:36:15.836 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-05-26T22:36:15.836 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-05-26T22:36:15.837 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-05-26T22:36:15.837 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-05-26T22:36:15.837 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-i801.ko 2024-05-26T22:36:15.837 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-isch.ko 2024-05-26T22:36:15.837 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-05-26T22:36:15.837 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-05-26T22:36:15.837 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-05-26T22:36:15.837 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-parport.ko 2024-05-26T22:36:15.838 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-05-26T22:36:15.838 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-05-26T22:36:15.838 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-05-26T22:36:15.838 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-05-26T22:36:15.838 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-05-26T22:36:15.838 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-05-26T22:36:15.838 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-via.ko 2024-05-26T22:36:15.839 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-05-26T22:36:15.839 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-05-26T22:36:15.839 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-dev.ko 2024-05-26T22:36:15.839 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-mux.ko 2024-05-26T22:36:15.839 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-05-26T22:36:15.839 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-smbus.ko 2024-05-26T22:36:15.839 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-stub.ko 2024-05-26T22:36:15.839 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband 2024-05-26T22:36:15.840 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core 2024-05-26T22:36:15.840 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_cm.ko 2024-05-26T22:36:15.840 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_core.ko 2024-05-26T22:36:15.840 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_umad.ko 2024-05-26T22:36:15.840 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-05-26T22:36:15.840 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/iw_cm.ko 2024-05-26T22:36:15.840 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/rdma_cm.ko 2024-05-26T22:36:15.840 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-05-26T22:36:15.840 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw 2024-05-26T22:36:15.841 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/cxgb4 2024-05-26T22:36:15.841 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-05-26T22:36:15.841 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx4 2024-05-26T22:36:15.841 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-05-26T22:36:15.841 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx5 2024-05-26T22:36:15.841 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-05-26T22:36:15.841 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mthca 2024-05-26T22:36:15.841 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-05-26T22:36:15.841 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp 2024-05-26T22:36:15.841 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/ipoib 2024-05-26T22:36:15.841 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-05-26T22:36:15.841 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/iser 2024-05-26T22:36:15.841 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-05-26T22:36:15.842 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/isert 2024-05-26T22:36:15.842 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-05-26T22:36:15.842 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srp 2024-05-26T22:36:15.842 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-05-26T22:36:15.842 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srpt 2024-05-26T22:36:15.842 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-05-26T22:36:15.842 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input 2024-05-26T22:36:15.842 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport 2024-05-26T22:36:15.842 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-05-26T22:36:15.842 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport/fm801-gp.ko 2024-05-26T22:36:15.843 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport/gameport.ko 2024-05-26T22:36:15.843 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/joydev.ko 2024-05-26T22:36:15.843 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/keyboard 2024-05-26T22:36:15.843 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/keyboard/gpio_keys.ko 2024-05-26T22:36:15.843 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse 2024-05-26T22:36:15.843 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/appletouch.ko 2024-05-26T22:36:15.843 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/bcm5974.ko 2024-05-26T22:36:15.843 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/cyapatp.ko 2024-05-26T22:36:15.843 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/elan_i2c.ko 2024-05-26T22:36:15.843 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/sermouse.ko 2024-05-26T22:36:15.843 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-05-26T22:36:15.843 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/synaptics_usb.ko 2024-05-26T22:36:15.843 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/vsxxxaa.ko 2024-05-26T22:36:15.844 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/rmi4 2024-05-26T22:36:15.844 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/rmi4/rmi_core.ko 2024-05-26T22:36:15.844 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio 2024-05-26T22:36:15.844 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/altera_ps2.ko 2024-05-26T22:36:15.844 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/arc_ps2.ko 2024-05-26T22:36:15.844 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-05-26T22:36:15.844 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/serio_raw.ko 2024-05-26T22:36:15.844 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/sparse-keymap.ko 2024-05-26T22:36:15.844 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds 2024-05-26T22:36:15.844 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-blinkm.ko 2024-05-26T22:36:15.844 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-lm3530.ko 2024-05-26T22:36:15.844 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-lp3944.ko 2024-05-26T22:36:15.844 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-ss4200.ko 2024-05-26T22:36:15.845 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger 2024-05-26T22:36:15.845 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-05-26T22:36:15.845 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-05-26T22:36:15.845 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-05-26T22:36:15.845 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-05-26T22:36:15.845 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-05-26T22:36:15.845 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-05-26T22:36:15.845 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-05-26T22:36:15.845 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-05-26T22:36:15.845 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-05-26T22:36:15.845 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md 2024-05-26T22:36:15.845 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/bcache 2024-05-26T22:36:15.845 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/bcache/bcache.ko 2024-05-26T22:36:15.846 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-bio-prison.ko 2024-05-26T22:36:15.846 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-cache-smq.ko 2024-05-26T22:36:15.846 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-cache.ko 2024-05-26T22:36:15.846 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-crypt.ko 2024-05-26T22:36:15.846 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-delay.ko 2024-05-26T22:36:15.846 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-flakey.ko 2024-05-26T22:36:15.846 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-log-userspace.ko 2024-05-26T22:36:15.846 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-multipath.ko 2024-05-26T22:36:15.846 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-queue-length.ko 2024-05-26T22:36:15.846 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-raid.ko 2024-05-26T22:36:15.846 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-round-robin.ko 2024-05-26T22:36:15.846 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-service-time.ko 2024-05-26T22:36:15.847 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-switch.ko 2024-05-26T22:36:15.847 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-thin-pool.ko 2024-05-26T22:36:15.847 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-verity.ko 2024-05-26T22:36:15.847 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/persistent-data 2024-05-26T22:36:15.847 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-05-26T22:36:15.847 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid0.ko 2024-05-26T22:36:15.847 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid1.ko 2024-05-26T22:36:15.847 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid10.ko 2024-05-26T22:36:15.847 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid456.ko 2024-05-26T22:36:15.847 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message 2024-05-26T22:36:15.847 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion 2024-05-26T22:36:15.847 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptbase.ko 2024-05-26T22:36:15.847 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptctl.ko 2024-05-26T22:36:15.847 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptfc.ko 2024-05-26T22:36:15.848 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptlan.ko 2024-05-26T22:36:15.848 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptsas.ko 2024-05-26T22:36:15.848 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptscsih.ko 2024-05-26T22:36:15.848 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptspi.ko 2024-05-26T22:36:15.848 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd 2024-05-26T22:36:15.848 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/lpc_ich.ko 2024-05-26T22:36:15.848 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/lpc_sch.ko 2024-05-26T22:36:15.848 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/mfd-core.ko 2024-05-26T22:36:15.848 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/sm501.ko 2024-05-26T22:36:15.848 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/viperboard.ko 2024-05-26T22:36:15.848 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/vx855.ko 2024-05-26T22:36:15.848 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/wl1273-core.ko 2024-05-26T22:36:15.848 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc 2024-05-26T22:36:15.849 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/altera-stapl 2024-05-26T22:36:15.849 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-05-26T22:36:15.849 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/apds9802als.ko 2024-05-26T22:36:15.849 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/apds990x.ko 2024-05-26T22:36:15.849 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/bh1770glc.ko 2024-05-26T22:36:15.849 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/cb710 2024-05-26T22:36:15.849 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/cb710/cb710.ko 2024-05-26T22:36:15.849 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom 2024-05-26T22:36:15.849 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom/at24.ko 2024-05-26T22:36:15.849 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-05-26T22:36:15.849 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom/max6875.ko 2024-05-26T22:36:15.849 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/enclosure.ko 2024-05-26T22:36:15.849 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/hpilo.ko 2024-05-26T22:36:15.850 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/isl29003.ko 2024-05-26T22:36:15.850 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/isl29020.ko 2024-05-26T22:36:15.850 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/lis3lv02d 2024-05-26T22:36:15.850 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-05-26T22:36:15.850 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-05-26T22:36:15.850 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei 2024-05-26T22:36:15.850 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei/mei-me.ko 2024-05-26T22:36:15.850 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei/mei-txe.ko 2024-05-26T22:36:15.850 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei/mei.ko 2024-05-26T22:36:15.850 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-gru 2024-05-26T22:36:15.850 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-gru/gru.ko 2024-05-26T22:36:15.851 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-xp 2024-05-26T22:36:15.851 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-xp/xp.ko 2024-05-26T22:36:15.851 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-xp/xpc.ko 2024-05-26T22:36:15.851 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-05-26T22:36:15.851 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/tifm_7xx1.ko 2024-05-26T22:36:15.851 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/tifm_core.ko 2024-05-26T22:36:15.851 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/tsl2550.ko 2024-05-26T22:36:15.851 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/vmw_balloon.ko 2024-05-26T22:36:15.851 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/vmw_vmci 2024-05-26T22:36:15.851 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-05-26T22:36:15.851 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net 2024-05-26T22:36:15.852 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/bonding 2024-05-26T22:36:15.852 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/bonding/bonding.ko 2024-05-26T22:36:15.852 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa 2024-05-26T22:36:15.852 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/b53 2024-05-26T22:36:15.852 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/b53/b53_common.ko 2024-05-26T22:36:15.852 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/bcm-sf2.ko 2024-05-26T22:36:15.852 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/mv88e6060.ko 2024-05-26T22:36:15.852 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/mv88e6xxx 2024-05-26T22:36:15.852 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-05-26T22:36:15.852 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dummy.ko 2024-05-26T22:36:15.852 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/eql.ko 2024-05-26T22:36:15.852 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet 2024-05-26T22:36:15.852 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com 2024-05-26T22:36:15.853 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-05-26T22:36:15.853 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-05-26T22:36:15.853 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-05-26T22:36:15.853 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-05-26T22:36:15.853 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390 2024-05-26T22:36:15.853 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/8390.ko 2024-05-26T22:36:15.853 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-05-26T22:36:15.853 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-05-26T22:36:15.853 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-05-26T22:36:15.853 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/adaptec 2024-05-26T22:36:15.853 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-05-26T22:36:15.853 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/agere 2024-05-26T22:36:15.853 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/agere/et131x.ko 2024-05-26T22:36:15.853 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/alteon 2024-05-26T22:36:15.854 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-05-26T22:36:15.854 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/altera 2024-05-26T22:36:15.854 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-05-26T22:36:15.854 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd 2024-05-26T22:36:15.854 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-05-26T22:36:15.854 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-05-26T22:36:15.854 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-05-26T22:36:15.854 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros 2024-05-26T22:36:15.854 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/alx 2024-05-26T22:36:15.854 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-05-26T22:36:15.854 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1c 2024-05-26T22:36:15.854 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-05-26T22:36:15.855 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1e 2024-05-26T22:36:15.855 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-05-26T22:36:15.855 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atlx 2024-05-26T22:36:15.855 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-05-26T22:36:15.855 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-05-26T22:36:15.855 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom 2024-05-26T22:36:15.855 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-05-26T22:36:15.855 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-05-26T22:36:15.855 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-05-26T22:36:15.855 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-05-26T22:36:15.855 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-05-26T22:36:15.855 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/genet 2024-05-26T22:36:15.855 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-05-26T22:36:15.856 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-05-26T22:36:15.856 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/brocade 2024-05-26T22:36:15.856 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/brocade/bna 2024-05-26T22:36:15.856 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-05-26T22:36:15.856 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio 2024-05-26T22:36:15.856 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb 2024-05-26T22:36:15.856 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-05-26T22:36:15.856 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-05-26T22:36:15.856 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-05-26T22:36:15.856 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-05-26T22:36:15.856 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-05-26T22:36:15.856 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-05-26T22:36:15.856 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-05-26T22:36:15.856 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-05-26T22:36:15.857 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-05-26T22:36:15.857 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/cisco 2024-05-26T22:36:15.857 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/cisco/enic 2024-05-26T22:36:15.857 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-05-26T22:36:15.857 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec 2024-05-26T22:36:15.857 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip 2024-05-26T22:36:15.857 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-05-26T22:36:15.857 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-05-26T22:36:15.857 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-05-26T22:36:15.857 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-05-26T22:36:15.857 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-05-26T22:36:15.858 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-05-26T22:36:15.858 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dlink 2024-05-26T22:36:15.858 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-05-26T22:36:15.858 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-05-26T22:36:15.858 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dnet.ko 2024-05-26T22:36:15.858 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/emulex 2024-05-26T22:36:15.858 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/emulex/benet 2024-05-26T22:36:15.858 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-05-26T22:36:15.858 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/ethoc.ko 2024-05-26T22:36:15.858 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/fealnx.ko 2024-05-26T22:36:15.858 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel 2024-05-26T22:36:15.858 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e100.ko 2024-05-26T22:36:15.859 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000 2024-05-26T22:36:15.859 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-05-26T22:36:15.859 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000e 2024-05-26T22:36:15.859 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-05-26T22:36:15.859 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/fm10k 2024-05-26T22:36:15.859 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-05-26T22:36:15.859 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/i40e 2024-05-26T22:36:15.859 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-05-26T22:36:15.859 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/iavf 2024-05-26T22:36:15.859 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-05-26T22:36:15.859 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igb 2024-05-26T22:36:15.859 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-05-26T22:36:15.859 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igbvf 2024-05-26T22:36:15.861 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-05-26T22:36:15.861 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbe 2024-05-26T22:36:15.861 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-05-26T22:36:15.861 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbevf 2024-05-26T22:36:15.861 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-05-26T22:36:15.861 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/jme.ko 2024-05-26T22:36:15.861 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell 2024-05-26T22:36:15.862 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-05-26T22:36:15.862 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell/skge.ko 2024-05-26T22:36:15.862 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-05-26T22:36:15.862 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox 2024-05-26T22:36:15.862 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx4 2024-05-26T22:36:15.862 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-05-26T22:36:15.862 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-05-26T22:36:15.862 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx5 2024-05-26T22:36:15.862 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-05-26T22:36:15.862 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-05-26T22:36:15.862 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/micrel 2024-05-26T22:36:15.862 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-05-26T22:36:15.862 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/myricom 2024-05-26T22:36:15.863 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/myricom/myri10ge 2024-05-26T22:36:15.863 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-05-26T22:36:15.863 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/natsemi 2024-05-26T22:36:15.863 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-05-26T22:36:15.863 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-05-26T22:36:15.863 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/neterion 2024-05-26T22:36:15.863 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-05-26T22:36:15.863 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/nvidia 2024-05-26T22:36:15.863 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-05-26T22:36:15.863 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/packetengines 2024-05-26T22:36:15.863 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-05-26T22:36:15.863 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-05-26T22:36:15.864 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic 2024-05-26T22:36:15.864 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/netxen 2024-05-26T22:36:15.864 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-05-26T22:36:15.864 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-05-26T22:36:15.864 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-05-26T22:36:15.864 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-05-26T22:36:15.864 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/rdc 2024-05-26T22:36:15.865 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-05-26T22:36:15.865 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek 2024-05-26T22:36:15.865 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-05-26T22:36:15.865 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-05-26T22:36:15.865 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/atp.ko 2024-05-26T22:36:15.865 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-05-26T22:36:15.866 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/rocker 2024-05-26T22:36:15.866 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-05-26T22:36:15.866 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sfc 2024-05-26T22:36:15.866 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-05-26T22:36:15.866 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/silan 2024-05-26T22:36:15.866 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-05-26T22:36:15.867 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sis 2024-05-26T22:36:15.867 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sis/sis190.ko 2024-05-26T22:36:15.867 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sis/sis900.ko 2024-05-26T22:36:15.867 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc 2024-05-26T22:36:15.867 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-05-26T22:36:15.867 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-05-26T22:36:15.868 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-05-26T22:36:15.868 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-05-26T22:36:15.868 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro 2024-05-26T22:36:15.868 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro/stmmac 2024-05-26T22:36:15.868 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-05-26T22:36:15.868 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-05-26T22:36:15.868 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun 2024-05-26T22:36:15.869 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/cassini.ko 2024-05-26T22:36:15.869 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/niu.ko 2024-05-26T22:36:15.869 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/sungem.ko 2024-05-26T22:36:15.869 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-05-26T22:36:15.869 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/tehuti 2024-05-26T22:36:15.869 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-05-26T22:36:15.870 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/ti 2024-05-26T22:36:15.870 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/ti/tlan.ko 2024-05-26T22:36:15.870 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/via 2024-05-26T22:36:15.870 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-05-26T22:36:15.870 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-05-26T22:36:15.870 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/wiznet 2024-05-26T22:36:15.871 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-05-26T22:36:15.871 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-05-26T22:36:15.871 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/xircom 2024-05-26T22:36:15.871 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-05-26T22:36:15.871 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/geneve.ko 2024-05-26T22:36:15.871 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/hyperv 2024-05-26T22:36:15.872 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-05-26T22:36:15.872 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ieee802154 2024-05-26T22:36:15.872 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ieee802154/fakelb.ko 2024-05-26T22:36:15.872 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ifb.ko 2024-05-26T22:36:15.872 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ipvlan 2024-05-26T22:36:15.873 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ipvlan/ipvlan.ko 2024-05-26T22:36:15.873 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/macvlan.ko 2024-05-26T22:36:15.873 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/macvtap.ko 2024-05-26T22:36:15.873 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio 2024-05-26T22:36:15.873 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio.ko 2024-05-26T22:36:15.873 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-05-26T22:36:15.874 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-05-26T22:36:15.874 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio/mdio-regmap.ko 2024-05-26T22:36:15.874 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mii.ko 2024-05-26T22:36:15.874 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/net_failover.ko 2024-05-26T22:36:15.874 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/netconsole.ko 2024-05-26T22:36:15.874 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/nlmon.ko 2024-05-26T22:36:15.875 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/pcs 2024-05-26T22:36:15.875 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/pcs/pcs-lynx.ko 2024-05-26T22:36:15.875 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-05-26T22:36:15.875 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy 2024-05-26T22:36:15.875 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/amd.ko 2024-05-26T22:36:15.875 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-05-26T22:36:15.876 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-05-26T22:36:15.876 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/bcm7xxx.ko 2024-05-26T22:36:15.876 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/bcm87xx.ko 2024-05-26T22:36:15.876 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/broadcom.ko 2024-05-26T22:36:15.876 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/cicada.ko 2024-05-26T22:36:15.876 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/davicom.ko 2024-05-26T22:36:15.877 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/dp83640.ko 2024-05-26T22:36:15.877 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/et1011c.ko 2024-05-26T22:36:15.877 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/icplus.ko 2024-05-26T22:36:15.877 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/lxt.ko 2024-05-26T22:36:15.877 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/marvell.ko 2024-05-26T22:36:15.877 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/micrel.ko 2024-05-26T22:36:15.878 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/national.ko 2024-05-26T22:36:15.878 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/phylink.ko 2024-05-26T22:36:15.878 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/qsemi.ko 2024-05-26T22:36:15.878 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/realtek.ko 2024-05-26T22:36:15.878 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/smsc.ko 2024-05-26T22:36:15.878 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/ste10Xp.ko 2024-05-26T22:36:15.879 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/vitesse.ko 2024-05-26T22:36:15.879 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp 2024-05-26T22:36:15.879 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/bsd_comp.ko 2024-05-26T22:36:15.879 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_async.ko 2024-05-26T22:36:15.879 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_deflate.ko 2024-05-26T22:36:15.879 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_generic.ko 2024-05-26T22:36:15.880 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_mppe.ko 2024-05-26T22:36:15.880 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_synctty.ko 2024-05-26T22:36:15.880 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/pppoe.ko 2024-05-26T22:36:15.880 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/pppox.ko 2024-05-26T22:36:15.880 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/pptp.ko 2024-05-26T22:36:15.880 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/slip 2024-05-26T22:36:15.881 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/slip/slhc.ko 2024-05-26T22:36:15.881 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/slip/slip.ko 2024-05-26T22:36:15.881 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/sungem_phy.ko 2024-05-26T22:36:15.881 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/tap.ko 2024-05-26T22:36:15.881 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team 2024-05-26T22:36:15.881 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team.ko 2024-05-26T22:36:15.882 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_activebackup.ko 2024-05-26T22:36:15.882 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_broadcast.ko 2024-05-26T22:36:15.882 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-05-26T22:36:15.882 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_random.ko 2024-05-26T22:36:15.882 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-05-26T22:36:15.883 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/tun.ko 2024-05-26T22:36:15.883 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/veth.ko 2024-05-26T22:36:15.883 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/virtio_net.ko 2024-05-26T22:36:15.883 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vmxnet3 2024-05-26T22:36:15.883 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-05-26T22:36:15.883 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vxlan 2024-05-26T22:36:15.884 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vxlan/vxlan.ko 2024-05-26T22:36:15.884 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/xen-netback 2024-05-26T22:36:15.884 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/xen-netback/xen-netback.ko 2024-05-26T22:36:15.884 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/xen-netfront.ko 2024-05-26T22:36:15.884 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ntb 2024-05-26T22:36:15.884 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ntb/ntb.ko 2024-05-26T22:36:15.885 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme 2024-05-26T22:36:15.885 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme/host 2024-05-26T22:36:15.885 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme/host/nvme-core.ko 2024-05-26T22:36:15.885 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme/host/nvme.ko 2024-05-26T22:36:15.885 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport 2024-05-26T22:36:15.885 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport.ko 2024-05-26T22:36:15.886 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport_cs.ko 2024-05-26T22:36:15.886 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport_pc.ko 2024-05-26T22:36:15.886 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport_serial.ko 2024-05-26T22:36:15.886 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci 2024-05-26T22:36:15.886 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/hotplug 2024-05-26T22:36:15.887 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-05-26T22:36:15.887 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/pcie 2024-05-26T22:36:15.887 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/pcie/aer_inject.ko 2024-05-26T22:36:15.887 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/xen-pcifront.ko 2024-05-26T22:36:15.887 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia 2024-05-26T22:36:15.887 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia/i82092.ko 2024-05-26T22:36:15.888 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia/pd6729.ko 2024-05-26T22:36:15.888 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia/yenta_socket.ko 2024-05-26T22:36:15.888 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pinctrl 2024-05-26T22:36:15.888 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pinctrl/intel 2024-05-26T22:36:15.888 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-05-26T22:36:15.888 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform 2024-05-26T22:36:15.889 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86 2024-05-26T22:36:15.889 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/acer-wmi.ko 2024-05-26T22:36:15.889 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/acerhdf.ko 2024-05-26T22:36:15.889 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/apple-gmux.ko 2024-05-26T22:36:15.889 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/asus-laptop.ko 2024-05-26T22:36:15.889 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-05-26T22:36:15.890 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/asus-wmi.ko 2024-05-26T22:36:15.890 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/classmate-laptop.ko 2024-05-26T22:36:15.890 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-05-26T22:36:15.890 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-05-26T22:36:15.890 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-05-26T22:36:15.891 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-05-26T22:36:15.891 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/hdaps.ko 2024-05-26T22:36:15.891 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/intel 2024-05-26T22:36:15.891 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-05-26T22:36:15.891 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/intel_ips.ko 2024-05-26T22:36:15.891 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/msi-wmi.ko 2024-05-26T22:36:15.891 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/mxm-wmi.ko 2024-05-26T22:36:15.891 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-05-26T22:36:15.891 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/samsung-laptop.ko 2024-05-26T22:36:15.892 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/samsung-q10.ko 2024-05-26T22:36:15.892 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/topstar-laptop.ko 2024-05-26T22:36:15.892 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-05-26T22:36:15.892 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/toshiba_haps.ko 2024-05-26T22:36:15.892 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/wmi-bmof.ko 2024-05-26T22:36:15.892 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/wmi.ko 2024-05-26T22:36:15.892 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/powercap 2024-05-26T22:36:15.892 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/powercap/intel_rapl_common.ko 2024-05-26T22:36:15.893 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/powercap/intel_rapl_msr.ko 2024-05-26T22:36:15.893 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps 2024-05-26T22:36:15.893 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients 2024-05-26T22:36:15.893 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients/pps-gpio.ko 2024-05-26T22:36:15.893 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients/pps-ldisc.ko 2024-05-26T22:36:15.893 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients/pps_parport.ko 2024-05-26T22:36:15.893 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/pps_core.ko 2024-05-26T22:36:15.893 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ptp 2024-05-26T22:36:15.894 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ptp/ptp.ko 2024-05-26T22:36:15.894 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ptp/ptp_kvm.ko 2024-05-26T22:36:15.894 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras 2024-05-26T22:36:15.894 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd 2024-05-26T22:36:15.894 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd/atl 2024-05-26T22:36:15.894 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-05-26T22:36:15.894 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd/fmpm.ko 2024-05-26T22:36:15.894 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc 2024-05-26T22:36:15.895 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-bq32k.ko 2024-05-26T22:36:15.895 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1286.ko 2024-05-26T22:36:15.895 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1307.ko 2024-05-26T22:36:15.895 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1374.ko 2024-05-26T22:36:15.895 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1511.ko 2024-05-26T22:36:15.895 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1553.ko 2024-05-26T22:36:15.895 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1672.ko 2024-05-26T22:36:15.895 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1742.ko 2024-05-26T22:36:15.895 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds2404.ko 2024-05-26T22:36:15.895 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds3232.ko 2024-05-26T22:36:15.896 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-em3027.ko 2024-05-26T22:36:15.896 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-fm3130.ko 2024-05-26T22:36:15.896 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-isl12022.ko 2024-05-26T22:36:15.896 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-isl1208.ko 2024-05-26T22:36:15.896 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-m41t80.ko 2024-05-26T22:36:15.896 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-m48t35.ko 2024-05-26T22:36:15.896 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-m48t59.ko 2024-05-26T22:36:15.896 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-max6900.ko 2024-05-26T22:36:15.896 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-msm6242.ko 2024-05-26T22:36:15.896 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf2127.ko 2024-05-26T22:36:15.896 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf85063.ko 2024-05-26T22:36:15.896 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf8523.ko 2024-05-26T22:36:15.896 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf8563.ko 2024-05-26T22:36:15.897 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf8583.ko 2024-05-26T22:36:15.897 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rp5c01.ko 2024-05-26T22:36:15.897 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rs5c372.ko 2024-05-26T22:36:15.897 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-05-26T22:36:15.897 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rx8025.ko 2024-05-26T22:36:15.897 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rx8581.ko 2024-05-26T22:36:15.897 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-05-26T22:36:15.897 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-x1205.ko 2024-05-26T22:36:15.897 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi 2024-05-26T22:36:15.897 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/3w-9xxx.ko 2024-05-26T22:36:15.897 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/3w-sas.ko 2024-05-26T22:36:15.897 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/3w-xxxx.ko 2024-05-26T22:36:15.897 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/BusLogic.ko 2024-05-26T22:36:15.898 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/a100u2w.ko 2024-05-26T22:36:15.898 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aacraid 2024-05-26T22:36:15.898 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aacraid/aacraid.ko 2024-05-26T22:36:15.898 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/advansys.ko 2024-05-26T22:36:15.898 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aic7xxx 2024-05-26T22:36:15.898 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-05-26T22:36:15.898 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-05-26T22:36:15.898 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/am53c974.ko 2024-05-26T22:36:15.898 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/arcmsr 2024-05-26T22:36:15.898 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-05-26T22:36:15.898 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/atp870u.ko 2024-05-26T22:36:15.898 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/be2iscsi 2024-05-26T22:36:15.898 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-05-26T22:36:15.899 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bfa 2024-05-26T22:36:15.899 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bfa/bfa.ko 2024-05-26T22:36:15.899 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2fc 2024-05-26T22:36:15.899 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-05-26T22:36:15.899 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2i 2024-05-26T22:36:15.899 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-05-26T22:36:15.899 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ch.ko 2024-05-26T22:36:15.899 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/csiostor 2024-05-26T22:36:15.899 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/csiostor/csiostor.ko 2024-05-26T22:36:15.899 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi 2024-05-26T22:36:15.899 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb3i 2024-05-26T22:36:15.899 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-05-26T22:36:15.899 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb4i 2024-05-26T22:36:15.899 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-05-26T22:36:15.900 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-05-26T22:36:15.900 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/dc395x.ko 2024-05-26T22:36:15.900 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler 2024-05-26T22:36:15.900 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-05-26T22:36:15.900 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-05-26T22:36:15.900 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-05-26T22:36:15.900 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-05-26T22:36:15.900 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/dmx3191d.ko 2024-05-26T22:36:15.900 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/esas2r 2024-05-26T22:36:15.900 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/esas2r/esas2r.ko 2024-05-26T22:36:15.900 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/esp_scsi.ko 2024-05-26T22:36:15.900 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fcoe 2024-05-26T22:36:15.900 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fcoe/fcoe.ko 2024-05-26T22:36:15.901 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-05-26T22:36:15.901 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fnic 2024-05-26T22:36:15.901 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fnic/fnic.ko 2024-05-26T22:36:15.901 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/hpsa.ko 2024-05-26T22:36:15.901 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/hptiop.ko 2024-05-26T22:36:15.901 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/hv_storvsc.ko 2024-05-26T22:36:15.901 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/initio.ko 2024-05-26T22:36:15.901 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ipr.ko 2024-05-26T22:36:15.901 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ips.ko 2024-05-26T22:36:15.901 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/isci 2024-05-26T22:36:15.901 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/isci/isci.ko 2024-05-26T22:36:15.901 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-05-26T22:36:15.901 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/iscsi_tcp.ko 2024-05-26T22:36:15.902 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libfc 2024-05-26T22:36:15.902 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libfc/libfc.ko 2024-05-26T22:36:15.902 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libiscsi.ko 2024-05-26T22:36:15.902 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libiscsi_tcp.ko 2024-05-26T22:36:15.902 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libsas 2024-05-26T22:36:15.902 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libsas/libsas.ko 2024-05-26T22:36:15.902 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/lpfc 2024-05-26T22:36:15.902 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/lpfc/lpfc.ko 2024-05-26T22:36:15.902 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid 2024-05-26T22:36:15.902 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid.ko 2024-05-26T22:36:15.902 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-05-26T22:36:15.902 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-05-26T22:36:15.902 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-05-26T22:36:15.903 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mpt3sas 2024-05-26T22:36:15.903 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-05-26T22:36:15.903 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mvsas 2024-05-26T22:36:15.903 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mvsas/mvsas.ko 2024-05-26T22:36:15.903 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mvumi.ko 2024-05-26T22:36:15.903 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/pm8001 2024-05-26T22:36:15.903 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-05-26T22:36:15.903 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/pmcraid.ko 2024-05-26T22:36:15.903 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla1280.ko 2024-05-26T22:36:15.903 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla2xxx 2024-05-26T22:36:15.903 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-05-26T22:36:15.903 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-05-26T22:36:15.903 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla4xxx 2024-05-26T22:36:15.903 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-05-26T22:36:15.904 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/raid_class.ko 2024-05-26T22:36:15.904 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_debug.ko 2024-05-26T22:36:15.904 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_fc.ko 2024-05-26T22:36:15.904 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-05-26T22:36:15.904 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_sas.ko 2024-05-26T22:36:15.904 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_spi.ko 2024-05-26T22:36:15.904 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_srp.ko 2024-05-26T22:36:15.904 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ses.ko 2024-05-26T22:36:15.904 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/st.ko 2024-05-26T22:36:15.904 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/stex.ko 2024-05-26T22:36:15.904 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/sym53c8xx_2 2024-05-26T22:36:15.904 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-05-26T22:36:15.904 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/virtio_scsi.ko 2024-05-26T22:36:15.905 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/vmw_pvscsi.ko 2024-05-26T22:36:15.905 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/wd719x.ko 2024-05-26T22:36:15.905 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ssb 2024-05-26T22:36:15.905 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ssb/ssb.ko 2024-05-26T22:36:15.905 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target 2024-05-26T22:36:15.905 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/iscsi 2024-05-26T22:36:15.905 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-05-26T22:36:15.905 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/loopback 2024-05-26T22:36:15.905 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/loopback/tcm_loop.ko 2024-05-26T22:36:15.905 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_file.ko 2024-05-26T22:36:15.905 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_iblock.ko 2024-05-26T22:36:15.905 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_mod.ko 2024-05-26T22:36:15.905 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_pscsi.ko 2024-05-26T22:36:15.906 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_user.ko 2024-05-26T22:36:15.906 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/tcm_fc 2024-05-26T22:36:15.906 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-05-26T22:36:15.906 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal 2024-05-26T22:36:15.906 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel 2024-05-26T22:36:15.906 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal 2024-05-26T22:36:15.906 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-05-26T22:36:15.906 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-05-26T22:36:15.906 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-05-26T22:36:15.906 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-05-26T22:36:15.906 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-05-26T22:36:15.906 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-05-26T22:36:15.907 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-05-26T22:36:15.907 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-05-26T22:36:15.907 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-05-26T22:36:15.907 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-05-26T22:36:15.907 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-05-26T22:36:15.907 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-05-26T22:36:15.907 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-05-26T22:36:15.907 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-05-26T22:36:15.907 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-05-26T22:36:15.907 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-05-26T22:36:15.907 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-05-26T22:36:15.907 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-05-26T22:36:15.907 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty 2024-05-26T22:36:15.907 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/ipwireless 2024-05-26T22:36:15.908 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-05-26T22:36:15.908 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/n_gsm.ko 2024-05-26T22:36:15.908 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/n_hdlc.ko 2024-05-26T22:36:15.908 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/nozomi.ko 2024-05-26T22:36:15.908 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial 2024-05-26T22:36:15.908 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/8250 2024-05-26T22:36:15.908 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-05-26T22:36:15.908 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/arc_uart.ko 2024-05-26T22:36:15.908 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/jsm 2024-05-26T22:36:15.908 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/jsm/jsm.ko 2024-05-26T22:36:15.908 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs 2024-05-26T22:36:15.908 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/core 2024-05-26T22:36:15.908 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/core/ufshcd-core.ko 2024-05-26T22:36:15.908 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/host 2024-05-26T22:36:15.909 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-05-26T22:36:15.909 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio 2024-05-26T22:36:15.909 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio.ko 2024-05-26T22:36:15.909 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_aec.ko 2024-05-26T22:36:15.909 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_cif.ko 2024-05-26T22:36:15.909 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_pci_generic.ko 2024-05-26T22:36:15.909 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_sercos3.ko 2024-05-26T22:36:15.909 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb 2024-05-26T22:36:15.909 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class 2024-05-26T22:36:15.909 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/cdc-acm.ko 2024-05-26T22:36:15.909 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/cdc-wdm.ko 2024-05-26T22:36:15.909 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/usblp.ko 2024-05-26T22:36:15.909 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/usbtmc.ko 2024-05-26T22:36:15.909 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/host 2024-05-26T22:36:15.910 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/host/sl811-hcd.ko 2024-05-26T22:36:15.910 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/image 2024-05-26T22:36:15.910 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/image/mdc800.ko 2024-05-26T22:36:15.910 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/image/microtek.ko 2024-05-26T22:36:15.910 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc 2024-05-26T22:36:15.910 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/adutux.ko 2024-05-26T22:36:15.910 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/appledisplay.ko 2024-05-26T22:36:15.910 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/emi26.ko 2024-05-26T22:36:15.910 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/emi62.ko 2024-05-26T22:36:15.910 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/ezusb.ko 2024-05-26T22:36:15.910 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/idmouse.ko 2024-05-26T22:36:15.910 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/iowarrior.ko 2024-05-26T22:36:15.910 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/isight_firmware.ko 2024-05-26T22:36:15.911 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/ldusb.ko 2024-05-26T22:36:15.911 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/legousbtower.ko 2024-05-26T22:36:15.911 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/sisusbvga 2024-05-26T22:36:15.911 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-05-26T22:36:15.911 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/trancevibrator.ko 2024-05-26T22:36:15.911 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/usb3503.ko 2024-05-26T22:36:15.911 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/usblcd.ko 2024-05-26T22:36:15.911 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/usbsevseg.ko 2024-05-26T22:36:15.911 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/uss720.ko 2024-05-26T22:36:15.911 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/yurex.ko 2024-05-26T22:36:15.911 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/phy 2024-05-26T22:36:15.911 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/phy/phy-generic.ko 2024-05-26T22:36:15.911 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial 2024-05-26T22:36:15.911 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/aircable.ko 2024-05-26T22:36:15.912 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ark3116.ko 2024-05-26T22:36:15.912 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/belkin_sa.ko 2024-05-26T22:36:15.912 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ch341.ko 2024-05-26T22:36:15.912 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/cp210x.ko 2024-05-26T22:36:15.912 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/cyberjack.ko 2024-05-26T22:36:15.912 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/cypress_m8.ko 2024-05-26T22:36:15.912 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/digi_acceleport.ko 2024-05-26T22:36:15.912 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/empeg.ko 2024-05-26T22:36:15.912 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ftdi_sio.ko 2024-05-26T22:36:15.912 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/garmin_gps.ko 2024-05-26T22:36:15.912 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/io_edgeport.ko 2024-05-26T22:36:15.912 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/io_ti.ko 2024-05-26T22:36:15.912 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ipaq.ko 2024-05-26T22:36:15.912 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ipw.ko 2024-05-26T22:36:15.913 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ir-usb.ko 2024-05-26T22:36:15.913 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-05-26T22:36:15.913 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/keyspan.ko 2024-05-26T22:36:15.913 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/keyspan_pda.ko 2024-05-26T22:36:15.913 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/kl5kusb105.ko 2024-05-26T22:36:15.913 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/kobil_sct.ko 2024-05-26T22:36:15.913 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/mct_u232.ko 2024-05-26T22:36:15.913 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/mos7720.ko 2024-05-26T22:36:15.913 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/mos7840.ko 2024-05-26T22:36:15.913 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/navman.ko 2024-05-26T22:36:15.913 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/omninet.ko 2024-05-26T22:36:15.913 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/opticon.ko 2024-05-26T22:36:15.913 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/option.ko 2024-05-26T22:36:15.913 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/oti6858.ko 2024-05-26T22:36:15.914 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/pl2303.ko 2024-05-26T22:36:15.914 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/qcaux.ko 2024-05-26T22:36:15.914 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/qcserial.ko 2024-05-26T22:36:15.914 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/quatech2.ko 2024-05-26T22:36:15.914 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/safe_serial.ko 2024-05-26T22:36:15.914 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/sierra.ko 2024-05-26T22:36:15.914 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/spcp8x5.ko 2024-05-26T22:36:15.914 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ssu100.ko 2024-05-26T22:36:15.914 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/symbolserial.ko 2024-05-26T22:36:15.914 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-05-26T22:36:15.914 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-05-26T22:36:15.914 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/usb_debug.ko 2024-05-26T22:36:15.914 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/usb_wwan.ko 2024-05-26T22:36:15.915 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/visor.ko 2024-05-26T22:36:15.915 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/whiteheat.ko 2024-05-26T22:36:15.915 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/xsens_mt.ko 2024-05-26T22:36:15.915 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage 2024-05-26T22:36:15.915 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/uas.ko 2024-05-26T22:36:15.915 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-alauda.ko 2024-05-26T22:36:15.915 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-cypress.ko 2024-05-26T22:36:15.915 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-datafab.ko 2024-05-26T22:36:15.915 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-05-26T22:36:15.915 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-freecom.ko 2024-05-26T22:36:15.915 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-isd200.ko 2024-05-26T22:36:15.915 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-05-26T22:36:15.915 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-karma.ko 2024-05-26T22:36:15.916 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-onetouch.ko 2024-05-26T22:36:15.916 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-realtek.ko 2024-05-26T22:36:15.916 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-sddr09.ko 2024-05-26T22:36:15.916 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-sddr55.ko 2024-05-26T22:36:15.916 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-usbat.ko 2024-05-26T22:36:15.916 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/usb-storage.ko 2024-05-26T22:36:15.916 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip 2024-05-26T22:36:15.916 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip/usbip-core.ko 2024-05-26T22:36:15.916 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip/usbip-host.ko 2024-05-26T22:36:15.916 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-05-26T22:36:15.916 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio 2024-05-26T22:36:15.916 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/pci 2024-05-26T22:36:15.916 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-05-26T22:36:15.916 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/pci/vfio-pci.ko 2024-05-26T22:36:15.917 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/vfio.ko 2024-05-26T22:36:15.917 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-05-26T22:36:15.917 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost 2024-05-26T22:36:15.917 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost.ko 2024-05-26T22:36:15.917 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost_iotlb.ko 2024-05-26T22:36:15.917 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost_net.ko 2024-05-26T22:36:15.917 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost_scsi.ko 2024-05-26T22:36:15.917 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video 2024-05-26T22:36:15.917 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight 2024-05-26T22:36:15.917 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/apple_bl.ko 2024-05-26T22:36:15.917 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/lcd.ko 2024-05-26T22:36:15.918 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/platform_lcd.ko 2024-05-26T22:36:15.918 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio 2024-05-26T22:36:15.918 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio.ko 2024-05-26T22:36:15.918 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_balloon.ko 2024-05-26T22:36:15.918 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_mmio.ko 2024-05-26T22:36:15.918 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_pci.ko 2024-05-26T22:36:15.918 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-05-26T22:36:15.918 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-05-26T22:36:15.918 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_ring.ko 2024-05-26T22:36:15.918 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1 2024-05-26T22:36:15.919 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/masters 2024-05-26T22:36:15.919 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/masters/ds2482.ko 2024-05-26T22:36:15.919 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/masters/ds2490.ko 2024-05-26T22:36:15.919 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves 2024-05-26T22:36:15.919 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-05-26T22:36:15.919 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-05-26T22:36:15.919 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-05-26T22:36:15.919 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-05-26T22:36:15.919 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-05-26T22:36:15.919 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-05-26T22:36:15.919 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-05-26T22:36:15.919 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-05-26T22:36:15.920 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-05-26T22:36:15.920 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_smem.ko 2024-05-26T22:36:15.920 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_therm.ko 2024-05-26T22:36:15.920 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/wire.ko 2024-05-26T22:36:15.920 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog 2024-05-26T22:36:15.920 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/alim1535_wdt.ko 2024-05-26T22:36:15.920 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/alim7101_wdt.ko 2024-05-26T22:36:15.920 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/f71808e_wdt.ko 2024-05-26T22:36:15.920 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/hpwdt.ko 2024-05-26T22:36:15.920 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/i6300esb.ko 2024-05-26T22:36:15.920 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-05-26T22:36:15.920 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/iTCO_wdt.ko 2024-05-26T22:36:15.920 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/ib700wdt.ko 2024-05-26T22:36:15.921 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/ibmasr.ko 2024-05-26T22:36:15.921 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-05-26T22:36:15.921 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/it8712f_wdt.ko 2024-05-26T22:36:15.921 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/it87_wdt.ko 2024-05-26T22:36:15.921 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/machzwd.ko 2024-05-26T22:36:15.921 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/nv_tco.ko 2024-05-26T22:36:15.921 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/pcwd_pci.ko 2024-05-26T22:36:15.921 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/pcwd_usb.ko 2024-05-26T22:36:15.921 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-05-26T22:36:15.921 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/sch311x_wdt.ko 2024-05-26T22:36:15.921 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/softdog.ko 2024-05-26T22:36:15.921 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/sp5100_tco.ko 2024-05-26T22:36:15.921 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/via_wdt.ko 2024-05-26T22:36:15.921 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-05-26T22:36:15.922 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/w83877f_wdt.ko 2024-05-26T22:36:15.922 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/w83977f_wdt.ko 2024-05-26T22:36:15.922 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/wdt_pci.ko 2024-05-26T22:36:15.922 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/xen_wdt.ko 2024-05-26T22:36:15.922 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen 2024-05-26T22:36:15.922 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-acpi-processor.ko 2024-05-26T22:36:15.922 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-evtchn.ko 2024-05-26T22:36:15.922 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-gntalloc.ko 2024-05-26T22:36:15.922 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-gntdev.ko 2024-05-26T22:36:15.922 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-pciback 2024-05-26T22:36:15.922 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-05-26T22:36:15.922 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-privcmd.ko 2024-05-26T22:36:15.923 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xenfs 2024-05-26T22:36:15.923 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xenfs/xenfs.ko 2024-05-26T22:36:15.923 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs 2024-05-26T22:36:15.923 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/9p 2024-05-26T22:36:15.923 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/9p/9p.ko 2024-05-26T22:36:15.923 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/affs 2024-05-26T22:36:15.923 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/affs/affs.ko 2024-05-26T22:36:15.923 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/befs 2024-05-26T22:36:15.923 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/befs/befs.ko 2024-05-26T22:36:15.923 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/binfmt_misc.ko 2024-05-26T22:36:15.923 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/btrfs 2024-05-26T22:36:15.923 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/btrfs/btrfs.ko 2024-05-26T22:36:15.923 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/cachefiles 2024-05-26T22:36:15.923 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/cachefiles/cachefiles.ko 2024-05-26T22:36:15.924 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ceph 2024-05-26T22:36:15.924 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ceph/ceph.ko 2024-05-26T22:36:15.924 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/coda 2024-05-26T22:36:15.924 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/coda/coda.ko 2024-05-26T22:36:15.924 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/cramfs 2024-05-26T22:36:15.924 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/cramfs/cramfs.ko 2024-05-26T22:36:15.924 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/dlm 2024-05-26T22:36:15.924 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/dlm/dlm.ko 2024-05-26T22:36:15.924 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ecryptfs 2024-05-26T22:36:15.924 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ecryptfs/ecryptfs.ko 2024-05-26T22:36:15.924 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/f2fs 2024-05-26T22:36:15.924 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/f2fs/f2fs.ko 2024-05-26T22:36:15.924 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fat 2024-05-26T22:36:15.924 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fat/fat.ko 2024-05-26T22:36:15.925 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fat/msdos.ko 2024-05-26T22:36:15.925 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fat/vfat.ko 2024-05-26T22:36:15.925 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fuse 2024-05-26T22:36:15.925 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fuse/cuse.ko 2024-05-26T22:36:15.925 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fuse/fuse.ko 2024-05-26T22:36:15.925 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/gfs2 2024-05-26T22:36:15.925 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/gfs2/gfs2.ko 2024-05-26T22:36:15.925 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfs 2024-05-26T22:36:15.925 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfs/hfs.ko 2024-05-26T22:36:15.925 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfsplus 2024-05-26T22:36:15.925 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfsplus/hfsplus.ko 2024-05-26T22:36:15.925 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/isofs 2024-05-26T22:36:15.925 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/isofs/isofs.ko 2024-05-26T22:36:15.925 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/jfs 2024-05-26T22:36:15.925 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/jfs/jfs.ko 2024-05-26T22:36:15.926 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/lockd 2024-05-26T22:36:15.926 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/lockd/lockd.ko 2024-05-26T22:36:15.926 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/minix 2024-05-26T22:36:15.926 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/minix/minix.ko 2024-05-26T22:36:15.926 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/netfs 2024-05-26T22:36:15.926 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/netfs/netfs.ko 2024-05-26T22:36:15.926 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs 2024-05-26T22:36:15.926 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/blocklayout 2024-05-26T22:36:15.926 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-05-26T22:36:15.926 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/filelayout 2024-05-26T22:36:15.926 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-05-26T22:36:15.926 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/flexfilelayout 2024-05-26T22:36:15.927 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-05-26T22:36:15.927 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/nfs.ko 2024-05-26T22:36:15.927 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/nfsv3.ko 2024-05-26T22:36:15.927 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/nfsv4.ko 2024-05-26T22:36:15.927 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs_common 2024-05-26T22:36:15.927 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs_common/grace.ko 2024-05-26T22:36:15.927 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs_common/nfs_acl.ko 2024-05-26T22:36:15.927 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfsd 2024-05-26T22:36:15.927 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfsd/nfsd.ko 2024-05-26T22:36:15.927 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nilfs2 2024-05-26T22:36:15.927 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nilfs2/nilfs2.ko 2024-05-26T22:36:15.927 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls 2024-05-26T22:36:15.927 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-celtic.ko 2024-05-26T22:36:15.927 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-centeuro.ko 2024-05-26T22:36:15.928 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-croatian.ko 2024-05-26T22:36:15.928 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-cyrillic.ko 2024-05-26T22:36:15.928 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-gaelic.ko 2024-05-26T22:36:15.928 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-greek.ko 2024-05-26T22:36:15.928 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-iceland.ko 2024-05-26T22:36:15.928 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-inuit.ko 2024-05-26T22:36:15.928 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-roman.ko 2024-05-26T22:36:15.928 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-romanian.ko 2024-05-26T22:36:15.928 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-turkish.ko 2024-05-26T22:36:15.928 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp1250.ko 2024-05-26T22:36:15.928 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp1251.ko 2024-05-26T22:36:15.928 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp1255.ko 2024-05-26T22:36:15.928 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp737.ko 2024-05-26T22:36:15.928 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp775.ko 2024-05-26T22:36:15.929 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp850.ko 2024-05-26T22:36:15.929 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp852.ko 2024-05-26T22:36:15.929 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp855.ko 2024-05-26T22:36:15.929 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp857.ko 2024-05-26T22:36:15.929 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp860.ko 2024-05-26T22:36:15.929 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp861.ko 2024-05-26T22:36:15.929 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp862.ko 2024-05-26T22:36:15.929 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp863.ko 2024-05-26T22:36:15.929 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp864.ko 2024-05-26T22:36:15.929 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp865.ko 2024-05-26T22:36:15.929 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp866.ko 2024-05-26T22:36:15.930 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp869.ko 2024-05-26T22:36:15.930 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp874.ko 2024-05-26T22:36:15.930 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp932.ko 2024-05-26T22:36:15.930 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp936.ko 2024-05-26T22:36:15.930 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp949.ko 2024-05-26T22:36:15.930 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp950.ko 2024-05-26T22:36:15.930 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_euc-jp.ko 2024-05-26T22:36:15.930 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-1.ko 2024-05-26T22:36:15.930 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-13.ko 2024-05-26T22:36:15.930 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-14.ko 2024-05-26T22:36:15.930 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-15.ko 2024-05-26T22:36:15.930 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-2.ko 2024-05-26T22:36:15.931 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-3.ko 2024-05-26T22:36:15.931 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-4.ko 2024-05-26T22:36:15.931 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-5.ko 2024-05-26T22:36:15.931 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-6.ko 2024-05-26T22:36:15.931 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-7.ko 2024-05-26T22:36:15.931 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-9.ko 2024-05-26T22:36:15.931 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_koi8-r.ko 2024-05-26T22:36:15.931 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_koi8-ru.ko 2024-05-26T22:36:15.931 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_koi8-u.ko 2024-05-26T22:36:15.931 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_ucs2_utils.ko 2024-05-26T22:36:15.931 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_utf8.ko 2024-05-26T22:36:15.931 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2 2024-05-26T22:36:15.931 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/cluster 2024-05-26T22:36:15.931 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-05-26T22:36:15.931 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlm 2024-05-26T22:36:15.932 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-05-26T22:36:15.932 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlmfs 2024-05-26T22:36:15.932 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-05-26T22:36:15.932 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2.ko 2024-05-26T22:36:15.932 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-05-26T22:36:15.932 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-05-26T22:36:15.932 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-05-26T22:36:15.932 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/overlayfs 2024-05-26T22:36:15.932 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/overlayfs/overlay.ko 2024-05-26T22:36:15.932 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/pstore 2024-05-26T22:36:15.932 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/pstore/ramoops.ko 2024-05-26T22:36:15.932 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/reiserfs 2024-05-26T22:36:15.932 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/reiserfs/reiserfs.ko 2024-05-26T22:36:15.933 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/romfs 2024-05-26T22:36:15.933 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/romfs/romfs.ko 2024-05-26T22:36:15.933 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb 2024-05-26T22:36:15.933 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/client 2024-05-26T22:36:15.933 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/client/cifs.ko 2024-05-26T22:36:15.933 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/common 2024-05-26T22:36:15.933 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/common/cifs_arc4.ko 2024-05-26T22:36:15.933 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/common/cifs_md4.ko 2024-05-26T22:36:15.933 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/squashfs 2024-05-26T22:36:15.933 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/squashfs/squashfs.ko 2024-05-26T22:36:15.933 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/sysv 2024-05-26T22:36:15.933 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/sysv/sysv.ko 2024-05-26T22:36:15.933 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/udf 2024-05-26T22:36:15.933 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/udf/udf.ko 2024-05-26T22:36:15.934 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ufs 2024-05-26T22:36:15.934 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ufs/ufs.ko 2024-05-26T22:36:15.934 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/xfs 2024-05-26T22:36:15.934 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/xfs/xfs.ko 2024-05-26T22:36:15.934 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel 2024-05-26T22:36:15.934 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/rcu 2024-05-26T22:36:15.934 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/rcu/rcutorture.ko 2024-05-26T22:36:15.934 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/torture.ko 2024-05-26T22:36:15.934 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/trace 2024-05-26T22:36:15.934 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-05-26T22:36:15.934 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib 2024-05-26T22:36:15.935 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/asn1_encoder.ko 2024-05-26T22:36:15.935 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crc-itu-t.ko 2024-05-26T22:36:15.935 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crc8.ko 2024-05-26T22:36:15.935 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crypto 2024-05-26T22:36:15.935 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crypto/libarc4.ko 2024-05-26T22:36:15.935 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crypto/libdes.ko 2024-05-26T22:36:15.935 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/libcrc32c.ko 2024-05-26T22:36:15.935 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/lru_cache.ko 2024-05-26T22:36:15.935 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/lz4 2024-05-26T22:36:15.935 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/lz4/lz4_compress.ko 2024-05-26T22:36:15.935 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/lz4/lz4hc_compress.ko 2024-05-26T22:36:15.935 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/math 2024-05-26T22:36:15.935 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/math/cordic.ko 2024-05-26T22:36:15.935 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/raid6 2024-05-26T22:36:15.936 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/raid6/raid6_pq.ko 2024-05-26T22:36:15.936 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/reed_solomon 2024-05-26T22:36:15.936 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/reed_solomon/reed_solomon.ko 2024-05-26T22:36:15.936 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/ts_bm.ko 2024-05-26T22:36:15.936 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/ts_fsm.ko 2024-05-26T22:36:15.936 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/ts_kmp.ko 2024-05-26T22:36:15.936 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/zstd 2024-05-26T22:36:15.936 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/zstd/zstd_compress.ko 2024-05-26T22:36:15.936 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/mm 2024-05-26T22:36:15.936 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/mm/hwpoison-inject.ko 2024-05-26T22:36:15.936 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net 2024-05-26T22:36:15.936 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan 2024-05-26T22:36:15.936 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/6lowpan.ko 2024-05-26T22:36:15.937 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_dest.ko 2024-05-26T22:36:15.937 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_fragment.ko 2024-05-26T22:36:15.937 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_hop.ko 2024-05-26T22:36:15.937 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_ipv6.ko 2024-05-26T22:36:15.937 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_mobility.ko 2024-05-26T22:36:15.937 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_routing.ko 2024-05-26T22:36:15.937 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_udp.ko 2024-05-26T22:36:15.937 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802 2024-05-26T22:36:15.937 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/garp.ko 2024-05-26T22:36:15.937 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/mrp.ko 2024-05-26T22:36:15.937 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/p8022.ko 2024-05-26T22:36:15.937 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/psnap.ko 2024-05-26T22:36:15.937 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/stp.ko 2024-05-26T22:36:15.937 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/8021q 2024-05-26T22:36:15.938 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/8021q/8021q.ko 2024-05-26T22:36:15.938 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p 2024-05-26T22:36:15.938 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet.ko 2024-05-26T22:36:15.938 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet_fd.ko 2024-05-26T22:36:15.938 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet_rdma.ko 2024-05-26T22:36:15.938 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet_virtio.ko 2024-05-26T22:36:15.938 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/batman-adv 2024-05-26T22:36:15.938 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/batman-adv/batman-adv.ko 2024-05-26T22:36:15.938 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge 2024-05-26T22:36:15.938 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/br_netfilter.ko 2024-05-26T22:36:15.938 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/bridge.ko 2024-05-26T22:36:15.939 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter 2024-05-26T22:36:15.939 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-05-26T22:36:15.939 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_among.ko 2024-05-26T22:36:15.939 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_arp.ko 2024-05-26T22:36:15.939 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-05-26T22:36:15.939 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-05-26T22:36:15.939 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_ip.ko 2024-05-26T22:36:15.939 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-05-26T22:36:15.939 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_limit.ko 2024-05-26T22:36:15.939 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_log.ko 2024-05-26T22:36:15.939 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_mark.ko 2024-05-26T22:36:15.939 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-05-26T22:36:15.939 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-05-26T22:36:15.939 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-05-26T22:36:15.940 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-05-26T22:36:15.940 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_snat.ko 2024-05-26T22:36:15.940 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_stp.ko 2024-05-26T22:36:15.940 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-05-26T22:36:15.940 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-05-26T22:36:15.940 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-05-26T22:36:15.940 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-05-26T22:36:15.940 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtables.ko 2024-05-26T22:36:15.940 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-05-26T22:36:15.940 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-05-26T22:36:15.940 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-05-26T22:36:15.940 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ceph 2024-05-26T22:36:15.940 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ceph/libceph.ko 2024-05-26T22:36:15.941 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/core 2024-05-26T22:36:15.941 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/core/failover.ko 2024-05-26T22:36:15.941 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/core/pktgen.ko 2024-05-26T22:36:15.941 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp 2024-05-26T22:36:15.941 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp.ko 2024-05-26T22:36:15.941 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp_diag.ko 2024-05-26T22:36:15.941 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp_ipv4.ko 2024-05-26T22:36:15.941 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp_ipv6.ko 2024-05-26T22:36:15.941 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dns_resolver 2024-05-26T22:36:15.941 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dns_resolver/dns_resolver.ko 2024-05-26T22:36:15.941 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa 2024-05-26T22:36:15.942 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/dsa_core.ko 2024-05-26T22:36:15.942 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_brcm.ko 2024-05-26T22:36:15.942 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_dsa.ko 2024-05-26T22:36:15.942 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_none.ko 2024-05-26T22:36:15.942 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_trailer.ko 2024-05-26T22:36:15.942 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154 2024-05-26T22:36:15.942 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/6lowpan 2024-05-26T22:36:15.942 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-05-26T22:36:15.942 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/ieee802154.ko 2024-05-26T22:36:15.942 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/ieee802154_socket.ko 2024-05-26T22:36:15.942 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4 2024-05-26T22:36:15.942 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ah4.ko 2024-05-26T22:36:15.943 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/esp4.ko 2024-05-26T22:36:15.943 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/fou.ko 2024-05-26T22:36:15.943 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/gre.ko 2024-05-26T22:36:15.943 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/inet_diag.ko 2024-05-26T22:36:15.943 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ip_gre.ko 2024-05-26T22:36:15.943 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ip_tunnel.ko 2024-05-26T22:36:15.943 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ip_vti.ko 2024-05-26T22:36:15.943 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ipcomp.ko 2024-05-26T22:36:15.943 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ipip.ko 2024-05-26T22:36:15.943 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter 2024-05-26T22:36:15.943 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/arp_tables.ko 2024-05-26T22:36:15.944 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-05-26T22:36:15.944 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-05-26T22:36:15.944 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-05-26T22:36:15.944 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-05-26T22:36:15.944 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-05-26T22:36:15.944 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-05-26T22:36:15.944 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-05-26T22:36:15.944 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-05-26T22:36:15.944 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-05-26T22:36:15.944 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_security.ko 2024-05-26T22:36:15.944 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-05-26T22:36:15.945 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-05-26T22:36:15.945 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-05-26T22:36:15.945 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-05-26T22:36:15.945 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-05-26T22:36:15.945 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-05-26T22:36:15.945 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-05-26T22:36:15.945 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-05-26T22:36:15.945 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-05-26T22:36:15.945 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-05-26T22:36:15.945 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_bic.ko 2024-05-26T22:36:15.945 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_dctcp.ko 2024-05-26T22:36:15.945 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_diag.ko 2024-05-26T22:36:15.945 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_highspeed.ko 2024-05-26T22:36:15.946 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_htcp.ko 2024-05-26T22:36:15.946 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_hybla.ko 2024-05-26T22:36:15.946 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_illinois.ko 2024-05-26T22:36:15.946 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_lp.ko 2024-05-26T22:36:15.946 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_scalable.ko 2024-05-26T22:36:15.946 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_vegas.ko 2024-05-26T22:36:15.946 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_veno.ko 2024-05-26T22:36:15.946 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_westwood.ko 2024-05-26T22:36:15.946 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_yeah.ko 2024-05-26T22:36:15.946 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tunnel4.ko 2024-05-26T22:36:15.946 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/udp_diag.ko 2024-05-26T22:36:15.946 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/udp_tunnel.ko 2024-05-26T22:36:15.946 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/xfrm4_tunnel.ko 2024-05-26T22:36:15.947 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6 2024-05-26T22:36:15.947 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ah6.ko 2024-05-26T22:36:15.947 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/esp6.ko 2024-05-26T22:36:15.947 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/fou6.ko 2024-05-26T22:36:15.947 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ip6_tunnel.ko 2024-05-26T22:36:15.947 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-05-26T22:36:15.947 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ip6_vti.ko 2024-05-26T22:36:15.947 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ipcomp6.ko 2024-05-26T22:36:15.947 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter 2024-05-26T22:36:15.947 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-05-26T22:36:15.947 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-05-26T22:36:15.947 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-05-26T22:36:15.947 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-05-26T22:36:15.947 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-05-26T22:36:15.948 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-05-26T22:36:15.948 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-05-26T22:36:15.948 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-05-26T22:36:15.948 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-05-26T22:36:15.948 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-05-26T22:36:15.948 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-05-26T22:36:15.948 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-05-26T22:36:15.948 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-05-26T22:36:15.948 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-05-26T22:36:15.948 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-05-26T22:36:15.948 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-05-26T22:36:15.948 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-05-26T22:36:15.948 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-05-26T22:36:15.948 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-05-26T22:36:15.949 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-05-26T22:36:15.949 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-05-26T22:36:15.949 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-05-26T22:36:15.949 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-05-26T22:36:15.949 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-05-26T22:36:15.949 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/sit.ko 2024-05-26T22:36:15.949 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/tunnel6.ko 2024-05-26T22:36:15.949 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/xfrm6_tunnel.ko 2024-05-26T22:36:15.949 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/key 2024-05-26T22:36:15.949 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/key/af_key.ko 2024-05-26T22:36:15.949 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp 2024-05-26T22:36:15.949 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_core.ko 2024-05-26T22:36:15.949 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_debugfs.ko 2024-05-26T22:36:15.949 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_eth.ko 2024-05-26T22:36:15.950 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_ip.ko 2024-05-26T22:36:15.950 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_ip6.ko 2024-05-26T22:36:15.950 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_netlink.ko 2024-05-26T22:36:15.950 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_ppp.ko 2024-05-26T22:36:15.950 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/llc 2024-05-26T22:36:15.950 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/llc/llc.ko 2024-05-26T22:36:15.950 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/mac802154 2024-05-26T22:36:15.950 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/mac802154/mac802154.ko 2024-05-26T22:36:15.950 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/mpls 2024-05-26T22:36:15.950 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/mpls/mpls_gso.ko 2024-05-26T22:36:15.950 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter 2024-05-26T22:36:15.950 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset 2024-05-26T22:36:15.951 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set.ko 2024-05-26T22:36:15.951 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-05-26T22:36:15.951 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-05-26T22:36:15.951 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-05-26T22:36:15.951 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-05-26T22:36:15.951 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-05-26T22:36:15.951 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-05-26T22:36:15.951 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-05-26T22:36:15.951 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-05-26T22:36:15.951 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-05-26T22:36:15.952 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-05-26T22:36:15.952 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-05-26T22:36:15.952 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-05-26T22:36:15.952 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-05-26T22:36:15.952 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-05-26T22:36:15.952 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-05-26T22:36:15.952 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs 2024-05-26T22:36:15.952 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs.ko 2024-05-26T22:36:15.952 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-05-26T22:36:15.952 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-05-26T22:36:15.952 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-05-26T22:36:15.952 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-05-26T22:36:15.952 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-05-26T22:36:15.952 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-05-26T22:36:15.953 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-05-26T22:36:15.953 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-05-26T22:36:15.953 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-05-26T22:36:15.953 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-05-26T22:36:15.953 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-05-26T22:36:15.953 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-05-26T22:36:15.953 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-05-26T22:36:15.953 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conncount.ko 2024-05-26T22:36:15.953 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack.ko 2024-05-26T22:36:15.953 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-05-26T22:36:15.953 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-05-26T22:36:15.953 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-05-26T22:36:15.954 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_h323.ko 2024-05-26T22:36:15.954 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_irc.ko 2024-05-26T22:36:15.954 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-05-26T22:36:15.954 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-05-26T22:36:15.954 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-05-26T22:36:15.954 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_sane.ko 2024-05-26T22:36:15.954 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_sip.ko 2024-05-26T22:36:15.954 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-05-26T22:36:15.954 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-05-26T22:36:15.954 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_dup_netdev.ko 2024-05-26T22:36:15.954 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_flow_table.ko 2024-05-26T22:36:15.954 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_flow_table_inet.ko 2024-05-26T22:36:15.955 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_log_syslog.ko 2024-05-26T22:36:15.955 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat.ko 2024-05-26T22:36:15.955 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_amanda.ko 2024-05-26T22:36:15.955 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_ftp.ko 2024-05-26T22:36:15.955 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_irc.ko 2024-05-26T22:36:15.955 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_sip.ko 2024-05-26T22:36:15.955 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_tftp.ko 2024-05-26T22:36:15.955 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_synproxy_core.ko 2024-05-26T22:36:15.955 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_tables.ko 2024-05-26T22:36:15.955 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink.ko 2024-05-26T22:36:15.955 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_acct.ko 2024-05-26T22:36:15.955 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_log.ko 2024-05-26T22:36:15.955 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_osf.ko 2024-05-26T22:36:15.956 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_queue.ko 2024-05-26T22:36:15.956 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_chain_nat.ko 2024-05-26T22:36:15.956 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_compat.ko 2024-05-26T22:36:15.956 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_ct.ko 2024-05-26T22:36:15.956 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_dup_netdev.ko 2024-05-26T22:36:15.956 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_fib.ko 2024-05-26T22:36:15.956 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_fwd_netdev.ko 2024-05-26T22:36:15.956 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_hash.ko 2024-05-26T22:36:15.957 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_limit.ko 2024-05-26T22:36:15.957 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_log.ko 2024-05-26T22:36:15.957 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_masq.ko 2024-05-26T22:36:15.957 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_nat.ko 2024-05-26T22:36:15.957 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_numgen.ko 2024-05-26T22:36:15.957 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_queue.ko 2024-05-26T22:36:15.957 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_quota.ko 2024-05-26T22:36:15.958 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_redir.ko 2024-05-26T22:36:15.958 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_reject.ko 2024-05-26T22:36:15.958 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_reject_inet.ko 2024-05-26T22:36:15.958 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_socket.ko 2024-05-26T22:36:15.958 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_tunnel.ko 2024-05-26T22:36:15.958 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_AUDIT.ko 2024-05-26T22:36:15.958 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CHECKSUM.ko 2024-05-26T22:36:15.958 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CLASSIFY.ko 2024-05-26T22:36:15.958 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-05-26T22:36:15.958 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CT.ko 2024-05-26T22:36:15.958 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_DSCP.ko 2024-05-26T22:36:15.959 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_HL.ko 2024-05-26T22:36:15.959 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_HMARK.ko 2024-05-26T22:36:15.959 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_IDLETIMER.ko 2024-05-26T22:36:15.959 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_LED.ko 2024-05-26T22:36:15.959 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_LOG.ko 2024-05-26T22:36:15.959 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_MASQUERADE.ko 2024-05-26T22:36:15.959 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_NETMAP.ko 2024-05-26T22:36:15.959 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_NFLOG.ko 2024-05-26T22:36:15.959 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_NFQUEUE.ko 2024-05-26T22:36:15.959 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_RATEEST.ko 2024-05-26T22:36:15.959 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_REDIRECT.ko 2024-05-26T22:36:15.959 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_SECMARK.ko 2024-05-26T22:36:15.959 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TCPMSS.ko 2024-05-26T22:36:15.959 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-05-26T22:36:15.960 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TEE.ko 2024-05-26T22:36:15.960 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TPROXY.ko 2024-05-26T22:36:15.960 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TRACE.ko 2024-05-26T22:36:15.960 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_addrtype.ko 2024-05-26T22:36:15.960 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_bpf.ko 2024-05-26T22:36:15.960 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_cgroup.ko 2024-05-26T22:36:15.960 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_cluster.ko 2024-05-26T22:36:15.960 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_comment.ko 2024-05-26T22:36:15.960 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connbytes.ko 2024-05-26T22:36:15.960 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connlabel.ko 2024-05-26T22:36:15.960 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connlimit.ko 2024-05-26T22:36:15.960 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connmark.ko 2024-05-26T22:36:15.960 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_conntrack.ko 2024-05-26T22:36:15.961 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_cpu.ko 2024-05-26T22:36:15.961 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_dccp.ko 2024-05-26T22:36:15.961 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_devgroup.ko 2024-05-26T22:36:15.961 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_dscp.ko 2024-05-26T22:36:15.961 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_ecn.ko 2024-05-26T22:36:15.961 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_esp.ko 2024-05-26T22:36:15.961 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_hashlimit.ko 2024-05-26T22:36:15.961 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_helper.ko 2024-05-26T22:36:15.961 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_hl.ko 2024-05-26T22:36:15.961 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_ipcomp.ko 2024-05-26T22:36:15.961 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_iprange.ko 2024-05-26T22:36:15.961 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_ipvs.ko 2024-05-26T22:36:15.961 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_l2tp.ko 2024-05-26T22:36:15.961 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_length.ko 2024-05-26T22:36:15.961 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_limit.ko 2024-05-26T22:36:15.962 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_mac.ko 2024-05-26T22:36:15.962 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_mark.ko 2024-05-26T22:36:15.962 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_multiport.ko 2024-05-26T22:36:15.962 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_nat.ko 2024-05-26T22:36:15.962 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_nfacct.ko 2024-05-26T22:36:15.962 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_osf.ko 2024-05-26T22:36:15.962 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_owner.ko 2024-05-26T22:36:15.962 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_physdev.ko 2024-05-26T22:36:15.962 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_pkttype.ko 2024-05-26T22:36:15.962 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_policy.ko 2024-05-26T22:36:15.962 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_quota.ko 2024-05-26T22:36:15.963 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_rateest.ko 2024-05-26T22:36:15.963 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_realm.ko 2024-05-26T22:36:15.963 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_recent.ko 2024-05-26T22:36:15.963 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_sctp.ko 2024-05-26T22:36:15.963 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_set.ko 2024-05-26T22:36:15.963 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_socket.ko 2024-05-26T22:36:15.963 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_state.ko 2024-05-26T22:36:15.964 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_statistic.ko 2024-05-26T22:36:15.964 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_string.ko 2024-05-26T22:36:15.964 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_tcpmss.ko 2024-05-26T22:36:15.964 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_time.ko 2024-05-26T22:36:15.964 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_u32.ko 2024-05-26T22:36:15.964 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netlink 2024-05-26T22:36:15.964 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netlink/netlink_diag.ko 2024-05-26T22:36:15.964 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/nsh 2024-05-26T22:36:15.964 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/nsh/nsh.ko 2024-05-26T22:36:15.964 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch 2024-05-26T22:36:15.964 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/openvswitch.ko 2024-05-26T22:36:15.964 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/vport-geneve.ko 2024-05-26T22:36:15.965 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/vport-gre.ko 2024-05-26T22:36:15.965 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/vport-vxlan.ko 2024-05-26T22:36:15.965 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/packet 2024-05-26T22:36:15.965 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/packet/af_packet_diag.ko 2024-05-26T22:36:15.965 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/rds 2024-05-26T22:36:15.965 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/rds/rds.ko 2024-05-26T22:36:15.965 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/rds/rds_rdma.ko 2024-05-26T22:36:15.965 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/rds/rds_tcp.ko 2024-05-26T22:36:15.965 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched 2024-05-26T22:36:15.965 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_csum.ko 2024-05-26T22:36:15.965 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_gact.ko 2024-05-26T22:36:15.965 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_mirred.ko 2024-05-26T22:36:15.965 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_nat.ko 2024-05-26T22:36:15.965 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_pedit.ko 2024-05-26T22:36:15.966 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_police.ko 2024-05-26T22:36:15.966 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_simple.ko 2024-05-26T22:36:15.966 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_skbedit.ko 2024-05-26T22:36:15.966 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_vlan.ko 2024-05-26T22:36:15.966 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_basic.ko 2024-05-26T22:36:15.966 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_bpf.ko 2024-05-26T22:36:15.966 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_flow.ko 2024-05-26T22:36:15.966 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_fw.ko 2024-05-26T22:36:15.966 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_route.ko 2024-05-26T22:36:15.966 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_u32.ko 2024-05-26T22:36:15.966 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_cmp.ko 2024-05-26T22:36:15.966 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_ipset.ko 2024-05-26T22:36:15.967 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_meta.ko 2024-05-26T22:36:15.967 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_nbyte.ko 2024-05-26T22:36:15.967 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_text.ko 2024-05-26T22:36:15.967 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_u32.ko 2024-05-26T22:36:15.967 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_choke.ko 2024-05-26T22:36:15.967 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_codel.ko 2024-05-26T22:36:15.967 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_drr.ko 2024-05-26T22:36:15.967 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_fq.ko 2024-05-26T22:36:15.967 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_gred.ko 2024-05-26T22:36:15.967 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_hfsc.ko 2024-05-26T22:36:15.967 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_hhf.ko 2024-05-26T22:36:15.967 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_htb.ko 2024-05-26T22:36:15.967 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_ingress.ko 2024-05-26T22:36:15.967 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_mqprio.ko 2024-05-26T22:36:15.968 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_mqprio_lib.ko 2024-05-26T22:36:15.968 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_multiq.ko 2024-05-26T22:36:15.968 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_netem.ko 2024-05-26T22:36:15.968 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_pie.ko 2024-05-26T22:36:15.968 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_plug.ko 2024-05-26T22:36:15.968 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_prio.ko 2024-05-26T22:36:15.968 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_qfq.ko 2024-05-26T22:36:15.968 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_red.ko 2024-05-26T22:36:15.968 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_sfb.ko 2024-05-26T22:36:15.968 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_sfq.ko 2024-05-26T22:36:15.968 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_tbf.ko 2024-05-26T22:36:15.968 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_teql.ko 2024-05-26T22:36:15.968 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sctp 2024-05-26T22:36:15.969 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sctp/sctp.ko 2024-05-26T22:36:15.969 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sctp/sctp_diag.ko 2024-05-26T22:36:15.969 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc 2024-05-26T22:36:15.969 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/auth_gss 2024-05-26T22:36:15.969 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-05-26T22:36:15.969 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-05-26T22:36:15.969 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/sunrpc.ko 2024-05-26T22:36:15.969 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/xprtrdma 2024-05-26T22:36:15.969 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-05-26T22:36:15.969 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/tipc 2024-05-26T22:36:15.969 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/tipc/tipc.ko 2024-05-26T22:36:15.969 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/tipc/tipc_diag.ko 2024-05-26T22:36:15.969 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/unix 2024-05-26T22:36:15.969 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/unix/unix_diag.ko 2024-05-26T22:36:15.970 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock 2024-05-26T22:36:15.970 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-05-26T22:36:15.970 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-05-26T22:36:15.970 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vsock.ko 2024-05-26T22:36:15.970 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vsock_diag.ko 2024-05-26T22:36:15.970 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vsock_loopback.ko 2024-05-26T22:36:15.970 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/xfrm 2024-05-26T22:36:15.970 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/xfrm/xfrm_ipcomp.ko 2024-05-26T22:36:15.970 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security 2024-05-26T22:36:15.970 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys 2024-05-26T22:36:15.970 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys/encrypted-keys 2024-05-26T22:36:15.970 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-05-26T22:36:15.970 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys/trusted-keys 2024-05-26T22:36:15.971 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys/trusted-keys/trusted.ko 2024-05-26T22:36:15.971 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.alias 2024-05-26T22:36:15.971 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.alias.bin 2024-05-26T22:36:15.971 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.builtin 2024-05-26T22:36:15.971 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.builtin.alias.bin 2024-05-26T22:36:15.971 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.builtin.bin 2024-05-26T22:36:15.971 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.builtin.modinfo 2024-05-26T22:36:15.971 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.dep 2024-05-26T22:36:15.971 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.dep.bin 2024-05-26T22:36:15.971 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.devname 2024-05-26T22:36:15.971 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.order 2024-05-26T22:36:15.971 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.softdep 2024-05-26T22:36:15.971 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.symbols 2024-05-26T22:36:15.972 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.symbols.bin 2024-05-26T22:36:15.972 INFO:teuthology.orchestra.run.smithi187.stdout:/lib/modules/6.9.0-gb9459a26b088/vmlinuz 2024-05-26T22:36:15.976 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-gb9459a26b088 2024-05-26T22:36:15.976 DEBUG:teuthology.orchestra.run.smithi187:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-05-26T22:36:15.999 INFO:teuthology.orchestra.run.smithi187.stdout:postinstall scriptlet (using /bin/sh): 2024-05-26T22:36:15.999 INFO:teuthology.orchestra.run.smithi187.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-05-26T22:36:15.999 INFO:teuthology.orchestra.run.smithi187.stdout: /usr/bin/kernel-install add 6.9.0-gb9459a26b088 /lib/modules/6.9.0-gb9459a26b088/vmlinuz 2024-05-26T22:36:15.999 INFO:teuthology.orchestra.run.smithi187.stdout:fi 2024-05-26T22:36:15.999 INFO:teuthology.orchestra.run.smithi187.stdout:for file in vmlinuz System.map config; do 2024-05-26T22:36:15.999 INFO:teuthology.orchestra.run.smithi187.stdout: if ! cmp --silent "/lib/modules/6.9.0-gb9459a26b088/${file}" "/boot/${file}-6.9.0-gb9459a26b088"; then 2024-05-26T22:36:15.999 INFO:teuthology.orchestra.run.smithi187.stdout: cp "/lib/modules/6.9.0-gb9459a26b088/${file}" "/boot/${file}-6.9.0-gb9459a26b088" 2024-05-26T22:36:15.999 INFO:teuthology.orchestra.run.smithi187.stdout: fi 2024-05-26T22:36:15.999 INFO:teuthology.orchestra.run.smithi187.stdout:done 2024-05-26T22:36:15.999 INFO:teuthology.orchestra.run.smithi187.stdout:if [ -d "/lib/modules/6.9.0-gb9459a26b088/dtb" ] && \ 2024-05-26T22:36:16.000 INFO:teuthology.orchestra.run.smithi187.stdout: ! diff -rq "/lib/modules/6.9.0-gb9459a26b088/dtb" "/boot/dtb-6.9.0-gb9459a26b088" >/dev/null 2>&1; then 2024-05-26T22:36:16.000 INFO:teuthology.orchestra.run.smithi187.stdout: rm -rf "/boot/dtb-6.9.0-gb9459a26b088" 2024-05-26T22:36:16.000 INFO:teuthology.orchestra.run.smithi187.stdout: cp -r "/lib/modules/6.9.0-gb9459a26b088/dtb" "/boot/dtb-6.9.0-gb9459a26b088" 2024-05-26T22:36:16.000 INFO:teuthology.orchestra.run.smithi187.stdout:fi 2024-05-26T22:36:16.000 INFO:teuthology.orchestra.run.smithi187.stdout:if [ ! -e "/lib/modules/6.9.0-gb9459a26b088/modules.dep" ]; then 2024-05-26T22:36:16.000 INFO:teuthology.orchestra.run.smithi187.stdout: /usr/sbin/depmod 6.9.0-gb9459a26b088 2024-05-26T22:36:16.000 INFO:teuthology.orchestra.run.smithi187.stdout:fi 2024-05-26T22:36:16.000 INFO:teuthology.orchestra.run.smithi187.stdout:preuninstall scriptlet (using /bin/sh): 2024-05-26T22:36:16.000 INFO:teuthology.orchestra.run.smithi187.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-05-26T22:36:16.001 INFO:teuthology.orchestra.run.smithi187.stdout:kernel-install remove 6.9.0-gb9459a26b088 2024-05-26T22:36:16.001 INFO:teuthology.orchestra.run.smithi187.stdout:fi 2024-05-26T22:36:16.002 DEBUG:teuthology.orchestra.run.smithi187:> sudo rpm -qi grub2-tools 2024-05-26T22:36:16.074 INFO:teuthology.orchestra.run.smithi187.stdout:Name : grub2-tools 2024-05-26T22:36:16.074 INFO:teuthology.orchestra.run.smithi187.stdout:Epoch : 1 2024-05-26T22:36:16.074 INFO:teuthology.orchestra.run.smithi187.stdout:Version : 2.06 2024-05-26T22:36:16.074 INFO:teuthology.orchestra.run.smithi187.stdout:Release : 68.el9 2024-05-26T22:36:16.074 INFO:teuthology.orchestra.run.smithi187.stdout:Architecture: x86_64 2024-05-26T22:36:16.074 INFO:teuthology.orchestra.run.smithi187.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-26T22:36:16.074 INFO:teuthology.orchestra.run.smithi187.stdout:Group : Unspecified 2024-05-26T22:36:16.074 INFO:teuthology.orchestra.run.smithi187.stdout:Size : 8264254 2024-05-26T22:36:16.074 INFO:teuthology.orchestra.run.smithi187.stdout:License : GPLv3+ 2024-05-26T22:36:16.074 INFO:teuthology.orchestra.run.smithi187.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-26T22:36:16.075 INFO:teuthology.orchestra.run.smithi187.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-26T22:36:16.075 INFO:teuthology.orchestra.run.smithi187.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-26T22:36:16.075 INFO:teuthology.orchestra.run.smithi187.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-26T22:36:16.075 INFO:teuthology.orchestra.run.smithi187.stdout:Packager : builder@centos.org 2024-05-26T22:36:16.075 INFO:teuthology.orchestra.run.smithi187.stdout:Vendor : CentOS 2024-05-26T22:36:16.075 INFO:teuthology.orchestra.run.smithi187.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-26T22:36:16.075 INFO:teuthology.orchestra.run.smithi187.stdout:Summary : Support tools for GRUB. 2024-05-26T22:36:16.075 INFO:teuthology.orchestra.run.smithi187.stdout:Description : 2024-05-26T22:36:16.075 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-05-26T22:36:16.075 INFO:teuthology.orchestra.run.smithi187.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-26T22:36:16.075 INFO:teuthology.orchestra.run.smithi187.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-26T22:36:16.075 INFO:teuthology.orchestra.run.smithi187.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-26T22:36:16.075 INFO:teuthology.orchestra.run.smithi187.stdout:hardware devices. 2024-05-26T22:36:16.075 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-05-26T22:36:16.075 INFO:teuthology.orchestra.run.smithi187.stdout:This subpackage provides tools for support of all platforms. 2024-05-26T22:36:16.076 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-26T22:36:16.076 INFO:teuthology.task.kernel:Updating grub on smithi187 to boot 6.9.0-gb9459a26b088 2024-05-26T22:36:16.077 DEBUG:teuthology.orchestra.run.smithi187:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-26T22:36:16.659 INFO:teuthology.orchestra.run.smithi187.stderr:Generating grub configuration file ... 2024-05-26T22:36:18.067 INFO:teuthology.orchestra.run.smithi187.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-26T22:36:18.092 INFO:teuthology.orchestra.run.smithi187.stderr:done 2024-05-26T22:36:18.094 DEBUG:teuthology.orchestra.run.smithi187:> mktemp 2024-05-26T22:36:18.109 INFO:teuthology.orchestra.run.smithi187.stdout:/tmp/tmp.3NG0oH3F9k 2024-05-26T22:36:18.109 DEBUG:teuthology.orchestra.run.smithi187:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.3NG0oH3F9k 2024-05-26T22:36:18.185 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod 0666 /tmp/tmp.3NG0oH3F9k 2024-05-26T22:36:18.330 DEBUG:teuthology.orchestra.remote:smithi187:/tmp/tmp.3NG0oH3F9k is 6KB 2024-05-26T22:36:18.349 DEBUG:teuthology.orchestra.run.smithi187:> rm -fr /tmp/tmp.3NG0oH3F9k 2024-05-26T22:36:18.363 DEBUG:teuthology.orchestra.run.smithi187:> sudo /bin/ls /boot/loader/entries || true 2024-05-26T22:36:18.425 INFO:teuthology.orchestra.run.smithi187.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-26T22:36:18.425 INFO:teuthology.orchestra.run.smithi187.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-26T22:36:18.425 INFO:teuthology.orchestra.run.smithi187.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-26T22:36:18.425 INFO:teuthology.orchestra.run.smithi187.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-gb9459a26b088.conf 2024-05-26T22:36:18.427 DEBUG:teuthology.orchestra.run.smithi187:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-gb9459a26b088 2024-05-26T22:36:18.562 DEBUG:teuthology.orchestra.run.smithi187:> sudo shutdown -r now 2024-05-26T22:36:33.906 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi131.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-05-26T22:36:33.906 DEBUG:teuthology.orchestra.run.smithi131:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-05-26T22:36:33.985 INFO:teuthology.orchestra.run.smithi131.stdout:/boot/System.map-6.9.0-gb9459a26b088 2024-05-26T22:36:33.985 INFO:teuthology.orchestra.run.smithi131.stdout:/boot/config-6.9.0-gb9459a26b088 2024-05-26T22:36:33.985 INFO:teuthology.orchestra.run.smithi131.stdout:/boot/vmlinuz-6.9.0-gb9459a26b088 2024-05-26T22:36:33.985 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/System.map 2024-05-26T22:36:33.986 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/config 2024-05-26T22:36:33.986 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel 2024-05-26T22:36:33.986 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch 2024-05-26T22:36:33.986 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86 2024-05-26T22:36:33.986 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto 2024-05-26T22:36:33.986 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-05-26T22:36:33.986 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-05-26T22:36:33.986 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-05-26T22:36:33.986 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-05-26T22:36:33.986 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-05-26T22:36:33.986 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-05-26T22:36:33.986 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-05-26T22:36:33.987 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/crc32c-intel.ko 2024-05-26T22:36:33.987 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-05-26T22:36:33.987 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-05-26T22:36:33.987 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-05-26T22:36:33.987 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-05-26T22:36:33.987 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/serpent-avx2.ko 2024-05-26T22:36:33.987 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-05-26T22:36:33.987 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-05-26T22:36:33.987 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-05-26T22:36:33.987 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-05-26T22:36:33.987 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-05-26T22:36:33.987 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-05-26T22:36:33.988 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-05-26T22:36:33.988 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel 2024-05-26T22:36:33.988 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel/cpu 2024-05-26T22:36:33.988 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel/cpu/mce 2024-05-26T22:36:33.988 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-05-26T22:36:33.988 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm 2024-05-26T22:36:33.988 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm/kvm-amd.ko 2024-05-26T22:36:33.988 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm/kvm-intel.ko 2024-05-26T22:36:33.988 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm/kvm.ko 2024-05-26T22:36:33.988 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto 2024-05-26T22:36:33.988 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/ansi_cprng.ko 2024-05-26T22:36:33.988 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/anubis.ko 2024-05-26T22:36:33.988 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/arc4.ko 2024-05-26T22:36:33.989 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx 2024-05-26T22:36:33.989 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_memcpy.ko 2024-05-26T22:36:33.989 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_pq.ko 2024-05-26T22:36:33.989 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_raid6_recov.ko 2024-05-26T22:36:33.989 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_tx.ko 2024-05-26T22:36:33.989 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_xor.ko 2024-05-26T22:36:33.989 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/raid6test.ko 2024-05-26T22:36:33.989 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/authenc.ko 2024-05-26T22:36:33.989 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/authencesn.ko 2024-05-26T22:36:33.989 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/blake2b_generic.ko 2024-05-26T22:36:33.989 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/blowfish_common.ko 2024-05-26T22:36:33.989 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/blowfish_generic.ko 2024-05-26T22:36:33.989 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/camellia_generic.ko 2024-05-26T22:36:33.990 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cast5_generic.ko 2024-05-26T22:36:33.990 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cast6_generic.ko 2024-05-26T22:36:33.990 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cast_common.ko 2024-05-26T22:36:33.990 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/ccm.ko 2024-05-26T22:36:33.990 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cmac.ko 2024-05-26T22:36:33.990 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/crc32_generic.ko 2024-05-26T22:36:33.990 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/crypto_engine.ko 2024-05-26T22:36:33.990 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/crypto_user.ko 2024-05-26T22:36:33.990 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cts.ko 2024-05-26T22:36:33.990 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/des_generic.ko 2024-05-26T22:36:33.990 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/dh_generic.ko 2024-05-26T22:36:33.991 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/echainiv.ko 2024-05-26T22:36:33.991 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/essiv.ko 2024-05-26T22:36:33.991 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/fcrypt.ko 2024-05-26T22:36:33.991 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/khazad.ko 2024-05-26T22:36:33.991 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/lz4.ko 2024-05-26T22:36:33.991 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/lz4hc.ko 2024-05-26T22:36:33.991 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/md4.ko 2024-05-26T22:36:33.992 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/michael_mic.ko 2024-05-26T22:36:33.992 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/pcbc.ko 2024-05-26T22:36:33.992 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/pcrypt.ko 2024-05-26T22:36:33.992 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/rmd160.ko 2024-05-26T22:36:33.992 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/seed.ko 2024-05-26T22:36:33.992 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/serpent_generic.ko 2024-05-26T22:36:33.992 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/tcrypt.ko 2024-05-26T22:36:33.992 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/tea.ko 2024-05-26T22:36:33.992 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/twofish_common.ko 2024-05-26T22:36:33.992 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/twofish_generic.ko 2024-05-26T22:36:33.992 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/vmac.ko 2024-05-26T22:36:33.992 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/wp512.ko 2024-05-26T22:36:33.992 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/xcbc.ko 2024-05-26T22:36:33.993 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/xor.ko 2024-05-26T22:36:33.993 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/xxhash_generic.ko 2024-05-26T22:36:33.993 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers 2024-05-26T22:36:33.993 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi 2024-05-26T22:36:33.993 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/acpi_ipmi.ko 2024-05-26T22:36:33.993 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/acpi_pad.ko 2024-05-26T22:36:33.993 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/ec_sys.ko 2024-05-26T22:36:33.993 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/platform_profile.ko 2024-05-26T22:36:33.993 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/sbs.ko 2024-05-26T22:36:33.993 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/sbshc.ko 2024-05-26T22:36:33.993 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/video.ko 2024-05-26T22:36:33.993 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata 2024-05-26T22:36:33.994 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/acard-ahci.ko 2024-05-26T22:36:33.994 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/ahci_platform.ko 2024-05-26T22:36:33.994 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/ata_generic.ko 2024-05-26T22:36:33.994 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/libahci_platform.ko 2024-05-26T22:36:33.994 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_acpi.ko 2024-05-26T22:36:33.994 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ali.ko 2024-05-26T22:36:33.994 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_amd.ko 2024-05-26T22:36:33.994 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_artop.ko 2024-05-26T22:36:33.994 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_atiixp.ko 2024-05-26T22:36:33.994 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_atp867x.ko 2024-05-26T22:36:33.994 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_cmd640.ko 2024-05-26T22:36:33.994 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_cmd64x.ko 2024-05-26T22:36:33.994 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_cypress.ko 2024-05-26T22:36:33.995 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_efar.ko 2024-05-26T22:36:33.995 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt366.ko 2024-05-26T22:36:33.995 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt37x.ko 2024-05-26T22:36:33.995 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt3x2n.ko 2024-05-26T22:36:33.995 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt3x3.ko 2024-05-26T22:36:33.995 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_it8213.ko 2024-05-26T22:36:33.995 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_it821x.ko 2024-05-26T22:36:33.995 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_jmicron.ko 2024-05-26T22:36:33.995 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_marvell.ko 2024-05-26T22:36:33.995 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_mpiix.ko 2024-05-26T22:36:33.995 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_netcell.ko 2024-05-26T22:36:33.995 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ninja32.ko 2024-05-26T22:36:33.995 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ns87410.ko 2024-05-26T22:36:33.996 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ns87415.ko 2024-05-26T22:36:33.996 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_oldpiix.ko 2024-05-26T22:36:33.996 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_opti.ko 2024-05-26T22:36:33.996 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_optidma.ko 2024-05-26T22:36:33.996 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_pcmcia.ko 2024-05-26T22:36:33.996 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_pdc2027x.ko 2024-05-26T22:36:33.996 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-05-26T22:36:33.996 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_piccolo.ko 2024-05-26T22:36:33.996 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_rdc.ko 2024-05-26T22:36:33.996 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_sch.ko 2024-05-26T22:36:33.996 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_serverworks.ko 2024-05-26T22:36:33.997 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_sil680.ko 2024-05-26T22:36:33.997 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_sis.ko 2024-05-26T22:36:33.997 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_sl82c105.ko 2024-05-26T22:36:33.997 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_triflex.ko 2024-05-26T22:36:33.997 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_via.ko 2024-05-26T22:36:33.997 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pdc_adma.ko 2024-05-26T22:36:33.997 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_inic162x.ko 2024-05-26T22:36:33.997 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_mv.ko 2024-05-26T22:36:33.997 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_nv.ko 2024-05-26T22:36:33.997 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_promise.ko 2024-05-26T22:36:33.997 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_qstor.ko 2024-05-26T22:36:33.997 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sil.ko 2024-05-26T22:36:33.997 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sil24.ko 2024-05-26T22:36:33.997 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sis.ko 2024-05-26T22:36:33.997 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_svw.ko 2024-05-26T22:36:33.998 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sx4.ko 2024-05-26T22:36:33.998 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_uli.ko 2024-05-26T22:36:33.998 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_via.ko 2024-05-26T22:36:33.998 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_vsc.ko 2024-05-26T22:36:33.998 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/auxdisplay 2024-05-26T22:36:33.998 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/auxdisplay/ks0108.ko 2024-05-26T22:36:33.998 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/base 2024-05-26T22:36:33.998 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/base/regmap 2024-05-26T22:36:33.998 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/base/regmap/regmap-i2c.ko 2024-05-26T22:36:33.998 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/base/regmap/regmap-mmio.ko 2024-05-26T22:36:33.998 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/bcma 2024-05-26T22:36:33.998 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/bcma/bcma.ko 2024-05-26T22:36:33.998 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block 2024-05-26T22:36:33.999 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/aoe 2024-05-26T22:36:33.999 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/aoe/aoe.ko 2024-05-26T22:36:33.999 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/brd.ko 2024-05-26T22:36:33.999 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/drbd 2024-05-26T22:36:33.999 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/drbd/drbd.ko 2024-05-26T22:36:33.999 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/floppy.ko 2024-05-26T22:36:33.999 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/loop.ko 2024-05-26T22:36:33.999 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/mtip32xx 2024-05-26T22:36:33.999 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-05-26T22:36:33.999 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/nbd.ko 2024-05-26T22:36:33.999 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/null_blk 2024-05-26T22:36:33.999 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/null_blk/null_blk.ko 2024-05-26T22:36:33.999 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/pktcdvd.ko 2024-05-26T22:36:34.000 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/rbd.ko 2024-05-26T22:36:34.000 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/virtio_blk.ko 2024-05-26T22:36:34.000 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/xen-blkback 2024-05-26T22:36:34.000 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-05-26T22:36:34.000 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/xen-blkfront.ko 2024-05-26T22:36:34.000 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/zram 2024-05-26T22:36:34.000 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/zram/zram.ko 2024-05-26T22:36:34.000 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char 2024-05-26T22:36:34.000 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hangcheck-timer.ko 2024-05-26T22:36:34.000 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random 2024-05-26T22:36:34.000 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/amd-rng.ko 2024-05-26T22:36:34.000 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/intel-rng.ko 2024-05-26T22:36:34.000 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-05-26T22:36:34.000 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/via-rng.ko 2024-05-26T22:36:34.001 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/virtio-rng.ko 2024-05-26T22:36:34.001 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi 2024-05-26T22:36:34.001 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-05-26T22:36:34.001 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-05-26T22:36:34.001 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-05-26T22:36:34.001 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_si.ko 2024-05-26T22:36:34.001 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-05-26T22:36:34.001 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-05-26T22:36:34.001 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/lp.ko 2024-05-26T22:36:34.001 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/mwave 2024-05-26T22:36:34.001 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/mwave/mwave.ko 2024-05-26T22:36:34.001 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ppdev.ko 2024-05-26T22:36:34.001 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tlclk.ko 2024-05-26T22:36:34.002 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm 2024-05-26T22:36:34.002 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm.ko 2024-05-26T22:36:34.002 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_atmel.ko 2024-05-26T22:36:34.002 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_infineon.ko 2024-05-26T22:36:34.002 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_nsc.ko 2024-05-26T22:36:34.002 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_tis.ko 2024-05-26T22:36:34.002 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-05-26T22:36:34.002 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/uv_mmtimer.ko 2024-05-26T22:36:34.002 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/virtio_console.ko 2024-05-26T22:36:34.002 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq 2024-05-26T22:36:34.002 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-05-26T22:36:34.002 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-05-26T22:36:34.003 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/p4-clockmod.ko 2024-05-26T22:36:34.003 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-05-26T22:36:34.003 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/powernow-k8.ko 2024-05-26T22:36:34.003 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/speedstep-lib.ko 2024-05-26T22:36:34.003 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto 2024-05-26T22:36:34.003 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/ccp 2024-05-26T22:36:34.003 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-05-26T22:36:34.003 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/ccp/ccp.ko 2024-05-26T22:36:34.003 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel 2024-05-26T22:36:34.003 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat 2024-05-26T22:36:34.003 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_common 2024-05-26T22:36:34.003 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-05-26T22:36:34.003 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-05-26T22:36:34.004 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-05-26T22:36:34.004 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/padlock-aes.ko 2024-05-26T22:36:34.004 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/padlock-sha.ko 2024-05-26T22:36:34.004 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/virtio 2024-05-26T22:36:34.004 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-05-26T22:36:34.004 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dca 2024-05-26T22:36:34.004 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dca/dca.ko 2024-05-26T22:36:34.004 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/devfreq 2024-05-26T22:36:34.004 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-05-26T22:36:34.004 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma 2024-05-26T22:36:34.005 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/dw 2024-05-26T22:36:34.005 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/dw/dw_dmac.ko 2024-05-26T22:36:34.005 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/ioat 2024-05-26T22:36:34.005 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/ioat/ioatdma.ko 2024-05-26T22:36:34.005 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac 2024-05-26T22:36:34.005 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/amd64_edac.ko 2024-05-26T22:36:34.005 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/e752x_edac.ko 2024-05-26T22:36:34.005 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/edac_mce_amd.ko 2024-05-26T22:36:34.005 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i3000_edac.ko 2024-05-26T22:36:34.005 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i3200_edac.ko 2024-05-26T22:36:34.005 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i5100_edac.ko 2024-05-26T22:36:34.005 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i5400_edac.ko 2024-05-26T22:36:34.006 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i7300_edac.ko 2024-05-26T22:36:34.006 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i7core_edac.ko 2024-05-26T22:36:34.006 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i82975x_edac.ko 2024-05-26T22:36:34.006 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/ie31200_edac.ko 2024-05-26T22:36:34.006 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/sb_edac.ko 2024-05-26T22:36:34.006 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/x38_edac.ko 2024-05-26T22:36:34.006 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/firmware 2024-05-26T22:36:34.006 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/firmware/edd.ko 2024-05-26T22:36:34.006 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/firmware/iscsi_ibft.ko 2024-05-26T22:36:34.006 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio 2024-05-26T22:36:34.006 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio/gpio-ich.ko 2024-05-26T22:36:34.007 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio/gpio-viperboard.ko 2024-05-26T22:36:34.007 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid 2024-05-26T22:36:34.007 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-appleir.ko 2024-05-26T22:36:34.007 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-aureal.ko 2024-05-26T22:36:34.007 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-axff.ko 2024-05-26T22:36:34.007 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-dr.ko 2024-05-26T22:36:34.007 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-elecom.ko 2024-05-26T22:36:34.007 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-elo.ko 2024-05-26T22:36:34.007 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-emsff.ko 2024-05-26T22:36:34.007 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-gaff.ko 2024-05-26T22:36:34.007 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-gt683r.ko 2024-05-26T22:36:34.008 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-gyration.ko 2024-05-26T22:36:34.008 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-holtek-kbd.ko 2024-05-26T22:36:34.008 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-holtek-mouse.ko 2024-05-26T22:36:34.008 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-holtekff.ko 2024-05-26T22:36:34.008 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-hyperv.ko 2024-05-26T22:36:34.008 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-icade.ko 2024-05-26T22:36:34.008 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-keytouch.ko 2024-05-26T22:36:34.008 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-kye.ko 2024-05-26T22:36:34.008 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-lcpower.ko 2024-05-26T22:36:34.008 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-led.ko 2024-05-26T22:36:34.008 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-lenovo.ko 2024-05-26T22:36:34.008 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-logitech-dj.ko 2024-05-26T22:36:34.009 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-05-26T22:36:34.009 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-multitouch.ko 2024-05-26T22:36:34.009 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-ortek.ko 2024-05-26T22:36:34.009 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-penmount.ko 2024-05-26T22:36:34.009 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-petalynx.ko 2024-05-26T22:36:34.009 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-picolcd.ko 2024-05-26T22:36:34.009 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-pl.ko 2024-05-26T22:36:34.009 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-plantronics.ko 2024-05-26T22:36:34.009 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-primax.ko 2024-05-26T22:36:34.009 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-rmi.ko 2024-05-26T22:36:34.010 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-arvo.ko 2024-05-26T22:36:34.010 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-common.ko 2024-05-26T22:36:34.010 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-isku.ko 2024-05-26T22:36:34.010 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-kone.ko 2024-05-26T22:36:34.010 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-05-26T22:36:34.010 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-konepure.ko 2024-05-26T22:36:34.010 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-05-26T22:36:34.010 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-lua.ko 2024-05-26T22:36:34.010 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-pyra.ko 2024-05-26T22:36:34.010 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-ryos.ko 2024-05-26T22:36:34.011 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-savu.ko 2024-05-26T22:36:34.011 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat.ko 2024-05-26T22:36:34.011 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-saitek.ko 2024-05-26T22:36:34.011 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-samsung.ko 2024-05-26T22:36:34.011 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sensor-hub.ko 2024-05-26T22:36:34.011 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sjoy.ko 2024-05-26T22:36:34.011 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sony.ko 2024-05-26T22:36:34.011 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-speedlink.ko 2024-05-26T22:36:34.011 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-steelseries.ko 2024-05-26T22:36:34.011 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sunplus.ko 2024-05-26T22:36:34.011 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-thrustmaster.ko 2024-05-26T22:36:34.011 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-tivo.ko 2024-05-26T22:36:34.012 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-tmff.ko 2024-05-26T22:36:34.012 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-topseed.ko 2024-05-26T22:36:34.012 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-twinhan.ko 2024-05-26T22:36:34.012 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-uclogic.ko 2024-05-26T22:36:34.012 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-waltop.ko 2024-05-26T22:36:34.012 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-wiimote.ko 2024-05-26T22:36:34.012 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-xinmo.ko 2024-05-26T22:36:34.012 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-zpff.ko 2024-05-26T22:36:34.012 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-zydacron.ko 2024-05-26T22:36:34.012 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/uhid.ko 2024-05-26T22:36:34.012 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/wacom.ko 2024-05-26T22:36:34.012 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv 2024-05-26T22:36:34.013 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv/hv_balloon.ko 2024-05-26T22:36:34.013 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv/hv_utils.ko 2024-05-26T22:36:34.013 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv/hv_vmbus.ko 2024-05-26T22:36:34.013 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon 2024-05-26T22:36:34.013 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/abituguru.ko 2024-05-26T22:36:34.013 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/abituguru3.ko 2024-05-26T22:36:34.013 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/acpi_power_meter.ko 2024-05-26T22:36:34.013 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ad7414.ko 2024-05-26T22:36:34.013 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ad7418.ko 2024-05-26T22:36:34.013 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adc128d818.ko 2024-05-26T22:36:34.013 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1025.ko 2024-05-26T22:36:34.013 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1026.ko 2024-05-26T22:36:34.013 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1029.ko 2024-05-26T22:36:34.013 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1031.ko 2024-05-26T22:36:34.014 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm9240.ko 2024-05-26T22:36:34.014 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ads7828.ko 2024-05-26T22:36:34.014 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7410.ko 2024-05-26T22:36:34.014 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7411.ko 2024-05-26T22:36:34.014 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7462.ko 2024-05-26T22:36:34.014 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7470.ko 2024-05-26T22:36:34.014 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7475.ko 2024-05-26T22:36:34.014 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7x10.ko 2024-05-26T22:36:34.014 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/amc6821.ko 2024-05-26T22:36:34.014 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/applesmc.ko 2024-05-26T22:36:34.014 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/asb100.ko 2024-05-26T22:36:34.014 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/asc7621.ko 2024-05-26T22:36:34.014 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/asus_atk0110.ko 2024-05-26T22:36:34.015 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/atxp1.ko 2024-05-26T22:36:34.015 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/coretemp.ko 2024-05-26T22:36:34.015 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-05-26T22:36:34.015 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/dme1737.ko 2024-05-26T22:36:34.015 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ds1621.ko 2024-05-26T22:36:34.015 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ds620.ko 2024-05-26T22:36:34.015 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/emc1403.ko 2024-05-26T22:36:34.015 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/emc6w201.ko 2024-05-26T22:36:34.015 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/f71805f.ko 2024-05-26T22:36:34.015 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/f71882fg.ko 2024-05-26T22:36:34.015 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/f75375s.ko 2024-05-26T22:36:34.015 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/fam15h_power.ko 2024-05-26T22:36:34.016 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/fschmd.ko 2024-05-26T22:36:34.016 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/g760a.ko 2024-05-26T22:36:34.016 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/g762.ko 2024-05-26T22:36:34.016 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/gl518sm.ko 2024-05-26T22:36:34.016 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/gl520sm.ko 2024-05-26T22:36:34.016 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/hwmon-vid.ko 2024-05-26T22:36:34.016 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/i5500_temp.ko 2024-05-26T22:36:34.016 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/i5k_amb.ko 2024-05-26T22:36:34.016 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ibmaem.ko 2024-05-26T22:36:34.016 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ibmpex.ko 2024-05-26T22:36:34.016 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ina209.ko 2024-05-26T22:36:34.016 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ina2xx.ko 2024-05-26T22:36:34.016 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/it87.ko 2024-05-26T22:36:34.016 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/k10temp.ko 2024-05-26T22:36:34.017 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/k8temp.ko 2024-05-26T22:36:34.017 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lineage-pem.ko 2024-05-26T22:36:34.017 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm63.ko 2024-05-26T22:36:34.017 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm73.ko 2024-05-26T22:36:34.017 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm75.ko 2024-05-26T22:36:34.017 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm77.ko 2024-05-26T22:36:34.017 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm78.ko 2024-05-26T22:36:34.017 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm80.ko 2024-05-26T22:36:34.017 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm83.ko 2024-05-26T22:36:34.017 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm85.ko 2024-05-26T22:36:34.017 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm87.ko 2024-05-26T22:36:34.018 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm90.ko 2024-05-26T22:36:34.018 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm92.ko 2024-05-26T22:36:34.018 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm93.ko 2024-05-26T22:36:34.018 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm95234.ko 2024-05-26T22:36:34.018 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm95241.ko 2024-05-26T22:36:34.018 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm95245.ko 2024-05-26T22:36:34.018 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc2945.ko 2024-05-26T22:36:34.018 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4151.ko 2024-05-26T22:36:34.018 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4215.ko 2024-05-26T22:36:34.018 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4222.ko 2024-05-26T22:36:34.018 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4245.ko 2024-05-26T22:36:34.018 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4260.ko 2024-05-26T22:36:34.019 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4261.ko 2024-05-26T22:36:34.019 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max16065.ko 2024-05-26T22:36:34.019 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max1619.ko 2024-05-26T22:36:34.019 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max1668.ko 2024-05-26T22:36:34.019 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max197.ko 2024-05-26T22:36:34.019 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max6639.ko 2024-05-26T22:36:34.019 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max6650.ko 2024-05-26T22:36:34.019 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max6697.ko 2024-05-26T22:36:34.019 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/mcp3021.ko 2024-05-26T22:36:34.019 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct6683.ko 2024-05-26T22:36:34.020 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct6775-core.ko 2024-05-26T22:36:34.020 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct6775.ko 2024-05-26T22:36:34.020 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct7802.ko 2024-05-26T22:36:34.020 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pc87360.ko 2024-05-26T22:36:34.020 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pc87427.ko 2024-05-26T22:36:34.020 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pcf8591.ko 2024-05-26T22:36:34.020 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus 2024-05-26T22:36:34.020 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-05-26T22:36:34.020 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-05-26T22:36:34.020 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-05-26T22:36:34.020 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/max16064.ko 2024-05-26T22:36:34.021 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/max34440.ko 2024-05-26T22:36:34.021 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/max8688.ko 2024-05-26T22:36:34.021 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-05-26T22:36:34.021 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-05-26T22:36:34.021 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-05-26T22:36:34.021 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-05-26T22:36:34.021 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-05-26T22:36:34.021 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-05-26T22:36:34.021 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/powr1220.ko 2024-05-26T22:36:34.021 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sch5627.ko 2024-05-26T22:36:34.021 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sch5636.ko 2024-05-26T22:36:34.022 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sch56xx-common.ko 2024-05-26T22:36:34.022 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sht15.ko 2024-05-26T22:36:34.022 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sht21.ko 2024-05-26T22:36:34.022 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/shtc1.ko 2024-05-26T22:36:34.022 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sis5595.ko 2024-05-26T22:36:34.022 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/smsc47b397.ko 2024-05-26T22:36:34.022 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/smsc47m1.ko 2024-05-26T22:36:34.022 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/smsc47m192.ko 2024-05-26T22:36:34.022 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/thmc50.ko 2024-05-26T22:36:34.022 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp102.ko 2024-05-26T22:36:34.022 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp103.ko 2024-05-26T22:36:34.022 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp401.ko 2024-05-26T22:36:34.022 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp421.ko 2024-05-26T22:36:34.023 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/via-cputemp.ko 2024-05-26T22:36:34.023 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/via686a.ko 2024-05-26T22:36:34.023 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/vt1211.ko 2024-05-26T22:36:34.023 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/vt8231.ko 2024-05-26T22:36:34.023 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83627ehf.ko 2024-05-26T22:36:34.023 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83627hf.ko 2024-05-26T22:36:34.023 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83781d.ko 2024-05-26T22:36:34.023 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83791d.ko 2024-05-26T22:36:34.023 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83792d.ko 2024-05-26T22:36:34.023 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83793.ko 2024-05-26T22:36:34.023 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83795.ko 2024-05-26T22:36:34.023 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83l785ts.ko 2024-05-26T22:36:34.023 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83l786ng.ko 2024-05-26T22:36:34.023 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c 2024-05-26T22:36:34.024 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/algos 2024-05-26T22:36:34.024 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-05-26T22:36:34.024 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-05-26T22:36:34.024 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses 2024-05-26T22:36:34.024 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-05-26T22:36:34.024 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-05-26T22:36:34.024 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-05-26T22:36:34.024 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-05-26T22:36:34.024 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-05-26T22:36:34.024 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-05-26T22:36:34.024 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-05-26T22:36:34.024 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-05-26T22:36:34.025 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-i801.ko 2024-05-26T22:36:34.025 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-isch.ko 2024-05-26T22:36:34.025 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-05-26T22:36:34.025 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-05-26T22:36:34.025 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-05-26T22:36:34.025 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-parport.ko 2024-05-26T22:36:34.025 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-05-26T22:36:34.025 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-05-26T22:36:34.025 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-05-26T22:36:34.025 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-05-26T22:36:34.026 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-05-26T22:36:34.026 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-05-26T22:36:34.026 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-via.ko 2024-05-26T22:36:34.026 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-05-26T22:36:34.026 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-05-26T22:36:34.026 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-dev.ko 2024-05-26T22:36:34.026 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-mux.ko 2024-05-26T22:36:34.026 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-05-26T22:36:34.027 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-smbus.ko 2024-05-26T22:36:34.027 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-stub.ko 2024-05-26T22:36:34.027 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband 2024-05-26T22:36:34.027 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core 2024-05-26T22:36:34.027 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_cm.ko 2024-05-26T22:36:34.027 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_core.ko 2024-05-26T22:36:34.027 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_umad.ko 2024-05-26T22:36:34.027 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-05-26T22:36:34.027 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/iw_cm.ko 2024-05-26T22:36:34.027 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/rdma_cm.ko 2024-05-26T22:36:34.027 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-05-26T22:36:34.027 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw 2024-05-26T22:36:34.027 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/cxgb4 2024-05-26T22:36:34.028 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-05-26T22:36:34.028 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx4 2024-05-26T22:36:34.028 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-05-26T22:36:34.028 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx5 2024-05-26T22:36:34.028 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-05-26T22:36:34.028 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mthca 2024-05-26T22:36:34.028 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-05-26T22:36:34.028 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp 2024-05-26T22:36:34.028 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/ipoib 2024-05-26T22:36:34.028 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-05-26T22:36:34.028 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/iser 2024-05-26T22:36:34.028 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-05-26T22:36:34.029 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/isert 2024-05-26T22:36:34.029 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-05-26T22:36:34.029 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srp 2024-05-26T22:36:34.029 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-05-26T22:36:34.029 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srpt 2024-05-26T22:36:34.029 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-05-26T22:36:34.030 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input 2024-05-26T22:36:34.030 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport 2024-05-26T22:36:34.030 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-05-26T22:36:34.030 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport/fm801-gp.ko 2024-05-26T22:36:34.030 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport/gameport.ko 2024-05-26T22:36:34.030 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/joydev.ko 2024-05-26T22:36:34.031 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/keyboard 2024-05-26T22:36:34.031 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/keyboard/gpio_keys.ko 2024-05-26T22:36:34.031 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse 2024-05-26T22:36:34.031 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/appletouch.ko 2024-05-26T22:36:34.031 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/bcm5974.ko 2024-05-26T22:36:34.031 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/cyapatp.ko 2024-05-26T22:36:34.032 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/elan_i2c.ko 2024-05-26T22:36:34.032 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/sermouse.ko 2024-05-26T22:36:34.032 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-05-26T22:36:34.032 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/synaptics_usb.ko 2024-05-26T22:36:34.032 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/vsxxxaa.ko 2024-05-26T22:36:34.032 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/rmi4 2024-05-26T22:36:34.033 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/rmi4/rmi_core.ko 2024-05-26T22:36:34.033 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio 2024-05-26T22:36:34.033 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/altera_ps2.ko 2024-05-26T22:36:34.033 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/arc_ps2.ko 2024-05-26T22:36:34.033 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-05-26T22:36:34.034 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/serio_raw.ko 2024-05-26T22:36:34.034 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/sparse-keymap.ko 2024-05-26T22:36:34.034 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds 2024-05-26T22:36:34.034 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-blinkm.ko 2024-05-26T22:36:34.034 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-lm3530.ko 2024-05-26T22:36:34.034 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-lp3944.ko 2024-05-26T22:36:34.034 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-ss4200.ko 2024-05-26T22:36:34.034 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger 2024-05-26T22:36:34.034 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-05-26T22:36:34.035 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-05-26T22:36:34.035 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-05-26T22:36:34.035 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-05-26T22:36:34.035 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-05-26T22:36:34.035 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-05-26T22:36:34.035 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-05-26T22:36:34.035 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-05-26T22:36:34.036 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-05-26T22:36:34.036 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md 2024-05-26T22:36:34.036 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/bcache 2024-05-26T22:36:34.036 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/bcache/bcache.ko 2024-05-26T22:36:34.036 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-bio-prison.ko 2024-05-26T22:36:34.036 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-cache-smq.ko 2024-05-26T22:36:34.036 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-cache.ko 2024-05-26T22:36:34.036 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-crypt.ko 2024-05-26T22:36:34.036 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-delay.ko 2024-05-26T22:36:34.036 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-flakey.ko 2024-05-26T22:36:34.036 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-log-userspace.ko 2024-05-26T22:36:34.036 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-multipath.ko 2024-05-26T22:36:34.036 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-queue-length.ko 2024-05-26T22:36:34.036 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-raid.ko 2024-05-26T22:36:34.036 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-round-robin.ko 2024-05-26T22:36:34.037 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-service-time.ko 2024-05-26T22:36:34.037 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-switch.ko 2024-05-26T22:36:34.037 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-thin-pool.ko 2024-05-26T22:36:34.037 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-verity.ko 2024-05-26T22:36:34.037 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/persistent-data 2024-05-26T22:36:34.037 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-05-26T22:36:34.037 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid0.ko 2024-05-26T22:36:34.037 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid1.ko 2024-05-26T22:36:34.037 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid10.ko 2024-05-26T22:36:34.037 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid456.ko 2024-05-26T22:36:34.037 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message 2024-05-26T22:36:34.037 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion 2024-05-26T22:36:34.037 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptbase.ko 2024-05-26T22:36:34.037 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptctl.ko 2024-05-26T22:36:34.038 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptfc.ko 2024-05-26T22:36:34.038 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptlan.ko 2024-05-26T22:36:34.038 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptsas.ko 2024-05-26T22:36:34.038 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptscsih.ko 2024-05-26T22:36:34.038 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptspi.ko 2024-05-26T22:36:34.038 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd 2024-05-26T22:36:34.038 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/lpc_ich.ko 2024-05-26T22:36:34.038 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/lpc_sch.ko 2024-05-26T22:36:34.038 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/mfd-core.ko 2024-05-26T22:36:34.038 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/sm501.ko 2024-05-26T22:36:34.038 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/viperboard.ko 2024-05-26T22:36:34.038 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/vx855.ko 2024-05-26T22:36:34.038 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/wl1273-core.ko 2024-05-26T22:36:34.039 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc 2024-05-26T22:36:34.039 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/altera-stapl 2024-05-26T22:36:34.039 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-05-26T22:36:34.039 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/apds9802als.ko 2024-05-26T22:36:34.039 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/apds990x.ko 2024-05-26T22:36:34.039 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/bh1770glc.ko 2024-05-26T22:36:34.039 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/cb710 2024-05-26T22:36:34.039 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/cb710/cb710.ko 2024-05-26T22:36:34.039 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom 2024-05-26T22:36:34.039 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom/at24.ko 2024-05-26T22:36:34.039 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-05-26T22:36:34.039 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom/max6875.ko 2024-05-26T22:36:34.039 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/enclosure.ko 2024-05-26T22:36:34.040 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/hpilo.ko 2024-05-26T22:36:34.040 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/isl29003.ko 2024-05-26T22:36:34.040 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/isl29020.ko 2024-05-26T22:36:34.040 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/lis3lv02d 2024-05-26T22:36:34.040 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-05-26T22:36:34.040 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-05-26T22:36:34.040 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei 2024-05-26T22:36:34.040 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei/mei-me.ko 2024-05-26T22:36:34.040 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei/mei-txe.ko 2024-05-26T22:36:34.040 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei/mei.ko 2024-05-26T22:36:34.040 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-gru 2024-05-26T22:36:34.040 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-gru/gru.ko 2024-05-26T22:36:34.040 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-xp 2024-05-26T22:36:34.040 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-xp/xp.ko 2024-05-26T22:36:34.041 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-xp/xpc.ko 2024-05-26T22:36:34.041 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-05-26T22:36:34.041 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/tifm_7xx1.ko 2024-05-26T22:36:34.041 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/tifm_core.ko 2024-05-26T22:36:34.041 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/tsl2550.ko 2024-05-26T22:36:34.041 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/vmw_balloon.ko 2024-05-26T22:36:34.041 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/vmw_vmci 2024-05-26T22:36:34.041 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-05-26T22:36:34.041 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net 2024-05-26T22:36:34.041 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/bonding 2024-05-26T22:36:34.041 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/bonding/bonding.ko 2024-05-26T22:36:34.041 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa 2024-05-26T22:36:34.042 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/b53 2024-05-26T22:36:34.042 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/b53/b53_common.ko 2024-05-26T22:36:34.042 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/bcm-sf2.ko 2024-05-26T22:36:34.042 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/mv88e6060.ko 2024-05-26T22:36:34.042 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/mv88e6xxx 2024-05-26T22:36:34.043 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-05-26T22:36:34.043 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dummy.ko 2024-05-26T22:36:34.043 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/eql.ko 2024-05-26T22:36:34.043 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet 2024-05-26T22:36:34.043 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com 2024-05-26T22:36:34.043 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-05-26T22:36:34.044 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-05-26T22:36:34.044 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-05-26T22:36:34.044 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-05-26T22:36:34.044 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390 2024-05-26T22:36:34.044 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/8390.ko 2024-05-26T22:36:34.044 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-05-26T22:36:34.044 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-05-26T22:36:34.044 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-05-26T22:36:34.044 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/adaptec 2024-05-26T22:36:34.045 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-05-26T22:36:34.045 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/agere 2024-05-26T22:36:34.045 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/agere/et131x.ko 2024-05-26T22:36:34.045 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/alteon 2024-05-26T22:36:34.045 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-05-26T22:36:34.045 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/altera 2024-05-26T22:36:34.045 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-05-26T22:36:34.045 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd 2024-05-26T22:36:34.045 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-05-26T22:36:34.045 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-05-26T22:36:34.045 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-05-26T22:36:34.045 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros 2024-05-26T22:36:34.045 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/alx 2024-05-26T22:36:34.045 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-05-26T22:36:34.046 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1c 2024-05-26T22:36:34.046 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-05-26T22:36:34.046 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1e 2024-05-26T22:36:34.046 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-05-26T22:36:34.046 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atlx 2024-05-26T22:36:34.046 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-05-26T22:36:34.046 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-05-26T22:36:34.046 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom 2024-05-26T22:36:34.046 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-05-26T22:36:34.046 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-05-26T22:36:34.046 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-05-26T22:36:34.047 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-05-26T22:36:34.047 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-05-26T22:36:34.047 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/genet 2024-05-26T22:36:34.047 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-05-26T22:36:34.047 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-05-26T22:36:34.047 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/brocade 2024-05-26T22:36:34.047 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/brocade/bna 2024-05-26T22:36:34.047 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-05-26T22:36:34.047 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio 2024-05-26T22:36:34.047 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb 2024-05-26T22:36:34.047 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-05-26T22:36:34.047 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-05-26T22:36:34.047 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-05-26T22:36:34.047 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-05-26T22:36:34.048 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-05-26T22:36:34.048 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-05-26T22:36:34.048 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-05-26T22:36:34.048 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-05-26T22:36:34.048 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-05-26T22:36:34.048 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/cisco 2024-05-26T22:36:34.048 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/cisco/enic 2024-05-26T22:36:34.048 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-05-26T22:36:34.048 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec 2024-05-26T22:36:34.048 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip 2024-05-26T22:36:34.048 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-05-26T22:36:34.048 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-05-26T22:36:34.048 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-05-26T22:36:34.048 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-05-26T22:36:34.048 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-05-26T22:36:34.049 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-05-26T22:36:34.049 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dlink 2024-05-26T22:36:34.049 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-05-26T22:36:34.049 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-05-26T22:36:34.049 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dnet.ko 2024-05-26T22:36:34.049 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/emulex 2024-05-26T22:36:34.049 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/emulex/benet 2024-05-26T22:36:34.049 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-05-26T22:36:34.049 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/ethoc.ko 2024-05-26T22:36:34.049 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/fealnx.ko 2024-05-26T22:36:34.049 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel 2024-05-26T22:36:34.049 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e100.ko 2024-05-26T22:36:34.049 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000 2024-05-26T22:36:34.050 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-05-26T22:36:34.050 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000e 2024-05-26T22:36:34.050 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-05-26T22:36:34.050 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/fm10k 2024-05-26T22:36:34.050 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-05-26T22:36:34.050 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/i40e 2024-05-26T22:36:34.050 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-05-26T22:36:34.050 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/iavf 2024-05-26T22:36:34.050 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-05-26T22:36:34.051 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igb 2024-05-26T22:36:34.051 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-05-26T22:36:34.051 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igbvf 2024-05-26T22:36:34.051 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-05-26T22:36:34.051 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbe 2024-05-26T22:36:34.051 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-05-26T22:36:34.051 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbevf 2024-05-26T22:36:34.051 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-05-26T22:36:34.051 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/jme.ko 2024-05-26T22:36:34.051 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell 2024-05-26T22:36:34.051 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-05-26T22:36:34.051 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell/skge.ko 2024-05-26T22:36:34.052 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-05-26T22:36:34.052 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox 2024-05-26T22:36:34.052 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx4 2024-05-26T22:36:34.052 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-05-26T22:36:34.052 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-05-26T22:36:34.052 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx5 2024-05-26T22:36:34.052 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-05-26T22:36:34.052 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-05-26T22:36:34.052 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/micrel 2024-05-26T22:36:34.052 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-05-26T22:36:34.052 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/myricom 2024-05-26T22:36:34.052 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/myricom/myri10ge 2024-05-26T22:36:34.053 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-05-26T22:36:34.053 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/natsemi 2024-05-26T22:36:34.053 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-05-26T22:36:34.053 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-05-26T22:36:34.053 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/neterion 2024-05-26T22:36:34.053 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-05-26T22:36:34.053 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/nvidia 2024-05-26T22:36:34.053 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-05-26T22:36:34.053 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/packetengines 2024-05-26T22:36:34.053 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-05-26T22:36:34.053 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-05-26T22:36:34.053 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic 2024-05-26T22:36:34.054 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/netxen 2024-05-26T22:36:34.054 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-05-26T22:36:34.054 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-05-26T22:36:34.054 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-05-26T22:36:34.054 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-05-26T22:36:34.054 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/rdc 2024-05-26T22:36:34.054 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-05-26T22:36:34.054 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek 2024-05-26T22:36:34.054 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-05-26T22:36:34.054 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-05-26T22:36:34.054 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/atp.ko 2024-05-26T22:36:34.057 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-05-26T22:36:34.058 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/rocker 2024-05-26T22:36:34.058 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-05-26T22:36:34.058 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sfc 2024-05-26T22:36:34.058 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-05-26T22:36:34.058 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/silan 2024-05-26T22:36:34.058 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-05-26T22:36:34.058 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sis 2024-05-26T22:36:34.059 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sis/sis190.ko 2024-05-26T22:36:34.059 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sis/sis900.ko 2024-05-26T22:36:34.059 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc 2024-05-26T22:36:34.059 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-05-26T22:36:34.059 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-05-26T22:36:34.059 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-05-26T22:36:34.059 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-05-26T22:36:34.059 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro 2024-05-26T22:36:34.059 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro/stmmac 2024-05-26T22:36:34.059 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-05-26T22:36:34.059 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-05-26T22:36:34.059 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun 2024-05-26T22:36:34.059 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/cassini.ko 2024-05-26T22:36:34.060 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/niu.ko 2024-05-26T22:36:34.060 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/sungem.ko 2024-05-26T22:36:34.060 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-05-26T22:36:34.060 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/tehuti 2024-05-26T22:36:34.060 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-05-26T22:36:34.060 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/ti 2024-05-26T22:36:34.060 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/ti/tlan.ko 2024-05-26T22:36:34.060 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/via 2024-05-26T22:36:34.060 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-05-26T22:36:34.060 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-05-26T22:36:34.060 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/wiznet 2024-05-26T22:36:34.060 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-05-26T22:36:34.061 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-05-26T22:36:34.061 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/xircom 2024-05-26T22:36:34.061 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-05-26T22:36:34.061 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/geneve.ko 2024-05-26T22:36:34.061 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/hyperv 2024-05-26T22:36:34.061 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-05-26T22:36:34.061 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ieee802154 2024-05-26T22:36:34.061 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ieee802154/fakelb.ko 2024-05-26T22:36:34.061 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ifb.ko 2024-05-26T22:36:34.061 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ipvlan 2024-05-26T22:36:34.061 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ipvlan/ipvlan.ko 2024-05-26T22:36:34.061 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/macvlan.ko 2024-05-26T22:36:34.061 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/macvtap.ko 2024-05-26T22:36:34.062 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio 2024-05-26T22:36:34.062 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio.ko 2024-05-26T22:36:34.062 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-05-26T22:36:34.062 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-05-26T22:36:34.062 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio/mdio-regmap.ko 2024-05-26T22:36:34.062 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mii.ko 2024-05-26T22:36:34.062 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/net_failover.ko 2024-05-26T22:36:34.062 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/netconsole.ko 2024-05-26T22:36:34.062 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/nlmon.ko 2024-05-26T22:36:34.062 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/pcs 2024-05-26T22:36:34.062 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/pcs/pcs-lynx.ko 2024-05-26T22:36:34.062 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-05-26T22:36:34.063 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy 2024-05-26T22:36:34.063 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/amd.ko 2024-05-26T22:36:34.063 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-05-26T22:36:34.063 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-05-26T22:36:34.063 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/bcm7xxx.ko 2024-05-26T22:36:34.063 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/bcm87xx.ko 2024-05-26T22:36:34.063 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/broadcom.ko 2024-05-26T22:36:34.063 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/cicada.ko 2024-05-26T22:36:34.063 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/davicom.ko 2024-05-26T22:36:34.063 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/dp83640.ko 2024-05-26T22:36:34.063 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/et1011c.ko 2024-05-26T22:36:34.063 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/icplus.ko 2024-05-26T22:36:34.063 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/lxt.ko 2024-05-26T22:36:34.064 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/marvell.ko 2024-05-26T22:36:34.064 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/micrel.ko 2024-05-26T22:36:34.064 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/national.ko 2024-05-26T22:36:34.064 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/phylink.ko 2024-05-26T22:36:34.064 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/qsemi.ko 2024-05-26T22:36:34.064 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/realtek.ko 2024-05-26T22:36:34.064 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/smsc.ko 2024-05-26T22:36:34.064 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/ste10Xp.ko 2024-05-26T22:36:34.064 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/vitesse.ko 2024-05-26T22:36:34.064 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp 2024-05-26T22:36:34.064 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/bsd_comp.ko 2024-05-26T22:36:34.064 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_async.ko 2024-05-26T22:36:34.064 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_deflate.ko 2024-05-26T22:36:34.064 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_generic.ko 2024-05-26T22:36:34.065 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_mppe.ko 2024-05-26T22:36:34.065 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_synctty.ko 2024-05-26T22:36:34.065 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/pppoe.ko 2024-05-26T22:36:34.065 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/pppox.ko 2024-05-26T22:36:34.065 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/pptp.ko 2024-05-26T22:36:34.065 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/slip 2024-05-26T22:36:34.065 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/slip/slhc.ko 2024-05-26T22:36:34.065 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/slip/slip.ko 2024-05-26T22:36:34.065 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/sungem_phy.ko 2024-05-26T22:36:34.065 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/tap.ko 2024-05-26T22:36:34.065 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team 2024-05-26T22:36:34.065 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team.ko 2024-05-26T22:36:34.065 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_activebackup.ko 2024-05-26T22:36:34.066 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_broadcast.ko 2024-05-26T22:36:34.066 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-05-26T22:36:34.066 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_random.ko 2024-05-26T22:36:34.066 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-05-26T22:36:34.066 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/tun.ko 2024-05-26T22:36:34.066 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/veth.ko 2024-05-26T22:36:34.066 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/virtio_net.ko 2024-05-26T22:36:34.066 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vmxnet3 2024-05-26T22:36:34.066 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-05-26T22:36:34.066 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vxlan 2024-05-26T22:36:34.066 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vxlan/vxlan.ko 2024-05-26T22:36:34.066 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/xen-netback 2024-05-26T22:36:34.066 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/xen-netback/xen-netback.ko 2024-05-26T22:36:34.067 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/xen-netfront.ko 2024-05-26T22:36:34.067 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ntb 2024-05-26T22:36:34.067 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ntb/ntb.ko 2024-05-26T22:36:34.067 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme 2024-05-26T22:36:34.067 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme/host 2024-05-26T22:36:34.067 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme/host/nvme-core.ko 2024-05-26T22:36:34.067 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme/host/nvme.ko 2024-05-26T22:36:34.067 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport 2024-05-26T22:36:34.067 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport.ko 2024-05-26T22:36:34.067 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport_cs.ko 2024-05-26T22:36:34.067 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport_pc.ko 2024-05-26T22:36:34.067 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport_serial.ko 2024-05-26T22:36:34.067 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci 2024-05-26T22:36:34.068 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/hotplug 2024-05-26T22:36:34.068 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-05-26T22:36:34.068 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/pcie 2024-05-26T22:36:34.068 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/pcie/aer_inject.ko 2024-05-26T22:36:34.068 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/xen-pcifront.ko 2024-05-26T22:36:34.068 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia 2024-05-26T22:36:34.068 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia/i82092.ko 2024-05-26T22:36:34.068 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia/pd6729.ko 2024-05-26T22:36:34.068 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia/yenta_socket.ko 2024-05-26T22:36:34.068 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pinctrl 2024-05-26T22:36:34.068 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pinctrl/intel 2024-05-26T22:36:34.068 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-05-26T22:36:34.068 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform 2024-05-26T22:36:34.069 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86 2024-05-26T22:36:34.069 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/acer-wmi.ko 2024-05-26T22:36:34.069 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/acerhdf.ko 2024-05-26T22:36:34.069 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/apple-gmux.ko 2024-05-26T22:36:34.069 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/asus-laptop.ko 2024-05-26T22:36:34.069 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-05-26T22:36:34.069 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/asus-wmi.ko 2024-05-26T22:36:34.069 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/classmate-laptop.ko 2024-05-26T22:36:34.069 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-05-26T22:36:34.069 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-05-26T22:36:34.069 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-05-26T22:36:34.069 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-05-26T22:36:34.069 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/hdaps.ko 2024-05-26T22:36:34.070 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/intel 2024-05-26T22:36:34.070 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-05-26T22:36:34.070 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/intel_ips.ko 2024-05-26T22:36:34.070 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/msi-wmi.ko 2024-05-26T22:36:34.070 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/mxm-wmi.ko 2024-05-26T22:36:34.070 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-05-26T22:36:34.070 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/samsung-laptop.ko 2024-05-26T22:36:34.070 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/samsung-q10.ko 2024-05-26T22:36:34.070 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/topstar-laptop.ko 2024-05-26T22:36:34.070 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-05-26T22:36:34.070 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/toshiba_haps.ko 2024-05-26T22:36:34.070 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/wmi-bmof.ko 2024-05-26T22:36:34.071 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/wmi.ko 2024-05-26T22:36:34.071 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/powercap 2024-05-26T22:36:34.071 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/powercap/intel_rapl_common.ko 2024-05-26T22:36:34.071 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/powercap/intel_rapl_msr.ko 2024-05-26T22:36:34.071 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps 2024-05-26T22:36:34.071 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients 2024-05-26T22:36:34.071 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients/pps-gpio.ko 2024-05-26T22:36:34.071 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients/pps-ldisc.ko 2024-05-26T22:36:34.071 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients/pps_parport.ko 2024-05-26T22:36:34.071 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/pps_core.ko 2024-05-26T22:36:34.071 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ptp 2024-05-26T22:36:34.072 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ptp/ptp.ko 2024-05-26T22:36:34.072 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ptp/ptp_kvm.ko 2024-05-26T22:36:34.072 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras 2024-05-26T22:36:34.072 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd 2024-05-26T22:36:34.072 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd/atl 2024-05-26T22:36:34.072 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-05-26T22:36:34.072 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd/fmpm.ko 2024-05-26T22:36:34.072 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc 2024-05-26T22:36:34.072 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-bq32k.ko 2024-05-26T22:36:34.072 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1286.ko 2024-05-26T22:36:34.072 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1307.ko 2024-05-26T22:36:34.072 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1374.ko 2024-05-26T22:36:34.072 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1511.ko 2024-05-26T22:36:34.072 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1553.ko 2024-05-26T22:36:34.073 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1672.ko 2024-05-26T22:36:34.073 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1742.ko 2024-05-26T22:36:34.073 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds2404.ko 2024-05-26T22:36:34.073 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds3232.ko 2024-05-26T22:36:34.073 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-em3027.ko 2024-05-26T22:36:34.073 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-fm3130.ko 2024-05-26T22:36:34.073 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-isl12022.ko 2024-05-26T22:36:34.073 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-isl1208.ko 2024-05-26T22:36:34.073 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-m41t80.ko 2024-05-26T22:36:34.073 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-m48t35.ko 2024-05-26T22:36:34.073 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-m48t59.ko 2024-05-26T22:36:34.073 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-max6900.ko 2024-05-26T22:36:34.074 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-msm6242.ko 2024-05-26T22:36:34.074 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf2127.ko 2024-05-26T22:36:34.074 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf85063.ko 2024-05-26T22:36:34.074 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf8523.ko 2024-05-26T22:36:34.074 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf8563.ko 2024-05-26T22:36:34.074 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf8583.ko 2024-05-26T22:36:34.074 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rp5c01.ko 2024-05-26T22:36:34.074 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rs5c372.ko 2024-05-26T22:36:34.074 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-05-26T22:36:34.074 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rx8025.ko 2024-05-26T22:36:34.074 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rx8581.ko 2024-05-26T22:36:34.074 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-05-26T22:36:34.074 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-x1205.ko 2024-05-26T22:36:34.075 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi 2024-05-26T22:36:34.075 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/3w-9xxx.ko 2024-05-26T22:36:34.075 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/3w-sas.ko 2024-05-26T22:36:34.075 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/3w-xxxx.ko 2024-05-26T22:36:34.075 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/BusLogic.ko 2024-05-26T22:36:34.075 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/a100u2w.ko 2024-05-26T22:36:34.075 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aacraid 2024-05-26T22:36:34.075 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aacraid/aacraid.ko 2024-05-26T22:36:34.075 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/advansys.ko 2024-05-26T22:36:34.075 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aic7xxx 2024-05-26T22:36:34.075 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-05-26T22:36:34.075 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-05-26T22:36:34.075 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/am53c974.ko 2024-05-26T22:36:34.076 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/arcmsr 2024-05-26T22:36:34.076 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-05-26T22:36:34.076 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/atp870u.ko 2024-05-26T22:36:34.076 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/be2iscsi 2024-05-26T22:36:34.076 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-05-26T22:36:34.076 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bfa 2024-05-26T22:36:34.076 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bfa/bfa.ko 2024-05-26T22:36:34.076 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2fc 2024-05-26T22:36:34.076 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-05-26T22:36:34.076 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2i 2024-05-26T22:36:34.076 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-05-26T22:36:34.076 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ch.ko 2024-05-26T22:36:34.076 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/csiostor 2024-05-26T22:36:34.076 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/csiostor/csiostor.ko 2024-05-26T22:36:34.077 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi 2024-05-26T22:36:34.077 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb3i 2024-05-26T22:36:34.077 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-05-26T22:36:34.077 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb4i 2024-05-26T22:36:34.077 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-05-26T22:36:34.077 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-05-26T22:36:34.077 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/dc395x.ko 2024-05-26T22:36:34.077 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler 2024-05-26T22:36:34.077 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-05-26T22:36:34.077 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-05-26T22:36:34.077 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-05-26T22:36:34.077 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-05-26T22:36:34.077 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/dmx3191d.ko 2024-05-26T22:36:34.078 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/esas2r 2024-05-26T22:36:34.078 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/esas2r/esas2r.ko 2024-05-26T22:36:34.078 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/esp_scsi.ko 2024-05-26T22:36:34.078 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fcoe 2024-05-26T22:36:34.078 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fcoe/fcoe.ko 2024-05-26T22:36:34.078 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-05-26T22:36:34.078 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fnic 2024-05-26T22:36:34.078 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fnic/fnic.ko 2024-05-26T22:36:34.078 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/hpsa.ko 2024-05-26T22:36:34.078 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/hptiop.ko 2024-05-26T22:36:34.079 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/hv_storvsc.ko 2024-05-26T22:36:34.079 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/initio.ko 2024-05-26T22:36:34.079 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ipr.ko 2024-05-26T22:36:34.079 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ips.ko 2024-05-26T22:36:34.079 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/isci 2024-05-26T22:36:34.079 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/isci/isci.ko 2024-05-26T22:36:34.079 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-05-26T22:36:34.079 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/iscsi_tcp.ko 2024-05-26T22:36:34.079 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libfc 2024-05-26T22:36:34.079 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libfc/libfc.ko 2024-05-26T22:36:34.079 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libiscsi.ko 2024-05-26T22:36:34.079 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libiscsi_tcp.ko 2024-05-26T22:36:34.079 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libsas 2024-05-26T22:36:34.079 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libsas/libsas.ko 2024-05-26T22:36:34.080 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/lpfc 2024-05-26T22:36:34.080 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/lpfc/lpfc.ko 2024-05-26T22:36:34.080 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid 2024-05-26T22:36:34.080 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid.ko 2024-05-26T22:36:34.080 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-05-26T22:36:34.080 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-05-26T22:36:34.080 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-05-26T22:36:34.080 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mpt3sas 2024-05-26T22:36:34.080 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-05-26T22:36:34.080 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mvsas 2024-05-26T22:36:34.080 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mvsas/mvsas.ko 2024-05-26T22:36:34.080 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mvumi.ko 2024-05-26T22:36:34.080 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/pm8001 2024-05-26T22:36:34.081 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-05-26T22:36:34.081 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/pmcraid.ko 2024-05-26T22:36:34.081 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla1280.ko 2024-05-26T22:36:34.081 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla2xxx 2024-05-26T22:36:34.081 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-05-26T22:36:34.081 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-05-26T22:36:34.081 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla4xxx 2024-05-26T22:36:34.081 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-05-26T22:36:34.081 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/raid_class.ko 2024-05-26T22:36:34.081 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_debug.ko 2024-05-26T22:36:34.081 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_fc.ko 2024-05-26T22:36:34.081 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-05-26T22:36:34.081 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_sas.ko 2024-05-26T22:36:34.082 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_spi.ko 2024-05-26T22:36:34.082 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_srp.ko 2024-05-26T22:36:34.082 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ses.ko 2024-05-26T22:36:34.082 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/st.ko 2024-05-26T22:36:34.082 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/stex.ko 2024-05-26T22:36:34.082 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/sym53c8xx_2 2024-05-26T22:36:34.082 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-05-26T22:36:34.082 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/virtio_scsi.ko 2024-05-26T22:36:34.082 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/vmw_pvscsi.ko 2024-05-26T22:36:34.082 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/wd719x.ko 2024-05-26T22:36:34.082 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ssb 2024-05-26T22:36:34.082 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ssb/ssb.ko 2024-05-26T22:36:34.083 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target 2024-05-26T22:36:34.083 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/iscsi 2024-05-26T22:36:34.083 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-05-26T22:36:34.083 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/loopback 2024-05-26T22:36:34.083 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/loopback/tcm_loop.ko 2024-05-26T22:36:34.083 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_file.ko 2024-05-26T22:36:34.083 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_iblock.ko 2024-05-26T22:36:34.083 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_mod.ko 2024-05-26T22:36:34.083 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_pscsi.ko 2024-05-26T22:36:34.083 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_user.ko 2024-05-26T22:36:34.083 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/tcm_fc 2024-05-26T22:36:34.083 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-05-26T22:36:34.083 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal 2024-05-26T22:36:34.083 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel 2024-05-26T22:36:34.084 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal 2024-05-26T22:36:34.084 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-05-26T22:36:34.084 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-05-26T22:36:34.084 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-05-26T22:36:34.084 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-05-26T22:36:34.084 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-05-26T22:36:34.084 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-05-26T22:36:34.084 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-05-26T22:36:34.084 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-05-26T22:36:34.084 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-05-26T22:36:34.085 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-05-26T22:36:34.085 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-05-26T22:36:34.085 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-05-26T22:36:34.085 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-05-26T22:36:34.085 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-05-26T22:36:34.085 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-05-26T22:36:34.085 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-05-26T22:36:34.085 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-05-26T22:36:34.085 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-05-26T22:36:34.085 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty 2024-05-26T22:36:34.085 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/ipwireless 2024-05-26T22:36:34.085 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-05-26T22:36:34.085 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/n_gsm.ko 2024-05-26T22:36:34.086 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/n_hdlc.ko 2024-05-26T22:36:34.086 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/nozomi.ko 2024-05-26T22:36:34.086 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial 2024-05-26T22:36:34.086 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/8250 2024-05-26T22:36:34.086 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-05-26T22:36:34.086 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/arc_uart.ko 2024-05-26T22:36:34.086 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/jsm 2024-05-26T22:36:34.086 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/jsm/jsm.ko 2024-05-26T22:36:34.086 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs 2024-05-26T22:36:34.086 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/core 2024-05-26T22:36:34.086 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/core/ufshcd-core.ko 2024-05-26T22:36:34.086 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/host 2024-05-26T22:36:34.086 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-05-26T22:36:34.087 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio 2024-05-26T22:36:34.087 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio.ko 2024-05-26T22:36:34.087 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_aec.ko 2024-05-26T22:36:34.087 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_cif.ko 2024-05-26T22:36:34.087 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_pci_generic.ko 2024-05-26T22:36:34.087 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_sercos3.ko 2024-05-26T22:36:34.087 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb 2024-05-26T22:36:34.087 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class 2024-05-26T22:36:34.087 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/cdc-acm.ko 2024-05-26T22:36:34.087 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/cdc-wdm.ko 2024-05-26T22:36:34.087 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/usblp.ko 2024-05-26T22:36:34.088 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/usbtmc.ko 2024-05-26T22:36:34.088 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/host 2024-05-26T22:36:34.088 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/host/sl811-hcd.ko 2024-05-26T22:36:34.088 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/image 2024-05-26T22:36:34.088 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/image/mdc800.ko 2024-05-26T22:36:34.088 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/image/microtek.ko 2024-05-26T22:36:34.088 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc 2024-05-26T22:36:34.088 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/adutux.ko 2024-05-26T22:36:34.088 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/appledisplay.ko 2024-05-26T22:36:34.088 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/emi26.ko 2024-05-26T22:36:34.088 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/emi62.ko 2024-05-26T22:36:34.088 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/ezusb.ko 2024-05-26T22:36:34.088 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/idmouse.ko 2024-05-26T22:36:34.089 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/iowarrior.ko 2024-05-26T22:36:34.089 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/isight_firmware.ko 2024-05-26T22:36:34.089 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/ldusb.ko 2024-05-26T22:36:34.089 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/legousbtower.ko 2024-05-26T22:36:34.089 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/sisusbvga 2024-05-26T22:36:34.089 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-05-26T22:36:34.089 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/trancevibrator.ko 2024-05-26T22:36:34.089 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/usb3503.ko 2024-05-26T22:36:34.089 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/usblcd.ko 2024-05-26T22:36:34.089 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/usbsevseg.ko 2024-05-26T22:36:34.090 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/uss720.ko 2024-05-26T22:36:34.090 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/yurex.ko 2024-05-26T22:36:34.090 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/phy 2024-05-26T22:36:34.090 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/phy/phy-generic.ko 2024-05-26T22:36:34.090 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial 2024-05-26T22:36:34.090 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/aircable.ko 2024-05-26T22:36:34.090 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ark3116.ko 2024-05-26T22:36:34.090 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/belkin_sa.ko 2024-05-26T22:36:34.090 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ch341.ko 2024-05-26T22:36:34.091 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/cp210x.ko 2024-05-26T22:36:34.091 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/cyberjack.ko 2024-05-26T22:36:34.091 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/cypress_m8.ko 2024-05-26T22:36:34.091 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/digi_acceleport.ko 2024-05-26T22:36:34.092 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/empeg.ko 2024-05-26T22:36:34.092 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ftdi_sio.ko 2024-05-26T22:36:34.092 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/garmin_gps.ko 2024-05-26T22:36:34.092 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/io_edgeport.ko 2024-05-26T22:36:34.092 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/io_ti.ko 2024-05-26T22:36:34.092 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ipaq.ko 2024-05-26T22:36:34.093 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ipw.ko 2024-05-26T22:36:34.093 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ir-usb.ko 2024-05-26T22:36:34.093 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-05-26T22:36:34.093 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/keyspan.ko 2024-05-26T22:36:34.093 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/keyspan_pda.ko 2024-05-26T22:36:34.093 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/kl5kusb105.ko 2024-05-26T22:36:34.093 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/kobil_sct.ko 2024-05-26T22:36:34.093 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/mct_u232.ko 2024-05-26T22:36:34.093 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/mos7720.ko 2024-05-26T22:36:34.093 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/mos7840.ko 2024-05-26T22:36:34.093 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/navman.ko 2024-05-26T22:36:34.093 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/omninet.ko 2024-05-26T22:36:34.093 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/opticon.ko 2024-05-26T22:36:34.094 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/option.ko 2024-05-26T22:36:34.094 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/oti6858.ko 2024-05-26T22:36:34.094 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/pl2303.ko 2024-05-26T22:36:34.094 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/qcaux.ko 2024-05-26T22:36:34.094 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/qcserial.ko 2024-05-26T22:36:34.094 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/quatech2.ko 2024-05-26T22:36:34.094 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/safe_serial.ko 2024-05-26T22:36:34.094 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/sierra.ko 2024-05-26T22:36:34.094 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/spcp8x5.ko 2024-05-26T22:36:34.094 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ssu100.ko 2024-05-26T22:36:34.094 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/symbolserial.ko 2024-05-26T22:36:34.094 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-05-26T22:36:34.095 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-05-26T22:36:34.095 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/usb_debug.ko 2024-05-26T22:36:34.095 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/usb_wwan.ko 2024-05-26T22:36:34.095 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/visor.ko 2024-05-26T22:36:34.095 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/whiteheat.ko 2024-05-26T22:36:34.095 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/xsens_mt.ko 2024-05-26T22:36:34.095 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage 2024-05-26T22:36:34.095 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/uas.ko 2024-05-26T22:36:34.095 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-alauda.ko 2024-05-26T22:36:34.095 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-cypress.ko 2024-05-26T22:36:34.096 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-datafab.ko 2024-05-26T22:36:34.096 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-05-26T22:36:34.096 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-freecom.ko 2024-05-26T22:36:34.096 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-isd200.ko 2024-05-26T22:36:34.096 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-05-26T22:36:34.096 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-karma.ko 2024-05-26T22:36:34.096 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-onetouch.ko 2024-05-26T22:36:34.096 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-realtek.ko 2024-05-26T22:36:34.096 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-sddr09.ko 2024-05-26T22:36:34.096 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-sddr55.ko 2024-05-26T22:36:34.096 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-usbat.ko 2024-05-26T22:36:34.096 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/usb-storage.ko 2024-05-26T22:36:34.096 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip 2024-05-26T22:36:34.097 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip/usbip-core.ko 2024-05-26T22:36:34.097 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip/usbip-host.ko 2024-05-26T22:36:34.097 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-05-26T22:36:34.097 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio 2024-05-26T22:36:34.097 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/pci 2024-05-26T22:36:34.097 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-05-26T22:36:34.097 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/pci/vfio-pci.ko 2024-05-26T22:36:34.097 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/vfio.ko 2024-05-26T22:36:34.097 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-05-26T22:36:34.097 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost 2024-05-26T22:36:34.097 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost.ko 2024-05-26T22:36:34.098 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost_iotlb.ko 2024-05-26T22:36:34.098 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost_net.ko 2024-05-26T22:36:34.098 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost_scsi.ko 2024-05-26T22:36:34.098 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video 2024-05-26T22:36:34.098 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight 2024-05-26T22:36:34.098 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/apple_bl.ko 2024-05-26T22:36:34.098 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/lcd.ko 2024-05-26T22:36:34.098 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/platform_lcd.ko 2024-05-26T22:36:34.098 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio 2024-05-26T22:36:34.098 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio.ko 2024-05-26T22:36:34.098 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_balloon.ko 2024-05-26T22:36:34.098 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_mmio.ko 2024-05-26T22:36:34.098 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_pci.ko 2024-05-26T22:36:34.099 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-05-26T22:36:34.099 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-05-26T22:36:34.099 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_ring.ko 2024-05-26T22:36:34.099 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1 2024-05-26T22:36:34.099 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/masters 2024-05-26T22:36:34.099 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/masters/ds2482.ko 2024-05-26T22:36:34.099 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/masters/ds2490.ko 2024-05-26T22:36:34.099 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves 2024-05-26T22:36:34.099 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-05-26T22:36:34.099 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-05-26T22:36:34.099 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-05-26T22:36:34.099 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-05-26T22:36:34.099 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-05-26T22:36:34.100 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-05-26T22:36:34.100 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-05-26T22:36:34.100 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-05-26T22:36:34.100 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-05-26T22:36:34.100 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_smem.ko 2024-05-26T22:36:34.100 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_therm.ko 2024-05-26T22:36:34.100 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/wire.ko 2024-05-26T22:36:34.100 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog 2024-05-26T22:36:34.100 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/alim1535_wdt.ko 2024-05-26T22:36:34.100 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/alim7101_wdt.ko 2024-05-26T22:36:34.100 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/f71808e_wdt.ko 2024-05-26T22:36:34.100 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/hpwdt.ko 2024-05-26T22:36:34.100 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/i6300esb.ko 2024-05-26T22:36:34.101 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-05-26T22:36:34.101 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/iTCO_wdt.ko 2024-05-26T22:36:34.101 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/ib700wdt.ko 2024-05-26T22:36:34.101 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/ibmasr.ko 2024-05-26T22:36:34.101 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-05-26T22:36:34.101 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/it8712f_wdt.ko 2024-05-26T22:36:34.101 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/it87_wdt.ko 2024-05-26T22:36:34.101 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/machzwd.ko 2024-05-26T22:36:34.101 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/nv_tco.ko 2024-05-26T22:36:34.101 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/pcwd_pci.ko 2024-05-26T22:36:34.101 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/pcwd_usb.ko 2024-05-26T22:36:34.101 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-05-26T22:36:34.101 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/sch311x_wdt.ko 2024-05-26T22:36:34.102 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/softdog.ko 2024-05-26T22:36:34.102 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/sp5100_tco.ko 2024-05-26T22:36:34.102 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/via_wdt.ko 2024-05-26T22:36:34.102 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-05-26T22:36:34.102 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/w83877f_wdt.ko 2024-05-26T22:36:34.102 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/w83977f_wdt.ko 2024-05-26T22:36:34.102 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/wdt_pci.ko 2024-05-26T22:36:34.102 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/xen_wdt.ko 2024-05-26T22:36:34.102 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen 2024-05-26T22:36:34.102 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-acpi-processor.ko 2024-05-26T22:36:34.102 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-evtchn.ko 2024-05-26T22:36:34.102 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-gntalloc.ko 2024-05-26T22:36:34.102 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-gntdev.ko 2024-05-26T22:36:34.103 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-pciback 2024-05-26T22:36:34.103 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-05-26T22:36:34.103 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-privcmd.ko 2024-05-26T22:36:34.103 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xenfs 2024-05-26T22:36:34.103 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xenfs/xenfs.ko 2024-05-26T22:36:34.103 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs 2024-05-26T22:36:34.103 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/9p 2024-05-26T22:36:34.103 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/9p/9p.ko 2024-05-26T22:36:34.103 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/affs 2024-05-26T22:36:34.103 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/affs/affs.ko 2024-05-26T22:36:34.103 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/befs 2024-05-26T22:36:34.103 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/befs/befs.ko 2024-05-26T22:36:34.103 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/binfmt_misc.ko 2024-05-26T22:36:34.104 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/btrfs 2024-05-26T22:36:34.104 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/btrfs/btrfs.ko 2024-05-26T22:36:34.104 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/cachefiles 2024-05-26T22:36:34.104 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/cachefiles/cachefiles.ko 2024-05-26T22:36:34.104 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ceph 2024-05-26T22:36:34.104 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ceph/ceph.ko 2024-05-26T22:36:34.104 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/coda 2024-05-26T22:36:34.104 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/coda/coda.ko 2024-05-26T22:36:34.104 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/cramfs 2024-05-26T22:36:34.104 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/cramfs/cramfs.ko 2024-05-26T22:36:34.104 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/dlm 2024-05-26T22:36:34.104 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/dlm/dlm.ko 2024-05-26T22:36:34.104 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ecryptfs 2024-05-26T22:36:34.104 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ecryptfs/ecryptfs.ko 2024-05-26T22:36:34.104 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/f2fs 2024-05-26T22:36:34.105 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/f2fs/f2fs.ko 2024-05-26T22:36:34.105 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fat 2024-05-26T22:36:34.105 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fat/fat.ko 2024-05-26T22:36:34.105 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fat/msdos.ko 2024-05-26T22:36:34.105 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fat/vfat.ko 2024-05-26T22:36:34.105 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fuse 2024-05-26T22:36:34.105 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fuse/cuse.ko 2024-05-26T22:36:34.105 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fuse/fuse.ko 2024-05-26T22:36:34.105 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/gfs2 2024-05-26T22:36:34.105 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/gfs2/gfs2.ko 2024-05-26T22:36:34.105 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfs 2024-05-26T22:36:34.105 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfs/hfs.ko 2024-05-26T22:36:34.105 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfsplus 2024-05-26T22:36:34.105 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfsplus/hfsplus.ko 2024-05-26T22:36:34.106 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/isofs 2024-05-26T22:36:34.106 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/isofs/isofs.ko 2024-05-26T22:36:34.106 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/jfs 2024-05-26T22:36:34.106 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/jfs/jfs.ko 2024-05-26T22:36:34.106 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/lockd 2024-05-26T22:36:34.106 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/lockd/lockd.ko 2024-05-26T22:36:34.106 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/minix 2024-05-26T22:36:34.106 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/minix/minix.ko 2024-05-26T22:36:34.106 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/netfs 2024-05-26T22:36:34.106 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/netfs/netfs.ko 2024-05-26T22:36:34.106 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs 2024-05-26T22:36:34.106 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/blocklayout 2024-05-26T22:36:34.106 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-05-26T22:36:34.107 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/filelayout 2024-05-26T22:36:34.107 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-05-26T22:36:34.107 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/flexfilelayout 2024-05-26T22:36:34.107 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-05-26T22:36:34.107 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/nfs.ko 2024-05-26T22:36:34.107 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/nfsv3.ko 2024-05-26T22:36:34.107 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/nfsv4.ko 2024-05-26T22:36:34.107 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs_common 2024-05-26T22:36:34.107 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs_common/grace.ko 2024-05-26T22:36:34.107 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs_common/nfs_acl.ko 2024-05-26T22:36:34.107 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfsd 2024-05-26T22:36:34.107 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfsd/nfsd.ko 2024-05-26T22:36:34.107 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nilfs2 2024-05-26T22:36:34.107 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nilfs2/nilfs2.ko 2024-05-26T22:36:34.107 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls 2024-05-26T22:36:34.108 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-celtic.ko 2024-05-26T22:36:34.108 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-centeuro.ko 2024-05-26T22:36:34.108 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-croatian.ko 2024-05-26T22:36:34.108 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-cyrillic.ko 2024-05-26T22:36:34.108 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-gaelic.ko 2024-05-26T22:36:34.108 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-greek.ko 2024-05-26T22:36:34.108 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-iceland.ko 2024-05-26T22:36:34.108 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-inuit.ko 2024-05-26T22:36:34.108 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-roman.ko 2024-05-26T22:36:34.108 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-romanian.ko 2024-05-26T22:36:34.108 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-turkish.ko 2024-05-26T22:36:34.108 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp1250.ko 2024-05-26T22:36:34.108 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp1251.ko 2024-05-26T22:36:34.108 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp1255.ko 2024-05-26T22:36:34.108 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp737.ko 2024-05-26T22:36:34.109 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp775.ko 2024-05-26T22:36:34.109 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp850.ko 2024-05-26T22:36:34.109 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp852.ko 2024-05-26T22:36:34.109 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp855.ko 2024-05-26T22:36:34.109 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp857.ko 2024-05-26T22:36:34.109 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp860.ko 2024-05-26T22:36:34.109 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp861.ko 2024-05-26T22:36:34.109 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp862.ko 2024-05-26T22:36:34.109 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp863.ko 2024-05-26T22:36:34.109 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp864.ko 2024-05-26T22:36:34.109 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp865.ko 2024-05-26T22:36:34.109 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp866.ko 2024-05-26T22:36:34.109 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp869.ko 2024-05-26T22:36:34.110 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp874.ko 2024-05-26T22:36:34.110 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp932.ko 2024-05-26T22:36:34.110 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp936.ko 2024-05-26T22:36:34.110 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp949.ko 2024-05-26T22:36:34.110 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp950.ko 2024-05-26T22:36:34.110 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_euc-jp.ko 2024-05-26T22:36:34.110 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-1.ko 2024-05-26T22:36:34.110 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-13.ko 2024-05-26T22:36:34.110 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-14.ko 2024-05-26T22:36:34.110 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-15.ko 2024-05-26T22:36:34.110 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-2.ko 2024-05-26T22:36:34.110 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-3.ko 2024-05-26T22:36:34.110 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-4.ko 2024-05-26T22:36:34.111 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-5.ko 2024-05-26T22:36:34.111 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-6.ko 2024-05-26T22:36:34.111 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-7.ko 2024-05-26T22:36:34.111 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-9.ko 2024-05-26T22:36:34.111 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_koi8-r.ko 2024-05-26T22:36:34.111 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_koi8-ru.ko 2024-05-26T22:36:34.111 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_koi8-u.ko 2024-05-26T22:36:34.111 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_ucs2_utils.ko 2024-05-26T22:36:34.111 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_utf8.ko 2024-05-26T22:36:34.111 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2 2024-05-26T22:36:34.111 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/cluster 2024-05-26T22:36:34.111 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-05-26T22:36:34.111 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlm 2024-05-26T22:36:34.112 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-05-26T22:36:34.112 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlmfs 2024-05-26T22:36:34.112 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-05-26T22:36:34.112 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2.ko 2024-05-26T22:36:34.112 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-05-26T22:36:34.112 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-05-26T22:36:34.112 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-05-26T22:36:34.112 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/overlayfs 2024-05-26T22:36:34.112 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/overlayfs/overlay.ko 2024-05-26T22:36:34.112 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/pstore 2024-05-26T22:36:34.112 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/pstore/ramoops.ko 2024-05-26T22:36:34.112 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/reiserfs 2024-05-26T22:36:34.112 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/reiserfs/reiserfs.ko 2024-05-26T22:36:34.112 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/romfs 2024-05-26T22:36:34.113 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/romfs/romfs.ko 2024-05-26T22:36:34.113 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb 2024-05-26T22:36:34.113 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/client 2024-05-26T22:36:34.113 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/client/cifs.ko 2024-05-26T22:36:34.113 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/common 2024-05-26T22:36:34.113 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/common/cifs_arc4.ko 2024-05-26T22:36:34.113 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/common/cifs_md4.ko 2024-05-26T22:36:34.113 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/squashfs 2024-05-26T22:36:34.113 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/squashfs/squashfs.ko 2024-05-26T22:36:34.113 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/sysv 2024-05-26T22:36:34.113 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/sysv/sysv.ko 2024-05-26T22:36:34.113 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/udf 2024-05-26T22:36:34.113 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/udf/udf.ko 2024-05-26T22:36:34.113 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ufs 2024-05-26T22:36:34.114 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ufs/ufs.ko 2024-05-26T22:36:34.114 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/xfs 2024-05-26T22:36:34.114 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/xfs/xfs.ko 2024-05-26T22:36:34.114 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel 2024-05-26T22:36:34.114 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/rcu 2024-05-26T22:36:34.114 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/rcu/rcutorture.ko 2024-05-26T22:36:34.114 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/torture.ko 2024-05-26T22:36:34.114 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/trace 2024-05-26T22:36:34.114 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-05-26T22:36:34.114 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib 2024-05-26T22:36:34.114 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/asn1_encoder.ko 2024-05-26T22:36:34.114 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crc-itu-t.ko 2024-05-26T22:36:34.114 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crc8.ko 2024-05-26T22:36:34.115 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crypto 2024-05-26T22:36:34.115 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crypto/libarc4.ko 2024-05-26T22:36:34.115 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crypto/libdes.ko 2024-05-26T22:36:34.115 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/libcrc32c.ko 2024-05-26T22:36:34.115 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/lru_cache.ko 2024-05-26T22:36:34.115 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/lz4 2024-05-26T22:36:34.115 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/lz4/lz4_compress.ko 2024-05-26T22:36:34.115 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/lz4/lz4hc_compress.ko 2024-05-26T22:36:34.115 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/math 2024-05-26T22:36:34.115 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/math/cordic.ko 2024-05-26T22:36:34.115 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/raid6 2024-05-26T22:36:34.115 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/raid6/raid6_pq.ko 2024-05-26T22:36:34.115 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/reed_solomon 2024-05-26T22:36:34.115 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/reed_solomon/reed_solomon.ko 2024-05-26T22:36:34.116 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/ts_bm.ko 2024-05-26T22:36:34.116 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/ts_fsm.ko 2024-05-26T22:36:34.116 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/ts_kmp.ko 2024-05-26T22:36:34.116 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/zstd 2024-05-26T22:36:34.116 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/zstd/zstd_compress.ko 2024-05-26T22:36:34.116 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/mm 2024-05-26T22:36:34.116 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/mm/hwpoison-inject.ko 2024-05-26T22:36:34.116 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net 2024-05-26T22:36:34.116 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan 2024-05-26T22:36:34.116 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/6lowpan.ko 2024-05-26T22:36:34.116 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_dest.ko 2024-05-26T22:36:34.116 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_fragment.ko 2024-05-26T22:36:34.116 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_hop.ko 2024-05-26T22:36:34.116 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_ipv6.ko 2024-05-26T22:36:34.117 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_mobility.ko 2024-05-26T22:36:34.117 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_routing.ko 2024-05-26T22:36:34.117 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_udp.ko 2024-05-26T22:36:34.117 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802 2024-05-26T22:36:34.117 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/garp.ko 2024-05-26T22:36:34.117 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/mrp.ko 2024-05-26T22:36:34.117 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/p8022.ko 2024-05-26T22:36:34.117 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/psnap.ko 2024-05-26T22:36:34.117 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/stp.ko 2024-05-26T22:36:34.117 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/8021q 2024-05-26T22:36:34.117 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/8021q/8021q.ko 2024-05-26T22:36:34.117 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p 2024-05-26T22:36:34.117 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet.ko 2024-05-26T22:36:34.118 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet_fd.ko 2024-05-26T22:36:34.118 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet_rdma.ko 2024-05-26T22:36:34.118 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet_virtio.ko 2024-05-26T22:36:34.118 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/batman-adv 2024-05-26T22:36:34.118 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/batman-adv/batman-adv.ko 2024-05-26T22:36:34.118 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge 2024-05-26T22:36:34.118 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/br_netfilter.ko 2024-05-26T22:36:34.118 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/bridge.ko 2024-05-26T22:36:34.118 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter 2024-05-26T22:36:34.118 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-05-26T22:36:34.118 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_among.ko 2024-05-26T22:36:34.118 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_arp.ko 2024-05-26T22:36:34.119 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-05-26T22:36:34.119 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-05-26T22:36:34.119 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_ip.ko 2024-05-26T22:36:34.119 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-05-26T22:36:34.119 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_limit.ko 2024-05-26T22:36:34.119 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_log.ko 2024-05-26T22:36:34.119 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_mark.ko 2024-05-26T22:36:34.119 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-05-26T22:36:34.119 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-05-26T22:36:34.119 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-05-26T22:36:34.119 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-05-26T22:36:34.119 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_snat.ko 2024-05-26T22:36:34.119 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_stp.ko 2024-05-26T22:36:34.120 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-05-26T22:36:34.120 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-05-26T22:36:34.120 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-05-26T22:36:34.120 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-05-26T22:36:34.120 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtables.ko 2024-05-26T22:36:34.120 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-05-26T22:36:34.120 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-05-26T22:36:34.120 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-05-26T22:36:34.120 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ceph 2024-05-26T22:36:34.120 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ceph/libceph.ko 2024-05-26T22:36:34.120 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/core 2024-05-26T22:36:34.121 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/core/failover.ko 2024-05-26T22:36:34.121 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/core/pktgen.ko 2024-05-26T22:36:34.121 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp 2024-05-26T22:36:34.121 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp.ko 2024-05-26T22:36:34.121 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp_diag.ko 2024-05-26T22:36:34.121 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp_ipv4.ko 2024-05-26T22:36:34.121 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp_ipv6.ko 2024-05-26T22:36:34.121 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dns_resolver 2024-05-26T22:36:34.121 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dns_resolver/dns_resolver.ko 2024-05-26T22:36:34.121 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa 2024-05-26T22:36:34.121 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/dsa_core.ko 2024-05-26T22:36:34.122 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_brcm.ko 2024-05-26T22:36:34.122 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_dsa.ko 2024-05-26T22:36:34.122 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_none.ko 2024-05-26T22:36:34.122 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_trailer.ko 2024-05-26T22:36:34.122 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154 2024-05-26T22:36:34.122 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/6lowpan 2024-05-26T22:36:34.122 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-05-26T22:36:34.122 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/ieee802154.ko 2024-05-26T22:36:34.122 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/ieee802154_socket.ko 2024-05-26T22:36:34.122 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4 2024-05-26T22:36:34.122 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ah4.ko 2024-05-26T22:36:34.122 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/esp4.ko 2024-05-26T22:36:34.123 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/fou.ko 2024-05-26T22:36:34.123 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/gre.ko 2024-05-26T22:36:34.123 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/inet_diag.ko 2024-05-26T22:36:34.123 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ip_gre.ko 2024-05-26T22:36:34.123 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ip_tunnel.ko 2024-05-26T22:36:34.123 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ip_vti.ko 2024-05-26T22:36:34.123 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ipcomp.ko 2024-05-26T22:36:34.123 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ipip.ko 2024-05-26T22:36:34.123 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter 2024-05-26T22:36:34.123 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/arp_tables.ko 2024-05-26T22:36:34.123 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-05-26T22:36:34.123 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-05-26T22:36:34.123 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-05-26T22:36:34.124 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-05-26T22:36:34.124 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-05-26T22:36:34.124 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-05-26T22:36:34.124 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-05-26T22:36:34.124 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-05-26T22:36:34.124 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-05-26T22:36:34.124 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_security.ko 2024-05-26T22:36:34.124 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-05-26T22:36:34.124 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-05-26T22:36:34.124 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-05-26T22:36:34.124 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-05-26T22:36:34.124 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-05-26T22:36:34.124 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-05-26T22:36:34.124 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-05-26T22:36:34.124 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-05-26T22:36:34.125 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-05-26T22:36:34.125 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-05-26T22:36:34.125 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_bic.ko 2024-05-26T22:36:34.125 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_dctcp.ko 2024-05-26T22:36:34.125 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_diag.ko 2024-05-26T22:36:34.125 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_highspeed.ko 2024-05-26T22:36:34.125 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_htcp.ko 2024-05-26T22:36:34.125 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_hybla.ko 2024-05-26T22:36:34.125 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_illinois.ko 2024-05-26T22:36:34.125 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_lp.ko 2024-05-26T22:36:34.125 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_scalable.ko 2024-05-26T22:36:34.125 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_vegas.ko 2024-05-26T22:36:34.125 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_veno.ko 2024-05-26T22:36:34.125 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_westwood.ko 2024-05-26T22:36:34.126 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_yeah.ko 2024-05-26T22:36:34.126 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tunnel4.ko 2024-05-26T22:36:34.126 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/udp_diag.ko 2024-05-26T22:36:34.126 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/udp_tunnel.ko 2024-05-26T22:36:34.126 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/xfrm4_tunnel.ko 2024-05-26T22:36:34.126 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6 2024-05-26T22:36:34.126 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ah6.ko 2024-05-26T22:36:34.126 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/esp6.ko 2024-05-26T22:36:34.126 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/fou6.ko 2024-05-26T22:36:34.126 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ip6_tunnel.ko 2024-05-26T22:36:34.126 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-05-26T22:36:34.126 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ip6_vti.ko 2024-05-26T22:36:34.126 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ipcomp6.ko 2024-05-26T22:36:34.127 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter 2024-05-26T22:36:34.127 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-05-26T22:36:34.127 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-05-26T22:36:34.127 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-05-26T22:36:34.127 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-05-26T22:36:34.127 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-05-26T22:36:34.127 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-05-26T22:36:34.127 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-05-26T22:36:34.127 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-05-26T22:36:34.127 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-05-26T22:36:34.127 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-05-26T22:36:34.127 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-05-26T22:36:34.127 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-05-26T22:36:34.128 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-05-26T22:36:34.128 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-05-26T22:36:34.128 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-05-26T22:36:34.128 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-05-26T22:36:34.128 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-05-26T22:36:34.128 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-05-26T22:36:34.128 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-05-26T22:36:34.128 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-05-26T22:36:34.128 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-05-26T22:36:34.128 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-05-26T22:36:34.128 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-05-26T22:36:34.128 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-05-26T22:36:34.128 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/sit.ko 2024-05-26T22:36:34.128 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/tunnel6.ko 2024-05-26T22:36:34.129 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/xfrm6_tunnel.ko 2024-05-26T22:36:34.129 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/key 2024-05-26T22:36:34.129 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/key/af_key.ko 2024-05-26T22:36:34.129 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp 2024-05-26T22:36:34.129 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_core.ko 2024-05-26T22:36:34.129 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_debugfs.ko 2024-05-26T22:36:34.129 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_eth.ko 2024-05-26T22:36:34.129 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_ip.ko 2024-05-26T22:36:34.129 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_ip6.ko 2024-05-26T22:36:34.129 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_netlink.ko 2024-05-26T22:36:34.129 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_ppp.ko 2024-05-26T22:36:34.129 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/llc 2024-05-26T22:36:34.130 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/llc/llc.ko 2024-05-26T22:36:34.130 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/mac802154 2024-05-26T22:36:34.130 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/mac802154/mac802154.ko 2024-05-26T22:36:34.130 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/mpls 2024-05-26T22:36:34.130 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/mpls/mpls_gso.ko 2024-05-26T22:36:34.130 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter 2024-05-26T22:36:34.130 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset 2024-05-26T22:36:34.130 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set.ko 2024-05-26T22:36:34.130 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-05-26T22:36:34.130 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-05-26T22:36:34.130 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-05-26T22:36:34.130 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-05-26T22:36:34.131 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-05-26T22:36:34.131 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-05-26T22:36:34.131 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-05-26T22:36:34.131 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-05-26T22:36:34.131 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-05-26T22:36:34.131 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-05-26T22:36:34.131 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-05-26T22:36:34.131 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-05-26T22:36:34.131 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-05-26T22:36:34.131 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-05-26T22:36:34.131 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-05-26T22:36:34.131 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs 2024-05-26T22:36:34.131 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs.ko 2024-05-26T22:36:34.132 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-05-26T22:36:34.132 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-05-26T22:36:34.132 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-05-26T22:36:34.132 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-05-26T22:36:34.132 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-05-26T22:36:34.132 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-05-26T22:36:34.132 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-05-26T22:36:34.132 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-05-26T22:36:34.132 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-05-26T22:36:34.132 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-05-26T22:36:34.132 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-05-26T22:36:34.132 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-05-26T22:36:34.133 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-05-26T22:36:34.133 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conncount.ko 2024-05-26T22:36:34.133 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack.ko 2024-05-26T22:36:34.133 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-05-26T22:36:34.133 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-05-26T22:36:34.133 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-05-26T22:36:34.133 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_h323.ko 2024-05-26T22:36:34.133 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_irc.ko 2024-05-26T22:36:34.133 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-05-26T22:36:34.133 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-05-26T22:36:34.133 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-05-26T22:36:34.133 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_sane.ko 2024-05-26T22:36:34.133 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_sip.ko 2024-05-26T22:36:34.134 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-05-26T22:36:34.134 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-05-26T22:36:34.134 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_dup_netdev.ko 2024-05-26T22:36:34.134 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_flow_table.ko 2024-05-26T22:36:34.134 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_flow_table_inet.ko 2024-05-26T22:36:34.134 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_log_syslog.ko 2024-05-26T22:36:34.134 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat.ko 2024-05-26T22:36:34.134 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_amanda.ko 2024-05-26T22:36:34.134 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_ftp.ko 2024-05-26T22:36:34.134 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_irc.ko 2024-05-26T22:36:34.134 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_sip.ko 2024-05-26T22:36:34.134 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_tftp.ko 2024-05-26T22:36:34.134 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_synproxy_core.ko 2024-05-26T22:36:34.135 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_tables.ko 2024-05-26T22:36:34.135 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink.ko 2024-05-26T22:36:34.135 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_acct.ko 2024-05-26T22:36:34.135 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_log.ko 2024-05-26T22:36:34.135 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_osf.ko 2024-05-26T22:36:34.135 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_queue.ko 2024-05-26T22:36:34.135 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_chain_nat.ko 2024-05-26T22:36:34.135 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_compat.ko 2024-05-26T22:36:34.135 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_ct.ko 2024-05-26T22:36:34.135 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_dup_netdev.ko 2024-05-26T22:36:34.135 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_fib.ko 2024-05-26T22:36:34.135 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_fwd_netdev.ko 2024-05-26T22:36:34.135 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_hash.ko 2024-05-26T22:36:34.135 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_limit.ko 2024-05-26T22:36:34.136 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_log.ko 2024-05-26T22:36:34.136 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_masq.ko 2024-05-26T22:36:34.136 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_nat.ko 2024-05-26T22:36:34.136 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_numgen.ko 2024-05-26T22:36:34.136 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_queue.ko 2024-05-26T22:36:34.136 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_quota.ko 2024-05-26T22:36:34.136 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_redir.ko 2024-05-26T22:36:34.136 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_reject.ko 2024-05-26T22:36:34.136 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_reject_inet.ko 2024-05-26T22:36:34.136 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_socket.ko 2024-05-26T22:36:34.136 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_tunnel.ko 2024-05-26T22:36:34.136 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_AUDIT.ko 2024-05-26T22:36:34.136 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CHECKSUM.ko 2024-05-26T22:36:34.136 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CLASSIFY.ko 2024-05-26T22:36:34.137 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-05-26T22:36:34.137 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CT.ko 2024-05-26T22:36:34.137 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_DSCP.ko 2024-05-26T22:36:34.137 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_HL.ko 2024-05-26T22:36:34.137 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_HMARK.ko 2024-05-26T22:36:34.137 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_IDLETIMER.ko 2024-05-26T22:36:34.137 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_LED.ko 2024-05-26T22:36:34.137 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_LOG.ko 2024-05-26T22:36:34.137 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_MASQUERADE.ko 2024-05-26T22:36:34.137 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_NETMAP.ko 2024-05-26T22:36:34.137 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_NFLOG.ko 2024-05-26T22:36:34.137 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_NFQUEUE.ko 2024-05-26T22:36:34.137 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_RATEEST.ko 2024-05-26T22:36:34.137 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_REDIRECT.ko 2024-05-26T22:36:34.138 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_SECMARK.ko 2024-05-26T22:36:34.138 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TCPMSS.ko 2024-05-26T22:36:34.138 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-05-26T22:36:34.138 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TEE.ko 2024-05-26T22:36:34.138 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TPROXY.ko 2024-05-26T22:36:34.138 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TRACE.ko 2024-05-26T22:36:34.138 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_addrtype.ko 2024-05-26T22:36:34.138 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_bpf.ko 2024-05-26T22:36:34.138 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_cgroup.ko 2024-05-26T22:36:34.138 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_cluster.ko 2024-05-26T22:36:34.138 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_comment.ko 2024-05-26T22:36:34.138 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connbytes.ko 2024-05-26T22:36:34.138 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connlabel.ko 2024-05-26T22:36:34.139 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connlimit.ko 2024-05-26T22:36:34.139 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connmark.ko 2024-05-26T22:36:34.139 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_conntrack.ko 2024-05-26T22:36:34.139 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_cpu.ko 2024-05-26T22:36:34.139 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_dccp.ko 2024-05-26T22:36:34.139 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_devgroup.ko 2024-05-26T22:36:34.139 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_dscp.ko 2024-05-26T22:36:34.139 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_ecn.ko 2024-05-26T22:36:34.139 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_esp.ko 2024-05-26T22:36:34.139 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_hashlimit.ko 2024-05-26T22:36:34.139 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_helper.ko 2024-05-26T22:36:34.139 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_hl.ko 2024-05-26T22:36:34.139 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_ipcomp.ko 2024-05-26T22:36:34.139 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_iprange.ko 2024-05-26T22:36:34.140 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_ipvs.ko 2024-05-26T22:36:34.140 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_l2tp.ko 2024-05-26T22:36:34.140 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_length.ko 2024-05-26T22:36:34.140 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_limit.ko 2024-05-26T22:36:34.140 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_mac.ko 2024-05-26T22:36:34.140 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_mark.ko 2024-05-26T22:36:34.140 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_multiport.ko 2024-05-26T22:36:34.140 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_nat.ko 2024-05-26T22:36:34.140 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_nfacct.ko 2024-05-26T22:36:34.140 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_osf.ko 2024-05-26T22:36:34.140 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_owner.ko 2024-05-26T22:36:34.140 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_physdev.ko 2024-05-26T22:36:34.141 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_pkttype.ko 2024-05-26T22:36:34.141 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_policy.ko 2024-05-26T22:36:34.141 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_quota.ko 2024-05-26T22:36:34.141 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_rateest.ko 2024-05-26T22:36:34.141 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_realm.ko 2024-05-26T22:36:34.141 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_recent.ko 2024-05-26T22:36:34.141 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_sctp.ko 2024-05-26T22:36:34.141 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_set.ko 2024-05-26T22:36:34.141 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_socket.ko 2024-05-26T22:36:34.141 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_state.ko 2024-05-26T22:36:34.141 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_statistic.ko 2024-05-26T22:36:34.141 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_string.ko 2024-05-26T22:36:34.142 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_tcpmss.ko 2024-05-26T22:36:34.142 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_time.ko 2024-05-26T22:36:34.142 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_u32.ko 2024-05-26T22:36:34.142 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netlink 2024-05-26T22:36:34.142 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netlink/netlink_diag.ko 2024-05-26T22:36:34.142 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/nsh 2024-05-26T22:36:34.142 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/nsh/nsh.ko 2024-05-26T22:36:34.142 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch 2024-05-26T22:36:34.142 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/openvswitch.ko 2024-05-26T22:36:34.142 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/vport-geneve.ko 2024-05-26T22:36:34.142 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/vport-gre.ko 2024-05-26T22:36:34.142 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/vport-vxlan.ko 2024-05-26T22:36:34.142 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/packet 2024-05-26T22:36:34.143 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/packet/af_packet_diag.ko 2024-05-26T22:36:34.143 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/rds 2024-05-26T22:36:34.143 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/rds/rds.ko 2024-05-26T22:36:34.143 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/rds/rds_rdma.ko 2024-05-26T22:36:34.143 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/rds/rds_tcp.ko 2024-05-26T22:36:34.143 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched 2024-05-26T22:36:34.143 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_csum.ko 2024-05-26T22:36:34.143 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_gact.ko 2024-05-26T22:36:34.143 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_mirred.ko 2024-05-26T22:36:34.143 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_nat.ko 2024-05-26T22:36:34.143 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_pedit.ko 2024-05-26T22:36:34.143 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_police.ko 2024-05-26T22:36:34.144 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_simple.ko 2024-05-26T22:36:34.144 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_skbedit.ko 2024-05-26T22:36:34.144 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_vlan.ko 2024-05-26T22:36:34.144 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_basic.ko 2024-05-26T22:36:34.144 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_bpf.ko 2024-05-26T22:36:34.144 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_flow.ko 2024-05-26T22:36:34.144 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_fw.ko 2024-05-26T22:36:34.144 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_route.ko 2024-05-26T22:36:34.144 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_u32.ko 2024-05-26T22:36:34.144 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_cmp.ko 2024-05-26T22:36:34.144 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_ipset.ko 2024-05-26T22:36:34.144 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_meta.ko 2024-05-26T22:36:34.144 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_nbyte.ko 2024-05-26T22:36:34.145 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_text.ko 2024-05-26T22:36:34.145 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_u32.ko 2024-05-26T22:36:34.145 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_choke.ko 2024-05-26T22:36:34.145 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_codel.ko 2024-05-26T22:36:34.145 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_drr.ko 2024-05-26T22:36:34.145 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_fq.ko 2024-05-26T22:36:34.145 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_gred.ko 2024-05-26T22:36:34.145 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_hfsc.ko 2024-05-26T22:36:34.145 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_hhf.ko 2024-05-26T22:36:34.145 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_htb.ko 2024-05-26T22:36:34.145 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_ingress.ko 2024-05-26T22:36:34.145 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_mqprio.ko 2024-05-26T22:36:34.145 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_mqprio_lib.ko 2024-05-26T22:36:34.146 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_multiq.ko 2024-05-26T22:36:34.146 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_netem.ko 2024-05-26T22:36:34.146 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_pie.ko 2024-05-26T22:36:34.146 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_plug.ko 2024-05-26T22:36:34.146 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_prio.ko 2024-05-26T22:36:34.146 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_qfq.ko 2024-05-26T22:36:34.146 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_red.ko 2024-05-26T22:36:34.146 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_sfb.ko 2024-05-26T22:36:34.146 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_sfq.ko 2024-05-26T22:36:34.146 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_tbf.ko 2024-05-26T22:36:34.146 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_teql.ko 2024-05-26T22:36:34.146 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sctp 2024-05-26T22:36:34.147 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sctp/sctp.ko 2024-05-26T22:36:34.147 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sctp/sctp_diag.ko 2024-05-26T22:36:34.147 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc 2024-05-26T22:36:34.147 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/auth_gss 2024-05-26T22:36:34.147 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-05-26T22:36:34.147 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-05-26T22:36:34.147 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/sunrpc.ko 2024-05-26T22:36:34.147 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/xprtrdma 2024-05-26T22:36:34.147 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-05-26T22:36:34.147 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/tipc 2024-05-26T22:36:34.147 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/tipc/tipc.ko 2024-05-26T22:36:34.147 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/tipc/tipc_diag.ko 2024-05-26T22:36:34.147 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/unix 2024-05-26T22:36:34.148 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/unix/unix_diag.ko 2024-05-26T22:36:34.148 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock 2024-05-26T22:36:34.148 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-05-26T22:36:34.148 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-05-26T22:36:34.148 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vsock.ko 2024-05-26T22:36:34.148 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vsock_diag.ko 2024-05-26T22:36:34.148 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vsock_loopback.ko 2024-05-26T22:36:34.148 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/xfrm 2024-05-26T22:36:34.148 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/xfrm/xfrm_ipcomp.ko 2024-05-26T22:36:34.148 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security 2024-05-26T22:36:34.148 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys 2024-05-26T22:36:34.148 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys/encrypted-keys 2024-05-26T22:36:34.148 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-05-26T22:36:34.149 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys/trusted-keys 2024-05-26T22:36:34.149 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys/trusted-keys/trusted.ko 2024-05-26T22:36:34.149 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.alias 2024-05-26T22:36:34.149 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.alias.bin 2024-05-26T22:36:34.149 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.builtin 2024-05-26T22:36:34.149 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.builtin.alias.bin 2024-05-26T22:36:34.149 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.builtin.bin 2024-05-26T22:36:34.149 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.builtin.modinfo 2024-05-26T22:36:34.149 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.dep 2024-05-26T22:36:34.149 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.dep.bin 2024-05-26T22:36:34.149 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.devname 2024-05-26T22:36:34.149 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.order 2024-05-26T22:36:34.149 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.softdep 2024-05-26T22:36:34.149 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.symbols 2024-05-26T22:36:34.150 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.symbols.bin 2024-05-26T22:36:34.150 INFO:teuthology.orchestra.run.smithi131.stdout:/lib/modules/6.9.0-gb9459a26b088/vmlinuz 2024-05-26T22:36:34.153 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-gb9459a26b088 2024-05-26T22:36:34.153 DEBUG:teuthology.orchestra.run.smithi131:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-05-26T22:36:34.175 INFO:teuthology.orchestra.run.smithi131.stdout:postinstall scriptlet (using /bin/sh): 2024-05-26T22:36:34.175 INFO:teuthology.orchestra.run.smithi131.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-05-26T22:36:34.175 INFO:teuthology.orchestra.run.smithi131.stdout: /usr/bin/kernel-install add 6.9.0-gb9459a26b088 /lib/modules/6.9.0-gb9459a26b088/vmlinuz 2024-05-26T22:36:34.175 INFO:teuthology.orchestra.run.smithi131.stdout:fi 2024-05-26T22:36:34.175 INFO:teuthology.orchestra.run.smithi131.stdout:for file in vmlinuz System.map config; do 2024-05-26T22:36:34.175 INFO:teuthology.orchestra.run.smithi131.stdout: if ! cmp --silent "/lib/modules/6.9.0-gb9459a26b088/${file}" "/boot/${file}-6.9.0-gb9459a26b088"; then 2024-05-26T22:36:34.175 INFO:teuthology.orchestra.run.smithi131.stdout: cp "/lib/modules/6.9.0-gb9459a26b088/${file}" "/boot/${file}-6.9.0-gb9459a26b088" 2024-05-26T22:36:34.175 INFO:teuthology.orchestra.run.smithi131.stdout: fi 2024-05-26T22:36:34.176 INFO:teuthology.orchestra.run.smithi131.stdout:done 2024-05-26T22:36:34.176 INFO:teuthology.orchestra.run.smithi131.stdout:if [ -d "/lib/modules/6.9.0-gb9459a26b088/dtb" ] && \ 2024-05-26T22:36:34.176 INFO:teuthology.orchestra.run.smithi131.stdout: ! diff -rq "/lib/modules/6.9.0-gb9459a26b088/dtb" "/boot/dtb-6.9.0-gb9459a26b088" >/dev/null 2>&1; then 2024-05-26T22:36:34.176 INFO:teuthology.orchestra.run.smithi131.stdout: rm -rf "/boot/dtb-6.9.0-gb9459a26b088" 2024-05-26T22:36:34.176 INFO:teuthology.orchestra.run.smithi131.stdout: cp -r "/lib/modules/6.9.0-gb9459a26b088/dtb" "/boot/dtb-6.9.0-gb9459a26b088" 2024-05-26T22:36:34.176 INFO:teuthology.orchestra.run.smithi131.stdout:fi 2024-05-26T22:36:34.176 INFO:teuthology.orchestra.run.smithi131.stdout:if [ ! -e "/lib/modules/6.9.0-gb9459a26b088/modules.dep" ]; then 2024-05-26T22:36:34.176 INFO:teuthology.orchestra.run.smithi131.stdout: /usr/sbin/depmod 6.9.0-gb9459a26b088 2024-05-26T22:36:34.176 INFO:teuthology.orchestra.run.smithi131.stdout:fi 2024-05-26T22:36:34.176 INFO:teuthology.orchestra.run.smithi131.stdout:preuninstall scriptlet (using /bin/sh): 2024-05-26T22:36:34.176 INFO:teuthology.orchestra.run.smithi131.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-05-26T22:36:34.176 INFO:teuthology.orchestra.run.smithi131.stdout:kernel-install remove 6.9.0-gb9459a26b088 2024-05-26T22:36:34.176 INFO:teuthology.orchestra.run.smithi131.stdout:fi 2024-05-26T22:36:34.177 DEBUG:teuthology.orchestra.run.smithi131:> sudo rpm -qi grub2-tools 2024-05-26T22:36:34.248 INFO:teuthology.orchestra.run.smithi131.stdout:Name : grub2-tools 2024-05-26T22:36:34.248 INFO:teuthology.orchestra.run.smithi131.stdout:Epoch : 1 2024-05-26T22:36:34.248 INFO:teuthology.orchestra.run.smithi131.stdout:Version : 2.06 2024-05-26T22:36:34.248 INFO:teuthology.orchestra.run.smithi131.stdout:Release : 68.el9 2024-05-26T22:36:34.248 INFO:teuthology.orchestra.run.smithi131.stdout:Architecture: x86_64 2024-05-26T22:36:34.248 INFO:teuthology.orchestra.run.smithi131.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-26T22:36:34.249 INFO:teuthology.orchestra.run.smithi131.stdout:Group : Unspecified 2024-05-26T22:36:34.249 INFO:teuthology.orchestra.run.smithi131.stdout:Size : 8264254 2024-05-26T22:36:34.249 INFO:teuthology.orchestra.run.smithi131.stdout:License : GPLv3+ 2024-05-26T22:36:34.249 INFO:teuthology.orchestra.run.smithi131.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-26T22:36:34.249 INFO:teuthology.orchestra.run.smithi131.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-26T22:36:34.249 INFO:teuthology.orchestra.run.smithi131.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-26T22:36:34.249 INFO:teuthology.orchestra.run.smithi131.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-26T22:36:34.249 INFO:teuthology.orchestra.run.smithi131.stdout:Packager : builder@centos.org 2024-05-26T22:36:34.249 INFO:teuthology.orchestra.run.smithi131.stdout:Vendor : CentOS 2024-05-26T22:36:34.249 INFO:teuthology.orchestra.run.smithi131.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-26T22:36:34.249 INFO:teuthology.orchestra.run.smithi131.stdout:Summary : Support tools for GRUB. 2024-05-26T22:36:34.249 INFO:teuthology.orchestra.run.smithi131.stdout:Description : 2024-05-26T22:36:34.250 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:36:34.250 INFO:teuthology.orchestra.run.smithi131.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-26T22:36:34.250 INFO:teuthology.orchestra.run.smithi131.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-26T22:36:34.250 INFO:teuthology.orchestra.run.smithi131.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-26T22:36:34.250 INFO:teuthology.orchestra.run.smithi131.stdout:hardware devices. 2024-05-26T22:36:34.250 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:36:34.250 INFO:teuthology.orchestra.run.smithi131.stdout:This subpackage provides tools for support of all platforms. 2024-05-26T22:36:34.251 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-26T22:36:34.251 INFO:teuthology.task.kernel:Updating grub on smithi131 to boot 6.9.0-gb9459a26b088 2024-05-26T22:36:34.251 DEBUG:teuthology.orchestra.run.smithi131:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-26T22:36:34.788 INFO:teuthology.orchestra.run.smithi131.stderr:Generating grub configuration file ... 2024-05-26T22:36:36.185 INFO:teuthology.orchestra.run.smithi131.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-26T22:36:36.227 INFO:teuthology.orchestra.run.smithi131.stderr:done 2024-05-26T22:36:36.229 DEBUG:teuthology.orchestra.run.smithi131:> mktemp 2024-05-26T22:36:36.260 INFO:teuthology.orchestra.run.smithi131.stdout:/tmp/tmp.uR9upBlcDI 2024-05-26T22:36:36.260 DEBUG:teuthology.orchestra.run.smithi131:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.uR9upBlcDI 2024-05-26T22:36:36.324 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod 0666 /tmp/tmp.uR9upBlcDI 2024-05-26T22:36:36.455 DEBUG:teuthology.orchestra.remote:smithi131:/tmp/tmp.uR9upBlcDI is 6KB 2024-05-26T22:36:36.505 DEBUG:teuthology.orchestra.run.smithi131:> rm -fr /tmp/tmp.uR9upBlcDI 2024-05-26T22:36:36.519 DEBUG:teuthology.orchestra.run.smithi131:> sudo /bin/ls /boot/loader/entries || true 2024-05-26T22:36:36.580 INFO:teuthology.orchestra.run.smithi131.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-26T22:36:36.580 INFO:teuthology.orchestra.run.smithi131.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-26T22:36:36.580 INFO:teuthology.orchestra.run.smithi131.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-26T22:36:36.581 INFO:teuthology.orchestra.run.smithi131.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-gb9459a26b088.conf 2024-05-26T22:36:36.582 DEBUG:teuthology.orchestra.run.smithi131:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-gb9459a26b088 2024-05-26T22:36:36.722 DEBUG:teuthology.orchestra.run.smithi131:> sudo shutdown -r now 2024-05-26T22:36:48.576 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2024-05-26T22:36:48.577 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi187.front.sepia.ceph.com' 2024-05-26T22:36:48.577 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi187.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-26T22:37:06.733 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-05-26T22:37:06.734 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi131.front.sepia.ceph.com' 2024-05-26T22:37:06.734 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-26T22:37:48.619 DEBUG:teuthology.orchestra.remote:timed out 2024-05-26T22:37:57.625 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi187.front.sepia.ceph.com' 2024-05-26T22:37:57.627 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi187.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-26T22:38:05.327 DEBUG:teuthology.orchestra.run.smithi187:> true 2024-05-26T22:38:06.091 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi187.front.sepia.ceph.com' 2024-05-26T22:38:06.091 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "6.9.0-gb9459a26b088"... 2024-05-26T22:38:06.091 DEBUG:teuthology.orchestra.run.smithi187:> uname -r 2024-05-26T22:38:06.132 INFO:teuthology.orchestra.run.smithi187.stdout:6.9.0-gb9459a26b088 2024-05-26T22:38:06.133 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-gb9459a26b088 vs 6.9.0-gb9459a26b088 2024-05-26T22:38:06.133 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-26T22:38:06.133 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-26T22:38:06.736 DEBUG:teuthology.orchestra.remote:timed out 2024-05-26T22:38:07.137 DEBUG:teuthology.parallel:result is None 2024-05-26T22:38:15.738 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi131.front.sepia.ceph.com' 2024-05-26T22:38:15.738 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-26T22:38:17.142 DEBUG:teuthology.orchestra.run.smithi131:> true 2024-05-26T22:38:17.726 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi131.front.sepia.ceph.com' 2024-05-26T22:38:17.726 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.9.0-gb9459a26b088"... 2024-05-26T22:38:17.727 DEBUG:teuthology.orchestra.run.smithi131:> uname -r 2024-05-26T22:38:17.767 INFO:teuthology.orchestra.run.smithi131.stdout:6.9.0-gb9459a26b088 2024-05-26T22:38:17.767 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-gb9459a26b088 vs 6.9.0-gb9459a26b088 2024-05-26T22:38:17.767 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-26T22:38:17.767 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-26T22:38:18.771 DEBUG:teuthology.parallel:result is None 2024-05-26T22:38:18.771 INFO:teuthology.run_tasks:Running task internal.base... 2024-05-26T22:38:18.778 INFO:teuthology.task.internal:Creating test directory... 2024-05-26T22:38:18.779 DEBUG:teuthology.orchestra.run.smithi131:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-26T22:38:18.784 DEBUG:teuthology.orchestra.run.smithi187:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-26T22:38:18.839 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-05-26T22:38:18.846 INFO:teuthology.run_tasks:Running task internal.archive... 2024-05-26T22:38:18.874 INFO:teuthology.task.internal:Creating archive directory... 2024-05-26T22:38:18.875 DEBUG:teuthology.orchestra.run.smithi131:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-26T22:38:18.880 DEBUG:teuthology.orchestra.run.smithi187:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-26T22:38:18.940 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-05-26T22:38:18.948 INFO:teuthology.task.internal:Enabling coredump saving... 2024-05-26T22:38:18.949 DEBUG:teuthology.orchestra.run.smithi131:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-26T22:38:18.954 DEBUG:teuthology.orchestra.run.smithi187:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-26T22:38:19.088 INFO:teuthology.orchestra.run.smithi131.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-26T22:38:19.100 INFO:teuthology.orchestra.run.smithi187.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-26T22:38:19.142 INFO:teuthology.orchestra.run.smithi131.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-26T22:38:19.392 INFO:teuthology.orchestra.run.smithi187.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-26T22:38:19.397 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-05-26T22:38:19.405 INFO:teuthology.task.internal:Configuring sudo... 2024-05-26T22:38:19.405 DEBUG:teuthology.orchestra.run.smithi131:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-26T22:38:19.411 DEBUG:teuthology.orchestra.run.smithi187:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-26T22:38:19.485 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-05-26T22:38:19.495 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-05-26T22:38:19.496 DEBUG:teuthology.orchestra.run.smithi131:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-26T22:38:19.500 DEBUG:teuthology.orchestra.run.smithi187:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-26T22:38:19.543 DEBUG:teuthology.orchestra.run.smithi131:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-26T22:38:19.645 DEBUG:teuthology.orchestra.run.smithi131:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-26T22:38:19.725 DEBUG:teuthology.orchestra.run.smithi131:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-26T22:38:19.786 DEBUG:teuthology.orchestra.run.smithi131:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-26T22:38:19.878 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-26T22:38:19.879 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-26T22:38:19.969 DEBUG:teuthology.orchestra.run.smithi187:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-26T22:38:20.079 DEBUG:teuthology.orchestra.run.smithi187:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-26T22:38:20.216 DEBUG:teuthology.orchestra.run.smithi187:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-26T22:38:20.328 DEBUG:teuthology.orchestra.run.smithi187:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-26T22:38:20.457 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-05-26T22:38:20.457 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-26T22:38:20.573 DEBUG:teuthology.orchestra.run.smithi131:> sudo service rsyslog restart 2024-05-26T22:38:20.578 DEBUG:teuthology.orchestra.run.smithi187:> sudo service rsyslog restart 2024-05-26T22:38:20.682 INFO:teuthology.orchestra.run.smithi131.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-26T22:38:20.800 INFO:teuthology.orchestra.run.smithi187.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-26T22:38:21.149 INFO:teuthology.run_tasks:Running task internal.timer... 2024-05-26T22:38:21.165 INFO:teuthology.task.internal:Starting timer... 2024-05-26T22:38:21.165 INFO:teuthology.run_tasks:Running task pcp... 2024-05-26T22:38:21.285 INFO:teuthology.run_tasks:Running task selinux... 2024-05-26T22:38:21.362 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-05-26T22:38:21.363 DEBUG:teuthology.orchestra.run.smithi131:> sudo service auditd rotate 2024-05-26T22:38:21.492 INFO:teuthology.orchestra.run.smithi131.stdout:Rotating logs: 2024-05-26T22:38:21.495 DEBUG:teuthology.orchestra.run.smithi187:> sudo service auditd rotate 2024-05-26T22:38:21.677 INFO:teuthology.orchestra.run.smithi187.stdout:Rotating logs: 2024-05-26T22:38:21.682 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-05-26T22:38:21.683 DEBUG:teuthology.orchestra.run.smithi131:> /usr/sbin/getenforce 2024-05-26T22:38:21.736 INFO:teuthology.orchestra.run.smithi131.stdout:Permissive 2024-05-26T22:38:21.736 DEBUG:teuthology.orchestra.run.smithi187:> /usr/sbin/getenforce 2024-05-26T22:38:21.820 INFO:teuthology.orchestra.run.smithi187.stdout:Permissive 2024-05-26T22:38:21.821 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi131.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi187.front.sepia.ceph.com': 'permissive'} 2024-05-26T22:38:21.821 DEBUG:teuthology.orchestra.run.smithi131:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-26T22:38:21.895 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-26T22:38:21.895 DEBUG:teuthology.orchestra.run.smithi187:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-26T22:38:21.954 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-26T22:38:21.954 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-05-26T22:38:21.955 DEBUG:teuthology.orchestra.run.smithi131:> sudo /usr/sbin/setenforce permissive 2024-05-26T22:38:22.089 DEBUG:teuthology.orchestra.run.smithi187:> sudo /usr/sbin/setenforce permissive 2024-05-26T22:38:22.164 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-05-26T22:38:22.175 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-05-26T22:38:22.178 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-05-26T22:38:22.294 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-05-26T22:38:22.304 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-05-26T22:38:22.305 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi131.front.sepia.ceph.com,smithi187.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-05-26T22:44:56.742 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi131.front.sepia.ceph.com'), Remote(name='ubuntu@smithi187.front.sepia.ceph.com')] 2024-05-26T22:44:56.743 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi131.front.sepia.ceph.com' 2024-05-26T22:44:56.744 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-26T22:44:56.849 DEBUG:teuthology.orchestra.run.smithi131:> true 2024-05-26T22:44:56.999 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi131.front.sepia.ceph.com' 2024-05-26T22:44:56.999 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi187.front.sepia.ceph.com' 2024-05-26T22:44:57.000 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi187.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-26T22:44:57.101 DEBUG:teuthology.orchestra.run.smithi187:> true 2024-05-26T22:44:57.258 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi187.front.sepia.ceph.com' 2024-05-26T22:44:57.258 INFO:teuthology.run_tasks:Running task clock... 2024-05-26T22:44:57.268 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-05-26T22:44:57.268 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-26T22:44:57.268 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-26T22:44:57.276 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-26T22:44:57.276 DEBUG:teuthology.orchestra.run.smithi187:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-26T22:44:57.363 INFO:teuthology.orchestra.run.smithi187.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-26T22:44:57.366 INFO:teuthology.orchestra.run.smithi131.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-26T22:44:57.402 INFO:teuthology.orchestra.run.smithi131.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-26T22:44:57.408 INFO:teuthology.orchestra.run.smithi187.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-26T22:44:57.475 INFO:teuthology.orchestra.run.smithi131.stderr:sudo: ntpd: command not found 2024-05-26T22:44:57.481 INFO:teuthology.orchestra.run.smithi187.stderr:sudo: ntpd: command not found 2024-05-26T22:44:57.505 INFO:teuthology.orchestra.run.smithi131.stdout:506 Cannot talk to daemon 2024-05-26T22:44:57.513 INFO:teuthology.orchestra.run.smithi187.stdout:506 Cannot talk to daemon 2024-05-26T22:44:57.542 INFO:teuthology.orchestra.run.smithi131.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-26T22:44:57.556 INFO:teuthology.orchestra.run.smithi187.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-26T22:44:57.581 INFO:teuthology.orchestra.run.smithi131.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-26T22:44:57.598 INFO:teuthology.orchestra.run.smithi187.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-26T22:44:57.714 INFO:teuthology.orchestra.run.smithi131.stderr:bash: line 1: ntpq: command not found 2024-05-26T22:44:57.721 INFO:teuthology.orchestra.run.smithi131.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-26T22:44:57.722 INFO:teuthology.orchestra.run.smithi131.stdout:=============================================================================== 2024-05-26T22:44:57.722 INFO:teuthology.orchestra.run.smithi131.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-26T22:44:57.722 INFO:teuthology.orchestra.run.smithi131.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-26T22:44:57.722 INFO:teuthology.orchestra.run.smithi131.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-26T22:44:57.722 INFO:teuthology.orchestra.run.smithi131.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-26T22:44:57.749 INFO:teuthology.orchestra.run.smithi187.stderr:bash: line 1: ntpq: command not found 2024-05-26T22:44:57.759 INFO:teuthology.orchestra.run.smithi187.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-26T22:44:57.759 INFO:teuthology.orchestra.run.smithi187.stdout:=============================================================================== 2024-05-26T22:44:57.759 INFO:teuthology.orchestra.run.smithi187.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-26T22:44:57.759 INFO:teuthology.orchestra.run.smithi187.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-26T22:44:57.759 INFO:teuthology.orchestra.run.smithi187.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-26T22:44:57.759 INFO:teuthology.orchestra.run.smithi187.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-26T22:44:57.761 INFO:teuthology.run_tasks:Running task install... 2024-05-26T22:44:57.769 DEBUG:teuthology.task.install:project ceph 2024-05-26T22:44:57.769 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '0ef46a246e1118e5adafa1857b4189e7d780c816'}} 2024-05-26T22:44:57.770 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '0ef46a246e1118e5adafa1857b4189e7d780c816'} 2024-05-26T22:44:57.770 INFO:teuthology.task.install:Using flavor: default 2024-05-26T22:44:57.775 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-05-26T22:44:57.775 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-05-26T22:44:57.776 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '0ef46a246e1118e5adafa1857b4189e7d780c816', 'tag': None, 'wait_for_package': False} 2024-05-26T22:44:57.776 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=0ef46a246e1118e5adafa1857b4189e7d780c816 2024-05-26T22:44:57.778 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '0ef46a246e1118e5adafa1857b4189e7d780c816', 'tag': None, 'wait_for_package': False} 2024-05-26T22:44:57.779 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=0ef46a246e1118e5adafa1857b4189e7d780c816 2024-05-26T22:44:57.925 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/squid/0ef46a246e1118e5adafa1857b4189e7d780c816/centos/9/flavors/default/ 2024-05-26T22:44:57.925 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2857.g0ef46a24 2024-05-26T22:44:57.929 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/squid/0ef46a246e1118e5adafa1857b4189e7d780c816/centos/9/flavors/default/ 2024-05-26T22:44:57.929 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2857.g0ef46a24 2024-05-26T22:44:58.097 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/squid/0ef46a246e1118e5adafa1857b4189e7d780c816/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/0ef46a246e1118e5adafa1857b4189e7d780c816/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/0ef46a246e1118e5adafa1857b4189e7d780c816/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-26T22:44:58.097 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-26T22:44:58.097 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-26T22:44:58.104 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/squid/0ef46a246e1118e5adafa1857b4189e7d780c816/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/0ef46a246e1118e5adafa1857b4189e7d780c816/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/0ef46a246e1118e5adafa1857b4189e7d780c816/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-26T22:44:58.104 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-05-26T22:44:58.104 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-26T22:44:58.186 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-26T22:44:58.186 DEBUG:teuthology.orchestra.run.smithi131:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/0ef46a246e1118e5adafa1857b4189e7d780c816/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-26T22:44:58.195 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-26T22:44:58.195 DEBUG:teuthology.orchestra.run.smithi187:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/0ef46a246e1118e5adafa1857b4189e7d780c816/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-26T22:44:58.312 DEBUG:teuthology.orchestra.run.smithi131:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-26T22:44:58.323 DEBUG:teuthology.orchestra.run.smithi187:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-26T22:44:58.423 DEBUG:teuthology.orchestra.run.smithi131:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-26T22:44:58.432 DEBUG:teuthology.orchestra.run.smithi187:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-26T22:44:58.498 INFO:teuthology.orchestra.run.smithi131.stdout:check_obsoletes = 1 2024-05-26T22:44:58.502 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum clean all 2024-05-26T22:44:58.505 INFO:teuthology.orchestra.run.smithi187.stdout:check_obsoletes = 1 2024-05-26T22:44:58.509 DEBUG:teuthology.orchestra.run.smithi187:> sudo yum clean all 2024-05-26T22:44:58.814 INFO:teuthology.orchestra.run.smithi131.stdout:45 files removed 2024-05-26T22:44:58.816 INFO:teuthology.orchestra.run.smithi187.stdout:45 files removed 2024-05-26T22:44:58.853 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-26T22:44:58.855 DEBUG:teuthology.orchestra.run.smithi187:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-26T22:44:59.717 INFO:teuthology.orchestra.run.smithi187.stdout:ceph packages for x86_64 196 kB/s | 83 kB 00:00 2024-05-26T22:44:59.748 INFO:teuthology.orchestra.run.smithi131.stdout:ceph packages for x86_64 182 kB/s | 83 kB 00:00 2024-05-26T22:45:00.201 INFO:teuthology.orchestra.run.smithi187.stdout:ceph noarch packages 30 kB/s | 11 kB 00:00 2024-05-26T22:45:00.232 INFO:teuthology.orchestra.run.smithi131.stdout:ceph noarch packages 30 kB/s | 11 kB 00:00 2024-05-26T22:45:00.675 INFO:teuthology.orchestra.run.smithi131.stdout:ceph source packages 5.3 kB/s | 1.9 kB 00:00 2024-05-26T22:45:00.742 INFO:teuthology.orchestra.run.smithi187.stdout:ceph source packages 4.1 kB/s | 1.9 kB 00:00 2024-05-26T22:45:01.323 INFO:teuthology.orchestra.run.smithi131.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.1 MB 00:00 2024-05-26T22:45:01.483 INFO:teuthology.orchestra.run.smithi187.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.1 MB 00:00 2024-05-26T22:45:03.809 INFO:teuthology.orchestra.run.smithi187.stdout:CentOS Stream 9 - AppStream 17 MB/s | 19 MB 00:01 2024-05-26T22:45:09.918 INFO:teuthology.orchestra.run.smithi187.stdout:CentOS Stream 9 - CRB 2.8 MB/s | 6.2 MB 00:02 2024-05-26T22:45:11.965 INFO:teuthology.orchestra.run.smithi131.stdout:CentOS Stream 9 - AppStream 2.0 MB/s | 19 MB 00:09 2024-05-26T22:45:12.602 INFO:teuthology.orchestra.run.smithi187.stdout:CentOS Stream 9 - Extras packages 13 kB/s | 17 kB 00:01 2024-05-26T22:45:16.383 INFO:teuthology.orchestra.run.smithi131.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.2 MB 00:00 2024-05-26T22:45:17.925 INFO:teuthology.orchestra.run.smithi131.stdout:CentOS Stream 9 - Extras packages 79 kB/s | 17 kB 00:00 2024-05-26T22:45:25.394 INFO:teuthology.orchestra.run.smithi187.stdout:Extra Packages for Enterprise Linux 1.7 MB/s | 21 MB 00:12 2024-05-26T22:45:31.468 INFO:teuthology.orchestra.run.smithi187.stdout:lab-extras 25 kB/s | 1.7 kB 00:00 2024-05-26T22:45:32.517 INFO:teuthology.orchestra.run.smithi131.stdout:Extra Packages for Enterprise Linux 1.5 MB/s | 21 MB 00:14 2024-05-26T22:45:33.898 INFO:teuthology.orchestra.run.smithi187.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-26T22:45:33.900 INFO:teuthology.orchestra.run.smithi187.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-26T22:45:33.921 INFO:teuthology.orchestra.run.smithi187.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-26T22:45:33.922 INFO:teuthology.orchestra.run.smithi187.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-26T22:45:33.923 INFO:teuthology.orchestra.run.smithi187.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-26T22:45:33.925 INFO:teuthology.orchestra.run.smithi187.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-26T22:45:33.927 INFO:teuthology.orchestra.run.smithi187.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-26T22:45:33.930 INFO:teuthology.orchestra.run.smithi187.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-26T22:45:33.932 INFO:teuthology.orchestra.run.smithi187.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-26T22:45:34.004 INFO:teuthology.orchestra.run.smithi187.stdout:Dependencies resolved. 2024-05-26T22:45:34.028 INFO:teuthology.orchestra.run.smithi187.stdout:======================================================================================= 2024-05-26T22:45:34.028 INFO:teuthology.orchestra.run.smithi187.stdout: Package Arch Version Repository Size 2024-05-26T22:45:34.029 INFO:teuthology.orchestra.run.smithi187.stdout:======================================================================================= 2024-05-26T22:45:34.029 INFO:teuthology.orchestra.run.smithi187.stdout:Installing: 2024-05-26T22:45:34.029 INFO:teuthology.orchestra.run.smithi187.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-26T22:45:34.029 INFO:teuthology.orchestra.run.smithi187.stdout: ceph x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 6.6 k 2024-05-26T22:45:34.029 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-base x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 5.4 M 2024-05-26T22:45:34.029 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-fuse x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 890 k 2024-05-26T22:45:34.029 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 145 k 2024-05-26T22:45:34.029 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 1.5 M 2024-05-26T22:45:34.029 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2857.g0ef46a24.el9 ceph-noarch 142 k 2024-05-26T22:45:34.029 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2857.g0ef46a24.el9 ceph-noarch 3.5 M 2024-05-26T22:45:34.029 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2857.g0ef46a24.el9 ceph-noarch 7.4 M 2024-05-26T22:45:34.029 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-rook noarch 2:19.0.0-2857.g0ef46a24.el9 ceph-noarch 49 k 2024-05-26T22:45:34.029 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-radosgw x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 10 M 2024-05-26T22:45:34.030 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-test x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 48 M 2024-05-26T22:45:34.030 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-volume noarch 2:19.0.0-2857.g0ef46a24.el9 ceph-noarch 264 k 2024-05-26T22:45:34.030 INFO:teuthology.orchestra.run.smithi187.stdout: cephadm noarch 2:19.0.0-2857.g0ef46a24.el9 ceph-noarch 762 k 2024-05-26T22:45:34.030 INFO:teuthology.orchestra.run.smithi187.stdout: cephfs-mirror x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 221 k 2024-05-26T22:45:34.030 INFO:teuthology.orchestra.run.smithi187.stdout: cephfs-shell x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 27 k 2024-05-26T22:45:34.030 INFO:teuthology.orchestra.run.smithi187.stdout: cephfs-top noarch 2:19.0.0-2857.g0ef46a24.el9 ceph-noarch 18 k 2024-05-26T22:45:34.030 INFO:teuthology.orchestra.run.smithi187.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-05-26T22:45:34.033 INFO:teuthology.orchestra.run.smithi187.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-26T22:45:34.033 INFO:teuthology.orchestra.run.smithi187.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-26T22:45:34.033 INFO:teuthology.orchestra.run.smithi187.stdout: libcephfs-devel x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 32 k 2024-05-26T22:45:34.033 INFO:teuthology.orchestra.run.smithi187.stdout: libcephfs2 x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 746 k 2024-05-26T22:45:34.033 INFO:teuthology.orchestra.run.smithi187.stdout: librados-devel x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 127 k 2024-05-26T22:45:34.033 INFO:teuthology.orchestra.run.smithi187.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-26T22:45:34.034 INFO:teuthology.orchestra.run.smithi187.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-26T22:45:34.034 INFO:teuthology.orchestra.run.smithi187.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-26T22:45:34.034 INFO:teuthology.orchestra.run.smithi187.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-26T22:45:34.034 INFO:teuthology.orchestra.run.smithi187.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-26T22:45:34.034 INFO:teuthology.orchestra.run.smithi187.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-26T22:45:34.034 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cephfs x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 162 k 2024-05-26T22:45:34.034 INFO:teuthology.orchestra.run.smithi187.stdout: python3-rados x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 321 k 2024-05-26T22:45:34.034 INFO:teuthology.orchestra.run.smithi187.stdout: python3-rbd x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 300 k 2024-05-26T22:45:34.034 INFO:teuthology.orchestra.run.smithi187.stdout: python3-rgw x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 99 k 2024-05-26T22:45:34.034 INFO:teuthology.orchestra.run.smithi187.stdout: rbd-fuse x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 85 k 2024-05-26T22:45:34.034 INFO:teuthology.orchestra.run.smithi187.stdout: rbd-mirror x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 3.1 M 2024-05-26T22:45:34.035 INFO:teuthology.orchestra.run.smithi187.stdout: rbd-nbd x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 170 k 2024-05-26T22:45:34.035 INFO:teuthology.orchestra.run.smithi187.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-26T22:45:34.035 INFO:teuthology.orchestra.run.smithi187.stdout:Upgrading: 2024-05-26T22:45:34.035 INFO:teuthology.orchestra.run.smithi187.stdout: NetworkManager x86_64 1:1.47.90-1.el9 baseos 2.3 M 2024-05-26T22:45:34.035 INFO:teuthology.orchestra.run.smithi187.stdout: NetworkManager-libnm x86_64 1:1.47.90-1.el9 baseos 1.8 M 2024-05-26T22:45:34.035 INFO:teuthology.orchestra.run.smithi187.stdout: NetworkManager-team x86_64 1:1.47.90-1.el9 baseos 41 k 2024-05-26T22:45:34.035 INFO:teuthology.orchestra.run.smithi187.stdout: NetworkManager-tui x86_64 1:1.47.90-1.el9 baseos 248 k 2024-05-26T22:45:34.035 INFO:teuthology.orchestra.run.smithi187.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-05-26T22:45:34.035 INFO:teuthology.orchestra.run.smithi187.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-05-26T22:45:34.036 INFO:teuthology.orchestra.run.smithi187.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-05-26T22:45:34.036 INFO:teuthology.orchestra.run.smithi187.stdout: librados2 x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 3.4 M 2024-05-26T22:45:34.036 INFO:teuthology.orchestra.run.smithi187.stdout: librbd1 x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 3.1 M 2024-05-26T22:45:34.036 INFO:teuthology.orchestra.run.smithi187.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-26T22:45:34.036 INFO:teuthology.orchestra.run.smithi187.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-26T22:45:34.036 INFO:teuthology.orchestra.run.smithi187.stdout:Installing dependencies: 2024-05-26T22:45:34.036 INFO:teuthology.orchestra.run.smithi187.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-26T22:45:34.036 INFO:teuthology.orchestra.run.smithi187.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-26T22:45:34.036 INFO:teuthology.orchestra.run.smithi187.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-26T22:45:34.036 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-common x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 21 M 2024-05-26T22:45:34.036 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2857.g0ef46a24.el9 ceph-noarch 32 k 2024-05-26T22:45:34.036 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mds x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 2.4 M 2024-05-26T22:45:34.036 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2857.g0ef46a24.el9 ceph-noarch 249 k 2024-05-26T22:45:34.037 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mon x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 4.7 M 2024-05-26T22:45:34.037 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-osd x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 17 M 2024-05-26T22:45:34.037 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2857.g0ef46a24.el9 ceph-noarch 16 k 2024-05-26T22:45:34.037 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-selinux x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 25 k 2024-05-26T22:45:34.037 INFO:teuthology.orchestra.run.smithi187.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-26T22:45:34.037 INFO:teuthology.orchestra.run.smithi187.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-26T22:45:34.037 INFO:teuthology.orchestra.run.smithi187.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-26T22:45:34.037 INFO:teuthology.orchestra.run.smithi187.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-26T22:45:34.037 INFO:teuthology.orchestra.run.smithi187.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-26T22:45:34.037 INFO:teuthology.orchestra.run.smithi187.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-26T22:45:34.037 INFO:teuthology.orchestra.run.smithi187.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-26T22:45:34.037 INFO:teuthology.orchestra.run.smithi187.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-26T22:45:34.037 INFO:teuthology.orchestra.run.smithi187.stdout: libcephsqlite x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 163 k 2024-05-26T22:45:34.038 INFO:teuthology.orchestra.run.smithi187.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-26T22:45:34.038 INFO:teuthology.orchestra.run.smithi187.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-26T22:45:34.038 INFO:teuthology.orchestra.run.smithi187.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-26T22:45:34.038 INFO:teuthology.orchestra.run.smithi187.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-26T22:45:34.038 INFO:teuthology.orchestra.run.smithi187.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-26T22:45:34.038 INFO:teuthology.orchestra.run.smithi187.stdout: libradosstriper1 x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 501 k 2024-05-26T22:45:34.038 INFO:teuthology.orchestra.run.smithi187.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-26T22:45:34.038 INFO:teuthology.orchestra.run.smithi187.stdout: librgw2 x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 5.3 M 2024-05-26T22:45:34.038 INFO:teuthology.orchestra.run.smithi187.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-26T22:45:34.038 INFO:teuthology.orchestra.run.smithi187.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-26T22:45:34.038 INFO:teuthology.orchestra.run.smithi187.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-26T22:45:34.038 INFO:teuthology.orchestra.run.smithi187.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-26T22:45:34.039 INFO:teuthology.orchestra.run.smithi187.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-26T22:45:34.039 INFO:teuthology.orchestra.run.smithi187.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-05-26T22:45:34.039 INFO:teuthology.orchestra.run.smithi187.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-26T22:45:34.039 INFO:teuthology.orchestra.run.smithi187.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-26T22:45:34.039 INFO:teuthology.orchestra.run.smithi187.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-26T22:45:34.039 INFO:teuthology.orchestra.run.smithi187.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-26T22:45:34.039 INFO:teuthology.orchestra.run.smithi187.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-26T22:45:34.039 INFO:teuthology.orchestra.run.smithi187.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-26T22:45:34.039 INFO:teuthology.orchestra.run.smithi187.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-26T22:45:34.039 INFO:teuthology.orchestra.run.smithi187.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-26T22:45:34.039 INFO:teuthology.orchestra.run.smithi187.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-26T22:45:34.039 INFO:teuthology.orchestra.run.smithi187.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-26T22:45:34.039 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-26T22:45:34.040 INFO:teuthology.orchestra.run.smithi187.stdout: python3-ceph-argparse x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 45 k 2024-05-26T22:45:34.040 INFO:teuthology.orchestra.run.smithi187.stdout: python3-ceph-common x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 135 k 2024-05-26T22:45:34.040 INFO:teuthology.orchestra.run.smithi187.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-26T22:45:34.040 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-26T22:45:34.040 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-26T22:45:34.040 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-26T22:45:34.040 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-26T22:45:34.040 INFO:teuthology.orchestra.run.smithi187.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-05-26T22:45:34.040 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-26T22:45:34.040 INFO:teuthology.orchestra.run.smithi187.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-26T22:45:34.040 INFO:teuthology.orchestra.run.smithi187.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-26T22:45:34.040 INFO:teuthology.orchestra.run.smithi187.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-26T22:45:34.040 INFO:teuthology.orchestra.run.smithi187.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-26T22:45:34.041 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-26T22:45:34.041 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-26T22:45:34.041 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-26T22:45:34.041 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-26T22:45:34.041 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-26T22:45:34.041 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-05-26T22:45:34.041 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-26T22:45:34.041 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-26T22:45:34.041 INFO:teuthology.orchestra.run.smithi187.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-26T22:45:34.041 INFO:teuthology.orchestra.run.smithi187.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-26T22:45:34.041 INFO:teuthology.orchestra.run.smithi187.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-26T22:45:34.041 INFO:teuthology.orchestra.run.smithi187.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-26T22:45:34.041 INFO:teuthology.orchestra.run.smithi187.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-26T22:45:34.042 INFO:teuthology.orchestra.run.smithi187.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-26T22:45:34.042 INFO:teuthology.orchestra.run.smithi187.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-26T22:45:34.042 INFO:teuthology.orchestra.run.smithi187.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-26T22:45:34.042 INFO:teuthology.orchestra.run.smithi187.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-26T22:45:34.042 INFO:teuthology.orchestra.run.smithi187.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-26T22:45:34.042 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-26T22:45:34.042 INFO:teuthology.orchestra.run.smithi187.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-26T22:45:34.043 INFO:teuthology.orchestra.run.smithi187.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-26T22:45:34.043 INFO:teuthology.orchestra.run.smithi187.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-26T22:45:34.043 INFO:teuthology.orchestra.run.smithi187.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-26T22:45:34.043 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-26T22:45:34.043 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-26T22:45:34.043 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-26T22:45:34.043 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-26T22:45:34.043 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-26T22:45:34.043 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-26T22:45:34.044 INFO:teuthology.orchestra.run.smithi187.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-26T22:45:34.044 INFO:teuthology.orchestra.run.smithi187.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-26T22:45:34.044 INFO:teuthology.orchestra.run.smithi187.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-26T22:45:34.044 INFO:teuthology.orchestra.run.smithi187.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-26T22:45:34.044 INFO:teuthology.orchestra.run.smithi187.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-26T22:45:34.044 INFO:teuthology.orchestra.run.smithi187.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-26T22:45:34.044 INFO:teuthology.orchestra.run.smithi187.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-26T22:45:34.044 INFO:teuthology.orchestra.run.smithi187.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-26T22:45:34.044 INFO:teuthology.orchestra.run.smithi187.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-26T22:45:34.044 INFO:teuthology.orchestra.run.smithi187.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-26T22:45:34.044 INFO:teuthology.orchestra.run.smithi187.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-26T22:45:34.044 INFO:teuthology.orchestra.run.smithi187.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-26T22:45:34.044 INFO:teuthology.orchestra.run.smithi187.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-26T22:45:34.045 INFO:teuthology.orchestra.run.smithi187.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-26T22:45:34.045 INFO:teuthology.orchestra.run.smithi187.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-26T22:45:34.045 INFO:teuthology.orchestra.run.smithi187.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-26T22:45:34.045 INFO:teuthology.orchestra.run.smithi187.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-26T22:45:34.045 INFO:teuthology.orchestra.run.smithi187.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-26T22:45:34.045 INFO:teuthology.orchestra.run.smithi187.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-26T22:45:34.045 INFO:teuthology.orchestra.run.smithi187.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-26T22:45:34.045 INFO:teuthology.orchestra.run.smithi187.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-26T22:45:34.046 INFO:teuthology.orchestra.run.smithi187.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-26T22:45:34.046 INFO:teuthology.orchestra.run.smithi187.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-26T22:45:34.046 INFO:teuthology.orchestra.run.smithi187.stdout:Installing weak dependencies: 2024-05-26T22:45:34.046 INFO:teuthology.orchestra.run.smithi187.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-26T22:45:34.046 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-05-26T22:45:34.046 INFO:teuthology.orchestra.run.smithi187.stdout:Transaction Summary 2024-05-26T22:45:34.046 INFO:teuthology.orchestra.run.smithi187.stdout:======================================================================================= 2024-05-26T22:45:34.046 INFO:teuthology.orchestra.run.smithi187.stdout:Install 148 Packages 2024-05-26T22:45:34.046 INFO:teuthology.orchestra.run.smithi187.stdout:Upgrade 11 Packages 2024-05-26T22:45:34.046 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-05-26T22:45:34.047 INFO:teuthology.orchestra.run.smithi187.stdout:Total download size: 225 M 2024-05-26T22:45:34.047 INFO:teuthology.orchestra.run.smithi187.stdout:Downloading Packages: 2024-05-26T22:45:35.178 INFO:teuthology.orchestra.run.smithi187.stdout:(1/159): ceph-19.0.0-2857.g0ef46a24.el9.x86_64. 24 kB/s | 6.6 kB 00:00 2024-05-26T22:45:35.478 INFO:teuthology.orchestra.run.smithi187.stdout:(2/159): ceph-fuse-19.0.0-2857.g0ef46a24.el9.x8 2.9 MB/s | 890 kB 00:00 2024-05-26T22:45:35.562 INFO:teuthology.orchestra.run.smithi187.stdout:(3/159): ceph-immutable-object-cache-19.0.0-285 1.7 MB/s | 145 kB 00:00 2024-05-26T22:45:35.712 INFO:teuthology.orchestra.run.smithi187.stdout:(4/159): ceph-base-19.0.0-2857.g0ef46a24.el9.x8 6.6 MB/s | 5.4 MB 00:00 2024-05-26T22:45:35.912 INFO:teuthology.orchestra.run.smithi187.stdout:(5/159): ceph-mgr-19.0.0-2857.g0ef46a24.el9.x86 7.5 MB/s | 1.5 MB 00:00 2024-05-26T22:45:36.087 INFO:teuthology.orchestra.run.smithi187.stdout:(6/159): ceph-mds-19.0.0-2857.g0ef46a24.el9.x86 4.5 MB/s | 2.4 MB 00:00 2024-05-26T22:45:36.446 INFO:teuthology.orchestra.run.smithi187.stdout:(7/159): ceph-mon-19.0.0-2857.g0ef46a24.el9.x86 8.8 MB/s | 4.7 MB 00:00 2024-05-26T22:45:37.000 INFO:teuthology.orchestra.run.smithi187.stdout:(8/159): ceph-common-19.0.0-2857.g0ef46a24.el9. 10 MB/s | 21 MB 00:02 2024-05-26T22:45:37.063 INFO:teuthology.orchestra.run.smithi187.stdout:(9/159): ceph-selinux-19.0.0-2857.g0ef46a24.el9 409 kB/s | 25 kB 00:00 2024-05-26T22:45:37.830 INFO:teuthology.orchestra.run.smithi187.stdout:(10/159): ceph-radosgw-19.0.0-2857.g0ef46a24.el 7.4 MB/s | 10 MB 00:01 2024-05-26T22:45:37.914 INFO:teuthology.orchestra.run.smithi187.stdout:(11/159): cephfs-mirror-19.0.0-2857.g0ef46a24.e 2.6 MB/s | 221 kB 00:00 2024-05-26T22:45:37.972 INFO:teuthology.orchestra.run.smithi187.stdout:(12/159): cephfs-shell-19.0.0-2857.g0ef46a24.el 469 kB/s | 27 kB 00:00 2024-05-26T22:45:38.039 INFO:teuthology.orchestra.run.smithi187.stdout:(13/159): libcephfs-devel-19.0.0-2857.g0ef46a24 483 kB/s | 32 kB 00:00 2024-05-26T22:45:38.198 INFO:teuthology.orchestra.run.smithi187.stdout:(14/159): libcephfs2-19.0.0-2857.g0ef46a24.el9. 4.6 MB/s | 746 kB 00:00 2024-05-26T22:45:38.265 INFO:teuthology.orchestra.run.smithi187.stdout:(15/159): libcephsqlite-19.0.0-2857.g0ef46a24.e 2.4 MB/s | 163 kB 00:00 2024-05-26T22:45:38.373 INFO:teuthology.orchestra.run.smithi187.stdout:(16/159): librados-devel-19.0.0-2857.g0ef46a24. 1.2 MB/s | 127 kB 00:00 2024-05-26T22:45:38.474 INFO:teuthology.orchestra.run.smithi187.stdout:(17/159): libradosstriper1-19.0.0-2857.g0ef46a2 4.9 MB/s | 501 kB 00:00 2024-05-26T22:45:38.676 INFO:teuthology.orchestra.run.smithi131.stdout:lab-extras 26 kB/s | 1.7 kB 00:00 2024-05-26T22:45:39.091 INFO:teuthology.orchestra.run.smithi187.stdout:(18/159): ceph-osd-19.0.0-2857.g0ef46a24.el9.x8 5.6 MB/s | 17 MB 00:03 2024-05-26T22:45:39.141 INFO:teuthology.orchestra.run.smithi187.stdout:(19/159): python3-ceph-argparse-19.0.0-2857.g0e 925 kB/s | 45 kB 00:00 2024-05-26T22:45:39.200 INFO:teuthology.orchestra.run.smithi187.stdout:(20/159): python3-ceph-common-19.0.0-2857.g0ef4 2.3 MB/s | 135 kB 00:00 2024-05-26T22:45:39.317 INFO:teuthology.orchestra.run.smithi187.stdout:(21/159): librgw2-19.0.0-2857.g0ef46a24.el9.x86 6.3 MB/s | 5.3 MB 00:00 2024-05-26T22:45:39.350 INFO:teuthology.orchestra.run.smithi187.stdout:(22/159): python3-cephfs-19.0.0-2857.g0ef46a24. 1.1 MB/s | 162 kB 00:00 2024-05-26T22:45:39.576 INFO:teuthology.orchestra.run.smithi187.stdout:(23/159): python3-rados-19.0.0-2857.g0ef46a24.e 1.2 MB/s | 321 kB 00:00 2024-05-26T22:45:39.609 INFO:teuthology.orchestra.run.smithi187.stdout:(24/159): python3-rbd-19.0.0-2857.g0ef46a24.el9 1.1 MB/s | 300 kB 00:00 2024-05-26T22:45:39.643 INFO:teuthology.orchestra.run.smithi187.stdout:(25/159): python3-rgw-19.0.0-2857.g0ef46a24.el9 1.5 MB/s | 99 kB 00:00 2024-05-26T22:45:39.669 INFO:teuthology.orchestra.run.smithi187.stdout:(26/159): rbd-fuse-19.0.0-2857.g0ef46a24.el9.x8 1.4 MB/s | 85 kB 00:00 2024-05-26T22:45:39.727 INFO:teuthology.orchestra.run.smithi187.stdout:(27/159): rbd-nbd-19.0.0-2857.g0ef46a24.el9.x86 2.9 MB/s | 170 kB 00:00 2024-05-26T22:45:39.794 INFO:teuthology.orchestra.run.smithi187.stdout:(28/159): ceph-grafana-dashboards-19.0.0-2857.g 483 kB/s | 32 kB 00:00 2024-05-26T22:45:39.877 INFO:teuthology.orchestra.run.smithi187.stdout:(29/159): ceph-mgr-cephadm-19.0.0-2857.g0ef46a2 1.7 MB/s | 142 kB 00:00 2024-05-26T22:45:40.019 INFO:teuthology.orchestra.run.smithi187.stdout:(30/159): rbd-mirror-19.0.0-2857.g0ef46a24.el9. 8.2 MB/s | 3.1 MB 00:00 2024-05-26T22:45:41.028 INFO:teuthology.orchestra.run.smithi187.stdout:(31/159): ceph-test-19.0.0-2857.g0ef46a24.el9.x 12 MB/s | 48 MB 00:03 2024-05-26T22:45:41.107 INFO:teuthology.orchestra.run.smithi131.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-26T22:45:41.109 INFO:teuthology.orchestra.run.smithi131.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-26T22:45:41.130 INFO:teuthology.orchestra.run.smithi131.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-26T22:45:41.131 INFO:teuthology.orchestra.run.smithi131.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-26T22:45:41.133 INFO:teuthology.orchestra.run.smithi131.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-26T22:45:41.133 INFO:teuthology.orchestra.run.smithi131.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-26T22:45:41.136 INFO:teuthology.orchestra.run.smithi131.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-26T22:45:41.140 INFO:teuthology.orchestra.run.smithi131.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-26T22:45:41.141 INFO:teuthology.orchestra.run.smithi131.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-26T22:45:41.161 INFO:teuthology.orchestra.run.smithi187.stdout:(32/159): ceph-mgr-dashboard-19.0.0-2857.g0ef46 2.7 MB/s | 3.5 MB 00:01 2024-05-26T22:45:41.195 INFO:teuthology.orchestra.run.smithi187.stdout:(33/159): ceph-mgr-modules-core-19.0.0-2857.g0e 1.5 MB/s | 249 kB 00:00 2024-05-26T22:45:41.215 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-05-26T22:45:41.236 INFO:teuthology.orchestra.run.smithi131.stdout:======================================================================================= 2024-05-26T22:45:41.237 INFO:teuthology.orchestra.run.smithi131.stdout: Package Arch Version Repository Size 2024-05-26T22:45:41.237 INFO:teuthology.orchestra.run.smithi131.stdout:======================================================================================= 2024-05-26T22:45:41.237 INFO:teuthology.orchestra.run.smithi131.stdout:Installing: 2024-05-26T22:45:41.237 INFO:teuthology.orchestra.run.smithi131.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-26T22:45:41.237 INFO:teuthology.orchestra.run.smithi131.stdout: ceph x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 6.6 k 2024-05-26T22:45:41.237 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-base x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 5.4 M 2024-05-26T22:45:41.237 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-fuse x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 890 k 2024-05-26T22:45:41.237 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 145 k 2024-05-26T22:45:41.237 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 1.5 M 2024-05-26T22:45:41.237 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2857.g0ef46a24.el9 ceph-noarch 142 k 2024-05-26T22:45:41.237 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2857.g0ef46a24.el9 ceph-noarch 3.5 M 2024-05-26T22:45:41.238 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2857.g0ef46a24.el9 ceph-noarch 7.4 M 2024-05-26T22:45:41.238 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-rook noarch 2:19.0.0-2857.g0ef46a24.el9 ceph-noarch 49 k 2024-05-26T22:45:41.238 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-radosgw x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 10 M 2024-05-26T22:45:41.238 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-test x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 48 M 2024-05-26T22:45:41.238 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-volume noarch 2:19.0.0-2857.g0ef46a24.el9 ceph-noarch 264 k 2024-05-26T22:45:41.238 INFO:teuthology.orchestra.run.smithi131.stdout: cephadm noarch 2:19.0.0-2857.g0ef46a24.el9 ceph-noarch 762 k 2024-05-26T22:45:41.238 INFO:teuthology.orchestra.run.smithi131.stdout: cephfs-mirror x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 221 k 2024-05-26T22:45:41.238 INFO:teuthology.orchestra.run.smithi131.stdout: cephfs-shell x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 27 k 2024-05-26T22:45:41.238 INFO:teuthology.orchestra.run.smithi131.stdout: cephfs-top noarch 2:19.0.0-2857.g0ef46a24.el9 ceph-noarch 18 k 2024-05-26T22:45:41.238 INFO:teuthology.orchestra.run.smithi131.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-05-26T22:45:41.238 INFO:teuthology.orchestra.run.smithi131.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-26T22:45:41.238 INFO:teuthology.orchestra.run.smithi131.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-26T22:45:41.239 INFO:teuthology.orchestra.run.smithi131.stdout: libcephfs-devel x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 32 k 2024-05-26T22:45:41.239 INFO:teuthology.orchestra.run.smithi131.stdout: libcephfs2 x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 746 k 2024-05-26T22:45:41.239 INFO:teuthology.orchestra.run.smithi131.stdout: librados-devel x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 127 k 2024-05-26T22:45:41.239 INFO:teuthology.orchestra.run.smithi131.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-26T22:45:41.239 INFO:teuthology.orchestra.run.smithi131.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-26T22:45:41.239 INFO:teuthology.orchestra.run.smithi131.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-26T22:45:41.239 INFO:teuthology.orchestra.run.smithi131.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-26T22:45:41.239 INFO:teuthology.orchestra.run.smithi131.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-26T22:45:41.239 INFO:teuthology.orchestra.run.smithi131.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-26T22:45:41.239 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cephfs x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 162 k 2024-05-26T22:45:41.239 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rados x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 321 k 2024-05-26T22:45:41.239 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rbd x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 300 k 2024-05-26T22:45:41.239 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rgw x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 99 k 2024-05-26T22:45:41.239 INFO:teuthology.orchestra.run.smithi131.stdout: rbd-fuse x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 85 k 2024-05-26T22:45:41.240 INFO:teuthology.orchestra.run.smithi131.stdout: rbd-mirror x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 3.1 M 2024-05-26T22:45:41.240 INFO:teuthology.orchestra.run.smithi131.stdout: rbd-nbd x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 170 k 2024-05-26T22:45:41.240 INFO:teuthology.orchestra.run.smithi131.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-26T22:45:41.240 INFO:teuthology.orchestra.run.smithi131.stdout:Upgrading: 2024-05-26T22:45:41.240 INFO:teuthology.orchestra.run.smithi131.stdout: NetworkManager x86_64 1:1.47.90-1.el9 baseos 2.3 M 2024-05-26T22:45:41.240 INFO:teuthology.orchestra.run.smithi131.stdout: NetworkManager-libnm x86_64 1:1.47.90-1.el9 baseos 1.8 M 2024-05-26T22:45:41.240 INFO:teuthology.orchestra.run.smithi131.stdout: NetworkManager-team x86_64 1:1.47.90-1.el9 baseos 41 k 2024-05-26T22:45:41.240 INFO:teuthology.orchestra.run.smithi131.stdout: NetworkManager-tui x86_64 1:1.47.90-1.el9 baseos 248 k 2024-05-26T22:45:41.240 INFO:teuthology.orchestra.run.smithi131.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-05-26T22:45:41.240 INFO:teuthology.orchestra.run.smithi131.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-05-26T22:45:41.240 INFO:teuthology.orchestra.run.smithi131.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-05-26T22:45:41.240 INFO:teuthology.orchestra.run.smithi131.stdout: librados2 x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 3.4 M 2024-05-26T22:45:41.240 INFO:teuthology.orchestra.run.smithi131.stdout: librbd1 x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 3.1 M 2024-05-26T22:45:41.241 INFO:teuthology.orchestra.run.smithi131.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-26T22:45:41.241 INFO:teuthology.orchestra.run.smithi131.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-26T22:45:41.241 INFO:teuthology.orchestra.run.smithi131.stdout:Installing dependencies: 2024-05-26T22:45:41.241 INFO:teuthology.orchestra.run.smithi131.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-26T22:45:41.241 INFO:teuthology.orchestra.run.smithi131.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-26T22:45:41.241 INFO:teuthology.orchestra.run.smithi131.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-26T22:45:41.241 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-common x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 21 M 2024-05-26T22:45:41.242 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2857.g0ef46a24.el9 ceph-noarch 32 k 2024-05-26T22:45:41.242 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mds x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 2.4 M 2024-05-26T22:45:41.242 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2857.g0ef46a24.el9 ceph-noarch 249 k 2024-05-26T22:45:41.242 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mon x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 4.7 M 2024-05-26T22:45:41.242 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-osd x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 17 M 2024-05-26T22:45:41.242 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2857.g0ef46a24.el9 ceph-noarch 16 k 2024-05-26T22:45:41.242 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-selinux x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 25 k 2024-05-26T22:45:41.242 INFO:teuthology.orchestra.run.smithi131.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-26T22:45:41.242 INFO:teuthology.orchestra.run.smithi131.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-26T22:45:41.242 INFO:teuthology.orchestra.run.smithi131.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-26T22:45:41.243 INFO:teuthology.orchestra.run.smithi131.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-26T22:45:41.243 INFO:teuthology.orchestra.run.smithi131.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-26T22:45:41.243 INFO:teuthology.orchestra.run.smithi131.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-26T22:45:41.243 INFO:teuthology.orchestra.run.smithi131.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-26T22:45:41.243 INFO:teuthology.orchestra.run.smithi131.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-26T22:45:41.243 INFO:teuthology.orchestra.run.smithi131.stdout: libcephsqlite x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 163 k 2024-05-26T22:45:41.243 INFO:teuthology.orchestra.run.smithi131.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-26T22:45:41.243 INFO:teuthology.orchestra.run.smithi131.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-26T22:45:41.243 INFO:teuthology.orchestra.run.smithi131.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-26T22:45:41.243 INFO:teuthology.orchestra.run.smithi131.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-26T22:45:41.243 INFO:teuthology.orchestra.run.smithi131.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-26T22:45:41.243 INFO:teuthology.orchestra.run.smithi131.stdout: libradosstriper1 x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 501 k 2024-05-26T22:45:41.244 INFO:teuthology.orchestra.run.smithi131.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-26T22:45:41.244 INFO:teuthology.orchestra.run.smithi131.stdout: librgw2 x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 5.3 M 2024-05-26T22:45:41.244 INFO:teuthology.orchestra.run.smithi131.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-26T22:45:41.244 INFO:teuthology.orchestra.run.smithi131.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-26T22:45:41.244 INFO:teuthology.orchestra.run.smithi131.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-26T22:45:41.244 INFO:teuthology.orchestra.run.smithi131.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-26T22:45:41.244 INFO:teuthology.orchestra.run.smithi131.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-26T22:45:41.244 INFO:teuthology.orchestra.run.smithi131.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-05-26T22:45:41.244 INFO:teuthology.orchestra.run.smithi131.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-26T22:45:41.244 INFO:teuthology.orchestra.run.smithi131.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-26T22:45:41.244 INFO:teuthology.orchestra.run.smithi131.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-26T22:45:41.244 INFO:teuthology.orchestra.run.smithi131.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-26T22:45:41.245 INFO:teuthology.orchestra.run.smithi131.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-26T22:45:41.245 INFO:teuthology.orchestra.run.smithi131.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-26T22:45:41.245 INFO:teuthology.orchestra.run.smithi131.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-26T22:45:41.245 INFO:teuthology.orchestra.run.smithi131.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-26T22:45:41.245 INFO:teuthology.orchestra.run.smithi131.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-26T22:45:41.245 INFO:teuthology.orchestra.run.smithi131.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-26T22:45:41.245 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-26T22:45:41.245 INFO:teuthology.orchestra.run.smithi131.stdout: python3-ceph-argparse x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 45 k 2024-05-26T22:45:41.245 INFO:teuthology.orchestra.run.smithi131.stdout: python3-ceph-common x86_64 2:19.0.0-2857.g0ef46a24.el9 ceph 135 k 2024-05-26T22:45:41.245 INFO:teuthology.orchestra.run.smithi131.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-26T22:45:41.245 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-26T22:45:41.245 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-26T22:45:41.246 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-26T22:45:41.246 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-26T22:45:41.246 INFO:teuthology.orchestra.run.smithi131.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-05-26T22:45:41.246 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-26T22:45:41.246 INFO:teuthology.orchestra.run.smithi131.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-26T22:45:41.246 INFO:teuthology.orchestra.run.smithi131.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-26T22:45:41.246 INFO:teuthology.orchestra.run.smithi131.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-26T22:45:41.246 INFO:teuthology.orchestra.run.smithi131.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-26T22:45:41.246 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-26T22:45:41.246 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-26T22:45:41.246 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-26T22:45:41.246 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-26T22:45:41.246 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-26T22:45:41.247 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-05-26T22:45:41.247 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-26T22:45:41.247 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-26T22:45:41.247 INFO:teuthology.orchestra.run.smithi131.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-26T22:45:41.247 INFO:teuthology.orchestra.run.smithi131.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-26T22:45:41.247 INFO:teuthology.orchestra.run.smithi131.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-26T22:45:41.247 INFO:teuthology.orchestra.run.smithi131.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-26T22:45:41.247 INFO:teuthology.orchestra.run.smithi131.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-26T22:45:41.247 INFO:teuthology.orchestra.run.smithi131.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-26T22:45:41.247 INFO:teuthology.orchestra.run.smithi131.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-26T22:45:41.247 INFO:teuthology.orchestra.run.smithi131.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-26T22:45:41.247 INFO:teuthology.orchestra.run.smithi131.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-26T22:45:41.247 INFO:teuthology.orchestra.run.smithi131.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-26T22:45:41.248 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-26T22:45:41.248 INFO:teuthology.orchestra.run.smithi131.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-26T22:45:41.248 INFO:teuthology.orchestra.run.smithi131.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-26T22:45:41.248 INFO:teuthology.orchestra.run.smithi131.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-26T22:45:41.248 INFO:teuthology.orchestra.run.smithi131.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-26T22:45:41.248 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-26T22:45:41.248 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-26T22:45:41.248 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-26T22:45:41.248 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-26T22:45:41.248 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-26T22:45:41.248 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-26T22:45:41.248 INFO:teuthology.orchestra.run.smithi131.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-26T22:45:41.248 INFO:teuthology.orchestra.run.smithi131.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-26T22:45:41.248 INFO:teuthology.orchestra.run.smithi131.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-26T22:45:41.249 INFO:teuthology.orchestra.run.smithi131.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-26T22:45:41.249 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-26T22:45:41.249 INFO:teuthology.orchestra.run.smithi131.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-26T22:45:41.249 INFO:teuthology.orchestra.run.smithi131.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-26T22:45:41.249 INFO:teuthology.orchestra.run.smithi131.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-26T22:45:41.249 INFO:teuthology.orchestra.run.smithi131.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-26T22:45:41.249 INFO:teuthology.orchestra.run.smithi131.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-26T22:45:41.249 INFO:teuthology.orchestra.run.smithi131.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-26T22:45:41.249 INFO:teuthology.orchestra.run.smithi131.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-26T22:45:41.249 INFO:teuthology.orchestra.run.smithi131.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-26T22:45:41.249 INFO:teuthology.orchestra.run.smithi131.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-26T22:45:41.249 INFO:teuthology.orchestra.run.smithi131.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-26T22:45:41.250 INFO:teuthology.orchestra.run.smithi131.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-26T22:45:41.250 INFO:teuthology.orchestra.run.smithi131.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-26T22:45:41.250 INFO:teuthology.orchestra.run.smithi131.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-26T22:45:41.250 INFO:teuthology.orchestra.run.smithi131.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-26T22:45:41.250 INFO:teuthology.orchestra.run.smithi131.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-26T22:45:41.250 INFO:teuthology.orchestra.run.smithi131.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-26T22:45:41.250 INFO:teuthology.orchestra.run.smithi131.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-26T22:45:41.250 INFO:teuthology.orchestra.run.smithi131.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-26T22:45:41.250 INFO:teuthology.orchestra.run.smithi131.stdout:Installing weak dependencies: 2024-05-26T22:45:41.250 INFO:teuthology.orchestra.run.smithi131.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-26T22:45:41.250 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:45:41.250 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction Summary 2024-05-26T22:45:41.250 INFO:teuthology.orchestra.run.smithi131.stdout:======================================================================================= 2024-05-26T22:45:41.251 INFO:teuthology.orchestra.run.smithi131.stdout:Install 148 Packages 2024-05-26T22:45:41.251 INFO:teuthology.orchestra.run.smithi131.stdout:Upgrade 11 Packages 2024-05-26T22:45:41.251 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:45:41.251 INFO:teuthology.orchestra.run.smithi187.stdout:(34/159): ceph-mgr-rook-19.0.0-2857.g0ef46a24.e 663 kB/s | 49 kB 00:00 2024-05-26T22:45:41.251 INFO:teuthology.orchestra.run.smithi131.stdout:Total download size: 225 M 2024-05-26T22:45:41.252 INFO:teuthology.orchestra.run.smithi131.stdout:Downloading Packages: 2024-05-26T22:45:41.271 INFO:teuthology.orchestra.run.smithi187.stdout:(35/159): ceph-prometheus-alerts-19.0.0-2857.g0 221 kB/s | 16 kB 00:00 2024-05-26T22:45:41.413 INFO:teuthology.orchestra.run.smithi187.stdout:(36/159): ceph-mgr-diskprediction-local-19.0.0- 5.3 MB/s | 7.4 MB 00:01 2024-05-26T22:45:41.471 INFO:teuthology.orchestra.run.smithi187.stdout:(37/159): ceph-volume-19.0.0-2857.g0ef46a24.el9 1.1 MB/s | 264 kB 00:00 2024-05-26T22:45:41.505 INFO:teuthology.orchestra.run.smithi187.stdout:(38/159): cephfs-top-19.0.0-2857.g0ef46a24.el9. 200 kB/s | 18 kB 00:00 2024-05-26T22:45:41.547 INFO:teuthology.orchestra.run.smithi187.stdout:(39/159): cephadm-19.0.0-2857.g0ef46a24.el9.noa 2.7 MB/s | 762 kB 00:00 2024-05-26T22:45:41.589 INFO:teuthology.orchestra.run.smithi187.stdout:(40/159): fuse-2.9.9-15.el9.x86_64.rpm 691 kB/s | 80 kB 00:00 2024-05-26T22:45:41.681 INFO:teuthology.orchestra.run.smithi187.stdout:(41/159): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.7 MB/s | 253 kB 00:00 2024-05-26T22:45:41.706 INFO:teuthology.orchestra.run.smithi187.stdout:(42/159): libquadmath-11.4.1-3.el9.x86_64.rpm 1.2 MB/s | 195 kB 00:00 2024-05-26T22:45:41.748 INFO:teuthology.orchestra.run.smithi187.stdout:(43/159): libgfortran-11.4.1-3.el9.x86_64.rpm 3.2 MB/s | 803 kB 00:00 2024-05-26T22:45:41.807 INFO:teuthology.orchestra.run.smithi187.stdout:(44/159): python3-ply-3.11-14.el9.noarch.rpm 1.0 MB/s | 106 kB 00:00 2024-05-26T22:45:41.840 INFO:teuthology.orchestra.run.smithi187.stdout:(45/159): python3-pycparser-2.20-6.el9.noarch.r 1.5 MB/s | 135 kB 00:00 2024-05-26T22:45:41.865 INFO:teuthology.orchestra.run.smithi187.stdout:(46/159): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-05-26T22:45:41.899 INFO:teuthology.orchestra.run.smithi187.stdout:(47/159): python3-urllib3-1.26.5-5.el9.noarch.r 3.6 MB/s | 215 kB 00:00 2024-05-26T22:45:41.950 INFO:teuthology.orchestra.run.smithi187.stdout:(48/159): python3-cryptography-36.0.1-4.el9.x86 4.6 MB/s | 1.2 MB 00:00 2024-05-26T22:45:42.042 INFO:teuthology.orchestra.run.smithi187.stdout:(49/159): automake-1.16.2-8.el9.noarch.rpm 3.9 MB/s | 689 kB 00:00 2024-05-26T22:45:42.067 INFO:teuthology.orchestra.run.smithi187.stdout:(50/159): boost-program-options-1.75.0-8.el9.x8 922 kB/s | 107 kB 00:00 2024-05-26T22:45:42.084 INFO:teuthology.orchestra.run.smithi187.stdout:(51/159): elfutils-libelf-devel-0.191-4.el9.x86 530 kB/s | 22 kB 00:00 2024-05-26T22:45:42.094 INFO:teuthology.orchestra.run.smithi131.stdout:(1/159): ceph-19.0.0-2857.g0ef46a24.el9.x86_64. 14 kB/s | 6.6 kB 00:00 2024-05-26T22:45:42.134 INFO:teuthology.orchestra.run.smithi187.stdout:(52/159): flexiblas-3.0.4-8.el9.x86_64.rpm 667 kB/s | 33 kB 00:00 2024-05-26T22:45:42.168 INFO:teuthology.orchestra.run.smithi187.stdout:(53/159): bison-3.7.4-5.el9.x86_64.rpm 3.4 MB/s | 943 kB 00:00 2024-05-26T22:45:42.201 INFO:teuthology.orchestra.run.smithi187.stdout:(54/159): flex-2.6.4-9.el9.x86_64.rpm 2.3 MB/s | 313 kB 00:00 2024-05-26T22:45:42.226 INFO:teuthology.orchestra.run.smithi187.stdout:(55/159): flexiblas-openblas-openmp-3.0.4-8.el9 316 kB/s | 18 kB 00:00 2024-05-26T22:45:42.252 INFO:teuthology.orchestra.run.smithi187.stdout:(56/159): libaio-devel-0.3.111-13.el9.x86_64.rp 226 kB/s | 11 kB 00:00 2024-05-26T22:45:42.285 INFO:teuthology.orchestra.run.smithi187.stdout:(57/159): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-05-26T22:45:42.311 INFO:teuthology.orchestra.run.smithi187.stdout:(58/159): librabbitmq-0.11.0-7.el9.x86_64.rpm 782 kB/s | 45 kB 00:00 2024-05-26T22:45:42.361 INFO:teuthology.orchestra.run.smithi187.stdout:(59/159): librdkafka-1.6.1-102.el9.x86_64.rpm 8.7 MB/s | 662 kB 00:00 2024-05-26T22:45:42.403 INFO:teuthology.orchestra.run.smithi187.stdout:(60/159): libtool-2.4.6-45.el9.x86_64.rpm 6.2 MB/s | 582 kB 00:00 2024-05-26T22:45:42.454 INFO:teuthology.orchestra.run.smithi187.stdout:(61/159): libuuid-devel-2.37.4-18.el9.x86_64.rp 279 kB/s | 25 kB 00:00 2024-05-26T22:45:42.479 INFO:teuthology.orchestra.run.smithi187.stdout:(62/159): libzstd-devel-1.5.1-2.el9.x86_64.rpm 638 kB/s | 47 kB 00:00 2024-05-26T22:45:42.512 INFO:teuthology.orchestra.run.smithi187.stdout:(63/159): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.0 MB/s | 292 kB 00:00 2024-05-26T22:45:42.527 INFO:teuthology.orchestra.run.smithi131.stdout:(2/159): ceph-fuse-19.0.0-2857.g0ef46a24.el9.x8 2.0 MB/s | 890 kB 00:00 2024-05-26T22:45:42.546 INFO:teuthology.orchestra.run.smithi187.stdout:(64/159): lua-5.4.4-4.el9.x86_64.rpm 2.8 MB/s | 188 kB 00:00 2024-05-26T22:45:42.571 INFO:teuthology.orchestra.run.smithi187.stdout:(65/159): openblas-0.3.21-2.el9.x86_64.rpm 608 kB/s | 35 kB 00:00 2024-05-26T22:45:42.594 INFO:teuthology.orchestra.run.smithi131.stdout:(3/159): ceph-immutable-object-cache-19.0.0-285 2.2 MB/s | 145 kB 00:00 2024-05-26T22:45:42.663 INFO:teuthology.orchestra.run.smithi187.stdout:(66/159): flexiblas-netlib-3.0.4-8.el9.x86_64.r 5.7 MB/s | 3.0 MB 00:00 2024-05-26T22:45:42.783 INFO:teuthology.orchestra.run.smithi187.stdout:(67/159): postgresql-13.14-1.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-05-26T22:45:42.856 INFO:teuthology.orchestra.run.smithi187.stdout:(68/159): postgresql-contrib-13.14-1.el9.x86_64 12 MB/s | 911 kB 00:00 2024-05-26T22:45:42.947 INFO:teuthology.orchestra.run.smithi187.stdout:(69/159): openblas-openmp-0.3.21-2.el9.x86_64.r 12 MB/s | 4.7 MB 00:00 2024-05-26T22:45:42.981 INFO:teuthology.orchestra.run.smithi187.stdout:(70/159): postgresql-private-libs-13.14-1.el9.x 1.0 MB/s | 133 kB 00:00 2024-05-26T22:45:42.994 INFO:teuthology.orchestra.run.smithi131.stdout:(4/159): ceph-mds-19.0.0-2857.g0ef46a24.el9.x86 5.9 MB/s | 2.4 MB 00:00 2024-05-26T22:45:43.073 INFO:teuthology.orchestra.run.smithi187.stdout:(71/159): openssl-devel-3.2.1-1.el9.x86_64.rpm 8.8 MB/s | 4.4 MB 00:00 2024-05-26T22:45:43.140 INFO:teuthology.orchestra.run.smithi187.stdout:(72/159): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-05-26T22:45:43.170 INFO:teuthology.orchestra.run.smithi131.stdout:(5/159): ceph-mgr-19.0.0-2857.g0ef46a24.el9.x86 8.6 MB/s | 1.5 MB 00:00 2024-05-26T22:45:43.182 INFO:teuthology.orchestra.run.smithi187.stdout:(73/159): protobuf-3.14.0-13.el9.x86_64.rpm 5.1 MB/s | 1.0 MB 00:00 2024-05-26T22:45:43.224 INFO:teuthology.orchestra.run.smithi187.stdout:(74/159): python3-colorama-0.4.6-3.el9.noarch.r 1.5 MB/s | 61 kB 00:00 2024-05-26T22:45:43.282 INFO:teuthology.orchestra.run.smithi187.stdout:(75/159): python3-devel-3.9.18-3.el9.x86_64.rpm 4.1 MB/s | 244 kB 00:00 2024-05-26T22:45:43.333 INFO:teuthology.orchestra.run.smithi187.stdout:(76/159): python3-jinja2-2.11.3-6.el9.noarch.rp 4.9 MB/s | 248 kB 00:00 2024-05-26T22:45:43.375 INFO:teuthology.orchestra.run.smithi187.stdout:(77/159): python3-jsonpatch-1.21-16.el9.noarch. 628 kB/s | 26 kB 00:00 2024-05-26T22:45:43.416 INFO:teuthology.orchestra.run.smithi187.stdout:(78/159): python3-jsonpointer-2.0-4.el9.noarch. 470 kB/s | 19 kB 00:00 2024-05-26T22:45:43.534 INFO:teuthology.orchestra.run.smithi187.stdout:(79/159): postgresql-server-13.14-1.el9.x86_64. 9.9 MB/s | 5.8 MB 00:00 2024-05-26T22:45:43.545 INFO:teuthology.orchestra.run.smithi131.stdout:(6/159): ceph-mon-19.0.0-2857.g0ef46a24.el9.x86 13 MB/s | 4.7 MB 00:00 2024-05-26T22:45:43.567 INFO:teuthology.orchestra.run.smithi187.stdout:(80/159): python3-mako-1.1.4-6.el9.noarch.rpm 1.1 MB/s | 172 kB 00:00 2024-05-26T22:45:43.592 INFO:teuthology.orchestra.run.smithi187.stdout:(81/159): python3-markupsafe-1.1.1-12.el9.x86_6 604 kB/s | 35 kB 00:00 2024-05-26T22:45:43.643 INFO:teuthology.orchestra.run.smithi187.stdout:(82/159): python3-numpy-f2py-1.20.1-5.el9.x86_6 5.0 MB/s | 250 kB 00:00 2024-05-26T22:45:43.785 INFO:teuthology.orchestra.run.smithi187.stdout:(83/159): python3-babel-2.9.1-2.el9.noarch.rpm 9.3 MB/s | 6.0 MB 00:00 2024-05-26T22:45:43.827 INFO:teuthology.orchestra.run.smithi187.stdout:(84/159): python3-oauthlib-3.1.1-5.el9.noarch.r 1.2 MB/s | 222 kB 00:00 2024-05-26T22:45:43.852 INFO:teuthology.orchestra.run.smithi187.stdout:(85/159): python3-packaging-20.9-5.el9.noarch.r 1.1 MB/s | 77 kB 00:00 2024-05-26T22:45:43.877 INFO:teuthology.orchestra.run.smithi187.stdout:(86/159): python3-prettytable-0.7.2-27.el9.noar 851 kB/s | 42 kB 00:00 2024-05-26T22:45:43.911 INFO:teuthology.orchestra.run.smithi187.stdout:(87/159): python3-protobuf-3.14.0-13.el9.noarch 4.5 MB/s | 269 kB 00:00 2024-05-26T22:45:43.945 INFO:teuthology.orchestra.run.smithi187.stdout:(88/159): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.3 MB/s | 159 kB 00:00 2024-05-26T22:45:43.979 INFO:teuthology.orchestra.run.smithi187.stdout:(89/159): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 279 kB 00:00 2024-05-26T22:45:44.004 INFO:teuthology.orchestra.run.smithi187.stdout:(90/159): python3-pytz-2021.1-5.el9.noarch.rpm 882 kB/s | 51 kB 00:00 2024-05-26T22:45:44.029 INFO:teuthology.orchestra.run.smithi187.stdout:(91/159): python3-requests-oauthlib-1.3.0-12.el 1.1 MB/s | 54 kB 00:00 2024-05-26T22:45:44.071 INFO:teuthology.orchestra.run.smithi187.stdout:(92/159): python3-toml-0.10.2-6.el9.noarch.rpm 1.0 MB/s | 42 kB 00:00 2024-05-26T22:45:44.130 INFO:teuthology.orchestra.run.smithi187.stdout:(93/159): python3-wcwidth-0.2.5-8.el9.noarch.rp 768 kB/s | 44 kB 00:00 2024-05-26T22:45:44.180 INFO:teuthology.orchestra.run.smithi187.stdout:(94/159): qatlib-24.02.0-1.el9.x86_64.rpm 4.4 MB/s | 221 kB 00:00 2024-05-26T22:45:44.280 INFO:teuthology.orchestra.run.smithi187.stdout:(95/159): python3-numpy-1.20.1-5.el9.x86_64.rpm 7.2 MB/s | 5.1 MB 00:00 2024-05-26T22:45:44.306 INFO:teuthology.orchestra.run.smithi187.stdout:(96/159): qatlib-service-24.02.0-1.el9.x86_64.r 288 kB/s | 36 kB 00:00 2024-05-26T22:45:44.331 INFO:teuthology.orchestra.run.smithi187.stdout:(97/159): qatzip-libs-1.2.0-1.el9.x86_64.rpm 957 kB/s | 47 kB 00:00 2024-05-26T22:45:44.364 INFO:teuthology.orchestra.run.smithi187.stdout:(98/159): socat-1.7.4.1-5.el9.x86_64.rpm 5.2 MB/s | 305 kB 00:00 2024-05-26T22:45:44.381 INFO:teuthology.orchestra.run.smithi187.stdout:(99/159): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-05-26T22:45:44.407 INFO:teuthology.orchestra.run.smithi187.stdout:(100/159): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 2.2 MB/s | 94 kB 00:00 2024-05-26T22:45:44.457 INFO:teuthology.orchestra.run.smithi187.stdout:(101/159): xmlstarlet-1.6.1-20.el9.x86_64.rpm 852 kB/s | 64 kB 00:00 2024-05-26T22:45:44.474 INFO:teuthology.orchestra.run.smithi187.stdout:(102/159): zlib-devel-1.2.11-41.el9.x86_64.rpm 679 kB/s | 45 kB 00:00 2024-05-26T22:45:44.529 INFO:teuthology.orchestra.run.smithi131.stdout:(7/159): ceph-osd-19.0.0-2857.g0ef46a24.el9.x86 17 MB/s | 17 MB 00:00 2024-05-26T22:45:44.791 INFO:teuthology.orchestra.run.smithi187.stdout:(103/159): python3-scipy-1.6.2-8.el9.x86_64.rpm 21 MB/s | 16 MB 00:00 2024-05-26T22:45:44.816 INFO:teuthology.orchestra.run.smithi187.stdout:(104/159): lua-devel-5.4.4-4.el9.x86_64.rpm 62 kB/s | 22 kB 00:00 2024-05-26T22:45:45.146 INFO:teuthology.orchestra.run.smithi131.stdout:(8/159): ceph-radosgw-19.0.0-2857.g0ef46a24.el9 17 MB/s | 10 MB 00:00 2024-05-26T22:45:45.196 INFO:teuthology.orchestra.run.smithi131.stdout:(9/159): ceph-selinux-19.0.0-2857.g0ef46a24.el9 514 kB/s | 25 kB 00:00 2024-05-26T22:45:45.325 INFO:teuthology.orchestra.run.smithi187.stdout:(105/159): protobuf-compiler-3.14.0-13.el9.x86_ 1.0 MB/s | 863 kB 00:00 2024-05-26T22:45:45.617 INFO:teuthology.orchestra.run.smithi187.stdout:(106/159): gperftools-libs-2.9.1-2.el9.x86_64.r 386 kB/s | 309 kB 00:00 2024-05-26T22:45:45.650 INFO:teuthology.orchestra.run.smithi187.stdout:(107/159): abseil-cpp-20211102.0-3.el9.x86_64.r 639 kB/s | 548 kB 00:00 2024-05-26T22:45:45.759 INFO:teuthology.orchestra.run.smithi187.stdout:(108/159): grpc-data-1.46.7-10.el9.noarch.rpm 45 kB/s | 19 kB 00:00 2024-05-26T22:45:45.809 INFO:teuthology.orchestra.run.smithi187.stdout:(109/159): libarrow-doc-9.0.0-11.el9.noarch.rpm 165 kB/s | 26 kB 00:00 2024-05-26T22:45:45.934 INFO:teuthology.orchestra.run.smithi187.stdout:(110/159): libunwind-1.6.2-1.el9.x86_64.rpm 542 kB/s | 67 kB 00:00 2024-05-26T22:45:45.993 INFO:teuthology.orchestra.run.smithi187.stdout:(111/159): liboath-2.6.7-2.el9.x86_64.rpm 208 kB/s | 49 kB 00:00 2024-05-26T22:45:46.068 INFO:teuthology.orchestra.run.smithi187.stdout:(112/159): luarocks-3.9.2-5.el9.noarch.rpm 1.1 MB/s | 151 kB 00:00 2024-05-26T22:45:46.185 INFO:teuthology.orchestra.run.smithi187.stdout:(113/159): libarrow-9.0.0-11.el9.x86_64.rpm 7.8 MB/s | 4.4 MB 00:00 2024-05-26T22:45:46.205 INFO:teuthology.orchestra.run.smithi131.stdout:(10/159): ceph-base-19.0.0-2857.g0ef46a24.el9.x 1.2 MB/s | 5.4 MB 00:04 2024-05-26T22:45:46.261 INFO:teuthology.orchestra.run.smithi187.stdout:(114/159): python3-asyncssh-2.13.2-5.el9.noarch 2.8 MB/s | 548 kB 00:00 2024-05-26T22:45:46.302 INFO:teuthology.orchestra.run.smithi187.stdout:(115/159): python3-bcrypt-3.2.2-1.el9.x86_64.rp 374 kB/s | 43 kB 00:00 2024-05-26T22:45:46.378 INFO:teuthology.orchestra.run.smithi187.stdout:(116/159): python3-cachetools-4.2.4-1.el9.noarc 277 kB/s | 32 kB 00:00 2024-05-26T22:45:46.420 INFO:teuthology.orchestra.run.smithi187.stdout:(117/159): python3-certifi-2021.10.8-2.el9.noar 129 kB/s | 15 kB 00:00 2024-05-26T22:45:46.422 INFO:teuthology.orchestra.run.smithi131.stdout:(11/159): cephfs-mirror-19.0.0-2857.g0ef46a24.e 1.0 MB/s | 221 kB 00:00 2024-05-26T22:45:46.478 INFO:teuthology.orchestra.run.smithi187.stdout:(118/159): parquet-libs-9.0.0-11.el9.x86_64.rpm 1.7 MB/s | 839 kB 00:00 2024-05-26T22:45:46.512 INFO:teuthology.orchestra.run.smithi187.stdout:(119/159): python3-cheroot-8.6.0-4.el9.noarch.r 1.3 MB/s | 172 kB 00:00 2024-05-26T22:45:46.522 INFO:teuthology.orchestra.run.smithi131.stdout:(12/159): cephfs-shell-19.0.0-2857.g0ef46a24.el 273 kB/s | 27 kB 00:00 2024-05-26T22:45:46.546 INFO:teuthology.orchestra.run.smithi187.stdout:(120/159): python3-cherrypy-18.6.1-2.el9.noarch 2.8 MB/s | 358 kB 00:00 2024-05-26T22:45:46.614 INFO:teuthology.orchestra.run.smithi131.stdout:(13/159): libcephfs-devel-19.0.0-2857.g0ef46a24 349 kB/s | 32 kB 00:00 2024-05-26T22:45:46.621 INFO:teuthology.orchestra.run.smithi187.stdout:(121/159): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.1 MB/s | 303 kB 00:00 2024-05-26T22:45:46.646 INFO:teuthology.orchestra.run.smithi187.stdout:(122/159): python3-google-auth-2.29.0-1.el9.noa 1.5 MB/s | 206 kB 00:00 2024-05-26T22:45:46.738 INFO:teuthology.orchestra.run.smithi187.stdout:(123/159): python3-grpcio-1.46.7-10.el9.x86_64. 11 MB/s | 2.0 MB 00:00 2024-05-26T22:45:46.785 INFO:teuthology.orchestra.run.smithi187.stdout:(124/159): python3-jaraco-8.2.1-3.el9.noarch.rp 77 kB/s | 11 kB 00:00 2024-05-26T22:45:46.822 INFO:teuthology.orchestra.run.smithi187.stdout:(125/159): python3-grpcio-tools-1.46.7-10.el9.x 721 kB/s | 144 kB 00:00 2024-05-26T22:45:46.847 INFO:teuthology.orchestra.run.smithi187.stdout:(126/159): python3-jaraco-classes-3.2.1-5.el9.n 164 kB/s | 18 kB 00:00 2024-05-26T22:45:46.889 INFO:teuthology.orchestra.run.smithi187.stdout:(127/159): python3-jaraco-collections-3.0.0-8.e 224 kB/s | 23 kB 00:00 2024-05-26T22:45:46.931 INFO:teuthology.orchestra.run.smithi187.stdout:(128/159): python3-jaraco-functools-3.5.0-2.el9 180 kB/s | 19 kB 00:00 2024-05-26T22:45:46.957 INFO:teuthology.orchestra.run.smithi187.stdout:(129/159): python3-jaraco-text-3.2.0-6.el9.noar 181 kB/s | 20 kB 00:00 2024-05-26T22:45:47.040 INFO:teuthology.orchestra.run.smithi187.stdout:(130/159): python3-kubernetes-26.1.0-2.el9.noar 6.8 MB/s | 1.0 MB 00:00 2024-05-26T22:45:47.065 INFO:teuthology.orchestra.run.smithi187.stdout:(131/159): python3-logutils-0.3.5-21.el9.noarch 348 kB/s | 46 kB 00:00 2024-05-26T22:45:47.091 INFO:teuthology.orchestra.run.smithi187.stdout:(132/159): python3-more-itertools-8.12.0-2.el9. 591 kB/s | 79 kB 00:00 2024-05-26T22:45:47.166 INFO:teuthology.orchestra.run.smithi187.stdout:(133/159): python3-natsort-7.1.1-5.el9.noarch.r 462 kB/s | 58 kB 00:00 2024-05-26T22:45:47.200 INFO:teuthology.orchestra.run.smithi187.stdout:(134/159): python3-pecan-1.4.2-3.el9.noarch.rpm 2.0 MB/s | 272 kB 00:00 2024-05-26T22:45:47.225 INFO:teuthology.orchestra.run.smithi187.stdout:(135/159): python3-portend-3.1.0-2.el9.noarch.r 123 kB/s | 16 kB 00:00 2024-05-26T22:45:47.275 INFO:teuthology.orchestra.run.smithi187.stdout:(136/159): python3-pyOpenSSL-21.0.0-1.el9.noarc 832 kB/s | 90 kB 00:00 2024-05-26T22:45:47.317 INFO:teuthology.orchestra.run.smithi187.stdout:(137/159): python3-pyperclip-1.8.2-2.el9.noarch 242 kB/s | 28 kB 00:00 2024-05-26T22:45:47.342 INFO:teuthology.orchestra.run.smithi187.stdout:(138/159): python3-repoze-lru-0.7-16.el9.noarch 264 kB/s | 31 kB 00:00 2024-05-26T22:45:47.409 INFO:teuthology.orchestra.run.smithi187.stdout:(139/159): python3-routes-2.5.1-5.el9.noarch.rp 1.4 MB/s | 188 kB 00:00 2024-05-26T22:45:47.466 INFO:teuthology.orchestra.run.smithi187.stdout:(140/159): python3-rsa-4.9-2.el9.noarch.rpm 399 kB/s | 59 kB 00:00 2024-05-26T22:45:47.501 INFO:teuthology.orchestra.run.smithi187.stdout:(141/159): python3-tempora-5.0.0-2.el9.noarch.r 226 kB/s | 36 kB 00:00 2024-05-26T22:45:47.526 INFO:teuthology.orchestra.run.smithi187.stdout:(142/159): python3-typing-extensions-4.4.0-2.el 437 kB/s | 51 kB 00:00 2024-05-26T22:45:47.602 INFO:teuthology.orchestra.run.smithi187.stdout:(143/159): python3-webob-1.8.7-6.el9.noarch.rpm 1.7 MB/s | 230 kB 00:00 2024-05-26T22:45:47.627 INFO:teuthology.orchestra.run.smithi187.stdout:(144/159): python3-websocket-client-1.2.3-2.el9 716 kB/s | 90 kB 00:00 2024-05-26T22:45:47.669 INFO:teuthology.orchestra.run.smithi187.stdout:(145/159): python3-werkzeug-2.0.3-3.el9.1.noarc 2.9 MB/s | 427 kB 00:00 2024-05-26T22:45:47.728 INFO:teuthology.orchestra.run.smithi187.stdout:(146/159): python3-zc-lockfile-2.0-10.el9.noarc 160 kB/s | 20 kB 00:00 2024-05-26T22:45:47.739 INFO:teuthology.orchestra.run.smithi131.stdout:(14/159): ceph-test-19.0.0-2857.g0ef46a24.el9.x 19 MB/s | 48 MB 00:02 2024-05-26T22:45:47.770 INFO:teuthology.orchestra.run.smithi187.stdout:(147/159): re2-20211101-3.el9.x86_64.rpm 1.3 MB/s | 192 kB 00:00 2024-05-26T22:45:47.772 INFO:teuthology.orchestra.run.smithi131.stdout:(15/159): libcephfs2-19.0.0-2857.g0ef46a24.el9. 644 kB/s | 746 kB 00:01 2024-05-26T22:45:47.823 INFO:teuthology.orchestra.run.smithi131.stdout:(16/159): libcephsqlite-19.0.0-2857.g0ef46a24.e 3.3 MB/s | 163 kB 00:00 2024-05-26T22:45:47.862 INFO:teuthology.orchestra.run.smithi187.stdout:(148/159): thrift-0.15.0-2.el9.x86_64.rpm 8.3 MB/s | 1.6 MB 00:00 2024-05-26T22:45:47.865 INFO:teuthology.orchestra.run.smithi131.stdout:(17/159): librados-devel-19.0.0-2857.g0ef46a24. 1.4 MB/s | 127 kB 00:00 2024-05-26T22:45:47.941 INFO:teuthology.orchestra.run.smithi131.stdout:(18/159): libradosstriper1-19.0.0-2857.g0ef46a2 4.2 MB/s | 501 kB 00:00 2024-05-26T22:45:48.016 INFO:teuthology.orchestra.run.smithi131.stdout:(19/159): python3-ceph-argparse-19.0.0-2857.g0e 609 kB/s | 45 kB 00:00 2024-05-26T22:45:48.083 INFO:teuthology.orchestra.run.smithi131.stdout:(20/159): python3-ceph-common-19.0.0-2857.g0ef4 2.0 MB/s | 135 kB 00:00 2024-05-26T22:45:48.133 INFO:teuthology.orchestra.run.smithi131.stdout:(21/159): python3-cephfs-19.0.0-2857.g0ef46a24. 3.2 MB/s | 162 kB 00:00 2024-05-26T22:45:48.138 INFO:teuthology.orchestra.run.smithi187.stdout:(149/159): librados2-19.0.0-2857.g0ef46a24.el9. 8.3 MB/s | 3.4 MB 00:00 2024-05-26T22:45:48.212 INFO:teuthology.orchestra.run.smithi187.stdout:(150/159): NetworkManager-1.47.90-1.el9.x86_64. 8.2 MB/s | 2.3 MB 00:00 2024-05-26T22:45:48.217 INFO:teuthology.orchestra.run.smithi131.stdout:(22/159): python3-rados-19.0.0-2857.g0ef46a24.e 3.8 MB/s | 321 kB 00:00 2024-05-26T22:45:48.254 INFO:teuthology.orchestra.run.smithi187.stdout:(151/159): NetworkManager-team-1.47.90-1.el9.x8 1.0 MB/s | 41 kB 00:00 2024-05-26T22:45:48.275 INFO:teuthology.orchestra.run.smithi131.stdout:(23/159): python3-rbd-19.0.0-2857.g0ef46a24.el9 5.1 MB/s | 300 kB 00:00 2024-05-26T22:45:48.304 INFO:teuthology.orchestra.run.smithi187.stdout:(152/159): NetworkManager-tui-1.47.90-1.el9.x86 4.9 MB/s | 248 kB 00:00 2024-05-26T22:45:48.326 INFO:teuthology.orchestra.run.smithi131.stdout:(24/159): python3-rgw-19.0.0-2857.g0ef46a24.el9 2.0 MB/s | 99 kB 00:00 2024-05-26T22:45:48.355 INFO:teuthology.orchestra.run.smithi187.stdout:(153/159): NetworkManager-libnm-1.47.90-1.el9.x 8.4 MB/s | 1.8 MB 00:00 2024-05-26T22:45:48.372 INFO:teuthology.orchestra.run.smithi187.stdout:(154/159): elfutils-debuginfod-client-0.191-4.e 555 kB/s | 37 kB 00:00 2024-05-26T22:45:48.376 INFO:teuthology.orchestra.run.smithi131.stdout:(25/159): rbd-fuse-19.0.0-2857.g0ef46a24.el9.x8 1.7 MB/s | 85 kB 00:00 2024-05-26T22:45:48.405 INFO:teuthology.orchestra.run.smithi187.stdout:(155/159): elfutils-libelf-0.191-4.el9.x86_64.r 4.1 MB/s | 207 kB 00:00 2024-05-26T22:45:48.439 INFO:teuthology.orchestra.run.smithi187.stdout:(156/159): elfutils-libs-0.191-4.el9.x86_64.rpm 3.8 MB/s | 260 kB 00:00 2024-05-26T22:45:48.514 INFO:teuthology.orchestra.run.smithi187.stdout:(157/159): openssl-3.2.1-1.el9.x86_64.rpm 13 MB/s | 1.3 MB 00:00 2024-05-26T22:45:48.598 INFO:teuthology.orchestra.run.smithi187.stdout:(158/159): openssl-libs-3.2.1-1.el9.x86_64.rpm 15 MB/s | 2.4 MB 00:00 2024-05-26T22:45:48.685 INFO:teuthology.orchestra.run.smithi131.stdout:(26/159): rbd-mirror-19.0.0-2857.g0ef46a24.el9. 10 MB/s | 3.1 MB 00:00 2024-05-26T22:45:48.706 INFO:teuthology.orchestra.run.smithi187.stdout:(159/159): librbd1-19.0.0-2857.g0ef46a24.el9.x8 3.4 MB/s | 3.1 MB 00:00 2024-05-26T22:45:48.718 INFO:teuthology.orchestra.run.smithi187.stdout:-------------------------------------------------------------------------------- 2024-05-26T22:45:48.719 INFO:teuthology.orchestra.run.smithi187.stdout:Total 15 MB/s | 225 MB 00:14 2024-05-26T22:45:48.735 INFO:teuthology.orchestra.run.smithi131.stdout:(27/159): rbd-nbd-19.0.0-2857.g0ef46a24.el9.x86 3.4 MB/s | 170 kB 00:00 2024-05-26T22:45:48.902 INFO:teuthology.orchestra.run.smithi131.stdout:(28/159): ceph-grafana-dashboards-19.0.0-2857.g 191 kB/s | 32 kB 00:00 2024-05-26T22:45:48.952 INFO:teuthology.orchestra.run.smithi131.stdout:(29/159): ceph-mgr-cephadm-19.0.0-2857.g0ef46a2 2.8 MB/s | 142 kB 00:00 2024-05-26T22:45:49.203 INFO:teuthology.orchestra.run.smithi131.stdout:(30/159): ceph-mgr-dashboard-19.0.0-2857.g0ef46 14 MB/s | 3.5 MB 00:00 2024-05-26T22:45:49.678 INFO:teuthology.orchestra.run.smithi131.stdout:(31/159): ceph-mgr-diskprediction-local-19.0.0- 16 MB/s | 7.4 MB 00:00 2024-05-26T22:45:49.728 INFO:teuthology.orchestra.run.smithi131.stdout:(32/159): ceph-mgr-modules-core-19.0.0-2857.g0e 5.0 MB/s | 249 kB 00:00 2024-05-26T22:45:49.814 INFO:teuthology.orchestra.run.smithi131.stdout:(33/159): ceph-mgr-rook-19.0.0-2857.g0ef46a24.e 587 kB/s | 49 kB 00:00 2024-05-26T22:45:50.037 INFO:teuthology.orchestra.run.smithi131.stdout:(34/159): librgw2-19.0.0-2857.g0ef46a24.el9.x86 2.4 MB/s | 5.3 MB 00:02 2024-05-26T22:45:50.063 INFO:teuthology.orchestra.run.smithi131.stdout:(35/159): ceph-prometheus-alerts-19.0.0-2857.g0 66 kB/s | 16 kB 00:00 2024-05-26T22:45:50.130 INFO:teuthology.orchestra.run.smithi131.stdout:(36/159): ceph-volume-19.0.0-2857.g0ef46a24.el9 2.8 MB/s | 264 kB 00:00 2024-05-26T22:45:50.180 INFO:teuthology.orchestra.run.smithi131.stdout:(37/159): cephadm-19.0.0-2857.g0ef46a24.el9.noa 6.4 MB/s | 762 kB 00:00 2024-05-26T22:45:50.197 INFO:teuthology.orchestra.run.smithi131.stdout:(38/159): cephfs-top-19.0.0-2857.g0ef46a24.el9. 276 kB/s | 18 kB 00:00 2024-05-26T22:45:50.314 INFO:teuthology.orchestra.run.smithi131.stdout:(39/159): fuse-2.9.9-15.el9.x86_64.rpm 603 kB/s | 80 kB 00:00 2024-05-26T22:45:50.389 INFO:teuthology.orchestra.run.smithi131.stdout:(40/159): libquadmath-11.4.1-3.el9.x86_64.rpm 2.6 MB/s | 195 kB 00:00 2024-05-26T22:45:50.423 INFO:teuthology.orchestra.run.smithi131.stdout:(41/159): libgfortran-11.4.1-3.el9.x86_64.rpm 3.5 MB/s | 803 kB 00:00 2024-05-26T22:45:50.457 INFO:teuthology.orchestra.run.smithi131.stdout:(42/159): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.7 MB/s | 253 kB 00:00 2024-05-26T22:45:50.533 INFO:teuthology.orchestra.run.smithi131.stdout:(43/159): python3-cryptography-36.0.1-4.el9.x86 11 MB/s | 1.2 MB 00:00 2024-05-26T22:45:50.558 INFO:teuthology.orchestra.run.smithi131.stdout:(44/159): python3-ply-3.11-14.el9.noarch.rpm 1.0 MB/s | 106 kB 00:00 2024-05-26T22:45:50.592 INFO:teuthology.orchestra.run.smithi131.stdout:(45/159): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-05-26T22:45:50.625 INFO:teuthology.orchestra.run.smithi131.stdout:(46/159): python3-requests-2.25.1-8.el9.noarch. 1.8 MB/s | 125 kB 00:00 2024-05-26T22:45:50.659 INFO:teuthology.orchestra.run.smithi131.stdout:(47/159): python3-urllib3-1.26.5-5.el9.noarch.r 3.2 MB/s | 215 kB 00:00 2024-05-26T22:45:50.801 INFO:teuthology.orchestra.run.smithi131.stdout:(48/159): automake-1.16.2-8.el9.noarch.rpm 3.9 MB/s | 689 kB 00:00 2024-05-26T22:45:50.843 INFO:teuthology.orchestra.run.smithi131.stdout:(49/159): boost-program-options-1.75.0-8.el9.x8 2.6 MB/s | 107 kB 00:00 2024-05-26T22:45:50.877 INFO:teuthology.orchestra.run.smithi131.stdout:(50/159): bison-3.7.4-5.el9.x86_64.rpm 4.3 MB/s | 943 kB 00:00 2024-05-26T22:45:50.902 INFO:teuthology.orchestra.run.smithi131.stdout:(51/159): elfutils-libelf-devel-0.191-4.el9.x86 374 kB/s | 22 kB 00:00 2024-05-26T22:45:50.935 INFO:teuthology.orchestra.run.smithi131.stdout:(52/159): flex-2.6.4-9.el9.x86_64.rpm 5.3 MB/s | 313 kB 00:00 2024-05-26T22:45:50.961 INFO:teuthology.orchestra.run.smithi131.stdout:(53/159): flexiblas-3.0.4-8.el9.x86_64.rpm 565 kB/s | 33 kB 00:00 2024-05-26T22:45:51.003 INFO:teuthology.orchestra.run.smithi131.stdout:(54/159): flexiblas-openblas-openmp-3.0.4-8.el9 447 kB/s | 18 kB 00:00 2024-05-26T22:45:51.045 INFO:teuthology.orchestra.run.smithi131.stdout:(55/159): libaio-devel-0.3.111-13.el9.x86_64.rp 273 kB/s | 11 kB 00:00 2024-05-26T22:45:51.095 INFO:teuthology.orchestra.run.smithi131.stdout:(56/159): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.2 MB/s | 160 kB 00:00 2024-05-26T22:45:51.354 INFO:teuthology.orchestra.run.smithi131.stdout:(57/159): ceph-common-19.0.0-2857.g0ef46a24.el9 2.2 MB/s | 21 MB 00:09 2024-05-26T22:45:51.379 INFO:teuthology.orchestra.run.smithi131.stdout:(58/159): librabbitmq-0.11.0-7.el9.x86_64.rpm 160 kB/s | 45 kB 00:00 2024-05-26T22:45:51.454 INFO:teuthology.orchestra.run.smithi131.stdout:(59/159): flexiblas-netlib-3.0.4-8.el9.x86_64.r 5.8 MB/s | 3.0 MB 00:00 2024-05-26T22:45:51.496 INFO:teuthology.orchestra.run.smithi131.stdout:(60/159): libtool-2.4.6-45.el9.x86_64.rpm 4.9 MB/s | 582 kB 00:00 2024-05-26T22:45:51.522 INFO:teuthology.orchestra.run.smithi131.stdout:(61/159): libuuid-devel-2.37.4-18.el9.x86_64.rp 384 kB/s | 25 kB 00:00 2024-05-26T22:45:51.547 INFO:teuthology.orchestra.run.smithi131.stdout:(62/159): libzstd-devel-1.5.1-2.el9.x86_64.rpm 955 kB/s | 47 kB 00:00 2024-05-26T22:45:51.581 INFO:teuthology.orchestra.run.smithi131.stdout:(63/159): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.9 MB/s | 292 kB 00:00 2024-05-26T22:45:51.615 INFO:teuthology.orchestra.run.smithi131.stdout:(64/159): lua-5.4.4-4.el9.x86_64.rpm 2.8 MB/s | 188 kB 00:00 2024-05-26T22:45:51.673 INFO:teuthology.orchestra.run.smithi131.stdout:(65/159): librdkafka-1.6.1-102.el9.x86_64.rpm 2.0 MB/s | 662 kB 00:00 2024-05-26T22:45:51.690 INFO:teuthology.orchestra.run.smithi131.stdout:(66/159): openblas-0.3.21-2.el9.x86_64.rpm 325 kB/s | 35 kB 00:00 2024-05-26T22:45:51.799 INFO:teuthology.orchestra.run.smithi131.stdout:(67/159): postgresql-13.14-1.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-05-26T22:45:51.866 INFO:teuthology.orchestra.run.smithi131.stdout:(68/159): postgresql-contrib-13.14-1.el9.x86_64 14 MB/s | 911 kB 00:00 2024-05-26T22:45:51.958 INFO:teuthology.orchestra.run.smithi131.stdout:(69/159): openssl-devel-3.2.1-1.el9.x86_64.rpm 16 MB/s | 4.4 MB 00:00 2024-05-26T22:45:51.992 INFO:teuthology.orchestra.run.smithi131.stdout:(70/159): postgresql-private-libs-13.14-1.el9.x 1.0 MB/s | 133 kB 00:00 2024-05-26T22:45:51.997 INFO:teuthology.orchestra.run.smithi187.stdout:Running transaction check 2024-05-26T22:45:52.092 INFO:teuthology.orchestra.run.smithi131.stdout:(71/159): openblas-openmp-0.3.21-2.el9.x86_64.r 9.9 MB/s | 4.7 MB 00:00 2024-05-26T22:45:52.134 INFO:teuthology.orchestra.run.smithi131.stdout:(72/159): python3-attrs-20.3.0-7.el9.noarch.rpm 2.1 MB/s | 87 kB 00:00 2024-05-26T22:45:52.184 INFO:teuthology.orchestra.run.smithi131.stdout:(73/159): protobuf-3.14.0-13.el9.x86_64.rpm 5.3 MB/s | 1.0 MB 00:00 2024-05-26T22:45:52.249 INFO:teuthology.orchestra.run.smithi187.stdout:Transaction check succeeded. 2024-05-26T22:45:52.250 INFO:teuthology.orchestra.run.smithi187.stdout:Running transaction test 2024-05-26T22:45:52.251 INFO:teuthology.orchestra.run.smithi131.stdout:(74/159): python3-colorama-0.4.6-3.el9.noarch.r 924 kB/s | 61 kB 00:00 2024-05-26T22:45:52.302 INFO:teuthology.orchestra.run.smithi131.stdout:(75/159): python3-devel-3.9.18-3.el9.x86_64.rpm 4.8 MB/s | 244 kB 00:00 2024-05-26T22:45:52.352 INFO:teuthology.orchestra.run.smithi131.stdout:(76/159): python3-jinja2-2.11.3-6.el9.noarch.rp 4.9 MB/s | 248 kB 00:00 2024-05-26T22:45:52.394 INFO:teuthology.orchestra.run.smithi131.stdout:(77/159): python3-jsonpatch-1.21-16.el9.noarch. 627 kB/s | 26 kB 00:00 2024-05-26T22:45:52.436 INFO:teuthology.orchestra.run.smithi131.stdout:(78/159): python3-jsonpointer-2.0-4.el9.noarch. 468 kB/s | 19 kB 00:00 2024-05-26T22:45:52.553 INFO:teuthology.orchestra.run.smithi131.stdout:(79/159): postgresql-server-13.14-1.el9.x86_64. 9.7 MB/s | 5.8 MB 00:00 2024-05-26T22:45:52.578 INFO:teuthology.orchestra.run.smithi131.stdout:(80/159): python3-mako-1.1.4-6.el9.noarch.rpm 1.2 MB/s | 172 kB 00:00 2024-05-26T22:45:52.620 INFO:teuthology.orchestra.run.smithi131.stdout:(81/159): python3-markupsafe-1.1.1-12.el9.x86_6 851 kB/s | 35 kB 00:00 2024-05-26T22:45:52.737 INFO:teuthology.orchestra.run.smithi131.stdout:(82/159): python3-babel-2.9.1-2.el9.noarch.rpm 9.9 MB/s | 6.0 MB 00:00 2024-05-26T22:45:52.770 INFO:teuthology.orchestra.run.smithi131.stdout:(83/159): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.6 MB/s | 250 kB 00:00 2024-05-26T22:45:52.821 INFO:teuthology.orchestra.run.smithi131.stdout:(84/159): python3-oauthlib-3.1.1-5.el9.noarch.r 2.6 MB/s | 222 kB 00:00 2024-05-26T22:45:52.846 INFO:teuthology.orchestra.run.smithi131.stdout:(85/159): python3-packaging-20.9-5.el9.noarch.r 1.0 MB/s | 77 kB 00:00 2024-05-26T22:45:52.872 INFO:teuthology.orchestra.run.smithi131.stdout:(86/159): python3-prettytable-0.7.2-27.el9.noar 854 kB/s | 42 kB 00:00 2024-05-26T22:45:52.905 INFO:teuthology.orchestra.run.smithi131.stdout:(87/159): python3-protobuf-3.14.0-13.el9.noarch 4.6 MB/s | 269 kB 00:00 2024-05-26T22:45:52.939 INFO:teuthology.orchestra.run.smithi131.stdout:(88/159): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.3 MB/s | 159 kB 00:00 2024-05-26T22:45:52.972 INFO:teuthology.orchestra.run.smithi131.stdout:(89/159): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 279 kB 00:00 2024-05-26T22:45:52.998 INFO:teuthology.orchestra.run.smithi131.stdout:(90/159): python3-pytz-2021.1-5.el9.noarch.rpm 877 kB/s | 51 kB 00:00 2024-05-26T22:45:53.023 INFO:teuthology.orchestra.run.smithi131.stdout:(91/159): python3-requests-oauthlib-1.3.0-12.el 1.1 MB/s | 54 kB 00:00 2024-05-26T22:45:53.066 INFO:teuthology.orchestra.run.smithi131.stdout:(92/159): python3-toml-0.10.2-6.el9.noarch.rpm 1.0 MB/s | 42 kB 00:00 2024-05-26T22:45:53.108 INFO:teuthology.orchestra.run.smithi131.stdout:(93/159): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.1 MB/s | 44 kB 00:00 2024-05-26T22:45:53.158 INFO:teuthology.orchestra.run.smithi131.stdout:(94/159): qatlib-24.02.0-1.el9.x86_64.rpm 4.4 MB/s | 221 kB 00:00 2024-05-26T22:45:53.200 INFO:teuthology.orchestra.run.smithi131.stdout:(95/159): qatlib-service-24.02.0-1.el9.x86_64.r 878 kB/s | 36 kB 00:00 2024-05-26T22:45:53.308 INFO:teuthology.orchestra.run.smithi131.stdout:(96/159): python3-numpy-1.20.1-5.el9.x86_64.rpm 7.0 MB/s | 5.1 MB 00:00 2024-05-26T22:45:53.334 INFO:teuthology.orchestra.run.smithi131.stdout:(97/159): qatzip-libs-1.2.0-1.el9.x86_64.rpm 357 kB/s | 47 kB 00:00 2024-05-26T22:45:53.393 INFO:teuthology.orchestra.run.smithi131.stdout:(98/159): socat-1.7.4.1-5.el9.x86_64.rpm 3.5 MB/s | 305 kB 00:00 2024-05-26T22:45:53.426 INFO:teuthology.orchestra.run.smithi131.stdout:(99/159): uuid-1.6.2-55.el9.x86_64.rpm 634 kB/s | 58 kB 00:00 2024-05-26T22:45:53.451 INFO:teuthology.orchestra.run.smithi131.stdout:(100/159): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.6 MB/s | 94 kB 00:00 2024-05-26T22:45:53.477 INFO:teuthology.orchestra.run.smithi131.stdout:(101/159): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.3 MB/s | 64 kB 00:00 2024-05-26T22:45:53.502 INFO:teuthology.orchestra.run.smithi131.stdout:(102/159): zlib-devel-1.2.11-41.el9.x86_64.rpm 906 kB/s | 45 kB 00:00 2024-05-26T22:45:53.586 INFO:teuthology.orchestra.run.smithi131.stdout:(103/159): lua-devel-5.4.4-4.el9.x86_64.rpm 207 kB/s | 22 kB 00:00 2024-05-26T22:45:53.870 INFO:teuthology.orchestra.run.smithi131.stdout:(104/159): python3-scipy-1.6.2-8.el9.x86_64.rpm 19 MB/s | 16 MB 00:00 2024-05-26T22:45:53.920 INFO:teuthology.orchestra.run.smithi131.stdout:(105/159): protobuf-compiler-3.14.0-13.el9.x86_ 2.0 MB/s | 863 kB 00:00 2024-05-26T22:45:54.028 INFO:teuthology.orchestra.run.smithi131.stdout:(106/159): abseil-cpp-20211102.0-3.el9.x86_64.r 1.2 MB/s | 548 kB 00:00 2024-05-26T22:45:54.062 INFO:teuthology.orchestra.run.smithi131.stdout:(107/159): gperftools-libs-2.9.1-2.el9.x86_64.r 1.6 MB/s | 309 kB 00:00 2024-05-26T22:45:54.087 INFO:teuthology.orchestra.run.smithi131.stdout:(108/159): grpc-data-1.46.7-10.el9.noarch.rpm 117 kB/s | 19 kB 00:00 2024-05-26T22:45:54.279 INFO:teuthology.orchestra.run.smithi131.stdout:(109/159): libarrow-9.0.0-11.el9.x86_64.rpm 18 MB/s | 4.4 MB 00:00 2024-05-26T22:45:54.304 INFO:teuthology.orchestra.run.smithi131.stdout:(110/159): libarrow-doc-9.0.0-11.el9.noarch.rpm 126 kB/s | 26 kB 00:00 2024-05-26T22:45:54.329 INFO:teuthology.orchestra.run.smithi131.stdout:(111/159): liboath-2.6.7-2.el9.x86_64.rpm 210 kB/s | 49 kB 00:00 2024-05-26T22:45:54.355 INFO:teuthology.orchestra.run.smithi131.stdout:(112/159): libunwind-1.6.2-1.el9.x86_64.rpm 1.4 MB/s | 67 kB 00:00 2024-05-26T22:45:54.388 INFO:teuthology.orchestra.run.smithi131.stdout:(113/159): luarocks-3.9.2-5.el9.noarch.rpm 1.8 MB/s | 151 kB 00:00 2024-05-26T22:45:54.422 INFO:teuthology.orchestra.run.smithi131.stdout:(114/159): parquet-libs-9.0.0-11.el9.x86_64.rpm 9.1 MB/s | 839 kB 00:00 2024-05-26T22:45:54.464 INFO:teuthology.orchestra.run.smithi131.stdout:(115/159): python3-asyncssh-2.13.2-5.el9.noarch 5.4 MB/s | 548 kB 00:00 2024-05-26T22:45:54.489 INFO:teuthology.orchestra.run.smithi131.stdout:(116/159): python3-bcrypt-3.2.2-1.el9.x86_64.rp 713 kB/s | 43 kB 00:00 2024-05-26T22:45:54.532 INFO:teuthology.orchestra.run.smithi131.stdout:(117/159): python3-cachetools-4.2.4-1.el9.noarc 312 kB/s | 32 kB 00:00 2024-05-26T22:45:54.565 INFO:teuthology.orchestra.run.smithi131.stdout:(118/159): python3-certifi-2021.10.8-2.el9.noar 151 kB/s | 15 kB 00:00 2024-05-26T22:45:54.598 INFO:teuthology.orchestra.run.smithi131.stdout:(119/159): python3-cheroot-8.6.0-4.el9.noarch.r 2.6 MB/s | 172 kB 00:00 2024-05-26T22:45:54.623 INFO:teuthology.orchestra.run.smithi131.stdout:(120/159): python3-cherrypy-18.6.1-2.el9.noarch 3.9 MB/s | 358 kB 00:00 2024-05-26T22:45:54.658 INFO:teuthology.orchestra.run.smithi131.stdout:(121/159): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.4 MB/s | 303 kB 00:00 2024-05-26T22:45:54.741 INFO:teuthology.orchestra.run.smithi131.stdout:(122/159): python3-google-auth-2.29.0-1.el9.noa 1.8 MB/s | 206 kB 00:00 2024-05-26T22:45:54.825 INFO:teuthology.orchestra.run.smithi131.stdout:(123/159): python3-grpcio-1.46.7-10.el9.x86_64. 10 MB/s | 2.0 MB 00:00 2024-05-26T22:45:54.842 INFO:teuthology.orchestra.run.smithi131.stdout:(124/159): python3-grpcio-tools-1.46.7-10.el9.x 788 kB/s | 144 kB 00:00 2024-05-26T22:45:54.867 INFO:teuthology.orchestra.run.smithi131.stdout:(125/159): python3-jaraco-8.2.1-3.el9.noarch.rp 93 kB/s | 11 kB 00:00 2024-05-26T22:45:54.892 INFO:teuthology.orchestra.run.smithi131.stdout:(126/159): python3-jaraco-classes-3.2.1-5.el9.n 360 kB/s | 18 kB 00:00 2024-05-26T22:45:54.918 INFO:teuthology.orchestra.run.smithi131.stdout:(127/159): python3-jaraco-collections-3.0.0-8.e 313 kB/s | 23 kB 00:00 2024-05-26T22:45:54.943 INFO:teuthology.orchestra.run.smithi131.stdout:(128/159): python3-jaraco-functools-3.5.0-2.el9 262 kB/s | 19 kB 00:00 2024-05-26T22:45:54.977 INFO:teuthology.orchestra.run.smithi131.stdout:(129/159): python3-jaraco-text-3.2.0-6.el9.noar 343 kB/s | 20 kB 00:00 2024-05-26T22:45:55.010 INFO:teuthology.orchestra.run.smithi131.stdout:(130/159): python3-kubernetes-26.1.0-2.el9.noar 11 MB/s | 1.0 MB 00:00 2024-05-26T22:45:55.022 INFO:teuthology.orchestra.run.smithi187.stdout:Transaction test succeeded. 2024-05-26T22:45:55.023 INFO:teuthology.orchestra.run.smithi187.stdout:Running transaction 2024-05-26T22:45:55.035 INFO:teuthology.orchestra.run.smithi131.stdout:(131/159): python3-logutils-0.3.5-21.el9.noarch 570 kB/s | 46 kB 00:00 2024-05-26T22:45:55.061 INFO:teuthology.orchestra.run.smithi131.stdout:(132/159): python3-more-itertools-8.12.0-2.el9. 1.6 MB/s | 79 kB 00:00 2024-05-26T22:45:55.086 INFO:teuthology.orchestra.run.smithi131.stdout:(133/159): python3-natsort-7.1.1-5.el9.noarch.r 775 kB/s | 58 kB 00:00 2024-05-26T22:45:55.128 INFO:teuthology.orchestra.run.smithi131.stdout:(134/159): python3-pecan-1.4.2-3.el9.noarch.rpm 2.9 MB/s | 272 kB 00:00 2024-05-26T22:45:55.153 INFO:teuthology.orchestra.run.smithi131.stdout:(135/159): python3-portend-3.1.0-2.el9.noarch.r 186 kB/s | 16 kB 00:00 2024-05-26T22:45:55.178 INFO:teuthology.orchestra.run.smithi131.stdout:(136/159): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.8 MB/s | 90 kB 00:00 2024-05-26T22:45:55.203 INFO:teuthology.orchestra.run.smithi131.stdout:(137/159): python3-pyperclip-1.8.2-2.el9.noarch 381 kB/s | 28 kB 00:00 2024-05-26T22:45:55.229 INFO:teuthology.orchestra.run.smithi131.stdout:(138/159): python3-repoze-lru-0.7-16.el9.noarch 418 kB/s | 31 kB 00:00 2024-05-26T22:45:55.263 INFO:teuthology.orchestra.run.smithi131.stdout:(139/159): python3-routes-2.5.1-5.el9.noarch.rp 3.2 MB/s | 188 kB 00:00 2024-05-26T22:45:55.288 INFO:teuthology.orchestra.run.smithi131.stdout:(140/159): python3-rsa-4.9-2.el9.noarch.rpm 718 kB/s | 59 kB 00:00 2024-05-26T22:45:55.313 INFO:teuthology.orchestra.run.smithi131.stdout:(141/159): python3-tempora-5.0.0-2.el9.noarch.r 439 kB/s | 36 kB 00:00 2024-05-26T22:45:55.338 INFO:teuthology.orchestra.run.smithi131.stdout:(142/159): python3-typing-extensions-4.4.0-2.el 691 kB/s | 51 kB 00:00 2024-05-26T22:45:55.372 INFO:teuthology.orchestra.run.smithi131.stdout:(143/159): python3-webob-1.8.7-6.el9.noarch.rpm 3.9 MB/s | 230 kB 00:00 2024-05-26T22:45:55.406 INFO:teuthology.orchestra.run.smithi131.stdout:(144/159): python3-websocket-client-1.2.3-2.el9 991 kB/s | 90 kB 00:00 2024-05-26T22:45:55.439 INFO:teuthology.orchestra.run.smithi131.stdout:(145/159): python3-werkzeug-2.0.3-3.el9.1.noarc 4.3 MB/s | 427 kB 00:00 2024-05-26T22:45:55.456 INFO:teuthology.orchestra.run.smithi131.stdout:(146/159): python3-zc-lockfile-2.0-10.el9.noarc 255 kB/s | 20 kB 00:00 2024-05-26T22:45:55.489 INFO:teuthology.orchestra.run.smithi131.stdout:(147/159): re2-20211101-3.el9.x86_64.rpm 3.8 MB/s | 192 kB 00:00 2024-05-26T22:45:55.557 INFO:teuthology.orchestra.run.smithi131.stdout:(148/159): thrift-0.15.0-2.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-05-26T22:45:55.757 INFO:teuthology.orchestra.run.smithi131.stdout:(149/159): librados2-19.0.0-2857.g0ef46a24.el9. 11 MB/s | 3.4 MB 00:00 2024-05-26T22:45:55.841 INFO:teuthology.orchestra.run.smithi131.stdout:(150/159): NetworkManager-1.47.90-1.el9.x86_64. 8.2 MB/s | 2.3 MB 00:00 2024-05-26T22:45:55.891 INFO:teuthology.orchestra.run.smithi131.stdout:(151/159): NetworkManager-team-1.47.90-1.el9.x8 844 kB/s | 41 kB 00:00 2024-05-26T22:45:55.949 INFO:teuthology.orchestra.run.smithi131.stdout:(152/159): NetworkManager-tui-1.47.90-1.el9.x86 4.2 MB/s | 248 kB 00:00 2024-05-26T22:45:56.000 INFO:teuthology.orchestra.run.smithi131.stdout:(153/159): elfutils-debuginfod-client-0.191-4.e 742 kB/s | 37 kB 00:00 2024-05-26T22:45:56.067 INFO:teuthology.orchestra.run.smithi131.stdout:(154/159): NetworkManager-libnm-1.47.90-1.el9.x 5.8 MB/s | 1.8 MB 00:00 2024-05-26T22:45:56.101 INFO:teuthology.orchestra.run.smithi131.stdout:(155/159): elfutils-libelf-0.191-4.el9.x86_64.r 2.0 MB/s | 207 kB 00:00 2024-05-26T22:45:56.151 INFO:teuthology.orchestra.run.smithi131.stdout:(156/159): elfutils-libs-0.191-4.el9.x86_64.rpm 3.1 MB/s | 260 kB 00:00 2024-05-26T22:45:56.218 INFO:teuthology.orchestra.run.smithi131.stdout:(157/159): openssl-3.2.1-1.el9.x86_64.rpm 12 MB/s | 1.3 MB 00:00 2024-05-26T22:45:56.326 INFO:teuthology.orchestra.run.smithi131.stdout:(158/159): openssl-libs-3.2.1-1.el9.x86_64.rpm 14 MB/s | 2.4 MB 00:00 2024-05-26T22:45:56.469 INFO:teuthology.orchestra.run.smithi131.stdout:(159/159): librbd1-19.0.0-2857.g0ef46a24.el9.x8 3.2 MB/s | 3.1 MB 00:00 2024-05-26T22:45:56.477 INFO:teuthology.orchestra.run.smithi131.stdout:-------------------------------------------------------------------------------- 2024-05-26T22:45:56.477 INFO:teuthology.orchestra.run.smithi131.stdout:Total 15 MB/s | 225 MB 00:15 2024-05-26T22:45:59.689 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction check 2024-05-26T22:45:59.939 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction check succeeded. 2024-05-26T22:45:59.939 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction test 2024-05-26T22:46:01.810 INFO:teuthology.orchestra.run.smithi187.stdout: Preparing : 1/1 2024-05-26T22:46:01.931 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/170 2024-05-26T22:46:02.037 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/170 2024-05-26T22:46:02.170 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/170 2024-05-26T22:46:02.228 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/170 2024-05-26T22:46:02.265 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/170 2024-05-26T22:46:02.395 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/170 2024-05-26T22:46:02.661 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction test succeeded. 2024-05-26T22:46:02.661 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction 2024-05-26T22:46:02.822 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/170 2024-05-26T22:46:02.938 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : librados2-2:19.0.0-2857.g0ef46a24.el9.x86_64 8/170 2024-05-26T22:46:03.184 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: librados2-2:19.0.0-2857.g0ef46a24.el9.x86_64 8/170 2024-05-26T22:46:03.213 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libcephfs2-2:19.0.0-2857.g0ef46a24.el9.x86_64 9/170 2024-05-26T22:46:03.400 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: libcephfs2-2:19.0.0-2857.g0ef46a24.el9.x86_64 9/170 2024-05-26T22:46:03.446 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-rados-2:19.0.0-2857.g0ef46a24.el9.x86_64 10/170 2024-05-26T22:46:03.527 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/170 2024-05-26T22:46:03.890 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/170 2024-05-26T22:46:03.957 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/170 2024-05-26T22:46:03.994 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libcephsqlite-2:19.0.0-2857.g0ef46a24.el9.x86_64 14/170 2024-05-26T22:46:04.148 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2857.g0ef46a24.el9.x86_64 14/170 2024-05-26T22:46:04.179 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libradosstriper1-2:19.0.0-2857.g0ef46a24.el9.x86 15/170 2024-05-26T22:46:04.284 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2857.g0ef46a24.el9.x86 15/170 2024-05-26T22:46:04.330 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-05-26T22:46:04.398 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-05-26T22:46:04.684 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/170 2024-05-26T22:46:05.019 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/170 2024-05-26T22:46:05.244 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 18/170 2024-05-26T22:46:05.358 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 18/170 2024-05-26T22:46:05.522 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-26T22:46:05.630 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-26T22:46:05.973 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-26T22:46:06.203 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : re2-1:20211101-3.el9.x86_64 20/170 2024-05-26T22:46:06.324 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/170 2024-05-26T22:46:06.383 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 22/170 2024-05-26T22:46:06.429 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : liboath-2.6.7-2.el9.x86_64 23/170 2024-05-26T22:46:06.535 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/170 2024-05-26T22:46:06.582 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 25/170 2024-05-26T22:46:06.646 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/170 2024-05-26T22:46:06.713 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/170 2024-05-26T22:46:06.753 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/170 2024-05-26T22:46:06.869 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/170 2024-05-26T22:46:06.938 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 30/170 2024-05-26T22:46:07.012 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/170 2024-05-26T22:46:07.071 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 32/170 2024-05-26T22:46:07.172 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 33/170 2024-05-26T22:46:07.216 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-ceph-common-2:19.0.0-2857.g0ef46a24.el9. 34/170 2024-05-26T22:46:07.271 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-ceph-argparse-2:19.0.0-2857.g0ef46a24.el 35/170 2024-05-26T22:46:07.306 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-cephfs-2:19.0.0-2857.g0ef46a24.el9.x86_6 36/170 2024-05-26T22:46:07.454 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/170 2024-05-26T22:46:07.512 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/170 2024-05-26T22:46:07.741 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/170 2024-05-26T22:46:07.922 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/170 2024-05-26T22:46:07.987 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/170 2024-05-26T22:46:08.712 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/170 2024-05-26T22:46:08.818 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/170 2024-05-26T22:46:08.863 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/170 2024-05-26T22:46:08.902 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/170 2024-05-26T22:46:08.968 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 46/170 2024-05-26T22:46:09.055 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : librados-devel-2:19.0.0-2857.g0ef46a24.el9.x86_6 47/170 2024-05-26T22:46:09.122 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 48/170 2024-05-26T22:46:09.317 INFO:teuthology.orchestra.run.smithi131.stdout: Preparing : 1/1 2024-05-26T22:46:09.324 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 49/170 2024-05-26T22:46:09.422 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/170 2024-05-26T22:46:09.432 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 50/170 2024-05-26T22:46:09.483 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/170 2024-05-26T22:46:09.512 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 51/170 2024-05-26T22:46:09.605 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/170 2024-05-26T22:46:09.611 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/170 2024-05-26T22:46:09.649 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 53/170 2024-05-26T22:46:09.665 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/170 2024-05-26T22:46:09.696 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/170 2024-05-26T22:46:09.712 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 54/170 2024-05-26T22:46:09.784 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/170 2024-05-26T22:46:09.786 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/170 2024-05-26T22:46:09.830 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/170 2024-05-26T22:46:09.892 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/170 2024-05-26T22:46:09.942 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/170 2024-05-26T22:46:10.128 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/170 2024-05-26T22:46:10.173 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/170 2024-05-26T22:46:10.225 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 61/170 2024-05-26T22:46:10.234 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/170 2024-05-26T22:46:10.328 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : librados2-2:19.0.0-2857.g0ef46a24.el9.x86_64 8/170 2024-05-26T22:46:10.346 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/170 2024-05-26T22:46:10.396 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 63/170 2024-05-26T22:46:10.451 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/170 2024-05-26T22:46:10.487 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/170 2024-05-26T22:46:10.499 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: librados2-2:19.0.0-2857.g0ef46a24.el9.x86_64 8/170 2024-05-26T22:46:10.520 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/170 2024-05-26T22:46:10.538 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libcephfs2-2:19.0.0-2857.g0ef46a24.el9.x86_64 9/170 2024-05-26T22:46:10.667 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: libcephfs2-2:19.0.0-2857.g0ef46a24.el9.x86_64 9/170 2024-05-26T22:46:10.712 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-rados-2:19.0.0-2857.g0ef46a24.el9.x86_64 10/170 2024-05-26T22:46:10.735 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/170 2024-05-26T22:46:10.803 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/170 2024-05-26T22:46:10.922 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/170 2024-05-26T22:46:10.977 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/170 2024-05-26T22:46:11.022 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/170 2024-05-26T22:46:11.095 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/170 2024-05-26T22:46:11.147 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/170 2024-05-26T22:46:11.185 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/170 2024-05-26T22:46:11.195 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/170 2024-05-26T22:46:11.246 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/170 2024-05-26T22:46:11.270 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/170 2024-05-26T22:46:11.292 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/170 2024-05-26T22:46:11.302 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libcephsqlite-2:19.0.0-2857.g0ef46a24.el9.x86_64 14/170 2024-05-26T22:46:11.342 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/170 2024-05-26T22:46:11.438 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/170 2024-05-26T22:46:11.455 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2857.g0ef46a24.el9.x86_64 14/170 2024-05-26T22:46:11.485 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/170 2024-05-26T22:46:11.487 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libradosstriper1-2:19.0.0-2857.g0ef46a24.el9.x86 15/170 2024-05-26T22:46:11.573 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2857.g0ef46a24.el9.x86 15/170 2024-05-26T22:46:11.605 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-05-26T22:46:11.638 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/170 2024-05-26T22:46:11.674 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-05-26T22:46:11.727 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/170 2024-05-26T22:46:11.774 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/170 2024-05-26T22:46:11.843 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/170 2024-05-26T22:46:11.987 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-packaging-20.9-5.el9.noarch 83/170 2024-05-26T22:46:12.007 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/170 2024-05-26T22:46:12.073 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 84/170 2024-05-26T22:46:12.350 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/170 2024-05-26T22:46:12.545 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 18/170 2024-05-26T22:46:12.616 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 18/170 2024-05-26T22:46:12.800 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-26T22:46:12.917 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-26T22:46:13.034 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 85/170 2024-05-26T22:46:13.093 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 86/170 2024-05-26T22:46:13.129 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 87/170 2024-05-26T22:46:13.243 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 88/170 2024-05-26T22:46:13.253 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-26T22:46:13.301 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 89/170 2024-05-26T22:46:13.355 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 90/170 2024-05-26T22:46:13.444 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : re2-1:20211101-3.el9.x86_64 20/170 2024-05-26T22:46:13.478 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 91/170 2024-05-26T22:46:13.529 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 92/170 2024-05-26T22:46:13.569 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/170 2024-05-26T22:46:13.615 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 22/170 2024-05-26T22:46:13.662 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : liboath-2.6.7-2.el9.x86_64 23/170 2024-05-26T22:46:13.755 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/170 2024-05-26T22:46:13.765 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : openblas-0.3.21-2.el9.x86_64 93/170 2024-05-26T22:46:13.800 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 94/170 2024-05-26T22:46:13.810 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 25/170 2024-05-26T22:46:13.872 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/170 2024-05-26T22:46:13.894 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 95/170 2024-05-26T22:46:13.923 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/170 2024-05-26T22:46:13.964 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/170 2024-05-26T22:46:14.079 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/170 2024-05-26T22:46:14.186 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 30/170 2024-05-26T22:46:14.224 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/170 2024-05-26T22:46:14.280 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 32/170 2024-05-26T22:46:14.398 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 33/170 2024-05-26T22:46:14.436 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-ceph-common-2:19.0.0-2857.g0ef46a24.el9. 34/170 2024-05-26T22:46:14.488 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-ceph-argparse-2:19.0.0-2857.g0ef46a24.el 35/170 2024-05-26T22:46:14.526 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-cephfs-2:19.0.0-2857.g0ef46a24.el9.x86_6 36/170 2024-05-26T22:46:14.645 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/170 2024-05-26T22:46:14.693 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 96/170 2024-05-26T22:46:14.726 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/170 2024-05-26T22:46:14.831 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/170 2024-05-26T22:46:14.858 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 97/170 2024-05-26T22:46:15.134 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/170 2024-05-26T22:46:15.208 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/170 2024-05-26T22:46:15.906 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/170 2024-05-26T22:46:16.022 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/170 2024-05-26T22:46:16.074 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/170 2024-05-26T22:46:16.123 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/170 2024-05-26T22:46:16.200 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 46/170 2024-05-26T22:46:16.253 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : librados-devel-2:19.0.0-2857.g0ef46a24.el9.x86_6 47/170 2024-05-26T22:46:16.325 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 48/170 2024-05-26T22:46:16.431 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 98/170 2024-05-26T22:46:16.526 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 99/170 2024-05-26T22:46:16.530 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 49/170 2024-05-26T22:46:16.562 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 100/170 2024-05-26T22:46:16.598 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 101/170 2024-05-26T22:46:16.601 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 50/170 2024-05-26T22:46:16.682 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 51/170 2024-05-26T22:46:16.777 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/170 2024-05-26T22:46:16.818 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 53/170 2024-05-26T22:46:16.857 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 54/170 2024-05-26T22:46:16.922 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/170 2024-05-26T22:46:16.966 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/170 2024-05-26T22:46:16.974 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 102/170 2024-05-26T22:46:17.031 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : librbd1-2:19.0.0-2857.g0ef46a24.el9.x86_64 103/170 2024-05-26T22:46:17.044 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/170 2024-05-26T22:46:17.106 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/170 2024-05-26T22:46:17.179 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: librbd1-2:19.0.0-2857.g0ef46a24.el9.x86_64 103/170 2024-05-26T22:46:17.213 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-rbd-2:19.0.0-2857.g0ef46a24.el9.x86_64 104/170 2024-05-26T22:46:17.273 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 105/170 2024-05-26T22:46:17.321 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/170 2024-05-26T22:46:17.376 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/170 2024-05-26T22:46:17.434 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 61/170 2024-05-26T22:46:17.561 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/170 2024-05-26T22:46:17.609 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 63/170 2024-05-26T22:46:17.655 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/170 2024-05-26T22:46:17.691 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/170 2024-05-26T22:46:17.724 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/170 2024-05-26T22:46:17.800 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 106/170 2024-05-26T22:46:17.834 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : librgw2-2:19.0.0-2857.g0ef46a24.el9.x86_64 107/170 2024-05-26T22:46:17.909 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: librgw2-2:19.0.0-2857.g0ef46a24.el9.x86_64 107/170 2024-05-26T22:46:17.932 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/170 2024-05-26T22:46:17.935 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-rgw-2:19.0.0-2857.g0ef46a24.el9.x86_64 108/170 2024-05-26T22:46:18.122 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/170 2024-05-26T22:46:18.180 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/170 2024-05-26T22:46:18.219 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/170 2024-05-26T22:46:18.265 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/170 2024-05-26T22:46:18.324 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/170 2024-05-26T22:46:18.374 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/170 2024-05-26T22:46:18.425 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/170 2024-05-26T22:46:18.471 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/170 2024-05-26T22:46:18.520 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/170 2024-05-26T22:46:18.586 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/170 2024-05-26T22:46:18.628 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/170 2024-05-26T22:46:18.768 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/170 2024-05-26T22:46:18.834 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/170 2024-05-26T22:46:18.885 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/170 2024-05-26T22:46:18.958 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/170 2024-05-26T22:46:19.123 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-packaging-20.9-5.el9.noarch 83/170 2024-05-26T22:46:19.199 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 84/170 2024-05-26T22:46:20.173 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 85/170 2024-05-26T22:46:20.231 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 86/170 2024-05-26T22:46:20.264 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 87/170 2024-05-26T22:46:20.388 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 88/170 2024-05-26T22:46:20.477 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 89/170 2024-05-26T22:46:20.538 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 90/170 2024-05-26T22:46:20.643 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-common-2:19.0.0-2857.g0ef46a24.el9.x86_64 109/170 2024-05-26T22:46:20.658 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 91/170 2024-05-26T22:46:20.682 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-common-2:19.0.0-2857.g0ef46a24.el9.x86_64 109/170 2024-05-26T22:46:20.708 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 92/170 2024-05-26T22:46:20.952 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : openblas-0.3.21-2.el9.x86_64 93/170 2024-05-26T22:46:20.979 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 94/170 2024-05-26T22:46:21.087 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 95/170 2024-05-26T22:46:21.314 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-common-2:19.0.0-2857.g0ef46a24.el9.x86_64 109/170 2024-05-26T22:46:21.361 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-base-2:19.0.0-2857.g0ef46a24.el9.x86_64 110/170 2024-05-26T22:46:21.561 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-base-2:19.0.0-2857.g0ef46a24.el9.x86_64 110/170 2024-05-26T22:46:21.561 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-26T22:46:21.561 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-26T22:46:21.561 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-05-26T22:46:21.595 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-selinux-2:19.0.0-2857.g0ef46a24.el9.x86_64 111/170 2024-05-26T22:46:21.923 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 96/170 2024-05-26T22:46:22.069 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 97/170 2024-05-26T22:46:23.690 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 98/170 2024-05-26T22:46:23.786 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 99/170 2024-05-26T22:46:23.830 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 100/170 2024-05-26T22:46:23.874 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 101/170 2024-05-26T22:46:24.272 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 102/170 2024-05-26T22:46:24.315 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : librbd1-2:19.0.0-2857.g0ef46a24.el9.x86_64 103/170 2024-05-26T22:46:24.436 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: librbd1-2:19.0.0-2857.g0ef46a24.el9.x86_64 103/170 2024-05-26T22:46:24.491 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-rbd-2:19.0.0-2857.g0ef46a24.el9.x86_64 104/170 2024-05-26T22:46:24.556 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 105/170 2024-05-26T22:46:25.093 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 106/170 2024-05-26T22:46:25.126 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : librgw2-2:19.0.0-2857.g0ef46a24.el9.x86_64 107/170 2024-05-26T22:46:25.227 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: librgw2-2:19.0.0-2857.g0ef46a24.el9.x86_64 107/170 2024-05-26T22:46:25.253 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-rgw-2:19.0.0-2857.g0ef46a24.el9.x86_64 108/170 2024-05-26T22:46:27.881 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-common-2:19.0.0-2857.g0ef46a24.el9.x86_64 109/170 2024-05-26T22:46:27.916 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-common-2:19.0.0-2857.g0ef46a24.el9.x86_64 109/170 2024-05-26T22:46:28.552 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-common-2:19.0.0-2857.g0ef46a24.el9.x86_64 109/170 2024-05-26T22:46:28.594 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-base-2:19.0.0-2857.g0ef46a24.el9.x86_64 110/170 2024-05-26T22:46:28.782 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-base-2:19.0.0-2857.g0ef46a24.el9.x86_64 110/170 2024-05-26T22:46:28.783 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-26T22:46:28.783 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-26T22:46:28.783 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:46:28.812 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-selinux-2:19.0.0-2857.g0ef46a24.el9.x86_64 111/170 2024-05-26T22:46:34.327 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2857.g0ef46a24.el9.x86_64 111/170 2024-05-26T22:46:34.328 INFO:teuthology.orchestra.run.smithi187.stdout:skipping the directory /sys 2024-05-26T22:46:34.328 INFO:teuthology.orchestra.run.smithi187.stdout:skipping the directory /proc 2024-05-26T22:46:34.328 INFO:teuthology.orchestra.run.smithi187.stdout:skipping the directory /mnt 2024-05-26T22:46:34.328 INFO:teuthology.orchestra.run.smithi187.stdout:skipping the directory /var/tmp 2024-05-26T22:46:34.328 INFO:teuthology.orchestra.run.smithi187.stdout:skipping the directory /home 2024-05-26T22:46:34.329 INFO:teuthology.orchestra.run.smithi187.stdout:skipping the directory /root 2024-05-26T22:46:34.329 INFO:teuthology.orchestra.run.smithi187.stdout:skipping the directory /tmp 2024-05-26T22:46:34.329 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-05-26T22:46:34.562 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-volume-2:19.0.0-2857.g0ef46a24.el9.noarch 112/170 2024-05-26T22:46:34.614 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-volume-2:19.0.0-2857.g0ef46a24.el9.noarch 112/170 2024-05-26T22:46:34.614 INFO:teuthology.orchestra.run.smithi187.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-26T22:46:34.615 INFO:teuthology.orchestra.run.smithi187.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-26T22:46:34.615 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-05-26T22:46:36.307 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-osd-2:19.0.0-2857.g0ef46a24.el9.x86_64 113/170 2024-05-26T22:46:36.385 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-osd-2:19.0.0-2857.g0ef46a24.el9.x86_64 113/170 2024-05-26T22:46:36.385 INFO:teuthology.orchestra.run.smithi187.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-26T22:46:36.385 INFO:teuthology.orchestra.run.smithi187.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-26T22:46:36.385 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-26T22:46:36.386 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-26T22:46:36.386 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-05-26T22:46:36.640 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-mds-2:19.0.0-2857.g0ef46a24.el9.x86_64 114/170 2024-05-26T22:46:36.711 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-mds-2:19.0.0-2857.g0ef46a24.el9.x86_64 114/170 2024-05-26T22:46:36.711 INFO:teuthology.orchestra.run.smithi187.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-26T22:46:36.711 INFO:teuthology.orchestra.run.smithi187.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-26T22:46:36.711 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-26T22:46:36.711 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-26T22:46:36.712 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-05-26T22:46:37.182 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-mon-2:19.0.0-2857.g0ef46a24.el9.x86_64 115/170 2024-05-26T22:46:37.252 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-mon-2:19.0.0-2857.g0ef46a24.el9.x86_64 115/170 2024-05-26T22:46:37.252 INFO:teuthology.orchestra.run.smithi187.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-26T22:46:37.252 INFO:teuthology.orchestra.run.smithi187.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-26T22:46:37.252 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-26T22:46:37.252 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-26T22:46:37.252 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-05-26T22:46:37.382 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : automake-1.16.2-8.el9.noarch 116/170 2024-05-26T22:46:37.451 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-ply-3.11-14.el9.noarch 117/170 2024-05-26T22:46:37.524 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 118/170 2024-05-26T22:46:37.752 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 119/170 2024-05-26T22:46:37.820 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 120/170 2024-05-26T22:46:37.905 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/170 2024-05-26T22:46:38.031 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 122/170 2024-05-26T22:46:38.187 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 123/170 2024-05-26T22:46:38.287 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 124/170 2024-05-26T22:46:38.344 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 125/170 2024-05-26T22:46:38.377 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : fuse-2.9.9-15.el9.x86_64 126/170 2024-05-26T22:46:39.035 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: cephadm-2:19.0.0-2857.g0ef46a24.el9.noarch 127/170 2024-05-26T22:46:39.084 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : cephadm-2:19.0.0-2857.g0ef46a24.el9.noarch 127/170 2024-05-26T22:46:39.140 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2857.g0ef46a24.e 128/170 2024-05-26T22:46:39.241 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2857.g0ef46a24. 129/170 2024-05-26T22:46:39.298 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2857.g0ef46a24.el9.noa 130/170 2024-05-26T22:46:40.604 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2857.g0ef46a24.el9.noa 130/170 2024-05-26T22:46:40.658 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2857.g0ef46a24.el9.n 131/170 2024-05-26T22:46:41.754 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2857.g0ef46a24.el9.n 131/170 2024-05-26T22:46:41.791 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2857.g0ef 132/170 2024-05-26T22:46:41.936 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2857.g0ef 132/170 2024-05-26T22:46:42.068 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2857.g0ef46a24.el9.x86_64 111/170 2024-05-26T22:46:42.068 INFO:teuthology.orchestra.run.smithi131.stdout:skipping the directory /sys 2024-05-26T22:46:42.068 INFO:teuthology.orchestra.run.smithi131.stdout:skipping the directory /proc 2024-05-26T22:46:42.068 INFO:teuthology.orchestra.run.smithi131.stdout:skipping the directory /mnt 2024-05-26T22:46:42.068 INFO:teuthology.orchestra.run.smithi131.stdout:skipping the directory /var/tmp 2024-05-26T22:46:42.068 INFO:teuthology.orchestra.run.smithi131.stdout:skipping the directory /home 2024-05-26T22:46:42.068 INFO:teuthology.orchestra.run.smithi131.stdout:skipping the directory /root 2024-05-26T22:46:42.068 INFO:teuthology.orchestra.run.smithi131.stdout:skipping the directory /tmp 2024-05-26T22:46:42.068 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:46:42.151 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2857.g0ef46a24.el 133/170 2024-05-26T22:46:42.185 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-mgr-2:19.0.0-2857.g0ef46a24.el9.x86_64 134/170 2024-05-26T22:46:42.254 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2857.g0ef46a24.el9.x86_64 134/170 2024-05-26T22:46:42.254 INFO:teuthology.orchestra.run.smithi187.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-26T22:46:42.254 INFO:teuthology.orchestra.run.smithi187.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-26T22:46:42.254 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-26T22:46:42.254 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-26T22:46:42.254 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-05-26T22:46:42.296 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-volume-2:19.0.0-2857.g0ef46a24.el9.noarch 112/170 2024-05-26T22:46:42.336 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-mgr-rook-2:19.0.0-2857.g0ef46a24.el9.noarch 135/170 2024-05-26T22:46:42.344 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-volume-2:19.0.0-2857.g0ef46a24.el9.noarch 112/170 2024-05-26T22:46:42.344 INFO:teuthology.orchestra.run.smithi131.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-26T22:46:42.344 INFO:teuthology.orchestra.run.smithi131.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-26T22:46:42.345 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:46:42.375 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2857.g0ef46a24.el9.noarch 135/170 2024-05-26T22:46:42.510 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-2:19.0.0-2857.g0ef46a24.el9.x86_64 136/170 2024-05-26T22:46:42.582 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-fuse-2:19.0.0-2857.g0ef46a24.el9.x86_64 137/170 2024-05-26T22:46:43.578 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libtool-2.4.6-45.el9.x86_64 138/170 2024-05-26T22:46:43.614 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-radosgw-2:19.0.0-2857.g0ef46a24.el9.x86_64 139/170 2024-05-26T22:46:43.709 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2857.g0ef46a24.el9.x86_64 139/170 2024-05-26T22:46:43.710 INFO:teuthology.orchestra.run.smithi187.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-26T22:46:43.710 INFO:teuthology.orchestra.run.smithi187.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-26T22:46:43.710 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-26T22:46:43.710 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-26T22:46:43.710 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-05-26T22:46:43.772 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2857.g0ef46 140/170 2024-05-26T22:46:43.841 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2857.g0ef46 140/170 2024-05-26T22:46:43.841 INFO:teuthology.orchestra.run.smithi187.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-26T22:46:43.841 INFO:teuthology.orchestra.run.smithi187.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-26T22:46:43.841 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-05-26T22:46:43.906 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : cephfs-mirror-2:19.0.0-2857.g0ef46a24.el9.x86_64 141/170 2024-05-26T22:46:43.962 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-osd-2:19.0.0-2857.g0ef46a24.el9.x86_64 113/170 2024-05-26T22:46:43.997 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2857.g0ef46a24.el9.x86_64 141/170 2024-05-26T22:46:43.997 INFO:teuthology.orchestra.run.smithi187.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-26T22:46:43.997 INFO:teuthology.orchestra.run.smithi187.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-26T22:46:43.997 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-26T22:46:43.997 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-26T22:46:43.998 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-05-26T22:46:44.050 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-osd-2:19.0.0-2857.g0ef46a24.el9.x86_64 113/170 2024-05-26T22:46:44.050 INFO:teuthology.orchestra.run.smithi131.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-26T22:46:44.050 INFO:teuthology.orchestra.run.smithi131.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-26T22:46:44.050 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-26T22:46:44.050 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-26T22:46:44.050 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:46:44.304 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mds-2:19.0.0-2857.g0ef46a24.el9.x86_64 114/170 2024-05-26T22:46:44.373 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-mds-2:19.0.0-2857.g0ef46a24.el9.x86_64 114/170 2024-05-26T22:46:44.373 INFO:teuthology.orchestra.run.smithi131.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-26T22:46:44.373 INFO:teuthology.orchestra.run.smithi131.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-26T22:46:44.373 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-26T22:46:44.373 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-26T22:46:44.373 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:46:44.390 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : rbd-mirror-2:19.0.0-2857.g0ef46a24.el9.x86_64 142/170 2024-05-26T22:46:44.466 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2857.g0ef46a24.el9.x86_64 142/170 2024-05-26T22:46:44.466 INFO:teuthology.orchestra.run.smithi187.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-26T22:46:44.466 INFO:teuthology.orchestra.run.smithi187.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-26T22:46:44.467 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-26T22:46:44.467 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-26T22:46:44.467 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-05-26T22:46:44.855 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mon-2:19.0.0-2857.g0ef46a24.el9.x86_64 115/170 2024-05-26T22:46:44.931 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-mon-2:19.0.0-2857.g0ef46a24.el9.x86_64 115/170 2024-05-26T22:46:44.931 INFO:teuthology.orchestra.run.smithi131.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-26T22:46:44.931 INFO:teuthology.orchestra.run.smithi131.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-26T22:46:44.931 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-26T22:46:44.932 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-26T22:46:44.932 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:46:45.064 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : automake-1.16.2-8.el9.noarch 116/170 2024-05-26T22:46:45.133 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-ply-3.11-14.el9.noarch 117/170 2024-05-26T22:46:45.222 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 118/170 2024-05-26T22:46:45.470 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 119/170 2024-05-26T22:46:45.526 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 120/170 2024-05-26T22:46:45.628 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/170 2024-05-26T22:46:45.739 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 122/170 2024-05-26T22:46:45.906 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 123/170 2024-05-26T22:46:45.967 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 124/170 2024-05-26T22:46:46.008 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 125/170 2024-05-26T22:46:46.040 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : fuse-2.9.9-15.el9.x86_64 126/170 2024-05-26T22:46:46.683 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: cephadm-2:19.0.0-2857.g0ef46a24.el9.noarch 127/170 2024-05-26T22:46:46.721 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : cephadm-2:19.0.0-2857.g0ef46a24.el9.noarch 127/170 2024-05-26T22:46:46.771 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2857.g0ef46a24.e 128/170 2024-05-26T22:46:46.876 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2857.g0ef46a24. 129/170 2024-05-26T22:46:46.908 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2857.g0ef46a24.el9.noa 130/170 2024-05-26T22:46:48.221 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2857.g0ef46a24.el9.noa 130/170 2024-05-26T22:46:48.268 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2857.g0ef46a24.el9.n 131/170 2024-05-26T22:46:49.366 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2857.g0ef46a24.el9.n 131/170 2024-05-26T22:46:49.387 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-test-2:19.0.0-2857.g0ef46a24.el9.x86_64 143/170 2024-05-26T22:46:49.401 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2857.g0ef 132/170 2024-05-26T22:46:49.433 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : rbd-fuse-2:19.0.0-2857.g0ef46a24.el9.x86_64 144/170 2024-05-26T22:46:49.488 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : rbd-nbd-2:19.0.0-2857.g0ef46a24.el9.x86_64 145/170 2024-05-26T22:46:49.545 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 146/170 2024-05-26T22:46:49.548 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2857.g0ef 132/170 2024-05-26T22:46:49.591 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 147/170 2024-05-26T22:46:49.754 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2857.g0ef46a24.el 133/170 2024-05-26T22:46:49.787 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mgr-2:19.0.0-2857.g0ef46a24.el9.x86_64 134/170 2024-05-26T22:46:49.886 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2857.g0ef46a24.el9.x86_64 134/170 2024-05-26T22:46:49.886 INFO:teuthology.orchestra.run.smithi131.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-26T22:46:49.886 INFO:teuthology.orchestra.run.smithi131.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-26T22:46:49.886 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-26T22:46:49.886 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-26T22:46:49.886 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:46:49.915 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : cephfs-shell-2:19.0.0-2857.g0ef46a24.el9.x86_64 148/170 2024-05-26T22:46:49.954 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mgr-rook-2:19.0.0-2857.g0ef46a24.el9.noarch 135/170 2024-05-26T22:46:49.966 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 149/170 2024-05-26T22:46:49.994 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2857.g0ef46a24.el9.noarch 135/170 2024-05-26T22:46:50.032 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libcephfs-devel-2:19.0.0-2857.g0ef46a24.el9.x86_ 150/170 2024-05-26T22:46:50.074 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 151/170 2024-05-26T22:46:50.128 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : NetworkManager-team-1:1.47.90-1.el9.x86_64 152/170 2024-05-26T22:46:50.128 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-2:19.0.0-2857.g0ef46a24.el9.x86_64 136/170 2024-05-26T22:46:50.192 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : NetworkManager-tui-1:1.47.90-1.el9.x86_64 153/170 2024-05-26T22:46:50.214 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-fuse-2:19.0.0-2857.g0ef46a24.el9.x86_64 137/170 2024-05-26T22:46:51.131 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-26T22:46:51.179 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-26T22:46:51.235 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-26T22:46:51.241 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libtool-2.4.6-45.el9.x86_64 138/170 2024-05-26T22:46:51.307 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-radosgw-2:19.0.0-2857.g0ef46a24.el9.x86_64 139/170 2024-05-26T22:46:51.372 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2857.g0ef46a24.el9.x86_64 139/170 2024-05-26T22:46:51.372 INFO:teuthology.orchestra.run.smithi131.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-26T22:46:51.373 INFO:teuthology.orchestra.run.smithi131.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-26T22:46:51.373 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-26T22:46:51.373 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-26T22:46:51.373 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:46:51.433 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2857.g0ef46 140/170 2024-05-26T22:46:51.501 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2857.g0ef46 140/170 2024-05-26T22:46:51.501 INFO:teuthology.orchestra.run.smithi131.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-26T22:46:51.501 INFO:teuthology.orchestra.run.smithi131.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-26T22:46:51.501 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:46:51.567 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : cephfs-mirror-2:19.0.0-2857.g0ef46a24.el9.x86_64 141/170 2024-05-26T22:46:51.633 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2857.g0ef46a24.el9.x86_64 141/170 2024-05-26T22:46:51.633 INFO:teuthology.orchestra.run.smithi131.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-26T22:46:51.633 INFO:teuthology.orchestra.run.smithi131.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-26T22:46:51.633 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-26T22:46:51.633 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-26T22:46:51.633 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:46:51.967 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : rbd-mirror-2:19.0.0-2857.g0ef46a24.el9.x86_64 142/170 2024-05-26T22:46:52.034 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2857.g0ef46a24.el9.x86_64 142/170 2024-05-26T22:46:52.034 INFO:teuthology.orchestra.run.smithi131.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-26T22:46:52.034 INFO:teuthology.orchestra.run.smithi131.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-26T22:46:52.034 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-26T22:46:52.034 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-26T22:46:52.034 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:46:52.562 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : cephfs-top-2:19.0.0-2857.g0ef46a24.el9.noarch 155/170 2024-05-26T22:46:52.677 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 156/170 2024-05-26T22:46:52.776 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 157/170 2024-05-26T22:46:52.975 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : flex-2.6.4-9.el9.x86_64 158/170 2024-05-26T22:46:53.013 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : bison-3.7.4-5.el9.x86_64 159/170 2024-05-26T22:46:53.077 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-05-26T22:46:53.113 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 161/170 2024-05-26T22:46:53.165 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 162/170 2024-05-26T22:46:53.165 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-05-26T22:46:53.322 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-05-26T22:46:53.367 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 164/170 2024-05-26T22:46:53.367 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 165/170 2024-05-26T22:46:53.495 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 165/170 2024-05-26T22:46:53.534 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 166/170 2024-05-26T22:46:53.568 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-26T22:46:53.568 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-26T22:46:53.905 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-26T22:46:53.905 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-05-26T22:46:54.059 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-05-26T22:46:54.120 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 169/170 2024-05-26T22:46:54.241 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-05-26T22:46:54.488 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-05-26T22:46:54.488 INFO:teuthology.orchestra.run.smithi187.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-05-26T22:46:54.488 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-05-26T22:46:56.946 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-test-2:19.0.0-2857.g0ef46a24.el9.x86_64 143/170 2024-05-26T22:46:56.988 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : rbd-fuse-2:19.0.0-2857.g0ef46a24.el9.x86_64 144/170 2024-05-26T22:46:57.037 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : rbd-nbd-2:19.0.0-2857.g0ef46a24.el9.x86_64 145/170 2024-05-26T22:46:57.088 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 146/170 2024-05-26T22:46:57.125 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 147/170 2024-05-26T22:46:57.409 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : cephfs-shell-2:19.0.0-2857.g0ef46a24.el9.x86_64 148/170 2024-05-26T22:46:57.485 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 149/170 2024-05-26T22:46:57.528 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libcephfs-devel-2:19.0.0-2857.g0ef46a24.el9.x86_ 150/170 2024-05-26T22:46:57.568 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 151/170 2024-05-26T22:46:57.617 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : NetworkManager-team-1:1.47.90-1.el9.x86_64 152/170 2024-05-26T22:46:57.686 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : NetworkManager-tui-1:1.47.90-1.el9.x86_64 153/170 2024-05-26T22:46:58.563 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-26T22:46:58.615 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-26T22:46:58.669 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-26T22:46:59.958 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : cephfs-top-2:19.0.0-2857.g0ef46a24.el9.noarch 155/170 2024-05-26T22:47:00.080 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 156/170 2024-05-26T22:47:00.143 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 157/170 2024-05-26T22:47:00.302 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : flex-2.6.4-9.el9.x86_64 158/170 2024-05-26T22:47:00.340 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : bison-3.7.4-5.el9.x86_64 159/170 2024-05-26T22:47:00.373 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-05-26T22:47:00.408 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 161/170 2024-05-26T22:47:00.452 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 162/170 2024-05-26T22:47:00.452 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-05-26T22:47:00.478 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-2:19.0.0-2857.g0ef46a24.el9.x86_64 1/170 2024-05-26T22:47:00.478 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-base-2:19.0.0-2857.g0ef46a24.el9.x86_64 2/170 2024-05-26T22:47:00.479 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-common-2:19.0.0-2857.g0ef46a24.el9.x86_64 3/170 2024-05-26T22:47:00.479 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-fuse-2:19.0.0-2857.g0ef46a24.el9.x86_64 4/170 2024-05-26T22:47:00.479 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2857.g0ef46 5/170 2024-05-26T22:47:00.479 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-mds-2:19.0.0-2857.g0ef46a24.el9.x86_64 6/170 2024-05-26T22:47:00.479 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-mgr-2:19.0.0-2857.g0ef46a24.el9.x86_64 7/170 2024-05-26T22:47:00.479 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-mon-2:19.0.0-2857.g0ef46a24.el9.x86_64 8/170 2024-05-26T22:47:00.479 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-osd-2:19.0.0-2857.g0ef46a24.el9.x86_64 9/170 2024-05-26T22:47:00.479 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-radosgw-2:19.0.0-2857.g0ef46a24.el9.x86_64 10/170 2024-05-26T22:47:00.479 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-selinux-2:19.0.0-2857.g0ef46a24.el9.x86_64 11/170 2024-05-26T22:47:00.479 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-test-2:19.0.0-2857.g0ef46a24.el9.x86_64 12/170 2024-05-26T22:47:00.479 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : cephfs-mirror-2:19.0.0-2857.g0ef46a24.el9.x86_64 13/170 2024-05-26T22:47:00.479 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : cephfs-shell-2:19.0.0-2857.g0ef46a24.el9.x86_64 14/170 2024-05-26T22:47:00.480 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libcephfs-devel-2:19.0.0-2857.g0ef46a24.el9.x86_ 15/170 2024-05-26T22:47:00.480 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libcephfs2-2:19.0.0-2857.g0ef46a24.el9.x86_64 16/170 2024-05-26T22:47:00.480 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libcephsqlite-2:19.0.0-2857.g0ef46a24.el9.x86_64 17/170 2024-05-26T22:47:00.480 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : librados-devel-2:19.0.0-2857.g0ef46a24.el9.x86_6 18/170 2024-05-26T22:47:00.480 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libradosstriper1-2:19.0.0-2857.g0ef46a24.el9.x86 19/170 2024-05-26T22:47:00.480 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : librgw2-2:19.0.0-2857.g0ef46a24.el9.x86_64 20/170 2024-05-26T22:47:00.480 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2857.g0ef46a24.el 21/170 2024-05-26T22:47:00.480 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-ceph-common-2:19.0.0-2857.g0ef46a24.el9. 22/170 2024-05-26T22:47:00.480 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-cephfs-2:19.0.0-2857.g0ef46a24.el9.x86_6 23/170 2024-05-26T22:47:00.480 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-rados-2:19.0.0-2857.g0ef46a24.el9.x86_64 24/170 2024-05-26T22:47:00.480 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-rbd-2:19.0.0-2857.g0ef46a24.el9.x86_64 25/170 2024-05-26T22:47:00.480 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-rgw-2:19.0.0-2857.g0ef46a24.el9.x86_64 26/170 2024-05-26T22:47:00.480 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : rbd-fuse-2:19.0.0-2857.g0ef46a24.el9.x86_64 27/170 2024-05-26T22:47:00.494 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : rbd-mirror-2:19.0.0-2857.g0ef46a24.el9.x86_64 28/170 2024-05-26T22:47:00.495 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : rbd-nbd-2:19.0.0-2857.g0ef46a24.el9.x86_64 29/170 2024-05-26T22:47:00.495 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2857.g0ef46a24. 30/170 2024-05-26T22:47:00.495 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2857.g0ef46a24.el9.noa 31/170 2024-05-26T22:47:00.495 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2857.g0ef46a24.el9.n 32/170 2024-05-26T22:47:00.495 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2857.g0ef 33/170 2024-05-26T22:47:00.495 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2857.g0ef46a24.el 34/170 2024-05-26T22:47:00.495 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2857.g0ef46a24.el9.noarch 35/170 2024-05-26T22:47:00.495 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2857.g0ef46a24.e 36/170 2024-05-26T22:47:00.495 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-volume-2:19.0.0-2857.g0ef46a24.el9.noarch 37/170 2024-05-26T22:47:00.495 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : cephadm-2:19.0.0-2857.g0ef46a24.el9.noarch 38/170 2024-05-26T22:47:00.495 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : cephfs-top-2:19.0.0-2857.g0ef46a24.el9.noarch 39/170 2024-05-26T22:47:00.496 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/170 2024-05-26T22:47:00.496 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/170 2024-05-26T22:47:00.496 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/170 2024-05-26T22:47:00.496 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/170 2024-05-26T22:47:00.496 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/170 2024-05-26T22:47:00.496 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/170 2024-05-26T22:47:00.496 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/170 2024-05-26T22:47:00.496 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/170 2024-05-26T22:47:00.496 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/170 2024-05-26T22:47:00.496 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/170 2024-05-26T22:47:00.496 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/170 2024-05-26T22:47:00.496 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/170 2024-05-26T22:47:00.497 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/170 2024-05-26T22:47:00.497 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/170 2024-05-26T22:47:00.497 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/170 2024-05-26T22:47:00.497 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/170 2024-05-26T22:47:00.497 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/170 2024-05-26T22:47:00.497 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/170 2024-05-26T22:47:00.497 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/170 2024-05-26T22:47:00.497 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/170 2024-05-26T22:47:00.497 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/170 2024-05-26T22:47:00.497 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/170 2024-05-26T22:47:00.497 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/170 2024-05-26T22:47:00.497 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/170 2024-05-26T22:47:00.497 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/170 2024-05-26T22:47:00.498 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/170 2024-05-26T22:47:00.498 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/170 2024-05-26T22:47:00.498 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/170 2024-05-26T22:47:00.498 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/170 2024-05-26T22:47:00.498 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/170 2024-05-26T22:47:00.498 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/170 2024-05-26T22:47:00.498 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/170 2024-05-26T22:47:00.498 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/170 2024-05-26T22:47:00.498 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/170 2024-05-26T22:47:00.498 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/170 2024-05-26T22:47:00.498 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/170 2024-05-26T22:47:00.498 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/170 2024-05-26T22:47:00.498 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/170 2024-05-26T22:47:00.499 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/170 2024-05-26T22:47:00.499 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/170 2024-05-26T22:47:00.499 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/170 2024-05-26T22:47:00.499 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/170 2024-05-26T22:47:00.499 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/170 2024-05-26T22:47:00.499 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/170 2024-05-26T22:47:00.499 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/170 2024-05-26T22:47:00.499 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/170 2024-05-26T22:47:00.499 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/170 2024-05-26T22:47:00.499 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/170 2024-05-26T22:47:00.499 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/170 2024-05-26T22:47:00.499 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/170 2024-05-26T22:47:00.499 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/170 2024-05-26T22:47:00.500 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/170 2024-05-26T22:47:00.500 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/170 2024-05-26T22:47:00.500 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/170 2024-05-26T22:47:00.500 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/170 2024-05-26T22:47:00.500 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/170 2024-05-26T22:47:00.500 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/170 2024-05-26T22:47:00.500 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/170 2024-05-26T22:47:00.500 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/170 2024-05-26T22:47:00.500 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 99/170 2024-05-26T22:47:00.500 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/170 2024-05-26T22:47:00.500 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 101/170 2024-05-26T22:47:00.500 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/170 2024-05-26T22:47:00.500 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/170 2024-05-26T22:47:00.501 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/170 2024-05-26T22:47:00.501 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/170 2024-05-26T22:47:00.501 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/170 2024-05-26T22:47:00.501 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/170 2024-05-26T22:47:00.501 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/170 2024-05-26T22:47:00.501 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/170 2024-05-26T22:47:00.501 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/170 2024-05-26T22:47:00.501 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/170 2024-05-26T22:47:00.501 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/170 2024-05-26T22:47:00.501 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/170 2024-05-26T22:47:00.501 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/170 2024-05-26T22:47:00.501 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/170 2024-05-26T22:47:00.501 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/170 2024-05-26T22:47:00.502 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/170 2024-05-26T22:47:00.502 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 118/170 2024-05-26T22:47:00.502 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/170 2024-05-26T22:47:00.502 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/170 2024-05-26T22:47:00.502 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/170 2024-05-26T22:47:00.502 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/170 2024-05-26T22:47:00.502 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/170 2024-05-26T22:47:00.502 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/170 2024-05-26T22:47:00.502 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/170 2024-05-26T22:47:00.502 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/170 2024-05-26T22:47:00.502 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/170 2024-05-26T22:47:00.502 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/170 2024-05-26T22:47:00.502 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/170 2024-05-26T22:47:00.503 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/170 2024-05-26T22:47:00.503 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/170 2024-05-26T22:47:00.503 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/170 2024-05-26T22:47:00.503 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/170 2024-05-26T22:47:00.503 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/170 2024-05-26T22:47:00.503 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/170 2024-05-26T22:47:00.503 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/170 2024-05-26T22:47:00.503 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/170 2024-05-26T22:47:00.503 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/170 2024-05-26T22:47:00.503 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/170 2024-05-26T22:47:00.504 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/170 2024-05-26T22:47:00.504 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/170 2024-05-26T22:47:00.504 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/170 2024-05-26T22:47:00.504 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/170 2024-05-26T22:47:00.504 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/170 2024-05-26T22:47:00.504 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/170 2024-05-26T22:47:00.504 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/170 2024-05-26T22:47:00.504 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/170 2024-05-26T22:47:00.504 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/170 2024-05-26T22:47:00.504 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : librados2-2:19.0.0-2857.g0ef46a24.el9.x86_64 149/170 2024-05-26T22:47:00.504 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/170 2024-05-26T22:47:00.504 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : librbd1-2:19.0.0-2857.g0ef46a24.el9.x86_64 151/170 2024-05-26T22:47:00.505 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/170 2024-05-26T22:47:00.505 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : NetworkManager-1:1.47.90-1.el9.x86_64 153/170 2024-05-26T22:47:00.505 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/170 2024-05-26T22:47:00.505 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 155/170 2024-05-26T22:47:00.505 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/170 2024-05-26T22:47:00.505 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : NetworkManager-team-1:1.47.90-1.el9.x86_64 157/170 2024-05-26T22:47:00.505 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/170 2024-05-26T22:47:00.505 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : NetworkManager-tui-1:1.47.90-1.el9.x86_64 159/170 2024-05-26T22:47:00.505 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-05-26T22:47:00.505 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/170 2024-05-26T22:47:00.505 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/170 2024-05-26T22:47:00.505 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/170 2024-05-26T22:47:00.505 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/170 2024-05-26T22:47:00.506 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/170 2024-05-26T22:47:00.506 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/170 2024-05-26T22:47:00.506 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 167/170 2024-05-26T22:47:00.506 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/170 2024-05-26T22:47:00.506 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 169/170 2024-05-26T22:47:00.617 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-05-26T22:47:00.662 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 164/170 2024-05-26T22:47:00.662 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 165/170 2024-05-26T22:47:00.799 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 165/170 2024-05-26T22:47:00.838 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 166/170 2024-05-26T22:47:00.871 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-26T22:47:00.871 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-26T22:47:01.209 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-26T22:47:01.209 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-05-26T22:47:01.363 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-05-26T22:47:01.407 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 169/170 2024-05-26T22:47:01.526 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-05-26T22:47:01.780 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-05-26T22:47:01.780 INFO:teuthology.orchestra.run.smithi131.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-05-26T22:47:01.780 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:47:02.835 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/170 2024-05-26T22:47:02.835 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-05-26T22:47:02.835 INFO:teuthology.orchestra.run.smithi187.stdout:Upgraded: 2024-05-26T22:47:02.835 INFO:teuthology.orchestra.run.smithi187.stdout: NetworkManager-1:1.47.90-1.el9.x86_64 2024-05-26T22:47:02.835 INFO:teuthology.orchestra.run.smithi187.stdout: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 2024-05-26T22:47:02.836 INFO:teuthology.orchestra.run.smithi187.stdout: NetworkManager-team-1:1.47.90-1.el9.x86_64 2024-05-26T22:47:02.836 INFO:teuthology.orchestra.run.smithi187.stdout: NetworkManager-tui-1:1.47.90-1.el9.x86_64 2024-05-26T22:47:02.836 INFO:teuthology.orchestra.run.smithi187.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-05-26T22:47:02.836 INFO:teuthology.orchestra.run.smithi187.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-05-26T22:47:02.836 INFO:teuthology.orchestra.run.smithi187.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-05-26T22:47:02.836 INFO:teuthology.orchestra.run.smithi187.stdout: librados2-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-26T22:47:02.836 INFO:teuthology.orchestra.run.smithi187.stdout: librbd1-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-26T22:47:02.836 INFO:teuthology.orchestra.run.smithi187.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-26T22:47:02.836 INFO:teuthology.orchestra.run.smithi187.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-26T22:47:02.836 INFO:teuthology.orchestra.run.smithi187.stdout:Installed: 2024-05-26T22:47:02.836 INFO:teuthology.orchestra.run.smithi187.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-26T22:47:02.836 INFO:teuthology.orchestra.run.smithi187.stdout: automake-1.16.2-8.el9.noarch 2024-05-26T22:47:02.836 INFO:teuthology.orchestra.run.smithi187.stdout: bison-3.7.4-5.el9.x86_64 2024-05-26T22:47:02.837 INFO:teuthology.orchestra.run.smithi187.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-26T22:47:02.837 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-26T22:47:02.837 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-base-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-26T22:47:02.837 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-common-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-26T22:47:02.837 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-fuse-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-26T22:47:02.837 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-grafana-dashboards-2:19.0.0-2857.g0ef46a24.el9.noarch 2024-05-26T22:47:02.837 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-immutable-object-cache-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-26T22:47:02.837 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mds-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-26T22:47:02.837 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-26T22:47:02.837 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-cephadm-2:19.0.0-2857.g0ef46a24.el9.noarch 2024-05-26T22:47:02.837 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-dashboard-2:19.0.0-2857.g0ef46a24.el9.noarch 2024-05-26T22:47:02.837 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2857.g0ef46a24.el9.noarch 2024-05-26T22:47:02.837 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-modules-core-2:19.0.0-2857.g0ef46a24.el9.noarch 2024-05-26T22:47:02.838 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-rook-2:19.0.0-2857.g0ef46a24.el9.noarch 2024-05-26T22:47:02.838 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mon-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-26T22:47:02.838 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-osd-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-26T22:47:02.838 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-prometheus-alerts-2:19.0.0-2857.g0ef46a24.el9.noarch 2024-05-26T22:47:02.838 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-radosgw-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-26T22:47:02.838 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-selinux-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-26T22:47:02.838 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-test-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-26T22:47:02.838 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-volume-2:19.0.0-2857.g0ef46a24.el9.noarch 2024-05-26T22:47:02.838 INFO:teuthology.orchestra.run.smithi187.stdout: cephadm-2:19.0.0-2857.g0ef46a24.el9.noarch 2024-05-26T22:47:02.838 INFO:teuthology.orchestra.run.smithi187.stdout: cephfs-mirror-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-26T22:47:02.838 INFO:teuthology.orchestra.run.smithi187.stdout: cephfs-shell-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-26T22:47:02.838 INFO:teuthology.orchestra.run.smithi187.stdout: cephfs-top-2:19.0.0-2857.g0ef46a24.el9.noarch 2024-05-26T22:47:02.838 INFO:teuthology.orchestra.run.smithi187.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-05-26T22:47:02.839 INFO:teuthology.orchestra.run.smithi187.stdout: flex-2.6.4-9.el9.x86_64 2024-05-26T22:47:02.839 INFO:teuthology.orchestra.run.smithi187.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-26T22:47:02.839 INFO:teuthology.orchestra.run.smithi187.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-26T22:47:02.839 INFO:teuthology.orchestra.run.smithi187.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-26T22:47:02.839 INFO:teuthology.orchestra.run.smithi187.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-26T22:47:02.839 INFO:teuthology.orchestra.run.smithi187.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-26T22:47:02.839 INFO:teuthology.orchestra.run.smithi187.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-26T22:47:02.839 INFO:teuthology.orchestra.run.smithi187.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-26T22:47:02.839 INFO:teuthology.orchestra.run.smithi187.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-26T22:47:02.839 INFO:teuthology.orchestra.run.smithi187.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-26T22:47:02.839 INFO:teuthology.orchestra.run.smithi187.stdout: libcephfs-devel-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-26T22:47:02.839 INFO:teuthology.orchestra.run.smithi187.stdout: libcephfs2-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-26T22:47:02.839 INFO:teuthology.orchestra.run.smithi187.stdout: libcephsqlite-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-26T22:47:02.840 INFO:teuthology.orchestra.run.smithi187.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-26T22:47:02.840 INFO:teuthology.orchestra.run.smithi187.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-26T22:47:02.840 INFO:teuthology.orchestra.run.smithi187.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-26T22:47:02.840 INFO:teuthology.orchestra.run.smithi187.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-26T22:47:02.840 INFO:teuthology.orchestra.run.smithi187.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-26T22:47:02.840 INFO:teuthology.orchestra.run.smithi187.stdout: librados-devel-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-26T22:47:02.840 INFO:teuthology.orchestra.run.smithi187.stdout: libradosstriper1-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-26T22:47:02.840 INFO:teuthology.orchestra.run.smithi187.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-26T22:47:02.840 INFO:teuthology.orchestra.run.smithi187.stdout: librgw2-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-26T22:47:02.840 INFO:teuthology.orchestra.run.smithi187.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-26T22:47:02.840 INFO:teuthology.orchestra.run.smithi187.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-26T22:47:02.840 INFO:teuthology.orchestra.run.smithi187.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-26T22:47:02.840 INFO:teuthology.orchestra.run.smithi187.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-26T22:47:02.841 INFO:teuthology.orchestra.run.smithi187.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-26T22:47:02.841 INFO:teuthology.orchestra.run.smithi187.stdout: lua-5.4.4-4.el9.x86_64 2024-05-26T22:47:02.841 INFO:teuthology.orchestra.run.smithi187.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-26T22:47:02.841 INFO:teuthology.orchestra.run.smithi187.stdout: luarocks-3.9.2-5.el9.noarch 2024-05-26T22:47:02.841 INFO:teuthology.orchestra.run.smithi187.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-26T22:47:02.841 INFO:teuthology.orchestra.run.smithi187.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-26T22:47:02.841 INFO:teuthology.orchestra.run.smithi187.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-26T22:47:02.841 INFO:teuthology.orchestra.run.smithi187.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-26T22:47:02.841 INFO:teuthology.orchestra.run.smithi187.stdout: postgresql-13.14-1.el9.x86_64 2024-05-26T22:47:02.841 INFO:teuthology.orchestra.run.smithi187.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-26T22:47:02.841 INFO:teuthology.orchestra.run.smithi187.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-26T22:47:02.841 INFO:teuthology.orchestra.run.smithi187.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-26T22:47:02.841 INFO:teuthology.orchestra.run.smithi187.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-26T22:47:02.842 INFO:teuthology.orchestra.run.smithi187.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-26T22:47:02.842 INFO:teuthology.orchestra.run.smithi187.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-26T22:47:02.842 INFO:teuthology.orchestra.run.smithi187.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-26T22:47:02.842 INFO:teuthology.orchestra.run.smithi187.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-26T22:47:02.842 INFO:teuthology.orchestra.run.smithi187.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-26T22:47:02.842 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-26T22:47:02.842 INFO:teuthology.orchestra.run.smithi187.stdout: python3-ceph-argparse-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-26T22:47:02.842 INFO:teuthology.orchestra.run.smithi187.stdout: python3-ceph-common-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-26T22:47:02.842 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cephfs-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-26T22:47:02.842 INFO:teuthology.orchestra.run.smithi187.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-26T22:47:02.842 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-26T22:47:02.842 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-26T22:47:02.842 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-26T22:47:02.843 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-26T22:47:02.843 INFO:teuthology.orchestra.run.smithi187.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-05-26T22:47:02.843 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-26T22:47:02.843 INFO:teuthology.orchestra.run.smithi187.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-26T22:47:02.843 INFO:teuthology.orchestra.run.smithi187.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-26T22:47:02.843 INFO:teuthology.orchestra.run.smithi187.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-26T22:47:02.843 INFO:teuthology.orchestra.run.smithi187.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-26T22:47:02.843 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-26T22:47:02.843 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-26T22:47:02.843 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-26T22:47:02.843 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-26T22:47:02.844 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-26T22:47:02.844 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-05-26T22:47:02.844 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-26T22:47:02.844 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-26T22:47:02.844 INFO:teuthology.orchestra.run.smithi187.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-26T22:47:02.844 INFO:teuthology.orchestra.run.smithi187.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-26T22:47:02.844 INFO:teuthology.orchestra.run.smithi187.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-26T22:47:02.844 INFO:teuthology.orchestra.run.smithi187.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-26T22:47:02.844 INFO:teuthology.orchestra.run.smithi187.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-26T22:47:02.844 INFO:teuthology.orchestra.run.smithi187.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-26T22:47:02.844 INFO:teuthology.orchestra.run.smithi187.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-26T22:47:02.844 INFO:teuthology.orchestra.run.smithi187.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-26T22:47:02.845 INFO:teuthology.orchestra.run.smithi187.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-26T22:47:02.845 INFO:teuthology.orchestra.run.smithi187.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-26T22:47:02.845 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-26T22:47:02.845 INFO:teuthology.orchestra.run.smithi187.stdout: python3-ply-3.11-14.el9.noarch 2024-05-26T22:47:02.845 INFO:teuthology.orchestra.run.smithi187.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-26T22:47:02.845 INFO:teuthology.orchestra.run.smithi187.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-26T22:47:02.845 INFO:teuthology.orchestra.run.smithi187.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-26T22:47:02.845 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-26T22:47:02.845 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-26T22:47:02.845 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-26T22:47:02.845 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-26T22:47:02.845 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-26T22:47:02.845 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-26T22:47:02.846 INFO:teuthology.orchestra.run.smithi187.stdout: python3-rados-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-26T22:47:02.846 INFO:teuthology.orchestra.run.smithi187.stdout: python3-rbd-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-26T22:47:02.846 INFO:teuthology.orchestra.run.smithi187.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-26T22:47:02.846 INFO:teuthology.orchestra.run.smithi187.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-26T22:47:02.846 INFO:teuthology.orchestra.run.smithi187.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-26T22:47:02.846 INFO:teuthology.orchestra.run.smithi187.stdout: python3-rgw-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-26T22:47:02.846 INFO:teuthology.orchestra.run.smithi187.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-26T22:47:02.846 INFO:teuthology.orchestra.run.smithi187.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-26T22:47:02.846 INFO:teuthology.orchestra.run.smithi187.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-26T22:47:02.846 INFO:teuthology.orchestra.run.smithi187.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-26T22:47:02.847 INFO:teuthology.orchestra.run.smithi187.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-26T22:47:02.847 INFO:teuthology.orchestra.run.smithi187.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-26T22:47:02.847 INFO:teuthology.orchestra.run.smithi187.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-26T22:47:02.847 INFO:teuthology.orchestra.run.smithi187.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-26T22:47:02.847 INFO:teuthology.orchestra.run.smithi187.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-26T22:47:02.847 INFO:teuthology.orchestra.run.smithi187.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-26T22:47:02.847 INFO:teuthology.orchestra.run.smithi187.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-26T22:47:02.847 INFO:teuthology.orchestra.run.smithi187.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-26T22:47:02.847 INFO:teuthology.orchestra.run.smithi187.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-26T22:47:02.847 INFO:teuthology.orchestra.run.smithi187.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-26T22:47:02.847 INFO:teuthology.orchestra.run.smithi187.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-26T22:47:02.847 INFO:teuthology.orchestra.run.smithi187.stdout: rbd-fuse-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-26T22:47:02.848 INFO:teuthology.orchestra.run.smithi187.stdout: rbd-mirror-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-26T22:47:02.848 INFO:teuthology.orchestra.run.smithi187.stdout: rbd-nbd-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-26T22:47:02.848 INFO:teuthology.orchestra.run.smithi187.stdout: re2-1:20211101-3.el9.x86_64 2024-05-26T22:47:02.848 INFO:teuthology.orchestra.run.smithi187.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-26T22:47:02.848 INFO:teuthology.orchestra.run.smithi187.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-26T22:47:02.848 INFO:teuthology.orchestra.run.smithi187.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-26T22:47:02.848 INFO:teuthology.orchestra.run.smithi187.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-26T22:47:02.848 INFO:teuthology.orchestra.run.smithi187.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-26T22:47:02.848 INFO:teuthology.orchestra.run.smithi187.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-26T22:47:02.849 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-05-26T22:47:02.849 INFO:teuthology.orchestra.run.smithi187.stdout:Complete! 2024-05-26T22:47:03.537 DEBUG:teuthology.parallel:result is None 2024-05-26T22:47:07.640 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-2:19.0.0-2857.g0ef46a24.el9.x86_64 1/170 2024-05-26T22:47:07.640 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-base-2:19.0.0-2857.g0ef46a24.el9.x86_64 2/170 2024-05-26T22:47:07.640 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-common-2:19.0.0-2857.g0ef46a24.el9.x86_64 3/170 2024-05-26T22:47:07.640 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-fuse-2:19.0.0-2857.g0ef46a24.el9.x86_64 4/170 2024-05-26T22:47:07.640 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2857.g0ef46 5/170 2024-05-26T22:47:07.640 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mds-2:19.0.0-2857.g0ef46a24.el9.x86_64 6/170 2024-05-26T22:47:07.640 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mgr-2:19.0.0-2857.g0ef46a24.el9.x86_64 7/170 2024-05-26T22:47:07.640 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mon-2:19.0.0-2857.g0ef46a24.el9.x86_64 8/170 2024-05-26T22:47:07.641 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-osd-2:19.0.0-2857.g0ef46a24.el9.x86_64 9/170 2024-05-26T22:47:07.641 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-radosgw-2:19.0.0-2857.g0ef46a24.el9.x86_64 10/170 2024-05-26T22:47:07.641 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-selinux-2:19.0.0-2857.g0ef46a24.el9.x86_64 11/170 2024-05-26T22:47:07.641 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-test-2:19.0.0-2857.g0ef46a24.el9.x86_64 12/170 2024-05-26T22:47:07.641 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : cephfs-mirror-2:19.0.0-2857.g0ef46a24.el9.x86_64 13/170 2024-05-26T22:47:07.641 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : cephfs-shell-2:19.0.0-2857.g0ef46a24.el9.x86_64 14/170 2024-05-26T22:47:07.641 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libcephfs-devel-2:19.0.0-2857.g0ef46a24.el9.x86_ 15/170 2024-05-26T22:47:07.641 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libcephfs2-2:19.0.0-2857.g0ef46a24.el9.x86_64 16/170 2024-05-26T22:47:07.641 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libcephsqlite-2:19.0.0-2857.g0ef46a24.el9.x86_64 17/170 2024-05-26T22:47:07.641 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librados-devel-2:19.0.0-2857.g0ef46a24.el9.x86_6 18/170 2024-05-26T22:47:07.641 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libradosstriper1-2:19.0.0-2857.g0ef46a24.el9.x86 19/170 2024-05-26T22:47:07.641 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librgw2-2:19.0.0-2857.g0ef46a24.el9.x86_64 20/170 2024-05-26T22:47:07.641 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2857.g0ef46a24.el 21/170 2024-05-26T22:47:07.642 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-ceph-common-2:19.0.0-2857.g0ef46a24.el9. 22/170 2024-05-26T22:47:07.642 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-cephfs-2:19.0.0-2857.g0ef46a24.el9.x86_6 23/170 2024-05-26T22:47:07.642 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-rados-2:19.0.0-2857.g0ef46a24.el9.x86_64 24/170 2024-05-26T22:47:07.642 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-rbd-2:19.0.0-2857.g0ef46a24.el9.x86_64 25/170 2024-05-26T22:47:07.642 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-rgw-2:19.0.0-2857.g0ef46a24.el9.x86_64 26/170 2024-05-26T22:47:07.656 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : rbd-fuse-2:19.0.0-2857.g0ef46a24.el9.x86_64 27/170 2024-05-26T22:47:07.656 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : rbd-mirror-2:19.0.0-2857.g0ef46a24.el9.x86_64 28/170 2024-05-26T22:47:07.656 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : rbd-nbd-2:19.0.0-2857.g0ef46a24.el9.x86_64 29/170 2024-05-26T22:47:07.656 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2857.g0ef46a24. 30/170 2024-05-26T22:47:07.656 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2857.g0ef46a24.el9.noa 31/170 2024-05-26T22:47:07.656 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2857.g0ef46a24.el9.n 32/170 2024-05-26T22:47:07.656 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2857.g0ef 33/170 2024-05-26T22:47:07.656 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2857.g0ef46a24.el 34/170 2024-05-26T22:47:07.656 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2857.g0ef46a24.el9.noarch 35/170 2024-05-26T22:47:07.656 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2857.g0ef46a24.e 36/170 2024-05-26T22:47:07.656 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-volume-2:19.0.0-2857.g0ef46a24.el9.noarch 37/170 2024-05-26T22:47:07.657 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : cephadm-2:19.0.0-2857.g0ef46a24.el9.noarch 38/170 2024-05-26T22:47:07.657 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : cephfs-top-2:19.0.0-2857.g0ef46a24.el9.noarch 39/170 2024-05-26T22:47:07.657 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/170 2024-05-26T22:47:07.657 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/170 2024-05-26T22:47:07.657 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/170 2024-05-26T22:47:07.657 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/170 2024-05-26T22:47:07.657 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/170 2024-05-26T22:47:07.657 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/170 2024-05-26T22:47:07.657 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/170 2024-05-26T22:47:07.657 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/170 2024-05-26T22:47:07.657 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/170 2024-05-26T22:47:07.657 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/170 2024-05-26T22:47:07.658 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/170 2024-05-26T22:47:07.658 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/170 2024-05-26T22:47:07.658 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/170 2024-05-26T22:47:07.658 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/170 2024-05-26T22:47:07.658 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/170 2024-05-26T22:47:07.658 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/170 2024-05-26T22:47:07.658 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/170 2024-05-26T22:47:07.658 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/170 2024-05-26T22:47:07.658 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/170 2024-05-26T22:47:07.658 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/170 2024-05-26T22:47:07.658 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/170 2024-05-26T22:47:07.658 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/170 2024-05-26T22:47:07.658 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/170 2024-05-26T22:47:07.659 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/170 2024-05-26T22:47:07.659 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/170 2024-05-26T22:47:07.659 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/170 2024-05-26T22:47:07.659 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/170 2024-05-26T22:47:07.659 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/170 2024-05-26T22:47:07.659 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/170 2024-05-26T22:47:07.659 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/170 2024-05-26T22:47:07.659 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/170 2024-05-26T22:47:07.659 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/170 2024-05-26T22:47:07.659 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/170 2024-05-26T22:47:07.659 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/170 2024-05-26T22:47:07.659 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/170 2024-05-26T22:47:07.659 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/170 2024-05-26T22:47:07.660 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/170 2024-05-26T22:47:07.660 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/170 2024-05-26T22:47:07.660 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/170 2024-05-26T22:47:07.660 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/170 2024-05-26T22:47:07.660 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/170 2024-05-26T22:47:07.660 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/170 2024-05-26T22:47:07.660 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/170 2024-05-26T22:47:07.660 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/170 2024-05-26T22:47:07.660 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/170 2024-05-26T22:47:07.660 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/170 2024-05-26T22:47:07.660 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/170 2024-05-26T22:47:07.660 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/170 2024-05-26T22:47:07.660 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/170 2024-05-26T22:47:07.661 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/170 2024-05-26T22:47:07.661 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/170 2024-05-26T22:47:07.661 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/170 2024-05-26T22:47:07.661 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/170 2024-05-26T22:47:07.661 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/170 2024-05-26T22:47:07.661 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/170 2024-05-26T22:47:07.661 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/170 2024-05-26T22:47:07.661 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/170 2024-05-26T22:47:07.661 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/170 2024-05-26T22:47:07.661 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/170 2024-05-26T22:47:07.661 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 99/170 2024-05-26T22:47:07.661 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/170 2024-05-26T22:47:07.661 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 101/170 2024-05-26T22:47:07.661 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/170 2024-05-26T22:47:07.662 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/170 2024-05-26T22:47:07.662 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/170 2024-05-26T22:47:07.662 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/170 2024-05-26T22:47:07.662 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/170 2024-05-26T22:47:07.662 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/170 2024-05-26T22:47:07.662 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/170 2024-05-26T22:47:07.662 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/170 2024-05-26T22:47:07.662 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/170 2024-05-26T22:47:07.662 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/170 2024-05-26T22:47:07.662 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/170 2024-05-26T22:47:07.662 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/170 2024-05-26T22:47:07.662 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/170 2024-05-26T22:47:07.662 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/170 2024-05-26T22:47:07.663 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/170 2024-05-26T22:47:07.663 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/170 2024-05-26T22:47:07.663 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 118/170 2024-05-26T22:47:07.663 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/170 2024-05-26T22:47:07.663 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/170 2024-05-26T22:47:07.663 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/170 2024-05-26T22:47:07.663 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/170 2024-05-26T22:47:07.663 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/170 2024-05-26T22:47:07.663 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/170 2024-05-26T22:47:07.663 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/170 2024-05-26T22:47:07.663 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/170 2024-05-26T22:47:07.663 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/170 2024-05-26T22:47:07.663 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/170 2024-05-26T22:47:07.663 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/170 2024-05-26T22:47:07.664 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/170 2024-05-26T22:47:07.664 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/170 2024-05-26T22:47:07.664 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/170 2024-05-26T22:47:07.664 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/170 2024-05-26T22:47:07.664 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/170 2024-05-26T22:47:07.664 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/170 2024-05-26T22:47:07.664 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/170 2024-05-26T22:47:07.664 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/170 2024-05-26T22:47:07.664 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/170 2024-05-26T22:47:07.664 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/170 2024-05-26T22:47:07.664 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/170 2024-05-26T22:47:07.664 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/170 2024-05-26T22:47:07.664 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/170 2024-05-26T22:47:07.664 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/170 2024-05-26T22:47:07.665 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/170 2024-05-26T22:47:07.665 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/170 2024-05-26T22:47:07.665 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/170 2024-05-26T22:47:07.665 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/170 2024-05-26T22:47:07.665 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/170 2024-05-26T22:47:07.665 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librados2-2:19.0.0-2857.g0ef46a24.el9.x86_64 149/170 2024-05-26T22:47:07.665 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/170 2024-05-26T22:47:07.665 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librbd1-2:19.0.0-2857.g0ef46a24.el9.x86_64 151/170 2024-05-26T22:47:07.665 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/170 2024-05-26T22:47:07.665 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : NetworkManager-1:1.47.90-1.el9.x86_64 153/170 2024-05-26T22:47:07.665 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/170 2024-05-26T22:47:07.665 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 155/170 2024-05-26T22:47:07.665 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/170 2024-05-26T22:47:07.665 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : NetworkManager-team-1:1.47.90-1.el9.x86_64 157/170 2024-05-26T22:47:07.666 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/170 2024-05-26T22:47:07.666 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : NetworkManager-tui-1:1.47.90-1.el9.x86_64 159/170 2024-05-26T22:47:07.666 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-05-26T22:47:07.666 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/170 2024-05-26T22:47:07.666 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/170 2024-05-26T22:47:07.666 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/170 2024-05-26T22:47:07.666 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/170 2024-05-26T22:47:07.666 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/170 2024-05-26T22:47:07.666 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/170 2024-05-26T22:47:07.666 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 167/170 2024-05-26T22:47:07.666 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/170 2024-05-26T22:47:07.666 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 169/170 2024-05-26T22:47:09.974 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/170 2024-05-26T22:47:09.974 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:47:09.974 INFO:teuthology.orchestra.run.smithi131.stdout:Upgraded: 2024-05-26T22:47:09.974 INFO:teuthology.orchestra.run.smithi131.stdout: NetworkManager-1:1.47.90-1.el9.x86_64 2024-05-26T22:47:09.974 INFO:teuthology.orchestra.run.smithi131.stdout: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 2024-05-26T22:47:09.974 INFO:teuthology.orchestra.run.smithi131.stdout: NetworkManager-team-1:1.47.90-1.el9.x86_64 2024-05-26T22:47:09.975 INFO:teuthology.orchestra.run.smithi131.stdout: NetworkManager-tui-1:1.47.90-1.el9.x86_64 2024-05-26T22:47:09.975 INFO:teuthology.orchestra.run.smithi131.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-05-26T22:47:09.975 INFO:teuthology.orchestra.run.smithi131.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-05-26T22:47:09.975 INFO:teuthology.orchestra.run.smithi131.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-05-26T22:47:09.975 INFO:teuthology.orchestra.run.smithi131.stdout: librados2-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-26T22:47:09.975 INFO:teuthology.orchestra.run.smithi131.stdout: librbd1-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-26T22:47:09.975 INFO:teuthology.orchestra.run.smithi131.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-26T22:47:09.975 INFO:teuthology.orchestra.run.smithi131.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-26T22:47:09.975 INFO:teuthology.orchestra.run.smithi131.stdout:Installed: 2024-05-26T22:47:09.975 INFO:teuthology.orchestra.run.smithi131.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-26T22:47:09.976 INFO:teuthology.orchestra.run.smithi131.stdout: automake-1.16.2-8.el9.noarch 2024-05-26T22:47:09.976 INFO:teuthology.orchestra.run.smithi131.stdout: bison-3.7.4-5.el9.x86_64 2024-05-26T22:47:09.976 INFO:teuthology.orchestra.run.smithi131.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-26T22:47:09.976 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-26T22:47:09.976 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-base-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-26T22:47:09.976 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-common-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-26T22:47:09.976 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-fuse-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-26T22:47:09.976 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-grafana-dashboards-2:19.0.0-2857.g0ef46a24.el9.noarch 2024-05-26T22:47:09.976 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-immutable-object-cache-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-26T22:47:09.976 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mds-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-26T22:47:09.976 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-26T22:47:09.977 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-cephadm-2:19.0.0-2857.g0ef46a24.el9.noarch 2024-05-26T22:47:09.977 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-dashboard-2:19.0.0-2857.g0ef46a24.el9.noarch 2024-05-26T22:47:09.977 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2857.g0ef46a24.el9.noarch 2024-05-26T22:47:09.977 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-modules-core-2:19.0.0-2857.g0ef46a24.el9.noarch 2024-05-26T22:47:09.977 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-rook-2:19.0.0-2857.g0ef46a24.el9.noarch 2024-05-26T22:47:09.977 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mon-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-26T22:47:09.977 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-osd-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-26T22:47:09.977 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-prometheus-alerts-2:19.0.0-2857.g0ef46a24.el9.noarch 2024-05-26T22:47:09.977 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-radosgw-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-26T22:47:09.977 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-selinux-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-26T22:47:09.978 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-test-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-26T22:47:09.978 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-volume-2:19.0.0-2857.g0ef46a24.el9.noarch 2024-05-26T22:47:09.978 INFO:teuthology.orchestra.run.smithi131.stdout: cephadm-2:19.0.0-2857.g0ef46a24.el9.noarch 2024-05-26T22:47:09.978 INFO:teuthology.orchestra.run.smithi131.stdout: cephfs-mirror-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-26T22:47:09.978 INFO:teuthology.orchestra.run.smithi131.stdout: cephfs-shell-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-26T22:47:09.978 INFO:teuthology.orchestra.run.smithi131.stdout: cephfs-top-2:19.0.0-2857.g0ef46a24.el9.noarch 2024-05-26T22:47:09.978 INFO:teuthology.orchestra.run.smithi131.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-05-26T22:47:09.978 INFO:teuthology.orchestra.run.smithi131.stdout: flex-2.6.4-9.el9.x86_64 2024-05-26T22:47:09.978 INFO:teuthology.orchestra.run.smithi131.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-26T22:47:09.978 INFO:teuthology.orchestra.run.smithi131.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-26T22:47:09.979 INFO:teuthology.orchestra.run.smithi131.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-26T22:47:09.979 INFO:teuthology.orchestra.run.smithi131.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-26T22:47:09.979 INFO:teuthology.orchestra.run.smithi131.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-26T22:47:09.979 INFO:teuthology.orchestra.run.smithi131.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-26T22:47:09.979 INFO:teuthology.orchestra.run.smithi131.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-26T22:47:09.979 INFO:teuthology.orchestra.run.smithi131.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-26T22:47:09.979 INFO:teuthology.orchestra.run.smithi131.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-26T22:47:09.979 INFO:teuthology.orchestra.run.smithi131.stdout: libcephfs-devel-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-26T22:47:09.979 INFO:teuthology.orchestra.run.smithi131.stdout: libcephfs2-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-26T22:47:09.979 INFO:teuthology.orchestra.run.smithi131.stdout: libcephsqlite-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-26T22:47:09.979 INFO:teuthology.orchestra.run.smithi131.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-26T22:47:09.980 INFO:teuthology.orchestra.run.smithi131.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-26T22:47:09.980 INFO:teuthology.orchestra.run.smithi131.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-26T22:47:09.980 INFO:teuthology.orchestra.run.smithi131.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-26T22:47:09.980 INFO:teuthology.orchestra.run.smithi131.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-26T22:47:09.980 INFO:teuthology.orchestra.run.smithi131.stdout: librados-devel-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-26T22:47:09.980 INFO:teuthology.orchestra.run.smithi131.stdout: libradosstriper1-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-26T22:47:09.980 INFO:teuthology.orchestra.run.smithi131.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-26T22:47:09.980 INFO:teuthology.orchestra.run.smithi131.stdout: librgw2-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-26T22:47:09.980 INFO:teuthology.orchestra.run.smithi131.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-26T22:47:09.980 INFO:teuthology.orchestra.run.smithi131.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-26T22:47:09.981 INFO:teuthology.orchestra.run.smithi131.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-26T22:47:09.981 INFO:teuthology.orchestra.run.smithi131.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-26T22:47:09.981 INFO:teuthology.orchestra.run.smithi131.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-26T22:47:09.981 INFO:teuthology.orchestra.run.smithi131.stdout: lua-5.4.4-4.el9.x86_64 2024-05-26T22:47:09.981 INFO:teuthology.orchestra.run.smithi131.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-26T22:47:09.981 INFO:teuthology.orchestra.run.smithi131.stdout: luarocks-3.9.2-5.el9.noarch 2024-05-26T22:47:09.981 INFO:teuthology.orchestra.run.smithi131.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-26T22:47:09.981 INFO:teuthology.orchestra.run.smithi131.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-26T22:47:09.981 INFO:teuthology.orchestra.run.smithi131.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-26T22:47:09.981 INFO:teuthology.orchestra.run.smithi131.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-26T22:47:09.982 INFO:teuthology.orchestra.run.smithi131.stdout: postgresql-13.14-1.el9.x86_64 2024-05-26T22:47:09.982 INFO:teuthology.orchestra.run.smithi131.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-26T22:47:09.982 INFO:teuthology.orchestra.run.smithi131.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-26T22:47:09.982 INFO:teuthology.orchestra.run.smithi131.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-26T22:47:09.982 INFO:teuthology.orchestra.run.smithi131.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-26T22:47:09.982 INFO:teuthology.orchestra.run.smithi131.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-26T22:47:09.982 INFO:teuthology.orchestra.run.smithi131.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-26T22:47:09.982 INFO:teuthology.orchestra.run.smithi131.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-26T22:47:09.982 INFO:teuthology.orchestra.run.smithi131.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-26T22:47:09.983 INFO:teuthology.orchestra.run.smithi131.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-26T22:47:09.983 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-26T22:47:09.983 INFO:teuthology.orchestra.run.smithi131.stdout: python3-ceph-argparse-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-26T22:47:09.983 INFO:teuthology.orchestra.run.smithi131.stdout: python3-ceph-common-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-26T22:47:09.983 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cephfs-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-26T22:47:09.983 INFO:teuthology.orchestra.run.smithi131.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-26T22:47:09.983 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-26T22:47:09.983 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-26T22:47:09.983 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-26T22:47:09.983 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-26T22:47:09.984 INFO:teuthology.orchestra.run.smithi131.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-05-26T22:47:09.984 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-26T22:47:09.984 INFO:teuthology.orchestra.run.smithi131.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-26T22:47:09.984 INFO:teuthology.orchestra.run.smithi131.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-26T22:47:09.984 INFO:teuthology.orchestra.run.smithi131.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-26T22:47:09.984 INFO:teuthology.orchestra.run.smithi131.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-26T22:47:09.984 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-26T22:47:09.984 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-26T22:47:09.984 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-26T22:47:09.985 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-26T22:47:09.985 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-26T22:47:09.985 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-05-26T22:47:09.985 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-26T22:47:09.985 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-26T22:47:09.985 INFO:teuthology.orchestra.run.smithi131.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-26T22:47:09.985 INFO:teuthology.orchestra.run.smithi131.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-26T22:47:09.985 INFO:teuthology.orchestra.run.smithi131.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-26T22:47:09.985 INFO:teuthology.orchestra.run.smithi131.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-26T22:47:09.985 INFO:teuthology.orchestra.run.smithi131.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-26T22:47:09.986 INFO:teuthology.orchestra.run.smithi131.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-26T22:47:09.986 INFO:teuthology.orchestra.run.smithi131.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-26T22:47:09.986 INFO:teuthology.orchestra.run.smithi131.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-26T22:47:09.986 INFO:teuthology.orchestra.run.smithi131.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-26T22:47:09.986 INFO:teuthology.orchestra.run.smithi131.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-26T22:47:09.986 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-26T22:47:09.986 INFO:teuthology.orchestra.run.smithi131.stdout: python3-ply-3.11-14.el9.noarch 2024-05-26T22:47:09.986 INFO:teuthology.orchestra.run.smithi131.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-26T22:47:09.986 INFO:teuthology.orchestra.run.smithi131.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-26T22:47:09.986 INFO:teuthology.orchestra.run.smithi131.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-26T22:47:09.986 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-26T22:47:09.986 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-26T22:47:09.986 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-26T22:47:09.987 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-26T22:47:09.987 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-26T22:47:09.987 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-26T22:47:09.987 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rados-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-26T22:47:09.987 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rbd-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-26T22:47:09.987 INFO:teuthology.orchestra.run.smithi131.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-26T22:47:09.987 INFO:teuthology.orchestra.run.smithi131.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-26T22:47:09.987 INFO:teuthology.orchestra.run.smithi131.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-26T22:47:09.987 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rgw-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-26T22:47:09.987 INFO:teuthology.orchestra.run.smithi131.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-26T22:47:09.987 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-26T22:47:09.987 INFO:teuthology.orchestra.run.smithi131.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-26T22:47:09.987 INFO:teuthology.orchestra.run.smithi131.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-26T22:47:09.988 INFO:teuthology.orchestra.run.smithi131.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-26T22:47:09.988 INFO:teuthology.orchestra.run.smithi131.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-26T22:47:09.988 INFO:teuthology.orchestra.run.smithi131.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-26T22:47:09.988 INFO:teuthology.orchestra.run.smithi131.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-26T22:47:09.988 INFO:teuthology.orchestra.run.smithi131.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-26T22:47:09.988 INFO:teuthology.orchestra.run.smithi131.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-26T22:47:09.988 INFO:teuthology.orchestra.run.smithi131.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-26T22:47:09.988 INFO:teuthology.orchestra.run.smithi131.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-26T22:47:09.988 INFO:teuthology.orchestra.run.smithi131.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-26T22:47:09.988 INFO:teuthology.orchestra.run.smithi131.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-26T22:47:09.988 INFO:teuthology.orchestra.run.smithi131.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-26T22:47:09.988 INFO:teuthology.orchestra.run.smithi131.stdout: rbd-fuse-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-26T22:47:09.988 INFO:teuthology.orchestra.run.smithi131.stdout: rbd-mirror-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-26T22:47:09.989 INFO:teuthology.orchestra.run.smithi131.stdout: rbd-nbd-2:19.0.0-2857.g0ef46a24.el9.x86_64 2024-05-26T22:47:09.989 INFO:teuthology.orchestra.run.smithi131.stdout: re2-1:20211101-3.el9.x86_64 2024-05-26T22:47:09.989 INFO:teuthology.orchestra.run.smithi131.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-26T22:47:09.989 INFO:teuthology.orchestra.run.smithi131.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-26T22:47:09.989 INFO:teuthology.orchestra.run.smithi131.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-26T22:47:09.989 INFO:teuthology.orchestra.run.smithi131.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-26T22:47:09.989 INFO:teuthology.orchestra.run.smithi131.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-26T22:47:09.990 INFO:teuthology.orchestra.run.smithi131.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-26T22:47:09.990 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:47:09.990 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-05-26T22:47:10.599 DEBUG:teuthology.parallel:result is None 2024-05-26T22:47:10.599 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=0ef46a246e1118e5adafa1857b4189e7d780c816 2024-05-26T22:47:10.745 DEBUG:teuthology.orchestra.run.smithi131:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-26T22:47:10.802 INFO:teuthology.orchestra.run.smithi131.stdout:19.0.0-2857.g0ef46a24.el9 2024-05-26T22:47:10.802 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2857.g0ef46a24.el9 2024-05-26T22:47:10.802 INFO:teuthology.task.install:The correct ceph version 19.0.0-2857.g0ef46a24 is installed. 2024-05-26T22:47:10.804 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=0ef46a246e1118e5adafa1857b4189e7d780c816 2024-05-26T22:47:10.954 DEBUG:teuthology.orchestra.run.smithi187:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-26T22:47:11.025 INFO:teuthology.orchestra.run.smithi187.stdout:19.0.0-2857.g0ef46a24.el9 2024-05-26T22:47:11.025 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2857.g0ef46a24.el9 2024-05-26T22:47:11.025 INFO:teuthology.task.install:The correct ceph version 19.0.0-2857.g0ef46a24 is installed. 2024-05-26T22:47:11.027 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-05-26T22:47:11.027 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-26T22:47:11.027 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-26T22:47:11.099 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-05-26T22:47:11.099 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-26T22:47:11.194 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-05-26T22:47:11.194 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-26T22:47:11.194 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/usr/bin/daemon-helper 2024-05-26T22:47:11.266 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-26T22:47:11.378 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-05-26T22:47:11.378 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd of=/usr/bin/daemon-helper 2024-05-26T22:47:11.468 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-26T22:47:11.549 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-05-26T22:47:11.549 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-26T22:47:11.550 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-26T22:47:11.618 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-26T22:47:11.725 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-05-26T22:47:11.726 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-26T22:47:11.812 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-26T22:47:11.934 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-05-26T22:47:11.934 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-26T22:47:11.934 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/usr/bin/stdin-killer 2024-05-26T22:47:11.998 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-26T22:47:12.099 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-05-26T22:47:12.100 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd of=/usr/bin/stdin-killer 2024-05-26T22:47:12.188 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-26T22:47:12.272 INFO:teuthology.run_tasks:Running task ceph... 2024-05-26T22:47:12.348 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-05-26T22:47:12.348 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod 777 /var/log/ceph 2024-05-26T22:47:12.353 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod 777 /var/log/ceph 2024-05-26T22:47:12.440 INFO:tasks.ceph:Disabling ceph logrotate... 2024-05-26T22:47:12.441 DEBUG:teuthology.orchestra.run.smithi131:> sudo rm -f -- /etc/logrotate.d/ceph 2024-05-26T22:47:12.445 DEBUG:teuthology.orchestra.run.smithi187:> sudo rm -f -- /etc/logrotate.d/ceph 2024-05-26T22:47:12.524 INFO:tasks.ceph:Creating extra log directories... 2024-05-26T22:47:12.524 DEBUG:teuthology.orchestra.run.smithi131:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-05-26T22:47:12.529 DEBUG:teuthology.orchestra.run.smithi187:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-05-26T22:47:12.620 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-05-26T22:47:12.621 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-05-26T22:47:12.621 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-05-26T22:47:12.621 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-26T22:47:12.621 DEBUG:teuthology.orchestra.run.smithi131:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-05-26T22:47:12.659 DEBUG:teuthology.orchestra.run.smithi131:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-05-26T22:47:12.835 DEBUG:teuthology.orchestra.run.smithi131:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-05-26T22:47:12.895 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-05-26T22:47:12.895 DEBUG:teuthology.orchestra.run.smithi187:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-05-26T22:47:12.951 DEBUG:teuthology.orchestra.run.smithi187:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-05-26T22:47:13.183 DEBUG:teuthology.orchestra.run.smithi187:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-05-26T22:47:13.277 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-05-26T22:47:13.277 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Replacing daemon mds'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-05-26T22:47:13.277 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-05-26_21:24:02-fs-squid-distro-default-smithi/7727073', 'branch': 'squid', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/exports}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7727073', 'kernel': {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'}, 'ktype': 'upstream', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'teuthology-2024-05-26_21:24:02-fs-squid-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'squid'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Replacing daemon mds'], 'sha1': '0ef46a246e1118e5adafa1857b4189e7d780c816'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '0ef46a246e1118e5adafa1857b4189e7d780c816'}}, 'kclient': {'syntax': 'v2'}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'subvolume_version': 2, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'squid', 'sha1': '7d7cb32d1d130135a98121a564d709025c2154b6'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 100, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 1555, 'sha1': '0ef46a246e1118e5adafa1857b4189e7d780c816', 'sleep_before_teardown': 0, 'subset': '10/512', 'suite': 'fs', 'suite_branch': 'squid', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_7d7cb32d1d130135a98121a564d709025c2154b6/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': '7d7cb32d1d130135a98121a564d709025c2154b6', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi131.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBM+HHdqoC8qRnkJTvrEdj3BbvQXlGnopC9WZsb4L3ON/a+jU2V1bp17hNvc2hmkmrRCpcdsegdRK2GiGTBht86E=', 'smithi187.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBau4rbBJGj2tQGP8Jid05zFpRKpZrP02Gdl97eTNpSIdAVmKwCB5s42zI05KYj+KQSZgVOqEBuYykhgUBFGmvY='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '0ef46a246e1118e5adafa1857b4189e7d780c816'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Replacing daemon mds'], 'sha1': '0ef46a246e1118e5adafa1857b4189e7d780c816', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'kclient': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_exports']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': 'e7ae7580c417d6ab27fda98fecadca5d9f85d63e', 'timestamp': '2024-05-26_21:24:02', 'tube': 'smithi', 'user': 'teuthology', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2698841'} 2024-05-26T22:47:13.278 DEBUG:teuthology.orchestra.run.smithi131:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-05-26T22:47:13.283 DEBUG:teuthology.orchestra.run.smithi187:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-05-26T22:47:13.337 DEBUG:teuthology.orchestra.run.smithi131:> sudo install -d -m0777 -- /var/run/ceph 2024-05-26T22:47:13.367 DEBUG:teuthology.orchestra.run.smithi187:> sudo install -d -m0777 -- /var/run/ceph 2024-05-26T22:47:13.458 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-26T22:47:13.458 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/scratch_devs of=/dev/stdout 2024-05-26T22:47:13.497 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-26T22:47:13.497 DEBUG:teuthology.orchestra.run.smithi131:> stat /dev/vg_nvme/lv_1 2024-05-26T22:47:13.578 INFO:teuthology.orchestra.run.smithi131.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-26T22:47:13.578 INFO:teuthology.orchestra.run.smithi131.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-26T22:47:13.578 INFO:teuthology.orchestra.run.smithi131.stdout:Device: 6h/6d Inode: 633 Links: 1 2024-05-26T22:47:13.578 INFO:teuthology.orchestra.run.smithi131.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-26T22:47:13.578 INFO:teuthology.orchestra.run.smithi131.stdout:Context: system_u:object_r:device_t:s0 2024-05-26T22:47:13.578 INFO:teuthology.orchestra.run.smithi131.stdout:Access: 2024-05-26 22:47:02.230508237 +0000 2024-05-26T22:47:13.578 INFO:teuthology.orchestra.run.smithi131.stdout:Modify: 2024-05-26 22:43:49.255104373 +0000 2024-05-26T22:47:13.578 INFO:teuthology.orchestra.run.smithi131.stdout:Change: 2024-05-26 22:43:49.255104373 +0000 2024-05-26T22:47:13.578 INFO:teuthology.orchestra.run.smithi131.stdout: Birth: 2024-05-26 22:43:49.255104373 +0000 2024-05-26T22:47:13.579 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-26T22:47:13.679 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records in 2024-05-26T22:47:13.680 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records out 2024-05-26T22:47:13.680 INFO:teuthology.orchestra.run.smithi131.stderr:512 bytes copied, 0.000419122 s, 1.2 MB/s 2024-05-26T22:47:13.683 DEBUG:teuthology.orchestra.run.smithi131:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-26T22:47:13.721 DEBUG:teuthology.orchestra.run.smithi131:> stat /dev/vg_nvme/lv_2 2024-05-26T22:47:13.799 INFO:teuthology.orchestra.run.smithi131.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-26T22:47:13.799 INFO:teuthology.orchestra.run.smithi131.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-26T22:47:13.799 INFO:teuthology.orchestra.run.smithi131.stdout:Device: 6h/6d Inode: 651 Links: 1 2024-05-26T22:47:13.799 INFO:teuthology.orchestra.run.smithi131.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-26T22:47:13.799 INFO:teuthology.orchestra.run.smithi131.stdout:Context: system_u:object_r:device_t:s0 2024-05-26T22:47:13.799 INFO:teuthology.orchestra.run.smithi131.stdout:Access: 2024-05-26 22:47:02.230508237 +0000 2024-05-26T22:47:13.800 INFO:teuthology.orchestra.run.smithi131.stdout:Modify: 2024-05-26 22:43:49.872091403 +0000 2024-05-26T22:47:13.800 INFO:teuthology.orchestra.run.smithi131.stdout:Change: 2024-05-26 22:43:49.872091403 +0000 2024-05-26T22:47:13.800 INFO:teuthology.orchestra.run.smithi131.stdout: Birth: 2024-05-26 22:43:49.872091403 +0000 2024-05-26T22:47:13.800 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-26T22:47:13.897 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records in 2024-05-26T22:47:13.897 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records out 2024-05-26T22:47:13.898 INFO:teuthology.orchestra.run.smithi131.stderr:512 bytes copied, 0.00037489 s, 1.4 MB/s 2024-05-26T22:47:13.900 DEBUG:teuthology.orchestra.run.smithi131:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-26T22:47:13.940 DEBUG:teuthology.orchestra.run.smithi131:> stat /dev/vg_nvme/lv_3 2024-05-26T22:47:14.018 INFO:teuthology.orchestra.run.smithi131.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-26T22:47:14.019 INFO:teuthology.orchestra.run.smithi131.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-26T22:47:14.019 INFO:teuthology.orchestra.run.smithi131.stdout:Device: 6h/6d Inode: 670 Links: 1 2024-05-26T22:47:14.019 INFO:teuthology.orchestra.run.smithi131.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-26T22:47:14.019 INFO:teuthology.orchestra.run.smithi131.stdout:Context: system_u:object_r:device_t:s0 2024-05-26T22:47:14.019 INFO:teuthology.orchestra.run.smithi131.stdout:Access: 2024-05-26 22:47:02.231508216 +0000 2024-05-26T22:47:14.019 INFO:teuthology.orchestra.run.smithi131.stdout:Modify: 2024-05-26 22:43:50.477078685 +0000 2024-05-26T22:47:14.019 INFO:teuthology.orchestra.run.smithi131.stdout:Change: 2024-05-26 22:43:50.477078685 +0000 2024-05-26T22:47:14.019 INFO:teuthology.orchestra.run.smithi131.stdout: Birth: 2024-05-26 22:43:50.477078685 +0000 2024-05-26T22:47:14.019 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-26T22:47:14.118 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records in 2024-05-26T22:47:14.118 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records out 2024-05-26T22:47:14.118 INFO:teuthology.orchestra.run.smithi131.stderr:512 bytes copied, 0.000280862 s, 1.8 MB/s 2024-05-26T22:47:14.121 DEBUG:teuthology.orchestra.run.smithi131:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-26T22:47:14.159 DEBUG:teuthology.orchestra.run.smithi131:> stat /dev/vg_nvme/lv_4 2024-05-26T22:47:14.238 INFO:teuthology.orchestra.run.smithi131.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-26T22:47:14.238 INFO:teuthology.orchestra.run.smithi131.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-26T22:47:14.238 INFO:teuthology.orchestra.run.smithi131.stdout:Device: 6h/6d Inode: 691 Links: 1 2024-05-26T22:47:14.238 INFO:teuthology.orchestra.run.smithi131.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-26T22:47:14.238 INFO:teuthology.orchestra.run.smithi131.stdout:Context: system_u:object_r:device_t:s0 2024-05-26T22:47:14.238 INFO:teuthology.orchestra.run.smithi131.stdout:Access: 2024-05-26 22:47:02.232508195 +0000 2024-05-26T22:47:14.238 INFO:teuthology.orchestra.run.smithi131.stdout:Modify: 2024-05-26 22:43:51.053066577 +0000 2024-05-26T22:47:14.238 INFO:teuthology.orchestra.run.smithi131.stdout:Change: 2024-05-26 22:43:51.053066577 +0000 2024-05-26T22:47:14.238 INFO:teuthology.orchestra.run.smithi131.stdout: Birth: 2024-05-26 22:43:51.053066577 +0000 2024-05-26T22:47:14.239 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-26T22:47:14.339 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records in 2024-05-26T22:47:14.340 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records out 2024-05-26T22:47:14.340 INFO:teuthology.orchestra.run.smithi131.stderr:512 bytes copied, 0.000295606 s, 1.7 MB/s 2024-05-26T22:47:14.342 DEBUG:teuthology.orchestra.run.smithi131:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-26T22:47:14.382 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-26T22:47:14.382 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-05-26T22:47:14.382 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/scratch_devs of=/dev/stdout 2024-05-26T22:47:14.439 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-26T22:47:14.440 DEBUG:teuthology.orchestra.run.smithi187:> stat /dev/vg_nvme/lv_1 2024-05-26T22:47:14.491 INFO:teuthology.orchestra.run.smithi187.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-26T22:47:14.491 INFO:teuthology.orchestra.run.smithi187.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-26T22:47:14.491 INFO:teuthology.orchestra.run.smithi187.stdout:Device: 6h/6d Inode: 638 Links: 1 2024-05-26T22:47:14.491 INFO:teuthology.orchestra.run.smithi187.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-26T22:47:14.491 INFO:teuthology.orchestra.run.smithi187.stdout:Context: system_u:object_r:device_t:s0 2024-05-26T22:47:14.491 INFO:teuthology.orchestra.run.smithi187.stdout:Access: 2024-05-26 22:46:54.951504315 +0000 2024-05-26T22:47:14.491 INFO:teuthology.orchestra.run.smithi187.stdout:Modify: 2024-05-26 22:43:53.534164581 +0000 2024-05-26T22:47:14.491 INFO:teuthology.orchestra.run.smithi187.stdout:Change: 2024-05-26 22:43:53.534164581 +0000 2024-05-26T22:47:14.491 INFO:teuthology.orchestra.run.smithi187.stdout: Birth: 2024-05-26 22:43:53.534164581 +0000 2024-05-26T22:47:14.492 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-26T22:47:14.570 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records in 2024-05-26T22:47:14.570 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records out 2024-05-26T22:47:14.570 INFO:teuthology.orchestra.run.smithi187.stderr:512 bytes copied, 0.000369516 s, 1.4 MB/s 2024-05-26T22:47:14.574 DEBUG:teuthology.orchestra.run.smithi187:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-26T22:47:14.629 DEBUG:teuthology.orchestra.run.smithi187:> stat /dev/vg_nvme/lv_2 2024-05-26T22:47:14.677 INFO:teuthology.orchestra.run.smithi187.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-26T22:47:14.677 INFO:teuthology.orchestra.run.smithi187.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-26T22:47:14.677 INFO:teuthology.orchestra.run.smithi187.stdout:Device: 6h/6d Inode: 659 Links: 1 2024-05-26T22:47:14.677 INFO:teuthology.orchestra.run.smithi187.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-26T22:47:14.677 INFO:teuthology.orchestra.run.smithi187.stdout:Context: system_u:object_r:device_t:s0 2024-05-26T22:47:14.677 INFO:teuthology.orchestra.run.smithi187.stdout:Access: 2024-05-26 22:46:54.952504299 +0000 2024-05-26T22:47:14.677 INFO:teuthology.orchestra.run.smithi187.stdout:Modify: 2024-05-26 22:43:54.154154869 +0000 2024-05-26T22:47:14.678 INFO:teuthology.orchestra.run.smithi187.stdout:Change: 2024-05-26 22:43:54.154154869 +0000 2024-05-26T22:47:14.678 INFO:teuthology.orchestra.run.smithi187.stdout: Birth: 2024-05-26 22:43:54.154154869 +0000 2024-05-26T22:47:14.678 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-26T22:47:14.756 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records in 2024-05-26T22:47:14.757 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records out 2024-05-26T22:47:14.757 INFO:teuthology.orchestra.run.smithi187.stderr:512 bytes copied, 0.000349685 s, 1.5 MB/s 2024-05-26T22:47:14.760 DEBUG:teuthology.orchestra.run.smithi187:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-26T22:47:14.814 DEBUG:teuthology.orchestra.run.smithi187:> stat /dev/vg_nvme/lv_3 2024-05-26T22:47:14.864 INFO:teuthology.orchestra.run.smithi187.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-26T22:47:14.864 INFO:teuthology.orchestra.run.smithi187.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-26T22:47:14.864 INFO:teuthology.orchestra.run.smithi187.stdout:Device: 6h/6d Inode: 678 Links: 1 2024-05-26T22:47:14.864 INFO:teuthology.orchestra.run.smithi187.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-26T22:47:14.864 INFO:teuthology.orchestra.run.smithi187.stdout:Context: system_u:object_r:device_t:s0 2024-05-26T22:47:14.864 INFO:teuthology.orchestra.run.smithi187.stdout:Access: 2024-05-26 22:46:54.953504284 +0000 2024-05-26T22:47:14.864 INFO:teuthology.orchestra.run.smithi187.stdout:Modify: 2024-05-26 22:43:54.779145078 +0000 2024-05-26T22:47:14.864 INFO:teuthology.orchestra.run.smithi187.stdout:Change: 2024-05-26 22:43:54.779145078 +0000 2024-05-26T22:47:14.864 INFO:teuthology.orchestra.run.smithi187.stdout: Birth: 2024-05-26 22:43:54.779145078 +0000 2024-05-26T22:47:14.865 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-26T22:47:14.945 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records in 2024-05-26T22:47:14.945 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records out 2024-05-26T22:47:14.945 INFO:teuthology.orchestra.run.smithi187.stderr:512 bytes copied, 0.000340982 s, 1.5 MB/s 2024-05-26T22:47:14.949 DEBUG:teuthology.orchestra.run.smithi187:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-26T22:47:15.003 DEBUG:teuthology.orchestra.run.smithi187:> stat /dev/vg_nvme/lv_4 2024-05-26T22:47:15.054 INFO:teuthology.orchestra.run.smithi187.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-26T22:47:15.055 INFO:teuthology.orchestra.run.smithi187.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-26T22:47:15.055 INFO:teuthology.orchestra.run.smithi187.stdout:Device: 6h/6d Inode: 694 Links: 1 2024-05-26T22:47:15.055 INFO:teuthology.orchestra.run.smithi187.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-26T22:47:15.055 INFO:teuthology.orchestra.run.smithi187.stdout:Context: system_u:object_r:device_t:s0 2024-05-26T22:47:15.055 INFO:teuthology.orchestra.run.smithi187.stdout:Access: 2024-05-26 22:46:54.953504284 +0000 2024-05-26T22:47:15.055 INFO:teuthology.orchestra.run.smithi187.stdout:Modify: 2024-05-26 22:43:55.392135475 +0000 2024-05-26T22:47:15.055 INFO:teuthology.orchestra.run.smithi187.stdout:Change: 2024-05-26 22:43:55.392135475 +0000 2024-05-26T22:47:15.055 INFO:teuthology.orchestra.run.smithi187.stdout: Birth: 2024-05-26 22:43:55.392135475 +0000 2024-05-26T22:47:15.056 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-26T22:47:15.132 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records in 2024-05-26T22:47:15.133 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records out 2024-05-26T22:47:15.133 INFO:teuthology.orchestra.run.smithi187.stderr:512 bytes copied, 0.000357241 s, 1.4 MB/s 2024-05-26T22:47:15.136 DEBUG:teuthology.orchestra.run.smithi187:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-26T22:47:15.189 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-26T22:47:15.189 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi131.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi187.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-05-26T22:47:15.189 INFO:tasks.ceph:Generating config... 2024-05-26T22:47:15.191 INFO:tasks.ceph:[client] client mount timeout = 600 2024-05-26T22:47:15.191 INFO:tasks.ceph:[client] debug client = 20 2024-05-26T22:47:15.191 INFO:tasks.ceph:[client] debug ms = 1 2024-05-26T22:47:15.191 INFO:tasks.ceph:[client] pid file = 2024-05-26T22:47:15.191 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-05-26T22:47:15.191 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-05-26T22:47:15.191 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-05-26T22:47:15.191 INFO:tasks.ceph:[mds] debug mds = 20 2024-05-26T22:47:15.191 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-05-26T22:47:15.192 INFO:tasks.ceph:[mds] debug ms = 1 2024-05-26T22:47:15.192 INFO:tasks.ceph:[mds] mds debug frag = True 2024-05-26T22:47:15.192 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-05-26T22:47:15.192 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-05-26T22:47:15.192 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-05-26T22:47:15.192 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-05-26T22:47:15.192 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-05-26T22:47:15.192 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-05-26T22:47:15.192 INFO:tasks.ceph:[mgr] debug client = 20 2024-05-26T22:47:15.192 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-05-26T22:47:15.192 INFO:tasks.ceph:[mgr] debug ms = 1 2024-05-26T22:47:15.192 INFO:tasks.ceph:[mon] debug mon = 20 2024-05-26T22:47:15.192 INFO:tasks.ceph:[mon] debug ms = 1 2024-05-26T22:47:15.192 INFO:tasks.ceph:[mon] debug paxos = 20 2024-05-26T22:47:15.193 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-05-26T22:47:15.193 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-05-26T22:47:15.193 INFO:tasks.ceph:[osd] bdev async discard = True 2024-05-26T22:47:15.193 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-05-26T22:47:15.193 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-05-26T22:47:15.193 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-05-26T22:47:15.193 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-05-26T22:47:15.193 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-05-26T22:47:15.193 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-05-26T22:47:15.193 INFO:tasks.ceph:[osd] debug ms = 1 2024-05-26T22:47:15.193 INFO:tasks.ceph:[osd] debug osd = 20 2024-05-26T22:47:15.193 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-05-26T22:47:15.194 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-05-26T22:47:15.194 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-05-26T22:47:15.194 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-05-26T22:47:15.194 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-05-26T22:47:15.194 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-05-26T22:47:15.194 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-05-26T22:47:15.194 INFO:tasks.ceph:Setting up mon.a... 2024-05-26T22:47:15.194 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-05-26T22:47:15.347 INFO:teuthology.orchestra.run.smithi131.stdout:creating /etc/ceph/ceph.keyring 2024-05-26T22:47:15.355 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-05-26T22:47:15.468 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-26T22:47:15.541 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.131'), ('mon.b', '172.21.15.187'), ('mon.c', '[v2:172.21.15.187:3301,v1:172.21.15.187:6790]')] 2024-05-26T22:47:15.541 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.131,172.21.15.187,[v2:172.21.15.187:3301,v1:172.21.15.187:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-05-26T22:47:15.542 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-26T22:47:15.542 DEBUG:teuthology.orchestra.run.smithi131:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-05-26T22:47:15.592 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.131 --add b 172.21.15.187 --addv c '[v2:172.21.15.187:3301,v1:172.21.15.187:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-05-26T22:47:15.741 INFO:teuthology.orchestra.run.smithi131.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-05-26T22:47:15.741 INFO:teuthology.orchestra.run.smithi131.stdout:monmaptool: generated fsid 46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f 2024-05-26T22:47:15.741 INFO:teuthology.orchestra.run.smithi131.stdout:setting min_mon_release = quincy 2024-05-26T22:47:15.741 INFO:teuthology.orchestra.run.smithi131.stdout:epoch 0 2024-05-26T22:47:15.741 INFO:teuthology.orchestra.run.smithi131.stdout:fsid 46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f 2024-05-26T22:47:15.741 INFO:teuthology.orchestra.run.smithi131.stdout:last_changed 2024-05-26T22:47:15.742487+0000 2024-05-26T22:47:15.741 INFO:teuthology.orchestra.run.smithi131.stdout:created 2024-05-26T22:47:15.742487+0000 2024-05-26T22:47:15.741 INFO:teuthology.orchestra.run.smithi131.stdout:min_mon_release 17 (quincy) 2024-05-26T22:47:15.741 INFO:teuthology.orchestra.run.smithi131.stdout:election_strategy: 1 2024-05-26T22:47:15.742 INFO:teuthology.orchestra.run.smithi131.stdout:0: [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] mon.a 2024-05-26T22:47:15.742 INFO:teuthology.orchestra.run.smithi131.stdout:1: [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] mon.b 2024-05-26T22:47:15.742 INFO:teuthology.orchestra.run.smithi131.stdout:2: [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] mon.c 2024-05-26T22:47:15.742 INFO:teuthology.orchestra.run.smithi131.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-05-26T22:47:15.744 DEBUG:teuthology.orchestra.run.smithi131:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-05-26T22:47:15.792 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f... 2024-05-26T22:47:15.793 DEBUG:teuthology.orchestra.run.smithi131:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-26T22:47:15.838 DEBUG:teuthology.orchestra.run.smithi187:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-26T22:47:15.975 INFO:teuthology.orchestra.run.smithi131.stdout:[global] 2024-05-26T22:47:15.975 INFO:teuthology.orchestra.run.smithi131.stdout: chdir = "" 2024-05-26T22:47:15.975 INFO:teuthology.orchestra.run.smithi131.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-26T22:47:15.975 INFO:teuthology.orchestra.run.smithi131.stdout: auth supported = cephx 2024-05-26T22:47:15.975 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:47:15.975 INFO:teuthology.orchestra.run.smithi131.stdout: filestore xattr use omap = true 2024-05-26T22:47:15.975 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:47:15.975 INFO:teuthology.orchestra.run.smithi131.stdout: mon clock drift allowed = 1.000 2024-05-26T22:47:15.976 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:47:15.976 INFO:teuthology.orchestra.run.smithi131.stdout: osd crush chooseleaf type = 0 2024-05-26T22:47:15.976 INFO:teuthology.orchestra.run.smithi131.stdout: auth debug = true 2024-05-26T22:47:15.976 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:47:15.976 INFO:teuthology.orchestra.run.smithi131.stdout: ms die on old message = true 2024-05-26T22:47:15.976 INFO:teuthology.orchestra.run.smithi131.stdout: ms die on bug = true 2024-05-26T22:47:15.976 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:47:15.976 INFO:teuthology.orchestra.run.smithi131.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-26T22:47:15.976 INFO:teuthology.orchestra.run.smithi131.stdout: mon pg warn max object skew = 0 2024-05-26T22:47:15.976 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:47:15.976 INFO:teuthology.orchestra.run.smithi131.stdout: # disable pg_autoscaler by default for new pools 2024-05-26T22:47:15.976 INFO:teuthology.orchestra.run.smithi131.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-26T22:47:15.976 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:47:15.977 INFO:teuthology.orchestra.run.smithi131.stdout: osd pool default size = 2 2024-05-26T22:47:15.977 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:47:15.977 INFO:teuthology.orchestra.run.smithi131.stdout: mon osd allow primary affinity = true 2024-05-26T22:47:15.977 INFO:teuthology.orchestra.run.smithi131.stdout: mon osd allow pg remap = true 2024-05-26T22:47:15.977 INFO:teuthology.orchestra.run.smithi131.stdout: mon warn on legacy crush tunables = false 2024-05-26T22:47:15.977 INFO:teuthology.orchestra.run.smithi131.stdout: mon warn on crush straw calc version zero = false 2024-05-26T22:47:15.977 INFO:teuthology.orchestra.run.smithi131.stdout: mon warn on no sortbitwise = false 2024-05-26T22:47:15.977 INFO:teuthology.orchestra.run.smithi131.stdout: mon warn on osd down out interval zero = false 2024-05-26T22:47:15.977 INFO:teuthology.orchestra.run.smithi131.stdout: mon warn on too few osds = false 2024-05-26T22:47:15.977 INFO:teuthology.orchestra.run.smithi131.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-26T22:47:15.977 INFO:teuthology.orchestra.run.smithi131.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-26T22:47:15.977 INFO:teuthology.orchestra.run.smithi131.stdout: mon_allow_pool_size_one = true 2024-05-26T22:47:15.978 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:47:15.978 INFO:teuthology.orchestra.run.smithi131.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-26T22:47:15.978 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:47:15.978 INFO:teuthology.orchestra.run.smithi131.stdout: osd default data pool replay window = 5 2024-05-26T22:47:15.978 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:47:15.978 INFO:teuthology.orchestra.run.smithi131.stdout: mon allow pool delete = true 2024-05-26T22:47:15.978 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:47:15.978 INFO:teuthology.orchestra.run.smithi131.stdout: mon cluster log file level = debug 2024-05-26T22:47:15.978 INFO:teuthology.orchestra.run.smithi131.stdout: debug asserts on shutdown = true 2024-05-26T22:47:15.978 INFO:teuthology.orchestra.run.smithi131.stdout: mon health detail to clog = false 2024-05-26T22:47:15.978 INFO:teuthology.orchestra.run.smithi131.stdout: mon host = "172.21.15.131,172.21.15.187,[v2:172.21.15.187:3301,v1:172.21.15.187:6790]" 2024-05-26T22:47:15.978 INFO:teuthology.orchestra.run.smithi131.stdout: ms die on skipped message = False 2024-05-26T22:47:15.979 INFO:teuthology.orchestra.run.smithi131.stdout: fsid = 46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f 2024-05-26T22:47:15.979 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:47:15.979 INFO:teuthology.orchestra.run.smithi131.stdout:[osd] 2024-05-26T22:47:15.979 INFO:teuthology.orchestra.run.smithi131.stdout: osd journal size = 100 2024-05-26T22:47:15.979 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:47:15.979 INFO:teuthology.orchestra.run.smithi131.stdout: osd scrub load threshold = 5.0 2024-05-26T22:47:15.979 INFO:teuthology.orchestra.run.smithi131.stdout: osd scrub max interval = 600 2024-05-26T22:47:15.979 INFO:teuthology.orchestra.run.smithi131.stdout: osd mclock profile = high_recovery_ops 2024-05-26T22:47:15.979 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:47:15.979 INFO:teuthology.orchestra.run.smithi131.stdout: osd recover clone overlap = true 2024-05-26T22:47:15.979 INFO:teuthology.orchestra.run.smithi131.stdout: osd recovery max chunk = 1048576 2024-05-26T22:47:15.979 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:47:15.979 INFO:teuthology.orchestra.run.smithi131.stdout: osd debug shutdown = true 2024-05-26T22:47:15.979 INFO:teuthology.orchestra.run.smithi131.stdout: osd debug op order = true 2024-05-26T22:47:15.980 INFO:teuthology.orchestra.run.smithi131.stdout: osd debug verify stray on activate = true 2024-05-26T22:47:15.980 INFO:teuthology.orchestra.run.smithi131.stdout: osd debug trim objects = true 2024-05-26T22:47:15.980 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:47:15.980 INFO:teuthology.orchestra.run.smithi131.stdout: osd open classes on start = true 2024-05-26T22:47:15.980 INFO:teuthology.orchestra.run.smithi131.stdout: osd debug pg log writeout = true 2024-05-26T22:47:15.980 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:47:15.980 INFO:teuthology.orchestra.run.smithi131.stdout: osd deep scrub update digest min age = 30 2024-05-26T22:47:15.980 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:47:15.980 INFO:teuthology.orchestra.run.smithi131.stdout: osd map max advance = 10 2024-05-26T22:47:15.980 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:47:15.980 INFO:teuthology.orchestra.run.smithi131.stdout: journal zero on create = true 2024-05-26T22:47:15.980 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:47:15.980 INFO:teuthology.orchestra.run.smithi131.stdout: filestore ondisk finisher threads = 3 2024-05-26T22:47:15.980 INFO:teuthology.orchestra.run.smithi131.stdout: filestore apply finisher threads = 3 2024-05-26T22:47:15.981 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:47:15.981 INFO:teuthology.orchestra.run.smithi131.stdout: bdev debug aio = true 2024-05-26T22:47:15.981 INFO:teuthology.orchestra.run.smithi131.stdout: osd debug misdirected ops = true 2024-05-26T22:47:15.981 INFO:teuthology.orchestra.run.smithi131.stdout: bdev async discard = True 2024-05-26T22:47:15.981 INFO:teuthology.orchestra.run.smithi131.stdout: bdev enable discard = True 2024-05-26T22:47:15.981 INFO:teuthology.orchestra.run.smithi131.stdout: bluestore allocator = bitmap 2024-05-26T22:47:15.981 INFO:teuthology.orchestra.run.smithi131.stdout: bluestore block size = 96636764160 2024-05-26T22:47:15.981 INFO:teuthology.orchestra.run.smithi131.stdout: bluestore fsck on mount = True 2024-05-26T22:47:15.981 INFO:teuthology.orchestra.run.smithi131.stdout: debug bluefs = 1/20 2024-05-26T22:47:15.981 INFO:teuthology.orchestra.run.smithi131.stdout: debug bluestore = 1/20 2024-05-26T22:47:15.981 INFO:teuthology.orchestra.run.smithi131.stdout: debug ms = 1 2024-05-26T22:47:15.982 INFO:teuthology.orchestra.run.smithi131.stdout: debug osd = 20 2024-05-26T22:47:15.982 INFO:teuthology.orchestra.run.smithi131.stdout: debug rocksdb = 4/10 2024-05-26T22:47:15.982 INFO:teuthology.orchestra.run.smithi131.stdout: mon osd backfillfull_ratio = 0.85 2024-05-26T22:47:15.982 INFO:teuthology.orchestra.run.smithi131.stdout: mon osd full ratio = 0.9 2024-05-26T22:47:15.982 INFO:teuthology.orchestra.run.smithi131.stdout: mon osd nearfull ratio = 0.8 2024-05-26T22:47:15.982 INFO:teuthology.orchestra.run.smithi131.stdout: osd failsafe full ratio = 0.95 2024-05-26T22:47:15.982 INFO:teuthology.orchestra.run.smithi131.stdout: osd objectstore = bluestore 2024-05-26T22:47:15.982 INFO:teuthology.orchestra.run.smithi131.stdout: osd op complaint time = 180 2024-05-26T22:47:15.982 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:47:15.982 INFO:teuthology.orchestra.run.smithi131.stdout:[mgr] 2024-05-26T22:47:15.982 INFO:teuthology.orchestra.run.smithi131.stdout: debug ms = 1 2024-05-26T22:47:15.982 INFO:teuthology.orchestra.run.smithi131.stdout: debug mgr = 20 2024-05-26T22:47:15.983 INFO:teuthology.orchestra.run.smithi131.stdout: debug mon = 20 2024-05-26T22:47:15.983 INFO:teuthology.orchestra.run.smithi131.stdout: debug auth = 20 2024-05-26T22:47:15.983 INFO:teuthology.orchestra.run.smithi131.stdout: mon reweight min pgs per osd = 4 2024-05-26T22:47:15.983 INFO:teuthology.orchestra.run.smithi131.stdout: mon reweight min bytes per osd = 10 2024-05-26T22:47:15.983 INFO:teuthology.orchestra.run.smithi131.stdout: mgr/telemetry/nag = false 2024-05-26T22:47:15.983 INFO:teuthology.orchestra.run.smithi131.stdout: debug client = 20 2024-05-26T22:47:15.983 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:47:15.983 INFO:teuthology.orchestra.run.smithi131.stdout:[mon] 2024-05-26T22:47:15.983 INFO:teuthology.orchestra.run.smithi131.stdout: debug ms = 1 2024-05-26T22:47:15.983 INFO:teuthology.orchestra.run.smithi131.stdout: debug mon = 20 2024-05-26T22:47:15.983 INFO:teuthology.orchestra.run.smithi131.stdout: debug paxos = 20 2024-05-26T22:47:15.983 INFO:teuthology.orchestra.run.smithi131.stdout: debug auth = 20 2024-05-26T22:47:15.983 INFO:teuthology.orchestra.run.smithi131.stdout: mon data avail warn = 5 2024-05-26T22:47:15.983 INFO:teuthology.orchestra.run.smithi131.stdout: mon mgr mkfs grace = 240 2024-05-26T22:47:15.984 INFO:teuthology.orchestra.run.smithi131.stdout: mon reweight min pgs per osd = 4 2024-05-26T22:47:15.984 INFO:teuthology.orchestra.run.smithi131.stdout: mon osd reporter subtree level = osd 2024-05-26T22:47:15.984 INFO:teuthology.orchestra.run.smithi131.stdout: mon osd prime pg temp = true 2024-05-26T22:47:15.984 INFO:teuthology.orchestra.run.smithi131.stdout: mon reweight min bytes per osd = 10 2024-05-26T22:47:15.984 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:47:15.984 INFO:teuthology.orchestra.run.smithi131.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-26T22:47:15.984 INFO:teuthology.orchestra.run.smithi131.stdout: auth mon ticket ttl = 660 # 11m 2024-05-26T22:47:15.984 INFO:teuthology.orchestra.run.smithi131.stdout: auth service ticket ttl = 240 # 4m 2024-05-26T22:47:15.984 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:47:15.984 INFO:teuthology.orchestra.run.smithi131.stdout: # don't complain about insecure global_id in the test suite 2024-05-26T22:47:15.984 INFO:teuthology.orchestra.run.smithi131.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-26T22:47:15.984 INFO:teuthology.orchestra.run.smithi131.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-26T22:47:15.984 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:47:15.985 INFO:teuthology.orchestra.run.smithi131.stdout: # 1m isn't quite enough 2024-05-26T22:47:15.985 INFO:teuthology.orchestra.run.smithi131.stdout: mon_down_mkfs_grace = 2m 2024-05-26T22:47:15.985 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:47:15.985 INFO:teuthology.orchestra.run.smithi131.stdout: mon_warn_on_filestore_osds = false 2024-05-26T22:47:15.985 INFO:teuthology.orchestra.run.smithi131.stdout: mon down mkfs grace = 300 2024-05-26T22:47:15.985 INFO:teuthology.orchestra.run.smithi131.stdout: mon op complaint time = 120 2024-05-26T22:47:15.985 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:47:15.985 INFO:teuthology.orchestra.run.smithi131.stdout:[client] 2024-05-26T22:47:15.985 INFO:teuthology.orchestra.run.smithi131.stdout: rgw cache enabled = true 2024-05-26T22:47:15.986 INFO:teuthology.orchestra.run.smithi131.stdout: rgw enable ops log = true 2024-05-26T22:47:15.986 INFO:teuthology.orchestra.run.smithi131.stdout: rgw enable usage log = true 2024-05-26T22:47:15.986 INFO:teuthology.orchestra.run.smithi131.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-26T22:47:15.986 INFO:teuthology.orchestra.run.smithi131.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-26T22:47:15.986 INFO:teuthology.orchestra.run.smithi131.stdout: client mount timeout = 600 2024-05-26T22:47:15.986 INFO:teuthology.orchestra.run.smithi131.stdout: debug client = 20 2024-05-26T22:47:15.986 INFO:teuthology.orchestra.run.smithi131.stdout: debug ms = 1 2024-05-26T22:47:15.986 INFO:teuthology.orchestra.run.smithi131.stdout: pid file = "" 2024-05-26T22:47:15.986 INFO:teuthology.orchestra.run.smithi131.stdout: rados mon op timeout = 900 2024-05-26T22:47:15.986 INFO:teuthology.orchestra.run.smithi131.stdout: rados osd op timeout = 900 2024-05-26T22:47:15.986 INFO:teuthology.orchestra.run.smithi131.stdout:[mon.a] 2024-05-26T22:47:15.986 INFO:teuthology.orchestra.run.smithi131.stdout:[mon.b] 2024-05-26T22:47:15.986 INFO:teuthology.orchestra.run.smithi131.stdout:[mon.c] 2024-05-26T22:47:15.987 INFO:teuthology.orchestra.run.smithi131.stdout:[mds.a] 2024-05-26T22:47:15.987 INFO:teuthology.orchestra.run.smithi131.stdout:[mds.c] 2024-05-26T22:47:15.987 INFO:teuthology.orchestra.run.smithi131.stdout:[mds.b] 2024-05-26T22:47:15.987 INFO:teuthology.orchestra.run.smithi131.stdout:[mds.d] 2024-05-26T22:47:15.987 INFO:teuthology.orchestra.run.smithi131.stdout:[mds] 2024-05-26T22:47:15.987 INFO:teuthology.orchestra.run.smithi131.stdout: debug mds = 20 2024-05-26T22:47:15.987 INFO:teuthology.orchestra.run.smithi131.stdout: debug mds balancer = 20 2024-05-26T22:47:15.988 INFO:teuthology.orchestra.run.smithi131.stdout: debug ms = 1 2024-05-26T22:47:15.988 INFO:teuthology.orchestra.run.smithi131.stdout: mds debug frag = True 2024-05-26T22:47:15.988 INFO:teuthology.orchestra.run.smithi131.stdout: mds debug scatterstat = True 2024-05-26T22:47:15.988 INFO:teuthology.orchestra.run.smithi131.stdout: mds op complaint time = 180 2024-05-26T22:47:15.988 INFO:teuthology.orchestra.run.smithi131.stdout: mds verify scatter = True 2024-05-26T22:47:15.988 INFO:teuthology.orchestra.run.smithi131.stdout: osd op complaint time = 180 2024-05-26T22:47:15.988 INFO:teuthology.orchestra.run.smithi131.stdout: rados mon op timeout = 900 2024-05-26T22:47:15.989 INFO:teuthology.orchestra.run.smithi131.stdout: rados osd op timeout = 900 2024-05-26T22:47:15.989 INFO:teuthology.orchestra.run.smithi187.stdout:[global] 2024-05-26T22:47:15.989 INFO:teuthology.orchestra.run.smithi187.stdout: chdir = "" 2024-05-26T22:47:15.990 INFO:teuthology.orchestra.run.smithi187.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-26T22:47:15.990 INFO:teuthology.orchestra.run.smithi187.stdout: auth supported = cephx 2024-05-26T22:47:15.990 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-05-26T22:47:15.990 INFO:teuthology.orchestra.run.smithi187.stdout: filestore xattr use omap = true 2024-05-26T22:47:15.990 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-05-26T22:47:15.991 INFO:teuthology.orchestra.run.smithi187.stdout: mon clock drift allowed = 1.000 2024-05-26T22:47:15.991 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-05-26T22:47:15.991 INFO:teuthology.orchestra.run.smithi187.stdout: osd crush chooseleaf type = 0 2024-05-26T22:47:15.991 INFO:teuthology.orchestra.run.smithi187.stdout: auth debug = true 2024-05-26T22:47:15.991 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-05-26T22:47:15.992 INFO:teuthology.orchestra.run.smithi187.stdout: ms die on old message = true 2024-05-26T22:47:15.992 INFO:teuthology.orchestra.run.smithi187.stdout: ms die on bug = true 2024-05-26T22:47:15.992 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-05-26T22:47:15.992 INFO:teuthology.orchestra.run.smithi187.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-26T22:47:15.992 INFO:teuthology.orchestra.run.smithi187.stdout: mon pg warn max object skew = 0 2024-05-26T22:47:15.993 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-05-26T22:47:15.993 INFO:teuthology.orchestra.run.smithi187.stdout: # disable pg_autoscaler by default for new pools 2024-05-26T22:47:15.993 INFO:teuthology.orchestra.run.smithi187.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-26T22:47:15.993 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-05-26T22:47:15.993 INFO:teuthology.orchestra.run.smithi187.stdout: osd pool default size = 2 2024-05-26T22:47:15.994 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-05-26T22:47:15.994 INFO:teuthology.orchestra.run.smithi187.stdout: mon osd allow primary affinity = true 2024-05-26T22:47:15.994 INFO:teuthology.orchestra.run.smithi187.stdout: mon osd allow pg remap = true 2024-05-26T22:47:15.994 INFO:teuthology.orchestra.run.smithi187.stdout: mon warn on legacy crush tunables = false 2024-05-26T22:47:15.995 INFO:teuthology.orchestra.run.smithi187.stdout: mon warn on crush straw calc version zero = false 2024-05-26T22:47:15.995 INFO:teuthology.orchestra.run.smithi187.stdout: mon warn on no sortbitwise = false 2024-05-26T22:47:15.995 INFO:teuthology.orchestra.run.smithi187.stdout: mon warn on osd down out interval zero = false 2024-05-26T22:47:15.995 INFO:teuthology.orchestra.run.smithi187.stdout: mon warn on too few osds = false 2024-05-26T22:47:15.996 INFO:teuthology.orchestra.run.smithi187.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-26T22:47:15.996 INFO:teuthology.orchestra.run.smithi187.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-26T22:47:15.996 INFO:teuthology.orchestra.run.smithi187.stdout: mon_allow_pool_size_one = true 2024-05-26T22:47:15.996 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-05-26T22:47:15.997 INFO:teuthology.orchestra.run.smithi187.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-26T22:47:15.997 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-05-26T22:47:15.997 INFO:teuthology.orchestra.run.smithi187.stdout: osd default data pool replay window = 5 2024-05-26T22:47:15.997 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-05-26T22:47:15.997 INFO:teuthology.orchestra.run.smithi187.stdout: mon allow pool delete = true 2024-05-26T22:47:15.998 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-05-26T22:47:15.998 INFO:teuthology.orchestra.run.smithi187.stdout: mon cluster log file level = debug 2024-05-26T22:47:15.998 INFO:teuthology.orchestra.run.smithi187.stdout: debug asserts on shutdown = true 2024-05-26T22:47:15.998 INFO:teuthology.orchestra.run.smithi187.stdout: mon health detail to clog = false 2024-05-26T22:47:15.998 INFO:teuthology.orchestra.run.smithi187.stdout: mon host = "172.21.15.131,172.21.15.187,[v2:172.21.15.187:3301,v1:172.21.15.187:6790]" 2024-05-26T22:47:15.999 INFO:teuthology.orchestra.run.smithi187.stdout: ms die on skipped message = False 2024-05-26T22:47:15.999 INFO:teuthology.orchestra.run.smithi187.stdout: fsid = 46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f 2024-05-26T22:47:15.999 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-05-26T22:47:15.999 INFO:teuthology.orchestra.run.smithi187.stdout:[osd] 2024-05-26T22:47:15.999 INFO:teuthology.orchestra.run.smithi187.stdout: osd journal size = 100 2024-05-26T22:47:15.999 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-05-26T22:47:16.000 INFO:teuthology.orchestra.run.smithi187.stdout: osd scrub load threshold = 5.0 2024-05-26T22:47:16.000 INFO:teuthology.orchestra.run.smithi187.stdout: osd scrub max interval = 600 2024-05-26T22:47:16.000 INFO:teuthology.orchestra.run.smithi187.stdout: osd mclock profile = high_recovery_ops 2024-05-26T22:47:16.000 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-05-26T22:47:16.000 INFO:teuthology.orchestra.run.smithi187.stdout: osd recover clone overlap = true 2024-05-26T22:47:16.000 INFO:teuthology.orchestra.run.smithi187.stdout: osd recovery max chunk = 1048576 2024-05-26T22:47:16.001 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-05-26T22:47:16.001 INFO:teuthology.orchestra.run.smithi187.stdout: osd debug shutdown = true 2024-05-26T22:47:16.001 INFO:teuthology.orchestra.run.smithi187.stdout: osd debug op order = true 2024-05-26T22:47:16.001 INFO:teuthology.orchestra.run.smithi187.stdout: osd debug verify stray on activate = true 2024-05-26T22:47:16.001 INFO:teuthology.orchestra.run.smithi187.stdout: osd debug trim objects = true 2024-05-26T22:47:16.002 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-05-26T22:47:16.002 INFO:teuthology.orchestra.run.smithi187.stdout: osd open classes on start = true 2024-05-26T22:47:16.002 INFO:teuthology.orchestra.run.smithi187.stdout: osd debug pg log writeout = true 2024-05-26T22:47:16.002 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-05-26T22:47:16.002 INFO:teuthology.orchestra.run.smithi187.stdout: osd deep scrub update digest min age = 30 2024-05-26T22:47:16.002 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-05-26T22:47:16.003 INFO:teuthology.orchestra.run.smithi187.stdout: osd map max advance = 10 2024-05-26T22:47:16.003 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-05-26T22:47:16.003 INFO:teuthology.orchestra.run.smithi187.stdout: journal zero on create = true 2024-05-26T22:47:16.003 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-05-26T22:47:16.003 INFO:teuthology.orchestra.run.smithi187.stdout: filestore ondisk finisher threads = 3 2024-05-26T22:47:16.003 INFO:teuthology.orchestra.run.smithi187.stdout: filestore apply finisher threads = 3 2024-05-26T22:47:16.004 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-05-26T22:47:16.004 INFO:teuthology.orchestra.run.smithi187.stdout: bdev debug aio = true 2024-05-26T22:47:16.004 INFO:teuthology.orchestra.run.smithi187.stdout: osd debug misdirected ops = true 2024-05-26T22:47:16.004 INFO:teuthology.orchestra.run.smithi187.stdout: bdev async discard = True 2024-05-26T22:47:16.004 INFO:teuthology.orchestra.run.smithi187.stdout: bdev enable discard = True 2024-05-26T22:47:16.004 INFO:teuthology.orchestra.run.smithi187.stdout: bluestore allocator = bitmap 2024-05-26T22:47:16.005 INFO:teuthology.orchestra.run.smithi187.stdout: bluestore block size = 96636764160 2024-05-26T22:47:16.005 INFO:teuthology.orchestra.run.smithi187.stdout: bluestore fsck on mount = True 2024-05-26T22:47:16.005 INFO:teuthology.orchestra.run.smithi187.stdout: debug bluefs = 1/20 2024-05-26T22:47:16.005 INFO:teuthology.orchestra.run.smithi187.stdout: debug bluestore = 1/20 2024-05-26T22:47:16.005 INFO:teuthology.orchestra.run.smithi187.stdout: debug ms = 1 2024-05-26T22:47:16.005 INFO:teuthology.orchestra.run.smithi187.stdout: debug osd = 20 2024-05-26T22:47:16.006 INFO:teuthology.orchestra.run.smithi187.stdout: debug rocksdb = 4/10 2024-05-26T22:47:16.006 INFO:teuthology.orchestra.run.smithi187.stdout: mon osd backfillfull_ratio = 0.85 2024-05-26T22:47:16.006 INFO:teuthology.orchestra.run.smithi187.stdout: mon osd full ratio = 0.9 2024-05-26T22:47:16.006 INFO:teuthology.orchestra.run.smithi187.stdout: mon osd nearfull ratio = 0.8 2024-05-26T22:47:16.006 INFO:teuthology.orchestra.run.smithi187.stdout: osd failsafe full ratio = 0.95 2024-05-26T22:47:16.006 INFO:teuthology.orchestra.run.smithi187.stdout: osd objectstore = bluestore 2024-05-26T22:47:16.007 INFO:teuthology.orchestra.run.smithi187.stdout: osd op complaint time = 180 2024-05-26T22:47:16.007 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-05-26T22:47:16.007 INFO:teuthology.orchestra.run.smithi187.stdout:[mgr] 2024-05-26T22:47:16.007 INFO:teuthology.orchestra.run.smithi187.stdout: debug ms = 1 2024-05-26T22:47:16.007 INFO:teuthology.orchestra.run.smithi187.stdout: debug mgr = 20 2024-05-26T22:47:16.008 INFO:teuthology.orchestra.run.smithi187.stdout: debug mon = 20 2024-05-26T22:47:16.008 INFO:teuthology.orchestra.run.smithi187.stdout: debug auth = 20 2024-05-26T22:47:16.008 INFO:teuthology.orchestra.run.smithi187.stdout: mon reweight min pgs per osd = 4 2024-05-26T22:47:16.008 INFO:teuthology.orchestra.run.smithi187.stdout: mon reweight min bytes per osd = 10 2024-05-26T22:47:16.008 INFO:teuthology.orchestra.run.smithi187.stdout: mgr/telemetry/nag = false 2024-05-26T22:47:16.008 INFO:teuthology.orchestra.run.smithi187.stdout: debug client = 20 2024-05-26T22:47:16.009 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-05-26T22:47:16.009 INFO:teuthology.orchestra.run.smithi187.stdout:[mon] 2024-05-26T22:47:16.009 INFO:teuthology.orchestra.run.smithi187.stdout: debug ms = 1 2024-05-26T22:47:16.009 INFO:teuthology.orchestra.run.smithi187.stdout: debug mon = 20 2024-05-26T22:47:16.009 INFO:teuthology.orchestra.run.smithi187.stdout: debug paxos = 20 2024-05-26T22:47:16.009 INFO:teuthology.orchestra.run.smithi187.stdout: debug auth = 20 2024-05-26T22:47:16.010 INFO:teuthology.orchestra.run.smithi187.stdout: mon data avail warn = 5 2024-05-26T22:47:16.010 INFO:teuthology.orchestra.run.smithi187.stdout: mon mgr mkfs grace = 240 2024-05-26T22:47:16.010 INFO:teuthology.orchestra.run.smithi187.stdout: mon reweight min pgs per osd = 4 2024-05-26T22:47:16.010 INFO:teuthology.orchestra.run.smithi187.stdout: mon osd reporter subtree level = osd 2024-05-26T22:47:16.011 INFO:teuthology.orchestra.run.smithi187.stdout: mon osd prime pg temp = true 2024-05-26T22:47:16.011 INFO:teuthology.orchestra.run.smithi187.stdout: mon reweight min bytes per osd = 10 2024-05-26T22:47:16.011 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-05-26T22:47:16.011 INFO:teuthology.orchestra.run.smithi187.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-26T22:47:16.011 INFO:teuthology.orchestra.run.smithi187.stdout: auth mon ticket ttl = 660 # 11m 2024-05-26T22:47:16.011 INFO:teuthology.orchestra.run.smithi187.stdout: auth service ticket ttl = 240 # 4m 2024-05-26T22:47:16.011 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-05-26T22:47:16.012 INFO:teuthology.orchestra.run.smithi187.stdout: # don't complain about insecure global_id in the test suite 2024-05-26T22:47:16.012 INFO:teuthology.orchestra.run.smithi187.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-26T22:47:16.012 INFO:teuthology.orchestra.run.smithi187.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-26T22:47:16.012 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-05-26T22:47:16.013 INFO:teuthology.orchestra.run.smithi187.stdout: # 1m isn't quite enough 2024-05-26T22:47:16.013 INFO:teuthology.orchestra.run.smithi187.stdout: mon_down_mkfs_grace = 2m 2024-05-26T22:47:16.013 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-05-26T22:47:16.013 INFO:teuthology.orchestra.run.smithi187.stdout: mon_warn_on_filestore_osds = false 2024-05-26T22:47:16.013 INFO:teuthology.orchestra.run.smithi187.stdout: mon down mkfs grace = 300 2024-05-26T22:47:16.013 INFO:teuthology.orchestra.run.smithi187.stdout: mon op complaint time = 120 2024-05-26T22:47:16.013 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-05-26T22:47:16.013 INFO:teuthology.orchestra.run.smithi187.stdout:[client] 2024-05-26T22:47:16.013 INFO:teuthology.orchestra.run.smithi187.stdout: rgw cache enabled = true 2024-05-26T22:47:16.013 INFO:teuthology.orchestra.run.smithi187.stdout: rgw enable ops log = true 2024-05-26T22:47:16.013 INFO:teuthology.orchestra.run.smithi187.stdout: rgw enable usage log = true 2024-05-26T22:47:16.014 INFO:teuthology.orchestra.run.smithi187.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-26T22:47:16.014 INFO:teuthology.orchestra.run.smithi187.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-26T22:47:16.014 INFO:teuthology.orchestra.run.smithi187.stdout: client mount timeout = 600 2024-05-26T22:47:16.014 INFO:teuthology.orchestra.run.smithi187.stdout: debug client = 20 2024-05-26T22:47:16.014 INFO:teuthology.orchestra.run.smithi187.stdout: debug ms = 1 2024-05-26T22:47:16.018 INFO:teuthology.orchestra.run.smithi187.stdout: pid file = "" 2024-05-26T22:47:16.018 INFO:teuthology.orchestra.run.smithi187.stdout: rados mon op timeout = 900 2024-05-26T22:47:16.018 INFO:teuthology.orchestra.run.smithi187.stdout: rados osd op timeout = 900 2024-05-26T22:47:16.018 INFO:teuthology.orchestra.run.smithi187.stdout:[mon.a] 2024-05-26T22:47:16.019 INFO:teuthology.orchestra.run.smithi187.stdout:[mon.b] 2024-05-26T22:47:16.019 INFO:teuthology.orchestra.run.smithi187.stdout:[mon.c] 2024-05-26T22:47:16.019 INFO:teuthology.orchestra.run.smithi187.stdout:[mds.a] 2024-05-26T22:47:16.019 INFO:teuthology.orchestra.run.smithi187.stdout:[mds.c] 2024-05-26T22:47:16.019 INFO:teuthology.orchestra.run.smithi187.stdout:[mds.b] 2024-05-26T22:47:16.019 INFO:teuthology.orchestra.run.smithi187.stdout:[mds.d] 2024-05-26T22:47:16.019 INFO:teuthology.orchestra.run.smithi187.stdout:[mds] 2024-05-26T22:47:16.020 INFO:teuthology.orchestra.run.smithi187.stdout: debug mds = 20 2024-05-26T22:47:16.020 INFO:teuthology.orchestra.run.smithi187.stdout: debug mds balancer = 20 2024-05-26T22:47:16.020 INFO:teuthology.orchestra.run.smithi187.stdout: debug ms = 1 2024-05-26T22:47:16.020 INFO:teuthology.orchestra.run.smithi187.stdout: mds debug frag = True 2024-05-26T22:47:16.020 INFO:teuthology.orchestra.run.smithi187.stdout: mds debug scatterstat = True 2024-05-26T22:47:16.020 INFO:teuthology.orchestra.run.smithi187.stdout: mds op complaint time = 180 2024-05-26T22:47:16.021 INFO:teuthology.orchestra.run.smithi187.stdout: mds verify scatter = True 2024-05-26T22:47:16.021 INFO:teuthology.orchestra.run.smithi187.stdout: osd op complaint time = 180 2024-05-26T22:47:16.021 INFO:teuthology.orchestra.run.smithi187.stdout: rados mon op timeout = 900 2024-05-26T22:47:16.021 INFO:teuthology.orchestra.run.smithi187.stdout: rados osd op timeout = 900 2024-05-26T22:47:16.025 INFO:tasks.ceph:Creating admin key on mon.a... 2024-05-26T22:47:16.025 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-05-26T22:47:16.151 INFO:tasks.ceph:Copying monmap to all nodes... 2024-05-26T22:47:16.151 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-26T22:47:16.151 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-05-26T22:47:16.197 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-26T22:47:16.197 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-05-26T22:47:16.296 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi131.front.sepia.ceph.com 2024-05-26T22:47:16.296 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-26T22:47:16.296 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/etc/ceph/ceph.keyring 2024-05-26T22:47:16.297 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-26T22:47:16.418 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-26T22:47:16.418 DEBUG:teuthology.orchestra.run.smithi131:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-05-26T22:47:16.517 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi187.front.sepia.ceph.com 2024-05-26T22:47:16.517 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-05-26T22:47:16.517 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd of=/etc/ceph/ceph.keyring 2024-05-26T22:47:16.518 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-26T22:47:16.649 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-05-26T22:47:16.649 DEBUG:teuthology.orchestra.run.smithi187:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-05-26T22:47:16.745 INFO:tasks.ceph:Setting up mon nodes... 2024-05-26T22:47:16.745 INFO:tasks.ceph:Setting up mgr nodes... 2024-05-26T22:47:16.745 DEBUG:teuthology.orchestra.run.smithi131:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-05-26T22:47:16.911 INFO:teuthology.orchestra.run.smithi131.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-05-26T22:47:16.921 DEBUG:teuthology.orchestra.run.smithi187:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-05-26T22:47:17.073 INFO:teuthology.orchestra.run.smithi187.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-05-26T22:47:17.082 INFO:tasks.ceph:Setting up mds nodes... 2024-05-26T22:47:17.083 DEBUG:teuthology.orchestra.run.smithi131:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-05-26T22:47:17.249 INFO:teuthology.orchestra.run.smithi131.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-05-26T22:47:17.258 DEBUG:teuthology.orchestra.run.smithi131:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-05-26T22:47:17.340 DEBUG:teuthology.orchestra.run.smithi131:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-05-26T22:47:17.494 INFO:teuthology.orchestra.run.smithi131.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-05-26T22:47:17.502 DEBUG:teuthology.orchestra.run.smithi131:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-05-26T22:47:17.576 DEBUG:teuthology.orchestra.run.smithi187:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-05-26T22:47:17.728 INFO:teuthology.orchestra.run.smithi187.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-05-26T22:47:17.736 DEBUG:teuthology.orchestra.run.smithi187:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-05-26T22:47:17.820 DEBUG:teuthology.orchestra.run.smithi187:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-05-26T22:47:17.968 INFO:teuthology.orchestra.run.smithi187.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-05-26T22:47:17.976 DEBUG:teuthology.orchestra.run.smithi187:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-05-26T22:47:18.051 INFO:tasks.ceph_client:Setting up client nodes... 2024-05-26T22:47:18.052 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-05-26T22:47:18.172 INFO:teuthology.orchestra.run.smithi131.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-05-26T22:47:18.210 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-05-26T22:47:18.321 INFO:teuthology.orchestra.run.smithi131.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-05-26T22:47:18.363 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-05-26T22:47:18.489 INFO:teuthology.orchestra.run.smithi187.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-05-26T22:47:18.530 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-05-26T22:47:18.647 INFO:teuthology.orchestra.run.smithi187.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-05-26T22:47:18.690 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-05-26T22:47:18.691 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi131.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi187.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-05-26T22:47:18.691 DEBUG:teuthology.orchestra.run.smithi131:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-05-26T22:47:18.780 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-26T22:47:18.780 INFO:tasks.ceph:role: osd.0 2024-05-26T22:47:18.780 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi131.front.sepia.ceph.com 2024-05-26T22:47:18.781 DEBUG:teuthology.orchestra.run.smithi131:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-05-26T22:47:18.871 INFO:teuthology.orchestra.run.smithi131.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-05-26T22:47:18.871 INFO:teuthology.orchestra.run.smithi131.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-26T22:47:18.872 INFO:teuthology.orchestra.run.smithi131.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-26T22:47:18.872 INFO:teuthology.orchestra.run.smithi131.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-26T22:47:18.872 INFO:teuthology.orchestra.run.smithi131.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-26T22:47:18.872 INFO:teuthology.orchestra.run.smithi131.stdout: = sunit=0 swidth=0 blks 2024-05-26T22:47:18.872 INFO:teuthology.orchestra.run.smithi131.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-26T22:47:18.872 INFO:teuthology.orchestra.run.smithi131.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-26T22:47:18.873 INFO:teuthology.orchestra.run.smithi131.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-26T22:47:18.873 INFO:teuthology.orchestra.run.smithi131.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-26T22:47:19.280 INFO:teuthology.orchestra.run.smithi131.stdout:Discarding blocks...Done. 2024-05-26T22:47:19.287 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi131.front.sepia.ceph.com -o noatime 2024-05-26T22:47:19.287 DEBUG:teuthology.orchestra.run.smithi131:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-05-26T22:47:19.451 DEBUG:teuthology.orchestra.run.smithi131:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-05-26T22:47:19.543 DEBUG:teuthology.orchestra.run.smithi131:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-05-26T22:47:19.615 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-26T22:47:19.615 INFO:tasks.ceph:role: osd.1 2024-05-26T22:47:19.615 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi131.front.sepia.ceph.com 2024-05-26T22:47:19.615 DEBUG:teuthology.orchestra.run.smithi131:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-05-26T22:47:19.690 INFO:teuthology.orchestra.run.smithi131.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-05-26T22:47:19.690 INFO:teuthology.orchestra.run.smithi131.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-26T22:47:19.690 INFO:teuthology.orchestra.run.smithi131.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-26T22:47:19.690 INFO:teuthology.orchestra.run.smithi131.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-26T22:47:19.690 INFO:teuthology.orchestra.run.smithi131.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-26T22:47:19.690 INFO:teuthology.orchestra.run.smithi131.stdout: = sunit=0 swidth=0 blks 2024-05-26T22:47:19.691 INFO:teuthology.orchestra.run.smithi131.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-26T22:47:19.691 INFO:teuthology.orchestra.run.smithi131.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-26T22:47:19.691 INFO:teuthology.orchestra.run.smithi131.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-26T22:47:19.691 INFO:teuthology.orchestra.run.smithi131.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-26T22:47:20.043 INFO:teuthology.orchestra.run.smithi131.stdout:Discarding blocks...Done. 2024-05-26T22:47:20.049 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi131.front.sepia.ceph.com -o noatime 2024-05-26T22:47:20.050 DEBUG:teuthology.orchestra.run.smithi131:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-05-26T22:47:20.157 DEBUG:teuthology.orchestra.run.smithi131:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-05-26T22:47:20.226 DEBUG:teuthology.orchestra.run.smithi131:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-05-26T22:47:20.288 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-26T22:47:20.288 INFO:tasks.ceph:role: osd.2 2024-05-26T22:47:20.288 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi131.front.sepia.ceph.com 2024-05-26T22:47:20.289 DEBUG:teuthology.orchestra.run.smithi131:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-05-26T22:47:20.364 INFO:teuthology.orchestra.run.smithi131.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-05-26T22:47:20.364 INFO:teuthology.orchestra.run.smithi131.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-26T22:47:20.365 INFO:teuthology.orchestra.run.smithi131.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-26T22:47:20.365 INFO:teuthology.orchestra.run.smithi131.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-26T22:47:20.365 INFO:teuthology.orchestra.run.smithi131.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-26T22:47:20.365 INFO:teuthology.orchestra.run.smithi131.stdout: = sunit=0 swidth=0 blks 2024-05-26T22:47:20.365 INFO:teuthology.orchestra.run.smithi131.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-26T22:47:20.365 INFO:teuthology.orchestra.run.smithi131.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-26T22:47:20.365 INFO:teuthology.orchestra.run.smithi131.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-26T22:47:20.365 INFO:teuthology.orchestra.run.smithi131.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-26T22:47:20.751 INFO:teuthology.orchestra.run.smithi131.stdout:Discarding blocks...Done. 2024-05-26T22:47:20.758 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi131.front.sepia.ceph.com -o noatime 2024-05-26T22:47:20.758 DEBUG:teuthology.orchestra.run.smithi131:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-05-26T22:47:20.869 DEBUG:teuthology.orchestra.run.smithi131:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-05-26T22:47:20.940 DEBUG:teuthology.orchestra.run.smithi131:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-05-26T22:47:21.008 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-26T22:47:21.008 INFO:tasks.ceph:role: osd.3 2024-05-26T22:47:21.008 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi131.front.sepia.ceph.com 2024-05-26T22:47:21.008 DEBUG:teuthology.orchestra.run.smithi131:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-05-26T22:47:21.084 INFO:teuthology.orchestra.run.smithi131.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-05-26T22:47:21.085 INFO:teuthology.orchestra.run.smithi131.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-26T22:47:21.085 INFO:teuthology.orchestra.run.smithi131.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-26T22:47:21.085 INFO:teuthology.orchestra.run.smithi131.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-26T22:47:21.085 INFO:teuthology.orchestra.run.smithi131.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-26T22:47:21.085 INFO:teuthology.orchestra.run.smithi131.stdout: = sunit=0 swidth=0 blks 2024-05-26T22:47:21.085 INFO:teuthology.orchestra.run.smithi131.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-26T22:47:21.085 INFO:teuthology.orchestra.run.smithi131.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-26T22:47:21.085 INFO:teuthology.orchestra.run.smithi131.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-26T22:47:21.086 INFO:teuthology.orchestra.run.smithi131.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-26T22:47:21.496 INFO:teuthology.orchestra.run.smithi131.stdout:Discarding blocks...Done. 2024-05-26T22:47:21.503 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi131.front.sepia.ceph.com -o noatime 2024-05-26T22:47:21.503 DEBUG:teuthology.orchestra.run.smithi131:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-05-26T22:47:21.672 DEBUG:teuthology.orchestra.run.smithi131:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-05-26T22:47:21.760 DEBUG:teuthology.orchestra.run.smithi131:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-26T22:47:21.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:21.909+0000 7fd5ca0f0640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-05-26T22:47:21.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:21.910+0000 7fd5ca0f0640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-05-26T22:47:21.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:21.910+0000 7fd5ca0f0640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-26T22:47:21.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:21.910+0000 7fd5ca0f0640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-26T22:47:21.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:21.910+0000 7fd5ca0f0640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-26T22:47:21.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:21.914+0000 7fd5ca0f0640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-05-26T22:47:25.131 DEBUG:teuthology.orchestra.run.smithi131:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-05-26T22:47:25.215 DEBUG:teuthology.orchestra.run.smithi131:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-26T22:47:25.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:25.335+0000 7fb09e22e640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-05-26T22:47:25.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:25.336+0000 7fb09e22e640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-05-26T22:47:25.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:25.336+0000 7fb09e22e640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-26T22:47:25.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:25.336+0000 7fb09e22e640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-26T22:47:25.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:25.336+0000 7fb09e22e640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-26T22:47:25.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:25.338+0000 7fb09e22e640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-05-26T22:47:28.547 DEBUG:teuthology.orchestra.run.smithi131:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-05-26T22:47:28.631 DEBUG:teuthology.orchestra.run.smithi131:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-26T22:47:28.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:28.756+0000 7f38136e8640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-05-26T22:47:28.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:28.757+0000 7f38136e8640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-05-26T22:47:28.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:28.757+0000 7f38136e8640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-26T22:47:28.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:28.757+0000 7f38136e8640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-26T22:47:28.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:28.757+0000 7f38136e8640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-26T22:47:28.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:28.759+0000 7f38136e8640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-05-26T22:47:31.977 DEBUG:teuthology.orchestra.run.smithi131:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-05-26T22:47:32.063 DEBUG:teuthology.orchestra.run.smithi131:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-26T22:47:32.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:32.195+0000 7f5384c1b640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-05-26T22:47:32.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:32.196+0000 7f5384c1b640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-05-26T22:47:32.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:32.196+0000 7f5384c1b640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-26T22:47:32.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:32.196+0000 7f5384c1b640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-26T22:47:32.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:32.196+0000 7f5384c1b640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-26T22:47:32.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:32.198+0000 7f5384c1b640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-05-26T22:47:35.156 DEBUG:teuthology.orchestra.run.smithi131:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-05-26T22:47:35.241 DEBUG:teuthology.orchestra.run.smithi187:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-05-26T22:47:35.331 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-26T22:47:35.332 INFO:tasks.ceph:role: osd.4 2024-05-26T22:47:35.332 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi187.front.sepia.ceph.com 2024-05-26T22:47:35.332 DEBUG:teuthology.orchestra.run.smithi187:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-05-26T22:47:35.422 INFO:teuthology.orchestra.run.smithi187.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-05-26T22:47:35.422 INFO:teuthology.orchestra.run.smithi187.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-26T22:47:35.422 INFO:teuthology.orchestra.run.smithi187.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-26T22:47:35.422 INFO:teuthology.orchestra.run.smithi187.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-26T22:47:35.422 INFO:teuthology.orchestra.run.smithi187.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-26T22:47:35.423 INFO:teuthology.orchestra.run.smithi187.stdout: = sunit=0 swidth=0 blks 2024-05-26T22:47:35.423 INFO:teuthology.orchestra.run.smithi187.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-26T22:47:35.423 INFO:teuthology.orchestra.run.smithi187.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-26T22:47:35.423 INFO:teuthology.orchestra.run.smithi187.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-26T22:47:35.423 INFO:teuthology.orchestra.run.smithi187.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-26T22:47:35.834 INFO:teuthology.orchestra.run.smithi187.stdout:Discarding blocks...Done. 2024-05-26T22:47:35.840 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi187.front.sepia.ceph.com -o noatime 2024-05-26T22:47:35.841 DEBUG:teuthology.orchestra.run.smithi187:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-05-26T22:47:36.004 DEBUG:teuthology.orchestra.run.smithi187:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-05-26T22:47:36.095 DEBUG:teuthology.orchestra.run.smithi187:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-05-26T22:47:36.177 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-26T22:47:36.178 INFO:tasks.ceph:role: osd.5 2024-05-26T22:47:36.178 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi187.front.sepia.ceph.com 2024-05-26T22:47:36.178 DEBUG:teuthology.orchestra.run.smithi187:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-05-26T22:47:36.255 INFO:teuthology.orchestra.run.smithi187.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-05-26T22:47:36.256 INFO:teuthology.orchestra.run.smithi187.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-26T22:47:36.256 INFO:teuthology.orchestra.run.smithi187.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-26T22:47:36.256 INFO:teuthology.orchestra.run.smithi187.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-26T22:47:36.256 INFO:teuthology.orchestra.run.smithi187.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-26T22:47:36.256 INFO:teuthology.orchestra.run.smithi187.stdout: = sunit=0 swidth=0 blks 2024-05-26T22:47:36.256 INFO:teuthology.orchestra.run.smithi187.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-26T22:47:36.257 INFO:teuthology.orchestra.run.smithi187.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-26T22:47:36.257 INFO:teuthology.orchestra.run.smithi187.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-26T22:47:36.257 INFO:teuthology.orchestra.run.smithi187.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-26T22:47:36.610 INFO:teuthology.orchestra.run.smithi187.stdout:Discarding blocks...Done. 2024-05-26T22:47:36.615 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi187.front.sepia.ceph.com -o noatime 2024-05-26T22:47:36.615 DEBUG:teuthology.orchestra.run.smithi187:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-05-26T22:47:36.732 DEBUG:teuthology.orchestra.run.smithi187:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-05-26T22:47:36.801 DEBUG:teuthology.orchestra.run.smithi187:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-05-26T22:47:36.864 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-26T22:47:36.864 INFO:tasks.ceph:role: osd.6 2024-05-26T22:47:36.864 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi187.front.sepia.ceph.com 2024-05-26T22:47:36.864 DEBUG:teuthology.orchestra.run.smithi187:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-05-26T22:47:36.934 INFO:teuthology.orchestra.run.smithi187.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-05-26T22:47:36.934 INFO:teuthology.orchestra.run.smithi187.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-26T22:47:36.934 INFO:teuthology.orchestra.run.smithi187.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-26T22:47:36.934 INFO:teuthology.orchestra.run.smithi187.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-26T22:47:36.935 INFO:teuthology.orchestra.run.smithi187.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-26T22:47:36.935 INFO:teuthology.orchestra.run.smithi187.stdout: = sunit=0 swidth=0 blks 2024-05-26T22:47:36.935 INFO:teuthology.orchestra.run.smithi187.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-26T22:47:36.935 INFO:teuthology.orchestra.run.smithi187.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-26T22:47:36.935 INFO:teuthology.orchestra.run.smithi187.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-26T22:47:36.935 INFO:teuthology.orchestra.run.smithi187.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-26T22:47:37.321 INFO:teuthology.orchestra.run.smithi187.stdout:Discarding blocks...Done. 2024-05-26T22:47:37.325 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi187.front.sepia.ceph.com -o noatime 2024-05-26T22:47:37.326 DEBUG:teuthology.orchestra.run.smithi187:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-05-26T22:47:37.432 DEBUG:teuthology.orchestra.run.smithi187:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-05-26T22:47:37.500 DEBUG:teuthology.orchestra.run.smithi187:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-05-26T22:47:37.566 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-26T22:47:37.566 INFO:tasks.ceph:role: osd.7 2024-05-26T22:47:37.566 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi187.front.sepia.ceph.com 2024-05-26T22:47:37.566 DEBUG:teuthology.orchestra.run.smithi187:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-05-26T22:47:37.638 INFO:teuthology.orchestra.run.smithi187.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-05-26T22:47:37.638 INFO:teuthology.orchestra.run.smithi187.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-26T22:47:37.639 INFO:teuthology.orchestra.run.smithi187.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-26T22:47:37.639 INFO:teuthology.orchestra.run.smithi187.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-26T22:47:37.639 INFO:teuthology.orchestra.run.smithi187.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-26T22:47:37.639 INFO:teuthology.orchestra.run.smithi187.stdout: = sunit=0 swidth=0 blks 2024-05-26T22:47:37.639 INFO:teuthology.orchestra.run.smithi187.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-26T22:47:37.639 INFO:teuthology.orchestra.run.smithi187.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-26T22:47:37.639 INFO:teuthology.orchestra.run.smithi187.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-26T22:47:37.639 INFO:teuthology.orchestra.run.smithi187.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-26T22:47:38.048 INFO:teuthology.orchestra.run.smithi187.stdout:Discarding blocks...Done. 2024-05-26T22:47:38.054 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi187.front.sepia.ceph.com -o noatime 2024-05-26T22:47:38.054 DEBUG:teuthology.orchestra.run.smithi187:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-05-26T22:47:38.198 DEBUG:teuthology.orchestra.run.smithi187:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-05-26T22:47:38.283 DEBUG:teuthology.orchestra.run.smithi187:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-26T22:47:38.441 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:38.442+0000 7f0f54b40640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-05-26T22:47:38.442 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:38.443+0000 7f0f54b40640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-05-26T22:47:38.443 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:38.443+0000 7f0f54b40640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-26T22:47:38.443 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:38.443+0000 7f0f54b40640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-26T22:47:38.443 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:38.443+0000 7f0f54b40640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-26T22:47:38.446 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:38.447+0000 7f0f54b40640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-05-26T22:47:41.699 DEBUG:teuthology.orchestra.run.smithi187:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-05-26T22:47:41.783 DEBUG:teuthology.orchestra.run.smithi187:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-26T22:47:41.912 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:41.912+0000 7fe0d3c1f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-05-26T22:47:41.912 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:41.912+0000 7fe0d3c1f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-05-26T22:47:41.912 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:41.912+0000 7fe0d3c1f640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-26T22:47:41.913 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:41.912+0000 7fe0d3c1f640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-26T22:47:41.913 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:41.912+0000 7fe0d3c1f640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-26T22:47:41.915 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:41.915+0000 7fe0d3c1f640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-05-26T22:47:43.278 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-26T22:47:43.287 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-26T22:47:45.137 DEBUG:teuthology.orchestra.run.smithi187:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-05-26T22:47:45.225 DEBUG:teuthology.orchestra.run.smithi187:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-26T22:47:45.359 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:45.359+0000 7fb74bc10640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-05-26T22:47:45.359 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:45.359+0000 7fb74bc10640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-05-26T22:47:45.359 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:45.359+0000 7fb74bc10640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-05-26T22:47:45.359 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:45.359+0000 7fb74bc10640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-05-26T22:47:45.359 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:45.359+0000 7fb74bc10640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-05-26T22:47:45.361 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:45.362+0000 7fb74bc10640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-05-26T22:47:48.570 DEBUG:teuthology.orchestra.run.smithi187:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-05-26T22:47:48.655 DEBUG:teuthology.orchestra.run.smithi187:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-26T22:47:48.782 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:48.782+0000 7f3f06c1b640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-05-26T22:47:48.783 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:48.783+0000 7f3f06c1b640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-05-26T22:47:48.783 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:48.783+0000 7f3f06c1b640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-05-26T22:47:48.783 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:48.783+0000 7f3f06c1b640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-05-26T22:47:48.783 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:48.783+0000 7f3f06c1b640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-05-26T22:47:48.785 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:48.786+0000 7f3f06c1b640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-05-26T22:47:51.773 DEBUG:teuthology.orchestra.run.smithi187:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-05-26T22:47:51.861 INFO:tasks.ceph:Reading keys from all nodes... 2024-05-26T22:47:51.861 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-26T22:47:51.861 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-05-26T22:47:51.949 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-26T22:47:51.949 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-05-26T22:47:52.033 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-26T22:47:52.033 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-05-26T22:47:52.115 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-26T22:47:52.115 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-05-26T22:47:52.197 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-26T22:47:52.198 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-05-26T22:47:52.280 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-26T22:47:52.281 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-05-26T22:47:52.365 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-26T22:47:52.365 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-05-26T22:47:52.445 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-05-26T22:47:52.445 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-05-26T22:47:52.535 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-05-26T22:47:52.535 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-05-26T22:47:52.618 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-05-26T22:47:52.618 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-05-26T22:47:52.703 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-05-26T22:47:52.703 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-05-26T22:47:52.789 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-05-26T22:47:52.790 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-05-26T22:47:52.871 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-05-26T22:47:52.871 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-05-26T22:47:52.946 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-05-26T22:47:52.947 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-05-26T22:47:53.067 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-26T22:47:53.067 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-05-26T22:47:53.125 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-26T22:47:53.125 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-05-26T22:47:53.182 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-05-26T22:47:53.182 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-05-26T22:47:53.237 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-05-26T22:47:53.237 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-05-26T22:47:53.288 INFO:tasks.ceph:Adding keys to all mons... 2024-05-26T22:47:53.288 DEBUG:teuthology.orchestra.run.smithi131:> sudo tee -a /etc/ceph/ceph.keyring 2024-05-26T22:47:53.296 DEBUG:teuthology.orchestra.run.smithi187:> sudo tee -a /etc/ceph/ceph.keyring 2024-05-26T22:47:53.374 INFO:teuthology.orchestra.run.smithi131.stdout:[mgr.y] 2024-05-26T22:47:53.374 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQD0u1NmX1RhNhAAo5mEOGzUyrdwFlB/XOXKnw== 2024-05-26T22:47:53.375 INFO:teuthology.orchestra.run.smithi131.stdout:[mds.a] 2024-05-26T22:47:53.375 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQD1u1NmTQf1DhAAdvEt7wP0IgKhNHOqjVAllA== 2024-05-26T22:47:53.375 INFO:teuthology.orchestra.run.smithi131.stdout:[mds.c] 2024-05-26T22:47:53.375 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQD1u1NmFjCHHRAAqix3Ujfsq2tnSU5+d/pSYA== 2024-05-26T22:47:53.375 INFO:teuthology.orchestra.run.smithi131.stdout:[osd.0] 2024-05-26T22:47:53.375 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQD5u1Nm+w1LNhAAsJLQkiKSh3KkMymlrZkcrw== 2024-05-26T22:47:53.375 INFO:teuthology.orchestra.run.smithi131.stdout:[osd.1] 2024-05-26T22:47:53.375 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQD9u1Nm4c8RFBAAcn9HQ+izML9rujJ/2/Q/BQ== 2024-05-26T22:47:53.375 INFO:teuthology.orchestra.run.smithi131.stdout:[osd.2] 2024-05-26T22:47:53.375 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQAAvFNmR5wmLRAAYEBAXPBZFq67iARZnwF3iw== 2024-05-26T22:47:53.375 INFO:teuthology.orchestra.run.smithi131.stdout:[osd.3] 2024-05-26T22:47:53.376 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQAEvFNmJ4G5CxAAnTbP8DinhmuvGNo1VBVzjQ== 2024-05-26T22:47:53.376 INFO:teuthology.orchestra.run.smithi131.stdout:[mgr.x] 2024-05-26T22:47:53.376 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQD1u1NmaWhyBBAAlU7Lq6HMjkCPu0kztJIAXQ== 2024-05-26T22:47:53.376 INFO:teuthology.orchestra.run.smithi131.stdout:[mds.b] 2024-05-26T22:47:53.376 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQD1u1NmHNZ8KxAAPgQ/ojbl+EDfPhqrkg7LPQ== 2024-05-26T22:47:53.376 INFO:teuthology.orchestra.run.smithi131.stdout:[mds.d] 2024-05-26T22:47:53.376 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQD1u1NmM4LPORAAhtQtr+Vfpf7vdVwjSW6xtA== 2024-05-26T22:47:53.376 INFO:teuthology.orchestra.run.smithi131.stdout:[osd.4] 2024-05-26T22:47:53.376 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQAKvFNmcw5rGhAAvnivP6kt3vcN9UPm/svKqw== 2024-05-26T22:47:53.376 INFO:teuthology.orchestra.run.smithi131.stdout:[osd.5] 2024-05-26T22:47:53.376 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQANvFNmTqlzNhAAGfrSvNNHYKGOBzRXo0TEdw== 2024-05-26T22:47:53.376 INFO:teuthology.orchestra.run.smithi131.stdout:[osd.6] 2024-05-26T22:47:53.377 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQARvFNmpzd6FRAAgi2RYlQPvfHNe0iP4bJrwg== 2024-05-26T22:47:53.377 INFO:teuthology.orchestra.run.smithi131.stdout:[osd.7] 2024-05-26T22:47:53.377 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQAUvFNmAkC8LhAAsmkKF4eloxdpCi/mrDtzqA== 2024-05-26T22:47:53.377 INFO:teuthology.orchestra.run.smithi131.stdout:[client.0] 2024-05-26T22:47:53.377 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQD2u1NmSGVWChAAH/wq3iaUs6ZELRhLKhmURQ== 2024-05-26T22:47:53.377 INFO:teuthology.orchestra.run.smithi131.stdout:[client.2] 2024-05-26T22:47:53.377 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQD2u1NmGEI9ExAAvJn1LRwmALkD7zh/AP07vQ== 2024-05-26T22:47:53.377 INFO:teuthology.orchestra.run.smithi131.stdout:[client.1] 2024-05-26T22:47:53.377 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQD2u1NmnA9CHRAA2XMmZQxysKweHNykiQUEeA== 2024-05-26T22:47:53.377 INFO:teuthology.orchestra.run.smithi131.stdout:[client.3] 2024-05-26T22:47:53.377 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQD2u1NmIHOtJhAAJCUm0kGqWu4rWVLsDkW6Eg== 2024-05-26T22:47:53.378 INFO:teuthology.orchestra.run.smithi187.stdout:[mgr.y] 2024-05-26T22:47:53.379 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQD0u1NmX1RhNhAAo5mEOGzUyrdwFlB/XOXKnw== 2024-05-26T22:47:53.379 INFO:teuthology.orchestra.run.smithi187.stdout:[mds.a] 2024-05-26T22:47:53.379 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQD1u1NmTQf1DhAAdvEt7wP0IgKhNHOqjVAllA== 2024-05-26T22:47:53.379 INFO:teuthology.orchestra.run.smithi187.stdout:[mds.c] 2024-05-26T22:47:53.379 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQD1u1NmFjCHHRAAqix3Ujfsq2tnSU5+d/pSYA== 2024-05-26T22:47:53.379 INFO:teuthology.orchestra.run.smithi187.stdout:[osd.0] 2024-05-26T22:47:53.379 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQD5u1Nm+w1LNhAAsJLQkiKSh3KkMymlrZkcrw== 2024-05-26T22:47:53.379 INFO:teuthology.orchestra.run.smithi187.stdout:[osd.1] 2024-05-26T22:47:53.379 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQD9u1Nm4c8RFBAAcn9HQ+izML9rujJ/2/Q/BQ== 2024-05-26T22:47:53.379 INFO:teuthology.orchestra.run.smithi187.stdout:[osd.2] 2024-05-26T22:47:53.379 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQAAvFNmR5wmLRAAYEBAXPBZFq67iARZnwF3iw== 2024-05-26T22:47:53.379 INFO:teuthology.orchestra.run.smithi187.stdout:[osd.3] 2024-05-26T22:47:53.379 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQAEvFNmJ4G5CxAAnTbP8DinhmuvGNo1VBVzjQ== 2024-05-26T22:47:53.380 INFO:teuthology.orchestra.run.smithi187.stdout:[mgr.x] 2024-05-26T22:47:53.380 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQD1u1NmaWhyBBAAlU7Lq6HMjkCPu0kztJIAXQ== 2024-05-26T22:47:53.380 INFO:teuthology.orchestra.run.smithi187.stdout:[mds.b] 2024-05-26T22:47:53.380 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQD1u1NmHNZ8KxAAPgQ/ojbl+EDfPhqrkg7LPQ== 2024-05-26T22:47:53.380 INFO:teuthology.orchestra.run.smithi187.stdout:[mds.d] 2024-05-26T22:47:53.380 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQD1u1NmM4LPORAAhtQtr+Vfpf7vdVwjSW6xtA== 2024-05-26T22:47:53.380 INFO:teuthology.orchestra.run.smithi187.stdout:[osd.4] 2024-05-26T22:47:53.380 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQAKvFNmcw5rGhAAvnivP6kt3vcN9UPm/svKqw== 2024-05-26T22:47:53.380 INFO:teuthology.orchestra.run.smithi187.stdout:[osd.5] 2024-05-26T22:47:53.380 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQANvFNmTqlzNhAAGfrSvNNHYKGOBzRXo0TEdw== 2024-05-26T22:47:53.380 INFO:teuthology.orchestra.run.smithi187.stdout:[osd.6] 2024-05-26T22:47:53.380 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQARvFNmpzd6FRAAgi2RYlQPvfHNe0iP4bJrwg== 2024-05-26T22:47:53.380 INFO:teuthology.orchestra.run.smithi187.stdout:[osd.7] 2024-05-26T22:47:53.380 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQAUvFNmAkC8LhAAsmkKF4eloxdpCi/mrDtzqA== 2024-05-26T22:47:53.381 INFO:teuthology.orchestra.run.smithi187.stdout:[client.0] 2024-05-26T22:47:53.381 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQD2u1NmSGVWChAAH/wq3iaUs6ZELRhLKhmURQ== 2024-05-26T22:47:53.381 INFO:teuthology.orchestra.run.smithi187.stdout:[client.2] 2024-05-26T22:47:53.381 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQD2u1NmGEI9ExAAvJn1LRwmALkD7zh/AP07vQ== 2024-05-26T22:47:53.381 INFO:teuthology.orchestra.run.smithi187.stdout:[client.1] 2024-05-26T22:47:53.381 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQD2u1NmnA9CHRAA2XMmZQxysKweHNykiQUEeA== 2024-05-26T22:47:53.381 INFO:teuthology.orchestra.run.smithi187.stdout:[client.3] 2024-05-26T22:47:53.381 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQD2u1NmIHOtJhAAJCUm0kGqWu4rWVLsDkW6Eg== 2024-05-26T22:47:53.382 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-26T22:47:53.426 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-26T22:47:53.556 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-26T22:47:53.563 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-26T22:47:53.697 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-26T22:47:53.702 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-26T22:47:53.836 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-26T22:47:53.841 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-26T22:47:53.973 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-26T22:47:53.980 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-26T22:47:54.115 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-26T22:47:54.123 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-26T22:47:54.252 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-26T22:47:54.259 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-26T22:47:54.389 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-26T22:47:54.396 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-26T22:47:54.532 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-26T22:47:54.539 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-26T22:47:54.672 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-26T22:47:54.679 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-26T22:47:54.810 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-26T22:47:54.816 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-26T22:47:54.946 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-26T22:47:54.953 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-26T22:47:55.086 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-26T22:47:55.093 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-26T22:47:55.225 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-26T22:47:55.229 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-26T22:47:55.362 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-26T22:47:55.367 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-26T22:47:55.503 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-26T22:47:55.509 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-26T22:47:55.642 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-26T22:47:55.647 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-26T22:47:55.781 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-26T22:47:55.786 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-26T22:47:55.924 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-05-26T22:47:55.924 DEBUG:teuthology.orchestra.run.smithi131:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-05-26T22:47:56.007 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-26T22:47:56.148 DEBUG:teuthology.orchestra.run.smithi131:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-05-26T22:47:56.215 DEBUG:teuthology.orchestra.run.smithi187:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-05-26T22:47:56.303 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-26T22:47:56.447 DEBUG:teuthology.orchestra.run.smithi187:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-05-26T22:47:56.519 DEBUG:teuthology.orchestra.run.smithi187:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-05-26T22:47:56.600 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-26T22:47:56.745 DEBUG:teuthology.orchestra.run.smithi187:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-05-26T22:47:56.819 DEBUG:teuthology.orchestra.run.smithi131:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-05-26T22:47:56.827 DEBUG:teuthology.orchestra.run.smithi187:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-05-26T22:47:56.876 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-05-26T22:47:56.877 INFO:tasks.ceph.mon.a:Restarting daemon 2024-05-26T22:47:56.877 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-05-26T22:47:56.882 INFO:tasks.ceph.mon.a:Started 2024-05-26T22:47:56.882 INFO:tasks.ceph.mon.b:Restarting daemon 2024-05-26T22:47:56.882 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-05-26T22:47:56.887 INFO:tasks.ceph.mon.b:Started 2024-05-26T22:47:56.887 INFO:tasks.ceph.mon.c:Restarting daemon 2024-05-26T22:47:56.887 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-05-26T22:47:56.892 INFO:tasks.ceph.mon.c:Started 2024-05-26T22:47:56.892 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-05-26T22:47:56.892 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-05-26T22:47:56.892 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-05-26T22:47:56.897 INFO:tasks.ceph.mgr.y:Started 2024-05-26T22:47:56.897 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-05-26T22:47:56.897 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-05-26T22:47:56.902 INFO:tasks.ceph.mgr.x:Started 2024-05-26T22:47:56.903 INFO:tasks.ceph:Setting crush tunables to default 2024-05-26T22:47:56.903 DEBUG:teuthology.orchestra.run.smithi131:> sudo ceph --cluster ceph osd crush tunables default 2024-05-26T22:47:57.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:57.074+0000 7ff8f1800640 1 Processor -- start 2024-05-26T22:47:57.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:57.074+0000 7ff8f1800640 1 -- start start 2024-05-26T22:47:57.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:57.075+0000 7ff8f1800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff8ec0d0460 0x7ff8ec06d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:47:57.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:57.075+0000 7ff8f1800640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8ec160180 con 0x7ff8ec0b5450 2024-05-26T22:47:57.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:57.075+0000 7ff8f1800640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8ec160460 con 0x7ff8ec06f420 2024-05-26T22:47:57.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:57.075+0000 7ff8f1800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff8ec160d90 con 0x7ff8ec0d0460 2024-05-26T22:47:57.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:57.076+0000 7ff8eaa00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7ff8ec0b5450 0x7ff8ec0b43c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:33088/0 (socket says 172.21.15.131:33088) 2024-05-26T22:47:57.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:57.076+0000 7ff8eaa00640 1 -- 172.21.15.131:0/1579932886 learned_addr learned my addr 172.21.15.131:0/1579932886 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:47:57.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:57.076+0000 7ff8eb400640 1 --2- 172.21.15.131:0/1579932886 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff8ec0d0460 0x7ff8ec06d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:47:57.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:57.076+0000 7ff8eb400640 1 -- 172.21.15.131:0/1579932886 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff8ec0d0460 msgr2=0x7ff8ec06d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-05-26T22:47:57.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:57.076+0000 7ff8eb400640 1 -- 172.21.15.131:0/1579932886 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff8ec0d0460 msgr2=0x7ff8ec06d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-26T22:47:57.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:57.076+0000 7ff8eb400640 1 --2- 172.21.15.131:0/1579932886 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff8ec0d0460 0x7ff8ec06d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-26T22:47:57.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:57.076+0000 7ff8eb400640 1 --2- 172.21.15.131:0/1579932886 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff8ec0d0460 0x7ff8ec06d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-26T22:47:57.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:57.277+0000 7ff8eb400640 1 --2- 172.21.15.131:0/1579932886 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff8ec0d0460 0x7ff8ec06d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:47:57.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:57.278+0000 7ff8eb400640 1 -- 172.21.15.131:0/1579932886 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff8ec0d0460 msgr2=0x7ff8ec06d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-05-26T22:47:57.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:57.278+0000 7ff8eb400640 1 -- 172.21.15.131:0/1579932886 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff8ec0d0460 msgr2=0x7ff8ec06d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-26T22:47:57.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:57.278+0000 7ff8eb400640 1 --2- 172.21.15.131:0/1579932886 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff8ec0d0460 0x7ff8ec06d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-26T22:47:57.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:57.278+0000 7ff8eb400640 1 --2- 172.21.15.131:0/1579932886 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff8ec0d0460 0x7ff8ec06d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-26T22:47:57.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:57.325+0000 7ff8ea000640 1 -- 172.21.15.131:0/1579932886 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2021684697 0 0) 0x7ff8ec160460 con 0x7ff8ec06f420 2024-05-26T22:47:57.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:57.325+0000 7ff8ea000640 1 -- 172.21.15.131:0/1579932886 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff8d0003520 con 0x7ff8ec06f420 2024-05-26T22:47:57.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:57.326+0000 7ff8ea000640 1 -- 172.21.15.131:0/1579932886 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2770020599 0 0) 0x7ff8ec160180 con 0x7ff8ec0b5450 2024-05-26T22:47:57.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:57.326+0000 7ff8ea000640 1 -- 172.21.15.131:0/1579932886 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff8ec160460 con 0x7ff8ec0b5450 2024-05-26T22:47:57.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:57.326+0000 7ff8ea000640 1 -- 172.21.15.131:0/1579932886 <== mon.1 v1:172.21.15.187:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1693662547 0 0) 0x7ff8d0003520 con 0x7ff8ec06f420 2024-05-26T22:47:57.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:57.326+0000 7ff8ea000640 1 -- 172.21.15.131:0/1579932886 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff8ec0d0460 msgr2=0x7ff8ec06d600 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-26T22:47:57.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:57.326+0000 7ff8ea000640 1 --2- 172.21.15.131:0/1579932886 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff8ec0d0460 0x7ff8ec06d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:47:57.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:57.326+0000 7ff8ea000640 1 -- 172.21.15.131:0/1579932886 >> v1:172.21.15.131:6789/0 conn(0x7ff8ec0b5450 legacy=0x7ff8ec0b43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:47:57.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:57.326+0000 7ff8ea000640 1 -- 172.21.15.131:0/1579932886 --> v1:172.21.15.187:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8ec160740 con 0x7ff8ec06f420 2024-05-26T22:47:57.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:57.327+0000 7ff8ea000640 1 -- 172.21.15.131:0/1579932886 <== mon.1 v1:172.21.15.187:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7ff8d40015f0 con 0x7ff8ec06f420 2024-05-26T22:47:57.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:57.327+0000 7ff8ea000640 1 -- 172.21.15.131:0/1579932886 >> v1:172.21.15.187:6789/0 conn(0x7ff8ec06f420 legacy=0x7ff8ec0cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:47:57.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:57.327+0000 7ff8ea000640 1 --2- 172.21.15.131:0/1579932886 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff8d0003b40 0x7ff8d0003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:47:57.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:57.327+0000 7ff8ea000640 1 --2- 172.21.15.131:0/1579932886 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff8d0004800 0x7ff8d0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:47:57.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:57.327+0000 7ff8ea000640 1 --2- 172.21.15.131:0/1579932886 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff8ec0d0460 0x7ff8d0005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:47:57.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:57.327+0000 7ff8ea000640 1 -- 172.21.15.131:0/1579932886 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff8ec160d90 con 0x7ff8d0004800 2024-05-26T22:47:57.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:57.327+0000 7ff8ea000640 1 -- 172.21.15.131:0/1579932886 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff8d000bb50 con 0x7ff8d0003b40 2024-05-26T22:47:57.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:57.327+0000 7ff8ea000640 1 -- 172.21.15.131:0/1579932886 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff8d000bd50 con 0x7ff8ec0d0460 2024-05-26T22:47:57.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:57.328+0000 7ff8e2000640 1 --2- 172.21.15.131:0/1579932886 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff8d0004800 0x7ff8d0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:47:57.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:57.328+0000 7ff8eaa00640 1 --2- 172.21.15.131:0/1579932886 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff8d0003b40 0x7ff8d0003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:47:57.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:57.329+0000 7ff8e2000640 1 -- 172.21.15.131:0/1579932886 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff8ec0d0460 msgr2=0x7ff8d0005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:47:57.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:57.329+0000 7ff8e2000640 1 --2- 172.21.15.131:0/1579932886 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff8ec0d0460 0x7ff8d0005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:47:57.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:57.329+0000 7ff8e2000640 1 -- 172.21.15.131:0/1579932886 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff8d0003b40 msgr2=0x7ff8d0003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:47:57.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:57.329+0000 7ff8e2000640 1 --2- 172.21.15.131:0/1579932886 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff8d0003b40 0x7ff8d0003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:47:57.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:57.329+0000 7ff8e2000640 1 -- 172.21.15.131:0/1579932886 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8d0003520 con 0x7ff8d0004800 2024-05-26T22:47:57.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:57.329+0000 7ff8eaa00640 1 --2- 172.21.15.131:0/1579932886 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff8d0003b40 0x7ff8d0003f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-26T22:47:57.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:57.332+0000 7ff8e2000640 1 --2- 172.21.15.131:0/1579932886 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff8d0004800 0x7ff8d0004c80 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff8d4007e50 tx=0x7ff8d0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cf4253c4da32587 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:47:57.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:57.334+0000 7ff8ea000640 1 -- 172.21.15.131:0/1579932886 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8d4036070 con 0x7ff8d0004800 2024-05-26T22:47:57.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:57.334+0000 7ff8ea000640 1 -- 172.21.15.131:0/1579932886 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8d4031620 con 0x7ff8d0004800 2024-05-26T22:47:57.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:57.334+0000 7ff8ea000640 1 -- 172.21.15.131:0/1579932886 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8d4007a70 con 0x7ff8d0004800 2024-05-26T22:47:57.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:57.336+0000 7ff8f1800640 1 -- 172.21.15.131:0/1579932886 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff8d0004800 msgr2=0x7ff8d0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:47:57.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:57.336+0000 7ff8f1800640 1 --2- 172.21.15.131:0/1579932886 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff8d0004800 0x7ff8d0004c80 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff8d4007e50 tx=0x7ff8d0005830 comp rx=0 tx=0).stop 2024-05-26T22:47:57.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:57.336+0000 7ff8eb400640 1 -- 172.21.15.131:0/1579932886 reap_dead start 2024-05-26T22:47:57.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:57.336+0000 7ff8f1800640 1 -- 172.21.15.131:0/1579932886 shutdown_connections 2024-05-26T22:47:57.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:57.336+0000 7ff8f1800640 1 -- 172.21.15.131:0/1579932886 >> 172.21.15.131:0/1579932886 conn(0x7ff8ec06bc90 msgr2=0x7ff8ec0b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:47:57.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:57.336+0000 7ff8f1800640 1 -- 172.21.15.131:0/1579932886 shutdown_connections 2024-05-26T22:47:57.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:57.336+0000 7ff8f1800640 1 -- 172.21.15.131:0/1579932886 wait complete. 2024-05-26T22:47:57.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:57.339+0000 7ff8f1800640 1 Processor -- start 2024-05-26T22:47:57.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:57.339+0000 7ff8f1800640 1 -- start start 2024-05-26T22:47:57.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:57.341+0000 7ff8f1800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff8d0004800 0x7ff8ec1588a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:47:57.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:57.341+0000 7ff8f1800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff8ec156400 0x7ff8ec15efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:47:57.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:57.341+0000 7ff8f1800640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff8ec15a220 0x7ff8ec15a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:47:57.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:57.341+0000 7ff8f1800640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff8ec15fde0 con 0x7ff8d0004800 2024-05-26T22:47:57.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:57.341+0000 7ff8f1800640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff8ec0b5830 con 0x7ff8ec15a220 2024-05-26T22:47:57.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:57.341+0000 7ff8f1800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff8ec15ff60 con 0x7ff8ec156400 2024-05-26T22:47:57.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:57.341+0000 7ff8eaa00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff8d0004800 0x7ff8ec1588a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:47:57.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:57.341+0000 7ff8eaa00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff8d0004800 0x7ff8ec1588a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33546/0 (socket says 172.21.15.131:33546) 2024-05-26T22:47:57.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:57.341+0000 7ff8eaa00640 1 -- 172.21.15.131:0/2326415097 learned_addr learned my addr 172.21.15.131:0/2326415097 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:47:57.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:57.342+0000 7ff8eb400640 1 --2- 172.21.15.131:0/2326415097 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff8ec15a220 0x7ff8ec15a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:47:57.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:57.342+0000 7ff8e2000640 1 --2- 172.21.15.131:0/2326415097 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff8ec156400 0x7ff8ec15efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:47:57.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:57.342+0000 7ff8eaa00640 1 -- 172.21.15.131:0/2326415097 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff8ec156400 msgr2=0x7ff8ec15efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:47:57.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:57.342+0000 7ff8eaa00640 1 --2- 172.21.15.131:0/2326415097 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff8ec156400 0x7ff8ec15efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:47:57.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:57.342+0000 7ff8eaa00640 1 -- 172.21.15.131:0/2326415097 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff8ec15a220 msgr2=0x7ff8ec15a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:47:57.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:57.342+0000 7ff8eaa00640 1 --2- 172.21.15.131:0/2326415097 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff8ec15a220 0x7ff8ec15a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:47:57.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:57.342+0000 7ff8eaa00640 1 -- 172.21.15.131:0/2326415097 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8ec156900 con 0x7ff8d0004800 2024-05-26T22:47:57.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:57.342+0000 7ff8eaa00640 1 --2- 172.21.15.131:0/2326415097 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff8d0004800 0x7ff8ec1588a0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc007bd0 tx=0x7ff8dc0018b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:47:57.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:57.343+0000 7ff8e3400640 1 -- 172.21.15.131:0/2326415097 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8dc012590 con 0x7ff8d0004800 2024-05-26T22:47:57.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:57.343+0000 7ff8f1800640 1 -- 172.21.15.131:0/2326415097 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8ec156be0 con 0x7ff8d0004800 2024-05-26T22:47:57.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:57.343+0000 7ff8f1800640 1 -- 172.21.15.131:0/2326415097 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8ec1571a0 con 0x7ff8d0004800 2024-05-26T22:47:57.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:57.345+0000 7ff8e3400640 1 -- 172.21.15.131:0/2326415097 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8dc018070 con 0x7ff8d0004800 2024-05-26T22:47:57.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:57.345+0000 7ff8f1800640 1 -- 172.21.15.131:0/2326415097 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8ec074e90 con 0x7ff8d0004800 2024-05-26T22:47:57.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:57.345+0000 7ff8e3400640 1 -- 172.21.15.131:0/2326415097 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8dc012930 con 0x7ff8d0004800 2024-05-26T22:47:57.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:57.345+0000 7ff8e3400640 1 -- 172.21.15.131:0/2326415097 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff8dc012bb0 con 0x7ff8d0004800 2024-05-26T22:47:57.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:57.345+0000 7ff8e3400640 1 -- 172.21.15.131:0/2326415097 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff8dc013570 con 0x7ff8d0004800 2024-05-26T22:47:57.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:57.348+0000 7ff8e3400640 1 -- 172.21.15.131:0/2326415097 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7ff8dc013a20 con 0x7ff8d0004800 2024-05-26T22:47:57.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:57.449+0000 7ff8f1800640 1 -- 172.21.15.131:0/2326415097 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7ff8ec0cee90 con 0x7ff8d0004800 2024-05-26T22:47:57.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:57.452+0000 7ff8e3400640 1 -- 172.21.15.131:0/2326415097 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7ff8dc013c00 con 0x7ff8d0004800 2024-05-26T22:47:57.452 INFO:teuthology.orchestra.run.smithi131.stderr:adjusted tunables profile to default 2024-05-26T22:47:57.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:57.454+0000 7ff8f1800640 1 -- 172.21.15.131:0/2326415097 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff8d0004800 msgr2=0x7ff8ec1588a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:47:57.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:57.454+0000 7ff8f1800640 1 --2- 172.21.15.131:0/2326415097 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff8d0004800 0x7ff8ec1588a0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc007bd0 tx=0x7ff8dc0018b0 comp rx=0 tx=0).stop 2024-05-26T22:47:57.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:57.454+0000 7ff8f1800640 1 -- 172.21.15.131:0/2326415097 shutdown_connections 2024-05-26T22:47:57.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:57.454+0000 7ff8f1800640 1 --2- 172.21.15.131:0/2326415097 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff8ec156400 0x7ff8ec15efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:47:57.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:57.454+0000 7ff8f1800640 1 --2- 172.21.15.131:0/2326415097 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff8ec15a220 0x7ff8ec15a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:47:57.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:57.454+0000 7ff8f1800640 1 --2- 172.21.15.131:0/2326415097 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff8d0004800 0x7ff8ec1588a0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:47:57.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:57.454+0000 7ff8f1800640 1 -- 172.21.15.131:0/2326415097 >> 172.21.15.131:0/2326415097 conn(0x7ff8ec06bc90 msgr2=0x7ff8ec0ce770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:47:57.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:57.454+0000 7ff8f1800640 1 -- 172.21.15.131:0/2326415097 shutdown_connections 2024-05-26T22:47:57.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:47:57.454+0000 7ff8f1800640 1 -- 172.21.15.131:0/2326415097 wait complete. 2024-05-26T22:47:57.471 INFO:tasks.ceph:check_enable_crimson: False 2024-05-26T22:47:57.471 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-05-26T22:47:57.471 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-26T22:47:57.471 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-05-26T22:47:57.501 INFO:tasks.ceph.mgr.y.smithi131.stderr:2024-05-26T22:47:57.501+0000 7f977c60d040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-26T22:47:57.540 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-26T22:47:57.540 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-05-26T22:47:57.661 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-26T22:47:57.661 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-05-26T22:47:57.693 INFO:tasks.ceph.mgr.y.smithi131.stderr:2024-05-26T22:47:57.692+0000 7f977c60d040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-26T22:47:57.759 INFO:tasks.ceph.mgr.y.smithi131.stderr:2024-05-26T22:47:57.759+0000 7f977c60d040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-26T22:47:57.769 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-26T22:47:57.769 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-05-26T22:47:57.831 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-05-26T22:47:57.831+0000 7f2d5eb2f040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-26T22:47:57.890 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-05-26T22:47:57.891 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-05-26T22:47:57.968 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-05-26T22:47:57.968 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-05-26T22:47:58.035 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-05-26T22:47:58.036+0000 7f2d5eb2f040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-26T22:47:58.037 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-05-26T22:47:58.038 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-05-26T22:47:58.056 INFO:tasks.ceph.mgr.y.smithi131.stderr:2024-05-26T22:47:58.056+0000 7f977c60d040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-26T22:47:58.104 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-05-26T22:47:58.104+0000 7f2d5eb2f040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-26T22:47:58.109 INFO:tasks.ceph.mgr.y.smithi131.stderr:2024-05-26T22:47:58.110+0000 7f977c60d040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-26T22:47:58.148 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-05-26T22:47:58.148 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-05-26T22:47:58.261 DEBUG:teuthology.orchestra.run.smithi187:> sudo ceph --cluster ceph osd new 814eaf62-a18d-494b-8e27-fd8fb88adea6 0 2024-05-26T22:47:58.418 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-05-26T22:47:58.418+0000 7f2d5eb2f040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-26T22:47:58.478 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-05-26T22:47:58.478+0000 7f2d5eb2f040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-26T22:47:58.482 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.482+0000 7effdf200640 1 Processor -- start 2024-05-26T22:47:58.482 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.482+0000 7effdf200640 1 -- start start 2024-05-26T22:47:58.483 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.484+0000 7effdf200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7effe015c180 0x7effe0072450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:47:58.484 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.484+0000 7effdf200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7effe006fe70 0x7effe006c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:47:58.484 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.484+0000 7effdf200640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7effe00cfb20 con 0x7effe006c860 2024-05-26T22:47:58.484 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.484+0000 7effdf200640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7effe006ccd0 con 0x7effe006fe70 2024-05-26T22:47:58.484 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.484+0000 7effdf200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7effe0072990 con 0x7effe015c180 2024-05-26T22:47:58.485 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.484+0000 7effdde00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7effe015c180 0x7effe0072450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:47:58.485 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.484+0000 7effdde00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7effe015c180 0x7effe0072450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.187:34290/0 (socket says 172.21.15.187:34290) 2024-05-26T22:47:58.485 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.484+0000 7effdde00640 1 -- 172.21.15.187:0/653171004 learned_addr learned my addr 172.21.15.187:0/653171004 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-05-26T22:47:58.485 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.484+0000 7effdd400640 1 --2- 172.21.15.187:0/653171004 >> v2:172.21.15.131:3300/0 conn(0x7effe006fe70 0x7effe006c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:47:58.485 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.485+0000 7effdde00640 1 -- 172.21.15.187:0/653171004 >> v2:172.21.15.131:3300/0 conn(0x7effe006fe70 msgr2=0x7effe006c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:47:58.485 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.485+0000 7effdde00640 1 --2- 172.21.15.187:0/653171004 >> v2:172.21.15.131:3300/0 conn(0x7effe006fe70 0x7effe006c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:47:58.485 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.485+0000 7effdde00640 1 -- 172.21.15.187:0/653171004 >> v1:172.21.15.187:6789/0 conn(0x7effe006c860 legacy=0x7effe015fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:47:58.485 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.485+0000 7effdca00640 1 -- 172.21.15.187:0/653171004 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2061086093 0 0) 0x7effd8001110 con 0x7effe006c860 2024-05-26T22:47:58.485 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.485+0000 7effdde00640 1 -- 172.21.15.187:0/653171004 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effe00cfda0 con 0x7effe015c180 2024-05-26T22:47:58.486 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.486+0000 7effdde00640 1 --2- 172.21.15.187:0/653171004 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7effe015c180 0x7effe0072450 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7effd0009e50 tx=0x7effd002a730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6ba304db88cfb3c server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:47:58.486 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.486+0000 7effdca00640 1 -- 172.21.15.187:0/653171004 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effd0036070 con 0x7effe015c180 2024-05-26T22:47:58.486 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.486+0000 7effdca00640 1 -- 172.21.15.187:0/653171004 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effd0031600 con 0x7effe015c180 2024-05-26T22:47:58.486 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.486+0000 7effdca00640 1 -- 172.21.15.187:0/653171004 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effd00319d0 con 0x7effe015c180 2024-05-26T22:47:58.487 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.487+0000 7effdf200640 1 -- 172.21.15.187:0/653171004 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7effe015c180 msgr2=0x7effe0072450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:47:58.487 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.487+0000 7effdf200640 1 --2- 172.21.15.187:0/653171004 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7effe015c180 0x7effe0072450 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7effd0009e50 tx=0x7effd002a730 comp rx=0 tx=0).stop 2024-05-26T22:47:58.487 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.487+0000 7effdf200640 1 -- 172.21.15.187:0/653171004 shutdown_connections 2024-05-26T22:47:58.487 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.487+0000 7effdf200640 1 --2- 172.21.15.187:0/653171004 >> v2:172.21.15.131:3300/0 conn(0x7effe006fe70 0x7effe006c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:47:58.487 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.487+0000 7effdf200640 1 --2- 172.21.15.187:0/653171004 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7effe015c180 0x7effe0072450 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:47:58.487 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.487+0000 7effdf200640 1 -- 172.21.15.187:0/653171004 >> 172.21.15.187:0/653171004 conn(0x7effe0071930 msgr2=0x7effe0071d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:47:58.487 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.487+0000 7effdf200640 1 -- 172.21.15.187:0/653171004 shutdown_connections 2024-05-26T22:47:58.487 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.487+0000 7effdf200640 1 -- 172.21.15.187:0/653171004 wait complete. 2024-05-26T22:47:58.488 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.488+0000 7effdf200640 1 Processor -- start 2024-05-26T22:47:58.488 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.489+0000 7effdf200640 1 -- start start 2024-05-26T22:47:58.490 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.489+0000 7effdf200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7effe006c860 0x7effe014aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:47:58.490 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.489+0000 7effdf200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7effe006fe70 0x7effe0157080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:47:58.490 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.490+0000 7effdf200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7effe015c180 0x7effe01575c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:47:58.490 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.490+0000 7effdf200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7effe00b3720 con 0x7effe015c180 2024-05-26T22:47:58.490 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.490+0000 7effdf200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7effe00d0360 con 0x7effe006c860 2024-05-26T22:47:58.490 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.490+0000 7effdf200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7effe00b3a00 con 0x7effe006fe70 2024-05-26T22:47:58.491 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.490+0000 7effdde00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7effe006c860 0x7effe014aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:47:58.491 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.490+0000 7effdde00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7effe006c860 0x7effe014aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.187:36354/0 (socket says 172.21.15.187:36354) 2024-05-26T22:47:58.491 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.490+0000 7effdde00640 1 -- 172.21.15.187:0/2893516951 learned_addr learned my addr 172.21.15.187:0/2893516951 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-05-26T22:47:58.491 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.490+0000 7effdd400640 1 --2- 172.21.15.187:0/2893516951 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7effe006fe70 0x7effe0157080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:47:58.491 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.490+0000 7effde800640 1 --2- 172.21.15.187:0/2893516951 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7effe015c180 0x7effe01575c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:47:58.491 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.490+0000 7effdde00640 1 -- 172.21.15.187:0/2893516951 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7effe006fe70 msgr2=0x7effe0157080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:47:58.491 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.490+0000 7effdde00640 1 --2- 172.21.15.187:0/2893516951 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7effe006fe70 0x7effe0157080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:47:58.491 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.490+0000 7effdde00640 1 -- 172.21.15.187:0/2893516951 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7effe015c180 msgr2=0x7effe01575c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:47:58.491 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.490+0000 7effdde00640 1 --2- 172.21.15.187:0/2893516951 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7effe015c180 0x7effe01575c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:47:58.491 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.490+0000 7effdde00640 1 -- 172.21.15.187:0/2893516951 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effe0148320 con 0x7effe006c860 2024-05-26T22:47:58.491 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.491+0000 7effdde00640 1 --2- 172.21.15.187:0/2893516951 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7effe006c860 0x7effe014aaa0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7effd0034040 tx=0x7effd0004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:47:58.493 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.491+0000 7effd6000640 1 -- 172.21.15.187:0/2893516951 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effd003f020 con 0x7effe006c860 2024-05-26T22:47:58.493 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.491+0000 7effd6000640 1 -- 172.21.15.187:0/2893516951 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effd0037040 con 0x7effe006c860 2024-05-26T22:47:58.493 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.491+0000 7effdf200640 1 -- 172.21.15.187:0/2893516951 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effe0148600 con 0x7effe006c860 2024-05-26T22:47:58.493 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.491+0000 7effd6000640 1 -- 172.21.15.187:0/2893516951 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effd0036070 con 0x7effe006c860 2024-05-26T22:47:58.493 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.491+0000 7effdf200640 1 -- 172.21.15.187:0/2893516951 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effe0148bc0 con 0x7effe006c860 2024-05-26T22:47:58.493 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.491+0000 7effd6000640 1 -- 172.21.15.187:0/2893516951 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7effd0042020 con 0x7effe006c860 2024-05-26T22:47:58.493 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.491+0000 7effd6000640 1 -- 172.21.15.187:0/2893516951 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7effd0044680 con 0x7effe006c860 2024-05-26T22:47:58.494 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.492+0000 7effdf200640 1 -- 172.21.15.187:0/2893516951 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effe00b0b30 con 0x7effe006c860 2024-05-26T22:47:58.495 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.495+0000 7effd6000640 1 -- 172.21.15.187:0/2893516951 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7effe00b0b30 con 0x7effe006c860 2024-05-26T22:47:58.573 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.573+0000 7effdf200640 1 -- 172.21.15.187:0/2893516951 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd new", "uuid": "814eaf62-a18d-494b-8e27-fd8fb88adea6", "id": 0} v 0) -- 0x7effe006f8e0 con 0x7effe006c860 2024-05-26T22:47:58.581 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.582+0000 7effd6000640 1 -- 172.21.15.187:0/2893516951 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "814eaf62-a18d-494b-8e27-fd8fb88adea6", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7effd0049030 con 0x7effe006c860 2024-05-26T22:47:58.582 INFO:teuthology.orchestra.run.smithi187.stdout:0 2024-05-26T22:47:58.583 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.583+0000 7effdf200640 1 -- 172.21.15.187:0/2893516951 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7effe006c860 msgr2=0x7effe014aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:47:58.583 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.583+0000 7effdf200640 1 --2- 172.21.15.187:0/2893516951 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7effe006c860 0x7effe014aaa0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7effd0034040 tx=0x7effd0004a40 comp rx=0 tx=0).stop 2024-05-26T22:47:58.583 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.583+0000 7effdf200640 1 -- 172.21.15.187:0/2893516951 shutdown_connections 2024-05-26T22:47:58.584 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.583+0000 7effdf200640 1 --2- 172.21.15.187:0/2893516951 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7effe006fe70 0x7effe0157080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:47:58.584 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.583+0000 7effdf200640 1 --2- 172.21.15.187:0/2893516951 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7effe015c180 0x7effe01575c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:47:58.584 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.583+0000 7effdf200640 1 --2- 172.21.15.187:0/2893516951 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7effe006c860 0x7effe014aaa0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:47:58.584 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.583+0000 7effdf200640 1 -- 172.21.15.187:0/2893516951 >> 172.21.15.187:0/2893516951 conn(0x7effe0071930 msgr2=0x7effe0150e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:47:58.584 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.583+0000 7effdf200640 1 -- 172.21.15.187:0/2893516951 shutdown_connections 2024-05-26T22:47:58.584 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.583+0000 7effdf200640 1 -- 172.21.15.187:0/2893516951 wait complete. 2024-05-26T22:47:58.599 DEBUG:teuthology.orchestra.run.smithi187:> sudo ceph --cluster ceph osd new 42e66287-f6c6-4e26-af3c-6dfaa3536d27 1 2024-05-26T22:47:58.783 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.783+0000 7f8657e00640 1 Processor -- start 2024-05-26T22:47:58.783 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.783+0000 7f8657e00640 1 -- start start 2024-05-26T22:47:58.784 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.784+0000 7f8657e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f865815e5b0 0x7f86580ced40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:47:58.785 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.784+0000 7f8657e00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f86580b3430 0x7f86580cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:47:58.785 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.784+0000 7f8657e00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86580d2780 con 0x7f86580cf7c0 2024-05-26T22:47:58.785 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.784+0000 7f8657e00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f86580d2a60 con 0x7f86580b3430 2024-05-26T22:47:58.785 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.784+0000 7f8657e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f86580d2d40 con 0x7f865815e5b0 2024-05-26T22:47:58.785 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.785+0000 7f8656a00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f865815e5b0 0x7f86580ced40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:47:58.785 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.785+0000 7f8657400640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f86580cf7c0 0x7f86580d09e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.187:53428/0 (socket says 172.21.15.187:53428) 2024-05-26T22:47:58.785 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.785+0000 7f8657400640 1 -- 172.21.15.187:0/623561231 learned_addr learned my addr 172.21.15.187:0/623561231 (peer_addr_for_me v1:172.21.15.187:0/0) 2024-05-26T22:47:58.785 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.785+0000 7f8656a00640 1 --2- 172.21.15.187:0/623561231 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f865815e5b0 0x7f86580ced40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.187:34308/0 (socket says 172.21.15.187:34308) 2024-05-26T22:47:58.785 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.785+0000 7f864fe00640 1 --2- 172.21.15.187:0/623561231 >> v2:172.21.15.187:3300/0 conn(0x7f86580b3430 0x7f86580cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:47:58.786 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.785+0000 7f8656a00640 1 -- 172.21.15.187:0/623561231 >> v2:172.21.15.187:3300/0 conn(0x7f86580b3430 msgr2=0x7f86580cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:47:58.786 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.785+0000 7f8656a00640 1 --2- 172.21.15.187:0/623561231 >> v2:172.21.15.187:3300/0 conn(0x7f86580b3430 0x7f86580cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:47:58.786 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.785+0000 7f8656a00640 1 -- 172.21.15.187:0/623561231 >> v1:172.21.15.187:6789/0 conn(0x7f86580cf7c0 legacy=0x7f86580d09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:47:58.786 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.785+0000 7f8656a00640 1 -- 172.21.15.187:0/623561231 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86580d3020 con 0x7f865815e5b0 2024-05-26T22:47:58.787 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.786+0000 7f8656a00640 1 --2- 172.21.15.187:0/623561231 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f865815e5b0 0x7f86580ced40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f86480097a0 tx=0x7f864802a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=209a0306bc11d81e server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:47:58.787 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.787+0000 7f8656000640 1 -- 172.21.15.187:0/623561231 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8648037070 con 0x7f865815e5b0 2024-05-26T22:47:58.787 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.787+0000 7f8656000640 1 -- 172.21.15.187:0/623561231 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f864802ae50 con 0x7f865815e5b0 2024-05-26T22:47:58.787 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.787+0000 7f8656000640 1 -- 172.21.15.187:0/623561231 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8648032660 con 0x7f865815e5b0 2024-05-26T22:47:58.788 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.787+0000 7f8657e00640 1 -- 172.21.15.187:0/623561231 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f865815e5b0 msgr2=0x7f86580ced40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:47:58.788 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.787+0000 7f8657e00640 1 --2- 172.21.15.187:0/623561231 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f865815e5b0 0x7f86580ced40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f86480097a0 tx=0x7f864802a100 comp rx=0 tx=0).stop 2024-05-26T22:47:58.788 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.787+0000 7f8657e00640 1 -- 172.21.15.187:0/623561231 shutdown_connections 2024-05-26T22:47:58.788 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.787+0000 7f8657e00640 1 --2- 172.21.15.187:0/623561231 >> v2:172.21.15.187:3300/0 conn(0x7f86580b3430 0x7f86580cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:47:58.788 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.787+0000 7f8657e00640 1 --2- 172.21.15.187:0/623561231 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f865815e5b0 0x7f86580ced40 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:47:58.788 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.787+0000 7f8657e00640 1 -- 172.21.15.187:0/623561231 >> 172.21.15.187:0/623561231 conn(0x7f865809d4d0 msgr2=0x7f86580d41a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:47:58.788 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.788+0000 7f8657e00640 1 -- 172.21.15.187:0/623561231 shutdown_connections 2024-05-26T22:47:58.788 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.788+0000 7f8657e00640 1 -- 172.21.15.187:0/623561231 wait complete. 2024-05-26T22:47:58.790 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.790+0000 7f8657e00640 1 Processor -- start 2024-05-26T22:47:58.790 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.790+0000 7f8657e00640 1 -- start start 2024-05-26T22:47:58.793 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.792+0000 7f8657e00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f86580b3430 0x7f86580cc120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:47:58.793 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.792+0000 7f8657e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f86580cf7c0 0x7f8658155490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:47:58.793 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.792+0000 7f8657e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f86580cdd20 0x7f86580ccdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:47:58.793 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.792+0000 7f8657e00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f86580b5030 con 0x7f86580cdd20 2024-05-26T22:47:58.793 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.792+0000 7f8657e00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f86580cfc10 con 0x7f86580b3430 2024-05-26T22:47:58.793 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.792+0000 7f8657e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f86580b51b0 con 0x7f86580cf7c0 2024-05-26T22:47:58.793 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.792+0000 7f8656a00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f86580b3430 0x7f86580cc120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:47:58.793 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.792+0000 7f864fe00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f86580cf7c0 0x7f8658155490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:47:58.793 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.792+0000 7f8656a00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f86580b3430 0x7f86580cc120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.187:36378/0 (socket says 172.21.15.187:36378) 2024-05-26T22:47:58.793 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.792+0000 7f8656a00640 1 -- 172.21.15.187:0/1269743905 learned_addr learned my addr 172.21.15.187:0/1269743905 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-05-26T22:47:58.794 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.792+0000 7f864fe00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f86580cf7c0 0x7f8658155490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.187:34318/0 (socket says 172.21.15.187:34318) 2024-05-26T22:47:58.794 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.793+0000 7f8656a00640 1 -- 172.21.15.187:0/1269743905 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f86580cf7c0 msgr2=0x7f8658155490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:47:58.794 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.793+0000 7f8656a00640 1 --2- 172.21.15.187:0/1269743905 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f86580cf7c0 0x7f8658155490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:47:58.794 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.793+0000 7f8656a00640 1 -- 172.21.15.187:0/1269743905 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f86580cdd20 msgr2=0x7f86580ccdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:47:58.794 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.793+0000 7f8656a00640 1 --2- 172.21.15.187:0/1269743905 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f86580cdd20 0x7f86580ccdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:47:58.794 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.793+0000 7f8656a00640 1 -- 172.21.15.187:0/1269743905 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86580cd5d0 con 0x7f86580b3430 2024-05-26T22:47:58.794 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.793+0000 7f8656a00640 1 --2- 172.21.15.187:0/1269743905 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f86580b3430 0x7f86580cc120 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f8648035040 tx=0x7f86480046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:47:58.795 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.793+0000 7f864ea00640 1 -- 172.21.15.187:0/1269743905 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8648041070 con 0x7f86580b3430 2024-05-26T22:47:58.795 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.793+0000 7f864ea00640 1 -- 172.21.15.187:0/1269743905 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f864803c460 con 0x7f86580b3430 2024-05-26T22:47:58.795 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.793+0000 7f864ea00640 1 -- 172.21.15.187:0/1269743905 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8648037040 con 0x7f86580b3430 2024-05-26T22:47:58.795 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.793+0000 7f8657e00640 1 -- 172.21.15.187:0/1269743905 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86580cb3a0 con 0x7f86580b3430 2024-05-26T22:47:58.795 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.794+0000 7f8657e00640 1 -- 172.21.15.187:0/1269743905 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86580cb640 con 0x7f86580b3430 2024-05-26T22:47:58.795 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.794+0000 7f864ea00640 1 -- 172.21.15.187:0/1269743905 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f864803a070 con 0x7f86580b3430 2024-05-26T22:47:58.795 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.794+0000 7f864ea00640 1 -- 172.21.15.187:0/1269743905 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7f8648045d50 con 0x7f86580b3430 2024-05-26T22:47:58.795 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.795+0000 7f8657e00640 1 -- 172.21.15.187:0/1269743905 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f865806c770 con 0x7f86580b3430 2024-05-26T22:47:58.797 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.797+0000 7f864ea00640 1 -- 172.21.15.187:0/1269743905 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f865806c770 con 0x7f86580b3430 2024-05-26T22:47:58.876 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.876+0000 7f8657e00640 1 -- 172.21.15.187:0/1269743905 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd new", "uuid": "42e66287-f6c6-4e26-af3c-6dfaa3536d27", "id": 1} v 0) -- 0x7f86580694e0 con 0x7f86580b3430 2024-05-26T22:47:58.884 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.884+0000 7f864ea00640 1 -- 172.21.15.187:0/1269743905 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "42e66287-f6c6-4e26-af3c-6dfaa3536d27", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7f8648045400 con 0x7f86580b3430 2024-05-26T22:47:58.884 INFO:teuthology.orchestra.run.smithi187.stdout:1 2024-05-26T22:47:58.886 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.885+0000 7f8657e00640 1 -- 172.21.15.187:0/1269743905 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f86580b3430 msgr2=0x7f86580cc120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:47:58.886 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.885+0000 7f8657e00640 1 --2- 172.21.15.187:0/1269743905 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f86580b3430 0x7f86580cc120 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f8648035040 tx=0x7f86480046c0 comp rx=0 tx=0).stop 2024-05-26T22:47:58.886 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.886+0000 7f8657e00640 1 -- 172.21.15.187:0/1269743905 shutdown_connections 2024-05-26T22:47:58.886 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.886+0000 7f8657e00640 1 --2- 172.21.15.187:0/1269743905 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f86580cf7c0 0x7f8658155490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:47:58.886 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.886+0000 7f8657e00640 1 --2- 172.21.15.187:0/1269743905 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f86580cdd20 0x7f86580ccdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:47:58.886 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.886+0000 7f8657e00640 1 --2- 172.21.15.187:0/1269743905 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f86580b3430 0x7f86580cc120 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:47:58.886 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.886+0000 7f8657e00640 1 -- 172.21.15.187:0/1269743905 >> 172.21.15.187:0/1269743905 conn(0x7f865809d4d0 msgr2=0x7f865806c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:47:58.887 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.886+0000 7f8657e00640 1 -- 172.21.15.187:0/1269743905 shutdown_connections 2024-05-26T22:47:58.887 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:58.886+0000 7f8657e00640 1 -- 172.21.15.187:0/1269743905 wait complete. 2024-05-26T22:47:58.904 DEBUG:teuthology.orchestra.run.smithi187:> sudo ceph --cluster ceph osd new 0da34673-04a2-4fd0-b57a-0ff2b679b043 2 2024-05-26T22:47:59.085 INFO:tasks.ceph.mgr.y.smithi131.stderr:2024-05-26T22:47:59.085+0000 7f977c60d040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-26T22:47:59.085 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.085+0000 7f4834c00640 1 Processor -- start 2024-05-26T22:47:59.086 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.085+0000 7f4834c00640 1 -- start start 2024-05-26T22:47:59.087 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.087+0000 7f4834c00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f483015e440 0x7f48300afe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:47:59.087 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.087+0000 7f4834c00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f48300b5580 0x7f483006c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:47:59.087 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.087+0000 7f4834c00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f483015e820 con 0x7f483006c390 2024-05-26T22:47:59.087 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.087+0000 7f4834c00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f483006b310 con 0x7f48300b5580 2024-05-26T22:47:59.087 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.087+0000 7f4834c00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f483006b5d0 con 0x7f483015e440 2024-05-26T22:47:59.087 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.087+0000 7f482de00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f483015e440 0x7f48300afe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:47:59.087 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.087+0000 7f482de00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f483015e440 0x7f48300afe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.187:34324/0 (socket says 172.21.15.187:34324) 2024-05-26T22:47:59.087 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.087+0000 7f482de00640 1 -- 172.21.15.187:0/1249982009 learned_addr learned my addr 172.21.15.187:0/1249982009 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-05-26T22:47:59.087 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.087+0000 7f482e800640 1 --2- 172.21.15.187:0/1249982009 >> v2:172.21.15.131:3300/0 conn(0x7f48300b5580 0x7f483006c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:47:59.088 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.088+0000 7f482de00640 1 -- 172.21.15.187:0/1249982009 >> v2:172.21.15.131:3300/0 conn(0x7f48300b5580 msgr2=0x7f483006c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:47:59.088 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.088+0000 7f482de00640 1 --2- 172.21.15.187:0/1249982009 >> v2:172.21.15.131:3300/0 conn(0x7f48300b5580 0x7f483006c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:47:59.088 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.088+0000 7f482de00640 1 -- 172.21.15.187:0/1249982009 >> v1:172.21.15.131:6789/0 conn(0x7f483006c390 legacy=0x7f48300b4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:47:59.088 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.088+0000 7f482de00640 1 -- 172.21.15.187:0/1249982009 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f483006b8b0 con 0x7f483015e440 2024-05-26T22:47:59.089 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.089+0000 7f482de00640 1 --2- 172.21.15.187:0/1249982009 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f483015e440 0x7f48300afe70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f4820009e50 tx=0x7f482002a6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2fb7c39a54374e68 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:47:59.090 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.089+0000 7f482ca00640 1 -- 172.21.15.187:0/1249982009 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4820036070 con 0x7f483015e440 2024-05-26T22:47:59.090 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.089+0000 7f482ca00640 1 -- 172.21.15.187:0/1249982009 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4820031600 con 0x7f483015e440 2024-05-26T22:47:59.090 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.089+0000 7f482ca00640 1 -- 172.21.15.187:0/1249982009 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48200319d0 con 0x7f483015e440 2024-05-26T22:47:59.090 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.090+0000 7f4834c00640 1 -- 172.21.15.187:0/1249982009 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f483015e440 msgr2=0x7f48300afe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:47:59.090 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.090+0000 7f4834c00640 1 --2- 172.21.15.187:0/1249982009 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f483015e440 0x7f48300afe70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f4820009e50 tx=0x7f482002a6c0 comp rx=0 tx=0).stop 2024-05-26T22:47:59.090 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.090+0000 7f4834c00640 1 -- 172.21.15.187:0/1249982009 shutdown_connections 2024-05-26T22:47:59.090 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.090+0000 7f4834c00640 1 --2- 172.21.15.187:0/1249982009 >> v2:172.21.15.131:3300/0 conn(0x7f48300b5580 0x7f483006c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:47:59.090 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.090+0000 7f4834c00640 1 --2- 172.21.15.187:0/1249982009 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f483015e440 0x7f48300afe70 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:47:59.090 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.090+0000 7f4834c00640 1 -- 172.21.15.187:0/1249982009 >> 172.21.15.187:0/1249982009 conn(0x7f4830072b00 msgr2=0x7f483006de50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:47:59.090 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.090+0000 7f4834c00640 1 -- 172.21.15.187:0/1249982009 shutdown_connections 2024-05-26T22:47:59.090 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.090+0000 7f4834c00640 1 -- 172.21.15.187:0/1249982009 wait complete. 2024-05-26T22:47:59.092 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.092+0000 7f4834c00640 1 Processor -- start 2024-05-26T22:47:59.092 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.092+0000 7f4834c00640 1 -- start start 2024-05-26T22:47:59.093 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.093+0000 7f4834c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f483006c390 0x7f4830161380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:47:59.093 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.093+0000 7f4834c00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f48300b5580 0x7f483014ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:47:59.093 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.093+0000 7f4834c00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f483015e440 0x7f483014e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:47:59.094 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.093+0000 7f4834c00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f48300add40 con 0x7f483006c390 2024-05-26T22:47:59.094 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.093+0000 7f4834c00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f483006be70 con 0x7f48300b5580 2024-05-26T22:47:59.094 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.093+0000 7f4834c00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f48300adec0 con 0x7f483015e440 2024-05-26T22:47:59.094 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.093+0000 7f482e800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f483015e440 0x7f483014e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:47:59.094 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.093+0000 7f482d400640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f48300b5580 0x7f483014ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:47:59.094 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.093+0000 7f482e800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f483015e440 0x7f483014e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.187:34334/0 (socket says 172.21.15.187:34334) 2024-05-26T22:47:59.094 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.093+0000 7f482e800640 1 -- 172.21.15.187:0/3968532666 learned_addr learned my addr 172.21.15.187:0/3968532666 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-05-26T22:47:59.094 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.094+0000 7f482de00640 1 --2- 172.21.15.187:0/3968532666 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f483006c390 0x7f4830161380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:47:59.094 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.094+0000 7f482d400640 1 -- 172.21.15.187:0/3968532666 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f483015e440 msgr2=0x7f483014e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:47:59.094 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.094+0000 7f482d400640 1 --2- 172.21.15.187:0/3968532666 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f483015e440 0x7f483014e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:47:59.094 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.094+0000 7f482d400640 1 -- 172.21.15.187:0/3968532666 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f483006c390 msgr2=0x7f4830161380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:47:59.094 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.094+0000 7f482d400640 1 --2- 172.21.15.187:0/3968532666 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f483006c390 0x7f4830161380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:47:59.095 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.094+0000 7f482d400640 1 -- 172.21.15.187:0/3968532666 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f483013fcf0 con 0x7f48300b5580 2024-05-26T22:47:59.095 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.094+0000 7f482e800640 1 --2- 172.21.15.187:0/3968532666 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f483015e440 0x7f483014e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-26T22:47:59.096 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.094+0000 7f482d400640 1 --2- 172.21.15.187:0/3968532666 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f48300b5580 0x7f483014ded0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f481800c410 tx=0x7f481800c8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:47:59.096 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.094+0000 7f4826000640 1 -- 172.21.15.187:0/3968532666 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4818017070 con 0x7f48300b5580 2024-05-26T22:47:59.096 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.094+0000 7f4826000640 1 -- 172.21.15.187:0/3968532666 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4818005230 con 0x7f48300b5580 2024-05-26T22:47:59.096 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.094+0000 7f4826000640 1 -- 172.21.15.187:0/3968532666 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48180045e0 con 0x7f48300b5580 2024-05-26T22:47:59.096 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.094+0000 7f4834c00640 1 -- 172.21.15.187:0/3968532666 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f483014d2f0 con 0x7f48300b5580 2024-05-26T22:47:59.096 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.095+0000 7f4834c00640 1 -- 172.21.15.187:0/3968532666 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f483014ed90 con 0x7f48300b5580 2024-05-26T22:47:59.096 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.095+0000 7f4826000640 1 -- 172.21.15.187:0/3968532666 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f481801a070 con 0x7f48300b5580 2024-05-26T22:47:59.096 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.095+0000 7f4826000640 1 -- 172.21.15.187:0/3968532666 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f4818007440 con 0x7f48300b5580 2024-05-26T22:47:59.096 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.096+0000 7f4834c00640 1 -- 172.21.15.187:0/3968532666 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48300695c0 con 0x7f48300b5580 2024-05-26T22:47:59.097 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.098+0000 7f4826000640 1 -- 172.21.15.187:0/3968532666 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f4818006e10 con 0x7f48300b5580 2024-05-26T22:47:59.179 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.179+0000 7f4834c00640 1 -- 172.21.15.187:0/3968532666 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd new", "uuid": "0da34673-04a2-4fd0-b57a-0ff2b679b043", "id": 2} v 0) -- 0x7f483015ebe0 con 0x7f48300b5580 2024-05-26T22:47:59.186 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.187+0000 7f4826000640 1 -- 172.21.15.187:0/3968532666 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "0da34673-04a2-4fd0-b57a-0ff2b679b043", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f48180067d0 con 0x7f48300b5580 2024-05-26T22:47:59.187 INFO:teuthology.orchestra.run.smithi187.stdout:2 2024-05-26T22:47:59.188 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.188+0000 7f4834c00640 1 -- 172.21.15.187:0/3968532666 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f48300b5580 msgr2=0x7f483014ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:47:59.188 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.188+0000 7f4834c00640 1 --2- 172.21.15.187:0/3968532666 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f48300b5580 0x7f483014ded0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f481800c410 tx=0x7f481800c8d0 comp rx=0 tx=0).stop 2024-05-26T22:47:59.188 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.188+0000 7f4834c00640 1 -- 172.21.15.187:0/3968532666 shutdown_connections 2024-05-26T22:47:59.188 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.188+0000 7f4834c00640 1 --2- 172.21.15.187:0/3968532666 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f483015e440 0x7f483014e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:47:59.189 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.188+0000 7f4834c00640 1 --2- 172.21.15.187:0/3968532666 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f48300b5580 0x7f483014ded0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:47:59.189 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.188+0000 7f4834c00640 1 --2- 172.21.15.187:0/3968532666 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f483006c390 0x7f4830161380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:47:59.189 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.188+0000 7f4834c00640 1 -- 172.21.15.187:0/3968532666 >> 172.21.15.187:0/3968532666 conn(0x7f4830072b00 msgr2=0x7f48300b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:47:59.189 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.188+0000 7f4834c00640 1 -- 172.21.15.187:0/3968532666 shutdown_connections 2024-05-26T22:47:59.189 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.188+0000 7f4834c00640 1 -- 172.21.15.187:0/3968532666 wait complete. 2024-05-26T22:47:59.190 INFO:tasks.ceph.mgr.y.smithi131.stderr:2024-05-26T22:47:59.190+0000 7f977c60d040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-26T22:47:59.206 DEBUG:teuthology.orchestra.run.smithi187:> sudo ceph --cluster ceph osd new cf117a27-d6b2-43d3-ae52-b47988bf8eed 3 2024-05-26T22:47:59.241 INFO:tasks.ceph.mgr.y.smithi131.stderr:2024-05-26T22:47:59.241+0000 7f977c60d040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-26T22:47:59.395 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.393+0000 7f6cefe00640 1 Processor -- start 2024-05-26T22:47:59.395 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.393+0000 7f6cefe00640 1 -- start start 2024-05-26T22:47:59.395 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.394+0000 7f6cefe00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6cf015e5b0 0x7f6cf00ced40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:47:59.396 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.394+0000 7f6cefe00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f6cf00b3430 0x7f6cf00cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:47:59.396 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.394+0000 7f6cefe00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6cf00d2780 con 0x7f6cf00cf7c0 2024-05-26T22:47:59.396 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.395+0000 7f6cefe00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f6cf00d2a60 con 0x7f6cf00b3430 2024-05-26T22:47:59.396 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.395+0000 7f6cefe00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f6cf00d2d40 con 0x7f6cf015e5b0 2024-05-26T22:47:59.396 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.395+0000 7f6ceea00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6cf015e5b0 0x7f6cf00ced40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:47:59.396 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.395+0000 7f6ceea00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6cf015e5b0 0x7f6cf00ced40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.187:34342/0 (socket says 172.21.15.187:34342) 2024-05-26T22:47:59.396 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.395+0000 7f6ceea00640 1 -- 172.21.15.187:0/3028135911 learned_addr learned my addr 172.21.15.187:0/3028135911 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-05-26T22:47:59.396 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.395+0000 7f6cee000640 1 --2- 172.21.15.187:0/3028135911 >> v2:172.21.15.131:3300/0 conn(0x7f6cf00b3430 0x7f6cf00cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:47:59.396 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.395+0000 7f6ceea00640 1 -- 172.21.15.187:0/3028135911 >> v2:172.21.15.131:3300/0 conn(0x7f6cf00b3430 msgr2=0x7f6cf00cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:47:59.396 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.395+0000 7f6ceea00640 1 --2- 172.21.15.187:0/3028135911 >> v2:172.21.15.131:3300/0 conn(0x7f6cf00b3430 0x7f6cf00cf280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:47:59.396 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.395+0000 7f6ceea00640 1 -- 172.21.15.187:0/3028135911 >> v1:172.21.15.131:6789/0 conn(0x7f6cf00cf7c0 legacy=0x7f6cf00d09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:47:59.396 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.395+0000 7f6ceea00640 1 -- 172.21.15.187:0/3028135911 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cf00d3020 con 0x7f6cf015e5b0 2024-05-26T22:47:59.398 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.396+0000 7f6ceea00640 1 --2- 172.21.15.187:0/3028135911 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6cf015e5b0 0x7f6cf00ced40 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6ce8009e50 tx=0x7f6ce802a630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de9b5c38d91d484e server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:47:59.398 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.397+0000 7f6ced600640 1 -- 172.21.15.187:0/3028135911 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ce8036070 con 0x7f6cf015e5b0 2024-05-26T22:47:59.398 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.397+0000 7f6ced600640 1 -- 172.21.15.187:0/3028135911 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ce8031600 con 0x7f6cf015e5b0 2024-05-26T22:47:59.398 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.397+0000 7f6ced600640 1 -- 172.21.15.187:0/3028135911 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ce80319d0 con 0x7f6cf015e5b0 2024-05-26T22:47:59.398 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.397+0000 7f6cefe00640 1 -- 172.21.15.187:0/3028135911 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6cf015e5b0 msgr2=0x7f6cf00ced40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:47:59.398 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.397+0000 7f6cefe00640 1 --2- 172.21.15.187:0/3028135911 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6cf015e5b0 0x7f6cf00ced40 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6ce8009e50 tx=0x7f6ce802a630 comp rx=0 tx=0).stop 2024-05-26T22:47:59.398 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.397+0000 7f6cefe00640 1 -- 172.21.15.187:0/3028135911 shutdown_connections 2024-05-26T22:47:59.398 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.397+0000 7f6cefe00640 1 --2- 172.21.15.187:0/3028135911 >> v2:172.21.15.131:3300/0 conn(0x7f6cf00b3430 0x7f6cf00cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:47:59.398 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.397+0000 7f6cefe00640 1 --2- 172.21.15.187:0/3028135911 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6cf015e5b0 0x7f6cf00ced40 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:47:59.398 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.397+0000 7f6cefe00640 1 -- 172.21.15.187:0/3028135911 >> 172.21.15.187:0/3028135911 conn(0x7f6cf009d4d0 msgr2=0x7f6cf00d41a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:47:59.399 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.397+0000 7f6cefe00640 1 -- 172.21.15.187:0/3028135911 shutdown_connections 2024-05-26T22:47:59.399 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.397+0000 7f6cefe00640 1 -- 172.21.15.187:0/3028135911 wait complete. 2024-05-26T22:47:59.399 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.399+0000 7f6cefe00640 1 Processor -- start 2024-05-26T22:47:59.399 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.399+0000 7f6cefe00640 1 -- start start 2024-05-26T22:47:59.401 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.400+0000 7f6cefe00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6cf00b3430 0x7f6cf00b83c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:47:59.402 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.400+0000 7f6cefe00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6cf00cf7c0 0x7f6cf01531e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:47:59.402 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.400+0000 7f6cefe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6cf00b6d60 0x7f6cf00c99b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:47:59.402 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.400+0000 7f6cefe00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f6cf00b5030 con 0x7f6cf00b6d60 2024-05-26T22:47:59.402 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.400+0000 7f6cefe00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6cf00cfc10 con 0x7f6cf00b3430 2024-05-26T22:47:59.402 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.400+0000 7f6cefe00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f6cf00b51b0 con 0x7f6cf00cf7c0 2024-05-26T22:47:59.402 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.400+0000 7f6cee000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6cf00cf7c0 0x7f6cf01531e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:47:59.402 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.400+0000 7f6ceea00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6cf00b3430 0x7f6cf00b83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:47:59.402 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.400+0000 7f6cee000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6cf00cf7c0 0x7f6cf01531e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.187:34358/0 (socket says 172.21.15.187:34358) 2024-05-26T22:47:59.402 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.400+0000 7f6cee000640 1 -- 172.21.15.187:0/794240221 learned_addr learned my addr 172.21.15.187:0/794240221 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-05-26T22:47:59.402 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.401+0000 7f6cef400640 1 --2- 172.21.15.187:0/794240221 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6cf00b6d60 0x7f6cf00c99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:47:59.402 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.401+0000 7f6cee000640 1 -- 172.21.15.187:0/794240221 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6cf00b3430 msgr2=0x7f6cf00b83c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:47:59.402 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.401+0000 7f6cee000640 1 --2- 172.21.15.187:0/794240221 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6cf00b3430 0x7f6cf00b83c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:47:59.403 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.401+0000 7f6cee000640 1 -- 172.21.15.187:0/794240221 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6cf00b6d60 msgr2=0x7f6cf00c99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:47:59.403 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.401+0000 7f6cee000640 1 --2- 172.21.15.187:0/794240221 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6cf00b6d60 0x7f6cf00c99b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:47:59.403 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.401+0000 7f6cee000640 1 -- 172.21.15.187:0/794240221 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cf00b7320 con 0x7f6cf00cf7c0 2024-05-26T22:47:59.403 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.401+0000 7f6ceea00640 1 --2- 172.21.15.187:0/794240221 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6cf00b3430 0x7f6cf00b83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-26T22:47:59.403 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.401+0000 7f6cee000640 1 --2- 172.21.15.187:0/794240221 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6cf00cf7c0 0x7f6cf01531e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc00c7f0 tx=0x7f6cdc00ccb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:47:59.403 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.401+0000 7f6ce6a00640 1 -- 172.21.15.187:0/794240221 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cdc015070 con 0x7f6cf00cf7c0 2024-05-26T22:47:59.403 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.401+0000 7f6ce6a00640 1 -- 172.21.15.187:0/794240221 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cdc004220 con 0x7f6cf00cf7c0 2024-05-26T22:47:59.403 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.401+0000 7f6cefe00640 1 -- 172.21.15.187:0/794240221 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cf00b7600 con 0x7f6cf00cf7c0 2024-05-26T22:47:59.403 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.401+0000 7f6ce6a00640 1 -- 172.21.15.187:0/794240221 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cdc007620 con 0x7f6cf00cf7c0 2024-05-26T22:47:59.403 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.402+0000 7f6cefe00640 1 -- 172.21.15.187:0/794240221 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cf00b7bc0 con 0x7f6cf00cf7c0 2024-05-26T22:47:59.403 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.402+0000 7f6ce6a00640 1 -- 172.21.15.187:0/794240221 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f6cdc0079d0 con 0x7f6cf00cf7c0 2024-05-26T22:47:59.403 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.402+0000 7f6ce6a00640 1 -- 172.21.15.187:0/794240221 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7f6cdc007cc0 con 0x7f6cf00cf7c0 2024-05-26T22:47:59.404 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.403+0000 7f6cefe00640 1 -- 172.21.15.187:0/794240221 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cf00694e0 con 0x7f6cf00cf7c0 2024-05-26T22:47:59.405 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.405+0000 7f6ce6a00640 1 -- 172.21.15.187:0/794240221 <== mon.2 v2:172.21.15.187:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f6cdc003610 con 0x7f6cf00cf7c0 2024-05-26T22:47:59.432 INFO:tasks.ceph.mgr.y.smithi131.stderr:2024-05-26T22:47:59.433+0000 7f977c60d040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-26T22:47:59.437 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-05-26T22:47:59.437+0000 7f2d5eb2f040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-26T22:47:59.486 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.484+0000 7f6cefe00640 1 -- 172.21.15.187:0/794240221 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "osd new", "uuid": "cf117a27-d6b2-43d3-ae52-b47988bf8eed", "id": 3} v 0) -- 0x7f6cf006ce50 con 0x7f6cf00cf7c0 2024-05-26T22:47:59.491 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.491+0000 7f6ce6a00640 1 -- 172.21.15.187:0/794240221 <== mon.2 v2:172.21.15.187:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "cf117a27-d6b2-43d3-ae52-b47988bf8eed", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7f6cdc003870 con 0x7f6cf00cf7c0 2024-05-26T22:47:59.491 INFO:teuthology.orchestra.run.smithi187.stdout:3 2024-05-26T22:47:59.493 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.493+0000 7f6cefe00640 1 -- 172.21.15.187:0/794240221 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6cf00cf7c0 msgr2=0x7f6cf01531e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:47:59.493 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.493+0000 7f6cefe00640 1 --2- 172.21.15.187:0/794240221 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6cf00cf7c0 0x7f6cf01531e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc00c7f0 tx=0x7f6cdc00ccb0 comp rx=0 tx=0).stop 2024-05-26T22:47:59.493 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.493+0000 7f6cefe00640 1 -- 172.21.15.187:0/794240221 shutdown_connections 2024-05-26T22:47:59.493 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.493+0000 7f6cefe00640 1 --2- 172.21.15.187:0/794240221 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6cf00cf7c0 0x7f6cf01531e0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:47:59.493 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.493+0000 7f6cefe00640 1 --2- 172.21.15.187:0/794240221 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6cf00b6d60 0x7f6cf00c99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:47:59.493 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.493+0000 7f6cefe00640 1 --2- 172.21.15.187:0/794240221 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6cf00b3430 0x7f6cf00b83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:47:59.493 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.493+0000 7f6cefe00640 1 -- 172.21.15.187:0/794240221 >> 172.21.15.187:0/794240221 conn(0x7f6cf009d4d0 msgr2=0x7f6cf00d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:47:59.493 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.493+0000 7f6cefe00640 1 -- 172.21.15.187:0/794240221 shutdown_connections 2024-05-26T22:47:59.494 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.493+0000 7f6cefe00640 1 -- 172.21.15.187:0/794240221 wait complete. 2024-05-26T22:47:59.495 INFO:tasks.ceph.mgr.y.smithi131.stderr:2024-05-26T22:47:59.495+0000 7f977c60d040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-26T22:47:59.509 DEBUG:teuthology.orchestra.run.smithi187:> sudo ceph --cluster ceph osd new a092ccfc-be89-4a84-88d5-0f781fa715eb 4 2024-05-26T22:47:59.547 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-05-26T22:47:59.547+0000 7f2d5eb2f040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-26T22:47:59.556 INFO:tasks.ceph.mgr.y.smithi131.stderr:2024-05-26T22:47:59.556+0000 7f977c60d040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-26T22:47:59.601 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-05-26T22:47:59.601+0000 7f2d5eb2f040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-26T22:47:59.682 INFO:tasks.ceph.mgr.y.smithi131.stderr:2024-05-26T22:47:59.683+0000 7f977c60d040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-26T22:47:59.683 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.684+0000 7f80e6e00640 1 Processor -- start 2024-05-26T22:47:59.683 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.684+0000 7f80e6e00640 1 -- start start 2024-05-26T22:47:59.685 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.685+0000 7f80e6e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f80e015e5b0 0x7f80e00ced40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:47:59.686 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.685+0000 7f80e6e00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f80e00b3430 0x7f80e00cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:47:59.686 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.685+0000 7f80e6e00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80e00d2780 con 0x7f80e00cf7c0 2024-05-26T22:47:59.686 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.685+0000 7f80e6e00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f80e00d2a60 con 0x7f80e015e5b0 2024-05-26T22:47:59.686 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.685+0000 7f80e6e00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f80e00d2d40 con 0x7f80e00b3430 2024-05-26T22:47:59.686 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.685+0000 7f80e5000640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f80e00b3430 0x7f80e00cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:47:59.686 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.685+0000 7f80e5000640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f80e00b3430 0x7f80e00cf280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.187:36406/0 (socket says 172.21.15.187:36406) 2024-05-26T22:47:59.686 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.685+0000 7f80e5000640 1 -- 172.21.15.187:0/409961082 learned_addr learned my addr 172.21.15.187:0/409961082 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-05-26T22:47:59.686 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.686+0000 7f80e5a00640 1 --2- 172.21.15.187:0/409961082 >> v2:172.21.15.131:3300/0 conn(0x7f80e015e5b0 0x7f80e00ced40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:47:59.686 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.686+0000 7f80e5000640 1 -- 172.21.15.187:0/409961082 >> v2:172.21.15.131:3300/0 conn(0x7f80e015e5b0 msgr2=0x7f80e00ced40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:47:59.686 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.686+0000 7f80e5000640 1 --2- 172.21.15.187:0/409961082 >> v2:172.21.15.131:3300/0 conn(0x7f80e015e5b0 0x7f80e00ced40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:47:59.686 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.686+0000 7f80e5000640 1 -- 172.21.15.187:0/409961082 >> v1:172.21.15.131:6789/0 conn(0x7f80e00cf7c0 legacy=0x7f80e00d09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:47:59.687 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.686+0000 7f80e5000640 1 -- 172.21.15.187:0/409961082 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80e00d3020 con 0x7f80e00b3430 2024-05-26T22:47:59.687 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.687+0000 7f80e5000640 1 --2- 172.21.15.187:0/409961082 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f80e00b3430 0x7f80e00cf280 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f80e0067a80 tx=0x7f80cc02a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51c6323c8f00d577 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:47:59.687 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.687+0000 7f80dfe00640 1 -- 172.21.15.187:0/409961082 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80cc037070 con 0x7f80e00b3430 2024-05-26T22:47:59.687 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.687+0000 7f80dfe00640 1 -- 172.21.15.187:0/409961082 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80cc032440 con 0x7f80e00b3430 2024-05-26T22:47:59.687 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.687+0000 7f80dfe00640 1 -- 172.21.15.187:0/409961082 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80cc032810 con 0x7f80e00b3430 2024-05-26T22:47:59.688 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.688+0000 7f80e6e00640 1 -- 172.21.15.187:0/409961082 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f80e00b3430 msgr2=0x7f80e00cf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:47:59.688 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.688+0000 7f80e6e00640 1 --2- 172.21.15.187:0/409961082 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f80e00b3430 0x7f80e00cf280 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f80e0067a80 tx=0x7f80cc02a1a0 comp rx=0 tx=0).stop 2024-05-26T22:47:59.688 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.688+0000 7f80e6e00640 1 -- 172.21.15.187:0/409961082 shutdown_connections 2024-05-26T22:47:59.688 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.688+0000 7f80e6e00640 1 --2- 172.21.15.187:0/409961082 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f80e00b3430 0x7f80e00cf280 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:47:59.688 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.688+0000 7f80e6e00640 1 --2- 172.21.15.187:0/409961082 >> v2:172.21.15.131:3300/0 conn(0x7f80e015e5b0 0x7f80e00ced40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:47:59.688 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.688+0000 7f80e6e00640 1 -- 172.21.15.187:0/409961082 >> 172.21.15.187:0/409961082 conn(0x7f80e009d4d0 msgr2=0x7f80e00d41a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:47:59.689 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.688+0000 7f80e6e00640 1 -- 172.21.15.187:0/409961082 shutdown_connections 2024-05-26T22:47:59.689 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.688+0000 7f80e6e00640 1 -- 172.21.15.187:0/409961082 wait complete. 2024-05-26T22:47:59.689 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.690+0000 7f80e6e00640 1 Processor -- start 2024-05-26T22:47:59.690 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.690+0000 7f80e6e00640 1 -- start start 2024-05-26T22:47:59.691 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.691+0000 7f80e6e00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f80e00b3430 0x7f80e00cab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:47:59.691 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.691+0000 7f80e6e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f80e00cf7c0 0x7f80e00c7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:47:59.691 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.691+0000 7f80e6e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f80e0159db0 0x7f80e01f7680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:47:59.691 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.691+0000 7f80e6e00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f80e00b5030 con 0x7f80e00cf7c0 2024-05-26T22:47:59.692 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.691+0000 7f80e6e00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f80e00cfbd0 con 0x7f80e00b3430 2024-05-26T22:47:59.692 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.691+0000 7f80e6e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f80e00b51b0 con 0x7f80e0159db0 2024-05-26T22:47:59.692 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.691+0000 7f80e5a00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f80e00b3430 0x7f80e00cab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:47:59.692 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.691+0000 7f80e5a00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f80e00b3430 0x7f80e00cab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.187:36408/0 (socket says 172.21.15.187:36408) 2024-05-26T22:47:59.692 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.691+0000 7f80e5a00640 1 -- 172.21.15.187:0/3643053789 learned_addr learned my addr 172.21.15.187:0/3643053789 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-05-26T22:47:59.692 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.691+0000 7f80e6400640 1 --2- 172.21.15.187:0/3643053789 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f80e0159db0 0x7f80e01f7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:47:59.692 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.691+0000 7f80e5000640 1 --2- 172.21.15.187:0/3643053789 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f80e00cf7c0 0x7f80e00c7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:47:59.692 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.692+0000 7f80e5a00640 1 -- 172.21.15.187:0/3643053789 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f80e0159db0 msgr2=0x7f80e01f7680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:47:59.692 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.692+0000 7f80e5a00640 1 --2- 172.21.15.187:0/3643053789 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f80e0159db0 0x7f80e01f7680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:47:59.692 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.692+0000 7f80e5a00640 1 -- 172.21.15.187:0/3643053789 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f80e00cf7c0 msgr2=0x7f80e00c7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:47:59.692 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.692+0000 7f80e5a00640 1 --2- 172.21.15.187:0/3643053789 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f80e00cf7c0 0x7f80e00c7cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:47:59.692 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.692+0000 7f80e5a00640 1 -- 172.21.15.187:0/3643053789 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80e00c8210 con 0x7f80e00b3430 2024-05-26T22:47:59.694 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.692+0000 7f80e5a00640 1 --2- 172.21.15.187:0/3643053789 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f80e00b3430 0x7f80e00cab00 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f80d400d800 tx=0x7f80d400dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:47:59.694 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.692+0000 7f80dd600640 1 -- 172.21.15.187:0/3643053789 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80d400b6c0 con 0x7f80e00b3430 2024-05-26T22:47:59.694 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.692+0000 7f80dd600640 1 -- 172.21.15.187:0/3643053789 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80d4019070 con 0x7f80e00b3430 2024-05-26T22:47:59.694 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.692+0000 7f80e6e00640 1 -- 172.21.15.187:0/3643053789 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80e00c84e0 con 0x7f80e00b3430 2024-05-26T22:47:59.694 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.692+0000 7f80dd600640 1 -- 172.21.15.187:0/3643053789 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80d400ba80 con 0x7f80e00b3430 2024-05-26T22:47:59.694 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.692+0000 7f80e6e00640 1 -- 172.21.15.187:0/3643053789 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80e00c8aa0 con 0x7f80e00b3430 2024-05-26T22:47:59.694 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.693+0000 7f80dd600640 1 -- 172.21.15.187:0/3643053789 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f80d400bc20 con 0x7f80e00b3430 2024-05-26T22:47:59.694 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.693+0000 7f80dd600640 1 -- 172.21.15.187:0/3643053789 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f80d401e970 con 0x7f80e00b3430 2024-05-26T22:47:59.694 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.693+0000 7f80e6e00640 1 -- 172.21.15.187:0/3643053789 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80e00694e0 con 0x7f80e00b3430 2024-05-26T22:47:59.695 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.696+0000 7f80dd600640 1 -- 172.21.15.187:0/3643053789 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f80d401ec10 con 0x7f80e00b3430 2024-05-26T22:47:59.775 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.775+0000 7f80e6e00640 1 -- 172.21.15.187:0/3643053789 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd new", "uuid": "a092ccfc-be89-4a84-88d5-0f781fa715eb", "id": 4} v 0) -- 0x7f80e00d3920 con 0x7f80e00b3430 2024-05-26T22:47:59.782 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.783+0000 7f80dd600640 1 -- 172.21.15.187:0/3643053789 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a092ccfc-be89-4a84-88d5-0f781fa715eb", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7f80d401e460 con 0x7f80e00b3430 2024-05-26T22:47:59.782 INFO:teuthology.orchestra.run.smithi187.stdout:4 2024-05-26T22:47:59.784 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.784+0000 7f80e6e00640 1 -- 172.21.15.187:0/3643053789 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f80e00b3430 msgr2=0x7f80e00cab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:47:59.784 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.784+0000 7f80e6e00640 1 --2- 172.21.15.187:0/3643053789 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f80e00b3430 0x7f80e00cab00 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f80d400d800 tx=0x7f80d400dcc0 comp rx=0 tx=0).stop 2024-05-26T22:47:59.784 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.784+0000 7f80e6e00640 1 -- 172.21.15.187:0/3643053789 shutdown_connections 2024-05-26T22:47:59.784 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.784+0000 7f80e6e00640 1 --2- 172.21.15.187:0/3643053789 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f80e0159db0 0x7f80e01f7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:47:59.784 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.784+0000 7f80e6e00640 1 --2- 172.21.15.187:0/3643053789 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f80e00cf7c0 0x7f80e00c7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:47:59.784 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.784+0000 7f80e6e00640 1 --2- 172.21.15.187:0/3643053789 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f80e00b3430 0x7f80e00cab00 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:47:59.784 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.784+0000 7f80e6e00640 1 -- 172.21.15.187:0/3643053789 >> 172.21.15.187:0/3643053789 conn(0x7f80e009d4d0 msgr2=0x7f80e00b0fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:47:59.784 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.784+0000 7f80e6e00640 1 -- 172.21.15.187:0/3643053789 shutdown_connections 2024-05-26T22:47:59.784 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.784+0000 7f80e6e00640 1 -- 172.21.15.187:0/3643053789 wait complete. 2024-05-26T22:47:59.800 DEBUG:teuthology.orchestra.run.smithi187:> sudo ceph --cluster ceph osd new 7513ba69-dd00-4a35-b190-9e8cc3b2bdc8 5 2024-05-26T22:47:59.801 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-05-26T22:47:59.801+0000 7f2d5eb2f040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-26T22:47:59.880 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-05-26T22:47:59.880+0000 7f2d5eb2f040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-26T22:47:59.944 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-05-26T22:47:59.944+0000 7f2d5eb2f040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-26T22:47:59.986 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.986+0000 7fbb59a00640 1 Processor -- start 2024-05-26T22:47:59.986 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.986+0000 7fbb59a00640 1 -- start start 2024-05-26T22:47:59.988 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.988+0000 7fbb59a00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fbb54161fd0 0x7fbb5406b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:47:59.988 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.988+0000 7fbb59a00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbb54162460 0x7fbb5406b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:47:59.989 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.988+0000 7fbb59a00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb541629b0 con 0x7fbb54144a70 2024-05-26T22:47:59.989 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.988+0000 7fbb59a00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fbb54144e50 con 0x7fbb54161fd0 2024-05-26T22:47:59.989 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.988+0000 7fbb59a00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fbb54162c90 con 0x7fbb54162460 2024-05-26T22:47:59.989 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.989+0000 7fbb4be00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fbb54161fd0 0x7fbb5406b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:47:59.989 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.989+0000 7fbb59000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbb54162460 0x7fbb5406b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:47:59.989 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.989+0000 7fbb4be00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fbb54161fd0 0x7fbb5406b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.187:36422/0 (socket says 172.21.15.187:36422) 2024-05-26T22:47:59.989 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.989+0000 7fbb4be00640 1 -- 172.21.15.187:0/3571948164 learned_addr learned my addr 172.21.15.187:0/3571948164 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-05-26T22:47:59.990 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.989+0000 7fbb59000640 1 -- 172.21.15.187:0/3571948164 >> v2:172.21.15.187:3300/0 conn(0x7fbb54161fd0 msgr2=0x7fbb5406b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:47:59.990 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.989+0000 7fbb59000640 1 --2- 172.21.15.187:0/3571948164 >> v2:172.21.15.187:3300/0 conn(0x7fbb54161fd0 0x7fbb5406b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:47:59.990 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.989+0000 7fbb59000640 1 -- 172.21.15.187:0/3571948164 >> v1:172.21.15.131:6789/0 conn(0x7fbb54144a70 legacy=0x7fbb5415f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:47:59.990 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.989+0000 7fbb59000640 1 -- 172.21.15.187:0/3571948164 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb5415ebf0 con 0x7fbb54162460 2024-05-26T22:47:59.990 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.990+0000 7fbb59000640 1 --2- 172.21.15.187:0/3571948164 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbb54162460 0x7fbb5406b850 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c009d90 tx=0x7fbb4c02a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c92e8c00f933f1d server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:47:59.991 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.990+0000 7fbb53400640 1 -- 172.21.15.187:0/3571948164 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb4c007740 con 0x7fbb54162460 2024-05-26T22:47:59.991 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.991+0000 7fbb53400640 1 -- 172.21.15.187:0/3571948164 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb4c0078e0 con 0x7fbb54162460 2024-05-26T22:47:59.991 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.991+0000 7fbb53400640 1 -- 172.21.15.187:0/3571948164 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb4c007cb0 con 0x7fbb54162460 2024-05-26T22:47:59.991 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.991+0000 7fbb59a00640 1 -- 172.21.15.187:0/3571948164 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbb54162460 msgr2=0x7fbb5406b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:47:59.992 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.991+0000 7fbb59a00640 1 --2- 172.21.15.187:0/3571948164 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbb54162460 0x7fbb5406b850 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c009d90 tx=0x7fbb4c02a660 comp rx=0 tx=0).stop 2024-05-26T22:47:59.992 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.991+0000 7fbb59a00640 1 -- 172.21.15.187:0/3571948164 shutdown_connections 2024-05-26T22:47:59.992 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.991+0000 7fbb59a00640 1 --2- 172.21.15.187:0/3571948164 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbb54162460 0x7fbb5406b850 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:47:59.992 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.991+0000 7fbb59a00640 1 --2- 172.21.15.187:0/3571948164 >> v2:172.21.15.187:3300/0 conn(0x7fbb54161fd0 0x7fbb5406b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:47:59.992 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.991+0000 7fbb59a00640 1 -- 172.21.15.187:0/3571948164 >> 172.21.15.187:0/3571948164 conn(0x7fbb540b2c70 msgr2=0x7fbb540b3080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:47:59.992 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.991+0000 7fbb59a00640 1 -- 172.21.15.187:0/3571948164 shutdown_connections 2024-05-26T22:47:59.992 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.991+0000 7fbb59a00640 1 -- 172.21.15.187:0/3571948164 wait complete. 2024-05-26T22:47:59.993 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.993+0000 7fbb59a00640 1 Processor -- start 2024-05-26T22:47:59.993 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.993+0000 7fbb59a00640 1 -- start start 2024-05-26T22:47:59.995 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.994+0000 7fbb59a00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb54144a70 0x7fbb54145ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:47:59.995 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.994+0000 7fbb59a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb54161fd0 0x7fbb54145270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:47:59.996 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.994+0000 7fbb59a00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbb54162460 0x7fbb54159870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:47:59.996 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.994+0000 7fbb59a00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fbb5406fa00 con 0x7fbb54161fd0 2024-05-26T22:47:59.996 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.994+0000 7fbb59a00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fbb5406f880 con 0x7fbb54144a70 2024-05-26T22:47:59.996 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.994+0000 7fbb59a00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fbb540cf3f0 con 0x7fbb54162460 2024-05-26T22:47:59.996 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.994+0000 7fbb53e00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb54144a70 0x7fbb54145ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:47:59.996 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.995+0000 7fbb59000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbb54162460 0x7fbb54159870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:47:59.996 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.995+0000 7fbb53e00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb54144a70 0x7fbb54145ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.187:36428/0 (socket says 172.21.15.187:36428) 2024-05-26T22:47:59.996 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.995+0000 7fbb53e00640 1 -- 172.21.15.187:0/1647386610 learned_addr learned my addr 172.21.15.187:0/1647386610 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-05-26T22:47:59.996 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.995+0000 7fbb4be00640 1 --2- 172.21.15.187:0/1647386610 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb54161fd0 0x7fbb54145270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:47:59.996 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.995+0000 7fbb59000640 1 -- 172.21.15.187:0/1647386610 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb54144a70 msgr2=0x7fbb54145ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:47:59.996 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.995+0000 7fbb59000640 1 --2- 172.21.15.187:0/1647386610 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb54144a70 0x7fbb54145ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:47:59.996 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.995+0000 7fbb59000640 1 -- 172.21.15.187:0/1647386610 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb54161fd0 msgr2=0x7fbb54145270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:47:59.997 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.995+0000 7fbb59000640 1 --2- 172.21.15.187:0/1647386610 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb54161fd0 0x7fbb54145270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:47:59.997 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.995+0000 7fbb59000640 1 -- 172.21.15.187:0/1647386610 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb540ccc80 con 0x7fbb54162460 2024-05-26T22:47:59.997 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.995+0000 7fbb59000640 1 --2- 172.21.15.187:0/1647386610 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbb54162460 0x7fbb54159870 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c031000 tx=0x7fbb4c004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:47:59.997 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.996+0000 7fbb50c00640 1 -- 172.21.15.187:0/1647386610 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb4c03f070 con 0x7fbb54162460 2024-05-26T22:47:59.997 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.996+0000 7fbb50c00640 1 -- 172.21.15.187:0/1647386610 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb4c031c00 con 0x7fbb54162460 2024-05-26T22:47:59.997 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.996+0000 7fbb50c00640 1 -- 172.21.15.187:0/1647386610 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb4c009450 con 0x7fbb54162460 2024-05-26T22:47:59.997 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.996+0000 7fbb59a00640 1 -- 172.21.15.187:0/1647386610 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb5413d080 con 0x7fbb54162460 2024-05-26T22:47:59.997 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.996+0000 7fbb59a00640 1 -- 172.21.15.187:0/1647386610 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb540ccef0 con 0x7fbb54162460 2024-05-26T22:47:59.998 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.996+0000 7fbb50c00640 1 -- 172.21.15.187:0/1647386610 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fbb4c044460 con 0x7fbb54162460 2024-05-26T22:47:59.998 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.996+0000 7fbb50c00640 1 -- 172.21.15.187:0/1647386610 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7fbb4c044cd0 con 0x7fbb54162460 2024-05-26T22:47:59.998 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.997+0000 7fbb59a00640 1 -- 172.21.15.187:0/1647386610 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb540694e0 con 0x7fbb54162460 2024-05-26T22:47:59.999 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:47:59.999+0000 7fbb50c00640 1 -- 172.21.15.187:0/1647386610 <== mon.2 v2:172.21.15.187:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fbb4c044750 con 0x7fbb54162460 2024-05-26T22:48:00.078 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.078+0000 7fbb59a00640 1 -- 172.21.15.187:0/1647386610 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "osd new", "uuid": "7513ba69-dd00-4a35-b190-9e8cc3b2bdc8", "id": 5} v 0) -- 0x7fbb5406c110 con 0x7fbb54162460 2024-05-26T22:48:00.079 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-05-26T22:48:00.080+0000 7f2d5eb2f040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-26T22:48:00.086 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.086+0000 7fbb50c00640 1 -- 172.21.15.187:0/1647386610 <== mon.2 v2:172.21.15.187:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7513ba69-dd00-4a35-b190-9e8cc3b2bdc8", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7fbb4c044930 con 0x7fbb54162460 2024-05-26T22:48:00.086 INFO:teuthology.orchestra.run.smithi187.stdout:5 2024-05-26T22:48:00.087 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.087+0000 7fbb59a00640 1 -- 172.21.15.187:0/1647386610 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbb54162460 msgr2=0x7fbb54159870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:00.087 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.087+0000 7fbb59a00640 1 --2- 172.21.15.187:0/1647386610 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbb54162460 0x7fbb54159870 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c031000 tx=0x7fbb4c004a40 comp rx=0 tx=0).stop 2024-05-26T22:48:00.088 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.087+0000 7fbb59a00640 1 -- 172.21.15.187:0/1647386610 shutdown_connections 2024-05-26T22:48:00.088 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.087+0000 7fbb59a00640 1 --2- 172.21.15.187:0/1647386610 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbb54162460 0x7fbb54159870 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:00.088 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.087+0000 7fbb59a00640 1 --2- 172.21.15.187:0/1647386610 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb54161fd0 0x7fbb54145270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:00.088 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.087+0000 7fbb59a00640 1 --2- 172.21.15.187:0/1647386610 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb54144a70 0x7fbb54145ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:00.088 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.087+0000 7fbb59a00640 1 -- 172.21.15.187:0/1647386610 >> 172.21.15.187:0/1647386610 conn(0x7fbb540b2c70 msgr2=0x7fbb5406b310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:00.088 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.088+0000 7fbb59a00640 1 -- 172.21.15.187:0/1647386610 shutdown_connections 2024-05-26T22:48:00.088 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.088+0000 7fbb59a00640 1 -- 172.21.15.187:0/1647386610 wait complete. 2024-05-26T22:48:00.104 DEBUG:teuthology.orchestra.run.smithi187:> sudo ceph --cluster ceph osd new 50081810-c093-4d2f-a4d7-b7c695c5400d 6 2024-05-26T22:48:00.140 INFO:tasks.ceph.mgr.y.smithi131.stderr:2024-05-26T22:48:00.140+0000 7f977c60d040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-26T22:48:00.283 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.283+0000 7f4a82800640 1 Processor -- start 2024-05-26T22:48:00.283 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.283+0000 7f4a82800640 1 -- start start 2024-05-26T22:48:00.285 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.285+0000 7f4a82800640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f4a7c0b5030 0x7f4a7c168f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:00.285 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.285+0000 7f4a82800640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a7c0d2780 con 0x7f4a7c0b3430 2024-05-26T22:48:00.285 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.285+0000 7f4a82800640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a7c0d2a60 con 0x7f4a7c15e5b0 2024-05-26T22:48:00.285 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.285+0000 7f4a82800640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f4a7c0d2d40 con 0x7f4a7c0b5030 2024-05-26T22:48:00.285 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.285+0000 7f4a81400640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f4a7c15e5b0 0x7f4a7c0ced40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.187:53434/0 (socket says 172.21.15.187:53434) 2024-05-26T22:48:00.285 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.285+0000 7f4a81400640 1 -- 172.21.15.187:0/286387299 learned_addr learned my addr 172.21.15.187:0/286387299 (peer_addr_for_me v1:172.21.15.187:0/0) 2024-05-26T22:48:00.287 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.285+0000 7f4a81e00640 1 --2- 172.21.15.187:0/286387299 >> v2:172.21.15.131:3300/0 conn(0x7f4a7c0b5030 0x7f4a7c168f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:00.287 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.285+0000 7f4a7be00640 1 -- 172.21.15.187:0/286387299 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3081942566 0 0) 0x7f4a7c0d2a60 con 0x7f4a7c15e5b0 2024-05-26T22:48:00.287 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.285+0000 7f4a7be00640 1 -- 172.21.15.187:0/286387299 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4a64003520 con 0x7f4a7c15e5b0 2024-05-26T22:48:00.287 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.286+0000 7f4a7be00640 1 -- 172.21.15.187:0/286387299 <== mon.1 v1:172.21.15.187:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 457645674 0 0) 0x7f4a64003520 con 0x7f4a7c15e5b0 2024-05-26T22:48:00.287 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.286+0000 7f4a7be00640 1 -- 172.21.15.187:0/286387299 >> v2:172.21.15.131:3300/0 conn(0x7f4a7c0b5030 msgr2=0x7f4a7c168f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:00.288 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.286+0000 7f4a7be00640 1 --2- 172.21.15.187:0/286387299 >> v2:172.21.15.131:3300/0 conn(0x7f4a7c0b5030 0x7f4a7c168f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:00.288 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.286+0000 7f4a7be00640 1 -- 172.21.15.187:0/286387299 >> v1:172.21.15.131:6789/0 conn(0x7f4a7c0b3430 legacy=0x7f4a7c0cf450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:00.288 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.286+0000 7f4a7be00640 1 -- 172.21.15.187:0/286387299 --> v1:172.21.15.187:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a7c0d3020 con 0x7f4a7c15e5b0 2024-05-26T22:48:00.288 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.286+0000 7f4a7be00640 1 -- 172.21.15.187:0/286387299 <== mon.1 v1:172.21.15.187:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7f4a70001670 con 0x7f4a7c15e5b0 2024-05-26T22:48:00.288 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.286+0000 7f4a7be00640 1 -- 172.21.15.187:0/286387299 >> v1:172.21.15.187:6789/0 conn(0x7f4a7c15e5b0 legacy=0x7f4a7c0ced40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:00.288 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.286+0000 7f4a7be00640 1 --2- 172.21.15.187:0/286387299 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4a640039b0 0x7f4a64003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:00.288 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.286+0000 7f4a7be00640 1 --2- 172.21.15.187:0/286387299 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a64004800 0x7f4a64004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:00.288 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.286+0000 7f4a7be00640 1 --2- 172.21.15.187:0/286387299 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4a640051c0 0x7f4a6400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:00.288 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.286+0000 7f4a7be00640 1 -- 172.21.15.187:0/286387299 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f4a7c0d2d40 con 0x7f4a64004800 2024-05-26T22:48:00.288 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.286+0000 7f4a7be00640 1 -- 172.21.15.187:0/286387299 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4a6400c160 con 0x7f4a640039b0 2024-05-26T22:48:00.288 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.286+0000 7f4a7be00640 1 -- 172.21.15.187:0/286387299 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f4a6400c360 con 0x7f4a640051c0 2024-05-26T22:48:00.289 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.286+0000 7f4a81400640 1 --2- 172.21.15.187:0/286387299 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4a640039b0 0x7f4a64003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:00.289 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.286+0000 7f4a81e00640 1 --2- 172.21.15.187:0/286387299 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4a640051c0 0x7f4a6400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:00.289 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.286+0000 7f4a80a00640 1 --2- 172.21.15.187:0/286387299 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a64004800 0x7f4a64004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:00.289 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.287+0000 7f4a81400640 1 -- 172.21.15.187:0/286387299 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4a640051c0 msgr2=0x7f4a6400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:00.289 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.287+0000 7f4a81400640 1 --2- 172.21.15.187:0/286387299 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4a640051c0 0x7f4a6400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:00.289 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.287+0000 7f4a81400640 1 -- 172.21.15.187:0/286387299 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a64004800 msgr2=0x7f4a64004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:00.289 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.287+0000 7f4a81400640 1 --2- 172.21.15.187:0/286387299 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a64004800 0x7f4a64004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:00.289 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.287+0000 7f4a81400640 1 -- 172.21.15.187:0/286387299 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a64003520 con 0x7f4a640039b0 2024-05-26T22:48:00.289 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.287+0000 7f4a81e00640 1 --2- 172.21.15.187:0/286387299 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4a640051c0 0x7f4a6400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-26T22:48:00.289 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.287+0000 7f4a81e00640 1 -- 172.21.15.187:0/286387299 reap_dead start 2024-05-26T22:48:00.290 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.288+0000 7f4a81400640 1 --2- 172.21.15.187:0/286387299 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4a640039b0 0x7f4a64003dd0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f4a7000c210 tx=0x7f4a64005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d91eb00076f290d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:00.290 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.288+0000 7f4a7be00640 1 -- 172.21.15.187:0/286387299 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a70007d30 con 0x7f4a640039b0 2024-05-26T22:48:00.291 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.288+0000 7f4a7be00640 1 -- 172.21.15.187:0/286387299 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a70005ad0 con 0x7f4a640039b0 2024-05-26T22:48:00.291 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.288+0000 7f4a7be00640 1 -- 172.21.15.187:0/286387299 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a70007070 con 0x7f4a640039b0 2024-05-26T22:48:00.291 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.289+0000 7f4a82800640 1 -- 172.21.15.187:0/286387299 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4a640039b0 msgr2=0x7f4a64003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:00.291 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.289+0000 7f4a82800640 1 --2- 172.21.15.187:0/286387299 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4a640039b0 0x7f4a64003dd0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f4a7000c210 tx=0x7f4a64005830 comp rx=0 tx=0).stop 2024-05-26T22:48:00.291 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.289+0000 7f4a82800640 1 -- 172.21.15.187:0/286387299 shutdown_connections 2024-05-26T22:48:00.291 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.289+0000 7f4a82800640 1 --2- 172.21.15.187:0/286387299 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4a640039b0 0x7f4a64003dd0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:00.291 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.289+0000 7f4a82800640 1 -- 172.21.15.187:0/286387299 >> 172.21.15.187:0/286387299 conn(0x7f4a7c09d4d0 msgr2=0x7f4a7c0d41a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:00.291 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.289+0000 7f4a82800640 1 -- 172.21.15.187:0/286387299 shutdown_connections 2024-05-26T22:48:00.291 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.289+0000 7f4a82800640 1 -- 172.21.15.187:0/286387299 wait complete. 2024-05-26T22:48:00.291 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.291+0000 7f4a82800640 1 Processor -- start 2024-05-26T22:48:00.291 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.291+0000 7f4a82800640 1 -- start start 2024-05-26T22:48:00.293 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.292+0000 7f4a82800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4a640039b0 0x7f4a7c1538f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:00.294 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.292+0000 7f4a82800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a7c0cda90 0x7f4a7c1f3e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:00.294 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.292+0000 7f4a82800640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4a7c0b8580 0x7f4a7c0b8a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:00.294 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.292+0000 7f4a82800640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f4a7c0b41a0 con 0x7f4a7c0cda90 2024-05-26T22:48:00.294 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.292+0000 7f4a82800640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4a7c0b4020 con 0x7f4a7c0b8580 2024-05-26T22:48:00.294 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.292+0000 7f4a82800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f4a7c0b4320 con 0x7f4a640039b0 2024-05-26T22:48:00.294 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.292+0000 7f4a81400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4a640039b0 0x7f4a7c1538f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:00.294 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.292+0000 7f4a81400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4a640039b0 0x7f4a7c1538f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.187:34414/0 (socket says 172.21.15.187:34414) 2024-05-26T22:48:00.294 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.292+0000 7f4a81400640 1 -- 172.21.15.187:0/1144635455 learned_addr learned my addr 172.21.15.187:0/1144635455 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-05-26T22:48:00.294 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.292+0000 7f4a81e00640 1 --2- 172.21.15.187:0/1144635455 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4a7c0b8580 0x7f4a7c0b8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:00.294 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.292+0000 7f4a80a00640 1 --2- 172.21.15.187:0/1144635455 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a7c0cda90 0x7f4a7c1f3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:00.294 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.293+0000 7f4a81400640 1 -- 172.21.15.187:0/1144635455 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4a7c0b8580 msgr2=0x7f4a7c0b8a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:00.294 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.293+0000 7f4a81400640 1 --2- 172.21.15.187:0/1144635455 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4a7c0b8580 0x7f4a7c0b8a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:00.295 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.293+0000 7f4a81400640 1 -- 172.21.15.187:0/1144635455 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a7c0cda90 msgr2=0x7f4a7c1f3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:00.295 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.293+0000 7f4a81400640 1 --2- 172.21.15.187:0/1144635455 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a7c0cda90 0x7f4a7c1f3e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:00.295 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.293+0000 7f4a81400640 1 -- 172.21.15.187:0/1144635455 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a7c1f43a0 con 0x7f4a640039b0 2024-05-26T22:48:00.295 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.293+0000 7f4a81400640 1 --2- 172.21.15.187:0/1144635455 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4a640039b0 0x7f4a7c1538f0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4a70001200 tx=0x7f4a70001a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:00.295 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.293+0000 7f4a79600640 1 -- 172.21.15.187:0/1144635455 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a70037040 con 0x7f4a640039b0 2024-05-26T22:48:00.295 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.293+0000 7f4a79600640 1 -- 172.21.15.187:0/1144635455 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a70041070 con 0x7f4a640039b0 2024-05-26T22:48:00.295 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.293+0000 7f4a79600640 1 -- 172.21.15.187:0/1144635455 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a700047e0 con 0x7f4a640039b0 2024-05-26T22:48:00.295 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.293+0000 7f4a82800640 1 -- 172.21.15.187:0/1144635455 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a7c14ab80 con 0x7f4a640039b0 2024-05-26T22:48:00.295 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.293+0000 7f4a82800640 1 -- 172.21.15.187:0/1144635455 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a7c0c9240 con 0x7f4a640039b0 2024-05-26T22:48:00.295 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.294+0000 7f4a79600640 1 -- 172.21.15.187:0/1144635455 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4a70004d10 con 0x7f4a640039b0 2024-05-26T22:48:00.295 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.294+0000 7f4a79600640 1 -- 172.21.15.187:0/1144635455 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7f4a70032d40 con 0x7f4a640039b0 2024-05-26T22:48:00.296 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.295+0000 7f4a82800640 1 -- 172.21.15.187:0/1144635455 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a7c0694e0 con 0x7f4a640039b0 2024-05-26T22:48:00.296 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.297+0000 7f4a79600640 1 -- 172.21.15.187:0/1144635455 <== mon.2 v2:172.21.15.187:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f4a70032750 con 0x7f4a640039b0 2024-05-26T22:48:00.377 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.377+0000 7f4a82800640 1 -- 172.21.15.187:0/1144635455 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "osd new", "uuid": "50081810-c093-4d2f-a4d7-b7c695c5400d", "id": 6} v 0) -- 0x7f4a7c0b5540 con 0x7f4a640039b0 2024-05-26T22:48:00.390 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.390+0000 7f4a79600640 1 -- 172.21.15.187:0/1144635455 <== mon.2 v2:172.21.15.187:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "50081810-c093-4d2f-a4d7-b7c695c5400d", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7f4a70032930 con 0x7f4a640039b0 2024-05-26T22:48:00.390 INFO:teuthology.orchestra.run.smithi187.stdout:6 2024-05-26T22:48:00.391 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.391+0000 7f4a82800640 1 -- 172.21.15.187:0/1144635455 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4a640039b0 msgr2=0x7f4a7c1538f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:00.391 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.391+0000 7f4a82800640 1 --2- 172.21.15.187:0/1144635455 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4a640039b0 0x7f4a7c1538f0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4a70001200 tx=0x7f4a70001a40 comp rx=0 tx=0).stop 2024-05-26T22:48:00.391 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.391+0000 7f4a82800640 1 -- 172.21.15.187:0/1144635455 shutdown_connections 2024-05-26T22:48:00.391 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.391+0000 7f4a82800640 1 --2- 172.21.15.187:0/1144635455 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4a7c0b8580 0x7f4a7c0b8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:00.391 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.391+0000 7f4a82800640 1 --2- 172.21.15.187:0/1144635455 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a7c0cda90 0x7f4a7c1f3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:00.391 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.391+0000 7f4a82800640 1 --2- 172.21.15.187:0/1144635455 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4a640039b0 0x7f4a7c1538f0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:00.391 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.391+0000 7f4a82800640 1 -- 172.21.15.187:0/1144635455 >> 172.21.15.187:0/1144635455 conn(0x7f4a7c09d4d0 msgr2=0x7f4a7c06cdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:00.392 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.391+0000 7f4a82800640 1 -- 172.21.15.187:0/1144635455 shutdown_connections 2024-05-26T22:48:00.392 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.391+0000 7f4a82800640 1 -- 172.21.15.187:0/1144635455 wait complete. 2024-05-26T22:48:00.408 DEBUG:teuthology.orchestra.run.smithi187:> sudo ceph --cluster ceph osd new 775dae05-54f5-4ad4-bbce-92863c6ed2ff 7 2024-05-26T22:48:00.454 INFO:tasks.ceph.mgr.y.smithi131.stderr:2024-05-26T22:48:00.454+0000 7f977c60d040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-26T22:48:00.514 INFO:tasks.ceph.mgr.y.smithi131.stderr:2024-05-26T22:48:00.515+0000 7f977c60d040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-26T22:48:00.566 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-05-26T22:48:00.566+0000 7f2d5eb2f040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-26T22:48:00.592 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.591+0000 7f2a1fe00640 1 Processor -- start 2024-05-26T22:48:00.592 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.591+0000 7f2a1fe00640 1 -- start start 2024-05-26T22:48:00.593 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.593+0000 7f2a1fe00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2a200b3430 0x7f2a200cf450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:00.593 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.593+0000 7f2a1fe00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f2a200cf990 0x7f2a200b4020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:00.593 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.593+0000 7f2a1fe00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a200d2780 con 0x7f2a2015e5b0 2024-05-26T22:48:00.593 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.593+0000 7f2a1fe00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f2a200d2a60 con 0x7f2a200cf990 2024-05-26T22:48:00.593 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.593+0000 7f2a1fe00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f2a200d2d40 con 0x7f2a200b3430 2024-05-26T22:48:00.593 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.593+0000 7f2a1e000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2a200b3430 0x7f2a200cf450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:00.593 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.593+0000 7f2a1e000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2a200b3430 0x7f2a200cf450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.187:34416/0 (socket says 172.21.15.187:34416) 2024-05-26T22:48:00.593 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.593+0000 7f2a1e000640 1 -- 172.21.15.187:0/1961004272 learned_addr learned my addr 172.21.15.187:0/1961004272 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-05-26T22:48:00.594 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.594+0000 7f2a1e000640 1 -- 172.21.15.187:0/1961004272 >> v2:172.21.15.187:3300/0 conn(0x7f2a200cf990 msgr2=0x7f2a200b4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:00.594 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.594+0000 7f2a1e000640 1 --2- 172.21.15.187:0/1961004272 >> v2:172.21.15.187:3300/0 conn(0x7f2a200cf990 0x7f2a200b4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:00.594 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.594+0000 7f2a1e000640 1 -- 172.21.15.187:0/1961004272 >> v1:172.21.15.131:6789/0 conn(0x7f2a2015e5b0 legacy=0x7f2a200ced40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:00.594 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.594+0000 7f2a1e000640 1 -- 172.21.15.187:0/1961004272 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a200d3020 con 0x7f2a200b3430 2024-05-26T22:48:00.595 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.595+0000 7f2a1e000640 1 --2- 172.21.15.187:0/1961004272 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2a200b3430 0x7f2a200cf450 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c009870 tx=0x7f2a0c02a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6ca258b27a7b374 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:00.595 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.595+0000 7f2a1d600640 1 -- 172.21.15.187:0/1961004272 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a0c037070 con 0x7f2a200b3430 2024-05-26T22:48:00.595 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.595+0000 7f2a1d600640 1 -- 172.21.15.187:0/1961004272 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a0c032440 con 0x7f2a200b3430 2024-05-26T22:48:00.595 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.595+0000 7f2a1d600640 1 -- 172.21.15.187:0/1961004272 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a0c032810 con 0x7f2a200b3430 2024-05-26T22:48:00.596 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.596+0000 7f2a1fe00640 1 -- 172.21.15.187:0/1961004272 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2a200b3430 msgr2=0x7f2a200cf450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:00.596 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.596+0000 7f2a1fe00640 1 --2- 172.21.15.187:0/1961004272 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2a200b3430 0x7f2a200cf450 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c009870 tx=0x7f2a0c02a1c0 comp rx=0 tx=0).stop 2024-05-26T22:48:00.596 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.596+0000 7f2a1fe00640 1 -- 172.21.15.187:0/1961004272 shutdown_connections 2024-05-26T22:48:00.597 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.596+0000 7f2a1fe00640 1 --2- 172.21.15.187:0/1961004272 >> v2:172.21.15.187:3300/0 conn(0x7f2a200cf990 0x7f2a200b4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:00.597 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.596+0000 7f2a1fe00640 1 --2- 172.21.15.187:0/1961004272 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2a200b3430 0x7f2a200cf450 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:00.597 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.596+0000 7f2a1fe00640 1 -- 172.21.15.187:0/1961004272 >> 172.21.15.187:0/1961004272 conn(0x7f2a2009d4d0 msgr2=0x7f2a200d41a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:00.597 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.596+0000 7f2a1fe00640 1 -- 172.21.15.187:0/1961004272 shutdown_connections 2024-05-26T22:48:00.597 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.596+0000 7f2a1fe00640 1 -- 172.21.15.187:0/1961004272 wait complete. 2024-05-26T22:48:00.597 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.598+0000 7f2a1fe00640 1 Processor -- start 2024-05-26T22:48:00.597 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.598+0000 7f2a1fe00640 1 -- start start 2024-05-26T22:48:00.600 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.599+0000 7f2a1fe00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2a200b3430 0x7f2a200cbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:00.600 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.599+0000 7f2a1fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a200cf990 0x7f2a201f3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:00.600 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.599+0000 7f2a1fe00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2a20159f90 0x7f2a201f7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:00.600 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.599+0000 7f2a1fe00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f2a200b4560 con 0x7f2a200cf990 2024-05-26T22:48:00.600 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.599+0000 7f2a1fe00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2a200d19f0 con 0x7f2a200b3430 2024-05-26T22:48:00.601 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.599+0000 7f2a1fe00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f2a200b46e0 con 0x7f2a20159f90 2024-05-26T22:48:00.601 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.599+0000 7f2a1ea00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2a200b3430 0x7f2a200cbcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:00.601 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.599+0000 7f2a1ea00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2a200b3430 0x7f2a200cbcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.187:36470/0 (socket says 172.21.15.187:36470) 2024-05-26T22:48:00.601 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.599+0000 7f2a1ea00640 1 -- 172.21.15.187:0/3084466720 learned_addr learned my addr 172.21.15.187:0/3084466720 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-05-26T22:48:00.601 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.599+0000 7f2a1f400640 1 --2- 172.21.15.187:0/3084466720 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2a20159f90 0x7f2a201f7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:00.601 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.599+0000 7f2a1e000640 1 --2- 172.21.15.187:0/3084466720 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a200cf990 0x7f2a201f3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:00.601 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.600+0000 7f2a1f400640 1 -- 172.21.15.187:0/3084466720 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2a200b3430 msgr2=0x7f2a200cbcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:00.601 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.600+0000 7f2a1f400640 1 --2- 172.21.15.187:0/3084466720 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2a200b3430 0x7f2a200cbcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:00.601 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.600+0000 7f2a1f400640 1 -- 172.21.15.187:0/3084466720 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a200cf990 msgr2=0x7f2a201f3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:00.601 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.600+0000 7f2a1f400640 1 --2- 172.21.15.187:0/3084466720 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a200cf990 0x7f2a201f3ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:00.601 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.600+0000 7f2a1f400640 1 -- 172.21.15.187:0/3084466720 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a201f4650 con 0x7f2a20159f90 2024-05-26T22:48:00.601 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.600+0000 7f2a1ea00640 1 --2- 172.21.15.187:0/3084466720 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2a200b3430 0x7f2a200cbcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-26T22:48:00.602 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.600+0000 7f2a1f400640 1 --2- 172.21.15.187:0/3084466720 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2a20159f90 0x7f2a201f7bd0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f2a1800be20 tx=0x7f2a1800bf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:00.602 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.600+0000 7f2a17400640 1 -- 172.21.15.187:0/3084466720 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a18019070 con 0x7f2a20159f90 2024-05-26T22:48:00.602 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.600+0000 7f2a17400640 1 -- 172.21.15.187:0/3084466720 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a18014440 con 0x7f2a20159f90 2024-05-26T22:48:00.602 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.600+0000 7f2a17400640 1 -- 172.21.15.187:0/3084466720 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a18014830 con 0x7f2a20159f90 2024-05-26T22:48:00.602 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.600+0000 7f2a1fe00640 1 -- 172.21.15.187:0/3084466720 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a201f4930 con 0x7f2a20159f90 2024-05-26T22:48:00.602 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.600+0000 7f2a1fe00640 1 -- 172.21.15.187:0/3084466720 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a201f76b0 con 0x7f2a20159f90 2024-05-26T22:48:00.602 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.601+0000 7f2a17400640 1 -- 172.21.15.187:0/3084466720 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f2a1801c070 con 0x7f2a20159f90 2024-05-26T22:48:00.602 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.601+0000 7f2a17400640 1 -- 172.21.15.187:0/3084466720 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7f2a1801da10 con 0x7f2a20159f90 2024-05-26T22:48:00.602 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.601+0000 7f2a1fe00640 1 -- 172.21.15.187:0/3084466720 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a201f74a0 con 0x7f2a20159f90 2024-05-26T22:48:00.605 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.605+0000 7f2a17400640 1 -- 172.21.15.187:0/3084466720 <== mon.2 v2:172.21.15.187:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f2a18023020 con 0x7f2a20159f90 2024-05-26T22:48:00.647 INFO:tasks.ceph.mgr.y.smithi131.stderr:2024-05-26T22:48:00.647+0000 7f977c60d040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-26T22:48:00.684 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.684+0000 7f2a1fe00640 1 -- 172.21.15.187:0/3084466720 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "osd new", "uuid": "775dae05-54f5-4ad4-bbce-92863c6ed2ff", "id": 7} v 0) -- 0x7f2a200694e0 con 0x7f2a20159f90 2024-05-26T22:48:00.694 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.694+0000 7f2a17400640 1 -- 172.21.15.187:0/3084466720 <== mon.2 v2:172.21.15.187:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "775dae05-54f5-4ad4-bbce-92863c6ed2ff", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7f2a1801e440 con 0x7f2a20159f90 2024-05-26T22:48:00.694 INFO:teuthology.orchestra.run.smithi187.stdout:7 2024-05-26T22:48:00.696 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.696+0000 7f2a1fe00640 1 -- 172.21.15.187:0/3084466720 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2a20159f90 msgr2=0x7f2a201f7bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:00.696 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.696+0000 7f2a1fe00640 1 --2- 172.21.15.187:0/3084466720 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2a20159f90 0x7f2a201f7bd0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f2a1800be20 tx=0x7f2a1800bf10 comp rx=0 tx=0).stop 2024-05-26T22:48:00.696 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.696+0000 7f2a1fe00640 1 -- 172.21.15.187:0/3084466720 shutdown_connections 2024-05-26T22:48:00.696 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.696+0000 7f2a1fe00640 1 --2- 172.21.15.187:0/3084466720 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2a20159f90 0x7f2a201f7bd0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:00.696 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.696+0000 7f2a1fe00640 1 --2- 172.21.15.187:0/3084466720 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a200cf990 0x7f2a201f3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:00.696 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.696+0000 7f2a1fe00640 1 --2- 172.21.15.187:0/3084466720 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2a200b3430 0x7f2a200cbcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:00.696 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.696+0000 7f2a1fe00640 1 -- 172.21.15.187:0/3084466720 >> 172.21.15.187:0/3084466720 conn(0x7f2a2009d4d0 msgr2=0x7f2a200ced00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:00.696 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.696+0000 7f2a1fe00640 1 -- 172.21.15.187:0/3084466720 shutdown_connections 2024-05-26T22:48:00.696 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:00.696+0000 7f2a1fe00640 1 -- 172.21.15.187:0/3084466720 wait complete. 2024-05-26T22:48:00.711 INFO:tasks.ceph.osd.0:Restarting daemon 2024-05-26T22:48:00.711 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-05-26T22:48:00.718 INFO:tasks.ceph.osd.0:Started 2024-05-26T22:48:00.718 INFO:tasks.ceph.osd.1:Restarting daemon 2024-05-26T22:48:00.719 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-05-26T22:48:00.725 INFO:tasks.ceph.osd.1:Started 2024-05-26T22:48:00.725 INFO:tasks.ceph.osd.2:Restarting daemon 2024-05-26T22:48:00.725 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-05-26T22:48:00.731 INFO:tasks.ceph.osd.2:Started 2024-05-26T22:48:00.731 INFO:tasks.ceph.osd.3:Restarting daemon 2024-05-26T22:48:00.731 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-05-26T22:48:00.736 INFO:tasks.ceph.osd.3:Started 2024-05-26T22:48:00.736 INFO:tasks.ceph.osd.4:Restarting daemon 2024-05-26T22:48:00.736 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-05-26T22:48:00.743 INFO:tasks.ceph.osd.4:Started 2024-05-26T22:48:00.744 INFO:tasks.ceph.osd.5:Restarting daemon 2024-05-26T22:48:00.744 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-05-26T22:48:00.749 INFO:tasks.ceph.osd.5:Started 2024-05-26T22:48:00.750 INFO:tasks.ceph.osd.6:Restarting daemon 2024-05-26T22:48:00.750 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-05-26T22:48:00.755 INFO:tasks.ceph.osd.6:Started 2024-05-26T22:48:00.755 INFO:tasks.ceph.osd.7:Restarting daemon 2024-05-26T22:48:00.755 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-05-26T22:48:00.760 INFO:tasks.ceph.osd.7:Started 2024-05-26T22:48:00.760 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-26T22:48:00.927 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-05-26T22:48:00.927+0000 7f2d5eb2f040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-26T22:48:00.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:00.963+0000 7fea22c00640 1 Processor -- start 2024-05-26T22:48:00.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:00.964+0000 7fea22c00640 1 -- start start 2024-05-26T22:48:00.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:00.965+0000 7fea22c00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fea1c06e8c0 0x7fea1c15d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:00.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:00.965+0000 7fea22c00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fea1c0cc7f0 0x7fea1c0ccc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:00.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:00.965+0000 7fea22c00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea1c160620 con 0x7fea1c156750 2024-05-26T22:48:00.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:00.965+0000 7fea22c00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fea1c0cd1a0 con 0x7fea1c06e8c0 2024-05-26T22:48:00.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:00.965+0000 7fea22c00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fea1c15cbd0 con 0x7fea1c0cc7f0 2024-05-26T22:48:00.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:00.965+0000 7fea20e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fea1c06e8c0 0x7fea1c15d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:00.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:00.965+0000 7fea20e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fea1c06e8c0 0x7fea1c15d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33586/0 (socket says 172.21.15.131:33586) 2024-05-26T22:48:00.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:00.965+0000 7fea20e00640 1 -- 172.21.15.131:0/3002625475 learned_addr learned my addr 172.21.15.131:0/3002625475 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:00.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:00.965+0000 7fea22200640 1 --2- 172.21.15.131:0/3002625475 >> v2:172.21.15.187:3300/0 conn(0x7fea1c0cc7f0 0x7fea1c0ccc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:00.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:00.966+0000 7fea20e00640 1 -- 172.21.15.131:0/3002625475 >> v2:172.21.15.187:3300/0 conn(0x7fea1c0cc7f0 msgr2=0x7fea1c0ccc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:00.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:00.966+0000 7fea20e00640 1 --2- 172.21.15.131:0/3002625475 >> v2:172.21.15.187:3300/0 conn(0x7fea1c0cc7f0 0x7fea1c0ccc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:00.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:00.966+0000 7fea20e00640 1 -- 172.21.15.131:0/3002625475 >> v1:172.21.15.131:6789/0 conn(0x7fea1c156750 legacy=0x7fea1c15fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:00.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:00.966+0000 7fea20e00640 1 -- 172.21.15.131:0/3002625475 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea1c160900 con 0x7fea1c06e8c0 2024-05-26T22:48:00.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:00.967+0000 7fea20e00640 1 --2- 172.21.15.131:0/3002625475 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea1c06e8c0 0x7fea1c15d060 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fea1c15a320 tx=0x7fea0802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0204fcc72307f0f server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:00.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:00.967+0000 7fea1be00640 1 -- 172.21.15.131:0/3002625475 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea08037070 con 0x7fea1c06e8c0 2024-05-26T22:48:00.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:00.967+0000 7fea1be00640 1 -- 172.21.15.131:0/3002625475 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea0802ae10 con 0x7fea1c06e8c0 2024-05-26T22:48:00.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:00.967+0000 7fea1be00640 1 -- 172.21.15.131:0/3002625475 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea08032660 con 0x7fea1c06e8c0 2024-05-26T22:48:00.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:00.968+0000 7fea22c00640 1 -- 172.21.15.131:0/3002625475 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea1c06e8c0 msgr2=0x7fea1c15d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:00.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:00.968+0000 7fea22c00640 1 --2- 172.21.15.131:0/3002625475 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea1c06e8c0 0x7fea1c15d060 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fea1c15a320 tx=0x7fea0802a180 comp rx=0 tx=0).stop 2024-05-26T22:48:00.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:00.968+0000 7fea22c00640 1 -- 172.21.15.131:0/3002625475 shutdown_connections 2024-05-26T22:48:00.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:00.968+0000 7fea22c00640 1 --2- 172.21.15.131:0/3002625475 >> v2:172.21.15.187:3300/0 conn(0x7fea1c0cc7f0 0x7fea1c0ccc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:00.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:00.968+0000 7fea22c00640 1 --2- 172.21.15.131:0/3002625475 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea1c06e8c0 0x7fea1c15d060 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:00.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:00.968+0000 7fea22c00640 1 -- 172.21.15.131:0/3002625475 >> 172.21.15.131:0/3002625475 conn(0x7fea1c0cff40 msgr2=0x7fea1c0d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:00.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:00.968+0000 7fea22c00640 1 -- 172.21.15.131:0/3002625475 shutdown_connections 2024-05-26T22:48:00.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:00.968+0000 7fea22c00640 1 -- 172.21.15.131:0/3002625475 wait complete. 2024-05-26T22:48:00.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:00.969+0000 7fea22c00640 1 Processor -- start 2024-05-26T22:48:00.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:00.970+0000 7fea22c00640 1 -- start start 2024-05-26T22:48:00.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:00.970+0000 7fea22c00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fea1c06e8c0 0x7fea1c131e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:00.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:00.970+0000 7fea22c00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fea1c0cc7f0 0x7fea1c156cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:00.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:00.970+0000 7fea22c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea1c156750 0x7fea1c155810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:00.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:00.970+0000 7fea22c00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fea1c156b30 con 0x7fea1c156750 2024-05-26T22:48:00.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:00.970+0000 7fea22c00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fea1c15c860 con 0x7fea1c06e8c0 2024-05-26T22:48:00.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:00.971+0000 7fea22c00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fea1c15d5a0 con 0x7fea1c0cc7f0 2024-05-26T22:48:00.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:00.971+0000 7fea22200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea1c156750 0x7fea1c155810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:00.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:00.971+0000 7fea20e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fea1c0cc7f0 0x7fea1c156cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:00.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:00.971+0000 7fea22200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea1c156750 0x7fea1c155810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33588/0 (socket says 172.21.15.131:33588) 2024-05-26T22:48:00.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:00.971+0000 7fea22200640 1 -- 172.21.15.131:0/384726921 learned_addr learned my addr 172.21.15.131:0/384726921 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:00.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:00.971+0000 7fea21800640 1 --2- 172.21.15.131:0/384726921 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fea1c06e8c0 0x7fea1c131e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:00.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:00.971+0000 7fea22200640 1 -- 172.21.15.131:0/384726921 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fea1c0cc7f0 msgr2=0x7fea1c156cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:00.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:00.971+0000 7fea22200640 1 --2- 172.21.15.131:0/384726921 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fea1c0cc7f0 0x7fea1c156cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:00.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:00.971+0000 7fea22200640 1 -- 172.21.15.131:0/384726921 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fea1c06e8c0 msgr2=0x7fea1c131e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:00.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:00.971+0000 7fea22200640 1 --2- 172.21.15.131:0/384726921 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fea1c06e8c0 0x7fea1c131e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:00.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:00.971+0000 7fea22200640 1 -- 172.21.15.131:0/384726921 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea1c1571f0 con 0x7fea1c156750 2024-05-26T22:48:00.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:00.972+0000 7fea22200640 1 --2- 172.21.15.131:0/384726921 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea1c156750 0x7fea1c155810 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fea1400efc0 tx=0x7fea1400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:00.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:00.972+0000 7fea19600640 1 -- 172.21.15.131:0/384726921 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea14018020 con 0x7fea1c156750 2024-05-26T22:48:00.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:00.972+0000 7fea19600640 1 -- 172.21.15.131:0/384726921 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea14014480 con 0x7fea1c156750 2024-05-26T22:48:00.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:00.972+0000 7fea19600640 1 -- 172.21.15.131:0/384726921 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea14014870 con 0x7fea1c156750 2024-05-26T22:48:00.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:00.972+0000 7fea22c00640 1 -- 172.21.15.131:0/384726921 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea1c158b60 con 0x7fea1c156750 2024-05-26T22:48:00.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:00.972+0000 7fea22c00640 1 -- 172.21.15.131:0/384726921 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea1c159120 con 0x7fea1c156750 2024-05-26T22:48:00.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:00.972+0000 7fea19600640 1 -- 172.21.15.131:0/384726921 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fea14005d10 con 0x7fea1c156750 2024-05-26T22:48:00.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:00.972+0000 7fea19600640 1 -- 172.21.15.131:0/384726921 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7fea14014da0 con 0x7fea1c156750 2024-05-26T22:48:00.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:00.973+0000 7fea22c00640 1 -- 172.21.15.131:0/384726921 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea1c0694e0 con 0x7fea1c156750 2024-05-26T22:48:00.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:00.976+0000 7fea19600640 1 -- 172.21.15.131:0/384726921 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fea14022070 con 0x7fea1c156750 2024-05-26T22:48:00.994 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-05-26T22:48:00.994+0000 7f2d5eb2f040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-26T22:48:01.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:01.039+0000 7fea22c00640 1 -- 172.21.15.131:0/384726921 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fea1c0ccd00 con 0x7fea1c156750 2024-05-26T22:48:01.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:01.040+0000 7fea19600640 1 -- 172.21.15.131:0/384726921 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7fea14010040 con 0x7fea1c156750 2024-05-26T22:48:01.040 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:48:01.040 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":10,"fsid":"46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f","created":"2024-05-26T22:47:57.288905+0000","modified":"2024-05-26T22:48:00.687341+0000","last_up_change":"0.000000","last_in_change":"2024-05-26T22:48:00.687341+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"814eaf62-a18d-494b-8e27-fd8fb88adea6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"42e66287-f6c6-4e26-af3c-6dfaa3536d27","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"0da34673-04a2-4fd0-b57a-0ff2b679b043","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"cf117a27-d6b2-43d3-ae52-b47988bf8eed","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"a092ccfc-be89-4a84-88d5-0f781fa715eb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"7513ba69-dd00-4a35-b190-9e8cc3b2bdc8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"50081810-c093-4d2f-a4d7-b7c695c5400d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"775dae05-54f5-4ad4-bbce-92863c6ed2ff","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-26T22:48:01.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:01.041+0000 7fea22c00640 1 -- 172.21.15.131:0/384726921 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea1c156750 msgr2=0x7fea1c155810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:01.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:01.041+0000 7fea22c00640 1 --2- 172.21.15.131:0/384726921 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea1c156750 0x7fea1c155810 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fea1400efc0 tx=0x7fea1400c490 comp rx=0 tx=0).stop 2024-05-26T22:48:01.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:01.042+0000 7fea22c00640 1 -- 172.21.15.131:0/384726921 shutdown_connections 2024-05-26T22:48:01.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:01.042+0000 7fea22c00640 1 --2- 172.21.15.131:0/384726921 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fea1c0cc7f0 0x7fea1c156cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:01.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:01.042+0000 7fea22c00640 1 --2- 172.21.15.131:0/384726921 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea1c156750 0x7fea1c155810 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:01.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:01.042+0000 7fea22c00640 1 --2- 172.21.15.131:0/384726921 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fea1c06e8c0 0x7fea1c131e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:01.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:01.042+0000 7fea22c00640 1 -- 172.21.15.131:0/384726921 >> 172.21.15.131:0/384726921 conn(0x7fea1c0cff40 msgr2=0x7fea1c142820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:01.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:01.042+0000 7fea22c00640 1 -- 172.21.15.131:0/384726921 shutdown_connections 2024-05-26T22:48:01.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:01.042+0000 7fea22c00640 1 -- 172.21.15.131:0/384726921 wait complete. 2024-05-26T22:48:01.053 INFO:tasks.ceph.mgr.y.smithi131.stderr:2024-05-26T22:48:01.051+0000 7f977c60d040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-26T22:48:01.057 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-05-26T22:48:01.058 INFO:tasks.ceph:Waiting for OSDs to come up 2024-05-26T22:48:01.109 INFO:tasks.ceph.mgr.y.smithi131.stderr:2024-05-26T22:48:01.109+0000 7f977c60d040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-26T22:48:01.125 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-05-26T22:48:01.125+0000 7f2d5eb2f040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-26T22:48:01.292 INFO:tasks.ceph.mgr.y.smithi131.stderr:2024-05-26T22:48:01.292+0000 7f977c60d040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-26T22:48:01.413 INFO:tasks.ceph.osd.0.smithi131.stderr:2024-05-26T22:48:01.414+0000 7f483ba2f640 -1 Falling back to public interface 2024-05-26T22:48:01.421 INFO:tasks.ceph.osd.3.smithi131.stderr:2024-05-26T22:48:01.422+0000 7fb779a3d640 -1 Falling back to public interface 2024-05-26T22:48:01.422 INFO:tasks.ceph.osd.5.smithi187.stderr:2024-05-26T22:48:01.422+0000 7f7c62ad9640 -1 Falling back to public interface 2024-05-26T22:48:01.423 INFO:tasks.ceph.osd.6.smithi187.stderr:2024-05-26T22:48:01.423+0000 7f61b0618640 -1 Falling back to public interface 2024-05-26T22:48:01.425 INFO:tasks.ceph.osd.5.smithi187.stderr:2024-05-26T22:48:01.425+0000 7f7c61600640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6809/0: (98) Address already in use 2024-05-26T22:48:01.427 INFO:tasks.ceph.osd.4.smithi187.stderr:2024-05-26T22:48:01.427+0000 7fee041cb640 -1 Falling back to public interface 2024-05-26T22:48:01.446 INFO:tasks.ceph.osd.2.smithi131.stderr:2024-05-26T22:48:01.447+0000 7efc6e661640 -1 Falling back to public interface 2024-05-26T22:48:01.454 INFO:tasks.ceph.osd.1.smithi131.stderr:2024-05-26T22:48:01.455+0000 7fcf75744640 -1 Falling back to public interface 2024-05-26T22:48:01.462 INFO:tasks.ceph.osd.7.smithi187.stderr:2024-05-26T22:48:01.460+0000 7fa2e900d640 -1 Falling back to public interface 2024-05-26T22:48:01.462 INFO:tasks.ceph.mgr.y.smithi131.stderr:2024-05-26T22:48:01.463+0000 7f977c60d040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-26T22:48:01.463 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-05-26T22:48:01.463+0000 7f2d5eb2f040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-26T22:48:01.523 INFO:tasks.ceph.mgr.y.smithi131.stderr:2024-05-26T22:48:01.524+0000 7f977c60d040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-26T22:48:01.525 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-05-26T22:48:01.525+0000 7f2d5eb2f040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-26T22:48:01.604 INFO:tasks.ceph.mgr.y.smithi131.stderr:2024-05-26T22:48:01.604+0000 7f977c60d040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-26T22:48:01.656 INFO:tasks.ceph.mgr.y.smithi131.stderr:2024-05-26T22:48:01.657+0000 7f977c60d040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-26T22:48:01.705 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-05-26T22:48:01.705+0000 7f2d5eb2f040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-26T22:48:01.865 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-05-26T22:48:01.873 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-05-26T22:48:01.874+0000 7f2d5eb2f040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-26T22:48:01.932 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-05-26T22:48:01.931+0000 7f2d5eb2f040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-26T22:48:02.013 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:02.014+0000 7f17f8e00640 1 Processor -- start 2024-05-26T22:48:02.014 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:02.014+0000 7f17f8e00640 1 -- start start 2024-05-26T22:48:02.016 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:02.015+0000 7f17f8e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f17f40b5d60 0x7f17f40b6140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:02.016 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:02.015+0000 7f17f8e00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f17f40d2340 0x7f17f40b9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:02.016 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:02.015+0000 7f17f8e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f17f40d3870 0x7f17f40d3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:02.016 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:02.015+0000 7f17f8e00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f17f40b6680 con 0x7f17f40d3870 2024-05-26T22:48:02.016 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:02.015+0000 7f17f8e00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f17f40d27d0 con 0x7f17f40d2340 2024-05-26T22:48:02.017 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:02.015+0000 7f17f8e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f17f40d2ab0 con 0x7f17f40b5d60 2024-05-26T22:48:02.017 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:02.015+0000 7f17f2800640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f17f40d3870 0x7f17f40d3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:02.017 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:02.015+0000 7f17f2800640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f17f40d3870 0x7f17f40d3cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33598/0 (socket says 172.21.15.131:33598) 2024-05-26T22:48:02.017 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:02.015+0000 7f17f2800640 1 -- 172.21.15.131:0/3486931054 learned_addr learned my addr 172.21.15.131:0/3486931054 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:02.017 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:02.015+0000 7f17ebe00640 1 --2- 172.21.15.131:0/3486931054 >> v2:172.21.15.187:3300/0 conn(0x7f17f40d2340 0x7f17f40b9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:02.017 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:02.015+0000 7f17f1e00640 1 --2- 172.21.15.131:0/3486931054 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f17f40b5d60 0x7f17f40b6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:02.017 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:02.016+0000 7f17f2800640 1 -- 172.21.15.131:0/3486931054 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f17f40b5d60 msgr2=0x7f17f40b6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:02.017 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:02.016+0000 7f17f2800640 1 --2- 172.21.15.131:0/3486931054 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f17f40b5d60 0x7f17f40b6140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:02.017 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:02.016+0000 7f17f2800640 1 -- 172.21.15.131:0/3486931054 >> v2:172.21.15.187:3300/0 conn(0x7f17f40d2340 msgr2=0x7f17f40b9bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:02.017 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:02.016+0000 7f17f2800640 1 --2- 172.21.15.131:0/3486931054 >> v2:172.21.15.187:3300/0 conn(0x7f17f40d2340 0x7f17f40b9bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:02.017 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:02.016+0000 7f17f2800640 1 -- 172.21.15.131:0/3486931054 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17f40d2d90 con 0x7f17f40d3870 2024-05-26T22:48:02.017 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:02.016+0000 7f17ebe00640 1 --2- 172.21.15.131:0/3486931054 >> v2:172.21.15.187:3300/0 conn(0x7f17f40d2340 0x7f17f40b9bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-26T22:48:02.018 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:02.017+0000 7f17f2800640 1 --2- 172.21.15.131:0/3486931054 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17f40d3870 0x7f17f40d3cc0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f17f40d5350 tx=0x7f17ec02a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19950f52abee42e3 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:02.018 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-05-26T22:48:02.015+0000 7f2d5eb2f040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-26T22:48:02.019 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:02.017+0000 7f17f1400640 1 -- 172.21.15.131:0/3486931054 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17ec036070 con 0x7f17f40d3870 2024-05-26T22:48:02.019 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:02.017+0000 7f17f1400640 1 -- 172.21.15.131:0/3486931054 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17ec031600 con 0x7f17f40d3870 2024-05-26T22:48:02.020 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:02.017+0000 7f17f1400640 1 -- 172.21.15.131:0/3486931054 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17ec0319d0 con 0x7f17f40d3870 2024-05-26T22:48:02.020 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:02.017+0000 7f17f8e00640 1 -- 172.21.15.131:0/3486931054 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17f40d3870 msgr2=0x7f17f40d3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:02.020 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:02.017+0000 7f17f8e00640 1 --2- 172.21.15.131:0/3486931054 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17f40d3870 0x7f17f40d3cc0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f17f40d5350 tx=0x7f17ec02a610 comp rx=0 tx=0).stop 2024-05-26T22:48:02.020 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:02.018+0000 7f17f8e00640 1 -- 172.21.15.131:0/3486931054 shutdown_connections 2024-05-26T22:48:02.020 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:02.018+0000 7f17f8e00640 1 --2- 172.21.15.131:0/3486931054 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17f40d3870 0x7f17f40d3cc0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:02.020 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:02.018+0000 7f17f8e00640 1 --2- 172.21.15.131:0/3486931054 >> v2:172.21.15.187:3300/0 conn(0x7f17f40d2340 0x7f17f40b9bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:02.020 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:02.018+0000 7f17f8e00640 1 --2- 172.21.15.131:0/3486931054 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f17f40b5d60 0x7f17f40b6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:02.020 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:02.018+0000 7f17f8e00640 1 -- 172.21.15.131:0/3486931054 >> 172.21.15.131:0/3486931054 conn(0x7f17f40676b0 msgr2=0x7f17f40d0d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:02.020 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:02.018+0000 7f17f8e00640 1 -- 172.21.15.131:0/3486931054 shutdown_connections 2024-05-26T22:48:02.021 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:02.018+0000 7f17f8e00640 1 -- 172.21.15.131:0/3486931054 wait complete. 2024-05-26T22:48:02.021 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:02.019+0000 7f17f8e00640 1 Processor -- start 2024-05-26T22:48:02.021 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:02.019+0000 7f17f8e00640 1 -- start start 2024-05-26T22:48:02.023 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:02.020+0000 7f17f8e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f17f40b5d60 0x7f17f4155110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:02.023 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:02.020+0000 7f17f8e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17f40d3870 0x7f17f415e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:02.023 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:02.020+0000 7f17f8e00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f17f415ef10 0x7f17f40b7370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:02.023 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:02.020+0000 7f17f8e00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f17f40ba290 con 0x7f17f40d3870 2024-05-26T22:48:02.023 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:02.020+0000 7f17f8e00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f17f40ba110 con 0x7f17f415ef10 2024-05-26T22:48:02.023 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:02.020+0000 7f17f8e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f17f415e6f0 con 0x7f17f40b5d60 2024-05-26T22:48:02.024 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:02.021+0000 7f17ebe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17f40d3870 0x7f17f415e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:02.024 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:02.021+0000 7f17ebe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17f40d3870 0x7f17f415e9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33608/0 (socket says 172.21.15.131:33608) 2024-05-26T22:48:02.024 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:02.021+0000 7f17ebe00640 1 -- 172.21.15.131:0/2567224300 learned_addr learned my addr 172.21.15.131:0/2567224300 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:02.024 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:02.021+0000 7f17f1e00640 1 --2- 172.21.15.131:0/2567224300 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f17f40b5d60 0x7f17f4155110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:02.024 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:02.021+0000 7f17f2800640 1 --2- 172.21.15.131:0/2567224300 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f17f415ef10 0x7f17f40b7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:02.024 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:02.021+0000 7f17ebe00640 1 -- 172.21.15.131:0/2567224300 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f17f40b5d60 msgr2=0x7f17f4155110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:02.024 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:02.021+0000 7f17ebe00640 1 --2- 172.21.15.131:0/2567224300 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f17f40b5d60 0x7f17f4155110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:02.024 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:02.021+0000 7f17ebe00640 1 -- 172.21.15.131:0/2567224300 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f17f415ef10 msgr2=0x7f17f40b7370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:02.024 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:02.021+0000 7f17ebe00640 1 --2- 172.21.15.131:0/2567224300 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f17f415ef10 0x7f17f40b7370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:02.025 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:02.021+0000 7f17ebe00640 1 -- 172.21.15.131:0/2567224300 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17f415f9e0 con 0x7f17f40d3870 2024-05-26T22:48:02.025 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:02.021+0000 7f17f2800640 1 --2- 172.21.15.131:0/2567224300 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f17f415ef10 0x7f17f40b7370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:48:02.025 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:02.021+0000 7f17ebe00640 1 --2- 172.21.15.131:0/2567224300 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17f40d3870 0x7f17f415e9d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f17dc00ec10 tx=0x7f17dc00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:02.025 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:02.021+0000 7f17ea000640 1 -- 172.21.15.131:0/2567224300 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17dc017070 con 0x7f17f40d3870 2024-05-26T22:48:02.025 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:02.022+0000 7f17ea000640 1 -- 172.21.15.131:0/2567224300 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17dc00cc30 con 0x7f17f40d3870 2024-05-26T22:48:02.025 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:02.022+0000 7f17ea000640 1 -- 172.21.15.131:0/2567224300 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17dc013470 con 0x7f17f40d3870 2024-05-26T22:48:02.025 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:02.022+0000 7f17f8e00640 1 -- 172.21.15.131:0/2567224300 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17f415f420 con 0x7f17f40d3870 2024-05-26T22:48:02.025 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:02.022+0000 7f17f8e00640 1 -- 172.21.15.131:0/2567224300 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17f415ff00 con 0x7f17f40d3870 2024-05-26T22:48:02.025 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:02.022+0000 7f17ea000640 1 -- 172.21.15.131:0/2567224300 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f17dc01a070 con 0x7f17f40d3870 2024-05-26T22:48:02.026 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:02.022+0000 7f17ea000640 1 -- 172.21.15.131:0/2567224300 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f17dc0066f0 con 0x7f17f40d3870 2024-05-26T22:48:02.026 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:02.023+0000 7f17f8e00640 1 -- 172.21.15.131:0/2567224300 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17f404f760 con 0x7f17f40d3870 2024-05-26T22:48:02.026 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:02.025+0000 7f17ea000640 1 -- 172.21.15.131:0/2567224300 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f17dc013610 con 0x7f17f40d3870 2024-05-26T22:48:02.069 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-05-26T22:48:02.069+0000 7f2d5eb2f040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-26T22:48:02.089 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:02.090+0000 7f17f8e00640 1 -- 172.21.15.131:0/2567224300 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f17f4155650 con 0x7f17f40d3870 2024-05-26T22:48:02.090 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:02.090+0000 7f17ea000640 1 -- 172.21.15.131:0/2567224300 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f17dc013a20 con 0x7f17f40d3870 2024-05-26T22:48:02.090 INFO:teuthology.misc.health.smithi131.stdout: 2024-05-26T22:48:02.090 INFO:teuthology.misc.health.smithi131.stdout:{"epoch":10,"fsid":"46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f","created":"2024-05-26T22:47:57.288905+0000","modified":"2024-05-26T22:48:00.687341+0000","last_up_change":"0.000000","last_in_change":"2024-05-26T22:48:00.687341+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"814eaf62-a18d-494b-8e27-fd8fb88adea6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"42e66287-f6c6-4e26-af3c-6dfaa3536d27","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"0da34673-04a2-4fd0-b57a-0ff2b679b043","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"cf117a27-d6b2-43d3-ae52-b47988bf8eed","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"a092ccfc-be89-4a84-88d5-0f781fa715eb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"7513ba69-dd00-4a35-b190-9e8cc3b2bdc8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"50081810-c093-4d2f-a4d7-b7c695c5400d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"775dae05-54f5-4ad4-bbce-92863c6ed2ff","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-26T22:48:02.091 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:02.092+0000 7f17f8e00640 1 -- 172.21.15.131:0/2567224300 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17f40d3870 msgr2=0x7f17f415e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:02.091 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:02.092+0000 7f17f8e00640 1 --2- 172.21.15.131:0/2567224300 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17f40d3870 0x7f17f415e9d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f17dc00ec10 tx=0x7f17dc00c520 comp rx=0 tx=0).stop 2024-05-26T22:48:02.091 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:02.092+0000 7f17f8e00640 1 -- 172.21.15.131:0/2567224300 shutdown_connections 2024-05-26T22:48:02.092 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:02.092+0000 7f17f8e00640 1 --2- 172.21.15.131:0/2567224300 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f17f415ef10 0x7f17f40b7370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:02.092 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:02.092+0000 7f17f8e00640 1 --2- 172.21.15.131:0/2567224300 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17f40d3870 0x7f17f415e9d0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:02.092 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:02.092+0000 7f17f8e00640 1 --2- 172.21.15.131:0/2567224300 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f17f40b5d60 0x7f17f4155110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:02.092 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:02.092+0000 7f17f8e00640 1 -- 172.21.15.131:0/2567224300 >> 172.21.15.131:0/2567224300 conn(0x7f17f40676b0 msgr2=0x7f17f40b8900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:02.092 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:02.092+0000 7f17f8e00640 1 -- 172.21.15.131:0/2567224300 shutdown_connections 2024-05-26T22:48:02.092 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:02.092+0000 7f17f8e00640 1 -- 172.21.15.131:0/2567224300 wait complete. 2024-05-26T22:48:02.104 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-05-26T22:48:02.221 INFO:tasks.ceph.mgr.y.smithi131.stderr:2024-05-26T22:48:02.221+0000 7f977c60d040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-26T22:48:02.356 INFO:tasks.ceph.mgr.y.smithi131.stderr:2024-05-26T22:48:02.356+0000 7f977c60d040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-26T22:48:02.621 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-05-26T22:48:02.622+0000 7f2d5eb2f040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-26T22:48:02.751 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-05-26T22:48:02.751+0000 7f2d5eb2f040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-26T22:48:03.387 INFO:tasks.ceph.mgr.y.smithi131.stderr:2024-05-26T22:48:03.387+0000 7f971ca00640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-05-26T22:48:03.391 INFO:tasks.ceph.mgr.y.smithi131.stderr:2024-05-26T22:48:03.392+0000 7f971ca00640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-05-26T22:48:04.993 INFO:tasks.ceph.osd.0.smithi131.stderr:2024-05-26T22:48:04.994+0000 7f483ba2f640 -1 osd.0 0 log_to_monitors true 2024-05-26T22:48:05.035 INFO:tasks.ceph.osd.4.smithi187.stderr:2024-05-26T22:48:05.035+0000 7fee041cb640 -1 osd.4 0 log_to_monitors true 2024-05-26T22:48:05.043 INFO:tasks.ceph.osd.6.smithi187.stderr:2024-05-26T22:48:05.043+0000 7f61b0618640 -1 osd.6 0 log_to_monitors true 2024-05-26T22:48:05.052 INFO:tasks.ceph.osd.2.smithi131.stderr:2024-05-26T22:48:05.052+0000 7efc6e661640 -1 osd.2 0 log_to_monitors true 2024-05-26T22:48:05.252 INFO:tasks.ceph.osd.5.smithi187.stderr:2024-05-26T22:48:05.253+0000 7f7c62ad9640 -1 osd.5 0 log_to_monitors true 2024-05-26T22:48:05.289 INFO:tasks.ceph.osd.3.smithi131.stderr:2024-05-26T22:48:05.289+0000 7fb779a3d640 -1 osd.3 0 log_to_monitors true 2024-05-26T22:48:05.291 INFO:tasks.ceph.osd.1.smithi131.stderr:2024-05-26T22:48:05.291+0000 7fcf75744640 -1 osd.1 0 log_to_monitors true 2024-05-26T22:48:05.301 INFO:tasks.ceph.osd.7.smithi187.stderr:2024-05-26T22:48:05.301+0000 7fa2e900d640 -1 osd.7 0 log_to_monitors true 2024-05-26T22:48:08.598 INFO:tasks.ceph.osd.1.smithi131.stderr:2024-05-26T22:48:08.599+0000 7fcf6fc00640 -1 osd.1 0 waiting for initial osdmap 2024-05-26T22:48:08.605 INFO:tasks.ceph.osd.1.smithi131.stderr:2024-05-26T22:48:08.606+0000 7fcf66600640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-26T22:48:08.615 INFO:tasks.ceph.osd.6.smithi187.stderr:2024-05-26T22:48:08.615+0000 7f61aa400640 -1 osd.6 0 waiting for initial osdmap 2024-05-26T22:48:08.628 INFO:tasks.ceph.osd.6.smithi187.stderr:2024-05-26T22:48:08.628+0000 7f61a0e00640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-26T22:48:08.645 INFO:tasks.ceph.osd.5.smithi187.stderr:2024-05-26T22:48:08.646+0000 7f7c5d000640 -1 osd.5 0 waiting for initial osdmap 2024-05-26T22:48:08.653 INFO:tasks.ceph.osd.5.smithi187.stderr:2024-05-26T22:48:08.653+0000 7f7c53a00640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-26T22:48:08.657 INFO:tasks.ceph.osd.4.smithi187.stderr:2024-05-26T22:48:08.658+0000 7fedfe600640 -1 osd.4 0 waiting for initial osdmap 2024-05-26T22:48:08.670 INFO:tasks.ceph.osd.4.smithi187.stderr:2024-05-26T22:48:08.670+0000 7fedf5000640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-26T22:48:08.695 INFO:tasks.ceph.osd.3.smithi131.stderr:2024-05-26T22:48:08.696+0000 7fb773e00640 -1 osd.3 0 waiting for initial osdmap 2024-05-26T22:48:08.699 INFO:tasks.ceph.osd.2.smithi131.stderr:2024-05-26T22:48:08.700+0000 7efc68c00640 -1 osd.2 0 waiting for initial osdmap 2024-05-26T22:48:08.704 INFO:tasks.ceph.osd.3.smithi131.stderr:2024-05-26T22:48:08.704+0000 7fb76a800640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-26T22:48:08.707 INFO:tasks.ceph.osd.2.smithi131.stderr:2024-05-26T22:48:08.708+0000 7efc5f600640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-26T22:48:08.719 INFO:tasks.ceph.osd.0.smithi131.stderr:2024-05-26T22:48:08.720+0000 7f4837200640 -1 osd.0 0 waiting for initial osdmap 2024-05-26T22:48:08.724 INFO:tasks.ceph.osd.0.smithi131.stderr:2024-05-26T22:48:08.725+0000 7f482be00640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-26T22:48:08.738 INFO:tasks.ceph.osd.7.smithi187.stderr:2024-05-26T22:48:08.739+0000 7fa2e3000640 -1 osd.7 0 waiting for initial osdmap 2024-05-26T22:48:08.747 INFO:tasks.ceph.osd.7.smithi187.stderr:2024-05-26T22:48:08.747+0000 7fa2d9a00640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-26T22:48:08.909 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-05-26T22:48:09.087 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:09.087+0000 7fc636e00640 1 Processor -- start 2024-05-26T22:48:09.087 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:09.088+0000 7fc636e00640 1 -- start start 2024-05-26T22:48:09.089 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:09.089+0000 7fc636e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc630159e50 0x7fc6300cc8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:09.089 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:09.089+0000 7fc636e00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6300b1d60 con 0x7fc63015d360 2024-05-26T22:48:09.089 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:09.089+0000 7fc636e00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6300b2040 con 0x7fc63006dee0 2024-05-26T22:48:09.089 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:09.089+0000 7fc636e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc6300ce660 con 0x7fc630159e50 2024-05-26T22:48:09.089 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:09.089+0000 7fc627400640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fc63015d360 0x7fc63006d7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:45510/0 (socket says 172.21.15.131:45510) 2024-05-26T22:48:09.089 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:09.089+0000 7fc627400640 1 -- 172.21.15.131:0/772631660 learned_addr learned my addr 172.21.15.131:0/772631660 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:48:09.092 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:09.090+0000 7fc62f400640 1 -- 172.21.15.131:0/772631660 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2836234596 0 0) 0x7fc6300b1d60 con 0x7fc63015d360 2024-05-26T22:48:09.092 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:09.090+0000 7fc62fe00640 1 --2- 172.21.15.131:0/772631660 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc630159e50 0x7fc6300cc8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:09.092 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:09.090+0000 7fc62f400640 1 -- 172.21.15.131:0/772631660 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc614003520 con 0x7fc63015d360 2024-05-26T22:48:09.092 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:09.090+0000 7fc62f400640 1 -- 172.21.15.131:0/772631660 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2983345802 0 0) 0x7fc614003520 con 0x7fc63015d360 2024-05-26T22:48:09.092 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:09.090+0000 7fc62f400640 1 -- 172.21.15.131:0/772631660 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc630159e50 msgr2=0x7fc6300cc8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:09.092 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:09.090+0000 7fc62f400640 1 --2- 172.21.15.131:0/772631660 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc630159e50 0x7fc6300cc8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:09.093 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:09.090+0000 7fc62f400640 1 -- 172.21.15.131:0/772631660 >> v1:172.21.15.187:6789/0 conn(0x7fc63006dee0 legacy=0x7fc6300b3500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:09.093 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:09.090+0000 7fc62f400640 1 -- 172.21.15.131:0/772631660 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6300b2320 con 0x7fc63015d360 2024-05-26T22:48:09.093 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:09.090+0000 7fc62f400640 1 -- 172.21.15.131:0/772631660 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7fc618001670 con 0x7fc63015d360 2024-05-26T22:48:09.093 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:09.090+0000 7fc62f400640 1 -- 172.21.15.131:0/772631660 >> v1:172.21.15.131:6789/0 conn(0x7fc63015d360 legacy=0x7fc63006d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:09.093 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:09.090+0000 7fc62f400640 1 --2- 172.21.15.131:0/772631660 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6140039e0 0x7fc614003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:09.093 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:09.090+0000 7fc62f400640 1 --2- 172.21.15.131:0/772631660 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc630159e50 0x7fc614004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:09.093 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:09.091+0000 7fc62f400640 1 --2- 172.21.15.131:0/772631660 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc614004d40 0x7fc6140051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:09.093 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:09.091+0000 7fc62f400640 1 -- 172.21.15.131:0/772631660 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc6300ce660 con 0x7fc6140039e0 2024-05-26T22:48:09.094 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:09.091+0000 7fc62f400640 1 -- 172.21.15.131:0/772631660 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc61400ba70 con 0x7fc614004d40 2024-05-26T22:48:09.094 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:09.091+0000 7fc62f400640 1 -- 172.21.15.131:0/772631660 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc61400bc70 con 0x7fc630159e50 2024-05-26T22:48:09.094 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:09.091+0000 7fc62fe00640 1 --2- 172.21.15.131:0/772631660 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6140039e0 0x7fc614003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:09.094 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:09.091+0000 7fc627400640 1 --2- 172.21.15.131:0/772631660 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc630159e50 0x7fc614004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:09.094 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:09.091+0000 7fc62fe00640 1 -- 172.21.15.131:0/772631660 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc630159e50 msgr2=0x7fc614004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:09.094 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:09.091+0000 7fc62fe00640 1 --2- 172.21.15.131:0/772631660 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc630159e50 0x7fc614004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:09.094 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:09.091+0000 7fc62fe00640 1 -- 172.21.15.131:0/772631660 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc614004d40 msgr2=0x7fc6140051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:09.094 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:09.091+0000 7fc634e00640 1 --2- 172.21.15.131:0/772631660 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc614004d40 0x7fc6140051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:09.094 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:09.091+0000 7fc62fe00640 1 --2- 172.21.15.131:0/772631660 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc614004d40 0x7fc6140051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:09.094 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:09.091+0000 7fc62fe00640 1 -- 172.21.15.131:0/772631660 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc614003520 con 0x7fc6140039e0 2024-05-26T22:48:09.095 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:09.092+0000 7fc62fe00640 1 --2- 172.21.15.131:0/772631660 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6140039e0 0x7fc614003e00 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc620009f90 tx=0x7fc614005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=537b1b9ca0b86cf7 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:09.095 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:09.092+0000 7fc62f400640 1 -- 172.21.15.131:0/772631660 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc620009c10 con 0x7fc6140039e0 2024-05-26T22:48:09.095 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:09.093+0000 7fc62f400640 1 -- 172.21.15.131:0/772631660 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc620036070 con 0x7fc6140039e0 2024-05-26T22:48:09.095 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:09.093+0000 7fc62f400640 1 -- 172.21.15.131:0/772631660 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc620031850 con 0x7fc6140039e0 2024-05-26T22:48:09.098 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:09.093+0000 7fc636e00640 1 -- 172.21.15.131:0/772631660 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6140039e0 msgr2=0x7fc614003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:09.098 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:09.093+0000 7fc636e00640 1 --2- 172.21.15.131:0/772631660 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6140039e0 0x7fc614003e00 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc620009f90 tx=0x7fc614005830 comp rx=0 tx=0).stop 2024-05-26T22:48:09.098 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:09.093+0000 7fc634e00640 1 -- 172.21.15.131:0/772631660 reap_dead start 2024-05-26T22:48:09.098 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:09.093+0000 7fc636e00640 1 -- 172.21.15.131:0/772631660 shutdown_connections 2024-05-26T22:48:09.098 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:09.093+0000 7fc636e00640 1 -- 172.21.15.131:0/772631660 >> 172.21.15.131:0/772631660 conn(0x7fc63006e610 msgr2=0x7fc63006ea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:09.099 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:09.093+0000 7fc636e00640 1 -- 172.21.15.131:0/772631660 shutdown_connections 2024-05-26T22:48:09.099 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:09.093+0000 7fc636e00640 1 -- 172.21.15.131:0/772631660 wait complete. 2024-05-26T22:48:09.099 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:09.095+0000 7fc636e00640 1 Processor -- start 2024-05-26T22:48:09.099 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:09.095+0000 7fc636e00640 1 -- start start 2024-05-26T22:48:09.099 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:09.096+0000 7fc636e00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc6140039e0 0x7fc63015e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:09.099 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:09.096+0000 7fc636e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6301f2f30 0x7fc63014bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:09.099 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:09.096+0000 7fc636e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc6301f3390 0x7fc6300b0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:09.099 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:09.096+0000 7fc636e00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc6300cca30 con 0x7fc6301f2f30 2024-05-26T22:48:09.099 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:09.096+0000 7fc636e00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc6300cc8b0 con 0x7fc6140039e0 2024-05-26T22:48:09.099 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:09.096+0000 7fc636e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc6300ccbb0 con 0x7fc6301f3390 2024-05-26T22:48:09.099 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:09.096+0000 7fc627400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6301f2f30 0x7fc63014bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:09.099 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:09.096+0000 7fc627400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6301f2f30 0x7fc63014bbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44580/0 (socket says 172.21.15.131:44580) 2024-05-26T22:48:09.099 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:09.096+0000 7fc627400640 1 -- 172.21.15.131:0/269653943 learned_addr learned my addr 172.21.15.131:0/269653943 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:09.100 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:09.097+0000 7fc62fe00640 1 --2- 172.21.15.131:0/269653943 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc6140039e0 0x7fc63015e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:09.100 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:09.097+0000 7fc634e00640 1 --2- 172.21.15.131:0/269653943 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc6301f3390 0x7fc6300b0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:09.100 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:09.097+0000 7fc627400640 1 -- 172.21.15.131:0/269653943 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc6301f3390 msgr2=0x7fc6300b0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:09.100 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:09.097+0000 7fc627400640 1 --2- 172.21.15.131:0/269653943 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc6301f3390 0x7fc6300b0bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:09.100 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:09.097+0000 7fc627400640 1 -- 172.21.15.131:0/269653943 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc6140039e0 msgr2=0x7fc63015e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:09.100 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:09.097+0000 7fc627400640 1 --2- 172.21.15.131:0/269653943 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc6140039e0 0x7fc63015e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:09.100 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:09.097+0000 7fc627400640 1 -- 172.21.15.131:0/269653943 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc63015db10 con 0x7fc6301f2f30 2024-05-26T22:48:09.100 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:09.097+0000 7fc62fe00640 1 --2- 172.21.15.131:0/269653943 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc6140039e0 0x7fc63015e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:48:09.100 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:09.097+0000 7fc627400640 1 --2- 172.21.15.131:0/269653943 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6301f2f30 0x7fc63014bbc0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc61800d810 tx=0x7fc61800dcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:09.100 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:09.097+0000 7fc62cc00640 1 -- 172.21.15.131:0/269653943 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc618019070 con 0x7fc6301f2f30 2024-05-26T22:48:09.100 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:09.097+0000 7fc62cc00640 1 -- 172.21.15.131:0/269653943 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6180074e0 con 0x7fc6301f2f30 2024-05-26T22:48:09.100 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:09.097+0000 7fc62cc00640 1 -- 172.21.15.131:0/269653943 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc618004a00 con 0x7fc6301f2f30 2024-05-26T22:48:09.100 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:09.097+0000 7fc636e00640 1 -- 172.21.15.131:0/269653943 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc63014fdf0 con 0x7fc6301f2f30 2024-05-26T22:48:09.101 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:09.097+0000 7fc636e00640 1 -- 172.21.15.131:0/269653943 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6301503b0 con 0x7fc6301f2f30 2024-05-26T22:48:09.101 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:09.098+0000 7fc636e00640 1 -- 172.21.15.131:0/269653943 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc63004f760 con 0x7fc6301f2f30 2024-05-26T22:48:09.101 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:09.099+0000 7fc62cc00640 1 -- 172.21.15.131:0/269653943 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 4) ==== 90442+0+0 (secure 0 0 0) 0x7fc61801c070 con 0x7fc6301f2f30 2024-05-26T22:48:09.101 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:09.100+0000 7fc62cc00640 1 --2- 172.21.15.131:0/269653943 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc60c06bda0 0x7fc60c06e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:09.102 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:09.100+0000 7fc62cc00640 1 -- 172.21.15.131:0/269653943 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2231+0+0 (secure 0 0 0) 0x7fc618089b70 con 0x7fc6301f2f30 2024-05-26T22:48:09.102 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:09.101+0000 7fc62fe00640 1 --2- 172.21.15.131:0/269653943 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc60c06bda0 0x7fc60c06e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:09.102 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:09.101+0000 7fc62fe00640 1 --2- 172.21.15.131:0/269653943 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc60c06bda0 0x7fc60c06e260 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fc620035000 tx=0x7fc62002f000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:09.105 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:09.106+0000 7fc62cc00640 1 -- 172.21.15.131:0/269653943 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fc63015db10 con 0x7fc6301f2f30 2024-05-26T22:48:09.295 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:09.295+0000 7fc636e00640 1 -- 172.21.15.131:0/269653943 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc6300d0970 con 0x7fc6301f2f30 2024-05-26T22:48:09.296 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:09.297+0000 7fc62cc00640 1 -- 172.21.15.131:0/269653943 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) ==== 74+0+7058 (secure 0 0 0) 0x7fc6180592f0 con 0x7fc6301f2f30 2024-05-26T22:48:09.297 INFO:teuthology.misc.health.smithi131.stdout: 2024-05-26T22:48:09.297 INFO:teuthology.misc.health.smithi131.stdout:{"epoch":12,"fsid":"46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f","created":"2024-05-26T22:47:57.288905+0000","modified":"2024-05-26T22:48:06.393721+0000","last_up_change":"0.000000","last_in_change":"2024-05-26T22:48:00.687341+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"814eaf62-a18d-494b-8e27-fd8fb88adea6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"42e66287-f6c6-4e26-af3c-6dfaa3536d27","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"0da34673-04a2-4fd0-b57a-0ff2b679b043","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"cf117a27-d6b2-43d3-ae52-b47988bf8eed","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"a092ccfc-be89-4a84-88d5-0f781fa715eb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"7513ba69-dd00-4a35-b190-9e8cc3b2bdc8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"50081810-c093-4d2f-a4d7-b7c695c5400d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"775dae05-54f5-4ad4-bbce-92863c6ed2ff","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-26T22:48:09.299 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:09.299+0000 7fc636e00640 1 -- 172.21.15.131:0/269653943 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc60c06bda0 msgr2=0x7fc60c06e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:09.299 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:09.299+0000 7fc636e00640 1 --2- 172.21.15.131:0/269653943 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc60c06bda0 0x7fc60c06e260 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fc620035000 tx=0x7fc62002f000 comp rx=0 tx=0).stop 2024-05-26T22:48:09.300 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:09.299+0000 7fc636e00640 1 -- 172.21.15.131:0/269653943 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6301f2f30 msgr2=0x7fc63014bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:09.300 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:09.299+0000 7fc636e00640 1 --2- 172.21.15.131:0/269653943 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6301f2f30 0x7fc63014bbc0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc61800d810 tx=0x7fc61800dcd0 comp rx=0 tx=0).stop 2024-05-26T22:48:09.300 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:09.300+0000 7fc636e00640 1 -- 172.21.15.131:0/269653943 shutdown_connections 2024-05-26T22:48:09.300 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:09.300+0000 7fc636e00640 1 --2- 172.21.15.131:0/269653943 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc6301f3390 0x7fc6300b0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:09.300 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:09.300+0000 7fc636e00640 1 --2- 172.21.15.131:0/269653943 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc60c06bda0 0x7fc60c06e260 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:09.300 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:09.300+0000 7fc636e00640 1 --2- 172.21.15.131:0/269653943 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6301f2f30 0x7fc63014bbc0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:09.300 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:09.300+0000 7fc636e00640 1 --2- 172.21.15.131:0/269653943 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc6140039e0 0x7fc63015e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:09.300 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:09.300+0000 7fc636e00640 1 -- 172.21.15.131:0/269653943 >> 172.21.15.131:0/269653943 conn(0x7fc63006e610 msgr2=0x7fc6300b3be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:09.300 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:09.300+0000 7fc636e00640 1 -- 172.21.15.131:0/269653943 shutdown_connections 2024-05-26T22:48:09.300 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:09.300+0000 7fc636e00640 1 -- 172.21.15.131:0/269653943 wait complete. 2024-05-26T22:48:09.313 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-05-26T22:48:13.473 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-26T22:48:13.482 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-26T22:48:16.120 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-05-26T22:48:16.311 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:16.311+0000 7f3221c00640 1 Processor -- start 2024-05-26T22:48:16.311 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:16.311+0000 7f3221c00640 1 -- start start 2024-05-26T22:48:16.312 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:16.313+0000 7f3221c00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f321c0d0bd0 0x7f321c0cd6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:16.313 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:16.313+0000 7f3221c00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f321c0d2cf0 con 0x7f321c0cf2a0 2024-05-26T22:48:16.313 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:16.313+0000 7f3221c00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f321c0d1340 con 0x7f321c0ce7b0 2024-05-26T22:48:16.313 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:16.313+0000 7f3221c00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f321c0cc580 con 0x7f321c0d0bd0 2024-05-26T22:48:16.313 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:16.313+0000 7f3221200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f321c0d0bd0 0x7f321c0cd6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:16.313 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:16.313+0000 7f3221200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f321c0d0bd0 0x7f321c0cd6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33794/0 (socket says 172.21.15.131:33794) 2024-05-26T22:48:16.313 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:16.313+0000 7f3221200640 1 -- 172.21.15.131:0/1927835180 learned_addr learned my addr 172.21.15.131:0/1927835180 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:16.314 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:16.313+0000 7f3221200640 1 -- 172.21.15.131:0/1927835180 >> v1:172.21.15.187:6789/0 conn(0x7f321c0ce7b0 legacy=0x7f321c0ceb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:16.314 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:16.313+0000 7f3221200640 1 -- 172.21.15.131:0/1927835180 >> v1:172.21.15.131:6789/0 conn(0x7f321c0cf2a0 legacy=0x7f321c0ccfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:16.314 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:16.313+0000 7f3221200640 1 -- 172.21.15.131:0/1927835180 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f321c0d1620 con 0x7f321c0d0bd0 2024-05-26T22:48:16.314 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:16.314+0000 7f3221200640 1 --2- 172.21.15.131:0/1927835180 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f321c0d0bd0 0x7f321c0cd6e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3214009e50 tx=0x7f321402a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1544f5adefa765c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:16.315 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:16.315+0000 7f321aa00640 1 -- 172.21.15.131:0/1927835180 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3214036070 con 0x7f321c0d0bd0 2024-05-26T22:48:16.315 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:16.315+0000 7f321aa00640 1 -- 172.21.15.131:0/1927835180 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3214031600 con 0x7f321c0d0bd0 2024-05-26T22:48:16.315 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:16.315+0000 7f321aa00640 1 -- 172.21.15.131:0/1927835180 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32140319d0 con 0x7f321c0d0bd0 2024-05-26T22:48:16.315 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:16.315+0000 7f3221c00640 1 -- 172.21.15.131:0/1927835180 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f321c0d0bd0 msgr2=0x7f321c0cd6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:16.316 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:16.315+0000 7f3221c00640 1 --2- 172.21.15.131:0/1927835180 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f321c0d0bd0 0x7f321c0cd6e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3214009e50 tx=0x7f321402a6d0 comp rx=0 tx=0).stop 2024-05-26T22:48:16.316 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:16.316+0000 7f3221c00640 1 -- 172.21.15.131:0/1927835180 shutdown_connections 2024-05-26T22:48:16.316 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:16.316+0000 7f3221c00640 1 --2- 172.21.15.131:0/1927835180 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f321c0d0bd0 0x7f321c0cd6e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:16.316 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:16.316+0000 7f3221c00640 1 -- 172.21.15.131:0/1927835180 >> 172.21.15.131:0/1927835180 conn(0x7f321c0d1f30 msgr2=0x7f321c0d2340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:16.316 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:16.316+0000 7f3221c00640 1 -- 172.21.15.131:0/1927835180 shutdown_connections 2024-05-26T22:48:16.316 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:16.316+0000 7f3221c00640 1 -- 172.21.15.131:0/1927835180 wait complete. 2024-05-26T22:48:16.317 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:16.318+0000 7f3221c00640 1 Processor -- start 2024-05-26T22:48:16.318 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:16.318+0000 7f3221c00640 1 -- start start 2024-05-26T22:48:16.319 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:16.319+0000 7f3221c00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f321c0ce7b0 0x7f321c1fd080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:16.319 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:16.319+0000 7f3221c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f321c0cf2a0 0x7f321c14b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:16.319 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:16.319+0000 7f3221c00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f321c0d0bd0 0x7f321c1f8f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:16.319 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:16.319+0000 7f3221c00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f321c161e30 con 0x7f321c0cf2a0 2024-05-26T22:48:16.319 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:16.319+0000 7f3221c00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f321c0ccab0 con 0x7f321c0ce7b0 2024-05-26T22:48:16.320 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:16.319+0000 7f3221c00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f321c0d0fe0 con 0x7f321c0d0bd0 2024-05-26T22:48:16.320 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:16.320+0000 7f321b400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f321c0cf2a0 0x7f321c14b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:16.320 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:16.320+0000 7f321b400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f321c0cf2a0 0x7f321c14b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33810/0 (socket says 172.21.15.131:33810) 2024-05-26T22:48:16.320 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:16.320+0000 7f321b400640 1 -- 172.21.15.131:0/72118140 learned_addr learned my addr 172.21.15.131:0/72118140 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:16.321 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:16.320+0000 7f321be00640 1 --2- 172.21.15.131:0/72118140 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f321c0ce7b0 0x7f321c1fd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:16.321 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:16.320+0000 7f3221200640 1 --2- 172.21.15.131:0/72118140 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f321c0d0bd0 0x7f321c1f8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:16.321 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:16.320+0000 7f321b400640 1 -- 172.21.15.131:0/72118140 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f321c0d0bd0 msgr2=0x7f321c1f8f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:16.321 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:16.320+0000 7f321b400640 1 --2- 172.21.15.131:0/72118140 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f321c0d0bd0 0x7f321c1f8f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:16.321 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:16.320+0000 7f321b400640 1 -- 172.21.15.131:0/72118140 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f321c0ce7b0 msgr2=0x7f321c1fd080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:16.322 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:16.320+0000 7f321b400640 1 --2- 172.21.15.131:0/72118140 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f321c0ce7b0 0x7f321c1fd080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:16.322 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:16.320+0000 7f321b400640 1 -- 172.21.15.131:0/72118140 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f321c1549d0 con 0x7f321c0cf2a0 2024-05-26T22:48:16.322 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:16.320+0000 7f3221200640 1 --2- 172.21.15.131:0/72118140 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f321c0d0bd0 0x7f321c1f8f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:48:16.322 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:16.320+0000 7f321b400640 1 --2- 172.21.15.131:0/72118140 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f321c0cf2a0 0x7f321c14b3f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f320800c3f0 tx=0x7f320800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:16.322 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:16.321+0000 7f3213e00640 1 -- 172.21.15.131:0/72118140 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32080052d0 con 0x7f321c0cf2a0 2024-05-26T22:48:16.322 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:16.321+0000 7f3213e00640 1 -- 172.21.15.131:0/72118140 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3208005490 con 0x7f321c0cf2a0 2024-05-26T22:48:16.322 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:16.321+0000 7f3213e00640 1 -- 172.21.15.131:0/72118140 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3208007260 con 0x7f321c0cf2a0 2024-05-26T22:48:16.322 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:16.321+0000 7f3221c00640 1 -- 172.21.15.131:0/72118140 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f321c1f9b10 con 0x7f321c0cf2a0 2024-05-26T22:48:16.322 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:16.321+0000 7f3221c00640 1 -- 172.21.15.131:0/72118140 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f321c1fc880 con 0x7f321c0cf2a0 2024-05-26T22:48:16.323 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:16.322+0000 7f3221c00640 1 -- 172.21.15.131:0/72118140 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f321c09ffc0 con 0x7f321c0cf2a0 2024-05-26T22:48:16.323 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:16.323+0000 7f3213e00640 1 -- 172.21.15.131:0/72118140 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3208007400 con 0x7f321c0cf2a0 2024-05-26T22:48:16.324 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:16.324+0000 7f3213e00640 1 --2- 172.21.15.131:0/72118140 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f31fc06be30 0x7f31fc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:16.324 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:16.324+0000 7f3213e00640 1 -- 172.21.15.131:0/72118140 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f320808a0a0 con 0x7f321c0cf2a0 2024-05-26T22:48:16.324 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:16.324+0000 7f321be00640 1 --2- 172.21.15.131:0/72118140 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f31fc06be30 0x7f31fc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:16.324 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:16.325+0000 7f321be00640 1 --2- 172.21.15.131:0/72118140 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f31fc06be30 0x7f31fc06e2f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f320c006fd0 tx=0x7f320c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:16.329 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:16.330+0000 7f3213e00640 1 -- 172.21.15.131:0/72118140 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f3208058d00 con 0x7f321c0cf2a0 2024-05-26T22:48:16.530 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:16.530+0000 7f3221c00640 1 -- 172.21.15.131:0/72118140 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f321c0734f0 con 0x7f321c0cf2a0 2024-05-26T22:48:16.530 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:16.531+0000 7f3213e00640 1 -- 172.21.15.131:0/72118140 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12908 (secure 0 0 0) 0x7f3208058600 con 0x7f321c0cf2a0 2024-05-26T22:48:16.531 INFO:teuthology.misc.health.smithi131.stdout: 2024-05-26T22:48:16.531 INFO:teuthology.misc.health.smithi131.stdout:{"epoch":16,"fsid":"46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f","created":"2024-05-26T22:47:57.288905+0000","modified":"2024-05-26T22:48:12.655278+0000","last_up_change":"2024-05-26T22:48:09.602207+0000","last_in_change":"2024-05-26T22:48:00.687341+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-26T22:48:10.463339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"814eaf62-a18d-494b-8e27-fd8fb88adea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6801","nonce":2350940051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6803","nonce":2350940051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6807","nonce":2350940051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6805","nonce":2350940051}]},"public_addr":"172.21.15.131:6801/2350940051","cluster_addr":"172.21.15.131:6803/2350940051","heartbeat_back_addr":"172.21.15.131:6807/2350940051","heartbeat_front_addr":"172.21.15.131:6805/2350940051","state":["exists","up"]},{"osd":1,"uuid":"42e66287-f6c6-4e26-af3c-6dfaa3536d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6825","nonce":1525528247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6827","nonce":1525528247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6831","nonce":1525528247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6829","nonce":1525528247}]},"public_addr":"172.21.15.131:6825/1525528247","cluster_addr":"172.21.15.131:6827/1525528247","heartbeat_back_addr":"172.21.15.131:6831/1525528247","heartbeat_front_addr":"172.21.15.131:6829/1525528247","state":["exists","up"]},{"osd":2,"uuid":"0da34673-04a2-4fd0-b57a-0ff2b679b043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6817","nonce":3627842210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6819","nonce":3627842210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6823","nonce":3627842210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6821","nonce":3627842210}]},"public_addr":"172.21.15.131:6817/3627842210","cluster_addr":"172.21.15.131:6819/3627842210","heartbeat_back_addr":"172.21.15.131:6823/3627842210","heartbeat_front_addr":"172.21.15.131:6821/3627842210","state":["exists","up"]},{"osd":3,"uuid":"cf117a27-d6b2-43d3-ae52-b47988bf8eed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6809","nonce":350790591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6811","nonce":350790591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6815","nonce":350790591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6813","nonce":350790591}]},"public_addr":"172.21.15.131:6809/350790591","cluster_addr":"172.21.15.131:6811/350790591","heartbeat_back_addr":"172.21.15.131:6815/350790591","heartbeat_front_addr":"172.21.15.131:6813/350790591","state":["exists","up"]},{"osd":4,"uuid":"a092ccfc-be89-4a84-88d5-0f781fa715eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6816","nonce":244560995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6819","nonce":244560995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6823","nonce":244560995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6821","nonce":244560995}]},"public_addr":"172.21.15.187:6816/244560995","cluster_addr":"172.21.15.187:6819/244560995","heartbeat_back_addr":"172.21.15.187:6823/244560995","heartbeat_front_addr":"172.21.15.187:6821/244560995","state":["exists","up"]},{"osd":5,"uuid":"7513ba69-dd00-4a35-b190-9e8cc3b2bdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6801","nonce":324619398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6803","nonce":324619398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6812","nonce":324619398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6807","nonce":324619398}]},"public_addr":"172.21.15.187:6801/324619398","cluster_addr":"172.21.15.187:6803/324619398","heartbeat_back_addr":"172.21.15.187:6812/324619398","heartbeat_front_addr":"172.21.15.187:6807/324619398","state":["exists","up"]},{"osd":6,"uuid":"50081810-c093-4d2f-a4d7-b7c695c5400d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6806","nonce":1391347786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6809","nonce":1391347786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6818","nonce":1391347786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6813","nonce":1391347786}]},"public_addr":"172.21.15.187:6806/1391347786","cluster_addr":"172.21.15.187:6809/1391347786","heartbeat_back_addr":"172.21.15.187:6818/1391347786","heartbeat_front_addr":"172.21.15.187:6813/1391347786","state":["exists","up"]},{"osd":7,"uuid":"775dae05-54f5-4ad4-bbce-92863c6ed2ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6825","nonce":1220065622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6827","nonce":1220065622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6831","nonce":1220065622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6829","nonce":1220065622}]},"public_addr":"172.21.15.187:6825/1220065622","cluster_addr":"172.21.15.187:6827/1220065622","heartbeat_back_addr":"172.21.15.187:6831/1220065622","heartbeat_front_addr":"172.21.15.187:6829/1220065622","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:05.983520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.299516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.067323+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.307748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.018742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.258775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.010912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.295355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-26T22:48:16.533 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:16.533+0000 7f3221c00640 1 -- 172.21.15.131:0/72118140 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f31fc06be30 msgr2=0x7f31fc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:16.533 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:16.533+0000 7f3221c00640 1 --2- 172.21.15.131:0/72118140 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f31fc06be30 0x7f31fc06e2f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f320c006fd0 tx=0x7f320c008040 comp rx=0 tx=0).stop 2024-05-26T22:48:16.533 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:16.533+0000 7f3221c00640 1 -- 172.21.15.131:0/72118140 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f321c0cf2a0 msgr2=0x7f321c14b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:16.534 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:16.533+0000 7f3221c00640 1 --2- 172.21.15.131:0/72118140 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f321c0cf2a0 0x7f321c14b3f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f320800c3f0 tx=0x7f320800c8b0 comp rx=0 tx=0).stop 2024-05-26T22:48:16.534 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:16.534+0000 7f3221c00640 1 -- 172.21.15.131:0/72118140 shutdown_connections 2024-05-26T22:48:16.534 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:16.534+0000 7f3221c00640 1 --2- 172.21.15.131:0/72118140 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f321c0d0bd0 0x7f321c1f8f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:16.534 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:16.534+0000 7f3221c00640 1 --2- 172.21.15.131:0/72118140 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f31fc06be30 0x7f31fc06e2f0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:16.534 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:16.534+0000 7f3221c00640 1 --2- 172.21.15.131:0/72118140 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f321c0cf2a0 0x7f321c14b3f0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:16.534 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:16.534+0000 7f3221c00640 1 --2- 172.21.15.131:0/72118140 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f321c0ce7b0 0x7f321c1fd080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:16.534 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:16.534+0000 7f3221c00640 1 -- 172.21.15.131:0/72118140 >> 172.21.15.131:0/72118140 conn(0x7f321c0d1f30 msgr2=0x7f321c0ceb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:16.534 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:16.534+0000 7f3221c00640 1 -- 172.21.15.131:0/72118140 shutdown_connections 2024-05-26T22:48:16.534 INFO:teuthology.misc.health.smithi131.stderr:2024-05-26T22:48:16.534+0000 7f3221c00640 1 -- 172.21.15.131:0/72118140 wait complete. 2024-05-26T22:48:16.548 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-05-26T22:48:16.548 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-05-26T22:48:16.548 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-26T22:48:16.548 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-26T22:48:16.555 INFO:tasks.ceph.mds.a:Started 2024-05-26T22:48:16.555 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-26T22:48:16.555 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-26T22:48:16.562 INFO:tasks.ceph.mds.c:Started 2024-05-26T22:48:16.562 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-26T22:48:16.562 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-26T22:48:16.570 INFO:tasks.ceph.mds.b:Started 2024-05-26T22:48:16.570 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-26T22:48:16.570 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-26T22:48:16.577 INFO:tasks.ceph.mds.d:Started 2024-05-26T22:48:16.578 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-05-26T22:48:16.578 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-26T22:48:16.698 INFO:tasks.ceph.mds.a.smithi131.stdout:starting mds.a at 2024-05-26T22:48:16.700 INFO:tasks.ceph.mds.c.smithi131.stdout:starting mds.c at 2024-05-26T22:48:16.730 INFO:tasks.ceph.mds.d.smithi187.stdout:starting mds.d at 2024-05-26T22:48:16.730 INFO:tasks.ceph.mds.b.smithi187.stdout:starting mds.b at 2024-05-26T22:48:16.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:16.767+0000 7fcf5a800640 1 Processor -- start 2024-05-26T22:48:16.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:16.767+0000 7fcf5a800640 1 -- start start 2024-05-26T22:48:16.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:16.769+0000 7fcf5a800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcf540ad250 0x7fcf540b3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:16.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:16.769+0000 7fcf5a800640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf5415cb20 con 0x7fcf5406ee20 2024-05-26T22:48:16.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:16.769+0000 7fcf5a800640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf5415ce00 con 0x7fcf541569d0 2024-05-26T22:48:16.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:16.769+0000 7fcf5a800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fcf5415d0e0 con 0x7fcf540ad250 2024-05-26T22:48:16.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:16.769+0000 7fcf53400640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fcf5406ee20 0x7fcf541562c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:40432/0 (socket says 172.21.15.131:40432) 2024-05-26T22:48:16.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:16.769+0000 7fcf53400640 1 -- 172.21.15.131:0/2949139466 learned_addr learned my addr 172.21.15.131:0/2949139466 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:48:16.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:16.769+0000 7fcf52a00640 1 --2- 172.21.15.131:0/2949139466 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcf540ad250 0x7fcf540b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:16.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:16.769+0000 7fcf52000640 1 -- 172.21.15.131:0/2949139466 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 428830455 0 0) 0x7fcf5415cb20 con 0x7fcf5406ee20 2024-05-26T22:48:16.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:16.769+0000 7fcf52000640 1 -- 172.21.15.131:0/2949139466 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcf38003520 con 0x7fcf5406ee20 2024-05-26T22:48:16.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:16.769+0000 7fcf52000640 1 -- 172.21.15.131:0/2949139466 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3446833721 0 0) 0x7fcf38003520 con 0x7fcf5406ee20 2024-05-26T22:48:16.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:16.769+0000 7fcf52000640 1 -- 172.21.15.131:0/2949139466 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcf540ad250 msgr2=0x7fcf540b3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:16.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:16.769+0000 7fcf52000640 1 --2- 172.21.15.131:0/2949139466 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcf540ad250 0x7fcf540b3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:16.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:16.769+0000 7fcf52000640 1 -- 172.21.15.131:0/2949139466 >> v1:172.21.15.187:6789/0 conn(0x7fcf541569d0 legacy=0x7fcf54160010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:16.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:16.769+0000 7fcf52000640 1 -- 172.21.15.131:0/2949139466 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf5415d3c0 con 0x7fcf5406ee20 2024-05-26T22:48:16.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:16.770+0000 7fcf52000640 1 -- 172.21.15.131:0/2949139466 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7fcf44001670 con 0x7fcf5406ee20 2024-05-26T22:48:16.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:16.770+0000 7fcf52000640 1 -- 172.21.15.131:0/2949139466 >> v1:172.21.15.131:6789/0 conn(0x7fcf5406ee20 legacy=0x7fcf541562c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:16.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:16.770+0000 7fcf52000640 1 --2- 172.21.15.131:0/2949139466 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf380039e0 0x7fcf38003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:16.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:16.770+0000 7fcf52000640 1 --2- 172.21.15.131:0/2949139466 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcf38004800 0x7fcf38004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:16.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:16.770+0000 7fcf52000640 1 --2- 172.21.15.131:0/2949139466 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcf540ad250 0x7fcf380051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:16.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:16.770+0000 7fcf52000640 1 -- 172.21.15.131:0/2949139466 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fcf5415d0e0 con 0x7fcf380039e0 2024-05-26T22:48:16.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:16.770+0000 7fcf52000640 1 -- 172.21.15.131:0/2949139466 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fcf3800baa0 con 0x7fcf38004800 2024-05-26T22:48:16.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:16.770+0000 7fcf52000640 1 -- 172.21.15.131:0/2949139466 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fcf3800bca0 con 0x7fcf540ad250 2024-05-26T22:48:16.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:16.770+0000 7fcf53400640 1 --2- 172.21.15.131:0/2949139466 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf380039e0 0x7fcf38003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:16.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:16.770+0000 7fcf53e00640 1 --2- 172.21.15.131:0/2949139466 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcf540ad250 0x7fcf380051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:16.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:16.770+0000 7fcf52a00640 1 --2- 172.21.15.131:0/2949139466 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcf38004800 0x7fcf38004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:16.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:16.770+0000 7fcf53400640 1 -- 172.21.15.131:0/2949139466 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcf540ad250 msgr2=0x7fcf380051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:16.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:16.770+0000 7fcf53400640 1 --2- 172.21.15.131:0/2949139466 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcf540ad250 0x7fcf380051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:16.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:16.770+0000 7fcf53400640 1 -- 172.21.15.131:0/2949139466 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcf38004800 msgr2=0x7fcf38004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:16.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:16.770+0000 7fcf53400640 1 --2- 172.21.15.131:0/2949139466 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcf38004800 0x7fcf38004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:16.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:16.770+0000 7fcf53400640 1 -- 172.21.15.131:0/2949139466 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf38003520 con 0x7fcf380039e0 2024-05-26T22:48:16.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:16.770+0000 7fcf53e00640 1 --2- 172.21.15.131:0/2949139466 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcf540ad250 0x7fcf380051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-26T22:48:16.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:16.771+0000 7fcf53400640 1 --2- 172.21.15.131:0/2949139466 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf380039e0 0x7fcf38003e00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fcf4400c210 tx=0x7fcf38005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=109ccb9a89833cda server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:16.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:16.771+0000 7fcf52000640 1 -- 172.21.15.131:0/2949139466 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf44007d10 con 0x7fcf380039e0 2024-05-26T22:48:16.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:16.771+0000 7fcf52000640 1 -- 172.21.15.131:0/2949139466 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf44005ad0 con 0x7fcf380039e0 2024-05-26T22:48:16.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:16.771+0000 7fcf52000640 1 -- 172.21.15.131:0/2949139466 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf44007070 con 0x7fcf380039e0 2024-05-26T22:48:16.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:16.772+0000 7fcf5a800640 1 -- 172.21.15.131:0/2949139466 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf380039e0 msgr2=0x7fcf38003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:16.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:16.772+0000 7fcf5a800640 1 --2- 172.21.15.131:0/2949139466 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf380039e0 0x7fcf38003e00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fcf4400c210 tx=0x7fcf38005830 comp rx=0 tx=0).stop 2024-05-26T22:48:16.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:16.772+0000 7fcf53e00640 1 -- 172.21.15.131:0/2949139466 reap_dead start 2024-05-26T22:48:16.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:16.772+0000 7fcf5a800640 1 -- 172.21.15.131:0/2949139466 shutdown_connections 2024-05-26T22:48:16.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:16.772+0000 7fcf5a800640 1 -- 172.21.15.131:0/2949139466 >> 172.21.15.131:0/2949139466 conn(0x7fcf540b1e90 msgr2=0x7fcf540b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:16.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:16.772+0000 7fcf5a800640 1 -- 172.21.15.131:0/2949139466 shutdown_connections 2024-05-26T22:48:16.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:16.772+0000 7fcf5a800640 1 -- 172.21.15.131:0/2949139466 wait complete. 2024-05-26T22:48:16.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:16.774+0000 7fcf5a800640 1 Processor -- start 2024-05-26T22:48:16.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:16.774+0000 7fcf5a800640 1 -- start start 2024-05-26T22:48:16.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:16.775+0000 7fcf5a800640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcf380039e0 0x7fcf54151030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:16.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:16.775+0000 7fcf5a800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf54140b60 0x7fcf5414b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:16.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:16.775+0000 7fcf5a800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcf541f6340 0x7fcf540b06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:16.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:16.775+0000 7fcf5a800640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fcf5415c900 con 0x7fcf54140b60 2024-05-26T22:48:16.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:16.775+0000 7fcf5a800640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fcf5415c780 con 0x7fcf380039e0 2024-05-26T22:48:16.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:16.775+0000 7fcf5a800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fcf5406f200 con 0x7fcf541f6340 2024-05-26T22:48:16.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:16.775+0000 7fcf52a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf54140b60 0x7fcf5414b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:16.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:16.775+0000 7fcf52a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf54140b60 0x7fcf5414b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33868/0 (socket says 172.21.15.131:33868) 2024-05-26T22:48:16.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:16.775+0000 7fcf52a00640 1 -- 172.21.15.131:0/1538344731 learned_addr learned my addr 172.21.15.131:0/1538344731 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:16.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:16.775+0000 7fcf53400640 1 --2- 172.21.15.131:0/1538344731 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcf380039e0 0x7fcf54151030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:16.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:16.775+0000 7fcf53e00640 1 --2- 172.21.15.131:0/1538344731 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcf541f6340 0x7fcf540b06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:16.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:16.775+0000 7fcf52a00640 1 -- 172.21.15.131:0/1538344731 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcf541f6340 msgr2=0x7fcf540b06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:16.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:16.775+0000 7fcf52a00640 1 --2- 172.21.15.131:0/1538344731 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcf541f6340 0x7fcf540b06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:16.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:16.775+0000 7fcf52a00640 1 -- 172.21.15.131:0/1538344731 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcf380039e0 msgr2=0x7fcf54151030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:16.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:16.775+0000 7fcf52a00640 1 --2- 172.21.15.131:0/1538344731 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcf380039e0 0x7fcf54151030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:16.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:16.775+0000 7fcf52a00640 1 -- 172.21.15.131:0/1538344731 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf5414c020 con 0x7fcf54140b60 2024-05-26T22:48:16.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:16.776+0000 7fcf52a00640 1 --2- 172.21.15.131:0/1538344731 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf54140b60 0x7fcf5414b6b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fcf400119d0 tx=0x7fcf40011dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:16.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:16.776+0000 7fcf4b400640 1 -- 172.21.15.131:0/1538344731 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf4000faf0 con 0x7fcf54140b60 2024-05-26T22:48:16.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:16.776+0000 7fcf4b400640 1 -- 172.21.15.131:0/1538344731 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf4000fc90 con 0x7fcf54140b60 2024-05-26T22:48:16.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:16.776+0000 7fcf4b400640 1 -- 172.21.15.131:0/1538344731 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf40014440 con 0x7fcf54140b60 2024-05-26T22:48:16.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:16.776+0000 7fcf5a800640 1 -- 172.21.15.131:0/1538344731 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf541f5860 con 0x7fcf54140b60 2024-05-26T22:48:16.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:16.776+0000 7fcf5a800640 1 -- 172.21.15.131:0/1538344731 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf541f5e20 con 0x7fcf54140b60 2024-05-26T22:48:16.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:16.777+0000 7fcf5a800640 1 -- 172.21.15.131:0/1538344731 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf540694e0 con 0x7fcf54140b60 2024-05-26T22:48:16.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:16.778+0000 7fcf4b400640 1 -- 172.21.15.131:0/1538344731 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcf40012040 con 0x7fcf54140b60 2024-05-26T22:48:16.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:16.779+0000 7fcf4b400640 1 --2- 172.21.15.131:0/1538344731 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fcf3406be30 0x7fcf3406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:16.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:16.779+0000 7fcf4b400640 1 -- 172.21.15.131:0/1538344731 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fcf4008b080 con 0x7fcf54140b60 2024-05-26T22:48:16.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:16.779+0000 7fcf53400640 1 --2- 172.21.15.131:0/1538344731 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fcf3406be30 0x7fcf3406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:16.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:16.780+0000 7fcf53400640 1 --2- 172.21.15.131:0/1538344731 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fcf3406be30 0x7fcf3406e2f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fcf44032000 tx=0x7fcf44034040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:16.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:16.783+0000 7fcf4b400640 1 -- 172.21.15.131:0/1538344731 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fcf40019030 con 0x7fcf54140b60 2024-05-26T22:48:16.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:16.979+0000 7fcf5a800640 1 -- 172.21.15.131:0/1538344731 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcf5406e890 con 0x7fcf54140b60 2024-05-26T22:48:16.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:16.980+0000 7fcf4b400640 1 -- 172.21.15.131:0/1538344731 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12908 (secure 0 0 0) 0x7fcf40059ce0 con 0x7fcf54140b60 2024-05-26T22:48:16.980 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:48:16.980 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":16,"fsid":"46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f","created":"2024-05-26T22:47:57.288905+0000","modified":"2024-05-26T22:48:12.655278+0000","last_up_change":"2024-05-26T22:48:09.602207+0000","last_in_change":"2024-05-26T22:48:00.687341+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-26T22:48:10.463339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"814eaf62-a18d-494b-8e27-fd8fb88adea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6801","nonce":2350940051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6803","nonce":2350940051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6807","nonce":2350940051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6805","nonce":2350940051}]},"public_addr":"172.21.15.131:6801/2350940051","cluster_addr":"172.21.15.131:6803/2350940051","heartbeat_back_addr":"172.21.15.131:6807/2350940051","heartbeat_front_addr":"172.21.15.131:6805/2350940051","state":["exists","up"]},{"osd":1,"uuid":"42e66287-f6c6-4e26-af3c-6dfaa3536d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6825","nonce":1525528247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6827","nonce":1525528247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6831","nonce":1525528247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6829","nonce":1525528247}]},"public_addr":"172.21.15.131:6825/1525528247","cluster_addr":"172.21.15.131:6827/1525528247","heartbeat_back_addr":"172.21.15.131:6831/1525528247","heartbeat_front_addr":"172.21.15.131:6829/1525528247","state":["exists","up"]},{"osd":2,"uuid":"0da34673-04a2-4fd0-b57a-0ff2b679b043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6817","nonce":3627842210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6819","nonce":3627842210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6823","nonce":3627842210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6821","nonce":3627842210}]},"public_addr":"172.21.15.131:6817/3627842210","cluster_addr":"172.21.15.131:6819/3627842210","heartbeat_back_addr":"172.21.15.131:6823/3627842210","heartbeat_front_addr":"172.21.15.131:6821/3627842210","state":["exists","up"]},{"osd":3,"uuid":"cf117a27-d6b2-43d3-ae52-b47988bf8eed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6809","nonce":350790591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6811","nonce":350790591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6815","nonce":350790591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6813","nonce":350790591}]},"public_addr":"172.21.15.131:6809/350790591","cluster_addr":"172.21.15.131:6811/350790591","heartbeat_back_addr":"172.21.15.131:6815/350790591","heartbeat_front_addr":"172.21.15.131:6813/350790591","state":["exists","up"]},{"osd":4,"uuid":"a092ccfc-be89-4a84-88d5-0f781fa715eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6816","nonce":244560995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6819","nonce":244560995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6823","nonce":244560995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6821","nonce":244560995}]},"public_addr":"172.21.15.187:6816/244560995","cluster_addr":"172.21.15.187:6819/244560995","heartbeat_back_addr":"172.21.15.187:6823/244560995","heartbeat_front_addr":"172.21.15.187:6821/244560995","state":["exists","up"]},{"osd":5,"uuid":"7513ba69-dd00-4a35-b190-9e8cc3b2bdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6801","nonce":324619398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6803","nonce":324619398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6812","nonce":324619398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6807","nonce":324619398}]},"public_addr":"172.21.15.187:6801/324619398","cluster_addr":"172.21.15.187:6803/324619398","heartbeat_back_addr":"172.21.15.187:6812/324619398","heartbeat_front_addr":"172.21.15.187:6807/324619398","state":["exists","up"]},{"osd":6,"uuid":"50081810-c093-4d2f-a4d7-b7c695c5400d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6806","nonce":1391347786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6809","nonce":1391347786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6818","nonce":1391347786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6813","nonce":1391347786}]},"public_addr":"172.21.15.187:6806/1391347786","cluster_addr":"172.21.15.187:6809/1391347786","heartbeat_back_addr":"172.21.15.187:6818/1391347786","heartbeat_front_addr":"172.21.15.187:6813/1391347786","state":["exists","up"]},{"osd":7,"uuid":"775dae05-54f5-4ad4-bbce-92863c6ed2ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6825","nonce":1220065622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6827","nonce":1220065622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6831","nonce":1220065622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6829","nonce":1220065622}]},"public_addr":"172.21.15.187:6825/1220065622","cluster_addr":"172.21.15.187:6827/1220065622","heartbeat_back_addr":"172.21.15.187:6831/1220065622","heartbeat_front_addr":"172.21.15.187:6829/1220065622","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:05.983520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.299516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.067323+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.307748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.018742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.258775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.010912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.295355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-26T22:48:16.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:16.982+0000 7fcf5a800640 1 -- 172.21.15.131:0/1538344731 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fcf3406be30 msgr2=0x7fcf3406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:16.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:16.982+0000 7fcf5a800640 1 --2- 172.21.15.131:0/1538344731 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fcf3406be30 0x7fcf3406e2f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fcf44032000 tx=0x7fcf44034040 comp rx=0 tx=0).stop 2024-05-26T22:48:16.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:16.982+0000 7fcf5a800640 1 -- 172.21.15.131:0/1538344731 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf54140b60 msgr2=0x7fcf5414b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:16.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:16.982+0000 7fcf5a800640 1 --2- 172.21.15.131:0/1538344731 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf54140b60 0x7fcf5414b6b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fcf400119d0 tx=0x7fcf40011dd0 comp rx=0 tx=0).stop 2024-05-26T22:48:16.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:16.983+0000 7fcf5a800640 1 -- 172.21.15.131:0/1538344731 shutdown_connections 2024-05-26T22:48:16.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:16.983+0000 7fcf5a800640 1 --2- 172.21.15.131:0/1538344731 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcf541f6340 0x7fcf540b06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:16.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:16.983+0000 7fcf5a800640 1 --2- 172.21.15.131:0/1538344731 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fcf3406be30 0x7fcf3406e2f0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:16.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:16.983+0000 7fcf5a800640 1 --2- 172.21.15.131:0/1538344731 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf54140b60 0x7fcf5414b6b0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:16.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:16.983+0000 7fcf5a800640 1 --2- 172.21.15.131:0/1538344731 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcf380039e0 0x7fcf54151030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:16.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:16.983+0000 7fcf5a800640 1 -- 172.21.15.131:0/1538344731 >> 172.21.15.131:0/1538344731 conn(0x7fcf540b1e90 msgr2=0x7fcf5415e980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:16.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:16.983+0000 7fcf5a800640 1 -- 172.21.15.131:0/1538344731 shutdown_connections 2024-05-26T22:48:16.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:16.983+0000 7fcf5a800640 1 -- 172.21.15.131:0/1538344731 wait complete. 2024-05-26T22:48:17.000 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-26T22:48:10.463339+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-26T22:48:17.000 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-26T22:48:17.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.190+0000 7f0ef9000640 1 Processor -- start 2024-05-26T22:48:17.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.190+0000 7f0ef9000640 1 -- start start 2024-05-26T22:48:17.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.192+0000 7f0ef9000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ef4145a10 0x7f0ef4072180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:17.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.192+0000 7f0ef9000640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f0ef40712d0 0x7f0ef406d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:17.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.192+0000 7f0ef9000640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ef4072d60 con 0x7f0ef406d790 2024-05-26T22:48:17.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.192+0000 7f0ef9000640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f0ef415c760 con 0x7f0ef40712d0 2024-05-26T22:48:17.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.192+0000 7f0ef9000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f0ef4072a80 con 0x7f0ef4145a10 2024-05-26T22:48:17.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.192+0000 7f0ef3400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ef4145a10 0x7f0ef4072180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:17.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.192+0000 7f0ef2a00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f0ef40712d0 0x7f0ef406d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:17.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.192+0000 7f0ef3400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ef4145a10 0x7f0ef4072180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.131:35814/0 (socket says 172.21.15.131:35814) 2024-05-26T22:48:17.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.192+0000 7f0ef3400640 1 -- 172.21.15.131:0/1397082743 learned_addr learned my addr 172.21.15.131:0/1397082743 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:17.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.193+0000 7f0ef3400640 1 -- 172.21.15.131:0/1397082743 >> v2:172.21.15.187:3300/0 conn(0x7f0ef40712d0 msgr2=0x7f0ef406d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:17.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.193+0000 7f0ef3400640 1 --2- 172.21.15.131:0/1397082743 >> v2:172.21.15.187:3300/0 conn(0x7f0ef40712d0 0x7f0ef406d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:17.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.193+0000 7f0ef3400640 1 -- 172.21.15.131:0/1397082743 >> v1:172.21.15.187:6789/0 conn(0x7f0ef406d790 legacy=0x7f0ef415fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:17.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.193+0000 7f0ef3400640 1 -- 172.21.15.131:0/1397082743 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ef40ceab0 con 0x7f0ef4145a10 2024-05-26T22:48:17.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.194+0000 7f0ef3400640 1 --2- 172.21.15.131:0/1397082743 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ef4145a10 0x7f0ef4072180 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0ee4009870 tx=0x7f0ee402a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70607314d6c6be96 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:17.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.195+0000 7f0ef2000640 1 -- 172.21.15.131:0/1397082743 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee4037070 con 0x7f0ef4145a10 2024-05-26T22:48:17.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.195+0000 7f0ef2000640 1 -- 172.21.15.131:0/1397082743 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ee4032440 con 0x7f0ef4145a10 2024-05-26T22:48:17.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.195+0000 7f0ef2000640 1 -- 172.21.15.131:0/1397082743 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee4032810 con 0x7f0ef4145a10 2024-05-26T22:48:17.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.195+0000 7f0ef9000640 1 -- 172.21.15.131:0/1397082743 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ef4145a10 msgr2=0x7f0ef4072180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:17.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.195+0000 7f0ef9000640 1 --2- 172.21.15.131:0/1397082743 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ef4145a10 0x7f0ef4072180 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0ee4009870 tx=0x7f0ee402a270 comp rx=0 tx=0).stop 2024-05-26T22:48:17.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.195+0000 7f0ef9000640 1 -- 172.21.15.131:0/1397082743 shutdown_connections 2024-05-26T22:48:17.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.195+0000 7f0ef9000640 1 --2- 172.21.15.131:0/1397082743 >> v2:172.21.15.187:3300/0 conn(0x7f0ef40712d0 0x7f0ef406d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:17.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.195+0000 7f0ef9000640 1 --2- 172.21.15.131:0/1397082743 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ef4145a10 0x7f0ef4072180 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:17.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.195+0000 7f0ef9000640 1 -- 172.21.15.131:0/1397082743 >> 172.21.15.131:0/1397082743 conn(0x7f0ef40b5660 msgr2=0x7f0ef40b5a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:17.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.195+0000 7f0ef9000640 1 -- 172.21.15.131:0/1397082743 shutdown_connections 2024-05-26T22:48:17.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.196+0000 7f0ef9000640 1 -- 172.21.15.131:0/1397082743 wait complete. 2024-05-26T22:48:17.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.197+0000 7f0ef9000640 1 Processor -- start 2024-05-26T22:48:17.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.197+0000 7f0ef9000640 1 -- start start 2024-05-26T22:48:17.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.198+0000 7f0ef9000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ef406d790 0x7f0ef41589d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:17.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.198+0000 7f0ef9000640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ef40712d0 0x7f0ef413dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:17.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.198+0000 7f0ef9000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ef4145a10 0x7f0ef413e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:17.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.198+0000 7f0ef9000640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef4145df0 con 0x7f0ef406d790 2024-05-26T22:48:17.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.198+0000 7f0ef9000640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef40cf070 con 0x7f0ef40712d0 2024-05-26T22:48:17.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.198+0000 7f0ef9000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f0ef40726c0 con 0x7f0ef4145a10 2024-05-26T22:48:17.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.199+0000 7f0ef3400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ef406d790 0x7f0ef41589d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:17.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.199+0000 7f0ef3400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ef406d790 0x7f0ef41589d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33876/0 (socket says 172.21.15.131:33876) 2024-05-26T22:48:17.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.199+0000 7f0ef3400640 1 -- 172.21.15.131:0/1351670149 learned_addr learned my addr 172.21.15.131:0/1351670149 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:17.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.199+0000 7f0ef3e00640 1 --2- 172.21.15.131:0/1351670149 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ef4145a10 0x7f0ef413e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:17.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.199+0000 7f0ef2a00640 1 --2- 172.21.15.131:0/1351670149 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ef40712d0 0x7f0ef413dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:17.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.199+0000 7f0ef3400640 1 -- 172.21.15.131:0/1351670149 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ef4145a10 msgr2=0x7f0ef413e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:17.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.199+0000 7f0ef3400640 1 --2- 172.21.15.131:0/1351670149 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ef4145a10 0x7f0ef413e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:17.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.199+0000 7f0ef3400640 1 -- 172.21.15.131:0/1351670149 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ef40712d0 msgr2=0x7f0ef413dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:17.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.199+0000 7f0ef3400640 1 --2- 172.21.15.131:0/1351670149 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ef40712d0 0x7f0ef413dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:17.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.199+0000 7f0ef3400640 1 -- 172.21.15.131:0/1351670149 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ef4140f30 con 0x7f0ef406d790 2024-05-26T22:48:17.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.199+0000 7f0ef3e00640 1 --2- 172.21.15.131:0/1351670149 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ef4145a10 0x7f0ef413e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:48:17.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.199+0000 7f0ef2a00640 1 --2- 172.21.15.131:0/1351670149 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ef40712d0 0x7f0ef413dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:48:17.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.199+0000 7f0ef3400640 1 --2- 172.21.15.131:0/1351670149 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ef406d790 0x7f0ef41589d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f0ee40090b0 tx=0x7f0ee40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:17.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.200+0000 7f0eeb400640 1 -- 172.21.15.131:0/1351670149 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee4041030 con 0x7f0ef406d790 2024-05-26T22:48:17.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.200+0000 7f0eeb400640 1 -- 172.21.15.131:0/1351670149 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ee4030440 con 0x7f0ef406d790 2024-05-26T22:48:17.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.200+0000 7f0ef9000640 1 -- 172.21.15.131:0/1351670149 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ef4141210 con 0x7f0ef406d790 2024-05-26T22:48:17.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.200+0000 7f0eeb400640 1 -- 172.21.15.131:0/1351670149 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee4037040 con 0x7f0ef406d790 2024-05-26T22:48:17.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.200+0000 7f0ef9000640 1 -- 172.21.15.131:0/1351670149 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ef4158f10 con 0x7f0ef406d790 2024-05-26T22:48:17.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.201+0000 7f0ef9000640 1 -- 172.21.15.131:0/1351670149 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ef40694e0 con 0x7f0ef406d790 2024-05-26T22:48:17.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.202+0000 7f0eeb400640 1 -- 172.21.15.131:0/1351670149 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0ee4044070 con 0x7f0ef406d790 2024-05-26T22:48:17.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.203+0000 7f0eeb400640 1 --2- 172.21.15.131:0/1351670149 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f0ed406be30 0x7f0ed406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:17.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.203+0000 7f0eeb400640 1 -- 172.21.15.131:0/1351670149 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f0ee40ab1d0 con 0x7f0ef406d790 2024-05-26T22:48:17.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.203+0000 7f0ef2a00640 1 --2- 172.21.15.131:0/1351670149 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f0ed406be30 0x7f0ed406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:17.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.203+0000 7f0ef2a00640 1 --2- 172.21.15.131:0/1351670149 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f0ed406be30 0x7f0ed406e2f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f0ef4143960 tx=0x7f0ee0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:17.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.208+0000 7f0eeb400640 1 -- 172.21.15.131:0/1351670149 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f0ee4079ec0 con 0x7f0ef406d790 2024-05-26T22:48:17.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.401+0000 7f0ef9000640 1 -- 172.21.15.131:0/1351670149 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0ef40b5ae0 con 0x7f0ef406d790 2024-05-26T22:48:17.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.402+0000 7f0eeb400640 1 -- 172.21.15.131:0/1351670149 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) ==== 93+0+10 (secure 0 0 0) 0x7f0ee40797c0 con 0x7f0ef406d790 2024-05-26T22:48:17.401 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2024-05-26T22:48:17.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.404+0000 7f0ef9000640 1 -- 172.21.15.131:0/1351670149 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f0ed406be30 msgr2=0x7f0ed406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:17.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.404+0000 7f0ef9000640 1 --2- 172.21.15.131:0/1351670149 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f0ed406be30 0x7f0ed406e2f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f0ef4143960 tx=0x7f0ee0008040 comp rx=0 tx=0).stop 2024-05-26T22:48:17.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.404+0000 7f0ef9000640 1 -- 172.21.15.131:0/1351670149 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ef406d790 msgr2=0x7f0ef41589d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:17.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.404+0000 7f0ef9000640 1 --2- 172.21.15.131:0/1351670149 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ef406d790 0x7f0ef41589d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f0ee40090b0 tx=0x7f0ee40046c0 comp rx=0 tx=0).stop 2024-05-26T22:48:17.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.404+0000 7f0ef9000640 1 -- 172.21.15.131:0/1351670149 shutdown_connections 2024-05-26T22:48:17.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.404+0000 7f0ef9000640 1 --2- 172.21.15.131:0/1351670149 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ef4145a10 0x7f0ef413e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:17.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.404+0000 7f0ef9000640 1 --2- 172.21.15.131:0/1351670149 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f0ed406be30 0x7f0ed406e2f0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:17.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.404+0000 7f0ef9000640 1 --2- 172.21.15.131:0/1351670149 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ef40712d0 0x7f0ef413dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:17.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.404+0000 7f0ef9000640 1 --2- 172.21.15.131:0/1351670149 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ef406d790 0x7f0ef41589d0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:17.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.404+0000 7f0ef9000640 1 -- 172.21.15.131:0/1351670149 >> 172.21.15.131:0/1351670149 conn(0x7f0ef40b5660 msgr2=0x7f0ef4071ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:17.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.405+0000 7f0ef9000640 1 -- 172.21.15.131:0/1351670149 shutdown_connections 2024-05-26T22:48:17.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.405+0000 7f0ef9000640 1 -- 172.21.15.131:0/1351670149 wait complete. 2024-05-26T22:48:17.422 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-26T22:48:17.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.620+0000 7f43eaa00640 1 Processor -- start 2024-05-26T22:48:17.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.620+0000 7f43eaa00640 1 -- start start 2024-05-26T22:48:17.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.621+0000 7f43eaa00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f43e415e320 0x7f43e415e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:17.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.621+0000 7f43eaa00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f43e4071500 0x7f43e40ce2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:17.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.621+0000 7f43eaa00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43e415ec40 con 0x7f43e4150f70 2024-05-26T22:48:17.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.621+0000 7f43eaa00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f43e40cfbe0 con 0x7f43e4071500 2024-05-26T22:48:17.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.621+0000 7f43eaa00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f43e40ceb40 con 0x7f43e415e320 2024-05-26T22:48:17.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.622+0000 7f43ea000640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f43e4150f70 0x7f43e41513c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:40438/0 (socket says 172.21.15.131:40438) 2024-05-26T22:48:17.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.622+0000 7f43ea000640 1 -- 172.21.15.131:0/3066737835 learned_addr learned my addr 172.21.15.131:0/3066737835 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:48:17.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.622+0000 7f43e8c00640 1 --2- 172.21.15.131:0/3066737835 >> v2:172.21.15.187:3300/0 conn(0x7f43e4071500 0x7f43e40ce2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:17.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.622+0000 7f43e3e00640 1 -- 172.21.15.131:0/3066737835 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1323674093 0 0) 0x7f43e415ec40 con 0x7f43e4150f70 2024-05-26T22:48:17.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.622+0000 7f43e9600640 1 --2- 172.21.15.131:0/3066737835 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f43e415e320 0x7f43e415e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:17.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.622+0000 7f43e3e00640 1 -- 172.21.15.131:0/3066737835 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f43cc003440 con 0x7f43e4150f70 2024-05-26T22:48:17.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.622+0000 7f43e3e00640 1 -- 172.21.15.131:0/3066737835 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2722417325 0 0) 0x7f43cc003440 con 0x7f43e4150f70 2024-05-26T22:48:17.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.622+0000 7f43e3e00640 1 -- 172.21.15.131:0/3066737835 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f43e415e320 msgr2=0x7f43e415e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:17.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.622+0000 7f43e3e00640 1 --2- 172.21.15.131:0/3066737835 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f43e415e320 0x7f43e415e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:17.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.622+0000 7f43e3e00640 1 -- 172.21.15.131:0/3066737835 >> v2:172.21.15.187:3300/0 conn(0x7f43e4071500 msgr2=0x7f43e40ce2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:17.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.622+0000 7f43e3e00640 1 --2- 172.21.15.131:0/3066737835 >> v2:172.21.15.187:3300/0 conn(0x7f43e4071500 0x7f43e40ce2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:17.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.623+0000 7f43e3e00640 1 -- 172.21.15.131:0/3066737835 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43e40cee20 con 0x7f43e4150f70 2024-05-26T22:48:17.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.623+0000 7f43e3e00640 1 -- 172.21.15.131:0/3066737835 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7f43dc0019f0 con 0x7f43e4150f70 2024-05-26T22:48:17.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.623+0000 7f43e3e00640 1 -- 172.21.15.131:0/3066737835 >> v1:172.21.15.131:6789/0 conn(0x7f43e4150f70 legacy=0x7f43e41513c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:17.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.623+0000 7f43e3e00640 1 --2- 172.21.15.131:0/3066737835 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f43e415e320 0x7f43cc003aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:17.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.623+0000 7f43e3e00640 1 --2- 172.21.15.131:0/3066737835 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f43cc004020 0x7f43cc00a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:17.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.623+0000 7f43e3e00640 1 --2- 172.21.15.131:0/3066737835 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f43cc00ae40 0x7f43cc00d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:17.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.623+0000 7f43e3e00640 1 -- 172.21.15.131:0/3066737835 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f43e40cfbe0 con 0x7f43cc00ae40 2024-05-26T22:48:17.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.623+0000 7f43e3e00640 1 -- 172.21.15.131:0/3066737835 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f43e40ceb40 con 0x7f43cc004020 2024-05-26T22:48:17.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.623+0000 7f43e3e00640 1 -- 172.21.15.131:0/3066737835 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f43cc00db70 con 0x7f43e415e320 2024-05-26T22:48:17.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.623+0000 7f43ea000640 1 --2- 172.21.15.131:0/3066737835 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f43cc00ae40 0x7f43cc00d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:17.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.623+0000 7f43e9600640 1 --2- 172.21.15.131:0/3066737835 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f43e415e320 0x7f43cc003aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:17.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.623+0000 7f43ea000640 1 -- 172.21.15.131:0/3066737835 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f43e415e320 msgr2=0x7f43cc003aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:17.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.623+0000 7f43e8c00640 1 --2- 172.21.15.131:0/3066737835 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f43cc004020 0x7f43cc00a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:17.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.623+0000 7f43ea000640 1 --2- 172.21.15.131:0/3066737835 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f43e415e320 0x7f43cc003aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:17.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.623+0000 7f43ea000640 1 -- 172.21.15.131:0/3066737835 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f43cc004020 msgr2=0x7f43cc00a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:17.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.623+0000 7f43ea000640 1 --2- 172.21.15.131:0/3066737835 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f43cc004020 0x7f43cc00a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:17.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.623+0000 7f43ea000640 1 -- 172.21.15.131:0/3066737835 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43cc003440 con 0x7f43cc00ae40 2024-05-26T22:48:17.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.624+0000 7f43ea000640 1 --2- 172.21.15.131:0/3066737835 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f43cc00ae40 0x7f43cc00d2b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f43dc00a880 tx=0x7f43cc003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa63b1a0c4b935dc server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:17.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.625+0000 7f43e3e00640 1 -- 172.21.15.131:0/3066737835 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43dc00a4a0 con 0x7f43cc00ae40 2024-05-26T22:48:17.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.625+0000 7f43e3e00640 1 -- 172.21.15.131:0/3066737835 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43dc02f070 con 0x7f43cc00ae40 2024-05-26T22:48:17.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.625+0000 7f43e3e00640 1 -- 172.21.15.131:0/3066737835 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43dc02bd40 con 0x7f43cc00ae40 2024-05-26T22:48:17.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.625+0000 7f43eaa00640 1 -- 172.21.15.131:0/3066737835 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f43cc00ae40 msgr2=0x7f43cc00d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:17.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.625+0000 7f43eaa00640 1 --2- 172.21.15.131:0/3066737835 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f43cc00ae40 0x7f43cc00d2b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f43dc00a880 tx=0x7f43cc003730 comp rx=0 tx=0).stop 2024-05-26T22:48:17.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.625+0000 7f43ea000640 1 -- 172.21.15.131:0/3066737835 reap_dead start 2024-05-26T22:48:17.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.625+0000 7f43eaa00640 1 -- 172.21.15.131:0/3066737835 shutdown_connections 2024-05-26T22:48:17.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.625+0000 7f43eaa00640 1 -- 172.21.15.131:0/3066737835 >> 172.21.15.131:0/3066737835 conn(0x7f43e40cff40 msgr2=0x7f43e40d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:17.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.625+0000 7f43eaa00640 1 -- 172.21.15.131:0/3066737835 shutdown_connections 2024-05-26T22:48:17.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.625+0000 7f43eaa00640 1 -- 172.21.15.131:0/3066737835 wait complete. 2024-05-26T22:48:17.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.627+0000 7f43eaa00640 1 Processor -- start 2024-05-26T22:48:17.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.627+0000 7f43eaa00640 1 -- start start 2024-05-26T22:48:17.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.628+0000 7f43eaa00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f43cc00ae40 0x7f43e41f0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:17.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.628+0000 7f43eaa00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f43e40b42b0 0x7f43e414d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:17.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.628+0000 7f43eaa00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f43e4151350 0x7f43e4151800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:17.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.628+0000 7f43eaa00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f43e40cf6c0 con 0x7f43e4151350 2024-05-26T22:48:17.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.628+0000 7f43eaa00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f43e40cf3e0 con 0x7f43e40b42b0 2024-05-26T22:48:17.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.628+0000 7f43eaa00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f43e40ce830 con 0x7f43cc00ae40 2024-05-26T22:48:17.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.628+0000 7f43ea000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f43e4151350 0x7f43e4151800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:17.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.628+0000 7f43ea000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f43e4151350 0x7f43e4151800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33894/0 (socket says 172.21.15.131:33894) 2024-05-26T22:48:17.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.628+0000 7f43ea000640 1 -- 172.21.15.131:0/1890757043 learned_addr learned my addr 172.21.15.131:0/1890757043 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:17.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.629+0000 7f43e8c00640 1 --2- 172.21.15.131:0/1890757043 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f43e40b42b0 0x7f43e414d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:17.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.629+0000 7f43ea000640 1 -- 172.21.15.131:0/1890757043 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f43cc00ae40 msgr2=0x7f43e41f0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:17.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.629+0000 7f43e9600640 1 --2- 172.21.15.131:0/1890757043 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f43cc00ae40 0x7f43e41f0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:17.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.629+0000 7f43ea000640 1 --2- 172.21.15.131:0/1890757043 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f43cc00ae40 0x7f43e41f0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:17.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.629+0000 7f43ea000640 1 -- 172.21.15.131:0/1890757043 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f43e40b42b0 msgr2=0x7f43e414d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:17.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.629+0000 7f43ea000640 1 --2- 172.21.15.131:0/1890757043 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f43e40b42b0 0x7f43e414d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:17.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.629+0000 7f43ea000640 1 -- 172.21.15.131:0/1890757043 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43e40b4a90 con 0x7f43e4151350 2024-05-26T22:48:17.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.629+0000 7f43e8c00640 1 --2- 172.21.15.131:0/1890757043 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f43e40b42b0 0x7f43e414d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:48:17.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.629+0000 7f43ea000640 1 --2- 172.21.15.131:0/1890757043 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f43e4151350 0x7f43e4151800 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f43dc00a9a0 tx=0x7f43e40ceff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:17.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.629+0000 7f43e1600640 1 -- 172.21.15.131:0/1890757043 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43dc044070 con 0x7f43e4151350 2024-05-26T22:48:17.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.630+0000 7f43e1600640 1 -- 172.21.15.131:0/1890757043 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43dc001a20 con 0x7f43e4151350 2024-05-26T22:48:17.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.630+0000 7f43e1600640 1 -- 172.21.15.131:0/1890757043 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43dc00a4a0 con 0x7f43e4151350 2024-05-26T22:48:17.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.630+0000 7f43eaa00640 1 -- 172.21.15.131:0/1890757043 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43e40b4d70 con 0x7f43e4151350 2024-05-26T22:48:17.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.630+0000 7f43eaa00640 1 -- 172.21.15.131:0/1890757043 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43e4152ff0 con 0x7f43e4151350 2024-05-26T22:48:17.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.631+0000 7f43eaa00640 1 -- 172.21.15.131:0/1890757043 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43e40694e0 con 0x7f43e4151350 2024-05-26T22:48:17.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.632+0000 7f43e1600640 1 -- 172.21.15.131:0/1890757043 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f43dc034670 con 0x7f43e4151350 2024-05-26T22:48:17.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.633+0000 7f43e1600640 1 --2- 172.21.15.131:0/1890757043 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f43c406be60 0x7f43c406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:17.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.633+0000 7f43e1600640 1 -- 172.21.15.131:0/1890757043 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f43dc0aaae0 con 0x7f43e4151350 2024-05-26T22:48:17.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.633+0000 7f43e9600640 1 --2- 172.21.15.131:0/1890757043 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f43c406be60 0x7f43c406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:17.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.634+0000 7f43e9600640 1 --2- 172.21.15.131:0/1890757043 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f43c406be60 0x7f43c406e320 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f43e4071b60 tx=0x7f43d8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:17.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.637+0000 7f43e1600640 1 -- 172.21.15.131:0/1890757043 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f43dc079740 con 0x7f43e4151350 2024-05-26T22:48:17.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.875+0000 7f43eaa00640 1 -- 172.21.15.131:0/1890757043 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f43e406e700 con 0x7f43e4151350 2024-05-26T22:48:17.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.876+0000 7f43e1600640 1 -- 172.21.15.131:0/1890757043 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3873 (secure 0 0 0) 0x7f43dc079040 con 0x7f43e4151350 2024-05-26T22:48:17.876 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:48:17.876 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/3266175650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6833","nonce":3266175650},{"type":"v1","addr":"172.21.15.187:6835","nonce":3266175650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/1862866305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1862866305},{"type":"v1","addr":"172.21.15.131:6835","nonce":1862866305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4269,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/618294324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":618294324},{"type":"v1","addr":"172.21.15.131:6837","nonce":618294324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4281,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6834/2547512415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":2547512415},{"type":"v1","addr":"172.21.15.187:6834","nonce":2547512415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-05-26T22:48:17.876 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 3 2024-05-26T22:48:17.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.878+0000 7f43eaa00640 1 -- 172.21.15.131:0/1890757043 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f43c406be60 msgr2=0x7f43c406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:17.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.878+0000 7f43eaa00640 1 --2- 172.21.15.131:0/1890757043 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f43c406be60 0x7f43c406e320 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f43e4071b60 tx=0x7f43d8008040 comp rx=0 tx=0).stop 2024-05-26T22:48:17.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.878+0000 7f43eaa00640 1 -- 172.21.15.131:0/1890757043 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f43e4151350 msgr2=0x7f43e4151800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:17.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.878+0000 7f43eaa00640 1 --2- 172.21.15.131:0/1890757043 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f43e4151350 0x7f43e4151800 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f43dc00a9a0 tx=0x7f43e40ceff0 comp rx=0 tx=0).stop 2024-05-26T22:48:17.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.878+0000 7f43eaa00640 1 -- 172.21.15.131:0/1890757043 shutdown_connections 2024-05-26T22:48:17.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.878+0000 7f43eaa00640 1 --2- 172.21.15.131:0/1890757043 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f43c406be60 0x7f43c406e320 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:17.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.878+0000 7f43eaa00640 1 --2- 172.21.15.131:0/1890757043 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f43e4151350 0x7f43e4151800 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:17.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.878+0000 7f43eaa00640 1 --2- 172.21.15.131:0/1890757043 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f43e40b42b0 0x7f43e414d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:17.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.878+0000 7f43eaa00640 1 --2- 172.21.15.131:0/1890757043 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f43cc00ae40 0x7f43e41f0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:17.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.878+0000 7f43eaa00640 1 -- 172.21.15.131:0/1890757043 >> 172.21.15.131:0/1890757043 conn(0x7f43e40cff40 msgr2=0x7f43e406d3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:17.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.879+0000 7f43eaa00640 1 -- 172.21.15.131:0/1890757043 shutdown_connections 2024-05-26T22:48:17.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:17.879+0000 7f43eaa00640 1 -- 172.21.15.131:0/1890757043 wait complete. 2024-05-26T22:48:17.897 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-26T22:48:18.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.083+0000 7fc352c00640 1 Processor -- start 2024-05-26T22:48:18.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.083+0000 7fc352c00640 1 -- start start 2024-05-26T22:48:18.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.085+0000 7fc352c00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc34c06ee20 0x7fc34c0b3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:18.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.085+0000 7fc352c00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc34c15cb20 con 0x7fc34c1569d0 2024-05-26T22:48:18.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.085+0000 7fc352c00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc34c15ce00 con 0x7fc34c0ad250 2024-05-26T22:48:18.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.085+0000 7fc352c00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc34c15d0e0 con 0x7fc34c06ee20 2024-05-26T22:48:18.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.085+0000 7fc350c00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fc34c1569d0 0x7fc34c160010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:40448/0 (socket says 172.21.15.131:40448) 2024-05-26T22:48:18.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.085+0000 7fc350c00640 1 -- 172.21.15.131:0/3233289287 learned_addr learned my addr 172.21.15.131:0/3233289287 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:48:18.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.086+0000 7fc34be00640 1 --2- 172.21.15.131:0/3233289287 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc34c06ee20 0x7fc34c0b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:18.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.086+0000 7fc34b400640 1 -- 172.21.15.131:0/3233289287 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3152850539 0 0) 0x7fc34c15cb20 con 0x7fc34c1569d0 2024-05-26T22:48:18.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.086+0000 7fc34b400640 1 -- 172.21.15.131:0/3233289287 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc330003520 con 0x7fc34c1569d0 2024-05-26T22:48:18.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.086+0000 7fc34b400640 1 -- 172.21.15.131:0/3233289287 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3476427484 0 0) 0x7fc330003520 con 0x7fc34c1569d0 2024-05-26T22:48:18.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.086+0000 7fc34b400640 1 -- 172.21.15.131:0/3233289287 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc34c06ee20 msgr2=0x7fc34c0b3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:18.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.086+0000 7fc34b400640 1 --2- 172.21.15.131:0/3233289287 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc34c06ee20 0x7fc34c0b3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:18.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.086+0000 7fc34b400640 1 -- 172.21.15.131:0/3233289287 >> v1:172.21.15.187:6789/0 conn(0x7fc34c0ad250 legacy=0x7fc34c1562c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:18.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.086+0000 7fc34b400640 1 -- 172.21.15.131:0/3233289287 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc34c15d3c0 con 0x7fc34c1569d0 2024-05-26T22:48:18.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.086+0000 7fc34b400640 1 -- 172.21.15.131:0/3233289287 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7fc3440019f0 con 0x7fc34c1569d0 2024-05-26T22:48:18.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.086+0000 7fc34b400640 1 -- 172.21.15.131:0/3233289287 >> v1:172.21.15.131:6789/0 conn(0x7fc34c1569d0 legacy=0x7fc34c160010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:18.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.086+0000 7fc34b400640 1 --2- 172.21.15.131:0/3233289287 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc3300039e0 0x7fc330003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:18.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.086+0000 7fc34b400640 1 --2- 172.21.15.131:0/3233289287 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc330004800 0x7fc330004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:18.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.086+0000 7fc34b400640 1 --2- 172.21.15.131:0/3233289287 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc34c06ee20 0x7fc3300051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:18.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.086+0000 7fc34b400640 1 -- 172.21.15.131:0/3233289287 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc34c15d0e0 con 0x7fc3300039e0 2024-05-26T22:48:18.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.086+0000 7fc34b400640 1 -- 172.21.15.131:0/3233289287 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc33000ba70 con 0x7fc330004800 2024-05-26T22:48:18.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.086+0000 7fc34b400640 1 -- 172.21.15.131:0/3233289287 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc33000bc70 con 0x7fc34c06ee20 2024-05-26T22:48:18.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.087+0000 7fc34be00640 1 --2- 172.21.15.131:0/3233289287 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc3300039e0 0x7fc330003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:18.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.087+0000 7fc343400640 1 --2- 172.21.15.131:0/3233289287 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc330004800 0x7fc330004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:18.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.087+0000 7fc350c00640 1 --2- 172.21.15.131:0/3233289287 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc34c06ee20 0x7fc3300051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:18.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.087+0000 7fc34be00640 1 -- 172.21.15.131:0/3233289287 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc34c06ee20 msgr2=0x7fc3300051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:18.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.087+0000 7fc34be00640 1 --2- 172.21.15.131:0/3233289287 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc34c06ee20 0x7fc3300051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:18.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.087+0000 7fc34be00640 1 -- 172.21.15.131:0/3233289287 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc330004800 msgr2=0x7fc330004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:18.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.087+0000 7fc34be00640 1 --2- 172.21.15.131:0/3233289287 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc330004800 0x7fc330004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:18.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.087+0000 7fc34be00640 1 -- 172.21.15.131:0/3233289287 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc330003520 con 0x7fc3300039e0 2024-05-26T22:48:18.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.088+0000 7fc34be00640 1 --2- 172.21.15.131:0/3233289287 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc3300039e0 0x7fc330003e00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc33c009a70 tx=0x7fc330005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d684ea36065cc2c server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:18.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.088+0000 7fc34b400640 1 -- 172.21.15.131:0/3233289287 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc33c036070 con 0x7fc3300039e0 2024-05-26T22:48:18.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.088+0000 7fc34b400640 1 -- 172.21.15.131:0/3233289287 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc33c031400 con 0x7fc3300039e0 2024-05-26T22:48:18.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.088+0000 7fc34b400640 1 -- 172.21.15.131:0/3233289287 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc33c009670 con 0x7fc3300039e0 2024-05-26T22:48:18.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.089+0000 7fc352c00640 1 -- 172.21.15.131:0/3233289287 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc3300039e0 msgr2=0x7fc330003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:18.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.089+0000 7fc352c00640 1 --2- 172.21.15.131:0/3233289287 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc3300039e0 0x7fc330003e00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc33c009a70 tx=0x7fc330005830 comp rx=0 tx=0).stop 2024-05-26T22:48:18.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.089+0000 7fc350c00640 1 -- 172.21.15.131:0/3233289287 reap_dead start 2024-05-26T22:48:18.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.089+0000 7fc352c00640 1 -- 172.21.15.131:0/3233289287 shutdown_connections 2024-05-26T22:48:18.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.089+0000 7fc352c00640 1 -- 172.21.15.131:0/3233289287 >> 172.21.15.131:0/3233289287 conn(0x7fc34c0b1e90 msgr2=0x7fc34c0b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:18.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.089+0000 7fc352c00640 1 -- 172.21.15.131:0/3233289287 shutdown_connections 2024-05-26T22:48:18.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.089+0000 7fc352c00640 1 -- 172.21.15.131:0/3233289287 wait complete. 2024-05-26T22:48:18.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.091+0000 7fc352c00640 1 Processor -- start 2024-05-26T22:48:18.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.091+0000 7fc352c00640 1 -- start start 2024-05-26T22:48:18.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.092+0000 7fc352c00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc3300039e0 0x7fc34c1f3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:18.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.092+0000 7fc352c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc34c150d50 0x7fc34c1511d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:18.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.092+0000 7fc352c00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc34c15b250 0x7fc34c15b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:18.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.092+0000 7fc352c00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc34c15c900 con 0x7fc34c150d50 2024-05-26T22:48:18.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.092+0000 7fc352c00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc34c15c780 con 0x7fc34c15b250 2024-05-26T22:48:18.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.092+0000 7fc352c00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc34c06f200 con 0x7fc3300039e0 2024-05-26T22:48:18.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.092+0000 7fc343400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc34c150d50 0x7fc34c1511d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:18.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.092+0000 7fc343400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc34c150d50 0x7fc34c1511d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33910/0 (socket says 172.21.15.131:33910) 2024-05-26T22:48:18.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.092+0000 7fc343400640 1 -- 172.21.15.131:0/2451837306 learned_addr learned my addr 172.21.15.131:0/2451837306 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:18.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.092+0000 7fc34be00640 1 --2- 172.21.15.131:0/2451837306 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc3300039e0 0x7fc34c1f3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:18.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.092+0000 7fc350c00640 1 --2- 172.21.15.131:0/2451837306 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc34c15b250 0x7fc34c15b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:18.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.092+0000 7fc343400640 1 -- 172.21.15.131:0/2451837306 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc3300039e0 msgr2=0x7fc34c1f3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:18.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.092+0000 7fc343400640 1 --2- 172.21.15.131:0/2451837306 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc3300039e0 0x7fc34c1f3bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:18.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.092+0000 7fc343400640 1 -- 172.21.15.131:0/2451837306 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc34c15b250 msgr2=0x7fc34c15b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:18.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.092+0000 7fc343400640 1 --2- 172.21.15.131:0/2451837306 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc34c15b250 0x7fc34c15b6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:18.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.092+0000 7fc343400640 1 -- 172.21.15.131:0/2451837306 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc34c071630 con 0x7fc34c150d50 2024-05-26T22:48:18.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.093+0000 7fc343400640 1 --2- 172.21.15.131:0/2451837306 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc34c150d50 0x7fc34c1511d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc33400e580 tx=0x7fc33400ea40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:18.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.093+0000 7fc348c00640 1 -- 172.21.15.131:0/2451837306 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3340052e0 con 0x7fc34c150d50 2024-05-26T22:48:18.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.093+0000 7fc348c00640 1 -- 172.21.15.131:0/2451837306 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc334018070 con 0x7fc34c150d50 2024-05-26T22:48:18.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.093+0000 7fc348c00640 1 -- 172.21.15.131:0/2451837306 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc334003810 con 0x7fc34c150d50 2024-05-26T22:48:18.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.093+0000 7fc352c00640 1 -- 172.21.15.131:0/2451837306 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc34c1518c0 con 0x7fc34c150d50 2024-05-26T22:48:18.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.093+0000 7fc352c00640 1 -- 172.21.15.131:0/2451837306 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc34c151e80 con 0x7fc34c150d50 2024-05-26T22:48:18.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.094+0000 7fc352c00640 1 -- 172.21.15.131:0/2451837306 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc34c0694e0 con 0x7fc34c150d50 2024-05-26T22:48:18.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.095+0000 7fc348c00640 1 -- 172.21.15.131:0/2451837306 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc3340039b0 con 0x7fc34c150d50 2024-05-26T22:48:18.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.096+0000 7fc348c00640 1 --2- 172.21.15.131:0/2451837306 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc32806be60 0x7fc32806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:18.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.096+0000 7fc348c00640 1 -- 172.21.15.131:0/2451837306 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fc33408a810 con 0x7fc34c150d50 2024-05-26T22:48:18.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.096+0000 7fc34be00640 1 --2- 172.21.15.131:0/2451837306 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc32806be60 0x7fc32806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:18.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.097+0000 7fc34be00640 1 --2- 172.21.15.131:0/2451837306 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc32806be60 0x7fc32806e320 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc33c033000 tx=0x7fc33c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:18.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.100+0000 7fc348c00640 1 -- 172.21.15.131:0/2451837306 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fc334059480 con 0x7fc34c150d50 2024-05-26T22:48:18.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.284+0000 7fc352c00640 1 -- 172.21.15.131:0/2451837306 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc34c15daa0 con 0x7fc34c150d50 2024-05-26T22:48:18.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.286+0000 7fc348c00640 1 -- 172.21.15.131:0/2451837306 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12908 (secure 0 0 0) 0x7fc334058d80 con 0x7fc34c150d50 2024-05-26T22:48:18.286 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:48:18.287 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":16,"fsid":"46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f","created":"2024-05-26T22:47:57.288905+0000","modified":"2024-05-26T22:48:12.655278+0000","last_up_change":"2024-05-26T22:48:09.602207+0000","last_in_change":"2024-05-26T22:48:00.687341+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-26T22:48:10.463339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"814eaf62-a18d-494b-8e27-fd8fb88adea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6801","nonce":2350940051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6803","nonce":2350940051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6807","nonce":2350940051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6805","nonce":2350940051}]},"public_addr":"172.21.15.131:6801/2350940051","cluster_addr":"172.21.15.131:6803/2350940051","heartbeat_back_addr":"172.21.15.131:6807/2350940051","heartbeat_front_addr":"172.21.15.131:6805/2350940051","state":["exists","up"]},{"osd":1,"uuid":"42e66287-f6c6-4e26-af3c-6dfaa3536d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6825","nonce":1525528247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6827","nonce":1525528247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6831","nonce":1525528247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6829","nonce":1525528247}]},"public_addr":"172.21.15.131:6825/1525528247","cluster_addr":"172.21.15.131:6827/1525528247","heartbeat_back_addr":"172.21.15.131:6831/1525528247","heartbeat_front_addr":"172.21.15.131:6829/1525528247","state":["exists","up"]},{"osd":2,"uuid":"0da34673-04a2-4fd0-b57a-0ff2b679b043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6817","nonce":3627842210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6819","nonce":3627842210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6823","nonce":3627842210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6821","nonce":3627842210}]},"public_addr":"172.21.15.131:6817/3627842210","cluster_addr":"172.21.15.131:6819/3627842210","heartbeat_back_addr":"172.21.15.131:6823/3627842210","heartbeat_front_addr":"172.21.15.131:6821/3627842210","state":["exists","up"]},{"osd":3,"uuid":"cf117a27-d6b2-43d3-ae52-b47988bf8eed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6809","nonce":350790591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6811","nonce":350790591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6815","nonce":350790591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6813","nonce":350790591}]},"public_addr":"172.21.15.131:6809/350790591","cluster_addr":"172.21.15.131:6811/350790591","heartbeat_back_addr":"172.21.15.131:6815/350790591","heartbeat_front_addr":"172.21.15.131:6813/350790591","state":["exists","up"]},{"osd":4,"uuid":"a092ccfc-be89-4a84-88d5-0f781fa715eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6816","nonce":244560995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6819","nonce":244560995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6823","nonce":244560995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6821","nonce":244560995}]},"public_addr":"172.21.15.187:6816/244560995","cluster_addr":"172.21.15.187:6819/244560995","heartbeat_back_addr":"172.21.15.187:6823/244560995","heartbeat_front_addr":"172.21.15.187:6821/244560995","state":["exists","up"]},{"osd":5,"uuid":"7513ba69-dd00-4a35-b190-9e8cc3b2bdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6801","nonce":324619398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6803","nonce":324619398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6812","nonce":324619398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6807","nonce":324619398}]},"public_addr":"172.21.15.187:6801/324619398","cluster_addr":"172.21.15.187:6803/324619398","heartbeat_back_addr":"172.21.15.187:6812/324619398","heartbeat_front_addr":"172.21.15.187:6807/324619398","state":["exists","up"]},{"osd":6,"uuid":"50081810-c093-4d2f-a4d7-b7c695c5400d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6806","nonce":1391347786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6809","nonce":1391347786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6818","nonce":1391347786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6813","nonce":1391347786}]},"public_addr":"172.21.15.187:6806/1391347786","cluster_addr":"172.21.15.187:6809/1391347786","heartbeat_back_addr":"172.21.15.187:6818/1391347786","heartbeat_front_addr":"172.21.15.187:6813/1391347786","state":["exists","up"]},{"osd":7,"uuid":"775dae05-54f5-4ad4-bbce-92863c6ed2ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6825","nonce":1220065622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6827","nonce":1220065622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6831","nonce":1220065622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6829","nonce":1220065622}]},"public_addr":"172.21.15.187:6825/1220065622","cluster_addr":"172.21.15.187:6827/1220065622","heartbeat_back_addr":"172.21.15.187:6831/1220065622","heartbeat_front_addr":"172.21.15.187:6829/1220065622","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:05.983520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.299516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.067323+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.307748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.018742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.258775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.010912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.295355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-26T22:48:18.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.288+0000 7fc352c00640 1 -- 172.21.15.131:0/2451837306 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc32806be60 msgr2=0x7fc32806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:18.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.288+0000 7fc352c00640 1 --2- 172.21.15.131:0/2451837306 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc32806be60 0x7fc32806e320 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc33c033000 tx=0x7fc33c008030 comp rx=0 tx=0).stop 2024-05-26T22:48:18.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.289+0000 7fc352c00640 1 -- 172.21.15.131:0/2451837306 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc34c150d50 msgr2=0x7fc34c1511d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:18.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.289+0000 7fc352c00640 1 --2- 172.21.15.131:0/2451837306 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc34c150d50 0x7fc34c1511d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc33400e580 tx=0x7fc33400ea40 comp rx=0 tx=0).stop 2024-05-26T22:48:18.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.289+0000 7fc352c00640 1 -- 172.21.15.131:0/2451837306 shutdown_connections 2024-05-26T22:48:18.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.289+0000 7fc352c00640 1 --2- 172.21.15.131:0/2451837306 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc32806be60 0x7fc32806e320 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:18.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.289+0000 7fc352c00640 1 --2- 172.21.15.131:0/2451837306 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc34c15b250 0x7fc34c15b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:18.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.289+0000 7fc352c00640 1 --2- 172.21.15.131:0/2451837306 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc34c150d50 0x7fc34c1511d0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:18.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.289+0000 7fc352c00640 1 --2- 172.21.15.131:0/2451837306 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc3300039e0 0x7fc34c1f3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:18.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.289+0000 7fc352c00640 1 -- 172.21.15.131:0/2451837306 >> 172.21.15.131:0/2451837306 conn(0x7fc34c0b1e90 msgr2=0x7fc34c0b1720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:18.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.289+0000 7fc352c00640 1 -- 172.21.15.131:0/2451837306 shutdown_connections 2024-05-26T22:48:18.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.289+0000 7fc352c00640 1 -- 172.21.15.131:0/2451837306 wait complete. 2024-05-26T22:48:18.306 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-26T22:48:10.463339+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-26T22:48:18.306 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-26T22:48:18.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.505+0000 7fa4cea00640 1 Processor -- start 2024-05-26T22:48:18.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.505+0000 7fa4cea00640 1 -- start start 2024-05-26T22:48:18.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.507+0000 7fa4cea00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fa4c80d0460 0x7fa4c806d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:18.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.507+0000 7fa4cea00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4c8160180 con 0x7fa4c806f420 2024-05-26T22:48:18.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.507+0000 7fa4cea00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4c8160460 con 0x7fa4c80b5450 2024-05-26T22:48:18.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.507+0000 7fa4cea00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fa4c8160d90 con 0x7fa4c80d0460 2024-05-26T22:48:18.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.507+0000 7fa4c7400640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fa4c806f420 0x7fa4c80cfd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:40462/0 (socket says 172.21.15.131:40462) 2024-05-26T22:48:18.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.507+0000 7fa4c7400640 1 -- 172.21.15.131:0/3824474939 learned_addr learned my addr 172.21.15.131:0/3824474939 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:48:18.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.508+0000 7fa4cca00640 1 --2- 172.21.15.131:0/3824474939 >> v2:172.21.15.187:3300/0 conn(0x7fa4c80d0460 0x7fa4c806d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:18.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.508+0000 7fa4c6a00640 1 -- 172.21.15.131:0/3824474939 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2829293051 0 0) 0x7fa4c8160180 con 0x7fa4c806f420 2024-05-26T22:48:18.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.508+0000 7fa4c6a00640 1 -- 172.21.15.131:0/3824474939 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa4ac003520 con 0x7fa4c806f420 2024-05-26T22:48:18.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.508+0000 7fa4c6a00640 1 -- 172.21.15.131:0/3824474939 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1020899791 0 0) 0x7fa4ac003520 con 0x7fa4c806f420 2024-05-26T22:48:18.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.508+0000 7fa4c6a00640 1 -- 172.21.15.131:0/3824474939 >> v2:172.21.15.187:3300/0 conn(0x7fa4c80d0460 msgr2=0x7fa4c806d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:18.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.508+0000 7fa4c6a00640 1 --2- 172.21.15.131:0/3824474939 >> v2:172.21.15.187:3300/0 conn(0x7fa4c80d0460 0x7fa4c806d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:18.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.508+0000 7fa4c6a00640 1 -- 172.21.15.131:0/3824474939 >> v1:172.21.15.187:6789/0 conn(0x7fa4c80b5450 legacy=0x7fa4c80b43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:18.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.508+0000 7fa4c6a00640 1 -- 172.21.15.131:0/3824474939 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4c8160740 con 0x7fa4c806f420 2024-05-26T22:48:18.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.508+0000 7fa4c6a00640 1 -- 172.21.15.131:0/3824474939 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7fa4b4001670 con 0x7fa4c806f420 2024-05-26T22:48:18.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.508+0000 7fa4c6a00640 1 -- 172.21.15.131:0/3824474939 >> v1:172.21.15.131:6789/0 conn(0x7fa4c806f420 legacy=0x7fa4c80cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:18.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.508+0000 7fa4c6a00640 1 --2- 172.21.15.131:0/3824474939 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa4ac0039b0 0x7fa4ac003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:18.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.508+0000 7fa4c6a00640 1 --2- 172.21.15.131:0/3824474939 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa4ac004800 0x7fa4ac004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:18.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.508+0000 7fa4c6a00640 1 --2- 172.21.15.131:0/3824474939 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4ac0051c0 0x7fa4ac00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:18.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.508+0000 7fa4c6a00640 1 -- 172.21.15.131:0/3824474939 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa4c8160d90 con 0x7fa4ac0051c0 2024-05-26T22:48:18.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.508+0000 7fa4c6a00640 1 -- 172.21.15.131:0/3824474939 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ac00c060 con 0x7fa4ac004800 2024-05-26T22:48:18.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.508+0000 7fa4c6a00640 1 -- 172.21.15.131:0/3824474939 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa4ac00c260 con 0x7fa4ac0039b0 2024-05-26T22:48:18.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.509+0000 7fa4cca00640 1 --2- 172.21.15.131:0/3824474939 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4ac0051c0 0x7fa4ac00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:18.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.509+0000 7fa4c7e00640 1 --2- 172.21.15.131:0/3824474939 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa4ac0039b0 0x7fa4ac003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:18.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.509+0000 7fa4c7400640 1 --2- 172.21.15.131:0/3824474939 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa4ac004800 0x7fa4ac004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:18.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.509+0000 7fa4cca00640 1 -- 172.21.15.131:0/3824474939 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa4ac0039b0 msgr2=0x7fa4ac003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:18.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.509+0000 7fa4cca00640 1 --2- 172.21.15.131:0/3824474939 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa4ac0039b0 0x7fa4ac003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:18.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.509+0000 7fa4cca00640 1 -- 172.21.15.131:0/3824474939 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa4ac004800 msgr2=0x7fa4ac004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:18.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.509+0000 7fa4cca00640 1 --2- 172.21.15.131:0/3824474939 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa4ac004800 0x7fa4ac004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:18.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.509+0000 7fa4cca00640 1 -- 172.21.15.131:0/3824474939 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4ac003520 con 0x7fa4ac0051c0 2024-05-26T22:48:18.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.509+0000 7fa4c7400640 1 --2- 172.21.15.131:0/3824474939 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa4ac004800 0x7fa4ac004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:48:18.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.510+0000 7fa4cca00640 1 -- 172.21.15.131:0/3824474939 reap_dead start 2024-05-26T22:48:18.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.510+0000 7fa4cca00640 1 --2- 172.21.15.131:0/3824474939 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4ac0051c0 0x7fa4ac00b9c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa4c0009fc0 tx=0x7fa4ac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6be3f563d678d1d2 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:18.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.510+0000 7fa4c6a00640 1 -- 172.21.15.131:0/3824474939 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4c0035020 con 0x7fa4ac0051c0 2024-05-26T22:48:18.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.510+0000 7fa4c6a00640 1 -- 172.21.15.131:0/3824474939 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4c0003b90 con 0x7fa4ac0051c0 2024-05-26T22:48:18.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.510+0000 7fa4c6a00640 1 -- 172.21.15.131:0/3824474939 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4c0009c10 con 0x7fa4ac0051c0 2024-05-26T22:48:18.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.511+0000 7fa4cea00640 1 -- 172.21.15.131:0/3824474939 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4ac0051c0 msgr2=0x7fa4ac00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:18.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.511+0000 7fa4cea00640 1 --2- 172.21.15.131:0/3824474939 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4ac0051c0 0x7fa4ac00b9c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa4c0009fc0 tx=0x7fa4ac005830 comp rx=0 tx=0).stop 2024-05-26T22:48:18.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.511+0000 7fa4cea00640 1 -- 172.21.15.131:0/3824474939 shutdown_connections 2024-05-26T22:48:18.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.511+0000 7fa4cea00640 1 --2- 172.21.15.131:0/3824474939 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4ac0051c0 0x7fa4ac00b9c0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:18.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.511+0000 7fa4cea00640 1 -- 172.21.15.131:0/3824474939 >> 172.21.15.131:0/3824474939 conn(0x7fa4c806bc90 msgr2=0x7fa4c80b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:18.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.511+0000 7fa4cea00640 1 -- 172.21.15.131:0/3824474939 shutdown_connections 2024-05-26T22:48:18.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.511+0000 7fa4cea00640 1 -- 172.21.15.131:0/3824474939 wait complete. 2024-05-26T22:48:18.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.513+0000 7fa4cea00640 1 Processor -- start 2024-05-26T22:48:18.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.513+0000 7fa4cea00640 1 -- start start 2024-05-26T22:48:18.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.514+0000 7fa4cea00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa4ac0051c0 0x7fa4c81464f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:18.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.514+0000 7fa4cea00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4c8150c10 0x7fa4c8153a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:18.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.514+0000 7fa4cea00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa4c80cdac0 0x7fa4c80cdf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:18.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.514+0000 7fa4cea00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa4c815fde0 con 0x7fa4c8150c10 2024-05-26T22:48:18.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.514+0000 7fa4cea00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa4c80b5830 con 0x7fa4c80cdac0 2024-05-26T22:48:18.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.514+0000 7fa4cea00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa4c815ff60 con 0x7fa4ac0051c0 2024-05-26T22:48:18.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.514+0000 7fa4c7400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4c8150c10 0x7fa4c8153a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:18.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.514+0000 7fa4c7400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4c8150c10 0x7fa4c8153a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33938/0 (socket says 172.21.15.131:33938) 2024-05-26T22:48:18.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.514+0000 7fa4c7400640 1 -- 172.21.15.131:0/1161716719 learned_addr learned my addr 172.21.15.131:0/1161716719 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:18.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.514+0000 7fa4cca00640 1 --2- 172.21.15.131:0/1161716719 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa4c80cdac0 0x7fa4c80cdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:18.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.514+0000 7fa4c7e00640 1 --2- 172.21.15.131:0/1161716719 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa4ac0051c0 0x7fa4c81464f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:18.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.515+0000 7fa4c7400640 1 -- 172.21.15.131:0/1161716719 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa4ac0051c0 msgr2=0x7fa4c81464f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:18.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.515+0000 7fa4c7400640 1 --2- 172.21.15.131:0/1161716719 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa4ac0051c0 0x7fa4c81464f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:18.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.515+0000 7fa4c7400640 1 -- 172.21.15.131:0/1161716719 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa4c80cdac0 msgr2=0x7fa4c80cdf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:18.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.515+0000 7fa4c7400640 1 --2- 172.21.15.131:0/1161716719 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa4c80cdac0 0x7fa4c80cdf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:18.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.515+0000 7fa4c7400640 1 -- 172.21.15.131:0/1161716719 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4c8153f40 con 0x7fa4c8150c10 2024-05-26T22:48:18.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.515+0000 7fa4c7e00640 1 --2- 172.21.15.131:0/1161716719 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa4ac0051c0 0x7fa4c81464f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:48:18.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.515+0000 7fa4cca00640 1 --2- 172.21.15.131:0/1161716719 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa4c80cdac0 0x7fa4c80cdf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:48:18.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.515+0000 7fa4c7400640 1 --2- 172.21.15.131:0/1161716719 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4c8150c10 0x7fa4c8153a00 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa4b400d810 tx=0x7fa4b400dcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:18.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.515+0000 7fa4bfe00640 1 -- 172.21.15.131:0/1161716719 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4b4019070 con 0x7fa4c8150c10 2024-05-26T22:48:18.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.515+0000 7fa4bfe00640 1 -- 172.21.15.131:0/1161716719 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4b40074e0 con 0x7fa4c8150c10 2024-05-26T22:48:18.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.515+0000 7fa4bfe00640 1 -- 172.21.15.131:0/1161716719 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4b40049e0 con 0x7fa4c8150c10 2024-05-26T22:48:18.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.515+0000 7fa4cea00640 1 -- 172.21.15.131:0/1161716719 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4c81f5d60 con 0x7fa4c8150c10 2024-05-26T22:48:18.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.515+0000 7fa4cea00640 1 -- 172.21.15.131:0/1161716719 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4c81f6320 con 0x7fa4c8150c10 2024-05-26T22:48:18.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.516+0000 7fa4cea00640 1 -- 172.21.15.131:0/1161716719 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4c80ce480 con 0x7fa4c8150c10 2024-05-26T22:48:18.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.517+0000 7fa4bfe00640 1 -- 172.21.15.131:0/1161716719 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa4b401c070 con 0x7fa4c8150c10 2024-05-26T22:48:18.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.518+0000 7fa4bfe00640 1 --2- 172.21.15.131:0/1161716719 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa4a806be80 0x7fa4a806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:18.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.518+0000 7fa4bfe00640 1 -- 172.21.15.131:0/1161716719 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fa4b408a4a0 con 0x7fa4c8150c10 2024-05-26T22:48:18.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.518+0000 7fa4c7e00640 1 --2- 172.21.15.131:0/1161716719 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa4a806be80 0x7fa4a806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:18.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.519+0000 7fa4c7e00640 1 --2- 172.21.15.131:0/1161716719 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa4a806be80 0x7fa4a806e340 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa4b8000c00 tx=0x7fa4b8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:18.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.522+0000 7fa4bfe00640 1 -- 172.21.15.131:0/1161716719 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fa4b4058bc0 con 0x7fa4c8150c10 2024-05-26T22:48:18.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.720+0000 7fa4cea00640 1 -- 172.21.15.131:0/1161716719 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa4c8074e90 con 0x7fa4c8150c10 2024-05-26T22:48:18.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.720+0000 7fa4bfe00640 1 -- 172.21.15.131:0/1161716719 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) ==== 93+0+10 (secure 0 0 0) 0x7fa4b405cec0 con 0x7fa4c8150c10 2024-05-26T22:48:18.720 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2024-05-26T22:48:18.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.723+0000 7fa4cea00640 1 -- 172.21.15.131:0/1161716719 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa4a806be80 msgr2=0x7fa4a806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:18.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.723+0000 7fa4cea00640 1 --2- 172.21.15.131:0/1161716719 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa4a806be80 0x7fa4a806e340 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa4b8000c00 tx=0x7fa4b8009040 comp rx=0 tx=0).stop 2024-05-26T22:48:18.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.723+0000 7fa4cea00640 1 -- 172.21.15.131:0/1161716719 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4c8150c10 msgr2=0x7fa4c8153a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:18.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.723+0000 7fa4cea00640 1 --2- 172.21.15.131:0/1161716719 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4c8150c10 0x7fa4c8153a00 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa4b400d810 tx=0x7fa4b400dcd0 comp rx=0 tx=0).stop 2024-05-26T22:48:18.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.723+0000 7fa4cea00640 1 -- 172.21.15.131:0/1161716719 shutdown_connections 2024-05-26T22:48:18.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.723+0000 7fa4cea00640 1 --2- 172.21.15.131:0/1161716719 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa4a806be80 0x7fa4a806e340 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:18.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.723+0000 7fa4cea00640 1 --2- 172.21.15.131:0/1161716719 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa4c80cdac0 0x7fa4c80cdf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:18.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.723+0000 7fa4cea00640 1 --2- 172.21.15.131:0/1161716719 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4c8150c10 0x7fa4c8153a00 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:18.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.723+0000 7fa4cea00640 1 --2- 172.21.15.131:0/1161716719 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa4ac0051c0 0x7fa4c81464f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:18.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.723+0000 7fa4cea00640 1 -- 172.21.15.131:0/1161716719 >> 172.21.15.131:0/1161716719 conn(0x7fa4c806bc90 msgr2=0x7fa4c80cc100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:18.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.723+0000 7fa4cea00640 1 -- 172.21.15.131:0/1161716719 shutdown_connections 2024-05-26T22:48:18.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.723+0000 7fa4cea00640 1 -- 172.21.15.131:0/1161716719 wait complete. 2024-05-26T22:48:18.742 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-26T22:48:18.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.939+0000 7f9529200640 1 Processor -- start 2024-05-26T22:48:18.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.939+0000 7f9529200640 1 -- start start 2024-05-26T22:48:18.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.940+0000 7f9529200640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f952406ebc0 0x7f95240cedf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:18.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.941+0000 7f9529200640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95240722a0 con 0x7f9524144340 2024-05-26T22:48:18.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.941+0000 7f9529200640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9524072580 con 0x7f9524071b70 2024-05-26T22:48:18.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.941+0000 7f9529200640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f9524072eb0 con 0x7f952406ebc0 2024-05-26T22:48:18.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.941+0000 7f9523400640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f9524144340 0x7f952406e4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:40476/0 (socket says 172.21.15.131:40476) 2024-05-26T22:48:18.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.941+0000 7f9523400640 1 -- 172.21.15.131:0/171871647 learned_addr learned my addr 172.21.15.131:0/171871647 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:48:18.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.941+0000 7f951be00640 1 --2- 172.21.15.131:0/171871647 >> v2:172.21.15.187:3300/0 conn(0x7f952406ebc0 0x7f95240cedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:18.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.941+0000 7f9522a00640 1 -- 172.21.15.131:0/171871647 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2635449851 0 0) 0x7f95240722a0 con 0x7f9524144340 2024-05-26T22:48:18.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.941+0000 7f9522a00640 1 -- 172.21.15.131:0/171871647 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9508003520 con 0x7f9524144340 2024-05-26T22:48:18.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.942+0000 7f9522a00640 1 -- 172.21.15.131:0/171871647 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4181168670 0 0) 0x7f9508003520 con 0x7f9524144340 2024-05-26T22:48:18.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.942+0000 7f9522a00640 1 -- 172.21.15.131:0/171871647 >> v2:172.21.15.187:3300/0 conn(0x7f952406ebc0 msgr2=0x7f95240cedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:18.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.942+0000 7f9522a00640 1 --2- 172.21.15.131:0/171871647 >> v2:172.21.15.187:3300/0 conn(0x7f952406ebc0 0x7f95240cedf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:18.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.942+0000 7f9522a00640 1 -- 172.21.15.131:0/171871647 >> v1:172.21.15.187:6789/0 conn(0x7f9524071b70 legacy=0x7f95240b3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:18.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.942+0000 7f9522a00640 1 -- 172.21.15.131:0/171871647 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9524072860 con 0x7f9524144340 2024-05-26T22:48:18.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.942+0000 7f9522a00640 1 -- 172.21.15.131:0/171871647 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7f9514001690 con 0x7f9524144340 2024-05-26T22:48:18.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.942+0000 7f9522a00640 1 -- 172.21.15.131:0/171871647 >> v1:172.21.15.131:6789/0 conn(0x7f9524144340 legacy=0x7f952406e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:18.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.942+0000 7f9522a00640 1 --2- 172.21.15.131:0/171871647 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9508003a00 0x7f9508003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:18.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.942+0000 7f9522a00640 1 --2- 172.21.15.131:0/171871647 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9508004800 0x7f9508004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:18.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.942+0000 7f9522a00640 1 --2- 172.21.15.131:0/171871647 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f95080051c0 0x7f950800dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:18.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.942+0000 7f9522a00640 1 -- 172.21.15.131:0/171871647 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f950800e2a0 con 0x7f9508003a00 2024-05-26T22:48:18.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.942+0000 7f9522a00640 1 -- 172.21.15.131:0/171871647 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f950800e450 con 0x7f9508004800 2024-05-26T22:48:18.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.942+0000 7f9522a00640 1 -- 172.21.15.131:0/171871647 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f950800e650 con 0x7f95080051c0 2024-05-26T22:48:18.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.942+0000 7f9523400640 1 --2- 172.21.15.131:0/171871647 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9508003a00 0x7f9508003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:18.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.942+0000 7f951be00640 1 --2- 172.21.15.131:0/171871647 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9508004800 0x7f9508004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:18.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.942+0000 7f9523e00640 1 --2- 172.21.15.131:0/171871647 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f95080051c0 0x7f950800dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:18.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.942+0000 7f9523400640 1 -- 172.21.15.131:0/171871647 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f95080051c0 msgr2=0x7f950800dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:18.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.942+0000 7f9523400640 1 --2- 172.21.15.131:0/171871647 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f95080051c0 0x7f950800dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:18.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.942+0000 7f9523400640 1 -- 172.21.15.131:0/171871647 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9508004800 msgr2=0x7f9508004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:18.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.942+0000 7f9523400640 1 --2- 172.21.15.131:0/171871647 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9508004800 0x7f9508004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:18.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.942+0000 7f9523400640 1 -- 172.21.15.131:0/171871647 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9508003520 con 0x7f9508003a00 2024-05-26T22:48:18.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.942+0000 7f9523e00640 1 -- 172.21.15.131:0/171871647 reap_dead start 2024-05-26T22:48:18.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.943+0000 7f9523400640 1 --2- 172.21.15.131:0/171871647 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9508003a00 0x7f9508003e20 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f951400c0e0 tx=0x7f9508005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f282287a3459ec27 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:18.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.944+0000 7f9522a00640 1 -- 172.21.15.131:0/171871647 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9514007c40 con 0x7f9508003a00 2024-05-26T22:48:18.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.944+0000 7f9522a00640 1 -- 172.21.15.131:0/171871647 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9514005af0 con 0x7f9508003a00 2024-05-26T22:48:18.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.944+0000 7f9522a00640 1 -- 172.21.15.131:0/171871647 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9514007070 con 0x7f9508003a00 2024-05-26T22:48:18.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.944+0000 7f9529200640 1 -- 172.21.15.131:0/171871647 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9508003a00 msgr2=0x7f9508003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:18.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.944+0000 7f9529200640 1 --2- 172.21.15.131:0/171871647 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9508003a00 0x7f9508003e20 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f951400c0e0 tx=0x7f9508005830 comp rx=0 tx=0).stop 2024-05-26T22:48:18.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.944+0000 7f9529200640 1 -- 172.21.15.131:0/171871647 shutdown_connections 2024-05-26T22:48:18.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.944+0000 7f9529200640 1 --2- 172.21.15.131:0/171871647 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9508003a00 0x7f9508003e20 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:18.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.944+0000 7f9529200640 1 -- 172.21.15.131:0/171871647 >> 172.21.15.131:0/171871647 conn(0x7f95240cff40 msgr2=0x7f95240d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:18.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.944+0000 7f9529200640 1 -- 172.21.15.131:0/171871647 shutdown_connections 2024-05-26T22:48:18.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.944+0000 7f9529200640 1 -- 172.21.15.131:0/171871647 wait complete. 2024-05-26T22:48:18.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.946+0000 7f9529200640 1 Processor -- start 2024-05-26T22:48:18.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.946+0000 7f9529200640 1 -- start start 2024-05-26T22:48:18.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.947+0000 7f9529200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9508003a00 0x7f952415e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:18.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.947+0000 7f9529200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f952415a050 0x7f95241401b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:18.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.947+0000 7f9529200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f952415a510 0x7f9524142030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:18.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.947+0000 7f9529200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f95240cedf0 con 0x7f952415a510 2024-05-26T22:48:18.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.947+0000 7f9529200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9524071f80 con 0x7f952415a050 2024-05-26T22:48:18.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.947+0000 7f9529200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f95240cef70 con 0x7f9508003a00 2024-05-26T22:48:18.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.947+0000 7f9523e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f952415a510 0x7f9524142030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:18.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.947+0000 7f9523e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f952415a510 0x7f9524142030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33944/0 (socket says 172.21.15.131:33944) 2024-05-26T22:48:18.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.947+0000 7f9523e00640 1 -- 172.21.15.131:0/1351092579 learned_addr learned my addr 172.21.15.131:0/1351092579 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:18.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.948+0000 7f951be00640 1 --2- 172.21.15.131:0/1351092579 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f952415a050 0x7f95241401b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:18.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.948+0000 7f9523400640 1 --2- 172.21.15.131:0/1351092579 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9508003a00 0x7f952415e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:18.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.948+0000 7f9523e00640 1 -- 172.21.15.131:0/1351092579 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9508003a00 msgr2=0x7f952415e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:18.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.948+0000 7f9523e00640 1 --2- 172.21.15.131:0/1351092579 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9508003a00 0x7f952415e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:18.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.948+0000 7f9523e00640 1 -- 172.21.15.131:0/1351092579 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f952415a050 msgr2=0x7f95241401b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:18.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.948+0000 7f9523e00640 1 --2- 172.21.15.131:0/1351092579 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f952415a050 0x7f95241401b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:18.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.948+0000 7f9523e00640 1 -- 172.21.15.131:0/1351092579 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f952415d170 con 0x7f952415a510 2024-05-26T22:48:18.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.948+0000 7f9523e00640 1 --2- 172.21.15.131:0/1351092579 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f952415a510 0x7f9524142030 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9524144e80 tx=0x7f951c00eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:18.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.948+0000 7f951b400640 1 -- 172.21.15.131:0/1351092579 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f951c003970 con 0x7f952415a510 2024-05-26T22:48:18.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.948+0000 7f951b400640 1 -- 172.21.15.131:0/1351092579 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f951c018070 con 0x7f952415a510 2024-05-26T22:48:18.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.948+0000 7f9529200640 1 -- 172.21.15.131:0/1351092579 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f952415d450 con 0x7f952415a510 2024-05-26T22:48:18.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.949+0000 7f951b400640 1 -- 172.21.15.131:0/1351092579 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f951c003d70 con 0x7f952415a510 2024-05-26T22:48:18.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.949+0000 7f9529200640 1 -- 172.21.15.131:0/1351092579 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9524143090 con 0x7f952415a510 2024-05-26T22:48:18.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.950+0000 7f9529200640 1 -- 172.21.15.131:0/1351092579 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95240694e0 con 0x7f952415a510 2024-05-26T22:48:18.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.951+0000 7f951b400640 1 -- 172.21.15.131:0/1351092579 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f951c0050d0 con 0x7f952415a510 2024-05-26T22:48:18.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.952+0000 7f951b400640 1 --2- 172.21.15.131:0/1351092579 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f950006be60 0x7f950006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:18.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.952+0000 7f951b400640 1 -- 172.21.15.131:0/1351092579 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f951c08a740 con 0x7f952415a510 2024-05-26T22:48:18.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.952+0000 7f9523400640 1 --2- 172.21.15.131:0/1351092579 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f950006be60 0x7f950006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:18.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.952+0000 7f9523400640 1 --2- 172.21.15.131:0/1351092579 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f950006be60 0x7f950006e320 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9514032000 tx=0x7f9514009030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:18.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:18.956+0000 7f951b400640 1 -- 172.21.15.131:0/1351092579 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f951c058d50 con 0x7f952415a510 2024-05-26T22:48:19.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.159+0000 7f9529200640 1 -- 172.21.15.131:0/1351092579 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f95240b1880 con 0x7f952415a510 2024-05-26T22:48:19.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.160+0000 7f951b400640 1 -- 172.21.15.131:0/1351092579 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) ==== 84+0+65 (secure 0 0 0) 0x7f951c05d050 con 0x7f952415a510 2024-05-26T22:48:19.160 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:48:19.160 INFO:teuthology.orchestra.run.smithi131.stdout:[ 2024-05-26T22:48:19.160 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:48:19.160 INFO:teuthology.orchestra.run.smithi131.stdout: "poolnum": 1, 2024-05-26T22:48:19.160 INFO:teuthology.orchestra.run.smithi131.stdout: "poolname": ".mgr" 2024-05-26T22:48:19.160 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:48:19.160 INFO:teuthology.orchestra.run.smithi131.stdout:] 2024-05-26T22:48:19.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.162+0000 7f9529200640 1 -- 172.21.15.131:0/1351092579 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f950006be60 msgr2=0x7f950006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:19.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.162+0000 7f9529200640 1 --2- 172.21.15.131:0/1351092579 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f950006be60 0x7f950006e320 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9514032000 tx=0x7f9514009030 comp rx=0 tx=0).stop 2024-05-26T22:48:19.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.162+0000 7f9529200640 1 -- 172.21.15.131:0/1351092579 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f952415a510 msgr2=0x7f9524142030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:19.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.162+0000 7f9529200640 1 --2- 172.21.15.131:0/1351092579 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f952415a510 0x7f9524142030 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9524144e80 tx=0x7f951c00eb90 comp rx=0 tx=0).stop 2024-05-26T22:48:19.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.163+0000 7f9529200640 1 -- 172.21.15.131:0/1351092579 shutdown_connections 2024-05-26T22:48:19.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.163+0000 7f9529200640 1 --2- 172.21.15.131:0/1351092579 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f950006be60 0x7f950006e320 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:19.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.163+0000 7f9529200640 1 --2- 172.21.15.131:0/1351092579 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f952415a510 0x7f9524142030 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:19.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.163+0000 7f9529200640 1 --2- 172.21.15.131:0/1351092579 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f952415a050 0x7f95241401b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:19.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.163+0000 7f9529200640 1 --2- 172.21.15.131:0/1351092579 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9508003a00 0x7f952415e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:19.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.163+0000 7f9529200640 1 -- 172.21.15.131:0/1351092579 >> 172.21.15.131:0/1351092579 conn(0x7f95240cff40 msgr2=0x7f95240b3c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:19.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.163+0000 7f9529200640 1 -- 172.21.15.131:0/1351092579 shutdown_connections 2024-05-26T22:48:19.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.163+0000 7f9529200640 1 -- 172.21.15.131:0/1351092579 wait complete. 2024-05-26T22:48:19.181 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-26T22:48:19.181 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-26T22:48:19.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.395+0000 7f67c7e00640 1 Processor -- start 2024-05-26T22:48:19.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.395+0000 7f67c7e00640 1 -- start start 2024-05-26T22:48:19.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.396+0000 7f67c7e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f67c813dee0 0x7f67c806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:19.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.396+0000 7f67c7e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f67c806da20 0x7f67c815b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:19.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.397+0000 7f67c7e00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67c813e2c0 con 0x7f67c815bc00 2024-05-26T22:48:19.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.397+0000 7f67c7e00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f67c815c430 con 0x7f67c806da20 2024-05-26T22:48:19.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.397+0000 7f67c7e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f67c815eb60 con 0x7f67c813dee0 2024-05-26T22:48:19.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.397+0000 7f67c7400640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f67c815bc00 0x7f67c80b1b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:40492/0 (socket says 172.21.15.131:40492) 2024-05-26T22:48:19.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.397+0000 7f67c7400640 1 -- 172.21.15.131:0/2787258543 learned_addr learned my addr 172.21.15.131:0/2787258543 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:48:19.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.397+0000 7f67c6000640 1 --2- 172.21.15.131:0/2787258543 >> v2:172.21.15.131:3300/0 conn(0x7f67c806da20 0x7f67c815b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:19.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.397+0000 7f67c6a00640 1 --2- 172.21.15.131:0/2787258543 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f67c813dee0 0x7f67c806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:19.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.398+0000 7f67c5600640 1 -- 172.21.15.131:0/2787258543 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 758206783 0 0) 0x7f67c813e2c0 con 0x7f67c815bc00 2024-05-26T22:48:19.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.398+0000 7f67c6000640 1 -- 172.21.15.131:0/2787258543 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f67c813dee0 msgr2=0x7f67c806d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:19.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.398+0000 7f67c6000640 1 --2- 172.21.15.131:0/2787258543 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f67c813dee0 0x7f67c806d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:19.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.398+0000 7f67c6000640 1 -- 172.21.15.131:0/2787258543 >> v1:172.21.15.131:6789/0 conn(0x7f67c815bc00 legacy=0x7f67c80b1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:19.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.398+0000 7f67c6000640 1 -- 172.21.15.131:0/2787258543 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67c815ee40 con 0x7f67c806da20 2024-05-26T22:48:19.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.399+0000 7f67c6000640 1 --2- 172.21.15.131:0/2787258543 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f67c806da20 0x7f67c815b6c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f67b40097a0 tx=0x7f67b402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c74497dbb38c94d6 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:19.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.399+0000 7f67c5600640 1 -- 172.21.15.131:0/2787258543 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67b4037070 con 0x7f67c806da20 2024-05-26T22:48:19.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.399+0000 7f67c5600640 1 -- 172.21.15.131:0/2787258543 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67b4032440 con 0x7f67c806da20 2024-05-26T22:48:19.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.399+0000 7f67c5600640 1 -- 172.21.15.131:0/2787258543 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67b4032810 con 0x7f67c806da20 2024-05-26T22:48:19.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.399+0000 7f67c7e00640 1 -- 172.21.15.131:0/2787258543 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f67c806da20 msgr2=0x7f67c815b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:19.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.399+0000 7f67c7e00640 1 --2- 172.21.15.131:0/2787258543 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f67c806da20 0x7f67c815b6c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f67b40097a0 tx=0x7f67b402a180 comp rx=0 tx=0).stop 2024-05-26T22:48:19.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.400+0000 7f67c7e00640 1 -- 172.21.15.131:0/2787258543 shutdown_connections 2024-05-26T22:48:19.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.400+0000 7f67c7e00640 1 --2- 172.21.15.131:0/2787258543 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f67c806da20 0x7f67c815b6c0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:19.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.400+0000 7f67c7e00640 1 --2- 172.21.15.131:0/2787258543 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f67c813dee0 0x7f67c806d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:19.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.400+0000 7f67c7e00640 1 -- 172.21.15.131:0/2787258543 >> 172.21.15.131:0/2787258543 conn(0x7f67c80b5410 msgr2=0x7f67c80b57f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:19.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.400+0000 7f67c7e00640 1 -- 172.21.15.131:0/2787258543 shutdown_connections 2024-05-26T22:48:19.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.400+0000 7f67c7e00640 1 -- 172.21.15.131:0/2787258543 wait complete. 2024-05-26T22:48:19.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.401+0000 7f67c7e00640 1 Processor -- start 2024-05-26T22:48:19.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.402+0000 7f67c7e00640 1 -- start start 2024-05-26T22:48:19.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.402+0000 7f67c7e00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f67c806da20 0x7f67c8137950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:19.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.402+0000 7f67c7e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f67c813dee0 0x7f67c80cb0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:19.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.402+0000 7f67c7e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f67c815bc00 0x7f67c813cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:19.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.402+0000 7f67c7e00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f67c80cf7c0 con 0x7f67c813dee0 2024-05-26T22:48:19.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.402+0000 7f67c7e00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f67c815f400 con 0x7f67c806da20 2024-05-26T22:48:19.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.402+0000 7f67c7e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f67c815c090 con 0x7f67c815bc00 2024-05-26T22:48:19.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.403+0000 7f67c6000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f67c813dee0 0x7f67c80cb0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:19.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.403+0000 7f67c6000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f67c813dee0 0x7f67c80cb0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33972/0 (socket says 172.21.15.131:33972) 2024-05-26T22:48:19.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.403+0000 7f67c6000640 1 -- 172.21.15.131:0/2389537978 learned_addr learned my addr 172.21.15.131:0/2389537978 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:19.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.403+0000 7f67c6000640 1 -- 172.21.15.131:0/2389537978 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f67c815bc00 msgr2=0x7f67c813cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:19.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.403+0000 7f67c6000640 1 --2- 172.21.15.131:0/2389537978 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f67c815bc00 0x7f67c813cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:19.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.403+0000 7f67c6000640 1 -- 172.21.15.131:0/2389537978 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f67c806da20 msgr2=0x7f67c8137950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:19.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.403+0000 7f67c6000640 1 --2- 172.21.15.131:0/2389537978 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f67c806da20 0x7f67c8137950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:19.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.403+0000 7f67c6000640 1 -- 172.21.15.131:0/2389537978 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67c80cb920 con 0x7f67c813dee0 2024-05-26T22:48:19.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.403+0000 7f67c6000640 1 --2- 172.21.15.131:0/2389537978 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f67c813dee0 0x7f67c80cb0d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f67b4035040 tx=0x7f67b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:19.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.404+0000 7f67bea00640 1 -- 172.21.15.131:0/2389537978 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67b4037040 con 0x7f67c813dee0 2024-05-26T22:48:19.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.404+0000 7f67bea00640 1 -- 172.21.15.131:0/2389537978 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67b4030500 con 0x7f67c813dee0 2024-05-26T22:48:19.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.404+0000 7f67bea00640 1 -- 172.21.15.131:0/2389537978 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67b4030910 con 0x7f67c813dee0 2024-05-26T22:48:19.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.404+0000 7f67c7e00640 1 -- 172.21.15.131:0/2389537978 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67c80cbc00 con 0x7f67c813dee0 2024-05-26T22:48:19.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.404+0000 7f67c7e00640 1 -- 172.21.15.131:0/2389537978 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67c80cc1c0 con 0x7f67c813dee0 2024-05-26T22:48:19.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.405+0000 7f67c7e00640 1 -- 172.21.15.131:0/2389537978 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67c80694e0 con 0x7f67c813dee0 2024-05-26T22:48:19.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.406+0000 7f67bea00640 1 -- 172.21.15.131:0/2389537978 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f67b4044020 con 0x7f67c813dee0 2024-05-26T22:48:19.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.407+0000 7f67bea00640 1 --2- 172.21.15.131:0/2389537978 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f67a806be30 0x7f67a806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:19.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.407+0000 7f67bea00640 1 -- 172.21.15.131:0/2389537978 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f67b40aad80 con 0x7f67c813dee0 2024-05-26T22:48:19.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.408+0000 7f67c6a00640 1 --2- 172.21.15.131:0/2389537978 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f67a806be30 0x7f67a806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:19.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.408+0000 7f67c6a00640 1 --2- 172.21.15.131:0/2389537978 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f67a806be30 0x7f67a806e2f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f67b8006a90 tx=0x7f67b8006a20 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:19.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.412+0000 7f67bea00640 1 -- 172.21.15.131:0/2389537978 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f67b4079a80 con 0x7f67c813dee0 2024-05-26T22:48:19.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.610+0000 7f67c7e00640 1 -- 172.21.15.131:0/2389537978 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f67c8159570 con 0x7f67c813dee0 2024-05-26T22:48:19.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.748+0000 7f67bea00640 1 -- 172.21.15.131:0/2389537978 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) ==== 138+0+0 (secure 0 0 0) 0x7f67b4079380 con 0x7f67c813dee0 2024-05-26T22:48:19.748 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_metadata' created 2024-05-26T22:48:19.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.752+0000 7f67c7e00640 1 -- 172.21.15.131:0/2389537978 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f67a806be30 msgr2=0x7f67a806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:19.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.752+0000 7f67c7e00640 1 --2- 172.21.15.131:0/2389537978 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f67a806be30 0x7f67a806e2f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f67b8006a90 tx=0x7f67b8006a20 comp rx=0 tx=0).stop 2024-05-26T22:48:19.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.752+0000 7f67c7e00640 1 -- 172.21.15.131:0/2389537978 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f67c813dee0 msgr2=0x7f67c80cb0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:19.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.752+0000 7f67c7e00640 1 --2- 172.21.15.131:0/2389537978 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f67c813dee0 0x7f67c80cb0d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f67b4035040 tx=0x7f67b40046c0 comp rx=0 tx=0).stop 2024-05-26T22:48:19.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.752+0000 7f67c7e00640 1 -- 172.21.15.131:0/2389537978 shutdown_connections 2024-05-26T22:48:19.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.752+0000 7f67c7e00640 1 --2- 172.21.15.131:0/2389537978 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f67c815bc00 0x7f67c813cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:19.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.752+0000 7f67c7e00640 1 --2- 172.21.15.131:0/2389537978 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f67a806be30 0x7f67a806e2f0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:19.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.752+0000 7f67c7e00640 1 --2- 172.21.15.131:0/2389537978 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f67c813dee0 0x7f67c80cb0d0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:19.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.752+0000 7f67c7e00640 1 --2- 172.21.15.131:0/2389537978 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f67c806da20 0x7f67c8137950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:19.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.752+0000 7f67c7e00640 1 -- 172.21.15.131:0/2389537978 >> 172.21.15.131:0/2389537978 conn(0x7f67c80b5410 msgr2=0x7f67c8159850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:19.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.753+0000 7f67c7e00640 1 -- 172.21.15.131:0/2389537978 shutdown_connections 2024-05-26T22:48:19.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.753+0000 7f67c7e00640 1 -- 172.21.15.131:0/2389537978 wait complete. 2024-05-26T22:48:19.782 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-26T22:48:19.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.967+0000 7f91bf200640 1 Processor -- start 2024-05-26T22:48:19.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.967+0000 7f91bf200640 1 -- start start 2024-05-26T22:48:19.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.968+0000 7f91bf200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f91c00d05a0 0x7f91c0166ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:19.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.968+0000 7f91bf200640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91c0072d60 con 0x7f91c0145a10 2024-05-26T22:48:19.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.968+0000 7f91bf200640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91c0072a80 con 0x7f91c00712d0 2024-05-26T22:48:19.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.968+0000 7f91bf200640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f91c015c760 con 0x7f91c00d05a0 2024-05-26T22:48:19.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.969+0000 7f91be800640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f91c00d05a0 0x7f91c0166ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:19.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.969+0000 7f91be800640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f91c00d05a0 0x7f91c0166ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33988/0 (socket says 172.21.15.131:33988) 2024-05-26T22:48:19.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.969+0000 7f91be800640 1 -- 172.21.15.131:0/3930145133 learned_addr learned my addr 172.21.15.131:0/3930145133 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:19.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.969+0000 7f91be800640 1 -- 172.21.15.131:0/3930145133 >> v1:172.21.15.187:6789/0 conn(0x7f91c00712d0 legacy=0x7f91c006d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:19.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.969+0000 7f91be800640 1 -- 172.21.15.131:0/3930145133 >> v1:172.21.15.131:6789/0 conn(0x7f91c0145a10 legacy=0x7f91c0072180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:19.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.969+0000 7f91be800640 1 -- 172.21.15.131:0/3930145133 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91c00ceab0 con 0x7f91c00d05a0 2024-05-26T22:48:19.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.969+0000 7f91bd400640 1 -- 172.21.15.131:0/3930145133 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 242186618 0 0) 0x7f91c0072d60 con 0x7f91c0145a10 2024-05-26T22:48:19.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.970+0000 7f91be800640 1 --2- 172.21.15.131:0/3930145133 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f91c00d05a0 0x7f91c0166ea0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f91b8009e50 tx=0x7f91b802a760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c8f01550f06992b server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:19.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.971+0000 7f91bd400640 1 -- 172.21.15.131:0/3930145133 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91b8036070 con 0x7f91c00d05a0 2024-05-26T22:48:19.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.971+0000 7f91bd400640 1 -- 172.21.15.131:0/3930145133 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91b8031600 con 0x7f91c00d05a0 2024-05-26T22:48:19.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.971+0000 7f91bd400640 1 -- 172.21.15.131:0/3930145133 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91b80319d0 con 0x7f91c00d05a0 2024-05-26T22:48:19.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.971+0000 7f91bf200640 1 -- 172.21.15.131:0/3930145133 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f91c00d05a0 msgr2=0x7f91c0166ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:19.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.971+0000 7f91bf200640 1 --2- 172.21.15.131:0/3930145133 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f91c00d05a0 0x7f91c0166ea0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f91b8009e50 tx=0x7f91b802a760 comp rx=0 tx=0).stop 2024-05-26T22:48:19.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.971+0000 7f91bf200640 1 -- 172.21.15.131:0/3930145133 shutdown_connections 2024-05-26T22:48:19.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.971+0000 7f91bf200640 1 --2- 172.21.15.131:0/3930145133 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f91c00d05a0 0x7f91c0166ea0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:19.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.971+0000 7f91bf200640 1 -- 172.21.15.131:0/3930145133 >> 172.21.15.131:0/3930145133 conn(0x7f91c00b5660 msgr2=0x7f91c00b5a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:19.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.971+0000 7f91bf200640 1 -- 172.21.15.131:0/3930145133 shutdown_connections 2024-05-26T22:48:19.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.971+0000 7f91bf200640 1 -- 172.21.15.131:0/3930145133 wait complete. 2024-05-26T22:48:19.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.973+0000 7f91bf200640 1 Processor -- start 2024-05-26T22:48:19.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.973+0000 7f91bf200640 1 -- start start 2024-05-26T22:48:19.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.974+0000 7f91bf200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f91c00712d0 0x7f91c014ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:19.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.974+0000 7f91bf200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f91c00d05a0 0x7f91c01427a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:19.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.974+0000 7f91bf200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f91c0145a10 0x7f91c015fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:19.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.974+0000 7f91bf200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f91c0145df0 con 0x7f91c00d05a0 2024-05-26T22:48:19.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.974+0000 7f91bf200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f91c00cf070 con 0x7f91c0145a10 2024-05-26T22:48:19.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.974+0000 7f91bf200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f91c006d960 con 0x7f91c00712d0 2024-05-26T22:48:19.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.974+0000 7f91b7e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f91c00d05a0 0x7f91c01427a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:19.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.974+0000 7f91b7e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f91c00d05a0 0x7f91c01427a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33992/0 (socket says 172.21.15.131:33992) 2024-05-26T22:48:19.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.974+0000 7f91b7e00640 1 -- 172.21.15.131:0/3015113346 learned_addr learned my addr 172.21.15.131:0/3015113346 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:19.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.975+0000 7f91bde00640 1 --2- 172.21.15.131:0/3015113346 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f91c00712d0 0x7f91c014ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:19.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.975+0000 7f91be800640 1 --2- 172.21.15.131:0/3015113346 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f91c0145a10 0x7f91c015fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:19.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.975+0000 7f91b7e00640 1 -- 172.21.15.131:0/3015113346 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f91c00712d0 msgr2=0x7f91c014ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:19.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.975+0000 7f91b7e00640 1 --2- 172.21.15.131:0/3015113346 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f91c00712d0 0x7f91c014ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:19.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.975+0000 7f91b7e00640 1 -- 172.21.15.131:0/3015113346 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f91c0145a10 msgr2=0x7f91c015fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:19.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.975+0000 7f91b7e00640 1 --2- 172.21.15.131:0/3015113346 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f91c0145a10 0x7f91c015fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:19.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.975+0000 7f91b7e00640 1 -- 172.21.15.131:0/3015113346 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91c014d350 con 0x7f91c00d05a0 2024-05-26T22:48:19.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.975+0000 7f91b7e00640 1 --2- 172.21.15.131:0/3015113346 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f91c00d05a0 0x7f91c01427a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f91a800c400 tx=0x7f91a800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:19.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.975+0000 7f91b6000640 1 -- 172.21.15.131:0/3015113346 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91a8011790 con 0x7f91c00d05a0 2024-05-26T22:48:19.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.975+0000 7f91b6000640 1 -- 172.21.15.131:0/3015113346 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91a8017070 con 0x7f91c00d05a0 2024-05-26T22:48:19.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.975+0000 7f91bf200640 1 -- 172.21.15.131:0/3015113346 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91c0142ce0 con 0x7f91c00d05a0 2024-05-26T22:48:19.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.975+0000 7f91b6000640 1 -- 172.21.15.131:0/3015113346 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91a8011bb0 con 0x7f91c00d05a0 2024-05-26T22:48:19.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.976+0000 7f91bf200640 1 -- 172.21.15.131:0/3015113346 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91c01f1f80 con 0x7f91c00d05a0 2024-05-26T22:48:19.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.977+0000 7f91bf200640 1 -- 172.21.15.131:0/3015113346 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91c00694e0 con 0x7f91c00d05a0 2024-05-26T22:48:19.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.978+0000 7f91b6000640 1 -- 172.21.15.131:0/3015113346 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f91a801c460 con 0x7f91c00d05a0 2024-05-26T22:48:19.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.979+0000 7f91b6000640 1 --2- 172.21.15.131:0/3015113346 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f919c06be30 0x7f919c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:19.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.979+0000 7f91b6000640 1 -- 172.21.15.131:0/3015113346 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 5227+0+0 (secure 0 0 0) 0x7f91a808aac0 con 0x7f91c00d05a0 2024-05-26T22:48:19.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.979+0000 7f91bde00640 1 --2- 172.21.15.131:0/3015113346 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f919c06be30 0x7f919c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:19.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.979+0000 7f91bde00640 1 --2- 172.21.15.131:0/3015113346 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f919c06be30 0x7f919c06e2f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f91b0007fd0 tx=0x7f91b0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:19.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:19.984+0000 7f91b6000640 1 -- 172.21.15.131:0/3015113346 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f91c014d350 con 0x7f91c00d05a0 2024-05-26T22:48:20.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:20.186+0000 7f91bf200640 1 -- 172.21.15.131:0/3015113346 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f91c006c320 con 0x7f91c00d05a0 2024-05-26T22:48:20.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:20.754+0000 7f91b6000640 1 -- 172.21.15.131:0/3015113346 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v18) ==== 170+0+0 (secure 0 0 0) 0x7f91a80596e0 con 0x7f91c00d05a0 2024-05-26T22:48:20.754 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_data' created 2024-05-26T22:48:20.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:20.758+0000 7f91bf200640 1 -- 172.21.15.131:0/3015113346 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f919c06be30 msgr2=0x7f919c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:20.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:20.758+0000 7f91bf200640 1 --2- 172.21.15.131:0/3015113346 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f919c06be30 0x7f919c06e2f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f91b0007fd0 tx=0x7f91b0009040 comp rx=0 tx=0).stop 2024-05-26T22:48:20.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:20.758+0000 7f91bf200640 1 -- 172.21.15.131:0/3015113346 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f91c00d05a0 msgr2=0x7f91c01427a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:20.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:20.758+0000 7f91bf200640 1 --2- 172.21.15.131:0/3015113346 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f91c00d05a0 0x7f91c01427a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f91a800c400 tx=0x7f91a800c8c0 comp rx=0 tx=0).stop 2024-05-26T22:48:20.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:20.759+0000 7f91bf200640 1 -- 172.21.15.131:0/3015113346 shutdown_connections 2024-05-26T22:48:20.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:20.759+0000 7f91bf200640 1 --2- 172.21.15.131:0/3015113346 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f919c06be30 0x7f919c06e2f0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:20.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:20.759+0000 7f91bf200640 1 --2- 172.21.15.131:0/3015113346 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f91c0145a10 0x7f91c015fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:20.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:20.759+0000 7f91bf200640 1 --2- 172.21.15.131:0/3015113346 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f91c00d05a0 0x7f91c01427a0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:20.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:20.759+0000 7f91bf200640 1 --2- 172.21.15.131:0/3015113346 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f91c00712d0 0x7f91c014ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:20.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:20.759+0000 7f91bf200640 1 -- 172.21.15.131:0/3015113346 >> 172.21.15.131:0/3015113346 conn(0x7f91c00b5660 msgr2=0x7f91c006d250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:20.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:20.759+0000 7f91bf200640 1 -- 172.21.15.131:0/3015113346 shutdown_connections 2024-05-26T22:48:20.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:20.759+0000 7f91bf200640 1 -- 172.21.15.131:0/3015113346 wait complete. 2024-05-26T22:48:20.788 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-26T22:48:20.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:20.967+0000 7f5f7dc00640 1 Processor -- start 2024-05-26T22:48:20.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:20.968+0000 7f5f7dc00640 1 -- start start 2024-05-26T22:48:20.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:20.969+0000 7f5f7dc00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f5f7806f420 0x7f5f780cfd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:20.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:20.969+0000 7f5f7dc00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5f780cf590 0x7f5f780d0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:20.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:20.969+0000 7f5f7dc00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f78160180 con 0x7f5f780b5450 2024-05-26T22:48:20.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:20.969+0000 7f5f7dc00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f5f78160d90 con 0x7f5f7806f420 2024-05-26T22:48:20.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:20.969+0000 7f5f7dc00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f5f78160460 con 0x7f5f780cf590 2024-05-26T22:48:20.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:20.969+0000 7f5f76a00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f5f780b5450 0x7f5f780b43c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:40504/0 (socket says 172.21.15.131:40504) 2024-05-26T22:48:20.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:20.969+0000 7f5f76a00640 1 -- 172.21.15.131:0/2798739080 learned_addr learned my addr 172.21.15.131:0/2798739080 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:48:20.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:20.970+0000 7f5f6e000640 1 --2- 172.21.15.131:0/2798739080 >> v2:172.21.15.131:3300/0 conn(0x7f5f7806f420 0x7f5f780cfd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:20.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:20.970+0000 7f5f77400640 1 --2- 172.21.15.131:0/2798739080 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5f780cf590 0x7f5f780d0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:20.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:20.970+0000 7f5f76000640 1 -- 172.21.15.131:0/2798739080 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 428514303 0 0) 0x7f5f78160180 con 0x7f5f780b5450 2024-05-26T22:48:20.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:20.970+0000 7f5f76000640 1 -- 172.21.15.131:0/2798739080 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5f5c003520 con 0x7f5f780b5450 2024-05-26T22:48:20.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:20.970+0000 7f5f6e000640 1 -- 172.21.15.131:0/2798739080 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5f780cf590 msgr2=0x7f5f780d0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:20.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:20.970+0000 7f5f6e000640 1 --2- 172.21.15.131:0/2798739080 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5f780cf590 0x7f5f780d0290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:20.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:20.970+0000 7f5f6e000640 1 -- 172.21.15.131:0/2798739080 >> v1:172.21.15.131:6789/0 conn(0x7f5f780b5450 legacy=0x7f5f780b43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:20.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:20.970+0000 7f5f6e000640 1 -- 172.21.15.131:0/2798739080 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f78160740 con 0x7f5f7806f420 2024-05-26T22:48:20.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:20.971+0000 7f5f6e000640 1 --2- 172.21.15.131:0/2798739080 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5f7806f420 0x7f5f780cfd50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5f600097a0 tx=0x7f5f6002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=808ed618c0dabbfe server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:20.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:20.971+0000 7f5f76000640 1 -- 172.21.15.131:0/2798739080 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f60037070 con 0x7f5f7806f420 2024-05-26T22:48:20.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:20.971+0000 7f5f76000640 1 -- 172.21.15.131:0/2798739080 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f60032440 con 0x7f5f7806f420 2024-05-26T22:48:20.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:20.971+0000 7f5f76000640 1 -- 172.21.15.131:0/2798739080 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f60032810 con 0x7f5f7806f420 2024-05-26T22:48:20.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:20.972+0000 7f5f7dc00640 1 -- 172.21.15.131:0/2798739080 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5f7806f420 msgr2=0x7f5f780cfd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:20.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:20.972+0000 7f5f7dc00640 1 --2- 172.21.15.131:0/2798739080 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5f7806f420 0x7f5f780cfd50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5f600097a0 tx=0x7f5f6002a180 comp rx=0 tx=0).stop 2024-05-26T22:48:20.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:20.972+0000 7f5f7dc00640 1 -- 172.21.15.131:0/2798739080 shutdown_connections 2024-05-26T22:48:20.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:20.972+0000 7f5f7dc00640 1 --2- 172.21.15.131:0/2798739080 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5f780cf590 0x7f5f780d0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:20.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:20.972+0000 7f5f7dc00640 1 --2- 172.21.15.131:0/2798739080 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5f7806f420 0x7f5f780cfd50 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:20.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:20.972+0000 7f5f7dc00640 1 -- 172.21.15.131:0/2798739080 >> 172.21.15.131:0/2798739080 conn(0x7f5f7806bc90 msgr2=0x7f5f780b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:20.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:20.972+0000 7f5f7dc00640 1 -- 172.21.15.131:0/2798739080 shutdown_connections 2024-05-26T22:48:20.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:20.972+0000 7f5f7dc00640 1 -- 172.21.15.131:0/2798739080 wait complete. 2024-05-26T22:48:20.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:20.974+0000 7f5f7dc00640 1 Processor -- start 2024-05-26T22:48:20.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:20.974+0000 7f5f7dc00640 1 -- start start 2024-05-26T22:48:20.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:20.975+0000 7f5f7dc00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5f7806f420 0x7f5f781492f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:20.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:20.975+0000 7f5f7dc00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5f780b5450 0x7f5f7809cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:20.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:20.975+0000 7f5f7dc00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5f780cf590 0x7f5f7814ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:20.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:20.975+0000 7f5f7dc00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f5f7815fde0 con 0x7f5f7806f420 2024-05-26T22:48:20.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:20.975+0000 7f5f7dc00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5f780b5830 con 0x7f5f780b5450 2024-05-26T22:48:20.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:20.975+0000 7f5f7dc00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f5f7815ff60 con 0x7f5f780cf590 2024-05-26T22:48:20.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:20.975+0000 7f5f76a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5f7806f420 0x7f5f781492f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:20.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:20.975+0000 7f5f76a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5f7806f420 0x7f5f781492f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:34010/0 (socket says 172.21.15.131:34010) 2024-05-26T22:48:20.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:20.975+0000 7f5f76a00640 1 -- 172.21.15.131:0/4233919354 learned_addr learned my addr 172.21.15.131:0/4233919354 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:20.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:20.976+0000 7f5f77400640 1 --2- 172.21.15.131:0/4233919354 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5f780cf590 0x7f5f7814ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:20.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:20.976+0000 7f5f76a00640 1 -- 172.21.15.131:0/4233919354 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5f780cf590 msgr2=0x7f5f7814ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:20.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:20.976+0000 7f5f76a00640 1 --2- 172.21.15.131:0/4233919354 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5f780cf590 0x7f5f7814ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:20.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:20.976+0000 7f5f76a00640 1 -- 172.21.15.131:0/4233919354 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5f780b5450 msgr2=0x7f5f7809cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:20.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:20.976+0000 7f5f6e000640 1 --2- 172.21.15.131:0/4233919354 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5f780b5450 0x7f5f7809cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:20.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:20.976+0000 7f5f76a00640 1 --2- 172.21.15.131:0/4233919354 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5f780b5450 0x7f5f7809cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:20.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:20.976+0000 7f5f76a00640 1 -- 172.21.15.131:0/4233919354 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f781587a0 con 0x7f5f7806f420 2024-05-26T22:48:20.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:20.976+0000 7f5f76a00640 1 --2- 172.21.15.131:0/4233919354 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5f7806f420 0x7f5f781492f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5f6800c430 tx=0x7f5f6800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:20.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:20.976+0000 7f5f6f400640 1 -- 172.21.15.131:0/4233919354 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f68017070 con 0x7f5f7806f420 2024-05-26T22:48:20.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:20.976+0000 7f5f6f400640 1 -- 172.21.15.131:0/4233919354 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f68005ce0 con 0x7f5f7806f420 2024-05-26T22:48:20.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:20.976+0000 7f5f7dc00640 1 -- 172.21.15.131:0/4233919354 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f78158a80 con 0x7f5f7806f420 2024-05-26T22:48:20.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:20.976+0000 7f5f6f400640 1 -- 172.21.15.131:0/4233919354 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f680072f0 con 0x7f5f7806f420 2024-05-26T22:48:20.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:20.977+0000 7f5f7dc00640 1 -- 172.21.15.131:0/4233919354 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f78158d60 con 0x7f5f7806f420 2024-05-26T22:48:20.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:20.978+0000 7f5f7dc00640 1 -- 172.21.15.131:0/4233919354 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f78074e90 con 0x7f5f7806f420 2024-05-26T22:48:20.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:20.979+0000 7f5f6f400640 1 -- 172.21.15.131:0/4233919354 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5f6801a070 con 0x7f5f7806f420 2024-05-26T22:48:20.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:20.980+0000 7f5f6f400640 1 --2- 172.21.15.131:0/4233919354 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f5f5406be30 0x7f5f5406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:20.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:20.980+0000 7f5f6f400640 1 -- 172.21.15.131:0/4233919354 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5609+0+0 (secure 0 0 0) 0x7f5f6808a430 con 0x7f5f7806f420 2024-05-26T22:48:20.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:20.981+0000 7f5f6e000640 1 --2- 172.21.15.131:0/4233919354 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f5f5406be30 0x7f5f5406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:20.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:20.981+0000 7f5f6e000640 1 --2- 172.21.15.131:0/4233919354 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f5f5406be30 0x7f5f5406e2f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5f60034000 tx=0x7f5f6002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:20.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:20.985+0000 7f5f6f400640 1 -- 172.21.15.131:0/4233919354 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f5f68058d10 con 0x7f5f7806f420 2024-05-26T22:48:21.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:21.208+0000 7f5f7dc00640 1 -- 172.21.15.131:0/4233919354 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f5f780cee90 con 0x7f5f7806f420 2024-05-26T22:48:21.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:21.799+0000 7f5f6f400640 1 -- 172.21.15.131:0/4233919354 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7f5f68058610 con 0x7f5f7806f420 2024-05-26T22:48:21.800 INFO:teuthology.orchestra.run.smithi131.stderr:new fs with metadata pool 2 and data pool 3 2024-05-26T22:48:21.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:21.805+0000 7f5f7dc00640 1 -- 172.21.15.131:0/4233919354 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f5f5406be30 msgr2=0x7f5f5406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:21.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:21.805+0000 7f5f7dc00640 1 --2- 172.21.15.131:0/4233919354 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f5f5406be30 0x7f5f5406e2f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5f60034000 tx=0x7f5f6002e000 comp rx=0 tx=0).stop 2024-05-26T22:48:21.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:21.805+0000 7f5f7dc00640 1 -- 172.21.15.131:0/4233919354 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5f7806f420 msgr2=0x7f5f781492f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:21.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:21.805+0000 7f5f7dc00640 1 --2- 172.21.15.131:0/4233919354 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5f7806f420 0x7f5f781492f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5f6800c430 tx=0x7f5f6800c8f0 comp rx=0 tx=0).stop 2024-05-26T22:48:21.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:21.806+0000 7f5f7dc00640 1 -- 172.21.15.131:0/4233919354 shutdown_connections 2024-05-26T22:48:21.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:21.806+0000 7f5f7dc00640 1 --2- 172.21.15.131:0/4233919354 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5f780cf590 0x7f5f7814ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:21.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:21.806+0000 7f5f7dc00640 1 --2- 172.21.15.131:0/4233919354 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f5f5406be30 0x7f5f5406e2f0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:21.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:21.806+0000 7f5f7dc00640 1 --2- 172.21.15.131:0/4233919354 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5f780b5450 0x7f5f7809cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:21.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:21.806+0000 7f5f7dc00640 1 --2- 172.21.15.131:0/4233919354 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5f7806f420 0x7f5f781492f0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:21.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:21.806+0000 7f5f7dc00640 1 -- 172.21.15.131:0/4233919354 >> 172.21.15.131:0/4233919354 conn(0x7f5f7806bc90 msgr2=0x7f5f780afb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:21.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:21.808+0000 7f5f7dc00640 1 -- 172.21.15.131:0/4233919354 shutdown_connections 2024-05-26T22:48:21.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:21.808+0000 7f5f7dc00640 1 -- 172.21.15.131:0/4233919354 wait complete. 2024-05-26T22:48:21.840 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-26T22:48:22.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.010+0000 7fd57d800640 1 Processor -- start 2024-05-26T22:48:22.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.011+0000 7fd57d800640 1 -- start start 2024-05-26T22:48:22.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.012+0000 7fd57d800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd578155840 0x7fd578071930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:22.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.012+0000 7fd57d800640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5780722a0 con 0x7fd57815c3d0 2024-05-26T22:48:22.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.012+0000 7fd57d800640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd578072580 con 0x7fd57806f660 2024-05-26T22:48:22.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.012+0000 7fd57d800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd578072860 con 0x7fd578155840 2024-05-26T22:48:22.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.012+0000 7fd56fe00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fd57815c3d0 0x7fd57806ef50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:40510/0 (socket says 172.21.15.131:40510) 2024-05-26T22:48:22.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.012+0000 7fd56fe00640 1 -- 172.21.15.131:0/3860698334 learned_addr learned my addr 172.21.15.131:0/3860698334 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:48:22.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.013+0000 7fd576a00640 1 --2- 172.21.15.131:0/3860698334 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd578155840 0x7fd578071930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:22.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.013+0000 7fd576000640 1 -- 172.21.15.131:0/3860698334 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2170721645 0 0) 0x7fd5780722a0 con 0x7fd57815c3d0 2024-05-26T22:48:22.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.013+0000 7fd576000640 1 -- 172.21.15.131:0/3860698334 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd55c003520 con 0x7fd57815c3d0 2024-05-26T22:48:22.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.013+0000 7fd576000640 1 -- 172.21.15.131:0/3860698334 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2339236320 0 0) 0x7fd55c003520 con 0x7fd57815c3d0 2024-05-26T22:48:22.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.013+0000 7fd576000640 1 -- 172.21.15.131:0/3860698334 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd578155840 msgr2=0x7fd578071930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:22.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.013+0000 7fd576000640 1 --2- 172.21.15.131:0/3860698334 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd578155840 0x7fd578071930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:22.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.013+0000 7fd576000640 1 -- 172.21.15.131:0/3860698334 >> v1:172.21.15.187:6789/0 conn(0x7fd57806f660 legacy=0x7fd5780ade40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:22.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.013+0000 7fd576000640 1 -- 172.21.15.131:0/3860698334 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd578072b40 con 0x7fd57815c3d0 2024-05-26T22:48:22.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.013+0000 7fd576000640 1 -- 172.21.15.131:0/3860698334 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7fd560001670 con 0x7fd57815c3d0 2024-05-26T22:48:22.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.013+0000 7fd576000640 1 -- 172.21.15.131:0/3860698334 >> v1:172.21.15.131:6789/0 conn(0x7fd57815c3d0 legacy=0x7fd57806ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:22.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.013+0000 7fd576000640 1 --2- 172.21.15.131:0/3860698334 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd578155840 0x7fd55c0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:22.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.013+0000 7fd576000640 1 --2- 172.21.15.131:0/3860698334 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd55c003fb0 0x7fd55c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:22.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.013+0000 7fd576000640 1 --2- 172.21.15.131:0/3860698334 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd55c004d40 0x7fd55c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:22.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.013+0000 7fd576000640 1 -- 172.21.15.131:0/3860698334 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fd578072860 con 0x7fd55c004d40 2024-05-26T22:48:22.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.013+0000 7fd576000640 1 -- 172.21.15.131:0/3860698334 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd55c00bb00 con 0x7fd55c003fb0 2024-05-26T22:48:22.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.013+0000 7fd576000640 1 -- 172.21.15.131:0/3860698334 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd55c00bd00 con 0x7fd578155840 2024-05-26T22:48:22.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.014+0000 7fd577400640 1 --2- 172.21.15.131:0/3860698334 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd55c004d40 0x7fd55c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:22.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.014+0000 7fd56fe00640 1 --2- 172.21.15.131:0/3860698334 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd55c003fb0 0x7fd55c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:22.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.014+0000 7fd576a00640 1 --2- 172.21.15.131:0/3860698334 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd578155840 0x7fd55c0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:22.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.014+0000 7fd577400640 1 -- 172.21.15.131:0/3860698334 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd578155840 msgr2=0x7fd55c0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:22.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.014+0000 7fd577400640 1 --2- 172.21.15.131:0/3860698334 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd578155840 0x7fd55c0039e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:22.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.014+0000 7fd577400640 1 -- 172.21.15.131:0/3860698334 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd55c003fb0 msgr2=0x7fd55c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:22.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.014+0000 7fd577400640 1 --2- 172.21.15.131:0/3860698334 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd55c003fb0 0x7fd55c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:22.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.014+0000 7fd577400640 1 -- 172.21.15.131:0/3860698334 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd55c003520 con 0x7fd55c004d40 2024-05-26T22:48:22.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.015+0000 7fd577400640 1 --2- 172.21.15.131:0/3860698334 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd55c004d40 0x7fd55c0051c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd570001a30 tx=0x7fd55c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=976fe72489a2e4ee server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:22.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.015+0000 7fd576000640 1 -- 172.21.15.131:0/3860698334 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd570001a70 con 0x7fd55c004d40 2024-05-26T22:48:22.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.015+0000 7fd576000640 1 -- 172.21.15.131:0/3860698334 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd570003b70 con 0x7fd55c004d40 2024-05-26T22:48:22.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.015+0000 7fd576000640 1 -- 172.21.15.131:0/3860698334 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5700050d0 con 0x7fd55c004d40 2024-05-26T22:48:22.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.016+0000 7fd57d800640 1 -- 172.21.15.131:0/3860698334 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd55c004d40 msgr2=0x7fd55c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:22.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.016+0000 7fd57d800640 1 --2- 172.21.15.131:0/3860698334 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd55c004d40 0x7fd55c0051c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd570001a30 tx=0x7fd55c005830 comp rx=0 tx=0).stop 2024-05-26T22:48:22.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.016+0000 7fd577400640 1 -- 172.21.15.131:0/3860698334 reap_dead start 2024-05-26T22:48:22.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.016+0000 7fd57d800640 1 -- 172.21.15.131:0/3860698334 shutdown_connections 2024-05-26T22:48:22.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.016+0000 7fd57d800640 1 -- 172.21.15.131:0/3860698334 >> 172.21.15.131:0/3860698334 conn(0x7fd5780b1e90 msgr2=0x7fd5780b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:22.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.016+0000 7fd57d800640 1 -- 172.21.15.131:0/3860698334 shutdown_connections 2024-05-26T22:48:22.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.016+0000 7fd57d800640 1 -- 172.21.15.131:0/3860698334 wait complete. 2024-05-26T22:48:22.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.018+0000 7fd57d800640 1 Processor -- start 2024-05-26T22:48:22.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.018+0000 7fd57d800640 1 -- start start 2024-05-26T22:48:22.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.019+0000 7fd57d800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd55c004d40 0x7fd5780ae470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:22.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.019+0000 7fd57d800640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd57809a280 0x7fd57809db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:22.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.019+0000 7fd57d800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd57815b6c0 0x7fd57815bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:22.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.019+0000 7fd57d800640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fd578071ab0 con 0x7fd55c004d40 2024-05-26T22:48:22.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.019+0000 7fd57d800640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd578071930 con 0x7fd57809a280 2024-05-26T22:48:22.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.019+0000 7fd57d800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd578071c30 con 0x7fd57815b6c0 2024-05-26T22:48:22.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.019+0000 7fd576a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd55c004d40 0x7fd5780ae470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:22.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.019+0000 7fd576a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd55c004d40 0x7fd5780ae470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:34040/0 (socket says 172.21.15.131:34040) 2024-05-26T22:48:22.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.019+0000 7fd576a00640 1 -- 172.21.15.131:0/1009774412 learned_addr learned my addr 172.21.15.131:0/1009774412 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:22.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.019+0000 7fd56fe00640 1 --2- 172.21.15.131:0/1009774412 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd57809a280 0x7fd57809db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:22.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.019+0000 7fd576a00640 1 -- 172.21.15.131:0/1009774412 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd57815b6c0 msgr2=0x7fd57815bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:22.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.019+0000 7fd576a00640 1 --2- 172.21.15.131:0/1009774412 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd57815b6c0 0x7fd57815bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:22.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.019+0000 7fd576a00640 1 -- 172.21.15.131:0/1009774412 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd57809a280 msgr2=0x7fd57809db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:22.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.019+0000 7fd576a00640 1 --2- 172.21.15.131:0/1009774412 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd57809a280 0x7fd57809db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:22.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.019+0000 7fd576a00640 1 -- 172.21.15.131:0/1009774412 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd57809cb00 con 0x7fd55c004d40 2024-05-26T22:48:22.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.020+0000 7fd576a00640 1 --2- 172.21.15.131:0/1009774412 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd55c004d40 0x7fd5780ae470 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd568010890 tx=0x7fd568010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:22.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.020+0000 7fd56ea00640 1 -- 172.21.15.131:0/1009774412 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd56800f040 con 0x7fd55c004d40 2024-05-26T22:48:22.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.020+0000 7fd56ea00640 1 -- 172.21.15.131:0/1009774412 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd56800eae0 con 0x7fd55c004d40 2024-05-26T22:48:22.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.020+0000 7fd56ea00640 1 -- 172.21.15.131:0/1009774412 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd56801c400 con 0x7fd55c004d40 2024-05-26T22:48:22.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.020+0000 7fd57d800640 1 -- 172.21.15.131:0/1009774412 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd57809a700 con 0x7fd55c004d40 2024-05-26T22:48:22.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.020+0000 7fd57d800640 1 -- 172.21.15.131:0/1009774412 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd57809d080 con 0x7fd55c004d40 2024-05-26T22:48:22.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.021+0000 7fd57d800640 1 -- 172.21.15.131:0/1009774412 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5780694e0 con 0x7fd55c004d40 2024-05-26T22:48:22.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.022+0000 7fd56ea00640 1 -- 172.21.15.131:0/1009774412 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd56801c5a0 con 0x7fd55c004d40 2024-05-26T22:48:22.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.023+0000 7fd56ea00640 1 --2- 172.21.15.131:0/1009774412 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fd55406be30 0x7fd55406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:22.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.024+0000 7fd56ea00640 1 -- 172.21.15.131:0/1009774412 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7fd56808b4b0 con 0x7fd55c004d40 2024-05-26T22:48:22.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.024+0000 7fd56fe00640 1 --2- 172.21.15.131:0/1009774412 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fd55406be30 0x7fd55406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:22.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.024+0000 7fd56fe00640 1 --2- 172.21.15.131:0/1009774412 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fd55406be30 0x7fd55406e2f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd560001c00 tx=0x7fd560000f70 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:22.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.028+0000 7fd56ea00640 1 -- 172.21.15.131:0/1009774412 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fd568018030 con 0x7fd55c004d40 2024-05-26T22:48:22.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.228+0000 7fd57d800640 1 -- 172.21.15.131:0/1009774412 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd57806f0a0 con 0x7fd55c004d40 2024-05-26T22:48:22.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.231+0000 7fd56ea00640 1 -- 172.21.15.131:0/1009774412 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+16642 (secure 0 0 0) 0x7fd568059dc0 con 0x7fd55c004d40 2024-05-26T22:48:22.231 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:48:22.231 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":19,"fsid":"46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f","created":"2024-05-26T22:47:57.288905+0000","modified":"2024-05-26T22:48:21.210216+0000","last_up_change":"2024-05-26T22:48:09.602207+0000","last_in_change":"2024-05-26T22:48:00.687341+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-26T22:48:10.463339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-26T22:48:19.663254+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-26T22:48:20.235751+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"814eaf62-a18d-494b-8e27-fd8fb88adea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6801","nonce":2350940051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6803","nonce":2350940051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6807","nonce":2350940051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6805","nonce":2350940051}]},"public_addr":"172.21.15.131:6801/2350940051","cluster_addr":"172.21.15.131:6803/2350940051","heartbeat_back_addr":"172.21.15.131:6807/2350940051","heartbeat_front_addr":"172.21.15.131:6805/2350940051","state":["exists","up"]},{"osd":1,"uuid":"42e66287-f6c6-4e26-af3c-6dfaa3536d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6825","nonce":1525528247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6827","nonce":1525528247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6831","nonce":1525528247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6829","nonce":1525528247}]},"public_addr":"172.21.15.131:6825/1525528247","cluster_addr":"172.21.15.131:6827/1525528247","heartbeat_back_addr":"172.21.15.131:6831/1525528247","heartbeat_front_addr":"172.21.15.131:6829/1525528247","state":["exists","up"]},{"osd":2,"uuid":"0da34673-04a2-4fd0-b57a-0ff2b679b043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6817","nonce":3627842210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6819","nonce":3627842210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6823","nonce":3627842210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6821","nonce":3627842210}]},"public_addr":"172.21.15.131:6817/3627842210","cluster_addr":"172.21.15.131:6819/3627842210","heartbeat_back_addr":"172.21.15.131:6823/3627842210","heartbeat_front_addr":"172.21.15.131:6821/3627842210","state":["exists","up"]},{"osd":3,"uuid":"cf117a27-d6b2-43d3-ae52-b47988bf8eed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6809","nonce":350790591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6811","nonce":350790591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6815","nonce":350790591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6813","nonce":350790591}]},"public_addr":"172.21.15.131:6809/350790591","cluster_addr":"172.21.15.131:6811/350790591","heartbeat_back_addr":"172.21.15.131:6815/350790591","heartbeat_front_addr":"172.21.15.131:6813/350790591","state":["exists","up"]},{"osd":4,"uuid":"a092ccfc-be89-4a84-88d5-0f781fa715eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6816","nonce":244560995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6819","nonce":244560995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6823","nonce":244560995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6821","nonce":244560995}]},"public_addr":"172.21.15.187:6816/244560995","cluster_addr":"172.21.15.187:6819/244560995","heartbeat_back_addr":"172.21.15.187:6823/244560995","heartbeat_front_addr":"172.21.15.187:6821/244560995","state":["exists","up"]},{"osd":5,"uuid":"7513ba69-dd00-4a35-b190-9e8cc3b2bdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6801","nonce":324619398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6803","nonce":324619398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6812","nonce":324619398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6807","nonce":324619398}]},"public_addr":"172.21.15.187:6801/324619398","cluster_addr":"172.21.15.187:6803/324619398","heartbeat_back_addr":"172.21.15.187:6812/324619398","heartbeat_front_addr":"172.21.15.187:6807/324619398","state":["exists","up"]},{"osd":6,"uuid":"50081810-c093-4d2f-a4d7-b7c695c5400d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6806","nonce":1391347786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6809","nonce":1391347786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6818","nonce":1391347786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6813","nonce":1391347786}]},"public_addr":"172.21.15.187:6806/1391347786","cluster_addr":"172.21.15.187:6809/1391347786","heartbeat_back_addr":"172.21.15.187:6818/1391347786","heartbeat_front_addr":"172.21.15.187:6813/1391347786","state":["exists","up"]},{"osd":7,"uuid":"775dae05-54f5-4ad4-bbce-92863c6ed2ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6825","nonce":1220065622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6827","nonce":1220065622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6831","nonce":1220065622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6829","nonce":1220065622}]},"public_addr":"172.21.15.187:6825/1220065622","cluster_addr":"172.21.15.187:6827/1220065622","heartbeat_back_addr":"172.21.15.187:6831/1220065622","heartbeat_front_addr":"172.21.15.187:6829/1220065622","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:05.983520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.299516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.067323+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.307748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.018742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.258775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.010912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.295355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-26T22:48:22.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.234+0000 7fd57d800640 1 -- 172.21.15.131:0/1009774412 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fd55406be30 msgr2=0x7fd55406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:22.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.234+0000 7fd57d800640 1 --2- 172.21.15.131:0/1009774412 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fd55406be30 0x7fd55406e2f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd560001c00 tx=0x7fd560000f70 comp rx=0 tx=0).stop 2024-05-26T22:48:22.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.234+0000 7fd57d800640 1 -- 172.21.15.131:0/1009774412 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd55c004d40 msgr2=0x7fd5780ae470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:22.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.234+0000 7fd57d800640 1 --2- 172.21.15.131:0/1009774412 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd55c004d40 0x7fd5780ae470 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd568010890 tx=0x7fd568010d50 comp rx=0 tx=0).stop 2024-05-26T22:48:22.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.234+0000 7fd57d800640 1 -- 172.21.15.131:0/1009774412 shutdown_connections 2024-05-26T22:48:22.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.234+0000 7fd57d800640 1 --2- 172.21.15.131:0/1009774412 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd57815b6c0 0x7fd57815bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:22.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.234+0000 7fd57d800640 1 --2- 172.21.15.131:0/1009774412 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fd55406be30 0x7fd55406e2f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:22.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.234+0000 7fd57d800640 1 --2- 172.21.15.131:0/1009774412 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd57809a280 0x7fd57809db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:22.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.234+0000 7fd57d800640 1 --2- 172.21.15.131:0/1009774412 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd55c004d40 0x7fd5780ae470 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:22.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.234+0000 7fd57d800640 1 -- 172.21.15.131:0/1009774412 >> 172.21.15.131:0/1009774412 conn(0x7fd5780b1e90 msgr2=0x7fd57815e220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:22.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.234+0000 7fd57d800640 1 -- 172.21.15.131:0/1009774412 shutdown_connections 2024-05-26T22:48:22.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.234+0000 7fd57d800640 1 -- 172.21.15.131:0/1009774412 wait complete. 2024-05-26T22:48:22.252 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-26T22:48:22.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.437+0000 7fbad3200640 1 Processor -- start 2024-05-26T22:48:22.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.437+0000 7fbad3200640 1 -- start start 2024-05-26T22:48:22.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.439+0000 7fbad3200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fbacc0b5b90 0x7fbacc0d23d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:22.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.439+0000 7fbad3200640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fbacc0b9990 0x7fbacc15ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:22.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.439+0000 7fbad3200640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbacc162330 con 0x7fbacc161de0 2024-05-26T22:48:22.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.439+0000 7fbad3200640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fbacc15e970 con 0x7fbacc0b5b90 2024-05-26T22:48:22.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.439+0000 7fbad3200640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fbacc0d2910 con 0x7fbacc0b9990 2024-05-26T22:48:22.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.439+0000 7fbad1e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fbacc0b5b90 0x7fbacc0d23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:22.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.439+0000 7fbad1e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fbacc0b5b90 0x7fbacc0d23d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:34054/0 (socket says 172.21.15.131:34054) 2024-05-26T22:48:22.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.439+0000 7fbad1e00640 1 -- 172.21.15.131:0/824408680 learned_addr learned my addr 172.21.15.131:0/824408680 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:22.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.439+0000 7fbad2800640 1 --2- 172.21.15.131:0/824408680 >> v2:172.21.15.187:3300/0 conn(0x7fbacc0b9990 0x7fbacc15ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:22.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.440+0000 7fbad1e00640 1 -- 172.21.15.131:0/824408680 >> v2:172.21.15.187:3300/0 conn(0x7fbacc0b9990 msgr2=0x7fbacc15ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:22.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.440+0000 7fbad1e00640 1 --2- 172.21.15.131:0/824408680 >> v2:172.21.15.187:3300/0 conn(0x7fbacc0b9990 0x7fbacc15ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:22.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.440+0000 7fbad1e00640 1 -- 172.21.15.131:0/824408680 >> v1:172.21.15.187:6789/0 conn(0x7fbacc161de0 legacy=0x7fbacc0b5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:22.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.440+0000 7fbad1e00640 1 -- 172.21.15.131:0/824408680 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbacc162610 con 0x7fbacc0b5b90 2024-05-26T22:48:22.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.441+0000 7fbad1e00640 1 --2- 172.21.15.131:0/824408680 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbacc0b5b90 0x7fbacc0d23d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fbacc06d3c0 tx=0x7fbac002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8986cf283cfbb22c server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:22.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.441+0000 7fbad0a00640 1 -- 172.21.15.131:0/824408680 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac0037070 con 0x7fbacc0b5b90 2024-05-26T22:48:22.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.441+0000 7fbad0a00640 1 -- 172.21.15.131:0/824408680 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbac0032440 con 0x7fbacc0b5b90 2024-05-26T22:48:22.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.441+0000 7fbad0a00640 1 -- 172.21.15.131:0/824408680 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac0032810 con 0x7fbacc0b5b90 2024-05-26T22:48:22.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.442+0000 7fbad3200640 1 -- 172.21.15.131:0/824408680 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbacc0b5b90 msgr2=0x7fbacc0d23d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:22.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.442+0000 7fbad3200640 1 --2- 172.21.15.131:0/824408680 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbacc0b5b90 0x7fbacc0d23d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fbacc06d3c0 tx=0x7fbac002a1a0 comp rx=0 tx=0).stop 2024-05-26T22:48:22.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.442+0000 7fbad3200640 1 -- 172.21.15.131:0/824408680 shutdown_connections 2024-05-26T22:48:22.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.442+0000 7fbad3200640 1 --2- 172.21.15.131:0/824408680 >> v2:172.21.15.187:3300/0 conn(0x7fbacc0b9990 0x7fbacc15ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:22.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.442+0000 7fbad3200640 1 --2- 172.21.15.131:0/824408680 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbacc0b5b90 0x7fbacc0d23d0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:22.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.442+0000 7fbad3200640 1 -- 172.21.15.131:0/824408680 >> 172.21.15.131:0/824408680 conn(0x7fbacc073070 msgr2=0x7fbacc0d0b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:22.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.442+0000 7fbad3200640 1 -- 172.21.15.131:0/824408680 shutdown_connections 2024-05-26T22:48:22.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.442+0000 7fbad3200640 1 -- 172.21.15.131:0/824408680 wait complete. 2024-05-26T22:48:22.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.444+0000 7fbad3200640 1 Processor -- start 2024-05-26T22:48:22.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.444+0000 7fbad3200640 1 -- start start 2024-05-26T22:48:22.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.445+0000 7fbad3200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbacc0b5b90 0x7fbacc1f6f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:22.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.445+0000 7fbad3200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbacc0b9990 0x7fbacc0d2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:22.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.445+0000 7fbad3200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbacc161de0 0x7fbacc0d3000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:22.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.445+0000 7fbad3200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fbacc0b9e20 con 0x7fbacc161de0 2024-05-26T22:48:22.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.445+0000 7fbad3200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fbacc162bd0 con 0x7fbacc0b9990 2024-05-26T22:48:22.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.445+0000 7fbad3200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fbacc0b9fa0 con 0x7fbacc0b5b90 2024-05-26T22:48:22.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.445+0000 7fbad2800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbacc161de0 0x7fbacc0d3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:22.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.445+0000 7fbad2800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbacc161de0 0x7fbacc0d3000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:34060/0 (socket says 172.21.15.131:34060) 2024-05-26T22:48:22.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.445+0000 7fbad2800640 1 -- 172.21.15.131:0/4257482523 learned_addr learned my addr 172.21.15.131:0/4257482523 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:22.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.445+0000 7fbad1e00640 1 --2- 172.21.15.131:0/4257482523 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbacc0b5b90 0x7fbacc1f6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:22.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.445+0000 7fbad1400640 1 --2- 172.21.15.131:0/4257482523 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbacc0b9990 0x7fbacc0d2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:22.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.445+0000 7fbad2800640 1 -- 172.21.15.131:0/4257482523 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbacc0b5b90 msgr2=0x7fbacc1f6f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:22.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.445+0000 7fbad2800640 1 --2- 172.21.15.131:0/4257482523 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbacc0b5b90 0x7fbacc1f6f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:22.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.445+0000 7fbad2800640 1 -- 172.21.15.131:0/4257482523 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbacc0b9990 msgr2=0x7fbacc0d2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:22.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.445+0000 7fbad2800640 1 --2- 172.21.15.131:0/4257482523 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbacc0b9990 0x7fbacc0d2ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:22.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.445+0000 7fbad2800640 1 -- 172.21.15.131:0/4257482523 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbacc0c9440 con 0x7fbacc161de0 2024-05-26T22:48:22.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.445+0000 7fbad1400640 1 --2- 172.21.15.131:0/4257482523 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbacc0b9990 0x7fbacc0d2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:48:22.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.446+0000 7fbad2800640 1 --2- 172.21.15.131:0/4257482523 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbacc161de0 0x7fbacc0d3000 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fbac8007e40 tx=0x7fbac800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:22.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.446+0000 7fbac6000640 1 -- 172.21.15.131:0/4257482523 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac8014620 con 0x7fbacc161de0 2024-05-26T22:48:22.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.446+0000 7fbac6000640 1 -- 172.21.15.131:0/4257482523 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbac8019070 con 0x7fbacc161de0 2024-05-26T22:48:22.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.446+0000 7fbac6000640 1 -- 172.21.15.131:0/4257482523 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac8014a20 con 0x7fbacc161de0 2024-05-26T22:48:22.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.446+0000 7fbad3200640 1 -- 172.21.15.131:0/4257482523 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbacc0c9720 con 0x7fbacc161de0 2024-05-26T22:48:22.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.446+0000 7fbad3200640 1 -- 172.21.15.131:0/4257482523 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbacc0c9ce0 con 0x7fbacc161de0 2024-05-26T22:48:22.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.447+0000 7fbad3200640 1 -- 172.21.15.131:0/4257482523 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbacc0694e0 con 0x7fbacc161de0 2024-05-26T22:48:22.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.448+0000 7fbac6000640 1 -- 172.21.15.131:0/4257482523 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbac8014bc0 con 0x7fbacc161de0 2024-05-26T22:48:22.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.449+0000 7fbac6000640 1 --2- 172.21.15.131:0/4257482523 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fbab006be30 0x7fbab006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:22.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.449+0000 7fbac6000640 1 -- 172.21.15.131:0/4257482523 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7fbac808b7b0 con 0x7fbacc161de0 2024-05-26T22:48:22.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.449+0000 7fbad1e00640 1 --2- 172.21.15.131:0/4257482523 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fbab006be30 0x7fbab006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:22.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.450+0000 7fbad1e00640 1 --2- 172.21.15.131:0/4257482523 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fbab006be30 0x7fbab006e2f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fbac0034000 tx=0x7fbac002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:22.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.454+0000 7fbac6000640 1 -- 172.21.15.131:0/4257482523 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fbac805a0c0 con 0x7fbacc161de0 2024-05-26T22:48:22.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.641+0000 7fbad3200640 1 -- 172.21.15.131:0/4257482523 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbacc06cfd0 con 0x7fbacc161de0 2024-05-26T22:48:22.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.645+0000 7fbac6000640 1 -- 172.21.15.131:0/4257482523 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+16642 (secure 0 0 0) 0x7fbac80599c0 con 0x7fbacc161de0 2024-05-26T22:48:22.645 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:48:22.645 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":19,"fsid":"46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f","created":"2024-05-26T22:47:57.288905+0000","modified":"2024-05-26T22:48:21.210216+0000","last_up_change":"2024-05-26T22:48:09.602207+0000","last_in_change":"2024-05-26T22:48:00.687341+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-26T22:48:10.463339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-26T22:48:19.663254+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-26T22:48:20.235751+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"814eaf62-a18d-494b-8e27-fd8fb88adea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6801","nonce":2350940051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6803","nonce":2350940051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6807","nonce":2350940051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6805","nonce":2350940051}]},"public_addr":"172.21.15.131:6801/2350940051","cluster_addr":"172.21.15.131:6803/2350940051","heartbeat_back_addr":"172.21.15.131:6807/2350940051","heartbeat_front_addr":"172.21.15.131:6805/2350940051","state":["exists","up"]},{"osd":1,"uuid":"42e66287-f6c6-4e26-af3c-6dfaa3536d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6825","nonce":1525528247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6827","nonce":1525528247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6831","nonce":1525528247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6829","nonce":1525528247}]},"public_addr":"172.21.15.131:6825/1525528247","cluster_addr":"172.21.15.131:6827/1525528247","heartbeat_back_addr":"172.21.15.131:6831/1525528247","heartbeat_front_addr":"172.21.15.131:6829/1525528247","state":["exists","up"]},{"osd":2,"uuid":"0da34673-04a2-4fd0-b57a-0ff2b679b043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6817","nonce":3627842210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6819","nonce":3627842210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6823","nonce":3627842210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6821","nonce":3627842210}]},"public_addr":"172.21.15.131:6817/3627842210","cluster_addr":"172.21.15.131:6819/3627842210","heartbeat_back_addr":"172.21.15.131:6823/3627842210","heartbeat_front_addr":"172.21.15.131:6821/3627842210","state":["exists","up"]},{"osd":3,"uuid":"cf117a27-d6b2-43d3-ae52-b47988bf8eed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6809","nonce":350790591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6811","nonce":350790591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6815","nonce":350790591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6813","nonce":350790591}]},"public_addr":"172.21.15.131:6809/350790591","cluster_addr":"172.21.15.131:6811/350790591","heartbeat_back_addr":"172.21.15.131:6815/350790591","heartbeat_front_addr":"172.21.15.131:6813/350790591","state":["exists","up"]},{"osd":4,"uuid":"a092ccfc-be89-4a84-88d5-0f781fa715eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6816","nonce":244560995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6819","nonce":244560995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6823","nonce":244560995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6821","nonce":244560995}]},"public_addr":"172.21.15.187:6816/244560995","cluster_addr":"172.21.15.187:6819/244560995","heartbeat_back_addr":"172.21.15.187:6823/244560995","heartbeat_front_addr":"172.21.15.187:6821/244560995","state":["exists","up"]},{"osd":5,"uuid":"7513ba69-dd00-4a35-b190-9e8cc3b2bdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6801","nonce":324619398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6803","nonce":324619398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6812","nonce":324619398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6807","nonce":324619398}]},"public_addr":"172.21.15.187:6801/324619398","cluster_addr":"172.21.15.187:6803/324619398","heartbeat_back_addr":"172.21.15.187:6812/324619398","heartbeat_front_addr":"172.21.15.187:6807/324619398","state":["exists","up"]},{"osd":6,"uuid":"50081810-c093-4d2f-a4d7-b7c695c5400d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6806","nonce":1391347786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6809","nonce":1391347786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6818","nonce":1391347786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6813","nonce":1391347786}]},"public_addr":"172.21.15.187:6806/1391347786","cluster_addr":"172.21.15.187:6809/1391347786","heartbeat_back_addr":"172.21.15.187:6818/1391347786","heartbeat_front_addr":"172.21.15.187:6813/1391347786","state":["exists","up"]},{"osd":7,"uuid":"775dae05-54f5-4ad4-bbce-92863c6ed2ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6825","nonce":1220065622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6827","nonce":1220065622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6831","nonce":1220065622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6829","nonce":1220065622}]},"public_addr":"172.21.15.187:6825/1220065622","cluster_addr":"172.21.15.187:6827/1220065622","heartbeat_back_addr":"172.21.15.187:6831/1220065622","heartbeat_front_addr":"172.21.15.187:6829/1220065622","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:05.983520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.299516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.067323+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.307748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.018742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.258775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.010912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.295355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-26T22:48:22.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.647+0000 7fbad3200640 1 -- 172.21.15.131:0/4257482523 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fbab006be30 msgr2=0x7fbab006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:22.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.647+0000 7fbad3200640 1 --2- 172.21.15.131:0/4257482523 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fbab006be30 0x7fbab006e2f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fbac0034000 tx=0x7fbac002e000 comp rx=0 tx=0).stop 2024-05-26T22:48:22.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.647+0000 7fbad3200640 1 -- 172.21.15.131:0/4257482523 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbacc161de0 msgr2=0x7fbacc0d3000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:22.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.647+0000 7fbad3200640 1 --2- 172.21.15.131:0/4257482523 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbacc161de0 0x7fbacc0d3000 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fbac8007e40 tx=0x7fbac800e4b0 comp rx=0 tx=0).stop 2024-05-26T22:48:22.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.647+0000 7fbad3200640 1 -- 172.21.15.131:0/4257482523 shutdown_connections 2024-05-26T22:48:22.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.647+0000 7fbad3200640 1 --2- 172.21.15.131:0/4257482523 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fbab006be30 0x7fbab006e2f0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:22.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.647+0000 7fbad3200640 1 --2- 172.21.15.131:0/4257482523 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbacc161de0 0x7fbacc0d3000 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:22.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.647+0000 7fbad3200640 1 --2- 172.21.15.131:0/4257482523 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbacc0b9990 0x7fbacc0d2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:22.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.647+0000 7fbad3200640 1 --2- 172.21.15.131:0/4257482523 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbacc0b5b90 0x7fbacc1f6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:22.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.647+0000 7fbad3200640 1 -- 172.21.15.131:0/4257482523 >> 172.21.15.131:0/4257482523 conn(0x7fbacc073070 msgr2=0x7fbacc06c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:22.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.648+0000 7fbad3200640 1 -- 172.21.15.131:0/4257482523 shutdown_connections 2024-05-26T22:48:22.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.648+0000 7fbad3200640 1 -- 172.21.15.131:0/4257482523 wait complete. 2024-05-26T22:48:22.664 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-26T22:48:22.738 INFO:tasks.ceph.mds.d.smithi187.stderr:2024-05-26T22:48:22.739+0000 7fd87d600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-26T22:48:22.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.866+0000 7fd7a1a00640 1 Processor -- start 2024-05-26T22:48:22.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.866+0000 7fd7a1a00640 1 -- start start 2024-05-26T22:48:22.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.867+0000 7fd7a1a00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fd79c0cc2d0 0x7fd79c06d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:22.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.867+0000 7fd7a1a00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd79c0cc780 con 0x7fd79c0cbef0 2024-05-26T22:48:22.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.867+0000 7fd7a1a00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd79c0cdca0 con 0x7fd79c06d860 2024-05-26T22:48:22.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.867+0000 7fd7a1a00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fd79c0ad600 con 0x7fd79c0cc2d0 2024-05-26T22:48:22.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.867+0000 7fd792000640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fd79c0cc2d0 0x7fd79c06d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:22.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.867+0000 7fd79aa00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fd79c0cbef0 0x7fd79c0acef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:40520/0 (socket says 172.21.15.131:40520) 2024-05-26T22:48:22.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.867+0000 7fd792000640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fd79c0cc2d0 0x7fd79c06d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:34064/0 (socket says 172.21.15.131:34064) 2024-05-26T22:48:22.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.868+0000 7fd79aa00640 1 -- 172.21.15.131:0/3391114470 learned_addr learned my addr 172.21.15.131:0/3391114470 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:48:22.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.868+0000 7fd792000640 1 -- 172.21.15.131:0/3391114470 >> v1:172.21.15.187:6789/0 conn(0x7fd79c06d860 legacy=0x7fd79c0cea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:22.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.868+0000 7fd79a000640 1 -- 172.21.15.131:0/3391114470 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2972804313 0 0) 0x7fd78c001110 con 0x7fd79c0cbef0 2024-05-26T22:48:22.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.868+0000 7fd792000640 1 -- 172.21.15.131:0/3391114470 >> v1:172.21.15.131:6789/0 conn(0x7fd79c0cbef0 legacy=0x7fd79c0acef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:22.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.868+0000 7fd792000640 1 -- 172.21.15.131:0/3391114470 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd79c0cdf80 con 0x7fd79c0cc2d0 2024-05-26T22:48:22.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.869+0000 7fd792000640 1 --2- 172.21.15.131:0/3391114470 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd79c0cc2d0 0x7fd79c06d320 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd78400ba40 tx=0x7fd78402c690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3350c6fcab2d726c server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:22.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.869+0000 7fd79a000640 1 -- 172.21.15.131:0/3391114470 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd78403a070 con 0x7fd79c0cc2d0 2024-05-26T22:48:22.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.869+0000 7fd79a000640 1 -- 172.21.15.131:0/3391114470 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd784032480 con 0x7fd79c0cc2d0 2024-05-26T22:48:22.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.869+0000 7fd79a000640 1 -- 172.21.15.131:0/3391114470 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd784032850 con 0x7fd79c0cc2d0 2024-05-26T22:48:22.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.870+0000 7fd7a1a00640 1 -- 172.21.15.131:0/3391114470 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd79c0cc2d0 msgr2=0x7fd79c06d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:22.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.870+0000 7fd7a1a00640 1 --2- 172.21.15.131:0/3391114470 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd79c0cc2d0 0x7fd79c06d320 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd78400ba40 tx=0x7fd78402c690 comp rx=0 tx=0).stop 2024-05-26T22:48:22.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.870+0000 7fd7a1a00640 1 -- 172.21.15.131:0/3391114470 shutdown_connections 2024-05-26T22:48:22.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.870+0000 7fd7a1a00640 1 --2- 172.21.15.131:0/3391114470 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd79c0cc2d0 0x7fd79c06d320 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:22.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.870+0000 7fd7a1a00640 1 -- 172.21.15.131:0/3391114470 >> 172.21.15.131:0/3391114470 conn(0x7fd79c06bc90 msgr2=0x7fd79c0b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:22.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.870+0000 7fd7a1a00640 1 -- 172.21.15.131:0/3391114470 shutdown_connections 2024-05-26T22:48:22.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.870+0000 7fd7a1a00640 1 -- 172.21.15.131:0/3391114470 wait complete. 2024-05-26T22:48:22.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.872+0000 7fd7a1a00640 1 Processor -- start 2024-05-26T22:48:22.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.872+0000 7fd7a1a00640 1 -- start start 2024-05-26T22:48:22.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.873+0000 7fd7a1a00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd79c06d860 0x7fd79c09b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:22.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.873+0000 7fd7a1a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd79c0cbef0 0x7fd79c1f4170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:22.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.873+0000 7fd7a1a00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd79c0cc2d0 0x7fd79c14ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:22.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.873+0000 7fd7a1a00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fd79c0ce540 con 0x7fd79c0cbef0 2024-05-26T22:48:22.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.873+0000 7fd7a1a00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd79c06dcd0 con 0x7fd79c06d860 2024-05-26T22:48:22.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.873+0000 7fd7a1a00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd79c0cf170 con 0x7fd79c0cc2d0 2024-05-26T22:48:22.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.873+0000 7fd792000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd79c0cbef0 0x7fd79c1f4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:22.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.873+0000 7fd792000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd79c0cbef0 0x7fd79c1f4170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:34074/0 (socket says 172.21.15.131:34074) 2024-05-26T22:48:22.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.873+0000 7fd792000640 1 -- 172.21.15.131:0/2991296118 learned_addr learned my addr 172.21.15.131:0/2991296118 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:22.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.874+0000 7fd79aa00640 1 --2- 172.21.15.131:0/2991296118 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd79c06d860 0x7fd79c09b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:22.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.874+0000 7fd792000640 1 -- 172.21.15.131:0/2991296118 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd79c0cc2d0 msgr2=0x7fd79c14ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:22.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.874+0000 7fd79b400640 1 --2- 172.21.15.131:0/2991296118 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd79c0cc2d0 0x7fd79c14ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:22.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.874+0000 7fd792000640 1 --2- 172.21.15.131:0/2991296118 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd79c0cc2d0 0x7fd79c14ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:22.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.874+0000 7fd792000640 1 -- 172.21.15.131:0/2991296118 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd79c06d860 msgr2=0x7fd79c09b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:22.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.874+0000 7fd792000640 1 --2- 172.21.15.131:0/2991296118 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd79c06d860 0x7fd79c09b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:22.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.874+0000 7fd792000640 1 -- 172.21.15.131:0/2991296118 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd79c14cf50 con 0x7fd79c0cbef0 2024-05-26T22:48:22.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.874+0000 7fd792000640 1 --2- 172.21.15.131:0/2991296118 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd79c0cbef0 0x7fd79c1f4170 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd784038040 tx=0x7fd7840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:22.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.874+0000 7fd793400640 1 -- 172.21.15.131:0/2991296118 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd78403a040 con 0x7fd79c0cbef0 2024-05-26T22:48:22.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.874+0000 7fd793400640 1 -- 172.21.15.131:0/2991296118 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd784045070 con 0x7fd79c0cbef0 2024-05-26T22:48:22.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.874+0000 7fd7a1a00640 1 -- 172.21.15.131:0/2991296118 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd79c1f4910 con 0x7fd79c0cbef0 2024-05-26T22:48:22.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.874+0000 7fd793400640 1 -- 172.21.15.131:0/2991296118 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd784032e60 con 0x7fd79c0cbef0 2024-05-26T22:48:22.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.875+0000 7fd7a1a00640 1 -- 172.21.15.131:0/2991296118 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd79c1f4ed0 con 0x7fd79c0cbef0 2024-05-26T22:48:22.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.876+0000 7fd7a1a00640 1 -- 172.21.15.131:0/2991296118 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd79c074e90 con 0x7fd79c0cbef0 2024-05-26T22:48:22.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.877+0000 7fd793400640 1 -- 172.21.15.131:0/2991296118 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd784035a60 con 0x7fd79c0cbef0 2024-05-26T22:48:22.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.878+0000 7fd793400640 1 --2- 172.21.15.131:0/2991296118 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fd77806be30 0x7fd77806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:22.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.878+0000 7fd793400640 1 -- 172.21.15.131:0/2991296118 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd7840abfb0 con 0x7fd79c0cbef0 2024-05-26T22:48:22.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.878+0000 7fd79aa00640 1 --2- 172.21.15.131:0/2991296118 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fd77806be30 0x7fd77806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:22.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.879+0000 7fd79aa00640 1 --2- 172.21.15.131:0/2991296118 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fd77806be30 0x7fd77806e2f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd78c000f60 tx=0x7fd78c007970 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:22.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:22.883+0000 7fd793400640 1 -- 172.21.15.131:0/2991296118 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fd78407a9f0 con 0x7fd79c0cbef0 2024-05-26T22:48:23.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:23.117+0000 7fd7a1a00640 1 -- 172.21.15.131:0/2991296118 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fd79c15a370 con 0x7fd79c0cbef0 2024-05-26T22:48:24.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:24.059+0000 7fd793400640 1 -- 172.21.15.131:0/2991296118 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v6) ==== 118+0+0 (secure 0 0 0) 0x7fd784048e40 con 0x7fd79c0cbef0 2024-05-26T22:48:24.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:24.062+0000 7fd7a1a00640 1 -- 172.21.15.131:0/2991296118 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fd77806be30 msgr2=0x7fd77806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:24.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:24.062+0000 7fd7a1a00640 1 --2- 172.21.15.131:0/2991296118 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fd77806be30 0x7fd77806e2f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd78c000f60 tx=0x7fd78c007970 comp rx=0 tx=0).stop 2024-05-26T22:48:24.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:24.062+0000 7fd7a1a00640 1 -- 172.21.15.131:0/2991296118 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd79c0cbef0 msgr2=0x7fd79c1f4170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:24.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:24.062+0000 7fd7a1a00640 1 --2- 172.21.15.131:0/2991296118 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd79c0cbef0 0x7fd79c1f4170 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd784038040 tx=0x7fd7840046c0 comp rx=0 tx=0).stop 2024-05-26T22:48:24.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:24.062+0000 7fd7a1a00640 1 -- 172.21.15.131:0/2991296118 shutdown_connections 2024-05-26T22:48:24.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:24.062+0000 7fd7a1a00640 1 --2- 172.21.15.131:0/2991296118 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd79c0cc2d0 0x7fd79c14ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:24.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:24.062+0000 7fd7a1a00640 1 --2- 172.21.15.131:0/2991296118 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fd77806be30 0x7fd77806e2f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:24.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:24.062+0000 7fd7a1a00640 1 --2- 172.21.15.131:0/2991296118 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd79c0cbef0 0x7fd79c1f4170 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:24.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:24.062+0000 7fd7a1a00640 1 --2- 172.21.15.131:0/2991296118 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd79c06d860 0x7fd79c09b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:24.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:24.062+0000 7fd7a1a00640 1 -- 172.21.15.131:0/2991296118 >> 172.21.15.131:0/2991296118 conn(0x7fd79c06bc90 msgr2=0x7fd79c0acef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:24.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:24.063+0000 7fd7a1a00640 1 -- 172.21.15.131:0/2991296118 shutdown_connections 2024-05-26T22:48:24.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:24.063+0000 7fd7a1a00640 1 -- 172.21.15.131:0/2991296118 wait complete. 2024-05-26T22:48:24.083 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-26T22:48:24.083 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-26T22:48:24.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:24.288+0000 7f9feac00640 1 Processor -- start 2024-05-26T22:48:24.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:24.288+0000 7f9feac00640 1 -- start start 2024-05-26T22:48:24.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:24.289+0000 7f9feac00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9fe415e340 0x7f9fe40b72e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:24.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:24.289+0000 7f9feac00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f9fe40b7820 0x7f9fe40d22b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:24.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:24.289+0000 7f9feac00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9fe415e720 con 0x7f9fe40d27f0 2024-05-26T22:48:24.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:24.289+0000 7f9feac00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f9fe40b88b0 con 0x7f9fe40b7820 2024-05-26T22:48:24.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:24.289+0000 7f9feac00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9fe40b7ca0 con 0x7f9fe415e340 2024-05-26T22:48:24.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:24.290+0000 7f9fe9800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9fe415e340 0x7f9fe40b72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:24.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:24.290+0000 7f9fe8e00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f9fe40b7820 0x7f9fe40d22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:24.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:24.290+0000 7f9fe9800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9fe415e340 0x7f9fe40b72e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.131:36018/0 (socket says 172.21.15.131:36018) 2024-05-26T22:48:24.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:24.290+0000 7f9fe9800640 1 -- 172.21.15.131:0/1899276763 learned_addr learned my addr 172.21.15.131:0/1899276763 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:24.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:24.291+0000 7f9fe9800640 1 -- 172.21.15.131:0/1899276763 >> v2:172.21.15.187:3300/0 conn(0x7f9fe40b7820 msgr2=0x7f9fe40d22b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:24.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:24.291+0000 7f9fe9800640 1 --2- 172.21.15.131:0/1899276763 >> v2:172.21.15.187:3300/0 conn(0x7f9fe40b7820 0x7f9fe40d22b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:24.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:24.291+0000 7f9fe9800640 1 -- 172.21.15.131:0/1899276763 >> v1:172.21.15.187:6789/0 conn(0x7f9fe40d27f0 legacy=0x7f9fe40b8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:24.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:24.291+0000 7f9fe9800640 1 -- 172.21.15.131:0/1899276763 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fe40b7f80 con 0x7f9fe415e340 2024-05-26T22:48:24.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:24.292+0000 7f9fe9800640 1 --2- 172.21.15.131:0/1899276763 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9fe415e340 0x7f9fe40b72e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f9fe40b8a30 tx=0x7f9fd802a2e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5e0d95062366ea5 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:24.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:24.292+0000 7f9fe3e00640 1 -- 172.21.15.131:0/1899276763 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fd8037070 con 0x7f9fe415e340 2024-05-26T22:48:24.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:24.293+0000 7f9fe3e00640 1 -- 172.21.15.131:0/1899276763 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fd8032640 con 0x7f9fe415e340 2024-05-26T22:48:24.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:24.293+0000 7f9fe3e00640 1 -- 172.21.15.131:0/1899276763 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fd8032a10 con 0x7f9fe415e340 2024-05-26T22:48:24.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:24.293+0000 7f9feac00640 1 -- 172.21.15.131:0/1899276763 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9fe415e340 msgr2=0x7f9fe40b72e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:24.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:24.293+0000 7f9feac00640 1 --2- 172.21.15.131:0/1899276763 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9fe415e340 0x7f9fe40b72e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f9fe40b8a30 tx=0x7f9fd802a2e0 comp rx=0 tx=0).stop 2024-05-26T22:48:24.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:24.293+0000 7f9feac00640 1 -- 172.21.15.131:0/1899276763 shutdown_connections 2024-05-26T22:48:24.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:24.293+0000 7f9feac00640 1 --2- 172.21.15.131:0/1899276763 >> v2:172.21.15.187:3300/0 conn(0x7f9fe40b7820 0x7f9fe40d22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:24.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:24.293+0000 7f9feac00640 1 --2- 172.21.15.131:0/1899276763 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9fe415e340 0x7f9fe40b72e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:24.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:24.293+0000 7f9feac00640 1 -- 172.21.15.131:0/1899276763 >> 172.21.15.131:0/1899276763 conn(0x7f9fe4072e60 msgr2=0x7f9fe4073270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:24.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:24.293+0000 7f9feac00640 1 -- 172.21.15.131:0/1899276763 shutdown_connections 2024-05-26T22:48:24.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:24.293+0000 7f9feac00640 1 -- 172.21.15.131:0/1899276763 wait complete. 2024-05-26T22:48:24.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:24.295+0000 7f9feac00640 1 Processor -- start 2024-05-26T22:48:24.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:24.295+0000 7f9feac00640 1 -- start start 2024-05-26T22:48:24.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:24.296+0000 7f9feac00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9fe40b7820 0x7f9fe4156240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:24.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:24.296+0000 7f9feac00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9fe40d27f0 0x7f9fe41533c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:24.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:24.296+0000 7f9feac00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9fe415e340 0x7f9fe41f6a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:24.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:24.296+0000 7f9feac00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe40b9450 con 0x7f9fe415e340 2024-05-26T22:48:24.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:24.296+0000 7f9feac00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe40b8540 con 0x7f9fe40d27f0 2024-05-26T22:48:24.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:24.296+0000 7f9feac00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9fe40b95d0 con 0x7f9fe40b7820 2024-05-26T22:48:24.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:24.297+0000 7f9fea200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9fe415e340 0x7f9fe41f6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:24.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:24.297+0000 7f9fea200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9fe415e340 0x7f9fe41f6a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53842/0 (socket says 172.21.15.131:53842) 2024-05-26T22:48:24.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:24.297+0000 7f9fea200640 1 -- 172.21.15.131:0/2671894102 learned_addr learned my addr 172.21.15.131:0/2671894102 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:24.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:24.297+0000 7f9fe8e00640 1 --2- 172.21.15.131:0/2671894102 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9fe40d27f0 0x7f9fe41533c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:24.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:24.297+0000 7f9fe9800640 1 --2- 172.21.15.131:0/2671894102 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9fe40b7820 0x7f9fe4156240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:24.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:24.297+0000 7f9fea200640 1 -- 172.21.15.131:0/2671894102 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9fe40b7820 msgr2=0x7f9fe4156240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:24.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:24.297+0000 7f9fea200640 1 --2- 172.21.15.131:0/2671894102 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9fe40b7820 0x7f9fe4156240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:24.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:24.297+0000 7f9fea200640 1 -- 172.21.15.131:0/2671894102 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9fe40d27f0 msgr2=0x7f9fe41533c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:24.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:24.297+0000 7f9fea200640 1 --2- 172.21.15.131:0/2671894102 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9fe40d27f0 0x7f9fe41533c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:24.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:24.297+0000 7f9fea200640 1 -- 172.21.15.131:0/2671894102 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fe41551a0 con 0x7f9fe415e340 2024-05-26T22:48:24.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:24.297+0000 7f9fe9800640 1 --2- 172.21.15.131:0/2671894102 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9fe40b7820 0x7f9fe4156240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:48:24.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:24.297+0000 7f9fea200640 1 --2- 172.21.15.131:0/2671894102 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9fe415e340 0x7f9fe41f6a80 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc00c730 tx=0x7f9fdc00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:24.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:24.297+0000 7f9fe1600640 1 -- 172.21.15.131:0/2671894102 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fdc003b70 con 0x7f9fe415e340 2024-05-26T22:48:24.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:24.298+0000 7f9fe1600640 1 -- 172.21.15.131:0/2671894102 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fdc003d10 con 0x7f9fe415e340 2024-05-26T22:48:24.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:24.298+0000 7f9feac00640 1 -- 172.21.15.131:0/2671894102 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fe4155480 con 0x7f9fe415e340 2024-05-26T22:48:24.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:24.298+0000 7f9fe1600640 1 -- 172.21.15.131:0/2671894102 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fdc00aa90 con 0x7f9fe415e340 2024-05-26T22:48:24.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:24.298+0000 7f9feac00640 1 -- 172.21.15.131:0/2671894102 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fe4155a40 con 0x7f9fe415e340 2024-05-26T22:48:24.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:24.299+0000 7f9feac00640 1 -- 172.21.15.131:0/2671894102 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fe40694e0 con 0x7f9fe415e340 2024-05-26T22:48:24.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:24.300+0000 7f9fe1600640 1 -- 172.21.15.131:0/2671894102 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9fdc00ac30 con 0x7f9fe415e340 2024-05-26T22:48:24.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:24.301+0000 7f9fe1600640 1 --2- 172.21.15.131:0/2671894102 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9fc406be30 0x7f9fc406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:24.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:24.301+0000 7f9fe1600640 1 -- 172.21.15.131:0/2671894102 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f9fdc08ad20 con 0x7f9fe415e340 2024-05-26T22:48:24.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:24.301+0000 7f9fe9800640 1 --2- 172.21.15.131:0/2671894102 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9fc406be30 0x7f9fc406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:24.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:24.301+0000 7f9fe9800640 1 --2- 172.21.15.131:0/2671894102 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9fc406be30 0x7f9fc406e2f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9fd8034000 tx=0x7f9fd802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:24.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:24.305+0000 7f9fe1600640 1 -- 172.21.15.131:0/2671894102 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f9fdc0596d0 con 0x7f9fe415e340 2024-05-26T22:48:24.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:24.548+0000 7f9feac00640 1 -- 172.21.15.131:0/2671894102 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f9fe40ca2b0 con 0x7f9fe415e340 2024-05-26T22:48:25.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.069+0000 7f9fe1600640 1 -- 172.21.15.131:0/2671894102 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v7) ==== 122+0+0 (secure 0 0 0) 0x7f9fdc058fd0 con 0x7f9fe415e340 2024-05-26T22:48:25.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.073+0000 7f9feac00640 1 -- 172.21.15.131:0/2671894102 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9fc406be30 msgr2=0x7f9fc406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:25.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.073+0000 7f9feac00640 1 --2- 172.21.15.131:0/2671894102 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9fc406be30 0x7f9fc406e2f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9fd8034000 tx=0x7f9fd802e000 comp rx=0 tx=0).stop 2024-05-26T22:48:25.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.074+0000 7f9feac00640 1 -- 172.21.15.131:0/2671894102 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9fe415e340 msgr2=0x7f9fe41f6a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:25.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.074+0000 7f9feac00640 1 --2- 172.21.15.131:0/2671894102 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9fe415e340 0x7f9fe41f6a80 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc00c730 tx=0x7f9fdc00cbf0 comp rx=0 tx=0).stop 2024-05-26T22:48:25.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.074+0000 7f9feac00640 1 -- 172.21.15.131:0/2671894102 shutdown_connections 2024-05-26T22:48:25.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.074+0000 7f9feac00640 1 --2- 172.21.15.131:0/2671894102 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9fc406be30 0x7f9fc406e2f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:25.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.074+0000 7f9feac00640 1 --2- 172.21.15.131:0/2671894102 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9fe415e340 0x7f9fe41f6a80 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:25.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.074+0000 7f9feac00640 1 --2- 172.21.15.131:0/2671894102 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9fe40d27f0 0x7f9fe41533c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:25.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.074+0000 7f9feac00640 1 --2- 172.21.15.131:0/2671894102 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9fe40b7820 0x7f9fe4156240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:25.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.074+0000 7f9feac00640 1 -- 172.21.15.131:0/2671894102 >> 172.21.15.131:0/2671894102 conn(0x7f9fe4072e60 msgr2=0x7f9fe40c7d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:25.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.074+0000 7f9feac00640 1 -- 172.21.15.131:0/2671894102 shutdown_connections 2024-05-26T22:48:25.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.074+0000 7f9feac00640 1 -- 172.21.15.131:0/2671894102 wait complete. 2024-05-26T22:48:25.096 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-26T22:48:25.096 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-26T22:48:25.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.303+0000 7f78df800640 1 Processor -- start 2024-05-26T22:48:25.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.303+0000 7f78df800640 1 -- start start 2024-05-26T22:48:25.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.304+0000 7f78df800640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f78d806ee20 0x7f78d80b3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:25.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.304+0000 7f78df800640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f78d81569d0 0x7f78d80af790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:25.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.304+0000 7f78df800640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78d815cb20 con 0x7f78d80ad250 2024-05-26T22:48:25.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.304+0000 7f78df800640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f78d815ce00 con 0x7f78d81569d0 2024-05-26T22:48:25.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.304+0000 7f78df800640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f78d815d0e0 con 0x7f78d806ee20 2024-05-26T22:48:25.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.305+0000 7f78dd800640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f78d81569d0 0x7f78d80af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:25.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.305+0000 7f78dd800640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f78d81569d0 0x7f78d80af790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53848/0 (socket says 172.21.15.131:53848) 2024-05-26T22:48:25.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.305+0000 7f78dd800640 1 -- 172.21.15.131:0/4083274492 learned_addr learned my addr 172.21.15.131:0/4083274492 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:25.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.305+0000 7f78dce00640 1 --2- 172.21.15.131:0/4083274492 >> v2:172.21.15.187:3300/0 conn(0x7f78d806ee20 0x7f78d80b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:25.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.305+0000 7f78dd800640 1 -- 172.21.15.131:0/4083274492 >> v2:172.21.15.187:3300/0 conn(0x7f78d806ee20 msgr2=0x7f78d80b3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:25.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.305+0000 7f78dd800640 1 --2- 172.21.15.131:0/4083274492 >> v2:172.21.15.187:3300/0 conn(0x7f78d806ee20 0x7f78d80b3190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:25.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.305+0000 7f78dd800640 1 -- 172.21.15.131:0/4083274492 >> v1:172.21.15.187:6789/0 conn(0x7f78d80ad250 legacy=0x7f78d81562c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:25.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.305+0000 7f78dd800640 1 -- 172.21.15.131:0/4083274492 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78d815d3c0 con 0x7f78d81569d0 2024-05-26T22:48:25.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.306+0000 7f78dd800640 1 --2- 172.21.15.131:0/4083274492 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f78d81569d0 0x7f78d80af790 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f78d8071090 tx=0x7f78d002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a26ff407be3f147a server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:25.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.307+0000 7f78d7400640 1 -- 172.21.15.131:0/4083274492 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d0037070 con 0x7f78d81569d0 2024-05-26T22:48:25.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.307+0000 7f78d7400640 1 -- 172.21.15.131:0/4083274492 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78d0032440 con 0x7f78d81569d0 2024-05-26T22:48:25.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.307+0000 7f78d7400640 1 -- 172.21.15.131:0/4083274492 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d0032810 con 0x7f78d81569d0 2024-05-26T22:48:25.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.307+0000 7f78df800640 1 -- 172.21.15.131:0/4083274492 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f78d81569d0 msgr2=0x7f78d80af790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:25.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.307+0000 7f78df800640 1 --2- 172.21.15.131:0/4083274492 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f78d81569d0 0x7f78d80af790 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f78d8071090 tx=0x7f78d002a1a0 comp rx=0 tx=0).stop 2024-05-26T22:48:25.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.307+0000 7f78df800640 1 -- 172.21.15.131:0/4083274492 shutdown_connections 2024-05-26T22:48:25.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.307+0000 7f78df800640 1 --2- 172.21.15.131:0/4083274492 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f78d81569d0 0x7f78d80af790 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:25.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.307+0000 7f78df800640 1 --2- 172.21.15.131:0/4083274492 >> v2:172.21.15.187:3300/0 conn(0x7f78d806ee20 0x7f78d80b3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:25.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.307+0000 7f78df800640 1 -- 172.21.15.131:0/4083274492 >> 172.21.15.131:0/4083274492 conn(0x7f78d80b1e90 msgr2=0x7f78d80b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:25.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.308+0000 7f78df800640 1 -- 172.21.15.131:0/4083274492 shutdown_connections 2024-05-26T22:48:25.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.308+0000 7f78df800640 1 -- 172.21.15.131:0/4083274492 wait complete. 2024-05-26T22:48:25.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.309+0000 7f78df800640 1 Processor -- start 2024-05-26T22:48:25.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.309+0000 7f78df800640 1 -- start start 2024-05-26T22:48:25.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.310+0000 7f78df800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f78d806ee20 0x7f78d81f2280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:25.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.310+0000 7f78df800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f78d80ad250 0x7f78d8151c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:25.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.310+0000 7f78df800640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f78d81569d0 0x7f78d809c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:25.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.310+0000 7f78df800640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f78d815c900 con 0x7f78d80ad250 2024-05-26T22:48:25.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.310+0000 7f78df800640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f78d815c780 con 0x7f78d81569d0 2024-05-26T22:48:25.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.310+0000 7f78df800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f78d806f200 con 0x7f78d806ee20 2024-05-26T22:48:25.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.311+0000 7f78d7e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f78d80ad250 0x7f78d8151c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:25.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.311+0000 7f78d7e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f78d80ad250 0x7f78d8151c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53852/0 (socket says 172.21.15.131:53852) 2024-05-26T22:48:25.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.311+0000 7f78d7e00640 1 -- 172.21.15.131:0/1823607408 learned_addr learned my addr 172.21.15.131:0/1823607408 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:25.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.311+0000 7f78dce00640 1 --2- 172.21.15.131:0/1823607408 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f78d806ee20 0x7f78d81f2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:25.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.311+0000 7f78d7e00640 1 -- 172.21.15.131:0/1823607408 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f78d806ee20 msgr2=0x7f78d81f2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:25.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.311+0000 7f78d7e00640 1 --2- 172.21.15.131:0/1823607408 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f78d806ee20 0x7f78d81f2280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:25.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.311+0000 7f78d7e00640 1 -- 172.21.15.131:0/1823607408 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f78d81569d0 msgr2=0x7f78d809c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:25.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.311+0000 7f78dd800640 1 --2- 172.21.15.131:0/1823607408 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f78d81569d0 0x7f78d809c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:25.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.311+0000 7f78d7e00640 1 --2- 172.21.15.131:0/1823607408 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f78d81569d0 0x7f78d809c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:25.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.311+0000 7f78d7e00640 1 -- 172.21.15.131:0/1823607408 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78d81e5100 con 0x7f78d80ad250 2024-05-26T22:48:25.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.311+0000 7f78dce00640 1 --2- 172.21.15.131:0/1823607408 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f78d806ee20 0x7f78d81f2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:48:25.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.311+0000 7f78d7e00640 1 --2- 172.21.15.131:0/1823607408 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f78d80ad250 0x7f78d8151c90 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f78c400c430 tx=0x7f78c400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:25.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.312+0000 7f78d4c00640 1 -- 172.21.15.131:0/1823607408 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78c4017070 con 0x7f78d80ad250 2024-05-26T22:48:25.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.312+0000 7f78d4c00640 1 -- 172.21.15.131:0/1823607408 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78c40035f0 con 0x7f78d80ad250 2024-05-26T22:48:25.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.312+0000 7f78df800640 1 -- 172.21.15.131:0/1823607408 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78d81e53e0 con 0x7f78d80ad250 2024-05-26T22:48:25.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.312+0000 7f78d4c00640 1 -- 172.21.15.131:0/1823607408 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78c40039e0 con 0x7f78d80ad250 2024-05-26T22:48:25.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.312+0000 7f78df800640 1 -- 172.21.15.131:0/1823607408 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78d81e5c80 con 0x7f78d80ad250 2024-05-26T22:48:25.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.313+0000 7f78df800640 1 -- 172.21.15.131:0/1823607408 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78d81e55b0 con 0x7f78d80ad250 2024-05-26T22:48:25.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.314+0000 7f78d4c00640 1 -- 172.21.15.131:0/1823607408 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f78c401a070 con 0x7f78d80ad250 2024-05-26T22:48:25.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.315+0000 7f78d4c00640 1 --2- 172.21.15.131:0/1823607408 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f78b806be30 0x7f78b806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:25.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.315+0000 7f78d4c00640 1 -- 172.21.15.131:0/1823607408 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f78c408a140 con 0x7f78d80ad250 2024-05-26T22:48:25.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.315+0000 7f78dce00640 1 --2- 172.21.15.131:0/1823607408 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f78b806be30 0x7f78b806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:25.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.316+0000 7f78dce00640 1 --2- 172.21.15.131:0/1823607408 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f78b806be30 0x7f78b806e2f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f78cc007f50 tx=0x7f78cc0074a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:25.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.319+0000 7f78d4c00640 1 -- 172.21.15.131:0/1823607408 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f78c40010c0 con 0x7f78d80ad250 2024-05-26T22:48:25.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.548+0000 7f78df800640 1 -- 172.21.15.131:0/1823607408 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f78d80694e0 con 0x7f78d80ad250 2024-05-26T22:48:25.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.550+0000 7f78d4c00640 1 -- 172.21.15.131:0/1823607408 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5184 (secure 0 0 0) 0x7f78c40584a0 con 0x7f78d80ad250 2024-05-26T22:48:25.550 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:48:25.550 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/3266175650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6833","nonce":3266175650},{"type":"v1","addr":"172.21.15.187:6835","nonce":3266175650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/1862866305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1862866305},{"type":"v1","addr":"172.21.15.131:6835","nonce":1862866305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4269,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/618294324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":618294324},{"type":"v1","addr":"172.21.15.131:6837","nonce":618294324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-26T22:48:21.211047+0000","modified":"2024-05-26T22:48:24.550610+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6834/2547512415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":2547512415},{"type":"v1","addr":"172.21.15.187:6834","nonce":2547512415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4281,"qdb_cluster":[4281]},"id":1}]} 2024-05-26T22:48:25.550 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 7 2024-05-26T22:48:25.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.552+0000 7f78df800640 1 -- 172.21.15.131:0/1823607408 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f78b806be30 msgr2=0x7f78b806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:25.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.552+0000 7f78df800640 1 --2- 172.21.15.131:0/1823607408 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f78b806be30 0x7f78b806e2f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f78cc007f50 tx=0x7f78cc0074a0 comp rx=0 tx=0).stop 2024-05-26T22:48:25.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.552+0000 7f78df800640 1 -- 172.21.15.131:0/1823607408 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f78d80ad250 msgr2=0x7f78d8151c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:25.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.552+0000 7f78df800640 1 --2- 172.21.15.131:0/1823607408 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f78d80ad250 0x7f78d8151c90 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f78c400c430 tx=0x7f78c400c8f0 comp rx=0 tx=0).stop 2024-05-26T22:48:25.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.552+0000 7f78df800640 1 -- 172.21.15.131:0/1823607408 shutdown_connections 2024-05-26T22:48:25.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.552+0000 7f78df800640 1 --2- 172.21.15.131:0/1823607408 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f78b806be30 0x7f78b806e2f0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:25.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.552+0000 7f78df800640 1 --2- 172.21.15.131:0/1823607408 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f78d81569d0 0x7f78d809c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:25.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.553+0000 7f78df800640 1 --2- 172.21.15.131:0/1823607408 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f78d80ad250 0x7f78d8151c90 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:25.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.553+0000 7f78df800640 1 --2- 172.21.15.131:0/1823607408 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f78d806ee20 0x7f78d81f2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:25.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.553+0000 7f78df800640 1 -- 172.21.15.131:0/1823607408 >> 172.21.15.131:0/1823607408 conn(0x7f78d80b1e90 msgr2=0x7f78d806d470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:25.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.553+0000 7f78df800640 1 -- 172.21.15.131:0/1823607408 shutdown_connections 2024-05-26T22:48:25.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.553+0000 7f78df800640 1 -- 172.21.15.131:0/1823607408 wait complete. 2024-05-26T22:48:25.570 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-26T22:48:25.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.762+0000 7f42c5800640 1 Processor -- start 2024-05-26T22:48:25.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.762+0000 7f42c5800640 1 -- start start 2024-05-26T22:48:25.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.764+0000 7f42c5800640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f42c0144340 0x7f42c00cedf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:25.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.764+0000 7f42c5800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f42c0071b70 0x7f42c006e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:25.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.764+0000 7f42c5800640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f42c006e9f0 0x7f42c015ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:25.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.764+0000 7f42c5800640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f42c006ee60 con 0x7f42c006e9f0 2024-05-26T22:48:25.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.764+0000 7f42c5800640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f42c0072eb0 con 0x7f42c0144340 2024-05-26T22:48:25.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.764+0000 7f42c5800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f42c0071fd0 con 0x7f42c0071b70 2024-05-26T22:48:25.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.764+0000 7f42c4e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f42c006e9f0 0x7f42c015ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:25.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.764+0000 7f42c4e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f42c006e9f0 0x7f42c015ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53866/0 (socket says 172.21.15.131:53866) 2024-05-26T22:48:25.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.764+0000 7f42c4e00640 1 -- 172.21.15.131:0/2016059238 learned_addr learned my addr 172.21.15.131:0/2016059238 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:25.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.764+0000 7f42bfe00640 1 --2- 172.21.15.131:0/2016059238 >> v2:172.21.15.187:3300/0 conn(0x7f42c0144340 0x7f42c00cedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:25.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.764+0000 7f42b7400640 1 --2- 172.21.15.131:0/2016059238 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f42c0071b70 0x7f42c006e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:25.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.765+0000 7f42c4e00640 1 -- 172.21.15.131:0/2016059238 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f42c0071b70 msgr2=0x7f42c006e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:25.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.765+0000 7f42c4e00640 1 --2- 172.21.15.131:0/2016059238 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f42c0071b70 0x7f42c006e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:25.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.765+0000 7f42c4e00640 1 -- 172.21.15.131:0/2016059238 >> v2:172.21.15.187:3300/0 conn(0x7f42c0144340 msgr2=0x7f42c00cedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:25.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.765+0000 7f42c4e00640 1 --2- 172.21.15.131:0/2016059238 >> v2:172.21.15.187:3300/0 conn(0x7f42c0144340 0x7f42c00cedf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:25.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.765+0000 7f42c4e00640 1 -- 172.21.15.131:0/2016059238 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42c00722a0 con 0x7f42c006e9f0 2024-05-26T22:48:25.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.766+0000 7f42c4e00640 1 --2- 172.21.15.131:0/2016059238 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f42c006e9f0 0x7f42c015ec30 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f42c00b2bd0 tx=0x7f42b802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e16d94213c7d6de8 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:25.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.766+0000 7f42bf400640 1 -- 172.21.15.131:0/2016059238 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b8037070 con 0x7f42c006e9f0 2024-05-26T22:48:25.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.766+0000 7f42bf400640 1 -- 172.21.15.131:0/2016059238 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42b8032440 con 0x7f42c006e9f0 2024-05-26T22:48:25.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.766+0000 7f42bf400640 1 -- 172.21.15.131:0/2016059238 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b8032810 con 0x7f42c006e9f0 2024-05-26T22:48:25.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.766+0000 7f42c5800640 1 -- 172.21.15.131:0/2016059238 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f42c006e9f0 msgr2=0x7f42c015ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:25.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.766+0000 7f42c5800640 1 --2- 172.21.15.131:0/2016059238 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f42c006e9f0 0x7f42c015ec30 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f42c00b2bd0 tx=0x7f42b802a1e0 comp rx=0 tx=0).stop 2024-05-26T22:48:25.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.767+0000 7f42c5800640 1 -- 172.21.15.131:0/2016059238 shutdown_connections 2024-05-26T22:48:25.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.767+0000 7f42c5800640 1 --2- 172.21.15.131:0/2016059238 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f42c006e9f0 0x7f42c015ec30 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:25.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.767+0000 7f42c5800640 1 --2- 172.21.15.131:0/2016059238 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f42c0071b70 0x7f42c006e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:25.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.767+0000 7f42c5800640 1 --2- 172.21.15.131:0/2016059238 >> v2:172.21.15.187:3300/0 conn(0x7f42c0144340 0x7f42c00cedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:25.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.767+0000 7f42c5800640 1 -- 172.21.15.131:0/2016059238 >> 172.21.15.131:0/2016059238 conn(0x7f42c00cff40 msgr2=0x7f42c00d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:25.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.767+0000 7f42c5800640 1 -- 172.21.15.131:0/2016059238 shutdown_connections 2024-05-26T22:48:25.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.767+0000 7f42c5800640 1 -- 172.21.15.131:0/2016059238 wait complete. 2024-05-26T22:48:25.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.768+0000 7f42c5800640 1 Processor -- start 2024-05-26T22:48:25.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.768+0000 7f42c5800640 1 -- start start 2024-05-26T22:48:25.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.769+0000 7f42c5800640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f42c006e9f0 0x7f42c00b3f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:25.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.769+0000 7f42c5800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f42c0071b70 0x7f42c01532e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:25.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.769+0000 7f42c5800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f42c0144340 0x7f42c0153820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:25.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.769+0000 7f42c5800640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f42c0072b40 con 0x7f42c0144340 2024-05-26T22:48:25.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.769+0000 7f42c5800640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f42c0072860 con 0x7f42c006e9f0 2024-05-26T22:48:25.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.769+0000 7f42c5800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f42c015f170 con 0x7f42c0071b70 2024-05-26T22:48:25.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.769+0000 7f42c4e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f42c0144340 0x7f42c0153820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:25.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.769+0000 7f42c4e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f42c0144340 0x7f42c0153820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53878/0 (socket says 172.21.15.131:53878) 2024-05-26T22:48:25.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.769+0000 7f42c4e00640 1 -- 172.21.15.131:0/2922447356 learned_addr learned my addr 172.21.15.131:0/2922447356 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:25.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.770+0000 7f42c4e00640 1 -- 172.21.15.131:0/2922447356 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f42c0071b70 msgr2=0x7f42c01532e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:25.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.770+0000 7f42c4e00640 1 --2- 172.21.15.131:0/2922447356 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f42c0071b70 0x7f42c01532e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:25.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.770+0000 7f42c4e00640 1 -- 172.21.15.131:0/2922447356 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f42c006e9f0 msgr2=0x7f42c00b3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:25.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.770+0000 7f42bfe00640 1 --2- 172.21.15.131:0/2922447356 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f42c006e9f0 0x7f42c00b3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:25.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.770+0000 7f42c4e00640 1 --2- 172.21.15.131:0/2922447356 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f42c006e9f0 0x7f42c00b3f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:25.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.770+0000 7f42c4e00640 1 -- 172.21.15.131:0/2922447356 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42c00adbe0 con 0x7f42c0144340 2024-05-26T22:48:25.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.770+0000 7f42c4e00640 1 --2- 172.21.15.131:0/2922447356 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f42c0144340 0x7f42c0153820 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f42b80090b0 tx=0x7f42b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:25.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.770+0000 7f42bcc00640 1 -- 172.21.15.131:0/2922447356 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b8037040 con 0x7f42c0144340 2024-05-26T22:48:25.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.770+0000 7f42bcc00640 1 -- 172.21.15.131:0/2922447356 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42b8042070 con 0x7f42c0144340 2024-05-26T22:48:25.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.770+0000 7f42c5800640 1 -- 172.21.15.131:0/2922447356 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42c00adec0 con 0x7f42c0144340 2024-05-26T22:48:25.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.770+0000 7f42bcc00640 1 -- 172.21.15.131:0/2922447356 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b8030920 con 0x7f42c0144340 2024-05-26T22:48:25.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.771+0000 7f42c5800640 1 -- 172.21.15.131:0/2922447356 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42c00ae480 con 0x7f42c0144340 2024-05-26T22:48:25.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.771+0000 7f42c5800640 1 -- 172.21.15.131:0/2922447356 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42c00694e0 con 0x7f42c0144340 2024-05-26T22:48:25.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.772+0000 7f42bcc00640 1 -- 172.21.15.131:0/2922447356 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f42b8030ac0 con 0x7f42c0144340 2024-05-26T22:48:25.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.773+0000 7f42bcc00640 1 --2- 172.21.15.131:0/2922447356 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f429c06be30 0x7f429c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:25.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.773+0000 7f42bcc00640 1 -- 172.21.15.131:0/2922447356 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f42b80ab820 con 0x7f42c0144340 2024-05-26T22:48:25.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.773+0000 7f42bfe00640 1 --2- 172.21.15.131:0/2922447356 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f429c06be30 0x7f429c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:25.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.774+0000 7f42bfe00640 1 --2- 172.21.15.131:0/2922447356 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f429c06be30 0x7f429c06e2f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f42b0006fd0 tx=0x7f42b0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:25.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.778+0000 7f42bcc00640 1 -- 172.21.15.131:0/2922447356 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f42b8079b70 con 0x7f42c0144340 2024-05-26T22:48:25.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.979+0000 7f42c5800640 1 -- 172.21.15.131:0/2922447356 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f42c00ce720 con 0x7f42c0144340 2024-05-26T22:48:25.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.981+0000 7f42bcc00640 1 -- 172.21.15.131:0/2922447356 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16629 (secure 0 0 0) 0x7f42b807de70 con 0x7f42c0144340 2024-05-26T22:48:25.982 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:48:25.982 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":20,"fsid":"46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f","created":"2024-05-26T22:47:57.288905+0000","modified":"2024-05-26T22:48:22.799298+0000","last_up_change":"2024-05-26T22:48:09.602207+0000","last_in_change":"2024-05-26T22:48:00.687341+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-26T22:48:10.463339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-26T22:48:19.663254+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-26T22:48:20.235751+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"814eaf62-a18d-494b-8e27-fd8fb88adea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6801","nonce":2350940051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6803","nonce":2350940051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6807","nonce":2350940051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6805","nonce":2350940051}]},"public_addr":"172.21.15.131:6801/2350940051","cluster_addr":"172.21.15.131:6803/2350940051","heartbeat_back_addr":"172.21.15.131:6807/2350940051","heartbeat_front_addr":"172.21.15.131:6805/2350940051","state":["exists","up"]},{"osd":1,"uuid":"42e66287-f6c6-4e26-af3c-6dfaa3536d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6825","nonce":1525528247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6827","nonce":1525528247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6831","nonce":1525528247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6829","nonce":1525528247}]},"public_addr":"172.21.15.131:6825/1525528247","cluster_addr":"172.21.15.131:6827/1525528247","heartbeat_back_addr":"172.21.15.131:6831/1525528247","heartbeat_front_addr":"172.21.15.131:6829/1525528247","state":["exists","up"]},{"osd":2,"uuid":"0da34673-04a2-4fd0-b57a-0ff2b679b043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6817","nonce":3627842210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6819","nonce":3627842210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6823","nonce":3627842210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6821","nonce":3627842210}]},"public_addr":"172.21.15.131:6817/3627842210","cluster_addr":"172.21.15.131:6819/3627842210","heartbeat_back_addr":"172.21.15.131:6823/3627842210","heartbeat_front_addr":"172.21.15.131:6821/3627842210","state":["exists","up"]},{"osd":3,"uuid":"cf117a27-d6b2-43d3-ae52-b47988bf8eed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6809","nonce":350790591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6811","nonce":350790591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6815","nonce":350790591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6813","nonce":350790591}]},"public_addr":"172.21.15.131:6809/350790591","cluster_addr":"172.21.15.131:6811/350790591","heartbeat_back_addr":"172.21.15.131:6815/350790591","heartbeat_front_addr":"172.21.15.131:6813/350790591","state":["exists","up"]},{"osd":4,"uuid":"a092ccfc-be89-4a84-88d5-0f781fa715eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6816","nonce":244560995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6819","nonce":244560995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6823","nonce":244560995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6821","nonce":244560995}]},"public_addr":"172.21.15.187:6816/244560995","cluster_addr":"172.21.15.187:6819/244560995","heartbeat_back_addr":"172.21.15.187:6823/244560995","heartbeat_front_addr":"172.21.15.187:6821/244560995","state":["exists","up"]},{"osd":5,"uuid":"7513ba69-dd00-4a35-b190-9e8cc3b2bdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6801","nonce":324619398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6803","nonce":324619398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6812","nonce":324619398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6807","nonce":324619398}]},"public_addr":"172.21.15.187:6801/324619398","cluster_addr":"172.21.15.187:6803/324619398","heartbeat_back_addr":"172.21.15.187:6812/324619398","heartbeat_front_addr":"172.21.15.187:6807/324619398","state":["exists","up"]},{"osd":6,"uuid":"50081810-c093-4d2f-a4d7-b7c695c5400d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6806","nonce":1391347786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6809","nonce":1391347786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6818","nonce":1391347786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6813","nonce":1391347786}]},"public_addr":"172.21.15.187:6806/1391347786","cluster_addr":"172.21.15.187:6809/1391347786","heartbeat_back_addr":"172.21.15.187:6818/1391347786","heartbeat_front_addr":"172.21.15.187:6813/1391347786","state":["exists","up"]},{"osd":7,"uuid":"775dae05-54f5-4ad4-bbce-92863c6ed2ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6825","nonce":1220065622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6827","nonce":1220065622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6831","nonce":1220065622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6829","nonce":1220065622}]},"public_addr":"172.21.15.187:6825/1220065622","cluster_addr":"172.21.15.187:6827/1220065622","heartbeat_back_addr":"172.21.15.187:6831/1220065622","heartbeat_front_addr":"172.21.15.187:6829/1220065622","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:05.983520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.299516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.067323+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.307748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.018742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.258775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.010912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.295355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-26T22:48:25.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.984+0000 7f42c5800640 1 -- 172.21.15.131:0/2922447356 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f429c06be30 msgr2=0x7f429c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:25.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.984+0000 7f42c5800640 1 --2- 172.21.15.131:0/2922447356 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f429c06be30 0x7f429c06e2f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f42b0006fd0 tx=0x7f42b0008040 comp rx=0 tx=0).stop 2024-05-26T22:48:25.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.984+0000 7f42c5800640 1 -- 172.21.15.131:0/2922447356 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f42c0144340 msgr2=0x7f42c0153820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:25.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.984+0000 7f42c5800640 1 --2- 172.21.15.131:0/2922447356 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f42c0144340 0x7f42c0153820 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f42b80090b0 tx=0x7f42b80046c0 comp rx=0 tx=0).stop 2024-05-26T22:48:25.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.985+0000 7f42c5800640 1 -- 172.21.15.131:0/2922447356 shutdown_connections 2024-05-26T22:48:25.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.985+0000 7f42c5800640 1 --2- 172.21.15.131:0/2922447356 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f42c0071b70 0x7f42c01532e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:25.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.985+0000 7f42c5800640 1 --2- 172.21.15.131:0/2922447356 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f429c06be30 0x7f429c06e2f0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:25.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.985+0000 7f42c5800640 1 --2- 172.21.15.131:0/2922447356 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f42c0144340 0x7f42c0153820 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:25.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.985+0000 7f42c5800640 1 --2- 172.21.15.131:0/2922447356 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f42c006e9f0 0x7f42c00b3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:25.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.985+0000 7f42c5800640 1 -- 172.21.15.131:0/2922447356 >> 172.21.15.131:0/2922447356 conn(0x7f42c00cff40 msgr2=0x7f42c006d3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:25.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.985+0000 7f42c5800640 1 -- 172.21.15.131:0/2922447356 shutdown_connections 2024-05-26T22:48:25.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:25.985+0000 7f42c5800640 1 -- 172.21.15.131:0/2922447356 wait complete. 2024-05-26T22:48:26.004 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-26T22:48:26.004 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-26T22:48:26.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.197+0000 7f9e3e600640 1 Processor -- start 2024-05-26T22:48:26.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.197+0000 7f9e3e600640 1 -- start start 2024-05-26T22:48:26.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.198+0000 7f9e3e600640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f9e3806ee20 0x7f9e380b3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:26.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.198+0000 7f9e3e600640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9e380ad250 0x7f9e380af790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:26.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.198+0000 7f9e3e600640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e3815cb20 con 0x7f9e3806f200 2024-05-26T22:48:26.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.198+0000 7f9e3e600640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f9e3815ce00 con 0x7f9e3806ee20 2024-05-26T22:48:26.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.198+0000 7f9e3e600640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9e3815d0e0 con 0x7f9e380ad250 2024-05-26T22:48:26.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.199+0000 7f9e37400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f9e3806ee20 0x7f9e380b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:26.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.199+0000 7f9e37e00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f9e3806f200 0x7f9e381562c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:48816/0 (socket says 172.21.15.131:48816) 2024-05-26T22:48:26.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.199+0000 7f9e37e00640 1 -- 172.21.15.131:0/1565009555 learned_addr learned my addr 172.21.15.131:0/1565009555 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:48:26.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.199+0000 7f9e36a00640 1 --2- 172.21.15.131:0/1565009555 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9e380ad250 0x7f9e380af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:26.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.199+0000 7f9e37400640 1 -- 172.21.15.131:0/1565009555 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9e380ad250 msgr2=0x7f9e380af790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:26.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.199+0000 7f9e37400640 1 --2- 172.21.15.131:0/1565009555 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9e380ad250 0x7f9e380af790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:26.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.199+0000 7f9e37400640 1 -- 172.21.15.131:0/1565009555 >> v1:172.21.15.131:6789/0 conn(0x7f9e3806f200 legacy=0x7f9e381562c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:26.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.199+0000 7f9e37400640 1 -- 172.21.15.131:0/1565009555 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e3815d3c0 con 0x7f9e3806ee20 2024-05-26T22:48:26.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.199+0000 7f9e36000640 1 -- 172.21.15.131:0/1565009555 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2111453168 0 0) 0x7f9e3815cb20 con 0x7f9e3806f200 2024-05-26T22:48:26.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.200+0000 7f9e37400640 1 --2- 172.21.15.131:0/1565009555 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9e3806ee20 0x7f9e380b3190 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9e280097a0 tx=0x7f9e2802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56e97cd61a60f79b server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:26.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.201+0000 7f9e36000640 1 -- 172.21.15.131:0/1565009555 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e28037070 con 0x7f9e3806ee20 2024-05-26T22:48:26.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.201+0000 7f9e36000640 1 -- 172.21.15.131:0/1565009555 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e28032440 con 0x7f9e3806ee20 2024-05-26T22:48:26.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.201+0000 7f9e36000640 1 -- 172.21.15.131:0/1565009555 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e28032810 con 0x7f9e3806ee20 2024-05-26T22:48:26.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.201+0000 7f9e3e600640 1 -- 172.21.15.131:0/1565009555 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9e3806ee20 msgr2=0x7f9e380b3190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:26.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.201+0000 7f9e3e600640 1 --2- 172.21.15.131:0/1565009555 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9e3806ee20 0x7f9e380b3190 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9e280097a0 tx=0x7f9e2802a180 comp rx=0 tx=0).stop 2024-05-26T22:48:26.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.201+0000 7f9e3e600640 1 -- 172.21.15.131:0/1565009555 shutdown_connections 2024-05-26T22:48:26.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.201+0000 7f9e3e600640 1 --2- 172.21.15.131:0/1565009555 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9e380ad250 0x7f9e380af790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:26.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.201+0000 7f9e3e600640 1 --2- 172.21.15.131:0/1565009555 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9e3806ee20 0x7f9e380b3190 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:26.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.201+0000 7f9e3e600640 1 -- 172.21.15.131:0/1565009555 >> 172.21.15.131:0/1565009555 conn(0x7f9e380b1e90 msgr2=0x7f9e380b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:26.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.201+0000 7f9e3e600640 1 -- 172.21.15.131:0/1565009555 shutdown_connections 2024-05-26T22:48:26.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.201+0000 7f9e3e600640 1 -- 172.21.15.131:0/1565009555 wait complete. 2024-05-26T22:48:26.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.203+0000 7f9e3e600640 1 Processor -- start 2024-05-26T22:48:26.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.203+0000 7f9e3e600640 1 -- start start 2024-05-26T22:48:26.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.204+0000 7f9e3e600640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9e3806ee20 0x7f9e381ee3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:26.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.204+0000 7f9e3e600640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9e3806f200 0x7f9e3809dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:26.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.204+0000 7f9e3e600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9e380ad250 0x7f9e3814f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:26.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.204+0000 7f9e3e600640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f9e3815c900 con 0x7f9e380ad250 2024-05-26T22:48:26.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.204+0000 7f9e3e600640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9e3815c780 con 0x7f9e3806f200 2024-05-26T22:48:26.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.204+0000 7f9e3e600640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9e381569d0 con 0x7f9e3806ee20 2024-05-26T22:48:26.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.204+0000 7f9e37e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9e380ad250 0x7f9e3814f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:26.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.204+0000 7f9e37e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9e380ad250 0x7f9e3814f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53886/0 (socket says 172.21.15.131:53886) 2024-05-26T22:48:26.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.204+0000 7f9e37e00640 1 -- 172.21.15.131:0/3894087788 learned_addr learned my addr 172.21.15.131:0/3894087788 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:26.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.205+0000 7f9e36a00640 1 --2- 172.21.15.131:0/3894087788 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9e3806f200 0x7f9e3809dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:26.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.205+0000 7f9e37400640 1 --2- 172.21.15.131:0/3894087788 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9e3806ee20 0x7f9e381ee3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:26.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.205+0000 7f9e37e00640 1 -- 172.21.15.131:0/3894087788 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9e3806ee20 msgr2=0x7f9e381ee3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:26.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.205+0000 7f9e37e00640 1 --2- 172.21.15.131:0/3894087788 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9e3806ee20 0x7f9e381ee3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:26.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.205+0000 7f9e37e00640 1 -- 172.21.15.131:0/3894087788 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9e3806f200 msgr2=0x7f9e3809dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:26.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.205+0000 7f9e37e00640 1 --2- 172.21.15.131:0/3894087788 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9e3806f200 0x7f9e3809dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:26.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.205+0000 7f9e37e00640 1 -- 172.21.15.131:0/3894087788 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e381f4480 con 0x7f9e380ad250 2024-05-26T22:48:26.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.205+0000 7f9e36a00640 1 --2- 172.21.15.131:0/3894087788 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9e3806f200 0x7f9e3809dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:48:26.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.205+0000 7f9e37e00640 1 --2- 172.21.15.131:0/3894087788 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9e380ad250 0x7f9e3814f140 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f9e3000c7e0 tx=0x7f9e3000cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:26.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.205+0000 7f9e2f400640 1 -- 172.21.15.131:0/3894087788 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e30018070 con 0x7f9e380ad250 2024-05-26T22:48:26.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.205+0000 7f9e2f400640 1 -- 172.21.15.131:0/3894087788 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e3000a640 con 0x7f9e380ad250 2024-05-26T22:48:26.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.206+0000 7f9e3e600640 1 -- 172.21.15.131:0/3894087788 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e381f7ee0 con 0x7f9e380ad250 2024-05-26T22:48:26.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.206+0000 7f9e2f400640 1 -- 172.21.15.131:0/3894087788 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e3000aa10 con 0x7f9e380ad250 2024-05-26T22:48:26.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.206+0000 7f9e3e600640 1 -- 172.21.15.131:0/3894087788 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e381f84a0 con 0x7f9e380ad250 2024-05-26T22:48:26.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.207+0000 7f9e3e600640 1 -- 172.21.15.131:0/3894087788 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e380694e0 con 0x7f9e380ad250 2024-05-26T22:48:26.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.208+0000 7f9e2f400640 1 -- 172.21.15.131:0/3894087788 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9e3001b070 con 0x7f9e380ad250 2024-05-26T22:48:26.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.209+0000 7f9e2f400640 1 --2- 172.21.15.131:0/3894087788 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9e1806be30 0x7f9e1806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:26.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.209+0000 7f9e2f400640 1 -- 172.21.15.131:0/3894087788 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f9e3008a420 con 0x7f9e380ad250 2024-05-26T22:48:26.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.209+0000 7f9e37400640 1 --2- 172.21.15.131:0/3894087788 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9e1806be30 0x7f9e1806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:26.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.210+0000 7f9e37400640 1 --2- 172.21.15.131:0/3894087788 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9e1806be30 0x7f9e1806e2f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9e28034000 tx=0x7f9e2802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:26.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.213+0000 7f9e2f400640 1 -- 172.21.15.131:0/3894087788 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f9e381f4480 con 0x7f9e380ad250 2024-05-26T22:48:26.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.394+0000 7f9e3e600640 1 -- 172.21.15.131:0/3894087788 --> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9e3815bf50 con 0x7f9e1806be30 2024-05-26T22:48:26.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.418+0000 7f9e2f400640 1 -- 172.21.15.131:0/3894087788 <== mgr.4104 v2:172.21.15.131:6832/3484832947 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251947 (secure 0 0 0) 0x7f9e3815bf50 con 0x7f9e1806be30 2024-05-26T22:48:26.423 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:48:26.425 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-05-26T22:48:26.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.425+0000 7f9e3e600640 1 -- 172.21.15.131:0/3894087788 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9e1806be30 msgr2=0x7f9e1806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:26.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.425+0000 7f9e3e600640 1 --2- 172.21.15.131:0/3894087788 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9e1806be30 0x7f9e1806e2f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9e28034000 tx=0x7f9e2802e000 comp rx=0 tx=0).stop 2024-05-26T22:48:26.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.425+0000 7f9e3e600640 1 -- 172.21.15.131:0/3894087788 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9e380ad250 msgr2=0x7f9e3814f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:26.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.425+0000 7f9e3e600640 1 --2- 172.21.15.131:0/3894087788 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9e380ad250 0x7f9e3814f140 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f9e3000c7e0 tx=0x7f9e3000cca0 comp rx=0 tx=0).stop 2024-05-26T22:48:26.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.425+0000 7f9e3e600640 1 -- 172.21.15.131:0/3894087788 shutdown_connections 2024-05-26T22:48:26.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.425+0000 7f9e3e600640 1 --2- 172.21.15.131:0/3894087788 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9e1806be30 0x7f9e1806e2f0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:26.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.425+0000 7f9e3e600640 1 --2- 172.21.15.131:0/3894087788 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9e380ad250 0x7f9e3814f140 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:26.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.425+0000 7f9e3e600640 1 --2- 172.21.15.131:0/3894087788 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9e3806f200 0x7f9e3809dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:26.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.425+0000 7f9e3e600640 1 --2- 172.21.15.131:0/3894087788 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9e3806ee20 0x7f9e381ee3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:26.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.425+0000 7f9e3e600640 1 -- 172.21.15.131:0/3894087788 >> 172.21.15.131:0/3894087788 conn(0x7f9e380b1e90 msgr2=0x7f9e380aee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:26.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.425+0000 7f9e3e600640 1 -- 172.21.15.131:0/3894087788 shutdown_connections 2024-05-26T22:48:26.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.425+0000 7f9e3e600640 1 -- 172.21.15.131:0/3894087788 wait complete. 2024-05-26T22:48:26.442 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-05-26T22:48:26.384454+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":157,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218992,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755728,"statfs":{"total":773094113280,"available":772869865472,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":39,"apply_latency_ms":39,"commit_latency_ns":39000000,"apply_latency_ns":39000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002054"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.509427+0000","last_change":"2024-05-26T22:48:21.831991+0000","last_active":"2024-05-26T22:48:23.509427+0000","last_peered":"2024-05-26T22:48:23.509427+0000","last_clean":"2024-05-26T22:48:23.509427+0000","last_became_active":"2024-05-26T22:48:21.831657+0000","last_became_peered":"2024-05-26T22:48:21.831657+0000","last_unstale":"2024-05-26T22:48:23.509427+0000","last_undegraded":"2024-05-26T22:48:23.509427+0000","last_fullsized":"2024-05-26T22:48:23.509427+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:30:45.689344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.822991+0000","last_change":"2024-05-26T22:48:21.813275+0000","last_active":"2024-05-26T22:48:22.822991+0000","last_peered":"2024-05-26T22:48:22.822991+0000","last_clean":"2024-05-26T22:48:22.822991+0000","last_became_active":"2024-05-26T22:48:21.812895+0000","last_became_peered":"2024-05-26T22:48:21.812895+0000","last_unstale":"2024-05-26T22:48:22.822991+0000","last_undegraded":"2024-05-26T22:48:22.822991+0000","last_fullsized":"2024-05-26T22:48:22.822991+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:59:43.907681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.809083+0000","last_change":"2024-05-26T22:48:21.805790+0000","last_active":"2024-05-26T22:48:22.809083+0000","last_peered":"2024-05-26T22:48:22.809083+0000","last_clean":"2024-05-26T22:48:22.809083+0000","last_became_active":"2024-05-26T22:48:21.805318+0000","last_became_peered":"2024-05-26T22:48:21.805318+0000","last_unstale":"2024-05-26T22:48:22.809083+0000","last_undegraded":"2024-05-26T22:48:22.809083+0000","last_fullsized":"2024-05-26T22:48:22.809083+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:43:21.391197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.506425+0000","last_change":"2024-05-26T22:48:21.831837+0000","last_active":"2024-05-26T22:48:23.506425+0000","last_peered":"2024-05-26T22:48:23.506425+0000","last_clean":"2024-05-26T22:48:23.506425+0000","last_became_active":"2024-05-26T22:48:21.829461+0000","last_became_peered":"2024-05-26T22:48:21.829461+0000","last_unstale":"2024-05-26T22:48:23.506425+0000","last_undegraded":"2024-05-26T22:48:23.506425+0000","last_fullsized":"2024-05-26T22:48:23.506425+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:56:39.208963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.508007+0000","last_change":"2024-05-26T22:48:21.821926+0000","last_active":"2024-05-26T22:48:23.508007+0000","last_peered":"2024-05-26T22:48:23.508007+0000","last_clean":"2024-05-26T22:48:23.508007+0000","last_became_active":"2024-05-26T22:48:21.821594+0000","last_became_peered":"2024-05-26T22:48:21.821594+0000","last_unstale":"2024-05-26T22:48:23.508007+0000","last_undegraded":"2024-05-26T22:48:23.508007+0000","last_fullsized":"2024-05-26T22:48:23.508007+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:16:22.847663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.506340+0000","last_change":"2024-05-26T22:48:21.823954+0000","last_active":"2024-05-26T22:48:23.506340+0000","last_peered":"2024-05-26T22:48:23.506340+0000","last_clean":"2024-05-26T22:48:23.506340+0000","last_became_active":"2024-05-26T22:48:21.821405+0000","last_became_peered":"2024-05-26T22:48:21.821405+0000","last_unstale":"2024-05-26T22:48:23.506340+0000","last_undegraded":"2024-05-26T22:48:23.506340+0000","last_fullsized":"2024-05-26T22:48:23.506340+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:33:39.923721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.506938+0000","last_change":"2024-05-26T22:48:21.821415+0000","last_active":"2024-05-26T22:48:23.506938+0000","last_peered":"2024-05-26T22:48:23.506938+0000","last_clean":"2024-05-26T22:48:23.506938+0000","last_became_active":"2024-05-26T22:48:21.821092+0000","last_became_peered":"2024-05-26T22:48:21.821092+0000","last_unstale":"2024-05-26T22:48:23.506938+0000","last_undegraded":"2024-05-26T22:48:23.506938+0000","last_fullsized":"2024-05-26T22:48:23.506938+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:56:39.208963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.811969+0000","last_change":"2024-05-26T22:48:21.847352+0000","last_active":"2024-05-26T22:48:22.811969+0000","last_peered":"2024-05-26T22:48:22.811969+0000","last_clean":"2024-05-26T22:48:22.811969+0000","last_became_active":"2024-05-26T22:48:21.846965+0000","last_became_peered":"2024-05-26T22:48:21.846965+0000","last_unstale":"2024-05-26T22:48:22.811969+0000","last_undegraded":"2024-05-26T22:48:22.811969+0000","last_fullsized":"2024-05-26T22:48:22.811969+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T23:40:55.485829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.509545+0000","last_change":"2024-05-26T22:48:21.847421+0000","last_active":"2024-05-26T22:48:23.509545+0000","last_peered":"2024-05-26T22:48:23.509545+0000","last_clean":"2024-05-26T22:48:23.509545+0000","last_became_active":"2024-05-26T22:48:21.846702+0000","last_became_peered":"2024-05-26T22:48:21.846702+0000","last_unstale":"2024-05-26T22:48:23.509545+0000","last_undegraded":"2024-05-26T22:48:23.509545+0000","last_fullsized":"2024-05-26T22:48:23.509545+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:27:35.210564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.810240+0000","last_change":"2024-05-26T22:48:21.831244+0000","last_active":"2024-05-26T22:48:22.810240+0000","last_peered":"2024-05-26T22:48:22.810240+0000","last_clean":"2024-05-26T22:48:22.810240+0000","last_became_active":"2024-05-26T22:48:21.830949+0000","last_became_peered":"2024-05-26T22:48:21.830949+0000","last_unstale":"2024-05-26T22:48:22.810240+0000","last_undegraded":"2024-05-26T22:48:22.810240+0000","last_fullsized":"2024-05-26T22:48:22.810240+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:59:43.907681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.810220+0000","last_change":"2024-05-26T22:48:21.836749+0000","last_active":"2024-05-26T22:48:22.810220+0000","last_peered":"2024-05-26T22:48:22.810220+0000","last_clean":"2024-05-26T22:48:22.810220+0000","last_became_active":"2024-05-26T22:48:21.836431+0000","last_became_peered":"2024-05-26T22:48:21.836431+0000","last_unstale":"2024-05-26T22:48:22.810220+0000","last_undegraded":"2024-05-26T22:48:22.810220+0000","last_fullsized":"2024-05-26T22:48:22.810220+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:38:00.344955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.505172+0000","last_change":"2024-05-26T22:48:21.819214+0000","last_active":"2024-05-26T22:48:23.505172+0000","last_peered":"2024-05-26T22:48:23.505172+0000","last_clean":"2024-05-26T22:48:23.505172+0000","last_became_active":"2024-05-26T22:48:21.818837+0000","last_became_peered":"2024-05-26T22:48:21.818837+0000","last_unstale":"2024-05-26T22:48:23.505172+0000","last_undegraded":"2024-05-26T22:48:23.505172+0000","last_fullsized":"2024-05-26T22:48:23.505172+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:42:32.978709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.812261+0000","last_change":"2024-05-26T22:48:21.830442+0000","last_active":"2024-05-26T22:48:22.812261+0000","last_peered":"2024-05-26T22:48:22.812261+0000","last_clean":"2024-05-26T22:48:22.812261+0000","last_became_active":"2024-05-26T22:48:21.830132+0000","last_became_peered":"2024-05-26T22:48:21.830132+0000","last_unstale":"2024-05-26T22:48:22.812261+0000","last_undegraded":"2024-05-26T22:48:22.812261+0000","last_fullsized":"2024-05-26T22:48:22.812261+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:43:21.391197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.808630+0000","last_change":"2024-05-26T22:48:21.818032+0000","last_active":"2024-05-26T22:48:22.808630+0000","last_peered":"2024-05-26T22:48:22.808630+0000","last_clean":"2024-05-26T22:48:22.808630+0000","last_became_active":"2024-05-26T22:48:21.817717+0000","last_became_peered":"2024-05-26T22:48:21.817717+0000","last_unstale":"2024-05-26T22:48:22.808630+0000","last_undegraded":"2024-05-26T22:48:22.808630+0000","last_fullsized":"2024-05-26T22:48:22.808630+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:27:34.959431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.822672+0000","last_change":"2024-05-26T22:48:21.823839+0000","last_active":"2024-05-26T22:48:22.822672+0000","last_peered":"2024-05-26T22:48:22.822672+0000","last_clean":"2024-05-26T22:48:22.822672+0000","last_became_active":"2024-05-26T22:48:21.823522+0000","last_became_peered":"2024-05-26T22:48:21.823522+0000","last_unstale":"2024-05-26T22:48:22.822672+0000","last_undegraded":"2024-05-26T22:48:22.822672+0000","last_fullsized":"2024-05-26T22:48:22.822672+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:52:25.073281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.504208+0000","last_change":"2024-05-26T22:48:21.819432+0000","last_active":"2024-05-26T22:48:23.504208+0000","last_peered":"2024-05-26T22:48:23.504208+0000","last_clean":"2024-05-26T22:48:23.504208+0000","last_became_active":"2024-05-26T22:48:21.819139+0000","last_became_peered":"2024-05-26T22:48:21.819139+0000","last_unstale":"2024-05-26T22:48:23.504208+0000","last_undegraded":"2024-05-26T22:48:23.504208+0000","last_fullsized":"2024-05-26T22:48:23.504208+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:21:52.108732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.507463+0000","last_change":"2024-05-26T22:48:21.842657+0000","last_active":"2024-05-26T22:48:23.507463+0000","last_peered":"2024-05-26T22:48:23.507463+0000","last_clean":"2024-05-26T22:48:23.507463+0000","last_became_active":"2024-05-26T22:48:21.842312+0000","last_became_peered":"2024-05-26T22:48:21.842312+0000","last_unstale":"2024-05-26T22:48:23.507463+0000","last_undegraded":"2024-05-26T22:48:23.507463+0000","last_fullsized":"2024-05-26T22:48:23.507463+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:49:40.791149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.503319+0000","last_change":"2024-05-26T22:48:21.812988+0000","last_active":"2024-05-26T22:48:23.503319+0000","last_peered":"2024-05-26T22:48:23.503319+0000","last_clean":"2024-05-26T22:48:23.503319+0000","last_became_active":"2024-05-26T22:48:21.812669+0000","last_became_peered":"2024-05-26T22:48:21.812669+0000","last_unstale":"2024-05-26T22:48:23.503319+0000","last_undegraded":"2024-05-26T22:48:23.503319+0000","last_fullsized":"2024-05-26T22:48:23.503319+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:16:02.119875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.509267+0000","last_change":"2024-05-26T22:48:21.830493+0000","last_active":"2024-05-26T22:48:23.509267+0000","last_peered":"2024-05-26T22:48:23.509267+0000","last_clean":"2024-05-26T22:48:23.509267+0000","last_became_active":"2024-05-26T22:48:21.830116+0000","last_became_peered":"2024-05-26T22:48:21.830116+0000","last_unstale":"2024-05-26T22:48:23.509267+0000","last_undegraded":"2024-05-26T22:48:23.509267+0000","last_fullsized":"2024-05-26T22:48:23.509267+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:45:06.922028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.809824+0000","last_change":"2024-05-26T22:48:21.829461+0000","last_active":"2024-05-26T22:48:22.809824+0000","last_peered":"2024-05-26T22:48:22.809824+0000","last_clean":"2024-05-26T22:48:22.809824+0000","last_became_active":"2024-05-26T22:48:21.829132+0000","last_became_peered":"2024-05-26T22:48:21.829132+0000","last_unstale":"2024-05-26T22:48:22.809824+0000","last_undegraded":"2024-05-26T22:48:22.809824+0000","last_fullsized":"2024-05-26T22:48:22.809824+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:52:25.073281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.809106+0000","last_change":"2024-05-26T22:48:21.818888+0000","last_active":"2024-05-26T22:48:22.809106+0000","last_peered":"2024-05-26T22:48:22.809106+0000","last_clean":"2024-05-26T22:48:22.809106+0000","last_became_active":"2024-05-26T22:48:21.818639+0000","last_became_peered":"2024-05-26T22:48:21.818639+0000","last_unstale":"2024-05-26T22:48:22.809106+0000","last_undegraded":"2024-05-26T22:48:22.809106+0000","last_fullsized":"2024-05-26T22:48:22.809106+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:38:00.344955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.506778+0000","last_change":"2024-05-26T22:48:21.842585+0000","last_active":"2024-05-26T22:48:23.506778+0000","last_peered":"2024-05-26T22:48:23.506778+0000","last_clean":"2024-05-26T22:48:23.506778+0000","last_became_active":"2024-05-26T22:48:21.842302+0000","last_became_peered":"2024-05-26T22:48:21.842302+0000","last_unstale":"2024-05-26T22:48:23.506778+0000","last_undegraded":"2024-05-26T22:48:23.506778+0000","last_fullsized":"2024-05-26T22:48:23.506778+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:08:10.330236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.811982+0000","last_change":"2024-05-26T22:48:21.830936+0000","last_active":"2024-05-26T22:48:22.811982+0000","last_peered":"2024-05-26T22:48:22.811982+0000","last_clean":"2024-05-26T22:48:22.811982+0000","last_became_active":"2024-05-26T22:48:21.830551+0000","last_became_peered":"2024-05-26T22:48:21.830551+0000","last_unstale":"2024-05-26T22:48:22.811982+0000","last_undegraded":"2024-05-26T22:48:22.811982+0000","last_fullsized":"2024-05-26T22:48:22.811982+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:27:34.959431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.508425+0000","last_change":"2024-05-26T22:48:21.837001+0000","last_active":"2024-05-26T22:48:23.508425+0000","last_peered":"2024-05-26T22:48:23.508425+0000","last_clean":"2024-05-26T22:48:23.508425+0000","last_became_active":"2024-05-26T22:48:21.836749+0000","last_became_peered":"2024-05-26T22:48:21.836749+0000","last_unstale":"2024-05-26T22:48:23.508425+0000","last_undegraded":"2024-05-26T22:48:23.508425+0000","last_fullsized":"2024-05-26T22:48:23.508425+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:30:45.689344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.812885+0000","last_change":"2024-05-26T22:48:21.834986+0000","last_active":"2024-05-26T22:48:22.812885+0000","last_peered":"2024-05-26T22:48:22.812885+0000","last_clean":"2024-05-26T22:48:22.812885+0000","last_became_active":"2024-05-26T22:48:21.834572+0000","last_became_peered":"2024-05-26T22:48:21.834572+0000","last_unstale":"2024-05-26T22:48:22.812885+0000","last_undegraded":"2024-05-26T22:48:22.812885+0000","last_fullsized":"2024-05-26T22:48:22.812885+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:33:39.923721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.810409+0000","last_change":"2024-05-26T22:48:21.824957+0000","last_active":"2024-05-26T22:48:22.810409+0000","last_peered":"2024-05-26T22:48:22.810409+0000","last_clean":"2024-05-26T22:48:22.810409+0000","last_became_active":"2024-05-26T22:48:21.822871+0000","last_became_peered":"2024-05-26T22:48:21.822871+0000","last_unstale":"2024-05-26T22:48:22.810409+0000","last_undegraded":"2024-05-26T22:48:22.810409+0000","last_fullsized":"2024-05-26T22:48:22.810409+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:59:10.543916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.822941+0000","last_change":"2024-05-26T22:48:21.827100+0000","last_active":"2024-05-26T22:48:22.822941+0000","last_peered":"2024-05-26T22:48:22.822941+0000","last_clean":"2024-05-26T22:48:22.822941+0000","last_became_active":"2024-05-26T22:48:21.826810+0000","last_became_peered":"2024-05-26T22:48:21.826810+0000","last_unstale":"2024-05-26T22:48:22.822941+0000","last_undegraded":"2024-05-26T22:48:22.822941+0000","last_fullsized":"2024-05-26T22:48:22.822941+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:38:00.344955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.813201+0000","last_change":"2024-05-26T22:48:21.836236+0000","last_active":"2024-05-26T22:48:22.813201+0000","last_peered":"2024-05-26T22:48:22.813201+0000","last_clean":"2024-05-26T22:48:22.813201+0000","last_became_active":"2024-05-26T22:48:21.834609+0000","last_became_peered":"2024-05-26T22:48:21.834609+0000","last_unstale":"2024-05-26T22:48:22.813201+0000","last_undegraded":"2024-05-26T22:48:22.813201+0000","last_fullsized":"2024-05-26T22:48:22.813201+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:08:10.330236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.809226+0000","last_change":"2024-05-26T22:48:21.821191+0000","last_active":"2024-05-26T22:48:22.809226+0000","last_peered":"2024-05-26T22:48:22.809226+0000","last_clean":"2024-05-26T22:48:22.809226+0000","last_became_active":"2024-05-26T22:48:21.820731+0000","last_became_peered":"2024-05-26T22:48:21.820731+0000","last_unstale":"2024-05-26T22:48:22.809226+0000","last_undegraded":"2024-05-26T22:48:22.809226+0000","last_fullsized":"2024-05-26T22:48:22.809226+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:59:43.907681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.507246+0000","last_change":"2024-05-26T22:48:21.836074+0000","last_active":"2024-05-26T22:48:23.507246+0000","last_peered":"2024-05-26T22:48:23.507246+0000","last_clean":"2024-05-26T22:48:23.507246+0000","last_became_active":"2024-05-26T22:48:21.831547+0000","last_became_peered":"2024-05-26T22:48:21.831547+0000","last_unstale":"2024-05-26T22:48:23.507246+0000","last_undegraded":"2024-05-26T22:48:23.507246+0000","last_fullsized":"2024-05-26T22:48:23.507246+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:34:58.598026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.503056+0000","last_change":"2024-05-26T22:48:21.815510+0000","last_active":"2024-05-26T22:48:23.503056+0000","last_peered":"2024-05-26T22:48:23.503056+0000","last_clean":"2024-05-26T22:48:23.503056+0000","last_became_active":"2024-05-26T22:48:21.815132+0000","last_became_peered":"2024-05-26T22:48:21.815132+0000","last_unstale":"2024-05-26T22:48:23.503056+0000","last_undegraded":"2024-05-26T22:48:23.503056+0000","last_fullsized":"2024-05-26T22:48:23.503056+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:56:00.075959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.809652+0000","last_change":"2024-05-26T22:48:21.811494+0000","last_active":"2024-05-26T22:48:22.809652+0000","last_peered":"2024-05-26T22:48:22.809652+0000","last_clean":"2024-05-26T22:48:22.809652+0000","last_became_active":"2024-05-26T22:48:21.809669+0000","last_became_peered":"2024-05-26T22:48:21.809669+0000","last_unstale":"2024-05-26T22:48:22.809652+0000","last_undegraded":"2024-05-26T22:48:22.809652+0000","last_fullsized":"2024-05-26T22:48:22.809652+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:27:34.959431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.504470+0000","last_change":"2024-05-26T22:48:21.824303+0000","last_active":"2024-05-26T22:48:23.504470+0000","last_peered":"2024-05-26T22:48:23.504470+0000","last_clean":"2024-05-26T22:48:23.504470+0000","last_became_active":"2024-05-26T22:48:21.823500+0000","last_became_peered":"2024-05-26T22:48:21.823500+0000","last_unstale":"2024-05-26T22:48:23.504470+0000","last_undegraded":"2024-05-26T22:48:23.504470+0000","last_fullsized":"2024-05-26T22:48:23.504470+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:16:02.119875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.812241+0000","last_change":"2024-05-26T22:48:20.802593+0000","last_active":"2024-05-26T22:48:22.812241+0000","last_peered":"2024-05-26T22:48:22.812241+0000","last_clean":"2024-05-26T22:48:22.812241+0000","last_became_active":"2024-05-26T22:48:20.802352+0000","last_became_peered":"2024-05-26T22:48:20.802352+0000","last_unstale":"2024-05-26T22:48:22.812241+0000","last_undegraded":"2024-05-26T22:48:22.812241+0000","last_fullsized":"2024-05-26T22:48:22.812241+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:39:51.267408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.809443+0000","last_change":"2024-05-26T22:48:21.832952+0000","last_active":"2024-05-26T22:48:22.809443+0000","last_peered":"2024-05-26T22:48:22.809443+0000","last_clean":"2024-05-26T22:48:22.809443+0000","last_became_active":"2024-05-26T22:48:21.832496+0000","last_became_peered":"2024-05-26T22:48:21.832496+0000","last_unstale":"2024-05-26T22:48:22.809443+0000","last_undegraded":"2024-05-26T22:48:22.809443+0000","last_fullsized":"2024-05-26T22:48:22.809443+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:59:10.543916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.810033+0000","last_change":"2024-05-26T22:48:20.789223+0000","last_active":"2024-05-26T22:48:22.810033+0000","last_peered":"2024-05-26T22:48:22.810033+0000","last_clean":"2024-05-26T22:48:22.810033+0000","last_became_active":"2024-05-26T22:48:20.787967+0000","last_became_peered":"2024-05-26T22:48:20.787967+0000","last_unstale":"2024-05-26T22:48:22.810033+0000","last_undegraded":"2024-05-26T22:48:22.810033+0000","last_fullsized":"2024-05-26T22:48:22.810033+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:08:01.698812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.812657+0000","last_change":"2024-05-26T22:48:20.784257+0000","last_active":"2024-05-26T22:48:22.812657+0000","last_peered":"2024-05-26T22:48:22.812657+0000","last_clean":"2024-05-26T22:48:22.812657+0000","last_became_active":"2024-05-26T22:48:20.783493+0000","last_became_peered":"2024-05-26T22:48:20.783493+0000","last_unstale":"2024-05-26T22:48:22.812657+0000","last_undegraded":"2024-05-26T22:48:22.812657+0000","last_fullsized":"2024-05-26T22:48:22.812657+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:30:48.024402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.813086+0000","last_change":"2024-05-26T22:48:21.834733+0000","last_active":"2024-05-26T22:48:22.813086+0000","last_peered":"2024-05-26T22:48:22.813086+0000","last_clean":"2024-05-26T22:48:22.813086+0000","last_became_active":"2024-05-26T22:48:21.834396+0000","last_became_peered":"2024-05-26T22:48:21.834396+0000","last_unstale":"2024-05-26T22:48:22.813086+0000","last_undegraded":"2024-05-26T22:48:22.813086+0000","last_fullsized":"2024-05-26T22:48:22.813086+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:56:39.208963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.506282+0000","last_change":"2024-05-26T22:48:20.798521+0000","last_active":"2024-05-26T22:48:23.506282+0000","last_peered":"2024-05-26T22:48:23.506282+0000","last_clean":"2024-05-26T22:48:23.506282+0000","last_became_active":"2024-05-26T22:48:20.798153+0000","last_became_peered":"2024-05-26T22:48:20.798153+0000","last_unstale":"2024-05-26T22:48:23.506282+0000","last_undegraded":"2024-05-26T22:48:23.506282+0000","last_fullsized":"2024-05-26T22:48:23.506282+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:10:49.991129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.823065+0000","last_change":"2024-05-26T22:48:21.832951+0000","last_active":"2024-05-26T22:48:22.823065+0000","last_peered":"2024-05-26T22:48:22.823065+0000","last_clean":"2024-05-26T22:48:22.823065+0000","last_became_active":"2024-05-26T22:48:21.831917+0000","last_became_peered":"2024-05-26T22:48:21.831917+0000","last_unstale":"2024-05-26T22:48:22.823065+0000","last_undegraded":"2024-05-26T22:48:22.823065+0000","last_fullsized":"2024-05-26T22:48:22.823065+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:59:10.543916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.810123+0000","last_change":"2024-05-26T22:48:20.801167+0000","last_active":"2024-05-26T22:48:22.810123+0000","last_peered":"2024-05-26T22:48:22.810123+0000","last_clean":"2024-05-26T22:48:22.810123+0000","last_became_active":"2024-05-26T22:48:20.800854+0000","last_became_peered":"2024-05-26T22:48:20.800854+0000","last_unstale":"2024-05-26T22:48:22.810123+0000","last_undegraded":"2024-05-26T22:48:22.810123+0000","last_fullsized":"2024-05-26T22:48:22.810123+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:39:51.267408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.509643+0000","last_change":"2024-05-26T22:48:21.841974+0000","last_active":"2024-05-26T22:48:23.509643+0000","last_peered":"2024-05-26T22:48:23.509643+0000","last_clean":"2024-05-26T22:48:23.509643+0000","last_became_active":"2024-05-26T22:48:21.840912+0000","last_became_peered":"2024-05-26T22:48:21.840912+0000","last_unstale":"2024-05-26T22:48:23.509643+0000","last_undegraded":"2024-05-26T22:48:23.509643+0000","last_fullsized":"2024-05-26T22:48:23.509643+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:27:35.210564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.509713+0000","last_change":"2024-05-26T22:48:20.798072+0000","last_active":"2024-05-26T22:48:23.509713+0000","last_peered":"2024-05-26T22:48:23.509713+0000","last_clean":"2024-05-26T22:48:23.509713+0000","last_became_active":"2024-05-26T22:48:20.797559+0000","last_became_peered":"2024-05-26T22:48:20.797559+0000","last_unstale":"2024-05-26T22:48:23.509713+0000","last_undegraded":"2024-05-26T22:48:23.509713+0000","last_fullsized":"2024-05-26T22:48:23.509713+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:52:24.060261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.809896+0000","last_change":"2024-05-26T22:48:21.838154+0000","last_active":"2024-05-26T22:48:22.809896+0000","last_peered":"2024-05-26T22:48:22.809896+0000","last_clean":"2024-05-26T22:48:22.809896+0000","last_became_active":"2024-05-26T22:48:21.837073+0000","last_became_peered":"2024-05-26T22:48:21.837073+0000","last_unstale":"2024-05-26T22:48:22.809896+0000","last_undegraded":"2024-05-26T22:48:22.809896+0000","last_fullsized":"2024-05-26T22:48:22.809896+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:56:39.208963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.822642+0000","last_change":"2024-05-26T22:48:20.796108+0000","last_active":"2024-05-26T22:48:22.822642+0000","last_peered":"2024-05-26T22:48:22.822642+0000","last_clean":"2024-05-26T22:48:22.822642+0000","last_became_active":"2024-05-26T22:48:20.795717+0000","last_became_peered":"2024-05-26T22:48:20.795717+0000","last_unstale":"2024-05-26T22:48:22.822642+0000","last_undegraded":"2024-05-26T22:48:22.822642+0000","last_fullsized":"2024-05-26T22:48:22.822642+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:39:51.267408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.505587+0000","last_change":"2024-05-26T22:48:21.823519+0000","last_active":"2024-05-26T22:48:23.505587+0000","last_peered":"2024-05-26T22:48:23.505587+0000","last_clean":"2024-05-26T22:48:23.505587+0000","last_became_active":"2024-05-26T22:48:21.823240+0000","last_became_peered":"2024-05-26T22:48:21.823240+0000","last_unstale":"2024-05-26T22:48:23.505587+0000","last_undegraded":"2024-05-26T22:48:23.505587+0000","last_fullsized":"2024-05-26T22:48:23.505587+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:33:51.651353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.809592+0000","last_change":"2024-05-26T22:48:20.792915+0000","last_active":"2024-05-26T22:48:22.809592+0000","last_peered":"2024-05-26T22:48:22.809592+0000","last_clean":"2024-05-26T22:48:22.809592+0000","last_became_active":"2024-05-26T22:48:20.792037+0000","last_became_peered":"2024-05-26T22:48:20.792037+0000","last_unstale":"2024-05-26T22:48:22.809592+0000","last_undegraded":"2024-05-26T22:48:22.809592+0000","last_fullsized":"2024-05-26T22:48:22.809592+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:11:29.128727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.508563+0000","last_change":"2024-05-26T22:48:21.839574+0000","last_active":"2024-05-26T22:48:23.508563+0000","last_peered":"2024-05-26T22:48:23.508563+0000","last_clean":"2024-05-26T22:48:23.508563+0000","last_became_active":"2024-05-26T22:48:21.834905+0000","last_became_peered":"2024-05-26T22:48:21.834905+0000","last_unstale":"2024-05-26T22:48:23.508563+0000","last_undegraded":"2024-05-26T22:48:23.508563+0000","last_fullsized":"2024-05-26T22:48:23.508563+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:34:58.598026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.809371+0000","last_change":"2024-05-26T22:48:20.792287+0000","last_active":"2024-05-26T22:48:22.809371+0000","last_peered":"2024-05-26T22:48:22.809371+0000","last_clean":"2024-05-26T22:48:22.809371+0000","last_became_active":"2024-05-26T22:48:20.791592+0000","last_became_peered":"2024-05-26T22:48:20.791592+0000","last_unstale":"2024-05-26T22:48:22.809371+0000","last_undegraded":"2024-05-26T22:48:22.809371+0000","last_fullsized":"2024-05-26T22:48:22.809371+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:30:48.024402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.808872+0000","last_change":"2024-05-26T22:48:21.824723+0000","last_active":"2024-05-26T22:48:22.808872+0000","last_peered":"2024-05-26T22:48:22.808872+0000","last_clean":"2024-05-26T22:48:22.808872+0000","last_became_active":"2024-05-26T22:48:21.824372+0000","last_became_peered":"2024-05-26T22:48:21.824372+0000","last_unstale":"2024-05-26T22:48:22.808872+0000","last_undegraded":"2024-05-26T22:48:22.808872+0000","last_fullsized":"2024-05-26T22:48:22.808872+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:52:25.073281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.507397+0000","last_change":"2024-05-26T22:48:20.802334+0000","last_active":"2024-05-26T22:48:23.507397+0000","last_peered":"2024-05-26T22:48:23.507397+0000","last_clean":"2024-05-26T22:48:23.507397+0000","last_became_active":"2024-05-26T22:48:20.802051+0000","last_became_peered":"2024-05-26T22:48:20.802051+0000","last_unstale":"2024-05-26T22:48:23.507397+0000","last_undegraded":"2024-05-26T22:48:23.507397+0000","last_fullsized":"2024-05-26T22:48:23.507397+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:55:59.062939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.508279+0000","last_change":"2024-05-26T22:48:21.830682+0000","last_active":"2024-05-26T22:48:23.508279+0000","last_peered":"2024-05-26T22:48:23.508279+0000","last_clean":"2024-05-26T22:48:23.508279+0000","last_became_active":"2024-05-26T22:48:21.830273+0000","last_became_peered":"2024-05-26T22:48:21.830273+0000","last_unstale":"2024-05-26T22:48:23.508279+0000","last_undegraded":"2024-05-26T22:48:23.508279+0000","last_fullsized":"2024-05-26T22:48:23.508279+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:15:00.790746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.809215+0000","last_change":"2024-05-26T22:48:20.791028+0000","last_active":"2024-05-26T22:48:22.809215+0000","last_peered":"2024-05-26T22:48:22.809215+0000","last_clean":"2024-05-26T22:48:22.809215+0000","last_became_active":"2024-05-26T22:48:20.790727+0000","last_became_peered":"2024-05-26T22:48:20.790727+0000","last_unstale":"2024-05-26T22:48:22.809215+0000","last_undegraded":"2024-05-26T22:48:22.809215+0000","last_fullsized":"2024-05-26T22:48:22.809215+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:20:41.886676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.809552+0000","last_change":"2024-05-26T22:48:21.809785+0000","last_active":"2024-05-26T22:48:22.809552+0000","last_peered":"2024-05-26T22:48:22.809552+0000","last_clean":"2024-05-26T22:48:22.809552+0000","last_became_active":"2024-05-26T22:48:21.809460+0000","last_became_peered":"2024-05-26T22:48:21.809460+0000","last_unstale":"2024-05-26T22:48:22.809552+0000","last_undegraded":"2024-05-26T22:48:22.809552+0000","last_fullsized":"2024-05-26T22:48:22.809552+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T23:40:55.485829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.502702+0000","last_change":"2024-05-26T22:48:20.788462+0000","last_active":"2024-05-26T22:48:23.502702+0000","last_peered":"2024-05-26T22:48:23.502702+0000","last_clean":"2024-05-26T22:48:23.502702+0000","last_became_active":"2024-05-26T22:48:20.787714+0000","last_became_peered":"2024-05-26T22:48:20.787714+0000","last_unstale":"2024-05-26T22:48:23.502702+0000","last_undegraded":"2024-05-26T22:48:23.502702+0000","last_fullsized":"2024-05-26T22:48:23.502702+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:08:01.698812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.507359+0000","last_change":"2024-05-26T22:48:21.823217+0000","last_active":"2024-05-26T22:48:23.507359+0000","last_peered":"2024-05-26T22:48:23.507359+0000","last_clean":"2024-05-26T22:48:23.507359+0000","last_became_active":"2024-05-26T22:48:21.822856+0000","last_became_peered":"2024-05-26T22:48:21.822856+0000","last_unstale":"2024-05-26T22:48:23.507359+0000","last_undegraded":"2024-05-26T22:48:23.507359+0000","last_fullsized":"2024-05-26T22:48:23.507359+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:08:10.330236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.505497+0000","last_change":"2024-05-26T22:48:20.793786+0000","last_active":"2024-05-26T22:48:23.505497+0000","last_peered":"2024-05-26T22:48:23.505497+0000","last_clean":"2024-05-26T22:48:23.505497+0000","last_became_active":"2024-05-26T22:48:20.793201+0000","last_became_peered":"2024-05-26T22:48:20.793201+0000","last_unstale":"2024-05-26T22:48:23.505497+0000","last_undegraded":"2024-05-26T22:48:23.505497+0000","last_fullsized":"2024-05-26T22:48:23.505497+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:30:48.024402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.509387+0000","last_change":"2024-05-26T22:48:21.832244+0000","last_active":"2024-05-26T22:48:23.509387+0000","last_peered":"2024-05-26T22:48:23.509387+0000","last_clean":"2024-05-26T22:48:23.509387+0000","last_became_active":"2024-05-26T22:48:21.831897+0000","last_became_peered":"2024-05-26T22:48:21.831897+0000","last_unstale":"2024-05-26T22:48:23.509387+0000","last_undegraded":"2024-05-26T22:48:23.509387+0000","last_fullsized":"2024-05-26T22:48:23.509387+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:49:40.791149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.503336+0000","last_change":"2024-05-26T22:48:20.791496+0000","last_active":"2024-05-26T22:48:23.503336+0000","last_peered":"2024-05-26T22:48:23.503336+0000","last_clean":"2024-05-26T22:48:23.503336+0000","last_became_active":"2024-05-26T22:48:20.790743+0000","last_became_peered":"2024-05-26T22:48:20.790743+0000","last_unstale":"2024-05-26T22:48:23.503336+0000","last_undegraded":"2024-05-26T22:48:23.503336+0000","last_fullsized":"2024-05-26T22:48:23.503336+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:20:41.886676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.509655+0000","last_change":"2024-05-26T22:48:21.837602+0000","last_active":"2024-05-26T22:48:23.509655+0000","last_peered":"2024-05-26T22:48:23.509655+0000","last_clean":"2024-05-26T22:48:23.509655+0000","last_became_active":"2024-05-26T22:48:21.834873+0000","last_became_peered":"2024-05-26T22:48:21.834873+0000","last_unstale":"2024-05-26T22:48:23.509655+0000","last_undegraded":"2024-05-26T22:48:23.509655+0000","last_fullsized":"2024-05-26T22:48:23.509655+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:12:50.184011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.822772+0000","last_change":"2024-05-26T22:48:20.782935+0000","last_active":"2024-05-26T22:48:22.822772+0000","last_peered":"2024-05-26T22:48:22.822772+0000","last_clean":"2024-05-26T22:48:22.822772+0000","last_became_active":"2024-05-26T22:48:20.782624+0000","last_became_peered":"2024-05-26T22:48:20.782624+0000","last_unstale":"2024-05-26T22:48:22.822772+0000","last_undegraded":"2024-05-26T22:48:22.822772+0000","last_fullsized":"2024-05-26T22:48:22.822772+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:55:59.062939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.507335+0000","last_change":"2024-05-26T22:48:21.836045+0000","last_active":"2024-05-26T22:48:23.507335+0000","last_peered":"2024-05-26T22:48:23.507335+0000","last_clean":"2024-05-26T22:48:23.507335+0000","last_became_active":"2024-05-26T22:48:21.830070+0000","last_became_peered":"2024-05-26T22:48:21.830070+0000","last_unstale":"2024-05-26T22:48:23.507335+0000","last_undegraded":"2024-05-26T22:48:23.507335+0000","last_fullsized":"2024-05-26T22:48:23.507335+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:11:40.258537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.503246+0000","last_change":"2024-05-26T22:48:20.805086+0000","last_active":"2024-05-26T22:48:23.503246+0000","last_peered":"2024-05-26T22:48:23.503246+0000","last_clean":"2024-05-26T22:48:23.503246+0000","last_became_active":"2024-05-26T22:48:20.804732+0000","last_became_peered":"2024-05-26T22:48:20.804732+0000","last_unstale":"2024-05-26T22:48:23.503246+0000","last_undegraded":"2024-05-26T22:48:23.503246+0000","last_fullsized":"2024-05-26T22:48:23.503246+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:11:29.128727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.509008+0000","last_change":"2024-05-26T22:48:21.840364+0000","last_active":"2024-05-26T22:48:23.509008+0000","last_peered":"2024-05-26T22:48:23.509008+0000","last_clean":"2024-05-26T22:48:23.509008+0000","last_became_active":"2024-05-26T22:48:21.839692+0000","last_became_peered":"2024-05-26T22:48:21.839692+0000","last_unstale":"2024-05-26T22:48:23.509008+0000","last_undegraded":"2024-05-26T22:48:23.509008+0000","last_fullsized":"2024-05-26T22:48:23.509008+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:11:40.258537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.506053+0000","last_change":"2024-05-26T22:48:12.068917+0000","last_active":"2024-05-26T22:48:23.506053+0000","last_peered":"2024-05-26T22:48:23.506053+0000","last_clean":"2024-05-26T22:48:23.506053+0000","last_became_active":"2024-05-26T22:48:12.065934+0000","last_became_peered":"2024-05-26T22:48:12.065934+0000","last_unstale":"2024-05-26T22:48:23.506053+0000","last_undegraded":"2024-05-26T22:48:23.506053+0000","last_fullsized":"2024-05-26T22:48:23.506053+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:10.628122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:10.628122+0000","last_clean_scrub_stamp":"2024-05-26T22:48:10.628122+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:14:58.895911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.506186+0000","last_change":"2024-05-26T22:48:20.800517+0000","last_active":"2024-05-26T22:48:23.506186+0000","last_peered":"2024-05-26T22:48:23.506186+0000","last_clean":"2024-05-26T22:48:23.506186+0000","last_became_active":"2024-05-26T22:48:20.797956+0000","last_became_peered":"2024-05-26T22:48:20.797956+0000","last_unstale":"2024-05-26T22:48:23.506186+0000","last_undegraded":"2024-05-26T22:48:23.506186+0000","last_fullsized":"2024-05-26T22:48:23.506186+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:39:51.267408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.812649+0000","last_change":"2024-05-26T22:48:21.844543+0000","last_active":"2024-05-26T22:48:22.812649+0000","last_peered":"2024-05-26T22:48:22.812649+0000","last_clean":"2024-05-26T22:48:22.812649+0000","last_became_active":"2024-05-26T22:48:21.844272+0000","last_became_peered":"2024-05-26T22:48:21.844272+0000","last_unstale":"2024-05-26T22:48:22.812649+0000","last_undegraded":"2024-05-26T22:48:22.812649+0000","last_fullsized":"2024-05-26T22:48:22.812649+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:59:10.543916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.505812+0000","last_change":"2024-05-26T22:48:20.799727+0000","last_active":"2024-05-26T22:48:23.505812+0000","last_peered":"2024-05-26T22:48:23.505812+0000","last_clean":"2024-05-26T22:48:23.505812+0000","last_became_active":"2024-05-26T22:48:20.799427+0000","last_became_peered":"2024-05-26T22:48:20.799427+0000","last_unstale":"2024-05-26T22:48:23.505812+0000","last_undegraded":"2024-05-26T22:48:23.505812+0000","last_fullsized":"2024-05-26T22:48:23.505812+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:55:59.062939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.808326+0000","last_change":"2024-05-26T22:48:21.832212+0000","last_active":"2024-05-26T22:48:22.808326+0000","last_peered":"2024-05-26T22:48:22.808326+0000","last_clean":"2024-05-26T22:48:22.808326+0000","last_became_active":"2024-05-26T22:48:21.831893+0000","last_became_peered":"2024-05-26T22:48:21.831893+0000","last_unstale":"2024-05-26T22:48:22.808326+0000","last_undegraded":"2024-05-26T22:48:22.808326+0000","last_fullsized":"2024-05-26T22:48:22.808326+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T23:40:55.485829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.502923+0000","last_change":"2024-05-26T22:48:20.785204+0000","last_active":"2024-05-26T22:48:23.502923+0000","last_peered":"2024-05-26T22:48:23.502923+0000","last_clean":"2024-05-26T22:48:23.502923+0000","last_became_active":"2024-05-26T22:48:20.784798+0000","last_became_peered":"2024-05-26T22:48:20.784798+0000","last_unstale":"2024-05-26T22:48:23.502923+0000","last_undegraded":"2024-05-26T22:48:23.502923+0000","last_fullsized":"2024-05-26T22:48:23.502923+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:30:48.024402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.822381+0000","last_change":"2024-05-26T22:48:21.811298+0000","last_active":"2024-05-26T22:48:22.822381+0000","last_peered":"2024-05-26T22:48:22.822381+0000","last_clean":"2024-05-26T22:48:22.822381+0000","last_became_active":"2024-05-26T22:48:21.807981+0000","last_became_peered":"2024-05-26T22:48:21.807981+0000","last_unstale":"2024-05-26T22:48:22.822381+0000","last_undegraded":"2024-05-26T22:48:22.822381+0000","last_fullsized":"2024-05-26T22:48:22.822381+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:27:34.959431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.507652+0000","last_change":"2024-05-26T22:48:20.794578+0000","last_active":"2024-05-26T22:48:23.507652+0000","last_peered":"2024-05-26T22:48:23.507652+0000","last_clean":"2024-05-26T22:48:23.507652+0000","last_became_active":"2024-05-26T22:48:20.794262+0000","last_became_peered":"2024-05-26T22:48:20.794262+0000","last_unstale":"2024-05-26T22:48:23.507652+0000","last_undegraded":"2024-05-26T22:48:23.507652+0000","last_fullsized":"2024-05-26T22:48:23.507652+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:16:01.106855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.505213+0000","last_change":"2024-05-26T22:48:21.826194+0000","last_active":"2024-05-26T22:48:23.505213+0000","last_peered":"2024-05-26T22:48:23.505213+0000","last_clean":"2024-05-26T22:48:23.505213+0000","last_became_active":"2024-05-26T22:48:21.825674+0000","last_became_peered":"2024-05-26T22:48:21.825674+0000","last_unstale":"2024-05-26T22:48:23.505213+0000","last_undegraded":"2024-05-26T22:48:23.505213+0000","last_fullsized":"2024-05-26T22:48:23.505213+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:01:59.805532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.822681+0000","last_change":"2024-05-26T22:48:20.788924+0000","last_active":"2024-05-26T22:48:22.822681+0000","last_peered":"2024-05-26T22:48:22.822681+0000","last_clean":"2024-05-26T22:48:22.822681+0000","last_became_active":"2024-05-26T22:48:20.788036+0000","last_became_peered":"2024-05-26T22:48:20.788036+0000","last_unstale":"2024-05-26T22:48:22.822681+0000","last_undegraded":"2024-05-26T22:48:22.822681+0000","last_fullsized":"2024-05-26T22:48:22.822681+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:10:49.991129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.507362+0000","last_change":"2024-05-26T22:48:21.833223+0000","last_active":"2024-05-26T22:48:23.507362+0000","last_peered":"2024-05-26T22:48:23.507362+0000","last_clean":"2024-05-26T22:48:23.507362+0000","last_became_active":"2024-05-26T22:48:21.831371+0000","last_became_peered":"2024-05-26T22:48:21.831371+0000","last_unstale":"2024-05-26T22:48:23.507362+0000","last_undegraded":"2024-05-26T22:48:23.507362+0000","last_fullsized":"2024-05-26T22:48:23.507362+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:45:06.922028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.509127+0000","last_change":"2024-05-26T22:48:20.800026+0000","last_active":"2024-05-26T22:48:23.509127+0000","last_peered":"2024-05-26T22:48:23.509127+0000","last_clean":"2024-05-26T22:48:23.509127+0000","last_became_active":"2024-05-26T22:48:20.799776+0000","last_became_peered":"2024-05-26T22:48:20.799776+0000","last_unstale":"2024-05-26T22:48:23.509127+0000","last_undegraded":"2024-05-26T22:48:23.509127+0000","last_fullsized":"2024-05-26T22:48:23.509127+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:33:50.638333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.822718+0000","last_change":"2024-05-26T22:48:21.811868+0000","last_active":"2024-05-26T22:48:22.822718+0000","last_peered":"2024-05-26T22:48:22.822718+0000","last_clean":"2024-05-26T22:48:22.822718+0000","last_became_active":"2024-05-26T22:48:21.811205+0000","last_became_peered":"2024-05-26T22:48:21.811205+0000","last_unstale":"2024-05-26T22:48:22.822718+0000","last_undegraded":"2024-05-26T22:48:22.822718+0000","last_fullsized":"2024-05-26T22:48:22.822718+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:43:21.391197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.823108+0000","last_change":"2024-05-26T22:48:20.786141+0000","last_active":"2024-05-26T22:48:22.823108+0000","last_peered":"2024-05-26T22:48:22.823108+0000","last_clean":"2024-05-26T22:48:22.823108+0000","last_became_active":"2024-05-26T22:48:20.785581+0000","last_became_peered":"2024-05-26T22:48:20.785581+0000","last_unstale":"2024-05-26T22:48:22.823108+0000","last_undegraded":"2024-05-26T22:48:22.823108+0000","last_fullsized":"2024-05-26T22:48:22.823108+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:16:01.106855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.812734+0000","last_change":"2024-05-26T22:48:21.828936+0000","last_active":"2024-05-26T22:48:22.812734+0000","last_peered":"2024-05-26T22:48:22.812734+0000","last_clean":"2024-05-26T22:48:22.812734+0000","last_became_active":"2024-05-26T22:48:21.828635+0000","last_became_peered":"2024-05-26T22:48:21.828635+0000","last_unstale":"2024-05-26T22:48:22.812734+0000","last_undegraded":"2024-05-26T22:48:22.812734+0000","last_fullsized":"2024-05-26T22:48:22.812734+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:21:52.108732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.810327+0000","last_change":"2024-05-26T22:48:20.798700+0000","last_active":"2024-05-26T22:48:22.810327+0000","last_peered":"2024-05-26T22:48:22.810327+0000","last_clean":"2024-05-26T22:48:22.810327+0000","last_became_active":"2024-05-26T22:48:20.796914+0000","last_became_peered":"2024-05-26T22:48:20.796914+0000","last_unstale":"2024-05-26T22:48:22.810327+0000","last_undegraded":"2024-05-26T22:48:22.810327+0000","last_fullsized":"2024-05-26T22:48:22.810327+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:10:49.991129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.809579+0000","last_change":"2024-05-26T22:48:21.811239+0000","last_active":"2024-05-26T22:48:22.809579+0000","last_peered":"2024-05-26T22:48:22.809579+0000","last_clean":"2024-05-26T22:48:22.809579+0000","last_became_active":"2024-05-26T22:48:21.810945+0000","last_became_peered":"2024-05-26T22:48:21.810945+0000","last_unstale":"2024-05-26T22:48:22.809579+0000","last_undegraded":"2024-05-26T22:48:22.809579+0000","last_fullsized":"2024-05-26T22:48:22.809579+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:21:52.108732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.505531+0000","last_change":"2024-05-26T22:48:20.802835+0000","last_active":"2024-05-26T22:48:23.505531+0000","last_peered":"2024-05-26T22:48:23.505531+0000","last_clean":"2024-05-26T22:48:23.505531+0000","last_became_active":"2024-05-26T22:48:20.802501+0000","last_became_peered":"2024-05-26T22:48:20.802501+0000","last_unstale":"2024-05-26T22:48:23.505531+0000","last_undegraded":"2024-05-26T22:48:23.505531+0000","last_fullsized":"2024-05-26T22:48:23.505531+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:39:51.267408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.504467+0000","last_change":"2024-05-26T22:48:21.832723+0000","last_active":"2024-05-26T22:48:23.504467+0000","last_peered":"2024-05-26T22:48:23.504467+0000","last_clean":"2024-05-26T22:48:23.504467+0000","last_became_active":"2024-05-26T22:48:21.824388+0000","last_became_peered":"2024-05-26T22:48:21.824388+0000","last_unstale":"2024-05-26T22:48:23.504467+0000","last_undegraded":"2024-05-26T22:48:23.504467+0000","last_fullsized":"2024-05-26T22:48:23.504467+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:56:00.075959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.822800+0000","last_change":"2024-05-26T22:48:20.797481+0000","last_active":"2024-05-26T22:48:22.822800+0000","last_peered":"2024-05-26T22:48:22.822800+0000","last_clean":"2024-05-26T22:48:22.822800+0000","last_became_active":"2024-05-26T22:48:20.797142+0000","last_became_peered":"2024-05-26T22:48:20.797142+0000","last_unstale":"2024-05-26T22:48:22.822800+0000","last_undegraded":"2024-05-26T22:48:22.822800+0000","last_fullsized":"2024-05-26T22:48:22.822800+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:08:01.698812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.812496+0000","last_change":"2024-05-26T22:48:21.819388+0000","last_active":"2024-05-26T22:48:22.812496+0000","last_peered":"2024-05-26T22:48:22.812496+0000","last_clean":"2024-05-26T22:48:22.812496+0000","last_became_active":"2024-05-26T22:48:21.817847+0000","last_became_peered":"2024-05-26T22:48:21.817847+0000","last_unstale":"2024-05-26T22:48:22.812496+0000","last_undegraded":"2024-05-26T22:48:22.812496+0000","last_fullsized":"2024-05-26T22:48:22.812496+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:38:00.344955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.822809+0000","last_change":"2024-05-26T22:48:20.785751+0000","last_active":"2024-05-26T22:48:22.822809+0000","last_peered":"2024-05-26T22:48:22.822809+0000","last_clean":"2024-05-26T22:48:22.822809+0000","last_became_active":"2024-05-26T22:48:20.785494+0000","last_became_peered":"2024-05-26T22:48:20.785494+0000","last_unstale":"2024-05-26T22:48:22.822809+0000","last_undegraded":"2024-05-26T22:48:22.822809+0000","last_fullsized":"2024-05-26T22:48:22.822809+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:20:41.886676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.812182+0000","last_change":"2024-05-26T22:48:21.825344+0000","last_active":"2024-05-26T22:48:22.812182+0000","last_peered":"2024-05-26T22:48:22.812182+0000","last_clean":"2024-05-26T22:48:22.812182+0000","last_became_active":"2024-05-26T22:48:21.823452+0000","last_became_peered":"2024-05-26T22:48:21.823452+0000","last_unstale":"2024-05-26T22:48:22.812182+0000","last_undegraded":"2024-05-26T22:48:22.812182+0000","last_fullsized":"2024-05-26T22:48:22.812182+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:52:25.073281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.508330+0000","last_change":"2024-05-26T22:48:20.803962+0000","last_active":"2024-05-26T22:48:23.508330+0000","last_peered":"2024-05-26T22:48:23.508330+0000","last_clean":"2024-05-26T22:48:23.508330+0000","last_became_active":"2024-05-26T22:48:20.803609+0000","last_became_peered":"2024-05-26T22:48:20.803609+0000","last_unstale":"2024-05-26T22:48:23.508330+0000","last_undegraded":"2024-05-26T22:48:23.508330+0000","last_fullsized":"2024-05-26T22:48:23.508330+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:16:01.106855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.809871+0000","last_change":"2024-05-26T22:48:21.829756+0000","last_active":"2024-05-26T22:48:22.809871+0000","last_peered":"2024-05-26T22:48:22.809871+0000","last_clean":"2024-05-26T22:48:22.809871+0000","last_became_active":"2024-05-26T22:48:21.829451+0000","last_became_peered":"2024-05-26T22:48:21.829451+0000","last_unstale":"2024-05-26T22:48:22.809871+0000","last_undegraded":"2024-05-26T22:48:22.809871+0000","last_fullsized":"2024-05-26T22:48:22.809871+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:43:21.391197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.506838+0000","last_change":"2024-05-26T22:48:21.832211+0000","last_active":"2024-05-26T22:48:23.506838+0000","last_peered":"2024-05-26T22:48:23.506838+0000","last_clean":"2024-05-26T22:48:23.506838+0000","last_became_active":"2024-05-26T22:48:21.831732+0000","last_became_peered":"2024-05-26T22:48:21.831732+0000","last_unstale":"2024-05-26T22:48:23.506838+0000","last_undegraded":"2024-05-26T22:48:23.506838+0000","last_fullsized":"2024-05-26T22:48:23.506838+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:16:22.847663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.505649+0000","last_change":"2024-05-26T22:48:20.805024+0000","last_active":"2024-05-26T22:48:23.505649+0000","last_peered":"2024-05-26T22:48:23.505649+0000","last_clean":"2024-05-26T22:48:23.505649+0000","last_became_active":"2024-05-26T22:48:20.804736+0000","last_became_peered":"2024-05-26T22:48:20.804736+0000","last_unstale":"2024-05-26T22:48:23.505649+0000","last_undegraded":"2024-05-26T22:48:23.505649+0000","last_fullsized":"2024-05-26T22:48:23.505649+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:42:31.965689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.506861+0000","last_change":"2024-05-26T22:48:21.832952+0000","last_active":"2024-05-26T22:48:23.506861+0000","last_peered":"2024-05-26T22:48:23.506861+0000","last_clean":"2024-05-26T22:48:23.506861+0000","last_became_active":"2024-05-26T22:48:21.831488+0000","last_became_peered":"2024-05-26T22:48:21.831488+0000","last_unstale":"2024-05-26T22:48:23.506861+0000","last_undegraded":"2024-05-26T22:48:23.506861+0000","last_fullsized":"2024-05-26T22:48:23.506861+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:15:00.790746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.508687+0000","last_change":"2024-05-26T22:48:20.799474+0000","last_active":"2024-05-26T22:48:23.508687+0000","last_peered":"2024-05-26T22:48:23.508687+0000","last_clean":"2024-05-26T22:48:23.508687+0000","last_became_active":"2024-05-26T22:48:20.799031+0000","last_became_peered":"2024-05-26T22:48:20.799031+0000","last_unstale":"2024-05-26T22:48:23.508687+0000","last_undegraded":"2024-05-26T22:48:23.508687+0000","last_fullsized":"2024-05-26T22:48:23.508687+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:01:58.792512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.809810+0000","last_change":"2024-05-26T22:48:21.837774+0000","last_active":"2024-05-26T22:48:22.809810+0000","last_peered":"2024-05-26T22:48:22.809810+0000","last_clean":"2024-05-26T22:48:22.809810+0000","last_became_active":"2024-05-26T22:48:21.837450+0000","last_became_peered":"2024-05-26T22:48:21.837450+0000","last_unstale":"2024-05-26T22:48:22.809810+0000","last_undegraded":"2024-05-26T22:48:22.809810+0000","last_fullsized":"2024-05-26T22:48:22.809810+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:33:39.923721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.812718+0000","last_change":"2024-05-26T22:48:20.788987+0000","last_active":"2024-05-26T22:48:22.812718+0000","last_peered":"2024-05-26T22:48:22.812718+0000","last_clean":"2024-05-26T22:48:22.812718+0000","last_became_active":"2024-05-26T22:48:20.788612+0000","last_became_peered":"2024-05-26T22:48:20.788612+0000","last_unstale":"2024-05-26T22:48:22.812718+0000","last_undegraded":"2024-05-26T22:48:22.812718+0000","last_fullsized":"2024-05-26T22:48:22.812718+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:20:41.886676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.812548+0000","last_change":"2024-05-26T22:48:21.826820+0000","last_active":"2024-05-26T22:48:22.812548+0000","last_peered":"2024-05-26T22:48:22.812548+0000","last_clean":"2024-05-26T22:48:22.812548+0000","last_became_active":"2024-05-26T22:48:21.826593+0000","last_became_peered":"2024-05-26T22:48:21.826593+0000","last_unstale":"2024-05-26T22:48:22.812548+0000","last_undegraded":"2024-05-26T22:48:22.812548+0000","last_fullsized":"2024-05-26T22:48:22.812548+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:59:43.907681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.809942+0000","last_change":"2024-05-26T22:48:20.796965+0000","last_active":"2024-05-26T22:48:22.809942+0000","last_peered":"2024-05-26T22:48:22.809942+0000","last_clean":"2024-05-26T22:48:22.809942+0000","last_became_active":"2024-05-26T22:48:20.796587+0000","last_became_peered":"2024-05-26T22:48:20.796587+0000","last_unstale":"2024-05-26T22:48:22.809942+0000","last_undegraded":"2024-05-26T22:48:22.809942+0000","last_fullsized":"2024-05-26T22:48:22.809942+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:11:29.128727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-26T22:48:26.447 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-26T22:48:26.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.682+0000 7f62ed400640 1 Processor -- start 2024-05-26T22:48:26.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.682+0000 7f62ed400640 1 -- start start 2024-05-26T22:48:26.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.684+0000 7f62ed400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f62e8144800 0x7f62e815fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:26.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.684+0000 7f62ed400640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f62e80ad870 0x7f62e80adcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:26.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.684+0000 7f62ed400640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62e815f7b0 con 0x7f62e815e9f0 2024-05-26T22:48:26.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.684+0000 7f62ed400640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f62e815f590 con 0x7f62e80ad870 2024-05-26T22:48:26.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.684+0000 7f62ed400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f62e80ae210 con 0x7f62e8144800 2024-05-26T22:48:26.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.684+0000 7f62dfe00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f62e80ad870 0x7f62e80adcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:26.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.684+0000 7f62e7e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f62e8144800 0x7f62e815fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:26.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.684+0000 7f62dfe00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f62e80ad870 0x7f62e80adcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.131:36288/0 (socket says 172.21.15.131:36288) 2024-05-26T22:48:26.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.684+0000 7f62dfe00640 1 -- 172.21.15.131:0/2640178266 learned_addr learned my addr 172.21.15.131:0/2640178266 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:26.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.685+0000 7f62e7e00640 1 -- 172.21.15.131:0/2640178266 >> v2:172.21.15.187:3300/0 conn(0x7f62e80ad870 msgr2=0x7f62e80adcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:26.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.685+0000 7f62e7e00640 1 --2- 172.21.15.131:0/2640178266 >> v2:172.21.15.187:3300/0 conn(0x7f62e80ad870 0x7f62e80adcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:26.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.685+0000 7f62e7e00640 1 -- 172.21.15.131:0/2640178266 >> v1:172.21.15.187:6789/0 conn(0x7f62e815e9f0 legacy=0x7f62e815ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:26.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.685+0000 7f62e7e00640 1 -- 172.21.15.131:0/2640178266 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62e80cde40 con 0x7f62e8144800 2024-05-26T22:48:26.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.685+0000 7f62dfe00640 1 --2- 172.21.15.131:0/2640178266 >> v2:172.21.15.187:3300/0 conn(0x7f62e80ad870 0x7f62e80adcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-26T22:48:26.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.687+0000 7f62e7e00640 1 --2- 172.21.15.131:0/2640178266 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f62e8144800 0x7f62e815fa90 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f62d80097a0 tx=0x7f62d802a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69f421dd0de795cf server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:26.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.687+0000 7f62e7400640 1 -- 172.21.15.131:0/2640178266 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62d8037070 con 0x7f62e8144800 2024-05-26T22:48:26.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.687+0000 7f62e7400640 1 -- 172.21.15.131:0/2640178266 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62d8032440 con 0x7f62e8144800 2024-05-26T22:48:26.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.687+0000 7f62e7400640 1 -- 172.21.15.131:0/2640178266 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62d8032810 con 0x7f62e8144800 2024-05-26T22:48:26.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.688+0000 7f62ed400640 1 -- 172.21.15.131:0/2640178266 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f62e8144800 msgr2=0x7f62e815fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:26.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.688+0000 7f62ed400640 1 --2- 172.21.15.131:0/2640178266 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f62e8144800 0x7f62e815fa90 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f62d80097a0 tx=0x7f62d802a210 comp rx=0 tx=0).stop 2024-05-26T22:48:26.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.688+0000 7f62ed400640 1 -- 172.21.15.131:0/2640178266 shutdown_connections 2024-05-26T22:48:26.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.688+0000 7f62ed400640 1 --2- 172.21.15.131:0/2640178266 >> v2:172.21.15.187:3300/0 conn(0x7f62e80ad870 0x7f62e80adcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:26.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.688+0000 7f62ed400640 1 --2- 172.21.15.131:0/2640178266 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f62e8144800 0x7f62e815fa90 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:26.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.688+0000 7f62ed400640 1 -- 172.21.15.131:0/2640178266 >> 172.21.15.131:0/2640178266 conn(0x7f62e80cff40 msgr2=0x7f62e80d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:26.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.688+0000 7f62ed400640 1 -- 172.21.15.131:0/2640178266 shutdown_connections 2024-05-26T22:48:26.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.688+0000 7f62ed400640 1 -- 172.21.15.131:0/2640178266 wait complete. 2024-05-26T22:48:26.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.690+0000 7f62ed400640 1 Processor -- start 2024-05-26T22:48:26.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.690+0000 7f62ed400640 1 -- start start 2024-05-26T22:48:26.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.691+0000 7f62ed400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f62e80ad870 0x7f62e8158840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:26.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.691+0000 7f62ed400640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f62e8144800 0x7f62e815b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:26.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.691+0000 7f62ed400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f62e815e9f0 0x7f62e815b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:26.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.691+0000 7f62ed400640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f62e80ce400 con 0x7f62e815e9f0 2024-05-26T22:48:26.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.691+0000 7f62ed400640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f62e815ffd0 con 0x7f62e8144800 2024-05-26T22:48:26.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.691+0000 7f62ed400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f62e8144be0 con 0x7f62e80ad870 2024-05-26T22:48:26.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.691+0000 7f62eca00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f62e815e9f0 0x7f62e815b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:26.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.691+0000 7f62eca00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f62e815e9f0 0x7f62e815b7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53888/0 (socket says 172.21.15.131:53888) 2024-05-26T22:48:26.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.691+0000 7f62eca00640 1 -- 172.21.15.131:0/2193859846 learned_addr learned my addr 172.21.15.131:0/2193859846 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:26.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.691+0000 7f62dfe00640 1 --2- 172.21.15.131:0/2193859846 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f62e8144800 0x7f62e815b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:26.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.691+0000 7f62e7e00640 1 --2- 172.21.15.131:0/2193859846 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f62e80ad870 0x7f62e8158840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:26.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.692+0000 7f62eca00640 1 -- 172.21.15.131:0/2193859846 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f62e80ad870 msgr2=0x7f62e8158840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:26.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.692+0000 7f62eca00640 1 --2- 172.21.15.131:0/2193859846 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f62e80ad870 0x7f62e8158840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:26.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.692+0000 7f62eca00640 1 -- 172.21.15.131:0/2193859846 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f62e8144800 msgr2=0x7f62e815b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:26.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.692+0000 7f62eca00640 1 --2- 172.21.15.131:0/2193859846 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f62e8144800 0x7f62e815b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:26.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.692+0000 7f62eca00640 1 -- 172.21.15.131:0/2193859846 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62e81f3e90 con 0x7f62e815e9f0 2024-05-26T22:48:26.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.692+0000 7f62eca00640 1 --2- 172.21.15.131:0/2193859846 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f62e815e9f0 0x7f62e815b7c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f62e000c6b0 tx=0x7f62e000cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:26.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.692+0000 7f62e4c00640 1 -- 172.21.15.131:0/2193859846 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e0003b70 con 0x7f62e815e9f0 2024-05-26T22:48:26.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.692+0000 7f62e4c00640 1 -- 172.21.15.131:0/2193859846 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62e0003d30 con 0x7f62e815e9f0 2024-05-26T22:48:26.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.692+0000 7f62ed400640 1 -- 172.21.15.131:0/2193859846 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62e81f4170 con 0x7f62e815e9f0 2024-05-26T22:48:26.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.692+0000 7f62e4c00640 1 -- 172.21.15.131:0/2193859846 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e0005310 con 0x7f62e815e9f0 2024-05-26T22:48:26.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.692+0000 7f62ed400640 1 -- 172.21.15.131:0/2193859846 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62e81f4730 con 0x7f62e815e9f0 2024-05-26T22:48:26.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.693+0000 7f62ed400640 1 -- 172.21.15.131:0/2193859846 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62e80694e0 con 0x7f62e815e9f0 2024-05-26T22:48:26.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.694+0000 7f62e4c00640 1 -- 172.21.15.131:0/2193859846 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f62e00054b0 con 0x7f62e815e9f0 2024-05-26T22:48:26.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.695+0000 7f62e4c00640 1 --2- 172.21.15.131:0/2193859846 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f62c406be30 0x7f62c406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:26.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.695+0000 7f62e4c00640 1 -- 172.21.15.131:0/2193859846 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f62e008ac80 con 0x7f62e815e9f0 2024-05-26T22:48:26.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.695+0000 7f62e7e00640 1 --2- 172.21.15.131:0/2193859846 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f62c406be30 0x7f62c406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:26.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.696+0000 7f62e7e00640 1 --2- 172.21.15.131:0/2193859846 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f62c406be30 0x7f62c406e2f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f62d8034000 tx=0x7f62d802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:26.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.699+0000 7f62e4c00640 1 -- 172.21.15.131:0/2193859846 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f62e00596c0 con 0x7f62e815e9f0 2024-05-26T22:48:26.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.896+0000 7f62ed400640 1 -- 172.21.15.131:0/2193859846 --> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f62e806c320 con 0x7f62c406be30 2024-05-26T22:48:26.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.908+0000 7f62e4c00640 1 -- 172.21.15.131:0/2193859846 <== mgr.4104 v2:172.21.15.131:6832/3484832947 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251947 (secure 0 0 0) 0x7f62e806c320 con 0x7f62c406be30 2024-05-26T22:48:26.909 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:48:26.912 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-05-26T22:48:26.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.914+0000 7f62ed400640 1 -- 172.21.15.131:0/2193859846 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f62c406be30 msgr2=0x7f62c406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:26.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.914+0000 7f62ed400640 1 --2- 172.21.15.131:0/2193859846 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f62c406be30 0x7f62c406e2f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f62d8034000 tx=0x7f62d802e000 comp rx=0 tx=0).stop 2024-05-26T22:48:26.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.914+0000 7f62ed400640 1 -- 172.21.15.131:0/2193859846 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f62e815e9f0 msgr2=0x7f62e815b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:26.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.914+0000 7f62ed400640 1 --2- 172.21.15.131:0/2193859846 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f62e815e9f0 0x7f62e815b7c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f62e000c6b0 tx=0x7f62e000cb70 comp rx=0 tx=0).stop 2024-05-26T22:48:26.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.915+0000 7f62ed400640 1 -- 172.21.15.131:0/2193859846 shutdown_connections 2024-05-26T22:48:26.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.915+0000 7f62ed400640 1 --2- 172.21.15.131:0/2193859846 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f62c406be30 0x7f62c406e2f0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:26.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.915+0000 7f62ed400640 1 --2- 172.21.15.131:0/2193859846 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f62e815e9f0 0x7f62e815b7c0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:26.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.915+0000 7f62ed400640 1 --2- 172.21.15.131:0/2193859846 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f62e8144800 0x7f62e815b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:26.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.915+0000 7f62ed400640 1 --2- 172.21.15.131:0/2193859846 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f62e80ad870 0x7f62e8158840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:26.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.915+0000 7f62ed400640 1 -- 172.21.15.131:0/2193859846 >> 172.21.15.131:0/2193859846 conn(0x7f62e80cff40 msgr2=0x7f62e80ce770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:26.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.915+0000 7f62ed400640 1 -- 172.21.15.131:0/2193859846 shutdown_connections 2024-05-26T22:48:26.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:26.915+0000 7f62ed400640 1 -- 172.21.15.131:0/2193859846 wait complete. 2024-05-26T22:48:26.933 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-05-26T22:48:26.384454+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":157,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218992,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755728,"statfs":{"total":773094113280,"available":772869865472,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":39,"apply_latency_ms":39,"commit_latency_ns":39000000,"apply_latency_ns":39000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002054"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.509427+0000","last_change":"2024-05-26T22:48:21.831991+0000","last_active":"2024-05-26T22:48:23.509427+0000","last_peered":"2024-05-26T22:48:23.509427+0000","last_clean":"2024-05-26T22:48:23.509427+0000","last_became_active":"2024-05-26T22:48:21.831657+0000","last_became_peered":"2024-05-26T22:48:21.831657+0000","last_unstale":"2024-05-26T22:48:23.509427+0000","last_undegraded":"2024-05-26T22:48:23.509427+0000","last_fullsized":"2024-05-26T22:48:23.509427+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:30:45.689344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.822991+0000","last_change":"2024-05-26T22:48:21.813275+0000","last_active":"2024-05-26T22:48:22.822991+0000","last_peered":"2024-05-26T22:48:22.822991+0000","last_clean":"2024-05-26T22:48:22.822991+0000","last_became_active":"2024-05-26T22:48:21.812895+0000","last_became_peered":"2024-05-26T22:48:21.812895+0000","last_unstale":"2024-05-26T22:48:22.822991+0000","last_undegraded":"2024-05-26T22:48:22.822991+0000","last_fullsized":"2024-05-26T22:48:22.822991+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:59:43.907681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.809083+0000","last_change":"2024-05-26T22:48:21.805790+0000","last_active":"2024-05-26T22:48:22.809083+0000","last_peered":"2024-05-26T22:48:22.809083+0000","last_clean":"2024-05-26T22:48:22.809083+0000","last_became_active":"2024-05-26T22:48:21.805318+0000","last_became_peered":"2024-05-26T22:48:21.805318+0000","last_unstale":"2024-05-26T22:48:22.809083+0000","last_undegraded":"2024-05-26T22:48:22.809083+0000","last_fullsized":"2024-05-26T22:48:22.809083+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:43:21.391197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.506425+0000","last_change":"2024-05-26T22:48:21.831837+0000","last_active":"2024-05-26T22:48:23.506425+0000","last_peered":"2024-05-26T22:48:23.506425+0000","last_clean":"2024-05-26T22:48:23.506425+0000","last_became_active":"2024-05-26T22:48:21.829461+0000","last_became_peered":"2024-05-26T22:48:21.829461+0000","last_unstale":"2024-05-26T22:48:23.506425+0000","last_undegraded":"2024-05-26T22:48:23.506425+0000","last_fullsized":"2024-05-26T22:48:23.506425+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:56:39.208963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.508007+0000","last_change":"2024-05-26T22:48:21.821926+0000","last_active":"2024-05-26T22:48:23.508007+0000","last_peered":"2024-05-26T22:48:23.508007+0000","last_clean":"2024-05-26T22:48:23.508007+0000","last_became_active":"2024-05-26T22:48:21.821594+0000","last_became_peered":"2024-05-26T22:48:21.821594+0000","last_unstale":"2024-05-26T22:48:23.508007+0000","last_undegraded":"2024-05-26T22:48:23.508007+0000","last_fullsized":"2024-05-26T22:48:23.508007+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:16:22.847663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.506340+0000","last_change":"2024-05-26T22:48:21.823954+0000","last_active":"2024-05-26T22:48:23.506340+0000","last_peered":"2024-05-26T22:48:23.506340+0000","last_clean":"2024-05-26T22:48:23.506340+0000","last_became_active":"2024-05-26T22:48:21.821405+0000","last_became_peered":"2024-05-26T22:48:21.821405+0000","last_unstale":"2024-05-26T22:48:23.506340+0000","last_undegraded":"2024-05-26T22:48:23.506340+0000","last_fullsized":"2024-05-26T22:48:23.506340+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:33:39.923721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.506938+0000","last_change":"2024-05-26T22:48:21.821415+0000","last_active":"2024-05-26T22:48:23.506938+0000","last_peered":"2024-05-26T22:48:23.506938+0000","last_clean":"2024-05-26T22:48:23.506938+0000","last_became_active":"2024-05-26T22:48:21.821092+0000","last_became_peered":"2024-05-26T22:48:21.821092+0000","last_unstale":"2024-05-26T22:48:23.506938+0000","last_undegraded":"2024-05-26T22:48:23.506938+0000","last_fullsized":"2024-05-26T22:48:23.506938+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:56:39.208963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.811969+0000","last_change":"2024-05-26T22:48:21.847352+0000","last_active":"2024-05-26T22:48:22.811969+0000","last_peered":"2024-05-26T22:48:22.811969+0000","last_clean":"2024-05-26T22:48:22.811969+0000","last_became_active":"2024-05-26T22:48:21.846965+0000","last_became_peered":"2024-05-26T22:48:21.846965+0000","last_unstale":"2024-05-26T22:48:22.811969+0000","last_undegraded":"2024-05-26T22:48:22.811969+0000","last_fullsized":"2024-05-26T22:48:22.811969+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T23:40:55.485829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.509545+0000","last_change":"2024-05-26T22:48:21.847421+0000","last_active":"2024-05-26T22:48:23.509545+0000","last_peered":"2024-05-26T22:48:23.509545+0000","last_clean":"2024-05-26T22:48:23.509545+0000","last_became_active":"2024-05-26T22:48:21.846702+0000","last_became_peered":"2024-05-26T22:48:21.846702+0000","last_unstale":"2024-05-26T22:48:23.509545+0000","last_undegraded":"2024-05-26T22:48:23.509545+0000","last_fullsized":"2024-05-26T22:48:23.509545+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:27:35.210564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.810240+0000","last_change":"2024-05-26T22:48:21.831244+0000","last_active":"2024-05-26T22:48:22.810240+0000","last_peered":"2024-05-26T22:48:22.810240+0000","last_clean":"2024-05-26T22:48:22.810240+0000","last_became_active":"2024-05-26T22:48:21.830949+0000","last_became_peered":"2024-05-26T22:48:21.830949+0000","last_unstale":"2024-05-26T22:48:22.810240+0000","last_undegraded":"2024-05-26T22:48:22.810240+0000","last_fullsized":"2024-05-26T22:48:22.810240+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:59:43.907681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.810220+0000","last_change":"2024-05-26T22:48:21.836749+0000","last_active":"2024-05-26T22:48:22.810220+0000","last_peered":"2024-05-26T22:48:22.810220+0000","last_clean":"2024-05-26T22:48:22.810220+0000","last_became_active":"2024-05-26T22:48:21.836431+0000","last_became_peered":"2024-05-26T22:48:21.836431+0000","last_unstale":"2024-05-26T22:48:22.810220+0000","last_undegraded":"2024-05-26T22:48:22.810220+0000","last_fullsized":"2024-05-26T22:48:22.810220+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:38:00.344955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.505172+0000","last_change":"2024-05-26T22:48:21.819214+0000","last_active":"2024-05-26T22:48:23.505172+0000","last_peered":"2024-05-26T22:48:23.505172+0000","last_clean":"2024-05-26T22:48:23.505172+0000","last_became_active":"2024-05-26T22:48:21.818837+0000","last_became_peered":"2024-05-26T22:48:21.818837+0000","last_unstale":"2024-05-26T22:48:23.505172+0000","last_undegraded":"2024-05-26T22:48:23.505172+0000","last_fullsized":"2024-05-26T22:48:23.505172+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:42:32.978709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.812261+0000","last_change":"2024-05-26T22:48:21.830442+0000","last_active":"2024-05-26T22:48:22.812261+0000","last_peered":"2024-05-26T22:48:22.812261+0000","last_clean":"2024-05-26T22:48:22.812261+0000","last_became_active":"2024-05-26T22:48:21.830132+0000","last_became_peered":"2024-05-26T22:48:21.830132+0000","last_unstale":"2024-05-26T22:48:22.812261+0000","last_undegraded":"2024-05-26T22:48:22.812261+0000","last_fullsized":"2024-05-26T22:48:22.812261+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:43:21.391197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.808630+0000","last_change":"2024-05-26T22:48:21.818032+0000","last_active":"2024-05-26T22:48:22.808630+0000","last_peered":"2024-05-26T22:48:22.808630+0000","last_clean":"2024-05-26T22:48:22.808630+0000","last_became_active":"2024-05-26T22:48:21.817717+0000","last_became_peered":"2024-05-26T22:48:21.817717+0000","last_unstale":"2024-05-26T22:48:22.808630+0000","last_undegraded":"2024-05-26T22:48:22.808630+0000","last_fullsized":"2024-05-26T22:48:22.808630+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:27:34.959431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.822672+0000","last_change":"2024-05-26T22:48:21.823839+0000","last_active":"2024-05-26T22:48:22.822672+0000","last_peered":"2024-05-26T22:48:22.822672+0000","last_clean":"2024-05-26T22:48:22.822672+0000","last_became_active":"2024-05-26T22:48:21.823522+0000","last_became_peered":"2024-05-26T22:48:21.823522+0000","last_unstale":"2024-05-26T22:48:22.822672+0000","last_undegraded":"2024-05-26T22:48:22.822672+0000","last_fullsized":"2024-05-26T22:48:22.822672+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:52:25.073281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.504208+0000","last_change":"2024-05-26T22:48:21.819432+0000","last_active":"2024-05-26T22:48:23.504208+0000","last_peered":"2024-05-26T22:48:23.504208+0000","last_clean":"2024-05-26T22:48:23.504208+0000","last_became_active":"2024-05-26T22:48:21.819139+0000","last_became_peered":"2024-05-26T22:48:21.819139+0000","last_unstale":"2024-05-26T22:48:23.504208+0000","last_undegraded":"2024-05-26T22:48:23.504208+0000","last_fullsized":"2024-05-26T22:48:23.504208+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:21:52.108732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.507463+0000","last_change":"2024-05-26T22:48:21.842657+0000","last_active":"2024-05-26T22:48:23.507463+0000","last_peered":"2024-05-26T22:48:23.507463+0000","last_clean":"2024-05-26T22:48:23.507463+0000","last_became_active":"2024-05-26T22:48:21.842312+0000","last_became_peered":"2024-05-26T22:48:21.842312+0000","last_unstale":"2024-05-26T22:48:23.507463+0000","last_undegraded":"2024-05-26T22:48:23.507463+0000","last_fullsized":"2024-05-26T22:48:23.507463+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:49:40.791149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.503319+0000","last_change":"2024-05-26T22:48:21.812988+0000","last_active":"2024-05-26T22:48:23.503319+0000","last_peered":"2024-05-26T22:48:23.503319+0000","last_clean":"2024-05-26T22:48:23.503319+0000","last_became_active":"2024-05-26T22:48:21.812669+0000","last_became_peered":"2024-05-26T22:48:21.812669+0000","last_unstale":"2024-05-26T22:48:23.503319+0000","last_undegraded":"2024-05-26T22:48:23.503319+0000","last_fullsized":"2024-05-26T22:48:23.503319+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:16:02.119875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.509267+0000","last_change":"2024-05-26T22:48:21.830493+0000","last_active":"2024-05-26T22:48:23.509267+0000","last_peered":"2024-05-26T22:48:23.509267+0000","last_clean":"2024-05-26T22:48:23.509267+0000","last_became_active":"2024-05-26T22:48:21.830116+0000","last_became_peered":"2024-05-26T22:48:21.830116+0000","last_unstale":"2024-05-26T22:48:23.509267+0000","last_undegraded":"2024-05-26T22:48:23.509267+0000","last_fullsized":"2024-05-26T22:48:23.509267+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:45:06.922028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.809824+0000","last_change":"2024-05-26T22:48:21.829461+0000","last_active":"2024-05-26T22:48:22.809824+0000","last_peered":"2024-05-26T22:48:22.809824+0000","last_clean":"2024-05-26T22:48:22.809824+0000","last_became_active":"2024-05-26T22:48:21.829132+0000","last_became_peered":"2024-05-26T22:48:21.829132+0000","last_unstale":"2024-05-26T22:48:22.809824+0000","last_undegraded":"2024-05-26T22:48:22.809824+0000","last_fullsized":"2024-05-26T22:48:22.809824+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:52:25.073281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.809106+0000","last_change":"2024-05-26T22:48:21.818888+0000","last_active":"2024-05-26T22:48:22.809106+0000","last_peered":"2024-05-26T22:48:22.809106+0000","last_clean":"2024-05-26T22:48:22.809106+0000","last_became_active":"2024-05-26T22:48:21.818639+0000","last_became_peered":"2024-05-26T22:48:21.818639+0000","last_unstale":"2024-05-26T22:48:22.809106+0000","last_undegraded":"2024-05-26T22:48:22.809106+0000","last_fullsized":"2024-05-26T22:48:22.809106+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:38:00.344955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.506778+0000","last_change":"2024-05-26T22:48:21.842585+0000","last_active":"2024-05-26T22:48:23.506778+0000","last_peered":"2024-05-26T22:48:23.506778+0000","last_clean":"2024-05-26T22:48:23.506778+0000","last_became_active":"2024-05-26T22:48:21.842302+0000","last_became_peered":"2024-05-26T22:48:21.842302+0000","last_unstale":"2024-05-26T22:48:23.506778+0000","last_undegraded":"2024-05-26T22:48:23.506778+0000","last_fullsized":"2024-05-26T22:48:23.506778+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:08:10.330236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.811982+0000","last_change":"2024-05-26T22:48:21.830936+0000","last_active":"2024-05-26T22:48:22.811982+0000","last_peered":"2024-05-26T22:48:22.811982+0000","last_clean":"2024-05-26T22:48:22.811982+0000","last_became_active":"2024-05-26T22:48:21.830551+0000","last_became_peered":"2024-05-26T22:48:21.830551+0000","last_unstale":"2024-05-26T22:48:22.811982+0000","last_undegraded":"2024-05-26T22:48:22.811982+0000","last_fullsized":"2024-05-26T22:48:22.811982+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:27:34.959431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.508425+0000","last_change":"2024-05-26T22:48:21.837001+0000","last_active":"2024-05-26T22:48:23.508425+0000","last_peered":"2024-05-26T22:48:23.508425+0000","last_clean":"2024-05-26T22:48:23.508425+0000","last_became_active":"2024-05-26T22:48:21.836749+0000","last_became_peered":"2024-05-26T22:48:21.836749+0000","last_unstale":"2024-05-26T22:48:23.508425+0000","last_undegraded":"2024-05-26T22:48:23.508425+0000","last_fullsized":"2024-05-26T22:48:23.508425+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:30:45.689344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.812885+0000","last_change":"2024-05-26T22:48:21.834986+0000","last_active":"2024-05-26T22:48:22.812885+0000","last_peered":"2024-05-26T22:48:22.812885+0000","last_clean":"2024-05-26T22:48:22.812885+0000","last_became_active":"2024-05-26T22:48:21.834572+0000","last_became_peered":"2024-05-26T22:48:21.834572+0000","last_unstale":"2024-05-26T22:48:22.812885+0000","last_undegraded":"2024-05-26T22:48:22.812885+0000","last_fullsized":"2024-05-26T22:48:22.812885+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:33:39.923721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.810409+0000","last_change":"2024-05-26T22:48:21.824957+0000","last_active":"2024-05-26T22:48:22.810409+0000","last_peered":"2024-05-26T22:48:22.810409+0000","last_clean":"2024-05-26T22:48:22.810409+0000","last_became_active":"2024-05-26T22:48:21.822871+0000","last_became_peered":"2024-05-26T22:48:21.822871+0000","last_unstale":"2024-05-26T22:48:22.810409+0000","last_undegraded":"2024-05-26T22:48:22.810409+0000","last_fullsized":"2024-05-26T22:48:22.810409+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:59:10.543916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.822941+0000","last_change":"2024-05-26T22:48:21.827100+0000","last_active":"2024-05-26T22:48:22.822941+0000","last_peered":"2024-05-26T22:48:22.822941+0000","last_clean":"2024-05-26T22:48:22.822941+0000","last_became_active":"2024-05-26T22:48:21.826810+0000","last_became_peered":"2024-05-26T22:48:21.826810+0000","last_unstale":"2024-05-26T22:48:22.822941+0000","last_undegraded":"2024-05-26T22:48:22.822941+0000","last_fullsized":"2024-05-26T22:48:22.822941+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:38:00.344955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.813201+0000","last_change":"2024-05-26T22:48:21.836236+0000","last_active":"2024-05-26T22:48:22.813201+0000","last_peered":"2024-05-26T22:48:22.813201+0000","last_clean":"2024-05-26T22:48:22.813201+0000","last_became_active":"2024-05-26T22:48:21.834609+0000","last_became_peered":"2024-05-26T22:48:21.834609+0000","last_unstale":"2024-05-26T22:48:22.813201+0000","last_undegraded":"2024-05-26T22:48:22.813201+0000","last_fullsized":"2024-05-26T22:48:22.813201+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:08:10.330236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.809226+0000","last_change":"2024-05-26T22:48:21.821191+0000","last_active":"2024-05-26T22:48:22.809226+0000","last_peered":"2024-05-26T22:48:22.809226+0000","last_clean":"2024-05-26T22:48:22.809226+0000","last_became_active":"2024-05-26T22:48:21.820731+0000","last_became_peered":"2024-05-26T22:48:21.820731+0000","last_unstale":"2024-05-26T22:48:22.809226+0000","last_undegraded":"2024-05-26T22:48:22.809226+0000","last_fullsized":"2024-05-26T22:48:22.809226+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:59:43.907681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.507246+0000","last_change":"2024-05-26T22:48:21.836074+0000","last_active":"2024-05-26T22:48:23.507246+0000","last_peered":"2024-05-26T22:48:23.507246+0000","last_clean":"2024-05-26T22:48:23.507246+0000","last_became_active":"2024-05-26T22:48:21.831547+0000","last_became_peered":"2024-05-26T22:48:21.831547+0000","last_unstale":"2024-05-26T22:48:23.507246+0000","last_undegraded":"2024-05-26T22:48:23.507246+0000","last_fullsized":"2024-05-26T22:48:23.507246+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:34:58.598026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.503056+0000","last_change":"2024-05-26T22:48:21.815510+0000","last_active":"2024-05-26T22:48:23.503056+0000","last_peered":"2024-05-26T22:48:23.503056+0000","last_clean":"2024-05-26T22:48:23.503056+0000","last_became_active":"2024-05-26T22:48:21.815132+0000","last_became_peered":"2024-05-26T22:48:21.815132+0000","last_unstale":"2024-05-26T22:48:23.503056+0000","last_undegraded":"2024-05-26T22:48:23.503056+0000","last_fullsized":"2024-05-26T22:48:23.503056+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:56:00.075959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.809652+0000","last_change":"2024-05-26T22:48:21.811494+0000","last_active":"2024-05-26T22:48:22.809652+0000","last_peered":"2024-05-26T22:48:22.809652+0000","last_clean":"2024-05-26T22:48:22.809652+0000","last_became_active":"2024-05-26T22:48:21.809669+0000","last_became_peered":"2024-05-26T22:48:21.809669+0000","last_unstale":"2024-05-26T22:48:22.809652+0000","last_undegraded":"2024-05-26T22:48:22.809652+0000","last_fullsized":"2024-05-26T22:48:22.809652+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:27:34.959431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.504470+0000","last_change":"2024-05-26T22:48:21.824303+0000","last_active":"2024-05-26T22:48:23.504470+0000","last_peered":"2024-05-26T22:48:23.504470+0000","last_clean":"2024-05-26T22:48:23.504470+0000","last_became_active":"2024-05-26T22:48:21.823500+0000","last_became_peered":"2024-05-26T22:48:21.823500+0000","last_unstale":"2024-05-26T22:48:23.504470+0000","last_undegraded":"2024-05-26T22:48:23.504470+0000","last_fullsized":"2024-05-26T22:48:23.504470+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:16:02.119875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.812241+0000","last_change":"2024-05-26T22:48:20.802593+0000","last_active":"2024-05-26T22:48:22.812241+0000","last_peered":"2024-05-26T22:48:22.812241+0000","last_clean":"2024-05-26T22:48:22.812241+0000","last_became_active":"2024-05-26T22:48:20.802352+0000","last_became_peered":"2024-05-26T22:48:20.802352+0000","last_unstale":"2024-05-26T22:48:22.812241+0000","last_undegraded":"2024-05-26T22:48:22.812241+0000","last_fullsized":"2024-05-26T22:48:22.812241+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:39:51.267408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.809443+0000","last_change":"2024-05-26T22:48:21.832952+0000","last_active":"2024-05-26T22:48:22.809443+0000","last_peered":"2024-05-26T22:48:22.809443+0000","last_clean":"2024-05-26T22:48:22.809443+0000","last_became_active":"2024-05-26T22:48:21.832496+0000","last_became_peered":"2024-05-26T22:48:21.832496+0000","last_unstale":"2024-05-26T22:48:22.809443+0000","last_undegraded":"2024-05-26T22:48:22.809443+0000","last_fullsized":"2024-05-26T22:48:22.809443+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:59:10.543916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.810033+0000","last_change":"2024-05-26T22:48:20.789223+0000","last_active":"2024-05-26T22:48:22.810033+0000","last_peered":"2024-05-26T22:48:22.810033+0000","last_clean":"2024-05-26T22:48:22.810033+0000","last_became_active":"2024-05-26T22:48:20.787967+0000","last_became_peered":"2024-05-26T22:48:20.787967+0000","last_unstale":"2024-05-26T22:48:22.810033+0000","last_undegraded":"2024-05-26T22:48:22.810033+0000","last_fullsized":"2024-05-26T22:48:22.810033+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:08:01.698812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.812657+0000","last_change":"2024-05-26T22:48:20.784257+0000","last_active":"2024-05-26T22:48:22.812657+0000","last_peered":"2024-05-26T22:48:22.812657+0000","last_clean":"2024-05-26T22:48:22.812657+0000","last_became_active":"2024-05-26T22:48:20.783493+0000","last_became_peered":"2024-05-26T22:48:20.783493+0000","last_unstale":"2024-05-26T22:48:22.812657+0000","last_undegraded":"2024-05-26T22:48:22.812657+0000","last_fullsized":"2024-05-26T22:48:22.812657+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:30:48.024402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.813086+0000","last_change":"2024-05-26T22:48:21.834733+0000","last_active":"2024-05-26T22:48:22.813086+0000","last_peered":"2024-05-26T22:48:22.813086+0000","last_clean":"2024-05-26T22:48:22.813086+0000","last_became_active":"2024-05-26T22:48:21.834396+0000","last_became_peered":"2024-05-26T22:48:21.834396+0000","last_unstale":"2024-05-26T22:48:22.813086+0000","last_undegraded":"2024-05-26T22:48:22.813086+0000","last_fullsized":"2024-05-26T22:48:22.813086+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:56:39.208963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.506282+0000","last_change":"2024-05-26T22:48:20.798521+0000","last_active":"2024-05-26T22:48:23.506282+0000","last_peered":"2024-05-26T22:48:23.506282+0000","last_clean":"2024-05-26T22:48:23.506282+0000","last_became_active":"2024-05-26T22:48:20.798153+0000","last_became_peered":"2024-05-26T22:48:20.798153+0000","last_unstale":"2024-05-26T22:48:23.506282+0000","last_undegraded":"2024-05-26T22:48:23.506282+0000","last_fullsized":"2024-05-26T22:48:23.506282+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:10:49.991129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.823065+0000","last_change":"2024-05-26T22:48:21.832951+0000","last_active":"2024-05-26T22:48:22.823065+0000","last_peered":"2024-05-26T22:48:22.823065+0000","last_clean":"2024-05-26T22:48:22.823065+0000","last_became_active":"2024-05-26T22:48:21.831917+0000","last_became_peered":"2024-05-26T22:48:21.831917+0000","last_unstale":"2024-05-26T22:48:22.823065+0000","last_undegraded":"2024-05-26T22:48:22.823065+0000","last_fullsized":"2024-05-26T22:48:22.823065+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:59:10.543916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.810123+0000","last_change":"2024-05-26T22:48:20.801167+0000","last_active":"2024-05-26T22:48:22.810123+0000","last_peered":"2024-05-26T22:48:22.810123+0000","last_clean":"2024-05-26T22:48:22.810123+0000","last_became_active":"2024-05-26T22:48:20.800854+0000","last_became_peered":"2024-05-26T22:48:20.800854+0000","last_unstale":"2024-05-26T22:48:22.810123+0000","last_undegraded":"2024-05-26T22:48:22.810123+0000","last_fullsized":"2024-05-26T22:48:22.810123+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:39:51.267408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.509643+0000","last_change":"2024-05-26T22:48:21.841974+0000","last_active":"2024-05-26T22:48:23.509643+0000","last_peered":"2024-05-26T22:48:23.509643+0000","last_clean":"2024-05-26T22:48:23.509643+0000","last_became_active":"2024-05-26T22:48:21.840912+0000","last_became_peered":"2024-05-26T22:48:21.840912+0000","last_unstale":"2024-05-26T22:48:23.509643+0000","last_undegraded":"2024-05-26T22:48:23.509643+0000","last_fullsized":"2024-05-26T22:48:23.509643+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:27:35.210564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.509713+0000","last_change":"2024-05-26T22:48:20.798072+0000","last_active":"2024-05-26T22:48:23.509713+0000","last_peered":"2024-05-26T22:48:23.509713+0000","last_clean":"2024-05-26T22:48:23.509713+0000","last_became_active":"2024-05-26T22:48:20.797559+0000","last_became_peered":"2024-05-26T22:48:20.797559+0000","last_unstale":"2024-05-26T22:48:23.509713+0000","last_undegraded":"2024-05-26T22:48:23.509713+0000","last_fullsized":"2024-05-26T22:48:23.509713+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:52:24.060261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.809896+0000","last_change":"2024-05-26T22:48:21.838154+0000","last_active":"2024-05-26T22:48:22.809896+0000","last_peered":"2024-05-26T22:48:22.809896+0000","last_clean":"2024-05-26T22:48:22.809896+0000","last_became_active":"2024-05-26T22:48:21.837073+0000","last_became_peered":"2024-05-26T22:48:21.837073+0000","last_unstale":"2024-05-26T22:48:22.809896+0000","last_undegraded":"2024-05-26T22:48:22.809896+0000","last_fullsized":"2024-05-26T22:48:22.809896+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:56:39.208963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.822642+0000","last_change":"2024-05-26T22:48:20.796108+0000","last_active":"2024-05-26T22:48:22.822642+0000","last_peered":"2024-05-26T22:48:22.822642+0000","last_clean":"2024-05-26T22:48:22.822642+0000","last_became_active":"2024-05-26T22:48:20.795717+0000","last_became_peered":"2024-05-26T22:48:20.795717+0000","last_unstale":"2024-05-26T22:48:22.822642+0000","last_undegraded":"2024-05-26T22:48:22.822642+0000","last_fullsized":"2024-05-26T22:48:22.822642+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:39:51.267408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.505587+0000","last_change":"2024-05-26T22:48:21.823519+0000","last_active":"2024-05-26T22:48:23.505587+0000","last_peered":"2024-05-26T22:48:23.505587+0000","last_clean":"2024-05-26T22:48:23.505587+0000","last_became_active":"2024-05-26T22:48:21.823240+0000","last_became_peered":"2024-05-26T22:48:21.823240+0000","last_unstale":"2024-05-26T22:48:23.505587+0000","last_undegraded":"2024-05-26T22:48:23.505587+0000","last_fullsized":"2024-05-26T22:48:23.505587+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:33:51.651353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.809592+0000","last_change":"2024-05-26T22:48:20.792915+0000","last_active":"2024-05-26T22:48:22.809592+0000","last_peered":"2024-05-26T22:48:22.809592+0000","last_clean":"2024-05-26T22:48:22.809592+0000","last_became_active":"2024-05-26T22:48:20.792037+0000","last_became_peered":"2024-05-26T22:48:20.792037+0000","last_unstale":"2024-05-26T22:48:22.809592+0000","last_undegraded":"2024-05-26T22:48:22.809592+0000","last_fullsized":"2024-05-26T22:48:22.809592+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:11:29.128727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.508563+0000","last_change":"2024-05-26T22:48:21.839574+0000","last_active":"2024-05-26T22:48:23.508563+0000","last_peered":"2024-05-26T22:48:23.508563+0000","last_clean":"2024-05-26T22:48:23.508563+0000","last_became_active":"2024-05-26T22:48:21.834905+0000","last_became_peered":"2024-05-26T22:48:21.834905+0000","last_unstale":"2024-05-26T22:48:23.508563+0000","last_undegraded":"2024-05-26T22:48:23.508563+0000","last_fullsized":"2024-05-26T22:48:23.508563+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:34:58.598026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.809371+0000","last_change":"2024-05-26T22:48:20.792287+0000","last_active":"2024-05-26T22:48:22.809371+0000","last_peered":"2024-05-26T22:48:22.809371+0000","last_clean":"2024-05-26T22:48:22.809371+0000","last_became_active":"2024-05-26T22:48:20.791592+0000","last_became_peered":"2024-05-26T22:48:20.791592+0000","last_unstale":"2024-05-26T22:48:22.809371+0000","last_undegraded":"2024-05-26T22:48:22.809371+0000","last_fullsized":"2024-05-26T22:48:22.809371+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:30:48.024402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.808872+0000","last_change":"2024-05-26T22:48:21.824723+0000","last_active":"2024-05-26T22:48:22.808872+0000","last_peered":"2024-05-26T22:48:22.808872+0000","last_clean":"2024-05-26T22:48:22.808872+0000","last_became_active":"2024-05-26T22:48:21.824372+0000","last_became_peered":"2024-05-26T22:48:21.824372+0000","last_unstale":"2024-05-26T22:48:22.808872+0000","last_undegraded":"2024-05-26T22:48:22.808872+0000","last_fullsized":"2024-05-26T22:48:22.808872+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:52:25.073281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.507397+0000","last_change":"2024-05-26T22:48:20.802334+0000","last_active":"2024-05-26T22:48:23.507397+0000","last_peered":"2024-05-26T22:48:23.507397+0000","last_clean":"2024-05-26T22:48:23.507397+0000","last_became_active":"2024-05-26T22:48:20.802051+0000","last_became_peered":"2024-05-26T22:48:20.802051+0000","last_unstale":"2024-05-26T22:48:23.507397+0000","last_undegraded":"2024-05-26T22:48:23.507397+0000","last_fullsized":"2024-05-26T22:48:23.507397+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:55:59.062939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.508279+0000","last_change":"2024-05-26T22:48:21.830682+0000","last_active":"2024-05-26T22:48:23.508279+0000","last_peered":"2024-05-26T22:48:23.508279+0000","last_clean":"2024-05-26T22:48:23.508279+0000","last_became_active":"2024-05-26T22:48:21.830273+0000","last_became_peered":"2024-05-26T22:48:21.830273+0000","last_unstale":"2024-05-26T22:48:23.508279+0000","last_undegraded":"2024-05-26T22:48:23.508279+0000","last_fullsized":"2024-05-26T22:48:23.508279+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:15:00.790746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.809215+0000","last_change":"2024-05-26T22:48:20.791028+0000","last_active":"2024-05-26T22:48:22.809215+0000","last_peered":"2024-05-26T22:48:22.809215+0000","last_clean":"2024-05-26T22:48:22.809215+0000","last_became_active":"2024-05-26T22:48:20.790727+0000","last_became_peered":"2024-05-26T22:48:20.790727+0000","last_unstale":"2024-05-26T22:48:22.809215+0000","last_undegraded":"2024-05-26T22:48:22.809215+0000","last_fullsized":"2024-05-26T22:48:22.809215+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:20:41.886676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.809552+0000","last_change":"2024-05-26T22:48:21.809785+0000","last_active":"2024-05-26T22:48:22.809552+0000","last_peered":"2024-05-26T22:48:22.809552+0000","last_clean":"2024-05-26T22:48:22.809552+0000","last_became_active":"2024-05-26T22:48:21.809460+0000","last_became_peered":"2024-05-26T22:48:21.809460+0000","last_unstale":"2024-05-26T22:48:22.809552+0000","last_undegraded":"2024-05-26T22:48:22.809552+0000","last_fullsized":"2024-05-26T22:48:22.809552+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T23:40:55.485829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.502702+0000","last_change":"2024-05-26T22:48:20.788462+0000","last_active":"2024-05-26T22:48:23.502702+0000","last_peered":"2024-05-26T22:48:23.502702+0000","last_clean":"2024-05-26T22:48:23.502702+0000","last_became_active":"2024-05-26T22:48:20.787714+0000","last_became_peered":"2024-05-26T22:48:20.787714+0000","last_unstale":"2024-05-26T22:48:23.502702+0000","last_undegraded":"2024-05-26T22:48:23.502702+0000","last_fullsized":"2024-05-26T22:48:23.502702+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:08:01.698812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.507359+0000","last_change":"2024-05-26T22:48:21.823217+0000","last_active":"2024-05-26T22:48:23.507359+0000","last_peered":"2024-05-26T22:48:23.507359+0000","last_clean":"2024-05-26T22:48:23.507359+0000","last_became_active":"2024-05-26T22:48:21.822856+0000","last_became_peered":"2024-05-26T22:48:21.822856+0000","last_unstale":"2024-05-26T22:48:23.507359+0000","last_undegraded":"2024-05-26T22:48:23.507359+0000","last_fullsized":"2024-05-26T22:48:23.507359+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:08:10.330236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.505497+0000","last_change":"2024-05-26T22:48:20.793786+0000","last_active":"2024-05-26T22:48:23.505497+0000","last_peered":"2024-05-26T22:48:23.505497+0000","last_clean":"2024-05-26T22:48:23.505497+0000","last_became_active":"2024-05-26T22:48:20.793201+0000","last_became_peered":"2024-05-26T22:48:20.793201+0000","last_unstale":"2024-05-26T22:48:23.505497+0000","last_undegraded":"2024-05-26T22:48:23.505497+0000","last_fullsized":"2024-05-26T22:48:23.505497+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:30:48.024402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.509387+0000","last_change":"2024-05-26T22:48:21.832244+0000","last_active":"2024-05-26T22:48:23.509387+0000","last_peered":"2024-05-26T22:48:23.509387+0000","last_clean":"2024-05-26T22:48:23.509387+0000","last_became_active":"2024-05-26T22:48:21.831897+0000","last_became_peered":"2024-05-26T22:48:21.831897+0000","last_unstale":"2024-05-26T22:48:23.509387+0000","last_undegraded":"2024-05-26T22:48:23.509387+0000","last_fullsized":"2024-05-26T22:48:23.509387+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:49:40.791149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.503336+0000","last_change":"2024-05-26T22:48:20.791496+0000","last_active":"2024-05-26T22:48:23.503336+0000","last_peered":"2024-05-26T22:48:23.503336+0000","last_clean":"2024-05-26T22:48:23.503336+0000","last_became_active":"2024-05-26T22:48:20.790743+0000","last_became_peered":"2024-05-26T22:48:20.790743+0000","last_unstale":"2024-05-26T22:48:23.503336+0000","last_undegraded":"2024-05-26T22:48:23.503336+0000","last_fullsized":"2024-05-26T22:48:23.503336+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:20:41.886676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.509655+0000","last_change":"2024-05-26T22:48:21.837602+0000","last_active":"2024-05-26T22:48:23.509655+0000","last_peered":"2024-05-26T22:48:23.509655+0000","last_clean":"2024-05-26T22:48:23.509655+0000","last_became_active":"2024-05-26T22:48:21.834873+0000","last_became_peered":"2024-05-26T22:48:21.834873+0000","last_unstale":"2024-05-26T22:48:23.509655+0000","last_undegraded":"2024-05-26T22:48:23.509655+0000","last_fullsized":"2024-05-26T22:48:23.509655+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:12:50.184011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.822772+0000","last_change":"2024-05-26T22:48:20.782935+0000","last_active":"2024-05-26T22:48:22.822772+0000","last_peered":"2024-05-26T22:48:22.822772+0000","last_clean":"2024-05-26T22:48:22.822772+0000","last_became_active":"2024-05-26T22:48:20.782624+0000","last_became_peered":"2024-05-26T22:48:20.782624+0000","last_unstale":"2024-05-26T22:48:22.822772+0000","last_undegraded":"2024-05-26T22:48:22.822772+0000","last_fullsized":"2024-05-26T22:48:22.822772+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:55:59.062939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.507335+0000","last_change":"2024-05-26T22:48:21.836045+0000","last_active":"2024-05-26T22:48:23.507335+0000","last_peered":"2024-05-26T22:48:23.507335+0000","last_clean":"2024-05-26T22:48:23.507335+0000","last_became_active":"2024-05-26T22:48:21.830070+0000","last_became_peered":"2024-05-26T22:48:21.830070+0000","last_unstale":"2024-05-26T22:48:23.507335+0000","last_undegraded":"2024-05-26T22:48:23.507335+0000","last_fullsized":"2024-05-26T22:48:23.507335+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:11:40.258537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.503246+0000","last_change":"2024-05-26T22:48:20.805086+0000","last_active":"2024-05-26T22:48:23.503246+0000","last_peered":"2024-05-26T22:48:23.503246+0000","last_clean":"2024-05-26T22:48:23.503246+0000","last_became_active":"2024-05-26T22:48:20.804732+0000","last_became_peered":"2024-05-26T22:48:20.804732+0000","last_unstale":"2024-05-26T22:48:23.503246+0000","last_undegraded":"2024-05-26T22:48:23.503246+0000","last_fullsized":"2024-05-26T22:48:23.503246+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:11:29.128727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.509008+0000","last_change":"2024-05-26T22:48:21.840364+0000","last_active":"2024-05-26T22:48:23.509008+0000","last_peered":"2024-05-26T22:48:23.509008+0000","last_clean":"2024-05-26T22:48:23.509008+0000","last_became_active":"2024-05-26T22:48:21.839692+0000","last_became_peered":"2024-05-26T22:48:21.839692+0000","last_unstale":"2024-05-26T22:48:23.509008+0000","last_undegraded":"2024-05-26T22:48:23.509008+0000","last_fullsized":"2024-05-26T22:48:23.509008+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:11:40.258537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.506053+0000","last_change":"2024-05-26T22:48:12.068917+0000","last_active":"2024-05-26T22:48:23.506053+0000","last_peered":"2024-05-26T22:48:23.506053+0000","last_clean":"2024-05-26T22:48:23.506053+0000","last_became_active":"2024-05-26T22:48:12.065934+0000","last_became_peered":"2024-05-26T22:48:12.065934+0000","last_unstale":"2024-05-26T22:48:23.506053+0000","last_undegraded":"2024-05-26T22:48:23.506053+0000","last_fullsized":"2024-05-26T22:48:23.506053+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:10.628122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:10.628122+0000","last_clean_scrub_stamp":"2024-05-26T22:48:10.628122+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:14:58.895911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.506186+0000","last_change":"2024-05-26T22:48:20.800517+0000","last_active":"2024-05-26T22:48:23.506186+0000","last_peered":"2024-05-26T22:48:23.506186+0000","last_clean":"2024-05-26T22:48:23.506186+0000","last_became_active":"2024-05-26T22:48:20.797956+0000","last_became_peered":"2024-05-26T22:48:20.797956+0000","last_unstale":"2024-05-26T22:48:23.506186+0000","last_undegraded":"2024-05-26T22:48:23.506186+0000","last_fullsized":"2024-05-26T22:48:23.506186+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:39:51.267408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.812649+0000","last_change":"2024-05-26T22:48:21.844543+0000","last_active":"2024-05-26T22:48:22.812649+0000","last_peered":"2024-05-26T22:48:22.812649+0000","last_clean":"2024-05-26T22:48:22.812649+0000","last_became_active":"2024-05-26T22:48:21.844272+0000","last_became_peered":"2024-05-26T22:48:21.844272+0000","last_unstale":"2024-05-26T22:48:22.812649+0000","last_undegraded":"2024-05-26T22:48:22.812649+0000","last_fullsized":"2024-05-26T22:48:22.812649+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:59:10.543916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.505812+0000","last_change":"2024-05-26T22:48:20.799727+0000","last_active":"2024-05-26T22:48:23.505812+0000","last_peered":"2024-05-26T22:48:23.505812+0000","last_clean":"2024-05-26T22:48:23.505812+0000","last_became_active":"2024-05-26T22:48:20.799427+0000","last_became_peered":"2024-05-26T22:48:20.799427+0000","last_unstale":"2024-05-26T22:48:23.505812+0000","last_undegraded":"2024-05-26T22:48:23.505812+0000","last_fullsized":"2024-05-26T22:48:23.505812+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:55:59.062939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.808326+0000","last_change":"2024-05-26T22:48:21.832212+0000","last_active":"2024-05-26T22:48:22.808326+0000","last_peered":"2024-05-26T22:48:22.808326+0000","last_clean":"2024-05-26T22:48:22.808326+0000","last_became_active":"2024-05-26T22:48:21.831893+0000","last_became_peered":"2024-05-26T22:48:21.831893+0000","last_unstale":"2024-05-26T22:48:22.808326+0000","last_undegraded":"2024-05-26T22:48:22.808326+0000","last_fullsized":"2024-05-26T22:48:22.808326+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T23:40:55.485829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.502923+0000","last_change":"2024-05-26T22:48:20.785204+0000","last_active":"2024-05-26T22:48:23.502923+0000","last_peered":"2024-05-26T22:48:23.502923+0000","last_clean":"2024-05-26T22:48:23.502923+0000","last_became_active":"2024-05-26T22:48:20.784798+0000","last_became_peered":"2024-05-26T22:48:20.784798+0000","last_unstale":"2024-05-26T22:48:23.502923+0000","last_undegraded":"2024-05-26T22:48:23.502923+0000","last_fullsized":"2024-05-26T22:48:23.502923+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:30:48.024402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.822381+0000","last_change":"2024-05-26T22:48:21.811298+0000","last_active":"2024-05-26T22:48:22.822381+0000","last_peered":"2024-05-26T22:48:22.822381+0000","last_clean":"2024-05-26T22:48:22.822381+0000","last_became_active":"2024-05-26T22:48:21.807981+0000","last_became_peered":"2024-05-26T22:48:21.807981+0000","last_unstale":"2024-05-26T22:48:22.822381+0000","last_undegraded":"2024-05-26T22:48:22.822381+0000","last_fullsized":"2024-05-26T22:48:22.822381+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:27:34.959431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.507652+0000","last_change":"2024-05-26T22:48:20.794578+0000","last_active":"2024-05-26T22:48:23.507652+0000","last_peered":"2024-05-26T22:48:23.507652+0000","last_clean":"2024-05-26T22:48:23.507652+0000","last_became_active":"2024-05-26T22:48:20.794262+0000","last_became_peered":"2024-05-26T22:48:20.794262+0000","last_unstale":"2024-05-26T22:48:23.507652+0000","last_undegraded":"2024-05-26T22:48:23.507652+0000","last_fullsized":"2024-05-26T22:48:23.507652+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:16:01.106855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.505213+0000","last_change":"2024-05-26T22:48:21.826194+0000","last_active":"2024-05-26T22:48:23.505213+0000","last_peered":"2024-05-26T22:48:23.505213+0000","last_clean":"2024-05-26T22:48:23.505213+0000","last_became_active":"2024-05-26T22:48:21.825674+0000","last_became_peered":"2024-05-26T22:48:21.825674+0000","last_unstale":"2024-05-26T22:48:23.505213+0000","last_undegraded":"2024-05-26T22:48:23.505213+0000","last_fullsized":"2024-05-26T22:48:23.505213+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:01:59.805532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.822681+0000","last_change":"2024-05-26T22:48:20.788924+0000","last_active":"2024-05-26T22:48:22.822681+0000","last_peered":"2024-05-26T22:48:22.822681+0000","last_clean":"2024-05-26T22:48:22.822681+0000","last_became_active":"2024-05-26T22:48:20.788036+0000","last_became_peered":"2024-05-26T22:48:20.788036+0000","last_unstale":"2024-05-26T22:48:22.822681+0000","last_undegraded":"2024-05-26T22:48:22.822681+0000","last_fullsized":"2024-05-26T22:48:22.822681+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:10:49.991129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.507362+0000","last_change":"2024-05-26T22:48:21.833223+0000","last_active":"2024-05-26T22:48:23.507362+0000","last_peered":"2024-05-26T22:48:23.507362+0000","last_clean":"2024-05-26T22:48:23.507362+0000","last_became_active":"2024-05-26T22:48:21.831371+0000","last_became_peered":"2024-05-26T22:48:21.831371+0000","last_unstale":"2024-05-26T22:48:23.507362+0000","last_undegraded":"2024-05-26T22:48:23.507362+0000","last_fullsized":"2024-05-26T22:48:23.507362+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:45:06.922028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.509127+0000","last_change":"2024-05-26T22:48:20.800026+0000","last_active":"2024-05-26T22:48:23.509127+0000","last_peered":"2024-05-26T22:48:23.509127+0000","last_clean":"2024-05-26T22:48:23.509127+0000","last_became_active":"2024-05-26T22:48:20.799776+0000","last_became_peered":"2024-05-26T22:48:20.799776+0000","last_unstale":"2024-05-26T22:48:23.509127+0000","last_undegraded":"2024-05-26T22:48:23.509127+0000","last_fullsized":"2024-05-26T22:48:23.509127+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:33:50.638333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.822718+0000","last_change":"2024-05-26T22:48:21.811868+0000","last_active":"2024-05-26T22:48:22.822718+0000","last_peered":"2024-05-26T22:48:22.822718+0000","last_clean":"2024-05-26T22:48:22.822718+0000","last_became_active":"2024-05-26T22:48:21.811205+0000","last_became_peered":"2024-05-26T22:48:21.811205+0000","last_unstale":"2024-05-26T22:48:22.822718+0000","last_undegraded":"2024-05-26T22:48:22.822718+0000","last_fullsized":"2024-05-26T22:48:22.822718+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:43:21.391197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.823108+0000","last_change":"2024-05-26T22:48:20.786141+0000","last_active":"2024-05-26T22:48:22.823108+0000","last_peered":"2024-05-26T22:48:22.823108+0000","last_clean":"2024-05-26T22:48:22.823108+0000","last_became_active":"2024-05-26T22:48:20.785581+0000","last_became_peered":"2024-05-26T22:48:20.785581+0000","last_unstale":"2024-05-26T22:48:22.823108+0000","last_undegraded":"2024-05-26T22:48:22.823108+0000","last_fullsized":"2024-05-26T22:48:22.823108+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:16:01.106855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.812734+0000","last_change":"2024-05-26T22:48:21.828936+0000","last_active":"2024-05-26T22:48:22.812734+0000","last_peered":"2024-05-26T22:48:22.812734+0000","last_clean":"2024-05-26T22:48:22.812734+0000","last_became_active":"2024-05-26T22:48:21.828635+0000","last_became_peered":"2024-05-26T22:48:21.828635+0000","last_unstale":"2024-05-26T22:48:22.812734+0000","last_undegraded":"2024-05-26T22:48:22.812734+0000","last_fullsized":"2024-05-26T22:48:22.812734+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:21:52.108732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.810327+0000","last_change":"2024-05-26T22:48:20.798700+0000","last_active":"2024-05-26T22:48:22.810327+0000","last_peered":"2024-05-26T22:48:22.810327+0000","last_clean":"2024-05-26T22:48:22.810327+0000","last_became_active":"2024-05-26T22:48:20.796914+0000","last_became_peered":"2024-05-26T22:48:20.796914+0000","last_unstale":"2024-05-26T22:48:22.810327+0000","last_undegraded":"2024-05-26T22:48:22.810327+0000","last_fullsized":"2024-05-26T22:48:22.810327+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:10:49.991129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.809579+0000","last_change":"2024-05-26T22:48:21.811239+0000","last_active":"2024-05-26T22:48:22.809579+0000","last_peered":"2024-05-26T22:48:22.809579+0000","last_clean":"2024-05-26T22:48:22.809579+0000","last_became_active":"2024-05-26T22:48:21.810945+0000","last_became_peered":"2024-05-26T22:48:21.810945+0000","last_unstale":"2024-05-26T22:48:22.809579+0000","last_undegraded":"2024-05-26T22:48:22.809579+0000","last_fullsized":"2024-05-26T22:48:22.809579+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:21:52.108732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.505531+0000","last_change":"2024-05-26T22:48:20.802835+0000","last_active":"2024-05-26T22:48:23.505531+0000","last_peered":"2024-05-26T22:48:23.505531+0000","last_clean":"2024-05-26T22:48:23.505531+0000","last_became_active":"2024-05-26T22:48:20.802501+0000","last_became_peered":"2024-05-26T22:48:20.802501+0000","last_unstale":"2024-05-26T22:48:23.505531+0000","last_undegraded":"2024-05-26T22:48:23.505531+0000","last_fullsized":"2024-05-26T22:48:23.505531+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:39:51.267408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.504467+0000","last_change":"2024-05-26T22:48:21.832723+0000","last_active":"2024-05-26T22:48:23.504467+0000","last_peered":"2024-05-26T22:48:23.504467+0000","last_clean":"2024-05-26T22:48:23.504467+0000","last_became_active":"2024-05-26T22:48:21.824388+0000","last_became_peered":"2024-05-26T22:48:21.824388+0000","last_unstale":"2024-05-26T22:48:23.504467+0000","last_undegraded":"2024-05-26T22:48:23.504467+0000","last_fullsized":"2024-05-26T22:48:23.504467+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:56:00.075959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.822800+0000","last_change":"2024-05-26T22:48:20.797481+0000","last_active":"2024-05-26T22:48:22.822800+0000","last_peered":"2024-05-26T22:48:22.822800+0000","last_clean":"2024-05-26T22:48:22.822800+0000","last_became_active":"2024-05-26T22:48:20.797142+0000","last_became_peered":"2024-05-26T22:48:20.797142+0000","last_unstale":"2024-05-26T22:48:22.822800+0000","last_undegraded":"2024-05-26T22:48:22.822800+0000","last_fullsized":"2024-05-26T22:48:22.822800+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:08:01.698812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.812496+0000","last_change":"2024-05-26T22:48:21.819388+0000","last_active":"2024-05-26T22:48:22.812496+0000","last_peered":"2024-05-26T22:48:22.812496+0000","last_clean":"2024-05-26T22:48:22.812496+0000","last_became_active":"2024-05-26T22:48:21.817847+0000","last_became_peered":"2024-05-26T22:48:21.817847+0000","last_unstale":"2024-05-26T22:48:22.812496+0000","last_undegraded":"2024-05-26T22:48:22.812496+0000","last_fullsized":"2024-05-26T22:48:22.812496+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:38:00.344955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.822809+0000","last_change":"2024-05-26T22:48:20.785751+0000","last_active":"2024-05-26T22:48:22.822809+0000","last_peered":"2024-05-26T22:48:22.822809+0000","last_clean":"2024-05-26T22:48:22.822809+0000","last_became_active":"2024-05-26T22:48:20.785494+0000","last_became_peered":"2024-05-26T22:48:20.785494+0000","last_unstale":"2024-05-26T22:48:22.822809+0000","last_undegraded":"2024-05-26T22:48:22.822809+0000","last_fullsized":"2024-05-26T22:48:22.822809+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:20:41.886676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.812182+0000","last_change":"2024-05-26T22:48:21.825344+0000","last_active":"2024-05-26T22:48:22.812182+0000","last_peered":"2024-05-26T22:48:22.812182+0000","last_clean":"2024-05-26T22:48:22.812182+0000","last_became_active":"2024-05-26T22:48:21.823452+0000","last_became_peered":"2024-05-26T22:48:21.823452+0000","last_unstale":"2024-05-26T22:48:22.812182+0000","last_undegraded":"2024-05-26T22:48:22.812182+0000","last_fullsized":"2024-05-26T22:48:22.812182+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:52:25.073281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.508330+0000","last_change":"2024-05-26T22:48:20.803962+0000","last_active":"2024-05-26T22:48:23.508330+0000","last_peered":"2024-05-26T22:48:23.508330+0000","last_clean":"2024-05-26T22:48:23.508330+0000","last_became_active":"2024-05-26T22:48:20.803609+0000","last_became_peered":"2024-05-26T22:48:20.803609+0000","last_unstale":"2024-05-26T22:48:23.508330+0000","last_undegraded":"2024-05-26T22:48:23.508330+0000","last_fullsized":"2024-05-26T22:48:23.508330+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:16:01.106855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.809871+0000","last_change":"2024-05-26T22:48:21.829756+0000","last_active":"2024-05-26T22:48:22.809871+0000","last_peered":"2024-05-26T22:48:22.809871+0000","last_clean":"2024-05-26T22:48:22.809871+0000","last_became_active":"2024-05-26T22:48:21.829451+0000","last_became_peered":"2024-05-26T22:48:21.829451+0000","last_unstale":"2024-05-26T22:48:22.809871+0000","last_undegraded":"2024-05-26T22:48:22.809871+0000","last_fullsized":"2024-05-26T22:48:22.809871+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:43:21.391197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.506838+0000","last_change":"2024-05-26T22:48:21.832211+0000","last_active":"2024-05-26T22:48:23.506838+0000","last_peered":"2024-05-26T22:48:23.506838+0000","last_clean":"2024-05-26T22:48:23.506838+0000","last_became_active":"2024-05-26T22:48:21.831732+0000","last_became_peered":"2024-05-26T22:48:21.831732+0000","last_unstale":"2024-05-26T22:48:23.506838+0000","last_undegraded":"2024-05-26T22:48:23.506838+0000","last_fullsized":"2024-05-26T22:48:23.506838+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:16:22.847663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.505649+0000","last_change":"2024-05-26T22:48:20.805024+0000","last_active":"2024-05-26T22:48:23.505649+0000","last_peered":"2024-05-26T22:48:23.505649+0000","last_clean":"2024-05-26T22:48:23.505649+0000","last_became_active":"2024-05-26T22:48:20.804736+0000","last_became_peered":"2024-05-26T22:48:20.804736+0000","last_unstale":"2024-05-26T22:48:23.505649+0000","last_undegraded":"2024-05-26T22:48:23.505649+0000","last_fullsized":"2024-05-26T22:48:23.505649+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:42:31.965689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.506861+0000","last_change":"2024-05-26T22:48:21.832952+0000","last_active":"2024-05-26T22:48:23.506861+0000","last_peered":"2024-05-26T22:48:23.506861+0000","last_clean":"2024-05-26T22:48:23.506861+0000","last_became_active":"2024-05-26T22:48:21.831488+0000","last_became_peered":"2024-05-26T22:48:21.831488+0000","last_unstale":"2024-05-26T22:48:23.506861+0000","last_undegraded":"2024-05-26T22:48:23.506861+0000","last_fullsized":"2024-05-26T22:48:23.506861+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:15:00.790746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.508687+0000","last_change":"2024-05-26T22:48:20.799474+0000","last_active":"2024-05-26T22:48:23.508687+0000","last_peered":"2024-05-26T22:48:23.508687+0000","last_clean":"2024-05-26T22:48:23.508687+0000","last_became_active":"2024-05-26T22:48:20.799031+0000","last_became_peered":"2024-05-26T22:48:20.799031+0000","last_unstale":"2024-05-26T22:48:23.508687+0000","last_undegraded":"2024-05-26T22:48:23.508687+0000","last_fullsized":"2024-05-26T22:48:23.508687+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:01:58.792512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.809810+0000","last_change":"2024-05-26T22:48:21.837774+0000","last_active":"2024-05-26T22:48:22.809810+0000","last_peered":"2024-05-26T22:48:22.809810+0000","last_clean":"2024-05-26T22:48:22.809810+0000","last_became_active":"2024-05-26T22:48:21.837450+0000","last_became_peered":"2024-05-26T22:48:21.837450+0000","last_unstale":"2024-05-26T22:48:22.809810+0000","last_undegraded":"2024-05-26T22:48:22.809810+0000","last_fullsized":"2024-05-26T22:48:22.809810+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:33:39.923721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.812718+0000","last_change":"2024-05-26T22:48:20.788987+0000","last_active":"2024-05-26T22:48:22.812718+0000","last_peered":"2024-05-26T22:48:22.812718+0000","last_clean":"2024-05-26T22:48:22.812718+0000","last_became_active":"2024-05-26T22:48:20.788612+0000","last_became_peered":"2024-05-26T22:48:20.788612+0000","last_unstale":"2024-05-26T22:48:22.812718+0000","last_undegraded":"2024-05-26T22:48:22.812718+0000","last_fullsized":"2024-05-26T22:48:22.812718+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:20:41.886676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.812548+0000","last_change":"2024-05-26T22:48:21.826820+0000","last_active":"2024-05-26T22:48:22.812548+0000","last_peered":"2024-05-26T22:48:22.812548+0000","last_clean":"2024-05-26T22:48:22.812548+0000","last_became_active":"2024-05-26T22:48:21.826593+0000","last_became_peered":"2024-05-26T22:48:21.826593+0000","last_unstale":"2024-05-26T22:48:22.812548+0000","last_undegraded":"2024-05-26T22:48:22.812548+0000","last_fullsized":"2024-05-26T22:48:22.812548+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:59:43.907681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.809942+0000","last_change":"2024-05-26T22:48:20.796965+0000","last_active":"2024-05-26T22:48:22.809942+0000","last_peered":"2024-05-26T22:48:22.809942+0000","last_clean":"2024-05-26T22:48:22.809942+0000","last_became_active":"2024-05-26T22:48:20.796587+0000","last_became_peered":"2024-05-26T22:48:20.796587+0000","last_unstale":"2024-05-26T22:48:22.809942+0000","last_undegraded":"2024-05-26T22:48:22.809942+0000","last_fullsized":"2024-05-26T22:48:22.809942+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:11:29.128727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-26T22:48:26.937 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-26T22:48:26.937 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-05-26T22:48:26.937 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-05-26T22:48:26.937 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-05-26T22:48:26.938 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-05-26T22:48:27.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.173+0000 7fe0ccc00640 1 Processor -- start 2024-05-26T22:48:27.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.173+0000 7fe0ccc00640 1 -- start start 2024-05-26T22:48:27.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.174+0000 7fe0ccc00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fe0c8156750 0x7fe0c815fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:27.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.174+0000 7fe0ccc00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fe0c80cc7f0 0x7fe0c80ccc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:27.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.174+0000 7fe0ccc00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0c8160620 con 0x7fe0c806e8c0 2024-05-26T22:48:27.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.174+0000 7fe0ccc00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fe0c815d770 con 0x7fe0c8156750 2024-05-26T22:48:27.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.174+0000 7fe0ccc00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fe0c80cd1a0 con 0x7fe0c80cc7f0 2024-05-26T22:48:27.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.174+0000 7fe0c7400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fe0c8156750 0x7fe0c815fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:27.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.174+0000 7fe0c7400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fe0c8156750 0x7fe0c815fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53900/0 (socket says 172.21.15.131:53900) 2024-05-26T22:48:27.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.174+0000 7fe0c7400640 1 -- 172.21.15.131:0/2244530859 learned_addr learned my addr 172.21.15.131:0/2244530859 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:27.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.174+0000 7fe0bfe00640 1 --1- 172.21.15.131:0/2244530859 >> v1:172.21.15.131:6789/0 conn(0x7fe0c806e8c0 0x7fe0c815d060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:48824/0 (socket says 172.21.15.131:48824) 2024-05-26T22:48:27.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.175+0000 7fe0c7e00640 1 --2- 172.21.15.131:0/2244530859 >> v2:172.21.15.187:3300/0 conn(0x7fe0c80cc7f0 0x7fe0c80ccc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:27.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.175+0000 7fe0c7400640 1 -- 172.21.15.131:0/2244530859 >> v2:172.21.15.187:3300/0 conn(0x7fe0c80cc7f0 msgr2=0x7fe0c80ccc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:27.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.175+0000 7fe0c7400640 1 --2- 172.21.15.131:0/2244530859 >> v2:172.21.15.187:3300/0 conn(0x7fe0c80cc7f0 0x7fe0c80ccc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:27.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.175+0000 7fe0c7400640 1 -- 172.21.15.131:0/2244530859 >> v1:172.21.15.131:6789/0 conn(0x7fe0c806e8c0 legacy=0x7fe0c815d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:27.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.175+0000 7fe0c7400640 1 -- 172.21.15.131:0/2244530859 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0c8160900 con 0x7fe0c8156750 2024-05-26T22:48:27.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.175+0000 7fe0c6a00640 1 -- 172.21.15.131:0/2244530859 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4260677525 0 0) 0x7fe0c8160620 con 0x7fe0c806e8c0 2024-05-26T22:48:27.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.176+0000 7fe0c7400640 1 --2- 172.21.15.131:0/2244530859 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe0c8156750 0x7fe0c815fd40 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe0c815a320 tx=0x7fe0b802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb1ff5f4121bd22 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:27.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.176+0000 7fe0c6a00640 1 -- 172.21.15.131:0/2244530859 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0b8037070 con 0x7fe0c8156750 2024-05-26T22:48:27.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.176+0000 7fe0c6a00640 1 -- 172.21.15.131:0/2244530859 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0b8032440 con 0x7fe0c8156750 2024-05-26T22:48:27.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.176+0000 7fe0c6a00640 1 -- 172.21.15.131:0/2244530859 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0b8032810 con 0x7fe0c8156750 2024-05-26T22:48:27.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.177+0000 7fe0ccc00640 1 -- 172.21.15.131:0/2244530859 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe0c8156750 msgr2=0x7fe0c815fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:27.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.177+0000 7fe0ccc00640 1 --2- 172.21.15.131:0/2244530859 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe0c8156750 0x7fe0c815fd40 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe0c815a320 tx=0x7fe0b802a270 comp rx=0 tx=0).stop 2024-05-26T22:48:27.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.177+0000 7fe0ccc00640 1 -- 172.21.15.131:0/2244530859 shutdown_connections 2024-05-26T22:48:27.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.177+0000 7fe0ccc00640 1 --2- 172.21.15.131:0/2244530859 >> v2:172.21.15.187:3300/0 conn(0x7fe0c80cc7f0 0x7fe0c80ccc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:27.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.177+0000 7fe0ccc00640 1 --2- 172.21.15.131:0/2244530859 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe0c8156750 0x7fe0c815fd40 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:27.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.177+0000 7fe0ccc00640 1 -- 172.21.15.131:0/2244530859 >> 172.21.15.131:0/2244530859 conn(0x7fe0c80cff40 msgr2=0x7fe0c80d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:27.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.177+0000 7fe0ccc00640 1 -- 172.21.15.131:0/2244530859 shutdown_connections 2024-05-26T22:48:27.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.177+0000 7fe0ccc00640 1 -- 172.21.15.131:0/2244530859 wait complete. 2024-05-26T22:48:27.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.179+0000 7fe0ccc00640 1 Processor -- start 2024-05-26T22:48:27.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.179+0000 7fe0ccc00640 1 -- start start 2024-05-26T22:48:27.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.180+0000 7fe0ccc00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe0c806e8c0 0x7fe0c81440d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:27.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.180+0000 7fe0ccc00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe0c80cc7f0 0x7fe0c8144610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:27.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.180+0000 7fe0ccc00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0c8156750 0x7fe0c80ae410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:27.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.180+0000 7fe0ccc00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c815c860 con 0x7fe0c80cc7f0 2024-05-26T22:48:27.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.180+0000 7fe0ccc00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c815cbd0 con 0x7fe0c8156750 2024-05-26T22:48:27.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.180+0000 7fe0ccc00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe0c8156b30 con 0x7fe0c806e8c0 2024-05-26T22:48:27.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.180+0000 7fe0bfe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe0c80cc7f0 0x7fe0c8144610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:27.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.180+0000 7fe0bfe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe0c80cc7f0 0x7fe0c8144610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53916/0 (socket says 172.21.15.131:53916) 2024-05-26T22:48:27.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.180+0000 7fe0bfe00640 1 -- 172.21.15.131:0/671980254 learned_addr learned my addr 172.21.15.131:0/671980254 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:27.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.180+0000 7fe0bfe00640 1 -- 172.21.15.131:0/671980254 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe0c806e8c0 msgr2=0x7fe0c81440d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:27.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.180+0000 7fe0c7e00640 1 --2- 172.21.15.131:0/671980254 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0c8156750 0x7fe0c80ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:27.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.180+0000 7fe0bfe00640 1 --2- 172.21.15.131:0/671980254 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe0c806e8c0 0x7fe0c81440d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:27.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.180+0000 7fe0bfe00640 1 -- 172.21.15.131:0/671980254 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0c8156750 msgr2=0x7fe0c80ae410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:27.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.180+0000 7fe0bfe00640 1 --2- 172.21.15.131:0/671980254 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0c8156750 0x7fe0c80ae410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:27.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.180+0000 7fe0bfe00640 1 -- 172.21.15.131:0/671980254 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0c80af190 con 0x7fe0c80cc7f0 2024-05-26T22:48:27.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.181+0000 7fe0bfe00640 1 --2- 172.21.15.131:0/671980254 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe0c80cc7f0 0x7fe0c8144610 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe0b000c4d0 tx=0x7fe0b000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:27.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.181+0000 7fe0bf400640 1 -- 172.21.15.131:0/671980254 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0b0018070 con 0x7fe0c80cc7f0 2024-05-26T22:48:27.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.181+0000 7fe0bf400640 1 -- 172.21.15.131:0/671980254 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0b0007500 con 0x7fe0c80cc7f0 2024-05-26T22:48:27.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.181+0000 7fe0bf400640 1 -- 172.21.15.131:0/671980254 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0b0004a20 con 0x7fe0c80cc7f0 2024-05-26T22:48:27.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.181+0000 7fe0ccc00640 1 -- 172.21.15.131:0/671980254 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0c80a96b0 con 0x7fe0c80cc7f0 2024-05-26T22:48:27.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.181+0000 7fe0ccc00640 1 -- 172.21.15.131:0/671980254 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0c80a9c70 con 0x7fe0c80cc7f0 2024-05-26T22:48:27.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.182+0000 7fe0ccc00640 1 -- 172.21.15.131:0/671980254 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0c80694e0 con 0x7fe0c80cc7f0 2024-05-26T22:48:27.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.183+0000 7fe0bf400640 1 -- 172.21.15.131:0/671980254 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe0b0004bc0 con 0x7fe0c80cc7f0 2024-05-26T22:48:27.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.184+0000 7fe0bf400640 1 --2- 172.21.15.131:0/671980254 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fe0a406be30 0x7fe0a406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:27.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.184+0000 7fe0bf400640 1 -- 172.21.15.131:0/671980254 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fe0b008a0f0 con 0x7fe0c80cc7f0 2024-05-26T22:48:27.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.184+0000 7fe0c7400640 1 --2- 172.21.15.131:0/671980254 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fe0a406be30 0x7fe0a406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:27.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.185+0000 7fe0c7400640 1 --2- 172.21.15.131:0/671980254 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fe0a406be30 0x7fe0a406e2f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe0b8034000 tx=0x7fe0b802c000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:27.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.189+0000 7fe0bf400640 1 -- 172.21.15.131:0/671980254 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fe0b0058b30 con 0x7fe0c80cc7f0 2024-05-26T22:48:27.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.436+0000 7fe0ccc00640 1 -- 172.21.15.131:0/671980254 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fe0c80a9e40 con 0x7fe0c80cc7f0 2024-05-26T22:48:27.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.445+0000 7fe0bf400640 1 -- 172.21.15.131:0/671980254 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+172946 (secure 0 0 0) 0x7fe0b0058430 con 0x7fe0c80cc7f0 2024-05-26T22:48:27.447 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:48:27.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.449+0000 7fe0ccc00640 1 -- 172.21.15.131:0/671980254 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fe0a406be30 msgr2=0x7fe0a406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:27.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.450+0000 7fe0ccc00640 1 --2- 172.21.15.131:0/671980254 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fe0a406be30 0x7fe0a406e2f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe0b8034000 tx=0x7fe0b802c000 comp rx=0 tx=0).stop 2024-05-26T22:48:27.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.450+0000 7fe0ccc00640 1 -- 172.21.15.131:0/671980254 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe0c80cc7f0 msgr2=0x7fe0c8144610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:27.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.450+0000 7fe0ccc00640 1 --2- 172.21.15.131:0/671980254 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe0c80cc7f0 0x7fe0c8144610 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe0b000c4d0 tx=0x7fe0b000c990 comp rx=0 tx=0).stop 2024-05-26T22:48:27.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.450+0000 7fe0ccc00640 1 -- 172.21.15.131:0/671980254 shutdown_connections 2024-05-26T22:48:27.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.450+0000 7fe0ccc00640 1 --2- 172.21.15.131:0/671980254 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fe0a406be30 0x7fe0a406e2f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:27.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.450+0000 7fe0ccc00640 1 --2- 172.21.15.131:0/671980254 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0c8156750 0x7fe0c80ae410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:27.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.450+0000 7fe0ccc00640 1 --2- 172.21.15.131:0/671980254 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe0c80cc7f0 0x7fe0c8144610 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:27.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.450+0000 7fe0ccc00640 1 --2- 172.21.15.131:0/671980254 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe0c806e8c0 0x7fe0c81440d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:27.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.450+0000 7fe0ccc00640 1 -- 172.21.15.131:0/671980254 >> 172.21.15.131:0/671980254 conn(0x7fe0c80cff40 msgr2=0x7fe0c80d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:27.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.450+0000 7fe0ccc00640 1 -- 172.21.15.131:0/671980254 shutdown_connections 2024-05-26T22:48:27.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.450+0000 7fe0ccc00640 1 -- 172.21.15.131:0/671980254 wait complete. 2024-05-26T22:48:27.469 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":5,"flags":0,"active_gid":4104,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":3484832947},{"type":"v1","addr":"172.21.15.131:6833","nonce":3484832947}]},"active_addr":"172.21.15.131:6833/3484832947","active_change":"2024-05-26T22:48:02.361256+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4111,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.131:0","nonce":1450637640}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.131:0","nonce":1478766804}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.131:0","nonce":1108000854}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.131:0","nonce":2767059717}]}]} 2024-05-26T22:48:27.472 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-05-26T22:48:27.472 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-05-26T22:48:27.473 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-26T22:48:27.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.669+0000 7f7487e00640 1 Processor -- start 2024-05-26T22:48:27.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.669+0000 7f7487e00640 1 -- start start 2024-05-26T22:48:27.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.671+0000 7f7487e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f74880b5b90 0x7f74880d23d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:27.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.671+0000 7f7487e00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7488162330 con 0x7f7488161de0 2024-05-26T22:48:27.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.671+0000 7f7487e00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7488162610 con 0x7f74880b9990 2024-05-26T22:48:27.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.671+0000 7f7487e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f748815e970 con 0x7f74880b5b90 2024-05-26T22:48:27.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.671+0000 7f7486000640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f7488161de0 0x7f74880b5f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:48840/0 (socket says 172.21.15.131:48840) 2024-05-26T22:48:27.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.671+0000 7f7486000640 1 -- 172.21.15.131:0/2094627784 learned_addr learned my addr 172.21.15.131:0/2094627784 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:48:27.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.672+0000 7f7486a00640 1 --2- 172.21.15.131:0/2094627784 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f74880b5b90 0x7f74880d23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:27.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.672+0000 7f7485600640 1 -- 172.21.15.131:0/2094627784 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3749300699 0 0) 0x7f7488162330 con 0x7f7488161de0 2024-05-26T22:48:27.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.672+0000 7f7485600640 1 -- 172.21.15.131:0/2094627784 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f746c003520 con 0x7f7488161de0 2024-05-26T22:48:27.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.672+0000 7f7485600640 1 -- 172.21.15.131:0/2094627784 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2754217151 0 0) 0x7f746c003520 con 0x7f7488161de0 2024-05-26T22:48:27.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.672+0000 7f7485600640 1 -- 172.21.15.131:0/2094627784 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f74880b5b90 msgr2=0x7f74880d23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:27.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.672+0000 7f7485600640 1 --2- 172.21.15.131:0/2094627784 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f74880b5b90 0x7f74880d23d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:27.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.672+0000 7f7485600640 1 -- 172.21.15.131:0/2094627784 >> v1:172.21.15.187:6789/0 conn(0x7f74880b9990 legacy=0x7f748815ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:27.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.672+0000 7f7485600640 1 -- 172.21.15.131:0/2094627784 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74881628f0 con 0x7f7488161de0 2024-05-26T22:48:27.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.672+0000 7f7485600640 1 -- 172.21.15.131:0/2094627784 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7f7474001670 con 0x7f7488161de0 2024-05-26T22:48:27.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.672+0000 7f7485600640 1 -- 172.21.15.131:0/2094627784 >> v1:172.21.15.131:6789/0 conn(0x7f7488161de0 legacy=0x7f74880b5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:27.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.672+0000 7f7485600640 1 --2- 172.21.15.131:0/2094627784 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f746c0039e0 0x7f746c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:27.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.672+0000 7f7485600640 1 --2- 172.21.15.131:0/2094627784 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f74880b5b90 0x7f746c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:27.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.672+0000 7f7485600640 1 --2- 172.21.15.131:0/2094627784 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f746c004d40 0x7f746c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:27.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.672+0000 7f7485600640 1 -- 172.21.15.131:0/2094627784 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f748815e970 con 0x7f746c004d40 2024-05-26T22:48:27.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.672+0000 7f7485600640 1 -- 172.21.15.131:0/2094627784 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f746c00bb50 con 0x7f746c0039e0 2024-05-26T22:48:27.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.672+0000 7f7485600640 1 -- 172.21.15.131:0/2094627784 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f746c00bd50 con 0x7f74880b5b90 2024-05-26T22:48:27.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.673+0000 7f7487400640 1 --2- 172.21.15.131:0/2094627784 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f746c004d40 0x7f746c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:27.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.673+0000 7f7486a00640 1 --2- 172.21.15.131:0/2094627784 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f746c0039e0 0x7f746c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:27.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.673+0000 7f7487400640 1 -- 172.21.15.131:0/2094627784 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f74880b5b90 msgr2=0x7f746c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:27.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.673+0000 7f7487400640 1 --2- 172.21.15.131:0/2094627784 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f74880b5b90 0x7f746c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:27.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.673+0000 7f7487400640 1 -- 172.21.15.131:0/2094627784 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f746c0039e0 msgr2=0x7f746c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:27.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.673+0000 7f7487400640 1 --2- 172.21.15.131:0/2094627784 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f746c0039e0 0x7f746c003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:27.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.673+0000 7f7487400640 1 -- 172.21.15.131:0/2094627784 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f746c003520 con 0x7f746c004d40 2024-05-26T22:48:27.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.674+0000 7f7487400640 1 --2- 172.21.15.131:0/2094627784 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f746c004d40 0x7f746c0051c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f7480001e50 tx=0x7f746c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cda0498ff5fd5709 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:27.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.674+0000 7f7485600640 1 -- 172.21.15.131:0/2094627784 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7480001a70 con 0x7f746c004d40 2024-05-26T22:48:27.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.674+0000 7f7485600640 1 -- 172.21.15.131:0/2094627784 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7480003970 con 0x7f746c004d40 2024-05-26T22:48:27.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.674+0000 7f7485600640 1 -- 172.21.15.131:0/2094627784 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7480003d60 con 0x7f746c004d40 2024-05-26T22:48:27.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.675+0000 7f7487e00640 1 -- 172.21.15.131:0/2094627784 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f746c004d40 msgr2=0x7f746c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:27.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.675+0000 7f7487e00640 1 --2- 172.21.15.131:0/2094627784 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f746c004d40 0x7f746c0051c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f7480001e50 tx=0x7f746c005830 comp rx=0 tx=0).stop 2024-05-26T22:48:27.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.675+0000 7f7487400640 1 -- 172.21.15.131:0/2094627784 reap_dead start 2024-05-26T22:48:27.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.675+0000 7f7487e00640 1 -- 172.21.15.131:0/2094627784 shutdown_connections 2024-05-26T22:48:27.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.675+0000 7f7487e00640 1 -- 172.21.15.131:0/2094627784 >> 172.21.15.131:0/2094627784 conn(0x7f7488073070 msgr2=0x7f74880d0b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:27.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.675+0000 7f7487e00640 1 -- 172.21.15.131:0/2094627784 shutdown_connections 2024-05-26T22:48:27.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.675+0000 7f7487e00640 1 -- 172.21.15.131:0/2094627784 wait complete. 2024-05-26T22:48:27.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.677+0000 7f7487e00640 1 Processor -- start 2024-05-26T22:48:27.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.677+0000 7f7487e00640 1 -- start start 2024-05-26T22:48:27.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.678+0000 7f7487e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f746c004d40 0x7f74880ccf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:27.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.678+0000 7f7487e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f74880cd450 0x7f748814e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:27.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.678+0000 7f7487e00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f748814e560 0x7f74880cb220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:27.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.678+0000 7f7487e00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f74880b9fa0 con 0x7f746c004d40 2024-05-26T22:48:27.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.678+0000 7f7487e00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f74880b9e20 con 0x7f748814e560 2024-05-26T22:48:27.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.678+0000 7f7487e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f748815e5e0 con 0x7f74880cd450 2024-05-26T22:48:27.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.678+0000 7f7486a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f746c004d40 0x7f74880ccf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:27.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.678+0000 7f7486a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f746c004d40 0x7f74880ccf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53936/0 (socket says 172.21.15.131:53936) 2024-05-26T22:48:27.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.678+0000 7f7486a00640 1 -- 172.21.15.131:0/2259013746 learned_addr learned my addr 172.21.15.131:0/2259013746 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:27.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.679+0000 7f7486a00640 1 -- 172.21.15.131:0/2259013746 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f74880cd450 msgr2=0x7f748814e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:27.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.679+0000 7f7486a00640 1 --2- 172.21.15.131:0/2259013746 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f74880cd450 0x7f748814e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:27.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.679+0000 7f7486a00640 1 -- 172.21.15.131:0/2259013746 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f748814e560 msgr2=0x7f74880cb220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:27.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.679+0000 7f7486a00640 1 --2- 172.21.15.131:0/2259013746 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f748814e560 0x7f74880cb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:27.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.679+0000 7f7486a00640 1 -- 172.21.15.131:0/2259013746 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74880cbe70 con 0x7f746c004d40 2024-05-26T22:48:27.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.679+0000 7f7486a00640 1 --2- 172.21.15.131:0/2259013746 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f746c004d40 0x7f74880ccf10 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f74780116a0 tx=0x7f7478011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:27.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.679+0000 7f747ea00640 1 -- 172.21.15.131:0/2259013746 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f747800f940 con 0x7f746c004d40 2024-05-26T22:48:27.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.679+0000 7f747ea00640 1 -- 172.21.15.131:0/2259013746 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f747800fae0 con 0x7f746c004d40 2024-05-26T22:48:27.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.679+0000 7f747ea00640 1 -- 172.21.15.131:0/2259013746 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f747800fe90 con 0x7f746c004d40 2024-05-26T22:48:27.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.679+0000 7f7487e00640 1 -- 172.21.15.131:0/2259013746 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74880cc150 con 0x7f746c004d40 2024-05-26T22:48:27.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.679+0000 7f7487e00640 1 -- 172.21.15.131:0/2259013746 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74880cc710 con 0x7f746c004d40 2024-05-26T22:48:27.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.680+0000 7f7487e00640 1 -- 172.21.15.131:0/2259013746 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74880694e0 con 0x7f746c004d40 2024-05-26T22:48:27.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.681+0000 7f747ea00640 1 -- 172.21.15.131:0/2259013746 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7478015460 con 0x7f746c004d40 2024-05-26T22:48:27.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.682+0000 7f747ea00640 1 --2- 172.21.15.131:0/2259013746 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f746806bfd0 0x7f746806e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:27.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.682+0000 7f747ea00640 1 -- 172.21.15.131:0/2259013746 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f747808b8c0 con 0x7f746c004d40 2024-05-26T22:48:27.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.682+0000 7f7486000640 1 --2- 172.21.15.131:0/2259013746 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f746806bfd0 0x7f746806e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:27.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.683+0000 7f7486000640 1 --2- 172.21.15.131:0/2259013746 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f746806bfd0 0x7f746806e490 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f74880b9600 tx=0x7f7474001500 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:27.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.687+0000 7f747ea00640 1 -- 172.21.15.131:0/2259013746 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f7478019030 con 0x7f746c004d40 2024-05-26T22:48:27.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.888+0000 7f7487e00640 1 -- 172.21.15.131:0/2259013746 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f74880d3820 con 0x7f746c004d40 2024-05-26T22:48:27.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.891+0000 7f747ea00640 1 -- 172.21.15.131:0/2259013746 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16629 (secure 0 0 0) 0x7f747805a1d0 con 0x7f746c004d40 2024-05-26T22:48:27.893 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:48:27.894 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":20,"fsid":"46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f","created":"2024-05-26T22:47:57.288905+0000","modified":"2024-05-26T22:48:22.799298+0000","last_up_change":"2024-05-26T22:48:09.602207+0000","last_in_change":"2024-05-26T22:48:00.687341+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-26T22:48:10.463339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-26T22:48:19.663254+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-26T22:48:20.235751+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"814eaf62-a18d-494b-8e27-fd8fb88adea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6801","nonce":2350940051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6803","nonce":2350940051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6807","nonce":2350940051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6805","nonce":2350940051}]},"public_addr":"172.21.15.131:6801/2350940051","cluster_addr":"172.21.15.131:6803/2350940051","heartbeat_back_addr":"172.21.15.131:6807/2350940051","heartbeat_front_addr":"172.21.15.131:6805/2350940051","state":["exists","up"]},{"osd":1,"uuid":"42e66287-f6c6-4e26-af3c-6dfaa3536d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6825","nonce":1525528247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6827","nonce":1525528247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6831","nonce":1525528247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6829","nonce":1525528247}]},"public_addr":"172.21.15.131:6825/1525528247","cluster_addr":"172.21.15.131:6827/1525528247","heartbeat_back_addr":"172.21.15.131:6831/1525528247","heartbeat_front_addr":"172.21.15.131:6829/1525528247","state":["exists","up"]},{"osd":2,"uuid":"0da34673-04a2-4fd0-b57a-0ff2b679b043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6817","nonce":3627842210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6819","nonce":3627842210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6823","nonce":3627842210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6821","nonce":3627842210}]},"public_addr":"172.21.15.131:6817/3627842210","cluster_addr":"172.21.15.131:6819/3627842210","heartbeat_back_addr":"172.21.15.131:6823/3627842210","heartbeat_front_addr":"172.21.15.131:6821/3627842210","state":["exists","up"]},{"osd":3,"uuid":"cf117a27-d6b2-43d3-ae52-b47988bf8eed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6809","nonce":350790591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6811","nonce":350790591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6815","nonce":350790591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6813","nonce":350790591}]},"public_addr":"172.21.15.131:6809/350790591","cluster_addr":"172.21.15.131:6811/350790591","heartbeat_back_addr":"172.21.15.131:6815/350790591","heartbeat_front_addr":"172.21.15.131:6813/350790591","state":["exists","up"]},{"osd":4,"uuid":"a092ccfc-be89-4a84-88d5-0f781fa715eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6816","nonce":244560995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6819","nonce":244560995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6823","nonce":244560995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6821","nonce":244560995}]},"public_addr":"172.21.15.187:6816/244560995","cluster_addr":"172.21.15.187:6819/244560995","heartbeat_back_addr":"172.21.15.187:6823/244560995","heartbeat_front_addr":"172.21.15.187:6821/244560995","state":["exists","up"]},{"osd":5,"uuid":"7513ba69-dd00-4a35-b190-9e8cc3b2bdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6801","nonce":324619398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6803","nonce":324619398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6812","nonce":324619398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6807","nonce":324619398}]},"public_addr":"172.21.15.187:6801/324619398","cluster_addr":"172.21.15.187:6803/324619398","heartbeat_back_addr":"172.21.15.187:6812/324619398","heartbeat_front_addr":"172.21.15.187:6807/324619398","state":["exists","up"]},{"osd":6,"uuid":"50081810-c093-4d2f-a4d7-b7c695c5400d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6806","nonce":1391347786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6809","nonce":1391347786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6818","nonce":1391347786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6813","nonce":1391347786}]},"public_addr":"172.21.15.187:6806/1391347786","cluster_addr":"172.21.15.187:6809/1391347786","heartbeat_back_addr":"172.21.15.187:6818/1391347786","heartbeat_front_addr":"172.21.15.187:6813/1391347786","state":["exists","up"]},{"osd":7,"uuid":"775dae05-54f5-4ad4-bbce-92863c6ed2ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6825","nonce":1220065622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6827","nonce":1220065622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6831","nonce":1220065622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6829","nonce":1220065622}]},"public_addr":"172.21.15.187:6825/1220065622","cluster_addr":"172.21.15.187:6827/1220065622","heartbeat_back_addr":"172.21.15.187:6831/1220065622","heartbeat_front_addr":"172.21.15.187:6829/1220065622","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:05.983520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.299516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.067323+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.307748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.018742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.258775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.010912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.295355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-26T22:48:27.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.894+0000 7f7487e00640 1 -- 172.21.15.131:0/2259013746 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f746806bfd0 msgr2=0x7f746806e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:27.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.894+0000 7f7487e00640 1 --2- 172.21.15.131:0/2259013746 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f746806bfd0 0x7f746806e490 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f74880b9600 tx=0x7f7474001500 comp rx=0 tx=0).stop 2024-05-26T22:48:27.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.894+0000 7f7487e00640 1 -- 172.21.15.131:0/2259013746 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f746c004d40 msgr2=0x7f74880ccf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:27.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.894+0000 7f7487e00640 1 --2- 172.21.15.131:0/2259013746 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f746c004d40 0x7f74880ccf10 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f74780116a0 tx=0x7f7478011b60 comp rx=0 tx=0).stop 2024-05-26T22:48:27.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.894+0000 7f7487e00640 1 -- 172.21.15.131:0/2259013746 shutdown_connections 2024-05-26T22:48:27.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.894+0000 7f7487e00640 1 --2- 172.21.15.131:0/2259013746 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f74880cd450 0x7f748814e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:27.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.894+0000 7f7487e00640 1 --2- 172.21.15.131:0/2259013746 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f746806bfd0 0x7f746806e490 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:27.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.894+0000 7f7487e00640 1 --2- 172.21.15.131:0/2259013746 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f748814e560 0x7f74880cb220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:27.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.894+0000 7f7487e00640 1 --2- 172.21.15.131:0/2259013746 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f746c004d40 0x7f74880ccf10 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:27.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.894+0000 7f7487e00640 1 -- 172.21.15.131:0/2259013746 >> 172.21.15.131:0/2259013746 conn(0x7f7488073070 msgr2=0x7f74880b42f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:27.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.894+0000 7f7487e00640 1 -- 172.21.15.131:0/2259013746 shutdown_connections 2024-05-26T22:48:27.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:27.894+0000 7f7487e00640 1 -- 172.21.15.131:0/2259013746 wait complete. 2024-05-26T22:48:27.913 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-05-26T22:48:27.913 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-26T22:48:28.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.116+0000 7f0d6d200640 1 Processor -- start 2024-05-26T22:48:28.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.116+0000 7f0d6d200640 1 -- start start 2024-05-26T22:48:28.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.117+0000 7f0d6d200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f0d6815c3d0 0x7f0d6806ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:28.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.117+0000 7f0d6d200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0d6806f490 0x7f0d680ade40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:28.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.117+0000 7f0d6d200640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d680722a0 con 0x7f0d68155840 2024-05-26T22:48:28.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.117+0000 7f0d6d200640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f0d68072040 con 0x7f0d6815c3d0 2024-05-26T22:48:28.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.117+0000 7f0d6d200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f0d68072580 con 0x7f0d6806f490 2024-05-26T22:48:28.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.118+0000 7f0d5d400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f0d6815c3d0 0x7f0d6806ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:28.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.118+0000 7f0d5d400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f0d6815c3d0 0x7f0d6806ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53948/0 (socket says 172.21.15.131:53948) 2024-05-26T22:48:28.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.118+0000 7f0d5d400640 1 -- 172.21.15.131:0/3112197224 learned_addr learned my addr 172.21.15.131:0/3112197224 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:28.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.118+0000 7f0d5d400640 1 -- 172.21.15.131:0/3112197224 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0d6806f490 msgr2=0x7f0d680ade40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:28.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.118+0000 7f0d66800640 1 --2- 172.21.15.131:0/3112197224 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0d6806f490 0x7f0d680ade40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:28.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.118+0000 7f0d5d400640 1 --2- 172.21.15.131:0/3112197224 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0d6806f490 0x7f0d680ade40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:28.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.118+0000 7f0d5d400640 1 -- 172.21.15.131:0/3112197224 >> v1:172.21.15.187:6789/0 conn(0x7f0d68155840 legacy=0x7f0d68071930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:28.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.118+0000 7f0d5d400640 1 -- 172.21.15.131:0/3112197224 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d68072860 con 0x7f0d6815c3d0 2024-05-26T22:48:28.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.119+0000 7f0d5d400640 1 --2- 172.21.15.131:0/3112197224 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0d6815c3d0 0x7f0d6806ef50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0d50009870 tx=0x7f0d5002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c180229880d939e server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:28.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.119+0000 7f0d65400640 1 -- 172.21.15.131:0/3112197224 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d50037070 con 0x7f0d6815c3d0 2024-05-26T22:48:28.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.119+0000 7f0d65400640 1 -- 172.21.15.131:0/3112197224 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d50032440 con 0x7f0d6815c3d0 2024-05-26T22:48:28.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.119+0000 7f0d65400640 1 -- 172.21.15.131:0/3112197224 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d50032810 con 0x7f0d6815c3d0 2024-05-26T22:48:28.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.120+0000 7f0d6d200640 1 -- 172.21.15.131:0/3112197224 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0d6815c3d0 msgr2=0x7f0d6806ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:28.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.120+0000 7f0d6d200640 1 --2- 172.21.15.131:0/3112197224 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0d6815c3d0 0x7f0d6806ef50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0d50009870 tx=0x7f0d5002a210 comp rx=0 tx=0).stop 2024-05-26T22:48:28.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.120+0000 7f0d6d200640 1 -- 172.21.15.131:0/3112197224 shutdown_connections 2024-05-26T22:48:28.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.120+0000 7f0d6d200640 1 --2- 172.21.15.131:0/3112197224 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0d6806f490 0x7f0d680ade40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:28.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.120+0000 7f0d6d200640 1 --2- 172.21.15.131:0/3112197224 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0d6815c3d0 0x7f0d6806ef50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:28.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.120+0000 7f0d6d200640 1 -- 172.21.15.131:0/3112197224 >> 172.21.15.131:0/3112197224 conn(0x7f0d680b1e90 msgr2=0x7f0d680b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:28.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.120+0000 7f0d6d200640 1 -- 172.21.15.131:0/3112197224 shutdown_connections 2024-05-26T22:48:28.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.120+0000 7f0d6d200640 1 -- 172.21.15.131:0/3112197224 wait complete. 2024-05-26T22:48:28.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.122+0000 7f0d6d200640 1 Processor -- start 2024-05-26T22:48:28.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.122+0000 7f0d6d200640 1 -- start start 2024-05-26T22:48:28.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.123+0000 7f0d6d200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0d6806f490 0x7f0d68151450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:28.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.123+0000 7f0d6d200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0d68155840 0x7f0d680b37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:28.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.123+0000 7f0d6d200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0d6815c3d0 0x7f0d680b3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:28.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.123+0000 7f0d6d200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f0d680ae380 con 0x7f0d6815c3d0 2024-05-26T22:48:28.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.123+0000 7f0d6d200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0d68072e20 con 0x7f0d68155840 2024-05-26T22:48:28.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.123+0000 7f0d6d200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f0d680ae500 con 0x7f0d6806f490 2024-05-26T22:48:28.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.123+0000 7f0d66800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0d6815c3d0 0x7f0d680b3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:28.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.123+0000 7f0d66800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0d6815c3d0 0x7f0d680b3ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53954/0 (socket says 172.21.15.131:53954) 2024-05-26T22:48:28.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.123+0000 7f0d66800640 1 -- 172.21.15.131:0/3117850735 learned_addr learned my addr 172.21.15.131:0/3117850735 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:28.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.124+0000 7f0d5d400640 1 --2- 172.21.15.131:0/3117850735 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0d68155840 0x7f0d680b37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:28.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.124+0000 7f0d66800640 1 -- 172.21.15.131:0/3117850735 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0d6806f490 msgr2=0x7f0d68151450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:28.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.124+0000 7f0d66800640 1 --2- 172.21.15.131:0/3117850735 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0d6806f490 0x7f0d68151450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:28.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.124+0000 7f0d66800640 1 -- 172.21.15.131:0/3117850735 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0d68155840 msgr2=0x7f0d680b37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:28.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.124+0000 7f0d66800640 1 --2- 172.21.15.131:0/3117850735 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0d68155840 0x7f0d680b37a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:28.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.124+0000 7f0d66800640 1 -- 172.21.15.131:0/3117850735 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d68148330 con 0x7f0d6815c3d0 2024-05-26T22:48:28.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.124+0000 7f0d66800640 1 --2- 172.21.15.131:0/3117850735 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0d6815c3d0 0x7f0d680b3ce0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0d6000cd70 tx=0x7f0d6000a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:28.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.124+0000 7f0d5ea00640 1 -- 172.21.15.131:0/3117850735 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d60003b90 con 0x7f0d6815c3d0 2024-05-26T22:48:28.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.124+0000 7f0d5ea00640 1 -- 172.21.15.131:0/3117850735 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d60003d30 con 0x7f0d6815c3d0 2024-05-26T22:48:28.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.124+0000 7f0d5ea00640 1 -- 172.21.15.131:0/3117850735 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d60004240 con 0x7f0d6815c3d0 2024-05-26T22:48:28.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.124+0000 7f0d6d200640 1 -- 172.21.15.131:0/3117850735 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d680b4310 con 0x7f0d6815c3d0 2024-05-26T22:48:28.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.124+0000 7f0d6d200640 1 -- 172.21.15.131:0/3117850735 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d68145720 con 0x7f0d6815c3d0 2024-05-26T22:48:28.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.125+0000 7f0d6d200640 1 -- 172.21.15.131:0/3117850735 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d680694e0 con 0x7f0d6815c3d0 2024-05-26T22:48:28.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.127+0000 7f0d5ea00640 1 -- 172.21.15.131:0/3117850735 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0d600043e0 con 0x7f0d6815c3d0 2024-05-26T22:48:28.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.127+0000 7f0d5ea00640 1 --2- 172.21.15.131:0/3117850735 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f0d4406be30 0x7f0d4406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:28.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.127+0000 7f0d5ea00640 1 -- 172.21.15.131:0/3117850735 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f0d6008b970 con 0x7f0d6815c3d0 2024-05-26T22:48:28.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.128+0000 7f0d65e00640 1 --2- 172.21.15.131:0/3117850735 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f0d4406be30 0x7f0d4406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:28.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.128+0000 7f0d65e00640 1 --2- 172.21.15.131:0/3117850735 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f0d4406be30 0x7f0d4406e2f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f0d68155130 tx=0x7f0d580079b0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:28.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.132+0000 7f0d5ea00640 1 -- 172.21.15.131:0/3117850735 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f0d6005a310 con 0x7f0d6815c3d0 2024-05-26T22:48:28.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.333+0000 7f0d6d200640 1 -- 172.21.15.131:0/3117850735 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0d6815a0d0 con 0x7f0d6815c3d0 2024-05-26T22:48:28.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.334+0000 7f0d5ea00640 1 -- 172.21.15.131:0/3117850735 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16629 (secure 0 0 0) 0x7f0d60059c10 con 0x7f0d6815c3d0 2024-05-26T22:48:28.335 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:48:28.335 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":20,"fsid":"46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f","created":"2024-05-26T22:47:57.288905+0000","modified":"2024-05-26T22:48:22.799298+0000","last_up_change":"2024-05-26T22:48:09.602207+0000","last_in_change":"2024-05-26T22:48:00.687341+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-26T22:48:10.463339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-26T22:48:19.663254+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-26T22:48:20.235751+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"814eaf62-a18d-494b-8e27-fd8fb88adea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6801","nonce":2350940051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6803","nonce":2350940051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6807","nonce":2350940051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6805","nonce":2350940051}]},"public_addr":"172.21.15.131:6801/2350940051","cluster_addr":"172.21.15.131:6803/2350940051","heartbeat_back_addr":"172.21.15.131:6807/2350940051","heartbeat_front_addr":"172.21.15.131:6805/2350940051","state":["exists","up"]},{"osd":1,"uuid":"42e66287-f6c6-4e26-af3c-6dfaa3536d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6825","nonce":1525528247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6827","nonce":1525528247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6831","nonce":1525528247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6829","nonce":1525528247}]},"public_addr":"172.21.15.131:6825/1525528247","cluster_addr":"172.21.15.131:6827/1525528247","heartbeat_back_addr":"172.21.15.131:6831/1525528247","heartbeat_front_addr":"172.21.15.131:6829/1525528247","state":["exists","up"]},{"osd":2,"uuid":"0da34673-04a2-4fd0-b57a-0ff2b679b043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6817","nonce":3627842210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6819","nonce":3627842210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6823","nonce":3627842210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6821","nonce":3627842210}]},"public_addr":"172.21.15.131:6817/3627842210","cluster_addr":"172.21.15.131:6819/3627842210","heartbeat_back_addr":"172.21.15.131:6823/3627842210","heartbeat_front_addr":"172.21.15.131:6821/3627842210","state":["exists","up"]},{"osd":3,"uuid":"cf117a27-d6b2-43d3-ae52-b47988bf8eed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6809","nonce":350790591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6811","nonce":350790591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6815","nonce":350790591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6813","nonce":350790591}]},"public_addr":"172.21.15.131:6809/350790591","cluster_addr":"172.21.15.131:6811/350790591","heartbeat_back_addr":"172.21.15.131:6815/350790591","heartbeat_front_addr":"172.21.15.131:6813/350790591","state":["exists","up"]},{"osd":4,"uuid":"a092ccfc-be89-4a84-88d5-0f781fa715eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6816","nonce":244560995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6819","nonce":244560995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6823","nonce":244560995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6821","nonce":244560995}]},"public_addr":"172.21.15.187:6816/244560995","cluster_addr":"172.21.15.187:6819/244560995","heartbeat_back_addr":"172.21.15.187:6823/244560995","heartbeat_front_addr":"172.21.15.187:6821/244560995","state":["exists","up"]},{"osd":5,"uuid":"7513ba69-dd00-4a35-b190-9e8cc3b2bdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6801","nonce":324619398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6803","nonce":324619398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6812","nonce":324619398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6807","nonce":324619398}]},"public_addr":"172.21.15.187:6801/324619398","cluster_addr":"172.21.15.187:6803/324619398","heartbeat_back_addr":"172.21.15.187:6812/324619398","heartbeat_front_addr":"172.21.15.187:6807/324619398","state":["exists","up"]},{"osd":6,"uuid":"50081810-c093-4d2f-a4d7-b7c695c5400d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6806","nonce":1391347786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6809","nonce":1391347786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6818","nonce":1391347786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6813","nonce":1391347786}]},"public_addr":"172.21.15.187:6806/1391347786","cluster_addr":"172.21.15.187:6809/1391347786","heartbeat_back_addr":"172.21.15.187:6818/1391347786","heartbeat_front_addr":"172.21.15.187:6813/1391347786","state":["exists","up"]},{"osd":7,"uuid":"775dae05-54f5-4ad4-bbce-92863c6ed2ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6825","nonce":1220065622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6827","nonce":1220065622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6831","nonce":1220065622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6829","nonce":1220065622}]},"public_addr":"172.21.15.187:6825/1220065622","cluster_addr":"172.21.15.187:6827/1220065622","heartbeat_back_addr":"172.21.15.187:6831/1220065622","heartbeat_front_addr":"172.21.15.187:6829/1220065622","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:05.983520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.299516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.067323+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.307748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.018742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.258775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.010912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.295355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-26T22:48:28.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.338+0000 7f0d6d200640 1 -- 172.21.15.131:0/3117850735 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f0d4406be30 msgr2=0x7f0d4406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:28.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.338+0000 7f0d6d200640 1 --2- 172.21.15.131:0/3117850735 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f0d4406be30 0x7f0d4406e2f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f0d68155130 tx=0x7f0d580079b0 comp rx=0 tx=0).stop 2024-05-26T22:48:28.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.338+0000 7f0d6d200640 1 -- 172.21.15.131:0/3117850735 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0d6815c3d0 msgr2=0x7f0d680b3ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:28.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.338+0000 7f0d6d200640 1 --2- 172.21.15.131:0/3117850735 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0d6815c3d0 0x7f0d680b3ce0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0d6000cd70 tx=0x7f0d6000a660 comp rx=0 tx=0).stop 2024-05-26T22:48:28.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.338+0000 7f0d6d200640 1 -- 172.21.15.131:0/3117850735 shutdown_connections 2024-05-26T22:48:28.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.338+0000 7f0d6d200640 1 --2- 172.21.15.131:0/3117850735 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f0d4406be30 0x7f0d4406e2f0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:28.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.338+0000 7f0d6d200640 1 --2- 172.21.15.131:0/3117850735 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0d6815c3d0 0x7f0d680b3ce0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:28.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.338+0000 7f0d6d200640 1 --2- 172.21.15.131:0/3117850735 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0d68155840 0x7f0d680b37a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:28.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.338+0000 7f0d6d200640 1 --2- 172.21.15.131:0/3117850735 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0d6806f490 0x7f0d68151450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:28.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.338+0000 7f0d6d200640 1 -- 172.21.15.131:0/3117850735 >> 172.21.15.131:0/3117850735 conn(0x7f0d680b1e90 msgr2=0x7f0d680b2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:28.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.338+0000 7f0d6d200640 1 -- 172.21.15.131:0/3117850735 shutdown_connections 2024-05-26T22:48:28.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.338+0000 7f0d6d200640 1 -- 172.21.15.131:0/3117850735 wait complete. 2024-05-26T22:48:28.357 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-05-26T22:48:28.357 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-05-26T22:48:28.357 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-05-26T22:48:28.358 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-05-26T22:48:28.358 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-05-26T22:48:28.359 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-05-26T22:48:28.359 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-05-26T22:48:28.359 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-05-26T22:48:28.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.651+0000 7f5891400640 1 Processor -- start 2024-05-26T22:48:28.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.651+0000 7f5891400640 1 -- start start 2024-05-26T22:48:28.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.655+0000 7f5891400640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f588c0b6220 0x7f588c0b6600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:28.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.655+0000 7f5891400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f588c0cf2b0 0x7f588c0b91a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:28.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.655+0000 7f5891400640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f588c0b6b40 con 0x7f588c0cf710 2024-05-26T22:48:28.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.655+0000 7f5891400640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f588c0b7eb0 con 0x7f588c0cf2b0 2024-05-26T22:48:28.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.655+0000 7f5891400640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f588c0b8d10 con 0x7f588c0b6220 2024-05-26T22:48:28.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.656+0000 7f5890a00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f588c0cf710 0x7f588c0b96e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:48842/0 (socket says 172.21.15.131:48842) 2024-05-26T22:48:28.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.656+0000 7f5890a00640 1 -- 172.21.15.131:0/4228777034 learned_addr learned my addr 172.21.15.131:0/4228777034 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:48:28.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.657+0000 7f588aa00640 1 -- 172.21.15.131:0/4228777034 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 505361965 0 0) 0x7f588c0b6b40 con 0x7f588c0cf710 2024-05-26T22:48:28.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.657+0000 7f588aa00640 1 -- 172.21.15.131:0/4228777034 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5870003520 con 0x7f588c0cf710 2024-05-26T22:48:28.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.657+0000 7f588b400640 1 --2- 172.21.15.131:0/4228777034 >> v2:172.21.15.131:3300/0 conn(0x7f588c0cf2b0 0x7f588c0b91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:28.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.658+0000 7f588aa00640 1 -- 172.21.15.131:0/4228777034 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1793988407 0 0) 0x7f5870003520 con 0x7f588c0cf710 2024-05-26T22:48:28.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.658+0000 7f588aa00640 1 -- 172.21.15.131:0/4228777034 >> v2:172.21.15.187:3300/0 conn(0x7f588c0b6220 msgr2=0x7f588c0b6600 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-26T22:48:28.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.658+0000 7f588aa00640 1 --2- 172.21.15.131:0/4228777034 >> v2:172.21.15.187:3300/0 conn(0x7f588c0b6220 0x7f588c0b6600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:28.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.658+0000 7f588aa00640 1 -- 172.21.15.131:0/4228777034 >> v2:172.21.15.131:3300/0 conn(0x7f588c0cf2b0 msgr2=0x7f588c0b91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:28.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.658+0000 7f588aa00640 1 --2- 172.21.15.131:0/4228777034 >> v2:172.21.15.131:3300/0 conn(0x7f588c0cf2b0 0x7f588c0b91a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:28.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.657+0000 7f64ff200640 1 Processor -- start 2024-05-26T22:48:28.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.658+0000 7f64ff200640 1 -- start start 2024-05-26T22:48:28.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.656+0000 7f3f89200640 1 Processor -- start 2024-05-26T22:48:28.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.658+0000 7f588aa00640 1 -- 172.21.15.131:0/4228777034 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f588c0b8100 con 0x7f588c0cf710 2024-05-26T22:48:28.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.658+0000 7f588aa00640 1 -- 172.21.15.131:0/4228777034 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7f58840019f0 con 0x7f588c0cf710 2024-05-26T22:48:28.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.658+0000 7f588aa00640 1 -- 172.21.15.131:0/4228777034 >> v1:172.21.15.131:6789/0 conn(0x7f588c0cf710 legacy=0x7f588c0b96e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:28.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.658+0000 7f588aa00640 1 --2- 172.21.15.131:0/4228777034 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f58700039b0 0x7f5870003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:28.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.658+0000 7f588aa00640 1 --2- 172.21.15.131:0/4228777034 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5870004800 0x7f5870004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:28.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.658+0000 7f588aa00640 1 --2- 172.21.15.131:0/4228777034 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f58700051c0 0x7f587000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:28.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.658+0000 7f588aa00640 1 -- 172.21.15.131:0/4228777034 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f588c0b7eb0 con 0x7f58700051c0 2024-05-26T22:48:28.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.658+0000 7f588aa00640 1 -- 172.21.15.131:0/4228777034 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f588c0b8d10 con 0x7f5870004800 2024-05-26T22:48:28.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.658+0000 7f588aa00640 1 -- 172.21.15.131:0/4228777034 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f587000c0e0 con 0x7f58700039b0 2024-05-26T22:48:28.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.658+0000 7f588b400640 1 --2- 172.21.15.131:0/4228777034 >> v2:172.21.15.131:3300/0 conn(0x7f588c0cf2b0 0x7f588c0b91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-26T22:48:28.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.656+0000 7f3f89200640 1 -- start start 2024-05-26T22:48:28.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.658+0000 7f3f89200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f7c0ead00 0x7f3f7c04fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:28.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.658+0000 7f3f89200640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f7c053720 con 0x7f3f7c006ea0 2024-05-26T22:48:28.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.658+0000 7f3f89200640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f7c053a00 con 0x7f3f7c04ffd0 2024-05-26T22:48:28.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.658+0000 7f3f89200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3f7c054330 con 0x7f3f7c0ead00 2024-05-26T22:48:28.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.659+0000 7f588b400640 1 --2- 172.21.15.131:0/4228777034 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5870004800 0x7f5870004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:28.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.659+0000 7f588be00640 1 --2- 172.21.15.131:0/4228777034 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f58700039b0 0x7f5870003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:28.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.659+0000 7f3f83400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f7c0ead00 0x7f3f7c04fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:28.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.659+0000 7f5890a00640 1 --2- 172.21.15.131:0/4228777034 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f58700051c0 0x7f587000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:28.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.660+0000 7f3f82a00640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f3f7c04ffd0 0x7f3f7c037bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.131:57610/0 (socket says 172.21.15.131:57610) 2024-05-26T22:48:28.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.660+0000 7f3f82a00640 1 -- 172.21.15.131:0/3491377688 learned_addr learned my addr 172.21.15.131:0/3491377688 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:48:28.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.659+0000 7f3f83e00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f3f7c006ea0 0x7f3f7c0382e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:48858/0 (socket says 172.21.15.131:48858) 2024-05-26T22:48:28.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.660+0000 7f64ff200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f64f800ab40 0x7f64f8009d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:28.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.661+0000 7f3f82000640 1 -- 172.21.15.131:0/3491377688 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2345945421 0 0) 0x7f3f7c053a00 con 0x7f3f7c04ffd0 2024-05-26T22:48:28.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.660+0000 7f64ff200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f64f80516e0 0x7f64f800a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:28.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.660+0000 7f64ff200640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64f8054d10 con 0x7f64f80e1310 2024-05-26T22:48:28.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.660+0000 7f64ff200640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f64f8055920 con 0x7f64f80516e0 2024-05-26T22:48:28.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.660+0000 7f64ff200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f64f8054ff0 con 0x7f64f800ab40 2024-05-26T22:48:28.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.661+0000 7f64fe800640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f64f80516e0 0x7f64f800a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:28.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.661+0000 7f64fde00640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f64f80e1310 0x7f64f803a170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.131:57624/0 (socket says 172.21.15.131:57624) 2024-05-26T22:48:28.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.661+0000 7f64fde00640 1 -- 172.21.15.131:0/4160380662 learned_addr learned my addr 172.21.15.131:0/4160380662 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:48:28.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.662+0000 7f64fd400640 1 --2- 172.21.15.131:0/4160380662 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f64f800ab40 0x7f64f8009d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:28.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.662+0000 7f64fe800640 1 -- 172.21.15.131:0/4160380662 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f64f800ab40 msgr2=0x7f64f8009d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:28.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.661+0000 7f3f82000640 1 -- 172.21.15.131:0/3491377688 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3f6c003520 con 0x7f3f7c04ffd0 2024-05-26T22:48:28.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.661+0000 7f3f82000640 1 -- 172.21.15.131:0/3491377688 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3954826163 0 0) 0x7f3f7c053720 con 0x7f3f7c006ea0 2024-05-26T22:48:28.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.661+0000 7f3f82000640 1 -- 172.21.15.131:0/3491377688 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3f7c053a00 con 0x7f3f7c006ea0 2024-05-26T22:48:28.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.662+0000 7f3f82000640 1 -- 172.21.15.131:0/3491377688 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3720088671 0 0) 0x7f3f7c053a00 con 0x7f3f7c006ea0 2024-05-26T22:48:28.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.662+0000 7f3f82000640 1 -- 172.21.15.131:0/3491377688 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f7c0ead00 msgr2=0x7f3f7c04fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:28.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.662+0000 7f3f82000640 1 --2- 172.21.15.131:0/3491377688 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f7c0ead00 0x7f3f7c04fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:28.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.662+0000 7f3f82000640 1 -- 172.21.15.131:0/3491377688 >> v1:172.21.15.187:6789/0 conn(0x7f3f7c04ffd0 legacy=0x7f3f7c037bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:28.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.662+0000 7f3f82000640 1 -- 172.21.15.131:0/3491377688 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f7c053ce0 con 0x7f3f7c006ea0 2024-05-26T22:48:28.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.662+0000 7f3f82000640 1 -- 172.21.15.131:0/3491377688 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7f3f70001670 con 0x7f3f7c006ea0 2024-05-26T22:48:28.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.662+0000 7f3f82000640 1 -- 172.21.15.131:0/3491377688 >> v1:172.21.15.131:6789/0 conn(0x7f3f7c006ea0 legacy=0x7f3f7c0382e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:28.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.662+0000 7f3f82000640 1 --2- 172.21.15.131:0/3491377688 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f6c004bd0 0x7f3f6c004fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:28.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.662+0000 7f3f82000640 1 --2- 172.21.15.131:0/3491377688 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3f6c0037a0 0x7f3f6c003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:28.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.662+0000 7f3f82000640 1 --2- 172.21.15.131:0/3491377688 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f6c004160 0x7f3f6c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:28.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.662+0000 7f3f82000640 1 -- 172.21.15.131:0/3491377688 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f3f7c054330 con 0x7f3f6c004bd0 2024-05-26T22:48:28.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.662+0000 7f3f82000640 1 -- 172.21.15.131:0/3491377688 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3f6c01d0a0 con 0x7f3f6c0037a0 2024-05-26T22:48:28.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.662+0000 7f3f82000640 1 -- 172.21.15.131:0/3491377688 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3f6c01d220 con 0x7f3f6c004160 2024-05-26T22:48:28.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.661+0000 7f588b400640 1 -- 172.21.15.131:0/4228777034 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f58700039b0 msgr2=0x7f5870003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:28.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.661+0000 7f588b400640 1 --2- 172.21.15.131:0/4228777034 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f58700039b0 0x7f5870003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:28.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.661+0000 7f588b400640 1 -- 172.21.15.131:0/4228777034 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f58700051c0 msgr2=0x7f587000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:28.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.661+0000 7f588b400640 1 --2- 172.21.15.131:0/4228777034 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f58700051c0 0x7f587000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:28.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.661+0000 7f588b400640 1 -- 172.21.15.131:0/4228777034 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f588c0b8100 con 0x7f5870004800 2024-05-26T22:48:28.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.662+0000 7f5890a00640 1 -- 172.21.15.131:0/4228777034 reap_dead start 2024-05-26T22:48:28.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.663+0000 7f3f83e00640 1 --2- 172.21.15.131:0/3491377688 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f6c004160 0x7f3f6c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:28.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.663+0000 7f8257800640 1 Processor -- start 2024-05-26T22:48:28.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.663+0000 7f8257800640 1 -- start start 2024-05-26T22:48:28.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.662+0000 7f64fe800640 1 --2- 172.21.15.131:0/4160380662 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f64f800ab40 0x7f64f8009d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:28.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.662+0000 7f64fe800640 1 -- 172.21.15.131:0/4160380662 >> v1:172.21.15.187:6789/0 conn(0x7f64f80e1310 legacy=0x7f64f803a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:28.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.662+0000 7f64fe800640 1 -- 172.21.15.131:0/4160380662 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64f80552d0 con 0x7f64f80516e0 2024-05-26T22:48:28.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.664+0000 7f588b400640 1 --2- 172.21.15.131:0/4228777034 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5870004800 0x7f5870004c80 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5878009c90 tx=0x7f5870005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99d5867dbb8dc3c3 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:28.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.664+0000 7fcbcf600640 1 Processor -- start 2024-05-26T22:48:28.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.664+0000 7f3f83e00640 1 -- 172.21.15.131:0/3491377688 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3f6c0037a0 msgr2=0x7f3f6c003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:28.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.664+0000 7f3f83e00640 1 --2- 172.21.15.131:0/3491377688 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3f6c0037a0 0x7f3f6c003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:28.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.664+0000 7f3f83e00640 1 -- 172.21.15.131:0/3491377688 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f6c004bd0 msgr2=0x7f3f6c004fb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-26T22:48:28.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.664+0000 7f3f83400640 1 --2- 172.21.15.131:0/3491377688 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f6c004bd0 0x7f3f6c004fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:28.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.664+0000 7fcbcf600640 1 -- start start 2024-05-26T22:48:28.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.664+0000 7f3f83e00640 1 --2- 172.21.15.131:0/3491377688 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f6c004bd0 0x7f3f6c004fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:28.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.664+0000 7f3f83e00640 1 -- 172.21.15.131:0/3491377688 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f7c053ce0 con 0x7f3f6c004160 2024-05-26T22:48:28.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.664+0000 7f3f83400640 1 --2- 172.21.15.131:0/3491377688 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f6c004bd0 0x7f3f6c004fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:48:28.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.665+0000 7f588aa00640 1 -- 172.21.15.131:0/4228777034 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5878036070 con 0x7f5870004800 2024-05-26T22:48:28.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.665+0000 7f588aa00640 1 -- 172.21.15.131:0/4228777034 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5878031400 con 0x7f5870004800 2024-05-26T22:48:28.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.665+0000 7f64fe800640 1 --2- 172.21.15.131:0/4160380662 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f64f80516e0 0x7f64f800a290 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f64f00097a0 tx=0x7f64f002a150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23bb32899ef9a032 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:28.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.665+0000 7f588aa00640 1 -- 172.21.15.131:0/4228777034 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5878009860 con 0x7f5870004800 2024-05-26T22:48:28.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.666+0000 7f8257800640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f82500d0560 0x7f8250072540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:28.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.666+0000 7f5891400640 1 -- 172.21.15.131:0/4228777034 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5870004800 msgr2=0x7f5870004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:28.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.666+0000 7f5891400640 1 --2- 172.21.15.131:0/4228777034 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5870004800 0x7f5870004c80 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5878009c90 tx=0x7f5870005830 comp rx=0 tx=0).stop 2024-05-26T22:48:28.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.666+0000 7f8257800640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f82500af230 0x7f82500cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:28.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.666+0000 7fcbcf600640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fcbc815e4e0 0x7fcbc80b9c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:28.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.666+0000 7fcbcf600640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fcbc80d2330 0x7fcbc80cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:28.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.666+0000 7f8257800640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82500733c0 con 0x7f8250165e30 2024-05-26T22:48:28.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.666+0000 7f8257800640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f8250072e90 con 0x7f82500d0560 2024-05-26T22:48:28.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.666+0000 7f8257800640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f82500cf540 con 0x7f82500af230 2024-05-26T22:48:28.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.666+0000 7fcbcf600640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcbc80d28f0 con 0x7fcbc80cf550 2024-05-26T22:48:28.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.666+0000 7fcbcf600640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fcbc80d2bd0 con 0x7fcbc815e4e0 2024-05-26T22:48:28.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.666+0000 7fcbcf600640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fcbc80d2eb0 con 0x7fcbc80d2330 2024-05-26T22:48:28.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.666+0000 7f8254e00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f8250165e30 0x7f82500cfe50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:48862/0 (socket says 172.21.15.131:48862) 2024-05-26T22:48:28.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.666+0000 7f8254e00640 1 -- 172.21.15.131:0/3216383424 learned_addr learned my addr 172.21.15.131:0/3216383424 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:48:28.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.666+0000 7f5891400640 1 -- 172.21.15.131:0/4228777034 shutdown_connections 2024-05-26T22:48:28.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.666+0000 7f5891400640 1 --2- 172.21.15.131:0/4228777034 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5870004800 0x7f5870004c80 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:28.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.666+0000 7f5891400640 1 -- 172.21.15.131:0/4228777034 >> 172.21.15.131:0/4228777034 conn(0x7f588c072e60 msgr2=0x7f588c073270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:28.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.667+0000 7f3f83e00640 1 --2- 172.21.15.131:0/3491377688 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f6c004160 0x7f3f6c00b9c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f3f7000c1c0 tx=0x7f3f6c005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb47cb2a5fbb4dac server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:28.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.666+0000 7f824fe00640 1 --2- 172.21.15.131:0/3216383424 >> v2:172.21.15.131:3300/0 conn(0x7f82500d0560 0x7f8250072540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:28.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.667+0000 7f5891400640 1 -- 172.21.15.131:0/4228777034 shutdown_connections 2024-05-26T22:48:28.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.667+0000 7fcbccc00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fcbc815e4e0 0x7fcbc80b9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:28.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.667+0000 7fcbccc00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fcbc815e4e0 0x7fcbc80b9c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54000/0 (socket says 172.21.15.131:54000) 2024-05-26T22:48:28.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.667+0000 7fcbccc00640 1 -- 172.21.15.131:0/1535701842 learned_addr learned my addr 172.21.15.131:0/1535701842 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:28.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.667+0000 7f5891400640 1 -- 172.21.15.131:0/4228777034 wait complete. 2024-05-26T22:48:28.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.667+0000 7f8255800640 1 --2- 172.21.15.131:0/3216383424 >> v2:172.21.15.187:3300/0 conn(0x7f82500af230 0x7f82500cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:28.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.667+0000 7fcbc7e00640 1 --2- 172.21.15.131:0/1535701842 >> v2:172.21.15.187:3300/0 conn(0x7fcbc80d2330 0x7fcbc80cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:28.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.667+0000 7fcbccc00640 1 -- 172.21.15.131:0/1535701842 >> v2:172.21.15.187:3300/0 conn(0x7fcbc80d2330 msgr2=0x7fcbc80cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:28.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.667+0000 7fcbccc00640 1 --2- 172.21.15.131:0/1535701842 >> v2:172.21.15.187:3300/0 conn(0x7fcbc80d2330 0x7fcbc80cf010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:28.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.667+0000 7fcbccc00640 1 -- 172.21.15.131:0/1535701842 >> v1:172.21.15.187:6789/0 conn(0x7fcbc80cf550 legacy=0x7fcbc80d3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:28.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.667+0000 7fcbccc00640 1 -- 172.21.15.131:0/1535701842 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbc80d3190 con 0x7fcbc815e4e0 2024-05-26T22:48:28.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.668+0000 7f824fe00640 1 -- 172.21.15.131:0/3216383424 >> v2:172.21.15.187:3300/0 conn(0x7f82500af230 msgr2=0x7f82500cdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:28.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.668+0000 7f824fe00640 1 --2- 172.21.15.131:0/3216383424 >> v2:172.21.15.187:3300/0 conn(0x7f82500af230 0x7f82500cdcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:28.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.668+0000 7f824f400640 1 -- 172.21.15.131:0/3216383424 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 532378845 0 0) 0x7f82500733c0 con 0x7f8250165e30 2024-05-26T22:48:28.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.668+0000 7f824fe00640 1 -- 172.21.15.131:0/3216383424 >> v1:172.21.15.131:6789/0 conn(0x7f8250165e30 legacy=0x7f82500cfe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:28.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.668+0000 7f824fe00640 1 -- 172.21.15.131:0/3216383424 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82500730e0 con 0x7f82500d0560 2024-05-26T22:48:28.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.669+0000 7f3f82000640 1 -- 172.21.15.131:0/3491377688 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f70007c70 con 0x7f3f6c004160 2024-05-26T22:48:28.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.669+0000 7f3f82000640 1 -- 172.21.15.131:0/3491377688 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f70007270 con 0x7f3f6c004160 2024-05-26T22:48:28.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.669+0000 7f3f82000640 1 -- 172.21.15.131:0/3491377688 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f70007c70 con 0x7f3f6c004160 2024-05-26T22:48:28.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.669+0000 7fcbccc00640 1 --2- 172.21.15.131:0/1535701842 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcbc815e4e0 0x7fcbc80b9c80 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fcbc80d0cb0 tx=0x7fcbb402a1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6d96ed71e95de7d server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:28.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.669+0000 7f824fe00640 1 --2- 172.21.15.131:0/3216383424 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f82500d0560 0x7f8250072540 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f82500d0fa0 tx=0x7f824002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f276046a7a639120 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:28.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.669+0000 7fcbc7400640 1 -- 172.21.15.131:0/1535701842 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb4037070 con 0x7fcbc815e4e0 2024-05-26T22:48:28.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.670+0000 7fcbc7400640 1 -- 172.21.15.131:0/1535701842 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbb4032440 con 0x7fcbc815e4e0 2024-05-26T22:48:28.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.670+0000 7f5891400640 1 Processor -- start 2024-05-26T22:48:28.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.670+0000 7f3f89200640 1 -- 172.21.15.131:0/3491377688 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f6c004160 msgr2=0x7f3f6c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:28.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.670+0000 7f3f89200640 1 --2- 172.21.15.131:0/3491377688 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f6c004160 0x7f3f6c00b9c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f3f7000c1c0 tx=0x7f3f6c005830 comp rx=0 tx=0).stop 2024-05-26T22:48:28.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.670+0000 7f5891400640 1 -- start start 2024-05-26T22:48:28.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.670+0000 7f64fca00640 1 -- 172.21.15.131:0/4160380662 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f0037070 con 0x7f64f80516e0 2024-05-26T22:48:28.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.670+0000 7f824f400640 1 -- 172.21.15.131:0/3216383424 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8240037070 con 0x7f82500d0560 2024-05-26T22:48:28.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.670+0000 7f824f400640 1 -- 172.21.15.131:0/3216383424 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8240032440 con 0x7f82500d0560 2024-05-26T22:48:28.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.670+0000 7f824f400640 1 -- 172.21.15.131:0/3216383424 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f824002ae60 con 0x7f82500d0560 2024-05-26T22:48:28.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.670+0000 7f64fca00640 1 -- 172.21.15.131:0/4160380662 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64f0032440 con 0x7f64f80516e0 2024-05-26T22:48:28.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.670+0000 7f64fca00640 1 -- 172.21.15.131:0/4160380662 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f002ae60 con 0x7f64f80516e0 2024-05-26T22:48:28.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.670+0000 7fcbc7400640 1 -- 172.21.15.131:0/1535701842 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb4032810 con 0x7fcbc815e4e0 2024-05-26T22:48:28.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.671+0000 7ff3c7800640 1 Processor -- start 2024-05-26T22:48:28.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.670+0000 7f3f83e00640 1 -- 172.21.15.131:0/3491377688 reap_dead start 2024-05-26T22:48:28.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.671+0000 7f3f89200640 1 -- 172.21.15.131:0/3491377688 shutdown_connections 2024-05-26T22:48:28.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.671+0000 7f3f89200640 1 -- 172.21.15.131:0/3491377688 >> 172.21.15.131:0/3491377688 conn(0x7f3f7c009fb0 msgr2=0x7f3f7c00a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:28.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.671+0000 7fcbcf600640 1 -- 172.21.15.131:0/1535701842 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcbc815e4e0 msgr2=0x7fcbc80b9c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:28.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.671+0000 7fcbcf600640 1 --2- 172.21.15.131:0/1535701842 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcbc815e4e0 0x7fcbc80b9c80 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fcbc80d0cb0 tx=0x7fcbb402a1f0 comp rx=0 tx=0).stop 2024-05-26T22:48:28.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.671+0000 7ff3c7800640 1 -- start start 2024-05-26T22:48:28.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.671+0000 7f3f89200640 1 -- 172.21.15.131:0/3491377688 shutdown_connections 2024-05-26T22:48:28.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.671+0000 7f3f89200640 1 -- 172.21.15.131:0/3491377688 wait complete. 2024-05-26T22:48:28.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.671+0000 7f8257800640 1 -- 172.21.15.131:0/3216383424 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f82500d0560 msgr2=0x7f8250072540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:28.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.671+0000 7f8257800640 1 --2- 172.21.15.131:0/3216383424 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f82500d0560 0x7f8250072540 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f82500d0fa0 tx=0x7f824002a110 comp rx=0 tx=0).stop 2024-05-26T22:48:28.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.672+0000 7f64ff200640 1 -- 172.21.15.131:0/4160380662 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f64f80516e0 msgr2=0x7f64f800a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:28.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.672+0000 7fcbcf600640 1 -- 172.21.15.131:0/1535701842 shutdown_connections 2024-05-26T22:48:28.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.672+0000 7f64ff200640 1 --2- 172.21.15.131:0/4160380662 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f64f80516e0 0x7f64f800a290 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f64f00097a0 tx=0x7f64f002a150 comp rx=0 tx=0).stop 2024-05-26T22:48:28.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.672+0000 7f5891400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5870004800 0x7f588c072080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:28.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.672+0000 7f5891400640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f588c15c790 0x7f588c203e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:28.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.673+0000 7f5891400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f588c2043b0 0x7f588c204830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:28.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.672+0000 7f64ff200640 1 -- 172.21.15.131:0/4160380662 shutdown_connections 2024-05-26T22:48:28.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.672+0000 7f64ff200640 1 --2- 172.21.15.131:0/4160380662 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f64f80516e0 0x7f64f800a290 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:28.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.672+0000 7f64ff200640 1 --2- 172.21.15.131:0/4160380662 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f64f800ab40 0x7f64f8009d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:28.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.672+0000 7f64ff200640 1 -- 172.21.15.131:0/4160380662 >> 172.21.15.131:0/4160380662 conn(0x7f64f8056cd0 msgr2=0x7f64f80570e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:28.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.672+0000 7fcbcf600640 1 --2- 172.21.15.131:0/1535701842 >> v2:172.21.15.187:3300/0 conn(0x7fcbc80d2330 0x7fcbc80cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:28.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.672+0000 7fcbcf600640 1 --2- 172.21.15.131:0/1535701842 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcbc815e4e0 0x7fcbc80b9c80 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:28.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.672+0000 7fcbcf600640 1 -- 172.21.15.131:0/1535701842 >> 172.21.15.131:0/1535701842 conn(0x7fcbc8072e90 msgr2=0x7fcbc80732a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:28.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.672+0000 7f8257800640 1 -- 172.21.15.131:0/3216383424 shutdown_connections 2024-05-26T22:48:28.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.672+0000 7f8257800640 1 --2- 172.21.15.131:0/3216383424 >> v2:172.21.15.187:3300/0 conn(0x7f82500af230 0x7f82500cdcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:28.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.672+0000 7f8257800640 1 --2- 172.21.15.131:0/3216383424 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f82500d0560 0x7f8250072540 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:28.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.672+0000 7f8257800640 1 -- 172.21.15.131:0/3216383424 >> 172.21.15.131:0/3216383424 conn(0x7f825006bc90 msgr2=0x7f82500b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:28.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.673+0000 7fcbcf600640 1 -- 172.21.15.131:0/1535701842 shutdown_connections 2024-05-26T22:48:28.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.673+0000 7fcbcf600640 1 -- 172.21.15.131:0/1535701842 wait complete. 2024-05-26T22:48:28.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.673+0000 7f5891400640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f588c0b89a0 con 0x7f588c2043b0 2024-05-26T22:48:28.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.673+0000 7f64ff200640 1 -- 172.21.15.131:0/4160380662 shutdown_connections 2024-05-26T22:48:28.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.673+0000 7f8257800640 1 -- 172.21.15.131:0/3216383424 shutdown_connections 2024-05-26T22:48:28.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.673+0000 7ff3c7800640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7ff3c0165e30 0x7ff3c0072540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:28.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.673+0000 7f8257800640 1 -- 172.21.15.131:0/3216383424 wait complete. 2024-05-26T22:48:28.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.673+0000 7ff3c7800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff3c00d0560 0x7ff3c00cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:28.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.673+0000 7ff3c7800640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3c00733c0 con 0x7ff3c00af230 2024-05-26T22:48:28.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.673+0000 7ff3c7800640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7ff3c0072e90 con 0x7ff3c0165e30 2024-05-26T22:48:28.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.673+0000 7ff3c7800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff3c00cf540 con 0x7ff3c00d0560 2024-05-26T22:48:28.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.673+0000 7f11c2200640 1 Processor -- start 2024-05-26T22:48:28.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.673+0000 7f5891400640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f588c0b86c0 con 0x7f588c15c790 2024-05-26T22:48:28.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.673+0000 7f5891400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f588c15e680 con 0x7f5870004800 2024-05-26T22:48:28.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.673+0000 7f588b400640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f588c15c790 0x7f588c203e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:28.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.673+0000 7f11c2200640 1 -- start start 2024-05-26T22:48:28.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.674+0000 7f588b400640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f588c15c790 0x7f588c203e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.131:36372/0 (socket says 172.21.15.131:36372) 2024-05-26T22:48:28.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.674+0000 7ff3bfe00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7ff3c00af230 0x7ff3c00cfe50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:48872/0 (socket says 172.21.15.131:48872) 2024-05-26T22:48:28.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.674+0000 7ff3bfe00640 1 -- 172.21.15.131:0/3593781002 learned_addr learned my addr 172.21.15.131:0/3593781002 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:48:28.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.674+0000 7ff3c4e00640 1 --2- 172.21.15.131:0/3593781002 >> v2:172.21.15.131:3300/0 conn(0x7ff3c0165e30 0x7ff3c0072540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:28.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.674+0000 7f3f89200640 1 Processor -- start 2024-05-26T22:48:28.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.674+0000 7f588be00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5870004800 0x7f588c072080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:28.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.674+0000 7f588be00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5870004800 0x7f588c072080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.131:43206/0 (socket says 172.21.15.131:43206) 2024-05-26T22:48:28.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.674+0000 7f588be00640 1 -- 172.21.15.131:0/476063634 learned_addr learned my addr 172.21.15.131:0/476063634 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:28.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.674+0000 7f3f89200640 1 -- start start 2024-05-26T22:48:28.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.675+0000 7f5890a00640 1 --2- 172.21.15.131:0/476063634 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f588c2043b0 0x7f588c204830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:28.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.674+0000 7ff3c5800640 1 --2- 172.21.15.131:0/3593781002 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff3c00d0560 0x7ff3c00cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:28.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.675+0000 7ff3c4e00640 1 -- 172.21.15.131:0/3593781002 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff3c00d0560 msgr2=0x7ff3c00cdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:28.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.675+0000 7ff3c4e00640 1 --2- 172.21.15.131:0/3593781002 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff3c00d0560 0x7ff3c00cdcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:28.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.675+0000 7ff3c4e00640 1 -- 172.21.15.131:0/3593781002 >> v1:172.21.15.131:6789/0 conn(0x7ff3c00af230 legacy=0x7ff3c00cfe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:28.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.675+0000 7ff3bf400640 1 -- 172.21.15.131:0/3593781002 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1830196341 0 0) 0x7ff3c00733c0 con 0x7ff3c00af230 2024-05-26T22:48:28.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.675+0000 7fcbcf600640 1 Processor -- start 2024-05-26T22:48:28.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.675+0000 7f5890a00640 1 -- 172.21.15.131:0/476063634 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5870004800 msgr2=0x7f588c072080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:28.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.675+0000 7f11c2200640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f11bc06bdf0 0x7f11bc072ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:28.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.676+0000 7f5890a00640 1 --2- 172.21.15.131:0/476063634 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5870004800 0x7f588c072080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:28.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.676+0000 7fcbcf600640 1 -- start start 2024-05-26T22:48:28.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.676+0000 7f5890a00640 1 -- 172.21.15.131:0/476063634 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f588c15c790 msgr2=0x7f588c203e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:28.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.676+0000 7f5890a00640 1 --2- 172.21.15.131:0/476063634 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f588c15c790 0x7f588c203e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:28.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.676+0000 7f8257800640 1 Processor -- start 2024-05-26T22:48:28.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.675+0000 7f11c2200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f11bc15a630 0x7f11bc071360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:28.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.676+0000 7f8257800640 1 -- start start 2024-05-26T22:48:28.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.675+0000 7ff3c4e00640 1 -- 172.21.15.131:0/3593781002 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3c00730e0 con 0x7ff3c0165e30 2024-05-26T22:48:28.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.676+0000 7f11bb400640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f11bc06bdf0 0x7f11bc072ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:28.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.676+0000 7f5890a00640 1 -- 172.21.15.131:0/476063634 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f588c06f7e0 con 0x7f588c2043b0 2024-05-26T22:48:28.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.676+0000 7f588b400640 1 --2- 172.21.15.131:0/476063634 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f588c15c790 0x7f588c203e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-26T22:48:28.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.676+0000 7f11bb400640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f11bc06bdf0 0x7f11bc072ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.131:36378/0 (socket says 172.21.15.131:36378) 2024-05-26T22:48:28.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.676+0000 7f11bbe00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f11bc0cfe50 0x7f11bc0cda90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:48886/0 (socket says 172.21.15.131:48886) 2024-05-26T22:48:28.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.676+0000 7f11bbe00640 1 -- 172.21.15.131:0/2451021581 learned_addr learned my addr 172.21.15.131:0/2451021581 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:48:28.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.677+0000 7f3f89200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f6c004160 0x7f3f7c04d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:28.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.677+0000 7f5890a00640 1 --2- 172.21.15.131:0/476063634 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f588c2043b0 0x7f588c204830 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f588c0cfaf0 tx=0x7f5884007430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:28.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.677+0000 7ff3c4e00640 1 --2- 172.21.15.131:0/3593781002 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff3c0165e30 0x7ff3c0072540 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff3b80097a0 tx=0x7ff3b802a0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f42d8f9865aa582f server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:28.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.677+0000 7f11c2200640 1 -- 172.21.15.131:0/2451021581 --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11bc0734c0 con 0x7f11bc0cfe50 2024-05-26T22:48:28.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.677+0000 7f11b2a00640 1 --2- 172.21.15.131:0/2451021581 >> v2:172.21.15.131:3300/0 conn(0x7f11bc15a630 0x7f11bc071360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:28.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.677+0000 7f3f89200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f7c0d8a40 0x7f3f7c036b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:28.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.677+0000 7ff3bf400640 1 -- 172.21.15.131:0/3593781002 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b8037070 con 0x7ff3c0165e30 2024-05-26T22:48:28.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.677+0000 7f3f89200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3f7c037080 0x7f3f7c0dcdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:28.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.677+0000 7f3f89200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f3f7c054940 con 0x7f3f6c004160 2024-05-26T22:48:28.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.677+0000 7f3f89200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3f7c0547c0 con 0x7f3f7c037080 2024-05-26T22:48:28.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.677+0000 7f3f89200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3f7c04f550 con 0x7f3f7c0d8a40 2024-05-26T22:48:28.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.677+0000 7f11c2200640 1 -- 172.21.15.131:0/2451021581 --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f11bc15e0a0 con 0x7f11bc15a630 2024-05-26T22:48:28.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.677+0000 7f11c2200640 1 -- 172.21.15.131:0/2451021581 --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f11bc15e940 con 0x7f11bc06bdf0 2024-05-26T22:48:28.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.675+0000 7f64ff200640 1 -- 172.21.15.131:0/4160380662 wait complete. 2024-05-26T22:48:28.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.677+0000 7f8257800640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f82500af230 0x7f82500ada30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:28.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.677+0000 7f8257800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f82500d0560 0x7f825009aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:28.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.678+0000 7fcbcf600640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcbc80cf550 0x7fcbc80cc180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:28.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.677+0000 7f8257800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8250165e30 0x7f825009af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:28.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.677+0000 7f64ff200640 1 Processor -- start 2024-05-26T22:48:28.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.677+0000 7ff3bf400640 1 -- 172.21.15.131:0/3593781002 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3b802ae40 con 0x7ff3c0165e30 2024-05-26T22:48:28.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.677+0000 7ff3bf400640 1 -- 172.21.15.131:0/3593781002 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b8037070 con 0x7ff3c0165e30 2024-05-26T22:48:28.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.678+0000 7f11baa00640 1 -- 172.21.15.131:0/2451021581 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1830381454 0 0) 0x7f11bc0734c0 con 0x7f11bc0cfe50 2024-05-26T22:48:28.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.678+0000 7f11baa00640 1 -- 172.21.15.131:0/2451021581 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f11a0003520 con 0x7f11bc0cfe50 2024-05-26T22:48:28.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.678+0000 7fcbcf600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcbc80d2330 0x7fcbc80b8da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:28.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.678+0000 7f64ff200640 1 -- start start 2024-05-26T22:48:28.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.678+0000 7fcbcf600640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcbc815e4e0 0x7fcbc80b92e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:28.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.678+0000 7f3f83e00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3f7c037080 0x7f3f7c0dcdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:28.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.678+0000 7f3f83400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f6c004160 0x7f3f7c04d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:28.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.678+0000 7f3f83e00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3f7c037080 0x7f3f7c0dcdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.131:36384/0 (socket says 172.21.15.131:36384) 2024-05-26T22:48:28.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.678+0000 7f3f83e00640 1 -- 172.21.15.131:0/2584714376 learned_addr learned my addr 172.21.15.131:0/2584714376 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:28.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.678+0000 7fcbcf600640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc80d3990 con 0x7fcbc80d2330 2024-05-26T22:48:28.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.677+0000 7f8257800640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f82500ce210 con 0x7f8250165e30 2024-05-26T22:48:28.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.678+0000 7fcbcf600640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc815e8c0 con 0x7fcbc815e4e0 2024-05-26T22:48:28.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.678+0000 7fcbcf600640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fcbc80ba1c0 con 0x7fcbc80cf550 2024-05-26T22:48:28.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.678+0000 7f8257800640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f82500cfac0 con 0x7f82500af230 2024-05-26T22:48:28.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.678+0000 7f11bb400640 1 -- 172.21.15.131:0/2451021581 >> v2:172.21.15.131:3300/0 conn(0x7f11bc15a630 msgr2=0x7f11bc071360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:28.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.678+0000 7f5883e00640 1 -- 172.21.15.131:0/476063634 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f588401a070 con 0x7f588c2043b0 2024-05-26T22:48:28.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.678+0000 7fcbccc00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcbc80cf550 0x7fcbc80cc180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:28.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.678+0000 7f11bb400640 1 --2- 172.21.15.131:0/2451021581 >> v2:172.21.15.131:3300/0 conn(0x7f11bc15a630 0x7f11bc071360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:28.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.678+0000 7f11bb400640 1 -- 172.21.15.131:0/2451021581 >> v1:172.21.15.131:6789/0 conn(0x7f11bc0cfe50 legacy=0x7f11bc0cda90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:28.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.678+0000 7f11bb400640 1 -- 172.21.15.131:0/2451021581 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11bc15e2f0 con 0x7f11bc06bdf0 2024-05-26T22:48:28.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.678+0000 7f8257800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f82500ce390 con 0x7f82500d0560 2024-05-26T22:48:28.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.678+0000 7fcbccc00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcbc80cf550 0x7fcbc80cc180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.131:43244/0 (socket says 172.21.15.131:43244) 2024-05-26T22:48:28.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.678+0000 7fcbccc00640 1 -- 172.21.15.131:0/1162736769 learned_addr learned my addr 172.21.15.131:0/1162736769 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:28.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.679+0000 7f5891400640 1 -- 172.21.15.131:0/476063634 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f588c06fa80 con 0x7f588c2043b0 2024-05-26T22:48:28.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.679+0000 7fcbcd600640 1 --2- 172.21.15.131:0/1162736769 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcbc815e4e0 0x7fcbc80b92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:28.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.678+0000 7f3f82a00640 1 --2- 172.21.15.131:0/2584714376 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f7c0d8a40 0x7f3f7c036b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:28.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.679+0000 7f8255800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8250165e30 0x7f825009af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:28.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.679+0000 7f8255800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8250165e30 0x7f825009af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54050/0 (socket says 172.21.15.131:54050) 2024-05-26T22:48:28.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.679+0000 7f8255800640 1 -- 172.21.15.131:0/1426942812 learned_addr learned my addr 172.21.15.131:0/1426942812 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:28.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.679+0000 7f3f83e00640 1 -- 172.21.15.131:0/2584714376 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f7c0d8a40 msgr2=0x7f3f7c036b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:28.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.679+0000 7f3f83e00640 1 --2- 172.21.15.131:0/2584714376 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f7c0d8a40 0x7f3f7c036b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:28.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.679+0000 7f3f83e00640 1 -- 172.21.15.131:0/2584714376 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f6c004160 msgr2=0x7f3f7c04d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:28.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.679+0000 7f3f83e00640 1 --2- 172.21.15.131:0/2584714376 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f6c004160 0x7f3f7c04d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:28.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.679+0000 7f3f83e00640 1 -- 172.21.15.131:0/2584714376 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f7c04de90 con 0x7f3f7c037080 2024-05-26T22:48:28.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.679+0000 7f5891400640 1 -- 172.21.15.131:0/476063634 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f588c0b32e0 con 0x7f588c2043b0 2024-05-26T22:48:28.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.679+0000 7ff3c7800640 1 -- 172.21.15.131:0/3593781002 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff3c0165e30 msgr2=0x7ff3c0072540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:28.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.679+0000 7ff3c7800640 1 --2- 172.21.15.131:0/3593781002 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff3c0165e30 0x7ff3c0072540 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff3b80097a0 tx=0x7ff3b802a0f0 comp rx=0 tx=0).stop 2024-05-26T22:48:28.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.679+0000 7fcbc7e00640 1 --2- 172.21.15.131:0/1162736769 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcbc80d2330 0x7fcbc80b8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:28.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.679+0000 7f5883e00640 1 -- 172.21.15.131:0/476063634 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f588400add0 con 0x7f588c2043b0 2024-05-26T22:48:28.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.680+0000 7fcbccc00640 1 -- 172.21.15.131:0/1162736769 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcbc815e4e0 msgr2=0x7fcbc80b92e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:28.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.680+0000 7fcbccc00640 1 --2- 172.21.15.131:0/1162736769 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcbc815e4e0 0x7fcbc80b92e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:28.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.680+0000 7fcbccc00640 1 -- 172.21.15.131:0/1162736769 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcbc80d2330 msgr2=0x7fcbc80b8da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:28.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.680+0000 7fcbccc00640 1 --2- 172.21.15.131:0/1162736769 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcbc80d2330 0x7fcbc80b8da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:28.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.680+0000 7fcbccc00640 1 -- 172.21.15.131:0/1162736769 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbc80b7d40 con 0x7fcbc80cf550 2024-05-26T22:48:28.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.680+0000 7f5883e00640 1 -- 172.21.15.131:0/476063634 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f588401f640 con 0x7f588c2043b0 2024-05-26T22:48:28.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.679+0000 7f824fe00640 1 --2- 172.21.15.131:0/1426942812 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f82500d0560 0x7f825009aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:28.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.679+0000 7f8254e00640 1 --2- 172.21.15.131:0/1426942812 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f82500af230 0x7f82500ada30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:28.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.680+0000 7ff3c7800640 1 -- 172.21.15.131:0/3593781002 shutdown_connections 2024-05-26T22:48:28.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.680+0000 7ff3c7800640 1 --2- 172.21.15.131:0/3593781002 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff3c00d0560 0x7ff3c00cdcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:28.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.680+0000 7ff3c7800640 1 --2- 172.21.15.131:0/3593781002 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff3c0165e30 0x7ff3c0072540 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:28.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.680+0000 7ff3c7800640 1 -- 172.21.15.131:0/3593781002 >> 172.21.15.131:0/3593781002 conn(0x7ff3c006bc90 msgr2=0x7ff3c00b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:28.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.680+0000 7f64ff200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f64f800ab40 0x7f64f817cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:28.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.680+0000 7f64ff200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f64f80516e0 0x7f64f81898d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:28.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.680+0000 7f64ff200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f64f80e1310 0x7f64f8189e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:28.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.681+0000 7f64ff200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f64f8054a20 con 0x7f64f80516e0 2024-05-26T22:48:28.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.680+0000 7fcbc7e00640 1 --2- 172.21.15.131:0/1162736769 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcbc80d2330 0x7fcbc80b8da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:48:28.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.680+0000 7fcbccc00640 1 --2- 172.21.15.131:0/1162736769 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcbc80cf550 0x7fcbc80cc180 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fcbb402a230 tx=0x7fcbb40046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:28.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.680+0000 7f3f83e00640 1 --2- 172.21.15.131:0/2584714376 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3f7c037080 0x7f3f7c0dcdd0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3f7002add0 tx=0x7f3f70001970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:28.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.681+0000 7f64ff200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f64f80548a0 con 0x7f64f800ab40 2024-05-26T22:48:28.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.681+0000 7f64ff200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f64f8055db0 con 0x7f64f80e1310 2024-05-26T22:48:28.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.681+0000 7f3a5d600640 1 Processor -- start 2024-05-26T22:48:28.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.681+0000 7f3a5d600640 1 -- start start 2024-05-26T22:48:28.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.681+0000 7f824fe00640 1 -- 172.21.15.131:0/1426942812 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f82500af230 msgr2=0x7f82500ada30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:28.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.681+0000 7f824fe00640 1 --2- 172.21.15.131:0/1426942812 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f82500af230 0x7f82500ada30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:28.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.681+0000 7f824fe00640 1 -- 172.21.15.131:0/1426942812 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8250165e30 msgr2=0x7f825009af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:28.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.681+0000 7f824fe00640 1 --2- 172.21.15.131:0/1426942812 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8250165e30 0x7f825009af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:28.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.681+0000 7fcbc4c00640 1 -- 172.21.15.131:0/1162736769 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb4041020 con 0x7fcbc80cf550 2024-05-26T22:48:28.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.682+0000 7fcbc4c00640 1 -- 172.21.15.131:0/1162736769 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbb40306c0 con 0x7fcbc80cf550 2024-05-26T22:48:28.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.682+0000 7fcbc4c00640 1 -- 172.21.15.131:0/1162736769 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb4037040 con 0x7fcbc80cf550 2024-05-26T22:48:28.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.681+0000 7f824fe00640 1 -- 172.21.15.131:0/1426942812 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f825009b480 con 0x7f82500d0560 2024-05-26T22:48:28.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.681+0000 7f8255800640 1 --2- 172.21.15.131:0/1426942812 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8250165e30 0x7f825009af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-26T22:48:28.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.681+0000 7f64fd400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f64f80516e0 0x7f64f81898d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:28.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.682+0000 7f64fd400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f64f80516e0 0x7f64f81898d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54064/0 (socket says 172.21.15.131:54064) 2024-05-26T22:48:28.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.682+0000 7f64fd400640 1 -- 172.21.15.131:0/2786745872 learned_addr learned my addr 172.21.15.131:0/2786745872 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:28.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.682+0000 7fcbcf600640 1 -- 172.21.15.131:0/1162736769 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbc80b7fe0 con 0x7fcbc80cf550 2024-05-26T22:48:28.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.682+0000 7fcbcf600640 1 -- 172.21.15.131:0/1162736769 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbc80b85a0 con 0x7fcbc80cf550 2024-05-26T22:48:28.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.682+0000 7f3f77400640 1 -- 172.21.15.131:0/2584714376 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f70007c70 con 0x7f3f7c037080 2024-05-26T22:48:28.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.680+0000 7ff3c7800640 1 -- 172.21.15.131:0/3593781002 shutdown_connections 2024-05-26T22:48:28.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.681+0000 7ff3c7800640 1 -- 172.21.15.131:0/3593781002 wait complete. 2024-05-26T22:48:28.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.682+0000 7f64fd400640 1 -- 172.21.15.131:0/2786745872 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f64f80e1310 msgr2=0x7f64f8189e10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-26T22:48:28.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.682+0000 7f64fd400640 1 --2- 172.21.15.131:0/2786745872 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f64f80e1310 0x7f64f8189e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:28.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.682+0000 7f64fd400640 1 -- 172.21.15.131:0/2786745872 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f64f800ab40 msgr2=0x7f64f817cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:28.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.682+0000 7f64fd400640 1 --2- 172.21.15.131:0/2786745872 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f64f800ab40 0x7f64f817cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:28.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.682+0000 7f64fd400640 1 -- 172.21.15.131:0/2786745872 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64f80e2e30 con 0x7f64f80516e0 2024-05-26T22:48:28.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.682+0000 7f3f89200640 1 -- 172.21.15.131:0/2584714376 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f7c04e170 con 0x7f3f7c037080 2024-05-26T22:48:28.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.682+0000 7f3f89200640 1 -- 172.21.15.131:0/2584714376 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f7c034c70 con 0x7f3f7c037080 2024-05-26T22:48:28.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.682+0000 7f824fe00640 1 --2- 172.21.15.131:0/1426942812 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f82500d0560 0x7f825009aa00 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f8240031000 tx=0x7f82400046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:28.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.680+0000 7f11bb400640 1 --2- 172.21.15.131:0/2451021581 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f11bc06bdf0 0x7f11bc072ec0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f11bc0cfe20 tx=0x7f11ac02ea00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=818a25f3a35ce34e server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:28.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.681+0000 7f11baa00640 1 -- 172.21.15.131:0/2451021581 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11ac037020 con 0x7f11bc06bdf0 2024-05-26T22:48:28.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.681+0000 7f11baa00640 1 -- 172.21.15.131:0/2451021581 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11ac00d4e0 con 0x7f11bc06bdf0 2024-05-26T22:48:28.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.681+0000 7f11baa00640 1 -- 172.21.15.131:0/2451021581 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11ac00b960 con 0x7f11bc06bdf0 2024-05-26T22:48:28.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.683+0000 7f64fd400640 1 --2- 172.21.15.131:0/2786745872 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f64f80516e0 0x7f64f81898d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f64ec00e980 tx=0x7f64ec00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:28.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.682+0000 7f11c2200640 1 -- 172.21.15.131:0/2451021581 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f11bc06bdf0 msgr2=0x7f11bc072ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:28.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.682+0000 7f11c2200640 1 --2- 172.21.15.131:0/2451021581 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f11bc06bdf0 0x7f11bc072ec0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f11bc0cfe20 tx=0x7f11ac02ea00 comp rx=0 tx=0).stop 2024-05-26T22:48:28.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.683+0000 7f3f77400640 1 -- 172.21.15.131:0/2584714376 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f70038040 con 0x7f3f7c037080 2024-05-26T22:48:28.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.683+0000 7f3f77400640 1 -- 172.21.15.131:0/2584714376 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f70007c70 con 0x7f3f7c037080 2024-05-26T22:48:28.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.683+0000 7f3a5d600640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f3a5815e500 0x7f3a58162a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:28.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.683+0000 7f3a5d600640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a580b6900 0x7f3a580d0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:28.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.683+0000 7f11c2200640 1 -- 172.21.15.131:0/2451021581 shutdown_connections 2024-05-26T22:48:28.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.683+0000 7f11c2200640 1 --2- 172.21.15.131:0/2451021581 >> v2:172.21.15.131:3300/0 conn(0x7f11bc15a630 0x7f11bc071360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:28.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.683+0000 7f11c2200640 1 --2- 172.21.15.131:0/2451021581 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f11bc06bdf0 0x7f11bc072ec0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:28.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.683+0000 7f11c2200640 1 -- 172.21.15.131:0/2451021581 >> 172.21.15.131:0/2451021581 conn(0x7f11bc06b940 msgr2=0x7f11bc0b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:28.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.683+0000 7f3a5d600640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a580b6d60 con 0x7f3a580d13b0 2024-05-26T22:48:28.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.683+0000 7f64f6000640 1 -- 172.21.15.131:0/2786745872 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64ec017070 con 0x7f64f80516e0 2024-05-26T22:48:28.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.683+0000 7f3a5d600640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f3a5815e8e0 con 0x7f3a5815e500 2024-05-26T22:48:28.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.683+0000 7f3a5d600640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3a580b5610 con 0x7f3a580b6900 2024-05-26T22:48:28.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.683+0000 7f3a56a00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f3a5815e500 0x7f3a58162a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:28.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.683+0000 7f64ff200640 1 -- 172.21.15.131:0/2786745872 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64f804df20 con 0x7f64f80516e0 2024-05-26T22:48:28.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.683+0000 7f64ff200640 1 -- 172.21.15.131:0/2786745872 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64f817d140 con 0x7f64f80516e0 2024-05-26T22:48:28.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.684+0000 7f824cc00640 1 -- 172.21.15.131:0/1426942812 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8240032e60 con 0x7f82500d0560 2024-05-26T22:48:28.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.684+0000 7f824cc00640 1 -- 172.21.15.131:0/1426942812 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8240030770 con 0x7f82500d0560 2024-05-26T22:48:28.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.684+0000 7f824cc00640 1 -- 172.21.15.131:0/1426942812 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8240037040 con 0x7f82500d0560 2024-05-26T22:48:28.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.684+0000 7f3a56a00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f3a5815e500 0x7f3a58162a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.131:36416/0 (socket says 172.21.15.131:36416) 2024-05-26T22:48:28.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.684+0000 7f3a56a00640 1 -- 172.21.15.131:0/1991851892 learned_addr learned my addr 172.21.15.131:0/1991851892 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:28.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.684+0000 7ff3c7800640 1 Processor -- start 2024-05-26T22:48:28.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.684+0000 7f3a56a00640 1 -- 172.21.15.131:0/1991851892 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a580b6900 msgr2=0x7f3a580d0e70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-26T22:48:28.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.684+0000 7f3a56a00640 1 --2- 172.21.15.131:0/1991851892 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a580b6900 0x7f3a580d0e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:28.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.684+0000 7f3a56a00640 1 -- 172.21.15.131:0/1991851892 >> v1:172.21.15.187:6789/0 conn(0x7f3a580d13b0 legacy=0x7f3a5806d640 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-26T22:48:28.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.684+0000 7f3a56a00640 1 -- 172.21.15.131:0/1991851892 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a580713d0 con 0x7f3a5815e500 2024-05-26T22:48:28.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.684+0000 7f8257800640 1 -- 172.21.15.131:0/1426942812 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f825009b6f0 con 0x7f82500d0560 2024-05-26T22:48:28.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.684+0000 7f8257800640 1 -- 172.21.15.131:0/1426942812 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f825009bd50 con 0x7f82500d0560 2024-05-26T22:48:28.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.683+0000 7f5891400640 1 -- 172.21.15.131:0/476063634 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5864001030 con 0x7f588c2043b0 2024-05-26T22:48:28.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.684+0000 7f64f6000640 1 -- 172.21.15.131:0/2786745872 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64ec00ca00 con 0x7f64f80516e0 2024-05-26T22:48:28.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.685+0000 7f11c2200640 1 -- 172.21.15.131:0/2451021581 shutdown_connections 2024-05-26T22:48:28.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.685+0000 7f3f89200640 1 -- 172.21.15.131:0/2584714376 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3f60001030 con 0x7f3f7c037080 2024-05-26T22:48:28.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.685+0000 7f8257800640 1 -- 172.21.15.131:0/1426942812 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f82500695c0 con 0x7f82500d0560 2024-05-26T22:48:28.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.684+0000 7ff3c7800640 1 -- start start 2024-05-26T22:48:28.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.684+0000 7f64f6000640 1 -- 172.21.15.131:0/2786745872 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64ec017070 con 0x7f64f80516e0 2024-05-26T22:48:28.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.685+0000 7f64ff200640 1 -- 172.21.15.131:0/2786745872 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f64d8001030 con 0x7f64f80516e0 2024-05-26T22:48:28.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.686+0000 7fcbcf600640 1 -- 172.21.15.131:0/1162736769 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fcbc80695c0 con 0x7fcbc80cf550 2024-05-26T22:48:28.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.686+0000 7ff3c7800640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff3c00af230 0x7ff3c01f8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:28.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.686+0000 7f3a56a00640 1 --2- 172.21.15.131:0/1991851892 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a5815e500 0x7f3a58162a90 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3a48009870 tx=0x7f3a4802a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c41b214891eb888 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:28.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.686+0000 7ff3c7800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff3c00d0560 0x7ff3c0160690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:28.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.686+0000 7f5883e00640 1 -- 172.21.15.131:0/476063634 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f588401d070 con 0x7f588c2043b0 2024-05-26T22:48:28.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.686+0000 7ff3c7800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff3c0165e30 0x7ff3c009ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:28.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.687+0000 7ff3c4e00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff3c00af230 0x7ff3c01f8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:28.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.687+0000 7ff3c4e00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff3c00af230 0x7ff3c01f8a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.131:36424/0 (socket says 172.21.15.131:36424) 2024-05-26T22:48:28.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.685+0000 7f11c2200640 1 -- 172.21.15.131:0/2451021581 wait complete. 2024-05-26T22:48:28.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.687+0000 7ff3c7800640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c00ce210 con 0x7ff3c0165e30 2024-05-26T22:48:28.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.687+0000 7ff3c7800640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c00cfac0 con 0x7ff3c00af230 2024-05-26T22:48:28.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.687+0000 7ff3c7800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff3c00ce390 con 0x7ff3c00d0560 2024-05-26T22:48:28.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.687+0000 7ff3c4e00640 1 -- 172.21.15.131:0/3918973407 learned_addr learned my addr 172.21.15.131:0/3918973407 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:28.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.687+0000 7ff3bfe00640 1 --2- 172.21.15.131:0/3918973407 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff3c00d0560 0x7ff3c0160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:28.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.687+0000 7f5883e00640 1 --2- 172.21.15.131:0/476063634 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f586c06be30 0x7f586c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:28.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.688+0000 7f5883e00640 1 -- 172.21.15.131:0/476063634 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f588408ad50 con 0x7f588c2043b0 2024-05-26T22:48:28.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.687+0000 7fcbc4c00640 1 -- 172.21.15.131:0/1162736769 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcbb4044020 con 0x7fcbc80cf550 2024-05-26T22:48:28.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.688+0000 7ff3c5800640 1 --2- 172.21.15.131:0/3918973407 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff3c0165e30 0x7ff3c009ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:28.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.688+0000 7f5883e00640 1 --2- 172.21.15.131:0/476063634 >> [v2:172.21.15.131:6800/2350940051,v1:172.21.15.131:6801/2350940051] conn(0x7f586c0740b0 0x7f586c0764d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:28.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.688+0000 7f5883e00640 1 -- 172.21.15.131:0/476063634 --> [v2:172.21.15.131:6800/2350940051,v1:172.21.15.131:6801/2350940051] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f586c076b80 con 0x7f586c0740b0 2024-05-26T22:48:28.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.688+0000 7f5883e00640 1 -- 172.21.15.131:0/476063634 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f588408d3b0 con 0x7f588c2043b0 2024-05-26T22:48:28.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.688+0000 7f3f77400640 1 -- 172.21.15.131:0/2584714376 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3f700047e0 con 0x7f3f7c037080 2024-05-26T22:48:28.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.688+0000 7ff3bfe00640 1 -- 172.21.15.131:0/3918973407 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff3c00af230 msgr2=0x7ff3c01f8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:28.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.688+0000 7ff3bfe00640 1 --2- 172.21.15.131:0/3918973407 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff3c00af230 0x7ff3c01f8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:28.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.688+0000 7ff3bfe00640 1 -- 172.21.15.131:0/3918973407 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff3c0165e30 msgr2=0x7ff3c009ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:28.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.688+0000 7f588be00640 1 --2- 172.21.15.131:0/476063634 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f586c06be30 0x7f586c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:28.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.688+0000 7ff3bfe00640 1 --2- 172.21.15.131:0/3918973407 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff3c0165e30 0x7ff3c009ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:28.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.688+0000 7ff3bfe00640 1 -- 172.21.15.131:0/3918973407 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3c009afa0 con 0x7ff3c00d0560 2024-05-26T22:48:28.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.689+0000 7ff3c5800640 1 --2- 172.21.15.131:0/3918973407 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff3c0165e30 0x7ff3c009ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-26T22:48:28.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.689+0000 7f3a55600640 1 -- 172.21.15.131:0/1991851892 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a48037070 con 0x7f3a5815e500 2024-05-26T22:48:28.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.689+0000 7ff3bfe00640 1 --2- 172.21.15.131:0/3918973407 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff3c00d0560 0x7ff3c0160690 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ff3b000c3e0 tx=0x7ff3b000c8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:28.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.689+0000 7f11c2200640 1 Processor -- start 2024-05-26T22:48:28.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.689+0000 7fcbc4c00640 1 --2- 172.21.15.131:0/1162736769 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fcba806be30 0x7fcba806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:28.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.689+0000 7f588be00640 1 --2- 172.21.15.131:0/476063634 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f586c06be30 0x7f586c06e2f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f587c00ba20 tx=0x7f587c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:28.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.689+0000 7f11c2200640 1 -- start start 2024-05-26T22:48:28.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.689+0000 7f3a55600640 1 -- 172.21.15.131:0/1991851892 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a48032440 con 0x7f3a5815e500 2024-05-26T22:48:28.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.689+0000 7f3a55600640 1 -- 172.21.15.131:0/1991851892 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a48032810 con 0x7f3a5815e500 2024-05-26T22:48:28.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.689+0000 7f588b400640 1 --2- 172.21.15.131:0/476063634 >> [v2:172.21.15.131:6800/2350940051,v1:172.21.15.131:6801/2350940051] conn(0x7f586c0740b0 0x7f586c0764d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:28.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.689+0000 7ff3bcc00640 1 -- 172.21.15.131:0/3918973407 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b00074e0 con 0x7ff3c00d0560 2024-05-26T22:48:28.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.690+0000 7ff3c7800640 1 -- 172.21.15.131:0/3918973407 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3c009b280 con 0x7ff3c00d0560 2024-05-26T22:48:28.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.689+0000 7fcbc4c00640 1 -- 172.21.15.131:0/1162736769 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fcbb40ab6d0 con 0x7fcbc80cf550 2024-05-26T22:48:28.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.690+0000 7fcbc4c00640 1 --2- 172.21.15.131:0/1162736769 >> [v2:172.21.15.131:6816/3627842210,v1:172.21.15.131:6817/3627842210] conn(0x7fcba8074050 0x7fcba8076470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:28.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.690+0000 7fcbc4c00640 1 -- 172.21.15.131:0/1162736769 --> [v2:172.21.15.131:6816/3627842210,v1:172.21.15.131:6817/3627842210] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fcbb4032440 con 0x7fcba8074050 2024-05-26T22:48:28.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.690+0000 7fcbc4c00640 1 -- 172.21.15.131:0/1162736769 <== mon.2 v2:172.21.15.187:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fcbb40b2050 con 0x7fcbc80cf550 2024-05-26T22:48:28.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.690+0000 7ff3c7800640 1 -- 172.21.15.131:0/3918973407 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3c009b840 con 0x7ff3c00d0560 2024-05-26T22:48:28.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.690+0000 7fcbc7e00640 1 --2- 172.21.15.131:0/1162736769 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fcba806be30 0x7fcba806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:28.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.690+0000 7fcbcd600640 1 --2- 172.21.15.131:0/1162736769 >> [v2:172.21.15.131:6816/3627842210,v1:172.21.15.131:6817/3627842210] conn(0x7fcba8074050 0x7fcba8076470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:28.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.690+0000 7f3f77400640 1 --2- 172.21.15.131:0/2584714376 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f3f6406c060 0x7f3f6406e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:28.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.690+0000 7f3a5d600640 1 -- 172.21.15.131:0/1991851892 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a5815e500 msgr2=0x7f3a58162a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:28.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.690+0000 7f3a5d600640 1 --2- 172.21.15.131:0/1991851892 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a5815e500 0x7f3a58162a90 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3a48009870 tx=0x7f3a4802a230 comp rx=0 tx=0).stop 2024-05-26T22:48:28.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.691+0000 7f3a5d600640 1 -- 172.21.15.131:0/1991851892 shutdown_connections 2024-05-26T22:48:28.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.691+0000 7f3a5d600640 1 --2- 172.21.15.131:0/1991851892 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a580b6900 0x7f3a580d0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:28.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.691+0000 7f3a5d600640 1 --2- 172.21.15.131:0/1991851892 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a5815e500 0x7f3a58162a90 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:28.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.691+0000 7f3a5d600640 1 -- 172.21.15.131:0/1991851892 >> 172.21.15.131:0/1991851892 conn(0x7f3a5806f6d0 msgr2=0x7f3a5806fae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:28.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.691+0000 7ff3bcc00640 1 -- 172.21.15.131:0/3918973407 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3b0007680 con 0x7ff3c00d0560 2024-05-26T22:48:28.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.691+0000 7f11c2200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f11bc06bdf0 0x7f11bc152f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:28.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.691+0000 7ff3bcc00640 1 -- 172.21.15.131:0/3918973407 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b00074e0 con 0x7ff3c00d0560 2024-05-26T22:48:28.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.691+0000 7f11c2200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f11bc0cfe50 0x7f11bc153ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:28.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.691+0000 7f3f77400640 1 -- 172.21.15.131:0/2584714376 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f3f700aa750 con 0x7f3f7c037080 2024-05-26T22:48:28.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.691+0000 7f3f77400640 1 --2- 172.21.15.131:0/2584714376 >> [v2:172.21.15.131:6808/350790591,v1:172.21.15.131:6809/350790591] conn(0x7f3f64074260 0x7f3f64076680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:28.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.691+0000 7f3f77400640 1 -- 172.21.15.131:0/2584714376 --> [v2:172.21.15.131:6808/350790591,v1:172.21.15.131:6809/350790591] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3f70007270 con 0x7f3f64074260 2024-05-26T22:48:28.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.691+0000 7f3f77400640 1 -- 172.21.15.131:0/2584714376 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f3f700b1050 con 0x7f3f7c037080 2024-05-26T22:48:28.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.691+0000 7f824cc00640 1 -- 172.21.15.131:0/1426942812 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8240044020 con 0x7f82500d0560 2024-05-26T22:48:28.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.691+0000 7f11c2200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f11bc15a630 0x7f11bc0b2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:28.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.691+0000 7f3f83400640 1 --2- 172.21.15.131:0/2584714376 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f3f6406c060 0x7f3f6406e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:28.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.692+0000 7f3a5d600640 1 -- 172.21.15.131:0/1991851892 shutdown_connections 2024-05-26T22:48:28.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.691+0000 7f11b2a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f11bc0cfe50 0x7f11bc153ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:28.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.692+0000 7f11c2200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f11bc15edd0 con 0x7f11bc0cfe50 2024-05-26T22:48:28.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.691+0000 7ff3c7800640 1 -- 172.21.15.131:0/3918973407 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff398001030 con 0x7ff3c00d0560 2024-05-26T22:48:28.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.692+0000 7f11b2a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f11bc0cfe50 0x7f11bc153ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54088/0 (socket says 172.21.15.131:54088) 2024-05-26T22:48:28.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.692+0000 7f11c2200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f11bc0718a0 con 0x7f11bc15a630 2024-05-26T22:48:28.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.692+0000 7f11c2200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f11bc154510 con 0x7f11bc06bdf0 2024-05-26T22:48:28.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.692+0000 7f588b400640 1 --2- 172.21.15.131:0/476063634 >> [v2:172.21.15.131:6800/2350940051,v1:172.21.15.131:6801/2350940051] conn(0x7f586c0740b0 0x7f586c0764d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:28.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.692+0000 7f11b2a00640 1 -- 172.21.15.131:0/325706192 learned_addr learned my addr 172.21.15.131:0/325706192 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:28.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.692+0000 7f11bbe00640 1 --2- 172.21.15.131:0/325706192 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f11bc15a630 0x7f11bc0b2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:28.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.692+0000 7fcbc7e00640 1 --2- 172.21.15.131:0/1162736769 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fcba806be30 0x7fcba806e2f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fcbc815d2a0 tx=0x7fcbc000b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:28.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.692+0000 7fcbcd600640 1 --2- 172.21.15.131:0/1162736769 >> [v2:172.21.15.131:6816/3627842210,v1:172.21.15.131:6817/3627842210] conn(0x7fcba8074050 0x7fcba8076470 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:28.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.691+0000 7f64f6000640 1 -- 172.21.15.131:0/2786745872 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f64ec01a070 con 0x7f64f80516e0 2024-05-26T22:48:28.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.693+0000 7f11bb400640 1 --2- 172.21.15.131:0/325706192 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f11bc06bdf0 0x7f11bc152f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:28.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.693+0000 7f11b2a00640 1 -- 172.21.15.131:0/325706192 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f11bc06bdf0 msgr2=0x7f11bc152f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:28.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.693+0000 7f11b2a00640 1 --2- 172.21.15.131:0/325706192 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f11bc06bdf0 0x7f11bc152f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:28.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.693+0000 7f11b2a00640 1 -- 172.21.15.131:0/325706192 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f11bc15a630 msgr2=0x7f11bc0b2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:28.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.693+0000 7f11b2a00640 1 --2- 172.21.15.131:0/325706192 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f11bc15a630 0x7f11bc0b2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:28.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.693+0000 7f11b2a00640 1 -- 172.21.15.131:0/325706192 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11bc1547f0 con 0x7f11bc0cfe50 2024-05-26T22:48:28.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.693+0000 7f64f6000640 1 --2- 172.21.15.131:0/2786745872 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f64e006c030 0x7f64e006e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:28.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.693+0000 7f64f6000640 1 -- 172.21.15.131:0/2786745872 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f64ec08aa40 con 0x7f64f80516e0 2024-05-26T22:48:28.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.693+0000 7f64fde00640 1 --2- 172.21.15.131:0/2786745872 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f64e006c030 0x7f64e006e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:28.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.693+0000 7f64f6000640 1 --2- 172.21.15.131:0/2786745872 >> [v2:172.21.15.131:6824/1525528247,v1:172.21.15.131:6825/1525528247] conn(0x7f64e0074250 0x7f64e0076670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:28.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.693+0000 7f64f6000640 1 -- 172.21.15.131:0/2786745872 --> [v2:172.21.15.131:6824/1525528247,v1:172.21.15.131:6825/1525528247] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f64ec022e50 con 0x7f64e0074250 2024-05-26T22:48:28.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.693+0000 7f64f6000640 1 -- 172.21.15.131:0/2786745872 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f64ec08f2a0 con 0x7f64f80516e0 2024-05-26T22:48:28.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.692+0000 7f3a5d600640 1 -- 172.21.15.131:0/1991851892 wait complete. 2024-05-26T22:48:28.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.693+0000 7f3a5d600640 1 Processor -- start 2024-05-26T22:48:28.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.694+0000 7f11b2a00640 1 --2- 172.21.15.131:0/325706192 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f11bc0cfe50 0x7f11bc153ac0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f11a400e9b0 tx=0x7f11a400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:28.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.694+0000 7f64fe800640 1 --2- 172.21.15.131:0/2786745872 >> [v2:172.21.15.131:6824/1525528247,v1:172.21.15.131:6825/1525528247] conn(0x7f64e0074250 0x7f64e0076670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:28.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.694+0000 7f11bb400640 1 --2- 172.21.15.131:0/325706192 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f11bc06bdf0 0x7f11bc152f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:48:28.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.694+0000 7f824cc00640 1 --2- 172.21.15.131:0/1426942812 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f823006bf00 0x7f823006e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:28.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.694+0000 7f3f83400640 1 --2- 172.21.15.131:0/2584714376 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f3f6406c060 0x7f3f6406e520 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3f8406a130 tx=0x7f3f8408d040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:28.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.694+0000 7f64fde00640 1 --2- 172.21.15.131:0/2786745872 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f64e006c030 0x7f64e006e4f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6500051060 tx=0x7f6500069c10 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:28.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.695+0000 7f3f82a00640 1 --2- 172.21.15.131:0/2584714376 >> [v2:172.21.15.131:6808/350790591,v1:172.21.15.131:6809/350790591] conn(0x7f3f64074260 0x7f3f64076680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:28.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.695+0000 7f3a5d600640 1 -- start start 2024-05-26T22:48:28.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.694+0000 7f11b3e00640 1 -- 172.21.15.131:0/325706192 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11a4017070 con 0x7f11bc0cfe50 2024-05-26T22:48:28.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.695+0000 7f11c2200640 1 -- 172.21.15.131:0/325706192 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11bc0b0610 con 0x7f11bc0cfe50 2024-05-26T22:48:28.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.694+0000 7f824cc00640 1 -- 172.21.15.131:0/1426942812 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f82400abab0 con 0x7f82500d0560 2024-05-26T22:48:28.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.694+0000 7f824cc00640 1 --2- 172.21.15.131:0/1426942812 >> [v2:172.21.15.187:6814/244560995,v1:172.21.15.187:6816/244560995] conn(0x7f8230074120 0x7f8230076540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:28.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.694+0000 7f824cc00640 1 -- 172.21.15.131:0/1426942812 --> [v2:172.21.15.187:6814/244560995,v1:172.21.15.187:6816/244560995] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8240008eb0 con 0x7f8230074120 2024-05-26T22:48:28.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.694+0000 7f824cc00640 1 -- 172.21.15.131:0/1426942812 <== mon.2 v2:172.21.15.187:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f82400abea0 con 0x7f82500d0560 2024-05-26T22:48:28.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.695+0000 7ff3bcc00640 1 -- 172.21.15.131:0/3918973407 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff3b0011070 con 0x7ff3c00d0560 2024-05-26T22:48:28.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.695+0000 7f5883e00640 1 -- 172.21.15.131:0/476063634 <== osd.0 v2:172.21.15.131:6800/2350940051 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f586c076b80 con 0x7f586c0740b0 2024-05-26T22:48:28.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.695+0000 7f8254e00640 1 --2- 172.21.15.131:0/1426942812 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f823006bf00 0x7f823006e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:28.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.695+0000 7f64fe800640 1 --2- 172.21.15.131:0/2786745872 >> [v2:172.21.15.131:6824/1525528247,v1:172.21.15.131:6825/1525528247] conn(0x7f64e0074250 0x7f64e0076670 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:28.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.696+0000 7f8255800640 1 --2- 172.21.15.131:0/1426942812 >> [v2:172.21.15.187:6814/244560995,v1:172.21.15.187:6816/244560995] conn(0x7f8230074120 0x7f8230076540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:28.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.696+0000 7f3a5d600640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a580b6900 0x7f3a58150d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:28.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.696+0000 7f3a5d600640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a580d13b0 0x7f3a5814eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:28.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.696+0000 7f3a5d600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a5815e500 0x7f3a581f4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:28.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.696+0000 7f3a5d600640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f3a5806dd50 con 0x7f3a5815e500 2024-05-26T22:48:28.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.696+0000 7f3a5d600640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3a580b52a0 con 0x7f3a580d13b0 2024-05-26T22:48:28.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.696+0000 7f3a5d600640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3a5806ded0 con 0x7f3a580b6900 2024-05-26T22:48:28.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.696+0000 7f3f82a00640 1 --2- 172.21.15.131:0/2584714376 >> [v2:172.21.15.131:6808/350790591,v1:172.21.15.131:6809/350790591] conn(0x7f3f64074260 0x7f3f64076680 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:28.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.696+0000 7fcbc4c00640 1 -- 172.21.15.131:0/1162736769 <== osd.2 v2:172.21.15.131:6816/3627842210 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcbb4032440 con 0x7fcba8074050 2024-05-26T22:48:28.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.696+0000 7ff3bcc00640 1 --2- 172.21.15.131:0/3918973407 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ff39c06c030 0x7ff39c06e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:28.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.696+0000 7ff3bcc00640 1 -- 172.21.15.131:0/3918973407 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff3b008ad00 con 0x7ff3c00d0560 2024-05-26T22:48:28.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.695+0000 7f11c2200640 1 -- 172.21.15.131:0/325706192 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11bc0b0070 con 0x7f11bc0cfe50 2024-05-26T22:48:28.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.696+0000 7f11c2200640 1 -- 172.21.15.131:0/325706192 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f11bc0b0430 con 0x7f11bc0cfe50 2024-05-26T22:48:28.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.696+0000 7f11b3e00640 1 -- 172.21.15.131:0/325706192 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11a400c9a0 con 0x7f11bc0cfe50 2024-05-26T22:48:28.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.696+0000 7f11b3e00640 1 -- 172.21.15.131:0/325706192 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11a4017070 con 0x7f11bc0cfe50 2024-05-26T22:48:28.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.697+0000 7f8254e00640 1 --2- 172.21.15.131:0/1426942812 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f823006bf00 0x7f823006e3c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f823c007fd0 tx=0x7f823c007de0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:28.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.697+0000 7ff3bcc00640 1 --2- 172.21.15.131:0/3918973407 >> [v2:172.21.15.187:6800/324619398,v1:172.21.15.187:6801/324619398] conn(0x7ff39c0741d0 0x7ff39c0765f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:28.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.697+0000 7f3a56a00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a580b6900 0x7f3a58150d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:28.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.697+0000 7f3a56a00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a580b6900 0x7f3a58150d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.131:43266/0 (socket says 172.21.15.131:43266) 2024-05-26T22:48:28.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.697+0000 7f3a56a00640 1 -- 172.21.15.131:0/808100511 learned_addr learned my addr 172.21.15.131:0/808100511 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:28.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.697+0000 7ff3bcc00640 1 -- 172.21.15.131:0/3918973407 --> [v2:172.21.15.187:6800/324619398,v1:172.21.15.187:6801/324619398] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff3b001ee50 con 0x7ff39c0741d0 2024-05-26T22:48:28.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.697+0000 7ff3bcc00640 1 -- 172.21.15.131:0/3918973407 <== mon.2 v2:172.21.15.187:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7ff3b008b190 con 0x7ff3c00d0560 2024-05-26T22:48:28.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.697+0000 7f64f6000640 1 -- 172.21.15.131:0/2786745872 <== osd.1 v2:172.21.15.131:6824/1525528247 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f64ec022e50 con 0x7f64e0074250 2024-05-26T22:48:28.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.697+0000 7f8255800640 1 --2- 172.21.15.131:0/1426942812 >> [v2:172.21.15.187:6814/244560995,v1:172.21.15.187:6816/244560995] conn(0x7f8230074120 0x7f8230076540 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:28.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.697+0000 7f3a56000640 1 --2- 172.21.15.131:0/808100511 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a580d13b0 0x7f3a5814eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:28.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.697+0000 7ff3c5800640 1 --2- 172.21.15.131:0/3918973407 >> [v2:172.21.15.187:6800/324619398,v1:172.21.15.187:6801/324619398] conn(0x7ff39c0741d0 0x7ff39c0765f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:28.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.698+0000 7ff3c4e00640 1 --2- 172.21.15.131:0/3918973407 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ff39c06c030 0x7ff39c06e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:28.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.698+0000 7f3a57400640 1 --2- 172.21.15.131:0/808100511 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a5815e500 0x7f3a581f4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:28.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.698+0000 7f3a56a00640 1 -- 172.21.15.131:0/808100511 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a580d13b0 msgr2=0x7f3a5814eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:28.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.698+0000 7f3a56a00640 1 --2- 172.21.15.131:0/808100511 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a580d13b0 0x7f3a5814eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:28.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.698+0000 7f3a56a00640 1 -- 172.21.15.131:0/808100511 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a5815e500 msgr2=0x7f3a581f4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:28.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.698+0000 7f3a56a00640 1 --2- 172.21.15.131:0/808100511 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a5815e500 0x7f3a581f4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:28.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.698+0000 7f3a56a00640 1 -- 172.21.15.131:0/808100511 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a581512c0 con 0x7f3a580b6900 2024-05-26T22:48:28.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.698+0000 7f3a57400640 1 --2- 172.21.15.131:0/808100511 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a5815e500 0x7f3a581f4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:48:28.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.699+0000 7f11b3e00640 1 -- 172.21.15.131:0/325706192 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f11a401a070 con 0x7f11bc0cfe50 2024-05-26T22:48:28.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.699+0000 7f3f77400640 1 -- 172.21.15.131:0/2584714376 <== osd.3 v2:172.21.15.131:6808/350790591 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3f70007270 con 0x7f3f64074260 2024-05-26T22:48:28.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.699+0000 7ff3c5800640 1 --2- 172.21.15.131:0/3918973407 >> [v2:172.21.15.187:6800/324619398,v1:172.21.15.187:6801/324619398] conn(0x7ff39c0741d0 0x7ff39c0765f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:28.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.699+0000 7f3a56a00640 1 --2- 172.21.15.131:0/808100511 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a580b6900 0x7f3a58150d80 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3a48009870 tx=0x7f3a480046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:28.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.699+0000 7f3a4ea00640 1 -- 172.21.15.131:0/808100511 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a48041070 con 0x7f3a580b6900 2024-05-26T22:48:28.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.700+0000 7ff3c4e00640 1 --2- 172.21.15.131:0/3918973407 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ff39c06c030 0x7ff39c06e4f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff3b80097a0 tx=0x7ff3b802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:28.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.700+0000 7f11b3e00640 1 --2- 172.21.15.131:0/325706192 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f119806c060 0x7f119806e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:28.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.700+0000 7f824cc00640 1 -- 172.21.15.131:0/1426942812 <== osd.4 v2:172.21.15.187:6814/244560995 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8240008eb0 con 0x7f8230074120 2024-05-26T22:48:28.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.700+0000 7f3a4ea00640 1 -- 172.21.15.131:0/808100511 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a4803c930 con 0x7f3a580b6900 2024-05-26T22:48:28.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.700+0000 7f3a4ea00640 1 -- 172.21.15.131:0/808100511 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a48037040 con 0x7f3a580b6900 2024-05-26T22:48:28.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.700+0000 7f11b3e00640 1 -- 172.21.15.131:0/325706192 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f11a408b1c0 con 0x7f11bc0cfe50 2024-05-26T22:48:28.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.700+0000 7f11bb400640 1 --2- 172.21.15.131:0/325706192 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f119806c060 0x7f119806e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:28.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.700+0000 7f11b3e00640 1 --2- 172.21.15.131:0/325706192 >> [v2:172.21.15.187:6824/1220065622,v1:172.21.15.187:6825/1220065622] conn(0x7f1198074200 0x7f1198076620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:28.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.700+0000 7f11b3e00640 1 -- 172.21.15.131:0/325706192 --> [v2:172.21.15.187:6824/1220065622,v1:172.21.15.187:6825/1220065622] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1198076cd0 con 0x7f1198074200 2024-05-26T22:48:28.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.700+0000 7f11b3e00640 1 -- 172.21.15.131:0/325706192 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f11a408b650 con 0x7f11bc0cfe50 2024-05-26T22:48:28.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.701+0000 7f11bb400640 1 --2- 172.21.15.131:0/325706192 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f119806c060 0x7f119806e520 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f11ac038000 tx=0x7f11ac00d000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:28.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.701+0000 7f11bbe00640 1 --2- 172.21.15.131:0/325706192 >> [v2:172.21.15.187:6824/1220065622,v1:172.21.15.187:6825/1220065622] conn(0x7f1198074200 0x7f1198076620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:28.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.701+0000 7f3a5d600640 1 -- 172.21.15.131:0/808100511 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a5814fd20 con 0x7f3a580b6900 2024-05-26T22:48:28.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.701+0000 7f3a5d600640 1 -- 172.21.15.131:0/808100511 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a581502a0 con 0x7f3a580b6900 2024-05-26T22:48:28.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.702+0000 7ff3bcc00640 1 -- 172.21.15.131:0/3918973407 <== osd.5 v2:172.21.15.187:6800/324619398 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff3b001ee50 con 0x7ff39c0741d0 2024-05-26T22:48:28.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.703+0000 7f11bbe00640 1 --2- 172.21.15.131:0/325706192 >> [v2:172.21.15.187:6824/1220065622,v1:172.21.15.187:6825/1220065622] conn(0x7f1198074200 0x7f1198076620 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:28.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.705+0000 7f11b3e00640 1 -- 172.21.15.131:0/325706192 <== osd.7 v2:172.21.15.187:6824/1220065622 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1198076cd0 con 0x7f1198074200 2024-05-26T22:48:28.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.703+0000 7f3a5d600640 1 -- 172.21.15.131:0/808100511 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3a580695c0 con 0x7f3a580b6900 2024-05-26T22:48:28.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.706+0000 7f3a4ea00640 1 -- 172.21.15.131:0/808100511 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3a48044070 con 0x7f3a580b6900 2024-05-26T22:48:28.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.707+0000 7f3a4ea00640 1 --2- 172.21.15.131:0/808100511 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f3a3806bf30 0x7f3a3806e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:28.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.707+0000 7f3a4ea00640 1 -- 172.21.15.131:0/808100511 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f3a480abac0 con 0x7f3a580b6900 2024-05-26T22:48:28.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.708+0000 7f3a4ea00640 1 --2- 172.21.15.131:0/808100511 >> [v2:172.21.15.187:6804/1391347786,v1:172.21.15.187:6806/1391347786] conn(0x7f3a38074150 0x7f3a38076570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:28.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.708+0000 7f3a4ea00640 1 -- 172.21.15.131:0/808100511 --> [v2:172.21.15.187:6804/1391347786,v1:172.21.15.187:6806/1391347786] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3a4804ee50 con 0x7f3a38074150 2024-05-26T22:48:28.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.708+0000 7f3a4ea00640 1 -- 172.21.15.131:0/808100511 <== mon.2 v2:172.21.15.187:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f3a480abeb0 con 0x7f3a580b6900 2024-05-26T22:48:28.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.708+0000 7f3a56000640 1 --2- 172.21.15.131:0/808100511 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f3a3806bf30 0x7f3a3806e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:28.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.709+0000 7f3a57400640 1 --2- 172.21.15.131:0/808100511 >> [v2:172.21.15.187:6804/1391347786,v1:172.21.15.187:6806/1391347786] conn(0x7f3a38074150 0x7f3a38076570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:28.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.710+0000 7f3a56000640 1 --2- 172.21.15.131:0/808100511 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f3a3806bf30 0x7f3a3806e3f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3a580b1690 tx=0x7f3a440090e0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:28.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.711+0000 7f3a57400640 1 --2- 172.21.15.131:0/808100511 >> [v2:172.21.15.187:6804/1391347786,v1:172.21.15.187:6806/1391347786] conn(0x7f3a38074150 0x7f3a38076570 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:28.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.713+0000 7f3a4ea00640 1 -- 172.21.15.131:0/808100511 <== osd.6 v2:172.21.15.187:6804/1391347786 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3a4804ee50 con 0x7f3a38074150 2024-05-26T22:48:28.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.719+0000 7fcbcf600640 1 -- 172.21.15.131:0/1162736769 --> [v2:172.21.15.131:6816/3627842210,v1:172.21.15.131:6817/3627842210] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fcbc81f7f10 con 0x7fcba8074050 2024-05-26T22:48:28.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.720+0000 7fcbc4c00640 1 -- 172.21.15.131:0/1162736769 <== osd.2 v2:172.21.15.131:6816/3627842210 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fcbc81f7f10 con 0x7fcba8074050 2024-05-26T22:48:28.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.721+0000 7fcbcf600640 1 -- 172.21.15.131:0/1162736769 >> [v2:172.21.15.131:6816/3627842210,v1:172.21.15.131:6817/3627842210] conn(0x7fcba8074050 msgr2=0x7fcba8076470 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:28.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.721+0000 7fcbcf600640 1 --2- 172.21.15.131:0/1162736769 >> [v2:172.21.15.131:6816/3627842210,v1:172.21.15.131:6817/3627842210] conn(0x7fcba8074050 0x7fcba8076470 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:28.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.722+0000 7fcbcf600640 1 -- 172.21.15.131:0/1162736769 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fcba806be30 msgr2=0x7fcba806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:28.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.722+0000 7fcbcf600640 1 --2- 172.21.15.131:0/1162736769 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fcba806be30 0x7fcba806e2f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fcbc815d2a0 tx=0x7fcbc000b040 comp rx=0 tx=0).stop 2024-05-26T22:48:28.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.722+0000 7fcbcf600640 1 -- 172.21.15.131:0/1162736769 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcbc80cf550 msgr2=0x7fcbc80cc180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:28.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.722+0000 7fcbcf600640 1 --2- 172.21.15.131:0/1162736769 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcbc80cf550 0x7fcbc80cc180 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fcbb402a230 tx=0x7fcbb40046c0 comp rx=0 tx=0).stop 2024-05-26T22:48:28.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.722+0000 7fcbcd600640 1 -- 172.21.15.131:0/1162736769 reap_dead start 2024-05-26T22:48:28.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.723+0000 7fcbcf600640 1 -- 172.21.15.131:0/1162736769 shutdown_connections 2024-05-26T22:48:28.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.723+0000 7fcbcf600640 1 -- 172.21.15.131:0/1162736769 >> 172.21.15.131:0/1162736769 conn(0x7fcbc8072e90 msgr2=0x7fcbc80c9df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:28.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.723+0000 7fcbcf600640 1 -- 172.21.15.131:0/1162736769 shutdown_connections 2024-05-26T22:48:28.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.724+0000 7fcbcf600640 1 -- 172.21.15.131:0/1162736769 wait complete. 2024-05-26T22:48:28.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.725+0000 7f64ff200640 1 -- 172.21.15.131:0/2786745872 --> [v2:172.21.15.131:6824/1525528247,v1:172.21.15.131:6825/1525528247] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f64d8002dd0 con 0x7f64e0074250 2024-05-26T22:48:28.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.726+0000 7f64f6000640 1 -- 172.21.15.131:0/2786745872 <== osd.1 v2:172.21.15.131:6824/1525528247 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f64d8002dd0 con 0x7f64e0074250 2024-05-26T22:48:28.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.726+0000 7f64ff200640 1 -- 172.21.15.131:0/2786745872 >> [v2:172.21.15.131:6824/1525528247,v1:172.21.15.131:6825/1525528247] conn(0x7f64e0074250 msgr2=0x7f64e0076670 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:28.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.726+0000 7f64ff200640 1 --2- 172.21.15.131:0/2786745872 >> [v2:172.21.15.131:6824/1525528247,v1:172.21.15.131:6825/1525528247] conn(0x7f64e0074250 0x7f64e0076670 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:28.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.727+0000 7f64ff200640 1 -- 172.21.15.131:0/2786745872 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f64e006c030 msgr2=0x7f64e006e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:28.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.727+0000 7f64ff200640 1 --2- 172.21.15.131:0/2786745872 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f64e006c030 0x7f64e006e4f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6500051060 tx=0x7f6500069c10 comp rx=0 tx=0).stop 2024-05-26T22:48:28.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.727+0000 7f64ff200640 1 -- 172.21.15.131:0/2786745872 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f64f80516e0 msgr2=0x7f64f81898d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:28.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.727+0000 7f64ff200640 1 --2- 172.21.15.131:0/2786745872 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f64f80516e0 0x7f64f81898d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f64ec00e980 tx=0x7f64ec00ee40 comp rx=0 tx=0).stop 2024-05-26T22:48:28.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.727+0000 7f64fe800640 1 -- 172.21.15.131:0/2786745872 reap_dead start 2024-05-26T22:48:28.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.727+0000 7f64ff200640 1 -- 172.21.15.131:0/2786745872 shutdown_connections 2024-05-26T22:48:28.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.727+0000 7f64ff200640 1 -- 172.21.15.131:0/2786745872 >> 172.21.15.131:0/2786745872 conn(0x7f64f8056cd0 msgr2=0x7f64f80e4d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:28.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.728+0000 7f64ff200640 1 -- 172.21.15.131:0/2786745872 shutdown_connections 2024-05-26T22:48:28.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.729+0000 7f64ff200640 1 -- 172.21.15.131:0/2786745872 wait complete. 2024-05-26T22:48:28.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.732+0000 7f3f89200640 1 -- 172.21.15.131:0/2584714376 --> [v2:172.21.15.131:6808/350790591,v1:172.21.15.131:6809/350790591] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3f60002dd0 con 0x7f3f64074260 2024-05-26T22:48:28.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.733+0000 7f3f77400640 1 -- 172.21.15.131:0/2584714376 <== osd.3 v2:172.21.15.131:6808/350790591 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f3f60002dd0 con 0x7f3f64074260 2024-05-26T22:48:28.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.733+0000 7f3f89200640 1 -- 172.21.15.131:0/2584714376 >> [v2:172.21.15.131:6808/350790591,v1:172.21.15.131:6809/350790591] conn(0x7f3f64074260 msgr2=0x7f3f64076680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:28.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.733+0000 7f3f89200640 1 --2- 172.21.15.131:0/2584714376 >> [v2:172.21.15.131:6808/350790591,v1:172.21.15.131:6809/350790591] conn(0x7f3f64074260 0x7f3f64076680 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:28.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.734+0000 7f8257800640 1 -- 172.21.15.131:0/1426942812 --> [v2:172.21.15.187:6814/244560995,v1:172.21.15.187:6816/244560995] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f825009c520 con 0x7f8230074120 2024-05-26T22:48:28.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.734+0000 7f5891400640 1 -- 172.21.15.131:0/476063634 --> [v2:172.21.15.131:6800/2350940051,v1:172.21.15.131:6801/2350940051] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5864002dd0 con 0x7f586c0740b0 2024-05-26T22:48:28.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.734+0000 7f5883e00640 1 -- 172.21.15.131:0/476063634 <== osd.0 v2:172.21.15.131:6800/2350940051 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f5864002dd0 con 0x7f586c0740b0 2024-05-26T22:48:28.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.734+0000 7f3f89200640 1 -- 172.21.15.131:0/2584714376 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f3f6406c060 msgr2=0x7f3f6406e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:28.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.734+0000 7f3f89200640 1 --2- 172.21.15.131:0/2584714376 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f3f6406c060 0x7f3f6406e520 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3f8406a130 tx=0x7f3f8408d040 comp rx=0 tx=0).stop 2024-05-26T22:48:28.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.734+0000 7f3f89200640 1 -- 172.21.15.131:0/2584714376 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3f7c037080 msgr2=0x7f3f7c0dcdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:28.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.734+0000 7f3f89200640 1 --2- 172.21.15.131:0/2584714376 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3f7c037080 0x7f3f7c0dcdd0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3f7002add0 tx=0x7f3f70001970 comp rx=0 tx=0).stop 2024-05-26T22:48:28.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.735+0000 7f3f83e00640 1 -- 172.21.15.131:0/2584714376 reap_dead start 2024-05-26T22:48:28.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.735+0000 7f3f89200640 1 -- 172.21.15.131:0/2584714376 shutdown_connections 2024-05-26T22:48:28.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.735+0000 7f3f89200640 1 -- 172.21.15.131:0/2584714376 >> 172.21.15.131:0/2584714376 conn(0x7f3f7c009fb0 msgr2=0x7f3f7c04ab40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:28.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.735+0000 7f3f89200640 1 -- 172.21.15.131:0/2584714376 shutdown_connections 2024-05-26T22:48:28.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.735+0000 7f3f89200640 1 -- 172.21.15.131:0/2584714376 wait complete. 2024-05-26T22:48:28.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.735+0000 7f5891400640 1 -- 172.21.15.131:0/476063634 >> [v2:172.21.15.131:6800/2350940051,v1:172.21.15.131:6801/2350940051] conn(0x7f586c0740b0 msgr2=0x7f586c0764d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:28.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.735+0000 7f5891400640 1 --2- 172.21.15.131:0/476063634 >> [v2:172.21.15.131:6800/2350940051,v1:172.21.15.131:6801/2350940051] conn(0x7f586c0740b0 0x7f586c0764d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:28.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.735+0000 7f5891400640 1 -- 172.21.15.131:0/476063634 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f586c06be30 msgr2=0x7f586c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:28.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.735+0000 7f5891400640 1 --2- 172.21.15.131:0/476063634 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f586c06be30 0x7f586c06e2f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f587c00ba20 tx=0x7f587c008040 comp rx=0 tx=0).stop 2024-05-26T22:48:28.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.735+0000 7f5891400640 1 -- 172.21.15.131:0/476063634 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f588c2043b0 msgr2=0x7f588c204830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:28.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.735+0000 7f824cc00640 1 -- 172.21.15.131:0/1426942812 <== osd.4 v2:172.21.15.187:6814/244560995 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f825009c520 con 0x7f8230074120 2024-05-26T22:48:28.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.735+0000 7f5891400640 1 --2- 172.21.15.131:0/476063634 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f588c2043b0 0x7f588c204830 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f588c0cfaf0 tx=0x7f5884007430 comp rx=0 tx=0).stop 2024-05-26T22:48:28.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.736+0000 7f5890a00640 1 -- 172.21.15.131:0/476063634 reap_dead start 2024-05-26T22:48:28.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.736+0000 7f5891400640 1 -- 172.21.15.131:0/476063634 shutdown_connections 2024-05-26T22:48:28.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.736+0000 7f8257800640 1 -- 172.21.15.131:0/1426942812 >> [v2:172.21.15.187:6814/244560995,v1:172.21.15.187:6816/244560995] conn(0x7f8230074120 msgr2=0x7f8230076540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:28.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.736+0000 7f8257800640 1 --2- 172.21.15.131:0/1426942812 >> [v2:172.21.15.187:6814/244560995,v1:172.21.15.187:6816/244560995] conn(0x7f8230074120 0x7f8230076540 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:28.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.736+0000 7f5891400640 1 -- 172.21.15.131:0/476063634 >> 172.21.15.131:0/476063634 conn(0x7f588c072e60 msgr2=0x7f588c160c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:28.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.736+0000 7f8257800640 1 -- 172.21.15.131:0/1426942812 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f823006bf00 msgr2=0x7f823006e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:28.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.736+0000 7f8257800640 1 --2- 172.21.15.131:0/1426942812 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f823006bf00 0x7f823006e3c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f823c007fd0 tx=0x7f823c007de0 comp rx=0 tx=0).stop 2024-05-26T22:48:28.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.736+0000 7f8257800640 1 -- 172.21.15.131:0/1426942812 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f82500d0560 msgr2=0x7f825009aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:28.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.736+0000 7f8257800640 1 --2- 172.21.15.131:0/1426942812 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f82500d0560 0x7f825009aa00 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f8240031000 tx=0x7f82400046c0 comp rx=0 tx=0).stop 2024-05-26T22:48:28.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.737+0000 7f8255800640 1 -- 172.21.15.131:0/1426942812 reap_dead start 2024-05-26T22:48:28.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.737+0000 7f8257800640 1 -- 172.21.15.131:0/1426942812 shutdown_connections 2024-05-26T22:48:28.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.737+0000 7f8257800640 1 -- 172.21.15.131:0/1426942812 >> 172.21.15.131:0/1426942812 conn(0x7f825006bc90 msgr2=0x7f8250163ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:28.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.737+0000 7f8257800640 1 -- 172.21.15.131:0/1426942812 shutdown_connections 2024-05-26T22:48:28.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.737+0000 7f8257800640 1 -- 172.21.15.131:0/1426942812 wait complete. 2024-05-26T22:48:28.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.738+0000 7f11c2200640 1 -- 172.21.15.131:0/325706192 --> [v2:172.21.15.187:6824/1220065622,v1:172.21.15.187:6825/1220065622] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f11bc0695c0 con 0x7f1198074200 2024-05-26T22:48:28.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.738+0000 7f5891400640 1 -- 172.21.15.131:0/476063634 shutdown_connections 2024-05-26T22:48:28.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.738+0000 7f5891400640 1 -- 172.21.15.131:0/476063634 wait complete. 2024-05-26T22:48:28.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.741+0000 7f11b3e00640 1 -- 172.21.15.131:0/325706192 <== osd.7 v2:172.21.15.187:6824/1220065622 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f11bc0695c0 con 0x7f1198074200 2024-05-26T22:48:28.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.740+0000 7ff3c7800640 1 -- 172.21.15.131:0/3918973407 --> [v2:172.21.15.187:6800/324619398,v1:172.21.15.187:6801/324619398] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff398002dd0 con 0x7ff39c0741d0 2024-05-26T22:48:28.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.743+0000 7ff3bcc00640 1 -- 172.21.15.131:0/3918973407 <== osd.5 v2:172.21.15.187:6800/324619398 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ff398002dd0 con 0x7ff39c0741d0 2024-05-26T22:48:28.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.743+0000 7f11c2200640 1 -- 172.21.15.131:0/325706192 >> [v2:172.21.15.187:6824/1220065622,v1:172.21.15.187:6825/1220065622] conn(0x7f1198074200 msgr2=0x7f1198076620 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:28.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.743+0000 7f11c2200640 1 --2- 172.21.15.131:0/325706192 >> [v2:172.21.15.187:6824/1220065622,v1:172.21.15.187:6825/1220065622] conn(0x7f1198074200 0x7f1198076620 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:28.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.744+0000 7f11c2200640 1 -- 172.21.15.131:0/325706192 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f119806c060 msgr2=0x7f119806e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:28.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.744+0000 7f11c2200640 1 --2- 172.21.15.131:0/325706192 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f119806c060 0x7f119806e520 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f11ac038000 tx=0x7f11ac00d000 comp rx=0 tx=0).stop 2024-05-26T22:48:28.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.744+0000 7f11c2200640 1 -- 172.21.15.131:0/325706192 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f11bc0cfe50 msgr2=0x7f11bc153ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:28.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.744+0000 7f11c2200640 1 --2- 172.21.15.131:0/325706192 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f11bc0cfe50 0x7f11bc153ac0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f11a400e9b0 tx=0x7f11a400ee70 comp rx=0 tx=0).stop 2024-05-26T22:48:28.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.744+0000 7f11bbe00640 1 -- 172.21.15.131:0/325706192 reap_dead start 2024-05-26T22:48:28.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.745+0000 7f11c2200640 1 -- 172.21.15.131:0/325706192 shutdown_connections 2024-05-26T22:48:28.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.745+0000 7f11c2200640 1 -- 172.21.15.131:0/325706192 >> 172.21.15.131:0/325706192 conn(0x7f11bc06b940 msgr2=0x7f11bc0ae2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:28.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.745+0000 7ff3c7800640 1 -- 172.21.15.131:0/3918973407 >> [v2:172.21.15.187:6800/324619398,v1:172.21.15.187:6801/324619398] conn(0x7ff39c0741d0 msgr2=0x7ff39c0765f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:28.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.745+0000 7ff3c7800640 1 --2- 172.21.15.131:0/3918973407 >> [v2:172.21.15.187:6800/324619398,v1:172.21.15.187:6801/324619398] conn(0x7ff39c0741d0 0x7ff39c0765f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:28.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.745+0000 7ff3c7800640 1 -- 172.21.15.131:0/3918973407 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ff39c06c030 msgr2=0x7ff39c06e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:28.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.745+0000 7ff3c7800640 1 --2- 172.21.15.131:0/3918973407 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ff39c06c030 0x7ff39c06e4f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff3b80097a0 tx=0x7ff3b802e000 comp rx=0 tx=0).stop 2024-05-26T22:48:28.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.745+0000 7ff3c7800640 1 -- 172.21.15.131:0/3918973407 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff3c00d0560 msgr2=0x7ff3c0160690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:28.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.745+0000 7ff3c7800640 1 --2- 172.21.15.131:0/3918973407 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff3c00d0560 0x7ff3c0160690 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ff3b000c3e0 tx=0x7ff3b000c8a0 comp rx=0 tx=0).stop 2024-05-26T22:48:28.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.745+0000 7ff3c5800640 1 -- 172.21.15.131:0/3918973407 reap_dead start 2024-05-26T22:48:28.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.746+0000 7ff3c7800640 1 -- 172.21.15.131:0/3918973407 shutdown_connections 2024-05-26T22:48:28.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.746+0000 7ff3c7800640 1 -- 172.21.15.131:0/3918973407 >> 172.21.15.131:0/3918973407 conn(0x7ff3c006bc90 msgr2=0x7ff3c006d520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:28.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.747+0000 7ff3c7800640 1 -- 172.21.15.131:0/3918973407 shutdown_connections 2024-05-26T22:48:28.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.747+0000 7ff3c7800640 1 -- 172.21.15.131:0/3918973407 wait complete. 2024-05-26T22:48:28.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.748+0000 7f11c2200640 1 -- 172.21.15.131:0/325706192 shutdown_connections 2024-05-26T22:48:28.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.748+0000 7f11c2200640 1 -- 172.21.15.131:0/325706192 wait complete. 2024-05-26T22:48:28.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.748+0000 7f3a5d600640 1 -- 172.21.15.131:0/808100511 --> [v2:172.21.15.187:6804/1391347786,v1:172.21.15.187:6806/1391347786] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3a580d2a00 con 0x7f3a38074150 2024-05-26T22:48:28.785 INFO:teuthology.orchestra.run.smithi131.stdout:55834574854 2024-05-26T22:48:28.786 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-05-26T22:48:28.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.750+0000 7f3a4ea00640 1 -- 172.21.15.131:0/808100511 <== osd.6 v2:172.21.15.187:6804/1391347786 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f3a580d2a00 con 0x7f3a38074150 2024-05-26T22:48:28.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.751+0000 7f3a5d600640 1 -- 172.21.15.131:0/808100511 >> [v2:172.21.15.187:6804/1391347786,v1:172.21.15.187:6806/1391347786] conn(0x7f3a38074150 msgr2=0x7f3a38076570 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:28.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.751+0000 7f3a5d600640 1 --2- 172.21.15.131:0/808100511 >> [v2:172.21.15.187:6804/1391347786,v1:172.21.15.187:6806/1391347786] conn(0x7f3a38074150 0x7f3a38076570 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:28.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.751+0000 7f3a5d600640 1 -- 172.21.15.131:0/808100511 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f3a3806bf30 msgr2=0x7f3a3806e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:28.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.751+0000 7f3a5d600640 1 --2- 172.21.15.131:0/808100511 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f3a3806bf30 0x7f3a3806e3f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3a580b1690 tx=0x7f3a440090e0 comp rx=0 tx=0).stop 2024-05-26T22:48:28.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.751+0000 7f3a5d600640 1 -- 172.21.15.131:0/808100511 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a580b6900 msgr2=0x7f3a58150d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:28.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.751+0000 7f3a5d600640 1 --2- 172.21.15.131:0/808100511 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a580b6900 0x7f3a58150d80 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3a48009870 tx=0x7f3a480046c0 comp rx=0 tx=0).stop 2024-05-26T22:48:28.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.751+0000 7f3a57400640 1 -- 172.21.15.131:0/808100511 reap_dead start 2024-05-26T22:48:28.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.751+0000 7f3a5d600640 1 -- 172.21.15.131:0/808100511 shutdown_connections 2024-05-26T22:48:28.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.751+0000 7f3a5d600640 1 -- 172.21.15.131:0/808100511 >> 172.21.15.131:0/808100511 conn(0x7f3a5806f6d0 msgr2=0x7f3a580b5aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:28.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.752+0000 7f3a5d600640 1 -- 172.21.15.131:0/808100511 shutdown_connections 2024-05-26T22:48:28.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:28.752+0000 7f3a5d600640 1 -- 172.21.15.131:0/808100511 wait complete. 2024-05-26T22:48:28.789 INFO:teuthology.orchestra.run.smithi131.stdout:55834574854 2024-05-26T22:48:28.789 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-05-26T22:48:28.789 INFO:teuthology.orchestra.run.smithi131.stdout:55834574854 2024-05-26T22:48:28.790 INFO:teuthology.orchestra.run.smithi131.stdout:55834574854 2024-05-26T22:48:28.791 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-05-26T22:48:28.791 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-05-26T22:48:28.792 INFO:teuthology.orchestra.run.smithi131.stdout:55834574854 2024-05-26T22:48:28.792 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-05-26T22:48:28.793 INFO:teuthology.orchestra.run.smithi131.stdout:55834574855 2024-05-26T22:48:28.793 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-05-26T22:48:28.794 INFO:teuthology.orchestra.run.smithi131.stdout:55834574854 2024-05-26T22:48:28.794 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-05-26T22:48:28.795 INFO:teuthology.orchestra.run.smithi131.stdout:55834574854 2024-05-26T22:48:28.795 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-05-26T22:48:29.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.107+0000 7fd916200640 1 Processor -- start 2024-05-26T22:48:29.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.107+0000 7f6035400640 1 Processor -- start 2024-05-26T22:48:29.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.107+0000 7fd916200640 1 -- start start 2024-05-26T22:48:29.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.109+0000 7fd916200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd9100d1190 0x7fd9100d25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:29.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.109+0000 7fd916200640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fd9100b7330 0x7fd9100b7790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:29.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.109+0000 7fd916200640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9100ba140 con 0x7fd9100b9ce0 2024-05-26T22:48:29.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.109+0000 7fd916200640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fd9100d2cd0 con 0x7fd9100b7330 2024-05-26T22:48:29.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.109+0000 7fd916200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd9100d2fb0 con 0x7fd9100d1190 2024-05-26T22:48:29.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.110+0000 7fd915800640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fd9100b7330 0x7fd9100b7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:29.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.110+0000 7fd915800640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fd9100b7330 0x7fd9100b7790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.131:36444/0 (socket says 172.21.15.131:36444) 2024-05-26T22:48:29.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.108+0000 7f6035400640 1 -- start start 2024-05-26T22:48:29.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.110+0000 7f6035400640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f602800a520 0x7f60280e4d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:29.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.110+0000 7f6035400640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6028051c90 con 0x7f60280e59b0 2024-05-26T22:48:29.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.110+0000 7f6035400640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6028051f70 con 0x7f60280e3f70 2024-05-26T22:48:29.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.110+0000 7f6035400640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f6028052250 con 0x7f602800a520 2024-05-26T22:48:29.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.110+0000 7fd915800640 1 -- 172.21.15.131:0/2718415554 learned_addr learned my addr 172.21.15.131:0/2718415554 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:29.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.110+0000 7fd914e00640 1 --2- 172.21.15.131:0/2718415554 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd9100d1190 0x7fd9100d25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:29.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.111+0000 7f602f400640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f602800a520 0x7f60280e4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:29.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.111+0000 7f602f400640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f602800a520 0x7f60280e4d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.131:36460/0 (socket says 172.21.15.131:36460) 2024-05-26T22:48:29.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.111+0000 7f602f400640 1 -- 172.21.15.131:0/372775581 learned_addr learned my addr 172.21.15.131:0/372775581 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:29.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.111+0000 7fd915800640 1 -- 172.21.15.131:0/2718415554 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd9100d1190 msgr2=0x7fd9100d25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:29.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.111+0000 7fd915800640 1 --2- 172.21.15.131:0/2718415554 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd9100d1190 0x7fd9100d25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:29.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.112+0000 7fd915800640 1 -- 172.21.15.131:0/2718415554 >> v1:172.21.15.187:6789/0 conn(0x7fd9100b9ce0 legacy=0x7fd9100d1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:29.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.112+0000 7fd915800640 1 -- 172.21.15.131:0/2718415554 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9100d3290 con 0x7fd9100b7330 2024-05-26T22:48:29.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.113+0000 7f602f400640 1 -- 172.21.15.131:0/372775581 >> v1:172.21.15.187:6789/0 conn(0x7f60280e3f70 legacy=0x7f60280e0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:29.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.113+0000 7f602f400640 1 -- 172.21.15.131:0/372775581 >> v1:172.21.15.131:6789/0 conn(0x7f60280e59b0 legacy=0x7f6028009e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:29.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.113+0000 7f602f400640 1 -- 172.21.15.131:0/372775581 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6028052530 con 0x7f602800a520 2024-05-26T22:48:29.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.115+0000 7fd915800640 1 --2- 172.21.15.131:0/2718415554 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd9100b7330 0x7fd9100b7790 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd908009e50 tx=0x7fd90802a710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b67110cf984bd2cf server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:29.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.115+0000 7f602f400640 1 --2- 172.21.15.131:0/372775581 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f602800a520 0x7f60280e4d40 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f601c009870 tx=0x7f601c02a280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=337df5c2d72c8b7c server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:29.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.116+0000 7f602ea00640 1 -- 172.21.15.131:0/372775581 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f601c037070 con 0x7f602800a520 2024-05-26T22:48:29.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.116+0000 7fd90f400640 1 -- 172.21.15.131:0/2718415554 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd908036070 con 0x7fd9100b7330 2024-05-26T22:48:29.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.117+0000 7fd90f400640 1 -- 172.21.15.131:0/2718415554 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd908031600 con 0x7fd9100b7330 2024-05-26T22:48:29.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.117+0000 7fd90f400640 1 -- 172.21.15.131:0/2718415554 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9080319d0 con 0x7fd9100b7330 2024-05-26T22:48:29.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.117+0000 7f602ea00640 1 -- 172.21.15.131:0/372775581 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f601c032440 con 0x7f602800a520 2024-05-26T22:48:29.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.117+0000 7f602ea00640 1 -- 172.21.15.131:0/372775581 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f601c032810 con 0x7f602800a520 2024-05-26T22:48:29.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.118+0000 7fd916200640 1 -- 172.21.15.131:0/2718415554 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd9100b7330 msgr2=0x7fd9100b7790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:29.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.118+0000 7fd916200640 1 --2- 172.21.15.131:0/2718415554 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd9100b7330 0x7fd9100b7790 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd908009e50 tx=0x7fd90802a710 comp rx=0 tx=0).stop 2024-05-26T22:48:29.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.118+0000 7f6035400640 1 -- 172.21.15.131:0/372775581 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f602800a520 msgr2=0x7f60280e4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:29.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.118+0000 7f6035400640 1 --2- 172.21.15.131:0/372775581 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f602800a520 0x7f60280e4d40 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f601c009870 tx=0x7f601c02a280 comp rx=0 tx=0).stop 2024-05-26T22:48:29.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.118+0000 7fd916200640 1 -- 172.21.15.131:0/2718415554 shutdown_connections 2024-05-26T22:48:29.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.118+0000 7fd916200640 1 --2- 172.21.15.131:0/2718415554 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd9100b7330 0x7fd9100b7790 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:29.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.118+0000 7fd916200640 1 --2- 172.21.15.131:0/2718415554 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd9100d1190 0x7fd9100d25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:29.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.118+0000 7fd916200640 1 -- 172.21.15.131:0/2718415554 >> 172.21.15.131:0/2718415554 conn(0x7fd910072e60 msgr2=0x7fd910073270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:29.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.119+0000 7fd916200640 1 -- 172.21.15.131:0/2718415554 shutdown_connections 2024-05-26T22:48:29.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.119+0000 7fd916200640 1 -- 172.21.15.131:0/2718415554 wait complete. 2024-05-26T22:48:29.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.119+0000 7f6035400640 1 -- 172.21.15.131:0/372775581 shutdown_connections 2024-05-26T22:48:29.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.119+0000 7f6035400640 1 --2- 172.21.15.131:0/372775581 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f602800a520 0x7f60280e4d40 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:29.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.119+0000 7f6035400640 1 -- 172.21.15.131:0/372775581 >> 172.21.15.131:0/372775581 conn(0x7f60280537b0 msgr2=0x7f6028053bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:29.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.119+0000 7f6035400640 1 -- 172.21.15.131:0/372775581 shutdown_connections 2024-05-26T22:48:29.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.119+0000 7f6035400640 1 -- 172.21.15.131:0/372775581 wait complete. 2024-05-26T22:48:29.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.119+0000 7f2176600640 1 Processor -- start 2024-05-26T22:48:29.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.119+0000 7f2176600640 1 -- start start 2024-05-26T22:48:29.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.121+0000 7f2176600640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f21700d03d0 0x7f21700d1470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:29.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.121+0000 7f2176600640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21700cf330 con 0x7f21700d2810 2024-05-26T22:48:29.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.121+0000 7f6035400640 1 Processor -- start 2024-05-26T22:48:29.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.121+0000 7f2176600640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21700cf610 con 0x7f217015e870 2024-05-26T22:48:29.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.121+0000 7f2176600640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f21700cff40 con 0x7f21700d03d0 2024-05-26T22:48:29.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.122+0000 7f6035400640 1 -- start start 2024-05-26T22:48:29.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.122+0000 7f216f400640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f21700d2810 0x7f21700d2bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:48900/0 (socket says 172.21.15.131:48900) 2024-05-26T22:48:29.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.122+0000 7fd916200640 1 Processor -- start 2024-05-26T22:48:29.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.122+0000 7f216f400640 1 -- 172.21.15.131:0/3008414486 learned_addr learned my addr 172.21.15.131:0/3008414486 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:48:29.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.123+0000 7f216fe00640 1 --2- 172.21.15.131:0/3008414486 >> v2:172.21.15.187:3300/0 conn(0x7f21700d03d0 0x7f21700d1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:29.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.123+0000 7fd916200640 1 -- start start 2024-05-26T22:48:29.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.123+0000 7f216e000640 1 -- 172.21.15.131:0/3008414486 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1846864533 0 0) 0x7f21700cf610 con 0x7f217015e870 2024-05-26T22:48:29.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.123+0000 7f6035400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f602800a520 0x7f602817b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:29.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.124+0000 7f216e000640 1 -- 172.21.15.131:0/3008414486 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2154003520 con 0x7f217015e870 2024-05-26T22:48:29.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.124+0000 7f6035400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f60280e3f70 0x7f602804aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:29.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.124+0000 7f6035400640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f60280e59b0 0x7f602817add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:29.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.124+0000 7f6035400640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f60280518a0 con 0x7f60280e3f70 2024-05-26T22:48:29.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.124+0000 7f6035400640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6028051720 con 0x7f60280e59b0 2024-05-26T22:48:29.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.124+0000 7f6035400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f6028051a20 con 0x7f602800a520 2024-05-26T22:48:29.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.124+0000 7f2f9f600640 1 Processor -- start 2024-05-26T22:48:29.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.124+0000 7f216e000640 1 -- 172.21.15.131:0/3008414486 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2735467559 0 0) 0x7f21700cf330 con 0x7f21700d2810 2024-05-26T22:48:29.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.124+0000 7fd916200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd9100b7330 0x7fd9101f3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:29.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.124+0000 7fd916200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd9100b9ce0 0x7fd91015f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:29.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.124+0000 7fd916200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd9100d1190 0x7fd9100cdbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:29.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.124+0000 7f216e000640 1 -- 172.21.15.131:0/3008414486 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f21700cf610 con 0x7f21700d2810 2024-05-26T22:48:29.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.125+0000 7fd916200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fd9100d3d70 con 0x7fd9100b9ce0 2024-05-26T22:48:29.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.125+0000 7fd916200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd9100d3850 con 0x7fd9100b7330 2024-05-26T22:48:29.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.125+0000 7fd916200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd91015a8e0 con 0x7fd9100d1190 2024-05-26T22:48:29.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.125+0000 7fd914e00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd9100b7330 0x7fd9101f3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:29.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.124+0000 7f602fe00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f602800a520 0x7f602817b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:29.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.124+0000 7f602f400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f60280e3f70 0x7f602804aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:29.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.124+0000 7f602f400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f60280e3f70 0x7f602804aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54110/0 (socket says 172.21.15.131:54110) 2024-05-26T22:48:29.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.124+0000 7f602f400640 1 -- 172.21.15.131:0/3303070870 learned_addr learned my addr 172.21.15.131:0/3303070870 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:29.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.124+0000 7f602fe00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f602800a520 0x7f602817b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.131:43278/0 (socket says 172.21.15.131:43278) 2024-05-26T22:48:29.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.125+0000 7f6034a00640 1 --2- 172.21.15.131:0/3303070870 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f60280e59b0 0x7f602817add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:29.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.125+0000 7fd90fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd9100b9ce0 0x7fd91015f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:29.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.125+0000 7fd914e00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd9100b7330 0x7fd9101f3c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.131:36484/0 (socket says 172.21.15.131:36484) 2024-05-26T22:48:29.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.125+0000 7fd914e00640 1 -- 172.21.15.131:0/1015536175 learned_addr learned my addr 172.21.15.131:0/1015536175 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:29.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.125+0000 7fd915800640 1 --2- 172.21.15.131:0/1015536175 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd9100d1190 0x7fd9100cdbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:29.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.125+0000 7f2f9f600640 1 -- start start 2024-05-26T22:48:29.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.125+0000 7f602f400640 1 -- 172.21.15.131:0/3303070870 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f602800a520 msgr2=0x7f602817b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:29.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.125+0000 7f602f400640 1 --2- 172.21.15.131:0/3303070870 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f602800a520 0x7f602817b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:29.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.125+0000 7f602f400640 1 -- 172.21.15.131:0/3303070870 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f60280e59b0 msgr2=0x7f602817add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:29.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.125+0000 7f602f400640 1 --2- 172.21.15.131:0/3303070870 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f60280e59b0 0x7f602817add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:29.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.125+0000 7f602f400640 1 -- 172.21.15.131:0/3303070870 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f602804af60 con 0x7f60280e3f70 2024-05-26T22:48:29.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.124+0000 7f216fe00640 1 -- 172.21.15.131:0/3008414486 >> v1:172.21.15.187:6789/0 conn(0x7f217015e870 legacy=0x7f21700b3b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:29.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.124+0000 7f216fe00640 1 -- 172.21.15.131:0/3008414486 >> v1:172.21.15.131:6789/0 conn(0x7f21700d2810 legacy=0x7f21700d2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:29.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.124+0000 7f216fe00640 1 -- 172.21.15.131:0/3008414486 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21700cf8f0 con 0x7f21700d03d0 2024-05-26T22:48:29.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.126+0000 7fd90fe00640 1 -- 172.21.15.131:0/1015536175 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd9100d1190 msgr2=0x7fd9100cdbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:29.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.126+0000 7fd90fe00640 1 --2- 172.21.15.131:0/1015536175 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd9100d1190 0x7fd9100cdbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:29.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.126+0000 7fd90fe00640 1 -- 172.21.15.131:0/1015536175 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd9100b7330 msgr2=0x7fd9101f3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:29.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.126+0000 7fd90fe00640 1 --2- 172.21.15.131:0/1015536175 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd9100b7330 0x7fd9101f3c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:29.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.126+0000 7fd90fe00640 1 -- 172.21.15.131:0/1015536175 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9100c8b80 con 0x7fd9100b9ce0 2024-05-26T22:48:29.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.127+0000 7f216fe00640 1 --2- 172.21.15.131:0/3008414486 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f21700d03d0 0x7f21700d1470 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f21600097a0 tx=0x7f216002a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cea8777571d70cef server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:29.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.127+0000 7fd914e00640 1 --2- 172.21.15.131:0/1015536175 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd9100b7330 0x7fd9101f3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-26T22:48:29.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.127+0000 7f602f400640 1 --2- 172.21.15.131:0/3303070870 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f60280e3f70 0x7f602804aa20 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f601c009870 tx=0x7f601c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:29.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.127+0000 7fd90fe00640 1 --2- 172.21.15.131:0/1015536175 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd9100b9ce0 0x7fd91015f8d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc00c3c0 tx=0x7fd8fc00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:29.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.127+0000 7f216e000640 1 -- 172.21.15.131:0/3008414486 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2160037070 con 0x7f21700d03d0 2024-05-26T22:48:29.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.128+0000 7fd90cc00640 1 -- 172.21.15.131:0/1015536175 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8fc017070 con 0x7fd9100b9ce0 2024-05-26T22:48:29.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.128+0000 7f216e000640 1 -- 172.21.15.131:0/3008414486 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2160032440 con 0x7f21700d03d0 2024-05-26T22:48:29.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.128+0000 7f216e000640 1 -- 172.21.15.131:0/3008414486 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2160032810 con 0x7f21700d03d0 2024-05-26T22:48:29.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.128+0000 7f2f9f600640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f2f9815e890 0x7f2f980cecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:29.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.128+0000 7f2f9f600640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f980cf320 con 0x7f2f980d1470 2024-05-26T22:48:29.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.128+0000 7f2f9f600640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f980cf600 con 0x7f2f980b77d0 2024-05-26T22:48:29.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.128+0000 7f2f9f600640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f2f980cf8e0 con 0x7f2f9815e890 2024-05-26T22:48:29.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.127+0000 7f6023e00640 1 -- 172.21.15.131:0/3303070870 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f601c041020 con 0x7f60280e3f70 2024-05-26T22:48:29.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.127+0000 7f6023e00640 1 -- 172.21.15.131:0/3303070870 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f601c030970 con 0x7f60280e3f70 2024-05-26T22:48:29.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.128+0000 7f6023e00640 1 -- 172.21.15.131:0/3303070870 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f601c037040 con 0x7f60280e3f70 2024-05-26T22:48:29.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.129+0000 7f2f8fe00640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f2f980b77d0 0x7f2f980d0d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.131:57670/0 (socket says 172.21.15.131:57670) 2024-05-26T22:48:29.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.129+0000 7f2f8fe00640 1 -- 172.21.15.131:0/2249793941 learned_addr learned my addr 172.21.15.131:0/2249793941 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:48:29.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.129+0000 7f2176600640 1 -- 172.21.15.131:0/3008414486 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f21700d03d0 msgr2=0x7f21700d1470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:29.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.129+0000 7f2176600640 1 --2- 172.21.15.131:0/3008414486 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f21700d03d0 0x7f21700d1470 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f21600097a0 tx=0x7f216002a270 comp rx=0 tx=0).stop 2024-05-26T22:48:29.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.129+0000 7f6035400640 1 -- 172.21.15.131:0/3303070870 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6028037b90 con 0x7f60280e3f70 2024-05-26T22:48:29.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.128+0000 7fd90cc00640 1 -- 172.21.15.131:0/1015536175 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8fc0052d0 con 0x7fd9100b9ce0 2024-05-26T22:48:29.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.129+0000 7f6035400640 1 -- 172.21.15.131:0/3303070870 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6028038150 con 0x7f60280e3f70 2024-05-26T22:48:29.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.129+0000 7f2f97e00640 1 -- 172.21.15.131:0/2249793941 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3427681381 0 0) 0x7f2f980cf600 con 0x7f2f980b77d0 2024-05-26T22:48:29.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.130+0000 7f2f97e00640 1 -- 172.21.15.131:0/2249793941 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2f7c003520 con 0x7f2f980b77d0 2024-05-26T22:48:29.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.130+0000 7f2f9cc00640 1 --2- 172.21.15.131:0/2249793941 >> v2:172.21.15.131:3300/0 conn(0x7f2f9815e890 0x7f2f980cecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:29.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.129+0000 7fd90cc00640 1 -- 172.21.15.131:0/1015536175 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8fc003830 con 0x7fd9100b9ce0 2024-05-26T22:48:29.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.129+0000 7fd916200640 1 -- 172.21.15.131:0/1015536175 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9100c8e60 con 0x7fd9100b9ce0 2024-05-26T22:48:29.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.130+0000 7fd916200640 1 -- 172.21.15.131:0/1015536175 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9100c9420 con 0x7fd9100b9ce0 2024-05-26T22:48:29.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.130+0000 7f2f97e00640 1 -- 172.21.15.131:0/2249793941 <== mon.1 v1:172.21.15.187:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3610193488 0 0) 0x7f2f7c003520 con 0x7f2f980b77d0 2024-05-26T22:48:29.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.130+0000 7f2f97e00640 1 -- 172.21.15.131:0/2249793941 >> v2:172.21.15.131:3300/0 conn(0x7f2f9815e890 msgr2=0x7f2f980cecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:29.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.130+0000 7f2f97e00640 1 --2- 172.21.15.131:0/2249793941 >> v2:172.21.15.131:3300/0 conn(0x7f2f9815e890 0x7f2f980cecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:29.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.130+0000 7f2f97e00640 1 -- 172.21.15.131:0/2249793941 >> v1:172.21.15.131:6789/0 conn(0x7f2f980d1470 legacy=0x7f2f980d2b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:29.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.131+0000 7f2176600640 1 -- 172.21.15.131:0/3008414486 shutdown_connections 2024-05-26T22:48:29.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.131+0000 7f2176600640 1 --2- 172.21.15.131:0/3008414486 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f21700d03d0 0x7f21700d1470 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:29.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.131+0000 7f2f97e00640 1 -- 172.21.15.131:0/2249793941 --> v1:172.21.15.187:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f980cfbc0 con 0x7f2f980b77d0 2024-05-26T22:48:29.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.131+0000 7f2f97e00640 1 -- 172.21.15.131:0/2249793941 <== mon.1 v1:172.21.15.187:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7f2f80001670 con 0x7f2f980b77d0 2024-05-26T22:48:29.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.131+0000 7f2f97e00640 1 -- 172.21.15.131:0/2249793941 >> v1:172.21.15.187:6789/0 conn(0x7f2f980b77d0 legacy=0x7f2f980d0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:29.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.131+0000 7f2f97e00640 1 --2- 172.21.15.131:0/2249793941 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f7c0039b0 0x7f2f7c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:29.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.131+0000 7f2f97e00640 1 --2- 172.21.15.131:0/2249793941 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2f7c004800 0x7f2f7c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:29.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.131+0000 7f2f97e00640 1 --2- 172.21.15.131:0/2249793941 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2f7c0051c0 0x7f2f7c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:29.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.131+0000 7f2f97e00640 1 -- 172.21.15.131:0/2249793941 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f2f980cf8e0 con 0x7f2f7c0051c0 2024-05-26T22:48:29.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.131+0000 7f2f97e00640 1 -- 172.21.15.131:0/2249793941 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2f7c00c060 con 0x7f2f7c0039b0 2024-05-26T22:48:29.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.131+0000 7f2f97e00640 1 -- 172.21.15.131:0/2249793941 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f2f7c00c260 con 0x7f2f7c004800 2024-05-26T22:48:29.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.131+0000 7f2176600640 1 -- 172.21.15.131:0/3008414486 >> 172.21.15.131:0/3008414486 conn(0x7f217009d5a0 msgr2=0x7f21700d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:29.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.132+0000 7f2176600640 1 -- 172.21.15.131:0/3008414486 shutdown_connections 2024-05-26T22:48:29.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.131+0000 7f6021600640 1 -- 172.21.15.131:0/3303070870 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6004005290 con 0x7f60280e3f70 2024-05-26T22:48:29.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.131+0000 7f9b4ac00640 1 Processor -- start 2024-05-26T22:48:29.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.132+0000 7f9b4ac00640 1 -- start start 2024-05-26T22:48:29.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.132+0000 7f2176600640 1 -- 172.21.15.131:0/3008414486 wait complete. 2024-05-26T22:48:29.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.132+0000 7f2f9d600640 1 --2- 172.21.15.131:0/2249793941 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2f7c0051c0 0x7f2f7c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:29.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.132+0000 7f2f8fe00640 1 --2- 172.21.15.131:0/2249793941 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2f7c004800 0x7f2f7c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:29.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.132+0000 7f347bc00640 1 Processor -- start 2024-05-26T22:48:29.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.132+0000 7f347bc00640 1 -- start start 2024-05-26T22:48:29.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.131+0000 7fd916200640 1 -- 172.21.15.131:0/1015536175 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9100694e0 con 0x7fd9100b9ce0 2024-05-26T22:48:29.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.134+0000 7f2f9cc00640 1 --2- 172.21.15.131:0/2249793941 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f7c0039b0 0x7f2f7c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:29.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.134+0000 7f9b4ac00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f9b4415e890 0x7f9b440cecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:29.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.134+0000 7f9b4ac00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f9b440d1470 0x7f9b44162cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:29.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.134+0000 7f9b4ac00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b440cf320 con 0x7f9b440b77d0 2024-05-26T22:48:29.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.134+0000 7f9b4ac00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f9b441631f0 con 0x7f9b4415e890 2024-05-26T22:48:29.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.134+0000 7f9b4ac00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f9b440cf600 con 0x7f9b440d1470 2024-05-26T22:48:29.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.134+0000 7fd90cc00640 1 -- 172.21.15.131:0/1015536175 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd8fc01a070 con 0x7fd9100b9ce0 2024-05-26T22:48:29.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.135+0000 7f2176600640 1 Processor -- start 2024-05-26T22:48:29.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.135+0000 7f9b48c00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f9b440d1470 0x7f9b44162cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:29.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.135+0000 7f9b48c00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f9b440d1470 0x7f9b44162cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.131:36500/0 (socket says 172.21.15.131:36500) 2024-05-26T22:48:29.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.136+0000 7f9b48c00640 1 -- 172.21.15.131:0/3570270680 learned_addr learned my addr 172.21.15.131:0/3570270680 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:29.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.136+0000 7ff2d0600640 1 Processor -- start 2024-05-26T22:48:29.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.135+0000 7f347bc00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f34740d04f0 0x7f3474073130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:29.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.136+0000 7f2176600640 1 -- start start 2024-05-26T22:48:29.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.135+0000 7f2f8fe00640 1 -- 172.21.15.131:0/2249793941 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f7c0039b0 msgr2=0x7f2f7c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:29.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.135+0000 7f2f8fe00640 1 --2- 172.21.15.131:0/2249793941 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f7c0039b0 0x7f2f7c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:29.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.135+0000 7f2f8fe00640 1 -- 172.21.15.131:0/2249793941 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2f7c0051c0 msgr2=0x7f2f7c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:29.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.135+0000 7f2f8fe00640 1 --2- 172.21.15.131:0/2249793941 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2f7c0051c0 0x7f2f7c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:29.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.135+0000 7f2f8fe00640 1 -- 172.21.15.131:0/2249793941 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f980cfbc0 con 0x7f2f7c004800 2024-05-26T22:48:29.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.135+0000 7f2f9d600640 1 -- 172.21.15.131:0/2249793941 reap_dead start 2024-05-26T22:48:29.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.136+0000 7fd90cc00640 1 --2- 172.21.15.131:0/1015536175 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fd8f006bf00 0x7fd8f006e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:29.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.136+0000 7f347bc00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3474160860 con 0x7f34740cda90 2024-05-26T22:48:29.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.136+0000 7f3479200640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f3474160480 0x7f34740cfde0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.131:57678/0 (socket says 172.21.15.131:57678) 2024-05-26T22:48:29.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.136+0000 7f3479200640 1 -- 172.21.15.131:0/1495779814 learned_addr learned my addr 172.21.15.131:0/1495779814 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:48:29.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.136+0000 7f347bc00640 1 -- 172.21.15.131:0/1495779814 --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f347406e3e0 con 0x7f3474160480 2024-05-26T22:48:29.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.136+0000 7f347bc00640 1 -- 172.21.15.131:0/1495779814 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f347406e6c0 con 0x7f34740d04f0 2024-05-26T22:48:29.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.136+0000 7f2f8fe00640 1 --2- 172.21.15.131:0/2249793941 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2f7c004800 0x7f2f7c004c80 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f2f80001050 tx=0x7f2f7c005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af8519db58b78d2e server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:29.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.136+0000 7ff2d0600640 1 -- start start 2024-05-26T22:48:29.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.137+0000 7fd90cc00640 1 -- 172.21.15.131:0/1015536175 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd8fc08a030 con 0x7fd9100b9ce0 2024-05-26T22:48:29.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.137+0000 7f3479c00640 1 --2- 172.21.15.131:0/1495779814 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f34740d04f0 0x7f3474073130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:29.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.137+0000 7f2f97e00640 1 -- 172.21.15.131:0/2249793941 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f80007d10 con 0x7f2f7c004800 2024-05-26T22:48:29.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.137+0000 7f2f97e00640 1 -- 172.21.15.131:0/2249793941 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f80005ad0 con 0x7f2f7c004800 2024-05-26T22:48:29.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.137+0000 7f2f97e00640 1 -- 172.21.15.131:0/2249793941 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f80007070 con 0x7f2f7c004800 2024-05-26T22:48:29.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.137+0000 7f3473400640 1 -- 172.21.15.131:0/1495779814 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3921662881 0 0) 0x7f347406e3e0 con 0x7f3474160480 2024-05-26T22:48:29.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.137+0000 7f3473400640 1 -- 172.21.15.131:0/1495779814 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f345c003520 con 0x7f3474160480 2024-05-26T22:48:29.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.137+0000 7f2176600640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f21700d03d0 0x7f21700ca520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:29.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.138+0000 7f2176600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f21700d2810 0x7f2170160bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:29.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.138+0000 7f3473400640 1 -- 172.21.15.131:0/1495779814 <== mon.1 v1:172.21.15.187:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1155376672 0 0) 0x7f345c003520 con 0x7f3474160480 2024-05-26T22:48:29.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.138+0000 7f3473400640 1 -- 172.21.15.131:0/1495779814 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f34740d04f0 msgr2=0x7f3474073130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:29.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.138+0000 7f3473400640 1 --2- 172.21.15.131:0/1495779814 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f34740d04f0 0x7f3474073130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:29.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.138+0000 7f3473400640 1 -- 172.21.15.131:0/1495779814 >> v1:172.21.15.131:6789/0 conn(0x7f34740cda90 legacy=0x7f3474072a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:29.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.138+0000 7f3473400640 1 -- 172.21.15.131:0/1495779814 --> v1:172.21.15.187:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f347406e9a0 con 0x7f3474160480 2024-05-26T22:48:29.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.138+0000 7f3473400640 1 -- 172.21.15.131:0/1495779814 <== mon.1 v1:172.21.15.187:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7f3468001670 con 0x7f3474160480 2024-05-26T22:48:29.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.138+0000 7f2176600640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f217015e870 0x7f21701e8e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:29.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.137+0000 7f9b48c00640 1 -- 172.21.15.131:0/3570270680 >> v2:172.21.15.131:3300/0 conn(0x7f9b4415e890 msgr2=0x7f9b440cecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:29.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.137+0000 7f9b48c00640 1 --2- 172.21.15.131:0/3570270680 >> v2:172.21.15.131:3300/0 conn(0x7f9b4415e890 0x7f9b440cecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:29.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.137+0000 7f9b48c00640 1 -- 172.21.15.131:0/3570270680 >> v1:172.21.15.131:6789/0 conn(0x7f9b440b77d0 legacy=0x7f9b440d0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:29.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.137+0000 7f9b48c00640 1 -- 172.21.15.131:0/3570270680 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b440cf8e0 con 0x7f9b440d1470 2024-05-26T22:48:29.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.138+0000 7f2176600640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f21700d4d60 con 0x7f21700d2810 2024-05-26T22:48:29.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.138+0000 7f216f400640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f21700d03d0 0x7f21700ca520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:29.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.138+0000 7f216f400640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f21700d03d0 0x7f21700ca520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.131:36516/0 (socket says 172.21.15.131:36516) 2024-05-26T22:48:29.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.138+0000 7f216f400640 1 -- 172.21.15.131:0/1409083103 learned_addr learned my addr 172.21.15.131:0/1409083103 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:29.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.138+0000 7ff2d0600640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7ff2c815e4e0 0x7ff2c80b9c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:29.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.139+0000 7f216fe00640 1 --2- 172.21.15.131:0/1409083103 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f217015e870 0x7f21701e8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:29.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.139+0000 7f2176600640 1 -- 172.21.15.131:0/1409083103 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f21700cf020 con 0x7f21700d03d0 2024-05-26T22:48:29.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.139+0000 7f2176600640 1 -- 172.21.15.131:0/1409083103 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f21700d4ee0 con 0x7f217015e870 2024-05-26T22:48:29.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.139+0000 7ff2d0600640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff2c80cf720 0x7ff2c80b7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:29.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.138+0000 7f3473400640 1 -- 172.21.15.131:0/1495779814 >> v1:172.21.15.187:6789/0 conn(0x7f3474160480 legacy=0x7f34740cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:29.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.138+0000 7f3473400640 1 --2- 172.21.15.131:0/1495779814 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f345c0039b0 0x7f345c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:29.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.138+0000 7f3473400640 1 --2- 172.21.15.131:0/1495779814 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f345c004800 0x7f345c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:29.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.138+0000 7f3473400640 1 --2- 172.21.15.131:0/1495779814 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f345c0051c0 0x7f345c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:29.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.138+0000 7f3473400640 1 -- 172.21.15.131:0/1495779814 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f347406e6c0 con 0x7f345c0039b0 2024-05-26T22:48:29.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.138+0000 7f3473400640 1 -- 172.21.15.131:0/1495779814 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f345c00c030 con 0x7f345c0051c0 2024-05-26T22:48:29.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.138+0000 7f3473400640 1 -- 172.21.15.131:0/1495779814 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f345c00c230 con 0x7f345c004800 2024-05-26T22:48:29.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.139+0000 7ff2d0600640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2c80d28f0 con 0x7ff2c80d2330 2024-05-26T22:48:29.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.139+0000 7ff2d0600640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7ff2c80d2bd0 con 0x7ff2c815e4e0 2024-05-26T22:48:29.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.139+0000 7ff2d0600640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff2c80d2eb0 con 0x7ff2c80cf720 2024-05-26T22:48:29.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.139+0000 7f2f9f600640 1 -- 172.21.15.131:0/2249793941 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2f7c004800 msgr2=0x7f2f7c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:29.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.139+0000 7f2f9f600640 1 --2- 172.21.15.131:0/2249793941 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2f7c004800 0x7f2f7c004c80 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f2f80001050 tx=0x7f2f7c005830 comp rx=0 tx=0).stop 2024-05-26T22:48:29.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.139+0000 7f6023e00640 1 -- 172.21.15.131:0/3303070870 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f601c044020 con 0x7f60280e3f70 2024-05-26T22:48:29.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.139+0000 7ff2ce600640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff2c80cf720 0x7ff2c80b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:29.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.139+0000 7f2f9f600640 1 -- 172.21.15.131:0/2249793941 shutdown_connections 2024-05-26T22:48:29.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.139+0000 7f2f9f600640 1 --2- 172.21.15.131:0/2249793941 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2f7c004800 0x7f2f7c004c80 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:29.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.139+0000 7f2f9f600640 1 -- 172.21.15.131:0/2249793941 >> 172.21.15.131:0/2249793941 conn(0x7f2f98073070 msgr2=0x7f2f9806c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:29.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.139+0000 7f3479c00640 1 --2- 172.21.15.131:0/1495779814 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f345c0051c0 0x7f345c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:29.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.140+0000 7ff2cdc00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7ff2c815e4e0 0x7ff2c80b9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:29.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.140+0000 7ff2cdc00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7ff2c815e4e0 0x7ff2c80b9c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.131:36532/0 (socket says 172.21.15.131:36532) 2024-05-26T22:48:29.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.140+0000 7ff2cdc00640 1 -- 172.21.15.131:0/1508089239 learned_addr learned my addr 172.21.15.131:0/1508089239 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:29.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.140+0000 7f2f9f600640 1 -- 172.21.15.131:0/2249793941 shutdown_connections 2024-05-26T22:48:29.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.140+0000 7f3473e00640 1 --2- 172.21.15.131:0/1495779814 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f345c004800 0x7f345c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:29.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.139+0000 7f216ea00640 1 --2- 172.21.15.131:0/1409083103 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f21700d2810 0x7f2170160bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:29.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.140+0000 7f2f9f600640 1 -- 172.21.15.131:0/2249793941 wait complete. 2024-05-26T22:48:29.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.140+0000 7fa4dde00640 1 Processor -- start 2024-05-26T22:48:29.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.140+0000 7fa4dde00640 1 -- start start 2024-05-26T22:48:29.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.140+0000 7f9b48c00640 1 --2- 172.21.15.131:0/3570270680 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9b440d1470 0x7f9b44162cb0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9b440d1fb0 tx=0x7f9b3002a2a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=49a739b8dce1aca0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:29.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.139+0000 7fd914e00640 1 --2- 172.21.15.131:0/1015536175 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fd8f006bf00 0x7fd8f006e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:29.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.140+0000 7f3479c00640 1 -- 172.21.15.131:0/1495779814 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f345c004800 msgr2=0x7f345c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:29.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.140+0000 7f3479200640 1 --2- 172.21.15.131:0/1495779814 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f345c0039b0 0x7f345c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:29.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.140+0000 7f3479c00640 1 --2- 172.21.15.131:0/1495779814 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f345c004800 0x7f345c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:29.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.140+0000 7f3479c00640 1 -- 172.21.15.131:0/1495779814 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f345c0039b0 msgr2=0x7f345c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:29.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.140+0000 7ff2cdc00640 1 -- 172.21.15.131:0/1508089239 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff2c80cf720 msgr2=0x7ff2c80b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:29.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.140+0000 7ff2cdc00640 1 --2- 172.21.15.131:0/1508089239 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff2c80cf720 0x7ff2c80b7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:29.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.140+0000 7ff2cdc00640 1 -- 172.21.15.131:0/1508089239 >> v1:172.21.15.187:6789/0 conn(0x7ff2c80d2330 legacy=0x7ff2c80cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:29.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.141+0000 7ff2cdc00640 1 -- 172.21.15.131:0/1508089239 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2c80d3190 con 0x7ff2c815e4e0 2024-05-26T22:48:29.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.141+0000 7ff2ce600640 1 --2- 172.21.15.131:0/1508089239 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff2c80cf720 0x7ff2c80b7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-26T22:48:29.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.142+0000 7fd914e00640 1 --2- 172.21.15.131:0/1015536175 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fd8f006bf00 0x7fd8f006e3c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd904009bb0 tx=0x7fd904008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:29.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.141+0000 7f216f400640 1 -- 172.21.15.131:0/1409083103 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f217015e870 msgr2=0x7f21701e8e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:29.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.141+0000 7f216f400640 1 --2- 172.21.15.131:0/1409083103 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f217015e870 0x7f21701e8e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:29.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.141+0000 7f216f400640 1 -- 172.21.15.131:0/1409083103 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f21700d2810 msgr2=0x7f2170160bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:29.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.142+0000 7f216f400640 1 --2- 172.21.15.131:0/1409083103 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f21700d2810 0x7f2170160bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:29.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.142+0000 7f216f400640 1 -- 172.21.15.131:0/1409083103 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21700c9760 con 0x7f21700d03d0 2024-05-26T22:48:29.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.142+0000 7f216ea00640 1 --2- 172.21.15.131:0/1409083103 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f21700d2810 0x7f2170160bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-26T22:48:29.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.140+0000 7f3479c00640 1 --2- 172.21.15.131:0/1495779814 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f345c0039b0 0x7f345c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:29.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.142+0000 7f3479c00640 1 -- 172.21.15.131:0/1495779814 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f347406e9a0 con 0x7f345c0051c0 2024-05-26T22:48:29.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.142+0000 7f3479200640 1 --2- 172.21.15.131:0/1495779814 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f345c0039b0 0x7f345c003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:48:29.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.142+0000 7fa4dde00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fa4d80b9f00 0x7fa4d80b5c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:29.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.143+0000 7f6023e00640 1 --2- 172.21.15.131:0/3303070870 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f601006bf00 0x7f601006e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:29.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.143+0000 7f6023e00640 1 -- 172.21.15.131:0/3303070870 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f601c0abd00 con 0x7f60280e3f70 2024-05-26T22:48:29.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.142+0000 7f2f9f600640 1 Processor -- start 2024-05-26T22:48:29.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.143+0000 7f2f9f600640 1 -- start start 2024-05-26T22:48:29.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.142+0000 7f9b42a00640 1 -- 172.21.15.131:0/3570270680 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b30037070 con 0x7f9b440d1470 2024-05-26T22:48:29.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.142+0000 7f9b42a00640 1 -- 172.21.15.131:0/3570270680 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b30032440 con 0x7f9b440d1470 2024-05-26T22:48:29.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.142+0000 7f9b42a00640 1 -- 172.21.15.131:0/3570270680 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b30037070 con 0x7f9b440d1470 2024-05-26T22:48:29.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.142+0000 7ff2cdc00640 1 --2- 172.21.15.131:0/1508089239 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff2c815e4e0 0x7ff2c80b9c80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff2b4009870 tx=0x7ff2b402a1e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c71ac39e3ffe2c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:29.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.143+0000 7ff2c7e00640 1 -- 172.21.15.131:0/1508089239 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b4037070 con 0x7ff2c815e4e0 2024-05-26T22:48:29.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.143+0000 7ff2c7e00640 1 -- 172.21.15.131:0/1508089239 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2b4032440 con 0x7ff2c815e4e0 2024-05-26T22:48:29.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.143+0000 7ff2c7e00640 1 -- 172.21.15.131:0/1508089239 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b4032810 con 0x7ff2c815e4e0 2024-05-26T22:48:29.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.142+0000 7fa4dde00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4d80d2e80 con 0x7fa4d815e3e0 2024-05-26T22:48:29.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.142+0000 7fa4dde00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4d80d3160 con 0x7fa4d80d3c40 2024-05-26T22:48:29.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.142+0000 7fa4dde00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fa4d80cf910 con 0x7fa4d80b9f00 2024-05-26T22:48:29.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.143+0000 7fa4d7e00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fa4d80b9f00 0x7fa4d80b5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:29.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.143+0000 7fa4d7e00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fa4d80b9f00 0x7fa4d80b5c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.131:36548/0 (socket says 172.21.15.131:36548) 2024-05-26T22:48:29.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.143+0000 7fa4d7e00640 1 -- 172.21.15.131:0/2353025357 learned_addr learned my addr 172.21.15.131:0/2353025357 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:29.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.144+0000 7fa4d7e00640 1 -- 172.21.15.131:0/2353025357 >> v1:172.21.15.187:6789/0 conn(0x7fa4d80d3c40 legacy=0x7fa4d80b7260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:29.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.144+0000 7f9b4ac00640 1 -- 172.21.15.131:0/3570270680 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9b440d1470 msgr2=0x7f9b44162cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:29.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.143+0000 7f602fe00640 1 --2- 172.21.15.131:0/3303070870 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f601006bf00 0x7f601006e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:29.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.144+0000 7f602fe00640 1 --2- 172.21.15.131:0/3303070870 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f601006bf00 0x7f601006e3c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6024000f60 tx=0x7f6024009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:29.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.144+0000 7f9b4ac00640 1 --2- 172.21.15.131:0/3570270680 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9b440d1470 0x7f9b44162cb0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9b440d1fb0 tx=0x7f9b3002a2a0 comp rx=0 tx=0).stop 2024-05-26T22:48:29.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.144+0000 7f6023e00640 1 -- 172.21.15.131:0/3303070870 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f601c04b030 con 0x7f60280e3f70 2024-05-26T22:48:29.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.142+0000 7f216f400640 1 --2- 172.21.15.131:0/1409083103 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f21700d03d0 0x7f21700ca520 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f216800cbc0 tx=0x7f216800a430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:29.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.144+0000 7ff2d0600640 1 -- 172.21.15.131:0/1508089239 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff2c815e4e0 msgr2=0x7ff2c80b9c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:29.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.145+0000 7ff2d0600640 1 --2- 172.21.15.131:0/1508089239 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff2c815e4e0 0x7ff2c80b9c80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff2b4009870 tx=0x7ff2b402a1e0 comp rx=0 tx=0).stop 2024-05-26T22:48:29.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.145+0000 7f2f9f600640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f7c004800 0x7f2f9814c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:29.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.143+0000 7f2167400640 1 -- 172.21.15.131:0/1409083103 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2168017070 con 0x7f21700d03d0 2024-05-26T22:48:29.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.143+0000 7f2176600640 1 -- 172.21.15.131:0/1409083103 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21700c9a40 con 0x7f21700d03d0 2024-05-26T22:48:29.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.143+0000 7f2167400640 1 -- 172.21.15.131:0/1409083103 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f216800ad40 con 0x7f21700d03d0 2024-05-26T22:48:29.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.144+0000 7f2167400640 1 -- 172.21.15.131:0/1409083103 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2168017070 con 0x7f21700d03d0 2024-05-26T22:48:29.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.145+0000 7f2176600640 1 -- 172.21.15.131:0/1409083103 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21700ca000 con 0x7f21700d03d0 2024-05-26T22:48:29.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.145+0000 7f2f9f600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2f9815bc30 0x7f2f9815c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:29.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.145+0000 7ff2d0600640 1 -- 172.21.15.131:0/1508089239 shutdown_connections 2024-05-26T22:48:29.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.145+0000 7ff2d0600640 1 --2- 172.21.15.131:0/1508089239 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff2c80cf720 0x7ff2c80b7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:29.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.145+0000 7ff2d0600640 1 --2- 172.21.15.131:0/1508089239 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff2c815e4e0 0x7ff2c80b9c80 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:29.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.145+0000 7ff2d0600640 1 -- 172.21.15.131:0/1508089239 >> 172.21.15.131:0/1508089239 conn(0x7ff2c8072e90 msgr2=0x7ff2c80732a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:29.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.145+0000 7f2f9f600640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2f981e9ad0 0x7f2f981e9f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:29.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.145+0000 7f2f9cc00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f7c004800 0x7f2f9814c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:29.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.145+0000 7f2f9f600640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f2f980cf060 con 0x7f2f9815bc30 2024-05-26T22:48:29.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.145+0000 7f2f9f600640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2f980ceee0 con 0x7f2f7c004800 2024-05-26T22:48:29.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.145+0000 7f2f9f600640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f2f98162cb0 con 0x7f2f981e9ad0 2024-05-26T22:48:29.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.145+0000 7f2f9cc00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f7c004800 0x7f2f9814c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.131:36558/0 (socket says 172.21.15.131:36558) 2024-05-26T22:48:29.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.145+0000 7f2f9cc00640 1 -- 172.21.15.131:0/1066447711 learned_addr learned my addr 172.21.15.131:0/1066447711 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:29.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.145+0000 7f9b4ac00640 1 -- 172.21.15.131:0/3570270680 shutdown_connections 2024-05-26T22:48:29.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.145+0000 7f9b4ac00640 1 --2- 172.21.15.131:0/3570270680 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9b440d1470 0x7f9b44162cb0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:29.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.145+0000 7f9b4ac00640 1 --2- 172.21.15.131:0/3570270680 >> v2:172.21.15.131:3300/0 conn(0x7f9b4415e890 0x7f9b440cecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:29.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.145+0000 7f9b4ac00640 1 -- 172.21.15.131:0/3570270680 >> 172.21.15.131:0/3570270680 conn(0x7f9b44073070 msgr2=0x7f9b4406c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:29.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.146+0000 7f2f8fe00640 1 --2- 172.21.15.131:0/1066447711 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2f9815bc30 0x7f2f9815c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:29.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.146+0000 7f3479c00640 1 --2- 172.21.15.131:0/1495779814 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f345c0051c0 0x7f345c00b9c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f34740d08d0 tx=0x7f345c005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92215b284f4c74a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:29.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.146+0000 7f2f9d600640 1 --2- 172.21.15.131:0/1066447711 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2f981e9ad0 0x7f2f981e9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:29.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.145+0000 7fa4d7e00640 1 -- 172.21.15.131:0/2353025357 >> v1:172.21.15.131:6789/0 conn(0x7fa4d815e3e0 legacy=0x7fa4d80cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:29.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.145+0000 7fa4d7e00640 1 -- 172.21.15.131:0/2353025357 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4d80d3440 con 0x7fa4d80b9f00 2024-05-26T22:48:29.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.146+0000 7f3473400640 1 -- 172.21.15.131:0/1495779814 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f346c036070 con 0x7f345c0051c0 2024-05-26T22:48:29.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.146+0000 7f3473400640 1 -- 172.21.15.131:0/1495779814 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f346c031600 con 0x7f345c0051c0 2024-05-26T22:48:29.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.146+0000 7f3473400640 1 -- 172.21.15.131:0/1495779814 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f346c009bd0 con 0x7f345c0051c0 2024-05-26T22:48:29.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.146+0000 7f2f9cc00640 1 -- 172.21.15.131:0/1066447711 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2f981e9ad0 msgr2=0x7f2f981e9f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:29.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.146+0000 7f2f9cc00640 1 --2- 172.21.15.131:0/1066447711 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2f981e9ad0 0x7f2f981e9f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:29.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.146+0000 7f2f9cc00640 1 -- 172.21.15.131:0/1066447711 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2f9815bc30 msgr2=0x7f2f9815c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:29.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.146+0000 7f2f9cc00640 1 --2- 172.21.15.131:0/1066447711 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2f9815bc30 0x7f2f9815c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:29.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.146+0000 7f2f9cc00640 1 -- 172.21.15.131:0/1066447711 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f9814cd20 con 0x7f2f7c004800 2024-05-26T22:48:29.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.147+0000 7ff2d0600640 1 -- 172.21.15.131:0/1508089239 shutdown_connections 2024-05-26T22:48:29.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.147+0000 7f2176600640 1 -- 172.21.15.131:0/1409083103 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21700695c0 con 0x7f21700d03d0 2024-05-26T22:48:29.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.147+0000 7f2f9cc00640 1 --2- 172.21.15.131:0/1066447711 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f7c004800 0x7f2f9814c7e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2f88009f00 tx=0x7f2f8800f580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:29.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.147+0000 7f347bc00640 1 -- 172.21.15.131:0/1495779814 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f345c0051c0 msgr2=0x7f345c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:29.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.147+0000 7f347bc00640 1 --2- 172.21.15.131:0/1495779814 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f345c0051c0 0x7f345c00b9c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f34740d08d0 tx=0x7f345c005830 comp rx=0 tx=0).stop 2024-05-26T22:48:29.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.147+0000 7f3479c00640 1 -- 172.21.15.131:0/1495779814 reap_dead start 2024-05-26T22:48:29.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.147+0000 7ff2d0600640 1 -- 172.21.15.131:0/1508089239 wait complete. 2024-05-26T22:48:29.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.148+0000 7f347bc00640 1 -- 172.21.15.131:0/1495779814 shutdown_connections 2024-05-26T22:48:29.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.148+0000 7f347bc00640 1 -- 172.21.15.131:0/1495779814 >> 172.21.15.131:0/1495779814 conn(0x7f347406b940 msgr2=0x7f34740b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:29.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.148+0000 7f347bc00640 1 -- 172.21.15.131:0/1495779814 shutdown_connections 2024-05-26T22:48:29.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.148+0000 7f347bc00640 1 -- 172.21.15.131:0/1495779814 wait complete. 2024-05-26T22:48:29.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.148+0000 7f9b4ac00640 1 -- 172.21.15.131:0/3570270680 shutdown_connections 2024-05-26T22:48:29.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.148+0000 7fa4d7e00640 1 --2- 172.21.15.131:0/2353025357 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa4d80b9f00 0x7fa4d80b5c50 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa4c4009870 tx=0x7fa4c402a300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c3d4d1ee831835b server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:29.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.148+0000 7f9b4ac00640 1 -- 172.21.15.131:0/3570270680 wait complete. 2024-05-26T22:48:29.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.150+0000 7ff2d0600640 1 Processor -- start 2024-05-26T22:48:29.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.150+0000 7ff2d0600640 1 -- start start 2024-05-26T22:48:29.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.150+0000 7f9b4ac00640 1 Processor -- start 2024-05-26T22:48:29.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.151+0000 7f347bc00640 1 Processor -- start 2024-05-26T22:48:29.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.151+0000 7f2167400640 1 -- 172.21.15.131:0/1409083103 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f216801a070 con 0x7f21700d03d0 2024-05-26T22:48:29.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.151+0000 7fd90cc00640 1 -- 172.21.15.131:0/1015536175 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fd8fc058400 con 0x7fd9100b9ce0 2024-05-26T22:48:29.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.151+0000 7f9b4ac00640 1 -- start start 2024-05-26T22:48:29.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.152+0000 7f9b4ac00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9b440b77d0 0x7f9b440b53a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:29.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.152+0000 7f9b4ac00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b440d1470 0x7f9b44160220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:29.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.152+0000 7f347bc00640 1 -- start start 2024-05-26T22:48:29.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.152+0000 7f9b4ac00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9b4415e890 0x7f9b44071c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:29.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.152+0000 7f9b4ac00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f9b440d2cc0 con 0x7f9b440d1470 2024-05-26T22:48:29.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.152+0000 7f9b4ac00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9b440d2b40 con 0x7f9b4415e890 2024-05-26T22:48:29.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.152+0000 7f9b4ac00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9b440d2e40 con 0x7f9b440b77d0 2024-05-26T22:48:29.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.152+0000 7f9b43e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9b440b77d0 0x7f9b440b53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:29.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.152+0000 7f9b43400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b440d1470 0x7f9b44160220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:29.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.153+0000 7f9b43e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9b440b77d0 0x7f9b440b53a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.131:43346/0 (socket says 172.21.15.131:43346) 2024-05-26T22:48:29.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.153+0000 7f9b43e00640 1 -- 172.21.15.131:0/896824633 learned_addr learned my addr 172.21.15.131:0/896824633 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:29.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.153+0000 7f2167400640 1 --2- 172.21.15.131:0/1409083103 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f215006c030 0x7f215006e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:29.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.153+0000 7f2167400640 1 -- 172.21.15.131:0/1409083103 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f216808ac30 con 0x7f21700d03d0 2024-05-26T22:48:29.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.153+0000 7f9b48c00640 1 --2- 172.21.15.131:0/896824633 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9b4415e890 0x7f9b44071c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:29.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.153+0000 7f347bc00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f345c0051c0 0x7f3474150b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:29.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.153+0000 7f347bc00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3474151080 0x7f347409aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:29.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.153+0000 7f347bc00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f347409af40 0x7f34740b2c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:29.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.153+0000 7f347bc00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f3474073130 con 0x7f347409af40 2024-05-26T22:48:29.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.153+0000 7f347bc00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f347406ef60 con 0x7f345c0051c0 2024-05-26T22:48:29.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.153+0000 7f347bc00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f34740732b0 con 0x7f3474151080 2024-05-26T22:48:29.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.153+0000 7f9b43400640 1 -- 172.21.15.131:0/896824633 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9b440b77d0 msgr2=0x7f9b440b53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:29.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.153+0000 7f9b43400640 1 --2- 172.21.15.131:0/896824633 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9b440b77d0 0x7f9b440b53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:29.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.153+0000 7f9b43400640 1 -- 172.21.15.131:0/896824633 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9b4415e890 msgr2=0x7f9b44071c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:29.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.153+0000 7f9b43400640 1 --2- 172.21.15.131:0/896824633 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9b4415e890 0x7f9b44071c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:29.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.153+0000 7f9b43400640 1 -- 172.21.15.131:0/896824633 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b4415f930 con 0x7f9b440d1470 2024-05-26T22:48:29.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.154+0000 7ff2d0600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff2c80cf720 0x7ff2c81f6da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:29.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.154+0000 7f3479c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f347409af40 0x7f34740b2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:29.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.154+0000 7f3479c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f347409af40 0x7f34740b2c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54204/0 (socket says 172.21.15.131:54204) 2024-05-26T22:48:29.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.154+0000 7f3479c00640 1 -- 172.21.15.131:0/1576776246 learned_addr learned my addr 172.21.15.131:0/1576776246 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:29.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.154+0000 7ff2d0600640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff2c80d2330 0x7ff2c80b4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:29.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.154+0000 7f3479200640 1 --2- 172.21.15.131:0/1576776246 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f345c0051c0 0x7f3474150b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:29.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.154+0000 7f3473e00640 1 --2- 172.21.15.131:0/1576776246 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3474151080 0x7f347409aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:29.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.154+0000 7ff2d0600640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff2c815e4e0 0x7ff2c80b5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:29.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.154+0000 7ff2cdc00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff2c80cf720 0x7ff2c81f6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:29.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.154+0000 7ff2d0600640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff2c80d3990 con 0x7ff2c80cf720 2024-05-26T22:48:29.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.154+0000 7ff2cdc00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff2c80cf720 0x7ff2c81f6da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54212/0 (socket says 172.21.15.131:54212) 2024-05-26T22:48:29.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.154+0000 7ff2cdc00640 1 -- 172.21.15.131:0/1144629340 learned_addr learned my addr 172.21.15.131:0/1144629340 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:29.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.154+0000 7ff2d0600640 1 -- 172.21.15.131:0/1144629340 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff2c815e8c0 con 0x7ff2c815e4e0 2024-05-26T22:48:29.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.155+0000 7ff2d0600640 1 -- 172.21.15.131:0/1144629340 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff2c80ba1c0 con 0x7ff2c80d2330 2024-05-26T22:48:29.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.155+0000 7f216ea00640 1 --2- 172.21.15.131:0/1409083103 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f215006c030 0x7f215006e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:29.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.154+0000 7f9b43400640 1 --2- 172.21.15.131:0/896824633 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b440d1470 0x7f9b44160220 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9b3400c750 tx=0x7f9b3400cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:29.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.155+0000 7f3479c00640 1 -- 172.21.15.131:0/1576776246 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3474151080 msgr2=0x7f347409aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:29.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.155+0000 7f3479c00640 1 --2- 172.21.15.131:0/1576776246 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3474151080 0x7f347409aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:29.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.155+0000 7f3479c00640 1 -- 172.21.15.131:0/1576776246 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f345c0051c0 msgr2=0x7f3474150b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:29.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.155+0000 7f3479c00640 1 --2- 172.21.15.131:0/1576776246 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f345c0051c0 0x7f3474150b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:29.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.155+0000 7f3479c00640 1 -- 172.21.15.131:0/1576776246 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34740b3880 con 0x7f347409af40 2024-05-26T22:48:29.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.155+0000 7f3479c00640 1 --2- 172.21.15.131:0/1576776246 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f347409af40 0x7f34740b2c00 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f346c02ae60 tx=0x7f346c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:29.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.155+0000 7f9b3be00640 1 -- 172.21.15.131:0/896824633 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b340050d0 con 0x7f9b440d1470 2024-05-26T22:48:29.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.155+0000 7f9b3be00640 1 -- 172.21.15.131:0/896824633 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b34005270 con 0x7f9b440d1470 2024-05-26T22:48:29.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.155+0000 7f9b3be00640 1 -- 172.21.15.131:0/896824633 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b3400a640 con 0x7f9b440d1470 2024-05-26T22:48:29.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.155+0000 7f9b4ac00640 1 -- 172.21.15.131:0/896824633 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b4415fc10 con 0x7f9b440d1470 2024-05-26T22:48:29.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.156+0000 7f216ea00640 1 --2- 172.21.15.131:0/1409083103 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f215006c030 0x7f215006e4f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f215c0011e0 tx=0x7f215c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:29.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.156+0000 7f3470c00640 1 -- 172.21.15.131:0/1576776246 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f346c036040 con 0x7f347409af40 2024-05-26T22:48:29.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.156+0000 7f347bc00640 1 -- 172.21.15.131:0/1576776246 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34740b3b60 con 0x7f347409af40 2024-05-26T22:48:29.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.156+0000 7f347bc00640 1 -- 172.21.15.131:0/1576776246 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34740b4120 con 0x7f347409af40 2024-05-26T22:48:29.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.155+0000 7ff2cd200640 1 --2- 172.21.15.131:0/1144629340 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff2c80d2330 0x7ff2c80b4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:29.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.155+0000 7ff2ce600640 1 --2- 172.21.15.131:0/1144629340 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff2c815e4e0 0x7ff2c80b5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:29.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.156+0000 7ff2cdc00640 1 -- 172.21.15.131:0/1144629340 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff2c80d2330 msgr2=0x7ff2c80b4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:29.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.156+0000 7ff2cdc00640 1 --2- 172.21.15.131:0/1144629340 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff2c80d2330 0x7ff2c80b4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:29.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.156+0000 7ff2cdc00640 1 -- 172.21.15.131:0/1144629340 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff2c815e4e0 msgr2=0x7ff2c80b5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:29.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.156+0000 7ff2cdc00640 1 --2- 172.21.15.131:0/1144629340 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff2c815e4e0 0x7ff2c80b5100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:29.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.156+0000 7ff2cdc00640 1 -- 172.21.15.131:0/1144629340 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2c815ebe0 con 0x7ff2c80cf720 2024-05-26T22:48:29.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.156+0000 7ff2ce600640 1 --2- 172.21.15.131:0/1144629340 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff2c815e4e0 0x7ff2c80b5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-26T22:48:29.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.157+0000 7ff2cdc00640 1 --2- 172.21.15.131:0/1144629340 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff2c80cf720 0x7ff2c81f6da0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff2b4000c00 tx=0x7ff2b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:29.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.156+0000 7f3470c00640 1 -- 172.21.15.131:0/1576776246 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f346c042070 con 0x7f347409af40 2024-05-26T22:48:29.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.156+0000 7f3470c00640 1 -- 172.21.15.131:0/1576776246 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f346c009bd0 con 0x7f347409af40 2024-05-26T22:48:29.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.156+0000 7f9b4ac00640 1 -- 172.21.15.131:0/896824633 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b44071400 con 0x7f9b440d1470 2024-05-26T22:48:29.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.157+0000 7ff2c5600640 1 -- 172.21.15.131:0/1144629340 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b4037040 con 0x7ff2c80cf720 2024-05-26T22:48:29.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.157+0000 7ff2c5600640 1 -- 172.21.15.131:0/1144629340 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2b4042070 con 0x7ff2c80cf720 2024-05-26T22:48:29.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.157+0000 7ff2c5600640 1 -- 172.21.15.131:0/1144629340 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b4030860 con 0x7ff2c80cf720 2024-05-26T22:48:29.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.158+0000 7ff2d0600640 1 -- 172.21.15.131:0/1144629340 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2c815eec0 con 0x7ff2c80cf720 2024-05-26T22:48:29.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.158+0000 7ff2d0600640 1 -- 172.21.15.131:0/1144629340 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2c815f480 con 0x7ff2c80cf720 2024-05-26T22:48:29.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.158+0000 7f3466000640 1 -- 172.21.15.131:0/1576776246 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3448005290 con 0x7f347409af40 2024-05-26T22:48:29.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.159+0000 7ff2d0600640 1 -- 172.21.15.131:0/1144629340 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2c80695c0 con 0x7ff2c80cf720 2024-05-26T22:48:29.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.159+0000 7f9b39600640 1 -- 172.21.15.131:0/896824633 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b18005290 con 0x7f9b440d1470 2024-05-26T22:48:29.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.161+0000 7f9b3be00640 1 -- 172.21.15.131:0/896824633 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9b3400a800 con 0x7f9b440d1470 2024-05-26T22:48:29.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.162+0000 7f9b3be00640 1 --2- 172.21.15.131:0/896824633 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9b2406be30 0x7f9b2406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:29.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.162+0000 7f9b3be00640 1 -- 172.21.15.131:0/896824633 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f9b3408a930 con 0x7f9b440d1470 2024-05-26T22:48:29.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.162+0000 7f9b43e00640 1 --2- 172.21.15.131:0/896824633 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9b2406be30 0x7f9b2406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:29.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.162+0000 7f3470c00640 1 -- 172.21.15.131:0/1576776246 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f346c005d10 con 0x7f347409af40 2024-05-26T22:48:29.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.163+0000 7f3470c00640 1 --2- 172.21.15.131:0/1576776246 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f345406be60 0x7f345406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:29.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.163+0000 7f9b43e00640 1 --2- 172.21.15.131:0/896824633 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9b2406be30 0x7f9b2406e2f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c00e250 tx=0x7f9b3c00b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:29.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.163+0000 7f3470c00640 1 -- 172.21.15.131:0/1576776246 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f346c0aaef0 con 0x7f347409af40 2024-05-26T22:48:29.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.163+0000 7f3479200640 1 --2- 172.21.15.131:0/1576776246 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f345406be60 0x7f345406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:29.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.163+0000 7ff2c5600640 1 -- 172.21.15.131:0/1144629340 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff2b4030a00 con 0x7ff2c80cf720 2024-05-26T22:48:29.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.164+0000 7f3479200640 1 --2- 172.21.15.131:0/1576776246 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f345406be60 0x7f345406e320 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3468001200 tx=0x7f3468009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:29.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.164+0000 7ff2c5600640 1 --2- 172.21.15.131:0/1144629340 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ff2a806be30 0x7ff2a806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:29.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.164+0000 7ff2c5600640 1 -- 172.21.15.131:0/1144629340 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff2b40ac130 con 0x7ff2c80cf720 2024-05-26T22:48:29.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.165+0000 7ff2cd200640 1 --2- 172.21.15.131:0/1144629340 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ff2a806be30 0x7ff2a806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:29.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.165+0000 7f2f95600640 1 -- 172.21.15.131:0/1066447711 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f88019070 con 0x7f2f7c004800 2024-05-26T22:48:29.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.165+0000 7fa4d7400640 1 -- 172.21.15.131:0/2353025357 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4c4037070 con 0x7fa4d80b9f00 2024-05-26T22:48:29.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.165+0000 7fa4d7400640 1 -- 172.21.15.131:0/2353025357 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4c402ae50 con 0x7fa4d80b9f00 2024-05-26T22:48:29.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.165+0000 7fa4d7400640 1 -- 172.21.15.131:0/2353025357 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4c4032860 con 0x7fa4d80b9f00 2024-05-26T22:48:29.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.165+0000 7ff2cd200640 1 --2- 172.21.15.131:0/1144629340 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ff2a806be30 0x7ff2a806e2f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff2c80cb4c0 tx=0x7ff2bc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:29.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.165+0000 7f2f9f600640 1 -- 172.21.15.131:0/1066447711 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f981e7fa0 con 0x7f2f7c004800 2024-05-26T22:48:29.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.166+0000 7f2f9f600640 1 -- 172.21.15.131:0/1066447711 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f981e8560 con 0x7f2f7c004800 2024-05-26T22:48:29.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.166+0000 7f2167400640 1 -- 172.21.15.131:0/1409083103 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f2168059070 con 0x7f21700d03d0 2024-05-26T22:48:29.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.166+0000 7f2f95600640 1 -- 172.21.15.131:0/1066447711 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f88003990 con 0x7f2f7c004800 2024-05-26T22:48:29.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.166+0000 7fa4dde00640 1 -- 172.21.15.131:0/2353025357 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa4d80b9f00 msgr2=0x7fa4d80b5c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:29.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.166+0000 7fa4dde00640 1 --2- 172.21.15.131:0/2353025357 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa4d80b9f00 0x7fa4d80b5c50 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa4c4009870 tx=0x7fa4c402a300 comp rx=0 tx=0).stop 2024-05-26T22:48:29.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.166+0000 7f2f95600640 1 -- 172.21.15.131:0/1066447711 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f88003d60 con 0x7f2f7c004800 2024-05-26T22:48:29.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.167+0000 7fa4dde00640 1 -- 172.21.15.131:0/2353025357 shutdown_connections 2024-05-26T22:48:29.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.167+0000 7fa4dde00640 1 --2- 172.21.15.131:0/2353025357 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa4d80b9f00 0x7fa4d80b5c50 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:29.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.167+0000 7fa4dde00640 1 -- 172.21.15.131:0/2353025357 >> 172.21.15.131:0/2353025357 conn(0x7fa4d8073070 msgr2=0x7fa4d80d0c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:29.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.167+0000 7fa4dde00640 1 -- 172.21.15.131:0/2353025357 shutdown_connections 2024-05-26T22:48:29.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.168+0000 7fa4dde00640 1 -- 172.21.15.131:0/2353025357 wait complete. 2024-05-26T22:48:29.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.168+0000 7f2f9f600640 1 -- 172.21.15.131:0/1066447711 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f70005290 con 0x7f2f7c004800 2024-05-26T22:48:29.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.169+0000 7fa4dde00640 1 Processor -- start 2024-05-26T22:48:29.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.170+0000 7fa4dde00640 1 -- start start 2024-05-26T22:48:29.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.170+0000 7ff2c5600640 1 -- 172.21.15.131:0/1144629340 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7ff2b403c380 con 0x7ff2c80cf720 2024-05-26T22:48:29.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.171+0000 7fa4dde00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa4d80b9f00 0x7fa4d806e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:29.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.170+0000 7f2f95600640 1 -- 172.21.15.131:0/1066447711 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2f8801c070 con 0x7f2f7c004800 2024-05-26T22:48:29.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.171+0000 7f2f95600640 1 --2- 172.21.15.131:0/1066447711 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f2f7406be30 0x7f2f7406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:29.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.171+0000 7f2f95600640 1 -- 172.21.15.131:0/1066447711 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f2f8808bf10 con 0x7f2f7c004800 2024-05-26T22:48:29.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.171+0000 7fa4dde00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa4d80d3c40 0x7fa4d815cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:29.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.171+0000 7fa4dde00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4d815e3e0 0x7fa4d815f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:29.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.171+0000 7fa4dde00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa4d815e7c0 con 0x7fa4d815e3e0 2024-05-26T22:48:29.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.171+0000 7fa4dde00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa4d80d2c30 con 0x7fa4d80d3c40 2024-05-26T22:48:29.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.171+0000 7fa4dde00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa4d80b6190 con 0x7fa4d80b9f00 2024-05-26T22:48:29.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.172+0000 7fa4d7e00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa4d80d3c40 0x7fa4d815cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:29.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.172+0000 7f2f8fe00640 1 --2- 172.21.15.131:0/1066447711 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f2f7406be30 0x7f2f7406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:29.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.172+0000 7fa4d7e00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa4d80d3c40 0x7fa4d815cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.131:36604/0 (socket says 172.21.15.131:36604) 2024-05-26T22:48:29.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.172+0000 7fa4d7e00640 1 -- 172.21.15.131:0/1020991371 learned_addr learned my addr 172.21.15.131:0/1020991371 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:29.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.172+0000 7fa4dca00640 1 --2- 172.21.15.131:0/1020991371 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa4d80b9f00 0x7fa4d806e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:29.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.173+0000 7fa4d7e00640 1 -- 172.21.15.131:0/1020991371 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa4d80b9f00 msgr2=0x7fa4d806e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:29.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.173+0000 7f2f8fe00640 1 --2- 172.21.15.131:0/1066447711 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f2f7406be30 0x7f2f7406e2f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2f80033000 tx=0x7f2f8002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:29.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.173+0000 7fa4d7e00640 1 --2- 172.21.15.131:0/1020991371 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa4d80b9f00 0x7fa4d806e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:29.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.173+0000 7fa4d7e00640 1 -- 172.21.15.131:0/1020991371 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4d815e3e0 msgr2=0x7fa4d815f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:29.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.173+0000 7fa4d7e00640 1 --2- 172.21.15.131:0/1020991371 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4d815e3e0 0x7fa4d815f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:29.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.173+0000 7fa4d7e00640 1 -- 172.21.15.131:0/1020991371 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4d80cbae0 con 0x7fa4d80d3c40 2024-05-26T22:48:29.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.173+0000 7fa4d7e00640 1 --2- 172.21.15.131:0/1020991371 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa4d80d3c40 0x7fa4d815cc20 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa4c4000c00 tx=0x7fa4c40046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:29.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.181+0000 7f9b3be00640 1 -- 172.21.15.131:0/896824633 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f9b34058bf0 con 0x7f9b440d1470 2024-05-26T22:48:29.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.182+0000 7fa4d4c00640 1 -- 172.21.15.131:0/1020991371 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4c4041070 con 0x7fa4d80d3c40 2024-05-26T22:48:29.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.182+0000 7fa4d4c00640 1 -- 172.21.15.131:0/1020991371 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4c403c890 con 0x7fa4d80d3c40 2024-05-26T22:48:29.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.182+0000 7fa4d4c00640 1 -- 172.21.15.131:0/1020991371 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4c4037040 con 0x7fa4d80d3c40 2024-05-26T22:48:29.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.182+0000 7fa4dde00640 1 -- 172.21.15.131:0/1020991371 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4d80cbdc0 con 0x7fa4d80d3c40 2024-05-26T22:48:29.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.183+0000 7fa4dde00640 1 -- 172.21.15.131:0/1020991371 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4d80cc380 con 0x7fa4d80d3c40 2024-05-26T22:48:29.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.184+0000 7f2f95600640 1 -- 172.21.15.131:0/1066447711 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f2f88010040 con 0x7f2f7c004800 2024-05-26T22:48:29.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.185+0000 7fa4dde00640 1 -- 172.21.15.131:0/1020991371 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4d80694e0 con 0x7fa4d80d3c40 2024-05-26T22:48:29.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.187+0000 7fa4d4c00640 1 -- 172.21.15.131:0/1020991371 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa4c4044070 con 0x7fa4d80d3c40 2024-05-26T22:48:29.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.188+0000 7fa4d4c00640 1 --2- 172.21.15.131:0/1020991371 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa4b806beb0 0x7fa4b806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:29.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.189+0000 7fa4d4c00640 1 -- 172.21.15.131:0/1020991371 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa4c4077c60 con 0x7fa4d80d3c40 2024-05-26T22:48:29.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.189+0000 7fa4dca00640 1 --2- 172.21.15.131:0/1020991371 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa4b806beb0 0x7fa4b806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:29.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.189+0000 7f3470c00640 1 -- 172.21.15.131:0/1576776246 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f346c079120 con 0x7f347409af40 2024-05-26T22:48:29.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.190+0000 7fa4dca00640 1 --2- 172.21.15.131:0/1020991371 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa4b806beb0 0x7fa4b806e370 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa4d80718a0 tx=0x7fa4cc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:29.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.203+0000 7fa4d4c00640 1 -- 172.21.15.131:0/1020991371 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fa4c4079bc0 con 0x7fa4d80d3c40 2024-05-26T22:48:29.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.556+0000 7f6021600640 1 -- 172.21.15.131:0/3303070870 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f6004005850 con 0x7f60280e3f70 2024-05-26T22:48:29.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.557+0000 7f6023e00640 1 -- 172.21.15.131:0/3303070870 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f601c07a840 con 0x7f60280e3f70 2024-05-26T22:48:29.557 INFO:teuthology.orchestra.run.smithi131.stdout:55834574853 2024-05-26T22:48:29.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.560+0000 7f6021600640 1 -- 172.21.15.131:0/3303070870 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f601006bf00 msgr2=0x7f601006e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:29.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.561+0000 7f6021600640 1 --2- 172.21.15.131:0/3303070870 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f601006bf00 0x7f601006e3c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6024000f60 tx=0x7f6024009040 comp rx=0 tx=0).stop 2024-05-26T22:48:29.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.561+0000 7f6021600640 1 -- 172.21.15.131:0/3303070870 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f60280e3f70 msgr2=0x7f602804aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:29.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.561+0000 7f6021600640 1 --2- 172.21.15.131:0/3303070870 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f60280e3f70 0x7f602804aa20 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f601c009870 tx=0x7f601c0046c0 comp rx=0 tx=0).stop 2024-05-26T22:48:29.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.562+0000 7f6021600640 1 -- 172.21.15.131:0/3303070870 shutdown_connections 2024-05-26T22:48:29.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.562+0000 7f6021600640 1 --2- 172.21.15.131:0/3303070870 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f601006bf00 0x7f601006e3c0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:29.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.562+0000 7f6021600640 1 --2- 172.21.15.131:0/3303070870 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f60280e59b0 0x7f602817add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:29.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.562+0000 7f6021600640 1 --2- 172.21.15.131:0/3303070870 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f60280e3f70 0x7f602804aa20 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:29.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.562+0000 7f6021600640 1 --2- 172.21.15.131:0/3303070870 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f602800a520 0x7f602817b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:29.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.562+0000 7f6021600640 1 -- 172.21.15.131:0/3303070870 >> 172.21.15.131:0/3303070870 conn(0x7f60280537b0 msgr2=0x7f60280e1180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:29.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.562+0000 7f6021600640 1 -- 172.21.15.131:0/3303070870 shutdown_connections 2024-05-26T22:48:29.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.562+0000 7f6021600640 1 -- 172.21.15.131:0/3303070870 wait complete. 2024-05-26T22:48:29.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.586+0000 7fd916200640 1 -- 172.21.15.131:0/1015536175 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fd9101f41c0 con 0x7fd9100b9ce0 2024-05-26T22:48:29.588 INFO:teuthology.orchestra.run.smithi131.stdout:55834574853 2024-05-26T22:48:29.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.587+0000 7fd90cc00640 1 -- 172.21.15.131:0/1015536175 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd8fc05c700 con 0x7fd9100b9ce0 2024-05-26T22:48:29.589 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.2 2024-05-26T22:48:29.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.589+0000 7f2176600640 1 -- 172.21.15.131:0/1409083103 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f21700d25f0 con 0x7f21700d03d0 2024-05-26T22:48:29.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.590+0000 7f2167400640 1 -- 172.21.15.131:0/1409083103 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f216805d370 con 0x7f21700d03d0 2024-05-26T22:48:29.590 INFO:teuthology.orchestra.run.smithi131.stdout:55834574853 2024-05-26T22:48:29.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.591+0000 7fd916200640 1 -- 172.21.15.131:0/1015536175 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fd8f006bf00 msgr2=0x7fd8f006e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:29.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.591+0000 7fd916200640 1 --2- 172.21.15.131:0/1015536175 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fd8f006bf00 0x7fd8f006e3c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd904009bb0 tx=0x7fd904008040 comp rx=0 tx=0).stop 2024-05-26T22:48:29.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.591+0000 7fd916200640 1 -- 172.21.15.131:0/1015536175 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd9100b9ce0 msgr2=0x7fd91015f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:29.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.591+0000 7fd916200640 1 --2- 172.21.15.131:0/1015536175 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd9100b9ce0 0x7fd91015f8d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc00c3c0 tx=0x7fd8fc00c880 comp rx=0 tx=0).stop 2024-05-26T22:48:29.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.592+0000 7fd916200640 1 -- 172.21.15.131:0/1015536175 shutdown_connections 2024-05-26T22:48:29.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.592+0000 7fd916200640 1 --2- 172.21.15.131:0/1015536175 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd9100d1190 0x7fd9100cdbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:29.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.592+0000 7fd916200640 1 --2- 172.21.15.131:0/1015536175 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fd8f006bf00 0x7fd8f006e3c0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:29.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.592+0000 7fd916200640 1 --2- 172.21.15.131:0/1015536175 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd9100b9ce0 0x7fd91015f8d0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:29.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.592+0000 7fd916200640 1 --2- 172.21.15.131:0/1015536175 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd9100b7330 0x7fd9101f3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:29.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.592+0000 7fd916200640 1 -- 172.21.15.131:0/1015536175 >> 172.21.15.131:0/1015536175 conn(0x7fd910072e60 msgr2=0x7fd9100b7710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:29.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.592+0000 7fd916200640 1 -- 172.21.15.131:0/1015536175 shutdown_connections 2024-05-26T22:48:29.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.592+0000 7fd916200640 1 -- 172.21.15.131:0/1015536175 wait complete. 2024-05-26T22:48:29.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.593+0000 7f2176600640 1 -- 172.21.15.131:0/1409083103 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f215006c030 msgr2=0x7f215006e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:29.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.593+0000 7f2176600640 1 --2- 172.21.15.131:0/1409083103 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f215006c030 0x7f215006e4f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f215c0011e0 tx=0x7f215c009040 comp rx=0 tx=0).stop 2024-05-26T22:48:29.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.593+0000 7f2176600640 1 -- 172.21.15.131:0/1409083103 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f21700d03d0 msgr2=0x7f21700ca520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:29.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.593+0000 7f2176600640 1 --2- 172.21.15.131:0/1409083103 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f21700d03d0 0x7f21700ca520 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f216800cbc0 tx=0x7f216800a430 comp rx=0 tx=0).stop 2024-05-26T22:48:29.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.594+0000 7f2176600640 1 -- 172.21.15.131:0/1409083103 shutdown_connections 2024-05-26T22:48:29.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.594+0000 7f2176600640 1 --2- 172.21.15.131:0/1409083103 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f217015e870 0x7f21701e8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:29.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.594+0000 7f2176600640 1 --2- 172.21.15.131:0/1409083103 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f215006c030 0x7f215006e4f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:29.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.594+0000 7f2176600640 1 --2- 172.21.15.131:0/1409083103 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f21700d2810 0x7f2170160bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:29.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.594+0000 7f2176600640 1 --2- 172.21.15.131:0/1409083103 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f21700d03d0 0x7f21700ca520 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:29.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.594+0000 7f2176600640 1 -- 172.21.15.131:0/1409083103 >> 172.21.15.131:0/1409083103 conn(0x7f217009d5a0 msgr2=0x7f21700b2550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:29.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.594+0000 7f2176600640 1 -- 172.21.15.131:0/1409083103 shutdown_connections 2024-05-26T22:48:29.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.594+0000 7f2176600640 1 -- 172.21.15.131:0/1409083103 wait complete. 2024-05-26T22:48:29.597 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.598+0000 7ff2d0600640 1 -- 172.21.15.131:0/1144629340 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7ff2c80b9d60 con 0x7ff2c80cf720 2024-05-26T22:48:29.598 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.599+0000 7ff2c5600640 1 -- 172.21.15.131:0/1144629340 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff2b407ab50 con 0x7ff2c80cf720 2024-05-26T22:48:29.601 INFO:teuthology.orchestra.run.smithi131.stdout:55834574853 2024-05-26T22:48:29.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.601+0000 7f2f9f600640 1 -- 172.21.15.131:0/1066447711 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f2f70005850 con 0x7f2f7c004800 2024-05-26T22:48:29.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.602+0000 7f2f95600640 1 -- 172.21.15.131:0/1066447711 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f2f8805a250 con 0x7f2f7c004800 2024-05-26T22:48:29.604 INFO:teuthology.orchestra.run.smithi131.stdout:55834574853 2024-05-26T22:48:29.604 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.603+0000 7ff2d0600640 1 -- 172.21.15.131:0/1144629340 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ff2a806be30 msgr2=0x7ff2a806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:29.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.603+0000 7ff2d0600640 1 --2- 172.21.15.131:0/1144629340 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ff2a806be30 0x7ff2a806e2f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff2c80cb4c0 tx=0x7ff2bc009040 comp rx=0 tx=0).stop 2024-05-26T22:48:29.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.603+0000 7ff2d0600640 1 -- 172.21.15.131:0/1144629340 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff2c80cf720 msgr2=0x7ff2c81f6da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:29.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.603+0000 7ff2d0600640 1 --2- 172.21.15.131:0/1144629340 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff2c80cf720 0x7ff2c81f6da0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff2b4000c00 tx=0x7ff2b40046c0 comp rx=0 tx=0).stop 2024-05-26T22:48:29.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.603+0000 7ff2d0600640 1 -- 172.21.15.131:0/1144629340 shutdown_connections 2024-05-26T22:48:29.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.603+0000 7ff2d0600640 1 --2- 172.21.15.131:0/1144629340 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff2c80d2330 0x7ff2c80b4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:29.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.603+0000 7ff2d0600640 1 --2- 172.21.15.131:0/1144629340 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ff2a806be30 0x7ff2a806e2f0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:29.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.603+0000 7ff2d0600640 1 --2- 172.21.15.131:0/1144629340 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff2c815e4e0 0x7ff2c80b5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:29.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.603+0000 7ff2d0600640 1 --2- 172.21.15.131:0/1144629340 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff2c80cf720 0x7ff2c81f6da0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:29.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.603+0000 7ff2d0600640 1 -- 172.21.15.131:0/1144629340 >> 172.21.15.131:0/1144629340 conn(0x7ff2c8072e90 msgr2=0x7ff2c81596d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:29.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.604+0000 7ff2d0600640 1 -- 172.21.15.131:0/1144629340 shutdown_connections 2024-05-26T22:48:29.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.604+0000 7ff2d0600640 1 -- 172.21.15.131:0/1144629340 wait complete. 2024-05-26T22:48:29.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.610+0000 7f2f9f600640 1 -- 172.21.15.131:0/1066447711 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f2f7406be30 msgr2=0x7f2f7406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:29.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.610+0000 7f2f9f600640 1 --2- 172.21.15.131:0/1066447711 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f2f7406be30 0x7f2f7406e2f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2f80033000 tx=0x7f2f8002e000 comp rx=0 tx=0).stop 2024-05-26T22:48:29.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.610+0000 7f2f9f600640 1 -- 172.21.15.131:0/1066447711 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f7c004800 msgr2=0x7f2f9814c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:29.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.610+0000 7f2f9f600640 1 --2- 172.21.15.131:0/1066447711 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f7c004800 0x7f2f9814c7e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2f88009f00 tx=0x7f2f8800f580 comp rx=0 tx=0).stop 2024-05-26T22:48:29.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.610+0000 7f2f9f600640 1 -- 172.21.15.131:0/1066447711 shutdown_connections 2024-05-26T22:48:29.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.610+0000 7f2f9f600640 1 --2- 172.21.15.131:0/1066447711 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2f981e9ad0 0x7f2f981e9f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:29.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.610+0000 7f2f9f600640 1 --2- 172.21.15.131:0/1066447711 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f2f7406be30 0x7f2f7406e2f0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:29.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.610+0000 7f2f9f600640 1 --2- 172.21.15.131:0/1066447711 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2f9815bc30 0x7f2f9815c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:29.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.610+0000 7f2f9f600640 1 --2- 172.21.15.131:0/1066447711 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f7c004800 0x7f2f9814c7e0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:29.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.610+0000 7f2f9f600640 1 -- 172.21.15.131:0/1066447711 >> 172.21.15.131:0/1066447711 conn(0x7f2f98073070 msgr2=0x7f2f980c7e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:29.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.610+0000 7f2f9f600640 1 -- 172.21.15.131:0/1066447711 shutdown_connections 2024-05-26T22:48:29.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.610+0000 7f2f9f600640 1 -- 172.21.15.131:0/1066447711 wait complete. 2024-05-26T22:48:29.619 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.1 2024-05-26T22:48:29.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.621+0000 7f9b39600640 1 -- 172.21.15.131:0/896824633 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f9b18005580 con 0x7f9b440d1470 2024-05-26T22:48:29.621 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.4 2024-05-26T22:48:29.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.622+0000 7f9b3be00640 1 -- 172.21.15.131:0/896824633 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f9b3405cef0 con 0x7f9b440d1470 2024-05-26T22:48:29.622 INFO:teuthology.orchestra.run.smithi131.stdout:55834574853 2024-05-26T22:48:29.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.625+0000 7f9b39600640 1 -- 172.21.15.131:0/896824633 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9b2406be30 msgr2=0x7f9b2406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:29.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.625+0000 7f9b39600640 1 --2- 172.21.15.131:0/896824633 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9b2406be30 0x7f9b2406e2f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c00e250 tx=0x7f9b3c00b040 comp rx=0 tx=0).stop 2024-05-26T22:48:29.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.625+0000 7f9b39600640 1 -- 172.21.15.131:0/896824633 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b440d1470 msgr2=0x7f9b44160220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:29.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.625+0000 7f9b39600640 1 --2- 172.21.15.131:0/896824633 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b440d1470 0x7f9b44160220 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9b3400c750 tx=0x7f9b3400cc10 comp rx=0 tx=0).stop 2024-05-26T22:48:29.625 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.7 2024-05-26T22:48:29.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.625+0000 7f9b39600640 1 -- 172.21.15.131:0/896824633 shutdown_connections 2024-05-26T22:48:29.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.625+0000 7f9b39600640 1 --2- 172.21.15.131:0/896824633 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9b2406be30 0x7f9b2406e2f0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:29.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.625+0000 7f9b39600640 1 --2- 172.21.15.131:0/896824633 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9b4415e890 0x7f9b44071c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:29.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.625+0000 7f9b39600640 1 --2- 172.21.15.131:0/896824633 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b440d1470 0x7f9b44160220 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:29.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.625+0000 7f9b39600640 1 --2- 172.21.15.131:0/896824633 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9b440b77d0 0x7f9b440b53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:29.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.625+0000 7f9b39600640 1 -- 172.21.15.131:0/896824633 >> 172.21.15.131:0/896824633 conn(0x7f9b44073070 msgr2=0x7f9b440d3110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:29.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.625+0000 7f9b39600640 1 -- 172.21.15.131:0/896824633 shutdown_connections 2024-05-26T22:48:29.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.625+0000 7f9b39600640 1 -- 172.21.15.131:0/896824633 wait complete. 2024-05-26T22:48:29.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.626+0000 7fa4dde00640 1 -- 172.21.15.131:0/1020991371 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fa4d80b4ca0 con 0x7fa4d80d3c40 2024-05-26T22:48:29.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.626+0000 7fa4d4c00640 1 -- 172.21.15.131:0/1020991371 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa4c407dec0 con 0x7fa4d80d3c40 2024-05-26T22:48:29.627 INFO:teuthology.orchestra.run.smithi131.stdout:55834574853 2024-05-26T22:48:29.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.627+0000 7f3466000640 1 -- 172.21.15.131:0/1576776246 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f3448005580 con 0x7f347409af40 2024-05-26T22:48:29.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.628+0000 7f3470c00640 1 -- 172.21.15.131:0/1576776246 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f346c07d420 con 0x7f347409af40 2024-05-26T22:48:29.628 INFO:teuthology.orchestra.run.smithi131.stdout:55834574854 2024-05-26T22:48:29.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.630+0000 7fa4dde00640 1 -- 172.21.15.131:0/1020991371 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa4b806beb0 msgr2=0x7fa4b806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:29.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.630+0000 7fa4dde00640 1 --2- 172.21.15.131:0/1020991371 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa4b806beb0 0x7fa4b806e370 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa4d80718a0 tx=0x7fa4cc009040 comp rx=0 tx=0).stop 2024-05-26T22:48:29.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.630+0000 7fa4dde00640 1 -- 172.21.15.131:0/1020991371 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa4d80d3c40 msgr2=0x7fa4d815cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:29.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.630+0000 7fa4dde00640 1 --2- 172.21.15.131:0/1020991371 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa4d80d3c40 0x7fa4d815cc20 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa4c4000c00 tx=0x7fa4c40046c0 comp rx=0 tx=0).stop 2024-05-26T22:48:29.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.630+0000 7fa4dde00640 1 -- 172.21.15.131:0/1020991371 shutdown_connections 2024-05-26T22:48:29.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.630+0000 7fa4dde00640 1 --2- 172.21.15.131:0/1020991371 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa4b806beb0 0x7fa4b806e370 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:29.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.630+0000 7fa4dde00640 1 --2- 172.21.15.131:0/1020991371 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4d815e3e0 0x7fa4d815f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:29.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.630+0000 7fa4dde00640 1 --2- 172.21.15.131:0/1020991371 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa4d80d3c40 0x7fa4d815cc20 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:29.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.630+0000 7fa4dde00640 1 --2- 172.21.15.131:0/1020991371 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa4d80b9f00 0x7fa4d806e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:29.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.630+0000 7fa4dde00640 1 -- 172.21.15.131:0/1020991371 >> 172.21.15.131:0/1020991371 conn(0x7fa4d8073070 msgr2=0x7fa4d80cea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:29.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.630+0000 7fa4dde00640 1 -- 172.21.15.131:0/1020991371 shutdown_connections 2024-05-26T22:48:29.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.630+0000 7fa4dde00640 1 -- 172.21.15.131:0/1020991371 wait complete. 2024-05-26T22:48:29.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.631+0000 7f3466000640 1 -- 172.21.15.131:0/1576776246 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f345406be60 msgr2=0x7f345406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:29.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.631+0000 7f3466000640 1 --2- 172.21.15.131:0/1576776246 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f345406be60 0x7f345406e320 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3468001200 tx=0x7f3468009040 comp rx=0 tx=0).stop 2024-05-26T22:48:29.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.631+0000 7f3466000640 1 -- 172.21.15.131:0/1576776246 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f347409af40 msgr2=0x7f34740b2c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:29.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.631+0000 7f3466000640 1 --2- 172.21.15.131:0/1576776246 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f347409af40 0x7f34740b2c00 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f346c02ae60 tx=0x7f346c009750 comp rx=0 tx=0).stop 2024-05-26T22:48:29.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.631+0000 7f3466000640 1 -- 172.21.15.131:0/1576776246 shutdown_connections 2024-05-26T22:48:29.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.631+0000 7f3466000640 1 --2- 172.21.15.131:0/1576776246 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3474151080 0x7f347409aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:29.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.631+0000 7f3466000640 1 --2- 172.21.15.131:0/1576776246 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f345406be60 0x7f345406e320 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:29.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.631+0000 7f3466000640 1 --2- 172.21.15.131:0/1576776246 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f347409af40 0x7f34740b2c00 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:29.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.631+0000 7f3466000640 1 --2- 172.21.15.131:0/1576776246 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f345c0051c0 0x7f3474150b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:29.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.631+0000 7f3466000640 1 -- 172.21.15.131:0/1576776246 >> 172.21.15.131:0/1576776246 conn(0x7f347406b940 msgr2=0x7f3474160df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:29.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.632+0000 7f3466000640 1 -- 172.21.15.131:0/1576776246 shutdown_connections 2024-05-26T22:48:29.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:29.632+0000 7f3466000640 1 -- 172.21.15.131:0/1576776246 wait complete. 2024-05-26T22:48:29.633 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.3 2024-05-26T22:48:29.646 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.0 2024-05-26T22:48:29.648 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.6 2024-05-26T22:48:29.650 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.5 2024-05-26T22:48:30.590 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-05-26T22:48:30.621 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-05-26T22:48:30.622 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-05-26T22:48:30.627 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-05-26T22:48:30.633 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-05-26T22:48:30.647 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-05-26T22:48:30.649 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-05-26T22:48:30.650 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-05-26T22:48:30.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.862+0000 7fbd7d000640 1 Processor -- start 2024-05-26T22:48:30.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.862+0000 7fbd7d000640 1 -- start start 2024-05-26T22:48:30.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.864+0000 7fbd7d000640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fbd700ead00 0x7fbd7004fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:30.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.864+0000 7fbd7d000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbd70006ea0 0x7fbd70007330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:30.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.866+0000 7fbd76a00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fbd7004ffd0 0x7fbd70037bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:48934/0 (socket says 172.21.15.131:48934) 2024-05-26T22:48:30.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.866+0000 7fbd77400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fbd700ead00 0x7fbd7004fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:30.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.866+0000 7fbd77400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fbd700ead00 0x7fbd7004fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54232/0 (socket says 172.21.15.131:54232) 2024-05-26T22:48:30.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.866+0000 7fbd76a00640 1 -- 172.21.15.131:0/1247852022 learned_addr learned my addr 172.21.15.131:0/1247852022 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:48:30.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.867+0000 7fbd77e00640 1 --2- 172.21.15.131:0/1247852022 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbd70006ea0 0x7fbd70007330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:30.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.867+0000 7fbd7d000640 1 -- 172.21.15.131:0/1247852022 --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd70053720 con 0x7fbd7004ffd0 2024-05-26T22:48:30.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.868+0000 7fbd7d000640 1 -- 172.21.15.131:0/1247852022 --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fbd70054330 con 0x7fbd700ead00 2024-05-26T22:48:30.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.868+0000 7fbd76000640 1 -- 172.21.15.131:0/1247852022 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 808584476 0 0) 0x7fbd70053720 con 0x7fbd7004ffd0 2024-05-26T22:48:30.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.868+0000 7fbd7d000640 1 -- 172.21.15.131:0/1247852022 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fbd70053a00 con 0x7fbd70006ea0 2024-05-26T22:48:30.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.869+0000 7fbd76000640 1 -- 172.21.15.131:0/1247852022 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbd60003520 con 0x7fbd7004ffd0 2024-05-26T22:48:30.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.869+0000 7fbd76000640 1 -- 172.21.15.131:0/1247852022 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1501248747 0 0) 0x7fbd60003520 con 0x7fbd7004ffd0 2024-05-26T22:48:30.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.870+0000 7fbd76000640 1 -- 172.21.15.131:0/1247852022 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbd70006ea0 msgr2=0x7fbd70007330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:30.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.870+0000 7fbd76000640 1 --2- 172.21.15.131:0/1247852022 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbd70006ea0 0x7fbd70007330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:30.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.870+0000 7fbd76000640 1 -- 172.21.15.131:0/1247852022 >> v2:172.21.15.131:3300/0 conn(0x7fbd700ead00 msgr2=0x7fbd7004fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:30.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.870+0000 7fbd76000640 1 --2- 172.21.15.131:0/1247852022 >> v2:172.21.15.131:3300/0 conn(0x7fbd700ead00 0x7fbd7004fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:30.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.870+0000 7fbd76000640 1 -- 172.21.15.131:0/1247852022 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd70053ce0 con 0x7fbd7004ffd0 2024-05-26T22:48:30.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.870+0000 7fbd76000640 1 -- 172.21.15.131:0/1247852022 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7fbd64001670 con 0x7fbd7004ffd0 2024-05-26T22:48:30.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.870+0000 7fbd76000640 1 -- 172.21.15.131:0/1247852022 >> v1:172.21.15.131:6789/0 conn(0x7fbd7004ffd0 legacy=0x7fbd70037bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:30.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.870+0000 7fbd76000640 1 --2- 172.21.15.131:0/1247852022 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbd70006ea0 0x7fbd600039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:30.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.870+0000 7fbd76000640 1 --2- 172.21.15.131:0/1247852022 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbd60003f30 0x7fbd60004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:30.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.870+0000 7fbd76000640 1 --2- 172.21.15.131:0/1247852022 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbd60004d40 0x7fbd600051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:30.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.871+0000 7fbd77e00640 1 --2- 172.21.15.131:0/1247852022 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbd60004d40 0x7fbd600051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:30.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.872+0000 7fbd76000640 1 -- 172.21.15.131:0/1247852022 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fbd70054330 con 0x7fbd60004d40 2024-05-26T22:48:30.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.872+0000 7fbd76000640 1 -- 172.21.15.131:0/1247852022 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fbd70053a00 con 0x7fbd60003f30 2024-05-26T22:48:30.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.872+0000 7fbd76000640 1 -- 172.21.15.131:0/1247852022 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fbd6000ba40 con 0x7fbd70006ea0 2024-05-26T22:48:30.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.872+0000 7fbd77400640 1 --2- 172.21.15.131:0/1247852022 >> v2:172.21.15.131:3300/0 conn(0x7fbd700ead00 0x7fbd7004fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-26T22:48:30.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.873+0000 7fbd77e00640 1 -- 172.21.15.131:0/1247852022 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbd70006ea0 msgr2=0x7fbd600039b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-26T22:48:30.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.873+0000 7fbd77400640 1 --2- 172.21.15.131:0/1247852022 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbd70006ea0 0x7fbd600039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:30.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.873+0000 7fbd77e00640 1 --2- 172.21.15.131:0/1247852022 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbd70006ea0 0x7fbd600039b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:30.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.873+0000 7fbd77e00640 1 -- 172.21.15.131:0/1247852022 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbd60003f30 msgr2=0x7fbd60004800 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-26T22:48:30.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.873+0000 7fbd77e00640 1 --2- 172.21.15.131:0/1247852022 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbd60003f30 0x7fbd60004800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:30.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.873+0000 7fbd77e00640 1 -- 172.21.15.131:0/1247852022 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd70053ce0 con 0x7fbd60004d40 2024-05-26T22:48:30.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.875+0000 7fbd77e00640 1 --2- 172.21.15.131:0/1247852022 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbd60004d40 0x7fbd600051c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c009aa0 tx=0x7fbd60005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=869fd6b8b516d077 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:30.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.876+0000 7fbd76000640 1 -- 172.21.15.131:0/1247852022 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd6c036070 con 0x7fbd60004d40 2024-05-26T22:48:30.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.876+0000 7fbd76000640 1 -- 172.21.15.131:0/1247852022 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd6c031400 con 0x7fbd60004d40 2024-05-26T22:48:30.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.877+0000 7fbd7d000640 1 -- 172.21.15.131:0/1247852022 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbd60004d40 msgr2=0x7fbd600051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:30.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.877+0000 7fbd76000640 1 -- 172.21.15.131:0/1247852022 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd6c036070 con 0x7fbd60004d40 2024-05-26T22:48:30.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.877+0000 7fbd7d000640 1 --2- 172.21.15.131:0/1247852022 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbd60004d40 0x7fbd600051c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c009aa0 tx=0x7fbd60005830 comp rx=0 tx=0).stop 2024-05-26T22:48:30.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.877+0000 7fbd77e00640 1 -- 172.21.15.131:0/1247852022 reap_dead start 2024-05-26T22:48:30.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.877+0000 7fbd7d000640 1 -- 172.21.15.131:0/1247852022 shutdown_connections 2024-05-26T22:48:30.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.877+0000 7fbd7d000640 1 -- 172.21.15.131:0/1247852022 >> 172.21.15.131:0/1247852022 conn(0x7fbd70009fb0 msgr2=0x7fbd7000a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:30.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.878+0000 7fbd7d000640 1 -- 172.21.15.131:0/1247852022 shutdown_connections 2024-05-26T22:48:30.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.878+0000 7fbd7d000640 1 -- 172.21.15.131:0/1247852022 wait complete. 2024-05-26T22:48:30.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.880+0000 7fbd7d000640 1 Processor -- start 2024-05-26T22:48:30.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.881+0000 7fbd7d000640 1 -- start start 2024-05-26T22:48:30.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.882+0000 7fbd7d000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbd60004d40 0x7fbd7017cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:30.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.882+0000 7fbd7d000640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbd700e8420 0x7fbd70047610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:30.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.882+0000 7fbd7d000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbd7004bb90 0x7fbd70043d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:30.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.882+0000 7fbd7d000640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fbd70054940 con 0x7fbd7004bb90 2024-05-26T22:48:30.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.882+0000 7fbd7d000640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fbd700547c0 con 0x7fbd700e8420 2024-05-26T22:48:30.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.882+0000 7fbd7d000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fbd7004f550 con 0x7fbd60004d40 2024-05-26T22:48:30.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.884+0000 7fbd76a00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbd700e8420 0x7fbd70047610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:30.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.884+0000 7fbd76a00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbd700e8420 0x7fbd70047610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.131:36612/0 (socket says 172.21.15.131:36612) 2024-05-26T22:48:30.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.884+0000 7fbd77400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbd60004d40 0x7fbd7017cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:30.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.884+0000 7fbd77e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbd7004bb90 0x7fbd70043d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:30.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.884+0000 7fbd76a00640 1 -- 172.21.15.131:0/3390834752 learned_addr learned my addr 172.21.15.131:0/3390834752 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:30.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.886+0000 7fbd76a00640 1 -- 172.21.15.131:0/3390834752 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbd60004d40 msgr2=0x7fbd7017cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:30.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.886+0000 7fbd76a00640 1 --2- 172.21.15.131:0/3390834752 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbd60004d40 0x7fbd7017cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:30.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.886+0000 7fbd76a00640 1 -- 172.21.15.131:0/3390834752 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbd7004bb90 msgr2=0x7fbd70043d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:30.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.886+0000 7fbd76a00640 1 --2- 172.21.15.131:0/3390834752 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbd7004bb90 0x7fbd70043d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:30.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.886+0000 7fbd76a00640 1 -- 172.21.15.131:0/3390834752 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd7017d5f0 con 0x7fbd700e8420 2024-05-26T22:48:30.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.886+0000 7fbd77e00640 1 --2- 172.21.15.131:0/3390834752 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbd7004bb90 0x7fbd70043d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-26T22:48:30.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.887+0000 7fbd77400640 1 --2- 172.21.15.131:0/3390834752 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbd60004d40 0x7fbd7017cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-26T22:48:30.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.888+0000 7fbd76a00640 1 --2- 172.21.15.131:0/3390834752 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbd700e8420 0x7fbd70047610 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbd6400c8c0 tx=0x7fbd6400cd80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:30.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.888+0000 7fbd6b400640 1 -- 172.21.15.131:0/3390834752 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd64018070 con 0x7fbd700e8420 2024-05-26T22:48:30.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.888+0000 7fbd6b400640 1 -- 172.21.15.131:0/3390834752 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd640074e0 con 0x7fbd700e8420 2024-05-26T22:48:30.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.888+0000 7fbd6b400640 1 -- 172.21.15.131:0/3390834752 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd64004a20 con 0x7fbd700e8420 2024-05-26T22:48:30.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.889+0000 7fbd7d000640 1 -- 172.21.15.131:0/3390834752 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd7017d8d0 con 0x7fbd700e8420 2024-05-26T22:48:30.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.889+0000 7fbd7d000640 1 -- 172.21.15.131:0/3390834752 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd70043820 con 0x7fbd700e8420 2024-05-26T22:48:30.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.893+0000 7fbd68c00640 1 -- 172.21.15.131:0/3390834752 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd4c005290 con 0x7fbd700e8420 2024-05-26T22:48:30.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.896+0000 7fbd6b400640 1 -- 172.21.15.131:0/3390834752 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbd64004bc0 con 0x7fbd700e8420 2024-05-26T22:48:30.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.897+0000 7fbd6b400640 1 --2- 172.21.15.131:0/3390834752 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fbd5806be30 0x7fbd5806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:30.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.898+0000 7fbd6b400640 1 -- 172.21.15.131:0/3390834752 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fbd6408a460 con 0x7fbd700e8420 2024-05-26T22:48:30.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.899+0000 7fbd77400640 1 --2- 172.21.15.131:0/3390834752 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fbd5806be30 0x7fbd5806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:30.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.900+0000 7fbd77400640 1 --2- 172.21.15.131:0/3390834752 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fbd5806be30 0x7fbd5806e2f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fbd7806a130 tx=0x7fbd7808d040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:30.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.901+0000 7f2b55200640 1 Processor -- start 2024-05-26T22:48:30.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.901+0000 7f2b55200640 1 -- start start 2024-05-26T22:48:30.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.903+0000 7f2b55200640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f2b500cf2e0 0x7f2b500cf730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:30.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.903+0000 7f2b55200640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b50158380 con 0x7f2b50157fa0 2024-05-26T22:48:30.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.903+0000 7f2b55200640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b5015e7a0 con 0x7f2b500cc400 2024-05-26T22:48:30.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.903+0000 7f2b55200640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f2b5015b9a0 con 0x7f2b500cf2e0 2024-05-26T22:48:30.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.904+0000 7f2b47e00640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f2b500cc400 0x7f2b5015b290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.131:57710/0 (socket says 172.21.15.131:57710) 2024-05-26T22:48:30.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.904+0000 7f2b47e00640 1 -- 172.21.15.131:0/1270504791 learned_addr learned my addr 172.21.15.131:0/1270504791 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:48:30.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.904+0000 7f2b4fe00640 1 --2- 172.21.15.131:0/1270504791 >> v2:172.21.15.187:3300/0 conn(0x7f2b500cf2e0 0x7f2b500cf730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:30.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.905+0000 7f2b4fe00640 1 -- 172.21.15.131:0/1270504791 >> v1:172.21.15.187:6789/0 conn(0x7f2b500cc400 legacy=0x7f2b5015b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:30.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.905+0000 7f2b4fe00640 1 -- 172.21.15.131:0/1270504791 >> v1:172.21.15.131:6789/0 conn(0x7f2b50157fa0 legacy=0x7f2b5015df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:30.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.905+0000 7f2b4ea00640 1 -- 172.21.15.131:0/1270504791 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 324773440 0 0) 0x7f2b50158380 con 0x7f2b50157fa0 2024-05-26T22:48:30.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.905+0000 7f2b4fe00640 1 -- 172.21.15.131:0/1270504791 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b5015ea80 con 0x7f2b500cf2e0 2024-05-26T22:48:30.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.906+0000 7f8261400640 1 Processor -- start 2024-05-26T22:48:30.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.907+0000 7f2b4fe00640 1 --2- 172.21.15.131:0/1270504791 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2b500cf2e0 0x7f2b500cf730 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2b4800a940 tx=0x7f2b4802a4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1f52c51c68f3760 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:30.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.907+0000 7f8261400640 1 -- start start 2024-05-26T22:48:30.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.909+0000 7f8261400640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f825c0d05d0 0x7f825c06df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:30.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.909+0000 7f8261400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f825c06e450 0x7f825c0cf320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:30.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.909+0000 7f8261400640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f825c15fa00 con 0x7f825c0cf860 2024-05-26T22:48:30.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.909+0000 7f8261400640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f825c0d09b0 con 0x7f825c06e450 2024-05-26T22:48:30.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.909+0000 7f8261400640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f825c15f720 con 0x7f825c0d05d0 2024-05-26T22:48:30.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.909+0000 7f8259e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f825c06e450 0x7f825c0cf320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:30.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.909+0000 7f8259e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f825c06e450 0x7f825c0cf320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54272/0 (socket says 172.21.15.131:54272) 2024-05-26T22:48:30.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.909+0000 7f8259e00640 1 -- 172.21.15.131:0/23670005 learned_addr learned my addr 172.21.15.131:0/23670005 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:30.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.909+0000 7f825b200640 1 --1- 172.21.15.131:0/23670005 >> v1:172.21.15.131:6789/0 conn(0x7f825c0cf860 0x7f825c0729b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:48950/0 (socket says 172.21.15.131:48950) 2024-05-26T22:48:30.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.909+0000 7f825a800640 1 --2- 172.21.15.131:0/23670005 >> v2:172.21.15.187:3300/0 conn(0x7f825c0d05d0 0x7f825c06df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:30.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.910+0000 7f8259e00640 1 -- 172.21.15.131:0/23670005 >> v2:172.21.15.187:3300/0 conn(0x7f825c0d05d0 msgr2=0x7f825c06df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:30.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.910+0000 7f8259e00640 1 --2- 172.21.15.131:0/23670005 >> v2:172.21.15.187:3300/0 conn(0x7f825c0d05d0 0x7f825c06df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:30.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.910+0000 7f8259e00640 1 -- 172.21.15.131:0/23670005 >> v1:172.21.15.131:6789/0 conn(0x7f825c0cf860 legacy=0x7f825c0729b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:30.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.910+0000 7f8259e00640 1 -- 172.21.15.131:0/23670005 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f825c0cfcd0 con 0x7f825c06e450 2024-05-26T22:48:30.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.910+0000 7f2b4ea00640 1 -- 172.21.15.131:0/1270504791 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b4800b040 con 0x7f2b500cf2e0 2024-05-26T22:48:30.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.911+0000 7fbd6b400640 1 -- 172.21.15.131:0/3390834752 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fbd64058720 con 0x7fbd700e8420 2024-05-26T22:48:30.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.910+0000 7f2b4ea00640 1 -- 172.21.15.131:0/1270504791 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b4802f710 con 0x7f2b500cf2e0 2024-05-26T22:48:30.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.910+0000 7f2b4ea00640 1 -- 172.21.15.131:0/1270504791 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b4802fae0 con 0x7f2b500cf2e0 2024-05-26T22:48:30.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.912+0000 7f8259e00640 1 --2- 172.21.15.131:0/23670005 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f825c06e450 0x7f825c0cf320 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f825c06c1d0 tx=0x7f824802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8053509fba593bc7 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:30.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.912+0000 7f2b55200640 1 -- 172.21.15.131:0/1270504791 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2b500cf2e0 msgr2=0x7f2b500cf730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:30.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.912+0000 7f2b55200640 1 --2- 172.21.15.131:0/1270504791 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2b500cf2e0 0x7f2b500cf730 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2b4800a940 tx=0x7f2b4802a4d0 comp rx=0 tx=0).stop 2024-05-26T22:48:30.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.913+0000 7f2b55200640 1 -- 172.21.15.131:0/1270504791 shutdown_connections 2024-05-26T22:48:30.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.913+0000 7f2b55200640 1 --2- 172.21.15.131:0/1270504791 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2b500cf2e0 0x7f2b500cf730 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:30.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.913+0000 7f2b55200640 1 -- 172.21.15.131:0/1270504791 >> 172.21.15.131:0/1270504791 conn(0x7f2b500b1c60 msgr2=0x7f2b500b2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:30.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.913+0000 7f8259400640 1 -- 172.21.15.131:0/23670005 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8248037070 con 0x7f825c06e450 2024-05-26T22:48:30.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.913+0000 7f8259400640 1 -- 172.21.15.131:0/23670005 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8248032440 con 0x7f825c06e450 2024-05-26T22:48:30.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.913+0000 7f8259400640 1 -- 172.21.15.131:0/23670005 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8248032810 con 0x7f825c06e450 2024-05-26T22:48:30.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.913+0000 7f2b55200640 1 -- 172.21.15.131:0/1270504791 shutdown_connections 2024-05-26T22:48:30.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.913+0000 7f2b55200640 1 -- 172.21.15.131:0/1270504791 wait complete. 2024-05-26T22:48:30.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.914+0000 7f8261400640 1 -- 172.21.15.131:0/23670005 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f825c06e450 msgr2=0x7f825c0cf320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:30.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.914+0000 7f8261400640 1 --2- 172.21.15.131:0/23670005 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f825c06e450 0x7f825c0cf320 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f825c06c1d0 tx=0x7f824802a270 comp rx=0 tx=0).stop 2024-05-26T22:48:30.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.914+0000 7f8261400640 1 -- 172.21.15.131:0/23670005 shutdown_connections 2024-05-26T22:48:30.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.914+0000 7f8261400640 1 --2- 172.21.15.131:0/23670005 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f825c06e450 0x7f825c0cf320 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:30.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.914+0000 7f8261400640 1 --2- 172.21.15.131:0/23670005 >> v2:172.21.15.187:3300/0 conn(0x7f825c0d05d0 0x7f825c06df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:30.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.914+0000 7f8261400640 1 -- 172.21.15.131:0/23670005 >> 172.21.15.131:0/23670005 conn(0x7f825c06b940 msgr2=0x7f825c0b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:30.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.915+0000 7f8261400640 1 -- 172.21.15.131:0/23670005 shutdown_connections 2024-05-26T22:48:30.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.915+0000 7f8261400640 1 -- 172.21.15.131:0/23670005 wait complete. 2024-05-26T22:48:30.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.915+0000 7f8ff7e00640 1 Processor -- start 2024-05-26T22:48:30.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.915+0000 7f8ff7e00640 1 -- start start 2024-05-26T22:48:30.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.917+0000 7f2b55200640 1 Processor -- start 2024-05-26T22:48:30.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.917+0000 7f8ff7e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8ff80b8d40 0x7f8ff80b9190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:30.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.917+0000 7f8ff7e00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ff80b96d0 con 0x7f8ff80b9f20 2024-05-26T22:48:30.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.917+0000 7f2b55200640 1 -- start start 2024-05-26T22:48:30.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.918+0000 7f8ff7e00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ff80b79f0 con 0x7f8ff815e340 2024-05-26T22:48:30.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.918+0000 7f8ff7e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f8ff80b88b0 con 0x7f8ff80b8d40 2024-05-26T22:48:30.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.918+0000 7f8261400640 1 Processor -- start 2024-05-26T22:48:30.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.918+0000 7f8ff6000640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f8ff80b9f20 0x7f8ff80d22b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:48960/0 (socket says 172.21.15.131:48960) 2024-05-26T22:48:30.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.918+0000 7f8ff6000640 1 -- 172.21.15.131:0/2715212818 learned_addr learned my addr 172.21.15.131:0/2715212818 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:48:30.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.919+0000 7f8ff7400640 1 --2- 172.21.15.131:0/2715212818 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8ff80b8d40 0x7f8ff80b9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:30.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.919+0000 7f8261400640 1 -- start start 2024-05-26T22:48:30.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.920+0000 7f2b55200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2b500cc400 0x7f2b50150ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:30.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.920+0000 7f2b55200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2b500cf2e0 0x7f2b5015c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:30.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.920+0000 7f8ff7400640 1 -- 172.21.15.131:0/2715212818 >> v1:172.21.15.187:6789/0 conn(0x7f8ff815e340 legacy=0x7f8ff80b72e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:30.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.920+0000 7f8ff7400640 1 -- 172.21.15.131:0/2715212818 >> v1:172.21.15.131:6789/0 conn(0x7f8ff80b9f20 legacy=0x7f8ff80d22b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:30.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.920+0000 7f8ff7400640 1 -- 172.21.15.131:0/2715212818 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ff815e720 con 0x7f8ff80b8d40 2024-05-26T22:48:30.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.920+0000 7f8261400640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f825c06e450 0x7f825c0aebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:30.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.920+0000 7f8261400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f825c0cf860 0x7f825c14f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:30.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.921+0000 7f2b4f400640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2b500cc400 0x7f2b50150ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:30.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.920+0000 7f8261400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f825c0d05d0 0x7f825c0b07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:30.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.921+0000 7f2b55200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2b50157fa0 0x7f2b50152270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:30.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.921+0000 7f2b4f400640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2b500cc400 0x7f2b50150ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.131:36640/0 (socket says 172.21.15.131:36640) 2024-05-26T22:48:30.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.920+0000 7f8261400640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f825c06d9f0 con 0x7f825c0cf860 2024-05-26T22:48:30.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.920+0000 7f8261400640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f825c06d710 con 0x7f825c06e450 2024-05-26T22:48:30.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.920+0000 7f8261400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f825c0730c0 con 0x7f825c0d05d0 2024-05-26T22:48:30.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.921+0000 7f2b4f400640 1 -- 172.21.15.131:0/2476994522 learned_addr learned my addr 172.21.15.131:0/2476994522 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:30.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.921+0000 7f825a800640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f825c06e450 0x7f825c0aebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:30.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.921+0000 7f2b47e00640 1 --2- 172.21.15.131:0/2476994522 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2b500cf2e0 0x7f2b5015c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:30.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.921+0000 7f2b4fe00640 1 --2- 172.21.15.131:0/2476994522 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2b50157fa0 0x7f2b50152270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:30.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.921+0000 7f8259e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f825c0cf860 0x7f825c14f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:30.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.921+0000 7f2416800640 1 Processor -- start 2024-05-26T22:48:30.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.921+0000 7f2416800640 1 -- start start 2024-05-26T22:48:30.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.921+0000 7f2b55200640 1 -- 172.21.15.131:0/2476994522 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f2b5015ad70 con 0x7f2b50157fa0 2024-05-26T22:48:30.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.922+0000 7f2b55200640 1 -- 172.21.15.131:0/2476994522 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2b500cfc70 con 0x7f2b500cc400 2024-05-26T22:48:30.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.922+0000 7f2b55200640 1 -- 172.21.15.131:0/2476994522 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f2b5015ed60 con 0x7f2b500cf2e0 2024-05-26T22:48:30.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.921+0000 7f825a800640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f825c06e450 0x7f825c0aebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.131:36650/0 (socket says 172.21.15.131:36650) 2024-05-26T22:48:30.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.921+0000 7f825a800640 1 -- 172.21.15.131:0/2000410794 learned_addr learned my addr 172.21.15.131:0/2000410794 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:30.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.921+0000 7f825b200640 1 --2- 172.21.15.131:0/2000410794 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f825c0d05d0 0x7f825c0b07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:30.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.922+0000 7f825a800640 1 -- 172.21.15.131:0/2000410794 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f825c0d05d0 msgr2=0x7f825c0b07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:30.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.922+0000 7f825a800640 1 --2- 172.21.15.131:0/2000410794 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f825c0d05d0 0x7f825c0b07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:30.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.922+0000 7f825a800640 1 -- 172.21.15.131:0/2000410794 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f825c0cf860 msgr2=0x7f825c14f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:30.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.922+0000 7f825a800640 1 --2- 172.21.15.131:0/2000410794 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f825c0cf860 0x7f825c14f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:30.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.922+0000 7f825a800640 1 -- 172.21.15.131:0/2000410794 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f825c0ad4b0 con 0x7f825c06e450 2024-05-26T22:48:30.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.922+0000 7f8259e00640 1 --2- 172.21.15.131:0/2000410794 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f825c0cf860 0x7f825c14f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-26T22:48:30.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.923+0000 7f825a800640 1 --2- 172.21.15.131:0/2000410794 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f825c06e450 0x7f825c0aebf0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f824c00e870 tx=0x7f824c00ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:30.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.923+0000 7f8ff7400640 1 --2- 172.21.15.131:0/2715212818 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8ff80b8d40 0x7f8ff80b9190 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f8fe8009dc0 tx=0x7f8fe802a760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d16287fb72146877 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:30.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.923+0000 7f8252a00640 1 -- 172.21.15.131:0/2000410794 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f824c017070 con 0x7f825c06e450 2024-05-26T22:48:30.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.923+0000 7f2b4fe00640 1 -- 172.21.15.131:0/2476994522 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2b500cf2e0 msgr2=0x7f2b5015c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:30.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.923+0000 7f2b4fe00640 1 --2- 172.21.15.131:0/2476994522 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2b500cf2e0 0x7f2b5015c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:30.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.923+0000 7f2b4fe00640 1 -- 172.21.15.131:0/2476994522 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2b500cc400 msgr2=0x7f2b50150ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:30.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.923+0000 7f2b4fe00640 1 --2- 172.21.15.131:0/2476994522 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2b500cc400 0x7f2b50150ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:30.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.923+0000 7f8261400640 1 -- 172.21.15.131:0/2000410794 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f825c0ad790 con 0x7f825c06e450 2024-05-26T22:48:30.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.923+0000 7f8261400640 1 -- 172.21.15.131:0/2000410794 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f825c0add50 con 0x7f825c06e450 2024-05-26T22:48:30.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.923+0000 7f8252a00640 1 -- 172.21.15.131:0/2000410794 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f824c00c940 con 0x7f825c06e450 2024-05-26T22:48:30.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.923+0000 7f8252a00640 1 -- 172.21.15.131:0/2000410794 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f824c00cd30 con 0x7f825c06e450 2024-05-26T22:48:30.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.923+0000 7f2416800640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f24100ceae0 0x7f241015c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:30.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.924+0000 7f2416800640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24100d2290 con 0x7f2410071230 2024-05-26T22:48:30.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.924+0000 7f2416800640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24100d2570 con 0x7f2410145c00 2024-05-26T22:48:30.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.924+0000 7f2416800640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f24100d2040 con 0x7f24100ceae0 2024-05-26T22:48:30.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.924+0000 7f240fe00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f2410071230 0x7f24100b6560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:48974/0 (socket says 172.21.15.131:48974) 2024-05-26T22:48:30.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.924+0000 7f240fe00640 1 -- 172.21.15.131:0/1863511748 learned_addr learned my addr 172.21.15.131:0/1863511748 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:48:30.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.924+0000 7f8ff5600640 1 -- 172.21.15.131:0/2715212818 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe8007740 con 0x7f8ff80b8d40 2024-05-26T22:48:30.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.924+0000 7f8ff5600640 1 -- 172.21.15.131:0/2715212818 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fe80078e0 con 0x7f8ff80b8d40 2024-05-26T22:48:30.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.924+0000 7f8ff5600640 1 -- 172.21.15.131:0/2715212818 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe8007740 con 0x7f8ff80b8d40 2024-05-26T22:48:30.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.925+0000 7f240f400640 1 --2- 172.21.15.131:0/1863511748 >> v2:172.21.15.187:3300/0 conn(0x7f24100ceae0 0x7f241015c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:30.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.925+0000 7f2b4fe00640 1 -- 172.21.15.131:0/2476994522 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b5015d6b0 con 0x7f2b50157fa0 2024-05-26T22:48:30.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.925+0000 7f240e000640 1 -- 172.21.15.131:0/1863511748 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3762262292 0 0) 0x7f24100d2290 con 0x7f2410071230 2024-05-26T22:48:30.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.925+0000 7f240e000640 1 -- 172.21.15.131:0/1863511748 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f23f4003520 con 0x7f2410071230 2024-05-26T22:48:30.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.925+0000 7f8ff7e00640 1 -- 172.21.15.131:0/2715212818 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8ff80b8d40 msgr2=0x7f8ff80b9190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:30.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.925+0000 7f2b4fe00640 1 --2- 172.21.15.131:0/2476994522 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2b50157fa0 0x7f2b50152270 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2b48031000 tx=0x7f2b480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:30.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.926+0000 7f2b47400640 1 -- 172.21.15.131:0/2476994522 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b48032cd0 con 0x7f2b50157fa0 2024-05-26T22:48:30.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.926+0000 7f2b47400640 1 -- 172.21.15.131:0/2476994522 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b4803b410 con 0x7f2b50157fa0 2024-05-26T22:48:30.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.926+0000 7f2b55200640 1 -- 172.21.15.131:0/2476994522 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b50151000 con 0x7f2b50157fa0 2024-05-26T22:48:30.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.926+0000 7f2b55200640 1 -- 172.21.15.131:0/2476994522 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b5015dc30 con 0x7f2b50157fa0 2024-05-26T22:48:30.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.927+0000 7f2b47400640 1 -- 172.21.15.131:0/2476994522 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b4803b800 con 0x7f2b50157fa0 2024-05-26T22:48:30.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.927+0000 7f8261400640 1 -- 172.21.15.131:0/2000410794 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8234005290 con 0x7f825c06e450 2024-05-26T22:48:30.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.925+0000 7f8ff7e00640 1 --2- 172.21.15.131:0/2715212818 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8ff80b8d40 0x7f8ff80b9190 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f8fe8009dc0 tx=0x7f8fe802a760 comp rx=0 tx=0).stop 2024-05-26T22:48:30.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.926+0000 7f8ff7e00640 1 -- 172.21.15.131:0/2715212818 shutdown_connections 2024-05-26T22:48:30.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.926+0000 7f8ff7e00640 1 --2- 172.21.15.131:0/2715212818 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8ff80b8d40 0x7f8ff80b9190 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:30.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.926+0000 7f8ff7e00640 1 -- 172.21.15.131:0/2715212818 >> 172.21.15.131:0/2715212818 conn(0x7f8ff8072e60 msgr2=0x7f8ff8073270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:30.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.926+0000 7f8ff7e00640 1 -- 172.21.15.131:0/2715212818 shutdown_connections 2024-05-26T22:48:30.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.925+0000 7f240f400640 1 -- 172.21.15.131:0/1863511748 >> v1:172.21.15.187:6789/0 conn(0x7f2410145c00 legacy=0x7f24100ceec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:30.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.926+0000 7f240f400640 1 -- 172.21.15.131:0/1863511748 >> v1:172.21.15.131:6789/0 conn(0x7f2410071230 legacy=0x7f24100b6560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:30.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.926+0000 7f240f400640 1 -- 172.21.15.131:0/1863511748 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24100d2850 con 0x7f24100ceae0 2024-05-26T22:48:30.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.926+0000 7f240e000640 1 -- 172.21.15.131:0/1863511748 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2561612432 0 0) 0x7f23f4003520 con 0x7f2410071230 2024-05-26T22:48:30.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.927+0000 7f8252a00640 1 -- 172.21.15.131:0/2000410794 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f824c01a070 con 0x7f825c06e450 2024-05-26T22:48:30.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.928+0000 7f240f400640 1 --2- 172.21.15.131:0/1863511748 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f24100ceae0 0x7f241015c860 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2408009d90 tx=0x7f240802a5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=833b4c1d47192989 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:30.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.929+0000 7f8252a00640 1 --2- 172.21.15.131:0/2000410794 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f823c06be30 0x7f823c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:30.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.927+0000 7f8ff7e00640 1 -- 172.21.15.131:0/2715212818 wait complete. 2024-05-26T22:48:30.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.929+0000 7f8ff7e00640 1 Processor -- start 2024-05-26T22:48:30.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.929+0000 7f8252a00640 1 -- 172.21.15.131:0/2000410794 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f824c08ad90 con 0x7f825c06e450 2024-05-26T22:48:30.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.930+0000 7f8259e00640 1 --2- 172.21.15.131:0/2000410794 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f823c06be30 0x7f823c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:30.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.930+0000 7f2b55200640 1 -- 172.21.15.131:0/2476994522 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b28005290 con 0x7f2b50157fa0 2024-05-26T22:48:30.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.930+0000 7f8ff7e00640 1 -- start start 2024-05-26T22:48:30.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.930+0000 7f2b47400640 1 -- 172.21.15.131:0/2476994522 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2b4803b9a0 con 0x7f2b50157fa0 2024-05-26T22:48:30.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.931+0000 7f8259e00640 1 --2- 172.21.15.131:0/2000410794 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f823c06be30 0x7f823c06e2f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8248034000 tx=0x7f824802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:30.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.931+0000 7f8ff7e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ff80b8d40 0x7f8ff80cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:30.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.931+0000 7f8ff7e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8ff80b9f20 0x7f8ff80c9db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:30.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.932+0000 7f8ff7e00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ff815e340 0x7f8ff80ca2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:30.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.932+0000 7f8ff6a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ff80b8d40 0x7f8ff80cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:30.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.932+0000 7f8ff7e00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff80b8260 con 0x7f8ff80b8d40 2024-05-26T22:48:30.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.932+0000 7f2b47400640 1 --2- 172.21.15.131:0/2476994522 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f2b2c06be80 0x7f2b2c06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:30.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.932+0000 7f2b47400640 1 -- 172.21.15.131:0/2476994522 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f2b480ac130 con 0x7f2b50157fa0 2024-05-26T22:48:30.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.932+0000 7f8ff6a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ff80b8d40 0x7f8ff80cf4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54304/0 (socket says 172.21.15.131:54304) 2024-05-26T22:48:30.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.932+0000 7f8ff6000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8ff80b9f20 0x7f8ff80c9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:30.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.932+0000 7f8ff6a00640 1 -- 172.21.15.131:0/1849661104 learned_addr learned my addr 172.21.15.131:0/1849661104 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:30.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.932+0000 7f8ff7e00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff80b7f80 con 0x7f8ff815e340 2024-05-26T22:48:30.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.932+0000 7f8ff7e00640 1 -- 172.21.15.131:0/1849661104 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f8ff80b8540 con 0x7f8ff80b9f20 2024-05-26T22:48:30.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.933+0000 7f2b4f400640 1 --2- 172.21.15.131:0/2476994522 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f2b2c06be80 0x7f2b2c06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:30.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.933+0000 7f8ff7400640 1 --2- 172.21.15.131:0/1849661104 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ff815e340 0x7f8ff80ca2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:30.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.933+0000 7f8ff6000640 1 -- 172.21.15.131:0/1849661104 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ff815e340 msgr2=0x7f8ff80ca2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:30.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.933+0000 7f8ff6000640 1 --2- 172.21.15.131:0/1849661104 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ff815e340 0x7f8ff80ca2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:30.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.933+0000 7f8ff6000640 1 -- 172.21.15.131:0/1849661104 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ff80b8d40 msgr2=0x7f8ff80cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:30.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.933+0000 7f8ff6000640 1 --2- 172.21.15.131:0/1849661104 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ff80b8d40 0x7f8ff80cf4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:30.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.933+0000 7f8ff6000640 1 -- 172.21.15.131:0/1849661104 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ff8203300 con 0x7f8ff80b9f20 2024-05-26T22:48:30.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.934+0000 7f2b4f400640 1 --2- 172.21.15.131:0/2476994522 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f2b2c06be80 0x7f2b2c06e340 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2b40001fc0 tx=0x7f2b40009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:30.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.934+0000 7f8ff6000640 1 --2- 172.21.15.131:0/1849661104 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8ff80b9f20 0x7f8ff80c9db0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f8ff000c470 tx=0x7f8ff0001e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:30.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.934+0000 7f8feea00640 1 -- 172.21.15.131:0/1849661104 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ff0017070 con 0x7f8ff80b9f20 2024-05-26T22:48:30.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.934+0000 7f8ff7e00640 1 -- 172.21.15.131:0/1849661104 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ff82035e0 con 0x7f8ff80b9f20 2024-05-26T22:48:30.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.934+0000 7f8ff7e00640 1 -- 172.21.15.131:0/1849661104 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ff806e840 con 0x7f8ff80b9f20 2024-05-26T22:48:30.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.935+0000 7f8feea00640 1 -- 172.21.15.131:0/1849661104 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ff0005ce0 con 0x7f8ff80b9f20 2024-05-26T22:48:30.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.935+0000 7f8feea00640 1 -- 172.21.15.131:0/1849661104 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ff00072d0 con 0x7f8ff80b9f20 2024-05-26T22:48:30.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.937+0000 7f8ff7e00640 1 -- 172.21.15.131:0/1849661104 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ff806ea40 con 0x7f8ff80b9f20 2024-05-26T22:48:30.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.938+0000 7f2b47400640 1 -- 172.21.15.131:0/2476994522 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f2b4803f030 con 0x7f2b50157fa0 2024-05-26T22:48:30.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.937+0000 7f8feea00640 1 -- 172.21.15.131:0/1849661104 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8ff001a070 con 0x7f8ff80b9f20 2024-05-26T22:48:30.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.939+0000 7f8feea00640 1 --2- 172.21.15.131:0/1849661104 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f8fd406be30 0x7f8fd406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:30.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.939+0000 7f8feea00640 1 -- 172.21.15.131:0/1849661104 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f8ff008a130 con 0x7f8ff80b9f20 2024-05-26T22:48:30.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.939+0000 7f8ff6a00640 1 --2- 172.21.15.131:0/1849661104 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f8fd406be30 0x7f8fd406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:30.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.940+0000 7f8ff6a00640 1 --2- 172.21.15.131:0/1849661104 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f8fd406be30 0x7f8fd406e2f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8fe0007fd0 tx=0x7f8fe0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:30.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.941+0000 7f240e000640 1 -- 172.21.15.131:0/1863511748 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2400001650 con 0x7f24100ceae0 2024-05-26T22:48:30.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.942+0000 7f240e000640 1 -- 172.21.15.131:0/1863511748 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2408007540 con 0x7f24100ceae0 2024-05-26T22:48:30.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.942+0000 7f8252a00640 1 -- 172.21.15.131:0/2000410794 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f824c058fc0 con 0x7f825c06e450 2024-05-26T22:48:30.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.942+0000 7f240e000640 1 -- 172.21.15.131:0/1863511748 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2408007910 con 0x7f24100ceae0 2024-05-26T22:48:30.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.942+0000 7f2416800640 1 -- 172.21.15.131:0/1863511748 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f24100ceae0 msgr2=0x7f241015c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:30.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.942+0000 7f2416800640 1 --2- 172.21.15.131:0/1863511748 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f24100ceae0 0x7f241015c860 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2408009d90 tx=0x7f240802a5f0 comp rx=0 tx=0).stop 2024-05-26T22:48:30.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.943+0000 7f2416800640 1 -- 172.21.15.131:0/1863511748 shutdown_connections 2024-05-26T22:48:30.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.943+0000 7f2416800640 1 --2- 172.21.15.131:0/1863511748 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f24100ceae0 0x7f241015c860 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:30.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.943+0000 7f2416800640 1 -- 172.21.15.131:0/1863511748 >> 172.21.15.131:0/1863511748 conn(0x7f241006f6d0 msgr2=0x7f241006fae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:30.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.943+0000 7f2416800640 1 -- 172.21.15.131:0/1863511748 shutdown_connections 2024-05-26T22:48:30.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.943+0000 7f2416800640 1 -- 172.21.15.131:0/1863511748 wait complete. 2024-05-26T22:48:30.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.945+0000 7f2416800640 1 Processor -- start 2024-05-26T22:48:30.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.945+0000 7f2416800640 1 -- start start 2024-05-26T22:48:30.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.946+0000 7f2416800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2410071230 0x7f24100b2340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:30.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.947+0000 7f2416800640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f24100ceae0 0x7f24101fc140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:30.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.947+0000 7f2416800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2410145c00 0x7f2410141700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:30.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.947+0000 7f2416800640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f24100716a0 con 0x7f2410145c00 2024-05-26T22:48:30.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.947+0000 7f2416800640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f24100d2e10 con 0x7f24100ceae0 2024-05-26T22:48:30.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.947+0000 7f240f400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2410071230 0x7f24100b2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:30.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.947+0000 7f240fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2410145c00 0x7f2410141700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:30.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.947+0000 7f240fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2410145c00 0x7f2410141700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54318/0 (socket says 172.21.15.131:54318) 2024-05-26T22:48:30.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.947+0000 7f240fe00640 1 -- 172.21.15.131:0/3024389575 learned_addr learned my addr 172.21.15.131:0/3024389575 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:30.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.947+0000 7f2416800640 1 -- 172.21.15.131:0/3024389575 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f2410145850 con 0x7f2410071230 2024-05-26T22:48:30.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.947+0000 7f240ea00640 1 --2- 172.21.15.131:0/3024389575 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f24100ceae0 0x7f24101fc140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:30.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.948+0000 7f240fe00640 1 -- 172.21.15.131:0/3024389575 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2410071230 msgr2=0x7f24100b2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:30.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.948+0000 7f240fe00640 1 --2- 172.21.15.131:0/3024389575 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2410071230 0x7f24100b2340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:30.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.948+0000 7f240fe00640 1 -- 172.21.15.131:0/3024389575 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f24100ceae0 msgr2=0x7f24101fc140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:30.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.948+0000 7f240fe00640 1 --2- 172.21.15.131:0/3024389575 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f24100ceae0 0x7f24101fc140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:30.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.948+0000 7f240fe00640 1 -- 172.21.15.131:0/3024389575 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24101fca40 con 0x7f2410145c00 2024-05-26T22:48:30.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.948+0000 7f240fe00640 1 --2- 172.21.15.131:0/3024389575 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2410145c00 0x7f2410141700 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f240000c4a0 tx=0x7f240000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:30.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.949+0000 7f2407400640 1 -- 172.21.15.131:0/3024389575 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2400018070 con 0x7f2410145c00 2024-05-26T22:48:30.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.949+0000 7f2407400640 1 -- 172.21.15.131:0/3024389575 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24000074e0 con 0x7f2410145c00 2024-05-26T22:48:30.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.949+0000 7f2407400640 1 -- 172.21.15.131:0/3024389575 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2400004a20 con 0x7f2410145c00 2024-05-26T22:48:30.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.949+0000 7f2416800640 1 -- 172.21.15.131:0/3024389575 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24101fcd20 con 0x7f2410145c00 2024-05-26T22:48:30.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.949+0000 7f2416800640 1 -- 172.21.15.131:0/3024389575 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24100b1e20 con 0x7f2410145c00 2024-05-26T22:48:30.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.951+0000 7f240f400640 1 --2- 172.21.15.131:0/3024389575 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2410071230 0x7f24100b2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:48:30.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.951+0000 7f2416800640 1 -- 172.21.15.131:0/3024389575 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24100b2020 con 0x7f2410145c00 2024-05-26T22:48:30.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.953+0000 7f2407400640 1 -- 172.21.15.131:0/3024389575 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f240001b070 con 0x7f2410145c00 2024-05-26T22:48:30.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.955+0000 7f2407400640 1 --2- 172.21.15.131:0/3024389575 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f23ec06bf00 0x7f23ec06e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:30.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.955+0000 7f2407400640 1 -- 172.21.15.131:0/3024389575 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f240008a720 con 0x7f2410145c00 2024-05-26T22:48:30.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.956+0000 7f240f400640 1 --2- 172.21.15.131:0/3024389575 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f23ec06bf00 0x7f23ec06e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:30.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.957+0000 7f240f400640 1 --2- 172.21.15.131:0/3024389575 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f23ec06bf00 0x7f23ec06e3c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2408034000 tx=0x7f2408033040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:30.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.957+0000 7f8feea00640 1 -- 172.21.15.131:0/1849661104 <== mon.2 v2:172.21.15.187:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f8ff806ea40 con 0x7f8ff80b9f20 2024-05-26T22:48:30.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.963+0000 7f2407400640 1 -- 172.21.15.131:0/3024389575 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f2400022020 con 0x7f2410145c00 2024-05-26T22:48:30.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.984+0000 7f14dbe00640 1 Processor -- start 2024-05-26T22:48:30.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.984+0000 7f14dbe00640 1 -- start start 2024-05-26T22:48:30.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.985+0000 7fe415800640 1 Processor -- start 2024-05-26T22:48:30.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.986+0000 7f618e400640 1 Processor -- start 2024-05-26T22:48:30.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.986+0000 7f14dbe00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f14dc15beb0 0x7f14dc071260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:30.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.986+0000 7fe415800640 1 -- start start 2024-05-26T22:48:30.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.986+0000 7f14dbe00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f14dc0b3bb0 0x7f14dc06d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:30.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.987+0000 7f14dbe00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14dc15eb60 con 0x7f14dc06d500 2024-05-26T22:48:30.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.987+0000 7f14dbe00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f14dc06dea0 con 0x7f14dc15beb0 2024-05-26T22:48:30.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.987+0000 7f14dbe00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f14dc15f770 con 0x7f14dc0b3bb0 2024-05-26T22:48:30.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.987+0000 7f14da000640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f14dc06d500 0x7f14dc0b34a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.131:57750/0 (socket says 172.21.15.131:57750) 2024-05-26T22:48:30.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.987+0000 7f14da000640 1 -- 172.21.15.131:0/3311770566 learned_addr learned my addr 172.21.15.131:0/3311770566 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:48:30.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.987+0000 7f14daa00640 1 --2- 172.21.15.131:0/3311770566 >> v2:172.21.15.131:3300/0 conn(0x7f14dc15beb0 0x7f14dc071260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:30.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.988+0000 7f618e400640 1 -- start start 2024-05-26T22:48:30.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.988+0000 7f14db400640 1 --2- 172.21.15.131:0/3311770566 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f14dc0b3bb0 0x7f14dc06d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:30.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.988+0000 7f14daa00640 1 -- 172.21.15.131:0/3311770566 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f14dc0b3bb0 msgr2=0x7f14dc06d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:30.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.988+0000 7f14daa00640 1 --2- 172.21.15.131:0/3311770566 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f14dc0b3bb0 0x7f14dc06d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:30.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.988+0000 7f14daa00640 1 -- 172.21.15.131:0/3311770566 >> v1:172.21.15.187:6789/0 conn(0x7f14dc06d500 legacy=0x7f14dc0b34a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:30.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.988+0000 7f14daa00640 1 -- 172.21.15.131:0/3311770566 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14dc15ee40 con 0x7f14dc15beb0 2024-05-26T22:48:30.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.990+0000 7f618e400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f61800394c0 0x7f618004f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:30.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.990+0000 7f618e400640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61800063c0 con 0x7f618003a590 2024-05-26T22:48:30.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.990+0000 7f618e400640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6180039cc0 con 0x7f6180005ec0 2024-05-26T22:48:30.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.990+0000 7f618e400640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f61800342f0 con 0x7f61800394c0 2024-05-26T22:48:30.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.990+0000 7f6187400640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f6180005ec0 0x7f61800dfe80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.131:57760/0 (socket says 172.21.15.131:57760) 2024-05-26T22:48:30.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.990+0000 7f6186a00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f618003a590 0x7f6180038db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:48982/0 (socket says 172.21.15.131:48982) 2024-05-26T22:48:30.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.990+0000 7f6186a00640 1 -- 172.21.15.131:0/2337435851 learned_addr learned my addr 172.21.15.131:0/2337435851 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:48:30.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.990+0000 7f6187e00640 1 --2- 172.21.15.131:0/2337435851 >> v2:172.21.15.131:3300/0 conn(0x7f61800394c0 0x7f618004f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:30.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.988+0000 7fe415800640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fe41015e5b0 0x7fe4100ced40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:30.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.988+0000 7fe415800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe4100b3430 0x7fe4100cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:30.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.988+0000 7fe415800640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4100d2780 con 0x7fe4100cf7c0 2024-05-26T22:48:30.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.988+0000 7fe415800640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fe4100d2a60 con 0x7fe41015e5b0 2024-05-26T22:48:30.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.988+0000 7fe415800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe4100d2d40 con 0x7fe4100b3430 2024-05-26T22:48:30.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.988+0000 7fe40fe00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fe41015e5b0 0x7fe4100ced40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:30.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.988+0000 7fe40fe00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fe41015e5b0 0x7fe4100ced40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.131:36696/0 (socket says 172.21.15.131:36696) 2024-05-26T22:48:30.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.988+0000 7fe40fe00640 1 -- 172.21.15.131:0/674073768 learned_addr learned my addr 172.21.15.131:0/674073768 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:30.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.989+0000 7fe40fe00640 1 -- 172.21.15.131:0/674073768 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe4100b3430 msgr2=0x7fe4100cf280 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-26T22:48:30.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.989+0000 7fe40fe00640 1 --2- 172.21.15.131:0/674073768 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe4100b3430 0x7fe4100cf280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:30.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.989+0000 7fe40fe00640 1 -- 172.21.15.131:0/674073768 >> v1:172.21.15.131:6789/0 conn(0x7fe4100cf7c0 legacy=0x7fe4100d09e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-26T22:48:30.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.989+0000 7fe40fe00640 1 -- 172.21.15.131:0/674073768 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4100d3020 con 0x7fe41015e5b0 2024-05-26T22:48:30.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.991+0000 7f6187e00640 1 -- 172.21.15.131:0/2337435851 >> v1:172.21.15.187:6789/0 conn(0x7f6180005ec0 legacy=0x7f61800dfe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:30.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.991+0000 7f6187e00640 1 -- 172.21.15.131:0/2337435851 >> v1:172.21.15.131:6789/0 conn(0x7f618003a590 legacy=0x7f6180038db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:30.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.991+0000 7f6187e00640 1 -- 172.21.15.131:0/2337435851 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6180039fa0 con 0x7f61800394c0 2024-05-26T22:48:30.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.991+0000 7fe40fe00640 1 --2- 172.21.15.131:0/674073768 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe41015e5b0 0x7fe4100ced40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe400011f90 tx=0x7fe40003b090 comp rx=0 tx=0).ready entity=mon.1 client_cookie=171fc8c472ecefac server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:30.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.990+0000 7f14daa00640 1 --2- 172.21.15.131:0/3311770566 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f14dc15beb0 0x7f14dc071260 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f14dc15f8f0 tx=0x7f14c802a0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3d55a7e28fd6813 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:30.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.992+0000 7fe40ea00640 1 -- 172.21.15.131:0/674073768 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe40003bda0 con 0x7fe41015e5b0 2024-05-26T22:48:30.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.992+0000 7fe40ea00640 1 -- 172.21.15.131:0/674073768 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4000110d0 con 0x7fe41015e5b0 2024-05-26T22:48:30.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.992+0000 7fe40ea00640 1 -- 172.21.15.131:0/674073768 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4000114a0 con 0x7fe41015e5b0 2024-05-26T22:48:30.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.992+0000 7f6187e00640 1 --2- 172.21.15.131:0/2337435851 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61800394c0 0x7f618004f490 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f618806a260 tx=0x7f61880a5e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4c63ab3115946f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:30.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.993+0000 7fe415800640 1 -- 172.21.15.131:0/674073768 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe41015e5b0 msgr2=0x7fe4100ced40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:30.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.993+0000 7fe415800640 1 --2- 172.21.15.131:0/674073768 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe41015e5b0 0x7fe4100ced40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe400011f90 tx=0x7fe40003b090 comp rx=0 tx=0).stop 2024-05-26T22:48:30.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.993+0000 7fe415800640 1 -- 172.21.15.131:0/674073768 shutdown_connections 2024-05-26T22:48:30.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.993+0000 7fe415800640 1 --2- 172.21.15.131:0/674073768 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe4100b3430 0x7fe4100cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:30.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.993+0000 7fe415800640 1 --2- 172.21.15.131:0/674073768 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe41015e5b0 0x7fe4100ced40 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:30.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.993+0000 7fe415800640 1 -- 172.21.15.131:0/674073768 >> 172.21.15.131:0/674073768 conn(0x7fe41009d4d0 msgr2=0x7fe4100d41a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:30.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.993+0000 7f6186000640 1 -- 172.21.15.131:0/2337435851 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61880b3070 con 0x7f61800394c0 2024-05-26T22:48:30.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.993+0000 7f6186000640 1 -- 172.21.15.131:0/2337435851 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61880a6c30 con 0x7f61800394c0 2024-05-26T22:48:30.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.993+0000 7f6186000640 1 -- 172.21.15.131:0/2337435851 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61880ae640 con 0x7f61800394c0 2024-05-26T22:48:30.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.993+0000 7f14d9600640 1 -- 172.21.15.131:0/3311770566 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c8037070 con 0x7f14dc15beb0 2024-05-26T22:48:30.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.993+0000 7fe415800640 1 -- 172.21.15.131:0/674073768 shutdown_connections 2024-05-26T22:48:30.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.993+0000 7fe415800640 1 -- 172.21.15.131:0/674073768 wait complete. 2024-05-26T22:48:30.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.994+0000 7f618e400640 1 -- 172.21.15.131:0/2337435851 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61800394c0 msgr2=0x7f618004f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:30.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.994+0000 7f618e400640 1 --2- 172.21.15.131:0/2337435851 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61800394c0 0x7f618004f490 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f618806a260 tx=0x7f61880a5e20 comp rx=0 tx=0).stop 2024-05-26T22:48:30.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.993+0000 7f14d9600640 1 -- 172.21.15.131:0/3311770566 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14c802ad30 con 0x7f14dc15beb0 2024-05-26T22:48:30.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.994+0000 7f14dbe00640 1 -- 172.21.15.131:0/3311770566 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f14dc15beb0 msgr2=0x7f14dc071260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:30.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.994+0000 7f14dbe00640 1 --2- 172.21.15.131:0/3311770566 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f14dc15beb0 0x7f14dc071260 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f14dc15f8f0 tx=0x7f14c802a0a0 comp rx=0 tx=0).stop 2024-05-26T22:48:30.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.994+0000 7f14dbe00640 1 -- 172.21.15.131:0/3311770566 shutdown_connections 2024-05-26T22:48:30.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.994+0000 7f14dbe00640 1 --2- 172.21.15.131:0/3311770566 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f14dc0b3bb0 0x7f14dc06d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:30.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.994+0000 7f14dbe00640 1 --2- 172.21.15.131:0/3311770566 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f14dc15beb0 0x7f14dc071260 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:30.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.994+0000 7f14dbe00640 1 -- 172.21.15.131:0/3311770566 >> 172.21.15.131:0/3311770566 conn(0x7f14dc0b1c60 msgr2=0x7f14dc0b2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:30.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.994+0000 7f14dbe00640 1 -- 172.21.15.131:0/3311770566 shutdown_connections 2024-05-26T22:48:30.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.995+0000 7f14dbe00640 1 -- 172.21.15.131:0/3311770566 wait complete. 2024-05-26T22:48:30.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.995+0000 7f618e400640 1 -- 172.21.15.131:0/2337435851 shutdown_connections 2024-05-26T22:48:30.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.995+0000 7f618e400640 1 --2- 172.21.15.131:0/2337435851 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61800394c0 0x7f618004f490 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:30.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.995+0000 7f618e400640 1 -- 172.21.15.131:0/2337435851 >> 172.21.15.131:0/2337435851 conn(0x7f6180009fe0 msgr2=0x7f618000a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:30.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.996+0000 7fe415800640 1 Processor -- start 2024-05-26T22:48:30.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.997+0000 7f14dbe00640 1 Processor -- start 2024-05-26T22:48:30.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.998+0000 7f14dbe00640 1 -- start start 2024-05-26T22:48:31.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.996+0000 7fe415800640 1 -- start start 2024-05-26T22:48:31.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.997+0000 7fe415800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe4100b3430 0x7fe4100cab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:31.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.998+0000 7fe415800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe4100cf7c0 0x7fe410157540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:31.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.998+0000 7fe415800640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe4100cc240 0x7fe4100c7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:31.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.998+0000 7fe415800640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe4100b5030 con 0x7fe4100b3430 2024-05-26T22:48:31.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.998+0000 7fe415800640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe4100cfc10 con 0x7fe4100cc240 2024-05-26T22:48:31.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.998+0000 7fe415800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe4100b51b0 con 0x7fe4100cf7c0 2024-05-26T22:48:31.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.996+0000 7f618e400640 1 -- 172.21.15.131:0/2337435851 shutdown_connections 2024-05-26T22:48:31.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.996+0000 7f618e400640 1 -- 172.21.15.131:0/2337435851 wait complete. 2024-05-26T22:48:31.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.998+0000 7f618e400640 1 Processor -- start 2024-05-26T22:48:31.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.999+0000 7f618e400640 1 -- start start 2024-05-26T22:48:31.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.998+0000 7fe40fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe4100b3430 0x7fe4100cab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:31.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.999+0000 7fe40f400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe4100cf7c0 0x7fe410157540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:31.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.999+0000 7fe40fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe4100b3430 0x7fe4100cab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54346/0 (socket says 172.21.15.131:54346) 2024-05-26T22:48:31.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.999+0000 7fe40f400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe4100cf7c0 0x7fe410157540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.131:43466/0 (socket says 172.21.15.131:43466) 2024-05-26T22:48:31.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:30.999+0000 7fe40fe00640 1 -- 172.21.15.131:0/3978024872 learned_addr learned my addr 172.21.15.131:0/3978024872 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:31.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.000+0000 7fe40fe00640 1 -- 172.21.15.131:0/3978024872 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe4100cf7c0 msgr2=0x7fe410157540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:31.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.000+0000 7fe40fe00640 1 --2- 172.21.15.131:0/3978024872 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe4100cf7c0 0x7fe410157540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:31.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.000+0000 7fe40fe00640 1 -- 172.21.15.131:0/3978024872 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe4100cc240 msgr2=0x7fe4100c7ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:31.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.000+0000 7fe40fe00640 1 --2- 172.21.15.131:0/3978024872 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe4100cc240 0x7fe4100c7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:31.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.000+0000 7fe40fe00640 1 -- 172.21.15.131:0/3978024872 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4100cdf80 con 0x7fe4100b3430 2024-05-26T22:48:31.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.000+0000 7f14dbe00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f14dc06d500 0x7f14dc1458a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:31.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.000+0000 7f14dbe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f14dc0b3bb0 0x7f14dc152bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:31.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.000+0000 7f14dbe00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f14dc15beb0 0x7f14dc15c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:31.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.000+0000 7f14dbe00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f14dc15c290 con 0x7f14dc0b3bb0 2024-05-26T22:48:31.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.000+0000 7f14dbe00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f14dc15f400 con 0x7f14dc15beb0 2024-05-26T22:48:31.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.000+0000 7f14dbe00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f14dc1620f0 con 0x7f14dc06d500 2024-05-26T22:48:31.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.000+0000 7f618e400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6180005ec0 0x7f61800db8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:31.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.000+0000 7f618e400640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f61800394c0 0x7f6180052c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:31.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.000+0000 7f618e400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f618003a590 0x7f618004de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:31.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.000+0000 7f618e400640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f6180039a60 con 0x7f618003a590 2024-05-26T22:48:31.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.001+0000 7f618e400640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f61800398e0 con 0x7f61800394c0 2024-05-26T22:48:31.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.001+0000 7f618e400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f618004f9d0 con 0x7f6180005ec0 2024-05-26T22:48:31.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.001+0000 7f6187400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6180005ec0 0x7f61800db8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:31.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.001+0000 7fe40f400640 1 --2- 172.21.15.131:0/3978024872 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe4100cf7c0 0x7fe410157540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-05-26T22:48:31.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.001+0000 7f14daa00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f14dc06d500 0x7f14dc1458a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:31.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.001+0000 7f14da000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f14dc0b3bb0 0x7f14dc152bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:31.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.001+0000 7f14da000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f14dc0b3bb0 0x7f14dc152bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54360/0 (socket says 172.21.15.131:54360) 2024-05-26T22:48:31.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.001+0000 7f14db400640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f14dc15beb0 0x7f14dc15c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:31.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.001+0000 7f14da000640 1 -- 172.21.15.131:0/1138845853 learned_addr learned my addr 172.21.15.131:0/1138845853 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:31.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.001+0000 7fe40fe00640 1 --2- 172.21.15.131:0/3978024872 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe4100b3430 0x7fe4100cab40 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe400011f90 tx=0x7fe4000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:31.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.002+0000 7f14daa00640 1 -- 172.21.15.131:0/1138845853 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f14dc15beb0 msgr2=0x7f14dc15c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:31.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.002+0000 7f14daa00640 1 --2- 172.21.15.131:0/1138845853 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f14dc15beb0 0x7f14dc15c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:31.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.002+0000 7f14daa00640 1 -- 172.21.15.131:0/1138845853 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f14dc0b3bb0 msgr2=0x7f14dc152bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:31.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.001+0000 7f6187e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f618003a590 0x7f618004de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:31.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.002+0000 7f6187e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f618003a590 0x7f618004de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54368/0 (socket says 172.21.15.131:54368) 2024-05-26T22:48:31.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.002+0000 7f6187e00640 1 -- 172.21.15.131:0/3707849925 learned_addr learned my addr 172.21.15.131:0/3707849925 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:31.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.002+0000 7f14daa00640 1 --2- 172.21.15.131:0/1138845853 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f14dc0b3bb0 0x7f14dc152bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:31.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.002+0000 7f14daa00640 1 -- 172.21.15.131:0/1138845853 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14dc1558c0 con 0x7f14dc06d500 2024-05-26T22:48:31.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.002+0000 7f14da000640 1 --2- 172.21.15.131:0/1138845853 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f14dc0b3bb0 0x7f14dc152bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-26T22:48:31.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.002+0000 7f6187e00640 1 -- 172.21.15.131:0/3707849925 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6180005ec0 msgr2=0x7f61800db8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:31.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.002+0000 7f6187e00640 1 --2- 172.21.15.131:0/3707849925 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6180005ec0 0x7f61800db8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:31.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.002+0000 7f6187e00640 1 -- 172.21.15.131:0/3707849925 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f61800394c0 msgr2=0x7f6180052c80 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-26T22:48:31.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.002+0000 7f6187e00640 1 --2- 172.21.15.131:0/3707849925 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f61800394c0 0x7f6180052c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:31.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.002+0000 7f6187e00640 1 -- 172.21.15.131:0/3707849925 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f618004b270 con 0x7f618003a590 2024-05-26T22:48:31.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.002+0000 7f6187400640 1 --2- 172.21.15.131:0/3707849925 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6180005ec0 0x7f61800db8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-26T22:48:31.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.002+0000 7f14daa00640 1 --2- 172.21.15.131:0/1138845853 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f14dc06d500 0x7f14dc1458a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f14c8031000 tx=0x7f14c80046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:31.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.003+0000 7f6187e00640 1 --2- 172.21.15.131:0/3707849925 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f618003a590 0x7f618004de40 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f618806a300 tx=0x7f618806a770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:31.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.003+0000 7f617b400640 1 -- 172.21.15.131:0/3707849925 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61880bd020 con 0x7f618003a590 2024-05-26T22:48:31.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.003+0000 7f617b400640 1 -- 172.21.15.131:0/3707849925 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61880b9620 con 0x7f618003a590 2024-05-26T22:48:31.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.003+0000 7f617b400640 1 -- 172.21.15.131:0/3707849925 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61880b3040 con 0x7f618003a590 2024-05-26T22:48:31.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.003+0000 7f618e400640 1 -- 172.21.15.131:0/3707849925 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61800e2ae0 con 0x7f618003a590 2024-05-26T22:48:31.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.003+0000 7fe407e00640 1 -- 172.21.15.131:0/3978024872 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe400052020 con 0x7fe4100b3430 2024-05-26T22:48:31.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.004+0000 7f14d2a00640 1 -- 172.21.15.131:0/1138845853 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c8037050 con 0x7f14dc06d500 2024-05-26T22:48:31.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.004+0000 7f14d2a00640 1 -- 172.21.15.131:0/1138845853 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14c8038040 con 0x7f14dc06d500 2024-05-26T22:48:31.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.004+0000 7f14d2a00640 1 -- 172.21.15.131:0/1138845853 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c8030b40 con 0x7f14dc06d500 2024-05-26T22:48:31.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.003+0000 7f618e400640 1 -- 172.21.15.131:0/3707849925 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f618004e380 con 0x7f618003a590 2024-05-26T22:48:31.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.004+0000 7fe407e00640 1 -- 172.21.15.131:0/3978024872 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe40000fe10 con 0x7fe4100b3430 2024-05-26T22:48:31.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.004+0000 7fe415800640 1 -- 172.21.15.131:0/3978024872 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4100c8c60 con 0x7fe4100b3430 2024-05-26T22:48:31.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.005+0000 7fe415800640 1 -- 172.21.15.131:0/3978024872 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4100ca5f0 con 0x7fe4100b3430 2024-05-26T22:48:31.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.006+0000 7f14dbe00640 1 -- 172.21.15.131:0/1138845853 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14dc155ba0 con 0x7f14dc06d500 2024-05-26T22:48:31.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.006+0000 7f14dbe00640 1 -- 172.21.15.131:0/1138845853 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f14dc156c80 con 0x7f14dc06d500 2024-05-26T22:48:31.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.006+0000 7fe415800640 1 -- 172.21.15.131:0/3978024872 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4100c9180 con 0x7fe4100b3430 2024-05-26T22:48:31.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.006+0000 7fe407e00640 1 -- 172.21.15.131:0/3978024872 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe40004d690 con 0x7fe4100b3430 2024-05-26T22:48:31.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.008+0000 7f14dbe00640 1 -- 172.21.15.131:0/1138845853 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14dc155d70 con 0x7f14dc06d500 2024-05-26T22:48:31.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.007+0000 7f618e400640 1 -- 172.21.15.131:0/3707849925 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61800053c0 con 0x7f618003a590 2024-05-26T22:48:31.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.009+0000 7f617b400640 1 -- 172.21.15.131:0/3707849925 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f61880c0020 con 0x7f618003a590 2024-05-26T22:48:31.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.011+0000 7f617b400640 1 --2- 172.21.15.131:0/3707849925 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f616806bf00 0x7f616806e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:31.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.011+0000 7f617b400640 1 -- 172.21.15.131:0/3707849925 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f6188127510 con 0x7f618003a590 2024-05-26T22:48:31.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.010+0000 7f14d2a00640 1 -- 172.21.15.131:0/1138845853 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f14c8045020 con 0x7f14dc06d500 2024-05-26T22:48:31.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.012+0000 7f14d2a00640 1 --2- 172.21.15.131:0/1138845853 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f14bc06be80 0x7f14bc06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:31.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.012+0000 7f14da000640 1 --2- 172.21.15.131:0/1138845853 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f14bc06be80 0x7f14bc06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:31.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.012+0000 7f14d2a00640 1 -- 172.21.15.131:0/1138845853 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f14c8032440 con 0x7f14dc06d500 2024-05-26T22:48:31.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.013+0000 7f6187400640 1 --2- 172.21.15.131:0/3707849925 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f616806bf00 0x7f616806e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:31.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.014+0000 7f14da000640 1 --2- 172.21.15.131:0/1138845853 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f14bc06be80 0x7f14bc06e340 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f14dc0b1920 tx=0x7f14cc000ed0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:31.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.014+0000 7f6187400640 1 --2- 172.21.15.131:0/3707849925 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f616806bf00 0x7f616806e3c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f6174007fd0 tx=0x7f6174007d60 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:31.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.011+0000 7fe407e00640 1 -- 172.21.15.131:0/3978024872 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe400055020 con 0x7fe4100b3430 2024-05-26T22:48:31.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.013+0000 7fe407e00640 1 --2- 172.21.15.131:0/3978024872 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fe3f006be80 0x7fe3f006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:31.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.013+0000 7fe407e00640 1 -- 172.21.15.131:0/3978024872 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fe4000bca20 con 0x7fe4100b3430 2024-05-26T22:48:31.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.015+0000 7fe40f400640 1 --2- 172.21.15.131:0/3978024872 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fe3f006be80 0x7fe3f006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:31.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.016+0000 7fe40f400640 1 --2- 172.21.15.131:0/3978024872 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fe3f006be80 0x7fe3f006e340 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe410161930 tx=0x7fe3fc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:31.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.016+0000 7fe407e00640 1 -- 172.21.15.131:0/3978024872 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fe40005c030 con 0x7fe4100b3430 2024-05-26T22:48:31.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.025+0000 7f617b400640 1 -- 172.21.15.131:0/3707849925 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f61880c7030 con 0x7f618003a590 2024-05-26T22:48:31.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.028+0000 7f14d2a00640 1 -- 172.21.15.131:0/1138845853 <== mon.2 v2:172.21.15.187:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f14c80791b0 con 0x7f14dc06d500 2024-05-26T22:48:31.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.354+0000 7fbd68c00640 1 -- 172.21.15.131:0/3390834752 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fbd4c005580 con 0x7fbd700e8420 2024-05-26T22:48:31.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.355+0000 7fbd6b400640 1 -- 172.21.15.131:0/3390834752 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fbd6405ca20 con 0x7fbd700e8420 2024-05-26T22:48:31.355 INFO:teuthology.orchestra.run.smithi131.stdout:55834574854 2024-05-26T22:48:31.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.358+0000 7fbd68c00640 1 -- 172.21.15.131:0/3390834752 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fbd5806be30 msgr2=0x7fbd5806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:31.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.358+0000 7fbd68c00640 1 --2- 172.21.15.131:0/3390834752 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fbd5806be30 0x7fbd5806e2f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fbd7806a130 tx=0x7fbd7808d040 comp rx=0 tx=0).stop 2024-05-26T22:48:31.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.359+0000 7fbd68c00640 1 -- 172.21.15.131:0/3390834752 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbd700e8420 msgr2=0x7fbd70047610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:31.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.359+0000 7fbd68c00640 1 --2- 172.21.15.131:0/3390834752 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbd700e8420 0x7fbd70047610 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbd6400c8c0 tx=0x7fbd6400cd80 comp rx=0 tx=0).stop 2024-05-26T22:48:31.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.359+0000 7fbd68c00640 1 -- 172.21.15.131:0/3390834752 shutdown_connections 2024-05-26T22:48:31.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.359+0000 7fbd68c00640 1 --2- 172.21.15.131:0/3390834752 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fbd5806be30 0x7fbd5806e2f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:31.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.359+0000 7fbd68c00640 1 --2- 172.21.15.131:0/3390834752 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbd7004bb90 0x7fbd70043d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:31.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.359+0000 7fbd68c00640 1 --2- 172.21.15.131:0/3390834752 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbd700e8420 0x7fbd70047610 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:31.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.359+0000 7fbd68c00640 1 --2- 172.21.15.131:0/3390834752 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbd60004d40 0x7fbd7017cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:31.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.359+0000 7fbd68c00640 1 -- 172.21.15.131:0/3390834752 >> 172.21.15.131:0/3390834752 conn(0x7fbd70009fb0 msgr2=0x7fbd70037b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:31.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.360+0000 7fbd68c00640 1 -- 172.21.15.131:0/3390834752 shutdown_connections 2024-05-26T22:48:31.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.360+0000 7fbd68c00640 1 -- 172.21.15.131:0/3390834752 wait complete. 2024-05-26T22:48:31.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.371+0000 7f2b55200640 1 -- 172.21.15.131:0/2476994522 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f2b28005580 con 0x7f2b50157fa0 2024-05-26T22:48:31.374 INFO:teuthology.orchestra.run.smithi131.stdout:55834574854 2024-05-26T22:48:31.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.372+0000 7f2b47400640 1 -- 172.21.15.131:0/2476994522 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f2b4807a500 con 0x7f2b50157fa0 2024-05-26T22:48:31.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.377+0000 7f2b55200640 1 -- 172.21.15.131:0/2476994522 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f2b2c06be80 msgr2=0x7f2b2c06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:31.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.377+0000 7f2b55200640 1 --2- 172.21.15.131:0/2476994522 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f2b2c06be80 0x7f2b2c06e340 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2b40001fc0 tx=0x7f2b40009040 comp rx=0 tx=0).stop 2024-05-26T22:48:31.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.377+0000 7f2b55200640 1 -- 172.21.15.131:0/2476994522 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2b50157fa0 msgr2=0x7f2b50152270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:31.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.377+0000 7f2b55200640 1 --2- 172.21.15.131:0/2476994522 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2b50157fa0 0x7f2b50152270 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2b48031000 tx=0x7f2b480046c0 comp rx=0 tx=0).stop 2024-05-26T22:48:31.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.377+0000 7f8261400640 1 -- 172.21.15.131:0/2000410794 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f8234005580 con 0x7f825c06e450 2024-05-26T22:48:31.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.377+0000 7f2b55200640 1 -- 172.21.15.131:0/2476994522 shutdown_connections 2024-05-26T22:48:31.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.377+0000 7f2b55200640 1 --2- 172.21.15.131:0/2476994522 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2b500cf2e0 0x7f2b5015c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:31.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.377+0000 7f2b55200640 1 --2- 172.21.15.131:0/2476994522 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f2b2c06be80 0x7f2b2c06e340 secure :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2b40001fc0 tx=0x7f2b40009040 comp rx=0 tx=0).stop 2024-05-26T22:48:31.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.377+0000 7f2b55200640 1 --2- 172.21.15.131:0/2476994522 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2b50157fa0 0x7f2b50152270 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:31.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.377+0000 7f2b55200640 1 --2- 172.21.15.131:0/2476994522 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2b500cc400 0x7f2b50150ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:31.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.377+0000 7f2b55200640 1 -- 172.21.15.131:0/2476994522 >> 172.21.15.131:0/2476994522 conn(0x7f2b500b1c60 msgr2=0x7f2b50154cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:31.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.377+0000 7f2b55200640 1 -- 172.21.15.131:0/2476994522 shutdown_connections 2024-05-26T22:48:31.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.378+0000 7f8252a00640 1 -- 172.21.15.131:0/2000410794 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f824c05d2c0 con 0x7f825c06e450 2024-05-26T22:48:31.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.377+0000 7f2b55200640 1 -- 172.21.15.131:0/2476994522 wait complete. 2024-05-26T22:48:31.379 INFO:teuthology.orchestra.run.smithi131.stdout:55834574854 2024-05-26T22:48:31.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.381+0000 7f8261400640 1 -- 172.21.15.131:0/2000410794 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f823c06be30 msgr2=0x7f823c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:31.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.381+0000 7f8261400640 1 --2- 172.21.15.131:0/2000410794 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f823c06be30 0x7f823c06e2f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8248034000 tx=0x7f824802e000 comp rx=0 tx=0).stop 2024-05-26T22:48:31.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.381+0000 7f8261400640 1 -- 172.21.15.131:0/2000410794 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f825c06e450 msgr2=0x7f825c0aebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:31.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.381+0000 7f8261400640 1 --2- 172.21.15.131:0/2000410794 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f825c06e450 0x7f825c0aebf0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f824c00e870 tx=0x7f824c00ed30 comp rx=0 tx=0).stop 2024-05-26T22:48:31.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.382+0000 7f8261400640 1 -- 172.21.15.131:0/2000410794 shutdown_connections 2024-05-26T22:48:31.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.382+0000 7f8261400640 1 --2- 172.21.15.131:0/2000410794 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f825c0d05d0 0x7f825c0b07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:31.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.382+0000 7f8261400640 1 --2- 172.21.15.131:0/2000410794 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f823c06be30 0x7f823c06e2f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:31.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.382+0000 7f8261400640 1 --2- 172.21.15.131:0/2000410794 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f825c0cf860 0x7f825c14f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:31.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.382+0000 7f8261400640 1 --2- 172.21.15.131:0/2000410794 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f825c06e450 0x7f825c0aebf0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:31.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.382+0000 7f8261400640 1 -- 172.21.15.131:0/2000410794 >> 172.21.15.131:0/2000410794 conn(0x7f825c06b940 msgr2=0x7f825c072970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:31.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.383+0000 7f8261400640 1 -- 172.21.15.131:0/2000410794 shutdown_connections 2024-05-26T22:48:31.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.383+0000 7f8261400640 1 -- 172.21.15.131:0/2000410794 wait complete. 2024-05-26T22:48:31.387 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.2 2024-05-26T22:48:31.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.387+0000 7f2416800640 1 -- 172.21.15.131:0/3024389575 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f24100695c0 con 0x7f2410145c00 2024-05-26T22:48:31.387 DEBUG:teuthology.parallel:result is None 2024-05-26T22:48:31.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.388+0000 7f2407400640 1 -- 172.21.15.131:0/3024389575 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f2400059140 con 0x7f2410145c00 2024-05-26T22:48:31.388 INFO:teuthology.orchestra.run.smithi131.stdout:55834574854 2024-05-26T22:48:31.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.392+0000 7f8ff7e00640 1 -- 172.21.15.131:0/1849661104 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f8ff806e630 con 0x7f8ff80b9f20 2024-05-26T22:48:31.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.392+0000 7f2416800640 1 -- 172.21.15.131:0/3024389575 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f23ec06bf00 msgr2=0x7f23ec06e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:31.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.392+0000 7f2416800640 1 --2- 172.21.15.131:0/3024389575 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f23ec06bf00 0x7f23ec06e3c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2408034000 tx=0x7f2408033040 comp rx=0 tx=0).stop 2024-05-26T22:48:31.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.392+0000 7f2416800640 1 -- 172.21.15.131:0/3024389575 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2410145c00 msgr2=0x7f2410141700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:31.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.392+0000 7f2416800640 1 --2- 172.21.15.131:0/3024389575 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2410145c00 0x7f2410141700 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f240000c4a0 tx=0x7f240000c960 comp rx=0 tx=0).stop 2024-05-26T22:48:31.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.393+0000 7f8feea00640 1 -- 172.21.15.131:0/1849661104 <== mon.2 v2:172.21.15.187:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f8ff00583f0 con 0x7f8ff80b9f20 2024-05-26T22:48:31.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.393+0000 7f2416800640 1 -- 172.21.15.131:0/3024389575 shutdown_connections 2024-05-26T22:48:31.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.393+0000 7f2416800640 1 --2- 172.21.15.131:0/3024389575 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f23ec06bf00 0x7f23ec06e3c0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:31.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.393+0000 7f2416800640 1 --2- 172.21.15.131:0/3024389575 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2410145c00 0x7f2410141700 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:31.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.393+0000 7f2416800640 1 --2- 172.21.15.131:0/3024389575 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f24100ceae0 0x7f24101fc140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:31.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.393+0000 7f2416800640 1 --2- 172.21.15.131:0/3024389575 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2410071230 0x7f24100b2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:31.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.393+0000 7f2416800640 1 -- 172.21.15.131:0/3024389575 >> 172.21.15.131:0/3024389575 conn(0x7f241006f6d0 msgr2=0x7f2410144880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:31.394 INFO:teuthology.orchestra.run.smithi131.stdout:55834574854 2024-05-26T22:48:31.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.393+0000 7f2416800640 1 -- 172.21.15.131:0/3024389575 shutdown_connections 2024-05-26T22:48:31.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.393+0000 7f2416800640 1 -- 172.21.15.131:0/3024389575 wait complete. 2024-05-26T22:48:31.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.397+0000 7f8ff7e00640 1 -- 172.21.15.131:0/1849661104 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f8fd406be30 msgr2=0x7f8fd406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:31.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.397+0000 7f8ff7e00640 1 --2- 172.21.15.131:0/1849661104 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f8fd406be30 0x7f8fd406e2f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8fe0007fd0 tx=0x7f8fe0009040 comp rx=0 tx=0).stop 2024-05-26T22:48:31.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.397+0000 7f8ff7e00640 1 -- 172.21.15.131:0/1849661104 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8ff80b9f20 msgr2=0x7f8ff80c9db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:31.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.397+0000 7f8ff7e00640 1 --2- 172.21.15.131:0/1849661104 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8ff80b9f20 0x7f8ff80c9db0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f8ff000c470 tx=0x7f8ff0001e70 comp rx=0 tx=0).stop 2024-05-26T22:48:31.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.397+0000 7f8ff7e00640 1 -- 172.21.15.131:0/1849661104 shutdown_connections 2024-05-26T22:48:31.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.397+0000 7f8ff7e00640 1 --2- 172.21.15.131:0/1849661104 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8ff80b9f20 0x7f8ff80c9db0 secure :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f8ff000c470 tx=0x7f8ff0001e70 comp rx=0 tx=0).stop 2024-05-26T22:48:31.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.397+0000 7f8ff7e00640 1 --2- 172.21.15.131:0/1849661104 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f8fd406be30 0x7f8fd406e2f0 secure :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8fe0007fd0 tx=0x7f8fe0009040 comp rx=0 tx=0).stop 2024-05-26T22:48:31.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.397+0000 7f8ff7e00640 1 --2- 172.21.15.131:0/1849661104 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ff815e340 0x7f8ff80ca2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:31.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.397+0000 7f8ff7e00640 1 --2- 172.21.15.131:0/1849661104 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ff80b8d40 0x7f8ff80cf4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:31.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.397+0000 7f8ff7e00640 1 -- 172.21.15.131:0/1849661104 >> 172.21.15.131:0/1849661104 conn(0x7f8ff8072e60 msgr2=0x7f8ff80d49d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:31.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.397+0000 7f8ff7e00640 1 -- 172.21.15.131:0/1849661104 shutdown_connections 2024-05-26T22:48:31.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.398+0000 7f8ff7e00640 1 -- 172.21.15.131:0/1849661104 wait complete. 2024-05-26T22:48:31.403 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.1 2024-05-26T22:48:31.403 DEBUG:teuthology.parallel:result is None 2024-05-26T22:48:31.409 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.4 2024-05-26T22:48:31.409 DEBUG:teuthology.parallel:result is None 2024-05-26T22:48:31.415 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.7 2024-05-26T22:48:31.415 DEBUG:teuthology.parallel:result is None 2024-05-26T22:48:31.418 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.3 2024-05-26T22:48:31.419 DEBUG:teuthology.parallel:result is None 2024-05-26T22:48:31.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.424+0000 7f618e400640 1 -- 172.21.15.131:0/3707849925 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f6180053a70 con 0x7f618003a590 2024-05-26T22:48:31.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.425+0000 7f617b400640 1 -- 172.21.15.131:0/3707849925 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f61880f5f30 con 0x7f618003a590 2024-05-26T22:48:31.425 INFO:teuthology.orchestra.run.smithi131.stdout:55834574854 2024-05-26T22:48:31.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.427+0000 7f618e400640 1 -- 172.21.15.131:0/3707849925 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f616806bf00 msgr2=0x7f616806e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:31.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.427+0000 7f618e400640 1 --2- 172.21.15.131:0/3707849925 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f616806bf00 0x7f616806e3c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f6174007fd0 tx=0x7f6174007d60 comp rx=0 tx=0).stop 2024-05-26T22:48:31.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.427+0000 7f618e400640 1 -- 172.21.15.131:0/3707849925 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f618003a590 msgr2=0x7f618004de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:31.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.427+0000 7f618e400640 1 --2- 172.21.15.131:0/3707849925 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f618003a590 0x7f618004de40 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f618806a300 tx=0x7f618806a770 comp rx=0 tx=0).stop 2024-05-26T22:48:31.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.427+0000 7f618e400640 1 -- 172.21.15.131:0/3707849925 shutdown_connections 2024-05-26T22:48:31.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.427+0000 7f618e400640 1 --2- 172.21.15.131:0/3707849925 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f616806bf00 0x7f616806e3c0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:31.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.427+0000 7f618e400640 1 --2- 172.21.15.131:0/3707849925 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f618003a590 0x7f618004de40 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:31.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.427+0000 7f618e400640 1 --2- 172.21.15.131:0/3707849925 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f61800394c0 0x7f6180052c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:31.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.427+0000 7f618e400640 1 --2- 172.21.15.131:0/3707849925 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6180005ec0 0x7f61800db8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:31.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.427+0000 7f618e400640 1 -- 172.21.15.131:0/3707849925 >> 172.21.15.131:0/3707849925 conn(0x7f6180009fe0 msgr2=0x7f6180037280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:31.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.427+0000 7f618e400640 1 -- 172.21.15.131:0/3707849925 shutdown_connections 2024-05-26T22:48:31.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.428+0000 7f618e400640 1 -- 172.21.15.131:0/3707849925 wait complete. 2024-05-26T22:48:31.443 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.0 2024-05-26T22:48:31.443 DEBUG:teuthology.parallel:result is None 2024-05-26T22:48:31.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.454+0000 7fe415800640 1 -- 172.21.15.131:0/3978024872 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fe4100694e0 con 0x7fe4100b3430 2024-05-26T22:48:31.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.455+0000 7fe407e00640 1 -- 172.21.15.131:0/3978024872 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe40008b470 con 0x7fe4100b3430 2024-05-26T22:48:31.455 INFO:teuthology.orchestra.run.smithi131.stdout:55834574854 2024-05-26T22:48:31.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.458+0000 7fe415800640 1 -- 172.21.15.131:0/3978024872 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fe3f006be80 msgr2=0x7fe3f006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:31.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.458+0000 7fe415800640 1 --2- 172.21.15.131:0/3978024872 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fe3f006be80 0x7fe3f006e340 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe410161930 tx=0x7fe3fc008040 comp rx=0 tx=0).stop 2024-05-26T22:48:31.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.458+0000 7fe415800640 1 -- 172.21.15.131:0/3978024872 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe4100b3430 msgr2=0x7fe4100cab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:31.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.458+0000 7fe415800640 1 --2- 172.21.15.131:0/3978024872 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe4100b3430 0x7fe4100cab40 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe400011f90 tx=0x7fe4000046c0 comp rx=0 tx=0).stop 2024-05-26T22:48:31.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.459+0000 7fe415800640 1 -- 172.21.15.131:0/3978024872 shutdown_connections 2024-05-26T22:48:31.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.459+0000 7fe415800640 1 --2- 172.21.15.131:0/3978024872 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe4100cf7c0 0x7fe410157540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:31.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.459+0000 7fe415800640 1 --2- 172.21.15.131:0/3978024872 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fe3f006be80 0x7fe3f006e340 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:31.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.459+0000 7fe415800640 1 --2- 172.21.15.131:0/3978024872 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe4100cc240 0x7fe4100c7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:31.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.459+0000 7fe415800640 1 --2- 172.21.15.131:0/3978024872 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe4100b3430 0x7fe4100cab40 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:31.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.459+0000 7fe415800640 1 -- 172.21.15.131:0/3978024872 >> 172.21.15.131:0/3978024872 conn(0x7fe41009d4d0 msgr2=0x7fe41006d5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:31.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.459+0000 7fe415800640 1 -- 172.21.15.131:0/3978024872 shutdown_connections 2024-05-26T22:48:31.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.459+0000 7fe415800640 1 -- 172.21.15.131:0/3978024872 wait complete. 2024-05-26T22:48:31.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.462+0000 7f14dbe00640 1 -- 172.21.15.131:0/1138845853 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f14dc0694e0 con 0x7f14dc06d500 2024-05-26T22:48:31.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.463+0000 7f14d2a00640 1 -- 172.21.15.131:0/1138845853 <== mon.2 v2:172.21.15.187:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f14c807d4b0 con 0x7f14dc06d500 2024-05-26T22:48:31.463 INFO:teuthology.orchestra.run.smithi131.stdout:55834574855 2024-05-26T22:48:31.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.467+0000 7f14dbe00640 1 -- 172.21.15.131:0/1138845853 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f14bc06be80 msgr2=0x7f14bc06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:31.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.467+0000 7f14dbe00640 1 --2- 172.21.15.131:0/1138845853 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f14bc06be80 0x7f14bc06e340 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f14dc0b1920 tx=0x7f14cc000ed0 comp rx=0 tx=0).stop 2024-05-26T22:48:31.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.467+0000 7f14dbe00640 1 -- 172.21.15.131:0/1138845853 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f14dc06d500 msgr2=0x7f14dc1458a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:31.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.467+0000 7f14dbe00640 1 --2- 172.21.15.131:0/1138845853 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f14dc06d500 0x7f14dc1458a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f14c8031000 tx=0x7f14c80046c0 comp rx=0 tx=0).stop 2024-05-26T22:48:31.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.467+0000 7f14dbe00640 1 -- 172.21.15.131:0/1138845853 shutdown_connections 2024-05-26T22:48:31.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.467+0000 7f14dbe00640 1 --2- 172.21.15.131:0/1138845853 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f14bc06be80 0x7f14bc06e340 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:31.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.467+0000 7f14dbe00640 1 --2- 172.21.15.131:0/1138845853 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f14dc15beb0 0x7f14dc15c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:31.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.467+0000 7f14dbe00640 1 --2- 172.21.15.131:0/1138845853 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f14dc0b3bb0 0x7f14dc152bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:31.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.467+0000 7f14dbe00640 1 --2- 172.21.15.131:0/1138845853 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f14dc06d500 0x7f14dc1458a0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:31.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.467+0000 7f14dbe00640 1 -- 172.21.15.131:0/1138845853 >> 172.21.15.131:0/1138845853 conn(0x7f14dc0b1c60 msgr2=0x7f14dc15b0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:31.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.467+0000 7f14dbe00640 1 -- 172.21.15.131:0/1138845853 shutdown_connections 2024-05-26T22:48:31.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.467+0000 7f14dbe00640 1 -- 172.21.15.131:0/1138845853 wait complete. 2024-05-26T22:48:31.481 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.6 2024-05-26T22:48:31.481 DEBUG:teuthology.parallel:result is None 2024-05-26T22:48:31.486 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.5 2024-05-26T22:48:31.486 DEBUG:teuthology.parallel:result is None 2024-05-26T22:48:31.487 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-26T22:48:31.487 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-26T22:48:31.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.724+0000 7f3a2b600640 1 Processor -- start 2024-05-26T22:48:31.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.724+0000 7f3a2b600640 1 -- start start 2024-05-26T22:48:31.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.726+0000 7f3a2b600640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f3a2406ee20 0x7f3a240b3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:31.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.726+0000 7f3a2b600640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f3a240ad250 0x7f3a240af790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:31.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.726+0000 7f3a2b600640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a2415cb20 con 0x7f3a2406f200 2024-05-26T22:48:31.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.726+0000 7f3a2b600640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f3a2415ce00 con 0x7f3a240ad250 2024-05-26T22:48:31.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.726+0000 7f3a2b600640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f3a2415d0e0 con 0x7f3a2406ee20 2024-05-26T22:48:31.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.726+0000 7f3a23e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f3a240ad250 0x7f3a240af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:31.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.726+0000 7f3a23e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f3a240ad250 0x7f3a240af790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54374/0 (socket says 172.21.15.131:54374) 2024-05-26T22:48:31.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.726+0000 7f3a23e00640 1 -- 172.21.15.131:0/3171890982 learned_addr learned my addr 172.21.15.131:0/3171890982 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:31.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.726+0000 7f3a28c00640 1 --2- 172.21.15.131:0/3171890982 >> v2:172.21.15.187:3300/0 conn(0x7f3a2406ee20 0x7f3a240b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:31.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.726+0000 7f3a23e00640 1 -- 172.21.15.131:0/3171890982 >> v2:172.21.15.187:3300/0 conn(0x7f3a2406ee20 msgr2=0x7f3a240b3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:31.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.726+0000 7f3a23e00640 1 --2- 172.21.15.131:0/3171890982 >> v2:172.21.15.187:3300/0 conn(0x7f3a2406ee20 0x7f3a240b3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:31.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.727+0000 7f3a23e00640 1 -- 172.21.15.131:0/3171890982 >> v1:172.21.15.187:6789/0 conn(0x7f3a2406f200 legacy=0x7f3a241562c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:31.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.727+0000 7f3a23e00640 1 -- 172.21.15.131:0/3171890982 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a2415d3c0 con 0x7f3a240ad250 2024-05-26T22:48:31.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.727+0000 7f3a28c00640 1 --2- 172.21.15.131:0/3171890982 >> v2:172.21.15.187:3300/0 conn(0x7f3a2406ee20 0x7f3a240b3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:48:31.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.727+0000 7f3a23e00640 1 --2- 172.21.15.131:0/3171890982 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a240ad250 0x7f3a240af790 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3a24071090 tx=0x7f3a1002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7ff36fce8ecb9c5 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:31.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.728+0000 7f3a23400640 1 -- 172.21.15.131:0/3171890982 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a10037070 con 0x7f3a240ad250 2024-05-26T22:48:31.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.728+0000 7f3a23400640 1 -- 172.21.15.131:0/3171890982 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a10032440 con 0x7f3a240ad250 2024-05-26T22:48:31.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.728+0000 7f3a23400640 1 -- 172.21.15.131:0/3171890982 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a10032810 con 0x7f3a240ad250 2024-05-26T22:48:31.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.728+0000 7f3a2b600640 1 -- 172.21.15.131:0/3171890982 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a240ad250 msgr2=0x7f3a240af790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:31.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.728+0000 7f3a2b600640 1 --2- 172.21.15.131:0/3171890982 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a240ad250 0x7f3a240af790 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3a24071090 tx=0x7f3a1002a1a0 comp rx=0 tx=0).stop 2024-05-26T22:48:31.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.729+0000 7f3a2b600640 1 -- 172.21.15.131:0/3171890982 shutdown_connections 2024-05-26T22:48:31.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.729+0000 7f3a2b600640 1 --2- 172.21.15.131:0/3171890982 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a240ad250 0x7f3a240af790 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:31.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.729+0000 7f3a2b600640 1 --2- 172.21.15.131:0/3171890982 >> v2:172.21.15.187:3300/0 conn(0x7f3a2406ee20 0x7f3a240b3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:31.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.729+0000 7f3a2b600640 1 -- 172.21.15.131:0/3171890982 >> 172.21.15.131:0/3171890982 conn(0x7f3a240b1e90 msgr2=0x7f3a240b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:31.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.729+0000 7f3a2b600640 1 -- 172.21.15.131:0/3171890982 shutdown_connections 2024-05-26T22:48:31.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.729+0000 7f3a2b600640 1 -- 172.21.15.131:0/3171890982 wait complete. 2024-05-26T22:48:31.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.730+0000 7f3a2b600640 1 Processor -- start 2024-05-26T22:48:31.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.730+0000 7f3a2b600640 1 -- start start 2024-05-26T22:48:31.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.731+0000 7f3a2b600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a2406ee20 0x7f3a241f8940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:31.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.731+0000 7f3a2b600640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a2406f200 0x7f3a240998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:31.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.731+0000 7f3a2b600640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a240ad250 0x7f3a2409d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:31.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.731+0000 7f3a2b600640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f3a2415c900 con 0x7f3a2406ee20 2024-05-26T22:48:31.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.731+0000 7f3a2b600640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3a2415c780 con 0x7f3a240ad250 2024-05-26T22:48:31.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.731+0000 7f3a2b600640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3a241569d0 con 0x7f3a2406f200 2024-05-26T22:48:31.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.731+0000 7f3a28c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a2406ee20 0x7f3a241f8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:31.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.732+0000 7f3a28c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a2406ee20 0x7f3a241f8940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54390/0 (socket says 172.21.15.131:54390) 2024-05-26T22:48:31.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.732+0000 7f3a28c00640 1 -- 172.21.15.131:0/857294916 learned_addr learned my addr 172.21.15.131:0/857294916 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:31.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.732+0000 7f3a23e00640 1 --2- 172.21.15.131:0/857294916 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a2406f200 0x7f3a240998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:31.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.732+0000 7f3a28c00640 1 -- 172.21.15.131:0/857294916 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a2406f200 msgr2=0x7f3a240998e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:31.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.732+0000 7f3a28c00640 1 --2- 172.21.15.131:0/857294916 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a2406f200 0x7f3a240998e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:31.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.732+0000 7f3a28c00640 1 -- 172.21.15.131:0/857294916 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a240ad250 msgr2=0x7f3a2409d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:31.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.732+0000 7f3a29600640 1 --2- 172.21.15.131:0/857294916 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a240ad250 0x7f3a2409d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:31.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.732+0000 7f3a28c00640 1 --2- 172.21.15.131:0/857294916 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a240ad250 0x7f3a2409d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:31.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.732+0000 7f3a28c00640 1 -- 172.21.15.131:0/857294916 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a241f8e80 con 0x7f3a2406ee20 2024-05-26T22:48:31.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.732+0000 7f3a29600640 1 --2- 172.21.15.131:0/857294916 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a240ad250 0x7f3a2409d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:48:31.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.732+0000 7f3a28c00640 1 --2- 172.21.15.131:0/857294916 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a2406ee20 0x7f3a241f8940 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f3a1800c7f0 tx=0x7f3a1800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:31.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.732+0000 7f3a20c00640 1 -- 172.21.15.131:0/857294916 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a18015070 con 0x7f3a2406ee20 2024-05-26T22:48:31.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.732+0000 7f3a20c00640 1 -- 172.21.15.131:0/857294916 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a18007a30 con 0x7f3a2406ee20 2024-05-26T22:48:31.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.733+0000 7f3a2b600640 1 -- 172.21.15.131:0/857294916 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a2409c7f0 con 0x7f3a2406ee20 2024-05-26T22:48:31.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.733+0000 7f3a20c00640 1 -- 172.21.15.131:0/857294916 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a18007e00 con 0x7f3a2406ee20 2024-05-26T22:48:31.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.733+0000 7f3a2b600640 1 -- 172.21.15.131:0/857294916 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a2409cdb0 con 0x7f3a2406ee20 2024-05-26T22:48:31.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.734+0000 7f3a2b600640 1 -- 172.21.15.131:0/857294916 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a240694e0 con 0x7f3a2406ee20 2024-05-26T22:48:31.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.735+0000 7f3a20c00640 1 -- 172.21.15.131:0/857294916 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3a18018070 con 0x7f3a2406ee20 2024-05-26T22:48:31.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.736+0000 7f3a20c00640 1 --2- 172.21.15.131:0/857294916 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f3a0406be60 0x7f3a0406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:31.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.736+0000 7f3a20c00640 1 -- 172.21.15.131:0/857294916 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f3a1808ade0 con 0x7f3a2406ee20 2024-05-26T22:48:31.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.736+0000 7f3a23e00640 1 --2- 172.21.15.131:0/857294916 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f3a0406be60 0x7f3a0406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:31.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.736+0000 7f3a23e00640 1 --2- 172.21.15.131:0/857294916 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f3a0406be60 0x7f3a0406e320 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3a10034000 tx=0x7f3a1002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:31.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.740+0000 7f3a20c00640 1 -- 172.21.15.131:0/857294916 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f3a241f8e80 con 0x7f3a2406ee20 2024-05-26T22:48:31.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.919+0000 7f3a2b600640 1 -- 172.21.15.131:0/857294916 --> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3a240b1c00 con 0x7f3a0406be60 2024-05-26T22:48:31.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.929+0000 7f3a20c00640 1 -- 172.21.15.131:0/857294916 <== mgr.4104 v2:172.21.15.131:6832/3484832947 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251836 (secure 0 0 0) 0x7f3a240b1c00 con 0x7f3a0406be60 2024-05-26T22:48:31.933 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:48:31.934 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-05-26T22:48:31.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.934+0000 7f3a2b600640 1 -- 172.21.15.131:0/857294916 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f3a0406be60 msgr2=0x7f3a0406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:31.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.934+0000 7f3a2b600640 1 --2- 172.21.15.131:0/857294916 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f3a0406be60 0x7f3a0406e320 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3a10034000 tx=0x7f3a1002e000 comp rx=0 tx=0).stop 2024-05-26T22:48:31.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.934+0000 7f3a2b600640 1 -- 172.21.15.131:0/857294916 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a2406ee20 msgr2=0x7f3a241f8940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:31.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.934+0000 7f3a2b600640 1 --2- 172.21.15.131:0/857294916 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a2406ee20 0x7f3a241f8940 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f3a1800c7f0 tx=0x7f3a1800cbf0 comp rx=0 tx=0).stop 2024-05-26T22:48:31.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.934+0000 7f3a2b600640 1 -- 172.21.15.131:0/857294916 shutdown_connections 2024-05-26T22:48:31.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.934+0000 7f3a2b600640 1 --2- 172.21.15.131:0/857294916 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a2406f200 0x7f3a240998e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:31.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.934+0000 7f3a2b600640 1 --2- 172.21.15.131:0/857294916 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f3a0406be60 0x7f3a0406e320 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:31.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.934+0000 7f3a2b600640 1 --2- 172.21.15.131:0/857294916 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a240ad250 0x7f3a2409d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:31.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.934+0000 7f3a2b600640 1 --2- 172.21.15.131:0/857294916 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a2406ee20 0x7f3a241f8940 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:31.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.934+0000 7f3a2b600640 1 -- 172.21.15.131:0/857294916 >> 172.21.15.131:0/857294916 conn(0x7f3a240b1e90 msgr2=0x7f3a2406e230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:31.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.935+0000 7f3a2b600640 1 -- 172.21.15.131:0/857294916 shutdown_connections 2024-05-26T22:48:31.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:31.935+0000 7f3a2b600640 1 -- 172.21.15.131:0/857294916 wait complete. 2024-05-26T22:48:31.953 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-05-26T22:48:30.385857+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218992,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755728,"statfs":{"total":773094113280,"available":772869865472,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.597430"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.509427+0000","last_change":"2024-05-26T22:48:21.831991+0000","last_active":"2024-05-26T22:48:23.509427+0000","last_peered":"2024-05-26T22:48:23.509427+0000","last_clean":"2024-05-26T22:48:23.509427+0000","last_became_active":"2024-05-26T22:48:21.831657+0000","last_became_peered":"2024-05-26T22:48:21.831657+0000","last_unstale":"2024-05-26T22:48:23.509427+0000","last_undegraded":"2024-05-26T22:48:23.509427+0000","last_fullsized":"2024-05-26T22:48:23.509427+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:30:45.689344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.822991+0000","last_change":"2024-05-26T22:48:21.813275+0000","last_active":"2024-05-26T22:48:22.822991+0000","last_peered":"2024-05-26T22:48:22.822991+0000","last_clean":"2024-05-26T22:48:22.822991+0000","last_became_active":"2024-05-26T22:48:21.812895+0000","last_became_peered":"2024-05-26T22:48:21.812895+0000","last_unstale":"2024-05-26T22:48:22.822991+0000","last_undegraded":"2024-05-26T22:48:22.822991+0000","last_fullsized":"2024-05-26T22:48:22.822991+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:59:43.907681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.809083+0000","last_change":"2024-05-26T22:48:21.805790+0000","last_active":"2024-05-26T22:48:22.809083+0000","last_peered":"2024-05-26T22:48:22.809083+0000","last_clean":"2024-05-26T22:48:22.809083+0000","last_became_active":"2024-05-26T22:48:21.805318+0000","last_became_peered":"2024-05-26T22:48:21.805318+0000","last_unstale":"2024-05-26T22:48:22.809083+0000","last_undegraded":"2024-05-26T22:48:22.809083+0000","last_fullsized":"2024-05-26T22:48:22.809083+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:43:21.391197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.506425+0000","last_change":"2024-05-26T22:48:21.831837+0000","last_active":"2024-05-26T22:48:23.506425+0000","last_peered":"2024-05-26T22:48:23.506425+0000","last_clean":"2024-05-26T22:48:23.506425+0000","last_became_active":"2024-05-26T22:48:21.829461+0000","last_became_peered":"2024-05-26T22:48:21.829461+0000","last_unstale":"2024-05-26T22:48:23.506425+0000","last_undegraded":"2024-05-26T22:48:23.506425+0000","last_fullsized":"2024-05-26T22:48:23.506425+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:56:39.208963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.508007+0000","last_change":"2024-05-26T22:48:21.821926+0000","last_active":"2024-05-26T22:48:23.508007+0000","last_peered":"2024-05-26T22:48:23.508007+0000","last_clean":"2024-05-26T22:48:23.508007+0000","last_became_active":"2024-05-26T22:48:21.821594+0000","last_became_peered":"2024-05-26T22:48:21.821594+0000","last_unstale":"2024-05-26T22:48:23.508007+0000","last_undegraded":"2024-05-26T22:48:23.508007+0000","last_fullsized":"2024-05-26T22:48:23.508007+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:16:22.847663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.506340+0000","last_change":"2024-05-26T22:48:21.823954+0000","last_active":"2024-05-26T22:48:23.506340+0000","last_peered":"2024-05-26T22:48:23.506340+0000","last_clean":"2024-05-26T22:48:23.506340+0000","last_became_active":"2024-05-26T22:48:21.821405+0000","last_became_peered":"2024-05-26T22:48:21.821405+0000","last_unstale":"2024-05-26T22:48:23.506340+0000","last_undegraded":"2024-05-26T22:48:23.506340+0000","last_fullsized":"2024-05-26T22:48:23.506340+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:33:39.923721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.506938+0000","last_change":"2024-05-26T22:48:21.821415+0000","last_active":"2024-05-26T22:48:23.506938+0000","last_peered":"2024-05-26T22:48:23.506938+0000","last_clean":"2024-05-26T22:48:23.506938+0000","last_became_active":"2024-05-26T22:48:21.821092+0000","last_became_peered":"2024-05-26T22:48:21.821092+0000","last_unstale":"2024-05-26T22:48:23.506938+0000","last_undegraded":"2024-05-26T22:48:23.506938+0000","last_fullsized":"2024-05-26T22:48:23.506938+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:56:39.208963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.811969+0000","last_change":"2024-05-26T22:48:21.847352+0000","last_active":"2024-05-26T22:48:22.811969+0000","last_peered":"2024-05-26T22:48:22.811969+0000","last_clean":"2024-05-26T22:48:22.811969+0000","last_became_active":"2024-05-26T22:48:21.846965+0000","last_became_peered":"2024-05-26T22:48:21.846965+0000","last_unstale":"2024-05-26T22:48:22.811969+0000","last_undegraded":"2024-05-26T22:48:22.811969+0000","last_fullsized":"2024-05-26T22:48:22.811969+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T23:40:55.485829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.509545+0000","last_change":"2024-05-26T22:48:21.847421+0000","last_active":"2024-05-26T22:48:23.509545+0000","last_peered":"2024-05-26T22:48:23.509545+0000","last_clean":"2024-05-26T22:48:23.509545+0000","last_became_active":"2024-05-26T22:48:21.846702+0000","last_became_peered":"2024-05-26T22:48:21.846702+0000","last_unstale":"2024-05-26T22:48:23.509545+0000","last_undegraded":"2024-05-26T22:48:23.509545+0000","last_fullsized":"2024-05-26T22:48:23.509545+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:27:35.210564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.810240+0000","last_change":"2024-05-26T22:48:21.831244+0000","last_active":"2024-05-26T22:48:22.810240+0000","last_peered":"2024-05-26T22:48:22.810240+0000","last_clean":"2024-05-26T22:48:22.810240+0000","last_became_active":"2024-05-26T22:48:21.830949+0000","last_became_peered":"2024-05-26T22:48:21.830949+0000","last_unstale":"2024-05-26T22:48:22.810240+0000","last_undegraded":"2024-05-26T22:48:22.810240+0000","last_fullsized":"2024-05-26T22:48:22.810240+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:59:43.907681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.810220+0000","last_change":"2024-05-26T22:48:21.836749+0000","last_active":"2024-05-26T22:48:22.810220+0000","last_peered":"2024-05-26T22:48:22.810220+0000","last_clean":"2024-05-26T22:48:22.810220+0000","last_became_active":"2024-05-26T22:48:21.836431+0000","last_became_peered":"2024-05-26T22:48:21.836431+0000","last_unstale":"2024-05-26T22:48:22.810220+0000","last_undegraded":"2024-05-26T22:48:22.810220+0000","last_fullsized":"2024-05-26T22:48:22.810220+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:38:00.344955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.505172+0000","last_change":"2024-05-26T22:48:21.819214+0000","last_active":"2024-05-26T22:48:23.505172+0000","last_peered":"2024-05-26T22:48:23.505172+0000","last_clean":"2024-05-26T22:48:23.505172+0000","last_became_active":"2024-05-26T22:48:21.818837+0000","last_became_peered":"2024-05-26T22:48:21.818837+0000","last_unstale":"2024-05-26T22:48:23.505172+0000","last_undegraded":"2024-05-26T22:48:23.505172+0000","last_fullsized":"2024-05-26T22:48:23.505172+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:42:32.978709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.812261+0000","last_change":"2024-05-26T22:48:21.830442+0000","last_active":"2024-05-26T22:48:22.812261+0000","last_peered":"2024-05-26T22:48:22.812261+0000","last_clean":"2024-05-26T22:48:22.812261+0000","last_became_active":"2024-05-26T22:48:21.830132+0000","last_became_peered":"2024-05-26T22:48:21.830132+0000","last_unstale":"2024-05-26T22:48:22.812261+0000","last_undegraded":"2024-05-26T22:48:22.812261+0000","last_fullsized":"2024-05-26T22:48:22.812261+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:43:21.391197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.808630+0000","last_change":"2024-05-26T22:48:21.818032+0000","last_active":"2024-05-26T22:48:22.808630+0000","last_peered":"2024-05-26T22:48:22.808630+0000","last_clean":"2024-05-26T22:48:22.808630+0000","last_became_active":"2024-05-26T22:48:21.817717+0000","last_became_peered":"2024-05-26T22:48:21.817717+0000","last_unstale":"2024-05-26T22:48:22.808630+0000","last_undegraded":"2024-05-26T22:48:22.808630+0000","last_fullsized":"2024-05-26T22:48:22.808630+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:27:34.959431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.822672+0000","last_change":"2024-05-26T22:48:21.823839+0000","last_active":"2024-05-26T22:48:22.822672+0000","last_peered":"2024-05-26T22:48:22.822672+0000","last_clean":"2024-05-26T22:48:22.822672+0000","last_became_active":"2024-05-26T22:48:21.823522+0000","last_became_peered":"2024-05-26T22:48:21.823522+0000","last_unstale":"2024-05-26T22:48:22.822672+0000","last_undegraded":"2024-05-26T22:48:22.822672+0000","last_fullsized":"2024-05-26T22:48:22.822672+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:52:25.073281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.504208+0000","last_change":"2024-05-26T22:48:21.819432+0000","last_active":"2024-05-26T22:48:23.504208+0000","last_peered":"2024-05-26T22:48:23.504208+0000","last_clean":"2024-05-26T22:48:23.504208+0000","last_became_active":"2024-05-26T22:48:21.819139+0000","last_became_peered":"2024-05-26T22:48:21.819139+0000","last_unstale":"2024-05-26T22:48:23.504208+0000","last_undegraded":"2024-05-26T22:48:23.504208+0000","last_fullsized":"2024-05-26T22:48:23.504208+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:21:52.108732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.507463+0000","last_change":"2024-05-26T22:48:21.842657+0000","last_active":"2024-05-26T22:48:23.507463+0000","last_peered":"2024-05-26T22:48:23.507463+0000","last_clean":"2024-05-26T22:48:23.507463+0000","last_became_active":"2024-05-26T22:48:21.842312+0000","last_became_peered":"2024-05-26T22:48:21.842312+0000","last_unstale":"2024-05-26T22:48:23.507463+0000","last_undegraded":"2024-05-26T22:48:23.507463+0000","last_fullsized":"2024-05-26T22:48:23.507463+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:49:40.791149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.503319+0000","last_change":"2024-05-26T22:48:21.812988+0000","last_active":"2024-05-26T22:48:23.503319+0000","last_peered":"2024-05-26T22:48:23.503319+0000","last_clean":"2024-05-26T22:48:23.503319+0000","last_became_active":"2024-05-26T22:48:21.812669+0000","last_became_peered":"2024-05-26T22:48:21.812669+0000","last_unstale":"2024-05-26T22:48:23.503319+0000","last_undegraded":"2024-05-26T22:48:23.503319+0000","last_fullsized":"2024-05-26T22:48:23.503319+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:16:02.119875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.509267+0000","last_change":"2024-05-26T22:48:21.830493+0000","last_active":"2024-05-26T22:48:23.509267+0000","last_peered":"2024-05-26T22:48:23.509267+0000","last_clean":"2024-05-26T22:48:23.509267+0000","last_became_active":"2024-05-26T22:48:21.830116+0000","last_became_peered":"2024-05-26T22:48:21.830116+0000","last_unstale":"2024-05-26T22:48:23.509267+0000","last_undegraded":"2024-05-26T22:48:23.509267+0000","last_fullsized":"2024-05-26T22:48:23.509267+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:45:06.922028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.809824+0000","last_change":"2024-05-26T22:48:21.829461+0000","last_active":"2024-05-26T22:48:22.809824+0000","last_peered":"2024-05-26T22:48:22.809824+0000","last_clean":"2024-05-26T22:48:22.809824+0000","last_became_active":"2024-05-26T22:48:21.829132+0000","last_became_peered":"2024-05-26T22:48:21.829132+0000","last_unstale":"2024-05-26T22:48:22.809824+0000","last_undegraded":"2024-05-26T22:48:22.809824+0000","last_fullsized":"2024-05-26T22:48:22.809824+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:52:25.073281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.809106+0000","last_change":"2024-05-26T22:48:21.818888+0000","last_active":"2024-05-26T22:48:22.809106+0000","last_peered":"2024-05-26T22:48:22.809106+0000","last_clean":"2024-05-26T22:48:22.809106+0000","last_became_active":"2024-05-26T22:48:21.818639+0000","last_became_peered":"2024-05-26T22:48:21.818639+0000","last_unstale":"2024-05-26T22:48:22.809106+0000","last_undegraded":"2024-05-26T22:48:22.809106+0000","last_fullsized":"2024-05-26T22:48:22.809106+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:38:00.344955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.506778+0000","last_change":"2024-05-26T22:48:21.842585+0000","last_active":"2024-05-26T22:48:23.506778+0000","last_peered":"2024-05-26T22:48:23.506778+0000","last_clean":"2024-05-26T22:48:23.506778+0000","last_became_active":"2024-05-26T22:48:21.842302+0000","last_became_peered":"2024-05-26T22:48:21.842302+0000","last_unstale":"2024-05-26T22:48:23.506778+0000","last_undegraded":"2024-05-26T22:48:23.506778+0000","last_fullsized":"2024-05-26T22:48:23.506778+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:08:10.330236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.811982+0000","last_change":"2024-05-26T22:48:21.830936+0000","last_active":"2024-05-26T22:48:22.811982+0000","last_peered":"2024-05-26T22:48:22.811982+0000","last_clean":"2024-05-26T22:48:22.811982+0000","last_became_active":"2024-05-26T22:48:21.830551+0000","last_became_peered":"2024-05-26T22:48:21.830551+0000","last_unstale":"2024-05-26T22:48:22.811982+0000","last_undegraded":"2024-05-26T22:48:22.811982+0000","last_fullsized":"2024-05-26T22:48:22.811982+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:27:34.959431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.508425+0000","last_change":"2024-05-26T22:48:21.837001+0000","last_active":"2024-05-26T22:48:23.508425+0000","last_peered":"2024-05-26T22:48:23.508425+0000","last_clean":"2024-05-26T22:48:23.508425+0000","last_became_active":"2024-05-26T22:48:21.836749+0000","last_became_peered":"2024-05-26T22:48:21.836749+0000","last_unstale":"2024-05-26T22:48:23.508425+0000","last_undegraded":"2024-05-26T22:48:23.508425+0000","last_fullsized":"2024-05-26T22:48:23.508425+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:30:45.689344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.812885+0000","last_change":"2024-05-26T22:48:21.834986+0000","last_active":"2024-05-26T22:48:22.812885+0000","last_peered":"2024-05-26T22:48:22.812885+0000","last_clean":"2024-05-26T22:48:22.812885+0000","last_became_active":"2024-05-26T22:48:21.834572+0000","last_became_peered":"2024-05-26T22:48:21.834572+0000","last_unstale":"2024-05-26T22:48:22.812885+0000","last_undegraded":"2024-05-26T22:48:22.812885+0000","last_fullsized":"2024-05-26T22:48:22.812885+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:33:39.923721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.810409+0000","last_change":"2024-05-26T22:48:21.824957+0000","last_active":"2024-05-26T22:48:22.810409+0000","last_peered":"2024-05-26T22:48:22.810409+0000","last_clean":"2024-05-26T22:48:22.810409+0000","last_became_active":"2024-05-26T22:48:21.822871+0000","last_became_peered":"2024-05-26T22:48:21.822871+0000","last_unstale":"2024-05-26T22:48:22.810409+0000","last_undegraded":"2024-05-26T22:48:22.810409+0000","last_fullsized":"2024-05-26T22:48:22.810409+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:59:10.543916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.822941+0000","last_change":"2024-05-26T22:48:21.827100+0000","last_active":"2024-05-26T22:48:22.822941+0000","last_peered":"2024-05-26T22:48:22.822941+0000","last_clean":"2024-05-26T22:48:22.822941+0000","last_became_active":"2024-05-26T22:48:21.826810+0000","last_became_peered":"2024-05-26T22:48:21.826810+0000","last_unstale":"2024-05-26T22:48:22.822941+0000","last_undegraded":"2024-05-26T22:48:22.822941+0000","last_fullsized":"2024-05-26T22:48:22.822941+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:38:00.344955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.813201+0000","last_change":"2024-05-26T22:48:21.836236+0000","last_active":"2024-05-26T22:48:22.813201+0000","last_peered":"2024-05-26T22:48:22.813201+0000","last_clean":"2024-05-26T22:48:22.813201+0000","last_became_active":"2024-05-26T22:48:21.834609+0000","last_became_peered":"2024-05-26T22:48:21.834609+0000","last_unstale":"2024-05-26T22:48:22.813201+0000","last_undegraded":"2024-05-26T22:48:22.813201+0000","last_fullsized":"2024-05-26T22:48:22.813201+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:08:10.330236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.809226+0000","last_change":"2024-05-26T22:48:21.821191+0000","last_active":"2024-05-26T22:48:22.809226+0000","last_peered":"2024-05-26T22:48:22.809226+0000","last_clean":"2024-05-26T22:48:22.809226+0000","last_became_active":"2024-05-26T22:48:21.820731+0000","last_became_peered":"2024-05-26T22:48:21.820731+0000","last_unstale":"2024-05-26T22:48:22.809226+0000","last_undegraded":"2024-05-26T22:48:22.809226+0000","last_fullsized":"2024-05-26T22:48:22.809226+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:59:43.907681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.507246+0000","last_change":"2024-05-26T22:48:21.836074+0000","last_active":"2024-05-26T22:48:23.507246+0000","last_peered":"2024-05-26T22:48:23.507246+0000","last_clean":"2024-05-26T22:48:23.507246+0000","last_became_active":"2024-05-26T22:48:21.831547+0000","last_became_peered":"2024-05-26T22:48:21.831547+0000","last_unstale":"2024-05-26T22:48:23.507246+0000","last_undegraded":"2024-05-26T22:48:23.507246+0000","last_fullsized":"2024-05-26T22:48:23.507246+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:34:58.598026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.503056+0000","last_change":"2024-05-26T22:48:21.815510+0000","last_active":"2024-05-26T22:48:23.503056+0000","last_peered":"2024-05-26T22:48:23.503056+0000","last_clean":"2024-05-26T22:48:23.503056+0000","last_became_active":"2024-05-26T22:48:21.815132+0000","last_became_peered":"2024-05-26T22:48:21.815132+0000","last_unstale":"2024-05-26T22:48:23.503056+0000","last_undegraded":"2024-05-26T22:48:23.503056+0000","last_fullsized":"2024-05-26T22:48:23.503056+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:56:00.075959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.809652+0000","last_change":"2024-05-26T22:48:21.811494+0000","last_active":"2024-05-26T22:48:22.809652+0000","last_peered":"2024-05-26T22:48:22.809652+0000","last_clean":"2024-05-26T22:48:22.809652+0000","last_became_active":"2024-05-26T22:48:21.809669+0000","last_became_peered":"2024-05-26T22:48:21.809669+0000","last_unstale":"2024-05-26T22:48:22.809652+0000","last_undegraded":"2024-05-26T22:48:22.809652+0000","last_fullsized":"2024-05-26T22:48:22.809652+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:27:34.959431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.504470+0000","last_change":"2024-05-26T22:48:21.824303+0000","last_active":"2024-05-26T22:48:23.504470+0000","last_peered":"2024-05-26T22:48:23.504470+0000","last_clean":"2024-05-26T22:48:23.504470+0000","last_became_active":"2024-05-26T22:48:21.823500+0000","last_became_peered":"2024-05-26T22:48:21.823500+0000","last_unstale":"2024-05-26T22:48:23.504470+0000","last_undegraded":"2024-05-26T22:48:23.504470+0000","last_fullsized":"2024-05-26T22:48:23.504470+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:16:02.119875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.812241+0000","last_change":"2024-05-26T22:48:20.802593+0000","last_active":"2024-05-26T22:48:22.812241+0000","last_peered":"2024-05-26T22:48:22.812241+0000","last_clean":"2024-05-26T22:48:22.812241+0000","last_became_active":"2024-05-26T22:48:20.802352+0000","last_became_peered":"2024-05-26T22:48:20.802352+0000","last_unstale":"2024-05-26T22:48:22.812241+0000","last_undegraded":"2024-05-26T22:48:22.812241+0000","last_fullsized":"2024-05-26T22:48:22.812241+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:39:51.267408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.809443+0000","last_change":"2024-05-26T22:48:21.832952+0000","last_active":"2024-05-26T22:48:22.809443+0000","last_peered":"2024-05-26T22:48:22.809443+0000","last_clean":"2024-05-26T22:48:22.809443+0000","last_became_active":"2024-05-26T22:48:21.832496+0000","last_became_peered":"2024-05-26T22:48:21.832496+0000","last_unstale":"2024-05-26T22:48:22.809443+0000","last_undegraded":"2024-05-26T22:48:22.809443+0000","last_fullsized":"2024-05-26T22:48:22.809443+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:59:10.543916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.810033+0000","last_change":"2024-05-26T22:48:20.789223+0000","last_active":"2024-05-26T22:48:22.810033+0000","last_peered":"2024-05-26T22:48:22.810033+0000","last_clean":"2024-05-26T22:48:22.810033+0000","last_became_active":"2024-05-26T22:48:20.787967+0000","last_became_peered":"2024-05-26T22:48:20.787967+0000","last_unstale":"2024-05-26T22:48:22.810033+0000","last_undegraded":"2024-05-26T22:48:22.810033+0000","last_fullsized":"2024-05-26T22:48:22.810033+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:08:01.698812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.812657+0000","last_change":"2024-05-26T22:48:20.784257+0000","last_active":"2024-05-26T22:48:22.812657+0000","last_peered":"2024-05-26T22:48:22.812657+0000","last_clean":"2024-05-26T22:48:22.812657+0000","last_became_active":"2024-05-26T22:48:20.783493+0000","last_became_peered":"2024-05-26T22:48:20.783493+0000","last_unstale":"2024-05-26T22:48:22.812657+0000","last_undegraded":"2024-05-26T22:48:22.812657+0000","last_fullsized":"2024-05-26T22:48:22.812657+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:30:48.024402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.813086+0000","last_change":"2024-05-26T22:48:21.834733+0000","last_active":"2024-05-26T22:48:22.813086+0000","last_peered":"2024-05-26T22:48:22.813086+0000","last_clean":"2024-05-26T22:48:22.813086+0000","last_became_active":"2024-05-26T22:48:21.834396+0000","last_became_peered":"2024-05-26T22:48:21.834396+0000","last_unstale":"2024-05-26T22:48:22.813086+0000","last_undegraded":"2024-05-26T22:48:22.813086+0000","last_fullsized":"2024-05-26T22:48:22.813086+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:56:39.208963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.506282+0000","last_change":"2024-05-26T22:48:20.798521+0000","last_active":"2024-05-26T22:48:23.506282+0000","last_peered":"2024-05-26T22:48:23.506282+0000","last_clean":"2024-05-26T22:48:23.506282+0000","last_became_active":"2024-05-26T22:48:20.798153+0000","last_became_peered":"2024-05-26T22:48:20.798153+0000","last_unstale":"2024-05-26T22:48:23.506282+0000","last_undegraded":"2024-05-26T22:48:23.506282+0000","last_fullsized":"2024-05-26T22:48:23.506282+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:10:49.991129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.823065+0000","last_change":"2024-05-26T22:48:21.832951+0000","last_active":"2024-05-26T22:48:22.823065+0000","last_peered":"2024-05-26T22:48:22.823065+0000","last_clean":"2024-05-26T22:48:22.823065+0000","last_became_active":"2024-05-26T22:48:21.831917+0000","last_became_peered":"2024-05-26T22:48:21.831917+0000","last_unstale":"2024-05-26T22:48:22.823065+0000","last_undegraded":"2024-05-26T22:48:22.823065+0000","last_fullsized":"2024-05-26T22:48:22.823065+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:59:10.543916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.810123+0000","last_change":"2024-05-26T22:48:20.801167+0000","last_active":"2024-05-26T22:48:22.810123+0000","last_peered":"2024-05-26T22:48:22.810123+0000","last_clean":"2024-05-26T22:48:22.810123+0000","last_became_active":"2024-05-26T22:48:20.800854+0000","last_became_peered":"2024-05-26T22:48:20.800854+0000","last_unstale":"2024-05-26T22:48:22.810123+0000","last_undegraded":"2024-05-26T22:48:22.810123+0000","last_fullsized":"2024-05-26T22:48:22.810123+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:39:51.267408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.509643+0000","last_change":"2024-05-26T22:48:21.841974+0000","last_active":"2024-05-26T22:48:23.509643+0000","last_peered":"2024-05-26T22:48:23.509643+0000","last_clean":"2024-05-26T22:48:23.509643+0000","last_became_active":"2024-05-26T22:48:21.840912+0000","last_became_peered":"2024-05-26T22:48:21.840912+0000","last_unstale":"2024-05-26T22:48:23.509643+0000","last_undegraded":"2024-05-26T22:48:23.509643+0000","last_fullsized":"2024-05-26T22:48:23.509643+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:27:35.210564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.509713+0000","last_change":"2024-05-26T22:48:20.798072+0000","last_active":"2024-05-26T22:48:23.509713+0000","last_peered":"2024-05-26T22:48:23.509713+0000","last_clean":"2024-05-26T22:48:23.509713+0000","last_became_active":"2024-05-26T22:48:20.797559+0000","last_became_peered":"2024-05-26T22:48:20.797559+0000","last_unstale":"2024-05-26T22:48:23.509713+0000","last_undegraded":"2024-05-26T22:48:23.509713+0000","last_fullsized":"2024-05-26T22:48:23.509713+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:52:24.060261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.809896+0000","last_change":"2024-05-26T22:48:21.838154+0000","last_active":"2024-05-26T22:48:22.809896+0000","last_peered":"2024-05-26T22:48:22.809896+0000","last_clean":"2024-05-26T22:48:22.809896+0000","last_became_active":"2024-05-26T22:48:21.837073+0000","last_became_peered":"2024-05-26T22:48:21.837073+0000","last_unstale":"2024-05-26T22:48:22.809896+0000","last_undegraded":"2024-05-26T22:48:22.809896+0000","last_fullsized":"2024-05-26T22:48:22.809896+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:56:39.208963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.822642+0000","last_change":"2024-05-26T22:48:20.796108+0000","last_active":"2024-05-26T22:48:22.822642+0000","last_peered":"2024-05-26T22:48:22.822642+0000","last_clean":"2024-05-26T22:48:22.822642+0000","last_became_active":"2024-05-26T22:48:20.795717+0000","last_became_peered":"2024-05-26T22:48:20.795717+0000","last_unstale":"2024-05-26T22:48:22.822642+0000","last_undegraded":"2024-05-26T22:48:22.822642+0000","last_fullsized":"2024-05-26T22:48:22.822642+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:39:51.267408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.505587+0000","last_change":"2024-05-26T22:48:21.823519+0000","last_active":"2024-05-26T22:48:23.505587+0000","last_peered":"2024-05-26T22:48:23.505587+0000","last_clean":"2024-05-26T22:48:23.505587+0000","last_became_active":"2024-05-26T22:48:21.823240+0000","last_became_peered":"2024-05-26T22:48:21.823240+0000","last_unstale":"2024-05-26T22:48:23.505587+0000","last_undegraded":"2024-05-26T22:48:23.505587+0000","last_fullsized":"2024-05-26T22:48:23.505587+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:33:51.651353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.809592+0000","last_change":"2024-05-26T22:48:20.792915+0000","last_active":"2024-05-26T22:48:22.809592+0000","last_peered":"2024-05-26T22:48:22.809592+0000","last_clean":"2024-05-26T22:48:22.809592+0000","last_became_active":"2024-05-26T22:48:20.792037+0000","last_became_peered":"2024-05-26T22:48:20.792037+0000","last_unstale":"2024-05-26T22:48:22.809592+0000","last_undegraded":"2024-05-26T22:48:22.809592+0000","last_fullsized":"2024-05-26T22:48:22.809592+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:11:29.128727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.508563+0000","last_change":"2024-05-26T22:48:21.839574+0000","last_active":"2024-05-26T22:48:23.508563+0000","last_peered":"2024-05-26T22:48:23.508563+0000","last_clean":"2024-05-26T22:48:23.508563+0000","last_became_active":"2024-05-26T22:48:21.834905+0000","last_became_peered":"2024-05-26T22:48:21.834905+0000","last_unstale":"2024-05-26T22:48:23.508563+0000","last_undegraded":"2024-05-26T22:48:23.508563+0000","last_fullsized":"2024-05-26T22:48:23.508563+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:34:58.598026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.809371+0000","last_change":"2024-05-26T22:48:20.792287+0000","last_active":"2024-05-26T22:48:22.809371+0000","last_peered":"2024-05-26T22:48:22.809371+0000","last_clean":"2024-05-26T22:48:22.809371+0000","last_became_active":"2024-05-26T22:48:20.791592+0000","last_became_peered":"2024-05-26T22:48:20.791592+0000","last_unstale":"2024-05-26T22:48:22.809371+0000","last_undegraded":"2024-05-26T22:48:22.809371+0000","last_fullsized":"2024-05-26T22:48:22.809371+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:30:48.024402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.808872+0000","last_change":"2024-05-26T22:48:21.824723+0000","last_active":"2024-05-26T22:48:22.808872+0000","last_peered":"2024-05-26T22:48:22.808872+0000","last_clean":"2024-05-26T22:48:22.808872+0000","last_became_active":"2024-05-26T22:48:21.824372+0000","last_became_peered":"2024-05-26T22:48:21.824372+0000","last_unstale":"2024-05-26T22:48:22.808872+0000","last_undegraded":"2024-05-26T22:48:22.808872+0000","last_fullsized":"2024-05-26T22:48:22.808872+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:52:25.073281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.507397+0000","last_change":"2024-05-26T22:48:20.802334+0000","last_active":"2024-05-26T22:48:23.507397+0000","last_peered":"2024-05-26T22:48:23.507397+0000","last_clean":"2024-05-26T22:48:23.507397+0000","last_became_active":"2024-05-26T22:48:20.802051+0000","last_became_peered":"2024-05-26T22:48:20.802051+0000","last_unstale":"2024-05-26T22:48:23.507397+0000","last_undegraded":"2024-05-26T22:48:23.507397+0000","last_fullsized":"2024-05-26T22:48:23.507397+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:55:59.062939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.508279+0000","last_change":"2024-05-26T22:48:21.830682+0000","last_active":"2024-05-26T22:48:23.508279+0000","last_peered":"2024-05-26T22:48:23.508279+0000","last_clean":"2024-05-26T22:48:23.508279+0000","last_became_active":"2024-05-26T22:48:21.830273+0000","last_became_peered":"2024-05-26T22:48:21.830273+0000","last_unstale":"2024-05-26T22:48:23.508279+0000","last_undegraded":"2024-05-26T22:48:23.508279+0000","last_fullsized":"2024-05-26T22:48:23.508279+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:15:00.790746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.809215+0000","last_change":"2024-05-26T22:48:20.791028+0000","last_active":"2024-05-26T22:48:22.809215+0000","last_peered":"2024-05-26T22:48:22.809215+0000","last_clean":"2024-05-26T22:48:22.809215+0000","last_became_active":"2024-05-26T22:48:20.790727+0000","last_became_peered":"2024-05-26T22:48:20.790727+0000","last_unstale":"2024-05-26T22:48:22.809215+0000","last_undegraded":"2024-05-26T22:48:22.809215+0000","last_fullsized":"2024-05-26T22:48:22.809215+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:20:41.886676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.809552+0000","last_change":"2024-05-26T22:48:21.809785+0000","last_active":"2024-05-26T22:48:22.809552+0000","last_peered":"2024-05-26T22:48:22.809552+0000","last_clean":"2024-05-26T22:48:22.809552+0000","last_became_active":"2024-05-26T22:48:21.809460+0000","last_became_peered":"2024-05-26T22:48:21.809460+0000","last_unstale":"2024-05-26T22:48:22.809552+0000","last_undegraded":"2024-05-26T22:48:22.809552+0000","last_fullsized":"2024-05-26T22:48:22.809552+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T23:40:55.485829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.502702+0000","last_change":"2024-05-26T22:48:20.788462+0000","last_active":"2024-05-26T22:48:23.502702+0000","last_peered":"2024-05-26T22:48:23.502702+0000","last_clean":"2024-05-26T22:48:23.502702+0000","last_became_active":"2024-05-26T22:48:20.787714+0000","last_became_peered":"2024-05-26T22:48:20.787714+0000","last_unstale":"2024-05-26T22:48:23.502702+0000","last_undegraded":"2024-05-26T22:48:23.502702+0000","last_fullsized":"2024-05-26T22:48:23.502702+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:08:01.698812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.507359+0000","last_change":"2024-05-26T22:48:21.823217+0000","last_active":"2024-05-26T22:48:23.507359+0000","last_peered":"2024-05-26T22:48:23.507359+0000","last_clean":"2024-05-26T22:48:23.507359+0000","last_became_active":"2024-05-26T22:48:21.822856+0000","last_became_peered":"2024-05-26T22:48:21.822856+0000","last_unstale":"2024-05-26T22:48:23.507359+0000","last_undegraded":"2024-05-26T22:48:23.507359+0000","last_fullsized":"2024-05-26T22:48:23.507359+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:08:10.330236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.505497+0000","last_change":"2024-05-26T22:48:20.793786+0000","last_active":"2024-05-26T22:48:23.505497+0000","last_peered":"2024-05-26T22:48:23.505497+0000","last_clean":"2024-05-26T22:48:23.505497+0000","last_became_active":"2024-05-26T22:48:20.793201+0000","last_became_peered":"2024-05-26T22:48:20.793201+0000","last_unstale":"2024-05-26T22:48:23.505497+0000","last_undegraded":"2024-05-26T22:48:23.505497+0000","last_fullsized":"2024-05-26T22:48:23.505497+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:30:48.024402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.509387+0000","last_change":"2024-05-26T22:48:21.832244+0000","last_active":"2024-05-26T22:48:23.509387+0000","last_peered":"2024-05-26T22:48:23.509387+0000","last_clean":"2024-05-26T22:48:23.509387+0000","last_became_active":"2024-05-26T22:48:21.831897+0000","last_became_peered":"2024-05-26T22:48:21.831897+0000","last_unstale":"2024-05-26T22:48:23.509387+0000","last_undegraded":"2024-05-26T22:48:23.509387+0000","last_fullsized":"2024-05-26T22:48:23.509387+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:49:40.791149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.503336+0000","last_change":"2024-05-26T22:48:20.791496+0000","last_active":"2024-05-26T22:48:23.503336+0000","last_peered":"2024-05-26T22:48:23.503336+0000","last_clean":"2024-05-26T22:48:23.503336+0000","last_became_active":"2024-05-26T22:48:20.790743+0000","last_became_peered":"2024-05-26T22:48:20.790743+0000","last_unstale":"2024-05-26T22:48:23.503336+0000","last_undegraded":"2024-05-26T22:48:23.503336+0000","last_fullsized":"2024-05-26T22:48:23.503336+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:20:41.886676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.509655+0000","last_change":"2024-05-26T22:48:21.837602+0000","last_active":"2024-05-26T22:48:23.509655+0000","last_peered":"2024-05-26T22:48:23.509655+0000","last_clean":"2024-05-26T22:48:23.509655+0000","last_became_active":"2024-05-26T22:48:21.834873+0000","last_became_peered":"2024-05-26T22:48:21.834873+0000","last_unstale":"2024-05-26T22:48:23.509655+0000","last_undegraded":"2024-05-26T22:48:23.509655+0000","last_fullsized":"2024-05-26T22:48:23.509655+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:12:50.184011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.822772+0000","last_change":"2024-05-26T22:48:20.782935+0000","last_active":"2024-05-26T22:48:22.822772+0000","last_peered":"2024-05-26T22:48:22.822772+0000","last_clean":"2024-05-26T22:48:22.822772+0000","last_became_active":"2024-05-26T22:48:20.782624+0000","last_became_peered":"2024-05-26T22:48:20.782624+0000","last_unstale":"2024-05-26T22:48:22.822772+0000","last_undegraded":"2024-05-26T22:48:22.822772+0000","last_fullsized":"2024-05-26T22:48:22.822772+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:55:59.062939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.507335+0000","last_change":"2024-05-26T22:48:21.836045+0000","last_active":"2024-05-26T22:48:23.507335+0000","last_peered":"2024-05-26T22:48:23.507335+0000","last_clean":"2024-05-26T22:48:23.507335+0000","last_became_active":"2024-05-26T22:48:21.830070+0000","last_became_peered":"2024-05-26T22:48:21.830070+0000","last_unstale":"2024-05-26T22:48:23.507335+0000","last_undegraded":"2024-05-26T22:48:23.507335+0000","last_fullsized":"2024-05-26T22:48:23.507335+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:11:40.258537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.503246+0000","last_change":"2024-05-26T22:48:20.805086+0000","last_active":"2024-05-26T22:48:23.503246+0000","last_peered":"2024-05-26T22:48:23.503246+0000","last_clean":"2024-05-26T22:48:23.503246+0000","last_became_active":"2024-05-26T22:48:20.804732+0000","last_became_peered":"2024-05-26T22:48:20.804732+0000","last_unstale":"2024-05-26T22:48:23.503246+0000","last_undegraded":"2024-05-26T22:48:23.503246+0000","last_fullsized":"2024-05-26T22:48:23.503246+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:11:29.128727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.509008+0000","last_change":"2024-05-26T22:48:21.840364+0000","last_active":"2024-05-26T22:48:23.509008+0000","last_peered":"2024-05-26T22:48:23.509008+0000","last_clean":"2024-05-26T22:48:23.509008+0000","last_became_active":"2024-05-26T22:48:21.839692+0000","last_became_peered":"2024-05-26T22:48:21.839692+0000","last_unstale":"2024-05-26T22:48:23.509008+0000","last_undegraded":"2024-05-26T22:48:23.509008+0000","last_fullsized":"2024-05-26T22:48:23.509008+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:11:40.258537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.506053+0000","last_change":"2024-05-26T22:48:12.068917+0000","last_active":"2024-05-26T22:48:23.506053+0000","last_peered":"2024-05-26T22:48:23.506053+0000","last_clean":"2024-05-26T22:48:23.506053+0000","last_became_active":"2024-05-26T22:48:12.065934+0000","last_became_peered":"2024-05-26T22:48:12.065934+0000","last_unstale":"2024-05-26T22:48:23.506053+0000","last_undegraded":"2024-05-26T22:48:23.506053+0000","last_fullsized":"2024-05-26T22:48:23.506053+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:10.628122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:10.628122+0000","last_clean_scrub_stamp":"2024-05-26T22:48:10.628122+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:14:58.895911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.506186+0000","last_change":"2024-05-26T22:48:20.800517+0000","last_active":"2024-05-26T22:48:23.506186+0000","last_peered":"2024-05-26T22:48:23.506186+0000","last_clean":"2024-05-26T22:48:23.506186+0000","last_became_active":"2024-05-26T22:48:20.797956+0000","last_became_peered":"2024-05-26T22:48:20.797956+0000","last_unstale":"2024-05-26T22:48:23.506186+0000","last_undegraded":"2024-05-26T22:48:23.506186+0000","last_fullsized":"2024-05-26T22:48:23.506186+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:39:51.267408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.812649+0000","last_change":"2024-05-26T22:48:21.844543+0000","last_active":"2024-05-26T22:48:22.812649+0000","last_peered":"2024-05-26T22:48:22.812649+0000","last_clean":"2024-05-26T22:48:22.812649+0000","last_became_active":"2024-05-26T22:48:21.844272+0000","last_became_peered":"2024-05-26T22:48:21.844272+0000","last_unstale":"2024-05-26T22:48:22.812649+0000","last_undegraded":"2024-05-26T22:48:22.812649+0000","last_fullsized":"2024-05-26T22:48:22.812649+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:59:10.543916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.505812+0000","last_change":"2024-05-26T22:48:20.799727+0000","last_active":"2024-05-26T22:48:23.505812+0000","last_peered":"2024-05-26T22:48:23.505812+0000","last_clean":"2024-05-26T22:48:23.505812+0000","last_became_active":"2024-05-26T22:48:20.799427+0000","last_became_peered":"2024-05-26T22:48:20.799427+0000","last_unstale":"2024-05-26T22:48:23.505812+0000","last_undegraded":"2024-05-26T22:48:23.505812+0000","last_fullsized":"2024-05-26T22:48:23.505812+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:55:59.062939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.808326+0000","last_change":"2024-05-26T22:48:21.832212+0000","last_active":"2024-05-26T22:48:22.808326+0000","last_peered":"2024-05-26T22:48:22.808326+0000","last_clean":"2024-05-26T22:48:22.808326+0000","last_became_active":"2024-05-26T22:48:21.831893+0000","last_became_peered":"2024-05-26T22:48:21.831893+0000","last_unstale":"2024-05-26T22:48:22.808326+0000","last_undegraded":"2024-05-26T22:48:22.808326+0000","last_fullsized":"2024-05-26T22:48:22.808326+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T23:40:55.485829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.502923+0000","last_change":"2024-05-26T22:48:20.785204+0000","last_active":"2024-05-26T22:48:23.502923+0000","last_peered":"2024-05-26T22:48:23.502923+0000","last_clean":"2024-05-26T22:48:23.502923+0000","last_became_active":"2024-05-26T22:48:20.784798+0000","last_became_peered":"2024-05-26T22:48:20.784798+0000","last_unstale":"2024-05-26T22:48:23.502923+0000","last_undegraded":"2024-05-26T22:48:23.502923+0000","last_fullsized":"2024-05-26T22:48:23.502923+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:30:48.024402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.822381+0000","last_change":"2024-05-26T22:48:21.811298+0000","last_active":"2024-05-26T22:48:22.822381+0000","last_peered":"2024-05-26T22:48:22.822381+0000","last_clean":"2024-05-26T22:48:22.822381+0000","last_became_active":"2024-05-26T22:48:21.807981+0000","last_became_peered":"2024-05-26T22:48:21.807981+0000","last_unstale":"2024-05-26T22:48:22.822381+0000","last_undegraded":"2024-05-26T22:48:22.822381+0000","last_fullsized":"2024-05-26T22:48:22.822381+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:27:34.959431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.507652+0000","last_change":"2024-05-26T22:48:20.794578+0000","last_active":"2024-05-26T22:48:23.507652+0000","last_peered":"2024-05-26T22:48:23.507652+0000","last_clean":"2024-05-26T22:48:23.507652+0000","last_became_active":"2024-05-26T22:48:20.794262+0000","last_became_peered":"2024-05-26T22:48:20.794262+0000","last_unstale":"2024-05-26T22:48:23.507652+0000","last_undegraded":"2024-05-26T22:48:23.507652+0000","last_fullsized":"2024-05-26T22:48:23.507652+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:16:01.106855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.505213+0000","last_change":"2024-05-26T22:48:21.826194+0000","last_active":"2024-05-26T22:48:23.505213+0000","last_peered":"2024-05-26T22:48:23.505213+0000","last_clean":"2024-05-26T22:48:23.505213+0000","last_became_active":"2024-05-26T22:48:21.825674+0000","last_became_peered":"2024-05-26T22:48:21.825674+0000","last_unstale":"2024-05-26T22:48:23.505213+0000","last_undegraded":"2024-05-26T22:48:23.505213+0000","last_fullsized":"2024-05-26T22:48:23.505213+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:01:59.805532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.822681+0000","last_change":"2024-05-26T22:48:20.788924+0000","last_active":"2024-05-26T22:48:22.822681+0000","last_peered":"2024-05-26T22:48:22.822681+0000","last_clean":"2024-05-26T22:48:22.822681+0000","last_became_active":"2024-05-26T22:48:20.788036+0000","last_became_peered":"2024-05-26T22:48:20.788036+0000","last_unstale":"2024-05-26T22:48:22.822681+0000","last_undegraded":"2024-05-26T22:48:22.822681+0000","last_fullsized":"2024-05-26T22:48:22.822681+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:10:49.991129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.507362+0000","last_change":"2024-05-26T22:48:21.833223+0000","last_active":"2024-05-26T22:48:23.507362+0000","last_peered":"2024-05-26T22:48:23.507362+0000","last_clean":"2024-05-26T22:48:23.507362+0000","last_became_active":"2024-05-26T22:48:21.831371+0000","last_became_peered":"2024-05-26T22:48:21.831371+0000","last_unstale":"2024-05-26T22:48:23.507362+0000","last_undegraded":"2024-05-26T22:48:23.507362+0000","last_fullsized":"2024-05-26T22:48:23.507362+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:45:06.922028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.509127+0000","last_change":"2024-05-26T22:48:20.800026+0000","last_active":"2024-05-26T22:48:23.509127+0000","last_peered":"2024-05-26T22:48:23.509127+0000","last_clean":"2024-05-26T22:48:23.509127+0000","last_became_active":"2024-05-26T22:48:20.799776+0000","last_became_peered":"2024-05-26T22:48:20.799776+0000","last_unstale":"2024-05-26T22:48:23.509127+0000","last_undegraded":"2024-05-26T22:48:23.509127+0000","last_fullsized":"2024-05-26T22:48:23.509127+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:33:50.638333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.822718+0000","last_change":"2024-05-26T22:48:21.811868+0000","last_active":"2024-05-26T22:48:22.822718+0000","last_peered":"2024-05-26T22:48:22.822718+0000","last_clean":"2024-05-26T22:48:22.822718+0000","last_became_active":"2024-05-26T22:48:21.811205+0000","last_became_peered":"2024-05-26T22:48:21.811205+0000","last_unstale":"2024-05-26T22:48:22.822718+0000","last_undegraded":"2024-05-26T22:48:22.822718+0000","last_fullsized":"2024-05-26T22:48:22.822718+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:43:21.391197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.823108+0000","last_change":"2024-05-26T22:48:20.786141+0000","last_active":"2024-05-26T22:48:22.823108+0000","last_peered":"2024-05-26T22:48:22.823108+0000","last_clean":"2024-05-26T22:48:22.823108+0000","last_became_active":"2024-05-26T22:48:20.785581+0000","last_became_peered":"2024-05-26T22:48:20.785581+0000","last_unstale":"2024-05-26T22:48:22.823108+0000","last_undegraded":"2024-05-26T22:48:22.823108+0000","last_fullsized":"2024-05-26T22:48:22.823108+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:16:01.106855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.812734+0000","last_change":"2024-05-26T22:48:21.828936+0000","last_active":"2024-05-26T22:48:22.812734+0000","last_peered":"2024-05-26T22:48:22.812734+0000","last_clean":"2024-05-26T22:48:22.812734+0000","last_became_active":"2024-05-26T22:48:21.828635+0000","last_became_peered":"2024-05-26T22:48:21.828635+0000","last_unstale":"2024-05-26T22:48:22.812734+0000","last_undegraded":"2024-05-26T22:48:22.812734+0000","last_fullsized":"2024-05-26T22:48:22.812734+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:21:52.108732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.810327+0000","last_change":"2024-05-26T22:48:20.798700+0000","last_active":"2024-05-26T22:48:22.810327+0000","last_peered":"2024-05-26T22:48:22.810327+0000","last_clean":"2024-05-26T22:48:22.810327+0000","last_became_active":"2024-05-26T22:48:20.796914+0000","last_became_peered":"2024-05-26T22:48:20.796914+0000","last_unstale":"2024-05-26T22:48:22.810327+0000","last_undegraded":"2024-05-26T22:48:22.810327+0000","last_fullsized":"2024-05-26T22:48:22.810327+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:10:49.991129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.809579+0000","last_change":"2024-05-26T22:48:21.811239+0000","last_active":"2024-05-26T22:48:22.809579+0000","last_peered":"2024-05-26T22:48:22.809579+0000","last_clean":"2024-05-26T22:48:22.809579+0000","last_became_active":"2024-05-26T22:48:21.810945+0000","last_became_peered":"2024-05-26T22:48:21.810945+0000","last_unstale":"2024-05-26T22:48:22.809579+0000","last_undegraded":"2024-05-26T22:48:22.809579+0000","last_fullsized":"2024-05-26T22:48:22.809579+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:21:52.108732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.505531+0000","last_change":"2024-05-26T22:48:20.802835+0000","last_active":"2024-05-26T22:48:23.505531+0000","last_peered":"2024-05-26T22:48:23.505531+0000","last_clean":"2024-05-26T22:48:23.505531+0000","last_became_active":"2024-05-26T22:48:20.802501+0000","last_became_peered":"2024-05-26T22:48:20.802501+0000","last_unstale":"2024-05-26T22:48:23.505531+0000","last_undegraded":"2024-05-26T22:48:23.505531+0000","last_fullsized":"2024-05-26T22:48:23.505531+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:39:51.267408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.504467+0000","last_change":"2024-05-26T22:48:21.832723+0000","last_active":"2024-05-26T22:48:23.504467+0000","last_peered":"2024-05-26T22:48:23.504467+0000","last_clean":"2024-05-26T22:48:23.504467+0000","last_became_active":"2024-05-26T22:48:21.824388+0000","last_became_peered":"2024-05-26T22:48:21.824388+0000","last_unstale":"2024-05-26T22:48:23.504467+0000","last_undegraded":"2024-05-26T22:48:23.504467+0000","last_fullsized":"2024-05-26T22:48:23.504467+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:56:00.075959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.822800+0000","last_change":"2024-05-26T22:48:20.797481+0000","last_active":"2024-05-26T22:48:22.822800+0000","last_peered":"2024-05-26T22:48:22.822800+0000","last_clean":"2024-05-26T22:48:22.822800+0000","last_became_active":"2024-05-26T22:48:20.797142+0000","last_became_peered":"2024-05-26T22:48:20.797142+0000","last_unstale":"2024-05-26T22:48:22.822800+0000","last_undegraded":"2024-05-26T22:48:22.822800+0000","last_fullsized":"2024-05-26T22:48:22.822800+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:08:01.698812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.812496+0000","last_change":"2024-05-26T22:48:21.819388+0000","last_active":"2024-05-26T22:48:22.812496+0000","last_peered":"2024-05-26T22:48:22.812496+0000","last_clean":"2024-05-26T22:48:22.812496+0000","last_became_active":"2024-05-26T22:48:21.817847+0000","last_became_peered":"2024-05-26T22:48:21.817847+0000","last_unstale":"2024-05-26T22:48:22.812496+0000","last_undegraded":"2024-05-26T22:48:22.812496+0000","last_fullsized":"2024-05-26T22:48:22.812496+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:38:00.344955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.822809+0000","last_change":"2024-05-26T22:48:20.785751+0000","last_active":"2024-05-26T22:48:22.822809+0000","last_peered":"2024-05-26T22:48:22.822809+0000","last_clean":"2024-05-26T22:48:22.822809+0000","last_became_active":"2024-05-26T22:48:20.785494+0000","last_became_peered":"2024-05-26T22:48:20.785494+0000","last_unstale":"2024-05-26T22:48:22.822809+0000","last_undegraded":"2024-05-26T22:48:22.822809+0000","last_fullsized":"2024-05-26T22:48:22.822809+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:20:41.886676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.812182+0000","last_change":"2024-05-26T22:48:21.825344+0000","last_active":"2024-05-26T22:48:22.812182+0000","last_peered":"2024-05-26T22:48:22.812182+0000","last_clean":"2024-05-26T22:48:22.812182+0000","last_became_active":"2024-05-26T22:48:21.823452+0000","last_became_peered":"2024-05-26T22:48:21.823452+0000","last_unstale":"2024-05-26T22:48:22.812182+0000","last_undegraded":"2024-05-26T22:48:22.812182+0000","last_fullsized":"2024-05-26T22:48:22.812182+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:52:25.073281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.508330+0000","last_change":"2024-05-26T22:48:20.803962+0000","last_active":"2024-05-26T22:48:23.508330+0000","last_peered":"2024-05-26T22:48:23.508330+0000","last_clean":"2024-05-26T22:48:23.508330+0000","last_became_active":"2024-05-26T22:48:20.803609+0000","last_became_peered":"2024-05-26T22:48:20.803609+0000","last_unstale":"2024-05-26T22:48:23.508330+0000","last_undegraded":"2024-05-26T22:48:23.508330+0000","last_fullsized":"2024-05-26T22:48:23.508330+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:16:01.106855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.809871+0000","last_change":"2024-05-26T22:48:21.829756+0000","last_active":"2024-05-26T22:48:22.809871+0000","last_peered":"2024-05-26T22:48:22.809871+0000","last_clean":"2024-05-26T22:48:22.809871+0000","last_became_active":"2024-05-26T22:48:21.829451+0000","last_became_peered":"2024-05-26T22:48:21.829451+0000","last_unstale":"2024-05-26T22:48:22.809871+0000","last_undegraded":"2024-05-26T22:48:22.809871+0000","last_fullsized":"2024-05-26T22:48:22.809871+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:43:21.391197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.506838+0000","last_change":"2024-05-26T22:48:21.832211+0000","last_active":"2024-05-26T22:48:23.506838+0000","last_peered":"2024-05-26T22:48:23.506838+0000","last_clean":"2024-05-26T22:48:23.506838+0000","last_became_active":"2024-05-26T22:48:21.831732+0000","last_became_peered":"2024-05-26T22:48:21.831732+0000","last_unstale":"2024-05-26T22:48:23.506838+0000","last_undegraded":"2024-05-26T22:48:23.506838+0000","last_fullsized":"2024-05-26T22:48:23.506838+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:16:22.847663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.505649+0000","last_change":"2024-05-26T22:48:20.805024+0000","last_active":"2024-05-26T22:48:23.505649+0000","last_peered":"2024-05-26T22:48:23.505649+0000","last_clean":"2024-05-26T22:48:23.505649+0000","last_became_active":"2024-05-26T22:48:20.804736+0000","last_became_peered":"2024-05-26T22:48:20.804736+0000","last_unstale":"2024-05-26T22:48:23.505649+0000","last_undegraded":"2024-05-26T22:48:23.505649+0000","last_fullsized":"2024-05-26T22:48:23.505649+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:42:31.965689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.506861+0000","last_change":"2024-05-26T22:48:21.832952+0000","last_active":"2024-05-26T22:48:23.506861+0000","last_peered":"2024-05-26T22:48:23.506861+0000","last_clean":"2024-05-26T22:48:23.506861+0000","last_became_active":"2024-05-26T22:48:21.831488+0000","last_became_peered":"2024-05-26T22:48:21.831488+0000","last_unstale":"2024-05-26T22:48:23.506861+0000","last_undegraded":"2024-05-26T22:48:23.506861+0000","last_fullsized":"2024-05-26T22:48:23.506861+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:15:00.790746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.508687+0000","last_change":"2024-05-26T22:48:20.799474+0000","last_active":"2024-05-26T22:48:23.508687+0000","last_peered":"2024-05-26T22:48:23.508687+0000","last_clean":"2024-05-26T22:48:23.508687+0000","last_became_active":"2024-05-26T22:48:20.799031+0000","last_became_peered":"2024-05-26T22:48:20.799031+0000","last_unstale":"2024-05-26T22:48:23.508687+0000","last_undegraded":"2024-05-26T22:48:23.508687+0000","last_fullsized":"2024-05-26T22:48:23.508687+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:01:58.792512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.809810+0000","last_change":"2024-05-26T22:48:21.837774+0000","last_active":"2024-05-26T22:48:22.809810+0000","last_peered":"2024-05-26T22:48:22.809810+0000","last_clean":"2024-05-26T22:48:22.809810+0000","last_became_active":"2024-05-26T22:48:21.837450+0000","last_became_peered":"2024-05-26T22:48:21.837450+0000","last_unstale":"2024-05-26T22:48:22.809810+0000","last_undegraded":"2024-05-26T22:48:22.809810+0000","last_fullsized":"2024-05-26T22:48:22.809810+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:33:39.923721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.812718+0000","last_change":"2024-05-26T22:48:20.788987+0000","last_active":"2024-05-26T22:48:22.812718+0000","last_peered":"2024-05-26T22:48:22.812718+0000","last_clean":"2024-05-26T22:48:22.812718+0000","last_became_active":"2024-05-26T22:48:20.788612+0000","last_became_peered":"2024-05-26T22:48:20.788612+0000","last_unstale":"2024-05-26T22:48:22.812718+0000","last_undegraded":"2024-05-26T22:48:22.812718+0000","last_fullsized":"2024-05-26T22:48:22.812718+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:20:41.886676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.812548+0000","last_change":"2024-05-26T22:48:21.826820+0000","last_active":"2024-05-26T22:48:22.812548+0000","last_peered":"2024-05-26T22:48:22.812548+0000","last_clean":"2024-05-26T22:48:22.812548+0000","last_became_active":"2024-05-26T22:48:21.826593+0000","last_became_peered":"2024-05-26T22:48:21.826593+0000","last_unstale":"2024-05-26T22:48:22.812548+0000","last_undegraded":"2024-05-26T22:48:22.812548+0000","last_fullsized":"2024-05-26T22:48:22.812548+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:59:43.907681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.809942+0000","last_change":"2024-05-26T22:48:20.796965+0000","last_active":"2024-05-26T22:48:22.809942+0000","last_peered":"2024-05-26T22:48:22.809942+0000","last_clean":"2024-05-26T22:48:22.809942+0000","last_became_active":"2024-05-26T22:48:20.796587+0000","last_became_peered":"2024-05-26T22:48:20.796587+0000","last_unstale":"2024-05-26T22:48:22.809942+0000","last_undegraded":"2024-05-26T22:48:22.809942+0000","last_fullsized":"2024-05-26T22:48:22.809942+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:11:29.128727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-26T22:48:31.960 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-26T22:48:32.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.192+0000 7fdef2e00640 1 Processor -- start 2024-05-26T22:48:32.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.193+0000 7fdef2e00640 1 -- start start 2024-05-26T22:48:32.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.194+0000 7fdef2e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fdeec144340 0x7fdeec0cedf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:32.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.194+0000 7fdef2e00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fdeec06ebc0 0x7fdeec0b3d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:32.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.194+0000 7fdef2e00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdeec072000 con 0x7fdeec071b70 2024-05-26T22:48:32.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.194+0000 7fdef2e00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fdeec072eb0 con 0x7fdeec144340 2024-05-26T22:48:32.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.194+0000 7fdef2e00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fdeec0722a0 con 0x7fdeec06ebc0 2024-05-26T22:48:32.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.194+0000 7fdef1a00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fdeec144340 0x7fdeec0cedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:32.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.195+0000 7fdef1a00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fdeec144340 0x7fdeec0cedf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54394/0 (socket says 172.21.15.131:54394) 2024-05-26T22:48:32.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.195+0000 7fdef1a00640 1 -- 172.21.15.131:0/1311378194 learned_addr learned my addr 172.21.15.131:0/1311378194 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:32.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.195+0000 7fdef1a00640 1 -- 172.21.15.131:0/1311378194 >> v2:172.21.15.187:3300/0 conn(0x7fdeec06ebc0 msgr2=0x7fdeec0b3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:32.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.195+0000 7fdef2400640 1 --2- 172.21.15.131:0/1311378194 >> v2:172.21.15.187:3300/0 conn(0x7fdeec06ebc0 0x7fdeec0b3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:32.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.195+0000 7fdef1a00640 1 --2- 172.21.15.131:0/1311378194 >> v2:172.21.15.187:3300/0 conn(0x7fdeec06ebc0 0x7fdeec0b3d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:32.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.195+0000 7fdef1a00640 1 -- 172.21.15.131:0/1311378194 >> v1:172.21.15.187:6789/0 conn(0x7fdeec071b70 legacy=0x7fdeec06e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:32.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.195+0000 7fdef1a00640 1 -- 172.21.15.131:0/1311378194 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdeec072580 con 0x7fdeec144340 2024-05-26T22:48:32.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.195+0000 7fdef2400640 1 --2- 172.21.15.131:0/1311378194 >> v2:172.21.15.187:3300/0 conn(0x7fdeec06ebc0 0x7fdeec0b3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:48:32.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.196+0000 7fdef1a00640 1 --2- 172.21.15.131:0/1311378194 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdeec144340 0x7fdeec0cedf0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fdeec073860 tx=0x7fdedc02c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b28106241f72f6b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:32.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.196+0000 7fdeebe00640 1 -- 172.21.15.131:0/1311378194 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdedc0325a0 con 0x7fdeec144340 2024-05-26T22:48:32.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.196+0000 7fdeebe00640 1 -- 172.21.15.131:0/1311378194 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdedc038070 con 0x7fdeec144340 2024-05-26T22:48:32.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.196+0000 7fdeebe00640 1 -- 172.21.15.131:0/1311378194 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdedc032960 con 0x7fdeec144340 2024-05-26T22:48:32.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.197+0000 7fdef2e00640 1 -- 172.21.15.131:0/1311378194 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdeec144340 msgr2=0x7fdeec0cedf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:32.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.197+0000 7fdef2e00640 1 --2- 172.21.15.131:0/1311378194 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdeec144340 0x7fdeec0cedf0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fdeec073860 tx=0x7fdedc02c8e0 comp rx=0 tx=0).stop 2024-05-26T22:48:32.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.197+0000 7fdef2e00640 1 -- 172.21.15.131:0/1311378194 shutdown_connections 2024-05-26T22:48:32.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.197+0000 7fdef2e00640 1 --2- 172.21.15.131:0/1311378194 >> v2:172.21.15.187:3300/0 conn(0x7fdeec06ebc0 0x7fdeec0b3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:32.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.197+0000 7fdef2e00640 1 --2- 172.21.15.131:0/1311378194 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdeec144340 0x7fdeec0cedf0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:32.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.197+0000 7fdef2e00640 1 -- 172.21.15.131:0/1311378194 >> 172.21.15.131:0/1311378194 conn(0x7fdeec0cff40 msgr2=0x7fdeec0d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:32.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.197+0000 7fdef2e00640 1 -- 172.21.15.131:0/1311378194 shutdown_connections 2024-05-26T22:48:32.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.197+0000 7fdef2e00640 1 -- 172.21.15.131:0/1311378194 wait complete. 2024-05-26T22:48:32.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.199+0000 7fdef2e00640 1 Processor -- start 2024-05-26T22:48:32.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.199+0000 7fdef2e00640 1 -- start start 2024-05-26T22:48:32.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.200+0000 7fdef2e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdeec06ebc0 0x7fdeec13b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:32.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.200+0000 7fdef2e00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdeec071b70 0x7fdeec13bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:32.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.200+0000 7fdef2e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdeec144340 0x7fdeec13c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:32.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.200+0000 7fdef2e00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fdeec0b4260 con 0x7fdeec06ebc0 2024-05-26T22:48:32.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.200+0000 7fdef2e00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fdeec072b40 con 0x7fdeec071b70 2024-05-26T22:48:32.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.200+0000 7fdef2e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fdeec0b43e0 con 0x7fdeec144340 2024-05-26T22:48:32.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.200+0000 7fdef1a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdeec06ebc0 0x7fdeec13b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:32.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.200+0000 7fdef1a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdeec06ebc0 0x7fdeec13b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54408/0 (socket says 172.21.15.131:54408) 2024-05-26T22:48:32.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.200+0000 7fdef1a00640 1 -- 172.21.15.131:0/2761020050 learned_addr learned my addr 172.21.15.131:0/2761020050 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:32.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.200+0000 7fdef1a00640 1 -- 172.21.15.131:0/2761020050 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdeec144340 msgr2=0x7fdeec13c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:32.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.200+0000 7fdef2400640 1 --2- 172.21.15.131:0/2761020050 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdeec144340 0x7fdeec13c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:32.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.200+0000 7fdef1a00640 1 --2- 172.21.15.131:0/2761020050 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdeec144340 0x7fdeec13c010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:32.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.200+0000 7fdef1a00640 1 -- 172.21.15.131:0/2761020050 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdeec071b70 msgr2=0x7fdeec13bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:32.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.200+0000 7fdef1000640 1 --2- 172.21.15.131:0/2761020050 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdeec071b70 0x7fdeec13bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:32.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.200+0000 7fdef1a00640 1 --2- 172.21.15.131:0/2761020050 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdeec071b70 0x7fdeec13bad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:32.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.200+0000 7fdef1a00640 1 -- 172.21.15.131:0/2761020050 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdeec1385f0 con 0x7fdeec06ebc0 2024-05-26T22:48:32.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.201+0000 7fdef1a00640 1 --2- 172.21.15.131:0/2761020050 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdeec06ebc0 0x7fdeec13b1b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fdedc000c00 tx=0x7fdedc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:32.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.201+0000 7fdee9600640 1 -- 172.21.15.131:0/2761020050 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdedc033840 con 0x7fdeec06ebc0 2024-05-26T22:48:32.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.201+0000 7fdee9600640 1 -- 172.21.15.131:0/2761020050 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdedc033a20 con 0x7fdeec06ebc0 2024-05-26T22:48:32.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.201+0000 7fdee9600640 1 -- 172.21.15.131:0/2761020050 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdedc033e10 con 0x7fdeec06ebc0 2024-05-26T22:48:32.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.201+0000 7fdef2e00640 1 -- 172.21.15.131:0/2761020050 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdeec13e010 con 0x7fdeec06ebc0 2024-05-26T22:48:32.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.201+0000 7fdef2e00640 1 -- 172.21.15.131:0/2761020050 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdeec138860 con 0x7fdeec06ebc0 2024-05-26T22:48:32.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.202+0000 7fdef2e00640 1 -- 172.21.15.131:0/2761020050 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdeec0ce4e0 con 0x7fdeec06ebc0 2024-05-26T22:48:32.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.203+0000 7fdee9600640 1 -- 172.21.15.131:0/2761020050 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdedc03c3f0 con 0x7fdeec06ebc0 2024-05-26T22:48:32.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.204+0000 7fdee9600640 1 --2- 172.21.15.131:0/2761020050 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fdecc06be60 0x7fdecc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:32.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.204+0000 7fdee9600640 1 -- 172.21.15.131:0/2761020050 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fdedc030030 con 0x7fdeec06ebc0 2024-05-26T22:48:32.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.204+0000 7fdef1000640 1 --2- 172.21.15.131:0/2761020050 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fdecc06be60 0x7fdecc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:32.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.205+0000 7fdef1000640 1 --2- 172.21.15.131:0/2761020050 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fdecc06be60 0x7fdecc06e320 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fdeec0cd7e0 tx=0x7fded8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:32.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.209+0000 7fdee9600640 1 -- 172.21.15.131:0/2761020050 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fdeec0ce4e0 con 0x7fdeec06ebc0 2024-05-26T22:48:32.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.390+0000 7fdef2e00640 1 -- 172.21.15.131:0/2761020050 --> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdeec15abc0 con 0x7fdecc06be60 2024-05-26T22:48:32.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.409+0000 7fdee9600640 1 -- 172.21.15.131:0/2761020050 <== mgr.4104 v2:172.21.15.131:6832/3484832947 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251837 (secure 0 0 0) 0x7fded80050d0 con 0x7fdecc06be60 2024-05-26T22:48:32.412 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:48:32.413 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-05-26T22:48:32.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.415+0000 7fdef2e00640 1 -- 172.21.15.131:0/2761020050 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fdecc06be60 msgr2=0x7fdecc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:32.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.415+0000 7fdef2e00640 1 --2- 172.21.15.131:0/2761020050 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fdecc06be60 0x7fdecc06e320 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fdeec0cd7e0 tx=0x7fded8009040 comp rx=0 tx=0).stop 2024-05-26T22:48:32.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.415+0000 7fdef2e00640 1 -- 172.21.15.131:0/2761020050 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdeec06ebc0 msgr2=0x7fdeec13b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:32.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.415+0000 7fdef2e00640 1 --2- 172.21.15.131:0/2761020050 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdeec06ebc0 0x7fdeec13b1b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fdedc000c00 tx=0x7fdedc0046c0 comp rx=0 tx=0).stop 2024-05-26T22:48:32.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.415+0000 7fdef2e00640 1 -- 172.21.15.131:0/2761020050 shutdown_connections 2024-05-26T22:48:32.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.415+0000 7fdef2e00640 1 --2- 172.21.15.131:0/2761020050 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdeec144340 0x7fdeec13c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:32.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.415+0000 7fdef2e00640 1 --2- 172.21.15.131:0/2761020050 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fdecc06be60 0x7fdecc06e320 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:32.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.415+0000 7fdef2e00640 1 --2- 172.21.15.131:0/2761020050 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdeec071b70 0x7fdeec13bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:32.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.415+0000 7fdef2e00640 1 --2- 172.21.15.131:0/2761020050 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdeec06ebc0 0x7fdeec13b1b0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:32.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.415+0000 7fdef2e00640 1 -- 172.21.15.131:0/2761020050 >> 172.21.15.131:0/2761020050 conn(0x7fdeec0cff40 msgr2=0x7fdeec0d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:32.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.415+0000 7fdef2e00640 1 -- 172.21.15.131:0/2761020050 shutdown_connections 2024-05-26T22:48:32.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.415+0000 7fdef2e00640 1 -- 172.21.15.131:0/2761020050 wait complete. 2024-05-26T22:48:32.433 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-05-26T22:48:32.386842+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218992,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755728,"statfs":{"total":773094113280,"available":772869865472,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.003776"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.509427+0000","last_change":"2024-05-26T22:48:21.831991+0000","last_active":"2024-05-26T22:48:23.509427+0000","last_peered":"2024-05-26T22:48:23.509427+0000","last_clean":"2024-05-26T22:48:23.509427+0000","last_became_active":"2024-05-26T22:48:21.831657+0000","last_became_peered":"2024-05-26T22:48:21.831657+0000","last_unstale":"2024-05-26T22:48:23.509427+0000","last_undegraded":"2024-05-26T22:48:23.509427+0000","last_fullsized":"2024-05-26T22:48:23.509427+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:30:45.689344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.822991+0000","last_change":"2024-05-26T22:48:21.813275+0000","last_active":"2024-05-26T22:48:22.822991+0000","last_peered":"2024-05-26T22:48:22.822991+0000","last_clean":"2024-05-26T22:48:22.822991+0000","last_became_active":"2024-05-26T22:48:21.812895+0000","last_became_peered":"2024-05-26T22:48:21.812895+0000","last_unstale":"2024-05-26T22:48:22.822991+0000","last_undegraded":"2024-05-26T22:48:22.822991+0000","last_fullsized":"2024-05-26T22:48:22.822991+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:59:43.907681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.809083+0000","last_change":"2024-05-26T22:48:21.805790+0000","last_active":"2024-05-26T22:48:22.809083+0000","last_peered":"2024-05-26T22:48:22.809083+0000","last_clean":"2024-05-26T22:48:22.809083+0000","last_became_active":"2024-05-26T22:48:21.805318+0000","last_became_peered":"2024-05-26T22:48:21.805318+0000","last_unstale":"2024-05-26T22:48:22.809083+0000","last_undegraded":"2024-05-26T22:48:22.809083+0000","last_fullsized":"2024-05-26T22:48:22.809083+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:43:21.391197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.506425+0000","last_change":"2024-05-26T22:48:21.831837+0000","last_active":"2024-05-26T22:48:23.506425+0000","last_peered":"2024-05-26T22:48:23.506425+0000","last_clean":"2024-05-26T22:48:23.506425+0000","last_became_active":"2024-05-26T22:48:21.829461+0000","last_became_peered":"2024-05-26T22:48:21.829461+0000","last_unstale":"2024-05-26T22:48:23.506425+0000","last_undegraded":"2024-05-26T22:48:23.506425+0000","last_fullsized":"2024-05-26T22:48:23.506425+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:56:39.208963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.508007+0000","last_change":"2024-05-26T22:48:21.821926+0000","last_active":"2024-05-26T22:48:23.508007+0000","last_peered":"2024-05-26T22:48:23.508007+0000","last_clean":"2024-05-26T22:48:23.508007+0000","last_became_active":"2024-05-26T22:48:21.821594+0000","last_became_peered":"2024-05-26T22:48:21.821594+0000","last_unstale":"2024-05-26T22:48:23.508007+0000","last_undegraded":"2024-05-26T22:48:23.508007+0000","last_fullsized":"2024-05-26T22:48:23.508007+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:16:22.847663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.506340+0000","last_change":"2024-05-26T22:48:21.823954+0000","last_active":"2024-05-26T22:48:23.506340+0000","last_peered":"2024-05-26T22:48:23.506340+0000","last_clean":"2024-05-26T22:48:23.506340+0000","last_became_active":"2024-05-26T22:48:21.821405+0000","last_became_peered":"2024-05-26T22:48:21.821405+0000","last_unstale":"2024-05-26T22:48:23.506340+0000","last_undegraded":"2024-05-26T22:48:23.506340+0000","last_fullsized":"2024-05-26T22:48:23.506340+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:33:39.923721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.506938+0000","last_change":"2024-05-26T22:48:21.821415+0000","last_active":"2024-05-26T22:48:23.506938+0000","last_peered":"2024-05-26T22:48:23.506938+0000","last_clean":"2024-05-26T22:48:23.506938+0000","last_became_active":"2024-05-26T22:48:21.821092+0000","last_became_peered":"2024-05-26T22:48:21.821092+0000","last_unstale":"2024-05-26T22:48:23.506938+0000","last_undegraded":"2024-05-26T22:48:23.506938+0000","last_fullsized":"2024-05-26T22:48:23.506938+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:56:39.208963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.811969+0000","last_change":"2024-05-26T22:48:21.847352+0000","last_active":"2024-05-26T22:48:22.811969+0000","last_peered":"2024-05-26T22:48:22.811969+0000","last_clean":"2024-05-26T22:48:22.811969+0000","last_became_active":"2024-05-26T22:48:21.846965+0000","last_became_peered":"2024-05-26T22:48:21.846965+0000","last_unstale":"2024-05-26T22:48:22.811969+0000","last_undegraded":"2024-05-26T22:48:22.811969+0000","last_fullsized":"2024-05-26T22:48:22.811969+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T23:40:55.485829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.509545+0000","last_change":"2024-05-26T22:48:21.847421+0000","last_active":"2024-05-26T22:48:23.509545+0000","last_peered":"2024-05-26T22:48:23.509545+0000","last_clean":"2024-05-26T22:48:23.509545+0000","last_became_active":"2024-05-26T22:48:21.846702+0000","last_became_peered":"2024-05-26T22:48:21.846702+0000","last_unstale":"2024-05-26T22:48:23.509545+0000","last_undegraded":"2024-05-26T22:48:23.509545+0000","last_fullsized":"2024-05-26T22:48:23.509545+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:27:35.210564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.810240+0000","last_change":"2024-05-26T22:48:21.831244+0000","last_active":"2024-05-26T22:48:22.810240+0000","last_peered":"2024-05-26T22:48:22.810240+0000","last_clean":"2024-05-26T22:48:22.810240+0000","last_became_active":"2024-05-26T22:48:21.830949+0000","last_became_peered":"2024-05-26T22:48:21.830949+0000","last_unstale":"2024-05-26T22:48:22.810240+0000","last_undegraded":"2024-05-26T22:48:22.810240+0000","last_fullsized":"2024-05-26T22:48:22.810240+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:59:43.907681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.810220+0000","last_change":"2024-05-26T22:48:21.836749+0000","last_active":"2024-05-26T22:48:22.810220+0000","last_peered":"2024-05-26T22:48:22.810220+0000","last_clean":"2024-05-26T22:48:22.810220+0000","last_became_active":"2024-05-26T22:48:21.836431+0000","last_became_peered":"2024-05-26T22:48:21.836431+0000","last_unstale":"2024-05-26T22:48:22.810220+0000","last_undegraded":"2024-05-26T22:48:22.810220+0000","last_fullsized":"2024-05-26T22:48:22.810220+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:38:00.344955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.505172+0000","last_change":"2024-05-26T22:48:21.819214+0000","last_active":"2024-05-26T22:48:23.505172+0000","last_peered":"2024-05-26T22:48:23.505172+0000","last_clean":"2024-05-26T22:48:23.505172+0000","last_became_active":"2024-05-26T22:48:21.818837+0000","last_became_peered":"2024-05-26T22:48:21.818837+0000","last_unstale":"2024-05-26T22:48:23.505172+0000","last_undegraded":"2024-05-26T22:48:23.505172+0000","last_fullsized":"2024-05-26T22:48:23.505172+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:42:32.978709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.812261+0000","last_change":"2024-05-26T22:48:21.830442+0000","last_active":"2024-05-26T22:48:22.812261+0000","last_peered":"2024-05-26T22:48:22.812261+0000","last_clean":"2024-05-26T22:48:22.812261+0000","last_became_active":"2024-05-26T22:48:21.830132+0000","last_became_peered":"2024-05-26T22:48:21.830132+0000","last_unstale":"2024-05-26T22:48:22.812261+0000","last_undegraded":"2024-05-26T22:48:22.812261+0000","last_fullsized":"2024-05-26T22:48:22.812261+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:43:21.391197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.808630+0000","last_change":"2024-05-26T22:48:21.818032+0000","last_active":"2024-05-26T22:48:22.808630+0000","last_peered":"2024-05-26T22:48:22.808630+0000","last_clean":"2024-05-26T22:48:22.808630+0000","last_became_active":"2024-05-26T22:48:21.817717+0000","last_became_peered":"2024-05-26T22:48:21.817717+0000","last_unstale":"2024-05-26T22:48:22.808630+0000","last_undegraded":"2024-05-26T22:48:22.808630+0000","last_fullsized":"2024-05-26T22:48:22.808630+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:27:34.959431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.822672+0000","last_change":"2024-05-26T22:48:21.823839+0000","last_active":"2024-05-26T22:48:22.822672+0000","last_peered":"2024-05-26T22:48:22.822672+0000","last_clean":"2024-05-26T22:48:22.822672+0000","last_became_active":"2024-05-26T22:48:21.823522+0000","last_became_peered":"2024-05-26T22:48:21.823522+0000","last_unstale":"2024-05-26T22:48:22.822672+0000","last_undegraded":"2024-05-26T22:48:22.822672+0000","last_fullsized":"2024-05-26T22:48:22.822672+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:52:25.073281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.504208+0000","last_change":"2024-05-26T22:48:21.819432+0000","last_active":"2024-05-26T22:48:23.504208+0000","last_peered":"2024-05-26T22:48:23.504208+0000","last_clean":"2024-05-26T22:48:23.504208+0000","last_became_active":"2024-05-26T22:48:21.819139+0000","last_became_peered":"2024-05-26T22:48:21.819139+0000","last_unstale":"2024-05-26T22:48:23.504208+0000","last_undegraded":"2024-05-26T22:48:23.504208+0000","last_fullsized":"2024-05-26T22:48:23.504208+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:21:52.108732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.507463+0000","last_change":"2024-05-26T22:48:21.842657+0000","last_active":"2024-05-26T22:48:23.507463+0000","last_peered":"2024-05-26T22:48:23.507463+0000","last_clean":"2024-05-26T22:48:23.507463+0000","last_became_active":"2024-05-26T22:48:21.842312+0000","last_became_peered":"2024-05-26T22:48:21.842312+0000","last_unstale":"2024-05-26T22:48:23.507463+0000","last_undegraded":"2024-05-26T22:48:23.507463+0000","last_fullsized":"2024-05-26T22:48:23.507463+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:49:40.791149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.503319+0000","last_change":"2024-05-26T22:48:21.812988+0000","last_active":"2024-05-26T22:48:23.503319+0000","last_peered":"2024-05-26T22:48:23.503319+0000","last_clean":"2024-05-26T22:48:23.503319+0000","last_became_active":"2024-05-26T22:48:21.812669+0000","last_became_peered":"2024-05-26T22:48:21.812669+0000","last_unstale":"2024-05-26T22:48:23.503319+0000","last_undegraded":"2024-05-26T22:48:23.503319+0000","last_fullsized":"2024-05-26T22:48:23.503319+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:16:02.119875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.509267+0000","last_change":"2024-05-26T22:48:21.830493+0000","last_active":"2024-05-26T22:48:23.509267+0000","last_peered":"2024-05-26T22:48:23.509267+0000","last_clean":"2024-05-26T22:48:23.509267+0000","last_became_active":"2024-05-26T22:48:21.830116+0000","last_became_peered":"2024-05-26T22:48:21.830116+0000","last_unstale":"2024-05-26T22:48:23.509267+0000","last_undegraded":"2024-05-26T22:48:23.509267+0000","last_fullsized":"2024-05-26T22:48:23.509267+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:45:06.922028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.809824+0000","last_change":"2024-05-26T22:48:21.829461+0000","last_active":"2024-05-26T22:48:22.809824+0000","last_peered":"2024-05-26T22:48:22.809824+0000","last_clean":"2024-05-26T22:48:22.809824+0000","last_became_active":"2024-05-26T22:48:21.829132+0000","last_became_peered":"2024-05-26T22:48:21.829132+0000","last_unstale":"2024-05-26T22:48:22.809824+0000","last_undegraded":"2024-05-26T22:48:22.809824+0000","last_fullsized":"2024-05-26T22:48:22.809824+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:52:25.073281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.809106+0000","last_change":"2024-05-26T22:48:21.818888+0000","last_active":"2024-05-26T22:48:22.809106+0000","last_peered":"2024-05-26T22:48:22.809106+0000","last_clean":"2024-05-26T22:48:22.809106+0000","last_became_active":"2024-05-26T22:48:21.818639+0000","last_became_peered":"2024-05-26T22:48:21.818639+0000","last_unstale":"2024-05-26T22:48:22.809106+0000","last_undegraded":"2024-05-26T22:48:22.809106+0000","last_fullsized":"2024-05-26T22:48:22.809106+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:38:00.344955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.506778+0000","last_change":"2024-05-26T22:48:21.842585+0000","last_active":"2024-05-26T22:48:23.506778+0000","last_peered":"2024-05-26T22:48:23.506778+0000","last_clean":"2024-05-26T22:48:23.506778+0000","last_became_active":"2024-05-26T22:48:21.842302+0000","last_became_peered":"2024-05-26T22:48:21.842302+0000","last_unstale":"2024-05-26T22:48:23.506778+0000","last_undegraded":"2024-05-26T22:48:23.506778+0000","last_fullsized":"2024-05-26T22:48:23.506778+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:08:10.330236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.811982+0000","last_change":"2024-05-26T22:48:21.830936+0000","last_active":"2024-05-26T22:48:22.811982+0000","last_peered":"2024-05-26T22:48:22.811982+0000","last_clean":"2024-05-26T22:48:22.811982+0000","last_became_active":"2024-05-26T22:48:21.830551+0000","last_became_peered":"2024-05-26T22:48:21.830551+0000","last_unstale":"2024-05-26T22:48:22.811982+0000","last_undegraded":"2024-05-26T22:48:22.811982+0000","last_fullsized":"2024-05-26T22:48:22.811982+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:27:34.959431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.508425+0000","last_change":"2024-05-26T22:48:21.837001+0000","last_active":"2024-05-26T22:48:23.508425+0000","last_peered":"2024-05-26T22:48:23.508425+0000","last_clean":"2024-05-26T22:48:23.508425+0000","last_became_active":"2024-05-26T22:48:21.836749+0000","last_became_peered":"2024-05-26T22:48:21.836749+0000","last_unstale":"2024-05-26T22:48:23.508425+0000","last_undegraded":"2024-05-26T22:48:23.508425+0000","last_fullsized":"2024-05-26T22:48:23.508425+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:30:45.689344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.812885+0000","last_change":"2024-05-26T22:48:21.834986+0000","last_active":"2024-05-26T22:48:22.812885+0000","last_peered":"2024-05-26T22:48:22.812885+0000","last_clean":"2024-05-26T22:48:22.812885+0000","last_became_active":"2024-05-26T22:48:21.834572+0000","last_became_peered":"2024-05-26T22:48:21.834572+0000","last_unstale":"2024-05-26T22:48:22.812885+0000","last_undegraded":"2024-05-26T22:48:22.812885+0000","last_fullsized":"2024-05-26T22:48:22.812885+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:33:39.923721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.810409+0000","last_change":"2024-05-26T22:48:21.824957+0000","last_active":"2024-05-26T22:48:22.810409+0000","last_peered":"2024-05-26T22:48:22.810409+0000","last_clean":"2024-05-26T22:48:22.810409+0000","last_became_active":"2024-05-26T22:48:21.822871+0000","last_became_peered":"2024-05-26T22:48:21.822871+0000","last_unstale":"2024-05-26T22:48:22.810409+0000","last_undegraded":"2024-05-26T22:48:22.810409+0000","last_fullsized":"2024-05-26T22:48:22.810409+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:59:10.543916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.822941+0000","last_change":"2024-05-26T22:48:21.827100+0000","last_active":"2024-05-26T22:48:22.822941+0000","last_peered":"2024-05-26T22:48:22.822941+0000","last_clean":"2024-05-26T22:48:22.822941+0000","last_became_active":"2024-05-26T22:48:21.826810+0000","last_became_peered":"2024-05-26T22:48:21.826810+0000","last_unstale":"2024-05-26T22:48:22.822941+0000","last_undegraded":"2024-05-26T22:48:22.822941+0000","last_fullsized":"2024-05-26T22:48:22.822941+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:38:00.344955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.813201+0000","last_change":"2024-05-26T22:48:21.836236+0000","last_active":"2024-05-26T22:48:22.813201+0000","last_peered":"2024-05-26T22:48:22.813201+0000","last_clean":"2024-05-26T22:48:22.813201+0000","last_became_active":"2024-05-26T22:48:21.834609+0000","last_became_peered":"2024-05-26T22:48:21.834609+0000","last_unstale":"2024-05-26T22:48:22.813201+0000","last_undegraded":"2024-05-26T22:48:22.813201+0000","last_fullsized":"2024-05-26T22:48:22.813201+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:08:10.330236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.809226+0000","last_change":"2024-05-26T22:48:21.821191+0000","last_active":"2024-05-26T22:48:22.809226+0000","last_peered":"2024-05-26T22:48:22.809226+0000","last_clean":"2024-05-26T22:48:22.809226+0000","last_became_active":"2024-05-26T22:48:21.820731+0000","last_became_peered":"2024-05-26T22:48:21.820731+0000","last_unstale":"2024-05-26T22:48:22.809226+0000","last_undegraded":"2024-05-26T22:48:22.809226+0000","last_fullsized":"2024-05-26T22:48:22.809226+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:59:43.907681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.507246+0000","last_change":"2024-05-26T22:48:21.836074+0000","last_active":"2024-05-26T22:48:23.507246+0000","last_peered":"2024-05-26T22:48:23.507246+0000","last_clean":"2024-05-26T22:48:23.507246+0000","last_became_active":"2024-05-26T22:48:21.831547+0000","last_became_peered":"2024-05-26T22:48:21.831547+0000","last_unstale":"2024-05-26T22:48:23.507246+0000","last_undegraded":"2024-05-26T22:48:23.507246+0000","last_fullsized":"2024-05-26T22:48:23.507246+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:34:58.598026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.503056+0000","last_change":"2024-05-26T22:48:21.815510+0000","last_active":"2024-05-26T22:48:23.503056+0000","last_peered":"2024-05-26T22:48:23.503056+0000","last_clean":"2024-05-26T22:48:23.503056+0000","last_became_active":"2024-05-26T22:48:21.815132+0000","last_became_peered":"2024-05-26T22:48:21.815132+0000","last_unstale":"2024-05-26T22:48:23.503056+0000","last_undegraded":"2024-05-26T22:48:23.503056+0000","last_fullsized":"2024-05-26T22:48:23.503056+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:56:00.075959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.809652+0000","last_change":"2024-05-26T22:48:21.811494+0000","last_active":"2024-05-26T22:48:22.809652+0000","last_peered":"2024-05-26T22:48:22.809652+0000","last_clean":"2024-05-26T22:48:22.809652+0000","last_became_active":"2024-05-26T22:48:21.809669+0000","last_became_peered":"2024-05-26T22:48:21.809669+0000","last_unstale":"2024-05-26T22:48:22.809652+0000","last_undegraded":"2024-05-26T22:48:22.809652+0000","last_fullsized":"2024-05-26T22:48:22.809652+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:27:34.959431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.504470+0000","last_change":"2024-05-26T22:48:21.824303+0000","last_active":"2024-05-26T22:48:23.504470+0000","last_peered":"2024-05-26T22:48:23.504470+0000","last_clean":"2024-05-26T22:48:23.504470+0000","last_became_active":"2024-05-26T22:48:21.823500+0000","last_became_peered":"2024-05-26T22:48:21.823500+0000","last_unstale":"2024-05-26T22:48:23.504470+0000","last_undegraded":"2024-05-26T22:48:23.504470+0000","last_fullsized":"2024-05-26T22:48:23.504470+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:16:02.119875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.812241+0000","last_change":"2024-05-26T22:48:20.802593+0000","last_active":"2024-05-26T22:48:22.812241+0000","last_peered":"2024-05-26T22:48:22.812241+0000","last_clean":"2024-05-26T22:48:22.812241+0000","last_became_active":"2024-05-26T22:48:20.802352+0000","last_became_peered":"2024-05-26T22:48:20.802352+0000","last_unstale":"2024-05-26T22:48:22.812241+0000","last_undegraded":"2024-05-26T22:48:22.812241+0000","last_fullsized":"2024-05-26T22:48:22.812241+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:39:51.267408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.809443+0000","last_change":"2024-05-26T22:48:21.832952+0000","last_active":"2024-05-26T22:48:22.809443+0000","last_peered":"2024-05-26T22:48:22.809443+0000","last_clean":"2024-05-26T22:48:22.809443+0000","last_became_active":"2024-05-26T22:48:21.832496+0000","last_became_peered":"2024-05-26T22:48:21.832496+0000","last_unstale":"2024-05-26T22:48:22.809443+0000","last_undegraded":"2024-05-26T22:48:22.809443+0000","last_fullsized":"2024-05-26T22:48:22.809443+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:59:10.543916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.810033+0000","last_change":"2024-05-26T22:48:20.789223+0000","last_active":"2024-05-26T22:48:22.810033+0000","last_peered":"2024-05-26T22:48:22.810033+0000","last_clean":"2024-05-26T22:48:22.810033+0000","last_became_active":"2024-05-26T22:48:20.787967+0000","last_became_peered":"2024-05-26T22:48:20.787967+0000","last_unstale":"2024-05-26T22:48:22.810033+0000","last_undegraded":"2024-05-26T22:48:22.810033+0000","last_fullsized":"2024-05-26T22:48:22.810033+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:08:01.698812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.812657+0000","last_change":"2024-05-26T22:48:20.784257+0000","last_active":"2024-05-26T22:48:22.812657+0000","last_peered":"2024-05-26T22:48:22.812657+0000","last_clean":"2024-05-26T22:48:22.812657+0000","last_became_active":"2024-05-26T22:48:20.783493+0000","last_became_peered":"2024-05-26T22:48:20.783493+0000","last_unstale":"2024-05-26T22:48:22.812657+0000","last_undegraded":"2024-05-26T22:48:22.812657+0000","last_fullsized":"2024-05-26T22:48:22.812657+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:30:48.024402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.813086+0000","last_change":"2024-05-26T22:48:21.834733+0000","last_active":"2024-05-26T22:48:22.813086+0000","last_peered":"2024-05-26T22:48:22.813086+0000","last_clean":"2024-05-26T22:48:22.813086+0000","last_became_active":"2024-05-26T22:48:21.834396+0000","last_became_peered":"2024-05-26T22:48:21.834396+0000","last_unstale":"2024-05-26T22:48:22.813086+0000","last_undegraded":"2024-05-26T22:48:22.813086+0000","last_fullsized":"2024-05-26T22:48:22.813086+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:56:39.208963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.506282+0000","last_change":"2024-05-26T22:48:20.798521+0000","last_active":"2024-05-26T22:48:23.506282+0000","last_peered":"2024-05-26T22:48:23.506282+0000","last_clean":"2024-05-26T22:48:23.506282+0000","last_became_active":"2024-05-26T22:48:20.798153+0000","last_became_peered":"2024-05-26T22:48:20.798153+0000","last_unstale":"2024-05-26T22:48:23.506282+0000","last_undegraded":"2024-05-26T22:48:23.506282+0000","last_fullsized":"2024-05-26T22:48:23.506282+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:10:49.991129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.823065+0000","last_change":"2024-05-26T22:48:21.832951+0000","last_active":"2024-05-26T22:48:22.823065+0000","last_peered":"2024-05-26T22:48:22.823065+0000","last_clean":"2024-05-26T22:48:22.823065+0000","last_became_active":"2024-05-26T22:48:21.831917+0000","last_became_peered":"2024-05-26T22:48:21.831917+0000","last_unstale":"2024-05-26T22:48:22.823065+0000","last_undegraded":"2024-05-26T22:48:22.823065+0000","last_fullsized":"2024-05-26T22:48:22.823065+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:59:10.543916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.810123+0000","last_change":"2024-05-26T22:48:20.801167+0000","last_active":"2024-05-26T22:48:22.810123+0000","last_peered":"2024-05-26T22:48:22.810123+0000","last_clean":"2024-05-26T22:48:22.810123+0000","last_became_active":"2024-05-26T22:48:20.800854+0000","last_became_peered":"2024-05-26T22:48:20.800854+0000","last_unstale":"2024-05-26T22:48:22.810123+0000","last_undegraded":"2024-05-26T22:48:22.810123+0000","last_fullsized":"2024-05-26T22:48:22.810123+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:39:51.267408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.509643+0000","last_change":"2024-05-26T22:48:21.841974+0000","last_active":"2024-05-26T22:48:23.509643+0000","last_peered":"2024-05-26T22:48:23.509643+0000","last_clean":"2024-05-26T22:48:23.509643+0000","last_became_active":"2024-05-26T22:48:21.840912+0000","last_became_peered":"2024-05-26T22:48:21.840912+0000","last_unstale":"2024-05-26T22:48:23.509643+0000","last_undegraded":"2024-05-26T22:48:23.509643+0000","last_fullsized":"2024-05-26T22:48:23.509643+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:27:35.210564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.509713+0000","last_change":"2024-05-26T22:48:20.798072+0000","last_active":"2024-05-26T22:48:23.509713+0000","last_peered":"2024-05-26T22:48:23.509713+0000","last_clean":"2024-05-26T22:48:23.509713+0000","last_became_active":"2024-05-26T22:48:20.797559+0000","last_became_peered":"2024-05-26T22:48:20.797559+0000","last_unstale":"2024-05-26T22:48:23.509713+0000","last_undegraded":"2024-05-26T22:48:23.509713+0000","last_fullsized":"2024-05-26T22:48:23.509713+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:52:24.060261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.809896+0000","last_change":"2024-05-26T22:48:21.838154+0000","last_active":"2024-05-26T22:48:22.809896+0000","last_peered":"2024-05-26T22:48:22.809896+0000","last_clean":"2024-05-26T22:48:22.809896+0000","last_became_active":"2024-05-26T22:48:21.837073+0000","last_became_peered":"2024-05-26T22:48:21.837073+0000","last_unstale":"2024-05-26T22:48:22.809896+0000","last_undegraded":"2024-05-26T22:48:22.809896+0000","last_fullsized":"2024-05-26T22:48:22.809896+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:56:39.208963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.822642+0000","last_change":"2024-05-26T22:48:20.796108+0000","last_active":"2024-05-26T22:48:22.822642+0000","last_peered":"2024-05-26T22:48:22.822642+0000","last_clean":"2024-05-26T22:48:22.822642+0000","last_became_active":"2024-05-26T22:48:20.795717+0000","last_became_peered":"2024-05-26T22:48:20.795717+0000","last_unstale":"2024-05-26T22:48:22.822642+0000","last_undegraded":"2024-05-26T22:48:22.822642+0000","last_fullsized":"2024-05-26T22:48:22.822642+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:39:51.267408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.505587+0000","last_change":"2024-05-26T22:48:21.823519+0000","last_active":"2024-05-26T22:48:23.505587+0000","last_peered":"2024-05-26T22:48:23.505587+0000","last_clean":"2024-05-26T22:48:23.505587+0000","last_became_active":"2024-05-26T22:48:21.823240+0000","last_became_peered":"2024-05-26T22:48:21.823240+0000","last_unstale":"2024-05-26T22:48:23.505587+0000","last_undegraded":"2024-05-26T22:48:23.505587+0000","last_fullsized":"2024-05-26T22:48:23.505587+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:33:51.651353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.809592+0000","last_change":"2024-05-26T22:48:20.792915+0000","last_active":"2024-05-26T22:48:22.809592+0000","last_peered":"2024-05-26T22:48:22.809592+0000","last_clean":"2024-05-26T22:48:22.809592+0000","last_became_active":"2024-05-26T22:48:20.792037+0000","last_became_peered":"2024-05-26T22:48:20.792037+0000","last_unstale":"2024-05-26T22:48:22.809592+0000","last_undegraded":"2024-05-26T22:48:22.809592+0000","last_fullsized":"2024-05-26T22:48:22.809592+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:11:29.128727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.508563+0000","last_change":"2024-05-26T22:48:21.839574+0000","last_active":"2024-05-26T22:48:23.508563+0000","last_peered":"2024-05-26T22:48:23.508563+0000","last_clean":"2024-05-26T22:48:23.508563+0000","last_became_active":"2024-05-26T22:48:21.834905+0000","last_became_peered":"2024-05-26T22:48:21.834905+0000","last_unstale":"2024-05-26T22:48:23.508563+0000","last_undegraded":"2024-05-26T22:48:23.508563+0000","last_fullsized":"2024-05-26T22:48:23.508563+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:34:58.598026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.809371+0000","last_change":"2024-05-26T22:48:20.792287+0000","last_active":"2024-05-26T22:48:22.809371+0000","last_peered":"2024-05-26T22:48:22.809371+0000","last_clean":"2024-05-26T22:48:22.809371+0000","last_became_active":"2024-05-26T22:48:20.791592+0000","last_became_peered":"2024-05-26T22:48:20.791592+0000","last_unstale":"2024-05-26T22:48:22.809371+0000","last_undegraded":"2024-05-26T22:48:22.809371+0000","last_fullsized":"2024-05-26T22:48:22.809371+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:30:48.024402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.808872+0000","last_change":"2024-05-26T22:48:21.824723+0000","last_active":"2024-05-26T22:48:22.808872+0000","last_peered":"2024-05-26T22:48:22.808872+0000","last_clean":"2024-05-26T22:48:22.808872+0000","last_became_active":"2024-05-26T22:48:21.824372+0000","last_became_peered":"2024-05-26T22:48:21.824372+0000","last_unstale":"2024-05-26T22:48:22.808872+0000","last_undegraded":"2024-05-26T22:48:22.808872+0000","last_fullsized":"2024-05-26T22:48:22.808872+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:52:25.073281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.507397+0000","last_change":"2024-05-26T22:48:20.802334+0000","last_active":"2024-05-26T22:48:23.507397+0000","last_peered":"2024-05-26T22:48:23.507397+0000","last_clean":"2024-05-26T22:48:23.507397+0000","last_became_active":"2024-05-26T22:48:20.802051+0000","last_became_peered":"2024-05-26T22:48:20.802051+0000","last_unstale":"2024-05-26T22:48:23.507397+0000","last_undegraded":"2024-05-26T22:48:23.507397+0000","last_fullsized":"2024-05-26T22:48:23.507397+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:55:59.062939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.508279+0000","last_change":"2024-05-26T22:48:21.830682+0000","last_active":"2024-05-26T22:48:23.508279+0000","last_peered":"2024-05-26T22:48:23.508279+0000","last_clean":"2024-05-26T22:48:23.508279+0000","last_became_active":"2024-05-26T22:48:21.830273+0000","last_became_peered":"2024-05-26T22:48:21.830273+0000","last_unstale":"2024-05-26T22:48:23.508279+0000","last_undegraded":"2024-05-26T22:48:23.508279+0000","last_fullsized":"2024-05-26T22:48:23.508279+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:15:00.790746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.809215+0000","last_change":"2024-05-26T22:48:20.791028+0000","last_active":"2024-05-26T22:48:22.809215+0000","last_peered":"2024-05-26T22:48:22.809215+0000","last_clean":"2024-05-26T22:48:22.809215+0000","last_became_active":"2024-05-26T22:48:20.790727+0000","last_became_peered":"2024-05-26T22:48:20.790727+0000","last_unstale":"2024-05-26T22:48:22.809215+0000","last_undegraded":"2024-05-26T22:48:22.809215+0000","last_fullsized":"2024-05-26T22:48:22.809215+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:20:41.886676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.809552+0000","last_change":"2024-05-26T22:48:21.809785+0000","last_active":"2024-05-26T22:48:22.809552+0000","last_peered":"2024-05-26T22:48:22.809552+0000","last_clean":"2024-05-26T22:48:22.809552+0000","last_became_active":"2024-05-26T22:48:21.809460+0000","last_became_peered":"2024-05-26T22:48:21.809460+0000","last_unstale":"2024-05-26T22:48:22.809552+0000","last_undegraded":"2024-05-26T22:48:22.809552+0000","last_fullsized":"2024-05-26T22:48:22.809552+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T23:40:55.485829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.502702+0000","last_change":"2024-05-26T22:48:20.788462+0000","last_active":"2024-05-26T22:48:23.502702+0000","last_peered":"2024-05-26T22:48:23.502702+0000","last_clean":"2024-05-26T22:48:23.502702+0000","last_became_active":"2024-05-26T22:48:20.787714+0000","last_became_peered":"2024-05-26T22:48:20.787714+0000","last_unstale":"2024-05-26T22:48:23.502702+0000","last_undegraded":"2024-05-26T22:48:23.502702+0000","last_fullsized":"2024-05-26T22:48:23.502702+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:08:01.698812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.507359+0000","last_change":"2024-05-26T22:48:21.823217+0000","last_active":"2024-05-26T22:48:23.507359+0000","last_peered":"2024-05-26T22:48:23.507359+0000","last_clean":"2024-05-26T22:48:23.507359+0000","last_became_active":"2024-05-26T22:48:21.822856+0000","last_became_peered":"2024-05-26T22:48:21.822856+0000","last_unstale":"2024-05-26T22:48:23.507359+0000","last_undegraded":"2024-05-26T22:48:23.507359+0000","last_fullsized":"2024-05-26T22:48:23.507359+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:08:10.330236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.505497+0000","last_change":"2024-05-26T22:48:20.793786+0000","last_active":"2024-05-26T22:48:23.505497+0000","last_peered":"2024-05-26T22:48:23.505497+0000","last_clean":"2024-05-26T22:48:23.505497+0000","last_became_active":"2024-05-26T22:48:20.793201+0000","last_became_peered":"2024-05-26T22:48:20.793201+0000","last_unstale":"2024-05-26T22:48:23.505497+0000","last_undegraded":"2024-05-26T22:48:23.505497+0000","last_fullsized":"2024-05-26T22:48:23.505497+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:30:48.024402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.509387+0000","last_change":"2024-05-26T22:48:21.832244+0000","last_active":"2024-05-26T22:48:23.509387+0000","last_peered":"2024-05-26T22:48:23.509387+0000","last_clean":"2024-05-26T22:48:23.509387+0000","last_became_active":"2024-05-26T22:48:21.831897+0000","last_became_peered":"2024-05-26T22:48:21.831897+0000","last_unstale":"2024-05-26T22:48:23.509387+0000","last_undegraded":"2024-05-26T22:48:23.509387+0000","last_fullsized":"2024-05-26T22:48:23.509387+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:49:40.791149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.503336+0000","last_change":"2024-05-26T22:48:20.791496+0000","last_active":"2024-05-26T22:48:23.503336+0000","last_peered":"2024-05-26T22:48:23.503336+0000","last_clean":"2024-05-26T22:48:23.503336+0000","last_became_active":"2024-05-26T22:48:20.790743+0000","last_became_peered":"2024-05-26T22:48:20.790743+0000","last_unstale":"2024-05-26T22:48:23.503336+0000","last_undegraded":"2024-05-26T22:48:23.503336+0000","last_fullsized":"2024-05-26T22:48:23.503336+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:20:41.886676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.509655+0000","last_change":"2024-05-26T22:48:21.837602+0000","last_active":"2024-05-26T22:48:23.509655+0000","last_peered":"2024-05-26T22:48:23.509655+0000","last_clean":"2024-05-26T22:48:23.509655+0000","last_became_active":"2024-05-26T22:48:21.834873+0000","last_became_peered":"2024-05-26T22:48:21.834873+0000","last_unstale":"2024-05-26T22:48:23.509655+0000","last_undegraded":"2024-05-26T22:48:23.509655+0000","last_fullsized":"2024-05-26T22:48:23.509655+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:12:50.184011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.822772+0000","last_change":"2024-05-26T22:48:20.782935+0000","last_active":"2024-05-26T22:48:22.822772+0000","last_peered":"2024-05-26T22:48:22.822772+0000","last_clean":"2024-05-26T22:48:22.822772+0000","last_became_active":"2024-05-26T22:48:20.782624+0000","last_became_peered":"2024-05-26T22:48:20.782624+0000","last_unstale":"2024-05-26T22:48:22.822772+0000","last_undegraded":"2024-05-26T22:48:22.822772+0000","last_fullsized":"2024-05-26T22:48:22.822772+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:55:59.062939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.507335+0000","last_change":"2024-05-26T22:48:21.836045+0000","last_active":"2024-05-26T22:48:23.507335+0000","last_peered":"2024-05-26T22:48:23.507335+0000","last_clean":"2024-05-26T22:48:23.507335+0000","last_became_active":"2024-05-26T22:48:21.830070+0000","last_became_peered":"2024-05-26T22:48:21.830070+0000","last_unstale":"2024-05-26T22:48:23.507335+0000","last_undegraded":"2024-05-26T22:48:23.507335+0000","last_fullsized":"2024-05-26T22:48:23.507335+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:11:40.258537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.503246+0000","last_change":"2024-05-26T22:48:20.805086+0000","last_active":"2024-05-26T22:48:23.503246+0000","last_peered":"2024-05-26T22:48:23.503246+0000","last_clean":"2024-05-26T22:48:23.503246+0000","last_became_active":"2024-05-26T22:48:20.804732+0000","last_became_peered":"2024-05-26T22:48:20.804732+0000","last_unstale":"2024-05-26T22:48:23.503246+0000","last_undegraded":"2024-05-26T22:48:23.503246+0000","last_fullsized":"2024-05-26T22:48:23.503246+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:11:29.128727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.509008+0000","last_change":"2024-05-26T22:48:21.840364+0000","last_active":"2024-05-26T22:48:23.509008+0000","last_peered":"2024-05-26T22:48:23.509008+0000","last_clean":"2024-05-26T22:48:23.509008+0000","last_became_active":"2024-05-26T22:48:21.839692+0000","last_became_peered":"2024-05-26T22:48:21.839692+0000","last_unstale":"2024-05-26T22:48:23.509008+0000","last_undegraded":"2024-05-26T22:48:23.509008+0000","last_fullsized":"2024-05-26T22:48:23.509008+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:11:40.258537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.506053+0000","last_change":"2024-05-26T22:48:12.068917+0000","last_active":"2024-05-26T22:48:23.506053+0000","last_peered":"2024-05-26T22:48:23.506053+0000","last_clean":"2024-05-26T22:48:23.506053+0000","last_became_active":"2024-05-26T22:48:12.065934+0000","last_became_peered":"2024-05-26T22:48:12.065934+0000","last_unstale":"2024-05-26T22:48:23.506053+0000","last_undegraded":"2024-05-26T22:48:23.506053+0000","last_fullsized":"2024-05-26T22:48:23.506053+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:10.628122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:10.628122+0000","last_clean_scrub_stamp":"2024-05-26T22:48:10.628122+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:14:58.895911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.506186+0000","last_change":"2024-05-26T22:48:20.800517+0000","last_active":"2024-05-26T22:48:23.506186+0000","last_peered":"2024-05-26T22:48:23.506186+0000","last_clean":"2024-05-26T22:48:23.506186+0000","last_became_active":"2024-05-26T22:48:20.797956+0000","last_became_peered":"2024-05-26T22:48:20.797956+0000","last_unstale":"2024-05-26T22:48:23.506186+0000","last_undegraded":"2024-05-26T22:48:23.506186+0000","last_fullsized":"2024-05-26T22:48:23.506186+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:39:51.267408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.812649+0000","last_change":"2024-05-26T22:48:21.844543+0000","last_active":"2024-05-26T22:48:22.812649+0000","last_peered":"2024-05-26T22:48:22.812649+0000","last_clean":"2024-05-26T22:48:22.812649+0000","last_became_active":"2024-05-26T22:48:21.844272+0000","last_became_peered":"2024-05-26T22:48:21.844272+0000","last_unstale":"2024-05-26T22:48:22.812649+0000","last_undegraded":"2024-05-26T22:48:22.812649+0000","last_fullsized":"2024-05-26T22:48:22.812649+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:59:10.543916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.505812+0000","last_change":"2024-05-26T22:48:20.799727+0000","last_active":"2024-05-26T22:48:23.505812+0000","last_peered":"2024-05-26T22:48:23.505812+0000","last_clean":"2024-05-26T22:48:23.505812+0000","last_became_active":"2024-05-26T22:48:20.799427+0000","last_became_peered":"2024-05-26T22:48:20.799427+0000","last_unstale":"2024-05-26T22:48:23.505812+0000","last_undegraded":"2024-05-26T22:48:23.505812+0000","last_fullsized":"2024-05-26T22:48:23.505812+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:55:59.062939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.808326+0000","last_change":"2024-05-26T22:48:21.832212+0000","last_active":"2024-05-26T22:48:22.808326+0000","last_peered":"2024-05-26T22:48:22.808326+0000","last_clean":"2024-05-26T22:48:22.808326+0000","last_became_active":"2024-05-26T22:48:21.831893+0000","last_became_peered":"2024-05-26T22:48:21.831893+0000","last_unstale":"2024-05-26T22:48:22.808326+0000","last_undegraded":"2024-05-26T22:48:22.808326+0000","last_fullsized":"2024-05-26T22:48:22.808326+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T23:40:55.485829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.502923+0000","last_change":"2024-05-26T22:48:20.785204+0000","last_active":"2024-05-26T22:48:23.502923+0000","last_peered":"2024-05-26T22:48:23.502923+0000","last_clean":"2024-05-26T22:48:23.502923+0000","last_became_active":"2024-05-26T22:48:20.784798+0000","last_became_peered":"2024-05-26T22:48:20.784798+0000","last_unstale":"2024-05-26T22:48:23.502923+0000","last_undegraded":"2024-05-26T22:48:23.502923+0000","last_fullsized":"2024-05-26T22:48:23.502923+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:30:48.024402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.822381+0000","last_change":"2024-05-26T22:48:21.811298+0000","last_active":"2024-05-26T22:48:22.822381+0000","last_peered":"2024-05-26T22:48:22.822381+0000","last_clean":"2024-05-26T22:48:22.822381+0000","last_became_active":"2024-05-26T22:48:21.807981+0000","last_became_peered":"2024-05-26T22:48:21.807981+0000","last_unstale":"2024-05-26T22:48:22.822381+0000","last_undegraded":"2024-05-26T22:48:22.822381+0000","last_fullsized":"2024-05-26T22:48:22.822381+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:27:34.959431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.507652+0000","last_change":"2024-05-26T22:48:20.794578+0000","last_active":"2024-05-26T22:48:23.507652+0000","last_peered":"2024-05-26T22:48:23.507652+0000","last_clean":"2024-05-26T22:48:23.507652+0000","last_became_active":"2024-05-26T22:48:20.794262+0000","last_became_peered":"2024-05-26T22:48:20.794262+0000","last_unstale":"2024-05-26T22:48:23.507652+0000","last_undegraded":"2024-05-26T22:48:23.507652+0000","last_fullsized":"2024-05-26T22:48:23.507652+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:16:01.106855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.505213+0000","last_change":"2024-05-26T22:48:21.826194+0000","last_active":"2024-05-26T22:48:23.505213+0000","last_peered":"2024-05-26T22:48:23.505213+0000","last_clean":"2024-05-26T22:48:23.505213+0000","last_became_active":"2024-05-26T22:48:21.825674+0000","last_became_peered":"2024-05-26T22:48:21.825674+0000","last_unstale":"2024-05-26T22:48:23.505213+0000","last_undegraded":"2024-05-26T22:48:23.505213+0000","last_fullsized":"2024-05-26T22:48:23.505213+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:01:59.805532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.822681+0000","last_change":"2024-05-26T22:48:20.788924+0000","last_active":"2024-05-26T22:48:22.822681+0000","last_peered":"2024-05-26T22:48:22.822681+0000","last_clean":"2024-05-26T22:48:22.822681+0000","last_became_active":"2024-05-26T22:48:20.788036+0000","last_became_peered":"2024-05-26T22:48:20.788036+0000","last_unstale":"2024-05-26T22:48:22.822681+0000","last_undegraded":"2024-05-26T22:48:22.822681+0000","last_fullsized":"2024-05-26T22:48:22.822681+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:10:49.991129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.507362+0000","last_change":"2024-05-26T22:48:21.833223+0000","last_active":"2024-05-26T22:48:23.507362+0000","last_peered":"2024-05-26T22:48:23.507362+0000","last_clean":"2024-05-26T22:48:23.507362+0000","last_became_active":"2024-05-26T22:48:21.831371+0000","last_became_peered":"2024-05-26T22:48:21.831371+0000","last_unstale":"2024-05-26T22:48:23.507362+0000","last_undegraded":"2024-05-26T22:48:23.507362+0000","last_fullsized":"2024-05-26T22:48:23.507362+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:45:06.922028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.509127+0000","last_change":"2024-05-26T22:48:20.800026+0000","last_active":"2024-05-26T22:48:23.509127+0000","last_peered":"2024-05-26T22:48:23.509127+0000","last_clean":"2024-05-26T22:48:23.509127+0000","last_became_active":"2024-05-26T22:48:20.799776+0000","last_became_peered":"2024-05-26T22:48:20.799776+0000","last_unstale":"2024-05-26T22:48:23.509127+0000","last_undegraded":"2024-05-26T22:48:23.509127+0000","last_fullsized":"2024-05-26T22:48:23.509127+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:33:50.638333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.822718+0000","last_change":"2024-05-26T22:48:21.811868+0000","last_active":"2024-05-26T22:48:22.822718+0000","last_peered":"2024-05-26T22:48:22.822718+0000","last_clean":"2024-05-26T22:48:22.822718+0000","last_became_active":"2024-05-26T22:48:21.811205+0000","last_became_peered":"2024-05-26T22:48:21.811205+0000","last_unstale":"2024-05-26T22:48:22.822718+0000","last_undegraded":"2024-05-26T22:48:22.822718+0000","last_fullsized":"2024-05-26T22:48:22.822718+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:43:21.391197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.823108+0000","last_change":"2024-05-26T22:48:20.786141+0000","last_active":"2024-05-26T22:48:22.823108+0000","last_peered":"2024-05-26T22:48:22.823108+0000","last_clean":"2024-05-26T22:48:22.823108+0000","last_became_active":"2024-05-26T22:48:20.785581+0000","last_became_peered":"2024-05-26T22:48:20.785581+0000","last_unstale":"2024-05-26T22:48:22.823108+0000","last_undegraded":"2024-05-26T22:48:22.823108+0000","last_fullsized":"2024-05-26T22:48:22.823108+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:16:01.106855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.812734+0000","last_change":"2024-05-26T22:48:21.828936+0000","last_active":"2024-05-26T22:48:22.812734+0000","last_peered":"2024-05-26T22:48:22.812734+0000","last_clean":"2024-05-26T22:48:22.812734+0000","last_became_active":"2024-05-26T22:48:21.828635+0000","last_became_peered":"2024-05-26T22:48:21.828635+0000","last_unstale":"2024-05-26T22:48:22.812734+0000","last_undegraded":"2024-05-26T22:48:22.812734+0000","last_fullsized":"2024-05-26T22:48:22.812734+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:21:52.108732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.810327+0000","last_change":"2024-05-26T22:48:20.798700+0000","last_active":"2024-05-26T22:48:22.810327+0000","last_peered":"2024-05-26T22:48:22.810327+0000","last_clean":"2024-05-26T22:48:22.810327+0000","last_became_active":"2024-05-26T22:48:20.796914+0000","last_became_peered":"2024-05-26T22:48:20.796914+0000","last_unstale":"2024-05-26T22:48:22.810327+0000","last_undegraded":"2024-05-26T22:48:22.810327+0000","last_fullsized":"2024-05-26T22:48:22.810327+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:10:49.991129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.809579+0000","last_change":"2024-05-26T22:48:21.811239+0000","last_active":"2024-05-26T22:48:22.809579+0000","last_peered":"2024-05-26T22:48:22.809579+0000","last_clean":"2024-05-26T22:48:22.809579+0000","last_became_active":"2024-05-26T22:48:21.810945+0000","last_became_peered":"2024-05-26T22:48:21.810945+0000","last_unstale":"2024-05-26T22:48:22.809579+0000","last_undegraded":"2024-05-26T22:48:22.809579+0000","last_fullsized":"2024-05-26T22:48:22.809579+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:21:52.108732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.505531+0000","last_change":"2024-05-26T22:48:20.802835+0000","last_active":"2024-05-26T22:48:23.505531+0000","last_peered":"2024-05-26T22:48:23.505531+0000","last_clean":"2024-05-26T22:48:23.505531+0000","last_became_active":"2024-05-26T22:48:20.802501+0000","last_became_peered":"2024-05-26T22:48:20.802501+0000","last_unstale":"2024-05-26T22:48:23.505531+0000","last_undegraded":"2024-05-26T22:48:23.505531+0000","last_fullsized":"2024-05-26T22:48:23.505531+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:39:51.267408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.504467+0000","last_change":"2024-05-26T22:48:21.832723+0000","last_active":"2024-05-26T22:48:23.504467+0000","last_peered":"2024-05-26T22:48:23.504467+0000","last_clean":"2024-05-26T22:48:23.504467+0000","last_became_active":"2024-05-26T22:48:21.824388+0000","last_became_peered":"2024-05-26T22:48:21.824388+0000","last_unstale":"2024-05-26T22:48:23.504467+0000","last_undegraded":"2024-05-26T22:48:23.504467+0000","last_fullsized":"2024-05-26T22:48:23.504467+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:56:00.075959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.822800+0000","last_change":"2024-05-26T22:48:20.797481+0000","last_active":"2024-05-26T22:48:22.822800+0000","last_peered":"2024-05-26T22:48:22.822800+0000","last_clean":"2024-05-26T22:48:22.822800+0000","last_became_active":"2024-05-26T22:48:20.797142+0000","last_became_peered":"2024-05-26T22:48:20.797142+0000","last_unstale":"2024-05-26T22:48:22.822800+0000","last_undegraded":"2024-05-26T22:48:22.822800+0000","last_fullsized":"2024-05-26T22:48:22.822800+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:08:01.698812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.812496+0000","last_change":"2024-05-26T22:48:21.819388+0000","last_active":"2024-05-26T22:48:22.812496+0000","last_peered":"2024-05-26T22:48:22.812496+0000","last_clean":"2024-05-26T22:48:22.812496+0000","last_became_active":"2024-05-26T22:48:21.817847+0000","last_became_peered":"2024-05-26T22:48:21.817847+0000","last_unstale":"2024-05-26T22:48:22.812496+0000","last_undegraded":"2024-05-26T22:48:22.812496+0000","last_fullsized":"2024-05-26T22:48:22.812496+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:38:00.344955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.822809+0000","last_change":"2024-05-26T22:48:20.785751+0000","last_active":"2024-05-26T22:48:22.822809+0000","last_peered":"2024-05-26T22:48:22.822809+0000","last_clean":"2024-05-26T22:48:22.822809+0000","last_became_active":"2024-05-26T22:48:20.785494+0000","last_became_peered":"2024-05-26T22:48:20.785494+0000","last_unstale":"2024-05-26T22:48:22.822809+0000","last_undegraded":"2024-05-26T22:48:22.822809+0000","last_fullsized":"2024-05-26T22:48:22.822809+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:20:41.886676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.812182+0000","last_change":"2024-05-26T22:48:21.825344+0000","last_active":"2024-05-26T22:48:22.812182+0000","last_peered":"2024-05-26T22:48:22.812182+0000","last_clean":"2024-05-26T22:48:22.812182+0000","last_became_active":"2024-05-26T22:48:21.823452+0000","last_became_peered":"2024-05-26T22:48:21.823452+0000","last_unstale":"2024-05-26T22:48:22.812182+0000","last_undegraded":"2024-05-26T22:48:22.812182+0000","last_fullsized":"2024-05-26T22:48:22.812182+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:52:25.073281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.508330+0000","last_change":"2024-05-26T22:48:20.803962+0000","last_active":"2024-05-26T22:48:23.508330+0000","last_peered":"2024-05-26T22:48:23.508330+0000","last_clean":"2024-05-26T22:48:23.508330+0000","last_became_active":"2024-05-26T22:48:20.803609+0000","last_became_peered":"2024-05-26T22:48:20.803609+0000","last_unstale":"2024-05-26T22:48:23.508330+0000","last_undegraded":"2024-05-26T22:48:23.508330+0000","last_fullsized":"2024-05-26T22:48:23.508330+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:16:01.106855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.809871+0000","last_change":"2024-05-26T22:48:21.829756+0000","last_active":"2024-05-26T22:48:22.809871+0000","last_peered":"2024-05-26T22:48:22.809871+0000","last_clean":"2024-05-26T22:48:22.809871+0000","last_became_active":"2024-05-26T22:48:21.829451+0000","last_became_peered":"2024-05-26T22:48:21.829451+0000","last_unstale":"2024-05-26T22:48:22.809871+0000","last_undegraded":"2024-05-26T22:48:22.809871+0000","last_fullsized":"2024-05-26T22:48:22.809871+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:43:21.391197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.506838+0000","last_change":"2024-05-26T22:48:21.832211+0000","last_active":"2024-05-26T22:48:23.506838+0000","last_peered":"2024-05-26T22:48:23.506838+0000","last_clean":"2024-05-26T22:48:23.506838+0000","last_became_active":"2024-05-26T22:48:21.831732+0000","last_became_peered":"2024-05-26T22:48:21.831732+0000","last_unstale":"2024-05-26T22:48:23.506838+0000","last_undegraded":"2024-05-26T22:48:23.506838+0000","last_fullsized":"2024-05-26T22:48:23.506838+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:16:22.847663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.505649+0000","last_change":"2024-05-26T22:48:20.805024+0000","last_active":"2024-05-26T22:48:23.505649+0000","last_peered":"2024-05-26T22:48:23.505649+0000","last_clean":"2024-05-26T22:48:23.505649+0000","last_became_active":"2024-05-26T22:48:20.804736+0000","last_became_peered":"2024-05-26T22:48:20.804736+0000","last_unstale":"2024-05-26T22:48:23.505649+0000","last_undegraded":"2024-05-26T22:48:23.505649+0000","last_fullsized":"2024-05-26T22:48:23.505649+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:42:31.965689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.506861+0000","last_change":"2024-05-26T22:48:21.832952+0000","last_active":"2024-05-26T22:48:23.506861+0000","last_peered":"2024-05-26T22:48:23.506861+0000","last_clean":"2024-05-26T22:48:23.506861+0000","last_became_active":"2024-05-26T22:48:21.831488+0000","last_became_peered":"2024-05-26T22:48:21.831488+0000","last_unstale":"2024-05-26T22:48:23.506861+0000","last_undegraded":"2024-05-26T22:48:23.506861+0000","last_fullsized":"2024-05-26T22:48:23.506861+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:15:00.790746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:23.508687+0000","last_change":"2024-05-26T22:48:20.799474+0000","last_active":"2024-05-26T22:48:23.508687+0000","last_peered":"2024-05-26T22:48:23.508687+0000","last_clean":"2024-05-26T22:48:23.508687+0000","last_became_active":"2024-05-26T22:48:20.799031+0000","last_became_peered":"2024-05-26T22:48:20.799031+0000","last_unstale":"2024-05-26T22:48:23.508687+0000","last_undegraded":"2024-05-26T22:48:23.508687+0000","last_fullsized":"2024-05-26T22:48:23.508687+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:01:58.792512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.809810+0000","last_change":"2024-05-26T22:48:21.837774+0000","last_active":"2024-05-26T22:48:22.809810+0000","last_peered":"2024-05-26T22:48:22.809810+0000","last_clean":"2024-05-26T22:48:22.809810+0000","last_became_active":"2024-05-26T22:48:21.837450+0000","last_became_peered":"2024-05-26T22:48:21.837450+0000","last_unstale":"2024-05-26T22:48:22.809810+0000","last_undegraded":"2024-05-26T22:48:22.809810+0000","last_fullsized":"2024-05-26T22:48:22.809810+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:33:39.923721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.812718+0000","last_change":"2024-05-26T22:48:20.788987+0000","last_active":"2024-05-26T22:48:22.812718+0000","last_peered":"2024-05-26T22:48:22.812718+0000","last_clean":"2024-05-26T22:48:22.812718+0000","last_became_active":"2024-05-26T22:48:20.788612+0000","last_became_peered":"2024-05-26T22:48:20.788612+0000","last_unstale":"2024-05-26T22:48:22.812718+0000","last_undegraded":"2024-05-26T22:48:22.812718+0000","last_fullsized":"2024-05-26T22:48:22.812718+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:20:41.886676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.812548+0000","last_change":"2024-05-26T22:48:21.826820+0000","last_active":"2024-05-26T22:48:22.812548+0000","last_peered":"2024-05-26T22:48:22.812548+0000","last_clean":"2024-05-26T22:48:22.812548+0000","last_became_active":"2024-05-26T22:48:21.826593+0000","last_became_peered":"2024-05-26T22:48:21.826593+0000","last_unstale":"2024-05-26T22:48:22.812548+0000","last_undegraded":"2024-05-26T22:48:22.812548+0000","last_fullsized":"2024-05-26T22:48:22.812548+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:20.748865+0000","last_clean_scrub_stamp":"2024-05-26T22:48:20.748865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:59:43.907681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-26T22:48:22.809942+0000","last_change":"2024-05-26T22:48:20.796965+0000","last_active":"2024-05-26T22:48:22.809942+0000","last_peered":"2024-05-26T22:48:22.809942+0000","last_clean":"2024-05-26T22:48:22.809942+0000","last_became_active":"2024-05-26T22:48:20.796587+0000","last_became_peered":"2024-05-26T22:48:20.796587+0000","last_unstale":"2024-05-26T22:48:22.809942+0000","last_undegraded":"2024-05-26T22:48:22.809942+0000","last_fullsized":"2024-05-26T22:48:22.809942+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:19.735845+0000","last_clean_scrub_stamp":"2024-05-26T22:48:19.735845+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:11:29.128727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-26T22:48:32.437 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-26T22:48:32.437 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-05-26T22:48:32.437 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-05-26T22:48:32.438 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-05-26T22:48:32.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.667+0000 7f4623200640 1 Processor -- start 2024-05-26T22:48:32.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.667+0000 7f4623200640 1 -- start start 2024-05-26T22:48:32.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.669+0000 7f4623200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f462414c090 0x7f46240b2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:32.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.669+0000 7f4623200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f462406df90 0x7f4624071160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:32.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.669+0000 7f4623200640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4624071990 con 0x7f46240cc760 2024-05-26T22:48:32.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.669+0000 7f4623200640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f4624071f50 con 0x7f462406df90 2024-05-26T22:48:32.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.669+0000 7f4623200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f46240ce150 con 0x7f462414c090 2024-05-26T22:48:32.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.669+0000 7f4621400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f462406df90 0x7f4624071160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:32.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.669+0000 7f4622800640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f46240cc760 0x7f46240ce5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:48998/0 (socket says 172.21.15.131:48998) 2024-05-26T22:48:32.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.669+0000 7f4621400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f462406df90 0x7f4624071160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54412/0 (socket says 172.21.15.131:54412) 2024-05-26T22:48:32.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.669+0000 7f4622800640 1 -- 172.21.15.131:0/285924623 learned_addr learned my addr 172.21.15.131:0/285924623 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:48:32.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.670+0000 7f4621e00640 1 --2- 172.21.15.131:0/285924623 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f462414c090 0x7f46240b2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:32.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.670+0000 7f4621400640 1 -- 172.21.15.131:0/285924623 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f462414c090 msgr2=0x7f46240b2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:32.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.670+0000 7f4621400640 1 --2- 172.21.15.131:0/285924623 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f462414c090 0x7f46240b2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:32.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.670+0000 7f4621400640 1 -- 172.21.15.131:0/285924623 >> v1:172.21.15.131:6789/0 conn(0x7f46240cc760 legacy=0x7f46240ce5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:32.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.670+0000 7f4621400640 1 -- 172.21.15.131:0/285924623 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4624071c70 con 0x7f462406df90 2024-05-26T22:48:32.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.671+0000 7f4621400640 1 --2- 172.21.15.131:0/285924623 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f462406df90 0x7f4624071160 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f460c0097d0 tx=0x7f460c02a120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd42cac863170d60 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:32.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.671+0000 7f4620a00640 1 -- 172.21.15.131:0/285924623 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f460c037070 con 0x7f462406df90 2024-05-26T22:48:32.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.671+0000 7f4620a00640 1 -- 172.21.15.131:0/285924623 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f460c02ae30 con 0x7f462406df90 2024-05-26T22:48:32.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.671+0000 7f4620a00640 1 -- 172.21.15.131:0/285924623 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f460c032640 con 0x7f462406df90 2024-05-26T22:48:32.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.672+0000 7f4623200640 1 -- 172.21.15.131:0/285924623 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f462406df90 msgr2=0x7f4624071160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:32.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.672+0000 7f4623200640 1 --2- 172.21.15.131:0/285924623 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f462406df90 0x7f4624071160 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f460c0097d0 tx=0x7f460c02a120 comp rx=0 tx=0).stop 2024-05-26T22:48:32.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.672+0000 7f4623200640 1 -- 172.21.15.131:0/285924623 shutdown_connections 2024-05-26T22:48:32.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.672+0000 7f4623200640 1 --2- 172.21.15.131:0/285924623 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f462406df90 0x7f4624071160 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:32.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.672+0000 7f4623200640 1 --2- 172.21.15.131:0/285924623 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f462414c090 0x7f46240b2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:32.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.672+0000 7f4623200640 1 -- 172.21.15.131:0/285924623 >> 172.21.15.131:0/285924623 conn(0x7f46240cff40 msgr2=0x7f46240d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:32.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.672+0000 7f4623200640 1 -- 172.21.15.131:0/285924623 shutdown_connections 2024-05-26T22:48:32.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.672+0000 7f4623200640 1 -- 172.21.15.131:0/285924623 wait complete. 2024-05-26T22:48:32.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.674+0000 7f4623200640 1 Processor -- start 2024-05-26T22:48:32.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.674+0000 7f4623200640 1 -- start start 2024-05-26T22:48:32.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.674+0000 7f4623200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f462406df90 0x7f46240b1e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:32.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.675+0000 7f4623200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f46240cc760 0x7f46241f7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:32.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.675+0000 7f4623200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f462414c090 0x7f4624154b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:32.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.675+0000 7f4623200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f462414c470 con 0x7f462406df90 2024-05-26T22:48:32.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.675+0000 7f4623200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f46240716a0 con 0x7f462414c090 2024-05-26T22:48:32.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.675+0000 7f4623200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f462414c5f0 con 0x7f46240cc760 2024-05-26T22:48:32.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.675+0000 7f4621e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f462406df90 0x7f46240b1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:32.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.675+0000 7f4621e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f462406df90 0x7f46240b1e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54428/0 (socket says 172.21.15.131:54428) 2024-05-26T22:48:32.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.675+0000 7f4621e00640 1 -- 172.21.15.131:0/803583579 learned_addr learned my addr 172.21.15.131:0/803583579 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:32.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.675+0000 7f4622800640 1 --2- 172.21.15.131:0/803583579 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f462414c090 0x7f4624154b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:32.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.675+0000 7f4621400640 1 --2- 172.21.15.131:0/803583579 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f46240cc760 0x7f46241f7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:32.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.675+0000 7f4621e00640 1 -- 172.21.15.131:0/803583579 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f46240cc760 msgr2=0x7f46241f7ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:32.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.675+0000 7f4621e00640 1 --2- 172.21.15.131:0/803583579 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f46240cc760 0x7f46241f7ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:32.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.675+0000 7f4621e00640 1 -- 172.21.15.131:0/803583579 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f462414c090 msgr2=0x7f4624154b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:32.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.675+0000 7f4621e00640 1 --2- 172.21.15.131:0/803583579 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f462414c090 0x7f4624154b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:32.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.675+0000 7f4621e00640 1 -- 172.21.15.131:0/803583579 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4624152ba0 con 0x7f462406df90 2024-05-26T22:48:32.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.675+0000 7f4621e00640 1 --2- 172.21.15.131:0/803583579 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f462406df90 0x7f46240b1e20 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f461c00d620 tx=0x7f461c00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:32.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.676+0000 7f461a000640 1 -- 172.21.15.131:0/803583579 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f461c00b620 con 0x7f462406df90 2024-05-26T22:48:32.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.676+0000 7f461a000640 1 -- 172.21.15.131:0/803583579 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f461c019070 con 0x7f462406df90 2024-05-26T22:48:32.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.676+0000 7f461a000640 1 -- 172.21.15.131:0/803583579 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f461c00ba00 con 0x7f462406df90 2024-05-26T22:48:32.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.676+0000 7f4623200640 1 -- 172.21.15.131:0/803583579 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4624152e60 con 0x7f462406df90 2024-05-26T22:48:32.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.676+0000 7f4623200640 1 -- 172.21.15.131:0/803583579 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4624153420 con 0x7f462406df90 2024-05-26T22:48:32.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.677+0000 7f4623200640 1 -- 172.21.15.131:0/803583579 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46240694e0 con 0x7f462406df90 2024-05-26T22:48:32.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.678+0000 7f461a000640 1 -- 172.21.15.131:0/803583579 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f461c00bba0 con 0x7f462406df90 2024-05-26T22:48:32.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.679+0000 7f461a000640 1 --2- 172.21.15.131:0/803583579 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f460006be30 0x7f460006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:32.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.679+0000 7f461a000640 1 -- 172.21.15.131:0/803583579 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f461c08b310 con 0x7f462406df90 2024-05-26T22:48:32.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.679+0000 7f4621400640 1 --2- 172.21.15.131:0/803583579 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f460006be30 0x7f460006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:32.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.679+0000 7f4621400640 1 --2- 172.21.15.131:0/803583579 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f460006be30 0x7f460006e2f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f460c034000 tx=0x7f460c02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:32.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.683+0000 7f461a000640 1 -- 172.21.15.131:0/803583579 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f461c059c20 con 0x7f462406df90 2024-05-26T22:48:32.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.901+0000 7f4623200640 1 -- 172.21.15.131:0/803583579 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f462406eb90 con 0x7f462406df90 2024-05-26T22:48:32.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.902+0000 7f461a000640 1 -- 172.21.15.131:0/803583579 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f461c059520 con 0x7f462406df90 2024-05-26T22:48:32.901 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:48:32.902 INFO:teuthology.orchestra.run.smithi131.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-26T22:48:32.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.904+0000 7f4623200640 1 -- 172.21.15.131:0/803583579 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f460006be30 msgr2=0x7f460006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:32.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.904+0000 7f4623200640 1 --2- 172.21.15.131:0/803583579 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f460006be30 0x7f460006e2f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f460c034000 tx=0x7f460c02e000 comp rx=0 tx=0).stop 2024-05-26T22:48:32.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.904+0000 7f4623200640 1 -- 172.21.15.131:0/803583579 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f462406df90 msgr2=0x7f46240b1e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:32.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.904+0000 7f4623200640 1 --2- 172.21.15.131:0/803583579 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f462406df90 0x7f46240b1e20 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f461c00d620 tx=0x7f461c00dae0 comp rx=0 tx=0).stop 2024-05-26T22:48:32.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.904+0000 7f4623200640 1 -- 172.21.15.131:0/803583579 shutdown_connections 2024-05-26T22:48:32.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.904+0000 7f4623200640 1 --2- 172.21.15.131:0/803583579 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f46240cc760 0x7f46241f7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:32.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.904+0000 7f4623200640 1 --2- 172.21.15.131:0/803583579 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f460006be30 0x7f460006e2f0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:32.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.904+0000 7f4623200640 1 --2- 172.21.15.131:0/803583579 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f462414c090 0x7f4624154b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:32.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.904+0000 7f4623200640 1 --2- 172.21.15.131:0/803583579 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f462406df90 0x7f46240b1e20 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:32.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.904+0000 7f4623200640 1 -- 172.21.15.131:0/803583579 >> 172.21.15.131:0/803583579 conn(0x7f46240cff40 msgr2=0x7f46241653d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:32.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.904+0000 7f4623200640 1 -- 172.21.15.131:0/803583579 shutdown_connections 2024-05-26T22:48:32.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:32.904+0000 7f4623200640 1 -- 172.21.15.131:0/803583579 wait complete. 2024-05-26T22:48:32.923 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-05-26T22:48:32.923 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-26T22:48:33.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.128+0000 7f4139800640 1 Processor -- start 2024-05-26T22:48:33.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.128+0000 7f4139800640 1 -- start start 2024-05-26T22:48:33.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.130+0000 7f4139800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f413415b1c0 0x7f41340b4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:33.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.130+0000 7f4139800640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f41340b51c0 0x7f413406e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:33.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.130+0000 7f4139800640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f413406c420 con 0x7f41340ce180 2024-05-26T22:48:33.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.130+0000 7f4139800640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f413406da30 con 0x7f41340b51c0 2024-05-26T22:48:33.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.130+0000 7f4139800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f413406c6c0 con 0x7f413415b1c0 2024-05-26T22:48:33.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.131+0000 7f4138e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f41340b51c0 0x7f413406e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:33.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.131+0000 7f4138e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f41340b51c0 0x7f413406e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54440/0 (socket says 172.21.15.131:54440) 2024-05-26T22:48:33.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.131+0000 7f4138e00640 1 -- 172.21.15.131:0/287066336 learned_addr learned my addr 172.21.15.131:0/287066336 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:33.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.131+0000 7f4133e00640 1 --2- 172.21.15.131:0/287066336 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f413415b1c0 0x7f41340b4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:33.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.131+0000 7f4138e00640 1 -- 172.21.15.131:0/287066336 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f413415b1c0 msgr2=0x7f41340b4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:33.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.131+0000 7f4138e00640 1 --2- 172.21.15.131:0/287066336 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f413415b1c0 0x7f41340b4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:33.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.131+0000 7f4138e00640 1 -- 172.21.15.131:0/287066336 >> v1:172.21.15.131:6789/0 conn(0x7f41340ce180 legacy=0x7f41340b4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:33.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.131+0000 7f4138e00640 1 -- 172.21.15.131:0/287066336 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f413406c9a0 con 0x7f41340b51c0 2024-05-26T22:48:33.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.133+0000 7f4138e00640 1 --2- 172.21.15.131:0/287066336 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f41340b51c0 0x7f413406e4d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f412c009e50 tx=0x7f412c02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b74298140d25648 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:33.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.133+0000 7f4132a00640 1 -- 172.21.15.131:0/287066336 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f412c036070 con 0x7f41340b51c0 2024-05-26T22:48:33.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.133+0000 7f4132a00640 1 -- 172.21.15.131:0/287066336 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f412c031600 con 0x7f41340b51c0 2024-05-26T22:48:33.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.133+0000 7f4132a00640 1 -- 172.21.15.131:0/287066336 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f412c0319d0 con 0x7f41340b51c0 2024-05-26T22:48:33.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.133+0000 7f4139800640 1 -- 172.21.15.131:0/287066336 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f41340b51c0 msgr2=0x7f413406e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:33.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.133+0000 7f4139800640 1 --2- 172.21.15.131:0/287066336 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f41340b51c0 0x7f413406e4d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f412c009e50 tx=0x7f412c02a710 comp rx=0 tx=0).stop 2024-05-26T22:48:33.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.134+0000 7f4139800640 1 -- 172.21.15.131:0/287066336 shutdown_connections 2024-05-26T22:48:33.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.134+0000 7f4139800640 1 --2- 172.21.15.131:0/287066336 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f41340b51c0 0x7f413406e4d0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:33.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.134+0000 7f4139800640 1 --2- 172.21.15.131:0/287066336 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f413415b1c0 0x7f41340b4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:33.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.134+0000 7f4139800640 1 -- 172.21.15.131:0/287066336 >> 172.21.15.131:0/287066336 conn(0x7f41340d07b0 msgr2=0x7f41340d0b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:33.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.134+0000 7f4139800640 1 -- 172.21.15.131:0/287066336 shutdown_connections 2024-05-26T22:48:33.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.134+0000 7f4139800640 1 -- 172.21.15.131:0/287066336 wait complete. 2024-05-26T22:48:33.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.135+0000 7f4139800640 1 Processor -- start 2024-05-26T22:48:33.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.136+0000 7f4139800640 1 -- start start 2024-05-26T22:48:33.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.136+0000 7f4139800640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f41340b51c0 0x7f413414d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:33.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.136+0000 7f4139800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f41340ce180 0x7f413415da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:33.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.136+0000 7f4139800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f413415b1c0 0x7f4134155230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:33.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.136+0000 7f4139800640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f41340675a0 con 0x7f413415b1c0 2024-05-26T22:48:33.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.136+0000 7f4139800640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f413406dfb0 con 0x7f41340b51c0 2024-05-26T22:48:33.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.136+0000 7f4139800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f41340b4770 con 0x7f41340ce180 2024-05-26T22:48:33.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.137+0000 7f4138e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f413415b1c0 0x7f4134155230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:33.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.137+0000 7f4138e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f413415b1c0 0x7f4134155230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54454/0 (socket says 172.21.15.131:54454) 2024-05-26T22:48:33.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.137+0000 7f4138e00640 1 -- 172.21.15.131:0/1703850090 learned_addr learned my addr 172.21.15.131:0/1703850090 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:33.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.137+0000 7f4133e00640 1 --2- 172.21.15.131:0/1703850090 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f41340b51c0 0x7f413414d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:33.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.137+0000 7f4133400640 1 --2- 172.21.15.131:0/1703850090 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f41340ce180 0x7f413415da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:33.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.137+0000 7f4138e00640 1 -- 172.21.15.131:0/1703850090 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f41340ce180 msgr2=0x7f413415da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:33.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.137+0000 7f4138e00640 1 --2- 172.21.15.131:0/1703850090 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f41340ce180 0x7f413415da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:33.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.137+0000 7f4138e00640 1 -- 172.21.15.131:0/1703850090 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f41340b51c0 msgr2=0x7f413414d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:33.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.137+0000 7f4138e00640 1 --2- 172.21.15.131:0/1703850090 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f41340b51c0 0x7f413414d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:33.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.137+0000 7f4138e00640 1 -- 172.21.15.131:0/1703850090 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41341546b0 con 0x7f413415b1c0 2024-05-26T22:48:33.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.137+0000 7f4133400640 1 --2- 172.21.15.131:0/1703850090 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f41340ce180 0x7f413415da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:48:33.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.137+0000 7f4138e00640 1 --2- 172.21.15.131:0/1703850090 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f413415b1c0 0x7f4134155230 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f412c009e50 tx=0x7f412c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:33.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.138+0000 7f412be00640 1 -- 172.21.15.131:0/1703850090 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f412c036050 con 0x7f413415b1c0 2024-05-26T22:48:33.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.138+0000 7f412be00640 1 -- 172.21.15.131:0/1703850090 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f412c042070 con 0x7f413415b1c0 2024-05-26T22:48:33.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.138+0000 7f4139800640 1 -- 172.21.15.131:0/1703850090 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41340af6a0 con 0x7f413415b1c0 2024-05-26T22:48:33.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.138+0000 7f412be00640 1 -- 172.21.15.131:0/1703850090 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f412c009510 con 0x7f413415b1c0 2024-05-26T22:48:33.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.138+0000 7f4139800640 1 -- 172.21.15.131:0/1703850090 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f413415d250 con 0x7f413415b1c0 2024-05-26T22:48:33.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.139+0000 7f4139800640 1 -- 172.21.15.131:0/1703850090 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41340694e0 con 0x7f413415b1c0 2024-05-26T22:48:33.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.140+0000 7f412be00640 1 -- 172.21.15.131:0/1703850090 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f412c007bd0 con 0x7f413415b1c0 2024-05-26T22:48:33.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.141+0000 7f412be00640 1 --2- 172.21.15.131:0/1703850090 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f411406be30 0x7f411406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:33.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.141+0000 7f412be00640 1 -- 172.21.15.131:0/1703850090 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f412c0abd00 con 0x7f413415b1c0 2024-05-26T22:48:33.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.141+0000 7f4133e00640 1 --2- 172.21.15.131:0/1703850090 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f411406be30 0x7f411406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:33.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.142+0000 7f4133e00640 1 --2- 172.21.15.131:0/1703850090 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f411406be30 0x7f411406e2f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4124006fd0 tx=0x7f4124008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:33.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.145+0000 7f412be00640 1 -- 172.21.15.131:0/1703850090 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f412c07a6a0 con 0x7f413415b1c0 2024-05-26T22:48:33.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.344+0000 7f4139800640 1 -- 172.21.15.131:0/1703850090 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f413406f960 con 0x7f413415b1c0 2024-05-26T22:48:33.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.348+0000 7f412be00640 1 -- 172.21.15.131:0/1703850090 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16629 (secure 0 0 0) 0x7f412c079fa0 con 0x7f413415b1c0 2024-05-26T22:48:33.348 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:48:33.348 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":20,"fsid":"46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f","created":"2024-05-26T22:47:57.288905+0000","modified":"2024-05-26T22:48:22.799298+0000","last_up_change":"2024-05-26T22:48:09.602207+0000","last_in_change":"2024-05-26T22:48:00.687341+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-26T22:48:10.463339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-26T22:48:19.663254+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-26T22:48:20.235751+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"814eaf62-a18d-494b-8e27-fd8fb88adea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6801","nonce":2350940051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6803","nonce":2350940051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6807","nonce":2350940051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6805","nonce":2350940051}]},"public_addr":"172.21.15.131:6801/2350940051","cluster_addr":"172.21.15.131:6803/2350940051","heartbeat_back_addr":"172.21.15.131:6807/2350940051","heartbeat_front_addr":"172.21.15.131:6805/2350940051","state":["exists","up"]},{"osd":1,"uuid":"42e66287-f6c6-4e26-af3c-6dfaa3536d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6825","nonce":1525528247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6827","nonce":1525528247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6831","nonce":1525528247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6829","nonce":1525528247}]},"public_addr":"172.21.15.131:6825/1525528247","cluster_addr":"172.21.15.131:6827/1525528247","heartbeat_back_addr":"172.21.15.131:6831/1525528247","heartbeat_front_addr":"172.21.15.131:6829/1525528247","state":["exists","up"]},{"osd":2,"uuid":"0da34673-04a2-4fd0-b57a-0ff2b679b043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6817","nonce":3627842210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6819","nonce":3627842210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6823","nonce":3627842210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6821","nonce":3627842210}]},"public_addr":"172.21.15.131:6817/3627842210","cluster_addr":"172.21.15.131:6819/3627842210","heartbeat_back_addr":"172.21.15.131:6823/3627842210","heartbeat_front_addr":"172.21.15.131:6821/3627842210","state":["exists","up"]},{"osd":3,"uuid":"cf117a27-d6b2-43d3-ae52-b47988bf8eed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6809","nonce":350790591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6811","nonce":350790591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6815","nonce":350790591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6813","nonce":350790591}]},"public_addr":"172.21.15.131:6809/350790591","cluster_addr":"172.21.15.131:6811/350790591","heartbeat_back_addr":"172.21.15.131:6815/350790591","heartbeat_front_addr":"172.21.15.131:6813/350790591","state":["exists","up"]},{"osd":4,"uuid":"a092ccfc-be89-4a84-88d5-0f781fa715eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6816","nonce":244560995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6819","nonce":244560995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6823","nonce":244560995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6821","nonce":244560995}]},"public_addr":"172.21.15.187:6816/244560995","cluster_addr":"172.21.15.187:6819/244560995","heartbeat_back_addr":"172.21.15.187:6823/244560995","heartbeat_front_addr":"172.21.15.187:6821/244560995","state":["exists","up"]},{"osd":5,"uuid":"7513ba69-dd00-4a35-b190-9e8cc3b2bdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6801","nonce":324619398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6803","nonce":324619398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6812","nonce":324619398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6807","nonce":324619398}]},"public_addr":"172.21.15.187:6801/324619398","cluster_addr":"172.21.15.187:6803/324619398","heartbeat_back_addr":"172.21.15.187:6812/324619398","heartbeat_front_addr":"172.21.15.187:6807/324619398","state":["exists","up"]},{"osd":6,"uuid":"50081810-c093-4d2f-a4d7-b7c695c5400d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6806","nonce":1391347786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6809","nonce":1391347786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6818","nonce":1391347786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6813","nonce":1391347786}]},"public_addr":"172.21.15.187:6806/1391347786","cluster_addr":"172.21.15.187:6809/1391347786","heartbeat_back_addr":"172.21.15.187:6818/1391347786","heartbeat_front_addr":"172.21.15.187:6813/1391347786","state":["exists","up"]},{"osd":7,"uuid":"775dae05-54f5-4ad4-bbce-92863c6ed2ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6825","nonce":1220065622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6827","nonce":1220065622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6831","nonce":1220065622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6829","nonce":1220065622}]},"public_addr":"172.21.15.187:6825/1220065622","cluster_addr":"172.21.15.187:6827/1220065622","heartbeat_back_addr":"172.21.15.187:6831/1220065622","heartbeat_front_addr":"172.21.15.187:6829/1220065622","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:05.983520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.299516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.067323+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.307748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.018742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.258775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.010912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.295355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-26T22:48:33.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.350+0000 7f4139800640 1 -- 172.21.15.131:0/1703850090 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f411406be30 msgr2=0x7f411406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:33.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.350+0000 7f4139800640 1 --2- 172.21.15.131:0/1703850090 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f411406be30 0x7f411406e2f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4124006fd0 tx=0x7f4124008040 comp rx=0 tx=0).stop 2024-05-26T22:48:33.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.350+0000 7f4139800640 1 -- 172.21.15.131:0/1703850090 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f413415b1c0 msgr2=0x7f4134155230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:33.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.350+0000 7f4139800640 1 --2- 172.21.15.131:0/1703850090 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f413415b1c0 0x7f4134155230 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f412c009e50 tx=0x7f412c004a40 comp rx=0 tx=0).stop 2024-05-26T22:48:33.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.350+0000 7f4139800640 1 -- 172.21.15.131:0/1703850090 shutdown_connections 2024-05-26T22:48:33.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.350+0000 7f4139800640 1 --2- 172.21.15.131:0/1703850090 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f41340ce180 0x7f413415da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:33.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.350+0000 7f4139800640 1 --2- 172.21.15.131:0/1703850090 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f411406be30 0x7f411406e2f0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:33.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.350+0000 7f4139800640 1 --2- 172.21.15.131:0/1703850090 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f413415b1c0 0x7f4134155230 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:33.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.350+0000 7f4139800640 1 --2- 172.21.15.131:0/1703850090 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f41340b51c0 0x7f413414d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:33.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.351+0000 7f4139800640 1 -- 172.21.15.131:0/1703850090 >> 172.21.15.131:0/1703850090 conn(0x7f41340d07b0 msgr2=0x7f41340aff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:33.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.351+0000 7f4139800640 1 -- 172.21.15.131:0/1703850090 shutdown_connections 2024-05-26T22:48:33.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.351+0000 7f4139800640 1 -- 172.21.15.131:0/1703850090 wait complete. 2024-05-26T22:48:33.370 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-26T22:48:10.463339+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-26T22:48:19.663254+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-26T22:48:20.235751+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-26T22:48:33.370 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-26T22:48:33.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.561+0000 7f5472800640 1 Processor -- start 2024-05-26T22:48:33.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.561+0000 7f5472800640 1 -- start start 2024-05-26T22:48:33.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.563+0000 7f5472800640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f546c15b730 0x7f546c15bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:33.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.563+0000 7f5472800640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f546c1400c0 0x7f546c1404f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:33.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.563+0000 7f5472800640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f546c13e820 con 0x7f546c06d250 2024-05-26T22:48:33.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.563+0000 7f5472800640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f546c071260 con 0x7f546c15b730 2024-05-26T22:48:33.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.563+0000 7f5472800640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f546c15c4a0 con 0x7f546c1400c0 2024-05-26T22:48:33.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.563+0000 7f5471400640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f546c06d250 0x7f546c06d630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:49014/0 (socket says 172.21.15.131:49014) 2024-05-26T22:48:33.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.563+0000 7f5471400640 1 -- 172.21.15.131:0/1871041818 learned_addr learned my addr 172.21.15.131:0/1871041818 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:48:33.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.563+0000 7f5470a00640 1 --2- 172.21.15.131:0/1871041818 >> v2:172.21.15.131:3300/0 conn(0x7f546c15b730 0x7f546c15bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:33.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.563+0000 7f5471e00640 1 --2- 172.21.15.131:0/1871041818 >> v2:172.21.15.187:3300/0 conn(0x7f546c1400c0 0x7f546c1404f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:33.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.563+0000 7f5470a00640 1 -- 172.21.15.131:0/1871041818 >> v2:172.21.15.187:3300/0 conn(0x7f546c1400c0 msgr2=0x7f546c1404f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:33.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.563+0000 7f5470a00640 1 --2- 172.21.15.131:0/1871041818 >> v2:172.21.15.187:3300/0 conn(0x7f546c1400c0 0x7f546c1404f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:33.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.563+0000 7f5470a00640 1 -- 172.21.15.131:0/1871041818 >> v1:172.21.15.131:6789/0 conn(0x7f546c06d250 legacy=0x7f546c06d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:33.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.563+0000 7f5470a00640 1 -- 172.21.15.131:0/1871041818 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f546c0714b0 con 0x7f546c15b730 2024-05-26T22:48:33.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.564+0000 7f5470a00640 1 --2- 172.21.15.131:0/1871041818 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f546c15b730 0x7f546c15bbc0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f54580097a0 tx=0x7f545802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=162062c577e7f5f0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:33.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.565+0000 7f546be00640 1 -- 172.21.15.131:0/1871041818 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5458037070 con 0x7f546c15b730 2024-05-26T22:48:33.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.565+0000 7f546be00640 1 -- 172.21.15.131:0/1871041818 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5458032440 con 0x7f546c15b730 2024-05-26T22:48:33.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.565+0000 7f546be00640 1 -- 172.21.15.131:0/1871041818 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5458032810 con 0x7f546c15b730 2024-05-26T22:48:33.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.565+0000 7f5472800640 1 -- 172.21.15.131:0/1871041818 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f546c15b730 msgr2=0x7f546c15bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:33.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.565+0000 7f5472800640 1 --2- 172.21.15.131:0/1871041818 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f546c15b730 0x7f546c15bbc0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f54580097a0 tx=0x7f545802a1e0 comp rx=0 tx=0).stop 2024-05-26T22:48:33.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.565+0000 7f5472800640 1 -- 172.21.15.131:0/1871041818 shutdown_connections 2024-05-26T22:48:33.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.565+0000 7f5472800640 1 --2- 172.21.15.131:0/1871041818 >> v2:172.21.15.187:3300/0 conn(0x7f546c1400c0 0x7f546c1404f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:33.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.565+0000 7f5472800640 1 --2- 172.21.15.131:0/1871041818 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f546c15b730 0x7f546c15bbc0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:33.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.565+0000 7f5472800640 1 -- 172.21.15.131:0/1871041818 >> 172.21.15.131:0/1871041818 conn(0x7f546c0b5660 msgr2=0x7f546c0b5a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:33.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.566+0000 7f5472800640 1 -- 172.21.15.131:0/1871041818 shutdown_connections 2024-05-26T22:48:33.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.566+0000 7f5472800640 1 -- 172.21.15.131:0/1871041818 wait complete. 2024-05-26T22:48:33.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.567+0000 7f5472800640 1 Processor -- start 2024-05-26T22:48:33.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.567+0000 7f5472800640 1 -- start start 2024-05-26T22:48:33.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.568+0000 7f5472800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f546c06d250 0x7f546c1ecfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:33.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.568+0000 7f5472800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f546c1400c0 0x7f546c1f0370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:33.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.568+0000 7f5472800640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f546c15b730 0x7f546c157990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:33.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.568+0000 7f5472800640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f546c15c100 con 0x7f546c06d250 2024-05-26T22:48:33.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.568+0000 7f5472800640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f546c071a70 con 0x7f546c15b730 2024-05-26T22:48:33.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.568+0000 7f5472800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f546c15c280 con 0x7f546c1400c0 2024-05-26T22:48:33.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.568+0000 7f5471400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f546c06d250 0x7f546c1ecfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:33.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.568+0000 7f5471400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f546c06d250 0x7f546c1ecfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54464/0 (socket says 172.21.15.131:54464) 2024-05-26T22:48:33.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.568+0000 7f5471400640 1 -- 172.21.15.131:0/2521612886 learned_addr learned my addr 172.21.15.131:0/2521612886 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:33.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.569+0000 7f5471400640 1 -- 172.21.15.131:0/2521612886 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f546c1400c0 msgr2=0x7f546c1f0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:33.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.569+0000 7f5471400640 1 --2- 172.21.15.131:0/2521612886 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f546c1400c0 0x7f546c1f0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:33.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.569+0000 7f5471400640 1 -- 172.21.15.131:0/2521612886 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f546c15b730 msgr2=0x7f546c157990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:33.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.569+0000 7f5471e00640 1 --2- 172.21.15.131:0/2521612886 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f546c15b730 0x7f546c157990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:33.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.569+0000 7f5471400640 1 --2- 172.21.15.131:0/2521612886 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f546c15b730 0x7f546c157990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:33.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.569+0000 7f5471400640 1 -- 172.21.15.131:0/2521612886 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f546c1f08b0 con 0x7f546c06d250 2024-05-26T22:48:33.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.569+0000 7f5471400640 1 --2- 172.21.15.131:0/2521612886 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f546c06d250 0x7f546c1ecfc0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f546000c400 tx=0x7f546000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:33.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.569+0000 7f5469600640 1 -- 172.21.15.131:0/2521612886 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5460017070 con 0x7f546c06d250 2024-05-26T22:48:33.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.569+0000 7f5469600640 1 -- 172.21.15.131:0/2521612886 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5460005ce0 con 0x7f546c06d250 2024-05-26T22:48:33.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.569+0000 7f5472800640 1 -- 172.21.15.131:0/2521612886 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f546c155a20 con 0x7f546c06d250 2024-05-26T22:48:33.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.570+0000 7f5469600640 1 -- 172.21.15.131:0/2521612886 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54600072f0 con 0x7f546c06d250 2024-05-26T22:48:33.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.570+0000 7f5472800640 1 -- 172.21.15.131:0/2521612886 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f546c155fa0 con 0x7f546c06d250 2024-05-26T22:48:33.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.571+0000 7f5472800640 1 -- 172.21.15.131:0/2521612886 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f546c0694e0 con 0x7f546c06d250 2024-05-26T22:48:33.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.572+0000 7f5469600640 1 -- 172.21.15.131:0/2521612886 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f546001a070 con 0x7f546c06d250 2024-05-26T22:48:33.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.573+0000 7f5469600640 1 --2- 172.21.15.131:0/2521612886 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f544c06be30 0x7f544c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:33.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.573+0000 7f5469600640 1 -- 172.21.15.131:0/2521612886 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f546008a2b0 con 0x7f546c06d250 2024-05-26T22:48:33.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.573+0000 7f5470a00640 1 --2- 172.21.15.131:0/2521612886 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f544c06be30 0x7f544c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:33.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.574+0000 7f5470a00640 1 --2- 172.21.15.131:0/2521612886 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f544c06be30 0x7f544c06e2f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5458034000 tx=0x7f545802c000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:33.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.577+0000 7f5469600640 1 -- 172.21.15.131:0/2521612886 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f546c1f08b0 con 0x7f546c06d250 2024-05-26T22:48:33.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.774+0000 7f5472800640 1 -- 172.21.15.131:0/2521612886 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f546c15e020 con 0x7f546c06d250 2024-05-26T22:48:33.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.775+0000 7f5469600640 1 -- 172.21.15.131:0/2521612886 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f5460058c50 con 0x7f546c06d250 2024-05-26T22:48:33.775 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2024-05-26T22:48:33.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.778+0000 7f5472800640 1 -- 172.21.15.131:0/2521612886 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f544c06be30 msgr2=0x7f544c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:33.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.778+0000 7f5472800640 1 --2- 172.21.15.131:0/2521612886 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f544c06be30 0x7f544c06e2f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5458034000 tx=0x7f545802c000 comp rx=0 tx=0).stop 2024-05-26T22:48:33.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.778+0000 7f5472800640 1 -- 172.21.15.131:0/2521612886 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f546c06d250 msgr2=0x7f546c1ecfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:33.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.778+0000 7f5472800640 1 --2- 172.21.15.131:0/2521612886 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f546c06d250 0x7f546c1ecfc0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f546000c400 tx=0x7f546000c8c0 comp rx=0 tx=0).stop 2024-05-26T22:48:33.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.778+0000 7f5472800640 1 -- 172.21.15.131:0/2521612886 shutdown_connections 2024-05-26T22:48:33.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.778+0000 7f5472800640 1 --2- 172.21.15.131:0/2521612886 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f546c1400c0 0x7f546c1f0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:33.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.778+0000 7f5472800640 1 --2- 172.21.15.131:0/2521612886 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f544c06be30 0x7f544c06e2f0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:33.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.778+0000 7f5472800640 1 --2- 172.21.15.131:0/2521612886 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f546c15b730 0x7f546c157990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:33.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.778+0000 7f5472800640 1 --2- 172.21.15.131:0/2521612886 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f546c06d250 0x7f546c1ecfc0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:33.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.778+0000 7f5472800640 1 -- 172.21.15.131:0/2521612886 >> 172.21.15.131:0/2521612886 conn(0x7f546c0b5660 msgr2=0x7f546c13f3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:33.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.778+0000 7f5472800640 1 -- 172.21.15.131:0/2521612886 shutdown_connections 2024-05-26T22:48:33.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.778+0000 7f5472800640 1 -- 172.21.15.131:0/2521612886 wait complete. 2024-05-26T22:48:33.797 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-26T22:48:33.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:33.999+0000 7f4fe7e00640 1 Processor -- start 2024-05-26T22:48:33.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.000+0000 7f4fe7e00640 1 -- start start 2024-05-26T22:48:34.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.001+0000 7f4fe7e00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f4fe8147050 0x7f4fe80d07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:34.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.001+0000 7f4fe7e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f4fe80d0ce0 0x7f4fe80d1f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:34.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.001+0000 7f4fe7e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4fe80d24d0 0x7f4fe8070cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:34.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.001+0000 7f4fe7e00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f4fe8147430 con 0x7f4fe80d0ce0 2024-05-26T22:48:34.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.001+0000 7f4fe7e00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f4fe8161fb0 con 0x7f4fe8147050 2024-05-26T22:48:34.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.001+0000 7f4fe7e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f4fe8162290 con 0x7f4fe80d24d0 2024-05-26T22:48:34.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.001+0000 7f4fe6000640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f4fe80d0ce0 0x7f4fe80d1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:34.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.001+0000 7f4fe6000640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f4fe80d0ce0 0x7f4fe80d1f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54468/0 (socket says 172.21.15.131:54468) 2024-05-26T22:48:34.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.001+0000 7f4fe6000640 1 -- 172.21.15.131:0/2532846451 learned_addr learned my addr 172.21.15.131:0/2532846451 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:34.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.001+0000 7f4fe6a00640 1 --2- 172.21.15.131:0/2532846451 >> v2:172.21.15.187:3300/0 conn(0x7f4fe8147050 0x7f4fe80d07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:34.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.002+0000 7f4fe6000640 1 -- 172.21.15.131:0/2532846451 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4fe80d24d0 msgr2=0x7f4fe8070cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:34.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.001+0000 7f4fe7400640 1 --2- 172.21.15.131:0/2532846451 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4fe80d24d0 0x7f4fe8070cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:34.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.002+0000 7f4fe6000640 1 --2- 172.21.15.131:0/2532846451 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4fe80d24d0 0x7f4fe8070cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:34.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.002+0000 7f4fe6000640 1 -- 172.21.15.131:0/2532846451 >> v2:172.21.15.187:3300/0 conn(0x7f4fe8147050 msgr2=0x7f4fe80d07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:34.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.002+0000 7f4fe6000640 1 --2- 172.21.15.131:0/2532846451 >> v2:172.21.15.187:3300/0 conn(0x7f4fe8147050 0x7f4fe80d07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:34.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.002+0000 7f4fe6000640 1 -- 172.21.15.131:0/2532846451 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fe8162570 con 0x7f4fe80d0ce0 2024-05-26T22:48:34.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.002+0000 7f4fe7400640 1 --2- 172.21.15.131:0/2532846451 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4fe80d24d0 0x7f4fe8070cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:48:34.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.003+0000 7f4fe6000640 1 --2- 172.21.15.131:0/2532846451 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4fe80d0ce0 0x7f4fe80d1f90 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4fe806ef50 tx=0x7f4fd002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c5bc7ede9d85c9a server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:34.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.003+0000 7f4fe5600640 1 -- 172.21.15.131:0/2532846451 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd0036070 con 0x7f4fe80d0ce0 2024-05-26T22:48:34.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.003+0000 7f4fe5600640 1 -- 172.21.15.131:0/2532846451 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fd0031600 con 0x7f4fe80d0ce0 2024-05-26T22:48:34.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.003+0000 7f4fe5600640 1 -- 172.21.15.131:0/2532846451 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd00319d0 con 0x7f4fe80d0ce0 2024-05-26T22:48:34.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.004+0000 7f4fe7e00640 1 -- 172.21.15.131:0/2532846451 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4fe80d0ce0 msgr2=0x7f4fe80d1f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:34.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.004+0000 7f4fe7e00640 1 --2- 172.21.15.131:0/2532846451 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4fe80d0ce0 0x7f4fe80d1f90 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4fe806ef50 tx=0x7f4fd002a6a0 comp rx=0 tx=0).stop 2024-05-26T22:48:34.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.004+0000 7f4fe7e00640 1 -- 172.21.15.131:0/2532846451 shutdown_connections 2024-05-26T22:48:34.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.004+0000 7f4fe7e00640 1 --2- 172.21.15.131:0/2532846451 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4fe80d24d0 0x7f4fe8070cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:34.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.004+0000 7f4fe7e00640 1 --2- 172.21.15.131:0/2532846451 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4fe80d0ce0 0x7f4fe80d1f90 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:34.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.004+0000 7f4fe7e00640 1 --2- 172.21.15.131:0/2532846451 >> v2:172.21.15.187:3300/0 conn(0x7f4fe8147050 0x7f4fe80d07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:34.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.004+0000 7f4fe7e00640 1 -- 172.21.15.131:0/2532846451 >> 172.21.15.131:0/2532846451 conn(0x7f4fe806f680 msgr2=0x7f4fe806fa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:34.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.004+0000 7f4fe7e00640 1 -- 172.21.15.131:0/2532846451 shutdown_connections 2024-05-26T22:48:34.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.004+0000 7f4fe7e00640 1 -- 172.21.15.131:0/2532846451 wait complete. 2024-05-26T22:48:34.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.005+0000 7f4fe7e00640 1 Processor -- start 2024-05-26T22:48:34.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.006+0000 7f4fe7e00640 1 -- start start 2024-05-26T22:48:34.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.006+0000 7f4fe7e00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4fe80d0ce0 0x7f4fe80b10c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:34.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.006+0000 7f4fe7e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4fe80d24d0 0x7f4fe8133e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:34.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.007+0000 7f4fe7e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4fe8147050 0x7f4fe80b6f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:34.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.007+0000 7f4fe7e00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe8071200 con 0x7f4fe8147050 2024-05-26T22:48:34.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.007+0000 7f4fe7e00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe80d2940 con 0x7f4fe80d0ce0 2024-05-26T22:48:34.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.007+0000 7f4fe7e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f4fe8071380 con 0x7f4fe80d24d0 2024-05-26T22:48:34.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.007+0000 7f4fe7400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4fe8147050 0x7f4fe80b6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:34.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.007+0000 7f4fe7400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4fe8147050 0x7f4fe80b6f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54476/0 (socket says 172.21.15.131:54476) 2024-05-26T22:48:34.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.007+0000 7f4fe7400640 1 -- 172.21.15.131:0/908284141 learned_addr learned my addr 172.21.15.131:0/908284141 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:34.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.007+0000 7f4fe6a00640 1 --2- 172.21.15.131:0/908284141 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4fe80d0ce0 0x7f4fe80b10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:34.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.007+0000 7f4fe6000640 1 --2- 172.21.15.131:0/908284141 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4fe80d24d0 0x7f4fe8133e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:34.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.007+0000 7f4fe7400640 1 -- 172.21.15.131:0/908284141 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4fe80d24d0 msgr2=0x7f4fe8133e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:34.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.007+0000 7f4fe7400640 1 --2- 172.21.15.131:0/908284141 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4fe80d24d0 0x7f4fe8133e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:34.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.007+0000 7f4fe7400640 1 -- 172.21.15.131:0/908284141 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4fe80d0ce0 msgr2=0x7f4fe80b10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:34.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.007+0000 7f4fe7400640 1 --2- 172.21.15.131:0/908284141 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4fe80d0ce0 0x7f4fe80b10c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:34.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.007+0000 7f4fe7400640 1 -- 172.21.15.131:0/908284141 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fe8134740 con 0x7f4fe8147050 2024-05-26T22:48:34.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.008+0000 7f4fe7400640 1 --2- 172.21.15.131:0/908284141 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4fe8147050 0x7f4fe80b6f90 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4fe000c7f0 tx=0x7f4fe000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:34.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.008+0000 7f4fdea00640 1 -- 172.21.15.131:0/908284141 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe0015070 con 0x7f4fe8147050 2024-05-26T22:48:34.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.008+0000 7f4fdea00640 1 -- 172.21.15.131:0/908284141 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fe0007820 con 0x7f4fe8147050 2024-05-26T22:48:34.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.008+0000 7f4fdea00640 1 -- 172.21.15.131:0/908284141 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe0007c10 con 0x7f4fe8147050 2024-05-26T22:48:34.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.008+0000 7f4fe7e00640 1 -- 172.21.15.131:0/908284141 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fe8134a20 con 0x7f4fe8147050 2024-05-26T22:48:34.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.008+0000 7f4fe7e00640 1 -- 172.21.15.131:0/908284141 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fe8134fe0 con 0x7f4fe8147050 2024-05-26T22:48:34.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.009+0000 7f4fe7e00640 1 -- 172.21.15.131:0/908284141 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fe8070cc0 con 0x7f4fe8147050 2024-05-26T22:48:34.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.011+0000 7f4fdea00640 1 -- 172.21.15.131:0/908284141 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4fe0018070 con 0x7f4fe8147050 2024-05-26T22:48:34.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.011+0000 7f4fdea00640 1 --2- 172.21.15.131:0/908284141 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f4fc406be30 0x7f4fc406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:34.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.012+0000 7f4fdea00640 1 -- 172.21.15.131:0/908284141 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f4fe008af40 con 0x7f4fe8147050 2024-05-26T22:48:34.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.012+0000 7f4fe6a00640 1 --2- 172.21.15.131:0/908284141 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f4fc406be30 0x7f4fc406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:34.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.012+0000 7f4fe6a00640 1 --2- 172.21.15.131:0/908284141 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f4fc406be30 0x7f4fc406e2f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4fd8006fd0 tx=0x7f4fd8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:34.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.016+0000 7f4fdea00640 1 -- 172.21.15.131:0/908284141 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f4fe8070cc0 con 0x7f4fe8147050 2024-05-26T22:48:34.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.218+0000 7f4fe7e00640 1 -- 172.21.15.131:0/908284141 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f4fe80cba80 con 0x7f4fe8147050 2024-05-26T22:48:34.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.219+0000 7f4fdea00640 1 -- 172.21.15.131:0/908284141 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f4fe0059a40 con 0x7f4fe8147050 2024-05-26T22:48:34.218 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-05-26T22:48:34.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.221+0000 7f4fe7e00640 1 -- 172.21.15.131:0/908284141 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f4fc406be30 msgr2=0x7f4fc406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:34.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.221+0000 7f4fe7e00640 1 --2- 172.21.15.131:0/908284141 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f4fc406be30 0x7f4fc406e2f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4fd8006fd0 tx=0x7f4fd8008040 comp rx=0 tx=0).stop 2024-05-26T22:48:34.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.221+0000 7f4fe7e00640 1 -- 172.21.15.131:0/908284141 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4fe8147050 msgr2=0x7f4fe80b6f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:34.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.221+0000 7f4fe7e00640 1 --2- 172.21.15.131:0/908284141 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4fe8147050 0x7f4fe80b6f90 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4fe000c7f0 tx=0x7f4fe000ccb0 comp rx=0 tx=0).stop 2024-05-26T22:48:34.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.221+0000 7f4fe7e00640 1 -- 172.21.15.131:0/908284141 shutdown_connections 2024-05-26T22:48:34.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.221+0000 7f4fe7e00640 1 --2- 172.21.15.131:0/908284141 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4fe80d24d0 0x7f4fe8133e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:34.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.221+0000 7f4fe7e00640 1 --2- 172.21.15.131:0/908284141 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f4fc406be30 0x7f4fc406e2f0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:34.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.221+0000 7f4fe7e00640 1 --2- 172.21.15.131:0/908284141 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4fe8147050 0x7f4fe80b6f90 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:34.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.221+0000 7f4fe7e00640 1 --2- 172.21.15.131:0/908284141 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4fe80d0ce0 0x7f4fe80b10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:34.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.221+0000 7f4fe7e00640 1 -- 172.21.15.131:0/908284141 >> 172.21.15.131:0/908284141 conn(0x7f4fe806f680 msgr2=0x7f4fe80b3fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:34.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.222+0000 7f4fe7e00640 1 -- 172.21.15.131:0/908284141 shutdown_connections 2024-05-26T22:48:34.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.222+0000 7f4fe7e00640 1 -- 172.21.15.131:0/908284141 wait complete. 2024-05-26T22:48:34.242 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-26T22:48:34.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.449+0000 7f30c2200640 1 Processor -- start 2024-05-26T22:48:34.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.450+0000 7f30c2200640 1 -- start start 2024-05-26T22:48:34.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.451+0000 7f30c2200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f30bc15e4e0 0x7f30bc0b9c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:34.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.451+0000 7f30c2200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f30bc0d2330 0x7f30bc0cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:34.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.451+0000 7f30c2200640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f30bc0cf550 0x7f30bc0d3c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:34.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.451+0000 7f30c2200640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f30bc0d28f0 con 0x7f30bc15e4e0 2024-05-26T22:48:34.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.451+0000 7f30c2200640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f30bc0d2bd0 con 0x7f30bc0cf550 2024-05-26T22:48:34.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.451+0000 7f30c2200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f30bc0d2eb0 con 0x7f30bc0d2330 2024-05-26T22:48:34.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.451+0000 7f30bb400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f30bc15e4e0 0x7f30bc0b9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:34.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.451+0000 7f30bb400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f30bc15e4e0 0x7f30bc0b9c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58080/0 (socket says 172.21.15.131:58080) 2024-05-26T22:48:34.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.451+0000 7f30bb400640 1 -- 172.21.15.131:0/3963094858 learned_addr learned my addr 172.21.15.131:0/3963094858 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:34.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.452+0000 7f30baa00640 1 --2- 172.21.15.131:0/3963094858 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f30bc0d2330 0x7f30bc0cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:34.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.452+0000 7f30bbe00640 1 --2- 172.21.15.131:0/3963094858 >> v2:172.21.15.187:3300/0 conn(0x7f30bc0cf550 0x7f30bc0d3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:34.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.452+0000 7f30bb400640 1 -- 172.21.15.131:0/3963094858 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f30bc0d2330 msgr2=0x7f30bc0cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:34.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.452+0000 7f30bb400640 1 --2- 172.21.15.131:0/3963094858 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f30bc0d2330 0x7f30bc0cf010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:34.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.452+0000 7f30bb400640 1 -- 172.21.15.131:0/3963094858 >> v2:172.21.15.187:3300/0 conn(0x7f30bc0cf550 msgr2=0x7f30bc0d3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:34.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.452+0000 7f30bb400640 1 --2- 172.21.15.131:0/3963094858 >> v2:172.21.15.187:3300/0 conn(0x7f30bc0cf550 0x7f30bc0d3c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:34.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.452+0000 7f30bb400640 1 -- 172.21.15.131:0/3963094858 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30bc0d3190 con 0x7f30bc15e4e0 2024-05-26T22:48:34.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.452+0000 7f30bbe00640 1 --2- 172.21.15.131:0/3963094858 >> v2:172.21.15.187:3300/0 conn(0x7f30bc0cf550 0x7f30bc0d3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:48:34.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.453+0000 7f30bb400640 1 --2- 172.21.15.131:0/3963094858 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f30bc15e4e0 0x7f30bc0b9c80 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f30bc0ce8b0 tx=0x7f30ac02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f2f1d23ff89303a server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:34.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.453+0000 7f30ba000640 1 -- 172.21.15.131:0/3963094858 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30ac036070 con 0x7f30bc15e4e0 2024-05-26T22:48:34.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.453+0000 7f30ba000640 1 -- 172.21.15.131:0/3963094858 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30ac031600 con 0x7f30bc15e4e0 2024-05-26T22:48:34.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.453+0000 7f30ba000640 1 -- 172.21.15.131:0/3963094858 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30ac0319d0 con 0x7f30bc15e4e0 2024-05-26T22:48:34.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.454+0000 7f30c2200640 1 -- 172.21.15.131:0/3963094858 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f30bc15e4e0 msgr2=0x7f30bc0b9c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:34.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.454+0000 7f30c2200640 1 --2- 172.21.15.131:0/3963094858 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f30bc15e4e0 0x7f30bc0b9c80 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f30bc0ce8b0 tx=0x7f30ac02a6a0 comp rx=0 tx=0).stop 2024-05-26T22:48:34.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.454+0000 7f30c2200640 1 -- 172.21.15.131:0/3963094858 shutdown_connections 2024-05-26T22:48:34.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.454+0000 7f30c2200640 1 --2- 172.21.15.131:0/3963094858 >> v2:172.21.15.187:3300/0 conn(0x7f30bc0cf550 0x7f30bc0d3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:34.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.454+0000 7f30c2200640 1 --2- 172.21.15.131:0/3963094858 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f30bc0d2330 0x7f30bc0cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:34.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.454+0000 7f30c2200640 1 --2- 172.21.15.131:0/3963094858 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f30bc15e4e0 0x7f30bc0b9c80 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:34.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.454+0000 7f30c2200640 1 -- 172.21.15.131:0/3963094858 >> 172.21.15.131:0/3963094858 conn(0x7f30bc072e90 msgr2=0x7f30bc0732a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:34.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.454+0000 7f30c2200640 1 -- 172.21.15.131:0/3963094858 shutdown_connections 2024-05-26T22:48:34.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.454+0000 7f30c2200640 1 -- 172.21.15.131:0/3963094858 wait complete. 2024-05-26T22:48:34.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.456+0000 7f30c2200640 1 Processor -- start 2024-05-26T22:48:34.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.456+0000 7f30c2200640 1 -- start start 2024-05-26T22:48:34.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.457+0000 7f30c2200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f30bc0cf550 0x7f30bc157d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:34.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.457+0000 7f30c2200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f30bc0d2330 0x7f30bc14e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:34.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.457+0000 7f30c2200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f30bc15e4e0 0x7f30bc1f40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:34.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.457+0000 7f30c2200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f30bc0d3990 con 0x7f30bc15e4e0 2024-05-26T22:48:34.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.457+0000 7f30c2200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f30bc15e8c0 con 0x7f30bc0cf550 2024-05-26T22:48:34.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.457+0000 7f30c2200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f30bc0ba1c0 con 0x7f30bc0d2330 2024-05-26T22:48:34.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.457+0000 7f30bbe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f30bc15e4e0 0x7f30bc1f40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:34.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.457+0000 7f30bbe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f30bc15e4e0 0x7f30bc1f40e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58082/0 (socket says 172.21.15.131:58082) 2024-05-26T22:48:34.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.457+0000 7f30bbe00640 1 -- 172.21.15.131:0/3821086815 learned_addr learned my addr 172.21.15.131:0/3821086815 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:34.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.457+0000 7f30baa00640 1 --2- 172.21.15.131:0/3821086815 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f30bc0d2330 0x7f30bc14e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:34.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.457+0000 7f30bb400640 1 --2- 172.21.15.131:0/3821086815 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f30bc0cf550 0x7f30bc157d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:34.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.458+0000 7f30bbe00640 1 -- 172.21.15.131:0/3821086815 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f30bc0d2330 msgr2=0x7f30bc14e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:34.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.458+0000 7f30bbe00640 1 --2- 172.21.15.131:0/3821086815 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f30bc0d2330 0x7f30bc14e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:34.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.458+0000 7f30bbe00640 1 -- 172.21.15.131:0/3821086815 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f30bc0cf550 msgr2=0x7f30bc157d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:34.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.458+0000 7f30bbe00640 1 --2- 172.21.15.131:0/3821086815 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f30bc0cf550 0x7f30bc157d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:34.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.458+0000 7f30bbe00640 1 -- 172.21.15.131:0/3821086815 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30bc0d4270 con 0x7f30bc15e4e0 2024-05-26T22:48:34.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.458+0000 7f30bbe00640 1 --2- 172.21.15.131:0/3821086815 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f30bc15e4e0 0x7f30bc1f40e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f30b400c7c0 tx=0x7f30b400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:34.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.458+0000 7f30b3400640 1 -- 172.21.15.131:0/3821086815 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30b4015020 con 0x7f30bc15e4e0 2024-05-26T22:48:34.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.458+0000 7f30b3400640 1 -- 172.21.15.131:0/3821086815 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30b4004230 con 0x7f30bc15e4e0 2024-05-26T22:48:34.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.458+0000 7f30b3400640 1 -- 172.21.15.131:0/3821086815 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30b4007620 con 0x7f30bc15e4e0 2024-05-26T22:48:34.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.458+0000 7f30c2200640 1 -- 172.21.15.131:0/3821086815 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30bc0d4550 con 0x7f30bc15e4e0 2024-05-26T22:48:34.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.458+0000 7f30c2200640 1 -- 172.21.15.131:0/3821086815 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30bc0c9780 con 0x7f30bc15e4e0 2024-05-26T22:48:34.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.459+0000 7f30c2200640 1 -- 172.21.15.131:0/3821086815 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30bc0695c0 con 0x7f30bc15e4e0 2024-05-26T22:48:34.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.461+0000 7f30b3400640 1 -- 172.21.15.131:0/3821086815 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f30b4018020 con 0x7f30bc15e4e0 2024-05-26T22:48:34.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.461+0000 7f30b3400640 1 --2- 172.21.15.131:0/3821086815 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f309806be30 0x7f309806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:34.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.462+0000 7f30b3400640 1 -- 172.21.15.131:0/3821086815 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f30b408afa0 con 0x7f30bc15e4e0 2024-05-26T22:48:34.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.462+0000 7f30bb400640 1 --2- 172.21.15.131:0/3821086815 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f309806be30 0x7f309806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:34.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.463+0000 7f30bb400640 1 --2- 172.21.15.131:0/3821086815 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f309806be30 0x7f309806e2f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f30ac033000 tx=0x7f30ac009000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:34.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.466+0000 7f30b3400640 1 -- 172.21.15.131:0/3821086815 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f30b401f020 con 0x7f30bc15e4e0 2024-05-26T22:48:34.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.668+0000 7f30c2200640 1 -- 172.21.15.131:0/3821086815 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f30bc0688d0 con 0x7f30bc15e4e0 2024-05-26T22:48:34.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.669+0000 7f30b3400640 1 -- 172.21.15.131:0/3821086815 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f30b4059970 con 0x7f30bc15e4e0 2024-05-26T22:48:34.669 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 64 2024-05-26T22:48:34.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.671+0000 7f30c2200640 1 -- 172.21.15.131:0/3821086815 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f309806be30 msgr2=0x7f309806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:34.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.671+0000 7f30c2200640 1 --2- 172.21.15.131:0/3821086815 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f309806be30 0x7f309806e2f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f30ac033000 tx=0x7f30ac009000 comp rx=0 tx=0).stop 2024-05-26T22:48:34.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.671+0000 7f30c2200640 1 -- 172.21.15.131:0/3821086815 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f30bc15e4e0 msgr2=0x7f30bc1f40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:34.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.671+0000 7f30c2200640 1 --2- 172.21.15.131:0/3821086815 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f30bc15e4e0 0x7f30bc1f40e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f30b400c7c0 tx=0x7f30b400cc80 comp rx=0 tx=0).stop 2024-05-26T22:48:34.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.672+0000 7f30c2200640 1 -- 172.21.15.131:0/3821086815 shutdown_connections 2024-05-26T22:48:34.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.672+0000 7f30c2200640 1 --2- 172.21.15.131:0/3821086815 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f30bc0d2330 0x7f30bc14e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:34.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.672+0000 7f30c2200640 1 --2- 172.21.15.131:0/3821086815 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f309806be30 0x7f309806e2f0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:34.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.672+0000 7f30c2200640 1 --2- 172.21.15.131:0/3821086815 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f30bc15e4e0 0x7f30bc1f40e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:34.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.672+0000 7f30c2200640 1 --2- 172.21.15.131:0/3821086815 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f30bc0cf550 0x7f30bc157d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:34.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.672+0000 7f30c2200640 1 -- 172.21.15.131:0/3821086815 >> 172.21.15.131:0/3821086815 conn(0x7f30bc072e90 msgr2=0x7f30bc0b48d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:34.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.672+0000 7f30c2200640 1 -- 172.21.15.131:0/3821086815 shutdown_connections 2024-05-26T22:48:34.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.672+0000 7f30c2200640 1 -- 172.21.15.131:0/3821086815 wait complete. 2024-05-26T22:48:34.691 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-26T22:48:34.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.884+0000 7f3b19800640 1 Processor -- start 2024-05-26T22:48:34.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.885+0000 7f3b19800640 1 -- start start 2024-05-26T22:48:34.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.886+0000 7f3b19800640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f3b1406fce0 0x7f3b1406de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:34.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.886+0000 7f3b19800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3b1415f560 0x7f3b140aebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:34.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.886+0000 7f3b19800640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b1406e4e0 con 0x7f3b14158c70 2024-05-26T22:48:34.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.886+0000 7f3b19800640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f3b140ae780 con 0x7f3b1406fce0 2024-05-26T22:48:34.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.886+0000 7f3b19800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3b1406f060 con 0x7f3b1415f560 2024-05-26T22:48:34.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.887+0000 7f3b12a00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f3b14158c70 0x7f3b1406f5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:40094/0 (socket says 172.21.15.131:40094) 2024-05-26T22:48:34.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.887+0000 7f3b12a00640 1 -- 172.21.15.131:0/2815659474 learned_addr learned my addr 172.21.15.131:0/2815659474 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:48:34.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.887+0000 7f3b12000640 1 --2- 172.21.15.131:0/2815659474 >> v2:172.21.15.131:3300/0 conn(0x7f3b1406fce0 0x7f3b1406de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:34.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.887+0000 7f3b13400640 1 --2- 172.21.15.131:0/2815659474 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3b1415f560 0x7f3b140aebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:34.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.887+0000 7f3b11600640 1 -- 172.21.15.131:0/2815659474 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3543377923 0 0) 0x7f3b1406e4e0 con 0x7f3b14158c70 2024-05-26T22:48:34.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.887+0000 7f3b11600640 1 -- 172.21.15.131:0/2815659474 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3af4003520 con 0x7f3b14158c70 2024-05-26T22:48:34.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.887+0000 7f3b12000640 1 -- 172.21.15.131:0/2815659474 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3b1415f560 msgr2=0x7f3b140aebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:34.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.887+0000 7f3b12000640 1 --2- 172.21.15.131:0/2815659474 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3b1415f560 0x7f3b140aebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:34.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.887+0000 7f3b12000640 1 -- 172.21.15.131:0/2815659474 >> v1:172.21.15.131:6789/0 conn(0x7f3b14158c70 legacy=0x7f3b1406f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:34.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.887+0000 7f3b12000640 1 -- 172.21.15.131:0/2815659474 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b1406e7c0 con 0x7f3b1406fce0 2024-05-26T22:48:34.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.888+0000 7f3b12000640 1 --2- 172.21.15.131:0/2815659474 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b1406fce0 0x7f3b1406de50 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3afc009d90 tx=0x7f3afc02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c54648187d38bb6d server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:34.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.889+0000 7f3b11600640 1 -- 172.21.15.131:0/2815659474 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3afc007740 con 0x7f3b1406fce0 2024-05-26T22:48:34.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.889+0000 7f3b11600640 1 -- 172.21.15.131:0/2815659474 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3afc0078e0 con 0x7f3b1406fce0 2024-05-26T22:48:34.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.889+0000 7f3b11600640 1 -- 172.21.15.131:0/2815659474 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3afc007cb0 con 0x7f3b1406fce0 2024-05-26T22:48:34.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.889+0000 7f3b19800640 1 -- 172.21.15.131:0/2815659474 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b1406fce0 msgr2=0x7f3b1406de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:34.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.889+0000 7f3b19800640 1 --2- 172.21.15.131:0/2815659474 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b1406fce0 0x7f3b1406de50 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3afc009d90 tx=0x7f3afc02a660 comp rx=0 tx=0).stop 2024-05-26T22:48:34.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.889+0000 7f3b19800640 1 -- 172.21.15.131:0/2815659474 shutdown_connections 2024-05-26T22:48:34.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.889+0000 7f3b19800640 1 --2- 172.21.15.131:0/2815659474 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3b1415f560 0x7f3b140aebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:34.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.889+0000 7f3b19800640 1 --2- 172.21.15.131:0/2815659474 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b1406fce0 0x7f3b1406de50 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:34.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.889+0000 7f3b19800640 1 -- 172.21.15.131:0/2815659474 >> 172.21.15.131:0/2815659474 conn(0x7f3b14072b00 msgr2=0x7f3b1406b310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:34.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.890+0000 7f3b19800640 1 -- 172.21.15.131:0/2815659474 shutdown_connections 2024-05-26T22:48:34.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.890+0000 7f3b19800640 1 -- 172.21.15.131:0/2815659474 wait complete. 2024-05-26T22:48:34.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.891+0000 7f3b19800640 1 Processor -- start 2024-05-26T22:48:34.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.891+0000 7f3b19800640 1 -- start start 2024-05-26T22:48:34.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.892+0000 7f3b19800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3b1406fce0 0x7f3b141f37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:34.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.892+0000 7f3b19800640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b14158c70 0x7f3b1414f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:34.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.892+0000 7f3b19800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b1415f560 0x7f3b14150080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:34.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.892+0000 7f3b19800640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f3b1415f9d0 con 0x7f3b1415f560 2024-05-26T22:48:34.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.892+0000 7f3b19800640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3b1406ed80 con 0x7f3b14158c70 2024-05-26T22:48:34.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.892+0000 7f3b19800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3b140af130 con 0x7f3b1406fce0 2024-05-26T22:48:34.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.893+0000 7f3b13400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b1415f560 0x7f3b14150080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:34.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.893+0000 7f3b13400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b1415f560 0x7f3b14150080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58104/0 (socket says 172.21.15.131:58104) 2024-05-26T22:48:34.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.893+0000 7f3b13400640 1 -- 172.21.15.131:0/276400764 learned_addr learned my addr 172.21.15.131:0/276400764 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:34.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.893+0000 7f3b12a00640 1 --2- 172.21.15.131:0/276400764 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3b1406fce0 0x7f3b141f37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:34.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.893+0000 7f3b12000640 1 --2- 172.21.15.131:0/276400764 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b14158c70 0x7f3b1414f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:34.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.893+0000 7f3b13400640 1 -- 172.21.15.131:0/276400764 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3b1406fce0 msgr2=0x7f3b141f37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:34.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.893+0000 7f3b13400640 1 --2- 172.21.15.131:0/276400764 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3b1406fce0 0x7f3b141f37d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:34.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.893+0000 7f3b13400640 1 -- 172.21.15.131:0/276400764 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b14158c70 msgr2=0x7f3b1414f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:34.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.893+0000 7f3b13400640 1 --2- 172.21.15.131:0/276400764 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b14158c70 0x7f3b1414f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:34.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.893+0000 7f3b13400640 1 -- 172.21.15.131:0/276400764 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b14154d20 con 0x7f3b1415f560 2024-05-26T22:48:34.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.893+0000 7f3b12000640 1 --2- 172.21.15.131:0/276400764 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b14158c70 0x7f3b1414f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:48:34.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.893+0000 7f3b13400640 1 --2- 172.21.15.131:0/276400764 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b1415f560 0x7f3b14150080 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c00d5d0 tx=0x7f3b0c00da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:34.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.894+0000 7f3b0aa00640 1 -- 172.21.15.131:0/276400764 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b0c0125f0 con 0x7f3b1415f560 2024-05-26T22:48:34.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.894+0000 7f3b0aa00640 1 -- 172.21.15.131:0/276400764 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b0c019070 con 0x7f3b1415f560 2024-05-26T22:48:34.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.894+0000 7f3b0aa00640 1 -- 172.21.15.131:0/276400764 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b0c0129f0 con 0x7f3b1415f560 2024-05-26T22:48:34.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.894+0000 7f3b19800640 1 -- 172.21.15.131:0/276400764 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b14155000 con 0x7f3b1415f560 2024-05-26T22:48:34.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.894+0000 7f3b19800640 1 -- 172.21.15.131:0/276400764 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b141555c0 con 0x7f3b1415f560 2024-05-26T22:48:34.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.895+0000 7f3b19800640 1 -- 172.21.15.131:0/276400764 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b140694e0 con 0x7f3b1415f560 2024-05-26T22:48:34.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.896+0000 7f3b0aa00640 1 -- 172.21.15.131:0/276400764 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3b0c012b90 con 0x7f3b1415f560 2024-05-26T22:48:34.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.898+0000 7f3b0aa00640 1 --2- 172.21.15.131:0/276400764 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f3af006be30 0x7f3af006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:34.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.898+0000 7f3b0aa00640 1 -- 172.21.15.131:0/276400764 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f3b0c08b2d0 con 0x7f3b1415f560 2024-05-26T22:48:34.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.898+0000 7f3b12a00640 1 --2- 172.21.15.131:0/276400764 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f3af006be30 0x7f3af006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:34.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.898+0000 7f3b12a00640 1 --2- 172.21.15.131:0/276400764 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f3af006be30 0x7f3af006e2f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3b04005880 tx=0x7f3b04009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:34.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:34.902+0000 7f3b0aa00640 1 -- 172.21.15.131:0/276400764 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f3b0c059bf0 con 0x7f3b1415f560 2024-05-26T22:48:35.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.134+0000 7f3b19800640 1 -- 172.21.15.131:0/276400764 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3b14067480 con 0x7f3b1415f560 2024-05-26T22:48:35.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.134+0000 7f3b0aa00640 1 -- 172.21.15.131:0/276400764 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5184 (secure 0 0 0) 0x7f3b0c0594f0 con 0x7f3b1415f560 2024-05-26T22:48:35.135 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:48:35.135 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/3266175650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6833","nonce":3266175650},{"type":"v1","addr":"172.21.15.187:6835","nonce":3266175650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/1862866305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1862866305},{"type":"v1","addr":"172.21.15.131:6835","nonce":1862866305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4269,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/618294324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":618294324},{"type":"v1","addr":"172.21.15.131:6837","nonce":618294324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-26T22:48:21.211047+0000","modified":"2024-05-26T22:48:24.550610+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6834/2547512415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":2547512415},{"type":"v1","addr":"172.21.15.187:6834","nonce":2547512415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4281,"qdb_cluster":[4281]},"id":1}]} 2024-05-26T22:48:35.135 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 7 2024-05-26T22:48:35.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.137+0000 7f3b19800640 1 -- 172.21.15.131:0/276400764 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f3af006be30 msgr2=0x7f3af006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:35.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.137+0000 7f3b19800640 1 --2- 172.21.15.131:0/276400764 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f3af006be30 0x7f3af006e2f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3b04005880 tx=0x7f3b04009040 comp rx=0 tx=0).stop 2024-05-26T22:48:35.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.137+0000 7f3b19800640 1 -- 172.21.15.131:0/276400764 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b1415f560 msgr2=0x7f3b14150080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:35.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.137+0000 7f3b19800640 1 --2- 172.21.15.131:0/276400764 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b1415f560 0x7f3b14150080 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c00d5d0 tx=0x7f3b0c00da90 comp rx=0 tx=0).stop 2024-05-26T22:48:35.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.137+0000 7f3b19800640 1 -- 172.21.15.131:0/276400764 shutdown_connections 2024-05-26T22:48:35.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.137+0000 7f3b19800640 1 --2- 172.21.15.131:0/276400764 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f3af006be30 0x7f3af006e2f0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:35.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.137+0000 7f3b19800640 1 --2- 172.21.15.131:0/276400764 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b1415f560 0x7f3b14150080 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:35.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.137+0000 7f3b19800640 1 --2- 172.21.15.131:0/276400764 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b14158c70 0x7f3b1414f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:35.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.137+0000 7f3b19800640 1 --2- 172.21.15.131:0/276400764 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3b1406fce0 0x7f3b141f37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:35.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.137+0000 7f3b19800640 1 -- 172.21.15.131:0/276400764 >> 172.21.15.131:0/276400764 conn(0x7f3b14072b00 msgr2=0x7f3b14161e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:35.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.137+0000 7f3b19800640 1 -- 172.21.15.131:0/276400764 shutdown_connections 2024-05-26T22:48:35.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.138+0000 7f3b19800640 1 -- 172.21.15.131:0/276400764 wait complete. 2024-05-26T22:48:35.156 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-26T22:48:35.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.345+0000 7fb2dfe00640 1 Processor -- start 2024-05-26T22:48:35.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.346+0000 7fb2dfe00640 1 -- start start 2024-05-26T22:48:35.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.347+0000 7fb2dfe00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fb2e006e8c0 0x7fb2e015d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:35.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.347+0000 7fb2dfe00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2e0160620 con 0x7fb2e00cc7f0 2024-05-26T22:48:35.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.347+0000 7fb2dfe00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2e0160900 con 0x7fb2e0156750 2024-05-26T22:48:35.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.347+0000 7fb2dfe00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fb2e015cbd0 con 0x7fb2e006e8c0 2024-05-26T22:48:35.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.348+0000 7fb2df400640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fb2e00cc7f0 0x7fb2e00cdc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:40100/0 (socket says 172.21.15.131:40100) 2024-05-26T22:48:35.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.348+0000 7fb2df400640 1 -- 172.21.15.131:0/1201933891 learned_addr learned my addr 172.21.15.131:0/1201933891 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:48:35.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.348+0000 7fb2d6000640 1 --2- 172.21.15.131:0/1201933891 >> v2:172.21.15.187:3300/0 conn(0x7fb2e006e8c0 0x7fb2e015d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:35.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.348+0000 7fb2de000640 1 -- 172.21.15.131:0/1201933891 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3623933266 0 0) 0x7fb2e0160620 con 0x7fb2e00cc7f0 2024-05-26T22:48:35.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.348+0000 7fb2de000640 1 -- 172.21.15.131:0/1201933891 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb2c4003520 con 0x7fb2e00cc7f0 2024-05-26T22:48:35.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.348+0000 7fb2de000640 1 -- 172.21.15.131:0/1201933891 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2341071027 0 0) 0x7fb2c4003520 con 0x7fb2e00cc7f0 2024-05-26T22:48:35.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.348+0000 7fb2de000640 1 -- 172.21.15.131:0/1201933891 >> v2:172.21.15.187:3300/0 conn(0x7fb2e006e8c0 msgr2=0x7fb2e015d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:35.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.348+0000 7fb2de000640 1 --2- 172.21.15.131:0/1201933891 >> v2:172.21.15.187:3300/0 conn(0x7fb2e006e8c0 0x7fb2e015d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:35.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.348+0000 7fb2de000640 1 -- 172.21.15.131:0/1201933891 >> v1:172.21.15.187:6789/0 conn(0x7fb2e0156750 legacy=0x7fb2e015fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:35.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.349+0000 7fb2de000640 1 -- 172.21.15.131:0/1201933891 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2e0160be0 con 0x7fb2e00cc7f0 2024-05-26T22:48:35.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.349+0000 7fb2de000640 1 -- 172.21.15.131:0/1201933891 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7fb2d80019f0 con 0x7fb2e00cc7f0 2024-05-26T22:48:35.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.349+0000 7fb2de000640 1 -- 172.21.15.131:0/1201933891 >> v1:172.21.15.131:6789/0 conn(0x7fb2e00cc7f0 legacy=0x7fb2e00cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:35.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.349+0000 7fb2de000640 1 --2- 172.21.15.131:0/1201933891 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb2c4003a00 0x7fb2c4003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:35.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.349+0000 7fb2de000640 1 --2- 172.21.15.131:0/1201933891 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb2c4004800 0x7fb2c4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:35.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.349+0000 7fb2de000640 1 --2- 172.21.15.131:0/1201933891 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb2c40051c0 0x7fb2c400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:35.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.349+0000 7fb2de000640 1 -- 172.21.15.131:0/1201933891 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e015cbd0 con 0x7fb2c4004800 2024-05-26T22:48:35.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.349+0000 7fb2de000640 1 -- 172.21.15.131:0/1201933891 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c400c0f0 con 0x7fb2c40051c0 2024-05-26T22:48:35.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.349+0000 7fb2de000640 1 -- 172.21.15.131:0/1201933891 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb2c400c2f0 con 0x7fb2c4003a00 2024-05-26T22:48:35.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.349+0000 7fb2d6000640 1 --2- 172.21.15.131:0/1201933891 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb2c4004800 0x7fb2c4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:35.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.349+0000 7fb2dea00640 1 --2- 172.21.15.131:0/1201933891 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb2c4003a00 0x7fb2c4003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:35.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.349+0000 7fb2d6000640 1 -- 172.21.15.131:0/1201933891 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb2c4003a00 msgr2=0x7fb2c4003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:35.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.349+0000 7fb2d6000640 1 --2- 172.21.15.131:0/1201933891 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb2c4003a00 0x7fb2c4003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:35.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.349+0000 7fb2d6000640 1 -- 172.21.15.131:0/1201933891 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb2c40051c0 msgr2=0x7fb2c400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:35.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.349+0000 7fb2d6000640 1 --2- 172.21.15.131:0/1201933891 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb2c40051c0 0x7fb2c400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:35.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.349+0000 7fb2d6000640 1 -- 172.21.15.131:0/1201933891 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2c4003520 con 0x7fb2c4004800 2024-05-26T22:48:35.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.349+0000 7fb2df400640 1 -- 172.21.15.131:0/1201933891 reap_dead start 2024-05-26T22:48:35.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.349+0000 7fb2dea00640 1 --2- 172.21.15.131:0/1201933891 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb2c4003a00 0x7fb2c4003e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:48:35.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.350+0000 7fb2d6000640 1 --2- 172.21.15.131:0/1201933891 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb2c4004800 0x7fb2c4004c80 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb2c8009ac0 tx=0x7fb2c4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=979f1d373ff43f9d server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:35.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.350+0000 7fb2de000640 1 -- 172.21.15.131:0/1201933891 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c8036070 con 0x7fb2c4004800 2024-05-26T22:48:35.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.351+0000 7fb2de000640 1 -- 172.21.15.131:0/1201933891 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2c8031400 con 0x7fb2c4004800 2024-05-26T22:48:35.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.351+0000 7fb2de000640 1 -- 172.21.15.131:0/1201933891 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c8009690 con 0x7fb2c4004800 2024-05-26T22:48:35.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.351+0000 7fb2dfe00640 1 -- 172.21.15.131:0/1201933891 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb2c4004800 msgr2=0x7fb2c4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:35.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.351+0000 7fb2dfe00640 1 --2- 172.21.15.131:0/1201933891 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb2c4004800 0x7fb2c4004c80 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb2c8009ac0 tx=0x7fb2c4005830 comp rx=0 tx=0).stop 2024-05-26T22:48:35.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.351+0000 7fb2dfe00640 1 -- 172.21.15.131:0/1201933891 shutdown_connections 2024-05-26T22:48:35.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.351+0000 7fb2dfe00640 1 --2- 172.21.15.131:0/1201933891 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb2c4004800 0x7fb2c4004c80 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:35.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.351+0000 7fb2dfe00640 1 -- 172.21.15.131:0/1201933891 >> 172.21.15.131:0/1201933891 conn(0x7fb2e00cff40 msgr2=0x7fb2e00d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:35.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.351+0000 7fb2dfe00640 1 -- 172.21.15.131:0/1201933891 shutdown_connections 2024-05-26T22:48:35.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.351+0000 7fb2dfe00640 1 -- 172.21.15.131:0/1201933891 wait complete. 2024-05-26T22:48:35.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.353+0000 7fb2dfe00640 1 Processor -- start 2024-05-26T22:48:35.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.353+0000 7fb2dfe00640 1 -- start start 2024-05-26T22:48:35.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.354+0000 7fb2dfe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb2c4004800 0x7fb2e0154a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:35.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.354+0000 7fb2dfe00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb2e00ab5a0 0x7fb2e00aba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:35.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.354+0000 7fb2dfe00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb2e00abf60 0x7fb2e0156cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:35.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.354+0000 7fb2dfe00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e00ccc60 con 0x7fb2c4004800 2024-05-26T22:48:35.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.354+0000 7fb2dfe00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e0156b30 con 0x7fb2e00ab5a0 2024-05-26T22:48:35.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.354+0000 7fb2dfe00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb2e00ccde0 con 0x7fb2e00abf60 2024-05-26T22:48:35.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.354+0000 7fb2dea00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb2c4004800 0x7fb2e0154a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:35.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.354+0000 7fb2dea00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb2c4004800 0x7fb2e0154a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58126/0 (socket says 172.21.15.131:58126) 2024-05-26T22:48:35.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.354+0000 7fb2dea00640 1 -- 172.21.15.131:0/3601217753 learned_addr learned my addr 172.21.15.131:0/3601217753 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:35.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.355+0000 7fb2df400640 1 --2- 172.21.15.131:0/3601217753 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb2e00abf60 0x7fb2e0156cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:35.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.355+0000 7fb2dea00640 1 -- 172.21.15.131:0/3601217753 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb2e00abf60 msgr2=0x7fb2e0156cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:35.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.355+0000 7fb2dea00640 1 --2- 172.21.15.131:0/3601217753 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb2e00abf60 0x7fb2e0156cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:35.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.355+0000 7fb2dea00640 1 -- 172.21.15.131:0/3601217753 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb2e00ab5a0 msgr2=0x7fb2e00aba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:35.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.355+0000 7fb2d6000640 1 --2- 172.21.15.131:0/3601217753 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb2e00ab5a0 0x7fb2e00aba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:35.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.355+0000 7fb2dea00640 1 --2- 172.21.15.131:0/3601217753 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb2e00ab5a0 0x7fb2e00aba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:35.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.355+0000 7fb2dea00640 1 -- 172.21.15.131:0/3601217753 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2e00aa9e0 con 0x7fb2c4004800 2024-05-26T22:48:35.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.355+0000 7fb2df400640 1 --2- 172.21.15.131:0/3601217753 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb2e00abf60 0x7fb2e0156cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:48:35.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.355+0000 7fb2dea00640 1 --2- 172.21.15.131:0/3601217753 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb2c4004800 0x7fb2e0154a20 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb2d000d4b0 tx=0x7fb2d000d970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:35.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.355+0000 7fb2d7e00640 1 -- 172.21.15.131:0/3601217753 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d0005480 con 0x7fb2c4004800 2024-05-26T22:48:35.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.355+0000 7fb2d7e00640 1 -- 172.21.15.131:0/3601217753 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2d0017070 con 0x7fb2c4004800 2024-05-26T22:48:35.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.355+0000 7fb2dfe00640 1 -- 172.21.15.131:0/3601217753 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2e00af350 con 0x7fb2c4004800 2024-05-26T22:48:35.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.355+0000 7fb2d7e00640 1 -- 172.21.15.131:0/3601217753 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d0003830 con 0x7fb2c4004800 2024-05-26T22:48:35.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.355+0000 7fb2dfe00640 1 -- 172.21.15.131:0/3601217753 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2e00ae590 con 0x7fb2c4004800 2024-05-26T22:48:35.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.356+0000 7fb2dfe00640 1 -- 172.21.15.131:0/3601217753 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2e00694e0 con 0x7fb2c4004800 2024-05-26T22:48:35.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.357+0000 7fb2d7e00640 1 -- 172.21.15.131:0/3601217753 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb2d00039d0 con 0x7fb2c4004800 2024-05-26T22:48:35.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.358+0000 7fb2d7e00640 1 --2- 172.21.15.131:0/3601217753 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb2bc06be60 0x7fb2bc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:35.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.358+0000 7fb2d7e00640 1 -- 172.21.15.131:0/3601217753 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb2d008acc0 con 0x7fb2c4004800 2024-05-26T22:48:35.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.358+0000 7fb2d6000640 1 --2- 172.21.15.131:0/3601217753 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb2bc06be60 0x7fb2bc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:35.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.359+0000 7fb2d6000640 1 --2- 172.21.15.131:0/3601217753 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb2bc06be60 0x7fb2bc06e320 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb2c8033000 tx=0x7fb2c8008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:35.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.362+0000 7fb2d7e00640 1 -- 172.21.15.131:0/3601217753 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fb2d0091080 con 0x7fb2c4004800 2024-05-26T22:48:35.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.549+0000 7fb2dfe00640 1 -- 172.21.15.131:0/3601217753 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb2e00cf740 con 0x7fb2c4004800 2024-05-26T22:48:35.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.551+0000 7fb2d7e00640 1 -- 172.21.15.131:0/3601217753 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16629 (secure 0 0 0) 0x7fb2e00af350 con 0x7fb2c4004800 2024-05-26T22:48:35.551 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:48:35.552 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":20,"fsid":"46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f","created":"2024-05-26T22:47:57.288905+0000","modified":"2024-05-26T22:48:22.799298+0000","last_up_change":"2024-05-26T22:48:09.602207+0000","last_in_change":"2024-05-26T22:48:00.687341+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-26T22:48:10.463339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-26T22:48:19.663254+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-26T22:48:20.235751+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"814eaf62-a18d-494b-8e27-fd8fb88adea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6801","nonce":2350940051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6803","nonce":2350940051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6807","nonce":2350940051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6805","nonce":2350940051}]},"public_addr":"172.21.15.131:6801/2350940051","cluster_addr":"172.21.15.131:6803/2350940051","heartbeat_back_addr":"172.21.15.131:6807/2350940051","heartbeat_front_addr":"172.21.15.131:6805/2350940051","state":["exists","up"]},{"osd":1,"uuid":"42e66287-f6c6-4e26-af3c-6dfaa3536d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6825","nonce":1525528247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6827","nonce":1525528247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6831","nonce":1525528247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6829","nonce":1525528247}]},"public_addr":"172.21.15.131:6825/1525528247","cluster_addr":"172.21.15.131:6827/1525528247","heartbeat_back_addr":"172.21.15.131:6831/1525528247","heartbeat_front_addr":"172.21.15.131:6829/1525528247","state":["exists","up"]},{"osd":2,"uuid":"0da34673-04a2-4fd0-b57a-0ff2b679b043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6817","nonce":3627842210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6819","nonce":3627842210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6823","nonce":3627842210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6821","nonce":3627842210}]},"public_addr":"172.21.15.131:6817/3627842210","cluster_addr":"172.21.15.131:6819/3627842210","heartbeat_back_addr":"172.21.15.131:6823/3627842210","heartbeat_front_addr":"172.21.15.131:6821/3627842210","state":["exists","up"]},{"osd":3,"uuid":"cf117a27-d6b2-43d3-ae52-b47988bf8eed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6809","nonce":350790591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6811","nonce":350790591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6815","nonce":350790591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6813","nonce":350790591}]},"public_addr":"172.21.15.131:6809/350790591","cluster_addr":"172.21.15.131:6811/350790591","heartbeat_back_addr":"172.21.15.131:6815/350790591","heartbeat_front_addr":"172.21.15.131:6813/350790591","state":["exists","up"]},{"osd":4,"uuid":"a092ccfc-be89-4a84-88d5-0f781fa715eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6816","nonce":244560995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6819","nonce":244560995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6823","nonce":244560995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6821","nonce":244560995}]},"public_addr":"172.21.15.187:6816/244560995","cluster_addr":"172.21.15.187:6819/244560995","heartbeat_back_addr":"172.21.15.187:6823/244560995","heartbeat_front_addr":"172.21.15.187:6821/244560995","state":["exists","up"]},{"osd":5,"uuid":"7513ba69-dd00-4a35-b190-9e8cc3b2bdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6801","nonce":324619398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6803","nonce":324619398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6812","nonce":324619398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6807","nonce":324619398}]},"public_addr":"172.21.15.187:6801/324619398","cluster_addr":"172.21.15.187:6803/324619398","heartbeat_back_addr":"172.21.15.187:6812/324619398","heartbeat_front_addr":"172.21.15.187:6807/324619398","state":["exists","up"]},{"osd":6,"uuid":"50081810-c093-4d2f-a4d7-b7c695c5400d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6806","nonce":1391347786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6809","nonce":1391347786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6818","nonce":1391347786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6813","nonce":1391347786}]},"public_addr":"172.21.15.187:6806/1391347786","cluster_addr":"172.21.15.187:6809/1391347786","heartbeat_back_addr":"172.21.15.187:6818/1391347786","heartbeat_front_addr":"172.21.15.187:6813/1391347786","state":["exists","up"]},{"osd":7,"uuid":"775dae05-54f5-4ad4-bbce-92863c6ed2ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6825","nonce":1220065622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6827","nonce":1220065622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6831","nonce":1220065622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6829","nonce":1220065622}]},"public_addr":"172.21.15.187:6825/1220065622","cluster_addr":"172.21.15.187:6827/1220065622","heartbeat_back_addr":"172.21.15.187:6831/1220065622","heartbeat_front_addr":"172.21.15.187:6829/1220065622","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:05.983520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.299516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.067323+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.307748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.018742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.258775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.010912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.295355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-26T22:48:35.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.554+0000 7fb2dfe00640 1 -- 172.21.15.131:0/3601217753 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb2bc06be60 msgr2=0x7fb2bc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:35.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.554+0000 7fb2dfe00640 1 --2- 172.21.15.131:0/3601217753 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb2bc06be60 0x7fb2bc06e320 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb2c8033000 tx=0x7fb2c8008030 comp rx=0 tx=0).stop 2024-05-26T22:48:35.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.554+0000 7fb2dfe00640 1 -- 172.21.15.131:0/3601217753 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb2c4004800 msgr2=0x7fb2e0154a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:35.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.554+0000 7fb2dfe00640 1 --2- 172.21.15.131:0/3601217753 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb2c4004800 0x7fb2e0154a20 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb2d000d4b0 tx=0x7fb2d000d970 comp rx=0 tx=0).stop 2024-05-26T22:48:35.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.554+0000 7fb2dfe00640 1 -- 172.21.15.131:0/3601217753 shutdown_connections 2024-05-26T22:48:35.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.554+0000 7fb2dfe00640 1 --2- 172.21.15.131:0/3601217753 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb2e00abf60 0x7fb2e0156cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:35.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.554+0000 7fb2dfe00640 1 --2- 172.21.15.131:0/3601217753 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb2bc06be60 0x7fb2bc06e320 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:35.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.554+0000 7fb2dfe00640 1 --2- 172.21.15.131:0/3601217753 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb2e00ab5a0 0x7fb2e00aba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:35.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.554+0000 7fb2dfe00640 1 --2- 172.21.15.131:0/3601217753 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb2c4004800 0x7fb2e0154a20 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:35.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.554+0000 7fb2dfe00640 1 -- 172.21.15.131:0/3601217753 >> 172.21.15.131:0/3601217753 conn(0x7fb2e00cff40 msgr2=0x7fb2e015cf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:35.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.554+0000 7fb2dfe00640 1 -- 172.21.15.131:0/3601217753 shutdown_connections 2024-05-26T22:48:35.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.554+0000 7fb2dfe00640 1 -- 172.21.15.131:0/3601217753 wait complete. 2024-05-26T22:48:35.571 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-26T22:48:10.463339+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-26T22:48:19.663254+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-26T22:48:20.235751+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-26T22:48:35.572 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-26T22:48:35.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.765+0000 7f2d26200640 1 Processor -- start 2024-05-26T22:48:35.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.765+0000 7f2d26200640 1 -- start start 2024-05-26T22:48:35.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.767+0000 7f2d26200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f2d2015b730 0x7f2d2015bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:35.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.767+0000 7f2d26200640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f2d201400c0 0x7f2d201404f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:35.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.767+0000 7f2d26200640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d2013e820 con 0x7f2d2006d250 2024-05-26T22:48:35.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.767+0000 7f2d26200640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f2d20071260 con 0x7f2d2015b730 2024-05-26T22:48:35.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.767+0000 7f2d26200640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f2d2015c4a0 con 0x7f2d201400c0 2024-05-26T22:48:35.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.767+0000 7f2d24e00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f2d2006d250 0x7f2d2006d630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:40106/0 (socket says 172.21.15.131:40106) 2024-05-26T22:48:35.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.767+0000 7f2d24e00640 1 -- 172.21.15.131:0/3180333104 learned_addr learned my addr 172.21.15.131:0/3180333104 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:48:35.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.767+0000 7f2d1fe00640 1 --2- 172.21.15.131:0/3180333104 >> v2:172.21.15.131:3300/0 conn(0x7f2d2015b730 0x7f2d2015bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:35.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.767+0000 7f2d1f400640 1 -- 172.21.15.131:0/3180333104 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1168802407 0 0) 0x7f2d2013e820 con 0x7f2d2006d250 2024-05-26T22:48:35.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.767+0000 7f2d1f400640 1 -- 172.21.15.131:0/3180333104 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2d08005730 con 0x7f2d2006d250 2024-05-26T22:48:35.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.767+0000 7f2d25800640 1 --2- 172.21.15.131:0/3180333104 >> v2:172.21.15.187:3300/0 conn(0x7f2d201400c0 0x7f2d201404f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:35.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.768+0000 7f2d1fe00640 1 -- 172.21.15.131:0/3180333104 >> v2:172.21.15.187:3300/0 conn(0x7f2d201400c0 msgr2=0x7f2d201404f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:35.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.768+0000 7f2d1fe00640 1 --2- 172.21.15.131:0/3180333104 >> v2:172.21.15.187:3300/0 conn(0x7f2d201400c0 0x7f2d201404f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:35.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.768+0000 7f2d1fe00640 1 -- 172.21.15.131:0/3180333104 >> v1:172.21.15.131:6789/0 conn(0x7f2d2006d250 legacy=0x7f2d2006d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:35.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.768+0000 7f2d1f400640 1 -- 172.21.15.131:0/3180333104 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2430513362 0 0) 0x7f2d08005730 con 0x7f2d2006d250 2024-05-26T22:48:35.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.768+0000 7f2d1fe00640 1 -- 172.21.15.131:0/3180333104 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d200714b0 con 0x7f2d2015b730 2024-05-26T22:48:35.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.769+0000 7f2d1fe00640 1 --2- 172.21.15.131:0/3180333104 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2d2015b730 0x7f2d2015bbc0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c0097a0 tx=0x7f2d0c02a160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1642ae298e65ecfb server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:35.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.769+0000 7f2d1f400640 1 -- 172.21.15.131:0/3180333104 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d0c037070 con 0x7f2d2015b730 2024-05-26T22:48:35.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.769+0000 7f2d1f400640 1 -- 172.21.15.131:0/3180333104 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d0c02abf0 con 0x7f2d2015b730 2024-05-26T22:48:35.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.769+0000 7f2d1f400640 1 -- 172.21.15.131:0/3180333104 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d14001670 con 0x7f2d2015b730 2024-05-26T22:48:35.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.769+0000 7f2d26200640 1 -- 172.21.15.131:0/3180333104 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2d2015b730 msgr2=0x7f2d2015bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:35.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.769+0000 7f2d26200640 1 --2- 172.21.15.131:0/3180333104 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2d2015b730 0x7f2d2015bbc0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c0097a0 tx=0x7f2d0c02a160 comp rx=0 tx=0).stop 2024-05-26T22:48:35.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.769+0000 7f2d26200640 1 -- 172.21.15.131:0/3180333104 shutdown_connections 2024-05-26T22:48:35.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.769+0000 7f2d26200640 1 --2- 172.21.15.131:0/3180333104 >> v2:172.21.15.187:3300/0 conn(0x7f2d201400c0 0x7f2d201404f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:35.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.769+0000 7f2d26200640 1 --2- 172.21.15.131:0/3180333104 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2d2015b730 0x7f2d2015bbc0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:35.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.769+0000 7f2d26200640 1 -- 172.21.15.131:0/3180333104 >> 172.21.15.131:0/3180333104 conn(0x7f2d200b5660 msgr2=0x7f2d200b5a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:35.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.769+0000 7f2d26200640 1 -- 172.21.15.131:0/3180333104 shutdown_connections 2024-05-26T22:48:35.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.770+0000 7f2d26200640 1 -- 172.21.15.131:0/3180333104 wait complete. 2024-05-26T22:48:35.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.771+0000 7f2d26200640 1 Processor -- start 2024-05-26T22:48:35.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.771+0000 7f2d26200640 1 -- start start 2024-05-26T22:48:35.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.772+0000 7f2d26200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2d2006d250 0x7f2d200ce470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:35.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.772+0000 7f2d26200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2d201400c0 0x7f2d201eded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:35.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.772+0000 7f2d26200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2d2015b730 0x7f2d20156840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:35.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.772+0000 7f2d26200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f2d2015c100 con 0x7f2d201400c0 2024-05-26T22:48:35.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.772+0000 7f2d26200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2d20071a70 con 0x7f2d2006d250 2024-05-26T22:48:35.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.772+0000 7f2d26200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f2d2015c280 con 0x7f2d2015b730 2024-05-26T22:48:35.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.772+0000 7f2d1fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2d201400c0 0x7f2d201eded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:35.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.772+0000 7f2d1fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2d201400c0 0x7f2d201eded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58146/0 (socket says 172.21.15.131:58146) 2024-05-26T22:48:35.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.772+0000 7f2d1fe00640 1 -- 172.21.15.131:0/2446367006 learned_addr learned my addr 172.21.15.131:0/2446367006 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:35.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.773+0000 7f2d1fe00640 1 -- 172.21.15.131:0/2446367006 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2d2015b730 msgr2=0x7f2d20156840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:35.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.773+0000 7f2d1fe00640 1 --2- 172.21.15.131:0/2446367006 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2d2015b730 0x7f2d20156840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:35.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.773+0000 7f2d1fe00640 1 -- 172.21.15.131:0/2446367006 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2d2006d250 msgr2=0x7f2d200ce470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:35.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.773+0000 7f2d1fe00640 1 --2- 172.21.15.131:0/2446367006 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2d2006d250 0x7f2d200ce470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:35.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.773+0000 7f2d1fe00640 1 -- 172.21.15.131:0/2446367006 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d20157320 con 0x7f2d201400c0 2024-05-26T22:48:35.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.773+0000 7f2d1fe00640 1 --2- 172.21.15.131:0/2446367006 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2d201400c0 0x7f2d201eded0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c035040 tx=0x7f2d0c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:35.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.773+0000 7f2d1cc00640 1 -- 172.21.15.131:0/2446367006 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d0c041020 con 0x7f2d201400c0 2024-05-26T22:48:35.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.773+0000 7f2d1cc00640 1 -- 172.21.15.131:0/2446367006 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d0c03d620 con 0x7f2d201400c0 2024-05-26T22:48:35.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.773+0000 7f2d1cc00640 1 -- 172.21.15.131:0/2446367006 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d0c037040 con 0x7f2d201400c0 2024-05-26T22:48:35.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.773+0000 7f2d26200640 1 -- 172.21.15.131:0/2446367006 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d200cd990 con 0x7f2d201400c0 2024-05-26T22:48:35.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.774+0000 7f2d26200640 1 -- 172.21.15.131:0/2446367006 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d200cdf50 con 0x7f2d201400c0 2024-05-26T22:48:35.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.775+0000 7f2d26200640 1 -- 172.21.15.131:0/2446367006 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d200694e0 con 0x7f2d201400c0 2024-05-26T22:48:35.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.776+0000 7f2d1cc00640 1 -- 172.21.15.131:0/2446367006 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2d0c044020 con 0x7f2d201400c0 2024-05-26T22:48:35.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.776+0000 7f2d1cc00640 1 --2- 172.21.15.131:0/2446367006 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f2d0006be30 0x7f2d0006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:35.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.777+0000 7f2d1cc00640 1 -- 172.21.15.131:0/2446367006 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f2d0c0ab460 con 0x7f2d201400c0 2024-05-26T22:48:35.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.777+0000 7f2d24e00640 1 --2- 172.21.15.131:0/2446367006 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f2d0006be30 0x7f2d0006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:35.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.777+0000 7f2d24e00640 1 --2- 172.21.15.131:0/2446367006 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f2d0006be30 0x7f2d0006e2f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2d14001760 tx=0x7f2d14001180 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:35.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.781+0000 7f2d1cc00640 1 -- 172.21.15.131:0/2446367006 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f2d0c04b030 con 0x7f2d201400c0 2024-05-26T22:48:35.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.972+0000 7f2d26200640 1 -- 172.21.15.131:0/2446367006 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2d200ce120 con 0x7f2d201400c0 2024-05-26T22:48:35.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.973+0000 7f2d1cc00640 1 -- 172.21.15.131:0/2446367006 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f2d0c079e00 con 0x7f2d201400c0 2024-05-26T22:48:35.973 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2024-05-26T22:48:35.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.976+0000 7f2d26200640 1 -- 172.21.15.131:0/2446367006 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f2d0006be30 msgr2=0x7f2d0006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:35.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.976+0000 7f2d26200640 1 --2- 172.21.15.131:0/2446367006 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f2d0006be30 0x7f2d0006e2f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2d14001760 tx=0x7f2d14001180 comp rx=0 tx=0).stop 2024-05-26T22:48:35.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.976+0000 7f2d26200640 1 -- 172.21.15.131:0/2446367006 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2d201400c0 msgr2=0x7f2d201eded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:35.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.976+0000 7f2d26200640 1 --2- 172.21.15.131:0/2446367006 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2d201400c0 0x7f2d201eded0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c035040 tx=0x7f2d0c0046c0 comp rx=0 tx=0).stop 2024-05-26T22:48:35.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.976+0000 7f2d26200640 1 -- 172.21.15.131:0/2446367006 shutdown_connections 2024-05-26T22:48:35.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.976+0000 7f2d26200640 1 --2- 172.21.15.131:0/2446367006 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2d2015b730 0x7f2d20156840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:35.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.976+0000 7f2d26200640 1 --2- 172.21.15.131:0/2446367006 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f2d0006be30 0x7f2d0006e2f0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:35.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.976+0000 7f2d26200640 1 --2- 172.21.15.131:0/2446367006 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2d201400c0 0x7f2d201eded0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:35.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.976+0000 7f2d26200640 1 --2- 172.21.15.131:0/2446367006 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2d2006d250 0x7f2d200ce470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:35.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.976+0000 7f2d26200640 1 -- 172.21.15.131:0/2446367006 >> 172.21.15.131:0/2446367006 conn(0x7f2d200b5660 msgr2=0x7f2d20159270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:35.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.976+0000 7f2d26200640 1 -- 172.21.15.131:0/2446367006 shutdown_connections 2024-05-26T22:48:35.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:35.976+0000 7f2d26200640 1 -- 172.21.15.131:0/2446367006 wait complete. 2024-05-26T22:48:35.993 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-26T22:48:36.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.191+0000 7fb284c00640 1 Processor -- start 2024-05-26T22:48:36.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.191+0000 7fb284c00640 1 -- start start 2024-05-26T22:48:36.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.193+0000 7fb284c00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb28015f6e0 0x7fb2800ad870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:36.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.193+0000 7fb284c00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fb28006b680 0x7fb28006bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:36.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.193+0000 7fb284c00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2800ae0c0 con 0x7fb2800af580 2024-05-26T22:48:36.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.193+0000 7fb284c00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fb2800ae3a0 con 0x7fb28006b680 2024-05-26T22:48:36.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.193+0000 7fb284c00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb2800ae680 con 0x7fb28015f6e0 2024-05-26T22:48:36.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.193+0000 7fb27e800640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fb28006b680 0x7fb28006bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:36.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.193+0000 7fb27e800640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fb28006b680 0x7fb28006bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58152/0 (socket says 172.21.15.131:58152) 2024-05-26T22:48:36.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.193+0000 7fb27e800640 1 -- 172.21.15.131:0/3211291201 learned_addr learned my addr 172.21.15.131:0/3211291201 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:36.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.193+0000 7fb27d400640 1 --2- 172.21.15.131:0/3211291201 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb28015f6e0 0x7fb2800ad870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:36.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.194+0000 7fb27e800640 1 -- 172.21.15.131:0/3211291201 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb28015f6e0 msgr2=0x7fb2800ad870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:36.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.194+0000 7fb27e800640 1 --2- 172.21.15.131:0/3211291201 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb28015f6e0 0x7fb2800ad870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:36.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.194+0000 7fb27e800640 1 -- 172.21.15.131:0/3211291201 >> v1:172.21.15.187:6789/0 conn(0x7fb2800af580 legacy=0x7fb2800af960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:36.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.194+0000 7fb27e800640 1 -- 172.21.15.131:0/3211291201 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2800ae960 con 0x7fb28006b680 2024-05-26T22:48:36.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.195+0000 7fb27e800640 1 --2- 172.21.15.131:0/3211291201 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb28006b680 0x7fb28006bb10 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb278009e50 tx=0x7fb27802a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d9f02de4e710ca1 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:36.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.195+0000 7fb27ca00640 1 -- 172.21.15.131:0/3211291201 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb278036070 con 0x7fb28006b680 2024-05-26T22:48:36.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.195+0000 7fb27ca00640 1 -- 172.21.15.131:0/3211291201 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb278031600 con 0x7fb28006b680 2024-05-26T22:48:36.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.195+0000 7fb27ca00640 1 -- 172.21.15.131:0/3211291201 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2780319d0 con 0x7fb28006b680 2024-05-26T22:48:36.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.196+0000 7fb284c00640 1 -- 172.21.15.131:0/3211291201 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb28006b680 msgr2=0x7fb28006bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:36.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.196+0000 7fb284c00640 1 --2- 172.21.15.131:0/3211291201 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb28006b680 0x7fb28006bb10 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb278009e50 tx=0x7fb27802a5f0 comp rx=0 tx=0).stop 2024-05-26T22:48:36.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.196+0000 7fb284c00640 1 -- 172.21.15.131:0/3211291201 shutdown_connections 2024-05-26T22:48:36.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.196+0000 7fb284c00640 1 --2- 172.21.15.131:0/3211291201 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb28006b680 0x7fb28006bb10 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:36.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.196+0000 7fb284c00640 1 --2- 172.21.15.131:0/3211291201 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb28015f6e0 0x7fb2800ad870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:36.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.196+0000 7fb284c00640 1 -- 172.21.15.131:0/3211291201 >> 172.21.15.131:0/3211291201 conn(0x7fb280072b00 msgr2=0x7fb28006de50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:36.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.196+0000 7fb284c00640 1 -- 172.21.15.131:0/3211291201 shutdown_connections 2024-05-26T22:48:36.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.196+0000 7fb284c00640 1 -- 172.21.15.131:0/3211291201 wait complete. 2024-05-26T22:48:36.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.198+0000 7fb284c00640 1 Processor -- start 2024-05-26T22:48:36.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.198+0000 7fb284c00640 1 -- start start 2024-05-26T22:48:36.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.199+0000 7fb284c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb28006b680 0x7fb28015b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:36.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.199+0000 7fb284c00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb2800af580 0x7fb280154910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:36.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.199+0000 7fb284c00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb28015f6e0 0x7fb2800ce920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:36.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.199+0000 7fb284c00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fb2800b6a40 con 0x7fb28006b680 2024-05-26T22:48:36.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.199+0000 7fb284c00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb2800b68c0 con 0x7fb2800af580 2024-05-26T22:48:36.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.199+0000 7fb284c00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb28006c050 con 0x7fb28015f6e0 2024-05-26T22:48:36.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.199+0000 7fb27de00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb28006b680 0x7fb28015b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:36.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.199+0000 7fb27de00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb28006b680 0x7fb28015b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58156/0 (socket says 172.21.15.131:58156) 2024-05-26T22:48:36.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.199+0000 7fb27de00640 1 -- 172.21.15.131:0/2047424587 learned_addr learned my addr 172.21.15.131:0/2047424587 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:36.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.199+0000 7fb27de00640 1 -- 172.21.15.131:0/2047424587 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb28015f6e0 msgr2=0x7fb2800ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:36.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.199+0000 7fb27e800640 1 --2- 172.21.15.131:0/2047424587 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb28015f6e0 0x7fb2800ce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:36.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.199+0000 7fb27d400640 1 --2- 172.21.15.131:0/2047424587 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb2800af580 0x7fb280154910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:36.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.199+0000 7fb27de00640 1 --2- 172.21.15.131:0/2047424587 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb28015f6e0 0x7fb2800ce920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:36.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.199+0000 7fb27de00640 1 -- 172.21.15.131:0/2047424587 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb2800af580 msgr2=0x7fb280154910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:36.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.199+0000 7fb27de00640 1 --2- 172.21.15.131:0/2047424587 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb2800af580 0x7fb280154910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:36.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.199+0000 7fb27de00640 1 -- 172.21.15.131:0/2047424587 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb280154e50 con 0x7fb28006b680 2024-05-26T22:48:36.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.199+0000 7fb27e800640 1 --2- 172.21.15.131:0/2047424587 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb28015f6e0 0x7fb2800ce920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:48:36.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.200+0000 7fb27de00640 1 --2- 172.21.15.131:0/2047424587 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb28006b680 0x7fb28015b8e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb27000c450 tx=0x7fb27000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:36.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.200+0000 7fb276000640 1 -- 172.21.15.131:0/2047424587 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2700052d0 con 0x7fb28006b680 2024-05-26T22:48:36.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.200+0000 7fb276000640 1 -- 172.21.15.131:0/2047424587 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb270005470 con 0x7fb28006b680 2024-05-26T22:48:36.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.200+0000 7fb276000640 1 -- 172.21.15.131:0/2047424587 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb270012870 con 0x7fb28006b680 2024-05-26T22:48:36.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.200+0000 7fb284c00640 1 -- 172.21.15.131:0/2047424587 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2800cd400 con 0x7fb28006b680 2024-05-26T22:48:36.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.200+0000 7fb284c00640 1 -- 172.21.15.131:0/2047424587 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2800cf990 con 0x7fb28006b680 2024-05-26T22:48:36.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.201+0000 7fb284c00640 1 -- 172.21.15.131:0/2047424587 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb28006e680 con 0x7fb28006b680 2024-05-26T22:48:36.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.202+0000 7fb276000640 1 -- 172.21.15.131:0/2047424587 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb270012a10 con 0x7fb28006b680 2024-05-26T22:48:36.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.203+0000 7fb276000640 1 --2- 172.21.15.131:0/2047424587 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb26006be30 0x7fb26006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:36.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.203+0000 7fb276000640 1 -- 172.21.15.131:0/2047424587 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb27008ab80 con 0x7fb28006b680 2024-05-26T22:48:36.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.203+0000 7fb27d400640 1 --2- 172.21.15.131:0/2047424587 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb26006be30 0x7fb26006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:36.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.204+0000 7fb27d400640 1 --2- 172.21.15.131:0/2047424587 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb26006be30 0x7fb26006e2f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb28015ec70 tx=0x7fb26c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:36.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.207+0000 7fb276000640 1 -- 172.21.15.131:0/2047424587 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fb28006e680 con 0x7fb28006b680 2024-05-26T22:48:36.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.394+0000 7fb284c00640 1 -- 172.21.15.131:0/2047424587 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb2800695c0 con 0x7fb28006b680 2024-05-26T22:48:36.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.395+0000 7fb276000640 1 -- 172.21.15.131:0/2047424587 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7fb2800cd400 con 0x7fb28006b680 2024-05-26T22:48:36.395 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-05-26T22:48:36.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.397+0000 7fb284c00640 1 -- 172.21.15.131:0/2047424587 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb26006be30 msgr2=0x7fb26006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:36.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.397+0000 7fb284c00640 1 --2- 172.21.15.131:0/2047424587 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb26006be30 0x7fb26006e2f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb28015ec70 tx=0x7fb26c008040 comp rx=0 tx=0).stop 2024-05-26T22:48:36.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.397+0000 7fb284c00640 1 -- 172.21.15.131:0/2047424587 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb28006b680 msgr2=0x7fb28015b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:36.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.397+0000 7fb284c00640 1 --2- 172.21.15.131:0/2047424587 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb28006b680 0x7fb28015b8e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb27000c450 tx=0x7fb27000c910 comp rx=0 tx=0).stop 2024-05-26T22:48:36.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.398+0000 7fb284c00640 1 -- 172.21.15.131:0/2047424587 shutdown_connections 2024-05-26T22:48:36.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.398+0000 7fb284c00640 1 --2- 172.21.15.131:0/2047424587 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb28015f6e0 0x7fb2800ce920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:36.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.398+0000 7fb284c00640 1 --2- 172.21.15.131:0/2047424587 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb26006be30 0x7fb26006e2f0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:36.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.398+0000 7fb284c00640 1 --2- 172.21.15.131:0/2047424587 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb2800af580 0x7fb280154910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:36.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.398+0000 7fb284c00640 1 --2- 172.21.15.131:0/2047424587 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb28006b680 0x7fb28015b8e0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:36.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.398+0000 7fb284c00640 1 -- 172.21.15.131:0/2047424587 >> 172.21.15.131:0/2047424587 conn(0x7fb280072b00 msgr2=0x7fb28006e880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:36.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.398+0000 7fb284c00640 1 -- 172.21.15.131:0/2047424587 shutdown_connections 2024-05-26T22:48:36.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.398+0000 7fb284c00640 1 -- 172.21.15.131:0/2047424587 wait complete. 2024-05-26T22:48:36.415 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-26T22:48:36.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.610+0000 7f199dc00640 1 Processor -- start 2024-05-26T22:48:36.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.610+0000 7f199dc00640 1 -- start start 2024-05-26T22:48:36.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.612+0000 7f199dc00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f1998155340 0x7f199815e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:36.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.612+0000 7f199dc00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1998073190 0x7f19980ae280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:36.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.612+0000 7f199dc00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f199815f860 con 0x7f19980b18b0 2024-05-26T22:48:36.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.612+0000 7f199dc00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f19980d11b0 con 0x7f1998155340 2024-05-26T22:48:36.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.612+0000 7f199dc00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f19980af590 con 0x7f1998073190 2024-05-26T22:48:36.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.612+0000 7f1997e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f1998155340 0x7f199815e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:36.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.612+0000 7f1997e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f1998155340 0x7f199815e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58166/0 (socket says 172.21.15.131:58166) 2024-05-26T22:48:36.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.612+0000 7f1997e00640 1 -- 172.21.15.131:0/442138032 learned_addr learned my addr 172.21.15.131:0/442138032 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:36.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.613+0000 7f199d200640 1 --2- 172.21.15.131:0/442138032 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1998073190 0x7f19980ae280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:36.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.613+0000 7f1997e00640 1 -- 172.21.15.131:0/442138032 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1998073190 msgr2=0x7f19980ae280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:36.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.613+0000 7f1997e00640 1 --2- 172.21.15.131:0/442138032 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1998073190 0x7f19980ae280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:36.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.613+0000 7f1997e00640 1 -- 172.21.15.131:0/442138032 >> v1:172.21.15.187:6789/0 conn(0x7f19980b18b0 legacy=0x7f1998072a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:36.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.613+0000 7f1997e00640 1 -- 172.21.15.131:0/442138032 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f199815efc0 con 0x7f1998155340 2024-05-26T22:48:36.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.614+0000 7f1997e00640 1 --2- 172.21.15.131:0/442138032 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1998155340 0x7f199815e900 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1988009870 tx=0x7f198802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe8f2f645aa7f7cd server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:36.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.614+0000 7f1996a00640 1 -- 172.21.15.131:0/442138032 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1988037070 con 0x7f1998155340 2024-05-26T22:48:36.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.614+0000 7f1996a00640 1 -- 172.21.15.131:0/442138032 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1988032440 con 0x7f1998155340 2024-05-26T22:48:36.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.614+0000 7f1996a00640 1 -- 172.21.15.131:0/442138032 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1988032810 con 0x7f1998155340 2024-05-26T22:48:36.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.615+0000 7f199dc00640 1 -- 172.21.15.131:0/442138032 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1998155340 msgr2=0x7f199815e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:36.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.615+0000 7f199dc00640 1 --2- 172.21.15.131:0/442138032 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1998155340 0x7f199815e900 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1988009870 tx=0x7f198802a180 comp rx=0 tx=0).stop 2024-05-26T22:48:36.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.615+0000 7f199dc00640 1 -- 172.21.15.131:0/442138032 shutdown_connections 2024-05-26T22:48:36.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.615+0000 7f199dc00640 1 --2- 172.21.15.131:0/442138032 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1998073190 0x7f19980ae280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:36.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.615+0000 7f199dc00640 1 --2- 172.21.15.131:0/442138032 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1998155340 0x7f199815e900 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:36.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.615+0000 7f199dc00640 1 -- 172.21.15.131:0/442138032 >> 172.21.15.131:0/442138032 conn(0x7f19980b57b0 msgr2=0x7f19980b5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:36.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.615+0000 7f199dc00640 1 -- 172.21.15.131:0/442138032 shutdown_connections 2024-05-26T22:48:36.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.615+0000 7f199dc00640 1 -- 172.21.15.131:0/442138032 wait complete. 2024-05-26T22:48:36.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.617+0000 7f199dc00640 1 Processor -- start 2024-05-26T22:48:36.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.618+0000 7f199dc00640 1 -- start start 2024-05-26T22:48:36.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.619+0000 7f199dc00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1998073190 0x7f1998161080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:36.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.619+0000 7f199dc00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f19980b18b0 0x7f1998150fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:36.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.619+0000 7f199dc00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1998155340 0x7f1998099520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:36.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.619+0000 7f199dc00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f1998155720 con 0x7f1998155340 2024-05-26T22:48:36.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.619+0000 7f199dc00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f199815f580 con 0x7f1998073190 2024-05-26T22:48:36.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.619+0000 7f199dc00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f19980b1d10 con 0x7f19980b18b0 2024-05-26T22:48:36.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.619+0000 7f199d200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1998155340 0x7f1998099520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:36.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.619+0000 7f199d200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1998155340 0x7f1998099520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58182/0 (socket says 172.21.15.131:58182) 2024-05-26T22:48:36.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.619+0000 7f199d200640 1 -- 172.21.15.131:0/3319072131 learned_addr learned my addr 172.21.15.131:0/3319072131 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:36.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.619+0000 7f1997e00640 1 --2- 172.21.15.131:0/3319072131 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1998073190 0x7f1998161080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:36.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.619+0000 7f1997400640 1 --2- 172.21.15.131:0/3319072131 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f19980b18b0 0x7f1998150fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:36.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.619+0000 7f199d200640 1 -- 172.21.15.131:0/3319072131 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f19980b18b0 msgr2=0x7f1998150fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:36.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.619+0000 7f199d200640 1 --2- 172.21.15.131:0/3319072131 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f19980b18b0 0x7f1998150fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:36.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.619+0000 7f199d200640 1 -- 172.21.15.131:0/3319072131 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1998073190 msgr2=0x7f1998161080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:36.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.619+0000 7f199d200640 1 --2- 172.21.15.131:0/3319072131 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1998073190 0x7f1998161080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:36.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.619+0000 7f199d200640 1 -- 172.21.15.131:0/3319072131 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19981501f0 con 0x7f1998155340 2024-05-26T22:48:36.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.620+0000 7f199d200640 1 --2- 172.21.15.131:0/3319072131 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1998155340 0x7f1998099520 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f1990007d40 tx=0x7f199000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:36.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.620+0000 7f198fe00640 1 -- 172.21.15.131:0/3319072131 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1990014420 con 0x7f1998155340 2024-05-26T22:48:36.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.620+0000 7f198fe00640 1 -- 172.21.15.131:0/3319072131 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1990019070 con 0x7f1998155340 2024-05-26T22:48:36.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.620+0000 7f198fe00640 1 -- 172.21.15.131:0/3319072131 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1990014820 con 0x7f1998155340 2024-05-26T22:48:36.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.620+0000 7f199dc00640 1 -- 172.21.15.131:0/3319072131 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19981504d0 con 0x7f1998155340 2024-05-26T22:48:36.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.620+0000 7f199dc00640 1 -- 172.21.15.131:0/3319072131 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1998150a90 con 0x7f1998155340 2024-05-26T22:48:36.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.622+0000 7f199dc00640 1 -- 172.21.15.131:0/3319072131 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19980722e0 con 0x7f1998155340 2024-05-26T22:48:36.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.622+0000 7f198fe00640 1 -- 172.21.15.131:0/3319072131 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f19900149c0 con 0x7f1998155340 2024-05-26T22:48:36.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.624+0000 7f198fe00640 1 --2- 172.21.15.131:0/3319072131 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f197806be30 0x7f197806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:36.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.624+0000 7f198fe00640 1 -- 172.21.15.131:0/3319072131 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f199008b610 con 0x7f1998155340 2024-05-26T22:48:36.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.624+0000 7f1997e00640 1 --2- 172.21.15.131:0/3319072131 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f197806be30 0x7f197806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:36.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.625+0000 7f1997e00640 1 --2- 172.21.15.131:0/3319072131 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f197806be30 0x7f197806e2f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f1988034000 tx=0x7f198802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:36.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.628+0000 7f198fe00640 1 -- 172.21.15.131:0/3319072131 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f19980722e0 con 0x7f1998155340 2024-05-26T22:48:36.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.828+0000 7f199dc00640 1 -- 172.21.15.131:0/3319072131 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f1998150c90 con 0x7f1998155340 2024-05-26T22:48:36.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.829+0000 7f198fe00640 1 -- 172.21.15.131:0/3319072131 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f1998150c90 con 0x7f1998155340 2024-05-26T22:48:36.829 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 64 2024-05-26T22:48:36.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.832+0000 7f199dc00640 1 -- 172.21.15.131:0/3319072131 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f197806be30 msgr2=0x7f197806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:36.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.832+0000 7f199dc00640 1 --2- 172.21.15.131:0/3319072131 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f197806be30 0x7f197806e2f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f1988034000 tx=0x7f198802e000 comp rx=0 tx=0).stop 2024-05-26T22:48:36.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.832+0000 7f199dc00640 1 -- 172.21.15.131:0/3319072131 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1998155340 msgr2=0x7f1998099520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:36.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.832+0000 7f199dc00640 1 --2- 172.21.15.131:0/3319072131 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1998155340 0x7f1998099520 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f1990007d40 tx=0x7f199000e4b0 comp rx=0 tx=0).stop 2024-05-26T22:48:36.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.832+0000 7f199dc00640 1 -- 172.21.15.131:0/3319072131 shutdown_connections 2024-05-26T22:48:36.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.832+0000 7f199dc00640 1 --2- 172.21.15.131:0/3319072131 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f19980b18b0 0x7f1998150fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:36.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.832+0000 7f199dc00640 1 --2- 172.21.15.131:0/3319072131 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f197806be30 0x7f197806e2f0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:36.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.832+0000 7f199dc00640 1 --2- 172.21.15.131:0/3319072131 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1998155340 0x7f1998099520 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:36.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.832+0000 7f199dc00640 1 --2- 172.21.15.131:0/3319072131 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1998073190 0x7f1998161080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:36.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.832+0000 7f199dc00640 1 -- 172.21.15.131:0/3319072131 >> 172.21.15.131:0/3319072131 conn(0x7f19980b57b0 msgr2=0x7f19980afa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:36.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.832+0000 7f199dc00640 1 -- 172.21.15.131:0/3319072131 shutdown_connections 2024-05-26T22:48:36.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:36.832+0000 7f199dc00640 1 -- 172.21.15.131:0/3319072131 wait complete. 2024-05-26T22:48:36.851 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-26T22:48:37.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.042+0000 7f85baa00640 1 Processor -- start 2024-05-26T22:48:37.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.043+0000 7f85baa00640 1 -- start start 2024-05-26T22:48:37.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.044+0000 7f85baa00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f85b415e380 0x7f85b40b3300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:37.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.044+0000 7f85baa00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f85b40b6300 0x7f85b40d2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:37.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.044+0000 7f85baa00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85b415e760 con 0x7f85b40b7310 2024-05-26T22:48:37.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.044+0000 7f85baa00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f85b40d2a50 con 0x7f85b40b6300 2024-05-26T22:48:37.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.044+0000 7f85baa00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f85b40cf330 con 0x7f85b415e380 2024-05-26T22:48:37.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.045+0000 7f85b9600640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f85b415e380 0x7f85b40b3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:37.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.045+0000 7f85ba000640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f85b40b6300 0x7f85b40d2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:37.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.045+0000 7f85b9600640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f85b415e380 0x7f85b40b3300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.131:58228/0 (socket says 172.21.15.131:58228) 2024-05-26T22:48:37.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.045+0000 7f85b9600640 1 -- 172.21.15.131:0/1096474891 learned_addr learned my addr 172.21.15.131:0/1096474891 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:37.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.045+0000 7f85b9600640 1 -- 172.21.15.131:0/1096474891 >> v2:172.21.15.187:3300/0 conn(0x7f85b40b6300 msgr2=0x7f85b40d2ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:37.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.045+0000 7f85b9600640 1 --2- 172.21.15.131:0/1096474891 >> v2:172.21.15.187:3300/0 conn(0x7f85b40b6300 0x7f85b40d2ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:37.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.045+0000 7f85b9600640 1 -- 172.21.15.131:0/1096474891 >> v1:172.21.15.187:6789/0 conn(0x7f85b40b7310 legacy=0x7f85b40b5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:37.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.045+0000 7f85b9600640 1 -- 172.21.15.131:0/1096474891 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85b40cf610 con 0x7f85b415e380 2024-05-26T22:48:37.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.047+0000 7f85b9600640 1 --2- 172.21.15.131:0/1096474891 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f85b415e380 0x7f85b40b3300 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f85a8009e50 tx=0x7f85a802a680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=87a4d03d8c3600f5 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:37.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.047+0000 7f85b3e00640 1 -- 172.21.15.131:0/1096474891 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a8036070 con 0x7f85b415e380 2024-05-26T22:48:37.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.048+0000 7f85b3e00640 1 -- 172.21.15.131:0/1096474891 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85a8031600 con 0x7f85b415e380 2024-05-26T22:48:37.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.048+0000 7f85b3e00640 1 -- 172.21.15.131:0/1096474891 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a80319d0 con 0x7f85b415e380 2024-05-26T22:48:37.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.048+0000 7f85baa00640 1 -- 172.21.15.131:0/1096474891 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f85b415e380 msgr2=0x7f85b40b3300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:37.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.048+0000 7f85baa00640 1 --2- 172.21.15.131:0/1096474891 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f85b415e380 0x7f85b40b3300 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f85a8009e50 tx=0x7f85a802a680 comp rx=0 tx=0).stop 2024-05-26T22:48:37.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.048+0000 7f85baa00640 1 -- 172.21.15.131:0/1096474891 shutdown_connections 2024-05-26T22:48:37.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.048+0000 7f85baa00640 1 --2- 172.21.15.131:0/1096474891 >> v2:172.21.15.187:3300/0 conn(0x7f85b40b6300 0x7f85b40d2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:37.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.048+0000 7f85baa00640 1 --2- 172.21.15.131:0/1096474891 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f85b415e380 0x7f85b40b3300 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:37.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.048+0000 7f85baa00640 1 -- 172.21.15.131:0/1096474891 >> 172.21.15.131:0/1096474891 conn(0x7f85b4073070 msgr2=0x7f85b40d0bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:37.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.049+0000 7f85baa00640 1 -- 172.21.15.131:0/1096474891 shutdown_connections 2024-05-26T22:48:37.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.049+0000 7f85baa00640 1 -- 172.21.15.131:0/1096474891 wait complete. 2024-05-26T22:48:37.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.051+0000 7f85baa00640 1 Processor -- start 2024-05-26T22:48:37.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.051+0000 7f85baa00640 1 -- start start 2024-05-26T22:48:37.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.052+0000 7f85baa00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f85b40b6300 0x7f85b4152de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:37.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.052+0000 7f85baa00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f85b40b7310 0x7f85b4160d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:37.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.052+0000 7f85baa00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f85b415e380 0x7f85b40cb240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:37.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.052+0000 7f85baa00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f85b4162d40 con 0x7f85b415e380 2024-05-26T22:48:37.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.052+0000 7f85baa00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f85b40cf8f0 con 0x7f85b40b6300 2024-05-26T22:48:37.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.052+0000 7f85baa00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f85b40b3840 con 0x7f85b40b7310 2024-05-26T22:48:37.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.052+0000 7f85ba000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f85b415e380 0x7f85b40cb240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:37.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.052+0000 7f85ba000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f85b415e380 0x7f85b40cb240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58190/0 (socket says 172.21.15.131:58190) 2024-05-26T22:48:37.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.052+0000 7f85ba000640 1 -- 172.21.15.131:0/1678491044 learned_addr learned my addr 172.21.15.131:0/1678491044 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:37.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.053+0000 7f85b9600640 1 --2- 172.21.15.131:0/1678491044 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f85b40b6300 0x7f85b4152de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:37.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.053+0000 7f85b8c00640 1 --2- 172.21.15.131:0/1678491044 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f85b40b7310 0x7f85b4160d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:37.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.053+0000 7f85ba000640 1 -- 172.21.15.131:0/1678491044 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f85b40b7310 msgr2=0x7f85b4160d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:37.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.053+0000 7f85ba000640 1 --2- 172.21.15.131:0/1678491044 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f85b40b7310 0x7f85b4160d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:37.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.053+0000 7f85ba000640 1 -- 172.21.15.131:0/1678491044 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f85b40b6300 msgr2=0x7f85b4152de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:37.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.053+0000 7f85ba000640 1 --2- 172.21.15.131:0/1678491044 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f85b40b6300 0x7f85b4152de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:37.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.053+0000 7f85ba000640 1 -- 172.21.15.131:0/1678491044 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85b40c91b0 con 0x7f85b415e380 2024-05-26T22:48:37.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.053+0000 7f85ba000640 1 --2- 172.21.15.131:0/1678491044 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f85b415e380 0x7f85b40cb240 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f85ac00e850 tx=0x7f85ac00ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:37.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.053+0000 7f85b1600640 1 -- 172.21.15.131:0/1678491044 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85ac017070 con 0x7f85b415e380 2024-05-26T22:48:37.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.053+0000 7f85b1600640 1 -- 172.21.15.131:0/1678491044 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85ac00c940 con 0x7f85b415e380 2024-05-26T22:48:37.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.053+0000 7f85b1600640 1 -- 172.21.15.131:0/1678491044 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85ac00cd30 con 0x7f85b415e380 2024-05-26T22:48:37.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.053+0000 7f85baa00640 1 -- 172.21.15.131:0/1678491044 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85b40c9490 con 0x7f85b415e380 2024-05-26T22:48:37.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.054+0000 7f85baa00640 1 -- 172.21.15.131:0/1678491044 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85b40c9a50 con 0x7f85b415e380 2024-05-26T22:48:37.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.055+0000 7f85baa00640 1 -- 172.21.15.131:0/1678491044 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85b40694e0 con 0x7f85b415e380 2024-05-26T22:48:37.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.056+0000 7f85b1600640 1 -- 172.21.15.131:0/1678491044 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f85ac01a070 con 0x7f85b415e380 2024-05-26T22:48:37.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.056+0000 7f85b1600640 1 --2- 172.21.15.131:0/1678491044 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f859406be30 0x7f859406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:37.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.057+0000 7f85b1600640 1 -- 172.21.15.131:0/1678491044 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f85ac08b260 con 0x7f85b415e380 2024-05-26T22:48:37.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.057+0000 7f85b9600640 1 --2- 172.21.15.131:0/1678491044 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f859406be30 0x7f859406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:37.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.057+0000 7f85b9600640 1 --2- 172.21.15.131:0/1678491044 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f859406be30 0x7f859406e2f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f85a8033000 tx=0x7f85a8009000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:37.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.061+0000 7f85b1600640 1 -- 172.21.15.131:0/1678491044 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f85ac059b70 con 0x7f85b415e380 2024-05-26T22:48:37.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.303+0000 7f85baa00640 1 -- 172.21.15.131:0/1678491044 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f85b40b30c0 con 0x7f85b415e380 2024-05-26T22:48:37.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.305+0000 7f85b1600640 1 -- 172.21.15.131:0/1678491044 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5184 (secure 0 0 0) 0x7f85ac059470 con 0x7f85b415e380 2024-05-26T22:48:37.306 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:48:37.306 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/3266175650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6833","nonce":3266175650},{"type":"v1","addr":"172.21.15.187:6835","nonce":3266175650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/1862866305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1862866305},{"type":"v1","addr":"172.21.15.131:6835","nonce":1862866305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4269,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/618294324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":618294324},{"type":"v1","addr":"172.21.15.131:6837","nonce":618294324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-26T22:48:21.211047+0000","modified":"2024-05-26T22:48:24.550610+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6834/2547512415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":2547512415},{"type":"v1","addr":"172.21.15.187:6834","nonce":2547512415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4281,"qdb_cluster":[4281]},"id":1}]} 2024-05-26T22:48:37.307 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 7 2024-05-26T22:48:37.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.307+0000 7f85baa00640 1 -- 172.21.15.131:0/1678491044 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f859406be30 msgr2=0x7f859406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:37.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.307+0000 7f85baa00640 1 --2- 172.21.15.131:0/1678491044 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f859406be30 0x7f859406e2f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f85a8033000 tx=0x7f85a8009000 comp rx=0 tx=0).stop 2024-05-26T22:48:37.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.307+0000 7f85baa00640 1 -- 172.21.15.131:0/1678491044 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f85b415e380 msgr2=0x7f85b40cb240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:37.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.307+0000 7f85baa00640 1 --2- 172.21.15.131:0/1678491044 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f85b415e380 0x7f85b40cb240 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f85ac00e850 tx=0x7f85ac00ed10 comp rx=0 tx=0).stop 2024-05-26T22:48:37.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.307+0000 7f85baa00640 1 -- 172.21.15.131:0/1678491044 shutdown_connections 2024-05-26T22:48:37.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.307+0000 7f85baa00640 1 --2- 172.21.15.131:0/1678491044 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f85b40b7310 0x7f85b4160d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:37.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.307+0000 7f85baa00640 1 --2- 172.21.15.131:0/1678491044 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f859406be30 0x7f859406e2f0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:37.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.307+0000 7f85baa00640 1 --2- 172.21.15.131:0/1678491044 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f85b415e380 0x7f85b40cb240 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:37.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.307+0000 7f85baa00640 1 --2- 172.21.15.131:0/1678491044 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f85b40b6300 0x7f85b4152de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:37.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.307+0000 7f85baa00640 1 -- 172.21.15.131:0/1678491044 >> 172.21.15.131:0/1678491044 conn(0x7f85b4073070 msgr2=0x7f85b40b4eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:37.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.308+0000 7f85baa00640 1 -- 172.21.15.131:0/1678491044 shutdown_connections 2024-05-26T22:48:37.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.308+0000 7f85baa00640 1 -- 172.21.15.131:0/1678491044 wait complete. 2024-05-26T22:48:37.327 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-26T22:48:37.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.524+0000 7f2b4fe00640 1 Processor -- start 2024-05-26T22:48:37.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.524+0000 7f2b4fe00640 1 -- start start 2024-05-26T22:48:37.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.526+0000 7f2b4fe00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2b50156750 0x7f2b5015fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:37.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.526+0000 7f2b4fe00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f2b5006e8c0 0x7f2b5015d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:37.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.526+0000 7f2b4fe00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f2b500cc7f0 0x7f2b500ccc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:37.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.526+0000 7f2b4fe00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f2b500cd1a0 con 0x7f2b5006e8c0 2024-05-26T22:48:37.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.526+0000 7f2b4fe00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f2b5015cbd0 con 0x7f2b500cc7f0 2024-05-26T22:48:37.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.526+0000 7f2b4fe00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f2b50160620 con 0x7f2b50156750 2024-05-26T22:48:37.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.526+0000 7f2b4e000640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f2b5006e8c0 0x7f2b5015d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:37.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.526+0000 7f2b4e000640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f2b5006e8c0 0x7f2b5015d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58200/0 (socket says 172.21.15.131:58200) 2024-05-26T22:48:37.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.526+0000 7f2b4e000640 1 -- 172.21.15.131:0/1329350160 learned_addr learned my addr 172.21.15.131:0/1329350160 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:37.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.526+0000 7f2b4f400640 1 --2- 172.21.15.131:0/1329350160 >> v2:172.21.15.187:3300/0 conn(0x7f2b500cc7f0 0x7f2b500ccc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:37.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.526+0000 7f2b4ea00640 1 --2- 172.21.15.131:0/1329350160 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2b50156750 0x7f2b5015fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:37.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.526+0000 7f2b4e000640 1 -- 172.21.15.131:0/1329350160 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2b50156750 msgr2=0x7f2b5015fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:37.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.526+0000 7f2b4e000640 1 --2- 172.21.15.131:0/1329350160 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2b50156750 0x7f2b5015fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:37.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.526+0000 7f2b4e000640 1 -- 172.21.15.131:0/1329350160 >> v2:172.21.15.187:3300/0 conn(0x7f2b500cc7f0 msgr2=0x7f2b500ccc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:37.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.526+0000 7f2b4e000640 1 --2- 172.21.15.131:0/1329350160 >> v2:172.21.15.187:3300/0 conn(0x7f2b500cc7f0 0x7f2b500ccc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:37.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.526+0000 7f2b4e000640 1 -- 172.21.15.131:0/1329350160 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b50160900 con 0x7f2b5006e8c0 2024-05-26T22:48:37.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.526+0000 7f2b4f400640 1 --2- 172.21.15.131:0/1329350160 >> v2:172.21.15.187:3300/0 conn(0x7f2b500cc7f0 0x7f2b500ccc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:48:37.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.528+0000 7f2b4e000640 1 --2- 172.21.15.131:0/1329350160 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2b5006e8c0 0x7f2b5015d060 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2b5006f570 tx=0x7f2b3c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6018c625f4902227 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:37.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.528+0000 7f2b4d600640 1 -- 172.21.15.131:0/1329350160 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b3c037070 con 0x7f2b5006e8c0 2024-05-26T22:48:37.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.528+0000 7f2b4d600640 1 -- 172.21.15.131:0/1329350160 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b3c032440 con 0x7f2b5006e8c0 2024-05-26T22:48:37.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.528+0000 7f2b4d600640 1 -- 172.21.15.131:0/1329350160 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b3c032810 con 0x7f2b5006e8c0 2024-05-26T22:48:37.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.529+0000 7f2b4fe00640 1 -- 172.21.15.131:0/1329350160 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2b5006e8c0 msgr2=0x7f2b5015d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:37.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.529+0000 7f2b4fe00640 1 --2- 172.21.15.131:0/1329350160 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2b5006e8c0 0x7f2b5015d060 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2b5006f570 tx=0x7f2b3c02a1e0 comp rx=0 tx=0).stop 2024-05-26T22:48:37.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.529+0000 7f2b4fe00640 1 -- 172.21.15.131:0/1329350160 shutdown_connections 2024-05-26T22:48:37.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.529+0000 7f2b4fe00640 1 --2- 172.21.15.131:0/1329350160 >> v2:172.21.15.187:3300/0 conn(0x7f2b500cc7f0 0x7f2b500ccc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:37.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.529+0000 7f2b4fe00640 1 --2- 172.21.15.131:0/1329350160 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2b5006e8c0 0x7f2b5015d060 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:37.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.529+0000 7f2b4fe00640 1 --2- 172.21.15.131:0/1329350160 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2b50156750 0x7f2b5015fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:37.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.529+0000 7f2b4fe00640 1 -- 172.21.15.131:0/1329350160 >> 172.21.15.131:0/1329350160 conn(0x7f2b500cff40 msgr2=0x7f2b500d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:37.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.529+0000 7f2b4fe00640 1 -- 172.21.15.131:0/1329350160 shutdown_connections 2024-05-26T22:48:37.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.529+0000 7f2b4fe00640 1 -- 172.21.15.131:0/1329350160 wait complete. 2024-05-26T22:48:37.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.531+0000 7f2b4fe00640 1 Processor -- start 2024-05-26T22:48:37.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.531+0000 7f2b4fe00640 1 -- start start 2024-05-26T22:48:37.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.532+0000 7f2b4fe00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2b5006e8c0 0x7f2b500aecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:37.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.532+0000 7f2b4fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2b500cc7f0 0x7f2b500b0470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:37.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.532+0000 7f2b4fe00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2b50156750 0x7f2b501f6a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:37.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.532+0000 7f2b4fe00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f2b50156b30 con 0x7f2b500cc7f0 2024-05-26T22:48:37.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.532+0000 7f2b4fe00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2b5015c860 con 0x7f2b50156750 2024-05-26T22:48:37.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.532+0000 7f2b4fe00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f2b50160280 con 0x7f2b5006e8c0 2024-05-26T22:48:37.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.532+0000 7f2b4e000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2b500cc7f0 0x7f2b500b0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:37.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.532+0000 7f2b4e000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2b500cc7f0 0x7f2b500b0470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58202/0 (socket says 172.21.15.131:58202) 2024-05-26T22:48:37.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.532+0000 7f2b4e000640 1 -- 172.21.15.131:0/432103027 learned_addr learned my addr 172.21.15.131:0/432103027 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:37.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.532+0000 7f2b4ea00640 1 --2- 172.21.15.131:0/432103027 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2b5006e8c0 0x7f2b500aecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:37.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.532+0000 7f2b4f400640 1 --2- 172.21.15.131:0/432103027 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2b50156750 0x7f2b501f6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:37.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.532+0000 7f2b4e000640 1 -- 172.21.15.131:0/432103027 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2b5006e8c0 msgr2=0x7f2b500aecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:37.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.532+0000 7f2b4e000640 1 --2- 172.21.15.131:0/432103027 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2b5006e8c0 0x7f2b500aecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:37.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.532+0000 7f2b4e000640 1 -- 172.21.15.131:0/432103027 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2b50156750 msgr2=0x7f2b501f6a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:37.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.532+0000 7f2b4e000640 1 --2- 172.21.15.131:0/432103027 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2b50156750 0x7f2b501f6a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:37.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.532+0000 7f2b4e000640 1 -- 172.21.15.131:0/432103027 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b500adc50 con 0x7f2b500cc7f0 2024-05-26T22:48:37.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.533+0000 7f2b4e000640 1 --2- 172.21.15.131:0/432103027 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2b500cc7f0 0x7f2b500b0470 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c02a220 tx=0x7f2b3c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:37.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.533+0000 7f2b46a00640 1 -- 172.21.15.131:0/432103027 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b3c041020 con 0x7f2b500cc7f0 2024-05-26T22:48:37.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.533+0000 7f2b46a00640 1 -- 172.21.15.131:0/432103027 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b3c0306c0 con 0x7f2b500cc7f0 2024-05-26T22:48:37.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.533+0000 7f2b46a00640 1 -- 172.21.15.131:0/432103027 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b3c037040 con 0x7f2b500cc7f0 2024-05-26T22:48:37.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.533+0000 7f2b4fe00640 1 -- 172.21.15.131:0/432103027 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b500adf30 con 0x7f2b500cc7f0 2024-05-26T22:48:37.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.533+0000 7f2b4fe00640 1 -- 172.21.15.131:0/432103027 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b500ae4f0 con 0x7f2b500cc7f0 2024-05-26T22:48:37.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.534+0000 7f2b4fe00640 1 -- 172.21.15.131:0/432103027 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b500694e0 con 0x7f2b500cc7f0 2024-05-26T22:48:37.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.535+0000 7f2b46a00640 1 -- 172.21.15.131:0/432103027 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2b3c044020 con 0x7f2b500cc7f0 2024-05-26T22:48:37.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.536+0000 7f2b46a00640 1 --2- 172.21.15.131:0/432103027 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f2b3006be60 0x7f2b3006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:37.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.536+0000 7f2b46a00640 1 -- 172.21.15.131:0/432103027 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f2b3c0ab670 con 0x7f2b500cc7f0 2024-05-26T22:48:37.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.537+0000 7f2b4ea00640 1 --2- 172.21.15.131:0/432103027 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f2b3006be60 0x7f2b3006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:37.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.537+0000 7f2b4ea00640 1 --2- 172.21.15.131:0/432103027 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f2b3006be60 0x7f2b3006e320 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2b40006fd0 tx=0x7f2b40008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:37.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.541+0000 7f2b46a00640 1 -- 172.21.15.131:0/432103027 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f2b3c04b030 con 0x7f2b500cc7f0 2024-05-26T22:48:37.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.743+0000 7f2b4fe00640 1 -- 172.21.15.131:0/432103027 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2b500d0600 con 0x7f2b500cc7f0 2024-05-26T22:48:37.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.746+0000 7f2b46a00640 1 -- 172.21.15.131:0/432103027 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16629 (secure 0 0 0) 0x7f2b3c07a010 con 0x7f2b500cc7f0 2024-05-26T22:48:37.746 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:48:37.747 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":20,"fsid":"46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f","created":"2024-05-26T22:47:57.288905+0000","modified":"2024-05-26T22:48:22.799298+0000","last_up_change":"2024-05-26T22:48:09.602207+0000","last_in_change":"2024-05-26T22:48:00.687341+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-26T22:48:10.463339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-26T22:48:19.663254+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-26T22:48:20.235751+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"814eaf62-a18d-494b-8e27-fd8fb88adea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6801","nonce":2350940051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6803","nonce":2350940051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6807","nonce":2350940051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6805","nonce":2350940051}]},"public_addr":"172.21.15.131:6801/2350940051","cluster_addr":"172.21.15.131:6803/2350940051","heartbeat_back_addr":"172.21.15.131:6807/2350940051","heartbeat_front_addr":"172.21.15.131:6805/2350940051","state":["exists","up"]},{"osd":1,"uuid":"42e66287-f6c6-4e26-af3c-6dfaa3536d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6825","nonce":1525528247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6827","nonce":1525528247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6831","nonce":1525528247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6829","nonce":1525528247}]},"public_addr":"172.21.15.131:6825/1525528247","cluster_addr":"172.21.15.131:6827/1525528247","heartbeat_back_addr":"172.21.15.131:6831/1525528247","heartbeat_front_addr":"172.21.15.131:6829/1525528247","state":["exists","up"]},{"osd":2,"uuid":"0da34673-04a2-4fd0-b57a-0ff2b679b043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6817","nonce":3627842210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6819","nonce":3627842210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6823","nonce":3627842210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6821","nonce":3627842210}]},"public_addr":"172.21.15.131:6817/3627842210","cluster_addr":"172.21.15.131:6819/3627842210","heartbeat_back_addr":"172.21.15.131:6823/3627842210","heartbeat_front_addr":"172.21.15.131:6821/3627842210","state":["exists","up"]},{"osd":3,"uuid":"cf117a27-d6b2-43d3-ae52-b47988bf8eed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6809","nonce":350790591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6811","nonce":350790591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6815","nonce":350790591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6813","nonce":350790591}]},"public_addr":"172.21.15.131:6809/350790591","cluster_addr":"172.21.15.131:6811/350790591","heartbeat_back_addr":"172.21.15.131:6815/350790591","heartbeat_front_addr":"172.21.15.131:6813/350790591","state":["exists","up"]},{"osd":4,"uuid":"a092ccfc-be89-4a84-88d5-0f781fa715eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6816","nonce":244560995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6819","nonce":244560995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6823","nonce":244560995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6821","nonce":244560995}]},"public_addr":"172.21.15.187:6816/244560995","cluster_addr":"172.21.15.187:6819/244560995","heartbeat_back_addr":"172.21.15.187:6823/244560995","heartbeat_front_addr":"172.21.15.187:6821/244560995","state":["exists","up"]},{"osd":5,"uuid":"7513ba69-dd00-4a35-b190-9e8cc3b2bdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6801","nonce":324619398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6803","nonce":324619398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6812","nonce":324619398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6807","nonce":324619398}]},"public_addr":"172.21.15.187:6801/324619398","cluster_addr":"172.21.15.187:6803/324619398","heartbeat_back_addr":"172.21.15.187:6812/324619398","heartbeat_front_addr":"172.21.15.187:6807/324619398","state":["exists","up"]},{"osd":6,"uuid":"50081810-c093-4d2f-a4d7-b7c695c5400d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6806","nonce":1391347786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6809","nonce":1391347786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6818","nonce":1391347786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6813","nonce":1391347786}]},"public_addr":"172.21.15.187:6806/1391347786","cluster_addr":"172.21.15.187:6809/1391347786","heartbeat_back_addr":"172.21.15.187:6818/1391347786","heartbeat_front_addr":"172.21.15.187:6813/1391347786","state":["exists","up"]},{"osd":7,"uuid":"775dae05-54f5-4ad4-bbce-92863c6ed2ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6825","nonce":1220065622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6827","nonce":1220065622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6831","nonce":1220065622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6829","nonce":1220065622}]},"public_addr":"172.21.15.187:6825/1220065622","cluster_addr":"172.21.15.187:6827/1220065622","heartbeat_back_addr":"172.21.15.187:6831/1220065622","heartbeat_front_addr":"172.21.15.187:6829/1220065622","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:05.983520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.299516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.067323+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.307748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.018742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.258775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.010912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.295355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-26T22:48:37.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.749+0000 7f2b4fe00640 1 -- 172.21.15.131:0/432103027 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f2b3006be60 msgr2=0x7f2b3006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:37.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.749+0000 7f2b4fe00640 1 --2- 172.21.15.131:0/432103027 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f2b3006be60 0x7f2b3006e320 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2b40006fd0 tx=0x7f2b40008040 comp rx=0 tx=0).stop 2024-05-26T22:48:37.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.749+0000 7f2b4fe00640 1 -- 172.21.15.131:0/432103027 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2b500cc7f0 msgr2=0x7f2b500b0470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:37.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.749+0000 7f2b4fe00640 1 --2- 172.21.15.131:0/432103027 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2b500cc7f0 0x7f2b500b0470 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c02a220 tx=0x7f2b3c0046c0 comp rx=0 tx=0).stop 2024-05-26T22:48:37.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.749+0000 7f2b4fe00640 1 -- 172.21.15.131:0/432103027 shutdown_connections 2024-05-26T22:48:37.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.749+0000 7f2b4fe00640 1 --2- 172.21.15.131:0/432103027 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f2b3006be60 0x7f2b3006e320 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:37.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.749+0000 7f2b4fe00640 1 --2- 172.21.15.131:0/432103027 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2b50156750 0x7f2b501f6a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:37.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.749+0000 7f2b4fe00640 1 --2- 172.21.15.131:0/432103027 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2b500cc7f0 0x7f2b500b0470 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:37.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.749+0000 7f2b4fe00640 1 --2- 172.21.15.131:0/432103027 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2b5006e8c0 0x7f2b500aecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:37.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.749+0000 7f2b4fe00640 1 -- 172.21.15.131:0/432103027 >> 172.21.15.131:0/432103027 conn(0x7f2b500cff40 msgr2=0x7f2b500b5a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:37.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.750+0000 7f2b4fe00640 1 -- 172.21.15.131:0/432103027 shutdown_connections 2024-05-26T22:48:37.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.750+0000 7f2b4fe00640 1 -- 172.21.15.131:0/432103027 wait complete. 2024-05-26T22:48:37.767 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-26T22:48:37.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.970+0000 7fa140000640 1 Processor -- start 2024-05-26T22:48:37.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.970+0000 7fa140000640 1 -- start start 2024-05-26T22:48:37.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.972+0000 7fa140000640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fa13806ee20 0x7fa1380b3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:37.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.972+0000 7fa140000640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fa1380ad250 0x7fa1380af790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:37.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.972+0000 7fa140000640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa13815cb20 con 0x7fa13806f200 2024-05-26T22:48:37.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.972+0000 7fa140000640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fa13815ce00 con 0x7fa1380ad250 2024-05-26T22:48:37.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.972+0000 7fa140000640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fa13815d0e0 con 0x7fa13806ee20 2024-05-26T22:48:37.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.972+0000 7fa13cc00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fa1380ad250 0x7fa1380af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:37.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.972+0000 7fa13cc00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fa1380ad250 0x7fa1380af790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58206/0 (socket says 172.21.15.131:58206) 2024-05-26T22:48:37.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.972+0000 7fa13cc00640 1 -- 172.21.15.131:0/1284142229 learned_addr learned my addr 172.21.15.131:0/1284142229 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:37.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.972+0000 7fa13d600640 1 --2- 172.21.15.131:0/1284142229 >> v2:172.21.15.187:3300/0 conn(0x7fa13806ee20 0x7fa1380b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:37.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.972+0000 7fa13cc00640 1 -- 172.21.15.131:0/1284142229 >> v2:172.21.15.187:3300/0 conn(0x7fa13806ee20 msgr2=0x7fa1380b3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:37.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.972+0000 7fa13cc00640 1 --2- 172.21.15.131:0/1284142229 >> v2:172.21.15.187:3300/0 conn(0x7fa13806ee20 0x7fa1380b3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:37.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.973+0000 7fa13cc00640 1 -- 172.21.15.131:0/1284142229 >> v1:172.21.15.187:6789/0 conn(0x7fa13806f200 legacy=0x7fa1381562c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:37.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.973+0000 7fa13cc00640 1 -- 172.21.15.131:0/1284142229 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa13815d3c0 con 0x7fa1380ad250 2024-05-26T22:48:37.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.974+0000 7fa13cc00640 1 --2- 172.21.15.131:0/1284142229 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa1380ad250 0x7fa1380af790 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa138071090 tx=0x7fa12402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=474bf533fa6fccc3 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:37.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.974+0000 7fa137e00640 1 -- 172.21.15.131:0/1284142229 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa124037070 con 0x7fa1380ad250 2024-05-26T22:48:37.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.974+0000 7fa137e00640 1 -- 172.21.15.131:0/1284142229 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa124032440 con 0x7fa1380ad250 2024-05-26T22:48:37.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.974+0000 7fa137e00640 1 -- 172.21.15.131:0/1284142229 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa124032810 con 0x7fa1380ad250 2024-05-26T22:48:37.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.974+0000 7fa140000640 1 -- 172.21.15.131:0/1284142229 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa1380ad250 msgr2=0x7fa1380af790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:37.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.974+0000 7fa140000640 1 --2- 172.21.15.131:0/1284142229 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa1380ad250 0x7fa1380af790 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa138071090 tx=0x7fa12402a1a0 comp rx=0 tx=0).stop 2024-05-26T22:48:37.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.975+0000 7fa140000640 1 -- 172.21.15.131:0/1284142229 shutdown_connections 2024-05-26T22:48:37.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.975+0000 7fa140000640 1 --2- 172.21.15.131:0/1284142229 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa1380ad250 0x7fa1380af790 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:37.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.975+0000 7fa140000640 1 --2- 172.21.15.131:0/1284142229 >> v2:172.21.15.187:3300/0 conn(0x7fa13806ee20 0x7fa1380b3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:37.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.975+0000 7fa140000640 1 -- 172.21.15.131:0/1284142229 >> 172.21.15.131:0/1284142229 conn(0x7fa1380b1e90 msgr2=0x7fa1380b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:37.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.975+0000 7fa140000640 1 -- 172.21.15.131:0/1284142229 shutdown_connections 2024-05-26T22:48:37.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.975+0000 7fa140000640 1 -- 172.21.15.131:0/1284142229 wait complete. 2024-05-26T22:48:37.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.976+0000 7fa140000640 1 Processor -- start 2024-05-26T22:48:37.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.977+0000 7fa140000640 1 -- start start 2024-05-26T22:48:37.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.977+0000 7fa140000640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa13806ee20 0x7fa13809c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:37.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.978+0000 7fa140000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa13806f200 0x7fa13815b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:37.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.978+0000 7fa140000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa1380ad250 0x7fa1381e6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:37.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.978+0000 7fa140000640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa13815c900 con 0x7fa1380ad250 2024-05-26T22:48:37.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.978+0000 7fa140000640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa13815c780 con 0x7fa13806ee20 2024-05-26T22:48:37.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.978+0000 7fa140000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa1381569d0 con 0x7fa13806f200 2024-05-26T22:48:37.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.978+0000 7fa13e000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa1380ad250 0x7fa1381e6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:37.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.978+0000 7fa13e000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa1380ad250 0x7fa1381e6ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58208/0 (socket says 172.21.15.131:58208) 2024-05-26T22:48:37.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.978+0000 7fa13e000640 1 -- 172.21.15.131:0/282909328 learned_addr learned my addr 172.21.15.131:0/282909328 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:37.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.978+0000 7fa13d600640 1 --2- 172.21.15.131:0/282909328 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa13806ee20 0x7fa13809c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:37.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.978+0000 7fa13cc00640 1 --2- 172.21.15.131:0/282909328 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa13806f200 0x7fa13815b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:37.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.978+0000 7fa13e000640 1 -- 172.21.15.131:0/282909328 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa13806f200 msgr2=0x7fa13815b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:37.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.978+0000 7fa13e000640 1 --2- 172.21.15.131:0/282909328 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa13806f200 0x7fa13815b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:37.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.978+0000 7fa13e000640 1 -- 172.21.15.131:0/282909328 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa13806ee20 msgr2=0x7fa13809c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:37.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.978+0000 7fa13e000640 1 --2- 172.21.15.131:0/282909328 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa13806ee20 0x7fa13809c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:37.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.978+0000 7fa13e000640 1 -- 172.21.15.131:0/282909328 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1381e78d0 con 0x7fa1380ad250 2024-05-26T22:48:37.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.978+0000 7fa13d600640 1 --2- 172.21.15.131:0/282909328 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa13806ee20 0x7fa13809c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:48:37.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.978+0000 7fa13e000640 1 --2- 172.21.15.131:0/282909328 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa1380ad250 0x7fa1381e6ef0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa13000c7b0 tx=0x7fa13000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:37.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.979+0000 7fa135600640 1 -- 172.21.15.131:0/282909328 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa130017070 con 0x7fa1380ad250 2024-05-26T22:48:37.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.979+0000 7fa135600640 1 -- 172.21.15.131:0/282909328 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa130003da0 con 0x7fa1380ad250 2024-05-26T22:48:37.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.979+0000 7fa135600640 1 -- 172.21.15.131:0/282909328 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa13000ce60 con 0x7fa1380ad250 2024-05-26T22:48:37.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.979+0000 7fa140000640 1 -- 172.21.15.131:0/282909328 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1381e7bb0 con 0x7fa1380ad250 2024-05-26T22:48:37.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.979+0000 7fa140000640 1 -- 172.21.15.131:0/282909328 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa138154ae0 con 0x7fa1380ad250 2024-05-26T22:48:37.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.980+0000 7fa140000640 1 -- 172.21.15.131:0/282909328 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1380694e0 con 0x7fa1380ad250 2024-05-26T22:48:37.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.981+0000 7fa135600640 1 -- 172.21.15.131:0/282909328 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa13001a070 con 0x7fa1380ad250 2024-05-26T22:48:37.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.982+0000 7fa135600640 1 --2- 172.21.15.131:0/282909328 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa11806be30 0x7fa11806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:37.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.982+0000 7fa135600640 1 -- 172.21.15.131:0/282909328 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa13008a510 con 0x7fa1380ad250 2024-05-26T22:48:37.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.982+0000 7fa13d600640 1 --2- 172.21.15.131:0/282909328 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa11806be30 0x7fa11806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:37.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.983+0000 7fa13d600640 1 --2- 172.21.15.131:0/282909328 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa11806be30 0x7fa11806e2f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa12c006fd0 tx=0x7fa12c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:37.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:37.986+0000 7fa135600640 1 -- 172.21.15.131:0/282909328 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fa130058eb0 con 0x7fa1380ad250 2024-05-26T22:48:38.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.227+0000 7fa140000640 1 -- 172.21.15.131:0/282909328 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa13806c590 con 0x7fa1380ad250 2024-05-26T22:48:38.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.229+0000 7fa135600640 1 -- 172.21.15.131:0/282909328 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5184 (secure 0 0 0) 0x7fa13005f080 con 0x7fa1380ad250 2024-05-26T22:48:38.229 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:48:38.229 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/3266175650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6833","nonce":3266175650},{"type":"v1","addr":"172.21.15.187:6835","nonce":3266175650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/1862866305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1862866305},{"type":"v1","addr":"172.21.15.131:6835","nonce":1862866305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4269,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/618294324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":618294324},{"type":"v1","addr":"172.21.15.131:6837","nonce":618294324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-26T22:48:21.211047+0000","modified":"2024-05-26T22:48:24.550610+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6834/2547512415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":2547512415},{"type":"v1","addr":"172.21.15.187:6834","nonce":2547512415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4281,"qdb_cluster":[4281]},"id":1}]} 2024-05-26T22:48:38.229 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 7 2024-05-26T22:48:38.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.231+0000 7fa140000640 1 -- 172.21.15.131:0/282909328 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa11806be30 msgr2=0x7fa11806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:38.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.231+0000 7fa140000640 1 --2- 172.21.15.131:0/282909328 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa11806be30 0x7fa11806e2f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa12c006fd0 tx=0x7fa12c008040 comp rx=0 tx=0).stop 2024-05-26T22:48:38.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.232+0000 7fa140000640 1 -- 172.21.15.131:0/282909328 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa1380ad250 msgr2=0x7fa1381e6ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:38.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.232+0000 7fa140000640 1 --2- 172.21.15.131:0/282909328 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa1380ad250 0x7fa1381e6ef0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa13000c7b0 tx=0x7fa13000cc70 comp rx=0 tx=0).stop 2024-05-26T22:48:38.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.232+0000 7fa140000640 1 -- 172.21.15.131:0/282909328 shutdown_connections 2024-05-26T22:48:38.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.232+0000 7fa140000640 1 --2- 172.21.15.131:0/282909328 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa13806f200 0x7fa13815b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:38.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.232+0000 7fa140000640 1 --2- 172.21.15.131:0/282909328 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa11806be30 0x7fa11806e2f0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:38.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.232+0000 7fa140000640 1 --2- 172.21.15.131:0/282909328 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa1380ad250 0x7fa1381e6ef0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:38.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.232+0000 7fa140000640 1 --2- 172.21.15.131:0/282909328 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa13806ee20 0x7fa13809c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:38.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.232+0000 7fa140000640 1 -- 172.21.15.131:0/282909328 >> 172.21.15.131:0/282909328 conn(0x7fa1380b1e90 msgr2=0x7fa13815bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:38.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.232+0000 7fa140000640 1 -- 172.21.15.131:0/282909328 shutdown_connections 2024-05-26T22:48:38.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.232+0000 7fa140000640 1 -- 172.21.15.131:0/282909328 wait complete. 2024-05-26T22:48:38.654 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-26T22:48:21.211047+0000', 'modified': '2024-05-26T22:48:24.550610+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4281}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4281': {'gid': 4281, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.187:6834/2547512415', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.187:6832', 'nonce': 2547512415}, {'type': 'v1', 'addr': '172.21.15.187:6834', 'nonce': 2547512415}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4281, 'qdb_cluster': [4281]} 2024-05-26T22:48:38.654 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-26T22:48:38.654 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-05-26T22:48:38.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.864+0000 7fb7ef600640 1 Processor -- start 2024-05-26T22:48:38.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.864+0000 7fb7ef600640 1 -- start start 2024-05-26T22:48:38.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.866+0000 7fb7ef600640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb7e806dd30 0x7fb7e806e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:38.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.866+0000 7fb7ef600640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fb7e80725a0 0x7fb7e80729d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:38.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.866+0000 7fb7ef600640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7e815c1c0 con 0x7fb7e8155770 2024-05-26T22:48:38.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.866+0000 7fb7ef600640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fb7e806e6d0 con 0x7fb7e80725a0 2024-05-26T22:48:38.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.866+0000 7fb7ef600640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb7e815c4a0 con 0x7fb7e806dd30 2024-05-26T22:48:38.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.866+0000 7fb7e7e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb7e806dd30 0x7fb7e806e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:38.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.866+0000 7fb7ed600640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fb7e80725a0 0x7fb7e80729d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:38.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.866+0000 7fb7ed600640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fb7e80725a0 0x7fb7e80729d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.131:51398/0 (socket says 172.21.15.131:51398) 2024-05-26T22:48:38.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.866+0000 7fb7ed600640 1 -- 172.21.15.131:0/3791228627 learned_addr learned my addr 172.21.15.131:0/3791228627 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:38.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.866+0000 7fb7e7e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb7e806dd30 0x7fb7e806e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.131:58280/0 (socket says 172.21.15.131:58280) 2024-05-26T22:48:38.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.867+0000 7fb7e7e00640 1 -- 172.21.15.131:0/3791228627 >> v2:172.21.15.187:3300/0 conn(0x7fb7e80725a0 msgr2=0x7fb7e80729d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:38.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.867+0000 7fb7e7e00640 1 --2- 172.21.15.131:0/3791228627 >> v2:172.21.15.187:3300/0 conn(0x7fb7e80725a0 0x7fb7e80729d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:38.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.867+0000 7fb7e7e00640 1 -- 172.21.15.131:0/3791228627 >> v1:172.21.15.187:6789/0 conn(0x7fb7e8155770 legacy=0x7fb7e815b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:38.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.867+0000 7fb7e7e00640 1 -- 172.21.15.131:0/3791228627 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7e8071da0 con 0x7fb7e806dd30 2024-05-26T22:48:38.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.868+0000 7fb7e7e00640 1 --2- 172.21.15.131:0/3791228627 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb7e806dd30 0x7fb7e806e190 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb7d8009d90 tx=0x7fb7d802a6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7765deafa4683198 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:38.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.869+0000 7fb7e7400640 1 -- 172.21.15.131:0/3791228627 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7d8007740 con 0x7fb7e806dd30 2024-05-26T22:48:38.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.869+0000 7fb7e7400640 1 -- 172.21.15.131:0/3791228627 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7d80078e0 con 0x7fb7e806dd30 2024-05-26T22:48:38.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.869+0000 7fb7e7400640 1 -- 172.21.15.131:0/3791228627 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7d8007cb0 con 0x7fb7e806dd30 2024-05-26T22:48:38.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.869+0000 7fb7ef600640 1 -- 172.21.15.131:0/3791228627 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb7e806dd30 msgr2=0x7fb7e806e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:38.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.869+0000 7fb7ef600640 1 --2- 172.21.15.131:0/3791228627 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb7e806dd30 0x7fb7e806e190 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb7d8009d90 tx=0x7fb7d802a6f0 comp rx=0 tx=0).stop 2024-05-26T22:48:38.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.870+0000 7fb7ef600640 1 -- 172.21.15.131:0/3791228627 shutdown_connections 2024-05-26T22:48:38.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.870+0000 7fb7ef600640 1 --2- 172.21.15.131:0/3791228627 >> v2:172.21.15.187:3300/0 conn(0x7fb7e80725a0 0x7fb7e80729d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:38.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.870+0000 7fb7ef600640 1 --2- 172.21.15.131:0/3791228627 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb7e806dd30 0x7fb7e806e190 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:38.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.870+0000 7fb7ef600640 1 -- 172.21.15.131:0/3791228627 >> 172.21.15.131:0/3791228627 conn(0x7fb7e806bc90 msgr2=0x7fb7e80b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:38.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.870+0000 7fb7ef600640 1 -- 172.21.15.131:0/3791228627 shutdown_connections 2024-05-26T22:48:38.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.870+0000 7fb7ef600640 1 -- 172.21.15.131:0/3791228627 wait complete. 2024-05-26T22:48:38.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.871+0000 7fb7ef600640 1 Processor -- start 2024-05-26T22:48:38.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.871+0000 7fb7ef600640 1 -- start start 2024-05-26T22:48:38.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.872+0000 7fb7ef600640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb7e806dd30 0x7fb7e80ada90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:38.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.872+0000 7fb7ef600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb7e80725a0 0x7fb7e81539f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:38.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.872+0000 7fb7ef600640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb7e8155770 0x7fb7e809c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:38.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.872+0000 7fb7ef600640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fb7e815c8c0 con 0x7fb7e80725a0 2024-05-26T22:48:38.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.872+0000 7fb7ef600640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb7e8155b50 con 0x7fb7e8155770 2024-05-26T22:48:38.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.872+0000 7fb7ef600640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb7e815be10 con 0x7fb7e806dd30 2024-05-26T22:48:38.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.873+0000 7fb7e7e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb7e80725a0 0x7fb7e81539f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:38.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.873+0000 7fb7e7e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb7e80725a0 0x7fb7e81539f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58216/0 (socket says 172.21.15.131:58216) 2024-05-26T22:48:38.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.873+0000 7fb7e7e00640 1 -- 172.21.15.131:0/422940806 learned_addr learned my addr 172.21.15.131:0/422940806 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:38.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.873+0000 7fb7ecc00640 1 --2- 172.21.15.131:0/422940806 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb7e806dd30 0x7fb7e80ada90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:38.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.873+0000 7fb7ed600640 1 --2- 172.21.15.131:0/422940806 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb7e8155770 0x7fb7e809c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:38.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.873+0000 7fb7e7e00640 1 -- 172.21.15.131:0/422940806 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb7e806dd30 msgr2=0x7fb7e80ada90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:38.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.873+0000 7fb7e7e00640 1 --2- 172.21.15.131:0/422940806 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb7e806dd30 0x7fb7e80ada90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:38.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.873+0000 7fb7e7e00640 1 -- 172.21.15.131:0/422940806 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb7e8155770 msgr2=0x7fb7e809c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:38.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.873+0000 7fb7e7e00640 1 --2- 172.21.15.131:0/422940806 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb7e8155770 0x7fb7e809c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:38.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.873+0000 7fb7e7e00640 1 -- 172.21.15.131:0/422940806 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7e809cb50 con 0x7fb7e80725a0 2024-05-26T22:48:38.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.873+0000 7fb7ecc00640 1 --2- 172.21.15.131:0/422940806 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb7e806dd30 0x7fb7e80ada90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-26T22:48:38.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.874+0000 7fb7e7e00640 1 --2- 172.21.15.131:0/422940806 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb7e80725a0 0x7fb7e81539f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb7d8009d90 tx=0x7fb7d8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:38.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.874+0000 7fb7e4c00640 1 -- 172.21.15.131:0/422940806 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7d803f070 con 0x7fb7e80725a0 2024-05-26T22:48:38.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.874+0000 7fb7e4c00640 1 -- 172.21.15.131:0/422940806 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7d8031c40 con 0x7fb7e80725a0 2024-05-26T22:48:38.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.874+0000 7fb7ef600640 1 -- 172.21.15.131:0/422940806 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7e809ce30 con 0x7fb7e80725a0 2024-05-26T22:48:38.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.874+0000 7fb7e4c00640 1 -- 172.21.15.131:0/422940806 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7d8009450 con 0x7fb7e80725a0 2024-05-26T22:48:38.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.874+0000 7fb7ef600640 1 -- 172.21.15.131:0/422940806 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7e809c840 con 0x7fb7e80725a0 2024-05-26T22:48:38.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.876+0000 7fb7e4c00640 1 -- 172.21.15.131:0/422940806 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb7d8044460 con 0x7fb7e80725a0 2024-05-26T22:48:38.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.877+0000 7fb7e4c00640 1 --2- 172.21.15.131:0/422940806 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb7c406be30 0x7fb7c406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:38.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.877+0000 7fb7e4c00640 1 -- 172.21.15.131:0/422940806 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb7d80ac6e0 con 0x7fb7e80725a0 2024-05-26T22:48:38.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.878+0000 7fb7ecc00640 1 --2- 172.21.15.131:0/422940806 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb7c406be30 0x7fb7c406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:38.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.878+0000 7fb7f2408740 1 Processor -- start 2024-05-26T22:48:38.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.878+0000 7fb7ecc00640 1 --2- 172.21.15.131:0/422940806 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb7c406be30 0x7fb7c406e2f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb7d0001700 tx=0x7fb7d0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:38.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.878+0000 7fb7f2408740 1 -- start start 2024-05-26T22:48:38.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.879+0000 7fb7f2408740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bb5bc3a300 0x55bb5bc3a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:38.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.879+0000 7fb7f2408740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55bb5bc378f0 0x55bb5bc0a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:38.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.879+0000 7fb7f2408740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bb5bc11860 0x55bb5bc0e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:38.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.879+0000 7fb7f2408740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55bb5bb72610 con 0x55bb5bc3a300 2024-05-26T22:48:38.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.879+0000 7fb7f2408740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55bb5bb1a5c0 con 0x55bb5bc11860 2024-05-26T22:48:38.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.879+0000 7fb7f2408740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x55bb5bb6e5d0 con 0x55bb5bc378f0 2024-05-26T22:48:38.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.879+0000 7fb7ecc00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bb5bc3a300 0x55bb5bc3a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:38.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.879+0000 7fb7ecc00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bb5bc3a300 0x55bb5bc3a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58232/0 (socket says 172.21.15.131:58232) 2024-05-26T22:48:38.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.879+0000 7fb7ecc00640 1 -- 172.21.15.131:0/2124526463 learned_addr learned my addr 172.21.15.131:0/2124526463 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:38.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.880+0000 7fb7ecc00640 1 -- 172.21.15.131:0/2124526463 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55bb5bc378f0 msgr2=0x55bb5bc0a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:38.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.880+0000 7fb7ecc00640 1 --2- 172.21.15.131:0/2124526463 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55bb5bc378f0 0x55bb5bc0a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:38.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.880+0000 7fb7ecc00640 1 -- 172.21.15.131:0/2124526463 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bb5bc11860 msgr2=0x55bb5bc0e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:38.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.880+0000 7fb7ecc00640 1 --2- 172.21.15.131:0/2124526463 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bb5bc11860 0x55bb5bc0e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:38.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.880+0000 7fb7ecc00640 1 -- 172.21.15.131:0/2124526463 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb5b8862d0 con 0x55bb5bc3a300 2024-05-26T22:48:38.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.880+0000 7fb7ecc00640 1 --2- 172.21.15.131:0/2124526463 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bb5bc3a300 0x55bb5bc3a6e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb7d0005150 tx=0x7fb7d0011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e71e2e11404fc31e server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:38.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.880+0000 7fb7dd200640 1 -- 172.21.15.131:0/2124526463 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7d00035f0 con 0x55bb5bc3a300 2024-05-26T22:48:38.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.880+0000 7fb7dd200640 1 -- 172.21.15.131:0/2124526463 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7d0003790 con 0x55bb5bc3a300 2024-05-26T22:48:38.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.880+0000 7fb7dd200640 1 -- 172.21.15.131:0/2124526463 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7d0003b60 con 0x55bb5bc3a300 2024-05-26T22:48:38.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.881+0000 7fb7f2408740 1 -- 172.21.15.131:0/2124526463 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bb5bc3a300 msgr2=0x55bb5bc3a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:38.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.881+0000 7fb7f2408740 1 --2- 172.21.15.131:0/2124526463 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bb5bc3a300 0x55bb5bc3a6e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb7d0005150 tx=0x7fb7d0011000 comp rx=0 tx=0).stop 2024-05-26T22:48:38.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.881+0000 7fb7f2408740 1 -- 172.21.15.131:0/2124526463 shutdown_connections 2024-05-26T22:48:38.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.881+0000 7fb7f2408740 1 --2- 172.21.15.131:0/2124526463 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55bb5bc378f0 0x55bb5bc0a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:38.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.881+0000 7fb7f2408740 1 --2- 172.21.15.131:0/2124526463 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bb5bc11860 0x55bb5bc0e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:38.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.881+0000 7fb7f2408740 1 --2- 172.21.15.131:0/2124526463 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bb5bc3a300 0x55bb5bc3a6e0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:38.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.881+0000 7fb7f2408740 1 -- 172.21.15.131:0/2124526463 >> 172.21.15.131:0/2124526463 conn(0x55bb5bc0cb00 msgr2=0x55bb5bc55110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:38.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.881+0000 7fb7f2408740 1 -- 172.21.15.131:0/2124526463 shutdown_connections 2024-05-26T22:48:38.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.881+0000 7fb7f2408740 1 -- 172.21.15.131:0/2124526463 wait complete. 2024-05-26T22:48:38.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.882+0000 7fb7f2408740 1 -- start start 2024-05-26T22:48:38.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.882+0000 7fb7f2408740 1 Processor -- start 2024-05-26T22:48:38.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.883+0000 7fb7f2408740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bb5bc378f0 0x55bb5bc49730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:38.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.883+0000 7fb7f2408740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bb5bc3a300 0x55bb5bc3df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:38.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.883+0000 7fb7f2408740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55bb5bc48590 0x55bb5bc3b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:38.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.883+0000 7fb7f2408740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55bb5bc37610 con 0x55bb5bc378f0 2024-05-26T22:48:38.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.883+0000 7fb7f2408740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55bb5bb2d910 con 0x55bb5bc3a300 2024-05-26T22:48:38.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.883+0000 7fb7f2408740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x55bb5bc58370 con 0x55bb5bc48590 2024-05-26T22:48:38.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.883+0000 7fb7f2408740 10 client.0 _ll_register_callbacks cb 0x55bb5bb69410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-26T22:48:38.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.883+0000 7fb7ecc00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bb5bc378f0 0x55bb5bc49730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:38.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.883+0000 7fb7ecc00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bb5bc378f0 0x55bb5bc49730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58238/0 (socket says 172.21.15.131:58238) 2024-05-26T22:48:38.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.883+0000 7fb7ecc00640 1 -- 172.21.15.131:0/543119040 learned_addr learned my addr 172.21.15.131:0/543119040 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:38.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.883+0000 7fb7ecc00640 1 -- 172.21.15.131:0/543119040 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55bb5bc48590 msgr2=0x55bb5bc3b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:38.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.883+0000 7fb7ecc00640 1 --2- 172.21.15.131:0/543119040 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55bb5bc48590 0x55bb5bc3b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:38.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.883+0000 7fb7ecc00640 1 -- 172.21.15.131:0/543119040 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bb5bc3a300 msgr2=0x55bb5bc3df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:38.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.883+0000 7fb7ecc00640 1 --2- 172.21.15.131:0/543119040 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bb5bc3a300 0x55bb5bc3df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:38.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.883+0000 7fb7ecc00640 1 -- 172.21.15.131:0/543119040 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb5b9506b0 con 0x55bb5bc378f0 2024-05-26T22:48:38.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.884+0000 7fb7ecc00640 1 --2- 172.21.15.131:0/543119040 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bb5bc378f0 0x55bb5bc49730 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb7d000ebb0 tx=0x7fb7d0013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0bc6c0741c27745 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:38.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.884+0000 7fb7d5600640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-26T22:48:38.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.884+0000 7fb7d5600640 1 -- 172.21.15.131:0/543119040 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7d0021650 con 0x55bb5bc378f0 2024-05-26T22:48:38.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.884+0000 7fb7d5600640 1 -- 172.21.15.131:0/543119040 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb7d002caf0 con 0x55bb5bc378f0 2024-05-26T22:48:38.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.884+0000 7fb7f2408740 1 -- 172.21.15.131:0/543119040 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7d0003790 con 0x55bb5bc378f0 2024-05-26T22:48:38.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.884+0000 7fb7d5600640 1 -- 172.21.15.131:0/543119040 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7fb7d002ccf0 con 0x55bb5bc378f0 2024-05-26T22:48:38.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.884+0000 7fb7f2408740 10 client.4650 fetch_fsmap learned FSMap version 7 2024-05-26T22:48:38.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.884+0000 7fb7f2408740 1 -- 172.21.15.131:0/543119040 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x55bb5bc47d90 con 0x55bb5bc378f0 2024-05-26T22:48:38.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.885+0000 7fb7d5600640 1 -- 172.21.15.131:0/543119040 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7d0003790 con 0x55bb5bc378f0 2024-05-26T22:48:38.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.885+0000 7fb7d5600640 1 -- 172.21.15.131:0/543119040 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 7) ==== 3494+0+0 (secure 0 0 0) 0x7fb7d0029070 con 0x55bb5bc378f0 2024-05-26T22:48:38.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.885+0000 7fb7f2408740 10 client.4650 fetch_fsmap finished waiting for FSMap version 7 2024-05-26T22:48:38.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.885+0000 7fb7f2408740 10 client.4650 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-26T22:48:38.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.885+0000 7fb7f2408740 20 client.4650 populate_metadata read hostname 'smithi131' 2024-05-26T22:48:38.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.885+0000 7fb7f2408740 1 --2- 172.21.15.131:0/543119040 >> [v2:172.21.15.187:6832/2547512415,v1:172.21.15.187:6834/2547512415] conn(0x55bb5bc715f0 0x55bb5bc719d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:38.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.885+0000 7fb7f2408740 4 client.4650 mds_command: new command op to 4281 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-26T22:48:38.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.885+0000 7fb7f2408740 1 -- 172.21.15.131:0/543119040 --> [v2:172.21.15.187:6832/2547512415,v1:172.21.15.187:6834/2547512415] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bb5bb7fd40 con 0x55bb5bc715f0 2024-05-26T22:48:38.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.885+0000 7fb7e7e00640 1 --2- 172.21.15.131:0/543119040 >> [v2:172.21.15.187:6832/2547512415,v1:172.21.15.187:6834/2547512415] conn(0x55bb5bc715f0 0x55bb5bc719d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:38.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.886+0000 7fb7e7e00640 1 --2- 172.21.15.131:0/543119040 >> [v2:172.21.15.187:6832/2547512415,v1:172.21.15.187:6834/2547512415] conn(0x55bb5bc715f0 0x55bb5bc719d0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49a1d2cb97a66373 server_cookie=76c41d6dafc18dde in_seq=0 out_seq=0 2024-05-26T22:48:38.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.886+0000 7fb7d5600640 10 client.4650 ms_handle_connect on v2:172.21.15.187:6832/2547512415 2024-05-26T22:48:38.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.888+0000 7fb7d5600640 1 -- 172.21.15.131:0/543119040 <== mds.0 v2:172.21.15.187:6832/2547512415 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55bb5bb7fd40 con 0x55bb5bc715f0 2024-05-26T22:48:38.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.888+0000 7fb7d5600640 10 client.4650 handle_command_reply: tid=0 2024-05-26T22:48:38.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.888+0000 7fb7f2408740 1 client.4650 shutdown 2024-05-26T22:48:38.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.888+0000 7fb7f2408740 1 -- 172.21.15.131:0/543119040 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bb5bc378f0 msgr2=0x55bb5bc49730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:38.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.888+0000 7fb7f2408740 1 --2- 172.21.15.131:0/543119040 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bb5bc378f0 0x55bb5bc49730 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb7d000ebb0 tx=0x7fb7d0013000 comp rx=0 tx=0).stop 2024-05-26T22:48:38.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.889+0000 7fb7f2408740 1 -- 172.21.15.131:0/543119040 shutdown_connections 2024-05-26T22:48:38.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.889+0000 7fb7f2408740 1 --2- 172.21.15.131:0/543119040 >> [v2:172.21.15.187:6832/2547512415,v1:172.21.15.187:6834/2547512415] conn(0x55bb5bc715f0 0x55bb5bc719d0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:38.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.889+0000 7fb7f2408740 1 --2- 172.21.15.131:0/543119040 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55bb5bc48590 0x55bb5bc3b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:38.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.889+0000 7fb7f2408740 1 --2- 172.21.15.131:0/543119040 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bb5bc3a300 0x55bb5bc3df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:38.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.889+0000 7fb7f2408740 1 --2- 172.21.15.131:0/543119040 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bb5bc378f0 0x55bb5bc49730 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:38.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.889+0000 7fb7d5600640 0 client.4650 ms_handle_reset on v2:172.21.15.187:6832/2547512415 2024-05-26T22:48:38.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.889+0000 7fb7f2408740 1 -- 172.21.15.131:0/543119040 >> 172.21.15.131:0/543119040 conn(0x55bb5bc11860 msgr2=0x55bb5bc0f110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:38.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.889+0000 7fb7f2408740 1 -- 172.21.15.131:0/543119040 shutdown_connections 2024-05-26T22:48:38.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.889+0000 7fb7f2408740 1 -- 172.21.15.131:0/543119040 wait complete. 2024-05-26T22:48:38.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.889+0000 7fb7f2408740 20 client.4650 trim_cache size 0 max 16384 2024-05-26T22:48:38.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.909+0000 7fb7f2408740 1 Processor -- start 2024-05-26T22:48:38.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.909+0000 7fb7f2408740 1 -- start start 2024-05-26T22:48:38.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.910+0000 7fb7f2408740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55bb5bc378f0 0x55bb5bc3bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:38.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.910+0000 7fb7f2408740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bb5bc3a300 0x55bb5bc3c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:38.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.910+0000 7fb7f2408740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bb5bc48590 0x55bb5bc3af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:38.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.910+0000 7fb7f2408740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fb7e8157d30 con 0x55bb5bc3a300 2024-05-26T22:48:38.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.910+0000 7fb7f2408740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb7e8157050 con 0x55bb5bc48590 2024-05-26T22:48:38.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.910+0000 7fb7f2408740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb7e81576a0 con 0x55bb5bc378f0 2024-05-26T22:48:38.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.910+0000 7fb7e7e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bb5bc3a300 0x55bb5bc3c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:38.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.910+0000 7fb7e7e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bb5bc3a300 0x55bb5bc3c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58246/0 (socket says 172.21.15.131:58246) 2024-05-26T22:48:38.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.910+0000 7fb7e7e00640 1 -- 172.21.15.131:0/830191304 learned_addr learned my addr 172.21.15.131:0/830191304 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:38.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.910+0000 7fb7ecc00640 1 --2- 172.21.15.131:0/830191304 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55bb5bc378f0 0x55bb5bc3bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:38.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.910+0000 7fb7e7e00640 1 -- 172.21.15.131:0/830191304 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55bb5bc378f0 msgr2=0x55bb5bc3bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:38.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.910+0000 7fb7e7e00640 1 --2- 172.21.15.131:0/830191304 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55bb5bc378f0 0x55bb5bc3bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:38.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.910+0000 7fb7ed600640 1 --2- 172.21.15.131:0/830191304 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bb5bc48590 0x55bb5bc3af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:38.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.910+0000 7fb7e7e00640 1 -- 172.21.15.131:0/830191304 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bb5bc48590 msgr2=0x55bb5bc3af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:38.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.910+0000 7fb7e7e00640 1 --2- 172.21.15.131:0/830191304 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bb5bc48590 0x55bb5bc3af70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:38.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.910+0000 7fb7e7e00640 1 -- 172.21.15.131:0/830191304 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb5bb6e750 con 0x55bb5bc3a300 2024-05-26T22:48:38.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.910+0000 7fb7ecc00640 1 --2- 172.21.15.131:0/830191304 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55bb5bc378f0 0x55bb5bc3bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:48:38.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.911+0000 7fb7e7e00640 1 --2- 172.21.15.131:0/830191304 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bb5bc3a300 0x55bb5bc3c210 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb7d802ace0 tx=0x7fb7d802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cee00712f064292f server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:38.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.911+0000 7fb7d7e00640 1 -- 172.21.15.131:0/830191304 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7d807eca0 con 0x55bb5bc3a300 2024-05-26T22:48:38.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.911+0000 7fb7d7e00640 1 -- 172.21.15.131:0/830191304 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7d80834d0 con 0x55bb5bc3a300 2024-05-26T22:48:38.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.911+0000 7fb7d7e00640 1 -- 172.21.15.131:0/830191304 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7d8078bf0 con 0x55bb5bc3a300 2024-05-26T22:48:38.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.911+0000 7fb7f2408740 1 -- 172.21.15.131:0/830191304 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bb5bc3a300 msgr2=0x55bb5bc3c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:38.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.911+0000 7fb7f2408740 1 --2- 172.21.15.131:0/830191304 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bb5bc3a300 0x55bb5bc3c210 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb7d802ace0 tx=0x7fb7d802a6f0 comp rx=0 tx=0).stop 2024-05-26T22:48:38.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.911+0000 7fb7f2408740 1 -- 172.21.15.131:0/830191304 shutdown_connections 2024-05-26T22:48:38.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.911+0000 7fb7f2408740 1 --2- 172.21.15.131:0/830191304 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bb5bc48590 0x55bb5bc3af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:38.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.911+0000 7fb7f2408740 1 --2- 172.21.15.131:0/830191304 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bb5bc3a300 0x55bb5bc3c210 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:38.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.911+0000 7fb7f2408740 1 --2- 172.21.15.131:0/830191304 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55bb5bc378f0 0x55bb5bc3bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:38.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.911+0000 7fb7f2408740 1 -- 172.21.15.131:0/830191304 >> 172.21.15.131:0/830191304 conn(0x55bb5bc11860 msgr2=0x55bb5bc0f110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:38.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.912+0000 7fb7f2408740 1 -- 172.21.15.131:0/830191304 shutdown_connections 2024-05-26T22:48:38.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.912+0000 7fb7f2408740 1 -- 172.21.15.131:0/830191304 wait complete. 2024-05-26T22:48:38.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.912+0000 7fb7f2408740 1 -- start start 2024-05-26T22:48:38.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.912+0000 7fb7f2408740 1 Processor -- start 2024-05-26T22:48:38.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.913+0000 7fb7f2408740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bb5bc378f0 0x55bb5bc3d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:38.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.913+0000 7fb7f2408740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55bb5bc3a300 0x55bb5bcdb030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:38.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.913+0000 7fb7f2408740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bb5bc48590 0x55bb5bc3dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:38.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.913+0000 7fb7f2408740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55bb5bc3e460 con 0x55bb5bc48590 2024-05-26T22:48:38.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.913+0000 7fb7f2408740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55bb5bc48070 con 0x55bb5bc378f0 2024-05-26T22:48:38.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.913+0000 7fb7f2408740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x55bb5bc6e450 con 0x55bb5bc3a300 2024-05-26T22:48:38.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.913+0000 7fb7f2408740 10 client.0 _ll_register_callbacks cb 0x7fb7d0007d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-26T22:48:38.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.914+0000 7fb7ed600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bb5bc48590 0x55bb5bc3dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:38.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.914+0000 7fb7ed600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bb5bc48590 0x55bb5bc3dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58248/0 (socket says 172.21.15.131:58248) 2024-05-26T22:48:38.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.914+0000 7fb7ed600640 1 -- 172.21.15.131:0/2644033697 learned_addr learned my addr 172.21.15.131:0/2644033697 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:38.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.914+0000 7fb7ed600640 1 -- 172.21.15.131:0/2644033697 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55bb5bc3a300 msgr2=0x55bb5bcdb030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:38.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.914+0000 7fb7ed600640 1 --2- 172.21.15.131:0/2644033697 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55bb5bc3a300 0x55bb5bcdb030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:38.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.914+0000 7fb7ed600640 1 -- 172.21.15.131:0/2644033697 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bb5bc378f0 msgr2=0x55bb5bc3d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:38.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.914+0000 7fb7ed600640 1 --2- 172.21.15.131:0/2644033697 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bb5bc378f0 0x55bb5bc3d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:38.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.914+0000 7fb7ed600640 1 -- 172.21.15.131:0/2644033697 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb5bc3e240 con 0x55bb5bc48590 2024-05-26T22:48:38.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.914+0000 7fb7ed600640 1 --2- 172.21.15.131:0/2644033697 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bb5bc48590 0x55bb5bc3dd00 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb7e0012300 tx=0x7fb7e00127c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=744c2800167d4f89 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:38.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.914+0000 7fb7d5600640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-26T22:48:38.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.914+0000 7fb7d5600640 1 -- 172.21.15.131:0/2644033697 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7e001a070 con 0x55bb5bc48590 2024-05-26T22:48:38.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.915+0000 7fb7d5600640 1 -- 172.21.15.131:0/2644033697 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb7e0015860 con 0x55bb5bc48590 2024-05-26T22:48:38.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.915+0000 7fb7f2408740 1 -- 172.21.15.131:0/2644033697 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7d80834d0 con 0x55bb5bc48590 2024-05-26T22:48:38.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.915+0000 7fb7d5600640 1 -- 172.21.15.131:0/2644033697 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7fb7e0025050 con 0x55bb5bc48590 2024-05-26T22:48:38.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.915+0000 7fb7f2408740 10 client.4656 fetch_fsmap learned FSMap version 7 2024-05-26T22:48:38.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.915+0000 7fb7f2408740 1 -- 172.21.15.131:0/2644033697 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x55bb5bc75bb0 con 0x55bb5bc48590 2024-05-26T22:48:38.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.915+0000 7fb7d5600640 1 -- 172.21.15.131:0/2644033697 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7d80834d0 con 0x55bb5bc48590 2024-05-26T22:48:38.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.915+0000 7fb7d5600640 1 -- 172.21.15.131:0/2644033697 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 7) ==== 3494+0+0 (secure 0 0 0) 0x7fb7e0028020 con 0x55bb5bc48590 2024-05-26T22:48:38.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.915+0000 7fb7f2408740 10 client.4656 fetch_fsmap finished waiting for FSMap version 7 2024-05-26T22:48:38.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.915+0000 7fb7f2408740 10 client.4656 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-26T22:48:38.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.915+0000 7fb7f2408740 20 client.4656 populate_metadata read hostname 'smithi131' 2024-05-26T22:48:38.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.915+0000 7fb7f2408740 1 --2- 172.21.15.131:0/2644033697 >> [v2:172.21.15.187:6832/2547512415,v1:172.21.15.187:6834/2547512415] conn(0x55bb5bc803b0 0x55bb5bc80790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:38.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.915+0000 7fb7f2408740 4 client.4656 mds_command: new command op to 4281 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-26T22:48:38.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.915+0000 7fb7f2408740 1 -- 172.21.15.131:0/2644033697 --> [v2:172.21.15.187:6832/2547512415,v1:172.21.15.187:6834/2547512415] -- command(tid 0: {"prefix": "status"}) -- 0x55bb5bb7a090 con 0x55bb5bc803b0 2024-05-26T22:48:38.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.916+0000 7fb7ecc00640 1 --2- 172.21.15.131:0/2644033697 >> [v2:172.21.15.187:6832/2547512415,v1:172.21.15.187:6834/2547512415] conn(0x55bb5bc803b0 0x55bb5bc80790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:38.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.917+0000 7fb7d5600640 10 client.4656 ms_handle_connect on v2:172.21.15.187:6832/2547512415 2024-05-26T22:48:38.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.917+0000 7fb7ecc00640 1 --2- 172.21.15.131:0/2644033697 >> [v2:172.21.15.187:6832/2547512415,v1:172.21.15.187:6834/2547512415] conn(0x55bb5bc803b0 0x55bb5bc80790 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2f854de0f398b10 server_cookie=c52d8d2e4e6cb40a in_seq=0 out_seq=0 2024-05-26T22:48:38.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.917+0000 7fb7d5600640 1 -- 172.21.15.131:0/2644033697 <== mds.0 v2:172.21.15.187:6832/2547512415 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x55bb5bb7a090 con 0x55bb5bc803b0 2024-05-26T22:48:38.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.917+0000 7fb7d5600640 10 client.4656 handle_command_reply: tid=0 2024-05-26T22:48:38.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.917+0000 7fb7f2408740 1 client.4656 shutdown 2024-05-26T22:48:38.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.917+0000 7fb7f2408740 1 -- 172.21.15.131:0/2644033697 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bb5bc48590 msgr2=0x55bb5bc3dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:38.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.917+0000 7fb7f2408740 1 --2- 172.21.15.131:0/2644033697 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bb5bc48590 0x55bb5bc3dd00 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb7e0012300 tx=0x7fb7e00127c0 comp rx=0 tx=0).stop 2024-05-26T22:48:38.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.918+0000 7fb7f2408740 1 -- 172.21.15.131:0/2644033697 shutdown_connections 2024-05-26T22:48:38.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.918+0000 7fb7f2408740 1 --2- 172.21.15.131:0/2644033697 >> [v2:172.21.15.187:6832/2547512415,v1:172.21.15.187:6834/2547512415] conn(0x55bb5bc803b0 0x55bb5bc80790 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:38.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.918+0000 7fb7f2408740 1 --2- 172.21.15.131:0/2644033697 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55bb5bc3a300 0x55bb5bcdb030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:38.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.918+0000 7fb7f2408740 1 --2- 172.21.15.131:0/2644033697 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bb5bc48590 0x55bb5bc3dd00 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:38.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.918+0000 7fb7f2408740 1 --2- 172.21.15.131:0/2644033697 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bb5bc378f0 0x55bb5bc3d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:38.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.918+0000 7fb7f2408740 1 -- 172.21.15.131:0/2644033697 >> 172.21.15.131:0/2644033697 conn(0x55bb5bc11860 msgr2=0x55bb5bc815d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:38.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.918+0000 7fb7d5600640 0 client.4656 ms_handle_reset on v2:172.21.15.187:6832/2547512415 2024-05-26T22:48:38.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.918+0000 7fb7f2408740 1 -- 172.21.15.131:0/2644033697 shutdown_connections 2024-05-26T22:48:38.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.918+0000 7fb7f2408740 1 -- 172.21.15.131:0/2644033697 wait complete. 2024-05-26T22:48:38.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.918+0000 7fb7f2408740 20 client.4656 trim_cache size 0 max 16384 2024-05-26T22:48:38.922 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-26T22:48:38.922 INFO:teuthology.orchestra.run.smithi131.stdout: "cluster_fsid": "46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f", 2024-05-26T22:48:38.922 INFO:teuthology.orchestra.run.smithi131.stdout: "whoami": 0, 2024-05-26T22:48:38.922 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 4281, 2024-05-26T22:48:38.922 INFO:teuthology.orchestra.run.smithi131.stdout: "want_state": "up:active", 2024-05-26T22:48:38.922 INFO:teuthology.orchestra.run.smithi131.stdout: "state": "up:active", 2024-05-26T22:48:38.922 INFO:teuthology.orchestra.run.smithi131.stdout: "fs_name": "cephfs", 2024-05-26T22:48:38.922 INFO:teuthology.orchestra.run.smithi131.stdout: "rank_uptime": 17.114874957000001, 2024-05-26T22:48:38.922 INFO:teuthology.orchestra.run.smithi131.stdout: "mdsmap_epoch": 7, 2024-05-26T22:48:38.923 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch": 20, 2024-05-26T22:48:38.923 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch_barrier": 20, 2024-05-26T22:48:38.923 INFO:teuthology.orchestra.run.smithi131.stdout: "uptime": 22.181775621 2024-05-26T22:48:38.923 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-26T22:48:38.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.918+0000 7fb7ef600640 1 -- 172.21.15.131:0/422940806 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb7c406be30 msgr2=0x7fb7c406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:38.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.918+0000 7fb7ef600640 1 --2- 172.21.15.131:0/422940806 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb7c406be30 0x7fb7c406e2f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb7d0001700 tx=0x7fb7d0009040 comp rx=0 tx=0).stop 2024-05-26T22:48:38.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.918+0000 7fb7ef600640 1 -- 172.21.15.131:0/422940806 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb7e80725a0 msgr2=0x7fb7e81539f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:38.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.918+0000 7fb7ef600640 1 --2- 172.21.15.131:0/422940806 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb7e80725a0 0x7fb7e81539f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb7d8009d90 tx=0x7fb7d8004a40 comp rx=0 tx=0).stop 2024-05-26T22:48:38.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.919+0000 7fb7ef600640 1 -- 172.21.15.131:0/422940806 shutdown_connections 2024-05-26T22:48:38.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.919+0000 7fb7ef600640 1 --2- 172.21.15.131:0/422940806 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb7c406be30 0x7fb7c406e2f0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:38.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.919+0000 7fb7ef600640 1 --2- 172.21.15.131:0/422940806 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb7e8155770 0x7fb7e809c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:38.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.919+0000 7fb7ef600640 1 --2- 172.21.15.131:0/422940806 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb7e80725a0 0x7fb7e81539f0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:38.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.919+0000 7fb7ef600640 1 --2- 172.21.15.131:0/422940806 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb7e806dd30 0x7fb7e80ada90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:38.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.919+0000 7fb7ef600640 1 -- 172.21.15.131:0/422940806 >> 172.21.15.131:0/422940806 conn(0x7fb7e806bc90 msgr2=0x7fb7e80b0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:38.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.919+0000 7fb7ef600640 1 -- 172.21.15.131:0/422940806 shutdown_connections 2024-05-26T22:48:38.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:38.919+0000 7fb7ef600640 1 -- 172.21.15.131:0/422940806 wait complete. 2024-05-26T22:48:38.935 INFO:teuthology.run_tasks:Running task sequential... 2024-05-26T22:48:38.946 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-05-26T22:48:38.946 INFO:teuthology.task.sequential:In sequential, running task print... 2024-05-26T22:48:38.947 INFO:teuthology.task.print:Enabling mgr modules 2024-05-26T22:48:38.947 INFO:teuthology.run_tasks:Running task kclient... 2024-05-26T22:48:38.988 INFO:tasks.kclient:Mounting kernel clients... 2024-05-26T22:48:38.988 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-05-26T22:48:38.988 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:48:38.989 DEBUG:teuthology.orchestra.run.smithi131:> ip netns list 2024-05-26T22:48:39.043 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:48:39.043 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link delete ceph-brx 2024-05-26T22:48:39.131 INFO:teuthology.orchestra.run.smithi131.stderr:Cannot find device "ceph-brx" 2024-05-26T22:48:39.134 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-26T22:48:39.135 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:48:39.135 DEBUG:teuthology.orchestra.run.smithi131:> ip netns list 2024-05-26T22:48:39.185 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:48:39.186 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link delete ceph-brx 2024-05-26T22:48:39.263 INFO:teuthology.orchestra.run.smithi131.stderr:Cannot find device "ceph-brx" 2024-05-26T22:48:39.266 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-26T22:48:39.266 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:48:39.266 DEBUG:teuthology.orchestra.run.smithi187:> ip netns list 2024-05-26T22:48:39.325 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:48:39.325 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip link delete ceph-brx 2024-05-26T22:48:39.415 INFO:teuthology.orchestra.run.smithi187.stderr:Cannot find device "ceph-brx" 2024-05-26T22:48:39.418 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-26T22:48:39.419 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:48:39.419 DEBUG:teuthology.orchestra.run.smithi187:> ip netns list 2024-05-26T22:48:39.471 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:48:39.471 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip link delete ceph-brx 2024-05-26T22:48:39.559 INFO:teuthology.orchestra.run.smithi187.stderr:Cannot find device "ceph-brx" 2024-05-26T22:48:39.562 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-26T22:48:39.563 INFO:tasks.kclient:client.0 config is {'syntax': 'v2'} 2024-05-26T22:48:39.563 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-26T22:48:39.563 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-26T22:48:39.563 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-26T22:48:39.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:39.768+0000 7f035f200640 1 Processor -- start 2024-05-26T22:48:39.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:39.768+0000 7f035f200640 1 -- start start 2024-05-26T22:48:39.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:39.770+0000 7f035f200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f036006ebc0 0x7f03600cedf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:39.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:39.770+0000 7f035f200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0360071b70 0x7f03600b3d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:39.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:39.770+0000 7f035f200640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03600722a0 con 0x7f0360144340 2024-05-26T22:48:39.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:39.770+0000 7f035f200640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f0360072eb0 con 0x7f036006ebc0 2024-05-26T22:48:39.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:39.770+0000 7f035f200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f0360072580 con 0x7f0360071b70 2024-05-26T22:48:39.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:39.770+0000 7f035d400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f036006ebc0 0x7f03600cedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:39.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:39.770+0000 7f035d400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f036006ebc0 0x7f03600cedf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58262/0 (socket says 172.21.15.131:58262) 2024-05-26T22:48:39.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:39.770+0000 7f035d400640 1 -- 172.21.15.131:0/441103531 learned_addr learned my addr 172.21.15.131:0/441103531 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:39.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:39.770+0000 7f035d400640 1 -- 172.21.15.131:0/441103531 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0360071b70 msgr2=0x7f03600b3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:39.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:39.770+0000 7f035e800640 1 --2- 172.21.15.131:0/441103531 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0360071b70 0x7f03600b3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:39.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:39.770+0000 7f035ca00640 1 -- 172.21.15.131:0/441103531 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1094624650 0 0) 0x7f03600722a0 con 0x7f0360144340 2024-05-26T22:48:39.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:39.771+0000 7f035d400640 1 --2- 172.21.15.131:0/441103531 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0360071b70 0x7f03600b3d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:39.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:39.771+0000 7f035d400640 1 -- 172.21.15.131:0/441103531 >> v1:172.21.15.131:6789/0 conn(0x7f0360144340 legacy=0x7f036006e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:39.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:39.771+0000 7f035d400640 1 -- 172.21.15.131:0/441103531 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0360072860 con 0x7f036006ebc0 2024-05-26T22:48:39.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:39.771+0000 7f035e800640 1 --2- 172.21.15.131:0/441103531 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0360071b70 0x7f03600b3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:48:39.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:39.772+0000 7f035d400640 1 --2- 172.21.15.131:0/441103531 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f036006ebc0 0x7f03600cedf0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f034c0098a0 tx=0x7f034c02a290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e129ee440b0f64c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:39.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:39.772+0000 7f035ca00640 1 -- 172.21.15.131:0/441103531 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f034c037070 con 0x7f036006ebc0 2024-05-26T22:48:39.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:39.772+0000 7f035ca00640 1 -- 172.21.15.131:0/441103531 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f034c032440 con 0x7f036006ebc0 2024-05-26T22:48:39.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:39.772+0000 7f035ca00640 1 -- 172.21.15.131:0/441103531 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f034c032810 con 0x7f036006ebc0 2024-05-26T22:48:39.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:39.772+0000 7f035f200640 1 -- 172.21.15.131:0/441103531 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f036006ebc0 msgr2=0x7f03600cedf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:39.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:39.772+0000 7f035f200640 1 --2- 172.21.15.131:0/441103531 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f036006ebc0 0x7f03600cedf0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f034c0098a0 tx=0x7f034c02a290 comp rx=0 tx=0).stop 2024-05-26T22:48:39.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:39.773+0000 7f035f200640 1 -- 172.21.15.131:0/441103531 shutdown_connections 2024-05-26T22:48:39.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:39.773+0000 7f035f200640 1 --2- 172.21.15.131:0/441103531 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0360071b70 0x7f03600b3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:39.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:39.773+0000 7f035f200640 1 --2- 172.21.15.131:0/441103531 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f036006ebc0 0x7f03600cedf0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:39.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:39.773+0000 7f035f200640 1 -- 172.21.15.131:0/441103531 >> 172.21.15.131:0/441103531 conn(0x7f03600cff40 msgr2=0x7f03600d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:39.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:39.773+0000 7f035f200640 1 -- 172.21.15.131:0/441103531 shutdown_connections 2024-05-26T22:48:39.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:39.773+0000 7f035f200640 1 -- 172.21.15.131:0/441103531 wait complete. 2024-05-26T22:48:39.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:39.774+0000 7f035f200640 1 Processor -- start 2024-05-26T22:48:39.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:39.774+0000 7f035f200640 1 -- start start 2024-05-26T22:48:39.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:39.775+0000 7f035f200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f036006ebc0 0x7f036013e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:39.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:39.775+0000 7f035f200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0360071b70 0x7f0360157750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:39.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:39.775+0000 7f035f200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0360144340 0x7f0360157c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:39.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:39.775+0000 7f035f200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f03600b4260 con 0x7f0360144340 2024-05-26T22:48:39.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:39.775+0000 7f035f200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0360071fa0 con 0x7f0360071b70 2024-05-26T22:48:39.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:39.775+0000 7f035f200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f03600b43e0 con 0x7f036006ebc0 2024-05-26T22:48:39.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:39.776+0000 7f035e800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0360144340 0x7f0360157c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:39.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:39.776+0000 7f035e800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0360144340 0x7f0360157c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58274/0 (socket says 172.21.15.131:58274) 2024-05-26T22:48:39.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:39.776+0000 7f035e800640 1 -- 172.21.15.131:0/2328188648 learned_addr learned my addr 172.21.15.131:0/2328188648 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:39.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:39.776+0000 7f035d400640 1 --2- 172.21.15.131:0/2328188648 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0360071b70 0x7f0360157750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:39.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:39.776+0000 7f035de00640 1 --2- 172.21.15.131:0/2328188648 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f036006ebc0 0x7f036013e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:39.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:39.776+0000 7f035e800640 1 -- 172.21.15.131:0/2328188648 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f036006ebc0 msgr2=0x7f036013e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:39.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:39.776+0000 7f035e800640 1 --2- 172.21.15.131:0/2328188648 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f036006ebc0 0x7f036013e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:39.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:39.776+0000 7f035e800640 1 -- 172.21.15.131:0/2328188648 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0360071b70 msgr2=0x7f0360157750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:39.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:39.776+0000 7f035e800640 1 --2- 172.21.15.131:0/2328188648 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0360071b70 0x7f0360157750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:39.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:39.776+0000 7f035e800640 1 -- 172.21.15.131:0/2328188648 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0360071400 con 0x7f0360144340 2024-05-26T22:48:39.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:39.776+0000 7f035de00640 1 --2- 172.21.15.131:0/2328188648 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f036006ebc0 0x7f036013e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:48:39.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:39.776+0000 7f035e800640 1 --2- 172.21.15.131:0/2328188648 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0360144340 0x7f0360157c90 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f035800cef0 tx=0x7f035800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:39.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:39.777+0000 7f0356000640 1 -- 172.21.15.131:0/2328188648 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f035801a070 con 0x7f0360144340 2024-05-26T22:48:39.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:39.777+0000 7f0356000640 1 -- 172.21.15.131:0/2328188648 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f035800d860 con 0x7f0360144340 2024-05-26T22:48:39.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:39.777+0000 7f035f200640 1 -- 172.21.15.131:0/2328188648 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03600716e0 con 0x7f0360144340 2024-05-26T22:48:39.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:39.777+0000 7f0356000640 1 -- 172.21.15.131:0/2328188648 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f035800dc50 con 0x7f0360144340 2024-05-26T22:48:39.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:39.777+0000 7f035f200640 1 -- 172.21.15.131:0/2328188648 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03600b5730 con 0x7f0360144340 2024-05-26T22:48:39.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:39.778+0000 7f035f200640 1 -- 172.21.15.131:0/2328188648 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03600694e0 con 0x7f0360144340 2024-05-26T22:48:39.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:39.779+0000 7f0356000640 1 -- 172.21.15.131:0/2328188648 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f035801d070 con 0x7f0360144340 2024-05-26T22:48:39.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:39.780+0000 7f0356000640 1 --2- 172.21.15.131:0/2328188648 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f034006be30 0x7f034006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:39.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:39.780+0000 7f0356000640 1 -- 172.21.15.131:0/2328188648 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f035808b7e0 con 0x7f0360144340 2024-05-26T22:48:39.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:39.780+0000 7f035de00640 1 --2- 172.21.15.131:0/2328188648 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f034006be30 0x7f034006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:39.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:39.781+0000 7f035de00640 1 --2- 172.21.15.131:0/2328188648 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f034006be30 0x7f034006e2f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f0350000f60 tx=0x7f0350009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:39.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:39.784+0000 7f0356000640 1 -- 172.21.15.131:0/2328188648 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f035805a1b0 con 0x7f0360144340 2024-05-26T22:48:40.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:40.022+0000 7f035f200640 1 -- 172.21.15.131:0/2328188648 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f03600718b0 con 0x7f0360144340 2024-05-26T22:48:40.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:40.023+0000 7f0356000640 1 -- 172.21.15.131:0/2328188648 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+73 (secure 0 0 0) 0x7f0358059ab0 con 0x7f0360144340 2024-05-26T22:48:40.023 INFO:teuthology.orchestra.run.smithi131.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-26T22:48:40.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:40.026+0000 7f035f200640 1 -- 172.21.15.131:0/2328188648 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f034006be30 msgr2=0x7f034006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:40.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:40.026+0000 7f035f200640 1 --2- 172.21.15.131:0/2328188648 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f034006be30 0x7f034006e2f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f0350000f60 tx=0x7f0350009040 comp rx=0 tx=0).stop 2024-05-26T22:48:40.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:40.026+0000 7f035f200640 1 -- 172.21.15.131:0/2328188648 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0360144340 msgr2=0x7f0360157c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:40.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:40.026+0000 7f035f200640 1 --2- 172.21.15.131:0/2328188648 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0360144340 0x7f0360157c90 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f035800cef0 tx=0x7f035800a710 comp rx=0 tx=0).stop 2024-05-26T22:48:40.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:40.026+0000 7f035f200640 1 -- 172.21.15.131:0/2328188648 shutdown_connections 2024-05-26T22:48:40.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:40.026+0000 7f035f200640 1 --2- 172.21.15.131:0/2328188648 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f034006be30 0x7f034006e2f0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:40.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:40.026+0000 7f035f200640 1 --2- 172.21.15.131:0/2328188648 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0360144340 0x7f0360157c90 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:40.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:40.026+0000 7f035f200640 1 --2- 172.21.15.131:0/2328188648 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0360071b70 0x7f0360157750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:40.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:40.026+0000 7f035f200640 1 --2- 172.21.15.131:0/2328188648 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f036006ebc0 0x7f036013e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:40.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:40.026+0000 7f035f200640 1 -- 172.21.15.131:0/2328188648 >> 172.21.15.131:0/2328188648 conn(0x7f03600cff40 msgr2=0x7f036013c930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:40.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:40.026+0000 7f035f200640 1 -- 172.21.15.131:0/2328188648 shutdown_connections 2024-05-26T22:48:40.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:40.026+0000 7f035f200640 1 -- 172.21.15.131:0/2328188648 wait complete. 2024-05-26T22:48:40.045 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-26T22:48:40.045 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-26T22:48:40.045 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi131.front.sepia.ceph.com 2024-05-26T22:48:40.045 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-26T22:48:40.045 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-26T22:48:40.046 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-26T22:48:40.046 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-26T22:48:40.046 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-26T22:48:40.046 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-26T22:48:40.046 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:48:40.046 DEBUG:teuthology.orchestra.run.smithi131:> ip addr 2024-05-26T22:48:40.092 INFO:teuthology.orchestra.run.smithi131.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-26T22:48:40.092 INFO:teuthology.orchestra.run.smithi131.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-26T22:48:40.092 INFO:teuthology.orchestra.run.smithi131.stdout: inet 127.0.0.1/8 scope host lo 2024-05-26T22:48:40.092 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-05-26T22:48:40.092 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 ::1/128 scope host 2024-05-26T22:48:40.092 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-05-26T22:48:40.092 INFO:teuthology.orchestra.run.smithi131.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-26T22:48:40.092 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:d8:fb:18 brd ff:ff:ff:ff:ff:ff 2024-05-26T22:48:40.092 INFO:teuthology.orchestra.run.smithi131.stdout: altname enp7s0f0 2024-05-26T22:48:40.093 INFO:teuthology.orchestra.run.smithi131.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-26T22:48:40.093 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:d8:fb:19 brd ff:ff:ff:ff:ff:ff 2024-05-26T22:48:40.093 INFO:teuthology.orchestra.run.smithi131.stdout: altname enp7s0f1 2024-05-26T22:48:40.093 INFO:teuthology.orchestra.run.smithi131.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-26T22:48:40.093 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:88:72:f8 brd ff:ff:ff:ff:ff:ff 2024-05-26T22:48:40.093 INFO:teuthology.orchestra.run.smithi131.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-26T22:48:40.093 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:88:72:f9 brd ff:ff:ff:ff:ff:ff 2024-05-26T22:48:40.093 INFO:teuthology.orchestra.run.smithi131.stdout: inet 172.21.15.131/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-26T22:48:40.093 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft 42577sec preferred_lft 42577sec 2024-05-26T22:48:40.093 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::ec4:7aff:fe88:72f9/64 scope link 2024-05-26T22:48:40.093 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-05-26T22:48:40.093 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-26T22:48:40.094 DEBUG:teuthology.orchestra.run.smithi131:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-26T22:48:40.094 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2024-05-26T22:48:40.094 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link add name ceph-brx type bridge 2024-05-26T22:48:40.094 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip addr flush dev ceph-brx 2024-05-26T22:48:40.094 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link set ceph-brx up 2024-05-26T22:48:40.094 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-26T22:48:40.094 DEBUG:teuthology.orchestra.run.smithi131:> ') 2024-05-26T22:48:40.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-26T22:48:40.597 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-26T22:48:40.607 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:48:40.607 DEBUG:teuthology.orchestra.run.smithi131:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-26T22:48:40.672 INFO:teuthology.orchestra.run.smithi131.stdout:1 2024-05-26T22:48:40.675 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:48:40.675 DEBUG:teuthology.orchestra.run.smithi131:> route 2024-05-26T22:48:40.734 INFO:teuthology.orchestra.run.smithi131.stdout:Kernel IP routing table 2024-05-26T22:48:40.734 INFO:teuthology.orchestra.run.smithi131.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-26T22:48:40.734 INFO:teuthology.orchestra.run.smithi131.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-26T22:48:40.735 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-26T22:48:40.735 INFO:teuthology.orchestra.run.smithi131.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-26T22:48:40.735 DEBUG:teuthology.orchestra.run.smithi131:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-26T22:48:40.735 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2024-05-26T22:48:40.735 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-26T22:48:40.735 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-26T22:48:40.735 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-26T22:48:40.735 DEBUG:teuthology.orchestra.run.smithi131:> ') 2024-05-26T22:48:40.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-26T22:48:41.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-26T22:48:41.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:48:41.470 DEBUG:teuthology.orchestra.run.smithi131:> ip netns list 2024-05-26T22:48:41.517 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:48:41.518 DEBUG:teuthology.orchestra.run.smithi131:> ip netns list-id 2024-05-26T22:48:41.609 DEBUG:teuthology.orchestra.run.smithi131:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-26T22:48:41.609 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2024-05-26T22:48:41.609 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-26T22:48:41.609 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-05-26T22:48:41.609 DEBUG:teuthology.orchestra.run.smithi131:> ') 2024-05-26T22:48:41.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:41 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-26T22:48:41.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-26T22:48:41.779 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-05-26T22:48:41.779 DEBUG:teuthology.orchestra.run.smithi131:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-26T22:48:41.780 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2024-05-26T22:48:41.780 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-05-26T22:48:41.780 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-26T22:48:41.780 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-05-26T22:48:41.780 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-05-26T22:48:41.780 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-05-26T22:48:41.780 DEBUG:teuthology.orchestra.run.smithi131:> ') 2024-05-26T22:48:41.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:41 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-26T22:48:42.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-26T22:48:42.063 DEBUG:teuthology.orchestra.run.smithi131:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-26T22:48:42.063 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2024-05-26T22:48:42.063 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link set brx.0 up 2024-05-26T22:48:42.063 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link set dev brx.0 master ceph-brx 2024-05-26T22:48:42.063 DEBUG:teuthology.orchestra.run.smithi131:> ') 2024-05-26T22:48:42.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-26T22:48:42.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-26T22:48:42.250 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-26T22:48:42.250 DEBUG:teuthology.orchestra.run.smithi131:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-26T22:48:42.288 INFO:teuthology.orchestra.run.smithi131.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-26T22:48:42.289 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-26T22:48:42.289 DEBUG:teuthology.orchestra.run.smithi131:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-26T22:48:42.378 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-05-26T22:48:42.378 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:48:42.378 DEBUG:teuthology.orchestra.run.smithi131:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-26T22:48:42.523 INFO:teuthology.orchestra.run.smithi131.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-26T22:48:42.523 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-26T22:48:43.217 INFO:teuthology.orchestra.run.smithi131.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-26T22:48:43.217 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-26T22:48:43.217 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: resolved to: "172.21.15.131:3300,172.21.15.187:3300,172.21.15.187:3301" 2024-05-26T22:48:43.217 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: trying mount with new device syntax: 0@46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f.cephfs=/ 2024-05-26T22:48:43.217 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.131:3300/172.21.15.187:3300/172.21.15.187:3301" will pass to kernel 2024-05-26T22:48:43.220 INFO:teuthology.orchestra.run.smithi131.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-26T22:48:43.221 INFO:teuthology.orchestra.run.smithi131.stdout: You just mounted a file system that supports labels which does not 2024-05-26T22:48:43.221 INFO:teuthology.orchestra.run.smithi131.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-26T22:48:43.221 INFO:teuthology.orchestra.run.smithi131.stdout: applications will generate AVC messages and not be allowed access to 2024-05-26T22:48:43.221 INFO:teuthology.orchestra.run.smithi131.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-26T22:48:43.225 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-26T22:48:43.226 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:48:43.226 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-26T22:48:43.309 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-26T22:48:43.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:43 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-26T22:48:43.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-26T22:48:43.455 INFO:teuthology.orchestra.run.smithi131.stdout:client4668 2024-05-26T22:48:43.456 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-26T22:48:43.456 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-26T22:48:43.504 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-26T22:48:43.592 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-26T22:48:43.599 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-26T22:48:43.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:43 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-26T22:48:43.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-26T22:48:43.681 INFO:teuthology.orchestra.run.smithi131.stdout:client4668 2024-05-26T22:48:43.681 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-26T22:48:43.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:43 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-26T22:48:43.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-26T22:48:43.820 INFO:teuthology.orchestra.run.smithi131.stdout:client4668 2024-05-26T22:48:43.820 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f.client4668/status') 2024-05-26T22:48:43.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:43 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-26T22:48:43.973 INFO:teuthology.orchestra.run.smithi131.stdout:instance: client.4668 (3)192.168.0.1:0/1267935264 2024-05-26T22:48:43.973 INFO:teuthology.orchestra.run.smithi131.stdout:blocklisted: false 2024-05-26T22:48:43.974 INFO:teuthology.orchestra.run.smithi131.stderr:0+1 records in 2024-05-26T22:48:43.974 INFO:teuthology.orchestra.run.smithi131.stderr:0+1 records out 2024-05-26T22:48:43.974 INFO:teuthology.orchestra.run.smithi131.stderr:69 bytes copied, 7.5999e-05 s, 908 kB/s 2024-05-26T22:48:43.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-26T22:48:43.987 INFO:tasks.kclient:client.2 config is {'syntax': 'v2'} 2024-05-26T22:48:43.987 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-26T22:48:43.987 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-05-26T22:48:43.987 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-26T22:48:44.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:44.192+0000 7fabe8800640 1 Processor -- start 2024-05-26T22:48:44.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:44.192+0000 7fabe8800640 1 -- start start 2024-05-26T22:48:44.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:44.194+0000 7fabe8800640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fabe0162780 0x7fabe00b3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:44.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:44.194+0000 7fabe8800640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fabe006dcc0 0x7fabe00ad950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:44.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:44.194+0000 7fabe8800640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fabe00ad170 con 0x7fabe006f600 2024-05-26T22:48:44.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:44.194+0000 7fabe8800640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fabe00b4510 con 0x7fabe0162780 2024-05-26T22:48:44.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:44.194+0000 7fabe8800640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fabe00b47f0 con 0x7fabe006dcc0 2024-05-26T22:48:44.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:44.194+0000 7fabe5e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fabe0162780 0x7fabe00b3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:44.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:44.194+0000 7fabe5e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fabe0162780 0x7fabe00b3ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58290/0 (socket says 172.21.15.131:58290) 2024-05-26T22:48:44.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:44.194+0000 7fabe5e00640 1 -- 172.21.15.131:0/1226574121 learned_addr learned my addr 172.21.15.131:0/1226574121 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:44.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:44.195+0000 7fabe6800640 1 --2- 172.21.15.131:0/1226574121 >> v2:172.21.15.187:3300/0 conn(0x7fabe006dcc0 0x7fabe00ad950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:44.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:44.195+0000 7fabe5e00640 1 -- 172.21.15.131:0/1226574121 >> v2:172.21.15.187:3300/0 conn(0x7fabe006dcc0 msgr2=0x7fabe00ad950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:44.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:44.195+0000 7fabe5e00640 1 --2- 172.21.15.131:0/1226574121 >> v2:172.21.15.187:3300/0 conn(0x7fabe006dcc0 0x7fabe00ad950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:44.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:44.195+0000 7fabe5e00640 1 -- 172.21.15.131:0/1226574121 >> v1:172.21.15.187:6789/0 conn(0x7fabe006f600 legacy=0x7fabe006d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:44.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:44.195+0000 7fabe5e00640 1 -- 172.21.15.131:0/1226574121 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabe00b4ad0 con 0x7fabe0162780 2024-05-26T22:48:44.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:44.196+0000 7fabe5e00640 1 --2- 172.21.15.131:0/1226574121 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fabe0162780 0x7fabe00b3ba0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fabe006d3b0 tx=0x7fabd402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c05bd8181a2414bd server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:44.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:44.196+0000 7fabe4a00640 1 -- 172.21.15.131:0/1226574121 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabd4037070 con 0x7fabe0162780 2024-05-26T22:48:44.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:44.196+0000 7fabe4a00640 1 -- 172.21.15.131:0/1226574121 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fabd4032440 con 0x7fabe0162780 2024-05-26T22:48:44.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:44.196+0000 7fabe4a00640 1 -- 172.21.15.131:0/1226574121 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabd4032810 con 0x7fabe0162780 2024-05-26T22:48:44.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:44.197+0000 7fabe8800640 1 -- 172.21.15.131:0/1226574121 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fabe0162780 msgr2=0x7fabe00b3ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:44.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:44.197+0000 7fabe8800640 1 --2- 172.21.15.131:0/1226574121 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fabe0162780 0x7fabe00b3ba0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fabe006d3b0 tx=0x7fabd402a230 comp rx=0 tx=0).stop 2024-05-26T22:48:44.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:44.197+0000 7fabe8800640 1 -- 172.21.15.131:0/1226574121 shutdown_connections 2024-05-26T22:48:44.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:44.197+0000 7fabe8800640 1 --2- 172.21.15.131:0/1226574121 >> v2:172.21.15.187:3300/0 conn(0x7fabe006dcc0 0x7fabe00ad950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:44.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:44.197+0000 7fabe8800640 1 --2- 172.21.15.131:0/1226574121 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fabe0162780 0x7fabe00b3ba0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:44.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:44.197+0000 7fabe8800640 1 -- 172.21.15.131:0/1226574121 >> 172.21.15.131:0/1226574121 conn(0x7fabe00b1e90 msgr2=0x7fabe00b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:44.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:44.197+0000 7fabe8800640 1 -- 172.21.15.131:0/1226574121 shutdown_connections 2024-05-26T22:48:44.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:44.197+0000 7fabe8800640 1 -- 172.21.15.131:0/1226574121 wait complete. 2024-05-26T22:48:44.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:44.199+0000 7fabe8800640 1 Processor -- start 2024-05-26T22:48:44.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:44.199+0000 7fabe8800640 1 -- start start 2024-05-26T22:48:44.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:44.200+0000 7fabe8800640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fabe006dcc0 0x7fabe0156d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:44.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:44.200+0000 7fabe8800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fabe006f600 0x7fabe009d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:44.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:44.200+0000 7fabe8800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fabe0162780 0x7fabe0072870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:44.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:44.200+0000 7fabe8800640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fabe00b4260 con 0x7fabe0162780 2024-05-26T22:48:44.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:44.200+0000 7fabe8800640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fabe00b40e0 con 0x7fabe006dcc0 2024-05-26T22:48:44.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:44.200+0000 7fabe8800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fabe00ade90 con 0x7fabe006f600 2024-05-26T22:48:44.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:44.200+0000 7fabe6800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fabe0162780 0x7fabe0072870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:44.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:44.200+0000 7fabe6800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fabe0162780 0x7fabe0072870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58292/0 (socket says 172.21.15.131:58292) 2024-05-26T22:48:44.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:44.200+0000 7fabe6800640 1 -- 172.21.15.131:0/2970150336 learned_addr learned my addr 172.21.15.131:0/2970150336 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:44.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:44.200+0000 7fabe5e00640 1 --2- 172.21.15.131:0/2970150336 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fabe006dcc0 0x7fabe0156d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:44.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:44.200+0000 7fabe5400640 1 --2- 172.21.15.131:0/2970150336 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fabe006f600 0x7fabe009d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:44.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:44.201+0000 7fabe6800640 1 -- 172.21.15.131:0/2970150336 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fabe006f600 msgr2=0x7fabe009d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:44.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:44.201+0000 7fabe6800640 1 --2- 172.21.15.131:0/2970150336 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fabe006f600 0x7fabe009d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:44.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:44.201+0000 7fabe6800640 1 -- 172.21.15.131:0/2970150336 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fabe006dcc0 msgr2=0x7fabe0156d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:44.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:44.201+0000 7fabe6800640 1 --2- 172.21.15.131:0/2970150336 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fabe006dcc0 0x7fabe0156d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:44.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:44.201+0000 7fabe6800640 1 -- 172.21.15.131:0/2970150336 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabe00ceb50 con 0x7fabe0162780 2024-05-26T22:48:44.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:44.201+0000 7fabe5e00640 1 --2- 172.21.15.131:0/2970150336 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fabe006dcc0 0x7fabe0156d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-26T22:48:44.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:44.201+0000 7fabe6800640 1 --2- 172.21.15.131:0/2970150336 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fabe0162780 0x7fabe0072870 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fabdc007e30 tx=0x7fabdc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:44.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:44.201+0000 7fabda000640 1 -- 172.21.15.131:0/2970150336 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabdc014620 con 0x7fabe0162780 2024-05-26T22:48:44.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:44.201+0000 7fabda000640 1 -- 172.21.15.131:0/2970150336 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fabdc019070 con 0x7fabe0162780 2024-05-26T22:48:44.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:44.201+0000 7fabe8800640 1 -- 172.21.15.131:0/2970150336 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabe0157270 con 0x7fabe0162780 2024-05-26T22:48:44.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:44.201+0000 7fabda000640 1 -- 172.21.15.131:0/2970150336 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabdc014a00 con 0x7fabe0162780 2024-05-26T22:48:44.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:44.201+0000 7fabe8800640 1 -- 172.21.15.131:0/2970150336 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabe00cf0d0 con 0x7fabe0162780 2024-05-26T22:48:44.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:44.202+0000 7fabe8800640 1 -- 172.21.15.131:0/2970150336 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabe00694e0 con 0x7fabe0162780 2024-05-26T22:48:44.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:44.204+0000 7fabda000640 1 -- 172.21.15.131:0/2970150336 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fabdc014ba0 con 0x7fabe0162780 2024-05-26T22:48:44.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:44.204+0000 7fabda000640 1 --2- 172.21.15.131:0/2970150336 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fabc406be30 0x7fabc406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:44.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:44.205+0000 7fabda000640 1 -- 172.21.15.131:0/2970150336 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fabdc08b810 con 0x7fabe0162780 2024-05-26T22:48:44.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:44.205+0000 7fabe5e00640 1 --2- 172.21.15.131:0/2970150336 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fabc406be30 0x7fabc406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:44.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:44.205+0000 7fabe5e00640 1 --2- 172.21.15.131:0/2970150336 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fabc406be30 0x7fabc406e2f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fabd4034000 tx=0x7fabd402e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:44.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:44.209+0000 7fabda000640 1 -- 172.21.15.131:0/2970150336 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fabdc05a120 con 0x7fabe0162780 2024-05-26T22:48:44.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:44.447+0000 7fabe8800640 1 -- 172.21.15.131:0/2970150336 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fabe00720f0 con 0x7fabe0162780 2024-05-26T22:48:44.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:44.448+0000 7fabda000640 1 -- 172.21.15.131:0/2970150336 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+73 (secure 0 0 0) 0x7fabdc059a20 con 0x7fabe0162780 2024-05-26T22:48:44.448 INFO:teuthology.orchestra.run.smithi131.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-26T22:48:44.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:44.451+0000 7fabe8800640 1 -- 172.21.15.131:0/2970150336 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fabc406be30 msgr2=0x7fabc406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:44.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:44.451+0000 7fabe8800640 1 --2- 172.21.15.131:0/2970150336 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fabc406be30 0x7fabc406e2f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fabd4034000 tx=0x7fabd402e000 comp rx=0 tx=0).stop 2024-05-26T22:48:44.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:44.451+0000 7fabe8800640 1 -- 172.21.15.131:0/2970150336 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fabe0162780 msgr2=0x7fabe0072870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:44.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:44.451+0000 7fabe8800640 1 --2- 172.21.15.131:0/2970150336 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fabe0162780 0x7fabe0072870 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fabdc007e30 tx=0x7fabdc00e4b0 comp rx=0 tx=0).stop 2024-05-26T22:48:44.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:44.451+0000 7fabe8800640 1 -- 172.21.15.131:0/2970150336 shutdown_connections 2024-05-26T22:48:44.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:44.451+0000 7fabe8800640 1 --2- 172.21.15.131:0/2970150336 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fabe006f600 0x7fabe009d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:44.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:44.451+0000 7fabe8800640 1 --2- 172.21.15.131:0/2970150336 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fabc406be30 0x7fabc406e2f0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:44.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:44.451+0000 7fabe8800640 1 --2- 172.21.15.131:0/2970150336 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fabe0162780 0x7fabe0072870 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:44.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:44.451+0000 7fabe8800640 1 --2- 172.21.15.131:0/2970150336 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fabe006dcc0 0x7fabe0156d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:44.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:44.451+0000 7fabe8800640 1 -- 172.21.15.131:0/2970150336 >> 172.21.15.131:0/2970150336 conn(0x7fabe00b1e90 msgr2=0x7fabe00b0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:44.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:44.451+0000 7fabe8800640 1 -- 172.21.15.131:0/2970150336 shutdown_connections 2024-05-26T22:48:44.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:44.452+0000 7fabe8800640 1 -- 172.21.15.131:0/2970150336 wait complete. 2024-05-26T22:48:44.469 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-26T22:48:44.469 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-26T22:48:44.469 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi131.front.sepia.ceph.com 2024-05-26T22:48:44.469 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-05-26T22:48:44.469 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-05-26T22:48:44.470 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-26T22:48:44.470 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-26T22:48:44.470 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-26T22:48:44.470 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-05-26T22:48:44.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:48:44.470 DEBUG:teuthology.orchestra.run.smithi131:> ip addr 2024-05-26T22:48:44.522 INFO:teuthology.orchestra.run.smithi131.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-26T22:48:44.522 INFO:teuthology.orchestra.run.smithi131.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-26T22:48:44.522 INFO:teuthology.orchestra.run.smithi131.stdout: inet 127.0.0.1/8 scope host lo 2024-05-26T22:48:44.522 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-05-26T22:48:44.522 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 ::1/128 scope host 2024-05-26T22:48:44.522 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-05-26T22:48:44.522 INFO:teuthology.orchestra.run.smithi131.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-26T22:48:44.523 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:d8:fb:18 brd ff:ff:ff:ff:ff:ff 2024-05-26T22:48:44.523 INFO:teuthology.orchestra.run.smithi131.stdout: altname enp7s0f0 2024-05-26T22:48:44.523 INFO:teuthology.orchestra.run.smithi131.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-26T22:48:44.523 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:d8:fb:19 brd ff:ff:ff:ff:ff:ff 2024-05-26T22:48:44.523 INFO:teuthology.orchestra.run.smithi131.stdout: altname enp7s0f1 2024-05-26T22:48:44.523 INFO:teuthology.orchestra.run.smithi131.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-26T22:48:44.523 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:88:72:f8 brd ff:ff:ff:ff:ff:ff 2024-05-26T22:48:44.523 INFO:teuthology.orchestra.run.smithi131.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-26T22:48:44.523 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:88:72:f9 brd ff:ff:ff:ff:ff:ff 2024-05-26T22:48:44.523 INFO:teuthology.orchestra.run.smithi131.stdout: inet 172.21.15.131/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-26T22:48:44.523 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft 42573sec preferred_lft 42573sec 2024-05-26T22:48:44.523 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::ec4:7aff:fe88:72f9/64 scope link 2024-05-26T22:48:44.523 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-05-26T22:48:44.524 INFO:teuthology.orchestra.run.smithi131.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-26T22:48:44.524 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 06:f4:05:86:a4:3e brd ff:ff:ff:ff:ff:ff 2024-05-26T22:48:44.524 INFO:teuthology.orchestra.run.smithi131.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-26T22:48:44.524 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-05-26T22:48:44.524 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::8802:8fff:fe41:fc6d/64 scope link 2024-05-26T22:48:44.524 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-05-26T22:48:44.524 INFO:teuthology.orchestra.run.smithi131.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-26T22:48:44.524 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 06:f4:05:86:a4:3e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-26T22:48:44.524 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::4f4:5ff:fe86:a43e/64 scope link 2024-05-26T22:48:44.524 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-05-26T22:48:44.525 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:48:44.525 DEBUG:teuthology.orchestra.run.smithi131:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-26T22:48:44.606 INFO:teuthology.orchestra.run.smithi131.stdout:1 2024-05-26T22:48:44.610 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:48:44.610 DEBUG:teuthology.orchestra.run.smithi131:> route 2024-05-26T22:48:44.661 INFO:teuthology.orchestra.run.smithi131.stdout:Kernel IP routing table 2024-05-26T22:48:44.662 INFO:teuthology.orchestra.run.smithi131.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-26T22:48:44.662 INFO:teuthology.orchestra.run.smithi131.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-26T22:48:44.662 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-26T22:48:44.662 INFO:teuthology.orchestra.run.smithi131.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-26T22:48:44.662 DEBUG:teuthology.orchestra.run.smithi131:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-26T22:48:44.662 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2024-05-26T22:48:44.663 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-26T22:48:44.663 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-26T22:48:44.663 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-26T22:48:44.663 DEBUG:teuthology.orchestra.run.smithi131:> ') 2024-05-26T22:48:44.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-26T22:48:44.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-26T22:48:44.869 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:48:44.869 DEBUG:teuthology.orchestra.run.smithi131:> ip netns list 2024-05-26T22:48:44.919 INFO:teuthology.orchestra.run.smithi131.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-26T22:48:44.919 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:48:44.920 DEBUG:teuthology.orchestra.run.smithi131:> ip netns list-id 2024-05-26T22:48:45.014 INFO:teuthology.orchestra.run.smithi131.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-05-26T22:48:45.015 DEBUG:teuthology.orchestra.run.smithi131:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-26T22:48:45.015 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2024-05-26T22:48:45.015 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-26T22:48:45.015 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-05-26T22:48:45.015 DEBUG:teuthology.orchestra.run.smithi131:> ') 2024-05-26T22:48:45.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-26T22:48:45.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-26T22:48:45.202 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:48:45.202 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-05-26T22:48:45.271 INFO:teuthology.orchestra.run.smithi131.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-26T22:48:45.271 INFO:teuthology.orchestra.run.smithi131.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-26T22:48:45.271 INFO:teuthology.orchestra.run.smithi131.stdout: inet 127.0.0.1/8 scope host lo 2024-05-26T22:48:45.271 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-05-26T22:48:45.271 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 ::1/128 scope host 2024-05-26T22:48:45.271 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-05-26T22:48:45.271 INFO:teuthology.orchestra.run.smithi131.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-26T22:48:45.272 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether a2:3c:fa:b8:23:3c brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-05-26T22:48:45.272 INFO:teuthology.orchestra.run.smithi131.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-05-26T22:48:45.272 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-05-26T22:48:45.272 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::a03c:faff:feb8:233c/64 scope link 2024-05-26T22:48:45.272 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-05-26T22:48:45.274 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:48:45.275 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-05-26T22:48:45.351 INFO:teuthology.orchestra.run.smithi131.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-26T22:48:45.351 INFO:teuthology.orchestra.run.smithi131.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-26T22:48:45.351 INFO:teuthology.orchestra.run.smithi131.stdout: inet 127.0.0.1/8 scope host lo 2024-05-26T22:48:45.351 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-05-26T22:48:45.352 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 ::1/128 scope host 2024-05-26T22:48:45.352 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-05-26T22:48:45.352 INFO:teuthology.orchestra.run.smithi131.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-26T22:48:45.352 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether a2:3c:fa:b8:23:3c brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-05-26T22:48:45.352 INFO:teuthology.orchestra.run.smithi131.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-05-26T22:48:45.352 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-05-26T22:48:45.352 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::a03c:faff:feb8:233c/64 scope link 2024-05-26T22:48:45.352 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-05-26T22:48:45.355 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-05-26T22:48:45.355 DEBUG:teuthology.orchestra.run.smithi131:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-26T22:48:45.355 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2024-05-26T22:48:45.355 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-05-26T22:48:45.355 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-05-26T22:48:45.356 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-05-26T22:48:45.356 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-05-26T22:48:45.356 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-05-26T22:48:45.356 DEBUG:teuthology.orchestra.run.smithi131:> ') 2024-05-26T22:48:45.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-26T22:48:45.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-26T22:48:45.644 DEBUG:teuthology.orchestra.run.smithi131:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-26T22:48:45.644 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2024-05-26T22:48:45.644 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link set brx.1 up 2024-05-26T22:48:45.644 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link set dev brx.1 master ceph-brx 2024-05-26T22:48:45.644 DEBUG:teuthology.orchestra.run.smithi131:> ') 2024-05-26T22:48:45.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-26T22:48:45.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-26T22:48:45.854 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-26T22:48:45.854 DEBUG:teuthology.orchestra.run.smithi131:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-05-26T22:48:45.901 INFO:teuthology.orchestra.run.smithi131.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-05-26T22:48:45.901 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-26T22:48:45.901 DEBUG:teuthology.orchestra.run.smithi131:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-05-26T22:48:45.996 INFO:tasks.cephfs.kernel_mount:mounting using device: 2@.cephfs=/ 2024-05-26T22:48:45.996 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:48:45.996 DEBUG:teuthology.orchestra.run.smithi131:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 2@.cephfs=/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-26T22:48:46.124 INFO:teuthology.orchestra.run.smithi131.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-26T22:48:46.125 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-26T22:48:46.162 INFO:teuthology.orchestra.run.smithi131.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-26T22:48:46.162 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-26T22:48:46.163 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: resolved to: "172.21.15.131:3300,172.21.15.187:3300,172.21.15.187:3301" 2024-05-26T22:48:46.163 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: trying mount with new device syntax: 2@46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f.cephfs=/ 2024-05-26T22:48:46.163 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,ms_mode=prefer-crc,key=2,mon_addr=172.21.15.131:3300/172.21.15.187:3300/172.21.15.187:3301" will pass to kernel 2024-05-26T22:48:46.165 INFO:teuthology.orchestra.run.smithi131.stdout:mount: /home/ubuntu/cephtest/mnt.2 does not contain SELinux labels. 2024-05-26T22:48:46.166 INFO:teuthology.orchestra.run.smithi131.stdout: You just mounted a file system that supports labels which does not 2024-05-26T22:48:46.166 INFO:teuthology.orchestra.run.smithi131.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-26T22:48:46.166 INFO:teuthology.orchestra.run.smithi131.stdout: applications will generate AVC messages and not be allowed access to 2024-05-26T22:48:46.166 INFO:teuthology.orchestra.run.smithi131.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-26T22:48:46.170 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-26T22:48:46.170 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:48:46.170 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-05-26T22:48:46.244 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-26T22:48:46.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:46 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-26T22:48:46.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-26T22:48:46.362 INFO:teuthology.orchestra.run.smithi131.stdout:client4513 2024-05-26T22:48:46.362 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-26T22:48:46.362 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-26T22:48:46.412 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-26T22:48:46.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:46 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-26T22:48:46.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-26T22:48:46.577 INFO:teuthology.orchestra.run.smithi131.stdout:client4513 2024-05-26T22:48:46.578 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-26T22:48:46.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:46 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-26T22:48:46.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-26T22:48:46.699 INFO:teuthology.orchestra.run.smithi131.stdout:client4513 2024-05-26T22:48:46.699 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f.client4513/status') 2024-05-26T22:48:46.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-26T22:48:46.835 INFO:teuthology.orchestra.run.smithi131.stdout:instance: client.4513 (3)172.21.15.131:0/2767576806 2024-05-26T22:48:46.835 INFO:teuthology.orchestra.run.smithi131.stdout:blocklisted: false 2024-05-26T22:48:46.835 INFO:teuthology.orchestra.run.smithi131.stderr:0+1 records in 2024-05-26T22:48:46.835 INFO:teuthology.orchestra.run.smithi131.stderr:0+1 records out 2024-05-26T22:48:46.835 INFO:teuthology.orchestra.run.smithi131.stderr:71 bytes copied, 9.5851e-05 s, 741 kB/s 2024-05-26T22:48:46.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-26T22:48:46.848 INFO:tasks.kclient:client.1 config is {'syntax': 'v2'} 2024-05-26T22:48:46.848 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-26T22:48:46.848 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-05-26T22:48:46.848 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-26T22:48:47.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:47.052+0000 7f8ab6600640 1 Processor -- start 2024-05-26T22:48:47.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:47.052+0000 7f8ab6600640 1 -- start start 2024-05-26T22:48:47.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:47.054+0000 7f8ab6600640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f8ab0156750 0x7f8ab015fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:47.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:47.054+0000 7f8ab6600640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8ab00cc7f0 0x7f8ab00ccca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:47.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:47.054+0000 7f8ab6600640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ab0160620 con 0x7f8ab006e8c0 2024-05-26T22:48:47.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:47.054+0000 7f8ab6600640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f8ab015d770 con 0x7f8ab0156750 2024-05-26T22:48:47.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:47.054+0000 7f8ab6600640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f8ab015cbd0 con 0x7f8ab00cc7f0 2024-05-26T22:48:47.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:47.054+0000 7f8aafe00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f8ab006e8c0 0x7f8ab015d060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:39242/0 (socket says 172.21.15.131:39242) 2024-05-26T22:48:47.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:47.054+0000 7f8aafe00640 1 -- 172.21.15.131:0/3690107291 learned_addr learned my addr 172.21.15.131:0/3690107291 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:48:47.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:47.054+0000 7f8ab5200640 1 --2- 172.21.15.131:0/3690107291 >> v2:172.21.15.187:3300/0 conn(0x7f8ab0156750 0x7f8ab015fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:47.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:47.054+0000 7f8ab5c00640 1 --2- 172.21.15.131:0/3690107291 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8ab00cc7f0 0x7f8ab00ccca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:47.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:47.055+0000 7f8aaf400640 1 -- 172.21.15.131:0/3690107291 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 639480607 0 0) 0x7f8ab0160620 con 0x7f8ab006e8c0 2024-05-26T22:48:47.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:47.055+0000 7f8aaf400640 1 -- 172.21.15.131:0/3690107291 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8a94003520 con 0x7f8ab006e8c0 2024-05-26T22:48:47.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:47.055+0000 7f8aaf400640 1 -- 172.21.15.131:0/3690107291 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2205667386 0 0) 0x7f8a94003520 con 0x7f8ab006e8c0 2024-05-26T22:48:47.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:47.055+0000 7f8aaf400640 1 -- 172.21.15.131:0/3690107291 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8ab00cc7f0 msgr2=0x7f8ab00ccca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:47.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:47.055+0000 7f8aaf400640 1 --2- 172.21.15.131:0/3690107291 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8ab00cc7f0 0x7f8ab00ccca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:47.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:47.055+0000 7f8aaf400640 1 -- 172.21.15.131:0/3690107291 >> v2:172.21.15.187:3300/0 conn(0x7f8ab0156750 msgr2=0x7f8ab015fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:47.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:47.055+0000 7f8aaf400640 1 --2- 172.21.15.131:0/3690107291 >> v2:172.21.15.187:3300/0 conn(0x7f8ab0156750 0x7f8ab015fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:47.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:47.055+0000 7f8aaf400640 1 -- 172.21.15.131:0/3690107291 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ab0160900 con 0x7f8ab006e8c0 2024-05-26T22:48:47.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:47.055+0000 7f8aaf400640 1 -- 172.21.15.131:0/3690107291 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7f8a9c001670 con 0x7f8ab006e8c0 2024-05-26T22:48:47.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:47.055+0000 7f8aaf400640 1 -- 172.21.15.131:0/3690107291 >> v1:172.21.15.131:6789/0 conn(0x7f8ab006e8c0 legacy=0x7f8ab015d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:47.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:47.055+0000 7f8aaf400640 1 --2- 172.21.15.131:0/3690107291 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8a940039b0 0x7f8a94003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:47.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:47.055+0000 7f8aaf400640 1 --2- 172.21.15.131:0/3690107291 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8a94004800 0x7f8a94004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:47.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:47.055+0000 7f8aaf400640 1 --2- 172.21.15.131:0/3690107291 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8ab00cc7f0 0x7f8a940051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:47.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:47.055+0000 7f8aaf400640 1 -- 172.21.15.131:0/3690107291 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab015d770 con 0x7f8a940039b0 2024-05-26T22:48:47.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:47.055+0000 7f8aaf400640 1 -- 172.21.15.131:0/3690107291 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab015cbd0 con 0x7f8a94004800 2024-05-26T22:48:47.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:47.055+0000 7f8aaf400640 1 -- 172.21.15.131:0/3690107291 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f8a9400b9c0 con 0x7f8ab00cc7f0 2024-05-26T22:48:47.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:47.055+0000 7f8ab5200640 1 --2- 172.21.15.131:0/3690107291 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8a940039b0 0x7f8a94003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:47.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:47.056+0000 7f8aafe00640 1 --2- 172.21.15.131:0/3690107291 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8a94004800 0x7f8a94004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:47.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:47.056+0000 7f8ab5c00640 1 --2- 172.21.15.131:0/3690107291 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8ab00cc7f0 0x7f8a940051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:47.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:47.056+0000 7f8ab5200640 1 -- 172.21.15.131:0/3690107291 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8ab00cc7f0 msgr2=0x7f8a940051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:47.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:47.056+0000 7f8ab5200640 1 --2- 172.21.15.131:0/3690107291 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8ab00cc7f0 0x7f8a940051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:47.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:47.056+0000 7f8ab5200640 1 -- 172.21.15.131:0/3690107291 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8a94004800 msgr2=0x7f8a94004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:47.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:47.056+0000 7f8ab5200640 1 --2- 172.21.15.131:0/3690107291 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8a94004800 0x7f8a94004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:47.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:47.056+0000 7f8ab5200640 1 -- 172.21.15.131:0/3690107291 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a94003520 con 0x7f8a940039b0 2024-05-26T22:48:47.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:47.057+0000 7f8ab5200640 1 --2- 172.21.15.131:0/3690107291 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8a940039b0 0x7f8a94003dd0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8aa0009fc0 tx=0x7f8a94005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d047e2a87dc72457 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:47.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:47.057+0000 7f8aaf400640 1 -- 172.21.15.131:0/3690107291 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa0009c10 con 0x7f8a940039b0 2024-05-26T22:48:47.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:47.057+0000 7f8aaf400640 1 -- 172.21.15.131:0/3690107291 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8aa0036070 con 0x7f8a940039b0 2024-05-26T22:48:47.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:47.057+0000 7f8aaf400640 1 -- 172.21.15.131:0/3690107291 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa002d070 con 0x7f8a940039b0 2024-05-26T22:48:47.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:47.058+0000 7f8ab6600640 1 -- 172.21.15.131:0/3690107291 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8a940039b0 msgr2=0x7f8a94003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:47.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:47.058+0000 7f8ab6600640 1 --2- 172.21.15.131:0/3690107291 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8a940039b0 0x7f8a94003dd0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8aa0009fc0 tx=0x7f8a94005830 comp rx=0 tx=0).stop 2024-05-26T22:48:47.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:47.058+0000 7f8ab5c00640 1 -- 172.21.15.131:0/3690107291 reap_dead start 2024-05-26T22:48:47.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:47.058+0000 7f8ab6600640 1 -- 172.21.15.131:0/3690107291 shutdown_connections 2024-05-26T22:48:47.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:47.058+0000 7f8ab6600640 1 -- 172.21.15.131:0/3690107291 >> 172.21.15.131:0/3690107291 conn(0x7f8ab00cff40 msgr2=0x7f8ab00d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:47.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:47.058+0000 7f8ab6600640 1 -- 172.21.15.131:0/3690107291 shutdown_connections 2024-05-26T22:48:47.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:47.058+0000 7f8ab6600640 1 -- 172.21.15.131:0/3690107291 wait complete. 2024-05-26T22:48:47.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:47.060+0000 7f8ab6600640 1 Processor -- start 2024-05-26T22:48:47.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:47.060+0000 7f8ab6600640 1 -- start start 2024-05-26T22:48:47.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:47.061+0000 7f8ab6600640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8a940039b0 0x7f8ab0138a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:47.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:47.061+0000 7f8ab6600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ab01fbdb0 0x7f8ab00b0700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:47.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:47.061+0000 7f8ab6600640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ab00b0c40 0x7f8ab015e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:47.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:47.061+0000 7f8ab6600640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab0156b30 con 0x7f8ab01fbdb0 2024-05-26T22:48:47.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:47.061+0000 7f8ab6600640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab015c860 con 0x7f8ab00b0c40 2024-05-26T22:48:47.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:47.061+0000 7f8ab6600640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f8ab0160280 con 0x7f8a940039b0 2024-05-26T22:48:47.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:47.061+0000 7f8aafe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ab01fbdb0 0x7f8ab00b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:47.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:47.061+0000 7f8aafe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ab01fbdb0 0x7f8ab00b0700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47122/0 (socket says 172.21.15.131:47122) 2024-05-26T22:48:47.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:47.061+0000 7f8aafe00640 1 -- 172.21.15.131:0/406014108 learned_addr learned my addr 172.21.15.131:0/406014108 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:47.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:47.061+0000 7f8ab5200640 1 --2- 172.21.15.131:0/406014108 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8a940039b0 0x7f8ab0138a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:47.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:47.061+0000 7f8ab5c00640 1 --2- 172.21.15.131:0/406014108 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ab00b0c40 0x7f8ab015e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:47.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:47.062+0000 7f8aafe00640 1 -- 172.21.15.131:0/406014108 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8a940039b0 msgr2=0x7f8ab0138a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:47.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:47.062+0000 7f8aafe00640 1 --2- 172.21.15.131:0/406014108 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8a940039b0 0x7f8ab0138a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:47.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:47.062+0000 7f8aafe00640 1 -- 172.21.15.131:0/406014108 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ab00b0c40 msgr2=0x7f8ab015e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:47.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:47.062+0000 7f8aafe00640 1 --2- 172.21.15.131:0/406014108 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ab00b0c40 0x7f8ab015e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:47.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:47.062+0000 7f8aafe00640 1 -- 172.21.15.131:0/406014108 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ab01fc520 con 0x7f8ab01fbdb0 2024-05-26T22:48:47.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:47.062+0000 7f8aafe00640 1 --2- 172.21.15.131:0/406014108 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ab01fbdb0 0x7f8ab00b0700 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c00e730 tx=0x7f8a9c00ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:47.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:47.062+0000 7f8aacc00640 1 -- 172.21.15.131:0/406014108 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a9c00c6c0 con 0x7f8ab01fbdb0 2024-05-26T22:48:47.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:47.062+0000 7f8aacc00640 1 -- 172.21.15.131:0/406014108 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a9c018070 con 0x7f8ab01fbdb0 2024-05-26T22:48:47.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:47.062+0000 7f8ab6600640 1 -- 172.21.15.131:0/406014108 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ab01fc800 con 0x7f8ab01fbdb0 2024-05-26T22:48:47.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:47.062+0000 7f8aacc00640 1 -- 172.21.15.131:0/406014108 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a9c00ca80 con 0x7f8ab01fbdb0 2024-05-26T22:48:47.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:47.062+0000 7f8ab6600640 1 -- 172.21.15.131:0/406014108 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ab015e270 con 0x7f8ab01fbdb0 2024-05-26T22:48:47.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:47.063+0000 7f8ab6600640 1 -- 172.21.15.131:0/406014108 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ab00694e0 con 0x7f8ab01fbdb0 2024-05-26T22:48:47.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:47.064+0000 7f8aacc00640 1 -- 172.21.15.131:0/406014108 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8a9c00cc20 con 0x7f8ab01fbdb0 2024-05-26T22:48:47.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:47.065+0000 7f8aacc00640 1 --2- 172.21.15.131:0/406014108 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f8a9006be60 0x7f8a9006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:47.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:47.065+0000 7f8aacc00640 1 -- 172.21.15.131:0/406014108 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f8a9c08ace0 con 0x7f8ab01fbdb0 2024-05-26T22:48:47.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:47.066+0000 7f8ab5200640 1 --2- 172.21.15.131:0/406014108 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f8a9006be60 0x7f8a9006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:47.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:47.066+0000 7f8ab5200640 1 --2- 172.21.15.131:0/406014108 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f8a9006be60 0x7f8a9006e320 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f8aa0035000 tx=0x7f8aa002f000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:47.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:47.070+0000 7f8aacc00640 1 -- 172.21.15.131:0/406014108 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f8a9c0595f0 con 0x7f8ab01fbdb0 2024-05-26T22:48:47.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:47.310+0000 7f8ab6600640 1 -- 172.21.15.131:0/406014108 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f8ab015a650 con 0x7f8ab01fbdb0 2024-05-26T22:48:47.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:47.311+0000 7f8aacc00640 1 -- 172.21.15.131:0/406014108 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+73 (secure 0 0 0) 0x7f8a9c058ef0 con 0x7f8ab01fbdb0 2024-05-26T22:48:47.311 INFO:teuthology.orchestra.run.smithi131.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-26T22:48:47.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:47.313+0000 7f8ab6600640 1 -- 172.21.15.131:0/406014108 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f8a9006be60 msgr2=0x7f8a9006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:47.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:47.313+0000 7f8ab6600640 1 --2- 172.21.15.131:0/406014108 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f8a9006be60 0x7f8a9006e320 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f8aa0035000 tx=0x7f8aa002f000 comp rx=0 tx=0).stop 2024-05-26T22:48:47.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:47.313+0000 7f8ab6600640 1 -- 172.21.15.131:0/406014108 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ab01fbdb0 msgr2=0x7f8ab00b0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:47.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:47.313+0000 7f8ab6600640 1 --2- 172.21.15.131:0/406014108 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ab01fbdb0 0x7f8ab00b0700 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c00e730 tx=0x7f8a9c00ebf0 comp rx=0 tx=0).stop 2024-05-26T22:48:47.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:47.313+0000 7f8ab6600640 1 -- 172.21.15.131:0/406014108 shutdown_connections 2024-05-26T22:48:47.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:47.313+0000 7f8ab6600640 1 --2- 172.21.15.131:0/406014108 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f8a9006be60 0x7f8a9006e320 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:47.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:47.313+0000 7f8ab6600640 1 --2- 172.21.15.131:0/406014108 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ab00b0c40 0x7f8ab015e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:47.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:47.313+0000 7f8ab6600640 1 --2- 172.21.15.131:0/406014108 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ab01fbdb0 0x7f8ab00b0700 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:47.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:47.313+0000 7f8ab6600640 1 --2- 172.21.15.131:0/406014108 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8a940039b0 0x7f8ab0138a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:47.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:47.313+0000 7f8ab6600640 1 -- 172.21.15.131:0/406014108 >> 172.21.15.131:0/406014108 conn(0x7f8ab00cff40 msgr2=0x7f8ab0142040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:47.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:47.314+0000 7f8ab6600640 1 -- 172.21.15.131:0/406014108 shutdown_connections 2024-05-26T22:48:47.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:47.314+0000 7f8ab6600640 1 -- 172.21.15.131:0/406014108 wait complete. 2024-05-26T22:48:47.333 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-26T22:48:47.333 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-26T22:48:47.333 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi187.front.sepia.ceph.com 2024-05-26T22:48:47.333 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-05-26T22:48:47.333 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-05-26T22:48:47.333 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-26T22:48:47.333 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-26T22:48:47.333 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-26T22:48:47.333 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-05-26T22:48:47.333 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:48:47.333 DEBUG:teuthology.orchestra.run.smithi187:> ip addr 2024-05-26T22:48:47.390 INFO:teuthology.orchestra.run.smithi187.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-26T22:48:47.390 INFO:teuthology.orchestra.run.smithi187.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-26T22:48:47.390 INFO:teuthology.orchestra.run.smithi187.stdout: inet 127.0.0.1/8 scope host lo 2024-05-26T22:48:47.390 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-05-26T22:48:47.391 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 ::1/128 scope host 2024-05-26T22:48:47.391 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-05-26T22:48:47.391 INFO:teuthology.orchestra.run.smithi187.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-26T22:48:47.391 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 0c:c4:7a:d9:36:78 brd ff:ff:ff:ff:ff:ff 2024-05-26T22:48:47.391 INFO:teuthology.orchestra.run.smithi187.stdout: altname enp7s0f0 2024-05-26T22:48:47.391 INFO:teuthology.orchestra.run.smithi187.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-26T22:48:47.391 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 0c:c4:7a:d9:36:79 brd ff:ff:ff:ff:ff:ff 2024-05-26T22:48:47.391 INFO:teuthology.orchestra.run.smithi187.stdout: altname enp7s0f1 2024-05-26T22:48:47.391 INFO:teuthology.orchestra.run.smithi187.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-26T22:48:47.391 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 0c:c4:7a:88:77:b0 brd ff:ff:ff:ff:ff:ff 2024-05-26T22:48:47.391 INFO:teuthology.orchestra.run.smithi187.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-26T22:48:47.392 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 0c:c4:7a:88:77:b1 brd ff:ff:ff:ff:ff:ff 2024-05-26T22:48:47.392 INFO:teuthology.orchestra.run.smithi187.stdout: inet 172.21.15.187/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-26T22:48:47.392 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft 42557sec preferred_lft 42557sec 2024-05-26T22:48:47.392 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 fe80::ec4:7aff:fe88:77b1/64 scope link 2024-05-26T22:48:47.392 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-05-26T22:48:47.393 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-26T22:48:47.393 DEBUG:teuthology.orchestra.run.smithi187:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-26T22:48:47.393 DEBUG:teuthology.orchestra.run.smithi187:> set -e 2024-05-26T22:48:47.393 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip link add name ceph-brx type bridge 2024-05-26T22:48:47.393 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip addr flush dev ceph-brx 2024-05-26T22:48:47.393 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip link set ceph-brx up 2024-05-26T22:48:47.394 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-26T22:48:47.394 DEBUG:teuthology.orchestra.run.smithi187:> ') 2024-05-26T22:48:47.495 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:47 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-26T22:48:47.889 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-26T22:48:47.899 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:48:47.900 DEBUG:teuthology.orchestra.run.smithi187:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-26T22:48:47.970 INFO:teuthology.orchestra.run.smithi187.stdout:1 2024-05-26T22:48:47.974 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:48:47.974 DEBUG:teuthology.orchestra.run.smithi187:> route 2024-05-26T22:48:48.024 INFO:teuthology.orchestra.run.smithi187.stdout:Kernel IP routing table 2024-05-26T22:48:48.025 INFO:teuthology.orchestra.run.smithi187.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-26T22:48:48.025 INFO:teuthology.orchestra.run.smithi187.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-26T22:48:48.025 INFO:teuthology.orchestra.run.smithi187.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-26T22:48:48.025 INFO:teuthology.orchestra.run.smithi187.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-26T22:48:48.026 DEBUG:teuthology.orchestra.run.smithi187:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-26T22:48:48.026 DEBUG:teuthology.orchestra.run.smithi187:> set -e 2024-05-26T22:48:48.026 DEBUG:teuthology.orchestra.run.smithi187:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-26T22:48:48.026 DEBUG:teuthology.orchestra.run.smithi187:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-26T22:48:48.026 DEBUG:teuthology.orchestra.run.smithi187:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-26T22:48:48.027 DEBUG:teuthology.orchestra.run.smithi187:> ') 2024-05-26T22:48:48.123 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-26T22:48:48.787 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-26T22:48:48.799 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:48:48.799 DEBUG:teuthology.orchestra.run.smithi187:> ip netns list 2024-05-26T22:48:48.851 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:48:48.851 DEBUG:teuthology.orchestra.run.smithi187:> ip netns list-id 2024-05-26T22:48:48.901 DEBUG:teuthology.orchestra.run.smithi187:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-26T22:48:48.901 DEBUG:teuthology.orchestra.run.smithi187:> set -e 2024-05-26T22:48:48.901 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-26T22:48:48.901 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-05-26T22:48:48.901 DEBUG:teuthology.orchestra.run.smithi187:> ') 2024-05-26T22:48:48.998 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-26T22:48:49.072 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-26T22:48:49.082 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-05-26T22:48:49.082 DEBUG:teuthology.orchestra.run.smithi187:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-26T22:48:49.083 DEBUG:teuthology.orchestra.run.smithi187:> set -e 2024-05-26T22:48:49.083 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-05-26T22:48:49.083 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-26T22:48:49.083 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-05-26T22:48:49.083 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-05-26T22:48:49.083 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-05-26T22:48:49.083 DEBUG:teuthology.orchestra.run.smithi187:> ') 2024-05-26T22:48:49.178 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-26T22:48:49.364 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-26T22:48:49.373 DEBUG:teuthology.orchestra.run.smithi187:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-26T22:48:49.373 DEBUG:teuthology.orchestra.run.smithi187:> set -e 2024-05-26T22:48:49.373 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip link set brx.0 up 2024-05-26T22:48:49.373 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip link set dev brx.0 master ceph-brx 2024-05-26T22:48:49.373 DEBUG:teuthology.orchestra.run.smithi187:> ') 2024-05-26T22:48:49.466 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-26T22:48:49.559 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-26T22:48:49.568 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-26T22:48:49.568 DEBUG:teuthology.orchestra.run.smithi187:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-05-26T22:48:49.614 INFO:teuthology.orchestra.run.smithi187.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-05-26T22:48:49.615 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-26T22:48:49.615 DEBUG:teuthology.orchestra.run.smithi187:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-05-26T22:48:49.707 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-05-26T22:48:49.707 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:48:49.707 DEBUG:teuthology.orchestra.run.smithi187:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-26T22:48:49.837 INFO:teuthology.orchestra.run.smithi187.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-26T22:48:49.837 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-26T22:48:50.513 INFO:teuthology.orchestra.run.smithi187.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-26T22:48:50.513 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-26T22:48:50.513 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: resolved to: "172.21.15.131:3300,172.21.15.187:3300,172.21.15.187:3301" 2024-05-26T22:48:50.513 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: trying mount with new device syntax: 1@46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f.cephfs=/ 2024-05-26T22:48:50.513 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.131:3300/172.21.15.187:3300/172.21.15.187:3301" will pass to kernel 2024-05-26T22:48:50.516 INFO:teuthology.orchestra.run.smithi187.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-05-26T22:48:50.516 INFO:teuthology.orchestra.run.smithi187.stdout: You just mounted a file system that supports labels which does not 2024-05-26T22:48:50.516 INFO:teuthology.orchestra.run.smithi187.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-26T22:48:50.517 INFO:teuthology.orchestra.run.smithi187.stdout: applications will generate AVC messages and not be allowed access to 2024-05-26T22:48:50.517 INFO:teuthology.orchestra.run.smithi187.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-26T22:48:50.522 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-26T22:48:50.522 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:48:50.522 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-05-26T22:48:50.603 DEBUG:teuthology.orchestra.run.smithi187:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-26T22:48:50.707 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:50 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-26T22:48:50.713 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-26T22:48:50.722 INFO:teuthology.orchestra.run.smithi187.stdout:client4522 2024-05-26T22:48:50.723 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-05-26T22:48:50.723 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-26T22:48:50.773 DEBUG:teuthology.orchestra.run.smithi187:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-26T22:48:50.926 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:50 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-26T22:48:50.930 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-26T22:48:50.940 INFO:teuthology.orchestra.run.smithi187.stdout:client4522 2024-05-26T22:48:50.940 DEBUG:teuthology.orchestra.run.smithi187:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-26T22:48:51.043 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:51 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-26T22:48:51.048 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-26T22:48:51.057 INFO:teuthology.orchestra.run.smithi187.stdout:client4522 2024-05-26T22:48:51.057 DEBUG:teuthology.orchestra.run.smithi187:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f.client4522/status') 2024-05-26T22:48:51.163 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-26T22:48:51.193 INFO:teuthology.orchestra.run.smithi187.stdout:instance: client.4522 (3)192.168.0.1:0/3032562193 2024-05-26T22:48:51.193 INFO:teuthology.orchestra.run.smithi187.stdout:blocklisted: false 2024-05-26T22:48:51.193 INFO:teuthology.orchestra.run.smithi187.stderr:0+1 records in 2024-05-26T22:48:51.193 INFO:teuthology.orchestra.run.smithi187.stderr:0+1 records out 2024-05-26T22:48:51.193 INFO:teuthology.orchestra.run.smithi187.stderr:69 bytes copied, 0.000102699 s, 672 kB/s 2024-05-26T22:48:51.196 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-26T22:48:51.206 INFO:tasks.kclient:client.3 config is {'syntax': 'v2'} 2024-05-26T22:48:51.206 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-26T22:48:51.207 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-05-26T22:48:51.207 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-26T22:48:51.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:51.416+0000 7fe7df200640 1 Processor -- start 2024-05-26T22:48:51.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:51.416+0000 7fe7df200640 1 -- start start 2024-05-26T22:48:51.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:51.418+0000 7fe7df200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe7e0144800 0x7fe7e015fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:51.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:51.418+0000 7fe7df200640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7e015f7b0 con 0x7fe7e00ad870 2024-05-26T22:48:51.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:51.418+0000 7fe7df200640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7e00cde40 con 0x7fe7e015f100 2024-05-26T22:48:51.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:51.418+0000 7fe7df200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe7e015f590 con 0x7fe7e0144800 2024-05-26T22:48:51.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:51.418+0000 7fe7dd400640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fe7e00ad870 0x7fe7e015e9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:39258/0 (socket says 172.21.15.131:39258) 2024-05-26T22:48:51.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:51.418+0000 7fe7dd400640 1 -- 172.21.15.131:0/4197526287 learned_addr learned my addr 172.21.15.131:0/4197526287 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:48:51.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:51.419+0000 7fe7dde00640 1 --2- 172.21.15.131:0/4197526287 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe7e0144800 0x7fe7e015fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:51.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:51.419+0000 7fe7dca00640 1 -- 172.21.15.131:0/4197526287 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1154390715 0 0) 0x7fe7e015f7b0 con 0x7fe7e00ad870 2024-05-26T22:48:51.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:51.419+0000 7fe7dca00640 1 -- 172.21.15.131:0/4197526287 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe7c4003520 con 0x7fe7e00ad870 2024-05-26T22:48:51.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:51.419+0000 7fe7dca00640 1 -- 172.21.15.131:0/4197526287 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1130968840 0 0) 0x7fe7c4003520 con 0x7fe7e00ad870 2024-05-26T22:48:51.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:51.419+0000 7fe7dca00640 1 -- 172.21.15.131:0/4197526287 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe7e0144800 msgr2=0x7fe7e015fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:51.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:51.419+0000 7fe7dca00640 1 --2- 172.21.15.131:0/4197526287 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe7e0144800 0x7fe7e015fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:51.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:51.419+0000 7fe7dca00640 1 -- 172.21.15.131:0/4197526287 >> v1:172.21.15.187:6789/0 conn(0x7fe7e015f100 legacy=0x7fe7e00ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:51.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:51.419+0000 7fe7dca00640 1 -- 172.21.15.131:0/4197526287 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7e00ce120 con 0x7fe7e00ad870 2024-05-26T22:48:51.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:51.419+0000 7fe7dca00640 1 -- 172.21.15.131:0/4197526287 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7fe7cc001670 con 0x7fe7e00ad870 2024-05-26T22:48:51.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:51.419+0000 7fe7dca00640 1 -- 172.21.15.131:0/4197526287 >> v1:172.21.15.131:6789/0 conn(0x7fe7e00ad870 legacy=0x7fe7e015e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:51.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:51.419+0000 7fe7dca00640 1 --2- 172.21.15.131:0/4197526287 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe7c40039e0 0x7fe7c4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:51.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:51.419+0000 7fe7dca00640 1 --2- 172.21.15.131:0/4197526287 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe7c4004800 0x7fe7c4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:51.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:51.419+0000 7fe7dca00640 1 --2- 172.21.15.131:0/4197526287 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe7e0144800 0x7fe7c40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:51.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:51.420+0000 7fe7dca00640 1 -- 172.21.15.131:0/4197526287 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe7e015f590 con 0x7fe7c4004800 2024-05-26T22:48:51.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:51.420+0000 7fe7dca00640 1 -- 172.21.15.131:0/4197526287 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe7c400bb00 con 0x7fe7c40039e0 2024-05-26T22:48:51.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:51.420+0000 7fe7dca00640 1 -- 172.21.15.131:0/4197526287 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe7c400bd00 con 0x7fe7e0144800 2024-05-26T22:48:51.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:51.420+0000 7fe7dd400640 1 --2- 172.21.15.131:0/4197526287 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe7c4004800 0x7fe7c4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:51.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:51.420+0000 7fe7dd400640 1 -- 172.21.15.131:0/4197526287 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe7e0144800 msgr2=0x7fe7c40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:51.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:51.420+0000 7fe7dd400640 1 --2- 172.21.15.131:0/4197526287 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe7e0144800 0x7fe7c40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:51.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:51.420+0000 7fe7dd400640 1 -- 172.21.15.131:0/4197526287 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe7c40039e0 msgr2=0x7fe7c4003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:51.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:51.420+0000 7fe7dd400640 1 --2- 172.21.15.131:0/4197526287 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe7c40039e0 0x7fe7c4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:51.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:51.420+0000 7fe7dd400640 1 -- 172.21.15.131:0/4197526287 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7c4003520 con 0x7fe7c4004800 2024-05-26T22:48:51.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:51.421+0000 7fe7dd400640 1 --2- 172.21.15.131:0/4197526287 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe7c4004800 0x7fe7c4004c80 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc00c210 tx=0x7fe7c4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46da580c362ce6be server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:51.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:51.421+0000 7fe7dca00640 1 -- 172.21.15.131:0/4197526287 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7cc007cf0 con 0x7fe7c4004800 2024-05-26T22:48:51.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:51.421+0000 7fe7dca00640 1 -- 172.21.15.131:0/4197526287 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7cc005af0 con 0x7fe7c4004800 2024-05-26T22:48:51.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:51.422+0000 7fe7dca00640 1 -- 172.21.15.131:0/4197526287 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7cc007070 con 0x7fe7c4004800 2024-05-26T22:48:51.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:51.422+0000 7fe7df200640 1 -- 172.21.15.131:0/4197526287 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe7c4004800 msgr2=0x7fe7c4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:51.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:51.422+0000 7fe7df200640 1 --2- 172.21.15.131:0/4197526287 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe7c4004800 0x7fe7c4004c80 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc00c210 tx=0x7fe7c4005830 comp rx=0 tx=0).stop 2024-05-26T22:48:51.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:51.422+0000 7fe7de800640 1 -- 172.21.15.131:0/4197526287 reap_dead start 2024-05-26T22:48:51.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:51.422+0000 7fe7df200640 1 -- 172.21.15.131:0/4197526287 shutdown_connections 2024-05-26T22:48:51.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:51.422+0000 7fe7df200640 1 -- 172.21.15.131:0/4197526287 >> 172.21.15.131:0/4197526287 conn(0x7fe7e00cff40 msgr2=0x7fe7e00d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:51.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:51.422+0000 7fe7df200640 1 -- 172.21.15.131:0/4197526287 shutdown_connections 2024-05-26T22:48:51.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:51.422+0000 7fe7df200640 1 -- 172.21.15.131:0/4197526287 wait complete. 2024-05-26T22:48:51.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:51.424+0000 7fe7df200640 1 Processor -- start 2024-05-26T22:48:51.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:51.424+0000 7fe7df200640 1 -- start start 2024-05-26T22:48:51.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:51.425+0000 7fe7df200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe7c4004800 0x7fe7e0133ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:51.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:51.425+0000 7fe7df200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe7e01343e0 0x7fe7e00af940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:51.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:51.425+0000 7fe7df200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe7e00afe80 0x7fe7e0156e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:51.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:51.425+0000 7fe7df200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe7e00adcd0 con 0x7fe7c4004800 2024-05-26T22:48:51.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:51.425+0000 7fe7df200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe7e0144be0 con 0x7fe7e01343e0 2024-05-26T22:48:51.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:51.425+0000 7fe7df200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe7e00ade50 con 0x7fe7e00afe80 2024-05-26T22:48:51.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:51.425+0000 7fe7dde00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe7c4004800 0x7fe7e0133ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:51.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:51.425+0000 7fe7dde00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe7c4004800 0x7fe7e0133ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47134/0 (socket says 172.21.15.131:47134) 2024-05-26T22:48:51.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:51.425+0000 7fe7dde00640 1 -- 172.21.15.131:0/2492325502 learned_addr learned my addr 172.21.15.131:0/2492325502 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:51.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:51.426+0000 7fe7dde00640 1 -- 172.21.15.131:0/2492325502 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe7e00afe80 msgr2=0x7fe7e0156e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:51.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:51.426+0000 7fe7dde00640 1 --2- 172.21.15.131:0/2492325502 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe7e00afe80 0x7fe7e0156e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:51.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:51.426+0000 7fe7dde00640 1 -- 172.21.15.131:0/2492325502 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe7e01343e0 msgr2=0x7fe7e00af940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:51.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:51.426+0000 7fe7dde00640 1 --2- 172.21.15.131:0/2492325502 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe7e01343e0 0x7fe7e00af940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:51.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:51.426+0000 7fe7dde00640 1 -- 172.21.15.131:0/2492325502 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7e0134c80 con 0x7fe7c4004800 2024-05-26T22:48:51.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:51.426+0000 7fe7dde00640 1 --2- 172.21.15.131:0/2492325502 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe7c4004800 0x7fe7e0133ea0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe7d000e930 tx=0x7fe7d000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:51.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:51.426+0000 7fe7d6000640 1 -- 172.21.15.131:0/2492325502 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d0017070 con 0x7fe7c4004800 2024-05-26T22:48:51.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:51.426+0000 7fe7d6000640 1 -- 172.21.15.131:0/2492325502 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7d000c8e0 con 0x7fe7c4004800 2024-05-26T22:48:51.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:51.426+0000 7fe7d6000640 1 -- 172.21.15.131:0/2492325502 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d000ccf0 con 0x7fe7c4004800 2024-05-26T22:48:51.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:51.426+0000 7fe7df200640 1 -- 172.21.15.131:0/2492325502 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7e015b680 con 0x7fe7c4004800 2024-05-26T22:48:51.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:51.426+0000 7fe7df200640 1 -- 172.21.15.131:0/2492325502 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7e00b3440 con 0x7fe7c4004800 2024-05-26T22:48:51.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:51.428+0000 7fe7df200640 1 -- 172.21.15.131:0/2492325502 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7e00b02a0 con 0x7fe7c4004800 2024-05-26T22:48:51.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:51.428+0000 7fe7d6000640 1 -- 172.21.15.131:0/2492325502 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe7d001a070 con 0x7fe7c4004800 2024-05-26T22:48:51.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:51.429+0000 7fe7d6000640 1 --2- 172.21.15.131:0/2492325502 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fe7c006be30 0x7fe7c006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:51.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:51.429+0000 7fe7d6000640 1 -- 172.21.15.131:0/2492325502 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fe7d008b1c0 con 0x7fe7c4004800 2024-05-26T22:48:51.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:51.430+0000 7fe7dd400640 1 --2- 172.21.15.131:0/2492325502 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fe7c006be30 0x7fe7c006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:51.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:51.430+0000 7fe7dd400640 1 --2- 172.21.15.131:0/2492325502 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fe7c006be30 0x7fe7c006e2f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe7c40057e0 tx=0x7fe7cc007000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:51.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:51.434+0000 7fe7d6000640 1 -- 172.21.15.131:0/2492325502 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fe7d0059b60 con 0x7fe7c4004800 2024-05-26T22:48:51.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:51.680+0000 7fe7df200640 1 -- 172.21.15.131:0/2492325502 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fe7e00694e0 con 0x7fe7c4004800 2024-05-26T22:48:51.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:51.681+0000 7fe7d6000640 1 -- 172.21.15.131:0/2492325502 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+73 (secure 0 0 0) 0x7fe7d0059460 con 0x7fe7c4004800 2024-05-26T22:48:51.681 INFO:teuthology.orchestra.run.smithi131.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-26T22:48:51.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:51.683+0000 7fe7df200640 1 -- 172.21.15.131:0/2492325502 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fe7c006be30 msgr2=0x7fe7c006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:51.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:51.683+0000 7fe7df200640 1 --2- 172.21.15.131:0/2492325502 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fe7c006be30 0x7fe7c006e2f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe7c40057e0 tx=0x7fe7cc007000 comp rx=0 tx=0).stop 2024-05-26T22:48:51.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:51.683+0000 7fe7df200640 1 -- 172.21.15.131:0/2492325502 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe7c4004800 msgr2=0x7fe7e0133ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:51.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:51.683+0000 7fe7df200640 1 --2- 172.21.15.131:0/2492325502 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe7c4004800 0x7fe7e0133ea0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe7d000e930 tx=0x7fe7d000edf0 comp rx=0 tx=0).stop 2024-05-26T22:48:51.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:51.684+0000 7fe7df200640 1 -- 172.21.15.131:0/2492325502 shutdown_connections 2024-05-26T22:48:51.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:51.684+0000 7fe7df200640 1 --2- 172.21.15.131:0/2492325502 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe7e00afe80 0x7fe7e0156e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:51.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:51.684+0000 7fe7df200640 1 --2- 172.21.15.131:0/2492325502 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fe7c006be30 0x7fe7c006e2f0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:51.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:51.684+0000 7fe7df200640 1 --2- 172.21.15.131:0/2492325502 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe7e01343e0 0x7fe7e00af940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:51.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:51.684+0000 7fe7df200640 1 --2- 172.21.15.131:0/2492325502 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe7c4004800 0x7fe7e0133ea0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:51.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:51.684+0000 7fe7df200640 1 -- 172.21.15.131:0/2492325502 >> 172.21.15.131:0/2492325502 conn(0x7fe7e00cff40 msgr2=0x7fe7e01610c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:51.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:51.684+0000 7fe7df200640 1 -- 172.21.15.131:0/2492325502 shutdown_connections 2024-05-26T22:48:51.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:51.684+0000 7fe7df200640 1 -- 172.21.15.131:0/2492325502 wait complete. 2024-05-26T22:48:51.704 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-26T22:48:51.704 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-26T22:48:51.704 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi187.front.sepia.ceph.com 2024-05-26T22:48:51.704 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-05-26T22:48:51.704 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-05-26T22:48:51.704 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-26T22:48:51.704 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-26T22:48:51.704 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-26T22:48:51.704 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-05-26T22:48:51.705 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:48:51.705 DEBUG:teuthology.orchestra.run.smithi187:> ip addr 2024-05-26T22:48:51.764 INFO:teuthology.orchestra.run.smithi187.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-26T22:48:51.764 INFO:teuthology.orchestra.run.smithi187.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-26T22:48:51.764 INFO:teuthology.orchestra.run.smithi187.stdout: inet 127.0.0.1/8 scope host lo 2024-05-26T22:48:51.764 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-05-26T22:48:51.764 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 ::1/128 scope host 2024-05-26T22:48:51.764 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-05-26T22:48:51.764 INFO:teuthology.orchestra.run.smithi187.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-26T22:48:51.765 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 0c:c4:7a:d9:36:78 brd ff:ff:ff:ff:ff:ff 2024-05-26T22:48:51.765 INFO:teuthology.orchestra.run.smithi187.stdout: altname enp7s0f0 2024-05-26T22:48:51.765 INFO:teuthology.orchestra.run.smithi187.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-26T22:48:51.765 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 0c:c4:7a:d9:36:79 brd ff:ff:ff:ff:ff:ff 2024-05-26T22:48:51.765 INFO:teuthology.orchestra.run.smithi187.stdout: altname enp7s0f1 2024-05-26T22:48:51.765 INFO:teuthology.orchestra.run.smithi187.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-26T22:48:51.765 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 0c:c4:7a:88:77:b0 brd ff:ff:ff:ff:ff:ff 2024-05-26T22:48:51.765 INFO:teuthology.orchestra.run.smithi187.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-26T22:48:51.765 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 0c:c4:7a:88:77:b1 brd ff:ff:ff:ff:ff:ff 2024-05-26T22:48:51.765 INFO:teuthology.orchestra.run.smithi187.stdout: inet 172.21.15.187/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-26T22:48:51.765 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft 42553sec preferred_lft 42553sec 2024-05-26T22:48:51.765 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 fe80::ec4:7aff:fe88:77b1/64 scope link 2024-05-26T22:48:51.765 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-05-26T22:48:51.766 INFO:teuthology.orchestra.run.smithi187.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-26T22:48:51.766 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 22:29:cd:a0:00:35 brd ff:ff:ff:ff:ff:ff 2024-05-26T22:48:51.766 INFO:teuthology.orchestra.run.smithi187.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-26T22:48:51.766 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-05-26T22:48:51.766 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 fe80::a496:8bff:fe67:8214/64 scope link 2024-05-26T22:48:51.766 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-05-26T22:48:51.766 INFO:teuthology.orchestra.run.smithi187.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-26T22:48:51.766 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether 22:29:cd:a0:00:35 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-26T22:48:51.766 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 fe80::2029:cdff:fea0:35/64 scope link 2024-05-26T22:48:51.766 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-05-26T22:48:51.767 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:48:51.767 DEBUG:teuthology.orchestra.run.smithi187:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-26T22:48:51.852 INFO:teuthology.orchestra.run.smithi187.stdout:1 2024-05-26T22:48:51.857 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:48:51.857 DEBUG:teuthology.orchestra.run.smithi187:> route 2024-05-26T22:48:51.908 INFO:teuthology.orchestra.run.smithi187.stdout:Kernel IP routing table 2024-05-26T22:48:51.908 INFO:teuthology.orchestra.run.smithi187.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-26T22:48:51.908 INFO:teuthology.orchestra.run.smithi187.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-26T22:48:51.908 INFO:teuthology.orchestra.run.smithi187.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-26T22:48:51.908 INFO:teuthology.orchestra.run.smithi187.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-26T22:48:51.909 DEBUG:teuthology.orchestra.run.smithi187:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-26T22:48:51.909 DEBUG:teuthology.orchestra.run.smithi187:> set -e 2024-05-26T22:48:51.909 DEBUG:teuthology.orchestra.run.smithi187:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-26T22:48:51.909 DEBUG:teuthology.orchestra.run.smithi187:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-26T22:48:51.909 DEBUG:teuthology.orchestra.run.smithi187:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-26T22:48:51.909 DEBUG:teuthology.orchestra.run.smithi187:> ') 2024-05-26T22:48:52.013 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-26T22:48:52.108 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-26T22:48:52.117 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:48:52.117 DEBUG:teuthology.orchestra.run.smithi187:> ip netns list 2024-05-26T22:48:52.165 INFO:teuthology.orchestra.run.smithi187.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-05-26T22:48:52.166 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:48:52.166 DEBUG:teuthology.orchestra.run.smithi187:> ip netns list-id 2024-05-26T22:48:52.216 INFO:teuthology.orchestra.run.smithi187.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-05-26T22:48:52.217 DEBUG:teuthology.orchestra.run.smithi187:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-26T22:48:52.217 DEBUG:teuthology.orchestra.run.smithi187:> set -e 2024-05-26T22:48:52.217 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-26T22:48:52.217 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-05-26T22:48:52.217 DEBUG:teuthology.orchestra.run.smithi187:> ') 2024-05-26T22:48:52.322 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-26T22:48:52.386 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-26T22:48:52.396 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:48:52.396 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-05-26T22:48:52.467 INFO:teuthology.orchestra.run.smithi187.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-26T22:48:52.467 INFO:teuthology.orchestra.run.smithi187.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-26T22:48:52.467 INFO:teuthology.orchestra.run.smithi187.stdout: inet 127.0.0.1/8 scope host lo 2024-05-26T22:48:52.467 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-05-26T22:48:52.467 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 ::1/128 scope host 2024-05-26T22:48:52.467 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-05-26T22:48:52.468 INFO:teuthology.orchestra.run.smithi187.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-26T22:48:52.468 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether e2:d3:4a:d0:7f:ab brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-05-26T22:48:52.468 INFO:teuthology.orchestra.run.smithi187.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-05-26T22:48:52.468 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-05-26T22:48:52.468 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 fe80::e0d3:4aff:fed0:7fab/64 scope link 2024-05-26T22:48:52.468 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-05-26T22:48:52.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:48:52.471 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-05-26T22:48:52.552 INFO:teuthology.orchestra.run.smithi187.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-26T22:48:52.552 INFO:teuthology.orchestra.run.smithi187.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-26T22:48:52.552 INFO:teuthology.orchestra.run.smithi187.stdout: inet 127.0.0.1/8 scope host lo 2024-05-26T22:48:52.552 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-05-26T22:48:52.552 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 ::1/128 scope host 2024-05-26T22:48:52.552 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-05-26T22:48:52.553 INFO:teuthology.orchestra.run.smithi187.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-26T22:48:52.553 INFO:teuthology.orchestra.run.smithi187.stdout: link/ether e2:d3:4a:d0:7f:ab brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-05-26T22:48:52.553 INFO:teuthology.orchestra.run.smithi187.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-05-26T22:48:52.553 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-05-26T22:48:52.553 INFO:teuthology.orchestra.run.smithi187.stdout: inet6 fe80::e0d3:4aff:fed0:7fab/64 scope link 2024-05-26T22:48:52.553 INFO:teuthology.orchestra.run.smithi187.stdout: valid_lft forever preferred_lft forever 2024-05-26T22:48:52.556 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-05-26T22:48:52.557 DEBUG:teuthology.orchestra.run.smithi187:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-26T22:48:52.557 DEBUG:teuthology.orchestra.run.smithi187:> set -e 2024-05-26T22:48:52.557 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-05-26T22:48:52.557 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-05-26T22:48:52.557 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-05-26T22:48:52.557 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-05-26T22:48:52.557 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-05-26T22:48:52.557 DEBUG:teuthology.orchestra.run.smithi187:> ') 2024-05-26T22:48:52.663 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-26T22:48:52.829 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-26T22:48:52.839 DEBUG:teuthology.orchestra.run.smithi187:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-26T22:48:52.839 DEBUG:teuthology.orchestra.run.smithi187:> set -e 2024-05-26T22:48:52.839 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip link set brx.1 up 2024-05-26T22:48:52.839 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip link set dev brx.1 master ceph-brx 2024-05-26T22:48:52.839 DEBUG:teuthology.orchestra.run.smithi187:> ') 2024-05-26T22:48:52.938 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-26T22:48:53.034 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-26T22:48:53.043 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-26T22:48:53.043 DEBUG:teuthology.orchestra.run.smithi187:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-05-26T22:48:53.089 INFO:teuthology.orchestra.run.smithi187.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-05-26T22:48:53.089 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-26T22:48:53.089 DEBUG:teuthology.orchestra.run.smithi187:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-05-26T22:48:53.186 INFO:tasks.cephfs.kernel_mount:mounting using device: 3@.cephfs=/ 2024-05-26T22:48:53.186 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:48:53.186 DEBUG:teuthology.orchestra.run.smithi187:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 3@.cephfs=/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-26T22:48:53.312 INFO:teuthology.orchestra.run.smithi187.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-26T22:48:53.312 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-26T22:48:53.345 INFO:teuthology.orchestra.run.smithi187.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-26T22:48:53.346 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-26T22:48:53.346 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: resolved to: "172.21.15.131:3300,172.21.15.187:3300,172.21.15.187:3301" 2024-05-26T22:48:53.346 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: trying mount with new device syntax: 3@46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f.cephfs=/ 2024-05-26T22:48:53.346 INFO:teuthology.orchestra.run.smithi187.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,ms_mode=prefer-crc,key=3,mon_addr=172.21.15.131:3300/172.21.15.187:3300/172.21.15.187:3301" will pass to kernel 2024-05-26T22:48:53.347 INFO:teuthology.orchestra.run.smithi187.stdout:mount: /home/ubuntu/cephtest/mnt.3 does not contain SELinux labels. 2024-05-26T22:48:53.347 INFO:teuthology.orchestra.run.smithi187.stdout: You just mounted a file system that supports labels which does not 2024-05-26T22:48:53.348 INFO:teuthology.orchestra.run.smithi187.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-26T22:48:53.348 INFO:teuthology.orchestra.run.smithi187.stdout: applications will generate AVC messages and not be allowed access to 2024-05-26T22:48:53.348 INFO:teuthology.orchestra.run.smithi187.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-26T22:48:53.352 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-26T22:48:53.352 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:48:53.352 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-05-26T22:48:53.427 DEBUG:teuthology.orchestra.run.smithi187:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-26T22:48:53.532 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:53 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-26T22:48:53.536 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-26T22:48:53.546 INFO:teuthology.orchestra.run.smithi187.stdout:client4528 2024-05-26T22:48:53.546 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-05-26T22:48:53.546 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-26T22:48:53.597 DEBUG:teuthology.orchestra.run.smithi187:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-26T22:48:53.700 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:53 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-26T22:48:53.705 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-26T22:48:53.715 INFO:teuthology.orchestra.run.smithi187.stdout:client4528 2024-05-26T22:48:53.715 DEBUG:teuthology.orchestra.run.smithi187:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-26T22:48:53.824 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:53 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-26T22:48:53.832 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-26T22:48:53.841 INFO:teuthology.orchestra.run.smithi187.stdout:client4528 2024-05-26T22:48:53.841 DEBUG:teuthology.orchestra.run.smithi187:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f.client4528/status') 2024-05-26T22:48:53.937 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:53 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-26T22:48:53.966 INFO:teuthology.orchestra.run.smithi187.stdout:instance: client.4528 (3)192.168.0.2:0/3122803215 2024-05-26T22:48:53.967 INFO:teuthology.orchestra.run.smithi187.stdout:blocklisted: false 2024-05-26T22:48:53.967 INFO:teuthology.orchestra.run.smithi187.stderr:0+1 records in 2024-05-26T22:48:53.967 INFO:teuthology.orchestra.run.smithi187.stderr:0+1 records out 2024-05-26T22:48:53.967 INFO:teuthology.orchestra.run.smithi187.stderr:69 bytes copied, 9.7014e-05 s, 711 kB/s 2024-05-26T22:48:53.969 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-26T22:48:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-26T22:48:53.979 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-05-26T22:48:53.996 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-26T22:48:54.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.208+0000 7fc527e00640 1 Processor -- start 2024-05-26T22:48:54.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.208+0000 7fc527e00640 1 -- start start 2024-05-26T22:48:54.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.210+0000 7fc527e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fc520071d80 0x7fc520072160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:54.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.210+0000 7fc527e00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5200726a0 con 0x7fc5200ad450 2024-05-26T22:48:54.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.210+0000 7fc527e00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5200b5190 con 0x7fc5200ad8e0 2024-05-26T22:48:54.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.210+0000 7fc527e00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fc52006dee0 con 0x7fc520071d80 2024-05-26T22:48:54.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.210+0000 7fc525400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fc520071d80 0x7fc520072160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:54.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.210+0000 7fc525400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fc520071d80 0x7fc520072160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47148/0 (socket says 172.21.15.131:47148) 2024-05-26T22:48:54.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.210+0000 7fc525400640 1 -- 172.21.15.131:0/3184220590 learned_addr learned my addr 172.21.15.131:0/3184220590 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:54.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.211+0000 7fc525400640 1 -- 172.21.15.131:0/3184220590 >> v1:172.21.15.187:6789/0 conn(0x7fc5200ad8e0 legacy=0x7fc5200b43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:54.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.211+0000 7fc525400640 1 -- 172.21.15.131:0/3184220590 >> v1:172.21.15.131:6789/0 conn(0x7fc5200ad450 legacy=0x7fc52006eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:54.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.211+0000 7fc525400640 1 -- 172.21.15.131:0/3184220590 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc52006e1c0 con 0x7fc520071d80 2024-05-26T22:48:54.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.211+0000 7fc51fe00640 1 -- 172.21.15.131:0/3184220590 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3977584655 0 0) 0x7fc5200726a0 con 0x7fc5200ad450 2024-05-26T22:48:54.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.212+0000 7fc525400640 1 --2- 172.21.15.131:0/3184220590 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc520071d80 0x7fc520072160 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc514009870 tx=0x7fc51402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7dbb586f3694765 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:54.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.212+0000 7fc51fe00640 1 -- 172.21.15.131:0/3184220590 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc514037070 con 0x7fc520071d80 2024-05-26T22:48:54.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.212+0000 7fc51fe00640 1 -- 172.21.15.131:0/3184220590 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc514032440 con 0x7fc520071d80 2024-05-26T22:48:54.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.212+0000 7fc51fe00640 1 -- 172.21.15.131:0/3184220590 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc514032810 con 0x7fc520071d80 2024-05-26T22:48:54.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.213+0000 7fc527e00640 1 -- 172.21.15.131:0/3184220590 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc520071d80 msgr2=0x7fc520072160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:54.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.213+0000 7fc527e00640 1 --2- 172.21.15.131:0/3184220590 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc520071d80 0x7fc520072160 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc514009870 tx=0x7fc51402a210 comp rx=0 tx=0).stop 2024-05-26T22:48:54.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.213+0000 7fc527e00640 1 -- 172.21.15.131:0/3184220590 shutdown_connections 2024-05-26T22:48:54.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.213+0000 7fc527e00640 1 --2- 172.21.15.131:0/3184220590 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc520071d80 0x7fc520072160 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:54.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.213+0000 7fc527e00640 1 -- 172.21.15.131:0/3184220590 >> 172.21.15.131:0/3184220590 conn(0x7fc5200b1e90 msgr2=0x7fc5200b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:54.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.213+0000 7fc527e00640 1 -- 172.21.15.131:0/3184220590 shutdown_connections 2024-05-26T22:48:54.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.213+0000 7fc527e00640 1 -- 172.21.15.131:0/3184220590 wait complete. 2024-05-26T22:48:54.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.215+0000 7fc527e00640 1 Processor -- start 2024-05-26T22:48:54.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.215+0000 7fc527e00640 1 -- start start 2024-05-26T22:48:54.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.216+0000 7fc527e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc520071d80 0x7fc52014ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:54.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.216+0000 7fc527e00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc5200ad450 0x7fc52015b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:54.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.216+0000 7fc527e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc52015bbd0 0x7fc520150370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:54.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.216+0000 7fc527e00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc52015e020 con 0x7fc52015bbd0 2024-05-26T22:48:54.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.216+0000 7fc527e00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc52006e780 con 0x7fc5200ad450 2024-05-26T22:48:54.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.216+0000 7fc527e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc52015e1a0 con 0x7fc520071d80 2024-05-26T22:48:54.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.216+0000 7fc525e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc52015bbd0 0x7fc520150370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:54.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.216+0000 7fc525e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc52015bbd0 0x7fc520150370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47162/0 (socket says 172.21.15.131:47162) 2024-05-26T22:48:54.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.216+0000 7fc525e00640 1 -- 172.21.15.131:0/1681715168 learned_addr learned my addr 172.21.15.131:0/1681715168 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:54.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.217+0000 7fc525400640 1 --2- 172.21.15.131:0/1681715168 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc520071d80 0x7fc52014ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:54.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.217+0000 7fc525e00640 1 -- 172.21.15.131:0/1681715168 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc520071d80 msgr2=0x7fc52014ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:54.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.217+0000 7fc525e00640 1 --2- 172.21.15.131:0/1681715168 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc520071d80 0x7fc52014ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:54.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.217+0000 7fc525e00640 1 -- 172.21.15.131:0/1681715168 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc5200ad450 msgr2=0x7fc52015b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:54.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.217+0000 7fc525e00640 1 --2- 172.21.15.131:0/1681715168 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc5200ad450 0x7fc52015b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:54.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.217+0000 7fc525e00640 1 -- 172.21.15.131:0/1681715168 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5200cd240 con 0x7fc52015bbd0 2024-05-26T22:48:54.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.217+0000 7fc525e00640 1 --2- 172.21.15.131:0/1681715168 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc52015bbd0 0x7fc520150370 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc51800b990 tx=0x7fc51800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:54.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.217+0000 7fc51d600640 1 -- 172.21.15.131:0/1681715168 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc518019070 con 0x7fc52015bbd0 2024-05-26T22:48:54.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.217+0000 7fc51d600640 1 -- 172.21.15.131:0/1681715168 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc51800cba0 con 0x7fc52015bbd0 2024-05-26T22:48:54.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.217+0000 7fc51d600640 1 -- 172.21.15.131:0/1681715168 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc518014440 con 0x7fc52015bbd0 2024-05-26T22:48:54.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.217+0000 7fc527e00640 1 -- 172.21.15.131:0/1681715168 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5200cd520 con 0x7fc52015bbd0 2024-05-26T22:48:54.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.218+0000 7fc527e00640 1 -- 172.21.15.131:0/1681715168 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5200cdae0 con 0x7fc52015bbd0 2024-05-26T22:48:54.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.219+0000 7fc527e00640 1 -- 172.21.15.131:0/1681715168 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5200cdce0 con 0x7fc52015bbd0 2024-05-26T22:48:54.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.220+0000 7fc51d600640 1 -- 172.21.15.131:0/1681715168 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc51801c070 con 0x7fc52015bbd0 2024-05-26T22:48:54.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.221+0000 7fc51d600640 1 --2- 172.21.15.131:0/1681715168 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc50406be30 0x7fc50406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:54.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.221+0000 7fc51d600640 1 -- 172.21.15.131:0/1681715168 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc51808abb0 con 0x7fc52015bbd0 2024-05-26T22:48:54.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.221+0000 7fc525400640 1 --2- 172.21.15.131:0/1681715168 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc50406be30 0x7fc50406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:54.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.222+0000 7fc525400640 1 --2- 172.21.15.131:0/1681715168 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc50406be30 0x7fc50406e2f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc514034000 tx=0x7fc51402e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:54.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.225+0000 7fc51d600640 1 -- 172.21.15.131:0/1681715168 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fc5200cdce0 con 0x7fc52015bbd0 2024-05-26T22:48:54.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.426+0000 7fc527e00640 1 -- 172.21.15.131:0/1681715168 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc5200694e0 con 0x7fc52015bbd0 2024-05-26T22:48:54.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.429+0000 7fc51d600640 1 -- 172.21.15.131:0/1681715168 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16629 (secure 0 0 0) 0x7fc518059560 con 0x7fc52015bbd0 2024-05-26T22:48:54.429 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:48:54.429 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":20,"fsid":"46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f","created":"2024-05-26T22:47:57.288905+0000","modified":"2024-05-26T22:48:22.799298+0000","last_up_change":"2024-05-26T22:48:09.602207+0000","last_in_change":"2024-05-26T22:48:00.687341+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-26T22:48:10.463339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-26T22:48:19.663254+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-26T22:48:20.235751+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"814eaf62-a18d-494b-8e27-fd8fb88adea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6801","nonce":2350940051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6803","nonce":2350940051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6807","nonce":2350940051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6805","nonce":2350940051}]},"public_addr":"172.21.15.131:6801/2350940051","cluster_addr":"172.21.15.131:6803/2350940051","heartbeat_back_addr":"172.21.15.131:6807/2350940051","heartbeat_front_addr":"172.21.15.131:6805/2350940051","state":["exists","up"]},{"osd":1,"uuid":"42e66287-f6c6-4e26-af3c-6dfaa3536d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6825","nonce":1525528247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6827","nonce":1525528247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6831","nonce":1525528247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6829","nonce":1525528247}]},"public_addr":"172.21.15.131:6825/1525528247","cluster_addr":"172.21.15.131:6827/1525528247","heartbeat_back_addr":"172.21.15.131:6831/1525528247","heartbeat_front_addr":"172.21.15.131:6829/1525528247","state":["exists","up"]},{"osd":2,"uuid":"0da34673-04a2-4fd0-b57a-0ff2b679b043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6817","nonce":3627842210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6819","nonce":3627842210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6823","nonce":3627842210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6821","nonce":3627842210}]},"public_addr":"172.21.15.131:6817/3627842210","cluster_addr":"172.21.15.131:6819/3627842210","heartbeat_back_addr":"172.21.15.131:6823/3627842210","heartbeat_front_addr":"172.21.15.131:6821/3627842210","state":["exists","up"]},{"osd":3,"uuid":"cf117a27-d6b2-43d3-ae52-b47988bf8eed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6809","nonce":350790591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6811","nonce":350790591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6815","nonce":350790591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6813","nonce":350790591}]},"public_addr":"172.21.15.131:6809/350790591","cluster_addr":"172.21.15.131:6811/350790591","heartbeat_back_addr":"172.21.15.131:6815/350790591","heartbeat_front_addr":"172.21.15.131:6813/350790591","state":["exists","up"]},{"osd":4,"uuid":"a092ccfc-be89-4a84-88d5-0f781fa715eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6816","nonce":244560995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6819","nonce":244560995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6823","nonce":244560995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6821","nonce":244560995}]},"public_addr":"172.21.15.187:6816/244560995","cluster_addr":"172.21.15.187:6819/244560995","heartbeat_back_addr":"172.21.15.187:6823/244560995","heartbeat_front_addr":"172.21.15.187:6821/244560995","state":["exists","up"]},{"osd":5,"uuid":"7513ba69-dd00-4a35-b190-9e8cc3b2bdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6801","nonce":324619398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6803","nonce":324619398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6812","nonce":324619398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6807","nonce":324619398}]},"public_addr":"172.21.15.187:6801/324619398","cluster_addr":"172.21.15.187:6803/324619398","heartbeat_back_addr":"172.21.15.187:6812/324619398","heartbeat_front_addr":"172.21.15.187:6807/324619398","state":["exists","up"]},{"osd":6,"uuid":"50081810-c093-4d2f-a4d7-b7c695c5400d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6806","nonce":1391347786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6809","nonce":1391347786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6818","nonce":1391347786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6813","nonce":1391347786}]},"public_addr":"172.21.15.187:6806/1391347786","cluster_addr":"172.21.15.187:6809/1391347786","heartbeat_back_addr":"172.21.15.187:6818/1391347786","heartbeat_front_addr":"172.21.15.187:6813/1391347786","state":["exists","up"]},{"osd":7,"uuid":"775dae05-54f5-4ad4-bbce-92863c6ed2ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6825","nonce":1220065622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6827","nonce":1220065622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6831","nonce":1220065622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6829","nonce":1220065622}]},"public_addr":"172.21.15.187:6825/1220065622","cluster_addr":"172.21.15.187:6827/1220065622","heartbeat_back_addr":"172.21.15.187:6831/1220065622","heartbeat_front_addr":"172.21.15.187:6829/1220065622","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:05.983520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.299516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.067323+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.307748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.018742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.258775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.010912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.295355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-26T22:48:54.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.432+0000 7fc527e00640 1 -- 172.21.15.131:0/1681715168 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc50406be30 msgr2=0x7fc50406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:54.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.432+0000 7fc527e00640 1 --2- 172.21.15.131:0/1681715168 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc50406be30 0x7fc50406e2f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc514034000 tx=0x7fc51402e000 comp rx=0 tx=0).stop 2024-05-26T22:48:54.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.432+0000 7fc527e00640 1 -- 172.21.15.131:0/1681715168 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc52015bbd0 msgr2=0x7fc520150370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:54.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.432+0000 7fc527e00640 1 --2- 172.21.15.131:0/1681715168 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc52015bbd0 0x7fc520150370 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc51800b990 tx=0x7fc51800be50 comp rx=0 tx=0).stop 2024-05-26T22:48:54.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.432+0000 7fc527e00640 1 -- 172.21.15.131:0/1681715168 shutdown_connections 2024-05-26T22:48:54.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.432+0000 7fc527e00640 1 --2- 172.21.15.131:0/1681715168 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc50406be30 0x7fc50406e2f0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:54.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.432+0000 7fc527e00640 1 --2- 172.21.15.131:0/1681715168 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc52015bbd0 0x7fc520150370 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:54.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.432+0000 7fc527e00640 1 --2- 172.21.15.131:0/1681715168 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc5200ad450 0x7fc52015b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:54.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.432+0000 7fc527e00640 1 --2- 172.21.15.131:0/1681715168 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc520071d80 0x7fc52014ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:54.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.432+0000 7fc527e00640 1 -- 172.21.15.131:0/1681715168 >> 172.21.15.131:0/1681715168 conn(0x7fc5200b1e90 msgr2=0x7fc52006d7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:54.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.432+0000 7fc527e00640 1 -- 172.21.15.131:0/1681715168 shutdown_connections 2024-05-26T22:48:54.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.432+0000 7fc527e00640 1 -- 172.21.15.131:0/1681715168 wait complete. 2024-05-26T22:48:54.452 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-26T22:48:10.463339+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-26T22:48:19.663254+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-26T22:48:20.235751+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-26T22:48:54.453 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-26T22:48:54.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.656+0000 7f4271600640 1 Processor -- start 2024-05-26T22:48:54.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.656+0000 7f4271600640 1 -- start start 2024-05-26T22:48:54.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.658+0000 7f4271600640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f426c0d0560 0x7f426c072e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:54.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.658+0000 7f4271600640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f426c072540 0x7f426c161ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:54.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.658+0000 7f4271600640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f426c0af8e0 con 0x7f426c06f180 2024-05-26T22:48:54.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.658+0000 7f4271600640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f426c0cfb30 con 0x7f426c0d0560 2024-05-26T22:48:54.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.658+0000 7f4271600640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f426c0cf200 con 0x7f426c072540 2024-05-26T22:48:54.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.658+0000 7f426a000640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f426c0d0560 0x7f426c072e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:54.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.658+0000 7f426a000640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f426c0d0560 0x7f426c072e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43756/0 (socket says 172.21.15.131:43756) 2024-05-26T22:48:54.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.658+0000 7f426a000640 1 -- 172.21.15.131:0/2535795846 learned_addr learned my addr 172.21.15.131:0/2535795846 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:54.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.659+0000 7f426a000640 1 -- 172.21.15.131:0/2535795846 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f426c072540 msgr2=0x7f426c161ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:54.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.659+0000 7f426b400640 1 --2- 172.21.15.131:0/2535795846 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f426c072540 0x7f426c161ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:54.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.659+0000 7f426a000640 1 --2- 172.21.15.131:0/2535795846 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f426c072540 0x7f426c161ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:54.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.659+0000 7f426a000640 1 -- 172.21.15.131:0/2535795846 >> v1:172.21.15.187:6789/0 conn(0x7f426c06f180 legacy=0x7f426c0cfe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:54.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.659+0000 7f426a000640 1 -- 172.21.15.131:0/2535795846 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f426c0cf4e0 con 0x7f426c0d0560 2024-05-26T22:48:54.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.660+0000 7f426a000640 1 --2- 172.21.15.131:0/2535795846 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f426c0d0560 0x7f426c072e90 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f426c0cfcb0 tx=0x7f425802a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8b191164096453d server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:54.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.660+0000 7f4269600640 1 -- 172.21.15.131:0/2535795846 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4258037070 con 0x7f426c0d0560 2024-05-26T22:48:54.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.660+0000 7f4269600640 1 -- 172.21.15.131:0/2535795846 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4258032440 con 0x7f426c0d0560 2024-05-26T22:48:54.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.660+0000 7f4269600640 1 -- 172.21.15.131:0/2535795846 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4258032810 con 0x7f426c0d0560 2024-05-26T22:48:54.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.661+0000 7f4271600640 1 -- 172.21.15.131:0/2535795846 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f426c0d0560 msgr2=0x7f426c072e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:54.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.661+0000 7f4271600640 1 --2- 172.21.15.131:0/2535795846 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f426c0d0560 0x7f426c072e90 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f426c0cfcb0 tx=0x7f425802a200 comp rx=0 tx=0).stop 2024-05-26T22:48:54.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.661+0000 7f4271600640 1 -- 172.21.15.131:0/2535795846 shutdown_connections 2024-05-26T22:48:54.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.661+0000 7f4271600640 1 --2- 172.21.15.131:0/2535795846 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f426c072540 0x7f426c161ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:54.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.661+0000 7f4271600640 1 --2- 172.21.15.131:0/2535795846 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f426c0d0560 0x7f426c072e90 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:54.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.661+0000 7f4271600640 1 -- 172.21.15.131:0/2535795846 >> 172.21.15.131:0/2535795846 conn(0x7f426c06bc90 msgr2=0x7f426c0b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:54.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.661+0000 7f4271600640 1 -- 172.21.15.131:0/2535795846 shutdown_connections 2024-05-26T22:48:54.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.661+0000 7f4271600640 1 -- 172.21.15.131:0/2535795846 wait complete. 2024-05-26T22:48:54.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.662+0000 7f4271600640 1 Processor -- start 2024-05-26T22:48:54.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.663+0000 7f4271600640 1 -- start start 2024-05-26T22:48:54.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.663+0000 7f4271600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f426c06f180 0x7f426c14cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:54.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.664+0000 7f4271600640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f426c072540 0x7f426c0b46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:54.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.664+0000 7f4271600640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f426c0d0560 0x7f426c160c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:54.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.664+0000 7f4271600640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f426c06f6e0 con 0x7f426c06f180 2024-05-26T22:48:54.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.664+0000 7f4271600640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f426c06f560 con 0x7f426c0d0560 2024-05-26T22:48:54.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.664+0000 7f4271600640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f426c0715a0 con 0x7f426c072540 2024-05-26T22:48:54.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.664+0000 7f426aa00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f426c06f180 0x7f426c14cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:54.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.664+0000 7f426aa00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f426c06f180 0x7f426c14cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43772/0 (socket says 172.21.15.131:43772) 2024-05-26T22:48:54.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.664+0000 7f426aa00640 1 -- 172.21.15.131:0/4136049897 learned_addr learned my addr 172.21.15.131:0/4136049897 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:54.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.664+0000 7f426aa00640 1 -- 172.21.15.131:0/4136049897 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f426c072540 msgr2=0x7f426c0b46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:54.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.664+0000 7f426aa00640 1 --2- 172.21.15.131:0/4136049897 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f426c072540 0x7f426c0b46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:54.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.664+0000 7f426aa00640 1 -- 172.21.15.131:0/4136049897 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f426c0d0560 msgr2=0x7f426c160c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:54.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.664+0000 7f426aa00640 1 --2- 172.21.15.131:0/4136049897 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f426c0d0560 0x7f426c160c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:54.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.664+0000 7f426aa00640 1 -- 172.21.15.131:0/4136049897 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f426c14d3f0 con 0x7f426c06f180 2024-05-26T22:48:54.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.664+0000 7f426aa00640 1 --2- 172.21.15.131:0/4136049897 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f426c06f180 0x7f426c14cc50 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f425c00c430 tx=0x7f425c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:54.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.665+0000 7f4262a00640 1 -- 172.21.15.131:0/4136049897 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f425c011790 con 0x7f426c06f180 2024-05-26T22:48:54.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.665+0000 7f4262a00640 1 -- 172.21.15.131:0/4136049897 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f425c017070 con 0x7f426c06f180 2024-05-26T22:48:54.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.665+0000 7f4262a00640 1 -- 172.21.15.131:0/4136049897 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f425c011b70 con 0x7f426c06f180 2024-05-26T22:48:54.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.665+0000 7f4271600640 1 -- 172.21.15.131:0/4136049897 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f426c14d6d0 con 0x7f426c06f180 2024-05-26T22:48:54.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.665+0000 7f4271600640 1 -- 172.21.15.131:0/4136049897 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f426c14dc90 con 0x7f426c06f180 2024-05-26T22:48:54.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.666+0000 7f4271600640 1 -- 172.21.15.131:0/4136049897 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f426c074e90 con 0x7f426c06f180 2024-05-26T22:48:54.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.667+0000 7f4262a00640 1 -- 172.21.15.131:0/4136049897 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f425c011d10 con 0x7f426c06f180 2024-05-26T22:48:54.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.668+0000 7f4262a00640 1 --2- 172.21.15.131:0/4136049897 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f424c06be30 0x7f424c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:54.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.668+0000 7f4262a00640 1 -- 172.21.15.131:0/4136049897 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f425c08ac40 con 0x7f426c06f180 2024-05-26T22:48:54.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.668+0000 7f426a000640 1 --2- 172.21.15.131:0/4136049897 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f424c06be30 0x7f424c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:54.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.669+0000 7f426a000640 1 --2- 172.21.15.131:0/4136049897 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f424c06be30 0x7f424c06e2f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4258034000 tx=0x7f425802c000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:54.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.672+0000 7f4262a00640 1 -- 172.21.15.131:0/4136049897 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f425c059680 con 0x7f426c06f180 2024-05-26T22:48:54.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.864+0000 7f4271600640 1 -- 172.21.15.131:0/4136049897 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f426c0b4200 con 0x7f426c06f180 2024-05-26T22:48:54.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.865+0000 7f4262a00640 1 -- 172.21.15.131:0/4136049897 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f425c058f80 con 0x7f426c06f180 2024-05-26T22:48:54.865 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2024-05-26T22:48:54.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.867+0000 7f4271600640 1 -- 172.21.15.131:0/4136049897 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f424c06be30 msgr2=0x7f424c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:54.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.867+0000 7f4271600640 1 --2- 172.21.15.131:0/4136049897 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f424c06be30 0x7f424c06e2f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4258034000 tx=0x7f425802c000 comp rx=0 tx=0).stop 2024-05-26T22:48:54.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.867+0000 7f4271600640 1 -- 172.21.15.131:0/4136049897 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f426c06f180 msgr2=0x7f426c14cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:54.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.867+0000 7f4271600640 1 --2- 172.21.15.131:0/4136049897 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f426c06f180 0x7f426c14cc50 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f425c00c430 tx=0x7f425c00c8f0 comp rx=0 tx=0).stop 2024-05-26T22:48:54.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.868+0000 7f4271600640 1 -- 172.21.15.131:0/4136049897 shutdown_connections 2024-05-26T22:48:54.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.868+0000 7f4271600640 1 --2- 172.21.15.131:0/4136049897 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f426c072540 0x7f426c0b46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:54.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.868+0000 7f4271600640 1 --2- 172.21.15.131:0/4136049897 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f424c06be30 0x7f424c06e2f0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:54.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.868+0000 7f4271600640 1 --2- 172.21.15.131:0/4136049897 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f426c0d0560 0x7f426c160c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:54.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.868+0000 7f4271600640 1 --2- 172.21.15.131:0/4136049897 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f426c06f180 0x7f426c14cc50 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:54.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.868+0000 7f4271600640 1 -- 172.21.15.131:0/4136049897 >> 172.21.15.131:0/4136049897 conn(0x7f426c06bc90 msgr2=0x7f426c15e890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:54.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.868+0000 7f4271600640 1 -- 172.21.15.131:0/4136049897 shutdown_connections 2024-05-26T22:48:54.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:54.868+0000 7f4271600640 1 -- 172.21.15.131:0/4136049897 wait complete. 2024-05-26T22:48:54.885 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-26T22:48:55.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.080+0000 7f55bae00640 1 Processor -- start 2024-05-26T22:48:55.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.080+0000 7f55bae00640 1 -- start start 2024-05-26T22:48:55.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.082+0000 7f55bae00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55b415e4e0 0x7f55b40b9c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:55.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.082+0000 7f55bae00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f55b40d2330 0x7f55b40cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:55.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.082+0000 7f55bae00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55b40d28f0 con 0x7f55b40cf550 2024-05-26T22:48:55.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.082+0000 7f55bae00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f55b40d2bd0 con 0x7f55b40d2330 2024-05-26T22:48:55.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.082+0000 7f55bae00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f55b40d2eb0 con 0x7f55b415e4e0 2024-05-26T22:48:55.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.082+0000 7f55ab400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f55b40d2330 0x7f55b40cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:55.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.082+0000 7f55ab400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f55b40d2330 0x7f55b40cf010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43782/0 (socket says 172.21.15.131:43782) 2024-05-26T22:48:55.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.082+0000 7f55ab400640 1 -- 172.21.15.131:0/218081496 learned_addr learned my addr 172.21.15.131:0/218081496 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:55.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.082+0000 7f55b8e00640 1 --1- 172.21.15.131:0/218081496 >> v1:172.21.15.131:6789/0 conn(0x7f55b40cf550 0x7f55b40d3c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:48082/0 (socket says 172.21.15.131:48082) 2024-05-26T22:48:55.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.083+0000 7f55b3e00640 1 --2- 172.21.15.131:0/218081496 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55b415e4e0 0x7f55b40b9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:55.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.083+0000 7f55ab400640 1 -- 172.21.15.131:0/218081496 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55b415e4e0 msgr2=0x7f55b40b9c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:55.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.083+0000 7f55ab400640 1 --2- 172.21.15.131:0/218081496 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55b415e4e0 0x7f55b40b9c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:55.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.083+0000 7f55ab400640 1 -- 172.21.15.131:0/218081496 >> v1:172.21.15.131:6789/0 conn(0x7f55b40cf550 legacy=0x7f55b40d3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:55.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.083+0000 7f55ab400640 1 -- 172.21.15.131:0/218081496 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55b40d3190 con 0x7f55b40d2330 2024-05-26T22:48:55.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.084+0000 7f55ab400640 1 --2- 172.21.15.131:0/218081496 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55b40d2330 0x7f55b40cf010 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f559c009870 tx=0x7f559c02a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d4220f1abfa7027 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:55.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.084+0000 7f55b3400640 1 -- 172.21.15.131:0/218081496 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f559c037070 con 0x7f55b40d2330 2024-05-26T22:48:55.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.084+0000 7f55b3400640 1 -- 172.21.15.131:0/218081496 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f559c032440 con 0x7f55b40d2330 2024-05-26T22:48:55.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.084+0000 7f55b3400640 1 -- 172.21.15.131:0/218081496 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f559c032810 con 0x7f55b40d2330 2024-05-26T22:48:55.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.085+0000 7f55bae00640 1 -- 172.21.15.131:0/218081496 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55b40d2330 msgr2=0x7f55b40cf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:55.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.085+0000 7f55bae00640 1 --2- 172.21.15.131:0/218081496 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55b40d2330 0x7f55b40cf010 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f559c009870 tx=0x7f559c02a250 comp rx=0 tx=0).stop 2024-05-26T22:48:55.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.085+0000 7f55bae00640 1 -- 172.21.15.131:0/218081496 shutdown_connections 2024-05-26T22:48:55.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.085+0000 7f55bae00640 1 --2- 172.21.15.131:0/218081496 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55b40d2330 0x7f55b40cf010 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:55.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.085+0000 7f55bae00640 1 --2- 172.21.15.131:0/218081496 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55b415e4e0 0x7f55b40b9c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:55.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.085+0000 7f55bae00640 1 -- 172.21.15.131:0/218081496 >> 172.21.15.131:0/218081496 conn(0x7f55b4072e90 msgr2=0x7f55b40732a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:55.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.085+0000 7f55bae00640 1 -- 172.21.15.131:0/218081496 shutdown_connections 2024-05-26T22:48:55.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.085+0000 7f55bae00640 1 -- 172.21.15.131:0/218081496 wait complete. 2024-05-26T22:48:55.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.087+0000 7f55bae00640 1 Processor -- start 2024-05-26T22:48:55.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.087+0000 7f55bae00640 1 -- start start 2024-05-26T22:48:55.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.088+0000 7f55bae00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55b40cf550 0x7f55b4161570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:55.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.088+0000 7f55bae00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55b40d2330 0x7f55b415f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:55.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.088+0000 7f55bae00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f55b415e4e0 0x7f55b4160200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:55.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.088+0000 7f55bae00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f55b40d3990 con 0x7f55b40d2330 2024-05-26T22:48:55.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.088+0000 7f55bae00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f55b415e8c0 con 0x7f55b415e4e0 2024-05-26T22:48:55.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.088+0000 7f55bae00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f55b40ba1c0 con 0x7f55b40cf550 2024-05-26T22:48:55.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.088+0000 7f55ab400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55b40d2330 0x7f55b415f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:55.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.088+0000 7f55ab400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55b40d2330 0x7f55b415f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43794/0 (socket says 172.21.15.131:43794) 2024-05-26T22:48:55.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.088+0000 7f55ab400640 1 -- 172.21.15.131:0/2683080997 learned_addr learned my addr 172.21.15.131:0/2683080997 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:55.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.088+0000 7f55ab400640 1 -- 172.21.15.131:0/2683080997 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55b40cf550 msgr2=0x7f55b4161570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:55.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.088+0000 7f55ab400640 1 --2- 172.21.15.131:0/2683080997 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55b40cf550 0x7f55b4161570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:55.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.088+0000 7f55ab400640 1 -- 172.21.15.131:0/2683080997 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f55b415e4e0 msgr2=0x7f55b4160200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:55.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.088+0000 7f55ab400640 1 --2- 172.21.15.131:0/2683080997 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f55b415e4e0 0x7f55b4160200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:55.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.088+0000 7f55ab400640 1 -- 172.21.15.131:0/2683080997 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55b40cac50 con 0x7f55b40d2330 2024-05-26T22:48:55.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.089+0000 7f55ab400640 1 --2- 172.21.15.131:0/2683080997 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55b40d2330 0x7f55b415f650 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f559c000c00 tx=0x7f559c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:55.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.089+0000 7f55b0c00640 1 -- 172.21.15.131:0/2683080997 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f559c037040 con 0x7f55b40d2330 2024-05-26T22:48:55.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.089+0000 7f55b0c00640 1 -- 172.21.15.131:0/2683080997 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f559c042070 con 0x7f55b40d2330 2024-05-26T22:48:55.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.089+0000 7f55bae00640 1 -- 172.21.15.131:0/2683080997 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55b41f8290 con 0x7f55b40d2330 2024-05-26T22:48:55.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.089+0000 7f55b0c00640 1 -- 172.21.15.131:0/2683080997 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f559c030bd0 con 0x7f55b40d2330 2024-05-26T22:48:55.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.089+0000 7f55bae00640 1 -- 172.21.15.131:0/2683080997 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55b415eb70 con 0x7f55b40d2330 2024-05-26T22:48:55.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.090+0000 7f55bae00640 1 -- 172.21.15.131:0/2683080997 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55b40695c0 con 0x7f55b40d2330 2024-05-26T22:48:55.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.091+0000 7f55b0c00640 1 -- 172.21.15.131:0/2683080997 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f559c047460 con 0x7f55b40d2330 2024-05-26T22:48:55.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.092+0000 7f55b0c00640 1 --2- 172.21.15.131:0/2683080997 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f559006be30 0x7f559006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:55.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.092+0000 7f55b0c00640 1 -- 172.21.15.131:0/2683080997 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f559c0abb40 con 0x7f55b40d2330 2024-05-26T22:48:55.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.092+0000 7f55b3e00640 1 --2- 172.21.15.131:0/2683080997 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f559006be30 0x7f559006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:55.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.093+0000 7f55b3e00640 1 --2- 172.21.15.131:0/2683080997 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f559006be30 0x7f559006e2f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f55b40cdfb0 tx=0x7f55a4007390 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:55.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.096+0000 7f55b0c00640 1 -- 172.21.15.131:0/2683080997 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f559c045360 con 0x7f55b40d2330 2024-05-26T22:48:55.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.294+0000 7f55bae00640 1 -- 172.21.15.131:0/2683080997 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f55b40cf100 con 0x7f55b40d2330 2024-05-26T22:48:55.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.295+0000 7f55b0c00640 1 -- 172.21.15.131:0/2683080997 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f559c07a450 con 0x7f55b40d2330 2024-05-26T22:48:55.295 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-05-26T22:48:55.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.298+0000 7f55bae00640 1 -- 172.21.15.131:0/2683080997 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f559006be30 msgr2=0x7f559006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:55.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.298+0000 7f55bae00640 1 --2- 172.21.15.131:0/2683080997 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f559006be30 0x7f559006e2f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f55b40cdfb0 tx=0x7f55a4007390 comp rx=0 tx=0).stop 2024-05-26T22:48:55.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.298+0000 7f55bae00640 1 -- 172.21.15.131:0/2683080997 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55b40d2330 msgr2=0x7f55b415f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:55.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.298+0000 7f55bae00640 1 --2- 172.21.15.131:0/2683080997 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55b40d2330 0x7f55b415f650 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f559c000c00 tx=0x7f559c0046c0 comp rx=0 tx=0).stop 2024-05-26T22:48:55.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.298+0000 7f55bae00640 1 -- 172.21.15.131:0/2683080997 shutdown_connections 2024-05-26T22:48:55.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.298+0000 7f55bae00640 1 --2- 172.21.15.131:0/2683080997 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f559006be30 0x7f559006e2f0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:55.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.298+0000 7f55bae00640 1 --2- 172.21.15.131:0/2683080997 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f55b415e4e0 0x7f55b4160200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:55.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.298+0000 7f55bae00640 1 --2- 172.21.15.131:0/2683080997 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55b40d2330 0x7f55b415f650 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:55.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.298+0000 7f55bae00640 1 --2- 172.21.15.131:0/2683080997 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55b40cf550 0x7f55b4161570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:55.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.298+0000 7f55bae00640 1 -- 172.21.15.131:0/2683080997 >> 172.21.15.131:0/2683080997 conn(0x7f55b4072e90 msgr2=0x7f55b40b7510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:55.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.298+0000 7f55bae00640 1 -- 172.21.15.131:0/2683080997 shutdown_connections 2024-05-26T22:48:55.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.298+0000 7f55bae00640 1 -- 172.21.15.131:0/2683080997 wait complete. 2024-05-26T22:48:55.316 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-26T22:48:55.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.517+0000 7f0fdf400640 1 Processor -- start 2024-05-26T22:48:55.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.518+0000 7f0fdf400640 1 -- start start 2024-05-26T22:48:55.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.519+0000 7f0fdf400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f0fd80b5450 0x7f0fd80b43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:55.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.519+0000 7f0fdf400640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f0fd806f420 0x7f0fd80cfd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:55.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.519+0000 7f0fdf400640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0fd80b4900 con 0x7f0fd80cf590 2024-05-26T22:48:55.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.519+0000 7f0fdf400640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f0fd8160d90 con 0x7f0fd80b5450 2024-05-26T22:48:55.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.519+0000 7f0fdf400640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f0fd8160180 con 0x7f0fd806f420 2024-05-26T22:48:55.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.520+0000 7f0fdca00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f0fd80b5450 0x7f0fd80b43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:55.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.520+0000 7f0fdca00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f0fd80b5450 0x7f0fd80b43c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43798/0 (socket says 172.21.15.131:43798) 2024-05-26T22:48:55.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.520+0000 7f0fdca00640 1 -- 172.21.15.131:0/1876129667 learned_addr learned my addr 172.21.15.131:0/1876129667 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:55.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.520+0000 7f0fdca00640 1 -- 172.21.15.131:0/1876129667 >> v2:172.21.15.187:3300/0 conn(0x7f0fd806f420 msgr2=0x7f0fd80cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:55.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.520+0000 7f0fd7e00640 1 --2- 172.21.15.131:0/1876129667 >> v2:172.21.15.187:3300/0 conn(0x7f0fd806f420 0x7f0fd80cfd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:55.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.520+0000 7f0fdca00640 1 --2- 172.21.15.131:0/1876129667 >> v2:172.21.15.187:3300/0 conn(0x7f0fd806f420 0x7f0fd80cfd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:55.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.520+0000 7f0fdca00640 1 -- 172.21.15.131:0/1876129667 >> v1:172.21.15.131:6789/0 conn(0x7f0fd80cf590 legacy=0x7f0fd806d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:55.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.520+0000 7f0fdca00640 1 -- 172.21.15.131:0/1876129667 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fd8160460 con 0x7f0fd80b5450 2024-05-26T22:48:55.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.520+0000 7f0fd7400640 1 -- 172.21.15.131:0/1876129667 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3503027153 0 0) 0x7f0fd80b4900 con 0x7f0fd80cf590 2024-05-26T22:48:55.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.520+0000 7f0fd7e00640 1 --2- 172.21.15.131:0/1876129667 >> v2:172.21.15.187:3300/0 conn(0x7f0fd806f420 0x7f0fd80cfd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:48:55.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.521+0000 7f0fdca00640 1 --2- 172.21.15.131:0/1876129667 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0fd80b5450 0x7f0fd80b43c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0fd80b35b0 tx=0x7f0fcc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8457292b5b453ab8 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:55.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.521+0000 7f0fd7400640 1 -- 172.21.15.131:0/1876129667 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fcc037070 con 0x7f0fd80b5450 2024-05-26T22:48:55.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.521+0000 7f0fd7400640 1 -- 172.21.15.131:0/1876129667 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fcc032440 con 0x7f0fd80b5450 2024-05-26T22:48:55.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.521+0000 7f0fd7400640 1 -- 172.21.15.131:0/1876129667 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fcc032810 con 0x7f0fd80b5450 2024-05-26T22:48:55.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.522+0000 7f0fdf400640 1 -- 172.21.15.131:0/1876129667 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0fd80b5450 msgr2=0x7f0fd80b43c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:55.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.522+0000 7f0fdf400640 1 --2- 172.21.15.131:0/1876129667 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0fd80b5450 0x7f0fd80b43c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0fd80b35b0 tx=0x7f0fcc02a270 comp rx=0 tx=0).stop 2024-05-26T22:48:55.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.522+0000 7f0fdf400640 1 -- 172.21.15.131:0/1876129667 shutdown_connections 2024-05-26T22:48:55.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.522+0000 7f0fdf400640 1 --2- 172.21.15.131:0/1876129667 >> v2:172.21.15.187:3300/0 conn(0x7f0fd806f420 0x7f0fd80cfd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:55.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.522+0000 7f0fdf400640 1 --2- 172.21.15.131:0/1876129667 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0fd80b5450 0x7f0fd80b43c0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:55.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.522+0000 7f0fdf400640 1 -- 172.21.15.131:0/1876129667 >> 172.21.15.131:0/1876129667 conn(0x7f0fd806bc90 msgr2=0x7f0fd80b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:55.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.522+0000 7f0fdf400640 1 -- 172.21.15.131:0/1876129667 shutdown_connections 2024-05-26T22:48:55.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.522+0000 7f0fdf400640 1 -- 172.21.15.131:0/1876129667 wait complete. 2024-05-26T22:48:55.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.524+0000 7f0fdf400640 1 Processor -- start 2024-05-26T22:48:55.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.524+0000 7f0fdf400640 1 -- start start 2024-05-26T22:48:55.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.525+0000 7f0fdf400640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0fd806f420 0x7f0fd8150820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:55.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.525+0000 7f0fdf400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0fd80b5450 0x7f0fd8152e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:55.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.525+0000 7f0fdf400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0fd80cf590 0x7f0fd815a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:55.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.525+0000 7f0fdf400640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f0fd80b5830 con 0x7f0fd80cf590 2024-05-26T22:48:55.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.525+0000 7f0fdf400640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0fd8160a20 con 0x7f0fd806f420 2024-05-26T22:48:55.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.525+0000 7f0fdf400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f0fd815fde0 con 0x7f0fd80b5450 2024-05-26T22:48:55.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.525+0000 7f0fdd400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0fd80cf590 0x7f0fd815a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:55.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.525+0000 7f0fdd400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0fd80cf590 0x7f0fd815a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43804/0 (socket says 172.21.15.131:43804) 2024-05-26T22:48:55.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.525+0000 7f0fdd400640 1 -- 172.21.15.131:0/2532131404 learned_addr learned my addr 172.21.15.131:0/2532131404 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:55.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.526+0000 7f0fdca00640 1 --2- 172.21.15.131:0/2532131404 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0fd806f420 0x7f0fd8150820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:55.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.526+0000 7f0fdd400640 1 -- 172.21.15.131:0/2532131404 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0fd80b5450 msgr2=0x7f0fd8152e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:55.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.526+0000 7f0fdd400640 1 --2- 172.21.15.131:0/2532131404 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0fd80b5450 0x7f0fd8152e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:55.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.526+0000 7f0fdd400640 1 -- 172.21.15.131:0/2532131404 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0fd806f420 msgr2=0x7f0fd8150820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:55.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.526+0000 7f0fdd400640 1 --2- 172.21.15.131:0/2532131404 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0fd806f420 0x7f0fd8150820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:55.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.526+0000 7f0fdd400640 1 -- 172.21.15.131:0/2532131404 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fd81540f0 con 0x7f0fd80cf590 2024-05-26T22:48:55.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.526+0000 7f0fdd400640 1 --2- 172.21.15.131:0/2532131404 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0fd80cf590 0x7f0fd815a4c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0fd000c790 tx=0x7f0fd000cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:55.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.526+0000 7f0fd4c00640 1 -- 172.21.15.131:0/2532131404 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fd0003da0 con 0x7f0fd80cf590 2024-05-26T22:48:55.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.526+0000 7f0fd4c00640 1 -- 172.21.15.131:0/2532131404 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fd0005070 con 0x7f0fd80cf590 2024-05-26T22:48:55.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.526+0000 7f0fd4c00640 1 -- 172.21.15.131:0/2532131404 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fd0005460 con 0x7f0fd80cf590 2024-05-26T22:48:55.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.526+0000 7f0fdf400640 1 -- 172.21.15.131:0/2532131404 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fd8154d40 con 0x7f0fd80cf590 2024-05-26T22:48:55.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.526+0000 7f0fdf400640 1 -- 172.21.15.131:0/2532131404 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fd8155300 con 0x7f0fd80cf590 2024-05-26T22:48:55.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.528+0000 7f0fdf400640 1 -- 172.21.15.131:0/2532131404 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fd8074e90 con 0x7f0fd80cf590 2024-05-26T22:48:55.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.528+0000 7f0fd4c00640 1 -- 172.21.15.131:0/2532131404 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0fd0005600 con 0x7f0fd80cf590 2024-05-26T22:48:55.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.530+0000 7f0fd4c00640 1 --2- 172.21.15.131:0/2532131404 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f0fbc06be30 0x7f0fbc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:55.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.530+0000 7f0fd4c00640 1 -- 172.21.15.131:0/2532131404 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f0fd008ae20 con 0x7f0fd80cf590 2024-05-26T22:48:55.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.530+0000 7f0fdca00640 1 --2- 172.21.15.131:0/2532131404 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f0fbc06be30 0x7f0fbc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:55.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.531+0000 7f0fdca00640 1 --2- 172.21.15.131:0/2532131404 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f0fbc06be30 0x7f0fbc06e2f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc034000 tx=0x7f0fcc02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:55.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.534+0000 7f0fd4c00640 1 -- 172.21.15.131:0/2532131404 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f0fd81540f0 con 0x7f0fd80cf590 2024-05-26T22:48:55.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.717+0000 7f0fdf400640 1 -- 172.21.15.131:0/2532131404 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f0fd80cee90 con 0x7f0fd80cf590 2024-05-26T22:48:55.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.718+0000 7f0fd4c00640 1 -- 172.21.15.131:0/2532131404 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f0fd0059730 con 0x7f0fd80cf590 2024-05-26T22:48:55.717 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 64 2024-05-26T22:48:55.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.720+0000 7f0fdf400640 1 -- 172.21.15.131:0/2532131404 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f0fbc06be30 msgr2=0x7f0fbc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:55.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.720+0000 7f0fdf400640 1 --2- 172.21.15.131:0/2532131404 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f0fbc06be30 0x7f0fbc06e2f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc034000 tx=0x7f0fcc02e000 comp rx=0 tx=0).stop 2024-05-26T22:48:55.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.720+0000 7f0fdf400640 1 -- 172.21.15.131:0/2532131404 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0fd80cf590 msgr2=0x7f0fd815a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:55.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.720+0000 7f0fdf400640 1 --2- 172.21.15.131:0/2532131404 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0fd80cf590 0x7f0fd815a4c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0fd000c790 tx=0x7f0fd000cc50 comp rx=0 tx=0).stop 2024-05-26T22:48:55.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.721+0000 7f0fdf400640 1 -- 172.21.15.131:0/2532131404 shutdown_connections 2024-05-26T22:48:55.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.721+0000 7f0fdf400640 1 --2- 172.21.15.131:0/2532131404 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0fd80b5450 0x7f0fd8152e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:55.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.721+0000 7f0fdf400640 1 --2- 172.21.15.131:0/2532131404 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f0fbc06be30 0x7f0fbc06e2f0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:55.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.721+0000 7f0fdf400640 1 --2- 172.21.15.131:0/2532131404 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0fd80cf590 0x7f0fd815a4c0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:55.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.721+0000 7f0fdf400640 1 --2- 172.21.15.131:0/2532131404 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0fd806f420 0x7f0fd8150820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:55.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.721+0000 7f0fdf400640 1 -- 172.21.15.131:0/2532131404 >> 172.21.15.131:0/2532131404 conn(0x7f0fd806bc90 msgr2=0x7f0fd80cfbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:55.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.721+0000 7f0fdf400640 1 -- 172.21.15.131:0/2532131404 shutdown_connections 2024-05-26T22:48:55.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.721+0000 7f0fdf400640 1 -- 172.21.15.131:0/2532131404 wait complete. 2024-05-26T22:48:55.737 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-26T22:48:55.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.930+0000 7f5ee6400640 1 Processor -- start 2024-05-26T22:48:55.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.930+0000 7f5ee6400640 1 -- start start 2024-05-26T22:48:55.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.931+0000 7f5ee6400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f5ee0155840 0x7f5ee0071930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:55.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.931+0000 7f5ee6400640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f5ee015c3d0 0x7f5ee006ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:55.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.931+0000 7f5ee6400640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ee00722a0 con 0x7f5ee006f490 2024-05-26T22:48:55.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.931+0000 7f5ee6400640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f5ee0072580 con 0x7f5ee0155840 2024-05-26T22:48:55.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.931+0000 7f5ee6400640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f5ee0072860 con 0x7f5ee015c3d0 2024-05-26T22:48:55.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.932+0000 7f5edf400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f5ee0155840 0x7f5ee0071930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:55.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.932+0000 7f5edfe00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f5ee006f490 0x7f5ee015ec80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:48096/0 (socket says 172.21.15.131:48096) 2024-05-26T22:48:55.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.932+0000 7f5edfe00640 1 -- 172.21.15.131:0/237323828 learned_addr learned my addr 172.21.15.131:0/237323828 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:48:55.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.932+0000 7f5edf400640 1 -- 172.21.15.131:0/237323828 >> v2:172.21.15.187:3300/0 conn(0x7f5ee015c3d0 msgr2=0x7f5ee006ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:55.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.932+0000 7f5edf400640 1 --2- 172.21.15.131:0/237323828 >> v2:172.21.15.187:3300/0 conn(0x7f5ee015c3d0 0x7f5ee006ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:55.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.932+0000 7f5edf400640 1 -- 172.21.15.131:0/237323828 >> v1:172.21.15.131:6789/0 conn(0x7f5ee006f490 legacy=0x7f5ee015ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:55.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.932+0000 7f5edf400640 1 -- 172.21.15.131:0/237323828 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ee0072b40 con 0x7f5ee0155840 2024-05-26T22:48:55.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.933+0000 7f5edf400640 1 --2- 172.21.15.131:0/237323828 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ee0155840 0x7f5ee0071930 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5ee0159280 tx=0x7f5ed002c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e51969e83a8e1232 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:55.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.933+0000 7f5ede000640 1 -- 172.21.15.131:0/237323828 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ed0032770 con 0x7f5ee0155840 2024-05-26T22:48:55.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.933+0000 7f5ede000640 1 -- 172.21.15.131:0/237323828 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ed0038070 con 0x7f5ee0155840 2024-05-26T22:48:55.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.933+0000 7f5ede000640 1 -- 172.21.15.131:0/237323828 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ed0032b30 con 0x7f5ee0155840 2024-05-26T22:48:55.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.934+0000 7f5ee6400640 1 -- 172.21.15.131:0/237323828 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ee0155840 msgr2=0x7f5ee0071930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:55.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.934+0000 7f5ee6400640 1 --2- 172.21.15.131:0/237323828 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ee0155840 0x7f5ee0071930 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5ee0159280 tx=0x7f5ed002c870 comp rx=0 tx=0).stop 2024-05-26T22:48:55.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.934+0000 7f5ee6400640 1 -- 172.21.15.131:0/237323828 shutdown_connections 2024-05-26T22:48:55.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.934+0000 7f5ee6400640 1 --2- 172.21.15.131:0/237323828 >> v2:172.21.15.187:3300/0 conn(0x7f5ee015c3d0 0x7f5ee006ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:55.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.934+0000 7f5ee6400640 1 --2- 172.21.15.131:0/237323828 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ee0155840 0x7f5ee0071930 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:55.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.934+0000 7f5ee6400640 1 -- 172.21.15.131:0/237323828 >> 172.21.15.131:0/237323828 conn(0x7f5ee00b1e90 msgr2=0x7f5ee00b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:55.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.934+0000 7f5ee6400640 1 -- 172.21.15.131:0/237323828 shutdown_connections 2024-05-26T22:48:55.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.934+0000 7f5ee6400640 1 -- 172.21.15.131:0/237323828 wait complete. 2024-05-26T22:48:55.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.936+0000 7f5ee6400640 1 Processor -- start 2024-05-26T22:48:55.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.936+0000 7f5ee6400640 1 -- start start 2024-05-26T22:48:55.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.937+0000 7f5ee6400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ee006f490 0x7f5ee01500c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:55.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.937+0000 7f5ee6400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5ee0155840 0x7f5ee0152be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:55.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.937+0000 7f5ee6400640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5ee015c3d0 0x7f5ee01f3720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:55.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.937+0000 7f5ee6400640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f5ee0071ff0 con 0x7f5ee006f490 2024-05-26T22:48:55.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.937+0000 7f5ee6400640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5ee0071e70 con 0x7f5ee015c3d0 2024-05-26T22:48:55.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.937+0000 7f5ee6400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f5ee00aee50 con 0x7f5ee0155840 2024-05-26T22:48:55.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.937+0000 7f5edf400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ee006f490 0x7f5ee01500c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:55.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.937+0000 7f5edf400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ee006f490 0x7f5ee01500c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43812/0 (socket says 172.21.15.131:43812) 2024-05-26T22:48:55.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.937+0000 7f5edf400640 1 -- 172.21.15.131:0/2167429490 learned_addr learned my addr 172.21.15.131:0/2167429490 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:55.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.937+0000 7f5edfe00640 1 --2- 172.21.15.131:0/2167429490 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5ee015c3d0 0x7f5ee01f3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:55.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.937+0000 7f5edf400640 1 -- 172.21.15.131:0/2167429490 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5ee0155840 msgr2=0x7f5ee0152be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:55.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.937+0000 7f5edf400640 1 --2- 172.21.15.131:0/2167429490 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5ee0155840 0x7f5ee0152be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:55.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.937+0000 7f5edf400640 1 -- 172.21.15.131:0/2167429490 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5ee015c3d0 msgr2=0x7f5ee01f3720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:55.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.937+0000 7f5edf400640 1 --2- 172.21.15.131:0/2167429490 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5ee015c3d0 0x7f5ee01f3720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:55.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.937+0000 7f5edf400640 1 -- 172.21.15.131:0/2167429490 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ee00cdef0 con 0x7f5ee006f490 2024-05-26T22:48:55.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.938+0000 7f5edf400640 1 --2- 172.21.15.131:0/2167429490 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ee006f490 0x7f5ee01500c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5ed002ce60 tx=0x7f5ed0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:55.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.938+0000 7f5ed7400640 1 -- 172.21.15.131:0/2167429490 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ed0033c40 con 0x7f5ee006f490 2024-05-26T22:48:55.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.938+0000 7f5ed7400640 1 -- 172.21.15.131:0/2167429490 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ed0032e50 con 0x7f5ee006f490 2024-05-26T22:48:55.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.938+0000 7f5ed7400640 1 -- 172.21.15.131:0/2167429490 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ed0046410 con 0x7f5ee006f490 2024-05-26T22:48:55.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.938+0000 7f5ee6400640 1 -- 172.21.15.131:0/2167429490 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ee00ce190 con 0x7f5ee006f490 2024-05-26T22:48:55.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.938+0000 7f5ee6400640 1 -- 172.21.15.131:0/2167429490 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ee0153120 con 0x7f5ee006f490 2024-05-26T22:48:55.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.939+0000 7f5ee6400640 1 -- 172.21.15.131:0/2167429490 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ee00694e0 con 0x7f5ee006f490 2024-05-26T22:48:55.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.940+0000 7f5ed7400640 1 -- 172.21.15.131:0/2167429490 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5ed0044020 con 0x7f5ee006f490 2024-05-26T22:48:55.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.941+0000 7f5ed7400640 1 --2- 172.21.15.131:0/2167429490 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f5ec006be30 0x7f5ec006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:55.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.941+0000 7f5ed7400640 1 -- 172.21.15.131:0/2167429490 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f5ed00abbc0 con 0x7f5ee006f490 2024-05-26T22:48:55.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.941+0000 7f5edea00640 1 --2- 172.21.15.131:0/2167429490 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f5ec006be30 0x7f5ec006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:55.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.942+0000 7f5edea00640 1 --2- 172.21.15.131:0/2167429490 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f5ec006be30 0x7f5ec006e2f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5ee0150600 tx=0x7f5ecc006bf0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:55.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:55.946+0000 7f5ed7400640 1 -- 172.21.15.131:0/2167429490 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f5ed007a560 con 0x7f5ee006f490 2024-05-26T22:48:56.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.146+0000 7f5ee6400640 1 -- 172.21.15.131:0/2167429490 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5ee00cc5d0 con 0x7f5ee006f490 2024-05-26T22:48:56.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.149+0000 7f5ed7400640 1 -- 172.21.15.131:0/2167429490 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16629 (secure 0 0 0) 0x7f5ed0079e60 con 0x7f5ee006f490 2024-05-26T22:48:56.149 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:48:56.150 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":20,"fsid":"46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f","created":"2024-05-26T22:47:57.288905+0000","modified":"2024-05-26T22:48:22.799298+0000","last_up_change":"2024-05-26T22:48:09.602207+0000","last_in_change":"2024-05-26T22:48:00.687341+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-26T22:48:10.463339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-26T22:48:19.663254+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-26T22:48:20.235751+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"814eaf62-a18d-494b-8e27-fd8fb88adea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6801","nonce":2350940051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6803","nonce":2350940051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6807","nonce":2350940051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6805","nonce":2350940051}]},"public_addr":"172.21.15.131:6801/2350940051","cluster_addr":"172.21.15.131:6803/2350940051","heartbeat_back_addr":"172.21.15.131:6807/2350940051","heartbeat_front_addr":"172.21.15.131:6805/2350940051","state":["exists","up"]},{"osd":1,"uuid":"42e66287-f6c6-4e26-af3c-6dfaa3536d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6825","nonce":1525528247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6827","nonce":1525528247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6831","nonce":1525528247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6829","nonce":1525528247}]},"public_addr":"172.21.15.131:6825/1525528247","cluster_addr":"172.21.15.131:6827/1525528247","heartbeat_back_addr":"172.21.15.131:6831/1525528247","heartbeat_front_addr":"172.21.15.131:6829/1525528247","state":["exists","up"]},{"osd":2,"uuid":"0da34673-04a2-4fd0-b57a-0ff2b679b043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6817","nonce":3627842210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6819","nonce":3627842210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6823","nonce":3627842210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6821","nonce":3627842210}]},"public_addr":"172.21.15.131:6817/3627842210","cluster_addr":"172.21.15.131:6819/3627842210","heartbeat_back_addr":"172.21.15.131:6823/3627842210","heartbeat_front_addr":"172.21.15.131:6821/3627842210","state":["exists","up"]},{"osd":3,"uuid":"cf117a27-d6b2-43d3-ae52-b47988bf8eed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6809","nonce":350790591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6811","nonce":350790591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6815","nonce":350790591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6813","nonce":350790591}]},"public_addr":"172.21.15.131:6809/350790591","cluster_addr":"172.21.15.131:6811/350790591","heartbeat_back_addr":"172.21.15.131:6815/350790591","heartbeat_front_addr":"172.21.15.131:6813/350790591","state":["exists","up"]},{"osd":4,"uuid":"a092ccfc-be89-4a84-88d5-0f781fa715eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6816","nonce":244560995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6819","nonce":244560995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6823","nonce":244560995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6821","nonce":244560995}]},"public_addr":"172.21.15.187:6816/244560995","cluster_addr":"172.21.15.187:6819/244560995","heartbeat_back_addr":"172.21.15.187:6823/244560995","heartbeat_front_addr":"172.21.15.187:6821/244560995","state":["exists","up"]},{"osd":5,"uuid":"7513ba69-dd00-4a35-b190-9e8cc3b2bdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6801","nonce":324619398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6803","nonce":324619398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6812","nonce":324619398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6807","nonce":324619398}]},"public_addr":"172.21.15.187:6801/324619398","cluster_addr":"172.21.15.187:6803/324619398","heartbeat_back_addr":"172.21.15.187:6812/324619398","heartbeat_front_addr":"172.21.15.187:6807/324619398","state":["exists","up"]},{"osd":6,"uuid":"50081810-c093-4d2f-a4d7-b7c695c5400d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6806","nonce":1391347786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6809","nonce":1391347786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6818","nonce":1391347786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6813","nonce":1391347786}]},"public_addr":"172.21.15.187:6806/1391347786","cluster_addr":"172.21.15.187:6809/1391347786","heartbeat_back_addr":"172.21.15.187:6818/1391347786","heartbeat_front_addr":"172.21.15.187:6813/1391347786","state":["exists","up"]},{"osd":7,"uuid":"775dae05-54f5-4ad4-bbce-92863c6ed2ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6825","nonce":1220065622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6827","nonce":1220065622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6831","nonce":1220065622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6829","nonce":1220065622}]},"public_addr":"172.21.15.187:6825/1220065622","cluster_addr":"172.21.15.187:6827/1220065622","heartbeat_back_addr":"172.21.15.187:6831/1220065622","heartbeat_front_addr":"172.21.15.187:6829/1220065622","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:05.983520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.299516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.067323+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.307748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.018742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.258775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.010912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.295355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-26T22:48:56.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.152+0000 7f5ee6400640 1 -- 172.21.15.131:0/2167429490 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f5ec006be30 msgr2=0x7f5ec006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:56.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.152+0000 7f5ee6400640 1 --2- 172.21.15.131:0/2167429490 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f5ec006be30 0x7f5ec006e2f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5ee0150600 tx=0x7f5ecc006bf0 comp rx=0 tx=0).stop 2024-05-26T22:48:56.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.152+0000 7f5ee6400640 1 -- 172.21.15.131:0/2167429490 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ee006f490 msgr2=0x7f5ee01500c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:56.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.152+0000 7f5ee6400640 1 --2- 172.21.15.131:0/2167429490 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ee006f490 0x7f5ee01500c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5ed002ce60 tx=0x7f5ed0004730 comp rx=0 tx=0).stop 2024-05-26T22:48:56.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.152+0000 7f5ee6400640 1 -- 172.21.15.131:0/2167429490 shutdown_connections 2024-05-26T22:48:56.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.152+0000 7f5ee6400640 1 --2- 172.21.15.131:0/2167429490 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5ee0155840 0x7f5ee0152be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:56.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.152+0000 7f5ee6400640 1 --2- 172.21.15.131:0/2167429490 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f5ec006be30 0x7f5ec006e2f0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:56.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.152+0000 7f5ee6400640 1 --2- 172.21.15.131:0/2167429490 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5ee015c3d0 0x7f5ee01f3720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:56.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.152+0000 7f5ee6400640 1 --2- 172.21.15.131:0/2167429490 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ee006f490 0x7f5ee01500c0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:56.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.152+0000 7f5ee6400640 1 -- 172.21.15.131:0/2167429490 >> 172.21.15.131:0/2167429490 conn(0x7f5ee00b1e90 msgr2=0x7f5ee00afef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:56.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.152+0000 7f5ee6400640 1 -- 172.21.15.131:0/2167429490 shutdown_connections 2024-05-26T22:48:56.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.152+0000 7f5ee6400640 1 -- 172.21.15.131:0/2167429490 wait complete. 2024-05-26T22:48:56.171 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-26T22:48:10.463339+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-26T22:48:19.663254+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-26T22:48:20.235751+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-26T22:48:56.171 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-26T22:48:56.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.370+0000 7fc709c00640 1 Processor -- start 2024-05-26T22:48:56.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.370+0000 7fc709c00640 1 -- start start 2024-05-26T22:48:56.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.371+0000 7fc709c00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fc70415bfa0 0x7fc704071500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:56.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.371+0000 7fc709c00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fc70406d960 0x7fc704071a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:56.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.371+0000 7fc709c00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7041552d0 con 0x7fc70415dc40 2024-05-26T22:48:56.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.371+0000 7fc709c00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fc7040b3f30 con 0x7fc70406d960 2024-05-26T22:48:56.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.371+0000 7fc709c00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fc704071f80 con 0x7fc70415bfa0 2024-05-26T22:48:56.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.372+0000 7fc703e00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fc70415dc40 0x7fc70406d250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:48100/0 (socket says 172.21.15.131:48100) 2024-05-26T22:48:56.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.372+0000 7fc703e00640 1 -- 172.21.15.131:0/222660229 learned_addr learned my addr 172.21.15.131:0/222660229 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:48:56.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.372+0000 7fc709200640 1 --2- 172.21.15.131:0/222660229 >> v2:172.21.15.131:3300/0 conn(0x7fc70406d960 0x7fc704071a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:56.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.372+0000 7fc6fb400640 1 --2- 172.21.15.131:0/222660229 >> v2:172.21.15.187:3300/0 conn(0x7fc70415bfa0 0x7fc704071500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:56.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.372+0000 7fc703400640 1 -- 172.21.15.131:0/222660229 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1269711694 0 0) 0x7fc7041552d0 con 0x7fc70415dc40 2024-05-26T22:48:56.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.372+0000 7fc703400640 1 -- 172.21.15.131:0/222660229 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc6e8003520 con 0x7fc70415dc40 2024-05-26T22:48:56.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.372+0000 7fc709200640 1 -- 172.21.15.131:0/222660229 >> v2:172.21.15.187:3300/0 conn(0x7fc70415bfa0 msgr2=0x7fc704071500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:56.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.372+0000 7fc709200640 1 --2- 172.21.15.131:0/222660229 >> v2:172.21.15.187:3300/0 conn(0x7fc70415bfa0 0x7fc704071500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:56.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.372+0000 7fc709200640 1 -- 172.21.15.131:0/222660229 >> v1:172.21.15.131:6789/0 conn(0x7fc70415dc40 legacy=0x7fc70406d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:56.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.372+0000 7fc709200640 1 -- 172.21.15.131:0/222660229 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc704072230 con 0x7fc70406d960 2024-05-26T22:48:56.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.372+0000 7fc6fb400640 1 --2- 172.21.15.131:0/222660229 >> v2:172.21.15.187:3300/0 conn(0x7fc70415bfa0 0x7fc704071500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:48:56.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.373+0000 7fc709200640 1 --2- 172.21.15.131:0/222660229 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc70406d960 0x7fc704071a40 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc009d90 tx=0x7fc6fc02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25c3c81299f9f6e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:56.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.374+0000 7fc703400640 1 -- 172.21.15.131:0/222660229 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6fc007740 con 0x7fc70406d960 2024-05-26T22:48:56.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.374+0000 7fc703400640 1 -- 172.21.15.131:0/222660229 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6fc0078e0 con 0x7fc70406d960 2024-05-26T22:48:56.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.374+0000 7fc703400640 1 -- 172.21.15.131:0/222660229 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6fc007cb0 con 0x7fc70406d960 2024-05-26T22:48:56.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.374+0000 7fc709c00640 1 -- 172.21.15.131:0/222660229 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc70406d960 msgr2=0x7fc704071a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:56.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.374+0000 7fc709c00640 1 --2- 172.21.15.131:0/222660229 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc70406d960 0x7fc704071a40 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc009d90 tx=0x7fc6fc02a6a0 comp rx=0 tx=0).stop 2024-05-26T22:48:56.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.374+0000 7fc709c00640 1 -- 172.21.15.131:0/222660229 shutdown_connections 2024-05-26T22:48:56.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.374+0000 7fc709c00640 1 --2- 172.21.15.131:0/222660229 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc70406d960 0x7fc704071a40 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:56.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.374+0000 7fc709c00640 1 --2- 172.21.15.131:0/222660229 >> v2:172.21.15.187:3300/0 conn(0x7fc70415bfa0 0x7fc704071500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:56.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.374+0000 7fc709c00640 1 -- 172.21.15.131:0/222660229 >> 172.21.15.131:0/222660229 conn(0x7fc7040b5660 msgr2=0x7fc7040b5a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:56.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.374+0000 7fc709c00640 1 -- 172.21.15.131:0/222660229 shutdown_connections 2024-05-26T22:48:56.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.374+0000 7fc709c00640 1 -- 172.21.15.131:0/222660229 wait complete. 2024-05-26T22:48:56.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.376+0000 7fc709c00640 1 Processor -- start 2024-05-26T22:48:56.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.376+0000 7fc709c00640 1 -- start start 2024-05-26T22:48:56.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.377+0000 7fc709c00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc70406d960 0x7fc704153e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:56.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.377+0000 7fc709c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc70415bfa0 0x7fc7041438c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:56.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.377+0000 7fc709c00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc70415dc40 0x7fc704143e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:56.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.377+0000 7fc709c00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc70415e020 con 0x7fc70415bfa0 2024-05-26T22:48:56.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.377+0000 7fc709c00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc7040b3bc0 con 0x7fc70406d960 2024-05-26T22:48:56.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.377+0000 7fc709c00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc70415e1a0 con 0x7fc70415dc40 2024-05-26T22:48:56.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.377+0000 7fc6fb400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc70415bfa0 0x7fc7041438c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:56.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.377+0000 7fc6fb400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc70415bfa0 0x7fc7041438c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43840/0 (socket says 172.21.15.131:43840) 2024-05-26T22:48:56.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.377+0000 7fc6fb400640 1 -- 172.21.15.131:0/538364909 learned_addr learned my addr 172.21.15.131:0/538364909 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:56.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.378+0000 7fc6fb400640 1 -- 172.21.15.131:0/538364909 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc70415dc40 msgr2=0x7fc704143e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:56.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.378+0000 7fc6fb400640 1 --2- 172.21.15.131:0/538364909 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc70415dc40 0x7fc704143e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:56.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.378+0000 7fc6fb400640 1 -- 172.21.15.131:0/538364909 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc70406d960 msgr2=0x7fc704153e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:56.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.378+0000 7fc703e00640 1 --2- 172.21.15.131:0/538364909 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc70406d960 0x7fc704153e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:56.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.378+0000 7fc6fb400640 1 --2- 172.21.15.131:0/538364909 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc70406d960 0x7fc704153e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:56.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.378+0000 7fc6fb400640 1 -- 172.21.15.131:0/538364909 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc70414cde0 con 0x7fc70415bfa0 2024-05-26T22:48:56.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.378+0000 7fc6fb400640 1 --2- 172.21.15.131:0/538364909 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc70415bfa0 0x7fc7041438c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc6ec00c7f0 tx=0x7fc6ec00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:56.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.378+0000 7fc700c00640 1 -- 172.21.15.131:0/538364909 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6ec015070 con 0x7fc70415bfa0 2024-05-26T22:48:56.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.378+0000 7fc700c00640 1 -- 172.21.15.131:0/538364909 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6ec007a30 con 0x7fc70415bfa0 2024-05-26T22:48:56.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.378+0000 7fc700c00640 1 -- 172.21.15.131:0/538364909 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6ec007e00 con 0x7fc70415bfa0 2024-05-26T22:48:56.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.378+0000 7fc709c00640 1 -- 172.21.15.131:0/538364909 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc70414c020 con 0x7fc70415bfa0 2024-05-26T22:48:56.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.379+0000 7fc709c00640 1 -- 172.21.15.131:0/538364909 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc70414c5e0 con 0x7fc70415bfa0 2024-05-26T22:48:56.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.380+0000 7fc709c00640 1 -- 172.21.15.131:0/538364909 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc70414cad0 con 0x7fc70415bfa0 2024-05-26T22:48:56.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.380+0000 7fc700c00640 1 -- 172.21.15.131:0/538364909 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc6ec018070 con 0x7fc70415bfa0 2024-05-26T22:48:56.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.382+0000 7fc700c00640 1 --2- 172.21.15.131:0/538364909 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc6e006be30 0x7fc6e006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:56.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.382+0000 7fc700c00640 1 -- 172.21.15.131:0/538364909 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc6ec08ae30 con 0x7fc70415bfa0 2024-05-26T22:48:56.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.382+0000 7fc703e00640 1 --2- 172.21.15.131:0/538364909 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc6e006be30 0x7fc6e006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:56.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.383+0000 7fc703e00640 1 --2- 172.21.15.131:0/538364909 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc6e006be30 0x7fc6e006e2f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc6f4000f60 tx=0x7fc6f4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:56.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.386+0000 7fc700c00640 1 -- 172.21.15.131:0/538364909 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fc6ec0597d0 con 0x7fc70415bfa0 2024-05-26T22:48:56.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.584+0000 7fc709c00640 1 -- 172.21.15.131:0/538364909 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc7040694e0 con 0x7fc70415bfa0 2024-05-26T22:48:56.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.585+0000 7fc700c00640 1 -- 172.21.15.131:0/538364909 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7fc6ec0590d0 con 0x7fc70415bfa0 2024-05-26T22:48:56.585 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2024-05-26T22:48:56.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.588+0000 7fc709c00640 1 -- 172.21.15.131:0/538364909 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc6e006be30 msgr2=0x7fc6e006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:56.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.588+0000 7fc709c00640 1 --2- 172.21.15.131:0/538364909 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc6e006be30 0x7fc6e006e2f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc6f4000f60 tx=0x7fc6f4009040 comp rx=0 tx=0).stop 2024-05-26T22:48:56.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.588+0000 7fc709c00640 1 -- 172.21.15.131:0/538364909 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc70415bfa0 msgr2=0x7fc7041438c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:56.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.588+0000 7fc709c00640 1 --2- 172.21.15.131:0/538364909 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc70415bfa0 0x7fc7041438c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc6ec00c7f0 tx=0x7fc6ec00cbf0 comp rx=0 tx=0).stop 2024-05-26T22:48:56.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.588+0000 7fc709c00640 1 -- 172.21.15.131:0/538364909 shutdown_connections 2024-05-26T22:48:56.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.588+0000 7fc709c00640 1 --2- 172.21.15.131:0/538364909 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc70415dc40 0x7fc704143e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:56.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.588+0000 7fc709c00640 1 --2- 172.21.15.131:0/538364909 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc6e006be30 0x7fc6e006e2f0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:56.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.588+0000 7fc709c00640 1 --2- 172.21.15.131:0/538364909 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc70415bfa0 0x7fc7041438c0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:56.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.588+0000 7fc709c00640 1 --2- 172.21.15.131:0/538364909 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc70406d960 0x7fc704153e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:56.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.588+0000 7fc709c00640 1 -- 172.21.15.131:0/538364909 >> 172.21.15.131:0/538364909 conn(0x7fc7040b5660 msgr2=0x7fc7040b5a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:56.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.588+0000 7fc709c00640 1 -- 172.21.15.131:0/538364909 shutdown_connections 2024-05-26T22:48:56.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.588+0000 7fc709c00640 1 -- 172.21.15.131:0/538364909 wait complete. 2024-05-26T22:48:56.606 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-26T22:48:56.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.808+0000 7f536d200640 1 Processor -- start 2024-05-26T22:48:56.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.808+0000 7f536d200640 1 -- start start 2024-05-26T22:48:56.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.809+0000 7f536d200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f5368155770 0x7f536815b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:56.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.809+0000 7f536d200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f536815bc40 0x7f536806dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:56.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.810+0000 7f536d200640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f536806e270 0x7f53680ceba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:56.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.810+0000 7f536d200640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f536806e6a0 con 0x7f5368155770 2024-05-26T22:48:56.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.810+0000 7f536d200640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f536815c1c0 con 0x7f536806e270 2024-05-26T22:48:56.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.810+0000 7f536d200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f536815c4a0 con 0x7f536815bc40 2024-05-26T22:48:56.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.810+0000 7f5365e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f5368155770 0x7f536815b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:56.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.810+0000 7f5365e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f5368155770 0x7f536815b700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43850/0 (socket says 172.21.15.131:43850) 2024-05-26T22:48:56.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.810+0000 7f5365e00640 1 -- 172.21.15.131:0/1989006079 learned_addr learned my addr 172.21.15.131:0/1989006079 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:56.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.810+0000 7f5365400640 1 --2- 172.21.15.131:0/1989006079 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f536815bc40 0x7f536806dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:56.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.811+0000 7f5366800640 1 --2- 172.21.15.131:0/1989006079 >> v2:172.21.15.187:3300/0 conn(0x7f536806e270 0x7f53680ceba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:56.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.811+0000 7f5365e00640 1 -- 172.21.15.131:0/1989006079 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f536815bc40 msgr2=0x7f536806dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:56.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.811+0000 7f5365e00640 1 --2- 172.21.15.131:0/1989006079 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f536815bc40 0x7f536806dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:56.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.811+0000 7f5365e00640 1 -- 172.21.15.131:0/1989006079 >> v2:172.21.15.187:3300/0 conn(0x7f536806e270 msgr2=0x7f53680ceba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:56.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.811+0000 7f5365e00640 1 --2- 172.21.15.131:0/1989006079 >> v2:172.21.15.187:3300/0 conn(0x7f536806e270 0x7f53680ceba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:56.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.811+0000 7f5365e00640 1 -- 172.21.15.131:0/1989006079 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5368071da0 con 0x7f5368155770 2024-05-26T22:48:56.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.812+0000 7f5365e00640 1 --2- 172.21.15.131:0/1989006079 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5368155770 0x7f536815b700 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f53680d0ed0 tx=0x7f535802c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3da2bce417e6cee server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:56.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.812+0000 7f5364a00640 1 -- 172.21.15.131:0/1989006079 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5358032430 con 0x7f5368155770 2024-05-26T22:48:56.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.812+0000 7f5364a00640 1 -- 172.21.15.131:0/1989006079 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5358037070 con 0x7f5368155770 2024-05-26T22:48:56.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.812+0000 7f5364a00640 1 -- 172.21.15.131:0/1989006079 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53580327f0 con 0x7f5368155770 2024-05-26T22:48:56.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.813+0000 7f536d200640 1 -- 172.21.15.131:0/1989006079 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5368155770 msgr2=0x7f536815b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:56.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.813+0000 7f536d200640 1 --2- 172.21.15.131:0/1989006079 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5368155770 0x7f536815b700 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f53680d0ed0 tx=0x7f535802c940 comp rx=0 tx=0).stop 2024-05-26T22:48:56.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.813+0000 7f536d200640 1 -- 172.21.15.131:0/1989006079 shutdown_connections 2024-05-26T22:48:56.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.813+0000 7f536d200640 1 --2- 172.21.15.131:0/1989006079 >> v2:172.21.15.187:3300/0 conn(0x7f536806e270 0x7f53680ceba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:56.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.813+0000 7f536d200640 1 --2- 172.21.15.131:0/1989006079 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f536815bc40 0x7f536806dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:56.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.813+0000 7f536d200640 1 --2- 172.21.15.131:0/1989006079 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5368155770 0x7f536815b700 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:56.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.813+0000 7f536d200640 1 -- 172.21.15.131:0/1989006079 >> 172.21.15.131:0/1989006079 conn(0x7f536806bc90 msgr2=0x7f53680b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:56.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.813+0000 7f536d200640 1 -- 172.21.15.131:0/1989006079 shutdown_connections 2024-05-26T22:48:56.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.813+0000 7f536d200640 1 -- 172.21.15.131:0/1989006079 wait complete. 2024-05-26T22:48:56.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.814+0000 7f536d200640 1 Processor -- start 2024-05-26T22:48:56.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.815+0000 7f536d200640 1 -- start start 2024-05-26T22:48:56.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.816+0000 7f536d200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f536806e270 0x7f536814ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:56.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.816+0000 7f536d200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5368155770 0x7f536814ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:56.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.816+0000 7f536d200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f536815bc40 0x7f53681526a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:56.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.816+0000 7f536d200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f536815c8c0 con 0x7f536806e270 2024-05-26T22:48:56.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.816+0000 7f536d200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5368155b50 con 0x7f5368155770 2024-05-26T22:48:56.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.816+0000 7f536d200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f536809de00 con 0x7f536815bc40 2024-05-26T22:48:56.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.816+0000 7f5365e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f536806e270 0x7f536814ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:56.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.816+0000 7f5365e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f536806e270 0x7f536814ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43856/0 (socket says 172.21.15.131:43856) 2024-05-26T22:48:56.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.816+0000 7f5365e00640 1 -- 172.21.15.131:0/4040356188 learned_addr learned my addr 172.21.15.131:0/4040356188 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:56.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.816+0000 7f5365e00640 1 -- 172.21.15.131:0/4040356188 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f536815bc40 msgr2=0x7f53681526a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:56.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.816+0000 7f5365e00640 1 --2- 172.21.15.131:0/4040356188 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f536815bc40 0x7f53681526a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:56.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.816+0000 7f5365e00640 1 -- 172.21.15.131:0/4040356188 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5368155770 msgr2=0x7f536814ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:56.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.816+0000 7f5365e00640 1 --2- 172.21.15.131:0/4040356188 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5368155770 0x7f536814ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:56.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.816+0000 7f5365e00640 1 -- 172.21.15.131:0/4040356188 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f536814d4b0 con 0x7f536806e270 2024-05-26T22:48:56.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.816+0000 7f5365e00640 1 --2- 172.21.15.131:0/4040356188 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f536806e270 0x7f536814ce80 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5358009f40 tx=0x7f5358004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:56.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.817+0000 7f535e000640 1 -- 172.21.15.131:0/4040356188 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5358041070 con 0x7f536806e270 2024-05-26T22:48:56.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.817+0000 7f535e000640 1 -- 172.21.15.131:0/4040356188 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f535803b850 con 0x7f536806e270 2024-05-26T22:48:56.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.817+0000 7f535e000640 1 -- 172.21.15.131:0/4040356188 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5358009510 con 0x7f536806e270 2024-05-26T22:48:56.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.817+0000 7f536d200640 1 -- 172.21.15.131:0/4040356188 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f536814d790 con 0x7f536806e270 2024-05-26T22:48:56.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.817+0000 7f536d200640 1 -- 172.21.15.131:0/4040356188 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f536814dd50 con 0x7f536806e270 2024-05-26T22:48:56.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.818+0000 7f536d200640 1 -- 172.21.15.131:0/4040356188 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5368074e90 con 0x7f536806e270 2024-05-26T22:48:56.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.819+0000 7f535e000640 1 -- 172.21.15.131:0/4040356188 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5358005d10 con 0x7f536806e270 2024-05-26T22:48:56.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.820+0000 7f535e000640 1 --2- 172.21.15.131:0/4040356188 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f534806be30 0x7f534806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:56.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.820+0000 7f535e000640 1 -- 172.21.15.131:0/4040356188 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f53580ab780 con 0x7f536806e270 2024-05-26T22:48:56.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.820+0000 7f5365400640 1 --2- 172.21.15.131:0/4040356188 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f534806be30 0x7f534806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:56.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.821+0000 7f5365400640 1 --2- 172.21.15.131:0/4040356188 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f534806be30 0x7f534806e2f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5354006fd0 tx=0x7f5354008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:56.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:56.824+0000 7f535e000640 1 -- 172.21.15.131:0/4040356188 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f535804b030 con 0x7f536806e270 2024-05-26T22:48:57.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.005+0000 7f536d200640 1 -- 172.21.15.131:0/4040356188 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f5368154da0 con 0x7f536806e270 2024-05-26T22:48:57.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.005+0000 7f535e000640 1 -- 172.21.15.131:0/4040356188 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f535807a090 con 0x7f536806e270 2024-05-26T22:48:57.005 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-05-26T22:48:57.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.008+0000 7f536d200640 1 -- 172.21.15.131:0/4040356188 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f534806be30 msgr2=0x7f534806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:57.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.008+0000 7f536d200640 1 --2- 172.21.15.131:0/4040356188 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f534806be30 0x7f534806e2f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5354006fd0 tx=0x7f5354008040 comp rx=0 tx=0).stop 2024-05-26T22:48:57.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.008+0000 7f536d200640 1 -- 172.21.15.131:0/4040356188 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f536806e270 msgr2=0x7f536814ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:57.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.008+0000 7f536d200640 1 --2- 172.21.15.131:0/4040356188 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f536806e270 0x7f536814ce80 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5358009f40 tx=0x7f5358004a40 comp rx=0 tx=0).stop 2024-05-26T22:48:57.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.008+0000 7f536d200640 1 -- 172.21.15.131:0/4040356188 shutdown_connections 2024-05-26T22:48:57.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.008+0000 7f536d200640 1 --2- 172.21.15.131:0/4040356188 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f536815bc40 0x7f53681526a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:57.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.008+0000 7f536d200640 1 --2- 172.21.15.131:0/4040356188 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f534806be30 0x7f534806e2f0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:57.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.008+0000 7f536d200640 1 --2- 172.21.15.131:0/4040356188 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5368155770 0x7f536814ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:57.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.008+0000 7f536d200640 1 --2- 172.21.15.131:0/4040356188 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f536806e270 0x7f536814ce80 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:57.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.008+0000 7f536d200640 1 -- 172.21.15.131:0/4040356188 >> 172.21.15.131:0/4040356188 conn(0x7f536806bc90 msgr2=0x7f53680b5280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:57.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.008+0000 7f536d200640 1 -- 172.21.15.131:0/4040356188 shutdown_connections 2024-05-26T22:48:57.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.008+0000 7f536d200640 1 -- 172.21.15.131:0/4040356188 wait complete. 2024-05-26T22:48:57.027 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-26T22:48:57.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.226+0000 7fbb8a000640 1 Processor -- start 2024-05-26T22:48:57.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.226+0000 7fbb8a000640 1 -- start start 2024-05-26T22:48:57.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.227+0000 7fbb8a000640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fbb8415dc40 0x7fbb8406d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:57.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.228+0000 7fbb8a000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbb84071c10 0x7fbb8406d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:57.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.228+0000 7fbb8a000640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb841552d0 con 0x7fbb8415bfa0 2024-05-26T22:48:57.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.228+0000 7fbb8a000640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fbb840b3f30 con 0x7fbb8415dc40 2024-05-26T22:48:57.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.228+0000 7fbb8a000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fbb84072230 con 0x7fbb84071c10 2024-05-26T22:48:57.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.228+0000 7fbb83e00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fbb8415bfa0 0x7fbb84071500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:48104/0 (socket says 172.21.15.131:48104) 2024-05-26T22:48:57.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.228+0000 7fbb83e00640 1 -- 172.21.15.131:0/3428367862 learned_addr learned my addr 172.21.15.131:0/3428367862 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:48:57.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.228+0000 7fbb88c00640 1 --2- 172.21.15.131:0/3428367862 >> v2:172.21.15.187:3300/0 conn(0x7fbb8415dc40 0x7fbb8406d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:57.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.228+0000 7fbb89600640 1 --2- 172.21.15.131:0/3428367862 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbb84071c10 0x7fbb8406d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:57.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.229+0000 7fbb88c00640 1 -- 172.21.15.131:0/3428367862 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbb84071c10 msgr2=0x7fbb8406d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:57.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.229+0000 7fbb88c00640 1 --2- 172.21.15.131:0/3428367862 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbb84071c10 0x7fbb8406d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:57.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.229+0000 7fbb88c00640 1 -- 172.21.15.131:0/3428367862 >> v1:172.21.15.131:6789/0 conn(0x7fbb8415bfa0 legacy=0x7fbb84071500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:57.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.229+0000 7fbb88c00640 1 -- 172.21.15.131:0/3428367862 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb84072510 con 0x7fbb8415dc40 2024-05-26T22:48:57.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.231+0000 7fbb88c00640 1 --2- 172.21.15.131:0/3428367862 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb8415dc40 0x7fbb8406d250 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fbb700097a0 tx=0x7fbb7002c390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=296112402266b43e server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:57.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.233+0000 7fbb83400640 1 -- 172.21.15.131:0/3428367862 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb70039070 con 0x7fbb8415dc40 2024-05-26T22:48:57.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.234+0000 7fbb83400640 1 -- 172.21.15.131:0/3428367862 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb70034640 con 0x7fbb8415dc40 2024-05-26T22:48:57.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.234+0000 7fbb83400640 1 -- 172.21.15.131:0/3428367862 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb70034a10 con 0x7fbb8415dc40 2024-05-26T22:48:57.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.234+0000 7fbb8a000640 1 -- 172.21.15.131:0/3428367862 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb8415dc40 msgr2=0x7fbb8406d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:57.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.234+0000 7fbb8a000640 1 --2- 172.21.15.131:0/3428367862 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb8415dc40 0x7fbb8406d250 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fbb700097a0 tx=0x7fbb7002c390 comp rx=0 tx=0).stop 2024-05-26T22:48:57.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.234+0000 7fbb8a000640 1 -- 172.21.15.131:0/3428367862 shutdown_connections 2024-05-26T22:48:57.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.234+0000 7fbb8a000640 1 --2- 172.21.15.131:0/3428367862 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbb84071c10 0x7fbb8406d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:57.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.234+0000 7fbb8a000640 1 --2- 172.21.15.131:0/3428367862 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb8415dc40 0x7fbb8406d250 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:57.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.234+0000 7fbb8a000640 1 -- 172.21.15.131:0/3428367862 >> 172.21.15.131:0/3428367862 conn(0x7fbb840b5660 msgr2=0x7fbb840b5a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:57.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.235+0000 7fbb8a000640 1 -- 172.21.15.131:0/3428367862 shutdown_connections 2024-05-26T22:48:57.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.235+0000 7fbb8a000640 1 -- 172.21.15.131:0/3428367862 wait complete. 2024-05-26T22:48:57.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.237+0000 7fbb8a000640 1 Processor -- start 2024-05-26T22:48:57.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.237+0000 7fbb8a000640 1 -- start start 2024-05-26T22:48:57.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.238+0000 7fbb8a000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb84071c10 0x7fbb84150f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:57.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.238+0000 7fbb8a000640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb8415bfa0 0x7fbb840cd910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:57.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.238+0000 7fbb8a000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbb8415dc40 0x7fbb84141eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:57.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.238+0000 7fbb8a000640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fbb8415e1a0 con 0x7fbb84071c10 2024-05-26T22:48:57.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.238+0000 7fbb8a000640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fbb8415e020 con 0x7fbb8415bfa0 2024-05-26T22:48:57.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.238+0000 7fbb8a000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fbb840b2080 con 0x7fbb8415dc40 2024-05-26T22:48:57.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.238+0000 7fbb88c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb84071c10 0x7fbb84150f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:57.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.238+0000 7fbb88c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb84071c10 0x7fbb84150f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43872/0 (socket says 172.21.15.131:43872) 2024-05-26T22:48:57.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.238+0000 7fbb88c00640 1 -- 172.21.15.131:0/1461083359 learned_addr learned my addr 172.21.15.131:0/1461083359 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:57.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.239+0000 7fbb88c00640 1 -- 172.21.15.131:0/1461083359 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbb8415dc40 msgr2=0x7fbb84141eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:57.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.239+0000 7fbb88c00640 1 --2- 172.21.15.131:0/1461083359 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbb8415dc40 0x7fbb84141eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:57.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.239+0000 7fbb88c00640 1 -- 172.21.15.131:0/1461083359 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb8415bfa0 msgr2=0x7fbb840cd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:57.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.239+0000 7fbb83e00640 1 --2- 172.21.15.131:0/1461083359 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb8415bfa0 0x7fbb840cd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:57.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.239+0000 7fbb88c00640 1 --2- 172.21.15.131:0/1461083359 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb8415bfa0 0x7fbb840cd910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:57.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.239+0000 7fbb88c00640 1 -- 172.21.15.131:0/1461083359 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb840ce010 con 0x7fbb84071c10 2024-05-26T22:48:57.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.239+0000 7fbb83e00640 1 --2- 172.21.15.131:0/1461083359 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb8415bfa0 0x7fbb840cd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:48:57.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.239+0000 7fbb88c00640 1 --2- 172.21.15.131:0/1461083359 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb84071c10 0x7fbb84150f40 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fbb70037040 tx=0x7fbb700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:57.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.240+0000 7fbb80c00640 1 -- 172.21.15.131:0/1461083359 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb70039040 con 0x7fbb84071c10 2024-05-26T22:48:57.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.240+0000 7fbb80c00640 1 -- 172.21.15.131:0/1461083359 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb70034440 con 0x7fbb84071c10 2024-05-26T22:48:57.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.240+0000 7fbb80c00640 1 -- 172.21.15.131:0/1461083359 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb70032950 con 0x7fbb84071c10 2024-05-26T22:48:57.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.240+0000 7fbb8a000640 1 -- 172.21.15.131:0/1461083359 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb840ce2f0 con 0x7fbb84071c10 2024-05-26T22:48:57.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.240+0000 7fbb8a000640 1 -- 172.21.15.131:0/1461083359 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb840ce8b0 con 0x7fbb84071c10 2024-05-26T22:48:57.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.241+0000 7fbb8a000640 1 -- 172.21.15.131:0/1461083359 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb840694e0 con 0x7fbb84071c10 2024-05-26T22:48:57.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.242+0000 7fbb80c00640 1 -- 172.21.15.131:0/1461083359 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbb70046020 con 0x7fbb84071c10 2024-05-26T22:48:57.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.243+0000 7fbb80c00640 1 --2- 172.21.15.131:0/1461083359 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fbb6406be30 0x7fbb6406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:57.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.243+0000 7fbb80c00640 1 -- 172.21.15.131:0/1461083359 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fbb700acf20 con 0x7fbb84071c10 2024-05-26T22:48:57.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.243+0000 7fbb83e00640 1 --2- 172.21.15.131:0/1461083359 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fbb6406be30 0x7fbb6406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:57.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.244+0000 7fbb83e00640 1 --2- 172.21.15.131:0/1461083359 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fbb6406be30 0x7fbb6406e2f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fbb840b13d0 tx=0x7fbb78009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:57.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.248+0000 7fbb80c00640 1 -- 172.21.15.131:0/1461083359 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fbb7007b960 con 0x7fbb84071c10 2024-05-26T22:48:57.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.450+0000 7fbb8a000640 1 -- 172.21.15.131:0/1461083359 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fbb840b35d0 con 0x7fbb84071c10 2024-05-26T22:48:57.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.450+0000 7fbb80c00640 1 -- 172.21.15.131:0/1461083359 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7fbb7007b260 con 0x7fbb84071c10 2024-05-26T22:48:57.450 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 64 2024-05-26T22:48:57.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.454+0000 7fbb8a000640 1 -- 172.21.15.131:0/1461083359 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fbb6406be30 msgr2=0x7fbb6406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:57.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.454+0000 7fbb8a000640 1 --2- 172.21.15.131:0/1461083359 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fbb6406be30 0x7fbb6406e2f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fbb840b13d0 tx=0x7fbb78009040 comp rx=0 tx=0).stop 2024-05-26T22:48:57.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.454+0000 7fbb8a000640 1 -- 172.21.15.131:0/1461083359 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb84071c10 msgr2=0x7fbb84150f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:57.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.454+0000 7fbb8a000640 1 --2- 172.21.15.131:0/1461083359 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb84071c10 0x7fbb84150f40 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fbb70037040 tx=0x7fbb700046c0 comp rx=0 tx=0).stop 2024-05-26T22:48:57.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.454+0000 7fbb8a000640 1 -- 172.21.15.131:0/1461083359 shutdown_connections 2024-05-26T22:48:57.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.454+0000 7fbb8a000640 1 --2- 172.21.15.131:0/1461083359 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbb8415dc40 0x7fbb84141eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:57.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.454+0000 7fbb8a000640 1 --2- 172.21.15.131:0/1461083359 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fbb6406be30 0x7fbb6406e2f0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:57.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.454+0000 7fbb8a000640 1 --2- 172.21.15.131:0/1461083359 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb8415bfa0 0x7fbb840cd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:57.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.454+0000 7fbb8a000640 1 --2- 172.21.15.131:0/1461083359 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb84071c10 0x7fbb84150f40 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:57.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.454+0000 7fbb8a000640 1 -- 172.21.15.131:0/1461083359 >> 172.21.15.131:0/1461083359 conn(0x7fbb840b5660 msgr2=0x7fbb8415d1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:57.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.454+0000 7fbb8a000640 1 -- 172.21.15.131:0/1461083359 shutdown_connections 2024-05-26T22:48:57.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.454+0000 7fbb8a000640 1 -- 172.21.15.131:0/1461083359 wait complete. 2024-05-26T22:48:57.473 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-26T22:48:57.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.671+0000 7f086b200640 1 Processor -- start 2024-05-26T22:48:57.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.671+0000 7f086b200640 1 -- start start 2024-05-26T22:48:57.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.674+0000 7f086b200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f086c156750 0x7f086c15fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:57.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.674+0000 7f086b200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f086c06e8c0 0x7f086c15d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:57.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.674+0000 7f086b200640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f086c160620 con 0x7f086c0cc7f0 2024-05-26T22:48:57.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.674+0000 7f086b200640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f086c15cbd0 con 0x7f086c156750 2024-05-26T22:48:57.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.674+0000 7f086b200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f086c160900 con 0x7f086c06e8c0 2024-05-26T22:48:57.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.674+0000 7f0869e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f086c156750 0x7f086c15fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:57.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.674+0000 7f0869e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f086c156750 0x7f086c15fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43876/0 (socket says 172.21.15.131:43876) 2024-05-26T22:48:57.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.674+0000 7f0869e00640 1 -- 172.21.15.131:0/949244303 learned_addr learned my addr 172.21.15.131:0/949244303 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:57.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.675+0000 7f0869400640 1 --2- 172.21.15.131:0/949244303 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f086c06e8c0 0x7f086c15d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:57.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.675+0000 7f0869e00640 1 -- 172.21.15.131:0/949244303 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f086c06e8c0 msgr2=0x7f086c15d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:57.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.675+0000 7f0869e00640 1 --2- 172.21.15.131:0/949244303 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f086c06e8c0 0x7f086c15d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:57.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.675+0000 7f0869e00640 1 -- 172.21.15.131:0/949244303 >> v1:172.21.15.131:6789/0 conn(0x7f086c0cc7f0 legacy=0x7f086c0cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:57.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.675+0000 7f0869e00640 1 -- 172.21.15.131:0/949244303 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f086c160be0 con 0x7f086c156750 2024-05-26T22:48:57.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.675+0000 7f0868a00640 1 -- 172.21.15.131:0/949244303 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4260527483 0 0) 0x7f086c160620 con 0x7f086c0cc7f0 2024-05-26T22:48:57.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.676+0000 7f0869e00640 1 --2- 172.21.15.131:0/949244303 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f086c156750 0x7f086c15fd40 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f085c009870 tx=0x7f085c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c89372b76e593ff server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:57.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.676+0000 7f0868a00640 1 -- 172.21.15.131:0/949244303 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f085c037070 con 0x7f086c156750 2024-05-26T22:48:57.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.677+0000 7f0868a00640 1 -- 172.21.15.131:0/949244303 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f085c032440 con 0x7f086c156750 2024-05-26T22:48:57.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.677+0000 7f0868a00640 1 -- 172.21.15.131:0/949244303 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f085c032810 con 0x7f086c156750 2024-05-26T22:48:57.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.677+0000 7f086b200640 1 -- 172.21.15.131:0/949244303 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f086c156750 msgr2=0x7f086c15fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:57.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.677+0000 7f086b200640 1 --2- 172.21.15.131:0/949244303 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f086c156750 0x7f086c15fd40 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f085c009870 tx=0x7f085c02a270 comp rx=0 tx=0).stop 2024-05-26T22:48:57.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.677+0000 7f086b200640 1 -- 172.21.15.131:0/949244303 shutdown_connections 2024-05-26T22:48:57.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.677+0000 7f086b200640 1 --2- 172.21.15.131:0/949244303 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f086c06e8c0 0x7f086c15d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:57.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.677+0000 7f086b200640 1 --2- 172.21.15.131:0/949244303 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f086c156750 0x7f086c15fd40 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:57.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.677+0000 7f086b200640 1 -- 172.21.15.131:0/949244303 >> 172.21.15.131:0/949244303 conn(0x7f086c0cff40 msgr2=0x7f086c0d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:57.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.677+0000 7f086b200640 1 -- 172.21.15.131:0/949244303 shutdown_connections 2024-05-26T22:48:57.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.677+0000 7f086b200640 1 -- 172.21.15.131:0/949244303 wait complete. 2024-05-26T22:48:57.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.679+0000 7f086b200640 1 Processor -- start 2024-05-26T22:48:57.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.679+0000 7f086b200640 1 -- start start 2024-05-26T22:48:57.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.680+0000 7f086b200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f086c06e8c0 0x7f086c155590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:57.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.680+0000 7f086b200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f086c0cc7f0 0x7f086c15e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:57.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.680+0000 7f086b200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f086c156750 0x7f086c15eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:57.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.680+0000 7f086b200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f086c160280 con 0x7f086c0cc7f0 2024-05-26T22:48:57.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.680+0000 7f086b200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f086c156b30 con 0x7f086c156750 2024-05-26T22:48:57.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.680+0000 7f086b200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f086c160400 con 0x7f086c06e8c0 2024-05-26T22:48:57.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.680+0000 7f0869400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f086c0cc7f0 0x7f086c15e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:57.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.681+0000 7f0869400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f086c0cc7f0 0x7f086c15e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43882/0 (socket says 172.21.15.131:43882) 2024-05-26T22:48:57.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.681+0000 7f0869400640 1 -- 172.21.15.131:0/2358916741 learned_addr learned my addr 172.21.15.131:0/2358916741 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:57.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.681+0000 7f0869400640 1 -- 172.21.15.131:0/2358916741 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f086c06e8c0 msgr2=0x7f086c155590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:57.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.681+0000 7f0869400640 1 --2- 172.21.15.131:0/2358916741 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f086c06e8c0 0x7f086c155590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:57.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.681+0000 7f0869400640 1 -- 172.21.15.131:0/2358916741 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f086c156750 msgr2=0x7f086c15eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:57.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.681+0000 7f0869400640 1 --2- 172.21.15.131:0/2358916741 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f086c156750 0x7f086c15eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:57.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.681+0000 7f0869400640 1 -- 172.21.15.131:0/2358916741 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f086c1f4b80 con 0x7f086c0cc7f0 2024-05-26T22:48:57.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.681+0000 7f0869400640 1 --2- 172.21.15.131:0/2358916741 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f086c0cc7f0 0x7f086c15e650 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f085800d5f0 tx=0x7f085800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:57.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.681+0000 7f0862000640 1 -- 172.21.15.131:0/2358916741 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08580125f0 con 0x7f086c0cc7f0 2024-05-26T22:48:57.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.682+0000 7f0862000640 1 -- 172.21.15.131:0/2358916741 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0858019070 con 0x7f086c0cc7f0 2024-05-26T22:48:57.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.682+0000 7f086b200640 1 -- 172.21.15.131:0/2358916741 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f086c154ab0 con 0x7f086c0cc7f0 2024-05-26T22:48:57.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.682+0000 7f0862000640 1 -- 172.21.15.131:0/2358916741 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08580129d0 con 0x7f086c0cc7f0 2024-05-26T22:48:57.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.682+0000 7f086b200640 1 -- 172.21.15.131:0/2358916741 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f086c155070 con 0x7f086c0cc7f0 2024-05-26T22:48:57.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.683+0000 7f086b200640 1 -- 172.21.15.131:0/2358916741 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f086c1f1910 con 0x7f086c0cc7f0 2024-05-26T22:48:57.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.684+0000 7f0862000640 1 -- 172.21.15.131:0/2358916741 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0858012b70 con 0x7f086c0cc7f0 2024-05-26T22:48:57.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.685+0000 7f0862000640 1 --2- 172.21.15.131:0/2358916741 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f084c06be30 0x7f084c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:57.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.685+0000 7f0862000640 1 -- 172.21.15.131:0/2358916741 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f085808b1e0 con 0x7f086c0cc7f0 2024-05-26T22:48:57.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.685+0000 7f0869e00640 1 --2- 172.21.15.131:0/2358916741 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f084c06be30 0x7f084c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:57.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.686+0000 7f0869e00640 1 --2- 172.21.15.131:0/2358916741 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f084c06be30 0x7f084c06e2f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f085c034000 tx=0x7f085c02c000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:57.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.689+0000 7f0862000640 1 -- 172.21.15.131:0/2358916741 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f0858059c20 con 0x7f086c0cc7f0 2024-05-26T22:48:57.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.889+0000 7f086b200640 1 -- 172.21.15.131:0/2358916741 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f086c0694e0 con 0x7f086c0cc7f0 2024-05-26T22:48:57.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.892+0000 7f0862000640 1 -- 172.21.15.131:0/2358916741 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16629 (secure 0 0 0) 0x7f0858059520 con 0x7f086c0cc7f0 2024-05-26T22:48:57.892 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:48:57.893 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":20,"fsid":"46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f","created":"2024-05-26T22:47:57.288905+0000","modified":"2024-05-26T22:48:22.799298+0000","last_up_change":"2024-05-26T22:48:09.602207+0000","last_in_change":"2024-05-26T22:48:00.687341+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-26T22:48:10.463339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-26T22:48:19.663254+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-26T22:48:20.235751+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"814eaf62-a18d-494b-8e27-fd8fb88adea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6801","nonce":2350940051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6803","nonce":2350940051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6807","nonce":2350940051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6805","nonce":2350940051}]},"public_addr":"172.21.15.131:6801/2350940051","cluster_addr":"172.21.15.131:6803/2350940051","heartbeat_back_addr":"172.21.15.131:6807/2350940051","heartbeat_front_addr":"172.21.15.131:6805/2350940051","state":["exists","up"]},{"osd":1,"uuid":"42e66287-f6c6-4e26-af3c-6dfaa3536d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6825","nonce":1525528247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6827","nonce":1525528247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6831","nonce":1525528247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6829","nonce":1525528247}]},"public_addr":"172.21.15.131:6825/1525528247","cluster_addr":"172.21.15.131:6827/1525528247","heartbeat_back_addr":"172.21.15.131:6831/1525528247","heartbeat_front_addr":"172.21.15.131:6829/1525528247","state":["exists","up"]},{"osd":2,"uuid":"0da34673-04a2-4fd0-b57a-0ff2b679b043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6817","nonce":3627842210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6819","nonce":3627842210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6823","nonce":3627842210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6821","nonce":3627842210}]},"public_addr":"172.21.15.131:6817/3627842210","cluster_addr":"172.21.15.131:6819/3627842210","heartbeat_back_addr":"172.21.15.131:6823/3627842210","heartbeat_front_addr":"172.21.15.131:6821/3627842210","state":["exists","up"]},{"osd":3,"uuid":"cf117a27-d6b2-43d3-ae52-b47988bf8eed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6809","nonce":350790591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6811","nonce":350790591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6815","nonce":350790591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6813","nonce":350790591}]},"public_addr":"172.21.15.131:6809/350790591","cluster_addr":"172.21.15.131:6811/350790591","heartbeat_back_addr":"172.21.15.131:6815/350790591","heartbeat_front_addr":"172.21.15.131:6813/350790591","state":["exists","up"]},{"osd":4,"uuid":"a092ccfc-be89-4a84-88d5-0f781fa715eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6816","nonce":244560995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6819","nonce":244560995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6823","nonce":244560995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6821","nonce":244560995}]},"public_addr":"172.21.15.187:6816/244560995","cluster_addr":"172.21.15.187:6819/244560995","heartbeat_back_addr":"172.21.15.187:6823/244560995","heartbeat_front_addr":"172.21.15.187:6821/244560995","state":["exists","up"]},{"osd":5,"uuid":"7513ba69-dd00-4a35-b190-9e8cc3b2bdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6801","nonce":324619398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6803","nonce":324619398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6812","nonce":324619398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6807","nonce":324619398}]},"public_addr":"172.21.15.187:6801/324619398","cluster_addr":"172.21.15.187:6803/324619398","heartbeat_back_addr":"172.21.15.187:6812/324619398","heartbeat_front_addr":"172.21.15.187:6807/324619398","state":["exists","up"]},{"osd":6,"uuid":"50081810-c093-4d2f-a4d7-b7c695c5400d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6806","nonce":1391347786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6809","nonce":1391347786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6818","nonce":1391347786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6813","nonce":1391347786}]},"public_addr":"172.21.15.187:6806/1391347786","cluster_addr":"172.21.15.187:6809/1391347786","heartbeat_back_addr":"172.21.15.187:6818/1391347786","heartbeat_front_addr":"172.21.15.187:6813/1391347786","state":["exists","up"]},{"osd":7,"uuid":"775dae05-54f5-4ad4-bbce-92863c6ed2ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6825","nonce":1220065622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6827","nonce":1220065622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6831","nonce":1220065622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6829","nonce":1220065622}]},"public_addr":"172.21.15.187:6825/1220065622","cluster_addr":"172.21.15.187:6827/1220065622","heartbeat_back_addr":"172.21.15.187:6831/1220065622","heartbeat_front_addr":"172.21.15.187:6829/1220065622","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:05.983520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.299516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.067323+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.307748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.018742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.258775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.010912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.295355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-26T22:48:57.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.895+0000 7f086b200640 1 -- 172.21.15.131:0/2358916741 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f084c06be30 msgr2=0x7f084c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:57.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.895+0000 7f086b200640 1 --2- 172.21.15.131:0/2358916741 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f084c06be30 0x7f084c06e2f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f085c034000 tx=0x7f085c02c000 comp rx=0 tx=0).stop 2024-05-26T22:48:57.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.895+0000 7f086b200640 1 -- 172.21.15.131:0/2358916741 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f086c0cc7f0 msgr2=0x7f086c15e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:57.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.895+0000 7f086b200640 1 --2- 172.21.15.131:0/2358916741 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f086c0cc7f0 0x7f086c15e650 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f085800d5f0 tx=0x7f085800dab0 comp rx=0 tx=0).stop 2024-05-26T22:48:57.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.895+0000 7f086b200640 1 -- 172.21.15.131:0/2358916741 shutdown_connections 2024-05-26T22:48:57.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.895+0000 7f086b200640 1 --2- 172.21.15.131:0/2358916741 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f084c06be30 0x7f084c06e2f0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:57.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.895+0000 7f086b200640 1 --2- 172.21.15.131:0/2358916741 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f086c156750 0x7f086c15eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:57.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.895+0000 7f086b200640 1 --2- 172.21.15.131:0/2358916741 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f086c0cc7f0 0x7f086c15e650 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:57.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.895+0000 7f086b200640 1 --2- 172.21.15.131:0/2358916741 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f086c06e8c0 0x7f086c155590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:57.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.895+0000 7f086b200640 1 -- 172.21.15.131:0/2358916741 >> 172.21.15.131:0/2358916741 conn(0x7f086c0cff40 msgr2=0x7f086c0ae050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:57.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.895+0000 7f086b200640 1 -- 172.21.15.131:0/2358916741 shutdown_connections 2024-05-26T22:48:57.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:57.895+0000 7f086b200640 1 -- 172.21.15.131:0/2358916741 wait complete. 2024-05-26T22:48:57.914 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-26T22:48:10.463339+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-26T22:48:19.663254+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-26T22:48:20.235751+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-26T22:48:57.915 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-26T22:48:58.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.118+0000 7f6cf6800640 1 Processor -- start 2024-05-26T22:48:58.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.118+0000 7f6cf6800640 1 -- start start 2024-05-26T22:48:58.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.120+0000 7f6cf6800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6cf0155770 0x7f6cf015b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:58.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.120+0000 7f6cf6800640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f6cf006e440 0x7f6cf00725a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:58.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.120+0000 7f6cf6800640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6cf015c1c0 con 0x7f6cf015bc40 2024-05-26T22:48:58.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.120+0000 7f6cf6800640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f6cf015c4a0 con 0x7f6cf006e440 2024-05-26T22:48:58.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.120+0000 7f6cf6800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f6cf0071da0 con 0x7f6cf0155770 2024-05-26T22:48:58.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.120+0000 7f6cefe00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f6cf006e440 0x7f6cf00725a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:58.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.120+0000 7f6cefe00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f6cf006e440 0x7f6cf00725a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43886/0 (socket says 172.21.15.131:43886) 2024-05-26T22:48:58.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.120+0000 7f6cefe00640 1 -- 172.21.15.131:0/3950353542 learned_addr learned my addr 172.21.15.131:0/3950353542 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:58.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.120+0000 7f6ce7e00640 1 --1- 172.21.15.131:0/3950353542 >> v1:172.21.15.131:6789/0 conn(0x7f6cf015bc40 0x7f6cf006dd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:48132/0 (socket says 172.21.15.131:48132) 2024-05-26T22:48:58.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.121+0000 7f6cef400640 1 --2- 172.21.15.131:0/3950353542 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6cf0155770 0x7f6cf015b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:58.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.121+0000 7f6cefe00640 1 -- 172.21.15.131:0/3950353542 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6cf0155770 msgr2=0x7f6cf015b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:58.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.121+0000 7f6cefe00640 1 --2- 172.21.15.131:0/3950353542 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6cf0155770 0x7f6cf015b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:58.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.121+0000 7f6cefe00640 1 -- 172.21.15.131:0/3950353542 >> v1:172.21.15.131:6789/0 conn(0x7f6cf015bc40 legacy=0x7f6cf006dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:58.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.121+0000 7f6cefe00640 1 -- 172.21.15.131:0/3950353542 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cf0072080 con 0x7f6cf006e440 2024-05-26T22:48:58.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.122+0000 7f6cefe00640 1 --2- 172.21.15.131:0/3950353542 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6cf006e440 0x7f6cf00725a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f6ce8009e50 tx=0x7f6ce802a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bf150402fd3a977 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:58.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.122+0000 7f6ceea00640 1 -- 172.21.15.131:0/3950353542 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ce8036070 con 0x7f6cf006e440 2024-05-26T22:48:58.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.122+0000 7f6ceea00640 1 -- 172.21.15.131:0/3950353542 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ce8031600 con 0x7f6cf006e440 2024-05-26T22:48:58.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.122+0000 7f6ceea00640 1 -- 172.21.15.131:0/3950353542 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ce80319d0 con 0x7f6cf006e440 2024-05-26T22:48:58.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.123+0000 7f6cf6800640 1 -- 172.21.15.131:0/3950353542 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6cf006e440 msgr2=0x7f6cf00725a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:58.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.123+0000 7f6cf6800640 1 --2- 172.21.15.131:0/3950353542 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6cf006e440 0x7f6cf00725a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f6ce8009e50 tx=0x7f6ce802a7a0 comp rx=0 tx=0).stop 2024-05-26T22:48:58.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.123+0000 7f6cf6800640 1 -- 172.21.15.131:0/3950353542 shutdown_connections 2024-05-26T22:48:58.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.123+0000 7f6cf6800640 1 --2- 172.21.15.131:0/3950353542 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6cf006e440 0x7f6cf00725a0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:58.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.123+0000 7f6cf6800640 1 --2- 172.21.15.131:0/3950353542 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6cf0155770 0x7f6cf015b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:58.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.123+0000 7f6cf6800640 1 -- 172.21.15.131:0/3950353542 >> 172.21.15.131:0/3950353542 conn(0x7f6cf006bc90 msgr2=0x7f6cf00b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:58.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.123+0000 7f6cf6800640 1 -- 172.21.15.131:0/3950353542 shutdown_connections 2024-05-26T22:48:58.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.123+0000 7f6cf6800640 1 -- 172.21.15.131:0/3950353542 wait complete. 2024-05-26T22:48:58.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.125+0000 7f6cf6800640 1 Processor -- start 2024-05-26T22:48:58.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.125+0000 7f6cf6800640 1 -- start start 2024-05-26T22:48:58.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.126+0000 7f6cf6800640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6cf006e440 0x7f6cf014bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:58.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.126+0000 7f6cf6800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6cf0155770 0x7f6cf009ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:58.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.126+0000 7f6cf6800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6cf015bc40 0x7f6cf01508f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:58.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.126+0000 7f6cf6800640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f6cf009de00 con 0x7f6cf0155770 2024-05-26T22:48:58.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.126+0000 7f6cf6800640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6cf015c8c0 con 0x7f6cf006e440 2024-05-26T22:48:58.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.126+0000 7f6cf6800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f6cf009df80 con 0x7f6cf015bc40 2024-05-26T22:48:58.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.126+0000 7f6ce7e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6cf0155770 0x7f6cf009ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:58.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.126+0000 7f6ce7e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6cf0155770 0x7f6cf009ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43888/0 (socket says 172.21.15.131:43888) 2024-05-26T22:48:58.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.126+0000 7f6ce7e00640 1 -- 172.21.15.131:0/1235967430 learned_addr learned my addr 172.21.15.131:0/1235967430 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:58.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.127+0000 7f6cef400640 1 --2- 172.21.15.131:0/1235967430 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6cf006e440 0x7f6cf014bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:58.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.127+0000 7f6ce7e00640 1 -- 172.21.15.131:0/1235967430 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6cf015bc40 msgr2=0x7f6cf01508f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:58.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.127+0000 7f6cefe00640 1 --2- 172.21.15.131:0/1235967430 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6cf015bc40 0x7f6cf01508f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:58.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.127+0000 7f6ce7e00640 1 --2- 172.21.15.131:0/1235967430 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6cf015bc40 0x7f6cf01508f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:58.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.127+0000 7f6ce7e00640 1 -- 172.21.15.131:0/1235967430 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6cf006e440 msgr2=0x7f6cf014bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:58.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.127+0000 7f6ce7e00640 1 --2- 172.21.15.131:0/1235967430 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6cf006e440 0x7f6cf014bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:58.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.127+0000 7f6ce7e00640 1 -- 172.21.15.131:0/1235967430 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cf014c2d0 con 0x7f6cf0155770 2024-05-26T22:48:58.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.127+0000 7f6cef400640 1 --2- 172.21.15.131:0/1235967430 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6cf006e440 0x7f6cf014bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:48:58.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.127+0000 7f6ce7e00640 1 --2- 172.21.15.131:0/1235967430 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6cf0155770 0x7f6cf009ba50 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f6cd800c390 tx=0x7f6cd800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:58.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.127+0000 7f6ce7400640 1 -- 172.21.15.131:0/1235967430 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cd8017070 con 0x7f6cf0155770 2024-05-26T22:48:58.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.127+0000 7f6ce7400640 1 -- 172.21.15.131:0/1235967430 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cd80064e0 con 0x7f6cf0155770 2024-05-26T22:48:58.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.127+0000 7f6cf6800640 1 -- 172.21.15.131:0/1235967430 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cf009c430 con 0x7f6cf0155770 2024-05-26T22:48:58.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.127+0000 7f6ce7400640 1 -- 172.21.15.131:0/1235967430 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cd8004a00 con 0x7f6cf0155770 2024-05-26T22:48:58.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.128+0000 7f6cf6800640 1 -- 172.21.15.131:0/1235967430 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cf009c9f0 con 0x7f6cf0155770 2024-05-26T22:48:58.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.129+0000 7f6cf6800640 1 -- 172.21.15.131:0/1235967430 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cf00adf00 con 0x7f6cf0155770 2024-05-26T22:48:58.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.129+0000 7f6ce7400640 1 -- 172.21.15.131:0/1235967430 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6cd801a070 con 0x7f6cf0155770 2024-05-26T22:48:58.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.130+0000 7f6ce7400640 1 --2- 172.21.15.131:0/1235967430 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f6ccc06be30 0x7f6ccc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:58.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.130+0000 7f6ce7400640 1 -- 172.21.15.131:0/1235967430 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f6cd808a270 con 0x7f6cf0155770 2024-05-26T22:48:58.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.131+0000 7f6cef400640 1 --2- 172.21.15.131:0/1235967430 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f6ccc06be30 0x7f6ccc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:58.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.131+0000 7f6cef400640 1 --2- 172.21.15.131:0/1235967430 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f6ccc06be30 0x7f6ccc06e2f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6ce0006fd0 tx=0x7f6ce0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:58.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.135+0000 7f6ce7400640 1 -- 172.21.15.131:0/1235967430 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f6cf00adf00 con 0x7f6cf0155770 2024-05-26T22:48:58.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.334+0000 7f6cf6800640 1 -- 172.21.15.131:0/1235967430 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6cf00695c0 con 0x7f6cf0155770 2024-05-26T22:48:58.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.335+0000 7f6ce7400640 1 -- 172.21.15.131:0/1235967430 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f6cd8058b80 con 0x7f6cf0155770 2024-05-26T22:48:58.335 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2024-05-26T22:48:58.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.337+0000 7f6cf6800640 1 -- 172.21.15.131:0/1235967430 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f6ccc06be30 msgr2=0x7f6ccc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:58.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.337+0000 7f6cf6800640 1 --2- 172.21.15.131:0/1235967430 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f6ccc06be30 0x7f6ccc06e2f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6ce0006fd0 tx=0x7f6ce0008040 comp rx=0 tx=0).stop 2024-05-26T22:48:58.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.337+0000 7f6cf6800640 1 -- 172.21.15.131:0/1235967430 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6cf0155770 msgr2=0x7f6cf009ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:58.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.337+0000 7f6cf6800640 1 --2- 172.21.15.131:0/1235967430 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6cf0155770 0x7f6cf009ba50 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f6cd800c390 tx=0x7f6cd800c850 comp rx=0 tx=0).stop 2024-05-26T22:48:58.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.337+0000 7f6cf6800640 1 -- 172.21.15.131:0/1235967430 shutdown_connections 2024-05-26T22:48:58.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.337+0000 7f6cf6800640 1 --2- 172.21.15.131:0/1235967430 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6cf015bc40 0x7f6cf01508f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:58.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.337+0000 7f6cf6800640 1 --2- 172.21.15.131:0/1235967430 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f6ccc06be30 0x7f6ccc06e2f0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:58.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.337+0000 7f6cf6800640 1 --2- 172.21.15.131:0/1235967430 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6cf0155770 0x7f6cf009ba50 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:58.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.337+0000 7f6cf6800640 1 --2- 172.21.15.131:0/1235967430 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6cf006e440 0x7f6cf014bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:58.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.337+0000 7f6cf6800640 1 -- 172.21.15.131:0/1235967430 >> 172.21.15.131:0/1235967430 conn(0x7f6cf006bc90 msgr2=0x7f6cf015a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:58.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.338+0000 7f6cf6800640 1 -- 172.21.15.131:0/1235967430 shutdown_connections 2024-05-26T22:48:58.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.338+0000 7f6cf6800640 1 -- 172.21.15.131:0/1235967430 wait complete. 2024-05-26T22:48:58.357 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-26T22:48:58.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.559+0000 7effbfc00640 1 Processor -- start 2024-05-26T22:48:58.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.559+0000 7effbfc00640 1 -- start start 2024-05-26T22:48:58.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.561+0000 7effbfc00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7effb806f180 0x7effb8072e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:58.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.561+0000 7effbfc00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7effb80d0560 0x7effb8161ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:58.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.561+0000 7effbfc00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7effb80af8e0 con 0x7effb8072540 2024-05-26T22:48:58.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.561+0000 7effbfc00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7effb80cfb30 con 0x7effb806f180 2024-05-26T22:48:58.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.561+0000 7effbfc00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7effb80cf200 con 0x7effb80d0560 2024-05-26T22:48:58.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.561+0000 7effbd200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7effb806f180 0x7effb8072e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:58.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.561+0000 7effb7e00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7effb8072540 0x7effb80cfe50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:48136/0 (socket says 172.21.15.131:48136) 2024-05-26T22:48:58.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.561+0000 7effb7e00640 1 -- 172.21.15.131:0/3427915848 learned_addr learned my addr 172.21.15.131:0/3427915848 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:48:58.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.561+0000 7effbd200640 1 -- 172.21.15.131:0/3427915848 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7effb80d0560 msgr2=0x7effb8161ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:58.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.561+0000 7effbd200640 1 --2- 172.21.15.131:0/3427915848 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7effb80d0560 0x7effb8161ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:58.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.561+0000 7effbd200640 1 -- 172.21.15.131:0/3427915848 >> v1:172.21.15.131:6789/0 conn(0x7effb8072540 legacy=0x7effb80cfe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:58.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.562+0000 7effbd200640 1 -- 172.21.15.131:0/3427915848 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effb80cf4e0 con 0x7effb806f180 2024-05-26T22:48:58.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.561+0000 7effb7400640 1 -- 172.21.15.131:0/3427915848 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1959346230 0 0) 0x7effb80af8e0 con 0x7effb8072540 2024-05-26T22:48:58.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.563+0000 7effbd200640 1 --2- 172.21.15.131:0/3427915848 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7effb806f180 0x7effb8072e90 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7effac0097a0 tx=0x7effac02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dc506780d43f8ef server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:58.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.563+0000 7effb7400640 1 -- 172.21.15.131:0/3427915848 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effac037070 con 0x7effb806f180 2024-05-26T22:48:58.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.563+0000 7effb7400640 1 -- 172.21.15.131:0/3427915848 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effac032440 con 0x7effb806f180 2024-05-26T22:48:58.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.563+0000 7effb7400640 1 -- 172.21.15.131:0/3427915848 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effac032810 con 0x7effb806f180 2024-05-26T22:48:58.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.564+0000 7effbfc00640 1 -- 172.21.15.131:0/3427915848 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7effb806f180 msgr2=0x7effb8072e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:58.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.564+0000 7effbfc00640 1 --2- 172.21.15.131:0/3427915848 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7effb806f180 0x7effb8072e90 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7effac0097a0 tx=0x7effac02a1a0 comp rx=0 tx=0).stop 2024-05-26T22:48:58.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.564+0000 7effbfc00640 1 -- 172.21.15.131:0/3427915848 shutdown_connections 2024-05-26T22:48:58.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.564+0000 7effbfc00640 1 --2- 172.21.15.131:0/3427915848 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7effb80d0560 0x7effb8161ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:58.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.564+0000 7effbfc00640 1 --2- 172.21.15.131:0/3427915848 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7effb806f180 0x7effb8072e90 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:58.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.564+0000 7effbfc00640 1 -- 172.21.15.131:0/3427915848 >> 172.21.15.131:0/3427915848 conn(0x7effb806bc90 msgr2=0x7effb80b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:58.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.564+0000 7effbfc00640 1 -- 172.21.15.131:0/3427915848 shutdown_connections 2024-05-26T22:48:58.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.564+0000 7effbfc00640 1 -- 172.21.15.131:0/3427915848 wait complete. 2024-05-26T22:48:58.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.566+0000 7effbfc00640 1 Processor -- start 2024-05-26T22:48:58.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.566+0000 7effbfc00640 1 -- start start 2024-05-26T22:48:58.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.567+0000 7effbfc00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7effb806f180 0x7effb80ce630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:58.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.567+0000 7effbfc00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7effb8072540 0x7effb81f7a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:58.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.567+0000 7effbfc00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7effb80d0560 0x7effb8151980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:58.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.567+0000 7effbfc00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7effb806f6e0 con 0x7effb8072540 2024-05-26T22:48:58.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.567+0000 7effbfc00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7effb806f560 con 0x7effb80d0560 2024-05-26T22:48:58.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.567+0000 7effbfc00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7effb80715a0 con 0x7effb806f180 2024-05-26T22:48:58.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.568+0000 7effb7e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7effb8072540 0x7effb81f7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:58.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.568+0000 7effb7e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7effb8072540 0x7effb81f7a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43908/0 (socket says 172.21.15.131:43908) 2024-05-26T22:48:58.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.568+0000 7effb7e00640 1 -- 172.21.15.131:0/3641735825 learned_addr learned my addr 172.21.15.131:0/3641735825 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:58.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.568+0000 7effbdc00640 1 --2- 172.21.15.131:0/3641735825 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7effb80d0560 0x7effb8151980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:58.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.568+0000 7effbd200640 1 --2- 172.21.15.131:0/3641735825 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7effb806f180 0x7effb80ce630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:58.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.568+0000 7effb7e00640 1 -- 172.21.15.131:0/3641735825 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7effb806f180 msgr2=0x7effb80ce630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:58.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.568+0000 7effb7e00640 1 --2- 172.21.15.131:0/3641735825 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7effb806f180 0x7effb80ce630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:58.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.568+0000 7effb7e00640 1 -- 172.21.15.131:0/3641735825 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7effb80d0560 msgr2=0x7effb8151980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:58.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.568+0000 7effb7e00640 1 --2- 172.21.15.131:0/3641735825 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7effb80d0560 0x7effb8151980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:58.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.568+0000 7effb7e00640 1 -- 172.21.15.131:0/3641735825 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effb80c9410 con 0x7effb8072540 2024-05-26T22:48:58.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.569+0000 7effb7e00640 1 --2- 172.21.15.131:0/3641735825 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7effb8072540 0x7effb81f7a30 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7effa400c4d0 tx=0x7effa400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:58.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.569+0000 7effb4c00640 1 -- 172.21.15.131:0/3641735825 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effa4017070 con 0x7effb8072540 2024-05-26T22:48:58.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.569+0000 7effb4c00640 1 -- 172.21.15.131:0/3641735825 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effa4005d00 con 0x7effb8072540 2024-05-26T22:48:58.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.569+0000 7effb4c00640 1 -- 172.21.15.131:0/3641735825 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effa40072b0 con 0x7effb8072540 2024-05-26T22:48:58.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.569+0000 7effbfc00640 1 -- 172.21.15.131:0/3641735825 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effb81f81a0 con 0x7effb8072540 2024-05-26T22:48:58.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.569+0000 7effbfc00640 1 -- 172.21.15.131:0/3641735825 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effb81f8760 con 0x7effb8072540 2024-05-26T22:48:58.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.570+0000 7effbfc00640 1 -- 172.21.15.131:0/3641735825 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effb8074e90 con 0x7effb8072540 2024-05-26T22:48:58.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.571+0000 7effb4c00640 1 -- 172.21.15.131:0/3641735825 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7effa401a070 con 0x7effb8072540 2024-05-26T22:48:58.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.573+0000 7effb4c00640 1 --2- 172.21.15.131:0/3641735825 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7eff9806be30 0x7eff9806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:58.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.573+0000 7effb4c00640 1 -- 172.21.15.131:0/3641735825 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7effa408a240 con 0x7effb8072540 2024-05-26T22:48:58.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.573+0000 7effbd200640 1 --2- 172.21.15.131:0/3641735825 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7eff9806be30 0x7eff9806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:58.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.574+0000 7effbd200640 1 --2- 172.21.15.131:0/3641735825 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7eff9806be30 0x7eff9806e2f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7effac034000 tx=0x7effac02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:58.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.578+0000 7effb4c00640 1 -- 172.21.15.131:0/3641735825 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7effa4058be0 con 0x7effb8072540 2024-05-26T22:48:58.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.773+0000 7effbfc00640 1 -- 172.21.15.131:0/3641735825 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7effb80c95e0 con 0x7effb8072540 2024-05-26T22:48:58.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.774+0000 7effb4c00640 1 -- 172.21.15.131:0/3641735825 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7effa40584e0 con 0x7effb8072540 2024-05-26T22:48:58.774 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-05-26T22:48:58.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.777+0000 7effbfc00640 1 -- 172.21.15.131:0/3641735825 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7eff9806be30 msgr2=0x7eff9806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:58.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.777+0000 7effbfc00640 1 --2- 172.21.15.131:0/3641735825 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7eff9806be30 0x7eff9806e2f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7effac034000 tx=0x7effac02e000 comp rx=0 tx=0).stop 2024-05-26T22:48:58.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.777+0000 7effbfc00640 1 -- 172.21.15.131:0/3641735825 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7effb8072540 msgr2=0x7effb81f7a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:58.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.777+0000 7effbfc00640 1 --2- 172.21.15.131:0/3641735825 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7effb8072540 0x7effb81f7a30 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7effa400c4d0 tx=0x7effa400c990 comp rx=0 tx=0).stop 2024-05-26T22:48:58.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.777+0000 7effbfc00640 1 -- 172.21.15.131:0/3641735825 shutdown_connections 2024-05-26T22:48:58.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.777+0000 7effbfc00640 1 --2- 172.21.15.131:0/3641735825 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7eff9806be30 0x7eff9806e2f0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:58.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.777+0000 7effbfc00640 1 --2- 172.21.15.131:0/3641735825 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7effb80d0560 0x7effb8151980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:58.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.777+0000 7effbfc00640 1 --2- 172.21.15.131:0/3641735825 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7effb8072540 0x7effb81f7a30 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:58.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.777+0000 7effbfc00640 1 --2- 172.21.15.131:0/3641735825 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7effb806f180 0x7effb80ce630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:58.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.777+0000 7effbfc00640 1 -- 172.21.15.131:0/3641735825 >> 172.21.15.131:0/3641735825 conn(0x7effb806bc90 msgr2=0x7effb815ea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:58.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.777+0000 7effbfc00640 1 -- 172.21.15.131:0/3641735825 shutdown_connections 2024-05-26T22:48:58.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:58.777+0000 7effbfc00640 1 -- 172.21.15.131:0/3641735825 wait complete. 2024-05-26T22:48:58.797 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-26T22:48:59.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.000+0000 7fcbe7200640 1 Processor -- start 2024-05-26T22:48:59.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.000+0000 7fcbe7200640 1 -- start start 2024-05-26T22:48:59.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.002+0000 7fcbe7200640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fcbe8145a10 0x7fcbe8072180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:59.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.002+0000 7fcbe7200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcbe80712d0 0x7fcbe806d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:59.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.002+0000 7fcbe7200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fcbe806d790 0x7fcbe815fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:59.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.002+0000 7fcbe7200640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fcbe815c760 con 0x7fcbe806d790 2024-05-26T22:48:59.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.002+0000 7fcbe7200640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fcbe8072d60 con 0x7fcbe8145a10 2024-05-26T22:48:59.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.002+0000 7fcbe7200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fcbe8072a80 con 0x7fcbe80712d0 2024-05-26T22:48:59.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.002+0000 7fcbe6800640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fcbe806d790 0x7fcbe815fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:59.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.002+0000 7fcbe6800640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fcbe806d790 0x7fcbe815fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43920/0 (socket says 172.21.15.131:43920) 2024-05-26T22:48:59.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.002+0000 7fcbe6800640 1 -- 172.21.15.131:0/4251385484 learned_addr learned my addr 172.21.15.131:0/4251385484 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:59.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.003+0000 7fcbe5e00640 1 --2- 172.21.15.131:0/4251385484 >> v2:172.21.15.187:3300/0 conn(0x7fcbe8145a10 0x7fcbe8072180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:59.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.003+0000 7fcbdd400640 1 --2- 172.21.15.131:0/4251385484 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcbe80712d0 0x7fcbe806d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:59.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.003+0000 7fcbe6800640 1 -- 172.21.15.131:0/4251385484 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcbe80712d0 msgr2=0x7fcbe806d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:59.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.003+0000 7fcbe6800640 1 --2- 172.21.15.131:0/4251385484 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcbe80712d0 0x7fcbe806d250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:59.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.003+0000 7fcbe6800640 1 -- 172.21.15.131:0/4251385484 >> v2:172.21.15.187:3300/0 conn(0x7fcbe8145a10 msgr2=0x7fcbe8072180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:59.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.003+0000 7fcbe6800640 1 --2- 172.21.15.131:0/4251385484 >> v2:172.21.15.187:3300/0 conn(0x7fcbe8145a10 0x7fcbe8072180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:59.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.003+0000 7fcbe6800640 1 -- 172.21.15.131:0/4251385484 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbe80ceab0 con 0x7fcbe806d790 2024-05-26T22:48:59.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.003+0000 7fcbe5e00640 1 --2- 172.21.15.131:0/4251385484 >> v2:172.21.15.187:3300/0 conn(0x7fcbe8145a10 0x7fcbe8072180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:48:59.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.004+0000 7fcbe6800640 1 --2- 172.21.15.131:0/4251385484 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcbe806d790 0x7fcbe815fd40 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fcbe806f770 tx=0x7fcbe002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1153beafdcc9824d server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:59.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.004+0000 7fcbe5400640 1 -- 172.21.15.131:0/4251385484 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbe0037070 con 0x7fcbe806d790 2024-05-26T22:48:59.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.004+0000 7fcbe5400640 1 -- 172.21.15.131:0/4251385484 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbe0032440 con 0x7fcbe806d790 2024-05-26T22:48:59.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.004+0000 7fcbe5400640 1 -- 172.21.15.131:0/4251385484 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbe0032810 con 0x7fcbe806d790 2024-05-26T22:48:59.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.005+0000 7fcbe7200640 1 -- 172.21.15.131:0/4251385484 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcbe806d790 msgr2=0x7fcbe815fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:59.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.005+0000 7fcbe7200640 1 --2- 172.21.15.131:0/4251385484 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcbe806d790 0x7fcbe815fd40 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fcbe806f770 tx=0x7fcbe002a1e0 comp rx=0 tx=0).stop 2024-05-26T22:48:59.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.005+0000 7fcbe7200640 1 -- 172.21.15.131:0/4251385484 shutdown_connections 2024-05-26T22:48:59.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.005+0000 7fcbe7200640 1 --2- 172.21.15.131:0/4251385484 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcbe806d790 0x7fcbe815fd40 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:59.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.005+0000 7fcbe7200640 1 --2- 172.21.15.131:0/4251385484 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcbe80712d0 0x7fcbe806d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:59.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.005+0000 7fcbe7200640 1 --2- 172.21.15.131:0/4251385484 >> v2:172.21.15.187:3300/0 conn(0x7fcbe8145a10 0x7fcbe8072180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:59.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.005+0000 7fcbe7200640 1 -- 172.21.15.131:0/4251385484 >> 172.21.15.131:0/4251385484 conn(0x7fcbe80b5660 msgr2=0x7fcbe80b5a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:59.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.005+0000 7fcbe7200640 1 -- 172.21.15.131:0/4251385484 shutdown_connections 2024-05-26T22:48:59.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.005+0000 7fcbe7200640 1 -- 172.21.15.131:0/4251385484 wait complete. 2024-05-26T22:48:59.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.007+0000 7fcbe7200640 1 Processor -- start 2024-05-26T22:48:59.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.007+0000 7fcbe7200640 1 -- start start 2024-05-26T22:48:59.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.008+0000 7fcbe7200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcbe806d790 0x7fcbe8159fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:59.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.008+0000 7fcbe7200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcbe80712d0 0x7fcbe813a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:59.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.008+0000 7fcbe7200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcbe8145a10 0x7fcbe813be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:59.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.008+0000 7fcbe7200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fcbe8145df0 con 0x7fcbe80712d0 2024-05-26T22:48:59.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.008+0000 7fcbe7200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fcbe80cf070 con 0x7fcbe806d790 2024-05-26T22:48:59.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.008+0000 7fcbe7200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fcbe80726c0 con 0x7fcbe8145a10 2024-05-26T22:48:59.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.008+0000 7fcbdd400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcbe80712d0 0x7fcbe813a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:59.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.008+0000 7fcbdd400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcbe80712d0 0x7fcbe813a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43930/0 (socket says 172.21.15.131:43930) 2024-05-26T22:48:59.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.008+0000 7fcbdd400640 1 -- 172.21.15.131:0/169710157 learned_addr learned my addr 172.21.15.131:0/169710157 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:59.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.008+0000 7fcbe5e00640 1 --2- 172.21.15.131:0/169710157 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcbe806d790 0x7fcbe8159fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:59.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.009+0000 7fcbe6800640 1 --2- 172.21.15.131:0/169710157 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcbe8145a10 0x7fcbe813be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:59.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.009+0000 7fcbdd400640 1 -- 172.21.15.131:0/169710157 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcbe8145a10 msgr2=0x7fcbe813be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:59.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.009+0000 7fcbdd400640 1 --2- 172.21.15.131:0/169710157 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcbe8145a10 0x7fcbe813be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:59.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.009+0000 7fcbdd400640 1 -- 172.21.15.131:0/169710157 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcbe806d790 msgr2=0x7fcbe8159fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:59.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.009+0000 7fcbdd400640 1 --2- 172.21.15.131:0/169710157 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcbe806d790 0x7fcbe8159fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:59.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.009+0000 7fcbdd400640 1 -- 172.21.15.131:0/169710157 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbe815a520 con 0x7fcbe80712d0 2024-05-26T22:48:59.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.009+0000 7fcbe5e00640 1 --2- 172.21.15.131:0/169710157 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcbe806d790 0x7fcbe8159fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-26T22:48:59.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.009+0000 7fcbdd400640 1 --2- 172.21.15.131:0/169710157 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcbe80712d0 0x7fcbe813a550 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fcbd000c7f0 tx=0x7fcbd000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:59.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.009+0000 7fcbdea00640 1 -- 172.21.15.131:0/169710157 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbd0015070 con 0x7fcbe80712d0 2024-05-26T22:48:59.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.009+0000 7fcbdea00640 1 -- 172.21.15.131:0/169710157 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbd0004220 con 0x7fcbe80712d0 2024-05-26T22:48:59.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.009+0000 7fcbdea00640 1 -- 172.21.15.131:0/169710157 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbd0007620 con 0x7fcbe80712d0 2024-05-26T22:48:59.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.009+0000 7fcbe7200640 1 -- 172.21.15.131:0/169710157 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbe813af30 con 0x7fcbe80712d0 2024-05-26T22:48:59.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.010+0000 7fcbe7200640 1 -- 172.21.15.131:0/169710157 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbe813b4f0 con 0x7fcbe80712d0 2024-05-26T22:48:59.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.011+0000 7fcbe7200640 1 -- 172.21.15.131:0/169710157 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbe80b2ca0 con 0x7fcbe80712d0 2024-05-26T22:48:59.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.012+0000 7fcbdea00640 1 -- 172.21.15.131:0/169710157 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcbd00079d0 con 0x7fcbe80712d0 2024-05-26T22:48:59.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.012+0000 7fcbdea00640 1 --2- 172.21.15.131:0/169710157 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fcbc406be60 0x7fcbc406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:59.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.013+0000 7fcbdea00640 1 -- 172.21.15.131:0/169710157 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fcbd008af10 con 0x7fcbe80712d0 2024-05-26T22:48:59.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.013+0000 7fcbe5e00640 1 --2- 172.21.15.131:0/169710157 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fcbc406be60 0x7fcbc406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:59.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.013+0000 7fcbe5e00640 1 --2- 172.21.15.131:0/169710157 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fcbc406be60 0x7fcbc406e320 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fcbd800e3d0 tx=0x7fcbd8007a10 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:59.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.017+0000 7fcbdea00640 1 -- 172.21.15.131:0/169710157 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fcbe80b2ca0 con 0x7fcbe80712d0 2024-05-26T22:48:59.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.214+0000 7fcbe7200640 1 -- 172.21.15.131:0/169710157 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fcbe80694e0 con 0x7fcbe80712d0 2024-05-26T22:48:59.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.215+0000 7fcbdea00640 1 -- 172.21.15.131:0/169710157 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7fcbe815a520 con 0x7fcbe80712d0 2024-05-26T22:48:59.215 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 64 2024-05-26T22:48:59.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.218+0000 7fcbe7200640 1 -- 172.21.15.131:0/169710157 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fcbc406be60 msgr2=0x7fcbc406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:59.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.218+0000 7fcbe7200640 1 --2- 172.21.15.131:0/169710157 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fcbc406be60 0x7fcbc406e320 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fcbd800e3d0 tx=0x7fcbd8007a10 comp rx=0 tx=0).stop 2024-05-26T22:48:59.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.218+0000 7fcbe7200640 1 -- 172.21.15.131:0/169710157 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcbe80712d0 msgr2=0x7fcbe813a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:59.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.218+0000 7fcbe7200640 1 --2- 172.21.15.131:0/169710157 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcbe80712d0 0x7fcbe813a550 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fcbd000c7f0 tx=0x7fcbd000ccb0 comp rx=0 tx=0).stop 2024-05-26T22:48:59.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.218+0000 7fcbe7200640 1 -- 172.21.15.131:0/169710157 shutdown_connections 2024-05-26T22:48:59.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.218+0000 7fcbe7200640 1 --2- 172.21.15.131:0/169710157 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcbe8145a10 0x7fcbe813be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:59.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.218+0000 7fcbe7200640 1 --2- 172.21.15.131:0/169710157 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fcbc406be60 0x7fcbc406e320 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:59.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.218+0000 7fcbe7200640 1 --2- 172.21.15.131:0/169710157 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcbe80712d0 0x7fcbe813a550 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:59.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.218+0000 7fcbe7200640 1 --2- 172.21.15.131:0/169710157 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcbe806d790 0x7fcbe8159fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:59.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.218+0000 7fcbe7200640 1 -- 172.21.15.131:0/169710157 >> 172.21.15.131:0/169710157 conn(0x7fcbe80b5660 msgr2=0x7fcbe8166150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:59.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.218+0000 7fcbe7200640 1 -- 172.21.15.131:0/169710157 shutdown_connections 2024-05-26T22:48:59.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.218+0000 7fcbe7200640 1 -- 172.21.15.131:0/169710157 wait complete. 2024-05-26T22:48:59.236 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-26T22:48:59.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.423+0000 7f1d6a400640 1 Processor -- start 2024-05-26T22:48:59.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.423+0000 7f1d6a400640 1 -- start start 2024-05-26T22:48:59.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.424+0000 7f1d6a400640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f1d64144340 0x7f1d640cedf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:59.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.425+0000 7f1d6a400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f1d6406ebc0 0x7f1d640b3d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:59.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.425+0000 7f1d6a400640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d64072000 con 0x7f1d64071b70 2024-05-26T22:48:59.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.425+0000 7f1d6a400640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f1d64072eb0 con 0x7f1d6406ebc0 2024-05-26T22:48:59.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.425+0000 7f1d6a400640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f1d640722a0 con 0x7f1d64144340 2024-05-26T22:48:59.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.425+0000 7f1d69a00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f1d6406ebc0 0x7f1d640b3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:59.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.425+0000 7f1d69a00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f1d6406ebc0 0x7f1d640b3d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43944/0 (socket says 172.21.15.131:43944) 2024-05-26T22:48:59.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.425+0000 7f1d69a00640 1 -- 172.21.15.131:0/4221475953 learned_addr learned my addr 172.21.15.131:0/4221475953 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:59.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.425+0000 7f1d69000640 1 --2- 172.21.15.131:0/4221475953 >> v2:172.21.15.187:3300/0 conn(0x7f1d64144340 0x7f1d640cedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:59.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.425+0000 7f1d69a00640 1 -- 172.21.15.131:0/4221475953 >> v2:172.21.15.187:3300/0 conn(0x7f1d64144340 msgr2=0x7f1d640cedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:59.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.425+0000 7f1d69a00640 1 --2- 172.21.15.131:0/4221475953 >> v2:172.21.15.187:3300/0 conn(0x7f1d64144340 0x7f1d640cedf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:59.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.425+0000 7f1d69a00640 1 -- 172.21.15.131:0/4221475953 >> v1:172.21.15.187:6789/0 conn(0x7f1d64071b70 legacy=0x7f1d6406e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:59.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.425+0000 7f1d69a00640 1 -- 172.21.15.131:0/4221475953 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d64072580 con 0x7f1d6406ebc0 2024-05-26T22:48:59.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.427+0000 7f1d69a00640 1 --2- 172.21.15.131:0/4221475953 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d6406ebc0 0x7f1d640b3d20 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f1d640b2bd0 tx=0x7f1d5c02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71b6bfa6e2e79829 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:59.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.427+0000 7f1d63400640 1 -- 172.21.15.131:0/4221475953 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d5c036070 con 0x7f1d6406ebc0 2024-05-26T22:48:59.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.427+0000 7f1d63400640 1 -- 172.21.15.131:0/4221475953 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d5c031600 con 0x7f1d6406ebc0 2024-05-26T22:48:59.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.427+0000 7f1d63400640 1 -- 172.21.15.131:0/4221475953 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d5c0319d0 con 0x7f1d6406ebc0 2024-05-26T22:48:59.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.427+0000 7f1d6a400640 1 -- 172.21.15.131:0/4221475953 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d6406ebc0 msgr2=0x7f1d640b3d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:59.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.427+0000 7f1d6a400640 1 --2- 172.21.15.131:0/4221475953 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d6406ebc0 0x7f1d640b3d20 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f1d640b2bd0 tx=0x7f1d5c02a6f0 comp rx=0 tx=0).stop 2024-05-26T22:48:59.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.428+0000 7f1d6a400640 1 -- 172.21.15.131:0/4221475953 shutdown_connections 2024-05-26T22:48:59.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.428+0000 7f1d6a400640 1 --2- 172.21.15.131:0/4221475953 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d6406ebc0 0x7f1d640b3d20 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:59.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.428+0000 7f1d6a400640 1 --2- 172.21.15.131:0/4221475953 >> v2:172.21.15.187:3300/0 conn(0x7f1d64144340 0x7f1d640cedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:59.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.428+0000 7f1d6a400640 1 -- 172.21.15.131:0/4221475953 >> 172.21.15.131:0/4221475953 conn(0x7f1d640cff40 msgr2=0x7f1d640d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:59.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.428+0000 7f1d6a400640 1 -- 172.21.15.131:0/4221475953 shutdown_connections 2024-05-26T22:48:59.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.428+0000 7f1d6a400640 1 -- 172.21.15.131:0/4221475953 wait complete. 2024-05-26T22:48:59.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.430+0000 7f1d6a400640 1 Processor -- start 2024-05-26T22:48:59.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.430+0000 7f1d6a400640 1 -- start start 2024-05-26T22:48:59.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.431+0000 7f1d6a400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1d6406ebc0 0x7f1d641406f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:59.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.431+0000 7f1d6a400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d64071b70 0x7f1d641f4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:59.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.431+0000 7f1d6a400640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1d64144340 0x7f1d6415a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:59.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.431+0000 7f1d6a400640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f1d640b4260 con 0x7f1d64071b70 2024-05-26T22:48:59.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.431+0000 7f1d6a400640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1d64072b40 con 0x7f1d64144340 2024-05-26T22:48:59.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.431+0000 7f1d6a400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1d640b43e0 con 0x7f1d6406ebc0 2024-05-26T22:48:59.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.431+0000 7f1d63e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d64071b70 0x7f1d641f4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:59.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.431+0000 7f1d63e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d64071b70 0x7f1d641f4fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43960/0 (socket says 172.21.15.131:43960) 2024-05-26T22:48:59.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.431+0000 7f1d63e00640 1 -- 172.21.15.131:0/904680545 learned_addr learned my addr 172.21.15.131:0/904680545 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:59.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.432+0000 7f1d69000640 1 --2- 172.21.15.131:0/904680545 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1d6406ebc0 0x7f1d641406f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:59.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.432+0000 7f1d69a00640 1 --2- 172.21.15.131:0/904680545 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1d64144340 0x7f1d6415a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:59.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.432+0000 7f1d63e00640 1 -- 172.21.15.131:0/904680545 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1d6406ebc0 msgr2=0x7f1d641406f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:59.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.432+0000 7f1d63e00640 1 --2- 172.21.15.131:0/904680545 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1d6406ebc0 0x7f1d641406f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:59.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.432+0000 7f1d63e00640 1 -- 172.21.15.131:0/904680545 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1d64144340 msgr2=0x7f1d6415a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:59.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.432+0000 7f1d63e00640 1 --2- 172.21.15.131:0/904680545 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1d64144340 0x7f1d6415a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:59.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.432+0000 7f1d63e00640 1 -- 172.21.15.131:0/904680545 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d64140db0 con 0x7f1d64071b70 2024-05-26T22:48:59.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.432+0000 7f1d63e00640 1 --2- 172.21.15.131:0/904680545 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d64071b70 0x7f1d641f4fa0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f1d5000c440 tx=0x7f1d5000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:59.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.432+0000 7f1d60c00640 1 -- 172.21.15.131:0/904680545 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d50017070 con 0x7f1d64071b70 2024-05-26T22:48:59.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.432+0000 7f1d60c00640 1 -- 172.21.15.131:0/904680545 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d500035f0 con 0x7f1d64071b70 2024-05-26T22:48:59.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.432+0000 7f1d60c00640 1 -- 172.21.15.131:0/904680545 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d500039e0 con 0x7f1d64071b70 2024-05-26T22:48:59.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.432+0000 7f1d6a400640 1 -- 172.21.15.131:0/904680545 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d6415bdb0 con 0x7f1d64071b70 2024-05-26T22:48:59.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.433+0000 7f1d6a400640 1 -- 172.21.15.131:0/904680545 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d6415c370 con 0x7f1d64071b70 2024-05-26T22:48:59.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.434+0000 7f1d6a400640 1 -- 172.21.15.131:0/904680545 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d640694e0 con 0x7f1d64071b70 2024-05-26T22:48:59.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.435+0000 7f1d60c00640 1 -- 172.21.15.131:0/904680545 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1d50003b80 con 0x7f1d64071b70 2024-05-26T22:48:59.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.436+0000 7f1d60c00640 1 --2- 172.21.15.131:0/904680545 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f1d4406be30 0x7f1d4406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:59.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.436+0000 7f1d60c00640 1 -- 172.21.15.131:0/904680545 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f1d5008a110 con 0x7f1d64071b70 2024-05-26T22:48:59.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.436+0000 7f1d69000640 1 --2- 172.21.15.131:0/904680545 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f1d4406be30 0x7f1d4406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:59.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.437+0000 7f1d69000640 1 --2- 172.21.15.131:0/904680545 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f1d4406be30 0x7f1d4406e2f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1d58006fd0 tx=0x7f1d58008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:59.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.441+0000 7f1d60c00640 1 -- 172.21.15.131:0/904680545 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f1d50058aa0 con 0x7f1d64071b70 2024-05-26T22:48:59.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.644+0000 7f1d6a400640 1 -- 172.21.15.131:0/904680545 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1d640ce720 con 0x7f1d64071b70 2024-05-26T22:48:59.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.647+0000 7f1d60c00640 1 -- 172.21.15.131:0/904680545 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16629 (secure 0 0 0) 0x7f1d500010c0 con 0x7f1d64071b70 2024-05-26T22:48:59.648 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:48:59.648 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":20,"fsid":"46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f","created":"2024-05-26T22:47:57.288905+0000","modified":"2024-05-26T22:48:22.799298+0000","last_up_change":"2024-05-26T22:48:09.602207+0000","last_in_change":"2024-05-26T22:48:00.687341+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-26T22:48:10.463339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-26T22:48:19.663254+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-26T22:48:20.235751+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"814eaf62-a18d-494b-8e27-fd8fb88adea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6801","nonce":2350940051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6803","nonce":2350940051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6807","nonce":2350940051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6805","nonce":2350940051}]},"public_addr":"172.21.15.131:6801/2350940051","cluster_addr":"172.21.15.131:6803/2350940051","heartbeat_back_addr":"172.21.15.131:6807/2350940051","heartbeat_front_addr":"172.21.15.131:6805/2350940051","state":["exists","up"]},{"osd":1,"uuid":"42e66287-f6c6-4e26-af3c-6dfaa3536d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6825","nonce":1525528247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6827","nonce":1525528247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6831","nonce":1525528247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6829","nonce":1525528247}]},"public_addr":"172.21.15.131:6825/1525528247","cluster_addr":"172.21.15.131:6827/1525528247","heartbeat_back_addr":"172.21.15.131:6831/1525528247","heartbeat_front_addr":"172.21.15.131:6829/1525528247","state":["exists","up"]},{"osd":2,"uuid":"0da34673-04a2-4fd0-b57a-0ff2b679b043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6817","nonce":3627842210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6819","nonce":3627842210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6823","nonce":3627842210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6821","nonce":3627842210}]},"public_addr":"172.21.15.131:6817/3627842210","cluster_addr":"172.21.15.131:6819/3627842210","heartbeat_back_addr":"172.21.15.131:6823/3627842210","heartbeat_front_addr":"172.21.15.131:6821/3627842210","state":["exists","up"]},{"osd":3,"uuid":"cf117a27-d6b2-43d3-ae52-b47988bf8eed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6809","nonce":350790591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6811","nonce":350790591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6815","nonce":350790591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6813","nonce":350790591}]},"public_addr":"172.21.15.131:6809/350790591","cluster_addr":"172.21.15.131:6811/350790591","heartbeat_back_addr":"172.21.15.131:6815/350790591","heartbeat_front_addr":"172.21.15.131:6813/350790591","state":["exists","up"]},{"osd":4,"uuid":"a092ccfc-be89-4a84-88d5-0f781fa715eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6816","nonce":244560995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6819","nonce":244560995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6823","nonce":244560995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6821","nonce":244560995}]},"public_addr":"172.21.15.187:6816/244560995","cluster_addr":"172.21.15.187:6819/244560995","heartbeat_back_addr":"172.21.15.187:6823/244560995","heartbeat_front_addr":"172.21.15.187:6821/244560995","state":["exists","up"]},{"osd":5,"uuid":"7513ba69-dd00-4a35-b190-9e8cc3b2bdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6801","nonce":324619398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6803","nonce":324619398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6812","nonce":324619398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6807","nonce":324619398}]},"public_addr":"172.21.15.187:6801/324619398","cluster_addr":"172.21.15.187:6803/324619398","heartbeat_back_addr":"172.21.15.187:6812/324619398","heartbeat_front_addr":"172.21.15.187:6807/324619398","state":["exists","up"]},{"osd":6,"uuid":"50081810-c093-4d2f-a4d7-b7c695c5400d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6806","nonce":1391347786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6809","nonce":1391347786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6818","nonce":1391347786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6813","nonce":1391347786}]},"public_addr":"172.21.15.187:6806/1391347786","cluster_addr":"172.21.15.187:6809/1391347786","heartbeat_back_addr":"172.21.15.187:6818/1391347786","heartbeat_front_addr":"172.21.15.187:6813/1391347786","state":["exists","up"]},{"osd":7,"uuid":"775dae05-54f5-4ad4-bbce-92863c6ed2ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6825","nonce":1220065622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6827","nonce":1220065622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6831","nonce":1220065622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6829","nonce":1220065622}]},"public_addr":"172.21.15.187:6825/1220065622","cluster_addr":"172.21.15.187:6827/1220065622","heartbeat_back_addr":"172.21.15.187:6831/1220065622","heartbeat_front_addr":"172.21.15.187:6829/1220065622","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:05.983520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.299516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.067323+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.307748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.018742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.258775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.010912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.295355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-26T22:48:59.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.650+0000 7f1d6a400640 1 -- 172.21.15.131:0/904680545 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f1d4406be30 msgr2=0x7f1d4406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:59.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.650+0000 7f1d6a400640 1 --2- 172.21.15.131:0/904680545 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f1d4406be30 0x7f1d4406e2f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1d58006fd0 tx=0x7f1d58008040 comp rx=0 tx=0).stop 2024-05-26T22:48:59.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.650+0000 7f1d6a400640 1 -- 172.21.15.131:0/904680545 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d64071b70 msgr2=0x7f1d641f4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:59.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.650+0000 7f1d6a400640 1 --2- 172.21.15.131:0/904680545 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d64071b70 0x7f1d641f4fa0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f1d5000c440 tx=0x7f1d5000c900 comp rx=0 tx=0).stop 2024-05-26T22:48:59.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.650+0000 7f1d6a400640 1 -- 172.21.15.131:0/904680545 shutdown_connections 2024-05-26T22:48:59.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.650+0000 7f1d6a400640 1 --2- 172.21.15.131:0/904680545 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f1d4406be30 0x7f1d4406e2f0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:59.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.650+0000 7f1d6a400640 1 --2- 172.21.15.131:0/904680545 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1d64144340 0x7f1d6415a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:59.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.650+0000 7f1d6a400640 1 --2- 172.21.15.131:0/904680545 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d64071b70 0x7f1d641f4fa0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:59.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.650+0000 7f1d6a400640 1 --2- 172.21.15.131:0/904680545 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1d6406ebc0 0x7f1d641406f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:59.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.650+0000 7f1d6a400640 1 -- 172.21.15.131:0/904680545 >> 172.21.15.131:0/904680545 conn(0x7f1d640cff40 msgr2=0x7f1d640d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:59.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.650+0000 7f1d6a400640 1 -- 172.21.15.131:0/904680545 shutdown_connections 2024-05-26T22:48:59.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.650+0000 7f1d6a400640 1 -- 172.21.15.131:0/904680545 wait complete. 2024-05-26T22:48:59.669 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-26T22:48:10.463339+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-26T22:48:19.663254+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-26T22:48:20.235751+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-26T22:48:59.670 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-26T22:48:59.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.874+0000 7f3193400640 1 Processor -- start 2024-05-26T22:48:59.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.875+0000 7f3193400640 1 -- start start 2024-05-26T22:48:59.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.877+0000 7f3193400640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f318c0b5450 0x7f318c0b43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:59.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.877+0000 7f3193400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f318c0d0460 0x7f318c06d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:59.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.877+0000 7f3193400640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f318c0b4900 con 0x7f318c06f420 2024-05-26T22:48:59.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.877+0000 7f3193400640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f318c160d90 con 0x7f318c0d0460 2024-05-26T22:48:59.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.877+0000 7f3193400640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f318c160180 con 0x7f318c0b5450 2024-05-26T22:48:59.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.877+0000 7f3191400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f318c0d0460 0x7f318c06d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:59.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.877+0000 7f3191400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f318c0d0460 0x7f318c06d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43970/0 (socket says 172.21.15.131:43970) 2024-05-26T22:48:59.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.877+0000 7f318be00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f318c06f420 0x7f318c0cfd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:48140/0 (socket says 172.21.15.131:48140) 2024-05-26T22:48:59.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.877+0000 7f3191400640 1 -- 172.21.15.131:0/2054966188 learned_addr learned my addr 172.21.15.131:0/2054966188 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:59.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.877+0000 7f3190a00640 1 --2- 172.21.15.131:0/2054966188 >> v2:172.21.15.187:3300/0 conn(0x7f318c0b5450 0x7f318c0b43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:59.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.878+0000 7f3191400640 1 -- 172.21.15.131:0/2054966188 >> v2:172.21.15.187:3300/0 conn(0x7f318c0b5450 msgr2=0x7f318c0b43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:48:59.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.878+0000 7f3191400640 1 --2- 172.21.15.131:0/2054966188 >> v2:172.21.15.187:3300/0 conn(0x7f318c0b5450 0x7f318c0b43c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:59.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.878+0000 7f3191400640 1 -- 172.21.15.131:0/2054966188 >> v1:172.21.15.131:6789/0 conn(0x7f318c06f420 legacy=0x7f318c0cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:59.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.878+0000 7f3191400640 1 -- 172.21.15.131:0/2054966188 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f318c160460 con 0x7f318c0d0460 2024-05-26T22:48:59.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.879+0000 7f3191400640 1 --2- 172.21.15.131:0/2054966188 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f318c0d0460 0x7f318c06d600 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3184009870 tx=0x7f318402a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dd37f2c3938e57 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:59.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.879+0000 7f318b400640 1 -- 172.21.15.131:0/2054966188 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3184037070 con 0x7f318c0d0460 2024-05-26T22:48:59.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.879+0000 7f318b400640 1 -- 172.21.15.131:0/2054966188 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3184032440 con 0x7f318c0d0460 2024-05-26T22:48:59.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.879+0000 7f318b400640 1 -- 172.21.15.131:0/2054966188 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3184032810 con 0x7f318c0d0460 2024-05-26T22:48:59.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.880+0000 7f3193400640 1 -- 172.21.15.131:0/2054966188 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f318c0d0460 msgr2=0x7f318c06d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:59.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.880+0000 7f3193400640 1 --2- 172.21.15.131:0/2054966188 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f318c0d0460 0x7f318c06d600 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3184009870 tx=0x7f318402a2a0 comp rx=0 tx=0).stop 2024-05-26T22:48:59.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.880+0000 7f3193400640 1 -- 172.21.15.131:0/2054966188 shutdown_connections 2024-05-26T22:48:59.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.880+0000 7f3193400640 1 --2- 172.21.15.131:0/2054966188 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f318c0d0460 0x7f318c06d600 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:59.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.880+0000 7f3193400640 1 --2- 172.21.15.131:0/2054966188 >> v2:172.21.15.187:3300/0 conn(0x7f318c0b5450 0x7f318c0b43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:59.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.880+0000 7f3193400640 1 -- 172.21.15.131:0/2054966188 >> 172.21.15.131:0/2054966188 conn(0x7f318c06bc90 msgr2=0x7f318c0b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:48:59.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.880+0000 7f3193400640 1 -- 172.21.15.131:0/2054966188 shutdown_connections 2024-05-26T22:48:59.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.880+0000 7f3193400640 1 -- 172.21.15.131:0/2054966188 wait complete. 2024-05-26T22:48:59.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.881+0000 7f3193400640 1 Processor -- start 2024-05-26T22:48:59.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.882+0000 7f3193400640 1 -- start start 2024-05-26T22:48:59.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.882+0000 7f3193400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f318c06f420 0x7f318c153e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:59.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.883+0000 7f3193400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f318c0b5450 0x7f318c15a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:59.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.883+0000 7f3193400640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f318c0d0460 0x7f318c15a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:59.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.883+0000 7f3193400640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f318c0b5830 con 0x7f318c06f420 2024-05-26T22:48:59.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.883+0000 7f3193400640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f318c160a20 con 0x7f318c0d0460 2024-05-26T22:48:59.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.883+0000 7f3193400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f318c15fde0 con 0x7f318c0b5450 2024-05-26T22:48:59.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.883+0000 7f3190a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f318c06f420 0x7f318c153e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:59.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.883+0000 7f3190a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f318c06f420 0x7f318c153e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43984/0 (socket says 172.21.15.131:43984) 2024-05-26T22:48:59.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.883+0000 7f3190a00640 1 -- 172.21.15.131:0/535924712 learned_addr learned my addr 172.21.15.131:0/535924712 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:48:59.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.883+0000 7f3190a00640 1 -- 172.21.15.131:0/535924712 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f318c0b5450 msgr2=0x7f318c15a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:59.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.883+0000 7f3190a00640 1 --2- 172.21.15.131:0/535924712 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f318c0b5450 0x7f318c15a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:59.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.883+0000 7f3190a00640 1 -- 172.21.15.131:0/535924712 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f318c0d0460 msgr2=0x7f318c15a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:48:59.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.883+0000 7f3191400640 1 --2- 172.21.15.131:0/535924712 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f318c0d0460 0x7f318c15a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:59.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.883+0000 7f3190a00640 1 --2- 172.21.15.131:0/535924712 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f318c0d0460 0x7f318c15a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:48:59.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.883+0000 7f3190a00640 1 -- 172.21.15.131:0/535924712 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f318c1e6830 con 0x7f318c06f420 2024-05-26T22:48:59.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.884+0000 7f3190a00640 1 --2- 172.21.15.131:0/535924712 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f318c06f420 0x7f318c153e30 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f317c00d830 tx=0x7f317c00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:59.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.884+0000 7f3188c00640 1 -- 172.21.15.131:0/535924712 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f317c00b6c0 con 0x7f318c06f420 2024-05-26T22:48:59.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.884+0000 7f3188c00640 1 -- 172.21.15.131:0/535924712 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f317c019070 con 0x7f318c06f420 2024-05-26T22:48:59.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.884+0000 7f3193400640 1 -- 172.21.15.131:0/535924712 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f318c1e6b10 con 0x7f318c06f420 2024-05-26T22:48:59.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.884+0000 7f3188c00640 1 -- 172.21.15.131:0/535924712 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f317c00ba80 con 0x7f318c06f420 2024-05-26T22:48:59.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.884+0000 7f3193400640 1 -- 172.21.15.131:0/535924712 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f318c15cbb0 con 0x7f318c06f420 2024-05-26T22:48:59.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.885+0000 7f3193400640 1 -- 172.21.15.131:0/535924712 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f318c074e90 con 0x7f318c06f420 2024-05-26T22:48:59.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.886+0000 7f3188c00640 1 -- 172.21.15.131:0/535924712 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f317c00bc20 con 0x7f318c06f420 2024-05-26T22:48:59.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.888+0000 7f3188c00640 1 --2- 172.21.15.131:0/535924712 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f316806be60 0x7f316806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:48:59.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.888+0000 7f3188c00640 1 -- 172.21.15.131:0/535924712 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f317c08b240 con 0x7f318c06f420 2024-05-26T22:48:59.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.888+0000 7f318be00640 1 --2- 172.21.15.131:0/535924712 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f316806be60 0x7f316806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:48:59.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.889+0000 7f318be00640 1 --2- 172.21.15.131:0/535924712 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f316806be60 0x7f316806e320 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f3174007fd0 tx=0x7f3174007d80 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:48:59.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:48:59.892+0000 7f3188c00640 1 -- 172.21.15.131:0/535924712 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f317c059c80 con 0x7f318c06f420 2024-05-26T22:49:00.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.092+0000 7f3193400640 1 -- 172.21.15.131:0/535924712 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f318c0695c0 con 0x7f318c06f420 2024-05-26T22:49:00.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.093+0000 7f3188c00640 1 -- 172.21.15.131:0/535924712 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f317c059580 con 0x7f318c06f420 2024-05-26T22:49:00.093 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2024-05-26T22:49:00.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.095+0000 7f3193400640 1 -- 172.21.15.131:0/535924712 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f316806be60 msgr2=0x7f316806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:00.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.095+0000 7f3193400640 1 --2- 172.21.15.131:0/535924712 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f316806be60 0x7f316806e320 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f3174007fd0 tx=0x7f3174007d80 comp rx=0 tx=0).stop 2024-05-26T22:49:00.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.095+0000 7f3193400640 1 -- 172.21.15.131:0/535924712 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f318c06f420 msgr2=0x7f318c153e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:00.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.095+0000 7f3193400640 1 --2- 172.21.15.131:0/535924712 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f318c06f420 0x7f318c153e30 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f317c00d830 tx=0x7f317c00dcf0 comp rx=0 tx=0).stop 2024-05-26T22:49:00.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.095+0000 7f3193400640 1 -- 172.21.15.131:0/535924712 shutdown_connections 2024-05-26T22:49:00.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.095+0000 7f3193400640 1 --2- 172.21.15.131:0/535924712 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f318c0b5450 0x7f318c15a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:00.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.095+0000 7f3193400640 1 --2- 172.21.15.131:0/535924712 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f316806be60 0x7f316806e320 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:00.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.095+0000 7f3193400640 1 --2- 172.21.15.131:0/535924712 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f318c0d0460 0x7f318c15a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:00.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.095+0000 7f3193400640 1 --2- 172.21.15.131:0/535924712 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f318c06f420 0x7f318c153e30 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:00.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.095+0000 7f3193400640 1 -- 172.21.15.131:0/535924712 >> 172.21.15.131:0/535924712 conn(0x7f318c06bc90 msgr2=0x7f318c0cd120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:00.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.096+0000 7f3193400640 1 -- 172.21.15.131:0/535924712 shutdown_connections 2024-05-26T22:49:00.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.096+0000 7f3193400640 1 -- 172.21.15.131:0/535924712 wait complete. 2024-05-26T22:49:00.114 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-26T22:49:00.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.321+0000 7f4904000640 1 Processor -- start 2024-05-26T22:49:00.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.321+0000 7f4904000640 1 -- start start 2024-05-26T22:49:00.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.322+0000 7f4904000640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f48fc06f180 0x7f48fc072e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:00.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.322+0000 7f4904000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48fc072540 0x7f48fc161ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:00.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.322+0000 7f4904000640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48fc0af8e0 con 0x7f48fc0cfe50 2024-05-26T22:49:00.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.322+0000 7f4904000640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f48fc0cfb30 con 0x7f48fc06f180 2024-05-26T22:49:00.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.322+0000 7f4904000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f48fc0cf200 con 0x7f48fc072540 2024-05-26T22:49:00.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.322+0000 7f4902000640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f48fc0cfe50 0x7f48fc0715a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:48150/0 (socket says 172.21.15.131:48150) 2024-05-26T22:49:00.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.322+0000 7f4902000640 1 -- 172.21.15.131:0/3477557212 learned_addr learned my addr 172.21.15.131:0/3477557212 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:49:00.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.323+0000 7f4900c00640 1 --2- 172.21.15.131:0/3477557212 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48fc072540 0x7f48fc161ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:00.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.323+0000 7f4901600640 1 --2- 172.21.15.131:0/3477557212 >> v2:172.21.15.187:3300/0 conn(0x7f48fc06f180 0x7f48fc072e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:00.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.323+0000 7f48fbe00640 1 -- 172.21.15.131:0/3477557212 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3498620692 0 0) 0x7f48fc0af8e0 con 0x7f48fc0cfe50 2024-05-26T22:49:00.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.323+0000 7f48fbe00640 1 -- 172.21.15.131:0/3477557212 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f48e4003520 con 0x7f48fc0cfe50 2024-05-26T22:49:00.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.323+0000 7f48fbe00640 1 -- 172.21.15.131:0/3477557212 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 995291014 0 0) 0x7f48e4003520 con 0x7f48fc0cfe50 2024-05-26T22:49:00.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.323+0000 7f48fbe00640 1 -- 172.21.15.131:0/3477557212 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48fc072540 msgr2=0x7f48fc161ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:00.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.323+0000 7f48fbe00640 1 --2- 172.21.15.131:0/3477557212 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48fc072540 0x7f48fc161ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:00.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.323+0000 7f48fbe00640 1 -- 172.21.15.131:0/3477557212 >> v2:172.21.15.187:3300/0 conn(0x7f48fc06f180 msgr2=0x7f48fc072e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:00.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.323+0000 7f48fbe00640 1 --2- 172.21.15.131:0/3477557212 >> v2:172.21.15.187:3300/0 conn(0x7f48fc06f180 0x7f48fc072e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:00.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.323+0000 7f48fbe00640 1 -- 172.21.15.131:0/3477557212 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48fc0cf4e0 con 0x7f48fc0cfe50 2024-05-26T22:49:00.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.323+0000 7f48fbe00640 1 -- 172.21.15.131:0/3477557212 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7f48f40019f0 con 0x7f48fc0cfe50 2024-05-26T22:49:00.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.323+0000 7f48fbe00640 1 -- 172.21.15.131:0/3477557212 >> v1:172.21.15.131:6789/0 conn(0x7f48fc0cfe50 legacy=0x7f48fc0715a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:00.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.323+0000 7f48fbe00640 1 --2- 172.21.15.131:0/3477557212 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48fc072540 0x7f48e40039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:00.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.323+0000 7f48fbe00640 1 --2- 172.21.15.131:0/3477557212 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48e4003f30 0x7f48e4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:00.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.323+0000 7f48fbe00640 1 --2- 172.21.15.131:0/3477557212 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f48e4004d40 0x7f48e40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:00.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.324+0000 7f48fbe00640 1 -- 172.21.15.131:0/3477557212 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f48fc0cfb30 con 0x7f48e4003f30 2024-05-26T22:49:00.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.324+0000 7f48fbe00640 1 -- 172.21.15.131:0/3477557212 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f48fc0cf200 con 0x7f48e4004d40 2024-05-26T22:49:00.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.324+0000 7f48fbe00640 1 -- 172.21.15.131:0/3477557212 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f48e400b9c0 con 0x7f48fc072540 2024-05-26T22:49:00.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.324+0000 7f4900c00640 1 --2- 172.21.15.131:0/3477557212 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48e4003f30 0x7f48e4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:00.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.324+0000 7f4901600640 1 --2- 172.21.15.131:0/3477557212 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48fc072540 0x7f48e40039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:00.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.324+0000 7f4900c00640 1 -- 172.21.15.131:0/3477557212 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48fc072540 msgr2=0x7f48e40039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:00.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.324+0000 7f4900c00640 1 --2- 172.21.15.131:0/3477557212 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48fc072540 0x7f48e40039b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:00.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.324+0000 7f4900c00640 1 -- 172.21.15.131:0/3477557212 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f48e4004d40 msgr2=0x7f48e40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:00.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.324+0000 7f4900c00640 1 --2- 172.21.15.131:0/3477557212 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f48e4004d40 0x7f48e40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:00.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.324+0000 7f4900c00640 1 -- 172.21.15.131:0/3477557212 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48e4003520 con 0x7f48e4003f30 2024-05-26T22:49:00.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.325+0000 7f4900c00640 1 --2- 172.21.15.131:0/3477557212 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48e4003f30 0x7f48e4004800 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f48e8009ac0 tx=0x7f48e4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5602ee28dcb931ad server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:00.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.325+0000 7f48fbe00640 1 -- 172.21.15.131:0/3477557212 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48e8036070 con 0x7f48e4003f30 2024-05-26T22:49:00.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.325+0000 7f48fbe00640 1 -- 172.21.15.131:0/3477557212 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48e8031400 con 0x7f48e4003f30 2024-05-26T22:49:00.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.325+0000 7f48fbe00640 1 -- 172.21.15.131:0/3477557212 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48e8009690 con 0x7f48e4003f30 2024-05-26T22:49:00.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.326+0000 7f4904000640 1 -- 172.21.15.131:0/3477557212 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48e4003f30 msgr2=0x7f48e4004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:00.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.326+0000 7f4904000640 1 --2- 172.21.15.131:0/3477557212 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48e4003f30 0x7f48e4004800 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f48e8009ac0 tx=0x7f48e4005830 comp rx=0 tx=0).stop 2024-05-26T22:49:00.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.326+0000 7f4902000640 1 -- 172.21.15.131:0/3477557212 reap_dead start 2024-05-26T22:49:00.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.326+0000 7f4904000640 1 -- 172.21.15.131:0/3477557212 shutdown_connections 2024-05-26T22:49:00.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.326+0000 7f4904000640 1 -- 172.21.15.131:0/3477557212 >> 172.21.15.131:0/3477557212 conn(0x7f48fc06bc90 msgr2=0x7f48fc0b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:00.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.326+0000 7f4904000640 1 -- 172.21.15.131:0/3477557212 shutdown_connections 2024-05-26T22:49:00.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.326+0000 7f4904000640 1 -- 172.21.15.131:0/3477557212 wait complete. 2024-05-26T22:49:00.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.328+0000 7f4904000640 1 Processor -- start 2024-05-26T22:49:00.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.328+0000 7f4904000640 1 -- start start 2024-05-26T22:49:00.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.329+0000 7f4904000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48e4003f30 0x7f48fc0b55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:00.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.329+0000 7f4904000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48fc150590 0x7f48fc0cc010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:00.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.329+0000 7f4904000640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f48fc09d290 0x7f48fc09d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:00.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.329+0000 7f4904000640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f48fc06f6e0 con 0x7f48fc150590 2024-05-26T22:49:00.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.329+0000 7f4904000640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f48fc06f560 con 0x7f48fc09d290 2024-05-26T22:49:00.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.329+0000 7f4904000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f48fc161ff0 con 0x7f48e4003f30 2024-05-26T22:49:00.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.329+0000 7f4900c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48fc150590 0x7f48fc0cc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:00.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.329+0000 7f4900c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48fc150590 0x7f48fc0cc010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43988/0 (socket says 172.21.15.131:43988) 2024-05-26T22:49:00.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.329+0000 7f4900c00640 1 -- 172.21.15.131:0/549174915 learned_addr learned my addr 172.21.15.131:0/549174915 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:00.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.330+0000 7f4901600640 1 --2- 172.21.15.131:0/549174915 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48e4003f30 0x7f48fc0b55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:00.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.330+0000 7f4900c00640 1 -- 172.21.15.131:0/549174915 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48e4003f30 msgr2=0x7f48fc0b55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:00.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.330+0000 7f4902000640 1 --2- 172.21.15.131:0/549174915 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f48fc09d290 0x7f48fc09d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:00.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.330+0000 7f4900c00640 1 --2- 172.21.15.131:0/549174915 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48e4003f30 0x7f48fc0b55d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:00.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.330+0000 7f4900c00640 1 -- 172.21.15.131:0/549174915 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f48fc09d290 msgr2=0x7f48fc09d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:00.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.330+0000 7f4900c00640 1 --2- 172.21.15.131:0/549174915 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f48fc09d290 0x7f48fc09d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:00.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.330+0000 7f4900c00640 1 -- 172.21.15.131:0/549174915 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48fc0b3100 con 0x7f48fc150590 2024-05-26T22:49:00.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.330+0000 7f4902000640 1 --2- 172.21.15.131:0/549174915 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f48fc09d290 0x7f48fc09d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:49:00.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.330+0000 7f4900c00640 1 --2- 172.21.15.131:0/549174915 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48fc150590 0x7f48fc0cc010 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f48e8034040 tx=0x7f48e8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:00.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.331+0000 7f48f9600640 1 -- 172.21.15.131:0/549174915 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48e8040020 con 0x7f48fc150590 2024-05-26T22:49:00.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.331+0000 7f48f9600640 1 -- 172.21.15.131:0/549174915 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48e8003c60 con 0x7f48fc150590 2024-05-26T22:49:00.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.331+0000 7f48f9600640 1 -- 172.21.15.131:0/549174915 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48e8036040 con 0x7f48fc150590 2024-05-26T22:49:00.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.331+0000 7f4904000640 1 -- 172.21.15.131:0/549174915 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48fc0b33e0 con 0x7f48fc150590 2024-05-26T22:49:00.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.331+0000 7f4904000640 1 -- 172.21.15.131:0/549174915 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48fc0b36c0 con 0x7f48fc150590 2024-05-26T22:49:00.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.332+0000 7f4904000640 1 -- 172.21.15.131:0/549174915 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48fc074e90 con 0x7f48fc150590 2024-05-26T22:49:00.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.333+0000 7f48f9600640 1 -- 172.21.15.131:0/549174915 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f48e8043020 con 0x7f48fc150590 2024-05-26T22:49:00.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.334+0000 7f48f9600640 1 --2- 172.21.15.131:0/549174915 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f48dc06be80 0x7f48dc06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:00.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.334+0000 7f48f9600640 1 -- 172.21.15.131:0/549174915 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f48e80aa980 con 0x7f48fc150590 2024-05-26T22:49:00.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.334+0000 7f4901600640 1 --2- 172.21.15.131:0/549174915 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f48dc06be80 0x7f48dc06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:00.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.335+0000 7f4901600640 1 --2- 172.21.15.131:0/549174915 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f48dc06be80 0x7f48dc06e340 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f48f0009990 tx=0x7f48f0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:00.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.339+0000 7f48f9600640 1 -- 172.21.15.131:0/549174915 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f48e80044e0 con 0x7f48fc150590 2024-05-26T22:49:00.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.543+0000 7f4904000640 1 -- 172.21.15.131:0/549174915 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f48fc0695c0 con 0x7f48fc150590 2024-05-26T22:49:00.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.544+0000 7f48f9600640 1 -- 172.21.15.131:0/549174915 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f48e8079310 con 0x7f48fc150590 2024-05-26T22:49:00.543 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-05-26T22:49:00.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.546+0000 7f4904000640 1 -- 172.21.15.131:0/549174915 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f48dc06be80 msgr2=0x7f48dc06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:00.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.546+0000 7f4904000640 1 --2- 172.21.15.131:0/549174915 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f48dc06be80 0x7f48dc06e340 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f48f0009990 tx=0x7f48f0008040 comp rx=0 tx=0).stop 2024-05-26T22:49:00.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.547+0000 7f4904000640 1 -- 172.21.15.131:0/549174915 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48fc150590 msgr2=0x7f48fc0cc010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:00.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.547+0000 7f4904000640 1 --2- 172.21.15.131:0/549174915 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48fc150590 0x7f48fc0cc010 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f48e8034040 tx=0x7f48e8004730 comp rx=0 tx=0).stop 2024-05-26T22:49:00.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.547+0000 7f4904000640 1 -- 172.21.15.131:0/549174915 shutdown_connections 2024-05-26T22:49:00.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.547+0000 7f4904000640 1 --2- 172.21.15.131:0/549174915 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f48dc06be80 0x7f48dc06e340 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:00.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.547+0000 7f4904000640 1 --2- 172.21.15.131:0/549174915 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f48fc09d290 0x7f48fc09d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:00.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.547+0000 7f4904000640 1 --2- 172.21.15.131:0/549174915 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48fc150590 0x7f48fc0cc010 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:00.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.547+0000 7f4904000640 1 --2- 172.21.15.131:0/549174915 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48e4003f30 0x7f48fc0b55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:00.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.547+0000 7f4904000640 1 -- 172.21.15.131:0/549174915 >> 172.21.15.131:0/549174915 conn(0x7f48fc06bc90 msgr2=0x7f48fc15c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:00.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.547+0000 7f4904000640 1 -- 172.21.15.131:0/549174915 shutdown_connections 2024-05-26T22:49:00.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.547+0000 7f4904000640 1 -- 172.21.15.131:0/549174915 wait complete. 2024-05-26T22:49:00.567 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-26T22:49:00.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.767+0000 7f75bfe00640 1 Processor -- start 2024-05-26T22:49:00.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.767+0000 7f75bfe00640 1 -- start start 2024-05-26T22:49:00.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.768+0000 7f75bfe00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f75c00b9f20 0x7f75c00d22b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:00.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.768+0000 7f75bfe00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f75c00d27f0 0x7f75c00b8d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:00.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.768+0000 7f75bfe00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75c00b79f0 con 0x7f75c015e340 2024-05-26T22:49:00.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.768+0000 7f75bfe00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f75c00b88b0 con 0x7f75c00d27f0 2024-05-26T22:49:00.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.768+0000 7f75bfe00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f75c015e720 con 0x7f75c00b9f20 2024-05-26T22:49:00.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.768+0000 7f75bea00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f75c015e340 0x7f75c00b72e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:48156/0 (socket says 172.21.15.131:48156) 2024-05-26T22:49:00.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.768+0000 7f75bea00640 1 -- 172.21.15.131:0/751776875 learned_addr learned my addr 172.21.15.131:0/751776875 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:49:00.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.768+0000 7f75bf400640 1 --2- 172.21.15.131:0/751776875 >> v2:172.21.15.131:3300/0 conn(0x7f75c00d27f0 0x7f75c00b8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:00.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.768+0000 7f75be000640 1 --2- 172.21.15.131:0/751776875 >> v2:172.21.15.187:3300/0 conn(0x7f75c00b9f20 0x7f75c00d22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:00.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.768+0000 7f75bd600640 1 -- 172.21.15.131:0/751776875 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2860467320 0 0) 0x7f75c00b79f0 con 0x7f75c015e340 2024-05-26T22:49:00.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.768+0000 7f75bd600640 1 -- 172.21.15.131:0/751776875 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f75a4003520 con 0x7f75c015e340 2024-05-26T22:49:00.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.768+0000 7f75bf400640 1 -- 172.21.15.131:0/751776875 >> v2:172.21.15.187:3300/0 conn(0x7f75c00b9f20 msgr2=0x7f75c00d22b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:00.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.768+0000 7f75bf400640 1 --2- 172.21.15.131:0/751776875 >> v2:172.21.15.187:3300/0 conn(0x7f75c00b9f20 0x7f75c00d22b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:00.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.768+0000 7f75bf400640 1 -- 172.21.15.131:0/751776875 >> v1:172.21.15.131:6789/0 conn(0x7f75c015e340 legacy=0x7f75c00b72e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:00.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.769+0000 7f75bf400640 1 -- 172.21.15.131:0/751776875 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75c00b7ca0 con 0x7f75c00d27f0 2024-05-26T22:49:00.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.769+0000 7f75be000640 1 --2- 172.21.15.131:0/751776875 >> v2:172.21.15.187:3300/0 conn(0x7f75c00b9f20 0x7f75c00d22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:49:00.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.770+0000 7f75bf400640 1 --2- 172.21.15.131:0/751776875 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75c00d27f0 0x7f75c00b8d40 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f75c00d1ac0 tx=0x7f75b802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7f69529db9f6ca0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:00.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.770+0000 7f75bd600640 1 -- 172.21.15.131:0/751776875 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75b8007740 con 0x7f75c00d27f0 2024-05-26T22:49:00.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.770+0000 7f75bd600640 1 -- 172.21.15.131:0/751776875 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75b80078e0 con 0x7f75c00d27f0 2024-05-26T22:49:00.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.770+0000 7f75bd600640 1 -- 172.21.15.131:0/751776875 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75b8007cb0 con 0x7f75c00d27f0 2024-05-26T22:49:00.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.770+0000 7f75bfe00640 1 -- 172.21.15.131:0/751776875 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75c00d27f0 msgr2=0x7f75c00b8d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:00.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.770+0000 7f75bfe00640 1 --2- 172.21.15.131:0/751776875 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75c00d27f0 0x7f75c00b8d40 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f75c00d1ac0 tx=0x7f75b802a710 comp rx=0 tx=0).stop 2024-05-26T22:49:00.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.771+0000 7f75bfe00640 1 -- 172.21.15.131:0/751776875 shutdown_connections 2024-05-26T22:49:00.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.771+0000 7f75bfe00640 1 --2- 172.21.15.131:0/751776875 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75c00d27f0 0x7f75c00b8d40 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:00.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.771+0000 7f75bfe00640 1 --2- 172.21.15.131:0/751776875 >> v2:172.21.15.187:3300/0 conn(0x7f75c00b9f20 0x7f75c00d22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:00.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.771+0000 7f75bfe00640 1 -- 172.21.15.131:0/751776875 >> 172.21.15.131:0/751776875 conn(0x7f75c0072e60 msgr2=0x7f75c0073270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:00.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.771+0000 7f75bfe00640 1 -- 172.21.15.131:0/751776875 shutdown_connections 2024-05-26T22:49:00.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.771+0000 7f75bfe00640 1 -- 172.21.15.131:0/751776875 wait complete. 2024-05-26T22:49:00.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.772+0000 7f75bfe00640 1 Processor -- start 2024-05-26T22:49:00.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.773+0000 7f75bfe00640 1 -- start start 2024-05-26T22:49:00.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.773+0000 7f75bfe00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f75c00b9f20 0x7f75c014f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:00.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.774+0000 7f75bfe00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f75c00d27f0 0x7f75c00c8c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:00.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.774+0000 7f75bfe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75c015e340 0x7f75c00c91b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:00.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.774+0000 7f75bfe00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f75c00b8540 con 0x7f75c015e340 2024-05-26T22:49:00.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.774+0000 7f75bfe00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f75c00b8260 con 0x7f75c00b9f20 2024-05-26T22:49:00.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.774+0000 7f75bfe00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f75c00b9280 con 0x7f75c00d27f0 2024-05-26T22:49:00.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.774+0000 7f75bf400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75c015e340 0x7f75c00c91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:00.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.774+0000 7f75bf400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75c015e340 0x7f75c00c91b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44000/0 (socket says 172.21.15.131:44000) 2024-05-26T22:49:00.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.774+0000 7f75bf400640 1 -- 172.21.15.131:0/1390496181 learned_addr learned my addr 172.21.15.131:0/1390496181 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:00.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.774+0000 7f75be000640 1 --2- 172.21.15.131:0/1390496181 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f75c00d27f0 0x7f75c00c8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:00.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.774+0000 7f75bea00640 1 --2- 172.21.15.131:0/1390496181 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f75c00b9f20 0x7f75c014f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:00.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.774+0000 7f75bf400640 1 -- 172.21.15.131:0/1390496181 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f75c00d27f0 msgr2=0x7f75c00c8c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:00.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.774+0000 7f75bf400640 1 --2- 172.21.15.131:0/1390496181 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f75c00d27f0 0x7f75c00c8c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:00.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.774+0000 7f75bf400640 1 -- 172.21.15.131:0/1390496181 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f75c00b9f20 msgr2=0x7f75c014f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:00.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.774+0000 7f75bf400640 1 --2- 172.21.15.131:0/1390496181 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f75c00b9f20 0x7f75c014f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:00.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.774+0000 7f75bf400640 1 -- 172.21.15.131:0/1390496181 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75c00c8080 con 0x7f75c015e340 2024-05-26T22:49:00.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.775+0000 7f75bf400640 1 --2- 172.21.15.131:0/1390496181 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75c015e340 0x7f75c00c91b0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f75b802ad00 tx=0x7f75b8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:00.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.775+0000 7f75b6a00640 1 -- 172.21.15.131:0/1390496181 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75b8040070 con 0x7f75c015e340 2024-05-26T22:49:00.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.775+0000 7f75b6a00640 1 -- 172.21.15.131:0/1390496181 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75b8031bc0 con 0x7f75c015e340 2024-05-26T22:49:00.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.775+0000 7f75bfe00640 1 -- 172.21.15.131:0/1390496181 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75c00c9810 con 0x7f75c015e340 2024-05-26T22:49:00.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.775+0000 7f75b6a00640 1 -- 172.21.15.131:0/1390496181 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75b8009450 con 0x7f75c015e340 2024-05-26T22:49:00.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.775+0000 7f75bfe00640 1 -- 172.21.15.131:0/1390496181 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75c00c9dd0 con 0x7f75c015e340 2024-05-26T22:49:00.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.776+0000 7f75bfe00640 1 -- 172.21.15.131:0/1390496181 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75c00694e0 con 0x7f75c015e340 2024-05-26T22:49:00.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.777+0000 7f75b6a00640 1 -- 172.21.15.131:0/1390496181 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f75b8039040 con 0x7f75c015e340 2024-05-26T22:49:00.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.778+0000 7f75b6a00640 1 --2- 172.21.15.131:0/1390496181 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f759c06be30 0x7f759c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:00.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.778+0000 7f75b6a00640 1 -- 172.21.15.131:0/1390496181 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f75b80ac7e0 con 0x7f75c015e340 2024-05-26T22:49:00.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.778+0000 7f75bea00640 1 --2- 172.21.15.131:0/1390496181 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f759c06be30 0x7f759c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:00.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.779+0000 7f75bea00640 1 --2- 172.21.15.131:0/1390496181 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f759c06be30 0x7f759c06e2f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f75b0005880 tx=0x7f75b0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:00.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.783+0000 7f75b6a00640 1 -- 172.21.15.131:0/1390496181 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f75b803ae60 con 0x7f75c015e340 2024-05-26T22:49:00.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.983+0000 7f75bfe00640 1 -- 172.21.15.131:0/1390496181 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f75c00cf8c0 con 0x7f75c015e340 2024-05-26T22:49:00.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.984+0000 7f75b6a00640 1 -- 172.21.15.131:0/1390496181 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f75b807b0f0 con 0x7f75c015e340 2024-05-26T22:49:00.985 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 64 2024-05-26T22:49:00.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.987+0000 7f75bfe00640 1 -- 172.21.15.131:0/1390496181 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f759c06be30 msgr2=0x7f759c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:00.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.987+0000 7f75bfe00640 1 --2- 172.21.15.131:0/1390496181 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f759c06be30 0x7f759c06e2f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f75b0005880 tx=0x7f75b0009040 comp rx=0 tx=0).stop 2024-05-26T22:49:00.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.987+0000 7f75bfe00640 1 -- 172.21.15.131:0/1390496181 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75c015e340 msgr2=0x7f75c00c91b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:00.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.987+0000 7f75bfe00640 1 --2- 172.21.15.131:0/1390496181 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75c015e340 0x7f75c00c91b0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f75b802ad00 tx=0x7f75b8004a40 comp rx=0 tx=0).stop 2024-05-26T22:49:00.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.987+0000 7f75bfe00640 1 -- 172.21.15.131:0/1390496181 shutdown_connections 2024-05-26T22:49:00.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.987+0000 7f75bfe00640 1 --2- 172.21.15.131:0/1390496181 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f75c00d27f0 0x7f75c00c8c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:00.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.987+0000 7f75bfe00640 1 --2- 172.21.15.131:0/1390496181 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f759c06be30 0x7f759c06e2f0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:00.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.987+0000 7f75bfe00640 1 --2- 172.21.15.131:0/1390496181 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75c015e340 0x7f75c00c91b0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:00.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.987+0000 7f75bfe00640 1 --2- 172.21.15.131:0/1390496181 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f75c00b9f20 0x7f75c014f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:00.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.987+0000 7f75bfe00640 1 -- 172.21.15.131:0/1390496181 >> 172.21.15.131:0/1390496181 conn(0x7f75c0072e60 msgr2=0x7f75c00ce7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:00.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.987+0000 7f75bfe00640 1 -- 172.21.15.131:0/1390496181 shutdown_connections 2024-05-26T22:49:00.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:00.987+0000 7f75bfe00640 1 -- 172.21.15.131:0/1390496181 wait complete. 2024-05-26T22:49:01.034 INFO:tasks.cephfs_test_runner:Starting test: test_ephemeral_pin_dist_conf_off (tasks.cephfs.test_exports.TestEphemeralPins) That turning off ephemeral distributed pin config prevents distribution. 2024-05-26T22:49:01.034 INFO:tasks.cephfs_test_runner:test_ephemeral_pin_dist_conf_off (tasks.cephfs.test_exports.TestEphemeralPins) 2024-05-26T22:49:01.036 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-26T22:49:01.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.243+0000 7fa011600640 1 Processor -- start 2024-05-26T22:49:01.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.243+0000 7fa011600640 1 -- start start 2024-05-26T22:49:01.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.244+0000 7fa011600640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa00c15c3d0 0x7fa00c06ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:01.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.244+0000 7fa011600640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fa00c06f490 0x7fa00c0ade40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:01.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.244+0000 7fa011600640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa00c0722a0 con 0x7fa00c155840 2024-05-26T22:49:01.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.244+0000 7fa011600640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fa00c072040 con 0x7fa00c06f490 2024-05-26T22:49:01.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.244+0000 7fa011600640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa00c072580 con 0x7fa00c15c3d0 2024-05-26T22:49:01.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.245+0000 7fa00b400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fa00c06f490 0x7fa00c0ade40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:01.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.245+0000 7fa00aa00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fa00c155840 0x7fa00c071930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:48162/0 (socket says 172.21.15.131:48162) 2024-05-26T22:49:01.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.245+0000 7fa00aa00640 1 -- 172.21.15.131:0/2124511951 learned_addr learned my addr 172.21.15.131:0/2124511951 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:49:01.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.245+0000 7fa00a000640 1 --2- 172.21.15.131:0/2124511951 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa00c15c3d0 0x7fa00c06ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:01.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.245+0000 7fa00b400640 1 -- 172.21.15.131:0/2124511951 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa00c15c3d0 msgr2=0x7fa00c06ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:01.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.245+0000 7fa00b400640 1 --2- 172.21.15.131:0/2124511951 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa00c15c3d0 0x7fa00c06ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:01.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.245+0000 7fa00b400640 1 -- 172.21.15.131:0/2124511951 >> v1:172.21.15.131:6789/0 conn(0x7fa00c155840 legacy=0x7fa00c071930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:01.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.245+0000 7fa00b400640 1 -- 172.21.15.131:0/2124511951 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa00c072860 con 0x7fa00c06f490 2024-05-26T22:49:01.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.245+0000 7fa00a000640 1 --2- 172.21.15.131:0/2124511951 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa00c15c3d0 0x7fa00c06ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:49:01.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.246+0000 7fa00b400640 1 --2- 172.21.15.131:0/2124511951 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa00c06f490 0x7fa00c0ade40 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa004009d90 tx=0x7fa00402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e46d59dafc9ac6c0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:01.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.246+0000 7fa009600640 1 -- 172.21.15.131:0/2124511951 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa004007740 con 0x7fa00c06f490 2024-05-26T22:49:01.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.246+0000 7fa009600640 1 -- 172.21.15.131:0/2124511951 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0040078e0 con 0x7fa00c06f490 2024-05-26T22:49:01.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.247+0000 7fa009600640 1 -- 172.21.15.131:0/2124511951 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa004007cb0 con 0x7fa00c06f490 2024-05-26T22:49:01.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.247+0000 7fa011600640 1 -- 172.21.15.131:0/2124511951 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa00c06f490 msgr2=0x7fa00c0ade40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:01.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.247+0000 7fa011600640 1 --2- 172.21.15.131:0/2124511951 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa00c06f490 0x7fa00c0ade40 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa004009d90 tx=0x7fa00402a660 comp rx=0 tx=0).stop 2024-05-26T22:49:01.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.247+0000 7fa011600640 1 -- 172.21.15.131:0/2124511951 shutdown_connections 2024-05-26T22:49:01.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.247+0000 7fa011600640 1 --2- 172.21.15.131:0/2124511951 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa00c06f490 0x7fa00c0ade40 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:01.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.247+0000 7fa011600640 1 --2- 172.21.15.131:0/2124511951 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa00c15c3d0 0x7fa00c06ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:01.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.247+0000 7fa011600640 1 -- 172.21.15.131:0/2124511951 >> 172.21.15.131:0/2124511951 conn(0x7fa00c0b1e90 msgr2=0x7fa00c0b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:01.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.247+0000 7fa011600640 1 -- 172.21.15.131:0/2124511951 shutdown_connections 2024-05-26T22:49:01.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.247+0000 7fa011600640 1 -- 172.21.15.131:0/2124511951 wait complete. 2024-05-26T22:49:01.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.249+0000 7fa011600640 1 Processor -- start 2024-05-26T22:49:01.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.249+0000 7fa011600640 1 -- start start 2024-05-26T22:49:01.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.250+0000 7fa011600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa00c06f490 0x7fa00c151030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:01.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.250+0000 7fa011600640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa00c155840 0x7fa00c156a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:01.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.250+0000 7fa011600640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa00c15c3d0 0x7fa00c167320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:01.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.250+0000 7fa011600640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa00c0ae380 con 0x7fa00c06f490 2024-05-26T22:49:01.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.250+0000 7fa011600640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa00c072e20 con 0x7fa00c15c3d0 2024-05-26T22:49:01.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.250+0000 7fa011600640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa00c0ae500 con 0x7fa00c155840 2024-05-26T22:49:01.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.250+0000 7fa00aa00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa00c06f490 0x7fa00c151030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:01.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.250+0000 7fa00aa00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa00c06f490 0x7fa00c151030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44028/0 (socket says 172.21.15.131:44028) 2024-05-26T22:49:01.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.250+0000 7fa00aa00640 1 -- 172.21.15.131:0/4194715017 learned_addr learned my addr 172.21.15.131:0/4194715017 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:01.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.251+0000 7fa00a000640 1 --2- 172.21.15.131:0/4194715017 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa00c155840 0x7fa00c156a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:01.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.251+0000 7fa00b400640 1 --2- 172.21.15.131:0/4194715017 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa00c15c3d0 0x7fa00c167320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:01.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.251+0000 7fa00aa00640 1 -- 172.21.15.131:0/4194715017 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa00c155840 msgr2=0x7fa00c156a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:01.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.251+0000 7fa00aa00640 1 --2- 172.21.15.131:0/4194715017 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa00c155840 0x7fa00c156a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:01.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.251+0000 7fa00aa00640 1 -- 172.21.15.131:0/4194715017 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa00c15c3d0 msgr2=0x7fa00c167320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:01.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.251+0000 7fa00aa00640 1 --2- 172.21.15.131:0/4194715017 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa00c15c3d0 0x7fa00c167320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:01.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.251+0000 7fa00aa00640 1 -- 172.21.15.131:0/4194715017 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa00c1f8520 con 0x7fa00c06f490 2024-05-26T22:49:01.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.251+0000 7fa00aa00640 1 --2- 172.21.15.131:0/4194715017 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa00c06f490 0x7fa00c151030 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f9ffc00c430 tx=0x7f9ffc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:01.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.252+0000 7fa002a00640 1 -- 172.21.15.131:0/4194715017 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ffc017070 con 0x7fa00c06f490 2024-05-26T22:49:01.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.252+0000 7fa002a00640 1 -- 172.21.15.131:0/4194715017 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ffc005ce0 con 0x7fa00c06f490 2024-05-26T22:49:01.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.252+0000 7fa002a00640 1 -- 172.21.15.131:0/4194715017 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ffc0072f0 con 0x7fa00c06f490 2024-05-26T22:49:01.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.252+0000 7fa011600640 1 -- 172.21.15.131:0/4194715017 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa00c150550 con 0x7fa00c06f490 2024-05-26T22:49:01.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.252+0000 7fa011600640 1 -- 172.21.15.131:0/4194715017 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa00c1f81b0 con 0x7fa00c06f490 2024-05-26T22:49:01.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.253+0000 7fa011600640 1 -- 172.21.15.131:0/4194715017 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa00c15bca0 con 0x7fa00c06f490 2024-05-26T22:49:01.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.254+0000 7fa002a00640 1 -- 172.21.15.131:0/4194715017 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9ffc01a070 con 0x7fa00c06f490 2024-05-26T22:49:01.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.255+0000 7fa002a00640 1 --2- 172.21.15.131:0/4194715017 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9fec06be30 0x7f9fec06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:01.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.255+0000 7fa002a00640 1 -- 172.21.15.131:0/4194715017 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f9ffc08a300 con 0x7fa00c06f490 2024-05-26T22:49:01.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.255+0000 7fa00a000640 1 --2- 172.21.15.131:0/4194715017 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9fec06be30 0x7f9fec06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:01.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.256+0000 7fa00a000640 1 --2- 172.21.15.131:0/4194715017 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9fec06be30 0x7f9fec06e2f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa00c159b80 tx=0x7f9ff8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:01.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.261+0000 7fa002a00640 1 -- 172.21.15.131:0/4194715017 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fa00c15bca0 con 0x7fa00c06f490 2024-05-26T22:49:01.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.460+0000 7fa011600640 1 -- 172.21.15.131:0/4194715017 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa00c0694e0 con 0x7fa00c06f490 2024-05-26T22:49:01.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.464+0000 7fa002a00640 1 -- 172.21.15.131:0/4194715017 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16629 (secure 0 0 0) 0x7f9ffc058c10 con 0x7fa00c06f490 2024-05-26T22:49:01.464 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:49:01.464 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":20,"fsid":"46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f","created":"2024-05-26T22:47:57.288905+0000","modified":"2024-05-26T22:48:22.799298+0000","last_up_change":"2024-05-26T22:48:09.602207+0000","last_in_change":"2024-05-26T22:48:00.687341+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-26T22:48:10.463339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-26T22:48:19.663254+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-26T22:48:20.235751+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"814eaf62-a18d-494b-8e27-fd8fb88adea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6801","nonce":2350940051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6803","nonce":2350940051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6807","nonce":2350940051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6805","nonce":2350940051}]},"public_addr":"172.21.15.131:6801/2350940051","cluster_addr":"172.21.15.131:6803/2350940051","heartbeat_back_addr":"172.21.15.131:6807/2350940051","heartbeat_front_addr":"172.21.15.131:6805/2350940051","state":["exists","up"]},{"osd":1,"uuid":"42e66287-f6c6-4e26-af3c-6dfaa3536d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6825","nonce":1525528247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6827","nonce":1525528247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6831","nonce":1525528247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6829","nonce":1525528247}]},"public_addr":"172.21.15.131:6825/1525528247","cluster_addr":"172.21.15.131:6827/1525528247","heartbeat_back_addr":"172.21.15.131:6831/1525528247","heartbeat_front_addr":"172.21.15.131:6829/1525528247","state":["exists","up"]},{"osd":2,"uuid":"0da34673-04a2-4fd0-b57a-0ff2b679b043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6817","nonce":3627842210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6819","nonce":3627842210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6823","nonce":3627842210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6821","nonce":3627842210}]},"public_addr":"172.21.15.131:6817/3627842210","cluster_addr":"172.21.15.131:6819/3627842210","heartbeat_back_addr":"172.21.15.131:6823/3627842210","heartbeat_front_addr":"172.21.15.131:6821/3627842210","state":["exists","up"]},{"osd":3,"uuid":"cf117a27-d6b2-43d3-ae52-b47988bf8eed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6809","nonce":350790591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6811","nonce":350790591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6815","nonce":350790591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6813","nonce":350790591}]},"public_addr":"172.21.15.131:6809/350790591","cluster_addr":"172.21.15.131:6811/350790591","heartbeat_back_addr":"172.21.15.131:6815/350790591","heartbeat_front_addr":"172.21.15.131:6813/350790591","state":["exists","up"]},{"osd":4,"uuid":"a092ccfc-be89-4a84-88d5-0f781fa715eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6816","nonce":244560995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6819","nonce":244560995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6823","nonce":244560995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6821","nonce":244560995}]},"public_addr":"172.21.15.187:6816/244560995","cluster_addr":"172.21.15.187:6819/244560995","heartbeat_back_addr":"172.21.15.187:6823/244560995","heartbeat_front_addr":"172.21.15.187:6821/244560995","state":["exists","up"]},{"osd":5,"uuid":"7513ba69-dd00-4a35-b190-9e8cc3b2bdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6801","nonce":324619398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6803","nonce":324619398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6812","nonce":324619398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6807","nonce":324619398}]},"public_addr":"172.21.15.187:6801/324619398","cluster_addr":"172.21.15.187:6803/324619398","heartbeat_back_addr":"172.21.15.187:6812/324619398","heartbeat_front_addr":"172.21.15.187:6807/324619398","state":["exists","up"]},{"osd":6,"uuid":"50081810-c093-4d2f-a4d7-b7c695c5400d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6806","nonce":1391347786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6809","nonce":1391347786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6818","nonce":1391347786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6813","nonce":1391347786}]},"public_addr":"172.21.15.187:6806/1391347786","cluster_addr":"172.21.15.187:6809/1391347786","heartbeat_back_addr":"172.21.15.187:6818/1391347786","heartbeat_front_addr":"172.21.15.187:6813/1391347786","state":["exists","up"]},{"osd":7,"uuid":"775dae05-54f5-4ad4-bbce-92863c6ed2ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6825","nonce":1220065622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6827","nonce":1220065622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6831","nonce":1220065622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6829","nonce":1220065622}]},"public_addr":"172.21.15.187:6825/1220065622","cluster_addr":"172.21.15.187:6827/1220065622","heartbeat_back_addr":"172.21.15.187:6831/1220065622","heartbeat_front_addr":"172.21.15.187:6829/1220065622","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:05.983520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.299516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.067323+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.307748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.018742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.258775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.010912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.295355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-26T22:49:01.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.466+0000 7fa011600640 1 -- 172.21.15.131:0/4194715017 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9fec06be30 msgr2=0x7f9fec06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:01.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.466+0000 7fa011600640 1 --2- 172.21.15.131:0/4194715017 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9fec06be30 0x7f9fec06e2f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa00c159b80 tx=0x7f9ff8008040 comp rx=0 tx=0).stop 2024-05-26T22:49:01.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.466+0000 7fa011600640 1 -- 172.21.15.131:0/4194715017 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa00c06f490 msgr2=0x7fa00c151030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:01.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.466+0000 7fa011600640 1 --2- 172.21.15.131:0/4194715017 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa00c06f490 0x7fa00c151030 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f9ffc00c430 tx=0x7f9ffc00c8f0 comp rx=0 tx=0).stop 2024-05-26T22:49:01.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.466+0000 7fa011600640 1 -- 172.21.15.131:0/4194715017 shutdown_connections 2024-05-26T22:49:01.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.466+0000 7fa011600640 1 --2- 172.21.15.131:0/4194715017 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa00c155840 0x7fa00c156a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:01.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.466+0000 7fa011600640 1 --2- 172.21.15.131:0/4194715017 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9fec06be30 0x7f9fec06e2f0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:01.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.467+0000 7fa011600640 1 --2- 172.21.15.131:0/4194715017 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa00c15c3d0 0x7fa00c167320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:01.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.467+0000 7fa011600640 1 --2- 172.21.15.131:0/4194715017 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa00c06f490 0x7fa00c151030 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:01.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.467+0000 7fa011600640 1 -- 172.21.15.131:0/4194715017 >> 172.21.15.131:0/4194715017 conn(0x7fa00c0b1e90 msgr2=0x7fa00c0b1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:01.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.467+0000 7fa011600640 1 -- 172.21.15.131:0/4194715017 shutdown_connections 2024-05-26T22:49:01.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.467+0000 7fa011600640 1 -- 172.21.15.131:0/4194715017 wait complete. 2024-05-26T22:49:01.487 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-26T22:48:10.463339+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-26T22:48:19.663254+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-26T22:48:20.235751+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-26T22:49:01.487 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-26T22:49:01.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.684+0000 7f855b200640 1 Processor -- start 2024-05-26T22:49:01.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.684+0000 7f855b200640 1 -- start start 2024-05-26T22:49:01.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.686+0000 7f855b200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f85540af580 0x7f855415f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:01.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.686+0000 7f855b200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f85540af960 0x7f85540ad870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:01.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.686+0000 7f855b200640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85540ae0c0 con 0x7f85541620a0 2024-05-26T22:49:01.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.686+0000 7f855b200640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f85540ae3a0 con 0x7f85540af580 2024-05-26T22:49:01.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.686+0000 7f855b200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f85540ae680 con 0x7f85540af960 2024-05-26T22:49:01.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.686+0000 7f8553e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f85540af580 0x7f855415f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:01.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.687+0000 7f8553e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f85540af580 0x7f855415f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44042/0 (socket says 172.21.15.131:44042) 2024-05-26T22:49:01.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.687+0000 7f8553e00640 1 -- 172.21.15.131:0/1335363508 learned_addr learned my addr 172.21.15.131:0/1335363508 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:01.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.687+0000 7f8553400640 1 --2- 172.21.15.131:0/1335363508 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f85540af960 0x7f85540ad870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:01.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.687+0000 7f8553e00640 1 -- 172.21.15.131:0/1335363508 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f85540af960 msgr2=0x7f85540ad870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:01.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.687+0000 7f8553e00640 1 --2- 172.21.15.131:0/1335363508 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f85540af960 0x7f85540ad870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:01.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.687+0000 7f8553e00640 1 -- 172.21.15.131:0/1335363508 >> v1:172.21.15.187:6789/0 conn(0x7f85541620a0 legacy=0x7f8554162550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:01.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.687+0000 7f8553e00640 1 -- 172.21.15.131:0/1335363508 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85540ae960 con 0x7f85540af580 2024-05-26T22:49:01.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.687+0000 7f8553400640 1 --2- 172.21.15.131:0/1335363508 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f85540af960 0x7f85540ad870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:49:01.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.688+0000 7f8553e00640 1 --2- 172.21.15.131:0/1335363508 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f85540af580 0x7f855415f6e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f8544009e50 tx=0x7f854402a5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d561849b950f52d0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:01.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.688+0000 7f8552a00640 1 -- 172.21.15.131:0/1335363508 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8544036070 con 0x7f85540af580 2024-05-26T22:49:01.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.688+0000 7f8552a00640 1 -- 172.21.15.131:0/1335363508 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8544031600 con 0x7f85540af580 2024-05-26T22:49:01.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.688+0000 7f8552a00640 1 -- 172.21.15.131:0/1335363508 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85440319d0 con 0x7f85540af580 2024-05-26T22:49:01.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.689+0000 7f855b200640 1 -- 172.21.15.131:0/1335363508 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f85540af580 msgr2=0x7f855415f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:01.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.689+0000 7f855b200640 1 --2- 172.21.15.131:0/1335363508 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f85540af580 0x7f855415f6e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f8544009e50 tx=0x7f854402a5b0 comp rx=0 tx=0).stop 2024-05-26T22:49:01.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.689+0000 7f855b200640 1 -- 172.21.15.131:0/1335363508 shutdown_connections 2024-05-26T22:49:01.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.689+0000 7f855b200640 1 --2- 172.21.15.131:0/1335363508 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f85540af960 0x7f85540ad870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:01.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.689+0000 7f855b200640 1 --2- 172.21.15.131:0/1335363508 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f85540af580 0x7f855415f6e0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:01.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.689+0000 7f855b200640 1 -- 172.21.15.131:0/1335363508 >> 172.21.15.131:0/1335363508 conn(0x7f8554072b00 msgr2=0x7f855406de50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:01.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.689+0000 7f855b200640 1 -- 172.21.15.131:0/1335363508 shutdown_connections 2024-05-26T22:49:01.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.689+0000 7f855b200640 1 -- 172.21.15.131:0/1335363508 wait complete. 2024-05-26T22:49:01.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.691+0000 7f855b200640 1 Processor -- start 2024-05-26T22:49:01.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.691+0000 7f855b200640 1 -- start start 2024-05-26T22:49:01.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.692+0000 7f855b200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f85540af580 0x7f855414f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:01.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.692+0000 7f855b200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f85540af960 0x7f85541601b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:01.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.693+0000 7f855b200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f85541620a0 0x7f85541606f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:01.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.693+0000 7f855b200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f85540b68c0 con 0x7f85541620a0 2024-05-26T22:49:01.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.693+0000 7f855b200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f85540afde0 con 0x7f85540af580 2024-05-26T22:49:01.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.693+0000 7f855b200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f85540b6a40 con 0x7f85540af960 2024-05-26T22:49:01.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.693+0000 7f8559200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f85541620a0 0x7f85541606f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:01.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.693+0000 7f8559200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f85541620a0 0x7f85541606f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44058/0 (socket says 172.21.15.131:44058) 2024-05-26T22:49:01.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.693+0000 7f8559200640 1 -- 172.21.15.131:0/2574015103 learned_addr learned my addr 172.21.15.131:0/2574015103 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:01.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.693+0000 7f8553e00640 1 --2- 172.21.15.131:0/2574015103 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f85540af580 0x7f855414f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:01.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.693+0000 7f8553400640 1 --2- 172.21.15.131:0/2574015103 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f85540af960 0x7f85541601b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:01.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.693+0000 7f8559200640 1 -- 172.21.15.131:0/2574015103 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f85540af960 msgr2=0x7f85541601b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:01.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.693+0000 7f8559200640 1 --2- 172.21.15.131:0/2574015103 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f85540af960 0x7f85541601b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:01.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.693+0000 7f8559200640 1 -- 172.21.15.131:0/2574015103 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f85540af580 msgr2=0x7f855414f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:01.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.693+0000 7f8559200640 1 --2- 172.21.15.131:0/2574015103 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f85540af580 0x7f855414f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:01.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.693+0000 7f8559200640 1 -- 172.21.15.131:0/2574015103 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85541f4880 con 0x7f85541620a0 2024-05-26T22:49:01.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.694+0000 7f8559200640 1 --2- 172.21.15.131:0/2574015103 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f85541620a0 0x7f85541606f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f854c00b440 tx=0x7f854c00b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:01.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.694+0000 7f854be00640 1 -- 172.21.15.131:0/2574015103 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f854c015020 con 0x7f85541620a0 2024-05-26T22:49:01.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.694+0000 7f854be00640 1 -- 172.21.15.131:0/2574015103 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f854c011830 con 0x7f85541620a0 2024-05-26T22:49:01.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.694+0000 7f854be00640 1 -- 172.21.15.131:0/2574015103 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f854c011c20 con 0x7f85541620a0 2024-05-26T22:49:01.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.694+0000 7f855b200640 1 -- 172.21.15.131:0/2574015103 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85541f4b60 con 0x7f85541620a0 2024-05-26T22:49:01.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.694+0000 7f855b200640 1 -- 172.21.15.131:0/2574015103 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85541f5120 con 0x7f85541620a0 2024-05-26T22:49:01.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.695+0000 7f855b200640 1 -- 172.21.15.131:0/2574015103 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85541e8c10 con 0x7f85541620a0 2024-05-26T22:49:01.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.696+0000 7f854be00640 1 -- 172.21.15.131:0/2574015103 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f854c018020 con 0x7f85541620a0 2024-05-26T22:49:01.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.697+0000 7f854be00640 1 --2- 172.21.15.131:0/2574015103 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f853006be30 0x7f853006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:01.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.697+0000 7f854be00640 1 -- 172.21.15.131:0/2574015103 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f854c08ab10 con 0x7f85541620a0 2024-05-26T22:49:01.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.697+0000 7f8553e00640 1 --2- 172.21.15.131:0/2574015103 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f853006be30 0x7f853006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:01.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.698+0000 7f8553e00640 1 --2- 172.21.15.131:0/2574015103 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f853006be30 0x7f853006e2f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8544033000 tx=0x7f8544009000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:01.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.701+0000 7f854be00640 1 -- 172.21.15.131:0/2574015103 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f854c01f030 con 0x7f85541620a0 2024-05-26T22:49:01.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.895+0000 7f855b200640 1 -- 172.21.15.131:0/2574015103 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f85540695c0 con 0x7f85541620a0 2024-05-26T22:49:01.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.896+0000 7f854be00640 1 -- 172.21.15.131:0/2574015103 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f854c0172c0 con 0x7f85541620a0 2024-05-26T22:49:01.896 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2024-05-26T22:49:01.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.898+0000 7f855b200640 1 -- 172.21.15.131:0/2574015103 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f853006be30 msgr2=0x7f853006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:01.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.898+0000 7f855b200640 1 --2- 172.21.15.131:0/2574015103 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f853006be30 0x7f853006e2f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8544033000 tx=0x7f8544009000 comp rx=0 tx=0).stop 2024-05-26T22:49:01.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.898+0000 7f855b200640 1 -- 172.21.15.131:0/2574015103 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f85541620a0 msgr2=0x7f85541606f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:01.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.898+0000 7f855b200640 1 --2- 172.21.15.131:0/2574015103 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f85541620a0 0x7f85541606f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f854c00b440 tx=0x7f854c00b900 comp rx=0 tx=0).stop 2024-05-26T22:49:01.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.899+0000 7f855b200640 1 -- 172.21.15.131:0/2574015103 shutdown_connections 2024-05-26T22:49:01.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.899+0000 7f855b200640 1 --2- 172.21.15.131:0/2574015103 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f85540af960 0x7f85541601b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:01.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.899+0000 7f855b200640 1 --2- 172.21.15.131:0/2574015103 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f853006be30 0x7f853006e2f0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:01.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.899+0000 7f855b200640 1 --2- 172.21.15.131:0/2574015103 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f85541620a0 0x7f85541606f0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:01.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.899+0000 7f855b200640 1 --2- 172.21.15.131:0/2574015103 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f85540af580 0x7f855414f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:01.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.899+0000 7f855b200640 1 -- 172.21.15.131:0/2574015103 >> 172.21.15.131:0/2574015103 conn(0x7f8554072b00 msgr2=0x7f85540b3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:01.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.899+0000 7f855b200640 1 -- 172.21.15.131:0/2574015103 shutdown_connections 2024-05-26T22:49:01.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:01.899+0000 7f855b200640 1 -- 172.21.15.131:0/2574015103 wait complete. 2024-05-26T22:49:01.917 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-26T22:49:02.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.103+0000 7f6e98400640 1 Processor -- start 2024-05-26T22:49:02.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.104+0000 7f6e98400640 1 -- start start 2024-05-26T22:49:02.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.105+0000 7f6e98400640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f6e90155770 0x7f6e9015b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:02.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.105+0000 7f6e98400640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e9015c1c0 con 0x7f6e9006e440 2024-05-26T22:49:02.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.105+0000 7f6e98400640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e9015c4a0 con 0x7f6e9015bc40 2024-05-26T22:49:02.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.105+0000 7f6e98400640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f6e90071da0 con 0x7f6e90155770 2024-05-26T22:49:02.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.105+0000 7f6e96400640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f6e9006e440 0x7f6e900725a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:48178/0 (socket says 172.21.15.131:48178) 2024-05-26T22:49:02.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.105+0000 7f6e96400640 1 -- 172.21.15.131:0/292464768 learned_addr learned my addr 172.21.15.131:0/292464768 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:49:02.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.106+0000 7f6e95a00640 1 --2- 172.21.15.131:0/292464768 >> v2:172.21.15.187:3300/0 conn(0x7f6e90155770 0x7f6e9015b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:02.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.106+0000 7f6e8fe00640 1 -- 172.21.15.131:0/292464768 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2319227184 0 0) 0x7f6e9015c1c0 con 0x7f6e9006e440 2024-05-26T22:49:02.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.106+0000 7f6e8fe00640 1 -- 172.21.15.131:0/292464768 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6e74003520 con 0x7f6e9006e440 2024-05-26T22:49:02.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.106+0000 7f6e8fe00640 1 -- 172.21.15.131:0/292464768 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2823388005 0 0) 0x7f6e74003520 con 0x7f6e9006e440 2024-05-26T22:49:02.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.106+0000 7f6e8fe00640 1 -- 172.21.15.131:0/292464768 >> v2:172.21.15.187:3300/0 conn(0x7f6e90155770 msgr2=0x7f6e9015b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:02.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.106+0000 7f6e8fe00640 1 --2- 172.21.15.131:0/292464768 >> v2:172.21.15.187:3300/0 conn(0x7f6e90155770 0x7f6e9015b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:02.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.106+0000 7f6e8fe00640 1 -- 172.21.15.131:0/292464768 >> v1:172.21.15.187:6789/0 conn(0x7f6e9015bc40 legacy=0x7f6e9006dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:02.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.106+0000 7f6e8fe00640 1 -- 172.21.15.131:0/292464768 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e90072080 con 0x7f6e9006e440 2024-05-26T22:49:02.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.106+0000 7f6e8fe00640 1 -- 172.21.15.131:0/292464768 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7f6e880019f0 con 0x7f6e9006e440 2024-05-26T22:49:02.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.106+0000 7f6e8fe00640 1 -- 172.21.15.131:0/292464768 >> v1:172.21.15.131:6789/0 conn(0x7f6e9006e440 legacy=0x7f6e900725a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:02.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.106+0000 7f6e8fe00640 1 --2- 172.21.15.131:0/292464768 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e740039b0 0x7f6e74003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:02.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.106+0000 7f6e8fe00640 1 --2- 172.21.15.131:0/292464768 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6e74004800 0x7f6e74004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:02.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.106+0000 7f6e8fe00640 1 --2- 172.21.15.131:0/292464768 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6e740051c0 0x7f6e7400dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:02.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.106+0000 7f6e8fe00640 1 -- 172.21.15.131:0/292464768 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f6e90071da0 con 0x7f6e740039b0 2024-05-26T22:49:02.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.106+0000 7f6e8fe00640 1 -- 172.21.15.131:0/292464768 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6e7400e280 con 0x7f6e74004800 2024-05-26T22:49:02.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.106+0000 7f6e8fe00640 1 -- 172.21.15.131:0/292464768 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f6e7400e480 con 0x7f6e740051c0 2024-05-26T22:49:02.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.107+0000 7f6e95a00640 1 --2- 172.21.15.131:0/292464768 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e740039b0 0x7f6e74003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:02.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.107+0000 7f6e95a00640 1 -- 172.21.15.131:0/292464768 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6e740051c0 msgr2=0x7f6e7400dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:02.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.107+0000 7f6e95a00640 1 --2- 172.21.15.131:0/292464768 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6e740051c0 0x7f6e7400dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:02.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.107+0000 7f6e95a00640 1 -- 172.21.15.131:0/292464768 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6e74004800 msgr2=0x7f6e74004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:02.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.107+0000 7f6e95a00640 1 --2- 172.21.15.131:0/292464768 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6e74004800 0x7f6e74004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:02.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.107+0000 7f6e95a00640 1 -- 172.21.15.131:0/292464768 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e74003520 con 0x7f6e740039b0 2024-05-26T22:49:02.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.107+0000 7f6e96400640 1 -- 172.21.15.131:0/292464768 reap_dead start 2024-05-26T22:49:02.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.108+0000 7f6e95a00640 1 --2- 172.21.15.131:0/292464768 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e740039b0 0x7f6e74003dd0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f6e80009ac0 tx=0x7f6e74005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b6103bd6ef66132 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:02.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.108+0000 7f6e8fe00640 1 -- 172.21.15.131:0/292464768 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e80036070 con 0x7f6e740039b0 2024-05-26T22:49:02.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.108+0000 7f6e8fe00640 1 -- 172.21.15.131:0/292464768 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e80031400 con 0x7f6e740039b0 2024-05-26T22:49:02.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.108+0000 7f6e8fe00640 1 -- 172.21.15.131:0/292464768 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e80009690 con 0x7f6e740039b0 2024-05-26T22:49:02.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.109+0000 7f6e98400640 1 -- 172.21.15.131:0/292464768 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e740039b0 msgr2=0x7f6e74003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:02.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.109+0000 7f6e98400640 1 --2- 172.21.15.131:0/292464768 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e740039b0 0x7f6e74003dd0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f6e80009ac0 tx=0x7f6e74005830 comp rx=0 tx=0).stop 2024-05-26T22:49:02.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.109+0000 7f6e98400640 1 -- 172.21.15.131:0/292464768 shutdown_connections 2024-05-26T22:49:02.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.109+0000 7f6e98400640 1 --2- 172.21.15.131:0/292464768 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e740039b0 0x7f6e74003dd0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:02.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.109+0000 7f6e98400640 1 -- 172.21.15.131:0/292464768 >> 172.21.15.131:0/292464768 conn(0x7f6e9006bc90 msgr2=0x7f6e900b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:02.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.109+0000 7f6e98400640 1 -- 172.21.15.131:0/292464768 shutdown_connections 2024-05-26T22:49:02.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.109+0000 7f6e98400640 1 -- 172.21.15.131:0/292464768 wait complete. 2024-05-26T22:49:02.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.111+0000 7f6e98400640 1 Processor -- start 2024-05-26T22:49:02.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.111+0000 7f6e98400640 1 -- start start 2024-05-26T22:49:02.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.112+0000 7f6e98400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e740039b0 0x7f6e9014f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:02.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.112+0000 7f6e98400640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6e901487f0 0x7f6e900cf080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:02.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.112+0000 7f6e98400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6e9014c7c0 0x7f6e9015a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:02.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.112+0000 7f6e98400640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f6e9009de00 con 0x7f6e740039b0 2024-05-26T22:49:02.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.112+0000 7f6e98400640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6e9015c8c0 con 0x7f6e901487f0 2024-05-26T22:49:02.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.112+0000 7f6e98400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f6e9009df80 con 0x7f6e9014c7c0 2024-05-26T22:49:02.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.112+0000 7f6e95a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e740039b0 0x7f6e9014f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:02.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.112+0000 7f6e95a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e740039b0 0x7f6e9014f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44080/0 (socket says 172.21.15.131:44080) 2024-05-26T22:49:02.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.112+0000 7f6e95a00640 1 -- 172.21.15.131:0/3220646343 learned_addr learned my addr 172.21.15.131:0/3220646343 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:02.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.112+0000 7f6e95000640 1 --2- 172.21.15.131:0/3220646343 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6e901487f0 0x7f6e900cf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:02.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.113+0000 7f6e96400640 1 --2- 172.21.15.131:0/3220646343 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6e9014c7c0 0x7f6e9015a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:02.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.113+0000 7f6e95a00640 1 -- 172.21.15.131:0/3220646343 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6e9014c7c0 msgr2=0x7f6e9015a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:02.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.113+0000 7f6e95a00640 1 --2- 172.21.15.131:0/3220646343 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6e9014c7c0 0x7f6e9015a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:02.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.113+0000 7f6e95a00640 1 -- 172.21.15.131:0/3220646343 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6e901487f0 msgr2=0x7f6e900cf080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:02.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.113+0000 7f6e95a00640 1 --2- 172.21.15.131:0/3220646343 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6e901487f0 0x7f6e900cf080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:02.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.113+0000 7f6e95a00640 1 -- 172.21.15.131:0/3220646343 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e9014e580 con 0x7f6e740039b0 2024-05-26T22:49:02.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.113+0000 7f6e96400640 1 --2- 172.21.15.131:0/3220646343 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6e9014c7c0 0x7f6e9015a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:49:02.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.113+0000 7f6e95a00640 1 --2- 172.21.15.131:0/3220646343 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e740039b0 0x7f6e9014f620 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f6e80034040 tx=0x7f6e80004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:02.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.113+0000 7f6e8d600640 1 -- 172.21.15.131:0/3220646343 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e8003f020 con 0x7f6e740039b0 2024-05-26T22:49:02.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.113+0000 7f6e8d600640 1 -- 172.21.15.131:0/3220646343 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e80037040 con 0x7f6e740039b0 2024-05-26T22:49:02.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.113+0000 7f6e98400640 1 -- 172.21.15.131:0/3220646343 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e9014e860 con 0x7f6e740039b0 2024-05-26T22:49:02.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.113+0000 7f6e8d600640 1 -- 172.21.15.131:0/3220646343 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e80036070 con 0x7f6e740039b0 2024-05-26T22:49:02.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.114+0000 7f6e98400640 1 -- 172.21.15.131:0/3220646343 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e9014ee20 con 0x7f6e740039b0 2024-05-26T22:49:02.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.115+0000 7f6e98400640 1 -- 172.21.15.131:0/3220646343 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e90074e90 con 0x7f6e740039b0 2024-05-26T22:49:02.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.115+0000 7f6e8d600640 1 -- 172.21.15.131:0/3220646343 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6e80042020 con 0x7f6e740039b0 2024-05-26T22:49:02.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.117+0000 7f6e8d600640 1 --2- 172.21.15.131:0/3220646343 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f6e7006be30 0x7f6e7006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:02.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.117+0000 7f6e8d600640 1 -- 172.21.15.131:0/3220646343 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f6e800aaee0 con 0x7f6e740039b0 2024-05-26T22:49:02.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.117+0000 7f6e95000640 1 --2- 172.21.15.131:0/3220646343 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f6e7006be30 0x7f6e7006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:02.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.118+0000 7f6e95000640 1 --2- 172.21.15.131:0/3220646343 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f6e7006be30 0x7f6e7006e2f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6e7c007fd0 tx=0x7f6e7c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:02.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.121+0000 7f6e8d600640 1 -- 172.21.15.131:0/3220646343 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f6e80049030 con 0x7f6e740039b0 2024-05-26T22:49:02.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.319+0000 7f6e98400640 1 -- 172.21.15.131:0/3220646343 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f6e900695c0 con 0x7f6e740039b0 2024-05-26T22:49:02.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.320+0000 7f6e8d600640 1 -- 172.21.15.131:0/3220646343 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f6e800798b0 con 0x7f6e740039b0 2024-05-26T22:49:02.319 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-05-26T22:49:02.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.322+0000 7f6e98400640 1 -- 172.21.15.131:0/3220646343 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f6e7006be30 msgr2=0x7f6e7006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:02.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.322+0000 7f6e98400640 1 --2- 172.21.15.131:0/3220646343 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f6e7006be30 0x7f6e7006e2f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6e7c007fd0 tx=0x7f6e7c009040 comp rx=0 tx=0).stop 2024-05-26T22:49:02.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.322+0000 7f6e98400640 1 -- 172.21.15.131:0/3220646343 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e740039b0 msgr2=0x7f6e9014f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:02.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.322+0000 7f6e98400640 1 --2- 172.21.15.131:0/3220646343 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e740039b0 0x7f6e9014f620 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f6e80034040 tx=0x7f6e80004730 comp rx=0 tx=0).stop 2024-05-26T22:49:02.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.322+0000 7f6e98400640 1 -- 172.21.15.131:0/3220646343 shutdown_connections 2024-05-26T22:49:02.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.322+0000 7f6e98400640 1 --2- 172.21.15.131:0/3220646343 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6e9014c7c0 0x7f6e9015a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:02.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.322+0000 7f6e98400640 1 --2- 172.21.15.131:0/3220646343 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f6e7006be30 0x7f6e7006e2f0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:02.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.322+0000 7f6e98400640 1 --2- 172.21.15.131:0/3220646343 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6e901487f0 0x7f6e900cf080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:02.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.322+0000 7f6e98400640 1 --2- 172.21.15.131:0/3220646343 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e740039b0 0x7f6e9014f620 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:02.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.323+0000 7f6e98400640 1 -- 172.21.15.131:0/3220646343 >> 172.21.15.131:0/3220646343 conn(0x7f6e9006bc90 msgr2=0x7f6e900aef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:02.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.323+0000 7f6e98400640 1 -- 172.21.15.131:0/3220646343 shutdown_connections 2024-05-26T22:49:02.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.323+0000 7f6e98400640 1 -- 172.21.15.131:0/3220646343 wait complete. 2024-05-26T22:49:02.342 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-26T22:49:02.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.550+0000 7fa17e600640 1 Processor -- start 2024-05-26T22:49:02.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.550+0000 7fa17e600640 1 -- start start 2024-05-26T22:49:02.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.552+0000 7fa17e600640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fa1780712d0 0x7fa17806d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:02.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.552+0000 7fa17e600640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa178072d60 con 0x7fa178145a10 2024-05-26T22:49:02.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.552+0000 7fa17e600640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa178072a80 con 0x7fa17806d790 2024-05-26T22:49:02.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.552+0000 7fa17e600640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fa17815c760 con 0x7fa1780712d0 2024-05-26T22:49:02.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.552+0000 7fa17d200640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fa178145a10 0x7fa178072180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:48188/0 (socket says 172.21.15.131:48188) 2024-05-26T22:49:02.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.552+0000 7fa17d200640 1 -- 172.21.15.131:0/4127712987 learned_addr learned my addr 172.21.15.131:0/4127712987 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:49:02.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.553+0000 7fa16fe00640 1 --2- 172.21.15.131:0/4127712987 >> v2:172.21.15.187:3300/0 conn(0x7fa1780712d0 0x7fa17806d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:02.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.553+0000 7fa177e00640 1 -- 172.21.15.131:0/4127712987 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1009935674 0 0) 0x7fa178072d60 con 0x7fa178145a10 2024-05-26T22:49:02.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.553+0000 7fa177e00640 1 -- 172.21.15.131:0/4127712987 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa15c003520 con 0x7fa178145a10 2024-05-26T22:49:02.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.553+0000 7fa177e00640 1 -- 172.21.15.131:0/4127712987 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2807203497 0 0) 0x7fa15c003520 con 0x7fa178145a10 2024-05-26T22:49:02.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.553+0000 7fa177e00640 1 -- 172.21.15.131:0/4127712987 >> v2:172.21.15.187:3300/0 conn(0x7fa1780712d0 msgr2=0x7fa17806d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:02.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.553+0000 7fa177e00640 1 --2- 172.21.15.131:0/4127712987 >> v2:172.21.15.187:3300/0 conn(0x7fa1780712d0 0x7fa17806d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:02.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.553+0000 7fa177e00640 1 -- 172.21.15.131:0/4127712987 >> v1:172.21.15.187:6789/0 conn(0x7fa17806d790 legacy=0x7fa1780cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:02.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.553+0000 7fa177e00640 1 -- 172.21.15.131:0/4127712987 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1780ceab0 con 0x7fa178145a10 2024-05-26T22:49:02.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.553+0000 7fa177e00640 1 -- 172.21.15.131:0/4127712987 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7fa1680019f0 con 0x7fa178145a10 2024-05-26T22:49:02.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.553+0000 7fa177e00640 1 -- 172.21.15.131:0/4127712987 >> v1:172.21.15.131:6789/0 conn(0x7fa178145a10 legacy=0x7fa178072180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:02.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.553+0000 7fa177e00640 1 --2- 172.21.15.131:0/4127712987 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa15c0039b0 0x7fa15c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:02.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.553+0000 7fa177e00640 1 --2- 172.21.15.131:0/4127712987 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa15c004800 0x7fa15c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:02.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.553+0000 7fa177e00640 1 --2- 172.21.15.131:0/4127712987 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa15c0051c0 0x7fa15c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:02.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.553+0000 7fa177e00640 1 -- 172.21.15.131:0/4127712987 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa17815c760 con 0x7fa15c0039b0 2024-05-26T22:49:02.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.553+0000 7fa177e00640 1 -- 172.21.15.131:0/4127712987 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa15c00c060 con 0x7fa15c004800 2024-05-26T22:49:02.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.553+0000 7fa177e00640 1 -- 172.21.15.131:0/4127712987 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa15c00c260 con 0x7fa15c0051c0 2024-05-26T22:49:02.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.553+0000 7fa17d200640 1 --2- 172.21.15.131:0/4127712987 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa15c0039b0 0x7fa15c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:02.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.554+0000 7fa17d200640 1 -- 172.21.15.131:0/4127712987 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa15c0051c0 msgr2=0x7fa15c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:02.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.554+0000 7fa17d200640 1 --2- 172.21.15.131:0/4127712987 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa15c0051c0 0x7fa15c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:02.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.554+0000 7fa17d200640 1 -- 172.21.15.131:0/4127712987 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa15c004800 msgr2=0x7fa15c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:02.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.554+0000 7fa17d200640 1 --2- 172.21.15.131:0/4127712987 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa15c004800 0x7fa15c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:02.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.554+0000 7fa17d200640 1 -- 172.21.15.131:0/4127712987 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa15c003520 con 0x7fa15c0039b0 2024-05-26T22:49:02.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.554+0000 7fa17dc00640 1 -- 172.21.15.131:0/4127712987 reap_dead start 2024-05-26T22:49:02.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.555+0000 7fa17d200640 1 --2- 172.21.15.131:0/4127712987 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa15c0039b0 0x7fa15c003dd0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa16800a860 tx=0x7fa15c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=319b2219b94b9e0d server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:02.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.555+0000 7fa177e00640 1 -- 172.21.15.131:0/4127712987 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa16800a480 con 0x7fa15c0039b0 2024-05-26T22:49:02.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.555+0000 7fa177e00640 1 -- 172.21.15.131:0/4127712987 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa16802f070 con 0x7fa15c0039b0 2024-05-26T22:49:02.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.555+0000 7fa177e00640 1 -- 172.21.15.131:0/4127712987 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa16802bd20 con 0x7fa15c0039b0 2024-05-26T22:49:02.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.556+0000 7fa17e600640 1 -- 172.21.15.131:0/4127712987 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa15c0039b0 msgr2=0x7fa15c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:02.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.556+0000 7fa17e600640 1 --2- 172.21.15.131:0/4127712987 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa15c0039b0 0x7fa15c003dd0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa16800a860 tx=0x7fa15c005830 comp rx=0 tx=0).stop 2024-05-26T22:49:02.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.556+0000 7fa17e600640 1 -- 172.21.15.131:0/4127712987 shutdown_connections 2024-05-26T22:49:02.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.556+0000 7fa17e600640 1 --2- 172.21.15.131:0/4127712987 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa15c0039b0 0x7fa15c003dd0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:02.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.556+0000 7fa17e600640 1 -- 172.21.15.131:0/4127712987 >> 172.21.15.131:0/4127712987 conn(0x7fa1780b5660 msgr2=0x7fa1780b5a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:02.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.556+0000 7fa17e600640 1 -- 172.21.15.131:0/4127712987 shutdown_connections 2024-05-26T22:49:02.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.556+0000 7fa17e600640 1 -- 172.21.15.131:0/4127712987 wait complete. 2024-05-26T22:49:02.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.558+0000 7fa17e600640 1 Processor -- start 2024-05-26T22:49:02.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.558+0000 7fa17e600640 1 -- start start 2024-05-26T22:49:02.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.559+0000 7fa17e600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa15c0039b0 0x7fa178142470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:02.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.559+0000 7fa17e600640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa1781f15b0 0x7fa178140910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:02.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.559+0000 7fa17e600640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa178140e50 0x7fa1781f7540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:02.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.559+0000 7fa17e600640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa1780cf070 con 0x7fa15c0039b0 2024-05-26T22:49:02.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.559+0000 7fa17e600640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa17806dbc0 con 0x7fa1781f15b0 2024-05-26T22:49:02.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.559+0000 7fa17e600640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa178145df0 con 0x7fa178140e50 2024-05-26T22:49:02.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.559+0000 7fa17d200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa15c0039b0 0x7fa178142470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:02.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.559+0000 7fa17d200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa15c0039b0 0x7fa178142470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44100/0 (socket says 172.21.15.131:44100) 2024-05-26T22:49:02.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.559+0000 7fa17d200640 1 -- 172.21.15.131:0/914646411 learned_addr learned my addr 172.21.15.131:0/914646411 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:02.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.559+0000 7fa16fe00640 1 --2- 172.21.15.131:0/914646411 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa1781f15b0 0x7fa178140910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:02.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.559+0000 7fa17d200640 1 -- 172.21.15.131:0/914646411 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa178140e50 msgr2=0x7fa1781f7540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:02.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.559+0000 7fa17d200640 1 --2- 172.21.15.131:0/914646411 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa178140e50 0x7fa1781f7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:02.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.559+0000 7fa17d200640 1 -- 172.21.15.131:0/914646411 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa1781f15b0 msgr2=0x7fa178140910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:02.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.559+0000 7fa17d200640 1 --2- 172.21.15.131:0/914646411 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa1781f15b0 0x7fa178140910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:02.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.559+0000 7fa17d200640 1 -- 172.21.15.131:0/914646411 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1781f4210 con 0x7fa15c0039b0 2024-05-26T22:49:02.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.560+0000 7fa17d200640 1 --2- 172.21.15.131:0/914646411 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa15c0039b0 0x7fa178142470 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa16800a860 tx=0x7fa16802d070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:02.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.560+0000 7fa175600640 1 -- 172.21.15.131:0/914646411 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa16800a480 con 0x7fa15c0039b0 2024-05-26T22:49:02.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.560+0000 7fa175600640 1 -- 172.21.15.131:0/914646411 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa168004d60 con 0x7fa15c0039b0 2024-05-26T22:49:02.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.560+0000 7fa175600640 1 -- 172.21.15.131:0/914646411 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa168034670 con 0x7fa15c0039b0 2024-05-26T22:49:02.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.560+0000 7fa17e600640 1 -- 172.21.15.131:0/914646411 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1781f44f0 con 0x7fa15c0039b0 2024-05-26T22:49:02.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.560+0000 7fa17e600640 1 -- 172.21.15.131:0/914646411 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1781f4ab0 con 0x7fa15c0039b0 2024-05-26T22:49:02.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.561+0000 7fa17e600640 1 -- 172.21.15.131:0/914646411 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1780694e0 con 0x7fa15c0039b0 2024-05-26T22:49:02.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.562+0000 7fa175600640 1 -- 172.21.15.131:0/914646411 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa168034810 con 0x7fa15c0039b0 2024-05-26T22:49:02.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.563+0000 7fa175600640 1 --2- 172.21.15.131:0/914646411 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa15406be30 0x7fa15406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:02.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.563+0000 7fa175600640 1 -- 172.21.15.131:0/914646411 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa1680ab4b0 con 0x7fa15c0039b0 2024-05-26T22:49:02.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.563+0000 7fa16fe00640 1 --2- 172.21.15.131:0/914646411 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa15406be30 0x7fa15406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:02.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.564+0000 7fa16fe00640 1 --2- 172.21.15.131:0/914646411 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa15406be30 0x7fa15406e2f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa1780b5240 tx=0x7fa160008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:02.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.568+0000 7fa175600640 1 -- 172.21.15.131:0/914646411 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fa168079e60 con 0x7fa15c0039b0 2024-05-26T22:49:02.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.750+0000 7fa17e600640 1 -- 172.21.15.131:0/914646411 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa178141cb0 con 0x7fa15c0039b0 2024-05-26T22:49:02.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.751+0000 7fa175600640 1 -- 172.21.15.131:0/914646411 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7fa168079760 con 0x7fa15c0039b0 2024-05-26T22:49:02.751 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 64 2024-05-26T22:49:02.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.754+0000 7fa17e600640 1 -- 172.21.15.131:0/914646411 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa15406be30 msgr2=0x7fa15406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:02.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.754+0000 7fa17e600640 1 --2- 172.21.15.131:0/914646411 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa15406be30 0x7fa15406e2f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa1780b5240 tx=0x7fa160008040 comp rx=0 tx=0).stop 2024-05-26T22:49:02.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.754+0000 7fa17e600640 1 -- 172.21.15.131:0/914646411 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa15c0039b0 msgr2=0x7fa178142470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:02.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.754+0000 7fa17e600640 1 --2- 172.21.15.131:0/914646411 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa15c0039b0 0x7fa178142470 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa16800a860 tx=0x7fa16802d070 comp rx=0 tx=0).stop 2024-05-26T22:49:02.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.754+0000 7fa17e600640 1 -- 172.21.15.131:0/914646411 shutdown_connections 2024-05-26T22:49:02.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.754+0000 7fa17e600640 1 --2- 172.21.15.131:0/914646411 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa178140e50 0x7fa1781f7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:02.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.754+0000 7fa17e600640 1 --2- 172.21.15.131:0/914646411 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa15406be30 0x7fa15406e2f0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:02.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.754+0000 7fa17e600640 1 --2- 172.21.15.131:0/914646411 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa1781f15b0 0x7fa178140910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:02.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.754+0000 7fa17e600640 1 --2- 172.21.15.131:0/914646411 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa15c0039b0 0x7fa178142470 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:02.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.754+0000 7fa17e600640 1 -- 172.21.15.131:0/914646411 >> 172.21.15.131:0/914646411 conn(0x7fa1780b5660 msgr2=0x7fa1780b5a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:02.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.754+0000 7fa17e600640 1 -- 172.21.15.131:0/914646411 shutdown_connections 2024-05-26T22:49:02.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.754+0000 7fa17e600640 1 -- 172.21.15.131:0/914646411 wait complete. 2024-05-26T22:49:02.772 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_exports.TestEphemeralPins.test_ephemeral_pin_dist_conf_off' 2024-05-26T22:49:02.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.950+0000 7f1b42800640 1 Processor -- start 2024-05-26T22:49:02.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.951+0000 7f1b42800640 1 -- start start 2024-05-26T22:49:02.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.952+0000 7f1b42800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1b3c0ad250 0x7f1b3c0b3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:02.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.952+0000 7f1b42800640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b3c15cb20 con 0x7f1b3c1569d0 2024-05-26T22:49:02.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.952+0000 7f1b42800640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b3c15ce00 con 0x7f1b3c06ee20 2024-05-26T22:49:02.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.952+0000 7f1b42800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1b3c15d0e0 con 0x7f1b3c0ad250 2024-05-26T22:49:02.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.952+0000 7f1b3be00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f1b3c1569d0 0x7f1b3c160010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:48198/0 (socket says 172.21.15.131:48198) 2024-05-26T22:49:02.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.952+0000 7f1b3be00640 1 -- 172.21.15.131:0/3412411388 learned_addr learned my addr 172.21.15.131:0/3412411388 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:49:02.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.953+0000 7f1b32a00640 1 --2- 172.21.15.131:0/3412411388 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1b3c0ad250 0x7f1b3c0b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:02.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.953+0000 7f1b3aa00640 1 -- 172.21.15.131:0/3412411388 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 114903103 0 0) 0x7f1b3c15cb20 con 0x7f1b3c1569d0 2024-05-26T22:49:02.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.953+0000 7f1b3aa00640 1 -- 172.21.15.131:0/3412411388 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1b20003520 con 0x7f1b3c1569d0 2024-05-26T22:49:02.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.953+0000 7f1b3aa00640 1 -- 172.21.15.131:0/3412411388 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2203998249 0 0) 0x7f1b20003520 con 0x7f1b3c1569d0 2024-05-26T22:49:02.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.953+0000 7f1b3aa00640 1 -- 172.21.15.131:0/3412411388 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1b3c0ad250 msgr2=0x7f1b3c0b3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:02.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.953+0000 7f1b3aa00640 1 --2- 172.21.15.131:0/3412411388 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1b3c0ad250 0x7f1b3c0b3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:02.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.953+0000 7f1b3aa00640 1 -- 172.21.15.131:0/3412411388 >> v1:172.21.15.187:6789/0 conn(0x7f1b3c06ee20 legacy=0x7f1b3c1562c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:02.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.953+0000 7f1b3aa00640 1 -- 172.21.15.131:0/3412411388 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b3c15d3c0 con 0x7f1b3c1569d0 2024-05-26T22:49:02.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.953+0000 7f1b3aa00640 1 -- 172.21.15.131:0/3412411388 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7f1b34001670 con 0x7f1b3c1569d0 2024-05-26T22:49:02.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.953+0000 7f1b3aa00640 1 -- 172.21.15.131:0/3412411388 >> v1:172.21.15.131:6789/0 conn(0x7f1b3c1569d0 legacy=0x7f1b3c160010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:02.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.953+0000 7f1b3aa00640 1 --2- 172.21.15.131:0/3412411388 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1b3c0ad250 0x7f1b200039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:02.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.953+0000 7f1b3aa00640 1 --2- 172.21.15.131:0/3412411388 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1b20004010 0x7f1b20004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:02.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.953+0000 7f1b3aa00640 1 --2- 172.21.15.131:0/3412411388 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1b20004d40 0x7f1b200051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:02.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.953+0000 7f1b3aa00640 1 -- 172.21.15.131:0/3412411388 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f1b3c15d0e0 con 0x7f1b20004010 2024-05-26T22:49:02.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.953+0000 7f1b3aa00640 1 -- 172.21.15.131:0/3412411388 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1b2000bb80 con 0x7f1b20004d40 2024-05-26T22:49:02.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.953+0000 7f1b3aa00640 1 -- 172.21.15.131:0/3412411388 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1b2000bd80 con 0x7f1b3c0ad250 2024-05-26T22:49:02.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.953+0000 7f1b32a00640 1 --2- 172.21.15.131:0/3412411388 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1b20004010 0x7f1b20004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:02.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.954+0000 7f1b32a00640 1 -- 172.21.15.131:0/3412411388 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1b3c0ad250 msgr2=0x7f1b200039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:02.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.954+0000 7f1b3be00640 1 --2- 172.21.15.131:0/3412411388 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1b20004d40 0x7f1b200051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:02.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.954+0000 7f1b3b400640 1 --2- 172.21.15.131:0/3412411388 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1b3c0ad250 0x7f1b200039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:02.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.954+0000 7f1b32a00640 1 --2- 172.21.15.131:0/3412411388 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1b3c0ad250 0x7f1b200039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:02.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.954+0000 7f1b32a00640 1 -- 172.21.15.131:0/3412411388 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1b20004d40 msgr2=0x7f1b200051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:02.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.954+0000 7f1b32a00640 1 --2- 172.21.15.131:0/3412411388 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1b20004d40 0x7f1b200051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:02.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.954+0000 7f1b32a00640 1 -- 172.21.15.131:0/3412411388 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b20003520 con 0x7f1b20004010 2024-05-26T22:49:02.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.954+0000 7f1b3be00640 1 --2- 172.21.15.131:0/3412411388 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1b20004d40 0x7f1b200051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:49:02.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.954+0000 7f1b3b400640 1 --2- 172.21.15.131:0/3412411388 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1b3c0ad250 0x7f1b200039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:49:02.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.955+0000 7f1b32a00640 1 --2- 172.21.15.131:0/3412411388 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1b20004010 0x7f1b20004800 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1b2400ba30 tx=0x7f1b20005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39f7808de85491ce server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:02.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.955+0000 7f1b3aa00640 1 -- 172.21.15.131:0/3412411388 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b24030070 con 0x7f1b20004010 2024-05-26T22:49:02.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.955+0000 7f1b3aa00640 1 -- 172.21.15.131:0/3412411388 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b2402ade0 con 0x7f1b20004010 2024-05-26T22:49:02.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.955+0000 7f1b3aa00640 1 -- 172.21.15.131:0/3412411388 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b2400b600 con 0x7f1b20004010 2024-05-26T22:49:02.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.956+0000 7f1b42800640 1 -- 172.21.15.131:0/3412411388 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1b20004010 msgr2=0x7f1b20004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:02.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.956+0000 7f1b42800640 1 --2- 172.21.15.131:0/3412411388 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1b20004010 0x7f1b20004800 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1b2400ba30 tx=0x7f1b20005830 comp rx=0 tx=0).stop 2024-05-26T22:49:02.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.956+0000 7f1b3be00640 1 -- 172.21.15.131:0/3412411388 reap_dead start 2024-05-26T22:49:02.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.956+0000 7f1b42800640 1 -- 172.21.15.131:0/3412411388 shutdown_connections 2024-05-26T22:49:02.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.956+0000 7f1b42800640 1 -- 172.21.15.131:0/3412411388 >> 172.21.15.131:0/3412411388 conn(0x7f1b3c0b1e90 msgr2=0x7f1b3c0b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:02.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.956+0000 7f1b42800640 1 -- 172.21.15.131:0/3412411388 shutdown_connections 2024-05-26T22:49:02.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.956+0000 7f1b42800640 1 -- 172.21.15.131:0/3412411388 wait complete. 2024-05-26T22:49:02.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.957+0000 7f1b42800640 1 Processor -- start 2024-05-26T22:49:02.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.958+0000 7f1b42800640 1 -- start start 2024-05-26T22:49:02.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.958+0000 7f1b42800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1b20004010 0x7f1b3c09c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:02.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.958+0000 7f1b42800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1b3c153380 0x7f1b3c09a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:02.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.958+0000 7f1b42800640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1b3c153800 0x7f1b3c1f27e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:02.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.958+0000 7f1b42800640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f1b3c15c900 con 0x7f1b3c153380 2024-05-26T22:49:02.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.958+0000 7f1b42800640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1b3c15c780 con 0x7f1b3c153800 2024-05-26T22:49:02.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.958+0000 7f1b42800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1b3c06f200 con 0x7f1b20004010 2024-05-26T22:49:02.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.959+0000 7f1b32a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1b3c153380 0x7f1b3c09a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:02.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.959+0000 7f1b32a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1b3c153380 0x7f1b3c09a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44124/0 (socket says 172.21.15.131:44124) 2024-05-26T22:49:02.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.959+0000 7f1b32a00640 1 -- 172.21.15.131:0/1579876502 learned_addr learned my addr 172.21.15.131:0/1579876502 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:02.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.959+0000 7f1b32a00640 1 -- 172.21.15.131:0/1579876502 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1b20004010 msgr2=0x7f1b3c09c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:02.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.959+0000 7f1b32a00640 1 --2- 172.21.15.131:0/1579876502 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1b20004010 0x7f1b3c09c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:02.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.959+0000 7f1b32a00640 1 -- 172.21.15.131:0/1579876502 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1b3c153800 msgr2=0x7f1b3c1f27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:02.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.959+0000 7f1b32a00640 1 --2- 172.21.15.131:0/1579876502 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1b3c153800 0x7f1b3c1f27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:02.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.959+0000 7f1b32a00640 1 -- 172.21.15.131:0/1579876502 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b3c13e0d0 con 0x7f1b3c153380 2024-05-26T22:49:02.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.959+0000 7f1b32a00640 1 --2- 172.21.15.131:0/1579876502 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1b3c153380 0x7f1b3c09a280 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f1b2400bb50 tx=0x7f1b240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:02.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.960+0000 7f1b33e00640 1 -- 172.21.15.131:0/1579876502 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b24043030 con 0x7f1b3c153380 2024-05-26T22:49:02.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.960+0000 7f1b33e00640 1 -- 172.21.15.131:0/1579876502 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b2402abe0 con 0x7f1b3c153380 2024-05-26T22:49:02.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.960+0000 7f1b33e00640 1 -- 172.21.15.131:0/1579876502 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b2400b600 con 0x7f1b3c153380 2024-05-26T22:49:02.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.960+0000 7f1b42800640 1 -- 172.21.15.131:0/1579876502 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b3c13e3b0 con 0x7f1b3c153380 2024-05-26T22:49:02.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.960+0000 7f1b42800640 1 -- 172.21.15.131:0/1579876502 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b3c13e970 con 0x7f1b3c153380 2024-05-26T22:49:02.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.961+0000 7f1b42800640 1 -- 172.21.15.131:0/1579876502 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b3c0694e0 con 0x7f1b3c153380 2024-05-26T22:49:02.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.962+0000 7f1b33e00640 1 -- 172.21.15.131:0/1579876502 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1b24046070 con 0x7f1b3c153380 2024-05-26T22:49:02.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.963+0000 7f1b33e00640 1 --2- 172.21.15.131:0/1579876502 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f1b1806be30 0x7f1b1806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:02.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.963+0000 7f1b33e00640 1 -- 172.21.15.131:0/1579876502 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f1b240ab9f0 con 0x7f1b3c153380 2024-05-26T22:49:02.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.963+0000 7f1b3b400640 1 --2- 172.21.15.131:0/1579876502 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f1b1806be30 0x7f1b1806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:02.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.964+0000 7f1b3b400640 1 --2- 172.21.15.131:0/1579876502 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f1b1806be30 0x7f1b1806e2f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c007fd0 tx=0x7f1b2c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:02.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:02.967+0000 7f1b33e00640 1 -- 172.21.15.131:0/1579876502 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f1b2407a270 con 0x7f1b3c153380 2024-05-26T22:49:03.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:03.235+0000 7f1b42800640 1 -- 172.21.15.131:0/1579876502 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_exports.TestEphemeralPins.test_ephemeral_pin_dist_conf_off"]} v 0) -- 0x7f1b3c06e660 con 0x7f1b3c153380 2024-05-26T22:49:03.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:03.279+0000 7f1b33e00640 1 -- 172.21.15.131:0/1579876502 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_exports.TestEphemeralPins.test_ephemeral_pin_dist_conf_off"]}]=0 v59) ==== 158+0+0 (secure 0 0 0) 0x7f1b24079b70 con 0x7f1b3c153380 2024-05-26T22:49:03.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:03.282+0000 7f1b42800640 1 -- 172.21.15.131:0/1579876502 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f1b1806be30 msgr2=0x7f1b1806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:03.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:03.282+0000 7f1b42800640 1 --2- 172.21.15.131:0/1579876502 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f1b1806be30 0x7f1b1806e2f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c007fd0 tx=0x7f1b2c009040 comp rx=0 tx=0).stop 2024-05-26T22:49:03.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:03.282+0000 7f1b42800640 1 -- 172.21.15.131:0/1579876502 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1b3c153380 msgr2=0x7f1b3c09a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:03.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:03.282+0000 7f1b42800640 1 --2- 172.21.15.131:0/1579876502 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1b3c153380 0x7f1b3c09a280 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f1b2400bb50 tx=0x7f1b240046c0 comp rx=0 tx=0).stop 2024-05-26T22:49:03.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:03.283+0000 7f1b42800640 1 -- 172.21.15.131:0/1579876502 shutdown_connections 2024-05-26T22:49:03.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:03.283+0000 7f1b42800640 1 --2- 172.21.15.131:0/1579876502 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f1b1806be30 0x7f1b1806e2f0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:03.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:03.283+0000 7f1b42800640 1 --2- 172.21.15.131:0/1579876502 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1b3c153800 0x7f1b3c1f27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:03.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:03.283+0000 7f1b42800640 1 --2- 172.21.15.131:0/1579876502 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1b3c153380 0x7f1b3c09a280 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:03.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:03.283+0000 7f1b42800640 1 --2- 172.21.15.131:0/1579876502 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1b20004010 0x7f1b3c09c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:03.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:03.283+0000 7f1b42800640 1 -- 172.21.15.131:0/1579876502 >> 172.21.15.131:0/1579876502 conn(0x7f1b3c0b1e90 msgr2=0x7f1b3c15ea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:03.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:03.284+0000 7f1b42800640 1 -- 172.21.15.131:0/1579876502 shutdown_connections 2024-05-26T22:49:03.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:03.284+0000 7f1b42800640 1 -- 172.21.15.131:0/1579876502 wait complete. 2024-05-26T22:49:03.305 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-26T22:49:03.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:03.493+0000 7f98edc00640 1 Processor -- start 2024-05-26T22:49:03.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:03.493+0000 7f98edc00640 1 -- start start 2024-05-26T22:49:03.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:03.494+0000 7f98edc00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f98e8145a10 0x7f98e8072180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:03.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:03.494+0000 7f98edc00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f98e806d960 0x7f98e80cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:03.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:03.494+0000 7f98edc00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98e8072d60 con 0x7f98e80712d0 2024-05-26T22:49:03.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:03.494+0000 7f98edc00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f98e815c760 con 0x7f98e8145a10 2024-05-26T22:49:03.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:03.494+0000 7f98edc00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f98e8072a80 con 0x7f98e806d960 2024-05-26T22:49:03.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:03.495+0000 7f98e7400640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f98e80712d0 0x7f98e806d250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:48210/0 (socket says 172.21.15.131:48210) 2024-05-26T22:49:03.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:03.495+0000 7f98e7400640 1 -- 172.21.15.131:0/2554815276 learned_addr learned my addr 172.21.15.131:0/2554815276 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:49:03.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:03.495+0000 7f98e7e00640 1 --2- 172.21.15.131:0/2554815276 >> v2:172.21.15.187:3300/0 conn(0x7f98e8145a10 0x7f98e8072180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:03.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:03.495+0000 7f98ed200640 1 --2- 172.21.15.131:0/2554815276 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f98e806d960 0x7f98e80cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:03.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:03.495+0000 7f98e6a00640 1 -- 172.21.15.131:0/2554815276 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 946443571 0 0) 0x7f98e8072d60 con 0x7f98e80712d0 2024-05-26T22:49:03.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:03.495+0000 7f98e6a00640 1 -- 172.21.15.131:0/2554815276 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f98cc003520 con 0x7f98e80712d0 2024-05-26T22:49:03.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:03.496+0000 7f98e6a00640 1 -- 172.21.15.131:0/2554815276 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1833294783 0 0) 0x7f98cc003520 con 0x7f98e80712d0 2024-05-26T22:49:03.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:03.496+0000 7f98e6a00640 1 -- 172.21.15.131:0/2554815276 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f98e806d960 msgr2=0x7f98e80cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:03.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:03.496+0000 7f98e6a00640 1 --2- 172.21.15.131:0/2554815276 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f98e806d960 0x7f98e80cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:03.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:03.496+0000 7f98e6a00640 1 -- 172.21.15.131:0/2554815276 >> v2:172.21.15.187:3300/0 conn(0x7f98e8145a10 msgr2=0x7f98e8072180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:03.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:03.496+0000 7f98e6a00640 1 --2- 172.21.15.131:0/2554815276 >> v2:172.21.15.187:3300/0 conn(0x7f98e8145a10 0x7f98e8072180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:03.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:03.496+0000 7f98e6a00640 1 -- 172.21.15.131:0/2554815276 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98e80ceab0 con 0x7f98e80712d0 2024-05-26T22:49:03.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:03.496+0000 7f98e6a00640 1 -- 172.21.15.131:0/2554815276 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7f98d4001670 con 0x7f98e80712d0 2024-05-26T22:49:03.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:03.496+0000 7f98e6a00640 1 -- 172.21.15.131:0/2554815276 >> v1:172.21.15.131:6789/0 conn(0x7f98e80712d0 legacy=0x7f98e806d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:03.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:03.496+0000 7f98e6a00640 1 --2- 172.21.15.131:0/2554815276 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f98cc0039b0 0x7f98cc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:03.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:03.496+0000 7f98e6a00640 1 --2- 172.21.15.131:0/2554815276 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f98cc004800 0x7f98cc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:03.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:03.496+0000 7f98e6a00640 1 --2- 172.21.15.131:0/2554815276 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f98e806d960 0x7f98cc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:03.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:03.496+0000 7f98e6a00640 1 -- 172.21.15.131:0/2554815276 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f98e815c760 con 0x7f98cc004800 2024-05-26T22:49:03.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:03.496+0000 7f98e6a00640 1 -- 172.21.15.131:0/2554815276 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f98e8072a80 con 0x7f98cc0039b0 2024-05-26T22:49:03.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:03.496+0000 7f98e6a00640 1 -- 172.21.15.131:0/2554815276 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f98cc00b9c0 con 0x7f98e806d960 2024-05-26T22:49:03.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:03.496+0000 7f98e7400640 1 --2- 172.21.15.131:0/2554815276 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f98cc004800 0x7f98cc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:03.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:03.496+0000 7f98e7e00640 1 --2- 172.21.15.131:0/2554815276 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f98cc0039b0 0x7f98cc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:03.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:03.496+0000 7f98e7400640 1 -- 172.21.15.131:0/2554815276 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f98e806d960 msgr2=0x7f98cc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:03.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:03.496+0000 7f98ed200640 1 --2- 172.21.15.131:0/2554815276 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f98e806d960 0x7f98cc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:03.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:03.496+0000 7f98e7400640 1 --2- 172.21.15.131:0/2554815276 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f98e806d960 0x7f98cc0051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:03.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:03.497+0000 7f98e7400640 1 -- 172.21.15.131:0/2554815276 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f98cc0039b0 msgr2=0x7f98cc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:03.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:03.497+0000 7f98e7400640 1 --2- 172.21.15.131:0/2554815276 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f98cc0039b0 0x7f98cc003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:03.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:03.497+0000 7f98e7400640 1 -- 172.21.15.131:0/2554815276 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98cc003520 con 0x7f98cc004800 2024-05-26T22:49:03.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:03.498+0000 7f98e7400640 1 --2- 172.21.15.131:0/2554815276 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f98cc004800 0x7f98cc004c80 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f98d400c160 tx=0x7f98cc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fefce4e41e3eef3e server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:03.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:03.498+0000 7f98e6a00640 1 -- 172.21.15.131:0/2554815276 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d4007c80 con 0x7f98cc004800 2024-05-26T22:49:03.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:03.498+0000 7f98e6a00640 1 -- 172.21.15.131:0/2554815276 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98d4005ad0 con 0x7f98cc004800 2024-05-26T22:49:03.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:03.498+0000 7f98e6a00640 1 -- 172.21.15.131:0/2554815276 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d4007070 con 0x7f98cc004800 2024-05-26T22:49:03.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:03.498+0000 7f98edc00640 1 -- 172.21.15.131:0/2554815276 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f98cc004800 msgr2=0x7f98cc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:03.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:03.498+0000 7f98edc00640 1 --2- 172.21.15.131:0/2554815276 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f98cc004800 0x7f98cc004c80 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f98d400c160 tx=0x7f98cc005830 comp rx=0 tx=0).stop 2024-05-26T22:49:03.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:03.498+0000 7f98ed200640 1 -- 172.21.15.131:0/2554815276 reap_dead start 2024-05-26T22:49:03.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:03.498+0000 7f98edc00640 1 -- 172.21.15.131:0/2554815276 shutdown_connections 2024-05-26T22:49:03.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:03.498+0000 7f98edc00640 1 -- 172.21.15.131:0/2554815276 >> 172.21.15.131:0/2554815276 conn(0x7f98e80b5660 msgr2=0x7f98e80b5a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:03.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:03.498+0000 7f98edc00640 1 -- 172.21.15.131:0/2554815276 shutdown_connections 2024-05-26T22:49:03.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:03.499+0000 7f98edc00640 1 -- 172.21.15.131:0/2554815276 wait complete. 2024-05-26T22:49:03.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:03.500+0000 7f98edc00640 1 Processor -- start 2024-05-26T22:49:03.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:03.500+0000 7f98edc00640 1 -- start start 2024-05-26T22:49:03.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:03.501+0000 7f98edc00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f98cc004800 0x7f98e815bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:03.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:03.501+0000 7f98edc00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f98e81449d0 0x7f98e81f7260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:03.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:03.501+0000 7f98edc00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f98e8144e50 0x7f98e81f77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:03.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:03.501+0000 7f98edc00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f98e8145df0 con 0x7f98e81449d0 2024-05-26T22:49:03.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:03.501+0000 7f98edc00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f98e80cf070 con 0x7f98e8144e50 2024-05-26T22:49:03.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:03.501+0000 7f98edc00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f98e80726c0 con 0x7f98cc004800 2024-05-26T22:49:03.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:03.501+0000 7f98e7400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f98e81449d0 0x7f98e81f7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:03.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:03.501+0000 7f98e7400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f98e81449d0 0x7f98e81f7260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44150/0 (socket says 172.21.15.131:44150) 2024-05-26T22:49:03.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:03.502+0000 7f98e7400640 1 -- 172.21.15.131:0/2704889018 learned_addr learned my addr 172.21.15.131:0/2704889018 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:03.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:03.502+0000 7f98ed200640 1 --2- 172.21.15.131:0/2704889018 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f98e8144e50 0x7f98e81f77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:03.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:03.502+0000 7f98e7e00640 1 --2- 172.21.15.131:0/2704889018 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f98cc004800 0x7f98e815bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:03.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:03.502+0000 7f98e7400640 1 -- 172.21.15.131:0/2704889018 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f98cc004800 msgr2=0x7f98e815bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:03.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:03.502+0000 7f98e7400640 1 --2- 172.21.15.131:0/2704889018 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f98cc004800 0x7f98e815bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:03.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:03.502+0000 7f98e7400640 1 -- 172.21.15.131:0/2704889018 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f98e8144e50 msgr2=0x7f98e81f77a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:03.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:03.502+0000 7f98e7400640 1 --2- 172.21.15.131:0/2704889018 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f98e8144e50 0x7f98e81f77a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:03.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:03.502+0000 7f98e7400640 1 -- 172.21.15.131:0/2704889018 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98e81431a0 con 0x7f98e81449d0 2024-05-26T22:49:03.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:03.502+0000 7f98ed200640 1 --2- 172.21.15.131:0/2704889018 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f98e8144e50 0x7f98e81f77a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:49:03.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:03.502+0000 7f98e7400640 1 --2- 172.21.15.131:0/2704889018 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f98e81449d0 0x7f98e81f7260 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f98d402ad60 tx=0x7f98d4001970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:03.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:03.502+0000 7f98dfe00640 1 -- 172.21.15.131:0/2704889018 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d4043070 con 0x7f98e81449d0 2024-05-26T22:49:03.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:03.502+0000 7f98dfe00640 1 -- 172.21.15.131:0/2704889018 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98d4038040 con 0x7f98e81449d0 2024-05-26T22:49:03.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:03.502+0000 7f98dfe00640 1 -- 172.21.15.131:0/2704889018 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d4007070 con 0x7f98e81449d0 2024-05-26T22:49:03.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:03.502+0000 7f98edc00640 1 -- 172.21.15.131:0/2704889018 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98e8143480 con 0x7f98e81449d0 2024-05-26T22:49:03.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:03.503+0000 7f98edc00640 1 -- 172.21.15.131:0/2704889018 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98e8143a40 con 0x7f98e81449d0 2024-05-26T22:49:03.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:03.504+0000 7f98edc00640 1 -- 172.21.15.131:0/2704889018 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98e80694e0 con 0x7f98e81449d0 2024-05-26T22:49:03.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:03.505+0000 7f98dfe00640 1 -- 172.21.15.131:0/2704889018 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f98d4007210 con 0x7f98e81449d0 2024-05-26T22:49:03.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:03.505+0000 7f98dfe00640 1 --2- 172.21.15.131:0/2704889018 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f98c806be30 0x7f98c806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:03.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:03.506+0000 7f98dfe00640 1 -- 172.21.15.131:0/2704889018 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5825+0+0 (secure 0 0 0) 0x7f98d40aaa50 con 0x7f98e81449d0 2024-05-26T22:49:03.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:03.506+0000 7f98e7e00640 1 --2- 172.21.15.131:0/2704889018 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f98c806be30 0x7f98c806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:03.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:03.506+0000 7f98e7e00640 1 --2- 172.21.15.131:0/2704889018 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f98c806be30 0x7f98c806e2f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f98d8009fc0 tx=0x7f98d800b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:03.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:03.510+0000 7f98dfe00640 1 -- 172.21.15.131:0/2704889018 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f98d40792f0 con 0x7f98e81449d0 2024-05-26T22:49:03.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:03.709+0000 7f98edc00640 1 -- 172.21.15.131:0/2704889018 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f98e80b0250 con 0x7f98e81449d0 2024-05-26T22:49:03.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:03.716+0000 7f98dfe00640 1 -- 172.21.15.131:0/2704889018 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) ==== 122+0+0 (secure 0 0 0) 0x7f98d4005aa0 con 0x7f98e81449d0 2024-05-26T22:49:03.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:03.719+0000 7f98edc00640 1 -- 172.21.15.131:0/2704889018 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f98c806be30 msgr2=0x7f98c806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:03.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:03.719+0000 7f98edc00640 1 --2- 172.21.15.131:0/2704889018 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f98c806be30 0x7f98c806e2f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f98d8009fc0 tx=0x7f98d800b040 comp rx=0 tx=0).stop 2024-05-26T22:49:03.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:03.719+0000 7f98edc00640 1 -- 172.21.15.131:0/2704889018 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f98e81449d0 msgr2=0x7f98e81f7260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:03.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:03.719+0000 7f98edc00640 1 --2- 172.21.15.131:0/2704889018 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f98e81449d0 0x7f98e81f7260 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f98d402ad60 tx=0x7f98d4001970 comp rx=0 tx=0).stop 2024-05-26T22:49:03.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:03.719+0000 7f98edc00640 1 -- 172.21.15.131:0/2704889018 shutdown_connections 2024-05-26T22:49:03.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:03.719+0000 7f98edc00640 1 --2- 172.21.15.131:0/2704889018 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f98c806be30 0x7f98c806e2f0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:03.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:03.719+0000 7f98edc00640 1 --2- 172.21.15.131:0/2704889018 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f98e8144e50 0x7f98e81f77a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:03.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:03.719+0000 7f98edc00640 1 --2- 172.21.15.131:0/2704889018 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f98e81449d0 0x7f98e81f7260 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:03.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:03.719+0000 7f98edc00640 1 --2- 172.21.15.131:0/2704889018 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f98cc004800 0x7f98e815bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:03.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:03.719+0000 7f98edc00640 1 -- 172.21.15.131:0/2704889018 >> 172.21.15.131:0/2704889018 conn(0x7f98e80b5660 msgr2=0x7f98e80b0a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:03.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:03.720+0000 7f98edc00640 1 -- 172.21.15.131:0/2704889018 shutdown_connections 2024-05-26T22:49:03.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:03.720+0000 7f98edc00640 1 -- 172.21.15.131:0/2704889018 wait complete. 2024-05-26T22:49:03.737 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-05-26T22:49:03.738 DEBUG:teuthology.orchestra.run.smithi131:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-26T22:49:03.845 DEBUG:teuthology.orchestra.run.smithi187:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-26T22:49:03.950 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-05-26T22:49:03.950 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-26T22:49:03.950 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-26T22:49:04.008 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-26T22:49:04.009 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-26T22:49:04.065 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-26T22:49:04.065 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-26T22:49:04.122 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-26T22:49:04.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.336+0000 7f6e25000640 1 Processor -- start 2024-05-26T22:49:04.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.336+0000 7f6e25000640 1 -- start start 2024-05-26T22:49:04.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.337+0000 7f6e25000640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f6e200ad250 0x7f6e200b3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:04.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.337+0000 7f6e25000640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e2015cb20 con 0x7f6e2006ee20 2024-05-26T22:49:04.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.337+0000 7f6e25000640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e2015ce00 con 0x7f6e201569d0 2024-05-26T22:49:04.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.337+0000 7f6e25000640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f6e2015d0e0 con 0x7f6e200ad250 2024-05-26T22:49:04.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.338+0000 7f6e1de00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f6e2006ee20 0x7f6e201562c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:46606/0 (socket says 172.21.15.131:46606) 2024-05-26T22:49:04.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.338+0000 7f6e1de00640 1 -- 172.21.15.131:0/3944685859 learned_addr learned my addr 172.21.15.131:0/3944685859 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:49:04.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.338+0000 7f6e1d400640 1 --2- 172.21.15.131:0/3944685859 >> v2:172.21.15.187:3300/0 conn(0x7f6e200ad250 0x7f6e200b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:04.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.338+0000 7f6e1ca00640 1 -- 172.21.15.131:0/3944685859 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 893338060 0 0) 0x7f6e2015cb20 con 0x7f6e2006ee20 2024-05-26T22:49:04.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.338+0000 7f6e1ca00640 1 -- 172.21.15.131:0/3944685859 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6e04003520 con 0x7f6e2006ee20 2024-05-26T22:49:04.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.339+0000 7f6e1ca00640 1 -- 172.21.15.131:0/3944685859 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2785346551 0 0) 0x7f6e04003520 con 0x7f6e2006ee20 2024-05-26T22:49:04.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.339+0000 7f6e1ca00640 1 -- 172.21.15.131:0/3944685859 >> v2:172.21.15.187:3300/0 conn(0x7f6e200ad250 msgr2=0x7f6e200b3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:04.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.339+0000 7f6e1ca00640 1 --2- 172.21.15.131:0/3944685859 >> v2:172.21.15.187:3300/0 conn(0x7f6e200ad250 0x7f6e200b3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:04.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.339+0000 7f6e1ca00640 1 -- 172.21.15.131:0/3944685859 >> v1:172.21.15.187:6789/0 conn(0x7f6e201569d0 legacy=0x7f6e20160010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:04.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.339+0000 7f6e1ca00640 1 -- 172.21.15.131:0/3944685859 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e2015d3c0 con 0x7f6e2006ee20 2024-05-26T22:49:04.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.339+0000 7f6e1ca00640 1 -- 172.21.15.131:0/3944685859 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7f6e100019f0 con 0x7f6e2006ee20 2024-05-26T22:49:04.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.339+0000 7f6e1ca00640 1 -- 172.21.15.131:0/3944685859 >> v1:172.21.15.131:6789/0 conn(0x7f6e2006ee20 legacy=0x7f6e201562c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:04.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.339+0000 7f6e1ca00640 1 --2- 172.21.15.131:0/3944685859 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e040039b0 0x7f6e04003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:04.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.339+0000 7f6e1ca00640 1 --2- 172.21.15.131:0/3944685859 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6e04004800 0x7f6e04004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:04.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.339+0000 7f6e1ca00640 1 --2- 172.21.15.131:0/3944685859 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6e040051c0 0x7f6e0400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:04.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.339+0000 7f6e1ca00640 1 -- 172.21.15.131:0/3944685859 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f6e2015d0e0 con 0x7f6e040039b0 2024-05-26T22:49:04.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.339+0000 7f6e1ca00640 1 -- 172.21.15.131:0/3944685859 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6e0400c160 con 0x7f6e04004800 2024-05-26T22:49:04.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.339+0000 7f6e1ca00640 1 -- 172.21.15.131:0/3944685859 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f6e0400c360 con 0x7f6e040051c0 2024-05-26T22:49:04.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.339+0000 7f6e1de00640 1 --2- 172.21.15.131:0/3944685859 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e040039b0 0x7f6e04003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:04.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.339+0000 7f6e1d400640 1 --2- 172.21.15.131:0/3944685859 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6e04004800 0x7f6e04004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:04.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.339+0000 7f6e1de00640 1 -- 172.21.15.131:0/3944685859 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6e040051c0 msgr2=0x7f6e0400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:04.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.339+0000 7f6e1de00640 1 --2- 172.21.15.131:0/3944685859 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6e040051c0 0x7f6e0400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:04.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.339+0000 7f6e1de00640 1 -- 172.21.15.131:0/3944685859 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6e04004800 msgr2=0x7f6e04004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:04.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.339+0000 7f6e1de00640 1 --2- 172.21.15.131:0/3944685859 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6e04004800 0x7f6e04004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:04.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.340+0000 7f6e1de00640 1 -- 172.21.15.131:0/3944685859 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e04003520 con 0x7f6e040039b0 2024-05-26T22:49:04.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.340+0000 7f6e1e800640 1 -- 172.21.15.131:0/3944685859 reap_dead start 2024-05-26T22:49:04.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.341+0000 7f6e1de00640 1 --2- 172.21.15.131:0/3944685859 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e040039b0 0x7f6e04003dd0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f6e1000a860 tx=0x7f6e04005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e789220c1a13bb1a server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:04.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.341+0000 7f6e1ca00640 1 -- 172.21.15.131:0/3944685859 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e1000a480 con 0x7f6e040039b0 2024-05-26T22:49:04.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.341+0000 7f6e1ca00640 1 -- 172.21.15.131:0/3944685859 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e1002f070 con 0x7f6e040039b0 2024-05-26T22:49:04.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.341+0000 7f6e1ca00640 1 -- 172.21.15.131:0/3944685859 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e1002bd20 con 0x7f6e040039b0 2024-05-26T22:49:04.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.341+0000 7f6e25000640 1 -- 172.21.15.131:0/3944685859 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e040039b0 msgr2=0x7f6e04003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:04.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.341+0000 7f6e25000640 1 --2- 172.21.15.131:0/3944685859 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e040039b0 0x7f6e04003dd0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f6e1000a860 tx=0x7f6e04005830 comp rx=0 tx=0).stop 2024-05-26T22:49:04.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.341+0000 7f6e25000640 1 -- 172.21.15.131:0/3944685859 shutdown_connections 2024-05-26T22:49:04.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.341+0000 7f6e25000640 1 --2- 172.21.15.131:0/3944685859 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e040039b0 0x7f6e04003dd0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:04.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.341+0000 7f6e25000640 1 -- 172.21.15.131:0/3944685859 >> 172.21.15.131:0/3944685859 conn(0x7f6e200b1e90 msgr2=0x7f6e200b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:04.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.342+0000 7f6e25000640 1 -- 172.21.15.131:0/3944685859 shutdown_connections 2024-05-26T22:49:04.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.342+0000 7f6e25000640 1 -- 172.21.15.131:0/3944685859 wait complete. 2024-05-26T22:49:04.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.343+0000 7f6e25000640 1 Processor -- start 2024-05-26T22:49:04.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.343+0000 7f6e25000640 1 -- start start 2024-05-26T22:49:04.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.344+0000 7f6e25000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6e040039b0 0x7f6e2009a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:04.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.344+0000 7f6e25000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e20142dd0 0x7f6e2009c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:04.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.344+0000 7f6e25000640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6e2009db50 0x7f6e201420e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:04.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.344+0000 7f6e25000640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f6e2015c900 con 0x7f6e20142dd0 2024-05-26T22:49:04.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.344+0000 7f6e25000640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6e2015c780 con 0x7f6e2009db50 2024-05-26T22:49:04.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.344+0000 7f6e25000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f6e2006f200 con 0x7f6e040039b0 2024-05-26T22:49:04.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.345+0000 7f6e1d400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e20142dd0 0x7f6e2009c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:04.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.345+0000 7f6e1d400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e20142dd0 0x7f6e2009c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41718/0 (socket says 172.21.15.131:41718) 2024-05-26T22:49:04.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.345+0000 7f6e1d400640 1 -- 172.21.15.131:0/292547131 learned_addr learned my addr 172.21.15.131:0/292547131 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:04.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.345+0000 7f6e1d400640 1 -- 172.21.15.131:0/292547131 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6e040039b0 msgr2=0x7f6e2009a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:04.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.345+0000 7f6e1d400640 1 --2- 172.21.15.131:0/292547131 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6e040039b0 0x7f6e2009a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:04.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.345+0000 7f6e1d400640 1 -- 172.21.15.131:0/292547131 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6e2009db50 msgr2=0x7f6e201420e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:04.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.345+0000 7f6e1d400640 1 --2- 172.21.15.131:0/292547131 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6e2009db50 0x7f6e201420e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:04.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.345+0000 7f6e1d400640 1 -- 172.21.15.131:0/292547131 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e2009c6b0 con 0x7f6e20142dd0 2024-05-26T22:49:04.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.345+0000 7f6e1d400640 1 --2- 172.21.15.131:0/292547131 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e20142dd0 0x7f6e2009c170 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f6e080115d0 tx=0x7f6e08011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:04.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.345+0000 7f6e16000640 1 -- 172.21.15.131:0/292547131 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e0800f8f0 con 0x7f6e20142dd0 2024-05-26T22:49:04.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.346+0000 7f6e16000640 1 -- 172.21.15.131:0/292547131 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e0800fa90 con 0x7f6e20142dd0 2024-05-26T22:49:04.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.346+0000 7f6e16000640 1 -- 172.21.15.131:0/292547131 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e0800fe60 con 0x7f6e20142dd0 2024-05-26T22:49:04.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.346+0000 7f6e25000640 1 -- 172.21.15.131:0/292547131 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e201400f0 con 0x7f6e20142dd0 2024-05-26T22:49:04.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.346+0000 7f6e25000640 1 -- 172.21.15.131:0/292547131 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e201406b0 con 0x7f6e20142dd0 2024-05-26T22:49:04.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.347+0000 7f6e25000640 1 -- 172.21.15.131:0/292547131 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e200694e0 con 0x7f6e20142dd0 2024-05-26T22:49:04.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.348+0000 7f6e16000640 1 -- 172.21.15.131:0/292547131 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6e08015460 con 0x7f6e20142dd0 2024-05-26T22:49:04.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.349+0000 7f6e16000640 1 --2- 172.21.15.131:0/292547131 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f6dfc06be30 0x7f6dfc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:04.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.349+0000 7f6e16000640 1 -- 172.21.15.131:0/292547131 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f6e0808b870 con 0x7f6e20142dd0 2024-05-26T22:49:04.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.349+0000 7f6e1de00640 1 --2- 172.21.15.131:0/292547131 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f6dfc06be30 0x7f6dfc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:04.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.350+0000 7f6e1de00640 1 --2- 172.21.15.131:0/292547131 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f6dfc06be30 0x7f6dfc06e2f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6e10033000 tx=0x7f6e10007000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:04.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.354+0000 7f6e16000640 1 -- 172.21.15.131:0/292547131 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f6e08019030 con 0x7f6e20142dd0 2024-05-26T22:49:04.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.553+0000 7f6e25000640 1 -- 172.21.15.131:0/292547131 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6e200b2550 con 0x7f6e20142dd0 2024-05-26T22:49:04.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.556+0000 7f6e16000640 1 -- 172.21.15.131:0/292547131 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16879 (secure 0 0 0) 0x7f6e0805a100 con 0x7f6e20142dd0 2024-05-26T22:49:04.556 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:49:04.556 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":22,"fsid":"46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f","created":"2024-05-26T22:47:57.288905+0000","modified":"2024-05-26T22:49:04.278988+0000","last_up_change":"2024-05-26T22:48:09.602207+0000","last_in_change":"2024-05-26T22:48:00.687341+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-26T22:48:10.463339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-26T22:48:19.663254+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-26T22:48:20.235751+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"814eaf62-a18d-494b-8e27-fd8fb88adea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6801","nonce":2350940051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6803","nonce":2350940051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6807","nonce":2350940051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6805","nonce":2350940051}]},"public_addr":"172.21.15.131:6801/2350940051","cluster_addr":"172.21.15.131:6803/2350940051","heartbeat_back_addr":"172.21.15.131:6807/2350940051","heartbeat_front_addr":"172.21.15.131:6805/2350940051","state":["exists","up"]},{"osd":1,"uuid":"42e66287-f6c6-4e26-af3c-6dfaa3536d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6825","nonce":1525528247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6827","nonce":1525528247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6831","nonce":1525528247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6829","nonce":1525528247}]},"public_addr":"172.21.15.131:6825/1525528247","cluster_addr":"172.21.15.131:6827/1525528247","heartbeat_back_addr":"172.21.15.131:6831/1525528247","heartbeat_front_addr":"172.21.15.131:6829/1525528247","state":["exists","up"]},{"osd":2,"uuid":"0da34673-04a2-4fd0-b57a-0ff2b679b043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6817","nonce":3627842210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6819","nonce":3627842210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6823","nonce":3627842210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6821","nonce":3627842210}]},"public_addr":"172.21.15.131:6817/3627842210","cluster_addr":"172.21.15.131:6819/3627842210","heartbeat_back_addr":"172.21.15.131:6823/3627842210","heartbeat_front_addr":"172.21.15.131:6821/3627842210","state":["exists","up"]},{"osd":3,"uuid":"cf117a27-d6b2-43d3-ae52-b47988bf8eed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6809","nonce":350790591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6811","nonce":350790591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6815","nonce":350790591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6813","nonce":350790591}]},"public_addr":"172.21.15.131:6809/350790591","cluster_addr":"172.21.15.131:6811/350790591","heartbeat_back_addr":"172.21.15.131:6815/350790591","heartbeat_front_addr":"172.21.15.131:6813/350790591","state":["exists","up"]},{"osd":4,"uuid":"a092ccfc-be89-4a84-88d5-0f781fa715eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6816","nonce":244560995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6819","nonce":244560995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6823","nonce":244560995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6821","nonce":244560995}]},"public_addr":"172.21.15.187:6816/244560995","cluster_addr":"172.21.15.187:6819/244560995","heartbeat_back_addr":"172.21.15.187:6823/244560995","heartbeat_front_addr":"172.21.15.187:6821/244560995","state":["exists","up"]},{"osd":5,"uuid":"7513ba69-dd00-4a35-b190-9e8cc3b2bdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6801","nonce":324619398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6803","nonce":324619398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6812","nonce":324619398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6807","nonce":324619398}]},"public_addr":"172.21.15.187:6801/324619398","cluster_addr":"172.21.15.187:6803/324619398","heartbeat_back_addr":"172.21.15.187:6812/324619398","heartbeat_front_addr":"172.21.15.187:6807/324619398","state":["exists","up"]},{"osd":6,"uuid":"50081810-c093-4d2f-a4d7-b7c695c5400d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6806","nonce":1391347786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6809","nonce":1391347786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6818","nonce":1391347786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6813","nonce":1391347786}]},"public_addr":"172.21.15.187:6806/1391347786","cluster_addr":"172.21.15.187:6809/1391347786","heartbeat_back_addr":"172.21.15.187:6818/1391347786","heartbeat_front_addr":"172.21.15.187:6813/1391347786","state":["exists","up"]},{"osd":7,"uuid":"775dae05-54f5-4ad4-bbce-92863c6ed2ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6825","nonce":1220065622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6827","nonce":1220065622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6831","nonce":1220065622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6829","nonce":1220065622}]},"public_addr":"172.21.15.187:6825/1220065622","cluster_addr":"172.21.15.187:6827/1220065622","heartbeat_back_addr":"172.21.15.187:6831/1220065622","heartbeat_front_addr":"172.21.15.187:6829/1220065622","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:05.983520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.299516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.067323+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.307748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.018742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.258775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.010912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.295355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.1e","mappings":[{"from":3,"to":2}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":7,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-26T22:49:04.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.559+0000 7f6e25000640 1 -- 172.21.15.131:0/292547131 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f6dfc06be30 msgr2=0x7f6dfc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:04.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.559+0000 7f6e25000640 1 --2- 172.21.15.131:0/292547131 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f6dfc06be30 0x7f6dfc06e2f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6e10033000 tx=0x7f6e10007000 comp rx=0 tx=0).stop 2024-05-26T22:49:04.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.559+0000 7f6e25000640 1 -- 172.21.15.131:0/292547131 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e20142dd0 msgr2=0x7f6e2009c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:04.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.559+0000 7f6e25000640 1 --2- 172.21.15.131:0/292547131 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e20142dd0 0x7f6e2009c170 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f6e080115d0 tx=0x7f6e08011a90 comp rx=0 tx=0).stop 2024-05-26T22:49:04.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.559+0000 7f6e25000640 1 -- 172.21.15.131:0/292547131 shutdown_connections 2024-05-26T22:49:04.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.559+0000 7f6e25000640 1 --2- 172.21.15.131:0/292547131 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f6dfc06be30 0x7f6dfc06e2f0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:04.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.559+0000 7f6e25000640 1 --2- 172.21.15.131:0/292547131 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6e2009db50 0x7f6e201420e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:04.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.559+0000 7f6e25000640 1 --2- 172.21.15.131:0/292547131 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e20142dd0 0x7f6e2009c170 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:04.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.559+0000 7f6e25000640 1 --2- 172.21.15.131:0/292547131 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6e040039b0 0x7f6e2009a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:04.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.559+0000 7f6e25000640 1 -- 172.21.15.131:0/292547131 >> 172.21.15.131:0/292547131 conn(0x7f6e200b1e90 msgr2=0x7f6e2015e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:04.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.559+0000 7f6e25000640 1 -- 172.21.15.131:0/292547131 shutdown_connections 2024-05-26T22:49:04.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.559+0000 7f6e25000640 1 -- 172.21.15.131:0/292547131 wait complete. 2024-05-26T22:49:04.578 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-26T22:48:10.463339+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-26T22:48:19.663254+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-26T22:48:20.235751+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-26T22:49:04.579 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-26T22:49:04.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.780+0000 7f5f39c00640 1 Processor -- start 2024-05-26T22:49:04.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.780+0000 7f5f39c00640 1 -- start start 2024-05-26T22:49:04.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.782+0000 7f5f39c00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5f34155770 0x7f5f3415b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:04.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.782+0000 7f5f39c00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f5f3415bc40 0x7f5f3406dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:04.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.782+0000 7f5f39c00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f5f3406e270 0x7f5f340ceba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:04.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.782+0000 7f5f39c00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f5f3406e6a0 con 0x7f5f3406e270 2024-05-26T22:49:04.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.782+0000 7f5f39c00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f5f3415c1c0 con 0x7f5f3415bc40 2024-05-26T22:49:04.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.782+0000 7f5f39c00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f5f3415c4a0 con 0x7f5f34155770 2024-05-26T22:49:04.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.783+0000 7f5f33400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f5f3406e270 0x7f5f340ceba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:04.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.783+0000 7f5f33400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f5f3406e270 0x7f5f340ceba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41732/0 (socket says 172.21.15.131:41732) 2024-05-26T22:49:04.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.783+0000 7f5f33400640 1 -- 172.21.15.131:0/188472930 learned_addr learned my addr 172.21.15.131:0/188472930 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:04.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.783+0000 7f5f32a00640 1 --2- 172.21.15.131:0/188472930 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5f34155770 0x7f5f3415b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:04.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.783+0000 7f5f32000640 1 --2- 172.21.15.131:0/188472930 >> v2:172.21.15.187:3300/0 conn(0x7f5f3415bc40 0x7f5f3406dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:04.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.783+0000 7f5f33400640 1 -- 172.21.15.131:0/188472930 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5f34155770 msgr2=0x7f5f3415b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:04.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.783+0000 7f5f33400640 1 --2- 172.21.15.131:0/188472930 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5f34155770 0x7f5f3415b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:04.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.783+0000 7f5f33400640 1 -- 172.21.15.131:0/188472930 >> v2:172.21.15.187:3300/0 conn(0x7f5f3415bc40 msgr2=0x7f5f3406dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:04.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.783+0000 7f5f33400640 1 --2- 172.21.15.131:0/188472930 >> v2:172.21.15.187:3300/0 conn(0x7f5f3415bc40 0x7f5f3406dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:04.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.783+0000 7f5f33400640 1 -- 172.21.15.131:0/188472930 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f34071da0 con 0x7f5f3406e270 2024-05-26T22:49:04.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.784+0000 7f5f33400640 1 --2- 172.21.15.131:0/188472930 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5f3406e270 0x7f5f340ceba0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f5f340cfd60 tx=0x7f5f2c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75390f3f954da757 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:04.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.784+0000 7f5f31600640 1 -- 172.21.15.131:0/188472930 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f2c037070 con 0x7f5f3406e270 2024-05-26T22:49:04.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.785+0000 7f5f31600640 1 -- 172.21.15.131:0/188472930 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f2c032440 con 0x7f5f3406e270 2024-05-26T22:49:04.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.785+0000 7f5f31600640 1 -- 172.21.15.131:0/188472930 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f2c032810 con 0x7f5f3406e270 2024-05-26T22:49:04.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.785+0000 7f5f39c00640 1 -- 172.21.15.131:0/188472930 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5f3406e270 msgr2=0x7f5f340ceba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:04.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.785+0000 7f5f39c00640 1 --2- 172.21.15.131:0/188472930 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5f3406e270 0x7f5f340ceba0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f5f340cfd60 tx=0x7f5f2c02a270 comp rx=0 tx=0).stop 2024-05-26T22:49:04.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.785+0000 7f5f39c00640 1 -- 172.21.15.131:0/188472930 shutdown_connections 2024-05-26T22:49:04.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.785+0000 7f5f39c00640 1 --2- 172.21.15.131:0/188472930 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5f3406e270 0x7f5f340ceba0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:04.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.785+0000 7f5f39c00640 1 --2- 172.21.15.131:0/188472930 >> v2:172.21.15.187:3300/0 conn(0x7f5f3415bc40 0x7f5f3406dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:04.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.785+0000 7f5f39c00640 1 --2- 172.21.15.131:0/188472930 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5f34155770 0x7f5f3415b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:04.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.785+0000 7f5f39c00640 1 -- 172.21.15.131:0/188472930 >> 172.21.15.131:0/188472930 conn(0x7f5f3406bc90 msgr2=0x7f5f340b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:04.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.785+0000 7f5f39c00640 1 -- 172.21.15.131:0/188472930 shutdown_connections 2024-05-26T22:49:04.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.786+0000 7f5f39c00640 1 -- 172.21.15.131:0/188472930 wait complete. 2024-05-26T22:49:04.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.788+0000 7f5f39c00640 1 Processor -- start 2024-05-26T22:49:04.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.788+0000 7f5f39c00640 1 -- start start 2024-05-26T22:49:04.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.789+0000 7f5f39c00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5f3406e270 0x7f5f340cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:04.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.789+0000 7f5f39c00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5f34155770 0x7f5f3409aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:04.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.789+0000 7f5f39c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5f3415bc40 0x7f5f3409afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:04.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.789+0000 7f5f39c00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f5f3415c8c0 con 0x7f5f3415bc40 2024-05-26T22:49:04.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.789+0000 7f5f39c00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5f34155b50 con 0x7f5f3406e270 2024-05-26T22:49:04.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.789+0000 7f5f39c00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f5f3409de00 con 0x7f5f34155770 2024-05-26T22:49:04.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.790+0000 7f5f33400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5f3415bc40 0x7f5f3409afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:04.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.790+0000 7f5f33400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5f3415bc40 0x7f5f3409afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41746/0 (socket says 172.21.15.131:41746) 2024-05-26T22:49:04.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.790+0000 7f5f33400640 1 -- 172.21.15.131:0/948411947 learned_addr learned my addr 172.21.15.131:0/948411947 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:04.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.790+0000 7f5f32000640 1 --2- 172.21.15.131:0/948411947 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5f34155770 0x7f5f3409aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:04.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.790+0000 7f5f32a00640 1 --2- 172.21.15.131:0/948411947 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5f3406e270 0x7f5f340cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:04.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.790+0000 7f5f33400640 1 -- 172.21.15.131:0/948411947 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5f34155770 msgr2=0x7f5f3409aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:04.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.790+0000 7f5f33400640 1 --2- 172.21.15.131:0/948411947 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5f34155770 0x7f5f3409aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:04.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.790+0000 7f5f33400640 1 -- 172.21.15.131:0/948411947 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5f3406e270 msgr2=0x7f5f340cffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:04.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.790+0000 7f5f33400640 1 --2- 172.21.15.131:0/948411947 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5f3406e270 0x7f5f340cffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:04.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.790+0000 7f5f33400640 1 -- 172.21.15.131:0/948411947 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f3409b7e0 con 0x7f5f3415bc40 2024-05-26T22:49:04.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.790+0000 7f5f33400640 1 --2- 172.21.15.131:0/948411947 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5f3415bc40 0x7f5f3409afb0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c0090b0 tx=0x7f5f2c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:04.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.791+0000 7f5f2aa00640 1 -- 172.21.15.131:0/948411947 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f2c037040 con 0x7f5f3415bc40 2024-05-26T22:49:04.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.791+0000 7f5f2aa00640 1 -- 172.21.15.131:0/948411947 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f2c042070 con 0x7f5f3415bc40 2024-05-26T22:49:04.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.791+0000 7f5f2aa00640 1 -- 172.21.15.131:0/948411947 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f2c030920 con 0x7f5f3415bc40 2024-05-26T22:49:04.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.791+0000 7f5f39c00640 1 -- 172.21.15.131:0/948411947 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f3409bda0 con 0x7f5f3415bc40 2024-05-26T22:49:04.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.791+0000 7f5f39c00640 1 -- 172.21.15.131:0/948411947 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f3409c360 con 0x7f5f3415bc40 2024-05-26T22:49:04.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.792+0000 7f5f39c00640 1 -- 172.21.15.131:0/948411947 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f34074e90 con 0x7f5f3415bc40 2024-05-26T22:49:04.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.793+0000 7f5f2aa00640 1 -- 172.21.15.131:0/948411947 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5f2c030ac0 con 0x7f5f3415bc40 2024-05-26T22:49:04.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.794+0000 7f5f2aa00640 1 --2- 172.21.15.131:0/948411947 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f5f1406be60 0x7f5f1406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:04.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.794+0000 7f5f2aa00640 1 -- 172.21.15.131:0/948411947 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f5f2c0ab7e0 con 0x7f5f3415bc40 2024-05-26T22:49:04.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.794+0000 7f5f32a00640 1 --2- 172.21.15.131:0/948411947 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f5f1406be60 0x7f5f1406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:04.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.795+0000 7f5f32a00640 1 --2- 172.21.15.131:0/948411947 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f5f1406be60 0x7f5f1406e320 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f5f24004450 tx=0x7f5f24003a30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:04.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.799+0000 7f5f2aa00640 1 -- 172.21.15.131:0/948411947 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f5f2c079ab0 con 0x7f5f3415bc40 2024-05-26T22:49:04.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.994+0000 7f5f39c00640 1 -- 172.21.15.131:0/948411947 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5f340695c0 con 0x7f5f3415bc40 2024-05-26T22:49:04.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.995+0000 7f5f2aa00640 1 -- 172.21.15.131:0/948411947 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7f5f2c07ddb0 con 0x7f5f3415bc40 2024-05-26T22:49:04.995 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2024-05-26T22:49:04.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.997+0000 7f5f39c00640 1 -- 172.21.15.131:0/948411947 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f5f1406be60 msgr2=0x7f5f1406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:04.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.997+0000 7f5f39c00640 1 --2- 172.21.15.131:0/948411947 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f5f1406be60 0x7f5f1406e320 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f5f24004450 tx=0x7f5f24003a30 comp rx=0 tx=0).stop 2024-05-26T22:49:04.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.997+0000 7f5f39c00640 1 -- 172.21.15.131:0/948411947 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5f3415bc40 msgr2=0x7f5f3409afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:04.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.997+0000 7f5f39c00640 1 --2- 172.21.15.131:0/948411947 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5f3415bc40 0x7f5f3409afb0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c0090b0 tx=0x7f5f2c0046c0 comp rx=0 tx=0).stop 2024-05-26T22:49:04.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.998+0000 7f5f39c00640 1 -- 172.21.15.131:0/948411947 shutdown_connections 2024-05-26T22:49:04.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.998+0000 7f5f39c00640 1 --2- 172.21.15.131:0/948411947 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5f34155770 0x7f5f3409aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:04.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.998+0000 7f5f39c00640 1 --2- 172.21.15.131:0/948411947 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f5f1406be60 0x7f5f1406e320 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:04.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.998+0000 7f5f39c00640 1 --2- 172.21.15.131:0/948411947 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5f3415bc40 0x7f5f3409afb0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:04.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.998+0000 7f5f39c00640 1 --2- 172.21.15.131:0/948411947 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5f3406e270 0x7f5f340cffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:04.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.998+0000 7f5f39c00640 1 -- 172.21.15.131:0/948411947 >> 172.21.15.131:0/948411947 conn(0x7f5f3406bc90 msgr2=0x7f5f3415a0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:04.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.998+0000 7f5f39c00640 1 -- 172.21.15.131:0/948411947 shutdown_connections 2024-05-26T22:49:04.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:04.998+0000 7f5f39c00640 1 -- 172.21.15.131:0/948411947 wait complete. 2024-05-26T22:49:05.016 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-26T22:49:05.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.221+0000 7f36fd200640 1 Processor -- start 2024-05-26T22:49:05.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.221+0000 7f36fd200640 1 -- start start 2024-05-26T22:49:05.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.222+0000 7f36fd200640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f36f80b5450 0x7f36f80b43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:05.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.222+0000 7f36fd200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f36f80d0460 0x7f36f806d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:05.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.223+0000 7f36fd200640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36f80b4900 con 0x7f36f806f420 2024-05-26T22:49:05.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.223+0000 7f36fd200640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f36f8160d90 con 0x7f36f80d0460 2024-05-26T22:49:05.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.223+0000 7f36fd200640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f36f8160180 con 0x7f36f80b5450 2024-05-26T22:49:05.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.223+0000 7f36f5400640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f36f806f420 0x7f36f80cfd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:46614/0 (socket says 172.21.15.131:46614) 2024-05-26T22:49:05.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.223+0000 7f36f6800640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f36f80d0460 0x7f36f806d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:05.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.223+0000 7f36f5400640 1 -- 172.21.15.131:0/1645362761 learned_addr learned my addr 172.21.15.131:0/1645362761 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:49:05.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.223+0000 7f36f4a00640 1 -- 172.21.15.131:0/1645362761 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1594184116 0 0) 0x7f36f80b4900 con 0x7f36f806f420 2024-05-26T22:49:05.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.223+0000 7f36f4a00640 1 -- 172.21.15.131:0/1645362761 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f36dc003520 con 0x7f36f806f420 2024-05-26T22:49:05.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.224+0000 7f36f6800640 1 -- 172.21.15.131:0/1645362761 >> v2:172.21.15.187:3300/0 conn(0x7f36f80b5450 msgr2=0x7f36f80b43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:05.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.224+0000 7f36f6800640 1 --2- 172.21.15.131:0/1645362761 >> v2:172.21.15.187:3300/0 conn(0x7f36f80b5450 0x7f36f80b43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:05.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.224+0000 7f36f6800640 1 -- 172.21.15.131:0/1645362761 >> v1:172.21.15.131:6789/0 conn(0x7f36f806f420 legacy=0x7f36f80cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:05.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.224+0000 7f36f6800640 1 -- 172.21.15.131:0/1645362761 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36f8160460 con 0x7f36f80d0460 2024-05-26T22:49:05.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.225+0000 7f36f6800640 1 --2- 172.21.15.131:0/1645362761 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f36f80d0460 0x7f36f806d600 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f36f0009d90 tx=0x7f36f002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f6e4732a450200c server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:05.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.225+0000 7f36f4a00640 1 -- 172.21.15.131:0/1645362761 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f0007740 con 0x7f36f80d0460 2024-05-26T22:49:05.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.225+0000 7f36f4a00640 1 -- 172.21.15.131:0/1645362761 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36f00078e0 con 0x7f36f80d0460 2024-05-26T22:49:05.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.225+0000 7f36f4a00640 1 -- 172.21.15.131:0/1645362761 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f0007cb0 con 0x7f36f80d0460 2024-05-26T22:49:05.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.225+0000 7f36fd200640 1 -- 172.21.15.131:0/1645362761 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f36f80d0460 msgr2=0x7f36f806d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:05.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.225+0000 7f36fd200640 1 --2- 172.21.15.131:0/1645362761 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f36f80d0460 0x7f36f806d600 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f36f0009d90 tx=0x7f36f002a6a0 comp rx=0 tx=0).stop 2024-05-26T22:49:05.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.226+0000 7f36fd200640 1 -- 172.21.15.131:0/1645362761 shutdown_connections 2024-05-26T22:49:05.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.226+0000 7f36fd200640 1 --2- 172.21.15.131:0/1645362761 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f36f80d0460 0x7f36f806d600 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:05.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.226+0000 7f36fd200640 1 --2- 172.21.15.131:0/1645362761 >> v2:172.21.15.187:3300/0 conn(0x7f36f80b5450 0x7f36f80b43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:05.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.226+0000 7f36fd200640 1 -- 172.21.15.131:0/1645362761 >> 172.21.15.131:0/1645362761 conn(0x7f36f806bc90 msgr2=0x7f36f80b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:05.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.226+0000 7f36fd200640 1 -- 172.21.15.131:0/1645362761 shutdown_connections 2024-05-26T22:49:05.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.226+0000 7f36fd200640 1 -- 172.21.15.131:0/1645362761 wait complete. 2024-05-26T22:49:05.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.227+0000 7f36fd200640 1 Processor -- start 2024-05-26T22:49:05.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.228+0000 7f36fd200640 1 -- start start 2024-05-26T22:49:05.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.228+0000 7f36fd200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f36f806f420 0x7f36f8142820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:05.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.229+0000 7f36fd200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f36f80b5450 0x7f36f81572b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:05.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.229+0000 7f36fd200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f36f80d0460 0x7f36f81577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:05.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.229+0000 7f36fd200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f36f80b5830 con 0x7f36f806f420 2024-05-26T22:49:05.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.229+0000 7f36fd200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f36f8160a20 con 0x7f36f80b5450 2024-05-26T22:49:05.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.229+0000 7f36fd200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f36f815fde0 con 0x7f36f80d0460 2024-05-26T22:49:05.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.229+0000 7f36f5e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f36f806f420 0x7f36f8142820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:05.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.229+0000 7f36f5e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f36f806f420 0x7f36f8142820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41772/0 (socket says 172.21.15.131:41772) 2024-05-26T22:49:05.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.229+0000 7f36f5e00640 1 -- 172.21.15.131:0/840002986 learned_addr learned my addr 172.21.15.131:0/840002986 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:05.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.229+0000 7f36f5e00640 1 -- 172.21.15.131:0/840002986 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f36f80d0460 msgr2=0x7f36f81577f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:05.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.229+0000 7f36f5e00640 1 --2- 172.21.15.131:0/840002986 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f36f80d0460 0x7f36f81577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:05.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.229+0000 7f36f5400640 1 --2- 172.21.15.131:0/840002986 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f36f80b5450 0x7f36f81572b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:05.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.229+0000 7f36f5e00640 1 -- 172.21.15.131:0/840002986 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f36f80b5450 msgr2=0x7f36f81572b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:05.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.229+0000 7f36f5e00640 1 --2- 172.21.15.131:0/840002986 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f36f80b5450 0x7f36f81572b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:05.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.229+0000 7f36f5e00640 1 -- 172.21.15.131:0/840002986 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36f80715b0 con 0x7f36f806f420 2024-05-26T22:49:05.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.230+0000 7f36f5e00640 1 --2- 172.21.15.131:0/840002986 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f36f806f420 0x7f36f8142820 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f36e800b4d0 tx=0x7f36e800b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:05.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.230+0000 7f36ee000640 1 -- 172.21.15.131:0/840002986 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e8014070 con 0x7f36f806f420 2024-05-26T22:49:05.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.230+0000 7f36ee000640 1 -- 172.21.15.131:0/840002986 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36e8010870 con 0x7f36f806f420 2024-05-26T22:49:05.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.230+0000 7f36fd200640 1 -- 172.21.15.131:0/840002986 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36f8071890 con 0x7f36f806f420 2024-05-26T22:49:05.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.230+0000 7f36ee000640 1 -- 172.21.15.131:0/840002986 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e8010c60 con 0x7f36f806f420 2024-05-26T22:49:05.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.230+0000 7f36fd200640 1 -- 172.21.15.131:0/840002986 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36f8071e50 con 0x7f36f806f420 2024-05-26T22:49:05.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.231+0000 7f36fd200640 1 -- 172.21.15.131:0/840002986 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36f8074e90 con 0x7f36f806f420 2024-05-26T22:49:05.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.232+0000 7f36ee000640 1 -- 172.21.15.131:0/840002986 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f36e8017070 con 0x7f36f806f420 2024-05-26T22:49:05.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.233+0000 7f36ee000640 1 --2- 172.21.15.131:0/840002986 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f36d806be30 0x7f36d806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:05.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.233+0000 7f36ee000640 1 -- 172.21.15.131:0/840002986 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f36e808ad20 con 0x7f36f806f420 2024-05-26T22:49:05.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.233+0000 7f36f5400640 1 --2- 172.21.15.131:0/840002986 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f36d806be30 0x7f36d806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:05.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.234+0000 7f36f5400640 1 --2- 172.21.15.131:0/840002986 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f36d806be30 0x7f36d806e2f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f36f81514d0 tx=0x7f36e4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:05.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.238+0000 7f36ee000640 1 -- 172.21.15.131:0/840002986 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f36e8059650 con 0x7f36f806f420 2024-05-26T22:49:05.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.436+0000 7f36fd200640 1 -- 172.21.15.131:0/840002986 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f36f80695c0 con 0x7f36f806f420 2024-05-26T22:49:05.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.437+0000 7f36ee000640 1 -- 172.21.15.131:0/840002986 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7f36e8058f50 con 0x7f36f806f420 2024-05-26T22:49:05.437 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-05-26T22:49:05.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.440+0000 7f36fd200640 1 -- 172.21.15.131:0/840002986 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f36d806be30 msgr2=0x7f36d806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:05.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.440+0000 7f36fd200640 1 --2- 172.21.15.131:0/840002986 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f36d806be30 0x7f36d806e2f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f36f81514d0 tx=0x7f36e4009040 comp rx=0 tx=0).stop 2024-05-26T22:49:05.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.440+0000 7f36fd200640 1 -- 172.21.15.131:0/840002986 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f36f806f420 msgr2=0x7f36f8142820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:05.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.440+0000 7f36fd200640 1 --2- 172.21.15.131:0/840002986 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f36f806f420 0x7f36f8142820 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f36e800b4d0 tx=0x7f36e800b990 comp rx=0 tx=0).stop 2024-05-26T22:49:05.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.440+0000 7f36fd200640 1 -- 172.21.15.131:0/840002986 shutdown_connections 2024-05-26T22:49:05.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.440+0000 7f36fd200640 1 --2- 172.21.15.131:0/840002986 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f36f80d0460 0x7f36f81577f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:05.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.440+0000 7f36fd200640 1 --2- 172.21.15.131:0/840002986 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f36d806be30 0x7f36d806e2f0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:05.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.440+0000 7f36fd200640 1 --2- 172.21.15.131:0/840002986 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f36f80b5450 0x7f36f81572b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:05.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.440+0000 7f36fd200640 1 --2- 172.21.15.131:0/840002986 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f36f806f420 0x7f36f8142820 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:05.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.440+0000 7f36fd200640 1 -- 172.21.15.131:0/840002986 >> 172.21.15.131:0/840002986 conn(0x7f36f806bc90 msgr2=0x7f36f80b1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:05.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.441+0000 7f36fd200640 1 -- 172.21.15.131:0/840002986 shutdown_connections 2024-05-26T22:49:05.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.441+0000 7f36fd200640 1 -- 172.21.15.131:0/840002986 wait complete. 2024-05-26T22:49:05.459 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-26T22:49:05.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.659+0000 7f1c79e00640 1 Processor -- start 2024-05-26T22:49:05.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.660+0000 7f1c79e00640 1 -- start start 2024-05-26T22:49:05.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.661+0000 7f1c79e00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f1c74165e30 0x7f1c74072540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:05.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.661+0000 7f1c79e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f1c740d0560 0x7f1c740cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:05.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.661+0000 7f1c79e00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c740733c0 con 0x7f1c740af230 2024-05-26T22:49:05.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.661+0000 7f1c79e00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f1c74072e90 con 0x7f1c740d0560 2024-05-26T22:49:05.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.661+0000 7f1c79e00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f1c740cf540 con 0x7f1c74165e30 2024-05-26T22:49:05.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.662+0000 7f1c72000640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f1c740af230 0x7f1c740cfe50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:46620/0 (socket says 172.21.15.131:46620) 2024-05-26T22:49:05.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.662+0000 7f1c72000640 1 -- 172.21.15.131:0/923870892 learned_addr learned my addr 172.21.15.131:0/923870892 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:49:05.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.662+0000 7f1c73400640 1 --2- 172.21.15.131:0/923870892 >> v2:172.21.15.131:3300/0 conn(0x7f1c740d0560 0x7f1c740cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:05.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.662+0000 7f1c72a00640 1 --2- 172.21.15.131:0/923870892 >> v2:172.21.15.187:3300/0 conn(0x7f1c74165e30 0x7f1c74072540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:05.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.662+0000 7f1c73400640 1 -- 172.21.15.131:0/923870892 >> v2:172.21.15.187:3300/0 conn(0x7f1c74165e30 msgr2=0x7f1c74072540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:05.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.662+0000 7f1c73400640 1 --2- 172.21.15.131:0/923870892 >> v2:172.21.15.187:3300/0 conn(0x7f1c74165e30 0x7f1c74072540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:05.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.662+0000 7f1c73400640 1 -- 172.21.15.131:0/923870892 >> v1:172.21.15.131:6789/0 conn(0x7f1c740af230 legacy=0x7f1c740cfe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:05.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.662+0000 7f1c73400640 1 -- 172.21.15.131:0/923870892 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c740730e0 con 0x7f1c740d0560 2024-05-26T22:49:05.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.662+0000 7f1c71600640 1 -- 172.21.15.131:0/923870892 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3098778047 0 0) 0x7f1c740733c0 con 0x7f1c740af230 2024-05-26T22:49:05.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.663+0000 7f1c73400640 1 --2- 172.21.15.131:0/923870892 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1c740d0560 0x7f1c740cdcd0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f1c740d0c40 tx=0x7f1c6c02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb65f6e178d222cf server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:05.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.664+0000 7f1c71600640 1 -- 172.21.15.131:0/923870892 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c6c037070 con 0x7f1c740d0560 2024-05-26T22:49:05.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.664+0000 7f1c71600640 1 -- 172.21.15.131:0/923870892 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c6c032440 con 0x7f1c740d0560 2024-05-26T22:49:05.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.664+0000 7f1c71600640 1 -- 172.21.15.131:0/923870892 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c6c02ae60 con 0x7f1c740d0560 2024-05-26T22:49:05.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.664+0000 7f1c79e00640 1 -- 172.21.15.131:0/923870892 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1c740d0560 msgr2=0x7f1c740cdcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:05.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.664+0000 7f1c79e00640 1 --2- 172.21.15.131:0/923870892 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1c740d0560 0x7f1c740cdcd0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f1c740d0c40 tx=0x7f1c6c02a110 comp rx=0 tx=0).stop 2024-05-26T22:49:05.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.665+0000 7f1c79e00640 1 -- 172.21.15.131:0/923870892 shutdown_connections 2024-05-26T22:49:05.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.665+0000 7f1c79e00640 1 --2- 172.21.15.131:0/923870892 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1c740d0560 0x7f1c740cdcd0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:05.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.665+0000 7f1c79e00640 1 --2- 172.21.15.131:0/923870892 >> v2:172.21.15.187:3300/0 conn(0x7f1c74165e30 0x7f1c74072540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:05.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.665+0000 7f1c79e00640 1 -- 172.21.15.131:0/923870892 >> 172.21.15.131:0/923870892 conn(0x7f1c7406bc90 msgr2=0x7f1c740b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:05.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.665+0000 7f1c79e00640 1 -- 172.21.15.131:0/923870892 shutdown_connections 2024-05-26T22:49:05.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.665+0000 7f1c79e00640 1 -- 172.21.15.131:0/923870892 wait complete. 2024-05-26T22:49:05.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.666+0000 7f1c79e00640 1 Processor -- start 2024-05-26T22:49:05.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.667+0000 7f1c79e00640 1 -- start start 2024-05-26T22:49:05.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.667+0000 7f1c79e00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1c740af230 0x7f1c74155220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:05.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.667+0000 7f1c79e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1c740d0560 0x7f1c7409b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:05.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.668+0000 7f1c79e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1c74165e30 0x7f1c740ae160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:05.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.668+0000 7f1c79e00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f1c740ce210 con 0x7f1c74165e30 2024-05-26T22:49:05.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.668+0000 7f1c79e00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1c740cfac0 con 0x7f1c740af230 2024-05-26T22:49:05.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.668+0000 7f1c79e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1c740ce390 con 0x7f1c740d0560 2024-05-26T22:49:05.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.668+0000 7f1c73400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1c74165e30 0x7f1c740ae160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:05.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.668+0000 7f1c73400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1c74165e30 0x7f1c740ae160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41792/0 (socket says 172.21.15.131:41792) 2024-05-26T22:49:05.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.668+0000 7f1c73400640 1 -- 172.21.15.131:0/1036463266 learned_addr learned my addr 172.21.15.131:0/1036463266 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:05.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.668+0000 7f1c72a00640 1 --2- 172.21.15.131:0/1036463266 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1c740af230 0x7f1c74155220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:05.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.668+0000 7f1c73400640 1 -- 172.21.15.131:0/1036463266 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1c740d0560 msgr2=0x7f1c7409b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:05.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.668+0000 7f1c73400640 1 --2- 172.21.15.131:0/1036463266 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1c740d0560 0x7f1c7409b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:05.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.668+0000 7f1c73400640 1 -- 172.21.15.131:0/1036463266 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1c740af230 msgr2=0x7f1c74155220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:05.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.668+0000 7f1c73400640 1 --2- 172.21.15.131:0/1036463266 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1c740af230 0x7f1c74155220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:05.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.668+0000 7f1c73400640 1 -- 172.21.15.131:0/1036463266 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c7409cf70 con 0x7f1c74165e30 2024-05-26T22:49:05.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.669+0000 7f1c73400640 1 --2- 172.21.15.131:0/1036463266 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1c74165e30 0x7f1c740ae160 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c009890 tx=0x7f1c6c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:05.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.669+0000 7f1c6b400640 1 -- 172.21.15.131:0/1036463266 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c6c037040 con 0x7f1c74165e30 2024-05-26T22:49:05.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.669+0000 7f1c6b400640 1 -- 172.21.15.131:0/1036463266 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c6c042070 con 0x7f1c74165e30 2024-05-26T22:49:05.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.669+0000 7f1c79e00640 1 -- 172.21.15.131:0/1036463266 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c740ad9a0 con 0x7f1c74165e30 2024-05-26T22:49:05.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.669+0000 7f1c6b400640 1 -- 172.21.15.131:0/1036463266 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c6c030b30 con 0x7f1c74165e30 2024-05-26T22:49:05.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.669+0000 7f1c79e00640 1 -- 172.21.15.131:0/1036463266 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c7409d4d0 con 0x7f1c74165e30 2024-05-26T22:49:05.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.670+0000 7f1c79e00640 1 -- 172.21.15.131:0/1036463266 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c740695c0 con 0x7f1c74165e30 2024-05-26T22:49:05.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.671+0000 7f1c6b400640 1 -- 172.21.15.131:0/1036463266 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1c6c030cd0 con 0x7f1c74165e30 2024-05-26T22:49:05.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.672+0000 7f1c6b400640 1 --2- 172.21.15.131:0/1036463266 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f1c5006be30 0x7f1c5006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:05.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.672+0000 7f1c6b400640 1 -- 172.21.15.131:0/1036463266 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f1c6c0abab0 con 0x7f1c74165e30 2024-05-26T22:49:05.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.672+0000 7f1c72a00640 1 --2- 172.21.15.131:0/1036463266 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f1c5006be30 0x7f1c5006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:05.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.673+0000 7f1c72a00640 1 --2- 172.21.15.131:0/1036463266 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f1c5006be30 0x7f1c5006e2f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1c64006fd0 tx=0x7f1c64008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:05.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.677+0000 7f1c6b400640 1 -- 172.21.15.131:0/1036463266 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f1c7409cf70 con 0x7f1c74165e30 2024-05-26T22:49:05.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.874+0000 7f1c79e00640 1 -- 172.21.15.131:0/1036463266 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f1c740cd340 con 0x7f1c74165e30 2024-05-26T22:49:05.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.875+0000 7f1c6b400640 1 -- 172.21.15.131:0/1036463266 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7f1c6c032e20 con 0x7f1c74165e30 2024-05-26T22:49:05.875 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 64 2024-05-26T22:49:05.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.878+0000 7f1c79e00640 1 -- 172.21.15.131:0/1036463266 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f1c5006be30 msgr2=0x7f1c5006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:05.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.878+0000 7f1c79e00640 1 --2- 172.21.15.131:0/1036463266 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f1c5006be30 0x7f1c5006e2f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1c64006fd0 tx=0x7f1c64008040 comp rx=0 tx=0).stop 2024-05-26T22:49:05.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.878+0000 7f1c79e00640 1 -- 172.21.15.131:0/1036463266 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1c74165e30 msgr2=0x7f1c740ae160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:05.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.878+0000 7f1c79e00640 1 --2- 172.21.15.131:0/1036463266 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1c74165e30 0x7f1c740ae160 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c009890 tx=0x7f1c6c0046c0 comp rx=0 tx=0).stop 2024-05-26T22:49:05.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.878+0000 7f1c79e00640 1 -- 172.21.15.131:0/1036463266 shutdown_connections 2024-05-26T22:49:05.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.878+0000 7f1c79e00640 1 --2- 172.21.15.131:0/1036463266 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1c740d0560 0x7f1c7409b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:05.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.878+0000 7f1c79e00640 1 --2- 172.21.15.131:0/1036463266 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f1c5006be30 0x7f1c5006e2f0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:05.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.878+0000 7f1c79e00640 1 --2- 172.21.15.131:0/1036463266 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1c74165e30 0x7f1c740ae160 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:05.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.878+0000 7f1c79e00640 1 --2- 172.21.15.131:0/1036463266 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1c740af230 0x7f1c74155220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:05.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.878+0000 7f1c79e00640 1 -- 172.21.15.131:0/1036463266 >> 172.21.15.131:0/1036463266 conn(0x7f1c7406bc90 msgr2=0x7f1c74071e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:05.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.878+0000 7f1c79e00640 1 -- 172.21.15.131:0/1036463266 shutdown_connections 2024-05-26T22:49:05.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:05.878+0000 7f1c79e00640 1 -- 172.21.15.131:0/1036463266 wait complete. 2024-05-26T22:49:05.898 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-26T22:49:06.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.100+0000 7fbd92a00640 1 Processor -- start 2024-05-26T22:49:06.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.101+0000 7fbd92a00640 1 -- start start 2024-05-26T22:49:06.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.102+0000 7fbd92a00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fbd8c0b9f20 0x7fbd8c0d22b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:06.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.102+0000 7fbd92a00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fbd8c0d27f0 0x7fbd8c0b8d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:06.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.102+0000 7fbd92a00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd8c0b79f0 con 0x7fbd8c15e340 2024-05-26T22:49:06.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.102+0000 7fbd92a00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fbd8c0b88b0 con 0x7fbd8c0d27f0 2024-05-26T22:49:06.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.102+0000 7fbd92a00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fbd8c15e720 con 0x7fbd8c0b9f20 2024-05-26T22:49:06.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.102+0000 7fbd92000640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fbd8c0d27f0 0x7fbd8c0b8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:06.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.102+0000 7fbd92000640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fbd8c0d27f0 0x7fbd8c0b8d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41796/0 (socket says 172.21.15.131:41796) 2024-05-26T22:49:06.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.102+0000 7fbd92000640 1 -- 172.21.15.131:0/749606503 learned_addr learned my addr 172.21.15.131:0/749606503 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:06.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.103+0000 7fbd90c00640 1 --2- 172.21.15.131:0/749606503 >> v2:172.21.15.187:3300/0 conn(0x7fbd8c0b9f20 0x7fbd8c0d22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:06.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.103+0000 7fbd92000640 1 -- 172.21.15.131:0/749606503 >> v2:172.21.15.187:3300/0 conn(0x7fbd8c0b9f20 msgr2=0x7fbd8c0d22b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:06.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.103+0000 7fbd92000640 1 --2- 172.21.15.131:0/749606503 >> v2:172.21.15.187:3300/0 conn(0x7fbd8c0b9f20 0x7fbd8c0d22b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:06.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.103+0000 7fbd92000640 1 -- 172.21.15.131:0/749606503 >> v1:172.21.15.187:6789/0 conn(0x7fbd8c15e340 legacy=0x7fbd8c0b72e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:06.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.103+0000 7fbd92000640 1 -- 172.21.15.131:0/749606503 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd8c0b7ca0 con 0x7fbd8c0d27f0 2024-05-26T22:49:06.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.104+0000 7fbd92000640 1 --2- 172.21.15.131:0/749606503 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbd8c0d27f0 0x7fbd8c0b8d40 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c0d1ac0 tx=0x7fbd8402a780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4c1c360e6ffe3a4 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:06.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.104+0000 7fbd8be00640 1 -- 172.21.15.131:0/749606503 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd84036070 con 0x7fbd8c0d27f0 2024-05-26T22:49:06.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.104+0000 7fbd8be00640 1 -- 172.21.15.131:0/749606503 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd84031600 con 0x7fbd8c0d27f0 2024-05-26T22:49:06.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.104+0000 7fbd8be00640 1 -- 172.21.15.131:0/749606503 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd840319d0 con 0x7fbd8c0d27f0 2024-05-26T22:49:06.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.105+0000 7fbd92a00640 1 -- 172.21.15.131:0/749606503 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbd8c0d27f0 msgr2=0x7fbd8c0b8d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:06.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.105+0000 7fbd92a00640 1 --2- 172.21.15.131:0/749606503 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbd8c0d27f0 0x7fbd8c0b8d40 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c0d1ac0 tx=0x7fbd8402a780 comp rx=0 tx=0).stop 2024-05-26T22:49:06.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.105+0000 7fbd92a00640 1 -- 172.21.15.131:0/749606503 shutdown_connections 2024-05-26T22:49:06.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.105+0000 7fbd92a00640 1 --2- 172.21.15.131:0/749606503 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbd8c0d27f0 0x7fbd8c0b8d40 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:06.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.105+0000 7fbd92a00640 1 --2- 172.21.15.131:0/749606503 >> v2:172.21.15.187:3300/0 conn(0x7fbd8c0b9f20 0x7fbd8c0d22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:06.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.105+0000 7fbd92a00640 1 -- 172.21.15.131:0/749606503 >> 172.21.15.131:0/749606503 conn(0x7fbd8c072e60 msgr2=0x7fbd8c073270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:06.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.105+0000 7fbd92a00640 1 -- 172.21.15.131:0/749606503 shutdown_connections 2024-05-26T22:49:06.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.105+0000 7fbd92a00640 1 -- 172.21.15.131:0/749606503 wait complete. 2024-05-26T22:49:06.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.107+0000 7fbd92a00640 1 Processor -- start 2024-05-26T22:49:06.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.107+0000 7fbd92a00640 1 -- start start 2024-05-26T22:49:06.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.108+0000 7fbd92a00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbd8c0b9f20 0x7fbd8c150280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:06.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.108+0000 7fbd92a00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbd8c0d27f0 0x7fbd8c15a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:06.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.108+0000 7fbd92a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbd8c15e340 0x7fbd8c15c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:06.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.108+0000 7fbd92a00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fbd8c0b8540 con 0x7fbd8c15e340 2024-05-26T22:49:06.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.108+0000 7fbd92a00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fbd8c0b8260 con 0x7fbd8c0b9f20 2024-05-26T22:49:06.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.108+0000 7fbd92a00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fbd8c0b9280 con 0x7fbd8c0d27f0 2024-05-26T22:49:06.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.108+0000 7fbd92000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbd8c15e340 0x7fbd8c15c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:06.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.108+0000 7fbd92000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbd8c15e340 0x7fbd8c15c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41800/0 (socket says 172.21.15.131:41800) 2024-05-26T22:49:06.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.108+0000 7fbd92000640 1 -- 172.21.15.131:0/3667035568 learned_addr learned my addr 172.21.15.131:0/3667035568 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:06.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.109+0000 7fbd92000640 1 -- 172.21.15.131:0/3667035568 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbd8c0d27f0 msgr2=0x7fbd8c15a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:06.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.109+0000 7fbd90c00640 1 --2- 172.21.15.131:0/3667035568 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbd8c0d27f0 0x7fbd8c15a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:06.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.109+0000 7fbd91600640 1 --2- 172.21.15.131:0/3667035568 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbd8c0b9f20 0x7fbd8c150280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:06.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.109+0000 7fbd92000640 1 --2- 172.21.15.131:0/3667035568 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbd8c0d27f0 0x7fbd8c15a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:06.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.109+0000 7fbd92000640 1 -- 172.21.15.131:0/3667035568 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbd8c0b9f20 msgr2=0x7fbd8c150280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:06.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.109+0000 7fbd92000640 1 --2- 172.21.15.131:0/3667035568 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbd8c0b9f20 0x7fbd8c150280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:06.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.109+0000 7fbd92000640 1 -- 172.21.15.131:0/3667035568 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd8c15ae90 con 0x7fbd8c15e340 2024-05-26T22:49:06.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.109+0000 7fbd92000640 1 --2- 172.21.15.131:0/3667035568 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbd8c15e340 0x7fbd8c15c8a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fbd84009f40 tx=0x7fbd84004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:06.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.109+0000 7fbd89600640 1 -- 172.21.15.131:0/3667035568 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd84036050 con 0x7fbd8c15e340 2024-05-26T22:49:06.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.109+0000 7fbd89600640 1 -- 172.21.15.131:0/3667035568 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd84042070 con 0x7fbd8c15e340 2024-05-26T22:49:06.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.109+0000 7fbd89600640 1 -- 172.21.15.131:0/3667035568 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd84009510 con 0x7fbd8c15e340 2024-05-26T22:49:06.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.109+0000 7fbd92a00640 1 -- 172.21.15.131:0/3667035568 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd8c15b170 con 0x7fbd8c15e340 2024-05-26T22:49:06.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.110+0000 7fbd92a00640 1 -- 172.21.15.131:0/3667035568 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd8c15b730 con 0x7fbd8c15e340 2024-05-26T22:49:06.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.111+0000 7fbd92a00640 1 -- 172.21.15.131:0/3667035568 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd8c0694e0 con 0x7fbd8c15e340 2024-05-26T22:49:06.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.112+0000 7fbd89600640 1 -- 172.21.15.131:0/3667035568 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbd84007bd0 con 0x7fbd8c15e340 2024-05-26T22:49:06.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.113+0000 7fbd89600640 1 --2- 172.21.15.131:0/3667035568 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fbd6c06be60 0x7fbd6c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:06.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.113+0000 7fbd89600640 1 -- 172.21.15.131:0/3667035568 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fbd840abe40 con 0x7fbd8c15e340 2024-05-26T22:49:06.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.113+0000 7fbd91600640 1 --2- 172.21.15.131:0/3667035568 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fbd6c06be60 0x7fbd6c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:06.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.114+0000 7fbd91600640 1 --2- 172.21.15.131:0/3667035568 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fbd6c06be60 0x7fbd6c06e320 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c00a750 tx=0x7fbd7c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:06.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.118+0000 7fbd89600640 1 -- 172.21.15.131:0/3667035568 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fbd84045360 con 0x7fbd8c15e340 2024-05-26T22:49:06.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.330+0000 7fbd92a00640 1 -- 172.21.15.131:0/3667035568 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fbd8c160770 con 0x7fbd8c15e340 2024-05-26T22:49:06.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.331+0000 7fbd89600640 1 -- 172.21.15.131:0/3667035568 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7fbd8407a770 con 0x7fbd8c15e340 2024-05-26T22:49:06.331 INFO:teuthology.orchestra.run.smithi131.stderr:listed 0 entries 2024-05-26T22:49:06.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.334+0000 7fbd92a00640 1 -- 172.21.15.131:0/3667035568 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fbd6c06be60 msgr2=0x7fbd6c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:06.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.334+0000 7fbd92a00640 1 --2- 172.21.15.131:0/3667035568 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fbd6c06be60 0x7fbd6c06e320 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c00a750 tx=0x7fbd7c008040 comp rx=0 tx=0).stop 2024-05-26T22:49:06.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.334+0000 7fbd92a00640 1 -- 172.21.15.131:0/3667035568 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbd8c15e340 msgr2=0x7fbd8c15c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:06.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.334+0000 7fbd92a00640 1 --2- 172.21.15.131:0/3667035568 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbd8c15e340 0x7fbd8c15c8a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fbd84009f40 tx=0x7fbd84004a40 comp rx=0 tx=0).stop 2024-05-26T22:49:06.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.334+0000 7fbd92a00640 1 -- 172.21.15.131:0/3667035568 shutdown_connections 2024-05-26T22:49:06.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.334+0000 7fbd92a00640 1 --2- 172.21.15.131:0/3667035568 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbd8c0d27f0 0x7fbd8c15a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:06.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.334+0000 7fbd92a00640 1 --2- 172.21.15.131:0/3667035568 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fbd6c06be60 0x7fbd6c06e320 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:06.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.334+0000 7fbd92a00640 1 --2- 172.21.15.131:0/3667035568 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbd8c15e340 0x7fbd8c15c8a0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:06.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.334+0000 7fbd92a00640 1 --2- 172.21.15.131:0/3667035568 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbd8c0b9f20 0x7fbd8c150280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:06.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.335+0000 7fbd92a00640 1 -- 172.21.15.131:0/3667035568 >> 172.21.15.131:0/3667035568 conn(0x7fbd8c072e60 msgr2=0x7fbd8c0d4aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:06.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.335+0000 7fbd92a00640 1 -- 172.21.15.131:0/3667035568 shutdown_connections 2024-05-26T22:49:06.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.335+0000 7fbd92a00640 1 -- 172.21.15.131:0/3667035568 wait complete. 2024-05-26T22:49:06.355 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-26T22:49:06.355 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:49:06.355 DEBUG:teuthology.orchestra.run.smithi131:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-05-26T22:49:06.437 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi131.front.sepia.ceph.com 2024-05-26T22:49:06.437 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:49:06.438 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-26T22:49:06.490 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-05-26T22:49:06.490 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-26T22:49:06.548 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-05-26T22:49:06.548 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-26T22:49:06.602 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-05-26T22:49:06.602 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-26T22:49:06.654 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-26T22:49:06.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.861+0000 7fbf6a200640 1 Processor -- start 2024-05-26T22:49:06.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.861+0000 7fbf6a200640 1 -- start start 2024-05-26T22:49:06.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.863+0000 7fbf6a200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fbf6406e8c0 0x7fbf6415d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:06.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.863+0000 7fbf6a200640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf64160620 con 0x7fbf640cc7f0 2024-05-26T22:49:06.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.863+0000 7fbf6a200640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf64160900 con 0x7fbf64156750 2024-05-26T22:49:06.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.863+0000 7fbf6a200640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fbf6415cbd0 con 0x7fbf6406e8c0 2024-05-26T22:49:06.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.863+0000 7fbf63e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fbf6406e8c0 0x7fbf6415d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:06.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.863+0000 7fbf63e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fbf6406e8c0 0x7fbf6415d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41810/0 (socket says 172.21.15.131:41810) 2024-05-26T22:49:06.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.863+0000 7fbf63e00640 1 -- 172.21.15.131:0/3027621361 learned_addr learned my addr 172.21.15.131:0/3027621361 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:06.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.863+0000 7fbf63e00640 1 -- 172.21.15.131:0/3027621361 >> v1:172.21.15.187:6789/0 conn(0x7fbf64156750 legacy=0x7fbf6415fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:06.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.863+0000 7fbf63e00640 1 -- 172.21.15.131:0/3027621361 >> v1:172.21.15.131:6789/0 conn(0x7fbf640cc7f0 legacy=0x7fbf640cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:06.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.863+0000 7fbf63e00640 1 -- 172.21.15.131:0/3027621361 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf64160be0 con 0x7fbf6406e8c0 2024-05-26T22:49:06.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.865+0000 7fbf63e00640 1 --2- 172.21.15.131:0/3027621361 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbf6406e8c0 0x7fbf6415d060 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fbf6415cd50 tx=0x7fbf5002a290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b305850b92b01ca5 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:06.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.865+0000 7fbf63400640 1 -- 172.21.15.131:0/3027621361 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf50037070 con 0x7fbf6406e8c0 2024-05-26T22:49:06.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.865+0000 7fbf63400640 1 -- 172.21.15.131:0/3027621361 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf50032440 con 0x7fbf6406e8c0 2024-05-26T22:49:06.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.865+0000 7fbf63400640 1 -- 172.21.15.131:0/3027621361 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf50032810 con 0x7fbf6406e8c0 2024-05-26T22:49:06.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.866+0000 7fbf6a200640 1 -- 172.21.15.131:0/3027621361 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbf6406e8c0 msgr2=0x7fbf6415d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:06.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.866+0000 7fbf6a200640 1 --2- 172.21.15.131:0/3027621361 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbf6406e8c0 0x7fbf6415d060 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fbf6415cd50 tx=0x7fbf5002a290 comp rx=0 tx=0).stop 2024-05-26T22:49:06.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.866+0000 7fbf6a200640 1 -- 172.21.15.131:0/3027621361 shutdown_connections 2024-05-26T22:49:06.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.866+0000 7fbf6a200640 1 --2- 172.21.15.131:0/3027621361 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbf6406e8c0 0x7fbf6415d060 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:06.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.866+0000 7fbf6a200640 1 -- 172.21.15.131:0/3027621361 >> 172.21.15.131:0/3027621361 conn(0x7fbf640cff40 msgr2=0x7fbf640d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:06.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.866+0000 7fbf6a200640 1 -- 172.21.15.131:0/3027621361 shutdown_connections 2024-05-26T22:49:06.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.866+0000 7fbf6a200640 1 -- 172.21.15.131:0/3027621361 wait complete. 2024-05-26T22:49:06.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.868+0000 7fbf6a200640 1 Processor -- start 2024-05-26T22:49:06.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.868+0000 7fbf6a200640 1 -- start start 2024-05-26T22:49:06.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.869+0000 7fbf6a200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbf6406e8c0 0x7fbf64157690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:06.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.869+0000 7fbf6a200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbf640cc7f0 0x7fbf64159540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:06.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.869+0000 7fbf6a200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbf64156750 0x7fbf641f3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:06.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.869+0000 7fbf6a200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fbf6415d5a0 con 0x7fbf6406e8c0 2024-05-26T22:49:06.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.869+0000 7fbf6a200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fbf64156b30 con 0x7fbf640cc7f0 2024-05-26T22:49:06.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.869+0000 7fbf6a200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fbf6415d720 con 0x7fbf64156750 2024-05-26T22:49:06.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.869+0000 7fbf68e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbf6406e8c0 0x7fbf64157690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:06.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.869+0000 7fbf68e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbf6406e8c0 0x7fbf64157690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41822/0 (socket says 172.21.15.131:41822) 2024-05-26T22:49:06.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.869+0000 7fbf68e00640 1 -- 172.21.15.131:0/4070997943 learned_addr learned my addr 172.21.15.131:0/4070997943 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:06.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.870+0000 7fbf63e00640 1 --2- 172.21.15.131:0/4070997943 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbf640cc7f0 0x7fbf64159540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:06.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.870+0000 7fbf68e00640 1 -- 172.21.15.131:0/4070997943 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbf64156750 msgr2=0x7fbf641f3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:06.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.870+0000 7fbf69800640 1 --2- 172.21.15.131:0/4070997943 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbf64156750 0x7fbf641f3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:06.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.870+0000 7fbf68e00640 1 --2- 172.21.15.131:0/4070997943 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbf64156750 0x7fbf641f3bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:06.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.870+0000 7fbf68e00640 1 -- 172.21.15.131:0/4070997943 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbf640cc7f0 msgr2=0x7fbf64159540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:06.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.870+0000 7fbf68e00640 1 --2- 172.21.15.131:0/4070997943 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbf640cc7f0 0x7fbf64159540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:06.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.870+0000 7fbf68e00640 1 -- 172.21.15.131:0/4070997943 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf64159a80 con 0x7fbf6406e8c0 2024-05-26T22:49:06.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.870+0000 7fbf63e00640 1 --2- 172.21.15.131:0/4070997943 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbf640cc7f0 0x7fbf64159540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-26T22:49:06.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.870+0000 7fbf68e00640 1 --2- 172.21.15.131:0/4070997943 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbf6406e8c0 0x7fbf64157690 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fbf5800b3e0 tx=0x7fbf5800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:06.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.870+0000 7fbf60c00640 1 -- 172.21.15.131:0/4070997943 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf58015020 con 0x7fbf6406e8c0 2024-05-26T22:49:06.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.871+0000 7fbf60c00640 1 -- 172.21.15.131:0/4070997943 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf58011850 con 0x7fbf6406e8c0 2024-05-26T22:49:06.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.871+0000 7fbf6a200640 1 -- 172.21.15.131:0/4070997943 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf64157ca0 con 0x7fbf6406e8c0 2024-05-26T22:49:06.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.871+0000 7fbf60c00640 1 -- 172.21.15.131:0/4070997943 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf58011c40 con 0x7fbf6406e8c0 2024-05-26T22:49:06.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.871+0000 7fbf6a200640 1 -- 172.21.15.131:0/4070997943 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf64158260 con 0x7fbf6406e8c0 2024-05-26T22:49:06.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.872+0000 7fbf6a200640 1 -- 172.21.15.131:0/4070997943 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf640694e0 con 0x7fbf6406e8c0 2024-05-26T22:49:06.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.873+0000 7fbf60c00640 1 -- 172.21.15.131:0/4070997943 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbf58018020 con 0x7fbf6406e8c0 2024-05-26T22:49:06.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.874+0000 7fbf60c00640 1 --2- 172.21.15.131:0/4070997943 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fbf4406be30 0x7fbf4406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:06.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.874+0000 7fbf60c00640 1 -- 172.21.15.131:0/4070997943 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fbf5808aec0 con 0x7fbf6406e8c0 2024-05-26T22:49:06.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.874+0000 7fbf63e00640 1 --2- 172.21.15.131:0/4070997943 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fbf4406be30 0x7fbf4406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:06.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.875+0000 7fbf63e00640 1 --2- 172.21.15.131:0/4070997943 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fbf4406be30 0x7fbf4406e2f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fbf50034000 tx=0x7fbf5002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:06.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:06.878+0000 7fbf60c00640 1 -- 172.21.15.131:0/4070997943 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fbf5801f020 con 0x7fbf6406e8c0 2024-05-26T22:49:07.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.075+0000 7fbf6a200640 1 -- 172.21.15.131:0/4070997943 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbf640adc80 con 0x7fbf6406e8c0 2024-05-26T22:49:07.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.077+0000 7fbf60c00640 1 -- 172.21.15.131:0/4070997943 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16879 (secure 0 0 0) 0x7fbf580172c0 con 0x7fbf6406e8c0 2024-05-26T22:49:07.077 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:49:07.077 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":22,"fsid":"46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f","created":"2024-05-26T22:47:57.288905+0000","modified":"2024-05-26T22:49:04.278988+0000","last_up_change":"2024-05-26T22:48:09.602207+0000","last_in_change":"2024-05-26T22:48:00.687341+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-26T22:48:10.463339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-26T22:48:19.663254+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-26T22:48:20.235751+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"814eaf62-a18d-494b-8e27-fd8fb88adea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6801","nonce":2350940051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6803","nonce":2350940051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6807","nonce":2350940051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6805","nonce":2350940051}]},"public_addr":"172.21.15.131:6801/2350940051","cluster_addr":"172.21.15.131:6803/2350940051","heartbeat_back_addr":"172.21.15.131:6807/2350940051","heartbeat_front_addr":"172.21.15.131:6805/2350940051","state":["exists","up"]},{"osd":1,"uuid":"42e66287-f6c6-4e26-af3c-6dfaa3536d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6825","nonce":1525528247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6827","nonce":1525528247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6831","nonce":1525528247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6829","nonce":1525528247}]},"public_addr":"172.21.15.131:6825/1525528247","cluster_addr":"172.21.15.131:6827/1525528247","heartbeat_back_addr":"172.21.15.131:6831/1525528247","heartbeat_front_addr":"172.21.15.131:6829/1525528247","state":["exists","up"]},{"osd":2,"uuid":"0da34673-04a2-4fd0-b57a-0ff2b679b043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6817","nonce":3627842210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6819","nonce":3627842210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6823","nonce":3627842210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6821","nonce":3627842210}]},"public_addr":"172.21.15.131:6817/3627842210","cluster_addr":"172.21.15.131:6819/3627842210","heartbeat_back_addr":"172.21.15.131:6823/3627842210","heartbeat_front_addr":"172.21.15.131:6821/3627842210","state":["exists","up"]},{"osd":3,"uuid":"cf117a27-d6b2-43d3-ae52-b47988bf8eed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6809","nonce":350790591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6811","nonce":350790591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6815","nonce":350790591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6813","nonce":350790591}]},"public_addr":"172.21.15.131:6809/350790591","cluster_addr":"172.21.15.131:6811/350790591","heartbeat_back_addr":"172.21.15.131:6815/350790591","heartbeat_front_addr":"172.21.15.131:6813/350790591","state":["exists","up"]},{"osd":4,"uuid":"a092ccfc-be89-4a84-88d5-0f781fa715eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6816","nonce":244560995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6819","nonce":244560995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6823","nonce":244560995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6821","nonce":244560995}]},"public_addr":"172.21.15.187:6816/244560995","cluster_addr":"172.21.15.187:6819/244560995","heartbeat_back_addr":"172.21.15.187:6823/244560995","heartbeat_front_addr":"172.21.15.187:6821/244560995","state":["exists","up"]},{"osd":5,"uuid":"7513ba69-dd00-4a35-b190-9e8cc3b2bdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6801","nonce":324619398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6803","nonce":324619398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6812","nonce":324619398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6807","nonce":324619398}]},"public_addr":"172.21.15.187:6801/324619398","cluster_addr":"172.21.15.187:6803/324619398","heartbeat_back_addr":"172.21.15.187:6812/324619398","heartbeat_front_addr":"172.21.15.187:6807/324619398","state":["exists","up"]},{"osd":6,"uuid":"50081810-c093-4d2f-a4d7-b7c695c5400d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6806","nonce":1391347786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6809","nonce":1391347786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6818","nonce":1391347786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6813","nonce":1391347786}]},"public_addr":"172.21.15.187:6806/1391347786","cluster_addr":"172.21.15.187:6809/1391347786","heartbeat_back_addr":"172.21.15.187:6818/1391347786","heartbeat_front_addr":"172.21.15.187:6813/1391347786","state":["exists","up"]},{"osd":7,"uuid":"775dae05-54f5-4ad4-bbce-92863c6ed2ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6825","nonce":1220065622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6827","nonce":1220065622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6831","nonce":1220065622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6829","nonce":1220065622}]},"public_addr":"172.21.15.187:6825/1220065622","cluster_addr":"172.21.15.187:6827/1220065622","heartbeat_back_addr":"172.21.15.187:6831/1220065622","heartbeat_front_addr":"172.21.15.187:6829/1220065622","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:05.983520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.299516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.067323+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.307748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.018742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.258775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.010912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.295355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.1e","mappings":[{"from":3,"to":2}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":7,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-26T22:49:07.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.079+0000 7fbf6a200640 1 -- 172.21.15.131:0/4070997943 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fbf4406be30 msgr2=0x7fbf4406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:07.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.079+0000 7fbf6a200640 1 --2- 172.21.15.131:0/4070997943 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fbf4406be30 0x7fbf4406e2f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fbf50034000 tx=0x7fbf5002e000 comp rx=0 tx=0).stop 2024-05-26T22:49:07.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.079+0000 7fbf6a200640 1 -- 172.21.15.131:0/4070997943 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbf6406e8c0 msgr2=0x7fbf64157690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:07.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.079+0000 7fbf6a200640 1 --2- 172.21.15.131:0/4070997943 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbf6406e8c0 0x7fbf64157690 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fbf5800b3e0 tx=0x7fbf5800b8a0 comp rx=0 tx=0).stop 2024-05-26T22:49:07.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.079+0000 7fbf6a200640 1 -- 172.21.15.131:0/4070997943 shutdown_connections 2024-05-26T22:49:07.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.079+0000 7fbf6a200640 1 --2- 172.21.15.131:0/4070997943 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbf64156750 0x7fbf641f3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:07.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.079+0000 7fbf6a200640 1 --2- 172.21.15.131:0/4070997943 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fbf4406be30 0x7fbf4406e2f0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:07.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.079+0000 7fbf6a200640 1 --2- 172.21.15.131:0/4070997943 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbf640cc7f0 0x7fbf64159540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:07.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.079+0000 7fbf6a200640 1 --2- 172.21.15.131:0/4070997943 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbf6406e8c0 0x7fbf64157690 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:07.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.079+0000 7fbf6a200640 1 -- 172.21.15.131:0/4070997943 >> 172.21.15.131:0/4070997943 conn(0x7fbf640cff40 msgr2=0x7fbf6415a430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:07.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.080+0000 7fbf6a200640 1 -- 172.21.15.131:0/4070997943 shutdown_connections 2024-05-26T22:49:07.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.080+0000 7fbf6a200640 1 -- 172.21.15.131:0/4070997943 wait complete. 2024-05-26T22:49:07.097 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-26T22:48:10.463339+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-26T22:48:19.663254+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-26T22:48:20.235751+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-26T22:49:07.098 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-26T22:49:07.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.283+0000 7f8309000640 1 Processor -- start 2024-05-26T22:49:07.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.283+0000 7f8309000640 1 -- start start 2024-05-26T22:49:07.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.285+0000 7f8309000640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f83040cbef0 0x7f83040cc2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:07.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.285+0000 7f8309000640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f83040acef0 0x7f830406d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:07.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.285+0000 7f8309000640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83040cdca0 con 0x7f83040ad3a0 2024-05-26T22:49:07.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.285+0000 7f8309000640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f83040cdf80 con 0x7f83040cbef0 2024-05-26T22:49:07.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.285+0000 7f8309000640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f83040ce260 con 0x7f83040acef0 2024-05-26T22:49:07.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.285+0000 7f8301e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f83040cbef0 0x7f83040cc2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:07.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.285+0000 7f8301e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f83040cbef0 0x7f83040cc2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41836/0 (socket says 172.21.15.131:41836) 2024-05-26T22:49:07.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.285+0000 7f8301e00640 1 -- 172.21.15.131:0/3818763884 learned_addr learned my addr 172.21.15.131:0/3818763884 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:07.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.286+0000 7f82f9400640 1 --2- 172.21.15.131:0/3818763884 >> v2:172.21.15.187:3300/0 conn(0x7f83040acef0 0x7f830406d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:07.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.286+0000 7f8301e00640 1 -- 172.21.15.131:0/3818763884 >> v2:172.21.15.187:3300/0 conn(0x7f83040acef0 msgr2=0x7f830406d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:07.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.286+0000 7f8301e00640 1 --2- 172.21.15.131:0/3818763884 >> v2:172.21.15.187:3300/0 conn(0x7f83040acef0 0x7f830406d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:07.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.286+0000 7f8301e00640 1 -- 172.21.15.131:0/3818763884 >> v1:172.21.15.131:6789/0 conn(0x7f83040ad3a0 legacy=0x7f83040cea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:07.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.286+0000 7f8301e00640 1 -- 172.21.15.131:0/3818763884 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83040ce540 con 0x7f83040cbef0 2024-05-26T22:49:07.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.286+0000 7f82f9400640 1 --2- 172.21.15.131:0/3818763884 >> v2:172.21.15.187:3300/0 conn(0x7f83040acef0 0x7f830406d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:49:07.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.286+0000 7f8301400640 1 -- 172.21.15.131:0/3818763884 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1131755396 0 0) 0x7f83040cdca0 con 0x7f83040ad3a0 2024-05-26T22:49:07.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.287+0000 7f8301e00640 1 --2- 172.21.15.131:0/3818763884 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f83040cbef0 0x7f83040cc2d0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f83040737f0 tx=0x7f82f402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a44ee3f15d74ec64 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:07.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.287+0000 7f8301400640 1 -- 172.21.15.131:0/3818763884 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82f4037070 con 0x7f83040cbef0 2024-05-26T22:49:07.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.287+0000 7f8301400640 1 -- 172.21.15.131:0/3818763884 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82f4032440 con 0x7f83040cbef0 2024-05-26T22:49:07.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.287+0000 7f8301400640 1 -- 172.21.15.131:0/3818763884 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82f4032810 con 0x7f83040cbef0 2024-05-26T22:49:07.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.288+0000 7f8309000640 1 -- 172.21.15.131:0/3818763884 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f83040cbef0 msgr2=0x7f83040cc2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:07.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.288+0000 7f8309000640 1 --2- 172.21.15.131:0/3818763884 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f83040cbef0 0x7f83040cc2d0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f83040737f0 tx=0x7f82f402a270 comp rx=0 tx=0).stop 2024-05-26T22:49:07.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.288+0000 7f8309000640 1 -- 172.21.15.131:0/3818763884 shutdown_connections 2024-05-26T22:49:07.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.288+0000 7f8309000640 1 --2- 172.21.15.131:0/3818763884 >> v2:172.21.15.187:3300/0 conn(0x7f83040acef0 0x7f830406d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:07.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.288+0000 7f8309000640 1 --2- 172.21.15.131:0/3818763884 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f83040cbef0 0x7f83040cc2d0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:07.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.288+0000 7f8309000640 1 -- 172.21.15.131:0/3818763884 >> 172.21.15.131:0/3818763884 conn(0x7f830406bc90 msgr2=0x7f83040b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:07.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.288+0000 7f8309000640 1 -- 172.21.15.131:0/3818763884 shutdown_connections 2024-05-26T22:49:07.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.288+0000 7f8309000640 1 -- 172.21.15.131:0/3818763884 wait complete. 2024-05-26T22:49:07.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.289+0000 7f8309000640 1 Processor -- start 2024-05-26T22:49:07.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.290+0000 7f8309000640 1 -- start start 2024-05-26T22:49:07.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.290+0000 7f8309000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f83040acef0 0x7f83040afb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:07.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.290+0000 7f8309000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f83040ad3a0 0x7f830409cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:07.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.290+0000 7f8309000640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f83040cbef0 0x7f83041459a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:07.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.290+0000 7f8309000640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f83040cf500 con 0x7f83040ad3a0 2024-05-26T22:49:07.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.290+0000 7f8309000640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f83040cf380 con 0x7f83040cbef0 2024-05-26T22:49:07.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.290+0000 7f8309000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f830406d860 con 0x7f83040acef0 2024-05-26T22:49:07.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.291+0000 7f82f9400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f83040ad3a0 0x7f830409cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:07.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.291+0000 7f82f9400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f83040ad3a0 0x7f830409cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41846/0 (socket says 172.21.15.131:41846) 2024-05-26T22:49:07.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.291+0000 7f82f9400640 1 -- 172.21.15.131:0/3628733102 learned_addr learned my addr 172.21.15.131:0/3628733102 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:07.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.291+0000 7f8301e00640 1 --2- 172.21.15.131:0/3628733102 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f83040acef0 0x7f83040afb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:07.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.291+0000 7f82f9400640 1 -- 172.21.15.131:0/3628733102 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f83040acef0 msgr2=0x7f83040afb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:07.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.291+0000 7f82f9400640 1 --2- 172.21.15.131:0/3628733102 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f83040acef0 0x7f83040afb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:07.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.291+0000 7f82f9400640 1 -- 172.21.15.131:0/3628733102 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f83040cbef0 msgr2=0x7f83041459a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:07.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.291+0000 7f82f9400640 1 --2- 172.21.15.131:0/3628733102 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f83040cbef0 0x7f83041459a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:07.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.291+0000 7f82f9400640 1 -- 172.21.15.131:0/3628733102 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f830409c740 con 0x7f83040ad3a0 2024-05-26T22:49:07.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.291+0000 7f82f9400640 1 --2- 172.21.15.131:0/3628733102 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f83040ad3a0 0x7f830409cbe0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f82ec00c820 tx=0x7f82ec00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:07.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.292+0000 7f82faa00640 1 -- 172.21.15.131:0/3628733102 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82ec015070 con 0x7f83040ad3a0 2024-05-26T22:49:07.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.292+0000 7f82faa00640 1 -- 172.21.15.131:0/3628733102 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82ec007a30 con 0x7f83040ad3a0 2024-05-26T22:49:07.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.292+0000 7f82faa00640 1 -- 172.21.15.131:0/3628733102 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82ec007de0 con 0x7f83040ad3a0 2024-05-26T22:49:07.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.292+0000 7f8309000640 1 -- 172.21.15.131:0/3628733102 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83040b00f0 con 0x7f83040ad3a0 2024-05-26T22:49:07.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.292+0000 7f8309000640 1 -- 172.21.15.131:0/3628733102 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83040b06b0 con 0x7f83040ad3a0 2024-05-26T22:49:07.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.293+0000 7f8309000640 1 -- 172.21.15.131:0/3628733102 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8304074e90 con 0x7f83040ad3a0 2024-05-26T22:49:07.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.294+0000 7f82faa00640 1 -- 172.21.15.131:0/3628733102 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f82ec018070 con 0x7f83040ad3a0 2024-05-26T22:49:07.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.295+0000 7f82faa00640 1 --2- 172.21.15.131:0/3628733102 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f82e006be30 0x7f82e006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:07.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.295+0000 7f82faa00640 1 -- 172.21.15.131:0/3628733102 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f82ec08ad40 con 0x7f83040ad3a0 2024-05-26T22:49:07.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.295+0000 7f8301e00640 1 --2- 172.21.15.131:0/3628733102 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f82e006be30 0x7f82e006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:07.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.295+0000 7f8301e00640 1 --2- 172.21.15.131:0/3628733102 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f82e006be30 0x7f82e006e2f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f82f4034000 tx=0x7f82f402e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:07.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.299+0000 7f82faa00640 1 -- 172.21.15.131:0/3628733102 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f82ec059670 con 0x7f83040ad3a0 2024-05-26T22:49:07.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.491+0000 7f8309000640 1 -- 172.21.15.131:0/3628733102 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f830406dbf0 con 0x7f83040ad3a0 2024-05-26T22:49:07.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.492+0000 7f82faa00640 1 -- 172.21.15.131:0/3628733102 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7f830406dbf0 con 0x7f83040ad3a0 2024-05-26T22:49:07.492 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2024-05-26T22:49:07.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.494+0000 7f8309000640 1 -- 172.21.15.131:0/3628733102 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f82e006be30 msgr2=0x7f82e006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:07.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.494+0000 7f8309000640 1 --2- 172.21.15.131:0/3628733102 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f82e006be30 0x7f82e006e2f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f82f4034000 tx=0x7f82f402e000 comp rx=0 tx=0).stop 2024-05-26T22:49:07.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.495+0000 7f8309000640 1 -- 172.21.15.131:0/3628733102 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f83040ad3a0 msgr2=0x7f830409cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:07.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.495+0000 7f8309000640 1 --2- 172.21.15.131:0/3628733102 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f83040ad3a0 0x7f830409cbe0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f82ec00c820 tx=0x7f82ec00cce0 comp rx=0 tx=0).stop 2024-05-26T22:49:07.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.495+0000 7f8309000640 1 -- 172.21.15.131:0/3628733102 shutdown_connections 2024-05-26T22:49:07.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.495+0000 7f8309000640 1 --2- 172.21.15.131:0/3628733102 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f82e006be30 0x7f82e006e2f0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:07.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.495+0000 7f8309000640 1 --2- 172.21.15.131:0/3628733102 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f83040cbef0 0x7f83041459a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:07.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.495+0000 7f8309000640 1 --2- 172.21.15.131:0/3628733102 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f83040ad3a0 0x7f830409cbe0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:07.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.495+0000 7f8309000640 1 --2- 172.21.15.131:0/3628733102 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f83040acef0 0x7f83040afb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:07.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.495+0000 7f8309000640 1 -- 172.21.15.131:0/3628733102 >> 172.21.15.131:0/3628733102 conn(0x7f830406bc90 msgr2=0x7f830415c110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:07.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.495+0000 7f8309000640 1 -- 172.21.15.131:0/3628733102 shutdown_connections 2024-05-26T22:49:07.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.495+0000 7f8309000640 1 -- 172.21.15.131:0/3628733102 wait complete. 2024-05-26T22:49:07.512 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-26T22:49:07.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.703+0000 7fe96fe00640 1 Processor -- start 2024-05-26T22:49:07.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.703+0000 7fe96fe00640 1 -- start start 2024-05-26T22:49:07.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.705+0000 7fe96fe00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fe970145a10 0x7fe970072180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:07.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.705+0000 7fe96fe00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe970072d60 con 0x7fe9700712d0 2024-05-26T22:49:07.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.705+0000 7fe96fe00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe970072a80 con 0x7fe97006d960 2024-05-26T22:49:07.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.705+0000 7fe96fe00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fe97015c760 con 0x7fe970145a10 2024-05-26T22:49:07.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.705+0000 7fe96ea00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fe970145a10 0x7fe970072180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:07.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.705+0000 7fe96e000640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fe9700712d0 0x7fe97006d250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:46646/0 (socket says 172.21.15.131:46646) 2024-05-26T22:49:07.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.705+0000 7fe96ea00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fe970145a10 0x7fe970072180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41860/0 (socket says 172.21.15.131:41860) 2024-05-26T22:49:07.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.705+0000 7fe96e000640 1 -- 172.21.15.131:0/51667834 learned_addr learned my addr 172.21.15.131:0/51667834 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:49:07.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.706+0000 7fe96ea00640 1 -- 172.21.15.131:0/51667834 >> v1:172.21.15.187:6789/0 conn(0x7fe97006d960 legacy=0x7fe9700cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:07.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.706+0000 7fe96ea00640 1 -- 172.21.15.131:0/51667834 >> v1:172.21.15.131:6789/0 conn(0x7fe9700712d0 legacy=0x7fe97006d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:07.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.706+0000 7fe96ea00640 1 -- 172.21.15.131:0/51667834 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9700ceab0 con 0x7fe970145a10 2024-05-26T22:49:07.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.707+0000 7fe96ea00640 1 --2- 172.21.15.131:0/51667834 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe970145a10 0x7fe970072180 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fe9600097a0 tx=0x7fe96002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfa02decfbd0f2f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:07.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.707+0000 7fe96d600640 1 -- 172.21.15.131:0/51667834 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe960037070 con 0x7fe970145a10 2024-05-26T22:49:07.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.707+0000 7fe96d600640 1 -- 172.21.15.131:0/51667834 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe960032440 con 0x7fe970145a10 2024-05-26T22:49:07.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.707+0000 7fe96d600640 1 -- 172.21.15.131:0/51667834 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe960032810 con 0x7fe970145a10 2024-05-26T22:49:07.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.707+0000 7fe96fe00640 1 -- 172.21.15.131:0/51667834 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe970145a10 msgr2=0x7fe970072180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:07.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.707+0000 7fe96fe00640 1 --2- 172.21.15.131:0/51667834 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe970145a10 0x7fe970072180 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fe9600097a0 tx=0x7fe96002a1e0 comp rx=0 tx=0).stop 2024-05-26T22:49:07.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.708+0000 7fe96fe00640 1 -- 172.21.15.131:0/51667834 shutdown_connections 2024-05-26T22:49:07.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.708+0000 7fe96fe00640 1 --2- 172.21.15.131:0/51667834 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe970145a10 0x7fe970072180 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:07.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.708+0000 7fe96fe00640 1 -- 172.21.15.131:0/51667834 >> 172.21.15.131:0/51667834 conn(0x7fe9700b5660 msgr2=0x7fe9700b5a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:07.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.708+0000 7fe96fe00640 1 -- 172.21.15.131:0/51667834 shutdown_connections 2024-05-26T22:49:07.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.708+0000 7fe96fe00640 1 -- 172.21.15.131:0/51667834 wait complete. 2024-05-26T22:49:07.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.710+0000 7fe96fe00640 1 Processor -- start 2024-05-26T22:49:07.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.710+0000 7fe96fe00640 1 -- start start 2024-05-26T22:49:07.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.712+0000 7fe96fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe97006d960 0x7fe9701f19e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:07.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.712+0000 7fe96fe00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe9700712d0 0x7fe970140e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:07.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.712+0000 7fe96fe00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe970145a10 0x7fe970158a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:07.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.712+0000 7fe96fe00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe970145df0 con 0x7fe97006d960 2024-05-26T22:49:07.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.712+0000 7fe96fe00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe9700cf070 con 0x7fe970145a10 2024-05-26T22:49:07.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.712+0000 7fe96fe00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe9700726c0 con 0x7fe9700712d0 2024-05-26T22:49:07.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.712+0000 7fe96ea00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe97006d960 0x7fe9701f19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:07.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.712+0000 7fe96ea00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe97006d960 0x7fe9701f19e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41862/0 (socket says 172.21.15.131:41862) 2024-05-26T22:49:07.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.712+0000 7fe96ea00640 1 -- 172.21.15.131:0/3818687598 learned_addr learned my addr 172.21.15.131:0/3818687598 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:07.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.712+0000 7fe96ea00640 1 -- 172.21.15.131:0/3818687598 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe9700712d0 msgr2=0x7fe970140e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:07.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.713+0000 7fe96ea00640 1 --2- 172.21.15.131:0/3818687598 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe9700712d0 0x7fe970140e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:07.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.713+0000 7fe96ea00640 1 -- 172.21.15.131:0/3818687598 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe970145a10 msgr2=0x7fe970158a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:07.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.713+0000 7fe96ea00640 1 --2- 172.21.15.131:0/3818687598 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe970145a10 0x7fe970158a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:07.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.713+0000 7fe96ea00640 1 -- 172.21.15.131:0/3818687598 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe97013fdb0 con 0x7fe97006d960 2024-05-26T22:49:07.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.713+0000 7fe96ea00640 1 --2- 172.21.15.131:0/3818687598 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe97006d960 0x7fe9701f19e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fe9600098c0 tx=0x7fe9600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:07.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.713+0000 7fe966a00640 1 -- 172.21.15.131:0/3818687598 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe960041070 con 0x7fe97006d960 2024-05-26T22:49:07.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.713+0000 7fe966a00640 1 -- 172.21.15.131:0/3818687598 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe96003c870 con 0x7fe97006d960 2024-05-26T22:49:07.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.713+0000 7fe966a00640 1 -- 172.21.15.131:0/3818687598 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe960037040 con 0x7fe97006d960 2024-05-26T22:49:07.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.713+0000 7fe96fe00640 1 -- 172.21.15.131:0/3818687598 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe970140090 con 0x7fe97006d960 2024-05-26T22:49:07.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.713+0000 7fe96fe00640 1 -- 172.21.15.131:0/3818687598 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe970140650 con 0x7fe97006d960 2024-05-26T22:49:07.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.715+0000 7fe95be00640 1 -- 172.21.15.131:0/3818687598 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe944005290 con 0x7fe97006d960 2024-05-26T22:49:07.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.715+0000 7fe966a00640 1 -- 172.21.15.131:0/3818687598 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe960044070 con 0x7fe97006d960 2024-05-26T22:49:07.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.716+0000 7fe966a00640 1 --2- 172.21.15.131:0/3818687598 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fe95006be30 0x7fe95006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:07.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.716+0000 7fe966a00640 1 -- 172.21.15.131:0/3818687598 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fe9600abd30 con 0x7fe97006d960 2024-05-26T22:49:07.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.716+0000 7fe96e000640 1 --2- 172.21.15.131:0/3818687598 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fe95006be30 0x7fe95006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:07.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.717+0000 7fe96e000640 1 --2- 172.21.15.131:0/3818687598 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fe95006be30 0x7fe95006e2f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe95c000f60 tx=0x7fe95c007a80 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:07.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.720+0000 7fe966a00640 1 -- 172.21.15.131:0/3818687598 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fe96007a6e0 con 0x7fe97006d960 2024-05-26T22:49:07.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.917+0000 7fe95be00640 1 -- 172.21.15.131:0/3818687598 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe944005d90 con 0x7fe97006d960 2024-05-26T22:49:07.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.918+0000 7fe966a00640 1 -- 172.21.15.131:0/3818687598 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7fe960079fe0 con 0x7fe97006d960 2024-05-26T22:49:07.918 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-05-26T22:49:07.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.921+0000 7fe95be00640 1 -- 172.21.15.131:0/3818687598 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fe95006be30 msgr2=0x7fe95006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:07.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.921+0000 7fe95be00640 1 --2- 172.21.15.131:0/3818687598 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fe95006be30 0x7fe95006e2f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe95c000f60 tx=0x7fe95c007a80 comp rx=0 tx=0).stop 2024-05-26T22:49:07.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.921+0000 7fe95be00640 1 -- 172.21.15.131:0/3818687598 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe97006d960 msgr2=0x7fe9701f19e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:07.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.921+0000 7fe95be00640 1 --2- 172.21.15.131:0/3818687598 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe97006d960 0x7fe9701f19e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fe9600098c0 tx=0x7fe9600046c0 comp rx=0 tx=0).stop 2024-05-26T22:49:07.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.921+0000 7fe95be00640 1 -- 172.21.15.131:0/3818687598 shutdown_connections 2024-05-26T22:49:07.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.921+0000 7fe95be00640 1 --2- 172.21.15.131:0/3818687598 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe9700712d0 0x7fe970140e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:07.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.921+0000 7fe95be00640 1 --2- 172.21.15.131:0/3818687598 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fe95006be30 0x7fe95006e2f0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:07.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.921+0000 7fe95be00640 1 --2- 172.21.15.131:0/3818687598 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe970145a10 0x7fe970158a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:07.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.921+0000 7fe95be00640 1 --2- 172.21.15.131:0/3818687598 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe97006d960 0x7fe9701f19e0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:07.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.921+0000 7fe95be00640 1 -- 172.21.15.131:0/3818687598 >> 172.21.15.131:0/3818687598 conn(0x7fe9700b5660 msgr2=0x7fe97015f560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:07.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.921+0000 7fe95be00640 1 -- 172.21.15.131:0/3818687598 shutdown_connections 2024-05-26T22:49:07.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:07.921+0000 7fe95be00640 1 -- 172.21.15.131:0/3818687598 wait complete. 2024-05-26T22:49:07.940 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-26T22:49:08.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.139+0000 7f563a800640 1 Processor -- start 2024-05-26T22:49:08.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.139+0000 7f563a800640 1 -- start start 2024-05-26T22:49:08.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.141+0000 7f563a800640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f563415b730 0x7f563415bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:08.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.141+0000 7f563a800640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f563413e820 con 0x7f56341400c0 2024-05-26T22:49:08.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.141+0000 7f563a800640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f563415c4a0 con 0x7f563406d250 2024-05-26T22:49:08.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.141+0000 7f563a800640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f5634071260 con 0x7f563415b730 2024-05-26T22:49:08.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.141+0000 7f5639e00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f56341400c0 0x7f56341404f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:46654/0 (socket says 172.21.15.131:46654) 2024-05-26T22:49:08.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.141+0000 7f5639e00640 1 -- 172.21.15.131:0/2172297592 learned_addr learned my addr 172.21.15.131:0/2172297592 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:49:08.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.142+0000 7f5638a00640 1 --2- 172.21.15.131:0/2172297592 >> v2:172.21.15.187:3300/0 conn(0x7f563415b730 0x7f563415bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:08.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.142+0000 7f5633e00640 1 -- 172.21.15.131:0/2172297592 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3198954373 0 0) 0x7f563413e820 con 0x7f56341400c0 2024-05-26T22:49:08.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.142+0000 7f5633e00640 1 -- 172.21.15.131:0/2172297592 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f561c003520 con 0x7f56341400c0 2024-05-26T22:49:08.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.142+0000 7f5633e00640 1 -- 172.21.15.131:0/2172297592 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3598985050 0 0) 0x7f561c003520 con 0x7f56341400c0 2024-05-26T22:49:08.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.142+0000 7f5633e00640 1 -- 172.21.15.131:0/2172297592 >> v2:172.21.15.187:3300/0 conn(0x7f563415b730 msgr2=0x7f563415bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:08.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.142+0000 7f5633e00640 1 --2- 172.21.15.131:0/2172297592 >> v2:172.21.15.187:3300/0 conn(0x7f563415b730 0x7f563415bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:08.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.142+0000 7f5633e00640 1 -- 172.21.15.131:0/2172297592 >> v1:172.21.15.187:6789/0 conn(0x7f563406d250 legacy=0x7f563406d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:08.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.142+0000 7f5633e00640 1 -- 172.21.15.131:0/2172297592 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56340714b0 con 0x7f56341400c0 2024-05-26T22:49:08.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.142+0000 7f5633e00640 1 -- 172.21.15.131:0/2172297592 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7f562c001670 con 0x7f56341400c0 2024-05-26T22:49:08.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.142+0000 7f5638a00640 1 --2- 172.21.15.131:0/2172297592 >> v2:172.21.15.187:3300/0 conn(0x7f563415b730 0x7f563415bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-26T22:49:08.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.142+0000 7f5633e00640 1 -- 172.21.15.131:0/2172297592 >> v1:172.21.15.131:6789/0 conn(0x7f56341400c0 legacy=0x7f56341404f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:08.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.142+0000 7f5633e00640 1 --2- 172.21.15.131:0/2172297592 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f561c0039b0 0x7f561c003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:08.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.142+0000 7f5633e00640 1 --2- 172.21.15.131:0/2172297592 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f561c004800 0x7f561c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:08.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.142+0000 7f5633e00640 1 --2- 172.21.15.131:0/2172297592 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f561c005210 0x7f561c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:08.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.142+0000 7f5633e00640 1 -- 172.21.15.131:0/2172297592 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f5634071260 con 0x7f561c005210 2024-05-26T22:49:08.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.142+0000 7f5633e00640 1 -- 172.21.15.131:0/2172297592 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f561c00c140 con 0x7f561c0039b0 2024-05-26T22:49:08.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.142+0000 7f5633e00640 1 -- 172.21.15.131:0/2172297592 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f561c00e550 con 0x7f561c004800 2024-05-26T22:49:08.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.143+0000 7f5639e00640 1 --2- 172.21.15.131:0/2172297592 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f561c005210 0x7f561c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:08.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.143+0000 7f5639400640 1 --2- 172.21.15.131:0/2172297592 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f561c0039b0 0x7f561c003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:08.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.143+0000 7f5638a00640 1 --2- 172.21.15.131:0/2172297592 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f561c004800 0x7f561c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:08.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.143+0000 7f5639e00640 1 -- 172.21.15.131:0/2172297592 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f561c004800 msgr2=0x7f561c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:08.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.143+0000 7f5639e00640 1 --2- 172.21.15.131:0/2172297592 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f561c004800 0x7f561c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:08.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.143+0000 7f5639e00640 1 -- 172.21.15.131:0/2172297592 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f561c0039b0 msgr2=0x7f561c003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:08.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.143+0000 7f5639e00640 1 --2- 172.21.15.131:0/2172297592 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f561c0039b0 0x7f561c003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:08.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.143+0000 7f5639e00640 1 -- 172.21.15.131:0/2172297592 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f561c003520 con 0x7f561c005210 2024-05-26T22:49:08.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.143+0000 7f5638a00640 1 --2- 172.21.15.131:0/2172297592 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f561c004800 0x7f561c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:49:08.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.144+0000 7f5639e00640 1 -- 172.21.15.131:0/2172297592 reap_dead start 2024-05-26T22:49:08.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.144+0000 7f5639e00640 1 --2- 172.21.15.131:0/2172297592 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f561c005210 0x7f561c00b9c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f562c00c240 tx=0x7f561c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1941dfb60285282b server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:08.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.144+0000 7f5633e00640 1 -- 172.21.15.131:0/2172297592 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f562c007cf0 con 0x7f561c005210 2024-05-26T22:49:08.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.144+0000 7f5633e00640 1 -- 172.21.15.131:0/2172297592 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f562c0058d0 con 0x7f561c005210 2024-05-26T22:49:08.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.144+0000 7f5633e00640 1 -- 172.21.15.131:0/2172297592 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f562c005ce0 con 0x7f561c005210 2024-05-26T22:49:08.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.145+0000 7f563a800640 1 -- 172.21.15.131:0/2172297592 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f561c005210 msgr2=0x7f561c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:08.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.145+0000 7f563a800640 1 --2- 172.21.15.131:0/2172297592 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f561c005210 0x7f561c00b9c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f562c00c240 tx=0x7f561c005830 comp rx=0 tx=0).stop 2024-05-26T22:49:08.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.145+0000 7f563a800640 1 -- 172.21.15.131:0/2172297592 shutdown_connections 2024-05-26T22:49:08.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.145+0000 7f563a800640 1 --2- 172.21.15.131:0/2172297592 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f561c005210 0x7f561c00b9c0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:08.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.145+0000 7f563a800640 1 -- 172.21.15.131:0/2172297592 >> 172.21.15.131:0/2172297592 conn(0x7f56340b5660 msgr2=0x7f56340b5a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:08.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.145+0000 7f563a800640 1 -- 172.21.15.131:0/2172297592 shutdown_connections 2024-05-26T22:49:08.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.145+0000 7f563a800640 1 -- 172.21.15.131:0/2172297592 wait complete. 2024-05-26T22:49:08.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.147+0000 7f563a800640 1 Processor -- start 2024-05-26T22:49:08.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.147+0000 7f563a800640 1 -- start start 2024-05-26T22:49:08.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.148+0000 7f563a800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f561c005210 0x7f5634155d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:08.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.148+0000 7f563a800640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5634158970 0x7f563413ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:08.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.148+0000 7f563a800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5634158df0 0x7f563413ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:08.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.148+0000 7f563a800640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f563406d630 con 0x7f5634158df0 2024-05-26T22:49:08.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.148+0000 7f563a800640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5634071a70 con 0x7f5634158970 2024-05-26T22:49:08.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.148+0000 7f563a800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f563406d7b0 con 0x7f561c005210 2024-05-26T22:49:08.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.148+0000 7f5639e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5634158df0 0x7f563413ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:08.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.148+0000 7f5639e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5634158df0 0x7f563413ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41880/0 (socket says 172.21.15.131:41880) 2024-05-26T22:49:08.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.148+0000 7f5639e00640 1 -- 172.21.15.131:0/752911229 learned_addr learned my addr 172.21.15.131:0/752911229 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:08.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.149+0000 7f5639400640 1 --2- 172.21.15.131:0/752911229 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f561c005210 0x7f5634155d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:08.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.149+0000 7f5638a00640 1 --2- 172.21.15.131:0/752911229 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5634158970 0x7f563413ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:08.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.149+0000 7f5639e00640 1 -- 172.21.15.131:0/752911229 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f561c005210 msgr2=0x7f5634155d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:08.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.149+0000 7f5639e00640 1 --2- 172.21.15.131:0/752911229 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f561c005210 0x7f5634155d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:08.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.149+0000 7f5639e00640 1 -- 172.21.15.131:0/752911229 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5634158970 msgr2=0x7f563413ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:08.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.149+0000 7f5639e00640 1 --2- 172.21.15.131:0/752911229 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5634158970 0x7f563413ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:08.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.149+0000 7f5639e00640 1 -- 172.21.15.131:0/752911229 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5634157910 con 0x7f5634158df0 2024-05-26T22:49:08.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.149+0000 7f5639e00640 1 --2- 172.21.15.131:0/752911229 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5634158df0 0x7f563413ef50 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f562c00c360 tx=0x7f5634071680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:08.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.149+0000 7f5631600640 1 -- 172.21.15.131:0/752911229 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f562c038040 con 0x7f5634158df0 2024-05-26T22:49:08.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.149+0000 7f5631600640 1 -- 172.21.15.131:0/752911229 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f562c000ed0 con 0x7f5634158df0 2024-05-26T22:49:08.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.150+0000 7f563a800640 1 -- 172.21.15.131:0/752911229 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5634157bb0 con 0x7f5634158df0 2024-05-26T22:49:08.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.150+0000 7f5631600640 1 -- 172.21.15.131:0/752911229 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f562c001c30 con 0x7f5634158df0 2024-05-26T22:49:08.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.150+0000 7f563a800640 1 -- 172.21.15.131:0/752911229 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5634158170 con 0x7f5634158df0 2024-05-26T22:49:08.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.151+0000 7f563a800640 1 -- 172.21.15.131:0/752911229 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56340694e0 con 0x7f5634158df0 2024-05-26T22:49:08.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.152+0000 7f5631600640 1 -- 172.21.15.131:0/752911229 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f562c043020 con 0x7f5634158df0 2024-05-26T22:49:08.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.153+0000 7f5631600640 1 --2- 172.21.15.131:0/752911229 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f561406be30 0x7f561406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:08.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.153+0000 7f5631600640 1 -- 172.21.15.131:0/752911229 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f562c0aa340 con 0x7f5634158df0 2024-05-26T22:49:08.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.153+0000 7f5639400640 1 --2- 172.21.15.131:0/752911229 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f561406be30 0x7f561406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:08.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.153+0000 7f5639400640 1 --2- 172.21.15.131:0/752911229 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f561406be30 0x7f561406e2f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5628007fd0 tx=0x7f5628009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:08.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.157+0000 7f5631600640 1 -- 172.21.15.131:0/752911229 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f562c044370 con 0x7f5634158df0 2024-05-26T22:49:08.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.357+0000 7f563a800640 1 -- 172.21.15.131:0/752911229 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f56340af9b0 con 0x7f5634158df0 2024-05-26T22:49:08.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.358+0000 7f5631600640 1 -- 172.21.15.131:0/752911229 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7f562c078bd0 con 0x7f5634158df0 2024-05-26T22:49:08.357 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 64 2024-05-26T22:49:08.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.360+0000 7f563a800640 1 -- 172.21.15.131:0/752911229 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f561406be30 msgr2=0x7f561406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:08.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.360+0000 7f563a800640 1 --2- 172.21.15.131:0/752911229 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f561406be30 0x7f561406e2f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5628007fd0 tx=0x7f5628009040 comp rx=0 tx=0).stop 2024-05-26T22:49:08.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.360+0000 7f563a800640 1 -- 172.21.15.131:0/752911229 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5634158df0 msgr2=0x7f563413ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:08.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.360+0000 7f563a800640 1 --2- 172.21.15.131:0/752911229 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5634158df0 0x7f563413ef50 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f562c00c360 tx=0x7f5634071680 comp rx=0 tx=0).stop 2024-05-26T22:49:08.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.361+0000 7f563a800640 1 -- 172.21.15.131:0/752911229 shutdown_connections 2024-05-26T22:49:08.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.361+0000 7f563a800640 1 --2- 172.21.15.131:0/752911229 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f561406be30 0x7f561406e2f0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:08.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.361+0000 7f563a800640 1 --2- 172.21.15.131:0/752911229 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5634158df0 0x7f563413ef50 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:08.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.361+0000 7f563a800640 1 --2- 172.21.15.131:0/752911229 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5634158970 0x7f563413ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:08.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.361+0000 7f563a800640 1 --2- 172.21.15.131:0/752911229 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f561c005210 0x7f5634155d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:08.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.361+0000 7f563a800640 1 -- 172.21.15.131:0/752911229 >> 172.21.15.131:0/752911229 conn(0x7f56340b5660 msgr2=0x7f5634159470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:08.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.361+0000 7f563a800640 1 -- 172.21.15.131:0/752911229 shutdown_connections 2024-05-26T22:49:08.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.361+0000 7f563a800640 1 -- 172.21.15.131:0/752911229 wait complete. 2024-05-26T22:49:08.379 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-26T22:49:08.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.566+0000 7f8a96200640 1 Processor -- start 2024-05-26T22:49:08.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.566+0000 7f8a96200640 1 -- start start 2024-05-26T22:49:08.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.568+0000 7f8a96200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8a90145a10 0x7f8a90072180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:08.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.568+0000 7f8a96200640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f8a9006d960 0x7f8a900cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:08.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.568+0000 7f8a96200640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a90072d60 con 0x7f8a900712d0 2024-05-26T22:49:08.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.568+0000 7f8a96200640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f8a9015c760 con 0x7f8a9006d960 2024-05-26T22:49:08.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.568+0000 7f8a96200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f8a90072a80 con 0x7f8a90145a10 2024-05-26T22:49:08.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.569+0000 7f8a87e00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f8a900712d0 0x7f8a9006d250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:46664/0 (socket says 172.21.15.131:46664) 2024-05-26T22:49:08.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.569+0000 7f8a87e00640 1 -- 172.21.15.131:0/1241182628 learned_addr learned my addr 172.21.15.131:0/1241182628 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:49:08.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.569+0000 7f8a94e00640 1 --2- 172.21.15.131:0/1241182628 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8a90145a10 0x7f8a90072180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:08.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.569+0000 7f8a95800640 1 --2- 172.21.15.131:0/1241182628 >> v2:172.21.15.187:3300/0 conn(0x7f8a9006d960 0x7f8a900cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:08.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.569+0000 7f8a8fe00640 1 -- 172.21.15.131:0/1241182628 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 968672972 0 0) 0x7f8a90072d60 con 0x7f8a900712d0 2024-05-26T22:49:08.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.569+0000 7f8a8fe00640 1 -- 172.21.15.131:0/1241182628 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8a74003520 con 0x7f8a900712d0 2024-05-26T22:49:08.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.569+0000 7f8a8fe00640 1 -- 172.21.15.131:0/1241182628 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3263070325 0 0) 0x7f8a74003520 con 0x7f8a900712d0 2024-05-26T22:49:08.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.569+0000 7f8a8fe00640 1 -- 172.21.15.131:0/1241182628 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8a90145a10 msgr2=0x7f8a90072180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:08.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.569+0000 7f8a8fe00640 1 --2- 172.21.15.131:0/1241182628 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8a90145a10 0x7f8a90072180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:08.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.569+0000 7f8a8fe00640 1 -- 172.21.15.131:0/1241182628 >> v2:172.21.15.187:3300/0 conn(0x7f8a9006d960 msgr2=0x7f8a900cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:08.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.569+0000 7f8a8fe00640 1 --2- 172.21.15.131:0/1241182628 >> v2:172.21.15.187:3300/0 conn(0x7f8a9006d960 0x7f8a900cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:08.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.570+0000 7f8a8fe00640 1 -- 172.21.15.131:0/1241182628 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a900ceab0 con 0x7f8a900712d0 2024-05-26T22:49:08.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.570+0000 7f8a8fe00640 1 -- 172.21.15.131:0/1241182628 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7f8a78001670 con 0x7f8a900712d0 2024-05-26T22:49:08.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.570+0000 7f8a8fe00640 1 -- 172.21.15.131:0/1241182628 >> v1:172.21.15.131:6789/0 conn(0x7f8a900712d0 legacy=0x7f8a9006d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:08.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.570+0000 7f8a8fe00640 1 --2- 172.21.15.131:0/1241182628 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8a740039b0 0x7f8a74003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:08.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.570+0000 7f8a8fe00640 1 --2- 172.21.15.131:0/1241182628 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8a90145a10 0x7f8a74004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:08.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.570+0000 7f8a8fe00640 1 --2- 172.21.15.131:0/1241182628 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8a74004d40 0x7f8a740051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:08.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.570+0000 7f8a8fe00640 1 -- 172.21.15.131:0/1241182628 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8a9015c760 con 0x7f8a74004d40 2024-05-26T22:49:08.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.570+0000 7f8a8fe00640 1 -- 172.21.15.131:0/1241182628 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8a90072a80 con 0x7f8a740039b0 2024-05-26T22:49:08.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.570+0000 7f8a8fe00640 1 -- 172.21.15.131:0/1241182628 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f8a7400b9c0 con 0x7f8a90145a10 2024-05-26T22:49:08.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.570+0000 7f8a95800640 1 --2- 172.21.15.131:0/1241182628 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8a74004d40 0x7f8a740051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:08.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.570+0000 7f8a94e00640 1 --2- 172.21.15.131:0/1241182628 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8a740039b0 0x7f8a74003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:08.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.570+0000 7f8a95800640 1 -- 172.21.15.131:0/1241182628 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8a90145a10 msgr2=0x7f8a74004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:08.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.570+0000 7f8a95800640 1 --2- 172.21.15.131:0/1241182628 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8a90145a10 0x7f8a74004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:08.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.570+0000 7f8a95800640 1 -- 172.21.15.131:0/1241182628 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8a740039b0 msgr2=0x7f8a74003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:08.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.570+0000 7f8a95800640 1 --2- 172.21.15.131:0/1241182628 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8a740039b0 0x7f8a74003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:08.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.570+0000 7f8a95800640 1 -- 172.21.15.131:0/1241182628 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a74003520 con 0x7f8a74004d40 2024-05-26T22:49:08.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.570+0000 7f8a94e00640 1 --2- 172.21.15.131:0/1241182628 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8a740039b0 0x7f8a74003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:49:08.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.571+0000 7f8a95800640 1 --2- 172.21.15.131:0/1241182628 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8a74004d40 0x7f8a740051c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f8a900b2900 tx=0x7f8a74005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f276f6f776c7d4e3 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:08.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.571+0000 7f8a8fe00640 1 -- 172.21.15.131:0/1241182628 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a88009c10 con 0x7f8a74004d40 2024-05-26T22:49:08.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.572+0000 7f8a8fe00640 1 -- 172.21.15.131:0/1241182628 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a88036070 con 0x7f8a74004d40 2024-05-26T22:49:08.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.572+0000 7f8a8fe00640 1 -- 172.21.15.131:0/1241182628 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a88031850 con 0x7f8a74004d40 2024-05-26T22:49:08.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.572+0000 7f8a96200640 1 -- 172.21.15.131:0/1241182628 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8a74004d40 msgr2=0x7f8a740051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:08.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.572+0000 7f8a96200640 1 --2- 172.21.15.131:0/1241182628 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8a74004d40 0x7f8a740051c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f8a900b2900 tx=0x7f8a74005830 comp rx=0 tx=0).stop 2024-05-26T22:49:08.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.572+0000 7f8a95800640 1 -- 172.21.15.131:0/1241182628 reap_dead start 2024-05-26T22:49:08.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.572+0000 7f8a96200640 1 -- 172.21.15.131:0/1241182628 shutdown_connections 2024-05-26T22:49:08.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.572+0000 7f8a96200640 1 -- 172.21.15.131:0/1241182628 >> 172.21.15.131:0/1241182628 conn(0x7f8a900b5660 msgr2=0x7f8a900b5a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:08.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.572+0000 7f8a96200640 1 -- 172.21.15.131:0/1241182628 shutdown_connections 2024-05-26T22:49:08.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.572+0000 7f8a96200640 1 -- 172.21.15.131:0/1241182628 wait complete. 2024-05-26T22:49:08.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.574+0000 7f8a96200640 1 Processor -- start 2024-05-26T22:49:08.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.574+0000 7f8a96200640 1 -- start start 2024-05-26T22:49:08.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.575+0000 7f8a96200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8a900cca50 0x7f8a900a9460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:08.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.575+0000 7f8a96200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8a901f1fc0 0x7f8a90143b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:08.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.575+0000 7f8a96200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8a900aa8b0 0x7f8a901440d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:08.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.575+0000 7f8a96200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8a90145df0 con 0x7f8a901f1fc0 2024-05-26T22:49:08.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.575+0000 7f8a96200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8a900cf070 con 0x7f8a900aa8b0 2024-05-26T22:49:08.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.575+0000 7f8a96200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f8a90072180 con 0x7f8a900cca50 2024-05-26T22:49:08.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.575+0000 7f8a87e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8a901f1fc0 0x7f8a90143b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:08.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.575+0000 7f8a87e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8a901f1fc0 0x7f8a90143b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41894/0 (socket says 172.21.15.131:41894) 2024-05-26T22:49:08.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.575+0000 7f8a87e00640 1 -- 172.21.15.131:0/2251917878 learned_addr learned my addr 172.21.15.131:0/2251917878 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:08.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.576+0000 7f8a87e00640 1 -- 172.21.15.131:0/2251917878 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8a900cca50 msgr2=0x7f8a900a9460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:08.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.576+0000 7f8a94e00640 1 --2- 172.21.15.131:0/2251917878 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8a900cca50 0x7f8a900a9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:08.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.576+0000 7f8a95800640 1 --2- 172.21.15.131:0/2251917878 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8a900aa8b0 0x7f8a901440d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:08.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.576+0000 7f8a87e00640 1 --2- 172.21.15.131:0/2251917878 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8a900cca50 0x7f8a900a9460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:08.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.576+0000 7f8a87e00640 1 -- 172.21.15.131:0/2251917878 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8a900aa8b0 msgr2=0x7f8a901440d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:08.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.576+0000 7f8a87e00640 1 --2- 172.21.15.131:0/2251917878 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8a900aa8b0 0x7f8a901440d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:08.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.576+0000 7f8a87e00640 1 -- 172.21.15.131:0/2251917878 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a9015c260 con 0x7f8a901f1fc0 2024-05-26T22:49:08.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.576+0000 7f8a87e00640 1 --2- 172.21.15.131:0/2251917878 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8a901f1fc0 0x7f8a90143b90 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f8a7800cad0 tx=0x7f8a7800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:08.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.576+0000 7f8a8d600640 1 -- 172.21.15.131:0/2251917878 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a78018070 con 0x7f8a901f1fc0 2024-05-26T22:49:08.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.576+0000 7f8a8d600640 1 -- 172.21.15.131:0/2251917878 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a78004cb0 con 0x7f8a901f1fc0 2024-05-26T22:49:08.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.576+0000 7f8a8d600640 1 -- 172.21.15.131:0/2251917878 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a78004e60 con 0x7f8a901f1fc0 2024-05-26T22:49:08.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.576+0000 7f8a96200640 1 -- 172.21.15.131:0/2251917878 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a901f24a0 con 0x7f8a901f1fc0 2024-05-26T22:49:08.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.576+0000 7f8a96200640 1 -- 172.21.15.131:0/2251917878 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a901f2a60 con 0x7f8a901f1fc0 2024-05-26T22:49:08.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.577+0000 7f8a96200640 1 -- 172.21.15.131:0/2251917878 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a900694e0 con 0x7f8a901f1fc0 2024-05-26T22:49:08.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.578+0000 7f8a8d600640 1 -- 172.21.15.131:0/2251917878 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8a78007310 con 0x7f8a901f1fc0 2024-05-26T22:49:08.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.579+0000 7f8a8d600640 1 --2- 172.21.15.131:0/2251917878 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f8a6c06be30 0x7f8a6c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:08.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.579+0000 7f8a8d600640 1 -- 172.21.15.131:0/2251917878 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f8a7808a570 con 0x7f8a901f1fc0 2024-05-26T22:49:08.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.579+0000 7f8a94e00640 1 --2- 172.21.15.131:0/2251917878 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f8a6c06be30 0x7f8a6c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:08.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.580+0000 7f8a94e00640 1 --2- 172.21.15.131:0/2251917878 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f8a6c06be30 0x7f8a6c06e2f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8a800099c0 tx=0x7f8a80008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:08.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.583+0000 7f8a8d600640 1 -- 172.21.15.131:0/2251917878 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f8a78058e00 con 0x7f8a901f1fc0 2024-05-26T22:49:08.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.795+0000 7f8a96200640 1 -- 172.21.15.131:0/2251917878 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f8a900b04c0 con 0x7f8a901f1fc0 2024-05-26T22:49:08.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.796+0000 7f8a8d600640 1 -- 172.21.15.131:0/2251917878 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7f8a78058700 con 0x7f8a901f1fc0 2024-05-26T22:49:08.795 INFO:teuthology.orchestra.run.smithi131.stderr:listed 0 entries 2024-05-26T22:49:08.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.799+0000 7f8a96200640 1 -- 172.21.15.131:0/2251917878 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f8a6c06be30 msgr2=0x7f8a6c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:08.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.799+0000 7f8a96200640 1 --2- 172.21.15.131:0/2251917878 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f8a6c06be30 0x7f8a6c06e2f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8a800099c0 tx=0x7f8a80008040 comp rx=0 tx=0).stop 2024-05-26T22:49:08.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.799+0000 7f8a96200640 1 -- 172.21.15.131:0/2251917878 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8a901f1fc0 msgr2=0x7f8a90143b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:08.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.799+0000 7f8a96200640 1 --2- 172.21.15.131:0/2251917878 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8a901f1fc0 0x7f8a90143b90 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f8a7800cad0 tx=0x7f8a7800cf90 comp rx=0 tx=0).stop 2024-05-26T22:49:08.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.799+0000 7f8a96200640 1 -- 172.21.15.131:0/2251917878 shutdown_connections 2024-05-26T22:49:08.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.799+0000 7f8a96200640 1 --2- 172.21.15.131:0/2251917878 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f8a6c06be30 0x7f8a6c06e2f0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:08.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.799+0000 7f8a96200640 1 --2- 172.21.15.131:0/2251917878 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8a900aa8b0 0x7f8a901440d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:08.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.799+0000 7f8a96200640 1 --2- 172.21.15.131:0/2251917878 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8a901f1fc0 0x7f8a90143b90 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:08.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.799+0000 7f8a96200640 1 --2- 172.21.15.131:0/2251917878 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8a900cca50 0x7f8a900a9460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:08.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.799+0000 7f8a96200640 1 -- 172.21.15.131:0/2251917878 >> 172.21.15.131:0/2251917878 conn(0x7f8a900b5660 msgr2=0x7f8a9015fca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:08.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.799+0000 7f8a96200640 1 -- 172.21.15.131:0/2251917878 shutdown_connections 2024-05-26T22:49:08.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:08.799+0000 7f8a96200640 1 -- 172.21.15.131:0/2251917878 wait complete. 2024-05-26T22:49:08.817 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-05-26T22:49:08.817 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:49:08.817 DEBUG:teuthology.orchestra.run.smithi187:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2024-05-26T22:49:08.912 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi187.front.sepia.ceph.com 2024-05-26T22:49:08.913 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:49:08.913 DEBUG:teuthology.orchestra.run.smithi187:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-05-26T22:49:08.976 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-26T22:49:08.976 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-26T22:49:09.034 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-26T22:49:09.034 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-26T22:49:09.088 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-26T22:49:09.088 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-26T22:49:09.143 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-26T22:49:09.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.348+0000 7f8255800640 1 Processor -- start 2024-05-26T22:49:09.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.348+0000 7f8255800640 1 -- start start 2024-05-26T22:49:09.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.349+0000 7f8255800640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f8250144340 0x7f82500cedf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:09.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.349+0000 7f8255800640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8250072000 con 0x7f825006ebc0 2024-05-26T22:49:09.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.349+0000 7f8255800640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82500722a0 con 0x7f8250071b70 2024-05-26T22:49:09.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.349+0000 7f8255800640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f8250072eb0 con 0x7f8250144340 2024-05-26T22:49:09.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.350+0000 7f8254e00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f825006ebc0 0x7f82500b3d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:46668/0 (socket says 172.21.15.131:46668) 2024-05-26T22:49:09.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.350+0000 7f8254e00640 1 -- 172.21.15.131:0/647804560 learned_addr learned my addr 172.21.15.131:0/647804560 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:49:09.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.350+0000 7f824fe00640 1 --2- 172.21.15.131:0/647804560 >> v2:172.21.15.187:3300/0 conn(0x7f8250144340 0x7f82500cedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:09.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.350+0000 7f824f400640 1 -- 172.21.15.131:0/647804560 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2831559634 0 0) 0x7f8250072000 con 0x7f825006ebc0 2024-05-26T22:49:09.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.350+0000 7f824f400640 1 -- 172.21.15.131:0/647804560 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8234003520 con 0x7f825006ebc0 2024-05-26T22:49:09.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.351+0000 7f824f400640 1 -- 172.21.15.131:0/647804560 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3241259854 0 0) 0x7f8234003520 con 0x7f825006ebc0 2024-05-26T22:49:09.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.351+0000 7f824f400640 1 -- 172.21.15.131:0/647804560 >> v2:172.21.15.187:3300/0 conn(0x7f8250144340 msgr2=0x7f82500cedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:09.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.351+0000 7f824f400640 1 --2- 172.21.15.131:0/647804560 >> v2:172.21.15.187:3300/0 conn(0x7f8250144340 0x7f82500cedf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:09.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.351+0000 7f824f400640 1 -- 172.21.15.131:0/647804560 >> v1:172.21.15.187:6789/0 conn(0x7f8250071b70 legacy=0x7f825006e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:09.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.351+0000 7f824f400640 1 -- 172.21.15.131:0/647804560 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8250072580 con 0x7f825006ebc0 2024-05-26T22:49:09.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.351+0000 7f824f400640 1 -- 172.21.15.131:0/647804560 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7f8248001670 con 0x7f825006ebc0 2024-05-26T22:49:09.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.351+0000 7f824f400640 1 -- 172.21.15.131:0/647804560 >> v1:172.21.15.131:6789/0 conn(0x7f825006ebc0 legacy=0x7f82500b3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:09.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.351+0000 7f824f400640 1 --2- 172.21.15.131:0/647804560 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8234003a00 0x7f8234003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:09.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.351+0000 7f824f400640 1 --2- 172.21.15.131:0/647804560 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8234004800 0x7f8234004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:09.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.351+0000 7f824f400640 1 --2- 172.21.15.131:0/647804560 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f82340051c0 0x7f823400dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:09.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.351+0000 7f824f400640 1 -- 172.21.15.131:0/647804560 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f823400e2a0 con 0x7f82340051c0 2024-05-26T22:49:09.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.351+0000 7f824f400640 1 -- 172.21.15.131:0/647804560 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f823400e450 con 0x7f8234004800 2024-05-26T22:49:09.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.351+0000 7f824f400640 1 -- 172.21.15.131:0/647804560 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f823400e650 con 0x7f8234003a00 2024-05-26T22:49:09.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.351+0000 7f8254e00640 1 --2- 172.21.15.131:0/647804560 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f82340051c0 0x7f823400dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:09.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.351+0000 7f8247400640 1 --2- 172.21.15.131:0/647804560 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8234004800 0x7f8234004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:09.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.352+0000 7f824fe00640 1 --2- 172.21.15.131:0/647804560 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8234003a00 0x7f8234003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:09.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.352+0000 7f8254e00640 1 -- 172.21.15.131:0/647804560 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8234003a00 msgr2=0x7f8234003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:09.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.352+0000 7f8254e00640 1 --2- 172.21.15.131:0/647804560 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8234003a00 0x7f8234003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:09.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.352+0000 7f8254e00640 1 -- 172.21.15.131:0/647804560 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8234004800 msgr2=0x7f8234004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:09.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.352+0000 7f8254e00640 1 --2- 172.21.15.131:0/647804560 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8234004800 0x7f8234004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:09.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.352+0000 7f8254e00640 1 -- 172.21.15.131:0/647804560 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8234003520 con 0x7f82340051c0 2024-05-26T22:49:09.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.353+0000 7f8254e00640 1 -- 172.21.15.131:0/647804560 reap_dead start 2024-05-26T22:49:09.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.353+0000 7f8254e00640 1 --2- 172.21.15.131:0/647804560 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f82340051c0 0x7f823400dbd0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f824800c0e0 tx=0x7f8234005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0fb4e1f21a3b4d server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:09.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.353+0000 7f824f400640 1 -- 172.21.15.131:0/647804560 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8248007c40 con 0x7f82340051c0 2024-05-26T22:49:09.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.353+0000 7f824f400640 1 -- 172.21.15.131:0/647804560 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8248005ad0 con 0x7f82340051c0 2024-05-26T22:49:09.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.353+0000 7f824f400640 1 -- 172.21.15.131:0/647804560 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8248007070 con 0x7f82340051c0 2024-05-26T22:49:09.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.354+0000 7f8255800640 1 -- 172.21.15.131:0/647804560 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f82340051c0 msgr2=0x7f823400dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:09.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.354+0000 7f8255800640 1 --2- 172.21.15.131:0/647804560 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f82340051c0 0x7f823400dbd0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f824800c0e0 tx=0x7f8234005830 comp rx=0 tx=0).stop 2024-05-26T22:49:09.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.354+0000 7f8255800640 1 -- 172.21.15.131:0/647804560 shutdown_connections 2024-05-26T22:49:09.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.354+0000 7f8255800640 1 --2- 172.21.15.131:0/647804560 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f82340051c0 0x7f823400dbd0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:09.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.354+0000 7f8255800640 1 -- 172.21.15.131:0/647804560 >> 172.21.15.131:0/647804560 conn(0x7f82500cff40 msgr2=0x7f82500d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:09.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.354+0000 7f8255800640 1 -- 172.21.15.131:0/647804560 shutdown_connections 2024-05-26T22:49:09.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.354+0000 7f8255800640 1 -- 172.21.15.131:0/647804560 wait complete. 2024-05-26T22:49:09.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.356+0000 7f8255800640 1 Processor -- start 2024-05-26T22:49:09.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.356+0000 7f8255800640 1 -- start start 2024-05-26T22:49:09.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.357+0000 7f8255800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f82340051c0 0x7f825015a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:09.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.357+0000 7f8255800640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f825015d230 0x7f825015d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:09.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.357+0000 7f8255800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f82500b00d0 0x7f82500b0550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:09.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.357+0000 7f8255800640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f82500cedf0 con 0x7f82500b00d0 2024-05-26T22:49:09.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.357+0000 7f8255800640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8250072b40 con 0x7f825015d230 2024-05-26T22:49:09.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.357+0000 7f8255800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f82500cef70 con 0x7f82340051c0 2024-05-26T22:49:09.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.357+0000 7f8254e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f82500b00d0 0x7f82500b0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:09.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.357+0000 7f8254e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f82500b00d0 0x7f82500b0550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41904/0 (socket says 172.21.15.131:41904) 2024-05-26T22:49:09.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.357+0000 7f8254e00640 1 -- 172.21.15.131:0/763354199 learned_addr learned my addr 172.21.15.131:0/763354199 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:09.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.357+0000 7f8247400640 1 --2- 172.21.15.131:0/763354199 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f825015d230 0x7f825015d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:09.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.357+0000 7f824fe00640 1 --2- 172.21.15.131:0/763354199 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f82340051c0 0x7f825015a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:09.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.357+0000 7f8254e00640 1 -- 172.21.15.131:0/763354199 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f82340051c0 msgr2=0x7f825015a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:09.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.357+0000 7f8254e00640 1 --2- 172.21.15.131:0/763354199 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f82340051c0 0x7f825015a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:09.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.357+0000 7f8254e00640 1 -- 172.21.15.131:0/763354199 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f825015d230 msgr2=0x7f825015d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:09.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.357+0000 7f8254e00640 1 --2- 172.21.15.131:0/763354199 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f825015d230 0x7f825015d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:09.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.357+0000 7f8254e00640 1 -- 172.21.15.131:0/763354199 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82501e4b10 con 0x7f82500b00d0 2024-05-26T22:49:09.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.357+0000 7f8247400640 1 --2- 172.21.15.131:0/763354199 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f825015d230 0x7f825015d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:49:09.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.357+0000 7f824fe00640 1 --2- 172.21.15.131:0/763354199 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f82340051c0 0x7f825015a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:49:09.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.358+0000 7f8254e00640 1 --2- 172.21.15.131:0/763354199 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f82500b00d0 0x7f82500b0550 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f824800c0e0 tx=0x7f82500ce260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:09.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.358+0000 7f824cc00640 1 -- 172.21.15.131:0/763354199 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8248040070 con 0x7f82500b00d0 2024-05-26T22:49:09.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.358+0000 7f824cc00640 1 -- 172.21.15.131:0/763354199 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82480047e0 con 0x7f82500b00d0 2024-05-26T22:49:09.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.358+0000 7f824cc00640 1 -- 172.21.15.131:0/763354199 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8248004bb0 con 0x7f82500b00d0 2024-05-26T22:49:09.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.358+0000 7f8255800640 1 -- 172.21.15.131:0/763354199 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82500b0e40 con 0x7f82500b00d0 2024-05-26T22:49:09.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.358+0000 7f8255800640 1 -- 172.21.15.131:0/763354199 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82501e5090 con 0x7f82500b00d0 2024-05-26T22:49:09.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.359+0000 7f8255800640 1 -- 172.21.15.131:0/763354199 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82500694e0 con 0x7f82500b00d0 2024-05-26T22:49:09.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.360+0000 7f824cc00640 1 -- 172.21.15.131:0/763354199 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8248043070 con 0x7f82500b00d0 2024-05-26T22:49:09.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.361+0000 7f824cc00640 1 --2- 172.21.15.131:0/763354199 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f822c06be30 0x7f822c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:09.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.361+0000 7f824cc00640 1 -- 172.21.15.131:0/763354199 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f82480a9fe0 con 0x7f82500b00d0 2024-05-26T22:49:09.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.361+0000 7f824fe00640 1 --2- 172.21.15.131:0/763354199 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f822c06be30 0x7f822c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:09.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.362+0000 7f824fe00640 1 --2- 172.21.15.131:0/763354199 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f822c06be30 0x7f822c06e2f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f82500b1550 tx=0x7f824000b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:09.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.366+0000 7f824cc00640 1 -- 172.21.15.131:0/763354199 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f8248078310 con 0x7f82500b00d0 2024-05-26T22:49:09.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.566+0000 7f8255800640 1 -- 172.21.15.131:0/763354199 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f82500ce720 con 0x7f82500b00d0 2024-05-26T22:49:09.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.569+0000 7f824cc00640 1 -- 172.21.15.131:0/763354199 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16879 (secure 0 0 0) 0x7f824807c610 con 0x7f82500b00d0 2024-05-26T22:49:09.569 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:49:09.569 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":22,"fsid":"46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f","created":"2024-05-26T22:47:57.288905+0000","modified":"2024-05-26T22:49:04.278988+0000","last_up_change":"2024-05-26T22:48:09.602207+0000","last_in_change":"2024-05-26T22:48:00.687341+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-26T22:48:10.463339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-26T22:48:19.663254+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-26T22:48:20.235751+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"814eaf62-a18d-494b-8e27-fd8fb88adea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6801","nonce":2350940051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6803","nonce":2350940051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6807","nonce":2350940051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6805","nonce":2350940051}]},"public_addr":"172.21.15.131:6801/2350940051","cluster_addr":"172.21.15.131:6803/2350940051","heartbeat_back_addr":"172.21.15.131:6807/2350940051","heartbeat_front_addr":"172.21.15.131:6805/2350940051","state":["exists","up"]},{"osd":1,"uuid":"42e66287-f6c6-4e26-af3c-6dfaa3536d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6825","nonce":1525528247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6827","nonce":1525528247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6831","nonce":1525528247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6829","nonce":1525528247}]},"public_addr":"172.21.15.131:6825/1525528247","cluster_addr":"172.21.15.131:6827/1525528247","heartbeat_back_addr":"172.21.15.131:6831/1525528247","heartbeat_front_addr":"172.21.15.131:6829/1525528247","state":["exists","up"]},{"osd":2,"uuid":"0da34673-04a2-4fd0-b57a-0ff2b679b043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6817","nonce":3627842210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6819","nonce":3627842210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6823","nonce":3627842210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6821","nonce":3627842210}]},"public_addr":"172.21.15.131:6817/3627842210","cluster_addr":"172.21.15.131:6819/3627842210","heartbeat_back_addr":"172.21.15.131:6823/3627842210","heartbeat_front_addr":"172.21.15.131:6821/3627842210","state":["exists","up"]},{"osd":3,"uuid":"cf117a27-d6b2-43d3-ae52-b47988bf8eed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6809","nonce":350790591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6811","nonce":350790591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6815","nonce":350790591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6813","nonce":350790591}]},"public_addr":"172.21.15.131:6809/350790591","cluster_addr":"172.21.15.131:6811/350790591","heartbeat_back_addr":"172.21.15.131:6815/350790591","heartbeat_front_addr":"172.21.15.131:6813/350790591","state":["exists","up"]},{"osd":4,"uuid":"a092ccfc-be89-4a84-88d5-0f781fa715eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6816","nonce":244560995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6819","nonce":244560995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6823","nonce":244560995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6821","nonce":244560995}]},"public_addr":"172.21.15.187:6816/244560995","cluster_addr":"172.21.15.187:6819/244560995","heartbeat_back_addr":"172.21.15.187:6823/244560995","heartbeat_front_addr":"172.21.15.187:6821/244560995","state":["exists","up"]},{"osd":5,"uuid":"7513ba69-dd00-4a35-b190-9e8cc3b2bdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6801","nonce":324619398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6803","nonce":324619398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6812","nonce":324619398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6807","nonce":324619398}]},"public_addr":"172.21.15.187:6801/324619398","cluster_addr":"172.21.15.187:6803/324619398","heartbeat_back_addr":"172.21.15.187:6812/324619398","heartbeat_front_addr":"172.21.15.187:6807/324619398","state":["exists","up"]},{"osd":6,"uuid":"50081810-c093-4d2f-a4d7-b7c695c5400d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6806","nonce":1391347786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6809","nonce":1391347786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6818","nonce":1391347786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6813","nonce":1391347786}]},"public_addr":"172.21.15.187:6806/1391347786","cluster_addr":"172.21.15.187:6809/1391347786","heartbeat_back_addr":"172.21.15.187:6818/1391347786","heartbeat_front_addr":"172.21.15.187:6813/1391347786","state":["exists","up"]},{"osd":7,"uuid":"775dae05-54f5-4ad4-bbce-92863c6ed2ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6825","nonce":1220065622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6827","nonce":1220065622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6831","nonce":1220065622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6829","nonce":1220065622}]},"public_addr":"172.21.15.187:6825/1220065622","cluster_addr":"172.21.15.187:6827/1220065622","heartbeat_back_addr":"172.21.15.187:6831/1220065622","heartbeat_front_addr":"172.21.15.187:6829/1220065622","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:05.983520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.299516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.067323+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.307748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.018742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.258775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.010912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.295355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.1e","mappings":[{"from":3,"to":2}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":7,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-26T22:49:09.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.572+0000 7f8255800640 1 -- 172.21.15.131:0/763354199 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f822c06be30 msgr2=0x7f822c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:09.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.572+0000 7f8255800640 1 --2- 172.21.15.131:0/763354199 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f822c06be30 0x7f822c06e2f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f82500b1550 tx=0x7f824000b040 comp rx=0 tx=0).stop 2024-05-26T22:49:09.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.572+0000 7f8255800640 1 -- 172.21.15.131:0/763354199 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f82500b00d0 msgr2=0x7f82500b0550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:09.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.572+0000 7f8255800640 1 --2- 172.21.15.131:0/763354199 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f82500b00d0 0x7f82500b0550 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f824800c0e0 tx=0x7f82500ce260 comp rx=0 tx=0).stop 2024-05-26T22:49:09.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.572+0000 7f8255800640 1 -- 172.21.15.131:0/763354199 shutdown_connections 2024-05-26T22:49:09.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.572+0000 7f8255800640 1 --2- 172.21.15.131:0/763354199 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f822c06be30 0x7f822c06e2f0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:09.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.572+0000 7f8255800640 1 --2- 172.21.15.131:0/763354199 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f82500b00d0 0x7f82500b0550 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:09.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.572+0000 7f8255800640 1 --2- 172.21.15.131:0/763354199 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f825015d230 0x7f825015d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:09.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.572+0000 7f8255800640 1 --2- 172.21.15.131:0/763354199 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f82340051c0 0x7f825015a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:09.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.572+0000 7f8255800640 1 -- 172.21.15.131:0/763354199 >> 172.21.15.131:0/763354199 conn(0x7f82500cff40 msgr2=0x7f82500b4db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:09.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.572+0000 7f8255800640 1 -- 172.21.15.131:0/763354199 shutdown_connections 2024-05-26T22:49:09.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.572+0000 7f8255800640 1 -- 172.21.15.131:0/763354199 wait complete. 2024-05-26T22:49:09.591 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-26T22:48:10.463339+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-26T22:48:19.663254+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-26T22:48:20.235751+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-26T22:49:09.591 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-26T22:49:09.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.797+0000 7f0825000640 1 Processor -- start 2024-05-26T22:49:09.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.798+0000 7f0825000640 1 -- start start 2024-05-26T22:49:09.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.799+0000 7f0825000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0820155770 0x7f082015b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:09.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.799+0000 7f0825000640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f082006e440 0x7f08200725a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:09.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.799+0000 7f0825000640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f082015c1c0 con 0x7f082015bc40 2024-05-26T22:49:09.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.800+0000 7f0825000640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f082015c4a0 con 0x7f082006e440 2024-05-26T22:49:09.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.800+0000 7f0825000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f0820071da0 con 0x7f0820155770 2024-05-26T22:49:09.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.800+0000 7f081d400640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f082015bc40 0x7f082006dd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:46670/0 (socket says 172.21.15.131:46670) 2024-05-26T22:49:09.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.800+0000 7f081d400640 1 -- 172.21.15.131:0/582377766 learned_addr learned my addr 172.21.15.131:0/582377766 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:49:09.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.800+0000 7f081e800640 1 --2- 172.21.15.131:0/582377766 >> v2:172.21.15.131:3300/0 conn(0x7f082006e440 0x7f08200725a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:09.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.800+0000 7f081de00640 1 --2- 172.21.15.131:0/582377766 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0820155770 0x7f082015b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:09.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.800+0000 7f081ca00640 1 -- 172.21.15.131:0/582377766 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3249444583 0 0) 0x7f082015c1c0 con 0x7f082015bc40 2024-05-26T22:49:09.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.800+0000 7f081ca00640 1 -- 172.21.15.131:0/582377766 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0800003520 con 0x7f082015bc40 2024-05-26T22:49:09.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.800+0000 7f081e800640 1 -- 172.21.15.131:0/582377766 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0820155770 msgr2=0x7f082015b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:09.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.800+0000 7f081e800640 1 --2- 172.21.15.131:0/582377766 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0820155770 0x7f082015b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:09.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.800+0000 7f081e800640 1 -- 172.21.15.131:0/582377766 >> v1:172.21.15.131:6789/0 conn(0x7f082015bc40 legacy=0x7f082006dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:09.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.800+0000 7f081e800640 1 -- 172.21.15.131:0/582377766 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0820072080 con 0x7f082006e440 2024-05-26T22:49:09.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.802+0000 7f081e800640 1 --2- 172.21.15.131:0/582377766 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f082006e440 0x7f08200725a0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f08180097a0 tx=0x7f081802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53c6d7e8f7f800e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:09.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.802+0000 7f081ca00640 1 -- 172.21.15.131:0/582377766 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0818037070 con 0x7f082006e440 2024-05-26T22:49:09.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.802+0000 7f081ca00640 1 -- 172.21.15.131:0/582377766 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0818032440 con 0x7f082006e440 2024-05-26T22:49:09.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.802+0000 7f081ca00640 1 -- 172.21.15.131:0/582377766 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0818032810 con 0x7f082006e440 2024-05-26T22:49:09.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.802+0000 7f0825000640 1 -- 172.21.15.131:0/582377766 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f082006e440 msgr2=0x7f08200725a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:09.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.802+0000 7f0825000640 1 --2- 172.21.15.131:0/582377766 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f082006e440 0x7f08200725a0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f08180097a0 tx=0x7f081802a210 comp rx=0 tx=0).stop 2024-05-26T22:49:09.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.803+0000 7f0825000640 1 -- 172.21.15.131:0/582377766 shutdown_connections 2024-05-26T22:49:09.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.803+0000 7f0825000640 1 --2- 172.21.15.131:0/582377766 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f082006e440 0x7f08200725a0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:09.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.803+0000 7f0825000640 1 --2- 172.21.15.131:0/582377766 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0820155770 0x7f082015b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:09.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.803+0000 7f0825000640 1 -- 172.21.15.131:0/582377766 >> 172.21.15.131:0/582377766 conn(0x7f082006bc90 msgr2=0x7f08200b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:09.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.803+0000 7f0825000640 1 -- 172.21.15.131:0/582377766 shutdown_connections 2024-05-26T22:49:09.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.803+0000 7f0825000640 1 -- 172.21.15.131:0/582377766 wait complete. 2024-05-26T22:49:09.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.804+0000 7f0825000640 1 Processor -- start 2024-05-26T22:49:09.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.804+0000 7f0825000640 1 -- start start 2024-05-26T22:49:09.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.805+0000 7f0825000640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f082006e440 0x7f082014a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:09.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.805+0000 7f0825000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0820155770 0x7f0820141540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:09.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.805+0000 7f0825000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f082015bc40 0x7f082014e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:09.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.805+0000 7f0825000640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f082009de00 con 0x7f0820155770 2024-05-26T22:49:09.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.805+0000 7f0825000640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f082015c8c0 con 0x7f082006e440 2024-05-26T22:49:09.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.805+0000 7f0825000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f082009df80 con 0x7f082015bc40 2024-05-26T22:49:09.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.806+0000 7f081d400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0820155770 0x7f0820141540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:09.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.806+0000 7f081d400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0820155770 0x7f0820141540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41922/0 (socket says 172.21.15.131:41922) 2024-05-26T22:49:09.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.806+0000 7f081d400640 1 -- 172.21.15.131:0/1492038308 learned_addr learned my addr 172.21.15.131:0/1492038308 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:09.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.806+0000 7f081de00640 1 --2- 172.21.15.131:0/1492038308 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f082006e440 0x7f082014a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:09.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.806+0000 7f081e800640 1 --2- 172.21.15.131:0/1492038308 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f082015bc40 0x7f082014e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:09.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.806+0000 7f081d400640 1 -- 172.21.15.131:0/1492038308 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f082015bc40 msgr2=0x7f082014e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:09.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.806+0000 7f081d400640 1 --2- 172.21.15.131:0/1492038308 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f082015bc40 0x7f082014e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:09.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.806+0000 7f081d400640 1 -- 172.21.15.131:0/1492038308 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f082006e440 msgr2=0x7f082014a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:09.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.806+0000 7f081d400640 1 --2- 172.21.15.131:0/1492038308 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f082006e440 0x7f082014a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:09.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.806+0000 7f081d400640 1 -- 172.21.15.131:0/1492038308 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f082009d7f0 con 0x7f0820155770 2024-05-26T22:49:09.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.807+0000 7f081d400640 1 --2- 172.21.15.131:0/1492038308 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0820155770 0x7f0820141540 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f080800cad0 tx=0x7f080800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:09.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.807+0000 7f0816000640 1 -- 172.21.15.131:0/1492038308 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0808017070 con 0x7f0820155770 2024-05-26T22:49:09.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.807+0000 7f0816000640 1 -- 172.21.15.131:0/1492038308 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f080800ac30 con 0x7f0820155770 2024-05-26T22:49:09.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.807+0000 7f0816000640 1 -- 172.21.15.131:0/1492038308 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0808012400 con 0x7f0820155770 2024-05-26T22:49:09.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.807+0000 7f0825000640 1 -- 172.21.15.131:0/1492038308 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f082009c750 con 0x7f0820155770 2024-05-26T22:49:09.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.807+0000 7f0825000640 1 -- 172.21.15.131:0/1492038308 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f082009cd10 con 0x7f0820155770 2024-05-26T22:49:09.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.808+0000 7f0825000640 1 -- 172.21.15.131:0/1492038308 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08200695c0 con 0x7f0820155770 2024-05-26T22:49:09.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.809+0000 7f0816000640 1 -- 172.21.15.131:0/1492038308 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f080801a070 con 0x7f0820155770 2024-05-26T22:49:09.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.810+0000 7f0816000640 1 --2- 172.21.15.131:0/1492038308 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f07fc06be30 0x7f07fc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:09.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.810+0000 7f0816000640 1 -- 172.21.15.131:0/1492038308 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f080808acd0 con 0x7f0820155770 2024-05-26T22:49:09.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.810+0000 7f081de00640 1 --2- 172.21.15.131:0/1492038308 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f07fc06be30 0x7f07fc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:09.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.811+0000 7f081de00640 1 --2- 172.21.15.131:0/1492038308 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f07fc06be30 0x7f07fc06e2f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f0810006fd0 tx=0x7f0810008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:09.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:09.814+0000 7f0816000640 1 -- 172.21.15.131:0/1492038308 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f08080595f0 con 0x7f0820155770 2024-05-26T22:49:10.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.021+0000 7f0825000640 1 -- 172.21.15.131:0/1492038308 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f08200adc40 con 0x7f0820155770 2024-05-26T22:49:10.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.021+0000 7f0816000640 1 -- 172.21.15.131:0/1492038308 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7f0808058ef0 con 0x7f0820155770 2024-05-26T22:49:10.021 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2024-05-26T22:49:10.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.024+0000 7f0825000640 1 -- 172.21.15.131:0/1492038308 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f07fc06be30 msgr2=0x7f07fc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:10.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.024+0000 7f0825000640 1 --2- 172.21.15.131:0/1492038308 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f07fc06be30 0x7f07fc06e2f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f0810006fd0 tx=0x7f0810008040 comp rx=0 tx=0).stop 2024-05-26T22:49:10.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.024+0000 7f0825000640 1 -- 172.21.15.131:0/1492038308 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0820155770 msgr2=0x7f0820141540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:10.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.024+0000 7f0825000640 1 --2- 172.21.15.131:0/1492038308 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0820155770 0x7f0820141540 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f080800cad0 tx=0x7f080800cf90 comp rx=0 tx=0).stop 2024-05-26T22:49:10.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.024+0000 7f0825000640 1 -- 172.21.15.131:0/1492038308 shutdown_connections 2024-05-26T22:49:10.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.024+0000 7f0825000640 1 --2- 172.21.15.131:0/1492038308 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f082015bc40 0x7f082014e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:10.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.024+0000 7f0825000640 1 --2- 172.21.15.131:0/1492038308 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f07fc06be30 0x7f07fc06e2f0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:10.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.024+0000 7f0825000640 1 --2- 172.21.15.131:0/1492038308 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0820155770 0x7f0820141540 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:10.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.024+0000 7f0825000640 1 --2- 172.21.15.131:0/1492038308 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f082006e440 0x7f082014a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:10.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.024+0000 7f0825000640 1 -- 172.21.15.131:0/1492038308 >> 172.21.15.131:0/1492038308 conn(0x7f082006bc90 msgr2=0x7f08200b4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:10.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.024+0000 7f0825000640 1 -- 172.21.15.131:0/1492038308 shutdown_connections 2024-05-26T22:49:10.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.024+0000 7f0825000640 1 -- 172.21.15.131:0/1492038308 wait complete. 2024-05-26T22:49:10.043 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-26T22:49:10.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.251+0000 7ff5a0e00640 1 Processor -- start 2024-05-26T22:49:10.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.252+0000 7ff5a0e00640 1 -- start start 2024-05-26T22:49:10.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.253+0000 7ff5a0e00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7ff59c155770 0x7ff59c15b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:10.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.253+0000 7ff5a0e00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff59c15c1c0 con 0x7ff59c06e440 2024-05-26T22:49:10.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.253+0000 7ff5a0e00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff59c15c4a0 con 0x7ff59c15bc40 2024-05-26T22:49:10.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.253+0000 7ff5a0e00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7ff59c071da0 con 0x7ff59c155770 2024-05-26T22:49:10.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.254+0000 7ff59a800640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7ff59c06e440 0x7ff59c0725a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:46680/0 (socket says 172.21.15.131:46680) 2024-05-26T22:49:10.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.254+0000 7ff59a800640 1 -- 172.21.15.131:0/1936797902 learned_addr learned my addr 172.21.15.131:0/1936797902 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:49:10.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.254+0000 7ff599e00640 1 --2- 172.21.15.131:0/1936797902 >> v2:172.21.15.187:3300/0 conn(0x7ff59c155770 0x7ff59c15b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:10.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.254+0000 7ff598a00640 1 -- 172.21.15.131:0/1936797902 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2544606086 0 0) 0x7ff59c15c1c0 con 0x7ff59c06e440 2024-05-26T22:49:10.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.254+0000 7ff598a00640 1 -- 172.21.15.131:0/1936797902 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff580005730 con 0x7ff59c06e440 2024-05-26T22:49:10.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.255+0000 7ff598a00640 1 -- 172.21.15.131:0/1936797902 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 421911359 0 0) 0x7ff580005730 con 0x7ff59c06e440 2024-05-26T22:49:10.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.255+0000 7ff598a00640 1 -- 172.21.15.131:0/1936797902 >> v2:172.21.15.187:3300/0 conn(0x7ff59c155770 msgr2=0x7ff59c15b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:10.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.255+0000 7ff598a00640 1 --2- 172.21.15.131:0/1936797902 >> v2:172.21.15.187:3300/0 conn(0x7ff59c155770 0x7ff59c15b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:10.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.255+0000 7ff598a00640 1 -- 172.21.15.131:0/1936797902 >> v1:172.21.15.187:6789/0 conn(0x7ff59c15bc40 legacy=0x7ff59c06dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:10.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.255+0000 7ff598a00640 1 -- 172.21.15.131:0/1936797902 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff59c072080 con 0x7ff59c06e440 2024-05-26T22:49:10.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.255+0000 7ff598a00640 1 -- 172.21.15.131:0/1936797902 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7ff594001670 con 0x7ff59c06e440 2024-05-26T22:49:10.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.255+0000 7ff598a00640 1 -- 172.21.15.131:0/1936797902 >> v1:172.21.15.131:6789/0 conn(0x7ff59c06e440 legacy=0x7ff59c0725a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:10.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.255+0000 7ff598a00640 1 --2- 172.21.15.131:0/1936797902 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff580005bc0 0x7ff580005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:10.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.255+0000 7ff598a00640 1 --2- 172.21.15.131:0/1936797902 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff580006a10 0x7ff580006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:10.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.255+0000 7ff598a00640 1 --2- 172.21.15.131:0/1936797902 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff5800073d0 0x7ff58000dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:10.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.255+0000 7ff598a00640 1 -- 172.21.15.131:0/1936797902 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff59c071da0 con 0x7ff580005bc0 2024-05-26T22:49:10.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.255+0000 7ff598a00640 1 -- 172.21.15.131:0/1936797902 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff58000e280 con 0x7ff580006a10 2024-05-26T22:49:10.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.255+0000 7ff598a00640 1 -- 172.21.15.131:0/1936797902 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff58000e480 con 0x7ff5800073d0 2024-05-26T22:49:10.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.255+0000 7ff599e00640 1 --2- 172.21.15.131:0/1936797902 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff580005bc0 0x7ff580005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:10.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.255+0000 7ff599e00640 1 -- 172.21.15.131:0/1936797902 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff5800073d0 msgr2=0x7ff58000dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:10.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.255+0000 7ff59a800640 1 --2- 172.21.15.131:0/1936797902 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff5800073d0 0x7ff58000dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:10.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.256+0000 7ff599400640 1 --2- 172.21.15.131:0/1936797902 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff580006a10 0x7ff580006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:10.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.256+0000 7ff599e00640 1 --2- 172.21.15.131:0/1936797902 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff5800073d0 0x7ff58000dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:10.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.256+0000 7ff599e00640 1 -- 172.21.15.131:0/1936797902 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff580006a10 msgr2=0x7ff580006e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:10.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.256+0000 7ff599e00640 1 --2- 172.21.15.131:0/1936797902 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff580006a10 0x7ff580006e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:10.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.256+0000 7ff599e00640 1 -- 172.21.15.131:0/1936797902 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff580005730 con 0x7ff580005bc0 2024-05-26T22:49:10.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.256+0000 7ff599400640 1 --2- 172.21.15.131:0/1936797902 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff580006a10 0x7ff580006e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:49:10.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.256+0000 7ff59a800640 1 --2- 172.21.15.131:0/1936797902 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff5800073d0 0x7ff58000dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:49:10.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.256+0000 7ff59a800640 1 -- 172.21.15.131:0/1936797902 reap_dead start 2024-05-26T22:49:10.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.257+0000 7ff599e00640 1 --2- 172.21.15.131:0/1936797902 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff580005bc0 0x7ff580005fe0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff58c009fc0 tx=0x7ff580007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6e18d2648715001 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:10.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.257+0000 7ff598a00640 1 -- 172.21.15.131:0/1936797902 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff58c009c10 con 0x7ff580005bc0 2024-05-26T22:49:10.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.257+0000 7ff598a00640 1 -- 172.21.15.131:0/1936797902 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff58c036070 con 0x7ff580005bc0 2024-05-26T22:49:10.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.257+0000 7ff598a00640 1 -- 172.21.15.131:0/1936797902 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff58c031880 con 0x7ff580005bc0 2024-05-26T22:49:10.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.257+0000 7ff5a0e00640 1 -- 172.21.15.131:0/1936797902 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff580005bc0 msgr2=0x7ff580005fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:10.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.257+0000 7ff5a0e00640 1 --2- 172.21.15.131:0/1936797902 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff580005bc0 0x7ff580005fe0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff58c009fc0 tx=0x7ff580007a40 comp rx=0 tx=0).stop 2024-05-26T22:49:10.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.258+0000 7ff5a0e00640 1 -- 172.21.15.131:0/1936797902 shutdown_connections 2024-05-26T22:49:10.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.258+0000 7ff5a0e00640 1 --2- 172.21.15.131:0/1936797902 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff580005bc0 0x7ff580005fe0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:10.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.258+0000 7ff5a0e00640 1 -- 172.21.15.131:0/1936797902 >> 172.21.15.131:0/1936797902 conn(0x7ff59c06bc90 msgr2=0x7ff59c0b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:10.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.258+0000 7ff5a0e00640 1 -- 172.21.15.131:0/1936797902 shutdown_connections 2024-05-26T22:49:10.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.258+0000 7ff5a0e00640 1 -- 172.21.15.131:0/1936797902 wait complete. 2024-05-26T22:49:10.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.259+0000 7ff5a0e00640 1 Processor -- start 2024-05-26T22:49:10.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.260+0000 7ff5a0e00640 1 -- start start 2024-05-26T22:49:10.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.261+0000 7ff5a0e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff580005bc0 0x7ff59c0afb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:10.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.261+0000 7ff5a0e00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff59c0b00c0 0x7ff59c14f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:10.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.261+0000 7ff5a0e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff59c14f5a0 0x7ff59c151250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:10.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.261+0000 7ff5a0e00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff59c09de00 con 0x7ff580005bc0 2024-05-26T22:49:10.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.261+0000 7ff5a0e00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff59c15c8c0 con 0x7ff59c0b00c0 2024-05-26T22:49:10.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.261+0000 7ff5a0e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff59c09df80 con 0x7ff59c14f5a0 2024-05-26T22:49:10.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.261+0000 7ff599e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff580005bc0 0x7ff59c0afb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:10.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.261+0000 7ff599e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff580005bc0 0x7ff59c0afb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41934/0 (socket says 172.21.15.131:41934) 2024-05-26T22:49:10.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.261+0000 7ff599e00640 1 -- 172.21.15.131:0/3126125974 learned_addr learned my addr 172.21.15.131:0/3126125974 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:10.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.261+0000 7ff59a800640 1 --2- 172.21.15.131:0/3126125974 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff59c14f5a0 0x7ff59c151250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:10.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.261+0000 7ff599e00640 1 -- 172.21.15.131:0/3126125974 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff59c14f5a0 msgr2=0x7ff59c151250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:10.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.261+0000 7ff599400640 1 --2- 172.21.15.131:0/3126125974 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff59c0b00c0 0x7ff59c14f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:10.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.261+0000 7ff599e00640 1 --2- 172.21.15.131:0/3126125974 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff59c14f5a0 0x7ff59c151250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:10.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.261+0000 7ff599e00640 1 -- 172.21.15.131:0/3126125974 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff59c0b00c0 msgr2=0x7ff59c14f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:10.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.261+0000 7ff599e00640 1 --2- 172.21.15.131:0/3126125974 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff59c0b00c0 0x7ff59c14f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:10.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.261+0000 7ff599e00640 1 -- 172.21.15.131:0/3126125974 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff59c151fd0 con 0x7ff580005bc0 2024-05-26T22:49:10.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.261+0000 7ff59a800640 1 --2- 172.21.15.131:0/3126125974 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff59c14f5a0 0x7ff59c151250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:49:10.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.262+0000 7ff599e00640 1 --2- 172.21.15.131:0/3126125974 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff580005bc0 0x7ff59c0afb80 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ff58c033040 tx=0x7ff58c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:10.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.262+0000 7ff592000640 1 -- 172.21.15.131:0/3126125974 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff58c009c10 con 0x7ff580005bc0 2024-05-26T22:49:10.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.262+0000 7ff592000640 1 -- 172.21.15.131:0/3126125974 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff58c003990 con 0x7ff580005bc0 2024-05-26T22:49:10.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.262+0000 7ff5a0e00640 1 -- 172.21.15.131:0/3126125974 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff59c1522b0 con 0x7ff580005bc0 2024-05-26T22:49:10.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.262+0000 7ff592000640 1 -- 172.21.15.131:0/3126125974 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff58c003d80 con 0x7ff580005bc0 2024-05-26T22:49:10.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.262+0000 7ff5a0e00640 1 -- 172.21.15.131:0/3126125974 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff59c152870 con 0x7ff580005bc0 2024-05-26T22:49:10.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.263+0000 7ff5a0e00640 1 -- 172.21.15.131:0/3126125974 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff59c0695c0 con 0x7ff580005bc0 2024-05-26T22:49:10.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.264+0000 7ff592000640 1 -- 172.21.15.131:0/3126125974 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff58c005d10 con 0x7ff580005bc0 2024-05-26T22:49:10.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.265+0000 7ff592000640 1 --2- 172.21.15.131:0/3126125974 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ff57806be30 0x7ff57806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:10.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.265+0000 7ff592000640 1 -- 172.21.15.131:0/3126125974 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7ff58c0ac270 con 0x7ff580005bc0 2024-05-26T22:49:10.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.265+0000 7ff599400640 1 --2- 172.21.15.131:0/3126125974 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ff57806be30 0x7ff57806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:10.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.266+0000 7ff599400640 1 --2- 172.21.15.131:0/3126125974 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ff57806be30 0x7ff57806e2f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff584000c00 tx=0x7ff584009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:10.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.269+0000 7ff592000640 1 -- 172.21.15.131:0/3126125974 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7ff58c07ab00 con 0x7ff580005bc0 2024-05-26T22:49:10.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.472+0000 7ff5a0e00640 1 -- 172.21.15.131:0/3126125974 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff59c06d6f0 con 0x7ff580005bc0 2024-05-26T22:49:10.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.472+0000 7ff592000640 1 -- 172.21.15.131:0/3126125974 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7ff58c07a400 con 0x7ff580005bc0 2024-05-26T22:49:10.473 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-05-26T22:49:10.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.475+0000 7ff5a0e00640 1 -- 172.21.15.131:0/3126125974 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ff57806be30 msgr2=0x7ff57806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:10.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.475+0000 7ff5a0e00640 1 --2- 172.21.15.131:0/3126125974 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ff57806be30 0x7ff57806e2f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff584000c00 tx=0x7ff584009040 comp rx=0 tx=0).stop 2024-05-26T22:49:10.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.475+0000 7ff5a0e00640 1 -- 172.21.15.131:0/3126125974 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff580005bc0 msgr2=0x7ff59c0afb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:10.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.475+0000 7ff5a0e00640 1 --2- 172.21.15.131:0/3126125974 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff580005bc0 0x7ff59c0afb80 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ff58c033040 tx=0x7ff58c009750 comp rx=0 tx=0).stop 2024-05-26T22:49:10.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.475+0000 7ff5a0e00640 1 -- 172.21.15.131:0/3126125974 shutdown_connections 2024-05-26T22:49:10.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.475+0000 7ff5a0e00640 1 --2- 172.21.15.131:0/3126125974 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff59c14f5a0 0x7ff59c151250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:10.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.475+0000 7ff5a0e00640 1 --2- 172.21.15.131:0/3126125974 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ff57806be30 0x7ff57806e2f0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:10.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.475+0000 7ff5a0e00640 1 --2- 172.21.15.131:0/3126125974 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff59c0b00c0 0x7ff59c14f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:10.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.475+0000 7ff5a0e00640 1 --2- 172.21.15.131:0/3126125974 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff580005bc0 0x7ff59c0afb80 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:10.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.475+0000 7ff5a0e00640 1 -- 172.21.15.131:0/3126125974 >> 172.21.15.131:0/3126125974 conn(0x7ff59c06bc90 msgr2=0x7ff59c15a1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:10.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.476+0000 7ff5a0e00640 1 -- 172.21.15.131:0/3126125974 shutdown_connections 2024-05-26T22:49:10.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.476+0000 7ff5a0e00640 1 -- 172.21.15.131:0/3126125974 wait complete. 2024-05-26T22:49:10.494 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-26T22:49:10.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.690+0000 7f61afe00640 1 Processor -- start 2024-05-26T22:49:10.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.690+0000 7f61afe00640 1 -- start start 2024-05-26T22:49:10.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.692+0000 7f61afe00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f61b01410d0 0x7f61b0165dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:10.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.692+0000 7f61afe00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61b013e820 con 0x7f61b015b730 2024-05-26T22:49:10.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.692+0000 7f61afe00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61b015c4a0 con 0x7f61b006d250 2024-05-26T22:49:10.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.692+0000 7f61afe00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f61b0071260 con 0x7f61b01410d0 2024-05-26T22:49:10.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.692+0000 7f61a6000640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f61b015b730 0x7f61b015bbc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:46684/0 (socket says 172.21.15.131:46684) 2024-05-26T22:49:10.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.692+0000 7f61a6000640 1 -- 172.21.15.131:0/353440461 learned_addr learned my addr 172.21.15.131:0/353440461 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:49:10.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.692+0000 7f61ae000640 1 -- 172.21.15.131:0/353440461 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3519236863 0 0) 0x7f61b013e820 con 0x7f61b015b730 2024-05-26T22:49:10.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.692+0000 7f61ae000640 1 -- 172.21.15.131:0/353440461 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6194003520 con 0x7f61b015b730 2024-05-26T22:49:10.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.693+0000 7f61af400640 1 --2- 172.21.15.131:0/353440461 >> v2:172.21.15.187:3300/0 conn(0x7f61b01410d0 0x7f61b0165dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:10.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.693+0000 7f61ae000640 1 -- 172.21.15.131:0/353440461 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3043690792 0 0) 0x7f6194003520 con 0x7f61b015b730 2024-05-26T22:49:10.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.693+0000 7f61ae000640 1 -- 172.21.15.131:0/353440461 >> v2:172.21.15.187:3300/0 conn(0x7f61b01410d0 msgr2=0x7f61b0165dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:10.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.693+0000 7f61ae000640 1 --2- 172.21.15.131:0/353440461 >> v2:172.21.15.187:3300/0 conn(0x7f61b01410d0 0x7f61b0165dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:10.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.693+0000 7f61ae000640 1 -- 172.21.15.131:0/353440461 >> v1:172.21.15.187:6789/0 conn(0x7f61b006d250 legacy=0x7f61b006d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:10.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.693+0000 7f61ae000640 1 -- 172.21.15.131:0/353440461 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61b00714b0 con 0x7f61b015b730 2024-05-26T22:49:10.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.693+0000 7f61ae000640 1 -- 172.21.15.131:0/353440461 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7f6198001670 con 0x7f61b015b730 2024-05-26T22:49:10.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.693+0000 7f61ae000640 1 -- 172.21.15.131:0/353440461 >> v1:172.21.15.131:6789/0 conn(0x7f61b015b730 legacy=0x7f61b015bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:10.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.693+0000 7f61ae000640 1 --2- 172.21.15.131:0/353440461 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6194003a30 0x7f6194003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:10.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.693+0000 7f61ae000640 1 --2- 172.21.15.131:0/353440461 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6194004800 0x7f6194004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:10.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.693+0000 7f61ae000640 1 --2- 172.21.15.131:0/353440461 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f61940052e0 0x7f619400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:10.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.693+0000 7f61ae000640 1 -- 172.21.15.131:0/353440461 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f61b0071260 con 0x7f6194004800 2024-05-26T22:49:10.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.693+0000 7f61ae000640 1 -- 172.21.15.131:0/353440461 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f619400c1b0 con 0x7f6194003a30 2024-05-26T22:49:10.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.693+0000 7f61ae000640 1 -- 172.21.15.131:0/353440461 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f619400c3b0 con 0x7f61940052e0 2024-05-26T22:49:10.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.693+0000 7f61a6000640 1 --2- 172.21.15.131:0/353440461 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6194004800 0x7f6194004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:10.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.694+0000 7f61aea00640 1 --2- 172.21.15.131:0/353440461 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6194003a30 0x7f6194003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:10.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.694+0000 7f61af400640 1 --2- 172.21.15.131:0/353440461 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f61940052e0 0x7f619400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:10.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.694+0000 7f61a6000640 1 -- 172.21.15.131:0/353440461 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f61940052e0 msgr2=0x7f619400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:10.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.694+0000 7f61a6000640 1 --2- 172.21.15.131:0/353440461 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f61940052e0 0x7f619400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:10.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.694+0000 7f61a6000640 1 -- 172.21.15.131:0/353440461 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6194003a30 msgr2=0x7f6194003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:10.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.694+0000 7f61a6000640 1 --2- 172.21.15.131:0/353440461 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6194003a30 0x7f6194003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:10.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.694+0000 7f61a6000640 1 -- 172.21.15.131:0/353440461 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6194003520 con 0x7f6194004800 2024-05-26T22:49:10.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.694+0000 7f61af400640 1 --2- 172.21.15.131:0/353440461 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f61940052e0 0x7f619400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:49:10.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.694+0000 7f61aea00640 1 --2- 172.21.15.131:0/353440461 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6194003a30 0x7f6194003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-26T22:49:10.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.694+0000 7f61af400640 1 -- 172.21.15.131:0/353440461 reap_dead start 2024-05-26T22:49:10.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.695+0000 7f61a6000640 1 --2- 172.21.15.131:0/353440461 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6194004800 0x7f6194004c80 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f6198007fd0 tx=0x7f6194005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=962ae294da5a23a6 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:10.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.695+0000 7f61ae000640 1 -- 172.21.15.131:0/353440461 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6198038070 con 0x7f6194004800 2024-05-26T22:49:10.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.695+0000 7f61ae000640 1 -- 172.21.15.131:0/353440461 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6198005af0 con 0x7f6194004800 2024-05-26T22:49:10.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.695+0000 7f61ae000640 1 -- 172.21.15.131:0/353440461 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6198007c20 con 0x7f6194004800 2024-05-26T22:49:10.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.696+0000 7f61afe00640 1 -- 172.21.15.131:0/353440461 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6194004800 msgr2=0x7f6194004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:10.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.696+0000 7f61afe00640 1 --2- 172.21.15.131:0/353440461 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6194004800 0x7f6194004c80 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f6198007fd0 tx=0x7f6194005830 comp rx=0 tx=0).stop 2024-05-26T22:49:10.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.696+0000 7f61afe00640 1 -- 172.21.15.131:0/353440461 shutdown_connections 2024-05-26T22:49:10.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.696+0000 7f61afe00640 1 --2- 172.21.15.131:0/353440461 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6194004800 0x7f6194004c80 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:10.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.696+0000 7f61afe00640 1 -- 172.21.15.131:0/353440461 >> 172.21.15.131:0/353440461 conn(0x7f61b00b5660 msgr2=0x7f61b00b5a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:10.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.696+0000 7f61afe00640 1 -- 172.21.15.131:0/353440461 shutdown_connections 2024-05-26T22:49:10.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.696+0000 7f61afe00640 1 -- 172.21.15.131:0/353440461 wait complete. 2024-05-26T22:49:10.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.698+0000 7f61afe00640 1 Processor -- start 2024-05-26T22:49:10.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.698+0000 7f61afe00640 1 -- start start 2024-05-26T22:49:10.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.699+0000 7f61afe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6194004800 0x7f61b0156330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:10.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.699+0000 7f61afe00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f61b013d230 0x7f61b00cfe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:10.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.699+0000 7f61afe00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f61b00d0370 0x7f61b00b3d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:10.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.699+0000 7f61afe00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f61b0141500 con 0x7f6194004800 2024-05-26T22:49:10.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.699+0000 7f61afe00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f61b0071a70 con 0x7f61b013d230 2024-05-26T22:49:10.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.699+0000 7f61afe00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f61b0141680 con 0x7f61b00d0370 2024-05-26T22:49:10.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.699+0000 7f61aea00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6194004800 0x7f61b0156330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:10.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.699+0000 7f61aea00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6194004800 0x7f61b0156330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41948/0 (socket says 172.21.15.131:41948) 2024-05-26T22:49:10.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.699+0000 7f61aea00640 1 -- 172.21.15.131:0/1310433568 learned_addr learned my addr 172.21.15.131:0/1310433568 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:10.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.699+0000 7f61a6000640 1 --2- 172.21.15.131:0/1310433568 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f61b013d230 0x7f61b00cfe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:10.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.700+0000 7f61af400640 1 --2- 172.21.15.131:0/1310433568 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f61b00d0370 0x7f61b00b3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:10.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.700+0000 7f61aea00640 1 -- 172.21.15.131:0/1310433568 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f61b00d0370 msgr2=0x7f61b00b3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:10.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.700+0000 7f61aea00640 1 --2- 172.21.15.131:0/1310433568 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f61b00d0370 0x7f61b00b3d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:10.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.700+0000 7f61aea00640 1 -- 172.21.15.131:0/1310433568 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f61b013d230 msgr2=0x7f61b00cfe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:10.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.700+0000 7f61aea00640 1 --2- 172.21.15.131:0/1310433568 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f61b013d230 0x7f61b00cfe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:10.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.700+0000 7f61aea00640 1 -- 172.21.15.131:0/1310433568 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61b013d6b0 con 0x7f6194004800 2024-05-26T22:49:10.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.700+0000 7f61aea00640 1 --2- 172.21.15.131:0/1310433568 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6194004800 0x7f61b0156330 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f61a000e630 tx=0x7f61a000eaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:10.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.700+0000 7f61a7400640 1 -- 172.21.15.131:0/1310433568 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61a00052e0 con 0x7f6194004800 2024-05-26T22:49:10.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.700+0000 7f61a7400640 1 -- 172.21.15.131:0/1310433568 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61a0018070 con 0x7f6194004800 2024-05-26T22:49:10.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.700+0000 7f61a7400640 1 -- 172.21.15.131:0/1310433568 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61a0003810 con 0x7f6194004800 2024-05-26T22:49:10.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.700+0000 7f61afe00640 1 -- 172.21.15.131:0/1310433568 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61b00d0980 con 0x7f6194004800 2024-05-26T22:49:10.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.700+0000 7f61afe00640 1 -- 172.21.15.131:0/1310433568 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61b0155850 con 0x7f6194004800 2024-05-26T22:49:10.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.702+0000 7f61afe00640 1 -- 172.21.15.131:0/1310433568 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61b00694e0 con 0x7f6194004800 2024-05-26T22:49:10.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.702+0000 7f61a7400640 1 -- 172.21.15.131:0/1310433568 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f61a00039b0 con 0x7f6194004800 2024-05-26T22:49:10.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.703+0000 7f61a7400640 1 --2- 172.21.15.131:0/1310433568 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f618c06be60 0x7f618c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:10.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.703+0000 7f61a7400640 1 -- 172.21.15.131:0/1310433568 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f61a008abe0 con 0x7f6194004800 2024-05-26T22:49:10.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.704+0000 7f61a6000640 1 --2- 172.21.15.131:0/1310433568 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f618c06be60 0x7f618c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:10.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.704+0000 7f61a6000640 1 --2- 172.21.15.131:0/1310433568 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f618c06be60 0x7f618c06e320 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6198035000 tx=0x7f6198009030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:10.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.708+0000 7f61a7400640 1 -- 172.21.15.131:0/1310433568 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f61a0059510 con 0x7f6194004800 2024-05-26T22:49:10.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.910+0000 7f61afe00640 1 -- 172.21.15.131:0/1310433568 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f61b00af9b0 con 0x7f6194004800 2024-05-26T22:49:10.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.911+0000 7f61a7400640 1 -- 172.21.15.131:0/1310433568 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7f61a001be40 con 0x7f6194004800 2024-05-26T22:49:10.910 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 64 2024-05-26T22:49:10.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.913+0000 7f61afe00640 1 -- 172.21.15.131:0/1310433568 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f618c06be60 msgr2=0x7f618c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:10.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.913+0000 7f61afe00640 1 --2- 172.21.15.131:0/1310433568 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f618c06be60 0x7f618c06e320 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6198035000 tx=0x7f6198009030 comp rx=0 tx=0).stop 2024-05-26T22:49:10.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.913+0000 7f61afe00640 1 -- 172.21.15.131:0/1310433568 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6194004800 msgr2=0x7f61b0156330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:10.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.913+0000 7f61afe00640 1 --2- 172.21.15.131:0/1310433568 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6194004800 0x7f61b0156330 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f61a000e630 tx=0x7f61a000eaf0 comp rx=0 tx=0).stop 2024-05-26T22:49:10.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.913+0000 7f61afe00640 1 -- 172.21.15.131:0/1310433568 shutdown_connections 2024-05-26T22:49:10.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.913+0000 7f61afe00640 1 --2- 172.21.15.131:0/1310433568 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f61b00d0370 0x7f61b00b3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:10.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.913+0000 7f61afe00640 1 --2- 172.21.15.131:0/1310433568 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f618c06be60 0x7f618c06e320 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:10.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.913+0000 7f61afe00640 1 --2- 172.21.15.131:0/1310433568 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f61b013d230 0x7f61b00cfe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:10.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.913+0000 7f61afe00640 1 --2- 172.21.15.131:0/1310433568 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6194004800 0x7f61b0156330 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:10.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.913+0000 7f61afe00640 1 -- 172.21.15.131:0/1310433568 >> 172.21.15.131:0/1310433568 conn(0x7f61b00b5660 msgr2=0x7f61b0071e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:10.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.913+0000 7f61afe00640 1 -- 172.21.15.131:0/1310433568 shutdown_connections 2024-05-26T22:49:10.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:10.913+0000 7f61afe00640 1 -- 172.21.15.131:0/1310433568 wait complete. 2024-05-26T22:49:10.932 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-26T22:49:11.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.134+0000 7fa8dba00640 1 Processor -- start 2024-05-26T22:49:11.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.135+0000 7fa8dba00640 1 -- start start 2024-05-26T22:49:11.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.136+0000 7fa8dba00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa8d406b770 0x7fa8d4072c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:11.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.136+0000 7fa8dba00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8d40cf770 con 0x7fa8d406b310 2024-05-26T22:49:11.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.136+0000 7fa8dba00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8d415e990 con 0x7fa8d415e5b0 2024-05-26T22:49:11.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.136+0000 7fa8dba00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa8d40ce8b0 con 0x7fa8d406b770 2024-05-26T22:49:11.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.137+0000 7fa8d3e00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fa8d406b310 0x7fa8d4072550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:46694/0 (socket says 172.21.15.131:46694) 2024-05-26T22:49:11.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.137+0000 7fa8d3e00640 1 -- 172.21.15.131:0/1078386338 learned_addr learned my addr 172.21.15.131:0/1078386338 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:49:11.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.137+0000 7fa8d9a00640 1 --2- 172.21.15.131:0/1078386338 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa8d406b770 0x7fa8d4072c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:11.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.137+0000 7fa8d3400640 1 -- 172.21.15.131:0/1078386338 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1790757114 0 0) 0x7fa8c0001110 con 0x7fa8d406b310 2024-05-26T22:49:11.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.137+0000 7fa8d3400640 1 -- 172.21.15.131:0/1078386338 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa8bc003520 con 0x7fa8d406b310 2024-05-26T22:49:11.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.137+0000 7fa8d3400640 1 -- 172.21.15.131:0/1078386338 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1635525859 0 0) 0x7fa8bc003520 con 0x7fa8d406b310 2024-05-26T22:49:11.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.137+0000 7fa8d3400640 1 -- 172.21.15.131:0/1078386338 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa8d406b770 msgr2=0x7fa8d4072c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:11.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.137+0000 7fa8d3400640 1 --2- 172.21.15.131:0/1078386338 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa8d406b770 0x7fa8d4072c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:11.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.137+0000 7fa8d3400640 1 -- 172.21.15.131:0/1078386338 >> v1:172.21.15.187:6789/0 conn(0x7fa8d415e5b0 legacy=0x7fa8d406e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:11.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.137+0000 7fa8d3400640 1 -- 172.21.15.131:0/1078386338 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8d40cdca0 con 0x7fa8d406b310 2024-05-26T22:49:11.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.138+0000 7fa8d3400640 1 -- 172.21.15.131:0/1078386338 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7fa8c0001840 con 0x7fa8d406b310 2024-05-26T22:49:11.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.138+0000 7fa8d3400640 1 -- 172.21.15.131:0/1078386338 >> v1:172.21.15.131:6789/0 conn(0x7fa8d406b310 legacy=0x7fa8d4072550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:11.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.138+0000 7fa8d3400640 1 --2- 172.21.15.131:0/1078386338 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa8d406b770 0x7fa8bc0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:11.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.138+0000 7fa8d3400640 1 --2- 172.21.15.131:0/1078386338 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8bc003f60 0x7fa8bc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:11.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.138+0000 7fa8d3400640 1 --2- 172.21.15.131:0/1078386338 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa8bc004d40 0x7fa8bc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:11.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.138+0000 7fa8d3400640 1 -- 172.21.15.131:0/1078386338 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d40ce8b0 con 0x7fa8bc003f60 2024-05-26T22:49:11.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.138+0000 7fa8d3400640 1 -- 172.21.15.131:0/1078386338 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa8bc00baa0 con 0x7fa8bc004d40 2024-05-26T22:49:11.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.138+0000 7fa8d3400640 1 -- 172.21.15.131:0/1078386338 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa8bc00bca0 con 0x7fa8d406b770 2024-05-26T22:49:11.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.138+0000 7fa8d3e00640 1 --2- 172.21.15.131:0/1078386338 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8bc003f60 0x7fa8bc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:11.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.138+0000 7fa8d9000640 1 --2- 172.21.15.131:0/1078386338 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa8d406b770 0x7fa8bc0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:11.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.138+0000 7fa8d3e00640 1 -- 172.21.15.131:0/1078386338 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa8d406b770 msgr2=0x7fa8bc0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:11.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.138+0000 7fa8d9a00640 1 --2- 172.21.15.131:0/1078386338 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa8bc004d40 0x7fa8bc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:11.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.138+0000 7fa8d3e00640 1 --2- 172.21.15.131:0/1078386338 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa8d406b770 0x7fa8bc0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:11.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.138+0000 7fa8d3e00640 1 -- 172.21.15.131:0/1078386338 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa8bc004d40 msgr2=0x7fa8bc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:11.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.138+0000 7fa8d3e00640 1 --2- 172.21.15.131:0/1078386338 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa8bc004d40 0x7fa8bc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:11.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.139+0000 7fa8d3e00640 1 -- 172.21.15.131:0/1078386338 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8bc003520 con 0x7fa8bc003f60 2024-05-26T22:49:11.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.139+0000 7fa8d9000640 1 --2- 172.21.15.131:0/1078386338 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa8d406b770 0x7fa8bc0039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:49:11.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.139+0000 7fa8d9a00640 1 --2- 172.21.15.131:0/1078386338 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa8bc004d40 0x7fa8bc0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:49:11.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.140+0000 7fa8d3e00640 1 --2- 172.21.15.131:0/1078386338 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8bc003f60 0x7fa8bc004800 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fa8c000c4c0 tx=0x7fa8bc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=812c52f90547b705 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:11.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.140+0000 7fa8d3400640 1 -- 172.21.15.131:0/1078386338 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c000c0e0 con 0x7fa8bc003f60 2024-05-26T22:49:11.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.140+0000 7fa8d3400640 1 -- 172.21.15.131:0/1078386338 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8c0034070 con 0x7fa8bc003f60 2024-05-26T22:49:11.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.140+0000 7fa8d3400640 1 -- 172.21.15.131:0/1078386338 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c0005d70 con 0x7fa8bc003f60 2024-05-26T22:49:11.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.140+0000 7fa8dba00640 1 -- 172.21.15.131:0/1078386338 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8bc003f60 msgr2=0x7fa8bc004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:11.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.140+0000 7fa8dba00640 1 --2- 172.21.15.131:0/1078386338 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8bc003f60 0x7fa8bc004800 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fa8c000c4c0 tx=0x7fa8bc005830 comp rx=0 tx=0).stop 2024-05-26T22:49:11.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.140+0000 7fa8d9a00640 1 -- 172.21.15.131:0/1078386338 reap_dead start 2024-05-26T22:49:11.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.141+0000 7fa8dba00640 1 -- 172.21.15.131:0/1078386338 shutdown_connections 2024-05-26T22:49:11.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.141+0000 7fa8dba00640 1 -- 172.21.15.131:0/1078386338 >> 172.21.15.131:0/1078386338 conn(0x7fa8d40b2d00 msgr2=0x7fa8d40b3110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:11.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.141+0000 7fa8dba00640 1 -- 172.21.15.131:0/1078386338 shutdown_connections 2024-05-26T22:49:11.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.141+0000 7fa8dba00640 1 -- 172.21.15.131:0/1078386338 wait complete. 2024-05-26T22:49:11.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.142+0000 7fa8dba00640 1 Processor -- start 2024-05-26T22:49:11.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.142+0000 7fa8dba00640 1 -- start start 2024-05-26T22:49:11.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.143+0000 7fa8dba00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa8bc003f60 0x7fa8d41f4830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:11.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.143+0000 7fa8dba00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa8d4159890 0x7fa8d41e8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:11.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.143+0000 7fa8dba00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8d41e9510 0x7fa8d41623a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:11.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.143+0000 7fa8dba00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d40ce540 con 0x7fa8d41e9510 2024-05-26T22:49:11.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.143+0000 7fa8dba00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d40ce260 con 0x7fa8bc003f60 2024-05-26T22:49:11.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.143+0000 7fa8dba00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa8d406bc00 con 0x7fa8d4159890 2024-05-26T22:49:11.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.144+0000 7fa8d9a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8d41e9510 0x7fa8d41623a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:11.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.144+0000 7fa8d9a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8d41e9510 0x7fa8d41623a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41974/0 (socket says 172.21.15.131:41974) 2024-05-26T22:49:11.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.144+0000 7fa8d9a00640 1 -- 172.21.15.131:0/1710980739 learned_addr learned my addr 172.21.15.131:0/1710980739 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:11.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.144+0000 7fa8d3e00640 1 --2- 172.21.15.131:0/1710980739 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa8d4159890 0x7fa8d41e8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:11.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.144+0000 7fa8d9000640 1 --2- 172.21.15.131:0/1710980739 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa8bc003f60 0x7fa8d41f4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:11.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.144+0000 7fa8d9a00640 1 -- 172.21.15.131:0/1710980739 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa8d4159890 msgr2=0x7fa8d41e8fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:11.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.144+0000 7fa8d9a00640 1 --2- 172.21.15.131:0/1710980739 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa8d4159890 0x7fa8d41e8fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:11.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.144+0000 7fa8d9a00640 1 -- 172.21.15.131:0/1710980739 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa8bc003f60 msgr2=0x7fa8d41f4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:11.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.144+0000 7fa8d9a00640 1 --2- 172.21.15.131:0/1710980739 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa8bc003f60 0x7fa8d41f4830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:11.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.144+0000 7fa8d9a00640 1 -- 172.21.15.131:0/1710980739 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8d415c100 con 0x7fa8d41e9510 2024-05-26T22:49:11.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.144+0000 7fa8d9000640 1 --2- 172.21.15.131:0/1710980739 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa8bc003f60 0x7fa8d41f4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:49:11.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.144+0000 7fa8d3e00640 1 --2- 172.21.15.131:0/1710980739 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa8d4159890 0x7fa8d41e8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:49:11.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.144+0000 7fa8d9a00640 1 --2- 172.21.15.131:0/1710980739 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8d41e9510 0x7fa8d41623a0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc009fd0 tx=0x7fa8cc00f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:11.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.145+0000 7fa8d0c00640 1 -- 172.21.15.131:0/1710980739 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8cc01c070 con 0x7fa8d41e9510 2024-05-26T22:49:11.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.145+0000 7fa8d0c00640 1 -- 172.21.15.131:0/1710980739 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8cc017400 con 0x7fa8d41e9510 2024-05-26T22:49:11.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.145+0000 7fa8d0c00640 1 -- 172.21.15.131:0/1710980739 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8cc0177d0 con 0x7fa8d41e9510 2024-05-26T22:49:11.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.145+0000 7fa8dba00640 1 -- 172.21.15.131:0/1710980739 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8d415c3e0 con 0x7fa8d41e9510 2024-05-26T22:49:11.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.145+0000 7fa8dba00640 1 -- 172.21.15.131:0/1710980739 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8d415c9a0 con 0x7fa8d41e9510 2024-05-26T22:49:11.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.146+0000 7fa8dba00640 1 -- 172.21.15.131:0/1710980739 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8d40695c0 con 0x7fa8d41e9510 2024-05-26T22:49:11.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.147+0000 7fa8d0c00640 1 -- 172.21.15.131:0/1710980739 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa8cc01f070 con 0x7fa8d41e9510 2024-05-26T22:49:11.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.148+0000 7fa8d0c00640 1 --2- 172.21.15.131:0/1710980739 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa8b406be30 0x7fa8b406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:11.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.148+0000 7fa8d0c00640 1 -- 172.21.15.131:0/1710980739 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fa8cc08b080 con 0x7fa8d41e9510 2024-05-26T22:49:11.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.148+0000 7fa8d9000640 1 --2- 172.21.15.131:0/1710980739 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa8b406be30 0x7fa8b406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:11.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.149+0000 7fa8d9000640 1 --2- 172.21.15.131:0/1710980739 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa8b406be30 0x7fa8b406e2f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa8c80016f0 tx=0x7fa8c8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:11.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.152+0000 7fa8d0c00640 1 -- 172.21.15.131:0/1710980739 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fa8cc012070 con 0x7fa8d41e9510 2024-05-26T22:49:11.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.361+0000 7fa8dba00640 1 -- 172.21.15.131:0/1710980739 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fa8d40b4ba0 con 0x7fa8d41e9510 2024-05-26T22:49:11.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.362+0000 7fa8d0c00640 1 -- 172.21.15.131:0/1710980739 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7fa8cc059910 con 0x7fa8d41e9510 2024-05-26T22:49:11.362 INFO:teuthology.orchestra.run.smithi131.stderr:listed 0 entries 2024-05-26T22:49:11.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.365+0000 7fa8dba00640 1 -- 172.21.15.131:0/1710980739 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa8b406be30 msgr2=0x7fa8b406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:11.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.365+0000 7fa8dba00640 1 --2- 172.21.15.131:0/1710980739 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa8b406be30 0x7fa8b406e2f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa8c80016f0 tx=0x7fa8c8009040 comp rx=0 tx=0).stop 2024-05-26T22:49:11.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.365+0000 7fa8dba00640 1 -- 172.21.15.131:0/1710980739 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8d41e9510 msgr2=0x7fa8d41623a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:11.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.365+0000 7fa8dba00640 1 --2- 172.21.15.131:0/1710980739 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8d41e9510 0x7fa8d41623a0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc009fd0 tx=0x7fa8cc00f4a0 comp rx=0 tx=0).stop 2024-05-26T22:49:11.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.365+0000 7fa8dba00640 1 -- 172.21.15.131:0/1710980739 shutdown_connections 2024-05-26T22:49:11.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.365+0000 7fa8dba00640 1 --2- 172.21.15.131:0/1710980739 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa8d4159890 0x7fa8d41e8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:11.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.365+0000 7fa8dba00640 1 --2- 172.21.15.131:0/1710980739 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa8b406be30 0x7fa8b406e2f0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:11.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.365+0000 7fa8dba00640 1 --2- 172.21.15.131:0/1710980739 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8d41e9510 0x7fa8d41623a0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:11.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.365+0000 7fa8dba00640 1 --2- 172.21.15.131:0/1710980739 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa8bc003f60 0x7fa8d41f4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:11.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.365+0000 7fa8dba00640 1 -- 172.21.15.131:0/1710980739 >> 172.21.15.131:0/1710980739 conn(0x7fa8d40b2d00 msgr2=0x7fa8d40b0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:11.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.365+0000 7fa8dba00640 1 -- 172.21.15.131:0/1710980739 shutdown_connections 2024-05-26T22:49:11.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.365+0000 7fa8dba00640 1 -- 172.21.15.131:0/1710980739 wait complete. 2024-05-26T22:49:11.382 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2024-05-26T22:49:11.382 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:49:11.382 DEBUG:teuthology.orchestra.run.smithi131:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2024-05-26T22:49:11.465 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi131.front.sepia.ceph.com 2024-05-26T22:49:11.465 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:49:11.465 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-05-26T22:49:11.527 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-05-26T22:49:11.527 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-26T22:49:11.583 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-05-26T22:49:11.584 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-26T22:49:11.637 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-05-26T22:49:11.637 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-26T22:49:11.692 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-26T22:49:11.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.899+0000 7f6489800640 1 Processor -- start 2024-05-26T22:49:11.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.899+0000 7f6489800640 1 -- start start 2024-05-26T22:49:11.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.900+0000 7f6489800640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f648406ee20 0x7f64840b3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:11.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.901+0000 7f6489800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f64840ad250 0x7f64840af790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:11.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.901+0000 7f6489800640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f648415cb20 con 0x7f648406f200 2024-05-26T22:49:11.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.901+0000 7f6489800640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f648415ce00 con 0x7f648406ee20 2024-05-26T22:49:11.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.901+0000 7f6489800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f648415d0e0 con 0x7f64840ad250 2024-05-26T22:49:11.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.901+0000 7f6482a00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f648406ee20 0x7f64840b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:11.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.901+0000 7f6482a00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f648406ee20 0x7f64840b3190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41978/0 (socket says 172.21.15.131:41978) 2024-05-26T22:49:11.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.901+0000 7f6482a00640 1 -- 172.21.15.131:0/1249796386 learned_addr learned my addr 172.21.15.131:0/1249796386 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:11.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.901+0000 7f6482a00640 1 -- 172.21.15.131:0/1249796386 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f64840ad250 msgr2=0x7f64840af790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:11.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.901+0000 7f647a000640 1 --2- 172.21.15.131:0/1249796386 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f64840ad250 0x7f64840af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:11.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.901+0000 7f6482a00640 1 --2- 172.21.15.131:0/1249796386 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f64840ad250 0x7f64840af790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:11.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.901+0000 7f6482a00640 1 -- 172.21.15.131:0/1249796386 >> v1:172.21.15.187:6789/0 conn(0x7f648406f200 legacy=0x7f64841562c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:11.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.901+0000 7f6482a00640 1 -- 172.21.15.131:0/1249796386 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f648415d3c0 con 0x7f648406ee20 2024-05-26T22:49:11.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.901+0000 7f647a000640 1 --2- 172.21.15.131:0/1249796386 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f64840ad250 0x7f64840af790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:49:11.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.902+0000 7f6482a00640 1 --2- 172.21.15.131:0/1249796386 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f648406ee20 0x7f64840b3190 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f6474009870 tx=0x7f647402a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccd07911e23a97c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:11.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.903+0000 7f6482000640 1 -- 172.21.15.131:0/1249796386 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6474037070 con 0x7f648406ee20 2024-05-26T22:49:11.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.903+0000 7f6482000640 1 -- 172.21.15.131:0/1249796386 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6474032440 con 0x7f648406ee20 2024-05-26T22:49:11.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.903+0000 7f6482000640 1 -- 172.21.15.131:0/1249796386 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6474032810 con 0x7f648406ee20 2024-05-26T22:49:11.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.903+0000 7f6489800640 1 -- 172.21.15.131:0/1249796386 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f648406ee20 msgr2=0x7f64840b3190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:11.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.903+0000 7f6489800640 1 --2- 172.21.15.131:0/1249796386 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f648406ee20 0x7f64840b3190 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f6474009870 tx=0x7f647402a200 comp rx=0 tx=0).stop 2024-05-26T22:49:11.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.904+0000 7f6489800640 1 -- 172.21.15.131:0/1249796386 shutdown_connections 2024-05-26T22:49:11.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.904+0000 7f6489800640 1 --2- 172.21.15.131:0/1249796386 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f64840ad250 0x7f64840af790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:11.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.904+0000 7f6489800640 1 --2- 172.21.15.131:0/1249796386 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f648406ee20 0x7f64840b3190 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:11.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.904+0000 7f6489800640 1 -- 172.21.15.131:0/1249796386 >> 172.21.15.131:0/1249796386 conn(0x7f64840b1e90 msgr2=0x7f64840b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:11.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.904+0000 7f6489800640 1 -- 172.21.15.131:0/1249796386 shutdown_connections 2024-05-26T22:49:11.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.904+0000 7f6489800640 1 -- 172.21.15.131:0/1249796386 wait complete. 2024-05-26T22:49:11.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.905+0000 7f6489800640 1 Processor -- start 2024-05-26T22:49:11.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.905+0000 7f6489800640 1 -- start start 2024-05-26T22:49:11.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.906+0000 7f6489800640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f648406ee20 0x7f64840cd910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:11.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.906+0000 7f6489800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f648406f200 0x7f64840cde50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:11.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.906+0000 7f6489800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f64840ad250 0x7f6484071590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:11.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.907+0000 7f6489800640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f648415c900 con 0x7f648406f200 2024-05-26T22:49:11.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.907+0000 7f6489800640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f648415c780 con 0x7f648406ee20 2024-05-26T22:49:11.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.907+0000 7f6489800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f64841569d0 con 0x7f64840ad250 2024-05-26T22:49:11.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.907+0000 7f647a000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f648406f200 0x7f64840cde50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:11.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.907+0000 7f647a000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f648406f200 0x7f64840cde50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41992/0 (socket says 172.21.15.131:41992) 2024-05-26T22:49:11.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.907+0000 7f647a000640 1 -- 172.21.15.131:0/2884593688 learned_addr learned my addr 172.21.15.131:0/2884593688 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:11.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.907+0000 7f647a000640 1 -- 172.21.15.131:0/2884593688 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f64840ad250 msgr2=0x7f6484071590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:11.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.907+0000 7f647a000640 1 --2- 172.21.15.131:0/2884593688 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f64840ad250 0x7f6484071590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:11.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.907+0000 7f6482a00640 1 --2- 172.21.15.131:0/2884593688 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f648406ee20 0x7f64840cd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:11.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.907+0000 7f647a000640 1 -- 172.21.15.131:0/2884593688 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f648406ee20 msgr2=0x7f64840cd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:11.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.907+0000 7f647a000640 1 --2- 172.21.15.131:0/2884593688 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f648406ee20 0x7f64840cd910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:11.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.907+0000 7f647a000640 1 -- 172.21.15.131:0/2884593688 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64840721a0 con 0x7f648406f200 2024-05-26T22:49:11.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.908+0000 7f647a000640 1 --2- 172.21.15.131:0/2884593688 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f648406f200 0x7f64840cde50 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f646c00ce00 tx=0x7f646c00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:11.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.908+0000 7f647b400640 1 -- 172.21.15.131:0/2884593688 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f646c00f890 con 0x7f648406f200 2024-05-26T22:49:11.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.908+0000 7f647b400640 1 -- 172.21.15.131:0/2884593688 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f646c00fa30 con 0x7f648406f200 2024-05-26T22:49:11.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.908+0000 7f647b400640 1 -- 172.21.15.131:0/2884593688 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f646c00fe40 con 0x7f648406f200 2024-05-26T22:49:11.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.908+0000 7f6489800640 1 -- 172.21.15.131:0/2884593688 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6484072480 con 0x7f648406f200 2024-05-26T22:49:11.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.908+0000 7f6489800640 1 -- 172.21.15.131:0/2884593688 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6484072a40 con 0x7f648406f200 2024-05-26T22:49:11.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.909+0000 7f6489800640 1 -- 172.21.15.131:0/2884593688 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64840694e0 con 0x7f648406f200 2024-05-26T22:49:11.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.910+0000 7f647b400640 1 -- 172.21.15.131:0/2884593688 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f646c018400 con 0x7f648406f200 2024-05-26T22:49:11.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.911+0000 7f647b400640 1 --2- 172.21.15.131:0/2884593688 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f646006be30 0x7f646006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:11.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.911+0000 7f647b400640 1 -- 172.21.15.131:0/2884593688 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f646c08bdd0 con 0x7f648406f200 2024-05-26T22:49:11.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.911+0000 7f6482a00640 1 --2- 172.21.15.131:0/2884593688 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f646006be30 0x7f646006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:11.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.912+0000 7f6482a00640 1 --2- 172.21.15.131:0/2884593688 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f646006be30 0x7f646006e2f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6474034000 tx=0x7f647402e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:11.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:11.915+0000 7f647b400640 1 -- 172.21.15.131:0/2884593688 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f646c014030 con 0x7f648406f200 2024-05-26T22:49:12.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.112+0000 7f6489800640 1 -- 172.21.15.131:0/2884593688 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f64840ae850 con 0x7f648406f200 2024-05-26T22:49:12.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.115+0000 7f647b400640 1 -- 172.21.15.131:0/2884593688 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16879 (secure 0 0 0) 0x7f646c00ae20 con 0x7f648406f200 2024-05-26T22:49:12.115 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:49:12.115 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":22,"fsid":"46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f","created":"2024-05-26T22:47:57.288905+0000","modified":"2024-05-26T22:49:04.278988+0000","last_up_change":"2024-05-26T22:48:09.602207+0000","last_in_change":"2024-05-26T22:48:00.687341+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-26T22:48:10.463339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-26T22:48:19.663254+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-26T22:48:20.235751+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"814eaf62-a18d-494b-8e27-fd8fb88adea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6801","nonce":2350940051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6803","nonce":2350940051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6807","nonce":2350940051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6805","nonce":2350940051}]},"public_addr":"172.21.15.131:6801/2350940051","cluster_addr":"172.21.15.131:6803/2350940051","heartbeat_back_addr":"172.21.15.131:6807/2350940051","heartbeat_front_addr":"172.21.15.131:6805/2350940051","state":["exists","up"]},{"osd":1,"uuid":"42e66287-f6c6-4e26-af3c-6dfaa3536d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6825","nonce":1525528247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6827","nonce":1525528247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6831","nonce":1525528247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6829","nonce":1525528247}]},"public_addr":"172.21.15.131:6825/1525528247","cluster_addr":"172.21.15.131:6827/1525528247","heartbeat_back_addr":"172.21.15.131:6831/1525528247","heartbeat_front_addr":"172.21.15.131:6829/1525528247","state":["exists","up"]},{"osd":2,"uuid":"0da34673-04a2-4fd0-b57a-0ff2b679b043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6817","nonce":3627842210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6819","nonce":3627842210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6823","nonce":3627842210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6821","nonce":3627842210}]},"public_addr":"172.21.15.131:6817/3627842210","cluster_addr":"172.21.15.131:6819/3627842210","heartbeat_back_addr":"172.21.15.131:6823/3627842210","heartbeat_front_addr":"172.21.15.131:6821/3627842210","state":["exists","up"]},{"osd":3,"uuid":"cf117a27-d6b2-43d3-ae52-b47988bf8eed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6809","nonce":350790591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6811","nonce":350790591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6815","nonce":350790591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6813","nonce":350790591}]},"public_addr":"172.21.15.131:6809/350790591","cluster_addr":"172.21.15.131:6811/350790591","heartbeat_back_addr":"172.21.15.131:6815/350790591","heartbeat_front_addr":"172.21.15.131:6813/350790591","state":["exists","up"]},{"osd":4,"uuid":"a092ccfc-be89-4a84-88d5-0f781fa715eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6816","nonce":244560995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6819","nonce":244560995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6823","nonce":244560995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6821","nonce":244560995}]},"public_addr":"172.21.15.187:6816/244560995","cluster_addr":"172.21.15.187:6819/244560995","heartbeat_back_addr":"172.21.15.187:6823/244560995","heartbeat_front_addr":"172.21.15.187:6821/244560995","state":["exists","up"]},{"osd":5,"uuid":"7513ba69-dd00-4a35-b190-9e8cc3b2bdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6801","nonce":324619398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6803","nonce":324619398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6812","nonce":324619398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6807","nonce":324619398}]},"public_addr":"172.21.15.187:6801/324619398","cluster_addr":"172.21.15.187:6803/324619398","heartbeat_back_addr":"172.21.15.187:6812/324619398","heartbeat_front_addr":"172.21.15.187:6807/324619398","state":["exists","up"]},{"osd":6,"uuid":"50081810-c093-4d2f-a4d7-b7c695c5400d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6806","nonce":1391347786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6809","nonce":1391347786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6818","nonce":1391347786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6813","nonce":1391347786}]},"public_addr":"172.21.15.187:6806/1391347786","cluster_addr":"172.21.15.187:6809/1391347786","heartbeat_back_addr":"172.21.15.187:6818/1391347786","heartbeat_front_addr":"172.21.15.187:6813/1391347786","state":["exists","up"]},{"osd":7,"uuid":"775dae05-54f5-4ad4-bbce-92863c6ed2ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6825","nonce":1220065622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6827","nonce":1220065622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6831","nonce":1220065622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6829","nonce":1220065622}]},"public_addr":"172.21.15.187:6825/1220065622","cluster_addr":"172.21.15.187:6827/1220065622","heartbeat_back_addr":"172.21.15.187:6831/1220065622","heartbeat_front_addr":"172.21.15.187:6829/1220065622","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:05.983520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.299516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.067323+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.307748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.018742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.258775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.010912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.295355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.1e","mappings":[{"from":3,"to":2}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":7,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-26T22:49:12.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.118+0000 7f6489800640 1 -- 172.21.15.131:0/2884593688 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f646006be30 msgr2=0x7f646006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:12.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.118+0000 7f6489800640 1 --2- 172.21.15.131:0/2884593688 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f646006be30 0x7f646006e2f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6474034000 tx=0x7f647402e000 comp rx=0 tx=0).stop 2024-05-26T22:49:12.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.118+0000 7f6489800640 1 -- 172.21.15.131:0/2884593688 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f648406f200 msgr2=0x7f64840cde50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:12.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.118+0000 7f6489800640 1 --2- 172.21.15.131:0/2884593688 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f648406f200 0x7f64840cde50 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f646c00ce00 tx=0x7f646c00a650 comp rx=0 tx=0).stop 2024-05-26T22:49:12.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.118+0000 7f6489800640 1 -- 172.21.15.131:0/2884593688 shutdown_connections 2024-05-26T22:49:12.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.118+0000 7f6489800640 1 --2- 172.21.15.131:0/2884593688 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f64840ad250 0x7f6484071590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:12.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.118+0000 7f6489800640 1 --2- 172.21.15.131:0/2884593688 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f646006be30 0x7f646006e2f0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:12.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.118+0000 7f6489800640 1 --2- 172.21.15.131:0/2884593688 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f648406f200 0x7f64840cde50 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:12.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.118+0000 7f6489800640 1 --2- 172.21.15.131:0/2884593688 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f648406ee20 0x7f64840cd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:12.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.118+0000 7f6489800640 1 -- 172.21.15.131:0/2884593688 >> 172.21.15.131:0/2884593688 conn(0x7f64840b1e90 msgr2=0x7f64840b16f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:12.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.118+0000 7f6489800640 1 -- 172.21.15.131:0/2884593688 shutdown_connections 2024-05-26T22:49:12.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.118+0000 7f6489800640 1 -- 172.21.15.131:0/2884593688 wait complete. 2024-05-26T22:49:12.137 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-26T22:48:10.463339+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-26T22:48:19.663254+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-26T22:48:20.235751+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-26T22:49:12.137 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-26T22:49:12.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.342+0000 7fc5a2a00640 1 Processor -- start 2024-05-26T22:49:12.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.342+0000 7fc5a2a00640 1 -- start start 2024-05-26T22:49:12.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.344+0000 7fc5a2a00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fc59c145a10 0x7fc59c072180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:12.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.344+0000 7fc5a2a00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc59c06d960 0x7fc59c0cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:12.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.344+0000 7fc5a2a00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc59c072d60 con 0x7fc59c0712d0 2024-05-26T22:49:12.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.344+0000 7fc5a2a00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fc59c15c760 con 0x7fc59c145a10 2024-05-26T22:49:12.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.344+0000 7fc5a2a00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc59c072a80 con 0x7fc59c06d960 2024-05-26T22:49:12.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.344+0000 7fc5a1600640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fc59c145a10 0x7fc59c072180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:12.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.344+0000 7fc5a0c00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fc59c0712d0 0x7fc59c06d250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:46704/0 (socket says 172.21.15.131:46704) 2024-05-26T22:49:12.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.344+0000 7fc5a0c00640 1 -- 172.21.15.131:0/1163634069 learned_addr learned my addr 172.21.15.131:0/1163634069 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:49:12.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.344+0000 7fc5a2000640 1 --2- 172.21.15.131:0/1163634069 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc59c06d960 0x7fc59c0cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:12.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.344+0000 7fc5a1600640 1 -- 172.21.15.131:0/1163634069 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc59c06d960 msgr2=0x7fc59c0cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:12.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.344+0000 7fc5a1600640 1 --2- 172.21.15.131:0/1163634069 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc59c06d960 0x7fc59c0cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:12.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.345+0000 7fc5a1600640 1 -- 172.21.15.131:0/1163634069 >> v1:172.21.15.131:6789/0 conn(0x7fc59c0712d0 legacy=0x7fc59c06d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:12.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.345+0000 7fc5a1600640 1 -- 172.21.15.131:0/1163634069 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc59c0ceab0 con 0x7fc59c145a10 2024-05-26T22:49:12.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.346+0000 7fc5a1600640 1 --2- 172.21.15.131:0/1163634069 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc59c145a10 0x7fc59c072180 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc5900097a0 tx=0x7fc59002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=337d5da3892873e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:12.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.346+0000 7fc59be00640 1 -- 172.21.15.131:0/1163634069 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc590037070 con 0x7fc59c145a10 2024-05-26T22:49:12.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.346+0000 7fc59be00640 1 -- 172.21.15.131:0/1163634069 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc590032440 con 0x7fc59c145a10 2024-05-26T22:49:12.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.346+0000 7fc59be00640 1 -- 172.21.15.131:0/1163634069 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc590032810 con 0x7fc59c145a10 2024-05-26T22:49:12.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.346+0000 7fc5a2a00640 1 -- 172.21.15.131:0/1163634069 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc59c145a10 msgr2=0x7fc59c072180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:12.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.346+0000 7fc5a2a00640 1 --2- 172.21.15.131:0/1163634069 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc59c145a10 0x7fc59c072180 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc5900097a0 tx=0x7fc59002a180 comp rx=0 tx=0).stop 2024-05-26T22:49:12.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.347+0000 7fc5a2a00640 1 -- 172.21.15.131:0/1163634069 shutdown_connections 2024-05-26T22:49:12.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.347+0000 7fc5a2a00640 1 --2- 172.21.15.131:0/1163634069 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc59c06d960 0x7fc59c0cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:12.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.347+0000 7fc5a2a00640 1 --2- 172.21.15.131:0/1163634069 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc59c145a10 0x7fc59c072180 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:12.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.347+0000 7fc5a2a00640 1 -- 172.21.15.131:0/1163634069 >> 172.21.15.131:0/1163634069 conn(0x7fc59c0b5660 msgr2=0x7fc59c0b5a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:12.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.347+0000 7fc5a2a00640 1 -- 172.21.15.131:0/1163634069 shutdown_connections 2024-05-26T22:49:12.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.347+0000 7fc5a2a00640 1 -- 172.21.15.131:0/1163634069 wait complete. 2024-05-26T22:49:12.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.349+0000 7fc5a2a00640 1 Processor -- start 2024-05-26T22:49:12.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.350+0000 7fc5a2a00640 1 -- start start 2024-05-26T22:49:12.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.351+0000 7fc5a2a00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc59c06d960 0x7fc59c1f7d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:12.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.351+0000 7fc5a2a00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc59c0712d0 0x7fc59c140b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:12.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.351+0000 7fc5a2a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc59c145a10 0x7fc59c1f3e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:12.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.351+0000 7fc5a2a00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc59c145df0 con 0x7fc59c145a10 2024-05-26T22:49:12.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.351+0000 7fc5a2a00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc59c0cf070 con 0x7fc59c06d960 2024-05-26T22:49:12.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.351+0000 7fc5a2a00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc59c0726c0 con 0x7fc59c0712d0 2024-05-26T22:49:12.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.351+0000 7fc5a2000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc59c145a10 0x7fc59c1f3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:12.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.351+0000 7fc5a2000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc59c145a10 0x7fc59c1f3e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42004/0 (socket says 172.21.15.131:42004) 2024-05-26T22:49:12.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.351+0000 7fc5a2000640 1 -- 172.21.15.131:0/2244953681 learned_addr learned my addr 172.21.15.131:0/2244953681 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:12.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.352+0000 7fc5a0c00640 1 --2- 172.21.15.131:0/2244953681 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc59c0712d0 0x7fc59c140b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:12.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.352+0000 7fc5a1600640 1 --2- 172.21.15.131:0/2244953681 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc59c06d960 0x7fc59c1f7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:12.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.352+0000 7fc5a2000640 1 -- 172.21.15.131:0/2244953681 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc59c0712d0 msgr2=0x7fc59c140b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:12.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.352+0000 7fc5a2000640 1 --2- 172.21.15.131:0/2244953681 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc59c0712d0 0x7fc59c140b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:12.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.352+0000 7fc5a2000640 1 -- 172.21.15.131:0/2244953681 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc59c06d960 msgr2=0x7fc59c1f7d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:12.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.352+0000 7fc5a2000640 1 --2- 172.21.15.131:0/2244953681 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc59c06d960 0x7fc59c1f7d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:12.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.352+0000 7fc5a2000640 1 -- 172.21.15.131:0/2244953681 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc59c1f47a0 con 0x7fc59c145a10 2024-05-26T22:49:12.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.352+0000 7fc5a1600640 1 --2- 172.21.15.131:0/2244953681 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc59c06d960 0x7fc59c1f7d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:49:12.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.352+0000 7fc5a2000640 1 --2- 172.21.15.131:0/2244953681 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc59c145a10 0x7fc59c1f3e30 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc594007de0 tx=0x7fc59400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:12.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.352+0000 7fc599600640 1 -- 172.21.15.131:0/2244953681 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc594014620 con 0x7fc59c145a10 2024-05-26T22:49:12.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.352+0000 7fc599600640 1 -- 172.21.15.131:0/2244953681 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc594019070 con 0x7fc59c145a10 2024-05-26T22:49:12.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.352+0000 7fc599600640 1 -- 172.21.15.131:0/2244953681 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc594014a20 con 0x7fc59c145a10 2024-05-26T22:49:12.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.352+0000 7fc5a2a00640 1 -- 172.21.15.131:0/2244953681 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc59c1f4a80 con 0x7fc59c145a10 2024-05-26T22:49:12.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.353+0000 7fc5a2a00640 1 -- 172.21.15.131:0/2244953681 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc59c1f7830 con 0x7fc59c145a10 2024-05-26T22:49:12.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.354+0000 7fc5a2a00640 1 -- 172.21.15.131:0/2244953681 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc59c0694e0 con 0x7fc59c145a10 2024-05-26T22:49:12.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.354+0000 7fc599600640 1 -- 172.21.15.131:0/2244953681 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc594014bc0 con 0x7fc59c145a10 2024-05-26T22:49:12.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.355+0000 7fc599600640 1 --2- 172.21.15.131:0/2244953681 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc57c06be30 0x7fc57c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:12.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.355+0000 7fc599600640 1 -- 172.21.15.131:0/2244953681 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fc59408b870 con 0x7fc59c145a10 2024-05-26T22:49:12.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.356+0000 7fc5a1600640 1 --2- 172.21.15.131:0/2244953681 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc57c06be30 0x7fc57c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:12.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.356+0000 7fc5a1600640 1 --2- 172.21.15.131:0/2244953681 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc57c06be30 0x7fc57c06e2f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc590034000 tx=0x7fc59002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:12.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.360+0000 7fc599600640 1 -- 172.21.15.131:0/2244953681 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fc59405a190 con 0x7fc59c145a10 2024-05-26T22:49:12.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.557+0000 7fc5a2a00640 1 -- 172.21.15.131:0/2244953681 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc59c06d6f0 con 0x7fc59c145a10 2024-05-26T22:49:12.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.557+0000 7fc599600640 1 -- 172.21.15.131:0/2244953681 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7fc594059a90 con 0x7fc59c145a10 2024-05-26T22:49:12.557 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2024-05-26T22:49:12.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.560+0000 7fc5a2a00640 1 -- 172.21.15.131:0/2244953681 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc57c06be30 msgr2=0x7fc57c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:12.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.560+0000 7fc5a2a00640 1 --2- 172.21.15.131:0/2244953681 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc57c06be30 0x7fc57c06e2f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc590034000 tx=0x7fc59002e000 comp rx=0 tx=0).stop 2024-05-26T22:49:12.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.560+0000 7fc5a2a00640 1 -- 172.21.15.131:0/2244953681 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc59c145a10 msgr2=0x7fc59c1f3e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:12.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.560+0000 7fc5a2a00640 1 --2- 172.21.15.131:0/2244953681 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc59c145a10 0x7fc59c1f3e30 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc594007de0 tx=0x7fc59400e4b0 comp rx=0 tx=0).stop 2024-05-26T22:49:12.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.560+0000 7fc5a2a00640 1 -- 172.21.15.131:0/2244953681 shutdown_connections 2024-05-26T22:49:12.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.560+0000 7fc5a2a00640 1 --2- 172.21.15.131:0/2244953681 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc59c0712d0 0x7fc59c140b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:12.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.560+0000 7fc5a2a00640 1 --2- 172.21.15.131:0/2244953681 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc57c06be30 0x7fc57c06e2f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:12.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.560+0000 7fc5a2a00640 1 --2- 172.21.15.131:0/2244953681 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc59c145a10 0x7fc59c1f3e30 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:12.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.560+0000 7fc5a2a00640 1 --2- 172.21.15.131:0/2244953681 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc59c06d960 0x7fc59c1f7d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:12.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.560+0000 7fc5a2a00640 1 -- 172.21.15.131:0/2244953681 >> 172.21.15.131:0/2244953681 conn(0x7fc59c0b5660 msgr2=0x7fc59c160560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:12.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.560+0000 7fc5a2a00640 1 -- 172.21.15.131:0/2244953681 shutdown_connections 2024-05-26T22:49:12.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.561+0000 7fc5a2a00640 1 -- 172.21.15.131:0/2244953681 wait complete. 2024-05-26T22:49:12.580 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-26T22:49:12.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.780+0000 7f7724800640 1 Processor -- start 2024-05-26T22:49:12.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.780+0000 7f7724800640 1 -- start start 2024-05-26T22:49:12.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.782+0000 7f7724800640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f771c155770 0x7f771c15b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:12.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.782+0000 7f7724800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f771c15bc40 0x7f771c06dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:12.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.782+0000 7f7724800640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f771c06e270 0x7f771c0ceba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:12.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.782+0000 7f7724800640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f771c06e6a0 con 0x7f771c06e270 2024-05-26T22:49:12.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.782+0000 7f7724800640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f771c15c1c0 con 0x7f771c155770 2024-05-26T22:49:12.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.782+0000 7f7724800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f771c15c4a0 con 0x7f771c15bc40 2024-05-26T22:49:12.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.782+0000 7f7722800640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f771c06e270 0x7f771c0ceba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:12.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.782+0000 7f7722800640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f771c06e270 0x7f771c0ceba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42008/0 (socket says 172.21.15.131:42008) 2024-05-26T22:49:12.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.782+0000 7f7722800640 1 -- 172.21.15.131:0/2066964 learned_addr learned my addr 172.21.15.131:0/2066964 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:12.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.782+0000 7f7721400640 1 --2- 172.21.15.131:0/2066964 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f771c15bc40 0x7f771c06dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:12.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.782+0000 7f7722800640 1 -- 172.21.15.131:0/2066964 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f771c15bc40 msgr2=0x7f771c06dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:12.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.782+0000 7f7722800640 1 --2- 172.21.15.131:0/2066964 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f771c15bc40 0x7f771c06dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:12.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.782+0000 7f7722800640 1 -- 172.21.15.131:0/2066964 >> v2:172.21.15.187:3300/0 conn(0x7f771c155770 msgr2=0x7f771c15b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:12.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.782+0000 7f7722800640 1 --2- 172.21.15.131:0/2066964 >> v2:172.21.15.187:3300/0 conn(0x7f771c155770 0x7f771c15b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:12.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.782+0000 7f7722800640 1 -- 172.21.15.131:0/2066964 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f771c071da0 con 0x7f771c06e270 2024-05-26T22:49:12.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.782+0000 7f7721400640 1 --2- 172.21.15.131:0/2066964 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f771c15bc40 0x7f771c06dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:49:12.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.783+0000 7f7722800640 1 --2- 172.21.15.131:0/2066964 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f771c06e270 0x7f771c0ceba0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f771c071790 tx=0x7f771802c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1db31cf13d96526d server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:12.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.784+0000 7f7720a00640 1 -- 172.21.15.131:0/2066964 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7718009600 con 0x7f771c06e270 2024-05-26T22:49:12.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.784+0000 7f7720a00640 1 -- 172.21.15.131:0/2066964 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7718037070 con 0x7f771c06e270 2024-05-26T22:49:12.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.784+0000 7f7720a00640 1 -- 172.21.15.131:0/2066964 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77180099c0 con 0x7f771c06e270 2024-05-26T22:49:12.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.784+0000 7f7724800640 1 -- 172.21.15.131:0/2066964 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f771c06e270 msgr2=0x7f771c0ceba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:12.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.784+0000 7f7724800640 1 --2- 172.21.15.131:0/2066964 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f771c06e270 0x7f771c0ceba0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f771c071790 tx=0x7f771802c930 comp rx=0 tx=0).stop 2024-05-26T22:49:12.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.784+0000 7f7724800640 1 -- 172.21.15.131:0/2066964 shutdown_connections 2024-05-26T22:49:12.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.784+0000 7f7724800640 1 --2- 172.21.15.131:0/2066964 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f771c06e270 0x7f771c0ceba0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:12.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.784+0000 7f7724800640 1 --2- 172.21.15.131:0/2066964 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f771c15bc40 0x7f771c06dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:12.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.784+0000 7f7724800640 1 --2- 172.21.15.131:0/2066964 >> v2:172.21.15.187:3300/0 conn(0x7f771c155770 0x7f771c15b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:12.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.784+0000 7f7724800640 1 -- 172.21.15.131:0/2066964 >> 172.21.15.131:0/2066964 conn(0x7f771c06bc90 msgr2=0x7f771c0b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:12.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.785+0000 7f7724800640 1 -- 172.21.15.131:0/2066964 shutdown_connections 2024-05-26T22:49:12.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.785+0000 7f7724800640 1 -- 172.21.15.131:0/2066964 wait complete. 2024-05-26T22:49:12.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.786+0000 7f7724800640 1 Processor -- start 2024-05-26T22:49:12.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.786+0000 7f7724800640 1 -- start start 2024-05-26T22:49:12.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.787+0000 7f7724800640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f771c06e270 0x7f771c1454e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:12.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.787+0000 7f7724800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f771c155770 0x7f771c152850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:12.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.787+0000 7f7724800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f771c15bc40 0x7f771c152d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:12.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.787+0000 7f7724800640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f771c15c8c0 con 0x7f771c15bc40 2024-05-26T22:49:12.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.787+0000 7f7724800640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f771c155b50 con 0x7f771c06e270 2024-05-26T22:49:12.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.787+0000 7f7724800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f771c09de00 con 0x7f771c155770 2024-05-26T22:49:12.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.788+0000 7f7722800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f771c15bc40 0x7f771c152d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:12.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.788+0000 7f7722800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f771c15bc40 0x7f771c152d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42018/0 (socket says 172.21.15.131:42018) 2024-05-26T22:49:12.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.788+0000 7f7722800640 1 -- 172.21.15.131:0/903470050 learned_addr learned my addr 172.21.15.131:0/903470050 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:12.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.788+0000 7f7721400640 1 --2- 172.21.15.131:0/903470050 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f771c155770 0x7f771c152850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:12.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.788+0000 7f7721e00640 1 --2- 172.21.15.131:0/903470050 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f771c06e270 0x7f771c1454e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:12.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.788+0000 7f7722800640 1 -- 172.21.15.131:0/903470050 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f771c155770 msgr2=0x7f771c152850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:12.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.788+0000 7f7722800640 1 --2- 172.21.15.131:0/903470050 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f771c155770 0x7f771c152850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:12.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.788+0000 7f7722800640 1 -- 172.21.15.131:0/903470050 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f771c06e270 msgr2=0x7f771c1454e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:12.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.788+0000 7f7722800640 1 --2- 172.21.15.131:0/903470050 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f771c06e270 0x7f771c1454e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:12.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.788+0000 7f7722800640 1 -- 172.21.15.131:0/903470050 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f771c153410 con 0x7f771c15bc40 2024-05-26T22:49:12.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.788+0000 7f7721e00640 1 --2- 172.21.15.131:0/903470050 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f771c06e270 0x7f771c1454e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:49:12.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.788+0000 7f7721400640 1 --2- 172.21.15.131:0/903470050 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f771c155770 0x7f771c152850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:49:12.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.788+0000 7f7722800640 1 --2- 172.21.15.131:0/903470050 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f771c15bc40 0x7f771c152d90 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f771800d240 tx=0x7f7718004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:12.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.789+0000 7f7716000640 1 -- 172.21.15.131:0/903470050 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7718009dc0 con 0x7f771c15bc40 2024-05-26T22:49:12.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.789+0000 7f7716000640 1 -- 172.21.15.131:0/903470050 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7718032420 con 0x7f771c15bc40 2024-05-26T22:49:12.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.789+0000 7f7716000640 1 -- 172.21.15.131:0/903470050 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7718032810 con 0x7f771c15bc40 2024-05-26T22:49:12.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.789+0000 7f7724800640 1 -- 172.21.15.131:0/903470050 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f771c1536f0 con 0x7f771c15bc40 2024-05-26T22:49:12.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.789+0000 7f7724800640 1 -- 172.21.15.131:0/903470050 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f771c153cb0 con 0x7f771c15bc40 2024-05-26T22:49:12.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.790+0000 7f7724800640 1 -- 172.21.15.131:0/903470050 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f771c074e90 con 0x7f771c15bc40 2024-05-26T22:49:12.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.791+0000 7f7716000640 1 -- 172.21.15.131:0/903470050 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f77180329b0 con 0x7f771c15bc40 2024-05-26T22:49:12.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.792+0000 7f7716000640 1 --2- 172.21.15.131:0/903470050 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f770006be30 0x7f770006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:12.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.792+0000 7f7716000640 1 -- 172.21.15.131:0/903470050 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f77180abe50 con 0x7f771c15bc40 2024-05-26T22:49:12.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.792+0000 7f7721e00640 1 --2- 172.21.15.131:0/903470050 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f770006be30 0x7f770006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:12.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.793+0000 7f7721e00640 1 --2- 172.21.15.131:0/903470050 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f770006be30 0x7f770006e2f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f771c1504f0 tx=0x7f7710008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:12.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.797+0000 7f7716000640 1 -- 172.21.15.131:0/903470050 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f771807a770 con 0x7f771c15bc40 2024-05-26T22:49:12.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.992+0000 7f7724800640 1 -- 172.21.15.131:0/903470050 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f771c0b1ea0 con 0x7f771c15bc40 2024-05-26T22:49:12.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.993+0000 7f7716000640 1 -- 172.21.15.131:0/903470050 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7f771807a070 con 0x7f771c15bc40 2024-05-26T22:49:12.993 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-05-26T22:49:12.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.995+0000 7f7724800640 1 -- 172.21.15.131:0/903470050 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f770006be30 msgr2=0x7f770006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:12.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.995+0000 7f7724800640 1 --2- 172.21.15.131:0/903470050 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f770006be30 0x7f770006e2f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f771c1504f0 tx=0x7f7710008040 comp rx=0 tx=0).stop 2024-05-26T22:49:12.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.995+0000 7f7724800640 1 -- 172.21.15.131:0/903470050 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f771c15bc40 msgr2=0x7f771c152d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:12.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.995+0000 7f7724800640 1 --2- 172.21.15.131:0/903470050 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f771c15bc40 0x7f771c152d90 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f771800d240 tx=0x7f7718004a40 comp rx=0 tx=0).stop 2024-05-26T22:49:12.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.996+0000 7f7724800640 1 -- 172.21.15.131:0/903470050 shutdown_connections 2024-05-26T22:49:12.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.996+0000 7f7724800640 1 --2- 172.21.15.131:0/903470050 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f771c155770 0x7f771c152850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:12.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.996+0000 7f7724800640 1 --2- 172.21.15.131:0/903470050 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f770006be30 0x7f770006e2f0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:12.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.996+0000 7f7724800640 1 --2- 172.21.15.131:0/903470050 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f771c15bc40 0x7f771c152d90 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:12.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.996+0000 7f7724800640 1 --2- 172.21.15.131:0/903470050 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f771c06e270 0x7f771c1454e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:12.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.996+0000 7f7724800640 1 -- 172.21.15.131:0/903470050 >> 172.21.15.131:0/903470050 conn(0x7f771c06bc90 msgr2=0x7f771c0b5260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:12.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.996+0000 7f7724800640 1 -- 172.21.15.131:0/903470050 shutdown_connections 2024-05-26T22:49:12.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:12.996+0000 7f7724800640 1 -- 172.21.15.131:0/903470050 wait complete. 2024-05-26T22:49:13.014 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-26T22:49:13.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.217+0000 7fc5cc600640 1 Processor -- start 2024-05-26T22:49:13.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.217+0000 7fc5cc600640 1 -- start start 2024-05-26T22:49:13.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.219+0000 7fc5cc600640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fc5c40b5450 0x7fc5c40b43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:13.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.219+0000 7fc5cc600640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fc5c40d0460 0x7fc5c406d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:13.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.219+0000 7fc5cc600640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5c40b4900 con 0x7fc5c406f420 2024-05-26T22:49:13.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.219+0000 7fc5cc600640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fc5c4160d90 con 0x7fc5c40d0460 2024-05-26T22:49:13.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.219+0000 7fc5cc600640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fc5c4160180 con 0x7fc5c40b5450 2024-05-26T22:49:13.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.219+0000 7fc5ca600640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fc5c40d0460 0x7fc5c406d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:13.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.219+0000 7fc5ca600640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fc5c40d0460 0x7fc5c406d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42028/0 (socket says 172.21.15.131:42028) 2024-05-26T22:49:13.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.219+0000 7fc5c9200640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fc5c406f420 0x7fc5c40cfd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:46720/0 (socket says 172.21.15.131:46720) 2024-05-26T22:49:13.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.219+0000 7fc5ca600640 1 -- 172.21.15.131:0/596918852 learned_addr learned my addr 172.21.15.131:0/596918852 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:13.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.220+0000 7fc5c9c00640 1 --2- 172.21.15.131:0/596918852 >> v2:172.21.15.187:3300/0 conn(0x7fc5c40b5450 0x7fc5c40b43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:13.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.220+0000 7fc5ca600640 1 -- 172.21.15.131:0/596918852 >> v2:172.21.15.187:3300/0 conn(0x7fc5c40b5450 msgr2=0x7fc5c40b43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:13.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.220+0000 7fc5ca600640 1 --2- 172.21.15.131:0/596918852 >> v2:172.21.15.187:3300/0 conn(0x7fc5c40b5450 0x7fc5c40b43c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:13.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.220+0000 7fc5ca600640 1 -- 172.21.15.131:0/596918852 >> v1:172.21.15.131:6789/0 conn(0x7fc5c406f420 legacy=0x7fc5c40cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:13.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.220+0000 7fc5c3e00640 1 -- 172.21.15.131:0/596918852 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 5162796 0 0) 0x7fc5c40b4900 con 0x7fc5c406f420 2024-05-26T22:49:13.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.220+0000 7fc5ca600640 1 -- 172.21.15.131:0/596918852 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5c4160460 con 0x7fc5c40d0460 2024-05-26T22:49:13.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.220+0000 7fc5c9c00640 1 --2- 172.21.15.131:0/596918852 >> v2:172.21.15.187:3300/0 conn(0x7fc5c40b5450 0x7fc5c40b43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-26T22:49:13.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.221+0000 7fc5ca600640 1 --2- 172.21.15.131:0/596918852 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc5c40d0460 0x7fc5c406d600 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc009e50 tx=0x7fc5bc02a760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9189f4e9aa709d6 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:13.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.221+0000 7fc5c3e00640 1 -- 172.21.15.131:0/596918852 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5bc036070 con 0x7fc5c40d0460 2024-05-26T22:49:13.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.221+0000 7fc5c3e00640 1 -- 172.21.15.131:0/596918852 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5bc031600 con 0x7fc5c40d0460 2024-05-26T22:49:13.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.221+0000 7fc5c3e00640 1 -- 172.21.15.131:0/596918852 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5bc0319d0 con 0x7fc5c40d0460 2024-05-26T22:49:13.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.222+0000 7fc5cc600640 1 -- 172.21.15.131:0/596918852 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc5c40d0460 msgr2=0x7fc5c406d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:13.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.222+0000 7fc5cc600640 1 --2- 172.21.15.131:0/596918852 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc5c40d0460 0x7fc5c406d600 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc009e50 tx=0x7fc5bc02a760 comp rx=0 tx=0).stop 2024-05-26T22:49:13.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.222+0000 7fc5cc600640 1 -- 172.21.15.131:0/596918852 shutdown_connections 2024-05-26T22:49:13.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.222+0000 7fc5cc600640 1 --2- 172.21.15.131:0/596918852 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc5c40d0460 0x7fc5c406d600 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:13.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.222+0000 7fc5cc600640 1 --2- 172.21.15.131:0/596918852 >> v2:172.21.15.187:3300/0 conn(0x7fc5c40b5450 0x7fc5c40b43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:13.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.222+0000 7fc5cc600640 1 -- 172.21.15.131:0/596918852 >> 172.21.15.131:0/596918852 conn(0x7fc5c406bc90 msgr2=0x7fc5c40b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:13.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.222+0000 7fc5cc600640 1 -- 172.21.15.131:0/596918852 shutdown_connections 2024-05-26T22:49:13.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.222+0000 7fc5cc600640 1 -- 172.21.15.131:0/596918852 wait complete. 2024-05-26T22:49:13.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.224+0000 7fc5cc600640 1 Processor -- start 2024-05-26T22:49:13.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.225+0000 7fc5cc600640 1 -- start start 2024-05-26T22:49:13.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.226+0000 7fc5cc600640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc5c406f420 0x7fc5c41500b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:13.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.226+0000 7fc5cc600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc5c40b5450 0x7fc5c4071360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:13.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.226+0000 7fc5cc600640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc5c40d0460 0x7fc5c41f7740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:13.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.226+0000 7fc5cc600640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc5c40b5830 con 0x7fc5c40b5450 2024-05-26T22:49:13.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.226+0000 7fc5cc600640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc5c4160a20 con 0x7fc5c40d0460 2024-05-26T22:49:13.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.226+0000 7fc5cc600640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc5c415fde0 con 0x7fc5c406f420 2024-05-26T22:49:13.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.226+0000 7fc5c9200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc5c40b5450 0x7fc5c4071360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:13.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.226+0000 7fc5c9200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc5c40b5450 0x7fc5c4071360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42036/0 (socket says 172.21.15.131:42036) 2024-05-26T22:49:13.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.226+0000 7fc5c9200640 1 -- 172.21.15.131:0/4175520754 learned_addr learned my addr 172.21.15.131:0/4175520754 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:13.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.226+0000 7fc5c9c00640 1 --2- 172.21.15.131:0/4175520754 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc5c406f420 0x7fc5c41500b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:13.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.226+0000 7fc5ca600640 1 --2- 172.21.15.131:0/4175520754 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc5c40d0460 0x7fc5c41f7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:13.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.227+0000 7fc5c9200640 1 -- 172.21.15.131:0/4175520754 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc5c406f420 msgr2=0x7fc5c41500b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:13.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.227+0000 7fc5c9200640 1 --2- 172.21.15.131:0/4175520754 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc5c406f420 0x7fc5c41500b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:13.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.227+0000 7fc5c9200640 1 -- 172.21.15.131:0/4175520754 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc5c40d0460 msgr2=0x7fc5c41f7740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:13.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.227+0000 7fc5c9200640 1 --2- 172.21.15.131:0/4175520754 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc5c40d0460 0x7fc5c41f7740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:13.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.227+0000 7fc5c9200640 1 -- 172.21.15.131:0/4175520754 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5c40ce950 con 0x7fc5c40b5450 2024-05-26T22:49:13.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.227+0000 7fc5c9200640 1 --2- 172.21.15.131:0/4175520754 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc5c40b5450 0x7fc5c4071360 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fc5b000c430 tx=0x7fc5b000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:13.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.227+0000 7fc5c1600640 1 -- 172.21.15.131:0/4175520754 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5b00074e0 con 0x7fc5c40b5450 2024-05-26T22:49:13.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.227+0000 7fc5c1600640 1 -- 172.21.15.131:0/4175520754 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5b00047e0 con 0x7fc5c40b5450 2024-05-26T22:49:13.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.227+0000 7fc5cc600640 1 -- 172.21.15.131:0/4175520754 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5c40ce3a0 con 0x7fc5c40b5450 2024-05-26T22:49:13.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.227+0000 7fc5c1600640 1 -- 172.21.15.131:0/4175520754 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5b0004bd0 con 0x7fc5c40b5450 2024-05-26T22:49:13.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.227+0000 7fc5cc600640 1 -- 172.21.15.131:0/4175520754 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5c41549d0 con 0x7fc5c40b5450 2024-05-26T22:49:13.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.228+0000 7fc5cc600640 1 -- 172.21.15.131:0/4175520754 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5c4074e90 con 0x7fc5c40b5450 2024-05-26T22:49:13.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.229+0000 7fc5c1600640 1 -- 172.21.15.131:0/4175520754 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc5b0011070 con 0x7fc5c40b5450 2024-05-26T22:49:13.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.230+0000 7fc5c1600640 1 --2- 172.21.15.131:0/4175520754 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc5a406be60 0x7fc5a406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:13.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.230+0000 7fc5c1600640 1 -- 172.21.15.131:0/4175520754 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fc5b008abb0 con 0x7fc5c40b5450 2024-05-26T22:49:13.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.231+0000 7fc5c9c00640 1 --2- 172.21.15.131:0/4175520754 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc5a406be60 0x7fc5a406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:13.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.231+0000 7fc5c9c00640 1 --2- 172.21.15.131:0/4175520754 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc5a406be60 0x7fc5a406e320 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc5b8006fd0 tx=0x7fc5b8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:13.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.235+0000 7fc5c1600640 1 -- 172.21.15.131:0/4175520754 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fc5b000e280 con 0x7fc5c40b5450 2024-05-26T22:49:13.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.437+0000 7fc5cc600640 1 -- 172.21.15.131:0/4175520754 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc5c40cee90 con 0x7fc5c40b5450 2024-05-26T22:49:13.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.438+0000 7fc5c1600640 1 -- 172.21.15.131:0/4175520754 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7fc5b0059440 con 0x7fc5c40b5450 2024-05-26T22:49:13.438 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 64 2024-05-26T22:49:13.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.441+0000 7fc5cc600640 1 -- 172.21.15.131:0/4175520754 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc5a406be60 msgr2=0x7fc5a406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:13.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.441+0000 7fc5cc600640 1 --2- 172.21.15.131:0/4175520754 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc5a406be60 0x7fc5a406e320 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc5b8006fd0 tx=0x7fc5b8008040 comp rx=0 tx=0).stop 2024-05-26T22:49:13.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.441+0000 7fc5cc600640 1 -- 172.21.15.131:0/4175520754 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc5c40b5450 msgr2=0x7fc5c4071360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:13.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.441+0000 7fc5cc600640 1 --2- 172.21.15.131:0/4175520754 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc5c40b5450 0x7fc5c4071360 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fc5b000c430 tx=0x7fc5b000c8f0 comp rx=0 tx=0).stop 2024-05-26T22:49:13.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.441+0000 7fc5cc600640 1 -- 172.21.15.131:0/4175520754 shutdown_connections 2024-05-26T22:49:13.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.441+0000 7fc5cc600640 1 --2- 172.21.15.131:0/4175520754 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc5a406be60 0x7fc5a406e320 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:13.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.441+0000 7fc5cc600640 1 --2- 172.21.15.131:0/4175520754 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc5c40d0460 0x7fc5c41f7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:13.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.441+0000 7fc5cc600640 1 --2- 172.21.15.131:0/4175520754 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc5c40b5450 0x7fc5c4071360 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:13.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.441+0000 7fc5cc600640 1 --2- 172.21.15.131:0/4175520754 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc5c406f420 0x7fc5c41500b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:13.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.441+0000 7fc5cc600640 1 -- 172.21.15.131:0/4175520754 >> 172.21.15.131:0/4175520754 conn(0x7fc5c406bc90 msgr2=0x7fc5c406e6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:13.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.441+0000 7fc5cc600640 1 -- 172.21.15.131:0/4175520754 shutdown_connections 2024-05-26T22:49:13.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.441+0000 7fc5cc600640 1 -- 172.21.15.131:0/4175520754 wait complete. 2024-05-26T22:49:13.460 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-26T22:49:13.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.655+0000 7f273ac00640 1 Processor -- start 2024-05-26T22:49:13.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.656+0000 7f273ac00640 1 -- start start 2024-05-26T22:49:13.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.657+0000 7f273ac00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f273415e340 0x7f27340b72e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:13.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.657+0000 7f273ac00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f27340b7820 0x7f27340d22b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:13.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.657+0000 7f273ac00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f273415e720 con 0x7f27340d27f0 2024-05-26T22:49:13.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.657+0000 7f273ac00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f27340b88b0 con 0x7f27340b7820 2024-05-26T22:49:13.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.657+0000 7f273ac00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f27340b7ca0 con 0x7f273415e340 2024-05-26T22:49:13.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.658+0000 7f2739800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f273415e340 0x7f27340b72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:13.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.658+0000 7f2738e00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f27340b7820 0x7f27340d22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:13.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.658+0000 7f2739800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f273415e340 0x7f27340b72e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.131:53408/0 (socket says 172.21.15.131:53408) 2024-05-26T22:49:13.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.658+0000 7f2739800640 1 -- 172.21.15.131:0/3752135801 learned_addr learned my addr 172.21.15.131:0/3752135801 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:13.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.658+0000 7f2738e00640 1 -- 172.21.15.131:0/3752135801 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f273415e340 msgr2=0x7f27340b72e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:13.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.658+0000 7f2738e00640 1 --2- 172.21.15.131:0/3752135801 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f273415e340 0x7f27340b72e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:13.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.659+0000 7f2738e00640 1 -- 172.21.15.131:0/3752135801 >> v1:172.21.15.187:6789/0 conn(0x7f27340d27f0 legacy=0x7f27340b8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:13.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.659+0000 7f2738e00640 1 -- 172.21.15.131:0/3752135801 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27340b7f80 con 0x7f27340b7820 2024-05-26T22:49:13.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.660+0000 7f2738e00640 1 --2- 172.21.15.131:0/3752135801 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f27340b7820 0x7f27340d22b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f27340b8a30 tx=0x7f272002a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c429d4a19be974e3 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:13.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.660+0000 7f2733e00640 1 -- 172.21.15.131:0/3752135801 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2720037070 con 0x7f27340b7820 2024-05-26T22:49:13.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.660+0000 7f2733e00640 1 -- 172.21.15.131:0/3752135801 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2720032440 con 0x7f27340b7820 2024-05-26T22:49:13.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.660+0000 7f2733e00640 1 -- 172.21.15.131:0/3752135801 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2720032810 con 0x7f27340b7820 2024-05-26T22:49:13.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.661+0000 7f273ac00640 1 -- 172.21.15.131:0/3752135801 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f27340b7820 msgr2=0x7f27340d22b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:13.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.661+0000 7f273ac00640 1 --2- 172.21.15.131:0/3752135801 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f27340b7820 0x7f27340d22b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f27340b8a30 tx=0x7f272002a210 comp rx=0 tx=0).stop 2024-05-26T22:49:13.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.661+0000 7f273ac00640 1 -- 172.21.15.131:0/3752135801 shutdown_connections 2024-05-26T22:49:13.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.661+0000 7f273ac00640 1 --2- 172.21.15.131:0/3752135801 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f27340b7820 0x7f27340d22b0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:13.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.661+0000 7f273ac00640 1 --2- 172.21.15.131:0/3752135801 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f273415e340 0x7f27340b72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:13.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.661+0000 7f273ac00640 1 -- 172.21.15.131:0/3752135801 >> 172.21.15.131:0/3752135801 conn(0x7f2734072e60 msgr2=0x7f2734073270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:13.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.661+0000 7f273ac00640 1 -- 172.21.15.131:0/3752135801 shutdown_connections 2024-05-26T22:49:13.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.661+0000 7f273ac00640 1 -- 172.21.15.131:0/3752135801 wait complete. 2024-05-26T22:49:13.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.663+0000 7f273ac00640 1 Processor -- start 2024-05-26T22:49:13.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.663+0000 7f273ac00640 1 -- start start 2024-05-26T22:49:13.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.664+0000 7f273ac00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f27340b7820 0x7f2734150280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:13.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.664+0000 7f273ac00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f27340d27f0 0x7f273415d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:13.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.664+0000 7f273ac00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f273415e340 0x7f273415d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:13.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.664+0000 7f273ac00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f27340b9450 con 0x7f273415e340 2024-05-26T22:49:13.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.664+0000 7f273ac00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f27340b8540 con 0x7f27340d27f0 2024-05-26T22:49:13.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.664+0000 7f273ac00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f27340b95d0 con 0x7f27340b7820 2024-05-26T22:49:13.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.664+0000 7f273a200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f273415e340 0x7f273415d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:13.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.664+0000 7f273a200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f273415e340 0x7f273415d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42048/0 (socket says 172.21.15.131:42048) 2024-05-26T22:49:13.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.664+0000 7f273a200640 1 -- 172.21.15.131:0/1218284894 learned_addr learned my addr 172.21.15.131:0/1218284894 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:13.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.665+0000 7f2738e00640 1 --2- 172.21.15.131:0/1218284894 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f27340d27f0 0x7f273415d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:13.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.665+0000 7f2739800640 1 --2- 172.21.15.131:0/1218284894 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f27340b7820 0x7f2734150280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:13.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.665+0000 7f273a200640 1 -- 172.21.15.131:0/1218284894 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f27340b7820 msgr2=0x7f2734150280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:13.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.665+0000 7f273a200640 1 --2- 172.21.15.131:0/1218284894 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f27340b7820 0x7f2734150280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:13.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.665+0000 7f273a200640 1 -- 172.21.15.131:0/1218284894 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f27340d27f0 msgr2=0x7f273415d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:13.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.665+0000 7f273a200640 1 --2- 172.21.15.131:0/1218284894 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f27340d27f0 0x7f273415d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:13.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.665+0000 7f273a200640 1 -- 172.21.15.131:0/1218284894 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27341546e0 con 0x7f273415e340 2024-05-26T22:49:13.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.665+0000 7f273a200640 1 --2- 172.21.15.131:0/1218284894 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f273415e340 0x7f273415d6e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f272c00c730 tx=0x7f272c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:13.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.665+0000 7f2731600640 1 -- 172.21.15.131:0/1218284894 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f272c016020 con 0x7f273415e340 2024-05-26T22:49:13.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.665+0000 7f2731600640 1 -- 172.21.15.131:0/1218284894 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f272c006d50 con 0x7f273415e340 2024-05-26T22:49:13.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.665+0000 7f2731600640 1 -- 172.21.15.131:0/1218284894 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f272c003bb0 con 0x7f273415e340 2024-05-26T22:49:13.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.665+0000 7f273ac00640 1 -- 172.21.15.131:0/1218284894 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27341549c0 con 0x7f273415e340 2024-05-26T22:49:13.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.666+0000 7f273ac00640 1 -- 172.21.15.131:0/1218284894 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27341554a0 con 0x7f273415e340 2024-05-26T22:49:13.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.667+0000 7f273ac00640 1 -- 172.21.15.131:0/1218284894 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27340694e0 con 0x7f273415e340 2024-05-26T22:49:13.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.668+0000 7f2731600640 1 -- 172.21.15.131:0/1218284894 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f272c00a7f0 con 0x7f273415e340 2024-05-26T22:49:13.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.668+0000 7f2731600640 1 --2- 172.21.15.131:0/1218284894 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f271406be30 0x7f271406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:13.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.668+0000 7f2731600640 1 -- 172.21.15.131:0/1218284894 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f272c08ab90 con 0x7f273415e340 2024-05-26T22:49:13.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.669+0000 7f2739800640 1 --2- 172.21.15.131:0/1218284894 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f271406be30 0x7f271406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:13.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.669+0000 7f2739800640 1 --2- 172.21.15.131:0/1218284894 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f271406be30 0x7f271406e2f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2728009aa0 tx=0x7f2728008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:13.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.673+0000 7f2731600640 1 -- 172.21.15.131:0/1218284894 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f272c020020 con 0x7f273415e340 2024-05-26T22:49:13.740 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-26T22:49:13.748 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-26T22:49:13.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.885+0000 7f273ac00640 1 -- 172.21.15.131:0/1218284894 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f2734160e50 con 0x7f273415e340 2024-05-26T22:49:13.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.886+0000 7f2731600640 1 -- 172.21.15.131:0/1218284894 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7f272c0594b0 con 0x7f273415e340 2024-05-26T22:49:13.886 INFO:teuthology.orchestra.run.smithi131.stderr:listed 0 entries 2024-05-26T22:49:13.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.889+0000 7f273ac00640 1 -- 172.21.15.131:0/1218284894 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f271406be30 msgr2=0x7f271406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:13.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.889+0000 7f273ac00640 1 --2- 172.21.15.131:0/1218284894 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f271406be30 0x7f271406e2f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2728009aa0 tx=0x7f2728008040 comp rx=0 tx=0).stop 2024-05-26T22:49:13.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.889+0000 7f273ac00640 1 -- 172.21.15.131:0/1218284894 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f273415e340 msgr2=0x7f273415d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:13.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.889+0000 7f273ac00640 1 --2- 172.21.15.131:0/1218284894 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f273415e340 0x7f273415d6e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f272c00c730 tx=0x7f272c00cbf0 comp rx=0 tx=0).stop 2024-05-26T22:49:13.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.889+0000 7f273ac00640 1 -- 172.21.15.131:0/1218284894 shutdown_connections 2024-05-26T22:49:13.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.889+0000 7f273ac00640 1 --2- 172.21.15.131:0/1218284894 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f271406be30 0x7f271406e2f0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:13.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.889+0000 7f273ac00640 1 --2- 172.21.15.131:0/1218284894 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f273415e340 0x7f273415d6e0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:13.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.889+0000 7f273ac00640 1 --2- 172.21.15.131:0/1218284894 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f27340d27f0 0x7f273415d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:13.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.889+0000 7f273ac00640 1 --2- 172.21.15.131:0/1218284894 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f27340b7820 0x7f2734150280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:13.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.889+0000 7f273ac00640 1 -- 172.21.15.131:0/1218284894 >> 172.21.15.131:0/1218284894 conn(0x7f2734072e60 msgr2=0x7f273415edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:13.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.889+0000 7f273ac00640 1 -- 172.21.15.131:0/1218284894 shutdown_connections 2024-05-26T22:49:13.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:13.889+0000 7f273ac00640 1 -- 172.21.15.131:0/1218284894 wait complete. 2024-05-26T22:49:13.908 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2024-05-26T22:49:13.908 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:49:13.908 DEBUG:teuthology.orchestra.run.smithi187:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2024-05-26T22:49:14.004 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi187.front.sepia.ceph.com 2024-05-26T22:49:14.004 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:49:14.004 DEBUG:teuthology.orchestra.run.smithi187:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-05-26T22:49:14.056 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-26T22:49:14.056 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-26T22:49:14.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.264+0000 7f5293e00640 1 Processor -- start 2024-05-26T22:49:14.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.264+0000 7f5293e00640 1 -- start start 2024-05-26T22:49:14.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.265+0000 7f5293e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f529406e8c0 0x7f529415d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:14.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.265+0000 7f5293e00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5294160620 con 0x7f5294156750 2024-05-26T22:49:14.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.266+0000 7f5293e00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5294160900 con 0x7f52940cc7f0 2024-05-26T22:49:14.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.266+0000 7f5293e00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f529415cbd0 con 0x7f529406e8c0 2024-05-26T22:49:14.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.266+0000 7f528a000640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f529406e8c0 0x7f529415d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:14.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.266+0000 7f5292a00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f5294156750 0x7f529415fd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:46728/0 (socket says 172.21.15.131:46728) 2024-05-26T22:49:14.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.266+0000 7f528a000640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f529406e8c0 0x7f529415d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42062/0 (socket says 172.21.15.131:42062) 2024-05-26T22:49:14.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.266+0000 7f5292a00640 1 -- 172.21.15.131:0/4210797538 learned_addr learned my addr 172.21.15.131:0/4210797538 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:49:14.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.266+0000 7f528a000640 1 -- 172.21.15.131:0/4210797538 >> v1:172.21.15.187:6789/0 conn(0x7f52940cc7f0 legacy=0x7f52940cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:14.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.266+0000 7f528a000640 1 -- 172.21.15.131:0/4210797538 >> v1:172.21.15.131:6789/0 conn(0x7f5294156750 legacy=0x7f529415fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:14.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.266+0000 7f528a000640 1 -- 172.21.15.131:0/4210797538 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5294160be0 con 0x7f529406e8c0 2024-05-26T22:49:14.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.267+0000 7f528a000640 1 --2- 172.21.15.131:0/4210797538 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f529406e8c0 0x7f529415d060 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f529415cd50 tx=0x7f527c02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95618d391979623f server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:14.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.268+0000 7f5292000640 1 -- 172.21.15.131:0/4210797538 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f527c037070 con 0x7f529406e8c0 2024-05-26T22:49:14.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.268+0000 7f5292000640 1 -- 172.21.15.131:0/4210797538 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f527c032440 con 0x7f529406e8c0 2024-05-26T22:49:14.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.268+0000 7f5292000640 1 -- 172.21.15.131:0/4210797538 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f527c032810 con 0x7f529406e8c0 2024-05-26T22:49:14.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.268+0000 7f5293e00640 1 -- 172.21.15.131:0/4210797538 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f529406e8c0 msgr2=0x7f529415d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:14.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.268+0000 7f5293e00640 1 --2- 172.21.15.131:0/4210797538 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f529406e8c0 0x7f529415d060 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f529415cd50 tx=0x7f527c02a1c0 comp rx=0 tx=0).stop 2024-05-26T22:49:14.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.268+0000 7f5293e00640 1 -- 172.21.15.131:0/4210797538 shutdown_connections 2024-05-26T22:49:14.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.268+0000 7f5293e00640 1 --2- 172.21.15.131:0/4210797538 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f529406e8c0 0x7f529415d060 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:14.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.268+0000 7f5293e00640 1 -- 172.21.15.131:0/4210797538 >> 172.21.15.131:0/4210797538 conn(0x7f52940cff40 msgr2=0x7f52940d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:14.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.268+0000 7f5293e00640 1 -- 172.21.15.131:0/4210797538 shutdown_connections 2024-05-26T22:49:14.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.268+0000 7f5293e00640 1 -- 172.21.15.131:0/4210797538 wait complete. 2024-05-26T22:49:14.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.270+0000 7f5293e00640 1 Processor -- start 2024-05-26T22:49:14.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.270+0000 7f5293e00640 1 -- start start 2024-05-26T22:49:14.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.271+0000 7f5293e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f529406e8c0 0x7f52940afa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:14.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.271+0000 7f5293e00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f52940cc7f0 0x7f5294143860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:14.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.271+0000 7f5293e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5294156750 0x7f52940b12e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:14.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.271+0000 7f5293e00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f529415d5a0 con 0x7f529406e8c0 2024-05-26T22:49:14.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.271+0000 7f5293e00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5294156b30 con 0x7f52940cc7f0 2024-05-26T22:49:14.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.271+0000 7f5293e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f529415d720 con 0x7f5294156750 2024-05-26T22:49:14.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.271+0000 7f5292a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f529406e8c0 0x7f52940afa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:14.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.271+0000 7f5292a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f529406e8c0 0x7f52940afa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42074/0 (socket says 172.21.15.131:42074) 2024-05-26T22:49:14.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.271+0000 7f5292a00640 1 -- 172.21.15.131:0/2198855617 learned_addr learned my addr 172.21.15.131:0/2198855617 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:14.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.272+0000 7f5292a00640 1 -- 172.21.15.131:0/2198855617 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5294156750 msgr2=0x7f52940b12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:14.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.272+0000 7f5292a00640 1 --2- 172.21.15.131:0/2198855617 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5294156750 0x7f52940b12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:14.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.272+0000 7f5292a00640 1 -- 172.21.15.131:0/2198855617 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f52940cc7f0 msgr2=0x7f5294143860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:14.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.272+0000 7f528a000640 1 --2- 172.21.15.131:0/2198855617 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f52940cc7f0 0x7f5294143860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:14.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.272+0000 7f5292a00640 1 --2- 172.21.15.131:0/2198855617 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f52940cc7f0 0x7f5294143860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:14.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.272+0000 7f5292a00640 1 -- 172.21.15.131:0/2198855617 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52940affc0 con 0x7f529406e8c0 2024-05-26T22:49:14.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.272+0000 7f5292a00640 1 --2- 172.21.15.131:0/2198855617 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f529406e8c0 0x7f52940afa80 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f528400c3f0 tx=0x7f528400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:14.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.272+0000 7f528b400640 1 -- 172.21.15.131:0/2198855617 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5284017070 con 0x7f529406e8c0 2024-05-26T22:49:14.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.273+0000 7f528b400640 1 -- 172.21.15.131:0/2198855617 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5284005ce0 con 0x7f529406e8c0 2024-05-26T22:49:14.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.273+0000 7f528b400640 1 -- 172.21.15.131:0/2198855617 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52840072f0 con 0x7f529406e8c0 2024-05-26T22:49:14.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.273+0000 7f5293e00640 1 -- 172.21.15.131:0/2198855617 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5294143da0 con 0x7f529406e8c0 2024-05-26T22:49:14.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.273+0000 7f5293e00640 1 -- 172.21.15.131:0/2198855617 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52940b0520 con 0x7f529406e8c0 2024-05-26T22:49:14.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.274+0000 7f5293e00640 1 -- 172.21.15.131:0/2198855617 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52940694e0 con 0x7f529406e8c0 2024-05-26T22:49:14.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.275+0000 7f528b400640 1 -- 172.21.15.131:0/2198855617 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f528401a070 con 0x7f529406e8c0 2024-05-26T22:49:14.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.276+0000 7f528b400640 1 --2- 172.21.15.131:0/2198855617 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f527006be30 0x7f527006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:14.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.276+0000 7f528b400640 1 -- 172.21.15.131:0/2198855617 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f528408a240 con 0x7f529406e8c0 2024-05-26T22:49:14.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.277+0000 7f528a000640 1 --2- 172.21.15.131:0/2198855617 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f527006be30 0x7f527006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:14.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.277+0000 7f528a000640 1 --2- 172.21.15.131:0/2198855617 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f527006be30 0x7f527006e2f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f527c034000 tx=0x7f527c02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:14.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.282+0000 7f528b400640 1 -- 172.21.15.131:0/2198855617 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f52940affc0 con 0x7f529406e8c0 2024-05-26T22:49:14.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.513+0000 7f5293e00640 1 -- 172.21.15.131:0/2198855617 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f52940b4ed0 con 0x7f529406e8c0 2024-05-26T22:49:14.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.515+0000 7f528b400640 1 -- 172.21.15.131:0/2198855617 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5184 (secure 0 0 0) 0x7f5284058b70 con 0x7f529406e8c0 2024-05-26T22:49:14.515 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:49:14.515 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/3266175650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6833","nonce":3266175650},{"type":"v1","addr":"172.21.15.187:6835","nonce":3266175650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/1862866305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1862866305},{"type":"v1","addr":"172.21.15.131:6835","nonce":1862866305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4269,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/618294324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":618294324},{"type":"v1","addr":"172.21.15.131:6837","nonce":618294324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-26T22:48:21.211047+0000","modified":"2024-05-26T22:48:24.550610+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6834/2547512415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":2547512415},{"type":"v1","addr":"172.21.15.187:6834","nonce":2547512415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4281,"qdb_cluster":[4281]},"id":1}]} 2024-05-26T22:49:14.515 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 7 2024-05-26T22:49:14.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.517+0000 7f5293e00640 1 -- 172.21.15.131:0/2198855617 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f527006be30 msgr2=0x7f527006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:14.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.517+0000 7f5293e00640 1 --2- 172.21.15.131:0/2198855617 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f527006be30 0x7f527006e2f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f527c034000 tx=0x7f527c02e000 comp rx=0 tx=0).stop 2024-05-26T22:49:14.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.517+0000 7f5293e00640 1 -- 172.21.15.131:0/2198855617 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f529406e8c0 msgr2=0x7f52940afa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:14.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.517+0000 7f5293e00640 1 --2- 172.21.15.131:0/2198855617 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f529406e8c0 0x7f52940afa80 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f528400c3f0 tx=0x7f528400c8b0 comp rx=0 tx=0).stop 2024-05-26T22:49:14.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.517+0000 7f5293e00640 1 -- 172.21.15.131:0/2198855617 shutdown_connections 2024-05-26T22:49:14.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.517+0000 7f5293e00640 1 --2- 172.21.15.131:0/2198855617 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5294156750 0x7f52940b12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:14.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.517+0000 7f5293e00640 1 --2- 172.21.15.131:0/2198855617 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f527006be30 0x7f527006e2f0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:14.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.517+0000 7f5293e00640 1 --2- 172.21.15.131:0/2198855617 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f52940cc7f0 0x7f5294143860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:14.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.517+0000 7f5293e00640 1 --2- 172.21.15.131:0/2198855617 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f529406e8c0 0x7f52940afa80 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:14.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.517+0000 7f5293e00640 1 -- 172.21.15.131:0/2198855617 >> 172.21.15.131:0/2198855617 conn(0x7f52940cff40 msgr2=0x7f529415b680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:14.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.518+0000 7f5293e00640 1 -- 172.21.15.131:0/2198855617 shutdown_connections 2024-05-26T22:49:14.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.518+0000 7f5293e00640 1 -- 172.21.15.131:0/2198855617 wait complete. 2024-05-26T22:49:14.536 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-26T22:49:14.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.738+0000 7f4f0b000640 1 Processor -- start 2024-05-26T22:49:14.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.738+0000 7f4f0b000640 1 -- start start 2024-05-26T22:49:14.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.739+0000 7f4f0b000640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f4f04155840 0x7f4f04071930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:14.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.739+0000 7f4f0b000640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f4f0415c3d0 0x7f4f0406ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:14.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.739+0000 7f4f0b000640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f040722a0 con 0x7f4f0406f490 2024-05-26T22:49:14.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.739+0000 7f4f0b000640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f4f04072580 con 0x7f4f04155840 2024-05-26T22:49:14.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.739+0000 7f4f0b000640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f4f04072860 con 0x7f4f0415c3d0 2024-05-26T22:49:14.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.740+0000 7f4f03e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f4f04155840 0x7f4f04071930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:14.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.740+0000 7f4f09000640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f4f0406f490 0x7f4f0415ec80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:49422/0 (socket says 172.21.15.131:49422) 2024-05-26T22:49:14.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.740+0000 7f4f09000640 1 -- 172.21.15.131:0/678014652 learned_addr learned my addr 172.21.15.131:0/678014652 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:49:14.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.740+0000 7f4f03e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f4f04155840 0x7f4f04071930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40628/0 (socket says 172.21.15.131:40628) 2024-05-26T22:49:14.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.740+0000 7f4f03e00640 1 -- 172.21.15.131:0/678014652 >> v2:172.21.15.187:3300/0 conn(0x7f4f0415c3d0 msgr2=0x7f4f0406ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:14.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.740+0000 7f4efbe00640 1 --2- 172.21.15.131:0/678014652 >> v2:172.21.15.187:3300/0 conn(0x7f4f0415c3d0 0x7f4f0406ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:14.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.740+0000 7f4f03400640 1 -- 172.21.15.131:0/678014652 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2553451670 0 0) 0x7f4f040722a0 con 0x7f4f0406f490 2024-05-26T22:49:14.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.740+0000 7f4f03e00640 1 --2- 172.21.15.131:0/678014652 >> v2:172.21.15.187:3300/0 conn(0x7f4f0415c3d0 0x7f4f0406ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:14.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.740+0000 7f4f03e00640 1 -- 172.21.15.131:0/678014652 >> v1:172.21.15.131:6789/0 conn(0x7f4f0406f490 legacy=0x7f4f0415ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:14.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.740+0000 7f4f03e00640 1 -- 172.21.15.131:0/678014652 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f04072b40 con 0x7f4f04155840 2024-05-26T22:49:14.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.740+0000 7f4efbe00640 1 --2- 172.21.15.131:0/678014652 >> v2:172.21.15.187:3300/0 conn(0x7f4f0415c3d0 0x7f4f0406ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:49:14.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.741+0000 7f4f03e00640 1 --2- 172.21.15.131:0/678014652 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f04155840 0x7f4f04071930 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f4f04159280 tx=0x7f4ef402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d314197bccdd3ad9 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:14.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.742+0000 7f4f03400640 1 -- 172.21.15.131:0/678014652 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef4037070 con 0x7f4f04155840 2024-05-26T22:49:14.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.742+0000 7f4f03400640 1 -- 172.21.15.131:0/678014652 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ef4032440 con 0x7f4f04155840 2024-05-26T22:49:14.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.742+0000 7f4f03400640 1 -- 172.21.15.131:0/678014652 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef4032810 con 0x7f4f04155840 2024-05-26T22:49:14.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.742+0000 7f4f0b000640 1 -- 172.21.15.131:0/678014652 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f04155840 msgr2=0x7f4f04071930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:14.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.742+0000 7f4f0b000640 1 --2- 172.21.15.131:0/678014652 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f04155840 0x7f4f04071930 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f4f04159280 tx=0x7f4ef402a1a0 comp rx=0 tx=0).stop 2024-05-26T22:49:14.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.743+0000 7f4f0b000640 1 -- 172.21.15.131:0/678014652 shutdown_connections 2024-05-26T22:49:14.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.743+0000 7f4f0b000640 1 --2- 172.21.15.131:0/678014652 >> v2:172.21.15.187:3300/0 conn(0x7f4f0415c3d0 0x7f4f0406ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:14.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.743+0000 7f4f0b000640 1 --2- 172.21.15.131:0/678014652 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f04155840 0x7f4f04071930 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:14.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.743+0000 7f4f0b000640 1 -- 172.21.15.131:0/678014652 >> 172.21.15.131:0/678014652 conn(0x7f4f040b1e90 msgr2=0x7f4f040b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:14.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.743+0000 7f4f0b000640 1 -- 172.21.15.131:0/678014652 shutdown_connections 2024-05-26T22:49:14.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.743+0000 7f4f0b000640 1 -- 172.21.15.131:0/678014652 wait complete. 2024-05-26T22:49:14.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.744+0000 7f4f0b000640 1 Processor -- start 2024-05-26T22:49:14.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.745+0000 7f4f0b000640 1 -- start start 2024-05-26T22:49:14.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.745+0000 7f4f0b000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f0406f490 0x7f4f0409c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:14.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.745+0000 7f4f0b000640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4f04155840 0x7f4f0414c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:14.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.745+0000 7f4f0b000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4f0415c3d0 0x7f4f0409a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:14.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.745+0000 7f4f0b000640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f4f04071ff0 con 0x7f4f0406f490 2024-05-26T22:49:14.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.745+0000 7f4f0b000640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4f04071e70 con 0x7f4f04155840 2024-05-26T22:49:14.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.745+0000 7f4f0b000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f4f040aee50 con 0x7f4f0415c3d0 2024-05-26T22:49:14.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.746+0000 7f4f03e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f0406f490 0x7f4f0409c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:14.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.746+0000 7f4f03e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f0406f490 0x7f4f0409c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40630/0 (socket says 172.21.15.131:40630) 2024-05-26T22:49:14.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.746+0000 7f4f03e00640 1 -- 172.21.15.131:0/1620371218 learned_addr learned my addr 172.21.15.131:0/1620371218 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:14.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.746+0000 7f4f03e00640 1 -- 172.21.15.131:0/1620371218 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4f0415c3d0 msgr2=0x7f4f0409a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:14.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.746+0000 7f4f03e00640 1 --2- 172.21.15.131:0/1620371218 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4f0415c3d0 0x7f4f0409a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:14.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.746+0000 7f4efbe00640 1 --2- 172.21.15.131:0/1620371218 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4f04155840 0x7f4f0414c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:14.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.746+0000 7f4f03e00640 1 -- 172.21.15.131:0/1620371218 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4f04155840 msgr2=0x7f4f0414c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:14.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.746+0000 7f4f03e00640 1 --2- 172.21.15.131:0/1620371218 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4f04155840 0x7f4f0414c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:14.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.746+0000 7f4f03e00640 1 -- 172.21.15.131:0/1620371218 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f0414ca40 con 0x7f4f0406f490 2024-05-26T22:49:14.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.746+0000 7f4f03e00640 1 --2- 172.21.15.131:0/1620371218 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f0406f490 0x7f4f0409c4f0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f4ef4000c00 tx=0x7f4ef40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:14.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.747+0000 7f4f00c00640 1 -- 172.21.15.131:0/1620371218 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef4037040 con 0x7f4f0406f490 2024-05-26T22:49:14.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.747+0000 7f4f00c00640 1 -- 172.21.15.131:0/1620371218 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ef4042070 con 0x7f4f0406f490 2024-05-26T22:49:14.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.747+0000 7f4f0b000640 1 -- 172.21.15.131:0/1620371218 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f0409b450 con 0x7f4f0406f490 2024-05-26T22:49:14.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.747+0000 7f4f00c00640 1 -- 172.21.15.131:0/1620371218 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef4030a50 con 0x7f4f0406f490 2024-05-26T22:49:14.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.747+0000 7f4f0b000640 1 -- 172.21.15.131:0/1620371218 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f0409ba10 con 0x7f4f0406f490 2024-05-26T22:49:14.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.748+0000 7f4f0b000640 1 -- 172.21.15.131:0/1620371218 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f0406d2a0 con 0x7f4f0406f490 2024-05-26T22:49:14.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.749+0000 7f4f00c00640 1 -- 172.21.15.131:0/1620371218 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4ef4030bf0 con 0x7f4f0406f490 2024-05-26T22:49:14.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.750+0000 7f4f00c00640 1 --2- 172.21.15.131:0/1620371218 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f4ee006be60 0x7f4ee006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:14.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.750+0000 7f4f00c00640 1 -- 172.21.15.131:0/1620371218 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f4ef40ab960 con 0x7f4f0406f490 2024-05-26T22:49:14.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.750+0000 7f4efbe00640 1 --2- 172.21.15.131:0/1620371218 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f4ee006be60 0x7f4ee006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:14.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.751+0000 7f4efbe00640 1 --2- 172.21.15.131:0/1620371218 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f4ee006be60 0x7f4ee006e320 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f4eec00b840 tx=0x7f4eec008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:14.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.754+0000 7f4f00c00640 1 -- 172.21.15.131:0/1620371218 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f4f0406d2a0 con 0x7f4f0406f490 2024-05-26T22:49:14.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.953+0000 7f4f0b000640 1 -- 172.21.15.131:0/1620371218 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4f040b0cf0 con 0x7f4f0406f490 2024-05-26T22:49:14.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.955+0000 7f4f00c00640 1 -- 172.21.15.131:0/1620371218 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16879 (secure 0 0 0) 0x7f4ef4045360 con 0x7f4f0406f490 2024-05-26T22:49:14.955 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:49:14.955 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":22,"fsid":"46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f","created":"2024-05-26T22:47:57.288905+0000","modified":"2024-05-26T22:49:04.278988+0000","last_up_change":"2024-05-26T22:48:09.602207+0000","last_in_change":"2024-05-26T22:48:00.687341+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-26T22:48:10.463339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-26T22:48:19.663254+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-26T22:48:20.235751+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"814eaf62-a18d-494b-8e27-fd8fb88adea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6801","nonce":2350940051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6803","nonce":2350940051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6807","nonce":2350940051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6805","nonce":2350940051}]},"public_addr":"172.21.15.131:6801/2350940051","cluster_addr":"172.21.15.131:6803/2350940051","heartbeat_back_addr":"172.21.15.131:6807/2350940051","heartbeat_front_addr":"172.21.15.131:6805/2350940051","state":["exists","up"]},{"osd":1,"uuid":"42e66287-f6c6-4e26-af3c-6dfaa3536d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6825","nonce":1525528247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6827","nonce":1525528247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6831","nonce":1525528247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6829","nonce":1525528247}]},"public_addr":"172.21.15.131:6825/1525528247","cluster_addr":"172.21.15.131:6827/1525528247","heartbeat_back_addr":"172.21.15.131:6831/1525528247","heartbeat_front_addr":"172.21.15.131:6829/1525528247","state":["exists","up"]},{"osd":2,"uuid":"0da34673-04a2-4fd0-b57a-0ff2b679b043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6817","nonce":3627842210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6819","nonce":3627842210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6823","nonce":3627842210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6821","nonce":3627842210}]},"public_addr":"172.21.15.131:6817/3627842210","cluster_addr":"172.21.15.131:6819/3627842210","heartbeat_back_addr":"172.21.15.131:6823/3627842210","heartbeat_front_addr":"172.21.15.131:6821/3627842210","state":["exists","up"]},{"osd":3,"uuid":"cf117a27-d6b2-43d3-ae52-b47988bf8eed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6809","nonce":350790591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6811","nonce":350790591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6815","nonce":350790591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6813","nonce":350790591}]},"public_addr":"172.21.15.131:6809/350790591","cluster_addr":"172.21.15.131:6811/350790591","heartbeat_back_addr":"172.21.15.131:6815/350790591","heartbeat_front_addr":"172.21.15.131:6813/350790591","state":["exists","up"]},{"osd":4,"uuid":"a092ccfc-be89-4a84-88d5-0f781fa715eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6816","nonce":244560995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6819","nonce":244560995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6823","nonce":244560995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6821","nonce":244560995}]},"public_addr":"172.21.15.187:6816/244560995","cluster_addr":"172.21.15.187:6819/244560995","heartbeat_back_addr":"172.21.15.187:6823/244560995","heartbeat_front_addr":"172.21.15.187:6821/244560995","state":["exists","up"]},{"osd":5,"uuid":"7513ba69-dd00-4a35-b190-9e8cc3b2bdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6801","nonce":324619398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6803","nonce":324619398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6812","nonce":324619398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6807","nonce":324619398}]},"public_addr":"172.21.15.187:6801/324619398","cluster_addr":"172.21.15.187:6803/324619398","heartbeat_back_addr":"172.21.15.187:6812/324619398","heartbeat_front_addr":"172.21.15.187:6807/324619398","state":["exists","up"]},{"osd":6,"uuid":"50081810-c093-4d2f-a4d7-b7c695c5400d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6806","nonce":1391347786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6809","nonce":1391347786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6818","nonce":1391347786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6813","nonce":1391347786}]},"public_addr":"172.21.15.187:6806/1391347786","cluster_addr":"172.21.15.187:6809/1391347786","heartbeat_back_addr":"172.21.15.187:6818/1391347786","heartbeat_front_addr":"172.21.15.187:6813/1391347786","state":["exists","up"]},{"osd":7,"uuid":"775dae05-54f5-4ad4-bbce-92863c6ed2ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6825","nonce":1220065622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6827","nonce":1220065622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6831","nonce":1220065622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6829","nonce":1220065622}]},"public_addr":"172.21.15.187:6825/1220065622","cluster_addr":"172.21.15.187:6827/1220065622","heartbeat_back_addr":"172.21.15.187:6831/1220065622","heartbeat_front_addr":"172.21.15.187:6829/1220065622","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:05.983520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.299516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.067323+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.307748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.018742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.258775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.010912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.295355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.1e","mappings":[{"from":3,"to":2}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":7,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-26T22:49:14.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.958+0000 7f4f0b000640 1 -- 172.21.15.131:0/1620371218 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f4ee006be60 msgr2=0x7f4ee006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:14.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.958+0000 7f4f0b000640 1 --2- 172.21.15.131:0/1620371218 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f4ee006be60 0x7f4ee006e320 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f4eec00b840 tx=0x7f4eec008040 comp rx=0 tx=0).stop 2024-05-26T22:49:14.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.958+0000 7f4f0b000640 1 -- 172.21.15.131:0/1620371218 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f0406f490 msgr2=0x7f4f0409c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:14.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.958+0000 7f4f0b000640 1 --2- 172.21.15.131:0/1620371218 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f0406f490 0x7f4f0409c4f0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f4ef4000c00 tx=0x7f4ef40046c0 comp rx=0 tx=0).stop 2024-05-26T22:49:14.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.958+0000 7f4f0b000640 1 -- 172.21.15.131:0/1620371218 shutdown_connections 2024-05-26T22:49:14.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.958+0000 7f4f0b000640 1 --2- 172.21.15.131:0/1620371218 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4f0415c3d0 0x7f4f0409a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:14.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.958+0000 7f4f0b000640 1 --2- 172.21.15.131:0/1620371218 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f4ee006be60 0x7f4ee006e320 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:14.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.958+0000 7f4f0b000640 1 --2- 172.21.15.131:0/1620371218 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4f04155840 0x7f4f0414c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:14.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.958+0000 7f4f0b000640 1 --2- 172.21.15.131:0/1620371218 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f0406f490 0x7f4f0409c4f0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:14.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.958+0000 7f4f0b000640 1 -- 172.21.15.131:0/1620371218 >> 172.21.15.131:0/1620371218 conn(0x7f4f040b1e90 msgr2=0x7f4f040b05e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:14.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.958+0000 7f4f0b000640 1 -- 172.21.15.131:0/1620371218 shutdown_connections 2024-05-26T22:49:14.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:14.958+0000 7f4f0b000640 1 -- 172.21.15.131:0/1620371218 wait complete. 2024-05-26T22:49:14.975 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-26T22:48:10.463339+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-26T22:48:19.663254+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-26T22:48:20.235751+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-26T22:49:14.976 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-26T22:49:15.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.181+0000 7fdc85200640 1 Processor -- start 2024-05-26T22:49:15.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.181+0000 7fdc85200640 1 -- start start 2024-05-26T22:49:15.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.183+0000 7fdc85200640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fdc80145a10 0x7fdc80072180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:15.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.183+0000 7fdc85200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdc800712d0 0x7fdc8006d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:15.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.183+0000 7fdc85200640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc80072d60 con 0x7fdc8006d790 2024-05-26T22:49:15.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.183+0000 7fdc85200640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fdc8015c760 con 0x7fdc80145a10 2024-05-26T22:49:15.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.183+0000 7fdc85200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fdc80072a80 con 0x7fdc800712d0 2024-05-26T22:49:15.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.183+0000 7fdc7fe00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fdc8006d790 0x7fdc8015fd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:49426/0 (socket says 172.21.15.131:49426) 2024-05-26T22:49:15.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.183+0000 7fdc7fe00640 1 -- 172.21.15.131:0/2070236600 learned_addr learned my addr 172.21.15.131:0/2070236600 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:49:15.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.183+0000 7fdc7f400640 1 --2- 172.21.15.131:0/2070236600 >> v2:172.21.15.187:3300/0 conn(0x7fdc80145a10 0x7fdc80072180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:15.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.183+0000 7fdc7ea00640 1 --2- 172.21.15.131:0/2070236600 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdc800712d0 0x7fdc8006d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:15.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.184+0000 7fdc7e000640 1 -- 172.21.15.131:0/2070236600 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 63069278 0 0) 0x7fdc80072d60 con 0x7fdc8006d790 2024-05-26T22:49:15.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.184+0000 7fdc7e000640 1 -- 172.21.15.131:0/2070236600 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdc64003520 con 0x7fdc8006d790 2024-05-26T22:49:15.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.184+0000 7fdc7e000640 1 -- 172.21.15.131:0/2070236600 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1348793543 0 0) 0x7fdc64003520 con 0x7fdc8006d790 2024-05-26T22:49:15.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.184+0000 7fdc7e000640 1 -- 172.21.15.131:0/2070236600 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdc800712d0 msgr2=0x7fdc8006d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:15.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.184+0000 7fdc7e000640 1 --2- 172.21.15.131:0/2070236600 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdc800712d0 0x7fdc8006d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:15.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.184+0000 7fdc7e000640 1 -- 172.21.15.131:0/2070236600 >> v2:172.21.15.187:3300/0 conn(0x7fdc80145a10 msgr2=0x7fdc80072180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:15.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.184+0000 7fdc7e000640 1 --2- 172.21.15.131:0/2070236600 >> v2:172.21.15.187:3300/0 conn(0x7fdc80145a10 0x7fdc80072180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:15.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.184+0000 7fdc7e000640 1 -- 172.21.15.131:0/2070236600 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc800ceab0 con 0x7fdc8006d790 2024-05-26T22:49:15.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.184+0000 7fdc7e000640 1 -- 172.21.15.131:0/2070236600 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7fdc780019f0 con 0x7fdc8006d790 2024-05-26T22:49:15.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.184+0000 7fdc7e000640 1 -- 172.21.15.131:0/2070236600 >> v1:172.21.15.131:6789/0 conn(0x7fdc8006d790 legacy=0x7fdc8015fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:15.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.184+0000 7fdc7e000640 1 --2- 172.21.15.131:0/2070236600 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdc800712d0 0x7fdc640039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:15.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.184+0000 7fdc7e000640 1 --2- 172.21.15.131:0/2070236600 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdc64003f30 0x7fdc64004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:15.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.184+0000 7fdc7e000640 1 --2- 172.21.15.131:0/2070236600 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdc64004d40 0x7fdc640051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:15.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.184+0000 7fdc7e000640 1 -- 172.21.15.131:0/2070236600 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fdc8015c760 con 0x7fdc64003f30 2024-05-26T22:49:15.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.184+0000 7fdc7e000640 1 -- 172.21.15.131:0/2070236600 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fdc80072a80 con 0x7fdc64004d40 2024-05-26T22:49:15.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.184+0000 7fdc7e000640 1 -- 172.21.15.131:0/2070236600 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fdc6400b9c0 con 0x7fdc800712d0 2024-05-26T22:49:15.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.185+0000 7fdc7ea00640 1 --2- 172.21.15.131:0/2070236600 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdc64003f30 0x7fdc64004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:15.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.185+0000 7fdc7f400640 1 --2- 172.21.15.131:0/2070236600 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdc800712d0 0x7fdc640039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:15.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.185+0000 7fdc7fe00640 1 --2- 172.21.15.131:0/2070236600 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdc64004d40 0x7fdc640051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:15.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.185+0000 7fdc7ea00640 1 -- 172.21.15.131:0/2070236600 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdc800712d0 msgr2=0x7fdc640039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:15.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.185+0000 7fdc7ea00640 1 --2- 172.21.15.131:0/2070236600 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdc800712d0 0x7fdc640039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:15.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.185+0000 7fdc7ea00640 1 -- 172.21.15.131:0/2070236600 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdc64004d40 msgr2=0x7fdc640051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:15.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.185+0000 7fdc7ea00640 1 --2- 172.21.15.131:0/2070236600 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdc64004d40 0x7fdc640051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:15.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.185+0000 7fdc7ea00640 1 -- 172.21.15.131:0/2070236600 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc64003520 con 0x7fdc64003f30 2024-05-26T22:49:15.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.185+0000 7fdc7fe00640 1 --2- 172.21.15.131:0/2070236600 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdc64004d40 0x7fdc640051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:49:15.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.186+0000 7fdc7ea00640 1 --2- 172.21.15.131:0/2070236600 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdc64003f30 0x7fdc64004800 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c009ac0 tx=0x7fdc64005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f10b5ea07eb94548 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:15.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.186+0000 7fdc7e000640 1 -- 172.21.15.131:0/2070236600 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc6c036070 con 0x7fdc64003f30 2024-05-26T22:49:15.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.186+0000 7fdc7e000640 1 -- 172.21.15.131:0/2070236600 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc6c031400 con 0x7fdc64003f30 2024-05-26T22:49:15.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.186+0000 7fdc7e000640 1 -- 172.21.15.131:0/2070236600 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc6c009690 con 0x7fdc64003f30 2024-05-26T22:49:15.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.187+0000 7fdc85200640 1 -- 172.21.15.131:0/2070236600 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdc64003f30 msgr2=0x7fdc64004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:15.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.187+0000 7fdc85200640 1 --2- 172.21.15.131:0/2070236600 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdc64003f30 0x7fdc64004800 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c009ac0 tx=0x7fdc64005830 comp rx=0 tx=0).stop 2024-05-26T22:49:15.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.187+0000 7fdc7fe00640 1 -- 172.21.15.131:0/2070236600 reap_dead start 2024-05-26T22:49:15.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.187+0000 7fdc85200640 1 -- 172.21.15.131:0/2070236600 shutdown_connections 2024-05-26T22:49:15.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.187+0000 7fdc85200640 1 -- 172.21.15.131:0/2070236600 >> 172.21.15.131:0/2070236600 conn(0x7fdc800b5660 msgr2=0x7fdc800b5a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:15.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.187+0000 7fdc85200640 1 -- 172.21.15.131:0/2070236600 shutdown_connections 2024-05-26T22:49:15.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.187+0000 7fdc85200640 1 -- 172.21.15.131:0/2070236600 wait complete. 2024-05-26T22:49:15.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.189+0000 7fdc85200640 1 Processor -- start 2024-05-26T22:49:15.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.189+0000 7fdc85200640 1 -- start start 2024-05-26T22:49:15.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.191+0000 7fdc85200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdc8015b940 0x7fdc801603e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:15.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.191+0000 7fdc85200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdc80135580 0x7fdc8015e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:15.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.191+0000 7fdc85200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdc8015e770 0x7fdc800a9460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:15.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.191+0000 7fdc85200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fdc80145df0 con 0x7fdc80135580 2024-05-26T22:49:15.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.191+0000 7fdc85200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fdc800cf070 con 0x7fdc8015b940 2024-05-26T22:49:15.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.191+0000 7fdc85200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fdc800726c0 con 0x7fdc8015e770 2024-05-26T22:49:15.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.191+0000 7fdc7ea00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdc80135580 0x7fdc8015e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:15.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.191+0000 7fdc7ea00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdc80135580 0x7fdc8015e230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40660/0 (socket says 172.21.15.131:40660) 2024-05-26T22:49:15.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.191+0000 7fdc7ea00640 1 -- 172.21.15.131:0/1876284103 learned_addr learned my addr 172.21.15.131:0/1876284103 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:15.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.191+0000 7fdc7f400640 1 --2- 172.21.15.131:0/1876284103 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdc8015b940 0x7fdc801603e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:15.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.191+0000 7fdc7fe00640 1 --2- 172.21.15.131:0/1876284103 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdc8015e770 0x7fdc800a9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:15.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.191+0000 7fdc7ea00640 1 -- 172.21.15.131:0/1876284103 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdc8015e770 msgr2=0x7fdc800a9460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:15.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.191+0000 7fdc7ea00640 1 --2- 172.21.15.131:0/1876284103 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdc8015e770 0x7fdc800a9460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:15.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.191+0000 7fdc7ea00640 1 -- 172.21.15.131:0/1876284103 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdc8015b940 msgr2=0x7fdc801603e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:15.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.191+0000 7fdc7ea00640 1 --2- 172.21.15.131:0/1876284103 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdc8015b940 0x7fdc801603e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:15.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.191+0000 7fdc7ea00640 1 -- 172.21.15.131:0/1876284103 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc800ac9b0 con 0x7fdc80135580 2024-05-26T22:49:15.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.192+0000 7fdc7ea00640 1 --2- 172.21.15.131:0/1876284103 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdc80135580 0x7fdc8015e230 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c02ab10 tx=0x7fdc6c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:15.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.192+0000 7fdc77400640 1 -- 172.21.15.131:0/1876284103 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc6c03f020 con 0x7fdc80135580 2024-05-26T22:49:15.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.192+0000 7fdc77400640 1 -- 172.21.15.131:0/1876284103 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc6c037040 con 0x7fdc80135580 2024-05-26T22:49:15.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.192+0000 7fdc85200640 1 -- 172.21.15.131:0/1876284103 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc800acc90 con 0x7fdc80135580 2024-05-26T22:49:15.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.192+0000 7fdc77400640 1 -- 172.21.15.131:0/1876284103 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc6c036070 con 0x7fdc80135580 2024-05-26T22:49:15.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.192+0000 7fdc85200640 1 -- 172.21.15.131:0/1876284103 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc800ad250 con 0x7fdc80135580 2024-05-26T22:49:15.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.193+0000 7fdc85200640 1 -- 172.21.15.131:0/1876284103 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc800694e0 con 0x7fdc80135580 2024-05-26T22:49:15.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.194+0000 7fdc77400640 1 -- 172.21.15.131:0/1876284103 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdc6c042070 con 0x7fdc80135580 2024-05-26T22:49:15.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.195+0000 7fdc77400640 1 --2- 172.21.15.131:0/1876284103 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fdc6006be60 0x7fdc6006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:15.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.195+0000 7fdc77400640 1 -- 172.21.15.131:0/1876284103 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fdc6c0aa180 con 0x7fdc80135580 2024-05-26T22:49:15.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.195+0000 7fdc7f400640 1 --2- 172.21.15.131:0/1876284103 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fdc6006be60 0x7fdc6006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:15.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.196+0000 7fdc7f400640 1 --2- 172.21.15.131:0/1876284103 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fdc6006be60 0x7fdc6006e320 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fdc700099d0 tx=0x7fdc70008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:15.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.199+0000 7fdc77400640 1 -- 172.21.15.131:0/1876284103 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fdc6c0044e0 con 0x7fdc80135580 2024-05-26T22:49:15.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.399+0000 7fdc85200640 1 -- 172.21.15.131:0/1876284103 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fdc800b5c10 con 0x7fdc80135580 2024-05-26T22:49:15.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.400+0000 7fdc77400640 1 -- 172.21.15.131:0/1876284103 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7fdc6c078aa0 con 0x7fdc80135580 2024-05-26T22:49:15.401 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2024-05-26T22:49:15.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.403+0000 7fdc85200640 1 -- 172.21.15.131:0/1876284103 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fdc6006be60 msgr2=0x7fdc6006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:15.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.403+0000 7fdc85200640 1 --2- 172.21.15.131:0/1876284103 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fdc6006be60 0x7fdc6006e320 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fdc700099d0 tx=0x7fdc70008040 comp rx=0 tx=0).stop 2024-05-26T22:49:15.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.403+0000 7fdc85200640 1 -- 172.21.15.131:0/1876284103 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdc80135580 msgr2=0x7fdc8015e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:15.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.403+0000 7fdc85200640 1 --2- 172.21.15.131:0/1876284103 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdc80135580 0x7fdc8015e230 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c02ab10 tx=0x7fdc6c004730 comp rx=0 tx=0).stop 2024-05-26T22:49:15.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.403+0000 7fdc85200640 1 -- 172.21.15.131:0/1876284103 shutdown_connections 2024-05-26T22:49:15.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.403+0000 7fdc85200640 1 --2- 172.21.15.131:0/1876284103 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdc8015e770 0x7fdc800a9460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:15.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.403+0000 7fdc85200640 1 --2- 172.21.15.131:0/1876284103 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fdc6006be60 0x7fdc6006e320 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:15.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.403+0000 7fdc85200640 1 --2- 172.21.15.131:0/1876284103 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdc80135580 0x7fdc8015e230 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:15.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.403+0000 7fdc85200640 1 --2- 172.21.15.131:0/1876284103 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdc8015b940 0x7fdc801603e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:15.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.403+0000 7fdc85200640 1 -- 172.21.15.131:0/1876284103 >> 172.21.15.131:0/1876284103 conn(0x7fdc800b5660 msgr2=0x7fdc8015c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:15.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.404+0000 7fdc85200640 1 -- 172.21.15.131:0/1876284103 shutdown_connections 2024-05-26T22:49:15.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.404+0000 7fdc85200640 1 -- 172.21.15.131:0/1876284103 wait complete. 2024-05-26T22:49:15.420 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-26T22:49:15.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.621+0000 7f3a53e00640 1 Processor -- start 2024-05-26T22:49:15.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.621+0000 7f3a53e00640 1 -- start start 2024-05-26T22:49:15.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.622+0000 7f3a53e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a540712d0 0x7f3a5406d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:15.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.622+0000 7f3a53e00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f3a5406d790 0x7f3a540cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:15.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.622+0000 7f3a53e00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a54072d60 con 0x7f3a54145a10 2024-05-26T22:49:15.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.623+0000 7f3a53e00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f3a5415c760 con 0x7f3a5406d790 2024-05-26T22:49:15.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.623+0000 7f3a53e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3a54072a80 con 0x7f3a540712d0 2024-05-26T22:49:15.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.623+0000 7f3a52a00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f3a54145a10 0x7f3a54072180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:49434/0 (socket says 172.21.15.131:49434) 2024-05-26T22:49:15.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.623+0000 7f3a52a00640 1 -- 172.21.15.131:0/3964184975 learned_addr learned my addr 172.21.15.131:0/3964184975 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:49:15.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.623+0000 7f3a53400640 1 --2- 172.21.15.131:0/3964184975 >> v2:172.21.15.187:3300/0 conn(0x7f3a5406d790 0x7f3a540cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:15.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.623+0000 7f3a52000640 1 --2- 172.21.15.131:0/3964184975 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a540712d0 0x7f3a5406d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:15.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.623+0000 7f3a51600640 1 -- 172.21.15.131:0/3964184975 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 109674220 0 0) 0x7f3a54072d60 con 0x7f3a54145a10 2024-05-26T22:49:15.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.623+0000 7f3a51600640 1 -- 172.21.15.131:0/3964184975 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3a38003520 con 0x7f3a54145a10 2024-05-26T22:49:15.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.624+0000 7f3a51600640 1 -- 172.21.15.131:0/3964184975 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2123948187 0 0) 0x7f3a38003520 con 0x7f3a54145a10 2024-05-26T22:49:15.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.624+0000 7f3a51600640 1 -- 172.21.15.131:0/3964184975 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a540712d0 msgr2=0x7f3a5406d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:15.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.624+0000 7f3a51600640 1 --2- 172.21.15.131:0/3964184975 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a540712d0 0x7f3a5406d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:15.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.624+0000 7f3a51600640 1 -- 172.21.15.131:0/3964184975 >> v2:172.21.15.187:3300/0 conn(0x7f3a5406d790 msgr2=0x7f3a540cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:15.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.624+0000 7f3a51600640 1 --2- 172.21.15.131:0/3964184975 >> v2:172.21.15.187:3300/0 conn(0x7f3a5406d790 0x7f3a540cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:15.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.624+0000 7f3a51600640 1 -- 172.21.15.131:0/3964184975 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a540ceab0 con 0x7f3a54145a10 2024-05-26T22:49:15.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.624+0000 7f3a51600640 1 -- 172.21.15.131:0/3964184975 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7f3a44001670 con 0x7f3a54145a10 2024-05-26T22:49:15.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.624+0000 7f3a51600640 1 -- 172.21.15.131:0/3964184975 >> v1:172.21.15.131:6789/0 conn(0x7f3a54145a10 legacy=0x7f3a54072180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:15.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.624+0000 7f3a51600640 1 --2- 172.21.15.131:0/3964184975 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a380039b0 0x7f3a38003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:15.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.624+0000 7f3a51600640 1 --2- 172.21.15.131:0/3964184975 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a540712d0 0x7f3a38004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:15.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.624+0000 7f3a51600640 1 --2- 172.21.15.131:0/3964184975 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a38004d40 0x7f3a380051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:15.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.624+0000 7f3a51600640 1 -- 172.21.15.131:0/3964184975 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f3a5415c760 con 0x7f3a38004d40 2024-05-26T22:49:15.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.624+0000 7f3a51600640 1 -- 172.21.15.131:0/3964184975 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3a54072a80 con 0x7f3a380039b0 2024-05-26T22:49:15.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.624+0000 7f3a51600640 1 -- 172.21.15.131:0/3964184975 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3a3800b9c0 con 0x7f3a540712d0 2024-05-26T22:49:15.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.624+0000 7f3a53400640 1 --2- 172.21.15.131:0/3964184975 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a38004d40 0x7f3a380051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:15.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.625+0000 7f3a52a00640 1 --2- 172.21.15.131:0/3964184975 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a380039b0 0x7f3a38003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:15.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.625+0000 7f3a53400640 1 -- 172.21.15.131:0/3964184975 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a540712d0 msgr2=0x7f3a38004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:15.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.625+0000 7f3a53400640 1 --2- 172.21.15.131:0/3964184975 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a540712d0 0x7f3a38004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:15.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.625+0000 7f3a53400640 1 -- 172.21.15.131:0/3964184975 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a380039b0 msgr2=0x7f3a38003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:15.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.625+0000 7f3a53400640 1 --2- 172.21.15.131:0/3964184975 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a380039b0 0x7f3a38003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:15.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.625+0000 7f3a53400640 1 -- 172.21.15.131:0/3964184975 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a38003520 con 0x7f3a38004d40 2024-05-26T22:49:15.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.626+0000 7f3a53400640 1 --2- 172.21.15.131:0/3964184975 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a38004d40 0x7f3a380051c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f3a4c009fc0 tx=0x7f3a38005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f84b69e85d088f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:15.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.626+0000 7f3a51600640 1 -- 172.21.15.131:0/3964184975 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a4c009c10 con 0x7f3a38004d40 2024-05-26T22:49:15.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.626+0000 7f3a51600640 1 -- 172.21.15.131:0/3964184975 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a4c036070 con 0x7f3a38004d40 2024-05-26T22:49:15.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.626+0000 7f3a51600640 1 -- 172.21.15.131:0/3964184975 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a4c02d070 con 0x7f3a38004d40 2024-05-26T22:49:15.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.627+0000 7f3a53e00640 1 -- 172.21.15.131:0/3964184975 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a38004d40 msgr2=0x7f3a380051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:15.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.627+0000 7f3a53e00640 1 --2- 172.21.15.131:0/3964184975 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a38004d40 0x7f3a380051c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f3a4c009fc0 tx=0x7f3a38005830 comp rx=0 tx=0).stop 2024-05-26T22:49:15.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.627+0000 7f3a53400640 1 -- 172.21.15.131:0/3964184975 reap_dead start 2024-05-26T22:49:15.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.627+0000 7f3a53e00640 1 -- 172.21.15.131:0/3964184975 shutdown_connections 2024-05-26T22:49:15.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.627+0000 7f3a53e00640 1 -- 172.21.15.131:0/3964184975 >> 172.21.15.131:0/3964184975 conn(0x7f3a540b5660 msgr2=0x7f3a540b5a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:15.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.627+0000 7f3a53e00640 1 -- 172.21.15.131:0/3964184975 shutdown_connections 2024-05-26T22:49:15.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.627+0000 7f3a53e00640 1 -- 172.21.15.131:0/3964184975 wait complete. 2024-05-26T22:49:15.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.629+0000 7f3a53e00640 1 Processor -- start 2024-05-26T22:49:15.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.629+0000 7f3a53e00640 1 -- start start 2024-05-26T22:49:15.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.630+0000 7f3a53e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a38004d40 0x7f3a5412b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:15.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.630+0000 7f3a53e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a54130c20 0x7f3a541f1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:15.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.630+0000 7f3a53e00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a541f2210 0x7f3a541f5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:15.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.630+0000 7f3a53e00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f3a54145df0 con 0x7f3a54130c20 2024-05-26T22:49:15.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.630+0000 7f3a53e00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3a540cf070 con 0x7f3a541f2210 2024-05-26T22:49:15.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.630+0000 7f3a53e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3a5406d250 con 0x7f3a38004d40 2024-05-26T22:49:15.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.630+0000 7f3a52000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a54130c20 0x7f3a541f1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:15.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.630+0000 7f3a52000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a54130c20 0x7f3a541f1cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40668/0 (socket says 172.21.15.131:40668) 2024-05-26T22:49:15.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.630+0000 7f3a52000640 1 -- 172.21.15.131:0/2594849321 learned_addr learned my addr 172.21.15.131:0/2594849321 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:15.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.630+0000 7f3a52000640 1 -- 172.21.15.131:0/2594849321 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a38004d40 msgr2=0x7f3a5412b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:15.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.630+0000 7f3a52000640 1 --2- 172.21.15.131:0/2594849321 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a38004d40 0x7f3a5412b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:15.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.630+0000 7f3a52000640 1 -- 172.21.15.131:0/2594849321 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a541f2210 msgr2=0x7f3a541f5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:15.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.630+0000 7f3a52000640 1 --2- 172.21.15.131:0/2594849321 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a541f2210 0x7f3a541f5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:15.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.630+0000 7f3a52000640 1 -- 172.21.15.131:0/2594849321 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a541311c0 con 0x7f3a54130c20 2024-05-26T22:49:15.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.631+0000 7f3a52000640 1 --2- 172.21.15.131:0/2594849321 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a54130c20 0x7f3a541f1cd0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f3a4000e850 tx=0x7f3a4000ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:15.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.631+0000 7f3a4b400640 1 -- 172.21.15.131:0/2594849321 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a40017070 con 0x7f3a54130c20 2024-05-26T22:49:15.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.631+0000 7f3a4b400640 1 -- 172.21.15.131:0/2594849321 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a4000c860 con 0x7f3a54130c20 2024-05-26T22:49:15.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.631+0000 7f3a4b400640 1 -- 172.21.15.131:0/2594849321 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a4000cc30 con 0x7f3a54130c20 2024-05-26T22:49:15.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.631+0000 7f3a53e00640 1 -- 172.21.15.131:0/2594849321 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a541f26c0 con 0x7f3a54130c20 2024-05-26T22:49:15.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.631+0000 7f3a53e00640 1 -- 172.21.15.131:0/2594849321 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a5413d960 con 0x7f3a54130c20 2024-05-26T22:49:15.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.632+0000 7f3a53e00640 1 -- 172.21.15.131:0/2594849321 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a540694e0 con 0x7f3a54130c20 2024-05-26T22:49:15.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.633+0000 7f3a4b400640 1 -- 172.21.15.131:0/2594849321 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3a4001a070 con 0x7f3a54130c20 2024-05-26T22:49:15.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.634+0000 7f3a4b400640 1 --2- 172.21.15.131:0/2594849321 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f3a3406be60 0x7f3a3406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:15.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.634+0000 7f3a4b400640 1 -- 172.21.15.131:0/2594849321 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f3a4008acb0 con 0x7f3a54130c20 2024-05-26T22:49:15.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.634+0000 7f3a52a00640 1 --2- 172.21.15.131:0/2594849321 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f3a3406be60 0x7f3a3406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:15.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.635+0000 7f3a52a00640 1 --2- 172.21.15.131:0/2594849321 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f3a3406be60 0x7f3a3406e320 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3a54160cb0 tx=0x7f3a44009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:15.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.638+0000 7f3a4b400640 1 -- 172.21.15.131:0/2594849321 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f3a400595d0 con 0x7f3a54130c20 2024-05-26T22:49:15.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.835+0000 7f3a53e00640 1 -- 172.21.15.131:0/2594849321 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f3a540afca0 con 0x7f3a54130c20 2024-05-26T22:49:15.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.836+0000 7f3a4b400640 1 -- 172.21.15.131:0/2594849321 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7f3a40058ed0 con 0x7f3a54130c20 2024-05-26T22:49:15.836 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-05-26T22:49:15.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.838+0000 7f3a53e00640 1 -- 172.21.15.131:0/2594849321 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f3a3406be60 msgr2=0x7f3a3406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:15.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.838+0000 7f3a53e00640 1 --2- 172.21.15.131:0/2594849321 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f3a3406be60 0x7f3a3406e320 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3a54160cb0 tx=0x7f3a44009040 comp rx=0 tx=0).stop 2024-05-26T22:49:15.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.838+0000 7f3a53e00640 1 -- 172.21.15.131:0/2594849321 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a54130c20 msgr2=0x7f3a541f1cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:15.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.838+0000 7f3a53e00640 1 --2- 172.21.15.131:0/2594849321 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a54130c20 0x7f3a541f1cd0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f3a4000e850 tx=0x7f3a4000ed10 comp rx=0 tx=0).stop 2024-05-26T22:49:15.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.839+0000 7f3a53e00640 1 -- 172.21.15.131:0/2594849321 shutdown_connections 2024-05-26T22:49:15.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.839+0000 7f3a53e00640 1 --2- 172.21.15.131:0/2594849321 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f3a3406be60 0x7f3a3406e320 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:15.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.839+0000 7f3a53e00640 1 --2- 172.21.15.131:0/2594849321 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a541f2210 0x7f3a541f5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:15.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.839+0000 7f3a53e00640 1 --2- 172.21.15.131:0/2594849321 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a54130c20 0x7f3a541f1cd0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:15.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.839+0000 7f3a53e00640 1 --2- 172.21.15.131:0/2594849321 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a38004d40 0x7f3a5412b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:15.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.839+0000 7f3a53e00640 1 -- 172.21.15.131:0/2594849321 >> 172.21.15.131:0/2594849321 conn(0x7f3a540b5660 msgr2=0x7f3a5413fd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:15.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.839+0000 7f3a53e00640 1 -- 172.21.15.131:0/2594849321 shutdown_connections 2024-05-26T22:49:15.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:15.839+0000 7f3a53e00640 1 -- 172.21.15.131:0/2594849321 wait complete. 2024-05-26T22:49:15.858 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-26T22:49:16.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.062+0000 7fb676600640 1 Processor -- start 2024-05-26T22:49:16.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.063+0000 7fb676600640 1 -- start start 2024-05-26T22:49:16.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.064+0000 7fb676600640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb67015e340 0x7fb6700b72e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:16.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.064+0000 7fb676600640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fb6700b7820 0x7fb6700d22b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:16.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.064+0000 7fb676600640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fb6700d27f0 0x7fb6700b8d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:16.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.064+0000 7fb676600640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fb6700b88b0 con 0x7fb6700d27f0 2024-05-26T22:49:16.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.064+0000 7fb676600640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fb67015e720 con 0x7fb6700b7820 2024-05-26T22:49:16.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.064+0000 7fb676600640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb6700b7ca0 con 0x7fb67015e340 2024-05-26T22:49:16.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.064+0000 7fb675c00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fb6700d27f0 0x7fb6700b8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:16.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.064+0000 7fb675c00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fb6700d27f0 0x7fb6700b8d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40682/0 (socket says 172.21.15.131:40682) 2024-05-26T22:49:16.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.064+0000 7fb675c00640 1 -- 172.21.15.131:0/24989957 learned_addr learned my addr 172.21.15.131:0/24989957 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:16.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.065+0000 7fb66fe00640 1 --2- 172.21.15.131:0/24989957 >> v2:172.21.15.187:3300/0 conn(0x7fb6700b7820 0x7fb6700d22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:16.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.065+0000 7fb675c00640 1 -- 172.21.15.131:0/24989957 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb67015e340 msgr2=0x7fb6700b72e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:16.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.065+0000 7fb675c00640 1 --2- 172.21.15.131:0/24989957 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb67015e340 0x7fb6700b72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:16.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.065+0000 7fb675c00640 1 -- 172.21.15.131:0/24989957 >> v2:172.21.15.187:3300/0 conn(0x7fb6700b7820 msgr2=0x7fb6700d22b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:16.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.065+0000 7fb675c00640 1 --2- 172.21.15.131:0/24989957 >> v2:172.21.15.187:3300/0 conn(0x7fb6700b7820 0x7fb6700d22b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:16.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.065+0000 7fb675c00640 1 -- 172.21.15.131:0/24989957 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6700b7f80 con 0x7fb6700d27f0 2024-05-26T22:49:16.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.065+0000 7fb66fe00640 1 --2- 172.21.15.131:0/24989957 >> v2:172.21.15.187:3300/0 conn(0x7fb6700b7820 0x7fb6700d22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:49:16.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.066+0000 7fb675c00640 1 --2- 172.21.15.131:0/24989957 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6700d27f0 0x7fb6700b8d40 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fb6700b7eb0 tx=0x7fb66802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f588c73ff78a965 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:16.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.066+0000 7fb66f400640 1 -- 172.21.15.131:0/24989957 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb668036070 con 0x7fb6700d27f0 2024-05-26T22:49:16.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.066+0000 7fb66f400640 1 -- 172.21.15.131:0/24989957 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb668031600 con 0x7fb6700d27f0 2024-05-26T22:49:16.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.066+0000 7fb66f400640 1 -- 172.21.15.131:0/24989957 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6680319d0 con 0x7fb6700d27f0 2024-05-26T22:49:16.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.067+0000 7fb676600640 1 -- 172.21.15.131:0/24989957 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6700d27f0 msgr2=0x7fb6700b8d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:16.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.067+0000 7fb676600640 1 --2- 172.21.15.131:0/24989957 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6700d27f0 0x7fb6700b8d40 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fb6700b7eb0 tx=0x7fb66802a710 comp rx=0 tx=0).stop 2024-05-26T22:49:16.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.067+0000 7fb676600640 1 -- 172.21.15.131:0/24989957 shutdown_connections 2024-05-26T22:49:16.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.067+0000 7fb676600640 1 --2- 172.21.15.131:0/24989957 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6700d27f0 0x7fb6700b8d40 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:16.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.067+0000 7fb676600640 1 --2- 172.21.15.131:0/24989957 >> v2:172.21.15.187:3300/0 conn(0x7fb6700b7820 0x7fb6700d22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:16.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.067+0000 7fb676600640 1 --2- 172.21.15.131:0/24989957 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb67015e340 0x7fb6700b72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:16.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.067+0000 7fb676600640 1 -- 172.21.15.131:0/24989957 >> 172.21.15.131:0/24989957 conn(0x7fb670072e60 msgr2=0x7fb670073270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:16.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.067+0000 7fb676600640 1 -- 172.21.15.131:0/24989957 shutdown_connections 2024-05-26T22:49:16.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.067+0000 7fb676600640 1 -- 172.21.15.131:0/24989957 wait complete. 2024-05-26T22:49:16.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.069+0000 7fb676600640 1 Processor -- start 2024-05-26T22:49:16.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.069+0000 7fb676600640 1 -- start start 2024-05-26T22:49:16.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.070+0000 7fb676600640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb6700b7820 0x7fb670144e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:16.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.070+0000 7fb676600640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb6700d27f0 0x7fb6701f3a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:16.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.070+0000 7fb676600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb67015e340 0x7fb670151f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:16.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.070+0000 7fb676600640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fb6700b9f20 con 0x7fb67015e340 2024-05-26T22:49:16.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.070+0000 7fb676600640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb6700b8540 con 0x7fb6700d27f0 2024-05-26T22:49:16.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.070+0000 7fb676600640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb6700ba0a0 con 0x7fb6700b7820 2024-05-26T22:49:16.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.070+0000 7fb675c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb67015e340 0x7fb670151f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:16.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.070+0000 7fb675c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb67015e340 0x7fb670151f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40686/0 (socket says 172.21.15.131:40686) 2024-05-26T22:49:16.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.070+0000 7fb675c00640 1 -- 172.21.15.131:0/3720726770 learned_addr learned my addr 172.21.15.131:0/3720726770 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:16.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.071+0000 7fb66fe00640 1 --2- 172.21.15.131:0/3720726770 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb6700d27f0 0x7fb6701f3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:16.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.071+0000 7fb675200640 1 --2- 172.21.15.131:0/3720726770 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb6700b7820 0x7fb670144e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:16.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.071+0000 7fb675c00640 1 -- 172.21.15.131:0/3720726770 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb6700b7820 msgr2=0x7fb670144e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:16.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.071+0000 7fb675c00640 1 --2- 172.21.15.131:0/3720726770 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb6700b7820 0x7fb670144e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:16.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.071+0000 7fb675c00640 1 -- 172.21.15.131:0/3720726770 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb6700d27f0 msgr2=0x7fb6701f3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:16.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.071+0000 7fb675c00640 1 --2- 172.21.15.131:0/3720726770 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb6700d27f0 0x7fb6701f3a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:16.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.071+0000 7fb675c00640 1 -- 172.21.15.131:0/3720726770 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb670150f20 con 0x7fb67015e340 2024-05-26T22:49:16.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.071+0000 7fb675c00640 1 --2- 172.21.15.131:0/3720726770 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb67015e340 0x7fb670151f50 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb66802ad00 tx=0x7fb668004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:16.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.071+0000 7fb66cc00640 1 -- 172.21.15.131:0/3720726770 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb668036050 con 0x7fb67015e340 2024-05-26T22:49:16.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.071+0000 7fb66cc00640 1 -- 172.21.15.131:0/3720726770 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb668042070 con 0x7fb67015e340 2024-05-26T22:49:16.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.071+0000 7fb66cc00640 1 -- 172.21.15.131:0/3720726770 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb668009510 con 0x7fb67015e340 2024-05-26T22:49:16.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.071+0000 7fb676600640 1 -- 172.21.15.131:0/3720726770 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6701f3f70 con 0x7fb67015e340 2024-05-26T22:49:16.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.071+0000 7fb676600640 1 -- 172.21.15.131:0/3720726770 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb670151470 con 0x7fb67015e340 2024-05-26T22:49:16.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.073+0000 7fb676600640 1 -- 172.21.15.131:0/3720726770 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb670147cb0 con 0x7fb67015e340 2024-05-26T22:49:16.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.074+0000 7fb66cc00640 1 -- 172.21.15.131:0/3720726770 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb668007bd0 con 0x7fb67015e340 2024-05-26T22:49:16.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.074+0000 7fb66cc00640 1 --2- 172.21.15.131:0/3720726770 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb65006be30 0x7fb65006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:16.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.075+0000 7fb66cc00640 1 -- 172.21.15.131:0/3720726770 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fb6680abe00 con 0x7fb67015e340 2024-05-26T22:49:16.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.075+0000 7fb675200640 1 --2- 172.21.15.131:0/3720726770 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb65006be30 0x7fb65006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:16.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.075+0000 7fb675200640 1 --2- 172.21.15.131:0/3720726770 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb65006be30 0x7fb65006e2f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb66400a770 tx=0x7fb664008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:16.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.079+0000 7fb66cc00640 1 -- 172.21.15.131:0/3720726770 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fb66807a7c0 con 0x7fb67015e340 2024-05-26T22:49:16.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.275+0000 7fb676600640 1 -- 172.21.15.131:0/3720726770 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb6701590f0 con 0x7fb67015e340 2024-05-26T22:49:16.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.276+0000 7fb66cc00640 1 -- 172.21.15.131:0/3720726770 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7fb66807a0c0 con 0x7fb67015e340 2024-05-26T22:49:16.276 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 64 2024-05-26T22:49:16.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.278+0000 7fb676600640 1 -- 172.21.15.131:0/3720726770 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb65006be30 msgr2=0x7fb65006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:16.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.278+0000 7fb676600640 1 --2- 172.21.15.131:0/3720726770 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb65006be30 0x7fb65006e2f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb66400a770 tx=0x7fb664008040 comp rx=0 tx=0).stop 2024-05-26T22:49:16.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.278+0000 7fb676600640 1 -- 172.21.15.131:0/3720726770 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb67015e340 msgr2=0x7fb670151f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:16.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.278+0000 7fb676600640 1 --2- 172.21.15.131:0/3720726770 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb67015e340 0x7fb670151f50 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb66802ad00 tx=0x7fb668004a40 comp rx=0 tx=0).stop 2024-05-26T22:49:16.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.279+0000 7fb676600640 1 -- 172.21.15.131:0/3720726770 shutdown_connections 2024-05-26T22:49:16.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.279+0000 7fb676600640 1 --2- 172.21.15.131:0/3720726770 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb65006be30 0x7fb65006e2f0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:16.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.279+0000 7fb676600640 1 --2- 172.21.15.131:0/3720726770 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb67015e340 0x7fb670151f50 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:16.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.279+0000 7fb676600640 1 --2- 172.21.15.131:0/3720726770 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb6700d27f0 0x7fb6701f3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:16.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.279+0000 7fb676600640 1 --2- 172.21.15.131:0/3720726770 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb6700b7820 0x7fb670144e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:16.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.279+0000 7fb676600640 1 -- 172.21.15.131:0/3720726770 >> 172.21.15.131:0/3720726770 conn(0x7fb670072e60 msgr2=0x7fb670157980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:16.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.279+0000 7fb676600640 1 -- 172.21.15.131:0/3720726770 shutdown_connections 2024-05-26T22:49:16.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.279+0000 7fb676600640 1 -- 172.21.15.131:0/3720726770 wait complete. 2024-05-26T22:49:16.298 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-26T22:49:16.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.501+0000 7fd4c3200640 1 Processor -- start 2024-05-26T22:49:16.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.501+0000 7fd4c3200640 1 -- start start 2024-05-26T22:49:16.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.503+0000 7fd4c3200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd4c415e200 0x7fd4c406d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:16.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.503+0000 7fd4c3200640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4c4070560 con 0x7fd4c41629e0 2024-05-26T22:49:16.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.503+0000 7fd4c3200640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4c406c820 con 0x7fd4c40ce8b0 2024-05-26T22:49:16.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.503+0000 7fd4c3200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd4c406cb00 con 0x7fd4c415e200 2024-05-26T22:49:16.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.503+0000 7fd4c1400640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fd4c41629e0 0x7fd4c406fd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:49450/0 (socket says 172.21.15.131:49450) 2024-05-26T22:49:16.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.503+0000 7fd4c1400640 1 -- 172.21.15.131:0/265508357 learned_addr learned my addr 172.21.15.131:0/265508357 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:49:16.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.504+0000 7fd4c1e00640 1 --2- 172.21.15.131:0/265508357 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd4c415e200 0x7fd4c406d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:16.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.504+0000 7fd4c0a00640 1 -- 172.21.15.131:0/265508357 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2258168882 0 0) 0x7fd4c4070560 con 0x7fd4c41629e0 2024-05-26T22:49:16.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.504+0000 7fd4c0a00640 1 -- 172.21.15.131:0/265508357 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd4a8003520 con 0x7fd4c41629e0 2024-05-26T22:49:16.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.504+0000 7fd4c0a00640 1 -- 172.21.15.131:0/265508357 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1109600395 0 0) 0x7fd4a8003520 con 0x7fd4c41629e0 2024-05-26T22:49:16.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.504+0000 7fd4c0a00640 1 -- 172.21.15.131:0/265508357 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd4c415e200 msgr2=0x7fd4c406d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:16.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.504+0000 7fd4c0a00640 1 --2- 172.21.15.131:0/265508357 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd4c415e200 0x7fd4c406d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:16.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.504+0000 7fd4c0a00640 1 -- 172.21.15.131:0/265508357 >> v1:172.21.15.187:6789/0 conn(0x7fd4c40ce8b0 legacy=0x7fd4c40cfff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:16.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.504+0000 7fd4c0a00640 1 -- 172.21.15.131:0/265508357 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4c406cde0 con 0x7fd4c41629e0 2024-05-26T22:49:16.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.504+0000 7fd4c0a00640 1 -- 172.21.15.131:0/265508357 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7fd4b40019f0 con 0x7fd4c41629e0 2024-05-26T22:49:16.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.504+0000 7fd4c0a00640 1 -- 172.21.15.131:0/265508357 >> v1:172.21.15.131:6789/0 conn(0x7fd4c41629e0 legacy=0x7fd4c406fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:16.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.504+0000 7fd4c0a00640 1 --2- 172.21.15.131:0/265508357 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd4a80039e0 0x7fd4a8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:16.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.504+0000 7fd4c0a00640 1 --2- 172.21.15.131:0/265508357 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd4a8004800 0x7fd4a8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:16.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.504+0000 7fd4c0a00640 1 --2- 172.21.15.131:0/265508357 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd4c415e200 0x7fd4a80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:16.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.504+0000 7fd4c0a00640 1 -- 172.21.15.131:0/265508357 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fd4c406cb00 con 0x7fd4a8004800 2024-05-26T22:49:16.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.504+0000 7fd4c0a00640 1 -- 172.21.15.131:0/265508357 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd4a800baa0 con 0x7fd4a80039e0 2024-05-26T22:49:16.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.504+0000 7fd4c0a00640 1 -- 172.21.15.131:0/265508357 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd4a800bca0 con 0x7fd4c415e200 2024-05-26T22:49:16.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.505+0000 7fd4c1400640 1 --2- 172.21.15.131:0/265508357 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd4a8004800 0x7fd4a8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:16.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.505+0000 7fd4c1e00640 1 --2- 172.21.15.131:0/265508357 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd4a80039e0 0x7fd4a8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:16.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.505+0000 7fd4c1400640 1 -- 172.21.15.131:0/265508357 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd4c415e200 msgr2=0x7fd4a80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:16.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.505+0000 7fd4c1400640 1 --2- 172.21.15.131:0/265508357 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd4c415e200 0x7fd4a80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:16.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.505+0000 7fd4c1400640 1 -- 172.21.15.131:0/265508357 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd4a80039e0 msgr2=0x7fd4a8003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:16.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.505+0000 7fd4c1400640 1 --2- 172.21.15.131:0/265508357 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd4a80039e0 0x7fd4a8003e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:16.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.505+0000 7fd4c1400640 1 -- 172.21.15.131:0/265508357 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4a8003520 con 0x7fd4a8004800 2024-05-26T22:49:16.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.506+0000 7fd4c1400640 1 --2- 172.21.15.131:0/265508357 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd4a8004800 0x7fd4a8004c80 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fd4b400a780 tx=0x7fd4a8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99582c0472056fa3 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:16.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.506+0000 7fd4c0a00640 1 -- 172.21.15.131:0/265508357 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4b400a3d0 con 0x7fd4a8004800 2024-05-26T22:49:16.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.506+0000 7fd4c0a00640 1 -- 172.21.15.131:0/265508357 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4b402f070 con 0x7fd4a8004800 2024-05-26T22:49:16.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.506+0000 7fd4c0a00640 1 -- 172.21.15.131:0/265508357 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4b402bc00 con 0x7fd4a8004800 2024-05-26T22:49:16.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.507+0000 7fd4c3200640 1 -- 172.21.15.131:0/265508357 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd4a8004800 msgr2=0x7fd4a8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:16.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.507+0000 7fd4c3200640 1 --2- 172.21.15.131:0/265508357 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd4a8004800 0x7fd4a8004c80 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fd4b400a780 tx=0x7fd4a8005830 comp rx=0 tx=0).stop 2024-05-26T22:49:16.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.507+0000 7fd4c2800640 1 -- 172.21.15.131:0/265508357 reap_dead start 2024-05-26T22:49:16.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.507+0000 7fd4c3200640 1 -- 172.21.15.131:0/265508357 shutdown_connections 2024-05-26T22:49:16.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.507+0000 7fd4c3200640 1 -- 172.21.15.131:0/265508357 >> 172.21.15.131:0/265508357 conn(0x7fd4c40d1ed0 msgr2=0x7fd4c40d22e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:16.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.507+0000 7fd4c3200640 1 -- 172.21.15.131:0/265508357 shutdown_connections 2024-05-26T22:49:16.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.507+0000 7fd4c3200640 1 -- 172.21.15.131:0/265508357 wait complete. 2024-05-26T22:49:16.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.509+0000 7fd4c3200640 1 Processor -- start 2024-05-26T22:49:16.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.509+0000 7fd4c3200640 1 -- start start 2024-05-26T22:49:16.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.510+0000 7fd4c3200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd4a8004800 0x7fd4c41f7640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:16.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.510+0000 7fd4c3200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd4c40b2a40 0x7fd4c40b2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:16.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.510+0000 7fd4c3200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd4c40cc730 0x7fd4c40ccbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:16.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.510+0000 7fd4c3200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fd4c415e760 con 0x7fd4c40b2a40 2024-05-26T22:49:16.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.510+0000 7fd4c3200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd4c415e5e0 con 0x7fd4c40cc730 2024-05-26T22:49:16.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.510+0000 7fd4c3200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd4c406d5e0 con 0x7fd4a8004800 2024-05-26T22:49:16.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.510+0000 7fd4c1400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd4c40b2a40 0x7fd4c40b2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:16.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.510+0000 7fd4c1400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd4c40b2a40 0x7fd4c40b2ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40712/0 (socket says 172.21.15.131:40712) 2024-05-26T22:49:16.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.510+0000 7fd4c1400640 1 -- 172.21.15.131:0/793802170 learned_addr learned my addr 172.21.15.131:0/793802170 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:16.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.510+0000 7fd4c2800640 1 --2- 172.21.15.131:0/793802170 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd4c40cc730 0x7fd4c40ccbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:16.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.510+0000 7fd4c1e00640 1 --2- 172.21.15.131:0/793802170 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd4a8004800 0x7fd4c41f7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:16.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.511+0000 7fd4c1400640 1 -- 172.21.15.131:0/793802170 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd4a8004800 msgr2=0x7fd4c41f7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:16.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.511+0000 7fd4c1400640 1 --2- 172.21.15.131:0/793802170 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd4a8004800 0x7fd4c41f7640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:16.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.511+0000 7fd4c1400640 1 -- 172.21.15.131:0/793802170 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd4c40cc730 msgr2=0x7fd4c40ccbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:16.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.511+0000 7fd4c1400640 1 --2- 172.21.15.131:0/793802170 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd4c40cc730 0x7fd4c40ccbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:16.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.511+0000 7fd4c1400640 1 -- 172.21.15.131:0/793802170 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4c40c9820 con 0x7fd4c40b2a40 2024-05-26T22:49:16.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.511+0000 7fd4c2800640 1 --2- 172.21.15.131:0/793802170 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd4c40cc730 0x7fd4c40ccbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:49:16.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.511+0000 7fd4c1400640 1 --2- 172.21.15.131:0/793802170 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd4c40b2a40 0x7fd4c40b2ec0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fd4b402b430 tx=0x7fd4b4001cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:16.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.511+0000 7fd4ba000640 1 -- 172.21.15.131:0/793802170 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4b402b980 con 0x7fd4c40b2a40 2024-05-26T22:49:16.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.511+0000 7fd4ba000640 1 -- 172.21.15.131:0/793802170 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4b402bb20 con 0x7fd4c40b2a40 2024-05-26T22:49:16.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.511+0000 7fd4ba000640 1 -- 172.21.15.131:0/793802170 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4b4004e30 con 0x7fd4c40b2a40 2024-05-26T22:49:16.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.511+0000 7fd4c3200640 1 -- 172.21.15.131:0/793802170 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4c40cd460 con 0x7fd4c40b2a40 2024-05-26T22:49:16.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.511+0000 7fd4c3200640 1 -- 172.21.15.131:0/793802170 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4c40cda20 con 0x7fd4c40b2a40 2024-05-26T22:49:16.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.512+0000 7fd4c3200640 1 -- 172.21.15.131:0/793802170 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4c40694e0 con 0x7fd4c40b2a40 2024-05-26T22:49:16.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.513+0000 7fd4ba000640 1 -- 172.21.15.131:0/793802170 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd4b403e070 con 0x7fd4c40b2a40 2024-05-26T22:49:16.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.514+0000 7fd4ba000640 1 --2- 172.21.15.131:0/793802170 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fd4a006be60 0x7fd4a006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:16.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.514+0000 7fd4ba000640 1 -- 172.21.15.131:0/793802170 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fd4b40ab770 con 0x7fd4c40b2a40 2024-05-26T22:49:16.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.515+0000 7fd4c1e00640 1 --2- 172.21.15.131:0/793802170 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fd4a006be60 0x7fd4a006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:16.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.515+0000 7fd4c1e00640 1 --2- 172.21.15.131:0/793802170 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fd4a006be60 0x7fd4a006e320 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd4c40ce510 tx=0x7fd4ac008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:16.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.519+0000 7fd4ba000640 1 -- 172.21.15.131:0/793802170 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fd4b407a090 con 0x7fd4c40b2a40 2024-05-26T22:49:16.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.747+0000 7fd4c3200640 1 -- 172.21.15.131:0/793802170 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd4c406c560 con 0x7fd4c40b2a40 2024-05-26T22:49:16.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.748+0000 7fd4ba000640 1 -- 172.21.15.131:0/793802170 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5184 (secure 0 0 0) 0x7fd4b4079990 con 0x7fd4c40b2a40 2024-05-26T22:49:16.748 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:49:16.748 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/3266175650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6833","nonce":3266175650},{"type":"v1","addr":"172.21.15.187:6835","nonce":3266175650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/1862866305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1862866305},{"type":"v1","addr":"172.21.15.131:6835","nonce":1862866305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4269,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/618294324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":618294324},{"type":"v1","addr":"172.21.15.131:6837","nonce":618294324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-26T22:48:21.211047+0000","modified":"2024-05-26T22:48:24.550610+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6834/2547512415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":2547512415},{"type":"v1","addr":"172.21.15.187:6834","nonce":2547512415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4281,"qdb_cluster":[4281]},"id":1}]} 2024-05-26T22:49:16.749 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 7 2024-05-26T22:49:16.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.751+0000 7fd4c3200640 1 -- 172.21.15.131:0/793802170 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fd4a006be60 msgr2=0x7fd4a006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:16.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.751+0000 7fd4c3200640 1 --2- 172.21.15.131:0/793802170 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fd4a006be60 0x7fd4a006e320 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd4c40ce510 tx=0x7fd4ac008040 comp rx=0 tx=0).stop 2024-05-26T22:49:16.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.751+0000 7fd4c3200640 1 -- 172.21.15.131:0/793802170 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd4c40b2a40 msgr2=0x7fd4c40b2ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:16.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.751+0000 7fd4c3200640 1 --2- 172.21.15.131:0/793802170 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd4c40b2a40 0x7fd4c40b2ec0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fd4b402b430 tx=0x7fd4b4001cf0 comp rx=0 tx=0).stop 2024-05-26T22:49:16.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.751+0000 7fd4c3200640 1 -- 172.21.15.131:0/793802170 shutdown_connections 2024-05-26T22:49:16.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.751+0000 7fd4c3200640 1 --2- 172.21.15.131:0/793802170 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fd4a006be60 0x7fd4a006e320 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:16.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.751+0000 7fd4c3200640 1 --2- 172.21.15.131:0/793802170 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd4c40cc730 0x7fd4c40ccbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:16.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.751+0000 7fd4c3200640 1 --2- 172.21.15.131:0/793802170 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd4c40b2a40 0x7fd4c40b2ec0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:16.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.751+0000 7fd4c3200640 1 --2- 172.21.15.131:0/793802170 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd4a8004800 0x7fd4c41f7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:16.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.751+0000 7fd4c3200640 1 -- 172.21.15.131:0/793802170 >> 172.21.15.131:0/793802170 conn(0x7fd4c40d1ed0 msgr2=0x7fd4c40d1470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:16.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.752+0000 7fd4c3200640 1 -- 172.21.15.131:0/793802170 shutdown_connections 2024-05-26T22:49:16.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.752+0000 7fd4c3200640 1 -- 172.21.15.131:0/793802170 wait complete. 2024-05-26T22:49:16.768 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-26T22:49:16.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.957+0000 7f57e0c00640 1 Processor -- start 2024-05-26T22:49:16.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.957+0000 7f57e0c00640 1 -- start start 2024-05-26T22:49:16.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.959+0000 7f57e0c00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f57dc06ee20 0x7f57dc0b3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:16.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.959+0000 7f57e0c00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f57dc0ad250 0x7f57dc0af790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:16.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.959+0000 7f57e0c00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57dc15cb20 con 0x7f57dc06f200 2024-05-26T22:49:16.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.959+0000 7f57e0c00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f57dc15ce00 con 0x7f57dc06ee20 2024-05-26T22:49:16.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.959+0000 7f57e0c00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f57dc15d0e0 con 0x7f57dc0ad250 2024-05-26T22:49:16.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.959+0000 7f57d9e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f57dc06ee20 0x7f57dc0b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:16.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.959+0000 7f57d9e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f57dc06ee20 0x7f57dc0b3190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40720/0 (socket says 172.21.15.131:40720) 2024-05-26T22:49:16.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.959+0000 7f57da800640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f57dc06f200 0x7f57dc1562c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:49460/0 (socket says 172.21.15.131:49460) 2024-05-26T22:49:16.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.959+0000 7f57d9e00640 1 -- 172.21.15.131:0/4022766998 learned_addr learned my addr 172.21.15.131:0/4022766998 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:16.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.959+0000 7f57d9400640 1 --2- 172.21.15.131:0/4022766998 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f57dc0ad250 0x7f57dc0af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:16.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.959+0000 7f57d9e00640 1 -- 172.21.15.131:0/4022766998 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f57dc0ad250 msgr2=0x7f57dc0af790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:16.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.959+0000 7f57d9e00640 1 --2- 172.21.15.131:0/4022766998 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f57dc0ad250 0x7f57dc0af790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:16.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.959+0000 7f57d9e00640 1 -- 172.21.15.131:0/4022766998 >> v1:172.21.15.131:6789/0 conn(0x7f57dc06f200 legacy=0x7f57dc1562c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:16.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.960+0000 7f57d9e00640 1 -- 172.21.15.131:0/4022766998 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57dc15d3c0 con 0x7f57dc06ee20 2024-05-26T22:49:16.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.960+0000 7f57d9e00640 1 --2- 172.21.15.131:0/4022766998 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f57dc06ee20 0x7f57dc0b3190 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f57cc009870 tx=0x7f57cc02a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aa5085906fc4e7c server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:16.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.961+0000 7f57d8a00640 1 -- 172.21.15.131:0/4022766998 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57cc037070 con 0x7f57dc06ee20 2024-05-26T22:49:16.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.961+0000 7f57d8a00640 1 -- 172.21.15.131:0/4022766998 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57cc032440 con 0x7f57dc06ee20 2024-05-26T22:49:16.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.961+0000 7f57d8a00640 1 -- 172.21.15.131:0/4022766998 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57cc032810 con 0x7f57dc06ee20 2024-05-26T22:49:16.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.961+0000 7f57e0c00640 1 -- 172.21.15.131:0/4022766998 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f57dc06ee20 msgr2=0x7f57dc0b3190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:16.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.961+0000 7f57e0c00640 1 --2- 172.21.15.131:0/4022766998 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f57dc06ee20 0x7f57dc0b3190 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f57cc009870 tx=0x7f57cc02a250 comp rx=0 tx=0).stop 2024-05-26T22:49:16.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.961+0000 7f57e0c00640 1 -- 172.21.15.131:0/4022766998 shutdown_connections 2024-05-26T22:49:16.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.961+0000 7f57e0c00640 1 --2- 172.21.15.131:0/4022766998 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f57dc0ad250 0x7f57dc0af790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:16.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.961+0000 7f57e0c00640 1 --2- 172.21.15.131:0/4022766998 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f57dc06ee20 0x7f57dc0b3190 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:16.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.961+0000 7f57e0c00640 1 -- 172.21.15.131:0/4022766998 >> 172.21.15.131:0/4022766998 conn(0x7f57dc0b1e90 msgr2=0x7f57dc0b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:16.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.962+0000 7f57e0c00640 1 -- 172.21.15.131:0/4022766998 shutdown_connections 2024-05-26T22:49:16.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.962+0000 7f57e0c00640 1 -- 172.21.15.131:0/4022766998 wait complete. 2024-05-26T22:49:16.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.963+0000 7f57e0c00640 1 Processor -- start 2024-05-26T22:49:16.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.963+0000 7f57e0c00640 1 -- start start 2024-05-26T22:49:16.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.964+0000 7f57e0c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f57dc06ee20 0x7f57dc1f8700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:16.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.964+0000 7f57e0c00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f57dc06f200 0x7f57dc14e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:16.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.964+0000 7f57e0c00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f57dc0ad250 0x7f57dc1f3f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:16.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.964+0000 7f57e0c00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f57dc15c900 con 0x7f57dc06ee20 2024-05-26T22:49:16.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.964+0000 7f57e0c00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f57dc15c780 con 0x7f57dc06f200 2024-05-26T22:49:16.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.964+0000 7f57e0c00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f57dc1569d0 con 0x7f57dc0ad250 2024-05-26T22:49:16.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.964+0000 7f57d9e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f57dc06ee20 0x7f57dc1f8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:16.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.964+0000 7f57d9e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f57dc06ee20 0x7f57dc1f8700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40726/0 (socket says 172.21.15.131:40726) 2024-05-26T22:49:16.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.965+0000 7f57d9e00640 1 -- 172.21.15.131:0/1228516816 learned_addr learned my addr 172.21.15.131:0/1228516816 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:16.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.965+0000 7f57d9e00640 1 -- 172.21.15.131:0/1228516816 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f57dc0ad250 msgr2=0x7f57dc1f3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:16.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.965+0000 7f57d9e00640 1 --2- 172.21.15.131:0/1228516816 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f57dc0ad250 0x7f57dc1f3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:16.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.965+0000 7f57d9e00640 1 -- 172.21.15.131:0/1228516816 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f57dc06f200 msgr2=0x7f57dc14e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:16.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.965+0000 7f57d9e00640 1 --2- 172.21.15.131:0/1228516816 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f57dc06f200 0x7f57dc14e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:16.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.965+0000 7f57d9e00640 1 -- 172.21.15.131:0/1228516816 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57dc14d830 con 0x7f57dc06ee20 2024-05-26T22:49:16.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.965+0000 7f57d9e00640 1 --2- 172.21.15.131:0/1228516816 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f57dc06ee20 0x7f57dc1f8700 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f57cc000c00 tx=0x7f57cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:16.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.965+0000 7f57d2000640 1 -- 172.21.15.131:0/1228516816 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57cc041020 con 0x7f57dc06ee20 2024-05-26T22:49:16.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.965+0000 7f57d2000640 1 -- 172.21.15.131:0/1228516816 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57cc0306c0 con 0x7f57dc06ee20 2024-05-26T22:49:16.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.966+0000 7f57d2000640 1 -- 172.21.15.131:0/1228516816 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57cc037040 con 0x7f57dc06ee20 2024-05-26T22:49:16.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.966+0000 7f57e0c00640 1 -- 172.21.15.131:0/1228516816 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57dc14db10 con 0x7f57dc06ee20 2024-05-26T22:49:16.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.966+0000 7f57e0c00640 1 -- 172.21.15.131:0/1228516816 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57dc14e0d0 con 0x7f57dc06ee20 2024-05-26T22:49:16.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.967+0000 7f57e0c00640 1 -- 172.21.15.131:0/1228516816 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57dc0694e0 con 0x7f57dc06ee20 2024-05-26T22:49:16.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.968+0000 7f57d2000640 1 -- 172.21.15.131:0/1228516816 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f57cc044020 con 0x7f57dc06ee20 2024-05-26T22:49:16.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.968+0000 7f57d2000640 1 --2- 172.21.15.131:0/1228516816 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f57bc06be30 0x7f57bc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:16.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.969+0000 7f57d2000640 1 -- 172.21.15.131:0/1228516816 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f57cc0ab7d0 con 0x7f57dc06ee20 2024-05-26T22:49:16.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.969+0000 7f57d9400640 1 --2- 172.21.15.131:0/1228516816 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f57bc06be30 0x7f57bc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:16.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.969+0000 7f57d9400640 1 --2- 172.21.15.131:0/1228516816 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f57bc06be30 0x7f57bc06e2f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f57c8006e80 tx=0x7f57c8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:16.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:16.973+0000 7f57d2000640 1 -- 172.21.15.131:0/1228516816 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f57cc04b030 con 0x7f57dc06ee20 2024-05-26T22:49:17.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.172+0000 7f57e0c00640 1 -- 172.21.15.131:0/1228516816 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f57dc15f100 con 0x7f57dc06ee20 2024-05-26T22:49:17.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.174+0000 7f57d2000640 1 -- 172.21.15.131:0/1228516816 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16879 (secure 0 0 0) 0x7f57cc07a060 con 0x7f57dc06ee20 2024-05-26T22:49:17.175 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:49:17.175 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":22,"fsid":"46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f","created":"2024-05-26T22:47:57.288905+0000","modified":"2024-05-26T22:49:04.278988+0000","last_up_change":"2024-05-26T22:48:09.602207+0000","last_in_change":"2024-05-26T22:48:00.687341+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-26T22:48:10.463339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-26T22:48:19.663254+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-26T22:48:20.235751+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"814eaf62-a18d-494b-8e27-fd8fb88adea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6801","nonce":2350940051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6803","nonce":2350940051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6807","nonce":2350940051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6805","nonce":2350940051}]},"public_addr":"172.21.15.131:6801/2350940051","cluster_addr":"172.21.15.131:6803/2350940051","heartbeat_back_addr":"172.21.15.131:6807/2350940051","heartbeat_front_addr":"172.21.15.131:6805/2350940051","state":["exists","up"]},{"osd":1,"uuid":"42e66287-f6c6-4e26-af3c-6dfaa3536d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6825","nonce":1525528247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6827","nonce":1525528247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6831","nonce":1525528247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6829","nonce":1525528247}]},"public_addr":"172.21.15.131:6825/1525528247","cluster_addr":"172.21.15.131:6827/1525528247","heartbeat_back_addr":"172.21.15.131:6831/1525528247","heartbeat_front_addr":"172.21.15.131:6829/1525528247","state":["exists","up"]},{"osd":2,"uuid":"0da34673-04a2-4fd0-b57a-0ff2b679b043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6817","nonce":3627842210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6819","nonce":3627842210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6823","nonce":3627842210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6821","nonce":3627842210}]},"public_addr":"172.21.15.131:6817/3627842210","cluster_addr":"172.21.15.131:6819/3627842210","heartbeat_back_addr":"172.21.15.131:6823/3627842210","heartbeat_front_addr":"172.21.15.131:6821/3627842210","state":["exists","up"]},{"osd":3,"uuid":"cf117a27-d6b2-43d3-ae52-b47988bf8eed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6809","nonce":350790591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6811","nonce":350790591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6815","nonce":350790591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6813","nonce":350790591}]},"public_addr":"172.21.15.131:6809/350790591","cluster_addr":"172.21.15.131:6811/350790591","heartbeat_back_addr":"172.21.15.131:6815/350790591","heartbeat_front_addr":"172.21.15.131:6813/350790591","state":["exists","up"]},{"osd":4,"uuid":"a092ccfc-be89-4a84-88d5-0f781fa715eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6816","nonce":244560995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6819","nonce":244560995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6823","nonce":244560995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6821","nonce":244560995}]},"public_addr":"172.21.15.187:6816/244560995","cluster_addr":"172.21.15.187:6819/244560995","heartbeat_back_addr":"172.21.15.187:6823/244560995","heartbeat_front_addr":"172.21.15.187:6821/244560995","state":["exists","up"]},{"osd":5,"uuid":"7513ba69-dd00-4a35-b190-9e8cc3b2bdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6801","nonce":324619398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6803","nonce":324619398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6812","nonce":324619398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6807","nonce":324619398}]},"public_addr":"172.21.15.187:6801/324619398","cluster_addr":"172.21.15.187:6803/324619398","heartbeat_back_addr":"172.21.15.187:6812/324619398","heartbeat_front_addr":"172.21.15.187:6807/324619398","state":["exists","up"]},{"osd":6,"uuid":"50081810-c093-4d2f-a4d7-b7c695c5400d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6806","nonce":1391347786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6809","nonce":1391347786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6818","nonce":1391347786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6813","nonce":1391347786}]},"public_addr":"172.21.15.187:6806/1391347786","cluster_addr":"172.21.15.187:6809/1391347786","heartbeat_back_addr":"172.21.15.187:6818/1391347786","heartbeat_front_addr":"172.21.15.187:6813/1391347786","state":["exists","up"]},{"osd":7,"uuid":"775dae05-54f5-4ad4-bbce-92863c6ed2ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6825","nonce":1220065622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6827","nonce":1220065622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6831","nonce":1220065622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6829","nonce":1220065622}]},"public_addr":"172.21.15.187:6825/1220065622","cluster_addr":"172.21.15.187:6827/1220065622","heartbeat_back_addr":"172.21.15.187:6831/1220065622","heartbeat_front_addr":"172.21.15.187:6829/1220065622","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:05.983520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.299516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.067323+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.307748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.018742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.258775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.010912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.295355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.1e","mappings":[{"from":3,"to":2}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":7,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-26T22:49:17.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.178+0000 7f57e0c00640 1 -- 172.21.15.131:0/1228516816 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f57bc06be30 msgr2=0x7f57bc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:17.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.178+0000 7f57e0c00640 1 --2- 172.21.15.131:0/1228516816 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f57bc06be30 0x7f57bc06e2f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f57c8006e80 tx=0x7f57c8008040 comp rx=0 tx=0).stop 2024-05-26T22:49:17.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.178+0000 7f57e0c00640 1 -- 172.21.15.131:0/1228516816 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f57dc06ee20 msgr2=0x7f57dc1f8700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:17.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.178+0000 7f57e0c00640 1 --2- 172.21.15.131:0/1228516816 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f57dc06ee20 0x7f57dc1f8700 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f57cc000c00 tx=0x7f57cc0046c0 comp rx=0 tx=0).stop 2024-05-26T22:49:17.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.178+0000 7f57e0c00640 1 -- 172.21.15.131:0/1228516816 shutdown_connections 2024-05-26T22:49:17.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.178+0000 7f57e0c00640 1 --2- 172.21.15.131:0/1228516816 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f57dc0ad250 0x7f57dc1f3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:17.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.178+0000 7f57e0c00640 1 --2- 172.21.15.131:0/1228516816 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f57bc06be30 0x7f57bc06e2f0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:17.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.178+0000 7f57e0c00640 1 --2- 172.21.15.131:0/1228516816 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f57dc06f200 0x7f57dc14e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:17.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.178+0000 7f57e0c00640 1 --2- 172.21.15.131:0/1228516816 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f57dc06ee20 0x7f57dc1f8700 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:17.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.178+0000 7f57e0c00640 1 -- 172.21.15.131:0/1228516816 >> 172.21.15.131:0/1228516816 conn(0x7f57dc0b1e90 msgr2=0x7f57dc15e650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:17.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.178+0000 7f57e0c00640 1 -- 172.21.15.131:0/1228516816 shutdown_connections 2024-05-26T22:49:17.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.178+0000 7f57e0c00640 1 -- 172.21.15.131:0/1228516816 wait complete. 2024-05-26T22:49:17.197 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-26T22:49:17.198 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-26T22:49:17.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.395+0000 7f1243200640 1 Processor -- start 2024-05-26T22:49:17.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.395+0000 7f1243200640 1 -- start start 2024-05-26T22:49:17.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.396+0000 7f1243200640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f12440b5b90 0x7f12440d23d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:17.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.396+0000 7f1243200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f1244161de0 0x7f12440b5f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:17.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.396+0000 7f1243200640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12440b64b0 con 0x7f12440b9990 2024-05-26T22:49:17.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.396+0000 7f1243200640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f124415e970 con 0x7f1244161de0 2024-05-26T22:49:17.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.396+0000 7f1243200640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f12440d2910 con 0x7f12440b5b90 2024-05-26T22:49:17.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.397+0000 7f1241400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f1244161de0 0x7f12440b5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:17.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.397+0000 7f1241400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f1244161de0 0x7f12440b5f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40732/0 (socket says 172.21.15.131:40732) 2024-05-26T22:49:17.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.397+0000 7f1241400640 1 -- 172.21.15.131:0/4232422391 learned_addr learned my addr 172.21.15.131:0/4232422391 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:17.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.397+0000 7f1241e00640 1 --2- 172.21.15.131:0/4232422391 >> v2:172.21.15.187:3300/0 conn(0x7f12440b5b90 0x7f12440d23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:17.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.397+0000 7f1241400640 1 -- 172.21.15.131:0/4232422391 >> v2:172.21.15.187:3300/0 conn(0x7f12440b5b90 msgr2=0x7f12440d23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:17.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.397+0000 7f1241400640 1 --2- 172.21.15.131:0/4232422391 >> v2:172.21.15.187:3300/0 conn(0x7f12440b5b90 0x7f12440d23d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:17.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.397+0000 7f1241400640 1 -- 172.21.15.131:0/4232422391 >> v1:172.21.15.131:6789/0 conn(0x7f12440b9990 legacy=0x7f124415ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:17.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.397+0000 7f1241400640 1 -- 172.21.15.131:0/4232422391 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1244162330 con 0x7f1244161de0 2024-05-26T22:49:17.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.398+0000 7f1241400640 1 --2- 172.21.15.131:0/4232422391 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1244161de0 0x7f12440b5f70 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f124415eaf0 tx=0x7f123002a1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f1789b188454b98 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:17.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.399+0000 7f1240a00640 1 -- 172.21.15.131:0/4232422391 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1230037070 con 0x7f1244161de0 2024-05-26T22:49:17.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.399+0000 7f1240a00640 1 -- 172.21.15.131:0/4232422391 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1230032440 con 0x7f1244161de0 2024-05-26T22:49:17.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.399+0000 7f1240a00640 1 -- 172.21.15.131:0/4232422391 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1230032810 con 0x7f1244161de0 2024-05-26T22:49:17.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.399+0000 7f1243200640 1 -- 172.21.15.131:0/4232422391 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1244161de0 msgr2=0x7f12440b5f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:17.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.399+0000 7f1243200640 1 --2- 172.21.15.131:0/4232422391 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1244161de0 0x7f12440b5f70 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f124415eaf0 tx=0x7f123002a1d0 comp rx=0 tx=0).stop 2024-05-26T22:49:17.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.399+0000 7f1243200640 1 -- 172.21.15.131:0/4232422391 shutdown_connections 2024-05-26T22:49:17.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.400+0000 7f1243200640 1 --2- 172.21.15.131:0/4232422391 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1244161de0 0x7f12440b5f70 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:17.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.400+0000 7f1243200640 1 --2- 172.21.15.131:0/4232422391 >> v2:172.21.15.187:3300/0 conn(0x7f12440b5b90 0x7f12440d23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:17.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.400+0000 7f1243200640 1 -- 172.21.15.131:0/4232422391 >> 172.21.15.131:0/4232422391 conn(0x7f1244073070 msgr2=0x7f12440d0b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:17.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.400+0000 7f1243200640 1 -- 172.21.15.131:0/4232422391 shutdown_connections 2024-05-26T22:49:17.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.400+0000 7f1243200640 1 -- 172.21.15.131:0/4232422391 wait complete. 2024-05-26T22:49:17.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.401+0000 7f1243200640 1 Processor -- start 2024-05-26T22:49:17.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.401+0000 7f1243200640 1 -- start start 2024-05-26T22:49:17.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.402+0000 7f1243200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f12440b5b90 0x7f1244159790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:17.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.402+0000 7f1243200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f12440b9990 0x7f12440cc230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:17.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.402+0000 7f1243200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1244161de0 0x7f12440cda40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:17.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.402+0000 7f1243200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f1244162bd0 con 0x7f12440b5b90 2024-05-26T22:49:17.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.402+0000 7f1243200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f12441628f0 con 0x7f12440b9990 2024-05-26T22:49:17.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.402+0000 7f1243200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f12440b9e00 con 0x7f1244161de0 2024-05-26T22:49:17.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.403+0000 7f1241e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f12440b5b90 0x7f1244159790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:17.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.403+0000 7f1241e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f12440b5b90 0x7f1244159790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40746/0 (socket says 172.21.15.131:40746) 2024-05-26T22:49:17.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.403+0000 7f1241e00640 1 -- 172.21.15.131:0/3333624788 learned_addr learned my addr 172.21.15.131:0/3333624788 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:17.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.403+0000 7f1241400640 1 --2- 172.21.15.131:0/3333624788 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f12440b9990 0x7f12440cc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:17.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.403+0000 7f1241e00640 1 -- 172.21.15.131:0/3333624788 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1244161de0 msgr2=0x7f12440cda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:17.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.403+0000 7f1242800640 1 --2- 172.21.15.131:0/3333624788 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1244161de0 0x7f12440cda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:17.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.403+0000 7f1241e00640 1 --2- 172.21.15.131:0/3333624788 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1244161de0 0x7f12440cda40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:17.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.403+0000 7f1241e00640 1 -- 172.21.15.131:0/3333624788 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f12440b9990 msgr2=0x7f12440cc230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:17.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.403+0000 7f1241e00640 1 --2- 172.21.15.131:0/3333624788 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f12440b9990 0x7f12440cc230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:17.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.403+0000 7f1241e00640 1 -- 172.21.15.131:0/3333624788 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12440cc9a0 con 0x7f12440b5b90 2024-05-26T22:49:17.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.403+0000 7f1241400640 1 --2- 172.21.15.131:0/3333624788 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f12440b9990 0x7f12440cc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:49:17.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.403+0000 7f1242800640 1 --2- 172.21.15.131:0/3333624788 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1244161de0 0x7f12440cda40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:49:17.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.403+0000 7f1241e00640 1 --2- 172.21.15.131:0/3333624788 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f12440b5b90 0x7f1244159790 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f123400c7f0 tx=0x7f123400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:17.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.404+0000 7f123a000640 1 -- 172.21.15.131:0/3333624788 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1234015070 con 0x7f12440b5b90 2024-05-26T22:49:17.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.404+0000 7f123a000640 1 -- 172.21.15.131:0/3333624788 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1234004240 con 0x7f12440b5b90 2024-05-26T22:49:17.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.404+0000 7f123a000640 1 -- 172.21.15.131:0/3333624788 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1234007620 con 0x7f12440b5b90 2024-05-26T22:49:17.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.404+0000 7f1243200640 1 -- 172.21.15.131:0/3333624788 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12440ccc80 con 0x7f12440b5b90 2024-05-26T22:49:17.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.404+0000 7f1243200640 1 -- 172.21.15.131:0/3333624788 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12440cd240 con 0x7f12440b5b90 2024-05-26T22:49:17.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.405+0000 7f1243200640 1 -- 172.21.15.131:0/3333624788 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12440694e0 con 0x7f12440b5b90 2024-05-26T22:49:17.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.406+0000 7f123a000640 1 -- 172.21.15.131:0/3333624788 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f12340077c0 con 0x7f12440b5b90 2024-05-26T22:49:17.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.407+0000 7f123a000640 1 --2- 172.21.15.131:0/3333624788 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f122406be30 0x7f122406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:17.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.407+0000 7f123a000640 1 -- 172.21.15.131:0/3333624788 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f123408b100 con 0x7f12440b5b90 2024-05-26T22:49:17.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.407+0000 7f1241400640 1 --2- 172.21.15.131:0/3333624788 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f122406be30 0x7f122406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:17.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.408+0000 7f1241400640 1 --2- 172.21.15.131:0/3333624788 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f122406be30 0x7f122406e2f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1230034000 tx=0x7f123002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:17.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.415+0000 7f123a000640 1 -- 172.21.15.131:0/3333624788 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f1234059a30 con 0x7f12440b5b90 2024-05-26T22:49:17.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.639+0000 7f1243200640 1 -- 172.21.15.131:0/3333624788 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f124415db90 con 0x7f12440b5b90 2024-05-26T22:49:17.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.640+0000 7f123a000640 1 -- 172.21.15.131:0/3333624788 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5184 (secure 0 0 0) 0x7f1234059330 con 0x7f12440b5b90 2024-05-26T22:49:17.640 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:49:17.640 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/3266175650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6833","nonce":3266175650},{"type":"v1","addr":"172.21.15.187:6835","nonce":3266175650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/1862866305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1862866305},{"type":"v1","addr":"172.21.15.131:6835","nonce":1862866305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4269,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/618294324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":618294324},{"type":"v1","addr":"172.21.15.131:6837","nonce":618294324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-26T22:48:21.211047+0000","modified":"2024-05-26T22:48:24.550610+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6834/2547512415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":2547512415},{"type":"v1","addr":"172.21.15.187:6834","nonce":2547512415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4281,"qdb_cluster":[4281]},"id":1}]} 2024-05-26T22:49:17.640 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 7 2024-05-26T22:49:17.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.642+0000 7f1243200640 1 -- 172.21.15.131:0/3333624788 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f122406be30 msgr2=0x7f122406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:17.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.642+0000 7f1243200640 1 --2- 172.21.15.131:0/3333624788 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f122406be30 0x7f122406e2f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1230034000 tx=0x7f123002e000 comp rx=0 tx=0).stop 2024-05-26T22:49:17.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.642+0000 7f1243200640 1 -- 172.21.15.131:0/3333624788 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f12440b5b90 msgr2=0x7f1244159790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:17.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.642+0000 7f1243200640 1 --2- 172.21.15.131:0/3333624788 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f12440b5b90 0x7f1244159790 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f123400c7f0 tx=0x7f123400ccb0 comp rx=0 tx=0).stop 2024-05-26T22:49:17.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.643+0000 7f1243200640 1 -- 172.21.15.131:0/3333624788 shutdown_connections 2024-05-26T22:49:17.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.643+0000 7f1243200640 1 --2- 172.21.15.131:0/3333624788 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1244161de0 0x7f12440cda40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:17.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.643+0000 7f1243200640 1 --2- 172.21.15.131:0/3333624788 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f122406be30 0x7f122406e2f0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:17.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.643+0000 7f1243200640 1 --2- 172.21.15.131:0/3333624788 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f12440b9990 0x7f12440cc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:17.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.643+0000 7f1243200640 1 --2- 172.21.15.131:0/3333624788 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f12440b5b90 0x7f1244159790 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:17.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.643+0000 7f1243200640 1 -- 172.21.15.131:0/3333624788 >> 172.21.15.131:0/3333624788 conn(0x7f1244073070 msgr2=0x7f124415ed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:17.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.643+0000 7f1243200640 1 -- 172.21.15.131:0/3333624788 shutdown_connections 2024-05-26T22:49:17.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.643+0000 7f1243200640 1 -- 172.21.15.131:0/3333624788 wait complete. 2024-05-26T22:49:17.662 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-26T22:49:17.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.861+0000 7f00d1400640 1 Processor -- start 2024-05-26T22:49:17.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.862+0000 7f00d1400640 1 -- start start 2024-05-26T22:49:17.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.863+0000 7f00d1400640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f00cc0ad450 0x7f00cc06d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:17.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.863+0000 7f00d1400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f00cc0ad830 0x7f00cc071d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:17.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.863+0000 7f00d1400640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00cc0b5190 con 0x7f00cc0722c0 2024-05-26T22:49:17.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.863+0000 7f00d1400640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f00cc06dee0 con 0x7f00cc0ad450 2024-05-26T22:49:17.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.863+0000 7f00d1400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f00cc06e1c0 con 0x7f00cc0ad830 2024-05-26T22:49:17.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.863+0000 7f00cb200640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f00cc0722c0 0x7f00cc06eca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:49490/0 (socket says 172.21.15.131:49490) 2024-05-26T22:49:17.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.863+0000 7f00cb200640 1 -- 172.21.15.131:0/2066567249 learned_addr learned my addr 172.21.15.131:0/2066567249 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:49:17.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.864+0000 7f00c1e00640 1 --2- 172.21.15.131:0/2066567249 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f00cc0ad830 0x7f00cc071d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:17.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.864+0000 7f00ca800640 1 --2- 172.21.15.131:0/2066567249 >> v2:172.21.15.187:3300/0 conn(0x7f00cc0ad450 0x7f00cc06d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:17.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.864+0000 7f00c9e00640 1 -- 172.21.15.131:0/2066567249 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 86097389 0 0) 0x7f00cc0b5190 con 0x7f00cc0722c0 2024-05-26T22:49:17.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.864+0000 7f00c9e00640 1 -- 172.21.15.131:0/2066567249 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f00b0003520 con 0x7f00cc0722c0 2024-05-26T22:49:17.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.864+0000 7f00c9e00640 1 -- 172.21.15.131:0/2066567249 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2067813359 0 0) 0x7f00b0003520 con 0x7f00cc0722c0 2024-05-26T22:49:17.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.864+0000 7f00c9e00640 1 -- 172.21.15.131:0/2066567249 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f00cc0ad830 msgr2=0x7f00cc071d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:17.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.864+0000 7f00c9e00640 1 --2- 172.21.15.131:0/2066567249 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f00cc0ad830 0x7f00cc071d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:17.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.864+0000 7f00c9e00640 1 -- 172.21.15.131:0/2066567249 >> v2:172.21.15.187:3300/0 conn(0x7f00cc0ad450 msgr2=0x7f00cc06d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:17.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.864+0000 7f00c9e00640 1 --2- 172.21.15.131:0/2066567249 >> v2:172.21.15.187:3300/0 conn(0x7f00cc0ad450 0x7f00cc06d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:17.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.864+0000 7f00c9e00640 1 -- 172.21.15.131:0/2066567249 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00cc06e4a0 con 0x7f00cc0722c0 2024-05-26T22:49:17.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.864+0000 7f00c9e00640 1 -- 172.21.15.131:0/2066567249 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7f00c4001670 con 0x7f00cc0722c0 2024-05-26T22:49:17.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.864+0000 7f00c9e00640 1 -- 172.21.15.131:0/2066567249 >> v1:172.21.15.131:6789/0 conn(0x7f00cc0722c0 legacy=0x7f00cc06eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:17.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.864+0000 7f00c9e00640 1 --2- 172.21.15.131:0/2066567249 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f00cc0ad830 0x7f00b00039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:17.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.865+0000 7f00c9e00640 1 --2- 172.21.15.131:0/2066567249 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f00b0003f80 0x7f00b0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:17.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.865+0000 7f00c9e00640 1 --2- 172.21.15.131:0/2066567249 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f00b0004d40 0x7f00b00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:17.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.865+0000 7f00c9e00640 1 -- 172.21.15.131:0/2066567249 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f00cc06dee0 con 0x7f00b0003f80 2024-05-26T22:49:17.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.865+0000 7f00c9e00640 1 -- 172.21.15.131:0/2066567249 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f00cc06e1c0 con 0x7f00b0004d40 2024-05-26T22:49:17.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.865+0000 7f00c9e00640 1 -- 172.21.15.131:0/2066567249 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f00b000dc80 con 0x7f00cc0ad830 2024-05-26T22:49:17.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.865+0000 7f00c1e00640 1 --2- 172.21.15.131:0/2066567249 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f00b0003f80 0x7f00b0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:17.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.865+0000 7f00cb200640 1 --2- 172.21.15.131:0/2066567249 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f00b0004d40 0x7f00b00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:17.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.865+0000 7f00ca800640 1 --2- 172.21.15.131:0/2066567249 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f00cc0ad830 0x7f00b00039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:17.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.865+0000 7f00c1e00640 1 -- 172.21.15.131:0/2066567249 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f00cc0ad830 msgr2=0x7f00b00039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:17.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.865+0000 7f00c1e00640 1 --2- 172.21.15.131:0/2066567249 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f00cc0ad830 0x7f00b00039b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:17.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.865+0000 7f00c1e00640 1 -- 172.21.15.131:0/2066567249 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f00b0004d40 msgr2=0x7f00b00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:17.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.865+0000 7f00c1e00640 1 --2- 172.21.15.131:0/2066567249 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f00b0004d40 0x7f00b00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:17.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.865+0000 7f00c1e00640 1 -- 172.21.15.131:0/2066567249 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00b0003520 con 0x7f00b0003f80 2024-05-26T22:49:17.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.865+0000 7f00cb200640 1 --2- 172.21.15.131:0/2066567249 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f00b0004d40 0x7f00b00051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:49:17.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.866+0000 7f00c1e00640 1 --2- 172.21.15.131:0/2066567249 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f00b0003f80 0x7f00b0004800 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f00b4009ac0 tx=0x7f00b0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7df30a6846964525 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:17.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.866+0000 7f00c9e00640 1 -- 172.21.15.131:0/2066567249 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b4036070 con 0x7f00b0003f80 2024-05-26T22:49:17.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.867+0000 7f00c9e00640 1 -- 172.21.15.131:0/2066567249 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00b4031400 con 0x7f00b0003f80 2024-05-26T22:49:17.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.867+0000 7f00c9e00640 1 -- 172.21.15.131:0/2066567249 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b4009690 con 0x7f00b0003f80 2024-05-26T22:49:17.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.867+0000 7f00d1400640 1 -- 172.21.15.131:0/2066567249 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f00b0003f80 msgr2=0x7f00b0004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:17.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.867+0000 7f00d1400640 1 --2- 172.21.15.131:0/2066567249 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f00b0003f80 0x7f00b0004800 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f00b4009ac0 tx=0x7f00b0005830 comp rx=0 tx=0).stop 2024-05-26T22:49:17.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.867+0000 7f00cb200640 1 -- 172.21.15.131:0/2066567249 reap_dead start 2024-05-26T22:49:17.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.867+0000 7f00d1400640 1 -- 172.21.15.131:0/2066567249 shutdown_connections 2024-05-26T22:49:17.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.867+0000 7f00d1400640 1 -- 172.21.15.131:0/2066567249 >> 172.21.15.131:0/2066567249 conn(0x7f00cc0b1e90 msgr2=0x7f00cc0b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:17.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.867+0000 7f00d1400640 1 -- 172.21.15.131:0/2066567249 shutdown_connections 2024-05-26T22:49:17.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.868+0000 7f00d1400640 1 -- 172.21.15.131:0/2066567249 wait complete. 2024-05-26T22:49:17.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.869+0000 7f00d1400640 1 Processor -- start 2024-05-26T22:49:17.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.869+0000 7f00d1400640 1 -- start start 2024-05-26T22:49:17.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.870+0000 7f00d1400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f00b0003f80 0x7f00cc1f3e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:17.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.870+0000 7f00d1400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f00cc0c9b30 0x7f00cc0c9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:17.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.870+0000 7f00d1400640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f00cc159b60 0x7f00cc159fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:17.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.870+0000 7f00d1400640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f00cc15e1a0 con 0x7f00cc0c9b30 2024-05-26T22:49:17.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.870+0000 7f00d1400640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f00cc15e020 con 0x7f00cc159b60 2024-05-26T22:49:17.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.870+0000 7f00d1400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f00cc06daf0 con 0x7f00b0003f80 2024-05-26T22:49:17.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.871+0000 7f00c1e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f00cc0c9b30 0x7f00cc0c9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:17.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.871+0000 7f00c1e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f00cc0c9b30 0x7f00cc0c9fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40766/0 (socket says 172.21.15.131:40766) 2024-05-26T22:49:17.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.871+0000 7f00c1e00640 1 -- 172.21.15.131:0/3140255117 learned_addr learned my addr 172.21.15.131:0/3140255117 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:17.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.871+0000 7f00ca800640 1 --2- 172.21.15.131:0/3140255117 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f00b0003f80 0x7f00cc1f3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:17.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.871+0000 7f00cb200640 1 --2- 172.21.15.131:0/3140255117 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f00cc159b60 0x7f00cc159fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:17.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.871+0000 7f00c1e00640 1 -- 172.21.15.131:0/3140255117 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f00b0003f80 msgr2=0x7f00cc1f3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:17.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.871+0000 7f00c1e00640 1 --2- 172.21.15.131:0/3140255117 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f00b0003f80 0x7f00cc1f3e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:17.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.871+0000 7f00c1e00640 1 -- 172.21.15.131:0/3140255117 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f00cc159b60 msgr2=0x7f00cc159fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:17.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.871+0000 7f00c1e00640 1 --2- 172.21.15.131:0/3140255117 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f00cc159b60 0x7f00cc159fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:17.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.871+0000 7f00c1e00640 1 -- 172.21.15.131:0/3140255117 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00cc15a740 con 0x7f00cc0c9b30 2024-05-26T22:49:17.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.871+0000 7f00ca800640 1 --2- 172.21.15.131:0/3140255117 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f00b0003f80 0x7f00cc1f3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:49:17.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.871+0000 7f00c1e00640 1 --2- 172.21.15.131:0/3140255117 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f00cc0c9b30 0x7f00cc0c9fb0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f00b4000c00 tx=0x7f00b4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:17.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.872+0000 7f00c3400640 1 -- 172.21.15.131:0/3140255117 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b4040020 con 0x7f00cc0c9b30 2024-05-26T22:49:17.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.872+0000 7f00c3400640 1 -- 172.21.15.131:0/3140255117 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00b4003c60 con 0x7f00cc0c9b30 2024-05-26T22:49:17.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.872+0000 7f00d1400640 1 -- 172.21.15.131:0/3140255117 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00cc15aa20 con 0x7f00cc0c9b30 2024-05-26T22:49:17.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.872+0000 7f00c3400640 1 -- 172.21.15.131:0/3140255117 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b4036040 con 0x7f00cc0c9b30 2024-05-26T22:49:17.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.872+0000 7f00d1400640 1 -- 172.21.15.131:0/3140255117 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00cc09ca40 con 0x7f00cc0c9b30 2024-05-26T22:49:17.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.873+0000 7f00d1400640 1 -- 172.21.15.131:0/3140255117 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00cc0b4bf0 con 0x7f00cc0c9b30 2024-05-26T22:49:17.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.874+0000 7f00c3400640 1 -- 172.21.15.131:0/3140255117 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f00b4043020 con 0x7f00cc0c9b30 2024-05-26T22:49:17.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.875+0000 7f00c3400640 1 --2- 172.21.15.131:0/3140255117 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f00a806be60 0x7f00a806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:17.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.875+0000 7f00c3400640 1 -- 172.21.15.131:0/3140255117 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f00b40aafb0 con 0x7f00cc0c9b30 2024-05-26T22:49:17.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.876+0000 7f00ca800640 1 --2- 172.21.15.131:0/3140255117 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f00a806be60 0x7f00a806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:17.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.876+0000 7f00ca800640 1 --2- 172.21.15.131:0/3140255117 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f00a806be60 0x7f00a806e320 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f00bc007460 tx=0x7f00bc00b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:17.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:17.879+0000 7f00c3400640 1 -- 172.21.15.131:0/3140255117 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f00cc0b4bf0 con 0x7f00cc0c9b30 2024-05-26T22:49:18.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.100+0000 7f00d1400640 1 -- 172.21.15.131:0/3140255117 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f00cc0694e0 con 0x7f00cc0c9b30 2024-05-26T22:49:18.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.101+0000 7f00c3400640 1 -- 172.21.15.131:0/3140255117 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5184 (secure 0 0 0) 0x7f00b404a070 con 0x7f00cc0c9b30 2024-05-26T22:49:18.101 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:49:18.101 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/3266175650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6833","nonce":3266175650},{"type":"v1","addr":"172.21.15.187:6835","nonce":3266175650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/1862866305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1862866305},{"type":"v1","addr":"172.21.15.131:6835","nonce":1862866305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4269,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/618294324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":618294324},{"type":"v1","addr":"172.21.15.131:6837","nonce":618294324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-26T22:48:21.211047+0000","modified":"2024-05-26T22:48:24.550610+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6834/2547512415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":2547512415},{"type":"v1","addr":"172.21.15.187:6834","nonce":2547512415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4281,"qdb_cluster":[4281]},"id":1}]} 2024-05-26T22:49:18.102 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 7 2024-05-26T22:49:18.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.104+0000 7f00d1400640 1 -- 172.21.15.131:0/3140255117 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f00a806be60 msgr2=0x7f00a806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:18.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.104+0000 7f00d1400640 1 --2- 172.21.15.131:0/3140255117 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f00a806be60 0x7f00a806e320 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f00bc007460 tx=0x7f00bc00b040 comp rx=0 tx=0).stop 2024-05-26T22:49:18.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.104+0000 7f00d1400640 1 -- 172.21.15.131:0/3140255117 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f00cc0c9b30 msgr2=0x7f00cc0c9fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:18.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.104+0000 7f00d1400640 1 --2- 172.21.15.131:0/3140255117 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f00cc0c9b30 0x7f00cc0c9fb0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f00b4000c00 tx=0x7f00b4004730 comp rx=0 tx=0).stop 2024-05-26T22:49:18.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.104+0000 7f00d1400640 1 -- 172.21.15.131:0/3140255117 shutdown_connections 2024-05-26T22:49:18.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.104+0000 7f00d1400640 1 --2- 172.21.15.131:0/3140255117 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f00a806be60 0x7f00a806e320 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:18.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.104+0000 7f00d1400640 1 --2- 172.21.15.131:0/3140255117 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f00cc159b60 0x7f00cc159fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:18.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.104+0000 7f00d1400640 1 --2- 172.21.15.131:0/3140255117 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f00cc0c9b30 0x7f00cc0c9fb0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:18.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.104+0000 7f00d1400640 1 --2- 172.21.15.131:0/3140255117 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f00b0003f80 0x7f00cc1f3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:18.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.104+0000 7f00d1400640 1 -- 172.21.15.131:0/3140255117 >> 172.21.15.131:0/3140255117 conn(0x7f00cc0b1e90 msgr2=0x7f00cc0cb3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:18.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.104+0000 7f00d1400640 1 -- 172.21.15.131:0/3140255117 shutdown_connections 2024-05-26T22:49:18.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.104+0000 7f00d1400640 1 -- 172.21.15.131:0/3140255117 wait complete. 2024-05-26T22:49:18.121 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-26T22:49:18.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.304+0000 7f7ce5600640 1 Processor -- start 2024-05-26T22:49:18.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.304+0000 7f7ce5600640 1 -- start start 2024-05-26T22:49:18.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.306+0000 7f7ce5600640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f7ce006b310 0x7f7ce006f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:18.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.306+0000 7f7ce5600640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f7ce006b6f0 0x7f7ce00ae410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:18.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.306+0000 7f7ce5600640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ce00aed60 con 0x7f7ce00ae950 2024-05-26T22:49:18.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.306+0000 7f7ce5600640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f7ce006e570 con 0x7f7ce006b310 2024-05-26T22:49:18.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.306+0000 7f7ce5600640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f7ce006e850 con 0x7f7ce006b6f0 2024-05-26T22:49:18.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.306+0000 7f7ce4c00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f7ce00ae950 0x7f7ce00b3e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:49494/0 (socket says 172.21.15.131:49494) 2024-05-26T22:49:18.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.306+0000 7f7ce4c00640 1 -- 172.21.15.131:0/2903011196 learned_addr learned my addr 172.21.15.131:0/2903011196 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:49:18.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.306+0000 7f7cdfe00640 1 --2- 172.21.15.131:0/2903011196 >> v2:172.21.15.131:3300/0 conn(0x7f7ce006b310 0x7f7ce006f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:18.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.307+0000 7f7cdf400640 1 --2- 172.21.15.131:0/2903011196 >> v2:172.21.15.187:3300/0 conn(0x7f7ce006b6f0 0x7f7ce00ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:18.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.307+0000 7f7cdea00640 1 -- 172.21.15.131:0/2903011196 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2326697926 0 0) 0x7f7ce00aed60 con 0x7f7ce00ae950 2024-05-26T22:49:18.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.307+0000 7f7cdea00640 1 -- 172.21.15.131:0/2903011196 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7cc4003520 con 0x7f7ce00ae950 2024-05-26T22:49:18.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.307+0000 7f7cdfe00640 1 -- 172.21.15.131:0/2903011196 >> v2:172.21.15.187:3300/0 conn(0x7f7ce006b6f0 msgr2=0x7f7ce00ae410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:18.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.307+0000 7f7cdfe00640 1 --2- 172.21.15.131:0/2903011196 >> v2:172.21.15.187:3300/0 conn(0x7f7ce006b6f0 0x7f7ce00ae410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:18.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.307+0000 7f7cdfe00640 1 -- 172.21.15.131:0/2903011196 >> v1:172.21.15.131:6789/0 conn(0x7f7ce00ae950 legacy=0x7f7ce00b3e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:18.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.307+0000 7f7cdfe00640 1 -- 172.21.15.131:0/2903011196 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ce006eb30 con 0x7f7ce006b310 2024-05-26T22:49:18.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.307+0000 7f7cdf400640 1 --2- 172.21.15.131:0/2903011196 >> v2:172.21.15.187:3300/0 conn(0x7f7ce006b6f0 0x7f7ce00ae410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-26T22:49:18.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.308+0000 7f7cdfe00640 1 --2- 172.21.15.131:0/2903011196 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ce006b310 0x7f7ce006f330 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f7ce0070d50 tx=0x7f7cd002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4a6be0bab762c07 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:18.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.308+0000 7f7cdea00640 1 -- 172.21.15.131:0/2903011196 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cd0037070 con 0x7f7ce006b310 2024-05-26T22:49:18.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.308+0000 7f7cdea00640 1 -- 172.21.15.131:0/2903011196 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cd0032440 con 0x7f7ce006b310 2024-05-26T22:49:18.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.308+0000 7f7cdea00640 1 -- 172.21.15.131:0/2903011196 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cd002ae60 con 0x7f7ce006b310 2024-05-26T22:49:18.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.309+0000 7f7ce5600640 1 -- 172.21.15.131:0/2903011196 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ce006b310 msgr2=0x7f7ce006f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:18.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.309+0000 7f7ce5600640 1 --2- 172.21.15.131:0/2903011196 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ce006b310 0x7f7ce006f330 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f7ce0070d50 tx=0x7f7cd002a110 comp rx=0 tx=0).stop 2024-05-26T22:49:18.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.309+0000 7f7ce5600640 1 -- 172.21.15.131:0/2903011196 shutdown_connections 2024-05-26T22:49:18.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.309+0000 7f7ce5600640 1 --2- 172.21.15.131:0/2903011196 >> v2:172.21.15.187:3300/0 conn(0x7f7ce006b6f0 0x7f7ce00ae410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:18.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.309+0000 7f7ce5600640 1 --2- 172.21.15.131:0/2903011196 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ce006b310 0x7f7ce006f330 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:18.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.309+0000 7f7ce5600640 1 -- 172.21.15.131:0/2903011196 >> 172.21.15.131:0/2903011196 conn(0x7f7ce00b2b90 msgr2=0x7f7ce00b2fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:18.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.309+0000 7f7ce5600640 1 -- 172.21.15.131:0/2903011196 shutdown_connections 2024-05-26T22:49:18.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.309+0000 7f7ce5600640 1 -- 172.21.15.131:0/2903011196 wait complete. 2024-05-26T22:49:18.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.311+0000 7f7ce5600640 1 Processor -- start 2024-05-26T22:49:18.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.311+0000 7f7ce5600640 1 -- start start 2024-05-26T22:49:18.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.312+0000 7f7ce5600640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ce006b310 0x7f7ce013d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:18.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.312+0000 7f7ce5600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ce006b6f0 0x7f7ce013de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:18.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.312+0000 7f7ce5600640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7ce00ae950 0x7f7ce01f4130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:18.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.312+0000 7f7ce5600640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f7ce00af1a0 con 0x7f7ce006b6f0 2024-05-26T22:49:18.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.312+0000 7f7ce5600640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7ce00af020 con 0x7f7ce006b310 2024-05-26T22:49:18.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.312+0000 7f7ce5600640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f7ce0147100 con 0x7f7ce00ae950 2024-05-26T22:49:18.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.312+0000 7f7cdf400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ce006b6f0 0x7f7ce013de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:18.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.312+0000 7f7cdf400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ce006b6f0 0x7f7ce013de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40784/0 (socket says 172.21.15.131:40784) 2024-05-26T22:49:18.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.312+0000 7f7cdf400640 1 -- 172.21.15.131:0/2291252237 learned_addr learned my addr 172.21.15.131:0/2291252237 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:18.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.312+0000 7f7cdfe00640 1 --2- 172.21.15.131:0/2291252237 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ce006b310 0x7f7ce013d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:18.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.312+0000 7f7ce4c00640 1 --2- 172.21.15.131:0/2291252237 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7ce00ae950 0x7f7ce01f4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:18.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.312+0000 7f7cdf400640 1 -- 172.21.15.131:0/2291252237 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7ce00ae950 msgr2=0x7f7ce01f4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:18.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.312+0000 7f7cdf400640 1 --2- 172.21.15.131:0/2291252237 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7ce00ae950 0x7f7ce01f4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:18.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.312+0000 7f7cdf400640 1 -- 172.21.15.131:0/2291252237 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ce006b310 msgr2=0x7f7ce013d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:18.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.312+0000 7f7cdf400640 1 --2- 172.21.15.131:0/2291252237 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ce006b310 0x7f7ce013d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:18.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.312+0000 7f7cdf400640 1 -- 172.21.15.131:0/2291252237 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ce01408e0 con 0x7f7ce006b6f0 2024-05-26T22:49:18.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.312+0000 7f7ce4c00640 1 --2- 172.21.15.131:0/2291252237 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7ce00ae950 0x7f7ce01f4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:49:18.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.313+0000 7f7cdf400640 1 --2- 172.21.15.131:0/2291252237 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ce006b6f0 0x7f7ce013de10 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc00e7e0 tx=0x7f7ccc00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:18.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.313+0000 7f7cd7e00640 1 -- 172.21.15.131:0/2291252237 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ccc00f040 con 0x7f7ce006b6f0 2024-05-26T22:49:18.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.313+0000 7f7cd7e00640 1 -- 172.21.15.131:0/2291252237 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ccc004030 con 0x7f7ce006b6f0 2024-05-26T22:49:18.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.313+0000 7f7cd7e00640 1 -- 172.21.15.131:0/2291252237 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ccc004400 con 0x7f7ce006b6f0 2024-05-26T22:49:18.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.313+0000 7f7ce5600640 1 -- 172.21.15.131:0/2291252237 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ce015f970 con 0x7f7ce006b6f0 2024-05-26T22:49:18.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.313+0000 7f7ce5600640 1 -- 172.21.15.131:0/2291252237 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ce0140bc0 con 0x7f7ce006b6f0 2024-05-26T22:49:18.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.315+0000 7f7ce5600640 1 -- 172.21.15.131:0/2291252237 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ce00694e0 con 0x7f7ce006b6f0 2024-05-26T22:49:18.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.315+0000 7f7cd7e00640 1 -- 172.21.15.131:0/2291252237 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7ccc00c6d0 con 0x7f7ce006b6f0 2024-05-26T22:49:18.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.317+0000 7f7cd7e00640 1 --2- 172.21.15.131:0/2291252237 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f7cc006be60 0x7f7cc006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:18.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.317+0000 7f7cd7e00640 1 -- 172.21.15.131:0/2291252237 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f7ccc08ac90 con 0x7f7ce006b6f0 2024-05-26T22:49:18.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.317+0000 7f7cdfe00640 1 --2- 172.21.15.131:0/2291252237 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f7cc006be60 0x7f7cc006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:18.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.318+0000 7f7cdfe00640 1 --2- 172.21.15.131:0/2291252237 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f7cc006be60 0x7f7cc006e320 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f7cd0034000 tx=0x7f7cd002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:18.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.321+0000 7f7cd7e00640 1 -- 172.21.15.131:0/2291252237 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f7ccc017020 con 0x7f7ce006b6f0 2024-05-26T22:49:18.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.521+0000 7f7ce5600640 1 -- 172.21.15.131:0/2291252237 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7ce00b1a30 con 0x7f7ce006b6f0 2024-05-26T22:49:18.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.523+0000 7f7cd7e00640 1 -- 172.21.15.131:0/2291252237 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16879 (secure 0 0 0) 0x7f7ccc0595b0 con 0x7f7ce006b6f0 2024-05-26T22:49:18.523 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:49:18.524 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":22,"fsid":"46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f","created":"2024-05-26T22:47:57.288905+0000","modified":"2024-05-26T22:49:04.278988+0000","last_up_change":"2024-05-26T22:48:09.602207+0000","last_in_change":"2024-05-26T22:48:00.687341+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-26T22:48:10.463339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-26T22:48:19.663254+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-26T22:48:20.235751+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"814eaf62-a18d-494b-8e27-fd8fb88adea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6801","nonce":2350940051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6803","nonce":2350940051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6807","nonce":2350940051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6805","nonce":2350940051}]},"public_addr":"172.21.15.131:6801/2350940051","cluster_addr":"172.21.15.131:6803/2350940051","heartbeat_back_addr":"172.21.15.131:6807/2350940051","heartbeat_front_addr":"172.21.15.131:6805/2350940051","state":["exists","up"]},{"osd":1,"uuid":"42e66287-f6c6-4e26-af3c-6dfaa3536d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6825","nonce":1525528247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6827","nonce":1525528247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6831","nonce":1525528247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6829","nonce":1525528247}]},"public_addr":"172.21.15.131:6825/1525528247","cluster_addr":"172.21.15.131:6827/1525528247","heartbeat_back_addr":"172.21.15.131:6831/1525528247","heartbeat_front_addr":"172.21.15.131:6829/1525528247","state":["exists","up"]},{"osd":2,"uuid":"0da34673-04a2-4fd0-b57a-0ff2b679b043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6817","nonce":3627842210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6819","nonce":3627842210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6823","nonce":3627842210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6821","nonce":3627842210}]},"public_addr":"172.21.15.131:6817/3627842210","cluster_addr":"172.21.15.131:6819/3627842210","heartbeat_back_addr":"172.21.15.131:6823/3627842210","heartbeat_front_addr":"172.21.15.131:6821/3627842210","state":["exists","up"]},{"osd":3,"uuid":"cf117a27-d6b2-43d3-ae52-b47988bf8eed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6809","nonce":350790591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6811","nonce":350790591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6815","nonce":350790591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6813","nonce":350790591}]},"public_addr":"172.21.15.131:6809/350790591","cluster_addr":"172.21.15.131:6811/350790591","heartbeat_back_addr":"172.21.15.131:6815/350790591","heartbeat_front_addr":"172.21.15.131:6813/350790591","state":["exists","up"]},{"osd":4,"uuid":"a092ccfc-be89-4a84-88d5-0f781fa715eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6816","nonce":244560995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6819","nonce":244560995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6823","nonce":244560995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6821","nonce":244560995}]},"public_addr":"172.21.15.187:6816/244560995","cluster_addr":"172.21.15.187:6819/244560995","heartbeat_back_addr":"172.21.15.187:6823/244560995","heartbeat_front_addr":"172.21.15.187:6821/244560995","state":["exists","up"]},{"osd":5,"uuid":"7513ba69-dd00-4a35-b190-9e8cc3b2bdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6801","nonce":324619398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6803","nonce":324619398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6812","nonce":324619398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6807","nonce":324619398}]},"public_addr":"172.21.15.187:6801/324619398","cluster_addr":"172.21.15.187:6803/324619398","heartbeat_back_addr":"172.21.15.187:6812/324619398","heartbeat_front_addr":"172.21.15.187:6807/324619398","state":["exists","up"]},{"osd":6,"uuid":"50081810-c093-4d2f-a4d7-b7c695c5400d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6806","nonce":1391347786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6809","nonce":1391347786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6818","nonce":1391347786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6813","nonce":1391347786}]},"public_addr":"172.21.15.187:6806/1391347786","cluster_addr":"172.21.15.187:6809/1391347786","heartbeat_back_addr":"172.21.15.187:6818/1391347786","heartbeat_front_addr":"172.21.15.187:6813/1391347786","state":["exists","up"]},{"osd":7,"uuid":"775dae05-54f5-4ad4-bbce-92863c6ed2ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6825","nonce":1220065622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6827","nonce":1220065622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6831","nonce":1220065622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6829","nonce":1220065622}]},"public_addr":"172.21.15.187:6825/1220065622","cluster_addr":"172.21.15.187:6827/1220065622","heartbeat_back_addr":"172.21.15.187:6831/1220065622","heartbeat_front_addr":"172.21.15.187:6829/1220065622","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:05.983520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.299516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.067323+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.307748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.018742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.258775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.010912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.295355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.1e","mappings":[{"from":3,"to":2}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":7,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-26T22:49:18.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.526+0000 7f7ce5600640 1 -- 172.21.15.131:0/2291252237 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f7cc006be60 msgr2=0x7f7cc006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:18.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.526+0000 7f7ce5600640 1 --2- 172.21.15.131:0/2291252237 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f7cc006be60 0x7f7cc006e320 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f7cd0034000 tx=0x7f7cd002e000 comp rx=0 tx=0).stop 2024-05-26T22:49:18.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.526+0000 7f7ce5600640 1 -- 172.21.15.131:0/2291252237 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ce006b6f0 msgr2=0x7f7ce013de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:18.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.526+0000 7f7ce5600640 1 --2- 172.21.15.131:0/2291252237 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ce006b6f0 0x7f7ce013de10 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc00e7e0 tx=0x7f7ccc00eca0 comp rx=0 tx=0).stop 2024-05-26T22:49:18.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.526+0000 7f7ce5600640 1 -- 172.21.15.131:0/2291252237 shutdown_connections 2024-05-26T22:49:18.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.526+0000 7f7ce5600640 1 --2- 172.21.15.131:0/2291252237 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7ce00ae950 0x7f7ce01f4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:18.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.526+0000 7f7ce5600640 1 --2- 172.21.15.131:0/2291252237 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f7cc006be60 0x7f7cc006e320 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:18.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.526+0000 7f7ce5600640 1 --2- 172.21.15.131:0/2291252237 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ce006b6f0 0x7f7ce013de10 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:18.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.526+0000 7f7ce5600640 1 --2- 172.21.15.131:0/2291252237 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ce006b310 0x7f7ce013d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:18.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.526+0000 7f7ce5600640 1 -- 172.21.15.131:0/2291252237 >> 172.21.15.131:0/2291252237 conn(0x7f7ce00b2b90 msgr2=0x7f7ce00b2230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:18.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.526+0000 7f7ce5600640 1 -- 172.21.15.131:0/2291252237 shutdown_connections 2024-05-26T22:49:18.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.526+0000 7f7ce5600640 1 -- 172.21.15.131:0/2291252237 wait complete. 2024-05-26T22:49:18.546 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-26T22:49:18.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.742+0000 7f6871400640 1 Processor -- start 2024-05-26T22:49:18.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.742+0000 7f6871400640 1 -- start start 2024-05-26T22:49:18.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.744+0000 7f6871400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f686c06dd30 0x7f686c06e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:18.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.744+0000 7f6871400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f686c0725a0 0x7f686c0729a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:18.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.744+0000 7f6871400640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f686c15c1c0 con 0x7f686c155770 2024-05-26T22:49:18.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.744+0000 7f6871400640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f686c15c4a0 con 0x7f686c06dd30 2024-05-26T22:49:18.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.744+0000 7f6871400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f686c071da0 con 0x7f686c0725a0 2024-05-26T22:49:18.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.744+0000 7f6863e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f686c06dd30 0x7f686c06e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:18.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.744+0000 7f6863e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f686c06dd30 0x7f686c06e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40800/0 (socket says 172.21.15.131:40800) 2024-05-26T22:49:18.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.744+0000 7f6863e00640 1 -- 172.21.15.131:0/2385645201 learned_addr learned my addr 172.21.15.131:0/2385645201 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:18.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.744+0000 7f686a800640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f686c155770 0x7f686c15b700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:49510/0 (socket says 172.21.15.131:49510) 2024-05-26T22:49:18.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.744+0000 7f686b200640 1 --2- 172.21.15.131:0/2385645201 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f686c0725a0 0x7f686c0729a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:18.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.745+0000 7f6863e00640 1 -- 172.21.15.131:0/2385645201 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f686c0725a0 msgr2=0x7f686c0729a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:18.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.745+0000 7f6863e00640 1 --2- 172.21.15.131:0/2385645201 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f686c0725a0 0x7f686c0729a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:18.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.745+0000 7f6863e00640 1 -- 172.21.15.131:0/2385645201 >> v1:172.21.15.131:6789/0 conn(0x7f686c155770 legacy=0x7f686c15b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:18.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.745+0000 7f6863e00640 1 -- 172.21.15.131:0/2385645201 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f686c072080 con 0x7f686c06dd30 2024-05-26T22:49:18.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.746+0000 7f6863e00640 1 --2- 172.21.15.131:0/2385645201 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f686c06dd30 0x7f686c06e1c0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f6854009870 tx=0x7f685402a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9b66e9ca9596e22 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:18.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.746+0000 7f6869e00640 1 -- 172.21.15.131:0/2385645201 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6854037070 con 0x7f686c06dd30 2024-05-26T22:49:18.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.746+0000 7f6869e00640 1 -- 172.21.15.131:0/2385645201 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f685402ae50 con 0x7f686c06dd30 2024-05-26T22:49:18.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.746+0000 7f6869e00640 1 -- 172.21.15.131:0/2385645201 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6854032860 con 0x7f686c06dd30 2024-05-26T22:49:18.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.746+0000 7f6871400640 1 -- 172.21.15.131:0/2385645201 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f686c06dd30 msgr2=0x7f686c06e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:18.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.746+0000 7f6871400640 1 --2- 172.21.15.131:0/2385645201 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f686c06dd30 0x7f686c06e1c0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f6854009870 tx=0x7f685402a300 comp rx=0 tx=0).stop 2024-05-26T22:49:18.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.747+0000 7f6871400640 1 -- 172.21.15.131:0/2385645201 shutdown_connections 2024-05-26T22:49:18.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.747+0000 7f6871400640 1 --2- 172.21.15.131:0/2385645201 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f686c0725a0 0x7f686c0729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:18.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.747+0000 7f6871400640 1 --2- 172.21.15.131:0/2385645201 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f686c06dd30 0x7f686c06e1c0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:18.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.747+0000 7f6871400640 1 -- 172.21.15.131:0/2385645201 >> 172.21.15.131:0/2385645201 conn(0x7f686c06bc90 msgr2=0x7f686c0b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:18.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.747+0000 7f6871400640 1 -- 172.21.15.131:0/2385645201 shutdown_connections 2024-05-26T22:49:18.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.747+0000 7f6871400640 1 -- 172.21.15.131:0/2385645201 wait complete. 2024-05-26T22:49:18.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.748+0000 7f6871400640 1 Processor -- start 2024-05-26T22:49:18.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.749+0000 7f6871400640 1 -- start start 2024-05-26T22:49:18.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.749+0000 7f6871400640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f686c06dd30 0x7f686c13c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:18.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.749+0000 7f6871400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f686c0725a0 0x7f686c1e9050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:18.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.749+0000 7f6871400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f686c155770 0x7f686c1e9590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:18.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.749+0000 7f6871400640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f686c15be10 con 0x7f686c0725a0 2024-05-26T22:49:18.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.749+0000 7f6871400640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f686c15c8c0 con 0x7f686c06dd30 2024-05-26T22:49:18.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.750+0000 7f6871400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f686c14fc60 con 0x7f686c155770 2024-05-26T22:49:18.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.750+0000 7f6863e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f686c0725a0 0x7f686c1e9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:18.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.750+0000 7f6863e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f686c0725a0 0x7f686c1e9050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40816/0 (socket says 172.21.15.131:40816) 2024-05-26T22:49:18.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.750+0000 7f6863e00640 1 -- 172.21.15.131:0/3077303829 learned_addr learned my addr 172.21.15.131:0/3077303829 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:18.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.750+0000 7f6863e00640 1 -- 172.21.15.131:0/3077303829 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f686c155770 msgr2=0x7f686c1e9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:18.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.750+0000 7f686b200640 1 --2- 172.21.15.131:0/3077303829 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f686c155770 0x7f686c1e9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:18.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.750+0000 7f6863e00640 1 --2- 172.21.15.131:0/3077303829 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f686c155770 0x7f686c1e9590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:18.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.750+0000 7f686a800640 1 --2- 172.21.15.131:0/3077303829 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f686c06dd30 0x7f686c13c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:18.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.750+0000 7f6863e00640 1 -- 172.21.15.131:0/3077303829 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f686c06dd30 msgr2=0x7f686c13c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:18.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.750+0000 7f6863e00640 1 --2- 172.21.15.131:0/3077303829 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f686c06dd30 0x7f686c13c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:18.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.750+0000 7f6863e00640 1 -- 172.21.15.131:0/3077303829 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f686c0b1260 con 0x7f686c0725a0 2024-05-26T22:49:18.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.750+0000 7f6863e00640 1 --2- 172.21.15.131:0/3077303829 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f686c0725a0 0x7f686c1e9050 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f6854000c00 tx=0x7f68540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:18.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.751+0000 7f6862a00640 1 -- 172.21.15.131:0/3077303829 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6854041070 con 0x7f686c0725a0 2024-05-26T22:49:18.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.751+0000 7f6862a00640 1 -- 172.21.15.131:0/3077303829 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f685403c670 con 0x7f686c0725a0 2024-05-26T22:49:18.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.751+0000 7f6862a00640 1 -- 172.21.15.131:0/3077303829 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6854037040 con 0x7f686c0725a0 2024-05-26T22:49:18.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.751+0000 7f6871400640 1 -- 172.21.15.131:0/3077303829 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f686c0b1540 con 0x7f686c0725a0 2024-05-26T22:49:18.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.751+0000 7f6871400640 1 -- 172.21.15.131:0/3077303829 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f686c13d470 con 0x7f686c0725a0 2024-05-26T22:49:18.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.752+0000 7f6871400640 1 -- 172.21.15.131:0/3077303829 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f686c074e90 con 0x7f686c0725a0 2024-05-26T22:49:18.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.753+0000 7f6862a00640 1 -- 172.21.15.131:0/3077303829 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6854044070 con 0x7f686c0725a0 2024-05-26T22:49:18.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.754+0000 7f6862a00640 1 --2- 172.21.15.131:0/3077303829 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f684806be30 0x7f684806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:18.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.754+0000 7f6862a00640 1 -- 172.21.15.131:0/3077303829 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f68540ab5c0 con 0x7f686c0725a0 2024-05-26T22:49:18.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.754+0000 7f686a800640 1 --2- 172.21.15.131:0/3077303829 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f684806be30 0x7f684806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:18.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.755+0000 7f686a800640 1 --2- 172.21.15.131:0/3077303829 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f684806be30 0x7f684806e2f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f686c0cfb80 tx=0x7f685c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:18.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.758+0000 7f6862a00640 1 -- 172.21.15.131:0/3077303829 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f6854079e50 con 0x7f686c0725a0 2024-05-26T22:49:18.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:18.997+0000 7f6871400640 1 -- 172.21.15.131:0/3077303829 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f686c0695c0 con 0x7f686c0725a0 2024-05-26T22:49:18.997 INFO:tasks.ceph.mon.a.smithi131.stderr:2024-05-26T22:49:18.998+0000 7fcf0d200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-26T22:49:19.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:19.001+0000 7f6862a00640 1 -- 172.21.15.131:0/3077303829 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v8) ==== 161+0+0 (secure 0 0 0) 0x7f6854079750 con 0x7f686c0725a0 2024-05-26T22:49:19.001 INFO:teuthology.orchestra.run.smithi131.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-26T22:49:19.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:19.005+0000 7f6871400640 1 -- 172.21.15.131:0/3077303829 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f684806be30 msgr2=0x7f684806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:19.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:19.005+0000 7f6871400640 1 --2- 172.21.15.131:0/3077303829 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f684806be30 0x7f684806e2f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f686c0cfb80 tx=0x7f685c009040 comp rx=0 tx=0).stop 2024-05-26T22:49:19.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:19.005+0000 7f6871400640 1 -- 172.21.15.131:0/3077303829 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f686c0725a0 msgr2=0x7f686c1e9050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:19.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:19.005+0000 7f6871400640 1 --2- 172.21.15.131:0/3077303829 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f686c0725a0 0x7f686c1e9050 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f6854000c00 tx=0x7f68540046c0 comp rx=0 tx=0).stop 2024-05-26T22:49:19.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:19.005+0000 7f6871400640 1 -- 172.21.15.131:0/3077303829 shutdown_connections 2024-05-26T22:49:19.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:19.005+0000 7f6871400640 1 --2- 172.21.15.131:0/3077303829 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f686c155770 0x7f686c1e9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:19.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:19.005+0000 7f6871400640 1 --2- 172.21.15.131:0/3077303829 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f684806be30 0x7f684806e2f0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:19.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:19.005+0000 7f6871400640 1 --2- 172.21.15.131:0/3077303829 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f686c0725a0 0x7f686c1e9050 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:19.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:19.005+0000 7f6871400640 1 --2- 172.21.15.131:0/3077303829 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f686c06dd30 0x7f686c13c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:19.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:19.005+0000 7f6871400640 1 -- 172.21.15.131:0/3077303829 >> 172.21.15.131:0/3077303829 conn(0x7f686c06bc90 msgr2=0x7f686c0b3a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:19.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:19.005+0000 7f6871400640 1 -- 172.21.15.131:0/3077303829 shutdown_connections 2024-05-26T22:49:19.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:19.005+0000 7f6871400640 1 -- 172.21.15.131:0/3077303829 wait complete. 2024-05-26T22:49:19.025 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-26T22:49:19.040 INFO:tasks.ceph.mds.d.smithi187.stderr: -8503> 2024-05-26T22:48:22.739+0000 7fd87d600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-26T22:49:19.135 INFO:tasks.ceph.mds.d.smithi187.stdout:starting mds.d at 2024-05-26T22:49:19.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:19.224+0000 7f2005a00640 1 Processor -- start 2024-05-26T22:49:19.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:19.224+0000 7f2005a00640 1 -- start start 2024-05-26T22:49:19.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:19.226+0000 7f2005a00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f200015e2b0 0x7f20000b9330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:19.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:19.226+0000 7f2005a00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f200015e690 0x7f20000b9870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:19.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:19.226+0000 7f2005a00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20000d0510 con 0x7f20000b9db0 2024-05-26T22:49:19.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:19.226+0000 7f2005a00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f20000d1120 con 0x7f200015e690 2024-05-26T22:49:19.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:19.226+0000 7f2005a00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f20000d15b0 con 0x7f200015e2b0 2024-05-26T22:49:19.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:19.226+0000 7f1fff400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f200015e690 0x7f20000b9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:19.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:19.226+0000 7f1fff400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f200015e690 0x7f20000b9870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40818/0 (socket says 172.21.15.131:40818) 2024-05-26T22:49:19.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:19.226+0000 7f1fff400640 1 -- 172.21.15.131:0/2403929725 learned_addr learned my addr 172.21.15.131:0/2403929725 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:19.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:19.227+0000 7f1fffe00640 1 --2- 172.21.15.131:0/2403929725 >> v2:172.21.15.187:3300/0 conn(0x7f200015e2b0 0x7f20000b9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:19.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:19.227+0000 7f1fff400640 1 -- 172.21.15.131:0/2403929725 >> v2:172.21.15.187:3300/0 conn(0x7f200015e2b0 msgr2=0x7f20000b9330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:19.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:19.227+0000 7f1fff400640 1 --2- 172.21.15.131:0/2403929725 >> v2:172.21.15.187:3300/0 conn(0x7f200015e2b0 0x7f20000b9330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:19.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:19.227+0000 7f1fff400640 1 -- 172.21.15.131:0/2403929725 >> v1:172.21.15.187:6789/0 conn(0x7f20000b9db0 legacy=0x7f2000162030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:19.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:19.227+0000 7f1fff400640 1 -- 172.21.15.131:0/2403929725 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20000d07f0 con 0x7f200015e690 2024-05-26T22:49:19.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:19.228+0000 7f1fff400640 1 --2- 172.21.15.131:0/2403929725 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f200015e690 0x7f20000b9870 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f200006c060 tx=0x7f1fec02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe7a43481fc963a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:19.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:19.228+0000 7f1ffea00640 1 -- 172.21.15.131:0/2403929725 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fec037070 con 0x7f200015e690 2024-05-26T22:49:19.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:19.229+0000 7f1ffea00640 1 -- 172.21.15.131:0/2403929725 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fec032440 con 0x7f200015e690 2024-05-26T22:49:19.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:19.229+0000 7f1ffea00640 1 -- 172.21.15.131:0/2403929725 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fec032810 con 0x7f200015e690 2024-05-26T22:49:19.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:19.229+0000 7f2005a00640 1 -- 172.21.15.131:0/2403929725 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f200015e690 msgr2=0x7f20000b9870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:19.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:19.229+0000 7f2005a00640 1 --2- 172.21.15.131:0/2403929725 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f200015e690 0x7f20000b9870 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f200006c060 tx=0x7f1fec02a230 comp rx=0 tx=0).stop 2024-05-26T22:49:19.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:19.229+0000 7f2005a00640 1 -- 172.21.15.131:0/2403929725 shutdown_connections 2024-05-26T22:49:19.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:19.229+0000 7f2005a00640 1 --2- 172.21.15.131:0/2403929725 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f200015e690 0x7f20000b9870 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:19.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:19.229+0000 7f2005a00640 1 --2- 172.21.15.131:0/2403929725 >> v2:172.21.15.187:3300/0 conn(0x7f200015e2b0 0x7f20000b9330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:19.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:19.229+0000 7f2005a00640 1 -- 172.21.15.131:0/2403929725 >> 172.21.15.131:0/2403929725 conn(0x7f200009d500 msgr2=0x7f20000d41c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:19.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:19.229+0000 7f2005a00640 1 -- 172.21.15.131:0/2403929725 shutdown_connections 2024-05-26T22:49:19.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:19.229+0000 7f2005a00640 1 -- 172.21.15.131:0/2403929725 wait complete. 2024-05-26T22:49:19.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:19.231+0000 7f2005a00640 1 Processor -- start 2024-05-26T22:49:19.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:19.231+0000 7f2005a00640 1 -- start start 2024-05-26T22:49:19.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:19.232+0000 7f2005a00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f20000b9db0 0x7f20000c9e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:19.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:19.232+0000 7f2005a00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f200015e2b0 0x7f20000cdc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:19.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:19.232+0000 7f2005a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f200015e690 0x7f20001f4200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:19.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:19.232+0000 7f2005a00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f20000b8020 con 0x7f200015e690 2024-05-26T22:49:19.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:19.232+0000 7f2005a00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f20000d0db0 con 0x7f200015e2b0 2024-05-26T22:49:19.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:19.232+0000 7f2005a00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f20000b81a0 con 0x7f20000b9db0 2024-05-26T22:49:19.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:19.232+0000 7f2005000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f200015e690 0x7f20001f4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:19.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:19.232+0000 7f2005000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f200015e690 0x7f20001f4200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40830/0 (socket says 172.21.15.131:40830) 2024-05-26T22:49:19.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:19.232+0000 7f2005000640 1 -- 172.21.15.131:0/2369974480 learned_addr learned my addr 172.21.15.131:0/2369974480 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:19.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:19.233+0000 7f1fffe00640 1 --2- 172.21.15.131:0/2369974480 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f20000b9db0 0x7f20000c9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:19.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:19.233+0000 7f1fff400640 1 --2- 172.21.15.131:0/2369974480 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f200015e2b0 0x7f20000cdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:19.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:19.233+0000 7f2005000640 1 -- 172.21.15.131:0/2369974480 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f20000b9db0 msgr2=0x7f20000c9e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:19.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:19.233+0000 7f2005000640 1 --2- 172.21.15.131:0/2369974480 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f20000b9db0 0x7f20000c9e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:19.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:19.233+0000 7f2005000640 1 -- 172.21.15.131:0/2369974480 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f200015e2b0 msgr2=0x7f20000cdc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:19.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:19.233+0000 7f2005000640 1 --2- 172.21.15.131:0/2369974480 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f200015e2b0 0x7f20000cdc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:19.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:19.233+0000 7f2005000640 1 -- 172.21.15.131:0/2369974480 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20000ca380 con 0x7f200015e690 2024-05-26T22:49:19.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:19.233+0000 7f1fffe00640 1 --2- 172.21.15.131:0/2369974480 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f20000b9db0 0x7f20000c9e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:49:19.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:19.233+0000 7f2005000640 1 --2- 172.21.15.131:0/2369974480 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f200015e690 0x7f20001f4200 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f1ff800c780 tx=0x7f1ff800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:19.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:19.233+0000 7f1ff7e00640 1 -- 172.21.15.131:0/2369974480 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff8017070 con 0x7f200015e690 2024-05-26T22:49:19.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:19.233+0000 7f1ff7e00640 1 -- 172.21.15.131:0/2369974480 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ff8003da0 con 0x7f200015e690 2024-05-26T22:49:19.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:19.233+0000 7f1ff7e00640 1 -- 172.21.15.131:0/2369974480 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff80052b0 con 0x7f200015e690 2024-05-26T22:49:19.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:19.233+0000 7f2005a00640 1 -- 172.21.15.131:0/2369974480 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2000160220 con 0x7f200015e690 2024-05-26T22:49:19.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:19.233+0000 7f2005a00640 1 -- 172.21.15.131:0/2369974480 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20001607e0 con 0x7f200015e690 2024-05-26T22:49:19.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:19.234+0000 7f2005a00640 1 -- 172.21.15.131:0/2369974480 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20000694e0 con 0x7f200015e690 2024-05-26T22:49:19.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:19.236+0000 7f1ff7e00640 1 -- 172.21.15.131:0/2369974480 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1ff8005450 con 0x7f200015e690 2024-05-26T22:49:19.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:19.237+0000 7f1ff7e00640 1 --2- 172.21.15.131:0/2369974480 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f1fe006be30 0x7f1fe006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:19.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:19.237+0000 7f1ff7e00640 1 -- 172.21.15.131:0/2369974480 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5911+0+0 (secure 0 0 0) 0x7f1ff808a4f0 con 0x7f200015e690 2024-05-26T22:49:19.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:19.237+0000 7f1fffe00640 1 --2- 172.21.15.131:0/2369974480 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f1fe006be30 0x7f1fe006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:19.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:19.238+0000 7f1fffe00640 1 --2- 172.21.15.131:0/2369974480 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f1fe006be30 0x7f1fe006e2f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1ff0006fd0 tx=0x7f1ff0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:19.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:19.241+0000 7f1ff7e00640 1 -- 172.21.15.131:0/2369974480 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f1ff8058d30 con 0x7f200015e690 2024-05-26T22:49:19.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:19.485+0000 7f2005a00640 1 -- 172.21.15.131:0/2369974480 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f20000d1770 con 0x7f200015e690 2024-05-26T22:49:20.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:20.005+0000 7f1ff7e00640 1 -- 172.21.15.131:0/2369974480 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v9) ==== 104+0+0 (secure 0 0 0) 0x7f1ff8058630 con 0x7f200015e690 2024-05-26T22:49:20.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:20.008+0000 7f2005a00640 1 -- 172.21.15.131:0/2369974480 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f1fe006be30 msgr2=0x7f1fe006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:20.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:20.008+0000 7f2005a00640 1 --2- 172.21.15.131:0/2369974480 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f1fe006be30 0x7f1fe006e2f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1ff0006fd0 tx=0x7f1ff0008040 comp rx=0 tx=0).stop 2024-05-26T22:49:20.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:20.009+0000 7f2005a00640 1 -- 172.21.15.131:0/2369974480 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f200015e690 msgr2=0x7f20001f4200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:20.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:20.009+0000 7f2005a00640 1 --2- 172.21.15.131:0/2369974480 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f200015e690 0x7f20001f4200 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f1ff800c780 tx=0x7f1ff800cc40 comp rx=0 tx=0).stop 2024-05-26T22:49:20.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:20.009+0000 7f2005a00640 1 -- 172.21.15.131:0/2369974480 shutdown_connections 2024-05-26T22:49:20.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:20.009+0000 7f2005a00640 1 --2- 172.21.15.131:0/2369974480 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f1fe006be30 0x7f1fe006e2f0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:20.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:20.009+0000 7f2005a00640 1 --2- 172.21.15.131:0/2369974480 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f200015e690 0x7f20001f4200 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:20.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:20.009+0000 7f2005a00640 1 --2- 172.21.15.131:0/2369974480 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f200015e2b0 0x7f20000cdc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:20.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:20.009+0000 7f2005a00640 1 --2- 172.21.15.131:0/2369974480 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f20000b9db0 0x7f20000c9e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:20.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:20.009+0000 7f2005a00640 1 -- 172.21.15.131:0/2369974480 >> 172.21.15.131:0/2369974480 conn(0x7f200009d500 msgr2=0x7f200015d910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:20.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:20.009+0000 7f2005a00640 1 -- 172.21.15.131:0/2369974480 shutdown_connections 2024-05-26T22:49:20.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:20.009+0000 7f2005a00640 1 -- 172.21.15.131:0/2369974480 wait complete. 2024-05-26T22:49:20.031 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-26T22:49:20.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:20.234+0000 7f10ea600640 1 Processor -- start 2024-05-26T22:49:20.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:20.235+0000 7f10ea600640 1 -- start start 2024-05-26T22:49:20.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:20.236+0000 7f10ea600640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f10e415ed30 0x7f10e40d0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:20.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:20.236+0000 7f10ea600640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f10e40d08a0 0x7f10e40b37b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:20.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:20.236+0000 7f10ea600640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10e415f190 con 0x7f10e406dd40 2024-05-26T22:49:20.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:20.236+0000 7f10ea600640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f10e415bb40 con 0x7f10e40d08a0 2024-05-26T22:49:20.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:20.236+0000 7f10ea600640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f10e415be20 con 0x7f10e415ed30 2024-05-26T22:49:20.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:20.237+0000 7f10e3400640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f10e406dd40 0x7f10e40b48b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:49516/0 (socket says 172.21.15.131:49516) 2024-05-26T22:49:20.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:20.237+0000 7f10e3400640 1 -- 172.21.15.131:0/1510451580 learned_addr learned my addr 172.21.15.131:0/1510451580 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:49:20.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:20.237+0000 7f10e2a00640 1 -- 172.21.15.131:0/1510451580 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 913828243 0 0) 0x7f10e415f190 con 0x7f10e406dd40 2024-05-26T22:49:20.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:20.237+0000 7f10e2a00640 1 -- 172.21.15.131:0/1510451580 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f10c8003520 con 0x7f10e406dd40 2024-05-26T22:49:20.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:20.237+0000 7f10daa00640 1 --2- 172.21.15.131:0/1510451580 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f10e415ed30 0x7f10e40d0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:20.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:20.237+0000 7f10e3e00640 1 --2- 172.21.15.131:0/1510451580 >> v2:172.21.15.187:3300/0 conn(0x7f10e40d08a0 0x7f10e40b37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:20.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:20.237+0000 7f10e2a00640 1 -- 172.21.15.131:0/1510451580 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2360477547 0 0) 0x7f10c8003520 con 0x7f10e406dd40 2024-05-26T22:49:20.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:20.237+0000 7f10e2a00640 1 -- 172.21.15.131:0/1510451580 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f10e415ed30 msgr2=0x7f10e40d0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:20.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:20.237+0000 7f10e2a00640 1 --2- 172.21.15.131:0/1510451580 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f10e415ed30 0x7f10e40d0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:20.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:20.237+0000 7f10e2a00640 1 -- 172.21.15.131:0/1510451580 >> v2:172.21.15.187:3300/0 conn(0x7f10e40d08a0 msgr2=0x7f10e40b37b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:20.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:20.237+0000 7f10e2a00640 1 --2- 172.21.15.131:0/1510451580 >> v2:172.21.15.187:3300/0 conn(0x7f10e40d08a0 0x7f10e40b37b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:20.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:20.237+0000 7f10e2a00640 1 -- 172.21.15.131:0/1510451580 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10e415c100 con 0x7f10e406dd40 2024-05-26T22:49:20.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:20.237+0000 7f10e2a00640 1 -- 172.21.15.131:0/1510451580 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7f10d4001670 con 0x7f10e406dd40 2024-05-26T22:49:20.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:20.238+0000 7f10e2a00640 1 -- 172.21.15.131:0/1510451580 >> v1:172.21.15.131:6789/0 conn(0x7f10e406dd40 legacy=0x7f10e40b48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:20.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:20.238+0000 7f10e2a00640 1 --2- 172.21.15.131:0/1510451580 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f10e415ed30 0x7f10c80039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:20.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:20.238+0000 7f10e2a00640 1 --2- 172.21.15.131:0/1510451580 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f10c8003fe0 0x7f10c8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:20.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:20.238+0000 7f10e2a00640 1 --2- 172.21.15.131:0/1510451580 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f10c8004d40 0x7f10c80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:20.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:20.238+0000 7f10e2a00640 1 -- 172.21.15.131:0/1510451580 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f10e415bb40 con 0x7f10c8003fe0 2024-05-26T22:49:20.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:20.238+0000 7f10e2a00640 1 -- 172.21.15.131:0/1510451580 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f10e415be20 con 0x7f10c8004d40 2024-05-26T22:49:20.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:20.238+0000 7f10e2a00640 1 -- 172.21.15.131:0/1510451580 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f10c800bb80 con 0x7f10e415ed30 2024-05-26T22:49:20.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:20.238+0000 7f10daa00640 1 --2- 172.21.15.131:0/1510451580 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f10c8003fe0 0x7f10c8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:20.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:20.238+0000 7f10e3e00640 1 --2- 172.21.15.131:0/1510451580 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f10c8004d40 0x7f10c80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:20.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:20.238+0000 7f10e3400640 1 --2- 172.21.15.131:0/1510451580 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f10e415ed30 0x7f10c80039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:20.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:20.238+0000 7f10daa00640 1 -- 172.21.15.131:0/1510451580 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f10e415ed30 msgr2=0x7f10c80039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:20.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:20.238+0000 7f10daa00640 1 --2- 172.21.15.131:0/1510451580 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f10e415ed30 0x7f10c80039b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:20.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:20.238+0000 7f10daa00640 1 -- 172.21.15.131:0/1510451580 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f10c8004d40 msgr2=0x7f10c80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:20.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:20.238+0000 7f10daa00640 1 --2- 172.21.15.131:0/1510451580 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f10c8004d40 0x7f10c80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:20.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:20.238+0000 7f10daa00640 1 -- 172.21.15.131:0/1510451580 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10c8003520 con 0x7f10c8003fe0 2024-05-26T22:49:20.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:20.238+0000 7f10e3e00640 1 --2- 172.21.15.131:0/1510451580 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f10c8004d40 0x7f10c80051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:49:20.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:20.239+0000 7f10daa00640 1 --2- 172.21.15.131:0/1510451580 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f10c8003fe0 0x7f10c8004800 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f10cc00b5c0 tx=0x7f10c8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74f78354f7c235cb server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:20.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:20.240+0000 7f10e2a00640 1 -- 172.21.15.131:0/1510451580 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10cc03a070 con 0x7f10c8003fe0 2024-05-26T22:49:20.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:20.240+0000 7f10e2a00640 1 -- 172.21.15.131:0/1510451580 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10cc035410 con 0x7f10c8003fe0 2024-05-26T22:49:20.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:20.240+0000 7f10e2a00640 1 -- 172.21.15.131:0/1510451580 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10cc00b600 con 0x7f10c8003fe0 2024-05-26T22:49:20.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:20.240+0000 7f10ea600640 1 -- 172.21.15.131:0/1510451580 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f10c8003fe0 msgr2=0x7f10c8004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:20.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:20.240+0000 7f10ea600640 1 --2- 172.21.15.131:0/1510451580 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f10c8003fe0 0x7f10c8004800 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f10cc00b5c0 tx=0x7f10c8005830 comp rx=0 tx=0).stop 2024-05-26T22:49:20.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:20.240+0000 7f10e3e00640 1 -- 172.21.15.131:0/1510451580 reap_dead start 2024-05-26T22:49:20.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:20.240+0000 7f10ea600640 1 -- 172.21.15.131:0/1510451580 shutdown_connections 2024-05-26T22:49:20.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:20.240+0000 7f10ea600640 1 -- 172.21.15.131:0/1510451580 >> 172.21.15.131:0/1510451580 conn(0x7f10e40b2110 msgr2=0x7f10e40b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:20.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:20.241+0000 7f10ea600640 1 -- 172.21.15.131:0/1510451580 shutdown_connections 2024-05-26T22:49:20.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:20.241+0000 7f10ea600640 1 -- 172.21.15.131:0/1510451580 wait complete. 2024-05-26T22:49:20.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:20.242+0000 7f10ea600640 1 Processor -- start 2024-05-26T22:49:20.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:20.242+0000 7f10ea600640 1 -- start start 2024-05-26T22:49:20.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:20.243+0000 7f10ea600640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f10c8003fe0 0x7f10e4158c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:20.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:20.243+0000 7f10ea600640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f10e41576a0 0x7f10e41ef3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:20.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:20.243+0000 7f10ea600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f10e409aa90 0x7f10e409b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:20.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:20.243+0000 7f10ea600640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f10e406e120 con 0x7f10e409aa90 2024-05-26T22:49:20.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:20.243+0000 7f10ea600640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f10e415b7d0 con 0x7f10c8003fe0 2024-05-26T22:49:20.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:20.243+0000 7f10ea600640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f10e40b4fc0 con 0x7f10e41576a0 2024-05-26T22:49:20.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:20.244+0000 7f10e3e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f10e409aa90 0x7f10e409b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:20.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:20.244+0000 7f10e3e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f10e409aa90 0x7f10e409b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40850/0 (socket says 172.21.15.131:40850) 2024-05-26T22:49:20.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:20.244+0000 7f10e3e00640 1 -- 172.21.15.131:0/3074803976 learned_addr learned my addr 172.21.15.131:0/3074803976 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:20.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:20.244+0000 7f10e3400640 1 --2- 172.21.15.131:0/3074803976 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f10c8003fe0 0x7f10e4158c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:20.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:20.244+0000 7f10daa00640 1 --2- 172.21.15.131:0/3074803976 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f10e41576a0 0x7f10e41ef3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:20.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:20.244+0000 7f10e3e00640 1 -- 172.21.15.131:0/3074803976 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f10e41576a0 msgr2=0x7f10e41ef3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:20.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:20.244+0000 7f10e3e00640 1 --2- 172.21.15.131:0/3074803976 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f10e41576a0 0x7f10e41ef3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:20.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:20.244+0000 7f10e3e00640 1 -- 172.21.15.131:0/3074803976 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f10c8003fe0 msgr2=0x7f10e4158c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:20.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:20.244+0000 7f10e3e00640 1 --2- 172.21.15.131:0/3074803976 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f10c8003fe0 0x7f10e4158c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:20.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:20.244+0000 7f10e3e00640 1 -- 172.21.15.131:0/3074803976 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10e4157ba0 con 0x7f10e409aa90 2024-05-26T22:49:20.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:20.244+0000 7f10e3400640 1 --2- 172.21.15.131:0/3074803976 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f10c8003fe0 0x7f10e4158c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:49:20.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:20.244+0000 7f10e3e00640 1 --2- 172.21.15.131:0/3074803976 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f10e409aa90 0x7f10e409b480 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f10e40b54a0 tx=0x7f10dc009d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:20.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:20.245+0000 7f10dbe00640 1 -- 172.21.15.131:0/3074803976 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10dc019070 con 0x7f10e409aa90 2024-05-26T22:49:20.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:20.245+0000 7f10dbe00640 1 -- 172.21.15.131:0/3074803976 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10dc007d10 con 0x7f10e409aa90 2024-05-26T22:49:20.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:20.245+0000 7f10ea600640 1 -- 172.21.15.131:0/3074803976 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10e4157e80 con 0x7f10e409aa90 2024-05-26T22:49:20.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:20.245+0000 7f10dbe00640 1 -- 172.21.15.131:0/3074803976 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10dc014640 con 0x7f10e409aa90 2024-05-26T22:49:20.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:20.245+0000 7f10ea600640 1 -- 172.21.15.131:0/3074803976 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10e4158440 con 0x7f10e409aa90 2024-05-26T22:49:20.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:20.246+0000 7f10ea600640 1 -- 172.21.15.131:0/3074803976 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10e4068440 con 0x7f10e409aa90 2024-05-26T22:49:20.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:20.247+0000 7f10dbe00640 1 -- 172.21.15.131:0/3074803976 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f10dc01c070 con 0x7f10e409aa90 2024-05-26T22:49:20.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:20.248+0000 7f10dbe00640 1 --2- 172.21.15.131:0/3074803976 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f10c006be30 0x7f10c006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:20.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:20.248+0000 7f10dbe00640 1 -- 172.21.15.131:0/3074803976 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5911+0+0 (secure 0 0 0) 0x7f10dc08aba0 con 0x7f10e409aa90 2024-05-26T22:49:20.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:20.248+0000 7f10e3400640 1 --2- 172.21.15.131:0/3074803976 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f10c006be30 0x7f10c006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:20.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:20.249+0000 7f10e3400640 1 --2- 172.21.15.131:0/3074803976 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f10c006be30 0x7f10c006e2f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f10d4001860 tx=0x7f10d4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:20.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:20.253+0000 7f10dbe00640 1 -- 172.21.15.131:0/3074803976 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f10dc0593e0 con 0x7f10e409aa90 2024-05-26T22:49:20.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:20.453+0000 7f10ea600640 1 -- 172.21.15.131:0/3074803976 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f10e41ef930 con 0x7f10e409aa90 2024-05-26T22:49:21.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:21.421+0000 7f10dbe00640 1 -- 172.21.15.131:0/3074803976 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v24) ==== 181+0+0 (secure 0 0 0) 0x7f10dc058ce0 con 0x7f10e409aa90 2024-05-26T22:49:21.421 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_metadata' removed 2024-05-26T22:49:21.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:21.424+0000 7f10ea600640 1 -- 172.21.15.131:0/3074803976 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f10c006be30 msgr2=0x7f10c006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:21.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:21.424+0000 7f10ea600640 1 --2- 172.21.15.131:0/3074803976 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f10c006be30 0x7f10c006e2f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f10d4001860 tx=0x7f10d4009040 comp rx=0 tx=0).stop 2024-05-26T22:49:21.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:21.424+0000 7f10ea600640 1 -- 172.21.15.131:0/3074803976 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f10e409aa90 msgr2=0x7f10e409b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:21.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:21.424+0000 7f10ea600640 1 --2- 172.21.15.131:0/3074803976 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f10e409aa90 0x7f10e409b480 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f10e40b54a0 tx=0x7f10dc009d40 comp rx=0 tx=0).stop 2024-05-26T22:49:21.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:21.425+0000 7f10ea600640 1 -- 172.21.15.131:0/3074803976 shutdown_connections 2024-05-26T22:49:21.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:21.425+0000 7f10ea600640 1 --2- 172.21.15.131:0/3074803976 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f10e41576a0 0x7f10e41ef3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:21.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:21.425+0000 7f10ea600640 1 --2- 172.21.15.131:0/3074803976 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f10c006be30 0x7f10c006e2f0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:21.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:21.425+0000 7f10ea600640 1 --2- 172.21.15.131:0/3074803976 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f10e409aa90 0x7f10e409b480 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:21.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:21.425+0000 7f10ea600640 1 --2- 172.21.15.131:0/3074803976 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f10c8003fe0 0x7f10e4158c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:21.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:21.425+0000 7f10ea600640 1 -- 172.21.15.131:0/3074803976 >> 172.21.15.131:0/3074803976 conn(0x7f10e40b2110 msgr2=0x7f10e4071340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:21.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:21.426+0000 7f10ea600640 1 -- 172.21.15.131:0/3074803976 shutdown_connections 2024-05-26T22:49:21.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:21.426+0000 7f10ea600640 1 -- 172.21.15.131:0/3074803976 wait complete. 2024-05-26T22:49:21.449 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-26T22:49:21.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:21.643+0000 7f0ec1000640 1 Processor -- start 2024-05-26T22:49:21.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:21.643+0000 7f0ec1000640 1 -- start start 2024-05-26T22:49:21.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:21.644+0000 7f0ec1000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ebc15e6b0 0x7f0ebc0b3790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:21.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:21.644+0000 7f0ec1000640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f0ebc0d4c30 0x7f0ebc06c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:21.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:21.645+0000 7f0ec1000640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ebc15ea90 con 0x7f0ebc0d2880 2024-05-26T22:49:21.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:21.645+0000 7f0ec1000640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f0ebc0cf4c0 con 0x7f0ebc0d4c30 2024-05-26T22:49:21.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:21.645+0000 7f0ec1000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f0ebc0ceeb0 con 0x7f0ebc15e6b0 2024-05-26T22:49:21.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:21.645+0000 7f0eba800640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f0ebc0d2880 0x7f0ebc0cf950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:49532/0 (socket says 172.21.15.131:49532) 2024-05-26T22:49:21.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:21.645+0000 7f0eba800640 1 -- 172.21.15.131:0/3277252832 learned_addr learned my addr 172.21.15.131:0/3277252832 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:49:21.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:21.645+0000 7f0eb9400640 1 --2- 172.21.15.131:0/3277252832 >> v2:172.21.15.187:3300/0 conn(0x7f0ebc0d4c30 0x7f0ebc06c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:21.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:21.645+0000 7f0eb9e00640 1 --2- 172.21.15.131:0/3277252832 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ebc15e6b0 0x7f0ebc0b3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:21.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:21.645+0000 7f0eb8a00640 1 -- 172.21.15.131:0/3277252832 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2299699733 0 0) 0x7f0ebc15ea90 con 0x7f0ebc0d2880 2024-05-26T22:49:21.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:21.645+0000 7f0eb8a00640 1 -- 172.21.15.131:0/3277252832 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0ea0003520 con 0x7f0ebc0d2880 2024-05-26T22:49:21.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:21.646+0000 7f0eb8a00640 1 -- 172.21.15.131:0/3277252832 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2446658531 0 0) 0x7f0ea0003520 con 0x7f0ebc0d2880 2024-05-26T22:49:21.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:21.646+0000 7f0eb8a00640 1 -- 172.21.15.131:0/3277252832 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ebc15e6b0 msgr2=0x7f0ebc0b3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:21.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:21.646+0000 7f0eb8a00640 1 --2- 172.21.15.131:0/3277252832 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ebc15e6b0 0x7f0ebc0b3790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:21.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:21.646+0000 7f0eb8a00640 1 -- 172.21.15.131:0/3277252832 >> v2:172.21.15.187:3300/0 conn(0x7f0ebc0d4c30 msgr2=0x7f0ebc06c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:21.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:21.646+0000 7f0eb8a00640 1 --2- 172.21.15.131:0/3277252832 >> v2:172.21.15.187:3300/0 conn(0x7f0ebc0d4c30 0x7f0ebc06c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:21.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:21.646+0000 7f0eb8a00640 1 -- 172.21.15.131:0/3277252832 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ebc0d1380 con 0x7f0ebc0d2880 2024-05-26T22:49:21.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:21.646+0000 7f0eb8a00640 1 -- 172.21.15.131:0/3277252832 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7f0eb4001670 con 0x7f0ebc0d2880 2024-05-26T22:49:21.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:21.646+0000 7f0eb8a00640 1 -- 172.21.15.131:0/3277252832 >> v1:172.21.15.131:6789/0 conn(0x7f0ebc0d2880 legacy=0x7f0ebc0cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:21.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:21.646+0000 7f0eb8a00640 1 --2- 172.21.15.131:0/3277252832 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ebc15e6b0 0x7f0ea00039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:21.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:21.646+0000 7f0eb8a00640 1 --2- 172.21.15.131:0/3277252832 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ea0003f30 0x7f0ea0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:21.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:21.646+0000 7f0eb8a00640 1 --2- 172.21.15.131:0/3277252832 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ea0004d40 0x7f0ea00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:21.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:21.646+0000 7f0eb8a00640 1 -- 172.21.15.131:0/3277252832 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f0ebc0cf4c0 con 0x7f0ea0004d40 2024-05-26T22:49:21.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:21.646+0000 7f0eb8a00640 1 -- 172.21.15.131:0/3277252832 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0ebc0ceeb0 con 0x7f0ea0003f30 2024-05-26T22:49:21.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:21.646+0000 7f0eb8a00640 1 -- 172.21.15.131:0/3277252832 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f0ea000ba10 con 0x7f0ebc15e6b0 2024-05-26T22:49:21.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:21.646+0000 7f0eba800640 1 --2- 172.21.15.131:0/3277252832 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ea0004d40 0x7f0ea00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:21.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:21.647+0000 7f0eb9400640 1 --2- 172.21.15.131:0/3277252832 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ea0003f30 0x7f0ea0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:21.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:21.647+0000 7f0eb9e00640 1 --2- 172.21.15.131:0/3277252832 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ebc15e6b0 0x7f0ea00039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:21.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:21.647+0000 7f0eba800640 1 -- 172.21.15.131:0/3277252832 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ebc15e6b0 msgr2=0x7f0ea00039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:21.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:21.647+0000 7f0eba800640 1 --2- 172.21.15.131:0/3277252832 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ebc15e6b0 0x7f0ea00039b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:21.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:21.647+0000 7f0eba800640 1 -- 172.21.15.131:0/3277252832 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ea0003f30 msgr2=0x7f0ea0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:21.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:21.647+0000 7f0eba800640 1 --2- 172.21.15.131:0/3277252832 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ea0003f30 0x7f0ea0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:21.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:21.647+0000 7f0eba800640 1 -- 172.21.15.131:0/3277252832 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ea0003520 con 0x7f0ea0004d40 2024-05-26T22:49:21.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:21.647+0000 7f0eb9400640 1 --2- 172.21.15.131:0/3277252832 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ea0003f30 0x7f0ea0004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:49:21.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:21.648+0000 7f0eba800640 1 --2- 172.21.15.131:0/3277252832 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ea0004d40 0x7f0ea00051c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f0eb400c210 tx=0x7f0ea0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3eb71425d073f00 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:21.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:21.648+0000 7f0eb8a00640 1 -- 172.21.15.131:0/3277252832 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb4007d50 con 0x7f0ea0004d40 2024-05-26T22:49:21.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:21.648+0000 7f0eb8a00640 1 -- 172.21.15.131:0/3277252832 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0eb4005ad0 con 0x7f0ea0004d40 2024-05-26T22:49:21.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:21.648+0000 7f0eb8a00640 1 -- 172.21.15.131:0/3277252832 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb4007070 con 0x7f0ea0004d40 2024-05-26T22:49:21.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:21.648+0000 7f0ec1000640 1 -- 172.21.15.131:0/3277252832 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ea0004d40 msgr2=0x7f0ea00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:21.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:21.648+0000 7f0ec1000640 1 --2- 172.21.15.131:0/3277252832 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ea0004d40 0x7f0ea00051c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f0eb400c210 tx=0x7f0ea0005830 comp rx=0 tx=0).stop 2024-05-26T22:49:21.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:21.648+0000 7f0eba800640 1 -- 172.21.15.131:0/3277252832 reap_dead start 2024-05-26T22:49:21.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:21.649+0000 7f0ec1000640 1 -- 172.21.15.131:0/3277252832 shutdown_connections 2024-05-26T22:49:21.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:21.649+0000 7f0ec1000640 1 -- 172.21.15.131:0/3277252832 >> 172.21.15.131:0/3277252832 conn(0x7f0ebc09d610 msgr2=0x7f0ebc0d42a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:21.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:21.649+0000 7f0ec1000640 1 -- 172.21.15.131:0/3277252832 shutdown_connections 2024-05-26T22:49:21.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:21.649+0000 7f0ec1000640 1 -- 172.21.15.131:0/3277252832 wait complete. 2024-05-26T22:49:21.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:21.650+0000 7f0ec1000640 1 Processor -- start 2024-05-26T22:49:21.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:21.651+0000 7f0ec1000640 1 -- start start 2024-05-26T22:49:21.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:21.651+0000 7f0ec1000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ea0004d40 0x7f0ebc1560e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:21.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:21.651+0000 7f0ec1000640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ebc15fd60 0x7f0ebc1601e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:21.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:21.651+0000 7f0ec1000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ebc0cbc80 0x7f0ebc0cc130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:21.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:21.651+0000 7f0ec1000640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f0ebc0d5090 con 0x7f0ea0004d40 2024-05-26T22:49:21.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:21.651+0000 7f0ec1000640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0ebc0cf150 con 0x7f0ebc15fd60 2024-05-26T22:49:21.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:21.651+0000 7f0ec1000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f0ebc0cc670 con 0x7f0ebc0cbc80 2024-05-26T22:49:21.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:21.652+0000 7f0eb9e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ea0004d40 0x7f0ebc1560e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:21.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:21.652+0000 7f0eb9e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ea0004d40 0x7f0ebc1560e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40862/0 (socket says 172.21.15.131:40862) 2024-05-26T22:49:21.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:21.652+0000 7f0eb9e00640 1 -- 172.21.15.131:0/3707172157 learned_addr learned my addr 172.21.15.131:0/3707172157 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:21.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:21.652+0000 7f0eb9e00640 1 -- 172.21.15.131:0/3707172157 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ebc0cbc80 msgr2=0x7f0ebc0cc130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:21.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:21.652+0000 7f0eb9400640 1 --2- 172.21.15.131:0/3707172157 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ebc15fd60 0x7f0ebc1601e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:21.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:21.652+0000 7f0eb9e00640 1 --2- 172.21.15.131:0/3707172157 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ebc0cbc80 0x7f0ebc0cc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:21.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:21.652+0000 7f0eb9e00640 1 -- 172.21.15.131:0/3707172157 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ebc15fd60 msgr2=0x7f0ebc1601e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:21.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:21.652+0000 7f0eb9e00640 1 --2- 172.21.15.131:0/3707172157 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ebc15fd60 0x7f0ebc1601e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:21.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:21.652+0000 7f0eb9e00640 1 -- 172.21.15.131:0/3707172157 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ebc1f7960 con 0x7f0ea0004d40 2024-05-26T22:49:21.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:21.652+0000 7f0eb9e00640 1 --2- 172.21.15.131:0/3707172157 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ea0004d40 0x7f0ebc1560e0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f0eac010810 tx=0x7f0eac010cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:21.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:21.653+0000 7f0eb2000640 1 -- 172.21.15.131:0/3707172157 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eac00f040 con 0x7f0ea0004d40 2024-05-26T22:49:21.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:21.653+0000 7f0eb2000640 1 -- 172.21.15.131:0/3707172157 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0eac00eb00 con 0x7f0ea0004d40 2024-05-26T22:49:21.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:21.653+0000 7f0ec1000640 1 -- 172.21.15.131:0/3707172157 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ebc1f7c40 con 0x7f0ea0004d40 2024-05-26T22:49:21.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:21.653+0000 7f0eb2000640 1 -- 172.21.15.131:0/3707172157 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eac01c400 con 0x7f0ea0004d40 2024-05-26T22:49:21.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:21.653+0000 7f0ec1000640 1 -- 172.21.15.131:0/3707172157 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ebc0b3790 con 0x7f0ea0004d40 2024-05-26T22:49:21.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:21.654+0000 7f0ec1000640 1 -- 172.21.15.131:0/3707172157 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ebc068440 con 0x7f0ea0004d40 2024-05-26T22:49:21.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:21.655+0000 7f0eb2000640 1 -- 172.21.15.131:0/3707172157 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0eac01c5a0 con 0x7f0ea0004d40 2024-05-26T22:49:21.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:21.656+0000 7f0eb2000640 1 --2- 172.21.15.131:0/3707172157 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f0e9c06be60 0x7f0e9c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:21.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:21.656+0000 7f0eb2000640 1 -- 172.21.15.131:0/3707172157 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5473+0+0 (secure 0 0 0) 0x7f0eac08b340 con 0x7f0ea0004d40 2024-05-26T22:49:21.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:21.656+0000 7f0eb9400640 1 --2- 172.21.15.131:0/3707172157 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f0e9c06be60 0x7f0e9c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:21.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:21.657+0000 7f0eb9400640 1 --2- 172.21.15.131:0/3707172157 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f0e9c06be60 0x7f0e9c06e320 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc160ae0 tx=0x7f0ea800b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:21.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:21.661+0000 7f0eb2000640 1 -- 172.21.15.131:0/3707172157 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f0eac018030 con 0x7f0ea0004d40 2024-05-26T22:49:21.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:21.871+0000 7f0ec1000640 1 -- 172.21.15.131:0/3707172157 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f0ebc0d1a90 con 0x7f0ea0004d40 2024-05-26T22:49:22.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:22.424+0000 7f0eb2000640 1 -- 172.21.15.131:0/3707172157 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v25) ==== 169+0+0 (secure 0 0 0) 0x7f0eac059d30 con 0x7f0ea0004d40 2024-05-26T22:49:22.424 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_data' removed 2024-05-26T22:49:22.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:22.428+0000 7f0ec1000640 1 -- 172.21.15.131:0/3707172157 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f0e9c06be60 msgr2=0x7f0e9c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:22.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:22.428+0000 7f0ec1000640 1 --2- 172.21.15.131:0/3707172157 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f0e9c06be60 0x7f0e9c06e320 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc160ae0 tx=0x7f0ea800b040 comp rx=0 tx=0).stop 2024-05-26T22:49:22.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:22.428+0000 7f0ec1000640 1 -- 172.21.15.131:0/3707172157 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ea0004d40 msgr2=0x7f0ebc1560e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:22.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:22.428+0000 7f0ec1000640 1 --2- 172.21.15.131:0/3707172157 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ea0004d40 0x7f0ebc1560e0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f0eac010810 tx=0x7f0eac010cd0 comp rx=0 tx=0).stop 2024-05-26T22:49:22.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:22.428+0000 7f0ec1000640 1 -- 172.21.15.131:0/3707172157 shutdown_connections 2024-05-26T22:49:22.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:22.428+0000 7f0ec1000640 1 --2- 172.21.15.131:0/3707172157 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ebc0cbc80 0x7f0ebc0cc130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:22.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:22.428+0000 7f0ec1000640 1 --2- 172.21.15.131:0/3707172157 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f0e9c06be60 0x7f0e9c06e320 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:22.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:22.428+0000 7f0ec1000640 1 --2- 172.21.15.131:0/3707172157 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ebc15fd60 0x7f0ebc1601e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:22.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:22.428+0000 7f0ec1000640 1 --2- 172.21.15.131:0/3707172157 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ea0004d40 0x7f0ebc1560e0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:22.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:22.428+0000 7f0ec1000640 1 -- 172.21.15.131:0/3707172157 >> 172.21.15.131:0/3707172157 conn(0x7f0ebc09d610 msgr2=0x7f0ebc0b20f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:22.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:22.429+0000 7f0ec1000640 1 -- 172.21.15.131:0/3707172157 shutdown_connections 2024-05-26T22:49:22.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:22.429+0000 7f0ec1000640 1 -- 172.21.15.131:0/3707172157 wait complete. 2024-05-26T22:49:22.455 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-26T22:49:22.456 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-26T22:49:22.456 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-26T22:49:22.456 INFO:teuthology.orchestra.run:waiting for 300 2024-05-26T22:49:22.456 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-26T22:49:22.456 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-26T22:49:22.456 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-26T22:49:22.457 INFO:teuthology.orchestra.run:waiting for 300 2024-05-26T22:49:22.457 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-26T22:49:22.457 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-26T22:49:22.457 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-26T22:49:22.457 INFO:teuthology.orchestra.run:waiting for 300 2024-05-26T22:49:22.457 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-26T22:49:22.457 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-26T22:49:22.457 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-26T22:49:22.458 INFO:teuthology.orchestra.run:waiting for 300 2024-05-26T22:49:22.477 INFO:tasks.ceph.mds.c:Stopped 2024-05-26T22:49:22.478 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-26T22:49:22.480 INFO:tasks.ceph.mds.b:Stopped 2024-05-26T22:49:22.480 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-26T22:49:22.481 INFO:tasks.ceph.mds.a:Stopped 2024-05-26T22:49:22.481 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-26T22:49:22.487 INFO:tasks.ceph.mds.d:Stopped 2024-05-26T22:49:22.487 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-26T22:49:22.532 INFO:tasks.ceph.mds.c:Started 2024-05-26T22:49:22.532 DEBUG:teuthology.parallel:result is None 2024-05-26T22:49:22.533 INFO:tasks.ceph.mds.b:Started 2024-05-26T22:49:22.533 DEBUG:teuthology.parallel:result is None 2024-05-26T22:49:22.537 INFO:tasks.ceph.mds.a:Started 2024-05-26T22:49:22.537 DEBUG:teuthology.parallel:result is None 2024-05-26T22:49:22.539 INFO:tasks.ceph.mds.d:Started 2024-05-26T22:49:22.540 DEBUG:teuthology.parallel:result is None 2024-05-26T22:49:22.540 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-26T22:49:22.679 INFO:tasks.ceph.mds.a.smithi131.stdout:starting mds.a at 2024-05-26T22:49:22.679 INFO:tasks.ceph.mds.c.smithi131.stdout:starting mds.c at 2024-05-26T22:49:22.682 INFO:tasks.ceph.mds.b.smithi187.stdout:starting mds.b at 2024-05-26T22:49:22.690 INFO:tasks.ceph.mds.d.smithi187.stdout:starting mds.d at 2024-05-26T22:49:22.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:22.716+0000 7f1d9d200640 1 Processor -- start 2024-05-26T22:49:22.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:22.717+0000 7f1d9d200640 1 -- start start 2024-05-26T22:49:22.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:22.718+0000 7f1d9d200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f1d9806f180 0x7f1d98072e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:22.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:22.718+0000 7f1d9d200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1d98072540 0x7f1d98161ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:22.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:22.718+0000 7f1d9d200640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d980af8e0 con 0x7f1d980cfe50 2024-05-26T22:49:22.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:22.718+0000 7f1d9d200640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f1d980cfb30 con 0x7f1d9806f180 2024-05-26T22:49:22.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:22.718+0000 7f1d9d200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1d980cf200 con 0x7f1d98072540 2024-05-26T22:49:22.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:22.718+0000 7f1d95e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f1d9806f180 0x7f1d98072e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:22.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:22.718+0000 7f1d95e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f1d9806f180 0x7f1d98072e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40920/0 (socket says 172.21.15.131:40920) 2024-05-26T22:49:22.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:22.718+0000 7f1d95e00640 1 -- 172.21.15.131:0/447286551 learned_addr learned my addr 172.21.15.131:0/447286551 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:22.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:22.718+0000 7f1d95400640 1 --2- 172.21.15.131:0/447286551 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1d98072540 0x7f1d98161ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:22.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:22.719+0000 7f1d95e00640 1 -- 172.21.15.131:0/447286551 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1d98072540 msgr2=0x7f1d98161ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:22.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:22.719+0000 7f1d95e00640 1 --2- 172.21.15.131:0/447286551 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1d98072540 0x7f1d98161ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:22.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:22.719+0000 7f1d95e00640 1 -- 172.21.15.131:0/447286551 >> v1:172.21.15.187:6789/0 conn(0x7f1d980cfe50 legacy=0x7f1d980715a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:22.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:22.719+0000 7f1d95e00640 1 -- 172.21.15.131:0/447286551 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d980cf4e0 con 0x7f1d9806f180 2024-05-26T22:49:22.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:22.720+0000 7f1d95e00640 1 --2- 172.21.15.131:0/447286551 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d9806f180 0x7f1d98072e90 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f1d88009870 tx=0x7f1d8802a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=759c227464c5540f server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:22.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:22.720+0000 7f1d94a00640 1 -- 172.21.15.131:0/447286551 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d88037070 con 0x7f1d9806f180 2024-05-26T22:49:22.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:22.720+0000 7f1d94a00640 1 -- 172.21.15.131:0/447286551 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d88032440 con 0x7f1d9806f180 2024-05-26T22:49:22.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:22.720+0000 7f1d94a00640 1 -- 172.21.15.131:0/447286551 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d88032810 con 0x7f1d9806f180 2024-05-26T22:49:22.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:22.720+0000 7f1d9d200640 1 -- 172.21.15.131:0/447286551 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d9806f180 msgr2=0x7f1d98072e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:22.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:22.721+0000 7f1d9d200640 1 --2- 172.21.15.131:0/447286551 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d9806f180 0x7f1d98072e90 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f1d88009870 tx=0x7f1d8802a250 comp rx=0 tx=0).stop 2024-05-26T22:49:22.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:22.721+0000 7f1d9d200640 1 -- 172.21.15.131:0/447286551 shutdown_connections 2024-05-26T22:49:22.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:22.721+0000 7f1d9d200640 1 --2- 172.21.15.131:0/447286551 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1d98072540 0x7f1d98161ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:22.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:22.721+0000 7f1d9d200640 1 --2- 172.21.15.131:0/447286551 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d9806f180 0x7f1d98072e90 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:22.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:22.721+0000 7f1d9d200640 1 -- 172.21.15.131:0/447286551 >> 172.21.15.131:0/447286551 conn(0x7f1d9806bc90 msgr2=0x7f1d980b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:22.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:22.721+0000 7f1d9d200640 1 -- 172.21.15.131:0/447286551 shutdown_connections 2024-05-26T22:49:22.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:22.721+0000 7f1d9d200640 1 -- 172.21.15.131:0/447286551 wait complete. 2024-05-26T22:49:22.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:22.722+0000 7f1d9d200640 1 Processor -- start 2024-05-26T22:49:22.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:22.723+0000 7f1d9d200640 1 -- start start 2024-05-26T22:49:22.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:22.723+0000 7f1d9d200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d9806f180 0x7f1d98151b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:22.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:22.723+0000 7f1d9d200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1d98072540 0x7f1d980b4300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:22.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:22.724+0000 7f1d9d200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1d980cfe50 0x7f1d980b4840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:22.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:22.724+0000 7f1d9d200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f1d9806f6e0 con 0x7f1d9806f180 2024-05-26T22:49:22.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:22.724+0000 7f1d9d200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1d9806f560 con 0x7f1d980cfe50 2024-05-26T22:49:22.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:22.724+0000 7f1d9d200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1d980d02e0 con 0x7f1d98072540 2024-05-26T22:49:22.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:22.724+0000 7f1d95e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d9806f180 0x7f1d98151b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:22.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:22.724+0000 7f1d95e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d9806f180 0x7f1d98151b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40926/0 (socket says 172.21.15.131:40926) 2024-05-26T22:49:22.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:22.724+0000 7f1d95e00640 1 -- 172.21.15.131:0/4135638926 learned_addr learned my addr 172.21.15.131:0/4135638926 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:22.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:22.724+0000 7f1d95400640 1 --2- 172.21.15.131:0/4135638926 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1d98072540 0x7f1d980b4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:22.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:22.724+0000 7f1d96800640 1 --2- 172.21.15.131:0/4135638926 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1d980cfe50 0x7f1d980b4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:22.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:22.724+0000 7f1d95e00640 1 -- 172.21.15.131:0/4135638926 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1d98072540 msgr2=0x7f1d980b4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:22.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:22.724+0000 7f1d95e00640 1 --2- 172.21.15.131:0/4135638926 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1d98072540 0x7f1d980b4300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:22.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:22.724+0000 7f1d95e00640 1 -- 172.21.15.131:0/4135638926 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1d980cfe50 msgr2=0x7f1d980b4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:22.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:22.724+0000 7f1d95e00640 1 --2- 172.21.15.131:0/4135638926 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1d980cfe50 0x7f1d980b4840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:22.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:22.724+0000 7f1d95e00640 1 -- 172.21.15.131:0/4135638926 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d980c9430 con 0x7f1d9806f180 2024-05-26T22:49:22.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:22.724+0000 7f1d95400640 1 --2- 172.21.15.131:0/4135638926 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1d98072540 0x7f1d980b4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-26T22:49:22.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:22.725+0000 7f1d95e00640 1 --2- 172.21.15.131:0/4135638926 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d9806f180 0x7f1d98151b80 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f1d88000c00 tx=0x7f1d880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:22.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:22.725+0000 7f1d8e000640 1 -- 172.21.15.131:0/4135638926 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d88037040 con 0x7f1d9806f180 2024-05-26T22:49:22.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:22.725+0000 7f1d8e000640 1 -- 172.21.15.131:0/4135638926 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d88042070 con 0x7f1d9806f180 2024-05-26T22:49:22.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:22.725+0000 7f1d8e000640 1 -- 172.21.15.131:0/4135638926 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d88030880 con 0x7f1d9806f180 2024-05-26T22:49:22.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:22.725+0000 7f1d9d200640 1 -- 172.21.15.131:0/4135638926 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d980c9710 con 0x7f1d9806f180 2024-05-26T22:49:22.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:22.725+0000 7f1d9d200640 1 -- 172.21.15.131:0/4135638926 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d980c9cd0 con 0x7f1d9806f180 2024-05-26T22:49:22.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:22.726+0000 7f1d9d200640 1 -- 172.21.15.131:0/4135638926 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d98074e90 con 0x7f1d9806f180 2024-05-26T22:49:22.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:22.727+0000 7f1d8e000640 1 -- 172.21.15.131:0/4135638926 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1d88030a20 con 0x7f1d9806f180 2024-05-26T22:49:22.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:22.728+0000 7f1d8e000640 1 --2- 172.21.15.131:0/4135638926 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f1d7806be60 0x7f1d7806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:22.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:22.728+0000 7f1d8e000640 1 -- 172.21.15.131:0/4135638926 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 4943+0+0 (secure 0 0 0) 0x7f1d880ab2e0 con 0x7f1d9806f180 2024-05-26T22:49:22.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:22.729+0000 7f1d95400640 1 --2- 172.21.15.131:0/4135638926 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f1d7806be60 0x7f1d7806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:22.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:22.729+0000 7f1d95400640 1 --2- 172.21.15.131:0/4135638926 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f1d7806be60 0x7f1d7806e320 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f1d980aff00 tx=0x7f1d84009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:22.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:22.733+0000 7f1d8e000640 1 -- 172.21.15.131:0/4135638926 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f1d880798a0 con 0x7f1d9806f180 2024-05-26T22:49:22.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:22.947+0000 7f1d9d200640 1 -- 172.21.15.131:0/4135638926 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f1d981603b0 con 0x7f1d9806f180 2024-05-26T22:49:23.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:23.459+0000 7f1d8e000640 1 -- 172.21.15.131:0/4135638926 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v27) ==== 97+0+0 (secure 0 0 0) 0x7f1d8807dba0 con 0x7f1d9806f180 2024-05-26T22:49:23.459 INFO:teuthology.orchestra.run.smithi131.stderr: removed all blocklist entries 2024-05-26T22:49:23.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:23.462+0000 7f1d9d200640 1 -- 172.21.15.131:0/4135638926 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f1d7806be60 msgr2=0x7f1d7806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:23.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:23.463+0000 7f1d9d200640 1 --2- 172.21.15.131:0/4135638926 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f1d7806be60 0x7f1d7806e320 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f1d980aff00 tx=0x7f1d84009190 comp rx=0 tx=0).stop 2024-05-26T22:49:23.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:23.463+0000 7f1d9d200640 1 -- 172.21.15.131:0/4135638926 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d9806f180 msgr2=0x7f1d98151b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:23.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:23.463+0000 7f1d9d200640 1 --2- 172.21.15.131:0/4135638926 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d9806f180 0x7f1d98151b80 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f1d88000c00 tx=0x7f1d880046c0 comp rx=0 tx=0).stop 2024-05-26T22:49:23.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:23.463+0000 7f1d9d200640 1 -- 172.21.15.131:0/4135638926 shutdown_connections 2024-05-26T22:49:23.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:23.463+0000 7f1d9d200640 1 --2- 172.21.15.131:0/4135638926 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1d98072540 0x7f1d980b4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:23.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:23.463+0000 7f1d9d200640 1 --2- 172.21.15.131:0/4135638926 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f1d7806be60 0x7f1d7806e320 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:23.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:23.463+0000 7f1d9d200640 1 --2- 172.21.15.131:0/4135638926 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1d980cfe50 0x7f1d980b4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:23.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:23.463+0000 7f1d9d200640 1 --2- 172.21.15.131:0/4135638926 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d9806f180 0x7f1d98151b80 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:23.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:23.463+0000 7f1d9d200640 1 -- 172.21.15.131:0/4135638926 >> 172.21.15.131:0/4135638926 conn(0x7f1d9806bc90 msgr2=0x7f1d98160a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:23.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:23.463+0000 7f1d9d200640 1 -- 172.21.15.131:0/4135638926 shutdown_connections 2024-05-26T22:49:23.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:23.463+0000 7f1d9d200640 1 -- 172.21.15.131:0/4135638926 wait complete. 2024-05-26T22:49:23.491 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-26T22:49:23.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:23.677+0000 7f641ca00640 1 Processor -- start 2024-05-26T22:49:23.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:23.677+0000 7f641ca00640 1 -- start start 2024-05-26T22:49:23.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:23.679+0000 7f641ca00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f6418166e60 0x7f6418169250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:23.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:23.679+0000 7f641ca00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64180d3f50 con 0x7f64180b5be0 2024-05-26T22:49:23.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:23.679+0000 7f641ca00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64180d2eb0 con 0x7f64180b9e70 2024-05-26T22:49:23.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:23.679+0000 7f641ca00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f64180d3ac0 con 0x7f6418166e60 2024-05-26T22:49:23.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:23.679+0000 7f6417200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f6418166e60 0x7f6418169250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:23.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:23.679+0000 7f6416800640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f64180b5be0 0x7f64180b5fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:49558/0 (socket says 172.21.15.131:49558) 2024-05-26T22:49:23.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:23.679+0000 7f6416800640 1 -- 172.21.15.131:0/1999734105 learned_addr learned my addr 172.21.15.131:0/1999734105 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:49:23.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:23.680+0000 7f6417200640 1 -- 172.21.15.131:0/1999734105 >> v1:172.21.15.187:6789/0 conn(0x7f64180b9e70 legacy=0x7f64180cef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:23.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:23.680+0000 7f6417200640 1 -- 172.21.15.131:0/1999734105 >> v1:172.21.15.131:6789/0 conn(0x7f64180b5be0 legacy=0x7f64180b5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:23.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:23.680+0000 7f6417200640 1 -- 172.21.15.131:0/1999734105 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64180d3190 con 0x7f6418166e60 2024-05-26T22:49:23.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:23.680+0000 7f6415400640 1 -- 172.21.15.131:0/1999734105 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3388284452 0 0) 0x7f64180d3f50 con 0x7f64180b5be0 2024-05-26T22:49:23.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:23.681+0000 7f6417200640 1 --2- 172.21.15.131:0/1999734105 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6418166e60 0x7f6418169250 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f6410009d90 tx=0x7f641002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca3290abd2a8f3f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:23.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:23.681+0000 7f6415400640 1 -- 172.21.15.131:0/1999734105 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6410007740 con 0x7f6418166e60 2024-05-26T22:49:23.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:23.681+0000 7f6415400640 1 -- 172.21.15.131:0/1999734105 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64100078e0 con 0x7f6418166e60 2024-05-26T22:49:23.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:23.681+0000 7f6415400640 1 -- 172.21.15.131:0/1999734105 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6410007cb0 con 0x7f6418166e60 2024-05-26T22:49:23.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:23.682+0000 7f641ca00640 1 -- 172.21.15.131:0/1999734105 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6418166e60 msgr2=0x7f6418169250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:23.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:23.682+0000 7f641ca00640 1 --2- 172.21.15.131:0/1999734105 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6418166e60 0x7f6418169250 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f6410009d90 tx=0x7f641002a6a0 comp rx=0 tx=0).stop 2024-05-26T22:49:23.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:23.682+0000 7f641ca00640 1 -- 172.21.15.131:0/1999734105 shutdown_connections 2024-05-26T22:49:23.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:23.682+0000 7f641ca00640 1 --2- 172.21.15.131:0/1999734105 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6418166e60 0x7f6418169250 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:23.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:23.682+0000 7f641ca00640 1 -- 172.21.15.131:0/1999734105 >> 172.21.15.131:0/1999734105 conn(0x7f6418073070 msgr2=0x7f64180d4160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:23.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:23.682+0000 7f641ca00640 1 -- 172.21.15.131:0/1999734105 shutdown_connections 2024-05-26T22:49:23.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:23.682+0000 7f641ca00640 1 -- 172.21.15.131:0/1999734105 wait complete. 2024-05-26T22:49:23.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:23.684+0000 7f641ca00640 1 Processor -- start 2024-05-26T22:49:23.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:23.684+0000 7f641ca00640 1 -- start start 2024-05-26T22:49:23.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:23.685+0000 7f641ca00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f64180b5be0 0x7f64180cb310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:23.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:23.685+0000 7f641ca00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f64180b9e70 0x7f641815c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:23.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:23.685+0000 7f641ca00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6418166e60 0x7f64180cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:23.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:23.685+0000 7f641ca00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f64180d3750 con 0x7f64180b9e70 2024-05-26T22:49:23.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:23.685+0000 7f641ca00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f64180cf640 con 0x7f64180b5be0 2024-05-26T22:49:23.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:23.685+0000 7f641ca00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f641815e6a0 con 0x7f6418166e60 2024-05-26T22:49:23.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:23.685+0000 7f6415e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f64180b9e70 0x7f641815c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:23.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:23.685+0000 7f6415e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f64180b9e70 0x7f641815c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40936/0 (socket says 172.21.15.131:40936) 2024-05-26T22:49:23.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:23.685+0000 7f6415e00640 1 -- 172.21.15.131:0/2961039737 learned_addr learned my addr 172.21.15.131:0/2961039737 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:23.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:23.685+0000 7f6416800640 1 --2- 172.21.15.131:0/2961039737 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f64180b5be0 0x7f64180cb310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:23.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:23.686+0000 7f6417200640 1 --2- 172.21.15.131:0/2961039737 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6418166e60 0x7f64180cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:23.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:23.686+0000 7f6415e00640 1 -- 172.21.15.131:0/2961039737 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6418166e60 msgr2=0x7f64180cd6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:23.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:23.686+0000 7f6415e00640 1 --2- 172.21.15.131:0/2961039737 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6418166e60 0x7f64180cd6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:23.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:23.686+0000 7f6415e00640 1 -- 172.21.15.131:0/2961039737 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f64180b5be0 msgr2=0x7f64180cb310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:23.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:23.686+0000 7f6415e00640 1 --2- 172.21.15.131:0/2961039737 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f64180b5be0 0x7f64180cb310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:23.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:23.686+0000 7f6415e00640 1 -- 172.21.15.131:0/2961039737 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6418153c90 con 0x7f64180b9e70 2024-05-26T22:49:23.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:23.686+0000 7f6415e00640 1 --2- 172.21.15.131:0/2961039737 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f64180b9e70 0x7f641815c1e0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f640000c400 tx=0x7f640000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:23.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:23.686+0000 7f640ea00640 1 -- 172.21.15.131:0/2961039737 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6400011790 con 0x7f64180b9e70 2024-05-26T22:49:23.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:23.686+0000 7f640ea00640 1 -- 172.21.15.131:0/2961039737 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6400017070 con 0x7f64180b9e70 2024-05-26T22:49:23.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:23.686+0000 7f641ca00640 1 -- 172.21.15.131:0/2961039737 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6418150660 con 0x7f64180b9e70 2024-05-26T22:49:23.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:23.686+0000 7f640ea00640 1 -- 172.21.15.131:0/2961039737 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6400011bb0 con 0x7f64180b9e70 2024-05-26T22:49:23.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:23.686+0000 7f641ca00640 1 -- 172.21.15.131:0/2961039737 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f641814fee0 con 0x7f64180b9e70 2024-05-26T22:49:23.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:23.688+0000 7f641ca00640 1 -- 172.21.15.131:0/2961039737 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64180694e0 con 0x7f64180b9e70 2024-05-26T22:49:23.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:23.689+0000 7f640ea00640 1 -- 172.21.15.131:0/2961039737 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f640001c460 con 0x7f64180b9e70 2024-05-26T22:49:23.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:23.690+0000 7f640ea00640 1 --2- 172.21.15.131:0/2961039737 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f63f406be30 0x7f63f406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:23.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:23.690+0000 7f640ea00640 1 -- 172.21.15.131:0/2961039737 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f640008aa80 con 0x7f64180b9e70 2024-05-26T22:49:23.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:23.690+0000 7f6416800640 1 --2- 172.21.15.131:0/2961039737 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f63f406be30 0x7f63f406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:23.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:23.691+0000 7f6416800640 1 --2- 172.21.15.131:0/2961039737 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f63f406be30 0x7f63f406e2f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6408000f60 tx=0x7f6408009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:23.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:23.694+0000 7f640ea00640 1 -- 172.21.15.131:0/2961039737 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f640008ce20 con 0x7f64180b9e70 2024-05-26T22:49:23.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:23.965+0000 7f641ca00640 1 -- 172.21.15.131:0/2961039737 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f64180b38a0 con 0x7f64180b9e70 2024-05-26T22:49:23.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:23.967+0000 7f640ea00640 1 -- 172.21.15.131:0/2961039737 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v6) ==== 80+0+6645 (secure 0 0 0) 0x7f64000596e0 con 0x7f64180b9e70 2024-05-26T22:49:23.967 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:49:23.967 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-26T22:49:23.967 INFO:teuthology.orchestra.run.smithi131.stdout: "auth_dump": [ 2024-05-26T22:49:23.967 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:49:23.967 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mds.a", 2024-05-26T22:49:23.967 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD1u1NmTQf1DhAAdvEt7wP0IgKhNHOqjVAllA==", 2024-05-26T22:49:23.967 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:49:23.967 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-05-26T22:49:23.968 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2024-05-26T22:49:23.968 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2024-05-26T22:49:23.968 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-05-26T22:49:23.968 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:49:23.968 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:49:23.968 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:49:23.968 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mds.b", 2024-05-26T22:49:23.968 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD1u1NmHNZ8KxAAPgQ/ojbl+EDfPhqrkg7LPQ==", 2024-05-26T22:49:23.968 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:49:23.968 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-05-26T22:49:23.968 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2024-05-26T22:49:23.968 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2024-05-26T22:49:23.968 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-05-26T22:49:23.969 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:49:23.969 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:49:23.969 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:49:23.969 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mds.c", 2024-05-26T22:49:23.969 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD1u1NmFjCHHRAAqix3Ujfsq2tnSU5+d/pSYA==", 2024-05-26T22:49:23.969 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:49:23.969 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-05-26T22:49:23.969 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2024-05-26T22:49:23.969 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2024-05-26T22:49:23.969 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-05-26T22:49:23.969 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:49:23.969 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:49:23.969 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:49:23.970 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mds.d", 2024-05-26T22:49:23.970 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD1u1NmM4LPORAAhtQtr+Vfpf7vdVwjSW6xtA==", 2024-05-26T22:49:23.970 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:49:23.970 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-05-26T22:49:23.970 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2024-05-26T22:49:23.970 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2024-05-26T22:49:23.970 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-05-26T22:49:23.970 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:49:23.970 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:49:23.970 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:49:23.970 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.0", 2024-05-26T22:49:23.970 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD5u1Nm+w1LNhAAsJLQkiKSh3KkMymlrZkcrw==", 2024-05-26T22:49:23.971 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:49:23.971 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-05-26T22:49:23.971 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-05-26T22:49:23.971 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-05-26T22:49:23.971 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:49:23.971 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:49:23.971 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:49:23.971 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.1", 2024-05-26T22:49:23.971 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD9u1Nm4c8RFBAAcn9HQ+izML9rujJ/2/Q/BQ==", 2024-05-26T22:49:23.971 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:49:23.971 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-05-26T22:49:23.971 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-05-26T22:49:23.971 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-05-26T22:49:23.971 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:49:23.972 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:49:23.972 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:49:23.972 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.2", 2024-05-26T22:49:23.972 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAAvFNmR5wmLRAAYEBAXPBZFq67iARZnwF3iw==", 2024-05-26T22:49:23.972 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:49:23.972 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-05-26T22:49:23.972 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-05-26T22:49:23.972 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-05-26T22:49:23.972 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:49:23.972 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:49:23.972 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:49:23.972 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.3", 2024-05-26T22:49:23.972 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAEvFNmJ4G5CxAAnTbP8DinhmuvGNo1VBVzjQ==", 2024-05-26T22:49:23.972 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:49:23.973 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-05-26T22:49:23.973 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-05-26T22:49:23.973 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-05-26T22:49:23.973 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:49:23.973 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:49:23.973 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:49:23.973 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.4", 2024-05-26T22:49:23.973 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAKvFNmcw5rGhAAvnivP6kt3vcN9UPm/svKqw==", 2024-05-26T22:49:23.973 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:49:23.973 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-05-26T22:49:23.973 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-05-26T22:49:23.973 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-05-26T22:49:23.973 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:49:23.974 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:49:23.974 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:49:23.974 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.5", 2024-05-26T22:49:23.974 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQANvFNmTqlzNhAAGfrSvNNHYKGOBzRXo0TEdw==", 2024-05-26T22:49:23.974 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:49:23.974 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-05-26T22:49:23.974 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-05-26T22:49:23.974 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-05-26T22:49:23.974 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:49:23.974 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:49:23.975 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:49:23.975 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.6", 2024-05-26T22:49:23.975 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQARvFNmpzd6FRAAgi2RYlQPvfHNe0iP4bJrwg==", 2024-05-26T22:49:23.975 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:49:23.975 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-05-26T22:49:23.975 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-05-26T22:49:23.975 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-05-26T22:49:23.975 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:49:23.975 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:49:23.975 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:49:23.975 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.7", 2024-05-26T22:49:23.975 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAUvFNmAkC8LhAAsmkKF4eloxdpCi/mrDtzqA==", 2024-05-26T22:49:23.975 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:49:23.975 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-05-26T22:49:23.976 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-05-26T22:49:23.976 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-05-26T22:49:23.976 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:49:23.976 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:49:23.976 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:49:23.976 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.0", 2024-05-26T22:49:23.976 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD2u1NmSGVWChAAH/wq3iaUs6ZELRhLKhmURQ==", 2024-05-26T22:49:23.976 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:49:23.976 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-05-26T22:49:23.976 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow r", 2024-05-26T22:49:23.976 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow rw", 2024-05-26T22:49:23.976 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow rwx" 2024-05-26T22:49:23.976 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:49:23.976 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:49:23.977 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:49:23.977 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.1", 2024-05-26T22:49:23.977 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD2u1NmnA9CHRAA2XMmZQxysKweHNykiQUEeA==", 2024-05-26T22:49:23.977 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:49:23.977 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-05-26T22:49:23.977 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow r", 2024-05-26T22:49:23.977 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow rw", 2024-05-26T22:49:23.977 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow rwx" 2024-05-26T22:49:23.977 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:49:23.977 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:49:23.977 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:49:23.977 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.2", 2024-05-26T22:49:23.977 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD2u1NmGEI9ExAAvJn1LRwmALkD7zh/AP07vQ==", 2024-05-26T22:49:23.977 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:49:23.978 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-05-26T22:49:23.978 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow r", 2024-05-26T22:49:23.978 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow rw", 2024-05-26T22:49:23.978 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow rwx" 2024-05-26T22:49:23.978 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:49:23.978 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:49:23.978 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:49:23.978 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.3", 2024-05-26T22:49:23.978 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD2u1NmIHOtJhAAJCUm0kGqWu4rWVLsDkW6Eg==", 2024-05-26T22:49:23.978 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:49:23.978 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-05-26T22:49:23.978 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow r", 2024-05-26T22:49:23.979 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow rw", 2024-05-26T22:49:23.979 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow rwx" 2024-05-26T22:49:23.979 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:49:23.979 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:49:23.979 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:49:23.979 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.admin", 2024-05-26T22:49:23.979 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD0u1NmLRGoCBAAhDeVJV14MTAIdUte067jig==", 2024-05-26T22:49:23.979 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:49:23.979 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow *", 2024-05-26T22:49:23.979 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2024-05-26T22:49:23.979 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2024-05-26T22:49:23.979 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-05-26T22:49:23.979 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:49:23.980 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:49:23.980 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:49:23.980 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.bootstrap-mds", 2024-05-26T22:49:23.980 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAdvFNmn85RERAAdyE5RmUZ7JEsRMOs4M2Eig==", 2024-05-26T22:49:23.980 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:49:23.980 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile bootstrap-mds" 2024-05-26T22:49:23.980 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:49:23.980 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:49:23.980 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:49:23.980 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.bootstrap-mgr", 2024-05-26T22:49:23.980 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAdvFNmCu1RERAA8+vzFFdtZo1pIa5eo6UjWw==", 2024-05-26T22:49:23.980 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:49:23.980 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile bootstrap-mgr" 2024-05-26T22:49:23.980 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:49:23.980 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:49:23.981 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:49:23.981 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.bootstrap-osd", 2024-05-26T22:49:23.981 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAdvFNmwwVSERAALHwQZi6kUBGiY5A+EFW/YA==", 2024-05-26T22:49:23.981 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:49:23.981 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile bootstrap-osd" 2024-05-26T22:49:23.981 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:49:23.981 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:49:23.981 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:49:23.981 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.bootstrap-rbd", 2024-05-26T22:49:23.981 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAdvFNmPh5SERAAjqOG9FVM+8rKe6QygDxOgg==", 2024-05-26T22:49:23.981 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:49:23.981 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile bootstrap-rbd" 2024-05-26T22:49:23.981 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:49:23.981 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:49:23.981 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:49:23.982 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-05-26T22:49:23.982 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAdvFNmvphSERAARYS1o/R+39CsJQlT50DlgQ==", 2024-05-26T22:49:23.982 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:49:23.982 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-05-26T22:49:23.982 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:49:23.982 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:49:23.982 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:49:23.982 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.bootstrap-rgw", 2024-05-26T22:49:23.982 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAdvFNm1bJSERAAoJhFXY1NMoPYuJ+35i+S1Q==", 2024-05-26T22:49:23.982 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:49:23.982 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile bootstrap-rgw" 2024-05-26T22:49:23.982 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:49:23.982 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:49:23.983 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:49:23.983 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mgr.x", 2024-05-26T22:49:23.983 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD1u1NmaWhyBBAAlU7Lq6HMjkCPu0kztJIAXQ==", 2024-05-26T22:49:23.983 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:49:23.983 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow *", 2024-05-26T22:49:23.983 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile mgr", 2024-05-26T22:49:23.983 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-05-26T22:49:23.983 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:49:23.983 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:49:23.983 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:49:23.983 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mgr.y", 2024-05-26T22:49:23.983 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD0u1NmX1RhNhAAo5mEOGzUyrdwFlB/XOXKnw==", 2024-05-26T22:49:23.983 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:49:23.983 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow *", 2024-05-26T22:49:23.984 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile mgr", 2024-05-26T22:49:23.984 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-05-26T22:49:23.984 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:49:23.984 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:49:23.984 INFO:teuthology.orchestra.run.smithi131.stdout: ] 2024-05-26T22:49:23.984 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-26T22:49:23.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:23.969+0000 7f641ca00640 1 -- 172.21.15.131:0/2961039737 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f63f406be30 msgr2=0x7f63f406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:23.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:23.969+0000 7f641ca00640 1 --2- 172.21.15.131:0/2961039737 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f63f406be30 0x7f63f406e2f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6408000f60 tx=0x7f6408009040 comp rx=0 tx=0).stop 2024-05-26T22:49:23.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:23.970+0000 7f641ca00640 1 -- 172.21.15.131:0/2961039737 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f64180b9e70 msgr2=0x7f641815c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:23.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:23.970+0000 7f641ca00640 1 --2- 172.21.15.131:0/2961039737 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f64180b9e70 0x7f641815c1e0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f640000c400 tx=0x7f640000c8c0 comp rx=0 tx=0).stop 2024-05-26T22:49:23.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:23.970+0000 7f641ca00640 1 -- 172.21.15.131:0/2961039737 shutdown_connections 2024-05-26T22:49:23.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:23.970+0000 7f641ca00640 1 --2- 172.21.15.131:0/2961039737 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6418166e60 0x7f64180cd6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:23.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:23.970+0000 7f641ca00640 1 --2- 172.21.15.131:0/2961039737 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f63f406be30 0x7f63f406e2f0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:23.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:23.970+0000 7f641ca00640 1 --2- 172.21.15.131:0/2961039737 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f64180b9e70 0x7f641815c1e0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:23.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:23.970+0000 7f641ca00640 1 --2- 172.21.15.131:0/2961039737 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f64180b5be0 0x7f64180cb310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:23.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:23.970+0000 7f641ca00640 1 -- 172.21.15.131:0/2961039737 >> 172.21.15.131:0/2961039737 conn(0x7f6418073070 msgr2=0x7f64180d0b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:23.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:23.970+0000 7f641ca00640 1 -- 172.21.15.131:0/2961039737 shutdown_connections 2024-05-26T22:49:23.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:23.970+0000 7f641ca00640 1 -- 172.21.15.131:0/2961039737 wait complete. 2024-05-26T22:49:23.988 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-05-26T22:49:24.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.189+0000 7f2071000640 1 Processor -- start 2024-05-26T22:49:24.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.189+0000 7f2071000640 1 -- start start 2024-05-26T22:49:24.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.190+0000 7f2071000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f206c06f180 0x7f206c072e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:24.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.191+0000 7f2071000640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f206c072540 0x7f206c161ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:24.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.191+0000 7f2071000640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f206c0af8e0 con 0x7f206c0cfe50 2024-05-26T22:49:24.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.191+0000 7f2071000640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f206c0cfb30 con 0x7f206c072540 2024-05-26T22:49:24.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.191+0000 7f2071000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f206c0cf200 con 0x7f206c06f180 2024-05-26T22:49:24.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.191+0000 7f2069400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f206c072540 0x7f206c161ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:24.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.191+0000 7f2069400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f206c072540 0x7f206c161ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40946/0 (socket says 172.21.15.131:40946) 2024-05-26T22:49:24.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.191+0000 7f2069400640 1 -- 172.21.15.131:0/2599478849 learned_addr learned my addr 172.21.15.131:0/2599478849 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:24.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.191+0000 7f2069e00640 1 --2- 172.21.15.131:0/2599478849 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f206c06f180 0x7f206c072e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:24.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.191+0000 7f2069400640 1 -- 172.21.15.131:0/2599478849 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f206c06f180 msgr2=0x7f206c072e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:24.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.191+0000 7f2069400640 1 --2- 172.21.15.131:0/2599478849 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f206c06f180 0x7f206c072e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:24.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.191+0000 7f2069400640 1 -- 172.21.15.131:0/2599478849 >> v1:172.21.15.187:6789/0 conn(0x7f206c0cfe50 legacy=0x7f206c0715a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:24.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.192+0000 7f2069400640 1 -- 172.21.15.131:0/2599478849 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f206c0cf4e0 con 0x7f206c072540 2024-05-26T22:49:24.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.193+0000 7f2069400640 1 --2- 172.21.15.131:0/2599478849 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f206c072540 0x7f206c161ff0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f2058009870 tx=0x7f205802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0bd691bc38d1cb3 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:24.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.193+0000 7f2068a00640 1 -- 172.21.15.131:0/2599478849 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2058037070 con 0x7f206c072540 2024-05-26T22:49:24.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.193+0000 7f2068a00640 1 -- 172.21.15.131:0/2599478849 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2058032440 con 0x7f206c072540 2024-05-26T22:49:24.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.193+0000 7f2068a00640 1 -- 172.21.15.131:0/2599478849 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2058032810 con 0x7f206c072540 2024-05-26T22:49:24.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.193+0000 7f2071000640 1 -- 172.21.15.131:0/2599478849 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f206c072540 msgr2=0x7f206c161ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:24.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.193+0000 7f2071000640 1 --2- 172.21.15.131:0/2599478849 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f206c072540 0x7f206c161ff0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f2058009870 tx=0x7f205802a210 comp rx=0 tx=0).stop 2024-05-26T22:49:24.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.194+0000 7f2071000640 1 -- 172.21.15.131:0/2599478849 shutdown_connections 2024-05-26T22:49:24.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.194+0000 7f2071000640 1 --2- 172.21.15.131:0/2599478849 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f206c072540 0x7f206c161ff0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:24.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.194+0000 7f2071000640 1 --2- 172.21.15.131:0/2599478849 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f206c06f180 0x7f206c072e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:24.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.194+0000 7f2071000640 1 -- 172.21.15.131:0/2599478849 >> 172.21.15.131:0/2599478849 conn(0x7f206c06bc90 msgr2=0x7f206c0b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:24.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.194+0000 7f2071000640 1 -- 172.21.15.131:0/2599478849 shutdown_connections 2024-05-26T22:49:24.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.194+0000 7f2071000640 1 -- 172.21.15.131:0/2599478849 wait complete. 2024-05-26T22:49:24.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.196+0000 7f2071000640 1 Processor -- start 2024-05-26T22:49:24.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.196+0000 7f2071000640 1 -- start start 2024-05-26T22:49:24.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.197+0000 7f2071000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f206c06f180 0x7f206c15bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:24.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.197+0000 7f2071000640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f206c072540 0x7f206c0ca050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:24.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.197+0000 7f2071000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f206c0cfe50 0x7f206c0afab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:24.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.197+0000 7f2071000640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f206c06f6e0 con 0x7f206c0cfe50 2024-05-26T22:49:24.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.197+0000 7f2071000640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f206c06f560 con 0x7f206c072540 2024-05-26T22:49:24.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.197+0000 7f2071000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f206c0d02e0 con 0x7f206c06f180 2024-05-26T22:49:24.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.198+0000 7f206a800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f206c0cfe50 0x7f206c0afab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:24.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.198+0000 7f206a800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f206c0cfe50 0x7f206c0afab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40950/0 (socket says 172.21.15.131:40950) 2024-05-26T22:49:24.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.198+0000 7f206a800640 1 -- 172.21.15.131:0/1221530768 learned_addr learned my addr 172.21.15.131:0/1221530768 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:24.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.198+0000 7f2069e00640 1 --2- 172.21.15.131:0/1221530768 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f206c06f180 0x7f206c15bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:24.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.198+0000 7f2069400640 1 --2- 172.21.15.131:0/1221530768 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f206c072540 0x7f206c0ca050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:24.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.198+0000 7f206a800640 1 -- 172.21.15.131:0/1221530768 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f206c06f180 msgr2=0x7f206c15bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:24.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.198+0000 7f206a800640 1 --2- 172.21.15.131:0/1221530768 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f206c06f180 0x7f206c15bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:24.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.198+0000 7f206a800640 1 -- 172.21.15.131:0/1221530768 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f206c072540 msgr2=0x7f206c0ca050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:24.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.198+0000 7f206a800640 1 --2- 172.21.15.131:0/1221530768 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f206c072540 0x7f206c0ca050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:24.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.198+0000 7f206a800640 1 -- 172.21.15.131:0/1221530768 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f206c0ce9e0 con 0x7f206c0cfe50 2024-05-26T22:49:24.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.198+0000 7f2069e00640 1 --2- 172.21.15.131:0/1221530768 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f206c06f180 0x7f206c15bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:49:24.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.198+0000 7f206a800640 1 --2- 172.21.15.131:0/1221530768 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f206c0cfe50 0x7f206c0afab0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f206400b970 tx=0x7f206400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:24.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.199+0000 7f2062000640 1 -- 172.21.15.131:0/1221530768 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2064019070 con 0x7f206c0cfe50 2024-05-26T22:49:24.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.199+0000 7f2062000640 1 -- 172.21.15.131:0/1221530768 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f206400cb40 con 0x7f206c0cfe50 2024-05-26T22:49:24.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.199+0000 7f2071000640 1 -- 172.21.15.131:0/1221530768 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f206c0c9290 con 0x7f206c0cfe50 2024-05-26T22:49:24.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.199+0000 7f2062000640 1 -- 172.21.15.131:0/1221530768 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2064014440 con 0x7f206c0cfe50 2024-05-26T22:49:24.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.199+0000 7f2071000640 1 -- 172.21.15.131:0/1221530768 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f206c0c9850 con 0x7f206c0cfe50 2024-05-26T22:49:24.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.200+0000 7f2071000640 1 -- 172.21.15.131:0/1221530768 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f206c074e90 con 0x7f206c0cfe50 2024-05-26T22:49:24.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.201+0000 7f2062000640 1 -- 172.21.15.131:0/1221530768 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f206401c070 con 0x7f206c0cfe50 2024-05-26T22:49:24.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.202+0000 7f2062000640 1 --2- 172.21.15.131:0/1221530768 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f204c06be60 0x7f204c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:24.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.202+0000 7f2062000640 1 -- 172.21.15.131:0/1221530768 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f206408a790 con 0x7f206c0cfe50 2024-05-26T22:49:24.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.202+0000 7f2069e00640 1 --2- 172.21.15.131:0/1221530768 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f204c06be60 0x7f204c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:24.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.203+0000 7f2069e00640 1 --2- 172.21.15.131:0/1221530768 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f204c06be60 0x7f204c06e320 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f205c006fd0 tx=0x7f205c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:24.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.206+0000 7f2062000640 1 -- 172.21.15.131:0/1221530768 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f2064059480 con 0x7f206c0cfe50 2024-05-26T22:49:24.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.459+0000 7f2071000640 1 -- 172.21.15.131:0/1221530768 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7f206c0aef00 con 0x7f206c0cfe50 2024-05-26T22:49:24.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.462+0000 7f2062000640 1 -- 172.21.15.131:0/1221530768 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v7) ==== 90+0+0 (secure 0 0 0) 0x7f2064058d80 con 0x7f206c0cfe50 2024-05-26T22:49:24.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.465+0000 7f2071000640 1 -- 172.21.15.131:0/1221530768 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f204c06be60 msgr2=0x7f204c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:24.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.465+0000 7f2071000640 1 --2- 172.21.15.131:0/1221530768 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f204c06be60 0x7f204c06e320 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f205c006fd0 tx=0x7f205c008040 comp rx=0 tx=0).stop 2024-05-26T22:49:24.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.465+0000 7f2071000640 1 -- 172.21.15.131:0/1221530768 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f206c0cfe50 msgr2=0x7f206c0afab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:24.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.465+0000 7f2071000640 1 --2- 172.21.15.131:0/1221530768 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f206c0cfe50 0x7f206c0afab0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f206400b970 tx=0x7f206400be30 comp rx=0 tx=0).stop 2024-05-26T22:49:24.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.465+0000 7f2071000640 1 -- 172.21.15.131:0/1221530768 shutdown_connections 2024-05-26T22:49:24.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.465+0000 7f2071000640 1 --2- 172.21.15.131:0/1221530768 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f204c06be60 0x7f204c06e320 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:24.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.465+0000 7f2071000640 1 --2- 172.21.15.131:0/1221530768 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f206c0cfe50 0x7f206c0afab0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:24.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.465+0000 7f2071000640 1 --2- 172.21.15.131:0/1221530768 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f206c072540 0x7f206c0ca050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:24.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.465+0000 7f2071000640 1 --2- 172.21.15.131:0/1221530768 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f206c06f180 0x7f206c15bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:24.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.465+0000 7f2071000640 1 -- 172.21.15.131:0/1221530768 >> 172.21.15.131:0/1221530768 conn(0x7f206c06bc90 msgr2=0x7f206c161390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:24.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.465+0000 7f2071000640 1 -- 172.21.15.131:0/1221530768 shutdown_connections 2024-05-26T22:49:24.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.465+0000 7f2071000640 1 -- 172.21.15.131:0/1221530768 wait complete. 2024-05-26T22:49:24.483 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-05-26T22:49:24.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.677+0000 7f9d15400640 1 Processor -- start 2024-05-26T22:49:24.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.677+0000 7f9d15400640 1 -- start start 2024-05-26T22:49:24.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.679+0000 7f9d15400640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f9d1015e320 0x7f9d100ae250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:24.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.679+0000 7f9d15400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d100b2f60 0x7f9d100cfe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:24.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.679+0000 7f9d15400640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d100b2440 con 0x7f9d100d0380 2024-05-26T22:49:24.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.679+0000 7f9d15400640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f9d100725e0 con 0x7f9d1015e320 2024-05-26T22:49:24.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.679+0000 7f9d15400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9d10161fe0 con 0x7f9d100b2f60 2024-05-26T22:49:24.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.680+0000 7f9d07400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d100b2f60 0x7f9d100cfe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:24.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.680+0000 7f9d14a00640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f9d100d0380 0x7f9d1006d250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.131:54804/0 (socket says 172.21.15.131:54804) 2024-05-26T22:49:24.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.680+0000 7f9d14a00640 1 -- 172.21.15.131:0/3067084051 learned_addr learned my addr 172.21.15.131:0/3067084051 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:49:24.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.680+0000 7f9d0fe00640 1 --2- 172.21.15.131:0/3067084051 >> v2:172.21.15.187:3300/0 conn(0x7f9d1015e320 0x7f9d100ae250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:24.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.681+0000 7f9d0f400640 1 -- 172.21.15.131:0/3067084051 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1251048383 0 0) 0x7f9d100b2440 con 0x7f9d100d0380 2024-05-26T22:49:24.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.681+0000 7f9d0f400640 1 -- 172.21.15.131:0/3067084051 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9cf4003520 con 0x7f9d100d0380 2024-05-26T22:49:24.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.681+0000 7f9d0fe00640 1 -- 172.21.15.131:0/3067084051 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d100b2f60 msgr2=0x7f9d100cfe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:24.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.681+0000 7f9d0fe00640 1 --2- 172.21.15.131:0/3067084051 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d100b2f60 0x7f9d100cfe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:24.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.681+0000 7f9d0fe00640 1 -- 172.21.15.131:0/3067084051 >> v1:172.21.15.187:6789/0 conn(0x7f9d100d0380 legacy=0x7f9d1006d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:24.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.681+0000 7f9d0fe00640 1 -- 172.21.15.131:0/3067084051 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d10162230 con 0x7f9d1015e320 2024-05-26T22:49:24.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.683+0000 7f9d0fe00640 1 --2- 172.21.15.131:0/3067084051 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d1015e320 0x7f9d100ae250 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9d000097a0 tx=0x7f9d0002a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ab388257691de90 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:24.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.684+0000 7f9d0f400640 1 -- 172.21.15.131:0/3067084051 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d00037070 con 0x7f9d1015e320 2024-05-26T22:49:24.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.684+0000 7f9d0f400640 1 -- 172.21.15.131:0/3067084051 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d00032440 con 0x7f9d1015e320 2024-05-26T22:49:24.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.684+0000 7f9d0f400640 1 -- 172.21.15.131:0/3067084051 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d00032810 con 0x7f9d1015e320 2024-05-26T22:49:24.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.684+0000 7f9d15400640 1 -- 172.21.15.131:0/3067084051 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d1015e320 msgr2=0x7f9d100ae250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:24.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.684+0000 7f9d15400640 1 --2- 172.21.15.131:0/3067084051 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d1015e320 0x7f9d100ae250 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9d000097a0 tx=0x7f9d0002a210 comp rx=0 tx=0).stop 2024-05-26T22:49:24.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.684+0000 7f9d15400640 1 -- 172.21.15.131:0/3067084051 shutdown_connections 2024-05-26T22:49:24.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.685+0000 7f9d15400640 1 --2- 172.21.15.131:0/3067084051 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d100b2f60 0x7f9d100cfe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:24.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.685+0000 7f9d15400640 1 --2- 172.21.15.131:0/3067084051 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d1015e320 0x7f9d100ae250 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:24.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.685+0000 7f9d15400640 1 -- 172.21.15.131:0/3067084051 >> 172.21.15.131:0/3067084051 conn(0x7f9d100b5660 msgr2=0x7f9d100b5a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:24.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.685+0000 7f9d15400640 1 -- 172.21.15.131:0/3067084051 shutdown_connections 2024-05-26T22:49:24.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.685+0000 7f9d15400640 1 -- 172.21.15.131:0/3067084051 wait complete. 2024-05-26T22:49:24.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.687+0000 7f9d15400640 1 Processor -- start 2024-05-26T22:49:24.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.687+0000 7f9d15400640 1 -- start start 2024-05-26T22:49:24.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.688+0000 7f9d15400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d100b2f60 0x7f9d10148370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:24.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.688+0000 7f9d15400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d100d0380 0x7f9d10145700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:24.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.688+0000 7f9d15400640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d1015e320 0x7f9d10146fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:24.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.688+0000 7f9d15400640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f9d10162ad0 con 0x7f9d100b2f60 2024-05-26T22:49:24.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.688+0000 7f9d15400640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9d101627f0 con 0x7f9d1015e320 2024-05-26T22:49:24.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.688+0000 7f9d15400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9d10162e40 con 0x7f9d100d0380 2024-05-26T22:49:24.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.688+0000 7f9d0fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d100b2f60 0x7f9d10148370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:24.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.688+0000 7f9d0fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d100b2f60 0x7f9d10148370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41648/0 (socket says 172.21.15.131:41648) 2024-05-26T22:49:24.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.688+0000 7f9d0fe00640 1 -- 172.21.15.131:0/522792394 learned_addr learned my addr 172.21.15.131:0/522792394 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:24.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.689+0000 7f9d07400640 1 --2- 172.21.15.131:0/522792394 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d100d0380 0x7f9d10145700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:24.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.689+0000 7f9d14a00640 1 --2- 172.21.15.131:0/522792394 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d1015e320 0x7f9d10146fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:24.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.689+0000 7f9d0fe00640 1 -- 172.21.15.131:0/522792394 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d100d0380 msgr2=0x7f9d10145700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:24.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.689+0000 7f9d0fe00640 1 --2- 172.21.15.131:0/522792394 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d100d0380 0x7f9d10145700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:24.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.689+0000 7f9d0fe00640 1 -- 172.21.15.131:0/522792394 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d1015e320 msgr2=0x7f9d10146fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:24.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.689+0000 7f9d0fe00640 1 --2- 172.21.15.131:0/522792394 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d1015e320 0x7f9d10146fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:24.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.689+0000 7f9d0fe00640 1 -- 172.21.15.131:0/522792394 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d10145c40 con 0x7f9d100b2f60 2024-05-26T22:49:24.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.689+0000 7f9d14a00640 1 --2- 172.21.15.131:0/522792394 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d1015e320 0x7f9d10146fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:49:24.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.689+0000 7f9d0fe00640 1 --2- 172.21.15.131:0/522792394 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d100b2f60 0x7f9d10148370 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f9d000097a0 tx=0x7f9d000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:24.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.690+0000 7f9d0cc00640 1 -- 172.21.15.131:0/522792394 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d00041070 con 0x7f9d100b2f60 2024-05-26T22:49:24.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.690+0000 7f9d0cc00640 1 -- 172.21.15.131:0/522792394 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d0003c870 con 0x7f9d100b2f60 2024-05-26T22:49:24.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.690+0000 7f9d0cc00640 1 -- 172.21.15.131:0/522792394 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d00037040 con 0x7f9d100b2f60 2024-05-26T22:49:24.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.690+0000 7f9d15400640 1 -- 172.21.15.131:0/522792394 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d101474f0 con 0x7f9d100b2f60 2024-05-26T22:49:24.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.690+0000 7f9d15400640 1 -- 172.21.15.131:0/522792394 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d101461f0 con 0x7f9d100b2f60 2024-05-26T22:49:24.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.691+0000 7f9d15400640 1 -- 172.21.15.131:0/522792394 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d100694e0 con 0x7f9d100b2f60 2024-05-26T22:49:24.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.692+0000 7f9d0cc00640 1 -- 172.21.15.131:0/522792394 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9d00044070 con 0x7f9d100b2f60 2024-05-26T22:49:24.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.693+0000 7f9d0cc00640 1 --2- 172.21.15.131:0/522792394 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9cec06be30 0x7f9cec06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:24.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.693+0000 7f9d0cc00640 1 -- 172.21.15.131:0/522792394 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f9d000ab720 con 0x7f9d100b2f60 2024-05-26T22:49:24.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.693+0000 7f9d07400640 1 --2- 172.21.15.131:0/522792394 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9cec06be30 0x7f9cec06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:24.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.694+0000 7f9d07400640 1 --2- 172.21.15.131:0/522792394 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9cec06be30 0x7f9cec06e2f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f9d101f7780 tx=0x7f9cf8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:24.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.697+0000 7f9d0cc00640 1 -- 172.21.15.131:0/522792394 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f9d0007a540 con 0x7f9d100b2f60 2024-05-26T22:49:24.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.955+0000 7f9d15400640 1 -- 172.21.15.131:0/522792394 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7f9d10159020 con 0x7f9d100b2f60 2024-05-26T22:49:24.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.960+0000 7f9d0cc00640 1 -- 172.21.15.131:0/522792394 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v8) ==== 90+0+0 (secure 0 0 0) 0x7f9d00079e40 con 0x7f9d100b2f60 2024-05-26T22:49:24.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.963+0000 7f9d15400640 1 -- 172.21.15.131:0/522792394 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9cec06be30 msgr2=0x7f9cec06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:24.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.963+0000 7f9d15400640 1 --2- 172.21.15.131:0/522792394 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9cec06be30 0x7f9cec06e2f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f9d101f7780 tx=0x7f9cf8008040 comp rx=0 tx=0).stop 2024-05-26T22:49:24.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.963+0000 7f9d15400640 1 -- 172.21.15.131:0/522792394 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d100b2f60 msgr2=0x7f9d10148370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:24.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.963+0000 7f9d15400640 1 --2- 172.21.15.131:0/522792394 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d100b2f60 0x7f9d10148370 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f9d000097a0 tx=0x7f9d000046c0 comp rx=0 tx=0).stop 2024-05-26T22:49:24.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.963+0000 7f9d15400640 1 -- 172.21.15.131:0/522792394 shutdown_connections 2024-05-26T22:49:24.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.963+0000 7f9d15400640 1 --2- 172.21.15.131:0/522792394 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d100d0380 0x7f9d10145700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:24.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.963+0000 7f9d15400640 1 --2- 172.21.15.131:0/522792394 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9cec06be30 0x7f9cec06e2f0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:24.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.963+0000 7f9d15400640 1 --2- 172.21.15.131:0/522792394 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d1015e320 0x7f9d10146fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:24.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.963+0000 7f9d15400640 1 --2- 172.21.15.131:0/522792394 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d100b2f60 0x7f9d10148370 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:24.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.963+0000 7f9d15400640 1 -- 172.21.15.131:0/522792394 >> 172.21.15.131:0/522792394 conn(0x7f9d100b5660 msgr2=0x7f9d100b4630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:24.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.963+0000 7f9d15400640 1 -- 172.21.15.131:0/522792394 shutdown_connections 2024-05-26T22:49:24.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:24.963+0000 7f9d15400640 1 -- 172.21.15.131:0/522792394 wait complete. 2024-05-26T22:49:24.981 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-05-26T22:49:25.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.166+0000 7fc772200640 1 Processor -- start 2024-05-26T22:49:25.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.167+0000 7fc772200640 1 -- start start 2024-05-26T22:49:25.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.168+0000 7fc772200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fc76c0b5450 0x7fc76c0b43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:25.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.168+0000 7fc772200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc76c0d0460 0x7fc76c06d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:25.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.168+0000 7fc772200640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc76c0b4900 con 0x7fc76c06f420 2024-05-26T22:49:25.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.168+0000 7fc772200640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fc76c160d90 con 0x7fc76c0b5450 2024-05-26T22:49:25.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.168+0000 7fc772200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc76c160180 con 0x7fc76c0d0460 2024-05-26T22:49:25.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.169+0000 7fc76b400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fc76c0b5450 0x7fc76c0b43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:25.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.169+0000 7fc76b400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fc76c0b5450 0x7fc76c0b43c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41664/0 (socket says 172.21.15.131:41664) 2024-05-26T22:49:25.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.169+0000 7fc76b400640 1 -- 172.21.15.131:0/1822033503 learned_addr learned my addr 172.21.15.131:0/1822033503 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:25.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.169+0000 7fc76be00640 1 --2- 172.21.15.131:0/1822033503 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc76c0d0460 0x7fc76c06d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:25.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.169+0000 7fc76b400640 1 -- 172.21.15.131:0/1822033503 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc76c0d0460 msgr2=0x7fc76c06d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:25.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.169+0000 7fc76b400640 1 --2- 172.21.15.131:0/1822033503 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc76c0d0460 0x7fc76c06d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:25.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.169+0000 7fc76b400640 1 -- 172.21.15.131:0/1822033503 >> v1:172.21.15.187:6789/0 conn(0x7fc76c06f420 legacy=0x7fc76c0cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:25.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.169+0000 7fc76b400640 1 -- 172.21.15.131:0/1822033503 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc76c160460 con 0x7fc76c0b5450 2024-05-26T22:49:25.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.170+0000 7fc76b400640 1 --2- 172.21.15.131:0/1822033503 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc76c0b5450 0x7fc76c0b43c0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fc75c009e50 tx=0x7fc75c02a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c34f5e2412466bf server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:25.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.171+0000 7fc76aa00640 1 -- 172.21.15.131:0/1822033503 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc75c036070 con 0x7fc76c0b5450 2024-05-26T22:49:25.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.171+0000 7fc76aa00640 1 -- 172.21.15.131:0/1822033503 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc75c031600 con 0x7fc76c0b5450 2024-05-26T22:49:25.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.171+0000 7fc76aa00640 1 -- 172.21.15.131:0/1822033503 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc75c0319d0 con 0x7fc76c0b5450 2024-05-26T22:49:25.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.171+0000 7fc772200640 1 -- 172.21.15.131:0/1822033503 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc76c0b5450 msgr2=0x7fc76c0b43c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:25.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.171+0000 7fc772200640 1 --2- 172.21.15.131:0/1822033503 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc76c0b5450 0x7fc76c0b43c0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fc75c009e50 tx=0x7fc75c02a6c0 comp rx=0 tx=0).stop 2024-05-26T22:49:25.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.171+0000 7fc772200640 1 -- 172.21.15.131:0/1822033503 shutdown_connections 2024-05-26T22:49:25.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.171+0000 7fc772200640 1 --2- 172.21.15.131:0/1822033503 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc76c0d0460 0x7fc76c06d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:25.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.171+0000 7fc772200640 1 --2- 172.21.15.131:0/1822033503 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc76c0b5450 0x7fc76c0b43c0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:25.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.171+0000 7fc772200640 1 -- 172.21.15.131:0/1822033503 >> 172.21.15.131:0/1822033503 conn(0x7fc76c06bc90 msgr2=0x7fc76c0b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:25.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.171+0000 7fc772200640 1 -- 172.21.15.131:0/1822033503 shutdown_connections 2024-05-26T22:49:25.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.171+0000 7fc772200640 1 -- 172.21.15.131:0/1822033503 wait complete. 2024-05-26T22:49:25.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.173+0000 7fc772200640 1 Processor -- start 2024-05-26T22:49:25.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.173+0000 7fc772200640 1 -- start start 2024-05-26T22:49:25.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.174+0000 7fc772200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc76c06f420 0x7fc76c15f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:25.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.174+0000 7fc772200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc76c0b5450 0x7fc76c14b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:25.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.174+0000 7fc772200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc76c0d0460 0x7fc76c157a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:25.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.174+0000 7fc772200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc76c0b5830 con 0x7fc76c0d0460 2024-05-26T22:49:25.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.174+0000 7fc772200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc76c160a20 con 0x7fc76c0b5450 2024-05-26T22:49:25.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.174+0000 7fc772200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc76c15fde0 con 0x7fc76c06f420 2024-05-26T22:49:25.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.174+0000 7fc76be00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc76c0d0460 0x7fc76c157a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:25.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.174+0000 7fc76be00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc76c0d0460 0x7fc76c157a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41672/0 (socket says 172.21.15.131:41672) 2024-05-26T22:49:25.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.174+0000 7fc76be00640 1 -- 172.21.15.131:0/3282228655 learned_addr learned my addr 172.21.15.131:0/3282228655 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:25.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.175+0000 7fc76be00640 1 -- 172.21.15.131:0/3282228655 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc76c06f420 msgr2=0x7fc76c15f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:25.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.175+0000 7fc76be00640 1 --2- 172.21.15.131:0/3282228655 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc76c06f420 0x7fc76c15f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:25.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.175+0000 7fc76be00640 1 -- 172.21.15.131:0/3282228655 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc76c0b5450 msgr2=0x7fc76c14b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:25.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.175+0000 7fc76be00640 1 --2- 172.21.15.131:0/3282228655 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc76c0b5450 0x7fc76c14b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:25.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.175+0000 7fc76be00640 1 -- 172.21.15.131:0/3282228655 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc76c1569a0 con 0x7fc76c0d0460 2024-05-26T22:49:25.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.175+0000 7fc76be00640 1 --2- 172.21.15.131:0/3282228655 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc76c0d0460 0x7fc76c157a40 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fc76400d800 tx=0x7fc76400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:25.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.175+0000 7fc763e00640 1 -- 172.21.15.131:0/3282228655 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc76400b6c0 con 0x7fc76c0d0460 2024-05-26T22:49:25.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.175+0000 7fc763e00640 1 -- 172.21.15.131:0/3282228655 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc764019070 con 0x7fc76c0d0460 2024-05-26T22:49:25.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.175+0000 7fc772200640 1 -- 172.21.15.131:0/3282228655 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc76c156c80 con 0x7fc76c0d0460 2024-05-26T22:49:25.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.175+0000 7fc763e00640 1 -- 172.21.15.131:0/3282228655 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc76400baa0 con 0x7fc76c0d0460 2024-05-26T22:49:25.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.175+0000 7fc772200640 1 -- 172.21.15.131:0/3282228655 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc76c157240 con 0x7fc76c0d0460 2024-05-26T22:49:25.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.176+0000 7fc772200640 1 -- 172.21.15.131:0/3282228655 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc76c0695c0 con 0x7fc76c0d0460 2024-05-26T22:49:25.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.177+0000 7fc763e00640 1 -- 172.21.15.131:0/3282228655 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc76400bc40 con 0x7fc76c0d0460 2024-05-26T22:49:25.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.178+0000 7fc763e00640 1 --2- 172.21.15.131:0/3282228655 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc74806be60 0x7fc74806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:25.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.178+0000 7fc763e00640 1 -- 172.21.15.131:0/3282228655 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7fc76408afd0 con 0x7fc76c0d0460 2024-05-26T22:49:25.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.178+0000 7fc76b400640 1 --2- 172.21.15.131:0/3282228655 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc74806be60 0x7fc74806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:25.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.179+0000 7fc76b400640 1 --2- 172.21.15.131:0/3282228655 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc74806be60 0x7fc74806e320 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc75c033000 tx=0x7fc75c02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:25.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.182+0000 7fc763e00640 1 -- 172.21.15.131:0/3282228655 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fc764059c30 con 0x7fc76c0d0460 2024-05-26T22:49:25.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.456+0000 7fc772200640 1 -- 172.21.15.131:0/3282228655 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7fc76c071640 con 0x7fc76c0d0460 2024-05-26T22:49:25.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.466+0000 7fc763e00640 1 -- 172.21.15.131:0/3282228655 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v9) ==== 90+0+0 (secure 0 0 0) 0x7fc764059530 con 0x7fc76c0d0460 2024-05-26T22:49:25.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.469+0000 7fc772200640 1 -- 172.21.15.131:0/3282228655 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc74806be60 msgr2=0x7fc74806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:25.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.469+0000 7fc772200640 1 --2- 172.21.15.131:0/3282228655 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc74806be60 0x7fc74806e320 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc75c033000 tx=0x7fc75c02e000 comp rx=0 tx=0).stop 2024-05-26T22:49:25.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.470+0000 7fc772200640 1 -- 172.21.15.131:0/3282228655 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc76c0d0460 msgr2=0x7fc76c157a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:25.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.470+0000 7fc772200640 1 --2- 172.21.15.131:0/3282228655 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc76c0d0460 0x7fc76c157a40 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fc76400d800 tx=0x7fc76400dcc0 comp rx=0 tx=0).stop 2024-05-26T22:49:25.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.470+0000 7fc772200640 1 -- 172.21.15.131:0/3282228655 shutdown_connections 2024-05-26T22:49:25.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.470+0000 7fc772200640 1 --2- 172.21.15.131:0/3282228655 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc74806be60 0x7fc74806e320 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:25.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.470+0000 7fc772200640 1 --2- 172.21.15.131:0/3282228655 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc76c0d0460 0x7fc76c157a40 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:25.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.470+0000 7fc772200640 1 --2- 172.21.15.131:0/3282228655 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc76c0b5450 0x7fc76c14b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:25.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.470+0000 7fc772200640 1 --2- 172.21.15.131:0/3282228655 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc76c06f420 0x7fc76c15f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:25.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.470+0000 7fc772200640 1 -- 172.21.15.131:0/3282228655 >> 172.21.15.131:0/3282228655 conn(0x7fc76c06bc90 msgr2=0x7fc76c0ce360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:25.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.470+0000 7fc772200640 1 -- 172.21.15.131:0/3282228655 shutdown_connections 2024-05-26T22:49:25.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.470+0000 7fc772200640 1 -- 172.21.15.131:0/3282228655 wait complete. 2024-05-26T22:49:25.489 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-05-26T22:49:25.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.675+0000 7f858aa00640 1 Processor -- start 2024-05-26T22:49:25.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.675+0000 7f858aa00640 1 -- start start 2024-05-26T22:49:25.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.676+0000 7f858aa00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f858415e320 0x7f85840ae250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:25.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.677+0000 7f858aa00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f85840b2f60 0x7f85840cfe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:25.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.677+0000 7f858aa00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85840b2440 con 0x7f85840d0380 2024-05-26T22:49:25.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.677+0000 7f858aa00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f85840725e0 con 0x7f85840b2f60 2024-05-26T22:49:25.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.677+0000 7f858aa00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f8584161fe0 con 0x7f858415e320 2024-05-26T22:49:25.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.677+0000 7f8588c00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f85840b2f60 0x7f85840cfe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:25.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.677+0000 7f8588c00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f85840b2f60 0x7f85840cfe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41684/0 (socket says 172.21.15.131:41684) 2024-05-26T22:49:25.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.677+0000 7f8588c00640 1 -- 172.21.15.131:0/526166425 learned_addr learned my addr 172.21.15.131:0/526166425 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:25.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.677+0000 7f858a000640 1 --1- 172.21.15.131:0/526166425 >> v1:172.21.15.131:6789/0 conn(0x7f85840d0380 0x7f858406d250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:37966/0 (socket says 172.21.15.131:37966) 2024-05-26T22:49:25.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.677+0000 7f8589600640 1 --2- 172.21.15.131:0/526166425 >> v2:172.21.15.187:3300/0 conn(0x7f858415e320 0x7f85840ae250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:25.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.677+0000 7f8588c00640 1 -- 172.21.15.131:0/526166425 >> v2:172.21.15.187:3300/0 conn(0x7f858415e320 msgr2=0x7f85840ae250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:25.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.677+0000 7f8588c00640 1 --2- 172.21.15.131:0/526166425 >> v2:172.21.15.187:3300/0 conn(0x7f858415e320 0x7f85840ae250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:25.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.677+0000 7f8588c00640 1 -- 172.21.15.131:0/526166425 >> v1:172.21.15.131:6789/0 conn(0x7f85840d0380 legacy=0x7f858406d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:25.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.677+0000 7f8588c00640 1 -- 172.21.15.131:0/526166425 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8584162230 con 0x7f85840b2f60 2024-05-26T22:49:25.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.677+0000 7f8583e00640 1 -- 172.21.15.131:0/526166425 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 86197031 0 0) 0x7f85840b2440 con 0x7f85840d0380 2024-05-26T22:49:25.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.678+0000 7f8588c00640 1 --2- 172.21.15.131:0/526166425 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f85840b2f60 0x7f85840cfe40 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f85840b2a10 tx=0x7f857402a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc957ca6c202f007 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:25.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.679+0000 7f8583e00640 1 -- 172.21.15.131:0/526166425 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8574037070 con 0x7f85840b2f60 2024-05-26T22:49:25.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.679+0000 7f8583e00640 1 -- 172.21.15.131:0/526166425 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f857402ae50 con 0x7f85840b2f60 2024-05-26T22:49:25.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.679+0000 7f8583e00640 1 -- 172.21.15.131:0/526166425 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8574032860 con 0x7f85840b2f60 2024-05-26T22:49:25.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.679+0000 7f858aa00640 1 -- 172.21.15.131:0/526166425 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f85840b2f60 msgr2=0x7f85840cfe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:25.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.679+0000 7f858aa00640 1 --2- 172.21.15.131:0/526166425 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f85840b2f60 0x7f85840cfe40 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f85840b2a10 tx=0x7f857402a300 comp rx=0 tx=0).stop 2024-05-26T22:49:25.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.679+0000 7f858aa00640 1 -- 172.21.15.131:0/526166425 shutdown_connections 2024-05-26T22:49:25.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.679+0000 7f858aa00640 1 --2- 172.21.15.131:0/526166425 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f85840b2f60 0x7f85840cfe40 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:25.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.679+0000 7f858aa00640 1 --2- 172.21.15.131:0/526166425 >> v2:172.21.15.187:3300/0 conn(0x7f858415e320 0x7f85840ae250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:25.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.679+0000 7f858aa00640 1 -- 172.21.15.131:0/526166425 >> 172.21.15.131:0/526166425 conn(0x7f85840b5660 msgr2=0x7f85840b5a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:25.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.679+0000 7f858aa00640 1 -- 172.21.15.131:0/526166425 shutdown_connections 2024-05-26T22:49:25.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.679+0000 7f858aa00640 1 -- 172.21.15.131:0/526166425 wait complete. 2024-05-26T22:49:25.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.681+0000 7f858aa00640 1 Processor -- start 2024-05-26T22:49:25.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.681+0000 7f858aa00640 1 -- start start 2024-05-26T22:49:25.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.682+0000 7f858aa00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f85840b2f60 0x7f85841f7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:25.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.682+0000 7f858aa00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f85840d0380 0x7f8584160ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:25.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.682+0000 7f858aa00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f858415e320 0x7f858415cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:25.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.682+0000 7f858aa00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8584162ad0 con 0x7f85840d0380 2024-05-26T22:49:25.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.682+0000 7f858aa00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f85841627f0 con 0x7f85840b2f60 2024-05-26T22:49:25.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.682+0000 7f858aa00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f8584162e40 con 0x7f858415e320 2024-05-26T22:49:25.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.682+0000 7f8588c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f85840d0380 0x7f8584160ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:25.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.682+0000 7f8588c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f85840d0380 0x7f8584160ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41692/0 (socket says 172.21.15.131:41692) 2024-05-26T22:49:25.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.682+0000 7f8588c00640 1 -- 172.21.15.131:0/1195508131 learned_addr learned my addr 172.21.15.131:0/1195508131 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:25.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.683+0000 7f8589600640 1 --2- 172.21.15.131:0/1195508131 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f85840b2f60 0x7f85841f7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:25.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.683+0000 7f8588c00640 1 -- 172.21.15.131:0/1195508131 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f858415e320 msgr2=0x7f858415cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:25.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.683+0000 7f8588c00640 1 --2- 172.21.15.131:0/1195508131 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f858415e320 0x7f858415cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:25.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.683+0000 7f8588c00640 1 -- 172.21.15.131:0/1195508131 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f85840b2f60 msgr2=0x7f85841f7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:25.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.683+0000 7f8588c00640 1 --2- 172.21.15.131:0/1195508131 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f85840b2f60 0x7f85841f7bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:25.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.683+0000 7f8588c00640 1 -- 172.21.15.131:0/1195508131 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f858415fa20 con 0x7f85840d0380 2024-05-26T22:49:25.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.683+0000 7f8588c00640 1 --2- 172.21.15.131:0/1195508131 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f85840d0380 0x7f8584160ac0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f85740090b0 tx=0x7f85740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:25.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.683+0000 7f8581600640 1 -- 172.21.15.131:0/1195508131 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8574037040 con 0x7f85840d0380 2024-05-26T22:49:25.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.683+0000 7f8581600640 1 -- 172.21.15.131:0/1195508131 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8574004030 con 0x7f85840d0380 2024-05-26T22:49:25.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.683+0000 7f8581600640 1 -- 172.21.15.131:0/1195508131 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8574004400 con 0x7f85840d0380 2024-05-26T22:49:25.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.683+0000 7f858aa00640 1 -- 172.21.15.131:0/1195508131 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f858415fd00 con 0x7f85840d0380 2024-05-26T22:49:25.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.683+0000 7f858aa00640 1 -- 172.21.15.131:0/1195508131 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85841602c0 con 0x7f85840d0380 2024-05-26T22:49:25.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.684+0000 7f858aa00640 1 -- 172.21.15.131:0/1195508131 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85840694e0 con 0x7f85840d0380 2024-05-26T22:49:25.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.686+0000 7f8581600640 1 -- 172.21.15.131:0/1195508131 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f857403a070 con 0x7f85840d0380 2024-05-26T22:49:25.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.686+0000 7f8581600640 1 --2- 172.21.15.131:0/1195508131 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f856806be30 0x7f856806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:25.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.686+0000 7f8581600640 1 -- 172.21.15.131:0/1195508131 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f85740ab060 con 0x7f85840d0380 2024-05-26T22:49:25.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.687+0000 7f8589600640 1 --2- 172.21.15.131:0/1195508131 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f856806be30 0x7f856806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:25.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.687+0000 7f8589600640 1 --2- 172.21.15.131:0/1195508131 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f856806be30 0x7f856806e2f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8578006fd0 tx=0x7f8578008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:25.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.691+0000 7f8581600640 1 -- 172.21.15.131:0/1195508131 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f8574041020 con 0x7f85840d0380 2024-05-26T22:49:25.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.958+0000 7f858aa00640 1 -- 172.21.15.131:0/1195508131 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7f85840b5e50 con 0x7f85840d0380 2024-05-26T22:49:25.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.963+0000 7f8581600640 1 -- 172.21.15.131:0/1195508131 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v10) ==== 90+0+0 (secure 0 0 0) 0x7f8574079d50 con 0x7f85840d0380 2024-05-26T22:49:25.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.965+0000 7f858aa00640 1 -- 172.21.15.131:0/1195508131 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f856806be30 msgr2=0x7f856806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:25.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.965+0000 7f858aa00640 1 --2- 172.21.15.131:0/1195508131 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f856806be30 0x7f856806e2f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8578006fd0 tx=0x7f8578008040 comp rx=0 tx=0).stop 2024-05-26T22:49:25.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.965+0000 7f858aa00640 1 -- 172.21.15.131:0/1195508131 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f85840d0380 msgr2=0x7f8584160ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:25.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.965+0000 7f858aa00640 1 --2- 172.21.15.131:0/1195508131 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f85840d0380 0x7f8584160ac0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f85740090b0 tx=0x7f85740046c0 comp rx=0 tx=0).stop 2024-05-26T22:49:25.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.965+0000 7f858aa00640 1 -- 172.21.15.131:0/1195508131 shutdown_connections 2024-05-26T22:49:25.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.965+0000 7f858aa00640 1 --2- 172.21.15.131:0/1195508131 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f858415e320 0x7f858415cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:25.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.965+0000 7f858aa00640 1 --2- 172.21.15.131:0/1195508131 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f856806be30 0x7f856806e2f0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:25.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.965+0000 7f858aa00640 1 --2- 172.21.15.131:0/1195508131 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f85840d0380 0x7f8584160ac0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:25.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.965+0000 7f858aa00640 1 --2- 172.21.15.131:0/1195508131 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f85840b2f60 0x7f85841f7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:25.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.965+0000 7f858aa00640 1 -- 172.21.15.131:0/1195508131 >> 172.21.15.131:0/1195508131 conn(0x7f85840b5660 msgr2=0x7f85840b3a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:25.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.966+0000 7f858aa00640 1 -- 172.21.15.131:0/1195508131 shutdown_connections 2024-05-26T22:49:25.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:25.966+0000 7f858aa00640 1 -- 172.21.15.131:0/1195508131 wait complete. 2024-05-26T22:49:25.983 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-05-26T22:49:26.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.184+0000 7f87ee200640 1 Processor -- start 2024-05-26T22:49:26.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.184+0000 7f87ee200640 1 -- start start 2024-05-26T22:49:26.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.186+0000 7f87ee200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f87e80b5450 0x7f87e80b43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:26.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.186+0000 7f87ee200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f87e806f420 0x7f87e80cfd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:26.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.186+0000 7f87ee200640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f87e80cf590 0x7f87e80d0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:26.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.186+0000 7f87ee200640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f87e8160d90 con 0x7f87e80b5450 2024-05-26T22:49:26.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.186+0000 7f87ee200640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f87e80b4900 con 0x7f87e80cf590 2024-05-26T22:49:26.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.186+0000 7f87ee200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f87e8160180 con 0x7f87e806f420 2024-05-26T22:49:26.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.186+0000 7f87e7400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f87e80b5450 0x7f87e80b43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:26.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.186+0000 7f87e7400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f87e80b5450 0x7f87e80b43c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41700/0 (socket says 172.21.15.131:41700) 2024-05-26T22:49:26.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.186+0000 7f87e7400640 1 -- 172.21.15.131:0/135323444 learned_addr learned my addr 172.21.15.131:0/135323444 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:26.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.187+0000 7f87e7400640 1 -- 172.21.15.131:0/135323444 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f87e806f420 msgr2=0x7f87e80cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:26.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.187+0000 7f87e7400640 1 --2- 172.21.15.131:0/135323444 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f87e806f420 0x7f87e80cfd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:26.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.187+0000 7f87e7400640 1 -- 172.21.15.131:0/135323444 >> v2:172.21.15.187:3300/0 conn(0x7f87e80cf590 msgr2=0x7f87e80d0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:26.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.187+0000 7f87e7400640 1 --2- 172.21.15.131:0/135323444 >> v2:172.21.15.187:3300/0 conn(0x7f87e80cf590 0x7f87e80d0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:26.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.187+0000 7f87e7400640 1 -- 172.21.15.131:0/135323444 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87e8160460 con 0x7f87e80b5450 2024-05-26T22:49:26.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.188+0000 7f87e7400640 1 --2- 172.21.15.131:0/135323444 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f87e80b5450 0x7f87e80b43c0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f87e815fd40 tx=0x7f87d802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32257f762f10f0d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:26.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.188+0000 7f87e6a00640 1 -- 172.21.15.131:0/135323444 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d8037070 con 0x7f87e80b5450 2024-05-26T22:49:26.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.188+0000 7f87e6a00640 1 -- 172.21.15.131:0/135323444 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87d8032440 con 0x7f87e80b5450 2024-05-26T22:49:26.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.188+0000 7f87e6a00640 1 -- 172.21.15.131:0/135323444 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d8032810 con 0x7f87e80b5450 2024-05-26T22:49:26.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.188+0000 7f87ee200640 1 -- 172.21.15.131:0/135323444 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f87e80b5450 msgr2=0x7f87e80b43c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:26.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.188+0000 7f87ee200640 1 --2- 172.21.15.131:0/135323444 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f87e80b5450 0x7f87e80b43c0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f87e815fd40 tx=0x7f87d802a1e0 comp rx=0 tx=0).stop 2024-05-26T22:49:26.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.189+0000 7f87ee200640 1 -- 172.21.15.131:0/135323444 shutdown_connections 2024-05-26T22:49:26.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.189+0000 7f87ee200640 1 --2- 172.21.15.131:0/135323444 >> v2:172.21.15.187:3300/0 conn(0x7f87e80cf590 0x7f87e80d0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:26.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.189+0000 7f87ee200640 1 --2- 172.21.15.131:0/135323444 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f87e806f420 0x7f87e80cfd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:26.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.189+0000 7f87ee200640 1 --2- 172.21.15.131:0/135323444 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f87e80b5450 0x7f87e80b43c0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:26.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.189+0000 7f87ee200640 1 -- 172.21.15.131:0/135323444 >> 172.21.15.131:0/135323444 conn(0x7f87e806bc90 msgr2=0x7f87e80b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:26.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.189+0000 7f87ee200640 1 -- 172.21.15.131:0/135323444 shutdown_connections 2024-05-26T22:49:26.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.189+0000 7f87ee200640 1 -- 172.21.15.131:0/135323444 wait complete. 2024-05-26T22:49:26.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.190+0000 7f87ee200640 1 Processor -- start 2024-05-26T22:49:26.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.190+0000 7f87ee200640 1 -- start start 2024-05-26T22:49:26.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.191+0000 7f87ee200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f87e806f420 0x7f87e8154a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:26.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.191+0000 7f87ee200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f87e80b5450 0x7f87e8146810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:26.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.191+0000 7f87ee200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f87e80cf590 0x7f87e8157070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:26.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.191+0000 7f87ee200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f87e80b5830 con 0x7f87e80cf590 2024-05-26T22:49:26.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.191+0000 7f87ee200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f87e8160a20 con 0x7f87e80b5450 2024-05-26T22:49:26.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.191+0000 7f87ee200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f87e815fde0 con 0x7f87e806f420 2024-05-26T22:49:26.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.192+0000 7f87e7e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f87e80cf590 0x7f87e8157070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:26.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.192+0000 7f87e7e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f87e80cf590 0x7f87e8157070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41708/0 (socket says 172.21.15.131:41708) 2024-05-26T22:49:26.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.192+0000 7f87e7e00640 1 -- 172.21.15.131:0/2588866288 learned_addr learned my addr 172.21.15.131:0/2588866288 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:26.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.192+0000 7f87e7e00640 1 -- 172.21.15.131:0/2588866288 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f87e806f420 msgr2=0x7f87e8154a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:26.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.192+0000 7f87e7400640 1 --2- 172.21.15.131:0/2588866288 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f87e806f420 0x7f87e8154a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:26.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.192+0000 7f87dfe00640 1 --2- 172.21.15.131:0/2588866288 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f87e80b5450 0x7f87e8146810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:26.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.192+0000 7f87e7e00640 1 --2- 172.21.15.131:0/2588866288 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f87e806f420 0x7f87e8154a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:26.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.192+0000 7f87e7e00640 1 -- 172.21.15.131:0/2588866288 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f87e80b5450 msgr2=0x7f87e8146810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:26.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.192+0000 7f87e7e00640 1 --2- 172.21.15.131:0/2588866288 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f87e80b5450 0x7f87e8146810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:26.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.192+0000 7f87e7e00640 1 -- 172.21.15.131:0/2588866288 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87e8154f60 con 0x7f87e80cf590 2024-05-26T22:49:26.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.192+0000 7f87dfe00640 1 --2- 172.21.15.131:0/2588866288 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f87e80b5450 0x7f87e8146810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:49:26.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.192+0000 7f87e7e00640 1 --2- 172.21.15.131:0/2588866288 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f87e80cf590 0x7f87e8157070 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f87e000b9a0 tx=0x7f87e000be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:26.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.193+0000 7f87df400640 1 -- 172.21.15.131:0/2588866288 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e0019070 con 0x7f87e80cf590 2024-05-26T22:49:26.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.193+0000 7f87df400640 1 -- 172.21.15.131:0/2588866288 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87e000cbc0 con 0x7f87e80cf590 2024-05-26T22:49:26.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.193+0000 7f87df400640 1 -- 172.21.15.131:0/2588866288 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e0014440 con 0x7f87e80cf590 2024-05-26T22:49:26.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.193+0000 7f87ee200640 1 -- 172.21.15.131:0/2588866288 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87e8146d50 con 0x7f87e80cf590 2024-05-26T22:49:26.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.193+0000 7f87ee200640 1 -- 172.21.15.131:0/2588866288 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87e81554e0 con 0x7f87e80cf590 2024-05-26T22:49:26.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.194+0000 7f87ee200640 1 -- 172.21.15.131:0/2588866288 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87e80695c0 con 0x7f87e80cf590 2024-05-26T22:49:26.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.195+0000 7f87df400640 1 -- 172.21.15.131:0/2588866288 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f87e001c070 con 0x7f87e80cf590 2024-05-26T22:49:26.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.196+0000 7f87df400640 1 --2- 172.21.15.131:0/2588866288 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f87c406be30 0x7f87c406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:26.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.196+0000 7f87df400640 1 -- 172.21.15.131:0/2588866288 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f87e008a900 con 0x7f87e80cf590 2024-05-26T22:49:26.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.196+0000 7f87e7400640 1 --2- 172.21.15.131:0/2588866288 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f87c406be30 0x7f87c406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:26.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.197+0000 7f87e7400640 1 --2- 172.21.15.131:0/2588866288 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f87c406be30 0x7f87c406e2f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f87d8034000 tx=0x7f87d802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:26.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.200+0000 7f87df400640 1 -- 172.21.15.131:0/2588866288 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f87e0059560 con 0x7f87e80cf590 2024-05-26T22:49:26.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.472+0000 7f87ee200640 1 -- 172.21.15.131:0/2588866288 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7f87e80b20e0 con 0x7f87e80cf590 2024-05-26T22:49:26.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.477+0000 7f87df400640 1 -- 172.21.15.131:0/2588866288 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v11) ==== 97+0+0 (secure 0 0 0) 0x7f87e0058e60 con 0x7f87e80cf590 2024-05-26T22:49:26.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.480+0000 7f87ee200640 1 -- 172.21.15.131:0/2588866288 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f87c406be30 msgr2=0x7f87c406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:26.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.480+0000 7f87ee200640 1 --2- 172.21.15.131:0/2588866288 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f87c406be30 0x7f87c406e2f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f87d8034000 tx=0x7f87d802e000 comp rx=0 tx=0).stop 2024-05-26T22:49:26.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.480+0000 7f87ee200640 1 -- 172.21.15.131:0/2588866288 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f87e80cf590 msgr2=0x7f87e8157070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:26.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.480+0000 7f87ee200640 1 --2- 172.21.15.131:0/2588866288 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f87e80cf590 0x7f87e8157070 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f87e000b9a0 tx=0x7f87e000be60 comp rx=0 tx=0).stop 2024-05-26T22:49:26.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.480+0000 7f87ee200640 1 -- 172.21.15.131:0/2588866288 shutdown_connections 2024-05-26T22:49:26.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.480+0000 7f87ee200640 1 --2- 172.21.15.131:0/2588866288 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f87c406be30 0x7f87c406e2f0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:26.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.480+0000 7f87ee200640 1 --2- 172.21.15.131:0/2588866288 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f87e80cf590 0x7f87e8157070 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:26.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.480+0000 7f87ee200640 1 --2- 172.21.15.131:0/2588866288 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f87e80b5450 0x7f87e8146810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:26.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.480+0000 7f87ee200640 1 --2- 172.21.15.131:0/2588866288 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f87e806f420 0x7f87e8154a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:26.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.480+0000 7f87ee200640 1 -- 172.21.15.131:0/2588866288 >> 172.21.15.131:0/2588866288 conn(0x7f87e806bc90 msgr2=0x7f87e815d9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:26.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.480+0000 7f87ee200640 1 -- 172.21.15.131:0/2588866288 shutdown_connections 2024-05-26T22:49:26.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.480+0000 7f87ee200640 1 -- 172.21.15.131:0/2588866288 wait complete. 2024-05-26T22:49:26.499 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-05-26T22:49:26.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.687+0000 7f4d25800640 1 Processor -- start 2024-05-26T22:49:26.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.688+0000 7f4d25800640 1 -- start start 2024-05-26T22:49:26.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.689+0000 7f4d25800640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f4d200b5450 0x7f4d200b43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:26.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.689+0000 7f4d25800640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f4d2006f420 0x7f4d200cfd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:26.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.689+0000 7f4d25800640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d200b4900 con 0x7f4d200cf590 2024-05-26T22:49:26.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.690+0000 7f4d25800640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f4d20160d90 con 0x7f4d2006f420 2024-05-26T22:49:26.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.690+0000 7f4d25800640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f4d20160180 con 0x7f4d200b5450 2024-05-26T22:49:26.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.690+0000 7f4d17e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f4d2006f420 0x7f4d200cfd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:26.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.690+0000 7f4d17e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f4d2006f420 0x7f4d200cfd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41716/0 (socket says 172.21.15.131:41716) 2024-05-26T22:49:26.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.690+0000 7f4d17e00640 1 -- 172.21.15.131:0/323154213 learned_addr learned my addr 172.21.15.131:0/323154213 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:26.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.690+0000 7f4d17e00640 1 -- 172.21.15.131:0/323154213 >> v2:172.21.15.187:3300/0 conn(0x7f4d200b5450 msgr2=0x7f4d200b43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:26.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.690+0000 7f4d1ea00640 1 --2- 172.21.15.131:0/323154213 >> v2:172.21.15.187:3300/0 conn(0x7f4d200b5450 0x7f4d200b43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:26.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.690+0000 7f4d17e00640 1 --2- 172.21.15.131:0/323154213 >> v2:172.21.15.187:3300/0 conn(0x7f4d200b5450 0x7f4d200b43c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:26.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.690+0000 7f4d17e00640 1 -- 172.21.15.131:0/323154213 >> v1:172.21.15.131:6789/0 conn(0x7f4d200cf590 legacy=0x7f4d2006d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:26.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.690+0000 7f4d17e00640 1 -- 172.21.15.131:0/323154213 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d20160460 con 0x7f4d2006f420 2024-05-26T22:49:26.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.691+0000 7f4d17e00640 1 --2- 172.21.15.131:0/323154213 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d2006f420 0x7f4d200cfd50 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f4d200d0c40 tx=0x7f4d0802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6334f11744ae23a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:26.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.692+0000 7f4d1e000640 1 -- 172.21.15.131:0/323154213 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d08037070 con 0x7f4d2006f420 2024-05-26T22:49:26.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.692+0000 7f4d1e000640 1 -- 172.21.15.131:0/323154213 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d08032440 con 0x7f4d2006f420 2024-05-26T22:49:26.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.692+0000 7f4d1e000640 1 -- 172.21.15.131:0/323154213 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d08032810 con 0x7f4d2006f420 2024-05-26T22:49:26.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.692+0000 7f4d25800640 1 -- 172.21.15.131:0/323154213 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d2006f420 msgr2=0x7f4d200cfd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:26.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.692+0000 7f4d25800640 1 --2- 172.21.15.131:0/323154213 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d2006f420 0x7f4d200cfd50 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f4d200d0c40 tx=0x7f4d0802a270 comp rx=0 tx=0).stop 2024-05-26T22:49:26.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.692+0000 7f4d25800640 1 -- 172.21.15.131:0/323154213 shutdown_connections 2024-05-26T22:49:26.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.692+0000 7f4d25800640 1 --2- 172.21.15.131:0/323154213 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d2006f420 0x7f4d200cfd50 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:26.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.692+0000 7f4d25800640 1 --2- 172.21.15.131:0/323154213 >> v2:172.21.15.187:3300/0 conn(0x7f4d200b5450 0x7f4d200b43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:26.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.692+0000 7f4d25800640 1 -- 172.21.15.131:0/323154213 >> 172.21.15.131:0/323154213 conn(0x7f4d2006bc90 msgr2=0x7f4d200b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:26.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.692+0000 7f4d25800640 1 -- 172.21.15.131:0/323154213 shutdown_connections 2024-05-26T22:49:26.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.692+0000 7f4d25800640 1 -- 172.21.15.131:0/323154213 wait complete. 2024-05-26T22:49:26.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.694+0000 7f4d25800640 1 Processor -- start 2024-05-26T22:49:26.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.694+0000 7f4d25800640 1 -- start start 2024-05-26T22:49:26.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.695+0000 7f4d25800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d2006f420 0x7f4d200cbd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:26.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.695+0000 7f4d25800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4d200b5450 0x7f4d2009ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:26.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.695+0000 7f4d25800640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4d200cf590 0x7f4d200ae070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:26.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.695+0000 7f4d25800640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f4d200b5830 con 0x7f4d2006f420 2024-05-26T22:49:26.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.695+0000 7f4d25800640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4d20160a20 con 0x7f4d200cf590 2024-05-26T22:49:26.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.695+0000 7f4d25800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f4d2015fde0 con 0x7f4d200b5450 2024-05-26T22:49:26.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.696+0000 7f4d1ea00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d2006f420 0x7f4d200cbd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:26.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.696+0000 7f4d1ea00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d2006f420 0x7f4d200cbd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41724/0 (socket says 172.21.15.131:41724) 2024-05-26T22:49:26.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.696+0000 7f4d1ea00640 1 -- 172.21.15.131:0/3696542526 learned_addr learned my addr 172.21.15.131:0/3696542526 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:26.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.696+0000 7f4d1ea00640 1 -- 172.21.15.131:0/3696542526 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4d200b5450 msgr2=0x7f4d2009ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:26.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.696+0000 7f4d1ea00640 1 --2- 172.21.15.131:0/3696542526 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4d200b5450 0x7f4d2009ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:26.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.696+0000 7f4d1ea00640 1 -- 172.21.15.131:0/3696542526 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4d200cf590 msgr2=0x7f4d200ae070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:26.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.696+0000 7f4d1f400640 1 --2- 172.21.15.131:0/3696542526 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4d200cf590 0x7f4d200ae070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:26.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.696+0000 7f4d1ea00640 1 --2- 172.21.15.131:0/3696542526 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4d200cf590 0x7f4d200ae070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:26.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.696+0000 7f4d1ea00640 1 -- 172.21.15.131:0/3696542526 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d2009d4f0 con 0x7f4d2006f420 2024-05-26T22:49:26.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.696+0000 7f4d1ea00640 1 --2- 172.21.15.131:0/3696542526 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d2006f420 0x7f4d200cbd60 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f4d1000c7f0 tx=0x7f4d1000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:26.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.697+0000 7f4d16a00640 1 -- 172.21.15.131:0/3696542526 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d10015070 con 0x7f4d2006f420 2024-05-26T22:49:26.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.697+0000 7f4d16a00640 1 -- 172.21.15.131:0/3696542526 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d10004240 con 0x7f4d2006f420 2024-05-26T22:49:26.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.697+0000 7f4d25800640 1 -- 172.21.15.131:0/3696542526 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d200c93c0 con 0x7f4d2006f420 2024-05-26T22:49:26.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.697+0000 7f4d16a00640 1 -- 172.21.15.131:0/3696542526 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d10007620 con 0x7f4d2006f420 2024-05-26T22:49:26.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.697+0000 7f4d25800640 1 -- 172.21.15.131:0/3696542526 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d200c9980 con 0x7f4d2006f420 2024-05-26T22:49:26.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.698+0000 7f4d25800640 1 -- 172.21.15.131:0/3696542526 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d20074e90 con 0x7f4d2006f420 2024-05-26T22:49:26.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.699+0000 7f4d16a00640 1 -- 172.21.15.131:0/3696542526 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4d100079d0 con 0x7f4d2006f420 2024-05-26T22:49:26.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.700+0000 7f4d16a00640 1 --2- 172.21.15.131:0/3696542526 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f4cfc06be30 0x7f4cfc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:26.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.700+0000 7f4d16a00640 1 -- 172.21.15.131:0/3696542526 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f4d1008aad0 con 0x7f4d2006f420 2024-05-26T22:49:26.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.700+0000 7f4d17e00640 1 --2- 172.21.15.131:0/3696542526 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f4cfc06be30 0x7f4cfc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:26.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.701+0000 7f4d17e00640 1 --2- 172.21.15.131:0/3696542526 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f4cfc06be30 0x7f4cfc06e2f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4d08034000 tx=0x7f4d0802c000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:26.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.704+0000 7f4d16a00640 1 -- 172.21.15.131:0/3696542526 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f4d10059860 con 0x7f4d2006f420 2024-05-26T22:49:26.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.982+0000 7f4d25800640 1 -- 172.21.15.131:0/3696542526 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7f4d200695c0 con 0x7f4d2006f420 2024-05-26T22:49:26.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.986+0000 7f4d16a00640 1 -- 172.21.15.131:0/3696542526 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v12) ==== 90+0+0 (secure 0 0 0) 0x7f4d10059160 con 0x7f4d2006f420 2024-05-26T22:49:26.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.989+0000 7f4d25800640 1 -- 172.21.15.131:0/3696542526 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f4cfc06be30 msgr2=0x7f4cfc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:26.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.989+0000 7f4d25800640 1 --2- 172.21.15.131:0/3696542526 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f4cfc06be30 0x7f4cfc06e2f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4d08034000 tx=0x7f4d0802c000 comp rx=0 tx=0).stop 2024-05-26T22:49:26.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.989+0000 7f4d25800640 1 -- 172.21.15.131:0/3696542526 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d2006f420 msgr2=0x7f4d200cbd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:26.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.989+0000 7f4d25800640 1 --2- 172.21.15.131:0/3696542526 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d2006f420 0x7f4d200cbd60 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f4d1000c7f0 tx=0x7f4d1000ccb0 comp rx=0 tx=0).stop 2024-05-26T22:49:26.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.989+0000 7f4d25800640 1 -- 172.21.15.131:0/3696542526 shutdown_connections 2024-05-26T22:49:26.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.989+0000 7f4d25800640 1 --2- 172.21.15.131:0/3696542526 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4d200b5450 0x7f4d2009ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:26.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.989+0000 7f4d25800640 1 --2- 172.21.15.131:0/3696542526 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f4cfc06be30 0x7f4cfc06e2f0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:26.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.989+0000 7f4d25800640 1 --2- 172.21.15.131:0/3696542526 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4d200cf590 0x7f4d200ae070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:26.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.989+0000 7f4d25800640 1 --2- 172.21.15.131:0/3696542526 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d2006f420 0x7f4d200cbd60 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:26.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.989+0000 7f4d25800640 1 -- 172.21.15.131:0/3696542526 >> 172.21.15.131:0/3696542526 conn(0x7f4d2006bc90 msgr2=0x7f4d200cc940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:26.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.989+0000 7f4d25800640 1 -- 172.21.15.131:0/3696542526 shutdown_connections 2024-05-26T22:49:26.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:26.989+0000 7f4d25800640 1 -- 172.21.15.131:0/3696542526 wait complete. 2024-05-26T22:49:27.008 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-26T22:49:27.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.209+0000 7febf9c00640 1 Processor -- start 2024-05-26T22:49:27.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.209+0000 7febf9c00640 1 -- start start 2024-05-26T22:49:27.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.210+0000 7febf9c00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7febf4144340 0x7febf40cedf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:27.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.210+0000 7febf9c00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7febf4071fd0 con 0x7febf4071b70 2024-05-26T22:49:27.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.210+0000 7febf9c00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7febf40722a0 con 0x7febf406ebc0 2024-05-26T22:49:27.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.210+0000 7febf9c00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7febf4072eb0 con 0x7febf4144340 2024-05-26T22:49:27.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.211+0000 7febf3400640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7febf4071b70 0x7febf406e4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:37986/0 (socket says 172.21.15.131:37986) 2024-05-26T22:49:27.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.211+0000 7febf3400640 1 -- 172.21.15.131:0/313942800 learned_addr learned my addr 172.21.15.131:0/313942800 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:49:27.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.211+0000 7febf3e00640 1 --2- 172.21.15.131:0/313942800 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7febf4144340 0x7febf40cedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:27.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.211+0000 7febf2a00640 1 -- 172.21.15.131:0/313942800 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3170127719 0 0) 0x7febf4071fd0 con 0x7febf4071b70 2024-05-26T22:49:27.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.211+0000 7febf2a00640 1 -- 172.21.15.131:0/313942800 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7febd8003520 con 0x7febf4071b70 2024-05-26T22:49:27.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.212+0000 7febf2a00640 1 -- 172.21.15.131:0/313942800 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1749577292 0 0) 0x7febd8003520 con 0x7febf4071b70 2024-05-26T22:49:27.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.212+0000 7febf2a00640 1 -- 172.21.15.131:0/313942800 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7febf4144340 msgr2=0x7febf40cedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:27.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.212+0000 7febf2a00640 1 --2- 172.21.15.131:0/313942800 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7febf4144340 0x7febf40cedf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:27.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.212+0000 7febf2a00640 1 -- 172.21.15.131:0/313942800 >> v1:172.21.15.187:6789/0 conn(0x7febf406ebc0 legacy=0x7febf40b3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:27.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.212+0000 7febf2a00640 1 -- 172.21.15.131:0/313942800 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febf4072580 con 0x7febf4071b70 2024-05-26T22:49:27.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.212+0000 7febf2a00640 1 -- 172.21.15.131:0/313942800 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7febdc001670 con 0x7febf4071b70 2024-05-26T22:49:27.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.212+0000 7febf2a00640 1 -- 172.21.15.131:0/313942800 >> v1:172.21.15.131:6789/0 conn(0x7febf4071b70 legacy=0x7febf406e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:27.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.212+0000 7febf2a00640 1 --2- 172.21.15.131:0/313942800 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7febf4144340 0x7febd80039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:27.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.212+0000 7febf2a00640 1 --2- 172.21.15.131:0/313942800 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7febd8003fb0 0x7febd8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:27.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.212+0000 7febf2a00640 1 --2- 172.21.15.131:0/313942800 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7febd8004d40 0x7febd80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:27.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.212+0000 7febf2a00640 1 -- 172.21.15.131:0/313942800 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7febd800bad0 con 0x7febd8003fb0 2024-05-26T22:49:27.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.212+0000 7febf2a00640 1 -- 172.21.15.131:0/313942800 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7febd800bc80 con 0x7febd8004d40 2024-05-26T22:49:27.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.212+0000 7febf2a00640 1 -- 172.21.15.131:0/313942800 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7febd800be80 con 0x7febf4144340 2024-05-26T22:49:27.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.212+0000 7febf3400640 1 --2- 172.21.15.131:0/313942800 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7febd8003fb0 0x7febd8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:27.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.212+0000 7febf9200640 1 --2- 172.21.15.131:0/313942800 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7febd8004d40 0x7febd80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:27.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.213+0000 7febf3400640 1 -- 172.21.15.131:0/313942800 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7febf4144340 msgr2=0x7febd80039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:27.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.213+0000 7febf3400640 1 --2- 172.21.15.131:0/313942800 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7febf4144340 0x7febd80039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:27.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.213+0000 7febf3400640 1 -- 172.21.15.131:0/313942800 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7febd8004d40 msgr2=0x7febd80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:27.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.213+0000 7febf3400640 1 --2- 172.21.15.131:0/313942800 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7febd8004d40 0x7febd80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:27.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.213+0000 7febf3400640 1 -- 172.21.15.131:0/313942800 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febd8003520 con 0x7febd8003fb0 2024-05-26T22:49:27.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.213+0000 7febf9200640 1 --2- 172.21.15.131:0/313942800 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7febd8004d40 0x7febd80051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:49:27.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.214+0000 7febf3400640 1 --2- 172.21.15.131:0/313942800 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7febd8003fb0 0x7febd8004800 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7febdc00c190 tx=0x7febd8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6110cfa1a91d54fe server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:27.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.214+0000 7febf2a00640 1 -- 172.21.15.131:0/313942800 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febdc007c60 con 0x7febd8003fb0 2024-05-26T22:49:27.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.214+0000 7febf2a00640 1 -- 172.21.15.131:0/313942800 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febdc005ad0 con 0x7febd8003fb0 2024-05-26T22:49:27.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.214+0000 7febf2a00640 1 -- 172.21.15.131:0/313942800 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febdc007070 con 0x7febd8003fb0 2024-05-26T22:49:27.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.214+0000 7febf9c00640 1 -- 172.21.15.131:0/313942800 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7febd8003fb0 msgr2=0x7febd8004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:27.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.214+0000 7febf9c00640 1 --2- 172.21.15.131:0/313942800 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7febd8003fb0 0x7febd8004800 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7febdc00c190 tx=0x7febd8005830 comp rx=0 tx=0).stop 2024-05-26T22:49:27.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.214+0000 7febf9200640 1 -- 172.21.15.131:0/313942800 reap_dead start 2024-05-26T22:49:27.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.215+0000 7febf9c00640 1 -- 172.21.15.131:0/313942800 shutdown_connections 2024-05-26T22:49:27.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.215+0000 7febf9c00640 1 -- 172.21.15.131:0/313942800 >> 172.21.15.131:0/313942800 conn(0x7febf40cff40 msgr2=0x7febf40d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:27.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.215+0000 7febf9c00640 1 -- 172.21.15.131:0/313942800 shutdown_connections 2024-05-26T22:49:27.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.215+0000 7febf9c00640 1 -- 172.21.15.131:0/313942800 wait complete. 2024-05-26T22:49:27.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.216+0000 7febf9c00640 1 Processor -- start 2024-05-26T22:49:27.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.217+0000 7febf9c00640 1 -- start start 2024-05-26T22:49:27.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.217+0000 7febf9c00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7febd8003fb0 0x7febf415dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:27.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.217+0000 7febf9c00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7febf413fae0 0x7febf406d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:27.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.218+0000 7febf9c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7febf413f520 0x7febf413e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:27.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.218+0000 7febf9c00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7febf40cedf0 con 0x7febf413f520 2024-05-26T22:49:27.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.218+0000 7febf9c00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7febf4072b40 con 0x7febf413fae0 2024-05-26T22:49:27.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.218+0000 7febf9c00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7febf40cef70 con 0x7febd8003fb0 2024-05-26T22:49:27.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.218+0000 7febf9200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7febf413f520 0x7febf413e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:27.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.218+0000 7febf9200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7febf413f520 0x7febf413e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41744/0 (socket says 172.21.15.131:41744) 2024-05-26T22:49:27.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.218+0000 7febf9200640 1 -- 172.21.15.131:0/3697841622 learned_addr learned my addr 172.21.15.131:0/3697841622 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:27.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.218+0000 7febf3e00640 1 --2- 172.21.15.131:0/3697841622 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7febd8003fb0 0x7febf415dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:27.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.218+0000 7febf3400640 1 --2- 172.21.15.131:0/3697841622 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7febf413fae0 0x7febf406d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:27.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.218+0000 7febf9200640 1 -- 172.21.15.131:0/3697841622 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7febd8003fb0 msgr2=0x7febf415dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:27.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.218+0000 7febf9200640 1 --2- 172.21.15.131:0/3697841622 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7febd8003fb0 0x7febf415dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:27.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.218+0000 7febf9200640 1 -- 172.21.15.131:0/3697841622 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7febf413fae0 msgr2=0x7febf406d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:27.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.218+0000 7febf9200640 1 --2- 172.21.15.131:0/3697841622 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7febf413fae0 0x7febf406d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:27.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.218+0000 7febf9200640 1 -- 172.21.15.131:0/3697841622 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febf415cb70 con 0x7febf413f520 2024-05-26T22:49:27.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.218+0000 7febf3400640 1 --2- 172.21.15.131:0/3697841622 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7febf413fae0 0x7febf406d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:49:27.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.219+0000 7febf9200640 1 --2- 172.21.15.131:0/3697841622 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7febf413f520 0x7febf413e9a0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7febec000f80 tx=0x7febec00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:27.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.219+0000 7febebe00640 1 -- 172.21.15.131:0/3697841622 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febec006d50 con 0x7febf413f520 2024-05-26T22:49:27.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.219+0000 7febebe00640 1 -- 172.21.15.131:0/3697841622 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febec003970 con 0x7febf413f520 2024-05-26T22:49:27.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.219+0000 7febebe00640 1 -- 172.21.15.131:0/3697841622 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febec003da0 con 0x7febf413f520 2024-05-26T22:49:27.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.219+0000 7febf9c00640 1 -- 172.21.15.131:0/3697841622 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febf415ce50 con 0x7febf413f520 2024-05-26T22:49:27.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.219+0000 7febf9c00640 1 -- 172.21.15.131:0/3697841622 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febf415d410 con 0x7febf413f520 2024-05-26T22:49:27.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.220+0000 7febf9c00640 1 -- 172.21.15.131:0/3697841622 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febf40694e0 con 0x7febf413f520 2024-05-26T22:49:27.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.221+0000 7febebe00640 1 -- 172.21.15.131:0/3697841622 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7febec01b020 con 0x7febf413f520 2024-05-26T22:49:27.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.222+0000 7febebe00640 1 --2- 172.21.15.131:0/3697841622 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7febd006be60 0x7febd006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:27.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.222+0000 7febebe00640 1 -- 172.21.15.131:0/3697841622 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7febec08a7c0 con 0x7febf413f520 2024-05-26T22:49:27.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.222+0000 7febf3e00640 1 --2- 172.21.15.131:0/3697841622 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7febd006be60 0x7febd006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:27.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.223+0000 7febf3e00640 1 --2- 172.21.15.131:0/3697841622 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7febd006be60 0x7febd006e320 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7febe40047b0 tx=0x7febe4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:27.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.228+0000 7febebe00640 1 -- 172.21.15.131:0/3697841622 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7febec059420 con 0x7febf413f520 2024-05-26T22:49:27.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.427+0000 7febf9c00640 1 -- 172.21.15.131:0/3697841622 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7febf40ce720 con 0x7febf413f520 2024-05-26T22:49:27.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.429+0000 7febebe00640 1 -- 172.21.15.131:0/3697841622 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) ==== 74+0+12915 (secure 0 0 0) 0x7febec058d20 con 0x7febf413f520 2024-05-26T22:49:27.429 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:49:27.429 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":29,"fsid":"46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f","created":"2024-05-26T22:47:57.288905+0000","modified":"2024-05-26T22:49:23.477339+0000","last_up_change":"2024-05-26T22:48:09.602207+0000","last_in_change":"2024-05-26T22:48:00.687341+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-26T22:48:10.463339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"814eaf62-a18d-494b-8e27-fd8fb88adea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6801","nonce":2350940051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6803","nonce":2350940051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6807","nonce":2350940051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6805","nonce":2350940051}]},"public_addr":"172.21.15.131:6801/2350940051","cluster_addr":"172.21.15.131:6803/2350940051","heartbeat_back_addr":"172.21.15.131:6807/2350940051","heartbeat_front_addr":"172.21.15.131:6805/2350940051","state":["exists","up"]},{"osd":1,"uuid":"42e66287-f6c6-4e26-af3c-6dfaa3536d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6825","nonce":1525528247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6827","nonce":1525528247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6831","nonce":1525528247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6829","nonce":1525528247}]},"public_addr":"172.21.15.131:6825/1525528247","cluster_addr":"172.21.15.131:6827/1525528247","heartbeat_back_addr":"172.21.15.131:6831/1525528247","heartbeat_front_addr":"172.21.15.131:6829/1525528247","state":["exists","up"]},{"osd":2,"uuid":"0da34673-04a2-4fd0-b57a-0ff2b679b043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6817","nonce":3627842210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6819","nonce":3627842210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6823","nonce":3627842210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6821","nonce":3627842210}]},"public_addr":"172.21.15.131:6817/3627842210","cluster_addr":"172.21.15.131:6819/3627842210","heartbeat_back_addr":"172.21.15.131:6823/3627842210","heartbeat_front_addr":"172.21.15.131:6821/3627842210","state":["exists","up"]},{"osd":3,"uuid":"cf117a27-d6b2-43d3-ae52-b47988bf8eed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6809","nonce":350790591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6811","nonce":350790591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6815","nonce":350790591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6813","nonce":350790591}]},"public_addr":"172.21.15.131:6809/350790591","cluster_addr":"172.21.15.131:6811/350790591","heartbeat_back_addr":"172.21.15.131:6815/350790591","heartbeat_front_addr":"172.21.15.131:6813/350790591","state":["exists","up"]},{"osd":4,"uuid":"a092ccfc-be89-4a84-88d5-0f781fa715eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6816","nonce":244560995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6819","nonce":244560995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6823","nonce":244560995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6821","nonce":244560995}]},"public_addr":"172.21.15.187:6816/244560995","cluster_addr":"172.21.15.187:6819/244560995","heartbeat_back_addr":"172.21.15.187:6823/244560995","heartbeat_front_addr":"172.21.15.187:6821/244560995","state":["exists","up"]},{"osd":5,"uuid":"7513ba69-dd00-4a35-b190-9e8cc3b2bdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6801","nonce":324619398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6803","nonce":324619398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6812","nonce":324619398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6807","nonce":324619398}]},"public_addr":"172.21.15.187:6801/324619398","cluster_addr":"172.21.15.187:6803/324619398","heartbeat_back_addr":"172.21.15.187:6812/324619398","heartbeat_front_addr":"172.21.15.187:6807/324619398","state":["exists","up"]},{"osd":6,"uuid":"50081810-c093-4d2f-a4d7-b7c695c5400d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6806","nonce":1391347786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6809","nonce":1391347786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6818","nonce":1391347786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6813","nonce":1391347786}]},"public_addr":"172.21.15.187:6806/1391347786","cluster_addr":"172.21.15.187:6809/1391347786","heartbeat_back_addr":"172.21.15.187:6818/1391347786","heartbeat_front_addr":"172.21.15.187:6813/1391347786","state":["exists","up"]},{"osd":7,"uuid":"775dae05-54f5-4ad4-bbce-92863c6ed2ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6825","nonce":1220065622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6827","nonce":1220065622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6831","nonce":1220065622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6829","nonce":1220065622}]},"public_addr":"172.21.15.187:6825/1220065622","cluster_addr":"172.21.15.187:6827/1220065622","heartbeat_back_addr":"172.21.15.187:6831/1220065622","heartbeat_front_addr":"172.21.15.187:6829/1220065622","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:05.983520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.299516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.067323+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.307748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.018742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.258775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.010912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.295355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-26T22:49:27.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.432+0000 7febf9c00640 1 -- 172.21.15.131:0/3697841622 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7febd006be60 msgr2=0x7febd006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:27.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.432+0000 7febf9c00640 1 --2- 172.21.15.131:0/3697841622 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7febd006be60 0x7febd006e320 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7febe40047b0 tx=0x7febe4008040 comp rx=0 tx=0).stop 2024-05-26T22:49:27.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.432+0000 7febf9c00640 1 -- 172.21.15.131:0/3697841622 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7febf413f520 msgr2=0x7febf413e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:27.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.432+0000 7febf9c00640 1 --2- 172.21.15.131:0/3697841622 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7febf413f520 0x7febf413e9a0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7febec000f80 tx=0x7febec00dc00 comp rx=0 tx=0).stop 2024-05-26T22:49:27.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.432+0000 7febf9c00640 1 -- 172.21.15.131:0/3697841622 shutdown_connections 2024-05-26T22:49:27.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.432+0000 7febf9c00640 1 --2- 172.21.15.131:0/3697841622 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7febd006be60 0x7febd006e320 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:27.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.432+0000 7febf9c00640 1 --2- 172.21.15.131:0/3697841622 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7febf413f520 0x7febf413e9a0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:27.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.432+0000 7febf9c00640 1 --2- 172.21.15.131:0/3697841622 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7febf413fae0 0x7febf406d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:27.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.432+0000 7febf9c00640 1 --2- 172.21.15.131:0/3697841622 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7febd8003fb0 0x7febf415dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:27.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.432+0000 7febf9c00640 1 -- 172.21.15.131:0/3697841622 >> 172.21.15.131:0/3697841622 conn(0x7febf40cff40 msgr2=0x7febf40d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:27.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.432+0000 7febf9c00640 1 -- 172.21.15.131:0/3697841622 shutdown_connections 2024-05-26T22:49:27.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.432+0000 7febf9c00640 1 -- 172.21.15.131:0/3697841622 wait complete. 2024-05-26T22:49:27.452 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-26T22:48:10.463339+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-26T22:49:27.453 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-26T22:49:27.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.658+0000 7f3d8a600640 1 Processor -- start 2024-05-26T22:49:27.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.658+0000 7f3d8a600640 1 -- start start 2024-05-26T22:49:27.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.660+0000 7f3d8a600640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f3d840af580 0x7f3d8415f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:27.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.660+0000 7f3d8a600640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f3d8406b680 0x7f3d8406bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:27.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.660+0000 7f3d8a600640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d840ae0c0 con 0x7f3d840af960 2024-05-26T22:49:27.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.660+0000 7f3d8a600640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f3d840ae3a0 con 0x7f3d8406b680 2024-05-26T22:49:27.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.660+0000 7f3d8a600640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f3d840ae680 con 0x7f3d840af580 2024-05-26T22:49:27.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.660+0000 7f3d83e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f3d8406b680 0x7f3d8406bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:27.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.661+0000 7f3d82a00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f3d840af960 0x7f3d840ad870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:38002/0 (socket says 172.21.15.131:38002) 2024-05-26T22:49:27.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.661+0000 7f3d82a00640 1 -- 172.21.15.131:0/2289578116 learned_addr learned my addr 172.21.15.131:0/2289578116 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:49:27.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.661+0000 7f3d83e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f3d8406b680 0x7f3d8406bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41760/0 (socket says 172.21.15.131:41760) 2024-05-26T22:49:27.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.661+0000 7f3d83400640 1 --2- 172.21.15.131:0/2289578116 >> v2:172.21.15.187:3300/0 conn(0x7f3d840af580 0x7f3d8415f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:27.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.661+0000 7f3d83e00640 1 -- 172.21.15.131:0/2289578116 >> v2:172.21.15.187:3300/0 conn(0x7f3d840af580 msgr2=0x7f3d8415f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:27.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.661+0000 7f3d83e00640 1 --2- 172.21.15.131:0/2289578116 >> v2:172.21.15.187:3300/0 conn(0x7f3d840af580 0x7f3d8415f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:27.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.661+0000 7f3d83e00640 1 -- 172.21.15.131:0/2289578116 >> v1:172.21.15.131:6789/0 conn(0x7f3d840af960 legacy=0x7f3d840ad870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:27.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.661+0000 7f3d83e00640 1 -- 172.21.15.131:0/2289578116 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d840ae960 con 0x7f3d8406b680 2024-05-26T22:49:27.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.661+0000 7f3d82000640 1 -- 172.21.15.131:0/2289578116 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3185184960 0 0) 0x7f3d840ae0c0 con 0x7f3d840af960 2024-05-26T22:49:27.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.662+0000 7f3d83e00640 1 --2- 172.21.15.131:0/2289578116 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3d8406b680 0x7f3d8406bb10 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f3d84070750 tx=0x7f3d7c02a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4345be7b22a34c28 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:27.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.662+0000 7f3d82000640 1 -- 172.21.15.131:0/2289578116 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d7c037070 con 0x7f3d8406b680 2024-05-26T22:49:27.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.662+0000 7f3d82000640 1 -- 172.21.15.131:0/2289578116 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d7c02add0 con 0x7f3d8406b680 2024-05-26T22:49:27.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.662+0000 7f3d82000640 1 -- 172.21.15.131:0/2289578116 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d7c032660 con 0x7f3d8406b680 2024-05-26T22:49:27.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.663+0000 7f3d8a600640 1 -- 172.21.15.131:0/2289578116 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3d8406b680 msgr2=0x7f3d8406bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:27.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.663+0000 7f3d8a600640 1 --2- 172.21.15.131:0/2289578116 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3d8406b680 0x7f3d8406bb10 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f3d84070750 tx=0x7f3d7c02a080 comp rx=0 tx=0).stop 2024-05-26T22:49:27.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.663+0000 7f3d8a600640 1 -- 172.21.15.131:0/2289578116 shutdown_connections 2024-05-26T22:49:27.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.663+0000 7f3d8a600640 1 --2- 172.21.15.131:0/2289578116 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3d8406b680 0x7f3d8406bb10 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:27.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.663+0000 7f3d8a600640 1 --2- 172.21.15.131:0/2289578116 >> v2:172.21.15.187:3300/0 conn(0x7f3d840af580 0x7f3d8415f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:27.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.663+0000 7f3d8a600640 1 -- 172.21.15.131:0/2289578116 >> 172.21.15.131:0/2289578116 conn(0x7f3d84072b00 msgr2=0x7f3d8406de50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:27.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.663+0000 7f3d8a600640 1 -- 172.21.15.131:0/2289578116 shutdown_connections 2024-05-26T22:49:27.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.663+0000 7f3d8a600640 1 -- 172.21.15.131:0/2289578116 wait complete. 2024-05-26T22:49:27.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.665+0000 7f3d8a600640 1 Processor -- start 2024-05-26T22:49:27.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.665+0000 7f3d8a600640 1 -- start start 2024-05-26T22:49:27.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.666+0000 7f3d8a600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3d8406b680 0x7f3d840b1f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:27.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.666+0000 7f3d8a600640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3d840af580 0x7f3d8414d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:27.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.666+0000 7f3d8a600640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3d840af960 0x7f3d84149050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:27.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.666+0000 7f3d8a600640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f3d840b6a40 con 0x7f3d8406b680 2024-05-26T22:49:27.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.666+0000 7f3d8a600640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3d840b68c0 con 0x7f3d840af580 2024-05-26T22:49:27.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.666+0000 7f3d8a600640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3d8406c050 con 0x7f3d840af960 2024-05-26T22:49:27.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.666+0000 7f3d83400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3d8406b680 0x7f3d840b1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:27.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.666+0000 7f3d83400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3d8406b680 0x7f3d840b1f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41774/0 (socket says 172.21.15.131:41774) 2024-05-26T22:49:27.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.666+0000 7f3d83400640 1 -- 172.21.15.131:0/1430690036 learned_addr learned my addr 172.21.15.131:0/1430690036 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:27.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.667+0000 7f3d82a00640 1 --2- 172.21.15.131:0/1430690036 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3d840af580 0x7f3d8414d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:27.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.667+0000 7f3d83400640 1 -- 172.21.15.131:0/1430690036 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3d840af960 msgr2=0x7f3d84149050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:27.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.667+0000 7f3d83400640 1 --2- 172.21.15.131:0/1430690036 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3d840af960 0x7f3d84149050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:27.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.667+0000 7f3d83400640 1 -- 172.21.15.131:0/1430690036 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3d840af580 msgr2=0x7f3d8414d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:27.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.667+0000 7f3d83400640 1 --2- 172.21.15.131:0/1430690036 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3d840af580 0x7f3d8414d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:27.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.667+0000 7f3d83400640 1 -- 172.21.15.131:0/1430690036 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d84147930 con 0x7f3d8406b680 2024-05-26T22:49:27.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.667+0000 7f3d82a00640 1 --2- 172.21.15.131:0/1430690036 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3d840af580 0x7f3d8414d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:49:27.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.667+0000 7f3d83400640 1 --2- 172.21.15.131:0/1430690036 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3d8406b680 0x7f3d840b1f80 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f3d74007ef0 tx=0x7f3d7400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:27.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.667+0000 7f3d7b400640 1 -- 172.21.15.131:0/1430690036 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d74014620 con 0x7f3d8406b680 2024-05-26T22:49:27.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.667+0000 7f3d7b400640 1 -- 172.21.15.131:0/1430690036 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d74019070 con 0x7f3d8406b680 2024-05-26T22:49:27.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.667+0000 7f3d7b400640 1 -- 172.21.15.131:0/1430690036 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d74014a00 con 0x7f3d8406b680 2024-05-26T22:49:27.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.667+0000 7f3d8a600640 1 -- 172.21.15.131:0/1430690036 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d841f8260 con 0x7f3d8406b680 2024-05-26T22:49:27.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.668+0000 7f3d8a600640 1 -- 172.21.15.131:0/1430690036 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d841f8820 con 0x7f3d8406b680 2024-05-26T22:49:27.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.669+0000 7f3d8a600640 1 -- 172.21.15.131:0/1430690036 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d840695c0 con 0x7f3d8406b680 2024-05-26T22:49:27.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.670+0000 7f3d7b400640 1 -- 172.21.15.131:0/1430690036 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3d74014ba0 con 0x7f3d8406b680 2024-05-26T22:49:27.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.670+0000 7f3d7b400640 1 --2- 172.21.15.131:0/1430690036 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f3d6406be30 0x7f3d6406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:27.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.671+0000 7f3d7b400640 1 -- 172.21.15.131:0/1430690036 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f3d7408b440 con 0x7f3d8406b680 2024-05-26T22:49:27.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.671+0000 7f3d82a00640 1 --2- 172.21.15.131:0/1430690036 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f3d6406be30 0x7f3d6406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:27.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.672+0000 7f3d82a00640 1 --2- 172.21.15.131:0/1430690036 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f3d6406be30 0x7f3d6406e2f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3d8413dc90 tx=0x7f3d70009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:27.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.675+0000 7f3d7b400640 1 -- 172.21.15.131:0/1430690036 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f3d7405a140 con 0x7f3d8406b680 2024-05-26T22:49:27.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.875+0000 7f3d8a600640 1 -- 172.21.15.131:0/1430690036 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3d84067320 con 0x7f3d8406b680 2024-05-26T22:49:27.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.876+0000 7f3d7b400640 1 -- 172.21.15.131:0/1430690036 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) ==== 93+0+10 (secure 0 0 0) 0x7f3d74059a40 con 0x7f3d8406b680 2024-05-26T22:49:27.876 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2024-05-26T22:49:27.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.879+0000 7f3d8a600640 1 -- 172.21.15.131:0/1430690036 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f3d6406be30 msgr2=0x7f3d6406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:27.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.879+0000 7f3d8a600640 1 --2- 172.21.15.131:0/1430690036 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f3d6406be30 0x7f3d6406e2f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3d8413dc90 tx=0x7f3d70009040 comp rx=0 tx=0).stop 2024-05-26T22:49:27.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.879+0000 7f3d8a600640 1 -- 172.21.15.131:0/1430690036 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3d8406b680 msgr2=0x7f3d840b1f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:27.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.879+0000 7f3d8a600640 1 --2- 172.21.15.131:0/1430690036 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3d8406b680 0x7f3d840b1f80 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f3d74007ef0 tx=0x7f3d7400e4b0 comp rx=0 tx=0).stop 2024-05-26T22:49:27.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.879+0000 7f3d8a600640 1 -- 172.21.15.131:0/1430690036 shutdown_connections 2024-05-26T22:49:27.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.879+0000 7f3d8a600640 1 --2- 172.21.15.131:0/1430690036 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3d840af960 0x7f3d84149050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:27.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.879+0000 7f3d8a600640 1 --2- 172.21.15.131:0/1430690036 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f3d6406be30 0x7f3d6406e2f0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:27.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.879+0000 7f3d8a600640 1 --2- 172.21.15.131:0/1430690036 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3d840af580 0x7f3d8414d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:27.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.879+0000 7f3d8a600640 1 --2- 172.21.15.131:0/1430690036 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3d8406b680 0x7f3d840b1f80 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:27.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.879+0000 7f3d8a600640 1 -- 172.21.15.131:0/1430690036 >> 172.21.15.131:0/1430690036 conn(0x7f3d84072b00 msgr2=0x7f3d84154220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:27.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.879+0000 7f3d8a600640 1 -- 172.21.15.131:0/1430690036 shutdown_connections 2024-05-26T22:49:27.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:27.879+0000 7f3d8a600640 1 -- 172.21.15.131:0/1430690036 wait complete. 2024-05-26T22:49:27.899 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-26T22:49:28.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.097+0000 7f0ac2600640 1 Processor -- start 2024-05-26T22:49:28.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.097+0000 7f0ac2600640 1 -- start start 2024-05-26T22:49:28.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.099+0000 7f0ac2600640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f0abc158c00 0x7f0abc06d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:28.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.099+0000 7f0ac2600640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0abc0b3e50 con 0x7f0abc0b38a0 2024-05-26T22:49:28.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.099+0000 7f0ac2600640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0abc0b4130 con 0x7f0abc06daf0 2024-05-26T22:49:28.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.099+0000 7f0ac2600640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f0abc06df50 con 0x7f0abc158c00 2024-05-26T22:49:28.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.099+0000 7f0abb400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f0abc158c00 0x7f0abc06d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:28.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.099+0000 7f0abbe00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f0abc0b38a0 0x7f0abc15fc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:38008/0 (socket says 172.21.15.131:38008) 2024-05-26T22:49:28.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.099+0000 7f0abb400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f0abc158c00 0x7f0abc06d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41782/0 (socket says 172.21.15.131:41782) 2024-05-26T22:49:28.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.099+0000 7f0abbe00640 1 -- 172.21.15.131:0/1767213025 learned_addr learned my addr 172.21.15.131:0/1767213025 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:49:28.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.100+0000 7f0abb400640 1 -- 172.21.15.131:0/1767213025 >> v1:172.21.15.187:6789/0 conn(0x7f0abc06daf0 legacy=0x7f0abc0b3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:28.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.100+0000 7f0abb400640 1 -- 172.21.15.131:0/1767213025 >> v1:172.21.15.131:6789/0 conn(0x7f0abc0b38a0 legacy=0x7f0abc15fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:28.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.100+0000 7f0abb400640 1 -- 172.21.15.131:0/1767213025 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0abc0b4410 con 0x7f0abc158c00 2024-05-26T22:49:28.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.101+0000 7f0abb400640 1 --2- 172.21.15.131:0/1767213025 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0abc158c00 0x7f0abc06d5b0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f0aac0097a0 tx=0x7f0aac02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3675db5a0d7abc4 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:28.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.101+0000 7f0aba000640 1 -- 172.21.15.131:0/1767213025 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aac037070 con 0x7f0abc158c00 2024-05-26T22:49:28.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.101+0000 7f0aba000640 1 -- 172.21.15.131:0/1767213025 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0aac02ae50 con 0x7f0abc158c00 2024-05-26T22:49:28.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.101+0000 7f0aba000640 1 -- 172.21.15.131:0/1767213025 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aac032660 con 0x7f0abc158c00 2024-05-26T22:49:28.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.102+0000 7f0ac2600640 1 -- 172.21.15.131:0/1767213025 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0abc158c00 msgr2=0x7f0abc06d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:28.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.102+0000 7f0ac2600640 1 --2- 172.21.15.131:0/1767213025 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0abc158c00 0x7f0abc06d5b0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f0aac0097a0 tx=0x7f0aac02a140 comp rx=0 tx=0).stop 2024-05-26T22:49:28.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.102+0000 7f0ac2600640 1 -- 172.21.15.131:0/1767213025 shutdown_connections 2024-05-26T22:49:28.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.102+0000 7f0ac2600640 1 --2- 172.21.15.131:0/1767213025 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0abc158c00 0x7f0abc06d5b0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:28.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.102+0000 7f0ac2600640 1 -- 172.21.15.131:0/1767213025 >> 172.21.15.131:0/1767213025 conn(0x7f0abc0b1e90 msgr2=0x7f0abc0b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:28.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.102+0000 7f0ac2600640 1 -- 172.21.15.131:0/1767213025 shutdown_connections 2024-05-26T22:49:28.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.102+0000 7f0ac2600640 1 -- 172.21.15.131:0/1767213025 wait complete. 2024-05-26T22:49:28.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.103+0000 7f0ac2600640 1 Processor -- start 2024-05-26T22:49:28.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.104+0000 7f0ac2600640 1 -- start start 2024-05-26T22:49:28.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.104+0000 7f0ac2600640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0abc06daf0 0x7f0abc0998c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:28.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.104+0000 7f0ac2600640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0abc0b38a0 0x7f0abc146e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:28.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.105+0000 7f0ac2600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0abc158c00 0x7f0abc1473b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:28.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.105+0000 7f0ac2600640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f0abc160360 con 0x7f0abc158c00 2024-05-26T22:49:28.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.105+0000 7f0ac2600640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0abc0b49d0 con 0x7f0abc06daf0 2024-05-26T22:49:28.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.105+0000 7f0ac2600640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f0abc1604e0 con 0x7f0abc0b38a0 2024-05-26T22:49:28.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.105+0000 7f0abbe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0abc158c00 0x7f0abc1473b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:28.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.105+0000 7f0abbe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0abc158c00 0x7f0abc1473b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41794/0 (socket says 172.21.15.131:41794) 2024-05-26T22:49:28.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.105+0000 7f0abbe00640 1 -- 172.21.15.131:0/3502918774 learned_addr learned my addr 172.21.15.131:0/3502918774 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:28.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.105+0000 7f0abbe00640 1 -- 172.21.15.131:0/3502918774 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0abc0b38a0 msgr2=0x7f0abc146e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:28.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.105+0000 7f0abaa00640 1 --2- 172.21.15.131:0/3502918774 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0abc0b38a0 0x7f0abc146e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:28.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.105+0000 7f0abbe00640 1 --2- 172.21.15.131:0/3502918774 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0abc0b38a0 0x7f0abc146e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:28.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.105+0000 7f0abbe00640 1 -- 172.21.15.131:0/3502918774 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0abc06daf0 msgr2=0x7f0abc0998c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:28.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.105+0000 7f0abbe00640 1 --2- 172.21.15.131:0/3502918774 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0abc06daf0 0x7f0abc0998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:28.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.105+0000 7f0abbe00640 1 -- 172.21.15.131:0/3502918774 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0abc15d830 con 0x7f0abc158c00 2024-05-26T22:49:28.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.106+0000 7f0abbe00640 1 --2- 172.21.15.131:0/3502918774 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0abc158c00 0x7f0abc1473b0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f0ab400cbc0 tx=0x7f0ab400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:28.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.106+0000 7f0ab3400640 1 -- 172.21.15.131:0/3502918774 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab4017070 con 0x7f0abc158c00 2024-05-26T22:49:28.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.106+0000 7f0ab3400640 1 -- 172.21.15.131:0/3502918774 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ab400ad40 con 0x7f0abc158c00 2024-05-26T22:49:28.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.106+0000 7f0ab3400640 1 -- 172.21.15.131:0/3502918774 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab4012600 con 0x7f0abc158c00 2024-05-26T22:49:28.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.106+0000 7f0ac2600640 1 -- 172.21.15.131:0/3502918774 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0abc15db10 con 0x7f0abc158c00 2024-05-26T22:49:28.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.106+0000 7f0ac2600640 1 -- 172.21.15.131:0/3502918774 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0abc15e0d0 con 0x7f0abc158c00 2024-05-26T22:49:28.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.107+0000 7f0ac2600640 1 -- 172.21.15.131:0/3502918774 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0abc0694e0 con 0x7f0abc158c00 2024-05-26T22:49:28.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.108+0000 7f0ab3400640 1 -- 172.21.15.131:0/3502918774 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0ab401a070 con 0x7f0abc158c00 2024-05-26T22:49:28.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.109+0000 7f0ab3400640 1 --2- 172.21.15.131:0/3502918774 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f0a9806be30 0x7f0a9806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:28.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.109+0000 7f0ab3400640 1 -- 172.21.15.131:0/3502918774 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f0ab408a800 con 0x7f0abc158c00 2024-05-26T22:49:28.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.109+0000 7f0abb400640 1 --2- 172.21.15.131:0/3502918774 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f0a9806be30 0x7f0a9806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:28.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.110+0000 7f0abb400640 1 --2- 172.21.15.131:0/3502918774 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f0a9806be30 0x7f0a9806e2f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0aac034000 tx=0x7f0aac02c000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:28.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.113+0000 7f0ab3400640 1 -- 172.21.15.131:0/3502918774 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f0ab4058eb0 con 0x7f0abc158c00 2024-05-26T22:49:28.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.315+0000 7f0ac2600640 1 -- 172.21.15.131:0/3502918774 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f0abc160d90 con 0x7f0abc158c00 2024-05-26T22:49:28.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.316+0000 7f0ab3400640 1 -- 172.21.15.131:0/3502918774 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v29) ==== 84+0+65 (secure 0 0 0) 0x7f0ab405d1b0 con 0x7f0abc158c00 2024-05-26T22:49:28.316 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:49:28.316 INFO:teuthology.orchestra.run.smithi131.stdout:[ 2024-05-26T22:49:28.316 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:49:28.316 INFO:teuthology.orchestra.run.smithi131.stdout: "poolnum": 1, 2024-05-26T22:49:28.316 INFO:teuthology.orchestra.run.smithi131.stdout: "poolname": ".mgr" 2024-05-26T22:49:28.316 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:49:28.316 INFO:teuthology.orchestra.run.smithi131.stdout:] 2024-05-26T22:49:28.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.319+0000 7f0ac2600640 1 -- 172.21.15.131:0/3502918774 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f0a9806be30 msgr2=0x7f0a9806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:28.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.319+0000 7f0ac2600640 1 --2- 172.21.15.131:0/3502918774 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f0a9806be30 0x7f0a9806e2f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0aac034000 tx=0x7f0aac02c000 comp rx=0 tx=0).stop 2024-05-26T22:49:28.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.319+0000 7f0ac2600640 1 -- 172.21.15.131:0/3502918774 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0abc158c00 msgr2=0x7f0abc1473b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:28.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.319+0000 7f0ac2600640 1 --2- 172.21.15.131:0/3502918774 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0abc158c00 0x7f0abc1473b0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f0ab400cbc0 tx=0x7f0ab400a430 comp rx=0 tx=0).stop 2024-05-26T22:49:28.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.319+0000 7f0ac2600640 1 -- 172.21.15.131:0/3502918774 shutdown_connections 2024-05-26T22:49:28.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.319+0000 7f0ac2600640 1 --2- 172.21.15.131:0/3502918774 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0abc0b38a0 0x7f0abc146e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:28.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.319+0000 7f0ac2600640 1 --2- 172.21.15.131:0/3502918774 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f0a9806be30 0x7f0a9806e2f0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:28.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.319+0000 7f0ac2600640 1 --2- 172.21.15.131:0/3502918774 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0abc158c00 0x7f0abc1473b0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:28.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.319+0000 7f0ac2600640 1 --2- 172.21.15.131:0/3502918774 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0abc06daf0 0x7f0abc0998c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:28.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.319+0000 7f0ac2600640 1 -- 172.21.15.131:0/3502918774 >> 172.21.15.131:0/3502918774 conn(0x7f0abc0b1e90 msgr2=0x7f0abc0b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:28.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.320+0000 7f0ac2600640 1 -- 172.21.15.131:0/3502918774 shutdown_connections 2024-05-26T22:49:28.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.320+0000 7f0ac2600640 1 -- 172.21.15.131:0/3502918774 wait complete. 2024-05-26T22:49:28.337 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-26T22:49:28.337 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-26T22:49:28.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.545+0000 7f3ff0000640 1 Processor -- start 2024-05-26T22:49:28.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.546+0000 7f3ff0000640 1 -- start start 2024-05-26T22:49:28.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.547+0000 7f3ff0000640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f3fe815f610 0x7f3fe806e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:28.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.547+0000 7f3ff0000640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3fe80b5e60 con 0x7f3fe8147e50 2024-05-26T22:49:28.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.547+0000 7f3ff0000640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3fe80b6140 con 0x7f3fe806e970 2024-05-26T22:49:28.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.547+0000 7f3ff0000640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f3fe8072400 con 0x7f3fe815f610 2024-05-26T22:49:28.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.547+0000 7f3fed600640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f3fe8147e50 0x7f3fe80b3f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:38024/0 (socket says 172.21.15.131:38024) 2024-05-26T22:49:28.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.547+0000 7f3fed600640 1 -- 172.21.15.131:0/791724138 learned_addr learned my addr 172.21.15.131:0/791724138 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:49:28.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.548+0000 7f3fe7e00640 1 -- 172.21.15.131:0/791724138 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2533492214 0 0) 0x7f3fe80b5e60 con 0x7f3fe8147e50 2024-05-26T22:49:28.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.548+0000 7f3fe7e00640 1 -- 172.21.15.131:0/791724138 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3fd0003520 con 0x7f3fe8147e50 2024-05-26T22:49:28.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.548+0000 7f3fecc00640 1 --2- 172.21.15.131:0/791724138 >> v2:172.21.15.187:3300/0 conn(0x7f3fe815f610 0x7f3fe806e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:28.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.548+0000 7f3fe7e00640 1 -- 172.21.15.131:0/791724138 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 16103804 0 0) 0x7f3fd0003520 con 0x7f3fe8147e50 2024-05-26T22:49:28.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.548+0000 7f3fe7e00640 1 -- 172.21.15.131:0/791724138 >> v2:172.21.15.187:3300/0 conn(0x7f3fe815f610 msgr2=0x7f3fe806e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:28.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.548+0000 7f3fe7e00640 1 --2- 172.21.15.131:0/791724138 >> v2:172.21.15.187:3300/0 conn(0x7f3fe815f610 0x7f3fe806e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:28.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.548+0000 7f3fe7e00640 1 -- 172.21.15.131:0/791724138 >> v1:172.21.15.187:6789/0 conn(0x7f3fe806e970 legacy=0x7f3fe806b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:28.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.548+0000 7f3fe7e00640 1 -- 172.21.15.131:0/791724138 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fe80726e0 con 0x7f3fe8147e50 2024-05-26T22:49:28.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.548+0000 7f3fe7e00640 1 -- 172.21.15.131:0/791724138 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7f3fdc001670 con 0x7f3fe8147e50 2024-05-26T22:49:28.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.548+0000 7f3fe7e00640 1 -- 172.21.15.131:0/791724138 >> v1:172.21.15.131:6789/0 conn(0x7f3fe8147e50 legacy=0x7f3fe80b3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:28.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.548+0000 7f3fe7e00640 1 --2- 172.21.15.131:0/791724138 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3fd0003a00 0x7f3fd0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:28.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.548+0000 7f3fe7e00640 1 --2- 172.21.15.131:0/791724138 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3fd0004800 0x7f3fd0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:28.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.548+0000 7f3fe7e00640 1 --2- 172.21.15.131:0/791724138 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3fd00051c0 0x7f3fd000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:28.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.548+0000 7f3fe7e00640 1 -- 172.21.15.131:0/791724138 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f3fe8072400 con 0x7f3fd0003a00 2024-05-26T22:49:28.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.548+0000 7f3fe7e00640 1 -- 172.21.15.131:0/791724138 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3fd000c0c0 con 0x7f3fd0004800 2024-05-26T22:49:28.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.548+0000 7f3fe7e00640 1 -- 172.21.15.131:0/791724138 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3fd000c2c0 con 0x7f3fd00051c0 2024-05-26T22:49:28.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.548+0000 7f3fed600640 1 --2- 172.21.15.131:0/791724138 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3fd0003a00 0x7f3fd0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:28.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.549+0000 7f3fecc00640 1 --2- 172.21.15.131:0/791724138 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3fd0004800 0x7f3fd0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:28.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.549+0000 7f3fed600640 1 -- 172.21.15.131:0/791724138 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3fd00051c0 msgr2=0x7f3fd000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:28.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.549+0000 7f3fed600640 1 --2- 172.21.15.131:0/791724138 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3fd00051c0 0x7f3fd000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:28.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.549+0000 7f3fed600640 1 -- 172.21.15.131:0/791724138 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3fd0004800 msgr2=0x7f3fd0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:28.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.549+0000 7f3fed600640 1 --2- 172.21.15.131:0/791724138 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3fd0004800 0x7f3fd0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:28.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.549+0000 7f3fed600640 1 -- 172.21.15.131:0/791724138 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fd0003520 con 0x7f3fd0003a00 2024-05-26T22:49:28.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.549+0000 7f3fee000640 1 -- 172.21.15.131:0/791724138 reap_dead start 2024-05-26T22:49:28.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.550+0000 7f3fed600640 1 --2- 172.21.15.131:0/791724138 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3fd0003a00 0x7f3fd0003e20 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc00c190 tx=0x7f3fd0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc667077e12dbfb9 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:28.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.550+0000 7f3fe7e00640 1 -- 172.21.15.131:0/791724138 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fdc007c60 con 0x7f3fd0003a00 2024-05-26T22:49:28.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.550+0000 7f3fe7e00640 1 -- 172.21.15.131:0/791724138 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3fdc005af0 con 0x7f3fd0003a00 2024-05-26T22:49:28.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.550+0000 7f3fe7e00640 1 -- 172.21.15.131:0/791724138 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fdc007070 con 0x7f3fd0003a00 2024-05-26T22:49:28.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.551+0000 7f3ff0000640 1 -- 172.21.15.131:0/791724138 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3fd0003a00 msgr2=0x7f3fd0003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:28.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.551+0000 7f3ff0000640 1 --2- 172.21.15.131:0/791724138 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3fd0003a00 0x7f3fd0003e20 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc00c190 tx=0x7f3fd0005830 comp rx=0 tx=0).stop 2024-05-26T22:49:28.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.551+0000 7f3ff0000640 1 -- 172.21.15.131:0/791724138 shutdown_connections 2024-05-26T22:49:28.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.551+0000 7f3ff0000640 1 --2- 172.21.15.131:0/791724138 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3fd0003a00 0x7f3fd0003e20 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:28.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.551+0000 7f3ff0000640 1 -- 172.21.15.131:0/791724138 >> 172.21.15.131:0/791724138 conn(0x7f3fe80b2c90 msgr2=0x7f3fe80b3070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:28.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.551+0000 7f3ff0000640 1 -- 172.21.15.131:0/791724138 shutdown_connections 2024-05-26T22:49:28.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.551+0000 7f3ff0000640 1 -- 172.21.15.131:0/791724138 wait complete. 2024-05-26T22:49:28.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.553+0000 7f3ff0000640 1 Processor -- start 2024-05-26T22:49:28.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.553+0000 7f3ff0000640 1 -- start start 2024-05-26T22:49:28.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.554+0000 7f3ff0000640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3fd0003a00 0x7f3fe81e7f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:28.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.554+0000 7f3ff0000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3fe80cfe80 0x7f3fe81e7670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:28.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.554+0000 7f3ff0000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3fe8145880 0x7f3fe8145d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:28.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.554+0000 7f3ff0000640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f3fe806e5b0 con 0x7f3fe8145880 2024-05-26T22:49:28.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.554+0000 7f3ff0000640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3fe806e430 con 0x7f3fd0003a00 2024-05-26T22:49:28.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.554+0000 7f3ff0000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3fe806e730 con 0x7f3fe80cfe80 2024-05-26T22:49:28.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.554+0000 7f3fee000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3fe8145880 0x7f3fe8145d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:28.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.554+0000 7f3fee000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3fe8145880 0x7f3fe8145d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41802/0 (socket says 172.21.15.131:41802) 2024-05-26T22:49:28.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.554+0000 7f3fee000640 1 -- 172.21.15.131:0/1203032721 learned_addr learned my addr 172.21.15.131:0/1203032721 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:28.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.554+0000 7f3fed600640 1 --2- 172.21.15.131:0/1203032721 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3fd0003a00 0x7f3fe81e7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:28.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.554+0000 7f3fecc00640 1 --2- 172.21.15.131:0/1203032721 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3fe80cfe80 0x7f3fe81e7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:28.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.555+0000 7f3fee000640 1 -- 172.21.15.131:0/1203032721 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3fe80cfe80 msgr2=0x7f3fe81e7670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:28.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.555+0000 7f3fee000640 1 --2- 172.21.15.131:0/1203032721 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3fe80cfe80 0x7f3fe81e7670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:28.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.555+0000 7f3fee000640 1 -- 172.21.15.131:0/1203032721 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3fd0003a00 msgr2=0x7f3fe81e7f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:28.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.555+0000 7f3fee000640 1 --2- 172.21.15.131:0/1203032721 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3fd0003a00 0x7f3fe81e7f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:28.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.555+0000 7f3fee000640 1 -- 172.21.15.131:0/1203032721 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fe8130d50 con 0x7f3fe8145880 2024-05-26T22:49:28.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.555+0000 7f3fee000640 1 --2- 172.21.15.131:0/1203032721 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3fe8145880 0x7f3fe8145d00 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f3fe809f0e0 tx=0x7f3fe000cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:28.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.555+0000 7f3fe5600640 1 -- 172.21.15.131:0/1203032721 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe0017070 con 0x7f3fe8145880 2024-05-26T22:49:28.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.555+0000 7f3fe5600640 1 -- 172.21.15.131:0/1203032721 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3fe0005d40 con 0x7f3fe8145880 2024-05-26T22:49:28.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.555+0000 7f3ff0000640 1 -- 172.21.15.131:0/1203032721 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fe8131030 con 0x7f3fe8145880 2024-05-26T22:49:28.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.555+0000 7f3fe5600640 1 -- 172.21.15.131:0/1203032721 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe0007310 con 0x7f3fe8145880 2024-05-26T22:49:28.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.555+0000 7f3ff0000640 1 -- 172.21.15.131:0/1203032721 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fe81315f0 con 0x7f3fe8145880 2024-05-26T22:49:28.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.556+0000 7f3ff0000640 1 -- 172.21.15.131:0/1203032721 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fe8068440 con 0x7f3fe8145880 2024-05-26T22:49:28.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.557+0000 7f3fe5600640 1 -- 172.21.15.131:0/1203032721 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3fe001a070 con 0x7f3fe8145880 2024-05-26T22:49:28.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.558+0000 7f3fe5600640 1 --2- 172.21.15.131:0/1203032721 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f3fcc06be30 0x7f3fcc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:28.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.558+0000 7f3fe5600640 1 -- 172.21.15.131:0/1203032721 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f3fe0089d90 con 0x7f3fe8145880 2024-05-26T22:49:28.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.558+0000 7f3fed600640 1 --2- 172.21.15.131:0/1203032721 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f3fcc06be30 0x7f3fcc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:28.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.559+0000 7f3fed600640 1 --2- 172.21.15.131:0/1203032721 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f3fcc06be30 0x7f3fcc06e2f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc00c160 tx=0x7f3fdc009030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:28.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.563+0000 7f3fe5600640 1 -- 172.21.15.131:0/1203032721 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f3fe0058a00 con 0x7f3fe8145880 2024-05-26T22:49:28.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:28.762+0000 7f3ff0000640 1 -- 172.21.15.131:0/1203032721 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f3fe815ea30 con 0x7f3fe8145880 2024-05-26T22:49:29.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:29.487+0000 7f3fe5600640 1 -- 172.21.15.131:0/1203032721 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v30) ==== 138+0+0 (secure 0 0 0) 0x7f3fe0058300 con 0x7f3fe8145880 2024-05-26T22:49:29.488 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_metadata' created 2024-05-26T22:49:29.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:29.491+0000 7f3ff0000640 1 -- 172.21.15.131:0/1203032721 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f3fcc06be30 msgr2=0x7f3fcc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:29.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:29.491+0000 7f3ff0000640 1 --2- 172.21.15.131:0/1203032721 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f3fcc06be30 0x7f3fcc06e2f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc00c160 tx=0x7f3fdc009030 comp rx=0 tx=0).stop 2024-05-26T22:49:29.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:29.491+0000 7f3ff0000640 1 -- 172.21.15.131:0/1203032721 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3fe8145880 msgr2=0x7f3fe8145d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:29.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:29.491+0000 7f3ff0000640 1 --2- 172.21.15.131:0/1203032721 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3fe8145880 0x7f3fe8145d00 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f3fe809f0e0 tx=0x7f3fe000cb90 comp rx=0 tx=0).stop 2024-05-26T22:49:29.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:29.491+0000 7f3ff0000640 1 -- 172.21.15.131:0/1203032721 shutdown_connections 2024-05-26T22:49:29.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:29.491+0000 7f3ff0000640 1 --2- 172.21.15.131:0/1203032721 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3fe80cfe80 0x7f3fe81e7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:29.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:29.491+0000 7f3ff0000640 1 --2- 172.21.15.131:0/1203032721 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f3fcc06be30 0x7f3fcc06e2f0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:29.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:29.491+0000 7f3ff0000640 1 --2- 172.21.15.131:0/1203032721 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3fe8145880 0x7f3fe8145d00 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:29.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:29.491+0000 7f3ff0000640 1 --2- 172.21.15.131:0/1203032721 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3fd0003a00 0x7f3fe81e7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:29.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:29.491+0000 7f3ff0000640 1 -- 172.21.15.131:0/1203032721 >> 172.21.15.131:0/1203032721 conn(0x7f3fe80b2c90 msgr2=0x7f3fe806b310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:29.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:29.491+0000 7f3ff0000640 1 -- 172.21.15.131:0/1203032721 shutdown_connections 2024-05-26T22:49:29.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:29.491+0000 7f3ff0000640 1 -- 172.21.15.131:0/1203032721 wait complete. 2024-05-26T22:49:29.516 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-26T22:49:29.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:29.710+0000 7f7179400640 1 Processor -- start 2024-05-26T22:49:29.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:29.710+0000 7f7179400640 1 -- start start 2024-05-26T22:49:29.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:29.712+0000 7f7179400640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f71740b5450 0x7f71740b43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:29.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:29.712+0000 7f7179400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f717406f420 0x7f71740cfd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:29.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:29.712+0000 7f7179400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f71740cf590 0x7f71740d0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:29.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:29.712+0000 7f7179400640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f7174160d90 con 0x7f71740cf590 2024-05-26T22:49:29.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:29.712+0000 7f7179400640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f71740b4900 con 0x7f71740b5450 2024-05-26T22:49:29.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:29.712+0000 7f7179400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f7174160180 con 0x7f717406f420 2024-05-26T22:49:29.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:29.712+0000 7f7173200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f71740cf590 0x7f71740d0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:29.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:29.713+0000 7f7173200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f71740cf590 0x7f71740d0290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41814/0 (socket says 172.21.15.131:41814) 2024-05-26T22:49:29.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:29.713+0000 7f7173200640 1 -- 172.21.15.131:0/2067602553 learned_addr learned my addr 172.21.15.131:0/2067602553 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:29.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:29.713+0000 7f7172800640 1 --2- 172.21.15.131:0/2067602553 >> v2:172.21.15.187:3300/0 conn(0x7f71740b5450 0x7f71740b43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:29.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:29.713+0000 7f7169e00640 1 --2- 172.21.15.131:0/2067602553 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f717406f420 0x7f71740cfd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:29.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:29.713+0000 7f7173200640 1 -- 172.21.15.131:0/2067602553 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f717406f420 msgr2=0x7f71740cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:29.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:29.713+0000 7f7173200640 1 --2- 172.21.15.131:0/2067602553 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f717406f420 0x7f71740cfd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:29.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:29.713+0000 7f7173200640 1 -- 172.21.15.131:0/2067602553 >> v2:172.21.15.187:3300/0 conn(0x7f71740b5450 msgr2=0x7f71740b43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:29.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:29.713+0000 7f7173200640 1 --2- 172.21.15.131:0/2067602553 >> v2:172.21.15.187:3300/0 conn(0x7f71740b5450 0x7f71740b43c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:29.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:29.713+0000 7f7173200640 1 -- 172.21.15.131:0/2067602553 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7174160460 con 0x7f71740cf590 2024-05-26T22:49:29.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:29.714+0000 7f7173200640 1 --2- 172.21.15.131:0/2067602553 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71740cf590 0x7f71740d0290 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f71740d0c40 tx=0x7f716c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc3e206e74915dd8 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:29.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:29.715+0000 7f7171e00640 1 -- 172.21.15.131:0/2067602553 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f716c037070 con 0x7f71740cf590 2024-05-26T22:49:29.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:29.715+0000 7f7171e00640 1 -- 172.21.15.131:0/2067602553 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f716c032440 con 0x7f71740cf590 2024-05-26T22:49:29.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:29.715+0000 7f7171e00640 1 -- 172.21.15.131:0/2067602553 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f716c032810 con 0x7f71740cf590 2024-05-26T22:49:29.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:29.715+0000 7f7179400640 1 -- 172.21.15.131:0/2067602553 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71740cf590 msgr2=0x7f71740d0290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:29.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:29.715+0000 7f7179400640 1 --2- 172.21.15.131:0/2067602553 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71740cf590 0x7f71740d0290 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f71740d0c40 tx=0x7f716c02a1e0 comp rx=0 tx=0).stop 2024-05-26T22:49:29.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:29.716+0000 7f7179400640 1 -- 172.21.15.131:0/2067602553 shutdown_connections 2024-05-26T22:49:29.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:29.716+0000 7f7179400640 1 --2- 172.21.15.131:0/2067602553 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71740cf590 0x7f71740d0290 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:29.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:29.716+0000 7f7179400640 1 --2- 172.21.15.131:0/2067602553 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f717406f420 0x7f71740cfd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:29.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:29.716+0000 7f7179400640 1 --2- 172.21.15.131:0/2067602553 >> v2:172.21.15.187:3300/0 conn(0x7f71740b5450 0x7f71740b43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:29.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:29.716+0000 7f7179400640 1 -- 172.21.15.131:0/2067602553 >> 172.21.15.131:0/2067602553 conn(0x7f717406bc90 msgr2=0x7f71740b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:29.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:29.716+0000 7f7179400640 1 -- 172.21.15.131:0/2067602553 shutdown_connections 2024-05-26T22:49:29.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:29.716+0000 7f7179400640 1 -- 172.21.15.131:0/2067602553 wait complete. 2024-05-26T22:49:29.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:29.717+0000 7f7179400640 1 Processor -- start 2024-05-26T22:49:29.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:29.718+0000 7f7179400640 1 -- start start 2024-05-26T22:49:29.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:29.718+0000 7f7179400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f717406f420 0x7f7174155b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:29.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:29.718+0000 7f7179400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71740b5450 0x7f71740cd7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:29.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:29.718+0000 7f7179400640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f71740cf590 0x7f71740cdd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:29.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:29.718+0000 7f7179400640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f71740b5830 con 0x7f71740b5450 2024-05-26T22:49:29.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:29.718+0000 7f7179400640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7174160a20 con 0x7f71740cf590 2024-05-26T22:49:29.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:29.718+0000 7f7179400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f717415fde0 con 0x7f717406f420 2024-05-26T22:49:29.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:29.719+0000 7f7169e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71740b5450 0x7f71740cd7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:29.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:29.719+0000 7f7169e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71740b5450 0x7f71740cd7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41824/0 (socket says 172.21.15.131:41824) 2024-05-26T22:49:29.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:29.719+0000 7f7169e00640 1 -- 172.21.15.131:0/3596573506 learned_addr learned my addr 172.21.15.131:0/3596573506 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:29.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:29.719+0000 7f7172800640 1 --2- 172.21.15.131:0/3596573506 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f717406f420 0x7f7174155b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:29.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:29.719+0000 7f7173200640 1 --2- 172.21.15.131:0/3596573506 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f71740cf590 0x7f71740cdd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:29.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:29.719+0000 7f7169e00640 1 -- 172.21.15.131:0/3596573506 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f717406f420 msgr2=0x7f7174155b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:29.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:29.719+0000 7f7169e00640 1 --2- 172.21.15.131:0/3596573506 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f717406f420 0x7f7174155b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:29.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:29.719+0000 7f7169e00640 1 -- 172.21.15.131:0/3596573506 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f71740cf590 msgr2=0x7f71740cdd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:29.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:29.719+0000 7f7169e00640 1 --2- 172.21.15.131:0/3596573506 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f71740cf590 0x7f71740cdd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:29.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:29.719+0000 7f7169e00640 1 -- 172.21.15.131:0/3596573506 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71741573a0 con 0x7f71740b5450 2024-05-26T22:49:29.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:29.719+0000 7f7173200640 1 --2- 172.21.15.131:0/3596573506 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f71740cf590 0x7f71740cdd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:49:29.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:29.720+0000 7f7169e00640 1 --2- 172.21.15.131:0/3596573506 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71740b5450 0x7f71740cd7d0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f715c00d800 tx=0x7f715c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:29.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:29.720+0000 7f716b400640 1 -- 172.21.15.131:0/3596573506 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f715c018030 con 0x7f71740b5450 2024-05-26T22:49:29.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:29.720+0000 7f716b400640 1 -- 172.21.15.131:0/3596573506 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f715c0096a0 con 0x7f71740b5450 2024-05-26T22:49:29.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:29.720+0000 7f716b400640 1 -- 172.21.15.131:0/3596573506 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f715c009870 con 0x7f71740b5450 2024-05-26T22:49:29.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:29.720+0000 7f7179400640 1 -- 172.21.15.131:0/3596573506 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7174157680 con 0x7f71740b5450 2024-05-26T22:49:29.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:29.720+0000 7f7179400640 1 -- 172.21.15.131:0/3596573506 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7174157c40 con 0x7f71740b5450 2024-05-26T22:49:29.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:29.721+0000 7f7179400640 1 -- 172.21.15.131:0/3596573506 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71740695c0 con 0x7f71740b5450 2024-05-26T22:49:29.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:29.722+0000 7f716b400640 1 -- 172.21.15.131:0/3596573506 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f715c01b070 con 0x7f71740b5450 2024-05-26T22:49:29.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:29.723+0000 7f716b400640 1 --2- 172.21.15.131:0/3596573506 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f715006be30 0x7f715006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:29.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:29.723+0000 7f716b400640 1 -- 172.21.15.131:0/3596573506 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5227+0+0 (secure 0 0 0) 0x7f715c08aff0 con 0x7f71740b5450 2024-05-26T22:49:29.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:29.723+0000 7f7172800640 1 --2- 172.21.15.131:0/3596573506 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f715006be30 0x7f715006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:29.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:29.724+0000 7f7172800640 1 --2- 172.21.15.131:0/3596573506 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f715006be30 0x7f715006e2f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f71640099c0 tx=0x7f7164003990 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:29.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:29.728+0000 7f716b400640 1 -- 172.21.15.131:0/3596573506 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f715c059b00 con 0x7f71740b5450 2024-05-26T22:49:29.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:29.932+0000 7f7179400640 1 -- 172.21.15.131:0/3596573506 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f717415bfb0 con 0x7f71740b5450 2024-05-26T22:49:30.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:30.493+0000 7f716b400640 1 -- 172.21.15.131:0/3596573506 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v31) ==== 170+0+0 (secure 0 0 0) 0x7f715c059400 con 0x7f71740b5450 2024-05-26T22:49:30.494 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_data' created 2024-05-26T22:49:30.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:30.497+0000 7f7179400640 1 -- 172.21.15.131:0/3596573506 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f715006be30 msgr2=0x7f715006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:30.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:30.497+0000 7f7179400640 1 --2- 172.21.15.131:0/3596573506 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f715006be30 0x7f715006e2f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f71640099c0 tx=0x7f7164003990 comp rx=0 tx=0).stop 2024-05-26T22:49:30.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:30.498+0000 7f7179400640 1 -- 172.21.15.131:0/3596573506 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71740b5450 msgr2=0x7f71740cd7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:30.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:30.498+0000 7f7179400640 1 --2- 172.21.15.131:0/3596573506 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71740b5450 0x7f71740cd7d0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f715c00d800 tx=0x7f715c00dcc0 comp rx=0 tx=0).stop 2024-05-26T22:49:30.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:30.498+0000 7f7179400640 1 -- 172.21.15.131:0/3596573506 shutdown_connections 2024-05-26T22:49:30.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:30.498+0000 7f7179400640 1 --2- 172.21.15.131:0/3596573506 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f715006be30 0x7f715006e2f0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:30.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:30.498+0000 7f7179400640 1 --2- 172.21.15.131:0/3596573506 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f71740cf590 0x7f71740cdd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:30.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:30.498+0000 7f7179400640 1 --2- 172.21.15.131:0/3596573506 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71740b5450 0x7f71740cd7d0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:30.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:30.498+0000 7f7179400640 1 --2- 172.21.15.131:0/3596573506 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f717406f420 0x7f7174155b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:30.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:30.498+0000 7f7179400640 1 -- 172.21.15.131:0/3596573506 >> 172.21.15.131:0/3596573506 conn(0x7f717406bc90 msgr2=0x7f7174071360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:30.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:30.498+0000 7f7179400640 1 -- 172.21.15.131:0/3596573506 shutdown_connections 2024-05-26T22:49:30.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:30.499+0000 7f7179400640 1 -- 172.21.15.131:0/3596573506 wait complete. 2024-05-26T22:49:30.534 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-26T22:49:30.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:30.709+0000 7f6537400640 1 Processor -- start 2024-05-26T22:49:30.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:30.709+0000 7f6537400640 1 -- start start 2024-05-26T22:49:30.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:30.711+0000 7f6537400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6530155770 0x7f653015b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:30.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:30.711+0000 7f6537400640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f653015c1c0 con 0x7f653015bc40 2024-05-26T22:49:30.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:30.711+0000 7f6537400640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f653015c4a0 con 0x7f653006e440 2024-05-26T22:49:30.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:30.711+0000 7f6537400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f6530071da0 con 0x7f6530155770 2024-05-26T22:49:30.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:30.711+0000 7f6527e00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f653015bc40 0x7f653006dd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:38034/0 (socket says 172.21.15.131:38034) 2024-05-26T22:49:30.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:30.711+0000 7f6527e00640 1 -- 172.21.15.131:0/4123714413 learned_addr learned my addr 172.21.15.131:0/4123714413 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:49:30.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:30.712+0000 7f6534a00640 1 --2- 172.21.15.131:0/4123714413 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6530155770 0x7f653015b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:30.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:30.712+0000 7f652fe00640 1 -- 172.21.15.131:0/4123714413 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4229013713 0 0) 0x7f653015c1c0 con 0x7f653015bc40 2024-05-26T22:49:30.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:30.712+0000 7f652fe00640 1 -- 172.21.15.131:0/4123714413 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6514003520 con 0x7f653015bc40 2024-05-26T22:49:30.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:30.712+0000 7f652fe00640 1 -- 172.21.15.131:0/4123714413 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1726117745 0 0) 0x7f6514003520 con 0x7f653015bc40 2024-05-26T22:49:30.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:30.712+0000 7f652fe00640 1 -- 172.21.15.131:0/4123714413 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6530155770 msgr2=0x7f653015b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:30.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:30.712+0000 7f652fe00640 1 --2- 172.21.15.131:0/4123714413 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6530155770 0x7f653015b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:30.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:30.712+0000 7f652fe00640 1 -- 172.21.15.131:0/4123714413 >> v1:172.21.15.187:6789/0 conn(0x7f653006e440 legacy=0x7f65300725a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:30.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:30.712+0000 7f652fe00640 1 -- 172.21.15.131:0/4123714413 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6530072080 con 0x7f653015bc40 2024-05-26T22:49:30.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:30.712+0000 7f652fe00640 1 -- 172.21.15.131:0/4123714413 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7f6518001670 con 0x7f653015bc40 2024-05-26T22:49:30.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:30.712+0000 7f652fe00640 1 -- 172.21.15.131:0/4123714413 >> v1:172.21.15.131:6789/0 conn(0x7f653015bc40 legacy=0x7f653006dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:30.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:30.712+0000 7f652fe00640 1 --2- 172.21.15.131:0/4123714413 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6530155770 0x7f65140039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:30.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:30.712+0000 7f652fe00640 1 --2- 172.21.15.131:0/4123714413 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6514004010 0x7f6514004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:30.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:30.712+0000 7f652fe00640 1 --2- 172.21.15.131:0/4123714413 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6514004d40 0x7f65140051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:30.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:30.712+0000 7f652fe00640 1 -- 172.21.15.131:0/4123714413 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f6530071da0 con 0x7f6514004d40 2024-05-26T22:49:30.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:30.712+0000 7f652fe00640 1 -- 172.21.15.131:0/4123714413 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f651400bc10 con 0x7f6514004010 2024-05-26T22:49:30.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:30.712+0000 7f652fe00640 1 -- 172.21.15.131:0/4123714413 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f651400be10 con 0x7f6530155770 2024-05-26T22:49:30.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:30.713+0000 7f6535400640 1 --2- 172.21.15.131:0/4123714413 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6514004d40 0x7f65140051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:30.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:30.713+0000 7f6534a00640 1 --2- 172.21.15.131:0/4123714413 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6530155770 0x7f65140039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:30.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:30.713+0000 7f6527e00640 1 --2- 172.21.15.131:0/4123714413 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6514004010 0x7f6514004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:30.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:30.713+0000 7f6535400640 1 -- 172.21.15.131:0/4123714413 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6530155770 msgr2=0x7f65140039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:30.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:30.713+0000 7f6535400640 1 --2- 172.21.15.131:0/4123714413 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6530155770 0x7f65140039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:30.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:30.713+0000 7f6535400640 1 -- 172.21.15.131:0/4123714413 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6514004010 msgr2=0x7f6514004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:30.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:30.713+0000 7f6535400640 1 --2- 172.21.15.131:0/4123714413 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6514004010 0x7f6514004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:30.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:30.713+0000 7f6535400640 1 -- 172.21.15.131:0/4123714413 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6514003520 con 0x7f6514004d40 2024-05-26T22:49:30.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:30.714+0000 7f6535400640 1 --2- 172.21.15.131:0/4123714413 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6514004d40 0x7f65140051c0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f6528001e50 tx=0x7f6514005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f4c10d13898d549 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:30.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:30.714+0000 7f652fe00640 1 -- 172.21.15.131:0/4123714413 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6528001a70 con 0x7f6514004d40 2024-05-26T22:49:30.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:30.714+0000 7f652fe00640 1 -- 172.21.15.131:0/4123714413 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6528003970 con 0x7f6514004d40 2024-05-26T22:49:30.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:30.714+0000 7f652fe00640 1 -- 172.21.15.131:0/4123714413 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6528003d60 con 0x7f6514004d40 2024-05-26T22:49:30.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:30.715+0000 7f6537400640 1 -- 172.21.15.131:0/4123714413 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6514004d40 msgr2=0x7f65140051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:30.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:30.715+0000 7f6537400640 1 --2- 172.21.15.131:0/4123714413 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6514004d40 0x7f65140051c0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f6528001e50 tx=0x7f6514005830 comp rx=0 tx=0).stop 2024-05-26T22:49:30.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:30.715+0000 7f6535400640 1 -- 172.21.15.131:0/4123714413 reap_dead start 2024-05-26T22:49:30.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:30.715+0000 7f6537400640 1 -- 172.21.15.131:0/4123714413 shutdown_connections 2024-05-26T22:49:30.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:30.715+0000 7f6537400640 1 -- 172.21.15.131:0/4123714413 >> 172.21.15.131:0/4123714413 conn(0x7f653006bc90 msgr2=0x7f65300b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:30.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:30.715+0000 7f6537400640 1 -- 172.21.15.131:0/4123714413 shutdown_connections 2024-05-26T22:49:30.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:30.715+0000 7f6537400640 1 -- 172.21.15.131:0/4123714413 wait complete. 2024-05-26T22:49:30.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:30.717+0000 7f6537400640 1 Processor -- start 2024-05-26T22:49:30.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:30.717+0000 7f6537400640 1 -- start start 2024-05-26T22:49:30.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:30.718+0000 7f6537400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6514004d40 0x7f653015a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:30.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:30.718+0000 7f6537400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f653009aa00 0x7f653009ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:30.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:30.718+0000 7f6537400640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6530133e30 0x7f65301342b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:30.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:30.718+0000 7f6537400640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f653009de00 con 0x7f6514004d40 2024-05-26T22:49:30.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:30.718+0000 7f6537400640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f653015c8c0 con 0x7f6530133e30 2024-05-26T22:49:30.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:30.718+0000 7f6537400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f653009df80 con 0x7f653009aa00 2024-05-26T22:49:30.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:30.718+0000 7f6534a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6514004d40 0x7f653015a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:30.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:30.718+0000 7f6534a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6514004d40 0x7f653015a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41840/0 (socket says 172.21.15.131:41840) 2024-05-26T22:49:30.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:30.718+0000 7f6534a00640 1 -- 172.21.15.131:0/2976262143 learned_addr learned my addr 172.21.15.131:0/2976262143 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:30.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:30.719+0000 7f6527e00640 1 --2- 172.21.15.131:0/2976262143 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f653009aa00 0x7f653009ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:30.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:30.719+0000 7f6534a00640 1 -- 172.21.15.131:0/2976262143 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f653009aa00 msgr2=0x7f653009ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:30.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:30.719+0000 7f6534a00640 1 --2- 172.21.15.131:0/2976262143 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f653009aa00 0x7f653009ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:30.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:30.719+0000 7f6534a00640 1 -- 172.21.15.131:0/2976262143 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6530133e30 msgr2=0x7f65301342b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:30.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:30.719+0000 7f6534a00640 1 --2- 172.21.15.131:0/2976262143 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6530133e30 0x7f65301342b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:30.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:30.719+0000 7f6534a00640 1 -- 172.21.15.131:0/2976262143 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6530134c20 con 0x7f6514004d40 2024-05-26T22:49:30.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:30.719+0000 7f6534a00640 1 --2- 172.21.15.131:0/2976262143 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6514004d40 0x7f653015a800 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f652000f5b0 tx=0x7f652000fa70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:30.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:30.719+0000 7f652d600640 1 -- 172.21.15.131:0/2976262143 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6520019020 con 0x7f6514004d40 2024-05-26T22:49:30.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:30.719+0000 7f652d600640 1 -- 172.21.15.131:0/2976262143 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6520014870 con 0x7f6514004d40 2024-05-26T22:49:30.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:30.719+0000 7f652d600640 1 -- 172.21.15.131:0/2976262143 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6520014c60 con 0x7f6514004d40 2024-05-26T22:49:30.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:30.719+0000 7f6537400640 1 -- 172.21.15.131:0/2976262143 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6530134f00 con 0x7f6514004d40 2024-05-26T22:49:30.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:30.719+0000 7f6537400640 1 -- 172.21.15.131:0/2976262143 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65301354c0 con 0x7f6514004d40 2024-05-26T22:49:30.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:30.720+0000 7f6537400640 1 -- 172.21.15.131:0/2976262143 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6530074e90 con 0x7f6514004d40 2024-05-26T22:49:30.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:30.722+0000 7f652d600640 1 -- 172.21.15.131:0/2976262143 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f652001c020 con 0x7f6514004d40 2024-05-26T22:49:30.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:30.722+0000 7f652d600640 1 --2- 172.21.15.131:0/2976262143 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f650c06be30 0x7f650c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:30.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:30.723+0000 7f652d600640 1 -- 172.21.15.131:0/2976262143 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5609+0+0 (secure 0 0 0) 0x7f652008b620 con 0x7f6514004d40 2024-05-26T22:49:30.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:30.723+0000 7f6527e00640 1 --2- 172.21.15.131:0/2976262143 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f650c06be30 0x7f650c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:30.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:30.724+0000 7f6527e00640 1 --2- 172.21.15.131:0/2976262143 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f650c06be30 0x7f650c06e2f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f653006f6a0 tx=0x7f6518009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:30.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:30.728+0000 7f652d600640 1 -- 172.21.15.131:0/2976262143 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f6520023020 con 0x7f6514004d40 2024-05-26T22:49:30.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:30.952+0000 7f6537400640 1 -- 172.21.15.131:0/2976262143 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f65300acef0 con 0x7f6514004d40 2024-05-26T22:49:31.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:31.532+0000 7f652d600640 1 -- 172.21.15.131:0/2976262143 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v12) ==== 172+0+0 (secure 0 0 0) 0x7f65300acef0 con 0x7f6514004d40 2024-05-26T22:49:31.533 INFO:teuthology.orchestra.run.smithi131.stderr:new fs with metadata pool 4 and data pool 5 2024-05-26T22:49:31.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:31.540+0000 7f6537400640 1 -- 172.21.15.131:0/2976262143 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f650c06be30 msgr2=0x7f650c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:31.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:31.540+0000 7f6537400640 1 --2- 172.21.15.131:0/2976262143 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f650c06be30 0x7f650c06e2f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f653006f6a0 tx=0x7f6518009040 comp rx=0 tx=0).stop 2024-05-26T22:49:31.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:31.540+0000 7f6537400640 1 -- 172.21.15.131:0/2976262143 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6514004d40 msgr2=0x7f653015a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:31.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:31.540+0000 7f6537400640 1 --2- 172.21.15.131:0/2976262143 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6514004d40 0x7f653015a800 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f652000f5b0 tx=0x7f652000fa70 comp rx=0 tx=0).stop 2024-05-26T22:49:31.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:31.541+0000 7f6537400640 1 -- 172.21.15.131:0/2976262143 shutdown_connections 2024-05-26T22:49:31.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:31.541+0000 7f6537400640 1 --2- 172.21.15.131:0/2976262143 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f653009aa00 0x7f653009ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:31.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:31.541+0000 7f6537400640 1 --2- 172.21.15.131:0/2976262143 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f650c06be30 0x7f650c06e2f0 secure :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f653006f6a0 tx=0x7f6518009040 comp rx=0 tx=0).stop 2024-05-26T22:49:31.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:31.541+0000 7f6537400640 1 --2- 172.21.15.131:0/2976262143 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6530133e30 0x7f65301342b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:31.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:31.541+0000 7f6537400640 1 --2- 172.21.15.131:0/2976262143 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6514004d40 0x7f653015a800 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:31.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:31.541+0000 7f6537400640 1 -- 172.21.15.131:0/2976262143 >> 172.21.15.131:0/2976262143 conn(0x7f653006bc90 msgr2=0x7f65300cd9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:31.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:31.542+0000 7f6537400640 1 -- 172.21.15.131:0/2976262143 shutdown_connections 2024-05-26T22:49:31.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:31.542+0000 7f6537400640 1 -- 172.21.15.131:0/2976262143 wait complete. 2024-05-26T22:49:31.575 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-26T22:49:31.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:31.742+0000 7f2584000640 1 Processor -- start 2024-05-26T22:49:31.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:31.742+0000 7f2584000640 1 -- start start 2024-05-26T22:49:31.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:31.744+0000 7f2584000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f257c072f50 0x7f257c0afda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:31.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:31.744+0000 7f2584000640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f257c0b02e0 0x7f257c06e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:31.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:31.744+0000 7f2584000640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f257c0b3f20 con 0x7f257c0d0820 2024-05-26T22:49:31.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:31.744+0000 7f2584000640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f257c0b41c0 con 0x7f257c0b02e0 2024-05-26T22:49:31.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:31.744+0000 7f2584000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f257c0b44a0 con 0x7f257c072f50 2024-05-26T22:49:31.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:31.744+0000 7f2580c00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f257c072f50 0x7f257c0afda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:31.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:31.744+0000 7f2580c00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f257c072f50 0x7f257c0afda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.131:42366/0 (socket says 172.21.15.131:42366) 2024-05-26T22:49:31.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:31.744+0000 7f2580c00640 1 -- 172.21.15.131:0/2357487140 learned_addr learned my addr 172.21.15.131:0/2357487140 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:31.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:31.744+0000 7f2581600640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f257c0d0820 0x7f257c15f490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.131:54868/0 (socket says 172.21.15.131:54868) 2024-05-26T22:49:31.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:31.745+0000 7f2582000640 1 --2- 172.21.15.131:0/2357487140 >> v2:172.21.15.187:3300/0 conn(0x7f257c0b02e0 0x7f257c06e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:31.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:31.745+0000 7f2580c00640 1 -- 172.21.15.131:0/2357487140 >> v2:172.21.15.187:3300/0 conn(0x7f257c0b02e0 msgr2=0x7f257c06e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:31.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:31.745+0000 7f2580c00640 1 --2- 172.21.15.131:0/2357487140 >> v2:172.21.15.187:3300/0 conn(0x7f257c0b02e0 0x7f257c06e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:31.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:31.745+0000 7f2580c00640 1 -- 172.21.15.131:0/2357487140 >> v1:172.21.15.187:6789/0 conn(0x7f257c0d0820 legacy=0x7f257c15f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:31.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:31.745+0000 7f2580c00640 1 -- 172.21.15.131:0/2357487140 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f257c072470 con 0x7f257c072f50 2024-05-26T22:49:31.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:31.745+0000 7f2582000640 1 --2- 172.21.15.131:0/2357487140 >> v2:172.21.15.187:3300/0 conn(0x7f257c0b02e0 0x7f257c06e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-26T22:49:31.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:31.747+0000 7f2580c00640 1 --2- 172.21.15.131:0/2357487140 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f257c072f50 0x7f257c0afda0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2568009870 tx=0x7f256802a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=943d2feca9384376 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:31.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:31.747+0000 7f257be00640 1 -- 172.21.15.131:0/2357487140 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2568037070 con 0x7f257c072f50 2024-05-26T22:49:31.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:31.747+0000 7f257be00640 1 -- 172.21.15.131:0/2357487140 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2568032440 con 0x7f257c072f50 2024-05-26T22:49:31.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:31.747+0000 7f257be00640 1 -- 172.21.15.131:0/2357487140 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2568032810 con 0x7f257c072f50 2024-05-26T22:49:31.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:31.748+0000 7f2584000640 1 -- 172.21.15.131:0/2357487140 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f257c072f50 msgr2=0x7f257c0afda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:31.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:31.748+0000 7f2584000640 1 --2- 172.21.15.131:0/2357487140 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f257c072f50 0x7f257c0afda0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2568009870 tx=0x7f256802a250 comp rx=0 tx=0).stop 2024-05-26T22:49:31.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:31.748+0000 7f2584000640 1 -- 172.21.15.131:0/2357487140 shutdown_connections 2024-05-26T22:49:31.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:31.748+0000 7f2584000640 1 --2- 172.21.15.131:0/2357487140 >> v2:172.21.15.187:3300/0 conn(0x7f257c0b02e0 0x7f257c06e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:31.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:31.748+0000 7f2584000640 1 --2- 172.21.15.131:0/2357487140 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f257c072f50 0x7f257c0afda0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:31.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:31.748+0000 7f2584000640 1 -- 172.21.15.131:0/2357487140 >> 172.21.15.131:0/2357487140 conn(0x7f257c0d0ff0 msgr2=0x7f257c0d1400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:31.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:31.748+0000 7f2584000640 1 -- 172.21.15.131:0/2357487140 shutdown_connections 2024-05-26T22:49:31.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:31.748+0000 7f2584000640 1 -- 172.21.15.131:0/2357487140 wait complete. 2024-05-26T22:49:31.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:31.750+0000 7f2584000640 1 Processor -- start 2024-05-26T22:49:31.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:31.750+0000 7f2584000640 1 -- start start 2024-05-26T22:49:31.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:31.751+0000 7f2584000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f257c072f50 0x7f257c15ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:31.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:31.751+0000 7f2584000640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f257c0b02e0 0x7f257c156ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:31.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:31.751+0000 7f2584000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f257c0d0820 0x7f257c1e78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:31.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:31.751+0000 7f2584000640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f257c0733d0 con 0x7f257c072f50 2024-05-26T22:49:31.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:31.751+0000 7f2584000640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f257c072a30 con 0x7f257c0b02e0 2024-05-26T22:49:31.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:31.751+0000 7f2584000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f257c0b5d20 con 0x7f257c0d0820 2024-05-26T22:49:31.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:31.751+0000 7f2581600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f257c072f50 0x7f257c15ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:31.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:31.751+0000 7f2581600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f257c072f50 0x7f257c15ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41844/0 (socket says 172.21.15.131:41844) 2024-05-26T22:49:31.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:31.751+0000 7f2581600640 1 -- 172.21.15.131:0/1028938361 learned_addr learned my addr 172.21.15.131:0/1028938361 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:31.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:31.751+0000 7f2582000640 1 --2- 172.21.15.131:0/1028938361 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f257c0d0820 0x7f257c1e78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:31.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:31.751+0000 7f2581600640 1 -- 172.21.15.131:0/1028938361 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f257c0d0820 msgr2=0x7f257c1e78a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:31.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:31.751+0000 7f2581600640 1 --2- 172.21.15.131:0/1028938361 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f257c0d0820 0x7f257c1e78a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:31.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:31.751+0000 7f2581600640 1 -- 172.21.15.131:0/1028938361 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f257c0b02e0 msgr2=0x7f257c156ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:31.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:31.751+0000 7f2581600640 1 --2- 172.21.15.131:0/1028938361 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f257c0b02e0 0x7f257c156ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:31.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:31.751+0000 7f2581600640 1 -- 172.21.15.131:0/1028938361 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f257c157430 con 0x7f257c072f50 2024-05-26T22:49:31.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:31.752+0000 7f2581600640 1 --2- 172.21.15.131:0/1028938361 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f257c072f50 0x7f257c15ade0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f257000c400 tx=0x7f257000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:31.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:31.752+0000 7f2579600640 1 -- 172.21.15.131:0/1028938361 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25700074e0 con 0x7f257c072f50 2024-05-26T22:49:31.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:31.752+0000 7f2579600640 1 -- 172.21.15.131:0/1028938361 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2570007680 con 0x7f257c072f50 2024-05-26T22:49:31.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:31.752+0000 7f2579600640 1 -- 172.21.15.131:0/1028938361 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2570004a00 con 0x7f257c072f50 2024-05-26T22:49:31.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:31.752+0000 7f2584000640 1 -- 172.21.15.131:0/1028938361 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f257c15a020 con 0x7f257c072f50 2024-05-26T22:49:31.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:31.752+0000 7f2584000640 1 -- 172.21.15.131:0/1028938361 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f257c15a5e0 con 0x7f257c072f50 2024-05-26T22:49:31.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:31.753+0000 7f2584000640 1 -- 172.21.15.131:0/1028938361 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f257c0694e0 con 0x7f257c072f50 2024-05-26T22:49:31.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:31.754+0000 7f2579600640 1 -- 172.21.15.131:0/1028938361 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f257001a020 con 0x7f257c072f50 2024-05-26T22:49:31.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:31.755+0000 7f2579600640 1 --2- 172.21.15.131:0/1028938361 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f255c06be30 0x7f255c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:31.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:31.755+0000 7f2579600640 1 -- 172.21.15.131:0/1028938361 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f257008a7b0 con 0x7f257c072f50 2024-05-26T22:49:31.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:31.755+0000 7f2580c00640 1 --2- 172.21.15.131:0/1028938361 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f255c06be30 0x7f255c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:31.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:31.756+0000 7f2580c00640 1 --2- 172.21.15.131:0/1028938361 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f255c06be30 0x7f255c06e2f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2568034000 tx=0x7f256802c000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:31.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:31.760+0000 7f2579600640 1 -- 172.21.15.131:0/1028938361 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f2570010300 con 0x7f257c072f50 2024-05-26T22:49:31.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:31.949+0000 7f2584000640 1 -- 172.21.15.131:0/1028938361 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f257c15daf0 con 0x7f257c072f50 2024-05-26T22:49:31.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:31.950+0000 7f2579600640 1 -- 172.21.15.131:0/1028938361 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) ==== 74+0+16706 (secure 0 0 0) 0x7f25700590c0 con 0x7f257c072f50 2024-05-26T22:49:31.950 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:49:31.950 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":32,"fsid":"46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f","created":"2024-05-26T22:47:57.288905+0000","modified":"2024-05-26T22:49:30.954432+0000","last_up_change":"2024-05-26T22:48:09.602207+0000","last_in_change":"2024-05-26T22:48:00.687341+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-26T22:48:10.463339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-05-26T22:49:28.813374+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-05-26T22:49:29.987297+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"814eaf62-a18d-494b-8e27-fd8fb88adea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6801","nonce":2350940051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6803","nonce":2350940051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6807","nonce":2350940051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6805","nonce":2350940051}]},"public_addr":"172.21.15.131:6801/2350940051","cluster_addr":"172.21.15.131:6803/2350940051","heartbeat_back_addr":"172.21.15.131:6807/2350940051","heartbeat_front_addr":"172.21.15.131:6805/2350940051","state":["exists","up"]},{"osd":1,"uuid":"42e66287-f6c6-4e26-af3c-6dfaa3536d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6825","nonce":1525528247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6827","nonce":1525528247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6831","nonce":1525528247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6829","nonce":1525528247}]},"public_addr":"172.21.15.131:6825/1525528247","cluster_addr":"172.21.15.131:6827/1525528247","heartbeat_back_addr":"172.21.15.131:6831/1525528247","heartbeat_front_addr":"172.21.15.131:6829/1525528247","state":["exists","up"]},{"osd":2,"uuid":"0da34673-04a2-4fd0-b57a-0ff2b679b043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6817","nonce":3627842210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6819","nonce":3627842210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6823","nonce":3627842210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6821","nonce":3627842210}]},"public_addr":"172.21.15.131:6817/3627842210","cluster_addr":"172.21.15.131:6819/3627842210","heartbeat_back_addr":"172.21.15.131:6823/3627842210","heartbeat_front_addr":"172.21.15.131:6821/3627842210","state":["exists","up"]},{"osd":3,"uuid":"cf117a27-d6b2-43d3-ae52-b47988bf8eed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6809","nonce":350790591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6811","nonce":350790591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6815","nonce":350790591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6813","nonce":350790591}]},"public_addr":"172.21.15.131:6809/350790591","cluster_addr":"172.21.15.131:6811/350790591","heartbeat_back_addr":"172.21.15.131:6815/350790591","heartbeat_front_addr":"172.21.15.131:6813/350790591","state":["exists","up"]},{"osd":4,"uuid":"a092ccfc-be89-4a84-88d5-0f781fa715eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6816","nonce":244560995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6819","nonce":244560995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6823","nonce":244560995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6821","nonce":244560995}]},"public_addr":"172.21.15.187:6816/244560995","cluster_addr":"172.21.15.187:6819/244560995","heartbeat_back_addr":"172.21.15.187:6823/244560995","heartbeat_front_addr":"172.21.15.187:6821/244560995","state":["exists","up"]},{"osd":5,"uuid":"7513ba69-dd00-4a35-b190-9e8cc3b2bdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6801","nonce":324619398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6803","nonce":324619398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6812","nonce":324619398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6807","nonce":324619398}]},"public_addr":"172.21.15.187:6801/324619398","cluster_addr":"172.21.15.187:6803/324619398","heartbeat_back_addr":"172.21.15.187:6812/324619398","heartbeat_front_addr":"172.21.15.187:6807/324619398","state":["exists","up"]},{"osd":6,"uuid":"50081810-c093-4d2f-a4d7-b7c695c5400d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6806","nonce":1391347786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6809","nonce":1391347786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6818","nonce":1391347786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6813","nonce":1391347786}]},"public_addr":"172.21.15.187:6806/1391347786","cluster_addr":"172.21.15.187:6809/1391347786","heartbeat_back_addr":"172.21.15.187:6818/1391347786","heartbeat_front_addr":"172.21.15.187:6813/1391347786","state":["exists","up"]},{"osd":7,"uuid":"775dae05-54f5-4ad4-bbce-92863c6ed2ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6825","nonce":1220065622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6827","nonce":1220065622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6831","nonce":1220065622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6829","nonce":1220065622}]},"public_addr":"172.21.15.187:6825/1220065622","cluster_addr":"172.21.15.187:6827/1220065622","heartbeat_back_addr":"172.21.15.187:6831/1220065622","heartbeat_front_addr":"172.21.15.187:6829/1220065622","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:05.983520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.299516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.067323+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.307748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.018742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.258775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.010912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.295355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-26T22:49:31.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:31.952+0000 7f2584000640 1 -- 172.21.15.131:0/1028938361 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f255c06be30 msgr2=0x7f255c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:31.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:31.952+0000 7f2584000640 1 --2- 172.21.15.131:0/1028938361 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f255c06be30 0x7f255c06e2f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2568034000 tx=0x7f256802c000 comp rx=0 tx=0).stop 2024-05-26T22:49:31.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:31.953+0000 7f2584000640 1 -- 172.21.15.131:0/1028938361 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f257c072f50 msgr2=0x7f257c15ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:31.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:31.953+0000 7f2584000640 1 --2- 172.21.15.131:0/1028938361 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f257c072f50 0x7f257c15ade0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f257000c400 tx=0x7f257000c8c0 comp rx=0 tx=0).stop 2024-05-26T22:49:31.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:31.953+0000 7f2584000640 1 -- 172.21.15.131:0/1028938361 shutdown_connections 2024-05-26T22:49:31.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:31.953+0000 7f2584000640 1 --2- 172.21.15.131:0/1028938361 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f257c0d0820 0x7f257c1e78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:31.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:31.953+0000 7f2584000640 1 --2- 172.21.15.131:0/1028938361 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f255c06be30 0x7f255c06e2f0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:31.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:31.953+0000 7f2584000640 1 --2- 172.21.15.131:0/1028938361 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f257c0b02e0 0x7f257c156ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:31.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:31.953+0000 7f2584000640 1 --2- 172.21.15.131:0/1028938361 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f257c072f50 0x7f257c15ade0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:31.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:31.953+0000 7f2584000640 1 -- 172.21.15.131:0/1028938361 >> 172.21.15.131:0/1028938361 conn(0x7f257c0d0ff0 msgr2=0x7f257c06b310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:31.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:31.953+0000 7f2584000640 1 -- 172.21.15.131:0/1028938361 shutdown_connections 2024-05-26T22:49:31.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:31.953+0000 7f2584000640 1 -- 172.21.15.131:0/1028938361 wait complete. 2024-05-26T22:49:31.970 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-26T22:49:32.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.162+0000 7f6bf6e00640 1 Processor -- start 2024-05-26T22:49:32.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.162+0000 7f6bf6e00640 1 -- start start 2024-05-26T22:49:32.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.164+0000 7f6bf6e00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f6bf0155840 0x7f6bf0071930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:32.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.164+0000 7f6bf6e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f6bf015c3d0 0x7f6bf006ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:32.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.164+0000 7f6bf6e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6bf006f490 0x7f6bf015ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:32.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.164+0000 7f6bf6e00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f6bf00722a0 con 0x7f6bf015c3d0 2024-05-26T22:49:32.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.164+0000 7f6bf6e00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f6bf0072580 con 0x7f6bf0155840 2024-05-26T22:49:32.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.164+0000 7f6bf6e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f6bf0072860 con 0x7f6bf006f490 2024-05-26T22:49:32.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.164+0000 7f6be7400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f6bf015c3d0 0x7f6bf006ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:32.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.164+0000 7f6be7400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f6bf015c3d0 0x7f6bf006ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41854/0 (socket says 172.21.15.131:41854) 2024-05-26T22:49:32.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.164+0000 7f6be7400640 1 -- 172.21.15.131:0/955268205 learned_addr learned my addr 172.21.15.131:0/955268205 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:32.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.165+0000 7f6befe00640 1 --2- 172.21.15.131:0/955268205 >> v2:172.21.15.187:3300/0 conn(0x7f6bf0155840 0x7f6bf0071930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:32.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.165+0000 7f6bf4e00640 1 --2- 172.21.15.131:0/955268205 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6bf006f490 0x7f6bf015ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:32.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.165+0000 7f6be7400640 1 -- 172.21.15.131:0/955268205 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6bf006f490 msgr2=0x7f6bf015ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:32.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.165+0000 7f6be7400640 1 --2- 172.21.15.131:0/955268205 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6bf006f490 0x7f6bf015ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:32.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.165+0000 7f6be7400640 1 -- 172.21.15.131:0/955268205 >> v2:172.21.15.187:3300/0 conn(0x7f6bf0155840 msgr2=0x7f6bf0071930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:32.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.165+0000 7f6be7400640 1 --2- 172.21.15.131:0/955268205 >> v2:172.21.15.187:3300/0 conn(0x7f6bf0155840 0x7f6bf0071930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:32.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.165+0000 7f6be7400640 1 -- 172.21.15.131:0/955268205 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bf0072b40 con 0x7f6bf015c3d0 2024-05-26T22:49:32.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.166+0000 7f6be7400640 1 --2- 172.21.15.131:0/955268205 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6bf015c3d0 0x7f6bf006ef50 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f6bf006d400 tx=0x7f6bd802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfbebba7a72c5a80 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:32.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.166+0000 7f6bef400640 1 -- 172.21.15.131:0/955268205 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bd8037070 con 0x7f6bf015c3d0 2024-05-26T22:49:32.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.166+0000 7f6bef400640 1 -- 172.21.15.131:0/955268205 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bd8032440 con 0x7f6bf015c3d0 2024-05-26T22:49:32.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.166+0000 7f6bef400640 1 -- 172.21.15.131:0/955268205 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bd8032810 con 0x7f6bf015c3d0 2024-05-26T22:49:32.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.167+0000 7f6bf6e00640 1 -- 172.21.15.131:0/955268205 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6bf015c3d0 msgr2=0x7f6bf006ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:32.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.167+0000 7f6bf6e00640 1 --2- 172.21.15.131:0/955268205 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6bf015c3d0 0x7f6bf006ef50 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f6bf006d400 tx=0x7f6bd802a1e0 comp rx=0 tx=0).stop 2024-05-26T22:49:32.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.167+0000 7f6bf6e00640 1 -- 172.21.15.131:0/955268205 shutdown_connections 2024-05-26T22:49:32.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.167+0000 7f6bf6e00640 1 --2- 172.21.15.131:0/955268205 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6bf006f490 0x7f6bf015ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:32.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.167+0000 7f6bf6e00640 1 --2- 172.21.15.131:0/955268205 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6bf015c3d0 0x7f6bf006ef50 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:32.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.167+0000 7f6bf6e00640 1 --2- 172.21.15.131:0/955268205 >> v2:172.21.15.187:3300/0 conn(0x7f6bf0155840 0x7f6bf0071930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:32.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.167+0000 7f6bf6e00640 1 -- 172.21.15.131:0/955268205 >> 172.21.15.131:0/955268205 conn(0x7f6bf00b1e90 msgr2=0x7f6bf00b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:32.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.167+0000 7f6bf6e00640 1 -- 172.21.15.131:0/955268205 shutdown_connections 2024-05-26T22:49:32.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.167+0000 7f6bf6e00640 1 -- 172.21.15.131:0/955268205 wait complete. 2024-05-26T22:49:32.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.169+0000 7f6bf6e00640 1 Processor -- start 2024-05-26T22:49:32.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.169+0000 7f6bf6e00640 1 -- start start 2024-05-26T22:49:32.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.170+0000 7f6bf6e00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6bf006f490 0x7f6bf0151120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:32.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.170+0000 7f6bf6e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6bf0155840 0x7f6bf01568d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:32.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.170+0000 7f6bf6e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6bf015c3d0 0x7f6bf015d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:32.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.170+0000 7f6bf6e00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f6bf0071ff0 con 0x7f6bf015c3d0 2024-05-26T22:49:32.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.170+0000 7f6bf6e00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6bf0071e70 con 0x7f6bf006f490 2024-05-26T22:49:32.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.170+0000 7f6bf6e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f6bf015f1c0 con 0x7f6bf0155840 2024-05-26T22:49:32.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.170+0000 7f6bf4e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6bf015c3d0 0x7f6bf015d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:32.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.170+0000 7f6bf4e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6bf015c3d0 0x7f6bf015d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41872/0 (socket says 172.21.15.131:41872) 2024-05-26T22:49:32.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.170+0000 7f6bf4e00640 1 -- 172.21.15.131:0/2717947289 learned_addr learned my addr 172.21.15.131:0/2717947289 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:32.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.170+0000 7f6befe00640 1 --2- 172.21.15.131:0/2717947289 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6bf006f490 0x7f6bf0151120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:32.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.170+0000 7f6be7400640 1 --2- 172.21.15.131:0/2717947289 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6bf0155840 0x7f6bf01568d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:32.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.170+0000 7f6bf4e00640 1 -- 172.21.15.131:0/2717947289 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6bf0155840 msgr2=0x7f6bf01568d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:32.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.170+0000 7f6bf4e00640 1 --2- 172.21.15.131:0/2717947289 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6bf0155840 0x7f6bf01568d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:32.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.170+0000 7f6bf4e00640 1 -- 172.21.15.131:0/2717947289 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6bf006f490 msgr2=0x7f6bf0151120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:32.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.170+0000 7f6bf4e00640 1 --2- 172.21.15.131:0/2717947289 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6bf006f490 0x7f6bf0151120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:32.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.170+0000 7f6bf4e00640 1 -- 172.21.15.131:0/2717947289 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bf0157240 con 0x7f6bf015c3d0 2024-05-26T22:49:32.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.170+0000 7f6be7400640 1 --2- 172.21.15.131:0/2717947289 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6bf0155840 0x7f6bf01568d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:49:32.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.171+0000 7f6bf4e00640 1 --2- 172.21.15.131:0/2717947289 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6bf015c3d0 0x7f6bf015d9b0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f6be800ccf0 tx=0x7f6be800a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:32.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.171+0000 7f6becc00640 1 -- 172.21.15.131:0/2717947289 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6be8003b90 con 0x7f6bf015c3d0 2024-05-26T22:49:32.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.171+0000 7f6becc00640 1 -- 172.21.15.131:0/2717947289 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6be8003d30 con 0x7f6bf015c3d0 2024-05-26T22:49:32.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.171+0000 7f6becc00640 1 -- 172.21.15.131:0/2717947289 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6be8004240 con 0x7f6bf015c3d0 2024-05-26T22:49:32.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.171+0000 7f6bf6e00640 1 -- 172.21.15.131:0/2717947289 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bf0159fd0 con 0x7f6bf015c3d0 2024-05-26T22:49:32.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.171+0000 7f6bf6e00640 1 -- 172.21.15.131:0/2717947289 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bf0156e10 con 0x7f6bf015c3d0 2024-05-26T22:49:32.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.172+0000 7f6bf6e00640 1 -- 172.21.15.131:0/2717947289 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bf00694e0 con 0x7f6bf015c3d0 2024-05-26T22:49:32.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.174+0000 7f6becc00640 1 -- 172.21.15.131:0/2717947289 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6be80043e0 con 0x7f6bf015c3d0 2024-05-26T22:49:32.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.175+0000 7f6becc00640 1 --2- 172.21.15.131:0/2717947289 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f6bcc06be30 0x7f6bcc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:32.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.175+0000 7f6becc00640 1 -- 172.21.15.131:0/2717947289 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f6be808b960 con 0x7f6bf015c3d0 2024-05-26T22:49:32.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.175+0000 7f6befe00640 1 --2- 172.21.15.131:0/2717947289 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f6bcc06be30 0x7f6bcc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:32.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.175+0000 7f6befe00640 1 --2- 172.21.15.131:0/2717947289 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f6bcc06be30 0x7f6bcc06e2f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6be0006fd0 tx=0x7f6be0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:32.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.179+0000 7f6becc00640 1 -- 172.21.15.131:0/2717947289 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f6be805a300 con 0x7f6bf015c3d0 2024-05-26T22:49:32.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.365+0000 7f6bf6e00640 1 -- 172.21.15.131:0/2717947289 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6bf00ad170 con 0x7f6bf015c3d0 2024-05-26T22:49:32.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.368+0000 7f6becc00640 1 -- 172.21.15.131:0/2717947289 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) ==== 74+0+16706 (secure 0 0 0) 0x7f6be8059c00 con 0x7f6bf015c3d0 2024-05-26T22:49:32.368 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:49:32.368 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":32,"fsid":"46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f","created":"2024-05-26T22:47:57.288905+0000","modified":"2024-05-26T22:49:30.954432+0000","last_up_change":"2024-05-26T22:48:09.602207+0000","last_in_change":"2024-05-26T22:48:00.687341+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-26T22:48:10.463339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-05-26T22:49:28.813374+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-05-26T22:49:29.987297+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"814eaf62-a18d-494b-8e27-fd8fb88adea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6801","nonce":2350940051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6803","nonce":2350940051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6807","nonce":2350940051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6805","nonce":2350940051}]},"public_addr":"172.21.15.131:6801/2350940051","cluster_addr":"172.21.15.131:6803/2350940051","heartbeat_back_addr":"172.21.15.131:6807/2350940051","heartbeat_front_addr":"172.21.15.131:6805/2350940051","state":["exists","up"]},{"osd":1,"uuid":"42e66287-f6c6-4e26-af3c-6dfaa3536d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6825","nonce":1525528247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6827","nonce":1525528247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6831","nonce":1525528247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6829","nonce":1525528247}]},"public_addr":"172.21.15.131:6825/1525528247","cluster_addr":"172.21.15.131:6827/1525528247","heartbeat_back_addr":"172.21.15.131:6831/1525528247","heartbeat_front_addr":"172.21.15.131:6829/1525528247","state":["exists","up"]},{"osd":2,"uuid":"0da34673-04a2-4fd0-b57a-0ff2b679b043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6817","nonce":3627842210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6819","nonce":3627842210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6823","nonce":3627842210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6821","nonce":3627842210}]},"public_addr":"172.21.15.131:6817/3627842210","cluster_addr":"172.21.15.131:6819/3627842210","heartbeat_back_addr":"172.21.15.131:6823/3627842210","heartbeat_front_addr":"172.21.15.131:6821/3627842210","state":["exists","up"]},{"osd":3,"uuid":"cf117a27-d6b2-43d3-ae52-b47988bf8eed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6809","nonce":350790591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6811","nonce":350790591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6815","nonce":350790591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6813","nonce":350790591}]},"public_addr":"172.21.15.131:6809/350790591","cluster_addr":"172.21.15.131:6811/350790591","heartbeat_back_addr":"172.21.15.131:6815/350790591","heartbeat_front_addr":"172.21.15.131:6813/350790591","state":["exists","up"]},{"osd":4,"uuid":"a092ccfc-be89-4a84-88d5-0f781fa715eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6816","nonce":244560995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6819","nonce":244560995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6823","nonce":244560995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6821","nonce":244560995}]},"public_addr":"172.21.15.187:6816/244560995","cluster_addr":"172.21.15.187:6819/244560995","heartbeat_back_addr":"172.21.15.187:6823/244560995","heartbeat_front_addr":"172.21.15.187:6821/244560995","state":["exists","up"]},{"osd":5,"uuid":"7513ba69-dd00-4a35-b190-9e8cc3b2bdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6801","nonce":324619398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6803","nonce":324619398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6812","nonce":324619398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6807","nonce":324619398}]},"public_addr":"172.21.15.187:6801/324619398","cluster_addr":"172.21.15.187:6803/324619398","heartbeat_back_addr":"172.21.15.187:6812/324619398","heartbeat_front_addr":"172.21.15.187:6807/324619398","state":["exists","up"]},{"osd":6,"uuid":"50081810-c093-4d2f-a4d7-b7c695c5400d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6806","nonce":1391347786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6809","nonce":1391347786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6818","nonce":1391347786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6813","nonce":1391347786}]},"public_addr":"172.21.15.187:6806/1391347786","cluster_addr":"172.21.15.187:6809/1391347786","heartbeat_back_addr":"172.21.15.187:6818/1391347786","heartbeat_front_addr":"172.21.15.187:6813/1391347786","state":["exists","up"]},{"osd":7,"uuid":"775dae05-54f5-4ad4-bbce-92863c6ed2ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6825","nonce":1220065622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6827","nonce":1220065622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6831","nonce":1220065622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6829","nonce":1220065622}]},"public_addr":"172.21.15.187:6825/1220065622","cluster_addr":"172.21.15.187:6827/1220065622","heartbeat_back_addr":"172.21.15.187:6831/1220065622","heartbeat_front_addr":"172.21.15.187:6829/1220065622","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:05.983520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.299516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.067323+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.307748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.018742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.258775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.010912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.295355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-26T22:49:32.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.371+0000 7f6bf6e00640 1 -- 172.21.15.131:0/2717947289 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f6bcc06be30 msgr2=0x7f6bcc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:32.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.371+0000 7f6bf6e00640 1 --2- 172.21.15.131:0/2717947289 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f6bcc06be30 0x7f6bcc06e2f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6be0006fd0 tx=0x7f6be0008040 comp rx=0 tx=0).stop 2024-05-26T22:49:32.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.371+0000 7f6bf6e00640 1 -- 172.21.15.131:0/2717947289 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6bf015c3d0 msgr2=0x7f6bf015d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:32.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.371+0000 7f6bf6e00640 1 --2- 172.21.15.131:0/2717947289 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6bf015c3d0 0x7f6bf015d9b0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f6be800ccf0 tx=0x7f6be800a660 comp rx=0 tx=0).stop 2024-05-26T22:49:32.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.371+0000 7f6bf6e00640 1 -- 172.21.15.131:0/2717947289 shutdown_connections 2024-05-26T22:49:32.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.371+0000 7f6bf6e00640 1 --2- 172.21.15.131:0/2717947289 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6bf0155840 0x7f6bf01568d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:32.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.371+0000 7f6bf6e00640 1 --2- 172.21.15.131:0/2717947289 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f6bcc06be30 0x7f6bcc06e2f0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:32.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.371+0000 7f6bf6e00640 1 --2- 172.21.15.131:0/2717947289 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6bf015c3d0 0x7f6bf015d9b0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:32.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.371+0000 7f6bf6e00640 1 --2- 172.21.15.131:0/2717947289 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6bf006f490 0x7f6bf0151120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:32.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.371+0000 7f6bf6e00640 1 -- 172.21.15.131:0/2717947289 >> 172.21.15.131:0/2717947289 conn(0x7f6bf00b1e90 msgr2=0x7f6bf006e830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:32.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.371+0000 7f6bf6e00640 1 -- 172.21.15.131:0/2717947289 shutdown_connections 2024-05-26T22:49:32.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.371+0000 7f6bf6e00640 1 -- 172.21.15.131:0/2717947289 wait complete. 2024-05-26T22:49:32.389 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-26T22:49:32.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.601+0000 7fab70200640 1 Processor -- start 2024-05-26T22:49:32.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.602+0000 7fab70200640 1 -- start start 2024-05-26T22:49:32.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.603+0000 7fab70200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fab68155770 0x7fab6815b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:32.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.603+0000 7fab70200640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fab6815bc40 0x7fab6806dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:32.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.603+0000 7fab70200640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab6806e680 con 0x7fab6806e270 2024-05-26T22:49:32.604 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.603+0000 7fab70200640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fab680735b0 con 0x7fab68155770 2024-05-26T22:49:32.604 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.603+0000 7fab70200640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fab6815c1c0 con 0x7fab6815bc40 2024-05-26T22:49:32.604 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.604+0000 7fab6d800640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fab68155770 0x7fab6815b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:32.604 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.604+0000 7fab6e200640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fab6806e270 0x7fab680ceba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:38036/0 (socket says 172.21.15.131:38036) 2024-05-26T22:49:32.604 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.604+0000 7fab6e200640 1 -- 172.21.15.131:0/2153019509 learned_addr learned my addr 172.21.15.131:0/2153019509 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:49:32.604 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.604+0000 7fab6ce00640 1 --2- 172.21.15.131:0/2153019509 >> v2:172.21.15.187:3300/0 conn(0x7fab6815bc40 0x7fab6806dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:32.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.604+0000 7fab6d800640 1 -- 172.21.15.131:0/2153019509 >> v2:172.21.15.187:3300/0 conn(0x7fab6815bc40 msgr2=0x7fab6806dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:32.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.604+0000 7fab6d800640 1 --2- 172.21.15.131:0/2153019509 >> v2:172.21.15.187:3300/0 conn(0x7fab6815bc40 0x7fab6806dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:32.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.604+0000 7fab6d800640 1 -- 172.21.15.131:0/2153019509 >> v1:172.21.15.131:6789/0 conn(0x7fab6806e270 legacy=0x7fab680ceba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:32.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.604+0000 7fab6d800640 1 -- 172.21.15.131:0/2153019509 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab6815c4a0 con 0x7fab68155770 2024-05-26T22:49:32.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.605+0000 7fab6d800640 1 --2- 172.21.15.131:0/2153019509 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab68155770 0x7fab6815b700 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fab6815c0d0 tx=0x7fab5c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=581a6c6ffd220958 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:32.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.606+0000 7fab67e00640 1 -- 172.21.15.131:0/2153019509 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab5c037070 con 0x7fab68155770 2024-05-26T22:49:32.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.606+0000 7fab67e00640 1 -- 172.21.15.131:0/2153019509 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab5c032440 con 0x7fab68155770 2024-05-26T22:49:32.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.606+0000 7fab67e00640 1 -- 172.21.15.131:0/2153019509 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab5c032810 con 0x7fab68155770 2024-05-26T22:49:32.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.606+0000 7fab70200640 1 -- 172.21.15.131:0/2153019509 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab68155770 msgr2=0x7fab6815b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:32.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.606+0000 7fab70200640 1 --2- 172.21.15.131:0/2153019509 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab68155770 0x7fab6815b700 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fab6815c0d0 tx=0x7fab5c02a230 comp rx=0 tx=0).stop 2024-05-26T22:49:32.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.606+0000 7fab70200640 1 -- 172.21.15.131:0/2153019509 shutdown_connections 2024-05-26T22:49:32.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.606+0000 7fab70200640 1 --2- 172.21.15.131:0/2153019509 >> v2:172.21.15.187:3300/0 conn(0x7fab6815bc40 0x7fab6806dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:32.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.606+0000 7fab70200640 1 --2- 172.21.15.131:0/2153019509 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab68155770 0x7fab6815b700 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:32.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.606+0000 7fab70200640 1 -- 172.21.15.131:0/2153019509 >> 172.21.15.131:0/2153019509 conn(0x7fab6806bc90 msgr2=0x7fab680b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:32.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.606+0000 7fab70200640 1 -- 172.21.15.131:0/2153019509 shutdown_connections 2024-05-26T22:49:32.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.606+0000 7fab70200640 1 -- 172.21.15.131:0/2153019509 wait complete. 2024-05-26T22:49:32.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.608+0000 7fab70200640 1 Processor -- start 2024-05-26T22:49:32.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.608+0000 7fab70200640 1 -- start start 2024-05-26T22:49:32.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.609+0000 7fab70200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab6806e270 0x7fab6809b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:32.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.609+0000 7fab70200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fab68155770 0x7fab680cc460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:32.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.609+0000 7fab70200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fab6815bc40 0x7fab680b0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:32.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.609+0000 7fab70200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fab68155b50 con 0x7fab6806e270 2024-05-26T22:49:32.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.609+0000 7fab70200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fab68072080 con 0x7fab68155770 2024-05-26T22:49:32.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.609+0000 7fab70200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fab6815c8c0 con 0x7fab6815bc40 2024-05-26T22:49:32.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.609+0000 7fab6d800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab6806e270 0x7fab6809b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:32.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.609+0000 7fab6d800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab6806e270 0x7fab6809b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41886/0 (socket says 172.21.15.131:41886) 2024-05-26T22:49:32.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.609+0000 7fab6d800640 1 -- 172.21.15.131:0/1123823892 learned_addr learned my addr 172.21.15.131:0/1123823892 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:32.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.610+0000 7fab6d800640 1 -- 172.21.15.131:0/1123823892 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fab6815bc40 msgr2=0x7fab680b0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:32.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.610+0000 7fab6d800640 1 --2- 172.21.15.131:0/1123823892 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fab6815bc40 0x7fab680b0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:32.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.610+0000 7fab6d800640 1 -- 172.21.15.131:0/1123823892 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fab68155770 msgr2=0x7fab680cc460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:32.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.610+0000 7fab6d800640 1 --2- 172.21.15.131:0/1123823892 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fab68155770 0x7fab680cc460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:32.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.610+0000 7fab6d800640 1 -- 172.21.15.131:0/1123823892 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab68140d40 con 0x7fab6806e270 2024-05-26T22:49:32.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.610+0000 7fab6d800640 1 --2- 172.21.15.131:0/1123823892 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab6806e270 0x7fab6809b1a0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fab5c000c00 tx=0x7fab5c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:32.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.610+0000 7fab65600640 1 -- 172.21.15.131:0/1123823892 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab5c037040 con 0x7fab6806e270 2024-05-26T22:49:32.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.610+0000 7fab65600640 1 -- 172.21.15.131:0/1123823892 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab5c042070 con 0x7fab6806e270 2024-05-26T22:49:32.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.610+0000 7fab65600640 1 -- 172.21.15.131:0/1123823892 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab5c030a70 con 0x7fab6806e270 2024-05-26T22:49:32.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.610+0000 7fab70200640 1 -- 172.21.15.131:0/1123823892 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab680ccb40 con 0x7fab6806e270 2024-05-26T22:49:32.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.611+0000 7fab70200640 1 -- 172.21.15.131:0/1123823892 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab680cd100 con 0x7fab6806e270 2024-05-26T22:49:32.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.612+0000 7fab70200640 1 -- 172.21.15.131:0/1123823892 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab68074e90 con 0x7fab6806e270 2024-05-26T22:49:32.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.613+0000 7fab65600640 1 -- 172.21.15.131:0/1123823892 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fab5c030c10 con 0x7fab6806e270 2024-05-26T22:49:32.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.613+0000 7fab65600640 1 --2- 172.21.15.131:0/1123823892 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fab4806be30 0x7fab4806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:32.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.613+0000 7fab65600640 1 -- 172.21.15.131:0/1123823892 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fab5c0ab980 con 0x7fab6806e270 2024-05-26T22:49:32.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.614+0000 7fab6ce00640 1 --2- 172.21.15.131:0/1123823892 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fab4806be30 0x7fab4806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:32.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.614+0000 7fab6ce00640 1 --2- 172.21.15.131:0/1123823892 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fab4806be30 0x7fab4806e2f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fab6815b270 tx=0x7fab54008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:32.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.618+0000 7fab65600640 1 -- 172.21.15.131:0/1123823892 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fab5c045360 con 0x7fab6806e270 2024-05-26T22:49:32.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:32.854+0000 7fab70200640 1 -- 172.21.15.131:0/1123823892 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fab680695c0 con 0x7fab6806e270 2024-05-26T22:49:33.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:33.538+0000 7fab65600640 1 -- 172.21.15.131:0/1123823892 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v14) ==== 118+0+0 (secure 0 0 0) 0x7fab5c07a330 con 0x7fab6806e270 2024-05-26T22:49:33.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:33.542+0000 7fab70200640 1 -- 172.21.15.131:0/1123823892 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fab4806be30 msgr2=0x7fab4806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:33.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:33.542+0000 7fab70200640 1 --2- 172.21.15.131:0/1123823892 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fab4806be30 0x7fab4806e2f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fab6815b270 tx=0x7fab54008040 comp rx=0 tx=0).stop 2024-05-26T22:49:33.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:33.542+0000 7fab70200640 1 -- 172.21.15.131:0/1123823892 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab6806e270 msgr2=0x7fab6809b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:33.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:33.542+0000 7fab70200640 1 --2- 172.21.15.131:0/1123823892 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab6806e270 0x7fab6809b1a0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fab5c000c00 tx=0x7fab5c0046c0 comp rx=0 tx=0).stop 2024-05-26T22:49:33.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:33.542+0000 7fab70200640 1 -- 172.21.15.131:0/1123823892 shutdown_connections 2024-05-26T22:49:33.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:33.542+0000 7fab70200640 1 --2- 172.21.15.131:0/1123823892 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fab6815bc40 0x7fab680b0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:33.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:33.542+0000 7fab70200640 1 --2- 172.21.15.131:0/1123823892 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fab4806be30 0x7fab4806e2f0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:33.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:33.542+0000 7fab70200640 1 --2- 172.21.15.131:0/1123823892 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fab68155770 0x7fab680cc460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:33.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:33.542+0000 7fab70200640 1 --2- 172.21.15.131:0/1123823892 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab6806e270 0x7fab6809b1a0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:33.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:33.542+0000 7fab70200640 1 -- 172.21.15.131:0/1123823892 >> 172.21.15.131:0/1123823892 conn(0x7fab6806bc90 msgr2=0x7fab680b3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:33.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:33.542+0000 7fab70200640 1 -- 172.21.15.131:0/1123823892 shutdown_connections 2024-05-26T22:49:33.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:33.542+0000 7fab70200640 1 -- 172.21.15.131:0/1123823892 wait complete. 2024-05-26T22:49:33.565 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-26T22:49:33.565 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-26T22:49:33.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:33.774+0000 7fb0b2e00640 1 Processor -- start 2024-05-26T22:49:33.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:33.775+0000 7fb0b2e00640 1 -- start start 2024-05-26T22:49:33.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:33.776+0000 7fb0b2e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fb0ac147be0 0x7fb0ac15fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:33.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:33.776+0000 7fb0b2e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb0ac15ff90 0x7fb0ac070d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:33.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:33.776+0000 7fb0b2e00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0ac0717d0 con 0x7fb0ac0712c0 2024-05-26T22:49:33.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:33.776+0000 7fb0b2e00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fb0ac06c8b0 con 0x7fb0ac147be0 2024-05-26T22:49:33.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:33.776+0000 7fb0b2e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb0ac071ab0 con 0x7fb0ac15ff90 2024-05-26T22:49:33.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:33.776+0000 7fb0abe00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fb0ac147be0 0x7fb0ac15fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:33.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:33.776+0000 7fb0b0e00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fb0ac0712c0 0x7fb0ac0b5410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:38040/0 (socket says 172.21.15.131:38040) 2024-05-26T22:49:33.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:33.776+0000 7fb0abe00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fb0ac147be0 0x7fb0ac15fa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41892/0 (socket says 172.21.15.131:41892) 2024-05-26T22:49:33.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:33.776+0000 7fb0b0e00640 1 -- 172.21.15.131:0/1799490124 learned_addr learned my addr 172.21.15.131:0/1799490124 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:49:33.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:33.777+0000 7fb0ab400640 1 --2- 172.21.15.131:0/1799490124 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb0ac15ff90 0x7fb0ac070d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:33.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:33.777+0000 7fb0abe00640 1 -- 172.21.15.131:0/1799490124 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb0ac15ff90 msgr2=0x7fb0ac070d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:33.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:33.777+0000 7fb0abe00640 1 --2- 172.21.15.131:0/1799490124 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb0ac15ff90 0x7fb0ac070d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:33.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:33.777+0000 7fb0abe00640 1 -- 172.21.15.131:0/1799490124 >> v1:172.21.15.131:6789/0 conn(0x7fb0ac0712c0 legacy=0x7fb0ac0b5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:33.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:33.777+0000 7fb0abe00640 1 -- 172.21.15.131:0/1799490124 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0ac06cb00 con 0x7fb0ac147be0 2024-05-26T22:49:33.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:33.777+0000 7fb0aaa00640 1 -- 172.21.15.131:0/1799490124 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 560221490 0 0) 0x7fb0ac0717d0 con 0x7fb0ac0712c0 2024-05-26T22:49:33.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:33.778+0000 7fb0abe00640 1 --2- 172.21.15.131:0/1799490124 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb0ac147be0 0x7fb0ac15fa50 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fb09c0097a0 tx=0x7fb09c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79a370d8cbff6ebe server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:33.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:33.778+0000 7fb0aaa00640 1 -- 172.21.15.131:0/1799490124 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb09c037070 con 0x7fb0ac147be0 2024-05-26T22:49:33.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:33.778+0000 7fb0aaa00640 1 -- 172.21.15.131:0/1799490124 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb09c032440 con 0x7fb0ac147be0 2024-05-26T22:49:33.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:33.778+0000 7fb0aaa00640 1 -- 172.21.15.131:0/1799490124 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb09c032810 con 0x7fb0ac147be0 2024-05-26T22:49:33.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:33.779+0000 7fb0b2e00640 1 -- 172.21.15.131:0/1799490124 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb0ac147be0 msgr2=0x7fb0ac15fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:33.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:33.779+0000 7fb0b2e00640 1 --2- 172.21.15.131:0/1799490124 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb0ac147be0 0x7fb0ac15fa50 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fb09c0097a0 tx=0x7fb09c02a180 comp rx=0 tx=0).stop 2024-05-26T22:49:33.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:33.779+0000 7fb0b2e00640 1 -- 172.21.15.131:0/1799490124 shutdown_connections 2024-05-26T22:49:33.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:33.779+0000 7fb0b2e00640 1 --2- 172.21.15.131:0/1799490124 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb0ac15ff90 0x7fb0ac070d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:33.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:33.779+0000 7fb0b2e00640 1 --2- 172.21.15.131:0/1799490124 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb0ac147be0 0x7fb0ac15fa50 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:33.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:33.779+0000 7fb0b2e00640 1 -- 172.21.15.131:0/1799490124 >> 172.21.15.131:0/1799490124 conn(0x7fb0ac0b3e00 msgr2=0x7fb0ac0b41e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:33.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:33.779+0000 7fb0b2e00640 1 -- 172.21.15.131:0/1799490124 shutdown_connections 2024-05-26T22:49:33.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:33.779+0000 7fb0b2e00640 1 -- 172.21.15.131:0/1799490124 wait complete. 2024-05-26T22:49:33.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:33.781+0000 7fb0b2e00640 1 Processor -- start 2024-05-26T22:49:33.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:33.781+0000 7fb0b2e00640 1 -- start start 2024-05-26T22:49:33.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:33.782+0000 7fb0b2e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb0ac0712c0 0x7fb0ac1f8330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:33.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:33.782+0000 7fb0b2e00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb0ac147be0 0x7fb0ac15cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:33.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:33.782+0000 7fb0b2e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb0ac15ff90 0x7fb0ac13f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:33.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:33.782+0000 7fb0b2e00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fb0ac1620f0 con 0x7fb0ac0712c0 2024-05-26T22:49:33.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:33.782+0000 7fb0b2e00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb0ac147fc0 con 0x7fb0ac147be0 2024-05-26T22:49:33.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:33.782+0000 7fb0b2e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb0ac162270 con 0x7fb0ac15ff90 2024-05-26T22:49:33.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:33.782+0000 7fb0abe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb0ac0712c0 0x7fb0ac1f8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:33.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:33.782+0000 7fb0abe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb0ac0712c0 0x7fb0ac1f8330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41906/0 (socket says 172.21.15.131:41906) 2024-05-26T22:49:33.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:33.782+0000 7fb0abe00640 1 -- 172.21.15.131:0/1688543757 learned_addr learned my addr 172.21.15.131:0/1688543757 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:33.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:33.782+0000 7fb0ab400640 1 --2- 172.21.15.131:0/1688543757 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb0ac147be0 0x7fb0ac15cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:33.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:33.782+0000 7fb0abe00640 1 -- 172.21.15.131:0/1688543757 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb0ac15ff90 msgr2=0x7fb0ac13f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:33.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:33.782+0000 7fb0abe00640 1 --2- 172.21.15.131:0/1688543757 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb0ac15ff90 0x7fb0ac13f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:33.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:33.782+0000 7fb0abe00640 1 -- 172.21.15.131:0/1688543757 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb0ac147be0 msgr2=0x7fb0ac15cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:33.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:33.782+0000 7fb0abe00640 1 --2- 172.21.15.131:0/1688543757 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb0ac147be0 0x7fb0ac15cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:33.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:33.782+0000 7fb0abe00640 1 -- 172.21.15.131:0/1688543757 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0ac1f8870 con 0x7fb0ac0712c0 2024-05-26T22:49:33.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:33.783+0000 7fb0abe00640 1 --2- 172.21.15.131:0/1688543757 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb0ac0712c0 0x7fb0ac1f8330 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fb09c000c00 tx=0x7fb09c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:33.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:33.783+0000 7fb0a3e00640 1 -- 172.21.15.131:0/1688543757 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb09c037040 con 0x7fb0ac0712c0 2024-05-26T22:49:33.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:33.783+0000 7fb0a3e00640 1 -- 172.21.15.131:0/1688543757 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb09c042070 con 0x7fb0ac0712c0 2024-05-26T22:49:33.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:33.783+0000 7fb0a3e00640 1 -- 172.21.15.131:0/1688543757 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb09c030bb0 con 0x7fb0ac0712c0 2024-05-26T22:49:33.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:33.783+0000 7fb0b2e00640 1 -- 172.21.15.131:0/1688543757 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0ac140260 con 0x7fb0ac0712c0 2024-05-26T22:49:33.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:33.783+0000 7fb0b2e00640 1 -- 172.21.15.131:0/1688543757 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0ac140820 con 0x7fb0ac0712c0 2024-05-26T22:49:33.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:33.784+0000 7fb0b2e00640 1 -- 172.21.15.131:0/1688543757 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0ac0695c0 con 0x7fb0ac0712c0 2024-05-26T22:49:33.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:33.785+0000 7fb0a3e00640 1 -- 172.21.15.131:0/1688543757 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb09c047460 con 0x7fb0ac0712c0 2024-05-26T22:49:33.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:33.786+0000 7fb0a3e00640 1 --2- 172.21.15.131:0/1688543757 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb08806be30 0x7fb08806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:33.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:33.786+0000 7fb0a3e00640 1 -- 172.21.15.131:0/1688543757 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fb09c0ab9e0 con 0x7fb0ac0712c0 2024-05-26T22:49:33.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:33.786+0000 7fb0ab400640 1 --2- 172.21.15.131:0/1688543757 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb08806be30 0x7fb08806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:33.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:33.787+0000 7fb0ab400640 1 --2- 172.21.15.131:0/1688543757 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb08806be30 0x7fb08806e2f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb094006fd0 tx=0x7fb094008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:33.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:33.790+0000 7fb0a3e00640 1 -- 172.21.15.131:0/1688543757 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fb0ac1f8870 con 0x7fb0ac0712c0 2024-05-26T22:49:34.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:34.031+0000 7fb0b2e00640 1 -- 172.21.15.131:0/1688543757 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fb0ac0b62c0 con 0x7fb0ac0712c0 2024-05-26T22:49:34.467 INFO:tasks.ceph.mds.c.smithi131.stderr:2024-05-26T22:49:34.467+0000 7fc331000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-26T22:49:34.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:34.558+0000 7fb0a3e00640 1 -- 172.21.15.131:0/1688543757 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v15) ==== 122+0+0 (secure 0 0 0) 0x7fb09c032e20 con 0x7fb0ac0712c0 2024-05-26T22:49:34.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:34.562+0000 7fb0b2e00640 1 -- 172.21.15.131:0/1688543757 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb08806be30 msgr2=0x7fb08806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:34.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:34.562+0000 7fb0b2e00640 1 --2- 172.21.15.131:0/1688543757 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb08806be30 0x7fb08806e2f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb094006fd0 tx=0x7fb094008040 comp rx=0 tx=0).stop 2024-05-26T22:49:34.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:34.562+0000 7fb0b2e00640 1 -- 172.21.15.131:0/1688543757 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb0ac0712c0 msgr2=0x7fb0ac1f8330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:34.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:34.562+0000 7fb0b2e00640 1 --2- 172.21.15.131:0/1688543757 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb0ac0712c0 0x7fb0ac1f8330 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fb09c000c00 tx=0x7fb09c0046c0 comp rx=0 tx=0).stop 2024-05-26T22:49:34.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:34.563+0000 7fb0b2e00640 1 -- 172.21.15.131:0/1688543757 shutdown_connections 2024-05-26T22:49:34.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:34.563+0000 7fb0b2e00640 1 --2- 172.21.15.131:0/1688543757 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb0ac15ff90 0x7fb0ac13f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:34.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:34.563+0000 7fb0b2e00640 1 --2- 172.21.15.131:0/1688543757 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb08806be30 0x7fb08806e2f0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:34.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:34.563+0000 7fb0b2e00640 1 --2- 172.21.15.131:0/1688543757 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb0ac147be0 0x7fb0ac15cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:34.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:34.563+0000 7fb0b2e00640 1 --2- 172.21.15.131:0/1688543757 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb0ac0712c0 0x7fb0ac1f8330 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:34.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:34.563+0000 7fb0b2e00640 1 -- 172.21.15.131:0/1688543757 >> 172.21.15.131:0/1688543757 conn(0x7fb0ac0b3e00 msgr2=0x7fb0ac161370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:34.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:34.563+0000 7fb0b2e00640 1 -- 172.21.15.131:0/1688543757 shutdown_connections 2024-05-26T22:49:34.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:34.563+0000 7fb0b2e00640 1 -- 172.21.15.131:0/1688543757 wait complete. 2024-05-26T22:49:34.584 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-26T22:49:34.584 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-26T22:49:34.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:34.786+0000 7f45bbe00640 1 Processor -- start 2024-05-26T22:49:34.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:34.786+0000 7f45bbe00640 1 -- start start 2024-05-26T22:49:34.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:34.788+0000 7f45bbe00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f45bc144800 0x7f45bc15fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:34.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:34.788+0000 7f45bbe00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f45bc15f100 0x7f45bc0add00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:34.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:34.788+0000 7f45bbe00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45bc15f7b0 con 0x7f45bc0ad870 2024-05-26T22:49:34.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:34.788+0000 7f45bbe00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f45bc15f570 con 0x7f45bc144800 2024-05-26T22:49:34.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:34.788+0000 7f45bbe00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f45bc15ffd0 con 0x7f45bc15f100 2024-05-26T22:49:34.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:34.789+0000 7f45bb400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f45bc15f100 0x7f45bc0add00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:34.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:34.789+0000 7f45bb400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f45bc15f100 0x7f45bc0add00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.131:40400/0 (socket says 172.21.15.131:40400) 2024-05-26T22:49:34.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:34.789+0000 7f45bb400640 1 -- 172.21.15.131:0/1683929824 learned_addr learned my addr 172.21.15.131:0/1683929824 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:34.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:34.789+0000 7f45baa00640 1 --2- 172.21.15.131:0/1683929824 >> v2:172.21.15.187:3300/0 conn(0x7f45bc144800 0x7f45bc15fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:34.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:34.790+0000 7f45bb400640 1 -- 172.21.15.131:0/1683929824 >> v2:172.21.15.187:3300/0 conn(0x7f45bc144800 msgr2=0x7f45bc15fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:34.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:34.790+0000 7f45bb400640 1 --2- 172.21.15.131:0/1683929824 >> v2:172.21.15.187:3300/0 conn(0x7f45bc144800 0x7f45bc15fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:34.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:34.790+0000 7f45bb400640 1 -- 172.21.15.131:0/1683929824 >> v1:172.21.15.187:6789/0 conn(0x7f45bc0ad870 legacy=0x7f45bc15e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:34.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:34.790+0000 7f45bb400640 1 -- 172.21.15.131:0/1683929824 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45bc0cde40 con 0x7f45bc15f100 2024-05-26T22:49:34.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:34.791+0000 7f45bb400640 1 --2- 172.21.15.131:0/1683929824 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f45bc15f100 0x7f45bc0add00 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f45b4009e50 tx=0x7f45b402a7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=556760689638b5e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:34.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:34.792+0000 7f45ba000640 1 -- 172.21.15.131:0/1683929824 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b4036070 con 0x7f45bc15f100 2024-05-26T22:49:34.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:34.792+0000 7f45ba000640 1 -- 172.21.15.131:0/1683929824 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45b4031600 con 0x7f45bc15f100 2024-05-26T22:49:34.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:34.792+0000 7f45ba000640 1 -- 172.21.15.131:0/1683929824 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b40319d0 con 0x7f45bc15f100 2024-05-26T22:49:34.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:34.792+0000 7f45bbe00640 1 -- 172.21.15.131:0/1683929824 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f45bc15f100 msgr2=0x7f45bc0add00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:34.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:34.793+0000 7f45bbe00640 1 --2- 172.21.15.131:0/1683929824 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f45bc15f100 0x7f45bc0add00 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f45b4009e50 tx=0x7f45b402a7a0 comp rx=0 tx=0).stop 2024-05-26T22:49:34.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:34.793+0000 7f45bbe00640 1 -- 172.21.15.131:0/1683929824 shutdown_connections 2024-05-26T22:49:34.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:34.793+0000 7f45bbe00640 1 --2- 172.21.15.131:0/1683929824 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f45bc15f100 0x7f45bc0add00 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:34.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:34.793+0000 7f45bbe00640 1 --2- 172.21.15.131:0/1683929824 >> v2:172.21.15.187:3300/0 conn(0x7f45bc144800 0x7f45bc15fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:34.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:34.793+0000 7f45bbe00640 1 -- 172.21.15.131:0/1683929824 >> 172.21.15.131:0/1683929824 conn(0x7f45bc0cff40 msgr2=0x7f45bc0d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:34.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:34.793+0000 7f45bbe00640 1 -- 172.21.15.131:0/1683929824 shutdown_connections 2024-05-26T22:49:34.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:34.793+0000 7f45bbe00640 1 -- 172.21.15.131:0/1683929824 wait complete. 2024-05-26T22:49:34.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:34.795+0000 7f45bbe00640 1 Processor -- start 2024-05-26T22:49:34.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:34.795+0000 7f45bbe00640 1 -- start start 2024-05-26T22:49:34.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:34.796+0000 7f45bbe00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f45bc0ad870 0x7f45bc0b1370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:34.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:34.796+0000 7f45bbe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f45bc144800 0x7f45bc0b18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:34.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:34.796+0000 7f45bbe00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f45bc15f100 0x7f45bc1f44d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:34.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:34.796+0000 7f45bbe00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f45bc144be0 con 0x7f45bc144800 2024-05-26T22:49:34.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:34.796+0000 7f45bbe00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f45bc0ce400 con 0x7f45bc15f100 2024-05-26T22:49:34.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:34.796+0000 7f45bbe00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f45bc0ce920 con 0x7f45bc0ad870 2024-05-26T22:49:34.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:34.796+0000 7f45b3e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f45bc144800 0x7f45bc0b18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:34.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:34.796+0000 7f45b3e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f45bc144800 0x7f45bc0b18b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53148/0 (socket says 172.21.15.131:53148) 2024-05-26T22:49:34.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:34.796+0000 7f45b3e00640 1 -- 172.21.15.131:0/1420114566 learned_addr learned my addr 172.21.15.131:0/1420114566 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:34.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:34.796+0000 7f45baa00640 1 --2- 172.21.15.131:0/1420114566 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f45bc0ad870 0x7f45bc0b1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:34.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:34.796+0000 7f45bb400640 1 --2- 172.21.15.131:0/1420114566 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f45bc15f100 0x7f45bc1f44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:34.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:34.797+0000 7f45b3e00640 1 -- 172.21.15.131:0/1420114566 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f45bc0ad870 msgr2=0x7f45bc0b1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:34.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:34.797+0000 7f45b3e00640 1 --2- 172.21.15.131:0/1420114566 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f45bc0ad870 0x7f45bc0b1370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:34.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:34.797+0000 7f45b3e00640 1 -- 172.21.15.131:0/1420114566 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f45bc15f100 msgr2=0x7f45bc1f44d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:34.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:34.797+0000 7f45b3e00640 1 --2- 172.21.15.131:0/1420114566 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f45bc15f100 0x7f45bc1f44d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:34.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:34.797+0000 7f45b3e00640 1 -- 172.21.15.131:0/1420114566 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45bc0b0fa0 con 0x7f45bc144800 2024-05-26T22:49:34.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:34.797+0000 7f45baa00640 1 --2- 172.21.15.131:0/1420114566 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f45bc0ad870 0x7f45bc0b1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-26T22:49:34.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:34.797+0000 7f45b3e00640 1 --2- 172.21.15.131:0/1420114566 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f45bc144800 0x7f45bc0b18b0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f45a400c430 tx=0x7f45a400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:34.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:34.797+0000 7f45b2a00640 1 -- 172.21.15.131:0/1420114566 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a40052d0 con 0x7f45bc144800 2024-05-26T22:49:34.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:34.797+0000 7f45b2a00640 1 -- 172.21.15.131:0/1420114566 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45a4005490 con 0x7f45bc144800 2024-05-26T22:49:34.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:34.797+0000 7f45b2a00640 1 -- 172.21.15.131:0/1420114566 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a4007260 con 0x7f45bc144800 2024-05-26T22:49:34.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:34.797+0000 7f45bbe00640 1 -- 172.21.15.131:0/1420114566 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45bc0aff00 con 0x7f45bc144800 2024-05-26T22:49:34.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:34.798+0000 7f45bbe00640 1 -- 172.21.15.131:0/1420114566 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45bc0b04c0 con 0x7f45bc144800 2024-05-26T22:49:34.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:34.799+0000 7f45bbe00640 1 -- 172.21.15.131:0/1420114566 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45bc0694e0 con 0x7f45bc144800 2024-05-26T22:49:34.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:34.800+0000 7f45b2a00640 1 -- 172.21.15.131:0/1420114566 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f45a4007400 con 0x7f45bc144800 2024-05-26T22:49:34.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:34.800+0000 7f45b2a00640 1 --2- 172.21.15.131:0/1420114566 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f459806be30 0x7f459806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:34.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:34.801+0000 7f45b2a00640 1 -- 172.21.15.131:0/1420114566 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f45a408a420 con 0x7f45bc144800 2024-05-26T22:49:34.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:34.801+0000 7f45baa00640 1 --2- 172.21.15.131:0/1420114566 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f459806be30 0x7f459806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:34.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:34.802+0000 7f45baa00640 1 --2- 172.21.15.131:0/1420114566 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f459806be30 0x7f459806e2f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f45ac009b90 tx=0x7f45ac008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:34.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:34.805+0000 7f45b2a00640 1 -- 172.21.15.131:0/1420114566 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f45a4058d30 con 0x7f45bc144800 2024-05-26T22:49:35.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.040+0000 7f45bbe00640 1 -- 172.21.15.131:0/1420114566 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f45bc06eb90 con 0x7f45bc144800 2024-05-26T22:49:35.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.042+0000 7f45b2a00640 1 -- 172.21.15.131:0/1420114566 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) ==== 94+0+5193 (secure 0 0 0) 0x7f45a4058630 con 0x7f45bc144800 2024-05-26T22:49:35.121 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:49:35.121 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/4292497989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":4292497989},{"type":"v1","addr":"172.21.15.187:6835","nonce":4292497989}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4678,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6836/4032294430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":4032294430},{"type":"v1","addr":"172.21.15.131:6836","nonce":4032294430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4687,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/3274403209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3274403209},{"type":"v1","addr":"172.21.15.187:6833","nonce":3274403209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-26T22:49:30.955525+0000","modified":"2024-05-26T22:49:34.033245+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5115},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5115":{"gid":5115,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.131:6837/2062704413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6835","nonce":2062704413},{"type":"v1","addr":"172.21.15.131:6837","nonce":2062704413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5115,"qdb_cluster":[5115]},"id":2}]} 2024-05-26T22:49:35.123 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 15 2024-05-26T22:49:35.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.045+0000 7f45bbe00640 1 -- 172.21.15.131:0/1420114566 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f459806be30 msgr2=0x7f459806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:35.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.045+0000 7f45bbe00640 1 --2- 172.21.15.131:0/1420114566 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f459806be30 0x7f459806e2f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f45ac009b90 tx=0x7f45ac008040 comp rx=0 tx=0).stop 2024-05-26T22:49:35.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.045+0000 7f45bbe00640 1 -- 172.21.15.131:0/1420114566 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f45bc144800 msgr2=0x7f45bc0b18b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:35.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.045+0000 7f45bbe00640 1 --2- 172.21.15.131:0/1420114566 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f45bc144800 0x7f45bc0b18b0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f45a400c430 tx=0x7f45a400c8f0 comp rx=0 tx=0).stop 2024-05-26T22:49:35.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.045+0000 7f45bbe00640 1 -- 172.21.15.131:0/1420114566 shutdown_connections 2024-05-26T22:49:35.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.045+0000 7f45bbe00640 1 --2- 172.21.15.131:0/1420114566 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f459806be30 0x7f459806e2f0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:35.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.045+0000 7f45bbe00640 1 --2- 172.21.15.131:0/1420114566 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f45bc15f100 0x7f45bc1f44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:35.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.045+0000 7f45bbe00640 1 --2- 172.21.15.131:0/1420114566 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f45bc144800 0x7f45bc0b18b0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:35.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.045+0000 7f45bbe00640 1 --2- 172.21.15.131:0/1420114566 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f45bc0ad870 0x7f45bc0b1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:35.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.045+0000 7f45bbe00640 1 -- 172.21.15.131:0/1420114566 >> 172.21.15.131:0/1420114566 conn(0x7f45bc0cff40 msgr2=0x7f45bc071370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:35.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.045+0000 7f45bbe00640 1 -- 172.21.15.131:0/1420114566 shutdown_connections 2024-05-26T22:49:35.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.045+0000 7f45bbe00640 1 -- 172.21.15.131:0/1420114566 wait complete. 2024-05-26T22:49:35.124 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-26T22:49:35.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.330+0000 7f84f3a00640 1 Processor -- start 2024-05-26T22:49:35.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.330+0000 7f84f3a00640 1 -- start start 2024-05-26T22:49:35.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.331+0000 7f84f3a00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f84ec155840 0x7f84ec071930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:35.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.331+0000 7f84f3a00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f84ec15c3d0 0x7f84ec06ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:35.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.331+0000 7f84f3a00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84ec0722a0 con 0x7f84ec06f490 2024-05-26T22:49:35.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.331+0000 7f84f3a00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f84ec072580 con 0x7f84ec15c3d0 2024-05-26T22:49:35.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.331+0000 7f84f3a00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f84ec072860 con 0x7f84ec155840 2024-05-26T22:49:35.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.332+0000 7f84e3e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f84ec15c3d0 0x7f84ec06ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:35.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.332+0000 7f84e3e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f84ec15c3d0 0x7f84ec06ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53154/0 (socket says 172.21.15.131:53154) 2024-05-26T22:49:35.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.332+0000 7f84e3e00640 1 -- 172.21.15.131:0/3937156569 learned_addr learned my addr 172.21.15.131:0/3937156569 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:35.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.332+0000 7f84e3e00640 1 -- 172.21.15.131:0/3937156569 >> v2:172.21.15.187:3300/0 conn(0x7f84ec155840 msgr2=0x7f84ec071930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:35.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.332+0000 7f84e3e00640 1 --2- 172.21.15.131:0/3937156569 >> v2:172.21.15.187:3300/0 conn(0x7f84ec155840 0x7f84ec071930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:35.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.332+0000 7f84e3e00640 1 -- 172.21.15.131:0/3937156569 >> v1:172.21.15.187:6789/0 conn(0x7f84ec06f490 legacy=0x7f84ec15ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:35.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.332+0000 7f84e3e00640 1 -- 172.21.15.131:0/3937156569 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84ec072b40 con 0x7f84ec15c3d0 2024-05-26T22:49:35.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.333+0000 7f84e3e00640 1 --2- 172.21.15.131:0/3937156569 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84ec15c3d0 0x7f84ec06ef50 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f84ec06d400 tx=0x7f84d402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=669435497e1db536 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:35.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.333+0000 7f84ebe00640 1 -- 172.21.15.131:0/3937156569 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d4037070 con 0x7f84ec15c3d0 2024-05-26T22:49:35.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.333+0000 7f84ebe00640 1 -- 172.21.15.131:0/3937156569 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84d4032440 con 0x7f84ec15c3d0 2024-05-26T22:49:35.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.334+0000 7f84ebe00640 1 -- 172.21.15.131:0/3937156569 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d4032810 con 0x7f84ec15c3d0 2024-05-26T22:49:35.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.334+0000 7f84f3a00640 1 -- 172.21.15.131:0/3937156569 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84ec15c3d0 msgr2=0x7f84ec06ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:35.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.334+0000 7f84f3a00640 1 --2- 172.21.15.131:0/3937156569 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84ec15c3d0 0x7f84ec06ef50 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f84ec06d400 tx=0x7f84d402a230 comp rx=0 tx=0).stop 2024-05-26T22:49:35.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.334+0000 7f84f3a00640 1 -- 172.21.15.131:0/3937156569 shutdown_connections 2024-05-26T22:49:35.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.334+0000 7f84f3a00640 1 --2- 172.21.15.131:0/3937156569 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84ec15c3d0 0x7f84ec06ef50 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:35.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.334+0000 7f84f3a00640 1 --2- 172.21.15.131:0/3937156569 >> v2:172.21.15.187:3300/0 conn(0x7f84ec155840 0x7f84ec071930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:35.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.334+0000 7f84f3a00640 1 -- 172.21.15.131:0/3937156569 >> 172.21.15.131:0/3937156569 conn(0x7f84ec0b1e90 msgr2=0x7f84ec0b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:35.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.334+0000 7f84f3a00640 1 -- 172.21.15.131:0/3937156569 shutdown_connections 2024-05-26T22:49:35.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.334+0000 7f84f3a00640 1 -- 172.21.15.131:0/3937156569 wait complete. 2024-05-26T22:49:35.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.336+0000 7f84f3a00640 1 Processor -- start 2024-05-26T22:49:35.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.336+0000 7f84f3a00640 1 -- start start 2024-05-26T22:49:35.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.337+0000 7f84f3a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84ec06f490 0x7f84ec0999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:35.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.337+0000 7f84f3a00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f84ec155840 0x7f84ec0b48d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:35.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.337+0000 7f84f3a00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f84ec15c3d0 0x7f84ec146cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:35.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.337+0000 7f84f3a00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f84ec071ff0 con 0x7f84ec06f490 2024-05-26T22:49:35.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.337+0000 7f84f3a00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f84ec071e70 con 0x7f84ec155840 2024-05-26T22:49:35.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.337+0000 7f84f3a00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f84ec0aee50 con 0x7f84ec15c3d0 2024-05-26T22:49:35.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.337+0000 7f84f1000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84ec06f490 0x7f84ec0999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:35.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.337+0000 7f84f1000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84ec06f490 0x7f84ec0999c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53168/0 (socket says 172.21.15.131:53168) 2024-05-26T22:49:35.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.337+0000 7f84f1000640 1 -- 172.21.15.131:0/1255677935 learned_addr learned my addr 172.21.15.131:0/1255677935 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:35.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.338+0000 7f84f1a00640 1 --2- 172.21.15.131:0/1255677935 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f84ec15c3d0 0x7f84ec146cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:35.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.338+0000 7f84f1000640 1 -- 172.21.15.131:0/1255677935 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f84ec15c3d0 msgr2=0x7f84ec146cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:35.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.338+0000 7f84e3e00640 1 --2- 172.21.15.131:0/1255677935 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f84ec155840 0x7f84ec0b48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:35.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.338+0000 7f84f1000640 1 --2- 172.21.15.131:0/1255677935 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f84ec15c3d0 0x7f84ec146cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:35.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.338+0000 7f84f1000640 1 -- 172.21.15.131:0/1255677935 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f84ec155840 msgr2=0x7f84ec0b48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:35.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.338+0000 7f84f1000640 1 --2- 172.21.15.131:0/1255677935 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f84ec155840 0x7f84ec0b48d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:35.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.338+0000 7f84f1000640 1 -- 172.21.15.131:0/1255677935 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84ec14b200 con 0x7f84ec06f490 2024-05-26T22:49:35.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.338+0000 7f84f1000640 1 --2- 172.21.15.131:0/1255677935 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84ec06f490 0x7f84ec0999c0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f84dc00ba80 tx=0x7f84dc00bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:35.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.338+0000 7f84e9600640 1 -- 172.21.15.131:0/1255677935 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84dc019070 con 0x7f84ec06f490 2024-05-26T22:49:35.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.338+0000 7f84e9600640 1 -- 172.21.15.131:0/1255677935 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84dc00cc20 con 0x7f84ec06f490 2024-05-26T22:49:35.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.338+0000 7f84e9600640 1 -- 172.21.15.131:0/1255677935 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84dc014440 con 0x7f84ec06f490 2024-05-26T22:49:35.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.338+0000 7f84f3a00640 1 -- 172.21.15.131:0/1255677935 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84ec14d450 con 0x7f84ec06f490 2024-05-26T22:49:35.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.338+0000 7f84f3a00640 1 -- 172.21.15.131:0/1255677935 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84ec0b2fe0 con 0x7f84ec06f490 2024-05-26T22:49:35.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.339+0000 7f84f3a00640 1 -- 172.21.15.131:0/1255677935 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84ec0694e0 con 0x7f84ec06f490 2024-05-26T22:49:35.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.340+0000 7f84e9600640 1 -- 172.21.15.131:0/1255677935 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f84dc01c070 con 0x7f84ec06f490 2024-05-26T22:49:35.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.341+0000 7f84e9600640 1 --2- 172.21.15.131:0/1255677935 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f84c806be60 0x7f84c806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:35.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.341+0000 7f84e9600640 1 -- 172.21.15.131:0/1255677935 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f84dc08ac00 con 0x7f84ec06f490 2024-05-26T22:49:35.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.341+0000 7f84e3e00640 1 --2- 172.21.15.131:0/1255677935 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f84c806be60 0x7f84c806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:35.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.342+0000 7f84e3e00640 1 --2- 172.21.15.131:0/1255677935 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f84c806be60 0x7f84c806e320 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f84d4034000 tx=0x7f84d402e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:35.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.346+0000 7f84e9600640 1 -- 172.21.15.131:0/1255677935 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f84dc0595a0 con 0x7f84ec06f490 2024-05-26T22:49:35.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.545+0000 7f84f3a00640 1 -- 172.21.15.131:0/1255677935 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f84ec152bf0 con 0x7f84ec06f490 2024-05-26T22:49:35.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.548+0000 7f84e9600640 1 -- 172.21.15.131:0/1255677935 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) ==== 74+0+16693 (secure 0 0 0) 0x7f84dc058ea0 con 0x7f84ec06f490 2024-05-26T22:49:35.548 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:49:35.548 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":33,"fsid":"46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f","created":"2024-05-26T22:47:57.288905+0000","modified":"2024-05-26T22:49:32.532828+0000","last_up_change":"2024-05-26T22:48:09.602207+0000","last_in_change":"2024-05-26T22:48:00.687341+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-26T22:48:10.463339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-05-26T22:49:28.813374+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-05-26T22:49:29.987297+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"814eaf62-a18d-494b-8e27-fd8fb88adea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6801","nonce":2350940051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6803","nonce":2350940051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6807","nonce":2350940051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6805","nonce":2350940051}]},"public_addr":"172.21.15.131:6801/2350940051","cluster_addr":"172.21.15.131:6803/2350940051","heartbeat_back_addr":"172.21.15.131:6807/2350940051","heartbeat_front_addr":"172.21.15.131:6805/2350940051","state":["exists","up"]},{"osd":1,"uuid":"42e66287-f6c6-4e26-af3c-6dfaa3536d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6825","nonce":1525528247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6827","nonce":1525528247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6831","nonce":1525528247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6829","nonce":1525528247}]},"public_addr":"172.21.15.131:6825/1525528247","cluster_addr":"172.21.15.131:6827/1525528247","heartbeat_back_addr":"172.21.15.131:6831/1525528247","heartbeat_front_addr":"172.21.15.131:6829/1525528247","state":["exists","up"]},{"osd":2,"uuid":"0da34673-04a2-4fd0-b57a-0ff2b679b043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6817","nonce":3627842210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6819","nonce":3627842210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6823","nonce":3627842210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6821","nonce":3627842210}]},"public_addr":"172.21.15.131:6817/3627842210","cluster_addr":"172.21.15.131:6819/3627842210","heartbeat_back_addr":"172.21.15.131:6823/3627842210","heartbeat_front_addr":"172.21.15.131:6821/3627842210","state":["exists","up"]},{"osd":3,"uuid":"cf117a27-d6b2-43d3-ae52-b47988bf8eed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6809","nonce":350790591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6811","nonce":350790591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6815","nonce":350790591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6813","nonce":350790591}]},"public_addr":"172.21.15.131:6809/350790591","cluster_addr":"172.21.15.131:6811/350790591","heartbeat_back_addr":"172.21.15.131:6815/350790591","heartbeat_front_addr":"172.21.15.131:6813/350790591","state":["exists","up"]},{"osd":4,"uuid":"a092ccfc-be89-4a84-88d5-0f781fa715eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6816","nonce":244560995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6819","nonce":244560995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6823","nonce":244560995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6821","nonce":244560995}]},"public_addr":"172.21.15.187:6816/244560995","cluster_addr":"172.21.15.187:6819/244560995","heartbeat_back_addr":"172.21.15.187:6823/244560995","heartbeat_front_addr":"172.21.15.187:6821/244560995","state":["exists","up"]},{"osd":5,"uuid":"7513ba69-dd00-4a35-b190-9e8cc3b2bdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6801","nonce":324619398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6803","nonce":324619398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6812","nonce":324619398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6807","nonce":324619398}]},"public_addr":"172.21.15.187:6801/324619398","cluster_addr":"172.21.15.187:6803/324619398","heartbeat_back_addr":"172.21.15.187:6812/324619398","heartbeat_front_addr":"172.21.15.187:6807/324619398","state":["exists","up"]},{"osd":6,"uuid":"50081810-c093-4d2f-a4d7-b7c695c5400d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6806","nonce":1391347786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6809","nonce":1391347786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6818","nonce":1391347786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6813","nonce":1391347786}]},"public_addr":"172.21.15.187:6806/1391347786","cluster_addr":"172.21.15.187:6809/1391347786","heartbeat_back_addr":"172.21.15.187:6818/1391347786","heartbeat_front_addr":"172.21.15.187:6813/1391347786","state":["exists","up"]},{"osd":7,"uuid":"775dae05-54f5-4ad4-bbce-92863c6ed2ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6825","nonce":1220065622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6827","nonce":1220065622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6831","nonce":1220065622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6829","nonce":1220065622}]},"public_addr":"172.21.15.187:6825/1220065622","cluster_addr":"172.21.15.187:6827/1220065622","heartbeat_back_addr":"172.21.15.187:6831/1220065622","heartbeat_front_addr":"172.21.15.187:6829/1220065622","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:05.983520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.299516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.067323+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.307748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.018742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.258775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.010912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.295355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-26T22:49:35.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.551+0000 7f84f3a00640 1 -- 172.21.15.131:0/1255677935 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f84c806be60 msgr2=0x7f84c806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:35.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.551+0000 7f84f3a00640 1 --2- 172.21.15.131:0/1255677935 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f84c806be60 0x7f84c806e320 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f84d4034000 tx=0x7f84d402e000 comp rx=0 tx=0).stop 2024-05-26T22:49:35.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.551+0000 7f84f3a00640 1 -- 172.21.15.131:0/1255677935 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84ec06f490 msgr2=0x7f84ec0999c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:35.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.551+0000 7f84f3a00640 1 --2- 172.21.15.131:0/1255677935 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84ec06f490 0x7f84ec0999c0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f84dc00ba80 tx=0x7f84dc00bf40 comp rx=0 tx=0).stop 2024-05-26T22:49:35.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.551+0000 7f84f3a00640 1 -- 172.21.15.131:0/1255677935 shutdown_connections 2024-05-26T22:49:35.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.551+0000 7f84f3a00640 1 --2- 172.21.15.131:0/1255677935 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f84ec15c3d0 0x7f84ec146cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:35.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.551+0000 7f84f3a00640 1 --2- 172.21.15.131:0/1255677935 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f84c806be60 0x7f84c806e320 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:35.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.551+0000 7f84f3a00640 1 --2- 172.21.15.131:0/1255677935 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f84ec155840 0x7f84ec0b48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:35.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.551+0000 7f84f3a00640 1 --2- 172.21.15.131:0/1255677935 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84ec06f490 0x7f84ec0999c0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:35.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.551+0000 7f84f3a00640 1 -- 172.21.15.131:0/1255677935 >> 172.21.15.131:0/1255677935 conn(0x7f84ec0b1e90 msgr2=0x7f84ec0cfeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:35.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.552+0000 7f84f3a00640 1 -- 172.21.15.131:0/1255677935 shutdown_connections 2024-05-26T22:49:35.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.552+0000 7f84f3a00640 1 -- 172.21.15.131:0/1255677935 wait complete. 2024-05-26T22:49:35.572 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-26T22:49:35.572 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-26T22:49:35.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.771+0000 7fddfa400640 1 Processor -- start 2024-05-26T22:49:35.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.771+0000 7fddfa400640 1 -- start start 2024-05-26T22:49:35.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.772+0000 7fddfa400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fddf4156750 0x7fddf415fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:35.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.772+0000 7fddfa400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fddf406e8c0 0x7fddf415d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:35.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.772+0000 7fddfa400640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fddf40cc7f0 0x7fddf40ccc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:35.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.772+0000 7fddfa400640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fddf40cd1a0 con 0x7fddf4156750 2024-05-26T22:49:35.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.772+0000 7fddfa400640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fddf415cbd0 con 0x7fddf40cc7f0 2024-05-26T22:49:35.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.772+0000 7fddfa400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fddf4160620 con 0x7fddf406e8c0 2024-05-26T22:49:35.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.773+0000 7fddf9000640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fddf4156750 0x7fddf415fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:35.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.773+0000 7fddf9000640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fddf4156750 0x7fddf415fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53182/0 (socket says 172.21.15.131:53182) 2024-05-26T22:49:35.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.773+0000 7fddf9000640 1 -- 172.21.15.131:0/2392617057 learned_addr learned my addr 172.21.15.131:0/2392617057 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:35.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.773+0000 7fddebe00640 1 --2- 172.21.15.131:0/2392617057 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fddf406e8c0 0x7fddf415d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:35.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.773+0000 7fddf9000640 1 -- 172.21.15.131:0/2392617057 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fddf406e8c0 msgr2=0x7fddf415d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:35.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.773+0000 7fddf9a00640 1 --2- 172.21.15.131:0/2392617057 >> v2:172.21.15.187:3300/0 conn(0x7fddf40cc7f0 0x7fddf40ccc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:35.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.773+0000 7fddf9000640 1 --2- 172.21.15.131:0/2392617057 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fddf406e8c0 0x7fddf415d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:35.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.773+0000 7fddf9000640 1 -- 172.21.15.131:0/2392617057 >> v2:172.21.15.187:3300/0 conn(0x7fddf40cc7f0 msgr2=0x7fddf40ccc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:35.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.773+0000 7fddf9000640 1 --2- 172.21.15.131:0/2392617057 >> v2:172.21.15.187:3300/0 conn(0x7fddf40cc7f0 0x7fddf40ccc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:35.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.773+0000 7fddf9000640 1 -- 172.21.15.131:0/2392617057 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddf4160900 con 0x7fddf4156750 2024-05-26T22:49:35.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.774+0000 7fddf9000640 1 --2- 172.21.15.131:0/2392617057 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fddf4156750 0x7fddf415fd40 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fddf406f570 tx=0x7fdde402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6521e0ad91c56aa2 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:35.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.775+0000 7fddf3e00640 1 -- 172.21.15.131:0/2392617057 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde4037070 con 0x7fddf4156750 2024-05-26T22:49:35.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.775+0000 7fddf3e00640 1 -- 172.21.15.131:0/2392617057 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdde4032440 con 0x7fddf4156750 2024-05-26T22:49:35.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.775+0000 7fddf3e00640 1 -- 172.21.15.131:0/2392617057 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde4032810 con 0x7fddf4156750 2024-05-26T22:49:35.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.775+0000 7fddfa400640 1 -- 172.21.15.131:0/2392617057 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fddf4156750 msgr2=0x7fddf415fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:35.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.775+0000 7fddfa400640 1 --2- 172.21.15.131:0/2392617057 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fddf4156750 0x7fddf415fd40 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fddf406f570 tx=0x7fdde402a1e0 comp rx=0 tx=0).stop 2024-05-26T22:49:35.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.775+0000 7fddfa400640 1 -- 172.21.15.131:0/2392617057 shutdown_connections 2024-05-26T22:49:35.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.775+0000 7fddfa400640 1 --2- 172.21.15.131:0/2392617057 >> v2:172.21.15.187:3300/0 conn(0x7fddf40cc7f0 0x7fddf40ccc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:35.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.775+0000 7fddfa400640 1 --2- 172.21.15.131:0/2392617057 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fddf406e8c0 0x7fddf415d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:35.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.775+0000 7fddfa400640 1 --2- 172.21.15.131:0/2392617057 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fddf4156750 0x7fddf415fd40 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:35.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.775+0000 7fddfa400640 1 -- 172.21.15.131:0/2392617057 >> 172.21.15.131:0/2392617057 conn(0x7fddf40cff40 msgr2=0x7fddf40d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:35.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.775+0000 7fddfa400640 1 -- 172.21.15.131:0/2392617057 shutdown_connections 2024-05-26T22:49:35.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.775+0000 7fddfa400640 1 -- 172.21.15.131:0/2392617057 wait complete. 2024-05-26T22:49:35.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.777+0000 7fddfa400640 1 Processor -- start 2024-05-26T22:49:35.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.777+0000 7fddfa400640 1 -- start start 2024-05-26T22:49:35.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.778+0000 7fddfa400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fddf406e8c0 0x7fddf415e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:35.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.778+0000 7fddfa400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fddf40cc7f0 0x7fddf41493b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:35.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.778+0000 7fddfa400640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fddf4156750 0x7fddf41498f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:35.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.778+0000 7fddfa400640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fddf4156b30 con 0x7fddf40cc7f0 2024-05-26T22:49:35.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.778+0000 7fddfa400640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fddf415c860 con 0x7fddf4156750 2024-05-26T22:49:35.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.778+0000 7fddfa400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fddf4160280 con 0x7fddf406e8c0 2024-05-26T22:49:35.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.778+0000 7fddebe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fddf40cc7f0 0x7fddf41493b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:35.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.779+0000 7fddebe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fddf40cc7f0 0x7fddf41493b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53188/0 (socket says 172.21.15.131:53188) 2024-05-26T22:49:35.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.779+0000 7fddebe00640 1 -- 172.21.15.131:0/4149279089 learned_addr learned my addr 172.21.15.131:0/4149279089 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:35.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.779+0000 7fddebe00640 1 -- 172.21.15.131:0/4149279089 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fddf406e8c0 msgr2=0x7fddf415e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:35.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.779+0000 7fddf9000640 1 --2- 172.21.15.131:0/4149279089 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fddf406e8c0 0x7fddf415e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:35.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.779+0000 7fddebe00640 1 --2- 172.21.15.131:0/4149279089 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fddf406e8c0 0x7fddf415e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:35.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.779+0000 7fddebe00640 1 -- 172.21.15.131:0/4149279089 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fddf4156750 msgr2=0x7fddf41498f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:35.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.779+0000 7fddebe00640 1 --2- 172.21.15.131:0/4149279089 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fddf4156750 0x7fddf41498f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:35.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.779+0000 7fddebe00640 1 -- 172.21.15.131:0/4149279089 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddf4153090 con 0x7fddf40cc7f0 2024-05-26T22:49:35.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.779+0000 7fddebe00640 1 --2- 172.21.15.131:0/4149279089 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fddf40cc7f0 0x7fddf41493b0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fdddc00c7f0 tx=0x7fdddc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:35.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.779+0000 7fddf1600640 1 -- 172.21.15.131:0/4149279089 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdddc015070 con 0x7fddf40cc7f0 2024-05-26T22:49:35.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.780+0000 7fddf1600640 1 -- 172.21.15.131:0/4149279089 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdddc004240 con 0x7fddf40cc7f0 2024-05-26T22:49:35.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.780+0000 7fddf1600640 1 -- 172.21.15.131:0/4149279089 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdddc007620 con 0x7fddf40cc7f0 2024-05-26T22:49:35.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.780+0000 7fddfa400640 1 -- 172.21.15.131:0/4149279089 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddf4153370 con 0x7fddf40cc7f0 2024-05-26T22:49:35.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.780+0000 7fddfa400640 1 -- 172.21.15.131:0/4149279089 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddf4153930 con 0x7fddf40cc7f0 2024-05-26T22:49:35.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.781+0000 7fddfa400640 1 -- 172.21.15.131:0/4149279089 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddf40694e0 con 0x7fddf40cc7f0 2024-05-26T22:49:35.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.782+0000 7fddf1600640 1 -- 172.21.15.131:0/4149279089 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdddc0079d0 con 0x7fddf40cc7f0 2024-05-26T22:49:35.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.783+0000 7fddf1600640 1 --2- 172.21.15.131:0/4149279089 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fddd006be30 0x7fddd006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:35.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.783+0000 7fddf1600640 1 -- 172.21.15.131:0/4149279089 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fdddc08b2c0 con 0x7fddf40cc7f0 2024-05-26T22:49:35.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.784+0000 7fddf9000640 1 --2- 172.21.15.131:0/4149279089 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fddd006be30 0x7fddd006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:35.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.784+0000 7fddf9000640 1 --2- 172.21.15.131:0/4149279089 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fddd006be30 0x7fddd006e2f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fdde4034000 tx=0x7fdde402e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:35.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.788+0000 7fddf1600640 1 -- 172.21.15.131:0/4149279089 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fdddc059c60 con 0x7fddf40cc7f0 2024-05-26T22:49:35.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:35.983+0000 7fddfa400640 1 -- 172.21.15.131:0/4149279089 --> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fddf415f750 con 0x7fddd006be30 2024-05-26T22:49:36.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.001+0000 7fddf1600640 1 -- 172.21.15.131:0/4149279089 <== mgr.4104 v2:172.21.15.131:6832/3484832947 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+239854 (secure 0 0 0) 0x7fddf415f750 con 0x7fddd006be30 2024-05-26T22:49:36.004 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:49:36.005 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-05-26T22:49:36.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.005+0000 7fddfa400640 1 -- 172.21.15.131:0/4149279089 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fddd006be30 msgr2=0x7fddd006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:36.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.005+0000 7fddfa400640 1 --2- 172.21.15.131:0/4149279089 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fddd006be30 0x7fddd006e2f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fdde4034000 tx=0x7fdde402e000 comp rx=0 tx=0).stop 2024-05-26T22:49:36.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.006+0000 7fddfa400640 1 -- 172.21.15.131:0/4149279089 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fddf40cc7f0 msgr2=0x7fddf41493b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:36.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.006+0000 7fddfa400640 1 --2- 172.21.15.131:0/4149279089 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fddf40cc7f0 0x7fddf41493b0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fdddc00c7f0 tx=0x7fdddc00ccb0 comp rx=0 tx=0).stop 2024-05-26T22:49:36.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.006+0000 7fddfa400640 1 -- 172.21.15.131:0/4149279089 shutdown_connections 2024-05-26T22:49:36.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.006+0000 7fddfa400640 1 --2- 172.21.15.131:0/4149279089 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fddd006be30 0x7fddd006e2f0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:36.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.006+0000 7fddfa400640 1 --2- 172.21.15.131:0/4149279089 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fddf4156750 0x7fddf41498f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:36.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.006+0000 7fddfa400640 1 --2- 172.21.15.131:0/4149279089 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fddf40cc7f0 0x7fddf41493b0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:36.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.006+0000 7fddfa400640 1 --2- 172.21.15.131:0/4149279089 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fddf406e8c0 0x7fddf415e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:36.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.006+0000 7fddfa400640 1 -- 172.21.15.131:0/4149279089 >> 172.21.15.131:0/4149279089 conn(0x7fddf40cff40 msgr2=0x7fddf40b4b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:36.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.006+0000 7fddfa400640 1 -- 172.21.15.131:0/4149279089 shutdown_connections 2024-05-26T22:49:36.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.006+0000 7fddfa400640 1 -- 172.21.15.131:0/4149279089 wait complete. 2024-05-26T22:49:36.025 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":68,"stamp":"2024-05-26T22:49:34.405515+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219952,"kb_used_data":3760,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754768,"statfs":{"total":773094113280,"available":772868882432,"internally_reserved":0,"allocated":3850240,"data_stored":2187230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.915105"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:49:30.488070+0000","last_change":"2024-05-26T22:49:30.488070+0000","last_active":"2024-05-26T22:49:30.488070+0000","last_peered":"2024-05-26T22:49:30.488070+0000","last_clean":"2024-05-26T22:49:30.488070+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:49:30.488070+0000","last_undegraded":"2024-05-26T22:49:30.488070+0000","last_fullsized":"2024-05-26T22:49:30.488070+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:49:30.488070+0000","last_change":"2024-05-26T22:49:30.488070+0000","last_active":"2024-05-26T22:49:30.488070+0000","last_peered":"2024-05-26T22:49:30.488070+0000","last_clean":"2024-05-26T22:49:30.488070+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:49:30.488070+0000","last_undegraded":"2024-05-26T22:49:30.488070+0000","last_fullsized":"2024-05-26T22:49:30.488070+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:49:30.488070+0000","last_change":"2024-05-26T22:49:30.488070+0000","last_active":"2024-05-26T22:49:30.488070+0000","last_peered":"2024-05-26T22:49:30.488070+0000","last_clean":"2024-05-26T22:49:30.488070+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:49:30.488070+0000","last_undegraded":"2024-05-26T22:49:30.488070+0000","last_fullsized":"2024-05-26T22:49:30.488070+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:49:30.488070+0000","last_change":"2024-05-26T22:49:30.488070+0000","last_active":"2024-05-26T22:49:30.488070+0000","last_peered":"2024-05-26T22:49:30.488070+0000","last_clean":"2024-05-26T22:49:30.488070+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:49:30.488070+0000","last_undegraded":"2024-05-26T22:49:30.488070+0000","last_fullsized":"2024-05-26T22:49:30.488070+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:49:30.488070+0000","last_change":"2024-05-26T22:49:30.488070+0000","last_active":"2024-05-26T22:49:30.488070+0000","last_peered":"2024-05-26T22:49:30.488070+0000","last_clean":"2024-05-26T22:49:30.488070+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:49:30.488070+0000","last_undegraded":"2024-05-26T22:49:30.488070+0000","last_fullsized":"2024-05-26T22:49:30.488070+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:49:30.488070+0000","last_change":"2024-05-26T22:49:30.488070+0000","last_active":"2024-05-26T22:49:30.488070+0000","last_peered":"2024-05-26T22:49:30.488070+0000","last_clean":"2024-05-26T22:49:30.488070+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:49:30.488070+0000","last_undegraded":"2024-05-26T22:49:30.488070+0000","last_fullsized":"2024-05-26T22:49:30.488070+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:49:30.488070+0000","last_change":"2024-05-26T22:49:30.488070+0000","last_active":"2024-05-26T22:49:30.488070+0000","last_peered":"2024-05-26T22:49:30.488070+0000","last_clean":"2024-05-26T22:49:30.488070+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:49:30.488070+0000","last_undegraded":"2024-05-26T22:49:30.488070+0000","last_fullsized":"2024-05-26T22:49:30.488070+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:49:30.488070+0000","last_change":"2024-05-26T22:49:30.488070+0000","last_active":"2024-05-26T22:49:30.488070+0000","last_peered":"2024-05-26T22:49:30.488070+0000","last_clean":"2024-05-26T22:49:30.488070+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:49:30.488070+0000","last_undegraded":"2024-05-26T22:49:30.488070+0000","last_fullsized":"2024-05-26T22:49:30.488070+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:49:30.488070+0000","last_change":"2024-05-26T22:49:30.488070+0000","last_active":"2024-05-26T22:49:30.488070+0000","last_peered":"2024-05-26T22:49:30.488070+0000","last_clean":"2024-05-26T22:49:30.488070+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:49:30.488070+0000","last_undegraded":"2024-05-26T22:49:30.488070+0000","last_fullsized":"2024-05-26T22:49:30.488070+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:49:30.488070+0000","last_change":"2024-05-26T22:49:30.488070+0000","last_active":"2024-05-26T22:49:30.488070+0000","last_peered":"2024-05-26T22:49:30.488070+0000","last_clean":"2024-05-26T22:49:30.488070+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:49:30.488070+0000","last_undegraded":"2024-05-26T22:49:30.488070+0000","last_fullsized":"2024-05-26T22:49:30.488070+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:49:30.488070+0000","last_change":"2024-05-26T22:49:30.488070+0000","last_active":"2024-05-26T22:49:30.488070+0000","last_peered":"2024-05-26T22:49:30.488070+0000","last_clean":"2024-05-26T22:49:30.488070+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:49:30.488070+0000","last_undegraded":"2024-05-26T22:49:30.488070+0000","last_fullsized":"2024-05-26T22:49:30.488070+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:49:30.488070+0000","last_change":"2024-05-26T22:49:30.488070+0000","last_active":"2024-05-26T22:49:30.488070+0000","last_peered":"2024-05-26T22:49:30.488070+0000","last_clean":"2024-05-26T22:49:30.488070+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:49:30.488070+0000","last_undegraded":"2024-05-26T22:49:30.488070+0000","last_fullsized":"2024-05-26T22:49:30.488070+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:49:30.488070+0000","last_change":"2024-05-26T22:49:30.488070+0000","last_active":"2024-05-26T22:49:30.488070+0000","last_peered":"2024-05-26T22:49:30.488070+0000","last_clean":"2024-05-26T22:49:30.488070+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:49:30.488070+0000","last_undegraded":"2024-05-26T22:49:30.488070+0000","last_fullsized":"2024-05-26T22:49:30.488070+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:49:30.488070+0000","last_change":"2024-05-26T22:49:30.488070+0000","last_active":"2024-05-26T22:49:30.488070+0000","last_peered":"2024-05-26T22:49:30.488070+0000","last_clean":"2024-05-26T22:49:30.488070+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:49:30.488070+0000","last_undegraded":"2024-05-26T22:49:30.488070+0000","last_fullsized":"2024-05-26T22:49:30.488070+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:49:30.488070+0000","last_change":"2024-05-26T22:49:30.488070+0000","last_active":"2024-05-26T22:49:30.488070+0000","last_peered":"2024-05-26T22:49:30.488070+0000","last_clean":"2024-05-26T22:49:30.488070+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:49:30.488070+0000","last_undegraded":"2024-05-26T22:49:30.488070+0000","last_fullsized":"2024-05-26T22:49:30.488070+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:49:30.488070+0000","last_change":"2024-05-26T22:49:30.488070+0000","last_active":"2024-05-26T22:49:30.488070+0000","last_peered":"2024-05-26T22:49:30.488070+0000","last_clean":"2024-05-26T22:49:30.488070+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:49:30.488070+0000","last_undegraded":"2024-05-26T22:49:30.488070+0000","last_fullsized":"2024-05-26T22:49:30.488070+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:49:30.488070+0000","last_change":"2024-05-26T22:49:30.488070+0000","last_active":"2024-05-26T22:49:30.488070+0000","last_peered":"2024-05-26T22:49:30.488070+0000","last_clean":"2024-05-26T22:49:30.488070+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:49:30.488070+0000","last_undegraded":"2024-05-26T22:49:30.488070+0000","last_fullsized":"2024-05-26T22:49:30.488070+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:49:30.488070+0000","last_change":"2024-05-26T22:49:30.488070+0000","last_active":"2024-05-26T22:49:30.488070+0000","last_peered":"2024-05-26T22:49:30.488070+0000","last_clean":"2024-05-26T22:49:30.488070+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:49:30.488070+0000","last_undegraded":"2024-05-26T22:49:30.488070+0000","last_fullsized":"2024-05-26T22:49:30.488070+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:49:30.488070+0000","last_change":"2024-05-26T22:49:30.488070+0000","last_active":"2024-05-26T22:49:30.488070+0000","last_peered":"2024-05-26T22:49:30.488070+0000","last_clean":"2024-05-26T22:49:30.488070+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:49:30.488070+0000","last_undegraded":"2024-05-26T22:49:30.488070+0000","last_fullsized":"2024-05-26T22:49:30.488070+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:49:30.488070+0000","last_change":"2024-05-26T22:49:30.488070+0000","last_active":"2024-05-26T22:49:30.488070+0000","last_peered":"2024-05-26T22:49:30.488070+0000","last_clean":"2024-05-26T22:49:30.488070+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:49:30.488070+0000","last_undegraded":"2024-05-26T22:49:30.488070+0000","last_fullsized":"2024-05-26T22:49:30.488070+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:49:30.488070+0000","last_change":"2024-05-26T22:49:30.488070+0000","last_active":"2024-05-26T22:49:30.488070+0000","last_peered":"2024-05-26T22:49:30.488070+0000","last_clean":"2024-05-26T22:49:30.488070+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:49:30.488070+0000","last_undegraded":"2024-05-26T22:49:30.488070+0000","last_fullsized":"2024-05-26T22:49:30.488070+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:49:30.488070+0000","last_change":"2024-05-26T22:49:30.488070+0000","last_active":"2024-05-26T22:49:30.488070+0000","last_peered":"2024-05-26T22:49:30.488070+0000","last_clean":"2024-05-26T22:49:30.488070+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:49:30.488070+0000","last_undegraded":"2024-05-26T22:49:30.488070+0000","last_fullsized":"2024-05-26T22:49:30.488070+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:49:30.488070+0000","last_change":"2024-05-26T22:49:30.488070+0000","last_active":"2024-05-26T22:49:30.488070+0000","last_peered":"2024-05-26T22:49:30.488070+0000","last_clean":"2024-05-26T22:49:30.488070+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:49:30.488070+0000","last_undegraded":"2024-05-26T22:49:30.488070+0000","last_fullsized":"2024-05-26T22:49:30.488070+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:49:30.488070+0000","last_change":"2024-05-26T22:49:30.488070+0000","last_active":"2024-05-26T22:49:30.488070+0000","last_peered":"2024-05-26T22:49:30.488070+0000","last_clean":"2024-05-26T22:49:30.488070+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:49:30.488070+0000","last_undegraded":"2024-05-26T22:49:30.488070+0000","last_fullsized":"2024-05-26T22:49:30.488070+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:49:30.488070+0000","last_change":"2024-05-26T22:49:30.488070+0000","last_active":"2024-05-26T22:49:30.488070+0000","last_peered":"2024-05-26T22:49:30.488070+0000","last_clean":"2024-05-26T22:49:30.488070+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:49:30.488070+0000","last_undegraded":"2024-05-26T22:49:30.488070+0000","last_fullsized":"2024-05-26T22:49:30.488070+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:49:30.488070+0000","last_change":"2024-05-26T22:49:30.488070+0000","last_active":"2024-05-26T22:49:30.488070+0000","last_peered":"2024-05-26T22:49:30.488070+0000","last_clean":"2024-05-26T22:49:30.488070+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:49:30.488070+0000","last_undegraded":"2024-05-26T22:49:30.488070+0000","last_fullsized":"2024-05-26T22:49:30.488070+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:49:30.488070+0000","last_change":"2024-05-26T22:49:30.488070+0000","last_active":"2024-05-26T22:49:30.488070+0000","last_peered":"2024-05-26T22:49:30.488070+0000","last_clean":"2024-05-26T22:49:30.488070+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:49:30.488070+0000","last_undegraded":"2024-05-26T22:49:30.488070+0000","last_fullsized":"2024-05-26T22:49:30.488070+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:49:30.488070+0000","last_change":"2024-05-26T22:49:30.488070+0000","last_active":"2024-05-26T22:49:30.488070+0000","last_peered":"2024-05-26T22:49:30.488070+0000","last_clean":"2024-05-26T22:49:30.488070+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:49:30.488070+0000","last_undegraded":"2024-05-26T22:49:30.488070+0000","last_fullsized":"2024-05-26T22:49:30.488070+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:49:30.488070+0000","last_change":"2024-05-26T22:49:30.488070+0000","last_active":"2024-05-26T22:49:30.488070+0000","last_peered":"2024-05-26T22:49:30.488070+0000","last_clean":"2024-05-26T22:49:30.488070+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:49:30.488070+0000","last_undegraded":"2024-05-26T22:49:30.488070+0000","last_fullsized":"2024-05-26T22:49:30.488070+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:49:30.488070+0000","last_change":"2024-05-26T22:49:30.488070+0000","last_active":"2024-05-26T22:49:30.488070+0000","last_peered":"2024-05-26T22:49:30.488070+0000","last_clean":"2024-05-26T22:49:30.488070+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:49:30.488070+0000","last_undegraded":"2024-05-26T22:49:30.488070+0000","last_fullsized":"2024-05-26T22:49:30.488070+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:49:30.488070+0000","last_change":"2024-05-26T22:49:30.488070+0000","last_active":"2024-05-26T22:49:30.488070+0000","last_peered":"2024-05-26T22:49:30.488070+0000","last_clean":"2024-05-26T22:49:30.488070+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:49:30.488070+0000","last_undegraded":"2024-05-26T22:49:30.488070+0000","last_fullsized":"2024-05-26T22:49:30.488070+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:49:30.488070+0000","last_change":"2024-05-26T22:49:30.488070+0000","last_active":"2024-05-26T22:49:30.488070+0000","last_peered":"2024-05-26T22:49:30.488070+0000","last_clean":"2024-05-26T22:49:30.488070+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:49:30.488070+0000","last_undegraded":"2024-05-26T22:49:30.488070+0000","last_fullsized":"2024-05-26T22:49:30.488070+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:49:30.488070+0000","last_change":"2024-05-26T22:49:30.488070+0000","last_active":"2024-05-26T22:49:30.488070+0000","last_peered":"2024-05-26T22:49:30.488070+0000","last_clean":"2024-05-26T22:49:30.488070+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:49:30.488070+0000","last_undegraded":"2024-05-26T22:49:30.488070+0000","last_fullsized":"2024-05-26T22:49:30.488070+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-05-26T22:49:29.495689+0000","last_change":"2024-05-26T22:49:29.495689+0000","last_active":"2024-05-26T22:49:29.474733+0000","last_peered":"2024-05-26T22:49:29.474733+0000","last_clean":"2024-05-26T22:49:29.474733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:49:29.495689+0000","last_undegraded":"2024-05-26T22:49:29.495689+0000","last_fullsized":"2024-05-26T22:49:29.495689+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_clean_scrub_stamp":"2024-05-26T22:49:29.474733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:49:30.488070+0000","last_change":"2024-05-26T22:49:30.488070+0000","last_active":"2024-05-26T22:49:30.488070+0000","last_peered":"2024-05-26T22:49:30.488070+0000","last_clean":"2024-05-26T22:49:30.488070+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:49:30.488070+0000","last_undegraded":"2024-05-26T22:49:30.488070+0000","last_fullsized":"2024-05-26T22:49:30.488070+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-05-26T22:49:30.081197+0000","last_change":"2024-05-26T22:49:30.081197+0000","last_active":"2024-05-26T22:49:29.474733+0000","last_peered":"2024-05-26T22:49:29.474733+0000","last_clean":"2024-05-26T22:49:29.474733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:49:30.081197+0000","last_undegraded":"2024-05-26T22:49:30.081197+0000","last_fullsized":"2024-05-26T22:49:30.081197+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_clean_scrub_stamp":"2024-05-26T22:49:29.474733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:49:30.488070+0000","last_change":"2024-05-26T22:49:30.488070+0000","last_active":"2024-05-26T22:49:30.488070+0000","last_peered":"2024-05-26T22:49:30.488070+0000","last_clean":"2024-05-26T22:49:30.488070+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:49:30.488070+0000","last_undegraded":"2024-05-26T22:49:30.488070+0000","last_fullsized":"2024-05-26T22:49:30.488070+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-05-26T22:49:29.505330+0000","last_change":"2024-05-26T22:49:29.505330+0000","last_active":"2024-05-26T22:49:29.474733+0000","last_peered":"2024-05-26T22:49:29.474733+0000","last_clean":"2024-05-26T22:49:29.474733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:49:29.505330+0000","last_undegraded":"2024-05-26T22:49:29.505330+0000","last_fullsized":"2024-05-26T22:49:29.505330+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_clean_scrub_stamp":"2024-05-26T22:49:29.474733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:49:30.488070+0000","last_change":"2024-05-26T22:49:30.488070+0000","last_active":"2024-05-26T22:49:30.488070+0000","last_peered":"2024-05-26T22:49:30.488070+0000","last_clean":"2024-05-26T22:49:30.488070+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:49:30.488070+0000","last_undegraded":"2024-05-26T22:49:30.488070+0000","last_fullsized":"2024-05-26T22:49:30.488070+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-05-26T22:49:29.495411+0000","last_change":"2024-05-26T22:49:29.495411+0000","last_active":"2024-05-26T22:49:29.474733+0000","last_peered":"2024-05-26T22:49:29.474733+0000","last_clean":"2024-05-26T22:49:29.474733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:49:29.495411+0000","last_undegraded":"2024-05-26T22:49:29.495411+0000","last_fullsized":"2024-05-26T22:49:29.495411+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_clean_scrub_stamp":"2024-05-26T22:49:29.474733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:49:30.488070+0000","last_change":"2024-05-26T22:49:30.488070+0000","last_active":"2024-05-26T22:49:30.488070+0000","last_peered":"2024-05-26T22:49:30.488070+0000","last_clean":"2024-05-26T22:49:30.488070+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:49:30.488070+0000","last_undegraded":"2024-05-26T22:49:30.488070+0000","last_fullsized":"2024-05-26T22:49:30.488070+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-05-26T22:49:29.498931+0000","last_change":"2024-05-26T22:49:29.498931+0000","last_active":"2024-05-26T22:49:29.474733+0000","last_peered":"2024-05-26T22:49:29.474733+0000","last_clean":"2024-05-26T22:49:29.474733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:49:29.498931+0000","last_undegraded":"2024-05-26T22:49:29.498931+0000","last_fullsized":"2024-05-26T22:49:29.498931+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_clean_scrub_stamp":"2024-05-26T22:49:29.474733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:49:30.488070+0000","last_change":"2024-05-26T22:49:30.488070+0000","last_active":"2024-05-26T22:49:30.488070+0000","last_peered":"2024-05-26T22:49:30.488070+0000","last_clean":"2024-05-26T22:49:30.488070+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:49:30.488070+0000","last_undegraded":"2024-05-26T22:49:30.488070+0000","last_fullsized":"2024-05-26T22:49:30.488070+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-05-26T22:49:29.496390+0000","last_change":"2024-05-26T22:49:29.496390+0000","last_active":"2024-05-26T22:49:29.474733+0000","last_peered":"2024-05-26T22:49:29.474733+0000","last_clean":"2024-05-26T22:49:29.474733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:49:29.496390+0000","last_undegraded":"2024-05-26T22:49:29.496390+0000","last_fullsized":"2024-05-26T22:49:29.496390+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_clean_scrub_stamp":"2024-05-26T22:49:29.474733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:49:30.488070+0000","last_change":"2024-05-26T22:49:30.488070+0000","last_active":"2024-05-26T22:49:30.488070+0000","last_peered":"2024-05-26T22:49:30.488070+0000","last_clean":"2024-05-26T22:49:30.488070+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:49:30.488070+0000","last_undegraded":"2024-05-26T22:49:30.488070+0000","last_fullsized":"2024-05-26T22:49:30.488070+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-05-26T22:49:29.502429+0000","last_change":"2024-05-26T22:49:29.502429+0000","last_active":"2024-05-26T22:49:29.474733+0000","last_peered":"2024-05-26T22:49:29.474733+0000","last_clean":"2024-05-26T22:49:29.474733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:49:29.502429+0000","last_undegraded":"2024-05-26T22:49:29.502429+0000","last_fullsized":"2024-05-26T22:49:29.502429+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_clean_scrub_stamp":"2024-05-26T22:49:29.474733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:49:30.488070+0000","last_change":"2024-05-26T22:49:30.488070+0000","last_active":"2024-05-26T22:49:30.488070+0000","last_peered":"2024-05-26T22:49:30.488070+0000","last_clean":"2024-05-26T22:49:30.488070+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:49:30.488070+0000","last_undegraded":"2024-05-26T22:49:30.488070+0000","last_fullsized":"2024-05-26T22:49:30.488070+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-05-26T22:49:29.496969+0000","last_change":"2024-05-26T22:49:29.496969+0000","last_active":"2024-05-26T22:49:29.474733+0000","last_peered":"2024-05-26T22:49:29.474733+0000","last_clean":"2024-05-26T22:49:29.474733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:49:29.496969+0000","last_undegraded":"2024-05-26T22:49:29.496969+0000","last_fullsized":"2024-05-26T22:49:29.496969+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_clean_scrub_stamp":"2024-05-26T22:49:29.474733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:49:30.488070+0000","last_change":"2024-05-26T22:49:30.488070+0000","last_active":"2024-05-26T22:49:30.488070+0000","last_peered":"2024-05-26T22:49:30.488070+0000","last_clean":"2024-05-26T22:49:30.488070+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:49:30.488070+0000","last_undegraded":"2024-05-26T22:49:30.488070+0000","last_fullsized":"2024-05-26T22:49:30.488070+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-05-26T22:49:29.501855+0000","last_change":"2024-05-26T22:49:29.501855+0000","last_active":"2024-05-26T22:49:29.474733+0000","last_peered":"2024-05-26T22:49:29.474733+0000","last_clean":"2024-05-26T22:49:29.474733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:49:29.501855+0000","last_undegraded":"2024-05-26T22:49:29.501855+0000","last_fullsized":"2024-05-26T22:49:29.501855+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_clean_scrub_stamp":"2024-05-26T22:49:29.474733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:49:30.488070+0000","last_change":"2024-05-26T22:49:30.488070+0000","last_active":"2024-05-26T22:49:30.488070+0000","last_peered":"2024-05-26T22:49:30.488070+0000","last_clean":"2024-05-26T22:49:30.488070+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:49:30.488070+0000","last_undegraded":"2024-05-26T22:49:30.488070+0000","last_fullsized":"2024-05-26T22:49:30.488070+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-05-26T22:49:30.078831+0000","last_change":"2024-05-26T22:49:30.078831+0000","last_active":"2024-05-26T22:49:29.474733+0000","last_peered":"2024-05-26T22:49:29.474733+0000","last_clean":"2024-05-26T22:49:29.474733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:49:30.078831+0000","last_undegraded":"2024-05-26T22:49:30.078831+0000","last_fullsized":"2024-05-26T22:49:30.078831+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_clean_scrub_stamp":"2024-05-26T22:49:29.474733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:49:30.488070+0000","last_change":"2024-05-26T22:49:30.488070+0000","last_active":"2024-05-26T22:49:30.488070+0000","last_peered":"2024-05-26T22:49:30.488070+0000","last_clean":"2024-05-26T22:49:30.488070+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:49:30.488070+0000","last_undegraded":"2024-05-26T22:49:30.488070+0000","last_fullsized":"2024-05-26T22:49:30.488070+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-05-26T22:49:29.505541+0000","last_change":"2024-05-26T22:49:29.505541+0000","last_active":"2024-05-26T22:49:29.474733+0000","last_peered":"2024-05-26T22:49:29.474733+0000","last_clean":"2024-05-26T22:49:29.474733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:49:29.505541+0000","last_undegraded":"2024-05-26T22:49:29.505541+0000","last_fullsized":"2024-05-26T22:49:29.505541+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_clean_scrub_stamp":"2024-05-26T22:49:29.474733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:49:30.488070+0000","last_change":"2024-05-26T22:49:30.488070+0000","last_active":"2024-05-26T22:49:30.488070+0000","last_peered":"2024-05-26T22:49:30.488070+0000","last_clean":"2024-05-26T22:49:30.488070+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:49:30.488070+0000","last_undegraded":"2024-05-26T22:49:30.488070+0000","last_fullsized":"2024-05-26T22:49:30.488070+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-05-26T22:49:29.503990+0000","last_change":"2024-05-26T22:49:29.503990+0000","last_active":"2024-05-26T22:49:29.474733+0000","last_peered":"2024-05-26T22:49:29.474733+0000","last_clean":"2024-05-26T22:49:29.474733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:49:29.503990+0000","last_undegraded":"2024-05-26T22:49:29.503990+0000","last_fullsized":"2024-05-26T22:49:29.503990+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_clean_scrub_stamp":"2024-05-26T22:49:29.474733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:49:30.488070+0000","last_change":"2024-05-26T22:49:30.488070+0000","last_active":"2024-05-26T22:49:30.488070+0000","last_peered":"2024-05-26T22:49:30.488070+0000","last_clean":"2024-05-26T22:49:30.488070+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:49:30.488070+0000","last_undegraded":"2024-05-26T22:49:30.488070+0000","last_fullsized":"2024-05-26T22:49:30.488070+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-05-26T22:49:29.497661+0000","last_change":"2024-05-26T22:49:29.497661+0000","last_active":"2024-05-26T22:49:29.474733+0000","last_peered":"2024-05-26T22:49:29.474733+0000","last_clean":"2024-05-26T22:49:29.474733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:49:29.497661+0000","last_undegraded":"2024-05-26T22:49:29.497661+0000","last_fullsized":"2024-05-26T22:49:29.497661+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_clean_scrub_stamp":"2024-05-26T22:49:29.474733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:49:30.488070+0000","last_change":"2024-05-26T22:49:30.488070+0000","last_active":"2024-05-26T22:49:30.488070+0000","last_peered":"2024-05-26T22:49:30.488070+0000","last_clean":"2024-05-26T22:49:30.488070+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:49:30.488070+0000","last_undegraded":"2024-05-26T22:49:30.488070+0000","last_fullsized":"2024-05-26T22:49:30.488070+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-05-26T22:49:29.500320+0000","last_change":"2024-05-26T22:49:29.500320+0000","last_active":"2024-05-26T22:49:29.474733+0000","last_peered":"2024-05-26T22:49:29.474733+0000","last_clean":"2024-05-26T22:49:29.474733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:49:29.500320+0000","last_undegraded":"2024-05-26T22:49:29.500320+0000","last_fullsized":"2024-05-26T22:49:29.500320+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_clean_scrub_stamp":"2024-05-26T22:49:29.474733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:49:30.488070+0000","last_change":"2024-05-26T22:49:30.488070+0000","last_active":"2024-05-26T22:49:30.488070+0000","last_peered":"2024-05-26T22:49:30.488070+0000","last_clean":"2024-05-26T22:49:30.488070+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:49:30.488070+0000","last_undegraded":"2024-05-26T22:49:30.488070+0000","last_fullsized":"2024-05-26T22:49:30.488070+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-05-26T22:49:29.503382+0000","last_change":"2024-05-26T22:49:29.503382+0000","last_active":"2024-05-26T22:49:29.474733+0000","last_peered":"2024-05-26T22:49:29.474733+0000","last_clean":"2024-05-26T22:49:29.474733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:49:29.503382+0000","last_undegraded":"2024-05-26T22:49:29.503382+0000","last_fullsized":"2024-05-26T22:49:29.503382+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_clean_scrub_stamp":"2024-05-26T22:49:29.474733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:49:30.488070+0000","last_change":"2024-05-26T22:49:30.488070+0000","last_active":"2024-05-26T22:49:30.488070+0000","last_peered":"2024-05-26T22:49:30.488070+0000","last_clean":"2024-05-26T22:49:30.488070+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:49:30.488070+0000","last_undegraded":"2024-05-26T22:49:30.488070+0000","last_fullsized":"2024-05-26T22:49:30.488070+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-05-26T22:49:29.501635+0000","last_change":"2024-05-26T22:49:29.501635+0000","last_active":"2024-05-26T22:49:29.474733+0000","last_peered":"2024-05-26T22:49:29.474733+0000","last_clean":"2024-05-26T22:49:29.474733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:49:29.501635+0000","last_undegraded":"2024-05-26T22:49:29.501635+0000","last_fullsized":"2024-05-26T22:49:29.501635+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_clean_scrub_stamp":"2024-05-26T22:49:29.474733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:49:30.488070+0000","last_change":"2024-05-26T22:49:30.488070+0000","last_active":"2024-05-26T22:49:30.488070+0000","last_peered":"2024-05-26T22:49:30.488070+0000","last_clean":"2024-05-26T22:49:30.488070+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:49:30.488070+0000","last_undegraded":"2024-05-26T22:49:30.488070+0000","last_fullsized":"2024-05-26T22:49:30.488070+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-05-26T22:49:29.501222+0000","last_change":"2024-05-26T22:49:29.501222+0000","last_active":"2024-05-26T22:49:29.474733+0000","last_peered":"2024-05-26T22:49:29.474733+0000","last_clean":"2024-05-26T22:49:29.474733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:49:29.501222+0000","last_undegraded":"2024-05-26T22:49:29.501222+0000","last_fullsized":"2024-05-26T22:49:29.501222+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_clean_scrub_stamp":"2024-05-26T22:49:29.474733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:49:30.488070+0000","last_change":"2024-05-26T22:49:30.488070+0000","last_active":"2024-05-26T22:49:30.488070+0000","last_peered":"2024-05-26T22:49:30.488070+0000","last_clean":"2024-05-26T22:49:30.488070+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:49:30.488070+0000","last_undegraded":"2024-05-26T22:49:30.488070+0000","last_fullsized":"2024-05-26T22:49:30.488070+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-05-26T22:49:29.499763+0000","last_change":"2024-05-26T22:49:29.499763+0000","last_active":"2024-05-26T22:49:29.474733+0000","last_peered":"2024-05-26T22:49:29.474733+0000","last_clean":"2024-05-26T22:49:29.474733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:49:29.499763+0000","last_undegraded":"2024-05-26T22:49:29.499763+0000","last_fullsized":"2024-05-26T22:49:29.499763+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_clean_scrub_stamp":"2024-05-26T22:49:29.474733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:49:30.488070+0000","last_change":"2024-05-26T22:49:30.488070+0000","last_active":"2024-05-26T22:49:30.488070+0000","last_peered":"2024-05-26T22:49:30.488070+0000","last_clean":"2024-05-26T22:49:30.488070+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:49:30.488070+0000","last_undegraded":"2024-05-26T22:49:30.488070+0000","last_fullsized":"2024-05-26T22:49:30.488070+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-05-26T22:49:29.499469+0000","last_change":"2024-05-26T22:49:29.499469+0000","last_active":"2024-05-26T22:49:29.474733+0000","last_peered":"2024-05-26T22:49:29.474733+0000","last_clean":"2024-05-26T22:49:29.474733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:49:29.499469+0000","last_undegraded":"2024-05-26T22:49:29.499469+0000","last_fullsized":"2024-05-26T22:49:29.499469+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_clean_scrub_stamp":"2024-05-26T22:49:29.474733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:49:30.488070+0000","last_change":"2024-05-26T22:49:30.488070+0000","last_active":"2024-05-26T22:49:30.488070+0000","last_peered":"2024-05-26T22:49:30.488070+0000","last_clean":"2024-05-26T22:49:30.488070+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:49:30.488070+0000","last_undegraded":"2024-05-26T22:49:30.488070+0000","last_fullsized":"2024-05-26T22:49:30.488070+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-05-26T22:49:29.498241+0000","last_change":"2024-05-26T22:49:29.498241+0000","last_active":"2024-05-26T22:49:29.474733+0000","last_peered":"2024-05-26T22:49:29.474733+0000","last_clean":"2024-05-26T22:49:29.474733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:49:29.498241+0000","last_undegraded":"2024-05-26T22:49:29.498241+0000","last_fullsized":"2024-05-26T22:49:29.498241+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_clean_scrub_stamp":"2024-05-26T22:49:29.474733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:49:30.488070+0000","last_change":"2024-05-26T22:49:30.488070+0000","last_active":"2024-05-26T22:49:30.488070+0000","last_peered":"2024-05-26T22:49:30.488070+0000","last_clean":"2024-05-26T22:49:30.488070+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:49:30.488070+0000","last_undegraded":"2024-05-26T22:49:30.488070+0000","last_fullsized":"2024-05-26T22:49:30.488070+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-05-26T22:49:30.080224+0000","last_change":"2024-05-26T22:49:30.080224+0000","last_active":"2024-05-26T22:49:29.474733+0000","last_peered":"2024-05-26T22:49:29.474733+0000","last_clean":"2024-05-26T22:49:29.474733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:49:30.080224+0000","last_undegraded":"2024-05-26T22:49:30.080224+0000","last_fullsized":"2024-05-26T22:49:30.080224+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_clean_scrub_stamp":"2024-05-26T22:49:29.474733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:49:30.488070+0000","last_change":"2024-05-26T22:49:30.488070+0000","last_active":"2024-05-26T22:49:30.488070+0000","last_peered":"2024-05-26T22:49:30.488070+0000","last_clean":"2024-05-26T22:49:30.488070+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:49:30.488070+0000","last_undegraded":"2024-05-26T22:49:30.488070+0000","last_fullsized":"2024-05-26T22:49:30.488070+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-05-26T22:49:29.495311+0000","last_change":"2024-05-26T22:49:29.495311+0000","last_active":"2024-05-26T22:49:29.474733+0000","last_peered":"2024-05-26T22:49:29.474733+0000","last_clean":"2024-05-26T22:49:29.474733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:49:29.495311+0000","last_undegraded":"2024-05-26T22:49:29.495311+0000","last_fullsized":"2024-05-26T22:49:29.495311+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_clean_scrub_stamp":"2024-05-26T22:49:29.474733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:49:30.488070+0000","last_change":"2024-05-26T22:49:30.488070+0000","last_active":"2024-05-26T22:49:30.488070+0000","last_peered":"2024-05-26T22:49:30.488070+0000","last_clean":"2024-05-26T22:49:30.488070+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:49:30.488070+0000","last_undegraded":"2024-05-26T22:49:30.488070+0000","last_fullsized":"2024-05-26T22:49:30.488070+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-05-26T22:49:29.498131+0000","last_change":"2024-05-26T22:49:29.498131+0000","last_active":"2024-05-26T22:49:29.474733+0000","last_peered":"2024-05-26T22:49:29.474733+0000","last_clean":"2024-05-26T22:49:29.474733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:49:29.498131+0000","last_undegraded":"2024-05-26T22:49:29.498131+0000","last_fullsized":"2024-05-26T22:49:29.498131+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_clean_scrub_stamp":"2024-05-26T22:49:29.474733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:49:30.488070+0000","last_change":"2024-05-26T22:49:30.488070+0000","last_active":"2024-05-26T22:49:30.488070+0000","last_peered":"2024-05-26T22:49:30.488070+0000","last_clean":"2024-05-26T22:49:30.488070+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:49:30.488070+0000","last_undegraded":"2024-05-26T22:49:30.488070+0000","last_fullsized":"2024-05-26T22:49:30.488070+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-05-26T22:49:29.503626+0000","last_change":"2024-05-26T22:49:29.503626+0000","last_active":"2024-05-26T22:49:29.474733+0000","last_peered":"2024-05-26T22:49:29.474733+0000","last_clean":"2024-05-26T22:49:29.474733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:49:29.503626+0000","last_undegraded":"2024-05-26T22:49:29.503626+0000","last_fullsized":"2024-05-26T22:49:29.503626+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_clean_scrub_stamp":"2024-05-26T22:49:29.474733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:49:30.488070+0000","last_change":"2024-05-26T22:49:30.488070+0000","last_active":"2024-05-26T22:49:30.488070+0000","last_peered":"2024-05-26T22:49:30.488070+0000","last_clean":"2024-05-26T22:49:30.488070+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:49:30.488070+0000","last_undegraded":"2024-05-26T22:49:30.488070+0000","last_fullsized":"2024-05-26T22:49:30.488070+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-05-26T22:49:29.500157+0000","last_change":"2024-05-26T22:49:29.500157+0000","last_active":"2024-05-26T22:49:29.474733+0000","last_peered":"2024-05-26T22:49:29.474733+0000","last_clean":"2024-05-26T22:49:29.474733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:49:29.500157+0000","last_undegraded":"2024-05-26T22:49:29.500157+0000","last_fullsized":"2024-05-26T22:49:29.500157+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_clean_scrub_stamp":"2024-05-26T22:49:29.474733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:49:30.488070+0000","last_change":"2024-05-26T22:49:30.488070+0000","last_active":"2024-05-26T22:49:30.488070+0000","last_peered":"2024-05-26T22:49:30.488070+0000","last_clean":"2024-05-26T22:49:30.488070+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:49:30.488070+0000","last_undegraded":"2024-05-26T22:49:30.488070+0000","last_fullsized":"2024-05-26T22:49:30.488070+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-05-26T22:49:30.080241+0000","last_change":"2024-05-26T22:49:30.080241+0000","last_active":"2024-05-26T22:49:29.474733+0000","last_peered":"2024-05-26T22:49:29.474733+0000","last_clean":"2024-05-26T22:49:29.474733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:49:30.080241+0000","last_undegraded":"2024-05-26T22:49:30.080241+0000","last_fullsized":"2024-05-26T22:49:30.080241+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_clean_scrub_stamp":"2024-05-26T22:49:29.474733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:49:30.488070+0000","last_change":"2024-05-26T22:49:30.488070+0000","last_active":"2024-05-26T22:49:30.488070+0000","last_peered":"2024-05-26T22:49:30.488070+0000","last_clean":"2024-05-26T22:49:30.488070+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:49:30.488070+0000","last_undegraded":"2024-05-26T22:49:30.488070+0000","last_fullsized":"2024-05-26T22:49:30.488070+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-05-26T22:49:29.499873+0000","last_change":"2024-05-26T22:49:29.499873+0000","last_active":"2024-05-26T22:49:29.474733+0000","last_peered":"2024-05-26T22:49:29.474733+0000","last_clean":"2024-05-26T22:49:29.474733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:49:29.499873+0000","last_undegraded":"2024-05-26T22:49:29.499873+0000","last_fullsized":"2024-05-26T22:49:29.499873+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_clean_scrub_stamp":"2024-05-26T22:49:29.474733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:49:30.488070+0000","last_change":"2024-05-26T22:49:30.488070+0000","last_active":"2024-05-26T22:49:30.488070+0000","last_peered":"2024-05-26T22:49:30.488070+0000","last_clean":"2024-05-26T22:49:30.488070+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:49:30.488070+0000","last_undegraded":"2024-05-26T22:49:30.488070+0000","last_fullsized":"2024-05-26T22:49:30.488070+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-05-26T22:49:30.078825+0000","last_change":"2024-05-26T22:49:30.078825+0000","last_active":"2024-05-26T22:49:29.474733+0000","last_peered":"2024-05-26T22:49:29.474733+0000","last_clean":"2024-05-26T22:49:29.474733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:49:30.078825+0000","last_undegraded":"2024-05-26T22:49:30.078825+0000","last_fullsized":"2024-05-26T22:49:30.078825+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_clean_scrub_stamp":"2024-05-26T22:49:29.474733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:49:30.488070+0000","last_change":"2024-05-26T22:49:30.488070+0000","last_active":"2024-05-26T22:49:30.488070+0000","last_peered":"2024-05-26T22:49:30.488070+0000","last_clean":"2024-05-26T22:49:30.488070+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:49:30.488070+0000","last_undegraded":"2024-05-26T22:49:30.488070+0000","last_fullsized":"2024-05-26T22:49:30.488070+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-05-26T22:49:29.499500+0000","last_change":"2024-05-26T22:49:29.499500+0000","last_active":"2024-05-26T22:49:29.474733+0000","last_peered":"2024-05-26T22:49:29.474733+0000","last_clean":"2024-05-26T22:49:29.474733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:49:29.499500+0000","last_undegraded":"2024-05-26T22:49:29.499500+0000","last_fullsized":"2024-05-26T22:49:29.499500+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_clean_scrub_stamp":"2024-05-26T22:49:29.474733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:49:30.488070+0000","last_change":"2024-05-26T22:49:30.488070+0000","last_active":"2024-05-26T22:49:30.488070+0000","last_peered":"2024-05-26T22:49:30.488070+0000","last_clean":"2024-05-26T22:49:30.488070+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:49:30.488070+0000","last_undegraded":"2024-05-26T22:49:30.488070+0000","last_fullsized":"2024-05-26T22:49:30.488070+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-05-26T22:49:29.496873+0000","last_change":"2024-05-26T22:49:29.496873+0000","last_active":"2024-05-26T22:49:29.474733+0000","last_peered":"2024-05-26T22:49:29.474733+0000","last_clean":"2024-05-26T22:49:29.474733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:49:29.496873+0000","last_undegraded":"2024-05-26T22:49:29.496873+0000","last_fullsized":"2024-05-26T22:49:29.496873+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_clean_scrub_stamp":"2024-05-26T22:49:29.474733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:49:30.488070+0000","last_change":"2024-05-26T22:49:30.488070+0000","last_active":"2024-05-26T22:49:30.488070+0000","last_peered":"2024-05-26T22:49:30.488070+0000","last_clean":"2024-05-26T22:49:30.488070+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:49:30.488070+0000","last_undegraded":"2024-05-26T22:49:30.488070+0000","last_fullsized":"2024-05-26T22:49:30.488070+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-05-26T22:49:29.502645+0000","last_change":"2024-05-26T22:49:29.502645+0000","last_active":"2024-05-26T22:49:29.474733+0000","last_peered":"2024-05-26T22:49:29.474733+0000","last_clean":"2024-05-26T22:49:29.474733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:49:29.502645+0000","last_undegraded":"2024-05-26T22:49:29.502645+0000","last_fullsized":"2024-05-26T22:49:29.502645+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_clean_scrub_stamp":"2024-05-26T22:49:29.474733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:49:30.488070+0000","last_change":"2024-05-26T22:49:30.488070+0000","last_active":"2024-05-26T22:49:30.488070+0000","last_peered":"2024-05-26T22:49:30.488070+0000","last_clean":"2024-05-26T22:49:30.488070+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:49:30.488070+0000","last_undegraded":"2024-05-26T22:49:30.488070+0000","last_fullsized":"2024-05-26T22:49:30.488070+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-05-26T22:49:29.495361+0000","last_change":"2024-05-26T22:49:29.495361+0000","last_active":"2024-05-26T22:49:29.474733+0000","last_peered":"2024-05-26T22:49:29.474733+0000","last_clean":"2024-05-26T22:49:29.474733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:49:29.495361+0000","last_undegraded":"2024-05-26T22:49:29.495361+0000","last_fullsized":"2024-05-26T22:49:29.495361+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_clean_scrub_stamp":"2024-05-26T22:49:29.474733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":188,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-05-26T22:49:29.508351+0000","last_change":"2024-05-26T22:48:12.068917+0000","last_active":"2024-05-26T22:49:29.508351+0000","last_peered":"2024-05-26T22:49:29.508351+0000","last_clean":"2024-05-26T22:49:29.508351+0000","last_became_active":"2024-05-26T22:48:12.065934+0000","last_became_peered":"2024-05-26T22:48:12.065934+0000","last_unstale":"2024-05-26T22:49:29.508351+0000","last_undegraded":"2024-05-26T22:49:29.508351+0000","last_fullsized":"2024-05-26T22:49:29.508351+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:10.628122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:10.628122+0000","last_clean_scrub_stamp":"2024-05-26T22:48:10.628122+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:14:58.895911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574867,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27928,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343912,"statfs":{"total":96636764160,"available":96608165888,"internally_reserved":0,"allocated":929792,"data_stored":717634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27352,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344488,"statfs":{"total":96636764160,"available":96608755712,"internally_reserved":0,"allocated":323584,"data_stored":121449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":335872,"data_stored":127266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":335872,"data_stored":127266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":335872,"data_stored":127266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":335872,"data_stored":127266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":335872,"data_stored":127266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574867,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27932,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343908,"statfs":{"total":96636764160,"available":96608161792,"internally_reserved":0,"allocated":917504,"data_stored":711817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-26T22:49:36.030 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-26T22:49:36.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.272+0000 7f3c76400640 1 Processor -- start 2024-05-26T22:49:36.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.272+0000 7f3c76400640 1 -- start start 2024-05-26T22:49:36.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.274+0000 7f3c76400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3c700d16d0 0x7f3c7015d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:36.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.274+0000 7f3c76400640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c700cfb40 con 0x7f3c700d0020 2024-05-26T22:49:36.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.274+0000 7f3c76400640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c700d1cb0 con 0x7f3c70156680 2024-05-26T22:49:36.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.274+0000 7f3c76400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3c700ce090 con 0x7f3c700d16d0 2024-05-26T22:49:36.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.274+0000 7f3c75a00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f3c700d0020 0x7f3c700d04b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:48418/0 (socket says 172.21.15.131:48418) 2024-05-26T22:49:36.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.274+0000 7f3c75a00640 1 -- 172.21.15.131:0/4023282561 learned_addr learned my addr 172.21.15.131:0/4023282561 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:49:36.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.275+0000 7f3c6fe00640 1 --2- 172.21.15.131:0/4023282561 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3c700d16d0 0x7f3c7015d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:36.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.275+0000 7f3c6f400640 1 -- 172.21.15.131:0/4023282561 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2555634971 0 0) 0x7f3c700cfb40 con 0x7f3c700d0020 2024-05-26T22:49:36.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.275+0000 7f3c6f400640 1 -- 172.21.15.131:0/4023282561 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3c54003520 con 0x7f3c700d0020 2024-05-26T22:49:36.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.275+0000 7f3c6f400640 1 -- 172.21.15.131:0/4023282561 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3095587794 0 0) 0x7f3c54003520 con 0x7f3c700d0020 2024-05-26T22:49:36.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.275+0000 7f3c6f400640 1 -- 172.21.15.131:0/4023282561 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3c700d16d0 msgr2=0x7f3c7015d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:36.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.275+0000 7f3c6f400640 1 --2- 172.21.15.131:0/4023282561 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3c700d16d0 0x7f3c7015d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:36.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.275+0000 7f3c6f400640 1 -- 172.21.15.131:0/4023282561 >> v1:172.21.15.187:6789/0 conn(0x7f3c70156680 legacy=0x7f3c7015d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:36.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.275+0000 7f3c6f400640 1 -- 172.21.15.131:0/4023282561 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c700ce370 con 0x7f3c700d0020 2024-05-26T22:49:36.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.275+0000 7f3c6f400640 1 -- 172.21.15.131:0/4023282561 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7f3c680019f0 con 0x7f3c700d0020 2024-05-26T22:49:36.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.275+0000 7f3c6f400640 1 -- 172.21.15.131:0/4023282561 >> v1:172.21.15.131:6789/0 conn(0x7f3c700d0020 legacy=0x7f3c700d04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:36.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.275+0000 7f3c6f400640 1 --2- 172.21.15.131:0/4023282561 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3c540039e0 0x7f3c54003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:36.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.275+0000 7f3c6f400640 1 --2- 172.21.15.131:0/4023282561 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3c54004800 0x7f3c54004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:36.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.275+0000 7f3c6f400640 1 --2- 172.21.15.131:0/4023282561 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3c700d16d0 0x7f3c540051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:36.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.275+0000 7f3c6f400640 1 -- 172.21.15.131:0/4023282561 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f3c700ce090 con 0x7f3c540039e0 2024-05-26T22:49:36.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.275+0000 7f3c6f400640 1 -- 172.21.15.131:0/4023282561 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3c5400baa0 con 0x7f3c54004800 2024-05-26T22:49:36.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.275+0000 7f3c6f400640 1 -- 172.21.15.131:0/4023282561 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3c5400bca0 con 0x7f3c700d16d0 2024-05-26T22:49:36.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.276+0000 7f3c75000640 1 --2- 172.21.15.131:0/4023282561 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3c540039e0 0x7f3c54003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:36.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.276+0000 7f3c6fe00640 1 --2- 172.21.15.131:0/4023282561 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3c54004800 0x7f3c54004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:36.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.276+0000 7f3c75a00640 1 --2- 172.21.15.131:0/4023282561 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3c700d16d0 0x7f3c540051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:36.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.276+0000 7f3c75000640 1 -- 172.21.15.131:0/4023282561 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3c700d16d0 msgr2=0x7f3c540051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:36.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.276+0000 7f3c75000640 1 --2- 172.21.15.131:0/4023282561 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3c700d16d0 0x7f3c540051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:36.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.276+0000 7f3c75000640 1 -- 172.21.15.131:0/4023282561 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3c54004800 msgr2=0x7f3c54004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:36.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.276+0000 7f3c75000640 1 --2- 172.21.15.131:0/4023282561 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3c54004800 0x7f3c54004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:36.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.276+0000 7f3c75000640 1 -- 172.21.15.131:0/4023282561 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c54003520 con 0x7f3c540039e0 2024-05-26T22:49:36.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.277+0000 7f3c75000640 1 --2- 172.21.15.131:0/4023282561 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3c540039e0 0x7f3c54003e00 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f3c60001720 tx=0x7f3c54005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f8dc83d0c7470d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:36.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.277+0000 7f3c6f400640 1 -- 172.21.15.131:0/4023282561 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c60001370 con 0x7f3c540039e0 2024-05-26T22:49:36.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.277+0000 7f3c6f400640 1 -- 172.21.15.131:0/4023282561 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c600052d0 con 0x7f3c540039e0 2024-05-26T22:49:36.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.277+0000 7f3c6f400640 1 -- 172.21.15.131:0/4023282561 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c6002ab90 con 0x7f3c540039e0 2024-05-26T22:49:36.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.278+0000 7f3c76400640 1 -- 172.21.15.131:0/4023282561 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3c540039e0 msgr2=0x7f3c54003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:36.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.278+0000 7f3c76400640 1 --2- 172.21.15.131:0/4023282561 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3c540039e0 0x7f3c54003e00 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f3c60001720 tx=0x7f3c54005830 comp rx=0 tx=0).stop 2024-05-26T22:49:36.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.278+0000 7f3c75a00640 1 -- 172.21.15.131:0/4023282561 reap_dead start 2024-05-26T22:49:36.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.278+0000 7f3c76400640 1 -- 172.21.15.131:0/4023282561 shutdown_connections 2024-05-26T22:49:36.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.278+0000 7f3c76400640 1 -- 172.21.15.131:0/4023282561 >> 172.21.15.131:0/4023282561 conn(0x7f3c700b3ba0 msgr2=0x7f3c700b3f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:36.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.278+0000 7f3c76400640 1 -- 172.21.15.131:0/4023282561 shutdown_connections 2024-05-26T22:49:36.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.278+0000 7f3c76400640 1 -- 172.21.15.131:0/4023282561 wait complete. 2024-05-26T22:49:36.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.280+0000 7f3c76400640 1 Processor -- start 2024-05-26T22:49:36.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.280+0000 7f3c76400640 1 -- start start 2024-05-26T22:49:36.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.281+0000 7f3c76400640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3c540039e0 0x7f3c701476b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:36.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.281+0000 7f3c76400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3c700b4c60 0x7f3c7015b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:36.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.281+0000 7f3c76400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3c700b50c0 0x7f3c7015bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:36.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.281+0000 7f3c76400640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f3c7015d930 con 0x7f3c700b4c60 2024-05-26T22:49:36.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.281+0000 7f3c76400640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3c7015d7b0 con 0x7f3c540039e0 2024-05-26T22:49:36.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.281+0000 7f3c76400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3c7015dab0 con 0x7f3c700b50c0 2024-05-26T22:49:36.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.281+0000 7f3c6fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3c700b4c60 0x7f3c7015b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:36.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.281+0000 7f3c6fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3c700b4c60 0x7f3c7015b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53212/0 (socket says 172.21.15.131:53212) 2024-05-26T22:49:36.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.281+0000 7f3c6fe00640 1 -- 172.21.15.131:0/775281613 learned_addr learned my addr 172.21.15.131:0/775281613 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:36.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.282+0000 7f3c75000640 1 --2- 172.21.15.131:0/775281613 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3c540039e0 0x7f3c701476b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:36.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.282+0000 7f3c6fe00640 1 -- 172.21.15.131:0/775281613 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3c700b50c0 msgr2=0x7f3c7015bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:36.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.282+0000 7f3c6fe00640 1 --2- 172.21.15.131:0/775281613 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3c700b50c0 0x7f3c7015bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:36.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.282+0000 7f3c6fe00640 1 -- 172.21.15.131:0/775281613 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3c540039e0 msgr2=0x7f3c701476b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:36.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.282+0000 7f3c6fe00640 1 --2- 172.21.15.131:0/775281613 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3c540039e0 0x7f3c701476b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:36.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.282+0000 7f3c6fe00640 1 -- 172.21.15.131:0/775281613 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c700b5620 con 0x7f3c700b4c60 2024-05-26T22:49:36.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.282+0000 7f3c6fe00640 1 --2- 172.21.15.131:0/775281613 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3c700b4c60 0x7f3c7015b6c0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c011630 tx=0x7f3c5c011af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:36.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.282+0000 7f3c6cc00640 1 -- 172.21.15.131:0/775281613 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c5c00f920 con 0x7f3c700b4c60 2024-05-26T22:49:36.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.282+0000 7f3c6cc00640 1 -- 172.21.15.131:0/775281613 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c5c00fac0 con 0x7f3c700b4c60 2024-05-26T22:49:36.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.282+0000 7f3c6cc00640 1 -- 172.21.15.131:0/775281613 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c5c00fe70 con 0x7f3c700b4c60 2024-05-26T22:49:36.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.282+0000 7f3c76400640 1 -- 172.21.15.131:0/775281613 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c700b5900 con 0x7f3c700b4c60 2024-05-26T22:49:36.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.282+0000 7f3c76400640 1 -- 172.21.15.131:0/775281613 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c700b5ec0 con 0x7f3c700b4c60 2024-05-26T22:49:36.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.284+0000 7f3c76400640 1 -- 172.21.15.131:0/775281613 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c700b6200 con 0x7f3c700b4c60 2024-05-26T22:49:36.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.284+0000 7f3c6cc00640 1 -- 172.21.15.131:0/775281613 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3c5c015460 con 0x7f3c700b4c60 2024-05-26T22:49:36.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.285+0000 7f3c6cc00640 1 --2- 172.21.15.131:0/775281613 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f3c5006be30 0x7f3c5006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:36.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.285+0000 7f3c6cc00640 1 -- 172.21.15.131:0/775281613 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f3c5c08b8b0 con 0x7f3c700b4c60 2024-05-26T22:49:36.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.286+0000 7f3c75000640 1 --2- 172.21.15.131:0/775281613 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f3c5006be30 0x7f3c5006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:36.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.286+0000 7f3c75000640 1 --2- 172.21.15.131:0/775281613 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f3c5006be30 0x7f3c5006e2f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3c60032000 tx=0x7f3c6002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:36.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.290+0000 7f3c6cc00640 1 -- 172.21.15.131:0/775281613 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f3c700b6200 con 0x7f3c700b4c60 2024-05-26T22:49:36.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.487+0000 7f3c76400640 1 -- 172.21.15.131:0/775281613 --> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3c7015d360 con 0x7f3c5006be30 2024-05-26T22:49:36.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.495+0000 7f3c6cc00640 1 -- 172.21.15.131:0/775281613 <== mgr.4104 v2:172.21.15.131:6832/3484832947 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251951 (secure 0 0 0) 0x7f3c7015d360 con 0x7f3c5006be30 2024-05-26T22:49:36.498 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:49:36.499 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-05-26T22:49:36.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.500+0000 7f3c76400640 1 -- 172.21.15.131:0/775281613 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f3c5006be30 msgr2=0x7f3c5006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:36.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.500+0000 7f3c76400640 1 --2- 172.21.15.131:0/775281613 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f3c5006be30 0x7f3c5006e2f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3c60032000 tx=0x7f3c6002e000 comp rx=0 tx=0).stop 2024-05-26T22:49:36.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.500+0000 7f3c76400640 1 -- 172.21.15.131:0/775281613 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3c700b4c60 msgr2=0x7f3c7015b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:36.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.500+0000 7f3c76400640 1 --2- 172.21.15.131:0/775281613 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3c700b4c60 0x7f3c7015b6c0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c011630 tx=0x7f3c5c011af0 comp rx=0 tx=0).stop 2024-05-26T22:49:36.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.500+0000 7f3c76400640 1 -- 172.21.15.131:0/775281613 shutdown_connections 2024-05-26T22:49:36.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.500+0000 7f3c76400640 1 --2- 172.21.15.131:0/775281613 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3c700b50c0 0x7f3c7015bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:36.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.500+0000 7f3c76400640 1 --2- 172.21.15.131:0/775281613 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f3c5006be30 0x7f3c5006e2f0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:36.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.500+0000 7f3c76400640 1 --2- 172.21.15.131:0/775281613 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3c700b4c60 0x7f3c7015b6c0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:36.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.500+0000 7f3c76400640 1 --2- 172.21.15.131:0/775281613 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3c540039e0 0x7f3c701476b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:36.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.500+0000 7f3c76400640 1 -- 172.21.15.131:0/775281613 >> 172.21.15.131:0/775281613 conn(0x7f3c700b3ba0 msgr2=0x7f3c7015a1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:36.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.501+0000 7f3c76400640 1 -- 172.21.15.131:0/775281613 shutdown_connections 2024-05-26T22:49:36.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.501+0000 7f3c76400640 1 -- 172.21.15.131:0/775281613 wait complete. 2024-05-26T22:49:36.519 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":69,"stamp":"2024-05-26T22:49:36.406100+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":186,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220316,"kb_used_data":4136,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754404,"statfs":{"total":773094113280,"available":772868509696,"internally_reserved":0,"allocated":4235264,"data_stored":2358420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":41,"apply_latency_ms":41,"commit_latency_ns":41000000,"apply_latency_ns":41000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001510"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-26T22:49:32.547016+0000","last_change":"2024-05-26T22:49:31.542598+0000","last_active":"2024-05-26T22:49:32.547016+0000","last_peered":"2024-05-26T22:49:32.547016+0000","last_clean":"2024-05-26T22:49:32.547016+0000","last_became_active":"2024-05-26T22:49:31.542287+0000","last_became_peered":"2024-05-26T22:49:31.542287+0000","last_unstale":"2024-05-26T22:49:32.547016+0000","last_undegraded":"2024-05-26T22:49:32.547016+0000","last_fullsized":"2024-05-26T22:49:32.547016+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:18:38.095776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-26T22:49:32.546982+0000","last_change":"2024-05-26T22:49:31.560448+0000","last_active":"2024-05-26T22:49:32.546982+0000","last_peered":"2024-05-26T22:49:32.546982+0000","last_clean":"2024-05-26T22:49:32.546982+0000","last_became_active":"2024-05-26T22:49:31.560140+0000","last_became_peered":"2024-05-26T22:49:31.560140+0000","last_unstale":"2024-05-26T22:49:32.546982+0000","last_undegraded":"2024-05-26T22:49:32.546982+0000","last_fullsized":"2024-05-26T22:49:32.546982+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:27:35.980326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-26T22:49:32.585691+0000","last_change":"2024-05-26T22:49:31.555967+0000","last_active":"2024-05-26T22:49:32.585691+0000","last_peered":"2024-05-26T22:49:32.585691+0000","last_clean":"2024-05-26T22:49:32.585691+0000","last_became_active":"2024-05-26T22:49:31.555670+0000","last_became_peered":"2024-05-26T22:49:31.555670+0000","last_unstale":"2024-05-26T22:49:32.585691+0000","last_undegraded":"2024-05-26T22:49:32.585691+0000","last_fullsized":"2024-05-26T22:49:32.585691+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T23:32:58.138139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-26T22:49:32.569410+0000","last_change":"2024-05-26T22:49:31.571341+0000","last_active":"2024-05-26T22:49:32.569410+0000","last_peered":"2024-05-26T22:49:32.569410+0000","last_clean":"2024-05-26T22:49:32.569410+0000","last_became_active":"2024-05-26T22:49:31.569238+0000","last_became_peered":"2024-05-26T22:49:31.569238+0000","last_unstale":"2024-05-26T22:49:32.569410+0000","last_undegraded":"2024-05-26T22:49:32.569410+0000","last_fullsized":"2024-05-26T22:49:32.569410+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:19:52.301442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-26T22:49:32.541828+0000","last_change":"2024-05-26T22:49:31.558680+0000","last_active":"2024-05-26T22:49:32.541828+0000","last_peered":"2024-05-26T22:49:32.541828+0000","last_clean":"2024-05-26T22:49:32.541828+0000","last_became_active":"2024-05-26T22:49:31.558347+0000","last_became_peered":"2024-05-26T22:49:31.558347+0000","last_unstale":"2024-05-26T22:49:32.541828+0000","last_undegraded":"2024-05-26T22:49:32.541828+0000","last_fullsized":"2024-05-26T22:49:32.541828+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:56:23.347904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-26T22:49:32.546132+0000","last_change":"2024-05-26T22:49:31.551765+0000","last_active":"2024-05-26T22:49:32.546132+0000","last_peered":"2024-05-26T22:49:32.546132+0000","last_clean":"2024-05-26T22:49:32.546132+0000","last_became_active":"2024-05-26T22:49:31.551480+0000","last_became_peered":"2024-05-26T22:49:31.551480+0000","last_unstale":"2024-05-26T22:49:32.546132+0000","last_undegraded":"2024-05-26T22:49:32.546132+0000","last_fullsized":"2024-05-26T22:49:32.546132+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T23:32:58.138139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-26T22:49:33.691215+0000","last_change":"2024-05-26T22:49:31.554793+0000","last_active":"2024-05-26T22:49:33.691215+0000","last_peered":"2024-05-26T22:49:33.691215+0000","last_clean":"2024-05-26T22:49:33.691215+0000","last_became_active":"2024-05-26T22:49:31.554579+0000","last_became_peered":"2024-05-26T22:49:31.554579+0000","last_unstale":"2024-05-26T22:49:33.691215+0000","last_undegraded":"2024-05-26T22:49:33.691215+0000","last_fullsized":"2024-05-26T22:49:33.691215+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:41:28.909927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-26T22:49:33.689467+0000","last_change":"2024-05-26T22:49:31.554757+0000","last_active":"2024-05-26T22:49:33.689467+0000","last_peered":"2024-05-26T22:49:33.689467+0000","last_clean":"2024-05-26T22:49:33.689467+0000","last_became_active":"2024-05-26T22:49:31.554383+0000","last_became_peered":"2024-05-26T22:49:31.554383+0000","last_unstale":"2024-05-26T22:49:33.689467+0000","last_undegraded":"2024-05-26T22:49:33.689467+0000","last_fullsized":"2024-05-26T22:49:33.689467+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:46:45.385281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-26T22:49:33.688740+0000","last_change":"2024-05-26T22:49:31.568331+0000","last_active":"2024-05-26T22:49:33.688740+0000","last_peered":"2024-05-26T22:49:33.688740+0000","last_clean":"2024-05-26T22:49:33.688740+0000","last_became_active":"2024-05-26T22:49:31.568005+0000","last_became_peered":"2024-05-26T22:49:31.568005+0000","last_unstale":"2024-05-26T22:49:33.688740+0000","last_undegraded":"2024-05-26T22:49:33.688740+0000","last_fullsized":"2024-05-26T22:49:33.688740+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:06:35.987200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-26T22:49:32.570311+0000","last_change":"2024-05-26T22:49:31.553052+0000","last_active":"2024-05-26T22:49:32.570311+0000","last_peered":"2024-05-26T22:49:32.570311+0000","last_clean":"2024-05-26T22:49:32.570311+0000","last_became_active":"2024-05-26T22:49:31.551825+0000","last_became_peered":"2024-05-26T22:49:31.551825+0000","last_unstale":"2024-05-26T22:49:32.570311+0000","last_undegraded":"2024-05-26T22:49:32.570311+0000","last_fullsized":"2024-05-26T22:49:32.570311+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:59:06.763141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-26T22:49:33.688837+0000","last_change":"2024-05-26T22:49:31.557775+0000","last_active":"2024-05-26T22:49:33.688837+0000","last_peered":"2024-05-26T22:49:33.688837+0000","last_clean":"2024-05-26T22:49:33.688837+0000","last_became_active":"2024-05-26T22:49:31.557458+0000","last_became_peered":"2024-05-26T22:49:31.557458+0000","last_unstale":"2024-05-26T22:49:33.688837+0000","last_undegraded":"2024-05-26T22:49:33.688837+0000","last_fullsized":"2024-05-26T22:49:33.688837+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:18:05.112774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-26T22:49:32.585602+0000","last_change":"2024-05-26T22:49:31.550039+0000","last_active":"2024-05-26T22:49:32.585602+0000","last_peered":"2024-05-26T22:49:32.585602+0000","last_clean":"2024-05-26T22:49:32.585602+0000","last_became_active":"2024-05-26T22:49:31.549757+0000","last_became_peered":"2024-05-26T22:49:31.549757+0000","last_unstale":"2024-05-26T22:49:32.585602+0000","last_undegraded":"2024-05-26T22:49:32.585602+0000","last_fullsized":"2024-05-26T22:49:32.585602+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T23:32:46.174213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-26T22:49:32.542081+0000","last_change":"2024-05-26T22:49:31.551753+0000","last_active":"2024-05-26T22:49:32.542081+0000","last_peered":"2024-05-26T22:49:32.542081+0000","last_clean":"2024-05-26T22:49:32.542081+0000","last_became_active":"2024-05-26T22:49:31.551189+0000","last_became_peered":"2024-05-26T22:49:31.551189+0000","last_unstale":"2024-05-26T22:49:32.542081+0000","last_undegraded":"2024-05-26T22:49:32.542081+0000","last_fullsized":"2024-05-26T22:49:32.542081+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:16:26.839871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-26T22:49:32.545966+0000","last_change":"2024-05-26T22:49:31.546411+0000","last_active":"2024-05-26T22:49:32.545966+0000","last_peered":"2024-05-26T22:49:32.545966+0000","last_clean":"2024-05-26T22:49:32.545966+0000","last_became_active":"2024-05-26T22:49:31.546137+0000","last_became_peered":"2024-05-26T22:49:31.546137+0000","last_unstale":"2024-05-26T22:49:32.545966+0000","last_undegraded":"2024-05-26T22:49:32.545966+0000","last_fullsized":"2024-05-26T22:49:32.545966+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:34:36.050113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-26T22:49:32.585563+0000","last_change":"2024-05-26T22:49:31.544816+0000","last_active":"2024-05-26T22:49:32.585563+0000","last_peered":"2024-05-26T22:49:32.585563+0000","last_clean":"2024-05-26T22:49:32.585563+0000","last_became_active":"2024-05-26T22:49:31.544373+0000","last_became_peered":"2024-05-26T22:49:31.544373+0000","last_unstale":"2024-05-26T22:49:32.585563+0000","last_undegraded":"2024-05-26T22:49:32.585563+0000","last_fullsized":"2024-05-26T22:49:32.585563+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:34:36.050113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-26T22:49:33.690863+0000","last_change":"2024-05-26T22:49:31.560627+0000","last_active":"2024-05-26T22:49:33.690863+0000","last_peered":"2024-05-26T22:49:33.690863+0000","last_clean":"2024-05-26T22:49:33.690863+0000","last_became_active":"2024-05-26T22:49:31.560391+0000","last_became_peered":"2024-05-26T22:49:31.560391+0000","last_unstale":"2024-05-26T22:49:33.690863+0000","last_undegraded":"2024-05-26T22:49:33.690863+0000","last_fullsized":"2024-05-26T22:49:33.690863+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:02:54.311565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-26T22:49:33.689973+0000","last_change":"2024-05-26T22:49:31.555132+0000","last_active":"2024-05-26T22:49:33.689973+0000","last_peered":"2024-05-26T22:49:33.689973+0000","last_clean":"2024-05-26T22:49:33.689973+0000","last_became_active":"2024-05-26T22:49:31.554252+0000","last_became_peered":"2024-05-26T22:49:31.554252+0000","last_unstale":"2024-05-26T22:49:33.689973+0000","last_undegraded":"2024-05-26T22:49:33.689973+0000","last_fullsized":"2024-05-26T22:49:33.689973+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T23:32:46.174213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-26T22:49:32.546886+0000","last_change":"2024-05-26T22:49:31.564350+0000","last_active":"2024-05-26T22:49:32.546886+0000","last_peered":"2024-05-26T22:49:32.546886+0000","last_clean":"2024-05-26T22:49:32.546886+0000","last_became_active":"2024-05-26T22:49:31.563960+0000","last_became_peered":"2024-05-26T22:49:31.563960+0000","last_unstale":"2024-05-26T22:49:32.546886+0000","last_undegraded":"2024-05-26T22:49:32.546886+0000","last_fullsized":"2024-05-26T22:49:32.546886+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:25:44.839029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-26T22:49:32.545596+0000","last_change":"2024-05-26T22:49:31.558270+0000","last_active":"2024-05-26T22:49:32.545596+0000","last_peered":"2024-05-26T22:49:32.545596+0000","last_clean":"2024-05-26T22:49:32.545596+0000","last_became_active":"2024-05-26T22:49:31.557944+0000","last_became_peered":"2024-05-26T22:49:31.557944+0000","last_unstale":"2024-05-26T22:49:32.545596+0000","last_undegraded":"2024-05-26T22:49:32.545596+0000","last_fullsized":"2024-05-26T22:49:32.545596+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T23:32:46.174213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-26T22:49:32.542346+0000","last_change":"2024-05-26T22:49:31.555429+0000","last_active":"2024-05-26T22:49:32.542346+0000","last_peered":"2024-05-26T22:49:32.542346+0000","last_clean":"2024-05-26T22:49:32.542346+0000","last_became_active":"2024-05-26T22:49:31.552698+0000","last_became_peered":"2024-05-26T22:49:31.552698+0000","last_unstale":"2024-05-26T22:49:32.542346+0000","last_undegraded":"2024-05-26T22:49:32.542346+0000","last_fullsized":"2024-05-26T22:49:32.542346+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T23:42:17.188960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-26T22:49:33.687991+0000","last_change":"2024-05-26T22:49:31.557309+0000","last_active":"2024-05-26T22:49:33.687991+0000","last_peered":"2024-05-26T22:49:33.687991+0000","last_clean":"2024-05-26T22:49:33.687991+0000","last_became_active":"2024-05-26T22:49:31.557025+0000","last_became_peered":"2024-05-26T22:49:31.557025+0000","last_unstale":"2024-05-26T22:49:33.687991+0000","last_undegraded":"2024-05-26T22:49:33.687991+0000","last_fullsized":"2024-05-26T22:49:33.687991+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:08:35.106387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-26T22:49:33.689425+0000","last_change":"2024-05-26T22:49:31.570794+0000","last_active":"2024-05-26T22:49:33.689425+0000","last_peered":"2024-05-26T22:49:33.689425+0000","last_clean":"2024-05-26T22:49:33.689425+0000","last_became_active":"2024-05-26T22:49:31.570478+0000","last_became_peered":"2024-05-26T22:49:31.570478+0000","last_unstale":"2024-05-26T22:49:33.689425+0000","last_undegraded":"2024-05-26T22:49:33.689425+0000","last_fullsized":"2024-05-26T22:49:33.689425+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:31:05.422346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-26T22:49:32.561931+0000","last_change":"2024-05-26T22:49:31.548342+0000","last_active":"2024-05-26T22:49:32.561931+0000","last_peered":"2024-05-26T22:49:32.561931+0000","last_clean":"2024-05-26T22:49:32.561931+0000","last_became_active":"2024-05-26T22:49:31.547579+0000","last_became_peered":"2024-05-26T22:49:31.547579+0000","last_unstale":"2024-05-26T22:49:32.561931+0000","last_undegraded":"2024-05-26T22:49:32.561931+0000","last_fullsized":"2024-05-26T22:49:32.561931+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:19:52.301442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-26T22:49:32.585046+0000","last_change":"2024-05-26T22:49:31.548813+0000","last_active":"2024-05-26T22:49:32.585046+0000","last_peered":"2024-05-26T22:49:32.585046+0000","last_clean":"2024-05-26T22:49:32.585046+0000","last_became_active":"2024-05-26T22:49:31.545763+0000","last_became_peered":"2024-05-26T22:49:31.545763+0000","last_unstale":"2024-05-26T22:49:32.585046+0000","last_undegraded":"2024-05-26T22:49:32.585046+0000","last_fullsized":"2024-05-26T22:49:32.585046+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:46:45.385281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-26T22:49:32.570354+0000","last_change":"2024-05-26T22:49:31.555491+0000","last_active":"2024-05-26T22:49:32.570354+0000","last_peered":"2024-05-26T22:49:32.570354+0000","last_clean":"2024-05-26T22:49:32.570354+0000","last_became_active":"2024-05-26T22:49:31.555026+0000","last_became_peered":"2024-05-26T22:49:31.555026+0000","last_unstale":"2024-05-26T22:49:32.570354+0000","last_undegraded":"2024-05-26T22:49:32.570354+0000","last_fullsized":"2024-05-26T22:49:32.570354+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T23:53:06.950292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-26T22:49:32.547135+0000","last_change":"2024-05-26T22:49:31.553505+0000","last_active":"2024-05-26T22:49:32.547135+0000","last_peered":"2024-05-26T22:49:32.547135+0000","last_clean":"2024-05-26T22:49:32.547135+0000","last_became_active":"2024-05-26T22:49:31.553139+0000","last_became_peered":"2024-05-26T22:49:31.553139+0000","last_unstale":"2024-05-26T22:49:32.547135+0000","last_undegraded":"2024-05-26T22:49:32.547135+0000","last_fullsized":"2024-05-26T22:49:32.547135+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:00:51.339564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-26T22:49:32.570296+0000","last_change":"2024-05-26T22:49:31.571712+0000","last_active":"2024-05-26T22:49:32.570296+0000","last_peered":"2024-05-26T22:49:32.570296+0000","last_clean":"2024-05-26T22:49:32.570296+0000","last_became_active":"2024-05-26T22:49:31.571320+0000","last_became_peered":"2024-05-26T22:49:31.571320+0000","last_unstale":"2024-05-26T22:49:32.570296+0000","last_undegraded":"2024-05-26T22:49:32.570296+0000","last_fullsized":"2024-05-26T22:49:32.570296+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:31:05.422346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-26T22:49:32.585571+0000","last_change":"2024-05-26T22:49:31.541276+0000","last_active":"2024-05-26T22:49:32.585571+0000","last_peered":"2024-05-26T22:49:32.585571+0000","last_clean":"2024-05-26T22:49:32.585571+0000","last_became_active":"2024-05-26T22:49:31.540847+0000","last_became_peered":"2024-05-26T22:49:31.540847+0000","last_unstale":"2024-05-26T22:49:32.585571+0000","last_undegraded":"2024-05-26T22:49:32.585571+0000","last_fullsized":"2024-05-26T22:49:32.585571+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:02:54.311565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-26T22:49:33.688587+0000","last_change":"2024-05-26T22:49:31.570878+0000","last_active":"2024-05-26T22:49:33.688587+0000","last_peered":"2024-05-26T22:49:33.688587+0000","last_clean":"2024-05-26T22:49:33.688587+0000","last_became_active":"2024-05-26T22:49:31.570556+0000","last_became_peered":"2024-05-26T22:49:31.570556+0000","last_unstale":"2024-05-26T22:49:33.688587+0000","last_undegraded":"2024-05-26T22:49:33.688587+0000","last_fullsized":"2024-05-26T22:49:33.688587+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:19:52.301442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-26T22:49:33.691373+0000","last_change":"2024-05-26T22:49:31.552278+0000","last_active":"2024-05-26T22:49:33.691373+0000","last_peered":"2024-05-26T22:49:33.691373+0000","last_clean":"2024-05-26T22:49:33.691373+0000","last_became_active":"2024-05-26T22:49:31.551275+0000","last_became_peered":"2024-05-26T22:49:31.551275+0000","last_unstale":"2024-05-26T22:49:33.691373+0000","last_undegraded":"2024-05-26T22:49:33.691373+0000","last_fullsized":"2024-05-26T22:49:33.691373+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:29:50.663367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-26T22:49:32.546482+0000","last_change":"2024-05-26T22:49:31.543549+0000","last_active":"2024-05-26T22:49:32.546482+0000","last_peered":"2024-05-26T22:49:32.546482+0000","last_clean":"2024-05-26T22:49:32.546482+0000","last_became_active":"2024-05-26T22:49:31.543222+0000","last_became_peered":"2024-05-26T22:49:31.543222+0000","last_unstale":"2024-05-26T22:49:32.546482+0000","last_undegraded":"2024-05-26T22:49:32.546482+0000","last_fullsized":"2024-05-26T22:49:32.546482+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:58:48.514425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-26T22:49:33.689874+0000","last_change":"2024-05-26T22:49:31.564617+0000","last_active":"2024-05-26T22:49:33.689874+0000","last_peered":"2024-05-26T22:49:33.689874+0000","last_clean":"2024-05-26T22:49:33.689874+0000","last_became_active":"2024-05-26T22:49:31.564054+0000","last_became_peered":"2024-05-26T22:49:31.564054+0000","last_unstale":"2024-05-26T22:49:33.689874+0000","last_undegraded":"2024-05-26T22:49:33.689874+0000","last_fullsized":"2024-05-26T22:49:33.689874+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:29:59.315295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-26T22:49:32.562575+0000","last_change":"2024-05-26T22:49:31.554876+0000","last_active":"2024-05-26T22:49:32.562575+0000","last_peered":"2024-05-26T22:49:32.562575+0000","last_clean":"2024-05-26T22:49:32.562575+0000","last_became_active":"2024-05-26T22:49:31.554629+0000","last_became_peered":"2024-05-26T22:49:31.554629+0000","last_unstale":"2024-05-26T22:49:32.562575+0000","last_undegraded":"2024-05-26T22:49:32.562575+0000","last_fullsized":"2024-05-26T22:49:32.562575+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:59:06.763141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"32'4","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-26T22:49:32.547137+0000","last_change":"2024-05-26T22:49:30.535774+0000","last_active":"2024-05-26T22:49:32.547137+0000","last_peered":"2024-05-26T22:49:32.547137+0000","last_clean":"2024-05-26T22:49:32.547137+0000","last_became_active":"2024-05-26T22:49:30.535397+0000","last_became_peered":"2024-05-26T22:49:30.535397+0000","last_unstale":"2024-05-26T22:49:32.547137+0000","last_undegraded":"2024-05-26T22:49:32.547137+0000","last_fullsized":"2024-05-26T22:49:32.547137+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_clean_scrub_stamp":"2024-05-26T22:49:29.474733+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T23:42:16.175623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-26T22:49:32.547009+0000","last_change":"2024-05-26T22:49:31.564007+0000","last_active":"2024-05-26T22:49:32.547009+0000","last_peered":"2024-05-26T22:49:32.547009+0000","last_clean":"2024-05-26T22:49:32.547009+0000","last_became_active":"2024-05-26T22:49:31.563262+0000","last_became_peered":"2024-05-26T22:49:31.563262+0000","last_unstale":"2024-05-26T22:49:32.547009+0000","last_undegraded":"2024-05-26T22:49:32.547009+0000","last_fullsized":"2024-05-26T22:49:32.547009+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:45:15.596716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-26T22:49:32.562374+0000","last_change":"2024-05-26T22:49:30.534388+0000","last_active":"2024-05-26T22:49:32.562374+0000","last_peered":"2024-05-26T22:49:32.562374+0000","last_clean":"2024-05-26T22:49:32.562374+0000","last_became_active":"2024-05-26T22:49:30.534077+0000","last_became_peered":"2024-05-26T22:49:30.534077+0000","last_unstale":"2024-05-26T22:49:32.562374+0000","last_undegraded":"2024-05-26T22:49:32.562374+0000","last_fullsized":"2024-05-26T22:49:32.562374+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_clean_scrub_stamp":"2024-05-26T22:49:29.474733+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T22:56:25.073062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-26T22:49:33.688954+0000","last_change":"2024-05-26T22:49:31.576827+0000","last_active":"2024-05-26T22:49:33.688954+0000","last_peered":"2024-05-26T22:49:33.688954+0000","last_clean":"2024-05-26T22:49:33.688954+0000","last_became_active":"2024-05-26T22:49:31.576500+0000","last_became_peered":"2024-05-26T22:49:31.576500+0000","last_unstale":"2024-05-26T22:49:33.688954+0000","last_undegraded":"2024-05-26T22:49:33.688954+0000","last_fullsized":"2024-05-26T22:49:33.688954+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:51:00.046866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-26T22:49:32.546263+0000","last_change":"2024-05-26T22:49:30.529805+0000","last_active":"2024-05-26T22:49:32.546263+0000","last_peered":"2024-05-26T22:49:32.546263+0000","last_clean":"2024-05-26T22:49:32.546263+0000","last_became_active":"2024-05-26T22:49:30.529446+0000","last_became_peered":"2024-05-26T22:49:30.529446+0000","last_unstale":"2024-05-26T22:49:32.546263+0000","last_undegraded":"2024-05-26T22:49:32.546263+0000","last_fullsized":"2024-05-26T22:49:32.546263+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_clean_scrub_stamp":"2024-05-26T22:49:29.474733+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:45:14.583379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-26T22:49:32.570550+0000","last_change":"2024-05-26T22:49:31.571473+0000","last_active":"2024-05-26T22:49:32.570550+0000","last_peered":"2024-05-26T22:49:32.570550+0000","last_clean":"2024-05-26T22:49:32.570550+0000","last_became_active":"2024-05-26T22:49:31.571166+0000","last_became_peered":"2024-05-26T22:49:31.571166+0000","last_unstale":"2024-05-26T22:49:32.570550+0000","last_undegraded":"2024-05-26T22:49:32.570550+0000","last_fullsized":"2024-05-26T22:49:32.570550+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:10:43.375671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-26T22:49:32.542355+0000","last_change":"2024-05-26T22:49:30.522082+0000","last_active":"2024-05-26T22:49:32.542355+0000","last_peered":"2024-05-26T22:49:32.542355+0000","last_clean":"2024-05-26T22:49:32.542355+0000","last_became_active":"2024-05-26T22:49:30.521749+0000","last_became_peered":"2024-05-26T22:49:30.521749+0000","last_unstale":"2024-05-26T22:49:32.542355+0000","last_undegraded":"2024-05-26T22:49:32.542355+0000","last_fullsized":"2024-05-26T22:49:32.542355+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_clean_scrub_stamp":"2024-05-26T22:49:29.474733+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:31:54.415212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-26T22:49:33.691433+0000","last_change":"2024-05-26T22:49:31.564454+0000","last_active":"2024-05-26T22:49:33.691433+0000","last_peered":"2024-05-26T22:49:33.691433+0000","last_clean":"2024-05-26T22:49:33.691433+0000","last_became_active":"2024-05-26T22:49:31.564159+0000","last_became_peered":"2024-05-26T22:49:31.564159+0000","last_unstale":"2024-05-26T22:49:33.691433+0000","last_undegraded":"2024-05-26T22:49:33.691433+0000","last_fullsized":"2024-05-26T22:49:33.691433+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:25:44.839029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-26T22:49:33.688719+0000","last_change":"2024-05-26T22:49:30.521824+0000","last_active":"2024-05-26T22:49:33.688719+0000","last_peered":"2024-05-26T22:49:33.688719+0000","last_clean":"2024-05-26T22:49:33.688719+0000","last_became_active":"2024-05-26T22:49:30.521512+0000","last_became_peered":"2024-05-26T22:49:30.521512+0000","last_unstale":"2024-05-26T22:49:33.688719+0000","last_undegraded":"2024-05-26T22:49:33.688719+0000","last_fullsized":"2024-05-26T22:49:33.688719+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_clean_scrub_stamp":"2024-05-26T22:49:29.474733+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:16:25.826534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-26T22:49:32.586083+0000","last_change":"2024-05-26T22:49:31.542570+0000","last_active":"2024-05-26T22:49:32.586083+0000","last_peered":"2024-05-26T22:49:32.586083+0000","last_clean":"2024-05-26T22:49:32.586083+0000","last_became_active":"2024-05-26T22:49:31.542247+0000","last_became_peered":"2024-05-26T22:49:31.542247+0000","last_unstale":"2024-05-26T22:49:32.586083+0000","last_undegraded":"2024-05-26T22:49:32.586083+0000","last_fullsized":"2024-05-26T22:49:32.586083+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:25:44.839029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-26T22:49:33.688696+0000","last_change":"2024-05-26T22:49:30.525125+0000","last_active":"2024-05-26T22:49:33.688696+0000","last_peered":"2024-05-26T22:49:33.688696+0000","last_clean":"2024-05-26T22:49:33.688696+0000","last_became_active":"2024-05-26T22:49:30.524802+0000","last_became_peered":"2024-05-26T22:49:30.524802+0000","last_unstale":"2024-05-26T22:49:33.688696+0000","last_undegraded":"2024-05-26T22:49:33.688696+0000","last_fullsized":"2024-05-26T22:49:33.688696+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_clean_scrub_stamp":"2024-05-26T22:49:29.474733+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:22:24.471003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-26T22:49:33.687881+0000","last_change":"2024-05-26T22:49:31.548513+0000","last_active":"2024-05-26T22:49:33.687881+0000","last_peered":"2024-05-26T22:49:33.687881+0000","last_clean":"2024-05-26T22:49:33.687881+0000","last_became_active":"2024-05-26T22:49:31.548133+0000","last_became_peered":"2024-05-26T22:49:31.548133+0000","last_unstale":"2024-05-26T22:49:33.687881+0000","last_undegraded":"2024-05-26T22:49:33.687881+0000","last_fullsized":"2024-05-26T22:49:33.687881+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:05:15.944937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-26T22:49:32.545350+0000","last_change":"2024-05-26T22:49:30.522141+0000","last_active":"2024-05-26T22:49:32.545350+0000","last_peered":"2024-05-26T22:49:32.545350+0000","last_clean":"2024-05-26T22:49:32.545350+0000","last_became_active":"2024-05-26T22:49:30.521725+0000","last_became_peered":"2024-05-26T22:49:30.521725+0000","last_unstale":"2024-05-26T22:49:32.545350+0000","last_undegraded":"2024-05-26T22:49:32.545350+0000","last_fullsized":"2024-05-26T22:49:32.545350+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_clean_scrub_stamp":"2024-05-26T22:49:29.474733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:17:12.484601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-26T22:49:32.569145+0000","last_change":"2024-05-26T22:49:31.549247+0000","last_active":"2024-05-26T22:49:32.569145+0000","last_peered":"2024-05-26T22:49:32.569145+0000","last_clean":"2024-05-26T22:49:32.569145+0000","last_became_active":"2024-05-26T22:49:31.548945+0000","last_became_peered":"2024-05-26T22:49:31.548945+0000","last_unstale":"2024-05-26T22:49:32.569145+0000","last_undegraded":"2024-05-26T22:49:32.569145+0000","last_fullsized":"2024-05-26T22:49:32.569145+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:08:35.106387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-26T22:49:33.688012+0000","last_change":"2024-05-26T22:49:30.531886+0000","last_active":"2024-05-26T22:49:33.688012+0000","last_peered":"2024-05-26T22:49:33.688012+0000","last_clean":"2024-05-26T22:49:33.688012+0000","last_became_active":"2024-05-26T22:49:30.531485+0000","last_became_peered":"2024-05-26T22:49:30.531485+0000","last_unstale":"2024-05-26T22:49:33.688012+0000","last_undegraded":"2024-05-26T22:49:33.688012+0000","last_fullsized":"2024-05-26T22:49:33.688012+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_clean_scrub_stamp":"2024-05-26T22:49:29.474733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:35:36.545132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-26T22:49:32.586124+0000","last_change":"2024-05-26T22:49:31.550686+0000","last_active":"2024-05-26T22:49:32.586124+0000","last_peered":"2024-05-26T22:49:32.586124+0000","last_clean":"2024-05-26T22:49:32.586124+0000","last_became_active":"2024-05-26T22:49:31.550356+0000","last_became_peered":"2024-05-26T22:49:31.550356+0000","last_unstale":"2024-05-26T22:49:32.586124+0000","last_undegraded":"2024-05-26T22:49:32.586124+0000","last_fullsized":"2024-05-26T22:49:32.586124+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:27:35.980326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-26T22:49:32.546731+0000","last_change":"2024-05-26T22:49:30.532848+0000","last_active":"2024-05-26T22:49:32.546731+0000","last_peered":"2024-05-26T22:49:32.546731+0000","last_clean":"2024-05-26T22:49:32.546731+0000","last_became_active":"2024-05-26T22:49:30.532598+0000","last_became_peered":"2024-05-26T22:49:30.532598+0000","last_unstale":"2024-05-26T22:49:32.546731+0000","last_undegraded":"2024-05-26T22:49:32.546731+0000","last_fullsized":"2024-05-26T22:49:32.546731+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_clean_scrub_stamp":"2024-05-26T22:49:29.474733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:05:14.931600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-26T22:49:32.546660+0000","last_change":"2024-05-26T22:49:31.558984+0000","last_active":"2024-05-26T22:49:32.546660+0000","last_peered":"2024-05-26T22:49:32.546660+0000","last_clean":"2024-05-26T22:49:32.546660+0000","last_became_active":"2024-05-26T22:49:31.557846+0000","last_became_peered":"2024-05-26T22:49:31.557846+0000","last_unstale":"2024-05-26T22:49:32.546660+0000","last_undegraded":"2024-05-26T22:49:32.546660+0000","last_fullsized":"2024-05-26T22:49:32.546660+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:29:50.663367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-26T22:49:32.562689+0000","last_change":"2024-05-26T22:49:30.524735+0000","last_active":"2024-05-26T22:49:32.562689+0000","last_peered":"2024-05-26T22:49:32.562689+0000","last_clean":"2024-05-26T22:49:32.562689+0000","last_became_active":"2024-05-26T22:49:30.524354+0000","last_became_peered":"2024-05-26T22:49:30.524354+0000","last_unstale":"2024-05-26T22:49:32.562689+0000","last_undegraded":"2024-05-26T22:49:32.562689+0000","last_fullsized":"2024-05-26T22:49:32.562689+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_clean_scrub_stamp":"2024-05-26T22:49:29.474733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:18:37.082439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-26T22:49:32.546546+0000","last_change":"2024-05-26T22:49:31.553641+0000","last_active":"2024-05-26T22:49:32.546546+0000","last_peered":"2024-05-26T22:49:32.546546+0000","last_clean":"2024-05-26T22:49:32.546546+0000","last_became_active":"2024-05-26T22:49:31.553346+0000","last_became_peered":"2024-05-26T22:49:31.553346+0000","last_unstale":"2024-05-26T22:49:32.546546+0000","last_undegraded":"2024-05-26T22:49:32.546546+0000","last_fullsized":"2024-05-26T22:49:32.546546+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:41:28.909927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"32'3","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-26T22:49:32.585571+0000","last_change":"2024-05-26T22:49:30.533471+0000","last_active":"2024-05-26T22:49:32.585571+0000","last_peered":"2024-05-26T22:49:32.585571+0000","last_clean":"2024-05-26T22:49:32.585571+0000","last_became_active":"2024-05-26T22:49:30.533236+0000","last_became_peered":"2024-05-26T22:49:30.533236+0000","last_unstale":"2024-05-26T22:49:32.585571+0000","last_undegraded":"2024-05-26T22:49:32.585571+0000","last_fullsized":"2024-05-26T22:49:32.585571+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_clean_scrub_stamp":"2024-05-26T22:49:29.474733+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:45:14.583379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-26T22:49:32.585368+0000","last_change":"2024-05-26T22:49:31.556041+0000","last_active":"2024-05-26T22:49:32.585368+0000","last_peered":"2024-05-26T22:49:32.585368+0000","last_clean":"2024-05-26T22:49:32.585368+0000","last_became_active":"2024-05-26T22:49:31.555773+0000","last_became_peered":"2024-05-26T22:49:31.555773+0000","last_unstale":"2024-05-26T22:49:32.585368+0000","last_undegraded":"2024-05-26T22:49:32.585368+0000","last_fullsized":"2024-05-26T22:49:32.585368+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:29:59.315295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"32'6","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-26T22:49:32.546436+0000","last_change":"2024-05-26T22:49:30.528930+0000","last_active":"2024-05-26T22:49:32.546436+0000","last_peered":"2024-05-26T22:49:32.546436+0000","last_clean":"2024-05-26T22:49:32.546436+0000","last_became_active":"2024-05-26T22:49:30.528680+0000","last_became_peered":"2024-05-26T22:49:30.528680+0000","last_unstale":"2024-05-26T22:49:32.546436+0000","last_undegraded":"2024-05-26T22:49:32.546436+0000","last_fullsized":"2024-05-26T22:49:32.546436+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_clean_scrub_stamp":"2024-05-26T22:49:29.474733+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:18:37.082439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-26T22:49:32.569528+0000","last_change":"2024-05-26T22:49:31.548391+0000","last_active":"2024-05-26T22:49:32.569528+0000","last_peered":"2024-05-26T22:49:32.569528+0000","last_clean":"2024-05-26T22:49:32.569528+0000","last_became_active":"2024-05-26T22:49:31.548105+0000","last_became_peered":"2024-05-26T22:49:31.548105+0000","last_unstale":"2024-05-26T22:49:32.569528+0000","last_undegraded":"2024-05-26T22:49:32.569528+0000","last_fullsized":"2024-05-26T22:49:32.569528+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:06:35.987200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-26T22:49:33.689005+0000","last_change":"2024-05-26T22:49:30.530952+0000","last_active":"2024-05-26T22:49:33.689005+0000","last_peered":"2024-05-26T22:49:33.689005+0000","last_clean":"2024-05-26T22:49:33.689005+0000","last_became_active":"2024-05-26T22:49:30.530732+0000","last_became_peered":"2024-05-26T22:49:30.530732+0000","last_unstale":"2024-05-26T22:49:33.689005+0000","last_undegraded":"2024-05-26T22:49:33.689005+0000","last_fullsized":"2024-05-26T22:49:33.689005+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_clean_scrub_stamp":"2024-05-26T22:49:29.474733+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T23:42:16.175623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-26T22:49:32.546005+0000","last_change":"2024-05-26T22:49:31.552851+0000","last_active":"2024-05-26T22:49:32.546005+0000","last_peered":"2024-05-26T22:49:32.546005+0000","last_clean":"2024-05-26T22:49:32.546005+0000","last_became_active":"2024-05-26T22:49:31.552591+0000","last_became_peered":"2024-05-26T22:49:31.552591+0000","last_unstale":"2024-05-26T22:49:32.546005+0000","last_undegraded":"2024-05-26T22:49:32.546005+0000","last_fullsized":"2024-05-26T22:49:32.546005+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:02:54.311565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-26T22:49:32.568864+0000","last_change":"2024-05-26T22:49:30.527636+0000","last_active":"2024-05-26T22:49:32.568864+0000","last_peered":"2024-05-26T22:49:32.568864+0000","last_clean":"2024-05-26T22:49:32.568864+0000","last_became_active":"2024-05-26T22:49:30.527293+0000","last_became_peered":"2024-05-26T22:49:30.527293+0000","last_unstale":"2024-05-26T22:49:32.568864+0000","last_undegraded":"2024-05-26T22:49:32.568864+0000","last_fullsized":"2024-05-26T22:49:32.568864+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_clean_scrub_stamp":"2024-05-26T22:49:29.474733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:35:36.545132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-26T22:49:33.690278+0000","last_change":"2024-05-26T22:49:31.564909+0000","last_active":"2024-05-26T22:49:33.690278+0000","last_peered":"2024-05-26T22:49:33.690278+0000","last_clean":"2024-05-26T22:49:33.690278+0000","last_became_active":"2024-05-26T22:49:31.564649+0000","last_became_peered":"2024-05-26T22:49:31.564649+0000","last_unstale":"2024-05-26T22:49:33.690278+0000","last_undegraded":"2024-05-26T22:49:33.690278+0000","last_fullsized":"2024-05-26T22:49:33.690278+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:34:36.050113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-26T22:49:32.545249+0000","last_change":"2024-05-26T22:49:30.532857+0000","last_active":"2024-05-26T22:49:32.545249+0000","last_peered":"2024-05-26T22:49:32.545249+0000","last_clean":"2024-05-26T22:49:32.545249+0000","last_became_active":"2024-05-26T22:49:30.532479+0000","last_became_peered":"2024-05-26T22:49:30.532479+0000","last_unstale":"2024-05-26T22:49:32.545249+0000","last_undegraded":"2024-05-26T22:49:32.545249+0000","last_fullsized":"2024-05-26T22:49:32.545249+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_clean_scrub_stamp":"2024-05-26T22:49:29.474733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:58:47.501088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-26T22:49:32.546545+0000","last_change":"2024-05-26T22:49:31.551983+0000","last_active":"2024-05-26T22:49:32.546545+0000","last_peered":"2024-05-26T22:49:32.546545+0000","last_clean":"2024-05-26T22:49:32.546545+0000","last_became_active":"2024-05-26T22:49:31.551480+0000","last_became_peered":"2024-05-26T22:49:31.551480+0000","last_unstale":"2024-05-26T22:49:32.546545+0000","last_undegraded":"2024-05-26T22:49:32.546545+0000","last_fullsized":"2024-05-26T22:49:32.546545+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T22:56:26.086399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"32'8","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-26T22:49:32.545467+0000","last_change":"2024-05-26T22:49:30.524556+0000","last_active":"2024-05-26T22:49:32.545467+0000","last_peered":"2024-05-26T22:49:32.545467+0000","last_clean":"2024-05-26T22:49:32.545467+0000","last_became_active":"2024-05-26T22:49:30.524258+0000","last_became_peered":"2024-05-26T22:49:30.524258+0000","last_unstale":"2024-05-26T22:49:32.545467+0000","last_undegraded":"2024-05-26T22:49:32.545467+0000","last_fullsized":"2024-05-26T22:49:32.545467+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_clean_scrub_stamp":"2024-05-26T22:49:29.474733+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T22:56:25.073062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-26T22:49:33.689116+0000","last_change":"2024-05-26T22:49:31.562928+0000","last_active":"2024-05-26T22:49:33.689116+0000","last_peered":"2024-05-26T22:49:33.689116+0000","last_clean":"2024-05-26T22:49:33.689116+0000","last_became_active":"2024-05-26T22:49:31.562617+0000","last_became_peered":"2024-05-26T22:49:31.562617+0000","last_unstale":"2024-05-26T22:49:33.689116+0000","last_undegraded":"2024-05-26T22:49:33.689116+0000","last_fullsized":"2024-05-26T22:49:33.689116+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:59:06.763141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-26T22:49:32.570310+0000","last_change":"2024-05-26T22:49:30.526382+0000","last_active":"2024-05-26T22:49:32.570310+0000","last_peered":"2024-05-26T22:49:32.570310+0000","last_clean":"2024-05-26T22:49:32.570310+0000","last_became_active":"2024-05-26T22:49:30.525846+0000","last_became_peered":"2024-05-26T22:49:30.525846+0000","last_unstale":"2024-05-26T22:49:32.570310+0000","last_undegraded":"2024-05-26T22:49:32.570310+0000","last_fullsized":"2024-05-26T22:49:32.570310+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_clean_scrub_stamp":"2024-05-26T22:49:29.474733+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:31:53.309746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-26T22:49:32.585759+0000","last_change":"2024-05-26T22:49:31.565000+0000","last_active":"2024-05-26T22:49:32.585759+0000","last_peered":"2024-05-26T22:49:32.585759+0000","last_clean":"2024-05-26T22:49:32.585759+0000","last_became_active":"2024-05-26T22:49:31.564675+0000","last_became_peered":"2024-05-26T22:49:31.564675+0000","last_unstale":"2024-05-26T22:49:32.585759+0000","last_undegraded":"2024-05-26T22:49:32.585759+0000","last_fullsized":"2024-05-26T22:49:32.585759+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:41:28.909927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-26T22:49:33.689298+0000","last_change":"2024-05-26T22:49:30.531872+0000","last_active":"2024-05-26T22:49:33.689298+0000","last_peered":"2024-05-26T22:49:33.689298+0000","last_clean":"2024-05-26T22:49:33.689298+0000","last_became_active":"2024-05-26T22:49:30.531433+0000","last_became_peered":"2024-05-26T22:49:30.531433+0000","last_unstale":"2024-05-26T22:49:33.689298+0000","last_undegraded":"2024-05-26T22:49:33.689298+0000","last_fullsized":"2024-05-26T22:49:33.689298+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_clean_scrub_stamp":"2024-05-26T22:49:29.474733+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:20:30.150930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-26T22:49:32.569715+0000","last_change":"2024-05-26T22:49:31.571094+0000","last_active":"2024-05-26T22:49:32.569715+0000","last_peered":"2024-05-26T22:49:32.569715+0000","last_clean":"2024-05-26T22:49:32.569715+0000","last_became_active":"2024-05-26T22:49:31.563951+0000","last_became_peered":"2024-05-26T22:49:31.563951+0000","last_unstale":"2024-05-26T22:49:32.569715+0000","last_undegraded":"2024-05-26T22:49:32.569715+0000","last_fullsized":"2024-05-26T22:49:32.569715+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:51:00.046866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-26T22:49:33.689497+0000","last_change":"2024-05-26T22:49:30.519755+0000","last_active":"2024-05-26T22:49:33.689497+0000","last_peered":"2024-05-26T22:49:33.689497+0000","last_clean":"2024-05-26T22:49:33.689497+0000","last_became_active":"2024-05-26T22:49:30.518432+0000","last_became_peered":"2024-05-26T22:49:30.518432+0000","last_unstale":"2024-05-26T22:49:33.689497+0000","last_undegraded":"2024-05-26T22:49:33.689497+0000","last_fullsized":"2024-05-26T22:49:33.689497+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_clean_scrub_stamp":"2024-05-26T22:49:29.474733+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:25:16.141222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-26T22:49:32.542974+0000","last_change":"2024-05-26T22:49:31.559867+0000","last_active":"2024-05-26T22:49:32.542974+0000","last_peered":"2024-05-26T22:49:32.542974+0000","last_clean":"2024-05-26T22:49:32.542974+0000","last_became_active":"2024-05-26T22:49:31.559601+0000","last_became_peered":"2024-05-26T22:49:31.559601+0000","last_unstale":"2024-05-26T22:49:32.542974+0000","last_undegraded":"2024-05-26T22:49:32.542974+0000","last_fullsized":"2024-05-26T22:49:32.542974+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:37:17.831291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-26T22:49:33.688977+0000","last_change":"2024-05-26T22:49:30.526658+0000","last_active":"2024-05-26T22:49:33.688977+0000","last_peered":"2024-05-26T22:49:33.688977+0000","last_clean":"2024-05-26T22:49:33.688977+0000","last_became_active":"2024-05-26T22:49:30.526308+0000","last_became_peered":"2024-05-26T22:49:30.526308+0000","last_unstale":"2024-05-26T22:49:33.688977+0000","last_undegraded":"2024-05-26T22:49:33.688977+0000","last_fullsized":"2024-05-26T22:49:33.688977+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_clean_scrub_stamp":"2024-05-26T22:49:29.474733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:37:20.139565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-26T22:49:32.541806+0000","last_change":"2024-05-26T22:49:31.561406+0000","last_active":"2024-05-26T22:49:32.541806+0000","last_peered":"2024-05-26T22:49:32.541806+0000","last_clean":"2024-05-26T22:49:32.541806+0000","last_became_active":"2024-05-26T22:49:31.560876+0000","last_became_peered":"2024-05-26T22:49:31.560876+0000","last_unstale":"2024-05-26T22:49:32.541806+0000","last_undegraded":"2024-05-26T22:49:32.541806+0000","last_fullsized":"2024-05-26T22:49:32.541806+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T22:49:42.451996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-26T22:49:32.562090+0000","last_change":"2024-05-26T22:49:30.529933+0000","last_active":"2024-05-26T22:49:32.562090+0000","last_peered":"2024-05-26T22:49:32.562090+0000","last_clean":"2024-05-26T22:49:32.562090+0000","last_became_active":"2024-05-26T22:49:30.529650+0000","last_became_peered":"2024-05-26T22:49:30.529650+0000","last_unstale":"2024-05-26T22:49:32.562090+0000","last_undegraded":"2024-05-26T22:49:32.562090+0000","last_fullsized":"2024-05-26T22:49:32.562090+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_clean_scrub_stamp":"2024-05-26T22:49:29.474733+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:48:06.612985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-26T22:49:32.542388+0000","last_change":"2024-05-26T22:49:31.550040+0000","last_active":"2024-05-26T22:49:32.542388+0000","last_peered":"2024-05-26T22:49:32.542388+0000","last_clean":"2024-05-26T22:49:32.542388+0000","last_became_active":"2024-05-26T22:49:31.549750+0000","last_became_peered":"2024-05-26T22:49:31.549750+0000","last_unstale":"2024-05-26T22:49:32.542388+0000","last_undegraded":"2024-05-26T22:49:32.542388+0000","last_fullsized":"2024-05-26T22:49:32.542388+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:35:37.558469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-26T22:49:32.546314+0000","last_change":"2024-05-26T22:49:30.529565+0000","last_active":"2024-05-26T22:49:32.546314+0000","last_peered":"2024-05-26T22:49:32.546314+0000","last_clean":"2024-05-26T22:49:32.546314+0000","last_became_active":"2024-05-26T22:49:30.528777+0000","last_became_peered":"2024-05-26T22:49:30.528777+0000","last_unstale":"2024-05-26T22:49:32.546314+0000","last_undegraded":"2024-05-26T22:49:32.546314+0000","last_fullsized":"2024-05-26T22:49:32.546314+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_clean_scrub_stamp":"2024-05-26T22:49:29.474733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:35:36.545132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-26T22:49:32.541578+0000","last_change":"2024-05-26T22:49:31.545663+0000","last_active":"2024-05-26T22:49:32.541578+0000","last_peered":"2024-05-26T22:49:32.541578+0000","last_clean":"2024-05-26T22:49:32.541578+0000","last_became_active":"2024-05-26T22:49:31.545397+0000","last_became_peered":"2024-05-26T22:49:31.545397+0000","last_unstale":"2024-05-26T22:49:32.541578+0000","last_undegraded":"2024-05-26T22:49:32.541578+0000","last_fullsized":"2024-05-26T22:49:32.541578+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:22:25.484340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-26T22:49:33.687672+0000","last_change":"2024-05-26T22:49:30.524213+0000","last_active":"2024-05-26T22:49:33.687672+0000","last_peered":"2024-05-26T22:49:33.687672+0000","last_clean":"2024-05-26T22:49:33.687672+0000","last_became_active":"2024-05-26T22:49:30.523770+0000","last_became_peered":"2024-05-26T22:49:30.523770+0000","last_unstale":"2024-05-26T22:49:33.687672+0000","last_undegraded":"2024-05-26T22:49:33.687672+0000","last_fullsized":"2024-05-26T22:49:33.687672+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_clean_scrub_stamp":"2024-05-26T22:49:29.474733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T22:49:41.438659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-26T22:49:33.691035+0000","last_change":"2024-05-26T22:49:31.554613+0000","last_active":"2024-05-26T22:49:33.691035+0000","last_peered":"2024-05-26T22:49:33.691035+0000","last_clean":"2024-05-26T22:49:33.691035+0000","last_became_active":"2024-05-26T22:49:31.554202+0000","last_became_peered":"2024-05-26T22:49:31.554202+0000","last_unstale":"2024-05-26T22:49:33.691035+0000","last_undegraded":"2024-05-26T22:49:33.691035+0000","last_fullsized":"2024-05-26T22:49:33.691035+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T23:32:58.138139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-26T22:49:32.584715+0000","last_change":"2024-05-26T22:49:30.529938+0000","last_active":"2024-05-26T22:49:32.584715+0000","last_peered":"2024-05-26T22:49:32.584715+0000","last_clean":"2024-05-26T22:49:32.584715+0000","last_became_active":"2024-05-26T22:49:30.529607+0000","last_became_peered":"2024-05-26T22:49:30.529607+0000","last_unstale":"2024-05-26T22:49:32.584715+0000","last_undegraded":"2024-05-26T22:49:32.584715+0000","last_fullsized":"2024-05-26T22:49:32.584715+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_clean_scrub_stamp":"2024-05-26T22:49:29.474733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:18:37.082439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-26T22:49:32.542978+0000","last_change":"2024-05-26T22:49:31.546429+0000","last_active":"2024-05-26T22:49:32.542978+0000","last_peered":"2024-05-26T22:49:32.542978+0000","last_clean":"2024-05-26T22:49:32.542978+0000","last_became_active":"2024-05-26T22:49:31.545970+0000","last_became_peered":"2024-05-26T22:49:31.545970+0000","last_unstale":"2024-05-26T22:49:32.542978+0000","last_undegraded":"2024-05-26T22:49:32.542978+0000","last_fullsized":"2024-05-26T22:49:32.542978+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:20:31.164267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-26T22:49:32.570491+0000","last_change":"2024-05-26T22:49:30.537095+0000","last_active":"2024-05-26T22:49:32.570491+0000","last_peered":"2024-05-26T22:49:32.570491+0000","last_clean":"2024-05-26T22:49:32.570491+0000","last_became_active":"2024-05-26T22:49:30.536827+0000","last_became_peered":"2024-05-26T22:49:30.536827+0000","last_unstale":"2024-05-26T22:49:32.570491+0000","last_undegraded":"2024-05-26T22:49:32.570491+0000","last_fullsized":"2024-05-26T22:49:32.570491+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_clean_scrub_stamp":"2024-05-26T22:49:29.474733+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:48:06.612985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-26T22:49:32.586047+0000","last_change":"2024-05-26T22:49:31.547854+0000","last_active":"2024-05-26T22:49:32.586047+0000","last_peered":"2024-05-26T22:49:32.586047+0000","last_clean":"2024-05-26T22:49:32.586047+0000","last_became_active":"2024-05-26T22:49:31.547509+0000","last_became_peered":"2024-05-26T22:49:31.547509+0000","last_unstale":"2024-05-26T22:49:32.586047+0000","last_undegraded":"2024-05-26T22:49:32.586047+0000","last_fullsized":"2024-05-26T22:49:32.586047+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:29:50.663367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"33'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-26T22:49:32.562738+0000","last_change":"2024-05-26T22:49:30.529049+0000","last_active":"2024-05-26T22:49:32.562738+0000","last_peered":"2024-05-26T22:49:32.562738+0000","last_clean":"2024-05-26T22:49:32.562738+0000","last_became_active":"2024-05-26T22:49:30.528867+0000","last_became_peered":"2024-05-26T22:49:30.528867+0000","last_unstale":"2024-05-26T22:49:32.562738+0000","last_undegraded":"2024-05-26T22:49:32.562738+0000","last_fullsized":"2024-05-26T22:49:32.562738+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_clean_scrub_stamp":"2024-05-26T22:49:29.474733+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:17:12.484601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-26T22:49:32.562345+0000","last_change":"2024-05-26T22:49:31.555662+0000","last_active":"2024-05-26T22:49:32.562345+0000","last_peered":"2024-05-26T22:49:32.562345+0000","last_clean":"2024-05-26T22:49:32.562345+0000","last_became_active":"2024-05-26T22:49:31.555330+0000","last_became_peered":"2024-05-26T22:49:31.555330+0000","last_unstale":"2024-05-26T22:49:32.562345+0000","last_undegraded":"2024-05-26T22:49:32.562345+0000","last_fullsized":"2024-05-26T22:49:32.562345+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:51:00.046866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-26T22:49:32.569760+0000","last_change":"2024-05-26T22:49:30.523981+0000","last_active":"2024-05-26T22:49:32.569760+0000","last_peered":"2024-05-26T22:49:32.569760+0000","last_clean":"2024-05-26T22:49:32.569760+0000","last_became_active":"2024-05-26T22:49:30.523424+0000","last_became_peered":"2024-05-26T22:49:30.523424+0000","last_unstale":"2024-05-26T22:49:32.569760+0000","last_undegraded":"2024-05-26T22:49:32.569760+0000","last_fullsized":"2024-05-26T22:49:32.569760+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_clean_scrub_stamp":"2024-05-26T22:49:29.474733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:58:47.501088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-26T22:49:32.546529+0000","last_change":"2024-05-26T22:49:31.543299+0000","last_active":"2024-05-26T22:49:32.546529+0000","last_peered":"2024-05-26T22:49:32.546529+0000","last_clean":"2024-05-26T22:49:32.546529+0000","last_became_active":"2024-05-26T22:49:31.542933+0000","last_became_peered":"2024-05-26T22:49:31.542933+0000","last_unstale":"2024-05-26T22:49:32.546529+0000","last_undegraded":"2024-05-26T22:49:32.546529+0000","last_fullsized":"2024-05-26T22:49:32.546529+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:17:13.497938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-26T22:49:32.562173+0000","last_change":"2024-05-26T22:49:30.532702+0000","last_active":"2024-05-26T22:49:32.562173+0000","last_peered":"2024-05-26T22:49:32.562173+0000","last_clean":"2024-05-26T22:49:32.562173+0000","last_became_active":"2024-05-26T22:49:30.532387+0000","last_became_peered":"2024-05-26T22:49:30.532387+0000","last_unstale":"2024-05-26T22:49:32.562173+0000","last_undegraded":"2024-05-26T22:49:32.562173+0000","last_fullsized":"2024-05-26T22:49:32.562173+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_clean_scrub_stamp":"2024-05-26T22:49:29.474733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:58:47.501088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-26T22:49:32.561961+0000","last_change":"2024-05-26T22:49:31.547259+0000","last_active":"2024-05-26T22:49:32.561961+0000","last_peered":"2024-05-26T22:49:32.561961+0000","last_clean":"2024-05-26T22:49:32.561961+0000","last_became_active":"2024-05-26T22:49:31.546888+0000","last_became_peered":"2024-05-26T22:49:31.546888+0000","last_unstale":"2024-05-26T22:49:32.561961+0000","last_undegraded":"2024-05-26T22:49:32.561961+0000","last_fullsized":"2024-05-26T22:49:32.561961+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:06:35.987200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-26T22:49:32.568975+0000","last_change":"2024-05-26T22:49:30.534502+0000","last_active":"2024-05-26T22:49:32.568975+0000","last_peered":"2024-05-26T22:49:32.568975+0000","last_clean":"2024-05-26T22:49:32.568975+0000","last_became_active":"2024-05-26T22:49:30.534253+0000","last_became_peered":"2024-05-26T22:49:30.534253+0000","last_unstale":"2024-05-26T22:49:32.568975+0000","last_undegraded":"2024-05-26T22:49:32.568975+0000","last_fullsized":"2024-05-26T22:49:32.568975+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_clean_scrub_stamp":"2024-05-26T22:49:29.474733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:20:30.150930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-26T22:49:32.545450+0000","last_change":"2024-05-26T22:49:31.562371+0000","last_active":"2024-05-26T22:49:32.545450+0000","last_peered":"2024-05-26T22:49:32.545450+0000","last_clean":"2024-05-26T22:49:32.545450+0000","last_became_active":"2024-05-26T22:49:31.562068+0000","last_became_peered":"2024-05-26T22:49:31.562068+0000","last_unstale":"2024-05-26T22:49:32.545450+0000","last_undegraded":"2024-05-26T22:49:32.545450+0000","last_fullsized":"2024-05-26T22:49:32.545450+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:29:59.315295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-26T22:49:33.687989+0000","last_change":"2024-05-26T22:49:30.537046+0000","last_active":"2024-05-26T22:49:33.687989+0000","last_peered":"2024-05-26T22:49:33.687989+0000","last_clean":"2024-05-26T22:49:33.687989+0000","last_became_active":"2024-05-26T22:49:30.536778+0000","last_became_peered":"2024-05-26T22:49:30.536778+0000","last_unstale":"2024-05-26T22:49:33.687989+0000","last_undegraded":"2024-05-26T22:49:33.687989+0000","last_fullsized":"2024-05-26T22:49:33.687989+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_clean_scrub_stamp":"2024-05-26T22:49:29.474733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:56:22.334567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-26T22:49:33.687839+0000","last_change":"2024-05-26T22:49:31.572522+0000","last_active":"2024-05-26T22:49:33.687839+0000","last_peered":"2024-05-26T22:49:33.687839+0000","last_clean":"2024-05-26T22:49:33.687839+0000","last_became_active":"2024-05-26T22:49:31.572182+0000","last_became_peered":"2024-05-26T22:49:31.572182+0000","last_unstale":"2024-05-26T22:49:33.687839+0000","last_undegraded":"2024-05-26T22:49:33.687839+0000","last_fullsized":"2024-05-26T22:49:33.687839+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:23:06.128279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-26T22:49:32.545195+0000","last_change":"2024-05-26T22:49:30.531986+0000","last_active":"2024-05-26T22:49:32.545195+0000","last_peered":"2024-05-26T22:49:32.545195+0000","last_clean":"2024-05-26T22:49:32.545195+0000","last_became_active":"2024-05-26T22:49:30.531697+0000","last_became_peered":"2024-05-26T22:49:30.531697+0000","last_unstale":"2024-05-26T22:49:32.545195+0000","last_undegraded":"2024-05-26T22:49:32.545195+0000","last_fullsized":"2024-05-26T22:49:32.545195+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_clean_scrub_stamp":"2024-05-26T22:49:29.474733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:23:05.114942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-26T22:49:33.688498+0000","last_change":"2024-05-26T22:49:31.562108+0000","last_active":"2024-05-26T22:49:33.688498+0000","last_peered":"2024-05-26T22:49:33.688498+0000","last_clean":"2024-05-26T22:49:33.688498+0000","last_became_active":"2024-05-26T22:49:31.561746+0000","last_became_peered":"2024-05-26T22:49:31.561746+0000","last_unstale":"2024-05-26T22:49:33.688498+0000","last_undegraded":"2024-05-26T22:49:33.688498+0000","last_fullsized":"2024-05-26T22:49:33.688498+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:30.488070+0000","last_clean_scrub_stamp":"2024-05-26T22:49:30.488070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:10:43.375671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-26T22:49:32.546446+0000","last_change":"2024-05-26T22:49:30.527931+0000","last_active":"2024-05-26T22:49:32.546446+0000","last_peered":"2024-05-26T22:49:32.546446+0000","last_clean":"2024-05-26T22:49:32.546446+0000","last_became_active":"2024-05-26T22:49:30.526765+0000","last_became_peered":"2024-05-26T22:49:30.526765+0000","last_unstale":"2024-05-26T22:49:32.546446+0000","last_undegraded":"2024-05-26T22:49:32.546446+0000","last_fullsized":"2024-05-26T22:49:32.546446+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:49:29.474733+0000","last_clean_scrub_stamp":"2024-05-26T22:49:29.474733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:16:25.826534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":194,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-26T22:49:33.690747+0000","last_change":"2024-05-26T22:48:12.068917+0000","last_active":"2024-05-26T22:49:33.690747+0000","last_peered":"2024-05-26T22:49:33.690747+0000","last_clean":"2024-05-26T22:49:33.690747+0000","last_became_active":"2024-05-26T22:48:12.065934+0000","last_became_peered":"2024-05-26T22:48:12.065934+0000","last_unstale":"2024-05-26T22:49:33.690747+0000","last_undegraded":"2024-05-26T22:49:33.690747+0000","last_fullsized":"2024-05-26T22:49:33.690747+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:10.628122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:10.628122+0000","last_clean_scrub_stamp":"2024-05-26T22:48:10.628122+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:14:58.895911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574868,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27968,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343872,"statfs":{"total":96636764160,"available":96608124928,"internally_reserved":0,"allocated":970752,"data_stored":737066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574868,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":380928,"data_stored":147224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574869,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":380928,"data_stored":147280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574868,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":380928,"data_stored":146712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574868,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":389120,"data_stored":148242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574868,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":376832,"data_stored":146678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574868,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":389120,"data_stored":148242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574868,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27972,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343868,"statfs":{"total":96636764160,"available":96608120832,"internally_reserved":0,"allocated":966656,"data_stored":736976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-26T22:49:36.523 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-26T22:49:36.523 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-26T22:49:36.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.778+0000 7f42a8a00640 1 Processor -- start 2024-05-26T22:49:36.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.778+0000 7f42a8a00640 1 -- start start 2024-05-26T22:49:36.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.780+0000 7f42a8a00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f42a40d0550 0x7f42a40ae250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:36.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.780+0000 7f42a8a00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42a40b2440 con 0x7f42a415e320 2024-05-26T22:49:36.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.780+0000 7f42a8a00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42a4162230 con 0x7f42a40b2f60 2024-05-26T22:49:36.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.780+0000 7f42a8a00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f42a4161fe0 con 0x7f42a40d0550 2024-05-26T22:49:36.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.780+0000 7f42a2800640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f42a415e320 0x7f42a40cfe40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:48420/0 (socket says 172.21.15.131:48420) 2024-05-26T22:49:36.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.780+0000 7f42a2800640 1 -- 172.21.15.131:0/2786395510 learned_addr learned my addr 172.21.15.131:0/2786395510 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:49:36.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.781+0000 7f42a1400640 1 -- 172.21.15.131:0/2786395510 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3578036804 0 0) 0x7f42a40b2440 con 0x7f42a415e320 2024-05-26T22:49:36.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.781+0000 7f42a1400640 1 -- 172.21.15.131:0/2786395510 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4288003520 con 0x7f42a415e320 2024-05-26T22:49:36.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.781+0000 7f42a1e00640 1 --2- 172.21.15.131:0/2786395510 >> v2:172.21.15.187:3300/0 conn(0x7f42a40d0550 0x7f42a40ae250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:36.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.781+0000 7f42a1400640 1 -- 172.21.15.131:0/2786395510 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3075643856 0 0) 0x7f4288003520 con 0x7f42a415e320 2024-05-26T22:49:36.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.781+0000 7f42a1400640 1 -- 172.21.15.131:0/2786395510 >> v2:172.21.15.187:3300/0 conn(0x7f42a40d0550 msgr2=0x7f42a40ae250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:36.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.781+0000 7f42a1400640 1 --2- 172.21.15.131:0/2786395510 >> v2:172.21.15.187:3300/0 conn(0x7f42a40d0550 0x7f42a40ae250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:36.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.781+0000 7f42a1400640 1 -- 172.21.15.131:0/2786395510 >> v1:172.21.15.187:6789/0 conn(0x7f42a40b2f60 legacy=0x7f42a40715d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:36.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.781+0000 7f42a1400640 1 -- 172.21.15.131:0/2786395510 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42a4162510 con 0x7f42a415e320 2024-05-26T22:49:36.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.781+0000 7f42a1400640 1 -- 172.21.15.131:0/2786395510 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7f4294001670 con 0x7f42a415e320 2024-05-26T22:49:36.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.781+0000 7f42a1400640 1 -- 172.21.15.131:0/2786395510 >> v1:172.21.15.131:6789/0 conn(0x7f42a415e320 legacy=0x7f42a40cfe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:36.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.781+0000 7f42a1400640 1 --2- 172.21.15.131:0/2786395510 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4288003a00 0x7f4288003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:36.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.781+0000 7f42a1400640 1 --2- 172.21.15.131:0/2786395510 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4288004800 0x7f4288004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:36.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.781+0000 7f42a1400640 1 --2- 172.21.15.131:0/2786395510 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4288005210 0x7f428800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:36.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.781+0000 7f42a1400640 1 -- 172.21.15.131:0/2786395510 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f42a4161fe0 con 0x7f4288005210 2024-05-26T22:49:36.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.781+0000 7f42a1400640 1 -- 172.21.15.131:0/2786395510 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f428800c120 con 0x7f4288004800 2024-05-26T22:49:36.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.781+0000 7f42a1400640 1 -- 172.21.15.131:0/2786395510 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f428800c320 con 0x7f4288003a00 2024-05-26T22:49:36.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.782+0000 7f42a3200640 1 --2- 172.21.15.131:0/2786395510 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4288005210 0x7f428800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:36.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.782+0000 7f42a1e00640 1 --2- 172.21.15.131:0/2786395510 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4288004800 0x7f4288004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:36.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.782+0000 7f42a2800640 1 --2- 172.21.15.131:0/2786395510 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4288003a00 0x7f4288003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:36.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.782+0000 7f42a3200640 1 -- 172.21.15.131:0/2786395510 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4288003a00 msgr2=0x7f4288003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:36.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.782+0000 7f42a3200640 1 --2- 172.21.15.131:0/2786395510 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4288003a00 0x7f4288003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:36.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.782+0000 7f42a3200640 1 -- 172.21.15.131:0/2786395510 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4288004800 msgr2=0x7f4288004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:36.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.782+0000 7f42a3200640 1 --2- 172.21.15.131:0/2786395510 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4288004800 0x7f4288004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:36.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.782+0000 7f42a3200640 1 -- 172.21.15.131:0/2786395510 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4288003520 con 0x7f4288005210 2024-05-26T22:49:36.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.782+0000 7f42a2800640 1 --2- 172.21.15.131:0/2786395510 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4288003a00 0x7f4288003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:49:36.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.783+0000 7f42a3200640 1 -- 172.21.15.131:0/2786395510 reap_dead start 2024-05-26T22:49:36.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.783+0000 7f42a3200640 1 --2- 172.21.15.131:0/2786395510 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4288005210 0x7f428800b9c0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f429c0044b0 tx=0x7f4288005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46d8a09ed4d0d895 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:36.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.783+0000 7f42a1400640 1 -- 172.21.15.131:0/2786395510 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f429c004080 con 0x7f4288005210 2024-05-26T22:49:36.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.783+0000 7f42a1400640 1 -- 172.21.15.131:0/2786395510 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f429c02ada0 con 0x7f4288005210 2024-05-26T22:49:36.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.783+0000 7f42a1400640 1 -- 172.21.15.131:0/2786395510 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f429c006d90 con 0x7f4288005210 2024-05-26T22:49:36.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.784+0000 7f42a8a00640 1 -- 172.21.15.131:0/2786395510 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4288005210 msgr2=0x7f428800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:36.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.784+0000 7f42a8a00640 1 --2- 172.21.15.131:0/2786395510 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4288005210 0x7f428800b9c0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f429c0044b0 tx=0x7f4288005830 comp rx=0 tx=0).stop 2024-05-26T22:49:36.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.784+0000 7f42a8a00640 1 -- 172.21.15.131:0/2786395510 shutdown_connections 2024-05-26T22:49:36.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.784+0000 7f42a8a00640 1 --2- 172.21.15.131:0/2786395510 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4288005210 0x7f428800b9c0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:36.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.784+0000 7f42a8a00640 1 -- 172.21.15.131:0/2786395510 >> 172.21.15.131:0/2786395510 conn(0x7f42a40b5660 msgr2=0x7f42a40b5a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:36.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.784+0000 7f42a8a00640 1 -- 172.21.15.131:0/2786395510 shutdown_connections 2024-05-26T22:49:36.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.784+0000 7f42a8a00640 1 -- 172.21.15.131:0/2786395510 wait complete. 2024-05-26T22:49:36.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.786+0000 7f42a8a00640 1 Processor -- start 2024-05-26T22:49:36.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.786+0000 7f42a8a00640 1 -- start start 2024-05-26T22:49:36.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.787+0000 7f42a8a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4288005210 0x7f42a41f3340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:36.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.787+0000 7f42a8a00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f42a415c9f0 0x7f42a415ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:36.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.787+0000 7f42a8a00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f42a415f590 0x7f42a415f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:36.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.787+0000 7f42a8a00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f42a4162e40 con 0x7f4288005210 2024-05-26T22:49:36.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.787+0000 7f42a8a00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f42a4162ad0 con 0x7f42a415f590 2024-05-26T22:49:36.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.787+0000 7f42a8a00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f42a40b3570 con 0x7f42a415c9f0 2024-05-26T22:49:36.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.787+0000 7f42a2800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4288005210 0x7f42a41f3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:36.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.788+0000 7f42a2800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4288005210 0x7f42a41f3340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53234/0 (socket says 172.21.15.131:53234) 2024-05-26T22:49:36.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.788+0000 7f42a2800640 1 -- 172.21.15.131:0/3355898471 learned_addr learned my addr 172.21.15.131:0/3355898471 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:36.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.788+0000 7f42a3200640 1 --2- 172.21.15.131:0/3355898471 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f42a415f590 0x7f42a415f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:36.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.788+0000 7f42a2800640 1 -- 172.21.15.131:0/3355898471 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f42a415c9f0 msgr2=0x7f42a415ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:36.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.788+0000 7f42a2800640 1 --2- 172.21.15.131:0/3355898471 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f42a415c9f0 0x7f42a415ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:36.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.788+0000 7f42a2800640 1 -- 172.21.15.131:0/3355898471 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f42a415f590 msgr2=0x7f42a415f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:36.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.788+0000 7f42a2800640 1 --2- 172.21.15.131:0/3355898471 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f42a415f590 0x7f42a415f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:36.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.788+0000 7f42a2800640 1 -- 172.21.15.131:0/3355898471 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42a415d3b0 con 0x7f4288005210 2024-05-26T22:49:36.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.788+0000 7f42a3200640 1 --2- 172.21.15.131:0/3355898471 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f42a415f590 0x7f42a415f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:49:36.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.788+0000 7f42a2800640 1 --2- 172.21.15.131:0/3355898471 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4288005210 0x7f42a41f3340 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f429400d7f0 tx=0x7f429400dcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:36.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.789+0000 7f429aa00640 1 -- 172.21.15.131:0/3355898471 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4294019070 con 0x7f4288005210 2024-05-26T22:49:36.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.789+0000 7f429aa00640 1 -- 172.21.15.131:0/3355898471 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42940074e0 con 0x7f4288005210 2024-05-26T22:49:36.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.789+0000 7f429aa00640 1 -- 172.21.15.131:0/3355898471 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4294004a00 con 0x7f4288005210 2024-05-26T22:49:36.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.789+0000 7f42a8a00640 1 -- 172.21.15.131:0/3355898471 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42a4160560 con 0x7f4288005210 2024-05-26T22:49:36.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.789+0000 7f42a8a00640 1 -- 172.21.15.131:0/3355898471 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42a4148660 con 0x7f4288005210 2024-05-26T22:49:36.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.790+0000 7f42a8a00640 1 -- 172.21.15.131:0/3355898471 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42a40694e0 con 0x7f4288005210 2024-05-26T22:49:36.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.791+0000 7f429aa00640 1 -- 172.21.15.131:0/3355898471 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4294004ba0 con 0x7f4288005210 2024-05-26T22:49:36.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.792+0000 7f429aa00640 1 --2- 172.21.15.131:0/3355898471 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f428406be30 0x7f428406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:36.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.792+0000 7f429aa00640 1 -- 172.21.15.131:0/3355898471 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f429408a9a0 con 0x7f4288005210 2024-05-26T22:49:36.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.792+0000 7f42a1e00640 1 --2- 172.21.15.131:0/3355898471 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f428406be30 0x7f428406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:36.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.793+0000 7f42a1e00640 1 --2- 172.21.15.131:0/3355898471 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f428406be30 0x7f428406e2f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f42a41608c0 tx=0x7f429000d280 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:36.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:36.796+0000 7f429aa00640 1 -- 172.21.15.131:0/3355898471 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f42940592b0 con 0x7f4288005210 2024-05-26T22:49:37.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:37.069+0000 7f42a8a00640 1 -- 172.21.15.131:0/3355898471 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f42a40cfc20 con 0x7f4288005210 2024-05-26T22:49:37.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:37.073+0000 7f429aa00640 1 -- 172.21.15.131:0/3355898471 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) ==== 190+0+147 (secure 0 0 0) 0x7f4294058bb0 con 0x7f4288005210 2024-05-26T22:49:37.073 INFO:teuthology.orchestra.run.smithi131.stdout:[client.0] 2024-05-26T22:49:37.073 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQD2u1NmSGVWChAAH/wq3iaUs6ZELRhLKhmURQ== 2024-05-26T22:49:37.073 INFO:teuthology.orchestra.run.smithi131.stdout: caps mds = "allow" 2024-05-26T22:49:37.073 INFO:teuthology.orchestra.run.smithi131.stdout: caps mon = "allow r" 2024-05-26T22:49:37.073 INFO:teuthology.orchestra.run.smithi131.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-05-26T22:49:37.074 INFO:teuthology.orchestra.run.smithi131.stderr:updated caps for client.0 2024-05-26T22:49:37.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:37.076+0000 7f42a8a00640 1 -- 172.21.15.131:0/3355898471 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f428406be30 msgr2=0x7f428406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:37.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:37.076+0000 7f42a8a00640 1 --2- 172.21.15.131:0/3355898471 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f428406be30 0x7f428406e2f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f42a41608c0 tx=0x7f429000d280 comp rx=0 tx=0).stop 2024-05-26T22:49:37.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:37.076+0000 7f42a8a00640 1 -- 172.21.15.131:0/3355898471 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4288005210 msgr2=0x7f42a41f3340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:37.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:37.076+0000 7f42a8a00640 1 --2- 172.21.15.131:0/3355898471 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4288005210 0x7f42a41f3340 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f429400d7f0 tx=0x7f429400dcb0 comp rx=0 tx=0).stop 2024-05-26T22:49:37.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:37.076+0000 7f42a8a00640 1 -- 172.21.15.131:0/3355898471 shutdown_connections 2024-05-26T22:49:37.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:37.076+0000 7f42a8a00640 1 --2- 172.21.15.131:0/3355898471 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f42a415c9f0 0x7f42a415ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:37.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:37.076+0000 7f42a8a00640 1 --2- 172.21.15.131:0/3355898471 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f428406be30 0x7f428406e2f0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:37.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:37.076+0000 7f42a8a00640 1 --2- 172.21.15.131:0/3355898471 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f42a415f590 0x7f42a415f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:37.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:37.076+0000 7f42a8a00640 1 --2- 172.21.15.131:0/3355898471 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4288005210 0x7f42a41f3340 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:37.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:37.076+0000 7f42a8a00640 1 -- 172.21.15.131:0/3355898471 >> 172.21.15.131:0/3355898471 conn(0x7f42a40b5660 msgr2=0x7f42a4071c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:37.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:37.076+0000 7f42a8a00640 1 -- 172.21.15.131:0/3355898471 shutdown_connections 2024-05-26T22:49:37.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:37.076+0000 7f42a8a00640 1 -- 172.21.15.131:0/3355898471 wait complete. 2024-05-26T22:49:37.096 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-26T22:49:37.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:37.292+0000 7fe544a00640 1 Processor -- start 2024-05-26T22:49:37.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:37.292+0000 7fe544a00640 1 -- start start 2024-05-26T22:49:37.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:37.293+0000 7fe544a00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fe5400b3000 0x7fe5400b33e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:37.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:37.293+0000 7fe544a00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fe540072400 0x7fe54006e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:37.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:37.293+0000 7fe544a00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5400b3920 con 0x7fe54006b310 2024-05-26T22:49:37.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:37.293+0000 7fe544a00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fe5400735d0 con 0x7fe5400b3000 2024-05-26T22:49:37.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:37.293+0000 7fe544a00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fe5400729c0 con 0x7fe540072400 2024-05-26T22:49:37.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:37.293+0000 7fe53f400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fe5400b3000 0x7fe5400b33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:37.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:37.294+0000 7fe53f400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fe5400b3000 0x7fe5400b33e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53248/0 (socket says 172.21.15.131:53248) 2024-05-26T22:49:37.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:37.294+0000 7fe53f400640 1 -- 172.21.15.131:0/2946860286 learned_addr learned my addr 172.21.15.131:0/2946860286 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:37.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:37.294+0000 7fe53f400640 1 -- 172.21.15.131:0/2946860286 >> v2:172.21.15.187:3300/0 conn(0x7fe540072400 msgr2=0x7fe54006e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:37.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:37.294+0000 7fe53f400640 1 --2- 172.21.15.131:0/2946860286 >> v2:172.21.15.187:3300/0 conn(0x7fe540072400 0x7fe54006e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:37.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:37.294+0000 7fe53f400640 1 -- 172.21.15.131:0/2946860286 >> v1:172.21.15.187:6789/0 conn(0x7fe54006b310 legacy=0x7fe54006b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:37.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:37.294+0000 7fe53f400640 1 -- 172.21.15.131:0/2946860286 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe540072ca0 con 0x7fe5400b3000 2024-05-26T22:49:37.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:37.295+0000 7fe53f400640 1 --2- 172.21.15.131:0/2946860286 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe5400b3000 0x7fe5400b33e0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fe5400d15d0 tx=0x7fe53002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bde2d80bf838f1f server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:37.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:37.295+0000 7fe53ea00640 1 -- 172.21.15.131:0/2946860286 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe530037070 con 0x7fe5400b3000 2024-05-26T22:49:37.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:37.295+0000 7fe53ea00640 1 -- 172.21.15.131:0/2946860286 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe530032440 con 0x7fe5400b3000 2024-05-26T22:49:37.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:37.295+0000 7fe53ea00640 1 -- 172.21.15.131:0/2946860286 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe530032810 con 0x7fe5400b3000 2024-05-26T22:49:37.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:37.296+0000 7fe544a00640 1 -- 172.21.15.131:0/2946860286 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe5400b3000 msgr2=0x7fe5400b33e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:37.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:37.296+0000 7fe544a00640 1 --2- 172.21.15.131:0/2946860286 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe5400b3000 0x7fe5400b33e0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fe5400d15d0 tx=0x7fe53002a1a0 comp rx=0 tx=0).stop 2024-05-26T22:49:37.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:37.296+0000 7fe544a00640 1 -- 172.21.15.131:0/2946860286 shutdown_connections 2024-05-26T22:49:37.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:37.296+0000 7fe544a00640 1 --2- 172.21.15.131:0/2946860286 >> v2:172.21.15.187:3300/0 conn(0x7fe540072400 0x7fe54006e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:37.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:37.296+0000 7fe544a00640 1 --2- 172.21.15.131:0/2946860286 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe5400b3000 0x7fe5400b33e0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:37.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:37.296+0000 7fe544a00640 1 -- 172.21.15.131:0/2946860286 >> 172.21.15.131:0/2946860286 conn(0x7fe540070100 msgr2=0x7fe540070510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:37.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:37.296+0000 7fe544a00640 1 -- 172.21.15.131:0/2946860286 shutdown_connections 2024-05-26T22:49:37.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:37.296+0000 7fe544a00640 1 -- 172.21.15.131:0/2946860286 wait complete. 2024-05-26T22:49:37.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:37.298+0000 7fe544a00640 1 Processor -- start 2024-05-26T22:49:37.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:37.298+0000 7fe544a00640 1 -- start start 2024-05-26T22:49:37.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:37.299+0000 7fe544a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe54006b310 0x7fe54015d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:37.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:37.299+0000 7fe544a00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe540072400 0x7fe5400af3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:37.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:37.299+0000 7fe544a00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe5400b3000 0x7fe5400af930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:37.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:37.299+0000 7fe544a00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe54006ea70 con 0x7fe54006b310 2024-05-26T22:49:37.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:37.299+0000 7fe544a00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe540073260 con 0x7fe540072400 2024-05-26T22:49:37.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:37.299+0000 7fe544a00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe540147e40 con 0x7fe5400b3000 2024-05-26T22:49:37.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:37.299+0000 7fe53f400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe54006b310 0x7fe54015d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:37.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:37.299+0000 7fe53f400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe54006b310 0x7fe54015d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53252/0 (socket says 172.21.15.131:53252) 2024-05-26T22:49:37.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:37.299+0000 7fe53f400640 1 -- 172.21.15.131:0/1192109449 learned_addr learned my addr 172.21.15.131:0/1192109449 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:37.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:37.299+0000 7fe53f400640 1 -- 172.21.15.131:0/1192109449 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe5400b3000 msgr2=0x7fe5400af930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:37.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:37.299+0000 7fe53f400640 1 --2- 172.21.15.131:0/1192109449 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe5400b3000 0x7fe5400af930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:37.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:37.299+0000 7fe53f400640 1 -- 172.21.15.131:0/1192109449 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe540072400 msgr2=0x7fe5400af3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:37.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:37.299+0000 7fe53f400640 1 --2- 172.21.15.131:0/1192109449 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe540072400 0x7fe5400af3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:37.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:37.299+0000 7fe53f400640 1 -- 172.21.15.131:0/1192109449 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe54015f110 con 0x7fe54006b310 2024-05-26T22:49:37.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:37.300+0000 7fe53f400640 1 --2- 172.21.15.131:0/1192109449 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe54006b310 0x7fe54015d1f0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fe530000c00 tx=0x7fe5300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:37.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:37.300+0000 7fe537400640 1 -- 172.21.15.131:0/1192109449 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe530041070 con 0x7fe54006b310 2024-05-26T22:49:37.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:37.300+0000 7fe537400640 1 -- 172.21.15.131:0/1192109449 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe53003c890 con 0x7fe54006b310 2024-05-26T22:49:37.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:37.300+0000 7fe537400640 1 -- 172.21.15.131:0/1192109449 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe530037040 con 0x7fe54006b310 2024-05-26T22:49:37.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:37.300+0000 7fe544a00640 1 -- 172.21.15.131:0/1192109449 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe54015f3f0 con 0x7fe54006b310 2024-05-26T22:49:37.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:37.300+0000 7fe544a00640 1 -- 172.21.15.131:0/1192109449 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe54015f9b0 con 0x7fe54006b310 2024-05-26T22:49:37.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:37.301+0000 7fe544a00640 1 -- 172.21.15.131:0/1192109449 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5400694e0 con 0x7fe54006b310 2024-05-26T22:49:37.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:37.302+0000 7fe537400640 1 -- 172.21.15.131:0/1192109449 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe530044070 con 0x7fe54006b310 2024-05-26T22:49:37.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:37.303+0000 7fe537400640 1 --2- 172.21.15.131:0/1192109449 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fe51c06be30 0x7fe51c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:37.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:37.303+0000 7fe537400640 1 -- 172.21.15.131:0/1192109449 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fe5300abbf0 con 0x7fe54006b310 2024-05-26T22:49:37.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:37.303+0000 7fe537e00640 1 --2- 172.21.15.131:0/1192109449 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fe51c06be30 0x7fe51c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:37.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:37.304+0000 7fe537e00640 1 --2- 172.21.15.131:0/1192109449 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fe51c06be30 0x7fe51c06e2f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe5400b15e0 tx=0x7fe528006c90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:37.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:37.308+0000 7fe537400640 1 -- 172.21.15.131:0/1192109449 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fe53007a470 con 0x7fe54006b310 2024-05-26T22:49:37.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:37.530+0000 7fe544a00640 1 -- 172.21.15.131:0/1192109449 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe540147320 con 0x7fe54006b310 2024-05-26T22:49:37.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:37.530+0000 7fe537400640 1 -- 172.21.15.131:0/1192109449 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) ==== 94+0+5193 (secure 0 0 0) 0x7fe530079d70 con 0x7fe54006b310 2024-05-26T22:49:37.531 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:49:37.531 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/4292497989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":4292497989},{"type":"v1","addr":"172.21.15.187:6835","nonce":4292497989}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4678,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6836/4032294430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":4032294430},{"type":"v1","addr":"172.21.15.131:6836","nonce":4032294430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4687,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/3274403209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3274403209},{"type":"v1","addr":"172.21.15.187:6833","nonce":3274403209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-26T22:49:30.955525+0000","modified":"2024-05-26T22:49:34.033245+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5115},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5115":{"gid":5115,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.131:6837/2062704413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6835","nonce":2062704413},{"type":"v1","addr":"172.21.15.131:6837","nonce":2062704413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5115,"qdb_cluster":[5115]},"id":2}]} 2024-05-26T22:49:37.531 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 15 2024-05-26T22:49:37.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:37.533+0000 7fe544a00640 1 -- 172.21.15.131:0/1192109449 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fe51c06be30 msgr2=0x7fe51c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:37.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:37.533+0000 7fe544a00640 1 --2- 172.21.15.131:0/1192109449 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fe51c06be30 0x7fe51c06e2f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe5400b15e0 tx=0x7fe528006c90 comp rx=0 tx=0).stop 2024-05-26T22:49:37.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:37.533+0000 7fe544a00640 1 -- 172.21.15.131:0/1192109449 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe54006b310 msgr2=0x7fe54015d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:37.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:37.533+0000 7fe544a00640 1 --2- 172.21.15.131:0/1192109449 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe54006b310 0x7fe54015d1f0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fe530000c00 tx=0x7fe5300046c0 comp rx=0 tx=0).stop 2024-05-26T22:49:37.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:37.533+0000 7fe544a00640 1 -- 172.21.15.131:0/1192109449 shutdown_connections 2024-05-26T22:49:37.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:37.533+0000 7fe544a00640 1 --2- 172.21.15.131:0/1192109449 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe5400b3000 0x7fe5400af930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:37.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:37.533+0000 7fe544a00640 1 --2- 172.21.15.131:0/1192109449 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fe51c06be30 0x7fe51c06e2f0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:37.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:37.533+0000 7fe544a00640 1 --2- 172.21.15.131:0/1192109449 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe540072400 0x7fe5400af3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:37.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:37.533+0000 7fe544a00640 1 --2- 172.21.15.131:0/1192109449 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe54006b310 0x7fe54015d1f0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:37.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:37.533+0000 7fe544a00640 1 -- 172.21.15.131:0/1192109449 >> 172.21.15.131:0/1192109449 conn(0x7fe540070100 msgr2=0x7fe5400b1d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:37.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:37.533+0000 7fe544a00640 1 -- 172.21.15.131:0/1192109449 shutdown_connections 2024-05-26T22:49:37.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:37.533+0000 7fe544a00640 1 -- 172.21.15.131:0/1192109449 wait complete. 2024-05-26T22:49:37.955 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 15, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-26T22:49:30.955525+0000', 'modified': '2024-05-26T22:49:34.033245+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5115}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5115': {'gid': 5115, 'name': 'c', 'rank': 0, 'incarnation': 12, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.131:6837/2062704413', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6835', 'nonce': 2062704413}, {'type': 'v1', 'addr': '172.21.15.131:6837', 'nonce': 2062704413}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5115, 'qdb_cluster': [5115]} 2024-05-26T22:49:37.956 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-26T22:49:37.956 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-05-26T22:49:38.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.171+0000 7f302f800640 1 Processor -- start 2024-05-26T22:49:38.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.171+0000 7f302f800640 1 -- start start 2024-05-26T22:49:38.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.173+0000 7f302f800640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f302806dd30 0x7f302806e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:38.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.173+0000 7f302f800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f30280725a0 0x7f30280729a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:38.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.173+0000 7f302f800640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f302815c1c0 con 0x7f3028155770 2024-05-26T22:49:38.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.173+0000 7f302f800640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f302815c4a0 con 0x7f302806dd30 2024-05-26T22:49:38.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.173+0000 7f302f800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3028071da0 con 0x7f30280725a0 2024-05-26T22:49:38.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.173+0000 7f3027e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f302806dd30 0x7f302806e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:38.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.173+0000 7f3027e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f302806dd30 0x7f302806e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53262/0 (socket says 172.21.15.131:53262) 2024-05-26T22:49:38.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.173+0000 7f3027e00640 1 -- 172.21.15.131:0/3093412032 learned_addr learned my addr 172.21.15.131:0/3093412032 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:38.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.174+0000 7f3027e00640 1 -- 172.21.15.131:0/3093412032 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f30280725a0 msgr2=0x7f30280729a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:38.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.174+0000 7f3027e00640 1 --2- 172.21.15.131:0/3093412032 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f30280725a0 0x7f30280729a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:38.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.174+0000 7f3027e00640 1 -- 172.21.15.131:0/3093412032 >> v1:172.21.15.187:6789/0 conn(0x7f3028155770 legacy=0x7f302815b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:38.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.174+0000 7f3027e00640 1 -- 172.21.15.131:0/3093412032 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3028072080 con 0x7f302806dd30 2024-05-26T22:49:38.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.175+0000 7f3027e00640 1 --2- 172.21.15.131:0/3093412032 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f302806dd30 0x7f302806e1c0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f3018009870 tx=0x7f301802a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1027915eeb10bb6 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:38.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.175+0000 7f3027400640 1 -- 172.21.15.131:0/3093412032 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3018037070 con 0x7f302806dd30 2024-05-26T22:49:38.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.175+0000 7f3027400640 1 -- 172.21.15.131:0/3093412032 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3018032440 con 0x7f302806dd30 2024-05-26T22:49:38.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.175+0000 7f3027400640 1 -- 172.21.15.131:0/3093412032 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3018032810 con 0x7f302806dd30 2024-05-26T22:49:38.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.176+0000 7f302f800640 1 -- 172.21.15.131:0/3093412032 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f302806dd30 msgr2=0x7f302806e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:38.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.176+0000 7f302f800640 1 --2- 172.21.15.131:0/3093412032 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f302806dd30 0x7f302806e1c0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f3018009870 tx=0x7f301802a2a0 comp rx=0 tx=0).stop 2024-05-26T22:49:38.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.176+0000 7f302f800640 1 -- 172.21.15.131:0/3093412032 shutdown_connections 2024-05-26T22:49:38.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.176+0000 7f302f800640 1 --2- 172.21.15.131:0/3093412032 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f30280725a0 0x7f30280729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:38.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.176+0000 7f302f800640 1 --2- 172.21.15.131:0/3093412032 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f302806dd30 0x7f302806e1c0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:38.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.176+0000 7f302f800640 1 -- 172.21.15.131:0/3093412032 >> 172.21.15.131:0/3093412032 conn(0x7f302806bc90 msgr2=0x7f30280b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:38.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.176+0000 7f302f800640 1 -- 172.21.15.131:0/3093412032 shutdown_connections 2024-05-26T22:49:38.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.176+0000 7f302f800640 1 -- 172.21.15.131:0/3093412032 wait complete. 2024-05-26T22:49:38.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.178+0000 7f302f800640 1 Processor -- start 2024-05-26T22:49:38.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.178+0000 7f302f800640 1 -- start start 2024-05-26T22:49:38.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.179+0000 7f302f800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f302806dd30 0x7f30281f7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:38.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.179+0000 7f302f800640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f30280725a0 0x7f302815a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:38.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.179+0000 7f302f800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3028155770 0x7f302815a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:38.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.179+0000 7f302f800640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f302815be10 con 0x7f302806dd30 2024-05-26T22:49:38.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.179+0000 7f302f800640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f302815c8c0 con 0x7f30280725a0 2024-05-26T22:49:38.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.179+0000 7f302f800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f30281675b0 con 0x7f3028155770 2024-05-26T22:49:38.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.179+0000 7f302ce00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f302806dd30 0x7f30281f7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:38.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.179+0000 7f302ce00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f302806dd30 0x7f30281f7ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53272/0 (socket says 172.21.15.131:53272) 2024-05-26T22:49:38.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.179+0000 7f302ce00640 1 -- 172.21.15.131:0/3982586743 learned_addr learned my addr 172.21.15.131:0/3982586743 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:38.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.179+0000 7f3027e00640 1 --2- 172.21.15.131:0/3982586743 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f30280725a0 0x7f302815a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:38.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.179+0000 7f302d800640 1 --2- 172.21.15.131:0/3982586743 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3028155770 0x7f302815a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:38.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.179+0000 7f302ce00640 1 -- 172.21.15.131:0/3982586743 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3028155770 msgr2=0x7f302815a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:38.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.179+0000 7f302ce00640 1 --2- 172.21.15.131:0/3982586743 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3028155770 0x7f302815a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:38.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.179+0000 7f302ce00640 1 -- 172.21.15.131:0/3982586743 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f30280725a0 msgr2=0x7f302815a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:38.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.179+0000 7f302ce00640 1 --2- 172.21.15.131:0/3982586743 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f30280725a0 0x7f302815a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:38.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.179+0000 7f302ce00640 1 -- 172.21.15.131:0/3982586743 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30281f84c0 con 0x7f302806dd30 2024-05-26T22:49:38.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.179+0000 7f3027e00640 1 --2- 172.21.15.131:0/3982586743 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f30280725a0 0x7f302815a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:49:38.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.180+0000 7f302ce00640 1 --2- 172.21.15.131:0/3982586743 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f302806dd30 0x7f30281f7ee0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f301000b410 tx=0x7f301000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:38.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.180+0000 7f3024c00640 1 -- 172.21.15.131:0/3982586743 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3010015020 con 0x7f302806dd30 2024-05-26T22:49:38.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.180+0000 7f3024c00640 1 -- 172.21.15.131:0/3982586743 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3010011830 con 0x7f302806dd30 2024-05-26T22:49:38.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.180+0000 7f302f800640 1 -- 172.21.15.131:0/3982586743 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30281f87a0 con 0x7f302806dd30 2024-05-26T22:49:38.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.180+0000 7f3024c00640 1 -- 172.21.15.131:0/3982586743 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3010011c20 con 0x7f302806dd30 2024-05-26T22:49:38.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.180+0000 7f302f800640 1 -- 172.21.15.131:0/3982586743 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30281f8d60 con 0x7f302806dd30 2024-05-26T22:49:38.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.182+0000 7f3024c00640 1 -- 172.21.15.131:0/3982586743 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3010018020 con 0x7f302806dd30 2024-05-26T22:49:38.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.183+0000 7f3024c00640 1 --2- 172.21.15.131:0/3982586743 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f300406be60 0x7f300406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:38.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.183+0000 7f3024c00640 1 -- 172.21.15.131:0/3982586743 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f301008a940 con 0x7f302806dd30 2024-05-26T22:49:38.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.184+0000 7f3027e00640 1 --2- 172.21.15.131:0/3982586743 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f300406be60 0x7f300406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:38.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.184+0000 7f303261e740 1 Processor -- start 2024-05-26T22:49:38.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.184+0000 7f303261e740 1 -- start start 2024-05-26T22:49:38.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.184+0000 7f3027e00640 1 --2- 172.21.15.131:0/3982586743 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f300406be60 0x7f300406e320 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3018034000 tx=0x7f301802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:38.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.185+0000 7f303261e740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55826cfe3c70 0x55826cf96890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:38.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.185+0000 7f303261e740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55826cfc6d90 0x55826cfc6110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:38.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.185+0000 7f303261e740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55826cfc6650 0x55826cf9b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:38.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.185+0000 7f303261e740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55826cefe610 con 0x55826cfe3c70 2024-05-26T22:49:38.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.185+0000 7f303261e740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55826cea65c0 con 0x55826cfc6650 2024-05-26T22:49:38.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.185+0000 7f303261e740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x55826cefa5d0 con 0x55826cfc6d90 2024-05-26T22:49:38.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.185+0000 7f302ce00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55826cfe3c70 0x55826cf96890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:38.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.185+0000 7f302ce00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55826cfe3c70 0x55826cf96890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53280/0 (socket says 172.21.15.131:53280) 2024-05-26T22:49:38.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.185+0000 7f302ce00640 1 -- 172.21.15.131:0/4241883754 learned_addr learned my addr 172.21.15.131:0/4241883754 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:38.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.185+0000 7f302ce00640 1 -- 172.21.15.131:0/4241883754 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55826cfc6d90 msgr2=0x55826cfc6110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:38.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.185+0000 7f302ce00640 1 --2- 172.21.15.131:0/4241883754 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55826cfc6d90 0x55826cfc6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:38.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.185+0000 7f302ce00640 1 -- 172.21.15.131:0/4241883754 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55826cfc6650 msgr2=0x55826cf9b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:38.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.185+0000 7f302ce00640 1 --2- 172.21.15.131:0/4241883754 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55826cfc6650 0x55826cf9b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:38.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.185+0000 7f302ce00640 1 -- 172.21.15.131:0/4241883754 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55826cc122d0 con 0x55826cfe3c70 2024-05-26T22:49:38.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.186+0000 7f302ce00640 1 --2- 172.21.15.131:0/4241883754 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55826cfe3c70 0x55826cf96890 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f3010054fb0 tx=0x7f301005d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c726d9f072dd4db server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:38.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.186+0000 7f301d600640 1 -- 172.21.15.131:0/4241883754 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3010056e50 con 0x55826cfe3c70 2024-05-26T22:49:38.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.186+0000 7f301d600640 1 -- 172.21.15.131:0/4241883754 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3010087070 con 0x55826cfe3c70 2024-05-26T22:49:38.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.186+0000 7f301d600640 1 -- 172.21.15.131:0/4241883754 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30100551c0 con 0x55826cfe3c70 2024-05-26T22:49:38.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.186+0000 7f303261e740 1 -- 172.21.15.131:0/4241883754 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55826cfe3c70 msgr2=0x55826cf96890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:38.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.186+0000 7f303261e740 1 --2- 172.21.15.131:0/4241883754 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55826cfe3c70 0x55826cf96890 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f3010054fb0 tx=0x7f301005d040 comp rx=0 tx=0).stop 2024-05-26T22:49:38.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.187+0000 7f303261e740 1 -- 172.21.15.131:0/4241883754 shutdown_connections 2024-05-26T22:49:38.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.187+0000 7f303261e740 1 --2- 172.21.15.131:0/4241883754 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55826cfc6d90 0x55826cfc6110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:38.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.187+0000 7f303261e740 1 --2- 172.21.15.131:0/4241883754 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55826cfc6650 0x55826cf9b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:38.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.187+0000 7f303261e740 1 --2- 172.21.15.131:0/4241883754 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55826cfe3c70 0x55826cf96890 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:38.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.187+0000 7f303261e740 1 -- 172.21.15.131:0/4241883754 >> 172.21.15.131:0/4241883754 conn(0x55826cf98b00 msgr2=0x55826cfe1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:38.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.187+0000 7f303261e740 1 -- 172.21.15.131:0/4241883754 shutdown_connections 2024-05-26T22:49:38.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.187+0000 7f303261e740 1 -- 172.21.15.131:0/4241883754 wait complete. 2024-05-26T22:49:38.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.187+0000 7f303261e740 1 -- start start 2024-05-26T22:49:38.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.188+0000 7f303261e740 1 Processor -- start 2024-05-26T22:49:38.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.189+0000 7f303261e740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55826cfc6d90 0x55826cfc12f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:38.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.189+0000 7f303261e740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55826cfe3c70 0x55826cfc7280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:38.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.189+0000 7f303261e740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55826cfe2240 0x55826cfe26c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:38.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.189+0000 7f303261e740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55826cefe610 con 0x55826cfc6d90 2024-05-26T22:49:38.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.189+0000 7f303261e740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55826ceb9910 con 0x55826cfe3c70 2024-05-26T22:49:38.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.189+0000 7f303261e740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x55826cf9aa20 con 0x55826cfe2240 2024-05-26T22:49:38.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.189+0000 7f303261e740 10 client.0 _ll_register_callbacks cb 0x55826cef5410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-26T22:49:38.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.189+0000 7f302ce00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55826cfc6d90 0x55826cfc12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:38.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.189+0000 7f302ce00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55826cfc6d90 0x55826cfc12f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53284/0 (socket says 172.21.15.131:53284) 2024-05-26T22:49:38.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.189+0000 7f302ce00640 1 -- 172.21.15.131:0/2981203591 learned_addr learned my addr 172.21.15.131:0/2981203591 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:38.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.189+0000 7f302ce00640 1 -- 172.21.15.131:0/2981203591 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55826cfe2240 msgr2=0x55826cfe26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:38.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.189+0000 7f302ce00640 1 --2- 172.21.15.131:0/2981203591 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55826cfe2240 0x55826cfe26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:38.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.189+0000 7f302ce00640 1 -- 172.21.15.131:0/2981203591 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55826cfe3c70 msgr2=0x55826cfc7280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:38.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.189+0000 7f302ce00640 1 --2- 172.21.15.131:0/2981203591 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55826cfe3c70 0x55826cfc7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:38.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.189+0000 7f302ce00640 1 -- 172.21.15.131:0/2981203591 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55826ccdc6b0 con 0x55826cfc6d90 2024-05-26T22:49:38.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.189+0000 7f302ce00640 1 --2- 172.21.15.131:0/2981203591 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55826cfc6d90 0x55826cfc12f0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f3010056e00 tx=0x7f301006d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81c9df35d19536a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:38.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.189+0000 7f3016000640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-26T22:49:38.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.190+0000 7f3016000640 1 -- 172.21.15.131:0/2981203591 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3010020d80 con 0x55826cfc6d90 2024-05-26T22:49:38.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.190+0000 7f3016000640 1 -- 172.21.15.131:0/2981203591 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f30100551c0 con 0x55826cfc6d90 2024-05-26T22:49:38.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.190+0000 7f303261e740 1 -- 172.21.15.131:0/2981203591 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3010087070 con 0x55826cfc6d90 2024-05-26T22:49:38.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.190+0000 7f3016000640 1 -- 172.21.15.131:0/2981203591 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=15) ==== 24+0+0 (secure 0 0 0) 0x7f3010057fa0 con 0x55826cfc6d90 2024-05-26T22:49:38.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.190+0000 7f303261e740 10 client.5292 fetch_fsmap learned FSMap version 15 2024-05-26T22:49:38.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.190+0000 7f303261e740 1 -- 172.21.15.131:0/2981203591 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=15,monmap=2+}) -- 0x55826cfd7f00 con 0x55826cfc6d90 2024-05-26T22:49:38.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.190+0000 7f3016000640 1 -- 172.21.15.131:0/2981203591 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3010087070 con 0x55826cfc6d90 2024-05-26T22:49:38.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.190+0000 7f3016000640 1 -- 172.21.15.131:0/2981203591 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 15) ==== 3494+0+0 (secure 0 0 0) 0x7f3010058140 con 0x55826cfc6d90 2024-05-26T22:49:38.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.190+0000 7f303261e740 10 client.5292 fetch_fsmap finished waiting for FSMap version 15 2024-05-26T22:49:38.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.190+0000 7f303261e740 10 client.5292 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-26T22:49:38.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.190+0000 7f303261e740 20 client.5292 populate_metadata read hostname 'smithi131' 2024-05-26T22:49:38.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.190+0000 7f303261e740 1 --2- 172.21.15.131:0/2981203591 >> [v2:172.21.15.131:6835/2062704413,v1:172.21.15.131:6837/2062704413] conn(0x55826cfd8bb0 0x55826cfd8f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:38.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.190+0000 7f303261e740 4 client.5292 mds_command: new command op to 5115 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-26T22:49:38.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.190+0000 7f303261e740 1 -- 172.21.15.131:0/2981203591 --> [v2:172.21.15.131:6835/2062704413,v1:172.21.15.131:6837/2062704413] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55826cf0bd40 con 0x55826cfd8bb0 2024-05-26T22:49:38.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.190+0000 7f3027e00640 1 --2- 172.21.15.131:0/2981203591 >> [v2:172.21.15.131:6835/2062704413,v1:172.21.15.131:6837/2062704413] conn(0x55826cfd8bb0 0x55826cfd8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:38.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.191+0000 7f3027e00640 1 --2- 172.21.15.131:0/2981203591 >> [v2:172.21.15.131:6835/2062704413,v1:172.21.15.131:6837/2062704413] conn(0x55826cfd8bb0 0x55826cfd8f90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3b54222f34070d server_cookie=134802486bfc17e3 in_seq=0 out_seq=0 2024-05-26T22:49:38.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.191+0000 7f3016000640 10 client.5292 ms_handle_connect on v2:172.21.15.131:6835/2062704413 2024-05-26T22:49:38.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.192+0000 7f3016000640 1 -- 172.21.15.131:0/2981203591 <== mds.0 v2:172.21.15.131:6835/2062704413 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55826cf0bd40 con 0x55826cfd8bb0 2024-05-26T22:49:38.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.192+0000 7f3016000640 10 client.5292 handle_command_reply: tid=0 2024-05-26T22:49:38.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.192+0000 7f303261e740 1 client.5292 shutdown 2024-05-26T22:49:38.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.192+0000 7f303261e740 1 -- 172.21.15.131:0/2981203591 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55826cfc6d90 msgr2=0x55826cfc12f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:38.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.192+0000 7f303261e740 1 --2- 172.21.15.131:0/2981203591 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55826cfc6d90 0x55826cfc12f0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f3010056e00 tx=0x7f301006d000 comp rx=0 tx=0).stop 2024-05-26T22:49:38.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.192+0000 7f303261e740 1 -- 172.21.15.131:0/2981203591 shutdown_connections 2024-05-26T22:49:38.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.192+0000 7f303261e740 1 --2- 172.21.15.131:0/2981203591 >> [v2:172.21.15.131:6835/2062704413,v1:172.21.15.131:6837/2062704413] conn(0x55826cfd8bb0 0x55826cfd8f90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:38.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.192+0000 7f303261e740 1 --2- 172.21.15.131:0/2981203591 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55826cfe2240 0x55826cfe26c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:38.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.192+0000 7f303261e740 1 --2- 172.21.15.131:0/2981203591 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55826cfe3c70 0x55826cfc7280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:38.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.192+0000 7f3016000640 0 client.5292 ms_handle_reset on v2:172.21.15.131:6835/2062704413 2024-05-26T22:49:38.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.192+0000 7f303261e740 1 --2- 172.21.15.131:0/2981203591 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55826cfc6d90 0x55826cfc12f0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:38.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.192+0000 7f303261e740 1 -- 172.21.15.131:0/2981203591 >> 172.21.15.131:0/2981203591 conn(0x55826cfc6650 msgr2=0x55826cf972f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:38.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.193+0000 7f303261e740 1 -- 172.21.15.131:0/2981203591 shutdown_connections 2024-05-26T22:49:38.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.193+0000 7f303261e740 1 -- 172.21.15.131:0/2981203591 wait complete. 2024-05-26T22:49:38.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.193+0000 7f303261e740 20 client.5292 trim_cache size 0 max 16384 2024-05-26T22:49:38.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.212+0000 7f303261e740 1 Processor -- start 2024-05-26T22:49:38.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.212+0000 7f303261e740 1 -- start start 2024-05-26T22:49:38.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.213+0000 7f303261e740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55826cfc6650 0x55826cfc8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:38.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.213+0000 7f303261e740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55826cfc6d90 0x55826cfe0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:38.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.213+0000 7f303261e740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55826cfe2240 0x55826cfe0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:38.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.213+0000 7f303261e740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f30281ffaf0 con 0x55826cfc6650 2024-05-26T22:49:38.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.213+0000 7f303261e740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f302814ab40 con 0x55826cfc6d90 2024-05-26T22:49:38.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.213+0000 7f303261e740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f30281ff4a0 con 0x55826cfe2240 2024-05-26T22:49:38.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.213+0000 7f302ce00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55826cfc6650 0x55826cfc8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:38.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.214+0000 7f302ce00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55826cfc6650 0x55826cfc8930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53296/0 (socket says 172.21.15.131:53296) 2024-05-26T22:49:38.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.214+0000 7f302ce00640 1 -- 172.21.15.131:0/3473142077 learned_addr learned my addr 172.21.15.131:0/3473142077 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:38.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.214+0000 7f302ce00640 1 -- 172.21.15.131:0/3473142077 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55826cfe2240 msgr2=0x55826cfe0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:38.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.214+0000 7f302ce00640 1 --2- 172.21.15.131:0/3473142077 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55826cfe2240 0x55826cfe0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:38.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.214+0000 7f302ce00640 1 -- 172.21.15.131:0/3473142077 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55826cfc6d90 msgr2=0x55826cfe0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:38.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.214+0000 7f302ce00640 1 --2- 172.21.15.131:0/3473142077 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55826cfc6d90 0x55826cfe0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:38.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.214+0000 7f302ce00640 1 -- 172.21.15.131:0/3473142077 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55826cfc0d20 con 0x55826cfc6650 2024-05-26T22:49:38.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.214+0000 7f302ce00640 1 --2- 172.21.15.131:0/3473142077 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55826cfc6650 0x55826cfc8930 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f3010088f30 tx=0x7f301006f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a18e0aa4e283f3f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:38.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.214+0000 7f301cc00640 1 -- 172.21.15.131:0/3473142077 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3010058920 con 0x55826cfc6650 2024-05-26T22:49:38.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.214+0000 7f301cc00640 1 -- 172.21.15.131:0/3473142077 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3010087070 con 0x55826cfc6650 2024-05-26T22:49:38.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.214+0000 7f301cc00640 1 -- 172.21.15.131:0/3473142077 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3010054df0 con 0x55826cfc6650 2024-05-26T22:49:38.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.215+0000 7f303261e740 1 -- 172.21.15.131:0/3473142077 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55826cfc6650 msgr2=0x55826cfc8930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:38.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.215+0000 7f303261e740 1 --2- 172.21.15.131:0/3473142077 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55826cfc6650 0x55826cfc8930 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f3010088f30 tx=0x7f301006f000 comp rx=0 tx=0).stop 2024-05-26T22:49:38.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.215+0000 7f303261e740 1 -- 172.21.15.131:0/3473142077 shutdown_connections 2024-05-26T22:49:38.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.215+0000 7f303261e740 1 --2- 172.21.15.131:0/3473142077 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55826cfe2240 0x55826cfe0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:38.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.215+0000 7f303261e740 1 --2- 172.21.15.131:0/3473142077 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55826cfc6d90 0x55826cfe0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:38.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.215+0000 7f303261e740 1 --2- 172.21.15.131:0/3473142077 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55826cfc6650 0x55826cfc8930 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:38.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.215+0000 7f303261e740 1 -- 172.21.15.131:0/3473142077 >> 172.21.15.131:0/3473142077 conn(0x55826cfbe350 msgr2=0x55826cfda0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:38.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.215+0000 7f303261e740 1 -- 172.21.15.131:0/3473142077 shutdown_connections 2024-05-26T22:49:38.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.215+0000 7f303261e740 1 -- 172.21.15.131:0/3473142077 wait complete. 2024-05-26T22:49:38.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.216+0000 7f303261e740 1 -- start start 2024-05-26T22:49:38.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.216+0000 7f303261e740 1 Processor -- start 2024-05-26T22:49:38.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.217+0000 7f303261e740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55826cfc6650 0x55826cff35f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:38.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.217+0000 7f303261e740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55826cfc6d90 0x55826d0630b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:38.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.217+0000 7f303261e740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55826cfe2240 0x55826cfe0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:38.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.217+0000 7f303261e740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55826cefe610 con 0x55826cfc6d90 2024-05-26T22:49:38.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.217+0000 7f303261e740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55826ceb9910 con 0x55826cfc6650 2024-05-26T22:49:38.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.217+0000 7f303261e740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x55826cff49f0 con 0x55826cfe2240 2024-05-26T22:49:38.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.217+0000 7f303261e740 10 client.0 _ll_register_callbacks cb 0x7f3010059360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-26T22:49:38.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.217+0000 7f3027e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55826cfc6d90 0x55826d0630b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:38.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.217+0000 7f3027e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55826cfc6d90 0x55826d0630b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53310/0 (socket says 172.21.15.131:53310) 2024-05-26T22:49:38.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.217+0000 7f3027e00640 1 -- 172.21.15.131:0/2214113084 learned_addr learned my addr 172.21.15.131:0/2214113084 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:38.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.217+0000 7f3027e00640 1 -- 172.21.15.131:0/2214113084 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55826cfe2240 msgr2=0x55826cfe0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:38.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.217+0000 7f3027e00640 1 --2- 172.21.15.131:0/2214113084 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55826cfe2240 0x55826cfe0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:38.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.217+0000 7f302ce00640 1 --2- 172.21.15.131:0/2214113084 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55826cfc6650 0x55826cff35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:38.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.217+0000 7f3027e00640 1 -- 172.21.15.131:0/2214113084 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55826cfc6650 msgr2=0x55826cff35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:38.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.218+0000 7f3027e00640 1 --2- 172.21.15.131:0/2214113084 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55826cfc6650 0x55826cff35f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:38.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.218+0000 7f3027e00640 1 -- 172.21.15.131:0/2214113084 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55826cfca4e0 con 0x55826cfc6d90 2024-05-26T22:49:38.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.218+0000 7f3027e00640 1 --2- 172.21.15.131:0/2214113084 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55826cfc6d90 0x55826d0630b0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f301802a890 tx=0x7f3018008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddb3cbba173cc750 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:38.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.218+0000 7f3016000640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-26T22:49:38.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.218+0000 7f3016000640 1 -- 172.21.15.131:0/2214113084 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3018037040 con 0x55826cfc6d90 2024-05-26T22:49:38.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.218+0000 7f303261e740 1 -- 172.21.15.131:0/2214113084 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3010087070 con 0x55826cfc6d90 2024-05-26T22:49:38.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.218+0000 7f3016000640 1 -- 172.21.15.131:0/2214113084 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f301804b3c0 con 0x55826cfc6d90 2024-05-26T22:49:38.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.218+0000 7f3016000640 1 -- 172.21.15.131:0/2214113084 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=15) ==== 24+0+0 (secure 0 0 0) 0x7f3018059050 con 0x55826cfc6d90 2024-05-26T22:49:38.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.218+0000 7f303261e740 10 client.5298 fetch_fsmap learned FSMap version 15 2024-05-26T22:49:38.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.218+0000 7f303261e740 1 -- 172.21.15.131:0/2214113084 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=15,monmap=2+}) -- 0x55826d063b70 con 0x55826cfc6d90 2024-05-26T22:49:38.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.219+0000 7f3016000640 1 -- 172.21.15.131:0/2214113084 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3010087070 con 0x55826cfc6d90 2024-05-26T22:49:38.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.219+0000 7f3016000640 1 -- 172.21.15.131:0/2214113084 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 15) ==== 3494+0+0 (secure 0 0 0) 0x7f301805c020 con 0x55826cfc6d90 2024-05-26T22:49:38.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.219+0000 7f303261e740 10 client.5298 fetch_fsmap finished waiting for FSMap version 15 2024-05-26T22:49:38.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.219+0000 7f303261e740 10 client.5298 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-26T22:49:38.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.219+0000 7f303261e740 20 client.5298 populate_metadata read hostname 'smithi131' 2024-05-26T22:49:38.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.219+0000 7f303261e740 1 --2- 172.21.15.131:0/2214113084 >> [v2:172.21.15.131:6835/2062704413,v1:172.21.15.131:6837/2062704413] conn(0x55826cfe3c70 0x55826cfd6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:38.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.219+0000 7f303261e740 4 client.5298 mds_command: new command op to 5115 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-26T22:49:38.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.219+0000 7f303261e740 1 -- 172.21.15.131:0/2214113084 --> [v2:172.21.15.131:6835/2062704413,v1:172.21.15.131:6837/2062704413] -- command(tid 0: {"prefix": "status"}) -- 0x55826cfdacd0 con 0x55826cfe3c70 2024-05-26T22:49:38.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.219+0000 7f302ce00640 1 --2- 172.21.15.131:0/2214113084 >> [v2:172.21.15.131:6835/2062704413,v1:172.21.15.131:6837/2062704413] conn(0x55826cfe3c70 0x55826cfd6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:38.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.219+0000 7f302ce00640 1 --2- 172.21.15.131:0/2214113084 >> [v2:172.21.15.131:6835/2062704413,v1:172.21.15.131:6837/2062704413] conn(0x55826cfe3c70 0x55826cfd6aa0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29161a0eafb9a122 server_cookie=ca53a88e4b72c312 in_seq=0 out_seq=0 2024-05-26T22:49:38.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.219+0000 7f3016000640 10 client.5298 ms_handle_connect on v2:172.21.15.131:6835/2062704413 2024-05-26T22:49:38.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.220+0000 7f3016000640 1 -- 172.21.15.131:0/2214113084 <== mds.0 v2:172.21.15.131:6835/2062704413 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x55826cfdacd0 con 0x55826cfe3c70 2024-05-26T22:49:38.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.220+0000 7f3016000640 10 client.5298 handle_command_reply: tid=0 2024-05-26T22:49:38.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.220+0000 7f303261e740 1 client.5298 shutdown 2024-05-26T22:49:38.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.220+0000 7f303261e740 1 -- 172.21.15.131:0/2214113084 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55826cfc6d90 msgr2=0x55826d0630b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:38.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.220+0000 7f303261e740 1 --2- 172.21.15.131:0/2214113084 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55826cfc6d90 0x55826d0630b0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f301802a890 tx=0x7f3018008030 comp rx=0 tx=0).stop 2024-05-26T22:49:38.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.220+0000 7f303261e740 1 -- 172.21.15.131:0/2214113084 shutdown_connections 2024-05-26T22:49:38.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.220+0000 7f303261e740 1 --2- 172.21.15.131:0/2214113084 >> [v2:172.21.15.131:6835/2062704413,v1:172.21.15.131:6837/2062704413] conn(0x55826cfe3c70 0x55826cfd6aa0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:38.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.220+0000 7f303261e740 1 --2- 172.21.15.131:0/2214113084 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55826cfe2240 0x55826cfe0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:38.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.220+0000 7f3016000640 0 client.5298 ms_handle_reset on v2:172.21.15.131:6835/2062704413 2024-05-26T22:49:38.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.220+0000 7f303261e740 1 --2- 172.21.15.131:0/2214113084 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55826cfc6d90 0x55826d0630b0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:38.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.220+0000 7f303261e740 1 --2- 172.21.15.131:0/2214113084 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55826cfc6650 0x55826cff35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:38.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.220+0000 7f303261e740 1 -- 172.21.15.131:0/2214113084 >> 172.21.15.131:0/2214113084 conn(0x55826cfbe350 msgr2=0x55826cf97580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:38.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.220+0000 7f303261e740 1 -- 172.21.15.131:0/2214113084 shutdown_connections 2024-05-26T22:49:38.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.220+0000 7f303261e740 1 -- 172.21.15.131:0/2214113084 wait complete. 2024-05-26T22:49:38.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.220+0000 7f303261e740 20 client.5298 trim_cache size 0 max 16384 2024-05-26T22:49:38.226 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-26T22:49:38.226 INFO:teuthology.orchestra.run.smithi131.stdout: "cluster_fsid": "46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f", 2024-05-26T22:49:38.226 INFO:teuthology.orchestra.run.smithi131.stdout: "whoami": 0, 2024-05-26T22:49:38.226 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 5115, 2024-05-26T22:49:38.226 INFO:teuthology.orchestra.run.smithi131.stdout: "want_state": "up:active", 2024-05-26T22:49:38.226 INFO:teuthology.orchestra.run.smithi131.stdout: "state": "up:active", 2024-05-26T22:49:38.226 INFO:teuthology.orchestra.run.smithi131.stdout: "fs_name": "cephfs", 2024-05-26T22:49:38.226 INFO:teuthology.orchestra.run.smithi131.stdout: "rank_uptime": 6.6834639549999997, 2024-05-26T22:49:38.226 INFO:teuthology.orchestra.run.smithi131.stdout: "mdsmap_epoch": 15, 2024-05-26T22:49:38.226 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch": 33, 2024-05-26T22:49:38.226 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch_barrier": 33, 2024-05-26T22:49:38.226 INFO:teuthology.orchestra.run.smithi131.stdout: "uptime": 15.535901911 2024-05-26T22:49:38.227 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-26T22:49:38.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.221+0000 7f302f800640 1 -- 172.21.15.131:0/3982586743 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f300406be60 msgr2=0x7f300406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:38.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.221+0000 7f302f800640 1 --2- 172.21.15.131:0/3982586743 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f300406be60 0x7f300406e320 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3018034000 tx=0x7f301802e000 comp rx=0 tx=0).stop 2024-05-26T22:49:38.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.221+0000 7f302f800640 1 -- 172.21.15.131:0/3982586743 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f302806dd30 msgr2=0x7f30281f7ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:38.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.221+0000 7f302f800640 1 --2- 172.21.15.131:0/3982586743 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f302806dd30 0x7f30281f7ee0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f301000b410 tx=0x7f301000b8d0 comp rx=0 tx=0).stop 2024-05-26T22:49:38.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.221+0000 7f302f800640 1 -- 172.21.15.131:0/3982586743 shutdown_connections 2024-05-26T22:49:38.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.221+0000 7f302f800640 1 --2- 172.21.15.131:0/3982586743 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3028155770 0x7f302815a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:38.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.221+0000 7f302f800640 1 --2- 172.21.15.131:0/3982586743 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f300406be60 0x7f300406e320 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:38.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.221+0000 7f302f800640 1 --2- 172.21.15.131:0/3982586743 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f30280725a0 0x7f302815a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:38.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.221+0000 7f302f800640 1 --2- 172.21.15.131:0/3982586743 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f302806dd30 0x7f30281f7ee0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:38.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.221+0000 7f302f800640 1 -- 172.21.15.131:0/3982586743 >> 172.21.15.131:0/3982586743 conn(0x7f302806bc90 msgr2=0x7f302814b300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:38.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.221+0000 7f302f800640 1 -- 172.21.15.131:0/3982586743 shutdown_connections 2024-05-26T22:49:38.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.221+0000 7f302f800640 1 -- 172.21.15.131:0/3982586743 wait complete. 2024-05-26T22:49:38.237 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-26T22:49:38.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.433+0000 7ffbabc00640 1 Processor -- start 2024-05-26T22:49:38.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.433+0000 7ffbabc00640 1 -- start start 2024-05-26T22:49:38.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.435+0000 7ffbabc00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffba4155840 0x7ffba4071930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:38.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.435+0000 7ffbabc00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7ffba415c3d0 0x7ffba406ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:38.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.435+0000 7ffbabc00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffba40722a0 con 0x7ffba406f490 2024-05-26T22:49:38.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.435+0000 7ffbabc00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7ffba4072580 con 0x7ffba415c3d0 2024-05-26T22:49:38.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.435+0000 7ffbabc00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ffba4072860 con 0x7ffba4155840 2024-05-26T22:49:38.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.435+0000 7ffba3e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7ffba415c3d0 0x7ffba406ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:38.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.435+0000 7ffba3e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7ffba415c3d0 0x7ffba406ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53322/0 (socket says 172.21.15.131:53322) 2024-05-26T22:49:38.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.435+0000 7ffba3e00640 1 -- 172.21.15.131:0/4017266253 learned_addr learned my addr 172.21.15.131:0/4017266253 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:38.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.435+0000 7ffba9200640 1 --2- 172.21.15.131:0/4017266253 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffba4155840 0x7ffba4071930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:38.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.436+0000 7ffba3e00640 1 -- 172.21.15.131:0/4017266253 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffba4155840 msgr2=0x7ffba4071930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:38.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.436+0000 7ffba3e00640 1 --2- 172.21.15.131:0/4017266253 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffba4155840 0x7ffba4071930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:38.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.436+0000 7ffba3e00640 1 -- 172.21.15.131:0/4017266253 >> v1:172.21.15.187:6789/0 conn(0x7ffba406f490 legacy=0x7ffba415ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:38.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.436+0000 7ffba3e00640 1 -- 172.21.15.131:0/4017266253 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffba4072b40 con 0x7ffba415c3d0 2024-05-26T22:49:38.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.437+0000 7ffba3e00640 1 --2- 172.21.15.131:0/4017266253 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffba415c3d0 0x7ffba406ef50 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7ffb90009870 tx=0x7ffb9002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3789ab8380909ac7 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:38.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.437+0000 7ffba3400640 1 -- 172.21.15.131:0/4017266253 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb90037070 con 0x7ffba415c3d0 2024-05-26T22:49:38.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.437+0000 7ffba3400640 1 -- 172.21.15.131:0/4017266253 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb90032440 con 0x7ffba415c3d0 2024-05-26T22:49:38.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.437+0000 7ffba3400640 1 -- 172.21.15.131:0/4017266253 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb90032810 con 0x7ffba415c3d0 2024-05-26T22:49:38.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.438+0000 7ffbabc00640 1 -- 172.21.15.131:0/4017266253 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffba415c3d0 msgr2=0x7ffba406ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:38.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.438+0000 7ffbabc00640 1 --2- 172.21.15.131:0/4017266253 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffba415c3d0 0x7ffba406ef50 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7ffb90009870 tx=0x7ffb9002a210 comp rx=0 tx=0).stop 2024-05-26T22:49:38.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.438+0000 7ffbabc00640 1 -- 172.21.15.131:0/4017266253 shutdown_connections 2024-05-26T22:49:38.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.438+0000 7ffbabc00640 1 --2- 172.21.15.131:0/4017266253 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffba415c3d0 0x7ffba406ef50 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:38.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.438+0000 7ffbabc00640 1 --2- 172.21.15.131:0/4017266253 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffba4155840 0x7ffba4071930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:38.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.438+0000 7ffbabc00640 1 -- 172.21.15.131:0/4017266253 >> 172.21.15.131:0/4017266253 conn(0x7ffba40b1e90 msgr2=0x7ffba40b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:38.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.438+0000 7ffbabc00640 1 -- 172.21.15.131:0/4017266253 shutdown_connections 2024-05-26T22:49:38.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.438+0000 7ffbabc00640 1 -- 172.21.15.131:0/4017266253 wait complete. 2024-05-26T22:49:38.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.440+0000 7ffbabc00640 1 Processor -- start 2024-05-26T22:49:38.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.440+0000 7ffbabc00640 1 -- start start 2024-05-26T22:49:38.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.441+0000 7ffbabc00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffba406f490 0x7ffba4146290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:38.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.441+0000 7ffbabc00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffba4155840 0x7ffba40b2d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:38.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.441+0000 7ffbabc00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffba415c3d0 0x7ffba40b37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:38.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.441+0000 7ffbabc00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ffba4071ff0 con 0x7ffba406f490 2024-05-26T22:49:38.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.441+0000 7ffbabc00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ffba4071e70 con 0x7ffba415c3d0 2024-05-26T22:49:38.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.441+0000 7ffbabc00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ffba40aee50 con 0x7ffba4155840 2024-05-26T22:49:38.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.441+0000 7ffba9200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffba406f490 0x7ffba4146290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:38.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.441+0000 7ffba9200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffba406f490 0x7ffba4146290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53338/0 (socket says 172.21.15.131:53338) 2024-05-26T22:49:38.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.441+0000 7ffba9200640 1 -- 172.21.15.131:0/2950717710 learned_addr learned my addr 172.21.15.131:0/2950717710 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:38.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.441+0000 7ffba9c00640 1 --2- 172.21.15.131:0/2950717710 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffba415c3d0 0x7ffba40b37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:38.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.441+0000 7ffba3e00640 1 --2- 172.21.15.131:0/2950717710 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffba4155840 0x7ffba40b2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:38.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.441+0000 7ffba9200640 1 -- 172.21.15.131:0/2950717710 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffba4155840 msgr2=0x7ffba40b2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:38.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.441+0000 7ffba9200640 1 --2- 172.21.15.131:0/2950717710 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffba4155840 0x7ffba40b2d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:38.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.441+0000 7ffba9200640 1 -- 172.21.15.131:0/2950717710 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffba415c3d0 msgr2=0x7ffba40b37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:38.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.441+0000 7ffba9200640 1 --2- 172.21.15.131:0/2950717710 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffba415c3d0 0x7ffba40b37a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:38.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.441+0000 7ffba9200640 1 -- 172.21.15.131:0/2950717710 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffba4147720 con 0x7ffba406f490 2024-05-26T22:49:38.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.442+0000 7ffba9200640 1 --2- 172.21.15.131:0/2950717710 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffba406f490 0x7ffba4146290 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7ffb9800d830 tx=0x7ffb9800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:38.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.442+0000 7ffba0c00640 1 -- 172.21.15.131:0/2950717710 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb9800b6c0 con 0x7ffba406f490 2024-05-26T22:49:38.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.442+0000 7ffba0c00640 1 -- 172.21.15.131:0/2950717710 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb98019070 con 0x7ffba406f490 2024-05-26T22:49:38.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.442+0000 7ffba0c00640 1 -- 172.21.15.131:0/2950717710 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb9800ba80 con 0x7ffba406f490 2024-05-26T22:49:38.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.442+0000 7ffbabc00640 1 -- 172.21.15.131:0/2950717710 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffba4147a00 con 0x7ffba406f490 2024-05-26T22:49:38.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.442+0000 7ffbabc00640 1 -- 172.21.15.131:0/2950717710 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffba4153240 con 0x7ffba406f490 2024-05-26T22:49:38.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.443+0000 7ffbabc00640 1 -- 172.21.15.131:0/2950717710 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffba40694e0 con 0x7ffba406f490 2024-05-26T22:49:38.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.444+0000 7ffba0c00640 1 -- 172.21.15.131:0/2950717710 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ffb9801e460 con 0x7ffba406f490 2024-05-26T22:49:38.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.445+0000 7ffba0c00640 1 --2- 172.21.15.131:0/2950717710 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ffb8406be30 0x7ffb8406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:38.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.445+0000 7ffba0c00640 1 -- 172.21.15.131:0/2950717710 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7ffb9808b5a0 con 0x7ffba406f490 2024-05-26T22:49:38.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.445+0000 7ffba3e00640 1 --2- 172.21.15.131:0/2950717710 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ffb8406be30 0x7ffb8406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:38.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.446+0000 7ffba3e00640 1 --2- 172.21.15.131:0/2950717710 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ffb8406be30 0x7ffb8406e2f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ffb90034000 tx=0x7ffb9002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:38.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.450+0000 7ffba0c00640 1 -- 172.21.15.131:0/2950717710 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7ffb98059f40 con 0x7ffba406f490 2024-05-26T22:49:38.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.694+0000 7ffbabc00640 1 -- 172.21.15.131:0/2950717710 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ffba406f170 con 0x7ffba406f490 2024-05-26T22:49:38.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.695+0000 7ffba0c00640 1 -- 172.21.15.131:0/2950717710 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v15) ==== 53+0+73 (secure 0 0 0) 0x7ffb98059840 con 0x7ffba406f490 2024-05-26T22:49:38.695 INFO:teuthology.orchestra.run.smithi131.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-26T22:49:38.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.697+0000 7ffbabc00640 1 -- 172.21.15.131:0/2950717710 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ffb8406be30 msgr2=0x7ffb8406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:38.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.697+0000 7ffbabc00640 1 --2- 172.21.15.131:0/2950717710 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ffb8406be30 0x7ffb8406e2f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ffb90034000 tx=0x7ffb9002e000 comp rx=0 tx=0).stop 2024-05-26T22:49:38.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.697+0000 7ffbabc00640 1 -- 172.21.15.131:0/2950717710 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffba406f490 msgr2=0x7ffba4146290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:38.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.697+0000 7ffbabc00640 1 --2- 172.21.15.131:0/2950717710 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffba406f490 0x7ffba4146290 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7ffb9800d830 tx=0x7ffb9800dcf0 comp rx=0 tx=0).stop 2024-05-26T22:49:38.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.698+0000 7ffbabc00640 1 -- 172.21.15.131:0/2950717710 shutdown_connections 2024-05-26T22:49:38.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.698+0000 7ffbabc00640 1 --2- 172.21.15.131:0/2950717710 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffba4155840 0x7ffba40b2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:38.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.698+0000 7ffbabc00640 1 --2- 172.21.15.131:0/2950717710 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ffb8406be30 0x7ffb8406e2f0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:38.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.698+0000 7ffbabc00640 1 --2- 172.21.15.131:0/2950717710 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffba415c3d0 0x7ffba40b37a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:38.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.698+0000 7ffbabc00640 1 --2- 172.21.15.131:0/2950717710 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffba406f490 0x7ffba4146290 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:38.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.698+0000 7ffbabc00640 1 -- 172.21.15.131:0/2950717710 >> 172.21.15.131:0/2950717710 conn(0x7ffba40b1e90 msgr2=0x7ffba406e820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:38.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.698+0000 7ffbabc00640 1 -- 172.21.15.131:0/2950717710 shutdown_connections 2024-05-26T22:49:38.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:38.698+0000 7ffbabc00640 1 -- 172.21.15.131:0/2950717710 wait complete. 2024-05-26T22:49:38.715 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-26T22:49:38.716 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-26T22:49:38.716 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi131.front.sepia.ceph.com 2024-05-26T22:49:38.716 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-26T22:49:38.716 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-26T22:49:38.716 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-26T22:49:38.716 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-26T22:49:38.716 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-26T22:49:38.716 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-26T22:49:38.716 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:49:38.716 DEBUG:teuthology.orchestra.run.smithi131:> ip addr 2024-05-26T22:49:38.765 INFO:teuthology.orchestra.run.smithi131.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-26T22:49:38.766 INFO:teuthology.orchestra.run.smithi131.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-26T22:49:38.766 INFO:teuthology.orchestra.run.smithi131.stdout: inet 127.0.0.1/8 scope host lo 2024-05-26T22:49:38.766 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-05-26T22:49:38.766 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 ::1/128 scope host 2024-05-26T22:49:38.766 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-05-26T22:49:38.766 INFO:teuthology.orchestra.run.smithi131.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-26T22:49:38.766 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:d8:fb:18 brd ff:ff:ff:ff:ff:ff 2024-05-26T22:49:38.766 INFO:teuthology.orchestra.run.smithi131.stdout: altname enp7s0f0 2024-05-26T22:49:38.767 INFO:teuthology.orchestra.run.smithi131.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-26T22:49:38.767 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:d8:fb:19 brd ff:ff:ff:ff:ff:ff 2024-05-26T22:49:38.767 INFO:teuthology.orchestra.run.smithi131.stdout: altname enp7s0f1 2024-05-26T22:49:38.767 INFO:teuthology.orchestra.run.smithi131.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-26T22:49:38.767 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:88:72:f8 brd ff:ff:ff:ff:ff:ff 2024-05-26T22:49:38.767 INFO:teuthology.orchestra.run.smithi131.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-26T22:49:38.767 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:88:72:f9 brd ff:ff:ff:ff:ff:ff 2024-05-26T22:49:38.767 INFO:teuthology.orchestra.run.smithi131.stdout: inet 172.21.15.131/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-26T22:49:38.767 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft 42518sec preferred_lft 42518sec 2024-05-26T22:49:38.767 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::ec4:7aff:fe88:72f9/64 scope link 2024-05-26T22:49:38.767 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-05-26T22:49:38.768 INFO:teuthology.orchestra.run.smithi131.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-26T22:49:38.768 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 06:f4:05:86:a4:3e brd ff:ff:ff:ff:ff:ff 2024-05-26T22:49:38.768 INFO:teuthology.orchestra.run.smithi131.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-26T22:49:38.768 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-05-26T22:49:38.768 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::8802:8fff:fe41:fc6d/64 scope link 2024-05-26T22:49:38.768 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-05-26T22:49:38.768 INFO:teuthology.orchestra.run.smithi131.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-26T22:49:38.768 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 06:f4:05:86:a4:3e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-26T22:49:38.768 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::4f4:5ff:fe86:a43e/64 scope link 2024-05-26T22:49:38.768 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-05-26T22:49:38.768 INFO:teuthology.orchestra.run.smithi131.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-26T22:49:38.769 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether ee:6f:35:b2:c0:5b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-26T22:49:38.769 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::ec6f:35ff:feb2:c05b/64 scope link 2024-05-26T22:49:38.769 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-05-26T22:49:38.769 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:49:38.769 DEBUG:teuthology.orchestra.run.smithi131:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-26T22:49:38.849 INFO:teuthology.orchestra.run.smithi131.stdout:1 2024-05-26T22:49:38.853 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:49:38.853 DEBUG:teuthology.orchestra.run.smithi131:> route 2024-05-26T22:49:38.904 INFO:teuthology.orchestra.run.smithi131.stdout:Kernel IP routing table 2024-05-26T22:49:38.904 INFO:teuthology.orchestra.run.smithi131.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-26T22:49:38.904 INFO:teuthology.orchestra.run.smithi131.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-26T22:49:38.905 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-26T22:49:38.905 INFO:teuthology.orchestra.run.smithi131.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-26T22:49:38.905 DEBUG:teuthology.orchestra.run.smithi131:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-26T22:49:38.905 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2024-05-26T22:49:38.905 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-26T22:49:38.905 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-26T22:49:38.905 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-26T22:49:38.905 DEBUG:teuthology.orchestra.run.smithi131:> ') 2024-05-26T22:49:39.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:39 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-26T22:49:39.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-26T22:49:39.115 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:49:39.115 DEBUG:teuthology.orchestra.run.smithi131:> ip netns list 2024-05-26T22:49:39.163 INFO:teuthology.orchestra.run.smithi131.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-26T22:49:39.164 INFO:teuthology.orchestra.run.smithi131.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-26T22:49:39.164 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-26T22:49:39.164 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:49:39.164 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link set brx.0 up 2024-05-26T22:49:39.250 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-26T22:49:39.251 DEBUG:teuthology.orchestra.run.smithi131:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-26T22:49:39.304 INFO:teuthology.orchestra.run.smithi131.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-26T22:49:39.304 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-26T22:49:39.305 DEBUG:teuthology.orchestra.run.smithi131:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-26T22:49:39.355 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-05-26T22:49:39.355 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:49:39.355 DEBUG:teuthology.orchestra.run.smithi131:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-26T22:49:39.488 INFO:teuthology.orchestra.run.smithi131.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-26T22:49:39.488 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-26T22:49:39.521 INFO:teuthology.orchestra.run.smithi131.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-26T22:49:39.521 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-26T22:49:39.521 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: resolved to: "172.21.15.131:3300,172.21.15.187:3300,172.21.15.187:3301" 2024-05-26T22:49:39.521 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: trying mount with new device syntax: 0@46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f.cephfs=/ 2024-05-26T22:49:39.521 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.131:3300/172.21.15.187:3300/172.21.15.187:3301" will pass to kernel 2024-05-26T22:49:39.523 INFO:teuthology.orchestra.run.smithi131.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-26T22:49:39.523 INFO:teuthology.orchestra.run.smithi131.stdout: You just mounted a file system that supports labels which does not 2024-05-26T22:49:39.523 INFO:teuthology.orchestra.run.smithi131.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-26T22:49:39.523 INFO:teuthology.orchestra.run.smithi131.stdout: applications will generate AVC messages and not be allowed access to 2024-05-26T22:49:39.523 INFO:teuthology.orchestra.run.smithi131.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-26T22:49:39.527 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-26T22:49:39.527 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:49:39.528 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-26T22:49:39.604 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-26T22:49:39.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:39 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-26T22:49:39.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-26T22:49:39.735 INFO:teuthology.orchestra.run.smithi131.stdout:client5307 2024-05-26T22:49:39.736 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-26T22:49:39.736 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-26T22:49:39.787 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_export_ephemeral_random True 2024-05-26T22:49:40.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.040+0000 7f37d1800640 1 Processor -- start 2024-05-26T22:49:40.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.040+0000 7f37d1800640 1 -- start start 2024-05-26T22:49:40.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.042+0000 7f37d1800640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f37cc0b4320 0x7f37cc0d09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:40.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.042+0000 7f37d1800640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f37cc15d130 0x7f37cc0b5400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:40.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.042+0000 7f37d1800640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37cc0d10a0 con 0x7f37cc145a60 2024-05-26T22:49:40.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.042+0000 7f37d1800640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f37cc0d1380 con 0x7f37cc15d130 2024-05-26T22:49:40.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.042+0000 7f37d1800640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f37cc0d1660 con 0x7f37cc0b4320 2024-05-26T22:49:40.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.042+0000 7f37cbe00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f37cc145a60 0x7f37cc0b3c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:48428/0 (socket says 172.21.15.131:48428) 2024-05-26T22:49:40.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.042+0000 7f37cbe00640 1 -- 172.21.15.131:0/2803154550 learned_addr learned my addr 172.21.15.131:0/2803154550 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:49:40.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.042+0000 7f37d0e00640 1 --2- 172.21.15.131:0/2803154550 >> v2:172.21.15.131:3300/0 conn(0x7f37cc15d130 0x7f37cc0b5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:40.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.043+0000 7f37cb400640 1 --2- 172.21.15.131:0/2803154550 >> v2:172.21.15.187:3300/0 conn(0x7f37cc0b4320 0x7f37cc0d09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:40.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.043+0000 7f37caa00640 1 -- 172.21.15.131:0/2803154550 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3660801103 0 0) 0x7f37cc0d10a0 con 0x7f37cc145a60 2024-05-26T22:49:40.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.043+0000 7f37caa00640 1 -- 172.21.15.131:0/2803154550 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f37b0003520 con 0x7f37cc145a60 2024-05-26T22:49:40.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.043+0000 7f37d0e00640 1 -- 172.21.15.131:0/2803154550 >> v2:172.21.15.187:3300/0 conn(0x7f37cc0b4320 msgr2=0x7f37cc0d09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:40.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.043+0000 7f37d0e00640 1 --2- 172.21.15.131:0/2803154550 >> v2:172.21.15.187:3300/0 conn(0x7f37cc0b4320 0x7f37cc0d09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:40.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.043+0000 7f37d0e00640 1 -- 172.21.15.131:0/2803154550 >> v1:172.21.15.131:6789/0 conn(0x7f37cc145a60 legacy=0x7f37cc0b3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:40.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.043+0000 7f37d0e00640 1 -- 172.21.15.131:0/2803154550 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37cc0d1940 con 0x7f37cc15d130 2024-05-26T22:49:40.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.043+0000 7f37cb400640 1 --2- 172.21.15.131:0/2803154550 >> v2:172.21.15.187:3300/0 conn(0x7f37cc0b4320 0x7f37cc0d09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-26T22:49:40.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.044+0000 7f37d0e00640 1 --2- 172.21.15.131:0/2803154550 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f37cc15d130 0x7f37cc0b5400 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f37cc0d3140 tx=0x7f37c402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21763fbbaecc0452 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:40.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.044+0000 7f37caa00640 1 -- 172.21.15.131:0/2803154550 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37c4037070 con 0x7f37cc15d130 2024-05-26T22:49:40.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.044+0000 7f37caa00640 1 -- 172.21.15.131:0/2803154550 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37c4032440 con 0x7f37cc15d130 2024-05-26T22:49:40.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.044+0000 7f37caa00640 1 -- 172.21.15.131:0/2803154550 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37c402ae60 con 0x7f37cc15d130 2024-05-26T22:49:40.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.045+0000 7f37d1800640 1 -- 172.21.15.131:0/2803154550 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f37cc15d130 msgr2=0x7f37cc0b5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:40.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.045+0000 7f37d1800640 1 --2- 172.21.15.131:0/2803154550 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f37cc15d130 0x7f37cc0b5400 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f37cc0d3140 tx=0x7f37c402a110 comp rx=0 tx=0).stop 2024-05-26T22:49:40.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.045+0000 7f37d1800640 1 -- 172.21.15.131:0/2803154550 shutdown_connections 2024-05-26T22:49:40.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.045+0000 7f37d1800640 1 --2- 172.21.15.131:0/2803154550 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f37cc15d130 0x7f37cc0b5400 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:40.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.045+0000 7f37d1800640 1 --2- 172.21.15.131:0/2803154550 >> v2:172.21.15.187:3300/0 conn(0x7f37cc0b4320 0x7f37cc0d09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:40.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.045+0000 7f37d1800640 1 -- 172.21.15.131:0/2803154550 >> 172.21.15.131:0/2803154550 conn(0x7f37cc067100 msgr2=0x7f37cc0674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:40.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.045+0000 7f37d1800640 1 -- 172.21.15.131:0/2803154550 shutdown_connections 2024-05-26T22:49:40.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.045+0000 7f37d1800640 1 -- 172.21.15.131:0/2803154550 wait complete. 2024-05-26T22:49:40.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.047+0000 7f37d1800640 1 Processor -- start 2024-05-26T22:49:40.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.047+0000 7f37d1800640 1 -- start start 2024-05-26T22:49:40.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.048+0000 7f37d1800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f37cc0b4320 0x7f37cc06df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:40.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.048+0000 7f37d1800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f37cc145a60 0x7f37cc153700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:40.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.048+0000 7f37d1800640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f37cc15d130 0x7f37cc1f38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:40.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.048+0000 7f37d1800640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f37cc145e40 con 0x7f37cc145a60 2024-05-26T22:49:40.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.048+0000 7f37d1800640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f37cc15d5c0 con 0x7f37cc15d130 2024-05-26T22:49:40.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.048+0000 7f37d1800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f37cc145fc0 con 0x7f37cc0b4320 2024-05-26T22:49:40.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.048+0000 7f37cb400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f37cc145a60 0x7f37cc153700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:40.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.048+0000 7f37cb400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f37cc145a60 0x7f37cc153700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53360/0 (socket says 172.21.15.131:53360) 2024-05-26T22:49:40.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.048+0000 7f37cb400640 1 -- 172.21.15.131:0/3491778612 learned_addr learned my addr 172.21.15.131:0/3491778612 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:40.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.049+0000 7f37cb400640 1 -- 172.21.15.131:0/3491778612 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f37cc0b4320 msgr2=0x7f37cc06df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:40.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.049+0000 7f37d0e00640 1 --2- 172.21.15.131:0/3491778612 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f37cc15d130 0x7f37cc1f38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:40.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.049+0000 7f37cb400640 1 --2- 172.21.15.131:0/3491778612 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f37cc0b4320 0x7f37cc06df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:40.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.049+0000 7f37cb400640 1 -- 172.21.15.131:0/3491778612 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f37cc15d130 msgr2=0x7f37cc1f38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:40.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.049+0000 7f37cb400640 1 --2- 172.21.15.131:0/3491778612 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f37cc15d130 0x7f37cc1f38e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:40.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.049+0000 7f37cb400640 1 -- 172.21.15.131:0/3491778612 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37cc153c40 con 0x7f37cc145a60 2024-05-26T22:49:40.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.049+0000 7f37cb400640 1 --2- 172.21.15.131:0/3491778612 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f37cc145a60 0x7f37cc153700 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f37b8009fa0 tx=0x7f37b800d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:40.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.049+0000 7f37c3e00640 1 -- 172.21.15.131:0/3491778612 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b801a070 con 0x7f37cc145a60 2024-05-26T22:49:40.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.049+0000 7f37c3e00640 1 -- 172.21.15.131:0/3491778612 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37b8003bb0 con 0x7f37cc145a60 2024-05-26T22:49:40.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.049+0000 7f37c3e00640 1 -- 172.21.15.131:0/3491778612 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b8004030 con 0x7f37cc145a60 2024-05-26T22:49:40.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.050+0000 7f37d1800640 1 -- 172.21.15.131:0/3491778612 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37cc1f3e20 con 0x7f37cc145a60 2024-05-26T22:49:40.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.050+0000 7f37d1800640 1 -- 172.21.15.131:0/3491778612 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37cc06cd80 con 0x7f37cc145a60 2024-05-26T22:49:40.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.051+0000 7f37d1800640 1 -- 172.21.15.131:0/3491778612 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37cc0694e0 con 0x7f37cc145a60 2024-05-26T22:49:40.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.052+0000 7f37c3e00640 1 -- 172.21.15.131:0/3491778612 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f37b801d070 con 0x7f37cc145a60 2024-05-26T22:49:40.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.053+0000 7f37c3e00640 1 --2- 172.21.15.131:0/3491778612 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f37ac06be30 0x7f37ac06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:40.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.053+0000 7f37c3e00640 1 -- 172.21.15.131:0/3491778612 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f37b808b060 con 0x7f37cc145a60 2024-05-26T22:49:40.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.053+0000 7f37cbe00640 1 --2- 172.21.15.131:0/3491778612 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f37ac06be30 0x7f37ac06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:40.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.053+0000 7f37cbe00640 1 --2- 172.21.15.131:0/3491778612 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f37ac06be30 0x7f37ac06e2f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f37bc000f60 tx=0x7f37bc0078f0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:40.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.057+0000 7f37c3e00640 1 -- 172.21.15.131:0/3491778612 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f37b8010070 con 0x7f37cc145a60 2024-05-26T22:49:40.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.258+0000 7f37d1800640 1 -- 172.21.15.131:0/3491778612 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command([{prefix=config set, name=mds_export_ephemeral_random}] v 0) -- 0x7f37cc0cf4a0 con 0x7f37cc145a60 2024-05-26T22:49:40.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.265+0000 7f37c3e00640 1 -- 172.21.15.131:0/3491778612 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_export_ephemeral_random}]=0 v11)=0 v11) ==== 128+0+0 (secure 0 0 0) 0x7f37b8059320 con 0x7f37cc145a60 2024-05-26T22:49:40.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.268+0000 7f37d1800640 1 -- 172.21.15.131:0/3491778612 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f37ac06be30 msgr2=0x7f37ac06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:40.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.268+0000 7f37d1800640 1 --2- 172.21.15.131:0/3491778612 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f37ac06be30 0x7f37ac06e2f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f37bc000f60 tx=0x7f37bc0078f0 comp rx=0 tx=0).stop 2024-05-26T22:49:40.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.268+0000 7f37d1800640 1 -- 172.21.15.131:0/3491778612 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f37cc145a60 msgr2=0x7f37cc153700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:40.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.268+0000 7f37d1800640 1 --2- 172.21.15.131:0/3491778612 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f37cc145a60 0x7f37cc153700 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f37b8009fa0 tx=0x7f37b800d560 comp rx=0 tx=0).stop 2024-05-26T22:49:40.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.268+0000 7f37d1800640 1 -- 172.21.15.131:0/3491778612 shutdown_connections 2024-05-26T22:49:40.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.268+0000 7f37d1800640 1 --2- 172.21.15.131:0/3491778612 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f37ac06be30 0x7f37ac06e2f0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:40.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.268+0000 7f37d1800640 1 --2- 172.21.15.131:0/3491778612 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f37cc15d130 0x7f37cc1f38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:40.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.268+0000 7f37d1800640 1 --2- 172.21.15.131:0/3491778612 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f37cc145a60 0x7f37cc153700 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:40.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.268+0000 7f37d1800640 1 --2- 172.21.15.131:0/3491778612 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f37cc0b4320 0x7f37cc06df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:40.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.268+0000 7f37d1800640 1 -- 172.21.15.131:0/3491778612 >> 172.21.15.131:0/3491778612 conn(0x7f37cc067100 msgr2=0x7f37cc0d2890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:40.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.269+0000 7f37d1800640 1 -- 172.21.15.131:0/3491778612 shutdown_connections 2024-05-26T22:49:40.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.269+0000 7f37d1800640 1 -- 172.21.15.131:0/3491778612 wait complete. 2024-05-26T22:49:40.289 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_export_ephemeral_distributed True 2024-05-26T22:49:40.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.497+0000 7fb680600640 1 Processor -- start 2024-05-26T22:49:40.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.497+0000 7fb680600640 1 -- start start 2024-05-26T22:49:40.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.499+0000 7fb680600640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fb6780b5450 0x7fb6780b43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:40.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.499+0000 7fb680600640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb67806f420 0x7fb6780cfd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:40.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.499+0000 7fb680600640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6780b4900 con 0x7fb6780cf590 2024-05-26T22:49:40.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.499+0000 7fb680600640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fb678160d90 con 0x7fb6780b5450 2024-05-26T22:49:40.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.499+0000 7fb680600640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb678160180 con 0x7fb67806f420 2024-05-26T22:49:40.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.499+0000 7fb67dc00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fb6780b5450 0x7fb6780b43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:40.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.499+0000 7fb67e600640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fb6780cf590 0x7fb67806d600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:48438/0 (socket says 172.21.15.131:48438) 2024-05-26T22:49:40.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.499+0000 7fb67e600640 1 -- 172.21.15.131:0/1836371236 learned_addr learned my addr 172.21.15.131:0/1836371236 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:49:40.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.500+0000 7fb67d200640 1 --2- 172.21.15.131:0/1836371236 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb67806f420 0x7fb6780cfd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:40.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.500+0000 7fb67dc00640 1 -- 172.21.15.131:0/1836371236 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb67806f420 msgr2=0x7fb6780cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:40.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.500+0000 7fb67dc00640 1 --2- 172.21.15.131:0/1836371236 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb67806f420 0x7fb6780cfd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:40.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.500+0000 7fb67dc00640 1 -- 172.21.15.131:0/1836371236 >> v1:172.21.15.131:6789/0 conn(0x7fb6780cf590 legacy=0x7fb67806d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:40.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.500+0000 7fb67dc00640 1 -- 172.21.15.131:0/1836371236 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb678160460 con 0x7fb6780b5450 2024-05-26T22:49:40.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.501+0000 7fb67dc00640 1 --2- 172.21.15.131:0/1836371236 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6780b5450 0x7fb6780b43c0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fb66c0097a0 tx=0x7fb66c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21a3a80df84e2234 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:40.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.501+0000 7fb677e00640 1 -- 172.21.15.131:0/1836371236 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb66c037070 con 0x7fb6780b5450 2024-05-26T22:49:40.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.501+0000 7fb677e00640 1 -- 172.21.15.131:0/1836371236 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb66c032440 con 0x7fb6780b5450 2024-05-26T22:49:40.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.501+0000 7fb677e00640 1 -- 172.21.15.131:0/1836371236 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb66c032810 con 0x7fb6780b5450 2024-05-26T22:49:40.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.502+0000 7fb680600640 1 -- 172.21.15.131:0/1836371236 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6780b5450 msgr2=0x7fb6780b43c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:40.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.502+0000 7fb680600640 1 --2- 172.21.15.131:0/1836371236 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6780b5450 0x7fb6780b43c0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fb66c0097a0 tx=0x7fb66c02a180 comp rx=0 tx=0).stop 2024-05-26T22:49:40.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.502+0000 7fb680600640 1 -- 172.21.15.131:0/1836371236 shutdown_connections 2024-05-26T22:49:40.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.502+0000 7fb680600640 1 --2- 172.21.15.131:0/1836371236 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb67806f420 0x7fb6780cfd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:40.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.502+0000 7fb680600640 1 --2- 172.21.15.131:0/1836371236 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6780b5450 0x7fb6780b43c0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:40.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.502+0000 7fb680600640 1 -- 172.21.15.131:0/1836371236 >> 172.21.15.131:0/1836371236 conn(0x7fb67806bc90 msgr2=0x7fb6780b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:40.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.502+0000 7fb680600640 1 -- 172.21.15.131:0/1836371236 shutdown_connections 2024-05-26T22:49:40.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.502+0000 7fb680600640 1 -- 172.21.15.131:0/1836371236 wait complete. 2024-05-26T22:49:40.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.504+0000 7fb680600640 1 Processor -- start 2024-05-26T22:49:40.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.504+0000 7fb680600640 1 -- start start 2024-05-26T22:49:40.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.505+0000 7fb680600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb67806f420 0x7fb6780b1300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:40.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.505+0000 7fb680600640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb6780b5450 0x7fb6781516e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:40.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.505+0000 7fb680600640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb6780cf590 0x7fb678151c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:40.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.505+0000 7fb680600640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fb6780b5830 con 0x7fb67806f420 2024-05-26T22:49:40.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.505+0000 7fb680600640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb678160a20 con 0x7fb6780b5450 2024-05-26T22:49:40.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.505+0000 7fb680600640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb67815fde0 con 0x7fb6780cf590 2024-05-26T22:49:40.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.505+0000 7fb67dc00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb67806f420 0x7fb6780b1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:40.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.506+0000 7fb67dc00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb67806f420 0x7fb6780b1300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53376/0 (socket says 172.21.15.131:53376) 2024-05-26T22:49:40.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.506+0000 7fb67dc00640 1 -- 172.21.15.131:0/2487288897 learned_addr learned my addr 172.21.15.131:0/2487288897 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:40.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.506+0000 7fb67e600640 1 --2- 172.21.15.131:0/2487288897 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb6780cf590 0x7fb678151c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:40.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.506+0000 7fb67d200640 1 --2- 172.21.15.131:0/2487288897 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb6780b5450 0x7fb6781516e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:40.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.506+0000 7fb67dc00640 1 -- 172.21.15.131:0/2487288897 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb6780cf590 msgr2=0x7fb678151c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:40.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.506+0000 7fb67dc00640 1 --2- 172.21.15.131:0/2487288897 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb6780cf590 0x7fb678151c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:40.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.506+0000 7fb67dc00640 1 -- 172.21.15.131:0/2487288897 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb6780b5450 msgr2=0x7fb6781516e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:40.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.506+0000 7fb67dc00640 1 --2- 172.21.15.131:0/2487288897 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb6780b5450 0x7fb6781516e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:40.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.506+0000 7fb67dc00640 1 -- 172.21.15.131:0/2487288897 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6781545b0 con 0x7fb67806f420 2024-05-26T22:49:40.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.506+0000 7fb67dc00640 1 --2- 172.21.15.131:0/2487288897 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb67806f420 0x7fb6780b1300 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fb66c035040 tx=0x7fb66c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:40.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.507+0000 7fb675600640 1 -- 172.21.15.131:0/2487288897 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb66c037040 con 0x7fb67806f420 2024-05-26T22:49:40.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.507+0000 7fb675600640 1 -- 172.21.15.131:0/2487288897 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb66c030770 con 0x7fb67806f420 2024-05-26T22:49:40.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.507+0000 7fb675600640 1 -- 172.21.15.131:0/2487288897 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb66c030b40 con 0x7fb67806f420 2024-05-26T22:49:40.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.507+0000 7fb680600640 1 -- 172.21.15.131:0/2487288897 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb678154890 con 0x7fb67806f420 2024-05-26T22:49:40.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.507+0000 7fb680600640 1 -- 172.21.15.131:0/2487288897 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6781561a0 con 0x7fb67806f420 2024-05-26T22:49:40.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.508+0000 7fb680600640 1 -- 172.21.15.131:0/2487288897 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6780b43f0 con 0x7fb67806f420 2024-05-26T22:49:40.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.509+0000 7fb675600640 1 -- 172.21.15.131:0/2487288897 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb66c044020 con 0x7fb67806f420 2024-05-26T22:49:40.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.510+0000 7fb675600640 1 --2- 172.21.15.131:0/2487288897 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb65806be30 0x7fb65806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:40.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.510+0000 7fb675600640 1 -- 172.21.15.131:0/2487288897 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fb66c0ab3a0 con 0x7fb67806f420 2024-05-26T22:49:40.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.510+0000 7fb67d200640 1 --2- 172.21.15.131:0/2487288897 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb65806be30 0x7fb65806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:40.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.511+0000 7fb67d200640 1 --2- 172.21.15.131:0/2487288897 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb65806be30 0x7fb65806e2f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb664006fd0 tx=0x7fb664008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:40.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.515+0000 7fb675600640 1 -- 172.21.15.131:0/2487288897 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fb6780b43f0 con 0x7fb67806f420 2024-05-26T22:49:40.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.707+0000 7fb680600640 1 -- 172.21.15.131:0/2487288897 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command([{prefix=config set, name=mds_export_ephemeral_distributed}] v 0) -- 0x7fb678155cb0 con 0x7fb67806f420 2024-05-26T22:49:40.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.712+0000 7fb675600640 1 -- 172.21.15.131:0/2487288897 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_export_ephemeral_distributed}]=0 v12)=0 v12) ==== 133+0+0 (secure 0 0 0) 0x7fb66c032e20 con 0x7fb67806f420 2024-05-26T22:49:40.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.715+0000 7fb680600640 1 -- 172.21.15.131:0/2487288897 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb65806be30 msgr2=0x7fb65806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:40.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.715+0000 7fb680600640 1 --2- 172.21.15.131:0/2487288897 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb65806be30 0x7fb65806e2f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb664006fd0 tx=0x7fb664008040 comp rx=0 tx=0).stop 2024-05-26T22:49:40.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.715+0000 7fb680600640 1 -- 172.21.15.131:0/2487288897 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb67806f420 msgr2=0x7fb6780b1300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:40.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.715+0000 7fb680600640 1 --2- 172.21.15.131:0/2487288897 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb67806f420 0x7fb6780b1300 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fb66c035040 tx=0x7fb66c0046c0 comp rx=0 tx=0).stop 2024-05-26T22:49:40.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.715+0000 7fb680600640 1 -- 172.21.15.131:0/2487288897 shutdown_connections 2024-05-26T22:49:40.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.715+0000 7fb680600640 1 --2- 172.21.15.131:0/2487288897 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb6780cf590 0x7fb678151c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:40.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.715+0000 7fb680600640 1 --2- 172.21.15.131:0/2487288897 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb65806be30 0x7fb65806e2f0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:40.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.715+0000 7fb680600640 1 --2- 172.21.15.131:0/2487288897 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb6780b5450 0x7fb6781516e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:40.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.715+0000 7fb680600640 1 --2- 172.21.15.131:0/2487288897 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb67806f420 0x7fb6780b1300 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:40.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.715+0000 7fb680600640 1 -- 172.21.15.131:0/2487288897 >> 172.21.15.131:0/2487288897 conn(0x7fb67806bc90 msgr2=0x7fb67815c750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:40.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.715+0000 7fb680600640 1 -- 172.21.15.131:0/2487288897 shutdown_connections 2024-05-26T22:49:40.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.715+0000 7fb680600640 1 -- 172.21.15.131:0/2487288897 wait complete. 2024-05-26T22:49:40.732 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_export_ephemeral_random_max 1.0 2024-05-26T22:49:40.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.927+0000 7fb16d000640 1 Processor -- start 2024-05-26T22:49:40.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.928+0000 7fb16d000640 1 -- start start 2024-05-26T22:49:40.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.929+0000 7fb16d000640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fb16806d250 0x7fb16806d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:40.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.929+0000 7fb16d000640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb16806db70 con 0x7fb16815b730 2024-05-26T22:49:40.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.929+0000 7fb16d000640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb16813e820 con 0x7fb1681400c0 2024-05-26T22:49:40.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.929+0000 7fb16d000640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fb168071260 con 0x7fb16806d250 2024-05-26T22:49:40.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.930+0000 7fb15fe00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fb16815b730 0x7fb16815bbc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:48450/0 (socket says 172.21.15.131:48450) 2024-05-26T22:49:40.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.930+0000 7fb15fe00640 1 -- 172.21.15.131:0/3700422226 learned_addr learned my addr 172.21.15.131:0/3700422226 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:49:40.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.930+0000 7fb167400640 1 --2- 172.21.15.131:0/3700422226 >> v2:172.21.15.187:3300/0 conn(0x7fb16806d250 0x7fb16806d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:40.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.930+0000 7fb166a00640 1 -- 172.21.15.131:0/3700422226 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2589620292 0 0) 0x7fb16806db70 con 0x7fb16815b730 2024-05-26T22:49:40.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.931+0000 7fb166a00640 1 -- 172.21.15.131:0/3700422226 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb14c005730 con 0x7fb16815b730 2024-05-26T22:49:40.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.931+0000 7fb166a00640 1 -- 172.21.15.131:0/3700422226 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2165530379 0 0) 0x7fb14c005730 con 0x7fb16815b730 2024-05-26T22:49:40.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.931+0000 7fb166a00640 1 -- 172.21.15.131:0/3700422226 >> v2:172.21.15.187:3300/0 conn(0x7fb16806d250 msgr2=0x7fb16806d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:40.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.931+0000 7fb166a00640 1 --2- 172.21.15.131:0/3700422226 >> v2:172.21.15.187:3300/0 conn(0x7fb16806d250 0x7fb16806d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:40.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.931+0000 7fb166a00640 1 -- 172.21.15.131:0/3700422226 >> v1:172.21.15.187:6789/0 conn(0x7fb1681400c0 legacy=0x7fb1681404f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:40.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.931+0000 7fb166a00640 1 -- 172.21.15.131:0/3700422226 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb16815c4a0 con 0x7fb16815b730 2024-05-26T22:49:40.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.931+0000 7fb166a00640 1 -- 172.21.15.131:0/3700422226 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7fb150001670 con 0x7fb16815b730 2024-05-26T22:49:40.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.931+0000 7fb166a00640 1 -- 172.21.15.131:0/3700422226 >> v1:172.21.15.131:6789/0 conn(0x7fb16815b730 legacy=0x7fb16815bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:40.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.931+0000 7fb166a00640 1 --2- 172.21.15.131:0/3700422226 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb14c005bc0 0x7fb14c005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:40.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.931+0000 7fb166a00640 1 --2- 172.21.15.131:0/3700422226 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb14c006a10 0x7fb14c006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:40.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.931+0000 7fb166a00640 1 --2- 172.21.15.131:0/3700422226 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb14c0073d0 0x7fb14c00dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:40.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.931+0000 7fb166a00640 1 -- 172.21.15.131:0/3700422226 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fb168071260 con 0x7fb14c005bc0 2024-05-26T22:49:40.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.931+0000 7fb166a00640 1 -- 172.21.15.131:0/3700422226 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb14c00e270 con 0x7fb14c006a10 2024-05-26T22:49:40.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.931+0000 7fb166a00640 1 -- 172.21.15.131:0/3700422226 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb14c00e470 con 0x7fb14c0073d0 2024-05-26T22:49:40.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.931+0000 7fb167400640 1 --2- 172.21.15.131:0/3700422226 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb14c005bc0 0x7fb14c005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:40.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.932+0000 7fb15fe00640 1 --2- 172.21.15.131:0/3700422226 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb14c006a10 0x7fb14c006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:40.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.932+0000 7fb167400640 1 -- 172.21.15.131:0/3700422226 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb14c0073d0 msgr2=0x7fb14c00dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:40.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.932+0000 7fb167400640 1 --2- 172.21.15.131:0/3700422226 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb14c0073d0 0x7fb14c00dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:40.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.932+0000 7fb167400640 1 -- 172.21.15.131:0/3700422226 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb14c006a10 msgr2=0x7fb14c006e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:40.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.932+0000 7fb167400640 1 --2- 172.21.15.131:0/3700422226 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb14c006a10 0x7fb14c006e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:40.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.932+0000 7fb167400640 1 -- 172.21.15.131:0/3700422226 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb14c005730 con 0x7fb14c005bc0 2024-05-26T22:49:40.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.932+0000 7fb167e00640 1 -- 172.21.15.131:0/3700422226 reap_dead start 2024-05-26T22:49:40.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.933+0000 7fb167400640 1 --2- 172.21.15.131:0/3700422226 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb14c005bc0 0x7fb14c005fe0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fb158009ac0 tx=0x7fb14c007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bcb7247c44b55fb server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:40.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.933+0000 7fb166a00640 1 -- 172.21.15.131:0/3700422226 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb158036070 con 0x7fb14c005bc0 2024-05-26T22:49:40.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.933+0000 7fb166a00640 1 -- 172.21.15.131:0/3700422226 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb158031400 con 0x7fb14c005bc0 2024-05-26T22:49:40.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.933+0000 7fb166a00640 1 -- 172.21.15.131:0/3700422226 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb158009690 con 0x7fb14c005bc0 2024-05-26T22:49:40.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.934+0000 7fb16d000640 1 -- 172.21.15.131:0/3700422226 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb14c005bc0 msgr2=0x7fb14c005fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:40.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.934+0000 7fb16d000640 1 --2- 172.21.15.131:0/3700422226 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb14c005bc0 0x7fb14c005fe0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fb158009ac0 tx=0x7fb14c007a40 comp rx=0 tx=0).stop 2024-05-26T22:49:40.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.934+0000 7fb16d000640 1 -- 172.21.15.131:0/3700422226 shutdown_connections 2024-05-26T22:49:40.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.934+0000 7fb16d000640 1 --2- 172.21.15.131:0/3700422226 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb14c005bc0 0x7fb14c005fe0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:40.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.934+0000 7fb16d000640 1 -- 172.21.15.131:0/3700422226 >> 172.21.15.131:0/3700422226 conn(0x7fb1680b5660 msgr2=0x7fb1680b5a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:40.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.934+0000 7fb16d000640 1 -- 172.21.15.131:0/3700422226 shutdown_connections 2024-05-26T22:49:40.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.934+0000 7fb16d000640 1 -- 172.21.15.131:0/3700422226 wait complete. 2024-05-26T22:49:40.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.935+0000 7fb16d000640 1 Processor -- start 2024-05-26T22:49:40.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.936+0000 7fb16d000640 1 -- start start 2024-05-26T22:49:40.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.936+0000 7fb16d000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb14c005bc0 0x7fb1681544c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:40.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.937+0000 7fb16d000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb1680b3ef0 0x7fb1680b4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:40.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.937+0000 7fb16d000640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb1681320b0 0x7fb168132530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:40.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.937+0000 7fb16d000640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fb168071a70 con 0x7fb14c005bc0 2024-05-26T22:49:40.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.937+0000 7fb16d000640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb168071790 con 0x7fb1681320b0 2024-05-26T22:49:40.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.937+0000 7fb16d000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb16806d630 con 0x7fb1680b3ef0 2024-05-26T22:49:40.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.937+0000 7fb167400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb14c005bc0 0x7fb1681544c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:40.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.937+0000 7fb167400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb14c005bc0 0x7fb1681544c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53404/0 (socket says 172.21.15.131:53404) 2024-05-26T22:49:40.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.937+0000 7fb167400640 1 -- 172.21.15.131:0/2244452860 learned_addr learned my addr 172.21.15.131:0/2244452860 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:40.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.937+0000 7fb167e00640 1 --2- 172.21.15.131:0/2244452860 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb1681320b0 0x7fb168132530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:40.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.937+0000 7fb15fe00640 1 --2- 172.21.15.131:0/2244452860 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb1680b3ef0 0x7fb1680b4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:40.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.937+0000 7fb167400640 1 -- 172.21.15.131:0/2244452860 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb1680b3ef0 msgr2=0x7fb1680b4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:40.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.937+0000 7fb167400640 1 --2- 172.21.15.131:0/2244452860 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb1680b3ef0 0x7fb1680b4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:40.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.937+0000 7fb167400640 1 -- 172.21.15.131:0/2244452860 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb1681320b0 msgr2=0x7fb168132530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:40.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.937+0000 7fb167400640 1 --2- 172.21.15.131:0/2244452860 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb1681320b0 0x7fb168132530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:40.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.937+0000 7fb167400640 1 -- 172.21.15.131:0/2244452860 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb168133eb0 con 0x7fb14c005bc0 2024-05-26T22:49:40.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.937+0000 7fb167e00640 1 --2- 172.21.15.131:0/2244452860 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb1681320b0 0x7fb168132530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:49:40.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.938+0000 7fb167400640 1 --2- 172.21.15.131:0/2244452860 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb14c005bc0 0x7fb1681544c0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fb158009be0 tx=0x7fb158004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:40.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.938+0000 7fb15f400640 1 -- 172.21.15.131:0/2244452860 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb15803f020 con 0x7fb14c005bc0 2024-05-26T22:49:40.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.938+0000 7fb15f400640 1 -- 172.21.15.131:0/2244452860 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb158037040 con 0x7fb14c005bc0 2024-05-26T22:49:40.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.938+0000 7fb16d000640 1 -- 172.21.15.131:0/2244452860 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb168132e10 con 0x7fb14c005bc0 2024-05-26T22:49:40.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.938+0000 7fb15f400640 1 -- 172.21.15.131:0/2244452860 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb158036070 con 0x7fb14c005bc0 2024-05-26T22:49:40.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.938+0000 7fb16d000640 1 -- 172.21.15.131:0/2244452860 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1681333d0 con 0x7fb14c005bc0 2024-05-26T22:49:40.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.939+0000 7fb16d000640 1 -- 172.21.15.131:0/2244452860 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb16813fd00 con 0x7fb14c005bc0 2024-05-26T22:49:40.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.940+0000 7fb15f400640 1 -- 172.21.15.131:0/2244452860 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb158042020 con 0x7fb14c005bc0 2024-05-26T22:49:40.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.942+0000 7fb15f400640 1 --2- 172.21.15.131:0/2244452860 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb14406be60 0x7fb14406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:40.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.942+0000 7fb15f400640 1 -- 172.21.15.131:0/2244452860 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fb1580aaec0 con 0x7fb14c005bc0 2024-05-26T22:49:40.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.942+0000 7fb15fe00640 1 --2- 172.21.15.131:0/2244452860 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb14406be60 0x7fb14406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:40.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.942+0000 7fb15fe00640 1 --2- 172.21.15.131:0/2244452860 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb14406be60 0x7fb14406e320 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb150000f60 tx=0x7fb150009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:40.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:40.946+0000 7fb15f400640 1 -- 172.21.15.131:0/2244452860 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fb16813fd00 con 0x7fb14c005bc0 2024-05-26T22:49:41.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:41.143+0000 7fb16d000640 1 -- 172.21.15.131:0/2244452860 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command([{prefix=config set, name=mds_export_ephemeral_random_max}] v 0) -- 0x7fb16806dd60 con 0x7fb14c005bc0 2024-05-26T22:49:41.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:41.149+0000 7fb15f400640 1 -- 172.21.15.131:0/2244452860 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_export_ephemeral_random_max}]=0 v13)=0 v13) ==== 131+0+0 (secure 0 0 0) 0x7fb158049030 con 0x7fb14c005bc0 2024-05-26T22:49:41.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:41.153+0000 7fb16d000640 1 -- 172.21.15.131:0/2244452860 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb14406be60 msgr2=0x7fb14406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:41.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:41.153+0000 7fb16d000640 1 --2- 172.21.15.131:0/2244452860 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb14406be60 0x7fb14406e320 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb150000f60 tx=0x7fb150009040 comp rx=0 tx=0).stop 2024-05-26T22:49:41.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:41.153+0000 7fb16d000640 1 -- 172.21.15.131:0/2244452860 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb14c005bc0 msgr2=0x7fb1681544c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:41.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:41.153+0000 7fb16d000640 1 --2- 172.21.15.131:0/2244452860 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb14c005bc0 0x7fb1681544c0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fb158009be0 tx=0x7fb158004730 comp rx=0 tx=0).stop 2024-05-26T22:49:41.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:41.153+0000 7fb16d000640 1 -- 172.21.15.131:0/2244452860 shutdown_connections 2024-05-26T22:49:41.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:41.153+0000 7fb16d000640 1 --2- 172.21.15.131:0/2244452860 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb1680b3ef0 0x7fb1680b4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:41.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:41.153+0000 7fb16d000640 1 --2- 172.21.15.131:0/2244452860 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb14406be60 0x7fb14406e320 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:41.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:41.153+0000 7fb16d000640 1 --2- 172.21.15.131:0/2244452860 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb1681320b0 0x7fb168132530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:41.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:41.153+0000 7fb16d000640 1 --2- 172.21.15.131:0/2244452860 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb14c005bc0 0x7fb1681544c0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:41.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:41.153+0000 7fb16d000640 1 -- 172.21.15.131:0/2244452860 >> 172.21.15.131:0/2244452860 conn(0x7fb1680b5660 msgr2=0x7fb16815daa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:41.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:41.153+0000 7fb16d000640 1 -- 172.21.15.131:0/2244452860 shutdown_connections 2024-05-26T22:49:41.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:41.153+0000 7fb16d000640 1 -- 172.21.15.131:0/2244452860 wait complete. 2024-05-26T22:49:41.172 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-05-26T22:49:41.172 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2024-05-26T22:49:41.173 DEBUG:teuthology.orchestra.run.smithi131:> 2024-05-26T22:49:41.173 DEBUG:teuthology.orchestra.run.smithi131:> # Use up a random number of inode numbers so the ephemeral pinning is not the same every test. 2024-05-26T22:49:41.173 DEBUG:teuthology.orchestra.run.smithi131:> mkdir .inode_number_thrash 2024-05-26T22:49:41.173 DEBUG:teuthology.orchestra.run.smithi131:> count=$((RANDOM % 1024)) 2024-05-26T22:49:41.173 DEBUG:teuthology.orchestra.run.smithi131:> for ((i = 0; i < count; i++)); do touch .inode_number_thrash/$i; done 2024-05-26T22:49:41.173 DEBUG:teuthology.orchestra.run.smithi131:> rm -rf .inode_number_thrash 2024-05-26T22:49:41.173 DEBUG:teuthology.orchestra.run.smithi131:> ') 2024-05-26T22:49:41.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:41 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-26T22:49:43.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-26T22:49:43.420 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2024-05-26T22:49:43.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:43.622+0000 7f63f0e00640 1 Processor -- start 2024-05-26T22:49:43.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:43.622+0000 7f63f0e00640 1 -- start start 2024-05-26T22:49:43.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:43.625+0000 7f63f0e00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f63ec06f420 0x7f63ec0cfd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:43.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:43.625+0000 7f63f0e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f63ec0cf590 0x7f63ec0d0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:43.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:43.625+0000 7f63f0e00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63ec160180 con 0x7f63ec0b5450 2024-05-26T22:49:43.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:43.625+0000 7f63f0e00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f63ec160d90 con 0x7f63ec06f420 2024-05-26T22:49:43.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:43.625+0000 7f63f0e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f63ec160460 con 0x7f63ec0cf590 2024-05-26T22:49:43.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:43.625+0000 7f63e9e00640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f63ec0b5450 0x7f63ec0b43c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.131:58116/0 (socket says 172.21.15.131:58116) 2024-05-26T22:49:43.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:43.625+0000 7f63e9e00640 1 -- 172.21.15.131:0/4278235253 learned_addr learned my addr 172.21.15.131:0/4278235253 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:49:43.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:43.625+0000 7f63e9400640 1 --2- 172.21.15.131:0/4278235253 >> v2:172.21.15.187:3300/0 conn(0x7f63ec06f420 0x7f63ec0cfd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:43.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:43.626+0000 7f63ea800640 1 --2- 172.21.15.131:0/4278235253 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f63ec0cf590 0x7f63ec0d0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:43.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:43.626+0000 7f63e8a00640 1 -- 172.21.15.131:0/4278235253 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4132158721 0 0) 0x7f63ec160180 con 0x7f63ec0b5450 2024-05-26T22:49:43.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:43.626+0000 7f63e8a00640 1 -- 172.21.15.131:0/4278235253 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f63d0003520 con 0x7f63ec0b5450 2024-05-26T22:49:43.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:43.627+0000 7f63e9400640 1 -- 172.21.15.131:0/4278235253 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f63ec0cf590 msgr2=0x7f63ec0d0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:43.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:43.627+0000 7f63e9400640 1 --2- 172.21.15.131:0/4278235253 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f63ec0cf590 0x7f63ec0d0290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:43.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:43.627+0000 7f63e9400640 1 -- 172.21.15.131:0/4278235253 >> v1:172.21.15.187:6789/0 conn(0x7f63ec0b5450 legacy=0x7f63ec0b43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:43.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:43.627+0000 7f63e9400640 1 -- 172.21.15.131:0/4278235253 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63ec160740 con 0x7f63ec06f420 2024-05-26T22:49:43.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:43.629+0000 7f63e9400640 1 --2- 172.21.15.131:0/4278235253 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f63ec06f420 0x7f63ec0cfd50 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f63d80097a0 tx=0x7f63d802a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f5f628de10188e1 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:43.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:43.630+0000 7f63e8a00640 1 -- 172.21.15.131:0/4278235253 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63d8037070 con 0x7f63ec06f420 2024-05-26T22:49:43.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:43.630+0000 7f63e8a00640 1 -- 172.21.15.131:0/4278235253 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63d8032440 con 0x7f63ec06f420 2024-05-26T22:49:43.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:43.630+0000 7f63e8a00640 1 -- 172.21.15.131:0/4278235253 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63d8032810 con 0x7f63ec06f420 2024-05-26T22:49:43.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:43.631+0000 7f63f0e00640 1 -- 172.21.15.131:0/4278235253 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f63ec06f420 msgr2=0x7f63ec0cfd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:43.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:43.631+0000 7f63f0e00640 1 --2- 172.21.15.131:0/4278235253 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f63ec06f420 0x7f63ec0cfd50 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f63d80097a0 tx=0x7f63d802a180 comp rx=0 tx=0).stop 2024-05-26T22:49:43.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:43.632+0000 7f63f0e00640 1 -- 172.21.15.131:0/4278235253 shutdown_connections 2024-05-26T22:49:43.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:43.632+0000 7f63f0e00640 1 --2- 172.21.15.131:0/4278235253 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f63ec0cf590 0x7f63ec0d0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:43.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:43.632+0000 7f63f0e00640 1 --2- 172.21.15.131:0/4278235253 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f63ec06f420 0x7f63ec0cfd50 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:43.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:43.632+0000 7f63f0e00640 1 -- 172.21.15.131:0/4278235253 >> 172.21.15.131:0/4278235253 conn(0x7f63ec06bc90 msgr2=0x7f63ec0b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:43.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:43.633+0000 7f63f0e00640 1 -- 172.21.15.131:0/4278235253 shutdown_connections 2024-05-26T22:49:43.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:43.633+0000 7f63f0e00640 1 -- 172.21.15.131:0/4278235253 wait complete. 2024-05-26T22:49:43.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:43.636+0000 7f63f0e00640 1 Processor -- start 2024-05-26T22:49:43.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:43.636+0000 7f63f0e00640 1 -- start start 2024-05-26T22:49:43.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:43.638+0000 7f63f0e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f63ec06f420 0x7f63ec146fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:43.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:43.638+0000 7f63f0e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63ec0b5450 0x7f63ec0ccc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:43.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:43.638+0000 7f63f0e00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f63ec0cf590 0x7f63ec0cd160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:43.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:43.638+0000 7f63f0e00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f63ec15fde0 con 0x7f63ec0b5450 2024-05-26T22:49:43.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:43.638+0000 7f63f0e00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f63ec0b5830 con 0x7f63ec0cf590 2024-05-26T22:49:43.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:43.638+0000 7f63f0e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f63ec15ff60 con 0x7f63ec06f420 2024-05-26T22:49:43.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:43.639+0000 7f63e9e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f63ec06f420 0x7f63ec146fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:43.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:43.639+0000 7f63e9400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63ec0b5450 0x7f63ec0ccc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:43.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:43.639+0000 7f63e9400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63ec0b5450 0x7f63ec0ccc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53416/0 (socket says 172.21.15.131:53416) 2024-05-26T22:49:43.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:43.639+0000 7f63e9400640 1 -- 172.21.15.131:0/3703060503 learned_addr learned my addr 172.21.15.131:0/3703060503 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:43.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:43.640+0000 7f63ea800640 1 --2- 172.21.15.131:0/3703060503 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f63ec0cf590 0x7f63ec0cd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:43.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:43.640+0000 7f63e9e00640 1 -- 172.21.15.131:0/3703060503 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f63ec0cf590 msgr2=0x7f63ec0cd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:43.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:43.640+0000 7f63e9e00640 1 --2- 172.21.15.131:0/3703060503 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f63ec0cf590 0x7f63ec0cd160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:43.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:43.640+0000 7f63e9e00640 1 -- 172.21.15.131:0/3703060503 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63ec0b5450 msgr2=0x7f63ec0ccc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:43.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:43.640+0000 7f63e9e00640 1 --2- 172.21.15.131:0/3703060503 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63ec0b5450 0x7f63ec0ccc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:43.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:43.640+0000 7f63e9e00640 1 -- 172.21.15.131:0/3703060503 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63ec09b720 con 0x7f63ec06f420 2024-05-26T22:49:43.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:43.640+0000 7f63e9400640 1 --2- 172.21.15.131:0/3703060503 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63ec0b5450 0x7f63ec0ccc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-26T22:49:43.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:43.640+0000 7f63e9e00640 1 --2- 172.21.15.131:0/3703060503 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f63ec06f420 0x7f63ec146fa0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f63dc00c400 tx=0x7f63dc00c8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:43.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:43.641+0000 7f63e2000640 1 -- 172.21.15.131:0/3703060503 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63dc017070 con 0x7f63ec06f420 2024-05-26T22:49:43.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:43.641+0000 7f63e2000640 1 -- 172.21.15.131:0/3703060503 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63dc005ce0 con 0x7f63ec06f420 2024-05-26T22:49:43.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:43.641+0000 7f63e2000640 1 -- 172.21.15.131:0/3703060503 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63dc0072f0 con 0x7f63ec06f420 2024-05-26T22:49:43.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:43.641+0000 7f63f0e00640 1 -- 172.21.15.131:0/3703060503 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63ec1474e0 con 0x7f63ec06f420 2024-05-26T22:49:43.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:43.641+0000 7f63f0e00640 1 -- 172.21.15.131:0/3703060503 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f63ec155e50 con 0x7f63ec06f420 2024-05-26T22:49:43.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:43.643+0000 7f63d7400640 1 -- 172.21.15.131:0/3703060503 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63ec074e90 con 0x7f63ec06f420 2024-05-26T22:49:43.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:43.645+0000 7f63e2000640 1 -- 172.21.15.131:0/3703060503 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f63dc01a070 con 0x7f63ec06f420 2024-05-26T22:49:43.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:43.646+0000 7f63e2000640 1 --2- 172.21.15.131:0/3703060503 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f63cc06be60 0x7f63cc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:43.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:43.646+0000 7f63e2000640 1 -- 172.21.15.131:0/3703060503 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f63dc08a8f0 con 0x7f63ec06f420 2024-05-26T22:49:43.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:43.646+0000 7f63e9400640 1 --2- 172.21.15.131:0/3703060503 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f63cc06be60 0x7f63cc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:43.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:43.648+0000 7f63e9400640 1 --2- 172.21.15.131:0/3703060503 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f63cc06be60 0x7f63cc06e320 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f63d802a1c0 tx=0x7f63d802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:43.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:43.651+0000 7f63e2000640 1 -- 172.21.15.131:0/3703060503 <== mon.2 v2:172.21.15.187:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f63ec09b720 con 0x7f63ec06f420 2024-05-26T22:49:43.873 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-26T22:49:43.881 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-26T22:49:43.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:43.889+0000 7f63d7400640 1 -- 172.21.15.131:0/3703060503 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7f63ec09e010 con 0x7f63ec06f420 2024-05-26T22:49:44.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:44.286+0000 7f63e2000640 1 -- 172.21.15.131:0/3703060503 <== mon.2 v2:172.21.15.187:3301/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v16) ==== 105+0+0 (secure 0 0 0) 0x7f63dc058c30 con 0x7f63ec06f420 2024-05-26T22:49:44.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:44.290+0000 7f63d7400640 1 -- 172.21.15.131:0/3703060503 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f63cc06be60 msgr2=0x7f63cc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:44.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:44.290+0000 7f63d7400640 1 --2- 172.21.15.131:0/3703060503 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f63cc06be60 0x7f63cc06e320 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f63d802a1c0 tx=0x7f63d802e000 comp rx=0 tx=0).stop 2024-05-26T22:49:44.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:44.290+0000 7f63d7400640 1 -- 172.21.15.131:0/3703060503 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f63ec06f420 msgr2=0x7f63ec146fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:44.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:44.290+0000 7f63d7400640 1 --2- 172.21.15.131:0/3703060503 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f63ec06f420 0x7f63ec146fa0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f63dc00c400 tx=0x7f63dc00c8c0 comp rx=0 tx=0).stop 2024-05-26T22:49:44.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:44.290+0000 7f63d7400640 1 -- 172.21.15.131:0/3703060503 shutdown_connections 2024-05-26T22:49:44.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:44.290+0000 7f63d7400640 1 --2- 172.21.15.131:0/3703060503 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f63cc06be60 0x7f63cc06e320 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:44.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:44.290+0000 7f63d7400640 1 --2- 172.21.15.131:0/3703060503 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f63ec0cf590 0x7f63ec0cd160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:44.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:44.290+0000 7f63d7400640 1 --2- 172.21.15.131:0/3703060503 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63ec0b5450 0x7f63ec0ccc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:44.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:44.290+0000 7f63d7400640 1 --2- 172.21.15.131:0/3703060503 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f63ec06f420 0x7f63ec146fa0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:44.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:44.290+0000 7f63d7400640 1 -- 172.21.15.131:0/3703060503 >> 172.21.15.131:0/3703060503 conn(0x7f63ec06bc90 msgr2=0x7f63ec0afb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:44.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:44.290+0000 7f63d7400640 1 -- 172.21.15.131:0/3703060503 shutdown_connections 2024-05-26T22:49:44.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:44.290+0000 7f63d7400640 1 -- 172.21.15.131:0/3703060503 wait complete. 2024-05-26T22:49:44.314 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-26T22:49:44.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:44.501+0000 7f6221a00640 1 Processor -- start 2024-05-26T22:49:44.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:44.501+0000 7f6221a00640 1 -- start start 2024-05-26T22:49:44.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:44.503+0000 7f6221a00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f621c0ad250 0x7f621c0b3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:44.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:44.503+0000 7f6221a00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f621c15cb20 con 0x7f621c1569d0 2024-05-26T22:49:44.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:44.503+0000 7f6221a00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f621c15ce00 con 0x7f621c06ee20 2024-05-26T22:49:44.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:44.503+0000 7f6221a00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f621c15d0e0 con 0x7f621c0ad250 2024-05-26T22:49:44.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:44.503+0000 7f621b400640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f621c1569d0 0x7f621c160010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:56824/0 (socket says 172.21.15.131:56824) 2024-05-26T22:49:44.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:44.503+0000 7f6213e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f621c0ad250 0x7f621c0b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:44.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:44.503+0000 7f621b400640 1 -- 172.21.15.131:0/1992798958 learned_addr learned my addr 172.21.15.131:0/1992798958 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:49:44.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:44.504+0000 7f6213e00640 1 -- 172.21.15.131:0/1992798958 >> v1:172.21.15.187:6789/0 conn(0x7f621c06ee20 legacy=0x7f621c1562c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:44.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:44.504+0000 7f6213e00640 1 -- 172.21.15.131:0/1992798958 >> v1:172.21.15.131:6789/0 conn(0x7f621c1569d0 legacy=0x7f621c160010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:44.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:44.504+0000 7f6213e00640 1 -- 172.21.15.131:0/1992798958 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f621c15d3c0 con 0x7f621c0ad250 2024-05-26T22:49:44.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:44.504+0000 7f621a000640 1 -- 172.21.15.131:0/1992798958 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 154536706 0 0) 0x7f621c15cb20 con 0x7f621c1569d0 2024-05-26T22:49:44.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:44.505+0000 7f6213e00640 1 --2- 172.21.15.131:0/1992798958 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f621c0ad250 0x7f621c0b3190 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f62040097a0 tx=0x7f620402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85350d69a735312d server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:44.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:44.505+0000 7f621a000640 1 -- 172.21.15.131:0/1992798958 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6204037070 con 0x7f621c0ad250 2024-05-26T22:49:44.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:44.505+0000 7f621a000640 1 -- 172.21.15.131:0/1992798958 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6204032440 con 0x7f621c0ad250 2024-05-26T22:49:44.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:44.505+0000 7f621a000640 1 -- 172.21.15.131:0/1992798958 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6204032810 con 0x7f621c0ad250 2024-05-26T22:49:44.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:44.506+0000 7f6221a00640 1 -- 172.21.15.131:0/1992798958 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f621c0ad250 msgr2=0x7f621c0b3190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:44.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:44.506+0000 7f6221a00640 1 --2- 172.21.15.131:0/1992798958 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f621c0ad250 0x7f621c0b3190 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f62040097a0 tx=0x7f620402a1a0 comp rx=0 tx=0).stop 2024-05-26T22:49:44.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:44.506+0000 7f6221a00640 1 -- 172.21.15.131:0/1992798958 shutdown_connections 2024-05-26T22:49:44.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:44.506+0000 7f6221a00640 1 --2- 172.21.15.131:0/1992798958 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f621c0ad250 0x7f621c0b3190 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:44.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:44.506+0000 7f6221a00640 1 -- 172.21.15.131:0/1992798958 >> 172.21.15.131:0/1992798958 conn(0x7f621c0b1e90 msgr2=0x7f621c0b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:44.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:44.506+0000 7f6221a00640 1 -- 172.21.15.131:0/1992798958 shutdown_connections 2024-05-26T22:49:44.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:44.506+0000 7f6221a00640 1 -- 172.21.15.131:0/1992798958 wait complete. 2024-05-26T22:49:44.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:44.508+0000 7f6221a00640 1 Processor -- start 2024-05-26T22:49:44.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:44.508+0000 7f6221a00640 1 -- start start 2024-05-26T22:49:44.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:44.509+0000 7f6221a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f621c06ee20 0x7f621c147440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:44.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:44.509+0000 7f6221a00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f621c0ad250 0x7f621c09d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:44.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:44.509+0000 7f6221a00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f621c1569d0 0x7f621c09da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:44.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:44.509+0000 7f6221a00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f621c15c900 con 0x7f621c06ee20 2024-05-26T22:49:44.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:44.509+0000 7f6221a00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f621c15c780 con 0x7f621c1569d0 2024-05-26T22:49:44.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:44.509+0000 7f6221a00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f621c06f200 con 0x7f621c0ad250 2024-05-26T22:49:44.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:44.509+0000 7f621aa00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f621c06ee20 0x7f621c147440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:44.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:44.509+0000 7f621aa00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f621c06ee20 0x7f621c147440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50152/0 (socket says 172.21.15.131:50152) 2024-05-26T22:49:44.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:44.509+0000 7f621aa00640 1 -- 172.21.15.131:0/2996677412 learned_addr learned my addr 172.21.15.131:0/2996677412 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:44.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:44.510+0000 7f621aa00640 1 -- 172.21.15.131:0/2996677412 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f621c0ad250 msgr2=0x7f621c09d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:44.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:44.510+0000 7f621aa00640 1 --2- 172.21.15.131:0/2996677412 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f621c0ad250 0x7f621c09d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:44.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:44.510+0000 7f621aa00640 1 -- 172.21.15.131:0/2996677412 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f621c1569d0 msgr2=0x7f621c09da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:44.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:44.510+0000 7f621aa00640 1 --2- 172.21.15.131:0/2996677412 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f621c1569d0 0x7f621c09da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:44.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:44.510+0000 7f621aa00640 1 -- 172.21.15.131:0/2996677412 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f621c09ad50 con 0x7f621c06ee20 2024-05-26T22:49:44.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:44.510+0000 7f621aa00640 1 --2- 172.21.15.131:0/2996677412 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f621c06ee20 0x7f621c147440 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f620c00c450 tx=0x7f620c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:44.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:44.510+0000 7f6212a00640 1 -- 172.21.15.131:0/2996677412 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f620c0052d0 con 0x7f621c06ee20 2024-05-26T22:49:44.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:44.510+0000 7f6212a00640 1 -- 172.21.15.131:0/2996677412 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f620c005470 con 0x7f621c06ee20 2024-05-26T22:49:44.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:44.510+0000 7f6212a00640 1 -- 172.21.15.131:0/2996677412 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f620c012870 con 0x7f621c06ee20 2024-05-26T22:49:44.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:44.510+0000 7f6221a00640 1 -- 172.21.15.131:0/2996677412 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f621c09b030 con 0x7f621c06ee20 2024-05-26T22:49:44.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:44.510+0000 7f6221a00640 1 -- 172.21.15.131:0/2996677412 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f621c09b5f0 con 0x7f621c06ee20 2024-05-26T22:49:44.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:44.511+0000 7f6221a00640 1 -- 172.21.15.131:0/2996677412 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f621c0694e0 con 0x7f621c06ee20 2024-05-26T22:49:44.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:44.512+0000 7f6212a00640 1 -- 172.21.15.131:0/2996677412 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f620c012a10 con 0x7f621c06ee20 2024-05-26T22:49:44.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:44.513+0000 7f6212a00640 1 --2- 172.21.15.131:0/2996677412 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f61f806be30 0x7f61f806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:44.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:44.513+0000 7f6212a00640 1 -- 172.21.15.131:0/2996677412 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f620c08ab30 con 0x7f621c06ee20 2024-05-26T22:49:44.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:44.514+0000 7f6213e00640 1 --2- 172.21.15.131:0/2996677412 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f61f806be30 0x7f61f806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:44.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:44.514+0000 7f6213e00640 1 --2- 172.21.15.131:0/2996677412 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f61f806be30 0x7f61f806e2f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6204034000 tx=0x7f620402c000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:44.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:44.519+0000 7f6212a00640 1 -- 172.21.15.131:0/2996677412 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f620c059440 con 0x7f621c06ee20 2024-05-26T22:49:44.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:44.756+0000 7f6221a00640 1 -- 172.21.15.131:0/2996677412 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f621c0af0b0 con 0x7f621c06ee20 2024-05-26T22:49:44.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:44.757+0000 7f6212a00640 1 -- 172.21.15.131:0/2996677412 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5209 (secure 0 0 0) 0x7f620c058d40 con 0x7f621c06ee20 2024-05-26T22:49:44.757 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:49:44.757 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/4292497989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":4292497989},{"type":"v1","addr":"172.21.15.187:6835","nonce":4292497989}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4678,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6836/4032294430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":4032294430},{"type":"v1","addr":"172.21.15.131:6836","nonce":4032294430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-26T22:49:30.955525+0000","modified":"2024-05-26T22:49:44.286161+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1],"up":{"mds_0":5115,"mds_1":4687},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4687":{"gid":4687,"name":"b","rank":1,"incarnation":17,"state":"up:creating","state_seq":1,"addr":"172.21.15.187:6833/3274403209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3274403209},{"type":"v1","addr":"172.21.15.187:6833","nonce":3274403209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5115":{"gid":5115,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.131:6837/2062704413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6835","nonce":2062704413},{"type":"v1","addr":"172.21.15.131:6837","nonce":2062704413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5115,"qdb_cluster":[5115]},"id":2}]} 2024-05-26T22:49:44.757 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 17 2024-05-26T22:49:44.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:44.761+0000 7f6221a00640 1 -- 172.21.15.131:0/2996677412 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f61f806be30 msgr2=0x7f61f806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:44.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:44.761+0000 7f6221a00640 1 --2- 172.21.15.131:0/2996677412 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f61f806be30 0x7f61f806e2f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6204034000 tx=0x7f620402c000 comp rx=0 tx=0).stop 2024-05-26T22:49:44.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:44.761+0000 7f6221a00640 1 -- 172.21.15.131:0/2996677412 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f621c06ee20 msgr2=0x7f621c147440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:44.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:44.761+0000 7f6221a00640 1 --2- 172.21.15.131:0/2996677412 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f621c06ee20 0x7f621c147440 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f620c00c450 tx=0x7f620c00c910 comp rx=0 tx=0).stop 2024-05-26T22:49:44.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:44.761+0000 7f6221a00640 1 -- 172.21.15.131:0/2996677412 shutdown_connections 2024-05-26T22:49:44.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:44.761+0000 7f6221a00640 1 --2- 172.21.15.131:0/2996677412 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f621c0ad250 0x7f621c09d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:44.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:44.761+0000 7f6221a00640 1 --2- 172.21.15.131:0/2996677412 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f61f806be30 0x7f61f806e2f0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:44.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:44.761+0000 7f6221a00640 1 --2- 172.21.15.131:0/2996677412 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f621c1569d0 0x7f621c09da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:44.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:44.761+0000 7f6221a00640 1 --2- 172.21.15.131:0/2996677412 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f621c06ee20 0x7f621c147440 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:44.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:44.761+0000 7f6221a00640 1 -- 172.21.15.131:0/2996677412 >> 172.21.15.131:0/2996677412 conn(0x7f621c0b1e90 msgr2=0x7f621c0b37b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:44.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:44.761+0000 7f6221a00640 1 -- 172.21.15.131:0/2996677412 shutdown_connections 2024-05-26T22:49:44.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:44.761+0000 7f6221a00640 1 -- 172.21.15.131:0/2996677412 wait complete. 2024-05-26T22:49:45.183 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-26T22:49:30.955525+0000', 'modified': '2024-05-26T22:49:44.286161+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1], 'up': {'mds_0': 5115, 'mds_1': 4687}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4687': {'gid': 4687, 'name': 'b', 'rank': 1, 'incarnation': 17, 'state': 'up:creating', 'state_seq': 1, 'addr': '172.21.15.187:6833/3274403209', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.187:6832', 'nonce': 3274403209}, {'type': 'v1', 'addr': '172.21.15.187:6833', 'nonce': 3274403209}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_5115': {'gid': 5115, 'name': 'c', 'rank': 0, 'incarnation': 12, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.131:6837/2062704413', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6835', 'nonce': 2062704413}, {'type': 'v1', 'addr': '172.21.15.131:6837', 'nonce': 2062704413}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5115, 'qdb_cluster': [5115]} 2024-05-26T22:49:45.183 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_4687:up:creating 2024-05-26T22:49:46.184 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-26T22:49:46.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:46.378+0000 7f7d67c00640 1 Processor -- start 2024-05-26T22:49:46.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:46.378+0000 7f7d67c00640 1 -- start start 2024-05-26T22:49:46.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:46.379+0000 7f7d67c00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f7d6015c3d0 0x7f7d6006ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:46.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:46.379+0000 7f7d67c00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f7d6006f490 0x7f7d600ade40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:46.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:46.379+0000 7f7d67c00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d600722a0 con 0x7f7d60155840 2024-05-26T22:49:46.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:46.379+0000 7f7d67c00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f7d60072040 con 0x7f7d6015c3d0 2024-05-26T22:49:46.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:46.379+0000 7f7d67c00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f7d60072580 con 0x7f7d6006f490 2024-05-26T22:49:46.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:46.379+0000 7f7d57e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f7d6015c3d0 0x7f7d6006ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:46.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:46.379+0000 7f7d57e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f7d6015c3d0 0x7f7d6006ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50158/0 (socket says 172.21.15.131:50158) 2024-05-26T22:49:46.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:46.379+0000 7f7d57e00640 1 -- 172.21.15.131:0/1506231886 learned_addr learned my addr 172.21.15.131:0/1506231886 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:46.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:46.380+0000 7f7d57e00640 1 -- 172.21.15.131:0/1506231886 >> v2:172.21.15.187:3300/0 conn(0x7f7d6006f490 msgr2=0x7f7d600ade40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:46.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:46.380+0000 7f7d57e00640 1 --2- 172.21.15.131:0/1506231886 >> v2:172.21.15.187:3300/0 conn(0x7f7d6006f490 0x7f7d600ade40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:46.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:46.380+0000 7f7d57e00640 1 -- 172.21.15.131:0/1506231886 >> v1:172.21.15.187:6789/0 conn(0x7f7d60155840 legacy=0x7f7d60071930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:46.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:46.380+0000 7f7d57e00640 1 -- 172.21.15.131:0/1506231886 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d60072860 con 0x7f7d6015c3d0 2024-05-26T22:49:46.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:46.381+0000 7f7d57e00640 1 --2- 172.21.15.131:0/1506231886 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d6015c3d0 0x7f7d6006ef50 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f7d60070d90 tx=0x7f7d4802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a2374fdb3203513 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:46.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:46.381+0000 7f7d5fe00640 1 -- 172.21.15.131:0/1506231886 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d48037070 con 0x7f7d6015c3d0 2024-05-26T22:49:46.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:46.381+0000 7f7d5fe00640 1 -- 172.21.15.131:0/1506231886 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d48032440 con 0x7f7d6015c3d0 2024-05-26T22:49:46.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:46.381+0000 7f7d5fe00640 1 -- 172.21.15.131:0/1506231886 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d48032810 con 0x7f7d6015c3d0 2024-05-26T22:49:46.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:46.382+0000 7f7d67c00640 1 -- 172.21.15.131:0/1506231886 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d6015c3d0 msgr2=0x7f7d6006ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:46.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:46.382+0000 7f7d67c00640 1 --2- 172.21.15.131:0/1506231886 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d6015c3d0 0x7f7d6006ef50 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f7d60070d90 tx=0x7f7d4802a230 comp rx=0 tx=0).stop 2024-05-26T22:49:46.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:46.382+0000 7f7d67c00640 1 -- 172.21.15.131:0/1506231886 shutdown_connections 2024-05-26T22:49:46.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:46.382+0000 7f7d67c00640 1 --2- 172.21.15.131:0/1506231886 >> v2:172.21.15.187:3300/0 conn(0x7f7d6006f490 0x7f7d600ade40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:46.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:46.382+0000 7f7d67c00640 1 --2- 172.21.15.131:0/1506231886 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d6015c3d0 0x7f7d6006ef50 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:46.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:46.382+0000 7f7d67c00640 1 -- 172.21.15.131:0/1506231886 >> 172.21.15.131:0/1506231886 conn(0x7f7d600b1e90 msgr2=0x7f7d600b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:46.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:46.382+0000 7f7d67c00640 1 -- 172.21.15.131:0/1506231886 shutdown_connections 2024-05-26T22:49:46.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:46.382+0000 7f7d67c00640 1 -- 172.21.15.131:0/1506231886 wait complete. 2024-05-26T22:49:46.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:46.384+0000 7f7d67c00640 1 Processor -- start 2024-05-26T22:49:46.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:46.384+0000 7f7d67c00640 1 -- start start 2024-05-26T22:49:46.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:46.385+0000 7f7d67c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d6006f490 0x7f7d6014de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:46.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:46.385+0000 7f7d67c00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d60155840 0x7f7d600afe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:46.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:46.385+0000 7f7d67c00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7d6015c3d0 0x7f7d600b0340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:46.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:46.385+0000 7f7d67c00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f7d600ae380 con 0x7f7d6006f490 2024-05-26T22:49:46.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:46.385+0000 7f7d67c00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7d60072e20 con 0x7f7d60155840 2024-05-26T22:49:46.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:46.385+0000 7f7d67c00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f7d600ae500 con 0x7f7d6015c3d0 2024-05-26T22:49:46.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:46.385+0000 7f7d65200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d6006f490 0x7f7d6014de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:46.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:46.385+0000 7f7d65200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d6006f490 0x7f7d6014de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50168/0 (socket says 172.21.15.131:50168) 2024-05-26T22:49:46.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:46.385+0000 7f7d65200640 1 -- 172.21.15.131:0/2646183700 learned_addr learned my addr 172.21.15.131:0/2646183700 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:46.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:46.385+0000 7f7d65c00640 1 --2- 172.21.15.131:0/2646183700 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7d6015c3d0 0x7f7d600b0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:46.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:46.385+0000 7f7d65200640 1 -- 172.21.15.131:0/2646183700 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7d6015c3d0 msgr2=0x7f7d600b0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:46.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:46.385+0000 7f7d65200640 1 --2- 172.21.15.131:0/2646183700 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7d6015c3d0 0x7f7d600b0340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:46.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:46.385+0000 7f7d57e00640 1 --2- 172.21.15.131:0/2646183700 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d60155840 0x7f7d600afe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:46.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:46.386+0000 7f7d65200640 1 -- 172.21.15.131:0/2646183700 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d60155840 msgr2=0x7f7d600afe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:46.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:46.386+0000 7f7d65200640 1 --2- 172.21.15.131:0/2646183700 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d60155840 0x7f7d600afe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:46.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:46.386+0000 7f7d65200640 1 -- 172.21.15.131:0/2646183700 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d6014e7e0 con 0x7f7d6006f490 2024-05-26T22:49:46.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:46.386+0000 7f7d65200640 1 --2- 172.21.15.131:0/2646183700 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d6006f490 0x7f7d6014de70 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f7d5000c440 tx=0x7f7d5000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:46.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:46.386+0000 7f7d5d600640 1 -- 172.21.15.131:0/2646183700 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d50017070 con 0x7f7d6006f490 2024-05-26T22:49:46.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:46.386+0000 7f7d5d600640 1 -- 172.21.15.131:0/2646183700 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d500035f0 con 0x7f7d6006f490 2024-05-26T22:49:46.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:46.386+0000 7f7d5d600640 1 -- 172.21.15.131:0/2646183700 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d500039e0 con 0x7f7d6006f490 2024-05-26T22:49:46.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:46.386+0000 7f7d67c00640 1 -- 172.21.15.131:0/2646183700 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d6014eac0 con 0x7f7d6006f490 2024-05-26T22:49:46.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:46.386+0000 7f7d67c00640 1 -- 172.21.15.131:0/2646183700 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d6014f080 con 0x7f7d6006f490 2024-05-26T22:49:46.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:46.388+0000 7f7d67c00640 1 -- 172.21.15.131:0/2646183700 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d600694e0 con 0x7f7d6006f490 2024-05-26T22:49:46.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:46.388+0000 7f7d5d600640 1 -- 172.21.15.131:0/2646183700 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7d5001a070 con 0x7f7d6006f490 2024-05-26T22:49:46.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:46.389+0000 7f7d5d600640 1 --2- 172.21.15.131:0/2646183700 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f7d3c06be30 0x7f7d3c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:46.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:46.389+0000 7f7d5d600640 1 -- 172.21.15.131:0/2646183700 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f7d5008a110 con 0x7f7d6006f490 2024-05-26T22:49:46.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:46.390+0000 7f7d57e00640 1 --2- 172.21.15.131:0/2646183700 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f7d3c06be30 0x7f7d3c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:46.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:46.390+0000 7f7d57e00640 1 --2- 172.21.15.131:0/2646183700 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f7d3c06be30 0x7f7d3c06e2f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7d48034000 tx=0x7f7d4802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:46.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:46.394+0000 7f7d5d600640 1 -- 172.21.15.131:0/2646183700 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f7d50058b50 con 0x7f7d6006f490 2024-05-26T22:49:46.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:46.631+0000 7f7d67c00640 1 -- 172.21.15.131:0/2646183700 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7d6006c590 con 0x7f7d6006f490 2024-05-26T22:49:46.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:46.632+0000 7f7d5d600640 1 -- 172.21.15.131:0/2646183700 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+5231 (secure 0 0 0) 0x7f7d500010c0 con 0x7f7d6006f490 2024-05-26T22:49:46.632 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:49:46.632 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":20,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/4292497989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":4292497989},{"type":"v1","addr":"172.21.15.187:6835","nonce":4292497989}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-26T22:49:30.955525+0000","modified":"2024-05-26T22:49:46.310449+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5115,"mds_1":4687,"mds_2":4678},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4678":{"gid":4678,"name":"a","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.131:6836/4032294430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":4032294430},{"type":"v1","addr":"172.21.15.131:6836","nonce":4032294430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4687":{"gid":4687,"name":"b","rank":1,"incarnation":17,"state":"up:active","state_seq":7,"addr":"172.21.15.187:6833/3274403209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3274403209},{"type":"v1","addr":"172.21.15.187:6833","nonce":3274403209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5115":{"gid":5115,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.131:6837/2062704413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6835","nonce":2062704413},{"type":"v1","addr":"172.21.15.131:6837","nonce":2062704413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5115,"qdb_cluster":[4678,4687,5115]},"id":2}]} 2024-05-26T22:49:46.633 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 20 2024-05-26T22:49:46.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:46.635+0000 7f7d67c00640 1 -- 172.21.15.131:0/2646183700 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f7d3c06be30 msgr2=0x7f7d3c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:46.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:46.635+0000 7f7d67c00640 1 --2- 172.21.15.131:0/2646183700 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f7d3c06be30 0x7f7d3c06e2f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7d48034000 tx=0x7f7d4802e000 comp rx=0 tx=0).stop 2024-05-26T22:49:46.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:46.635+0000 7f7d67c00640 1 -- 172.21.15.131:0/2646183700 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d6006f490 msgr2=0x7f7d6014de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:46.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:46.635+0000 7f7d67c00640 1 --2- 172.21.15.131:0/2646183700 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d6006f490 0x7f7d6014de70 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f7d5000c440 tx=0x7f7d5000c900 comp rx=0 tx=0).stop 2024-05-26T22:49:46.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:46.635+0000 7f7d67c00640 1 -- 172.21.15.131:0/2646183700 shutdown_connections 2024-05-26T22:49:46.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:46.635+0000 7f7d67c00640 1 --2- 172.21.15.131:0/2646183700 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7d6015c3d0 0x7f7d600b0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:46.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:46.635+0000 7f7d67c00640 1 --2- 172.21.15.131:0/2646183700 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f7d3c06be30 0x7f7d3c06e2f0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:46.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:46.635+0000 7f7d67c00640 1 --2- 172.21.15.131:0/2646183700 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d60155840 0x7f7d600afe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:46.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:46.635+0000 7f7d67c00640 1 --2- 172.21.15.131:0/2646183700 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d6006f490 0x7f7d6014de70 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:46.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:46.635+0000 7f7d67c00640 1 -- 172.21.15.131:0/2646183700 >> 172.21.15.131:0/2646183700 conn(0x7f7d600b1e90 msgr2=0x7f7d600b2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:46.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:46.635+0000 7f7d67c00640 1 -- 172.21.15.131:0/2646183700 shutdown_connections 2024-05-26T22:49:46.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:46.635+0000 7f7d67c00640 1 -- 172.21.15.131:0/2646183700 wait complete. 2024-05-26T22:49:47.056 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 20, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-26T22:49:30.955525+0000', 'modified': '2024-05-26T22:49:46.310449+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 5115, 'mds_1': 4687, 'mds_2': 4678}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4678': {'gid': 4678, 'name': 'a', 'rank': 2, 'incarnation': 19, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.131:6836/4032294430', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6834', 'nonce': 4032294430}, {'type': 'v1', 'addr': '172.21.15.131:6836', 'nonce': 4032294430}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4687': {'gid': 4687, 'name': 'b', 'rank': 1, 'incarnation': 17, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.187:6833/3274403209', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.187:6832', 'nonce': 3274403209}, {'type': 'v1', 'addr': '172.21.15.187:6833', 'nonce': 3274403209}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_5115': {'gid': 5115, 'name': 'c', 'rank': 0, 'incarnation': 12, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.131:6837/2062704413', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6835', 'nonce': 2062704413}, {'type': 'v1', 'addr': '172.21.15.131:6837', 'nonce': 2062704413}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5115, 'qdb_cluster': [4678, 4687, 5115]} 2024-05-26T22:49:47.056 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-05-26T22:49:47.056 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-05-26T22:49:47.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.254+0000 7fbbe5800640 1 Processor -- start 2024-05-26T22:49:47.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.254+0000 7fbbe5800640 1 -- start start 2024-05-26T22:49:47.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.256+0000 7fbbe5800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbbe00b5450 0x7fbbe00b43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:47.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.256+0000 7fbbe5800640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fbbe006f420 0x7fbbe00cfd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:47.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.256+0000 7fbbe5800640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbbe00b4900 con 0x7fbbe00cf590 2024-05-26T22:49:47.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.256+0000 7fbbe5800640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fbbe0160d90 con 0x7fbbe006f420 2024-05-26T22:49:47.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.256+0000 7fbbe5800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fbbe0160180 con 0x7fbbe00b5450 2024-05-26T22:49:47.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.256+0000 7fbbdf400640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fbbe00cf590 0x7fbbe006d600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:56838/0 (socket says 172.21.15.131:56838) 2024-05-26T22:49:47.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.256+0000 7fbbdf400640 1 -- 172.21.15.131:0/2754270205 learned_addr learned my addr 172.21.15.131:0/2754270205 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:49:47.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.256+0000 7fbbdea00640 1 --2- 172.21.15.131:0/2754270205 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbbe00b5450 0x7fbbe00b43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:47.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.256+0000 7fbbde000640 1 --2- 172.21.15.131:0/2754270205 >> v2:172.21.15.187:3300/0 conn(0x7fbbe006f420 0x7fbbe00cfd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:47.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.256+0000 7fbbdd600640 1 -- 172.21.15.131:0/2754270205 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 164910931 0 0) 0x7fbbe00b4900 con 0x7fbbe00cf590 2024-05-26T22:49:47.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.257+0000 7fbbdd600640 1 -- 172.21.15.131:0/2754270205 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbbc0003520 con 0x7fbbe00cf590 2024-05-26T22:49:47.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.257+0000 7fbbdd600640 1 -- 172.21.15.131:0/2754270205 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2916313720 0 0) 0x7fbbc0003520 con 0x7fbbe00cf590 2024-05-26T22:49:47.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.257+0000 7fbbdd600640 1 -- 172.21.15.131:0/2754270205 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbbe00b5450 msgr2=0x7fbbe00b43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:47.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.257+0000 7fbbdd600640 1 --2- 172.21.15.131:0/2754270205 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbbe00b5450 0x7fbbe00b43c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:47.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.257+0000 7fbbdd600640 1 -- 172.21.15.131:0/2754270205 >> v2:172.21.15.187:3300/0 conn(0x7fbbe006f420 msgr2=0x7fbbe00cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:47.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.257+0000 7fbbdd600640 1 --2- 172.21.15.131:0/2754270205 >> v2:172.21.15.187:3300/0 conn(0x7fbbe006f420 0x7fbbe00cfd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:47.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.257+0000 7fbbdd600640 1 -- 172.21.15.131:0/2754270205 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbe0160460 con 0x7fbbe00cf590 2024-05-26T22:49:47.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.257+0000 7fbbdd600640 1 -- 172.21.15.131:0/2754270205 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7fbbd8001670 con 0x7fbbe00cf590 2024-05-26T22:49:47.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.257+0000 7fbbdd600640 1 -- 172.21.15.131:0/2754270205 >> v1:172.21.15.131:6789/0 conn(0x7fbbe00cf590 legacy=0x7fbbe006d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:47.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.257+0000 7fbbdd600640 1 --2- 172.21.15.131:0/2754270205 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbbc00039b0 0x7fbbc0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:47.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.257+0000 7fbbdd600640 1 --2- 172.21.15.131:0/2754270205 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbbc0004800 0x7fbbc0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:47.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.257+0000 7fbbdd600640 1 --2- 172.21.15.131:0/2754270205 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbbe00b5450 0x7fbbc00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:47.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.257+0000 7fbbdd600640 1 -- 172.21.15.131:0/2754270205 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe0160d90 con 0x7fbbc0004800 2024-05-26T22:49:47.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.257+0000 7fbbdd600640 1 -- 172.21.15.131:0/2754270205 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe0160180 con 0x7fbbc00039b0 2024-05-26T22:49:47.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.257+0000 7fbbdd600640 1 -- 172.21.15.131:0/2754270205 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fbbc000ba10 con 0x7fbbe00b5450 2024-05-26T22:49:47.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.257+0000 7fbbde000640 1 --2- 172.21.15.131:0/2754270205 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbbc0004800 0x7fbbc0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:47.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.258+0000 7fbbde000640 1 -- 172.21.15.131:0/2754270205 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbbe00b5450 msgr2=0x7fbbc00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:47.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.258+0000 7fbbde000640 1 --2- 172.21.15.131:0/2754270205 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbbe00b5450 0x7fbbc00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:47.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.258+0000 7fbbde000640 1 -- 172.21.15.131:0/2754270205 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbbc00039b0 msgr2=0x7fbbc0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:47.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.258+0000 7fbbde000640 1 --2- 172.21.15.131:0/2754270205 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbbc00039b0 0x7fbbc0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:47.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.258+0000 7fbbde000640 1 -- 172.21.15.131:0/2754270205 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbc0003520 con 0x7fbbc0004800 2024-05-26T22:49:47.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.259+0000 7fbbde000640 1 --2- 172.21.15.131:0/2754270205 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbbc0004800 0x7fbbc0004c80 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fbbc8007360 tx=0x7fbbc0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3301bcb6a23f051 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:47.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.259+0000 7fbbdd600640 1 -- 172.21.15.131:0/2754270205 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc8009be0 con 0x7fbbc0004800 2024-05-26T22:49:47.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.259+0000 7fbbdd600640 1 -- 172.21.15.131:0/2754270205 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbbc802e620 con 0x7fbbc0004800 2024-05-26T22:49:47.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.259+0000 7fbbdd600640 1 -- 172.21.15.131:0/2754270205 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc802e9f0 con 0x7fbbc0004800 2024-05-26T22:49:47.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.260+0000 7fbbe5800640 1 -- 172.21.15.131:0/2754270205 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbbc0004800 msgr2=0x7fbbc0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:47.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.260+0000 7fbbe5800640 1 --2- 172.21.15.131:0/2754270205 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbbc0004800 0x7fbbc0004c80 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fbbc8007360 tx=0x7fbbc0005830 comp rx=0 tx=0).stop 2024-05-26T22:49:47.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.260+0000 7fbbdf400640 1 -- 172.21.15.131:0/2754270205 reap_dead start 2024-05-26T22:49:47.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.260+0000 7fbbe5800640 1 -- 172.21.15.131:0/2754270205 shutdown_connections 2024-05-26T22:49:47.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.260+0000 7fbbe5800640 1 -- 172.21.15.131:0/2754270205 >> 172.21.15.131:0/2754270205 conn(0x7fbbe006bc90 msgr2=0x7fbbe00b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:47.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.260+0000 7fbbe5800640 1 -- 172.21.15.131:0/2754270205 shutdown_connections 2024-05-26T22:49:47.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.260+0000 7fbbe5800640 1 -- 172.21.15.131:0/2754270205 wait complete. 2024-05-26T22:49:47.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.262+0000 7fbbe5800640 1 Processor -- start 2024-05-26T22:49:47.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.262+0000 7fbbe5800640 1 -- start start 2024-05-26T22:49:47.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.263+0000 7fbbe5800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbbc0004800 0x7fbbe01506b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:47.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.263+0000 7fbbe5800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbbe0156130 0x7fbbe0154ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:47.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.263+0000 7fbbe5800640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbbe01550e0 0x7fbbe0155570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:47.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.263+0000 7fbbe5800640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe00b5830 con 0x7fbbc0004800 2024-05-26T22:49:47.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.263+0000 7fbbe5800640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe0160a20 con 0x7fbbe01550e0 2024-05-26T22:49:47.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.263+0000 7fbbe5800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fbbe015fde0 con 0x7fbbe0156130 2024-05-26T22:49:47.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.263+0000 7fbbdea00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbbc0004800 0x7fbbe01506b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:47.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.263+0000 7fbbdea00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbbc0004800 0x7fbbe01506b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50172/0 (socket says 172.21.15.131:50172) 2024-05-26T22:49:47.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.263+0000 7fbbdea00640 1 -- 172.21.15.131:0/2886345535 learned_addr learned my addr 172.21.15.131:0/2886345535 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:47.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.263+0000 7fbbdea00640 1 -- 172.21.15.131:0/2886345535 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbbe0156130 msgr2=0x7fbbe0154ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:47.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.263+0000 7fbbde000640 1 --2- 172.21.15.131:0/2886345535 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbbe0156130 0x7fbbe0154ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:47.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.263+0000 7fbbdf400640 1 --2- 172.21.15.131:0/2886345535 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbbe01550e0 0x7fbbe0155570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:47.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.263+0000 7fbbdea00640 1 --2- 172.21.15.131:0/2886345535 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbbe0156130 0x7fbbe0154ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:47.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.263+0000 7fbbdea00640 1 -- 172.21.15.131:0/2886345535 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbbe01550e0 msgr2=0x7fbbe0155570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:47.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.263+0000 7fbbdea00640 1 --2- 172.21.15.131:0/2886345535 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbbe01550e0 0x7fbbe0155570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:47.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.263+0000 7fbbdea00640 1 -- 172.21.15.131:0/2886345535 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbe0150bf0 con 0x7fbbc0004800 2024-05-26T22:49:47.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.264+0000 7fbbdea00640 1 --2- 172.21.15.131:0/2886345535 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbbc0004800 0x7fbbe01506b0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fbbd000e7a0 tx=0x7fbbd000ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:47.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.264+0000 7fbbd6a00640 1 -- 172.21.15.131:0/2886345535 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd0017070 con 0x7fbbc0004800 2024-05-26T22:49:47.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.264+0000 7fbbd6a00640 1 -- 172.21.15.131:0/2886345535 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbbd000c940 con 0x7fbbc0004800 2024-05-26T22:49:47.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.264+0000 7fbbe5800640 1 -- 172.21.15.131:0/2886345535 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbe01526c0 con 0x7fbbc0004800 2024-05-26T22:49:47.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.264+0000 7fbbd6a00640 1 -- 172.21.15.131:0/2886345535 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd000cd10 con 0x7fbbc0004800 2024-05-26T22:49:47.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.264+0000 7fbbe5800640 1 -- 172.21.15.131:0/2886345535 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbe0152c80 con 0x7fbbc0004800 2024-05-26T22:49:47.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.266+0000 7fbbd6a00640 1 -- 172.21.15.131:0/2886345535 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbbd001a070 con 0x7fbbc0004800 2024-05-26T22:49:47.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.267+0000 7fbbd6a00640 1 --2- 172.21.15.131:0/2886345535 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fbbbc06be30 0x7fbbbc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:47.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.267+0000 7fbbd6a00640 1 -- 172.21.15.131:0/2886345535 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fbbd008ac80 con 0x7fbbc0004800 2024-05-26T22:49:47.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.267+0000 7fbbde000640 1 --2- 172.21.15.131:0/2886345535 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fbbbc06be30 0x7fbbbc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:47.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.268+0000 7fbbe85ca740 1 Processor -- start 2024-05-26T22:49:47.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.268+0000 7fbbe85ca740 1 -- start start 2024-05-26T22:49:47.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.268+0000 7fbbde000640 1 --2- 172.21.15.131:0/2886345535 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fbbbc06be30 0x7fbbbc06e2f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fbbc8035000 tx=0x7fbbc8038040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:47.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.269+0000 7fbbe85ca740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559dde942f60 0x559dde946d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:47.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.269+0000 7fbbe85ca740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559dde91ac70 0x559dde943340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:47.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.269+0000 7fbbe85ca740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559dde943880 0x559dde946110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:47.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.269+0000 7fbbe85ca740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x559dde87e610 con 0x559dde942f60 2024-05-26T22:49:47.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.269+0000 7fbbe85ca740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x559dde8265c0 con 0x559dde943880 2024-05-26T22:49:47.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.269+0000 7fbbe85ca740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x559dde87a5d0 con 0x559dde91ac70 2024-05-26T22:49:47.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.269+0000 7fbbdea00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559dde942f60 0x559dde946d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:47.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.269+0000 7fbbdea00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559dde942f60 0x559dde946d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50186/0 (socket says 172.21.15.131:50186) 2024-05-26T22:49:47.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.269+0000 7fbbdea00640 1 -- 172.21.15.131:0/3239790021 learned_addr learned my addr 172.21.15.131:0/3239790021 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:47.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.269+0000 7fbbde000640 1 --2- 172.21.15.131:0/3239790021 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559dde91ac70 0x559dde943340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:47.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.269+0000 7fbbdea00640 1 -- 172.21.15.131:0/3239790021 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559dde91ac70 msgr2=0x559dde943340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:47.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.269+0000 7fbbdea00640 1 --2- 172.21.15.131:0/3239790021 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559dde91ac70 0x559dde943340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:47.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.269+0000 7fbbdea00640 1 -- 172.21.15.131:0/3239790021 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559dde943880 msgr2=0x559dde946110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:47.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.269+0000 7fbbdea00640 1 --2- 172.21.15.131:0/3239790021 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559dde943880 0x559dde946110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:47.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.269+0000 7fbbdea00640 1 -- 172.21.15.131:0/3239790021 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559dde5922d0 con 0x559dde942f60 2024-05-26T22:49:47.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.269+0000 7fbbde000640 1 --2- 172.21.15.131:0/3239790021 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559dde91ac70 0x559dde943340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:49:47.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.270+0000 7fbbdea00640 1 --2- 172.21.15.131:0/3239790021 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559dde942f60 0x559dde946d60 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fbbd0058950 tx=0x7fbbd0058980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66171d14569154be server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:47.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.270+0000 7fbbcf400640 1 -- 172.21.15.131:0/3239790021 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd0021040 con 0x559dde942f60 2024-05-26T22:49:47.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.270+0000 7fbbcf400640 1 -- 172.21.15.131:0/3239790021 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbbd0061b00 con 0x559dde942f60 2024-05-26T22:49:47.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.270+0000 7fbbcf400640 1 -- 172.21.15.131:0/3239790021 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd0055590 con 0x559dde942f60 2024-05-26T22:49:47.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.270+0000 7fbbe85ca740 1 -- 172.21.15.131:0/3239790021 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559dde942f60 msgr2=0x559dde946d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:47.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.270+0000 7fbbe85ca740 1 --2- 172.21.15.131:0/3239790021 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559dde942f60 0x559dde946d60 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fbbd0058950 tx=0x7fbbd0058980 comp rx=0 tx=0).stop 2024-05-26T22:49:47.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.270+0000 7fbbe85ca740 1 -- 172.21.15.131:0/3239790021 shutdown_connections 2024-05-26T22:49:47.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.270+0000 7fbbe85ca740 1 --2- 172.21.15.131:0/3239790021 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559dde91ac70 0x559dde943340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:47.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.270+0000 7fbbe85ca740 1 --2- 172.21.15.131:0/3239790021 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559dde943880 0x559dde946110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:47.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.270+0000 7fbbe85ca740 1 --2- 172.21.15.131:0/3239790021 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559dde942f60 0x559dde946d60 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:47.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.270+0000 7fbbe85ca740 1 -- 172.21.15.131:0/3239790021 >> 172.21.15.131:0/3239790021 conn(0x559dde918b00 msgr2=0x559dde961110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:47.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.271+0000 7fbbe85ca740 1 -- 172.21.15.131:0/3239790021 shutdown_connections 2024-05-26T22:49:47.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.271+0000 7fbbe85ca740 1 -- 172.21.15.131:0/3239790021 wait complete. 2024-05-26T22:49:47.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.271+0000 7fbbe85ca740 1 -- start start 2024-05-26T22:49:47.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.271+0000 7fbbe85ca740 1 Processor -- start 2024-05-26T22:49:47.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.272+0000 7fbbe85ca740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559dde942f60 0x559dde961880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:47.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.272+0000 7fbbe85ca740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559dde943880 0x559dde939990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:47.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.272+0000 7fbbe85ca740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559dde962c30 0x559dde949690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:47.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.272+0000 7fbbe85ca740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x559dde916e50 con 0x559dde962c30 2024-05-26T22:49:47.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.272+0000 7fbbe85ca740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x559dde839910 con 0x559dde943880 2024-05-26T22:49:47.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.272+0000 7fbbe85ca740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x559dde916890 con 0x559dde942f60 2024-05-26T22:49:47.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.272+0000 7fbbe85ca740 10 client.0 _ll_register_callbacks cb 0x559dde875410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-26T22:49:47.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.273+0000 7fbbdf400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559dde962c30 0x559dde949690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:47.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.273+0000 7fbbdf400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559dde962c30 0x559dde949690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50196/0 (socket says 172.21.15.131:50196) 2024-05-26T22:49:47.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.273+0000 7fbbdf400640 1 -- 172.21.15.131:0/3472729940 learned_addr learned my addr 172.21.15.131:0/3472729940 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:47.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.273+0000 7fbbdf400640 1 -- 172.21.15.131:0/3472729940 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559dde942f60 msgr2=0x559dde961880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:47.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.273+0000 7fbbdf400640 1 --2- 172.21.15.131:0/3472729940 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559dde942f60 0x559dde961880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:47.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.273+0000 7fbbdf400640 1 -- 172.21.15.131:0/3472729940 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559dde943880 msgr2=0x559dde939990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:47.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.273+0000 7fbbdf400640 1 --2- 172.21.15.131:0/3472729940 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559dde943880 0x559dde939990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:47.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.273+0000 7fbbdf400640 1 -- 172.21.15.131:0/3472729940 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559dde65c6b0 con 0x559dde962c30 2024-05-26T22:49:47.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.273+0000 7fbbdf400640 1 --2- 172.21.15.131:0/3472729940 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559dde962c30 0x559dde949690 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fbbe00cf970 tx=0x7fbbd800d5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ebd610a6b68265f server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:47.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.273+0000 7fbbc7e00640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-26T22:49:47.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.274+0000 7fbbc7e00640 1 -- 172.21.15.131:0/3472729940 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd80072d0 con 0x559dde962c30 2024-05-26T22:49:47.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.274+0000 7fbbc7e00640 1 -- 172.21.15.131:0/3472729940 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fbbd8007470 con 0x559dde962c30 2024-05-26T22:49:47.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.274+0000 7fbbe85ca740 1 -- 172.21.15.131:0/3472729940 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbbd0061b00 con 0x559dde962c30 2024-05-26T22:49:47.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.274+0000 7fbbc7e00640 1 -- 172.21.15.131:0/3472729940 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fbbd8007670 con 0x559dde962c30 2024-05-26T22:49:47.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.274+0000 7fbbe85ca740 10 client.5361 fetch_fsmap learned FSMap version 20 2024-05-26T22:49:47.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.274+0000 7fbbe85ca740 1 -- 172.21.15.131:0/3472729940 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=2+}) -- 0x559dde9639d0 con 0x559dde962c30 2024-05-26T22:49:47.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.274+0000 7fbbc7e00640 1 -- 172.21.15.131:0/3472729940 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbbd0061b00 con 0x559dde962c30 2024-05-26T22:49:47.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.274+0000 7fbbc7e00640 1 -- 172.21.15.131:0/3472729940 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 20) ==== 3534+0+0 (secure 0 0 0) 0x7fbbd80047e0 con 0x559dde962c30 2024-05-26T22:49:47.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.274+0000 7fbbe85ca740 10 client.5361 fetch_fsmap finished waiting for FSMap version 20 2024-05-26T22:49:47.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.274+0000 7fbbe85ca740 10 client.5361 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-26T22:49:47.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.274+0000 7fbbe85ca740 20 client.5361 populate_metadata read hostname 'smithi131' 2024-05-26T22:49:47.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.274+0000 7fbbe85ca740 1 --2- 172.21.15.131:0/3472729940 >> [v2:172.21.15.131:6834/4032294430,v1:172.21.15.131:6836/4032294430] conn(0x559dde9793b0 0x559dde941480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:47.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.274+0000 7fbbe85ca740 4 client.5361 mds_command: new command op to 4678 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-26T22:49:47.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.274+0000 7fbbe85ca740 1 -- 172.21.15.131:0/3472729940 --> [v2:172.21.15.131:6834/4032294430,v1:172.21.15.131:6836/4032294430] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559dde88bd40 con 0x559dde9793b0 2024-05-26T22:49:47.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.275+0000 7fbbdea00640 1 --2- 172.21.15.131:0/3472729940 >> [v2:172.21.15.131:6834/4032294430,v1:172.21.15.131:6836/4032294430] conn(0x559dde9793b0 0x559dde941480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:47.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.275+0000 7fbbdea00640 1 --2- 172.21.15.131:0/3472729940 >> [v2:172.21.15.131:6834/4032294430,v1:172.21.15.131:6836/4032294430] conn(0x559dde9793b0 0x559dde941480 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=995c1ac6f655fb60 server_cookie=bc31053dfbdd0c26 in_seq=0 out_seq=0 2024-05-26T22:49:47.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.275+0000 7fbbc7e00640 10 client.5361 ms_handle_connect on v2:172.21.15.131:6834/4032294430 2024-05-26T22:49:47.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.276+0000 7fbbc7e00640 1 -- 172.21.15.131:0/3472729940 <== mds.2 v2:172.21.15.131:6834/4032294430 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7fbbd0077c70 con 0x559dde9793b0 2024-05-26T22:49:47.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.276+0000 7fbbc7e00640 10 client.5361 handle_command_reply: tid=0 2024-05-26T22:49:47.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.277+0000 7fbbe85ca740 1 client.5361 shutdown 2024-05-26T22:49:47.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.277+0000 7fbbe85ca740 1 -- 172.21.15.131:0/3472729940 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559dde962c30 msgr2=0x559dde949690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:47.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.277+0000 7fbbe85ca740 1 --2- 172.21.15.131:0/3472729940 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559dde962c30 0x559dde949690 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fbbe00cf970 tx=0x7fbbd800d5e0 comp rx=0 tx=0).stop 2024-05-26T22:49:47.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.277+0000 7fbbe85ca740 1 -- 172.21.15.131:0/3472729940 shutdown_connections 2024-05-26T22:49:47.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.277+0000 7fbbe85ca740 1 --2- 172.21.15.131:0/3472729940 >> [v2:172.21.15.131:6834/4032294430,v1:172.21.15.131:6836/4032294430] conn(0x559dde9793b0 0x559dde941480 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:47.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.277+0000 7fbbe85ca740 1 --2- 172.21.15.131:0/3472729940 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559dde962c30 0x559dde949690 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:47.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.277+0000 7fbbe85ca740 1 --2- 172.21.15.131:0/3472729940 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559dde943880 0x559dde939990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:47.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.277+0000 7fbbc7e00640 0 client.5361 ms_handle_reset on v2:172.21.15.131:6834/4032294430 2024-05-26T22:49:47.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.277+0000 7fbbe85ca740 1 --2- 172.21.15.131:0/3472729940 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559dde942f60 0x559dde961880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:47.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.277+0000 7fbbe85ca740 1 -- 172.21.15.131:0/3472729940 >> 172.21.15.131:0/3472729940 conn(0x559dde91ac70 msgr2=0x559dde91c3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:47.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.277+0000 7fbbe85ca740 1 -- 172.21.15.131:0/3472729940 shutdown_connections 2024-05-26T22:49:47.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.277+0000 7fbbe85ca740 1 -- 172.21.15.131:0/3472729940 wait complete. 2024-05-26T22:49:47.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.277+0000 7fbbe85ca740 20 client.5361 trim_cache size 0 max 16384 2024-05-26T22:49:47.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.297+0000 7fbbe85ca740 1 Processor -- start 2024-05-26T22:49:47.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.297+0000 7fbbe85ca740 1 -- start start 2024-05-26T22:49:47.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.298+0000 7fbbe85ca740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559dde942f60 0x559dde977e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:47.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.298+0000 7fbbe85ca740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559dde943880 0x559dde9783c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:47.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.298+0000 7fbbe85ca740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559dde962c30 0x559dde915920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:47.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.298+0000 7fbbe85ca740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe0072a20 con 0x559dde942f60 2024-05-26T22:49:47.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.298+0000 7fbbe85ca740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe00730f0 con 0x559dde943880 2024-05-26T22:49:47.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.298+0000 7fbbe85ca740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fbbe0157500 con 0x559dde962c30 2024-05-26T22:49:47.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.298+0000 7fbbdea00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559dde942f60 0x559dde977e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:47.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.299+0000 7fbbdea00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559dde942f60 0x559dde977e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50208/0 (socket says 172.21.15.131:50208) 2024-05-26T22:49:47.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.299+0000 7fbbdea00640 1 -- 172.21.15.131:0/3202202155 learned_addr learned my addr 172.21.15.131:0/3202202155 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:47.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.299+0000 7fbbdea00640 1 -- 172.21.15.131:0/3202202155 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559dde962c30 msgr2=0x559dde915920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:47.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.299+0000 7fbbdea00640 1 --2- 172.21.15.131:0/3202202155 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559dde962c30 0x559dde915920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:47.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.299+0000 7fbbdea00640 1 -- 172.21.15.131:0/3202202155 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559dde943880 msgr2=0x559dde9783c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:47.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.299+0000 7fbbdea00640 1 --2- 172.21.15.131:0/3202202155 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559dde943880 0x559dde9783c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:47.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.299+0000 7fbbdea00640 1 -- 172.21.15.131:0/3202202155 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559dde9436b0 con 0x559dde942f60 2024-05-26T22:49:47.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.299+0000 7fbbdea00640 1 --2- 172.21.15.131:0/3202202155 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559dde942f60 0x559dde977e80 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fbbd00590e0 tx=0x7fbbd001e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c94802c2083df46 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:47.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.299+0000 7fbbcea00640 1 -- 172.21.15.131:0/3202202155 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd0021040 con 0x559dde942f60 2024-05-26T22:49:47.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.299+0000 7fbbcea00640 1 -- 172.21.15.131:0/3202202155 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbbd0072070 con 0x559dde942f60 2024-05-26T22:49:47.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.299+0000 7fbbcea00640 1 -- 172.21.15.131:0/3202202155 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd0058370 con 0x559dde942f60 2024-05-26T22:49:47.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.300+0000 7fbbe85ca740 1 -- 172.21.15.131:0/3202202155 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559dde942f60 msgr2=0x559dde977e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:47.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.300+0000 7fbbe85ca740 1 --2- 172.21.15.131:0/3202202155 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559dde942f60 0x559dde977e80 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fbbd00590e0 tx=0x7fbbd001e070 comp rx=0 tx=0).stop 2024-05-26T22:49:47.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.300+0000 7fbbe85ca740 1 -- 172.21.15.131:0/3202202155 shutdown_connections 2024-05-26T22:49:47.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.300+0000 7fbbe85ca740 1 --2- 172.21.15.131:0/3202202155 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559dde962c30 0x559dde915920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:47.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.300+0000 7fbbe85ca740 1 --2- 172.21.15.131:0/3202202155 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559dde943880 0x559dde9783c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:47.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.300+0000 7fbbe85ca740 1 --2- 172.21.15.131:0/3202202155 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559dde942f60 0x559dde977e80 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:47.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.300+0000 7fbbe85ca740 1 -- 172.21.15.131:0/3202202155 >> 172.21.15.131:0/3202202155 conn(0x559dde91ac70 msgr2=0x559dde941450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:47.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.300+0000 7fbbe85ca740 1 -- 172.21.15.131:0/3202202155 shutdown_connections 2024-05-26T22:49:47.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.300+0000 7fbbe85ca740 1 -- 172.21.15.131:0/3202202155 wait complete. 2024-05-26T22:49:47.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.301+0000 7fbbe85ca740 1 -- start start 2024-05-26T22:49:47.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.301+0000 7fbbe85ca740 1 Processor -- start 2024-05-26T22:49:47.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.302+0000 7fbbe85ca740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559dde942f60 0x559dde93db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:47.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.302+0000 7fbbe85ca740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559dde943880 0x559dde9e33e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:47.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.302+0000 7fbbe85ca740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559dde962c30 0x559dde964520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:47.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.302+0000 7fbbe85ca740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x559dde979230 con 0x559dde962c30 2024-05-26T22:49:47.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.302+0000 7fbbe85ca740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x559dde940710 con 0x559dde943880 2024-05-26T22:49:47.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.302+0000 7fbbe85ca740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x559dde979880 con 0x559dde942f60 2024-05-26T22:49:47.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.302+0000 7fbbe85ca740 10 client.0 _ll_register_callbacks cb 0x7fbbd80016d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-26T22:49:47.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.302+0000 7fbbdf400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559dde962c30 0x559dde964520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:47.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.302+0000 7fbbdf400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559dde962c30 0x559dde964520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50222/0 (socket says 172.21.15.131:50222) 2024-05-26T22:49:47.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.302+0000 7fbbdf400640 1 -- 172.21.15.131:0/1039232246 learned_addr learned my addr 172.21.15.131:0/1039232246 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:47.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.302+0000 7fbbdea00640 1 --2- 172.21.15.131:0/1039232246 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559dde942f60 0x559dde93db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:47.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.303+0000 7fbbde000640 1 --2- 172.21.15.131:0/1039232246 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559dde943880 0x559dde9e33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:47.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.303+0000 7fbbdf400640 1 -- 172.21.15.131:0/1039232246 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559dde942f60 msgr2=0x559dde93db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:47.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.303+0000 7fbbdf400640 1 --2- 172.21.15.131:0/1039232246 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559dde942f60 0x559dde93db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:47.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.303+0000 7fbbdf400640 1 -- 172.21.15.131:0/1039232246 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559dde943880 msgr2=0x559dde9e33e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:47.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.303+0000 7fbbdf400640 1 --2- 172.21.15.131:0/1039232246 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559dde943880 0x559dde9e33e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:47.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.303+0000 7fbbdf400640 1 -- 172.21.15.131:0/1039232246 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559dde957f10 con 0x559dde962c30 2024-05-26T22:49:47.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.303+0000 7fbbdf400640 1 --2- 172.21.15.131:0/1039232246 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559dde962c30 0x559dde964520 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fbbd8000c00 tx=0x7fbbd0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93263f869b9292dc server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:47.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.303+0000 7fbbc7e00640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-26T22:49:47.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.303+0000 7fbbc7e00640 1 -- 172.21.15.131:0/1039232246 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd8023070 con 0x559dde962c30 2024-05-26T22:49:47.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.303+0000 7fbbc7e00640 1 -- 172.21.15.131:0/1039232246 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fbbd80248c0 con 0x559dde962c30 2024-05-26T22:49:47.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.303+0000 7fbbe85ca740 1 -- 172.21.15.131:0/1039232246 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbbd0072070 con 0x559dde962c30 2024-05-26T22:49:47.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.303+0000 7fbbc7e00640 1 -- 172.21.15.131:0/1039232246 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fbbd802f050 con 0x559dde962c30 2024-05-26T22:49:47.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.303+0000 7fbbe85ca740 10 client.5367 fetch_fsmap learned FSMap version 20 2024-05-26T22:49:47.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.303+0000 7fbbe85ca740 1 -- 172.21.15.131:0/1039232246 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=2+}) -- 0x559dde957c30 con 0x559dde962c30 2024-05-26T22:49:47.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.304+0000 7fbbc7e00640 1 -- 172.21.15.131:0/1039232246 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbbd0072070 con 0x559dde962c30 2024-05-26T22:49:47.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.304+0000 7fbbc7e00640 1 -- 172.21.15.131:0/1039232246 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 20) ==== 3534+0+0 (secure 0 0 0) 0x7fbbd8032020 con 0x559dde962c30 2024-05-26T22:49:47.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.304+0000 7fbbe85ca740 10 client.5367 fetch_fsmap finished waiting for FSMap version 20 2024-05-26T22:49:47.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.304+0000 7fbbe85ca740 10 client.5367 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-26T22:49:47.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.304+0000 7fbbe85ca740 20 client.5367 populate_metadata read hostname 'smithi131' 2024-05-26T22:49:47.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.304+0000 7fbbe85ca740 1 --2- 172.21.15.131:0/1039232246 >> [v2:172.21.15.131:6834/4032294430,v1:172.21.15.131:6836/4032294430] conn(0x559dde91a460 0x559dde960140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:47.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.304+0000 7fbbe85ca740 4 client.5367 mds_command: new command op to 4678 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-26T22:49:47.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.304+0000 7fbbe85ca740 1 -- 172.21.15.131:0/1039232246 --> [v2:172.21.15.131:6834/4032294430,v1:172.21.15.131:6836/4032294430] -- command(tid 0: {"prefix": "status"}) -- 0x559dde886090 con 0x559dde91a460 2024-05-26T22:49:47.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.304+0000 7fbbdea00640 1 --2- 172.21.15.131:0/1039232246 >> [v2:172.21.15.131:6834/4032294430,v1:172.21.15.131:6836/4032294430] conn(0x559dde91a460 0x559dde960140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:47.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.305+0000 7fbbc7e00640 10 client.5367 ms_handle_connect on v2:172.21.15.131:6834/4032294430 2024-05-26T22:49:47.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.304+0000 7fbbdea00640 1 --2- 172.21.15.131:0/1039232246 >> [v2:172.21.15.131:6834/4032294430,v1:172.21.15.131:6836/4032294430] conn(0x559dde91a460 0x559dde960140 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c94f791f5bf78097 server_cookie=85769d16c6ae4072 in_seq=0 out_seq=0 2024-05-26T22:49:47.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.305+0000 7fbbc7e00640 1 -- 172.21.15.131:0/1039232246 <== mds.2 v2:172.21.15.131:6834/4032294430 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x559dde886090 con 0x559dde91a460 2024-05-26T22:49:47.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.305+0000 7fbbc7e00640 10 client.5367 handle_command_reply: tid=0 2024-05-26T22:49:47.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.305+0000 7fbbe85ca740 1 client.5367 shutdown 2024-05-26T22:49:47.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.305+0000 7fbbe85ca740 1 -- 172.21.15.131:0/1039232246 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559dde962c30 msgr2=0x559dde964520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:47.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.305+0000 7fbbe85ca740 1 --2- 172.21.15.131:0/1039232246 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559dde962c30 0x559dde964520 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fbbd8000c00 tx=0x7fbbd0004730 comp rx=0 tx=0).stop 2024-05-26T22:49:47.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.305+0000 7fbbe85ca740 1 -- 172.21.15.131:0/1039232246 shutdown_connections 2024-05-26T22:49:47.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.305+0000 7fbbe85ca740 1 --2- 172.21.15.131:0/1039232246 >> [v2:172.21.15.131:6834/4032294430,v1:172.21.15.131:6836/4032294430] conn(0x559dde91a460 0x559dde960140 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:47.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.305+0000 7fbbe85ca740 1 --2- 172.21.15.131:0/1039232246 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559dde962c30 0x559dde964520 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:47.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.305+0000 7fbbe85ca740 1 --2- 172.21.15.131:0/1039232246 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559dde943880 0x559dde9e33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:47.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.305+0000 7fbbe85ca740 1 --2- 172.21.15.131:0/1039232246 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559dde942f60 0x559dde93db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:47.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.305+0000 7fbbe85ca740 1 -- 172.21.15.131:0/1039232246 >> 172.21.15.131:0/1039232246 conn(0x559dde91ac70 msgr2=0x559dde93a760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:47.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.305+0000 7fbbc7e00640 0 client.5367 ms_handle_reset on v2:172.21.15.131:6834/4032294430 2024-05-26T22:49:47.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.305+0000 7fbbe85ca740 1 -- 172.21.15.131:0/1039232246 shutdown_connections 2024-05-26T22:49:47.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.305+0000 7fbbe85ca740 1 -- 172.21.15.131:0/1039232246 wait complete. 2024-05-26T22:49:47.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.305+0000 7fbbe85ca740 20 client.5367 trim_cache size 0 max 16384 2024-05-26T22:49:47.309 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-26T22:49:47.309 INFO:teuthology.orchestra.run.smithi131.stdout: "cluster_fsid": "46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f", 2024-05-26T22:49:47.309 INFO:teuthology.orchestra.run.smithi131.stdout: "whoami": 2, 2024-05-26T22:49:47.309 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 4678, 2024-05-26T22:49:47.310 INFO:teuthology.orchestra.run.smithi131.stdout: "want_state": "up:active", 2024-05-26T22:49:47.310 INFO:teuthology.orchestra.run.smithi131.stdout: "state": "up:active", 2024-05-26T22:49:47.310 INFO:teuthology.orchestra.run.smithi131.stdout: "fs_name": "cephfs", 2024-05-26T22:49:47.310 INFO:teuthology.orchestra.run.smithi131.stdout: "rank_uptime": 1.9924464820000001, 2024-05-26T22:49:47.310 INFO:teuthology.orchestra.run.smithi131.stdout: "mdsmap_epoch": 20, 2024-05-26T22:49:47.310 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch": 33, 2024-05-26T22:49:47.310 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch_barrier": 33, 2024-05-26T22:49:47.310 INFO:teuthology.orchestra.run.smithi131.stdout: "uptime": 24.621072069 2024-05-26T22:49:47.310 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-26T22:49:47.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.306+0000 7fbbe5800640 1 -- 172.21.15.131:0/2886345535 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fbbbc06be30 msgr2=0x7fbbbc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:47.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.306+0000 7fbbe5800640 1 --2- 172.21.15.131:0/2886345535 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fbbbc06be30 0x7fbbbc06e2f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fbbc8035000 tx=0x7fbbc8038040 comp rx=0 tx=0).stop 2024-05-26T22:49:47.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.306+0000 7fbbe5800640 1 -- 172.21.15.131:0/2886345535 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbbc0004800 msgr2=0x7fbbe01506b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:47.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.306+0000 7fbbe5800640 1 --2- 172.21.15.131:0/2886345535 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbbc0004800 0x7fbbe01506b0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fbbd000e7a0 tx=0x7fbbd000ec60 comp rx=0 tx=0).stop 2024-05-26T22:49:47.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.306+0000 7fbbe5800640 1 -- 172.21.15.131:0/2886345535 shutdown_connections 2024-05-26T22:49:47.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.306+0000 7fbbe5800640 1 --2- 172.21.15.131:0/2886345535 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbbe0156130 0x7fbbe0154ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:47.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.306+0000 7fbbe5800640 1 --2- 172.21.15.131:0/2886345535 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fbbbc06be30 0x7fbbbc06e2f0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:47.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.306+0000 7fbbe5800640 1 --2- 172.21.15.131:0/2886345535 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbbe01550e0 0x7fbbe0155570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:47.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.306+0000 7fbbe5800640 1 --2- 172.21.15.131:0/2886345535 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbbc0004800 0x7fbbe01506b0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:47.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.306+0000 7fbbe5800640 1 -- 172.21.15.131:0/2886345535 >> 172.21.15.131:0/2886345535 conn(0x7fbbe006bc90 msgr2=0x7fbbe00b43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:47.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.306+0000 7fbbe5800640 1 -- 172.21.15.131:0/2886345535 shutdown_connections 2024-05-26T22:49:47.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.306+0000 7fbbe5800640 1 -- 172.21.15.131:0/2886345535 wait complete. 2024-05-26T22:49:47.322 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-05-26T22:49:47.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.497+0000 7fa2fbe00640 1 Processor -- start 2024-05-26T22:49:47.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.498+0000 7fa2fbe00640 1 -- start start 2024-05-26T22:49:47.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.499+0000 7fa2fbe00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fa2fc0712d0 0x7fa2fc06d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:47.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.499+0000 7fa2fbe00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2fc072d60 con 0x7fa2fc145a10 2024-05-26T22:49:47.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.499+0000 7fa2fbe00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2fc072a80 con 0x7fa2fc06d790 2024-05-26T22:49:47.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.499+0000 7fa2fbe00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fa2fc15c760 con 0x7fa2fc0712d0 2024-05-26T22:49:47.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.499+0000 7fa2faa00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fa2fc145a10 0x7fa2fc072180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:56854/0 (socket says 172.21.15.131:56854) 2024-05-26T22:49:47.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.499+0000 7fa2faa00640 1 -- 172.21.15.131:0/4036242885 learned_addr learned my addr 172.21.15.131:0/4036242885 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:49:47.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.500+0000 7fa2fa000640 1 -- 172.21.15.131:0/4036242885 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1156075610 0 0) 0x7fa2fc072d60 con 0x7fa2fc145a10 2024-05-26T22:49:47.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.500+0000 7fa2fa000640 1 -- 172.21.15.131:0/4036242885 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa2e0003520 con 0x7fa2fc145a10 2024-05-26T22:49:47.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.500+0000 7fa2f2000640 1 --2- 172.21.15.131:0/4036242885 >> v2:172.21.15.187:3300/0 conn(0x7fa2fc0712d0 0x7fa2fc06d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:47.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.500+0000 7fa2fa000640 1 -- 172.21.15.131:0/4036242885 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3758690064 0 0) 0x7fa2e0003520 con 0x7fa2fc145a10 2024-05-26T22:49:47.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.500+0000 7fa2fa000640 1 -- 172.21.15.131:0/4036242885 >> v2:172.21.15.187:3300/0 conn(0x7fa2fc0712d0 msgr2=0x7fa2fc06d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:47.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.500+0000 7fa2fa000640 1 --2- 172.21.15.131:0/4036242885 >> v2:172.21.15.187:3300/0 conn(0x7fa2fc0712d0 0x7fa2fc06d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:47.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.500+0000 7fa2fa000640 1 -- 172.21.15.131:0/4036242885 >> v1:172.21.15.187:6789/0 conn(0x7fa2fc06d790 legacy=0x7fa2fc0cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:47.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.500+0000 7fa2fa000640 1 -- 172.21.15.131:0/4036242885 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2fc0ceab0 con 0x7fa2fc145a10 2024-05-26T22:49:47.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.500+0000 7fa2fa000640 1 -- 172.21.15.131:0/4036242885 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7fa2ec0019f0 con 0x7fa2fc145a10 2024-05-26T22:49:47.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.500+0000 7fa2fa000640 1 -- 172.21.15.131:0/4036242885 >> v1:172.21.15.131:6789/0 conn(0x7fa2fc145a10 legacy=0x7fa2fc072180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:47.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.500+0000 7fa2fa000640 1 --2- 172.21.15.131:0/4036242885 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa2e00039b0 0x7fa2e0003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:47.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.500+0000 7fa2fa000640 1 --2- 172.21.15.131:0/4036242885 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa2e0004800 0x7fa2e0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:47.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.500+0000 7fa2fa000640 1 --2- 172.21.15.131:0/4036242885 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa2e00051c0 0x7fa2e000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:47.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.500+0000 7fa2fa000640 1 -- 172.21.15.131:0/4036242885 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa2fc15c760 con 0x7fa2e0004800 2024-05-26T22:49:47.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.500+0000 7fa2fa000640 1 -- 172.21.15.131:0/4036242885 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e000c0f0 con 0x7fa2e00051c0 2024-05-26T22:49:47.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.500+0000 7fa2fa000640 1 -- 172.21.15.131:0/4036242885 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa2e000c2f0 con 0x7fa2e00039b0 2024-05-26T22:49:47.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.500+0000 7fa2f2000640 1 --2- 172.21.15.131:0/4036242885 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa2e0004800 0x7fa2e0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:47.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.501+0000 7fa2f2000640 1 -- 172.21.15.131:0/4036242885 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa2e00039b0 msgr2=0x7fa2e0003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:47.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.501+0000 7fa2f2000640 1 --2- 172.21.15.131:0/4036242885 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa2e00039b0 0x7fa2e0003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:47.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.501+0000 7fa2f2000640 1 -- 172.21.15.131:0/4036242885 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa2e00051c0 msgr2=0x7fa2e000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:47.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.501+0000 7fa2f2000640 1 --2- 172.21.15.131:0/4036242885 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa2e00051c0 0x7fa2e000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:47.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.501+0000 7fa2f2000640 1 -- 172.21.15.131:0/4036242885 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2e0003520 con 0x7fa2e0004800 2024-05-26T22:49:47.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.501+0000 7fa2fb400640 1 -- 172.21.15.131:0/4036242885 reap_dead start 2024-05-26T22:49:47.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.502+0000 7fa2f2000640 1 --2- 172.21.15.131:0/4036242885 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa2e0004800 0x7fa2e0004c80 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fa2fc071740 tx=0x7fa2e0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45933bfa7441d3e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:47.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.502+0000 7fa2fa000640 1 -- 172.21.15.131:0/4036242885 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2e4030070 con 0x7fa2e0004800 2024-05-26T22:49:47.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.502+0000 7fa2fa000640 1 -- 172.21.15.131:0/4036242885 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2e402adb0 con 0x7fa2e0004800 2024-05-26T22:49:47.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.502+0000 7fa2fa000640 1 -- 172.21.15.131:0/4036242885 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2e400b600 con 0x7fa2e0004800 2024-05-26T22:49:47.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.502+0000 7fa2fbe00640 1 -- 172.21.15.131:0/4036242885 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa2e0004800 msgr2=0x7fa2e0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:47.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.502+0000 7fa2fbe00640 1 --2- 172.21.15.131:0/4036242885 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa2e0004800 0x7fa2e0004c80 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fa2fc071740 tx=0x7fa2e0005830 comp rx=0 tx=0).stop 2024-05-26T22:49:47.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.503+0000 7fa2fbe00640 1 -- 172.21.15.131:0/4036242885 shutdown_connections 2024-05-26T22:49:47.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.503+0000 7fa2fbe00640 1 --2- 172.21.15.131:0/4036242885 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa2e0004800 0x7fa2e0004c80 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:47.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.503+0000 7fa2fbe00640 1 -- 172.21.15.131:0/4036242885 >> 172.21.15.131:0/4036242885 conn(0x7fa2fc0b5660 msgr2=0x7fa2fc0b5a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:47.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.503+0000 7fa2fbe00640 1 -- 172.21.15.131:0/4036242885 shutdown_connections 2024-05-26T22:49:47.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.503+0000 7fa2fbe00640 1 -- 172.21.15.131:0/4036242885 wait complete. 2024-05-26T22:49:47.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.504+0000 7fa2fbe00640 1 Processor -- start 2024-05-26T22:49:47.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.504+0000 7fa2fbe00640 1 -- start start 2024-05-26T22:49:47.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.505+0000 7fa2fbe00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa2e0004800 0x7fa2fc0b4b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:47.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.505+0000 7fa2fbe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa2fc0aed80 0x7fa2fc1f7550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:47.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.505+0000 7fa2fbe00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa2fc1f7a90 0x7fa2fc14d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:47.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.505+0000 7fa2fbe00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa2fc0cf070 con 0x7fa2fc0aed80 2024-05-26T22:49:47.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.505+0000 7fa2fbe00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa2fc06dbc0 con 0x7fa2e0004800 2024-05-26T22:49:47.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.505+0000 7fa2fbe00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa2fc145df0 con 0x7fa2fc1f7a90 2024-05-26T22:49:47.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.506+0000 7fa2f2000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa2fc0aed80 0x7fa2fc1f7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:47.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.506+0000 7fa2f2000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa2fc0aed80 0x7fa2fc1f7550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50250/0 (socket says 172.21.15.131:50250) 2024-05-26T22:49:47.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.506+0000 7fa2f2000640 1 -- 172.21.15.131:0/1648563319 learned_addr learned my addr 172.21.15.131:0/1648563319 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:47.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.506+0000 7fa2f2000640 1 -- 172.21.15.131:0/1648563319 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa2fc1f7a90 msgr2=0x7fa2fc14d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:47.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.506+0000 7fa2f2000640 1 --2- 172.21.15.131:0/1648563319 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa2fc1f7a90 0x7fa2fc14d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:47.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.506+0000 7fa2f2000640 1 -- 172.21.15.131:0/1648563319 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa2e0004800 msgr2=0x7fa2fc0b4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:47.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.506+0000 7fa2f2000640 1 --2- 172.21.15.131:0/1648563319 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa2e0004800 0x7fa2fc0b4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:47.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.506+0000 7fa2f2000640 1 -- 172.21.15.131:0/1648563319 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2fc0b36f0 con 0x7fa2fc0aed80 2024-05-26T22:49:47.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.506+0000 7fa2f2000640 1 --2- 172.21.15.131:0/1648563319 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa2fc0aed80 0x7fa2fc1f7550 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fa2e400baf0 tx=0x7fa2e4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:47.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.507+0000 7fa2f3400640 1 -- 172.21.15.131:0/1648563319 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2e4043020 con 0x7fa2fc0aed80 2024-05-26T22:49:47.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.507+0000 7fa2f3400640 1 -- 172.21.15.131:0/1648563319 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2e403f680 con 0x7fa2fc0aed80 2024-05-26T22:49:47.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.507+0000 7fa2fbe00640 1 -- 172.21.15.131:0/1648563319 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2fc0b39d0 con 0x7fa2fc0aed80 2024-05-26T22:49:47.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.507+0000 7fa2f3400640 1 -- 172.21.15.131:0/1648563319 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2e400b600 con 0x7fa2fc0aed80 2024-05-26T22:49:47.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.507+0000 7fa2fbe00640 1 -- 172.21.15.131:0/1648563319 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2fc0b3f90 con 0x7fa2fc0aed80 2024-05-26T22:49:47.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.509+0000 7fa2f3400640 1 -- 172.21.15.131:0/1648563319 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa2e4046020 con 0x7fa2fc0aed80 2024-05-26T22:49:47.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.510+0000 7fa2f3400640 1 --2- 172.21.15.131:0/1648563319 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa2d806be30 0x7fa2d806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:47.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.510+0000 7fa2f3400640 1 -- 172.21.15.131:0/1648563319 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fa2e40abeb0 con 0x7fa2fc0aed80 2024-05-26T22:49:47.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.510+0000 7fa2faa00640 1 --2- 172.21.15.131:0/1648563319 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa2d806be30 0x7fa2d806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:47.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.511+0000 7fa304384740 1 Processor -- start 2024-05-26T22:49:47.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.511+0000 7fa2faa00640 1 --2- 172.21.15.131:0/1648563319 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa2d806be30 0x7fa2d806e2f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec007fd0 tx=0x7fa2ec0018a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:47.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.511+0000 7fa304384740 1 -- start start 2024-05-26T22:49:47.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.512+0000 7fa304384740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560d14d8ae50 0x560d14d3da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:47.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.512+0000 7fa304384740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560d14d6df70 0x560d14d6d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:47.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.512+0000 7fa304384740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560d14d6d830 0x560d14d42640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:47.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.512+0000 7fa304384740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x560d14ca5610 con 0x560d14d8ae50 2024-05-26T22:49:47.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.512+0000 7fa304384740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x560d14c4d5c0 con 0x560d14d6d830 2024-05-26T22:49:47.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.512+0000 7fa304384740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x560d14ca15d0 con 0x560d14d6df70 2024-05-26T22:49:47.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.512+0000 7fa2faa00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560d14d8ae50 0x560d14d3da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:47.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.512+0000 7fa2faa00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560d14d8ae50 0x560d14d3da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50260/0 (socket says 172.21.15.131:50260) 2024-05-26T22:49:47.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.512+0000 7fa2faa00640 1 -- 172.21.15.131:0/3297097299 learned_addr learned my addr 172.21.15.131:0/3297097299 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:47.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.512+0000 7fa2faa00640 1 -- 172.21.15.131:0/3297097299 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560d14d6df70 msgr2=0x560d14d6d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:47.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.512+0000 7fa2faa00640 1 --2- 172.21.15.131:0/3297097299 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560d14d6df70 0x560d14d6d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:47.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.512+0000 7fa2faa00640 1 -- 172.21.15.131:0/3297097299 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560d14d6d830 msgr2=0x560d14d42640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:47.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.512+0000 7fa2faa00640 1 --2- 172.21.15.131:0/3297097299 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560d14d6d830 0x560d14d42640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:47.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.512+0000 7fa2faa00640 1 -- 172.21.15.131:0/3297097299 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d149b92d0 con 0x560d14d8ae50 2024-05-26T22:49:47.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.513+0000 7fa2faa00640 1 --2- 172.21.15.131:0/3297097299 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560d14d8ae50 0x560d14d3da70 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec005e70 tx=0x7fa2ec00f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a3b12a44f9ffacc server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:47.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.513+0000 7fa2eb000640 1 -- 172.21.15.131:0/3297097299 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2ec0077e0 con 0x560d14d8ae50 2024-05-26T22:49:47.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.513+0000 7fa2eb000640 1 -- 172.21.15.131:0/3297097299 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2ec007980 con 0x560d14d8ae50 2024-05-26T22:49:47.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.513+0000 7fa2eb000640 1 -- 172.21.15.131:0/3297097299 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2ec012660 con 0x560d14d8ae50 2024-05-26T22:49:47.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.513+0000 7fa304384740 1 -- 172.21.15.131:0/3297097299 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560d14d8ae50 msgr2=0x560d14d3da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:47.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.513+0000 7fa304384740 1 --2- 172.21.15.131:0/3297097299 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560d14d8ae50 0x560d14d3da70 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec005e70 tx=0x7fa2ec00f000 comp rx=0 tx=0).stop 2024-05-26T22:49:47.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.514+0000 7fa304384740 1 -- 172.21.15.131:0/3297097299 shutdown_connections 2024-05-26T22:49:47.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.514+0000 7fa304384740 1 --2- 172.21.15.131:0/3297097299 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560d14d6df70 0x560d14d6d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:47.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.514+0000 7fa304384740 1 --2- 172.21.15.131:0/3297097299 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560d14d6d830 0x560d14d42640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:47.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.514+0000 7fa304384740 1 --2- 172.21.15.131:0/3297097299 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560d14d8ae50 0x560d14d3da70 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:47.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.514+0000 7fa304384740 1 -- 172.21.15.131:0/3297097299 >> 172.21.15.131:0/3297097299 conn(0x560d14d3fce0 msgr2=0x560d14d882f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:47.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.514+0000 7fa304384740 1 -- 172.21.15.131:0/3297097299 shutdown_connections 2024-05-26T22:49:47.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.514+0000 7fa304384740 1 -- 172.21.15.131:0/3297097299 wait complete. 2024-05-26T22:49:47.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.514+0000 7fa304384740 1 -- start start 2024-05-26T22:49:47.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.515+0000 7fa304384740 1 Processor -- start 2024-05-26T22:49:47.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.515+0000 7fa304384740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560d14d6df70 0x560d14d6f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:47.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.515+0000 7fa304384740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560d14d8ae50 0x560d14d57d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:47.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.515+0000 7fa304384740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560d14d70700 0x560d14d89920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:47.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.515+0000 7fa304384740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x560d14d41c00 con 0x560d14d70700 2024-05-26T22:49:47.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.515+0000 7fa304384740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x560d14c60910 con 0x560d14d8ae50 2024-05-26T22:49:47.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.515+0000 7fa304384740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x560d14d40ff0 con 0x560d14d6df70 2024-05-26T22:49:47.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.516+0000 7fa304384740 10 client.0 _ll_register_callbacks cb 0x560d14c9c410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-26T22:49:47.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.516+0000 7fa2fb400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560d14d70700 0x560d14d89920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:47.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.516+0000 7fa2fb400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560d14d70700 0x560d14d89920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50266/0 (socket says 172.21.15.131:50266) 2024-05-26T22:49:47.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.516+0000 7fa2fb400640 1 -- 172.21.15.131:0/3732292714 learned_addr learned my addr 172.21.15.131:0/3732292714 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:47.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.516+0000 7fa2faa00640 1 --2- 172.21.15.131:0/3732292714 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560d14d6df70 0x560d14d6f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:47.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.516+0000 7fa2fb400640 1 -- 172.21.15.131:0/3732292714 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560d14d6df70 msgr2=0x560d14d6f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:47.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.516+0000 7fa2fb400640 1 --2- 172.21.15.131:0/3732292714 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560d14d6df70 0x560d14d6f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:47.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.516+0000 7fa2fb400640 1 -- 172.21.15.131:0/3732292714 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560d14d8ae50 msgr2=0x560d14d57d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:47.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.516+0000 7fa2fb400640 1 --2- 172.21.15.131:0/3732292714 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560d14d8ae50 0x560d14d57d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:47.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.516+0000 7fa2fb400640 1 -- 172.21.15.131:0/3732292714 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d14a836b0 con 0x560d14d70700 2024-05-26T22:49:47.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.516+0000 7fa2fb400640 1 --2- 172.21.15.131:0/3732292714 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560d14d70700 0x560d14d89920 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fa2fc0d0c60 tx=0x7fa2f400c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2e7fbc64ceb230 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:47.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.516+0000 7fa2df400640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-26T22:49:47.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.517+0000 7fa2df400640 1 -- 172.21.15.131:0/3732292714 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f400cd40 con 0x560d14d70700 2024-05-26T22:49:47.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.517+0000 7fa2df400640 1 -- 172.21.15.131:0/3732292714 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fa2f4014ac0 con 0x560d14d70700 2024-05-26T22:49:47.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.517+0000 7fa304384740 1 -- 172.21.15.131:0/3732292714 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2ec007980 con 0x560d14d70700 2024-05-26T22:49:47.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.517+0000 7fa2df400640 1 -- 172.21.15.131:0/3732292714 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fa2f4007070 con 0x560d14d70700 2024-05-26T22:49:47.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.517+0000 7fa304384740 10 client.5379 fetch_fsmap learned FSMap version 20 2024-05-26T22:49:47.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.517+0000 7fa304384740 1 -- 172.21.15.131:0/3732292714 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=2+}) -- 0x560d14d7f730 con 0x560d14d70700 2024-05-26T22:49:47.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.517+0000 7fa2df400640 1 -- 172.21.15.131:0/3732292714 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2ec007980 con 0x560d14d70700 2024-05-26T22:49:47.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.517+0000 7fa2df400640 1 -- 172.21.15.131:0/3732292714 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 20) ==== 3534+0+0 (secure 0 0 0) 0x7fa2f4007210 con 0x560d14d70700 2024-05-26T22:49:47.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.517+0000 7fa304384740 10 client.5379 fetch_fsmap finished waiting for FSMap version 20 2024-05-26T22:49:47.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.517+0000 7fa304384740 10 client.5379 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-26T22:49:47.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.517+0000 7fa304384740 20 client.5379 populate_metadata read hostname 'smithi131' 2024-05-26T22:49:47.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.517+0000 7fa304384740 1 --2- 172.21.15.131:0/3732292714 >> [v2:172.21.15.187:6832/3274403209,v1:172.21.15.187:6833/3274403209] conn(0x560d14d9c460 0x560d14d80480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:47.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.517+0000 7fa304384740 4 client.5379 mds_command: new command op to 4687 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-26T22:49:47.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.517+0000 7fa304384740 1 -- 172.21.15.131:0/3732292714 --> [v2:172.21.15.187:6832/3274403209,v1:172.21.15.187:6833/3274403209] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560d14cb2d40 con 0x560d14d9c460 2024-05-26T22:49:47.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.518+0000 7fa2faa00640 1 --2- 172.21.15.131:0/3732292714 >> [v2:172.21.15.187:6832/3274403209,v1:172.21.15.187:6833/3274403209] conn(0x560d14d9c460 0x560d14d80480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:47.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.519+0000 7fa2faa00640 1 --2- 172.21.15.131:0/3732292714 >> [v2:172.21.15.187:6832/3274403209,v1:172.21.15.187:6833/3274403209] conn(0x560d14d9c460 0x560d14d80480 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=373d4f1ed73677ec server_cookie=91d09eca6791e726 in_seq=0 out_seq=0 2024-05-26T22:49:47.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.519+0000 7fa2df400640 10 client.5379 ms_handle_connect on v2:172.21.15.187:6832/3274403209 2024-05-26T22:49:47.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.521+0000 7fa2df400640 1 -- 172.21.15.131:0/3732292714 <== mds.1 v2:172.21.15.187:6832/3274403209 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560d14cb2d40 con 0x560d14d9c460 2024-05-26T22:49:47.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.521+0000 7fa2df400640 10 client.5379 handle_command_reply: tid=0 2024-05-26T22:49:47.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.521+0000 7fa304384740 1 client.5379 shutdown 2024-05-26T22:49:47.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.522+0000 7fa304384740 1 -- 172.21.15.131:0/3732292714 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560d14d70700 msgr2=0x560d14d89920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:47.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.522+0000 7fa304384740 1 --2- 172.21.15.131:0/3732292714 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560d14d70700 0x560d14d89920 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fa2fc0d0c60 tx=0x7fa2f400c4b0 comp rx=0 tx=0).stop 2024-05-26T22:49:47.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.522+0000 7fa304384740 1 -- 172.21.15.131:0/3732292714 shutdown_connections 2024-05-26T22:49:47.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.522+0000 7fa304384740 1 --2- 172.21.15.131:0/3732292714 >> [v2:172.21.15.187:6832/3274403209,v1:172.21.15.187:6833/3274403209] conn(0x560d14d9c460 0x560d14d80480 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:47.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.522+0000 7fa304384740 1 --2- 172.21.15.131:0/3732292714 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560d14d70700 0x560d14d89920 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:47.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.522+0000 7fa304384740 1 --2- 172.21.15.131:0/3732292714 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560d14d8ae50 0x560d14d57d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:47.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.522+0000 7fa304384740 1 --2- 172.21.15.131:0/3732292714 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560d14d6df70 0x560d14d6f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:47.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.522+0000 7fa2df400640 0 client.5379 ms_handle_reset on v2:172.21.15.187:6832/3274403209 2024-05-26T22:49:47.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.522+0000 7fa304384740 1 -- 172.21.15.131:0/3732292714 >> 172.21.15.131:0/3732292714 conn(0x560d14d6d830 msgr2=0x560d14d3e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:47.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.522+0000 7fa304384740 1 -- 172.21.15.131:0/3732292714 shutdown_connections 2024-05-26T22:49:47.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.522+0000 7fa304384740 1 -- 172.21.15.131:0/3732292714 wait complete. 2024-05-26T22:49:47.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.522+0000 7fa304384740 20 client.5379 trim_cache size 0 max 16384 2024-05-26T22:49:47.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.544+0000 7fa304384740 1 Processor -- start 2024-05-26T22:49:47.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.544+0000 7fa304384740 1 -- start start 2024-05-26T22:49:47.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.545+0000 7fa304384740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560d14d6df70 0x560d14dac3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:47.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.545+0000 7fa304384740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560d14d70700 0x560d14d86770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:47.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.545+0000 7fa304384740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560d14d8ae50 0x560d14d86cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:47.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.545+0000 7fa304384740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa2fc2000e0 con 0x560d14d6df70 2024-05-26T22:49:47.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.545+0000 7fa304384740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa2fc200790 con 0x560d14d8ae50 2024-05-26T22:49:47.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.545+0000 7fa304384740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa2fc14c5a0 con 0x560d14d70700 2024-05-26T22:49:47.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.545+0000 7fa2faa00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560d14d6df70 0x560d14dac3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:47.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.545+0000 7fa2faa00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560d14d6df70 0x560d14dac3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50274/0 (socket says 172.21.15.131:50274) 2024-05-26T22:49:47.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.545+0000 7fa2faa00640 1 -- 172.21.15.131:0/1333332084 learned_addr learned my addr 172.21.15.131:0/1333332084 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:47.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.546+0000 7fa2f2000640 1 --2- 172.21.15.131:0/1333332084 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560d14d70700 0x560d14d86770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:47.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.546+0000 7fa2fb400640 1 --2- 172.21.15.131:0/1333332084 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560d14d8ae50 0x560d14d86cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:47.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.546+0000 7fa2faa00640 1 -- 172.21.15.131:0/1333332084 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560d14d70700 msgr2=0x560d14d86770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:47.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.546+0000 7fa2faa00640 1 --2- 172.21.15.131:0/1333332084 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560d14d70700 0x560d14d86770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:47.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.546+0000 7fa2faa00640 1 -- 172.21.15.131:0/1333332084 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560d14d8ae50 msgr2=0x560d14d86cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:47.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.546+0000 7fa2faa00640 1 --2- 172.21.15.131:0/1333332084 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560d14d8ae50 0x560d14d86cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:47.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.546+0000 7fa2faa00640 1 -- 172.21.15.131:0/1333332084 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d14d39a00 con 0x560d14d6df70 2024-05-26T22:49:47.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.546+0000 7fa2faa00640 1 --2- 172.21.15.131:0/1333332084 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560d14d6df70 0x560d14dac3a0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec0074a0 tx=0x7fa2ec007000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=841f6f954731d323 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:47.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.546+0000 7fa2ea600640 1 -- 172.21.15.131:0/1333332084 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2ec030070 con 0x560d14d6df70 2024-05-26T22:49:47.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.546+0000 7fa2ea600640 1 -- 172.21.15.131:0/1333332084 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2ec007860 con 0x560d14d6df70 2024-05-26T22:49:47.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.546+0000 7fa2ea600640 1 -- 172.21.15.131:0/1333332084 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2ec02b5b0 con 0x560d14d6df70 2024-05-26T22:49:47.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.547+0000 7fa304384740 1 -- 172.21.15.131:0/1333332084 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560d14d6df70 msgr2=0x560d14dac3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:47.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.547+0000 7fa304384740 1 --2- 172.21.15.131:0/1333332084 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560d14d6df70 0x560d14dac3a0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec0074a0 tx=0x7fa2ec007000 comp rx=0 tx=0).stop 2024-05-26T22:49:47.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.547+0000 7fa304384740 1 -- 172.21.15.131:0/1333332084 shutdown_connections 2024-05-26T22:49:47.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.547+0000 7fa304384740 1 --2- 172.21.15.131:0/1333332084 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560d14d70700 0x560d14d86770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:47.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.547+0000 7fa304384740 1 --2- 172.21.15.131:0/1333332084 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560d14d8ae50 0x560d14d86cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:47.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.547+0000 7fa304384740 1 --2- 172.21.15.131:0/1333332084 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560d14d6df70 0x560d14dac3a0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:47.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.547+0000 7fa304384740 1 -- 172.21.15.131:0/1333332084 >> 172.21.15.131:0/1333332084 conn(0x560d14d6d830 msgr2=0x560d14d3e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:47.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.547+0000 7fa304384740 1 -- 172.21.15.131:0/1333332084 shutdown_connections 2024-05-26T22:49:47.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.547+0000 7fa304384740 1 -- 172.21.15.131:0/1333332084 wait complete. 2024-05-26T22:49:47.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.548+0000 7fa304384740 1 -- start start 2024-05-26T22:49:47.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.548+0000 7fa304384740 1 Processor -- start 2024-05-26T22:49:47.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.549+0000 7fa304384740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560d14d6df70 0x560d14d86b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:47.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.549+0000 7fa304384740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560d14d70700 0x560d14dab1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:47.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.550+0000 7fa304384740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560d14d8ae50 0x560d14d3cee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:47.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.550+0000 7fa304384740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x560d14d9c2e0 con 0x560d14d6df70 2024-05-26T22:49:47.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.550+0000 7fa304384740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x560d14d80cd0 con 0x560d14d70700 2024-05-26T22:49:47.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.550+0000 7fa304384740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x560d14d9c900 con 0x560d14d8ae50 2024-05-26T22:49:47.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.550+0000 7fa304384740 10 client.0 _ll_register_callbacks cb 0x7fa2f4007b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-26T22:49:47.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.550+0000 7fa2faa00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560d14d6df70 0x560d14d86b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:47.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.550+0000 7fa2faa00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560d14d6df70 0x560d14d86b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50284/0 (socket says 172.21.15.131:50284) 2024-05-26T22:49:47.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.550+0000 7fa2faa00640 1 -- 172.21.15.131:0/361184822 learned_addr learned my addr 172.21.15.131:0/361184822 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:47.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.550+0000 7fa2faa00640 1 -- 172.21.15.131:0/361184822 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560d14d8ae50 msgr2=0x560d14d3cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:47.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.550+0000 7fa2faa00640 1 --2- 172.21.15.131:0/361184822 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560d14d8ae50 0x560d14d3cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:47.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.550+0000 7fa2faa00640 1 -- 172.21.15.131:0/361184822 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560d14d70700 msgr2=0x560d14dab1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:47.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.550+0000 7fa2faa00640 1 --2- 172.21.15.131:0/361184822 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560d14d70700 0x560d14dab1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:47.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.550+0000 7fa2faa00640 1 -- 172.21.15.131:0/361184822 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d14d870b0 con 0x560d14d6df70 2024-05-26T22:49:47.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.550+0000 7fa2faa00640 1 --2- 172.21.15.131:0/361184822 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560d14d6df70 0x560d14d86b70 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec007040 tx=0x7fa2ec0012c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da391f514f8d3264 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:47.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.550+0000 7fa2df400640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-26T22:49:47.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.551+0000 7fa2df400640 1 -- 172.21.15.131:0/361184822 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2ec030040 con 0x560d14d6df70 2024-05-26T22:49:47.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.551+0000 7fa2df400640 1 -- 172.21.15.131:0/361184822 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fa2ec02b3b0 con 0x560d14d6df70 2024-05-26T22:49:47.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.551+0000 7fa304384740 1 -- 172.21.15.131:0/361184822 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2ec007860 con 0x560d14d6df70 2024-05-26T22:49:47.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.551+0000 7fa2df400640 1 -- 172.21.15.131:0/361184822 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fa2ec041050 con 0x560d14d6df70 2024-05-26T22:49:47.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.551+0000 7fa304384740 10 client.5385 fetch_fsmap learned FSMap version 20 2024-05-26T22:49:47.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.551+0000 7fa304384740 1 -- 172.21.15.131:0/361184822 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=2+}) -- 0x560d14d7e6e0 con 0x560d14d6df70 2024-05-26T22:49:47.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.551+0000 7fa2df400640 1 -- 172.21.15.131:0/361184822 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2ec007860 con 0x560d14d6df70 2024-05-26T22:49:47.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.552+0000 7fa2df400640 1 -- 172.21.15.131:0/361184822 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 20) ==== 3534+0+0 (secure 0 0 0) 0x7fa2ec044020 con 0x560d14d6df70 2024-05-26T22:49:47.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.552+0000 7fa304384740 10 client.5385 fetch_fsmap finished waiting for FSMap version 20 2024-05-26T22:49:47.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.552+0000 7fa304384740 10 client.5385 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-26T22:49:47.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.552+0000 7fa304384740 20 client.5385 populate_metadata read hostname 'smithi131' 2024-05-26T22:49:47.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.552+0000 7fa304384740 1 --2- 172.21.15.131:0/361184822 >> [v2:172.21.15.187:6832/3274403209,v1:172.21.15.187:6833/3274403209] conn(0x560d14dec430 0x560d14dec810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:47.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.552+0000 7fa304384740 4 client.5385 mds_command: new command op to 4687 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-26T22:49:47.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.552+0000 7fa304384740 1 -- 172.21.15.131:0/361184822 --> [v2:172.21.15.187:6832/3274403209,v1:172.21.15.187:6833/3274403209] -- command(tid 0: {"prefix": "status"}) -- 0x560d14cad090 con 0x560d14dec430 2024-05-26T22:49:47.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.552+0000 7fa2f2000640 1 --2- 172.21.15.131:0/361184822 >> [v2:172.21.15.187:6832/3274403209,v1:172.21.15.187:6833/3274403209] conn(0x560d14dec430 0x560d14dec810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:47.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.553+0000 7fa2f2000640 1 --2- 172.21.15.131:0/361184822 >> [v2:172.21.15.187:6832/3274403209,v1:172.21.15.187:6833/3274403209] conn(0x560d14dec430 0x560d14dec810 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1142d1352e5eebad server_cookie=d8935ddc83a465e1 in_seq=0 out_seq=0 2024-05-26T22:49:47.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.553+0000 7fa2df400640 10 client.5385 ms_handle_connect on v2:172.21.15.187:6832/3274403209 2024-05-26T22:49:47.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.554+0000 7fa2df400640 1 -- 172.21.15.131:0/361184822 <== mds.1 v2:172.21.15.187:6832/3274403209 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x560d14cad090 con 0x560d14dec430 2024-05-26T22:49:47.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.554+0000 7fa2df400640 10 client.5385 handle_command_reply: tid=0 2024-05-26T22:49:47.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.554+0000 7fa304384740 1 client.5385 shutdown 2024-05-26T22:49:47.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.555+0000 7fa304384740 1 -- 172.21.15.131:0/361184822 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560d14d6df70 msgr2=0x560d14d86b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:47.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.555+0000 7fa304384740 1 --2- 172.21.15.131:0/361184822 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560d14d6df70 0x560d14d86b70 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec007040 tx=0x7fa2ec0012c0 comp rx=0 tx=0).stop 2024-05-26T22:49:47.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.555+0000 7fa304384740 1 -- 172.21.15.131:0/361184822 shutdown_connections 2024-05-26T22:49:47.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.555+0000 7fa304384740 1 --2- 172.21.15.131:0/361184822 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560d14d8ae50 0x560d14d3cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:47.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.555+0000 7fa304384740 1 --2- 172.21.15.131:0/361184822 >> [v2:172.21.15.187:6832/3274403209,v1:172.21.15.187:6833/3274403209] conn(0x560d14dec430 0x560d14dec810 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:47.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.555+0000 7fa304384740 1 --2- 172.21.15.131:0/361184822 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560d14d70700 0x560d14dab1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:47.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.555+0000 7fa304384740 1 --2- 172.21.15.131:0/361184822 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560d14d6df70 0x560d14d86b70 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:47.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.555+0000 7fa304384740 1 -- 172.21.15.131:0/361184822 >> 172.21.15.131:0/361184822 conn(0x560d14d6d830 msgr2=0x560d14d88f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:47.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.555+0000 7fa2df400640 0 client.5385 ms_handle_reset on v2:172.21.15.187:6832/3274403209 2024-05-26T22:49:47.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.555+0000 7fa304384740 1 -- 172.21.15.131:0/361184822 shutdown_connections 2024-05-26T22:49:47.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.555+0000 7fa304384740 1 -- 172.21.15.131:0/361184822 wait complete. 2024-05-26T22:49:47.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.555+0000 7fa304384740 20 client.5385 trim_cache size 0 max 16384 2024-05-26T22:49:47.557 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-26T22:49:47.557 INFO:teuthology.orchestra.run.smithi131.stdout: "cluster_fsid": "46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f", 2024-05-26T22:49:47.557 INFO:teuthology.orchestra.run.smithi131.stdout: "whoami": 1, 2024-05-26T22:49:47.557 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 4687, 2024-05-26T22:49:47.557 INFO:teuthology.orchestra.run.smithi131.stdout: "want_state": "up:active", 2024-05-26T22:49:47.557 INFO:teuthology.orchestra.run.smithi131.stdout: "state": "up:active", 2024-05-26T22:49:47.557 INFO:teuthology.orchestra.run.smithi131.stdout: "fs_name": "cephfs", 2024-05-26T22:49:47.557 INFO:teuthology.orchestra.run.smithi131.stdout: "rank_uptime": 3.2625185509999999, 2024-05-26T22:49:47.557 INFO:teuthology.orchestra.run.smithi131.stdout: "mdsmap_epoch": 20, 2024-05-26T22:49:47.557 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch": 33, 2024-05-26T22:49:47.557 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch_barrier": 33, 2024-05-26T22:49:47.557 INFO:teuthology.orchestra.run.smithi131.stdout: "uptime": 24.866796975 2024-05-26T22:49:47.557 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-26T22:49:47.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.556+0000 7fa2fbe00640 1 -- 172.21.15.131:0/1648563319 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa2d806be30 msgr2=0x7fa2d806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:47.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.556+0000 7fa2fbe00640 1 --2- 172.21.15.131:0/1648563319 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa2d806be30 0x7fa2d806e2f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec007fd0 tx=0x7fa2ec0018a0 comp rx=0 tx=0).stop 2024-05-26T22:49:47.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.556+0000 7fa2fbe00640 1 -- 172.21.15.131:0/1648563319 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa2fc0aed80 msgr2=0x7fa2fc1f7550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:47.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.556+0000 7fa2fbe00640 1 --2- 172.21.15.131:0/1648563319 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa2fc0aed80 0x7fa2fc1f7550 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fa2e400baf0 tx=0x7fa2e4004730 comp rx=0 tx=0).stop 2024-05-26T22:49:47.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.556+0000 7fa2fbe00640 1 -- 172.21.15.131:0/1648563319 shutdown_connections 2024-05-26T22:49:47.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.556+0000 7fa2fbe00640 1 --2- 172.21.15.131:0/1648563319 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa2fc1f7a90 0x7fa2fc14d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:47.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.556+0000 7fa2fbe00640 1 --2- 172.21.15.131:0/1648563319 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa2d806be30 0x7fa2d806e2f0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:47.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.556+0000 7fa2fbe00640 1 --2- 172.21.15.131:0/1648563319 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa2fc0aed80 0x7fa2fc1f7550 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:47.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.556+0000 7fa2fbe00640 1 --2- 172.21.15.131:0/1648563319 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa2e0004800 0x7fa2fc0b4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:47.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.556+0000 7fa2fbe00640 1 -- 172.21.15.131:0/1648563319 >> 172.21.15.131:0/1648563319 conn(0x7fa2fc0b5660 msgr2=0x7fa2fc0afb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:47.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.556+0000 7fa2fbe00640 1 -- 172.21.15.131:0/1648563319 shutdown_connections 2024-05-26T22:49:47.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.556+0000 7fa2fbe00640 1 -- 172.21.15.131:0/1648563319 wait complete. 2024-05-26T22:49:47.575 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-05-26T22:49:47.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.766+0000 7fa98be00640 1 Processor -- start 2024-05-26T22:49:47.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.767+0000 7fa98be00640 1 -- start start 2024-05-26T22:49:47.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.769+0000 7fa98be00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fa98c0d0550 0x7fa98c0ae250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:47.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.769+0000 7fa98be00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa98c0b2440 con 0x7fa98c0b2f60 2024-05-26T22:49:47.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.769+0000 7fa98be00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa98c162230 con 0x7fa98c15e320 2024-05-26T22:49:47.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.769+0000 7fa98be00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fa98c161fe0 con 0x7fa98c0d0550 2024-05-26T22:49:47.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.769+0000 7fa982000640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fa98c0d0550 0x7fa98c0ae250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:47.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.769+0000 7fa98b400640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fa98c0b2f60 0x7fa98c0715d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:56862/0 (socket says 172.21.15.131:56862) 2024-05-26T22:49:47.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.769+0000 7fa98b400640 1 -- 172.21.15.131:0/250649849 learned_addr learned my addr 172.21.15.131:0/250649849 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:49:47.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.769+0000 7fa982000640 1 -- 172.21.15.131:0/250649849 >> v1:172.21.15.187:6789/0 conn(0x7fa98c15e320 legacy=0x7fa98c0cfe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:47.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.769+0000 7fa982000640 1 -- 172.21.15.131:0/250649849 >> v1:172.21.15.131:6789/0 conn(0x7fa98c0b2f60 legacy=0x7fa98c0715d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:47.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.769+0000 7fa982000640 1 -- 172.21.15.131:0/250649849 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa98c162510 con 0x7fa98c0d0550 2024-05-26T22:49:47.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.769+0000 7fa98a000640 1 -- 172.21.15.131:0/250649849 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4135742823 0 0) 0x7fa98c0b2440 con 0x7fa98c0b2f60 2024-05-26T22:49:47.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.770+0000 7fa982000640 1 --2- 172.21.15.131:0/250649849 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa98c0d0550 0x7fa98c0ae250 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fa9740097a0 tx=0x7fa97402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a63ddd11b605db9 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:47.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.770+0000 7fa98a000640 1 -- 172.21.15.131:0/250649849 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa974037070 con 0x7fa98c0d0550 2024-05-26T22:49:47.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.771+0000 7fa98a000640 1 -- 172.21.15.131:0/250649849 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa974032440 con 0x7fa98c0d0550 2024-05-26T22:49:47.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.771+0000 7fa98a000640 1 -- 172.21.15.131:0/250649849 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa974032810 con 0x7fa98c0d0550 2024-05-26T22:49:47.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.771+0000 7fa98be00640 1 -- 172.21.15.131:0/250649849 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa98c0d0550 msgr2=0x7fa98c0ae250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:47.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.771+0000 7fa98be00640 1 --2- 172.21.15.131:0/250649849 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa98c0d0550 0x7fa98c0ae250 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fa9740097a0 tx=0x7fa97402a270 comp rx=0 tx=0).stop 2024-05-26T22:49:47.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.771+0000 7fa98be00640 1 -- 172.21.15.131:0/250649849 shutdown_connections 2024-05-26T22:49:47.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.771+0000 7fa98be00640 1 --2- 172.21.15.131:0/250649849 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa98c0d0550 0x7fa98c0ae250 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:47.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.771+0000 7fa98be00640 1 -- 172.21.15.131:0/250649849 >> 172.21.15.131:0/250649849 conn(0x7fa98c0b5660 msgr2=0x7fa98c0b5a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:47.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.771+0000 7fa98be00640 1 -- 172.21.15.131:0/250649849 shutdown_connections 2024-05-26T22:49:47.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.771+0000 7fa98be00640 1 -- 172.21.15.131:0/250649849 wait complete. 2024-05-26T22:49:47.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.773+0000 7fa98be00640 1 Processor -- start 2024-05-26T22:49:47.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.773+0000 7fa98be00640 1 -- start start 2024-05-26T22:49:47.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.774+0000 7fa98be00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa98c0b2f60 0x7fa98c160160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:47.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.774+0000 7fa98be00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa98c0d0550 0x7fa98c0c5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:47.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.774+0000 7fa98be00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa98c15e320 0x7fa98c15cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:47.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.774+0000 7fa98be00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa98c162e40 con 0x7fa98c0d0550 2024-05-26T22:49:47.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.774+0000 7fa98be00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa98c162ad0 con 0x7fa98c15e320 2024-05-26T22:49:47.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.774+0000 7fa98be00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa98c0b3570 con 0x7fa98c0b2f60 2024-05-26T22:49:47.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.774+0000 7fa982000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa98c0d0550 0x7fa98c0c5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:47.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.774+0000 7fa982000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa98c0d0550 0x7fa98c0c5100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50302/0 (socket says 172.21.15.131:50302) 2024-05-26T22:49:47.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.774+0000 7fa982000640 1 -- 172.21.15.131:0/1168172898 learned_addr learned my addr 172.21.15.131:0/1168172898 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:47.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.775+0000 7fa98b400640 1 --2- 172.21.15.131:0/1168172898 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa98c15e320 0x7fa98c15cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:47.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.775+0000 7fa98aa00640 1 --2- 172.21.15.131:0/1168172898 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa98c0b2f60 0x7fa98c160160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:47.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.775+0000 7fa982000640 1 -- 172.21.15.131:0/1168172898 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa98c0b2f60 msgr2=0x7fa98c160160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:47.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.775+0000 7fa982000640 1 --2- 172.21.15.131:0/1168172898 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa98c0b2f60 0x7fa98c160160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:47.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.775+0000 7fa982000640 1 -- 172.21.15.131:0/1168172898 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa98c15e320 msgr2=0x7fa98c15cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:47.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.775+0000 7fa982000640 1 --2- 172.21.15.131:0/1168172898 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa98c15e320 0x7fa98c15cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:47.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.775+0000 7fa982000640 1 -- 172.21.15.131:0/1168172898 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa98c160740 con 0x7fa98c0d0550 2024-05-26T22:49:47.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.775+0000 7fa98b400640 1 --2- 172.21.15.131:0/1168172898 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa98c15e320 0x7fa98c15cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:49:47.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.775+0000 7fa982000640 1 --2- 172.21.15.131:0/1168172898 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa98c0d0550 0x7fa98c0c5100 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fa9740097a0 tx=0x7fa9740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:47.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.775+0000 7fa983e00640 1 -- 172.21.15.131:0/1168172898 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa974041070 con 0x7fa98c0d0550 2024-05-26T22:49:47.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.776+0000 7fa983e00640 1 -- 172.21.15.131:0/1168172898 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa97403c870 con 0x7fa98c0d0550 2024-05-26T22:49:47.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.776+0000 7fa98be00640 1 -- 172.21.15.131:0/1168172898 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa98c160a20 con 0x7fa98c0d0550 2024-05-26T22:49:47.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.776+0000 7fa983e00640 1 -- 172.21.15.131:0/1168172898 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa974037040 con 0x7fa98c0d0550 2024-05-26T22:49:47.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.776+0000 7fa98be00640 1 -- 172.21.15.131:0/1168172898 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa98c160fe0 con 0x7fa98c0d0550 2024-05-26T22:49:47.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.778+0000 7fa983e00640 1 -- 172.21.15.131:0/1168172898 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa974044070 con 0x7fa98c0d0550 2024-05-26T22:49:47.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.779+0000 7fa983e00640 1 --2- 172.21.15.131:0/1168172898 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa96806be30 0x7fa96806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:47.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.779+0000 7fa983e00640 1 -- 172.21.15.131:0/1168172898 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fa9740abbc0 con 0x7fa98c0d0550 2024-05-26T22:49:47.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.779+0000 7fa98aa00640 1 --2- 172.21.15.131:0/1168172898 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa96806be30 0x7fa96806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:47.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.779+0000 7fa994d87740 1 Processor -- start 2024-05-26T22:49:47.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.779+0000 7fa994d87740 1 -- start start 2024-05-26T22:49:47.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.779+0000 7fa98aa00640 1 --2- 172.21.15.131:0/1168172898 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa96806be30 0x7fa96806e2f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa97c003d90 tx=0x7fa97c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:47.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.780+0000 7fa994d87740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564e2d647e50 0x564e2d5faa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:47.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.780+0000 7fa994d87740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564e2d62af70 0x564e2d62a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:47.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.780+0000 7fa994d87740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x564e2d62a830 0x564e2d5ff640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:47.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.780+0000 7fa994d87740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x564e2d562610 con 0x564e2d647e50 2024-05-26T22:49:47.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.780+0000 7fa994d87740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x564e2d50a5c0 con 0x564e2d62af70 2024-05-26T22:49:47.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.780+0000 7fa994d87740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x564e2d55e5d0 con 0x564e2d62a830 2024-05-26T22:49:47.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.780+0000 7fa98aa00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564e2d647e50 0x564e2d5faa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:47.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.781+0000 7fa98aa00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564e2d647e50 0x564e2d5faa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50312/0 (socket says 172.21.15.131:50312) 2024-05-26T22:49:47.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.781+0000 7fa98aa00640 1 -- 172.21.15.131:0/1063254182 learned_addr learned my addr 172.21.15.131:0/1063254182 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:47.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.781+0000 7fa98aa00640 1 -- 172.21.15.131:0/1063254182 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x564e2d62a830 msgr2=0x564e2d5ff640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:47.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.781+0000 7fa98aa00640 1 --2- 172.21.15.131:0/1063254182 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x564e2d62a830 0x564e2d5ff640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:47.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.781+0000 7fa98aa00640 1 -- 172.21.15.131:0/1063254182 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564e2d62af70 msgr2=0x564e2d62a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:47.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.781+0000 7fa98aa00640 1 --2- 172.21.15.131:0/1063254182 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564e2d62af70 0x564e2d62a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:47.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.781+0000 7fa98aa00640 1 -- 172.21.15.131:0/1063254182 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e2d2762d0 con 0x564e2d647e50 2024-05-26T22:49:47.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.781+0000 7fa98aa00640 1 --2- 172.21.15.131:0/1063254182 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564e2d647e50 0x564e2d5faa70 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fa97c00d8e0 tx=0x7fa97c010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=932da87f686e55de server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:47.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.781+0000 7fa97b400640 1 -- 172.21.15.131:0/1063254182 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa97c024070 con 0x564e2d647e50 2024-05-26T22:49:47.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.781+0000 7fa97b400640 1 -- 172.21.15.131:0/1063254182 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa97c014db0 con 0x564e2d647e50 2024-05-26T22:49:47.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.781+0000 7fa97b400640 1 -- 172.21.15.131:0/1063254182 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa97c01f640 con 0x564e2d647e50 2024-05-26T22:49:47.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.782+0000 7fa994d87740 1 -- 172.21.15.131:0/1063254182 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564e2d647e50 msgr2=0x564e2d5faa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:47.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.782+0000 7fa994d87740 1 --2- 172.21.15.131:0/1063254182 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564e2d647e50 0x564e2d5faa70 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fa97c00d8e0 tx=0x7fa97c010000 comp rx=0 tx=0).stop 2024-05-26T22:49:47.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.782+0000 7fa994d87740 1 -- 172.21.15.131:0/1063254182 shutdown_connections 2024-05-26T22:49:47.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.782+0000 7fa994d87740 1 --2- 172.21.15.131:0/1063254182 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x564e2d62a830 0x564e2d5ff640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:47.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.782+0000 7fa994d87740 1 --2- 172.21.15.131:0/1063254182 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564e2d62af70 0x564e2d62a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:47.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.782+0000 7fa994d87740 1 --2- 172.21.15.131:0/1063254182 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564e2d647e50 0x564e2d5faa70 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:47.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.782+0000 7fa994d87740 1 -- 172.21.15.131:0/1063254182 >> 172.21.15.131:0/1063254182 conn(0x564e2d5fcce0 msgr2=0x564e2d6452f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:47.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.782+0000 7fa994d87740 1 -- 172.21.15.131:0/1063254182 shutdown_connections 2024-05-26T22:49:47.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.782+0000 7fa994d87740 1 -- 172.21.15.131:0/1063254182 wait complete. 2024-05-26T22:49:47.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.783+0000 7fa994d87740 1 -- start start 2024-05-26T22:49:47.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.783+0000 7fa994d87740 1 Processor -- start 2024-05-26T22:49:47.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.784+0000 7fa994d87740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564e2d62af70 0x564e2d63e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:47.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.784+0000 7fa994d87740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564e2d647e50 0x564e2d62dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:47.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.784+0000 7fa994d87740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x564e2d6558d0 0x564e2d655d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:47.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.784+0000 7fa994d87740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x564e2d5fec00 con 0x564e2d647e50 2024-05-26T22:49:47.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.784+0000 7fa994d87740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x564e2d51d910 con 0x564e2d62af70 2024-05-26T22:49:47.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.784+0000 7fa994d87740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x564e2d5fdff0 con 0x564e2d6558d0 2024-05-26T22:49:47.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.784+0000 7fa994d87740 10 client.0 _ll_register_callbacks cb 0x564e2d559410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-26T22:49:47.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.784+0000 7fa982000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564e2d647e50 0x564e2d62dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:47.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.784+0000 7fa982000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564e2d647e50 0x564e2d62dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50322/0 (socket says 172.21.15.131:50322) 2024-05-26T22:49:47.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.784+0000 7fa982000640 1 -- 172.21.15.131:0/545709655 learned_addr learned my addr 172.21.15.131:0/545709655 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:47.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.785+0000 7fa982000640 1 -- 172.21.15.131:0/545709655 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x564e2d6558d0 msgr2=0x564e2d655d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:47.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.785+0000 7fa982000640 1 --2- 172.21.15.131:0/545709655 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x564e2d6558d0 0x564e2d655d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:47.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.785+0000 7fa982000640 1 -- 172.21.15.131:0/545709655 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564e2d62af70 msgr2=0x564e2d63e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:47.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.785+0000 7fa982000640 1 --2- 172.21.15.131:0/545709655 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564e2d62af70 0x564e2d63e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:47.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.785+0000 7fa982000640 1 -- 172.21.15.131:0/545709655 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e2d3406b0 con 0x564e2d647e50 2024-05-26T22:49:47.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.785+0000 7fa982000640 1 --2- 172.21.15.131:0/545709655 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564e2d647e50 0x564e2d62dd20 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fa974076350 tx=0x7fa97407a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83c7dac65cbae1f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:47.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.785+0000 7fa96fe00640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-26T22:49:47.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.785+0000 7fa96fe00640 1 -- 172.21.15.131:0/545709655 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa97404b040 con 0x564e2d647e50 2024-05-26T22:49:47.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.785+0000 7fa96fe00640 1 -- 172.21.15.131:0/545709655 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fa974082640 con 0x564e2d647e50 2024-05-26T22:49:47.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.785+0000 7fa994d87740 1 -- 172.21.15.131:0/545709655 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa97c014db0 con 0x564e2d647e50 2024-05-26T22:49:47.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.785+0000 7fa96fe00640 1 -- 172.21.15.131:0/545709655 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fa97403ce60 con 0x564e2d647e50 2024-05-26T22:49:47.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.785+0000 7fa994d87740 10 client.5400 fetch_fsmap learned FSMap version 20 2024-05-26T22:49:47.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.785+0000 7fa994d87740 1 -- 172.21.15.131:0/545709655 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=2+}) -- 0x564e2d62e260 con 0x564e2d647e50 2024-05-26T22:49:47.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.786+0000 7fa96fe00640 1 -- 172.21.15.131:0/545709655 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa97c014db0 con 0x564e2d647e50 2024-05-26T22:49:47.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.786+0000 7fa96fe00640 1 -- 172.21.15.131:0/545709655 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 20) ==== 3534+0+0 (secure 0 0 0) 0x7fa974079340 con 0x564e2d647e50 2024-05-26T22:49:47.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.786+0000 7fa994d87740 10 client.5400 fetch_fsmap finished waiting for FSMap version 20 2024-05-26T22:49:47.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.786+0000 7fa994d87740 10 client.5400 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-26T22:49:47.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.786+0000 7fa994d87740 20 client.5400 populate_metadata read hostname 'smithi131' 2024-05-26T22:49:47.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.786+0000 7fa994d87740 1 --2- 172.21.15.131:0/545709655 >> [v2:172.21.15.131:6835/2062704413,v1:172.21.15.131:6837/2062704413] conn(0x564e2d63c730 0x564e2d6641e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:47.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.786+0000 7fa994d87740 4 client.5400 mds_command: new command op to 5115 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-26T22:49:47.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.786+0000 7fa994d87740 1 -- 172.21.15.131:0/545709655 --> [v2:172.21.15.131:6835/2062704413,v1:172.21.15.131:6837/2062704413] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564e2d56fd40 con 0x564e2d63c730 2024-05-26T22:49:47.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.786+0000 7fa98aa00640 1 --2- 172.21.15.131:0/545709655 >> [v2:172.21.15.131:6835/2062704413,v1:172.21.15.131:6837/2062704413] conn(0x564e2d63c730 0x564e2d6641e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:47.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.786+0000 7fa98aa00640 1 --2- 172.21.15.131:0/545709655 >> [v2:172.21.15.131:6835/2062704413,v1:172.21.15.131:6837/2062704413] conn(0x564e2d63c730 0x564e2d6641e0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33c64b726fee1f01 server_cookie=9b4f47bc76a29cd7 in_seq=0 out_seq=0 2024-05-26T22:49:47.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.786+0000 7fa96fe00640 10 client.5400 ms_handle_connect on v2:172.21.15.131:6835/2062704413 2024-05-26T22:49:47.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.787+0000 7fa96fe00640 1 -- 172.21.15.131:0/545709655 <== mds.0 v2:172.21.15.131:6835/2062704413 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564e2d56fd40 con 0x564e2d63c730 2024-05-26T22:49:47.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.787+0000 7fa96fe00640 10 client.5400 handle_command_reply: tid=0 2024-05-26T22:49:47.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.787+0000 7fa994d87740 1 client.5400 shutdown 2024-05-26T22:49:47.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.787+0000 7fa994d87740 1 -- 172.21.15.131:0/545709655 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564e2d647e50 msgr2=0x564e2d62dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:47.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.787+0000 7fa994d87740 1 --2- 172.21.15.131:0/545709655 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564e2d647e50 0x564e2d62dd20 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fa974076350 tx=0x7fa97407a730 comp rx=0 tx=0).stop 2024-05-26T22:49:47.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.787+0000 7fa994d87740 1 -- 172.21.15.131:0/545709655 shutdown_connections 2024-05-26T22:49:47.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.787+0000 7fa994d87740 1 --2- 172.21.15.131:0/545709655 >> [v2:172.21.15.131:6835/2062704413,v1:172.21.15.131:6837/2062704413] conn(0x564e2d63c730 0x564e2d6641e0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:47.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.787+0000 7fa994d87740 1 --2- 172.21.15.131:0/545709655 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x564e2d6558d0 0x564e2d655d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:47.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.787+0000 7fa994d87740 1 --2- 172.21.15.131:0/545709655 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564e2d647e50 0x564e2d62dd20 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:47.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.787+0000 7fa96fe00640 0 client.5400 ms_handle_reset on v2:172.21.15.131:6835/2062704413 2024-05-26T22:49:47.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.787+0000 7fa994d87740 1 --2- 172.21.15.131:0/545709655 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564e2d62af70 0x564e2d63e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:47.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.787+0000 7fa994d87740 1 -- 172.21.15.131:0/545709655 >> 172.21.15.131:0/545709655 conn(0x564e2d62a830 msgr2=0x564e2d5fb4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:47.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.788+0000 7fa994d87740 1 -- 172.21.15.131:0/545709655 shutdown_connections 2024-05-26T22:49:47.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.788+0000 7fa994d87740 1 -- 172.21.15.131:0/545709655 wait complete. 2024-05-26T22:49:47.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.788+0000 7fa994d87740 20 client.5400 trim_cache size 0 max 16384 2024-05-26T22:49:47.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.809+0000 7fa994d87740 1 Processor -- start 2024-05-26T22:49:47.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.809+0000 7fa994d87740 1 -- start start 2024-05-26T22:49:47.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.809+0000 7fa994d87740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x564e2d62af70 0x564e2d5f9b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:47.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.810+0000 7fa994d87740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564e2d647e50 0x564e2d6bdb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:47.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.810+0000 7fa994d87740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564e2d6558d0 0x564e2d6be0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:47.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.810+0000 7fa994d87740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa98c148c10 con 0x564e2d647e50 2024-05-26T22:49:47.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.810+0000 7fa994d87740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa98c2001b0 con 0x564e2d6558d0 2024-05-26T22:49:47.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.810+0000 7fa994d87740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa98c145b40 con 0x564e2d62af70 2024-05-26T22:49:47.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.810+0000 7fa982000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564e2d647e50 0x564e2d6bdb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:47.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.810+0000 7fa982000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564e2d647e50 0x564e2d6bdb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50328/0 (socket says 172.21.15.131:50328) 2024-05-26T22:49:47.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.810+0000 7fa982000640 1 -- 172.21.15.131:0/2389592919 learned_addr learned my addr 172.21.15.131:0/2389592919 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:47.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.810+0000 7fa982000640 1 -- 172.21.15.131:0/2389592919 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x564e2d62af70 msgr2=0x564e2d5f9b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:47.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.810+0000 7fa982000640 1 --2- 172.21.15.131:0/2389592919 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x564e2d62af70 0x564e2d5f9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:47.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.810+0000 7fa982000640 1 -- 172.21.15.131:0/2389592919 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564e2d6558d0 msgr2=0x564e2d6be0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:47.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.810+0000 7fa982000640 1 --2- 172.21.15.131:0/2389592919 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564e2d6558d0 0x564e2d6be0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:47.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.810+0000 7fa982000640 1 -- 172.21.15.131:0/2389592919 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e2d6556f0 con 0x564e2d647e50 2024-05-26T22:49:47.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.810+0000 7fa982000640 1 --2- 172.21.15.131:0/2389592919 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564e2d647e50 0x564e2d6bdb60 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fa974035040 tx=0x7fa974008000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47b1be41adbdd392 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:47.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.810+0000 7fa97aa00640 1 -- 172.21.15.131:0/2389592919 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa97404b040 con 0x564e2d647e50 2024-05-26T22:49:47.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.810+0000 7fa97aa00640 1 -- 172.21.15.131:0/2389592919 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa974079340 con 0x564e2d647e50 2024-05-26T22:49:47.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.810+0000 7fa97aa00640 1 -- 172.21.15.131:0/2389592919 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa974082880 con 0x564e2d647e50 2024-05-26T22:49:47.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.810+0000 7fa994d87740 1 -- 172.21.15.131:0/2389592919 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564e2d647e50 msgr2=0x564e2d6bdb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:47.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.810+0000 7fa994d87740 1 --2- 172.21.15.131:0/2389592919 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564e2d647e50 0x564e2d6bdb60 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fa974035040 tx=0x7fa974008000 comp rx=0 tx=0).stop 2024-05-26T22:49:47.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.810+0000 7fa994d87740 1 -- 172.21.15.131:0/2389592919 shutdown_connections 2024-05-26T22:49:47.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.810+0000 7fa994d87740 1 --2- 172.21.15.131:0/2389592919 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564e2d6558d0 0x564e2d6be0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:47.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.810+0000 7fa994d87740 1 --2- 172.21.15.131:0/2389592919 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564e2d647e50 0x564e2d6bdb60 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:47.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.810+0000 7fa994d87740 1 --2- 172.21.15.131:0/2389592919 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x564e2d62af70 0x564e2d5f9b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:47.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.810+0000 7fa994d87740 1 -- 172.21.15.131:0/2389592919 >> 172.21.15.131:0/2389592919 conn(0x564e2d62a830 msgr2=0x564e2d63d060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:47.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.810+0000 7fa994d87740 1 -- 172.21.15.131:0/2389592919 shutdown_connections 2024-05-26T22:49:47.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.810+0000 7fa994d87740 1 -- 172.21.15.131:0/2389592919 wait complete. 2024-05-26T22:49:47.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.811+0000 7fa994d87740 1 -- start start 2024-05-26T22:49:47.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.811+0000 7fa994d87740 1 Processor -- start 2024-05-26T22:49:47.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.812+0000 7fa994d87740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564e2d62af70 0x564e2d6bcd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:47.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.812+0000 7fa994d87740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564e2d647e50 0x564e2d6465b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:47.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.812+0000 7fa994d87740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x564e2d6558d0 0x564e2d65b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:47.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.812+0000 7fa994d87740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x564e2d62c0b0 con 0x564e2d62af70 2024-05-26T22:49:47.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.812+0000 7fa994d87740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x564e2d63dd70 con 0x564e2d647e50 2024-05-26T22:49:47.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.812+0000 7fa994d87740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x564e2d62baa0 con 0x564e2d6558d0 2024-05-26T22:49:47.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.812+0000 7fa994d87740 10 client.0 _ll_register_callbacks cb 0x7fa974009400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-26T22:49:47.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.812+0000 7fa98aa00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564e2d62af70 0x564e2d6bcd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:47.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.812+0000 7fa98aa00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564e2d62af70 0x564e2d6bcd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50338/0 (socket says 172.21.15.131:50338) 2024-05-26T22:49:47.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.812+0000 7fa98aa00640 1 -- 172.21.15.131:0/2071794069 learned_addr learned my addr 172.21.15.131:0/2071794069 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:47.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.813+0000 7fa98aa00640 1 -- 172.21.15.131:0/2071794069 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x564e2d6558d0 msgr2=0x564e2d65b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:47.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.813+0000 7fa98aa00640 1 --2- 172.21.15.131:0/2071794069 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x564e2d6558d0 0x564e2d65b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:47.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.813+0000 7fa98aa00640 1 -- 172.21.15.131:0/2071794069 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564e2d647e50 msgr2=0x564e2d6465b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:47.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.813+0000 7fa98aa00640 1 --2- 172.21.15.131:0/2071794069 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564e2d647e50 0x564e2d6465b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:47.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.813+0000 7fa98aa00640 1 -- 172.21.15.131:0/2071794069 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e2d645bc0 con 0x564e2d62af70 2024-05-26T22:49:47.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.813+0000 7fa98aa00640 1 --2- 172.21.15.131:0/2071794069 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564e2d62af70 0x564e2d6bcd70 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fa97c00d8b0 tx=0x7fa97c003760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85dd4b93efa6e9a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:47.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.813+0000 7fa96fe00640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-26T22:49:47.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.813+0000 7fa96fe00640 1 -- 172.21.15.131:0/2071794069 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa97c024040 con 0x564e2d62af70 2024-05-26T22:49:47.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.813+0000 7fa96fe00640 1 -- 172.21.15.131:0/2071794069 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fa97c02c460 con 0x564e2d62af70 2024-05-26T22:49:47.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.813+0000 7fa994d87740 1 -- 172.21.15.131:0/2071794069 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa974079340 con 0x564e2d62af70 2024-05-26T22:49:47.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.813+0000 7fa96fe00640 1 -- 172.21.15.131:0/2071794069 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fa97c03e050 con 0x564e2d62af70 2024-05-26T22:49:47.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.813+0000 7fa994d87740 10 client.5406 fetch_fsmap learned FSMap version 20 2024-05-26T22:49:47.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.813+0000 7fa994d87740 1 -- 172.21.15.131:0/2071794069 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=2+}) -- 0x564e2d6456a0 con 0x564e2d62af70 2024-05-26T22:49:47.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.814+0000 7fa96fe00640 1 -- 172.21.15.131:0/2071794069 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa974079340 con 0x564e2d62af70 2024-05-26T22:49:47.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.814+0000 7fa96fe00640 1 -- 172.21.15.131:0/2071794069 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 20) ==== 3534+0+0 (secure 0 0 0) 0x7fa97c041020 con 0x564e2d62af70 2024-05-26T22:49:47.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.814+0000 7fa994d87740 10 client.5406 fetch_fsmap finished waiting for FSMap version 20 2024-05-26T22:49:47.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.814+0000 7fa994d87740 10 client.5406 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-26T22:49:47.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.814+0000 7fa994d87740 20 client.5406 populate_metadata read hostname 'smithi131' 2024-05-26T22:49:47.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.814+0000 7fa994d87740 1 --2- 172.21.15.131:0/2071794069 >> [v2:172.21.15.131:6835/2062704413,v1:172.21.15.131:6837/2062704413] conn(0x564e2d65d990 0x564e2d63ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:47.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.814+0000 7fa994d87740 4 client.5406 mds_command: new command op to 5115 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-26T22:49:47.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.814+0000 7fa994d87740 1 -- 172.21.15.131:0/2071794069 --> [v2:172.21.15.131:6835/2062704413,v1:172.21.15.131:6837/2062704413] -- command(tid 0: {"prefix": "status"}) -- 0x564e2d5fc130 con 0x564e2d65d990 2024-05-26T22:49:47.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.814+0000 7fa982000640 1 --2- 172.21.15.131:0/2071794069 >> [v2:172.21.15.131:6835/2062704413,v1:172.21.15.131:6837/2062704413] conn(0x564e2d65d990 0x564e2d63ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:47.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.814+0000 7fa982000640 1 --2- 172.21.15.131:0/2071794069 >> [v2:172.21.15.131:6835/2062704413,v1:172.21.15.131:6837/2062704413] conn(0x564e2d65d990 0x564e2d63ac80 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79ec50d52bae6386 server_cookie=f2bf7fc0a9be3a47 in_seq=0 out_seq=0 2024-05-26T22:49:47.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.815+0000 7fa96fe00640 10 client.5406 ms_handle_connect on v2:172.21.15.131:6835/2062704413 2024-05-26T22:49:47.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.815+0000 7fa96fe00640 1 -- 172.21.15.131:0/2071794069 <== mds.0 v2:172.21.15.131:6835/2062704413 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x564e2d5fc130 con 0x564e2d65d990 2024-05-26T22:49:47.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.815+0000 7fa96fe00640 10 client.5406 handle_command_reply: tid=0 2024-05-26T22:49:47.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.815+0000 7fa994d87740 1 client.5406 shutdown 2024-05-26T22:49:47.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.815+0000 7fa994d87740 1 -- 172.21.15.131:0/2071794069 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564e2d62af70 msgr2=0x564e2d6bcd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:47.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.815+0000 7fa994d87740 1 --2- 172.21.15.131:0/2071794069 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564e2d62af70 0x564e2d6bcd70 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fa97c00d8b0 tx=0x7fa97c003760 comp rx=0 tx=0).stop 2024-05-26T22:49:47.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.815+0000 7fa994d87740 1 -- 172.21.15.131:0/2071794069 shutdown_connections 2024-05-26T22:49:47.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.815+0000 7fa994d87740 1 --2- 172.21.15.131:0/2071794069 >> [v2:172.21.15.131:6835/2062704413,v1:172.21.15.131:6837/2062704413] conn(0x564e2d65d990 0x564e2d63ac80 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:47.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.815+0000 7fa994d87740 1 --2- 172.21.15.131:0/2071794069 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x564e2d6558d0 0x564e2d65b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:47.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.815+0000 7fa994d87740 1 --2- 172.21.15.131:0/2071794069 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564e2d647e50 0x564e2d6465b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:47.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.815+0000 7fa994d87740 1 --2- 172.21.15.131:0/2071794069 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564e2d62af70 0x564e2d6bcd70 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:47.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.815+0000 7fa96fe00640 0 client.5406 ms_handle_reset on v2:172.21.15.131:6835/2062704413 2024-05-26T22:49:47.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.815+0000 7fa994d87740 1 -- 172.21.15.131:0/2071794069 >> 172.21.15.131:0/2071794069 conn(0x564e2d62a830 msgr2=0x564e2d5f9d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:47.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.815+0000 7fa994d87740 1 -- 172.21.15.131:0/2071794069 shutdown_connections 2024-05-26T22:49:47.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.815+0000 7fa994d87740 1 -- 172.21.15.131:0/2071794069 wait complete. 2024-05-26T22:49:47.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.815+0000 7fa994d87740 20 client.5406 trim_cache size 0 max 16384 2024-05-26T22:49:47.820 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-26T22:49:47.820 INFO:teuthology.orchestra.run.smithi131.stdout: "cluster_fsid": "46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f", 2024-05-26T22:49:47.820 INFO:teuthology.orchestra.run.smithi131.stdout: "whoami": 0, 2024-05-26T22:49:47.820 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 5115, 2024-05-26T22:49:47.820 INFO:teuthology.orchestra.run.smithi131.stdout: "want_state": "up:active", 2024-05-26T22:49:47.820 INFO:teuthology.orchestra.run.smithi131.stdout: "state": "up:active", 2024-05-26T22:49:47.821 INFO:teuthology.orchestra.run.smithi131.stdout: "fs_name": "cephfs", 2024-05-26T22:49:47.821 INFO:teuthology.orchestra.run.smithi131.stdout: "rank_uptime": 16.278481735, 2024-05-26T22:49:47.821 INFO:teuthology.orchestra.run.smithi131.stdout: "mdsmap_epoch": 20, 2024-05-26T22:49:47.821 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch": 33, 2024-05-26T22:49:47.821 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch_barrier": 33, 2024-05-26T22:49:47.821 INFO:teuthology.orchestra.run.smithi131.stdout: "uptime": 25.130920114999999 2024-05-26T22:49:47.821 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-26T22:49:47.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.816+0000 7fa98be00640 1 -- 172.21.15.131:0/1168172898 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa96806be30 msgr2=0x7fa96806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:47.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.816+0000 7fa98be00640 1 --2- 172.21.15.131:0/1168172898 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa96806be30 0x7fa96806e2f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa97c003d90 tx=0x7fa97c009040 comp rx=0 tx=0).stop 2024-05-26T22:49:47.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.816+0000 7fa98be00640 1 -- 172.21.15.131:0/1168172898 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa98c0d0550 msgr2=0x7fa98c0c5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:47.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.816+0000 7fa98be00640 1 --2- 172.21.15.131:0/1168172898 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa98c0d0550 0x7fa98c0c5100 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fa9740097a0 tx=0x7fa9740046c0 comp rx=0 tx=0).stop 2024-05-26T22:49:47.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.816+0000 7fa98be00640 1 -- 172.21.15.131:0/1168172898 shutdown_connections 2024-05-26T22:49:47.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.816+0000 7fa98be00640 1 --2- 172.21.15.131:0/1168172898 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa96806be30 0x7fa96806e2f0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:47.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.816+0000 7fa98be00640 1 --2- 172.21.15.131:0/1168172898 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa98c15e320 0x7fa98c15cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:47.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.816+0000 7fa98be00640 1 --2- 172.21.15.131:0/1168172898 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa98c0d0550 0x7fa98c0c5100 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:47.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.816+0000 7fa98be00640 1 --2- 172.21.15.131:0/1168172898 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa98c0b2f60 0x7fa98c160160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:47.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.816+0000 7fa98be00640 1 -- 172.21.15.131:0/1168172898 >> 172.21.15.131:0/1168172898 conn(0x7fa98c0b5660 msgr2=0x7fa98c0b11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:47.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.816+0000 7fa98be00640 1 -- 172.21.15.131:0/1168172898 shutdown_connections 2024-05-26T22:49:47.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47.816+0000 7fa98be00640 1 -- 172.21.15.131:0/1168172898 wait complete. 2024-05-26T22:49:47.831 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-05-26T22:49:47.832 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-26T22:49:47.832 DEBUG:teuthology.orchestra.run.smithi131:> mkdir -p tree 2024-05-26T22:49:47.832 DEBUG:teuthology.orchestra.run.smithi131:> 2024-05-26T22:49:47.832 DEBUG:teuthology.orchestra.run.smithi131:> 2024-05-26T22:49:47.832 DEBUG:teuthology.orchestra.run.smithi131:> 2024-05-26T22:49:47.832 DEBUG:teuthology.orchestra.run.smithi131:> for ((i = 0; i < 100; i++)); do 2024-05-26T22:49:47.832 DEBUG:teuthology.orchestra.run.smithi131:> mkdir -p "tree/$i" 2024-05-26T22:49:47.832 DEBUG:teuthology.orchestra.run.smithi131:> echo file > "tree/$i/file" 2024-05-26T22:49:47.832 DEBUG:teuthology.orchestra.run.smithi131:> done 2024-05-26T22:49:47.832 DEBUG:teuthology.orchestra.run.smithi131:> ') 2024-05-26T22:49:47.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:47 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-26T22:49:47.920 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree 2024-05-26T22:49:47.926 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i = 0 )) 2024-05-26T22:49:47.926 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:49:47.927 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/0 2024-05-26T22:49:47.930 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:49:47.938 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:49:47.939 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:49:47.939 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/1 2024-05-26T22:49:47.942 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:49:47.944 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:49:47.944 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:49:47.944 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/2 2024-05-26T22:49:47.948 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:49:47.950 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:49:47.950 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:49:47.950 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/3 2024-05-26T22:49:47.954 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:49:47.956 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:49:47.956 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:49:47.956 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/4 2024-05-26T22:49:47.960 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:49:47.960 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:49:47.960 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:49:47.961 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/5 2024-05-26T22:49:47.964 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:49:47.965 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:49:47.965 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:49:47.966 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/6 2024-05-26T22:49:47.969 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:49:47.971 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:49:47.971 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:49:47.971 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/7 2024-05-26T22:49:47.976 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:49:47.977 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:49:47.977 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:49:47.977 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/8 2024-05-26T22:49:47.982 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:49:47.983 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:49:47.984 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:49:47.984 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/9 2024-05-26T22:49:47.988 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:49:47.990 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:49:47.990 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:49:47.991 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/10 2024-05-26T22:49:47.995 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:49:47.997 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:49:47.997 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:49:47.997 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/11 2024-05-26T22:49:48.002 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:49:48.004 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:49:48.004 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:49:48.004 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/12 2024-05-26T22:49:48.009 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:49:48.011 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:49:48.011 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:49:48.011 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/13 2024-05-26T22:49:48.016 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:49:48.018 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:49:48.018 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:49:48.018 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/14 2024-05-26T22:49:48.023 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:49:48.025 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:49:48.025 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:49:48.026 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/15 2024-05-26T22:49:48.030 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:49:48.031 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:49:48.031 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:49:48.031 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/16 2024-05-26T22:49:48.035 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:49:48.037 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:49:48.037 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:49:48.037 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/17 2024-05-26T22:49:48.042 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:49:48.044 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:49:48.044 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:49:48.044 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/18 2024-05-26T22:49:48.049 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:49:48.051 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:49:48.051 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:49:48.051 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/19 2024-05-26T22:49:48.056 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:49:48.058 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:49:48.058 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:49:48.059 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/20 2024-05-26T22:49:48.064 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:49:48.065 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:49:48.065 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:49:48.066 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/21 2024-05-26T22:49:48.071 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:49:48.073 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:49:48.073 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:49:48.073 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/22 2024-05-26T22:49:48.078 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:49:48.080 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:49:48.080 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:49:48.081 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/23 2024-05-26T22:49:48.086 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:49:48.088 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:49:48.088 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:49:48.088 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/24 2024-05-26T22:49:48.093 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:49:48.094 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:49:48.094 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:49:48.095 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/25 2024-05-26T22:49:48.099 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:49:48.101 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:49:48.101 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:49:48.101 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/26 2024-05-26T22:49:48.106 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:49:48.107 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:49:48.108 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:49:48.108 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/27 2024-05-26T22:49:48.112 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:49:48.115 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:49:48.116 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:49:48.116 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/28 2024-05-26T22:49:48.122 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:49:48.124 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:49:48.124 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:49:48.125 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/29 2024-05-26T22:49:48.130 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:49:48.132 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:49:48.132 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:49:48.133 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/30 2024-05-26T22:49:48.138 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:49:48.140 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:49:48.140 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:49:48.140 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/31 2024-05-26T22:49:48.146 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:49:48.148 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:49:48.148 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:49:48.148 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/32 2024-05-26T22:49:48.153 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:49:48.155 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:49:48.156 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:49:48.156 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/33 2024-05-26T22:49:48.161 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:49:48.163 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:49:48.163 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:49:48.164 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/34 2024-05-26T22:49:48.169 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:49:48.171 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:49:48.171 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:49:48.172 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/35 2024-05-26T22:49:48.177 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:49:48.179 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:49:48.179 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:49:48.180 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/36 2024-05-26T22:49:48.185 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:49:48.188 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:49:48.188 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:49:48.188 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/37 2024-05-26T22:49:48.193 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:49:48.195 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:49:48.196 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:49:48.196 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/38 2024-05-26T22:49:48.201 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:49:48.203 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:49:48.203 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:49:48.203 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/39 2024-05-26T22:49:48.209 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:49:48.211 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:49:48.211 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:49:48.211 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/40 2024-05-26T22:49:48.217 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:49:48.219 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:49:48.219 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:49:48.219 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/41 2024-05-26T22:49:48.224 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:49:48.227 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:49:48.227 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:49:48.227 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/42 2024-05-26T22:49:48.232 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:49:48.234 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:49:48.234 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:49:48.234 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/43 2024-05-26T22:49:48.241 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:49:48.243 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:49:48.243 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:49:48.243 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/44 2024-05-26T22:49:48.248 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:49:48.250 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:49:48.250 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:49:48.250 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/45 2024-05-26T22:49:48.256 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:49:48.258 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:49:48.258 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:49:48.258 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/46 2024-05-26T22:49:48.263 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:49:48.266 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:49:48.266 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:49:48.266 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/47 2024-05-26T22:49:48.272 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:49:48.275 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:49:48.275 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:49:48.275 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/48 2024-05-26T22:49:48.280 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:49:48.282 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:49:48.283 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:49:48.283 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/49 2024-05-26T22:49:48.288 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:49:48.291 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:49:48.291 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:49:48.291 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/50 2024-05-26T22:49:48.296 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:49:48.299 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:49:48.299 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:49:48.299 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/51 2024-05-26T22:49:48.305 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:49:48.307 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:49:48.307 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:49:48.308 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/52 2024-05-26T22:49:48.313 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:49:48.315 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:49:48.315 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:49:48.316 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/53 2024-05-26T22:49:48.321 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:49:48.324 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:49:48.324 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:49:48.324 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/54 2024-05-26T22:49:48.331 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:49:48.332 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:49:48.332 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:49:48.333 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/55 2024-05-26T22:49:48.339 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:49:48.341 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:49:48.341 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:49:48.341 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/56 2024-05-26T22:49:48.347 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:49:48.350 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:49:48.350 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:49:48.350 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/57 2024-05-26T22:49:48.356 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:49:48.358 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:49:48.358 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:49:48.359 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/58 2024-05-26T22:49:48.364 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:49:48.367 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:49:48.367 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:49:48.367 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/59 2024-05-26T22:49:48.372 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:49:48.375 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:49:48.375 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:49:48.375 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/60 2024-05-26T22:49:48.380 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:49:48.383 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:49:48.383 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:49:48.383 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/61 2024-05-26T22:49:48.388 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:49:48.394 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:49:48.394 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:49:48.394 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/62 2024-05-26T22:49:48.397 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:49:48.400 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:49:48.400 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:49:48.400 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/63 2024-05-26T22:49:48.406 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:49:48.407 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:49:48.407 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:49:48.407 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/64 2024-05-26T22:49:48.412 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:49:48.413 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:49:48.414 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:49:48.414 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/65 2024-05-26T22:49:48.419 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:49:48.422 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:49:48.422 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:49:48.422 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/66 2024-05-26T22:49:48.427 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:49:48.430 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:49:48.430 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:49:48.430 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/67 2024-05-26T22:49:48.435 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:49:48.438 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:49:48.438 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:49:48.438 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/68 2024-05-26T22:49:48.444 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:49:48.446 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:49:48.446 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:49:48.446 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/69 2024-05-26T22:49:48.452 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:49:48.454 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:49:48.454 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:49:48.454 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/70 2024-05-26T22:49:48.461 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:49:48.463 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:49:48.463 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:49:48.463 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/71 2024-05-26T22:49:48.469 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:49:48.471 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:49:48.472 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:49:48.472 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/72 2024-05-26T22:49:48.477 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:49:48.480 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:49:48.480 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:49:48.480 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/73 2024-05-26T22:49:48.485 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:49:48.488 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:49:48.488 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:49:48.488 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/74 2024-05-26T22:49:48.494 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:49:48.496 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:49:48.496 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:49:48.496 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/75 2024-05-26T22:49:48.502 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:49:48.504 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:49:48.504 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:49:48.504 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/76 2024-05-26T22:49:48.510 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:49:48.512 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:49:48.512 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:49:48.513 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/77 2024-05-26T22:49:48.518 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:49:48.520 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:49:48.520 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:49:48.521 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/78 2024-05-26T22:49:48.526 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:49:48.528 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:49:48.529 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:49:48.529 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/79 2024-05-26T22:49:48.534 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:49:48.537 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:49:48.537 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:49:48.537 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/80 2024-05-26T22:49:48.542 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:49:48.544 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:49:48.544 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:49:48.545 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/81 2024-05-26T22:49:48.550 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:49:48.552 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:49:48.553 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:49:48.553 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/82 2024-05-26T22:49:48.559 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:49:48.561 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:49:48.561 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:49:48.561 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/83 2024-05-26T22:49:48.567 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:49:48.570 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:49:48.570 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:49:48.570 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/84 2024-05-26T22:49:48.576 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:49:48.578 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:49:48.578 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:49:48.578 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/85 2024-05-26T22:49:48.584 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:49:48.585 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:49:48.585 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:49:48.585 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/86 2024-05-26T22:49:48.590 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:49:48.592 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:49:48.592 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:49:48.592 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/87 2024-05-26T22:49:48.598 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:49:48.602 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:49:48.602 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:49:48.602 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/88 2024-05-26T22:49:48.604 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:49:48.605 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:49:48.605 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:49:48.605 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/89 2024-05-26T22:49:48.610 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:49:48.611 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:49:48.612 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:49:48.612 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/90 2024-05-26T22:49:48.615 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:49:48.616 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:49:48.617 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:49:48.617 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/91 2024-05-26T22:49:48.620 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:49:48.621 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:49:48.621 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:49:48.621 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/92 2024-05-26T22:49:48.626 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:49:48.628 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:49:48.628 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:49:48.628 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/93 2024-05-26T22:49:48.632 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:49:48.634 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:49:48.634 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:49:48.634 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/94 2024-05-26T22:49:48.638 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:49:48.640 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:49:48.640 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:49:48.640 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/95 2024-05-26T22:49:48.644 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:49:48.646 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:49:48.646 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:49:48.646 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/96 2024-05-26T22:49:48.651 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:49:48.653 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:49:48.653 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:49:48.653 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/97 2024-05-26T22:49:48.658 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:49:48.660 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:49:48.661 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:49:48.661 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/98 2024-05-26T22:49:48.665 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:49:48.667 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:49:48.667 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:49:48.668 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/99 2024-05-26T22:49:48.672 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:49:48.674 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:49:48.675 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:49:48.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-26T22:49:48.687 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_export_ephemeral_distributed False 2024-05-26T22:49:48.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:48.893+0000 7f8322c00640 1 Processor -- start 2024-05-26T22:49:48.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:48.893+0000 7f8322c00640 1 -- start start 2024-05-26T22:49:48.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:48.895+0000 7f8322c00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f831c0712d0 0x7f831c06d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:48.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:48.895+0000 7f8322c00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f831c06d790 0x7f831c0cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:48.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:48.895+0000 7f8322c00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f831c072d60 con 0x7f831c145a10 2024-05-26T22:49:48.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:48.895+0000 7f8322c00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f831c15c760 con 0x7f831c06d790 2024-05-26T22:49:48.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:48.895+0000 7f8322c00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f831c072a80 con 0x7f831c0712d0 2024-05-26T22:49:48.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:48.895+0000 7f8321800640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f831c145a10 0x7f831c072180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:56878/0 (socket says 172.21.15.131:56878) 2024-05-26T22:49:48.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:48.895+0000 7f8322200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f831c06d790 0x7f831c0cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:48.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:48.895+0000 7f8321800640 1 -- 172.21.15.131:0/1259024593 learned_addr learned my addr 172.21.15.131:0/1259024593 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:49:48.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:48.896+0000 7f8320e00640 1 --2- 172.21.15.131:0/1259024593 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f831c0712d0 0x7f831c06d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:48.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:48.896+0000 7f8322200640 1 -- 172.21.15.131:0/1259024593 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f831c0712d0 msgr2=0x7f831c06d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:49:48.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:48.896+0000 7f8322200640 1 --2- 172.21.15.131:0/1259024593 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f831c0712d0 0x7f831c06d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:48.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:48.896+0000 7f8322200640 1 -- 172.21.15.131:0/1259024593 >> v1:172.21.15.131:6789/0 conn(0x7f831c145a10 legacy=0x7f831c072180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:48.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:48.896+0000 7f8322200640 1 -- 172.21.15.131:0/1259024593 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f831c0ceab0 con 0x7f831c06d790 2024-05-26T22:49:48.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:48.897+0000 7f8322200640 1 --2- 172.21.15.131:0/1259024593 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f831c06d790 0x7f831c0cf590 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f8314009d90 tx=0x7f831402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad6ea559409b15d0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:48.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:48.897+0000 7f831be00640 1 -- 172.21.15.131:0/1259024593 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8314007740 con 0x7f831c06d790 2024-05-26T22:49:48.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:48.897+0000 7f831be00640 1 -- 172.21.15.131:0/1259024593 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83140078e0 con 0x7f831c06d790 2024-05-26T22:49:48.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:48.897+0000 7f831be00640 1 -- 172.21.15.131:0/1259024593 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8314007cb0 con 0x7f831c06d790 2024-05-26T22:49:48.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:48.898+0000 7f8322c00640 1 -- 172.21.15.131:0/1259024593 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f831c06d790 msgr2=0x7f831c0cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:48.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:48.898+0000 7f8322c00640 1 --2- 172.21.15.131:0/1259024593 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f831c06d790 0x7f831c0cf590 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f8314009d90 tx=0x7f831402a660 comp rx=0 tx=0).stop 2024-05-26T22:49:48.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:48.898+0000 7f8322c00640 1 -- 172.21.15.131:0/1259024593 shutdown_connections 2024-05-26T22:49:48.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:48.898+0000 7f8322c00640 1 --2- 172.21.15.131:0/1259024593 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f831c06d790 0x7f831c0cf590 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:48.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:48.898+0000 7f8322c00640 1 --2- 172.21.15.131:0/1259024593 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f831c0712d0 0x7f831c06d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:48.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:48.898+0000 7f8322c00640 1 -- 172.21.15.131:0/1259024593 >> 172.21.15.131:0/1259024593 conn(0x7f831c0b5660 msgr2=0x7f831c0b5a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:48.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:48.898+0000 7f8322c00640 1 -- 172.21.15.131:0/1259024593 shutdown_connections 2024-05-26T22:49:48.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:48.898+0000 7f8322c00640 1 -- 172.21.15.131:0/1259024593 wait complete. 2024-05-26T22:49:48.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:48.900+0000 7f8322c00640 1 Processor -- start 2024-05-26T22:49:48.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:48.900+0000 7f8322c00640 1 -- start start 2024-05-26T22:49:48.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:48.901+0000 7f8322c00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f831c06d790 0x7f831c15acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:48.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:48.901+0000 7f8322c00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f831c0712d0 0x7f831c1f3d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:48.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:48.901+0000 7f8322c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f831c145a10 0x7f831c141f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:48.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:48.901+0000 7f8322c00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f831c145df0 con 0x7f831c145a10 2024-05-26T22:49:48.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:48.901+0000 7f8322c00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f831c0cf070 con 0x7f831c0712d0 2024-05-26T22:49:48.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:48.901+0000 7f8322c00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f831c0cfad0 con 0x7f831c06d790 2024-05-26T22:49:48.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:48.901+0000 7f8322200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f831c145a10 0x7f831c141f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:48.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:48.901+0000 7f8322200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f831c145a10 0x7f831c141f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50366/0 (socket says 172.21.15.131:50366) 2024-05-26T22:49:48.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:48.901+0000 7f8322200640 1 -- 172.21.15.131:0/1431858493 learned_addr learned my addr 172.21.15.131:0/1431858493 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:49:48.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:48.902+0000 7f8321800640 1 --2- 172.21.15.131:0/1431858493 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f831c06d790 0x7f831c15acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:48.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:48.902+0000 7f8320e00640 1 --2- 172.21.15.131:0/1431858493 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f831c0712d0 0x7f831c1f3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:48.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:48.902+0000 7f8322200640 1 -- 172.21.15.131:0/1431858493 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f831c06d790 msgr2=0x7f831c15acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:48.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:48.902+0000 7f8322200640 1 --2- 172.21.15.131:0/1431858493 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f831c06d790 0x7f831c15acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:48.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:48.902+0000 7f8322200640 1 -- 172.21.15.131:0/1431858493 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f831c0712d0 msgr2=0x7f831c1f3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:48.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:48.902+0000 7f8322200640 1 --2- 172.21.15.131:0/1431858493 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f831c0712d0 0x7f831c1f3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:48.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:48.902+0000 7f8322200640 1 -- 172.21.15.131:0/1431858493 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f831c1f4580 con 0x7f831c145a10 2024-05-26T22:49:48.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:48.902+0000 7f8320e00640 1 --2- 172.21.15.131:0/1431858493 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f831c0712d0 0x7f831c1f3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:49:48.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:48.902+0000 7f8322200640 1 --2- 172.21.15.131:0/1431858493 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f831c145a10 0x7f831c141f60 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f8314009eb0 tx=0x7f8314004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:48.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:48.902+0000 7f8319600640 1 -- 172.21.15.131:0/1431858493 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8314040070 con 0x7f831c145a10 2024-05-26T22:49:48.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:48.902+0000 7f8319600640 1 -- 172.21.15.131:0/1431858493 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8314031bc0 con 0x7f831c145a10 2024-05-26T22:49:48.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:48.902+0000 7f8319600640 1 -- 172.21.15.131:0/1431858493 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8314045460 con 0x7f831c145a10 2024-05-26T22:49:48.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:48.903+0000 7f8322c00640 1 -- 172.21.15.131:0/1431858493 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f831c1f4860 con 0x7f831c145a10 2024-05-26T22:49:48.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:48.903+0000 7f8322c00640 1 -- 172.21.15.131:0/1431858493 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f831c1f5100 con 0x7f831c145a10 2024-05-26T22:49:48.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:48.904+0000 7f8322c00640 1 -- 172.21.15.131:0/1431858493 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f831c1f4ef0 con 0x7f831c145a10 2024-05-26T22:49:48.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:48.905+0000 7f8319600640 1 -- 172.21.15.131:0/1431858493 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8314039040 con 0x7f831c145a10 2024-05-26T22:49:48.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:48.906+0000 7f8319600640 1 --2- 172.21.15.131:0/1431858493 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f830006be30 0x7f830006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:49:48.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:48.906+0000 7f8319600640 1 -- 172.21.15.131:0/1431858493 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f83140ac800 con 0x7f831c145a10 2024-05-26T22:49:48.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:48.906+0000 7f8321800640 1 --2- 172.21.15.131:0/1431858493 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f830006be30 0x7f830006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:49:48.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:48.907+0000 7f8321800640 1 --2- 172.21.15.131:0/1431858493 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f830006be30 0x7f830006e2f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f831c0ae4d0 tx=0x7f8310009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:49:48.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:48.910+0000 7f8319600640 1 -- 172.21.15.131:0/1431858493 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f831407b1a0 con 0x7f831c145a10 2024-05-26T22:49:49.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:49.106+0000 7f8322c00640 1 -- 172.21.15.131:0/1431858493 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command([{prefix=config set, name=mds_export_ephemeral_distributed}] v 0) -- 0x7f831c0694e0 con 0x7f831c145a10 2024-05-26T22:49:49.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:49.111+0000 7f8319600640 1 -- 172.21.15.131:0/1431858493 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_export_ephemeral_distributed}]=0 v14)=0 v14) ==== 134+0+0 (secure 0 0 0) 0x7f8314043e40 con 0x7f831c145a10 2024-05-26T22:49:49.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:49.115+0000 7f8322c00640 1 -- 172.21.15.131:0/1431858493 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f830006be30 msgr2=0x7f830006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:49.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:49.115+0000 7f8322c00640 1 --2- 172.21.15.131:0/1431858493 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f830006be30 0x7f830006e2f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f831c0ae4d0 tx=0x7f8310009040 comp rx=0 tx=0).stop 2024-05-26T22:49:49.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:49.115+0000 7f8322c00640 1 -- 172.21.15.131:0/1431858493 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f831c145a10 msgr2=0x7f831c141f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:49:49.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:49.115+0000 7f8322c00640 1 --2- 172.21.15.131:0/1431858493 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f831c145a10 0x7f831c141f60 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f8314009eb0 tx=0x7f8314004a40 comp rx=0 tx=0).stop 2024-05-26T22:49:49.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:49.115+0000 7f8322c00640 1 -- 172.21.15.131:0/1431858493 shutdown_connections 2024-05-26T22:49:49.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:49.115+0000 7f8322c00640 1 --2- 172.21.15.131:0/1431858493 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f830006be30 0x7f830006e2f0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:49.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:49.115+0000 7f8322c00640 1 --2- 172.21.15.131:0/1431858493 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f831c145a10 0x7f831c141f60 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:49.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:49.115+0000 7f8322c00640 1 --2- 172.21.15.131:0/1431858493 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f831c0712d0 0x7f831c1f3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:49.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:49.115+0000 7f8322c00640 1 --2- 172.21.15.131:0/1431858493 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f831c06d790 0x7f831c15acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:49:49.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:49.115+0000 7f8322c00640 1 -- 172.21.15.131:0/1431858493 >> 172.21.15.131:0/1431858493 conn(0x7f831c0b5660 msgr2=0x7f831c0b5a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:49:49.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:49.115+0000 7f8322c00640 1 -- 172.21.15.131:0/1431858493 shutdown_connections 2024-05-26T22:49:49.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:49:49.115+0000 7f8322c00640 1 -- 172.21.15.131:0/1431858493 wait complete. 2024-05-26T22:49:49.135 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:49:49.135 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin.distributed -v 1 tree) 2024-05-26T22:50:04.322 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-05-26T22:50:04.555 INFO:teuthology.orchestra.run.smithi131.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"1370","version":"1370","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"827","version":"827","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":1,"authpin":0},"nref":3}}] 2024-05-26T22:50:04.555 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "1370", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "1370" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 1, "subtree": 1 }, "projected_version": "827", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "827" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-05-26T22:50:04.556 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-05-26T22:50:04.556 DEBUG:teuthology.orchestra.run.smithi131:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-26T22:50:04.658 DEBUG:teuthology.orchestra.run.smithi187:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-26T22:50:04.792 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-05-26T22:50:04.792 INFO:tasks.cephfs.mount:Terminating background process 2024-05-26T22:50:04.792 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-26T22:50:04.792 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-26T22:50:04.851 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-26T22:50:04.851 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-26T22:50:04.903 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-26T22:50:05.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.106+0000 7f4626400640 1 Processor -- start 2024-05-26T22:50:05.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.106+0000 7f4626400640 1 -- start start 2024-05-26T22:50:05.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.107+0000 7f4626400640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f462006ee20 0x7f46200b3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:05.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.107+0000 7f4626400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f46200ad250 0x7f46200af790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:05.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.107+0000 7f4626400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f462006f200 0x7f46201562c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:05.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.107+0000 7f4626400640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f462015cb20 con 0x7f462006f200 2024-05-26T22:50:05.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.107+0000 7f4626400640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f462015ce00 con 0x7f462006ee20 2024-05-26T22:50:05.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.107+0000 7f4626400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f462015d0e0 con 0x7f46200ad250 2024-05-26T22:50:05.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.107+0000 7f461fe00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f462006f200 0x7f46201562c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:05.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.107+0000 7f461fe00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f462006f200 0x7f46201562c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37624/0 (socket says 172.21.15.131:37624) 2024-05-26T22:50:05.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.107+0000 7f461fe00640 1 -- 172.21.15.131:0/171979753 learned_addr learned my addr 172.21.15.131:0/171979753 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:05.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.108+0000 7f461f400640 1 --2- 172.21.15.131:0/171979753 >> v2:172.21.15.187:3300/0 conn(0x7f462006ee20 0x7f46200b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:05.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.108+0000 7f461fe00640 1 -- 172.21.15.131:0/171979753 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f46200ad250 msgr2=0x7f46200af790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:05.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.108+0000 7f461fe00640 1 --2- 172.21.15.131:0/171979753 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f46200ad250 0x7f46200af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:05.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.108+0000 7f461fe00640 1 -- 172.21.15.131:0/171979753 >> v2:172.21.15.187:3300/0 conn(0x7f462006ee20 msgr2=0x7f46200b3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:05.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.108+0000 7f461fe00640 1 --2- 172.21.15.131:0/171979753 >> v2:172.21.15.187:3300/0 conn(0x7f462006ee20 0x7f46200b3190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:05.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.108+0000 7f461fe00640 1 -- 172.21.15.131:0/171979753 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f462015d3c0 con 0x7f462006f200 2024-05-26T22:50:05.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.109+0000 7f461fe00640 1 --2- 172.21.15.131:0/171979753 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f462006f200 0x7f46201562c0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f4620071090 tx=0x7f461802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49bac39be216b152 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:05.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.109+0000 7f461e000640 1 -- 172.21.15.131:0/171979753 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4618037070 con 0x7f462006f200 2024-05-26T22:50:05.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.109+0000 7f461e000640 1 -- 172.21.15.131:0/171979753 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4618032440 con 0x7f462006f200 2024-05-26T22:50:05.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.109+0000 7f461e000640 1 -- 172.21.15.131:0/171979753 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4618032810 con 0x7f462006f200 2024-05-26T22:50:05.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.109+0000 7f4626400640 1 -- 172.21.15.131:0/171979753 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f462006f200 msgr2=0x7f46201562c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:05.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.109+0000 7f4626400640 1 --2- 172.21.15.131:0/171979753 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f462006f200 0x7f46201562c0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f4620071090 tx=0x7f461802a1e0 comp rx=0 tx=0).stop 2024-05-26T22:50:05.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.110+0000 7f4626400640 1 -- 172.21.15.131:0/171979753 shutdown_connections 2024-05-26T22:50:05.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.110+0000 7f4626400640 1 --2- 172.21.15.131:0/171979753 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f462006f200 0x7f46201562c0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:05.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.110+0000 7f4626400640 1 --2- 172.21.15.131:0/171979753 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f46200ad250 0x7f46200af790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:05.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.110+0000 7f4626400640 1 --2- 172.21.15.131:0/171979753 >> v2:172.21.15.187:3300/0 conn(0x7f462006ee20 0x7f46200b3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:05.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.110+0000 7f4626400640 1 -- 172.21.15.131:0/171979753 >> 172.21.15.131:0/171979753 conn(0x7f46200b1e90 msgr2=0x7f46200b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:05.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.110+0000 7f4626400640 1 -- 172.21.15.131:0/171979753 shutdown_connections 2024-05-26T22:50:05.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.110+0000 7f4626400640 1 -- 172.21.15.131:0/171979753 wait complete. 2024-05-26T22:50:05.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.111+0000 7f4626400640 1 Processor -- start 2024-05-26T22:50:05.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.112+0000 7f4626400640 1 -- start start 2024-05-26T22:50:05.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.113+0000 7f4626400640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f462006ee20 0x7f46201f7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:05.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.113+0000 7f4626400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f462006f200 0x7f46201f7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:05.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.113+0000 7f4626400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f46200ad250 0x7f46200b0010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:05.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.113+0000 7f4626400640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f462015c900 con 0x7f46200ad250 2024-05-26T22:50:05.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.113+0000 7f4626400640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f462015c780 con 0x7f462006ee20 2024-05-26T22:50:05.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.113+0000 7f4626400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f4620156800 con 0x7f462006f200 2024-05-26T22:50:05.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.113+0000 7f461fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f46200ad250 0x7f46200b0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:05.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.113+0000 7f461fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f46200ad250 0x7f46200b0010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37628/0 (socket says 172.21.15.131:37628) 2024-05-26T22:50:05.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.113+0000 7f461fe00640 1 -- 172.21.15.131:0/2502627175 learned_addr learned my addr 172.21.15.131:0/2502627175 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:05.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.113+0000 7f461f400640 1 --2- 172.21.15.131:0/2502627175 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f462006ee20 0x7f46201f7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:05.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.113+0000 7f461ea00640 1 --2- 172.21.15.131:0/2502627175 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f462006f200 0x7f46201f7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:05.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.113+0000 7f461fe00640 1 -- 172.21.15.131:0/2502627175 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f462006f200 msgr2=0x7f46201f7dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:05.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.113+0000 7f461fe00640 1 --2- 172.21.15.131:0/2502627175 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f462006f200 0x7f46201f7dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:05.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.113+0000 7f461fe00640 1 -- 172.21.15.131:0/2502627175 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f462006ee20 msgr2=0x7f46201f7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:05.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.113+0000 7f461fe00640 1 --2- 172.21.15.131:0/2502627175 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f462006ee20 0x7f46201f7880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:05.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.114+0000 7f461fe00640 1 -- 172.21.15.131:0/2502627175 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46200b0cd0 con 0x7f46200ad250 2024-05-26T22:50:05.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.114+0000 7f461ea00640 1 --2- 172.21.15.131:0/2502627175 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f462006f200 0x7f46201f7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:50:05.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.114+0000 7f461fe00640 1 --2- 172.21.15.131:0/2502627175 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f46200ad250 0x7f46200b0010 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f46180090b0 tx=0x7f46180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:05.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.114+0000 7f4617400640 1 -- 172.21.15.131:0/2502627175 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4618041020 con 0x7f46200ad250 2024-05-26T22:50:05.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.114+0000 7f4617400640 1 -- 172.21.15.131:0/2502627175 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4618030990 con 0x7f46200ad250 2024-05-26T22:50:05.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.114+0000 7f4626400640 1 -- 172.21.15.131:0/2502627175 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4620143d70 con 0x7f46200ad250 2024-05-26T22:50:05.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.114+0000 7f4617400640 1 -- 172.21.15.131:0/2502627175 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4618037040 con 0x7f46200ad250 2024-05-26T22:50:05.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.114+0000 7f4626400640 1 -- 172.21.15.131:0/2502627175 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46201437b0 con 0x7f46200ad250 2024-05-26T22:50:05.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.116+0000 7f4626400640 1 -- 172.21.15.131:0/2502627175 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46200694e0 con 0x7f46200ad250 2024-05-26T22:50:05.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.117+0000 7f4617400640 1 -- 172.21.15.131:0/2502627175 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4618044020 con 0x7f46200ad250 2024-05-26T22:50:05.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.117+0000 7f4617400640 1 --2- 172.21.15.131:0/2502627175 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f45fc06be60 0x7f45fc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:05.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.118+0000 7f4617400640 1 -- 172.21.15.131:0/2502627175 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f46180ab870 con 0x7f46200ad250 2024-05-26T22:50:05.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.118+0000 7f461f400640 1 --2- 172.21.15.131:0/2502627175 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f45fc06be60 0x7f45fc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:05.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.119+0000 7f461f400640 1 --2- 172.21.15.131:0/2502627175 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f45fc06be60 0x7f45fc06e320 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f461000a970 tx=0x7f4610008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:05.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.123+0000 7f4617400640 1 -- 172.21.15.131:0/2502627175 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f461804b030 con 0x7f46200ad250 2024-05-26T22:50:05.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.323+0000 7f4626400640 1 -- 172.21.15.131:0/2502627175 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f462006e710 con 0x7f46200ad250 2024-05-26T22:50:05.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.326+0000 7f4617400640 1 -- 172.21.15.131:0/2502627175 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) ==== 74+0+16693 (secure 0 0 0) 0x7f461807a180 con 0x7f46200ad250 2024-05-26T22:50:05.326 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:50:05.327 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":33,"fsid":"46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f","created":"2024-05-26T22:47:57.288905+0000","modified":"2024-05-26T22:49:32.532828+0000","last_up_change":"2024-05-26T22:48:09.602207+0000","last_in_change":"2024-05-26T22:48:00.687341+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-26T22:48:10.463339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-05-26T22:49:28.813374+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-05-26T22:49:29.987297+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"814eaf62-a18d-494b-8e27-fd8fb88adea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6801","nonce":2350940051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6803","nonce":2350940051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6807","nonce":2350940051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6805","nonce":2350940051}]},"public_addr":"172.21.15.131:6801/2350940051","cluster_addr":"172.21.15.131:6803/2350940051","heartbeat_back_addr":"172.21.15.131:6807/2350940051","heartbeat_front_addr":"172.21.15.131:6805/2350940051","state":["exists","up"]},{"osd":1,"uuid":"42e66287-f6c6-4e26-af3c-6dfaa3536d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6825","nonce":1525528247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6827","nonce":1525528247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6831","nonce":1525528247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6829","nonce":1525528247}]},"public_addr":"172.21.15.131:6825/1525528247","cluster_addr":"172.21.15.131:6827/1525528247","heartbeat_back_addr":"172.21.15.131:6831/1525528247","heartbeat_front_addr":"172.21.15.131:6829/1525528247","state":["exists","up"]},{"osd":2,"uuid":"0da34673-04a2-4fd0-b57a-0ff2b679b043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6817","nonce":3627842210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6819","nonce":3627842210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6823","nonce":3627842210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6821","nonce":3627842210}]},"public_addr":"172.21.15.131:6817/3627842210","cluster_addr":"172.21.15.131:6819/3627842210","heartbeat_back_addr":"172.21.15.131:6823/3627842210","heartbeat_front_addr":"172.21.15.131:6821/3627842210","state":["exists","up"]},{"osd":3,"uuid":"cf117a27-d6b2-43d3-ae52-b47988bf8eed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6809","nonce":350790591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6811","nonce":350790591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6815","nonce":350790591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6813","nonce":350790591}]},"public_addr":"172.21.15.131:6809/350790591","cluster_addr":"172.21.15.131:6811/350790591","heartbeat_back_addr":"172.21.15.131:6815/350790591","heartbeat_front_addr":"172.21.15.131:6813/350790591","state":["exists","up"]},{"osd":4,"uuid":"a092ccfc-be89-4a84-88d5-0f781fa715eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6816","nonce":244560995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6819","nonce":244560995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6823","nonce":244560995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6821","nonce":244560995}]},"public_addr":"172.21.15.187:6816/244560995","cluster_addr":"172.21.15.187:6819/244560995","heartbeat_back_addr":"172.21.15.187:6823/244560995","heartbeat_front_addr":"172.21.15.187:6821/244560995","state":["exists","up"]},{"osd":5,"uuid":"7513ba69-dd00-4a35-b190-9e8cc3b2bdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6801","nonce":324619398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6803","nonce":324619398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6812","nonce":324619398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6807","nonce":324619398}]},"public_addr":"172.21.15.187:6801/324619398","cluster_addr":"172.21.15.187:6803/324619398","heartbeat_back_addr":"172.21.15.187:6812/324619398","heartbeat_front_addr":"172.21.15.187:6807/324619398","state":["exists","up"]},{"osd":6,"uuid":"50081810-c093-4d2f-a4d7-b7c695c5400d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6806","nonce":1391347786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6809","nonce":1391347786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6818","nonce":1391347786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6813","nonce":1391347786}]},"public_addr":"172.21.15.187:6806/1391347786","cluster_addr":"172.21.15.187:6809/1391347786","heartbeat_back_addr":"172.21.15.187:6818/1391347786","heartbeat_front_addr":"172.21.15.187:6813/1391347786","state":["exists","up"]},{"osd":7,"uuid":"775dae05-54f5-4ad4-bbce-92863c6ed2ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6825","nonce":1220065622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6827","nonce":1220065622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6831","nonce":1220065622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6829","nonce":1220065622}]},"public_addr":"172.21.15.187:6825/1220065622","cluster_addr":"172.21.15.187:6827/1220065622","heartbeat_back_addr":"172.21.15.187:6831/1220065622","heartbeat_front_addr":"172.21.15.187:6829/1220065622","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:05.983520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.299516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.067323+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.307748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.018742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.258775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.010912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.295355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-26T22:50:05.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.329+0000 7f4626400640 1 -- 172.21.15.131:0/2502627175 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f45fc06be60 msgr2=0x7f45fc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:05.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.329+0000 7f4626400640 1 --2- 172.21.15.131:0/2502627175 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f45fc06be60 0x7f45fc06e320 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f461000a970 tx=0x7f4610008040 comp rx=0 tx=0).stop 2024-05-26T22:50:05.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.329+0000 7f4626400640 1 -- 172.21.15.131:0/2502627175 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f46200ad250 msgr2=0x7f46200b0010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:05.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.329+0000 7f4626400640 1 --2- 172.21.15.131:0/2502627175 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f46200ad250 0x7f46200b0010 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f46180090b0 tx=0x7f46180046c0 comp rx=0 tx=0).stop 2024-05-26T22:50:05.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.329+0000 7f4626400640 1 -- 172.21.15.131:0/2502627175 shutdown_connections 2024-05-26T22:50:05.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.329+0000 7f4626400640 1 --2- 172.21.15.131:0/2502627175 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f462006f200 0x7f46201f7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:05.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.329+0000 7f4626400640 1 --2- 172.21.15.131:0/2502627175 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f45fc06be60 0x7f45fc06e320 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:05.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.329+0000 7f4626400640 1 --2- 172.21.15.131:0/2502627175 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f46200ad250 0x7f46200b0010 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:05.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.329+0000 7f4626400640 1 --2- 172.21.15.131:0/2502627175 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f462006ee20 0x7f46201f7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:05.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.329+0000 7f4626400640 1 -- 172.21.15.131:0/2502627175 >> 172.21.15.131:0/2502627175 conn(0x7f46200b1e90 msgr2=0x7f462015afe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:05.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.329+0000 7f4626400640 1 -- 172.21.15.131:0/2502627175 shutdown_connections 2024-05-26T22:50:05.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.330+0000 7f4626400640 1 -- 172.21.15.131:0/2502627175 wait complete. 2024-05-26T22:50:05.348 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-26T22:48:10.463339+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-26T22:49:28.813374+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-05-26T22:49:29.987297+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-26T22:50:05.349 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-26T22:50:05.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.552+0000 7f62f1400640 1 Processor -- start 2024-05-26T22:50:05.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.552+0000 7f62f1400640 1 -- start start 2024-05-26T22:50:05.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.554+0000 7f62f1400640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f62ec0712d0 0x7f62ec06d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:05.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.554+0000 7f62f1400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f62ec06d790 0x7f62ec0cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:05.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.554+0000 7f62f1400640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62ec072d60 con 0x7f62ec145a10 2024-05-26T22:50:05.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.554+0000 7f62f1400640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f62ec06dc00 con 0x7f62ec0712d0 2024-05-26T22:50:05.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.554+0000 7f62f1400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f62ec15c760 con 0x7f62ec06d790 2024-05-26T22:50:05.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.554+0000 7f62ebe00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f62ec145a10 0x7f62ec072180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:36310/0 (socket says 172.21.15.131:36310) 2024-05-26T22:50:05.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.554+0000 7f62ebe00640 1 -- 172.21.15.131:0/622487830 learned_addr learned my addr 172.21.15.131:0/622487830 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:50:05.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.554+0000 7f62eb400640 1 --2- 172.21.15.131:0/622487830 >> v2:172.21.15.187:3300/0 conn(0x7f62ec0712d0 0x7f62ec06d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:05.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.554+0000 7f62f0a00640 1 --2- 172.21.15.131:0/622487830 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f62ec06d790 0x7f62ec0cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:05.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.554+0000 7f62eaa00640 1 -- 172.21.15.131:0/622487830 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 430681887 0 0) 0x7f62ec072d60 con 0x7f62ec145a10 2024-05-26T22:50:05.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.554+0000 7f62eaa00640 1 -- 172.21.15.131:0/622487830 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f62d0003520 con 0x7f62ec145a10 2024-05-26T22:50:05.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.555+0000 7f62eaa00640 1 -- 172.21.15.131:0/622487830 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 821074876 0 0) 0x7f62d0003520 con 0x7f62ec145a10 2024-05-26T22:50:05.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.555+0000 7f62eaa00640 1 -- 172.21.15.131:0/622487830 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f62ec06d790 msgr2=0x7f62ec0cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:05.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.555+0000 7f62eaa00640 1 --2- 172.21.15.131:0/622487830 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f62ec06d790 0x7f62ec0cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:05.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.555+0000 7f62eaa00640 1 -- 172.21.15.131:0/622487830 >> v2:172.21.15.187:3300/0 conn(0x7f62ec0712d0 msgr2=0x7f62ec06d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:05.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.555+0000 7f62eaa00640 1 --2- 172.21.15.131:0/622487830 >> v2:172.21.15.187:3300/0 conn(0x7f62ec0712d0 0x7f62ec06d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:05.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.555+0000 7f62eaa00640 1 -- 172.21.15.131:0/622487830 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62ec072a80 con 0x7f62ec145a10 2024-05-26T22:50:05.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.555+0000 7f62eaa00640 1 -- 172.21.15.131:0/622487830 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7f62dc001670 con 0x7f62ec145a10 2024-05-26T22:50:05.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.555+0000 7f62eaa00640 1 -- 172.21.15.131:0/622487830 >> v1:172.21.15.131:6789/0 conn(0x7f62ec145a10 legacy=0x7f62ec072180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:05.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.555+0000 7f62eaa00640 1 --2- 172.21.15.131:0/622487830 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f62d00039b0 0x7f62d0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:05.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.555+0000 7f62eaa00640 1 --2- 172.21.15.131:0/622487830 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f62d0004800 0x7f62d0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:05.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.555+0000 7f62eaa00640 1 --2- 172.21.15.131:0/622487830 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f62ec06d790 0x7f62d00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:05.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.555+0000 7f62eaa00640 1 -- 172.21.15.131:0/622487830 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f62ec06dc00 con 0x7f62d00039b0 2024-05-26T22:50:05.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.555+0000 7f62eaa00640 1 -- 172.21.15.131:0/622487830 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f62ec15c760 con 0x7f62d0004800 2024-05-26T22:50:05.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.555+0000 7f62eaa00640 1 -- 172.21.15.131:0/622487830 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f62d000fdd0 con 0x7f62ec06d790 2024-05-26T22:50:05.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.555+0000 7f62ebe00640 1 --2- 172.21.15.131:0/622487830 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f62d00039b0 0x7f62d0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:05.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.556+0000 7f62eb400640 1 --2- 172.21.15.131:0/622487830 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f62d0004800 0x7f62d0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:05.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.556+0000 7f62f0a00640 1 --2- 172.21.15.131:0/622487830 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f62ec06d790 0x7f62d00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:05.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.556+0000 7f62ebe00640 1 -- 172.21.15.131:0/622487830 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f62ec06d790 msgr2=0x7f62d00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:05.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.556+0000 7f62ebe00640 1 --2- 172.21.15.131:0/622487830 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f62ec06d790 0x7f62d00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:05.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.556+0000 7f62ebe00640 1 -- 172.21.15.131:0/622487830 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f62d0004800 msgr2=0x7f62d0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:05.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.556+0000 7f62ebe00640 1 --2- 172.21.15.131:0/622487830 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f62d0004800 0x7f62d0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:05.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.556+0000 7f62ebe00640 1 -- 172.21.15.131:0/622487830 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62d0003520 con 0x7f62d00039b0 2024-05-26T22:50:05.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.557+0000 7f62ebe00640 1 --2- 172.21.15.131:0/622487830 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f62d00039b0 0x7f62d0003dd0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f62dc00c190 tx=0x7f62d0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69fd51e544239028 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:05.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.557+0000 7f62eaa00640 1 -- 172.21.15.131:0/622487830 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62dc007c80 con 0x7f62d00039b0 2024-05-26T22:50:05.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.557+0000 7f62eaa00640 1 -- 172.21.15.131:0/622487830 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62dc02e070 con 0x7f62d00039b0 2024-05-26T22:50:05.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.557+0000 7f62eaa00640 1 -- 172.21.15.131:0/622487830 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62dc005d30 con 0x7f62d00039b0 2024-05-26T22:50:05.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.558+0000 7f62f1400640 1 -- 172.21.15.131:0/622487830 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f62d00039b0 msgr2=0x7f62d0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:05.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.558+0000 7f62f1400640 1 --2- 172.21.15.131:0/622487830 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f62d00039b0 0x7f62d0003dd0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f62dc00c190 tx=0x7f62d0005830 comp rx=0 tx=0).stop 2024-05-26T22:50:05.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.558+0000 7f62f0a00640 1 -- 172.21.15.131:0/622487830 reap_dead start 2024-05-26T22:50:05.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.558+0000 7f62f1400640 1 -- 172.21.15.131:0/622487830 shutdown_connections 2024-05-26T22:50:05.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.558+0000 7f62f1400640 1 -- 172.21.15.131:0/622487830 >> 172.21.15.131:0/622487830 conn(0x7f62ec0b5660 msgr2=0x7f62ec0b5a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:05.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.558+0000 7f62f1400640 1 -- 172.21.15.131:0/622487830 shutdown_connections 2024-05-26T22:50:05.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.558+0000 7f62f1400640 1 -- 172.21.15.131:0/622487830 wait complete. 2024-05-26T22:50:05.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.560+0000 7f62f1400640 1 Processor -- start 2024-05-26T22:50:05.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.560+0000 7f62f1400640 1 -- start start 2024-05-26T22:50:05.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.561+0000 7f62f1400640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f62d00039b0 0x7f62ec1f26b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:05.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.561+0000 7f62f1400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f62ec15e6f0 0x7f62ec1f0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:05.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.561+0000 7f62f1400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f62ec1f06d0 0x7f62ec0b30c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:05.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.561+0000 7f62f1400640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f62ec0cf070 con 0x7f62ec15e6f0 2024-05-26T22:50:05.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.561+0000 7f62f1400640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f62ec0ced90 con 0x7f62d00039b0 2024-05-26T22:50:05.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.561+0000 7f62f1400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f62ec145df0 con 0x7f62ec1f06d0 2024-05-26T22:50:05.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.562+0000 7f62eb400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f62ec15e6f0 0x7f62ec1f0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:05.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.562+0000 7f62eb400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f62ec15e6f0 0x7f62ec1f0190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37658/0 (socket says 172.21.15.131:37658) 2024-05-26T22:50:05.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.562+0000 7f62eb400640 1 -- 172.21.15.131:0/33408914 learned_addr learned my addr 172.21.15.131:0/33408914 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:05.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.562+0000 7f62f0a00640 1 --2- 172.21.15.131:0/33408914 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f62ec1f06d0 0x7f62ec0b30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:05.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.562+0000 7f62ebe00640 1 --2- 172.21.15.131:0/33408914 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f62d00039b0 0x7f62ec1f26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:05.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.562+0000 7f62eb400640 1 -- 172.21.15.131:0/33408914 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f62ec1f06d0 msgr2=0x7f62ec0b30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:05.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.562+0000 7f62eb400640 1 --2- 172.21.15.131:0/33408914 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f62ec1f06d0 0x7f62ec0b30c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:05.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.562+0000 7f62eb400640 1 -- 172.21.15.131:0/33408914 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f62d00039b0 msgr2=0x7f62ec1f26b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:05.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.562+0000 7f62eb400640 1 --2- 172.21.15.131:0/33408914 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f62d00039b0 0x7f62ec1f26b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:05.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.562+0000 7f62eb400640 1 -- 172.21.15.131:0/33408914 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62ec15eb80 con 0x7f62ec15e6f0 2024-05-26T22:50:05.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.562+0000 7f62ebe00640 1 --2- 172.21.15.131:0/33408914 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f62d00039b0 0x7f62ec1f26b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:50:05.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.562+0000 7f62f0a00640 1 --2- 172.21.15.131:0/33408914 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f62ec1f06d0 0x7f62ec0b30c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:50:05.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.562+0000 7f62eb400640 1 --2- 172.21.15.131:0/33408914 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f62ec15e6f0 0x7f62ec1f0190 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f62d8011590 tx=0x7f62d8011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:05.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.563+0000 7f62e3e00640 1 -- 172.21.15.131:0/33408914 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62d8004220 con 0x7f62ec15e6f0 2024-05-26T22:50:05.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.563+0000 7f62e3e00640 1 -- 172.21.15.131:0/33408914 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62d80043c0 con 0x7f62ec15e6f0 2024-05-26T22:50:05.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.563+0000 7f62e3e00640 1 -- 172.21.15.131:0/33408914 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62d8015870 con 0x7f62ec15e6f0 2024-05-26T22:50:05.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.563+0000 7f62f1400640 1 -- 172.21.15.131:0/33408914 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62ec15ee60 con 0x7f62ec15e6f0 2024-05-26T22:50:05.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.563+0000 7f62f1400640 1 -- 172.21.15.131:0/33408914 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62ec15f420 con 0x7f62ec15e6f0 2024-05-26T22:50:05.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.564+0000 7f62f1400640 1 -- 172.21.15.131:0/33408914 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62ec0694e0 con 0x7f62ec15e6f0 2024-05-26T22:50:05.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.565+0000 7f62e3e00640 1 -- 172.21.15.131:0/33408914 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f62d8015a10 con 0x7f62ec15e6f0 2024-05-26T22:50:05.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.566+0000 7f62e3e00640 1 --2- 172.21.15.131:0/33408914 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f62cc06be60 0x7f62cc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:05.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.566+0000 7f62e3e00640 1 -- 172.21.15.131:0/33408914 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f62d808afd0 con 0x7f62ec15e6f0 2024-05-26T22:50:05.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.566+0000 7f62ebe00640 1 --2- 172.21.15.131:0/33408914 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f62cc06be60 0x7f62cc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:05.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.567+0000 7f62ebe00640 1 --2- 172.21.15.131:0/33408914 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f62cc06be60 0x7f62cc06e320 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f62dc032000 tx=0x7f62dc007000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:05.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.571+0000 7f62e3e00640 1 -- 172.21.15.131:0/33408914 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f62d8019030 con 0x7f62ec15e6f0 2024-05-26T22:50:05.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.772+0000 7f62f1400640 1 -- 172.21.15.131:0/33408914 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f62ec0cda60 con 0x7f62ec15e6f0 2024-05-26T22:50:05.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.773+0000 7f62e3e00640 1 -- 172.21.15.131:0/33408914 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v33) ==== 93+0+10 (secure 0 0 0) 0x7f62d8059970 con 0x7f62ec15e6f0 2024-05-26T22:50:05.773 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2024-05-26T22:50:05.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.776+0000 7f62f1400640 1 -- 172.21.15.131:0/33408914 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f62cc06be60 msgr2=0x7f62cc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:05.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.776+0000 7f62f1400640 1 --2- 172.21.15.131:0/33408914 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f62cc06be60 0x7f62cc06e320 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f62dc032000 tx=0x7f62dc007000 comp rx=0 tx=0).stop 2024-05-26T22:50:05.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.776+0000 7f62f1400640 1 -- 172.21.15.131:0/33408914 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f62ec15e6f0 msgr2=0x7f62ec1f0190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:05.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.776+0000 7f62f1400640 1 --2- 172.21.15.131:0/33408914 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f62ec15e6f0 0x7f62ec1f0190 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f62d8011590 tx=0x7f62d8011a50 comp rx=0 tx=0).stop 2024-05-26T22:50:05.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.776+0000 7f62f1400640 1 -- 172.21.15.131:0/33408914 shutdown_connections 2024-05-26T22:50:05.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.776+0000 7f62f1400640 1 --2- 172.21.15.131:0/33408914 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f62ec1f06d0 0x7f62ec0b30c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:05.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.776+0000 7f62f1400640 1 --2- 172.21.15.131:0/33408914 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f62cc06be60 0x7f62cc06e320 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:05.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.776+0000 7f62f1400640 1 --2- 172.21.15.131:0/33408914 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f62ec15e6f0 0x7f62ec1f0190 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:05.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.776+0000 7f62f1400640 1 --2- 172.21.15.131:0/33408914 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f62d00039b0 0x7f62ec1f26b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:05.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.776+0000 7f62f1400640 1 -- 172.21.15.131:0/33408914 >> 172.21.15.131:0/33408914 conn(0x7f62ec0b5660 msgr2=0x7f62ec0b14b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:05.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.776+0000 7f62f1400640 1 -- 172.21.15.131:0/33408914 shutdown_connections 2024-05-26T22:50:05.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.776+0000 7f62f1400640 1 -- 172.21.15.131:0/33408914 wait complete. 2024-05-26T22:50:05.793 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-26T22:50:05.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.986+0000 7fc121800640 1 Processor -- start 2024-05-26T22:50:05.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.986+0000 7fc121800640 1 -- start start 2024-05-26T22:50:05.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.988+0000 7fc121800640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fc11c0d0460 0x7fc11c06d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:05.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.988+0000 7fc121800640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc11c160180 con 0x7fc11c0b5450 2024-05-26T22:50:05.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.988+0000 7fc121800640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc11c160460 con 0x7fc11c06f420 2024-05-26T22:50:05.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.988+0000 7fc121800640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fc11c160d90 con 0x7fc11c0d0460 2024-05-26T22:50:05.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.988+0000 7fc11b400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fc11c0d0460 0x7fc11c06d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:05.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.988+0000 7fc11aa00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fc11c0b5450 0x7fc11c0b43c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:36320/0 (socket says 172.21.15.131:36320) 2024-05-26T22:50:05.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.988+0000 7fc11aa00640 1 -- 172.21.15.131:0/1872665145 learned_addr learned my addr 172.21.15.131:0/1872665145 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:50:05.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.989+0000 7fc11b400640 1 -- 172.21.15.131:0/1872665145 >> v1:172.21.15.187:6789/0 conn(0x7fc11c06f420 legacy=0x7fc11c0cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:05.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.989+0000 7fc11b400640 1 -- 172.21.15.131:0/1872665145 >> v1:172.21.15.131:6789/0 conn(0x7fc11c0b5450 legacy=0x7fc11c0b43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:05.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.989+0000 7fc11b400640 1 -- 172.21.15.131:0/1872665145 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc11c160740 con 0x7fc11c0d0460 2024-05-26T22:50:05.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.989+0000 7fc11a000640 1 -- 172.21.15.131:0/1872665145 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2403464776 0 0) 0x7fc11c160180 con 0x7fc11c0b5450 2024-05-26T22:50:05.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.990+0000 7fc11b400640 1 --2- 172.21.15.131:0/1872665145 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc11c0d0460 0x7fc11c06d600 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fc1140097a0 tx=0x7fc11402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59e7b4338e331c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:05.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.990+0000 7fc11a000640 1 -- 172.21.15.131:0/1872665145 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc114037070 con 0x7fc11c0d0460 2024-05-26T22:50:05.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.990+0000 7fc11a000640 1 -- 172.21.15.131:0/1872665145 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc114032440 con 0x7fc11c0d0460 2024-05-26T22:50:05.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.990+0000 7fc11a000640 1 -- 172.21.15.131:0/1872665145 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc114032810 con 0x7fc11c0d0460 2024-05-26T22:50:05.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.991+0000 7fc121800640 1 -- 172.21.15.131:0/1872665145 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc11c0d0460 msgr2=0x7fc11c06d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:05.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.991+0000 7fc121800640 1 --2- 172.21.15.131:0/1872665145 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc11c0d0460 0x7fc11c06d600 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fc1140097a0 tx=0x7fc11402a1e0 comp rx=0 tx=0).stop 2024-05-26T22:50:05.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.991+0000 7fc121800640 1 -- 172.21.15.131:0/1872665145 shutdown_connections 2024-05-26T22:50:05.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.991+0000 7fc121800640 1 --2- 172.21.15.131:0/1872665145 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc11c0d0460 0x7fc11c06d600 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:05.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.991+0000 7fc121800640 1 -- 172.21.15.131:0/1872665145 >> 172.21.15.131:0/1872665145 conn(0x7fc11c06bc90 msgr2=0x7fc11c0b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:05.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.991+0000 7fc121800640 1 -- 172.21.15.131:0/1872665145 shutdown_connections 2024-05-26T22:50:05.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.991+0000 7fc121800640 1 -- 172.21.15.131:0/1872665145 wait complete. 2024-05-26T22:50:05.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.993+0000 7fc121800640 1 Processor -- start 2024-05-26T22:50:05.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.993+0000 7fc121800640 1 -- start start 2024-05-26T22:50:05.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.994+0000 7fc121800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc11c06f420 0x7fc11c1469d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:05.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.994+0000 7fc121800640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc11c0b5450 0x7fc11c1f2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:05.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.994+0000 7fc121800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc11c0d0460 0x7fc11c155fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:05.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.994+0000 7fc121800640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc11c15fde0 con 0x7fc11c06f420 2024-05-26T22:50:05.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.994+0000 7fc121800640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc11c0b5830 con 0x7fc11c0b5450 2024-05-26T22:50:05.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.994+0000 7fc121800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc11c15ff60 con 0x7fc11c0d0460 2024-05-26T22:50:05.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.994+0000 7fc11aa00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc11c06f420 0x7fc11c1469d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:05.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.994+0000 7fc11aa00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc11c06f420 0x7fc11c1469d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37666/0 (socket says 172.21.15.131:37666) 2024-05-26T22:50:05.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.994+0000 7fc11aa00640 1 -- 172.21.15.131:0/1444524432 learned_addr learned my addr 172.21.15.131:0/1444524432 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:05.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.995+0000 7fc112000640 1 --2- 172.21.15.131:0/1444524432 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc11c0b5450 0x7fc11c1f2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:05.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.995+0000 7fc11aa00640 1 -- 172.21.15.131:0/1444524432 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc11c0d0460 msgr2=0x7fc11c155fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:05.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.995+0000 7fc11b400640 1 --2- 172.21.15.131:0/1444524432 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc11c0d0460 0x7fc11c155fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:05.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.995+0000 7fc11aa00640 1 --2- 172.21.15.131:0/1444524432 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc11c0d0460 0x7fc11c155fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:05.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.995+0000 7fc11aa00640 1 -- 172.21.15.131:0/1444524432 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc11c0b5450 msgr2=0x7fc11c1f2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:05.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.995+0000 7fc11aa00640 1 --2- 172.21.15.131:0/1444524432 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc11c0b5450 0x7fc11c1f2dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:05.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.995+0000 7fc11aa00640 1 -- 172.21.15.131:0/1444524432 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc11c1f3300 con 0x7fc11c06f420 2024-05-26T22:50:05.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.995+0000 7fc112000640 1 --2- 172.21.15.131:0/1444524432 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc11c0b5450 0x7fc11c1f2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:50:05.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.995+0000 7fc11aa00640 1 --2- 172.21.15.131:0/1444524432 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc11c06f420 0x7fc11c1469d0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fc10c00c890 tx=0x7fc10c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:05.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.995+0000 7fc113400640 1 -- 172.21.15.131:0/1444524432 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc10c018070 con 0x7fc11c06f420 2024-05-26T22:50:05.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.995+0000 7fc113400640 1 -- 172.21.15.131:0/1444524432 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc10c00a640 con 0x7fc11c06f420 2024-05-26T22:50:05.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.995+0000 7fc113400640 1 -- 172.21.15.131:0/1444524432 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc10c00aa30 con 0x7fc11c06f420 2024-05-26T22:50:05.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.995+0000 7fc121800640 1 -- 172.21.15.131:0/1444524432 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc11c154f30 con 0x7fc11c06f420 2024-05-26T22:50:05.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.996+0000 7fc121800640 1 -- 172.21.15.131:0/1444524432 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc11c1554f0 con 0x7fc11c06f420 2024-05-26T22:50:05.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.997+0000 7fc121800640 1 -- 172.21.15.131:0/1444524432 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc11c074e90 con 0x7fc11c06f420 2024-05-26T22:50:05.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.998+0000 7fc113400640 1 -- 172.21.15.131:0/1444524432 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc10c00abd0 con 0x7fc11c06f420 2024-05-26T22:50:05.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.999+0000 7fc113400640 1 --2- 172.21.15.131:0/1444524432 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc0f806be30 0x7fc0f806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:05.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.999+0000 7fc113400640 1 -- 172.21.15.131:0/1444524432 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fc10c08a510 con 0x7fc11c06f420 2024-05-26T22:50:05.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:05.999+0000 7fc112000640 1 --2- 172.21.15.131:0/1444524432 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc0f806be30 0x7fc0f806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:05.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.000+0000 7fc112000640 1 --2- 172.21.15.131:0/1444524432 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc0f806be30 0x7fc0f806e2f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fc11c153d90 tx=0x7fc104008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:06.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.003+0000 7fc113400640 1 -- 172.21.15.131:0/1444524432 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fc11c1f3300 con 0x7fc11c06f420 2024-05-26T22:50:06.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.200+0000 7fc121800640 1 -- 172.21.15.131:0/1444524432 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc11c0cee90 con 0x7fc11c06f420 2024-05-26T22:50:06.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.201+0000 7fc113400640 1 -- 172.21.15.131:0/1444524432 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v33) ==== 104+0+11 (secure 0 0 0) 0x7fc10c058e20 con 0x7fc11c06f420 2024-05-26T22:50:06.201 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-05-26T22:50:06.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.204+0000 7fc121800640 1 -- 172.21.15.131:0/1444524432 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc0f806be30 msgr2=0x7fc0f806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:06.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.204+0000 7fc121800640 1 --2- 172.21.15.131:0/1444524432 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc0f806be30 0x7fc0f806e2f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fc11c153d90 tx=0x7fc104008040 comp rx=0 tx=0).stop 2024-05-26T22:50:06.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.204+0000 7fc121800640 1 -- 172.21.15.131:0/1444524432 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc11c06f420 msgr2=0x7fc11c1469d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:06.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.204+0000 7fc121800640 1 --2- 172.21.15.131:0/1444524432 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc11c06f420 0x7fc11c1469d0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fc10c00c890 tx=0x7fc10c00cd50 comp rx=0 tx=0).stop 2024-05-26T22:50:06.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.204+0000 7fc121800640 1 -- 172.21.15.131:0/1444524432 shutdown_connections 2024-05-26T22:50:06.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.204+0000 7fc121800640 1 --2- 172.21.15.131:0/1444524432 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc11c0d0460 0x7fc11c155fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:06.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.204+0000 7fc121800640 1 --2- 172.21.15.131:0/1444524432 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc0f806be30 0x7fc0f806e2f0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:06.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.204+0000 7fc121800640 1 --2- 172.21.15.131:0/1444524432 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc11c0b5450 0x7fc11c1f2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:06.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.204+0000 7fc121800640 1 --2- 172.21.15.131:0/1444524432 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc11c06f420 0x7fc11c1469d0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:06.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.204+0000 7fc121800640 1 -- 172.21.15.131:0/1444524432 >> 172.21.15.131:0/1444524432 conn(0x7fc11c06bc90 msgr2=0x7fc11c15d9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:06.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.204+0000 7fc121800640 1 -- 172.21.15.131:0/1444524432 shutdown_connections 2024-05-26T22:50:06.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.204+0000 7fc121800640 1 -- 172.21.15.131:0/1444524432 wait complete. 2024-05-26T22:50:06.221 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-26T22:50:06.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.423+0000 7fdd62200640 1 Processor -- start 2024-05-26T22:50:06.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.424+0000 7fdd62200640 1 -- start start 2024-05-26T22:50:06.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.425+0000 7fdd62200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fdd5c06d250 0x7fdd5c06d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:06.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.425+0000 7fdd62200640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd5c06db70 con 0x7fdd5c1400c0 2024-05-26T22:50:06.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.425+0000 7fdd62200640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd5c13e820 con 0x7fdd5c15b730 2024-05-26T22:50:06.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.425+0000 7fdd62200640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fdd5c071260 con 0x7fdd5c06d250 2024-05-26T22:50:06.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.425+0000 7fdd60e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fdd5c06d250 0x7fdd5c06d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:06.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.425+0000 7fdd60e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fdd5c06d250 0x7fdd5c06d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37676/0 (socket says 172.21.15.131:37676) 2024-05-26T22:50:06.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.425+0000 7fdd60e00640 1 -- 172.21.15.131:0/3202479530 learned_addr learned my addr 172.21.15.131:0/3202479530 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:06.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.426+0000 7fdd60e00640 1 -- 172.21.15.131:0/3202479530 >> v1:172.21.15.187:6789/0 conn(0x7fdd5c15b730 legacy=0x7fdd5c15bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:06.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.426+0000 7fdd60e00640 1 -- 172.21.15.131:0/3202479530 >> v1:172.21.15.131:6789/0 conn(0x7fdd5c1400c0 legacy=0x7fdd5c1404f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:06.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.426+0000 7fdd60e00640 1 -- 172.21.15.131:0/3202479530 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd5c15c4a0 con 0x7fdd5c06d250 2024-05-26T22:50:06.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.427+0000 7fdd60e00640 1 --2- 172.21.15.131:0/3202479530 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd5c06d250 0x7fdd5c06d630 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c009870 tx=0x7fdd4c02a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6f260ec29abb1f server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:06.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.427+0000 7fdd5b400640 1 -- 172.21.15.131:0/3202479530 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd4c037070 con 0x7fdd5c06d250 2024-05-26T22:50:06.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.427+0000 7fdd5b400640 1 -- 172.21.15.131:0/3202479530 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd4c032440 con 0x7fdd5c06d250 2024-05-26T22:50:06.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.427+0000 7fdd5b400640 1 -- 172.21.15.131:0/3202479530 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd4c032810 con 0x7fdd5c06d250 2024-05-26T22:50:06.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.428+0000 7fdd62200640 1 -- 172.21.15.131:0/3202479530 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd5c06d250 msgr2=0x7fdd5c06d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:06.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.428+0000 7fdd62200640 1 --2- 172.21.15.131:0/3202479530 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd5c06d250 0x7fdd5c06d630 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c009870 tx=0x7fdd4c02a2a0 comp rx=0 tx=0).stop 2024-05-26T22:50:06.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.428+0000 7fdd62200640 1 -- 172.21.15.131:0/3202479530 shutdown_connections 2024-05-26T22:50:06.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.428+0000 7fdd62200640 1 --2- 172.21.15.131:0/3202479530 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd5c06d250 0x7fdd5c06d630 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:06.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.428+0000 7fdd62200640 1 -- 172.21.15.131:0/3202479530 >> 172.21.15.131:0/3202479530 conn(0x7fdd5c0b5660 msgr2=0x7fdd5c0b5a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:06.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.428+0000 7fdd62200640 1 -- 172.21.15.131:0/3202479530 shutdown_connections 2024-05-26T22:50:06.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.428+0000 7fdd62200640 1 -- 172.21.15.131:0/3202479530 wait complete. 2024-05-26T22:50:06.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.430+0000 7fdd62200640 1 Processor -- start 2024-05-26T22:50:06.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.430+0000 7fdd62200640 1 -- start start 2024-05-26T22:50:06.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.431+0000 7fdd62200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd5c06d250 0x7fdd5c1f2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:06.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.431+0000 7fdd62200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdd5c1400c0 0x7fdd5c0cfeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:06.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.431+0000 7fdd62200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdd5c15b730 0x7fdd5c157a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:06.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.431+0000 7fdd62200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fdd5c071a70 con 0x7fdd5c06d250 2024-05-26T22:50:06.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.431+0000 7fdd62200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fdd5c071790 con 0x7fdd5c1400c0 2024-05-26T22:50:06.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.431+0000 7fdd62200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fdd5c072fa0 con 0x7fdd5c15b730 2024-05-26T22:50:06.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.431+0000 7fdd60e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd5c06d250 0x7fdd5c1f2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:06.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.431+0000 7fdd60e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd5c06d250 0x7fdd5c1f2eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37690/0 (socket says 172.21.15.131:37690) 2024-05-26T22:50:06.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.431+0000 7fdd60e00640 1 -- 172.21.15.131:0/1240038525 learned_addr learned my addr 172.21.15.131:0/1240038525 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:06.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.432+0000 7fdd60e00640 1 -- 172.21.15.131:0/1240038525 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdd5c15b730 msgr2=0x7fdd5c157a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:06.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.432+0000 7fdd60e00640 1 --2- 172.21.15.131:0/1240038525 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdd5c15b730 0x7fdd5c157a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:06.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.432+0000 7fdd60e00640 1 -- 172.21.15.131:0/1240038525 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdd5c1400c0 msgr2=0x7fdd5c0cfeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:06.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.432+0000 7fdd60e00640 1 --2- 172.21.15.131:0/1240038525 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdd5c1400c0 0x7fdd5c0cfeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:06.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.432+0000 7fdd60e00640 1 -- 172.21.15.131:0/1240038525 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd5c1560f0 con 0x7fdd5c06d250 2024-05-26T22:50:06.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.432+0000 7fdd60e00640 1 --2- 172.21.15.131:0/1240038525 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd5c06d250 0x7fdd5c1f2eb0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c009870 tx=0x7fdd4c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:06.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.432+0000 7fdd58c00640 1 -- 172.21.15.131:0/1240038525 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd4c041020 con 0x7fdd5c06d250 2024-05-26T22:50:06.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.432+0000 7fdd58c00640 1 -- 172.21.15.131:0/1240038525 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd4c030970 con 0x7fdd5c06d250 2024-05-26T22:50:06.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.432+0000 7fdd62200640 1 -- 172.21.15.131:0/1240038525 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd5c1563d0 con 0x7fdd5c06d250 2024-05-26T22:50:06.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.432+0000 7fdd58c00640 1 -- 172.21.15.131:0/1240038525 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd4c037040 con 0x7fdd5c06d250 2024-05-26T22:50:06.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.432+0000 7fdd62200640 1 -- 172.21.15.131:0/1240038525 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd5c156990 con 0x7fdd5c06d250 2024-05-26T22:50:06.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.433+0000 7fdd62200640 1 -- 172.21.15.131:0/1240038525 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd5c0694e0 con 0x7fdd5c06d250 2024-05-26T22:50:06.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.434+0000 7fdd58c00640 1 -- 172.21.15.131:0/1240038525 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdd4c044020 con 0x7fdd5c06d250 2024-05-26T22:50:06.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.435+0000 7fdd58c00640 1 --2- 172.21.15.131:0/1240038525 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fdd3806be30 0x7fdd3806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:06.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.436+0000 7fdd58c00640 1 -- 172.21.15.131:0/1240038525 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fdd4c0ab920 con 0x7fdd5c06d250 2024-05-26T22:50:06.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.436+0000 7fdd5be00640 1 --2- 172.21.15.131:0/1240038525 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fdd3806be30 0x7fdd3806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:06.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.437+0000 7fdd5be00640 1 --2- 172.21.15.131:0/1240038525 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fdd3806be30 0x7fdd3806e2f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c0b1ab0 tx=0x7fdd44009230 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:06.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.440+0000 7fdd58c00640 1 -- 172.21.15.131:0/1240038525 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fdd4c04b030 con 0x7fdd5c06d250 2024-05-26T22:50:06.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.641+0000 7fdd62200640 1 -- 172.21.15.131:0/1240038525 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fdd5c13eb30 con 0x7fdd5c06d250 2024-05-26T22:50:06.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.642+0000 7fdd58c00640 1 -- 172.21.15.131:0/1240038525 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v33) ==== 100+0+11 (secure 0 0 0) 0x7fdd4c07a230 con 0x7fdd5c06d250 2024-05-26T22:50:06.642 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 64 2024-05-26T22:50:06.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.645+0000 7fdd62200640 1 -- 172.21.15.131:0/1240038525 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fdd3806be30 msgr2=0x7fdd3806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:06.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.645+0000 7fdd62200640 1 --2- 172.21.15.131:0/1240038525 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fdd3806be30 0x7fdd3806e2f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c0b1ab0 tx=0x7fdd44009230 comp rx=0 tx=0).stop 2024-05-26T22:50:06.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.645+0000 7fdd62200640 1 -- 172.21.15.131:0/1240038525 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd5c06d250 msgr2=0x7fdd5c1f2eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:06.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.645+0000 7fdd62200640 1 --2- 172.21.15.131:0/1240038525 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd5c06d250 0x7fdd5c1f2eb0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c009870 tx=0x7fdd4c0046c0 comp rx=0 tx=0).stop 2024-05-26T22:50:06.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.645+0000 7fdd62200640 1 -- 172.21.15.131:0/1240038525 shutdown_connections 2024-05-26T22:50:06.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.645+0000 7fdd62200640 1 --2- 172.21.15.131:0/1240038525 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdd5c15b730 0x7fdd5c157a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:06.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.645+0000 7fdd62200640 1 --2- 172.21.15.131:0/1240038525 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fdd3806be30 0x7fdd3806e2f0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:06.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.645+0000 7fdd62200640 1 --2- 172.21.15.131:0/1240038525 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdd5c1400c0 0x7fdd5c0cfeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:06.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.645+0000 7fdd62200640 1 --2- 172.21.15.131:0/1240038525 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd5c06d250 0x7fdd5c1f2eb0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:06.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.645+0000 7fdd62200640 1 -- 172.21.15.131:0/1240038525 >> 172.21.15.131:0/1240038525 conn(0x7fdd5c0b5660 msgr2=0x7fdd5c159310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:06.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.645+0000 7fdd62200640 1 -- 172.21.15.131:0/1240038525 shutdown_connections 2024-05-26T22:50:06.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.645+0000 7fdd62200640 1 -- 172.21.15.131:0/1240038525 wait complete. 2024-05-26T22:50:06.664 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-26T22:50:06.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.867+0000 7fc793a00640 1 Processor -- start 2024-05-26T22:50:06.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.867+0000 7fc793a00640 1 -- start start 2024-05-26T22:50:06.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.868+0000 7fc793a00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fc78c0b5450 0x7fc78c0b43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:06.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.868+0000 7fc793a00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc78c0d0460 0x7fc78c06d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:06.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.868+0000 7fc793a00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc78c0b4900 con 0x7fc78c06f420 2024-05-26T22:50:06.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.869+0000 7fc793a00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fc78c160d90 con 0x7fc78c0b5450 2024-05-26T22:50:06.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.869+0000 7fc793a00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc78c160180 con 0x7fc78c0d0460 2024-05-26T22:50:06.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.869+0000 7fc78be00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fc78c06f420 0x7fc78c0cfd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:36336/0 (socket says 172.21.15.131:36336) 2024-05-26T22:50:06.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.869+0000 7fc78be00640 1 -- 172.21.15.131:0/994068191 learned_addr learned my addr 172.21.15.131:0/994068191 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:50:06.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.869+0000 7fc78b400640 1 -- 172.21.15.131:0/994068191 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2804315826 0 0) 0x7fc78c0b4900 con 0x7fc78c06f420 2024-05-26T22:50:06.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.869+0000 7fc791000640 1 --2- 172.21.15.131:0/994068191 >> v2:172.21.15.187:3300/0 conn(0x7fc78c0b5450 0x7fc78c0b43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:06.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.869+0000 7fc78b400640 1 -- 172.21.15.131:0/994068191 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc774003520 con 0x7fc78c06f420 2024-05-26T22:50:06.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.869+0000 7fc791a00640 1 --2- 172.21.15.131:0/994068191 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc78c0d0460 0x7fc78c06d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:06.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.870+0000 7fc78b400640 1 -- 172.21.15.131:0/994068191 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 56478242 0 0) 0x7fc774003520 con 0x7fc78c06f420 2024-05-26T22:50:06.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.870+0000 7fc78b400640 1 -- 172.21.15.131:0/994068191 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc78c0d0460 msgr2=0x7fc78c06d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:06.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.870+0000 7fc78b400640 1 --2- 172.21.15.131:0/994068191 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc78c0d0460 0x7fc78c06d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:06.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.870+0000 7fc78b400640 1 -- 172.21.15.131:0/994068191 >> v2:172.21.15.187:3300/0 conn(0x7fc78c0b5450 msgr2=0x7fc78c0b43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:06.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.870+0000 7fc78b400640 1 --2- 172.21.15.131:0/994068191 >> v2:172.21.15.187:3300/0 conn(0x7fc78c0b5450 0x7fc78c0b43c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:06.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.870+0000 7fc78b400640 1 -- 172.21.15.131:0/994068191 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc78c160460 con 0x7fc78c06f420 2024-05-26T22:50:06.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.870+0000 7fc78b400640 1 -- 172.21.15.131:0/994068191 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7fc778001670 con 0x7fc78c06f420 2024-05-26T22:50:06.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.870+0000 7fc78b400640 1 -- 172.21.15.131:0/994068191 >> v1:172.21.15.131:6789/0 conn(0x7fc78c06f420 legacy=0x7fc78c0cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:06.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.870+0000 7fc78b400640 1 --2- 172.21.15.131:0/994068191 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7740039b0 0x7fc774003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:06.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.870+0000 7fc78b400640 1 --2- 172.21.15.131:0/994068191 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc774004800 0x7fc774004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:06.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.870+0000 7fc78b400640 1 --2- 172.21.15.131:0/994068191 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc78c0d0460 0x7fc7740051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:06.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.870+0000 7fc78b400640 1 -- 172.21.15.131:0/994068191 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc78c160d90 con 0x7fc7740039b0 2024-05-26T22:50:06.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.870+0000 7fc78b400640 1 -- 172.21.15.131:0/994068191 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc78c160180 con 0x7fc774004800 2024-05-26T22:50:06.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.870+0000 7fc78b400640 1 -- 172.21.15.131:0/994068191 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc77400ba40 con 0x7fc78c0d0460 2024-05-26T22:50:06.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.870+0000 7fc791000640 1 --2- 172.21.15.131:0/994068191 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7740039b0 0x7fc774003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:06.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.870+0000 7fc78be00640 1 --2- 172.21.15.131:0/994068191 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc774004800 0x7fc774004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:06.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.870+0000 7fc791000640 1 -- 172.21.15.131:0/994068191 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc78c0d0460 msgr2=0x7fc7740051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:06.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.870+0000 7fc791000640 1 --2- 172.21.15.131:0/994068191 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc78c0d0460 0x7fc7740051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:06.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.870+0000 7fc791000640 1 -- 172.21.15.131:0/994068191 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc774004800 msgr2=0x7fc774004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:06.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.870+0000 7fc791000640 1 --2- 172.21.15.131:0/994068191 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc774004800 0x7fc774004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:06.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.870+0000 7fc791000640 1 -- 172.21.15.131:0/994068191 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc774003520 con 0x7fc7740039b0 2024-05-26T22:50:06.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.871+0000 7fc791000640 1 --2- 172.21.15.131:0/994068191 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7740039b0 0x7fc774003dd0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fc780009ac0 tx=0x7fc774005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56f4f8167dc499cc server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:06.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.872+0000 7fc78b400640 1 -- 172.21.15.131:0/994068191 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc780036070 con 0x7fc7740039b0 2024-05-26T22:50:06.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.872+0000 7fc78b400640 1 -- 172.21.15.131:0/994068191 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc780031400 con 0x7fc7740039b0 2024-05-26T22:50:06.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.872+0000 7fc78b400640 1 -- 172.21.15.131:0/994068191 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc780009690 con 0x7fc7740039b0 2024-05-26T22:50:06.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.872+0000 7fc793a00640 1 -- 172.21.15.131:0/994068191 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7740039b0 msgr2=0x7fc774003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:06.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.872+0000 7fc793a00640 1 --2- 172.21.15.131:0/994068191 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7740039b0 0x7fc774003dd0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fc780009ac0 tx=0x7fc774005830 comp rx=0 tx=0).stop 2024-05-26T22:50:06.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.872+0000 7fc791a00640 1 -- 172.21.15.131:0/994068191 reap_dead start 2024-05-26T22:50:06.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.872+0000 7fc793a00640 1 -- 172.21.15.131:0/994068191 shutdown_connections 2024-05-26T22:50:06.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.872+0000 7fc793a00640 1 -- 172.21.15.131:0/994068191 >> 172.21.15.131:0/994068191 conn(0x7fc78c06bc90 msgr2=0x7fc78c0b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:06.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.872+0000 7fc793a00640 1 -- 172.21.15.131:0/994068191 shutdown_connections 2024-05-26T22:50:06.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.872+0000 7fc793a00640 1 -- 172.21.15.131:0/994068191 wait complete. 2024-05-26T22:50:06.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.874+0000 7fc793a00640 1 Processor -- start 2024-05-26T22:50:06.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.874+0000 7fc793a00640 1 -- start start 2024-05-26T22:50:06.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.875+0000 7fc793a00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc7740039b0 0x7fc78c09cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:06.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.875+0000 7fc793a00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc78c1578f0 0x7fc78c146ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:06.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.875+0000 7fc793a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc78c15e910 0x7fc78c15edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:06.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.875+0000 7fc793a00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc78c0b5830 con 0x7fc78c15e910 2024-05-26T22:50:06.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.875+0000 7fc793a00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc78c160a20 con 0x7fc78c1578f0 2024-05-26T22:50:06.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.875+0000 7fc793a00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc78c15fde0 con 0x7fc7740039b0 2024-05-26T22:50:06.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.875+0000 7fc791a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc78c15e910 0x7fc78c15edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:06.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.876+0000 7fc791a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc78c15e910 0x7fc78c15edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37708/0 (socket says 172.21.15.131:37708) 2024-05-26T22:50:06.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.876+0000 7fc791a00640 1 -- 172.21.15.131:0/500746349 learned_addr learned my addr 172.21.15.131:0/500746349 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:06.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.876+0000 7fc791000640 1 --2- 172.21.15.131:0/500746349 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc7740039b0 0x7fc78c09cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:06.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.876+0000 7fc791a00640 1 -- 172.21.15.131:0/500746349 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc7740039b0 msgr2=0x7fc78c09cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:06.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.876+0000 7fc791a00640 1 --2- 172.21.15.131:0/500746349 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc7740039b0 0x7fc78c09cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:06.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.876+0000 7fc791a00640 1 -- 172.21.15.131:0/500746349 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc78c1578f0 msgr2=0x7fc78c146ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:06.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.876+0000 7fc791a00640 1 --2- 172.21.15.131:0/500746349 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc78c1578f0 0x7fc78c146ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:06.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.876+0000 7fc791a00640 1 -- 172.21.15.131:0/500746349 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc78c147010 con 0x7fc78c15e910 2024-05-26T22:50:06.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.876+0000 7fc791a00640 1 --2- 172.21.15.131:0/500746349 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc78c15e910 0x7fc78c15edc0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fc78c0cf350 tx=0x7fc78400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:06.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.876+0000 7fc788c00640 1 -- 172.21.15.131:0/500746349 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc78400ebc0 con 0x7fc78c15e910 2024-05-26T22:50:06.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.876+0000 7fc788c00640 1 -- 172.21.15.131:0/500746349 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc78400ed80 con 0x7fc78c15e910 2024-05-26T22:50:06.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.876+0000 7fc788c00640 1 -- 172.21.15.131:0/500746349 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc784003bd0 con 0x7fc78c15e910 2024-05-26T22:50:06.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.876+0000 7fc793a00640 1 -- 172.21.15.131:0/500746349 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc78c09d170 con 0x7fc78c15e910 2024-05-26T22:50:06.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.877+0000 7fc793a00640 1 -- 172.21.15.131:0/500746349 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc78c09d730 con 0x7fc78c15e910 2024-05-26T22:50:06.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.878+0000 7fc793a00640 1 -- 172.21.15.131:0/500746349 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc78c074e90 con 0x7fc78c15e910 2024-05-26T22:50:06.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.879+0000 7fc788c00640 1 -- 172.21.15.131:0/500746349 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc78401b020 con 0x7fc78c15e910 2024-05-26T22:50:06.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.879+0000 7fc788c00640 1 --2- 172.21.15.131:0/500746349 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc76c06be60 0x7fc76c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:06.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.880+0000 7fc788c00640 1 -- 172.21.15.131:0/500746349 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fc78408ad30 con 0x7fc78c15e910 2024-05-26T22:50:06.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.880+0000 7fc791000640 1 --2- 172.21.15.131:0/500746349 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc76c06be60 0x7fc76c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:06.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.880+0000 7fc791000640 1 --2- 172.21.15.131:0/500746349 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc76c06be60 0x7fc76c06e320 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc780033000 tx=0x7fc780008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:06.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:06.884+0000 7fc788c00640 1 -- 172.21.15.131:0/500746349 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fc78c147010 con 0x7fc78c15e910 2024-05-26T22:50:07.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:07.095+0000 7fc793a00640 1 -- 172.21.15.131:0/500746349 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc78c0cee90 con 0x7fc78c15e910 2024-05-26T22:50:07.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:07.096+0000 7fc788c00640 1 -- 172.21.15.131:0/500746349 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v33) ==== 80+0+0 (secure 0 0 0) 0x7fc784059640 con 0x7fc78c15e910 2024-05-26T22:50:07.095 INFO:teuthology.orchestra.run.smithi131.stderr:listed 0 entries 2024-05-26T22:50:07.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:07.098+0000 7fc793a00640 1 -- 172.21.15.131:0/500746349 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc76c06be60 msgr2=0x7fc76c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:07.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:07.098+0000 7fc793a00640 1 --2- 172.21.15.131:0/500746349 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc76c06be60 0x7fc76c06e320 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc780033000 tx=0x7fc780008030 comp rx=0 tx=0).stop 2024-05-26T22:50:07.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:07.098+0000 7fc793a00640 1 -- 172.21.15.131:0/500746349 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc78c15e910 msgr2=0x7fc78c15edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:07.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:07.098+0000 7fc793a00640 1 --2- 172.21.15.131:0/500746349 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc78c15e910 0x7fc78c15edc0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fc78c0cf350 tx=0x7fc78400e4b0 comp rx=0 tx=0).stop 2024-05-26T22:50:07.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:07.099+0000 7fc793a00640 1 -- 172.21.15.131:0/500746349 shutdown_connections 2024-05-26T22:50:07.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:07.099+0000 7fc793a00640 1 --2- 172.21.15.131:0/500746349 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc76c06be60 0x7fc76c06e320 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:07.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:07.099+0000 7fc793a00640 1 --2- 172.21.15.131:0/500746349 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc78c15e910 0x7fc78c15edc0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:07.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:07.099+0000 7fc793a00640 1 --2- 172.21.15.131:0/500746349 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc78c1578f0 0x7fc78c146ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:07.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:07.099+0000 7fc793a00640 1 --2- 172.21.15.131:0/500746349 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc7740039b0 0x7fc78c09cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:07.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:07.099+0000 7fc793a00640 1 -- 172.21.15.131:0/500746349 >> 172.21.15.131:0/500746349 conn(0x7fc78c06bc90 msgr2=0x7fc78c0cd9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:07.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:07.099+0000 7fc793a00640 1 -- 172.21.15.131:0/500746349 shutdown_connections 2024-05-26T22:50:07.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:07.099+0000 7fc793a00640 1 -- 172.21.15.131:0/500746349 wait complete. 2024-05-26T22:50:07.116 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-26T22:50:07.116 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:50:07.116 DEBUG:teuthology.orchestra.run.smithi131:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-26T22:50:07.285 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi131.front.sepia.ceph.com 2024-05-26T22:50:07.285 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:50:07.285 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-26T22:50:07.324 INFO:tasks.cephfs.mount:Terminating background process 2024-05-26T22:50:07.324 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-05-26T22:50:07.324 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-26T22:50:07.367 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-26T22:50:07.367 INFO:tasks.cephfs.mount:Terminating background process 2024-05-26T22:50:07.368 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-26T22:50:07.368 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-26T22:50:07.411 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-26T22:50:07.411 INFO:tasks.cephfs.mount:Terminating background process 2024-05-26T22:50:07.411 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-05-26T22:50:07.411 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-26T22:50:07.463 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-26T22:50:07.463 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-26T22:50:07.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:07.665+0000 7fe03e400640 1 Processor -- start 2024-05-26T22:50:07.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:07.665+0000 7fe03e400640 1 -- start start 2024-05-26T22:50:07.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:07.667+0000 7fe03e400640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fe038162780 0x7fe0380b3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:07.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:07.667+0000 7fe03e400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fe03806dcc0 0x7fe0380ad950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:07.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:07.667+0000 7fe03e400640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0380ad170 con 0x7fe03806f600 2024-05-26T22:50:07.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:07.667+0000 7fe03e400640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fe0380b4510 con 0x7fe03806dcc0 2024-05-26T22:50:07.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:07.667+0000 7fe03e400640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fe0380b47f0 con 0x7fe038162780 2024-05-26T22:50:07.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:07.667+0000 7fe037e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fe03806dcc0 0x7fe0380ad950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:07.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:07.668+0000 7fe037e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fe03806dcc0 0x7fe0380ad950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37716/0 (socket says 172.21.15.131:37716) 2024-05-26T22:50:07.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:07.668+0000 7fe037e00640 1 -- 172.21.15.131:0/1002122801 learned_addr learned my addr 172.21.15.131:0/1002122801 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:07.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:07.668+0000 7fe037400640 1 --2- 172.21.15.131:0/1002122801 >> v2:172.21.15.187:3300/0 conn(0x7fe038162780 0x7fe0380b3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:07.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:07.668+0000 7fe037e00640 1 -- 172.21.15.131:0/1002122801 >> v2:172.21.15.187:3300/0 conn(0x7fe038162780 msgr2=0x7fe0380b3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:07.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:07.668+0000 7fe037e00640 1 --2- 172.21.15.131:0/1002122801 >> v2:172.21.15.187:3300/0 conn(0x7fe038162780 0x7fe0380b3ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:07.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:07.668+0000 7fe037e00640 1 -- 172.21.15.131:0/1002122801 >> v1:172.21.15.187:6789/0 conn(0x7fe03806f600 legacy=0x7fe03806d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:07.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:07.668+0000 7fe037e00640 1 -- 172.21.15.131:0/1002122801 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0380b4ad0 con 0x7fe03806dcc0 2024-05-26T22:50:07.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:07.668+0000 7fe037400640 1 --2- 172.21.15.131:0/1002122801 >> v2:172.21.15.187:3300/0 conn(0x7fe038162780 0x7fe0380b3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:50:07.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:07.669+0000 7fe037e00640 1 --2- 172.21.15.131:0/1002122801 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe03806dcc0 0x7fe0380ad950 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fe03806c2f0 tx=0x7fe03002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd6d2ba01ee69aa server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:07.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:07.669+0000 7fe036000640 1 -- 172.21.15.131:0/1002122801 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe030037070 con 0x7fe03806dcc0 2024-05-26T22:50:07.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:07.669+0000 7fe036000640 1 -- 172.21.15.131:0/1002122801 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe030032440 con 0x7fe03806dcc0 2024-05-26T22:50:07.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:07.669+0000 7fe036000640 1 -- 172.21.15.131:0/1002122801 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe030032810 con 0x7fe03806dcc0 2024-05-26T22:50:07.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:07.670+0000 7fe03e400640 1 -- 172.21.15.131:0/1002122801 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe03806dcc0 msgr2=0x7fe0380ad950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:07.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:07.670+0000 7fe03e400640 1 --2- 172.21.15.131:0/1002122801 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe03806dcc0 0x7fe0380ad950 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fe03806c2f0 tx=0x7fe03002a270 comp rx=0 tx=0).stop 2024-05-26T22:50:07.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:07.670+0000 7fe03e400640 1 -- 172.21.15.131:0/1002122801 shutdown_connections 2024-05-26T22:50:07.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:07.670+0000 7fe03e400640 1 --2- 172.21.15.131:0/1002122801 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe03806dcc0 0x7fe0380ad950 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:07.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:07.670+0000 7fe03e400640 1 --2- 172.21.15.131:0/1002122801 >> v2:172.21.15.187:3300/0 conn(0x7fe038162780 0x7fe0380b3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:07.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:07.670+0000 7fe03e400640 1 -- 172.21.15.131:0/1002122801 >> 172.21.15.131:0/1002122801 conn(0x7fe0380b1e90 msgr2=0x7fe0380b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:07.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:07.670+0000 7fe03e400640 1 -- 172.21.15.131:0/1002122801 shutdown_connections 2024-05-26T22:50:07.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:07.670+0000 7fe03e400640 1 -- 172.21.15.131:0/1002122801 wait complete. 2024-05-26T22:50:07.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:07.672+0000 7fe03e400640 1 Processor -- start 2024-05-26T22:50:07.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:07.672+0000 7fe03e400640 1 -- start start 2024-05-26T22:50:07.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:07.673+0000 7fe03e400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe03806dcc0 0x7fe03809be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:07.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:07.673+0000 7fe03e400640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe03806f600 0x7fe03809dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:07.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:07.673+0000 7fe03e400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe038162780 0x7fe038153160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:07.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:07.673+0000 7fe03e400640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe0380b4260 con 0x7fe03806dcc0 2024-05-26T22:50:07.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:07.673+0000 7fe03e400640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe0380b40e0 con 0x7fe03806f600 2024-05-26T22:50:07.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:07.673+0000 7fe03e400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe0380ade90 con 0x7fe038162780 2024-05-26T22:50:07.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:07.674+0000 7fe037400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe03806dcc0 0x7fe03809be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:07.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:07.674+0000 7fe037400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe03806dcc0 0x7fe03809be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37720/0 (socket says 172.21.15.131:37720) 2024-05-26T22:50:07.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:07.674+0000 7fe037400640 1 -- 172.21.15.131:0/3520492159 learned_addr learned my addr 172.21.15.131:0/3520492159 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:07.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:07.674+0000 7fe037e00640 1 --2- 172.21.15.131:0/3520492159 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe038162780 0x7fe038153160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:07.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:07.674+0000 7fe037400640 1 -- 172.21.15.131:0/3520492159 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe038162780 msgr2=0x7fe038153160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:07.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:07.674+0000 7fe036a00640 1 --2- 172.21.15.131:0/3520492159 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe03806f600 0x7fe03809dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:07.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:07.674+0000 7fe037400640 1 --2- 172.21.15.131:0/3520492159 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe038162780 0x7fe038153160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:07.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:07.674+0000 7fe037400640 1 -- 172.21.15.131:0/3520492159 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe03806f600 msgr2=0x7fe03809dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:07.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:07.674+0000 7fe037400640 1 --2- 172.21.15.131:0/3520492159 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe03806f600 0x7fe03809dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:07.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:07.674+0000 7fe037400640 1 -- 172.21.15.131:0/3520492159 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe03809b570 con 0x7fe03806dcc0 2024-05-26T22:50:07.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:07.674+0000 7fe037400640 1 --2- 172.21.15.131:0/3520492159 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe03806dcc0 0x7fe03809be00 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fe02800ce00 tx=0x7fe02800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:07.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:07.675+0000 7fe02f400640 1 -- 172.21.15.131:0/3520492159 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe02800f680 con 0x7fe03806dcc0 2024-05-26T22:50:07.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:07.675+0000 7fe02f400640 1 -- 172.21.15.131:0/3520492159 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe02800f820 con 0x7fe03806dcc0 2024-05-26T22:50:07.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:07.675+0000 7fe02f400640 1 -- 172.21.15.131:0/3520492159 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe02800fc30 con 0x7fe03806dcc0 2024-05-26T22:50:07.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:07.675+0000 7fe03e400640 1 -- 172.21.15.131:0/3520492159 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe03809a4d0 con 0x7fe03806dcc0 2024-05-26T22:50:07.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:07.675+0000 7fe03e400640 1 -- 172.21.15.131:0/3520492159 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe03809aa90 con 0x7fe03806dcc0 2024-05-26T22:50:07.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:07.676+0000 7fe03e400640 1 -- 172.21.15.131:0/3520492159 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0380694e0 con 0x7fe03806dcc0 2024-05-26T22:50:07.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:07.677+0000 7fe02f400640 1 -- 172.21.15.131:0/3520492159 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe028018400 con 0x7fe03806dcc0 2024-05-26T22:50:07.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:07.678+0000 7fe02f400640 1 --2- 172.21.15.131:0/3520492159 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fe01806be30 0x7fe01806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:07.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:07.678+0000 7fe02f400640 1 -- 172.21.15.131:0/3520492159 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fe02808bb50 con 0x7fe03806dcc0 2024-05-26T22:50:07.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:07.678+0000 7fe036a00640 1 --2- 172.21.15.131:0/3520492159 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fe01806be30 0x7fe01806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:07.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:07.679+0000 7fe036a00640 1 --2- 172.21.15.131:0/3520492159 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fe01806be30 0x7fe01806e2f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe038164dc0 tx=0x7fe024009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:07.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:07.683+0000 7fe02f400640 1 -- 172.21.15.131:0/3520492159 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fe028014030 con 0x7fe03806dcc0 2024-05-26T22:50:07.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:07.912+0000 7fe03e400640 1 -- 172.21.15.131:0/3520492159 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe03806e350 con 0x7fe03806dcc0 2024-05-26T22:50:07.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:07.914+0000 7fe02f400640 1 -- 172.21.15.131:0/3520492159 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+5231 (secure 0 0 0) 0x7fe028059ea0 con 0x7fe03806dcc0 2024-05-26T22:50:07.913 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:50:07.914 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":20,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/4292497989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":4292497989},{"type":"v1","addr":"172.21.15.187:6835","nonce":4292497989}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-26T22:49:30.955525+0000","modified":"2024-05-26T22:49:46.310449+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5115,"mds_1":4687,"mds_2":4678},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4678":{"gid":4678,"name":"a","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.131:6836/4032294430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":4032294430},{"type":"v1","addr":"172.21.15.131:6836","nonce":4032294430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4687":{"gid":4687,"name":"b","rank":1,"incarnation":17,"state":"up:active","state_seq":7,"addr":"172.21.15.187:6833/3274403209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3274403209},{"type":"v1","addr":"172.21.15.187:6833","nonce":3274403209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5115":{"gid":5115,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.131:6837/2062704413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6835","nonce":2062704413},{"type":"v1","addr":"172.21.15.131:6837","nonce":2062704413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5115,"qdb_cluster":[4678,4687,5115]},"id":2}]} 2024-05-26T22:50:07.914 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 20 2024-05-26T22:50:07.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:07.916+0000 7fe03e400640 1 -- 172.21.15.131:0/3520492159 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fe01806be30 msgr2=0x7fe01806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:07.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:07.916+0000 7fe03e400640 1 --2- 172.21.15.131:0/3520492159 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fe01806be30 0x7fe01806e2f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe038164dc0 tx=0x7fe024009040 comp rx=0 tx=0).stop 2024-05-26T22:50:07.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:07.916+0000 7fe03e400640 1 -- 172.21.15.131:0/3520492159 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe03806dcc0 msgr2=0x7fe03809be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:07.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:07.916+0000 7fe03e400640 1 --2- 172.21.15.131:0/3520492159 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe03806dcc0 0x7fe03809be00 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fe02800ce00 tx=0x7fe02800a650 comp rx=0 tx=0).stop 2024-05-26T22:50:07.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:07.916+0000 7fe03e400640 1 -- 172.21.15.131:0/3520492159 shutdown_connections 2024-05-26T22:50:07.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:07.916+0000 7fe03e400640 1 --2- 172.21.15.131:0/3520492159 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe038162780 0x7fe038153160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:07.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:07.916+0000 7fe03e400640 1 --2- 172.21.15.131:0/3520492159 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fe01806be30 0x7fe01806e2f0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:07.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:07.916+0000 7fe03e400640 1 --2- 172.21.15.131:0/3520492159 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe03806f600 0x7fe03809dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:07.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:07.916+0000 7fe03e400640 1 --2- 172.21.15.131:0/3520492159 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe03806dcc0 0x7fe03809be00 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:07.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:07.916+0000 7fe03e400640 1 -- 172.21.15.131:0/3520492159 >> 172.21.15.131:0/3520492159 conn(0x7fe0380b1e90 msgr2=0x7fe03806d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:07.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:07.916+0000 7fe03e400640 1 -- 172.21.15.131:0/3520492159 shutdown_connections 2024-05-26T22:50:07.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:07.917+0000 7fe03e400640 1 -- 172.21.15.131:0/3520492159 wait complete. 2024-05-26T22:50:07.934 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-26T22:50:08.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.133+0000 7f74d5a00640 1 Processor -- start 2024-05-26T22:50:08.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.134+0000 7f74d5a00640 1 -- start start 2024-05-26T22:50:08.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.135+0000 7f74d5a00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f74d006dcc0 0x7f74d00b3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:08.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.135+0000 7f74d5a00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f74d00b36d0 0x7f74d015fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:08.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.135+0000 7f74d5a00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74d00b3e50 con 0x7f74d0158c00 2024-05-26T22:50:08.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.135+0000 7f74d5a00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f74d00b4130 con 0x7f74d00b36d0 2024-05-26T22:50:08.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.135+0000 7f74d5a00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f74d00b4410 con 0x7f74d006dcc0 2024-05-26T22:50:08.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.136+0000 7f74cf400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f74d00b36d0 0x7f74d015fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:08.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.136+0000 7f74cf400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f74d00b36d0 0x7f74d015fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37728/0 (socket says 172.21.15.131:37728) 2024-05-26T22:50:08.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.136+0000 7f74cf400640 1 -- 172.21.15.131:0/1976175270 learned_addr learned my addr 172.21.15.131:0/1976175270 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:08.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.136+0000 7f74c6000640 1 --2- 172.21.15.131:0/1976175270 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f74d006dcc0 0x7f74d00b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:08.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.136+0000 7f74cf400640 1 -- 172.21.15.131:0/1976175270 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f74d006dcc0 msgr2=0x7f74d00b3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:08.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.136+0000 7f74cf400640 1 --2- 172.21.15.131:0/1976175270 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f74d006dcc0 0x7f74d00b3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:08.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.136+0000 7f74cf400640 1 -- 172.21.15.131:0/1976175270 >> v1:172.21.15.187:6789/0 conn(0x7f74d0158c00 legacy=0x7f74d006d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:08.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.136+0000 7f74cf400640 1 -- 172.21.15.131:0/1976175270 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74d00b46f0 con 0x7f74d00b36d0 2024-05-26T22:50:08.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.137+0000 7f74cf400640 1 --2- 172.21.15.131:0/1976175270 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f74d00b36d0 0x7f74d015fc50 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f74c8009e50 tx=0x7f74c802a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9a1803bed5124a8 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:08.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.137+0000 7f74ce000640 1 -- 172.21.15.131:0/1976175270 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c8036070 con 0x7f74d00b36d0 2024-05-26T22:50:08.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.138+0000 7f74ce000640 1 -- 172.21.15.131:0/1976175270 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74c8031600 con 0x7f74d00b36d0 2024-05-26T22:50:08.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.138+0000 7f74ce000640 1 -- 172.21.15.131:0/1976175270 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c80319d0 con 0x7f74d00b36d0 2024-05-26T22:50:08.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.138+0000 7f74d5a00640 1 -- 172.21.15.131:0/1976175270 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f74d00b36d0 msgr2=0x7f74d015fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:08.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.138+0000 7f74d5a00640 1 --2- 172.21.15.131:0/1976175270 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f74d00b36d0 0x7f74d015fc50 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f74c8009e50 tx=0x7f74c802a630 comp rx=0 tx=0).stop 2024-05-26T22:50:08.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.138+0000 7f74d5a00640 1 -- 172.21.15.131:0/1976175270 shutdown_connections 2024-05-26T22:50:08.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.138+0000 7f74d5a00640 1 --2- 172.21.15.131:0/1976175270 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f74d00b36d0 0x7f74d015fc50 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:08.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.138+0000 7f74d5a00640 1 --2- 172.21.15.131:0/1976175270 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f74d006dcc0 0x7f74d00b3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:08.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.138+0000 7f74d5a00640 1 -- 172.21.15.131:0/1976175270 >> 172.21.15.131:0/1976175270 conn(0x7f74d00b1e90 msgr2=0x7f74d00b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:08.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.138+0000 7f74d5a00640 1 -- 172.21.15.131:0/1976175270 shutdown_connections 2024-05-26T22:50:08.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.138+0000 7f74d5a00640 1 -- 172.21.15.131:0/1976175270 wait complete. 2024-05-26T22:50:08.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.140+0000 7f74d5a00640 1 Processor -- start 2024-05-26T22:50:08.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.140+0000 7f74d5a00640 1 -- start start 2024-05-26T22:50:08.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.141+0000 7f74d5a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f74d006dcc0 0x7f74d0159a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:08.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.141+0000 7f74d5a00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f74d00b36d0 0x7f74d0153320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:08.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.141+0000 7f74d5a00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f74d0158c00 0x7f74d015e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:08.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.141+0000 7f74d5a00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f74d0160190 con 0x7f74d006dcc0 2024-05-26T22:50:08.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.141+0000 7f74d5a00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f74d00b3b60 con 0x7f74d0158c00 2024-05-26T22:50:08.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.141+0000 7f74d5a00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f74d0160310 con 0x7f74d00b36d0 2024-05-26T22:50:08.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.141+0000 7f74cea00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f74d006dcc0 0x7f74d0159a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:08.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.141+0000 7f74cea00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f74d006dcc0 0x7f74d0159a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37736/0 (socket says 172.21.15.131:37736) 2024-05-26T22:50:08.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.141+0000 7f74cea00640 1 -- 172.21.15.131:0/2771431249 learned_addr learned my addr 172.21.15.131:0/2771431249 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:08.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.142+0000 7f74cea00640 1 -- 172.21.15.131:0/2771431249 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f74d00b36d0 msgr2=0x7f74d0153320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:08.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.142+0000 7f74cea00640 1 --2- 172.21.15.131:0/2771431249 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f74d00b36d0 0x7f74d0153320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:08.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.142+0000 7f74cea00640 1 -- 172.21.15.131:0/2771431249 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f74d0158c00 msgr2=0x7f74d015e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:08.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.142+0000 7f74cea00640 1 --2- 172.21.15.131:0/2771431249 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f74d0158c00 0x7f74d015e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:08.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.142+0000 7f74cea00640 1 -- 172.21.15.131:0/2771431249 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74d00b0260 con 0x7f74d006dcc0 2024-05-26T22:50:08.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.142+0000 7f74cea00640 1 --2- 172.21.15.131:0/2771431249 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f74d006dcc0 0x7f74d0159a80 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f74c000b3f0 tx=0x7f74c000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:08.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.142+0000 7f74c7400640 1 -- 172.21.15.131:0/2771431249 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c0015020 con 0x7f74d006dcc0 2024-05-26T22:50:08.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.142+0000 7f74c7400640 1 -- 172.21.15.131:0/2771431249 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74c0011830 con 0x7f74d006dcc0 2024-05-26T22:50:08.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.142+0000 7f74c7400640 1 -- 172.21.15.131:0/2771431249 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c0011c20 con 0x7f74d006dcc0 2024-05-26T22:50:08.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.142+0000 7f74d5a00640 1 -- 172.21.15.131:0/2771431249 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74d015a110 con 0x7f74d006dcc0 2024-05-26T22:50:08.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.142+0000 7f74d5a00640 1 -- 172.21.15.131:0/2771431249 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74d015dfb0 con 0x7f74d006dcc0 2024-05-26T22:50:08.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.143+0000 7f74d5a00640 1 -- 172.21.15.131:0/2771431249 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74d00694e0 con 0x7f74d006dcc0 2024-05-26T22:50:08.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.144+0000 7f74c7400640 1 -- 172.21.15.131:0/2771431249 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f74c0018020 con 0x7f74d006dcc0 2024-05-26T22:50:08.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.145+0000 7f74c7400640 1 --2- 172.21.15.131:0/2771431249 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f74ac06be30 0x7f74ac06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:08.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.145+0000 7f74c7400640 1 -- 172.21.15.131:0/2771431249 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f74c008aab0 con 0x7f74d006dcc0 2024-05-26T22:50:08.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.146+0000 7f74c6000640 1 --2- 172.21.15.131:0/2771431249 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f74ac06be30 0x7f74ac06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:08.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.146+0000 7f74c6000640 1 --2- 172.21.15.131:0/2771431249 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f74ac06be30 0x7f74ac06e2f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f74d00af570 tx=0x7f74b8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:08.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.150+0000 7f74c7400640 1 -- 172.21.15.131:0/2771431249 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f74c001f030 con 0x7f74d006dcc0 2024-05-26T22:50:08.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.354+0000 7f74d5a00640 1 -- 172.21.15.131:0/2771431249 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f74d015c3b0 con 0x7f74d006dcc0 2024-05-26T22:50:08.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.356+0000 7f74c7400640 1 -- 172.21.15.131:0/2771431249 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) ==== 74+0+16693 (secure 0 0 0) 0x7f74c00172c0 con 0x7f74d006dcc0 2024-05-26T22:50:08.357 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:50:08.357 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":33,"fsid":"46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f","created":"2024-05-26T22:47:57.288905+0000","modified":"2024-05-26T22:49:32.532828+0000","last_up_change":"2024-05-26T22:48:09.602207+0000","last_in_change":"2024-05-26T22:48:00.687341+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-26T22:48:10.463339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-05-26T22:49:28.813374+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-05-26T22:49:29.987297+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"814eaf62-a18d-494b-8e27-fd8fb88adea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6801","nonce":2350940051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6803","nonce":2350940051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6807","nonce":2350940051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6805","nonce":2350940051}]},"public_addr":"172.21.15.131:6801/2350940051","cluster_addr":"172.21.15.131:6803/2350940051","heartbeat_back_addr":"172.21.15.131:6807/2350940051","heartbeat_front_addr":"172.21.15.131:6805/2350940051","state":["exists","up"]},{"osd":1,"uuid":"42e66287-f6c6-4e26-af3c-6dfaa3536d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6825","nonce":1525528247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6827","nonce":1525528247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6831","nonce":1525528247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6829","nonce":1525528247}]},"public_addr":"172.21.15.131:6825/1525528247","cluster_addr":"172.21.15.131:6827/1525528247","heartbeat_back_addr":"172.21.15.131:6831/1525528247","heartbeat_front_addr":"172.21.15.131:6829/1525528247","state":["exists","up"]},{"osd":2,"uuid":"0da34673-04a2-4fd0-b57a-0ff2b679b043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6817","nonce":3627842210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6819","nonce":3627842210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6823","nonce":3627842210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6821","nonce":3627842210}]},"public_addr":"172.21.15.131:6817/3627842210","cluster_addr":"172.21.15.131:6819/3627842210","heartbeat_back_addr":"172.21.15.131:6823/3627842210","heartbeat_front_addr":"172.21.15.131:6821/3627842210","state":["exists","up"]},{"osd":3,"uuid":"cf117a27-d6b2-43d3-ae52-b47988bf8eed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6809","nonce":350790591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6811","nonce":350790591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6815","nonce":350790591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6813","nonce":350790591}]},"public_addr":"172.21.15.131:6809/350790591","cluster_addr":"172.21.15.131:6811/350790591","heartbeat_back_addr":"172.21.15.131:6815/350790591","heartbeat_front_addr":"172.21.15.131:6813/350790591","state":["exists","up"]},{"osd":4,"uuid":"a092ccfc-be89-4a84-88d5-0f781fa715eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6816","nonce":244560995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6819","nonce":244560995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6823","nonce":244560995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6821","nonce":244560995}]},"public_addr":"172.21.15.187:6816/244560995","cluster_addr":"172.21.15.187:6819/244560995","heartbeat_back_addr":"172.21.15.187:6823/244560995","heartbeat_front_addr":"172.21.15.187:6821/244560995","state":["exists","up"]},{"osd":5,"uuid":"7513ba69-dd00-4a35-b190-9e8cc3b2bdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6801","nonce":324619398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6803","nonce":324619398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6812","nonce":324619398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6807","nonce":324619398}]},"public_addr":"172.21.15.187:6801/324619398","cluster_addr":"172.21.15.187:6803/324619398","heartbeat_back_addr":"172.21.15.187:6812/324619398","heartbeat_front_addr":"172.21.15.187:6807/324619398","state":["exists","up"]},{"osd":6,"uuid":"50081810-c093-4d2f-a4d7-b7c695c5400d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6806","nonce":1391347786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6809","nonce":1391347786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6818","nonce":1391347786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6813","nonce":1391347786}]},"public_addr":"172.21.15.187:6806/1391347786","cluster_addr":"172.21.15.187:6809/1391347786","heartbeat_back_addr":"172.21.15.187:6818/1391347786","heartbeat_front_addr":"172.21.15.187:6813/1391347786","state":["exists","up"]},{"osd":7,"uuid":"775dae05-54f5-4ad4-bbce-92863c6ed2ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6825","nonce":1220065622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6827","nonce":1220065622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6831","nonce":1220065622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6829","nonce":1220065622}]},"public_addr":"172.21.15.187:6825/1220065622","cluster_addr":"172.21.15.187:6827/1220065622","heartbeat_back_addr":"172.21.15.187:6831/1220065622","heartbeat_front_addr":"172.21.15.187:6829/1220065622","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:05.983520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.299516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.067323+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.307748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.018742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.258775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.010912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.295355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-26T22:50:08.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.359+0000 7f74d5a00640 1 -- 172.21.15.131:0/2771431249 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f74ac06be30 msgr2=0x7f74ac06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:08.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.359+0000 7f74d5a00640 1 --2- 172.21.15.131:0/2771431249 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f74ac06be30 0x7f74ac06e2f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f74d00af570 tx=0x7f74b8008040 comp rx=0 tx=0).stop 2024-05-26T22:50:08.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.359+0000 7f74d5a00640 1 -- 172.21.15.131:0/2771431249 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f74d006dcc0 msgr2=0x7f74d0159a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:08.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.359+0000 7f74d5a00640 1 --2- 172.21.15.131:0/2771431249 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f74d006dcc0 0x7f74d0159a80 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f74c000b3f0 tx=0x7f74c000b8b0 comp rx=0 tx=0).stop 2024-05-26T22:50:08.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.360+0000 7f74d5a00640 1 -- 172.21.15.131:0/2771431249 shutdown_connections 2024-05-26T22:50:08.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.360+0000 7f74d5a00640 1 --2- 172.21.15.131:0/2771431249 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f74d00b36d0 0x7f74d0153320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:08.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.360+0000 7f74d5a00640 1 --2- 172.21.15.131:0/2771431249 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f74ac06be30 0x7f74ac06e2f0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:08.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.360+0000 7f74d5a00640 1 --2- 172.21.15.131:0/2771431249 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f74d0158c00 0x7f74d015e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:08.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.360+0000 7f74d5a00640 1 --2- 172.21.15.131:0/2771431249 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f74d006dcc0 0x7f74d0159a80 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:08.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.360+0000 7f74d5a00640 1 -- 172.21.15.131:0/2771431249 >> 172.21.15.131:0/2771431249 conn(0x7f74d00b1e90 msgr2=0x7f74d0157560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:08.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.360+0000 7f74d5a00640 1 -- 172.21.15.131:0/2771431249 shutdown_connections 2024-05-26T22:50:08.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.360+0000 7f74d5a00640 1 -- 172.21.15.131:0/2771431249 wait complete. 2024-05-26T22:50:08.379 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-26T22:48:10.463339+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-26T22:49:28.813374+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-05-26T22:49:29.987297+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-26T22:50:08.379 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-26T22:50:08.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.570+0000 7f7319800640 1 Processor -- start 2024-05-26T22:50:08.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.570+0000 7f7319800640 1 -- start start 2024-05-26T22:50:08.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.572+0000 7f7319800640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f73140d1190 0x7f73140d25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:08.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.572+0000 7f7319800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f73140b9ce0 0x7f73140d1570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:08.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.572+0000 7f7319800640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f73140b5c80 0x7f73140b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:08.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.572+0000 7f7319800640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f73140ba140 con 0x7f73140b5c80 2024-05-26T22:50:08.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.572+0000 7f7319800640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f73140d1ab0 con 0x7f73140d1190 2024-05-26T22:50:08.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.572+0000 7f7319800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f73140d2cd0 con 0x7f73140b9ce0 2024-05-26T22:50:08.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.573+0000 7f7318e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f73140b5c80 0x7f73140b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:08.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.573+0000 7f7318e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f73140b5c80 0x7f73140b60e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37750/0 (socket says 172.21.15.131:37750) 2024-05-26T22:50:08.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.573+0000 7f7318e00640 1 -- 172.21.15.131:0/3307304004 learned_addr learned my addr 172.21.15.131:0/3307304004 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:08.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.573+0000 7f730be00640 1 --2- 172.21.15.131:0/3307304004 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f73140b9ce0 0x7f73140d1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:08.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.573+0000 7f7313e00640 1 --2- 172.21.15.131:0/3307304004 >> v2:172.21.15.187:3300/0 conn(0x7f73140d1190 0x7f73140d25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:08.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.573+0000 7f7318e00640 1 -- 172.21.15.131:0/3307304004 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f73140b9ce0 msgr2=0x7f73140d1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:08.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.573+0000 7f7318e00640 1 --2- 172.21.15.131:0/3307304004 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f73140b9ce0 0x7f73140d1570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:08.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.573+0000 7f7318e00640 1 -- 172.21.15.131:0/3307304004 >> v2:172.21.15.187:3300/0 conn(0x7f73140d1190 msgr2=0x7f73140d25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:08.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.573+0000 7f7318e00640 1 --2- 172.21.15.131:0/3307304004 >> v2:172.21.15.187:3300/0 conn(0x7f73140d1190 0x7f73140d25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:08.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.573+0000 7f7318e00640 1 -- 172.21.15.131:0/3307304004 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73140d2fb0 con 0x7f73140b5c80 2024-05-26T22:50:08.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.573+0000 7f730be00640 1 --2- 172.21.15.131:0/3307304004 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f73140b9ce0 0x7f73140d1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-26T22:50:08.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.574+0000 7f7318e00640 1 --2- 172.21.15.131:0/3307304004 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f73140b5c80 0x7f73140b60e0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f731406bf00 tx=0x7f730c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=130fd2a79b5ef01f server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:08.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.575+0000 7f7313400640 1 -- 172.21.15.131:0/3307304004 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f730c036070 con 0x7f73140b5c80 2024-05-26T22:50:08.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.575+0000 7f7313400640 1 -- 172.21.15.131:0/3307304004 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f730c031600 con 0x7f73140b5c80 2024-05-26T22:50:08.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.575+0000 7f7313400640 1 -- 172.21.15.131:0/3307304004 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f730c0319d0 con 0x7f73140b5c80 2024-05-26T22:50:08.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.575+0000 7f7319800640 1 -- 172.21.15.131:0/3307304004 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f73140b5c80 msgr2=0x7f73140b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:08.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.575+0000 7f7319800640 1 --2- 172.21.15.131:0/3307304004 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f73140b5c80 0x7f73140b60e0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f731406bf00 tx=0x7f730c02a6a0 comp rx=0 tx=0).stop 2024-05-26T22:50:08.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.575+0000 7f7319800640 1 -- 172.21.15.131:0/3307304004 shutdown_connections 2024-05-26T22:50:08.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.575+0000 7f7319800640 1 --2- 172.21.15.131:0/3307304004 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f73140b5c80 0x7f73140b60e0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:08.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.575+0000 7f7319800640 1 --2- 172.21.15.131:0/3307304004 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f73140b9ce0 0x7f73140d1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:08.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.575+0000 7f7319800640 1 --2- 172.21.15.131:0/3307304004 >> v2:172.21.15.187:3300/0 conn(0x7f73140d1190 0x7f73140d25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:08.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.575+0000 7f7319800640 1 -- 172.21.15.131:0/3307304004 >> 172.21.15.131:0/3307304004 conn(0x7f7314072e60 msgr2=0x7f7314073270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:08.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.576+0000 7f7319800640 1 -- 172.21.15.131:0/3307304004 shutdown_connections 2024-05-26T22:50:08.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.576+0000 7f7319800640 1 -- 172.21.15.131:0/3307304004 wait complete. 2024-05-26T22:50:08.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.577+0000 7f7319800640 1 Processor -- start 2024-05-26T22:50:08.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.577+0000 7f7319800640 1 -- start start 2024-05-26T22:50:08.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.578+0000 7f7319800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f73140b5c80 0x7f73140c8790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:08.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.578+0000 7f7319800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f73140b9ce0 0x7f73140cb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:08.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.578+0000 7f7319800640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f73140d1190 0x7f7314153720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:08.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.578+0000 7f7319800640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f73140d3850 con 0x7f73140b9ce0 2024-05-26T22:50:08.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.578+0000 7f7319800640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f73140d3570 con 0x7f73140d1190 2024-05-26T22:50:08.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.578+0000 7f7319800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f73140d3d70 con 0x7f73140b5c80 2024-05-26T22:50:08.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.578+0000 7f730be00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f73140b9ce0 0x7f73140cb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:08.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.578+0000 7f730be00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f73140b9ce0 0x7f73140cb9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37760/0 (socket says 172.21.15.131:37760) 2024-05-26T22:50:08.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.578+0000 7f730be00640 1 -- 172.21.15.131:0/4069447480 learned_addr learned my addr 172.21.15.131:0/4069447480 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:08.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.579+0000 7f730be00640 1 -- 172.21.15.131:0/4069447480 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f73140b5c80 msgr2=0x7f73140c8790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:08.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.579+0000 7f730be00640 1 --2- 172.21.15.131:0/4069447480 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f73140b5c80 0x7f73140c8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:08.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.579+0000 7f7318e00640 1 --2- 172.21.15.131:0/4069447480 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f73140d1190 0x7f7314153720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:08.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.579+0000 7f730be00640 1 -- 172.21.15.131:0/4069447480 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f73140d1190 msgr2=0x7f7314153720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:08.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.579+0000 7f730be00640 1 --2- 172.21.15.131:0/4069447480 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f73140d1190 0x7f7314153720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:08.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.579+0000 7f730be00640 1 -- 172.21.15.131:0/4069447480 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73140cbee0 con 0x7f73140b9ce0 2024-05-26T22:50:08.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.579+0000 7f730be00640 1 --2- 172.21.15.131:0/4069447480 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f73140b9ce0 0x7f73140cb9a0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f72fc00e840 tx=0x7f72fc00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:08.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.579+0000 7f7310c00640 1 -- 172.21.15.131:0/4069447480 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72fc00f040 con 0x7f73140b9ce0 2024-05-26T22:50:08.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.579+0000 7f7310c00640 1 -- 172.21.15.131:0/4069447480 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72fc004030 con 0x7f73140b9ce0 2024-05-26T22:50:08.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.579+0000 7f7319800640 1 -- 172.21.15.131:0/4069447480 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73140cc1b0 con 0x7f73140b9ce0 2024-05-26T22:50:08.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.579+0000 7f7310c00640 1 -- 172.21.15.131:0/4069447480 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72fc004400 con 0x7f73140b9ce0 2024-05-26T22:50:08.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.580+0000 7f7319800640 1 -- 172.21.15.131:0/4069447480 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73140cc770 con 0x7f73140b9ce0 2024-05-26T22:50:08.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.581+0000 7f7319800640 1 -- 172.21.15.131:0/4069447480 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73140694e0 con 0x7f73140b9ce0 2024-05-26T22:50:08.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.581+0000 7f7310c00640 1 -- 172.21.15.131:0/4069447480 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f72fc00c6d0 con 0x7f73140b9ce0 2024-05-26T22:50:08.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.582+0000 7f7310c00640 1 --2- 172.21.15.131:0/4069447480 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f72f006be60 0x7f72f006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:08.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.582+0000 7f7310c00640 1 -- 172.21.15.131:0/4069447480 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f72fc08aab0 con 0x7f73140b9ce0 2024-05-26T22:50:08.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.583+0000 7f7313e00640 1 --2- 172.21.15.131:0/4069447480 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f72f006be60 0x7f72f006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:08.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.583+0000 7f7313e00640 1 --2- 172.21.15.131:0/4069447480 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f72f006be60 0x7f72f006e320 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f7304006fd0 tx=0x7f7304008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:08.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.587+0000 7f7310c00640 1 -- 172.21.15.131:0/4069447480 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f72fc017020 con 0x7f73140b9ce0 2024-05-26T22:50:08.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.770+0000 7f7319800640 1 -- 172.21.15.131:0/4069447480 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f73140cf6b0 con 0x7f73140b9ce0 2024-05-26T22:50:08.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.771+0000 7f7310c00640 1 -- 172.21.15.131:0/4069447480 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v33) ==== 93+0+10 (secure 0 0 0) 0x7f72fc059450 con 0x7f73140b9ce0 2024-05-26T22:50:08.771 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2024-05-26T22:50:08.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.773+0000 7f7319800640 1 -- 172.21.15.131:0/4069447480 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f72f006be60 msgr2=0x7f72f006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:08.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.773+0000 7f7319800640 1 --2- 172.21.15.131:0/4069447480 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f72f006be60 0x7f72f006e320 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f7304006fd0 tx=0x7f7304008040 comp rx=0 tx=0).stop 2024-05-26T22:50:08.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.773+0000 7f7319800640 1 -- 172.21.15.131:0/4069447480 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f73140b9ce0 msgr2=0x7f73140cb9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:08.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.773+0000 7f7319800640 1 --2- 172.21.15.131:0/4069447480 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f73140b9ce0 0x7f73140cb9a0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f72fc00e840 tx=0x7f72fc00ed00 comp rx=0 tx=0).stop 2024-05-26T22:50:08.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.773+0000 7f7319800640 1 -- 172.21.15.131:0/4069447480 shutdown_connections 2024-05-26T22:50:08.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.773+0000 7f7319800640 1 --2- 172.21.15.131:0/4069447480 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f72f006be60 0x7f72f006e320 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:08.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.773+0000 7f7319800640 1 --2- 172.21.15.131:0/4069447480 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f73140d1190 0x7f7314153720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:08.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.773+0000 7f7319800640 1 --2- 172.21.15.131:0/4069447480 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f73140b9ce0 0x7f73140cb9a0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:08.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.773+0000 7f7319800640 1 --2- 172.21.15.131:0/4069447480 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f73140b5c80 0x7f73140c8790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:08.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.773+0000 7f7319800640 1 -- 172.21.15.131:0/4069447480 >> 172.21.15.131:0/4069447480 conn(0x7f7314072e60 msgr2=0x7f73140b3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:08.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.773+0000 7f7319800640 1 -- 172.21.15.131:0/4069447480 shutdown_connections 2024-05-26T22:50:08.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.773+0000 7f7319800640 1 -- 172.21.15.131:0/4069447480 wait complete. 2024-05-26T22:50:08.791 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-26T22:50:08.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.989+0000 7f50ef200640 1 Processor -- start 2024-05-26T22:50:08.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.989+0000 7f50ef200640 1 -- start start 2024-05-26T22:50:08.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.990+0000 7f50ef200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f50f0145a10 0x7f50f0072180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:08.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.990+0000 7f50ef200640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f50f00712d0 0x7f50f006d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:08.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.991+0000 7f50ef200640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50f0072d60 con 0x7f50f006d790 2024-05-26T22:50:08.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.991+0000 7f50ef200640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f50f015c760 con 0x7f50f00712d0 2024-05-26T22:50:08.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.991+0000 7f50ef200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f50f0072a80 con 0x7f50f0145a10 2024-05-26T22:50:08.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.991+0000 7f50ee800640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f50f006d790 0x7f50f015fd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:36342/0 (socket says 172.21.15.131:36342) 2024-05-26T22:50:08.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.991+0000 7f50ee800640 1 -- 172.21.15.131:0/4167098371 learned_addr learned my addr 172.21.15.131:0/4167098371 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:50:08.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.991+0000 7f50ede00640 1 --2- 172.21.15.131:0/4167098371 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f50f0145a10 0x7f50f0072180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:08.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.991+0000 7f50e5400640 1 --2- 172.21.15.131:0/4167098371 >> v2:172.21.15.187:3300/0 conn(0x7f50f00712d0 0x7f50f006d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:08.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.991+0000 7f50ed400640 1 -- 172.21.15.131:0/4167098371 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1019037837 0 0) 0x7f50f0072d60 con 0x7f50f006d790 2024-05-26T22:50:08.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.991+0000 7f50ed400640 1 -- 172.21.15.131:0/4167098371 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f50d4003520 con 0x7f50f006d790 2024-05-26T22:50:08.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.992+0000 7f50ed400640 1 -- 172.21.15.131:0/4167098371 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1754232341 0 0) 0x7f50d4003520 con 0x7f50f006d790 2024-05-26T22:50:08.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.992+0000 7f50ed400640 1 -- 172.21.15.131:0/4167098371 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f50f0145a10 msgr2=0x7f50f0072180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:08.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.992+0000 7f50ed400640 1 --2- 172.21.15.131:0/4167098371 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f50f0145a10 0x7f50f0072180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:08.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.992+0000 7f50ed400640 1 -- 172.21.15.131:0/4167098371 >> v2:172.21.15.187:3300/0 conn(0x7f50f00712d0 msgr2=0x7f50f006d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:08.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.992+0000 7f50ed400640 1 --2- 172.21.15.131:0/4167098371 >> v2:172.21.15.187:3300/0 conn(0x7f50f00712d0 0x7f50f006d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:08.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.992+0000 7f50ed400640 1 -- 172.21.15.131:0/4167098371 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50f00ceab0 con 0x7f50f006d790 2024-05-26T22:50:08.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.992+0000 7f50ed400640 1 -- 172.21.15.131:0/4167098371 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7f50e8001670 con 0x7f50f006d790 2024-05-26T22:50:08.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.992+0000 7f50ed400640 1 -- 172.21.15.131:0/4167098371 >> v1:172.21.15.131:6789/0 conn(0x7f50f006d790 legacy=0x7f50f015fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:08.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.992+0000 7f50ed400640 1 --2- 172.21.15.131:0/4167098371 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50d40039b0 0x7f50d4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:08.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.992+0000 7f50ed400640 1 --2- 172.21.15.131:0/4167098371 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f50f0145a10 0x7f50d4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:08.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.992+0000 7f50ed400640 1 --2- 172.21.15.131:0/4167098371 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f50d4004d40 0x7f50d40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:08.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.992+0000 7f50ed400640 1 -- 172.21.15.131:0/4167098371 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f50f015c760 con 0x7f50d40039b0 2024-05-26T22:50:08.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.992+0000 7f50ed400640 1 -- 172.21.15.131:0/4167098371 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f50f0072a80 con 0x7f50d4004d40 2024-05-26T22:50:08.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.992+0000 7f50ed400640 1 -- 172.21.15.131:0/4167098371 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f50d400b9c0 con 0x7f50f0145a10 2024-05-26T22:50:08.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.992+0000 7f50ede00640 1 --2- 172.21.15.131:0/4167098371 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50d40039b0 0x7f50d4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:08.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.993+0000 7f50ede00640 1 -- 172.21.15.131:0/4167098371 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f50f0145a10 msgr2=0x7f50d4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:08.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.993+0000 7f50ede00640 1 --2- 172.21.15.131:0/4167098371 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f50f0145a10 0x7f50d4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:08.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.993+0000 7f50ede00640 1 -- 172.21.15.131:0/4167098371 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f50d4004d40 msgr2=0x7f50d40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:08.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.993+0000 7f50ede00640 1 --2- 172.21.15.131:0/4167098371 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f50d4004d40 0x7f50d40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:08.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.993+0000 7f50ede00640 1 -- 172.21.15.131:0/4167098371 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50d4003520 con 0x7f50d40039b0 2024-05-26T22:50:08.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.994+0000 7f50ede00640 1 --2- 172.21.15.131:0/4167098371 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50d40039b0 0x7f50d4003dd0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f50e000e170 tx=0x7f50d4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a01f227537a48641 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:08.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.994+0000 7f50ed400640 1 -- 172.21.15.131:0/4167098371 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e0034020 con 0x7f50d40039b0 2024-05-26T22:50:08.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.994+0000 7f50ed400640 1 -- 172.21.15.131:0/4167098371 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50e0030430 con 0x7f50d40039b0 2024-05-26T22:50:08.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.994+0000 7f50ed400640 1 -- 172.21.15.131:0/4167098371 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e0009c10 con 0x7f50d40039b0 2024-05-26T22:50:08.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.994+0000 7f50ef200640 1 -- 172.21.15.131:0/4167098371 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50d40039b0 msgr2=0x7f50d4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:08.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.994+0000 7f50ef200640 1 --2- 172.21.15.131:0/4167098371 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50d40039b0 0x7f50d4003dd0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f50e000e170 tx=0x7f50d4005830 comp rx=0 tx=0).stop 2024-05-26T22:50:08.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.994+0000 7f50ee800640 1 -- 172.21.15.131:0/4167098371 reap_dead start 2024-05-26T22:50:08.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.995+0000 7f50ef200640 1 -- 172.21.15.131:0/4167098371 shutdown_connections 2024-05-26T22:50:08.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.995+0000 7f50ef200640 1 -- 172.21.15.131:0/4167098371 >> 172.21.15.131:0/4167098371 conn(0x7f50f00b5660 msgr2=0x7f50f00b5a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:08.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.995+0000 7f50ef200640 1 -- 172.21.15.131:0/4167098371 shutdown_connections 2024-05-26T22:50:08.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.995+0000 7f50ef200640 1 -- 172.21.15.131:0/4167098371 wait complete. 2024-05-26T22:50:08.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.996+0000 7f50ef200640 1 Processor -- start 2024-05-26T22:50:08.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.996+0000 7f50ef200640 1 -- start start 2024-05-26T22:50:08.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.997+0000 7f50ef200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50d40039b0 0x7f50f01f1620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:08.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.997+0000 7f50ef200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f50f0160130 0x7f50f01605b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:08.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.997+0000 7f50ef200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f50f01f1dd0 0x7f50f01f2250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:08.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.997+0000 7f50ef200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f50f0145df0 con 0x7f50d40039b0 2024-05-26T22:50:08.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.997+0000 7f50ef200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f50f00cf070 con 0x7f50f01f1dd0 2024-05-26T22:50:08.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.997+0000 7f50ef200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f50f00d05a0 con 0x7f50f0160130 2024-05-26T22:50:08.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.998+0000 7f50ede00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50d40039b0 0x7f50f01f1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:08.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.998+0000 7f50ede00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50d40039b0 0x7f50f01f1620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37790/0 (socket says 172.21.15.131:37790) 2024-05-26T22:50:08.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.998+0000 7f50ede00640 1 -- 172.21.15.131:0/3611637755 learned_addr learned my addr 172.21.15.131:0/3611637755 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:08.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.998+0000 7f50e5400640 1 --2- 172.21.15.131:0/3611637755 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f50f0160130 0x7f50f01605b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:08.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.998+0000 7f50ee800640 1 --2- 172.21.15.131:0/3611637755 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f50f01f1dd0 0x7f50f01f2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:08.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.998+0000 7f50ede00640 1 -- 172.21.15.131:0/3611637755 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f50f0160130 msgr2=0x7f50f01605b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:08.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.998+0000 7f50ede00640 1 --2- 172.21.15.131:0/3611637755 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f50f0160130 0x7f50f01605b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:08.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.998+0000 7f50ede00640 1 -- 172.21.15.131:0/3611637755 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f50f01f1dd0 msgr2=0x7f50f01f2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:09.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.998+0000 7f50ede00640 1 --2- 172.21.15.131:0/3611637755 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f50f01f1dd0 0x7f50f01f2250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:09.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.998+0000 7f50ede00640 1 -- 172.21.15.131:0/3611637755 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50f01f5310 con 0x7f50d40039b0 2024-05-26T22:50:09.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.998+0000 7f50ede00640 1 --2- 172.21.15.131:0/3611637755 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50d40039b0 0x7f50f01f1620 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f50e0033040 tx=0x7f50e0009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:09.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.999+0000 7f50e6a00640 1 -- 172.21.15.131:0/3611637755 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e0034040 con 0x7f50d40039b0 2024-05-26T22:50:09.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.999+0000 7f50e6a00640 1 -- 172.21.15.131:0/3611637755 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50e0003bb0 con 0x7f50d40039b0 2024-05-26T22:50:09.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.999+0000 7f50e6a00640 1 -- 172.21.15.131:0/3611637755 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e0004020 con 0x7f50d40039b0 2024-05-26T22:50:09.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.999+0000 7f50ef200640 1 -- 172.21.15.131:0/3611637755 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50f01f55f0 con 0x7f50d40039b0 2024-05-26T22:50:09.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:08.999+0000 7f50ef200640 1 -- 172.21.15.131:0/3611637755 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50f014b2f0 con 0x7f50d40039b0 2024-05-26T22:50:09.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.000+0000 7f50ef200640 1 -- 172.21.15.131:0/3611637755 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50f00694e0 con 0x7f50d40039b0 2024-05-26T22:50:09.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.001+0000 7f50e6a00640 1 -- 172.21.15.131:0/3611637755 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f50e0037020 con 0x7f50d40039b0 2024-05-26T22:50:09.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.002+0000 7f50e6a00640 1 --2- 172.21.15.131:0/3611637755 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f50cc06be30 0x7f50cc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:09.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.002+0000 7f50e6a00640 1 -- 172.21.15.131:0/3611637755 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f50e00abc10 con 0x7f50d40039b0 2024-05-26T22:50:09.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.002+0000 7f50e5400640 1 --2- 172.21.15.131:0/3611637755 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f50cc06be30 0x7f50cc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:09.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.003+0000 7f50e5400640 1 --2- 172.21.15.131:0/3611637755 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f50cc06be30 0x7f50cc06e2f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f50d8006a90 tx=0x7f50d8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:09.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.006+0000 7f50e6a00640 1 -- 172.21.15.131:0/3611637755 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f50e007a650 con 0x7f50d40039b0 2024-05-26T22:50:09.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.204+0000 7f50ef200640 1 -- 172.21.15.131:0/3611637755 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f50f00afa90 con 0x7f50d40039b0 2024-05-26T22:50:09.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.204+0000 7f50e6a00640 1 -- 172.21.15.131:0/3611637755 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v33) ==== 104+0+11 (secure 0 0 0) 0x7f50e0079f50 con 0x7f50d40039b0 2024-05-26T22:50:09.204 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-05-26T22:50:09.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.207+0000 7f50ef200640 1 -- 172.21.15.131:0/3611637755 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f50cc06be30 msgr2=0x7f50cc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:09.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.207+0000 7f50ef200640 1 --2- 172.21.15.131:0/3611637755 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f50cc06be30 0x7f50cc06e2f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f50d8006a90 tx=0x7f50d8008040 comp rx=0 tx=0).stop 2024-05-26T22:50:09.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.207+0000 7f50ef200640 1 -- 172.21.15.131:0/3611637755 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50d40039b0 msgr2=0x7f50f01f1620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:09.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.207+0000 7f50ef200640 1 --2- 172.21.15.131:0/3611637755 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50d40039b0 0x7f50f01f1620 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f50e0033040 tx=0x7f50e0009750 comp rx=0 tx=0).stop 2024-05-26T22:50:09.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.208+0000 7f50ef200640 1 -- 172.21.15.131:0/3611637755 shutdown_connections 2024-05-26T22:50:09.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.208+0000 7f50ef200640 1 --2- 172.21.15.131:0/3611637755 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f50f0160130 0x7f50f01605b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:09.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.208+0000 7f50ef200640 1 --2- 172.21.15.131:0/3611637755 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f50cc06be30 0x7f50cc06e2f0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:09.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.208+0000 7f50ef200640 1 --2- 172.21.15.131:0/3611637755 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f50f01f1dd0 0x7f50f01f2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:09.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.208+0000 7f50ef200640 1 --2- 172.21.15.131:0/3611637755 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50d40039b0 0x7f50f01f1620 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:09.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.208+0000 7f50ef200640 1 -- 172.21.15.131:0/3611637755 >> 172.21.15.131:0/3611637755 conn(0x7f50f00b5660 msgr2=0x7f50f00b0de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:09.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.208+0000 7f50ef200640 1 -- 172.21.15.131:0/3611637755 shutdown_connections 2024-05-26T22:50:09.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.208+0000 7f50ef200640 1 -- 172.21.15.131:0/3611637755 wait complete. 2024-05-26T22:50:09.226 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-26T22:50:09.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.429+0000 7f05e9e00640 1 Processor -- start 2024-05-26T22:50:09.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.429+0000 7f05e9e00640 1 -- start start 2024-05-26T22:50:09.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.431+0000 7f05e9e00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f05e40d05a0 0x7f05e4166ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:09.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.431+0000 7f05e9e00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05e4072d60 con 0x7f05e4145a10 2024-05-26T22:50:09.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.431+0000 7f05e9e00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05e4072a80 con 0x7f05e40712d0 2024-05-26T22:50:09.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.431+0000 7f05e9e00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f05e415c760 con 0x7f05e40d05a0 2024-05-26T22:50:09.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.431+0000 7f05e8a00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f05e4145a10 0x7f05e4072180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:36348/0 (socket says 172.21.15.131:36348) 2024-05-26T22:50:09.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.431+0000 7f05e8a00640 1 -- 172.21.15.131:0/6347314 learned_addr learned my addr 172.21.15.131:0/6347314 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:50:09.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.431+0000 7f05e9400640 1 --2- 172.21.15.131:0/6347314 >> v2:172.21.15.187:3300/0 conn(0x7f05e40d05a0 0x7f05e4166ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:09.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.431+0000 7f05e3e00640 1 -- 172.21.15.131:0/6347314 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2616173532 0 0) 0x7f05e4072d60 con 0x7f05e4145a10 2024-05-26T22:50:09.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.432+0000 7f05e3e00640 1 -- 172.21.15.131:0/6347314 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f05c4003520 con 0x7f05e4145a10 2024-05-26T22:50:09.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.432+0000 7f05e3e00640 1 -- 172.21.15.131:0/6347314 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1908890473 0 0) 0x7f05c4003520 con 0x7f05e4145a10 2024-05-26T22:50:09.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.432+0000 7f05e3e00640 1 -- 172.21.15.131:0/6347314 >> v2:172.21.15.187:3300/0 conn(0x7f05e40d05a0 msgr2=0x7f05e4166ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:09.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.432+0000 7f05e3e00640 1 --2- 172.21.15.131:0/6347314 >> v2:172.21.15.187:3300/0 conn(0x7f05e40d05a0 0x7f05e4166ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:09.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.432+0000 7f05e3e00640 1 -- 172.21.15.131:0/6347314 >> v1:172.21.15.187:6789/0 conn(0x7f05e40712d0 legacy=0x7f05e406d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:09.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.432+0000 7f05e3e00640 1 -- 172.21.15.131:0/6347314 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05e40ceab0 con 0x7f05e4145a10 2024-05-26T22:50:09.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.432+0000 7f05e3e00640 1 -- 172.21.15.131:0/6347314 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7f05d4001670 con 0x7f05e4145a10 2024-05-26T22:50:09.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.432+0000 7f05e3e00640 1 -- 172.21.15.131:0/6347314 >> v1:172.21.15.131:6789/0 conn(0x7f05e4145a10 legacy=0x7f05e4072180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:09.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.432+0000 7f05e3e00640 1 --2- 172.21.15.131:0/6347314 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05c40039b0 0x7f05c4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:09.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.432+0000 7f05e3e00640 1 --2- 172.21.15.131:0/6347314 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f05c4004800 0x7f05c4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:09.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.432+0000 7f05e3e00640 1 --2- 172.21.15.131:0/6347314 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f05c40051c0 0x7f05c400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:09.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.432+0000 7f05e3e00640 1 -- 172.21.15.131:0/6347314 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f05e415c760 con 0x7f05c40039b0 2024-05-26T22:50:09.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.432+0000 7f05e3e00640 1 -- 172.21.15.131:0/6347314 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f05c400c060 con 0x7f05c4004800 2024-05-26T22:50:09.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.432+0000 7f05e3e00640 1 -- 172.21.15.131:0/6347314 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f05c400c260 con 0x7f05c40051c0 2024-05-26T22:50:09.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.432+0000 7f05e8a00640 1 --2- 172.21.15.131:0/6347314 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05c40039b0 0x7f05c4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:09.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.433+0000 7f05dbe00640 1 --2- 172.21.15.131:0/6347314 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f05c4004800 0x7f05c4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:09.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.433+0000 7f05e8a00640 1 -- 172.21.15.131:0/6347314 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f05c40051c0 msgr2=0x7f05c400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:09.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.433+0000 7f05e8a00640 1 --2- 172.21.15.131:0/6347314 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f05c40051c0 0x7f05c400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:09.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.433+0000 7f05e8a00640 1 -- 172.21.15.131:0/6347314 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f05c4004800 msgr2=0x7f05c4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:09.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.433+0000 7f05e8a00640 1 --2- 172.21.15.131:0/6347314 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f05c4004800 0x7f05c4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:09.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.433+0000 7f05e8a00640 1 -- 172.21.15.131:0/6347314 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05c4003520 con 0x7f05c40039b0 2024-05-26T22:50:09.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.433+0000 7f05e9400640 1 -- 172.21.15.131:0/6347314 reap_dead start 2024-05-26T22:50:09.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.434+0000 7f05e8a00640 1 --2- 172.21.15.131:0/6347314 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05c40039b0 0x7f05c4003dd0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f05d400c240 tx=0x7f05c4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0b036667ec993ae server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:09.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.434+0000 7f05e3e00640 1 -- 172.21.15.131:0/6347314 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05d4007d10 con 0x7f05c40039b0 2024-05-26T22:50:09.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.434+0000 7f05e3e00640 1 -- 172.21.15.131:0/6347314 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05d40058d0 con 0x7f05c40039b0 2024-05-26T22:50:09.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.434+0000 7f05e3e00640 1 -- 172.21.15.131:0/6347314 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05d4005d40 con 0x7f05c40039b0 2024-05-26T22:50:09.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.435+0000 7f05e9e00640 1 -- 172.21.15.131:0/6347314 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05c40039b0 msgr2=0x7f05c4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:09.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.435+0000 7f05e9e00640 1 --2- 172.21.15.131:0/6347314 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05c40039b0 0x7f05c4003dd0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f05d400c240 tx=0x7f05c4005830 comp rx=0 tx=0).stop 2024-05-26T22:50:09.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.435+0000 7f05e9e00640 1 -- 172.21.15.131:0/6347314 shutdown_connections 2024-05-26T22:50:09.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.435+0000 7f05e9e00640 1 --2- 172.21.15.131:0/6347314 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05c40039b0 0x7f05c4003dd0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:09.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.435+0000 7f05e9e00640 1 -- 172.21.15.131:0/6347314 >> 172.21.15.131:0/6347314 conn(0x7f05e40b5660 msgr2=0x7f05e40b5a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:09.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.435+0000 7f05e9e00640 1 -- 172.21.15.131:0/6347314 shutdown_connections 2024-05-26T22:50:09.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.435+0000 7f05e9e00640 1 -- 172.21.15.131:0/6347314 wait complete. 2024-05-26T22:50:09.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.437+0000 7f05e9e00640 1 Processor -- start 2024-05-26T22:50:09.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.437+0000 7f05e9e00640 1 -- start start 2024-05-26T22:50:09.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.438+0000 7f05e9e00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f05c40039b0 0x7f05e41f4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:09.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.438+0000 7f05e9e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f05e40c5a10 0x7f05e413b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:09.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.438+0000 7f05e9e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05e40c7a00 0x7f05e40c7e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:09.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.438+0000 7f05e9e00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f05e4145df0 con 0x7f05e40c7a00 2024-05-26T22:50:09.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.438+0000 7f05e9e00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f05e40cf070 con 0x7f05c40039b0 2024-05-26T22:50:09.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.438+0000 7f05e9e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f05e4072180 con 0x7f05e40c5a10 2024-05-26T22:50:09.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.438+0000 7f05e9400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05e40c7a00 0x7f05e40c7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:09.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.438+0000 7f05e9400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05e40c7a00 0x7f05e40c7e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37802/0 (socket says 172.21.15.131:37802) 2024-05-26T22:50:09.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.438+0000 7f05e9400640 1 -- 172.21.15.131:0/895874036 learned_addr learned my addr 172.21.15.131:0/895874036 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:09.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.438+0000 7f05e8a00640 1 --2- 172.21.15.131:0/895874036 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f05c40039b0 0x7f05e41f4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:09.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.438+0000 7f05e9400640 1 -- 172.21.15.131:0/895874036 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f05e40c5a10 msgr2=0x7f05e413b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:09.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.438+0000 7f05dbe00640 1 --2- 172.21.15.131:0/895874036 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f05e40c5a10 0x7f05e413b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:09.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.438+0000 7f05e9400640 1 --2- 172.21.15.131:0/895874036 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f05e40c5a10 0x7f05e413b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:09.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.438+0000 7f05e9400640 1 -- 172.21.15.131:0/895874036 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f05c40039b0 msgr2=0x7f05e41f4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:09.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.438+0000 7f05e9400640 1 --2- 172.21.15.131:0/895874036 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f05c40039b0 0x7f05e41f4790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:09.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.438+0000 7f05e9400640 1 -- 172.21.15.131:0/895874036 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05e40c5310 con 0x7f05e40c7a00 2024-05-26T22:50:09.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.438+0000 7f05e8a00640 1 --2- 172.21.15.131:0/895874036 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f05c40039b0 0x7f05e41f4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:50:09.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.439+0000 7f05e9400640 1 --2- 172.21.15.131:0/895874036 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05e40c7a00 0x7f05e40c7e90 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f05e40b28e0 tx=0x7f05dc00ef30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:09.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.439+0000 7f05e1600640 1 -- 172.21.15.131:0/895874036 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05dc00bba0 con 0x7f05e40c7a00 2024-05-26T22:50:09.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.439+0000 7f05e1600640 1 -- 172.21.15.131:0/895874036 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05dc01a070 con 0x7f05e40c7a00 2024-05-26T22:50:09.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.439+0000 7f05e1600640 1 -- 172.21.15.131:0/895874036 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05dc014400 con 0x7f05e40c7a00 2024-05-26T22:50:09.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.439+0000 7f05e9e00640 1 -- 172.21.15.131:0/895874036 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05e40c5ef0 con 0x7f05e40c7a00 2024-05-26T22:50:09.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.439+0000 7f05e9e00640 1 -- 172.21.15.131:0/895874036 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05e40c64b0 con 0x7f05e40c7a00 2024-05-26T22:50:09.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.440+0000 7f05e9e00640 1 -- 172.21.15.131:0/895874036 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05e40694e0 con 0x7f05e40c7a00 2024-05-26T22:50:09.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.441+0000 7f05e1600640 1 -- 172.21.15.131:0/895874036 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f05dc005d10 con 0x7f05e40c7a00 2024-05-26T22:50:09.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.442+0000 7f05e1600640 1 --2- 172.21.15.131:0/895874036 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f05c006be30 0x7f05c006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:09.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.442+0000 7f05e1600640 1 -- 172.21.15.131:0/895874036 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f05dc08afd0 con 0x7f05e40c7a00 2024-05-26T22:50:09.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.442+0000 7f05e8a00640 1 --2- 172.21.15.131:0/895874036 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f05c006be30 0x7f05c006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:09.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.443+0000 7f05e8a00640 1 --2- 172.21.15.131:0/895874036 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f05c006be30 0x7f05c006e2f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f05d4033000 tx=0x7f05d4035040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:09.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.448+0000 7f05e1600640 1 -- 172.21.15.131:0/895874036 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f05dc024020 con 0x7f05e40c7a00 2024-05-26T22:50:09.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.650+0000 7f05e9e00640 1 -- 172.21.15.131:0/895874036 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f05e40c57f0 con 0x7f05e40c7a00 2024-05-26T22:50:09.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.651+0000 7f05e1600640 1 -- 172.21.15.131:0/895874036 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v33) ==== 100+0+11 (secure 0 0 0) 0x7f05dc059980 con 0x7f05e40c7a00 2024-05-26T22:50:09.651 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 64 2024-05-26T22:50:09.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.654+0000 7f05e9e00640 1 -- 172.21.15.131:0/895874036 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f05c006be30 msgr2=0x7f05c006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:09.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.654+0000 7f05e9e00640 1 --2- 172.21.15.131:0/895874036 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f05c006be30 0x7f05c006e2f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f05d4033000 tx=0x7f05d4035040 comp rx=0 tx=0).stop 2024-05-26T22:50:09.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.654+0000 7f05e9e00640 1 -- 172.21.15.131:0/895874036 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05e40c7a00 msgr2=0x7f05e40c7e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:09.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.654+0000 7f05e9e00640 1 --2- 172.21.15.131:0/895874036 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05e40c7a00 0x7f05e40c7e90 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f05e40b28e0 tx=0x7f05dc00ef30 comp rx=0 tx=0).stop 2024-05-26T22:50:09.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.655+0000 7f05e9e00640 1 -- 172.21.15.131:0/895874036 shutdown_connections 2024-05-26T22:50:09.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.655+0000 7f05e9e00640 1 --2- 172.21.15.131:0/895874036 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f05e40c5a10 0x7f05e413b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:09.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.655+0000 7f05e9e00640 1 --2- 172.21.15.131:0/895874036 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f05c006be30 0x7f05c006e2f0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:09.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.655+0000 7f05e9e00640 1 --2- 172.21.15.131:0/895874036 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05e40c7a00 0x7f05e40c7e90 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:09.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.655+0000 7f05e9e00640 1 --2- 172.21.15.131:0/895874036 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f05c40039b0 0x7f05e41f4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:09.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.655+0000 7f05e9e00640 1 -- 172.21.15.131:0/895874036 >> 172.21.15.131:0/895874036 conn(0x7f05e40b5660 msgr2=0x7f05e40b0ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:09.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.655+0000 7f05e9e00640 1 -- 172.21.15.131:0/895874036 shutdown_connections 2024-05-26T22:50:09.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.655+0000 7f05e9e00640 1 -- 172.21.15.131:0/895874036 wait complete. 2024-05-26T22:50:09.673 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-26T22:50:09.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.874+0000 7f2a61600640 1 Processor -- start 2024-05-26T22:50:09.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.874+0000 7f2a61600640 1 -- start start 2024-05-26T22:50:09.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.875+0000 7f2a61600640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f2a5c155840 0x7f2a5c071930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:09.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.876+0000 7f2a61600640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a5c0722a0 con 0x7f2a5c06f660 2024-05-26T22:50:09.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.876+0000 7f2a61600640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a5c072580 con 0x7f2a5c15c3d0 2024-05-26T22:50:09.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.876+0000 7f2a61600640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f2a5c072860 con 0x7f2a5c155840 2024-05-26T22:50:09.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.876+0000 7f2a5b400640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f2a5c06f660 0x7f2a5c0ade40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:36356/0 (socket says 172.21.15.131:36356) 2024-05-26T22:50:09.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.876+0000 7f2a5b400640 1 -- 172.21.15.131:0/1242516357 learned_addr learned my addr 172.21.15.131:0/1242516357 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:50:09.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.876+0000 7f2a5aa00640 1 --2- 172.21.15.131:0/1242516357 >> v2:172.21.15.187:3300/0 conn(0x7f2a5c155840 0x7f2a5c071930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:09.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.876+0000 7f2a59600640 1 -- 172.21.15.131:0/1242516357 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 394467116 0 0) 0x7f2a5c0722a0 con 0x7f2a5c06f660 2024-05-26T22:50:09.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.876+0000 7f2a59600640 1 -- 172.21.15.131:0/1242516357 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2a40003520 con 0x7f2a5c06f660 2024-05-26T22:50:09.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.877+0000 7f2a59600640 1 -- 172.21.15.131:0/1242516357 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2460807129 0 0) 0x7f2a40003520 con 0x7f2a5c06f660 2024-05-26T22:50:09.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.877+0000 7f2a59600640 1 -- 172.21.15.131:0/1242516357 >> v2:172.21.15.187:3300/0 conn(0x7f2a5c155840 msgr2=0x7f2a5c071930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:09.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.877+0000 7f2a59600640 1 --2- 172.21.15.131:0/1242516357 >> v2:172.21.15.187:3300/0 conn(0x7f2a5c155840 0x7f2a5c071930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:09.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.877+0000 7f2a59600640 1 -- 172.21.15.131:0/1242516357 >> v1:172.21.15.187:6789/0 conn(0x7f2a5c15c3d0 legacy=0x7f2a5c06ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:09.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.877+0000 7f2a59600640 1 -- 172.21.15.131:0/1242516357 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a5c072b40 con 0x7f2a5c06f660 2024-05-26T22:50:09.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.877+0000 7f2a59600640 1 -- 172.21.15.131:0/1242516357 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7f2a540019f0 con 0x7f2a5c06f660 2024-05-26T22:50:09.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.877+0000 7f2a59600640 1 -- 172.21.15.131:0/1242516357 >> v1:172.21.15.131:6789/0 conn(0x7f2a5c06f660 legacy=0x7f2a5c0ade40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:09.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.877+0000 7f2a59600640 1 --2- 172.21.15.131:0/1242516357 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a400039b0 0x7f2a40003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:09.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.877+0000 7f2a59600640 1 --2- 172.21.15.131:0/1242516357 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2a40004800 0x7f2a40004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:09.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.877+0000 7f2a59600640 1 --2- 172.21.15.131:0/1242516357 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2a400051c0 0x7f2a4000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:09.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.877+0000 7f2a59600640 1 -- 172.21.15.131:0/1242516357 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f2a5c072860 con 0x7f2a400039b0 2024-05-26T22:50:09.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.877+0000 7f2a59600640 1 -- 172.21.15.131:0/1242516357 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2a4000c060 con 0x7f2a40004800 2024-05-26T22:50:09.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.877+0000 7f2a59600640 1 -- 172.21.15.131:0/1242516357 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f2a4000c260 con 0x7f2a400051c0 2024-05-26T22:50:09.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.877+0000 7f2a5aa00640 1 --2- 172.21.15.131:0/1242516357 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a400039b0 0x7f2a40003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:09.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.878+0000 7f2a5b400640 1 --2- 172.21.15.131:0/1242516357 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2a400051c0 0x7f2a4000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:09.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.878+0000 7f2a5a000640 1 --2- 172.21.15.131:0/1242516357 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2a40004800 0x7f2a40004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:09.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.878+0000 7f2a5aa00640 1 -- 172.21.15.131:0/1242516357 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2a400051c0 msgr2=0x7f2a4000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:09.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.878+0000 7f2a5aa00640 1 --2- 172.21.15.131:0/1242516357 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2a400051c0 0x7f2a4000b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:09.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.878+0000 7f2a5aa00640 1 -- 172.21.15.131:0/1242516357 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2a40004800 msgr2=0x7f2a40004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:09.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.878+0000 7f2a5aa00640 1 --2- 172.21.15.131:0/1242516357 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2a40004800 0x7f2a40004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:09.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.878+0000 7f2a5aa00640 1 -- 172.21.15.131:0/1242516357 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a40003520 con 0x7f2a400039b0 2024-05-26T22:50:09.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.878+0000 7f2a5a000640 1 --2- 172.21.15.131:0/1242516357 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2a40004800 0x7f2a40004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:50:09.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.878+0000 7f2a5b400640 1 -- 172.21.15.131:0/1242516357 reap_dead start 2024-05-26T22:50:09.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.879+0000 7f2a5aa00640 1 --2- 172.21.15.131:0/1242516357 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a400039b0 0x7f2a40003dd0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c009ac0 tx=0x7f2a40005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=870c6a3fe3f7cc92 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:09.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.879+0000 7f2a59600640 1 -- 172.21.15.131:0/1242516357 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a4c036070 con 0x7f2a400039b0 2024-05-26T22:50:09.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.879+0000 7f2a59600640 1 -- 172.21.15.131:0/1242516357 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a4c031400 con 0x7f2a400039b0 2024-05-26T22:50:09.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.879+0000 7f2a59600640 1 -- 172.21.15.131:0/1242516357 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a4c009690 con 0x7f2a400039b0 2024-05-26T22:50:09.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.879+0000 7f2a61600640 1 -- 172.21.15.131:0/1242516357 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a400039b0 msgr2=0x7f2a40003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:09.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.879+0000 7f2a61600640 1 --2- 172.21.15.131:0/1242516357 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a400039b0 0x7f2a40003dd0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c009ac0 tx=0x7f2a40005830 comp rx=0 tx=0).stop 2024-05-26T22:50:09.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.880+0000 7f2a61600640 1 -- 172.21.15.131:0/1242516357 shutdown_connections 2024-05-26T22:50:09.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.880+0000 7f2a61600640 1 --2- 172.21.15.131:0/1242516357 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a400039b0 0x7f2a40003dd0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:09.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.880+0000 7f2a61600640 1 -- 172.21.15.131:0/1242516357 >> 172.21.15.131:0/1242516357 conn(0x7f2a5c0b1e90 msgr2=0x7f2a5c0b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:09.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.880+0000 7f2a61600640 1 -- 172.21.15.131:0/1242516357 shutdown_connections 2024-05-26T22:50:09.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.880+0000 7f2a61600640 1 -- 172.21.15.131:0/1242516357 wait complete. 2024-05-26T22:50:09.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.881+0000 7f2a61600640 1 Processor -- start 2024-05-26T22:50:09.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.882+0000 7f2a61600640 1 -- start start 2024-05-26T22:50:09.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.882+0000 7f2a61600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a400039b0 0x7f2a5c14a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:09.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.882+0000 7f2a61600640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2a5c0ccb30 0x7f2a5c0b37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:09.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.882+0000 7f2a61600640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2a5c0b3ce0 0x7f2a5c1f7710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:09.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.882+0000 7f2a61600640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f2a5c071ab0 con 0x7f2a400039b0 2024-05-26T22:50:09.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.882+0000 7f2a61600640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2a5c071930 con 0x7f2a5c0b3ce0 2024-05-26T22:50:09.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.882+0000 7f2a61600640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f2a5c071c30 con 0x7f2a5c0ccb30 2024-05-26T22:50:09.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.883+0000 7f2a5aa00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a400039b0 0x7f2a5c14a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:09.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.883+0000 7f2a5aa00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a400039b0 0x7f2a5c14a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37834/0 (socket says 172.21.15.131:37834) 2024-05-26T22:50:09.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.883+0000 7f2a5aa00640 1 -- 172.21.15.131:0/994577051 learned_addr learned my addr 172.21.15.131:0/994577051 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:09.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.883+0000 7f2a5a000640 1 --2- 172.21.15.131:0/994577051 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2a5c0ccb30 0x7f2a5c0b37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:09.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.883+0000 7f2a5b400640 1 --2- 172.21.15.131:0/994577051 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2a5c0b3ce0 0x7f2a5c1f7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:09.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.883+0000 7f2a5aa00640 1 -- 172.21.15.131:0/994577051 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2a5c0ccb30 msgr2=0x7f2a5c0b37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:09.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.883+0000 7f2a5aa00640 1 --2- 172.21.15.131:0/994577051 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2a5c0ccb30 0x7f2a5c0b37a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:09.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.883+0000 7f2a5aa00640 1 -- 172.21.15.131:0/994577051 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2a5c0b3ce0 msgr2=0x7f2a5c1f7710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:09.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.883+0000 7f2a5aa00640 1 --2- 172.21.15.131:0/994577051 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2a5c0b3ce0 0x7f2a5c1f7710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:09.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.883+0000 7f2a5aa00640 1 -- 172.21.15.131:0/994577051 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a5c154bf0 con 0x7f2a400039b0 2024-05-26T22:50:09.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.883+0000 7f2a5aa00640 1 --2- 172.21.15.131:0/994577051 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a400039b0 0x7f2a5c14a5f0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c034040 tx=0x7f2a4c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:09.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.884+0000 7f2a52a00640 1 -- 172.21.15.131:0/994577051 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a4c03f020 con 0x7f2a400039b0 2024-05-26T22:50:09.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.884+0000 7f2a52a00640 1 -- 172.21.15.131:0/994577051 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a4c037040 con 0x7f2a400039b0 2024-05-26T22:50:09.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.884+0000 7f2a52a00640 1 -- 172.21.15.131:0/994577051 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a4c036070 con 0x7f2a400039b0 2024-05-26T22:50:09.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.884+0000 7f2a61600640 1 -- 172.21.15.131:0/994577051 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a5c147b80 con 0x7f2a400039b0 2024-05-26T22:50:09.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.884+0000 7f2a61600640 1 -- 172.21.15.131:0/994577051 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a5c14c150 con 0x7f2a400039b0 2024-05-26T22:50:09.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.885+0000 7f2a61600640 1 -- 172.21.15.131:0/994577051 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a5c1550a0 con 0x7f2a400039b0 2024-05-26T22:50:09.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.886+0000 7f2a52a00640 1 -- 172.21.15.131:0/994577051 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2a4c042020 con 0x7f2a400039b0 2024-05-26T22:50:09.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.887+0000 7f2a52a00640 1 --2- 172.21.15.131:0/994577051 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f2a3c06be30 0x7f2a3c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:09.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.887+0000 7f2a52a00640 1 -- 172.21.15.131:0/994577051 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f2a4c0aae90 con 0x7f2a400039b0 2024-05-26T22:50:09.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.887+0000 7f2a5a000640 1 --2- 172.21.15.131:0/994577051 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f2a3c06be30 0x7f2a3c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:09.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.888+0000 7f2a5a000640 1 --2- 172.21.15.131:0/994577051 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f2a3c06be30 0x7f2a3c06e2f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2a48000c00 tx=0x7f2a48009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:09.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:09.891+0000 7f2a52a00640 1 -- 172.21.15.131:0/994577051 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f2a5c1550a0 con 0x7f2a400039b0 2024-05-26T22:50:10.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.128+0000 7f2a61600640 1 -- 172.21.15.131:0/994577051 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2a5c0694e0 con 0x7f2a400039b0 2024-05-26T22:50:10.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.129+0000 7f2a52a00640 1 -- 172.21.15.131:0/994577051 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+5231 (secure 0 0 0) 0x7f2a4c049030 con 0x7f2a400039b0 2024-05-26T22:50:10.129 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:50:10.129 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":20,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/4292497989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":4292497989},{"type":"v1","addr":"172.21.15.187:6835","nonce":4292497989}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-26T22:49:30.955525+0000","modified":"2024-05-26T22:49:46.310449+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5115,"mds_1":4687,"mds_2":4678},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4678":{"gid":4678,"name":"a","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.131:6836/4032294430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":4032294430},{"type":"v1","addr":"172.21.15.131:6836","nonce":4032294430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4687":{"gid":4687,"name":"b","rank":1,"incarnation":17,"state":"up:active","state_seq":7,"addr":"172.21.15.187:6833/3274403209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3274403209},{"type":"v1","addr":"172.21.15.187:6833","nonce":3274403209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5115":{"gid":5115,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.131:6837/2062704413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6835","nonce":2062704413},{"type":"v1","addr":"172.21.15.131:6837","nonce":2062704413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5115,"qdb_cluster":[4678,4687,5115]},"id":2}]} 2024-05-26T22:50:10.130 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 20 2024-05-26T22:50:10.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.132+0000 7f2a61600640 1 -- 172.21.15.131:0/994577051 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f2a3c06be30 msgr2=0x7f2a3c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:10.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.132+0000 7f2a61600640 1 --2- 172.21.15.131:0/994577051 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f2a3c06be30 0x7f2a3c06e2f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2a48000c00 tx=0x7f2a48009040 comp rx=0 tx=0).stop 2024-05-26T22:50:10.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.132+0000 7f2a61600640 1 -- 172.21.15.131:0/994577051 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a400039b0 msgr2=0x7f2a5c14a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:10.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.132+0000 7f2a61600640 1 --2- 172.21.15.131:0/994577051 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a400039b0 0x7f2a5c14a5f0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c034040 tx=0x7f2a4c004730 comp rx=0 tx=0).stop 2024-05-26T22:50:10.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.132+0000 7f2a61600640 1 -- 172.21.15.131:0/994577051 shutdown_connections 2024-05-26T22:50:10.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.132+0000 7f2a61600640 1 --2- 172.21.15.131:0/994577051 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2a5c0ccb30 0x7f2a5c0b37a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:10.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.132+0000 7f2a61600640 1 --2- 172.21.15.131:0/994577051 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f2a3c06be30 0x7f2a3c06e2f0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:10.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.132+0000 7f2a61600640 1 --2- 172.21.15.131:0/994577051 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2a5c0b3ce0 0x7f2a5c1f7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:10.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.132+0000 7f2a61600640 1 --2- 172.21.15.131:0/994577051 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a400039b0 0x7f2a5c14a5f0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:10.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.132+0000 7f2a61600640 1 -- 172.21.15.131:0/994577051 >> 172.21.15.131:0/994577051 conn(0x7f2a5c0b1e90 msgr2=0x7f2a5c06e4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:10.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.132+0000 7f2a61600640 1 -- 172.21.15.131:0/994577051 shutdown_connections 2024-05-26T22:50:10.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.132+0000 7f2a61600640 1 -- 172.21.15.131:0/994577051 wait complete. 2024-05-26T22:50:10.151 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-26T22:50:10.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.338+0000 7f5879200640 1 Processor -- start 2024-05-26T22:50:10.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.339+0000 7f5879200640 1 -- start start 2024-05-26T22:50:10.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.340+0000 7f5879200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5874156750 0x7f587415fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:10.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.340+0000 7f5879200640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f587406e8c0 0x7f587415d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:10.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.340+0000 7f5879200640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5874160620 con 0x7f58740cc7f0 2024-05-26T22:50:10.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.340+0000 7f5879200640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f587415cbd0 con 0x7f587406e8c0 2024-05-26T22:50:10.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.340+0000 7f5879200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f5874160900 con 0x7f5874156750 2024-05-26T22:50:10.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.341+0000 7f5873400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5874156750 0x7f587415fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:10.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.341+0000 7f5873400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5874156750 0x7f587415fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.131:53748/0 (socket says 172.21.15.131:53748) 2024-05-26T22:50:10.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.341+0000 7f5873400640 1 -- 172.21.15.131:0/957645444 learned_addr learned my addr 172.21.15.131:0/957645444 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:10.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.341+0000 7f5872a00640 1 --2- 172.21.15.131:0/957645444 >> v2:172.21.15.187:3300/0 conn(0x7f587406e8c0 0x7f587415d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:10.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.342+0000 7f5873400640 1 -- 172.21.15.131:0/957645444 >> v2:172.21.15.187:3300/0 conn(0x7f587406e8c0 msgr2=0x7f587415d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:10.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.342+0000 7f5873400640 1 --2- 172.21.15.131:0/957645444 >> v2:172.21.15.187:3300/0 conn(0x7f587406e8c0 0x7f587415d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:10.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.342+0000 7f5873400640 1 -- 172.21.15.131:0/957645444 >> v1:172.21.15.187:6789/0 conn(0x7f58740cc7f0 legacy=0x7f58740cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:10.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.342+0000 7f5873400640 1 -- 172.21.15.131:0/957645444 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5874160be0 con 0x7f5874156750 2024-05-26T22:50:10.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.343+0000 7f5873400640 1 --2- 172.21.15.131:0/957645444 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5874156750 0x7f587415fd40 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5864009870 tx=0x7f586402a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=232b879ff0fc79fa server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:10.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.344+0000 7f5872000640 1 -- 172.21.15.131:0/957645444 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5864037070 con 0x7f5874156750 2024-05-26T22:50:10.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.344+0000 7f5872000640 1 -- 172.21.15.131:0/957645444 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5864032440 con 0x7f5874156750 2024-05-26T22:50:10.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.344+0000 7f5872000640 1 -- 172.21.15.131:0/957645444 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5864032810 con 0x7f5874156750 2024-05-26T22:50:10.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.345+0000 7f5879200640 1 -- 172.21.15.131:0/957645444 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5874156750 msgr2=0x7f587415fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:10.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.345+0000 7f5879200640 1 --2- 172.21.15.131:0/957645444 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5874156750 0x7f587415fd40 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5864009870 tx=0x7f586402a270 comp rx=0 tx=0).stop 2024-05-26T22:50:10.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.345+0000 7f5879200640 1 -- 172.21.15.131:0/957645444 shutdown_connections 2024-05-26T22:50:10.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.345+0000 7f5879200640 1 --2- 172.21.15.131:0/957645444 >> v2:172.21.15.187:3300/0 conn(0x7f587406e8c0 0x7f587415d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:10.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.345+0000 7f5879200640 1 --2- 172.21.15.131:0/957645444 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5874156750 0x7f587415fd40 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:10.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.345+0000 7f5879200640 1 -- 172.21.15.131:0/957645444 >> 172.21.15.131:0/957645444 conn(0x7f58740cff40 msgr2=0x7f58740d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:10.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.345+0000 7f5879200640 1 -- 172.21.15.131:0/957645444 shutdown_connections 2024-05-26T22:50:10.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.345+0000 7f5879200640 1 -- 172.21.15.131:0/957645444 wait complete. 2024-05-26T22:50:10.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.347+0000 7f5879200640 1 Processor -- start 2024-05-26T22:50:10.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.347+0000 7f5879200640 1 -- start start 2024-05-26T22:50:10.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.348+0000 7f5879200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f587406e8c0 0x7f587414d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:10.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.348+0000 7f5879200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f58740cc7f0 0x7f58740b11d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:10.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.348+0000 7f5879200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5874156750 0x7f58741f0020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:10.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.348+0000 7f5879200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f5874160280 con 0x7f5874156750 2024-05-26T22:50:10.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.348+0000 7f5879200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5874156b30 con 0x7f587406e8c0 2024-05-26T22:50:10.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.348+0000 7f5879200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f5874160400 con 0x7f58740cc7f0 2024-05-26T22:50:10.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.348+0000 7f5873e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5874156750 0x7f58741f0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:10.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.348+0000 7f5873e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5874156750 0x7f58741f0020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37844/0 (socket says 172.21.15.131:37844) 2024-05-26T22:50:10.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.348+0000 7f5873e00640 1 -- 172.21.15.131:0/114286617 learned_addr learned my addr 172.21.15.131:0/114286617 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:10.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.349+0000 7f5873400640 1 --2- 172.21.15.131:0/114286617 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f587406e8c0 0x7f587414d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:10.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.349+0000 7f5872a00640 1 --2- 172.21.15.131:0/114286617 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f58740cc7f0 0x7f58740b11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:10.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.349+0000 7f5873e00640 1 -- 172.21.15.131:0/114286617 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f58740cc7f0 msgr2=0x7f58740b11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:10.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.349+0000 7f5873e00640 1 --2- 172.21.15.131:0/114286617 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f58740cc7f0 0x7f58740b11d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:10.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.349+0000 7f5873e00640 1 -- 172.21.15.131:0/114286617 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f587406e8c0 msgr2=0x7f587414d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:10.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.349+0000 7f5873e00640 1 --2- 172.21.15.131:0/114286617 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f587406e8c0 0x7f587414d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:10.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.349+0000 7f5873e00640 1 -- 172.21.15.131:0/114286617 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f587414e370 con 0x7f5874156750 2024-05-26T22:50:10.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.349+0000 7f5873e00640 1 --2- 172.21.15.131:0/114286617 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5874156750 0x7f58741f0020 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f586c00c7a0 tx=0x7f586c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:10.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.349+0000 7f586b400640 1 -- 172.21.15.131:0/114286617 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f586c003b70 con 0x7f5874156750 2024-05-26T22:50:10.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.349+0000 7f586b400640 1 -- 172.21.15.131:0/114286617 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f586c003d30 con 0x7f5874156750 2024-05-26T22:50:10.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.350+0000 7f586b400640 1 -- 172.21.15.131:0/114286617 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f586c00aa90 con 0x7f5874156750 2024-05-26T22:50:10.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.350+0000 7f5879200640 1 -- 172.21.15.131:0/114286617 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f587414e650 con 0x7f5874156750 2024-05-26T22:50:10.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.350+0000 7f5879200640 1 -- 172.21.15.131:0/114286617 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f587414ec10 con 0x7f5874156750 2024-05-26T22:50:10.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.351+0000 7f5879200640 1 -- 172.21.15.131:0/114286617 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58740694e0 con 0x7f5874156750 2024-05-26T22:50:10.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.352+0000 7f586b400640 1 -- 172.21.15.131:0/114286617 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f586c00ac30 con 0x7f5874156750 2024-05-26T22:50:10.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.353+0000 7f586b400640 1 --2- 172.21.15.131:0/114286617 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f585406be30 0x7f585406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:10.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.353+0000 7f586b400640 1 -- 172.21.15.131:0/114286617 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f586c08ae10 con 0x7f5874156750 2024-05-26T22:50:10.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.353+0000 7f5873400640 1 --2- 172.21.15.131:0/114286617 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f585406be30 0x7f585406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:10.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.354+0000 7f5873400640 1 --2- 172.21.15.131:0/114286617 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f585406be30 0x7f585406e2f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f5864034000 tx=0x7f586402e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:10.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.358+0000 7f586b400640 1 -- 172.21.15.131:0/114286617 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f586c0597c0 con 0x7f5874156750 2024-05-26T22:50:10.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.560+0000 7f5879200640 1 -- 172.21.15.131:0/114286617 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f58740cea10 con 0x7f5874156750 2024-05-26T22:50:10.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.563+0000 7f586b400640 1 -- 172.21.15.131:0/114286617 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) ==== 74+0+16693 (secure 0 0 0) 0x7f586c0590c0 con 0x7f5874156750 2024-05-26T22:50:10.563 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:50:10.564 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":33,"fsid":"46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f","created":"2024-05-26T22:47:57.288905+0000","modified":"2024-05-26T22:49:32.532828+0000","last_up_change":"2024-05-26T22:48:09.602207+0000","last_in_change":"2024-05-26T22:48:00.687341+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-26T22:48:10.463339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-05-26T22:49:28.813374+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-05-26T22:49:29.987297+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"814eaf62-a18d-494b-8e27-fd8fb88adea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6801","nonce":2350940051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6803","nonce":2350940051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6807","nonce":2350940051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6805","nonce":2350940051}]},"public_addr":"172.21.15.131:6801/2350940051","cluster_addr":"172.21.15.131:6803/2350940051","heartbeat_back_addr":"172.21.15.131:6807/2350940051","heartbeat_front_addr":"172.21.15.131:6805/2350940051","state":["exists","up"]},{"osd":1,"uuid":"42e66287-f6c6-4e26-af3c-6dfaa3536d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6825","nonce":1525528247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6827","nonce":1525528247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6831","nonce":1525528247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6829","nonce":1525528247}]},"public_addr":"172.21.15.131:6825/1525528247","cluster_addr":"172.21.15.131:6827/1525528247","heartbeat_back_addr":"172.21.15.131:6831/1525528247","heartbeat_front_addr":"172.21.15.131:6829/1525528247","state":["exists","up"]},{"osd":2,"uuid":"0da34673-04a2-4fd0-b57a-0ff2b679b043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6817","nonce":3627842210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6819","nonce":3627842210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6823","nonce":3627842210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6821","nonce":3627842210}]},"public_addr":"172.21.15.131:6817/3627842210","cluster_addr":"172.21.15.131:6819/3627842210","heartbeat_back_addr":"172.21.15.131:6823/3627842210","heartbeat_front_addr":"172.21.15.131:6821/3627842210","state":["exists","up"]},{"osd":3,"uuid":"cf117a27-d6b2-43d3-ae52-b47988bf8eed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6809","nonce":350790591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6811","nonce":350790591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6815","nonce":350790591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6813","nonce":350790591}]},"public_addr":"172.21.15.131:6809/350790591","cluster_addr":"172.21.15.131:6811/350790591","heartbeat_back_addr":"172.21.15.131:6815/350790591","heartbeat_front_addr":"172.21.15.131:6813/350790591","state":["exists","up"]},{"osd":4,"uuid":"a092ccfc-be89-4a84-88d5-0f781fa715eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6816","nonce":244560995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6819","nonce":244560995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6823","nonce":244560995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6821","nonce":244560995}]},"public_addr":"172.21.15.187:6816/244560995","cluster_addr":"172.21.15.187:6819/244560995","heartbeat_back_addr":"172.21.15.187:6823/244560995","heartbeat_front_addr":"172.21.15.187:6821/244560995","state":["exists","up"]},{"osd":5,"uuid":"7513ba69-dd00-4a35-b190-9e8cc3b2bdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6801","nonce":324619398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6803","nonce":324619398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6812","nonce":324619398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6807","nonce":324619398}]},"public_addr":"172.21.15.187:6801/324619398","cluster_addr":"172.21.15.187:6803/324619398","heartbeat_back_addr":"172.21.15.187:6812/324619398","heartbeat_front_addr":"172.21.15.187:6807/324619398","state":["exists","up"]},{"osd":6,"uuid":"50081810-c093-4d2f-a4d7-b7c695c5400d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6806","nonce":1391347786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6809","nonce":1391347786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6818","nonce":1391347786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6813","nonce":1391347786}]},"public_addr":"172.21.15.187:6806/1391347786","cluster_addr":"172.21.15.187:6809/1391347786","heartbeat_back_addr":"172.21.15.187:6818/1391347786","heartbeat_front_addr":"172.21.15.187:6813/1391347786","state":["exists","up"]},{"osd":7,"uuid":"775dae05-54f5-4ad4-bbce-92863c6ed2ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6825","nonce":1220065622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6827","nonce":1220065622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6831","nonce":1220065622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6829","nonce":1220065622}]},"public_addr":"172.21.15.187:6825/1220065622","cluster_addr":"172.21.15.187:6827/1220065622","heartbeat_back_addr":"172.21.15.187:6831/1220065622","heartbeat_front_addr":"172.21.15.187:6829/1220065622","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:05.983520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.299516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.067323+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.307748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.018742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.258775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.010912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.295355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-26T22:50:10.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.566+0000 7f5879200640 1 -- 172.21.15.131:0/114286617 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f585406be30 msgr2=0x7f585406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:10.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.566+0000 7f5879200640 1 --2- 172.21.15.131:0/114286617 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f585406be30 0x7f585406e2f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f5864034000 tx=0x7f586402e000 comp rx=0 tx=0).stop 2024-05-26T22:50:10.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.566+0000 7f5879200640 1 -- 172.21.15.131:0/114286617 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5874156750 msgr2=0x7f58741f0020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:10.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.566+0000 7f5879200640 1 --2- 172.21.15.131:0/114286617 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5874156750 0x7f58741f0020 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f586c00c7a0 tx=0x7f586c00cc60 comp rx=0 tx=0).stop 2024-05-26T22:50:10.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.566+0000 7f5879200640 1 -- 172.21.15.131:0/114286617 shutdown_connections 2024-05-26T22:50:10.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.566+0000 7f5879200640 1 --2- 172.21.15.131:0/114286617 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f58740cc7f0 0x7f58740b11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:10.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.566+0000 7f5879200640 1 --2- 172.21.15.131:0/114286617 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f585406be30 0x7f585406e2f0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:10.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.566+0000 7f5879200640 1 --2- 172.21.15.131:0/114286617 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5874156750 0x7f58741f0020 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:10.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.566+0000 7f5879200640 1 --2- 172.21.15.131:0/114286617 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f587406e8c0 0x7f587414d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:10.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.566+0000 7f5879200640 1 -- 172.21.15.131:0/114286617 >> 172.21.15.131:0/114286617 conn(0x7f58740cff40 msgr2=0x7f5874071c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:10.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.567+0000 7f5879200640 1 -- 172.21.15.131:0/114286617 shutdown_connections 2024-05-26T22:50:10.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.567+0000 7f5879200640 1 -- 172.21.15.131:0/114286617 wait complete. 2024-05-26T22:50:10.587 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-26T22:50:10.587 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-26T22:50:10.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.780+0000 7fdd15600640 1 Processor -- start 2024-05-26T22:50:10.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.780+0000 7fdd15600640 1 -- start start 2024-05-26T22:50:10.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.782+0000 7fdd15600640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fdd1006ee20 0x7fdd100b3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:10.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.782+0000 7fdd15600640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd1015cb20 con 0x7fdd100ad250 2024-05-26T22:50:10.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.782+0000 7fdd15600640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd1015ce00 con 0x7fdd101569d0 2024-05-26T22:50:10.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.782+0000 7fdd15600640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fdd1015d0e0 con 0x7fdd1006ee20 2024-05-26T22:50:10.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.782+0000 7fdd0ea00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fdd1006ee20 0x7fdd100b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:10.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.782+0000 7fdd06000640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fdd100ad250 0x7fdd101562c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:36360/0 (socket says 172.21.15.131:36360) 2024-05-26T22:50:10.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.782+0000 7fdd0ea00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fdd1006ee20 0x7fdd100b3190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37852/0 (socket says 172.21.15.131:37852) 2024-05-26T22:50:10.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.782+0000 7fdd06000640 1 -- 172.21.15.131:0/1402387119 learned_addr learned my addr 172.21.15.131:0/1402387119 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:50:10.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.783+0000 7fdd0ea00640 1 -- 172.21.15.131:0/1402387119 >> v1:172.21.15.187:6789/0 conn(0x7fdd101569d0 legacy=0x7fdd10160010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:10.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.783+0000 7fdd0ea00640 1 -- 172.21.15.131:0/1402387119 >> v1:172.21.15.131:6789/0 conn(0x7fdd100ad250 legacy=0x7fdd101562c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:10.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.783+0000 7fdd0ea00640 1 -- 172.21.15.131:0/1402387119 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd1015d3c0 con 0x7fdd1006ee20 2024-05-26T22:50:10.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.784+0000 7fdd0ea00640 1 --2- 172.21.15.131:0/1402387119 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd1006ee20 0x7fdd100b3190 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fdd00009d90 tx=0x7fdd0002a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bfaefd0ff41a3c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:10.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.784+0000 7fdd0e000640 1 -- 172.21.15.131:0/1402387119 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd00031600 con 0x7fdd1006ee20 2024-05-26T22:50:10.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.784+0000 7fdd0e000640 1 -- 172.21.15.131:0/1402387119 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd000317a0 con 0x7fdd1006ee20 2024-05-26T22:50:10.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.784+0000 7fdd0e000640 1 -- 172.21.15.131:0/1402387119 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd00031b70 con 0x7fdd1006ee20 2024-05-26T22:50:10.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.785+0000 7fdd15600640 1 -- 172.21.15.131:0/1402387119 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd1006ee20 msgr2=0x7fdd100b3190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:10.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.785+0000 7fdd15600640 1 --2- 172.21.15.131:0/1402387119 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd1006ee20 0x7fdd100b3190 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fdd00009d90 tx=0x7fdd0002a610 comp rx=0 tx=0).stop 2024-05-26T22:50:10.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.785+0000 7fdd15600640 1 -- 172.21.15.131:0/1402387119 shutdown_connections 2024-05-26T22:50:10.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.785+0000 7fdd15600640 1 --2- 172.21.15.131:0/1402387119 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd1006ee20 0x7fdd100b3190 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:10.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.785+0000 7fdd15600640 1 -- 172.21.15.131:0/1402387119 >> 172.21.15.131:0/1402387119 conn(0x7fdd100b1e90 msgr2=0x7fdd100b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:10.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.785+0000 7fdd15600640 1 -- 172.21.15.131:0/1402387119 shutdown_connections 2024-05-26T22:50:10.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.785+0000 7fdd15600640 1 -- 172.21.15.131:0/1402387119 wait complete. 2024-05-26T22:50:10.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.787+0000 7fdd15600640 1 Processor -- start 2024-05-26T22:50:10.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.787+0000 7fdd15600640 1 -- start start 2024-05-26T22:50:10.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.788+0000 7fdd15600640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdd1006ee20 0x7fdd10142310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:10.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.788+0000 7fdd15600640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdd100ad250 0x7fdd1009a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:10.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.788+0000 7fdd15600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd101569d0 0x7fdd10144d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:10.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.788+0000 7fdd15600640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fdd1015c900 con 0x7fdd101569d0 2024-05-26T22:50:10.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.788+0000 7fdd15600640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fdd1015c780 con 0x7fdd1006ee20 2024-05-26T22:50:10.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.788+0000 7fdd15600640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fdd1006f200 con 0x7fdd100ad250 2024-05-26T22:50:10.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.788+0000 7fdd0f400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd101569d0 0x7fdd10144d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:10.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.788+0000 7fdd0f400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd101569d0 0x7fdd10144d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37860/0 (socket says 172.21.15.131:37860) 2024-05-26T22:50:10.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.788+0000 7fdd0f400640 1 -- 172.21.15.131:0/677928146 learned_addr learned my addr 172.21.15.131:0/677928146 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:10.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.788+0000 7fdd0ea00640 1 --2- 172.21.15.131:0/677928146 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdd1006ee20 0x7fdd10142310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:10.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.788+0000 7fdd0f400640 1 -- 172.21.15.131:0/677928146 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdd100ad250 msgr2=0x7fdd1009a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:10.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.788+0000 7fdd0f400640 1 --2- 172.21.15.131:0/677928146 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdd100ad250 0x7fdd1009a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:10.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.788+0000 7fdd0f400640 1 -- 172.21.15.131:0/677928146 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdd1006ee20 msgr2=0x7fdd10142310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:10.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.788+0000 7fdd0f400640 1 --2- 172.21.15.131:0/677928146 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdd1006ee20 0x7fdd10142310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:10.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.788+0000 7fdd0f400640 1 -- 172.21.15.131:0/677928146 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd10142a10 con 0x7fdd101569d0 2024-05-26T22:50:10.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.788+0000 7fdd0ea00640 1 --2- 172.21.15.131:0/677928146 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdd1006ee20 0x7fdd10142310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:50:10.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.789+0000 7fdd0f400640 1 --2- 172.21.15.131:0/677928146 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd101569d0 0x7fdd10144d30 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fdd0800b410 tx=0x7fdd0800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:10.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.789+0000 7fdd07400640 1 -- 172.21.15.131:0/677928146 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd08015030 con 0x7fdd101569d0 2024-05-26T22:50:10.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.789+0000 7fdd07400640 1 -- 172.21.15.131:0/677928146 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd08004020 con 0x7fdd101569d0 2024-05-26T22:50:10.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.789+0000 7fdd07400640 1 -- 172.21.15.131:0/677928146 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd08004450 con 0x7fdd101569d0 2024-05-26T22:50:10.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.789+0000 7fdd15600640 1 -- 172.21.15.131:0/677928146 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd10142cf0 con 0x7fdd101569d0 2024-05-26T22:50:10.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.789+0000 7fdd15600640 1 -- 172.21.15.131:0/677928146 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd101432b0 con 0x7fdd101569d0 2024-05-26T22:50:10.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.790+0000 7fdd15600640 1 -- 172.21.15.131:0/677928146 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd100694e0 con 0x7fdd101569d0 2024-05-26T22:50:10.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.791+0000 7fdd07400640 1 -- 172.21.15.131:0/677928146 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdd08018070 con 0x7fdd101569d0 2024-05-26T22:50:10.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.792+0000 7fdd07400640 1 --2- 172.21.15.131:0/677928146 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fdcec06be30 0x7fdcec06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:10.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.792+0000 7fdd07400640 1 -- 172.21.15.131:0/677928146 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fdd0808aa90 con 0x7fdd101569d0 2024-05-26T22:50:10.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.792+0000 7fdd0ea00640 1 --2- 172.21.15.131:0/677928146 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fdcec06be30 0x7fdcec06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:10.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.793+0000 7fdd0ea00640 1 --2- 172.21.15.131:0/677928146 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fdcec06be30 0x7fdcec06e2f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fdd00034000 tx=0x7fdd0002f000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:10.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:10.796+0000 7fdd07400640 1 -- 172.21.15.131:0/677928146 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fdd08059310 con 0x7fdd101569d0 2024-05-26T22:50:11.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.035+0000 7fdd15600640 1 -- 172.21.15.131:0/677928146 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdd1006e110 con 0x7fdd101569d0 2024-05-26T22:50:11.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.037+0000 7fdd07400640 1 -- 172.21.15.131:0/677928146 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+5231 (secure 0 0 0) 0x7fdd08058c10 con 0x7fdd101569d0 2024-05-26T22:50:11.037 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:50:11.037 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":20,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/4292497989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":4292497989},{"type":"v1","addr":"172.21.15.187:6835","nonce":4292497989}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-26T22:49:30.955525+0000","modified":"2024-05-26T22:49:46.310449+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5115,"mds_1":4687,"mds_2":4678},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4678":{"gid":4678,"name":"a","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.131:6836/4032294430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":4032294430},{"type":"v1","addr":"172.21.15.131:6836","nonce":4032294430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4687":{"gid":4687,"name":"b","rank":1,"incarnation":17,"state":"up:active","state_seq":7,"addr":"172.21.15.187:6833/3274403209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3274403209},{"type":"v1","addr":"172.21.15.187:6833","nonce":3274403209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5115":{"gid":5115,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.131:6837/2062704413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6835","nonce":2062704413},{"type":"v1","addr":"172.21.15.131:6837","nonce":2062704413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5115,"qdb_cluster":[4678,4687,5115]},"id":2}]} 2024-05-26T22:50:11.037 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 20 2024-05-26T22:50:11.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.040+0000 7fdd15600640 1 -- 172.21.15.131:0/677928146 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fdcec06be30 msgr2=0x7fdcec06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:11.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.040+0000 7fdd15600640 1 --2- 172.21.15.131:0/677928146 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fdcec06be30 0x7fdcec06e2f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fdd00034000 tx=0x7fdd0002f000 comp rx=0 tx=0).stop 2024-05-26T22:50:11.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.040+0000 7fdd15600640 1 -- 172.21.15.131:0/677928146 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd101569d0 msgr2=0x7fdd10144d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:11.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.040+0000 7fdd15600640 1 --2- 172.21.15.131:0/677928146 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd101569d0 0x7fdd10144d30 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fdd0800b410 tx=0x7fdd0800b8d0 comp rx=0 tx=0).stop 2024-05-26T22:50:11.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.040+0000 7fdd15600640 1 -- 172.21.15.131:0/677928146 shutdown_connections 2024-05-26T22:50:11.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.040+0000 7fdd15600640 1 --2- 172.21.15.131:0/677928146 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdd100ad250 0x7fdd1009a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:11.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.040+0000 7fdd15600640 1 --2- 172.21.15.131:0/677928146 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fdcec06be30 0x7fdcec06e2f0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:11.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.040+0000 7fdd15600640 1 --2- 172.21.15.131:0/677928146 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd101569d0 0x7fdd10144d30 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:11.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.040+0000 7fdd15600640 1 --2- 172.21.15.131:0/677928146 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdd1006ee20 0x7fdd10142310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:11.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.040+0000 7fdd15600640 1 -- 172.21.15.131:0/677928146 >> 172.21.15.131:0/677928146 conn(0x7fdd100b1e90 msgr2=0x7fdd100b1710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:11.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.040+0000 7fdd15600640 1 -- 172.21.15.131:0/677928146 shutdown_connections 2024-05-26T22:50:11.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.040+0000 7fdd15600640 1 -- 172.21.15.131:0/677928146 wait complete. 2024-05-26T22:50:11.059 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-26T22:50:11.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.259+0000 7f335a200640 1 Processor -- start 2024-05-26T22:50:11.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.259+0000 7f335a200640 1 -- start start 2024-05-26T22:50:11.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.261+0000 7f335a200640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f335406e8c0 0x7f335415d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:11.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.261+0000 7f335a200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f33540cc7f0 0x7f33540ccca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:11.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.261+0000 7f335a200640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3354160620 con 0x7f3354156750 2024-05-26T22:50:11.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.261+0000 7f335a200640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f335415cbd0 con 0x7f335406e8c0 2024-05-26T22:50:11.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.261+0000 7f335a200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3354160900 con 0x7f33540cc7f0 2024-05-26T22:50:11.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.261+0000 7f3358e00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f3354156750 0x7f335415fd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:36364/0 (socket says 172.21.15.131:36364) 2024-05-26T22:50:11.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.261+0000 7f3358e00640 1 -- 172.21.15.131:0/1439313970 learned_addr learned my addr 172.21.15.131:0/1439313970 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:50:11.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.261+0000 7f3353e00640 1 --2- 172.21.15.131:0/1439313970 >> v2:172.21.15.187:3300/0 conn(0x7f335406e8c0 0x7f335415d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:11.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.261+0000 7f3353400640 1 -- 172.21.15.131:0/1439313970 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3852708688 0 0) 0x7f3354160620 con 0x7f3354156750 2024-05-26T22:50:11.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.261+0000 7f3359800640 1 --2- 172.21.15.131:0/1439313970 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f33540cc7f0 0x7f33540ccca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:11.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.261+0000 7f3353400640 1 -- 172.21.15.131:0/1439313970 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f333c003520 con 0x7f3354156750 2024-05-26T22:50:11.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.262+0000 7f3353400640 1 -- 172.21.15.131:0/1439313970 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2340105929 0 0) 0x7f333c003520 con 0x7f3354156750 2024-05-26T22:50:11.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.262+0000 7f3353400640 1 -- 172.21.15.131:0/1439313970 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f33540cc7f0 msgr2=0x7f33540ccca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:11.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.262+0000 7f3353400640 1 --2- 172.21.15.131:0/1439313970 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f33540cc7f0 0x7f33540ccca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:11.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.262+0000 7f3353400640 1 -- 172.21.15.131:0/1439313970 >> v2:172.21.15.187:3300/0 conn(0x7f335406e8c0 msgr2=0x7f335415d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:11.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.262+0000 7f3353400640 1 --2- 172.21.15.131:0/1439313970 >> v2:172.21.15.187:3300/0 conn(0x7f335406e8c0 0x7f335415d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:11.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.262+0000 7f3353400640 1 -- 172.21.15.131:0/1439313970 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3354160be0 con 0x7f3354156750 2024-05-26T22:50:11.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.262+0000 7f3353400640 1 -- 172.21.15.131:0/1439313970 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7f3348001670 con 0x7f3354156750 2024-05-26T22:50:11.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.262+0000 7f3353400640 1 -- 172.21.15.131:0/1439313970 >> v1:172.21.15.131:6789/0 conn(0x7f3354156750 legacy=0x7f335415fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:11.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.262+0000 7f3353400640 1 --2- 172.21.15.131:0/1439313970 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f333c0039b0 0x7f333c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:11.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.262+0000 7f3353400640 1 --2- 172.21.15.131:0/1439313970 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f333c004800 0x7f333c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:11.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.262+0000 7f3353400640 1 --2- 172.21.15.131:0/1439313970 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f33540cc7f0 0x7f333c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:11.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.262+0000 7f3353400640 1 -- 172.21.15.131:0/1439313970 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f335415cbd0 con 0x7f333c0039b0 2024-05-26T22:50:11.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.262+0000 7f3353400640 1 -- 172.21.15.131:0/1439313970 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3354160900 con 0x7f333c004800 2024-05-26T22:50:11.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.262+0000 7f3353400640 1 -- 172.21.15.131:0/1439313970 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f333c00ba70 con 0x7f33540cc7f0 2024-05-26T22:50:11.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.262+0000 7f3358e00640 1 --2- 172.21.15.131:0/1439313970 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f333c0039b0 0x7f333c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:11.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.263+0000 7f3353e00640 1 --2- 172.21.15.131:0/1439313970 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f333c004800 0x7f333c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:11.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.263+0000 7f3359800640 1 --2- 172.21.15.131:0/1439313970 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f33540cc7f0 0x7f333c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:11.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.263+0000 7f3358e00640 1 -- 172.21.15.131:0/1439313970 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f33540cc7f0 msgr2=0x7f333c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:11.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.263+0000 7f3358e00640 1 --2- 172.21.15.131:0/1439313970 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f33540cc7f0 0x7f333c0051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:11.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.263+0000 7f3358e00640 1 -- 172.21.15.131:0/1439313970 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f333c004800 msgr2=0x7f333c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:11.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.263+0000 7f3358e00640 1 --2- 172.21.15.131:0/1439313970 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f333c004800 0x7f333c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:11.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.263+0000 7f3358e00640 1 -- 172.21.15.131:0/1439313970 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f333c003520 con 0x7f333c0039b0 2024-05-26T22:50:11.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.264+0000 7f3358e00640 1 --2- 172.21.15.131:0/1439313970 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f333c0039b0 0x7f333c003dd0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f334800c210 tx=0x7f333c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6245ebf8302ce00 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:11.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.264+0000 7f3353400640 1 -- 172.21.15.131:0/1439313970 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3348007d30 con 0x7f333c0039b0 2024-05-26T22:50:11.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.264+0000 7f3353400640 1 -- 172.21.15.131:0/1439313970 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33480058d0 con 0x7f333c0039b0 2024-05-26T22:50:11.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.264+0000 7f3353400640 1 -- 172.21.15.131:0/1439313970 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3348005ca0 con 0x7f333c0039b0 2024-05-26T22:50:11.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.265+0000 7f335a200640 1 -- 172.21.15.131:0/1439313970 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f333c0039b0 msgr2=0x7f333c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:11.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.265+0000 7f335a200640 1 --2- 172.21.15.131:0/1439313970 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f333c0039b0 0x7f333c003dd0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f334800c210 tx=0x7f333c005830 comp rx=0 tx=0).stop 2024-05-26T22:50:11.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.265+0000 7f3359800640 1 -- 172.21.15.131:0/1439313970 reap_dead start 2024-05-26T22:50:11.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.265+0000 7f335a200640 1 -- 172.21.15.131:0/1439313970 shutdown_connections 2024-05-26T22:50:11.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.265+0000 7f335a200640 1 -- 172.21.15.131:0/1439313970 >> 172.21.15.131:0/1439313970 conn(0x7f33540cff40 msgr2=0x7f33540d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:11.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.265+0000 7f335a200640 1 -- 172.21.15.131:0/1439313970 shutdown_connections 2024-05-26T22:50:11.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.265+0000 7f335a200640 1 -- 172.21.15.131:0/1439313970 wait complete. 2024-05-26T22:50:11.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.267+0000 7f335a200640 1 Processor -- start 2024-05-26T22:50:11.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.267+0000 7f335a200640 1 -- start start 2024-05-26T22:50:11.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.268+0000 7f335a200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f333c0039b0 0x7f335413e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:11.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.268+0000 7f335a200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f33541576c0 0x7f3354155cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:11.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.268+0000 7f335a200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f33540b0360 0x7f33541585a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:11.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.268+0000 7f335a200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f335415d5a0 con 0x7f33540b0360 2024-05-26T22:50:11.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.268+0000 7f335a200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3354156b30 con 0x7f33541576c0 2024-05-26T22:50:11.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.268+0000 7f335a200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f335415d720 con 0x7f333c0039b0 2024-05-26T22:50:11.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.268+0000 7f3359800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f33540b0360 0x7f33541585a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:11.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.268+0000 7f3359800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f33540b0360 0x7f33541585a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37882/0 (socket says 172.21.15.131:37882) 2024-05-26T22:50:11.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.268+0000 7f3359800640 1 -- 172.21.15.131:0/994688395 learned_addr learned my addr 172.21.15.131:0/994688395 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:11.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.268+0000 7f3353e00640 1 --2- 172.21.15.131:0/994688395 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f33541576c0 0x7f3354155cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:11.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.268+0000 7f3358e00640 1 --2- 172.21.15.131:0/994688395 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f333c0039b0 0x7f335413e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:11.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.269+0000 7f3359800640 1 -- 172.21.15.131:0/994688395 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f333c0039b0 msgr2=0x7f335413e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:11.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.269+0000 7f3359800640 1 --2- 172.21.15.131:0/994688395 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f333c0039b0 0x7f335413e2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:11.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.269+0000 7f3359800640 1 -- 172.21.15.131:0/994688395 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f33541576c0 msgr2=0x7f3354155cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:11.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.269+0000 7f3359800640 1 --2- 172.21.15.131:0/994688395 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f33541576c0 0x7f3354155cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:11.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.269+0000 7f3359800640 1 -- 172.21.15.131:0/994688395 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33540b2950 con 0x7f33540b0360 2024-05-26T22:50:11.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.269+0000 7f3359800640 1 --2- 172.21.15.131:0/994688395 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f33540b0360 0x7f33541585a0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f334c009820 tx=0x7f334c009ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:11.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.269+0000 7f3350c00640 1 -- 172.21.15.131:0/994688395 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f334c019070 con 0x7f33540b0360 2024-05-26T22:50:11.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.269+0000 7f3350c00640 1 -- 172.21.15.131:0/994688395 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f334c007d40 con 0x7f33540b0360 2024-05-26T22:50:11.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.269+0000 7f3350c00640 1 -- 172.21.15.131:0/994688395 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f334c014640 con 0x7f33540b0360 2024-05-26T22:50:11.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.269+0000 7f335a200640 1 -- 172.21.15.131:0/994688395 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33540ada10 con 0x7f33540b0360 2024-05-26T22:50:11.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.269+0000 7f335a200640 1 -- 172.21.15.131:0/994688395 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33540adfd0 con 0x7f33540b0360 2024-05-26T22:50:11.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.270+0000 7f335a200640 1 -- 172.21.15.131:0/994688395 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f335415c040 con 0x7f33540b0360 2024-05-26T22:50:11.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.271+0000 7f3350c00640 1 -- 172.21.15.131:0/994688395 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f334c01c070 con 0x7f33540b0360 2024-05-26T22:50:11.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.273+0000 7f3350c00640 1 --2- 172.21.15.131:0/994688395 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f333406be30 0x7f333406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:11.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.273+0000 7f3350c00640 1 -- 172.21.15.131:0/994688395 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f334c08aaa0 con 0x7f33540b0360 2024-05-26T22:50:11.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.273+0000 7f3358e00640 1 --2- 172.21.15.131:0/994688395 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f333406be30 0x7f333406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:11.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.274+0000 7f3358e00640 1 --2- 172.21.15.131:0/994688395 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f333406be30 0x7f333406e2f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3348033000 tx=0x7f3348009030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:11.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.278+0000 7f3350c00640 1 -- 172.21.15.131:0/994688395 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f335415c040 con 0x7f33540b0360 2024-05-26T22:50:11.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.514+0000 7f335a200640 1 -- 172.21.15.131:0/994688395 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f33540694e0 con 0x7f33540b0360 2024-05-26T22:50:11.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.516+0000 7f3350c00640 1 -- 172.21.15.131:0/994688395 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+5231 (secure 0 0 0) 0x7f334c0593b0 con 0x7f33540b0360 2024-05-26T22:50:11.516 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:50:11.516 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":20,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/4292497989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":4292497989},{"type":"v1","addr":"172.21.15.187:6835","nonce":4292497989}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-26T22:49:30.955525+0000","modified":"2024-05-26T22:49:46.310449+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5115,"mds_1":4687,"mds_2":4678},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4678":{"gid":4678,"name":"a","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.131:6836/4032294430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":4032294430},{"type":"v1","addr":"172.21.15.131:6836","nonce":4032294430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4687":{"gid":4687,"name":"b","rank":1,"incarnation":17,"state":"up:active","state_seq":7,"addr":"172.21.15.187:6833/3274403209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3274403209},{"type":"v1","addr":"172.21.15.187:6833","nonce":3274403209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5115":{"gid":5115,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.131:6837/2062704413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6835","nonce":2062704413},{"type":"v1","addr":"172.21.15.131:6837","nonce":2062704413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5115,"qdb_cluster":[4678,4687,5115]},"id":2}]} 2024-05-26T22:50:11.516 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 20 2024-05-26T22:50:11.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.518+0000 7f335a200640 1 -- 172.21.15.131:0/994688395 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f333406be30 msgr2=0x7f333406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:11.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.518+0000 7f335a200640 1 --2- 172.21.15.131:0/994688395 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f333406be30 0x7f333406e2f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3348033000 tx=0x7f3348009030 comp rx=0 tx=0).stop 2024-05-26T22:50:11.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.518+0000 7f335a200640 1 -- 172.21.15.131:0/994688395 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f33540b0360 msgr2=0x7f33541585a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:11.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.518+0000 7f335a200640 1 --2- 172.21.15.131:0/994688395 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f33540b0360 0x7f33541585a0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f334c009820 tx=0x7f334c009ce0 comp rx=0 tx=0).stop 2024-05-26T22:50:11.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.518+0000 7f335a200640 1 -- 172.21.15.131:0/994688395 shutdown_connections 2024-05-26T22:50:11.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.518+0000 7f335a200640 1 --2- 172.21.15.131:0/994688395 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f333406be30 0x7f333406e2f0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:11.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.518+0000 7f335a200640 1 --2- 172.21.15.131:0/994688395 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f33540b0360 0x7f33541585a0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:11.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.518+0000 7f335a200640 1 --2- 172.21.15.131:0/994688395 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f33541576c0 0x7f3354155cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:11.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.518+0000 7f335a200640 1 --2- 172.21.15.131:0/994688395 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f333c0039b0 0x7f335413e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:11.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.518+0000 7f335a200640 1 -- 172.21.15.131:0/994688395 >> 172.21.15.131:0/994688395 conn(0x7f33540cff40 msgr2=0x7f335415b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:11.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.519+0000 7f335a200640 1 -- 172.21.15.131:0/994688395 shutdown_connections 2024-05-26T22:50:11.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.519+0000 7f335a200640 1 -- 172.21.15.131:0/994688395 wait complete. 2024-05-26T22:50:11.538 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-26T22:50:11.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.736+0000 7fd27b600640 1 Processor -- start 2024-05-26T22:50:11.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.736+0000 7fd27b600640 1 -- start start 2024-05-26T22:50:11.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.737+0000 7fd27b600640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd2740ad450 0x7fd27406eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:11.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.737+0000 7fd27b600640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2740b5190 con 0x7fd274071d80 2024-05-26T22:50:11.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.737+0000 7fd27b600640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd27406dee0 con 0x7fd2740ad8b0 2024-05-26T22:50:11.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.737+0000 7fd27b600640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd27406e1c0 con 0x7fd2740ad450 2024-05-26T22:50:11.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.738+0000 7fd278c00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fd274071d80 0x7fd274072160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:36378/0 (socket says 172.21.15.131:36378) 2024-05-26T22:50:11.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.738+0000 7fd278c00640 1 -- 172.21.15.131:0/1336392250 learned_addr learned my addr 172.21.15.131:0/1336392250 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:50:11.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.738+0000 7fd273e00640 1 --2- 172.21.15.131:0/1336392250 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd2740ad450 0x7fd27406eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:11.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.738+0000 7fd273400640 1 -- 172.21.15.131:0/1336392250 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2868596416 0 0) 0x7fd2740b5190 con 0x7fd274071d80 2024-05-26T22:50:11.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.738+0000 7fd273400640 1 -- 172.21.15.131:0/1336392250 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd25c003520 con 0x7fd274071d80 2024-05-26T22:50:11.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.738+0000 7fd273400640 1 -- 172.21.15.131:0/1336392250 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 516149146 0 0) 0x7fd25c003520 con 0x7fd274071d80 2024-05-26T22:50:11.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.738+0000 7fd273400640 1 -- 172.21.15.131:0/1336392250 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd2740ad450 msgr2=0x7fd27406eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:11.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.738+0000 7fd273400640 1 --2- 172.21.15.131:0/1336392250 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd2740ad450 0x7fd27406eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:11.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.738+0000 7fd273400640 1 -- 172.21.15.131:0/1336392250 >> v1:172.21.15.187:6789/0 conn(0x7fd2740ad8b0 legacy=0x7fd27406f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:11.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.738+0000 7fd273400640 1 -- 172.21.15.131:0/1336392250 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd27406e4a0 con 0x7fd274071d80 2024-05-26T22:50:11.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.738+0000 7fd273400640 1 -- 172.21.15.131:0/1336392250 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7fd268001670 con 0x7fd274071d80 2024-05-26T22:50:11.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.739+0000 7fd273400640 1 -- 172.21.15.131:0/1336392250 >> v1:172.21.15.131:6789/0 conn(0x7fd274071d80 legacy=0x7fd274072160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:11.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.739+0000 7fd273400640 1 --2- 172.21.15.131:0/1336392250 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd25c0039e0 0x7fd25c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:11.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.739+0000 7fd273400640 1 --2- 172.21.15.131:0/1336392250 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd2740ad450 0x7fd25c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:11.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.739+0000 7fd273400640 1 --2- 172.21.15.131:0/1336392250 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd25c004d90 0x7fd25c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:11.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.739+0000 7fd273400640 1 -- 172.21.15.131:0/1336392250 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fd27406e1c0 con 0x7fd25c004d90 2024-05-26T22:50:11.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.739+0000 7fd273400640 1 -- 172.21.15.131:0/1336392250 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd25c00bc40 con 0x7fd25c0039e0 2024-05-26T22:50:11.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.739+0000 7fd273400640 1 -- 172.21.15.131:0/1336392250 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd25c00be40 con 0x7fd2740ad450 2024-05-26T22:50:11.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.739+0000 7fd279600640 1 --2- 172.21.15.131:0/1336392250 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd25c004d90 0x7fd25c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:11.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.739+0000 7fd273e00640 1 --2- 172.21.15.131:0/1336392250 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd2740ad450 0x7fd25c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:11.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.739+0000 7fd278c00640 1 --2- 172.21.15.131:0/1336392250 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd25c0039e0 0x7fd25c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:11.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.739+0000 7fd279600640 1 -- 172.21.15.131:0/1336392250 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd2740ad450 msgr2=0x7fd25c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:11.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.739+0000 7fd279600640 1 --2- 172.21.15.131:0/1336392250 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd2740ad450 0x7fd25c004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:11.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.739+0000 7fd279600640 1 -- 172.21.15.131:0/1336392250 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd25c0039e0 msgr2=0x7fd25c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:11.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.739+0000 7fd279600640 1 --2- 172.21.15.131:0/1336392250 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd25c0039e0 0x7fd25c003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:11.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.739+0000 7fd279600640 1 -- 172.21.15.131:0/1336392250 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd25c003520 con 0x7fd25c004d90 2024-05-26T22:50:11.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.739+0000 7fd278c00640 1 --2- 172.21.15.131:0/1336392250 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd25c0039e0 0x7fd25c003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:50:11.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.740+0000 7fd279600640 1 --2- 172.21.15.131:0/1336392250 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd25c004d90 0x7fd25c005210 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fd274071540 tx=0x7fd25c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeca529d8ec53a82 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:11.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.741+0000 7fd273400640 1 -- 172.21.15.131:0/1336392250 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd26c001a70 con 0x7fd25c004d90 2024-05-26T22:50:11.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.741+0000 7fd273400640 1 -- 172.21.15.131:0/1336392250 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd26c02adf0 con 0x7fd25c004d90 2024-05-26T22:50:11.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.741+0000 7fd273400640 1 -- 172.21.15.131:0/1336392250 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd26c003bb0 con 0x7fd25c004d90 2024-05-26T22:50:11.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.741+0000 7fd27b600640 1 -- 172.21.15.131:0/1336392250 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd25c004d90 msgr2=0x7fd25c005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:11.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.741+0000 7fd27b600640 1 --2- 172.21.15.131:0/1336392250 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd25c004d90 0x7fd25c005210 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fd274071540 tx=0x7fd25c005830 comp rx=0 tx=0).stop 2024-05-26T22:50:11.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.741+0000 7fd279600640 1 -- 172.21.15.131:0/1336392250 reap_dead start 2024-05-26T22:50:11.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.741+0000 7fd27b600640 1 -- 172.21.15.131:0/1336392250 shutdown_connections 2024-05-26T22:50:11.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.741+0000 7fd27b600640 1 -- 172.21.15.131:0/1336392250 >> 172.21.15.131:0/1336392250 conn(0x7fd2740b1e90 msgr2=0x7fd2740b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:11.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.741+0000 7fd27b600640 1 -- 172.21.15.131:0/1336392250 shutdown_connections 2024-05-26T22:50:11.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.741+0000 7fd27b600640 1 -- 172.21.15.131:0/1336392250 wait complete. 2024-05-26T22:50:11.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.743+0000 7fd27b600640 1 Processor -- start 2024-05-26T22:50:11.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.743+0000 7fd27b600640 1 -- start start 2024-05-26T22:50:11.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.744+0000 7fd27b600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd25c004d90 0x7fd2740999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:11.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.744+0000 7fd27b600640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd274143770 0x7fd274155390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:11.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.744+0000 7fd27b600640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd2741558d0 0x7fd27415b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:11.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.744+0000 7fd27b600640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fd27415e1a0 con 0x7fd25c004d90 2024-05-26T22:50:11.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.744+0000 7fd27b600640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd27415e020 con 0x7fd2741558d0 2024-05-26T22:50:11.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.744+0000 7fd27b600640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd27406eca0 con 0x7fd274143770 2024-05-26T22:50:11.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.744+0000 7fd278c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd25c004d90 0x7fd2740999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:11.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.744+0000 7fd278c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd25c004d90 0x7fd2740999c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37906/0 (socket says 172.21.15.131:37906) 2024-05-26T22:50:11.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.744+0000 7fd278c00640 1 -- 172.21.15.131:0/2280028237 learned_addr learned my addr 172.21.15.131:0/2280028237 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:11.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.745+0000 7fd278c00640 1 -- 172.21.15.131:0/2280028237 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd274143770 msgr2=0x7fd274155390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:11.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.745+0000 7fd278c00640 1 --2- 172.21.15.131:0/2280028237 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd274143770 0x7fd274155390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:11.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.745+0000 7fd278c00640 1 -- 172.21.15.131:0/2280028237 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd2741558d0 msgr2=0x7fd27415b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:11.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.745+0000 7fd278c00640 1 --2- 172.21.15.131:0/2280028237 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd2741558d0 0x7fd27415b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:11.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.745+0000 7fd278c00640 1 -- 172.21.15.131:0/2280028237 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd274143ef0 con 0x7fd25c004d90 2024-05-26T22:50:11.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.745+0000 7fd278c00640 1 --2- 172.21.15.131:0/2280028237 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd25c004d90 0x7fd2740999c0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fd26800d820 tx=0x7fd26800dce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:11.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.745+0000 7fd270c00640 1 -- 172.21.15.131:0/2280028237 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd268019070 con 0x7fd25c004d90 2024-05-26T22:50:11.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.745+0000 7fd270c00640 1 -- 172.21.15.131:0/2280028237 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2680074e0 con 0x7fd25c004d90 2024-05-26T22:50:11.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.745+0000 7fd270c00640 1 -- 172.21.15.131:0/2280028237 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2680049e0 con 0x7fd25c004d90 2024-05-26T22:50:11.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.745+0000 7fd27b600640 1 -- 172.21.15.131:0/2280028237 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2741441d0 con 0x7fd25c004d90 2024-05-26T22:50:11.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.746+0000 7fd27b600640 1 -- 172.21.15.131:0/2280028237 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd274144790 con 0x7fd25c004d90 2024-05-26T22:50:11.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.747+0000 7fd27b600640 1 -- 172.21.15.131:0/2280028237 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd274155cf0 con 0x7fd25c004d90 2024-05-26T22:50:11.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.748+0000 7fd270c00640 1 -- 172.21.15.131:0/2280028237 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd268004b80 con 0x7fd25c004d90 2024-05-26T22:50:11.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.748+0000 7fd270c00640 1 --2- 172.21.15.131:0/2280028237 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fd25406be60 0x7fd25406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:11.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.749+0000 7fd270c00640 1 -- 172.21.15.131:0/2280028237 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fd26808a7f0 con 0x7fd25c004d90 2024-05-26T22:50:11.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.749+0000 7fd273e00640 1 --2- 172.21.15.131:0/2280028237 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fd25406be60 0x7fd25406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:11.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.750+0000 7fd273e00640 1 --2- 172.21.15.131:0/2280028237 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fd25406be60 0x7fd25406e320 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fd260006a90 tx=0x7fd260006a20 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:11.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.753+0000 7fd270c00640 1 -- 172.21.15.131:0/2280028237 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fd268059100 con 0x7fd25c004d90 2024-05-26T22:50:11.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.950+0000 7fd27b600640 1 -- 172.21.15.131:0/2280028237 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd2740694e0 con 0x7fd25c004d90 2024-05-26T22:50:11.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.953+0000 7fd270c00640 1 -- 172.21.15.131:0/2280028237 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) ==== 74+0+16693 (secure 0 0 0) 0x7fd268058a00 con 0x7fd25c004d90 2024-05-26T22:50:11.953 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:50:11.953 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":33,"fsid":"46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f","created":"2024-05-26T22:47:57.288905+0000","modified":"2024-05-26T22:49:32.532828+0000","last_up_change":"2024-05-26T22:48:09.602207+0000","last_in_change":"2024-05-26T22:48:00.687341+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-26T22:48:10.463339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-05-26T22:49:28.813374+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-05-26T22:49:29.987297+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"814eaf62-a18d-494b-8e27-fd8fb88adea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6801","nonce":2350940051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6803","nonce":2350940051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6807","nonce":2350940051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6805","nonce":2350940051}]},"public_addr":"172.21.15.131:6801/2350940051","cluster_addr":"172.21.15.131:6803/2350940051","heartbeat_back_addr":"172.21.15.131:6807/2350940051","heartbeat_front_addr":"172.21.15.131:6805/2350940051","state":["exists","up"]},{"osd":1,"uuid":"42e66287-f6c6-4e26-af3c-6dfaa3536d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6825","nonce":1525528247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6827","nonce":1525528247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6831","nonce":1525528247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6829","nonce":1525528247}]},"public_addr":"172.21.15.131:6825/1525528247","cluster_addr":"172.21.15.131:6827/1525528247","heartbeat_back_addr":"172.21.15.131:6831/1525528247","heartbeat_front_addr":"172.21.15.131:6829/1525528247","state":["exists","up"]},{"osd":2,"uuid":"0da34673-04a2-4fd0-b57a-0ff2b679b043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6817","nonce":3627842210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6819","nonce":3627842210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6823","nonce":3627842210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6821","nonce":3627842210}]},"public_addr":"172.21.15.131:6817/3627842210","cluster_addr":"172.21.15.131:6819/3627842210","heartbeat_back_addr":"172.21.15.131:6823/3627842210","heartbeat_front_addr":"172.21.15.131:6821/3627842210","state":["exists","up"]},{"osd":3,"uuid":"cf117a27-d6b2-43d3-ae52-b47988bf8eed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6809","nonce":350790591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6811","nonce":350790591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6815","nonce":350790591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6813","nonce":350790591}]},"public_addr":"172.21.15.131:6809/350790591","cluster_addr":"172.21.15.131:6811/350790591","heartbeat_back_addr":"172.21.15.131:6815/350790591","heartbeat_front_addr":"172.21.15.131:6813/350790591","state":["exists","up"]},{"osd":4,"uuid":"a092ccfc-be89-4a84-88d5-0f781fa715eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6816","nonce":244560995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6819","nonce":244560995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6823","nonce":244560995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6821","nonce":244560995}]},"public_addr":"172.21.15.187:6816/244560995","cluster_addr":"172.21.15.187:6819/244560995","heartbeat_back_addr":"172.21.15.187:6823/244560995","heartbeat_front_addr":"172.21.15.187:6821/244560995","state":["exists","up"]},{"osd":5,"uuid":"7513ba69-dd00-4a35-b190-9e8cc3b2bdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6801","nonce":324619398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6803","nonce":324619398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6812","nonce":324619398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6807","nonce":324619398}]},"public_addr":"172.21.15.187:6801/324619398","cluster_addr":"172.21.15.187:6803/324619398","heartbeat_back_addr":"172.21.15.187:6812/324619398","heartbeat_front_addr":"172.21.15.187:6807/324619398","state":["exists","up"]},{"osd":6,"uuid":"50081810-c093-4d2f-a4d7-b7c695c5400d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6806","nonce":1391347786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6809","nonce":1391347786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6818","nonce":1391347786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6813","nonce":1391347786}]},"public_addr":"172.21.15.187:6806/1391347786","cluster_addr":"172.21.15.187:6809/1391347786","heartbeat_back_addr":"172.21.15.187:6818/1391347786","heartbeat_front_addr":"172.21.15.187:6813/1391347786","state":["exists","up"]},{"osd":7,"uuid":"775dae05-54f5-4ad4-bbce-92863c6ed2ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6825","nonce":1220065622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6827","nonce":1220065622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6831","nonce":1220065622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6829","nonce":1220065622}]},"public_addr":"172.21.15.187:6825/1220065622","cluster_addr":"172.21.15.187:6827/1220065622","heartbeat_back_addr":"172.21.15.187:6831/1220065622","heartbeat_front_addr":"172.21.15.187:6829/1220065622","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:05.983520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.299516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.067323+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.307748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.018742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.258775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.010912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.295355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-26T22:50:11.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.956+0000 7fd27b600640 1 -- 172.21.15.131:0/2280028237 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fd25406be60 msgr2=0x7fd25406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:11.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.956+0000 7fd27b600640 1 --2- 172.21.15.131:0/2280028237 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fd25406be60 0x7fd25406e320 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fd260006a90 tx=0x7fd260006a20 comp rx=0 tx=0).stop 2024-05-26T22:50:11.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.956+0000 7fd27b600640 1 -- 172.21.15.131:0/2280028237 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd25c004d90 msgr2=0x7fd2740999c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:11.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.956+0000 7fd27b600640 1 --2- 172.21.15.131:0/2280028237 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd25c004d90 0x7fd2740999c0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fd26800d820 tx=0x7fd26800dce0 comp rx=0 tx=0).stop 2024-05-26T22:50:11.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.956+0000 7fd27b600640 1 -- 172.21.15.131:0/2280028237 shutdown_connections 2024-05-26T22:50:11.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.956+0000 7fd27b600640 1 --2- 172.21.15.131:0/2280028237 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd274143770 0x7fd274155390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:11.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.956+0000 7fd27b600640 1 --2- 172.21.15.131:0/2280028237 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fd25406be60 0x7fd25406e320 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:11.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.956+0000 7fd27b600640 1 --2- 172.21.15.131:0/2280028237 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd2741558d0 0x7fd27415b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:11.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.956+0000 7fd27b600640 1 --2- 172.21.15.131:0/2280028237 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd25c004d90 0x7fd2740999c0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:11.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.956+0000 7fd27b600640 1 -- 172.21.15.131:0/2280028237 >> 172.21.15.131:0/2280028237 conn(0x7fd2740b1e90 msgr2=0x7fd2741598e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:11.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.956+0000 7fd27b600640 1 -- 172.21.15.131:0/2280028237 shutdown_connections 2024-05-26T22:50:11.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:11.956+0000 7fd27b600640 1 -- 172.21.15.131:0/2280028237 wait complete. 2024-05-26T22:50:11.975 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-26T22:50:12.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.177+0000 7f7018e00640 1 Processor -- start 2024-05-26T22:50:12.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.177+0000 7f7018e00640 1 -- start start 2024-05-26T22:50:12.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.178+0000 7f7018e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f701406e1c0 0x7f7014072cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:12.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.179+0000 7f7018e00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f701415c1c0 con 0x7f7014155770 2024-05-26T22:50:12.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.179+0000 7f7018e00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f701415c4a0 con 0x7f701406dd30 2024-05-26T22:50:12.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.179+0000 7f7018e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f7014071da0 con 0x7f701406e1c0 2024-05-26T22:50:12.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.179+0000 7f7011e00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f7014155770 0x7f701415b700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:36394/0 (socket says 172.21.15.131:36394) 2024-05-26T22:50:12.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.179+0000 7f7011e00640 1 -- 172.21.15.131:0/4288764559 learned_addr learned my addr 172.21.15.131:0/4288764559 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:50:12.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.179+0000 7f7011400640 1 -- 172.21.15.131:0/4288764559 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3314553710 0 0) 0x7f701415c1c0 con 0x7f7014155770 2024-05-26T22:50:12.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.179+0000 7f7011400640 1 -- 172.21.15.131:0/4288764559 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6ff8003520 con 0x7f7014155770 2024-05-26T22:50:12.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.180+0000 7f7011400640 1 -- 172.21.15.131:0/4288764559 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3436013430 0 0) 0x7f6ff8003520 con 0x7f7014155770 2024-05-26T22:50:12.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.180+0000 7f7011400640 1 -- 172.21.15.131:0/4288764559 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f701406e1c0 msgr2=0x7f7014072cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:12.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.180+0000 7f7011400640 1 --2- 172.21.15.131:0/4288764559 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f701406e1c0 0x7f7014072cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:12.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.180+0000 7f7011400640 1 -- 172.21.15.131:0/4288764559 >> v1:172.21.15.187:6789/0 conn(0x7f701406dd30 legacy=0x7f70140725a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:12.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.180+0000 7f7011400640 1 -- 172.21.15.131:0/4288764559 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7014072080 con 0x7f7014155770 2024-05-26T22:50:12.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.180+0000 7f7011400640 1 -- 172.21.15.131:0/4288764559 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7f7004001670 con 0x7f7014155770 2024-05-26T22:50:12.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.180+0000 7f7011400640 1 -- 172.21.15.131:0/4288764559 >> v1:172.21.15.131:6789/0 conn(0x7f7014155770 legacy=0x7f701415b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:12.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.180+0000 7f7011400640 1 --2- 172.21.15.131:0/4288764559 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f701406e1c0 0x7f6ff80039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:12.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.180+0000 7f7011400640 1 --2- 172.21.15.131:0/4288764559 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6ff80040d0 0x7f6ff8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:12.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.180+0000 7f7011400640 1 --2- 172.21.15.131:0/4288764559 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6ff8004db0 0x7f6ff8005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:12.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.180+0000 7f7011400640 1 -- 172.21.15.131:0/4288764559 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f7014071da0 con 0x7f6ff8004db0 2024-05-26T22:50:12.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.180+0000 7f7011400640 1 -- 172.21.15.131:0/4288764559 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6ff800bcb0 con 0x7f6ff80040d0 2024-05-26T22:50:12.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.180+0000 7f7011400640 1 -- 172.21.15.131:0/4288764559 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f6ff800e0c0 con 0x7f701406e1c0 2024-05-26T22:50:12.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.180+0000 7f7012800640 1 --2- 172.21.15.131:0/4288764559 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6ff8004db0 0x7f6ff8005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:12.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.180+0000 7f7011e00640 1 --2- 172.21.15.131:0/4288764559 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f701406e1c0 0x7f6ff80039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:12.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.181+0000 7f7012800640 1 -- 172.21.15.131:0/4288764559 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f701406e1c0 msgr2=0x7f6ff80039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:12.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.181+0000 7f7012800640 1 --2- 172.21.15.131:0/4288764559 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f701406e1c0 0x7f6ff80039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:12.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.181+0000 7f700be00640 1 --2- 172.21.15.131:0/4288764559 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6ff80040d0 0x7f6ff8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:12.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.181+0000 7f7012800640 1 -- 172.21.15.131:0/4288764559 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6ff80040d0 msgr2=0x7f6ff8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:12.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.181+0000 7f7012800640 1 --2- 172.21.15.131:0/4288764559 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6ff80040d0 0x7f6ff8004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:12.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.181+0000 7f7012800640 1 -- 172.21.15.131:0/4288764559 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ff8003520 con 0x7f6ff8004db0 2024-05-26T22:50:12.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.182+0000 7f7012800640 1 --2- 172.21.15.131:0/4288764559 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6ff8004db0 0x7f6ff8005230 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f700c009e40 tx=0x7f6ff8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d48f4d2045e7576b server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:12.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.182+0000 7f7011400640 1 -- 172.21.15.131:0/4288764559 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f700c009a10 con 0x7f6ff8004db0 2024-05-26T22:50:12.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.182+0000 7f7011400640 1 -- 172.21.15.131:0/4288764559 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f700c036070 con 0x7f6ff8004db0 2024-05-26T22:50:12.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.182+0000 7f7011400640 1 -- 172.21.15.131:0/4288764559 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f700c0318a0 con 0x7f6ff8004db0 2024-05-26T22:50:12.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.182+0000 7f7018e00640 1 -- 172.21.15.131:0/4288764559 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6ff8004db0 msgr2=0x7f6ff8005230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:12.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.182+0000 7f7018e00640 1 --2- 172.21.15.131:0/4288764559 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6ff8004db0 0x7f6ff8005230 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f700c009e40 tx=0x7f6ff8005830 comp rx=0 tx=0).stop 2024-05-26T22:50:12.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.182+0000 7f7012800640 1 -- 172.21.15.131:0/4288764559 reap_dead start 2024-05-26T22:50:12.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.183+0000 7f7018e00640 1 -- 172.21.15.131:0/4288764559 shutdown_connections 2024-05-26T22:50:12.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.183+0000 7f7018e00640 1 -- 172.21.15.131:0/4288764559 >> 172.21.15.131:0/4288764559 conn(0x7f701406bc90 msgr2=0x7f70140b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:12.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.183+0000 7f7018e00640 1 -- 172.21.15.131:0/4288764559 shutdown_connections 2024-05-26T22:50:12.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.183+0000 7f7018e00640 1 -- 172.21.15.131:0/4288764559 wait complete. 2024-05-26T22:50:12.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.184+0000 7f7018e00640 1 Processor -- start 2024-05-26T22:50:12.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.185+0000 7f7018e00640 1 -- start start 2024-05-26T22:50:12.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.185+0000 7f7018e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6ff8004db0 0x7f7014152d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:12.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.185+0000 7f7018e00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f701414bb90 0x7f70141549c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:12.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.185+0000 7f7018e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f70140b0b90 0x7f70140b1010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:12.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.186+0000 7f7018e00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f701415c8c0 con 0x7f6ff8004db0 2024-05-26T22:50:12.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.186+0000 7f7018e00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f701406e5c0 con 0x7f701414bb90 2024-05-26T22:50:12.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.186+0000 7f7018e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f701415be10 con 0x7f70140b0b90 2024-05-26T22:50:12.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.186+0000 7f7011e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6ff8004db0 0x7f7014152d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:12.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.186+0000 7f7011e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6ff8004db0 0x7f7014152d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37928/0 (socket says 172.21.15.131:37928) 2024-05-26T22:50:12.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.186+0000 7f7011e00640 1 -- 172.21.15.131:0/332899974 learned_addr learned my addr 172.21.15.131:0/332899974 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:12.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.186+0000 7f700be00640 1 --2- 172.21.15.131:0/332899974 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f701414bb90 0x7f70141549c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:12.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.186+0000 7f7012800640 1 --2- 172.21.15.131:0/332899974 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f70140b0b90 0x7f70140b1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:12.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.186+0000 7f7011e00640 1 -- 172.21.15.131:0/332899974 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f70140b0b90 msgr2=0x7f70140b1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:12.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.186+0000 7f7011e00640 1 --2- 172.21.15.131:0/332899974 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f70140b0b90 0x7f70140b1010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:12.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.186+0000 7f7011e00640 1 -- 172.21.15.131:0/332899974 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f701414bb90 msgr2=0x7f70141549c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:12.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.186+0000 7f7011e00640 1 --2- 172.21.15.131:0/332899974 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f701414bb90 0x7f70141549c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:12.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.186+0000 7f7011e00640 1 -- 172.21.15.131:0/332899974 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f701414c010 con 0x7f6ff8004db0 2024-05-26T22:50:12.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.186+0000 7f700be00640 1 --2- 172.21.15.131:0/332899974 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f701414bb90 0x7f70141549c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:50:12.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.187+0000 7f7011e00640 1 --2- 172.21.15.131:0/332899974 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6ff8004db0 0x7f7014152d60 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f700400e870 tx=0x7f700400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:12.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.187+0000 7f700a000640 1 -- 172.21.15.131:0/332899974 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f700400c870 con 0x7f6ff8004db0 2024-05-26T22:50:12.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.187+0000 7f700a000640 1 -- 172.21.15.131:0/332899974 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7004018070 con 0x7f6ff8004db0 2024-05-26T22:50:12.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.187+0000 7f7018e00640 1 -- 172.21.15.131:0/332899974 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70141533f0 con 0x7f6ff8004db0 2024-05-26T22:50:12.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.187+0000 7f700a000640 1 -- 172.21.15.131:0/332899974 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f700400cc70 con 0x7f6ff8004db0 2024-05-26T22:50:12.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.187+0000 7f7018e00640 1 -- 172.21.15.131:0/332899974 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70141539b0 con 0x7f6ff8004db0 2024-05-26T22:50:12.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.188+0000 7f7018e00640 1 -- 172.21.15.131:0/332899974 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7014074e90 con 0x7f6ff8004db0 2024-05-26T22:50:12.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.189+0000 7f700a000640 1 -- 172.21.15.131:0/332899974 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f700401d460 con 0x7f6ff8004db0 2024-05-26T22:50:12.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.190+0000 7f700a000640 1 --2- 172.21.15.131:0/332899974 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f6ff006be30 0x7f6ff006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:12.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.190+0000 7f700a000640 1 -- 172.21.15.131:0/332899974 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f700408adc0 con 0x7f6ff8004db0 2024-05-26T22:50:12.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.190+0000 7f700be00640 1 --2- 172.21.15.131:0/332899974 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f6ff006be30 0x7f6ff006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:12.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.191+0000 7f700be00640 1 --2- 172.21.15.131:0/332899974 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f6ff006be30 0x7f6ff006e2f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f7014154640 tx=0x7f6ffc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:12.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.194+0000 7f700a000640 1 -- 172.21.15.131:0/332899974 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f70040596d0 con 0x7f6ff8004db0 2024-05-26T22:50:12.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.435+0000 7f7018e00640 1 -- 172.21.15.131:0/332899974 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f70140695c0 con 0x7f6ff8004db0 2024-05-26T22:50:12.436 INFO:tasks.ceph.mon.a.smithi131.stderr:2024-05-26T22:50:12.437+0000 7fcf0d200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-26T22:50:12.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.440+0000 7f700a000640 1 -- 172.21.15.131:0/332899974 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v21) ==== 161+0+0 (secure 0 0 0) 0x7f7004058fd0 con 0x7f6ff8004db0 2024-05-26T22:50:12.440 INFO:teuthology.orchestra.run.smithi131.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-26T22:50:12.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.444+0000 7f7018e00640 1 -- 172.21.15.131:0/332899974 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f6ff006be30 msgr2=0x7f6ff006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:12.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.444+0000 7f7018e00640 1 --2- 172.21.15.131:0/332899974 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f6ff006be30 0x7f6ff006e2f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f7014154640 tx=0x7f6ffc009040 comp rx=0 tx=0).stop 2024-05-26T22:50:12.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.444+0000 7f7018e00640 1 -- 172.21.15.131:0/332899974 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6ff8004db0 msgr2=0x7f7014152d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:12.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.444+0000 7f7018e00640 1 --2- 172.21.15.131:0/332899974 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6ff8004db0 0x7f7014152d60 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f700400e870 tx=0x7f700400ed30 comp rx=0 tx=0).stop 2024-05-26T22:50:12.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.444+0000 7f7018e00640 1 -- 172.21.15.131:0/332899974 shutdown_connections 2024-05-26T22:50:12.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.444+0000 7f7018e00640 1 --2- 172.21.15.131:0/332899974 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f70140b0b90 0x7f70140b1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:12.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.444+0000 7f7018e00640 1 --2- 172.21.15.131:0/332899974 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f6ff006be30 0x7f6ff006e2f0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:12.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.444+0000 7f7018e00640 1 --2- 172.21.15.131:0/332899974 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f701414bb90 0x7f70141549c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:12.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.444+0000 7f7018e00640 1 --2- 172.21.15.131:0/332899974 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6ff8004db0 0x7f7014152d60 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:12.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.444+0000 7f7018e00640 1 -- 172.21.15.131:0/332899974 >> 172.21.15.131:0/332899974 conn(0x7f701406bc90 msgr2=0x7f70140cfd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:12.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.444+0000 7f7018e00640 1 -- 172.21.15.131:0/332899974 shutdown_connections 2024-05-26T22:50:12.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.444+0000 7f7018e00640 1 -- 172.21.15.131:0/332899974 wait complete. 2024-05-26T22:50:12.466 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-26T22:50:12.502 INFO:tasks.ceph.mds.a.smithi131.stdout:starting mds.a at 2024-05-26T22:50:12.516 INFO:tasks.ceph.mds.b.smithi187.stdout:starting mds.b at 2024-05-26T22:50:12.548 INFO:tasks.ceph.mds.c.smithi131.stdout:starting mds.c at 2024-05-26T22:50:12.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.647+0000 7f208c800640 1 Processor -- start 2024-05-26T22:50:12.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.647+0000 7f208c800640 1 -- start start 2024-05-26T22:50:12.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.648+0000 7f208c800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2084165e30 0x7f2084072540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:12.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.648+0000 7f208c800640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f20840d0560 0x7f20840cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:12.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.648+0000 7f208c800640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20840733c0 con 0x7f20840af230 2024-05-26T22:50:12.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.648+0000 7f208c800640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f2084072e90 con 0x7f20840d0560 2024-05-26T22:50:12.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.648+0000 7f208c800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f20840cf540 con 0x7f2084165e30 2024-05-26T22:50:12.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.649+0000 7f2089400640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f20840af230 0x7f20840cfe50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:36418/0 (socket says 172.21.15.131:36418) 2024-05-26T22:50:12.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.649+0000 7f2089400640 1 -- 172.21.15.131:0/4013074993 learned_addr learned my addr 172.21.15.131:0/4013074993 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:50:12.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.649+0000 7f2088a00640 1 -- 172.21.15.131:0/4013074993 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1330077511 0 0) 0x7f20840733c0 con 0x7f20840af230 2024-05-26T22:50:12.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.649+0000 7f2088a00640 1 -- 172.21.15.131:0/4013074993 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f206c003520 con 0x7f20840af230 2024-05-26T22:50:12.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.649+0000 7f2089e00640 1 --2- 172.21.15.131:0/4013074993 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2084165e30 0x7f2084072540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:12.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.649+0000 7f208a800640 1 --2- 172.21.15.131:0/4013074993 >> v2:172.21.15.187:3300/0 conn(0x7f20840d0560 0x7f20840cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:12.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.650+0000 7f2088a00640 1 -- 172.21.15.131:0/4013074993 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 341491880 0 0) 0x7f206c003520 con 0x7f20840af230 2024-05-26T22:50:12.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.650+0000 7f2088a00640 1 -- 172.21.15.131:0/4013074993 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2084165e30 msgr2=0x7f2084072540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:12.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.650+0000 7f2088a00640 1 --2- 172.21.15.131:0/4013074993 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2084165e30 0x7f2084072540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:12.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.650+0000 7f2088a00640 1 -- 172.21.15.131:0/4013074993 >> v2:172.21.15.187:3300/0 conn(0x7f20840d0560 msgr2=0x7f20840cdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:12.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.650+0000 7f2088a00640 1 --2- 172.21.15.131:0/4013074993 >> v2:172.21.15.187:3300/0 conn(0x7f20840d0560 0x7f20840cdcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:12.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.650+0000 7f2088a00640 1 -- 172.21.15.131:0/4013074993 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20840730e0 con 0x7f20840af230 2024-05-26T22:50:12.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.650+0000 7f2088a00640 1 -- 172.21.15.131:0/4013074993 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7f2074001670 con 0x7f20840af230 2024-05-26T22:50:12.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.650+0000 7f2088a00640 1 -- 172.21.15.131:0/4013074993 >> v1:172.21.15.131:6789/0 conn(0x7f20840af230 legacy=0x7f20840cfe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:12.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.650+0000 7f2088a00640 1 --2- 172.21.15.131:0/4013074993 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f206c0039b0 0x7f206c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:12.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.650+0000 7f2088a00640 1 --2- 172.21.15.131:0/4013074993 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f206c004800 0x7f206c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:12.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.650+0000 7f2088a00640 1 --2- 172.21.15.131:0/4013074993 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2084165e30 0x7f206c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:12.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.650+0000 7f2088a00640 1 -- 172.21.15.131:0/4013074993 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f2084072e90 con 0x7f206c0039b0 2024-05-26T22:50:12.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.650+0000 7f2088a00640 1 -- 172.21.15.131:0/4013074993 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f20840cf540 con 0x7f206c004800 2024-05-26T22:50:12.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.650+0000 7f2088a00640 1 -- 172.21.15.131:0/4013074993 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f206c00ba70 con 0x7f2084165e30 2024-05-26T22:50:12.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.650+0000 7f2089e00640 1 --2- 172.21.15.131:0/4013074993 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f206c0039b0 0x7f206c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:12.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.651+0000 7f2089400640 1 --2- 172.21.15.131:0/4013074993 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f206c004800 0x7f206c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:12.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.651+0000 7f208a800640 1 --2- 172.21.15.131:0/4013074993 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2084165e30 0x7f206c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:12.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.651+0000 7f2089e00640 1 -- 172.21.15.131:0/4013074993 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2084165e30 msgr2=0x7f206c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:12.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.651+0000 7f2089e00640 1 --2- 172.21.15.131:0/4013074993 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2084165e30 0x7f206c0051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:12.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.651+0000 7f2089e00640 1 -- 172.21.15.131:0/4013074993 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f206c004800 msgr2=0x7f206c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:12.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.651+0000 7f2089e00640 1 --2- 172.21.15.131:0/4013074993 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f206c004800 0x7f206c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:12.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.651+0000 7f2089e00640 1 -- 172.21.15.131:0/4013074993 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f206c003520 con 0x7f206c0039b0 2024-05-26T22:50:12.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.651+0000 7f2089400640 1 --2- 172.21.15.131:0/4013074993 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f206c004800 0x7f206c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:50:12.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.652+0000 7f2089e00640 1 --2- 172.21.15.131:0/4013074993 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f206c0039b0 0x7f206c003dd0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f207800ba00 tx=0x7f206c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=597b3a6aec850650 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:12.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.652+0000 7f2088a00640 1 -- 172.21.15.131:0/4013074993 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2078030070 con 0x7f206c0039b0 2024-05-26T22:50:12.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.652+0000 7f2088a00640 1 -- 172.21.15.131:0/4013074993 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f207802acc0 con 0x7f206c0039b0 2024-05-26T22:50:12.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.652+0000 7f2088a00640 1 -- 172.21.15.131:0/4013074993 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f207800b600 con 0x7f206c0039b0 2024-05-26T22:50:12.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.652+0000 7f208c800640 1 -- 172.21.15.131:0/4013074993 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f206c0039b0 msgr2=0x7f206c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:12.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.652+0000 7f208c800640 1 --2- 172.21.15.131:0/4013074993 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f206c0039b0 0x7f206c003dd0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f207800ba00 tx=0x7f206c005830 comp rx=0 tx=0).stop 2024-05-26T22:50:12.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.653+0000 7f208a800640 1 -- 172.21.15.131:0/4013074993 reap_dead start 2024-05-26T22:50:12.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.653+0000 7f208c800640 1 -- 172.21.15.131:0/4013074993 shutdown_connections 2024-05-26T22:50:12.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.653+0000 7f208c800640 1 -- 172.21.15.131:0/4013074993 >> 172.21.15.131:0/4013074993 conn(0x7f208406bc90 msgr2=0x7f20840b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:12.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.653+0000 7f208c800640 1 -- 172.21.15.131:0/4013074993 shutdown_connections 2024-05-26T22:50:12.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.653+0000 7f208c800640 1 -- 172.21.15.131:0/4013074993 wait complete. 2024-05-26T22:50:12.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.655+0000 7f208c800640 1 Processor -- start 2024-05-26T22:50:12.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.655+0000 7f208c800640 1 -- start start 2024-05-26T22:50:12.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.656+0000 7f208c800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f206c0039b0 0x7f208415e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:12.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.656+0000 7f208c800640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2084162830 0x7f2084164540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:12.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.656+0000 7f208c800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2084164a80 0x7f208415f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:12.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.656+0000 7f208c800640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f2084072540 con 0x7f206c0039b0 2024-05-26T22:50:12.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.656+0000 7f208c800640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f20840cfac0 con 0x7f2084162830 2024-05-26T22:50:12.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.656+0000 7f208c800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f20840726c0 con 0x7f2084164a80 2024-05-26T22:50:12.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.656+0000 7f2089e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f206c0039b0 0x7f208415e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:12.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.656+0000 7f2089e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f206c0039b0 0x7f208415e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37988/0 (socket says 172.21.15.131:37988) 2024-05-26T22:50:12.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.656+0000 7f2089e00640 1 -- 172.21.15.131:0/4271152901 learned_addr learned my addr 172.21.15.131:0/4271152901 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:12.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.656+0000 7f2089400640 1 --2- 172.21.15.131:0/4271152901 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2084162830 0x7f2084164540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:12.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.656+0000 7f2089e00640 1 -- 172.21.15.131:0/4271152901 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2084164a80 msgr2=0x7f208415f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:12.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.656+0000 7f2089e00640 1 --2- 172.21.15.131:0/4271152901 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2084164a80 0x7f208415f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:12.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.656+0000 7f2089e00640 1 -- 172.21.15.131:0/4271152901 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2084162830 msgr2=0x7f2084164540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:12.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.656+0000 7f2089e00640 1 --2- 172.21.15.131:0/4271152901 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2084162830 0x7f2084164540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:12.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.656+0000 7f2089e00640 1 -- 172.21.15.131:0/4271152901 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f208409d290 con 0x7f206c0039b0 2024-05-26T22:50:12.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.657+0000 7f2089e00640 1 --2- 172.21.15.131:0/4271152901 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f206c0039b0 0x7f208415e870 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f207800ba30 tx=0x7f2078004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:12.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.657+0000 7f207e000640 1 -- 172.21.15.131:0/4271152901 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2078043020 con 0x7f206c0039b0 2024-05-26T22:50:12.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.657+0000 7f207e000640 1 -- 172.21.15.131:0/4271152901 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f207803f680 con 0x7f206c0039b0 2024-05-26T22:50:12.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.657+0000 7f208c800640 1 -- 172.21.15.131:0/4271152901 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f208409d570 con 0x7f206c0039b0 2024-05-26T22:50:12.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.657+0000 7f207e000640 1 -- 172.21.15.131:0/4271152901 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f207800b600 con 0x7f206c0039b0 2024-05-26T22:50:12.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.657+0000 7f208c800640 1 -- 172.21.15.131:0/4271152901 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f208409ce20 con 0x7f206c0039b0 2024-05-26T22:50:12.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.658+0000 7f208c800640 1 -- 172.21.15.131:0/4271152901 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20840695c0 con 0x7f206c0039b0 2024-05-26T22:50:12.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.659+0000 7f207e000640 1 -- 172.21.15.131:0/4271152901 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2078046020 con 0x7f206c0039b0 2024-05-26T22:50:12.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.660+0000 7f207e000640 1 --2- 172.21.15.131:0/4271152901 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f206806be30 0x7f206806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:12.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.660+0000 7f207e000640 1 -- 172.21.15.131:0/4271152901 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5967+0+0 (secure 0 0 0) 0x7f20780ab8b0 con 0x7f206c0039b0 2024-05-26T22:50:12.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.660+0000 7f2089400640 1 --2- 172.21.15.131:0/4271152901 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f206806be30 0x7f206806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:12.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.661+0000 7f2089400640 1 --2- 172.21.15.131:0/4271152901 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f206806be30 0x7f206806e2f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f2084071e40 tx=0x7f2074009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:12.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.665+0000 7f207e000640 1 -- 172.21.15.131:0/4271152901 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f207804d070 con 0x7f206c0039b0 2024-05-26T22:50:12.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:12.899+0000 7f208c800640 1 -- 172.21.15.131:0/4271152901 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f2084162c10 con 0x7f206c0039b0 2024-05-26T22:50:13.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:13.444+0000 7f207e000640 1 -- 172.21.15.131:0/4271152901 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v22) ==== 104+0+0 (secure 0 0 0) 0x7f207807a0c0 con 0x7f206c0039b0 2024-05-26T22:50:13.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:13.448+0000 7f208c800640 1 -- 172.21.15.131:0/4271152901 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f206806be30 msgr2=0x7f206806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:13.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:13.448+0000 7f208c800640 1 --2- 172.21.15.131:0/4271152901 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f206806be30 0x7f206806e2f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f2084071e40 tx=0x7f2074009040 comp rx=0 tx=0).stop 2024-05-26T22:50:13.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:13.448+0000 7f208c800640 1 -- 172.21.15.131:0/4271152901 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f206c0039b0 msgr2=0x7f208415e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:13.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:13.448+0000 7f208c800640 1 --2- 172.21.15.131:0/4271152901 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f206c0039b0 0x7f208415e870 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f207800ba30 tx=0x7f2078004730 comp rx=0 tx=0).stop 2024-05-26T22:50:13.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:13.448+0000 7f208c800640 1 -- 172.21.15.131:0/4271152901 shutdown_connections 2024-05-26T22:50:13.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:13.448+0000 7f208c800640 1 --2- 172.21.15.131:0/4271152901 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2084164a80 0x7f208415f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:13.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:13.448+0000 7f208c800640 1 --2- 172.21.15.131:0/4271152901 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f206806be30 0x7f206806e2f0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:13.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:13.448+0000 7f208c800640 1 --2- 172.21.15.131:0/4271152901 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2084162830 0x7f2084164540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:13.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:13.448+0000 7f208c800640 1 --2- 172.21.15.131:0/4271152901 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f206c0039b0 0x7f208415e870 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:13.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:13.448+0000 7f208c800640 1 -- 172.21.15.131:0/4271152901 >> 172.21.15.131:0/4271152901 conn(0x7f208406bc90 msgr2=0x7f2084071340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:13.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:13.448+0000 7f208c800640 1 -- 172.21.15.131:0/4271152901 shutdown_connections 2024-05-26T22:50:13.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:13.448+0000 7f208c800640 1 -- 172.21.15.131:0/4271152901 wait complete. 2024-05-26T22:50:13.470 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-26T22:50:13.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:13.722+0000 7fa559800640 1 Processor -- start 2024-05-26T22:50:13.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:13.722+0000 7fa559800640 1 -- start start 2024-05-26T22:50:13.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:13.723+0000 7fa559800640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fa5540cf8f0 0x7fa554072250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:13.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:13.723+0000 7fa559800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa55406ef70 0x7fa5540d07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:13.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:13.724+0000 7fa559800640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5540ccb70 con 0x7fa5540d0360 2024-05-26T22:50:13.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:13.724+0000 7fa559800640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fa5540d0d00 con 0x7fa5540cf8f0 2024-05-26T22:50:13.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:13.724+0000 7fa559800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa5540cce50 con 0x7fa55406ef70 2024-05-26T22:50:13.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:13.724+0000 7fa552a00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fa5540cf8f0 0x7fa554072250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:13.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:13.724+0000 7fa552a00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fa5540cf8f0 0x7fa554072250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.131:60762/0 (socket says 172.21.15.131:60762) 2024-05-26T22:50:13.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:13.724+0000 7fa552a00640 1 -- 172.21.15.131:0/2121011555 learned_addr learned my addr 172.21.15.131:0/2121011555 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:13.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:13.724+0000 7fa553400640 1 --2- 172.21.15.131:0/2121011555 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa55406ef70 0x7fa5540d07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:13.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:13.725+0000 7fa552a00640 1 -- 172.21.15.131:0/2121011555 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa55406ef70 msgr2=0x7fa5540d07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:13.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:13.725+0000 7fa552a00640 1 --2- 172.21.15.131:0/2121011555 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa55406ef70 0x7fa5540d07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:13.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:13.725+0000 7fa552a00640 1 -- 172.21.15.131:0/2121011555 >> v1:172.21.15.187:6789/0 conn(0x7fa5540d0360 legacy=0x7fa55406e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:13.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:13.725+0000 7fa552a00640 1 -- 172.21.15.131:0/2121011555 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5540cd130 con 0x7fa5540cf8f0 2024-05-26T22:50:13.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:13.727+0000 7fa552a00640 1 --2- 172.21.15.131:0/2121011555 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa5540cf8f0 0x7fa554072250 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa544009870 tx=0x7fa54402a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9122bf2e3be26f5c server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:13.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:13.727+0000 7fa551600640 1 -- 172.21.15.131:0/2121011555 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa544037070 con 0x7fa5540cf8f0 2024-05-26T22:50:13.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:13.727+0000 7fa551600640 1 -- 172.21.15.131:0/2121011555 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa544032440 con 0x7fa5540cf8f0 2024-05-26T22:50:13.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:13.727+0000 7fa551600640 1 -- 172.21.15.131:0/2121011555 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa544032810 con 0x7fa5540cf8f0 2024-05-26T22:50:13.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:13.728+0000 7fa559800640 1 -- 172.21.15.131:0/2121011555 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa5540cf8f0 msgr2=0x7fa554072250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:13.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:13.728+0000 7fa559800640 1 --2- 172.21.15.131:0/2121011555 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa5540cf8f0 0x7fa554072250 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa544009870 tx=0x7fa54402a270 comp rx=0 tx=0).stop 2024-05-26T22:50:13.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:13.728+0000 7fa559800640 1 -- 172.21.15.131:0/2121011555 shutdown_connections 2024-05-26T22:50:13.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:13.728+0000 7fa559800640 1 --2- 172.21.15.131:0/2121011555 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa55406ef70 0x7fa5540d07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:13.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:13.728+0000 7fa559800640 1 --2- 172.21.15.131:0/2121011555 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa5540cf8f0 0x7fa554072250 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:13.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:13.728+0000 7fa559800640 1 -- 172.21.15.131:0/2121011555 >> 172.21.15.131:0/2121011555 conn(0x7fa5540b2110 msgr2=0x7fa5540b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:13.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:13.728+0000 7fa559800640 1 -- 172.21.15.131:0/2121011555 shutdown_connections 2024-05-26T22:50:13.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:13.728+0000 7fa559800640 1 -- 172.21.15.131:0/2121011555 wait complete. 2024-05-26T22:50:13.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:13.730+0000 7fa559800640 1 Processor -- start 2024-05-26T22:50:13.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:13.731+0000 7fa559800640 1 -- start start 2024-05-26T22:50:13.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:13.732+0000 7fa559800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa55406ef70 0x7fa55414fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:13.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:13.732+0000 7fa559800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa5540cf8f0 0x7fa5540ad9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:13.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:13.732+0000 7fa559800640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa5540d0360 0x7fa5540c4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:13.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:13.732+0000 7fa559800640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa55415ffd0 con 0x7fa5540cf8f0 2024-05-26T22:50:13.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:13.732+0000 7fa559800640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa5540cfcd0 con 0x7fa5540d0360 2024-05-26T22:50:13.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:13.732+0000 7fa559800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa554160150 con 0x7fa55406ef70 2024-05-26T22:50:13.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:13.732+0000 7fa552000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa5540cf8f0 0x7fa5540ad9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:13.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:13.732+0000 7fa552000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa5540cf8f0 0x7fa5540ad9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37992/0 (socket says 172.21.15.131:37992) 2024-05-26T22:50:13.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:13.732+0000 7fa552000640 1 -- 172.21.15.131:0/3459821089 learned_addr learned my addr 172.21.15.131:0/3459821089 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:13.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:13.732+0000 7fa552000640 1 -- 172.21.15.131:0/3459821089 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa55406ef70 msgr2=0x7fa55414fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:13.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:13.732+0000 7fa552000640 1 --2- 172.21.15.131:0/3459821089 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa55406ef70 0x7fa55414fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:13.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:13.732+0000 7fa552000640 1 -- 172.21.15.131:0/3459821089 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa5540d0360 msgr2=0x7fa5540c4f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:13.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:13.732+0000 7fa553400640 1 --2- 172.21.15.131:0/3459821089 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa5540d0360 0x7fa5540c4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:13.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:13.732+0000 7fa552000640 1 --2- 172.21.15.131:0/3459821089 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa5540d0360 0x7fa5540c4f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:13.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:13.733+0000 7fa552000640 1 -- 172.21.15.131:0/3459821089 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5540ae3a0 con 0x7fa5540cf8f0 2024-05-26T22:50:13.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:13.733+0000 7fa552000640 1 --2- 172.21.15.131:0/3459821089 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa5540cf8f0 0x7fa5540ad9e0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fa54000c430 tx=0x7fa54000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:13.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:13.733+0000 7fa54aa00640 1 -- 172.21.15.131:0/3459821089 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5400052d0 con 0x7fa5540cf8f0 2024-05-26T22:50:13.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:13.733+0000 7fa54aa00640 1 -- 172.21.15.131:0/3459821089 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa540005490 con 0x7fa5540cf8f0 2024-05-26T22:50:13.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:13.733+0000 7fa54aa00640 1 -- 172.21.15.131:0/3459821089 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa540007260 con 0x7fa5540cf8f0 2024-05-26T22:50:13.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:13.733+0000 7fa559800640 1 -- 172.21.15.131:0/3459821089 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5540ae680 con 0x7fa5540cf8f0 2024-05-26T22:50:13.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:13.733+0000 7fa559800640 1 -- 172.21.15.131:0/3459821089 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa554158c40 con 0x7fa5540cf8f0 2024-05-26T22:50:13.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:13.734+0000 7fa559800640 1 -- 172.21.15.131:0/3459821089 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa554068440 con 0x7fa5540cf8f0 2024-05-26T22:50:13.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:13.735+0000 7fa54aa00640 1 -- 172.21.15.131:0/3459821089 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa540007400 con 0x7fa5540cf8f0 2024-05-26T22:50:13.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:13.736+0000 7fa54aa00640 1 --2- 172.21.15.131:0/3459821089 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa53406be30 0x7fa53406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:13.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:13.736+0000 7fa54aa00640 1 -- 172.21.15.131:0/3459821089 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5967+0+0 (secure 0 0 0) 0x7fa54008a490 con 0x7fa5540cf8f0 2024-05-26T22:50:13.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:13.736+0000 7fa552a00640 1 --2- 172.21.15.131:0/3459821089 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa53406be30 0x7fa53406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:13.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:13.737+0000 7fa552a00640 1 --2- 172.21.15.131:0/3459821089 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa53406be30 0x7fa53406e2f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa544034000 tx=0x7fa54402c000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:13.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:13.741+0000 7fa54aa00640 1 -- 172.21.15.131:0/3459821089 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fa540058d30 con 0x7fa5540cf8f0 2024-05-26T22:50:13.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:13.939+0000 7fa559800640 1 -- 172.21.15.131:0/3459821089 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fa5540ce970 con 0x7fa5540cf8f0 2024-05-26T22:50:14.001 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-26T22:50:14.010 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-26T22:50:14.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:14.447+0000 7fa54aa00640 1 -- 172.21.15.131:0/3459821089 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v35) ==== 181+0+0 (secure 0 0 0) 0x7fa540058630 con 0x7fa5540cf8f0 2024-05-26T22:50:14.448 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_metadata' removed 2024-05-26T22:50:14.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:14.452+0000 7fa559800640 1 -- 172.21.15.131:0/3459821089 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa53406be30 msgr2=0x7fa53406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:14.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:14.452+0000 7fa559800640 1 --2- 172.21.15.131:0/3459821089 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa53406be30 0x7fa53406e2f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa544034000 tx=0x7fa54402c000 comp rx=0 tx=0).stop 2024-05-26T22:50:14.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:14.452+0000 7fa559800640 1 -- 172.21.15.131:0/3459821089 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa5540cf8f0 msgr2=0x7fa5540ad9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:14.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:14.452+0000 7fa559800640 1 --2- 172.21.15.131:0/3459821089 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa5540cf8f0 0x7fa5540ad9e0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fa54000c430 tx=0x7fa54000c8f0 comp rx=0 tx=0).stop 2024-05-26T22:50:14.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:14.453+0000 7fa559800640 1 -- 172.21.15.131:0/3459821089 shutdown_connections 2024-05-26T22:50:14.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:14.453+0000 7fa559800640 1 --2- 172.21.15.131:0/3459821089 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa53406be30 0x7fa53406e2f0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:14.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:14.453+0000 7fa559800640 1 --2- 172.21.15.131:0/3459821089 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa5540d0360 0x7fa5540c4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:14.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:14.453+0000 7fa559800640 1 --2- 172.21.15.131:0/3459821089 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa5540cf8f0 0x7fa5540ad9e0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:14.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:14.453+0000 7fa559800640 1 --2- 172.21.15.131:0/3459821089 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa55406ef70 0x7fa55414fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:14.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:14.453+0000 7fa559800640 1 -- 172.21.15.131:0/3459821089 >> 172.21.15.131:0/3459821089 conn(0x7fa5540b2110 msgr2=0x7fa55406b310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:14.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:14.456+0000 7fa559800640 1 -- 172.21.15.131:0/3459821089 shutdown_connections 2024-05-26T22:50:14.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:14.456+0000 7fa559800640 1 -- 172.21.15.131:0/3459821089 wait complete. 2024-05-26T22:50:14.480 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-26T22:50:14.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:14.670+0000 7f51a7a00640 1 Processor -- start 2024-05-26T22:50:14.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:14.670+0000 7f51a7a00640 1 -- start start 2024-05-26T22:50:14.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:14.671+0000 7f51a7a00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f51a00cf8f0 0x7f51a0072250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:14.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:14.672+0000 7f51a7a00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51a00ccb70 con 0x7f51a00d0360 2024-05-26T22:50:14.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:14.672+0000 7f51a7a00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51a00cce50 con 0x7f51a006ef70 2024-05-26T22:50:14.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:14.672+0000 7f51a7a00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f51a00cd130 con 0x7f51a00cf8f0 2024-05-26T22:50:14.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:14.672+0000 7f5197e00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f51a00d0360 0x7f51a006e860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:46738/0 (socket says 172.21.15.131:46738) 2024-05-26T22:50:14.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:14.672+0000 7f5197e00640 1 -- 172.21.15.131:0/1121480062 learned_addr learned my addr 172.21.15.131:0/1121480062 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:50:14.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:14.672+0000 7f51a5000640 1 --2- 172.21.15.131:0/1121480062 >> v2:172.21.15.187:3300/0 conn(0x7f51a00cf8f0 0x7f51a0072250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:14.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:14.672+0000 7f519fe00640 1 -- 172.21.15.131:0/1121480062 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 448973586 0 0) 0x7f51a00ccb70 con 0x7f51a00d0360 2024-05-26T22:50:14.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:14.672+0000 7f519fe00640 1 -- 172.21.15.131:0/1121480062 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5184003520 con 0x7f51a00d0360 2024-05-26T22:50:14.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:14.673+0000 7f519fe00640 1 -- 172.21.15.131:0/1121480062 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2746731875 0 0) 0x7f5184003520 con 0x7f51a00d0360 2024-05-26T22:50:14.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:14.673+0000 7f519fe00640 1 -- 172.21.15.131:0/1121480062 >> v2:172.21.15.187:3300/0 conn(0x7f51a00cf8f0 msgr2=0x7f51a0072250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:14.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:14.673+0000 7f519fe00640 1 --2- 172.21.15.131:0/1121480062 >> v2:172.21.15.187:3300/0 conn(0x7f51a00cf8f0 0x7f51a0072250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:14.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:14.673+0000 7f519fe00640 1 -- 172.21.15.131:0/1121480062 >> v1:172.21.15.187:6789/0 conn(0x7f51a006ef70 legacy=0x7f51a015ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:14.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:14.673+0000 7f519fe00640 1 -- 172.21.15.131:0/1121480062 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51a00cd410 con 0x7f51a00d0360 2024-05-26T22:50:14.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:14.673+0000 7f519fe00640 1 -- 172.21.15.131:0/1121480062 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7f5188001670 con 0x7f51a00d0360 2024-05-26T22:50:14.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:14.673+0000 7f519fe00640 1 -- 172.21.15.131:0/1121480062 >> v1:172.21.15.131:6789/0 conn(0x7f51a00d0360 legacy=0x7f51a006e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:14.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:14.673+0000 7f519fe00640 1 --2- 172.21.15.131:0/1121480062 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f51840039b0 0x7f5184003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:14.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:14.673+0000 7f519fe00640 1 --2- 172.21.15.131:0/1121480062 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5184004800 0x7f5184004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:14.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:14.673+0000 7f519fe00640 1 --2- 172.21.15.131:0/1121480062 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f51840051c0 0x7f518400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:14.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:14.673+0000 7f519fe00640 1 -- 172.21.15.131:0/1121480062 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f51a00cd130 con 0x7f51840051c0 2024-05-26T22:50:14.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:14.673+0000 7f519fe00640 1 -- 172.21.15.131:0/1121480062 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f518400c0f0 con 0x7f51840039b0 2024-05-26T22:50:14.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:14.673+0000 7f519fe00640 1 -- 172.21.15.131:0/1121480062 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f518400c2f0 con 0x7f5184004800 2024-05-26T22:50:14.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:14.673+0000 7f51a5a00640 1 --2- 172.21.15.131:0/1121480062 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f51840051c0 0x7f518400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:14.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:14.674+0000 7f51a5000640 1 --2- 172.21.15.131:0/1121480062 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f51840039b0 0x7f5184003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:14.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:14.674+0000 7f5197e00640 1 --2- 172.21.15.131:0/1121480062 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5184004800 0x7f5184004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:14.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:14.674+0000 7f51a5a00640 1 -- 172.21.15.131:0/1121480062 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5184004800 msgr2=0x7f5184004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:14.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:14.674+0000 7f51a5a00640 1 --2- 172.21.15.131:0/1121480062 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5184004800 0x7f5184004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:14.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:14.674+0000 7f51a5a00640 1 -- 172.21.15.131:0/1121480062 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f51840039b0 msgr2=0x7f5184003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:14.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:14.674+0000 7f51a5a00640 1 --2- 172.21.15.131:0/1121480062 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f51840039b0 0x7f5184003db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:14.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:14.674+0000 7f51a5a00640 1 -- 172.21.15.131:0/1121480062 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5184003520 con 0x7f51840051c0 2024-05-26T22:50:14.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:14.674+0000 7f51a5000640 1 --2- 172.21.15.131:0/1121480062 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f51840039b0 0x7f5184003db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:50:14.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:14.675+0000 7f51a5a00640 1 -- 172.21.15.131:0/1121480062 reap_dead start 2024-05-26T22:50:14.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:14.675+0000 7f51a5a00640 1 --2- 172.21.15.131:0/1121480062 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f51840051c0 0x7f518400b9c0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f51a006f400 tx=0x7f5184005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acf1348beb9f930e server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:14.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:14.675+0000 7f519fe00640 1 -- 172.21.15.131:0/1121480062 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5198004080 con 0x7f51840051c0 2024-05-26T22:50:14.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:14.675+0000 7f519fe00640 1 -- 172.21.15.131:0/1121480062 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5198006d70 con 0x7f51840051c0 2024-05-26T22:50:14.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:14.675+0000 7f519fe00640 1 -- 172.21.15.131:0/1121480062 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f519802ad70 con 0x7f51840051c0 2024-05-26T22:50:14.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:14.675+0000 7f51a7a00640 1 -- 172.21.15.131:0/1121480062 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f51840051c0 msgr2=0x7f518400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:14.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:14.675+0000 7f51a7a00640 1 --2- 172.21.15.131:0/1121480062 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f51840051c0 0x7f518400b9c0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f51a006f400 tx=0x7f5184005830 comp rx=0 tx=0).stop 2024-05-26T22:50:14.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:14.676+0000 7f51a7a00640 1 -- 172.21.15.131:0/1121480062 shutdown_connections 2024-05-26T22:50:14.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:14.676+0000 7f51a7a00640 1 --2- 172.21.15.131:0/1121480062 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f51840051c0 0x7f518400b9c0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:14.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:14.676+0000 7f51a7a00640 1 -- 172.21.15.131:0/1121480062 >> 172.21.15.131:0/1121480062 conn(0x7f51a00b2110 msgr2=0x7f51a00b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:14.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:14.676+0000 7f51a7a00640 1 -- 172.21.15.131:0/1121480062 shutdown_connections 2024-05-26T22:50:14.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:14.676+0000 7f51a7a00640 1 -- 172.21.15.131:0/1121480062 wait complete. 2024-05-26T22:50:14.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:14.677+0000 7f51a7a00640 1 Processor -- start 2024-05-26T22:50:14.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:14.678+0000 7f51a7a00640 1 -- start start 2024-05-26T22:50:14.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:14.679+0000 7f51a7a00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f51840051c0 0x7f51a0158c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:14.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:14.679+0000 7f51a7a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f51a00c9370 0x7f51a0159290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:14.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:14.679+0000 7f51a7a00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f51a00c97f0 0x7f51a01597d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:14.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:14.679+0000 7f51a7a00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f51a00723d0 con 0x7f51a00c9370 2024-05-26T22:50:14.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:14.679+0000 7f51a7a00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f51a0072250 con 0x7f51840051c0 2024-05-26T22:50:14.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:14.679+0000 7f51a7a00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f51a0072550 con 0x7f51a00c97f0 2024-05-26T22:50:14.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:14.679+0000 7f5197e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f51a00c9370 0x7f51a0159290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:14.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:14.679+0000 7f5197e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f51a00c9370 0x7f51a0159290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60508/0 (socket says 172.21.15.131:60508) 2024-05-26T22:50:14.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:14.679+0000 7f5197e00640 1 -- 172.21.15.131:0/2204955937 learned_addr learned my addr 172.21.15.131:0/2204955937 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:14.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:14.679+0000 7f51a5000640 1 --2- 172.21.15.131:0/2204955937 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f51840051c0 0x7f51a0158c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:14.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:14.679+0000 7f51a5a00640 1 --2- 172.21.15.131:0/2204955937 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f51a00c97f0 0x7f51a01597d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:14.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:14.680+0000 7f5197e00640 1 -- 172.21.15.131:0/2204955937 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f51a00c97f0 msgr2=0x7f51a01597d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:14.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:14.680+0000 7f5197e00640 1 --2- 172.21.15.131:0/2204955937 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f51a00c97f0 0x7f51a01597d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:14.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:14.680+0000 7f5197e00640 1 -- 172.21.15.131:0/2204955937 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f51840051c0 msgr2=0x7f51a0158c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:14.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:14.680+0000 7f5197e00640 1 --2- 172.21.15.131:0/2204955937 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f51840051c0 0x7f51a0158c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:14.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:14.680+0000 7f5197e00640 1 -- 172.21.15.131:0/2204955937 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51a00c9dc0 con 0x7f51a00c9370 2024-05-26T22:50:14.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:14.680+0000 7f5197e00640 1 --2- 172.21.15.131:0/2204955937 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f51a00c9370 0x7f51a0159290 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f518800d770 tx=0x7f518800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:14.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:14.680+0000 7f519d600640 1 -- 172.21.15.131:0/2204955937 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5188019070 con 0x7f51a00c9370 2024-05-26T22:50:14.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:14.680+0000 7f519d600640 1 -- 172.21.15.131:0/2204955937 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51880074e0 con 0x7f51a00c9370 2024-05-26T22:50:14.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:14.680+0000 7f519d600640 1 -- 172.21.15.131:0/2204955937 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5188004a00 con 0x7f51a00c9370 2024-05-26T22:50:14.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:14.680+0000 7f51a7a00640 1 -- 172.21.15.131:0/2204955937 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51a00ca0a0 con 0x7f51a00c9370 2024-05-26T22:50:14.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:14.680+0000 7f51a7a00640 1 -- 172.21.15.131:0/2204955937 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51a00ca660 con 0x7f51a00c9370 2024-05-26T22:50:14.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:14.681+0000 7f51a7a00640 1 -- 172.21.15.131:0/2204955937 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51a0068440 con 0x7f51a00c9370 2024-05-26T22:50:14.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:14.682+0000 7f519d600640 1 -- 172.21.15.131:0/2204955937 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f518801c070 con 0x7f51a00c9370 2024-05-26T22:50:14.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:14.683+0000 7f519d600640 1 --2- 172.21.15.131:0/2204955937 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f517c06be60 0x7f517c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:14.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:14.683+0000 7f519d600640 1 -- 172.21.15.131:0/2204955937 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5529+0+0 (secure 0 0 0) 0x7f518808a840 con 0x7f51a00c9370 2024-05-26T22:50:14.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:14.684+0000 7f51a5000640 1 --2- 172.21.15.131:0/2204955937 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f517c06be60 0x7f517c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:14.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:14.684+0000 7f51a5000640 1 --2- 172.21.15.131:0/2204955937 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f517c06be60 0x7f517c06e320 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5190006fd0 tx=0x7f5190008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:14.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:14.688+0000 7f519d600640 1 -- 172.21.15.131:0/2204955937 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f518800de20 con 0x7f51a00c9370 2024-05-26T22:50:14.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:14.890+0000 7f51a7a00640 1 -- 172.21.15.131:0/2204955937 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f51a00ca830 con 0x7f51a00c9370 2024-05-26T22:50:15.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:15.461+0000 7f519d600640 1 -- 172.21.15.131:0/2204955937 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v36) ==== 169+0+0 (secure 0 0 0) 0x7f5188059200 con 0x7f51a00c9370 2024-05-26T22:50:15.461 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_data' removed 2024-05-26T22:50:15.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:15.467+0000 7f51a7a00640 1 -- 172.21.15.131:0/2204955937 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f517c06be60 msgr2=0x7f517c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:15.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:15.467+0000 7f51a7a00640 1 --2- 172.21.15.131:0/2204955937 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f517c06be60 0x7f517c06e320 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5190006fd0 tx=0x7f5190008040 comp rx=0 tx=0).stop 2024-05-26T22:50:15.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:15.467+0000 7f51a7a00640 1 -- 172.21.15.131:0/2204955937 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f51a00c9370 msgr2=0x7f51a0159290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:15.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:15.467+0000 7f51a7a00640 1 --2- 172.21.15.131:0/2204955937 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f51a00c9370 0x7f51a0159290 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f518800d770 tx=0x7f518800dc30 comp rx=0 tx=0).stop 2024-05-26T22:50:15.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:15.468+0000 7f51a7a00640 1 -- 172.21.15.131:0/2204955937 shutdown_connections 2024-05-26T22:50:15.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:15.468+0000 7f51a7a00640 1 --2- 172.21.15.131:0/2204955937 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f51a00c97f0 0x7f51a01597d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:15.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:15.468+0000 7f51a7a00640 1 --2- 172.21.15.131:0/2204955937 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f517c06be60 0x7f517c06e320 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:15.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:15.468+0000 7f51a7a00640 1 --2- 172.21.15.131:0/2204955937 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f51a00c9370 0x7f51a0159290 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:15.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:15.468+0000 7f51a7a00640 1 --2- 172.21.15.131:0/2204955937 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f51840051c0 0x7f51a0158c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:15.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:15.468+0000 7f51a7a00640 1 -- 172.21.15.131:0/2204955937 >> 172.21.15.131:0/2204955937 conn(0x7f51a00b2110 msgr2=0x7f51a00ae660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:15.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:15.468+0000 7f51a7a00640 1 -- 172.21.15.131:0/2204955937 shutdown_connections 2024-05-26T22:50:15.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:15.468+0000 7f51a7a00640 1 -- 172.21.15.131:0/2204955937 wait complete. 2024-05-26T22:50:15.496 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_export_ephemeral_random_max 2024-05-26T22:50:15.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:15.682+0000 7f61c7000640 1 Processor -- start 2024-05-26T22:50:15.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:15.682+0000 7f61c7000640 1 -- start start 2024-05-26T22:50:15.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:15.683+0000 7f61c7000640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f61c00d03b0 0x7f61c015c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:15.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:15.683+0000 7f61c7000640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f61c015cd30 0x7f61c0144ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:15.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:15.683+0000 7f61c7000640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61c015fcc0 con 0x7f61c0156660 2024-05-26T22:50:15.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:15.684+0000 7f61c7000640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f61c015ffa0 con 0x7f61c015cd30 2024-05-26T22:50:15.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:15.684+0000 7f61c7000640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f61c0160280 con 0x7f61c00d03b0 2024-05-26T22:50:15.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:15.684+0000 7f61c6600640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f61c015cd30 0x7f61c0144ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:15.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:15.684+0000 7f61c5c00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f61c0156660 0x7f61c00cfca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:46748/0 (socket says 172.21.15.131:46748) 2024-05-26T22:50:15.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:15.684+0000 7f61c6600640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f61c015cd30 0x7f61c0144ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60518/0 (socket says 172.21.15.131:60518) 2024-05-26T22:50:15.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:15.684+0000 7f61c5c00640 1 -- 172.21.15.131:0/1797434944 learned_addr learned my addr 172.21.15.131:0/1797434944 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:50:15.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:15.684+0000 7f61c5200640 1 --2- 172.21.15.131:0/1797434944 >> v2:172.21.15.187:3300/0 conn(0x7f61c00d03b0 0x7f61c015c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:15.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:15.684+0000 7f61c6600640 1 -- 172.21.15.131:0/1797434944 >> v2:172.21.15.187:3300/0 conn(0x7f61c00d03b0 msgr2=0x7f61c015c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:15.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:15.684+0000 7f61c6600640 1 --2- 172.21.15.131:0/1797434944 >> v2:172.21.15.187:3300/0 conn(0x7f61c00d03b0 0x7f61c015c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:15.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:15.684+0000 7f61c6600640 1 -- 172.21.15.131:0/1797434944 >> v1:172.21.15.131:6789/0 conn(0x7f61c0156660 legacy=0x7f61c00cfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:15.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:15.684+0000 7f61c6600640 1 -- 172.21.15.131:0/1797434944 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61c0160560 con 0x7f61c015cd30 2024-05-26T22:50:15.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:15.684+0000 7f61bfe00640 1 -- 172.21.15.131:0/1797434944 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1329847055 0 0) 0x7f61c015fcc0 con 0x7f61c0156660 2024-05-26T22:50:15.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:15.684+0000 7f61c5200640 1 --2- 172.21.15.131:0/1797434944 >> v2:172.21.15.187:3300/0 conn(0x7f61c00d03b0 0x7f61c015c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:50:15.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:15.685+0000 7f61c6600640 1 --2- 172.21.15.131:0/1797434944 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61c015cd30 0x7f61c0144ec0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f61c006ea40 tx=0x7f61b802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48adbd51ea1b00da server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:15.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:15.686+0000 7f61bfe00640 1 -- 172.21.15.131:0/1797434944 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b8007740 con 0x7f61c015cd30 2024-05-26T22:50:15.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:15.686+0000 7f61bfe00640 1 -- 172.21.15.131:0/1797434944 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61b80078e0 con 0x7f61c015cd30 2024-05-26T22:50:15.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:15.686+0000 7f61bfe00640 1 -- 172.21.15.131:0/1797434944 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b8007cb0 con 0x7f61c015cd30 2024-05-26T22:50:15.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:15.686+0000 7f61c7000640 1 -- 172.21.15.131:0/1797434944 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61c015cd30 msgr2=0x7f61c0144ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:15.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:15.686+0000 7f61c7000640 1 --2- 172.21.15.131:0/1797434944 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61c015cd30 0x7f61c0144ec0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f61c006ea40 tx=0x7f61b802a680 comp rx=0 tx=0).stop 2024-05-26T22:50:15.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:15.687+0000 7f61c7000640 1 -- 172.21.15.131:0/1797434944 shutdown_connections 2024-05-26T22:50:15.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:15.687+0000 7f61c7000640 1 --2- 172.21.15.131:0/1797434944 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61c015cd30 0x7f61c0144ec0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:15.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:15.687+0000 7f61c7000640 1 --2- 172.21.15.131:0/1797434944 >> v2:172.21.15.187:3300/0 conn(0x7f61c00d03b0 0x7f61c015c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:15.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:15.687+0000 7f61c7000640 1 -- 172.21.15.131:0/1797434944 >> 172.21.15.131:0/1797434944 conn(0x7f61c00cc7f0 msgr2=0x7f61c00ccbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:15.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:15.687+0000 7f61c7000640 1 -- 172.21.15.131:0/1797434944 shutdown_connections 2024-05-26T22:50:15.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:15.687+0000 7f61c7000640 1 -- 172.21.15.131:0/1797434944 wait complete. 2024-05-26T22:50:15.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:15.688+0000 7f61c7000640 1 Processor -- start 2024-05-26T22:50:15.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:15.689+0000 7f61c7000640 1 -- start start 2024-05-26T22:50:15.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:15.690+0000 7f61c7000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61c00d03b0 0x7f61c01e9640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:15.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:15.690+0000 7f61c7000640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f61c0156660 0x7f61c01e6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:15.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:15.690+0000 7f61c7000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f61c015cd30 0x7f61c00b0150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:15.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:15.690+0000 7f61c7000640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f61c0160a80 con 0x7f61c00d03b0 2024-05-26T22:50:15.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:15.690+0000 7f61c7000640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f61c0156a40 con 0x7f61c0156660 2024-05-26T22:50:15.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:15.690+0000 7f61c7000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f61c015d140 con 0x7f61c015cd30 2024-05-26T22:50:15.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:15.690+0000 7f61c5c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61c00d03b0 0x7f61c01e9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:15.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:15.690+0000 7f61c5c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61c00d03b0 0x7f61c01e9640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60530/0 (socket says 172.21.15.131:60530) 2024-05-26T22:50:15.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:15.690+0000 7f61c5c00640 1 -- 172.21.15.131:0/3118360101 learned_addr learned my addr 172.21.15.131:0/3118360101 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:15.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:15.690+0000 7f61c5c00640 1 -- 172.21.15.131:0/3118360101 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f61c015cd30 msgr2=0x7f61c00b0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:15.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:15.690+0000 7f61c5c00640 1 --2- 172.21.15.131:0/3118360101 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f61c015cd30 0x7f61c00b0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:15.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:15.690+0000 7f61c5c00640 1 -- 172.21.15.131:0/3118360101 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f61c0156660 msgr2=0x7f61c01e6fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:15.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:15.690+0000 7f61c5c00640 1 --2- 172.21.15.131:0/3118360101 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f61c0156660 0x7f61c01e6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:15.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:15.690+0000 7f61c5c00640 1 -- 172.21.15.131:0/3118360101 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61c01e7510 con 0x7f61c00d03b0 2024-05-26T22:50:15.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:15.691+0000 7f61c5c00640 1 --2- 172.21.15.131:0/3118360101 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61c00d03b0 0x7f61c01e9640 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f61b400c400 tx=0x7f61b400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:15.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:15.691+0000 7f61bd600640 1 -- 172.21.15.131:0/3118360101 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b4017070 con 0x7f61c00d03b0 2024-05-26T22:50:15.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:15.691+0000 7f61bd600640 1 -- 172.21.15.131:0/3118360101 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61b4005ce0 con 0x7f61c00d03b0 2024-05-26T22:50:15.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:15.691+0000 7f61bd600640 1 -- 172.21.15.131:0/3118360101 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b4007310 con 0x7f61c00d03b0 2024-05-26T22:50:15.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:15.691+0000 7f61c7000640 1 -- 172.21.15.131:0/3118360101 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61c01e77b0 con 0x7f61c00d03b0 2024-05-26T22:50:15.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:15.691+0000 7f61c7000640 1 -- 172.21.15.131:0/3118360101 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61c01e7d70 con 0x7f61c00d03b0 2024-05-26T22:50:15.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:15.692+0000 7f61c7000640 1 -- 172.21.15.131:0/3118360101 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61c00694e0 con 0x7f61c00d03b0 2024-05-26T22:50:15.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:15.693+0000 7f61bd600640 1 -- 172.21.15.131:0/3118360101 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f61b401a070 con 0x7f61c00d03b0 2024-05-26T22:50:15.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:15.694+0000 7f61bd600640 1 --2- 172.21.15.131:0/3118360101 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f61a006be30 0x7f61a006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:15.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:15.694+0000 7f61bd600640 1 -- 172.21.15.131:0/3118360101 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5115+0+0 (secure 0 0 0) 0x7f61b408a110 con 0x7f61c00d03b0 2024-05-26T22:50:15.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:15.695+0000 7f61c5200640 1 --2- 172.21.15.131:0/3118360101 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f61a006be30 0x7f61a006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:15.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:15.695+0000 7f61c5200640 1 --2- 172.21.15.131:0/3118360101 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f61a006be30 0x7f61a006e2f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f61ac006fd0 tx=0x7f61ac008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:15.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:15.699+0000 7f61bd600640 1 -- 172.21.15.131:0/3118360101 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f61b4058c70 con 0x7f61c00d03b0 2024-05-26T22:50:15.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:15.897+0000 7f61c7000640 1 -- 172.21.15.131:0/3118360101 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_export_ephemeral_random_max"} v 0) -- 0x7f61c006ede0 con 0x7f61c00d03b0 2024-05-26T22:50:15.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:15.904+0000 7f61bd600640 1 -- 172.21.15.131:0/3118360101 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_export_ephemeral_random_max"}]=0 v15) ==== 114+0+0 (secure 0 0 0) 0x7f61b4058570 con 0x7f61c00d03b0 2024-05-26T22:50:15.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:15.907+0000 7f61c7000640 1 -- 172.21.15.131:0/3118360101 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f61a006be30 msgr2=0x7f61a006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:15.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:15.907+0000 7f61c7000640 1 --2- 172.21.15.131:0/3118360101 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f61a006be30 0x7f61a006e2f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f61ac006fd0 tx=0x7f61ac008040 comp rx=0 tx=0).stop 2024-05-26T22:50:15.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:15.907+0000 7f61c7000640 1 -- 172.21.15.131:0/3118360101 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61c00d03b0 msgr2=0x7f61c01e9640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:15.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:15.907+0000 7f61c7000640 1 --2- 172.21.15.131:0/3118360101 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61c00d03b0 0x7f61c01e9640 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f61b400c400 tx=0x7f61b400c8c0 comp rx=0 tx=0).stop 2024-05-26T22:50:15.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:15.907+0000 7f61c7000640 1 -- 172.21.15.131:0/3118360101 shutdown_connections 2024-05-26T22:50:15.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:15.907+0000 7f61c7000640 1 --2- 172.21.15.131:0/3118360101 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f61c015cd30 0x7f61c00b0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:15.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:15.907+0000 7f61c7000640 1 --2- 172.21.15.131:0/3118360101 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f61a006be30 0x7f61a006e2f0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:15.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:15.907+0000 7f61c7000640 1 --2- 172.21.15.131:0/3118360101 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f61c0156660 0x7f61c01e6fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:15.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:15.907+0000 7f61c7000640 1 --2- 172.21.15.131:0/3118360101 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61c00d03b0 0x7f61c01e9640 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:15.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:15.907+0000 7f61c7000640 1 -- 172.21.15.131:0/3118360101 >> 172.21.15.131:0/3118360101 conn(0x7f61c00cc7f0 msgr2=0x7f61c00b4870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:15.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:15.908+0000 7f61c7000640 1 -- 172.21.15.131:0/3118360101 shutdown_connections 2024-05-26T22:50:15.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:15.908+0000 7f61c7000640 1 -- 172.21.15.131:0/3118360101 wait complete. 2024-05-26T22:50:15.926 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_export_ephemeral_random 2024-05-26T22:50:16.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.129+0000 7f897ec00640 1 Processor -- start 2024-05-26T22:50:16.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.129+0000 7f897ec00640 1 -- start start 2024-05-26T22:50:16.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.131+0000 7f897ec00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f89780b5450 0x7f89780b43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:16.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.131+0000 7f897ec00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f89780d0460 0x7f897806d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:16.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.131+0000 7f897ec00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89780b4900 con 0x7f897806f420 2024-05-26T22:50:16.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.131+0000 7f897ec00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f8978160d90 con 0x7f89780d0460 2024-05-26T22:50:16.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.131+0000 7f897ec00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f8978160180 con 0x7f89780b5450 2024-05-26T22:50:16.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.131+0000 7f897cc00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f89780d0460 0x7f897806d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:16.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.131+0000 7f8977400640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f897806f420 0x7f89780cfd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:46750/0 (socket says 172.21.15.131:46750) 2024-05-26T22:50:16.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.131+0000 7f8977400640 1 -- 172.21.15.131:0/1504275293 learned_addr learned my addr 172.21.15.131:0/1504275293 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:50:16.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.132+0000 7f8977e00640 1 --2- 172.21.15.131:0/1504275293 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f89780b5450 0x7f89780b43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:16.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.132+0000 7f897cc00640 1 -- 172.21.15.131:0/1504275293 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f89780b5450 msgr2=0x7f89780b43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:16.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.132+0000 7f897cc00640 1 --2- 172.21.15.131:0/1504275293 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f89780b5450 0x7f89780b43c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:16.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.132+0000 7f897cc00640 1 -- 172.21.15.131:0/1504275293 >> v1:172.21.15.131:6789/0 conn(0x7f897806f420 legacy=0x7f89780cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:16.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.132+0000 7f897cc00640 1 -- 172.21.15.131:0/1504275293 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8978160460 con 0x7f89780d0460 2024-05-26T22:50:16.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.133+0000 7f897cc00640 1 --2- 172.21.15.131:0/1504275293 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f89780d0460 0x7f897806d600 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f8970009d90 tx=0x7f897002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e20aacdacda932b server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:16.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.133+0000 7f8976a00640 1 -- 172.21.15.131:0/1504275293 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8970007740 con 0x7f89780d0460 2024-05-26T22:50:16.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.133+0000 7f8976a00640 1 -- 172.21.15.131:0/1504275293 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89700078e0 con 0x7f89780d0460 2024-05-26T22:50:16.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.133+0000 7f8976a00640 1 -- 172.21.15.131:0/1504275293 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8970007cb0 con 0x7f89780d0460 2024-05-26T22:50:16.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.134+0000 7f897ec00640 1 -- 172.21.15.131:0/1504275293 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f89780d0460 msgr2=0x7f897806d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:16.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.134+0000 7f897ec00640 1 --2- 172.21.15.131:0/1504275293 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f89780d0460 0x7f897806d600 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f8970009d90 tx=0x7f897002a660 comp rx=0 tx=0).stop 2024-05-26T22:50:16.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.134+0000 7f897ec00640 1 -- 172.21.15.131:0/1504275293 shutdown_connections 2024-05-26T22:50:16.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.134+0000 7f897ec00640 1 --2- 172.21.15.131:0/1504275293 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f89780d0460 0x7f897806d600 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:16.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.134+0000 7f897ec00640 1 --2- 172.21.15.131:0/1504275293 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f89780b5450 0x7f89780b43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:16.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.134+0000 7f897ec00640 1 -- 172.21.15.131:0/1504275293 >> 172.21.15.131:0/1504275293 conn(0x7f897806bc90 msgr2=0x7f89780b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:16.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.134+0000 7f897ec00640 1 -- 172.21.15.131:0/1504275293 shutdown_connections 2024-05-26T22:50:16.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.134+0000 7f897ec00640 1 -- 172.21.15.131:0/1504275293 wait complete. 2024-05-26T22:50:16.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.136+0000 7f897ec00640 1 Processor -- start 2024-05-26T22:50:16.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.136+0000 7f897ec00640 1 -- start start 2024-05-26T22:50:16.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.137+0000 7f897ec00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f897806f420 0x7f8978154ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:16.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.137+0000 7f897ec00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f89780b5450 0x7f89781f34c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:16.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.137+0000 7f897ec00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f89780d0460 0x7f8978072950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:16.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.137+0000 7f897ec00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f89780b5830 con 0x7f89780b5450 2024-05-26T22:50:16.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.137+0000 7f897ec00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8978160a20 con 0x7f89780d0460 2024-05-26T22:50:16.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.137+0000 7f897ec00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f897815fde0 con 0x7f897806f420 2024-05-26T22:50:16.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.137+0000 7f8977400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f89780b5450 0x7f89781f34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:16.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.137+0000 7f8977400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f89780b5450 0x7f89781f34c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60542/0 (socket says 172.21.15.131:60542) 2024-05-26T22:50:16.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.137+0000 7f8977400640 1 -- 172.21.15.131:0/4136230401 learned_addr learned my addr 172.21.15.131:0/4136230401 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:16.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.138+0000 7f8977400640 1 -- 172.21.15.131:0/4136230401 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f897806f420 msgr2=0x7f8978154ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:16.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.138+0000 7f8977400640 1 --2- 172.21.15.131:0/4136230401 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f897806f420 0x7f8978154ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:16.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.138+0000 7f8977400640 1 -- 172.21.15.131:0/4136230401 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f89780d0460 msgr2=0x7f8978072950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:16.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.138+0000 7f897cc00640 1 --2- 172.21.15.131:0/4136230401 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f89780d0460 0x7f8978072950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:16.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.138+0000 7f8977400640 1 --2- 172.21.15.131:0/4136230401 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f89780d0460 0x7f8978072950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:16.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.138+0000 7f8977400640 1 -- 172.21.15.131:0/4136230401 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89781551d0 con 0x7f89780b5450 2024-05-26T22:50:16.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.138+0000 7f8977400640 1 --2- 172.21.15.131:0/4136230401 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f89780b5450 0x7f89781f34c0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f896400c3c0 tx=0x7f896400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:16.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.138+0000 7f896fe00640 1 -- 172.21.15.131:0/4136230401 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8964017070 con 0x7f89780b5450 2024-05-26T22:50:16.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.138+0000 7f896fe00640 1 -- 172.21.15.131:0/4136230401 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8964005ce0 con 0x7f89780b5450 2024-05-26T22:50:16.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.138+0000 7f897ec00640 1 -- 172.21.15.131:0/4136230401 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89781554b0 con 0x7f89780b5450 2024-05-26T22:50:16.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.139+0000 7f896fe00640 1 -- 172.21.15.131:0/4136230401 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89640072f0 con 0x7f89780b5450 2024-05-26T22:50:16.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.139+0000 7f897ec00640 1 -- 172.21.15.131:0/4136230401 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8978155a70 con 0x7f89780b5450 2024-05-26T22:50:16.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.140+0000 7f897ec00640 1 -- 172.21.15.131:0/4136230401 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8978074e90 con 0x7f89780b5450 2024-05-26T22:50:16.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.141+0000 7f896fe00640 1 -- 172.21.15.131:0/4136230401 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f896401a070 con 0x7f89780b5450 2024-05-26T22:50:16.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.142+0000 7f896fe00640 1 --2- 172.21.15.131:0/4136230401 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f895806be30 0x7f895806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:16.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.142+0000 7f896fe00640 1 -- 172.21.15.131:0/4136230401 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5115+0+0 (secure 0 0 0) 0x7f8964089ee0 con 0x7f89780b5450 2024-05-26T22:50:16.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.142+0000 7f8977e00640 1 --2- 172.21.15.131:0/4136230401 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f895806be30 0x7f895806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:16.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.143+0000 7f8977e00640 1 --2- 172.21.15.131:0/4136230401 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f895806be30 0x7f895806e2f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f89680098f0 tx=0x7f8968008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:16.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.147+0000 7f896fe00640 1 -- 172.21.15.131:0/4136230401 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f8964058b70 con 0x7f89780b5450 2024-05-26T22:50:16.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.344+0000 7f897ec00640 1 -- 172.21.15.131:0/4136230401 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_export_ephemeral_random"} v 0) -- 0x7f89780695c0 con 0x7f89780b5450 2024-05-26T22:50:16.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.350+0000 7f896fe00640 1 -- 172.21.15.131:0/4136230401 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_export_ephemeral_random"}]=0 v16) ==== 110+0+0 (secure 0 0 0) 0x7f8964058470 con 0x7f89780b5450 2024-05-26T22:50:16.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.354+0000 7f897ec00640 1 -- 172.21.15.131:0/4136230401 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f895806be30 msgr2=0x7f895806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:16.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.354+0000 7f897ec00640 1 --2- 172.21.15.131:0/4136230401 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f895806be30 0x7f895806e2f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f89680098f0 tx=0x7f8968008040 comp rx=0 tx=0).stop 2024-05-26T22:50:16.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.354+0000 7f897ec00640 1 -- 172.21.15.131:0/4136230401 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f89780b5450 msgr2=0x7f89781f34c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:16.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.354+0000 7f897ec00640 1 --2- 172.21.15.131:0/4136230401 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f89780b5450 0x7f89781f34c0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f896400c3c0 tx=0x7f896400c880 comp rx=0 tx=0).stop 2024-05-26T22:50:16.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.354+0000 7f897ec00640 1 -- 172.21.15.131:0/4136230401 shutdown_connections 2024-05-26T22:50:16.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.354+0000 7f897ec00640 1 --2- 172.21.15.131:0/4136230401 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f895806be30 0x7f895806e2f0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:16.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.354+0000 7f897ec00640 1 --2- 172.21.15.131:0/4136230401 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f89780d0460 0x7f8978072950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:16.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.354+0000 7f897ec00640 1 --2- 172.21.15.131:0/4136230401 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f89780b5450 0x7f89781f34c0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:16.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.354+0000 7f897ec00640 1 --2- 172.21.15.131:0/4136230401 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f897806f420 0x7f8978154ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:16.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.354+0000 7f897ec00640 1 -- 172.21.15.131:0/4136230401 >> 172.21.15.131:0/4136230401 conn(0x7f897806bc90 msgr2=0x7f89780b1300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:16.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.354+0000 7f897ec00640 1 -- 172.21.15.131:0/4136230401 shutdown_connections 2024-05-26T22:50:16.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.354+0000 7f897ec00640 1 -- 172.21.15.131:0/4136230401 wait complete. 2024-05-26T22:50:16.374 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_export_ephemeral_distributed 2024-05-26T22:50:16.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.579+0000 7fc8f2400640 1 Processor -- start 2024-05-26T22:50:16.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.579+0000 7fc8f2400640 1 -- start start 2024-05-26T22:50:16.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.580+0000 7fc8f2400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc8ec06dd30 0x7fc8ec06e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:16.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.580+0000 7fc8f2400640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8ec15c1c0 con 0x7fc8ec155770 2024-05-26T22:50:16.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.581+0000 7fc8f2400640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8ec15c4a0 con 0x7fc8ec0725a0 2024-05-26T22:50:16.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.581+0000 7fc8f2400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc8ec06e6d0 con 0x7fc8ec06dd30 2024-05-26T22:50:16.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.581+0000 7fc8eb400640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fc8ec155770 0x7fc8ec15b700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:46756/0 (socket says 172.21.15.131:46756) 2024-05-26T22:50:16.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.581+0000 7fc8eb400640 1 -- 172.21.15.131:0/3178176563 learned_addr learned my addr 172.21.15.131:0/3178176563 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:50:16.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.581+0000 7fc8eaa00640 1 -- 172.21.15.131:0/3178176563 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3428530950 0 0) 0x7fc8ec15c1c0 con 0x7fc8ec155770 2024-05-26T22:50:16.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.581+0000 7fc8e2a00640 1 --2- 172.21.15.131:0/3178176563 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc8ec06dd30 0x7fc8ec06e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:16.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.581+0000 7fc8eaa00640 1 -- 172.21.15.131:0/3178176563 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc8d0005730 con 0x7fc8ec155770 2024-05-26T22:50:16.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.581+0000 7fc8eaa00640 1 -- 172.21.15.131:0/3178176563 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2946720944 0 0) 0x7fc8d0005730 con 0x7fc8ec155770 2024-05-26T22:50:16.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.581+0000 7fc8eaa00640 1 -- 172.21.15.131:0/3178176563 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc8ec06dd30 msgr2=0x7fc8ec06e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:16.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.581+0000 7fc8eaa00640 1 --2- 172.21.15.131:0/3178176563 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc8ec06dd30 0x7fc8ec06e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:16.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.582+0000 7fc8eaa00640 1 -- 172.21.15.131:0/3178176563 >> v1:172.21.15.187:6789/0 conn(0x7fc8ec0725a0 legacy=0x7fc8ec0729d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:16.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.582+0000 7fc8eaa00640 1 -- 172.21.15.131:0/3178176563 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8ec071da0 con 0x7fc8ec155770 2024-05-26T22:50:16.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.582+0000 7fc8eaa00640 1 -- 172.21.15.131:0/3178176563 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7fc8dc0019f0 con 0x7fc8ec155770 2024-05-26T22:50:16.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.582+0000 7fc8eaa00640 1 -- 172.21.15.131:0/3178176563 >> v1:172.21.15.131:6789/0 conn(0x7fc8ec155770 legacy=0x7fc8ec15b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:16.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.582+0000 7fc8eaa00640 1 --2- 172.21.15.131:0/3178176563 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc8d0005bf0 0x7fc8d0006010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:16.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.582+0000 7fc8eaa00640 1 --2- 172.21.15.131:0/3178176563 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc8ec06dd30 0x7fc8d0006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:16.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.582+0000 7fc8eaa00640 1 --2- 172.21.15.131:0/3178176563 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc8d0006f50 0x7fc8d00073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:16.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.582+0000 7fc8eaa00640 1 -- 172.21.15.131:0/3178176563 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc8d000ddf0 con 0x7fc8d0006f50 2024-05-26T22:50:16.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.582+0000 7fc8eaa00640 1 -- 172.21.15.131:0/3178176563 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc8d000df70 con 0x7fc8d0005bf0 2024-05-26T22:50:16.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.582+0000 7fc8eaa00640 1 -- 172.21.15.131:0/3178176563 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc8d000e170 con 0x7fc8ec06dd30 2024-05-26T22:50:16.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.582+0000 7fc8ebe00640 1 --2- 172.21.15.131:0/3178176563 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc8d0006f50 0x7fc8d00073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:16.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.582+0000 7fc8eb400640 1 --2- 172.21.15.131:0/3178176563 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc8d0005bf0 0x7fc8d0006010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:16.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.582+0000 7fc8e2a00640 1 --2- 172.21.15.131:0/3178176563 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc8ec06dd30 0x7fc8d0006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:16.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.583+0000 7fc8ebe00640 1 -- 172.21.15.131:0/3178176563 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc8ec06dd30 msgr2=0x7fc8d0006a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:16.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.583+0000 7fc8ebe00640 1 --2- 172.21.15.131:0/3178176563 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc8ec06dd30 0x7fc8d0006a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:16.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.583+0000 7fc8ebe00640 1 -- 172.21.15.131:0/3178176563 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc8d0005bf0 msgr2=0x7fc8d0006010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:16.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.583+0000 7fc8ebe00640 1 --2- 172.21.15.131:0/3178176563 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc8d0005bf0 0x7fc8d0006010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:16.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.583+0000 7fc8ebe00640 1 -- 172.21.15.131:0/3178176563 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8d0005730 con 0x7fc8d0006f50 2024-05-26T22:50:16.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.584+0000 7fc8ebe00640 1 --2- 172.21.15.131:0/3178176563 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc8d0006f50 0x7fc8d00073d0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fc8e4001750 tx=0x7fc8d0007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d58ec7354b396183 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:16.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.584+0000 7fc8eaa00640 1 -- 172.21.15.131:0/3178176563 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e4001370 con 0x7fc8d0006f50 2024-05-26T22:50:16.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.584+0000 7fc8eaa00640 1 -- 172.21.15.131:0/3178176563 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8e40052d0 con 0x7fc8d0006f50 2024-05-26T22:50:16.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.584+0000 7fc8eaa00640 1 -- 172.21.15.131:0/3178176563 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e402abe0 con 0x7fc8d0006f50 2024-05-26T22:50:16.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.584+0000 7fc8f2400640 1 -- 172.21.15.131:0/3178176563 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc8d0006f50 msgr2=0x7fc8d00073d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:16.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.584+0000 7fc8f2400640 1 --2- 172.21.15.131:0/3178176563 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc8d0006f50 0x7fc8d00073d0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fc8e4001750 tx=0x7fc8d0007a40 comp rx=0 tx=0).stop 2024-05-26T22:50:16.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.584+0000 7fc8ebe00640 1 -- 172.21.15.131:0/3178176563 reap_dead start 2024-05-26T22:50:16.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.585+0000 7fc8f2400640 1 -- 172.21.15.131:0/3178176563 shutdown_connections 2024-05-26T22:50:16.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.585+0000 7fc8f2400640 1 -- 172.21.15.131:0/3178176563 >> 172.21.15.131:0/3178176563 conn(0x7fc8ec06bc90 msgr2=0x7fc8ec0b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:16.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.585+0000 7fc8f2400640 1 -- 172.21.15.131:0/3178176563 shutdown_connections 2024-05-26T22:50:16.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.585+0000 7fc8f2400640 1 -- 172.21.15.131:0/3178176563 wait complete. 2024-05-26T22:50:16.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.586+0000 7fc8f2400640 1 Processor -- start 2024-05-26T22:50:16.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.587+0000 7fc8f2400640 1 -- start start 2024-05-26T22:50:16.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.587+0000 7fc8f2400640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc8d0006f50 0x7fc8ec09d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:16.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.587+0000 7fc8f2400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc8ec1efc20 0x7fc8ec1f00a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:16.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.588+0000 7fc8f2400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc8ec1f05e0 0x7fc8ec0affc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:16.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.588+0000 7fc8f2400640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc8ec15c8c0 con 0x7fc8ec1f05e0 2024-05-26T22:50:16.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.588+0000 7fc8f2400640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc8ec155b50 con 0x7fc8d0006f50 2024-05-26T22:50:16.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.588+0000 7fc8f2400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc8ec15be10 con 0x7fc8ec1efc20 2024-05-26T22:50:16.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.588+0000 7fc8ebe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc8ec1f05e0 0x7fc8ec0affc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:16.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.588+0000 7fc8ebe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc8ec1f05e0 0x7fc8ec0affc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60568/0 (socket says 172.21.15.131:60568) 2024-05-26T22:50:16.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.588+0000 7fc8ebe00640 1 -- 172.21.15.131:0/2469478670 learned_addr learned my addr 172.21.15.131:0/2469478670 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:16.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.588+0000 7fc8eb400640 1 --2- 172.21.15.131:0/2469478670 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc8d0006f50 0x7fc8ec09d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:16.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.588+0000 7fc8e2a00640 1 --2- 172.21.15.131:0/2469478670 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc8ec1efc20 0x7fc8ec1f00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:16.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.588+0000 7fc8ebe00640 1 -- 172.21.15.131:0/2469478670 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc8ec1efc20 msgr2=0x7fc8ec1f00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:16.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.588+0000 7fc8ebe00640 1 --2- 172.21.15.131:0/2469478670 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc8ec1efc20 0x7fc8ec1f00a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:16.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.588+0000 7fc8ebe00640 1 -- 172.21.15.131:0/2469478670 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc8d0006f50 msgr2=0x7fc8ec09d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:16.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.588+0000 7fc8ebe00640 1 --2- 172.21.15.131:0/2469478670 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc8d0006f50 0x7fc8ec09d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:16.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.588+0000 7fc8ebe00640 1 -- 172.21.15.131:0/2469478670 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8ec0af800 con 0x7fc8ec1f05e0 2024-05-26T22:50:16.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.589+0000 7fc8ebe00640 1 --2- 172.21.15.131:0/2469478670 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc8ec1f05e0 0x7fc8ec0affc0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fc8e402a590 tx=0x7fc8e4001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:16.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.589+0000 7fc8e3e00640 1 -- 172.21.15.131:0/2469478670 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e4005480 con 0x7fc8ec1f05e0 2024-05-26T22:50:16.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.589+0000 7fc8e3e00640 1 -- 172.21.15.131:0/2469478670 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8e4040070 con 0x7fc8ec1f05e0 2024-05-26T22:50:16.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.589+0000 7fc8f2400640 1 -- 172.21.15.131:0/2469478670 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8ec1f0a90 con 0x7fc8ec1f05e0 2024-05-26T22:50:16.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.589+0000 7fc8e3e00640 1 -- 172.21.15.131:0/2469478670 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e4007240 con 0x7fc8ec1f05e0 2024-05-26T22:50:16.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.589+0000 7fc8f2400640 1 -- 172.21.15.131:0/2469478670 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8ec14ecd0 con 0x7fc8ec1f05e0 2024-05-26T22:50:16.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.590+0000 7fc8f2400640 1 -- 172.21.15.131:0/2469478670 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8ec074e90 con 0x7fc8ec1f05e0 2024-05-26T22:50:16.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.591+0000 7fc8e3e00640 1 -- 172.21.15.131:0/2469478670 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc8e403a070 con 0x7fc8ec1f05e0 2024-05-26T22:50:16.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.592+0000 7fc8e3e00640 1 --2- 172.21.15.131:0/2469478670 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc8c806be30 0x7fc8c806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:16.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.592+0000 7fc8e3e00640 1 -- 172.21.15.131:0/2469478670 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5115+0+0 (secure 0 0 0) 0x7fc8e40aa2b0 con 0x7fc8ec1f05e0 2024-05-26T22:50:16.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.592+0000 7fc8eb400640 1 --2- 172.21.15.131:0/2469478670 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc8c806be30 0x7fc8c806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:16.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.593+0000 7fc8eb400640 1 --2- 172.21.15.131:0/2469478670 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc8c806be30 0x7fc8c806e2f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc002230 tx=0x7fc8dc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:16.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.597+0000 7fc8e3e00640 1 -- 172.21.15.131:0/2469478670 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fc8e4078e10 con 0x7fc8ec1f05e0 2024-05-26T22:50:16.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.796+0000 7fc8f2400640 1 -- 172.21.15.131:0/2469478670 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_export_ephemeral_distributed"} v 0) -- 0x7fc8ec0cca30 con 0x7fc8ec1f05e0 2024-05-26T22:50:16.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.801+0000 7fc8e3e00640 1 -- 172.21.15.131:0/2469478670 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_export_ephemeral_distributed"}]=0 v17) ==== 115+0+0 (secure 0 0 0) 0x7fc8e4078710 con 0x7fc8ec1f05e0 2024-05-26T22:50:16.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.804+0000 7fc8f2400640 1 -- 172.21.15.131:0/2469478670 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc8c806be30 msgr2=0x7fc8c806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:16.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.804+0000 7fc8f2400640 1 --2- 172.21.15.131:0/2469478670 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc8c806be30 0x7fc8c806e2f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc002230 tx=0x7fc8dc009040 comp rx=0 tx=0).stop 2024-05-26T22:50:16.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.804+0000 7fc8f2400640 1 -- 172.21.15.131:0/2469478670 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc8ec1f05e0 msgr2=0x7fc8ec0affc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:16.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.804+0000 7fc8f2400640 1 --2- 172.21.15.131:0/2469478670 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc8ec1f05e0 0x7fc8ec0affc0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fc8e402a590 tx=0x7fc8e4001120 comp rx=0 tx=0).stop 2024-05-26T22:50:16.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.805+0000 7fc8f2400640 1 -- 172.21.15.131:0/2469478670 shutdown_connections 2024-05-26T22:50:16.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.805+0000 7fc8f2400640 1 --2- 172.21.15.131:0/2469478670 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc8ec1efc20 0x7fc8ec1f00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:16.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.805+0000 7fc8f2400640 1 --2- 172.21.15.131:0/2469478670 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc8c806be30 0x7fc8c806e2f0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:16.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.805+0000 7fc8f2400640 1 --2- 172.21.15.131:0/2469478670 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc8ec1f05e0 0x7fc8ec0affc0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:16.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.805+0000 7fc8f2400640 1 --2- 172.21.15.131:0/2469478670 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc8d0006f50 0x7fc8ec09d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:16.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.805+0000 7fc8f2400640 1 -- 172.21.15.131:0/2469478670 >> 172.21.15.131:0/2469478670 conn(0x7fc8ec06bc90 msgr2=0x7fc8ec0aef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:16.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.805+0000 7fc8f2400640 1 -- 172.21.15.131:0/2469478670 shutdown_connections 2024-05-26T22:50:16.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:16.805+0000 7fc8f2400640 1 -- 172.21.15.131:0/2469478670 wait complete. 2024-05-26T22:50:16.824 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-26T22:50:17.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.010+0000 7fb361800640 1 Processor -- start 2024-05-26T22:50:17.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.010+0000 7fb361800640 1 -- start start 2024-05-26T22:50:17.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.011+0000 7fb361800640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fb35c0b5450 0x7fb35c0b43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:17.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.011+0000 7fb361800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb35c06f420 0x7fb35c0cfd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:17.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.012+0000 7fb361800640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb35c0b4900 con 0x7fb35c0cf590 2024-05-26T22:50:17.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.012+0000 7fb361800640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fb35c160d90 con 0x7fb35c0b5450 2024-05-26T22:50:17.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.012+0000 7fb361800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb35c160180 con 0x7fb35c06f420 2024-05-26T22:50:17.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.012+0000 7fb35aa00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fb35c0b5450 0x7fb35c0b43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:17.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.012+0000 7fb35a000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb35c06f420 0x7fb35c0cfd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:17.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.012+0000 7fb35b400640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7fb35c0cf590 0x7fb35c06d600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.131:49174/0 (socket says 172.21.15.131:49174) 2024-05-26T22:50:17.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.012+0000 7fb35b400640 1 -- 172.21.15.131:0/858463881 learned_addr learned my addr 172.21.15.131:0/858463881 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:50:17.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.013+0000 7fb359600640 1 -- 172.21.15.131:0/858463881 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1316431869 0 0) 0x7fb35c0b4900 con 0x7fb35c0cf590 2024-05-26T22:50:17.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.013+0000 7fb359600640 1 -- 172.21.15.131:0/858463881 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb340003520 con 0x7fb35c0cf590 2024-05-26T22:50:17.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.013+0000 7fb35a000640 1 -- 172.21.15.131:0/858463881 >> v2:172.21.15.187:3300/0 conn(0x7fb35c0b5450 msgr2=0x7fb35c0b43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:17.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.013+0000 7fb35a000640 1 --2- 172.21.15.131:0/858463881 >> v2:172.21.15.187:3300/0 conn(0x7fb35c0b5450 0x7fb35c0b43c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:17.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.013+0000 7fb35a000640 1 -- 172.21.15.131:0/858463881 >> v1:172.21.15.187:6789/0 conn(0x7fb35c0cf590 legacy=0x7fb35c06d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:17.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.013+0000 7fb35a000640 1 -- 172.21.15.131:0/858463881 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb35c160460 con 0x7fb35c06f420 2024-05-26T22:50:17.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.014+0000 7fb35a000640 1 --2- 172.21.15.131:0/858463881 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb35c06f420 0x7fb35c0cfd50 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb3480097a0 tx=0x7fb34802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cbfbac7c4006b191 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:17.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.015+0000 7fb359600640 1 -- 172.21.15.131:0/858463881 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb348037070 con 0x7fb35c06f420 2024-05-26T22:50:17.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.015+0000 7fb359600640 1 -- 172.21.15.131:0/858463881 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb348032440 con 0x7fb35c06f420 2024-05-26T22:50:17.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.015+0000 7fb359600640 1 -- 172.21.15.131:0/858463881 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb348032810 con 0x7fb35c06f420 2024-05-26T22:50:17.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.015+0000 7fb361800640 1 -- 172.21.15.131:0/858463881 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb35c06f420 msgr2=0x7fb35c0cfd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:17.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.015+0000 7fb361800640 1 --2- 172.21.15.131:0/858463881 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb35c06f420 0x7fb35c0cfd50 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb3480097a0 tx=0x7fb34802a180 comp rx=0 tx=0).stop 2024-05-26T22:50:17.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.016+0000 7fb361800640 1 -- 172.21.15.131:0/858463881 shutdown_connections 2024-05-26T22:50:17.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.016+0000 7fb361800640 1 --2- 172.21.15.131:0/858463881 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb35c06f420 0x7fb35c0cfd50 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:17.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.016+0000 7fb361800640 1 --2- 172.21.15.131:0/858463881 >> v2:172.21.15.187:3300/0 conn(0x7fb35c0b5450 0x7fb35c0b43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:17.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.016+0000 7fb361800640 1 -- 172.21.15.131:0/858463881 >> 172.21.15.131:0/858463881 conn(0x7fb35c06bc90 msgr2=0x7fb35c0b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:17.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.016+0000 7fb361800640 1 -- 172.21.15.131:0/858463881 shutdown_connections 2024-05-26T22:50:17.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.016+0000 7fb361800640 1 -- 172.21.15.131:0/858463881 wait complete. 2024-05-26T22:50:17.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.017+0000 7fb361800640 1 Processor -- start 2024-05-26T22:50:17.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.018+0000 7fb361800640 1 -- start start 2024-05-26T22:50:17.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.019+0000 7fb361800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb35c06f420 0x7fb35c1f7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:17.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.019+0000 7fb361800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb35c0b5450 0x7fb35c0c7d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:17.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.019+0000 7fb361800640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb35c0cf590 0x7fb35c1f4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:17.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.019+0000 7fb361800640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fb35c0b5830 con 0x7fb35c06f420 2024-05-26T22:50:17.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.019+0000 7fb361800640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb35c160a20 con 0x7fb35c0cf590 2024-05-26T22:50:17.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.019+0000 7fb361800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb35c15fde0 con 0x7fb35c0b5450 2024-05-26T22:50:17.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.019+0000 7fb35aa00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb35c06f420 0x7fb35c1f7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:17.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.019+0000 7fb35aa00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb35c06f420 0x7fb35c1f7250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60582/0 (socket says 172.21.15.131:60582) 2024-05-26T22:50:17.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.019+0000 7fb35aa00640 1 -- 172.21.15.131:0/4047132693 learned_addr learned my addr 172.21.15.131:0/4047132693 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:17.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.019+0000 7fb35a000640 1 --2- 172.21.15.131:0/4047132693 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb35c0b5450 0x7fb35c0c7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:17.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.019+0000 7fb35b400640 1 --2- 172.21.15.131:0/4047132693 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb35c0cf590 0x7fb35c1f4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:17.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.019+0000 7fb35aa00640 1 -- 172.21.15.131:0/4047132693 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb35c0b5450 msgr2=0x7fb35c0c7d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:17.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.019+0000 7fb35aa00640 1 --2- 172.21.15.131:0/4047132693 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb35c0b5450 0x7fb35c0c7d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:17.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.019+0000 7fb35aa00640 1 -- 172.21.15.131:0/4047132693 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb35c0cf590 msgr2=0x7fb35c1f4dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:17.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.019+0000 7fb35aa00640 1 --2- 172.21.15.131:0/4047132693 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb35c0cf590 0x7fb35c1f4dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:17.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.019+0000 7fb35aa00640 1 -- 172.21.15.131:0/4047132693 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb35c1f8120 con 0x7fb35c06f420 2024-05-26T22:50:17.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.019+0000 7fb35a000640 1 --2- 172.21.15.131:0/4047132693 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb35c0b5450 0x7fb35c0c7d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:50:17.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.020+0000 7fb35aa00640 1 --2- 172.21.15.131:0/4047132693 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb35c06f420 0x7fb35c1f7250 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fb34c00e9b0 tx=0x7fb34c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:17.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.020+0000 7fb352a00640 1 -- 172.21.15.131:0/4047132693 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb34c017070 con 0x7fb35c06f420 2024-05-26T22:50:17.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.020+0000 7fb352a00640 1 -- 172.21.15.131:0/4047132693 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb34c00c9a0 con 0x7fb35c06f420 2024-05-26T22:50:17.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.020+0000 7fb352a00640 1 -- 172.21.15.131:0/4047132693 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb34c00cd90 con 0x7fb35c06f420 2024-05-26T22:50:17.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.020+0000 7fb361800640 1 -- 172.21.15.131:0/4047132693 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb35c1f7880 con 0x7fb35c06f420 2024-05-26T22:50:17.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.020+0000 7fb361800640 1 -- 172.21.15.131:0/4047132693 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb35c1f7e40 con 0x7fb35c06f420 2024-05-26T22:50:17.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.021+0000 7fb361800640 1 -- 172.21.15.131:0/4047132693 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb35c074e90 con 0x7fb35c06f420 2024-05-26T22:50:17.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.022+0000 7fb352a00640 1 -- 172.21.15.131:0/4047132693 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb34c01a070 con 0x7fb35c06f420 2024-05-26T22:50:17.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.024+0000 7fb352a00640 1 --2- 172.21.15.131:0/4047132693 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb33c06be30 0x7fb33c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:17.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.024+0000 7fb352a00640 1 -- 172.21.15.131:0/4047132693 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5115+0+0 (secure 0 0 0) 0x7fb34c08afe0 con 0x7fb35c06f420 2024-05-26T22:50:17.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.024+0000 7fb35a000640 1 --2- 172.21.15.131:0/4047132693 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb33c06be30 0x7fb33c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:17.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.025+0000 7fb35a000640 1 --2- 172.21.15.131:0/4047132693 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb33c06be30 0x7fb33c06e2f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb348034000 tx=0x7fb34802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:17.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.028+0000 7fb352a00640 1 -- 172.21.15.131:0/4047132693 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fb34c059c10 con 0x7fb35c06f420 2024-05-26T22:50:17.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.226+0000 7fb361800640 1 -- 172.21.15.131:0/4047132693 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fb35c0695c0 con 0x7fb35c06f420 2024-05-26T22:50:17.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.232+0000 7fb352a00640 1 -- 172.21.15.131:0/4047132693 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v18) ==== 104+0+0 (secure 0 0 0) 0x7fb34c059510 con 0x7fb35c06f420 2024-05-26T22:50:17.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.235+0000 7fb361800640 1 -- 172.21.15.131:0/4047132693 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb33c06be30 msgr2=0x7fb33c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:17.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.235+0000 7fb361800640 1 --2- 172.21.15.131:0/4047132693 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb33c06be30 0x7fb33c06e2f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb348034000 tx=0x7fb34802e000 comp rx=0 tx=0).stop 2024-05-26T22:50:17.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.235+0000 7fb361800640 1 -- 172.21.15.131:0/4047132693 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb35c06f420 msgr2=0x7fb35c1f7250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:17.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.235+0000 7fb361800640 1 --2- 172.21.15.131:0/4047132693 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb35c06f420 0x7fb35c1f7250 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fb34c00e9b0 tx=0x7fb34c00ee70 comp rx=0 tx=0).stop 2024-05-26T22:50:17.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.235+0000 7fb361800640 1 -- 172.21.15.131:0/4047132693 shutdown_connections 2024-05-26T22:50:17.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.235+0000 7fb361800640 1 --2- 172.21.15.131:0/4047132693 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb35c0b5450 0x7fb35c0c7d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:17.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.235+0000 7fb361800640 1 --2- 172.21.15.131:0/4047132693 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb33c06be30 0x7fb33c06e2f0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:17.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.235+0000 7fb361800640 1 --2- 172.21.15.131:0/4047132693 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb35c0cf590 0x7fb35c1f4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:17.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.235+0000 7fb361800640 1 --2- 172.21.15.131:0/4047132693 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb35c06f420 0x7fb35c1f7250 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:17.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.235+0000 7fb361800640 1 -- 172.21.15.131:0/4047132693 >> 172.21.15.131:0/4047132693 conn(0x7fb35c06bc90 msgr2=0x7fb35c0cc940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:17.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.235+0000 7fb361800640 1 -- 172.21.15.131:0/4047132693 shutdown_connections 2024-05-26T22:50:17.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.235+0000 7fb361800640 1 -- 172.21.15.131:0/4047132693 wait complete. 2024-05-26T22:50:17.253 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_exports.TestEphemeralPins.test_ephemeral_pin_dist_conf_off' 2024-05-26T22:50:17.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.447+0000 7f03bf400640 1 Processor -- start 2024-05-26T22:50:17.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.448+0000 7f03bf400640 1 -- start start 2024-05-26T22:50:17.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.449+0000 7f03bf400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f03b806ee20 0x7f03b80b3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:17.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.449+0000 7f03bf400640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03b815cb20 con 0x7f03b80ad250 2024-05-26T22:50:17.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.449+0000 7f03bf400640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03b815ce00 con 0x7f03b81569d0 2024-05-26T22:50:17.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.449+0000 7f03bf400640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f03b815d0e0 con 0x7f03b806ee20 2024-05-26T22:50:17.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.449+0000 7f03b7e00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f03b80ad250 0x7f03b81562c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:46766/0 (socket says 172.21.15.131:46766) 2024-05-26T22:50:17.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.449+0000 7f03bca00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f03b806ee20 0x7f03b80b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:17.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.449+0000 7f03b7e00640 1 -- 172.21.15.131:0/2719225821 learned_addr learned my addr 172.21.15.131:0/2719225821 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:50:17.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.450+0000 7f03bca00640 1 -- 172.21.15.131:0/2719225821 >> v1:172.21.15.187:6789/0 conn(0x7f03b81569d0 legacy=0x7f03b8160010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:17.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.450+0000 7f03bca00640 1 -- 172.21.15.131:0/2719225821 >> v1:172.21.15.131:6789/0 conn(0x7f03b80ad250 legacy=0x7f03b81562c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:17.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.450+0000 7f03bca00640 1 -- 172.21.15.131:0/2719225821 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03b815d3c0 con 0x7f03b806ee20 2024-05-26T22:50:17.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.450+0000 7f03b7400640 1 -- 172.21.15.131:0/2719225821 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1873786346 0 0) 0x7f03b815cb20 con 0x7f03b80ad250 2024-05-26T22:50:17.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.451+0000 7f03bca00640 1 --2- 172.21.15.131:0/2719225821 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03b806ee20 0x7f03b80b3190 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f03ac0097a0 tx=0x7f03ac02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f27839bd9c507420 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:17.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.451+0000 7f03b7400640 1 -- 172.21.15.131:0/2719225821 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03ac037070 con 0x7f03b806ee20 2024-05-26T22:50:17.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.452+0000 7f03b7400640 1 -- 172.21.15.131:0/2719225821 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03ac02ae50 con 0x7f03b806ee20 2024-05-26T22:50:17.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.452+0000 7f03b7400640 1 -- 172.21.15.131:0/2719225821 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03ac032660 con 0x7f03b806ee20 2024-05-26T22:50:17.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.452+0000 7f03bf400640 1 -- 172.21.15.131:0/2719225821 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03b806ee20 msgr2=0x7f03b80b3190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:17.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.452+0000 7f03bf400640 1 --2- 172.21.15.131:0/2719225821 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03b806ee20 0x7f03b80b3190 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f03ac0097a0 tx=0x7f03ac02a140 comp rx=0 tx=0).stop 2024-05-26T22:50:17.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.452+0000 7f03bf400640 1 -- 172.21.15.131:0/2719225821 shutdown_connections 2024-05-26T22:50:17.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.452+0000 7f03bf400640 1 --2- 172.21.15.131:0/2719225821 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03b806ee20 0x7f03b80b3190 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:17.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.452+0000 7f03bf400640 1 -- 172.21.15.131:0/2719225821 >> 172.21.15.131:0/2719225821 conn(0x7f03b80b1e90 msgr2=0x7f03b80b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:17.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.452+0000 7f03bf400640 1 -- 172.21.15.131:0/2719225821 shutdown_connections 2024-05-26T22:50:17.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.452+0000 7f03bf400640 1 -- 172.21.15.131:0/2719225821 wait complete. 2024-05-26T22:50:17.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.454+0000 7f03bf400640 1 Processor -- start 2024-05-26T22:50:17.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.454+0000 7f03bf400640 1 -- start start 2024-05-26T22:50:17.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.455+0000 7f03bf400640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f03b806ee20 0x7f03b81f2a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:17.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.455+0000 7f03bf400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03b80ad250 0x7f03b81e5b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:17.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.455+0000 7f03bf400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f03b81569d0 0x7f03b809a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:17.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.455+0000 7f03bf400640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f03b815c900 con 0x7f03b80ad250 2024-05-26T22:50:17.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.455+0000 7f03bf400640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f03b815c780 con 0x7f03b806ee20 2024-05-26T22:50:17.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.455+0000 7f03bf400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f03b806f200 con 0x7f03b81569d0 2024-05-26T22:50:17.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.455+0000 7f03b7e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03b80ad250 0x7f03b81e5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:17.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.455+0000 7f03b7e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03b80ad250 0x7f03b81e5b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60612/0 (socket says 172.21.15.131:60612) 2024-05-26T22:50:17.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.455+0000 7f03b7e00640 1 -- 172.21.15.131:0/566246171 learned_addr learned my addr 172.21.15.131:0/566246171 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:17.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.456+0000 7f03b7e00640 1 -- 172.21.15.131:0/566246171 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f03b81569d0 msgr2=0x7f03b809a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:17.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.456+0000 7f03b7e00640 1 --2- 172.21.15.131:0/566246171 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f03b81569d0 0x7f03b809a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:17.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.456+0000 7f03bca00640 1 --2- 172.21.15.131:0/566246171 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f03b806ee20 0x7f03b81f2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:17.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.456+0000 7f03b7e00640 1 -- 172.21.15.131:0/566246171 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f03b806ee20 msgr2=0x7f03b81f2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:17.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.456+0000 7f03b7e00640 1 --2- 172.21.15.131:0/566246171 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f03b806ee20 0x7f03b81f2a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:17.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.456+0000 7f03b7e00640 1 -- 172.21.15.131:0/566246171 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03b81e6290 con 0x7f03b80ad250 2024-05-26T22:50:17.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.456+0000 7f03b7e00640 1 --2- 172.21.15.131:0/566246171 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03b80ad250 0x7f03b81e5b20 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f03a400c4e0 tx=0x7f03a400c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:17.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.456+0000 7f03b4c00640 1 -- 172.21.15.131:0/566246171 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03a4018070 con 0x7f03b80ad250 2024-05-26T22:50:17.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.457+0000 7f03b4c00640 1 -- 172.21.15.131:0/566246171 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03a4007500 con 0x7f03b80ad250 2024-05-26T22:50:17.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.457+0000 7f03b4c00640 1 -- 172.21.15.131:0/566246171 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03a4004a20 con 0x7f03b80ad250 2024-05-26T22:50:17.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.457+0000 7f03bf400640 1 -- 172.21.15.131:0/566246171 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03b81e6570 con 0x7f03b80ad250 2024-05-26T22:50:17.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.457+0000 7f03bf400640 1 -- 172.21.15.131:0/566246171 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03b809cf70 con 0x7f03b80ad250 2024-05-26T22:50:17.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.458+0000 7f03bf400640 1 -- 172.21.15.131:0/566246171 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03b80ae890 con 0x7f03b80ad250 2024-05-26T22:50:17.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.459+0000 7f03b4c00640 1 -- 172.21.15.131:0/566246171 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f03a401b070 con 0x7f03b80ad250 2024-05-26T22:50:17.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.460+0000 7f03b4c00640 1 --2- 172.21.15.131:0/566246171 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f039806be30 0x7f039806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:17.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.460+0000 7f03b4c00640 1 -- 172.21.15.131:0/566246171 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5115+0+0 (secure 0 0 0) 0x7f03a4089ff0 con 0x7f03b80ad250 2024-05-26T22:50:17.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.460+0000 7f03bca00640 1 --2- 172.21.15.131:0/566246171 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f039806be30 0x7f039806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:17.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.460+0000 7f03bca00640 1 --2- 172.21.15.131:0/566246171 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f039806be30 0x7f039806e2f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f03ac034000 tx=0x7f03ac02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:17.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.465+0000 7f03b4c00640 1 -- 172.21.15.131:0/566246171 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f03b80ae890 con 0x7f03b80ad250 2024-05-26T22:50:17.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:17.732+0000 7f03bf400640 1 -- 172.21.15.131:0/566246171 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_exports.TestEphemeralPins.test_ephemeral_pin_dist_conf_off"]} v 0) -- 0x7f03b80694e0 con 0x7f03b80ad250 2024-05-26T22:50:18.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:18.477+0000 7f03b4c00640 1 -- 172.21.15.131:0/566246171 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_exports.TestEphemeralPins.test_ephemeral_pin_dist_conf_off"]}]=0 v122) ==== 155+0+0 (secure 0 0 0) 0x7f03a4058c70 con 0x7f03b80ad250 2024-05-26T22:50:18.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:18.481+0000 7f03bf400640 1 -- 172.21.15.131:0/566246171 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f039806be30 msgr2=0x7f039806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:18.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:18.481+0000 7f03bf400640 1 --2- 172.21.15.131:0/566246171 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f039806be30 0x7f039806e2f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f03ac034000 tx=0x7f03ac02e000 comp rx=0 tx=0).stop 2024-05-26T22:50:18.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:18.481+0000 7f03bf400640 1 -- 172.21.15.131:0/566246171 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03b80ad250 msgr2=0x7f03b81e5b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:18.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:18.481+0000 7f03bf400640 1 --2- 172.21.15.131:0/566246171 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03b80ad250 0x7f03b81e5b20 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f03a400c4e0 tx=0x7f03a400c9a0 comp rx=0 tx=0).stop 2024-05-26T22:50:18.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:18.481+0000 7f03bf400640 1 -- 172.21.15.131:0/566246171 shutdown_connections 2024-05-26T22:50:18.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:18.481+0000 7f03bf400640 1 --2- 172.21.15.131:0/566246171 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f03b81569d0 0x7f03b809a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:18.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:18.481+0000 7f03bf400640 1 --2- 172.21.15.131:0/566246171 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f039806be30 0x7f039806e2f0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:18.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:18.481+0000 7f03bf400640 1 --2- 172.21.15.131:0/566246171 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03b80ad250 0x7f03b81e5b20 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:18.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:18.481+0000 7f03bf400640 1 --2- 172.21.15.131:0/566246171 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f03b806ee20 0x7f03b81f2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:18.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:18.481+0000 7f03bf400640 1 -- 172.21.15.131:0/566246171 >> 172.21.15.131:0/566246171 conn(0x7f03b80b1e90 msgr2=0x7f03b80b23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:18.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:18.481+0000 7f03bf400640 1 -- 172.21.15.131:0/566246171 shutdown_connections 2024-05-26T22:50:18.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:18.481+0000 7f03bf400640 1 -- 172.21.15.131:0/566246171 wait complete. 2024-05-26T22:50:18.502 INFO:tasks.cephfs_test_runner:That turning off ephemeral distributed pin config prevents distribution. ... ok 2024-05-26T22:50:18.502 INFO:tasks.cephfs_test_runner:Starting test: test_ephemeral_pin_dist_failover (tasks.cephfs.test_exports.TestEphemeralPins) That MDS failover does not cause unnecessary migrations. 2024-05-26T22:50:18.502 INFO:tasks.cephfs_test_runner:test_ephemeral_pin_dist_failover (tasks.cephfs.test_exports.TestEphemeralPins) 2024-05-26T22:50:18.503 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-26T22:50:18.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:18.695+0000 7f257ec00640 1 Processor -- start 2024-05-26T22:50:18.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:18.695+0000 7f257ec00640 1 -- start start 2024-05-26T22:50:18.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:18.696+0000 7f257ec00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f25781569d0 0x7f25780b3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:18.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:18.696+0000 7f257ec00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f257815cb20 con 0x7f25780ad250 2024-05-26T22:50:18.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:18.696+0000 7f257ec00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f257815ce00 con 0x7f257806ee20 2024-05-26T22:50:18.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:18.696+0000 7f257ec00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f257815d0e0 con 0x7f25781569d0 2024-05-26T22:50:18.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:18.697+0000 7f257cc00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f25781569d0 0x7f25780b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:18.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:18.697+0000 7f256f400640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f25780ad250 0x7f2578160010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:46768/0 (socket says 172.21.15.131:46768) 2024-05-26T22:50:18.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:18.697+0000 7f256f400640 1 -- 172.21.15.131:0/3693185438 learned_addr learned my addr 172.21.15.131:0/3693185438 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:50:18.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:18.697+0000 7f257cc00640 1 -- 172.21.15.131:0/3693185438 >> v1:172.21.15.187:6789/0 conn(0x7f257806ee20 legacy=0x7f25781562c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:18.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:18.697+0000 7f257cc00640 1 -- 172.21.15.131:0/3693185438 >> v1:172.21.15.131:6789/0 conn(0x7f25780ad250 legacy=0x7f2578160010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:18.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:18.697+0000 7f257cc00640 1 -- 172.21.15.131:0/3693185438 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f257815d3c0 con 0x7f25781569d0 2024-05-26T22:50:18.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:18.698+0000 7f257cc00640 1 --2- 172.21.15.131:0/3693185438 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f25781569d0 0x7f25780b3190 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f2570009d90 tx=0x7f257002a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f5738a1b131a6a1 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:18.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:18.698+0000 7f2577400640 1 -- 172.21.15.131:0/3693185438 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2570031600 con 0x7f25781569d0 2024-05-26T22:50:18.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:18.698+0000 7f2577400640 1 -- 172.21.15.131:0/3693185438 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25700317a0 con 0x7f25781569d0 2024-05-26T22:50:18.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:18.699+0000 7f2577400640 1 -- 172.21.15.131:0/3693185438 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2570031b70 con 0x7f25781569d0 2024-05-26T22:50:18.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:18.699+0000 7f257ec00640 1 -- 172.21.15.131:0/3693185438 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f25781569d0 msgr2=0x7f25780b3190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:18.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:18.699+0000 7f257ec00640 1 --2- 172.21.15.131:0/3693185438 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f25781569d0 0x7f25780b3190 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f2570009d90 tx=0x7f257002a610 comp rx=0 tx=0).stop 2024-05-26T22:50:18.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:18.699+0000 7f257ec00640 1 -- 172.21.15.131:0/3693185438 shutdown_connections 2024-05-26T22:50:18.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:18.699+0000 7f257ec00640 1 --2- 172.21.15.131:0/3693185438 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f25781569d0 0x7f25780b3190 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:18.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:18.699+0000 7f257ec00640 1 -- 172.21.15.131:0/3693185438 >> 172.21.15.131:0/3693185438 conn(0x7f25780b1e90 msgr2=0x7f25780b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:18.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:18.699+0000 7f257ec00640 1 -- 172.21.15.131:0/3693185438 shutdown_connections 2024-05-26T22:50:18.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:18.699+0000 7f257ec00640 1 -- 172.21.15.131:0/3693185438 wait complete. 2024-05-26T22:50:18.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:18.701+0000 7f257ec00640 1 Processor -- start 2024-05-26T22:50:18.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:18.701+0000 7f257ec00640 1 -- start start 2024-05-26T22:50:18.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:18.702+0000 7f257ec00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f257806ee20 0x7f25781e6c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:18.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:18.702+0000 7f257ec00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f25780ad250 0x7f25781475d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:18.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:18.702+0000 7f257ec00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f25781569d0 0x7f25781e74c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:18.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:18.702+0000 7f257ec00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f257815c900 con 0x7f25781569d0 2024-05-26T22:50:18.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:18.702+0000 7f257ec00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f257815c780 con 0x7f25780ad250 2024-05-26T22:50:18.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:18.702+0000 7f257ec00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f257806f200 con 0x7f257806ee20 2024-05-26T22:50:18.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:18.702+0000 7f257cc00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f25781569d0 0x7f25781e74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:18.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:18.702+0000 7f257cc00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f25781569d0 0x7f25781e74c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60630/0 (socket says 172.21.15.131:60630) 2024-05-26T22:50:18.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:18.702+0000 7f257cc00640 1 -- 172.21.15.131:0/1061579273 learned_addr learned my addr 172.21.15.131:0/1061579273 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:18.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:18.703+0000 7f2577e00640 1 --2- 172.21.15.131:0/1061579273 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f257806ee20 0x7f25781e6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:18.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:18.703+0000 7f256f400640 1 --2- 172.21.15.131:0/1061579273 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f25780ad250 0x7f25781475d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:18.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:18.703+0000 7f257cc00640 1 -- 172.21.15.131:0/1061579273 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f257806ee20 msgr2=0x7f25781e6c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:18.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:18.703+0000 7f257cc00640 1 --2- 172.21.15.131:0/1061579273 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f257806ee20 0x7f25781e6c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:18.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:18.703+0000 7f257cc00640 1 -- 172.21.15.131:0/1061579273 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f25780ad250 msgr2=0x7f25781475d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:18.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:18.703+0000 7f257cc00640 1 --2- 172.21.15.131:0/1061579273 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f25780ad250 0x7f25781475d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:18.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:18.703+0000 7f257cc00640 1 -- 172.21.15.131:0/1061579273 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25781f8bd0 con 0x7f25781569d0 2024-05-26T22:50:18.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:18.703+0000 7f2577e00640 1 --2- 172.21.15.131:0/1061579273 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f257806ee20 0x7f25781e6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:50:18.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:18.703+0000 7f257cc00640 1 --2- 172.21.15.131:0/1061579273 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f25781569d0 0x7f25781e74c0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f2570009eb0 tx=0x7f2570004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:18.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:18.703+0000 7f2574c00640 1 -- 172.21.15.131:0/1061579273 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2570040070 con 0x7f25781569d0 2024-05-26T22:50:18.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:18.704+0000 7f2574c00640 1 -- 172.21.15.131:0/1061579273 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2570030b60 con 0x7f25781569d0 2024-05-26T22:50:18.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:18.704+0000 7f257ec00640 1 -- 172.21.15.131:0/1061579273 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25781e5ea0 con 0x7f25781569d0 2024-05-26T22:50:18.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:18.704+0000 7f2574c00640 1 -- 172.21.15.131:0/1061579273 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f257003c420 con 0x7f25781569d0 2024-05-26T22:50:18.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:18.704+0000 7f257ec00640 1 -- 172.21.15.131:0/1061579273 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25781e6460 con 0x7f25781569d0 2024-05-26T22:50:18.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:18.705+0000 7f257ec00640 1 -- 172.21.15.131:0/1061579273 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25780694e0 con 0x7f25781569d0 2024-05-26T22:50:18.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:18.706+0000 7f2574c00640 1 -- 172.21.15.131:0/1061579273 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2570039040 con 0x7f25781569d0 2024-05-26T22:50:18.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:18.706+0000 7f2574c00640 1 --2- 172.21.15.131:0/1061579273 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f255406be30 0x7f255406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:18.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:18.707+0000 7f2574c00640 1 -- 172.21.15.131:0/1061579273 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5115+0+0 (secure 0 0 0) 0x7f25700abf90 con 0x7f25781569d0 2024-05-26T22:50:18.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:18.707+0000 7f2577e00640 1 --2- 172.21.15.131:0/1061579273 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f255406be30 0x7f255406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:18.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:18.707+0000 7f2577e00640 1 --2- 172.21.15.131:0/1061579273 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f255406be30 0x7f255406e2f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f2568003eb0 tx=0x7f2568009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:18.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:18.711+0000 7f2574c00640 1 -- 172.21.15.131:0/1061579273 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f257007a5f0 con 0x7f25781569d0 2024-05-26T22:50:18.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:18.912+0000 7f257ec00640 1 -- 172.21.15.131:0/1061579273 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f257806e160 con 0x7f25781569d0 2024-05-26T22:50:18.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:18.914+0000 7f2574c00640 1 -- 172.21.15.131:0/1061579273 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) ==== 74+0+13310 (secure 0 0 0) 0x7f257007e8f0 con 0x7f25781569d0 2024-05-26T22:50:18.914 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:50:18.915 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":36,"fsid":"46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f","created":"2024-05-26T22:47:57.288905+0000","modified":"2024-05-26T22:50:15.449164+0000","last_up_change":"2024-05-26T22:48:09.602207+0000","last_in_change":"2024-05-26T22:48:00.687341+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-26T22:48:10.463339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"814eaf62-a18d-494b-8e27-fd8fb88adea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6801","nonce":2350940051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6803","nonce":2350940051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6807","nonce":2350940051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6805","nonce":2350940051}]},"public_addr":"172.21.15.131:6801/2350940051","cluster_addr":"172.21.15.131:6803/2350940051","heartbeat_back_addr":"172.21.15.131:6807/2350940051","heartbeat_front_addr":"172.21.15.131:6805/2350940051","state":["exists","up"]},{"osd":1,"uuid":"42e66287-f6c6-4e26-af3c-6dfaa3536d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6825","nonce":1525528247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6827","nonce":1525528247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6831","nonce":1525528247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6829","nonce":1525528247}]},"public_addr":"172.21.15.131:6825/1525528247","cluster_addr":"172.21.15.131:6827/1525528247","heartbeat_back_addr":"172.21.15.131:6831/1525528247","heartbeat_front_addr":"172.21.15.131:6829/1525528247","state":["exists","up"]},{"osd":2,"uuid":"0da34673-04a2-4fd0-b57a-0ff2b679b043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6817","nonce":3627842210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6819","nonce":3627842210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6823","nonce":3627842210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6821","nonce":3627842210}]},"public_addr":"172.21.15.131:6817/3627842210","cluster_addr":"172.21.15.131:6819/3627842210","heartbeat_back_addr":"172.21.15.131:6823/3627842210","heartbeat_front_addr":"172.21.15.131:6821/3627842210","state":["exists","up"]},{"osd":3,"uuid":"cf117a27-d6b2-43d3-ae52-b47988bf8eed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6809","nonce":350790591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6811","nonce":350790591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6815","nonce":350790591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6813","nonce":350790591}]},"public_addr":"172.21.15.131:6809/350790591","cluster_addr":"172.21.15.131:6811/350790591","heartbeat_back_addr":"172.21.15.131:6815/350790591","heartbeat_front_addr":"172.21.15.131:6813/350790591","state":["exists","up"]},{"osd":4,"uuid":"a092ccfc-be89-4a84-88d5-0f781fa715eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6816","nonce":244560995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6819","nonce":244560995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6823","nonce":244560995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6821","nonce":244560995}]},"public_addr":"172.21.15.187:6816/244560995","cluster_addr":"172.21.15.187:6819/244560995","heartbeat_back_addr":"172.21.15.187:6823/244560995","heartbeat_front_addr":"172.21.15.187:6821/244560995","state":["exists","up"]},{"osd":5,"uuid":"7513ba69-dd00-4a35-b190-9e8cc3b2bdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6801","nonce":324619398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6803","nonce":324619398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6812","nonce":324619398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6807","nonce":324619398}]},"public_addr":"172.21.15.187:6801/324619398","cluster_addr":"172.21.15.187:6803/324619398","heartbeat_back_addr":"172.21.15.187:6812/324619398","heartbeat_front_addr":"172.21.15.187:6807/324619398","state":["exists","up"]},{"osd":6,"uuid":"50081810-c093-4d2f-a4d7-b7c695c5400d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6806","nonce":1391347786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6809","nonce":1391347786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6818","nonce":1391347786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6813","nonce":1391347786}]},"public_addr":"172.21.15.187:6806/1391347786","cluster_addr":"172.21.15.187:6809/1391347786","heartbeat_back_addr":"172.21.15.187:6818/1391347786","heartbeat_front_addr":"172.21.15.187:6813/1391347786","state":["exists","up"]},{"osd":7,"uuid":"775dae05-54f5-4ad4-bbce-92863c6ed2ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6825","nonce":1220065622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6827","nonce":1220065622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6831","nonce":1220065622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6829","nonce":1220065622}]},"public_addr":"172.21.15.187:6825/1220065622","cluster_addr":"172.21.15.187:6827/1220065622","heartbeat_back_addr":"172.21.15.187:6831/1220065622","heartbeat_front_addr":"172.21.15.187:6829/1220065622","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:05.983520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.299516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.067323+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.307748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.018742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.258775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.010912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.295355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.187:6833/3274403209":"2024-05-27T22:50:12.437645+0000","172.21.15.187:6832/3274403209":"2024-05-27T22:50:12.437645+0000","172.21.15.131:6834/4032294430":"2024-05-27T22:50:12.437614+0000","172.21.15.131:6837/2062704413":"2024-05-27T22:50:12.437661+0000","172.21.15.131:6836/4032294430":"2024-05-27T22:50:12.437614+0000","172.21.15.131:6835/2062704413":"2024-05-27T22:50:12.437661+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-26T22:50:18.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:18.917+0000 7f257ec00640 1 -- 172.21.15.131:0/1061579273 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f255406be30 msgr2=0x7f255406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:18.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:18.917+0000 7f257ec00640 1 --2- 172.21.15.131:0/1061579273 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f255406be30 0x7f255406e2f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f2568003eb0 tx=0x7f2568009040 comp rx=0 tx=0).stop 2024-05-26T22:50:18.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:18.917+0000 7f257ec00640 1 -- 172.21.15.131:0/1061579273 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f25781569d0 msgr2=0x7f25781e74c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:18.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:18.917+0000 7f257ec00640 1 --2- 172.21.15.131:0/1061579273 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f25781569d0 0x7f25781e74c0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f2570009eb0 tx=0x7f2570004a40 comp rx=0 tx=0).stop 2024-05-26T22:50:18.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:18.918+0000 7f257ec00640 1 -- 172.21.15.131:0/1061579273 shutdown_connections 2024-05-26T22:50:18.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:18.918+0000 7f257ec00640 1 --2- 172.21.15.131:0/1061579273 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f255406be30 0x7f255406e2f0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:18.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:18.918+0000 7f257ec00640 1 --2- 172.21.15.131:0/1061579273 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f25781569d0 0x7f25781e74c0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:18.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:18.918+0000 7f257ec00640 1 --2- 172.21.15.131:0/1061579273 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f25780ad250 0x7f25781475d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:18.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:18.918+0000 7f257ec00640 1 --2- 172.21.15.131:0/1061579273 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f257806ee20 0x7f25781e6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:18.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:18.918+0000 7f257ec00640 1 -- 172.21.15.131:0/1061579273 >> 172.21.15.131:0/1061579273 conn(0x7f25780b1e90 msgr2=0x7f257815ea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:18.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:18.918+0000 7f257ec00640 1 -- 172.21.15.131:0/1061579273 shutdown_connections 2024-05-26T22:50:18.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:18.918+0000 7f257ec00640 1 -- 172.21.15.131:0/1061579273 wait complete. 2024-05-26T22:50:18.936 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-26T22:48:10.463339+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-26T22:50:18.937 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-26T22:50:19.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.140+0000 7fa39e600640 1 Processor -- start 2024-05-26T22:50:19.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.140+0000 7fa39e600640 1 -- start start 2024-05-26T22:50:19.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.142+0000 7fa39e600640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fa3980b5450 0x7fa3980b43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:19.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.142+0000 7fa39e600640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fa3980d0460 0x7fa39806d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:19.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.142+0000 7fa39e600640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3980b4900 con 0x7fa39806f420 2024-05-26T22:50:19.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.142+0000 7fa39e600640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fa398160d90 con 0x7fa3980b5450 2024-05-26T22:50:19.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.142+0000 7fa39e600640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fa398160180 con 0x7fa3980d0460 2024-05-26T22:50:19.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.142+0000 7fa397400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fa3980b5450 0x7fa3980b43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:19.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.142+0000 7fa397400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fa3980b5450 0x7fa3980b43c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60644/0 (socket says 172.21.15.131:60644) 2024-05-26T22:50:19.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.142+0000 7fa397400640 1 -- 172.21.15.131:0/1116959066 learned_addr learned my addr 172.21.15.131:0/1116959066 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:19.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.143+0000 7fa397e00640 1 --2- 172.21.15.131:0/1116959066 >> v2:172.21.15.187:3300/0 conn(0x7fa3980d0460 0x7fa39806d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:19.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.143+0000 7fa397400640 1 -- 172.21.15.131:0/1116959066 >> v2:172.21.15.187:3300/0 conn(0x7fa3980d0460 msgr2=0x7fa39806d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:19.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.143+0000 7fa397400640 1 --2- 172.21.15.131:0/1116959066 >> v2:172.21.15.187:3300/0 conn(0x7fa3980d0460 0x7fa39806d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:19.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.143+0000 7fa397400640 1 -- 172.21.15.131:0/1116959066 >> v1:172.21.15.187:6789/0 conn(0x7fa39806f420 legacy=0x7fa3980cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:19.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.143+0000 7fa397400640 1 -- 172.21.15.131:0/1116959066 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa398160460 con 0x7fa3980b5450 2024-05-26T22:50:19.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.143+0000 7fa397e00640 1 --2- 172.21.15.131:0/1116959066 >> v2:172.21.15.187:3300/0 conn(0x7fa3980d0460 0x7fa39806d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:50:19.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.144+0000 7fa397400640 1 --2- 172.21.15.131:0/1116959066 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa3980b5450 0x7fa3980b43c0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fa388009870 tx=0x7fa38802a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=205af1dfe413c66 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:19.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.144+0000 7fa396a00640 1 -- 172.21.15.131:0/1116959066 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa388037070 con 0x7fa3980b5450 2024-05-26T22:50:19.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.144+0000 7fa396a00640 1 -- 172.21.15.131:0/1116959066 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa388032440 con 0x7fa3980b5450 2024-05-26T22:50:19.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.144+0000 7fa396a00640 1 -- 172.21.15.131:0/1116959066 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa388032810 con 0x7fa3980b5450 2024-05-26T22:50:19.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.145+0000 7fa39e600640 1 -- 172.21.15.131:0/1116959066 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa3980b5450 msgr2=0x7fa3980b43c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:19.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.145+0000 7fa39e600640 1 --2- 172.21.15.131:0/1116959066 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa3980b5450 0x7fa3980b43c0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fa388009870 tx=0x7fa38802a250 comp rx=0 tx=0).stop 2024-05-26T22:50:19.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.145+0000 7fa39e600640 1 -- 172.21.15.131:0/1116959066 shutdown_connections 2024-05-26T22:50:19.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.145+0000 7fa39e600640 1 --2- 172.21.15.131:0/1116959066 >> v2:172.21.15.187:3300/0 conn(0x7fa3980d0460 0x7fa39806d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:19.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.145+0000 7fa39e600640 1 --2- 172.21.15.131:0/1116959066 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa3980b5450 0x7fa3980b43c0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:19.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.145+0000 7fa39e600640 1 -- 172.21.15.131:0/1116959066 >> 172.21.15.131:0/1116959066 conn(0x7fa39806bc90 msgr2=0x7fa3980b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:19.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.145+0000 7fa39e600640 1 -- 172.21.15.131:0/1116959066 shutdown_connections 2024-05-26T22:50:19.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.145+0000 7fa39e600640 1 -- 172.21.15.131:0/1116959066 wait complete. 2024-05-26T22:50:19.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.147+0000 7fa39e600640 1 Processor -- start 2024-05-26T22:50:19.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.148+0000 7fa39e600640 1 -- start start 2024-05-26T22:50:19.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.149+0000 7fa39e600640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa39806f420 0x7fa3981492f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:19.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.149+0000 7fa39e600640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa3980b5450 0x7fa39815d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:19.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.149+0000 7fa39e600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa3980d0460 0x7fa398158420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:19.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.149+0000 7fa39e600640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa3980b5830 con 0x7fa3980d0460 2024-05-26T22:50:19.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.149+0000 7fa39e600640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa398160a20 con 0x7fa3980b5450 2024-05-26T22:50:19.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.149+0000 7fa39e600640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa39815fde0 con 0x7fa39806f420 2024-05-26T22:50:19.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.149+0000 7fa397e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa3980d0460 0x7fa398158420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:19.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.149+0000 7fa397e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa3980d0460 0x7fa398158420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60656/0 (socket says 172.21.15.131:60656) 2024-05-26T22:50:19.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.149+0000 7fa397e00640 1 -- 172.21.15.131:0/784086809 learned_addr learned my addr 172.21.15.131:0/784086809 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:19.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.149+0000 7fa397400640 1 --2- 172.21.15.131:0/784086809 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa39806f420 0x7fa3981492f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:19.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.149+0000 7fa38fe00640 1 --2- 172.21.15.131:0/784086809 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa3980b5450 0x7fa39815d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:19.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.150+0000 7fa397e00640 1 -- 172.21.15.131:0/784086809 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa39806f420 msgr2=0x7fa3981492f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:19.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.150+0000 7fa397e00640 1 --2- 172.21.15.131:0/784086809 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa39806f420 0x7fa3981492f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:19.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.150+0000 7fa397e00640 1 -- 172.21.15.131:0/784086809 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa3980b5450 msgr2=0x7fa39815d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:19.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.150+0000 7fa397e00640 1 --2- 172.21.15.131:0/784086809 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa3980b5450 0x7fa39815d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:19.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.150+0000 7fa397e00640 1 -- 172.21.15.131:0/784086809 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa398149830 con 0x7fa3980d0460 2024-05-26T22:50:19.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.150+0000 7fa397e00640 1 --2- 172.21.15.131:0/784086809 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa3980d0460 0x7fa398158420 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fa39000cef0 tx=0x7fa39000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:19.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.150+0000 7fa38f400640 1 -- 172.21.15.131:0/784086809 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa39001a070 con 0x7fa3980d0460 2024-05-26T22:50:19.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.150+0000 7fa38f400640 1 -- 172.21.15.131:0/784086809 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa39000d860 con 0x7fa3980d0460 2024-05-26T22:50:19.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.150+0000 7fa39e600640 1 -- 172.21.15.131:0/784086809 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa39815e1f0 con 0x7fa3980d0460 2024-05-26T22:50:19.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.150+0000 7fa38f400640 1 -- 172.21.15.131:0/784086809 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa39000dc50 con 0x7fa3980d0460 2024-05-26T22:50:19.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.151+0000 7fa39e600640 1 -- 172.21.15.131:0/784086809 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa39815e7b0 con 0x7fa3980d0460 2024-05-26T22:50:19.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.152+0000 7fa39e600640 1 -- 172.21.15.131:0/784086809 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3980b1bf0 con 0x7fa3980d0460 2024-05-26T22:50:19.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.153+0000 7fa38f400640 1 -- 172.21.15.131:0/784086809 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa39001d070 con 0x7fa3980d0460 2024-05-26T22:50:19.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.154+0000 7fa38f400640 1 --2- 172.21.15.131:0/784086809 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa37406be30 0x7fa37406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:19.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.154+0000 7fa38f400640 1 -- 172.21.15.131:0/784086809 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5115+0+0 (secure 0 0 0) 0x7fa39008b4f0 con 0x7fa3980d0460 2024-05-26T22:50:19.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.154+0000 7fa397400640 1 --2- 172.21.15.131:0/784086809 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa37406be30 0x7fa37406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:19.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.155+0000 7fa397400640 1 --2- 172.21.15.131:0/784086809 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa37406be30 0x7fa37406e2f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa388034000 tx=0x7fa38802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:19.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.158+0000 7fa38f400640 1 -- 172.21.15.131:0/784086809 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fa3980b1bf0 con 0x7fa3980d0460 2024-05-26T22:50:19.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.355+0000 7fa39e600640 1 -- 172.21.15.131:0/784086809 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa3980cee90 con 0x7fa3980d0460 2024-05-26T22:50:19.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.356+0000 7fa38f400640 1 -- 172.21.15.131:0/784086809 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v36) ==== 93+0+10 (secure 0 0 0) 0x7fa39005a1b0 con 0x7fa3980d0460 2024-05-26T22:50:19.356 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2024-05-26T22:50:19.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.358+0000 7fa39e600640 1 -- 172.21.15.131:0/784086809 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa37406be30 msgr2=0x7fa37406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:19.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.358+0000 7fa39e600640 1 --2- 172.21.15.131:0/784086809 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa37406be30 0x7fa37406e2f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa388034000 tx=0x7fa38802e000 comp rx=0 tx=0).stop 2024-05-26T22:50:19.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.358+0000 7fa39e600640 1 -- 172.21.15.131:0/784086809 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa3980d0460 msgr2=0x7fa398158420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:19.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.358+0000 7fa39e600640 1 --2- 172.21.15.131:0/784086809 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa3980d0460 0x7fa398158420 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fa39000cef0 tx=0x7fa39000a710 comp rx=0 tx=0).stop 2024-05-26T22:50:19.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.358+0000 7fa39e600640 1 -- 172.21.15.131:0/784086809 shutdown_connections 2024-05-26T22:50:19.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.358+0000 7fa39e600640 1 --2- 172.21.15.131:0/784086809 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa37406be30 0x7fa37406e2f0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:19.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.358+0000 7fa39e600640 1 --2- 172.21.15.131:0/784086809 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa3980d0460 0x7fa398158420 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:19.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.358+0000 7fa39e600640 1 --2- 172.21.15.131:0/784086809 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa3980b5450 0x7fa39815d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:19.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.358+0000 7fa39e600640 1 --2- 172.21.15.131:0/784086809 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa39806f420 0x7fa3981492f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:19.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.358+0000 7fa39e600640 1 -- 172.21.15.131:0/784086809 >> 172.21.15.131:0/784086809 conn(0x7fa39806bc90 msgr2=0x7fa3980b1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:19.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.359+0000 7fa39e600640 1 -- 172.21.15.131:0/784086809 shutdown_connections 2024-05-26T22:50:19.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.359+0000 7fa39e600640 1 -- 172.21.15.131:0/784086809 wait complete. 2024-05-26T22:50:19.377 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_exports.TestEphemeralPins.test_ephemeral_pin_dist_failover' 2024-05-26T22:50:19.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.579+0000 7f72f2400640 1 Processor -- start 2024-05-26T22:50:19.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.580+0000 7f72f2400640 1 -- start start 2024-05-26T22:50:19.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.581+0000 7f72f2400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f72ec06dcc0 0x7f72ec0b3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:19.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.581+0000 7f72f2400640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72ec0b3e50 con 0x7f72ec158c00 2024-05-26T22:50:19.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.581+0000 7f72f2400640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72ec0b4130 con 0x7f72ec0b36d0 2024-05-26T22:50:19.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.581+0000 7f72f2400640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f72ec0b4410 con 0x7f72ec06dcc0 2024-05-26T22:50:19.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.581+0000 7f72eb400640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f72ec158c00 0x7f72ec06d5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:46774/0 (socket says 172.21.15.131:46774) 2024-05-26T22:50:19.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.581+0000 7f72eb400640 1 -- 172.21.15.131:0/1071218729 learned_addr learned my addr 172.21.15.131:0/1071218729 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:50:19.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.582+0000 7f72e3e00640 1 --2- 172.21.15.131:0/1071218729 >> v2:172.21.15.131:3300/0 conn(0x7f72ec06dcc0 0x7f72ec0b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:19.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.582+0000 7f72eaa00640 1 -- 172.21.15.131:0/1071218729 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 425478612 0 0) 0x7f72ec0b3e50 con 0x7f72ec158c00 2024-05-26T22:50:19.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.582+0000 7f72eaa00640 1 -- 172.21.15.131:0/1071218729 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f72d0003520 con 0x7f72ec158c00 2024-05-26T22:50:19.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.582+0000 7f72eaa00640 1 -- 172.21.15.131:0/1071218729 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2719329948 0 0) 0x7f72d0003520 con 0x7f72ec158c00 2024-05-26T22:50:19.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.582+0000 7f72eaa00640 1 -- 172.21.15.131:0/1071218729 >> v2:172.21.15.131:3300/0 conn(0x7f72ec06dcc0 msgr2=0x7f72ec0b3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:19.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.582+0000 7f72eaa00640 1 --2- 172.21.15.131:0/1071218729 >> v2:172.21.15.131:3300/0 conn(0x7f72ec06dcc0 0x7f72ec0b3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:19.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.582+0000 7f72eaa00640 1 -- 172.21.15.131:0/1071218729 >> v1:172.21.15.187:6789/0 conn(0x7f72ec0b36d0 legacy=0x7f72ec15fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:19.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.582+0000 7f72eaa00640 1 -- 172.21.15.131:0/1071218729 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72ec0b46f0 con 0x7f72ec158c00 2024-05-26T22:50:19.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.582+0000 7f72eaa00640 1 -- 172.21.15.131:0/1071218729 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7f72dc001670 con 0x7f72ec158c00 2024-05-26T22:50:19.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.582+0000 7f72e3e00640 1 --2- 172.21.15.131:0/1071218729 >> v2:172.21.15.131:3300/0 conn(0x7f72ec06dcc0 0x7f72ec0b3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-26T22:50:19.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.582+0000 7f72eaa00640 1 -- 172.21.15.131:0/1071218729 >> v1:172.21.15.131:6789/0 conn(0x7f72ec158c00 legacy=0x7f72ec06d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:19.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.582+0000 7f72eaa00640 1 --2- 172.21.15.131:0/1071218729 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f72d00039b0 0x7f72d0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:19.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.582+0000 7f72eaa00640 1 --2- 172.21.15.131:0/1071218729 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f72d0004800 0x7f72d0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:19.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.582+0000 7f72eaa00640 1 --2- 172.21.15.131:0/1071218729 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f72d00051c0 0x7f72d000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:19.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.582+0000 7f72eaa00640 1 -- 172.21.15.131:0/1071218729 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f72ec0b4410 con 0x7f72d00051c0 2024-05-26T22:50:19.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.582+0000 7f72eaa00640 1 -- 172.21.15.131:0/1071218729 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f72d000c160 con 0x7f72d0004800 2024-05-26T22:50:19.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.582+0000 7f72eaa00640 1 -- 172.21.15.131:0/1071218729 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f72d000c360 con 0x7f72d00039b0 2024-05-26T22:50:19.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.583+0000 7f72ebe00640 1 --2- 172.21.15.131:0/1071218729 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f72d00051c0 0x7f72d000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:19.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.583+0000 7f72ebe00640 1 -- 172.21.15.131:0/1071218729 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f72d00039b0 msgr2=0x7f72d0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:19.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.583+0000 7f72ebe00640 1 --2- 172.21.15.131:0/1071218729 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f72d00039b0 0x7f72d0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:19.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.583+0000 7f72ebe00640 1 -- 172.21.15.131:0/1071218729 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f72d0004800 msgr2=0x7f72d0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:19.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.583+0000 7f72ebe00640 1 --2- 172.21.15.131:0/1071218729 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f72d0004800 0x7f72d0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:19.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.583+0000 7f72ebe00640 1 -- 172.21.15.131:0/1071218729 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72d0003520 con 0x7f72d00051c0 2024-05-26T22:50:19.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.584+0000 7f72ebe00640 1 -- 172.21.15.131:0/1071218729 reap_dead start 2024-05-26T22:50:19.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.584+0000 7f72ebe00640 1 --2- 172.21.15.131:0/1071218729 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f72d00051c0 0x7f72d000b9c0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f72e4001e50 tx=0x7f72d0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf6d5a4dcad4d618 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:19.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.584+0000 7f72eaa00640 1 -- 172.21.15.131:0/1071218729 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72e4001a70 con 0x7f72d00051c0 2024-05-26T22:50:19.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.584+0000 7f72eaa00640 1 -- 172.21.15.131:0/1071218729 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72e4003970 con 0x7f72d00051c0 2024-05-26T22:50:19.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.584+0000 7f72eaa00640 1 -- 172.21.15.131:0/1071218729 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72e4003d60 con 0x7f72d00051c0 2024-05-26T22:50:19.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.585+0000 7f72f2400640 1 -- 172.21.15.131:0/1071218729 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f72d00051c0 msgr2=0x7f72d000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:19.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.585+0000 7f72f2400640 1 --2- 172.21.15.131:0/1071218729 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f72d00051c0 0x7f72d000b9c0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f72e4001e50 tx=0x7f72d0005830 comp rx=0 tx=0).stop 2024-05-26T22:50:19.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.585+0000 7f72f2400640 1 -- 172.21.15.131:0/1071218729 shutdown_connections 2024-05-26T22:50:19.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.585+0000 7f72f2400640 1 --2- 172.21.15.131:0/1071218729 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f72d00051c0 0x7f72d000b9c0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:19.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.585+0000 7f72f2400640 1 -- 172.21.15.131:0/1071218729 >> 172.21.15.131:0/1071218729 conn(0x7f72ec0b1e90 msgr2=0x7f72ec0b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:19.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.585+0000 7f72f2400640 1 -- 172.21.15.131:0/1071218729 shutdown_connections 2024-05-26T22:50:19.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.585+0000 7f72f2400640 1 -- 172.21.15.131:0/1071218729 wait complete. 2024-05-26T22:50:19.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.587+0000 7f72f2400640 1 Processor -- start 2024-05-26T22:50:19.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.587+0000 7f72f2400640 1 -- start start 2024-05-26T22:50:19.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.588+0000 7f72f2400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f72d00051c0 0x7f72ec159a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:19.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.588+0000 7f72f2400640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f72ec099820 0x7f72ec1f4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:19.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.588+0000 7f72f2400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f72ec15ef20 0x7f72ec1f4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:19.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.588+0000 7f72f2400640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f72ec0b3190 con 0x7f72d00051c0 2024-05-26T22:50:19.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.588+0000 7f72f2400640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f72ec0b3b60 con 0x7f72ec099820 2024-05-26T22:50:19.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.588+0000 7f72f2400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f72ec0b3310 con 0x7f72ec15ef20 2024-05-26T22:50:19.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.588+0000 7f72eb400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f72d00051c0 0x7f72ec159a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:19.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.588+0000 7f72eb400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f72d00051c0 0x7f72ec159a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60702/0 (socket says 172.21.15.131:60702) 2024-05-26T22:50:19.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.588+0000 7f72eb400640 1 -- 172.21.15.131:0/2864242941 learned_addr learned my addr 172.21.15.131:0/2864242941 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:19.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.589+0000 7f72eb400640 1 -- 172.21.15.131:0/2864242941 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f72ec15ef20 msgr2=0x7f72ec1f4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:19.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.589+0000 7f72eb400640 1 --2- 172.21.15.131:0/2864242941 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f72ec15ef20 0x7f72ec1f4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:19.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.589+0000 7f72eb400640 1 -- 172.21.15.131:0/2864242941 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f72ec099820 msgr2=0x7f72ec1f4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:19.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.589+0000 7f72eb400640 1 --2- 172.21.15.131:0/2864242941 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f72ec099820 0x7f72ec1f4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:19.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.589+0000 7f72eb400640 1 -- 172.21.15.131:0/2864242941 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72ec15a1f0 con 0x7f72d00051c0 2024-05-26T22:50:19.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.589+0000 7f72eb400640 1 --2- 172.21.15.131:0/2864242941 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f72d00051c0 0x7f72ec159a80 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f72dc00c7a0 tx=0x7f72dc00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:19.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.589+0000 7f72e3400640 1 -- 172.21.15.131:0/2864242941 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72dc018070 con 0x7f72d00051c0 2024-05-26T22:50:19.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.589+0000 7f72e3400640 1 -- 172.21.15.131:0/2864242941 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72dc007500 con 0x7f72d00051c0 2024-05-26T22:50:19.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.589+0000 7f72e3400640 1 -- 172.21.15.131:0/2864242941 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72dc004a20 con 0x7f72d00051c0 2024-05-26T22:50:19.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.589+0000 7f72f2400640 1 -- 172.21.15.131:0/2864242941 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72ec15a4d0 con 0x7f72d00051c0 2024-05-26T22:50:19.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.589+0000 7f72f2400640 1 -- 172.21.15.131:0/2864242941 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72ec15aa90 con 0x7f72d00051c0 2024-05-26T22:50:19.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.591+0000 7f72f2400640 1 -- 172.21.15.131:0/2864242941 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72ec06dae0 con 0x7f72d00051c0 2024-05-26T22:50:19.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.592+0000 7f72e3400640 1 -- 172.21.15.131:0/2864242941 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f72dc004bc0 con 0x7f72d00051c0 2024-05-26T22:50:19.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.592+0000 7f72e3400640 1 --2- 172.21.15.131:0/2864242941 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f72c806be30 0x7f72c806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:19.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.593+0000 7f72e3400640 1 -- 172.21.15.131:0/2864242941 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5115+0+0 (secure 0 0 0) 0x7f72dc08a2e0 con 0x7f72d00051c0 2024-05-26T22:50:19.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.593+0000 7f72e3e00640 1 --2- 172.21.15.131:0/2864242941 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f72c806be30 0x7f72c806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:19.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.593+0000 7f72e3e00640 1 --2- 172.21.15.131:0/2864242941 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f72c806be30 0x7f72c806e2f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f72d4006a90 tx=0x7f72d4006a20 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:19.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.597+0000 7f72e3400640 1 -- 172.21.15.131:0/2864242941 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f72ec06dae0 con 0x7f72d00051c0 2024-05-26T22:50:19.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:19.866+0000 7f72f2400640 1 -- 172.21.15.131:0/2864242941 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_exports.TestEphemeralPins.test_ephemeral_pin_dist_failover"]} v 0) -- 0x7f72ec0694e0 con 0x7f72d00051c0 2024-05-26T22:50:20.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:20.490+0000 7f72e3400640 1 -- 172.21.15.131:0/2864242941 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_exports.TestEphemeralPins.test_ephemeral_pin_dist_failover"]}]=0 v124) ==== 158+0+0 (secure 0 0 0) 0x7f72dc058ee0 con 0x7f72d00051c0 2024-05-26T22:50:20.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:20.494+0000 7f72f2400640 1 -- 172.21.15.131:0/2864242941 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f72c806be30 msgr2=0x7f72c806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:20.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:20.494+0000 7f72f2400640 1 --2- 172.21.15.131:0/2864242941 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f72c806be30 0x7f72c806e2f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f72d4006a90 tx=0x7f72d4006a20 comp rx=0 tx=0).stop 2024-05-26T22:50:20.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:20.494+0000 7f72f2400640 1 -- 172.21.15.131:0/2864242941 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f72d00051c0 msgr2=0x7f72ec159a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:20.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:20.494+0000 7f72f2400640 1 --2- 172.21.15.131:0/2864242941 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f72d00051c0 0x7f72ec159a80 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f72dc00c7a0 tx=0x7f72dc00cc60 comp rx=0 tx=0).stop 2024-05-26T22:50:20.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:20.494+0000 7f72f2400640 1 -- 172.21.15.131:0/2864242941 shutdown_connections 2024-05-26T22:50:20.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:20.494+0000 7f72f2400640 1 --2- 172.21.15.131:0/2864242941 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f72ec15ef20 0x7f72ec1f4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:20.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:20.494+0000 7f72f2400640 1 --2- 172.21.15.131:0/2864242941 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f72c806be30 0x7f72c806e2f0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:20.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:20.494+0000 7f72f2400640 1 --2- 172.21.15.131:0/2864242941 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f72ec099820 0x7f72ec1f4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:20.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:20.494+0000 7f72f2400640 1 --2- 172.21.15.131:0/2864242941 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f72d00051c0 0x7f72ec159a80 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:20.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:20.494+0000 7f72f2400640 1 -- 172.21.15.131:0/2864242941 >> 172.21.15.131:0/2864242941 conn(0x7f72ec0b1e90 msgr2=0x7f72ec0cfb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:20.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:20.494+0000 7f72f2400640 1 -- 172.21.15.131:0/2864242941 shutdown_connections 2024-05-26T22:50:20.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:20.494+0000 7f72f2400640 1 -- 172.21.15.131:0/2864242941 wait complete. 2024-05-26T22:50:20.514 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-26T22:50:20.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:20.721+0000 7f149de00640 1 Processor -- start 2024-05-26T22:50:20.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:20.721+0000 7f149de00640 1 -- start start 2024-05-26T22:50:20.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:20.722+0000 7f149de00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f149806f420 0x7f14980cfd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:20.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:20.722+0000 7f149de00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f14980cf590 0x7f14980d0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:20.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:20.722+0000 7f149de00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1498160180 con 0x7f14980b5450 2024-05-26T22:50:20.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:20.722+0000 7f149de00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f1498160d90 con 0x7f14980cf590 2024-05-26T22:50:20.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:20.722+0000 7f149de00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1498160460 con 0x7f149806f420 2024-05-26T22:50:20.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:20.723+0000 7f1496a00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f14980b5450 0x7f14980b43c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:46788/0 (socket says 172.21.15.131:46788) 2024-05-26T22:50:20.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:20.723+0000 7f1496a00640 1 -- 172.21.15.131:0/2481871657 learned_addr learned my addr 172.21.15.131:0/2481871657 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:50:20.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:20.723+0000 7f1496000640 1 -- 172.21.15.131:0/2481871657 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 520109135 0 0) 0x7f1498160180 con 0x7f14980b5450 2024-05-26T22:50:20.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:20.723+0000 7f1496000640 1 -- 172.21.15.131:0/2481871657 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f147c003520 con 0x7f14980b5450 2024-05-26T22:50:20.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:20.723+0000 7f1497400640 1 --2- 172.21.15.131:0/2481871657 >> v2:172.21.15.187:3300/0 conn(0x7f14980cf590 0x7f14980d0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:20.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:20.723+0000 7f148e000640 1 --2- 172.21.15.131:0/2481871657 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f149806f420 0x7f14980cfd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:20.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:20.724+0000 7f1496000640 1 -- 172.21.15.131:0/2481871657 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2835228405 0 0) 0x7f147c003520 con 0x7f14980b5450 2024-05-26T22:50:20.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:20.724+0000 7f1496000640 1 -- 172.21.15.131:0/2481871657 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f149806f420 msgr2=0x7f14980cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:20.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:20.724+0000 7f1496000640 1 --2- 172.21.15.131:0/2481871657 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f149806f420 0x7f14980cfd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:20.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:20.724+0000 7f1496000640 1 -- 172.21.15.131:0/2481871657 >> v2:172.21.15.187:3300/0 conn(0x7f14980cf590 msgr2=0x7f14980d0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:20.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:20.724+0000 7f1496000640 1 --2- 172.21.15.131:0/2481871657 >> v2:172.21.15.187:3300/0 conn(0x7f14980cf590 0x7f14980d0290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:20.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:20.724+0000 7f1496000640 1 -- 172.21.15.131:0/2481871657 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1498160740 con 0x7f14980b5450 2024-05-26T22:50:20.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:20.724+0000 7f1496000640 1 -- 172.21.15.131:0/2481871657 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7f1488001670 con 0x7f14980b5450 2024-05-26T22:50:20.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:20.724+0000 7f1497400640 1 --2- 172.21.15.131:0/2481871657 >> v2:172.21.15.187:3300/0 conn(0x7f14980cf590 0x7f14980d0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-26T22:50:20.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:20.724+0000 7f1496000640 1 -- 172.21.15.131:0/2481871657 >> v1:172.21.15.131:6789/0 conn(0x7f14980b5450 legacy=0x7f14980b43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:20.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:20.724+0000 7f1496000640 1 --2- 172.21.15.131:0/2481871657 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f149806f420 0x7f147c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:20.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:20.724+0000 7f1496000640 1 --2- 172.21.15.131:0/2481871657 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f147c004030 0x7f147c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:20.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:20.724+0000 7f1496000640 1 --2- 172.21.15.131:0/2481871657 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f147c004d40 0x7f147c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:20.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:20.724+0000 7f1496000640 1 -- 172.21.15.131:0/2481871657 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f1498160d90 con 0x7f147c004030 2024-05-26T22:50:20.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:20.724+0000 7f1496000640 1 -- 172.21.15.131:0/2481871657 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1498160460 con 0x7f147c004d40 2024-05-26T22:50:20.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:20.724+0000 7f1496000640 1 -- 172.21.15.131:0/2481871657 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f147c00bb20 con 0x7f149806f420 2024-05-26T22:50:20.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:20.724+0000 7f148e000640 1 --2- 172.21.15.131:0/2481871657 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f147c004030 0x7f147c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:20.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:20.724+0000 7f1496a00640 1 --2- 172.21.15.131:0/2481871657 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f149806f420 0x7f147c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:20.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:20.724+0000 7f1497400640 1 --2- 172.21.15.131:0/2481871657 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f147c004d40 0x7f147c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:20.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:20.724+0000 7f148e000640 1 -- 172.21.15.131:0/2481871657 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f149806f420 msgr2=0x7f147c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:20.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:20.724+0000 7f148e000640 1 --2- 172.21.15.131:0/2481871657 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f149806f420 0x7f147c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:20.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:20.724+0000 7f148e000640 1 -- 172.21.15.131:0/2481871657 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f147c004d40 msgr2=0x7f147c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:20.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:20.724+0000 7f148e000640 1 --2- 172.21.15.131:0/2481871657 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f147c004d40 0x7f147c0051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:20.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:20.724+0000 7f148e000640 1 -- 172.21.15.131:0/2481871657 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f147c003520 con 0x7f147c004030 2024-05-26T22:50:20.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:20.725+0000 7f148e000640 1 --2- 172.21.15.131:0/2481871657 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f147c004030 0x7f147c004800 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f148000ba30 tx=0x7f147c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76d008874c818fc9 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:20.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:20.726+0000 7f1496000640 1 -- 172.21.15.131:0/2481871657 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1480030070 con 0x7f147c004030 2024-05-26T22:50:20.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:20.726+0000 7f1496000640 1 -- 172.21.15.131:0/2481871657 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f148002ade0 con 0x7f147c004030 2024-05-26T22:50:20.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:20.726+0000 7f1496000640 1 -- 172.21.15.131:0/2481871657 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f148000b600 con 0x7f147c004030 2024-05-26T22:50:20.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:20.726+0000 7f149de00640 1 -- 172.21.15.131:0/2481871657 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f147c004030 msgr2=0x7f147c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:20.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:20.726+0000 7f149de00640 1 --2- 172.21.15.131:0/2481871657 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f147c004030 0x7f147c004800 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f148000ba30 tx=0x7f147c005830 comp rx=0 tx=0).stop 2024-05-26T22:50:20.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:20.726+0000 7f1497400640 1 -- 172.21.15.131:0/2481871657 reap_dead start 2024-05-26T22:50:20.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:20.726+0000 7f149de00640 1 -- 172.21.15.131:0/2481871657 shutdown_connections 2024-05-26T22:50:20.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:20.726+0000 7f149de00640 1 -- 172.21.15.131:0/2481871657 >> 172.21.15.131:0/2481871657 conn(0x7f149806bc90 msgr2=0x7f14980b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:20.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:20.726+0000 7f149de00640 1 -- 172.21.15.131:0/2481871657 shutdown_connections 2024-05-26T22:50:20.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:20.726+0000 7f149de00640 1 -- 172.21.15.131:0/2481871657 wait complete. 2024-05-26T22:50:20.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:20.728+0000 7f149de00640 1 Processor -- start 2024-05-26T22:50:20.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:20.728+0000 7f149de00640 1 -- start start 2024-05-26T22:50:20.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:20.729+0000 7f149de00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f147c004030 0x7f14980ccbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:20.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:20.729+0000 7f149de00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1498148d60 0x7f14981f31f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:20.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:20.729+0000 7f149de00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f14981f3730 0x7f14981f7360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:20.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:20.729+0000 7f149de00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f149815fde0 con 0x7f1498148d60 2024-05-26T22:50:20.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:20.729+0000 7f149de00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f14980b5830 con 0x7f14981f3730 2024-05-26T22:50:20.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:20.729+0000 7f149de00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f149815ff60 con 0x7f147c004030 2024-05-26T22:50:20.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:20.729+0000 7f148e000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1498148d60 0x7f14981f31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:20.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:20.729+0000 7f148e000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1498148d60 0x7f14981f31f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60726/0 (socket says 172.21.15.131:60726) 2024-05-26T22:50:20.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:20.729+0000 7f148e000640 1 -- 172.21.15.131:0/3227853726 learned_addr learned my addr 172.21.15.131:0/3227853726 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:20.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:20.730+0000 7f1496a00640 1 --2- 172.21.15.131:0/3227853726 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f147c004030 0x7f14980ccbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:20.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:20.730+0000 7f1497400640 1 --2- 172.21.15.131:0/3227853726 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f14981f3730 0x7f14981f7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:20.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:20.730+0000 7f148e000640 1 -- 172.21.15.131:0/3227853726 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f147c004030 msgr2=0x7f14980ccbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:20.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:20.730+0000 7f148e000640 1 --2- 172.21.15.131:0/3227853726 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f147c004030 0x7f14980ccbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:20.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:20.730+0000 7f148e000640 1 -- 172.21.15.131:0/3227853726 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f14981f3730 msgr2=0x7f14981f7360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:20.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:20.730+0000 7f148e000640 1 --2- 172.21.15.131:0/3227853726 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f14981f3730 0x7f14981f7360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:20.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:20.730+0000 7f148e000640 1 -- 172.21.15.131:0/3227853726 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14981f3bb0 con 0x7f1498148d60 2024-05-26T22:50:20.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:20.730+0000 7f148e000640 1 --2- 172.21.15.131:0/3227853726 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1498148d60 0x7f14981f31f0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f148002a8d0 tx=0x7f14800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:20.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:20.730+0000 7f148fe00640 1 -- 172.21.15.131:0/3227853726 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1480043030 con 0x7f1498148d60 2024-05-26T22:50:20.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:20.731+0000 7f148fe00640 1 -- 172.21.15.131:0/3227853726 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f148002abe0 con 0x7f1498148d60 2024-05-26T22:50:20.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:20.731+0000 7f148fe00640 1 -- 172.21.15.131:0/3227853726 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f148000b600 con 0x7f1498148d60 2024-05-26T22:50:20.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:20.731+0000 7f149de00640 1 -- 172.21.15.131:0/3227853726 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14981f8090 con 0x7f1498148d60 2024-05-26T22:50:20.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:20.731+0000 7f149de00640 1 -- 172.21.15.131:0/3227853726 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1498148560 con 0x7f1498148d60 2024-05-26T22:50:20.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:20.732+0000 7f149de00640 1 -- 172.21.15.131:0/3227853726 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14980695c0 con 0x7f1498148d60 2024-05-26T22:50:20.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:20.733+0000 7f148fe00640 1 -- 172.21.15.131:0/3227853726 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1480046070 con 0x7f1498148d60 2024-05-26T22:50:20.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:20.733+0000 7f148fe00640 1 --2- 172.21.15.131:0/3227853726 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f147406be30 0x7f147406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:20.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:20.734+0000 7f148fe00640 1 -- 172.21.15.131:0/3227853726 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5115+0+0 (secure 0 0 0) 0x7f14800ab620 con 0x7f1498148d60 2024-05-26T22:50:20.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:20.734+0000 7f1496a00640 1 --2- 172.21.15.131:0/3227853726 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f147406be30 0x7f147406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:20.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:20.734+0000 7f1496a00640 1 --2- 172.21.15.131:0/3227853726 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f147406be30 0x7f147406e2f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f14880057f0 tx=0x7f1488000f50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:20.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:20.738+0000 7f148fe00640 1 -- 172.21.15.131:0/3227853726 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f148007a180 con 0x7f1498148d60 2024-05-26T22:50:20.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:20.936+0000 7f149de00640 1 -- 172.21.15.131:0/3227853726 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f14980cee90 con 0x7f1498148d60 2024-05-26T22:50:20.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:20.943+0000 7f148fe00640 1 -- 172.21.15.131:0/3227853726 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v19)=0 v19) ==== 122+0+0 (secure 0 0 0) 0x7f1480079a80 con 0x7f1498148d60 2024-05-26T22:50:20.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:20.947+0000 7f149de00640 1 -- 172.21.15.131:0/3227853726 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f147406be30 msgr2=0x7f147406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:20.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:20.947+0000 7f149de00640 1 --2- 172.21.15.131:0/3227853726 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f147406be30 0x7f147406e2f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f14880057f0 tx=0x7f1488000f50 comp rx=0 tx=0).stop 2024-05-26T22:50:20.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:20.947+0000 7f149de00640 1 -- 172.21.15.131:0/3227853726 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1498148d60 msgr2=0x7f14981f31f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:20.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:20.947+0000 7f149de00640 1 --2- 172.21.15.131:0/3227853726 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1498148d60 0x7f14981f31f0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f148002a8d0 tx=0x7f14800046c0 comp rx=0 tx=0).stop 2024-05-26T22:50:20.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:20.947+0000 7f149de00640 1 -- 172.21.15.131:0/3227853726 shutdown_connections 2024-05-26T22:50:20.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:20.947+0000 7f149de00640 1 --2- 172.21.15.131:0/3227853726 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f147406be30 0x7f147406e2f0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:20.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:20.947+0000 7f149de00640 1 --2- 172.21.15.131:0/3227853726 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f14981f3730 0x7f14981f7360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:20.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:20.947+0000 7f149de00640 1 --2- 172.21.15.131:0/3227853726 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1498148d60 0x7f14981f31f0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:20.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:20.947+0000 7f149de00640 1 --2- 172.21.15.131:0/3227853726 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f147c004030 0x7f14980ccbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:20.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:20.947+0000 7f149de00640 1 -- 172.21.15.131:0/3227853726 >> 172.21.15.131:0/3227853726 conn(0x7f149806bc90 msgr2=0x7f14980b4380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:20.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:20.947+0000 7f149de00640 1 -- 172.21.15.131:0/3227853726 shutdown_connections 2024-05-26T22:50:20.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:20.947+0000 7f149de00640 1 -- 172.21.15.131:0/3227853726 wait complete. 2024-05-26T22:50:20.967 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-05-26T22:50:20.967 DEBUG:teuthology.orchestra.run.smithi131:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-26T22:50:21.064 DEBUG:teuthology.orchestra.run.smithi187:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-26T22:50:21.174 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-05-26T22:50:21.174 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-26T22:50:21.174 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-26T22:50:21.231 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-26T22:50:21.232 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-05-26T22:50:21.232 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-26T22:50:21.288 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-26T22:50:21.289 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-26T22:50:21.289 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-26T22:50:21.347 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-26T22:50:21.347 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-05-26T22:50:21.347 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-26T22:50:21.402 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-26T22:50:21.402 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-26T22:50:21.402 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-26T22:50:21.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:21.609+0000 7f0ae9400640 1 Processor -- start 2024-05-26T22:50:21.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:21.609+0000 7f0ae9400640 1 -- start start 2024-05-26T22:50:21.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:21.611+0000 7f0ae9400640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f0ae40ad250 0x7f0ae40b3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:21.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:21.611+0000 7f0ae9400640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ae415cb20 con 0x7f0ae406ee20 2024-05-26T22:50:21.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:21.611+0000 7f0ae9400640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ae415ce00 con 0x7f0ae41569d0 2024-05-26T22:50:21.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:21.611+0000 7f0ae9400640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f0ae415d0e0 con 0x7f0ae40ad250 2024-05-26T22:50:21.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:21.611+0000 7f0ae2800640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f0ae406ee20 0x7f0ae41562c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:46796/0 (socket says 172.21.15.131:46796) 2024-05-26T22:50:21.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:21.611+0000 7f0ae2800640 1 -- 172.21.15.131:0/3466404735 learned_addr learned my addr 172.21.15.131:0/3466404735 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:50:21.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:21.611+0000 7f0ae1e00640 1 --2- 172.21.15.131:0/3466404735 >> v2:172.21.15.187:3300/0 conn(0x7f0ae40ad250 0x7f0ae40b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:21.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:21.611+0000 7f0ae1400640 1 -- 172.21.15.131:0/3466404735 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2507405869 0 0) 0x7f0ae415cb20 con 0x7f0ae406ee20 2024-05-26T22:50:21.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:21.612+0000 7f0ae1400640 1 -- 172.21.15.131:0/3466404735 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0ac8003520 con 0x7f0ae406ee20 2024-05-26T22:50:21.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:21.612+0000 7f0ae1400640 1 -- 172.21.15.131:0/3466404735 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2245748274 0 0) 0x7f0ac8003520 con 0x7f0ae406ee20 2024-05-26T22:50:21.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:21.612+0000 7f0ae1400640 1 -- 172.21.15.131:0/3466404735 >> v2:172.21.15.187:3300/0 conn(0x7f0ae40ad250 msgr2=0x7f0ae40b3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:21.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:21.612+0000 7f0ae1400640 1 --2- 172.21.15.131:0/3466404735 >> v2:172.21.15.187:3300/0 conn(0x7f0ae40ad250 0x7f0ae40b3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:21.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:21.612+0000 7f0ae1400640 1 -- 172.21.15.131:0/3466404735 >> v1:172.21.15.187:6789/0 conn(0x7f0ae41569d0 legacy=0x7f0ae4160010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:21.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:21.612+0000 7f0ae1400640 1 -- 172.21.15.131:0/3466404735 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ae415d3c0 con 0x7f0ae406ee20 2024-05-26T22:50:21.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:21.612+0000 7f0ae1400640 1 -- 172.21.15.131:0/3466404735 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7f0ad40019f0 con 0x7f0ae406ee20 2024-05-26T22:50:21.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:21.612+0000 7f0ae1400640 1 -- 172.21.15.131:0/3466404735 >> v1:172.21.15.131:6789/0 conn(0x7f0ae406ee20 legacy=0x7f0ae41562c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:21.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:21.612+0000 7f0ae1400640 1 --2- 172.21.15.131:0/3466404735 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ac80039b0 0x7f0ac8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:21.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:21.612+0000 7f0ae1400640 1 --2- 172.21.15.131:0/3466404735 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ac8004800 0x7f0ac8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:21.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:21.612+0000 7f0ae1400640 1 --2- 172.21.15.131:0/3466404735 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ac80052e0 0x7f0ac800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:21.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:21.612+0000 7f0ae1400640 1 -- 172.21.15.131:0/3466404735 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae415d0e0 con 0x7f0ac8004800 2024-05-26T22:50:21.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:21.612+0000 7f0ae1400640 1 -- 172.21.15.131:0/3466404735 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac800c1b0 con 0x7f0ac80052e0 2024-05-26T22:50:21.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:21.612+0000 7f0ae1400640 1 -- 172.21.15.131:0/3466404735 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f0ac800c3b0 con 0x7f0ac80039b0 2024-05-26T22:50:21.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:21.612+0000 7f0ae1e00640 1 --2- 172.21.15.131:0/3466404735 >> v2:172.21.15.187:3300/0 conn(0x7f0ae40ad250 0x7f0ae40b3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-26T22:50:21.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:21.612+0000 7f0ae1e00640 1 --2- 172.21.15.131:0/3466404735 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ac8004800 0x7f0ac8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:21.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:21.613+0000 7f0ae3200640 1 --2- 172.21.15.131:0/3466404735 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ac80052e0 0x7f0ac800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:21.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:21.613+0000 7f0ae2800640 1 --2- 172.21.15.131:0/3466404735 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ac80039b0 0x7f0ac8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:21.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:21.613+0000 7f0ae1e00640 1 -- 172.21.15.131:0/3466404735 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ac80039b0 msgr2=0x7f0ac8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:21.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:21.613+0000 7f0ae1e00640 1 --2- 172.21.15.131:0/3466404735 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ac80039b0 0x7f0ac8003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:21.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:21.613+0000 7f0ae1e00640 1 -- 172.21.15.131:0/3466404735 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ac80052e0 msgr2=0x7f0ac800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:21.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:21.613+0000 7f0ae1e00640 1 --2- 172.21.15.131:0/3466404735 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ac80052e0 0x7f0ac800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:21.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:21.613+0000 7f0ae1e00640 1 -- 172.21.15.131:0/3466404735 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ac8003520 con 0x7f0ac8004800 2024-05-26T22:50:21.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:21.613+0000 7f0ae3200640 1 --2- 172.21.15.131:0/3466404735 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ac80052e0 0x7f0ac800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:50:21.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:21.613+0000 7f0ae3200640 1 -- 172.21.15.131:0/3466404735 reap_dead start 2024-05-26T22:50:21.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:21.614+0000 7f0ae1e00640 1 --2- 172.21.15.131:0/3466404735 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ac8004800 0x7f0ac8004c80 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f0ad0006fd0 tx=0x7f0ac8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43930a85c6e4a700 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:21.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:21.614+0000 7f0ae1400640 1 -- 172.21.15.131:0/3466404735 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad0037070 con 0x7f0ac8004800 2024-05-26T22:50:21.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:21.614+0000 7f0ae1400640 1 -- 172.21.15.131:0/3466404735 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ad002ac50 con 0x7f0ac8004800 2024-05-26T22:50:21.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:21.614+0000 7f0ae1400640 1 -- 172.21.15.131:0/3466404735 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad000f040 con 0x7f0ac8004800 2024-05-26T22:50:21.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:21.615+0000 7f0ae9400640 1 -- 172.21.15.131:0/3466404735 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ac8004800 msgr2=0x7f0ac8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:21.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:21.615+0000 7f0ae9400640 1 --2- 172.21.15.131:0/3466404735 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ac8004800 0x7f0ac8004c80 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f0ad0006fd0 tx=0x7f0ac8005830 comp rx=0 tx=0).stop 2024-05-26T22:50:21.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:21.615+0000 7f0ae9400640 1 -- 172.21.15.131:0/3466404735 shutdown_connections 2024-05-26T22:50:21.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:21.615+0000 7f0ae9400640 1 --2- 172.21.15.131:0/3466404735 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ac8004800 0x7f0ac8004c80 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:21.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:21.615+0000 7f0ae9400640 1 -- 172.21.15.131:0/3466404735 >> 172.21.15.131:0/3466404735 conn(0x7f0ae40b1e90 msgr2=0x7f0ae40b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:21.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:21.615+0000 7f0ae9400640 1 -- 172.21.15.131:0/3466404735 shutdown_connections 2024-05-26T22:50:21.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:21.615+0000 7f0ae9400640 1 -- 172.21.15.131:0/3466404735 wait complete. 2024-05-26T22:50:21.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:21.616+0000 7f0ae9400640 1 Processor -- start 2024-05-26T22:50:21.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:21.617+0000 7f0ae9400640 1 -- start start 2024-05-26T22:50:21.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:21.617+0000 7f0ae9400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ac8004800 0x7f0ae41f3be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:21.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:21.617+0000 7f0ae9400640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ae4147080 0x7f0ae4072b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:21.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:21.618+0000 7f0ae9400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ae4073060 0x7f0ae415b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:21.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:21.618+0000 7f0ae9400640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae415c900 con 0x7f0ae4073060 2024-05-26T22:50:21.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:21.618+0000 7f0ae9400640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae415c780 con 0x7f0ae4147080 2024-05-26T22:50:21.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:21.618+0000 7f0ae9400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f0ae406f200 con 0x7f0ac8004800 2024-05-26T22:50:21.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:21.618+0000 7f0ae3200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ae4073060 0x7f0ae415b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:21.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:21.618+0000 7f0ae3200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ae4073060 0x7f0ae415b230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60738/0 (socket says 172.21.15.131:60738) 2024-05-26T22:50:21.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:21.618+0000 7f0ae3200640 1 -- 172.21.15.131:0/722424735 learned_addr learned my addr 172.21.15.131:0/722424735 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:21.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:21.618+0000 7f0ae2800640 1 --2- 172.21.15.131:0/722424735 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ac8004800 0x7f0ae41f3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:21.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:21.618+0000 7f0ae3200640 1 -- 172.21.15.131:0/722424735 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ac8004800 msgr2=0x7f0ae41f3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:21.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:21.618+0000 7f0ae3200640 1 --2- 172.21.15.131:0/722424735 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ac8004800 0x7f0ae41f3be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:21.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:21.618+0000 7f0ae3200640 1 -- 172.21.15.131:0/722424735 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ae4147080 msgr2=0x7f0ae4072b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:21.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:21.618+0000 7f0ae3200640 1 --2- 172.21.15.131:0/722424735 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ae4147080 0x7f0ae4072b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:21.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:21.618+0000 7f0ae3200640 1 -- 172.21.15.131:0/722424735 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ae409b520 con 0x7f0ae4073060 2024-05-26T22:50:21.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:21.618+0000 7f0ae2800640 1 --2- 172.21.15.131:0/722424735 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ac8004800 0x7f0ae41f3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:50:21.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:21.618+0000 7f0ae3200640 1 --2- 172.21.15.131:0/722424735 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ae4073060 0x7f0ae415b230 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f0ae40b2f50 tx=0x7f0adc00d660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:21.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:21.619+0000 7f0adaa00640 1 -- 172.21.15.131:0/722424735 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0adc018070 con 0x7f0ae4073060 2024-05-26T22:50:21.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:21.619+0000 7f0adaa00640 1 -- 172.21.15.131:0/722424735 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0adc00de60 con 0x7f0ae4073060 2024-05-26T22:50:21.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:21.619+0000 7f0adaa00640 1 -- 172.21.15.131:0/722424735 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0adc003830 con 0x7f0ae4073060 2024-05-26T22:50:21.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:21.619+0000 7f0ae9400640 1 -- 172.21.15.131:0/722424735 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ae409b800 con 0x7f0ae4073060 2024-05-26T22:50:21.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:21.619+0000 7f0ae9400640 1 -- 172.21.15.131:0/722424735 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ae409bdc0 con 0x7f0ae4073060 2024-05-26T22:50:21.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:21.620+0000 7f0ae9400640 1 -- 172.21.15.131:0/722424735 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ae40694e0 con 0x7f0ae4073060 2024-05-26T22:50:21.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:21.621+0000 7f0adaa00640 1 -- 172.21.15.131:0/722424735 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0adc01b070 con 0x7f0ae4073060 2024-05-26T22:50:21.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:21.622+0000 7f0adaa00640 1 --2- 172.21.15.131:0/722424735 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f0ac406be30 0x7f0ac406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:21.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:21.622+0000 7f0adaa00640 1 -- 172.21.15.131:0/722424735 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5115+0+0 (secure 0 0 0) 0x7f0adc08a100 con 0x7f0ae4073060 2024-05-26T22:50:21.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:21.622+0000 7f0ae2800640 1 --2- 172.21.15.131:0/722424735 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f0ac406be30 0x7f0ac406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:21.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:21.623+0000 7f0ae2800640 1 --2- 172.21.15.131:0/722424735 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f0ac406be30 0x7f0ac406e2f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f0ae41f78c0 tx=0x7f0ad4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:21.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:21.627+0000 7f0adaa00640 1 -- 172.21.15.131:0/722424735 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f0adc058cf0 con 0x7f0ae4073060 2024-05-26T22:50:21.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:21.860+0000 7f0ae9400640 1 -- 172.21.15.131:0/722424735 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0ae406c590 con 0x7f0ae4073060 2024-05-26T22:50:21.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:21.860+0000 7f0adaa00640 1 -- 172.21.15.131:0/722424735 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+3881 (secure 0 0 0) 0x7f0adc0585f0 con 0x7f0ae4073060 2024-05-26T22:50:21.860 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:50:21.861 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":22,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/4292497989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":4292497989},{"type":"v1","addr":"172.21.15.187:6835","nonce":4292497989}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4849,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/3714922523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3714922523},{"type":"v1","addr":"172.21.15.187:6833","nonce":3714922523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5523,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6836/3294551724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":3294551724},{"type":"v1","addr":"172.21.15.131:6836","nonce":3294551724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5538,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/2873336536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6835","nonce":2873336536},{"type":"v1","addr":"172.21.15.131:6837","nonce":2873336536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[]} 2024-05-26T22:50:21.861 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 22 2024-05-26T22:50:21.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:21.863+0000 7f0ae9400640 1 -- 172.21.15.131:0/722424735 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f0ac406be30 msgr2=0x7f0ac406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:21.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:21.863+0000 7f0ae9400640 1 --2- 172.21.15.131:0/722424735 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f0ac406be30 0x7f0ac406e2f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f0ae41f78c0 tx=0x7f0ad4009040 comp rx=0 tx=0).stop 2024-05-26T22:50:21.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:21.863+0000 7f0ae9400640 1 -- 172.21.15.131:0/722424735 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ae4073060 msgr2=0x7f0ae415b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:21.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:21.863+0000 7f0ae9400640 1 --2- 172.21.15.131:0/722424735 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ae4073060 0x7f0ae415b230 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f0ae40b2f50 tx=0x7f0adc00d660 comp rx=0 tx=0).stop 2024-05-26T22:50:21.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:21.863+0000 7f0ae9400640 1 -- 172.21.15.131:0/722424735 shutdown_connections 2024-05-26T22:50:21.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:21.863+0000 7f0ae9400640 1 --2- 172.21.15.131:0/722424735 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f0ac406be30 0x7f0ac406e2f0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:21.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:21.863+0000 7f0ae9400640 1 --2- 172.21.15.131:0/722424735 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ae4073060 0x7f0ae415b230 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:21.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:21.863+0000 7f0ae9400640 1 --2- 172.21.15.131:0/722424735 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ae4147080 0x7f0ae4072b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:21.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:21.863+0000 7f0ae9400640 1 --2- 172.21.15.131:0/722424735 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0ac8004800 0x7f0ae41f3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:21.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:21.863+0000 7f0ae9400640 1 -- 172.21.15.131:0/722424735 >> 172.21.15.131:0/722424735 conn(0x7f0ae40b1e90 msgr2=0x7f0ae406e480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:21.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:21.863+0000 7f0ae9400640 1 -- 172.21.15.131:0/722424735 shutdown_connections 2024-05-26T22:50:21.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:21.863+0000 7f0ae9400640 1 -- 172.21.15.131:0/722424735 wait complete. 2024-05-26T22:50:21.880 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-26T22:50:21.880 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-26T22:50:21.881 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-26T22:50:21.881 INFO:teuthology.orchestra.run:waiting for 300 2024-05-26T22:50:21.881 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-26T22:50:21.881 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-26T22:50:21.881 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-26T22:50:21.881 INFO:teuthology.orchestra.run:waiting for 300 2024-05-26T22:50:21.881 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-26T22:50:21.882 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-26T22:50:21.882 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-26T22:50:21.882 INFO:teuthology.orchestra.run:waiting for 300 2024-05-26T22:50:21.882 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-26T22:50:21.882 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-26T22:50:21.882 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-26T22:50:21.882 INFO:teuthology.orchestra.run:waiting for 300 2024-05-26T22:50:21.897 INFO:tasks.ceph.mds.c:Stopped 2024-05-26T22:50:21.897 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-26T22:50:21.899 INFO:tasks.ceph.mds.a:Stopped 2024-05-26T22:50:21.899 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-26T22:50:21.904 INFO:tasks.ceph.mds.d:Stopped 2024-05-26T22:50:21.904 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-26T22:50:21.906 INFO:tasks.ceph.mds.b:Stopped 2024-05-26T22:50:21.906 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-26T22:50:21.946 INFO:tasks.ceph.mds.c:Started 2024-05-26T22:50:21.947 DEBUG:teuthology.parallel:result is None 2024-05-26T22:50:21.952 INFO:tasks.ceph.mds.d:Started 2024-05-26T22:50:21.952 DEBUG:teuthology.parallel:result is None 2024-05-26T22:50:21.953 INFO:tasks.ceph.mds.a:Started 2024-05-26T22:50:21.953 DEBUG:teuthology.parallel:result is None 2024-05-26T22:50:21.958 INFO:tasks.ceph.mds.b:Started 2024-05-26T22:50:21.959 DEBUG:teuthology.parallel:result is None 2024-05-26T22:50:21.959 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-26T22:50:22.090 INFO:tasks.ceph.mds.a.smithi131.stdout:starting mds.a at 2024-05-26T22:50:22.091 INFO:tasks.ceph.mds.c.smithi131.stdout:starting mds.c at 2024-05-26T22:50:22.104 INFO:tasks.ceph.mds.b.smithi187.stdout:starting mds.b at 2024-05-26T22:50:22.105 INFO:tasks.ceph.mds.d.smithi187.stdout:starting mds.d at 2024-05-26T22:50:22.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.132+0000 7f0b8ce00640 1 Processor -- start 2024-05-26T22:50:22.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.132+0000 7f0b8ce00640 1 -- start start 2024-05-26T22:50:22.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.134+0000 7f0b8ce00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0b880b5450 0x7f0b880b43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:22.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.134+0000 7f0b8ce00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f0b8806f420 0x7f0b880cfd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:22.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.134+0000 7f0b8ce00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f0b880cf590 0x7f0b880d0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:22.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.134+0000 7f0b8ce00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f0b88160d90 con 0x7f0b880cf590 2024-05-26T22:50:22.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.134+0000 7f0b8ce00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f0b880b4900 con 0x7f0b8806f420 2024-05-26T22:50:22.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.134+0000 7f0b8ce00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f0b88160180 con 0x7f0b880b5450 2024-05-26T22:50:22.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.134+0000 7f0b86800640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f0b880cf590 0x7f0b880d0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:22.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.134+0000 7f0b86800640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f0b880cf590 0x7f0b880d0290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60768/0 (socket says 172.21.15.131:60768) 2024-05-26T22:50:22.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.134+0000 7f0b86800640 1 -- 172.21.15.131:0/4139068176 learned_addr learned my addr 172.21.15.131:0/4139068176 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:22.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.134+0000 7f0b85e00640 1 --2- 172.21.15.131:0/4139068176 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0b880b5450 0x7f0b880b43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:22.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.134+0000 7f0b7fe00640 1 --2- 172.21.15.131:0/4139068176 >> v2:172.21.15.187:3300/0 conn(0x7f0b8806f420 0x7f0b880cfd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:22.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.134+0000 7f0b86800640 1 -- 172.21.15.131:0/4139068176 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0b880b5450 msgr2=0x7f0b880b43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:22.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.134+0000 7f0b86800640 1 --2- 172.21.15.131:0/4139068176 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0b880b5450 0x7f0b880b43c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:22.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.134+0000 7f0b86800640 1 -- 172.21.15.131:0/4139068176 >> v2:172.21.15.187:3300/0 conn(0x7f0b8806f420 msgr2=0x7f0b880cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:22.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.134+0000 7f0b86800640 1 --2- 172.21.15.131:0/4139068176 >> v2:172.21.15.187:3300/0 conn(0x7f0b8806f420 0x7f0b880cfd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:22.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.134+0000 7f0b86800640 1 -- 172.21.15.131:0/4139068176 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b88160460 con 0x7f0b880cf590 2024-05-26T22:50:22.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.135+0000 7f0b86800640 1 --2- 172.21.15.131:0/4139068176 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0b880cf590 0x7f0b880d0290 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f0b88073750 tx=0x7f0b8002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1ac39bb5d561414 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:22.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.136+0000 7f0b85400640 1 -- 172.21.15.131:0/4139068176 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b80036070 con 0x7f0b880cf590 2024-05-26T22:50:22.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.136+0000 7f0b85400640 1 -- 172.21.15.131:0/4139068176 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b80031600 con 0x7f0b880cf590 2024-05-26T22:50:22.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.136+0000 7f0b85400640 1 -- 172.21.15.131:0/4139068176 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b800319d0 con 0x7f0b880cf590 2024-05-26T22:50:22.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.136+0000 7f0b8ce00640 1 -- 172.21.15.131:0/4139068176 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0b880cf590 msgr2=0x7f0b880d0290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:22.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.136+0000 7f0b8ce00640 1 --2- 172.21.15.131:0/4139068176 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0b880cf590 0x7f0b880d0290 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f0b88073750 tx=0x7f0b8002a6a0 comp rx=0 tx=0).stop 2024-05-26T22:50:22.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.136+0000 7f0b8ce00640 1 -- 172.21.15.131:0/4139068176 shutdown_connections 2024-05-26T22:50:22.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.136+0000 7f0b8ce00640 1 --2- 172.21.15.131:0/4139068176 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0b880cf590 0x7f0b880d0290 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:22.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.136+0000 7f0b8ce00640 1 --2- 172.21.15.131:0/4139068176 >> v2:172.21.15.187:3300/0 conn(0x7f0b8806f420 0x7f0b880cfd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:22.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.136+0000 7f0b8ce00640 1 --2- 172.21.15.131:0/4139068176 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0b880b5450 0x7f0b880b43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:22.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.136+0000 7f0b8ce00640 1 -- 172.21.15.131:0/4139068176 >> 172.21.15.131:0/4139068176 conn(0x7f0b8806bc90 msgr2=0x7f0b880b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:22.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.136+0000 7f0b8ce00640 1 -- 172.21.15.131:0/4139068176 shutdown_connections 2024-05-26T22:50:22.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.136+0000 7f0b8ce00640 1 -- 172.21.15.131:0/4139068176 wait complete. 2024-05-26T22:50:22.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.138+0000 7f0b8ce00640 1 Processor -- start 2024-05-26T22:50:22.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.138+0000 7f0b8ce00640 1 -- start start 2024-05-26T22:50:22.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.139+0000 7f0b8ce00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0b8806f420 0x7f0b880c4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:22.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.139+0000 7f0b8ce00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0b880b5450 0x7f0b880b1530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:22.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.139+0000 7f0b8ce00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0b880cf590 0x7f0b88159a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:22.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.139+0000 7f0b8ce00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f0b880b5830 con 0x7f0b8806f420 2024-05-26T22:50:22.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.139+0000 7f0b8ce00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0b88160a20 con 0x7f0b880b5450 2024-05-26T22:50:22.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.139+0000 7f0b8ce00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f0b8815fde0 con 0x7f0b880cf590 2024-05-26T22:50:22.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.139+0000 7f0b85e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0b8806f420 0x7f0b880c4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:22.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.139+0000 7f0b85e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0b8806f420 0x7f0b880c4e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60774/0 (socket says 172.21.15.131:60774) 2024-05-26T22:50:22.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.139+0000 7f0b85e00640 1 -- 172.21.15.131:0/1436335678 learned_addr learned my addr 172.21.15.131:0/1436335678 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:22.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.139+0000 7f0b7fe00640 1 --2- 172.21.15.131:0/1436335678 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0b880b5450 0x7f0b880b1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:22.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.140+0000 7f0b86800640 1 --2- 172.21.15.131:0/1436335678 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0b880cf590 0x7f0b88159a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:22.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.140+0000 7f0b85e00640 1 -- 172.21.15.131:0/1436335678 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0b880cf590 msgr2=0x7f0b88159a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:22.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.140+0000 7f0b85e00640 1 --2- 172.21.15.131:0/1436335678 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0b880cf590 0x7f0b88159a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:22.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.140+0000 7f0b85e00640 1 -- 172.21.15.131:0/1436335678 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0b880b5450 msgr2=0x7f0b880b1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:22.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.140+0000 7f0b85e00640 1 --2- 172.21.15.131:0/1436335678 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0b880b5450 0x7f0b880b1530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:22.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.140+0000 7f0b85e00640 1 -- 172.21.15.131:0/1436335678 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b881f5500 con 0x7f0b8806f420 2024-05-26T22:50:22.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.140+0000 7f0b86800640 1 --2- 172.21.15.131:0/1436335678 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0b880cf590 0x7f0b88159a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-26T22:50:22.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.140+0000 7f0b7fe00640 1 --2- 172.21.15.131:0/1436335678 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0b880b5450 0x7f0b880b1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-26T22:50:22.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.140+0000 7f0b85e00640 1 --2- 172.21.15.131:0/1436335678 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0b8806f420 0x7f0b880c4e80 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f0b7800d5f0 tx=0x7f0b7800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:22.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.140+0000 7f0b7e000640 1 -- 172.21.15.131:0/1436335678 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b780125f0 con 0x7f0b8806f420 2024-05-26T22:50:22.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.140+0000 7f0b7e000640 1 -- 172.21.15.131:0/1436335678 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b78019070 con 0x7f0b8806f420 2024-05-26T22:50:22.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.140+0000 7f0b8ce00640 1 -- 172.21.15.131:0/1436335678 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b881f57e0 con 0x7f0b8806f420 2024-05-26T22:50:22.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.140+0000 7f0b7e000640 1 -- 172.21.15.131:0/1436335678 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b780129d0 con 0x7f0b8806f420 2024-05-26T22:50:22.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.141+0000 7f0b8ce00640 1 -- 172.21.15.131:0/1436335678 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b881f5da0 con 0x7f0b8806f420 2024-05-26T22:50:22.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.142+0000 7f0b8ce00640 1 -- 172.21.15.131:0/1436335678 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b88074e90 con 0x7f0b8806f420 2024-05-26T22:50:22.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.143+0000 7f0b7e000640 1 -- 172.21.15.131:0/1436335678 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0b78012b70 con 0x7f0b8806f420 2024-05-26T22:50:22.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.143+0000 7f0b7e000640 1 --2- 172.21.15.131:0/1436335678 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f0b6406be30 0x7f0b6406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:22.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.144+0000 7f0b7e000640 1 -- 172.21.15.131:0/1436335678 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5115+0+0 (secure 0 0 0) 0x7f0b7808afb0 con 0x7f0b8806f420 2024-05-26T22:50:22.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.144+0000 7f0b7fe00640 1 --2- 172.21.15.131:0/1436335678 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f0b6406be30 0x7f0b6406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:22.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.144+0000 7f0b7fe00640 1 --2- 172.21.15.131:0/1436335678 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f0b6406be30 0x7f0b6406e2f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f0b880aecf0 tx=0x7f0b70008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:22.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.148+0000 7f0b7e000640 1 -- 172.21.15.131:0/1436335678 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f0b78059ba0 con 0x7f0b8806f420 2024-05-26T22:50:22.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.361+0000 7f0b8ce00640 1 -- 172.21.15.131:0/1436335678 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f0b880695c0 con 0x7f0b8806f420 2024-05-26T22:50:22.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.515+0000 7f0b7e000640 1 -- 172.21.15.131:0/1436335678 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v38) ==== 97+0+0 (secure 0 0 0) 0x7f0b780594a0 con 0x7f0b8806f420 2024-05-26T22:50:22.515 INFO:teuthology.orchestra.run.smithi131.stderr: removed all blocklist entries 2024-05-26T22:50:22.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.520+0000 7f0b8ce00640 1 -- 172.21.15.131:0/1436335678 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f0b6406be30 msgr2=0x7f0b6406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:22.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.520+0000 7f0b8ce00640 1 --2- 172.21.15.131:0/1436335678 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f0b6406be30 0x7f0b6406e2f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f0b880aecf0 tx=0x7f0b70008040 comp rx=0 tx=0).stop 2024-05-26T22:50:22.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.520+0000 7f0b8ce00640 1 -- 172.21.15.131:0/1436335678 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0b8806f420 msgr2=0x7f0b880c4e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:22.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.520+0000 7f0b8ce00640 1 --2- 172.21.15.131:0/1436335678 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0b8806f420 0x7f0b880c4e80 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f0b7800d5f0 tx=0x7f0b7800dab0 comp rx=0 tx=0).stop 2024-05-26T22:50:22.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.520+0000 7f0b8ce00640 1 -- 172.21.15.131:0/1436335678 shutdown_connections 2024-05-26T22:50:22.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.520+0000 7f0b8ce00640 1 --2- 172.21.15.131:0/1436335678 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0b880cf590 0x7f0b88159a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:22.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.520+0000 7f0b8ce00640 1 --2- 172.21.15.131:0/1436335678 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f0b6406be30 0x7f0b6406e2f0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:22.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.520+0000 7f0b8ce00640 1 --2- 172.21.15.131:0/1436335678 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0b880b5450 0x7f0b880b1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:22.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.520+0000 7f0b8ce00640 1 --2- 172.21.15.131:0/1436335678 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0b8806f420 0x7f0b880c4e80 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:22.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.520+0000 7f0b8ce00640 1 -- 172.21.15.131:0/1436335678 >> 172.21.15.131:0/1436335678 conn(0x7f0b8806bc90 msgr2=0x7f0b880cc100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:22.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.520+0000 7f0b8ce00640 1 -- 172.21.15.131:0/1436335678 shutdown_connections 2024-05-26T22:50:22.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.520+0000 7f0b8ce00640 1 -- 172.21.15.131:0/1436335678 wait complete. 2024-05-26T22:50:22.543 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-26T22:50:22.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.732+0000 7f6729600640 1 Processor -- start 2024-05-26T22:50:22.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.732+0000 7f6729600640 1 -- start start 2024-05-26T22:50:22.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.734+0000 7f6729600640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f67240cc7f0 0x7f67240ccc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:22.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.734+0000 7f6729600640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6724160620 con 0x7f672406e8c0 2024-05-26T22:50:22.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.734+0000 7f6729600640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6724160900 con 0x7f6724156750 2024-05-26T22:50:22.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.734+0000 7f6729600640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f672415d770 con 0x7f67240cc7f0 2024-05-26T22:50:22.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.734+0000 7f6723400640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f672406e8c0 0x7f672415d060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:46818/0 (socket says 172.21.15.131:46818) 2024-05-26T22:50:22.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.734+0000 7f6723400640 1 -- 172.21.15.131:0/1654044070 learned_addr learned my addr 172.21.15.131:0/1654044070 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:50:22.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.734+0000 7f6728c00640 1 --2- 172.21.15.131:0/1654044070 >> v2:172.21.15.131:3300/0 conn(0x7f67240cc7f0 0x7f67240ccc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:22.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.735+0000 7f6728c00640 1 -- 172.21.15.131:0/1654044070 >> v1:172.21.15.187:6789/0 conn(0x7f6724156750 legacy=0x7f672415fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:22.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.735+0000 7f6728c00640 1 -- 172.21.15.131:0/1654044070 >> v1:172.21.15.131:6789/0 conn(0x7f672406e8c0 legacy=0x7f672415d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:22.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.735+0000 7f6728c00640 1 -- 172.21.15.131:0/1654044070 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6724160be0 con 0x7f67240cc7f0 2024-05-26T22:50:22.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.736+0000 7f6728c00640 1 --2- 172.21.15.131:0/1654044070 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f67240cc7f0 0x7f67240ccc60 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f671c009d90 tx=0x7f671c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e3db4aed14c2842 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:22.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.736+0000 7f6722a00640 1 -- 172.21.15.131:0/1654044070 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f671c007740 con 0x7f67240cc7f0 2024-05-26T22:50:22.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.736+0000 7f6722a00640 1 -- 172.21.15.131:0/1654044070 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f671c0078e0 con 0x7f67240cc7f0 2024-05-26T22:50:22.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.736+0000 7f6722a00640 1 -- 172.21.15.131:0/1654044070 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f671c007cb0 con 0x7f67240cc7f0 2024-05-26T22:50:22.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.736+0000 7f6729600640 1 -- 172.21.15.131:0/1654044070 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f67240cc7f0 msgr2=0x7f67240ccc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:22.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.736+0000 7f6729600640 1 --2- 172.21.15.131:0/1654044070 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f67240cc7f0 0x7f67240ccc60 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f671c009d90 tx=0x7f671c02a6a0 comp rx=0 tx=0).stop 2024-05-26T22:50:22.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.736+0000 7f6729600640 1 -- 172.21.15.131:0/1654044070 shutdown_connections 2024-05-26T22:50:22.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.736+0000 7f6729600640 1 --2- 172.21.15.131:0/1654044070 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f67240cc7f0 0x7f67240ccc60 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:22.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.736+0000 7f6729600640 1 -- 172.21.15.131:0/1654044070 >> 172.21.15.131:0/1654044070 conn(0x7f67240cff40 msgr2=0x7f67240d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:22.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.737+0000 7f6729600640 1 -- 172.21.15.131:0/1654044070 shutdown_connections 2024-05-26T22:50:22.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.737+0000 7f6729600640 1 -- 172.21.15.131:0/1654044070 wait complete. 2024-05-26T22:50:22.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.738+0000 7f6729600640 1 Processor -- start 2024-05-26T22:50:22.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.738+0000 7f6729600640 1 -- start start 2024-05-26T22:50:22.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.739+0000 7f6729600640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f672406e8c0 0x7f6724141390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:22.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.739+0000 7f6729600640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f67240cc7f0 0x7f6724155d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:22.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.739+0000 7f6729600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6724156750 0x7f6724134260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:22.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.739+0000 7f6729600640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f672415cbd0 con 0x7f6724156750 2024-05-26T22:50:22.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.739+0000 7f6729600640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f67240cd1a0 con 0x7f672406e8c0 2024-05-26T22:50:22.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.739+0000 7f6729600640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f6724156b30 con 0x7f67240cc7f0 2024-05-26T22:50:22.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.740+0000 7f6728c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6724156750 0x7f6724134260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:22.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.740+0000 7f6728c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6724156750 0x7f6724134260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60780/0 (socket says 172.21.15.131:60780) 2024-05-26T22:50:22.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.740+0000 7f6728c00640 1 -- 172.21.15.131:0/1940256116 learned_addr learned my addr 172.21.15.131:0/1940256116 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:22.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.740+0000 7f6728c00640 1 -- 172.21.15.131:0/1940256116 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f67240cc7f0 msgr2=0x7f6724155d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:22.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.740+0000 7f6728c00640 1 --2- 172.21.15.131:0/1940256116 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f67240cc7f0 0x7f6724155d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:22.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.740+0000 7f6728c00640 1 -- 172.21.15.131:0/1940256116 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f672406e8c0 msgr2=0x7f6724141390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:22.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.740+0000 7f6723e00640 1 --2- 172.21.15.131:0/1940256116 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f672406e8c0 0x7f6724141390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:22.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.740+0000 7f6728c00640 1 --2- 172.21.15.131:0/1940256116 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f672406e8c0 0x7f6724141390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:22.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.740+0000 7f6728c00640 1 -- 172.21.15.131:0/1940256116 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67241349d0 con 0x7f6724156750 2024-05-26T22:50:22.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.740+0000 7f6728c00640 1 --2- 172.21.15.131:0/1940256116 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6724156750 0x7f6724134260 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f671c009eb0 tx=0x7f671c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:22.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.740+0000 7f671be00640 1 -- 172.21.15.131:0/1940256116 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f671c040070 con 0x7f6724156750 2024-05-26T22:50:22.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.741+0000 7f671be00640 1 -- 172.21.15.131:0/1940256116 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f671c031c40 con 0x7f6724156750 2024-05-26T22:50:22.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.741+0000 7f6729600640 1 -- 172.21.15.131:0/1940256116 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6724134cb0 con 0x7f6724156750 2024-05-26T22:50:22.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.741+0000 7f671be00640 1 -- 172.21.15.131:0/1940256116 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f671c03c420 con 0x7f6724156750 2024-05-26T22:50:22.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.741+0000 7f6729600640 1 -- 172.21.15.131:0/1940256116 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6724135270 con 0x7f6724156750 2024-05-26T22:50:22.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.742+0000 7f6729600640 1 -- 172.21.15.131:0/1940256116 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67240694e0 con 0x7f6724156750 2024-05-26T22:50:22.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.743+0000 7f671be00640 1 -- 172.21.15.131:0/1940256116 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f671c039040 con 0x7f6724156750 2024-05-26T22:50:22.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.744+0000 7f671be00640 1 --2- 172.21.15.131:0/1940256116 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f670406be30 0x7f670406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:22.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.744+0000 7f671be00640 1 -- 172.21.15.131:0/1940256116 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 4857+0+0 (secure 0 0 0) 0x7f671c0ac1c0 con 0x7f6724156750 2024-05-26T22:50:22.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.744+0000 7f6723e00640 1 --2- 172.21.15.131:0/1940256116 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f670406be30 0x7f670406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:22.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.744+0000 7f6723e00640 1 --2- 172.21.15.131:0/1940256116 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f670406be30 0x7f670406e2f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f671400a750 tx=0x7f6714008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:22.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:22.748+0000 7f671be00640 1 -- 172.21.15.131:0/1940256116 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f671c07aec0 con 0x7f6724156750 2024-05-26T22:50:23.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.016+0000 7f6729600640 1 -- 172.21.15.131:0/1940256116 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f6724144f30 con 0x7f6724156750 2024-05-26T22:50:23.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.017+0000 7f671be00640 1 -- 172.21.15.131:0/1940256116 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v13) ==== 80+0+5304 (secure 0 0 0) 0x7f671c07a7c0 con 0x7f6724156750 2024-05-26T22:50:23.017 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:50:23.017 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-26T22:50:23.017 INFO:teuthology.orchestra.run.smithi131.stdout: "auth_dump": [ 2024-05-26T22:50:23.017 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:50:23.017 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mds.a", 2024-05-26T22:50:23.017 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD1u1NmTQf1DhAAdvEt7wP0IgKhNHOqjVAllA==", 2024-05-26T22:50:23.017 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:50:23.017 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-05-26T22:50:23.017 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2024-05-26T22:50:23.017 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2024-05-26T22:50:23.017 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-05-26T22:50:23.018 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:50:23.018 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:23.018 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:50:23.018 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mds.b", 2024-05-26T22:50:23.018 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD1u1NmHNZ8KxAAPgQ/ojbl+EDfPhqrkg7LPQ==", 2024-05-26T22:50:23.018 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:50:23.018 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-05-26T22:50:23.018 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2024-05-26T22:50:23.018 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2024-05-26T22:50:23.018 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-05-26T22:50:23.018 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:50:23.018 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:23.018 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:50:23.018 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mds.c", 2024-05-26T22:50:23.019 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD1u1NmFjCHHRAAqix3Ujfsq2tnSU5+d/pSYA==", 2024-05-26T22:50:23.019 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:50:23.019 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-05-26T22:50:23.019 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2024-05-26T22:50:23.019 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2024-05-26T22:50:23.019 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-05-26T22:50:23.019 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:50:23.019 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:23.019 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:50:23.019 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mds.d", 2024-05-26T22:50:23.019 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD1u1NmM4LPORAAhtQtr+Vfpf7vdVwjSW6xtA==", 2024-05-26T22:50:23.019 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:50:23.019 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-05-26T22:50:23.020 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2024-05-26T22:50:23.020 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2024-05-26T22:50:23.020 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-05-26T22:50:23.020 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:50:23.020 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:23.020 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:50:23.020 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.0", 2024-05-26T22:50:23.020 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD5u1Nm+w1LNhAAsJLQkiKSh3KkMymlrZkcrw==", 2024-05-26T22:50:23.020 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:50:23.020 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-05-26T22:50:23.020 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-05-26T22:50:23.020 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-05-26T22:50:23.020 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:50:23.020 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:23.021 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:50:23.021 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.1", 2024-05-26T22:50:23.021 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD9u1Nm4c8RFBAAcn9HQ+izML9rujJ/2/Q/BQ==", 2024-05-26T22:50:23.021 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:50:23.021 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-05-26T22:50:23.021 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-05-26T22:50:23.021 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-05-26T22:50:23.021 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:50:23.021 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:23.021 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:50:23.021 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.2", 2024-05-26T22:50:23.021 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAAvFNmR5wmLRAAYEBAXPBZFq67iARZnwF3iw==", 2024-05-26T22:50:23.021 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:50:23.021 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-05-26T22:50:23.022 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-05-26T22:50:23.022 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-05-26T22:50:23.022 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:50:23.022 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:23.022 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:50:23.022 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.3", 2024-05-26T22:50:23.022 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAEvFNmJ4G5CxAAnTbP8DinhmuvGNo1VBVzjQ==", 2024-05-26T22:50:23.022 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:50:23.022 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-05-26T22:50:23.022 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-05-26T22:50:23.022 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-05-26T22:50:23.022 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:50:23.022 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:23.023 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:50:23.023 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.4", 2024-05-26T22:50:23.023 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAKvFNmcw5rGhAAvnivP6kt3vcN9UPm/svKqw==", 2024-05-26T22:50:23.023 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:50:23.023 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-05-26T22:50:23.023 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-05-26T22:50:23.023 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-05-26T22:50:23.023 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:50:23.023 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:23.023 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:50:23.023 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.5", 2024-05-26T22:50:23.023 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQANvFNmTqlzNhAAGfrSvNNHYKGOBzRXo0TEdw==", 2024-05-26T22:50:23.023 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:50:23.023 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-05-26T22:50:23.024 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-05-26T22:50:23.024 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-05-26T22:50:23.024 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:50:23.024 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:23.024 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:50:23.024 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.6", 2024-05-26T22:50:23.024 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQARvFNmpzd6FRAAgi2RYlQPvfHNe0iP4bJrwg==", 2024-05-26T22:50:23.024 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:50:23.024 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-05-26T22:50:23.024 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-05-26T22:50:23.024 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-05-26T22:50:23.024 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:50:23.024 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:23.024 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:50:23.025 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.7", 2024-05-26T22:50:23.025 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAUvFNmAkC8LhAAsmkKF4eloxdpCi/mrDtzqA==", 2024-05-26T22:50:23.025 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:50:23.025 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-05-26T22:50:23.025 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-05-26T22:50:23.025 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-05-26T22:50:23.025 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:50:23.025 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:23.025 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:50:23.025 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.0", 2024-05-26T22:50:23.025 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD2u1NmSGVWChAAH/wq3iaUs6ZELRhLKhmURQ==", 2024-05-26T22:50:23.025 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:50:23.025 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-05-26T22:50:23.025 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow r", 2024-05-26T22:50:23.026 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-05-26T22:50:23.026 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:50:23.026 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:23.026 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:50:23.026 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.1", 2024-05-26T22:50:23.026 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD2u1NmnA9CHRAA2XMmZQxysKweHNykiQUEeA==", 2024-05-26T22:50:23.026 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:50:23.026 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-05-26T22:50:23.026 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow r", 2024-05-26T22:50:23.027 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow rw", 2024-05-26T22:50:23.027 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow rwx" 2024-05-26T22:50:23.027 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:50:23.027 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:23.027 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:50:23.027 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.2", 2024-05-26T22:50:23.027 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD2u1NmGEI9ExAAvJn1LRwmALkD7zh/AP07vQ==", 2024-05-26T22:50:23.027 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:50:23.027 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-05-26T22:50:23.027 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow r", 2024-05-26T22:50:23.027 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow rw", 2024-05-26T22:50:23.027 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow rwx" 2024-05-26T22:50:23.027 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:50:23.029 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:23.029 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:50:23.029 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.3", 2024-05-26T22:50:23.029 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD2u1NmIHOtJhAAJCUm0kGqWu4rWVLsDkW6Eg==", 2024-05-26T22:50:23.030 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:50:23.030 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-05-26T22:50:23.030 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow r", 2024-05-26T22:50:23.030 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow rw", 2024-05-26T22:50:23.030 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow rwx" 2024-05-26T22:50:23.030 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:50:23.030 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:23.030 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:50:23.030 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.admin", 2024-05-26T22:50:23.030 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD0u1NmLRGoCBAAhDeVJV14MTAIdUte067jig==", 2024-05-26T22:50:23.030 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:50:23.030 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow *", 2024-05-26T22:50:23.030 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2024-05-26T22:50:23.031 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2024-05-26T22:50:23.031 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-05-26T22:50:23.031 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:50:23.031 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:23.031 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:50:23.031 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mgr.x", 2024-05-26T22:50:23.031 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD1u1NmaWhyBBAAlU7Lq6HMjkCPu0kztJIAXQ==", 2024-05-26T22:50:23.031 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:50:23.031 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow *", 2024-05-26T22:50:23.031 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile mgr", 2024-05-26T22:50:23.031 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-05-26T22:50:23.031 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:50:23.031 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:23.031 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:50:23.032 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mgr.y", 2024-05-26T22:50:23.032 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD0u1NmX1RhNhAAo5mEOGzUyrdwFlB/XOXKnw==", 2024-05-26T22:50:23.032 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:50:23.032 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow *", 2024-05-26T22:50:23.032 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile mgr", 2024-05-26T22:50:23.032 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-05-26T22:50:23.032 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:50:23.032 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:50:23.032 INFO:teuthology.orchestra.run.smithi131.stdout: ] 2024-05-26T22:50:23.032 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-26T22:50:23.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.019+0000 7f6729600640 1 -- 172.21.15.131:0/1940256116 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f670406be30 msgr2=0x7f670406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:23.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.019+0000 7f6729600640 1 --2- 172.21.15.131:0/1940256116 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f670406be30 0x7f670406e2f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f671400a750 tx=0x7f6714008040 comp rx=0 tx=0).stop 2024-05-26T22:50:23.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.019+0000 7f6729600640 1 -- 172.21.15.131:0/1940256116 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6724156750 msgr2=0x7f6724134260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:23.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.019+0000 7f6729600640 1 --2- 172.21.15.131:0/1940256116 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6724156750 0x7f6724134260 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f671c009eb0 tx=0x7f671c004a40 comp rx=0 tx=0).stop 2024-05-26T22:50:23.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.020+0000 7f6729600640 1 -- 172.21.15.131:0/1940256116 shutdown_connections 2024-05-26T22:50:23.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.020+0000 7f6729600640 1 --2- 172.21.15.131:0/1940256116 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f67240cc7f0 0x7f6724155d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:23.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.020+0000 7f6729600640 1 --2- 172.21.15.131:0/1940256116 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f670406be30 0x7f670406e2f0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:23.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.020+0000 7f6729600640 1 --2- 172.21.15.131:0/1940256116 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6724156750 0x7f6724134260 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:23.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.020+0000 7f6729600640 1 --2- 172.21.15.131:0/1940256116 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f672406e8c0 0x7f6724141390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:23.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.020+0000 7f6729600640 1 -- 172.21.15.131:0/1940256116 >> 172.21.15.131:0/1940256116 conn(0x7f67240cff40 msgr2=0x7f672415d060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:23.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.020+0000 7f6729600640 1 -- 172.21.15.131:0/1940256116 shutdown_connections 2024-05-26T22:50:23.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.020+0000 7f6729600640 1 -- 172.21.15.131:0/1940256116 wait complete. 2024-05-26T22:50:23.037 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-26T22:50:23.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.221+0000 7f4116400640 1 Processor -- start 2024-05-26T22:50:23.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.221+0000 7f4116400640 1 -- start start 2024-05-26T22:50:23.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.222+0000 7f4116400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f411006ee20 0x7f41100b3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:23.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.222+0000 7f4116400640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f41100ad250 0x7f41100af790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:23.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.222+0000 7f4116400640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f411015cb20 con 0x7f411006f200 2024-05-26T22:50:23.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.222+0000 7f4116400640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f411015ce00 con 0x7f41100ad250 2024-05-26T22:50:23.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.222+0000 7f4116400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f411015d0e0 con 0x7f411006ee20 2024-05-26T22:50:23.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.223+0000 7f410f400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f411006ee20 0x7f41100b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:23.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.223+0000 7f4107e00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f41100ad250 0x7f41100af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:23.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.223+0000 7f410fe00640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f411006f200 0x7f41101562c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.131:49248/0 (socket says 172.21.15.131:49248) 2024-05-26T22:50:23.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.223+0000 7f410fe00640 1 -- 172.21.15.131:0/3819294197 learned_addr learned my addr 172.21.15.131:0/3819294197 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:50:23.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.223+0000 7f410f400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f411006ee20 0x7f41100b3190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.131:42704/0 (socket says 172.21.15.131:42704) 2024-05-26T22:50:23.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.224+0000 7f410ea00640 1 -- 172.21.15.131:0/3819294197 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3798552930 0 0) 0x7f411015cb20 con 0x7f411006f200 2024-05-26T22:50:23.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.224+0000 7f410ea00640 1 -- 172.21.15.131:0/3819294197 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f40f4003520 con 0x7f411006f200 2024-05-26T22:50:23.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.224+0000 7f4107e00640 1 -- 172.21.15.131:0/3819294197 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f411006ee20 msgr2=0x7f41100b3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:23.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.224+0000 7f4107e00640 1 --2- 172.21.15.131:0/3819294197 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f411006ee20 0x7f41100b3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:23.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.224+0000 7f4107e00640 1 -- 172.21.15.131:0/3819294197 >> v1:172.21.15.187:6789/0 conn(0x7f411006f200 legacy=0x7f41101562c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:23.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.224+0000 7f4107e00640 1 -- 172.21.15.131:0/3819294197 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f411015d3c0 con 0x7f41100ad250 2024-05-26T22:50:23.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.225+0000 7f4107e00640 1 --2- 172.21.15.131:0/3819294197 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f41100ad250 0x7f41100af790 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f40f800dbc0 tx=0x7f40f802e490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13d626a351ad2bc3 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:23.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.226+0000 7f410ea00640 1 -- 172.21.15.131:0/3819294197 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f803a070 con 0x7f41100ad250 2024-05-26T22:50:23.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.226+0000 7f410ea00640 1 -- 172.21.15.131:0/3819294197 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40f800b650 con 0x7f41100ad250 2024-05-26T22:50:23.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.226+0000 7f410ea00640 1 -- 172.21.15.131:0/3819294197 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f800ba20 con 0x7f41100ad250 2024-05-26T22:50:23.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.227+0000 7f4116400640 1 -- 172.21.15.131:0/3819294197 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f41100ad250 msgr2=0x7f41100af790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:23.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.227+0000 7f4116400640 1 --2- 172.21.15.131:0/3819294197 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f41100ad250 0x7f41100af790 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f40f800dbc0 tx=0x7f40f802e490 comp rx=0 tx=0).stop 2024-05-26T22:50:23.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.227+0000 7f4116400640 1 -- 172.21.15.131:0/3819294197 shutdown_connections 2024-05-26T22:50:23.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.227+0000 7f4116400640 1 --2- 172.21.15.131:0/3819294197 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f41100ad250 0x7f41100af790 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:23.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.227+0000 7f4116400640 1 --2- 172.21.15.131:0/3819294197 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f411006ee20 0x7f41100b3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:23.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.227+0000 7f4116400640 1 -- 172.21.15.131:0/3819294197 >> 172.21.15.131:0/3819294197 conn(0x7f41100b1e90 msgr2=0x7f41100b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:23.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.227+0000 7f4116400640 1 -- 172.21.15.131:0/3819294197 shutdown_connections 2024-05-26T22:50:23.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.227+0000 7f4116400640 1 -- 172.21.15.131:0/3819294197 wait complete. 2024-05-26T22:50:23.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.228+0000 7f4116400640 1 Processor -- start 2024-05-26T22:50:23.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.229+0000 7f4116400640 1 -- start start 2024-05-26T22:50:23.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.229+0000 7f4116400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f411006ee20 0x7f411014e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:23.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.229+0000 7f4116400640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f411006f200 0x7f411015e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:23.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.230+0000 7f4116400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f41100ad250 0x7f411015eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:23.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.230+0000 7f4116400640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f411015c900 con 0x7f41100ad250 2024-05-26T22:50:23.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.230+0000 7f4116400640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f411015c780 con 0x7f411006f200 2024-05-26T22:50:23.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.230+0000 7f4116400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f41101569d0 con 0x7f411006ee20 2024-05-26T22:50:23.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.230+0000 7f410fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f41100ad250 0x7f411015eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:23.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.230+0000 7f410fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f41100ad250 0x7f411015eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60784/0 (socket says 172.21.15.131:60784) 2024-05-26T22:50:23.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.230+0000 7f410fe00640 1 -- 172.21.15.131:0/7335330 learned_addr learned my addr 172.21.15.131:0/7335330 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:23.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.230+0000 7f410f400640 1 --2- 172.21.15.131:0/7335330 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f411006ee20 0x7f411014e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:23.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.230+0000 7f410fe00640 1 -- 172.21.15.131:0/7335330 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f411006ee20 msgr2=0x7f411014e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:23.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.230+0000 7f410fe00640 1 --2- 172.21.15.131:0/7335330 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f411006ee20 0x7f411014e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:23.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.230+0000 7f410fe00640 1 -- 172.21.15.131:0/7335330 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f411006f200 msgr2=0x7f411015e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:23.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.230+0000 7f4107e00640 1 --2- 172.21.15.131:0/7335330 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f411006f200 0x7f411015e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:23.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.230+0000 7f410fe00640 1 --2- 172.21.15.131:0/7335330 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f411006f200 0x7f411015e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:23.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.230+0000 7f410fe00640 1 -- 172.21.15.131:0/7335330 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4110154340 con 0x7f41100ad250 2024-05-26T22:50:23.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.231+0000 7f410fe00640 1 --2- 172.21.15.131:0/7335330 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f41100ad250 0x7f411015eeb0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f410800cbe0 tx=0x7f410800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:23.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.231+0000 7f4107400640 1 -- 172.21.15.131:0/7335330 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4108017070 con 0x7f41100ad250 2024-05-26T22:50:23.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.231+0000 7f4107400640 1 -- 172.21.15.131:0/7335330 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f410800ad40 con 0x7f41100ad250 2024-05-26T22:50:23.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.231+0000 7f4116400640 1 -- 172.21.15.131:0/7335330 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f411014eaf0 con 0x7f41100ad250 2024-05-26T22:50:23.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.231+0000 7f4107400640 1 -- 172.21.15.131:0/7335330 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4108012600 con 0x7f41100ad250 2024-05-26T22:50:23.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.231+0000 7f4116400640 1 -- 172.21.15.131:0/7335330 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f411014fbb0 con 0x7f41100ad250 2024-05-26T22:50:23.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.232+0000 7f4116400640 1 -- 172.21.15.131:0/7335330 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41100694e0 con 0x7f41100ad250 2024-05-26T22:50:23.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.233+0000 7f4107400640 1 -- 172.21.15.131:0/7335330 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f410801a070 con 0x7f41100ad250 2024-05-26T22:50:23.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.234+0000 7f4107400640 1 --2- 172.21.15.131:0/7335330 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f40ec06be60 0x7f40ec06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:23.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.235+0000 7f4107400640 1 -- 172.21.15.131:0/7335330 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 4857+0+0 (secure 0 0 0) 0x7f410808a870 con 0x7f41100ad250 2024-05-26T22:50:23.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.235+0000 7f410f400640 1 --2- 172.21.15.131:0/7335330 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f40ec06be60 0x7f40ec06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:23.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.235+0000 7f410f400640 1 --2- 172.21.15.131:0/7335330 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f40ec06be60 0x7f40ec06e320 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4100009a10 tx=0x7f4100008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:23.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.239+0000 7f4107400640 1 -- 172.21.15.131:0/7335330 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f411014eaf0 con 0x7f41100ad250 2024-05-26T22:50:23.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.435+0000 7f4116400640 1 -- 172.21.15.131:0/7335330 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f411006e7c0 con 0x7f41100ad250 2024-05-26T22:50:23.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.437+0000 7f4107400640 1 -- 172.21.15.131:0/7335330 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) ==== 74+0+12915 (secure 0 0 0) 0x7f4110154340 con 0x7f41100ad250 2024-05-26T22:50:23.437 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:50:23.438 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":40,"fsid":"46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f","created":"2024-05-26T22:47:57.288905+0000","modified":"2024-05-26T22:50:22.529766+0000","last_up_change":"2024-05-26T22:48:09.602207+0000","last_in_change":"2024-05-26T22:48:00.687341+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-26T22:48:10.463339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"814eaf62-a18d-494b-8e27-fd8fb88adea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6801","nonce":2350940051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6803","nonce":2350940051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6807","nonce":2350940051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6805","nonce":2350940051}]},"public_addr":"172.21.15.131:6801/2350940051","cluster_addr":"172.21.15.131:6803/2350940051","heartbeat_back_addr":"172.21.15.131:6807/2350940051","heartbeat_front_addr":"172.21.15.131:6805/2350940051","state":["exists","up"]},{"osd":1,"uuid":"42e66287-f6c6-4e26-af3c-6dfaa3536d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6825","nonce":1525528247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6827","nonce":1525528247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6831","nonce":1525528247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6829","nonce":1525528247}]},"public_addr":"172.21.15.131:6825/1525528247","cluster_addr":"172.21.15.131:6827/1525528247","heartbeat_back_addr":"172.21.15.131:6831/1525528247","heartbeat_front_addr":"172.21.15.131:6829/1525528247","state":["exists","up"]},{"osd":2,"uuid":"0da34673-04a2-4fd0-b57a-0ff2b679b043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6817","nonce":3627842210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6819","nonce":3627842210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6823","nonce":3627842210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6821","nonce":3627842210}]},"public_addr":"172.21.15.131:6817/3627842210","cluster_addr":"172.21.15.131:6819/3627842210","heartbeat_back_addr":"172.21.15.131:6823/3627842210","heartbeat_front_addr":"172.21.15.131:6821/3627842210","state":["exists","up"]},{"osd":3,"uuid":"cf117a27-d6b2-43d3-ae52-b47988bf8eed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6809","nonce":350790591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6811","nonce":350790591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6815","nonce":350790591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6813","nonce":350790591}]},"public_addr":"172.21.15.131:6809/350790591","cluster_addr":"172.21.15.131:6811/350790591","heartbeat_back_addr":"172.21.15.131:6815/350790591","heartbeat_front_addr":"172.21.15.131:6813/350790591","state":["exists","up"]},{"osd":4,"uuid":"a092ccfc-be89-4a84-88d5-0f781fa715eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6816","nonce":244560995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6819","nonce":244560995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6823","nonce":244560995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6821","nonce":244560995}]},"public_addr":"172.21.15.187:6816/244560995","cluster_addr":"172.21.15.187:6819/244560995","heartbeat_back_addr":"172.21.15.187:6823/244560995","heartbeat_front_addr":"172.21.15.187:6821/244560995","state":["exists","up"]},{"osd":5,"uuid":"7513ba69-dd00-4a35-b190-9e8cc3b2bdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6801","nonce":324619398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6803","nonce":324619398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6812","nonce":324619398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6807","nonce":324619398}]},"public_addr":"172.21.15.187:6801/324619398","cluster_addr":"172.21.15.187:6803/324619398","heartbeat_back_addr":"172.21.15.187:6812/324619398","heartbeat_front_addr":"172.21.15.187:6807/324619398","state":["exists","up"]},{"osd":6,"uuid":"50081810-c093-4d2f-a4d7-b7c695c5400d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6806","nonce":1391347786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6809","nonce":1391347786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6818","nonce":1391347786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6813","nonce":1391347786}]},"public_addr":"172.21.15.187:6806/1391347786","cluster_addr":"172.21.15.187:6809/1391347786","heartbeat_back_addr":"172.21.15.187:6818/1391347786","heartbeat_front_addr":"172.21.15.187:6813/1391347786","state":["exists","up"]},{"osd":7,"uuid":"775dae05-54f5-4ad4-bbce-92863c6ed2ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6825","nonce":1220065622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6827","nonce":1220065622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6831","nonce":1220065622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6829","nonce":1220065622}]},"public_addr":"172.21.15.187:6825/1220065622","cluster_addr":"172.21.15.187:6827/1220065622","heartbeat_back_addr":"172.21.15.187:6831/1220065622","heartbeat_front_addr":"172.21.15.187:6829/1220065622","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:05.983520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.299516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.067323+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.307748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.018742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.258775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.010912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.295355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-26T22:50:23.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.439+0000 7f4116400640 1 -- 172.21.15.131:0/7335330 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f40ec06be60 msgr2=0x7f40ec06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:23.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.439+0000 7f4116400640 1 --2- 172.21.15.131:0/7335330 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f40ec06be60 0x7f40ec06e320 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4100009a10 tx=0x7f4100008040 comp rx=0 tx=0).stop 2024-05-26T22:50:23.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.439+0000 7f4116400640 1 -- 172.21.15.131:0/7335330 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f41100ad250 msgr2=0x7f411015eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:23.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.439+0000 7f4116400640 1 --2- 172.21.15.131:0/7335330 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f41100ad250 0x7f411015eeb0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f410800cbe0 tx=0x7f410800a430 comp rx=0 tx=0).stop 2024-05-26T22:50:23.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.439+0000 7f4116400640 1 -- 172.21.15.131:0/7335330 shutdown_connections 2024-05-26T22:50:23.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.440+0000 7f4116400640 1 --2- 172.21.15.131:0/7335330 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f40ec06be60 0x7f40ec06e320 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:23.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.440+0000 7f4116400640 1 --2- 172.21.15.131:0/7335330 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f41100ad250 0x7f411015eeb0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:23.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.440+0000 7f4116400640 1 --2- 172.21.15.131:0/7335330 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f411006f200 0x7f411015e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:23.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.440+0000 7f4116400640 1 --2- 172.21.15.131:0/7335330 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f411006ee20 0x7f411014e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:23.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.440+0000 7f4116400640 1 -- 172.21.15.131:0/7335330 >> 172.21.15.131:0/7335330 conn(0x7f41100b1e90 msgr2=0x7f41100b3750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:23.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.440+0000 7f4116400640 1 -- 172.21.15.131:0/7335330 shutdown_connections 2024-05-26T22:50:23.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.440+0000 7f4116400640 1 -- 172.21.15.131:0/7335330 wait complete. 2024-05-26T22:50:23.458 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-26T22:48:10.463339+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-26T22:50:23.459 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-26T22:50:23.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.653+0000 7f66be000640 1 Processor -- start 2024-05-26T22:50:23.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.653+0000 7f66be000640 1 -- start start 2024-05-26T22:50:23.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.655+0000 7f66be000640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f66b80b5450 0x7f66b80b43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:23.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.655+0000 7f66be000640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f66b806f420 0x7f66b80cfd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:23.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.655+0000 7f66be000640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66b80b4900 con 0x7f66b80cf590 2024-05-26T22:50:23.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.655+0000 7f66be000640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f66b8160d90 con 0x7f66b80b5450 2024-05-26T22:50:23.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.655+0000 7f66be000640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f66b8160180 con 0x7f66b806f420 2024-05-26T22:50:23.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.655+0000 7f66b7400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f66b80b5450 0x7f66b80b43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:23.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.655+0000 7f66b7e00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f66b80cf590 0x7f66b806d600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:46822/0 (socket says 172.21.15.131:46822) 2024-05-26T22:50:23.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.655+0000 7f66b7e00640 1 -- 172.21.15.131:0/280211027 learned_addr learned my addr 172.21.15.131:0/280211027 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:50:23.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.656+0000 7f66b7400640 1 -- 172.21.15.131:0/280211027 >> v2:172.21.15.187:3300/0 conn(0x7f66b806f420 msgr2=0x7f66b80cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:23.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.656+0000 7f66b7400640 1 --2- 172.21.15.131:0/280211027 >> v2:172.21.15.187:3300/0 conn(0x7f66b806f420 0x7f66b80cfd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:23.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.656+0000 7f66b7400640 1 -- 172.21.15.131:0/280211027 >> v1:172.21.15.131:6789/0 conn(0x7f66b80cf590 legacy=0x7f66b806d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:23.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.656+0000 7f66b7400640 1 -- 172.21.15.131:0/280211027 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66b8160460 con 0x7f66b80b5450 2024-05-26T22:50:23.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.656+0000 7f66b6a00640 1 -- 172.21.15.131:0/280211027 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2150287511 0 0) 0x7f66b80b4900 con 0x7f66b80cf590 2024-05-26T22:50:23.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.657+0000 7f66b7400640 1 --2- 172.21.15.131:0/280211027 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f66b80b5450 0x7f66b80b43c0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f66b80b35b0 tx=0x7f66a802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2137a55813b278cf server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:23.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.657+0000 7f66b6a00640 1 -- 172.21.15.131:0/280211027 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a8037070 con 0x7f66b80b5450 2024-05-26T22:50:23.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.657+0000 7f66b6a00640 1 -- 172.21.15.131:0/280211027 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66a8032440 con 0x7f66b80b5450 2024-05-26T22:50:23.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.657+0000 7f66b6a00640 1 -- 172.21.15.131:0/280211027 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a8032810 con 0x7f66b80b5450 2024-05-26T22:50:23.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.657+0000 7f66be000640 1 -- 172.21.15.131:0/280211027 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f66b80b5450 msgr2=0x7f66b80b43c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:23.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.657+0000 7f66be000640 1 --2- 172.21.15.131:0/280211027 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f66b80b5450 0x7f66b80b43c0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f66b80b35b0 tx=0x7f66a802a1a0 comp rx=0 tx=0).stop 2024-05-26T22:50:23.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.658+0000 7f66be000640 1 -- 172.21.15.131:0/280211027 shutdown_connections 2024-05-26T22:50:23.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.658+0000 7f66be000640 1 --2- 172.21.15.131:0/280211027 >> v2:172.21.15.187:3300/0 conn(0x7f66b806f420 0x7f66b80cfd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:23.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.658+0000 7f66be000640 1 --2- 172.21.15.131:0/280211027 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f66b80b5450 0x7f66b80b43c0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:23.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.658+0000 7f66be000640 1 -- 172.21.15.131:0/280211027 >> 172.21.15.131:0/280211027 conn(0x7f66b806bc90 msgr2=0x7f66b80b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:23.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.658+0000 7f66be000640 1 -- 172.21.15.131:0/280211027 shutdown_connections 2024-05-26T22:50:23.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.658+0000 7f66be000640 1 -- 172.21.15.131:0/280211027 wait complete. 2024-05-26T22:50:23.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.659+0000 7f66be000640 1 Processor -- start 2024-05-26T22:50:23.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.659+0000 7f66be000640 1 -- start start 2024-05-26T22:50:23.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.660+0000 7f66be000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f66b806f420 0x7f66b809aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:23.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.660+0000 7f66be000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f66b80b5450 0x7f66b809c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:23.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.660+0000 7f66be000640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f66b80cf590 0x7f66b81483c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:23.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.660+0000 7f66be000640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f66b80b5830 con 0x7f66b80b5450 2024-05-26T22:50:23.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.660+0000 7f66be000640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f66b8160a20 con 0x7f66b80cf590 2024-05-26T22:50:23.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.660+0000 7f66be000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f66b815fde0 con 0x7f66b806f420 2024-05-26T22:50:23.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.661+0000 7f66aea00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f66b80b5450 0x7f66b809c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:23.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.661+0000 7f66aea00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f66b80b5450 0x7f66b809c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60810/0 (socket says 172.21.15.131:60810) 2024-05-26T22:50:23.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.661+0000 7f66aea00640 1 -- 172.21.15.131:0/4192557295 learned_addr learned my addr 172.21.15.131:0/4192557295 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:23.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.661+0000 7f66b7e00640 1 --2- 172.21.15.131:0/4192557295 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f66b80cf590 0x7f66b81483c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:23.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.661+0000 7f66aea00640 1 -- 172.21.15.131:0/4192557295 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f66b806f420 msgr2=0x7f66b809aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:23.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.661+0000 7f66aea00640 1 --2- 172.21.15.131:0/4192557295 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f66b806f420 0x7f66b809aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:23.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.661+0000 7f66aea00640 1 -- 172.21.15.131:0/4192557295 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f66b80cf590 msgr2=0x7f66b81483c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:23.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.661+0000 7f66aea00640 1 --2- 172.21.15.131:0/4192557295 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f66b80cf590 0x7f66b81483c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:23.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.661+0000 7f66aea00640 1 -- 172.21.15.131:0/4192557295 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66b809dc70 con 0x7f66b80b5450 2024-05-26T22:50:23.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.661+0000 7f66aea00640 1 --2- 172.21.15.131:0/4192557295 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f66b80b5450 0x7f66b809c970 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f66a000b410 tx=0x7f66a000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:23.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.662+0000 7f66afe00640 1 -- 172.21.15.131:0/4192557295 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a0015020 con 0x7f66b80b5450 2024-05-26T22:50:23.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.662+0000 7f66afe00640 1 -- 172.21.15.131:0/4192557295 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66a0011850 con 0x7f66b80b5450 2024-05-26T22:50:23.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.662+0000 7f66afe00640 1 -- 172.21.15.131:0/4192557295 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a0011c40 con 0x7f66b80b5450 2024-05-26T22:50:23.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.662+0000 7f66be000640 1 -- 172.21.15.131:0/4192557295 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66b809d3d0 con 0x7f66b80b5450 2024-05-26T22:50:23.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.662+0000 7f66be000640 1 -- 172.21.15.131:0/4192557295 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66b809d990 con 0x7f66b80b5450 2024-05-26T22:50:23.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.663+0000 7f66be000640 1 -- 172.21.15.131:0/4192557295 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66b8074e90 con 0x7f66b80b5450 2024-05-26T22:50:23.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.664+0000 7f66afe00640 1 -- 172.21.15.131:0/4192557295 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f66a0018020 con 0x7f66b80b5450 2024-05-26T22:50:23.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.665+0000 7f66afe00640 1 --2- 172.21.15.131:0/4192557295 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f669406be30 0x7f669406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:23.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.665+0000 7f66afe00640 1 -- 172.21.15.131:0/4192557295 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 4857+0+0 (secure 0 0 0) 0x7f66a008aa00 con 0x7f66b80b5450 2024-05-26T22:50:23.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.665+0000 7f66b7400640 1 --2- 172.21.15.131:0/4192557295 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f669406be30 0x7f669406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:23.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.665+0000 7f66b7400640 1 --2- 172.21.15.131:0/4192557295 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f669406be30 0x7f669406e2f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f66a8034000 tx=0x7f66a802c000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:23.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.669+0000 7f66afe00640 1 -- 172.21.15.131:0/4192557295 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f66a001f020 con 0x7f66b80b5450 2024-05-26T22:50:23.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.865+0000 7f66be000640 1 -- 172.21.15.131:0/4192557295 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f66b80695c0 con 0x7f66b80b5450 2024-05-26T22:50:23.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.866+0000 7f66afe00640 1 -- 172.21.15.131:0/4192557295 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v40) ==== 93+0+10 (secure 0 0 0) 0x7f66a00172c0 con 0x7f66b80b5450 2024-05-26T22:50:23.866 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2024-05-26T22:50:23.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.868+0000 7f66be000640 1 -- 172.21.15.131:0/4192557295 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f669406be30 msgr2=0x7f669406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:23.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.868+0000 7f66be000640 1 --2- 172.21.15.131:0/4192557295 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f669406be30 0x7f669406e2f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f66a8034000 tx=0x7f66a802c000 comp rx=0 tx=0).stop 2024-05-26T22:50:23.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.868+0000 7f66be000640 1 -- 172.21.15.131:0/4192557295 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f66b80b5450 msgr2=0x7f66b809c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:23.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.868+0000 7f66be000640 1 --2- 172.21.15.131:0/4192557295 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f66b80b5450 0x7f66b809c970 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f66a000b410 tx=0x7f66a000b8d0 comp rx=0 tx=0).stop 2024-05-26T22:50:23.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.869+0000 7f66be000640 1 -- 172.21.15.131:0/4192557295 shutdown_connections 2024-05-26T22:50:23.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.869+0000 7f66be000640 1 --2- 172.21.15.131:0/4192557295 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f669406be30 0x7f669406e2f0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:23.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.869+0000 7f66be000640 1 --2- 172.21.15.131:0/4192557295 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f66b80cf590 0x7f66b81483c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:23.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.869+0000 7f66be000640 1 --2- 172.21.15.131:0/4192557295 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f66b80b5450 0x7f66b809c970 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:23.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.869+0000 7f66be000640 1 --2- 172.21.15.131:0/4192557295 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f66b806f420 0x7f66b809aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:23.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.869+0000 7f66be000640 1 -- 172.21.15.131:0/4192557295 >> 172.21.15.131:0/4192557295 conn(0x7f66b806bc90 msgr2=0x7f66b815edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:23.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.869+0000 7f66be000640 1 -- 172.21.15.131:0/4192557295 shutdown_connections 2024-05-26T22:50:23.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:23.869+0000 7f66be000640 1 -- 172.21.15.131:0/4192557295 wait complete. 2024-05-26T22:50:23.887 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-26T22:50:24.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.083+0000 7f5beb000640 1 Processor -- start 2024-05-26T22:50:24.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.083+0000 7f5beb000640 1 -- start start 2024-05-26T22:50:24.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.084+0000 7f5beb000640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f5be4155840 0x7f5be4071930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:24.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.085+0000 7f5beb000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5be415c3d0 0x7f5be406ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:24.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.085+0000 7f5beb000640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5be40722a0 con 0x7f5be406f490 2024-05-26T22:50:24.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.085+0000 7f5beb000640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f5be4072580 con 0x7f5be4155840 2024-05-26T22:50:24.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.085+0000 7f5beb000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f5be4072860 con 0x7f5be415c3d0 2024-05-26T22:50:24.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.085+0000 7f5be3e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f5be4155840 0x7f5be4071930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:24.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.085+0000 7f5be3e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f5be4155840 0x7f5be4071930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60822/0 (socket says 172.21.15.131:60822) 2024-05-26T22:50:24.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.085+0000 7f5be3e00640 1 -- 172.21.15.131:0/3188399051 learned_addr learned my addr 172.21.15.131:0/3188399051 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:24.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.085+0000 7f5be3400640 1 --2- 172.21.15.131:0/3188399051 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5be415c3d0 0x7f5be406ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:24.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.085+0000 7f5be3e00640 1 -- 172.21.15.131:0/3188399051 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5be415c3d0 msgr2=0x7f5be406ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:24.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.085+0000 7f5be3e00640 1 --2- 172.21.15.131:0/3188399051 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5be415c3d0 0x7f5be406ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:24.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.085+0000 7f5be3e00640 1 -- 172.21.15.131:0/3188399051 >> v1:172.21.15.187:6789/0 conn(0x7f5be406f490 legacy=0x7f5be415ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:24.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.085+0000 7f5be3e00640 1 -- 172.21.15.131:0/3188399051 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5be4072b40 con 0x7f5be4155840 2024-05-26T22:50:24.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.085+0000 7f5be3400640 1 --2- 172.21.15.131:0/3188399051 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5be415c3d0 0x7f5be406ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:50:24.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.086+0000 7f5be3e00640 1 --2- 172.21.15.131:0/3188399051 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5be4155840 0x7f5be4071930 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f5bd400bb10 tx=0x7f5bd402c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2baf59d15ba6db2b server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:24.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.087+0000 7f5be2a00640 1 -- 172.21.15.131:0/3188399051 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bd4032790 con 0x7f5be4155840 2024-05-26T22:50:24.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.087+0000 7f5be2a00640 1 -- 172.21.15.131:0/3188399051 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bd4038070 con 0x7f5be4155840 2024-05-26T22:50:24.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.087+0000 7f5be2a00640 1 -- 172.21.15.131:0/3188399051 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bd4032b50 con 0x7f5be4155840 2024-05-26T22:50:24.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.087+0000 7f5beb000640 1 -- 172.21.15.131:0/3188399051 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5be4155840 msgr2=0x7f5be4071930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:24.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.087+0000 7f5beb000640 1 --2- 172.21.15.131:0/3188399051 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5be4155840 0x7f5be4071930 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f5bd400bb10 tx=0x7f5bd402c8d0 comp rx=0 tx=0).stop 2024-05-26T22:50:24.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.087+0000 7f5beb000640 1 -- 172.21.15.131:0/3188399051 shutdown_connections 2024-05-26T22:50:24.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.087+0000 7f5beb000640 1 --2- 172.21.15.131:0/3188399051 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5be415c3d0 0x7f5be406ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:24.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.087+0000 7f5beb000640 1 --2- 172.21.15.131:0/3188399051 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5be4155840 0x7f5be4071930 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:24.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.087+0000 7f5beb000640 1 -- 172.21.15.131:0/3188399051 >> 172.21.15.131:0/3188399051 conn(0x7f5be40b1e90 msgr2=0x7f5be40b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:24.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.088+0000 7f5beb000640 1 -- 172.21.15.131:0/3188399051 shutdown_connections 2024-05-26T22:50:24.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.088+0000 7f5beb000640 1 -- 172.21.15.131:0/3188399051 wait complete. 2024-05-26T22:50:24.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.089+0000 7f5beb000640 1 Processor -- start 2024-05-26T22:50:24.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.089+0000 7f5beb000640 1 -- start start 2024-05-26T22:50:24.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.090+0000 7f5beb000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5be406f490 0x7f5be414e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:24.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.090+0000 7f5beb000640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5be4155840 0x7f5be40cda90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:24.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.090+0000 7f5beb000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5be415c3d0 0x7f5be40cb390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:24.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.090+0000 7f5beb000640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f5be4071ff0 con 0x7f5be406f490 2024-05-26T22:50:24.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.090+0000 7f5beb000640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5be4071e70 con 0x7f5be4155840 2024-05-26T22:50:24.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.090+0000 7f5beb000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f5be40aee50 con 0x7f5be415c3d0 2024-05-26T22:50:24.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.091+0000 7f5be3e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5be406f490 0x7f5be414e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:24.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.091+0000 7f5be3e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5be406f490 0x7f5be414e0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60826/0 (socket says 172.21.15.131:60826) 2024-05-26T22:50:24.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.091+0000 7f5be3e00640 1 -- 172.21.15.131:0/4092104482 learned_addr learned my addr 172.21.15.131:0/4092104482 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:24.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.091+0000 7f5be3e00640 1 -- 172.21.15.131:0/4092104482 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5be415c3d0 msgr2=0x7f5be40cb390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:24.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.091+0000 7f5be3e00640 1 --2- 172.21.15.131:0/4092104482 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5be415c3d0 0x7f5be40cb390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:24.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.091+0000 7f5be3e00640 1 -- 172.21.15.131:0/4092104482 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5be4155840 msgr2=0x7f5be40cda90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:24.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.091+0000 7f5be3e00640 1 --2- 172.21.15.131:0/4092104482 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5be4155840 0x7f5be40cda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:24.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.091+0000 7f5be3e00640 1 -- 172.21.15.131:0/4092104482 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5be40cee90 con 0x7f5be406f490 2024-05-26T22:50:24.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.091+0000 7f5be3e00640 1 --2- 172.21.15.131:0/4092104482 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5be406f490 0x7f5be414e0f0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f5bd4000c00 tx=0x7f5bd4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:24.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.091+0000 7f5bdbe00640 1 -- 172.21.15.131:0/4092104482 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bd4043070 con 0x7f5be406f490 2024-05-26T22:50:24.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.092+0000 7f5bdbe00640 1 -- 172.21.15.131:0/4092104482 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bd403c640 con 0x7f5be406f490 2024-05-26T22:50:24.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.092+0000 7f5beb000640 1 -- 172.21.15.131:0/4092104482 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5be40cccd0 con 0x7f5be406f490 2024-05-26T22:50:24.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.092+0000 7f5bdbe00640 1 -- 172.21.15.131:0/4092104482 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bd403ca30 con 0x7f5be406f490 2024-05-26T22:50:24.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.092+0000 7f5beb000640 1 -- 172.21.15.131:0/4092104482 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5be40cd290 con 0x7f5be406f490 2024-05-26T22:50:24.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.093+0000 7f5beb000640 1 -- 172.21.15.131:0/4092104482 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5be40694e0 con 0x7f5be406f490 2024-05-26T22:50:24.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.094+0000 7f5bdbe00640 1 -- 172.21.15.131:0/4092104482 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5bd4046070 con 0x7f5be406f490 2024-05-26T22:50:24.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.094+0000 7f5bdbe00640 1 --2- 172.21.15.131:0/4092104482 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f5bc406be30 0x7f5bc406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:24.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.095+0000 7f5bdbe00640 1 -- 172.21.15.131:0/4092104482 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 4857+0+0 (secure 0 0 0) 0x7f5bd40ab1d0 con 0x7f5be406f490 2024-05-26T22:50:24.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.095+0000 7f5be3400640 1 --2- 172.21.15.131:0/4092104482 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f5bc406be30 0x7f5bc406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:24.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.096+0000 7f5be3400640 1 --2- 172.21.15.131:0/4092104482 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f5bc406be30 0x7f5bc406e2f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f5bd0006fd0 tx=0x7f5bd0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:24.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.099+0000 7f5bdbe00640 1 -- 172.21.15.131:0/4092104482 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f5be40cee90 con 0x7f5be406f490 2024-05-26T22:50:24.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.299+0000 7f5beb000640 1 -- 172.21.15.131:0/4092104482 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f5be406e5c0 con 0x7f5be406f490 2024-05-26T22:50:24.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.300+0000 7f5bdbe00640 1 -- 172.21.15.131:0/4092104482 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v40) ==== 84+0+65 (secure 0 0 0) 0x7f5bd4079ec0 con 0x7f5be406f490 2024-05-26T22:50:24.300 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:50:24.300 INFO:teuthology.orchestra.run.smithi131.stdout:[ 2024-05-26T22:50:24.300 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:50:24.300 INFO:teuthology.orchestra.run.smithi131.stdout: "poolnum": 1, 2024-05-26T22:50:24.300 INFO:teuthology.orchestra.run.smithi131.stdout: "poolname": ".mgr" 2024-05-26T22:50:24.300 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:50:24.300 INFO:teuthology.orchestra.run.smithi131.stdout:] 2024-05-26T22:50:24.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.302+0000 7f5beb000640 1 -- 172.21.15.131:0/4092104482 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f5bc406be30 msgr2=0x7f5bc406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:24.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.302+0000 7f5beb000640 1 --2- 172.21.15.131:0/4092104482 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f5bc406be30 0x7f5bc406e2f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f5bd0006fd0 tx=0x7f5bd0008040 comp rx=0 tx=0).stop 2024-05-26T22:50:24.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.302+0000 7f5beb000640 1 -- 172.21.15.131:0/4092104482 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5be406f490 msgr2=0x7f5be414e0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:24.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.302+0000 7f5beb000640 1 --2- 172.21.15.131:0/4092104482 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5be406f490 0x7f5be414e0f0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f5bd4000c00 tx=0x7f5bd4004730 comp rx=0 tx=0).stop 2024-05-26T22:50:24.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.303+0000 7f5beb000640 1 -- 172.21.15.131:0/4092104482 shutdown_connections 2024-05-26T22:50:24.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.303+0000 7f5beb000640 1 --2- 172.21.15.131:0/4092104482 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5be415c3d0 0x7f5be40cb390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:24.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.303+0000 7f5beb000640 1 --2- 172.21.15.131:0/4092104482 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f5bc406be30 0x7f5bc406e2f0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:24.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.303+0000 7f5beb000640 1 --2- 172.21.15.131:0/4092104482 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5be4155840 0x7f5be40cda90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:24.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.303+0000 7f5beb000640 1 --2- 172.21.15.131:0/4092104482 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5be406f490 0x7f5be414e0f0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:24.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.303+0000 7f5beb000640 1 -- 172.21.15.131:0/4092104482 >> 172.21.15.131:0/4092104482 conn(0x7f5be40b1e90 msgr2=0x7f5be415cd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:24.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.303+0000 7f5beb000640 1 -- 172.21.15.131:0/4092104482 shutdown_connections 2024-05-26T22:50:24.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.303+0000 7f5beb000640 1 -- 172.21.15.131:0/4092104482 wait complete. 2024-05-26T22:50:24.320 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-26T22:50:24.320 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-26T22:50:24.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.528+0000 7fd66fe00640 1 Processor -- start 2024-05-26T22:50:24.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.528+0000 7fd66fe00640 1 -- start start 2024-05-26T22:50:24.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.529+0000 7fd66fe00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fd67013dee0 0x7fd67006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:24.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.529+0000 7fd66fe00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fd67015bdd0 0x7fd6700ce7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:24.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.529+0000 7fd66fe00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd67013e2c0 con 0x7fd67006da20 2024-05-26T22:50:24.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.529+0000 7fd66fe00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fd67015c430 con 0x7fd67013dee0 2024-05-26T22:50:24.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.529+0000 7fd66fe00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fd67015eb60 con 0x7fd67015bdd0 2024-05-26T22:50:24.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.529+0000 7fd66ea00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fd67013dee0 0x7fd67006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:24.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.529+0000 7fd66ea00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fd67013dee0 0x7fd67006d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49868/0 (socket says 172.21.15.131:49868) 2024-05-26T22:50:24.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.529+0000 7fd66ea00640 1 -- 172.21.15.131:0/2600596613 learned_addr learned my addr 172.21.15.131:0/2600596613 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:24.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.530+0000 7fd66f400640 1 --2- 172.21.15.131:0/2600596613 >> v2:172.21.15.187:3300/0 conn(0x7fd67015bdd0 0x7fd6700ce7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:24.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.530+0000 7fd66ea00640 1 -- 172.21.15.131:0/2600596613 >> v2:172.21.15.187:3300/0 conn(0x7fd67015bdd0 msgr2=0x7fd6700ce7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:24.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.530+0000 7fd66ea00640 1 --2- 172.21.15.131:0/2600596613 >> v2:172.21.15.187:3300/0 conn(0x7fd67015bdd0 0x7fd6700ce7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:24.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.530+0000 7fd66ea00640 1 -- 172.21.15.131:0/2600596613 >> v1:172.21.15.187:6789/0 conn(0x7fd67006da20 legacy=0x7fd67015b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:24.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.530+0000 7fd66ea00640 1 -- 172.21.15.131:0/2600596613 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd67015ee40 con 0x7fd67013dee0 2024-05-26T22:50:24.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.531+0000 7fd66ea00640 1 --2- 172.21.15.131:0/2600596613 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd67013dee0 0x7fd67006d4e0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fd6700afb30 tx=0x7fd66002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8efdb45728d619a server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:24.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.531+0000 7fd66e000640 1 -- 172.21.15.131:0/2600596613 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd660037070 con 0x7fd67013dee0 2024-05-26T22:50:24.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.531+0000 7fd66e000640 1 -- 172.21.15.131:0/2600596613 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd660032440 con 0x7fd67013dee0 2024-05-26T22:50:24.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.531+0000 7fd66e000640 1 -- 172.21.15.131:0/2600596613 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd660032810 con 0x7fd67013dee0 2024-05-26T22:50:24.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.532+0000 7fd66fe00640 1 -- 172.21.15.131:0/2600596613 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd67013dee0 msgr2=0x7fd67006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:24.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.532+0000 7fd66fe00640 1 --2- 172.21.15.131:0/2600596613 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd67013dee0 0x7fd67006d4e0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fd6700afb30 tx=0x7fd66002a230 comp rx=0 tx=0).stop 2024-05-26T22:50:24.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.532+0000 7fd66fe00640 1 -- 172.21.15.131:0/2600596613 shutdown_connections 2024-05-26T22:50:24.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.532+0000 7fd66fe00640 1 --2- 172.21.15.131:0/2600596613 >> v2:172.21.15.187:3300/0 conn(0x7fd67015bdd0 0x7fd6700ce7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:24.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.532+0000 7fd66fe00640 1 --2- 172.21.15.131:0/2600596613 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd67013dee0 0x7fd67006d4e0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:24.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.532+0000 7fd66fe00640 1 -- 172.21.15.131:0/2600596613 >> 172.21.15.131:0/2600596613 conn(0x7fd6700b5410 msgr2=0x7fd6700b57f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:24.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.532+0000 7fd66fe00640 1 -- 172.21.15.131:0/2600596613 shutdown_connections 2024-05-26T22:50:24.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.532+0000 7fd66fe00640 1 -- 172.21.15.131:0/2600596613 wait complete. 2024-05-26T22:50:24.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.534+0000 7fd66fe00640 1 Processor -- start 2024-05-26T22:50:24.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.534+0000 7fd66fe00640 1 -- start start 2024-05-26T22:50:24.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.535+0000 7fd66fe00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd67006da20 0x7fd67013ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:24.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.535+0000 7fd66fe00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd67013dee0 0x7fd6700af8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:24.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.535+0000 7fd66fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd67015bdd0 0x7fd67013cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:24.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.535+0000 7fd66fe00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fd67015f920 con 0x7fd67015bdd0 2024-05-26T22:50:24.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.535+0000 7fd66fe00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd67015f400 con 0x7fd67013dee0 2024-05-26T22:50:24.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.535+0000 7fd66fe00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd6700b09d0 con 0x7fd67006da20 2024-05-26T22:50:24.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.535+0000 7fd66f400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd67015bdd0 0x7fd67013cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:24.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.535+0000 7fd66f400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd67015bdd0 0x7fd67013cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49884/0 (socket says 172.21.15.131:49884) 2024-05-26T22:50:24.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.535+0000 7fd66f400640 1 -- 172.21.15.131:0/330640534 learned_addr learned my addr 172.21.15.131:0/330640534 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:24.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.535+0000 7fd667e00640 1 --2- 172.21.15.131:0/330640534 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd67013dee0 0x7fd6700af8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:24.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.536+0000 7fd66ea00640 1 --2- 172.21.15.131:0/330640534 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd67006da20 0x7fd67013ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:24.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.536+0000 7fd66f400640 1 -- 172.21.15.131:0/330640534 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd67006da20 msgr2=0x7fd67013ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:24.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.536+0000 7fd66f400640 1 --2- 172.21.15.131:0/330640534 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd67006da20 0x7fd67013ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:24.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.536+0000 7fd66f400640 1 -- 172.21.15.131:0/330640534 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd67013dee0 msgr2=0x7fd6700af8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:24.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.536+0000 7fd66f400640 1 --2- 172.21.15.131:0/330640534 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd67013dee0 0x7fd6700af8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:24.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.536+0000 7fd66f400640 1 -- 172.21.15.131:0/330640534 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6700b0130 con 0x7fd67015bdd0 2024-05-26T22:50:24.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.536+0000 7fd667e00640 1 --2- 172.21.15.131:0/330640534 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd67013dee0 0x7fd6700af8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:50:24.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.536+0000 7fd66ea00640 1 --2- 172.21.15.131:0/330640534 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd67006da20 0x7fd67013ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:50:24.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.536+0000 7fd66f400640 1 --2- 172.21.15.131:0/330640534 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd67015bdd0 0x7fd67013cc20 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fd66800cc60 tx=0x7fd66800a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:24.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.536+0000 7fd666a00640 1 -- 172.21.15.131:0/330640534 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd668003990 con 0x7fd67015bdd0 2024-05-26T22:50:24.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.536+0000 7fd666a00640 1 -- 172.21.15.131:0/330640534 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd668003b30 con 0x7fd67015bdd0 2024-05-26T22:50:24.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.536+0000 7fd666a00640 1 -- 172.21.15.131:0/330640534 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd668004020 con 0x7fd67015bdd0 2024-05-26T22:50:24.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.536+0000 7fd66fe00640 1 -- 172.21.15.131:0/330640534 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6700b0410 con 0x7fd67015bdd0 2024-05-26T22:50:24.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.536+0000 7fd66fe00640 1 -- 172.21.15.131:0/330640534 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6700afe20 con 0x7fd67015bdd0 2024-05-26T22:50:24.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.538+0000 7fd66fe00640 1 -- 172.21.15.131:0/330640534 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6700694e0 con 0x7fd67015bdd0 2024-05-26T22:50:24.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.539+0000 7fd666a00640 1 -- 172.21.15.131:0/330640534 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd6680041c0 con 0x7fd67015bdd0 2024-05-26T22:50:24.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.539+0000 7fd666a00640 1 --2- 172.21.15.131:0/330640534 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fd64c06be30 0x7fd64c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:24.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.540+0000 7fd666a00640 1 -- 172.21.15.131:0/330640534 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 4857+0+0 (secure 0 0 0) 0x7fd66808aec0 con 0x7fd67015bdd0 2024-05-26T22:50:24.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.540+0000 7fd66ea00640 1 --2- 172.21.15.131:0/330640534 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fd64c06be30 0x7fd64c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:24.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.540+0000 7fd66ea00640 1 --2- 172.21.15.131:0/330640534 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fd64c06be30 0x7fd64c06e2f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fd660034000 tx=0x7fd66002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:24.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.544+0000 7fd666a00640 1 -- 172.21.15.131:0/330640534 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fd668014030 con 0x7fd67015bdd0 2024-05-26T22:50:24.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:24.742+0000 7fd66fe00640 1 -- 172.21.15.131:0/330640534 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fd67015faa0 con 0x7fd67015bdd0 2024-05-26T22:50:25.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:25.539+0000 7fd666a00640 1 -- 172.21.15.131:0/330640534 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v41) ==== 138+0+0 (secure 0 0 0) 0x7fd668059c50 con 0x7fd67015bdd0 2024-05-26T22:50:25.539 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_metadata' created 2024-05-26T22:50:25.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:25.543+0000 7fd66fe00640 1 -- 172.21.15.131:0/330640534 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fd64c06be30 msgr2=0x7fd64c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:25.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:25.543+0000 7fd66fe00640 1 --2- 172.21.15.131:0/330640534 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fd64c06be30 0x7fd64c06e2f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fd660034000 tx=0x7fd66002e000 comp rx=0 tx=0).stop 2024-05-26T22:50:25.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:25.544+0000 7fd66fe00640 1 -- 172.21.15.131:0/330640534 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd67015bdd0 msgr2=0x7fd67013cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:25.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:25.544+0000 7fd66fe00640 1 --2- 172.21.15.131:0/330640534 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd67015bdd0 0x7fd67013cc20 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fd66800cc60 tx=0x7fd66800a5c0 comp rx=0 tx=0).stop 2024-05-26T22:50:25.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:25.544+0000 7fd66fe00640 1 -- 172.21.15.131:0/330640534 shutdown_connections 2024-05-26T22:50:25.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:25.544+0000 7fd66fe00640 1 --2- 172.21.15.131:0/330640534 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fd64c06be30 0x7fd64c06e2f0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:25.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:25.544+0000 7fd66fe00640 1 --2- 172.21.15.131:0/330640534 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd67015bdd0 0x7fd67013cc20 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:25.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:25.544+0000 7fd66fe00640 1 --2- 172.21.15.131:0/330640534 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd67013dee0 0x7fd6700af8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:25.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:25.544+0000 7fd66fe00640 1 --2- 172.21.15.131:0/330640534 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd67006da20 0x7fd67013ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:25.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:25.544+0000 7fd66fe00640 1 -- 172.21.15.131:0/330640534 >> 172.21.15.131:0/330640534 conn(0x7fd6700b5410 msgr2=0x7fd67015aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:25.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:25.544+0000 7fd66fe00640 1 -- 172.21.15.131:0/330640534 shutdown_connections 2024-05-26T22:50:25.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:25.545+0000 7fd66fe00640 1 -- 172.21.15.131:0/330640534 wait complete. 2024-05-26T22:50:25.570 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-26T22:50:25.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:25.774+0000 7f9b55a00640 1 Processor -- start 2024-05-26T22:50:25.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:25.774+0000 7f9b55a00640 1 -- start start 2024-05-26T22:50:25.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:25.776+0000 7f9b55a00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f9b500b5450 0x7f9b500b43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:25.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:25.776+0000 7f9b55a00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f9b5006f420 0x7f9b500cfd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:25.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:25.776+0000 7f9b55a00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b500b4900 con 0x7f9b500cf590 2024-05-26T22:50:25.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:25.776+0000 7f9b55a00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f9b50160d90 con 0x7f9b500b5450 2024-05-26T22:50:25.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:25.776+0000 7f9b55a00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f9b50160180 con 0x7f9b5006f420 2024-05-26T22:50:25.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:25.776+0000 7f9b4ea00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f9b500b5450 0x7f9b500b43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:25.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:25.776+0000 7f9b4ea00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f9b500b5450 0x7f9b500b43c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49892/0 (socket says 172.21.15.131:49892) 2024-05-26T22:50:25.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:25.776+0000 7f9b4ea00640 1 -- 172.21.15.131:0/1702417018 learned_addr learned my addr 172.21.15.131:0/1702417018 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:25.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:25.777+0000 7f9b4e000640 1 --2- 172.21.15.131:0/1702417018 >> v2:172.21.15.187:3300/0 conn(0x7f9b5006f420 0x7f9b500cfd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:25.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:25.777+0000 7f9b4ea00640 1 -- 172.21.15.131:0/1702417018 >> v2:172.21.15.187:3300/0 conn(0x7f9b5006f420 msgr2=0x7f9b500cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:25.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:25.777+0000 7f9b4ea00640 1 --2- 172.21.15.131:0/1702417018 >> v2:172.21.15.187:3300/0 conn(0x7f9b5006f420 0x7f9b500cfd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:25.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:25.777+0000 7f9b4ea00640 1 -- 172.21.15.131:0/1702417018 >> v1:172.21.15.131:6789/0 conn(0x7f9b500cf590 legacy=0x7f9b5006d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:25.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:25.777+0000 7f9b4ea00640 1 -- 172.21.15.131:0/1702417018 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b50160460 con 0x7f9b500b5450 2024-05-26T22:50:25.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:25.777+0000 7f9b4d600640 1 -- 172.21.15.131:0/1702417018 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2644898546 0 0) 0x7f9b500b4900 con 0x7f9b500cf590 2024-05-26T22:50:25.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:25.779+0000 7f9b4ea00640 1 --2- 172.21.15.131:0/1702417018 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b500b5450 0x7f9b500b43c0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f9b500b35b0 tx=0x7f9b4002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11155c955e8d96e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:25.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:25.779+0000 7f9b4d600640 1 -- 172.21.15.131:0/1702417018 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b40037070 con 0x7f9b500b5450 2024-05-26T22:50:25.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:25.779+0000 7f9b4d600640 1 -- 172.21.15.131:0/1702417018 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b40032440 con 0x7f9b500b5450 2024-05-26T22:50:25.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:25.779+0000 7f9b4d600640 1 -- 172.21.15.131:0/1702417018 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b40032810 con 0x7f9b500b5450 2024-05-26T22:50:25.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:25.780+0000 7f9b55a00640 1 -- 172.21.15.131:0/1702417018 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b500b5450 msgr2=0x7f9b500b43c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:25.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:25.780+0000 7f9b55a00640 1 --2- 172.21.15.131:0/1702417018 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b500b5450 0x7f9b500b43c0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f9b500b35b0 tx=0x7f9b4002a270 comp rx=0 tx=0).stop 2024-05-26T22:50:25.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:25.780+0000 7f9b55a00640 1 -- 172.21.15.131:0/1702417018 shutdown_connections 2024-05-26T22:50:25.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:25.780+0000 7f9b55a00640 1 --2- 172.21.15.131:0/1702417018 >> v2:172.21.15.187:3300/0 conn(0x7f9b5006f420 0x7f9b500cfd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:25.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:25.780+0000 7f9b55a00640 1 --2- 172.21.15.131:0/1702417018 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b500b5450 0x7f9b500b43c0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:25.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:25.780+0000 7f9b55a00640 1 -- 172.21.15.131:0/1702417018 >> 172.21.15.131:0/1702417018 conn(0x7f9b5006bc90 msgr2=0x7f9b500b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:25.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:25.780+0000 7f9b55a00640 1 -- 172.21.15.131:0/1702417018 shutdown_connections 2024-05-26T22:50:25.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:25.780+0000 7f9b55a00640 1 -- 172.21.15.131:0/1702417018 wait complete. 2024-05-26T22:50:25.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:25.782+0000 7f9b55a00640 1 Processor -- start 2024-05-26T22:50:25.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:25.782+0000 7f9b55a00640 1 -- start start 2024-05-26T22:50:25.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:25.783+0000 7f9b55a00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9b5006f420 0x7f9b500c4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:25.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:25.783+0000 7f9b55a00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9b500b5450 0x7f9b501f7450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:25.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:25.783+0000 7f9b55a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b500cf590 0x7f9b501f7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:25.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:25.783+0000 7f9b55a00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f9b500b5830 con 0x7f9b500cf590 2024-05-26T22:50:25.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:25.783+0000 7f9b55a00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9b50160a20 con 0x7f9b500b5450 2024-05-26T22:50:25.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:25.783+0000 7f9b55a00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9b5015fde0 con 0x7f9b5006f420 2024-05-26T22:50:25.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:25.783+0000 7f9b4f400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b500cf590 0x7f9b501f7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:25.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:25.783+0000 7f9b4f400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b500cf590 0x7f9b501f7990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49900/0 (socket says 172.21.15.131:49900) 2024-05-26T22:50:25.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:25.783+0000 7f9b4f400640 1 -- 172.21.15.131:0/1166407197 learned_addr learned my addr 172.21.15.131:0/1166407197 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:25.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:25.783+0000 7f9b4e000640 1 --2- 172.21.15.131:0/1166407197 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9b500b5450 0x7f9b501f7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:25.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:25.783+0000 7f9b4ea00640 1 --2- 172.21.15.131:0/1166407197 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9b5006f420 0x7f9b500c4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:25.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:25.784+0000 7f9b4f400640 1 -- 172.21.15.131:0/1166407197 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9b5006f420 msgr2=0x7f9b500c4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:25.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:25.784+0000 7f9b4f400640 1 --2- 172.21.15.131:0/1166407197 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9b5006f420 0x7f9b500c4e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:25.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:25.784+0000 7f9b4f400640 1 -- 172.21.15.131:0/1166407197 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9b500b5450 msgr2=0x7f9b501f7450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:25.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:25.784+0000 7f9b4f400640 1 --2- 172.21.15.131:0/1166407197 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9b500b5450 0x7f9b501f7450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:25.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:25.784+0000 7f9b4f400640 1 -- 172.21.15.131:0/1166407197 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b5009c9d0 con 0x7f9b500cf590 2024-05-26T22:50:25.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:25.784+0000 7f9b4e000640 1 --2- 172.21.15.131:0/1166407197 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9b500b5450 0x7f9b501f7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-26T22:50:25.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:25.784+0000 7f9b4f400640 1 --2- 172.21.15.131:0/1166407197 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b500cf590 0x7f9b501f7990 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f9b4800c760 tx=0x7f9b4800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:25.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:25.784+0000 7f9b46a00640 1 -- 172.21.15.131:0/1166407197 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b48003da0 con 0x7f9b500cf590 2024-05-26T22:50:25.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:25.784+0000 7f9b46a00640 1 -- 172.21.15.131:0/1166407197 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b48005070 con 0x7f9b500cf590 2024-05-26T22:50:25.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:25.784+0000 7f9b46a00640 1 -- 172.21.15.131:0/1166407197 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b48005460 con 0x7f9b500cf590 2024-05-26T22:50:25.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:25.784+0000 7f9b55a00640 1 -- 172.21.15.131:0/1166407197 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b5009ccb0 con 0x7f9b500cf590 2024-05-26T22:50:25.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:25.785+0000 7f9b55a00640 1 -- 172.21.15.131:0/1166407197 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b5009d270 con 0x7f9b500cf590 2024-05-26T22:50:25.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:25.786+0000 7f9b55a00640 1 -- 172.21.15.131:0/1166407197 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b50074e90 con 0x7f9b500cf590 2024-05-26T22:50:25.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:25.787+0000 7f9b46a00640 1 -- 172.21.15.131:0/1166407197 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9b48005600 con 0x7f9b500cf590 2024-05-26T22:50:25.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:25.788+0000 7f9b46a00640 1 --2- 172.21.15.131:0/1166407197 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9b3006be30 0x7f9b3006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:25.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:25.788+0000 7f9b46a00640 1 -- 172.21.15.131:0/1166407197 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 5227+0+0 (secure 0 0 0) 0x7f9b4808ac30 con 0x7f9b500cf590 2024-05-26T22:50:25.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:25.788+0000 7f9b4ea00640 1 --2- 172.21.15.131:0/1166407197 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9b3006be30 0x7f9b3006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:25.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:25.789+0000 7f9b4ea00640 1 --2- 172.21.15.131:0/1166407197 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9b3006be30 0x7f9b3006e2f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f9b40034000 tx=0x7f9b4002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:25.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:25.793+0000 7f9b46a00640 1 -- 172.21.15.131:0/1166407197 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f9b480597b0 con 0x7f9b500cf590 2024-05-26T22:50:25.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:25.987+0000 7f9b55a00640 1 -- 172.21.15.131:0/1166407197 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f9b500c5420 con 0x7f9b500cf590 2024-05-26T22:50:26.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:26.546+0000 7f9b46a00640 1 -- 172.21.15.131:0/1166407197 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v42) ==== 170+0+0 (secure 0 0 0) 0x7f9b500c5420 con 0x7f9b500cf590 2024-05-26T22:50:26.546 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_data' created 2024-05-26T22:50:26.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:26.549+0000 7f9b55a00640 1 -- 172.21.15.131:0/1166407197 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9b3006be30 msgr2=0x7f9b3006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:26.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:26.549+0000 7f9b55a00640 1 --2- 172.21.15.131:0/1166407197 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9b3006be30 0x7f9b3006e2f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f9b40034000 tx=0x7f9b4002e000 comp rx=0 tx=0).stop 2024-05-26T22:50:26.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:26.549+0000 7f9b55a00640 1 -- 172.21.15.131:0/1166407197 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b500cf590 msgr2=0x7f9b501f7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:26.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:26.549+0000 7f9b55a00640 1 --2- 172.21.15.131:0/1166407197 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b500cf590 0x7f9b501f7990 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f9b4800c760 tx=0x7f9b4800cc20 comp rx=0 tx=0).stop 2024-05-26T22:50:26.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:26.550+0000 7f9b55a00640 1 -- 172.21.15.131:0/1166407197 shutdown_connections 2024-05-26T22:50:26.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:26.550+0000 7f9b55a00640 1 --2- 172.21.15.131:0/1166407197 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9b3006be30 0x7f9b3006e2f0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:26.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:26.550+0000 7f9b55a00640 1 --2- 172.21.15.131:0/1166407197 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b500cf590 0x7f9b501f7990 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:26.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:26.550+0000 7f9b55a00640 1 --2- 172.21.15.131:0/1166407197 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9b500b5450 0x7f9b501f7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:26.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:26.550+0000 7f9b55a00640 1 --2- 172.21.15.131:0/1166407197 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9b5006f420 0x7f9b500c4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:26.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:26.550+0000 7f9b55a00640 1 -- 172.21.15.131:0/1166407197 >> 172.21.15.131:0/1166407197 conn(0x7f9b5006bc90 msgr2=0x7f9b500cfbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:26.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:26.550+0000 7f9b55a00640 1 -- 172.21.15.131:0/1166407197 shutdown_connections 2024-05-26T22:50:26.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:26.550+0000 7f9b55a00640 1 -- 172.21.15.131:0/1166407197 wait complete. 2024-05-26T22:50:26.582 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-26T22:50:26.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:26.759+0000 7ff37e000640 1 Processor -- start 2024-05-26T22:50:26.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:26.759+0000 7ff37e000640 1 -- start start 2024-05-26T22:50:26.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:26.761+0000 7ff37e000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff3780b5450 0x7ff3780b43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:26.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:26.761+0000 7ff37e000640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7ff3780d0460 0x7ff37806d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:26.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:26.761+0000 7ff37e000640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3780b4900 con 0x7ff37806f420 2024-05-26T22:50:26.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:26.761+0000 7ff37e000640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7ff378160d90 con 0x7ff3780d0460 2024-05-26T22:50:26.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:26.761+0000 7ff37e000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff378160180 con 0x7ff3780b5450 2024-05-26T22:50:26.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:26.761+0000 7ff376a00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7ff37806f420 0x7ff3780cfd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:38424/0 (socket says 172.21.15.131:38424) 2024-05-26T22:50:26.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:26.761+0000 7ff376a00640 1 -- 172.21.15.131:0/1710004884 learned_addr learned my addr 172.21.15.131:0/1710004884 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:50:26.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:26.762+0000 7ff376000640 1 -- 172.21.15.131:0/1710004884 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1703714367 0 0) 0x7ff3780b4900 con 0x7ff37806f420 2024-05-26T22:50:26.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:26.762+0000 7ff376000640 1 -- 172.21.15.131:0/1710004884 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff35c003520 con 0x7ff37806f420 2024-05-26T22:50:26.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:26.762+0000 7ff377e00640 1 --2- 172.21.15.131:0/1710004884 >> v2:172.21.15.187:3300/0 conn(0x7ff3780d0460 0x7ff37806d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:26.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:26.762+0000 7ff377400640 1 --2- 172.21.15.131:0/1710004884 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff3780b5450 0x7ff3780b43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:26.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:26.762+0000 7ff376000640 1 -- 172.21.15.131:0/1710004884 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1810081430 0 0) 0x7ff35c003520 con 0x7ff37806f420 2024-05-26T22:50:26.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:26.762+0000 7ff376000640 1 -- 172.21.15.131:0/1710004884 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff3780b5450 msgr2=0x7ff3780b43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:26.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:26.762+0000 7ff376000640 1 --2- 172.21.15.131:0/1710004884 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff3780b5450 0x7ff3780b43c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:26.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:26.762+0000 7ff376000640 1 -- 172.21.15.131:0/1710004884 >> v2:172.21.15.187:3300/0 conn(0x7ff3780d0460 msgr2=0x7ff37806d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:26.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:26.762+0000 7ff376000640 1 --2- 172.21.15.131:0/1710004884 >> v2:172.21.15.187:3300/0 conn(0x7ff3780d0460 0x7ff37806d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:26.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:26.762+0000 7ff376000640 1 -- 172.21.15.131:0/1710004884 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff378160460 con 0x7ff37806f420 2024-05-26T22:50:26.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:26.762+0000 7ff376000640 1 -- 172.21.15.131:0/1710004884 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7ff360001670 con 0x7ff37806f420 2024-05-26T22:50:26.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:26.762+0000 7ff376000640 1 -- 172.21.15.131:0/1710004884 >> v1:172.21.15.131:6789/0 conn(0x7ff37806f420 legacy=0x7ff3780cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:26.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:26.762+0000 7ff376000640 1 --2- 172.21.15.131:0/1710004884 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff35c0039b0 0x7ff35c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:26.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:26.762+0000 7ff376000640 1 --2- 172.21.15.131:0/1710004884 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff35c004800 0x7ff35c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:26.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:26.762+0000 7ff376000640 1 --2- 172.21.15.131:0/1710004884 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff3780b5450 0x7ff35c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:26.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:26.762+0000 7ff376000640 1 -- 172.21.15.131:0/1710004884 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff378160d90 con 0x7ff35c004800 2024-05-26T22:50:26.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:26.762+0000 7ff376000640 1 -- 172.21.15.131:0/1710004884 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff378160180 con 0x7ff35c0039b0 2024-05-26T22:50:26.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:26.762+0000 7ff376000640 1 -- 172.21.15.131:0/1710004884 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff35c00ba70 con 0x7ff3780b5450 2024-05-26T22:50:26.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:26.763+0000 7ff376a00640 1 --2- 172.21.15.131:0/1710004884 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff35c004800 0x7ff35c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:26.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:26.763+0000 7ff377400640 1 --2- 172.21.15.131:0/1710004884 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff35c0039b0 0x7ff35c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:26.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:26.763+0000 7ff376a00640 1 -- 172.21.15.131:0/1710004884 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff3780b5450 msgr2=0x7ff35c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:26.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:26.763+0000 7ff376a00640 1 --2- 172.21.15.131:0/1710004884 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff3780b5450 0x7ff35c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:26.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:26.763+0000 7ff376a00640 1 -- 172.21.15.131:0/1710004884 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff35c0039b0 msgr2=0x7ff35c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:26.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:26.763+0000 7ff376a00640 1 --2- 172.21.15.131:0/1710004884 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff35c0039b0 0x7ff35c003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:26.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:26.763+0000 7ff376a00640 1 -- 172.21.15.131:0/1710004884 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff35c003520 con 0x7ff35c004800 2024-05-26T22:50:26.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:26.764+0000 7ff376a00640 1 --2- 172.21.15.131:0/1710004884 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff35c004800 0x7ff35c004c80 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7ff36000c210 tx=0x7ff35c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dccaf27bacc2fed0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:26.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:26.764+0000 7ff376000640 1 -- 172.21.15.131:0/1710004884 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff360007d30 con 0x7ff35c004800 2024-05-26T22:50:26.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:26.764+0000 7ff376000640 1 -- 172.21.15.131:0/1710004884 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3600058d0 con 0x7ff35c004800 2024-05-26T22:50:26.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:26.764+0000 7ff376000640 1 -- 172.21.15.131:0/1710004884 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff360005ce0 con 0x7ff35c004800 2024-05-26T22:50:26.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:26.765+0000 7ff37e000640 1 -- 172.21.15.131:0/1710004884 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff35c004800 msgr2=0x7ff35c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:26.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:26.765+0000 7ff37e000640 1 --2- 172.21.15.131:0/1710004884 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff35c004800 0x7ff35c004c80 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7ff36000c210 tx=0x7ff35c005830 comp rx=0 tx=0).stop 2024-05-26T22:50:26.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:26.765+0000 7ff377e00640 1 -- 172.21.15.131:0/1710004884 reap_dead start 2024-05-26T22:50:26.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:26.765+0000 7ff37e000640 1 -- 172.21.15.131:0/1710004884 shutdown_connections 2024-05-26T22:50:26.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:26.765+0000 7ff37e000640 1 -- 172.21.15.131:0/1710004884 >> 172.21.15.131:0/1710004884 conn(0x7ff37806bc90 msgr2=0x7ff3780b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:26.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:26.765+0000 7ff37e000640 1 -- 172.21.15.131:0/1710004884 shutdown_connections 2024-05-26T22:50:26.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:26.765+0000 7ff37e000640 1 -- 172.21.15.131:0/1710004884 wait complete. 2024-05-26T22:50:26.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:26.767+0000 7ff37e000640 1 Processor -- start 2024-05-26T22:50:26.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:26.767+0000 7ff37e000640 1 -- start start 2024-05-26T22:50:26.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:26.768+0000 7ff37e000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff35c004800 0x7ff37814a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:26.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:26.768+0000 7ff37e000640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff37815aed0 0x7ff3781e9f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:26.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:26.768+0000 7ff37e000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff37815dfb0 0x7ff37815e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:26.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:26.768+0000 7ff37e000640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff3780b5830 con 0x7ff37815dfb0 2024-05-26T22:50:26.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:26.768+0000 7ff37e000640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff378160a20 con 0x7ff37815aed0 2024-05-26T22:50:26.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:26.768+0000 7ff37e000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff37815fde0 con 0x7ff35c004800 2024-05-26T22:50:26.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:26.768+0000 7ff377e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff37815dfb0 0x7ff37815e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:26.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:26.768+0000 7ff377e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff37815dfb0 0x7ff37815e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49920/0 (socket says 172.21.15.131:49920) 2024-05-26T22:50:26.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:26.768+0000 7ff377e00640 1 -- 172.21.15.131:0/1661824202 learned_addr learned my addr 172.21.15.131:0/1661824202 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:26.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:26.768+0000 7ff376a00640 1 --2- 172.21.15.131:0/1661824202 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff37815aed0 0x7ff3781e9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:26.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:26.768+0000 7ff377400640 1 --2- 172.21.15.131:0/1661824202 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff35c004800 0x7ff37814a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:26.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:26.769+0000 7ff377e00640 1 -- 172.21.15.131:0/1661824202 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff35c004800 msgr2=0x7ff37814a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:26.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:26.769+0000 7ff377e00640 1 --2- 172.21.15.131:0/1661824202 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff35c004800 0x7ff37814a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:26.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:26.769+0000 7ff377e00640 1 -- 172.21.15.131:0/1661824202 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff37815aed0 msgr2=0x7ff3781e9f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:26.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:26.769+0000 7ff377e00640 1 --2- 172.21.15.131:0/1661824202 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff37815aed0 0x7ff3781e9f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:26.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:26.769+0000 7ff377e00640 1 -- 172.21.15.131:0/1661824202 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff37814aec0 con 0x7ff37815dfb0 2024-05-26T22:50:26.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:26.769+0000 7ff376a00640 1 --2- 172.21.15.131:0/1661824202 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff37815aed0 0x7ff3781e9f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:50:26.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:26.769+0000 7ff377e00640 1 --2- 172.21.15.131:0/1661824202 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff37815dfb0 0x7ff37815e440 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7ff3780d0c70 tx=0x7ff37000d820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:26.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:26.769+0000 7ff36f400640 1 -- 172.21.15.131:0/1661824202 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff370004020 con 0x7ff37815dfb0 2024-05-26T22:50:26.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:26.769+0000 7ff36f400640 1 -- 172.21.15.131:0/1661824202 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3700041e0 con 0x7ff37815dfb0 2024-05-26T22:50:26.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:26.769+0000 7ff36f400640 1 -- 172.21.15.131:0/1661824202 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3700043d0 con 0x7ff37815dfb0 2024-05-26T22:50:26.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:26.769+0000 7ff37e000640 1 -- 172.21.15.131:0/1661824202 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3781e5d30 con 0x7ff37815dfb0 2024-05-26T22:50:26.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:26.769+0000 7ff37e000640 1 -- 172.21.15.131:0/1661824202 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3781e62f0 con 0x7ff37815dfb0 2024-05-26T22:50:26.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:26.770+0000 7ff37e000640 1 -- 172.21.15.131:0/1661824202 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3781e6db0 con 0x7ff37815dfb0 2024-05-26T22:50:26.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:26.772+0000 7ff36f400640 1 -- 172.21.15.131:0/1661824202 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff37001a020 con 0x7ff37815dfb0 2024-05-26T22:50:26.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:26.772+0000 7ff36f400640 1 --2- 172.21.15.131:0/1661824202 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ff35406be30 0x7ff35406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:26.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:26.773+0000 7ff36f400640 1 -- 172.21.15.131:0/1661824202 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 5609+0+0 (secure 0 0 0) 0x7ff37008a580 con 0x7ff37815dfb0 2024-05-26T22:50:26.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:26.773+0000 7ff377400640 1 --2- 172.21.15.131:0/1661824202 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ff35406be30 0x7ff35406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:26.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:26.773+0000 7ff377400640 1 --2- 172.21.15.131:0/1661824202 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ff35406be30 0x7ff35406e2f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff368009e50 tx=0x7ff36800b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:26.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:26.778+0000 7ff36f400640 1 -- 172.21.15.131:0/1661824202 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7ff37814aec0 con 0x7ff37815dfb0 2024-05-26T22:50:26.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:26.999+0000 7ff37e000640 1 -- 172.21.15.131:0/1661824202 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7ff378074e90 con 0x7ff37815dfb0 2024-05-26T22:50:27.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:27.593+0000 7ff36f400640 1 -- 172.21.15.131:0/1661824202 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v25) ==== 172+0+0 (secure 0 0 0) 0x7ff370059020 con 0x7ff37815dfb0 2024-05-26T22:50:27.594 INFO:teuthology.orchestra.run.smithi131.stderr:new fs with metadata pool 6 and data pool 7 2024-05-26T22:50:27.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:27.601+0000 7ff37e000640 1 -- 172.21.15.131:0/1661824202 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ff35406be30 msgr2=0x7ff35406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:27.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:27.601+0000 7ff37e000640 1 --2- 172.21.15.131:0/1661824202 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ff35406be30 0x7ff35406e2f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff368009e50 tx=0x7ff36800b040 comp rx=0 tx=0).stop 2024-05-26T22:50:27.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:27.601+0000 7ff37e000640 1 -- 172.21.15.131:0/1661824202 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff37815dfb0 msgr2=0x7ff37815e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:27.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:27.601+0000 7ff37e000640 1 --2- 172.21.15.131:0/1661824202 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff37815dfb0 0x7ff37815e440 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7ff3780d0c70 tx=0x7ff37000d820 comp rx=0 tx=0).stop 2024-05-26T22:50:27.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:27.601+0000 7ff37e000640 1 -- 172.21.15.131:0/1661824202 shutdown_connections 2024-05-26T22:50:27.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:27.601+0000 7ff37e000640 1 --2- 172.21.15.131:0/1661824202 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ff35406be30 0x7ff35406e2f0 secure :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff368009e50 tx=0x7ff36800b040 comp rx=0 tx=0).stop 2024-05-26T22:50:27.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:27.601+0000 7ff37e000640 1 --2- 172.21.15.131:0/1661824202 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff37815dfb0 0x7ff37815e440 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:27.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:27.601+0000 7ff37e000640 1 --2- 172.21.15.131:0/1661824202 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff37815aed0 0x7ff3781e9f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:27.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:27.601+0000 7ff37e000640 1 --2- 172.21.15.131:0/1661824202 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff35c004800 0x7ff37814a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:27.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:27.601+0000 7ff37e000640 1 -- 172.21.15.131:0/1661824202 >> 172.21.15.131:0/1661824202 conn(0x7ff37806bc90 msgr2=0x7ff3780b4bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:27.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:27.606+0000 7ff37e000640 1 -- 172.21.15.131:0/1661824202 shutdown_connections 2024-05-26T22:50:27.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:27.606+0000 7ff37e000640 1 -- 172.21.15.131:0/1661824202 wait complete. 2024-05-26T22:50:27.636 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-26T22:50:27.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:27.803+0000 7fed8ac00640 1 Processor -- start 2024-05-26T22:50:27.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:27.803+0000 7fed8ac00640 1 -- start start 2024-05-26T22:50:27.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:27.805+0000 7fed8ac00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fed8406e8c0 0x7fed8415d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:27.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:27.805+0000 7fed8ac00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fed840cc7f0 0x7fed840ccc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:27.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:27.805+0000 7fed8ac00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed84160620 con 0x7fed84156750 2024-05-26T22:50:27.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:27.805+0000 7fed8ac00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fed840cd1a0 con 0x7fed840cc7f0 2024-05-26T22:50:27.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:27.805+0000 7fed8ac00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fed8415cbd0 con 0x7fed8406e8c0 2024-05-26T22:50:27.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:27.805+0000 7fed8a200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fed840cc7f0 0x7fed840ccc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:27.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:27.805+0000 7fed89800640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fed84156750 0x7fed8415fd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:38440/0 (socket says 172.21.15.131:38440) 2024-05-26T22:50:27.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:27.805+0000 7fed89800640 1 -- 172.21.15.131:0/1900652103 learned_addr learned my addr 172.21.15.131:0/1900652103 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:50:27.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:27.805+0000 7fed88e00640 1 --2- 172.21.15.131:0/1900652103 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fed8406e8c0 0x7fed8415d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:27.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:27.805+0000 7fed8a200640 1 -- 172.21.15.131:0/1900652103 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fed8406e8c0 msgr2=0x7fed8415d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:27.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:27.805+0000 7fed8a200640 1 --2- 172.21.15.131:0/1900652103 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fed8406e8c0 0x7fed8415d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:27.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:27.805+0000 7fed8a200640 1 -- 172.21.15.131:0/1900652103 >> v1:172.21.15.131:6789/0 conn(0x7fed84156750 legacy=0x7fed8415fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:27.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:27.805+0000 7fed8a200640 1 -- 172.21.15.131:0/1900652103 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed84160900 con 0x7fed840cc7f0 2024-05-26T22:50:27.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:27.805+0000 7fed83e00640 1 -- 172.21.15.131:0/1900652103 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4090581275 0 0) 0x7fed84160620 con 0x7fed84156750 2024-05-26T22:50:27.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:27.806+0000 7fed8a200640 1 --2- 172.21.15.131:0/1900652103 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed840cc7f0 0x7fed840ccc60 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fed7c009d90 tx=0x7fed7c02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6484e2d30ebb06a server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:27.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:27.807+0000 7fed83e00640 1 -- 172.21.15.131:0/1900652103 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed7c007740 con 0x7fed840cc7f0 2024-05-26T22:50:27.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:27.807+0000 7fed83e00640 1 -- 172.21.15.131:0/1900652103 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed7c0078e0 con 0x7fed840cc7f0 2024-05-26T22:50:27.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:27.807+0000 7fed83e00640 1 -- 172.21.15.131:0/1900652103 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed7c007cb0 con 0x7fed840cc7f0 2024-05-26T22:50:27.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:27.807+0000 7fed8ac00640 1 -- 172.21.15.131:0/1900652103 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed840cc7f0 msgr2=0x7fed840ccc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:27.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:27.807+0000 7fed8ac00640 1 --2- 172.21.15.131:0/1900652103 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed840cc7f0 0x7fed840ccc60 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fed7c009d90 tx=0x7fed7c02a660 comp rx=0 tx=0).stop 2024-05-26T22:50:27.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:27.807+0000 7fed8ac00640 1 -- 172.21.15.131:0/1900652103 shutdown_connections 2024-05-26T22:50:27.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:27.807+0000 7fed8ac00640 1 --2- 172.21.15.131:0/1900652103 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed840cc7f0 0x7fed840ccc60 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:27.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:27.807+0000 7fed8ac00640 1 --2- 172.21.15.131:0/1900652103 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fed8406e8c0 0x7fed8415d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:27.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:27.807+0000 7fed8ac00640 1 -- 172.21.15.131:0/1900652103 >> 172.21.15.131:0/1900652103 conn(0x7fed840cff40 msgr2=0x7fed840d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:27.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:27.807+0000 7fed8ac00640 1 -- 172.21.15.131:0/1900652103 shutdown_connections 2024-05-26T22:50:27.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:27.807+0000 7fed8ac00640 1 -- 172.21.15.131:0/1900652103 wait complete. 2024-05-26T22:50:27.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:27.809+0000 7fed8ac00640 1 Processor -- start 2024-05-26T22:50:27.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:27.809+0000 7fed8ac00640 1 -- start start 2024-05-26T22:50:27.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:27.810+0000 7fed8ac00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fed8406e8c0 0x7fed840b26b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:27.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:27.810+0000 7fed8ac00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fed840cc7f0 0x7fed841f41d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:27.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:27.810+0000 7fed8ac00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed84156750 0x7fed841f4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:27.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:27.810+0000 7fed8ac00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fed84156b30 con 0x7fed84156750 2024-05-26T22:50:27.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:27.810+0000 7fed8ac00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fed8415c860 con 0x7fed8406e8c0 2024-05-26T22:50:27.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:27.810+0000 7fed8ac00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fed8415d5a0 con 0x7fed840cc7f0 2024-05-26T22:50:27.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:27.810+0000 7fed8a200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed84156750 0x7fed841f4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:27.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:27.810+0000 7fed8a200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed84156750 0x7fed841f4710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49952/0 (socket says 172.21.15.131:49952) 2024-05-26T22:50:27.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:27.810+0000 7fed8a200640 1 -- 172.21.15.131:0/717050483 learned_addr learned my addr 172.21.15.131:0/717050483 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:27.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:27.811+0000 7fed89800640 1 --2- 172.21.15.131:0/717050483 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fed8406e8c0 0x7fed840b26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:27.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:27.811+0000 7fed8a200640 1 -- 172.21.15.131:0/717050483 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fed840cc7f0 msgr2=0x7fed841f41d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:27.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:27.811+0000 7fed88e00640 1 --2- 172.21.15.131:0/717050483 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fed840cc7f0 0x7fed841f41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:27.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:27.811+0000 7fed8a200640 1 --2- 172.21.15.131:0/717050483 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fed840cc7f0 0x7fed841f41d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:27.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:27.811+0000 7fed8a200640 1 -- 172.21.15.131:0/717050483 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fed8406e8c0 msgr2=0x7fed840b26b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:27.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:27.811+0000 7fed8a200640 1 --2- 172.21.15.131:0/717050483 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fed8406e8c0 0x7fed840b26b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:27.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:27.811+0000 7fed8a200640 1 -- 172.21.15.131:0/717050483 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed841f4c50 con 0x7fed84156750 2024-05-26T22:50:27.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:27.811+0000 7fed8a200640 1 --2- 172.21.15.131:0/717050483 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed84156750 0x7fed841f4710 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fed7c009eb0 tx=0x7fed7c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:27.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:27.811+0000 7fed81600640 1 -- 172.21.15.131:0/717050483 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed7c040070 con 0x7fed84156750 2024-05-26T22:50:27.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:27.811+0000 7fed81600640 1 -- 172.21.15.131:0/717050483 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed7c031c90 con 0x7fed84156750 2024-05-26T22:50:27.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:27.811+0000 7fed81600640 1 -- 172.21.15.131:0/717050483 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed7c045680 con 0x7fed84156750 2024-05-26T22:50:27.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:27.811+0000 7fed8ac00640 1 -- 172.21.15.131:0/717050483 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed840b1610 con 0x7fed84156750 2024-05-26T22:50:27.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:27.812+0000 7fed8ac00640 1 -- 172.21.15.131:0/717050483 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed840b1bd0 con 0x7fed84156750 2024-05-26T22:50:27.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:27.813+0000 7fed8ac00640 1 -- 172.21.15.131:0/717050483 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed840694e0 con 0x7fed84156750 2024-05-26T22:50:27.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:27.813+0000 7fed81600640 1 -- 172.21.15.131:0/717050483 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fed7c039040 con 0x7fed84156750 2024-05-26T22:50:27.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:27.814+0000 7fed81600640 1 --2- 172.21.15.131:0/717050483 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fed6406be30 0x7fed6406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:27.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:27.814+0000 7fed81600640 1 -- 172.21.15.131:0/717050483 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7fed7c0ac820 con 0x7fed84156750 2024-05-26T22:50:27.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:27.815+0000 7fed89800640 1 --2- 172.21.15.131:0/717050483 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fed6406be30 0x7fed6406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:27.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:27.816+0000 7fed89800640 1 --2- 172.21.15.131:0/717050483 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fed6406be30 0x7fed6406e2f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fed78000f60 tx=0x7fed78009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:27.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:27.819+0000 7fed81600640 1 -- 172.21.15.131:0/717050483 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fed7c07b280 con 0x7fed84156750 2024-05-26T22:50:28.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.016+0000 7fed8ac00640 1 -- 172.21.15.131:0/717050483 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fed840ae180 con 0x7fed84156750 2024-05-26T22:50:28.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.019+0000 7fed81600640 1 -- 172.21.15.131:0/717050483 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) ==== 74+0+16650 (secure 0 0 0) 0x7fed7c07ab80 con 0x7fed84156750 2024-05-26T22:50:28.019 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:50:28.019 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":43,"fsid":"46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f","created":"2024-05-26T22:47:57.288905+0000","modified":"2024-05-26T22:50:27.000862+0000","last_up_change":"2024-05-26T22:48:09.602207+0000","last_in_change":"2024-05-26T22:48:00.687341+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-26T22:48:10.463339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-05-26T22:50:24.793406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-05-26T22:50:26.041636+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"814eaf62-a18d-494b-8e27-fd8fb88adea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6801","nonce":2350940051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6803","nonce":2350940051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6807","nonce":2350940051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6805","nonce":2350940051}]},"public_addr":"172.21.15.131:6801/2350940051","cluster_addr":"172.21.15.131:6803/2350940051","heartbeat_back_addr":"172.21.15.131:6807/2350940051","heartbeat_front_addr":"172.21.15.131:6805/2350940051","state":["exists","up"]},{"osd":1,"uuid":"42e66287-f6c6-4e26-af3c-6dfaa3536d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6825","nonce":1525528247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6827","nonce":1525528247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6831","nonce":1525528247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6829","nonce":1525528247}]},"public_addr":"172.21.15.131:6825/1525528247","cluster_addr":"172.21.15.131:6827/1525528247","heartbeat_back_addr":"172.21.15.131:6831/1525528247","heartbeat_front_addr":"172.21.15.131:6829/1525528247","state":["exists","up"]},{"osd":2,"uuid":"0da34673-04a2-4fd0-b57a-0ff2b679b043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6817","nonce":3627842210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6819","nonce":3627842210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6823","nonce":3627842210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6821","nonce":3627842210}]},"public_addr":"172.21.15.131:6817/3627842210","cluster_addr":"172.21.15.131:6819/3627842210","heartbeat_back_addr":"172.21.15.131:6823/3627842210","heartbeat_front_addr":"172.21.15.131:6821/3627842210","state":["exists","up"]},{"osd":3,"uuid":"cf117a27-d6b2-43d3-ae52-b47988bf8eed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6809","nonce":350790591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6811","nonce":350790591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6815","nonce":350790591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6813","nonce":350790591}]},"public_addr":"172.21.15.131:6809/350790591","cluster_addr":"172.21.15.131:6811/350790591","heartbeat_back_addr":"172.21.15.131:6815/350790591","heartbeat_front_addr":"172.21.15.131:6813/350790591","state":["exists","up"]},{"osd":4,"uuid":"a092ccfc-be89-4a84-88d5-0f781fa715eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6816","nonce":244560995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6819","nonce":244560995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6823","nonce":244560995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6821","nonce":244560995}]},"public_addr":"172.21.15.187:6816/244560995","cluster_addr":"172.21.15.187:6819/244560995","heartbeat_back_addr":"172.21.15.187:6823/244560995","heartbeat_front_addr":"172.21.15.187:6821/244560995","state":["exists","up"]},{"osd":5,"uuid":"7513ba69-dd00-4a35-b190-9e8cc3b2bdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6801","nonce":324619398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6803","nonce":324619398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6812","nonce":324619398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6807","nonce":324619398}]},"public_addr":"172.21.15.187:6801/324619398","cluster_addr":"172.21.15.187:6803/324619398","heartbeat_back_addr":"172.21.15.187:6812/324619398","heartbeat_front_addr":"172.21.15.187:6807/324619398","state":["exists","up"]},{"osd":6,"uuid":"50081810-c093-4d2f-a4d7-b7c695c5400d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6806","nonce":1391347786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6809","nonce":1391347786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6818","nonce":1391347786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6813","nonce":1391347786}]},"public_addr":"172.21.15.187:6806/1391347786","cluster_addr":"172.21.15.187:6809/1391347786","heartbeat_back_addr":"172.21.15.187:6818/1391347786","heartbeat_front_addr":"172.21.15.187:6813/1391347786","state":["exists","up"]},{"osd":7,"uuid":"775dae05-54f5-4ad4-bbce-92863c6ed2ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6825","nonce":1220065622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6827","nonce":1220065622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6831","nonce":1220065622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6829","nonce":1220065622}]},"public_addr":"172.21.15.187:6825/1220065622","cluster_addr":"172.21.15.187:6827/1220065622","heartbeat_back_addr":"172.21.15.187:6831/1220065622","heartbeat_front_addr":"172.21.15.187:6829/1220065622","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:05.983520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.299516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.067323+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.307748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.018742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.258775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.010912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.295355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-26T22:50:28.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.021+0000 7fed8ac00640 1 -- 172.21.15.131:0/717050483 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fed6406be30 msgr2=0x7fed6406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:28.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.021+0000 7fed8ac00640 1 --2- 172.21.15.131:0/717050483 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fed6406be30 0x7fed6406e2f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fed78000f60 tx=0x7fed78009040 comp rx=0 tx=0).stop 2024-05-26T22:50:28.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.021+0000 7fed8ac00640 1 -- 172.21.15.131:0/717050483 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed84156750 msgr2=0x7fed841f4710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:28.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.021+0000 7fed8ac00640 1 --2- 172.21.15.131:0/717050483 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed84156750 0x7fed841f4710 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fed7c009eb0 tx=0x7fed7c004a40 comp rx=0 tx=0).stop 2024-05-26T22:50:28.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.021+0000 7fed8ac00640 1 -- 172.21.15.131:0/717050483 shutdown_connections 2024-05-26T22:50:28.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.021+0000 7fed8ac00640 1 --2- 172.21.15.131:0/717050483 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fed840cc7f0 0x7fed841f41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:28.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.021+0000 7fed8ac00640 1 --2- 172.21.15.131:0/717050483 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fed6406be30 0x7fed6406e2f0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:28.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.021+0000 7fed8ac00640 1 --2- 172.21.15.131:0/717050483 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed84156750 0x7fed841f4710 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:28.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.021+0000 7fed8ac00640 1 --2- 172.21.15.131:0/717050483 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fed8406e8c0 0x7fed840b26b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:28.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.022+0000 7fed8ac00640 1 -- 172.21.15.131:0/717050483 >> 172.21.15.131:0/717050483 conn(0x7fed840cff40 msgr2=0x7fed840b4b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:28.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.022+0000 7fed8ac00640 1 -- 172.21.15.131:0/717050483 shutdown_connections 2024-05-26T22:50:28.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.022+0000 7fed8ac00640 1 -- 172.21.15.131:0/717050483 wait complete. 2024-05-26T22:50:28.039 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-26T22:50:28.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.228+0000 7f4581600640 1 Processor -- start 2024-05-26T22:50:28.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.228+0000 7f4581600640 1 -- start start 2024-05-26T22:50:28.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.230+0000 7f4581600640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f457c156750 0x7f457c15fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:28.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.230+0000 7f4581600640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f457c06e8c0 0x7f457c15d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:28.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.230+0000 7f4581600640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f457c0cc7f0 0x7f457c0ccc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:28.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.230+0000 7f4581600640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f457c0cd1a0 con 0x7f457c0cc7f0 2024-05-26T22:50:28.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.230+0000 7f4581600640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f457c15cbd0 con 0x7f457c156750 2024-05-26T22:50:28.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.230+0000 7f4581600640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f457c160620 con 0x7f457c06e8c0 2024-05-26T22:50:28.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.230+0000 7f4580c00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f457c0cc7f0 0x7f457c0ccc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:28.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.230+0000 7f4580c00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f457c0cc7f0 0x7f457c0ccc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49968/0 (socket says 172.21.15.131:49968) 2024-05-26T22:50:28.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.230+0000 7f4580c00640 1 -- 172.21.15.131:0/1442719250 learned_addr learned my addr 172.21.15.131:0/1442719250 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:28.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.230+0000 7f457b400640 1 --2- 172.21.15.131:0/1442719250 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f457c06e8c0 0x7f457c15d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:28.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.230+0000 7f457be00640 1 --2- 172.21.15.131:0/1442719250 >> v2:172.21.15.187:3300/0 conn(0x7f457c156750 0x7f457c15fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:28.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.231+0000 7f4580c00640 1 -- 172.21.15.131:0/1442719250 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f457c06e8c0 msgr2=0x7f457c15d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:28.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.231+0000 7f4580c00640 1 --2- 172.21.15.131:0/1442719250 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f457c06e8c0 0x7f457c15d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:28.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.231+0000 7f4580c00640 1 -- 172.21.15.131:0/1442719250 >> v2:172.21.15.187:3300/0 conn(0x7f457c156750 msgr2=0x7f457c15fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:28.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.231+0000 7f4580c00640 1 --2- 172.21.15.131:0/1442719250 >> v2:172.21.15.187:3300/0 conn(0x7f457c156750 0x7f457c15fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:28.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.231+0000 7f4580c00640 1 -- 172.21.15.131:0/1442719250 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f457c160900 con 0x7f457c0cc7f0 2024-05-26T22:50:28.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.231+0000 7f457be00640 1 --2- 172.21.15.131:0/1442719250 >> v2:172.21.15.187:3300/0 conn(0x7f457c156750 0x7f457c15fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:50:28.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.232+0000 7f4580c00640 1 --2- 172.21.15.131:0/1442719250 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f457c0cc7f0 0x7f457c0ccc60 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f457c0cdb10 tx=0x7f457402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b1291eb340ac288 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:28.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.232+0000 7f457aa00640 1 -- 172.21.15.131:0/1442719250 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4574036070 con 0x7f457c0cc7f0 2024-05-26T22:50:28.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.232+0000 7f457aa00640 1 -- 172.21.15.131:0/1442719250 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4574031600 con 0x7f457c0cc7f0 2024-05-26T22:50:28.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.232+0000 7f457aa00640 1 -- 172.21.15.131:0/1442719250 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45740319d0 con 0x7f457c0cc7f0 2024-05-26T22:50:28.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.233+0000 7f4581600640 1 -- 172.21.15.131:0/1442719250 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f457c0cc7f0 msgr2=0x7f457c0ccc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:28.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.233+0000 7f4581600640 1 --2- 172.21.15.131:0/1442719250 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f457c0cc7f0 0x7f457c0ccc60 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f457c0cdb10 tx=0x7f457402a6a0 comp rx=0 tx=0).stop 2024-05-26T22:50:28.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.233+0000 7f4581600640 1 -- 172.21.15.131:0/1442719250 shutdown_connections 2024-05-26T22:50:28.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.233+0000 7f4581600640 1 --2- 172.21.15.131:0/1442719250 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f457c0cc7f0 0x7f457c0ccc60 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:28.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.233+0000 7f4581600640 1 --2- 172.21.15.131:0/1442719250 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f457c06e8c0 0x7f457c15d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:28.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.233+0000 7f4581600640 1 --2- 172.21.15.131:0/1442719250 >> v2:172.21.15.187:3300/0 conn(0x7f457c156750 0x7f457c15fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:28.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.233+0000 7f4581600640 1 -- 172.21.15.131:0/1442719250 >> 172.21.15.131:0/1442719250 conn(0x7f457c0cff40 msgr2=0x7f457c0d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:28.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.233+0000 7f4581600640 1 -- 172.21.15.131:0/1442719250 shutdown_connections 2024-05-26T22:50:28.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.233+0000 7f4581600640 1 -- 172.21.15.131:0/1442719250 wait complete. 2024-05-26T22:50:28.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.234+0000 7f4581600640 1 Processor -- start 2024-05-26T22:50:28.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.235+0000 7f4581600640 1 -- start start 2024-05-26T22:50:28.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.235+0000 7f4581600640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f457c06e8c0 0x7f457c13d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:28.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.235+0000 7f4581600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f457c0cc7f0 0x7f457c13e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:28.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.235+0000 7f4581600640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f457c156750 0x7f457c15b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:28.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.235+0000 7f4581600640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f457c156b30 con 0x7f457c0cc7f0 2024-05-26T22:50:28.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.235+0000 7f4581600640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f457c15c860 con 0x7f457c06e8c0 2024-05-26T22:50:28.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.235+0000 7f4581600640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f457c160280 con 0x7f457c156750 2024-05-26T22:50:28.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.236+0000 7f457b400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f457c0cc7f0 0x7f457c13e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:28.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.236+0000 7f457b400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f457c0cc7f0 0x7f457c13e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49982/0 (socket says 172.21.15.131:49982) 2024-05-26T22:50:28.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.236+0000 7f457b400640 1 -- 172.21.15.131:0/3545687673 learned_addr learned my addr 172.21.15.131:0/3545687673 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:28.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.236+0000 7f457be00640 1 --2- 172.21.15.131:0/3545687673 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f457c06e8c0 0x7f457c13d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:28.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.236+0000 7f457b400640 1 -- 172.21.15.131:0/3545687673 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f457c156750 msgr2=0x7f457c15b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:28.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.236+0000 7f457b400640 1 --2- 172.21.15.131:0/3545687673 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f457c156750 0x7f457c15b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:28.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.236+0000 7f457b400640 1 -- 172.21.15.131:0/3545687673 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f457c06e8c0 msgr2=0x7f457c13d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:28.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.236+0000 7f457b400640 1 --2- 172.21.15.131:0/3545687673 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f457c06e8c0 0x7f457c13d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:28.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.236+0000 7f457b400640 1 -- 172.21.15.131:0/3545687673 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f457c13f180 con 0x7f457c0cc7f0 2024-05-26T22:50:28.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.236+0000 7f457b400640 1 --2- 172.21.15.131:0/3545687673 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f457c0cc7f0 0x7f457c13e7c0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f456800d800 tx=0x7f456800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:28.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.237+0000 7f4573e00640 1 -- 172.21.15.131:0/3545687673 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f456800b6c0 con 0x7f457c0cc7f0 2024-05-26T22:50:28.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.237+0000 7f4573e00640 1 -- 172.21.15.131:0/3545687673 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4568019070 con 0x7f457c0cc7f0 2024-05-26T22:50:28.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.237+0000 7f4573e00640 1 -- 172.21.15.131:0/3545687673 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f456800ba80 con 0x7f457c0cc7f0 2024-05-26T22:50:28.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.237+0000 7f4581600640 1 -- 172.21.15.131:0/3545687673 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f457c13f460 con 0x7f457c0cc7f0 2024-05-26T22:50:28.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.237+0000 7f4581600640 1 -- 172.21.15.131:0/3545687673 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f457c158020 con 0x7f457c0cc7f0 2024-05-26T22:50:28.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.238+0000 7f4581600640 1 -- 172.21.15.131:0/3545687673 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f457c0694e0 con 0x7f457c0cc7f0 2024-05-26T22:50:28.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.239+0000 7f4573e00640 1 -- 172.21.15.131:0/3545687673 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f456800bc20 con 0x7f457c0cc7f0 2024-05-26T22:50:28.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.240+0000 7f4573e00640 1 --2- 172.21.15.131:0/3545687673 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f455c070240 0x7f455c072700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:28.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.240+0000 7f4573e00640 1 -- 172.21.15.131:0/3545687673 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7f456808b330 con 0x7f457c0cc7f0 2024-05-26T22:50:28.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.240+0000 7f457be00640 1 --2- 172.21.15.131:0/3545687673 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f455c070240 0x7f455c072700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:28.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.240+0000 7f457be00640 1 --2- 172.21.15.131:0/3545687673 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f455c070240 0x7f455c072700 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f456c00b7c0 tx=0x7f456c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:28.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.244+0000 7f4573e00640 1 -- 172.21.15.131:0/3545687673 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f4568059ce0 con 0x7f457c0cc7f0 2024-05-26T22:50:28.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.431+0000 7f4581600640 1 -- 172.21.15.131:0/3545687673 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f457c13dcb0 con 0x7f457c0cc7f0 2024-05-26T22:50:28.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.432+0000 7f4573e00640 1 -- 172.21.15.131:0/3545687673 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) ==== 74+0+16650 (secure 0 0 0) 0x7f45680595e0 con 0x7f457c0cc7f0 2024-05-26T22:50:28.433 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:50:28.433 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":43,"fsid":"46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f","created":"2024-05-26T22:47:57.288905+0000","modified":"2024-05-26T22:50:27.000862+0000","last_up_change":"2024-05-26T22:48:09.602207+0000","last_in_change":"2024-05-26T22:48:00.687341+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-26T22:48:10.463339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-05-26T22:50:24.793406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-05-26T22:50:26.041636+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"814eaf62-a18d-494b-8e27-fd8fb88adea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6801","nonce":2350940051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6803","nonce":2350940051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6807","nonce":2350940051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6805","nonce":2350940051}]},"public_addr":"172.21.15.131:6801/2350940051","cluster_addr":"172.21.15.131:6803/2350940051","heartbeat_back_addr":"172.21.15.131:6807/2350940051","heartbeat_front_addr":"172.21.15.131:6805/2350940051","state":["exists","up"]},{"osd":1,"uuid":"42e66287-f6c6-4e26-af3c-6dfaa3536d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6825","nonce":1525528247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6827","nonce":1525528247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6831","nonce":1525528247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6829","nonce":1525528247}]},"public_addr":"172.21.15.131:6825/1525528247","cluster_addr":"172.21.15.131:6827/1525528247","heartbeat_back_addr":"172.21.15.131:6831/1525528247","heartbeat_front_addr":"172.21.15.131:6829/1525528247","state":["exists","up"]},{"osd":2,"uuid":"0da34673-04a2-4fd0-b57a-0ff2b679b043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6817","nonce":3627842210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6819","nonce":3627842210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6823","nonce":3627842210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6821","nonce":3627842210}]},"public_addr":"172.21.15.131:6817/3627842210","cluster_addr":"172.21.15.131:6819/3627842210","heartbeat_back_addr":"172.21.15.131:6823/3627842210","heartbeat_front_addr":"172.21.15.131:6821/3627842210","state":["exists","up"]},{"osd":3,"uuid":"cf117a27-d6b2-43d3-ae52-b47988bf8eed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6809","nonce":350790591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6811","nonce":350790591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6815","nonce":350790591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6813","nonce":350790591}]},"public_addr":"172.21.15.131:6809/350790591","cluster_addr":"172.21.15.131:6811/350790591","heartbeat_back_addr":"172.21.15.131:6815/350790591","heartbeat_front_addr":"172.21.15.131:6813/350790591","state":["exists","up"]},{"osd":4,"uuid":"a092ccfc-be89-4a84-88d5-0f781fa715eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6816","nonce":244560995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6819","nonce":244560995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6823","nonce":244560995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6821","nonce":244560995}]},"public_addr":"172.21.15.187:6816/244560995","cluster_addr":"172.21.15.187:6819/244560995","heartbeat_back_addr":"172.21.15.187:6823/244560995","heartbeat_front_addr":"172.21.15.187:6821/244560995","state":["exists","up"]},{"osd":5,"uuid":"7513ba69-dd00-4a35-b190-9e8cc3b2bdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6801","nonce":324619398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6803","nonce":324619398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6812","nonce":324619398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6807","nonce":324619398}]},"public_addr":"172.21.15.187:6801/324619398","cluster_addr":"172.21.15.187:6803/324619398","heartbeat_back_addr":"172.21.15.187:6812/324619398","heartbeat_front_addr":"172.21.15.187:6807/324619398","state":["exists","up"]},{"osd":6,"uuid":"50081810-c093-4d2f-a4d7-b7c695c5400d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6806","nonce":1391347786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6809","nonce":1391347786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6818","nonce":1391347786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6813","nonce":1391347786}]},"public_addr":"172.21.15.187:6806/1391347786","cluster_addr":"172.21.15.187:6809/1391347786","heartbeat_back_addr":"172.21.15.187:6818/1391347786","heartbeat_front_addr":"172.21.15.187:6813/1391347786","state":["exists","up"]},{"osd":7,"uuid":"775dae05-54f5-4ad4-bbce-92863c6ed2ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6825","nonce":1220065622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6827","nonce":1220065622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6831","nonce":1220065622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6829","nonce":1220065622}]},"public_addr":"172.21.15.187:6825/1220065622","cluster_addr":"172.21.15.187:6827/1220065622","heartbeat_back_addr":"172.21.15.187:6831/1220065622","heartbeat_front_addr":"172.21.15.187:6829/1220065622","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:05.983520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.299516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.067323+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.307748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.018742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.258775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.010912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.295355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-26T22:50:28.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.435+0000 7f4581600640 1 -- 172.21.15.131:0/3545687673 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f455c070240 msgr2=0x7f455c072700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:28.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.435+0000 7f4581600640 1 --2- 172.21.15.131:0/3545687673 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f455c070240 0x7f455c072700 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f456c00b7c0 tx=0x7f456c008040 comp rx=0 tx=0).stop 2024-05-26T22:50:28.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.435+0000 7f4581600640 1 -- 172.21.15.131:0/3545687673 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f457c0cc7f0 msgr2=0x7f457c13e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:28.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.435+0000 7f4581600640 1 --2- 172.21.15.131:0/3545687673 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f457c0cc7f0 0x7f457c13e7c0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f456800d800 tx=0x7f456800dcc0 comp rx=0 tx=0).stop 2024-05-26T22:50:28.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.435+0000 7f4581600640 1 -- 172.21.15.131:0/3545687673 shutdown_connections 2024-05-26T22:50:28.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.435+0000 7f4581600640 1 --2- 172.21.15.131:0/3545687673 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f457c156750 0x7f457c15b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:28.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.435+0000 7f4581600640 1 --2- 172.21.15.131:0/3545687673 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f455c070240 0x7f455c072700 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:28.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.435+0000 7f4581600640 1 --2- 172.21.15.131:0/3545687673 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f457c0cc7f0 0x7f457c13e7c0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:28.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.435+0000 7f4581600640 1 --2- 172.21.15.131:0/3545687673 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f457c06e8c0 0x7f457c13d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:28.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.435+0000 7f4581600640 1 -- 172.21.15.131:0/3545687673 >> 172.21.15.131:0/3545687673 conn(0x7f457c0cff40 msgr2=0x7f457c0d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:28.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.436+0000 7f4581600640 1 -- 172.21.15.131:0/3545687673 shutdown_connections 2024-05-26T22:50:28.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.436+0000 7f4581600640 1 -- 172.21.15.131:0/3545687673 wait complete. 2024-05-26T22:50:28.453 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-26T22:50:28.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.655+0000 7f6b23600640 1 Processor -- start 2024-05-26T22:50:28.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.655+0000 7f6b23600640 1 -- start start 2024-05-26T22:50:28.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.657+0000 7f6b23600640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f6b1c15e890 0x7f6b1c0cecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:28.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.657+0000 7f6b23600640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b1c0cf320 con 0x7f6b1c0b77d0 2024-05-26T22:50:28.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.657+0000 7f6b23600640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b1c0cf600 con 0x7f6b1c0d1470 2024-05-26T22:50:28.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.657+0000 7f6b23600640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f6b1c0cf8e0 con 0x7f6b1c15e890 2024-05-26T22:50:28.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.657+0000 7f6b13e00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f6b1c0b77d0 0x7f6b1c0d0d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:38448/0 (socket says 172.21.15.131:38448) 2024-05-26T22:50:28.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.657+0000 7f6b13e00640 1 -- 172.21.15.131:0/1934213907 learned_addr learned my addr 172.21.15.131:0/1934213907 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:50:28.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.657+0000 7f6b20c00640 1 --2- 172.21.15.131:0/1934213907 >> v2:172.21.15.187:3300/0 conn(0x7f6b1c15e890 0x7f6b1c0cecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:28.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.658+0000 7f6b1be00640 1 -- 172.21.15.131:0/1934213907 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 268280145 0 0) 0x7f6b1c0cf320 con 0x7f6b1c0b77d0 2024-05-26T22:50:28.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.658+0000 7f6b1be00640 1 -- 172.21.15.131:0/1934213907 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6afc003520 con 0x7f6b1c0b77d0 2024-05-26T22:50:28.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.658+0000 7f6b1be00640 1 -- 172.21.15.131:0/1934213907 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3029206941 0 0) 0x7f6afc003520 con 0x7f6b1c0b77d0 2024-05-26T22:50:28.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.658+0000 7f6b1be00640 1 -- 172.21.15.131:0/1934213907 >> v2:172.21.15.187:3300/0 conn(0x7f6b1c15e890 msgr2=0x7f6b1c0cecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:28.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.658+0000 7f6b1be00640 1 --2- 172.21.15.131:0/1934213907 >> v2:172.21.15.187:3300/0 conn(0x7f6b1c15e890 0x7f6b1c0cecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:28.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.658+0000 7f6b1be00640 1 -- 172.21.15.131:0/1934213907 >> v1:172.21.15.187:6789/0 conn(0x7f6b1c0d1470 legacy=0x7f6b1c0d2b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:28.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.658+0000 7f6b1be00640 1 -- 172.21.15.131:0/1934213907 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b1c0cfbc0 con 0x7f6b1c0b77d0 2024-05-26T22:50:28.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.658+0000 7f6b1be00640 1 -- 172.21.15.131:0/1934213907 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7f6b0c001670 con 0x7f6b1c0b77d0 2024-05-26T22:50:28.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.658+0000 7f6b1be00640 1 -- 172.21.15.131:0/1934213907 >> v1:172.21.15.131:6789/0 conn(0x7f6b1c0b77d0 legacy=0x7f6b1c0d0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:28.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.658+0000 7f6b1be00640 1 --2- 172.21.15.131:0/1934213907 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6afc003a00 0x7f6afc003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:28.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.658+0000 7f6b1be00640 1 --2- 172.21.15.131:0/1934213907 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6afc004800 0x7f6afc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:28.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.658+0000 7f6b1be00640 1 --2- 172.21.15.131:0/1934213907 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6afc0051c0 0x7f6afc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:28.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.658+0000 7f6b1be00640 1 -- 172.21.15.131:0/1934213907 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f6b1c0cf8e0 con 0x7f6afc003a00 2024-05-26T22:50:28.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.658+0000 7f6b1be00640 1 -- 172.21.15.131:0/1934213907 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6afc00c0c0 con 0x7f6afc004800 2024-05-26T22:50:28.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.658+0000 7f6b1be00640 1 -- 172.21.15.131:0/1934213907 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f6afc00c2c0 con 0x7f6afc0051c0 2024-05-26T22:50:28.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.658+0000 7f6b20c00640 1 --2- 172.21.15.131:0/1934213907 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6afc003a00 0x7f6afc003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:28.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.659+0000 7f6b13e00640 1 --2- 172.21.15.131:0/1934213907 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6afc004800 0x7f6afc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:28.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.659+0000 7f6b20c00640 1 -- 172.21.15.131:0/1934213907 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6afc0051c0 msgr2=0x7f6afc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:28.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.659+0000 7f6b20c00640 1 --2- 172.21.15.131:0/1934213907 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6afc0051c0 0x7f6afc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:28.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.659+0000 7f6b20c00640 1 -- 172.21.15.131:0/1934213907 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6afc004800 msgr2=0x7f6afc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:28.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.659+0000 7f6b20c00640 1 --2- 172.21.15.131:0/1934213907 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6afc004800 0x7f6afc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:28.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.659+0000 7f6b20c00640 1 -- 172.21.15.131:0/1934213907 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6afc003520 con 0x7f6afc003a00 2024-05-26T22:50:28.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.659+0000 7f6b13e00640 1 --2- 172.21.15.131:0/1934213907 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6afc004800 0x7f6afc004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:50:28.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.659+0000 7f6b21600640 1 -- 172.21.15.131:0/1934213907 reap_dead start 2024-05-26T22:50:28.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.660+0000 7f6b20c00640 1 --2- 172.21.15.131:0/1934213907 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6afc003a00 0x7f6afc003e20 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f6b04009ac0 tx=0x7f6afc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=566eaf7e9fcdd552 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:28.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.660+0000 7f6b1be00640 1 -- 172.21.15.131:0/1934213907 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b04036070 con 0x7f6afc003a00 2024-05-26T22:50:28.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.660+0000 7f6b1be00640 1 -- 172.21.15.131:0/1934213907 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b04031400 con 0x7f6afc003a00 2024-05-26T22:50:28.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.660+0000 7f6b1be00640 1 -- 172.21.15.131:0/1934213907 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b04009690 con 0x7f6afc003a00 2024-05-26T22:50:28.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.661+0000 7f6b23600640 1 -- 172.21.15.131:0/1934213907 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6afc003a00 msgr2=0x7f6afc003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:28.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.661+0000 7f6b23600640 1 --2- 172.21.15.131:0/1934213907 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6afc003a00 0x7f6afc003e20 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f6b04009ac0 tx=0x7f6afc005830 comp rx=0 tx=0).stop 2024-05-26T22:50:28.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.661+0000 7f6b23600640 1 -- 172.21.15.131:0/1934213907 shutdown_connections 2024-05-26T22:50:28.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.661+0000 7f6b23600640 1 --2- 172.21.15.131:0/1934213907 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6afc003a00 0x7f6afc003e20 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:28.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.661+0000 7f6b23600640 1 -- 172.21.15.131:0/1934213907 >> 172.21.15.131:0/1934213907 conn(0x7f6b1c073070 msgr2=0x7f6b1c06c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:28.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.661+0000 7f6b23600640 1 -- 172.21.15.131:0/1934213907 shutdown_connections 2024-05-26T22:50:28.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.661+0000 7f6b23600640 1 -- 172.21.15.131:0/1934213907 wait complete. 2024-05-26T22:50:28.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.662+0000 7f6b23600640 1 Processor -- start 2024-05-26T22:50:28.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.663+0000 7f6b23600640 1 -- start start 2024-05-26T22:50:28.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.663+0000 7f6b23600640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6afc003a00 0x7f6b1c15ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:28.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.663+0000 7f6b23600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6b1c157e20 0x7f6b1c157580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:28.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.664+0000 7f6b23600640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6b1c161100 0x7f6b1c161580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:28.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.664+0000 7f6b23600640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f6b1c0cf060 con 0x7f6b1c157e20 2024-05-26T22:50:28.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.664+0000 7f6b23600640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6b1c0ceee0 con 0x7f6b1c161100 2024-05-26T22:50:28.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.664+0000 7f6b23600640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f6b1c162cb0 con 0x7f6afc003a00 2024-05-26T22:50:28.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.664+0000 7f6b13e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6b1c157e20 0x7f6b1c157580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:28.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.664+0000 7f6b13e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6b1c157e20 0x7f6b1c157580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50010/0 (socket says 172.21.15.131:50010) 2024-05-26T22:50:28.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.664+0000 7f6b13e00640 1 -- 172.21.15.131:0/1320321115 learned_addr learned my addr 172.21.15.131:0/1320321115 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:28.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.664+0000 7f6b20c00640 1 --2- 172.21.15.131:0/1320321115 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6afc003a00 0x7f6b1c15ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:28.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.664+0000 7f6b21600640 1 --2- 172.21.15.131:0/1320321115 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6b1c161100 0x7f6b1c161580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:28.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.664+0000 7f6b13e00640 1 -- 172.21.15.131:0/1320321115 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6afc003a00 msgr2=0x7f6b1c15ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:28.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.664+0000 7f6b13e00640 1 --2- 172.21.15.131:0/1320321115 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6afc003a00 0x7f6b1c15ec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:28.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.664+0000 7f6b13e00640 1 -- 172.21.15.131:0/1320321115 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6b1c161100 msgr2=0x7f6b1c161580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:28.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.664+0000 7f6b13e00640 1 --2- 172.21.15.131:0/1320321115 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6b1c161100 0x7f6b1c161580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:28.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.664+0000 7f6b13e00640 1 -- 172.21.15.131:0/1320321115 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b1c15f1b0 con 0x7f6b1c157e20 2024-05-26T22:50:28.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.665+0000 7f6b13e00640 1 --2- 172.21.15.131:0/1320321115 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6b1c157e20 0x7f6b1c157580 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c00d770 tx=0x7f6b0c00db70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:28.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.665+0000 7f6b19600640 1 -- 172.21.15.131:0/1320321115 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b0c019070 con 0x7f6b1c157e20 2024-05-26T22:50:28.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.665+0000 7f6b19600640 1 -- 172.21.15.131:0/1320321115 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b0c0072d0 con 0x7f6b1c157e20 2024-05-26T22:50:28.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.665+0000 7f6b23600640 1 -- 172.21.15.131:0/1320321115 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b1c15f690 con 0x7f6b1c157e20 2024-05-26T22:50:28.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.665+0000 7f6b19600640 1 -- 172.21.15.131:0/1320321115 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b0c0047e0 con 0x7f6b1c157e20 2024-05-26T22:50:28.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.665+0000 7f6b23600640 1 -- 172.21.15.131:0/1320321115 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b1c15b580 con 0x7f6b1c157e20 2024-05-26T22:50:28.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.666+0000 7f6b23600640 1 -- 172.21.15.131:0/1320321115 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b1c15b780 con 0x7f6b1c157e20 2024-05-26T22:50:28.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.667+0000 7f6b19600640 1 -- 172.21.15.131:0/1320321115 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6b0c01c070 con 0x7f6b1c157e20 2024-05-26T22:50:28.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.668+0000 7f6b19600640 1 --2- 172.21.15.131:0/1320321115 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f6af806be30 0x7f6af806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:28.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.668+0000 7f6b19600640 1 -- 172.21.15.131:0/1320321115 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f6b0c08a8a0 con 0x7f6b1c157e20 2024-05-26T22:50:28.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.668+0000 7f6b20c00640 1 --2- 172.21.15.131:0/1320321115 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f6af806be30 0x7f6af806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:28.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.669+0000 7f6b20c00640 1 --2- 172.21.15.131:0/1320321115 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f6af806be30 0x7f6af806e2f0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f6b04033000 tx=0x7f6b04008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:28.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.672+0000 7f6b19600640 1 -- 172.21.15.131:0/1320321115 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f6b0c0591b0 con 0x7f6b1c157e20 2024-05-26T22:50:28.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:28.908+0000 7f6b23600640 1 -- 172.21.15.131:0/1320321115 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f6b1c0695c0 con 0x7f6b1c157e20 2024-05-26T22:50:29.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:29.612+0000 7f6b19600640 1 -- 172.21.15.131:0/1320321115 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v28) ==== 118+0+0 (secure 0 0 0) 0x7f6b0c058ab0 con 0x7f6b1c157e20 2024-05-26T22:50:29.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:29.616+0000 7f6b23600640 1 -- 172.21.15.131:0/1320321115 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f6af806be30 msgr2=0x7f6af806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:29.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:29.616+0000 7f6b23600640 1 --2- 172.21.15.131:0/1320321115 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f6af806be30 0x7f6af806e2f0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f6b04033000 tx=0x7f6b04008030 comp rx=0 tx=0).stop 2024-05-26T22:50:29.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:29.616+0000 7f6b23600640 1 -- 172.21.15.131:0/1320321115 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6b1c157e20 msgr2=0x7f6b1c157580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:29.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:29.616+0000 7f6b23600640 1 --2- 172.21.15.131:0/1320321115 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6b1c157e20 0x7f6b1c157580 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c00d770 tx=0x7f6b0c00db70 comp rx=0 tx=0).stop 2024-05-26T22:50:29.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:29.616+0000 7f6b23600640 1 -- 172.21.15.131:0/1320321115 shutdown_connections 2024-05-26T22:50:29.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:29.616+0000 7f6b23600640 1 --2- 172.21.15.131:0/1320321115 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f6af806be30 0x7f6af806e2f0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:29.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:29.616+0000 7f6b23600640 1 --2- 172.21.15.131:0/1320321115 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6b1c161100 0x7f6b1c161580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:29.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:29.616+0000 7f6b23600640 1 --2- 172.21.15.131:0/1320321115 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6b1c157e20 0x7f6b1c157580 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:29.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:29.616+0000 7f6b23600640 1 --2- 172.21.15.131:0/1320321115 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6afc003a00 0x7f6b1c15ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:29.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:29.616+0000 7f6b23600640 1 -- 172.21.15.131:0/1320321115 >> 172.21.15.131:0/1320321115 conn(0x7f6b1c073070 msgr2=0x7f6b1c0c8010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:29.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:29.617+0000 7f6b23600640 1 -- 172.21.15.131:0/1320321115 shutdown_connections 2024-05-26T22:50:29.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:29.617+0000 7f6b23600640 1 -- 172.21.15.131:0/1320321115 wait complete. 2024-05-26T22:50:29.639 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-26T22:50:29.639 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-26T22:50:29.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:29.847+0000 7f6026200640 1 Processor -- start 2024-05-26T22:50:29.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:29.847+0000 7f6026200640 1 -- start start 2024-05-26T22:50:29.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:29.848+0000 7f6026200640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f602006f180 0x7f6020072e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:29.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:29.848+0000 7f6026200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f6020072540 0x7f6020161ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:29.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:29.848+0000 7f6026200640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60200af8e0 con 0x7f60200cfe50 2024-05-26T22:50:29.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:29.848+0000 7f6026200640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f60200cfb30 con 0x7f6020072540 2024-05-26T22:50:29.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:29.848+0000 7f6026200640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f60200cf200 con 0x7f602006f180 2024-05-26T22:50:29.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:29.849+0000 7f601ea00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f6020072540 0x7f6020161ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:29.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:29.849+0000 7f601ea00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f6020072540 0x7f6020161ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50022/0 (socket says 172.21.15.131:50022) 2024-05-26T22:50:29.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:29.849+0000 7f601ea00640 1 -- 172.21.15.131:0/3892752982 learned_addr learned my addr 172.21.15.131:0/3892752982 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:29.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:29.849+0000 7f601ea00640 1 -- 172.21.15.131:0/3892752982 >> v2:172.21.15.187:3300/0 conn(0x7f602006f180 msgr2=0x7f6020072e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:29.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:29.849+0000 7f601f400640 1 --2- 172.21.15.131:0/3892752982 >> v2:172.21.15.187:3300/0 conn(0x7f602006f180 0x7f6020072e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:29.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:29.849+0000 7f601ea00640 1 --2- 172.21.15.131:0/3892752982 >> v2:172.21.15.187:3300/0 conn(0x7f602006f180 0x7f6020072e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:29.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:29.849+0000 7f601ea00640 1 -- 172.21.15.131:0/3892752982 >> v1:172.21.15.131:6789/0 conn(0x7f60200cfe50 legacy=0x7f60200715a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:29.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:29.849+0000 7f601ea00640 1 -- 172.21.15.131:0/3892752982 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60200cf4e0 con 0x7f6020072540 2024-05-26T22:50:29.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:29.849+0000 7f601f400640 1 --2- 172.21.15.131:0/3892752982 >> v2:172.21.15.187:3300/0 conn(0x7f602006f180 0x7f6020072e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:50:29.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:29.850+0000 7f601ea00640 1 --2- 172.21.15.131:0/3892752982 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6020072540 0x7f6020161ff0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f60200d0c40 tx=0x7f601002a740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=199a42f056d8c6ef server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:29.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:29.851+0000 7f601e000640 1 -- 172.21.15.131:0/3892752982 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6010036070 con 0x7f6020072540 2024-05-26T22:50:29.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:29.851+0000 7f601e000640 1 -- 172.21.15.131:0/3892752982 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6010031600 con 0x7f6020072540 2024-05-26T22:50:29.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:29.851+0000 7f601e000640 1 -- 172.21.15.131:0/3892752982 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60100319d0 con 0x7f6020072540 2024-05-26T22:50:29.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:29.851+0000 7f6026200640 1 -- 172.21.15.131:0/3892752982 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6020072540 msgr2=0x7f6020161ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:29.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:29.851+0000 7f6026200640 1 --2- 172.21.15.131:0/3892752982 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6020072540 0x7f6020161ff0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f60200d0c40 tx=0x7f601002a740 comp rx=0 tx=0).stop 2024-05-26T22:50:29.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:29.851+0000 7f6026200640 1 -- 172.21.15.131:0/3892752982 shutdown_connections 2024-05-26T22:50:29.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:29.851+0000 7f6026200640 1 --2- 172.21.15.131:0/3892752982 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6020072540 0x7f6020161ff0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:29.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:29.851+0000 7f6026200640 1 --2- 172.21.15.131:0/3892752982 >> v2:172.21.15.187:3300/0 conn(0x7f602006f180 0x7f6020072e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:29.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:29.851+0000 7f6026200640 1 -- 172.21.15.131:0/3892752982 >> 172.21.15.131:0/3892752982 conn(0x7f602006bc90 msgr2=0x7f60200b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:29.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:29.851+0000 7f6026200640 1 -- 172.21.15.131:0/3892752982 shutdown_connections 2024-05-26T22:50:29.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:29.851+0000 7f6026200640 1 -- 172.21.15.131:0/3892752982 wait complete. 2024-05-26T22:50:29.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:29.853+0000 7f6026200640 1 Processor -- start 2024-05-26T22:50:29.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:29.853+0000 7f6026200640 1 -- start start 2024-05-26T22:50:29.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:29.854+0000 7f6026200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f602006f180 0x7f60201583e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:29.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:29.854+0000 7f6026200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6020072540 0x7f60201f7610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:29.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:29.854+0000 7f6026200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f60200cfe50 0x7f60200adf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:29.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:29.854+0000 7f6026200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f602006f6e0 con 0x7f6020072540 2024-05-26T22:50:29.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:29.854+0000 7f6026200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f602006f560 con 0x7f602006f180 2024-05-26T22:50:29.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:29.854+0000 7f6026200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f6020071cb0 con 0x7f60200cfe50 2024-05-26T22:50:29.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:29.854+0000 7f601ea00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6020072540 0x7f60201f7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:29.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:29.854+0000 7f601ea00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6020072540 0x7f60201f7610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50026/0 (socket says 172.21.15.131:50026) 2024-05-26T22:50:29.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:29.854+0000 7f601ea00640 1 -- 172.21.15.131:0/1933914534 learned_addr learned my addr 172.21.15.131:0/1933914534 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:29.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:29.855+0000 7f601f400640 1 --2- 172.21.15.131:0/1933914534 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f602006f180 0x7f60201583e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:29.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:29.855+0000 7f601ea00640 1 -- 172.21.15.131:0/1933914534 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f60200cfe50 msgr2=0x7f60200adf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:29.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:29.855+0000 7f601ea00640 1 --2- 172.21.15.131:0/1933914534 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f60200cfe50 0x7f60200adf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:29.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:29.855+0000 7f601ea00640 1 -- 172.21.15.131:0/1933914534 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f602006f180 msgr2=0x7f60201583e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:29.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:29.855+0000 7f601ea00640 1 --2- 172.21.15.131:0/1933914534 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f602006f180 0x7f60201583e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:29.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:29.855+0000 7f601ea00640 1 -- 172.21.15.131:0/1933914534 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60201f7ef0 con 0x7f6020072540 2024-05-26T22:50:29.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:29.855+0000 7f601ea00640 1 --2- 172.21.15.131:0/1933914534 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6020072540 0x7f60201f7610 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f6010034040 tx=0x7f6010004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:29.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:29.856+0000 7f6017400640 1 -- 172.21.15.131:0/1933914534 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6010036050 con 0x7f6020072540 2024-05-26T22:50:29.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:29.856+0000 7f6017400640 1 -- 172.21.15.131:0/1933914534 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6010042070 con 0x7f6020072540 2024-05-26T22:50:29.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:29.856+0000 7f6017400640 1 -- 172.21.15.131:0/1933914534 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6010009510 con 0x7f6020072540 2024-05-26T22:50:29.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:29.856+0000 7f6026200640 1 -- 172.21.15.131:0/1933914534 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60201f81d0 con 0x7f6020072540 2024-05-26T22:50:29.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:29.856+0000 7f6026200640 1 -- 172.21.15.131:0/1933914534 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60201f5280 con 0x7f6020072540 2024-05-26T22:50:29.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:29.857+0000 7f6026200640 1 -- 172.21.15.131:0/1933914534 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60200695c0 con 0x7f6020072540 2024-05-26T22:50:29.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:29.858+0000 7f6017400640 1 -- 172.21.15.131:0/1933914534 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6010007c30 con 0x7f6020072540 2024-05-26T22:50:29.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:29.859+0000 7f6017400640 1 --2- 172.21.15.131:0/1933914534 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f5ffc06be60 0x7f5ffc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:29.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:29.859+0000 7f6017400640 1 -- 172.21.15.131:0/1933914534 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f60100abd30 con 0x7f6020072540 2024-05-26T22:50:29.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:29.859+0000 7f601f400640 1 --2- 172.21.15.131:0/1933914534 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f5ffc06be60 0x7f5ffc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:29.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:29.860+0000 7f601f400640 1 --2- 172.21.15.131:0/1933914534 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f5ffc06be60 0x7f5ffc06e320 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f602015ca20 tx=0x7f6008008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:29.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:29.864+0000 7f6017400640 1 -- 172.21.15.131:0/1933914534 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f601007a770 con 0x7f6020072540 2024-05-26T22:50:30.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:30.095+0000 7f6026200640 1 -- 172.21.15.131:0/1933914534 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f60200b16d0 con 0x7f6020072540 2024-05-26T22:50:30.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:30.615+0000 7f6017400640 1 -- 172.21.15.131:0/1933914534 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v29) ==== 122+0+0 (secure 0 0 0) 0x7f601007a070 con 0x7f6020072540 2024-05-26T22:50:30.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:30.619+0000 7f6026200640 1 -- 172.21.15.131:0/1933914534 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f5ffc06be60 msgr2=0x7f5ffc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:30.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:30.619+0000 7f6026200640 1 --2- 172.21.15.131:0/1933914534 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f5ffc06be60 0x7f5ffc06e320 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f602015ca20 tx=0x7f6008008040 comp rx=0 tx=0).stop 2024-05-26T22:50:30.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:30.619+0000 7f6026200640 1 -- 172.21.15.131:0/1933914534 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6020072540 msgr2=0x7f60201f7610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:30.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:30.619+0000 7f6026200640 1 --2- 172.21.15.131:0/1933914534 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6020072540 0x7f60201f7610 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f6010034040 tx=0x7f6010004a40 comp rx=0 tx=0).stop 2024-05-26T22:50:30.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:30.619+0000 7f6026200640 1 -- 172.21.15.131:0/1933914534 shutdown_connections 2024-05-26T22:50:30.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:30.619+0000 7f6026200640 1 --2- 172.21.15.131:0/1933914534 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f60200cfe50 0x7f60200adf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:30.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:30.619+0000 7f6026200640 1 --2- 172.21.15.131:0/1933914534 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f5ffc06be60 0x7f5ffc06e320 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:30.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:30.620+0000 7f6026200640 1 --2- 172.21.15.131:0/1933914534 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6020072540 0x7f60201f7610 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:30.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:30.620+0000 7f6026200640 1 --2- 172.21.15.131:0/1933914534 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f602006f180 0x7f60201583e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:30.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:30.620+0000 7f6026200640 1 -- 172.21.15.131:0/1933914534 >> 172.21.15.131:0/1933914534 conn(0x7f602006bc90 msgr2=0x7f60200d0280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:30.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:30.620+0000 7f6026200640 1 -- 172.21.15.131:0/1933914534 shutdown_connections 2024-05-26T22:50:30.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:30.620+0000 7f6026200640 1 -- 172.21.15.131:0/1933914534 wait complete. 2024-05-26T22:50:30.640 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-26T22:50:30.641 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-26T22:50:30.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:30.837+0000 7f2fd2800640 1 Processor -- start 2024-05-26T22:50:30.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:30.837+0000 7f2fd2800640 1 -- start start 2024-05-26T22:50:30.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:30.839+0000 7f2fd2800640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f2fcc0cedf0 0x7f2fcc071b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:30.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:30.839+0000 7f2fd2800640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2fcc0722a0 con 0x7f2fcc06ebc0 2024-05-26T22:50:30.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:30.839+0000 7f2fd2800640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2fcc072580 con 0x7f2fcc144340 2024-05-26T22:50:30.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:30.839+0000 7f2fd2800640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f2fcc072eb0 con 0x7f2fcc0cedf0 2024-05-26T22:50:30.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:30.839+0000 7f2fd0a00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f2fcc06ebc0 0x7f2fcc0b3d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:38460/0 (socket says 172.21.15.131:38460) 2024-05-26T22:50:30.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:30.839+0000 7f2fd0a00640 1 -- 172.21.15.131:0/3896654174 learned_addr learned my addr 172.21.15.131:0/3896654174 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:50:30.841 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:30.839+0000 7f2fd1e00640 1 --2- 172.21.15.131:0/3896654174 >> v2:172.21.15.187:3300/0 conn(0x7f2fcc0cedf0 0x7f2fcc071b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:30.841 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:30.839+0000 7f2fcbe00640 1 -- 172.21.15.131:0/3896654174 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 689373361 0 0) 0x7f2fcc0722a0 con 0x7f2fcc06ebc0 2024-05-26T22:50:30.841 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:30.839+0000 7f2fcbe00640 1 -- 172.21.15.131:0/3896654174 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2fb4003520 con 0x7f2fcc06ebc0 2024-05-26T22:50:30.841 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:30.840+0000 7f2fcbe00640 1 -- 172.21.15.131:0/3896654174 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4038192890 0 0) 0x7f2fb4003520 con 0x7f2fcc06ebc0 2024-05-26T22:50:30.841 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:30.840+0000 7f2fcbe00640 1 -- 172.21.15.131:0/3896654174 >> v2:172.21.15.187:3300/0 conn(0x7f2fcc0cedf0 msgr2=0x7f2fcc071b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:30.841 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:30.840+0000 7f2fcbe00640 1 --2- 172.21.15.131:0/3896654174 >> v2:172.21.15.187:3300/0 conn(0x7f2fcc0cedf0 0x7f2fcc071b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:30.841 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:30.840+0000 7f2fcbe00640 1 -- 172.21.15.131:0/3896654174 >> v1:172.21.15.187:6789/0 conn(0x7f2fcc144340 legacy=0x7f2fcc06e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:30.841 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:30.840+0000 7f2fcbe00640 1 -- 172.21.15.131:0/3896654174 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fcc072860 con 0x7f2fcc06ebc0 2024-05-26T22:50:30.841 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:30.840+0000 7f2fcbe00640 1 -- 172.21.15.131:0/3896654174 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7f2fb8001690 con 0x7f2fcc06ebc0 2024-05-26T22:50:30.841 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:30.840+0000 7f2fcbe00640 1 -- 172.21.15.131:0/3896654174 >> v1:172.21.15.131:6789/0 conn(0x7f2fcc06ebc0 legacy=0x7f2fcc0b3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:30.841 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:30.840+0000 7f2fcbe00640 1 --2- 172.21.15.131:0/3896654174 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2fb4003a00 0x7f2fb4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:30.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:30.840+0000 7f2fcbe00640 1 --2- 172.21.15.131:0/3896654174 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2fb4004800 0x7f2fb4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:30.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:30.840+0000 7f2fcbe00640 1 --2- 172.21.15.131:0/3896654174 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2fb4005210 0x7f2fb400dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:30.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:30.840+0000 7f2fcbe00640 1 -- 172.21.15.131:0/3896654174 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb400e340 con 0x7f2fb4004800 2024-05-26T22:50:30.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:30.840+0000 7f2fcbe00640 1 -- 172.21.15.131:0/3896654174 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb400e4f0 con 0x7f2fb4005210 2024-05-26T22:50:30.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:30.840+0000 7f2fcbe00640 1 -- 172.21.15.131:0/3896654174 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f2fb400e6f0 con 0x7f2fb4003a00 2024-05-26T22:50:30.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:30.840+0000 7f2fd1e00640 1 --2- 172.21.15.131:0/3896654174 >> v2:172.21.15.187:3300/0 conn(0x7f2fcc0cedf0 0x7f2fcc071b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-26T22:50:30.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:30.840+0000 7f2fd0a00640 1 --2- 172.21.15.131:0/3896654174 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2fb4004800 0x7f2fb4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:30.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:30.841+0000 7f2fd0a00640 1 -- 172.21.15.131:0/3896654174 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2fb4003a00 msgr2=0x7f2fb4003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:30.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:30.841+0000 7f2fd1400640 1 --2- 172.21.15.131:0/3896654174 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2fb4003a00 0x7f2fb4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:30.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:30.841+0000 7f2fd0a00640 1 --2- 172.21.15.131:0/3896654174 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2fb4003a00 0x7f2fb4003e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:30.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:30.841+0000 7f2fd0a00640 1 -- 172.21.15.131:0/3896654174 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2fb4005210 msgr2=0x7f2fb400dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:30.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:30.841+0000 7f2fd1e00640 1 --2- 172.21.15.131:0/3896654174 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2fb4005210 0x7f2fb400dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:30.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:30.841+0000 7f2fd0a00640 1 --2- 172.21.15.131:0/3896654174 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2fb4005210 0x7f2fb400dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:30.843 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:30.841+0000 7f2fd1e00640 1 -- 172.21.15.131:0/3896654174 reap_dead start 2024-05-26T22:50:30.843 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:30.841+0000 7f2fd0a00640 1 -- 172.21.15.131:0/3896654174 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fb4003520 con 0x7f2fb4004800 2024-05-26T22:50:30.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:30.842+0000 7f2fd0a00640 1 --2- 172.21.15.131:0/3896654174 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2fb4004800 0x7f2fb4004c80 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f2fb800c160 tx=0x7f2fb4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23a42323cb6dfa9e server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:30.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:30.842+0000 7f2fcbe00640 1 -- 172.21.15.131:0/3896654174 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb8007c60 con 0x7f2fb4004800 2024-05-26T22:50:30.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:30.842+0000 7f2fcbe00640 1 -- 172.21.15.131:0/3896654174 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fb8005af0 con 0x7f2fb4004800 2024-05-26T22:50:30.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:30.842+0000 7f2fcbe00640 1 -- 172.21.15.131:0/3896654174 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb8007070 con 0x7f2fb4004800 2024-05-26T22:50:30.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:30.842+0000 7f2fd2800640 1 -- 172.21.15.131:0/3896654174 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2fb4004800 msgr2=0x7f2fb4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:30.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:30.842+0000 7f2fd2800640 1 --2- 172.21.15.131:0/3896654174 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2fb4004800 0x7f2fb4004c80 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f2fb800c160 tx=0x7f2fb4005830 comp rx=0 tx=0).stop 2024-05-26T22:50:30.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:30.843+0000 7f2fd2800640 1 -- 172.21.15.131:0/3896654174 shutdown_connections 2024-05-26T22:50:30.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:30.843+0000 7f2fd2800640 1 --2- 172.21.15.131:0/3896654174 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2fb4004800 0x7f2fb4004c80 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:30.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:30.843+0000 7f2fd2800640 1 -- 172.21.15.131:0/3896654174 >> 172.21.15.131:0/3896654174 conn(0x7f2fcc0cff40 msgr2=0x7f2fcc0d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:30.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:30.843+0000 7f2fd2800640 1 -- 172.21.15.131:0/3896654174 shutdown_connections 2024-05-26T22:50:30.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:30.843+0000 7f2fd2800640 1 -- 172.21.15.131:0/3896654174 wait complete. 2024-05-26T22:50:30.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:30.844+0000 7f2fd2800640 1 Processor -- start 2024-05-26T22:50:30.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:30.844+0000 7f2fd2800640 1 -- start start 2024-05-26T22:50:30.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:30.845+0000 7f2fd2800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2fb4004800 0x7f2fcc15aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:30.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:30.845+0000 7f2fd2800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2fcc0b0d00 0x7f2fcc15b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:30.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:30.845+0000 7f2fd2800640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2fcc1f43d0 0x7f2fcc13fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:30.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:30.845+0000 7f2fd2800640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f2fcc071b70 con 0x7f2fb4004800 2024-05-26T22:50:30.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:30.845+0000 7f2fd2800640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2fcc0cf200 con 0x7f2fcc1f43d0 2024-05-26T22:50:30.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:30.845+0000 7f2fd2800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f2fcc071cf0 con 0x7f2fcc0b0d00 2024-05-26T22:50:30.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:30.846+0000 7f2fd1400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2fb4004800 0x7f2fcc15aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:30.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:30.846+0000 7f2fd1400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2fb4004800 0x7f2fcc15aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50054/0 (socket says 172.21.15.131:50054) 2024-05-26T22:50:30.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:30.846+0000 7f2fd1400640 1 -- 172.21.15.131:0/2349903606 learned_addr learned my addr 172.21.15.131:0/2349903606 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:30.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:30.846+0000 7f2fd1400640 1 -- 172.21.15.131:0/2349903606 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2fcc0b0d00 msgr2=0x7f2fcc15b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:30.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:30.846+0000 7f2fd1400640 1 --2- 172.21.15.131:0/2349903606 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2fcc0b0d00 0x7f2fcc15b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:30.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:30.846+0000 7f2fd1400640 1 -- 172.21.15.131:0/2349903606 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2fcc1f43d0 msgr2=0x7f2fcc13fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:30.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:30.846+0000 7f2fd1400640 1 --2- 172.21.15.131:0/2349903606 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2fcc1f43d0 0x7f2fcc13fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:30.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:30.846+0000 7f2fd1400640 1 -- 172.21.15.131:0/2349903606 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fcc15bee0 con 0x7f2fb4004800 2024-05-26T22:50:30.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:30.846+0000 7f2fd1400640 1 --2- 172.21.15.131:0/2349903606 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2fb4004800 0x7f2fcc15aa10 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f2fc000d5a0 tx=0x7f2fc000da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:30.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:30.846+0000 7f2fc9600640 1 -- 172.21.15.131:0/2349903606 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc0005480 con 0x7f2fb4004800 2024-05-26T22:50:30.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:30.847+0000 7f2fc9600640 1 -- 172.21.15.131:0/2349903606 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fc0017070 con 0x7f2fb4004800 2024-05-26T22:50:30.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:30.847+0000 7f2fd2800640 1 -- 172.21.15.131:0/2349903606 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fcc15c1c0 con 0x7f2fb4004800 2024-05-26T22:50:30.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:30.847+0000 7f2fc9600640 1 -- 172.21.15.131:0/2349903606 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc0003810 con 0x7f2fb4004800 2024-05-26T22:50:30.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:30.847+0000 7f2fd2800640 1 -- 172.21.15.131:0/2349903606 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fcc15bbf0 con 0x7f2fb4004800 2024-05-26T22:50:30.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:30.848+0000 7f2fd2800640 1 -- 172.21.15.131:0/2349903606 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fcc0694e0 con 0x7f2fb4004800 2024-05-26T22:50:30.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:30.849+0000 7f2fc9600640 1 -- 172.21.15.131:0/2349903606 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2fc00039b0 con 0x7f2fb4004800 2024-05-26T22:50:30.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:30.849+0000 7f2fc9600640 1 --2- 172.21.15.131:0/2349903606 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f2fac06be30 0x7f2fac06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:30.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:30.849+0000 7f2fc9600640 1 -- 172.21.15.131:0/2349903606 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f2fc008ac20 con 0x7f2fb4004800 2024-05-26T22:50:30.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:30.850+0000 7f2fd0a00640 1 --2- 172.21.15.131:0/2349903606 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f2fac06be30 0x7f2fac06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:30.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:30.850+0000 7f2fd0a00640 1 --2- 172.21.15.131:0/2349903606 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f2fac06be30 0x7f2fac06e2f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f2fb8032000 tx=0x7f2fb8007000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:30.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:30.854+0000 7f2fc9600640 1 -- 172.21.15.131:0/2349903606 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f2fc000f2f0 con 0x7f2fb4004800 2024-05-26T22:50:31.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.090+0000 7f2fd2800640 1 -- 172.21.15.131:0/2349903606 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2fcc0ce720 con 0x7f2fb4004800 2024-05-26T22:50:31.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.092+0000 7f2fc9600640 1 -- 172.21.15.131:0/2349903606 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+5187 (secure 0 0 0) 0x7f2fc00595c0 con 0x7f2fb4004800 2024-05-26T22:50:31.092 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:50:31.092 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":29,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4661,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/265040431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6835","nonce":265040431},{"type":"v1","addr":"172.21.15.131:6837","nonce":265040431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24},{"gid":4670,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6834/1027783169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1027783169},{"type":"v1","addr":"172.21.15.187:6834","nonce":1027783169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24},{"gid":4918,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/3429891101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6833","nonce":3429891101},{"type":"v1","addr":"172.21.15.187:6835","nonce":3429891101}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-26T22:50:27.001665+0000","modified":"2024-05-26T22:50:30.097328+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5637},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5637":{"gid":5637,"name":"c","rank":0,"incarnation":25,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6836/650770835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":650770835},{"type":"v1","addr":"172.21.15.131:6836","nonce":650770835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5637,"qdb_cluster":[5637]},"id":3}]} 2024-05-26T22:50:31.092 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 29 2024-05-26T22:50:31.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.095+0000 7f2fd2800640 1 -- 172.21.15.131:0/2349903606 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f2fac06be30 msgr2=0x7f2fac06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:31.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.095+0000 7f2fd2800640 1 --2- 172.21.15.131:0/2349903606 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f2fac06be30 0x7f2fac06e2f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f2fb8032000 tx=0x7f2fb8007000 comp rx=0 tx=0).stop 2024-05-26T22:50:31.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.095+0000 7f2fd2800640 1 -- 172.21.15.131:0/2349903606 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2fb4004800 msgr2=0x7f2fcc15aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:31.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.095+0000 7f2fd2800640 1 --2- 172.21.15.131:0/2349903606 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2fb4004800 0x7f2fcc15aa10 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f2fc000d5a0 tx=0x7f2fc000da60 comp rx=0 tx=0).stop 2024-05-26T22:50:31.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.095+0000 7f2fd2800640 1 -- 172.21.15.131:0/2349903606 shutdown_connections 2024-05-26T22:50:31.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.095+0000 7f2fd2800640 1 --2- 172.21.15.131:0/2349903606 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2fcc0b0d00 0x7f2fcc15b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:31.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.095+0000 7f2fd2800640 1 --2- 172.21.15.131:0/2349903606 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f2fac06be30 0x7f2fac06e2f0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:31.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.095+0000 7f2fd2800640 1 --2- 172.21.15.131:0/2349903606 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2fcc1f43d0 0x7f2fcc13fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:31.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.095+0000 7f2fd2800640 1 --2- 172.21.15.131:0/2349903606 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2fb4004800 0x7f2fcc15aa10 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:31.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.095+0000 7f2fd2800640 1 -- 172.21.15.131:0/2349903606 >> 172.21.15.131:0/2349903606 conn(0x7f2fcc0cff40 msgr2=0x7f2fcc1439a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:31.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.096+0000 7f2fd2800640 1 -- 172.21.15.131:0/2349903606 shutdown_connections 2024-05-26T22:50:31.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.096+0000 7f2fd2800640 1 -- 172.21.15.131:0/2349903606 wait complete. 2024-05-26T22:50:31.113 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-26T22:50:31.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.296+0000 7f8ed8c00640 1 Processor -- start 2024-05-26T22:50:31.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.296+0000 7f8ed8c00640 1 -- start start 2024-05-26T22:50:31.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.297+0000 7f8ed8c00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8ed4158c00 0x7f8ed406d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:31.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.297+0000 7f8ed8c00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f8ed40b38a0 0x7f8ed415fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:31.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.297+0000 7f8ed8c00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ed40b3e50 con 0x7f8ed406daf0 2024-05-26T22:50:31.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.297+0000 7f8ed8c00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f8ed406df50 con 0x7f8ed40b38a0 2024-05-26T22:50:31.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.297+0000 7f8ed8c00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f8ed40b4130 con 0x7f8ed4158c00 2024-05-26T22:50:31.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.298+0000 7f8ed2800640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f8ed40b38a0 0x7f8ed415fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:31.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.298+0000 7f8ed2800640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f8ed40b38a0 0x7f8ed415fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50056/0 (socket says 172.21.15.131:50056) 2024-05-26T22:50:31.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.298+0000 7f8ed2800640 1 -- 172.21.15.131:0/32679378 learned_addr learned my addr 172.21.15.131:0/32679378 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:31.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.298+0000 7f8ed1e00640 1 --2- 172.21.15.131:0/32679378 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8ed4158c00 0x7f8ed406d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:31.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.298+0000 7f8ed2800640 1 -- 172.21.15.131:0/32679378 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8ed4158c00 msgr2=0x7f8ed406d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:31.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.298+0000 7f8ed2800640 1 --2- 172.21.15.131:0/32679378 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8ed4158c00 0x7f8ed406d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:31.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.298+0000 7f8ed2800640 1 -- 172.21.15.131:0/32679378 >> v1:172.21.15.131:6789/0 conn(0x7f8ed406daf0 legacy=0x7f8ed40b3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:31.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.298+0000 7f8ed2800640 1 -- 172.21.15.131:0/32679378 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ed40b4410 con 0x7f8ed40b38a0 2024-05-26T22:50:31.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.299+0000 7f8ed2800640 1 --2- 172.21.15.131:0/32679378 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ed40b38a0 0x7f8ed415fc50 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc009e80 tx=0x7f8ecc02a6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94e2e07ca276fdc0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:31.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.300+0000 7f8ed1400640 1 -- 172.21.15.131:0/32679378 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ecc036070 con 0x7f8ed40b38a0 2024-05-26T22:50:31.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.300+0000 7f8ed1400640 1 -- 172.21.15.131:0/32679378 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ecc031600 con 0x7f8ed40b38a0 2024-05-26T22:50:31.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.300+0000 7f8ed1400640 1 -- 172.21.15.131:0/32679378 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ecc0319d0 con 0x7f8ed40b38a0 2024-05-26T22:50:31.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.300+0000 7f8ed8c00640 1 -- 172.21.15.131:0/32679378 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ed40b38a0 msgr2=0x7f8ed415fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:31.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.300+0000 7f8ed8c00640 1 --2- 172.21.15.131:0/32679378 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ed40b38a0 0x7f8ed415fc50 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc009e80 tx=0x7f8ecc02a6b0 comp rx=0 tx=0).stop 2024-05-26T22:50:31.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.300+0000 7f8ed8c00640 1 -- 172.21.15.131:0/32679378 shutdown_connections 2024-05-26T22:50:31.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.300+0000 7f8ed8c00640 1 --2- 172.21.15.131:0/32679378 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ed40b38a0 0x7f8ed415fc50 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:31.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.300+0000 7f8ed8c00640 1 --2- 172.21.15.131:0/32679378 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8ed4158c00 0x7f8ed406d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:31.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.300+0000 7f8ed8c00640 1 -- 172.21.15.131:0/32679378 >> 172.21.15.131:0/32679378 conn(0x7f8ed40b1e90 msgr2=0x7f8ed40b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:31.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.300+0000 7f8ed8c00640 1 -- 172.21.15.131:0/32679378 shutdown_connections 2024-05-26T22:50:31.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.301+0000 7f8ed8c00640 1 -- 172.21.15.131:0/32679378 wait complete. 2024-05-26T22:50:31.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.303+0000 7f8ed8c00640 1 Processor -- start 2024-05-26T22:50:31.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.303+0000 7f8ed8c00640 1 -- start start 2024-05-26T22:50:31.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.304+0000 7f8ed8c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ed406daf0 0x7f8ed409a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:31.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.304+0000 7f8ed8c00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ed40b38a0 0x7f8ed409ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:31.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.304+0000 7f8ed8c00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8ed4158c00 0x7f8ed409d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:31.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.304+0000 7f8ed8c00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed4160190 con 0x7f8ed406daf0 2024-05-26T22:50:31.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.304+0000 7f8ed8c00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed40b49d0 con 0x7f8ed40b38a0 2024-05-26T22:50:31.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.304+0000 7f8ed8c00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f8ed4160310 con 0x7f8ed4158c00 2024-05-26T22:50:31.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.304+0000 7f8ed1e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ed406daf0 0x7f8ed409a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:31.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.304+0000 7f8ed1e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ed406daf0 0x7f8ed409a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50068/0 (socket says 172.21.15.131:50068) 2024-05-26T22:50:31.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.304+0000 7f8ed1e00640 1 -- 172.21.15.131:0/782433039 learned_addr learned my addr 172.21.15.131:0/782433039 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:31.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.305+0000 7f8ec9400640 1 --2- 172.21.15.131:0/782433039 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ed40b38a0 0x7f8ed409ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:31.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.305+0000 7f8ed2800640 1 --2- 172.21.15.131:0/782433039 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8ed4158c00 0x7f8ed409d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:31.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.305+0000 7f8ed1e00640 1 -- 172.21.15.131:0/782433039 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8ed4158c00 msgr2=0x7f8ed409d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:31.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.305+0000 7f8ed1e00640 1 --2- 172.21.15.131:0/782433039 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8ed4158c00 0x7f8ed409d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:31.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.305+0000 7f8ed1e00640 1 -- 172.21.15.131:0/782433039 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ed40b38a0 msgr2=0x7f8ed409ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:31.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.305+0000 7f8ed1e00640 1 --2- 172.21.15.131:0/782433039 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ed40b38a0 0x7f8ed409ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:31.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.305+0000 7f8ed1e00640 1 -- 172.21.15.131:0/782433039 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ed4157ce0 con 0x7f8ed406daf0 2024-05-26T22:50:31.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.305+0000 7f8ed2800640 1 --2- 172.21.15.131:0/782433039 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8ed4158c00 0x7f8ed409d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:50:31.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.305+0000 7f8ec9400640 1 --2- 172.21.15.131:0/782433039 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ed40b38a0 0x7f8ed409ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:50:31.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.305+0000 7f8ed1e00640 1 --2- 172.21.15.131:0/782433039 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ed406daf0 0x7f8ed409a5d0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f8ec400d620 tx=0x7f8ec400dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:31.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.305+0000 7f8ecaa00640 1 -- 172.21.15.131:0/782433039 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec400b620 con 0x7f8ed406daf0 2024-05-26T22:50:31.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.305+0000 7f8ecaa00640 1 -- 172.21.15.131:0/782433039 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ec4019070 con 0x7f8ed406daf0 2024-05-26T22:50:31.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.305+0000 7f8ed8c00640 1 -- 172.21.15.131:0/782433039 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ed41591a0 con 0x7f8ed406daf0 2024-05-26T22:50:31.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.305+0000 7f8ecaa00640 1 -- 172.21.15.131:0/782433039 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec400ba00 con 0x7f8ed406daf0 2024-05-26T22:50:31.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.306+0000 7f8ed8c00640 1 -- 172.21.15.131:0/782433039 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ed4159760 con 0x7f8ed406daf0 2024-05-26T22:50:31.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.307+0000 7f8ed8c00640 1 -- 172.21.15.131:0/782433039 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ed40694e0 con 0x7f8ed406daf0 2024-05-26T22:50:31.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.308+0000 7f8ecaa00640 1 -- 172.21.15.131:0/782433039 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8ec400bba0 con 0x7f8ed406daf0 2024-05-26T22:50:31.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.309+0000 7f8ecaa00640 1 --2- 172.21.15.131:0/782433039 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f8eb006be30 0x7f8eb006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:31.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.310+0000 7f8ecaa00640 1 -- 172.21.15.131:0/782433039 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f8ec408b200 con 0x7f8ed406daf0 2024-05-26T22:50:31.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.310+0000 7f8ec9400640 1 --2- 172.21.15.131:0/782433039 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f8eb006be30 0x7f8eb006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:31.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.310+0000 7f8ec9400640 1 --2- 172.21.15.131:0/782433039 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f8eb006be30 0x7f8eb006e2f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f8ed41f76d0 tx=0x7f8ebc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:31.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.314+0000 7f8ecaa00640 1 -- 172.21.15.131:0/782433039 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f8ec4059c40 con 0x7f8ed406daf0 2024-05-26T22:50:31.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.510+0000 7f8ed8c00640 1 -- 172.21.15.131:0/782433039 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8ed4158340 con 0x7f8ed406daf0 2024-05-26T22:50:31.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.513+0000 7f8ecaa00640 1 -- 172.21.15.131:0/782433039 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) ==== 74+0+16637 (secure 0 0 0) 0x7f8ec4059540 con 0x7f8ed406daf0 2024-05-26T22:50:31.514 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:50:31.514 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":44,"fsid":"46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f","created":"2024-05-26T22:47:57.288905+0000","modified":"2024-05-26T22:50:28.592544+0000","last_up_change":"2024-05-26T22:48:09.602207+0000","last_in_change":"2024-05-26T22:48:00.687341+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-26T22:48:10.463339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-05-26T22:50:24.793406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-05-26T22:50:26.041636+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"814eaf62-a18d-494b-8e27-fd8fb88adea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6801","nonce":2350940051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6803","nonce":2350940051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6807","nonce":2350940051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6805","nonce":2350940051}]},"public_addr":"172.21.15.131:6801/2350940051","cluster_addr":"172.21.15.131:6803/2350940051","heartbeat_back_addr":"172.21.15.131:6807/2350940051","heartbeat_front_addr":"172.21.15.131:6805/2350940051","state":["exists","up"]},{"osd":1,"uuid":"42e66287-f6c6-4e26-af3c-6dfaa3536d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6825","nonce":1525528247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6827","nonce":1525528247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6831","nonce":1525528247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6829","nonce":1525528247}]},"public_addr":"172.21.15.131:6825/1525528247","cluster_addr":"172.21.15.131:6827/1525528247","heartbeat_back_addr":"172.21.15.131:6831/1525528247","heartbeat_front_addr":"172.21.15.131:6829/1525528247","state":["exists","up"]},{"osd":2,"uuid":"0da34673-04a2-4fd0-b57a-0ff2b679b043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6817","nonce":3627842210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6819","nonce":3627842210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6823","nonce":3627842210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6821","nonce":3627842210}]},"public_addr":"172.21.15.131:6817/3627842210","cluster_addr":"172.21.15.131:6819/3627842210","heartbeat_back_addr":"172.21.15.131:6823/3627842210","heartbeat_front_addr":"172.21.15.131:6821/3627842210","state":["exists","up"]},{"osd":3,"uuid":"cf117a27-d6b2-43d3-ae52-b47988bf8eed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6809","nonce":350790591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6811","nonce":350790591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6815","nonce":350790591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6813","nonce":350790591}]},"public_addr":"172.21.15.131:6809/350790591","cluster_addr":"172.21.15.131:6811/350790591","heartbeat_back_addr":"172.21.15.131:6815/350790591","heartbeat_front_addr":"172.21.15.131:6813/350790591","state":["exists","up"]},{"osd":4,"uuid":"a092ccfc-be89-4a84-88d5-0f781fa715eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6816","nonce":244560995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6819","nonce":244560995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6823","nonce":244560995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6821","nonce":244560995}]},"public_addr":"172.21.15.187:6816/244560995","cluster_addr":"172.21.15.187:6819/244560995","heartbeat_back_addr":"172.21.15.187:6823/244560995","heartbeat_front_addr":"172.21.15.187:6821/244560995","state":["exists","up"]},{"osd":5,"uuid":"7513ba69-dd00-4a35-b190-9e8cc3b2bdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6801","nonce":324619398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6803","nonce":324619398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6812","nonce":324619398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6807","nonce":324619398}]},"public_addr":"172.21.15.187:6801/324619398","cluster_addr":"172.21.15.187:6803/324619398","heartbeat_back_addr":"172.21.15.187:6812/324619398","heartbeat_front_addr":"172.21.15.187:6807/324619398","state":["exists","up"]},{"osd":6,"uuid":"50081810-c093-4d2f-a4d7-b7c695c5400d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6806","nonce":1391347786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6809","nonce":1391347786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6818","nonce":1391347786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6813","nonce":1391347786}]},"public_addr":"172.21.15.187:6806/1391347786","cluster_addr":"172.21.15.187:6809/1391347786","heartbeat_back_addr":"172.21.15.187:6818/1391347786","heartbeat_front_addr":"172.21.15.187:6813/1391347786","state":["exists","up"]},{"osd":7,"uuid":"775dae05-54f5-4ad4-bbce-92863c6ed2ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6825","nonce":1220065622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6827","nonce":1220065622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6831","nonce":1220065622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6829","nonce":1220065622}]},"public_addr":"172.21.15.187:6825/1220065622","cluster_addr":"172.21.15.187:6827/1220065622","heartbeat_back_addr":"172.21.15.187:6831/1220065622","heartbeat_front_addr":"172.21.15.187:6829/1220065622","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:05.983520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.299516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.067323+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.307748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.018742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.258775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.010912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.295355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-26T22:50:31.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.516+0000 7f8ed8c00640 1 -- 172.21.15.131:0/782433039 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f8eb006be30 msgr2=0x7f8eb006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:31.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.516+0000 7f8ed8c00640 1 --2- 172.21.15.131:0/782433039 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f8eb006be30 0x7f8eb006e2f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f8ed41f76d0 tx=0x7f8ebc009040 comp rx=0 tx=0).stop 2024-05-26T22:50:31.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.516+0000 7f8ed8c00640 1 -- 172.21.15.131:0/782433039 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ed406daf0 msgr2=0x7f8ed409a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:31.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.516+0000 7f8ed8c00640 1 --2- 172.21.15.131:0/782433039 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ed406daf0 0x7f8ed409a5d0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f8ec400d620 tx=0x7f8ec400dae0 comp rx=0 tx=0).stop 2024-05-26T22:50:31.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.516+0000 7f8ed8c00640 1 -- 172.21.15.131:0/782433039 shutdown_connections 2024-05-26T22:50:31.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.516+0000 7f8ed8c00640 1 --2- 172.21.15.131:0/782433039 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8ed4158c00 0x7f8ed409d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:31.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.516+0000 7f8ed8c00640 1 --2- 172.21.15.131:0/782433039 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f8eb006be30 0x7f8eb006e2f0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:31.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.516+0000 7f8ed8c00640 1 --2- 172.21.15.131:0/782433039 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ed40b38a0 0x7f8ed409ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:31.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.516+0000 7f8ed8c00640 1 --2- 172.21.15.131:0/782433039 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ed406daf0 0x7f8ed409a5d0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:31.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.516+0000 7f8ed8c00640 1 -- 172.21.15.131:0/782433039 >> 172.21.15.131:0/782433039 conn(0x7f8ed40b1e90 msgr2=0x7f8ed40b0890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:31.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.516+0000 7f8ed8c00640 1 -- 172.21.15.131:0/782433039 shutdown_connections 2024-05-26T22:50:31.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.516+0000 7f8ed8c00640 1 -- 172.21.15.131:0/782433039 wait complete. 2024-05-26T22:50:31.534 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-26T22:50:31.534 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-26T22:50:31.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.723+0000 7f50ebe00640 1 Processor -- start 2024-05-26T22:50:31.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.723+0000 7f50ebe00640 1 -- start start 2024-05-26T22:50:31.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.725+0000 7f50ebe00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f50ec0cc7f0 0x7f50ec0ccc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:31.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.725+0000 7f50ebe00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50ec160620 con 0x7f50ec06e8c0 2024-05-26T22:50:31.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.725+0000 7f50ebe00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50ec160900 con 0x7f50ec156750 2024-05-26T22:50:31.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.725+0000 7f50ebe00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f50ec15d770 con 0x7f50ec0cc7f0 2024-05-26T22:50:31.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.725+0000 7f50ea000640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f50ec06e8c0 0x7f50ec15d060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:38476/0 (socket says 172.21.15.131:38476) 2024-05-26T22:50:31.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.725+0000 7f50ea000640 1 -- 172.21.15.131:0/2838380326 learned_addr learned my addr 172.21.15.131:0/2838380326 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:50:31.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.725+0000 7f50eb400640 1 --2- 172.21.15.131:0/2838380326 >> v2:172.21.15.187:3300/0 conn(0x7f50ec0cc7f0 0x7f50ec0ccc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:31.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.726+0000 7f50e9600640 1 -- 172.21.15.131:0/2838380326 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3720197769 0 0) 0x7f50ec160620 con 0x7f50ec06e8c0 2024-05-26T22:50:31.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.726+0000 7f50e9600640 1 -- 172.21.15.131:0/2838380326 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f50d0003520 con 0x7f50ec06e8c0 2024-05-26T22:50:31.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.726+0000 7f50e9600640 1 -- 172.21.15.131:0/2838380326 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2089009555 0 0) 0x7f50d0003520 con 0x7f50ec06e8c0 2024-05-26T22:50:31.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.726+0000 7f50e9600640 1 -- 172.21.15.131:0/2838380326 >> v2:172.21.15.187:3300/0 conn(0x7f50ec0cc7f0 msgr2=0x7f50ec0ccc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:31.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.726+0000 7f50e9600640 1 --2- 172.21.15.131:0/2838380326 >> v2:172.21.15.187:3300/0 conn(0x7f50ec0cc7f0 0x7f50ec0ccc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:31.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.726+0000 7f50e9600640 1 -- 172.21.15.131:0/2838380326 >> v1:172.21.15.187:6789/0 conn(0x7f50ec156750 legacy=0x7f50ec15fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:31.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.726+0000 7f50e9600640 1 -- 172.21.15.131:0/2838380326 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50ec160be0 con 0x7f50ec06e8c0 2024-05-26T22:50:31.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.726+0000 7f50e9600640 1 -- 172.21.15.131:0/2838380326 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7f50d8001670 con 0x7f50ec06e8c0 2024-05-26T22:50:31.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.726+0000 7f50e9600640 1 -- 172.21.15.131:0/2838380326 >> v1:172.21.15.131:6789/0 conn(0x7f50ec06e8c0 legacy=0x7f50ec15d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:31.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.726+0000 7f50e9600640 1 --2- 172.21.15.131:0/2838380326 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f50d0003a00 0x7f50d0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:31.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.726+0000 7f50e9600640 1 --2- 172.21.15.131:0/2838380326 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50d0004800 0x7f50d0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:31.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.726+0000 7f50e9600640 1 --2- 172.21.15.131:0/2838380326 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f50d00051c0 0x7f50d000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:31.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.726+0000 7f50e9600640 1 -- 172.21.15.131:0/2838380326 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f50ec15d770 con 0x7f50d0004800 2024-05-26T22:50:31.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.726+0000 7f50e9600640 1 -- 172.21.15.131:0/2838380326 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f50d000c090 con 0x7f50d00051c0 2024-05-26T22:50:31.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.726+0000 7f50e9600640 1 -- 172.21.15.131:0/2838380326 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f50d000c290 con 0x7f50d0003a00 2024-05-26T22:50:31.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.726+0000 7f50ea000640 1 --2- 172.21.15.131:0/2838380326 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50d0004800 0x7f50d0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:31.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.727+0000 7f50ea000640 1 -- 172.21.15.131:0/2838380326 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f50d0003a00 msgr2=0x7f50d0003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:31.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.727+0000 7f50ea000640 1 --2- 172.21.15.131:0/2838380326 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f50d0003a00 0x7f50d0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:31.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.727+0000 7f50ea000640 1 -- 172.21.15.131:0/2838380326 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f50d00051c0 msgr2=0x7f50d000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:31.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.727+0000 7f50ea000640 1 --2- 172.21.15.131:0/2838380326 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f50d00051c0 0x7f50d000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:31.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.727+0000 7f50ea000640 1 -- 172.21.15.131:0/2838380326 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50d0003520 con 0x7f50d0004800 2024-05-26T22:50:31.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.727+0000 7f50eb400640 1 -- 172.21.15.131:0/2838380326 reap_dead start 2024-05-26T22:50:31.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.728+0000 7f50ea000640 1 --2- 172.21.15.131:0/2838380326 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50d0004800 0x7f50d0004c80 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f50d800c110 tx=0x7f50d0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a9b6d78b9f3f384 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:31.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.728+0000 7f50e9600640 1 -- 172.21.15.131:0/2838380326 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50d8007c40 con 0x7f50d0004800 2024-05-26T22:50:31.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.728+0000 7f50e9600640 1 -- 172.21.15.131:0/2838380326 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50d8005af0 con 0x7f50d0004800 2024-05-26T22:50:31.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.728+0000 7f50e9600640 1 -- 172.21.15.131:0/2838380326 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50d8007070 con 0x7f50d0004800 2024-05-26T22:50:31.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.728+0000 7f50ebe00640 1 -- 172.21.15.131:0/2838380326 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50d0004800 msgr2=0x7f50d0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:31.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.728+0000 7f50ebe00640 1 --2- 172.21.15.131:0/2838380326 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50d0004800 0x7f50d0004c80 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f50d800c110 tx=0x7f50d0005830 comp rx=0 tx=0).stop 2024-05-26T22:50:31.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.728+0000 7f50ebe00640 1 -- 172.21.15.131:0/2838380326 shutdown_connections 2024-05-26T22:50:31.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.728+0000 7f50ebe00640 1 --2- 172.21.15.131:0/2838380326 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50d0004800 0x7f50d0004c80 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:31.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.728+0000 7f50ebe00640 1 -- 172.21.15.131:0/2838380326 >> 172.21.15.131:0/2838380326 conn(0x7f50ec0cff40 msgr2=0x7f50ec0d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:31.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.729+0000 7f50ebe00640 1 -- 172.21.15.131:0/2838380326 shutdown_connections 2024-05-26T22:50:31.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.729+0000 7f50ebe00640 1 -- 172.21.15.131:0/2838380326 wait complete. 2024-05-26T22:50:31.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.730+0000 7f50ebe00640 1 Processor -- start 2024-05-26T22:50:31.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.730+0000 7f50ebe00640 1 -- start start 2024-05-26T22:50:31.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.731+0000 7f50ebe00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f50d0004800 0x7f50ec138230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:31.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.731+0000 7f50ebe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50ec0b5c70 0x7f50ec1f12a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:31.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.731+0000 7f50ebe00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f50ec158f70 0x7f50ec1593f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:31.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.731+0000 7f50ebe00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f50ec15cbd0 con 0x7f50ec0b5c70 2024-05-26T22:50:31.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.731+0000 7f50ebe00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f50ec0cd1a0 con 0x7f50d0004800 2024-05-26T22:50:31.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.731+0000 7f50ebe00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f50ec156b30 con 0x7f50ec158f70 2024-05-26T22:50:31.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.731+0000 7f50ea000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50ec0b5c70 0x7f50ec1f12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:31.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.731+0000 7f50ea000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50ec0b5c70 0x7f50ec1f12a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50082/0 (socket says 172.21.15.131:50082) 2024-05-26T22:50:31.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.731+0000 7f50ea000640 1 -- 172.21.15.131:0/2540946098 learned_addr learned my addr 172.21.15.131:0/2540946098 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:31.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.732+0000 7f50ea000640 1 -- 172.21.15.131:0/2540946098 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f50ec158f70 msgr2=0x7f50ec1593f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:31.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.732+0000 7f50ea000640 1 --2- 172.21.15.131:0/2540946098 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f50ec158f70 0x7f50ec1593f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:31.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.732+0000 7f50ea000640 1 -- 172.21.15.131:0/2540946098 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f50d0004800 msgr2=0x7f50ec138230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:31.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.732+0000 7f50ea000640 1 --2- 172.21.15.131:0/2540946098 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f50d0004800 0x7f50ec138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:31.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.732+0000 7f50ea000640 1 -- 172.21.15.131:0/2540946098 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50ec0b60f0 con 0x7f50ec0b5c70 2024-05-26T22:50:31.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.732+0000 7f50ea000640 1 --2- 172.21.15.131:0/2540946098 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50ec0b5c70 0x7f50ec1f12a0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f50d800c110 tx=0x7f50ec160db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:31.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.732+0000 7f50e2a00640 1 -- 172.21.15.131:0/2540946098 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50d8040030 con 0x7f50ec0b5c70 2024-05-26T22:50:31.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.732+0000 7f50e2a00640 1 -- 172.21.15.131:0/2540946098 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50d8001430 con 0x7f50ec0b5c70 2024-05-26T22:50:31.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.732+0000 7f50e2a00640 1 -- 172.21.15.131:0/2540946098 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50d8037040 con 0x7f50ec0b5c70 2024-05-26T22:50:31.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.732+0000 7f50ebe00640 1 -- 172.21.15.131:0/2540946098 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50ec0afb20 con 0x7f50ec0b5c70 2024-05-26T22:50:31.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.733+0000 7f50ebe00640 1 -- 172.21.15.131:0/2540946098 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50ec0afdc0 con 0x7f50ec0b5c70 2024-05-26T22:50:31.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.734+0000 7f50ebe00640 1 -- 172.21.15.131:0/2540946098 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50ec0694e0 con 0x7f50ec0b5c70 2024-05-26T22:50:31.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.735+0000 7f50e2a00640 1 -- 172.21.15.131:0/2540946098 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f50d8043070 con 0x7f50ec0b5c70 2024-05-26T22:50:31.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.735+0000 7f50e2a00640 1 --2- 172.21.15.131:0/2540946098 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f50cc06be60 0x7f50cc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:31.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.736+0000 7f50e2a00640 1 -- 172.21.15.131:0/2540946098 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f50d80aa9b0 con 0x7f50ec0b5c70 2024-05-26T22:50:31.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.736+0000 7f50eaa00640 1 --2- 172.21.15.131:0/2540946098 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f50cc06be60 0x7f50cc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:31.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.736+0000 7f50eaa00640 1 --2- 172.21.15.131:0/2540946098 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f50cc06be60 0x7f50cc06e320 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f50ec159e70 tx=0x7f50dc0079b0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:31.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.740+0000 7f50e2a00640 1 -- 172.21.15.131:0/2540946098 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f50d8005ac0 con 0x7f50ec0b5c70 2024-05-26T22:50:31.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.932+0000 7f50ebe00640 1 -- 172.21.15.131:0/2540946098 --> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f50ec0af970 con 0x7f50cc06be60 2024-05-26T22:50:31.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.942+0000 7f50e2a00640 1 -- 172.21.15.131:0/2540946098 <== mgr.4104 v2:172.21.15.131:6832/3484832947 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+237202 (secure 0 0 0) 0x7f50ec0af970 con 0x7f50cc06be60 2024-05-26T22:50:31.945 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:50:31.947 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-05-26T22:50:31.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.947+0000 7f50ebe00640 1 -- 172.21.15.131:0/2540946098 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f50cc06be60 msgr2=0x7f50cc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:31.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.947+0000 7f50ebe00640 1 --2- 172.21.15.131:0/2540946098 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f50cc06be60 0x7f50cc06e320 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f50ec159e70 tx=0x7f50dc0079b0 comp rx=0 tx=0).stop 2024-05-26T22:50:31.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.947+0000 7f50ebe00640 1 -- 172.21.15.131:0/2540946098 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50ec0b5c70 msgr2=0x7f50ec1f12a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:31.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.947+0000 7f50ebe00640 1 --2- 172.21.15.131:0/2540946098 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50ec0b5c70 0x7f50ec1f12a0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f50d800c110 tx=0x7f50ec160db0 comp rx=0 tx=0).stop 2024-05-26T22:50:31.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.947+0000 7f50ebe00640 1 -- 172.21.15.131:0/2540946098 shutdown_connections 2024-05-26T22:50:31.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.947+0000 7f50ebe00640 1 --2- 172.21.15.131:0/2540946098 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f50ec158f70 0x7f50ec1593f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:31.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.947+0000 7f50ebe00640 1 --2- 172.21.15.131:0/2540946098 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f50cc06be60 0x7f50cc06e320 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:31.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.947+0000 7f50ebe00640 1 --2- 172.21.15.131:0/2540946098 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50ec0b5c70 0x7f50ec1f12a0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:31.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.947+0000 7f50ebe00640 1 --2- 172.21.15.131:0/2540946098 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f50d0004800 0x7f50ec138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:31.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.947+0000 7f50ebe00640 1 -- 172.21.15.131:0/2540946098 >> 172.21.15.131:0/2540946098 conn(0x7f50ec0cff40 msgr2=0x7f50ec0cf260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:31.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.947+0000 7f50ebe00640 1 -- 172.21.15.131:0/2540946098 shutdown_connections 2024-05-26T22:50:31.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:31.947+0000 7f50ebe00640 1 -- 172.21.15.131:0/2540946098 wait complete. 2024-05-26T22:50:31.965 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":107,"stamp":"2024-05-26T22:50:30.424662+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":229320,"kb_used_data":4808,"kb_used_omap":12,"kb_used_meta":224371,"kb_avail":754745400,"statfs":{"total":773094113280,"available":772859289600,"internally_reserved":0,"allocated":4923392,"data_stored":2665864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":229756465},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.882072"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:25.526760+0000","last_change":"2024-05-26T22:50:25.526760+0000","last_active":"2024-05-26T22:50:25.526760+0000","last_peered":"2024-05-26T22:50:25.526760+0000","last_clean":"2024-05-26T22:50:25.526760+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:25.526760+0000","last_undegraded":"2024-05-26T22:50:25.526760+0000","last_fullsized":"2024-05-26T22:50:25.526760+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:25.526760+0000","last_change":"2024-05-26T22:50:25.526760+0000","last_active":"2024-05-26T22:50:25.526760+0000","last_peered":"2024-05-26T22:50:25.526760+0000","last_clean":"2024-05-26T22:50:25.526760+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:25.526760+0000","last_undegraded":"2024-05-26T22:50:25.526760+0000","last_fullsized":"2024-05-26T22:50:25.526760+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:25.526760+0000","last_change":"2024-05-26T22:50:25.526760+0000","last_active":"2024-05-26T22:50:25.526760+0000","last_peered":"2024-05-26T22:50:25.526760+0000","last_clean":"2024-05-26T22:50:25.526760+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:25.526760+0000","last_undegraded":"2024-05-26T22:50:25.526760+0000","last_fullsized":"2024-05-26T22:50:25.526760+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:25.526760+0000","last_change":"2024-05-26T22:50:25.526760+0000","last_active":"2024-05-26T22:50:25.526760+0000","last_peered":"2024-05-26T22:50:25.526760+0000","last_clean":"2024-05-26T22:50:25.526760+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:25.526760+0000","last_undegraded":"2024-05-26T22:50:25.526760+0000","last_fullsized":"2024-05-26T22:50:25.526760+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:25.526760+0000","last_change":"2024-05-26T22:50:25.526760+0000","last_active":"2024-05-26T22:50:25.526760+0000","last_peered":"2024-05-26T22:50:25.526760+0000","last_clean":"2024-05-26T22:50:25.526760+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:25.526760+0000","last_undegraded":"2024-05-26T22:50:25.526760+0000","last_fullsized":"2024-05-26T22:50:25.526760+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:25.526760+0000","last_change":"2024-05-26T22:50:25.526760+0000","last_active":"2024-05-26T22:50:25.526760+0000","last_peered":"2024-05-26T22:50:25.526760+0000","last_clean":"2024-05-26T22:50:25.526760+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:25.526760+0000","last_undegraded":"2024-05-26T22:50:25.526760+0000","last_fullsized":"2024-05-26T22:50:25.526760+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:25.526760+0000","last_change":"2024-05-26T22:50:25.526760+0000","last_active":"2024-05-26T22:50:25.526760+0000","last_peered":"2024-05-26T22:50:25.526760+0000","last_clean":"2024-05-26T22:50:25.526760+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:25.526760+0000","last_undegraded":"2024-05-26T22:50:25.526760+0000","last_fullsized":"2024-05-26T22:50:25.526760+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:25.526760+0000","last_change":"2024-05-26T22:50:25.526760+0000","last_active":"2024-05-26T22:50:25.526760+0000","last_peered":"2024-05-26T22:50:25.526760+0000","last_clean":"2024-05-26T22:50:25.526760+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:25.526760+0000","last_undegraded":"2024-05-26T22:50:25.526760+0000","last_fullsized":"2024-05-26T22:50:25.526760+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:25.526760+0000","last_change":"2024-05-26T22:50:25.526760+0000","last_active":"2024-05-26T22:50:25.526760+0000","last_peered":"2024-05-26T22:50:25.526760+0000","last_clean":"2024-05-26T22:50:25.526760+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:25.526760+0000","last_undegraded":"2024-05-26T22:50:25.526760+0000","last_fullsized":"2024-05-26T22:50:25.526760+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:25.526760+0000","last_change":"2024-05-26T22:50:25.526760+0000","last_active":"2024-05-26T22:50:25.526760+0000","last_peered":"2024-05-26T22:50:25.526760+0000","last_clean":"2024-05-26T22:50:25.526760+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:25.526760+0000","last_undegraded":"2024-05-26T22:50:25.526760+0000","last_fullsized":"2024-05-26T22:50:25.526760+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:25.526760+0000","last_change":"2024-05-26T22:50:25.526760+0000","last_active":"2024-05-26T22:50:25.526760+0000","last_peered":"2024-05-26T22:50:25.526760+0000","last_clean":"2024-05-26T22:50:25.526760+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:25.526760+0000","last_undegraded":"2024-05-26T22:50:25.526760+0000","last_fullsized":"2024-05-26T22:50:25.526760+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:25.526760+0000","last_change":"2024-05-26T22:50:25.526760+0000","last_active":"2024-05-26T22:50:25.526760+0000","last_peered":"2024-05-26T22:50:25.526760+0000","last_clean":"2024-05-26T22:50:25.526760+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:25.526760+0000","last_undegraded":"2024-05-26T22:50:25.526760+0000","last_fullsized":"2024-05-26T22:50:25.526760+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:25.526760+0000","last_change":"2024-05-26T22:50:25.526760+0000","last_active":"2024-05-26T22:50:25.526760+0000","last_peered":"2024-05-26T22:50:25.526760+0000","last_clean":"2024-05-26T22:50:25.526760+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:25.526760+0000","last_undegraded":"2024-05-26T22:50:25.526760+0000","last_fullsized":"2024-05-26T22:50:25.526760+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:25.526760+0000","last_change":"2024-05-26T22:50:25.526760+0000","last_active":"2024-05-26T22:50:25.526760+0000","last_peered":"2024-05-26T22:50:25.526760+0000","last_clean":"2024-05-26T22:50:25.526760+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:25.526760+0000","last_undegraded":"2024-05-26T22:50:25.526760+0000","last_fullsized":"2024-05-26T22:50:25.526760+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:25.526760+0000","last_change":"2024-05-26T22:50:25.526760+0000","last_active":"2024-05-26T22:50:25.526760+0000","last_peered":"2024-05-26T22:50:25.526760+0000","last_clean":"2024-05-26T22:50:25.526760+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:25.526760+0000","last_undegraded":"2024-05-26T22:50:25.526760+0000","last_fullsized":"2024-05-26T22:50:25.526760+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:25.526760+0000","last_change":"2024-05-26T22:50:25.526760+0000","last_active":"2024-05-26T22:50:25.526760+0000","last_peered":"2024-05-26T22:50:25.526760+0000","last_clean":"2024-05-26T22:50:25.526760+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:25.526760+0000","last_undegraded":"2024-05-26T22:50:25.526760+0000","last_fullsized":"2024-05-26T22:50:25.526760+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:25.526760+0000","last_change":"2024-05-26T22:50:25.526760+0000","last_active":"2024-05-26T22:50:25.526760+0000","last_peered":"2024-05-26T22:50:25.526760+0000","last_clean":"2024-05-26T22:50:25.526760+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:25.526760+0000","last_undegraded":"2024-05-26T22:50:25.526760+0000","last_fullsized":"2024-05-26T22:50:25.526760+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:25.526760+0000","last_change":"2024-05-26T22:50:25.526760+0000","last_active":"2024-05-26T22:50:25.526760+0000","last_peered":"2024-05-26T22:50:25.526760+0000","last_clean":"2024-05-26T22:50:25.526760+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:25.526760+0000","last_undegraded":"2024-05-26T22:50:25.526760+0000","last_fullsized":"2024-05-26T22:50:25.526760+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:25.526760+0000","last_change":"2024-05-26T22:50:25.526760+0000","last_active":"2024-05-26T22:50:25.526760+0000","last_peered":"2024-05-26T22:50:25.526760+0000","last_clean":"2024-05-26T22:50:25.526760+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:25.526760+0000","last_undegraded":"2024-05-26T22:50:25.526760+0000","last_fullsized":"2024-05-26T22:50:25.526760+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:25.526760+0000","last_change":"2024-05-26T22:50:25.526760+0000","last_active":"2024-05-26T22:50:25.526760+0000","last_peered":"2024-05-26T22:50:25.526760+0000","last_clean":"2024-05-26T22:50:25.526760+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:25.526760+0000","last_undegraded":"2024-05-26T22:50:25.526760+0000","last_fullsized":"2024-05-26T22:50:25.526760+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:25.526760+0000","last_change":"2024-05-26T22:50:25.526760+0000","last_active":"2024-05-26T22:50:25.526760+0000","last_peered":"2024-05-26T22:50:25.526760+0000","last_clean":"2024-05-26T22:50:25.526760+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:25.526760+0000","last_undegraded":"2024-05-26T22:50:25.526760+0000","last_fullsized":"2024-05-26T22:50:25.526760+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:25.526760+0000","last_change":"2024-05-26T22:50:25.526760+0000","last_active":"2024-05-26T22:50:25.526760+0000","last_peered":"2024-05-26T22:50:25.526760+0000","last_clean":"2024-05-26T22:50:25.526760+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:25.526760+0000","last_undegraded":"2024-05-26T22:50:25.526760+0000","last_fullsized":"2024-05-26T22:50:25.526760+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:25.526760+0000","last_change":"2024-05-26T22:50:25.526760+0000","last_active":"2024-05-26T22:50:25.526760+0000","last_peered":"2024-05-26T22:50:25.526760+0000","last_clean":"2024-05-26T22:50:25.526760+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:25.526760+0000","last_undegraded":"2024-05-26T22:50:25.526760+0000","last_fullsized":"2024-05-26T22:50:25.526760+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:25.526760+0000","last_change":"2024-05-26T22:50:25.526760+0000","last_active":"2024-05-26T22:50:25.526760+0000","last_peered":"2024-05-26T22:50:25.526760+0000","last_clean":"2024-05-26T22:50:25.526760+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:25.526760+0000","last_undegraded":"2024-05-26T22:50:25.526760+0000","last_fullsized":"2024-05-26T22:50:25.526760+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:25.526760+0000","last_change":"2024-05-26T22:50:25.526760+0000","last_active":"2024-05-26T22:50:25.526760+0000","last_peered":"2024-05-26T22:50:25.526760+0000","last_clean":"2024-05-26T22:50:25.526760+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:25.526760+0000","last_undegraded":"2024-05-26T22:50:25.526760+0000","last_fullsized":"2024-05-26T22:50:25.526760+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:25.526760+0000","last_change":"2024-05-26T22:50:25.526760+0000","last_active":"2024-05-26T22:50:25.526760+0000","last_peered":"2024-05-26T22:50:25.526760+0000","last_clean":"2024-05-26T22:50:25.526760+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:25.526760+0000","last_undegraded":"2024-05-26T22:50:25.526760+0000","last_fullsized":"2024-05-26T22:50:25.526760+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:25.526760+0000","last_change":"2024-05-26T22:50:25.526760+0000","last_active":"2024-05-26T22:50:25.526760+0000","last_peered":"2024-05-26T22:50:25.526760+0000","last_clean":"2024-05-26T22:50:25.526760+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:25.526760+0000","last_undegraded":"2024-05-26T22:50:25.526760+0000","last_fullsized":"2024-05-26T22:50:25.526760+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:25.526760+0000","last_change":"2024-05-26T22:50:25.526760+0000","last_active":"2024-05-26T22:50:25.526760+0000","last_peered":"2024-05-26T22:50:25.526760+0000","last_clean":"2024-05-26T22:50:25.526760+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:25.526760+0000","last_undegraded":"2024-05-26T22:50:25.526760+0000","last_fullsized":"2024-05-26T22:50:25.526760+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:25.526760+0000","last_change":"2024-05-26T22:50:25.526760+0000","last_active":"2024-05-26T22:50:25.526760+0000","last_peered":"2024-05-26T22:50:25.526760+0000","last_clean":"2024-05-26T22:50:25.526760+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:25.526760+0000","last_undegraded":"2024-05-26T22:50:25.526760+0000","last_fullsized":"2024-05-26T22:50:25.526760+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:25.526760+0000","last_change":"2024-05-26T22:50:25.526760+0000","last_active":"2024-05-26T22:50:25.526760+0000","last_peered":"2024-05-26T22:50:25.526760+0000","last_clean":"2024-05-26T22:50:25.526760+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:25.526760+0000","last_undegraded":"2024-05-26T22:50:25.526760+0000","last_fullsized":"2024-05-26T22:50:25.526760+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:25.526760+0000","last_change":"2024-05-26T22:50:25.526760+0000","last_active":"2024-05-26T22:50:25.526760+0000","last_peered":"2024-05-26T22:50:25.526760+0000","last_clean":"2024-05-26T22:50:25.526760+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:25.526760+0000","last_undegraded":"2024-05-26T22:50:25.526760+0000","last_fullsized":"2024-05-26T22:50:25.526760+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:25.526760+0000","last_change":"2024-05-26T22:50:25.526760+0000","last_active":"2024-05-26T22:50:25.526760+0000","last_peered":"2024-05-26T22:50:25.526760+0000","last_clean":"2024-05-26T22:50:25.526760+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:25.526760+0000","last_undegraded":"2024-05-26T22:50:25.526760+0000","last_fullsized":"2024-05-26T22:50:25.526760+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":207,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-26T22:50:24.830482+0000","last_change":"2024-05-26T22:48:12.068917+0000","last_active":"2024-05-26T22:50:24.830482+0000","last_peered":"2024-05-26T22:50:24.830482+0000","last_clean":"2024-05-26T22:50:24.830482+0000","last_became_active":"2024-05-26T22:48:12.065934+0000","last_became_peered":"2024-05-26T22:48:12.065934+0000","last_unstale":"2024-05-26T22:50:24.830482+0000","last_undegraded":"2024-05-26T22:50:24.830482+0000","last_fullsized":"2024-05-26T22:50:24.830482+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:10.628122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:10.628122+0000","last_clean_scrub_stamp":"2024-05-26T22:48:10.628122+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:14:58.895911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574878,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32216,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339624,"statfs":{"total":96636764160,"available":96603774976,"internally_reserved":0,"allocated":1060864,"data_stored":776009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574878,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27488,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344352,"statfs":{"total":96636764160,"available":96608616448,"internally_reserved":0,"allocated":466944,"data_stored":185641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574879,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27480,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344360,"statfs":{"total":96636764160,"available":96608624640,"internally_reserved":0,"allocated":466944,"data_stored":185641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574878,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27476,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344364,"statfs":{"total":96636764160,"available":96608628736,"internally_reserved":0,"allocated":466944,"data_stored":185641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574878,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27476,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344364,"statfs":{"total":96636764160,"available":96608628736,"internally_reserved":0,"allocated":466944,"data_stored":185641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574878,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27488,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344352,"statfs":{"total":96636764160,"available":96608616448,"internally_reserved":0,"allocated":466944,"data_stored":185641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574878,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27480,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344360,"statfs":{"total":96636764160,"available":96608624640,"internally_reserved":0,"allocated":466944,"data_stored":185641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574878,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32216,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339624,"statfs":{"total":96636764160,"available":96603774976,"internally_reserved":0,"allocated":1060864,"data_stored":776009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-26T22:50:31.969 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-26T22:50:32.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.206+0000 7f1b40e00640 1 Processor -- start 2024-05-26T22:50:32.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.206+0000 7f1b40e00640 1 -- start start 2024-05-26T22:50:32.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.207+0000 7f1b40e00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f1b3c0cedf0 0x7f1b3c071b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:32.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.208+0000 7f1b40e00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b3c0722a0 con 0x7f1b3c144340 2024-05-26T22:50:32.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.208+0000 7f1b40e00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b3c072580 con 0x7f1b3c06ebc0 2024-05-26T22:50:32.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.208+0000 7f1b40e00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f1b3c072eb0 con 0x7f1b3c0cedf0 2024-05-26T22:50:32.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.208+0000 7f1b3b400640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f1b3c144340 0x7f1b3c06e4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:38484/0 (socket says 172.21.15.131:38484) 2024-05-26T22:50:32.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.208+0000 7f1b3b400640 1 -- 172.21.15.131:0/1009151288 learned_addr learned my addr 172.21.15.131:0/1009151288 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:50:32.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.208+0000 7f1b3be00640 1 --2- 172.21.15.131:0/1009151288 >> v2:172.21.15.187:3300/0 conn(0x7f1b3c0cedf0 0x7f1b3c071b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:32.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.208+0000 7f1b3aa00640 1 -- 172.21.15.131:0/1009151288 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3556741651 0 0) 0x7f1b3c0722a0 con 0x7f1b3c144340 2024-05-26T22:50:32.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.208+0000 7f1b3aa00640 1 -- 172.21.15.131:0/1009151288 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1b20003520 con 0x7f1b3c144340 2024-05-26T22:50:32.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.209+0000 7f1b3aa00640 1 -- 172.21.15.131:0/1009151288 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2576663691 0 0) 0x7f1b20003520 con 0x7f1b3c144340 2024-05-26T22:50:32.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.209+0000 7f1b3aa00640 1 -- 172.21.15.131:0/1009151288 >> v2:172.21.15.187:3300/0 conn(0x7f1b3c0cedf0 msgr2=0x7f1b3c071b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:32.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.209+0000 7f1b3aa00640 1 --2- 172.21.15.131:0/1009151288 >> v2:172.21.15.187:3300/0 conn(0x7f1b3c0cedf0 0x7f1b3c071b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:32.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.209+0000 7f1b3aa00640 1 -- 172.21.15.131:0/1009151288 >> v1:172.21.15.187:6789/0 conn(0x7f1b3c06ebc0 legacy=0x7f1b3c0b3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:32.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.209+0000 7f1b3aa00640 1 -- 172.21.15.131:0/1009151288 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b3c072860 con 0x7f1b3c144340 2024-05-26T22:50:32.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.209+0000 7f1b3aa00640 1 -- 172.21.15.131:0/1009151288 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7f1b2c001690 con 0x7f1b3c144340 2024-05-26T22:50:32.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.209+0000 7f1b3aa00640 1 -- 172.21.15.131:0/1009151288 >> v1:172.21.15.131:6789/0 conn(0x7f1b3c144340 legacy=0x7f1b3c06e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:32.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.209+0000 7f1b3aa00640 1 --2- 172.21.15.131:0/1009151288 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1b200039b0 0x7f1b20003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:32.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.209+0000 7f1b3aa00640 1 --2- 172.21.15.131:0/1009151288 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1b20004800 0x7f1b20004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:32.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.209+0000 7f1b3aa00640 1 --2- 172.21.15.131:0/1009151288 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1b200051c0 0x7f1b2000dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:32.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.209+0000 7f1b3aa00640 1 -- 172.21.15.131:0/1009151288 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f1b2000e2f0 con 0x7f1b20004800 2024-05-26T22:50:32.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.209+0000 7f1b3aa00640 1 -- 172.21.15.131:0/1009151288 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1b2000e4a0 con 0x7f1b200039b0 2024-05-26T22:50:32.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.209+0000 7f1b3aa00640 1 -- 172.21.15.131:0/1009151288 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1b2000e6a0 con 0x7f1b200051c0 2024-05-26T22:50:32.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.209+0000 7f1b33e00640 1 --2- 172.21.15.131:0/1009151288 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1b20004800 0x7f1b20004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:32.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.210+0000 7f1b3b400640 1 --2- 172.21.15.131:0/1009151288 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1b200039b0 0x7f1b20003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:32.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.210+0000 7f1b3be00640 1 --2- 172.21.15.131:0/1009151288 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1b200051c0 0x7f1b2000dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:32.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.210+0000 7f1b33e00640 1 -- 172.21.15.131:0/1009151288 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1b200051c0 msgr2=0x7f1b2000dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:32.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.210+0000 7f1b33e00640 1 --2- 172.21.15.131:0/1009151288 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1b200051c0 0x7f1b2000dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:32.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.210+0000 7f1b33e00640 1 -- 172.21.15.131:0/1009151288 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1b200039b0 msgr2=0x7f1b20003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:32.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.210+0000 7f1b33e00640 1 --2- 172.21.15.131:0/1009151288 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1b200039b0 0x7f1b20003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:32.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.210+0000 7f1b33e00640 1 -- 172.21.15.131:0/1009151288 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b20003520 con 0x7f1b20004800 2024-05-26T22:50:32.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.210+0000 7f1b3be00640 1 -- 172.21.15.131:0/1009151288 reap_dead start 2024-05-26T22:50:32.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.211+0000 7f1b33e00640 1 --2- 172.21.15.131:0/1009151288 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1b20004800 0x7f1b20004c80 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f1b24001750 tx=0x7f1b20005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1db1aed981d6377b server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:32.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.211+0000 7f1b3aa00640 1 -- 172.21.15.131:0/1009151288 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b24001370 con 0x7f1b20004800 2024-05-26T22:50:32.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.211+0000 7f1b3aa00640 1 -- 172.21.15.131:0/1009151288 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b240052d0 con 0x7f1b20004800 2024-05-26T22:50:32.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.211+0000 7f1b3aa00640 1 -- 172.21.15.131:0/1009151288 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b2402abc0 con 0x7f1b20004800 2024-05-26T22:50:32.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.212+0000 7f1b40e00640 1 -- 172.21.15.131:0/1009151288 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1b20004800 msgr2=0x7f1b20004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:32.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.212+0000 7f1b40e00640 1 --2- 172.21.15.131:0/1009151288 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1b20004800 0x7f1b20004c80 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f1b24001750 tx=0x7f1b20005830 comp rx=0 tx=0).stop 2024-05-26T22:50:32.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.212+0000 7f1b40e00640 1 -- 172.21.15.131:0/1009151288 shutdown_connections 2024-05-26T22:50:32.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.212+0000 7f1b40e00640 1 --2- 172.21.15.131:0/1009151288 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1b20004800 0x7f1b20004c80 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:32.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.212+0000 7f1b40e00640 1 -- 172.21.15.131:0/1009151288 >> 172.21.15.131:0/1009151288 conn(0x7f1b3c0cff40 msgr2=0x7f1b3c0d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:32.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.212+0000 7f1b40e00640 1 -- 172.21.15.131:0/1009151288 shutdown_connections 2024-05-26T22:50:32.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.212+0000 7f1b40e00640 1 -- 172.21.15.131:0/1009151288 wait complete. 2024-05-26T22:50:32.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.213+0000 7f1b40e00640 1 Processor -- start 2024-05-26T22:50:32.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.214+0000 7f1b40e00640 1 -- start start 2024-05-26T22:50:32.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.214+0000 7f1b40e00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1b20004800 0x7f1b3c15aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:32.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.215+0000 7f1b40e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1b3c13ee90 0x7f1b3c0c4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:32.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.215+0000 7f1b40e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1b3c06d3a0 0x7f1b3c06d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:32.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.215+0000 7f1b40e00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f1b3c071b70 con 0x7f1b3c06d3a0 2024-05-26T22:50:32.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.215+0000 7f1b40e00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1b3c0cf200 con 0x7f1b20004800 2024-05-26T22:50:32.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.215+0000 7f1b40e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1b3c071cf0 con 0x7f1b3c13ee90 2024-05-26T22:50:32.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.215+0000 7f1b3be00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1b3c06d3a0 0x7f1b3c06d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:32.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.215+0000 7f1b3be00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1b3c06d3a0 0x7f1b3c06d820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50088/0 (socket says 172.21.15.131:50088) 2024-05-26T22:50:32.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.215+0000 7f1b3be00640 1 -- 172.21.15.131:0/2866345553 learned_addr learned my addr 172.21.15.131:0/2866345553 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:32.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.215+0000 7f1b33e00640 1 --2- 172.21.15.131:0/2866345553 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1b3c13ee90 0x7f1b3c0c4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:32.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.215+0000 7f1b3b400640 1 --2- 172.21.15.131:0/2866345553 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1b20004800 0x7f1b3c15aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:32.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.215+0000 7f1b3be00640 1 -- 172.21.15.131:0/2866345553 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1b3c13ee90 msgr2=0x7f1b3c0c4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:32.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.215+0000 7f1b3be00640 1 --2- 172.21.15.131:0/2866345553 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1b3c13ee90 0x7f1b3c0c4d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:32.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.215+0000 7f1b3be00640 1 -- 172.21.15.131:0/2866345553 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1b20004800 msgr2=0x7f1b3c15aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:32.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.215+0000 7f1b3be00640 1 --2- 172.21.15.131:0/2866345553 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1b20004800 0x7f1b3c15aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:32.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.215+0000 7f1b3be00640 1 -- 172.21.15.131:0/2866345553 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b3c0b0470 con 0x7f1b3c06d3a0 2024-05-26T22:50:32.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.215+0000 7f1b33e00640 1 --2- 172.21.15.131:0/2866345553 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1b3c13ee90 0x7f1b3c0c4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:50:32.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.216+0000 7f1b3be00640 1 --2- 172.21.15.131:0/2866345553 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1b3c06d3a0 0x7f1b3c06d820 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f1b34009890 tx=0x7f1b34009d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:32.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.216+0000 7f1b33400640 1 -- 172.21.15.131:0/2866345553 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b34019070 con 0x7f1b3c06d3a0 2024-05-26T22:50:32.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.216+0000 7f1b33400640 1 -- 172.21.15.131:0/2866345553 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b34007d40 con 0x7f1b3c06d3a0 2024-05-26T22:50:32.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.216+0000 7f1b40e00640 1 -- 172.21.15.131:0/2866345553 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b3c0b0750 con 0x7f1b3c06d3a0 2024-05-26T22:50:32.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.216+0000 7f1b33400640 1 -- 172.21.15.131:0/2866345553 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b34014640 con 0x7f1b3c06d3a0 2024-05-26T22:50:32.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.216+0000 7f1b40e00640 1 -- 172.21.15.131:0/2866345553 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b3c0b0d10 con 0x7f1b3c06d3a0 2024-05-26T22:50:32.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.217+0000 7f1b40e00640 1 -- 172.21.15.131:0/2866345553 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b3c0694e0 con 0x7f1b3c06d3a0 2024-05-26T22:50:32.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.218+0000 7f1b33400640 1 -- 172.21.15.131:0/2866345553 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1b3401c070 con 0x7f1b3c06d3a0 2024-05-26T22:50:32.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.219+0000 7f1b33400640 1 --2- 172.21.15.131:0/2866345553 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f1b1806be30 0x7f1b1806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:32.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.219+0000 7f1b33400640 1 -- 172.21.15.131:0/2866345553 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f1b3408aa00 con 0x7f1b3c06d3a0 2024-05-26T22:50:32.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.219+0000 7f1b3b400640 1 --2- 172.21.15.131:0/2866345553 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f1b1806be30 0x7f1b1806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:32.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.220+0000 7f1b3b400640 1 --2- 172.21.15.131:0/2866345553 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f1b1806be30 0x7f1b1806e2f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c001050 tx=0x7f1b2c001150 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:32.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.224+0000 7f1b33400640 1 -- 172.21.15.131:0/2866345553 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f1b34059310 con 0x7f1b3c06d3a0 2024-05-26T22:50:32.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.417+0000 7f1b40e00640 1 -- 172.21.15.131:0/2866345553 --> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1b3c0ce720 con 0x7f1b1806be30 2024-05-26T22:50:32.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.426+0000 7f1b33400640 1 -- 172.21.15.131:0/2866345553 <== mgr.4104 v2:172.21.15.131:6832/3484832947 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+237202 (secure 0 0 0) 0x7f1b3c0ce720 con 0x7f1b1806be30 2024-05-26T22:50:32.429 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:50:32.430 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-05-26T22:50:32.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.432+0000 7f1b40e00640 1 -- 172.21.15.131:0/2866345553 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f1b1806be30 msgr2=0x7f1b1806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:32.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.432+0000 7f1b40e00640 1 --2- 172.21.15.131:0/2866345553 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f1b1806be30 0x7f1b1806e2f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c001050 tx=0x7f1b2c001150 comp rx=0 tx=0).stop 2024-05-26T22:50:32.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.432+0000 7f1b40e00640 1 -- 172.21.15.131:0/2866345553 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1b3c06d3a0 msgr2=0x7f1b3c06d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:32.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.432+0000 7f1b40e00640 1 --2- 172.21.15.131:0/2866345553 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1b3c06d3a0 0x7f1b3c06d820 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f1b34009890 tx=0x7f1b34009d50 comp rx=0 tx=0).stop 2024-05-26T22:50:32.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.432+0000 7f1b40e00640 1 -- 172.21.15.131:0/2866345553 shutdown_connections 2024-05-26T22:50:32.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.432+0000 7f1b40e00640 1 --2- 172.21.15.131:0/2866345553 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1b3c13ee90 0x7f1b3c0c4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:32.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.432+0000 7f1b40e00640 1 --2- 172.21.15.131:0/2866345553 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f1b1806be30 0x7f1b1806e2f0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:32.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.432+0000 7f1b40e00640 1 --2- 172.21.15.131:0/2866345553 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1b3c06d3a0 0x7f1b3c06d820 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:32.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.432+0000 7f1b40e00640 1 --2- 172.21.15.131:0/2866345553 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1b20004800 0x7f1b3c15aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:32.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.432+0000 7f1b40e00640 1 -- 172.21.15.131:0/2866345553 >> 172.21.15.131:0/2866345553 conn(0x7f1b3c0cff40 msgr2=0x7f1b3c142460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:32.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.432+0000 7f1b40e00640 1 -- 172.21.15.131:0/2866345553 shutdown_connections 2024-05-26T22:50:32.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.432+0000 7f1b40e00640 1 -- 172.21.15.131:0/2866345553 wait complete. 2024-05-26T22:50:32.452 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":107,"stamp":"2024-05-26T22:50:30.424662+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":229320,"kb_used_data":4808,"kb_used_omap":12,"kb_used_meta":224371,"kb_avail":754745400,"statfs":{"total":773094113280,"available":772859289600,"internally_reserved":0,"allocated":4923392,"data_stored":2665864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":229756465},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.882072"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:25.526760+0000","last_change":"2024-05-26T22:50:25.526760+0000","last_active":"2024-05-26T22:50:25.526760+0000","last_peered":"2024-05-26T22:50:25.526760+0000","last_clean":"2024-05-26T22:50:25.526760+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:25.526760+0000","last_undegraded":"2024-05-26T22:50:25.526760+0000","last_fullsized":"2024-05-26T22:50:25.526760+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:25.526760+0000","last_change":"2024-05-26T22:50:25.526760+0000","last_active":"2024-05-26T22:50:25.526760+0000","last_peered":"2024-05-26T22:50:25.526760+0000","last_clean":"2024-05-26T22:50:25.526760+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:25.526760+0000","last_undegraded":"2024-05-26T22:50:25.526760+0000","last_fullsized":"2024-05-26T22:50:25.526760+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:25.526760+0000","last_change":"2024-05-26T22:50:25.526760+0000","last_active":"2024-05-26T22:50:25.526760+0000","last_peered":"2024-05-26T22:50:25.526760+0000","last_clean":"2024-05-26T22:50:25.526760+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:25.526760+0000","last_undegraded":"2024-05-26T22:50:25.526760+0000","last_fullsized":"2024-05-26T22:50:25.526760+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:25.526760+0000","last_change":"2024-05-26T22:50:25.526760+0000","last_active":"2024-05-26T22:50:25.526760+0000","last_peered":"2024-05-26T22:50:25.526760+0000","last_clean":"2024-05-26T22:50:25.526760+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:25.526760+0000","last_undegraded":"2024-05-26T22:50:25.526760+0000","last_fullsized":"2024-05-26T22:50:25.526760+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:25.526760+0000","last_change":"2024-05-26T22:50:25.526760+0000","last_active":"2024-05-26T22:50:25.526760+0000","last_peered":"2024-05-26T22:50:25.526760+0000","last_clean":"2024-05-26T22:50:25.526760+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:25.526760+0000","last_undegraded":"2024-05-26T22:50:25.526760+0000","last_fullsized":"2024-05-26T22:50:25.526760+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:25.526760+0000","last_change":"2024-05-26T22:50:25.526760+0000","last_active":"2024-05-26T22:50:25.526760+0000","last_peered":"2024-05-26T22:50:25.526760+0000","last_clean":"2024-05-26T22:50:25.526760+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:25.526760+0000","last_undegraded":"2024-05-26T22:50:25.526760+0000","last_fullsized":"2024-05-26T22:50:25.526760+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:25.526760+0000","last_change":"2024-05-26T22:50:25.526760+0000","last_active":"2024-05-26T22:50:25.526760+0000","last_peered":"2024-05-26T22:50:25.526760+0000","last_clean":"2024-05-26T22:50:25.526760+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:25.526760+0000","last_undegraded":"2024-05-26T22:50:25.526760+0000","last_fullsized":"2024-05-26T22:50:25.526760+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:25.526760+0000","last_change":"2024-05-26T22:50:25.526760+0000","last_active":"2024-05-26T22:50:25.526760+0000","last_peered":"2024-05-26T22:50:25.526760+0000","last_clean":"2024-05-26T22:50:25.526760+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:25.526760+0000","last_undegraded":"2024-05-26T22:50:25.526760+0000","last_fullsized":"2024-05-26T22:50:25.526760+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:25.526760+0000","last_change":"2024-05-26T22:50:25.526760+0000","last_active":"2024-05-26T22:50:25.526760+0000","last_peered":"2024-05-26T22:50:25.526760+0000","last_clean":"2024-05-26T22:50:25.526760+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:25.526760+0000","last_undegraded":"2024-05-26T22:50:25.526760+0000","last_fullsized":"2024-05-26T22:50:25.526760+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:25.526760+0000","last_change":"2024-05-26T22:50:25.526760+0000","last_active":"2024-05-26T22:50:25.526760+0000","last_peered":"2024-05-26T22:50:25.526760+0000","last_clean":"2024-05-26T22:50:25.526760+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:25.526760+0000","last_undegraded":"2024-05-26T22:50:25.526760+0000","last_fullsized":"2024-05-26T22:50:25.526760+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:25.526760+0000","last_change":"2024-05-26T22:50:25.526760+0000","last_active":"2024-05-26T22:50:25.526760+0000","last_peered":"2024-05-26T22:50:25.526760+0000","last_clean":"2024-05-26T22:50:25.526760+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:25.526760+0000","last_undegraded":"2024-05-26T22:50:25.526760+0000","last_fullsized":"2024-05-26T22:50:25.526760+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:25.526760+0000","last_change":"2024-05-26T22:50:25.526760+0000","last_active":"2024-05-26T22:50:25.526760+0000","last_peered":"2024-05-26T22:50:25.526760+0000","last_clean":"2024-05-26T22:50:25.526760+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:25.526760+0000","last_undegraded":"2024-05-26T22:50:25.526760+0000","last_fullsized":"2024-05-26T22:50:25.526760+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:25.526760+0000","last_change":"2024-05-26T22:50:25.526760+0000","last_active":"2024-05-26T22:50:25.526760+0000","last_peered":"2024-05-26T22:50:25.526760+0000","last_clean":"2024-05-26T22:50:25.526760+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:25.526760+0000","last_undegraded":"2024-05-26T22:50:25.526760+0000","last_fullsized":"2024-05-26T22:50:25.526760+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:25.526760+0000","last_change":"2024-05-26T22:50:25.526760+0000","last_active":"2024-05-26T22:50:25.526760+0000","last_peered":"2024-05-26T22:50:25.526760+0000","last_clean":"2024-05-26T22:50:25.526760+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:25.526760+0000","last_undegraded":"2024-05-26T22:50:25.526760+0000","last_fullsized":"2024-05-26T22:50:25.526760+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:25.526760+0000","last_change":"2024-05-26T22:50:25.526760+0000","last_active":"2024-05-26T22:50:25.526760+0000","last_peered":"2024-05-26T22:50:25.526760+0000","last_clean":"2024-05-26T22:50:25.526760+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:25.526760+0000","last_undegraded":"2024-05-26T22:50:25.526760+0000","last_fullsized":"2024-05-26T22:50:25.526760+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:25.526760+0000","last_change":"2024-05-26T22:50:25.526760+0000","last_active":"2024-05-26T22:50:25.526760+0000","last_peered":"2024-05-26T22:50:25.526760+0000","last_clean":"2024-05-26T22:50:25.526760+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:25.526760+0000","last_undegraded":"2024-05-26T22:50:25.526760+0000","last_fullsized":"2024-05-26T22:50:25.526760+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:25.526760+0000","last_change":"2024-05-26T22:50:25.526760+0000","last_active":"2024-05-26T22:50:25.526760+0000","last_peered":"2024-05-26T22:50:25.526760+0000","last_clean":"2024-05-26T22:50:25.526760+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:25.526760+0000","last_undegraded":"2024-05-26T22:50:25.526760+0000","last_fullsized":"2024-05-26T22:50:25.526760+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:25.526760+0000","last_change":"2024-05-26T22:50:25.526760+0000","last_active":"2024-05-26T22:50:25.526760+0000","last_peered":"2024-05-26T22:50:25.526760+0000","last_clean":"2024-05-26T22:50:25.526760+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:25.526760+0000","last_undegraded":"2024-05-26T22:50:25.526760+0000","last_fullsized":"2024-05-26T22:50:25.526760+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:25.526760+0000","last_change":"2024-05-26T22:50:25.526760+0000","last_active":"2024-05-26T22:50:25.526760+0000","last_peered":"2024-05-26T22:50:25.526760+0000","last_clean":"2024-05-26T22:50:25.526760+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:25.526760+0000","last_undegraded":"2024-05-26T22:50:25.526760+0000","last_fullsized":"2024-05-26T22:50:25.526760+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:25.526760+0000","last_change":"2024-05-26T22:50:25.526760+0000","last_active":"2024-05-26T22:50:25.526760+0000","last_peered":"2024-05-26T22:50:25.526760+0000","last_clean":"2024-05-26T22:50:25.526760+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:25.526760+0000","last_undegraded":"2024-05-26T22:50:25.526760+0000","last_fullsized":"2024-05-26T22:50:25.526760+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:25.526760+0000","last_change":"2024-05-26T22:50:25.526760+0000","last_active":"2024-05-26T22:50:25.526760+0000","last_peered":"2024-05-26T22:50:25.526760+0000","last_clean":"2024-05-26T22:50:25.526760+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:25.526760+0000","last_undegraded":"2024-05-26T22:50:25.526760+0000","last_fullsized":"2024-05-26T22:50:25.526760+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:25.526760+0000","last_change":"2024-05-26T22:50:25.526760+0000","last_active":"2024-05-26T22:50:25.526760+0000","last_peered":"2024-05-26T22:50:25.526760+0000","last_clean":"2024-05-26T22:50:25.526760+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:25.526760+0000","last_undegraded":"2024-05-26T22:50:25.526760+0000","last_fullsized":"2024-05-26T22:50:25.526760+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:25.526760+0000","last_change":"2024-05-26T22:50:25.526760+0000","last_active":"2024-05-26T22:50:25.526760+0000","last_peered":"2024-05-26T22:50:25.526760+0000","last_clean":"2024-05-26T22:50:25.526760+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:25.526760+0000","last_undegraded":"2024-05-26T22:50:25.526760+0000","last_fullsized":"2024-05-26T22:50:25.526760+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:25.526760+0000","last_change":"2024-05-26T22:50:25.526760+0000","last_active":"2024-05-26T22:50:25.526760+0000","last_peered":"2024-05-26T22:50:25.526760+0000","last_clean":"2024-05-26T22:50:25.526760+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:25.526760+0000","last_undegraded":"2024-05-26T22:50:25.526760+0000","last_fullsized":"2024-05-26T22:50:25.526760+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:25.526760+0000","last_change":"2024-05-26T22:50:25.526760+0000","last_active":"2024-05-26T22:50:25.526760+0000","last_peered":"2024-05-26T22:50:25.526760+0000","last_clean":"2024-05-26T22:50:25.526760+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:25.526760+0000","last_undegraded":"2024-05-26T22:50:25.526760+0000","last_fullsized":"2024-05-26T22:50:25.526760+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:25.526760+0000","last_change":"2024-05-26T22:50:25.526760+0000","last_active":"2024-05-26T22:50:25.526760+0000","last_peered":"2024-05-26T22:50:25.526760+0000","last_clean":"2024-05-26T22:50:25.526760+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:25.526760+0000","last_undegraded":"2024-05-26T22:50:25.526760+0000","last_fullsized":"2024-05-26T22:50:25.526760+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:25.526760+0000","last_change":"2024-05-26T22:50:25.526760+0000","last_active":"2024-05-26T22:50:25.526760+0000","last_peered":"2024-05-26T22:50:25.526760+0000","last_clean":"2024-05-26T22:50:25.526760+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:25.526760+0000","last_undegraded":"2024-05-26T22:50:25.526760+0000","last_fullsized":"2024-05-26T22:50:25.526760+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:25.526760+0000","last_change":"2024-05-26T22:50:25.526760+0000","last_active":"2024-05-26T22:50:25.526760+0000","last_peered":"2024-05-26T22:50:25.526760+0000","last_clean":"2024-05-26T22:50:25.526760+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:25.526760+0000","last_undegraded":"2024-05-26T22:50:25.526760+0000","last_fullsized":"2024-05-26T22:50:25.526760+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:25.526760+0000","last_change":"2024-05-26T22:50:25.526760+0000","last_active":"2024-05-26T22:50:25.526760+0000","last_peered":"2024-05-26T22:50:25.526760+0000","last_clean":"2024-05-26T22:50:25.526760+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:25.526760+0000","last_undegraded":"2024-05-26T22:50:25.526760+0000","last_fullsized":"2024-05-26T22:50:25.526760+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:25.526760+0000","last_change":"2024-05-26T22:50:25.526760+0000","last_active":"2024-05-26T22:50:25.526760+0000","last_peered":"2024-05-26T22:50:25.526760+0000","last_clean":"2024-05-26T22:50:25.526760+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:25.526760+0000","last_undegraded":"2024-05-26T22:50:25.526760+0000","last_fullsized":"2024-05-26T22:50:25.526760+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:25.526760+0000","last_change":"2024-05-26T22:50:25.526760+0000","last_active":"2024-05-26T22:50:25.526760+0000","last_peered":"2024-05-26T22:50:25.526760+0000","last_clean":"2024-05-26T22:50:25.526760+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:25.526760+0000","last_undegraded":"2024-05-26T22:50:25.526760+0000","last_fullsized":"2024-05-26T22:50:25.526760+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:26.540322+0000","last_change":"2024-05-26T22:50:26.540322+0000","last_active":"2024-05-26T22:50:26.540322+0000","last_peered":"2024-05-26T22:50:26.540322+0000","last_clean":"2024-05-26T22:50:26.540322+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:26.540322+0000","last_undegraded":"2024-05-26T22:50:26.540322+0000","last_fullsized":"2024-05-26T22:50:26.540322+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-26T22:50:25.526760+0000","last_change":"2024-05-26T22:50:25.526760+0000","last_active":"2024-05-26T22:50:25.526760+0000","last_peered":"2024-05-26T22:50:25.526760+0000","last_clean":"2024-05-26T22:50:25.526760+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-26T22:50:25.526760+0000","last_undegraded":"2024-05-26T22:50:25.526760+0000","last_fullsized":"2024-05-26T22:50:25.526760+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":207,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-26T22:50:24.830482+0000","last_change":"2024-05-26T22:48:12.068917+0000","last_active":"2024-05-26T22:50:24.830482+0000","last_peered":"2024-05-26T22:50:24.830482+0000","last_clean":"2024-05-26T22:50:24.830482+0000","last_became_active":"2024-05-26T22:48:12.065934+0000","last_became_peered":"2024-05-26T22:48:12.065934+0000","last_unstale":"2024-05-26T22:50:24.830482+0000","last_undegraded":"2024-05-26T22:50:24.830482+0000","last_fullsized":"2024-05-26T22:50:24.830482+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:10.628122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:10.628122+0000","last_clean_scrub_stamp":"2024-05-26T22:48:10.628122+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:14:58.895911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574878,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32216,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339624,"statfs":{"total":96636764160,"available":96603774976,"internally_reserved":0,"allocated":1060864,"data_stored":776009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574878,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27488,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344352,"statfs":{"total":96636764160,"available":96608616448,"internally_reserved":0,"allocated":466944,"data_stored":185641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574879,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27480,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344360,"statfs":{"total":96636764160,"available":96608624640,"internally_reserved":0,"allocated":466944,"data_stored":185641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574878,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27476,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344364,"statfs":{"total":96636764160,"available":96608628736,"internally_reserved":0,"allocated":466944,"data_stored":185641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574878,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27476,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344364,"statfs":{"total":96636764160,"available":96608628736,"internally_reserved":0,"allocated":466944,"data_stored":185641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574878,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27488,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344352,"statfs":{"total":96636764160,"available":96608616448,"internally_reserved":0,"allocated":466944,"data_stored":185641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574878,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27480,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344360,"statfs":{"total":96636764160,"available":96608624640,"internally_reserved":0,"allocated":466944,"data_stored":185641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574878,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32216,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339624,"statfs":{"total":96636764160,"available":96603774976,"internally_reserved":0,"allocated":1060864,"data_stored":776009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-26T22:50:32.455 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-26T22:50:32.512 INFO:tasks.ceph.mds.c.smithi131.stderr:2024-05-26T22:50:32.512+0000 7efef9c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-26T22:50:32.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.703+0000 7fa354200640 1 Processor -- start 2024-05-26T22:50:32.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.703+0000 7fa354200640 1 -- start start 2024-05-26T22:50:32.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.705+0000 7fa354200640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fa34c06ee20 0x7fa34c0b3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:32.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.705+0000 7fa354200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fa34c1569d0 0x7fa34c0af790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:32.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.705+0000 7fa354200640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa34c15cb20 con 0x7fa34c0ad250 2024-05-26T22:50:32.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.705+0000 7fa354200640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fa34c15ce00 con 0x7fa34c1569d0 2024-05-26T22:50:32.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.705+0000 7fa354200640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fa34c15d0e0 con 0x7fa34c06ee20 2024-05-26T22:50:32.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.705+0000 7fa350e00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fa34c0ad250 0x7fa34c1562c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:38486/0 (socket says 172.21.15.131:38486) 2024-05-26T22:50:32.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.705+0000 7fa350e00640 1 -- 172.21.15.131:0/3279620170 learned_addr learned my addr 172.21.15.131:0/3279620170 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:50:32.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.705+0000 7fa352200640 1 --2- 172.21.15.131:0/3279620170 >> v2:172.21.15.131:3300/0 conn(0x7fa34c1569d0 0x7fa34c0af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:32.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.705+0000 7fa351800640 1 --2- 172.21.15.131:0/3279620170 >> v2:172.21.15.187:3300/0 conn(0x7fa34c06ee20 0x7fa34c0b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:32.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.705+0000 7fa34be00640 1 -- 172.21.15.131:0/3279620170 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2678292750 0 0) 0x7fa34c15cb20 con 0x7fa34c0ad250 2024-05-26T22:50:32.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.706+0000 7fa352200640 1 -- 172.21.15.131:0/3279620170 >> v2:172.21.15.187:3300/0 conn(0x7fa34c06ee20 msgr2=0x7fa34c0b3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:32.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.706+0000 7fa352200640 1 --2- 172.21.15.131:0/3279620170 >> v2:172.21.15.187:3300/0 conn(0x7fa34c06ee20 0x7fa34c0b3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:32.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.706+0000 7fa352200640 1 -- 172.21.15.131:0/3279620170 >> v1:172.21.15.131:6789/0 conn(0x7fa34c0ad250 legacy=0x7fa34c1562c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:32.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.706+0000 7fa352200640 1 -- 172.21.15.131:0/3279620170 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa34c15d3c0 con 0x7fa34c1569d0 2024-05-26T22:50:32.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.707+0000 7fa352200640 1 --2- 172.21.15.131:0/3279620170 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa34c1569d0 0x7fa34c0af790 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fa34c071090 tx=0x7fa34402a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef7c867c72d45d47 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:32.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.707+0000 7fa34be00640 1 -- 172.21.15.131:0/3279620170 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa344007740 con 0x7fa34c1569d0 2024-05-26T22:50:32.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.707+0000 7fa34be00640 1 -- 172.21.15.131:0/3279620170 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3440078e0 con 0x7fa34c1569d0 2024-05-26T22:50:32.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.707+0000 7fa34be00640 1 -- 172.21.15.131:0/3279620170 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa344007cb0 con 0x7fa34c1569d0 2024-05-26T22:50:32.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.707+0000 7fa354200640 1 -- 172.21.15.131:0/3279620170 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa34c1569d0 msgr2=0x7fa34c0af790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:32.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.707+0000 7fa354200640 1 --2- 172.21.15.131:0/3279620170 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa34c1569d0 0x7fa34c0af790 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fa34c071090 tx=0x7fa34402a5f0 comp rx=0 tx=0).stop 2024-05-26T22:50:32.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.708+0000 7fa354200640 1 -- 172.21.15.131:0/3279620170 shutdown_connections 2024-05-26T22:50:32.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.708+0000 7fa354200640 1 --2- 172.21.15.131:0/3279620170 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa34c1569d0 0x7fa34c0af790 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:32.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.708+0000 7fa354200640 1 --2- 172.21.15.131:0/3279620170 >> v2:172.21.15.187:3300/0 conn(0x7fa34c06ee20 0x7fa34c0b3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:32.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.708+0000 7fa354200640 1 -- 172.21.15.131:0/3279620170 >> 172.21.15.131:0/3279620170 conn(0x7fa34c0b1e90 msgr2=0x7fa34c0b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:32.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.708+0000 7fa354200640 1 -- 172.21.15.131:0/3279620170 shutdown_connections 2024-05-26T22:50:32.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.708+0000 7fa354200640 1 -- 172.21.15.131:0/3279620170 wait complete. 2024-05-26T22:50:32.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.709+0000 7fa354200640 1 Processor -- start 2024-05-26T22:50:32.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.710+0000 7fa354200640 1 -- start start 2024-05-26T22:50:32.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.710+0000 7fa354200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa34c06ee20 0x7fa34c09dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:32.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.710+0000 7fa354200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa34c0ad250 0x7fa34c147970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:32.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.710+0000 7fa354200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa34c1569d0 0x7fa34c153f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:32.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.710+0000 7fa354200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa34c15c900 con 0x7fa34c1569d0 2024-05-26T22:50:32.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.710+0000 7fa354200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa34c15c780 con 0x7fa34c0ad250 2024-05-26T22:50:32.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.710+0000 7fa354200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa34c06f200 con 0x7fa34c06ee20 2024-05-26T22:50:32.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.711+0000 7fa352200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa34c1569d0 0x7fa34c153f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:32.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.711+0000 7fa352200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa34c1569d0 0x7fa34c153f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50096/0 (socket says 172.21.15.131:50096) 2024-05-26T22:50:32.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.711+0000 7fa352200640 1 -- 172.21.15.131:0/2593641325 learned_addr learned my addr 172.21.15.131:0/2593641325 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:32.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.711+0000 7fa351800640 1 --2- 172.21.15.131:0/2593641325 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa34c06ee20 0x7fa34c09dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:32.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.711+0000 7fa350e00640 1 --2- 172.21.15.131:0/2593641325 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa34c0ad250 0x7fa34c147970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:32.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.711+0000 7fa352200640 1 -- 172.21.15.131:0/2593641325 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa34c06ee20 msgr2=0x7fa34c09dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:32.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.711+0000 7fa352200640 1 --2- 172.21.15.131:0/2593641325 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa34c06ee20 0x7fa34c09dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:32.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.711+0000 7fa352200640 1 -- 172.21.15.131:0/2593641325 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa34c0ad250 msgr2=0x7fa34c147970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:32.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.711+0000 7fa352200640 1 --2- 172.21.15.131:0/2593641325 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa34c0ad250 0x7fa34c147970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:32.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.711+0000 7fa352200640 1 -- 172.21.15.131:0/2593641325 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa34c09cf90 con 0x7fa34c1569d0 2024-05-26T22:50:32.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.711+0000 7fa351800640 1 --2- 172.21.15.131:0/2593641325 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa34c06ee20 0x7fa34c09dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:50:32.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.712+0000 7fa352200640 1 --2- 172.21.15.131:0/2593641325 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa34c1569d0 0x7fa34c153f10 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fa34402abe0 tx=0x7fa344004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:32.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.712+0000 7fa349600640 1 -- 172.21.15.131:0/2593641325 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa34403f070 con 0x7fa34c1569d0 2024-05-26T22:50:32.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.712+0000 7fa349600640 1 -- 172.21.15.131:0/2593641325 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa344031bc0 con 0x7fa34c1569d0 2024-05-26T22:50:32.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.712+0000 7fa349600640 1 -- 172.21.15.131:0/2593641325 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa344044460 con 0x7fa34c1569d0 2024-05-26T22:50:32.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.712+0000 7fa354200640 1 -- 172.21.15.131:0/2593641325 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa34c09d270 con 0x7fa34c1569d0 2024-05-26T22:50:32.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.712+0000 7fa354200640 1 -- 172.21.15.131:0/2593641325 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa34c09e290 con 0x7fa34c1569d0 2024-05-26T22:50:32.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.713+0000 7fa354200640 1 -- 172.21.15.131:0/2593641325 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa34c0694e0 con 0x7fa34c1569d0 2024-05-26T22:50:32.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.714+0000 7fa349600640 1 -- 172.21.15.131:0/2593641325 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa34403b410 con 0x7fa34c1569d0 2024-05-26T22:50:32.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.715+0000 7fa349600640 1 --2- 172.21.15.131:0/2593641325 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa32c06be30 0x7fa32c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:32.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.715+0000 7fa349600640 1 -- 172.21.15.131:0/2593641325 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7fa3440ac710 con 0x7fa34c1569d0 2024-05-26T22:50:32.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.715+0000 7fa351800640 1 --2- 172.21.15.131:0/2593641325 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa32c06be30 0x7fa32c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:32.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.716+0000 7fa351800640 1 --2- 172.21.15.131:0/2593641325 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa32c06be30 0x7fa32c06e2f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fa340006fd0 tx=0x7fa340008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:32.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.719+0000 7fa349600640 1 -- 172.21.15.131:0/2593641325 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fa34407b0b0 con 0x7fa34c1569d0 2024-05-26T22:50:32.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.914+0000 7fa354200640 1 -- 172.21.15.131:0/2593641325 --> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa34c06c590 con 0x7fa32c06be30 2024-05-26T22:50:32.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.934+0000 7fa349600640 1 -- 172.21.15.131:0/2593641325 <== mgr.4104 v2:172.21.15.131:6832/3484832947 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251959 (secure 0 0 0) 0x7fa34c06c590 con 0x7fa32c06be30 2024-05-26T22:50:32.937 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:50:32.938 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-05-26T22:50:32.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.940+0000 7fa354200640 1 -- 172.21.15.131:0/2593641325 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa32c06be30 msgr2=0x7fa32c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:32.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.940+0000 7fa354200640 1 --2- 172.21.15.131:0/2593641325 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa32c06be30 0x7fa32c06e2f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fa340006fd0 tx=0x7fa340008040 comp rx=0 tx=0).stop 2024-05-26T22:50:32.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.940+0000 7fa354200640 1 -- 172.21.15.131:0/2593641325 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa34c1569d0 msgr2=0x7fa34c153f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:32.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.940+0000 7fa354200640 1 --2- 172.21.15.131:0/2593641325 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa34c1569d0 0x7fa34c153f10 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fa34402abe0 tx=0x7fa344004a40 comp rx=0 tx=0).stop 2024-05-26T22:50:32.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.940+0000 7fa354200640 1 -- 172.21.15.131:0/2593641325 shutdown_connections 2024-05-26T22:50:32.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.940+0000 7fa354200640 1 --2- 172.21.15.131:0/2593641325 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa32c06be30 0x7fa32c06e2f0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:32.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.940+0000 7fa354200640 1 --2- 172.21.15.131:0/2593641325 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa34c1569d0 0x7fa34c153f10 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:32.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.940+0000 7fa354200640 1 --2- 172.21.15.131:0/2593641325 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa34c0ad250 0x7fa34c147970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:32.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.940+0000 7fa354200640 1 --2- 172.21.15.131:0/2593641325 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa34c06ee20 0x7fa34c09dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:32.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.940+0000 7fa354200640 1 -- 172.21.15.131:0/2593641325 >> 172.21.15.131:0/2593641325 conn(0x7fa34c0b1e90 msgr2=0x7fa34c15d8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:32.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.940+0000 7fa354200640 1 -- 172.21.15.131:0/2593641325 shutdown_connections 2024-05-26T22:50:32.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:32.940+0000 7fa354200640 1 -- 172.21.15.131:0/2593641325 wait complete. 2024-05-26T22:50:32.960 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":108,"stamp":"2024-05-26T22:50:32.426341+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":149,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":229756,"kb_used_data":5256,"kb_used_omap":12,"kb_used_meta":224371,"kb_avail":754744964,"statfs":{"total":773094113280,"available":772858843136,"internally_reserved":0,"allocated":5382144,"data_stored":2871956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":229756465},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":42,"apply_latency_ms":42,"commit_latency_ns":42000000,"apply_latency_ns":42000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.003203"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:29.031378+0000","last_change":"2024-05-26T22:50:27.610657+0000","last_active":"2024-05-26T22:50:29.031378+0000","last_peered":"2024-05-26T22:50:29.031378+0000","last_clean":"2024-05-26T22:50:29.031378+0000","last_became_active":"2024-05-26T22:50:27.610367+0000","last_became_peered":"2024-05-26T22:50:27.610367+0000","last_unstale":"2024-05-26T22:50:29.031378+0000","last_undegraded":"2024-05-26T22:50:29.031378+0000","last_fullsized":"2024-05-26T22:50:29.031378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:44:57.492554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:29.030017+0000","last_change":"2024-05-26T22:50:27.621789+0000","last_active":"2024-05-26T22:50:29.030017+0000","last_peered":"2024-05-26T22:50:29.030017+0000","last_clean":"2024-05-26T22:50:29.030017+0000","last_became_active":"2024-05-26T22:50:27.621501+0000","last_became_peered":"2024-05-26T22:50:27.621501+0000","last_unstale":"2024-05-26T22:50:29.030017+0000","last_undegraded":"2024-05-26T22:50:29.030017+0000","last_fullsized":"2024-05-26T22:50:29.030017+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:15:23.061792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.617060+0000","last_change":"2024-05-26T22:50:27.602597+0000","last_active":"2024-05-26T22:50:28.617060+0000","last_peered":"2024-05-26T22:50:28.617060+0000","last_clean":"2024-05-26T22:50:28.617060+0000","last_became_active":"2024-05-26T22:50:27.602270+0000","last_became_peered":"2024-05-26T22:50:27.602270+0000","last_unstale":"2024-05-26T22:50:28.617060+0000","last_undegraded":"2024-05-26T22:50:28.617060+0000","last_fullsized":"2024-05-26T22:50:28.617060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:48:40.238018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.607638+0000","last_change":"2024-05-26T22:50:27.606124+0000","last_active":"2024-05-26T22:50:28.607638+0000","last_peered":"2024-05-26T22:50:28.607638+0000","last_clean":"2024-05-26T22:50:28.607638+0000","last_became_active":"2024-05-26T22:50:27.605799+0000","last_became_peered":"2024-05-26T22:50:27.605799+0000","last_unstale":"2024-05-26T22:50:28.607638+0000","last_undegraded":"2024-05-26T22:50:28.607638+0000","last_fullsized":"2024-05-26T22:50:28.607638+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:40:21.264809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:29.029641+0000","last_change":"2024-05-26T22:50:27.622010+0000","last_active":"2024-05-26T22:50:29.029641+0000","last_peered":"2024-05-26T22:50:29.029641+0000","last_clean":"2024-05-26T22:50:29.029641+0000","last_became_active":"2024-05-26T22:50:27.621491+0000","last_became_peered":"2024-05-26T22:50:27.621491+0000","last_unstale":"2024-05-26T22:50:29.029641+0000","last_undegraded":"2024-05-26T22:50:29.029641+0000","last_fullsized":"2024-05-26T22:50:29.029641+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:58:06.516711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.610098+0000","last_change":"2024-05-26T22:50:27.619003+0000","last_active":"2024-05-26T22:50:28.610098+0000","last_peered":"2024-05-26T22:50:28.610098+0000","last_clean":"2024-05-26T22:50:28.610098+0000","last_became_active":"2024-05-26T22:50:27.613040+0000","last_became_peered":"2024-05-26T22:50:27.613040+0000","last_unstale":"2024-05-26T22:50:28.610098+0000","last_undegraded":"2024-05-26T22:50:28.610098+0000","last_fullsized":"2024-05-26T22:50:28.610098+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:21:37.744153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:29.030212+0000","last_change":"2024-05-26T22:50:27.612767+0000","last_active":"2024-05-26T22:50:29.030212+0000","last_peered":"2024-05-26T22:50:29.030212+0000","last_clean":"2024-05-26T22:50:29.030212+0000","last_became_active":"2024-05-26T22:50:27.612139+0000","last_became_peered":"2024-05-26T22:50:27.612139+0000","last_unstale":"2024-05-26T22:50:29.030212+0000","last_undegraded":"2024-05-26T22:50:29.030212+0000","last_fullsized":"2024-05-26T22:50:29.030212+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:34:05.951317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.606233+0000","last_change":"2024-05-26T22:50:27.593124+0000","last_active":"2024-05-26T22:50:28.606233+0000","last_peered":"2024-05-26T22:50:28.606233+0000","last_clean":"2024-05-26T22:50:28.606233+0000","last_became_active":"2024-05-26T22:50:27.591829+0000","last_became_peered":"2024-05-26T22:50:27.591829+0000","last_unstale":"2024-05-26T22:50:28.606233+0000","last_undegraded":"2024-05-26T22:50:28.606233+0000","last_fullsized":"2024-05-26T22:50:28.606233+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:52:28.184259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.608451+0000","last_change":"2024-05-26T22:50:27.609384+0000","last_active":"2024-05-26T22:50:28.608451+0000","last_peered":"2024-05-26T22:50:28.608451+0000","last_clean":"2024-05-26T22:50:28.608451+0000","last_became_active":"2024-05-26T22:50:27.604237+0000","last_became_peered":"2024-05-26T22:50:27.604237+0000","last_unstale":"2024-05-26T22:50:28.608451+0000","last_undegraded":"2024-05-26T22:50:28.608451+0000","last_fullsized":"2024-05-26T22:50:28.608451+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:15:23.061792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.608544+0000","last_change":"2024-05-26T22:50:27.593515+0000","last_active":"2024-05-26T22:50:28.608544+0000","last_peered":"2024-05-26T22:50:28.608544+0000","last_clean":"2024-05-26T22:50:28.608544+0000","last_became_active":"2024-05-26T22:50:27.592190+0000","last_became_peered":"2024-05-26T22:50:27.592190+0000","last_unstale":"2024-05-26T22:50:28.608544+0000","last_undegraded":"2024-05-26T22:50:28.608544+0000","last_fullsized":"2024-05-26T22:50:28.608544+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:55:33.647672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.608476+0000","last_change":"2024-05-26T22:50:27.605243+0000","last_active":"2024-05-26T22:50:28.608476+0000","last_peered":"2024-05-26T22:50:28.608476+0000","last_clean":"2024-05-26T22:50:28.608476+0000","last_became_active":"2024-05-26T22:50:27.604931+0000","last_became_peered":"2024-05-26T22:50:27.604931+0000","last_unstale":"2024-05-26T22:50:28.608476+0000","last_undegraded":"2024-05-26T22:50:28.608476+0000","last_fullsized":"2024-05-26T22:50:28.608476+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:21:16.701772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:29.029545+0000","last_change":"2024-05-26T22:50:27.627122+0000","last_active":"2024-05-26T22:50:29.029545+0000","last_peered":"2024-05-26T22:50:29.029545+0000","last_clean":"2024-05-26T22:50:29.029545+0000","last_became_active":"2024-05-26T22:50:27.626847+0000","last_became_peered":"2024-05-26T22:50:27.626847+0000","last_unstale":"2024-05-26T22:50:29.029545+0000","last_undegraded":"2024-05-26T22:50:29.029545+0000","last_fullsized":"2024-05-26T22:50:29.029545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:40:21.264809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:29.028177+0000","last_change":"2024-05-26T22:50:27.616958+0000","last_active":"2024-05-26T22:50:29.028177+0000","last_peered":"2024-05-26T22:50:29.028177+0000","last_clean":"2024-05-26T22:50:29.028177+0000","last_became_active":"2024-05-26T22:50:27.616551+0000","last_became_peered":"2024-05-26T22:50:27.616551+0000","last_unstale":"2024-05-26T22:50:29.028177+0000","last_undegraded":"2024-05-26T22:50:29.028177+0000","last_fullsized":"2024-05-26T22:50:29.028177+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:14:28.234483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.607145+0000","last_change":"2024-05-26T22:50:27.608517+0000","last_active":"2024-05-26T22:50:28.607145+0000","last_peered":"2024-05-26T22:50:28.607145+0000","last_clean":"2024-05-26T22:50:28.607145+0000","last_became_active":"2024-05-26T22:50:27.606950+0000","last_became_peered":"2024-05-26T22:50:27.606950+0000","last_unstale":"2024-05-26T22:50:28.607145+0000","last_undegraded":"2024-05-26T22:50:28.607145+0000","last_fullsized":"2024-05-26T22:50:28.607145+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:28:42.535219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.609790+0000","last_change":"2024-05-26T22:50:27.608959+0000","last_active":"2024-05-26T22:50:28.609790+0000","last_peered":"2024-05-26T22:50:28.609790+0000","last_clean":"2024-05-26T22:50:28.609790+0000","last_became_active":"2024-05-26T22:50:27.608700+0000","last_became_peered":"2024-05-26T22:50:27.608700+0000","last_unstale":"2024-05-26T22:50:28.609790+0000","last_undegraded":"2024-05-26T22:50:28.609790+0000","last_fullsized":"2024-05-26T22:50:28.609790+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:12:46.452654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.608690+0000","last_change":"2024-05-26T22:50:27.611336+0000","last_active":"2024-05-26T22:50:28.608690+0000","last_peered":"2024-05-26T22:50:28.608690+0000","last_clean":"2024-05-26T22:50:28.608690+0000","last_became_active":"2024-05-26T22:50:27.611063+0000","last_became_peered":"2024-05-26T22:50:27.611063+0000","last_unstale":"2024-05-26T22:50:28.608690+0000","last_undegraded":"2024-05-26T22:50:28.608690+0000","last_fullsized":"2024-05-26T22:50:28.608690+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:53:36.611294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.617634+0000","last_change":"2024-05-26T22:50:27.611141+0000","last_active":"2024-05-26T22:50:28.617634+0000","last_peered":"2024-05-26T22:50:28.617634+0000","last_clean":"2024-05-26T22:50:28.617634+0000","last_became_active":"2024-05-26T22:50:27.610706+0000","last_became_peered":"2024-05-26T22:50:27.610706+0000","last_unstale":"2024-05-26T22:50:28.617634+0000","last_undegraded":"2024-05-26T22:50:28.617634+0000","last_fullsized":"2024-05-26T22:50:28.617634+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:29:00.413315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.617367+0000","last_change":"2024-05-26T22:50:27.625629+0000","last_active":"2024-05-26T22:50:28.617367+0000","last_peered":"2024-05-26T22:50:28.617367+0000","last_clean":"2024-05-26T22:50:28.617367+0000","last_became_active":"2024-05-26T22:50:27.625326+0000","last_became_peered":"2024-05-26T22:50:27.625326+0000","last_unstale":"2024-05-26T22:50:28.617367+0000","last_undegraded":"2024-05-26T22:50:28.617367+0000","last_fullsized":"2024-05-26T22:50:28.617367+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:52:28.184259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.610528+0000","last_change":"2024-05-26T22:50:27.612043+0000","last_active":"2024-05-26T22:50:28.610528+0000","last_peered":"2024-05-26T22:50:28.610528+0000","last_clean":"2024-05-26T22:50:28.610528+0000","last_became_active":"2024-05-26T22:50:27.611759+0000","last_became_peered":"2024-05-26T22:50:27.611759+0000","last_unstale":"2024-05-26T22:50:28.610528+0000","last_undegraded":"2024-05-26T22:50:28.610528+0000","last_fullsized":"2024-05-26T22:50:28.610528+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:54:47.587339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:29.028648+0000","last_change":"2024-05-26T22:50:27.620986+0000","last_active":"2024-05-26T22:50:29.028648+0000","last_peered":"2024-05-26T22:50:29.028648+0000","last_clean":"2024-05-26T22:50:29.028648+0000","last_became_active":"2024-05-26T22:50:27.620701+0000","last_became_peered":"2024-05-26T22:50:27.620701+0000","last_unstale":"2024-05-26T22:50:29.028648+0000","last_undegraded":"2024-05-26T22:50:29.028648+0000","last_fullsized":"2024-05-26T22:50:29.028648+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:13:05.372755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.610175+0000","last_change":"2024-05-26T22:50:27.623707+0000","last_active":"2024-05-26T22:50:28.610175+0000","last_peered":"2024-05-26T22:50:28.610175+0000","last_clean":"2024-05-26T22:50:28.610175+0000","last_became_active":"2024-05-26T22:50:27.623450+0000","last_became_peered":"2024-05-26T22:50:27.623450+0000","last_unstale":"2024-05-26T22:50:28.610175+0000","last_undegraded":"2024-05-26T22:50:28.610175+0000","last_fullsized":"2024-05-26T22:50:28.610175+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:21:39.447850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.607328+0000","last_change":"2024-05-26T22:50:27.595025+0000","last_active":"2024-05-26T22:50:28.607328+0000","last_peered":"2024-05-26T22:50:28.607328+0000","last_clean":"2024-05-26T22:50:28.607328+0000","last_became_active":"2024-05-26T22:50:27.594473+0000","last_became_peered":"2024-05-26T22:50:27.594473+0000","last_unstale":"2024-05-26T22:50:28.607328+0000","last_undegraded":"2024-05-26T22:50:28.607328+0000","last_fullsized":"2024-05-26T22:50:28.607328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:44:14.357986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.609269+0000","last_change":"2024-05-26T22:50:27.609501+0000","last_active":"2024-05-26T22:50:28.609269+0000","last_peered":"2024-05-26T22:50:28.609269+0000","last_clean":"2024-05-26T22:50:28.609269+0000","last_became_active":"2024-05-26T22:50:27.608925+0000","last_became_peered":"2024-05-26T22:50:27.608925+0000","last_unstale":"2024-05-26T22:50:28.609269+0000","last_undegraded":"2024-05-26T22:50:28.609269+0000","last_fullsized":"2024-05-26T22:50:28.609269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:58:29.789956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.608671+0000","last_change":"2024-05-26T22:50:27.606676+0000","last_active":"2024-05-26T22:50:28.608671+0000","last_peered":"2024-05-26T22:50:28.608671+0000","last_clean":"2024-05-26T22:50:28.608671+0000","last_became_active":"2024-05-26T22:50:27.606399+0000","last_became_peered":"2024-05-26T22:50:27.606399+0000","last_unstale":"2024-05-26T22:50:28.608671+0000","last_undegraded":"2024-05-26T22:50:28.608671+0000","last_fullsized":"2024-05-26T22:50:28.608671+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:07:39.345324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.609416+0000","last_change":"2024-05-26T22:50:27.603472+0000","last_active":"2024-05-26T22:50:28.609416+0000","last_peered":"2024-05-26T22:50:28.609416+0000","last_clean":"2024-05-26T22:50:28.609416+0000","last_became_active":"2024-05-26T22:50:27.602661+0000","last_became_peered":"2024-05-26T22:50:27.602661+0000","last_unstale":"2024-05-26T22:50:28.609416+0000","last_undegraded":"2024-05-26T22:50:28.609416+0000","last_fullsized":"2024-05-26T22:50:28.609416+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:26:43.963641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:29.028316+0000","last_change":"2024-05-26T22:50:27.621771+0000","last_active":"2024-05-26T22:50:29.028316+0000","last_peered":"2024-05-26T22:50:29.028316+0000","last_clean":"2024-05-26T22:50:29.028316+0000","last_became_active":"2024-05-26T22:50:27.621425+0000","last_became_peered":"2024-05-26T22:50:27.621425+0000","last_unstale":"2024-05-26T22:50:29.028316+0000","last_undegraded":"2024-05-26T22:50:29.028316+0000","last_fullsized":"2024-05-26T22:50:29.028316+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:04:18.699067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.616839+0000","last_change":"2024-05-26T22:50:27.607252+0000","last_active":"2024-05-26T22:50:28.616839+0000","last_peered":"2024-05-26T22:50:28.616839+0000","last_clean":"2024-05-26T22:50:28.616839+0000","last_became_active":"2024-05-26T22:50:27.606329+0000","last_became_peered":"2024-05-26T22:50:27.606329+0000","last_unstale":"2024-05-26T22:50:28.616839+0000","last_undegraded":"2024-05-26T22:50:28.616839+0000","last_fullsized":"2024-05-26T22:50:28.616839+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:14:10.443873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:29.029593+0000","last_change":"2024-05-26T22:50:27.626991+0000","last_active":"2024-05-26T22:50:29.029593+0000","last_peered":"2024-05-26T22:50:29.029593+0000","last_clean":"2024-05-26T22:50:29.029593+0000","last_became_active":"2024-05-26T22:50:27.626702+0000","last_became_peered":"2024-05-26T22:50:27.626702+0000","last_unstale":"2024-05-26T22:50:29.029593+0000","last_undegraded":"2024-05-26T22:50:29.029593+0000","last_fullsized":"2024-05-26T22:50:29.029593+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:52:44.229379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.609538+0000","last_change":"2024-05-26T22:50:27.602948+0000","last_active":"2024-05-26T22:50:28.609538+0000","last_peered":"2024-05-26T22:50:28.609538+0000","last_clean":"2024-05-26T22:50:28.609538+0000","last_became_active":"2024-05-26T22:50:27.602641+0000","last_became_peered":"2024-05-26T22:50:27.602641+0000","last_unstale":"2024-05-26T22:50:28.609538+0000","last_undegraded":"2024-05-26T22:50:28.609538+0000","last_fullsized":"2024-05-26T22:50:28.609538+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:28:13.522438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.616295+0000","last_change":"2024-05-26T22:50:27.613332+0000","last_active":"2024-05-26T22:50:28.616295+0000","last_peered":"2024-05-26T22:50:28.616295+0000","last_clean":"2024-05-26T22:50:28.616295+0000","last_became_active":"2024-05-26T22:50:27.613002+0000","last_became_peered":"2024-05-26T22:50:27.613002+0000","last_unstale":"2024-05-26T22:50:28.616295+0000","last_undegraded":"2024-05-26T22:50:28.616295+0000","last_fullsized":"2024-05-26T22:50:28.616295+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:35:16.414543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.609812+0000","last_change":"2024-05-26T22:50:27.609655+0000","last_active":"2024-05-26T22:50:28.609812+0000","last_peered":"2024-05-26T22:50:28.609812+0000","last_clean":"2024-05-26T22:50:28.609812+0000","last_became_active":"2024-05-26T22:50:27.609367+0000","last_became_peered":"2024-05-26T22:50:27.609367+0000","last_unstale":"2024-05-26T22:50:28.609812+0000","last_undegraded":"2024-05-26T22:50:28.609812+0000","last_fullsized":"2024-05-26T22:50:28.609812+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:48:07.563526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:29.027930+0000","last_change":"2024-05-26T22:50:27.613465+0000","last_active":"2024-05-26T22:50:29.027930+0000","last_peered":"2024-05-26T22:50:29.027930+0000","last_clean":"2024-05-26T22:50:29.027930+0000","last_became_active":"2024-05-26T22:50:27.613133+0000","last_became_peered":"2024-05-26T22:50:27.613133+0000","last_unstale":"2024-05-26T22:50:29.027930+0000","last_undegraded":"2024-05-26T22:50:29.027930+0000","last_fullsized":"2024-05-26T22:50:29.027930+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:28:32.032578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.608109+0000","last_change":"2024-05-26T22:50:27.625540+0000","last_active":"2024-05-26T22:50:28.608109+0000","last_peered":"2024-05-26T22:50:28.608109+0000","last_clean":"2024-05-26T22:50:28.608109+0000","last_became_active":"2024-05-26T22:50:27.624727+0000","last_became_peered":"2024-05-26T22:50:27.624727+0000","last_unstale":"2024-05-26T22:50:28.608109+0000","last_undegraded":"2024-05-26T22:50:28.608109+0000","last_fullsized":"2024-05-26T22:50:28.608109+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:57:10.223342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"43'4","reported_seq":18,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.611114+0000","last_change":"2024-05-26T22:50:26.581794+0000","last_active":"2024-05-26T22:50:28.611114+0000","last_peered":"2024-05-26T22:50:28.611114+0000","last_clean":"2024-05-26T22:50:28.611114+0000","last_became_active":"2024-05-26T22:50:26.581504+0000","last_became_peered":"2024-05-26T22:50:26.581504+0000","last_unstale":"2024-05-26T22:50:28.611114+0000","last_undegraded":"2024-05-26T22:50:28.611114+0000","last_fullsized":"2024-05-26T22:50:28.611114+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:44:13.344424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.609437+0000","last_change":"2024-05-26T22:50:27.607318+0000","last_active":"2024-05-26T22:50:28.609437+0000","last_peered":"2024-05-26T22:50:28.609437+0000","last_clean":"2024-05-26T22:50:28.609437+0000","last_became_active":"2024-05-26T22:50:27.607013+0000","last_became_peered":"2024-05-26T22:50:27.607013+0000","last_unstale":"2024-05-26T22:50:28.609437+0000","last_undegraded":"2024-05-26T22:50:28.609437+0000","last_fullsized":"2024-05-26T22:50:28.609437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:23:58.424666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"43'1","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.617636+0000","last_change":"2024-05-26T22:50:26.582366+0000","last_active":"2024-05-26T22:50:28.617636+0000","last_peered":"2024-05-26T22:50:28.617636+0000","last_clean":"2024-05-26T22:50:28.617636+0000","last_became_active":"2024-05-26T22:50:26.582022+0000","last_became_peered":"2024-05-26T22:50:26.582022+0000","last_unstale":"2024-05-26T22:50:28.617636+0000","last_undegraded":"2024-05-26T22:50:28.617636+0000","last_fullsized":"2024-05-26T22:50:28.617636+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:58:28.776394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:29.029938+0000","last_change":"2024-05-26T22:50:27.622401+0000","last_active":"2024-05-26T22:50:29.029938+0000","last_peered":"2024-05-26T22:50:29.029938+0000","last_clean":"2024-05-26T22:50:29.029938+0000","last_became_active":"2024-05-26T22:50:27.622072+0000","last_became_peered":"2024-05-26T22:50:27.622072+0000","last_unstale":"2024-05-26T22:50:29.029938+0000","last_undegraded":"2024-05-26T22:50:29.029938+0000","last_fullsized":"2024-05-26T22:50:29.029938+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:34:05.951317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"43'2","reported_seq":16,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.607043+0000","last_change":"2024-05-26T22:50:26.579628+0000","last_active":"2024-05-26T22:50:28.607043+0000","last_peered":"2024-05-26T22:50:28.607043+0000","last_clean":"2024-05-26T22:50:28.607043+0000","last_became_active":"2024-05-26T22:50:26.579316+0000","last_became_peered":"2024-05-26T22:50:26.579316+0000","last_unstale":"2024-05-26T22:50:28.607043+0000","last_undegraded":"2024-05-26T22:50:28.607043+0000","last_fullsized":"2024-05-26T22:50:28.607043+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:53:35.597732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.607130+0000","last_change":"2024-05-26T22:50:27.609117+0000","last_active":"2024-05-26T22:50:28.607130+0000","last_peered":"2024-05-26T22:50:28.607130+0000","last_clean":"2024-05-26T22:50:28.607130+0000","last_became_active":"2024-05-26T22:50:27.601651+0000","last_became_peered":"2024-05-26T22:50:27.601651+0000","last_unstale":"2024-05-26T22:50:28.607130+0000","last_undegraded":"2024-05-26T22:50:28.607130+0000","last_fullsized":"2024-05-26T22:50:28.607130+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:33:34.699738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"43'2","reported_seq":16,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:29.030473+0000","last_change":"2024-05-26T22:50:26.589034+0000","last_active":"2024-05-26T22:50:29.030473+0000","last_peered":"2024-05-26T22:50:29.030473+0000","last_clean":"2024-05-26T22:50:29.030473+0000","last_became_active":"2024-05-26T22:50:26.587800+0000","last_became_peered":"2024-05-26T22:50:26.587800+0000","last_unstale":"2024-05-26T22:50:29.030473+0000","last_undegraded":"2024-05-26T22:50:29.030473+0000","last_fullsized":"2024-05-26T22:50:29.030473+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:52:27.170697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.616330+0000","last_change":"2024-05-26T22:50:27.610168+0000","last_active":"2024-05-26T22:50:28.616330+0000","last_peered":"2024-05-26T22:50:28.616330+0000","last_clean":"2024-05-26T22:50:28.616330+0000","last_became_active":"2024-05-26T22:50:27.609872+0000","last_became_peered":"2024-05-26T22:50:27.609872+0000","last_unstale":"2024-05-26T22:50:28.616330+0000","last_undegraded":"2024-05-26T22:50:28.616330+0000","last_fullsized":"2024-05-26T22:50:28.616330+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:09:00.534190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"43'1","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.610258+0000","last_change":"2024-05-26T22:50:26.588331+0000","last_active":"2024-05-26T22:50:28.610258+0000","last_peered":"2024-05-26T22:50:28.610258+0000","last_clean":"2024-05-26T22:50:28.610258+0000","last_became_active":"2024-05-26T22:50:26.588083+0000","last_became_peered":"2024-05-26T22:50:26.588083+0000","last_unstale":"2024-05-26T22:50:28.610258+0000","last_undegraded":"2024-05-26T22:50:28.610258+0000","last_fullsized":"2024-05-26T22:50:28.610258+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:52:27.170697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.607891+0000","last_change":"2024-05-26T22:50:27.611774+0000","last_active":"2024-05-26T22:50:28.607891+0000","last_peered":"2024-05-26T22:50:28.607891+0000","last_clean":"2024-05-26T22:50:28.607891+0000","last_became_active":"2024-05-26T22:50:27.604920+0000","last_became_peered":"2024-05-26T22:50:27.604920+0000","last_unstale":"2024-05-26T22:50:28.607891+0000","last_undegraded":"2024-05-26T22:50:28.607891+0000","last_fullsized":"2024-05-26T22:50:28.607891+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:58:06.516711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"43'1","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.616637+0000","last_change":"2024-05-26T22:50:26.585020+0000","last_active":"2024-05-26T22:50:28.616637+0000","last_peered":"2024-05-26T22:50:28.616637+0000","last_clean":"2024-05-26T22:50:28.616637+0000","last_became_active":"2024-05-26T22:50:26.584736+0000","last_became_peered":"2024-05-26T22:50:26.584736+0000","last_unstale":"2024-05-26T22:50:28.616637+0000","last_undegraded":"2024-05-26T22:50:28.616637+0000","last_fullsized":"2024-05-26T22:50:28.616637+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:42:43.319527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.606395+0000","last_change":"2024-05-26T22:50:27.593506+0000","last_active":"2024-05-26T22:50:28.606395+0000","last_peered":"2024-05-26T22:50:28.606395+0000","last_clean":"2024-05-26T22:50:28.606395+0000","last_became_active":"2024-05-26T22:50:27.592798+0000","last_became_peered":"2024-05-26T22:50:27.592798+0000","last_unstale":"2024-05-26T22:50:28.606395+0000","last_undegraded":"2024-05-26T22:50:28.606395+0000","last_fullsized":"2024-05-26T22:50:28.606395+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:06:08.865730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:29.028885+0000","last_change":"2024-05-26T22:50:26.574522+0000","last_active":"2024-05-26T22:50:29.028885+0000","last_peered":"2024-05-26T22:50:29.028885+0000","last_clean":"2024-05-26T22:50:29.028885+0000","last_became_active":"2024-05-26T22:50:26.574033+0000","last_became_peered":"2024-05-26T22:50:26.574033+0000","last_unstale":"2024-05-26T22:50:29.028885+0000","last_undegraded":"2024-05-26T22:50:29.028885+0000","last_fullsized":"2024-05-26T22:50:29.028885+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:29:03.868288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.608513+0000","last_change":"2024-05-26T22:50:27.609584+0000","last_active":"2024-05-26T22:50:28.608513+0000","last_peered":"2024-05-26T22:50:28.608513+0000","last_clean":"2024-05-26T22:50:28.608513+0000","last_became_active":"2024-05-26T22:50:27.609305+0000","last_became_peered":"2024-05-26T22:50:27.609305+0000","last_unstale":"2024-05-26T22:50:28.608513+0000","last_undegraded":"2024-05-26T22:50:28.608513+0000","last_fullsized":"2024-05-26T22:50:28.608513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:42:44.333089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.606424+0000","last_change":"2024-05-26T22:50:26.581544+0000","last_active":"2024-05-26T22:50:28.606424+0000","last_peered":"2024-05-26T22:50:28.606424+0000","last_clean":"2024-05-26T22:50:28.606424+0000","last_became_active":"2024-05-26T22:50:26.581228+0000","last_became_peered":"2024-05-26T22:50:26.581228+0000","last_unstale":"2024-05-26T22:50:28.606424+0000","last_undegraded":"2024-05-26T22:50:28.606424+0000","last_fullsized":"2024-05-26T22:50:28.606424+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:19:59.957522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.617537+0000","last_change":"2024-05-26T22:50:27.625496+0000","last_active":"2024-05-26T22:50:28.617537+0000","last_peered":"2024-05-26T22:50:28.617537+0000","last_clean":"2024-05-26T22:50:28.617537+0000","last_became_active":"2024-05-26T22:50:27.625123+0000","last_became_peered":"2024-05-26T22:50:27.625123+0000","last_unstale":"2024-05-26T22:50:28.617537+0000","last_undegraded":"2024-05-26T22:50:28.617537+0000","last_fullsized":"2024-05-26T22:50:28.617537+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:06:08.865730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.617772+0000","last_change":"2024-05-26T22:50:26.583932+0000","last_active":"2024-05-26T22:50:28.617772+0000","last_peered":"2024-05-26T22:50:28.617772+0000","last_clean":"2024-05-26T22:50:28.617772+0000","last_became_active":"2024-05-26T22:50:26.582486+0000","last_became_peered":"2024-05-26T22:50:26.582486+0000","last_unstale":"2024-05-26T22:50:28.617772+0000","last_undegraded":"2024-05-26T22:50:28.617772+0000","last_fullsized":"2024-05-26T22:50:28.617772+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:53:35.597732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:29.031528+0000","last_change":"2024-05-26T22:50:27.611333+0000","last_active":"2024-05-26T22:50:29.031528+0000","last_peered":"2024-05-26T22:50:29.031528+0000","last_clean":"2024-05-26T22:50:29.031528+0000","last_became_active":"2024-05-26T22:50:27.610941+0000","last_became_peered":"2024-05-26T22:50:27.610941+0000","last_unstale":"2024-05-26T22:50:29.031528+0000","last_undegraded":"2024-05-26T22:50:29.031528+0000","last_fullsized":"2024-05-26T22:50:29.031528+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:13:34.494539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.608745+0000","last_change":"2024-05-26T22:50:26.587905+0000","last_active":"2024-05-26T22:50:28.608745+0000","last_peered":"2024-05-26T22:50:28.608745+0000","last_clean":"2024-05-26T22:50:28.608745+0000","last_became_active":"2024-05-26T22:50:26.587642+0000","last_became_peered":"2024-05-26T22:50:26.587642+0000","last_unstale":"2024-05-26T22:50:28.608745+0000","last_undegraded":"2024-05-26T22:50:28.608745+0000","last_fullsized":"2024-05-26T22:50:28.608745+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:53:35.597732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.610624+0000","last_change":"2024-05-26T22:50:27.630178+0000","last_active":"2024-05-26T22:50:28.610624+0000","last_peered":"2024-05-26T22:50:28.610624+0000","last_clean":"2024-05-26T22:50:28.610624+0000","last_became_active":"2024-05-26T22:50:27.629834+0000","last_became_peered":"2024-05-26T22:50:27.629834+0000","last_unstale":"2024-05-26T22:50:28.610624+0000","last_undegraded":"2024-05-26T22:50:28.610624+0000","last_fullsized":"2024-05-26T22:50:28.610624+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:04:47.607246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"43'3","reported_seq":17,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:29.031132+0000","last_change":"2024-05-26T22:50:26.591431+0000","last_active":"2024-05-26T22:50:29.031132+0000","last_peered":"2024-05-26T22:50:29.031132+0000","last_clean":"2024-05-26T22:50:29.031132+0000","last_became_active":"2024-05-26T22:50:26.591180+0000","last_became_peered":"2024-05-26T22:50:26.591180+0000","last_unstale":"2024-05-26T22:50:29.031132+0000","last_undegraded":"2024-05-26T22:50:29.031132+0000","last_fullsized":"2024-05-26T22:50:29.031132+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:08:59.520628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:29.030606+0000","last_change":"2024-05-26T22:50:27.612043+0000","last_active":"2024-05-26T22:50:29.030606+0000","last_peered":"2024-05-26T22:50:29.030606+0000","last_clean":"2024-05-26T22:50:29.030606+0000","last_became_active":"2024-05-26T22:50:27.611779+0000","last_became_peered":"2024-05-26T22:50:27.611779+0000","last_unstale":"2024-05-26T22:50:29.030606+0000","last_undegraded":"2024-05-26T22:50:29.030606+0000","last_fullsized":"2024-05-26T22:50:29.030606+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:21:16.701772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"43'6","reported_seq":20,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:29.028548+0000","last_change":"2024-05-26T22:50:26.585012+0000","last_active":"2024-05-26T22:50:29.028548+0000","last_peered":"2024-05-26T22:50:29.028548+0000","last_clean":"2024-05-26T22:50:29.028548+0000","last_became_active":"2024-05-26T22:50:26.584653+0000","last_became_peered":"2024-05-26T22:50:26.584653+0000","last_unstale":"2024-05-26T22:50:29.028548+0000","last_undegraded":"2024-05-26T22:50:29.028548+0000","last_fullsized":"2024-05-26T22:50:29.028548+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:19:00.151465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.609458+0000","last_change":"2024-05-26T22:50:27.620675+0000","last_active":"2024-05-26T22:50:28.609458+0000","last_peered":"2024-05-26T22:50:28.609458+0000","last_clean":"2024-05-26T22:50:28.609458+0000","last_became_active":"2024-05-26T22:50:27.620365+0000","last_became_peered":"2024-05-26T22:50:27.620365+0000","last_unstale":"2024-05-26T22:50:28.609458+0000","last_undegraded":"2024-05-26T22:50:28.609458+0000","last_fullsized":"2024-05-26T22:50:28.609458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:15:35.289066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"43'2","reported_seq":16,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.610360+0000","last_change":"2024-05-26T22:50:26.577323+0000","last_active":"2024-05-26T22:50:28.610360+0000","last_peered":"2024-05-26T22:50:28.610360+0000","last_clean":"2024-05-26T22:50:28.610360+0000","last_became_active":"2024-05-26T22:50:26.576604+0000","last_became_peered":"2024-05-26T22:50:26.576604+0000","last_unstale":"2024-05-26T22:50:28.610360+0000","last_undegraded":"2024-05-26T22:50:28.610360+0000","last_fullsized":"2024-05-26T22:50:28.610360+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:06:07.852168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.608226+0000","last_change":"2024-05-26T22:50:27.615536+0000","last_active":"2024-05-26T22:50:28.608226+0000","last_peered":"2024-05-26T22:50:28.608226+0000","last_clean":"2024-05-26T22:50:28.608226+0000","last_became_active":"2024-05-26T22:50:27.615152+0000","last_became_peered":"2024-05-26T22:50:27.615152+0000","last_unstale":"2024-05-26T22:50:28.608226+0000","last_undegraded":"2024-05-26T22:50:28.608226+0000","last_fullsized":"2024-05-26T22:50:28.608226+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:34:05.951317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:29.030456+0000","last_change":"2024-05-26T22:50:26.589863+0000","last_active":"2024-05-26T22:50:29.030456+0000","last_peered":"2024-05-26T22:50:29.030456+0000","last_clean":"2024-05-26T22:50:29.030456+0000","last_became_active":"2024-05-26T22:50:26.589578+0000","last_became_peered":"2024-05-26T22:50:26.589578+0000","last_unstale":"2024-05-26T22:50:29.030456+0000","last_undegraded":"2024-05-26T22:50:29.030456+0000","last_fullsized":"2024-05-26T22:50:29.030456+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:29:03.868288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.609765+0000","last_change":"2024-05-26T22:50:27.619152+0000","last_active":"2024-05-26T22:50:28.609765+0000","last_peered":"2024-05-26T22:50:28.609765+0000","last_clean":"2024-05-26T22:50:28.609765+0000","last_became_active":"2024-05-26T22:50:27.618811+0000","last_became_peered":"2024-05-26T22:50:27.618811+0000","last_unstale":"2024-05-26T22:50:28.609765+0000","last_undegraded":"2024-05-26T22:50:28.609765+0000","last_fullsized":"2024-05-26T22:50:28.609765+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:13:34.494539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.609954+0000","last_change":"2024-05-26T22:50:26.579106+0000","last_active":"2024-05-26T22:50:28.609954+0000","last_peered":"2024-05-26T22:50:28.609954+0000","last_clean":"2024-05-26T22:50:28.609954+0000","last_became_active":"2024-05-26T22:50:26.578796+0000","last_became_peered":"2024-05-26T22:50:26.578796+0000","last_unstale":"2024-05-26T22:50:28.609954+0000","last_undegraded":"2024-05-26T22:50:28.609954+0000","last_fullsized":"2024-05-26T22:50:28.609954+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:48:39.224457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:29.030463+0000","last_change":"2024-05-26T22:50:27.616914+0000","last_active":"2024-05-26T22:50:29.030463+0000","last_peered":"2024-05-26T22:50:29.030463+0000","last_clean":"2024-05-26T22:50:29.030463+0000","last_became_active":"2024-05-26T22:50:27.616624+0000","last_became_peered":"2024-05-26T22:50:27.616624+0000","last_unstale":"2024-05-26T22:50:29.030463+0000","last_undegraded":"2024-05-26T22:50:29.030463+0000","last_fullsized":"2024-05-26T22:50:29.030463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:15:23.061792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"43'8","reported_seq":22,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.606500+0000","last_change":"2024-05-26T22:50:26.583566+0000","last_active":"2024-05-26T22:50:28.606500+0000","last_peered":"2024-05-26T22:50:28.606500+0000","last_clean":"2024-05-26T22:50:28.606500+0000","last_became_active":"2024-05-26T22:50:26.583235+0000","last_became_peered":"2024-05-26T22:50:26.583235+0000","last_unstale":"2024-05-26T22:50:28.606500+0000","last_undegraded":"2024-05-26T22:50:28.606500+0000","last_fullsized":"2024-05-26T22:50:28.606500+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:42:43.319527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:29.028557+0000","last_change":"2024-05-26T22:50:27.606939+0000","last_active":"2024-05-26T22:50:29.028557+0000","last_peered":"2024-05-26T22:50:29.028557+0000","last_clean":"2024-05-26T22:50:29.028557+0000","last_became_active":"2024-05-26T22:50:27.606619+0000","last_became_peered":"2024-05-26T22:50:27.606619+0000","last_unstale":"2024-05-26T22:50:29.028557+0000","last_undegraded":"2024-05-26T22:50:29.028557+0000","last_fullsized":"2024-05-26T22:50:29.028557+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:44:11.226829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"43'1","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:29.028496+0000","last_change":"2024-05-26T22:50:26.579397+0000","last_active":"2024-05-26T22:50:29.028496+0000","last_peered":"2024-05-26T22:50:29.028496+0000","last_clean":"2024-05-26T22:50:29.028496+0000","last_became_active":"2024-05-26T22:50:26.579019+0000","last_became_peered":"2024-05-26T22:50:26.579019+0000","last_unstale":"2024-05-26T22:50:29.028496+0000","last_undegraded":"2024-05-26T22:50:29.028496+0000","last_fullsized":"2024-05-26T22:50:29.028496+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:47:40.423971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.608654+0000","last_change":"2024-05-26T22:50:27.626388+0000","last_active":"2024-05-26T22:50:28.608654+0000","last_peered":"2024-05-26T22:50:28.608654+0000","last_clean":"2024-05-26T22:50:28.608654+0000","last_became_active":"2024-05-26T22:50:27.625515+0000","last_became_peered":"2024-05-26T22:50:27.625515+0000","last_unstale":"2024-05-26T22:50:28.608654+0000","last_undegraded":"2024-05-26T22:50:28.608654+0000","last_fullsized":"2024-05-26T22:50:28.608654+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:44:57.492554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"43'2","reported_seq":16,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.617792+0000","last_change":"2024-05-26T22:50:26.584807+0000","last_active":"2024-05-26T22:50:28.617792+0000","last_peered":"2024-05-26T22:50:28.617792+0000","last_clean":"2024-05-26T22:50:28.617792+0000","last_became_active":"2024-05-26T22:50:26.584427+0000","last_became_peered":"2024-05-26T22:50:26.584427+0000","last_unstale":"2024-05-26T22:50:28.617792+0000","last_undegraded":"2024-05-26T22:50:28.617792+0000","last_fullsized":"2024-05-26T22:50:28.617792+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:26:42.950079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.610112+0000","last_change":"2024-05-26T22:50:27.619929+0000","last_active":"2024-05-26T22:50:28.610112+0000","last_peered":"2024-05-26T22:50:28.610112+0000","last_clean":"2024-05-26T22:50:28.610112+0000","last_became_active":"2024-05-26T22:50:27.619626+0000","last_became_peered":"2024-05-26T22:50:27.619626+0000","last_unstale":"2024-05-26T22:50:28.610112+0000","last_undegraded":"2024-05-26T22:50:28.610112+0000","last_fullsized":"2024-05-26T22:50:28.610112+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:57:10.223342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"43'1","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:29.029470+0000","last_change":"2024-05-26T22:50:26.576296+0000","last_active":"2024-05-26T22:50:29.029470+0000","last_peered":"2024-05-26T22:50:29.029470+0000","last_clean":"2024-05-26T22:50:29.029470+0000","last_became_active":"2024-05-26T22:50:26.575800+0000","last_became_peered":"2024-05-26T22:50:26.575800+0000","last_unstale":"2024-05-26T22:50:29.029470+0000","last_undegraded":"2024-05-26T22:50:29.029470+0000","last_fullsized":"2024-05-26T22:50:29.029470+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:35:15.400981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.610347+0000","last_change":"2024-05-26T22:50:27.606967+0000","last_active":"2024-05-26T22:50:28.610347+0000","last_peered":"2024-05-26T22:50:28.610347+0000","last_clean":"2024-05-26T22:50:28.610347+0000","last_became_active":"2024-05-26T22:50:27.606245+0000","last_became_peered":"2024-05-26T22:50:27.606245+0000","last_unstale":"2024-05-26T22:50:28.610347+0000","last_undegraded":"2024-05-26T22:50:28.610347+0000","last_fullsized":"2024-05-26T22:50:28.610347+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:31:49.984660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.610553+0000","last_change":"2024-05-26T22:50:26.582011+0000","last_active":"2024-05-26T22:50:28.610553+0000","last_peered":"2024-05-26T22:50:28.610553+0000","last_clean":"2024-05-26T22:50:28.610553+0000","last_became_active":"2024-05-26T22:50:26.581706+0000","last_became_peered":"2024-05-26T22:50:26.581706+0000","last_unstale":"2024-05-26T22:50:28.610553+0000","last_undegraded":"2024-05-26T22:50:28.610553+0000","last_fullsized":"2024-05-26T22:50:28.610553+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:28:59.399753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.610117+0000","last_change":"2024-05-26T22:50:27.613787+0000","last_active":"2024-05-26T22:50:28.610117+0000","last_peered":"2024-05-26T22:50:28.610117+0000","last_clean":"2024-05-26T22:50:28.610117+0000","last_became_active":"2024-05-26T22:50:27.612338+0000","last_became_peered":"2024-05-26T22:50:27.612338+0000","last_unstale":"2024-05-26T22:50:28.610117+0000","last_undegraded":"2024-05-26T22:50:28.610117+0000","last_fullsized":"2024-05-26T22:50:28.610117+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:49:03.875973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"43'2","reported_seq":16,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.610465+0000","last_change":"2024-05-26T22:50:26.588207+0000","last_active":"2024-05-26T22:50:28.610465+0000","last_peered":"2024-05-26T22:50:28.610465+0000","last_clean":"2024-05-26T22:50:28.610465+0000","last_became_active":"2024-05-26T22:50:26.587873+0000","last_became_peered":"2024-05-26T22:50:26.587873+0000","last_unstale":"2024-05-26T22:50:28.610465+0000","last_undegraded":"2024-05-26T22:50:28.610465+0000","last_fullsized":"2024-05-26T22:50:28.610465+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:28:41.521657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.605988+0000","last_change":"2024-05-26T22:50:27.610411+0000","last_active":"2024-05-26T22:50:28.605988+0000","last_peered":"2024-05-26T22:50:28.605988+0000","last_clean":"2024-05-26T22:50:28.605988+0000","last_became_active":"2024-05-26T22:50:27.610166+0000","last_became_peered":"2024-05-26T22:50:27.610166+0000","last_unstale":"2024-05-26T22:50:28.605988+0000","last_undegraded":"2024-05-26T22:50:28.605988+0000","last_fullsized":"2024-05-26T22:50:28.605988+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:28:13.522438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:29.029044+0000","last_change":"2024-05-26T22:50:26.581924+0000","last_active":"2024-05-26T22:50:29.029044+0000","last_peered":"2024-05-26T22:50:29.029044+0000","last_clean":"2024-05-26T22:50:29.029044+0000","last_became_active":"2024-05-26T22:50:26.581631+0000","last_became_peered":"2024-05-26T22:50:26.581631+0000","last_unstale":"2024-05-26T22:50:29.029044+0000","last_undegraded":"2024-05-26T22:50:29.029044+0000","last_fullsized":"2024-05-26T22:50:29.029044+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:25:43.834484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:29.027973+0000","last_change":"2024-05-26T22:50:27.618550+0000","last_active":"2024-05-26T22:50:29.027973+0000","last_peered":"2024-05-26T22:50:29.027973+0000","last_clean":"2024-05-26T22:50:29.027973+0000","last_became_active":"2024-05-26T22:50:27.618345+0000","last_became_peered":"2024-05-26T22:50:27.618345+0000","last_unstale":"2024-05-26T22:50:29.027973+0000","last_undegraded":"2024-05-26T22:50:29.027973+0000","last_fullsized":"2024-05-26T22:50:29.027973+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:01:47.391816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.606366+0000","last_change":"2024-05-26T22:50:26.584763+0000","last_active":"2024-05-26T22:50:28.606366+0000","last_peered":"2024-05-26T22:50:28.606366+0000","last_clean":"2024-05-26T22:50:28.606366+0000","last_became_active":"2024-05-26T22:50:26.584324+0000","last_became_peered":"2024-05-26T22:50:26.584324+0000","last_unstale":"2024-05-26T22:50:28.606366+0000","last_undegraded":"2024-05-26T22:50:28.606366+0000","last_fullsized":"2024-05-26T22:50:28.606366+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T23:11:53.223115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.606427+0000","last_change":"2024-05-26T22:50:27.608890+0000","last_active":"2024-05-26T22:50:28.606427+0000","last_peered":"2024-05-26T22:50:28.606427+0000","last_clean":"2024-05-26T22:50:28.606427+0000","last_became_active":"2024-05-26T22:50:27.604269+0000","last_became_peered":"2024-05-26T22:50:27.604269+0000","last_unstale":"2024-05-26T22:50:28.606427+0000","last_undegraded":"2024-05-26T22:50:28.606427+0000","last_fullsized":"2024-05-26T22:50:28.606427+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:29:00.413315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:29.028877+0000","last_change":"2024-05-26T22:50:26.589965+0000","last_active":"2024-05-26T22:50:29.028877+0000","last_peered":"2024-05-26T22:50:29.028877+0000","last_clean":"2024-05-26T22:50:29.028877+0000","last_became_active":"2024-05-26T22:50:26.589604+0000","last_became_peered":"2024-05-26T22:50:26.589604+0000","last_unstale":"2024-05-26T22:50:29.028877+0000","last_undegraded":"2024-05-26T22:50:29.028877+0000","last_fullsized":"2024-05-26T22:50:29.028877+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:48:39.224457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.609943+0000","last_change":"2024-05-26T22:50:27.611710+0000","last_active":"2024-05-26T22:50:28.609943+0000","last_peered":"2024-05-26T22:50:28.609943+0000","last_clean":"2024-05-26T22:50:28.609943+0000","last_became_active":"2024-05-26T22:50:27.610934+0000","last_became_peered":"2024-05-26T22:50:27.610934+0000","last_unstale":"2024-05-26T22:50:28.609943+0000","last_undegraded":"2024-05-26T22:50:28.609943+0000","last_fullsized":"2024-05-26T22:50:28.609943+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:33:34.699738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"44'1","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.614426+0000","last_change":"2024-05-26T22:50:26.587049+0000","last_active":"2024-05-26T22:50:28.614426+0000","last_peered":"2024-05-26T22:50:28.614426+0000","last_clean":"2024-05-26T22:50:28.614426+0000","last_became_active":"2024-05-26T22:50:26.586723+0000","last_became_peered":"2024-05-26T22:50:26.586723+0000","last_unstale":"2024-05-26T22:50:28.614426+0000","last_undegraded":"2024-05-26T22:50:28.614426+0000","last_fullsized":"2024-05-26T22:50:28.614426+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:52:43.215817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.607713+0000","last_change":"2024-05-26T22:50:27.605859+0000","last_active":"2024-05-26T22:50:28.607713+0000","last_peered":"2024-05-26T22:50:28.607713+0000","last_clean":"2024-05-26T22:50:28.607713+0000","last_became_active":"2024-05-26T22:50:27.605404+0000","last_became_peered":"2024-05-26T22:50:27.605404+0000","last_unstale":"2024-05-26T22:50:28.607713+0000","last_undegraded":"2024-05-26T22:50:28.607713+0000","last_fullsized":"2024-05-26T22:50:28.607713+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:52:44.229379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:29.031279+0000","last_change":"2024-05-26T22:50:26.591419+0000","last_active":"2024-05-26T22:50:29.031279+0000","last_peered":"2024-05-26T22:50:29.031279+0000","last_clean":"2024-05-26T22:50:29.031279+0000","last_became_active":"2024-05-26T22:50:26.591179+0000","last_became_peered":"2024-05-26T22:50:26.591179+0000","last_unstale":"2024-05-26T22:50:29.031279+0000","last_undegraded":"2024-05-26T22:50:29.031279+0000","last_fullsized":"2024-05-26T22:50:29.031279+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:14:09.430311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:29.031095+0000","last_change":"2024-05-26T22:50:27.615442+0000","last_active":"2024-05-26T22:50:29.031095+0000","last_peered":"2024-05-26T22:50:29.031095+0000","last_clean":"2024-05-26T22:50:29.031095+0000","last_became_active":"2024-05-26T22:50:27.615157+0000","last_became_peered":"2024-05-26T22:50:27.615157+0000","last_unstale":"2024-05-26T22:50:29.031095+0000","last_undegraded":"2024-05-26T22:50:29.031095+0000","last_fullsized":"2024-05-26T22:50:29.031095+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:55:33.647672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.606290+0000","last_change":"2024-05-26T22:50:26.578562+0000","last_active":"2024-05-26T22:50:28.606290+0000","last_peered":"2024-05-26T22:50:28.606290+0000","last_clean":"2024-05-26T22:50:28.606290+0000","last_became_active":"2024-05-26T22:50:26.577480+0000","last_became_peered":"2024-05-26T22:50:26.577480+0000","last_unstale":"2024-05-26T22:50:28.606290+0000","last_undegraded":"2024-05-26T22:50:28.606290+0000","last_fullsized":"2024-05-26T22:50:28.606290+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:07:38.331762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.610258+0000","last_change":"2024-05-26T22:50:27.617767+0000","last_active":"2024-05-26T22:50:28.610258+0000","last_peered":"2024-05-26T22:50:28.610258+0000","last_clean":"2024-05-26T22:50:28.610258+0000","last_became_active":"2024-05-26T22:50:27.613401+0000","last_became_peered":"2024-05-26T22:50:27.613401+0000","last_unstale":"2024-05-26T22:50:28.610258+0000","last_undegraded":"2024-05-26T22:50:28.610258+0000","last_fullsized":"2024-05-26T22:50:28.610258+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:46:46.492897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:29.031017+0000","last_change":"2024-05-26T22:50:26.587128+0000","last_active":"2024-05-26T22:50:29.031017+0000","last_peered":"2024-05-26T22:50:29.031017+0000","last_clean":"2024-05-26T22:50:29.031017+0000","last_became_active":"2024-05-26T22:50:26.586847+0000","last_became_peered":"2024-05-26T22:50:26.586847+0000","last_unstale":"2024-05-26T22:50:29.031017+0000","last_undegraded":"2024-05-26T22:50:29.031017+0000","last_fullsized":"2024-05-26T22:50:29.031017+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:35:15.400981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.610312+0000","last_change":"2024-05-26T22:50:27.625960+0000","last_active":"2024-05-26T22:50:28.610312+0000","last_peered":"2024-05-26T22:50:28.610312+0000","last_clean":"2024-05-26T22:50:28.610312+0000","last_became_active":"2024-05-26T22:50:27.625621+0000","last_became_peered":"2024-05-26T22:50:27.625621+0000","last_unstale":"2024-05-26T22:50:28.610312+0000","last_undegraded":"2024-05-26T22:50:28.610312+0000","last_fullsized":"2024-05-26T22:50:28.610312+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:26:50.858068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.616671+0000","last_change":"2024-05-26T22:50:26.585664+0000","last_active":"2024-05-26T22:50:28.616671+0000","last_peered":"2024-05-26T22:50:28.616671+0000","last_clean":"2024-05-26T22:50:28.616671+0000","last_became_active":"2024-05-26T22:50:26.585056+0000","last_became_peered":"2024-05-26T22:50:26.585056+0000","last_unstale":"2024-05-26T22:50:28.616671+0000","last_undegraded":"2024-05-26T22:50:28.616671+0000","last_fullsized":"2024-05-26T22:50:28.616671+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:07:38.331762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:29.029128+0000","last_change":"2024-05-26T22:50:27.612641+0000","last_active":"2024-05-26T22:50:29.029128+0000","last_peered":"2024-05-26T22:50:29.029128+0000","last_clean":"2024-05-26T22:50:29.029128+0000","last_became_active":"2024-05-26T22:50:27.612332+0000","last_became_peered":"2024-05-26T22:50:27.612332+0000","last_unstale":"2024-05-26T22:50:29.029128+0000","last_undegraded":"2024-05-26T22:50:29.029128+0000","last_fullsized":"2024-05-26T22:50:29.029128+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:44:14.357986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.605807+0000","last_change":"2024-05-26T22:50:26.579162+0000","last_active":"2024-05-26T22:50:28.605807+0000","last_peered":"2024-05-26T22:50:28.605807+0000","last_clean":"2024-05-26T22:50:28.605807+0000","last_became_active":"2024-05-26T22:50:26.578107+0000","last_became_peered":"2024-05-26T22:50:26.578107+0000","last_unstale":"2024-05-26T22:50:28.605807+0000","last_undegraded":"2024-05-26T22:50:28.605807+0000","last_fullsized":"2024-05-26T22:50:28.605807+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:58:28.776394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.616297+0000","last_change":"2024-05-26T22:50:27.619099+0000","last_active":"2024-05-26T22:50:28.616297+0000","last_peered":"2024-05-26T22:50:28.616297+0000","last_clean":"2024-05-26T22:50:28.616297+0000","last_became_active":"2024-05-26T22:50:27.616954+0000","last_became_peered":"2024-05-26T22:50:27.616954+0000","last_unstale":"2024-05-26T22:50:28.616297+0000","last_undegraded":"2024-05-26T22:50:28.616297+0000","last_fullsized":"2024-05-26T22:50:28.616297+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:29:04.881850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.606557+0000","last_change":"2024-05-26T22:50:26.577478+0000","last_active":"2024-05-26T22:50:28.606557+0000","last_peered":"2024-05-26T22:50:28.606557+0000","last_clean":"2024-05-26T22:50:28.606557+0000","last_became_active":"2024-05-26T22:50:26.576506+0000","last_became_peered":"2024-05-26T22:50:26.576506+0000","last_unstale":"2024-05-26T22:50:28.606557+0000","last_undegraded":"2024-05-26T22:50:28.606557+0000","last_fullsized":"2024-05-26T22:50:28.606557+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:07:38.331762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.608750+0000","last_change":"2024-05-26T22:50:27.620262+0000","last_active":"2024-05-26T22:50:28.608750+0000","last_peered":"2024-05-26T22:50:28.608750+0000","last_clean":"2024-05-26T22:50:28.608750+0000","last_became_active":"2024-05-26T22:50:27.619951+0000","last_became_peered":"2024-05-26T22:50:27.619951+0000","last_unstale":"2024-05-26T22:50:28.608750+0000","last_undegraded":"2024-05-26T22:50:28.608750+0000","last_fullsized":"2024-05-26T22:50:28.608750+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:13:34.494539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"43'2","reported_seq":16,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:29.030612+0000","last_change":"2024-05-26T22:50:26.577747+0000","last_active":"2024-05-26T22:50:29.030612+0000","last_peered":"2024-05-26T22:50:29.030612+0000","last_clean":"2024-05-26T22:50:29.030612+0000","last_became_active":"2024-05-26T22:50:26.577408+0000","last_became_peered":"2024-05-26T22:50:26.577408+0000","last_unstale":"2024-05-26T22:50:29.030612+0000","last_undegraded":"2024-05-26T22:50:29.030612+0000","last_fullsized":"2024-05-26T22:50:29.030612+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:08:59.520628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":215,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:29.029658+0000","last_change":"2024-05-26T22:48:12.068917+0000","last_active":"2024-05-26T22:50:29.029658+0000","last_peered":"2024-05-26T22:50:29.029658+0000","last_clean":"2024-05-26T22:50:29.029658+0000","last_became_active":"2024-05-26T22:48:12.065934+0000","last_became_peered":"2024-05-26T22:48:12.065934+0000","last_unstale":"2024-05-26T22:50:29.029658+0000","last_undegraded":"2024-05-26T22:50:29.029658+0000","last_fullsized":"2024-05-26T22:50:29.029658+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:10.628122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:10.628122+0000","last_clean_scrub_stamp":"2024-05-26T22:48:10.628122+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:14:58.895911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574879,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32268,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339572,"statfs":{"total":96636764160,"available":96603721728,"internally_reserved":0,"allocated":1114112,"data_stored":801258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574879,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27532,"kb_used_data":512,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344308,"statfs":{"total":96636764160,"available":96608571392,"internally_reserved":0,"allocated":524288,"data_stored":211472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574880,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27528,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344312,"statfs":{"total":96636764160,"available":96608575488,"internally_reserved":0,"allocated":520192,"data_stored":210870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574879,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27536,"kb_used_data":516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344304,"statfs":{"total":96636764160,"available":96608567296,"internally_reserved":0,"allocated":528384,"data_stored":212412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574879,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27544,"kb_used_data":516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344296,"statfs":{"total":96636764160,"available":96608559104,"internally_reserved":0,"allocated":528384,"data_stored":211438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574879,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27540,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344300,"statfs":{"total":96636764160,"available":96608563200,"internally_reserved":0,"allocated":532480,"data_stored":212434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574879,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27536,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344304,"statfs":{"total":96636764160,"available":96608567296,"internally_reserved":0,"allocated":520192,"data_stored":210870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574879,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32272,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339568,"statfs":{"total":96636764160,"available":96603717632,"internally_reserved":0,"allocated":1114112,"data_stored":801202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-26T22:50:32.964 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-05-26T22:50:33.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.210+0000 7f9f83e00640 1 Processor -- start 2024-05-26T22:50:33.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.211+0000 7f9f83e00640 1 -- start start 2024-05-26T22:50:33.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.212+0000 7f9f83e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f9f7c06ee20 0x7f9f7c0b3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:33.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.212+0000 7f9f83e00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f7c15cb20 con 0x7f9f7c1569d0 2024-05-26T22:50:33.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.212+0000 7f9f83e00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f7c15ce00 con 0x7f9f7c0ad250 2024-05-26T22:50:33.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.212+0000 7f9f83e00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f9f7c15d0e0 con 0x7f9f7c06ee20 2024-05-26T22:50:33.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.213+0000 7f9f81e00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f9f7c1569d0 0x7f9f7c160010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:38502/0 (socket says 172.21.15.131:38502) 2024-05-26T22:50:33.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.213+0000 7f9f81e00640 1 -- 172.21.15.131:0/2618785952 learned_addr learned my addr 172.21.15.131:0/2618785952 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:50:33.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.213+0000 7f9f81400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f9f7c06ee20 0x7f9f7c0b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:33.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.213+0000 7f9f81400640 1 -- 172.21.15.131:0/2618785952 >> v1:172.21.15.187:6789/0 conn(0x7f9f7c0ad250 legacy=0x7f9f7c1562c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:33.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.213+0000 7f9f81400640 1 -- 172.21.15.131:0/2618785952 >> v1:172.21.15.131:6789/0 conn(0x7f9f7c1569d0 legacy=0x7f9f7c160010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:33.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.213+0000 7f9f81400640 1 -- 172.21.15.131:0/2618785952 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f7c15d3c0 con 0x7f9f7c06ee20 2024-05-26T22:50:33.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.213+0000 7f9f7be00640 1 -- 172.21.15.131:0/2618785952 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2789278393 0 0) 0x7f9f7c15cb20 con 0x7f9f7c1569d0 2024-05-26T22:50:33.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.214+0000 7f9f81400640 1 --2- 172.21.15.131:0/2618785952 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f7c06ee20 0x7f9f7c0b3190 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f9f700097a0 tx=0x7f9f7002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee8dbcfe7cb2a3e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:33.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.215+0000 7f9f7be00640 1 -- 172.21.15.131:0/2618785952 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f70037070 con 0x7f9f7c06ee20 2024-05-26T22:50:33.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.215+0000 7f9f7be00640 1 -- 172.21.15.131:0/2618785952 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f7002ae50 con 0x7f9f7c06ee20 2024-05-26T22:50:33.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.215+0000 7f9f7be00640 1 -- 172.21.15.131:0/2618785952 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f70032660 con 0x7f9f7c06ee20 2024-05-26T22:50:33.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.215+0000 7f9f83e00640 1 -- 172.21.15.131:0/2618785952 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f7c06ee20 msgr2=0x7f9f7c0b3190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:33.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.215+0000 7f9f83e00640 1 --2- 172.21.15.131:0/2618785952 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f7c06ee20 0x7f9f7c0b3190 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f9f700097a0 tx=0x7f9f7002a140 comp rx=0 tx=0).stop 2024-05-26T22:50:33.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.215+0000 7f9f83e00640 1 -- 172.21.15.131:0/2618785952 shutdown_connections 2024-05-26T22:50:33.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.215+0000 7f9f83e00640 1 --2- 172.21.15.131:0/2618785952 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f7c06ee20 0x7f9f7c0b3190 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:33.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.215+0000 7f9f83e00640 1 -- 172.21.15.131:0/2618785952 >> 172.21.15.131:0/2618785952 conn(0x7f9f7c0b1e90 msgr2=0x7f9f7c0b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:33.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.215+0000 7f9f83e00640 1 -- 172.21.15.131:0/2618785952 shutdown_connections 2024-05-26T22:50:33.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.215+0000 7f9f83e00640 1 -- 172.21.15.131:0/2618785952 wait complete. 2024-05-26T22:50:33.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.217+0000 7f9f83e00640 1 Processor -- start 2024-05-26T22:50:33.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.217+0000 7f9f83e00640 1 -- start start 2024-05-26T22:50:33.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.218+0000 7f9f83e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f7c06ee20 0x7f9f7c1e7620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:33.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.218+0000 7f9f83e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9f7c0ad250 0x7f9f7c15b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:33.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.218+0000 7f9f83e00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9f7c1569d0 0x7f9f7c151f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:33.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.218+0000 7f9f83e00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f9f7c15c900 con 0x7f9f7c06ee20 2024-05-26T22:50:33.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.218+0000 7f9f83e00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9f7c15c780 con 0x7f9f7c1569d0 2024-05-26T22:50:33.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.218+0000 7f9f83e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9f7c06f200 con 0x7f9f7c0ad250 2024-05-26T22:50:33.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.218+0000 7f9f81400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f7c06ee20 0x7f9f7c1e7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:33.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.218+0000 7f9f81400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f7c06ee20 0x7f9f7c1e7620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50120/0 (socket says 172.21.15.131:50120) 2024-05-26T22:50:33.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.218+0000 7f9f81400640 1 -- 172.21.15.131:0/3145491456 learned_addr learned my addr 172.21.15.131:0/3145491456 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:33.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.219+0000 7f9f81400640 1 -- 172.21.15.131:0/3145491456 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9f7c0ad250 msgr2=0x7f9f7c15b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:33.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.219+0000 7f9f81400640 1 --2- 172.21.15.131:0/3145491456 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9f7c0ad250 0x7f9f7c15b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:33.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.219+0000 7f9f81400640 1 -- 172.21.15.131:0/3145491456 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9f7c1569d0 msgr2=0x7f9f7c151f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:33.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.219+0000 7f9f81400640 1 --2- 172.21.15.131:0/3145491456 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9f7c1569d0 0x7f9f7c151f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:33.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.219+0000 7f9f81400640 1 -- 172.21.15.131:0/3145491456 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f7c1e6580 con 0x7f9f7c06ee20 2024-05-26T22:50:33.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.219+0000 7f9f81400640 1 --2- 172.21.15.131:0/3145491456 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f7c06ee20 0x7f9f7c1e7620 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f9f700097a0 tx=0x7f9f700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:33.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.219+0000 7f9f79600640 1 -- 172.21.15.131:0/3145491456 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f70042070 con 0x7f9f7c06ee20 2024-05-26T22:50:33.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.219+0000 7f9f79600640 1 -- 172.21.15.131:0/3145491456 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f7003e670 con 0x7f9f7c06ee20 2024-05-26T22:50:33.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.219+0000 7f9f79600640 1 -- 172.21.15.131:0/3145491456 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f70037040 con 0x7f9f7c06ee20 2024-05-26T22:50:33.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.219+0000 7f9f83e00640 1 -- 172.21.15.131:0/3145491456 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f7c1e6860 con 0x7f9f7c06ee20 2024-05-26T22:50:33.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.220+0000 7f9f83e00640 1 -- 172.21.15.131:0/3145491456 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f7c1e6e20 con 0x7f9f7c06ee20 2024-05-26T22:50:33.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.221+0000 7f9f83e00640 1 -- 172.21.15.131:0/3145491456 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f7c0694e0 con 0x7f9f7c06ee20 2024-05-26T22:50:33.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.222+0000 7f9f79600640 1 -- 172.21.15.131:0/3145491456 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9f70045070 con 0x7f9f7c06ee20 2024-05-26T22:50:33.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.223+0000 7f9f79600640 1 --2- 172.21.15.131:0/3145491456 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9f5c06be30 0x7f9f5c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:33.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.223+0000 7f9f79600640 1 -- 172.21.15.131:0/3145491456 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f9f700ab780 con 0x7f9f7c06ee20 2024-05-26T22:50:33.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.223+0000 7f9f80a00640 1 --2- 172.21.15.131:0/3145491456 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9f5c06be30 0x7f9f5c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:33.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.224+0000 7f9f80a00640 1 --2- 172.21.15.131:0/3145491456 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9f5c06be30 0x7f9f5c06e2f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c0b1790 tx=0x7f9f680079d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:33.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.227+0000 7f9f79600640 1 -- 172.21.15.131:0/3145491456 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f9f7007a240 con 0x7f9f7c06ee20 2024-05-26T22:50:33.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.502+0000 7f9f83e00640 1 -- 172.21.15.131:0/3145491456 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f9f7c06d100 con 0x7f9f7c06ee20 2024-05-26T22:50:33.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.503+0000 7f9f79600640 1 -- 172.21.15.131:0/3145491456 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1174 (secure 0 0 0) 0x7f9f70079b40 con 0x7f9f7c06ee20 2024-05-26T22:50:33.503 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:50:33.503 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":156,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":44,"num_osds":8,"num_up_osds":8,"osd_up_since":1716763689,"num_in_osds":8,"osd_in_since":1716763680,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":24,"data_bytes":592778,"bytes_used":235270144,"bytes_avail":772858843136,"bytes_total":773094113280,"write_bytes_sec":2388,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":29,"id":3,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":3,"rank":0,"name":"c","status":"up:active","gid":5637}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":4,"modified":"2024-05-26T22:49:18.401449+0000","services":{"mds":{"daemons":{"summary":"","d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-05-26T22:50:33.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.507+0000 7f9f83e00640 1 -- 172.21.15.131:0/3145491456 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9f5c06be30 msgr2=0x7f9f5c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:33.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.507+0000 7f9f83e00640 1 --2- 172.21.15.131:0/3145491456 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9f5c06be30 0x7f9f5c06e2f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c0b1790 tx=0x7f9f680079d0 comp rx=0 tx=0).stop 2024-05-26T22:50:33.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.507+0000 7f9f83e00640 1 -- 172.21.15.131:0/3145491456 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f7c06ee20 msgr2=0x7f9f7c1e7620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:33.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.507+0000 7f9f83e00640 1 --2- 172.21.15.131:0/3145491456 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f7c06ee20 0x7f9f7c1e7620 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f9f700097a0 tx=0x7f9f700046c0 comp rx=0 tx=0).stop 2024-05-26T22:50:33.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.507+0000 7f9f83e00640 1 -- 172.21.15.131:0/3145491456 shutdown_connections 2024-05-26T22:50:33.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.507+0000 7f9f83e00640 1 --2- 172.21.15.131:0/3145491456 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9f7c0ad250 0x7f9f7c15b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:33.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.507+0000 7f9f83e00640 1 --2- 172.21.15.131:0/3145491456 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9f5c06be30 0x7f9f5c06e2f0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:33.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.507+0000 7f9f83e00640 1 --2- 172.21.15.131:0/3145491456 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9f7c1569d0 0x7f9f7c151f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:33.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.507+0000 7f9f83e00640 1 --2- 172.21.15.131:0/3145491456 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f7c06ee20 0x7f9f7c1e7620 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:33.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.507+0000 7f9f83e00640 1 -- 172.21.15.131:0/3145491456 >> 172.21.15.131:0/3145491456 conn(0x7f9f7c0b1e90 msgr2=0x7f9f7c06d470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:33.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.507+0000 7f9f83e00640 1 -- 172.21.15.131:0/3145491456 shutdown_connections 2024-05-26T22:50:33.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.507+0000 7f9f83e00640 1 -- 172.21.15.131:0/3145491456 wait complete. 2024-05-26T22:50:33.524 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-05-26T22:50:33.525 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-26T22:50:33.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.721+0000 7fad93e00640 1 Processor -- start 2024-05-26T22:50:33.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.721+0000 7fad93e00640 1 -- start start 2024-05-26T22:50:33.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.723+0000 7fad93e00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fad9415f4a0 0x7fad940b36b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:33.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.723+0000 7fad93e00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad94162c10 con 0x7fad9406f4a0 2024-05-26T22:50:33.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.723+0000 7fad93e00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad9406eb30 con 0x7fad94067680 2024-05-26T22:50:33.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.723+0000 7fad93e00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fad9406da90 con 0x7fad9415f4a0 2024-05-26T22:50:33.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.723+0000 7fad93400640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fad9406f4a0 0x7fad94162500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:38512/0 (socket says 172.21.15.131:38512) 2024-05-26T22:50:33.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.724+0000 7fad93400640 1 -- 172.21.15.131:0/3960668499 learned_addr learned my addr 172.21.15.131:0/3960668499 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:50:33.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.724+0000 7fad92000640 1 -- 172.21.15.131:0/3960668499 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2467597578 0 0) 0x7fad94162c10 con 0x7fad9406f4a0 2024-05-26T22:50:33.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.724+0000 7fad92000640 1 -- 172.21.15.131:0/3960668499 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fad78003520 con 0x7fad9406f4a0 2024-05-26T22:50:33.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.724+0000 7fad92a00640 1 --2- 172.21.15.131:0/3960668499 >> v2:172.21.15.187:3300/0 conn(0x7fad9415f4a0 0x7fad940b36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:33.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.724+0000 7fad92000640 1 -- 172.21.15.131:0/3960668499 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2402009210 0 0) 0x7fad78003520 con 0x7fad9406f4a0 2024-05-26T22:50:33.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.724+0000 7fad92000640 1 -- 172.21.15.131:0/3960668499 >> v2:172.21.15.187:3300/0 conn(0x7fad9415f4a0 msgr2=0x7fad940b36b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:33.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.724+0000 7fad92000640 1 --2- 172.21.15.131:0/3960668499 >> v2:172.21.15.187:3300/0 conn(0x7fad9415f4a0 0x7fad940b36b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:33.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.724+0000 7fad92000640 1 -- 172.21.15.131:0/3960668499 >> v1:172.21.15.187:6789/0 conn(0x7fad94067680 legacy=0x7fad94161df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:33.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.724+0000 7fad92000640 1 -- 172.21.15.131:0/3960668499 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad9406dd70 con 0x7fad9406f4a0 2024-05-26T22:50:33.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.724+0000 7fad92000640 1 -- 172.21.15.131:0/3960668499 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7fad8c0019f0 con 0x7fad9406f4a0 2024-05-26T22:50:33.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.724+0000 7fad92000640 1 -- 172.21.15.131:0/3960668499 >> v1:172.21.15.131:6789/0 conn(0x7fad9406f4a0 legacy=0x7fad94162500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:33.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.724+0000 7fad92000640 1 --2- 172.21.15.131:0/3960668499 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fad780039b0 0x7fad78003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:33.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.724+0000 7fad92000640 1 --2- 172.21.15.131:0/3960668499 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fad78004800 0x7fad78004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:33.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.724+0000 7fad92000640 1 --2- 172.21.15.131:0/3960668499 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad780051c0 0x7fad7800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:33.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.725+0000 7fad92000640 1 -- 172.21.15.131:0/3960668499 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fad9406da90 con 0x7fad780051c0 2024-05-26T22:50:33.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.725+0000 7fad92000640 1 -- 172.21.15.131:0/3960668499 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fad7800c0f0 con 0x7fad78004800 2024-05-26T22:50:33.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.725+0000 7fad92000640 1 -- 172.21.15.131:0/3960668499 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fad7800c2f0 con 0x7fad780039b0 2024-05-26T22:50:33.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.725+0000 7fad93400640 1 --2- 172.21.15.131:0/3960668499 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad780051c0 0x7fad7800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:33.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.725+0000 7fad92a00640 1 --2- 172.21.15.131:0/3960668499 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fad780039b0 0x7fad78003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:33.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.725+0000 7fad8a000640 1 --2- 172.21.15.131:0/3960668499 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fad78004800 0x7fad78004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:33.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.725+0000 7fad93400640 1 -- 172.21.15.131:0/3960668499 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fad780039b0 msgr2=0x7fad78003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:33.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.725+0000 7fad93400640 1 --2- 172.21.15.131:0/3960668499 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fad780039b0 0x7fad78003db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:33.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.725+0000 7fad93400640 1 -- 172.21.15.131:0/3960668499 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fad78004800 msgr2=0x7fad78004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:33.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.725+0000 7fad93400640 1 --2- 172.21.15.131:0/3960668499 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fad78004800 0x7fad78004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:33.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.725+0000 7fad93400640 1 -- 172.21.15.131:0/3960668499 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad78003520 con 0x7fad780051c0 2024-05-26T22:50:33.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.725+0000 7fad92a00640 1 --2- 172.21.15.131:0/3960668499 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fad780039b0 0x7fad78003db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:50:33.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.726+0000 7fad93400640 1 -- 172.21.15.131:0/3960668499 reap_dead start 2024-05-26T22:50:33.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.726+0000 7fad93400640 1 --2- 172.21.15.131:0/3960668499 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad780051c0 0x7fad7800b9c0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fad8c00a8f0 tx=0x7fad78005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af824cd768ff3e03 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:33.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.726+0000 7fad92000640 1 -- 172.21.15.131:0/3960668499 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad8c00a4c0 con 0x7fad780051c0 2024-05-26T22:50:33.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.727+0000 7fad92000640 1 -- 172.21.15.131:0/3960668499 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad8c02f070 con 0x7fad780051c0 2024-05-26T22:50:33.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.727+0000 7fad92000640 1 -- 172.21.15.131:0/3960668499 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad8c02bcb0 con 0x7fad780051c0 2024-05-26T22:50:33.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.727+0000 7fad93e00640 1 -- 172.21.15.131:0/3960668499 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad780051c0 msgr2=0x7fad7800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:33.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.727+0000 7fad93e00640 1 --2- 172.21.15.131:0/3960668499 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad780051c0 0x7fad7800b9c0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fad8c00a8f0 tx=0x7fad78005830 comp rx=0 tx=0).stop 2024-05-26T22:50:33.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.727+0000 7fad93e00640 1 -- 172.21.15.131:0/3960668499 shutdown_connections 2024-05-26T22:50:33.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.727+0000 7fad93e00640 1 --2- 172.21.15.131:0/3960668499 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad780051c0 0x7fad7800b9c0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:33.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.727+0000 7fad93e00640 1 -- 172.21.15.131:0/3960668499 >> 172.21.15.131:0/3960668499 conn(0x7fad940d0600 msgr2=0x7fad940d09e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:33.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.727+0000 7fad93e00640 1 -- 172.21.15.131:0/3960668499 shutdown_connections 2024-05-26T22:50:33.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.727+0000 7fad93e00640 1 -- 172.21.15.131:0/3960668499 wait complete. 2024-05-26T22:50:33.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.729+0000 7fad93e00640 1 Processor -- start 2024-05-26T22:50:33.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.729+0000 7fad93e00640 1 -- start start 2024-05-26T22:50:33.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.730+0000 7fad93e00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fad780051c0 0x7fad9414d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:33.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.730+0000 7fad93e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad94149b30 0x7fad940cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:33.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.730+0000 7fad93e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fad94146b40 0x7fad94146ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:33.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.730+0000 7fad93e00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fad9406e610 con 0x7fad94149b30 2024-05-26T22:50:33.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.730+0000 7fad93e00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fad9406e330 con 0x7fad780051c0 2024-05-26T22:50:33.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.730+0000 7fad93e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fad9415e800 con 0x7fad94146b40 2024-05-26T22:50:33.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.730+0000 7fad8a000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad94149b30 0x7fad940cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:33.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.730+0000 7fad8a000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad94149b30 0x7fad940cf5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50144/0 (socket says 172.21.15.131:50144) 2024-05-26T22:50:33.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.730+0000 7fad8a000640 1 -- 172.21.15.131:0/536379269 learned_addr learned my addr 172.21.15.131:0/536379269 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:33.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.731+0000 7fad92a00640 1 --2- 172.21.15.131:0/536379269 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fad780051c0 0x7fad9414d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:33.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.731+0000 7fad93400640 1 --2- 172.21.15.131:0/536379269 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fad94146b40 0x7fad94146ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:33.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.731+0000 7fad8a000640 1 -- 172.21.15.131:0/536379269 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fad94146b40 msgr2=0x7fad94146ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:33.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.731+0000 7fad8a000640 1 --2- 172.21.15.131:0/536379269 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fad94146b40 0x7fad94146ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:33.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.731+0000 7fad8a000640 1 -- 172.21.15.131:0/536379269 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fad780051c0 msgr2=0x7fad9414d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:33.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.731+0000 7fad8a000640 1 --2- 172.21.15.131:0/536379269 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fad780051c0 0x7fad9414d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:33.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.731+0000 7fad8a000640 1 -- 172.21.15.131:0/536379269 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad940cd8e0 con 0x7fad94149b30 2024-05-26T22:50:33.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.731+0000 7fad8a000640 1 --2- 172.21.15.131:0/536379269 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad94149b30 0x7fad940cf5f0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fad7c00d4e0 tx=0x7fad7c00d9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:33.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.731+0000 7fad8be00640 1 -- 172.21.15.131:0/536379269 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad7c016070 con 0x7fad94149b30 2024-05-26T22:50:33.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.731+0000 7fad8be00640 1 -- 172.21.15.131:0/536379269 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad7c012800 con 0x7fad94149b30 2024-05-26T22:50:33.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.731+0000 7fad93e00640 1 -- 172.21.15.131:0/536379269 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad940cdbc0 con 0x7fad94149b30 2024-05-26T22:50:33.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.732+0000 7fad8be00640 1 -- 172.21.15.131:0/536379269 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad7c012c30 con 0x7fad94149b30 2024-05-26T22:50:33.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.732+0000 7fad93e00640 1 -- 172.21.15.131:0/536379269 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad940cfb30 con 0x7fad94149b30 2024-05-26T22:50:33.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.733+0000 7fad93e00640 1 -- 172.21.15.131:0/536379269 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad940694e0 con 0x7fad94149b30 2024-05-26T22:50:33.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.734+0000 7fad8be00640 1 -- 172.21.15.131:0/536379269 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fad7c019070 con 0x7fad94149b30 2024-05-26T22:50:33.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.734+0000 7fad8be00640 1 --2- 172.21.15.131:0/536379269 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fad7006be30 0x7fad7006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:33.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.735+0000 7fad8be00640 1 -- 172.21.15.131:0/536379269 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7fad7c08ad70 con 0x7fad94149b30 2024-05-26T22:50:33.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.735+0000 7fad92a00640 1 --2- 172.21.15.131:0/536379269 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fad7006be30 0x7fad7006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:33.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.735+0000 7fad92a00640 1 --2- 172.21.15.131:0/536379269 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fad7006be30 0x7fad7006e2f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fad84006fd0 tx=0x7fad84008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:33.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.739+0000 7fad8be00640 1 -- 172.21.15.131:0/536379269 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fad7c059710 con 0x7fad94149b30 2024-05-26T22:50:33.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.919+0000 7fad93e00640 1 -- 172.21.15.131:0/536379269 --> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fad940ccb50 con 0x7fad7006be30 2024-05-26T22:50:33.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.927+0000 7fad8be00640 1 -- 172.21.15.131:0/536379269 <== mgr.4104 v2:172.21.15.131:6832/3484832947 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251959 (secure 0 0 0) 0x7fad940ccb50 con 0x7fad7006be30 2024-05-26T22:50:33.928 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:50:33.931 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-05-26T22:50:33.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.932+0000 7fad93e00640 1 -- 172.21.15.131:0/536379269 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fad7006be30 msgr2=0x7fad7006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:33.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.932+0000 7fad93e00640 1 --2- 172.21.15.131:0/536379269 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fad7006be30 0x7fad7006e2f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fad84006fd0 tx=0x7fad84008040 comp rx=0 tx=0).stop 2024-05-26T22:50:33.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.932+0000 7fad93e00640 1 -- 172.21.15.131:0/536379269 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad94149b30 msgr2=0x7fad940cf5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:33.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.932+0000 7fad93e00640 1 --2- 172.21.15.131:0/536379269 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad94149b30 0x7fad940cf5f0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fad7c00d4e0 tx=0x7fad7c00d9a0 comp rx=0 tx=0).stop 2024-05-26T22:50:33.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.932+0000 7fad93e00640 1 -- 172.21.15.131:0/536379269 shutdown_connections 2024-05-26T22:50:33.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.932+0000 7fad93e00640 1 --2- 172.21.15.131:0/536379269 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fad94146b40 0x7fad94146ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:33.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.932+0000 7fad93e00640 1 --2- 172.21.15.131:0/536379269 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fad7006be30 0x7fad7006e2f0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:33.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.932+0000 7fad93e00640 1 --2- 172.21.15.131:0/536379269 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad94149b30 0x7fad940cf5f0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:33.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.932+0000 7fad93e00640 1 --2- 172.21.15.131:0/536379269 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fad780051c0 0x7fad9414d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:33.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.933+0000 7fad93e00640 1 -- 172.21.15.131:0/536379269 >> 172.21.15.131:0/536379269 conn(0x7fad940d0600 msgr2=0x7fad940b1270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:33.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.933+0000 7fad93e00640 1 -- 172.21.15.131:0/536379269 shutdown_connections 2024-05-26T22:50:33.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:33.933+0000 7fad93e00640 1 -- 172.21.15.131:0/536379269 wait complete. 2024-05-26T22:50:33.951 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":108,"stamp":"2024-05-26T22:50:32.426341+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":149,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":229756,"kb_used_data":5256,"kb_used_omap":12,"kb_used_meta":224371,"kb_avail":754744964,"statfs":{"total":773094113280,"available":772858843136,"internally_reserved":0,"allocated":5382144,"data_stored":2871956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":229756465},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":42,"apply_latency_ms":42,"commit_latency_ns":42000000,"apply_latency_ns":42000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.003203"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:29.031378+0000","last_change":"2024-05-26T22:50:27.610657+0000","last_active":"2024-05-26T22:50:29.031378+0000","last_peered":"2024-05-26T22:50:29.031378+0000","last_clean":"2024-05-26T22:50:29.031378+0000","last_became_active":"2024-05-26T22:50:27.610367+0000","last_became_peered":"2024-05-26T22:50:27.610367+0000","last_unstale":"2024-05-26T22:50:29.031378+0000","last_undegraded":"2024-05-26T22:50:29.031378+0000","last_fullsized":"2024-05-26T22:50:29.031378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:44:57.492554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:29.030017+0000","last_change":"2024-05-26T22:50:27.621789+0000","last_active":"2024-05-26T22:50:29.030017+0000","last_peered":"2024-05-26T22:50:29.030017+0000","last_clean":"2024-05-26T22:50:29.030017+0000","last_became_active":"2024-05-26T22:50:27.621501+0000","last_became_peered":"2024-05-26T22:50:27.621501+0000","last_unstale":"2024-05-26T22:50:29.030017+0000","last_undegraded":"2024-05-26T22:50:29.030017+0000","last_fullsized":"2024-05-26T22:50:29.030017+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:15:23.061792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.617060+0000","last_change":"2024-05-26T22:50:27.602597+0000","last_active":"2024-05-26T22:50:28.617060+0000","last_peered":"2024-05-26T22:50:28.617060+0000","last_clean":"2024-05-26T22:50:28.617060+0000","last_became_active":"2024-05-26T22:50:27.602270+0000","last_became_peered":"2024-05-26T22:50:27.602270+0000","last_unstale":"2024-05-26T22:50:28.617060+0000","last_undegraded":"2024-05-26T22:50:28.617060+0000","last_fullsized":"2024-05-26T22:50:28.617060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:48:40.238018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.607638+0000","last_change":"2024-05-26T22:50:27.606124+0000","last_active":"2024-05-26T22:50:28.607638+0000","last_peered":"2024-05-26T22:50:28.607638+0000","last_clean":"2024-05-26T22:50:28.607638+0000","last_became_active":"2024-05-26T22:50:27.605799+0000","last_became_peered":"2024-05-26T22:50:27.605799+0000","last_unstale":"2024-05-26T22:50:28.607638+0000","last_undegraded":"2024-05-26T22:50:28.607638+0000","last_fullsized":"2024-05-26T22:50:28.607638+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:40:21.264809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:29.029641+0000","last_change":"2024-05-26T22:50:27.622010+0000","last_active":"2024-05-26T22:50:29.029641+0000","last_peered":"2024-05-26T22:50:29.029641+0000","last_clean":"2024-05-26T22:50:29.029641+0000","last_became_active":"2024-05-26T22:50:27.621491+0000","last_became_peered":"2024-05-26T22:50:27.621491+0000","last_unstale":"2024-05-26T22:50:29.029641+0000","last_undegraded":"2024-05-26T22:50:29.029641+0000","last_fullsized":"2024-05-26T22:50:29.029641+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:58:06.516711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.610098+0000","last_change":"2024-05-26T22:50:27.619003+0000","last_active":"2024-05-26T22:50:28.610098+0000","last_peered":"2024-05-26T22:50:28.610098+0000","last_clean":"2024-05-26T22:50:28.610098+0000","last_became_active":"2024-05-26T22:50:27.613040+0000","last_became_peered":"2024-05-26T22:50:27.613040+0000","last_unstale":"2024-05-26T22:50:28.610098+0000","last_undegraded":"2024-05-26T22:50:28.610098+0000","last_fullsized":"2024-05-26T22:50:28.610098+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:21:37.744153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:29.030212+0000","last_change":"2024-05-26T22:50:27.612767+0000","last_active":"2024-05-26T22:50:29.030212+0000","last_peered":"2024-05-26T22:50:29.030212+0000","last_clean":"2024-05-26T22:50:29.030212+0000","last_became_active":"2024-05-26T22:50:27.612139+0000","last_became_peered":"2024-05-26T22:50:27.612139+0000","last_unstale":"2024-05-26T22:50:29.030212+0000","last_undegraded":"2024-05-26T22:50:29.030212+0000","last_fullsized":"2024-05-26T22:50:29.030212+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:34:05.951317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.606233+0000","last_change":"2024-05-26T22:50:27.593124+0000","last_active":"2024-05-26T22:50:28.606233+0000","last_peered":"2024-05-26T22:50:28.606233+0000","last_clean":"2024-05-26T22:50:28.606233+0000","last_became_active":"2024-05-26T22:50:27.591829+0000","last_became_peered":"2024-05-26T22:50:27.591829+0000","last_unstale":"2024-05-26T22:50:28.606233+0000","last_undegraded":"2024-05-26T22:50:28.606233+0000","last_fullsized":"2024-05-26T22:50:28.606233+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:52:28.184259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.608451+0000","last_change":"2024-05-26T22:50:27.609384+0000","last_active":"2024-05-26T22:50:28.608451+0000","last_peered":"2024-05-26T22:50:28.608451+0000","last_clean":"2024-05-26T22:50:28.608451+0000","last_became_active":"2024-05-26T22:50:27.604237+0000","last_became_peered":"2024-05-26T22:50:27.604237+0000","last_unstale":"2024-05-26T22:50:28.608451+0000","last_undegraded":"2024-05-26T22:50:28.608451+0000","last_fullsized":"2024-05-26T22:50:28.608451+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:15:23.061792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.608544+0000","last_change":"2024-05-26T22:50:27.593515+0000","last_active":"2024-05-26T22:50:28.608544+0000","last_peered":"2024-05-26T22:50:28.608544+0000","last_clean":"2024-05-26T22:50:28.608544+0000","last_became_active":"2024-05-26T22:50:27.592190+0000","last_became_peered":"2024-05-26T22:50:27.592190+0000","last_unstale":"2024-05-26T22:50:28.608544+0000","last_undegraded":"2024-05-26T22:50:28.608544+0000","last_fullsized":"2024-05-26T22:50:28.608544+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:55:33.647672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.608476+0000","last_change":"2024-05-26T22:50:27.605243+0000","last_active":"2024-05-26T22:50:28.608476+0000","last_peered":"2024-05-26T22:50:28.608476+0000","last_clean":"2024-05-26T22:50:28.608476+0000","last_became_active":"2024-05-26T22:50:27.604931+0000","last_became_peered":"2024-05-26T22:50:27.604931+0000","last_unstale":"2024-05-26T22:50:28.608476+0000","last_undegraded":"2024-05-26T22:50:28.608476+0000","last_fullsized":"2024-05-26T22:50:28.608476+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:21:16.701772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:29.029545+0000","last_change":"2024-05-26T22:50:27.627122+0000","last_active":"2024-05-26T22:50:29.029545+0000","last_peered":"2024-05-26T22:50:29.029545+0000","last_clean":"2024-05-26T22:50:29.029545+0000","last_became_active":"2024-05-26T22:50:27.626847+0000","last_became_peered":"2024-05-26T22:50:27.626847+0000","last_unstale":"2024-05-26T22:50:29.029545+0000","last_undegraded":"2024-05-26T22:50:29.029545+0000","last_fullsized":"2024-05-26T22:50:29.029545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:40:21.264809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:29.028177+0000","last_change":"2024-05-26T22:50:27.616958+0000","last_active":"2024-05-26T22:50:29.028177+0000","last_peered":"2024-05-26T22:50:29.028177+0000","last_clean":"2024-05-26T22:50:29.028177+0000","last_became_active":"2024-05-26T22:50:27.616551+0000","last_became_peered":"2024-05-26T22:50:27.616551+0000","last_unstale":"2024-05-26T22:50:29.028177+0000","last_undegraded":"2024-05-26T22:50:29.028177+0000","last_fullsized":"2024-05-26T22:50:29.028177+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:14:28.234483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.607145+0000","last_change":"2024-05-26T22:50:27.608517+0000","last_active":"2024-05-26T22:50:28.607145+0000","last_peered":"2024-05-26T22:50:28.607145+0000","last_clean":"2024-05-26T22:50:28.607145+0000","last_became_active":"2024-05-26T22:50:27.606950+0000","last_became_peered":"2024-05-26T22:50:27.606950+0000","last_unstale":"2024-05-26T22:50:28.607145+0000","last_undegraded":"2024-05-26T22:50:28.607145+0000","last_fullsized":"2024-05-26T22:50:28.607145+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:28:42.535219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.609790+0000","last_change":"2024-05-26T22:50:27.608959+0000","last_active":"2024-05-26T22:50:28.609790+0000","last_peered":"2024-05-26T22:50:28.609790+0000","last_clean":"2024-05-26T22:50:28.609790+0000","last_became_active":"2024-05-26T22:50:27.608700+0000","last_became_peered":"2024-05-26T22:50:27.608700+0000","last_unstale":"2024-05-26T22:50:28.609790+0000","last_undegraded":"2024-05-26T22:50:28.609790+0000","last_fullsized":"2024-05-26T22:50:28.609790+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:12:46.452654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.608690+0000","last_change":"2024-05-26T22:50:27.611336+0000","last_active":"2024-05-26T22:50:28.608690+0000","last_peered":"2024-05-26T22:50:28.608690+0000","last_clean":"2024-05-26T22:50:28.608690+0000","last_became_active":"2024-05-26T22:50:27.611063+0000","last_became_peered":"2024-05-26T22:50:27.611063+0000","last_unstale":"2024-05-26T22:50:28.608690+0000","last_undegraded":"2024-05-26T22:50:28.608690+0000","last_fullsized":"2024-05-26T22:50:28.608690+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:53:36.611294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.617634+0000","last_change":"2024-05-26T22:50:27.611141+0000","last_active":"2024-05-26T22:50:28.617634+0000","last_peered":"2024-05-26T22:50:28.617634+0000","last_clean":"2024-05-26T22:50:28.617634+0000","last_became_active":"2024-05-26T22:50:27.610706+0000","last_became_peered":"2024-05-26T22:50:27.610706+0000","last_unstale":"2024-05-26T22:50:28.617634+0000","last_undegraded":"2024-05-26T22:50:28.617634+0000","last_fullsized":"2024-05-26T22:50:28.617634+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:29:00.413315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.617367+0000","last_change":"2024-05-26T22:50:27.625629+0000","last_active":"2024-05-26T22:50:28.617367+0000","last_peered":"2024-05-26T22:50:28.617367+0000","last_clean":"2024-05-26T22:50:28.617367+0000","last_became_active":"2024-05-26T22:50:27.625326+0000","last_became_peered":"2024-05-26T22:50:27.625326+0000","last_unstale":"2024-05-26T22:50:28.617367+0000","last_undegraded":"2024-05-26T22:50:28.617367+0000","last_fullsized":"2024-05-26T22:50:28.617367+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:52:28.184259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.610528+0000","last_change":"2024-05-26T22:50:27.612043+0000","last_active":"2024-05-26T22:50:28.610528+0000","last_peered":"2024-05-26T22:50:28.610528+0000","last_clean":"2024-05-26T22:50:28.610528+0000","last_became_active":"2024-05-26T22:50:27.611759+0000","last_became_peered":"2024-05-26T22:50:27.611759+0000","last_unstale":"2024-05-26T22:50:28.610528+0000","last_undegraded":"2024-05-26T22:50:28.610528+0000","last_fullsized":"2024-05-26T22:50:28.610528+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:54:47.587339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:29.028648+0000","last_change":"2024-05-26T22:50:27.620986+0000","last_active":"2024-05-26T22:50:29.028648+0000","last_peered":"2024-05-26T22:50:29.028648+0000","last_clean":"2024-05-26T22:50:29.028648+0000","last_became_active":"2024-05-26T22:50:27.620701+0000","last_became_peered":"2024-05-26T22:50:27.620701+0000","last_unstale":"2024-05-26T22:50:29.028648+0000","last_undegraded":"2024-05-26T22:50:29.028648+0000","last_fullsized":"2024-05-26T22:50:29.028648+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:13:05.372755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.610175+0000","last_change":"2024-05-26T22:50:27.623707+0000","last_active":"2024-05-26T22:50:28.610175+0000","last_peered":"2024-05-26T22:50:28.610175+0000","last_clean":"2024-05-26T22:50:28.610175+0000","last_became_active":"2024-05-26T22:50:27.623450+0000","last_became_peered":"2024-05-26T22:50:27.623450+0000","last_unstale":"2024-05-26T22:50:28.610175+0000","last_undegraded":"2024-05-26T22:50:28.610175+0000","last_fullsized":"2024-05-26T22:50:28.610175+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:21:39.447850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.607328+0000","last_change":"2024-05-26T22:50:27.595025+0000","last_active":"2024-05-26T22:50:28.607328+0000","last_peered":"2024-05-26T22:50:28.607328+0000","last_clean":"2024-05-26T22:50:28.607328+0000","last_became_active":"2024-05-26T22:50:27.594473+0000","last_became_peered":"2024-05-26T22:50:27.594473+0000","last_unstale":"2024-05-26T22:50:28.607328+0000","last_undegraded":"2024-05-26T22:50:28.607328+0000","last_fullsized":"2024-05-26T22:50:28.607328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:44:14.357986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.609269+0000","last_change":"2024-05-26T22:50:27.609501+0000","last_active":"2024-05-26T22:50:28.609269+0000","last_peered":"2024-05-26T22:50:28.609269+0000","last_clean":"2024-05-26T22:50:28.609269+0000","last_became_active":"2024-05-26T22:50:27.608925+0000","last_became_peered":"2024-05-26T22:50:27.608925+0000","last_unstale":"2024-05-26T22:50:28.609269+0000","last_undegraded":"2024-05-26T22:50:28.609269+0000","last_fullsized":"2024-05-26T22:50:28.609269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:58:29.789956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.608671+0000","last_change":"2024-05-26T22:50:27.606676+0000","last_active":"2024-05-26T22:50:28.608671+0000","last_peered":"2024-05-26T22:50:28.608671+0000","last_clean":"2024-05-26T22:50:28.608671+0000","last_became_active":"2024-05-26T22:50:27.606399+0000","last_became_peered":"2024-05-26T22:50:27.606399+0000","last_unstale":"2024-05-26T22:50:28.608671+0000","last_undegraded":"2024-05-26T22:50:28.608671+0000","last_fullsized":"2024-05-26T22:50:28.608671+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:07:39.345324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.609416+0000","last_change":"2024-05-26T22:50:27.603472+0000","last_active":"2024-05-26T22:50:28.609416+0000","last_peered":"2024-05-26T22:50:28.609416+0000","last_clean":"2024-05-26T22:50:28.609416+0000","last_became_active":"2024-05-26T22:50:27.602661+0000","last_became_peered":"2024-05-26T22:50:27.602661+0000","last_unstale":"2024-05-26T22:50:28.609416+0000","last_undegraded":"2024-05-26T22:50:28.609416+0000","last_fullsized":"2024-05-26T22:50:28.609416+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:26:43.963641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:29.028316+0000","last_change":"2024-05-26T22:50:27.621771+0000","last_active":"2024-05-26T22:50:29.028316+0000","last_peered":"2024-05-26T22:50:29.028316+0000","last_clean":"2024-05-26T22:50:29.028316+0000","last_became_active":"2024-05-26T22:50:27.621425+0000","last_became_peered":"2024-05-26T22:50:27.621425+0000","last_unstale":"2024-05-26T22:50:29.028316+0000","last_undegraded":"2024-05-26T22:50:29.028316+0000","last_fullsized":"2024-05-26T22:50:29.028316+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:04:18.699067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.616839+0000","last_change":"2024-05-26T22:50:27.607252+0000","last_active":"2024-05-26T22:50:28.616839+0000","last_peered":"2024-05-26T22:50:28.616839+0000","last_clean":"2024-05-26T22:50:28.616839+0000","last_became_active":"2024-05-26T22:50:27.606329+0000","last_became_peered":"2024-05-26T22:50:27.606329+0000","last_unstale":"2024-05-26T22:50:28.616839+0000","last_undegraded":"2024-05-26T22:50:28.616839+0000","last_fullsized":"2024-05-26T22:50:28.616839+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:14:10.443873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:29.029593+0000","last_change":"2024-05-26T22:50:27.626991+0000","last_active":"2024-05-26T22:50:29.029593+0000","last_peered":"2024-05-26T22:50:29.029593+0000","last_clean":"2024-05-26T22:50:29.029593+0000","last_became_active":"2024-05-26T22:50:27.626702+0000","last_became_peered":"2024-05-26T22:50:27.626702+0000","last_unstale":"2024-05-26T22:50:29.029593+0000","last_undegraded":"2024-05-26T22:50:29.029593+0000","last_fullsized":"2024-05-26T22:50:29.029593+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:52:44.229379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.609538+0000","last_change":"2024-05-26T22:50:27.602948+0000","last_active":"2024-05-26T22:50:28.609538+0000","last_peered":"2024-05-26T22:50:28.609538+0000","last_clean":"2024-05-26T22:50:28.609538+0000","last_became_active":"2024-05-26T22:50:27.602641+0000","last_became_peered":"2024-05-26T22:50:27.602641+0000","last_unstale":"2024-05-26T22:50:28.609538+0000","last_undegraded":"2024-05-26T22:50:28.609538+0000","last_fullsized":"2024-05-26T22:50:28.609538+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:28:13.522438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.616295+0000","last_change":"2024-05-26T22:50:27.613332+0000","last_active":"2024-05-26T22:50:28.616295+0000","last_peered":"2024-05-26T22:50:28.616295+0000","last_clean":"2024-05-26T22:50:28.616295+0000","last_became_active":"2024-05-26T22:50:27.613002+0000","last_became_peered":"2024-05-26T22:50:27.613002+0000","last_unstale":"2024-05-26T22:50:28.616295+0000","last_undegraded":"2024-05-26T22:50:28.616295+0000","last_fullsized":"2024-05-26T22:50:28.616295+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:35:16.414543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.609812+0000","last_change":"2024-05-26T22:50:27.609655+0000","last_active":"2024-05-26T22:50:28.609812+0000","last_peered":"2024-05-26T22:50:28.609812+0000","last_clean":"2024-05-26T22:50:28.609812+0000","last_became_active":"2024-05-26T22:50:27.609367+0000","last_became_peered":"2024-05-26T22:50:27.609367+0000","last_unstale":"2024-05-26T22:50:28.609812+0000","last_undegraded":"2024-05-26T22:50:28.609812+0000","last_fullsized":"2024-05-26T22:50:28.609812+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:48:07.563526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:29.027930+0000","last_change":"2024-05-26T22:50:27.613465+0000","last_active":"2024-05-26T22:50:29.027930+0000","last_peered":"2024-05-26T22:50:29.027930+0000","last_clean":"2024-05-26T22:50:29.027930+0000","last_became_active":"2024-05-26T22:50:27.613133+0000","last_became_peered":"2024-05-26T22:50:27.613133+0000","last_unstale":"2024-05-26T22:50:29.027930+0000","last_undegraded":"2024-05-26T22:50:29.027930+0000","last_fullsized":"2024-05-26T22:50:29.027930+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:28:32.032578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.608109+0000","last_change":"2024-05-26T22:50:27.625540+0000","last_active":"2024-05-26T22:50:28.608109+0000","last_peered":"2024-05-26T22:50:28.608109+0000","last_clean":"2024-05-26T22:50:28.608109+0000","last_became_active":"2024-05-26T22:50:27.624727+0000","last_became_peered":"2024-05-26T22:50:27.624727+0000","last_unstale":"2024-05-26T22:50:28.608109+0000","last_undegraded":"2024-05-26T22:50:28.608109+0000","last_fullsized":"2024-05-26T22:50:28.608109+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:57:10.223342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"43'4","reported_seq":18,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.611114+0000","last_change":"2024-05-26T22:50:26.581794+0000","last_active":"2024-05-26T22:50:28.611114+0000","last_peered":"2024-05-26T22:50:28.611114+0000","last_clean":"2024-05-26T22:50:28.611114+0000","last_became_active":"2024-05-26T22:50:26.581504+0000","last_became_peered":"2024-05-26T22:50:26.581504+0000","last_unstale":"2024-05-26T22:50:28.611114+0000","last_undegraded":"2024-05-26T22:50:28.611114+0000","last_fullsized":"2024-05-26T22:50:28.611114+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:44:13.344424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.609437+0000","last_change":"2024-05-26T22:50:27.607318+0000","last_active":"2024-05-26T22:50:28.609437+0000","last_peered":"2024-05-26T22:50:28.609437+0000","last_clean":"2024-05-26T22:50:28.609437+0000","last_became_active":"2024-05-26T22:50:27.607013+0000","last_became_peered":"2024-05-26T22:50:27.607013+0000","last_unstale":"2024-05-26T22:50:28.609437+0000","last_undegraded":"2024-05-26T22:50:28.609437+0000","last_fullsized":"2024-05-26T22:50:28.609437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:23:58.424666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"43'1","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.617636+0000","last_change":"2024-05-26T22:50:26.582366+0000","last_active":"2024-05-26T22:50:28.617636+0000","last_peered":"2024-05-26T22:50:28.617636+0000","last_clean":"2024-05-26T22:50:28.617636+0000","last_became_active":"2024-05-26T22:50:26.582022+0000","last_became_peered":"2024-05-26T22:50:26.582022+0000","last_unstale":"2024-05-26T22:50:28.617636+0000","last_undegraded":"2024-05-26T22:50:28.617636+0000","last_fullsized":"2024-05-26T22:50:28.617636+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:58:28.776394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:29.029938+0000","last_change":"2024-05-26T22:50:27.622401+0000","last_active":"2024-05-26T22:50:29.029938+0000","last_peered":"2024-05-26T22:50:29.029938+0000","last_clean":"2024-05-26T22:50:29.029938+0000","last_became_active":"2024-05-26T22:50:27.622072+0000","last_became_peered":"2024-05-26T22:50:27.622072+0000","last_unstale":"2024-05-26T22:50:29.029938+0000","last_undegraded":"2024-05-26T22:50:29.029938+0000","last_fullsized":"2024-05-26T22:50:29.029938+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:34:05.951317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"43'2","reported_seq":16,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.607043+0000","last_change":"2024-05-26T22:50:26.579628+0000","last_active":"2024-05-26T22:50:28.607043+0000","last_peered":"2024-05-26T22:50:28.607043+0000","last_clean":"2024-05-26T22:50:28.607043+0000","last_became_active":"2024-05-26T22:50:26.579316+0000","last_became_peered":"2024-05-26T22:50:26.579316+0000","last_unstale":"2024-05-26T22:50:28.607043+0000","last_undegraded":"2024-05-26T22:50:28.607043+0000","last_fullsized":"2024-05-26T22:50:28.607043+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:53:35.597732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.607130+0000","last_change":"2024-05-26T22:50:27.609117+0000","last_active":"2024-05-26T22:50:28.607130+0000","last_peered":"2024-05-26T22:50:28.607130+0000","last_clean":"2024-05-26T22:50:28.607130+0000","last_became_active":"2024-05-26T22:50:27.601651+0000","last_became_peered":"2024-05-26T22:50:27.601651+0000","last_unstale":"2024-05-26T22:50:28.607130+0000","last_undegraded":"2024-05-26T22:50:28.607130+0000","last_fullsized":"2024-05-26T22:50:28.607130+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:33:34.699738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"43'2","reported_seq":16,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:29.030473+0000","last_change":"2024-05-26T22:50:26.589034+0000","last_active":"2024-05-26T22:50:29.030473+0000","last_peered":"2024-05-26T22:50:29.030473+0000","last_clean":"2024-05-26T22:50:29.030473+0000","last_became_active":"2024-05-26T22:50:26.587800+0000","last_became_peered":"2024-05-26T22:50:26.587800+0000","last_unstale":"2024-05-26T22:50:29.030473+0000","last_undegraded":"2024-05-26T22:50:29.030473+0000","last_fullsized":"2024-05-26T22:50:29.030473+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:52:27.170697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.616330+0000","last_change":"2024-05-26T22:50:27.610168+0000","last_active":"2024-05-26T22:50:28.616330+0000","last_peered":"2024-05-26T22:50:28.616330+0000","last_clean":"2024-05-26T22:50:28.616330+0000","last_became_active":"2024-05-26T22:50:27.609872+0000","last_became_peered":"2024-05-26T22:50:27.609872+0000","last_unstale":"2024-05-26T22:50:28.616330+0000","last_undegraded":"2024-05-26T22:50:28.616330+0000","last_fullsized":"2024-05-26T22:50:28.616330+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:09:00.534190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"43'1","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.610258+0000","last_change":"2024-05-26T22:50:26.588331+0000","last_active":"2024-05-26T22:50:28.610258+0000","last_peered":"2024-05-26T22:50:28.610258+0000","last_clean":"2024-05-26T22:50:28.610258+0000","last_became_active":"2024-05-26T22:50:26.588083+0000","last_became_peered":"2024-05-26T22:50:26.588083+0000","last_unstale":"2024-05-26T22:50:28.610258+0000","last_undegraded":"2024-05-26T22:50:28.610258+0000","last_fullsized":"2024-05-26T22:50:28.610258+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:52:27.170697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.607891+0000","last_change":"2024-05-26T22:50:27.611774+0000","last_active":"2024-05-26T22:50:28.607891+0000","last_peered":"2024-05-26T22:50:28.607891+0000","last_clean":"2024-05-26T22:50:28.607891+0000","last_became_active":"2024-05-26T22:50:27.604920+0000","last_became_peered":"2024-05-26T22:50:27.604920+0000","last_unstale":"2024-05-26T22:50:28.607891+0000","last_undegraded":"2024-05-26T22:50:28.607891+0000","last_fullsized":"2024-05-26T22:50:28.607891+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:58:06.516711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"43'1","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.616637+0000","last_change":"2024-05-26T22:50:26.585020+0000","last_active":"2024-05-26T22:50:28.616637+0000","last_peered":"2024-05-26T22:50:28.616637+0000","last_clean":"2024-05-26T22:50:28.616637+0000","last_became_active":"2024-05-26T22:50:26.584736+0000","last_became_peered":"2024-05-26T22:50:26.584736+0000","last_unstale":"2024-05-26T22:50:28.616637+0000","last_undegraded":"2024-05-26T22:50:28.616637+0000","last_fullsized":"2024-05-26T22:50:28.616637+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:42:43.319527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.606395+0000","last_change":"2024-05-26T22:50:27.593506+0000","last_active":"2024-05-26T22:50:28.606395+0000","last_peered":"2024-05-26T22:50:28.606395+0000","last_clean":"2024-05-26T22:50:28.606395+0000","last_became_active":"2024-05-26T22:50:27.592798+0000","last_became_peered":"2024-05-26T22:50:27.592798+0000","last_unstale":"2024-05-26T22:50:28.606395+0000","last_undegraded":"2024-05-26T22:50:28.606395+0000","last_fullsized":"2024-05-26T22:50:28.606395+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:06:08.865730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:29.028885+0000","last_change":"2024-05-26T22:50:26.574522+0000","last_active":"2024-05-26T22:50:29.028885+0000","last_peered":"2024-05-26T22:50:29.028885+0000","last_clean":"2024-05-26T22:50:29.028885+0000","last_became_active":"2024-05-26T22:50:26.574033+0000","last_became_peered":"2024-05-26T22:50:26.574033+0000","last_unstale":"2024-05-26T22:50:29.028885+0000","last_undegraded":"2024-05-26T22:50:29.028885+0000","last_fullsized":"2024-05-26T22:50:29.028885+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:29:03.868288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.608513+0000","last_change":"2024-05-26T22:50:27.609584+0000","last_active":"2024-05-26T22:50:28.608513+0000","last_peered":"2024-05-26T22:50:28.608513+0000","last_clean":"2024-05-26T22:50:28.608513+0000","last_became_active":"2024-05-26T22:50:27.609305+0000","last_became_peered":"2024-05-26T22:50:27.609305+0000","last_unstale":"2024-05-26T22:50:28.608513+0000","last_undegraded":"2024-05-26T22:50:28.608513+0000","last_fullsized":"2024-05-26T22:50:28.608513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:42:44.333089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.606424+0000","last_change":"2024-05-26T22:50:26.581544+0000","last_active":"2024-05-26T22:50:28.606424+0000","last_peered":"2024-05-26T22:50:28.606424+0000","last_clean":"2024-05-26T22:50:28.606424+0000","last_became_active":"2024-05-26T22:50:26.581228+0000","last_became_peered":"2024-05-26T22:50:26.581228+0000","last_unstale":"2024-05-26T22:50:28.606424+0000","last_undegraded":"2024-05-26T22:50:28.606424+0000","last_fullsized":"2024-05-26T22:50:28.606424+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:19:59.957522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.617537+0000","last_change":"2024-05-26T22:50:27.625496+0000","last_active":"2024-05-26T22:50:28.617537+0000","last_peered":"2024-05-26T22:50:28.617537+0000","last_clean":"2024-05-26T22:50:28.617537+0000","last_became_active":"2024-05-26T22:50:27.625123+0000","last_became_peered":"2024-05-26T22:50:27.625123+0000","last_unstale":"2024-05-26T22:50:28.617537+0000","last_undegraded":"2024-05-26T22:50:28.617537+0000","last_fullsized":"2024-05-26T22:50:28.617537+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:06:08.865730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.617772+0000","last_change":"2024-05-26T22:50:26.583932+0000","last_active":"2024-05-26T22:50:28.617772+0000","last_peered":"2024-05-26T22:50:28.617772+0000","last_clean":"2024-05-26T22:50:28.617772+0000","last_became_active":"2024-05-26T22:50:26.582486+0000","last_became_peered":"2024-05-26T22:50:26.582486+0000","last_unstale":"2024-05-26T22:50:28.617772+0000","last_undegraded":"2024-05-26T22:50:28.617772+0000","last_fullsized":"2024-05-26T22:50:28.617772+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:53:35.597732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:29.031528+0000","last_change":"2024-05-26T22:50:27.611333+0000","last_active":"2024-05-26T22:50:29.031528+0000","last_peered":"2024-05-26T22:50:29.031528+0000","last_clean":"2024-05-26T22:50:29.031528+0000","last_became_active":"2024-05-26T22:50:27.610941+0000","last_became_peered":"2024-05-26T22:50:27.610941+0000","last_unstale":"2024-05-26T22:50:29.031528+0000","last_undegraded":"2024-05-26T22:50:29.031528+0000","last_fullsized":"2024-05-26T22:50:29.031528+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:13:34.494539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.608745+0000","last_change":"2024-05-26T22:50:26.587905+0000","last_active":"2024-05-26T22:50:28.608745+0000","last_peered":"2024-05-26T22:50:28.608745+0000","last_clean":"2024-05-26T22:50:28.608745+0000","last_became_active":"2024-05-26T22:50:26.587642+0000","last_became_peered":"2024-05-26T22:50:26.587642+0000","last_unstale":"2024-05-26T22:50:28.608745+0000","last_undegraded":"2024-05-26T22:50:28.608745+0000","last_fullsized":"2024-05-26T22:50:28.608745+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:53:35.597732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.610624+0000","last_change":"2024-05-26T22:50:27.630178+0000","last_active":"2024-05-26T22:50:28.610624+0000","last_peered":"2024-05-26T22:50:28.610624+0000","last_clean":"2024-05-26T22:50:28.610624+0000","last_became_active":"2024-05-26T22:50:27.629834+0000","last_became_peered":"2024-05-26T22:50:27.629834+0000","last_unstale":"2024-05-26T22:50:28.610624+0000","last_undegraded":"2024-05-26T22:50:28.610624+0000","last_fullsized":"2024-05-26T22:50:28.610624+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:04:47.607246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"43'3","reported_seq":17,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:29.031132+0000","last_change":"2024-05-26T22:50:26.591431+0000","last_active":"2024-05-26T22:50:29.031132+0000","last_peered":"2024-05-26T22:50:29.031132+0000","last_clean":"2024-05-26T22:50:29.031132+0000","last_became_active":"2024-05-26T22:50:26.591180+0000","last_became_peered":"2024-05-26T22:50:26.591180+0000","last_unstale":"2024-05-26T22:50:29.031132+0000","last_undegraded":"2024-05-26T22:50:29.031132+0000","last_fullsized":"2024-05-26T22:50:29.031132+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:08:59.520628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:29.030606+0000","last_change":"2024-05-26T22:50:27.612043+0000","last_active":"2024-05-26T22:50:29.030606+0000","last_peered":"2024-05-26T22:50:29.030606+0000","last_clean":"2024-05-26T22:50:29.030606+0000","last_became_active":"2024-05-26T22:50:27.611779+0000","last_became_peered":"2024-05-26T22:50:27.611779+0000","last_unstale":"2024-05-26T22:50:29.030606+0000","last_undegraded":"2024-05-26T22:50:29.030606+0000","last_fullsized":"2024-05-26T22:50:29.030606+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:21:16.701772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"43'6","reported_seq":20,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:29.028548+0000","last_change":"2024-05-26T22:50:26.585012+0000","last_active":"2024-05-26T22:50:29.028548+0000","last_peered":"2024-05-26T22:50:29.028548+0000","last_clean":"2024-05-26T22:50:29.028548+0000","last_became_active":"2024-05-26T22:50:26.584653+0000","last_became_peered":"2024-05-26T22:50:26.584653+0000","last_unstale":"2024-05-26T22:50:29.028548+0000","last_undegraded":"2024-05-26T22:50:29.028548+0000","last_fullsized":"2024-05-26T22:50:29.028548+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:19:00.151465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.609458+0000","last_change":"2024-05-26T22:50:27.620675+0000","last_active":"2024-05-26T22:50:28.609458+0000","last_peered":"2024-05-26T22:50:28.609458+0000","last_clean":"2024-05-26T22:50:28.609458+0000","last_became_active":"2024-05-26T22:50:27.620365+0000","last_became_peered":"2024-05-26T22:50:27.620365+0000","last_unstale":"2024-05-26T22:50:28.609458+0000","last_undegraded":"2024-05-26T22:50:28.609458+0000","last_fullsized":"2024-05-26T22:50:28.609458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:15:35.289066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"43'2","reported_seq":16,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.610360+0000","last_change":"2024-05-26T22:50:26.577323+0000","last_active":"2024-05-26T22:50:28.610360+0000","last_peered":"2024-05-26T22:50:28.610360+0000","last_clean":"2024-05-26T22:50:28.610360+0000","last_became_active":"2024-05-26T22:50:26.576604+0000","last_became_peered":"2024-05-26T22:50:26.576604+0000","last_unstale":"2024-05-26T22:50:28.610360+0000","last_undegraded":"2024-05-26T22:50:28.610360+0000","last_fullsized":"2024-05-26T22:50:28.610360+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:06:07.852168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.608226+0000","last_change":"2024-05-26T22:50:27.615536+0000","last_active":"2024-05-26T22:50:28.608226+0000","last_peered":"2024-05-26T22:50:28.608226+0000","last_clean":"2024-05-26T22:50:28.608226+0000","last_became_active":"2024-05-26T22:50:27.615152+0000","last_became_peered":"2024-05-26T22:50:27.615152+0000","last_unstale":"2024-05-26T22:50:28.608226+0000","last_undegraded":"2024-05-26T22:50:28.608226+0000","last_fullsized":"2024-05-26T22:50:28.608226+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:34:05.951317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:29.030456+0000","last_change":"2024-05-26T22:50:26.589863+0000","last_active":"2024-05-26T22:50:29.030456+0000","last_peered":"2024-05-26T22:50:29.030456+0000","last_clean":"2024-05-26T22:50:29.030456+0000","last_became_active":"2024-05-26T22:50:26.589578+0000","last_became_peered":"2024-05-26T22:50:26.589578+0000","last_unstale":"2024-05-26T22:50:29.030456+0000","last_undegraded":"2024-05-26T22:50:29.030456+0000","last_fullsized":"2024-05-26T22:50:29.030456+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:29:03.868288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.609765+0000","last_change":"2024-05-26T22:50:27.619152+0000","last_active":"2024-05-26T22:50:28.609765+0000","last_peered":"2024-05-26T22:50:28.609765+0000","last_clean":"2024-05-26T22:50:28.609765+0000","last_became_active":"2024-05-26T22:50:27.618811+0000","last_became_peered":"2024-05-26T22:50:27.618811+0000","last_unstale":"2024-05-26T22:50:28.609765+0000","last_undegraded":"2024-05-26T22:50:28.609765+0000","last_fullsized":"2024-05-26T22:50:28.609765+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:13:34.494539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.609954+0000","last_change":"2024-05-26T22:50:26.579106+0000","last_active":"2024-05-26T22:50:28.609954+0000","last_peered":"2024-05-26T22:50:28.609954+0000","last_clean":"2024-05-26T22:50:28.609954+0000","last_became_active":"2024-05-26T22:50:26.578796+0000","last_became_peered":"2024-05-26T22:50:26.578796+0000","last_unstale":"2024-05-26T22:50:28.609954+0000","last_undegraded":"2024-05-26T22:50:28.609954+0000","last_fullsized":"2024-05-26T22:50:28.609954+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:48:39.224457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:29.030463+0000","last_change":"2024-05-26T22:50:27.616914+0000","last_active":"2024-05-26T22:50:29.030463+0000","last_peered":"2024-05-26T22:50:29.030463+0000","last_clean":"2024-05-26T22:50:29.030463+0000","last_became_active":"2024-05-26T22:50:27.616624+0000","last_became_peered":"2024-05-26T22:50:27.616624+0000","last_unstale":"2024-05-26T22:50:29.030463+0000","last_undegraded":"2024-05-26T22:50:29.030463+0000","last_fullsized":"2024-05-26T22:50:29.030463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:15:23.061792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"43'8","reported_seq":22,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.606500+0000","last_change":"2024-05-26T22:50:26.583566+0000","last_active":"2024-05-26T22:50:28.606500+0000","last_peered":"2024-05-26T22:50:28.606500+0000","last_clean":"2024-05-26T22:50:28.606500+0000","last_became_active":"2024-05-26T22:50:26.583235+0000","last_became_peered":"2024-05-26T22:50:26.583235+0000","last_unstale":"2024-05-26T22:50:28.606500+0000","last_undegraded":"2024-05-26T22:50:28.606500+0000","last_fullsized":"2024-05-26T22:50:28.606500+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:42:43.319527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:29.028557+0000","last_change":"2024-05-26T22:50:27.606939+0000","last_active":"2024-05-26T22:50:29.028557+0000","last_peered":"2024-05-26T22:50:29.028557+0000","last_clean":"2024-05-26T22:50:29.028557+0000","last_became_active":"2024-05-26T22:50:27.606619+0000","last_became_peered":"2024-05-26T22:50:27.606619+0000","last_unstale":"2024-05-26T22:50:29.028557+0000","last_undegraded":"2024-05-26T22:50:29.028557+0000","last_fullsized":"2024-05-26T22:50:29.028557+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:44:11.226829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"43'1","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:29.028496+0000","last_change":"2024-05-26T22:50:26.579397+0000","last_active":"2024-05-26T22:50:29.028496+0000","last_peered":"2024-05-26T22:50:29.028496+0000","last_clean":"2024-05-26T22:50:29.028496+0000","last_became_active":"2024-05-26T22:50:26.579019+0000","last_became_peered":"2024-05-26T22:50:26.579019+0000","last_unstale":"2024-05-26T22:50:29.028496+0000","last_undegraded":"2024-05-26T22:50:29.028496+0000","last_fullsized":"2024-05-26T22:50:29.028496+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:47:40.423971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.608654+0000","last_change":"2024-05-26T22:50:27.626388+0000","last_active":"2024-05-26T22:50:28.608654+0000","last_peered":"2024-05-26T22:50:28.608654+0000","last_clean":"2024-05-26T22:50:28.608654+0000","last_became_active":"2024-05-26T22:50:27.625515+0000","last_became_peered":"2024-05-26T22:50:27.625515+0000","last_unstale":"2024-05-26T22:50:28.608654+0000","last_undegraded":"2024-05-26T22:50:28.608654+0000","last_fullsized":"2024-05-26T22:50:28.608654+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:44:57.492554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"43'2","reported_seq":16,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.617792+0000","last_change":"2024-05-26T22:50:26.584807+0000","last_active":"2024-05-26T22:50:28.617792+0000","last_peered":"2024-05-26T22:50:28.617792+0000","last_clean":"2024-05-26T22:50:28.617792+0000","last_became_active":"2024-05-26T22:50:26.584427+0000","last_became_peered":"2024-05-26T22:50:26.584427+0000","last_unstale":"2024-05-26T22:50:28.617792+0000","last_undegraded":"2024-05-26T22:50:28.617792+0000","last_fullsized":"2024-05-26T22:50:28.617792+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:26:42.950079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.610112+0000","last_change":"2024-05-26T22:50:27.619929+0000","last_active":"2024-05-26T22:50:28.610112+0000","last_peered":"2024-05-26T22:50:28.610112+0000","last_clean":"2024-05-26T22:50:28.610112+0000","last_became_active":"2024-05-26T22:50:27.619626+0000","last_became_peered":"2024-05-26T22:50:27.619626+0000","last_unstale":"2024-05-26T22:50:28.610112+0000","last_undegraded":"2024-05-26T22:50:28.610112+0000","last_fullsized":"2024-05-26T22:50:28.610112+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:57:10.223342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"43'1","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:29.029470+0000","last_change":"2024-05-26T22:50:26.576296+0000","last_active":"2024-05-26T22:50:29.029470+0000","last_peered":"2024-05-26T22:50:29.029470+0000","last_clean":"2024-05-26T22:50:29.029470+0000","last_became_active":"2024-05-26T22:50:26.575800+0000","last_became_peered":"2024-05-26T22:50:26.575800+0000","last_unstale":"2024-05-26T22:50:29.029470+0000","last_undegraded":"2024-05-26T22:50:29.029470+0000","last_fullsized":"2024-05-26T22:50:29.029470+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:35:15.400981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.610347+0000","last_change":"2024-05-26T22:50:27.606967+0000","last_active":"2024-05-26T22:50:28.610347+0000","last_peered":"2024-05-26T22:50:28.610347+0000","last_clean":"2024-05-26T22:50:28.610347+0000","last_became_active":"2024-05-26T22:50:27.606245+0000","last_became_peered":"2024-05-26T22:50:27.606245+0000","last_unstale":"2024-05-26T22:50:28.610347+0000","last_undegraded":"2024-05-26T22:50:28.610347+0000","last_fullsized":"2024-05-26T22:50:28.610347+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:31:49.984660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.610553+0000","last_change":"2024-05-26T22:50:26.582011+0000","last_active":"2024-05-26T22:50:28.610553+0000","last_peered":"2024-05-26T22:50:28.610553+0000","last_clean":"2024-05-26T22:50:28.610553+0000","last_became_active":"2024-05-26T22:50:26.581706+0000","last_became_peered":"2024-05-26T22:50:26.581706+0000","last_unstale":"2024-05-26T22:50:28.610553+0000","last_undegraded":"2024-05-26T22:50:28.610553+0000","last_fullsized":"2024-05-26T22:50:28.610553+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:28:59.399753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.610117+0000","last_change":"2024-05-26T22:50:27.613787+0000","last_active":"2024-05-26T22:50:28.610117+0000","last_peered":"2024-05-26T22:50:28.610117+0000","last_clean":"2024-05-26T22:50:28.610117+0000","last_became_active":"2024-05-26T22:50:27.612338+0000","last_became_peered":"2024-05-26T22:50:27.612338+0000","last_unstale":"2024-05-26T22:50:28.610117+0000","last_undegraded":"2024-05-26T22:50:28.610117+0000","last_fullsized":"2024-05-26T22:50:28.610117+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:49:03.875973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"43'2","reported_seq":16,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.610465+0000","last_change":"2024-05-26T22:50:26.588207+0000","last_active":"2024-05-26T22:50:28.610465+0000","last_peered":"2024-05-26T22:50:28.610465+0000","last_clean":"2024-05-26T22:50:28.610465+0000","last_became_active":"2024-05-26T22:50:26.587873+0000","last_became_peered":"2024-05-26T22:50:26.587873+0000","last_unstale":"2024-05-26T22:50:28.610465+0000","last_undegraded":"2024-05-26T22:50:28.610465+0000","last_fullsized":"2024-05-26T22:50:28.610465+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:28:41.521657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.605988+0000","last_change":"2024-05-26T22:50:27.610411+0000","last_active":"2024-05-26T22:50:28.605988+0000","last_peered":"2024-05-26T22:50:28.605988+0000","last_clean":"2024-05-26T22:50:28.605988+0000","last_became_active":"2024-05-26T22:50:27.610166+0000","last_became_peered":"2024-05-26T22:50:27.610166+0000","last_unstale":"2024-05-26T22:50:28.605988+0000","last_undegraded":"2024-05-26T22:50:28.605988+0000","last_fullsized":"2024-05-26T22:50:28.605988+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:28:13.522438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:29.029044+0000","last_change":"2024-05-26T22:50:26.581924+0000","last_active":"2024-05-26T22:50:29.029044+0000","last_peered":"2024-05-26T22:50:29.029044+0000","last_clean":"2024-05-26T22:50:29.029044+0000","last_became_active":"2024-05-26T22:50:26.581631+0000","last_became_peered":"2024-05-26T22:50:26.581631+0000","last_unstale":"2024-05-26T22:50:29.029044+0000","last_undegraded":"2024-05-26T22:50:29.029044+0000","last_fullsized":"2024-05-26T22:50:29.029044+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:25:43.834484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:29.027973+0000","last_change":"2024-05-26T22:50:27.618550+0000","last_active":"2024-05-26T22:50:29.027973+0000","last_peered":"2024-05-26T22:50:29.027973+0000","last_clean":"2024-05-26T22:50:29.027973+0000","last_became_active":"2024-05-26T22:50:27.618345+0000","last_became_peered":"2024-05-26T22:50:27.618345+0000","last_unstale":"2024-05-26T22:50:29.027973+0000","last_undegraded":"2024-05-26T22:50:29.027973+0000","last_fullsized":"2024-05-26T22:50:29.027973+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:01:47.391816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.606366+0000","last_change":"2024-05-26T22:50:26.584763+0000","last_active":"2024-05-26T22:50:28.606366+0000","last_peered":"2024-05-26T22:50:28.606366+0000","last_clean":"2024-05-26T22:50:28.606366+0000","last_became_active":"2024-05-26T22:50:26.584324+0000","last_became_peered":"2024-05-26T22:50:26.584324+0000","last_unstale":"2024-05-26T22:50:28.606366+0000","last_undegraded":"2024-05-26T22:50:28.606366+0000","last_fullsized":"2024-05-26T22:50:28.606366+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T23:11:53.223115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.606427+0000","last_change":"2024-05-26T22:50:27.608890+0000","last_active":"2024-05-26T22:50:28.606427+0000","last_peered":"2024-05-26T22:50:28.606427+0000","last_clean":"2024-05-26T22:50:28.606427+0000","last_became_active":"2024-05-26T22:50:27.604269+0000","last_became_peered":"2024-05-26T22:50:27.604269+0000","last_unstale":"2024-05-26T22:50:28.606427+0000","last_undegraded":"2024-05-26T22:50:28.606427+0000","last_fullsized":"2024-05-26T22:50:28.606427+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:29:00.413315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:29.028877+0000","last_change":"2024-05-26T22:50:26.589965+0000","last_active":"2024-05-26T22:50:29.028877+0000","last_peered":"2024-05-26T22:50:29.028877+0000","last_clean":"2024-05-26T22:50:29.028877+0000","last_became_active":"2024-05-26T22:50:26.589604+0000","last_became_peered":"2024-05-26T22:50:26.589604+0000","last_unstale":"2024-05-26T22:50:29.028877+0000","last_undegraded":"2024-05-26T22:50:29.028877+0000","last_fullsized":"2024-05-26T22:50:29.028877+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:48:39.224457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.609943+0000","last_change":"2024-05-26T22:50:27.611710+0000","last_active":"2024-05-26T22:50:28.609943+0000","last_peered":"2024-05-26T22:50:28.609943+0000","last_clean":"2024-05-26T22:50:28.609943+0000","last_became_active":"2024-05-26T22:50:27.610934+0000","last_became_peered":"2024-05-26T22:50:27.610934+0000","last_unstale":"2024-05-26T22:50:28.609943+0000","last_undegraded":"2024-05-26T22:50:28.609943+0000","last_fullsized":"2024-05-26T22:50:28.609943+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:33:34.699738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"44'1","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.614426+0000","last_change":"2024-05-26T22:50:26.587049+0000","last_active":"2024-05-26T22:50:28.614426+0000","last_peered":"2024-05-26T22:50:28.614426+0000","last_clean":"2024-05-26T22:50:28.614426+0000","last_became_active":"2024-05-26T22:50:26.586723+0000","last_became_peered":"2024-05-26T22:50:26.586723+0000","last_unstale":"2024-05-26T22:50:28.614426+0000","last_undegraded":"2024-05-26T22:50:28.614426+0000","last_fullsized":"2024-05-26T22:50:28.614426+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:52:43.215817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.607713+0000","last_change":"2024-05-26T22:50:27.605859+0000","last_active":"2024-05-26T22:50:28.607713+0000","last_peered":"2024-05-26T22:50:28.607713+0000","last_clean":"2024-05-26T22:50:28.607713+0000","last_became_active":"2024-05-26T22:50:27.605404+0000","last_became_peered":"2024-05-26T22:50:27.605404+0000","last_unstale":"2024-05-26T22:50:28.607713+0000","last_undegraded":"2024-05-26T22:50:28.607713+0000","last_fullsized":"2024-05-26T22:50:28.607713+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:52:44.229379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:29.031279+0000","last_change":"2024-05-26T22:50:26.591419+0000","last_active":"2024-05-26T22:50:29.031279+0000","last_peered":"2024-05-26T22:50:29.031279+0000","last_clean":"2024-05-26T22:50:29.031279+0000","last_became_active":"2024-05-26T22:50:26.591179+0000","last_became_peered":"2024-05-26T22:50:26.591179+0000","last_unstale":"2024-05-26T22:50:29.031279+0000","last_undegraded":"2024-05-26T22:50:29.031279+0000","last_fullsized":"2024-05-26T22:50:29.031279+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:14:09.430311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:29.031095+0000","last_change":"2024-05-26T22:50:27.615442+0000","last_active":"2024-05-26T22:50:29.031095+0000","last_peered":"2024-05-26T22:50:29.031095+0000","last_clean":"2024-05-26T22:50:29.031095+0000","last_became_active":"2024-05-26T22:50:27.615157+0000","last_became_peered":"2024-05-26T22:50:27.615157+0000","last_unstale":"2024-05-26T22:50:29.031095+0000","last_undegraded":"2024-05-26T22:50:29.031095+0000","last_fullsized":"2024-05-26T22:50:29.031095+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:55:33.647672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.606290+0000","last_change":"2024-05-26T22:50:26.578562+0000","last_active":"2024-05-26T22:50:28.606290+0000","last_peered":"2024-05-26T22:50:28.606290+0000","last_clean":"2024-05-26T22:50:28.606290+0000","last_became_active":"2024-05-26T22:50:26.577480+0000","last_became_peered":"2024-05-26T22:50:26.577480+0000","last_unstale":"2024-05-26T22:50:28.606290+0000","last_undegraded":"2024-05-26T22:50:28.606290+0000","last_fullsized":"2024-05-26T22:50:28.606290+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:07:38.331762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.610258+0000","last_change":"2024-05-26T22:50:27.617767+0000","last_active":"2024-05-26T22:50:28.610258+0000","last_peered":"2024-05-26T22:50:28.610258+0000","last_clean":"2024-05-26T22:50:28.610258+0000","last_became_active":"2024-05-26T22:50:27.613401+0000","last_became_peered":"2024-05-26T22:50:27.613401+0000","last_unstale":"2024-05-26T22:50:28.610258+0000","last_undegraded":"2024-05-26T22:50:28.610258+0000","last_fullsized":"2024-05-26T22:50:28.610258+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:46:46.492897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:29.031017+0000","last_change":"2024-05-26T22:50:26.587128+0000","last_active":"2024-05-26T22:50:29.031017+0000","last_peered":"2024-05-26T22:50:29.031017+0000","last_clean":"2024-05-26T22:50:29.031017+0000","last_became_active":"2024-05-26T22:50:26.586847+0000","last_became_peered":"2024-05-26T22:50:26.586847+0000","last_unstale":"2024-05-26T22:50:29.031017+0000","last_undegraded":"2024-05-26T22:50:29.031017+0000","last_fullsized":"2024-05-26T22:50:29.031017+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:35:15.400981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.610312+0000","last_change":"2024-05-26T22:50:27.625960+0000","last_active":"2024-05-26T22:50:28.610312+0000","last_peered":"2024-05-26T22:50:28.610312+0000","last_clean":"2024-05-26T22:50:28.610312+0000","last_became_active":"2024-05-26T22:50:27.625621+0000","last_became_peered":"2024-05-26T22:50:27.625621+0000","last_unstale":"2024-05-26T22:50:28.610312+0000","last_undegraded":"2024-05-26T22:50:28.610312+0000","last_fullsized":"2024-05-26T22:50:28.610312+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:26:50.858068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.616671+0000","last_change":"2024-05-26T22:50:26.585664+0000","last_active":"2024-05-26T22:50:28.616671+0000","last_peered":"2024-05-26T22:50:28.616671+0000","last_clean":"2024-05-26T22:50:28.616671+0000","last_became_active":"2024-05-26T22:50:26.585056+0000","last_became_peered":"2024-05-26T22:50:26.585056+0000","last_unstale":"2024-05-26T22:50:28.616671+0000","last_undegraded":"2024-05-26T22:50:28.616671+0000","last_fullsized":"2024-05-26T22:50:28.616671+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:07:38.331762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:29.029128+0000","last_change":"2024-05-26T22:50:27.612641+0000","last_active":"2024-05-26T22:50:29.029128+0000","last_peered":"2024-05-26T22:50:29.029128+0000","last_clean":"2024-05-26T22:50:29.029128+0000","last_became_active":"2024-05-26T22:50:27.612332+0000","last_became_peered":"2024-05-26T22:50:27.612332+0000","last_unstale":"2024-05-26T22:50:29.029128+0000","last_undegraded":"2024-05-26T22:50:29.029128+0000","last_fullsized":"2024-05-26T22:50:29.029128+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:44:14.357986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.605807+0000","last_change":"2024-05-26T22:50:26.579162+0000","last_active":"2024-05-26T22:50:28.605807+0000","last_peered":"2024-05-26T22:50:28.605807+0000","last_clean":"2024-05-26T22:50:28.605807+0000","last_became_active":"2024-05-26T22:50:26.578107+0000","last_became_peered":"2024-05-26T22:50:26.578107+0000","last_unstale":"2024-05-26T22:50:28.605807+0000","last_undegraded":"2024-05-26T22:50:28.605807+0000","last_fullsized":"2024-05-26T22:50:28.605807+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:58:28.776394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.616297+0000","last_change":"2024-05-26T22:50:27.619099+0000","last_active":"2024-05-26T22:50:28.616297+0000","last_peered":"2024-05-26T22:50:28.616297+0000","last_clean":"2024-05-26T22:50:28.616297+0000","last_became_active":"2024-05-26T22:50:27.616954+0000","last_became_peered":"2024-05-26T22:50:27.616954+0000","last_unstale":"2024-05-26T22:50:28.616297+0000","last_undegraded":"2024-05-26T22:50:28.616297+0000","last_fullsized":"2024-05-26T22:50:28.616297+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:29:04.881850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.606557+0000","last_change":"2024-05-26T22:50:26.577478+0000","last_active":"2024-05-26T22:50:28.606557+0000","last_peered":"2024-05-26T22:50:28.606557+0000","last_clean":"2024-05-26T22:50:28.606557+0000","last_became_active":"2024-05-26T22:50:26.576506+0000","last_became_peered":"2024-05-26T22:50:26.576506+0000","last_unstale":"2024-05-26T22:50:28.606557+0000","last_undegraded":"2024-05-26T22:50:28.606557+0000","last_fullsized":"2024-05-26T22:50:28.606557+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:07:38.331762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.608750+0000","last_change":"2024-05-26T22:50:27.620262+0000","last_active":"2024-05-26T22:50:28.608750+0000","last_peered":"2024-05-26T22:50:28.608750+0000","last_clean":"2024-05-26T22:50:28.608750+0000","last_became_active":"2024-05-26T22:50:27.619951+0000","last_became_peered":"2024-05-26T22:50:27.619951+0000","last_unstale":"2024-05-26T22:50:28.608750+0000","last_undegraded":"2024-05-26T22:50:28.608750+0000","last_fullsized":"2024-05-26T22:50:28.608750+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:13:34.494539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"43'2","reported_seq":16,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:29.030612+0000","last_change":"2024-05-26T22:50:26.577747+0000","last_active":"2024-05-26T22:50:29.030612+0000","last_peered":"2024-05-26T22:50:29.030612+0000","last_clean":"2024-05-26T22:50:29.030612+0000","last_became_active":"2024-05-26T22:50:26.577408+0000","last_became_peered":"2024-05-26T22:50:26.577408+0000","last_unstale":"2024-05-26T22:50:29.030612+0000","last_undegraded":"2024-05-26T22:50:29.030612+0000","last_fullsized":"2024-05-26T22:50:29.030612+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:08:59.520628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":215,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:29.029658+0000","last_change":"2024-05-26T22:48:12.068917+0000","last_active":"2024-05-26T22:50:29.029658+0000","last_peered":"2024-05-26T22:50:29.029658+0000","last_clean":"2024-05-26T22:50:29.029658+0000","last_became_active":"2024-05-26T22:48:12.065934+0000","last_became_peered":"2024-05-26T22:48:12.065934+0000","last_unstale":"2024-05-26T22:50:29.029658+0000","last_undegraded":"2024-05-26T22:50:29.029658+0000","last_fullsized":"2024-05-26T22:50:29.029658+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:10.628122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:10.628122+0000","last_clean_scrub_stamp":"2024-05-26T22:48:10.628122+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:14:58.895911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574879,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32268,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339572,"statfs":{"total":96636764160,"available":96603721728,"internally_reserved":0,"allocated":1114112,"data_stored":801258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574879,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27532,"kb_used_data":512,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344308,"statfs":{"total":96636764160,"available":96608571392,"internally_reserved":0,"allocated":524288,"data_stored":211472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574880,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27528,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344312,"statfs":{"total":96636764160,"available":96608575488,"internally_reserved":0,"allocated":520192,"data_stored":210870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574879,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27536,"kb_used_data":516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344304,"statfs":{"total":96636764160,"available":96608567296,"internally_reserved":0,"allocated":528384,"data_stored":212412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574879,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27544,"kb_used_data":516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344296,"statfs":{"total":96636764160,"available":96608559104,"internally_reserved":0,"allocated":528384,"data_stored":211438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574879,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27540,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344300,"statfs":{"total":96636764160,"available":96608563200,"internally_reserved":0,"allocated":532480,"data_stored":212434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574879,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27536,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344304,"statfs":{"total":96636764160,"available":96608567296,"internally_reserved":0,"allocated":520192,"data_stored":210870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574879,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32272,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339568,"statfs":{"total":96636764160,"available":96603717632,"internally_reserved":0,"allocated":1114112,"data_stored":801202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-26T22:50:36.955 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-26T22:50:37.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.165+0000 7f943d600640 1 Processor -- start 2024-05-26T22:50:37.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.165+0000 7f943d600640 1 -- start start 2024-05-26T22:50:37.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.166+0000 7f943d600640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f9438145a80 0x7f94380b3a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:37.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.166+0000 7f943d600640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f943815cd70 0x7f94380b5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:37.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.167+0000 7f943d600640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9438145e60 con 0x7f94380b3f90 2024-05-26T22:50:37.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.167+0000 7f943d600640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f943815d870 con 0x7f9438145a80 2024-05-26T22:50:37.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.167+0000 7f943d600640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f94380b5510 con 0x7f943815cd70 2024-05-26T22:50:37.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.167+0000 7f9437e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f9438145a80 0x7f94380b3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:37.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.167+0000 7f942f400640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f94380b3f90 0x7f943815c660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:58694/0 (socket says 172.21.15.131:58694) 2024-05-26T22:50:37.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.167+0000 7f942f400640 1 -- 172.21.15.131:0/24542535 learned_addr learned my addr 172.21.15.131:0/24542535 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:50:37.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.167+0000 7f943cc00640 1 --2- 172.21.15.131:0/24542535 >> v2:172.21.15.187:3300/0 conn(0x7f943815cd70 0x7f94380b5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:37.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.168+0000 7f9437400640 1 -- 172.21.15.131:0/24542535 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1089446453 0 0) 0x7f9438145e60 con 0x7f94380b3f90 2024-05-26T22:50:37.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.168+0000 7f9437400640 1 -- 172.21.15.131:0/24542535 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f941c003520 con 0x7f94380b3f90 2024-05-26T22:50:37.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.168+0000 7f9437400640 1 -- 172.21.15.131:0/24542535 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1810452605 0 0) 0x7f941c003520 con 0x7f94380b3f90 2024-05-26T22:50:37.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.168+0000 7f9437e00640 1 -- 172.21.15.131:0/24542535 >> v2:172.21.15.187:3300/0 conn(0x7f943815cd70 msgr2=0x7f94380b5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:37.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.168+0000 7f9437e00640 1 --2- 172.21.15.131:0/24542535 >> v2:172.21.15.187:3300/0 conn(0x7f943815cd70 0x7f94380b5fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:37.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.168+0000 7f9437e00640 1 -- 172.21.15.131:0/24542535 >> v1:172.21.15.131:6789/0 conn(0x7f94380b3f90 legacy=0x7f943815c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:37.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.168+0000 7f9437e00640 1 -- 172.21.15.131:0/24542535 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f943815d180 con 0x7f9438145a80 2024-05-26T22:50:37.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.169+0000 7f9437e00640 1 --2- 172.21.15.131:0/24542535 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9438145a80 0x7f94380b3a50 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f94380ce1d0 tx=0x7f942802a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c37651ab0c01756a server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:37.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.169+0000 7f9437400640 1 -- 172.21.15.131:0/24542535 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9428037070 con 0x7f9438145a80 2024-05-26T22:50:37.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.169+0000 7f9437400640 1 -- 172.21.15.131:0/24542535 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f942802abc0 con 0x7f9438145a80 2024-05-26T22:50:37.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.169+0000 7f9437400640 1 -- 172.21.15.131:0/24542535 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9420001670 con 0x7f9438145a80 2024-05-26T22:50:37.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.170+0000 7f943d600640 1 -- 172.21.15.131:0/24542535 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9438145a80 msgr2=0x7f94380b3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:37.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.170+0000 7f943d600640 1 --2- 172.21.15.131:0/24542535 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9438145a80 0x7f94380b3a50 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f94380ce1d0 tx=0x7f942802a130 comp rx=0 tx=0).stop 2024-05-26T22:50:37.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.170+0000 7f943d600640 1 -- 172.21.15.131:0/24542535 shutdown_connections 2024-05-26T22:50:37.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.170+0000 7f943d600640 1 --2- 172.21.15.131:0/24542535 >> v2:172.21.15.187:3300/0 conn(0x7f943815cd70 0x7f94380b5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:37.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.170+0000 7f943d600640 1 --2- 172.21.15.131:0/24542535 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9438145a80 0x7f94380b3a50 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:37.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.170+0000 7f943d600640 1 -- 172.21.15.131:0/24542535 >> 172.21.15.131:0/24542535 conn(0x7f943806b940 msgr2=0x7f94380b72e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:37.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.170+0000 7f943d600640 1 -- 172.21.15.131:0/24542535 shutdown_connections 2024-05-26T22:50:37.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.170+0000 7f943d600640 1 -- 172.21.15.131:0/24542535 wait complete. 2024-05-26T22:50:37.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.172+0000 7f943d600640 1 Processor -- start 2024-05-26T22:50:37.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.172+0000 7f943d600640 1 -- start start 2024-05-26T22:50:37.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.173+0000 7f943d600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f94380b3f90 0x7f94380b2c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:37.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.173+0000 7f943d600640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9438145a80 0x7f9438157560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:37.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.173+0000 7f943d600640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f943815cd70 0x7f9438157aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:37.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.173+0000 7f943d600640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f94380b5ab0 con 0x7f94380b3f90 2024-05-26T22:50:37.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.173+0000 7f943d600640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f94380b57d0 con 0x7f943815cd70 2024-05-26T22:50:37.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.173+0000 7f943d600640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f94380b6510 con 0x7f9438145a80 2024-05-26T22:50:37.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.173+0000 7f9437e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f94380b3f90 0x7f94380b2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:37.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.173+0000 7f9437e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f94380b3f90 0x7f94380b2c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41722/0 (socket says 172.21.15.131:41722) 2024-05-26T22:50:37.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.173+0000 7f9437e00640 1 -- 172.21.15.131:0/522631165 learned_addr learned my addr 172.21.15.131:0/522631165 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:37.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.174+0000 7f9437e00640 1 -- 172.21.15.131:0/522631165 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9438145a80 msgr2=0x7f9438157560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:37.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.174+0000 7f9437e00640 1 --2- 172.21.15.131:0/522631165 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9438145a80 0x7f9438157560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:37.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.174+0000 7f9437e00640 1 -- 172.21.15.131:0/522631165 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f943815cd70 msgr2=0x7f9438157aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:37.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.174+0000 7f943cc00640 1 --2- 172.21.15.131:0/522631165 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f943815cd70 0x7f9438157aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:37.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.174+0000 7f9437e00640 1 --2- 172.21.15.131:0/522631165 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f943815cd70 0x7f9438157aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:37.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.174+0000 7f9437e00640 1 -- 172.21.15.131:0/522631165 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94381f3f60 con 0x7f94380b3f90 2024-05-26T22:50:37.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.174+0000 7f9437e00640 1 --2- 172.21.15.131:0/522631165 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f94380b3f90 0x7f94380b2c80 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f9428035040 tx=0x7f94280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:37.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.174+0000 7f9434c00640 1 -- 172.21.15.131:0/522631165 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9428041020 con 0x7f94380b3f90 2024-05-26T22:50:37.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.174+0000 7f9434c00640 1 -- 172.21.15.131:0/522631165 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f942803d640 con 0x7f94380b3f90 2024-05-26T22:50:37.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.174+0000 7f9434c00640 1 -- 172.21.15.131:0/522631165 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9428037040 con 0x7f94380b3f90 2024-05-26T22:50:37.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.174+0000 7f943d600640 1 -- 172.21.15.131:0/522631165 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94381f4240 con 0x7f94380b3f90 2024-05-26T22:50:37.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.174+0000 7f943d600640 1 -- 172.21.15.131:0/522631165 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94381f4800 con 0x7f94380b3f90 2024-05-26T22:50:37.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.176+0000 7f943d600640 1 -- 172.21.15.131:0/522631165 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94380694e0 con 0x7f94380b3f90 2024-05-26T22:50:37.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.177+0000 7f9434c00640 1 -- 172.21.15.131:0/522631165 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9428044020 con 0x7f94380b3f90 2024-05-26T22:50:37.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.177+0000 7f9434c00640 1 --2- 172.21.15.131:0/522631165 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f941406be30 0x7f941406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:37.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.178+0000 7f9434c00640 1 -- 172.21.15.131:0/522631165 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f94280ab4c0 con 0x7f94380b3f90 2024-05-26T22:50:37.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.178+0000 7f942f400640 1 --2- 172.21.15.131:0/522631165 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f941406be30 0x7f941406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:37.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.178+0000 7f942f400640 1 --2- 172.21.15.131:0/522631165 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f941406be30 0x7f941406e2f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f94381f17b0 tx=0x7f9420001020 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:37.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.182+0000 7f9434c00640 1 -- 172.21.15.131:0/522631165 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f942804b030 con 0x7f94380b3f90 2024-05-26T22:50:37.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.379+0000 7f943d600640 1 -- 172.21.15.131:0/522631165 --> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f94380745a0 con 0x7f941406be30 2024-05-26T22:50:37.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.388+0000 7f9434c00640 1 -- 172.21.15.131:0/522631165 <== mgr.4104 v2:172.21.15.131:6832/3484832947 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251848 (secure 0 0 0) 0x7f94380745a0 con 0x7f941406be30 2024-05-26T22:50:37.391 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:50:37.393 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-05-26T22:50:37.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.393+0000 7f943d600640 1 -- 172.21.15.131:0/522631165 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f941406be30 msgr2=0x7f941406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:37.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.393+0000 7f943d600640 1 --2- 172.21.15.131:0/522631165 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f941406be30 0x7f941406e2f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f94381f17b0 tx=0x7f9420001020 comp rx=0 tx=0).stop 2024-05-26T22:50:37.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.393+0000 7f943d600640 1 -- 172.21.15.131:0/522631165 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f94380b3f90 msgr2=0x7f94380b2c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:37.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.393+0000 7f943d600640 1 --2- 172.21.15.131:0/522631165 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f94380b3f90 0x7f94380b2c80 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f9428035040 tx=0x7f94280046c0 comp rx=0 tx=0).stop 2024-05-26T22:50:37.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.393+0000 7f943d600640 1 -- 172.21.15.131:0/522631165 shutdown_connections 2024-05-26T22:50:37.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.393+0000 7f943d600640 1 --2- 172.21.15.131:0/522631165 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9438145a80 0x7f9438157560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:37.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.393+0000 7f943d600640 1 --2- 172.21.15.131:0/522631165 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f941406be30 0x7f941406e2f0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:37.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.393+0000 7f943d600640 1 --2- 172.21.15.131:0/522631165 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f943815cd70 0x7f9438157aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:37.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.393+0000 7f943d600640 1 --2- 172.21.15.131:0/522631165 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f94380b3f90 0x7f94380b2c80 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:37.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.393+0000 7f943d600640 1 -- 172.21.15.131:0/522631165 >> 172.21.15.131:0/522631165 conn(0x7f943806b940 msgr2=0x7f943806c700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:37.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.394+0000 7f943d600640 1 -- 172.21.15.131:0/522631165 shutdown_connections 2024-05-26T22:50:37.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.394+0000 7f943d600640 1 -- 172.21.15.131:0/522631165 wait complete. 2024-05-26T22:50:37.413 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":110,"stamp":"2024-05-26T22:50:36.427458+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":229756,"kb_used_data":5256,"kb_used_omap":12,"kb_used_meta":224371,"kb_avail":754744964,"statfs":{"total":773094113280,"available":772858843136,"internally_reserved":0,"allocated":5382144,"data_stored":2871956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":229756465},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.851903"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:29.031378+0000","last_change":"2024-05-26T22:50:27.610657+0000","last_active":"2024-05-26T22:50:29.031378+0000","last_peered":"2024-05-26T22:50:29.031378+0000","last_clean":"2024-05-26T22:50:29.031378+0000","last_became_active":"2024-05-26T22:50:27.610367+0000","last_became_peered":"2024-05-26T22:50:27.610367+0000","last_unstale":"2024-05-26T22:50:29.031378+0000","last_undegraded":"2024-05-26T22:50:29.031378+0000","last_fullsized":"2024-05-26T22:50:29.031378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:44:57.492554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:29.030017+0000","last_change":"2024-05-26T22:50:27.621789+0000","last_active":"2024-05-26T22:50:29.030017+0000","last_peered":"2024-05-26T22:50:29.030017+0000","last_clean":"2024-05-26T22:50:29.030017+0000","last_became_active":"2024-05-26T22:50:27.621501+0000","last_became_peered":"2024-05-26T22:50:27.621501+0000","last_unstale":"2024-05-26T22:50:29.030017+0000","last_undegraded":"2024-05-26T22:50:29.030017+0000","last_fullsized":"2024-05-26T22:50:29.030017+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:15:23.061792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.617060+0000","last_change":"2024-05-26T22:50:27.602597+0000","last_active":"2024-05-26T22:50:28.617060+0000","last_peered":"2024-05-26T22:50:28.617060+0000","last_clean":"2024-05-26T22:50:28.617060+0000","last_became_active":"2024-05-26T22:50:27.602270+0000","last_became_peered":"2024-05-26T22:50:27.602270+0000","last_unstale":"2024-05-26T22:50:28.617060+0000","last_undegraded":"2024-05-26T22:50:28.617060+0000","last_fullsized":"2024-05-26T22:50:28.617060+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:48:40.238018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.607638+0000","last_change":"2024-05-26T22:50:27.606124+0000","last_active":"2024-05-26T22:50:28.607638+0000","last_peered":"2024-05-26T22:50:28.607638+0000","last_clean":"2024-05-26T22:50:28.607638+0000","last_became_active":"2024-05-26T22:50:27.605799+0000","last_became_peered":"2024-05-26T22:50:27.605799+0000","last_unstale":"2024-05-26T22:50:28.607638+0000","last_undegraded":"2024-05-26T22:50:28.607638+0000","last_fullsized":"2024-05-26T22:50:28.607638+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:40:21.264809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:29.029641+0000","last_change":"2024-05-26T22:50:27.622010+0000","last_active":"2024-05-26T22:50:29.029641+0000","last_peered":"2024-05-26T22:50:29.029641+0000","last_clean":"2024-05-26T22:50:29.029641+0000","last_became_active":"2024-05-26T22:50:27.621491+0000","last_became_peered":"2024-05-26T22:50:27.621491+0000","last_unstale":"2024-05-26T22:50:29.029641+0000","last_undegraded":"2024-05-26T22:50:29.029641+0000","last_fullsized":"2024-05-26T22:50:29.029641+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:58:06.516711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.610098+0000","last_change":"2024-05-26T22:50:27.619003+0000","last_active":"2024-05-26T22:50:28.610098+0000","last_peered":"2024-05-26T22:50:28.610098+0000","last_clean":"2024-05-26T22:50:28.610098+0000","last_became_active":"2024-05-26T22:50:27.613040+0000","last_became_peered":"2024-05-26T22:50:27.613040+0000","last_unstale":"2024-05-26T22:50:28.610098+0000","last_undegraded":"2024-05-26T22:50:28.610098+0000","last_fullsized":"2024-05-26T22:50:28.610098+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:21:37.744153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:29.030212+0000","last_change":"2024-05-26T22:50:27.612767+0000","last_active":"2024-05-26T22:50:29.030212+0000","last_peered":"2024-05-26T22:50:29.030212+0000","last_clean":"2024-05-26T22:50:29.030212+0000","last_became_active":"2024-05-26T22:50:27.612139+0000","last_became_peered":"2024-05-26T22:50:27.612139+0000","last_unstale":"2024-05-26T22:50:29.030212+0000","last_undegraded":"2024-05-26T22:50:29.030212+0000","last_fullsized":"2024-05-26T22:50:29.030212+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:34:05.951317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.606233+0000","last_change":"2024-05-26T22:50:27.593124+0000","last_active":"2024-05-26T22:50:28.606233+0000","last_peered":"2024-05-26T22:50:28.606233+0000","last_clean":"2024-05-26T22:50:28.606233+0000","last_became_active":"2024-05-26T22:50:27.591829+0000","last_became_peered":"2024-05-26T22:50:27.591829+0000","last_unstale":"2024-05-26T22:50:28.606233+0000","last_undegraded":"2024-05-26T22:50:28.606233+0000","last_fullsized":"2024-05-26T22:50:28.606233+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:52:28.184259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.608451+0000","last_change":"2024-05-26T22:50:27.609384+0000","last_active":"2024-05-26T22:50:28.608451+0000","last_peered":"2024-05-26T22:50:28.608451+0000","last_clean":"2024-05-26T22:50:28.608451+0000","last_became_active":"2024-05-26T22:50:27.604237+0000","last_became_peered":"2024-05-26T22:50:27.604237+0000","last_unstale":"2024-05-26T22:50:28.608451+0000","last_undegraded":"2024-05-26T22:50:28.608451+0000","last_fullsized":"2024-05-26T22:50:28.608451+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:15:23.061792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.608544+0000","last_change":"2024-05-26T22:50:27.593515+0000","last_active":"2024-05-26T22:50:28.608544+0000","last_peered":"2024-05-26T22:50:28.608544+0000","last_clean":"2024-05-26T22:50:28.608544+0000","last_became_active":"2024-05-26T22:50:27.592190+0000","last_became_peered":"2024-05-26T22:50:27.592190+0000","last_unstale":"2024-05-26T22:50:28.608544+0000","last_undegraded":"2024-05-26T22:50:28.608544+0000","last_fullsized":"2024-05-26T22:50:28.608544+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:55:33.647672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.608476+0000","last_change":"2024-05-26T22:50:27.605243+0000","last_active":"2024-05-26T22:50:28.608476+0000","last_peered":"2024-05-26T22:50:28.608476+0000","last_clean":"2024-05-26T22:50:28.608476+0000","last_became_active":"2024-05-26T22:50:27.604931+0000","last_became_peered":"2024-05-26T22:50:27.604931+0000","last_unstale":"2024-05-26T22:50:28.608476+0000","last_undegraded":"2024-05-26T22:50:28.608476+0000","last_fullsized":"2024-05-26T22:50:28.608476+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:21:16.701772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:29.029545+0000","last_change":"2024-05-26T22:50:27.627122+0000","last_active":"2024-05-26T22:50:29.029545+0000","last_peered":"2024-05-26T22:50:29.029545+0000","last_clean":"2024-05-26T22:50:29.029545+0000","last_became_active":"2024-05-26T22:50:27.626847+0000","last_became_peered":"2024-05-26T22:50:27.626847+0000","last_unstale":"2024-05-26T22:50:29.029545+0000","last_undegraded":"2024-05-26T22:50:29.029545+0000","last_fullsized":"2024-05-26T22:50:29.029545+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:40:21.264809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:29.028177+0000","last_change":"2024-05-26T22:50:27.616958+0000","last_active":"2024-05-26T22:50:29.028177+0000","last_peered":"2024-05-26T22:50:29.028177+0000","last_clean":"2024-05-26T22:50:29.028177+0000","last_became_active":"2024-05-26T22:50:27.616551+0000","last_became_peered":"2024-05-26T22:50:27.616551+0000","last_unstale":"2024-05-26T22:50:29.028177+0000","last_undegraded":"2024-05-26T22:50:29.028177+0000","last_fullsized":"2024-05-26T22:50:29.028177+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:14:28.234483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.607145+0000","last_change":"2024-05-26T22:50:27.608517+0000","last_active":"2024-05-26T22:50:28.607145+0000","last_peered":"2024-05-26T22:50:28.607145+0000","last_clean":"2024-05-26T22:50:28.607145+0000","last_became_active":"2024-05-26T22:50:27.606950+0000","last_became_peered":"2024-05-26T22:50:27.606950+0000","last_unstale":"2024-05-26T22:50:28.607145+0000","last_undegraded":"2024-05-26T22:50:28.607145+0000","last_fullsized":"2024-05-26T22:50:28.607145+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:28:42.535219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.609790+0000","last_change":"2024-05-26T22:50:27.608959+0000","last_active":"2024-05-26T22:50:28.609790+0000","last_peered":"2024-05-26T22:50:28.609790+0000","last_clean":"2024-05-26T22:50:28.609790+0000","last_became_active":"2024-05-26T22:50:27.608700+0000","last_became_peered":"2024-05-26T22:50:27.608700+0000","last_unstale":"2024-05-26T22:50:28.609790+0000","last_undegraded":"2024-05-26T22:50:28.609790+0000","last_fullsized":"2024-05-26T22:50:28.609790+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:12:46.452654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.608690+0000","last_change":"2024-05-26T22:50:27.611336+0000","last_active":"2024-05-26T22:50:28.608690+0000","last_peered":"2024-05-26T22:50:28.608690+0000","last_clean":"2024-05-26T22:50:28.608690+0000","last_became_active":"2024-05-26T22:50:27.611063+0000","last_became_peered":"2024-05-26T22:50:27.611063+0000","last_unstale":"2024-05-26T22:50:28.608690+0000","last_undegraded":"2024-05-26T22:50:28.608690+0000","last_fullsized":"2024-05-26T22:50:28.608690+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:53:36.611294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.617634+0000","last_change":"2024-05-26T22:50:27.611141+0000","last_active":"2024-05-26T22:50:28.617634+0000","last_peered":"2024-05-26T22:50:28.617634+0000","last_clean":"2024-05-26T22:50:28.617634+0000","last_became_active":"2024-05-26T22:50:27.610706+0000","last_became_peered":"2024-05-26T22:50:27.610706+0000","last_unstale":"2024-05-26T22:50:28.617634+0000","last_undegraded":"2024-05-26T22:50:28.617634+0000","last_fullsized":"2024-05-26T22:50:28.617634+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:29:00.413315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.617367+0000","last_change":"2024-05-26T22:50:27.625629+0000","last_active":"2024-05-26T22:50:28.617367+0000","last_peered":"2024-05-26T22:50:28.617367+0000","last_clean":"2024-05-26T22:50:28.617367+0000","last_became_active":"2024-05-26T22:50:27.625326+0000","last_became_peered":"2024-05-26T22:50:27.625326+0000","last_unstale":"2024-05-26T22:50:28.617367+0000","last_undegraded":"2024-05-26T22:50:28.617367+0000","last_fullsized":"2024-05-26T22:50:28.617367+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:52:28.184259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.610528+0000","last_change":"2024-05-26T22:50:27.612043+0000","last_active":"2024-05-26T22:50:28.610528+0000","last_peered":"2024-05-26T22:50:28.610528+0000","last_clean":"2024-05-26T22:50:28.610528+0000","last_became_active":"2024-05-26T22:50:27.611759+0000","last_became_peered":"2024-05-26T22:50:27.611759+0000","last_unstale":"2024-05-26T22:50:28.610528+0000","last_undegraded":"2024-05-26T22:50:28.610528+0000","last_fullsized":"2024-05-26T22:50:28.610528+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:54:47.587339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:29.028648+0000","last_change":"2024-05-26T22:50:27.620986+0000","last_active":"2024-05-26T22:50:29.028648+0000","last_peered":"2024-05-26T22:50:29.028648+0000","last_clean":"2024-05-26T22:50:29.028648+0000","last_became_active":"2024-05-26T22:50:27.620701+0000","last_became_peered":"2024-05-26T22:50:27.620701+0000","last_unstale":"2024-05-26T22:50:29.028648+0000","last_undegraded":"2024-05-26T22:50:29.028648+0000","last_fullsized":"2024-05-26T22:50:29.028648+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:13:05.372755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.610175+0000","last_change":"2024-05-26T22:50:27.623707+0000","last_active":"2024-05-26T22:50:28.610175+0000","last_peered":"2024-05-26T22:50:28.610175+0000","last_clean":"2024-05-26T22:50:28.610175+0000","last_became_active":"2024-05-26T22:50:27.623450+0000","last_became_peered":"2024-05-26T22:50:27.623450+0000","last_unstale":"2024-05-26T22:50:28.610175+0000","last_undegraded":"2024-05-26T22:50:28.610175+0000","last_fullsized":"2024-05-26T22:50:28.610175+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:21:39.447850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.607328+0000","last_change":"2024-05-26T22:50:27.595025+0000","last_active":"2024-05-26T22:50:28.607328+0000","last_peered":"2024-05-26T22:50:28.607328+0000","last_clean":"2024-05-26T22:50:28.607328+0000","last_became_active":"2024-05-26T22:50:27.594473+0000","last_became_peered":"2024-05-26T22:50:27.594473+0000","last_unstale":"2024-05-26T22:50:28.607328+0000","last_undegraded":"2024-05-26T22:50:28.607328+0000","last_fullsized":"2024-05-26T22:50:28.607328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:44:14.357986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.609269+0000","last_change":"2024-05-26T22:50:27.609501+0000","last_active":"2024-05-26T22:50:28.609269+0000","last_peered":"2024-05-26T22:50:28.609269+0000","last_clean":"2024-05-26T22:50:28.609269+0000","last_became_active":"2024-05-26T22:50:27.608925+0000","last_became_peered":"2024-05-26T22:50:27.608925+0000","last_unstale":"2024-05-26T22:50:28.609269+0000","last_undegraded":"2024-05-26T22:50:28.609269+0000","last_fullsized":"2024-05-26T22:50:28.609269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:58:29.789956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.608671+0000","last_change":"2024-05-26T22:50:27.606676+0000","last_active":"2024-05-26T22:50:28.608671+0000","last_peered":"2024-05-26T22:50:28.608671+0000","last_clean":"2024-05-26T22:50:28.608671+0000","last_became_active":"2024-05-26T22:50:27.606399+0000","last_became_peered":"2024-05-26T22:50:27.606399+0000","last_unstale":"2024-05-26T22:50:28.608671+0000","last_undegraded":"2024-05-26T22:50:28.608671+0000","last_fullsized":"2024-05-26T22:50:28.608671+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:07:39.345324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.609416+0000","last_change":"2024-05-26T22:50:27.603472+0000","last_active":"2024-05-26T22:50:28.609416+0000","last_peered":"2024-05-26T22:50:28.609416+0000","last_clean":"2024-05-26T22:50:28.609416+0000","last_became_active":"2024-05-26T22:50:27.602661+0000","last_became_peered":"2024-05-26T22:50:27.602661+0000","last_unstale":"2024-05-26T22:50:28.609416+0000","last_undegraded":"2024-05-26T22:50:28.609416+0000","last_fullsized":"2024-05-26T22:50:28.609416+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:26:43.963641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:29.028316+0000","last_change":"2024-05-26T22:50:27.621771+0000","last_active":"2024-05-26T22:50:29.028316+0000","last_peered":"2024-05-26T22:50:29.028316+0000","last_clean":"2024-05-26T22:50:29.028316+0000","last_became_active":"2024-05-26T22:50:27.621425+0000","last_became_peered":"2024-05-26T22:50:27.621425+0000","last_unstale":"2024-05-26T22:50:29.028316+0000","last_undegraded":"2024-05-26T22:50:29.028316+0000","last_fullsized":"2024-05-26T22:50:29.028316+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:04:18.699067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.616839+0000","last_change":"2024-05-26T22:50:27.607252+0000","last_active":"2024-05-26T22:50:28.616839+0000","last_peered":"2024-05-26T22:50:28.616839+0000","last_clean":"2024-05-26T22:50:28.616839+0000","last_became_active":"2024-05-26T22:50:27.606329+0000","last_became_peered":"2024-05-26T22:50:27.606329+0000","last_unstale":"2024-05-26T22:50:28.616839+0000","last_undegraded":"2024-05-26T22:50:28.616839+0000","last_fullsized":"2024-05-26T22:50:28.616839+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:14:10.443873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:29.029593+0000","last_change":"2024-05-26T22:50:27.626991+0000","last_active":"2024-05-26T22:50:29.029593+0000","last_peered":"2024-05-26T22:50:29.029593+0000","last_clean":"2024-05-26T22:50:29.029593+0000","last_became_active":"2024-05-26T22:50:27.626702+0000","last_became_peered":"2024-05-26T22:50:27.626702+0000","last_unstale":"2024-05-26T22:50:29.029593+0000","last_undegraded":"2024-05-26T22:50:29.029593+0000","last_fullsized":"2024-05-26T22:50:29.029593+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:52:44.229379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.609538+0000","last_change":"2024-05-26T22:50:27.602948+0000","last_active":"2024-05-26T22:50:28.609538+0000","last_peered":"2024-05-26T22:50:28.609538+0000","last_clean":"2024-05-26T22:50:28.609538+0000","last_became_active":"2024-05-26T22:50:27.602641+0000","last_became_peered":"2024-05-26T22:50:27.602641+0000","last_unstale":"2024-05-26T22:50:28.609538+0000","last_undegraded":"2024-05-26T22:50:28.609538+0000","last_fullsized":"2024-05-26T22:50:28.609538+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:28:13.522438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.616295+0000","last_change":"2024-05-26T22:50:27.613332+0000","last_active":"2024-05-26T22:50:28.616295+0000","last_peered":"2024-05-26T22:50:28.616295+0000","last_clean":"2024-05-26T22:50:28.616295+0000","last_became_active":"2024-05-26T22:50:27.613002+0000","last_became_peered":"2024-05-26T22:50:27.613002+0000","last_unstale":"2024-05-26T22:50:28.616295+0000","last_undegraded":"2024-05-26T22:50:28.616295+0000","last_fullsized":"2024-05-26T22:50:28.616295+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:35:16.414543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.609812+0000","last_change":"2024-05-26T22:50:27.609655+0000","last_active":"2024-05-26T22:50:28.609812+0000","last_peered":"2024-05-26T22:50:28.609812+0000","last_clean":"2024-05-26T22:50:28.609812+0000","last_became_active":"2024-05-26T22:50:27.609367+0000","last_became_peered":"2024-05-26T22:50:27.609367+0000","last_unstale":"2024-05-26T22:50:28.609812+0000","last_undegraded":"2024-05-26T22:50:28.609812+0000","last_fullsized":"2024-05-26T22:50:28.609812+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:48:07.563526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:29.027930+0000","last_change":"2024-05-26T22:50:27.613465+0000","last_active":"2024-05-26T22:50:29.027930+0000","last_peered":"2024-05-26T22:50:29.027930+0000","last_clean":"2024-05-26T22:50:29.027930+0000","last_became_active":"2024-05-26T22:50:27.613133+0000","last_became_peered":"2024-05-26T22:50:27.613133+0000","last_unstale":"2024-05-26T22:50:29.027930+0000","last_undegraded":"2024-05-26T22:50:29.027930+0000","last_fullsized":"2024-05-26T22:50:29.027930+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:28:32.032578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.608109+0000","last_change":"2024-05-26T22:50:27.625540+0000","last_active":"2024-05-26T22:50:28.608109+0000","last_peered":"2024-05-26T22:50:28.608109+0000","last_clean":"2024-05-26T22:50:28.608109+0000","last_became_active":"2024-05-26T22:50:27.624727+0000","last_became_peered":"2024-05-26T22:50:27.624727+0000","last_unstale":"2024-05-26T22:50:28.608109+0000","last_undegraded":"2024-05-26T22:50:28.608109+0000","last_fullsized":"2024-05-26T22:50:28.608109+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:57:10.223342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"43'4","reported_seq":18,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.611114+0000","last_change":"2024-05-26T22:50:26.581794+0000","last_active":"2024-05-26T22:50:28.611114+0000","last_peered":"2024-05-26T22:50:28.611114+0000","last_clean":"2024-05-26T22:50:28.611114+0000","last_became_active":"2024-05-26T22:50:26.581504+0000","last_became_peered":"2024-05-26T22:50:26.581504+0000","last_unstale":"2024-05-26T22:50:28.611114+0000","last_undegraded":"2024-05-26T22:50:28.611114+0000","last_fullsized":"2024-05-26T22:50:28.611114+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:44:13.344424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.609437+0000","last_change":"2024-05-26T22:50:27.607318+0000","last_active":"2024-05-26T22:50:28.609437+0000","last_peered":"2024-05-26T22:50:28.609437+0000","last_clean":"2024-05-26T22:50:28.609437+0000","last_became_active":"2024-05-26T22:50:27.607013+0000","last_became_peered":"2024-05-26T22:50:27.607013+0000","last_unstale":"2024-05-26T22:50:28.609437+0000","last_undegraded":"2024-05-26T22:50:28.609437+0000","last_fullsized":"2024-05-26T22:50:28.609437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:23:58.424666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"43'1","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.617636+0000","last_change":"2024-05-26T22:50:26.582366+0000","last_active":"2024-05-26T22:50:28.617636+0000","last_peered":"2024-05-26T22:50:28.617636+0000","last_clean":"2024-05-26T22:50:28.617636+0000","last_became_active":"2024-05-26T22:50:26.582022+0000","last_became_peered":"2024-05-26T22:50:26.582022+0000","last_unstale":"2024-05-26T22:50:28.617636+0000","last_undegraded":"2024-05-26T22:50:28.617636+0000","last_fullsized":"2024-05-26T22:50:28.617636+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:58:28.776394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:29.029938+0000","last_change":"2024-05-26T22:50:27.622401+0000","last_active":"2024-05-26T22:50:29.029938+0000","last_peered":"2024-05-26T22:50:29.029938+0000","last_clean":"2024-05-26T22:50:29.029938+0000","last_became_active":"2024-05-26T22:50:27.622072+0000","last_became_peered":"2024-05-26T22:50:27.622072+0000","last_unstale":"2024-05-26T22:50:29.029938+0000","last_undegraded":"2024-05-26T22:50:29.029938+0000","last_fullsized":"2024-05-26T22:50:29.029938+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:34:05.951317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"43'2","reported_seq":16,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.607043+0000","last_change":"2024-05-26T22:50:26.579628+0000","last_active":"2024-05-26T22:50:28.607043+0000","last_peered":"2024-05-26T22:50:28.607043+0000","last_clean":"2024-05-26T22:50:28.607043+0000","last_became_active":"2024-05-26T22:50:26.579316+0000","last_became_peered":"2024-05-26T22:50:26.579316+0000","last_unstale":"2024-05-26T22:50:28.607043+0000","last_undegraded":"2024-05-26T22:50:28.607043+0000","last_fullsized":"2024-05-26T22:50:28.607043+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:53:35.597732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.607130+0000","last_change":"2024-05-26T22:50:27.609117+0000","last_active":"2024-05-26T22:50:28.607130+0000","last_peered":"2024-05-26T22:50:28.607130+0000","last_clean":"2024-05-26T22:50:28.607130+0000","last_became_active":"2024-05-26T22:50:27.601651+0000","last_became_peered":"2024-05-26T22:50:27.601651+0000","last_unstale":"2024-05-26T22:50:28.607130+0000","last_undegraded":"2024-05-26T22:50:28.607130+0000","last_fullsized":"2024-05-26T22:50:28.607130+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:33:34.699738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"43'2","reported_seq":16,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:29.030473+0000","last_change":"2024-05-26T22:50:26.589034+0000","last_active":"2024-05-26T22:50:29.030473+0000","last_peered":"2024-05-26T22:50:29.030473+0000","last_clean":"2024-05-26T22:50:29.030473+0000","last_became_active":"2024-05-26T22:50:26.587800+0000","last_became_peered":"2024-05-26T22:50:26.587800+0000","last_unstale":"2024-05-26T22:50:29.030473+0000","last_undegraded":"2024-05-26T22:50:29.030473+0000","last_fullsized":"2024-05-26T22:50:29.030473+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:52:27.170697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.616330+0000","last_change":"2024-05-26T22:50:27.610168+0000","last_active":"2024-05-26T22:50:28.616330+0000","last_peered":"2024-05-26T22:50:28.616330+0000","last_clean":"2024-05-26T22:50:28.616330+0000","last_became_active":"2024-05-26T22:50:27.609872+0000","last_became_peered":"2024-05-26T22:50:27.609872+0000","last_unstale":"2024-05-26T22:50:28.616330+0000","last_undegraded":"2024-05-26T22:50:28.616330+0000","last_fullsized":"2024-05-26T22:50:28.616330+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:09:00.534190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"43'1","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.610258+0000","last_change":"2024-05-26T22:50:26.588331+0000","last_active":"2024-05-26T22:50:28.610258+0000","last_peered":"2024-05-26T22:50:28.610258+0000","last_clean":"2024-05-26T22:50:28.610258+0000","last_became_active":"2024-05-26T22:50:26.588083+0000","last_became_peered":"2024-05-26T22:50:26.588083+0000","last_unstale":"2024-05-26T22:50:28.610258+0000","last_undegraded":"2024-05-26T22:50:28.610258+0000","last_fullsized":"2024-05-26T22:50:28.610258+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:52:27.170697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.607891+0000","last_change":"2024-05-26T22:50:27.611774+0000","last_active":"2024-05-26T22:50:28.607891+0000","last_peered":"2024-05-26T22:50:28.607891+0000","last_clean":"2024-05-26T22:50:28.607891+0000","last_became_active":"2024-05-26T22:50:27.604920+0000","last_became_peered":"2024-05-26T22:50:27.604920+0000","last_unstale":"2024-05-26T22:50:28.607891+0000","last_undegraded":"2024-05-26T22:50:28.607891+0000","last_fullsized":"2024-05-26T22:50:28.607891+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:58:06.516711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"43'1","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.616637+0000","last_change":"2024-05-26T22:50:26.585020+0000","last_active":"2024-05-26T22:50:28.616637+0000","last_peered":"2024-05-26T22:50:28.616637+0000","last_clean":"2024-05-26T22:50:28.616637+0000","last_became_active":"2024-05-26T22:50:26.584736+0000","last_became_peered":"2024-05-26T22:50:26.584736+0000","last_unstale":"2024-05-26T22:50:28.616637+0000","last_undegraded":"2024-05-26T22:50:28.616637+0000","last_fullsized":"2024-05-26T22:50:28.616637+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:42:43.319527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.606395+0000","last_change":"2024-05-26T22:50:27.593506+0000","last_active":"2024-05-26T22:50:28.606395+0000","last_peered":"2024-05-26T22:50:28.606395+0000","last_clean":"2024-05-26T22:50:28.606395+0000","last_became_active":"2024-05-26T22:50:27.592798+0000","last_became_peered":"2024-05-26T22:50:27.592798+0000","last_unstale":"2024-05-26T22:50:28.606395+0000","last_undegraded":"2024-05-26T22:50:28.606395+0000","last_fullsized":"2024-05-26T22:50:28.606395+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:06:08.865730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:29.028885+0000","last_change":"2024-05-26T22:50:26.574522+0000","last_active":"2024-05-26T22:50:29.028885+0000","last_peered":"2024-05-26T22:50:29.028885+0000","last_clean":"2024-05-26T22:50:29.028885+0000","last_became_active":"2024-05-26T22:50:26.574033+0000","last_became_peered":"2024-05-26T22:50:26.574033+0000","last_unstale":"2024-05-26T22:50:29.028885+0000","last_undegraded":"2024-05-26T22:50:29.028885+0000","last_fullsized":"2024-05-26T22:50:29.028885+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:29:03.868288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.608513+0000","last_change":"2024-05-26T22:50:27.609584+0000","last_active":"2024-05-26T22:50:28.608513+0000","last_peered":"2024-05-26T22:50:28.608513+0000","last_clean":"2024-05-26T22:50:28.608513+0000","last_became_active":"2024-05-26T22:50:27.609305+0000","last_became_peered":"2024-05-26T22:50:27.609305+0000","last_unstale":"2024-05-26T22:50:28.608513+0000","last_undegraded":"2024-05-26T22:50:28.608513+0000","last_fullsized":"2024-05-26T22:50:28.608513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:42:44.333089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.606424+0000","last_change":"2024-05-26T22:50:26.581544+0000","last_active":"2024-05-26T22:50:28.606424+0000","last_peered":"2024-05-26T22:50:28.606424+0000","last_clean":"2024-05-26T22:50:28.606424+0000","last_became_active":"2024-05-26T22:50:26.581228+0000","last_became_peered":"2024-05-26T22:50:26.581228+0000","last_unstale":"2024-05-26T22:50:28.606424+0000","last_undegraded":"2024-05-26T22:50:28.606424+0000","last_fullsized":"2024-05-26T22:50:28.606424+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:19:59.957522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.617537+0000","last_change":"2024-05-26T22:50:27.625496+0000","last_active":"2024-05-26T22:50:28.617537+0000","last_peered":"2024-05-26T22:50:28.617537+0000","last_clean":"2024-05-26T22:50:28.617537+0000","last_became_active":"2024-05-26T22:50:27.625123+0000","last_became_peered":"2024-05-26T22:50:27.625123+0000","last_unstale":"2024-05-26T22:50:28.617537+0000","last_undegraded":"2024-05-26T22:50:28.617537+0000","last_fullsized":"2024-05-26T22:50:28.617537+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:06:08.865730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.617772+0000","last_change":"2024-05-26T22:50:26.583932+0000","last_active":"2024-05-26T22:50:28.617772+0000","last_peered":"2024-05-26T22:50:28.617772+0000","last_clean":"2024-05-26T22:50:28.617772+0000","last_became_active":"2024-05-26T22:50:26.582486+0000","last_became_peered":"2024-05-26T22:50:26.582486+0000","last_unstale":"2024-05-26T22:50:28.617772+0000","last_undegraded":"2024-05-26T22:50:28.617772+0000","last_fullsized":"2024-05-26T22:50:28.617772+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:53:35.597732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:29.031528+0000","last_change":"2024-05-26T22:50:27.611333+0000","last_active":"2024-05-26T22:50:29.031528+0000","last_peered":"2024-05-26T22:50:29.031528+0000","last_clean":"2024-05-26T22:50:29.031528+0000","last_became_active":"2024-05-26T22:50:27.610941+0000","last_became_peered":"2024-05-26T22:50:27.610941+0000","last_unstale":"2024-05-26T22:50:29.031528+0000","last_undegraded":"2024-05-26T22:50:29.031528+0000","last_fullsized":"2024-05-26T22:50:29.031528+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:13:34.494539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.608745+0000","last_change":"2024-05-26T22:50:26.587905+0000","last_active":"2024-05-26T22:50:28.608745+0000","last_peered":"2024-05-26T22:50:28.608745+0000","last_clean":"2024-05-26T22:50:28.608745+0000","last_became_active":"2024-05-26T22:50:26.587642+0000","last_became_peered":"2024-05-26T22:50:26.587642+0000","last_unstale":"2024-05-26T22:50:28.608745+0000","last_undegraded":"2024-05-26T22:50:28.608745+0000","last_fullsized":"2024-05-26T22:50:28.608745+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:53:35.597732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.610624+0000","last_change":"2024-05-26T22:50:27.630178+0000","last_active":"2024-05-26T22:50:28.610624+0000","last_peered":"2024-05-26T22:50:28.610624+0000","last_clean":"2024-05-26T22:50:28.610624+0000","last_became_active":"2024-05-26T22:50:27.629834+0000","last_became_peered":"2024-05-26T22:50:27.629834+0000","last_unstale":"2024-05-26T22:50:28.610624+0000","last_undegraded":"2024-05-26T22:50:28.610624+0000","last_fullsized":"2024-05-26T22:50:28.610624+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:04:47.607246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"43'3","reported_seq":17,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:29.031132+0000","last_change":"2024-05-26T22:50:26.591431+0000","last_active":"2024-05-26T22:50:29.031132+0000","last_peered":"2024-05-26T22:50:29.031132+0000","last_clean":"2024-05-26T22:50:29.031132+0000","last_became_active":"2024-05-26T22:50:26.591180+0000","last_became_peered":"2024-05-26T22:50:26.591180+0000","last_unstale":"2024-05-26T22:50:29.031132+0000","last_undegraded":"2024-05-26T22:50:29.031132+0000","last_fullsized":"2024-05-26T22:50:29.031132+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:08:59.520628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:29.030606+0000","last_change":"2024-05-26T22:50:27.612043+0000","last_active":"2024-05-26T22:50:29.030606+0000","last_peered":"2024-05-26T22:50:29.030606+0000","last_clean":"2024-05-26T22:50:29.030606+0000","last_became_active":"2024-05-26T22:50:27.611779+0000","last_became_peered":"2024-05-26T22:50:27.611779+0000","last_unstale":"2024-05-26T22:50:29.030606+0000","last_undegraded":"2024-05-26T22:50:29.030606+0000","last_fullsized":"2024-05-26T22:50:29.030606+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:21:16.701772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"43'6","reported_seq":20,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:29.028548+0000","last_change":"2024-05-26T22:50:26.585012+0000","last_active":"2024-05-26T22:50:29.028548+0000","last_peered":"2024-05-26T22:50:29.028548+0000","last_clean":"2024-05-26T22:50:29.028548+0000","last_became_active":"2024-05-26T22:50:26.584653+0000","last_became_peered":"2024-05-26T22:50:26.584653+0000","last_unstale":"2024-05-26T22:50:29.028548+0000","last_undegraded":"2024-05-26T22:50:29.028548+0000","last_fullsized":"2024-05-26T22:50:29.028548+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:19:00.151465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.609458+0000","last_change":"2024-05-26T22:50:27.620675+0000","last_active":"2024-05-26T22:50:28.609458+0000","last_peered":"2024-05-26T22:50:28.609458+0000","last_clean":"2024-05-26T22:50:28.609458+0000","last_became_active":"2024-05-26T22:50:27.620365+0000","last_became_peered":"2024-05-26T22:50:27.620365+0000","last_unstale":"2024-05-26T22:50:28.609458+0000","last_undegraded":"2024-05-26T22:50:28.609458+0000","last_fullsized":"2024-05-26T22:50:28.609458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:15:35.289066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"43'2","reported_seq":16,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.610360+0000","last_change":"2024-05-26T22:50:26.577323+0000","last_active":"2024-05-26T22:50:28.610360+0000","last_peered":"2024-05-26T22:50:28.610360+0000","last_clean":"2024-05-26T22:50:28.610360+0000","last_became_active":"2024-05-26T22:50:26.576604+0000","last_became_peered":"2024-05-26T22:50:26.576604+0000","last_unstale":"2024-05-26T22:50:28.610360+0000","last_undegraded":"2024-05-26T22:50:28.610360+0000","last_fullsized":"2024-05-26T22:50:28.610360+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:06:07.852168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.608226+0000","last_change":"2024-05-26T22:50:27.615536+0000","last_active":"2024-05-26T22:50:28.608226+0000","last_peered":"2024-05-26T22:50:28.608226+0000","last_clean":"2024-05-26T22:50:28.608226+0000","last_became_active":"2024-05-26T22:50:27.615152+0000","last_became_peered":"2024-05-26T22:50:27.615152+0000","last_unstale":"2024-05-26T22:50:28.608226+0000","last_undegraded":"2024-05-26T22:50:28.608226+0000","last_fullsized":"2024-05-26T22:50:28.608226+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:34:05.951317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:29.030456+0000","last_change":"2024-05-26T22:50:26.589863+0000","last_active":"2024-05-26T22:50:29.030456+0000","last_peered":"2024-05-26T22:50:29.030456+0000","last_clean":"2024-05-26T22:50:29.030456+0000","last_became_active":"2024-05-26T22:50:26.589578+0000","last_became_peered":"2024-05-26T22:50:26.589578+0000","last_unstale":"2024-05-26T22:50:29.030456+0000","last_undegraded":"2024-05-26T22:50:29.030456+0000","last_fullsized":"2024-05-26T22:50:29.030456+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:29:03.868288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.609765+0000","last_change":"2024-05-26T22:50:27.619152+0000","last_active":"2024-05-26T22:50:28.609765+0000","last_peered":"2024-05-26T22:50:28.609765+0000","last_clean":"2024-05-26T22:50:28.609765+0000","last_became_active":"2024-05-26T22:50:27.618811+0000","last_became_peered":"2024-05-26T22:50:27.618811+0000","last_unstale":"2024-05-26T22:50:28.609765+0000","last_undegraded":"2024-05-26T22:50:28.609765+0000","last_fullsized":"2024-05-26T22:50:28.609765+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:13:34.494539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.609954+0000","last_change":"2024-05-26T22:50:26.579106+0000","last_active":"2024-05-26T22:50:28.609954+0000","last_peered":"2024-05-26T22:50:28.609954+0000","last_clean":"2024-05-26T22:50:28.609954+0000","last_became_active":"2024-05-26T22:50:26.578796+0000","last_became_peered":"2024-05-26T22:50:26.578796+0000","last_unstale":"2024-05-26T22:50:28.609954+0000","last_undegraded":"2024-05-26T22:50:28.609954+0000","last_fullsized":"2024-05-26T22:50:28.609954+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:48:39.224457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:29.030463+0000","last_change":"2024-05-26T22:50:27.616914+0000","last_active":"2024-05-26T22:50:29.030463+0000","last_peered":"2024-05-26T22:50:29.030463+0000","last_clean":"2024-05-26T22:50:29.030463+0000","last_became_active":"2024-05-26T22:50:27.616624+0000","last_became_peered":"2024-05-26T22:50:27.616624+0000","last_unstale":"2024-05-26T22:50:29.030463+0000","last_undegraded":"2024-05-26T22:50:29.030463+0000","last_fullsized":"2024-05-26T22:50:29.030463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:15:23.061792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"43'8","reported_seq":22,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.606500+0000","last_change":"2024-05-26T22:50:26.583566+0000","last_active":"2024-05-26T22:50:28.606500+0000","last_peered":"2024-05-26T22:50:28.606500+0000","last_clean":"2024-05-26T22:50:28.606500+0000","last_became_active":"2024-05-26T22:50:26.583235+0000","last_became_peered":"2024-05-26T22:50:26.583235+0000","last_unstale":"2024-05-26T22:50:28.606500+0000","last_undegraded":"2024-05-26T22:50:28.606500+0000","last_fullsized":"2024-05-26T22:50:28.606500+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:42:43.319527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:29.028557+0000","last_change":"2024-05-26T22:50:27.606939+0000","last_active":"2024-05-26T22:50:29.028557+0000","last_peered":"2024-05-26T22:50:29.028557+0000","last_clean":"2024-05-26T22:50:29.028557+0000","last_became_active":"2024-05-26T22:50:27.606619+0000","last_became_peered":"2024-05-26T22:50:27.606619+0000","last_unstale":"2024-05-26T22:50:29.028557+0000","last_undegraded":"2024-05-26T22:50:29.028557+0000","last_fullsized":"2024-05-26T22:50:29.028557+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:44:11.226829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"43'1","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:29.028496+0000","last_change":"2024-05-26T22:50:26.579397+0000","last_active":"2024-05-26T22:50:29.028496+0000","last_peered":"2024-05-26T22:50:29.028496+0000","last_clean":"2024-05-26T22:50:29.028496+0000","last_became_active":"2024-05-26T22:50:26.579019+0000","last_became_peered":"2024-05-26T22:50:26.579019+0000","last_unstale":"2024-05-26T22:50:29.028496+0000","last_undegraded":"2024-05-26T22:50:29.028496+0000","last_fullsized":"2024-05-26T22:50:29.028496+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:47:40.423971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.608654+0000","last_change":"2024-05-26T22:50:27.626388+0000","last_active":"2024-05-26T22:50:28.608654+0000","last_peered":"2024-05-26T22:50:28.608654+0000","last_clean":"2024-05-26T22:50:28.608654+0000","last_became_active":"2024-05-26T22:50:27.625515+0000","last_became_peered":"2024-05-26T22:50:27.625515+0000","last_unstale":"2024-05-26T22:50:28.608654+0000","last_undegraded":"2024-05-26T22:50:28.608654+0000","last_fullsized":"2024-05-26T22:50:28.608654+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:44:57.492554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"43'2","reported_seq":16,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.617792+0000","last_change":"2024-05-26T22:50:26.584807+0000","last_active":"2024-05-26T22:50:28.617792+0000","last_peered":"2024-05-26T22:50:28.617792+0000","last_clean":"2024-05-26T22:50:28.617792+0000","last_became_active":"2024-05-26T22:50:26.584427+0000","last_became_peered":"2024-05-26T22:50:26.584427+0000","last_unstale":"2024-05-26T22:50:28.617792+0000","last_undegraded":"2024-05-26T22:50:28.617792+0000","last_fullsized":"2024-05-26T22:50:28.617792+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:26:42.950079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.610112+0000","last_change":"2024-05-26T22:50:27.619929+0000","last_active":"2024-05-26T22:50:28.610112+0000","last_peered":"2024-05-26T22:50:28.610112+0000","last_clean":"2024-05-26T22:50:28.610112+0000","last_became_active":"2024-05-26T22:50:27.619626+0000","last_became_peered":"2024-05-26T22:50:27.619626+0000","last_unstale":"2024-05-26T22:50:28.610112+0000","last_undegraded":"2024-05-26T22:50:28.610112+0000","last_fullsized":"2024-05-26T22:50:28.610112+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:57:10.223342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"43'1","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:29.029470+0000","last_change":"2024-05-26T22:50:26.576296+0000","last_active":"2024-05-26T22:50:29.029470+0000","last_peered":"2024-05-26T22:50:29.029470+0000","last_clean":"2024-05-26T22:50:29.029470+0000","last_became_active":"2024-05-26T22:50:26.575800+0000","last_became_peered":"2024-05-26T22:50:26.575800+0000","last_unstale":"2024-05-26T22:50:29.029470+0000","last_undegraded":"2024-05-26T22:50:29.029470+0000","last_fullsized":"2024-05-26T22:50:29.029470+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:35:15.400981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.610347+0000","last_change":"2024-05-26T22:50:27.606967+0000","last_active":"2024-05-26T22:50:28.610347+0000","last_peered":"2024-05-26T22:50:28.610347+0000","last_clean":"2024-05-26T22:50:28.610347+0000","last_became_active":"2024-05-26T22:50:27.606245+0000","last_became_peered":"2024-05-26T22:50:27.606245+0000","last_unstale":"2024-05-26T22:50:28.610347+0000","last_undegraded":"2024-05-26T22:50:28.610347+0000","last_fullsized":"2024-05-26T22:50:28.610347+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:31:49.984660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.610553+0000","last_change":"2024-05-26T22:50:26.582011+0000","last_active":"2024-05-26T22:50:28.610553+0000","last_peered":"2024-05-26T22:50:28.610553+0000","last_clean":"2024-05-26T22:50:28.610553+0000","last_became_active":"2024-05-26T22:50:26.581706+0000","last_became_peered":"2024-05-26T22:50:26.581706+0000","last_unstale":"2024-05-26T22:50:28.610553+0000","last_undegraded":"2024-05-26T22:50:28.610553+0000","last_fullsized":"2024-05-26T22:50:28.610553+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:28:59.399753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.610117+0000","last_change":"2024-05-26T22:50:27.613787+0000","last_active":"2024-05-26T22:50:28.610117+0000","last_peered":"2024-05-26T22:50:28.610117+0000","last_clean":"2024-05-26T22:50:28.610117+0000","last_became_active":"2024-05-26T22:50:27.612338+0000","last_became_peered":"2024-05-26T22:50:27.612338+0000","last_unstale":"2024-05-26T22:50:28.610117+0000","last_undegraded":"2024-05-26T22:50:28.610117+0000","last_fullsized":"2024-05-26T22:50:28.610117+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:49:03.875973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"43'2","reported_seq":16,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.610465+0000","last_change":"2024-05-26T22:50:26.588207+0000","last_active":"2024-05-26T22:50:28.610465+0000","last_peered":"2024-05-26T22:50:28.610465+0000","last_clean":"2024-05-26T22:50:28.610465+0000","last_became_active":"2024-05-26T22:50:26.587873+0000","last_became_peered":"2024-05-26T22:50:26.587873+0000","last_unstale":"2024-05-26T22:50:28.610465+0000","last_undegraded":"2024-05-26T22:50:28.610465+0000","last_fullsized":"2024-05-26T22:50:28.610465+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:28:41.521657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.605988+0000","last_change":"2024-05-26T22:50:27.610411+0000","last_active":"2024-05-26T22:50:28.605988+0000","last_peered":"2024-05-26T22:50:28.605988+0000","last_clean":"2024-05-26T22:50:28.605988+0000","last_became_active":"2024-05-26T22:50:27.610166+0000","last_became_peered":"2024-05-26T22:50:27.610166+0000","last_unstale":"2024-05-26T22:50:28.605988+0000","last_undegraded":"2024-05-26T22:50:28.605988+0000","last_fullsized":"2024-05-26T22:50:28.605988+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:28:13.522438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:29.029044+0000","last_change":"2024-05-26T22:50:26.581924+0000","last_active":"2024-05-26T22:50:29.029044+0000","last_peered":"2024-05-26T22:50:29.029044+0000","last_clean":"2024-05-26T22:50:29.029044+0000","last_became_active":"2024-05-26T22:50:26.581631+0000","last_became_peered":"2024-05-26T22:50:26.581631+0000","last_unstale":"2024-05-26T22:50:29.029044+0000","last_undegraded":"2024-05-26T22:50:29.029044+0000","last_fullsized":"2024-05-26T22:50:29.029044+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:25:43.834484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:29.027973+0000","last_change":"2024-05-26T22:50:27.618550+0000","last_active":"2024-05-26T22:50:29.027973+0000","last_peered":"2024-05-26T22:50:29.027973+0000","last_clean":"2024-05-26T22:50:29.027973+0000","last_became_active":"2024-05-26T22:50:27.618345+0000","last_became_peered":"2024-05-26T22:50:27.618345+0000","last_unstale":"2024-05-26T22:50:29.027973+0000","last_undegraded":"2024-05-26T22:50:29.027973+0000","last_fullsized":"2024-05-26T22:50:29.027973+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:01:47.391816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.606366+0000","last_change":"2024-05-26T22:50:26.584763+0000","last_active":"2024-05-26T22:50:28.606366+0000","last_peered":"2024-05-26T22:50:28.606366+0000","last_clean":"2024-05-26T22:50:28.606366+0000","last_became_active":"2024-05-26T22:50:26.584324+0000","last_became_peered":"2024-05-26T22:50:26.584324+0000","last_unstale":"2024-05-26T22:50:28.606366+0000","last_undegraded":"2024-05-26T22:50:28.606366+0000","last_fullsized":"2024-05-26T22:50:28.606366+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T23:11:53.223115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.606427+0000","last_change":"2024-05-26T22:50:27.608890+0000","last_active":"2024-05-26T22:50:28.606427+0000","last_peered":"2024-05-26T22:50:28.606427+0000","last_clean":"2024-05-26T22:50:28.606427+0000","last_became_active":"2024-05-26T22:50:27.604269+0000","last_became_peered":"2024-05-26T22:50:27.604269+0000","last_unstale":"2024-05-26T22:50:28.606427+0000","last_undegraded":"2024-05-26T22:50:28.606427+0000","last_fullsized":"2024-05-26T22:50:28.606427+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:29:00.413315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:29.028877+0000","last_change":"2024-05-26T22:50:26.589965+0000","last_active":"2024-05-26T22:50:29.028877+0000","last_peered":"2024-05-26T22:50:29.028877+0000","last_clean":"2024-05-26T22:50:29.028877+0000","last_became_active":"2024-05-26T22:50:26.589604+0000","last_became_peered":"2024-05-26T22:50:26.589604+0000","last_unstale":"2024-05-26T22:50:29.028877+0000","last_undegraded":"2024-05-26T22:50:29.028877+0000","last_fullsized":"2024-05-26T22:50:29.028877+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:48:39.224457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.609943+0000","last_change":"2024-05-26T22:50:27.611710+0000","last_active":"2024-05-26T22:50:28.609943+0000","last_peered":"2024-05-26T22:50:28.609943+0000","last_clean":"2024-05-26T22:50:28.609943+0000","last_became_active":"2024-05-26T22:50:27.610934+0000","last_became_peered":"2024-05-26T22:50:27.610934+0000","last_unstale":"2024-05-26T22:50:28.609943+0000","last_undegraded":"2024-05-26T22:50:28.609943+0000","last_fullsized":"2024-05-26T22:50:28.609943+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:33:34.699738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"44'1","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.614426+0000","last_change":"2024-05-26T22:50:26.587049+0000","last_active":"2024-05-26T22:50:28.614426+0000","last_peered":"2024-05-26T22:50:28.614426+0000","last_clean":"2024-05-26T22:50:28.614426+0000","last_became_active":"2024-05-26T22:50:26.586723+0000","last_became_peered":"2024-05-26T22:50:26.586723+0000","last_unstale":"2024-05-26T22:50:28.614426+0000","last_undegraded":"2024-05-26T22:50:28.614426+0000","last_fullsized":"2024-05-26T22:50:28.614426+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:52:43.215817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.607713+0000","last_change":"2024-05-26T22:50:27.605859+0000","last_active":"2024-05-26T22:50:28.607713+0000","last_peered":"2024-05-26T22:50:28.607713+0000","last_clean":"2024-05-26T22:50:28.607713+0000","last_became_active":"2024-05-26T22:50:27.605404+0000","last_became_peered":"2024-05-26T22:50:27.605404+0000","last_unstale":"2024-05-26T22:50:28.607713+0000","last_undegraded":"2024-05-26T22:50:28.607713+0000","last_fullsized":"2024-05-26T22:50:28.607713+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:52:44.229379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:29.031279+0000","last_change":"2024-05-26T22:50:26.591419+0000","last_active":"2024-05-26T22:50:29.031279+0000","last_peered":"2024-05-26T22:50:29.031279+0000","last_clean":"2024-05-26T22:50:29.031279+0000","last_became_active":"2024-05-26T22:50:26.591179+0000","last_became_peered":"2024-05-26T22:50:26.591179+0000","last_unstale":"2024-05-26T22:50:29.031279+0000","last_undegraded":"2024-05-26T22:50:29.031279+0000","last_fullsized":"2024-05-26T22:50:29.031279+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:14:09.430311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:29.031095+0000","last_change":"2024-05-26T22:50:27.615442+0000","last_active":"2024-05-26T22:50:29.031095+0000","last_peered":"2024-05-26T22:50:29.031095+0000","last_clean":"2024-05-26T22:50:29.031095+0000","last_became_active":"2024-05-26T22:50:27.615157+0000","last_became_peered":"2024-05-26T22:50:27.615157+0000","last_unstale":"2024-05-26T22:50:29.031095+0000","last_undegraded":"2024-05-26T22:50:29.031095+0000","last_fullsized":"2024-05-26T22:50:29.031095+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:55:33.647672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.606290+0000","last_change":"2024-05-26T22:50:26.578562+0000","last_active":"2024-05-26T22:50:28.606290+0000","last_peered":"2024-05-26T22:50:28.606290+0000","last_clean":"2024-05-26T22:50:28.606290+0000","last_became_active":"2024-05-26T22:50:26.577480+0000","last_became_peered":"2024-05-26T22:50:26.577480+0000","last_unstale":"2024-05-26T22:50:28.606290+0000","last_undegraded":"2024-05-26T22:50:28.606290+0000","last_fullsized":"2024-05-26T22:50:28.606290+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:07:38.331762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.610258+0000","last_change":"2024-05-26T22:50:27.617767+0000","last_active":"2024-05-26T22:50:28.610258+0000","last_peered":"2024-05-26T22:50:28.610258+0000","last_clean":"2024-05-26T22:50:28.610258+0000","last_became_active":"2024-05-26T22:50:27.613401+0000","last_became_peered":"2024-05-26T22:50:27.613401+0000","last_unstale":"2024-05-26T22:50:28.610258+0000","last_undegraded":"2024-05-26T22:50:28.610258+0000","last_fullsized":"2024-05-26T22:50:28.610258+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:46:46.492897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:29.031017+0000","last_change":"2024-05-26T22:50:26.587128+0000","last_active":"2024-05-26T22:50:29.031017+0000","last_peered":"2024-05-26T22:50:29.031017+0000","last_clean":"2024-05-26T22:50:29.031017+0000","last_became_active":"2024-05-26T22:50:26.586847+0000","last_became_peered":"2024-05-26T22:50:26.586847+0000","last_unstale":"2024-05-26T22:50:29.031017+0000","last_undegraded":"2024-05-26T22:50:29.031017+0000","last_fullsized":"2024-05-26T22:50:29.031017+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:35:15.400981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.610312+0000","last_change":"2024-05-26T22:50:27.625960+0000","last_active":"2024-05-26T22:50:28.610312+0000","last_peered":"2024-05-26T22:50:28.610312+0000","last_clean":"2024-05-26T22:50:28.610312+0000","last_became_active":"2024-05-26T22:50:27.625621+0000","last_became_peered":"2024-05-26T22:50:27.625621+0000","last_unstale":"2024-05-26T22:50:28.610312+0000","last_undegraded":"2024-05-26T22:50:28.610312+0000","last_fullsized":"2024-05-26T22:50:28.610312+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:26:50.858068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.616671+0000","last_change":"2024-05-26T22:50:26.585664+0000","last_active":"2024-05-26T22:50:28.616671+0000","last_peered":"2024-05-26T22:50:28.616671+0000","last_clean":"2024-05-26T22:50:28.616671+0000","last_became_active":"2024-05-26T22:50:26.585056+0000","last_became_peered":"2024-05-26T22:50:26.585056+0000","last_unstale":"2024-05-26T22:50:28.616671+0000","last_undegraded":"2024-05-26T22:50:28.616671+0000","last_fullsized":"2024-05-26T22:50:28.616671+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:07:38.331762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:29.029128+0000","last_change":"2024-05-26T22:50:27.612641+0000","last_active":"2024-05-26T22:50:29.029128+0000","last_peered":"2024-05-26T22:50:29.029128+0000","last_clean":"2024-05-26T22:50:29.029128+0000","last_became_active":"2024-05-26T22:50:27.612332+0000","last_became_peered":"2024-05-26T22:50:27.612332+0000","last_unstale":"2024-05-26T22:50:29.029128+0000","last_undegraded":"2024-05-26T22:50:29.029128+0000","last_fullsized":"2024-05-26T22:50:29.029128+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:44:14.357986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.605807+0000","last_change":"2024-05-26T22:50:26.579162+0000","last_active":"2024-05-26T22:50:28.605807+0000","last_peered":"2024-05-26T22:50:28.605807+0000","last_clean":"2024-05-26T22:50:28.605807+0000","last_became_active":"2024-05-26T22:50:26.578107+0000","last_became_peered":"2024-05-26T22:50:26.578107+0000","last_unstale":"2024-05-26T22:50:28.605807+0000","last_undegraded":"2024-05-26T22:50:28.605807+0000","last_fullsized":"2024-05-26T22:50:28.605807+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:58:28.776394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.616297+0000","last_change":"2024-05-26T22:50:27.619099+0000","last_active":"2024-05-26T22:50:28.616297+0000","last_peered":"2024-05-26T22:50:28.616297+0000","last_clean":"2024-05-26T22:50:28.616297+0000","last_became_active":"2024-05-26T22:50:27.616954+0000","last_became_peered":"2024-05-26T22:50:27.616954+0000","last_unstale":"2024-05-26T22:50:28.616297+0000","last_undegraded":"2024-05-26T22:50:28.616297+0000","last_fullsized":"2024-05-26T22:50:28.616297+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:29:04.881850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.606557+0000","last_change":"2024-05-26T22:50:26.577478+0000","last_active":"2024-05-26T22:50:28.606557+0000","last_peered":"2024-05-26T22:50:28.606557+0000","last_clean":"2024-05-26T22:50:28.606557+0000","last_became_active":"2024-05-26T22:50:26.576506+0000","last_became_peered":"2024-05-26T22:50:26.576506+0000","last_unstale":"2024-05-26T22:50:28.606557+0000","last_undegraded":"2024-05-26T22:50:28.606557+0000","last_fullsized":"2024-05-26T22:50:28.606557+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:07:38.331762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:28.608750+0000","last_change":"2024-05-26T22:50:27.620262+0000","last_active":"2024-05-26T22:50:28.608750+0000","last_peered":"2024-05-26T22:50:28.608750+0000","last_clean":"2024-05-26T22:50:28.608750+0000","last_became_active":"2024-05-26T22:50:27.619951+0000","last_became_peered":"2024-05-26T22:50:27.619951+0000","last_unstale":"2024-05-26T22:50:28.608750+0000","last_undegraded":"2024-05-26T22:50:28.608750+0000","last_fullsized":"2024-05-26T22:50:28.608750+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:26.540322+0000","last_clean_scrub_stamp":"2024-05-26T22:50:26.540322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:13:34.494539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"43'2","reported_seq":16,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:29.030612+0000","last_change":"2024-05-26T22:50:26.577747+0000","last_active":"2024-05-26T22:50:29.030612+0000","last_peered":"2024-05-26T22:50:29.030612+0000","last_clean":"2024-05-26T22:50:29.030612+0000","last_became_active":"2024-05-26T22:50:26.577408+0000","last_became_peered":"2024-05-26T22:50:26.577408+0000","last_unstale":"2024-05-26T22:50:29.030612+0000","last_undegraded":"2024-05-26T22:50:29.030612+0000","last_fullsized":"2024-05-26T22:50:29.030612+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:50:25.526760+0000","last_clean_scrub_stamp":"2024-05-26T22:50:25.526760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:08:59.520628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":215,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-05-26T22:50:29.029658+0000","last_change":"2024-05-26T22:48:12.068917+0000","last_active":"2024-05-26T22:50:29.029658+0000","last_peered":"2024-05-26T22:50:29.029658+0000","last_clean":"2024-05-26T22:50:29.029658+0000","last_became_active":"2024-05-26T22:48:12.065934+0000","last_became_peered":"2024-05-26T22:48:12.065934+0000","last_unstale":"2024-05-26T22:50:29.029658+0000","last_undegraded":"2024-05-26T22:50:29.029658+0000","last_fullsized":"2024-05-26T22:50:29.029658+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:10.628122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:10.628122+0000","last_clean_scrub_stamp":"2024-05-26T22:48:10.628122+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:14:58.895911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574880,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32268,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339572,"statfs":{"total":96636764160,"available":96603721728,"internally_reserved":0,"allocated":1114112,"data_stored":801258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574880,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27532,"kb_used_data":512,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344308,"statfs":{"total":96636764160,"available":96608571392,"internally_reserved":0,"allocated":524288,"data_stored":211472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574881,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27528,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344312,"statfs":{"total":96636764160,"available":96608575488,"internally_reserved":0,"allocated":520192,"data_stored":210870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574880,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27536,"kb_used_data":516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344304,"statfs":{"total":96636764160,"available":96608567296,"internally_reserved":0,"allocated":528384,"data_stored":212412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574880,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27544,"kb_used_data":516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344296,"statfs":{"total":96636764160,"available":96608559104,"internally_reserved":0,"allocated":528384,"data_stored":211438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574880,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27540,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344300,"statfs":{"total":96636764160,"available":96608563200,"internally_reserved":0,"allocated":532480,"data_stored":212434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574880,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27536,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344304,"statfs":{"total":96636764160,"available":96608567296,"internally_reserved":0,"allocated":520192,"data_stored":210870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574880,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32272,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339568,"statfs":{"total":96636764160,"available":96603717632,"internally_reserved":0,"allocated":1114112,"data_stored":801202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-26T22:50:37.417 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-26T22:50:37.417 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-26T22:50:37.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.670+0000 7ff8e0a00640 1 Processor -- start 2024-05-26T22:50:37.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.670+0000 7ff8e0a00640 1 -- start start 2024-05-26T22:50:37.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.672+0000 7ff8e0a00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7ff8dc06dd30 0x7ff8dc06e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:37.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.672+0000 7ff8e0a00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8dc15c1c0 con 0x7ff8dc0725a0 2024-05-26T22:50:37.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.672+0000 7ff8e0a00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8dc15c4a0 con 0x7ff8dc155770 2024-05-26T22:50:37.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.672+0000 7ff8e0a00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7ff8dc071da0 con 0x7ff8dc06dd30 2024-05-26T22:50:37.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.672+0000 7ff8d9400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7ff8dc06dd30 0x7ff8dc06e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:37.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.672+0000 7ff8d9400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7ff8dc06dd30 0x7ff8dc06e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41726/0 (socket says 172.21.15.131:41726) 2024-05-26T22:50:37.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.672+0000 7ff8d9400640 1 -- 172.21.15.131:0/285259842 learned_addr learned my addr 172.21.15.131:0/285259842 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:37.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.673+0000 7ff8d9400640 1 -- 172.21.15.131:0/285259842 >> v1:172.21.15.187:6789/0 conn(0x7ff8dc155770 legacy=0x7ff8dc15b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:37.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.673+0000 7ff8d9400640 1 -- 172.21.15.131:0/285259842 >> v1:172.21.15.131:6789/0 conn(0x7ff8dc0725a0 legacy=0x7ff8dc0729a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:37.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.673+0000 7ff8d9400640 1 -- 172.21.15.131:0/285259842 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8dc072080 con 0x7ff8dc06dd30 2024-05-26T22:50:37.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.674+0000 7ff8d9400640 1 --2- 172.21.15.131:0/285259842 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff8dc06dd30 0x7ff8dc06e1c0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7ff8c4009870 tx=0x7ff8c402c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb54ef953d29e120 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:37.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.674+0000 7ff8d8a00640 1 -- 172.21.15.131:0/285259842 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8c4038070 con 0x7ff8dc06dd30 2024-05-26T22:50:37.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.674+0000 7ff8d8a00640 1 -- 172.21.15.131:0/285259842 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8c4031710 con 0x7ff8dc06dd30 2024-05-26T22:50:37.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.674+0000 7ff8d8a00640 1 -- 172.21.15.131:0/285259842 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8c4031ae0 con 0x7ff8dc06dd30 2024-05-26T22:50:37.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.675+0000 7ff8e0a00640 1 -- 172.21.15.131:0/285259842 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff8dc06dd30 msgr2=0x7ff8dc06e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:37.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.675+0000 7ff8e0a00640 1 --2- 172.21.15.131:0/285259842 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff8dc06dd30 0x7ff8dc06e1c0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7ff8c4009870 tx=0x7ff8c402c540 comp rx=0 tx=0).stop 2024-05-26T22:50:37.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.675+0000 7ff8e0a00640 1 -- 172.21.15.131:0/285259842 shutdown_connections 2024-05-26T22:50:37.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.675+0000 7ff8e0a00640 1 --2- 172.21.15.131:0/285259842 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff8dc06dd30 0x7ff8dc06e1c0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:37.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.675+0000 7ff8e0a00640 1 -- 172.21.15.131:0/285259842 >> 172.21.15.131:0/285259842 conn(0x7ff8dc06bc90 msgr2=0x7ff8dc0b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:37.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.675+0000 7ff8e0a00640 1 -- 172.21.15.131:0/285259842 shutdown_connections 2024-05-26T22:50:37.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.675+0000 7ff8e0a00640 1 -- 172.21.15.131:0/285259842 wait complete. 2024-05-26T22:50:37.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.677+0000 7ff8e0a00640 1 Processor -- start 2024-05-26T22:50:37.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.677+0000 7ff8e0a00640 1 -- start start 2024-05-26T22:50:37.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.679+0000 7ff8e0a00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff8dc06dd30 0x7ff8dc0cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:37.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.679+0000 7ff8e0a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff8dc0725a0 0x7ff8dc14f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:37.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.679+0000 7ff8e0a00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff8dc155770 0x7ff8dc15a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:37.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.679+0000 7ff8e0a00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff8dc15be10 con 0x7ff8dc0725a0 2024-05-26T22:50:37.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.679+0000 7ff8e0a00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff8dc15c8c0 con 0x7ff8dc155770 2024-05-26T22:50:37.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.679+0000 7ff8e0a00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff8dc1f41f0 con 0x7ff8dc06dd30 2024-05-26T22:50:37.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.679+0000 7ff8d9400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff8dc0725a0 0x7ff8dc14f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:37.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.679+0000 7ff8d9400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff8dc0725a0 0x7ff8dc14f420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41734/0 (socket says 172.21.15.131:41734) 2024-05-26T22:50:37.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.679+0000 7ff8d9400640 1 -- 172.21.15.131:0/2293746508 learned_addr learned my addr 172.21.15.131:0/2293746508 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:37.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.679+0000 7ff8d9e00640 1 --2- 172.21.15.131:0/2293746508 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff8dc06dd30 0x7ff8dc0cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:37.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.679+0000 7ff8d9400640 1 -- 172.21.15.131:0/2293746508 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff8dc06dd30 msgr2=0x7ff8dc0cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:37.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.679+0000 7ff8d9400640 1 --2- 172.21.15.131:0/2293746508 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff8dc06dd30 0x7ff8dc0cf3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:37.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.679+0000 7ff8da800640 1 --2- 172.21.15.131:0/2293746508 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff8dc155770 0x7ff8dc15a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:37.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.679+0000 7ff8d9400640 1 -- 172.21.15.131:0/2293746508 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff8dc155770 msgr2=0x7ff8dc15a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:37.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.679+0000 7ff8d9400640 1 --2- 172.21.15.131:0/2293746508 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff8dc155770 0x7ff8dc15a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:37.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.679+0000 7ff8d9400640 1 -- 172.21.15.131:0/2293746508 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8dc1524c0 con 0x7ff8dc0725a0 2024-05-26T22:50:37.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.680+0000 7ff8d9e00640 1 --2- 172.21.15.131:0/2293746508 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff8dc06dd30 0x7ff8dc0cf3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:50:37.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.680+0000 7ff8d9400640 1 --2- 172.21.15.131:0/2293746508 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff8dc0725a0 0x7ff8dc14f420 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7ff8c4009870 tx=0x7ff8c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:37.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.680+0000 7ff8d2000640 1 -- 172.21.15.131:0/2293746508 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8c4038040 con 0x7ff8dc0725a0 2024-05-26T22:50:37.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.680+0000 7ff8d2000640 1 -- 172.21.15.131:0/2293746508 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8c4043070 con 0x7ff8dc0725a0 2024-05-26T22:50:37.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.680+0000 7ff8e0a00640 1 -- 172.21.15.131:0/2293746508 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8dc1527a0 con 0x7ff8dc0725a0 2024-05-26T22:50:37.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.680+0000 7ff8d2000640 1 -- 172.21.15.131:0/2293746508 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8c4033a70 con 0x7ff8dc0725a0 2024-05-26T22:50:37.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.680+0000 7ff8e0a00640 1 -- 172.21.15.131:0/2293746508 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8dc152d60 con 0x7ff8dc0725a0 2024-05-26T22:50:37.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.682+0000 7ff8e0a00640 1 -- 172.21.15.131:0/2293746508 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8dc074e90 con 0x7ff8dc0725a0 2024-05-26T22:50:37.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.683+0000 7ff8d2000640 1 -- 172.21.15.131:0/2293746508 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff8c4033c10 con 0x7ff8dc0725a0 2024-05-26T22:50:37.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.683+0000 7ff8d2000640 1 --2- 172.21.15.131:0/2293746508 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ff8b806be30 0x7ff8b806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:37.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.684+0000 7ff8d2000640 1 -- 172.21.15.131:0/2293746508 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7ff8c40abb90 con 0x7ff8dc0725a0 2024-05-26T22:50:37.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.684+0000 7ff8d9e00640 1 --2- 172.21.15.131:0/2293746508 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ff8b806be30 0x7ff8b806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:37.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.684+0000 7ff8d9e00640 1 --2- 172.21.15.131:0/2293746508 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ff8b806be30 0x7ff8b806e2f0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ff8cc007e90 tx=0x7ff8cc0073a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:37.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.688+0000 7ff8d2000640 1 -- 172.21.15.131:0/2293746508 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7ff8c4046360 con 0x7ff8dc0725a0 2024-05-26T22:50:37.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.957+0000 7ff8e0a00640 1 -- 172.21.15.131:0/2293746508 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7ff8dc0695c0 con 0x7ff8dc0725a0 2024-05-26T22:50:37.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.961+0000 7ff8d2000640 1 -- 172.21.15.131:0/2293746508 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v14) ==== 190+0+147 (secure 0 0 0) 0x7ff8c407a5d0 con 0x7ff8dc0725a0 2024-05-26T22:50:37.962 INFO:teuthology.orchestra.run.smithi131.stdout:[client.0] 2024-05-26T22:50:37.962 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQD2u1NmSGVWChAAH/wq3iaUs6ZELRhLKhmURQ== 2024-05-26T22:50:37.962 INFO:teuthology.orchestra.run.smithi131.stdout: caps mds = "allow" 2024-05-26T22:50:37.962 INFO:teuthology.orchestra.run.smithi131.stdout: caps mon = "allow r" 2024-05-26T22:50:37.962 INFO:teuthology.orchestra.run.smithi131.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-05-26T22:50:37.962 INFO:teuthology.orchestra.run.smithi131.stderr:updated caps for client.0 2024-05-26T22:50:37.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.964+0000 7ff8e0a00640 1 -- 172.21.15.131:0/2293746508 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ff8b806be30 msgr2=0x7ff8b806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:37.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.964+0000 7ff8e0a00640 1 --2- 172.21.15.131:0/2293746508 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ff8b806be30 0x7ff8b806e2f0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ff8cc007e90 tx=0x7ff8cc0073a0 comp rx=0 tx=0).stop 2024-05-26T22:50:37.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.964+0000 7ff8e0a00640 1 -- 172.21.15.131:0/2293746508 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff8dc0725a0 msgr2=0x7ff8dc14f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:37.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.964+0000 7ff8e0a00640 1 --2- 172.21.15.131:0/2293746508 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff8dc0725a0 0x7ff8dc14f420 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7ff8c4009870 tx=0x7ff8c40046c0 comp rx=0 tx=0).stop 2024-05-26T22:50:37.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.964+0000 7ff8e0a00640 1 -- 172.21.15.131:0/2293746508 shutdown_connections 2024-05-26T22:50:37.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.964+0000 7ff8e0a00640 1 --2- 172.21.15.131:0/2293746508 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ff8b806be30 0x7ff8b806e2f0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:37.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.964+0000 7ff8e0a00640 1 --2- 172.21.15.131:0/2293746508 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff8dc155770 0x7ff8dc15a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:37.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.964+0000 7ff8e0a00640 1 --2- 172.21.15.131:0/2293746508 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff8dc0725a0 0x7ff8dc14f420 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:37.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.964+0000 7ff8e0a00640 1 --2- 172.21.15.131:0/2293746508 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff8dc06dd30 0x7ff8dc0cf3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:37.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.964+0000 7ff8e0a00640 1 -- 172.21.15.131:0/2293746508 >> 172.21.15.131:0/2293746508 conn(0x7ff8dc06bc90 msgr2=0x7ff8dc0cfd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:37.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.965+0000 7ff8e0a00640 1 -- 172.21.15.131:0/2293746508 shutdown_connections 2024-05-26T22:50:37.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:37.965+0000 7ff8e0a00640 1 -- 172.21.15.131:0/2293746508 wait complete. 2024-05-26T22:50:37.984 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-26T22:50:38.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:38.182+0000 7ff39b000640 1 Processor -- start 2024-05-26T22:50:38.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:38.182+0000 7ff39b000640 1 -- start start 2024-05-26T22:50:38.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:38.184+0000 7ff39b000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff3940cc7f0 0x7ff3940ccca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:38.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:38.184+0000 7ff39b000640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff394160620 con 0x7ff394156750 2024-05-26T22:50:38.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:38.184+0000 7ff39b000640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff394160900 con 0x7ff39406e8c0 2024-05-26T22:50:38.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:38.184+0000 7ff39b000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff39415d770 con 0x7ff3940cc7f0 2024-05-26T22:50:38.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:38.184+0000 7ff399c00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7ff394156750 0x7ff39415fd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:58706/0 (socket says 172.21.15.131:58706) 2024-05-26T22:50:38.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:38.184+0000 7ff399c00640 1 -- 172.21.15.131:0/3421365881 learned_addr learned my addr 172.21.15.131:0/3421365881 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:50:38.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:38.184+0000 7ff39a600640 1 --2- 172.21.15.131:0/3421365881 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff3940cc7f0 0x7ff3940ccca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:38.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:38.184+0000 7ff393e00640 1 -- 172.21.15.131:0/3421365881 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 423474687 0 0) 0x7ff394160620 con 0x7ff394156750 2024-05-26T22:50:38.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:38.185+0000 7ff393e00640 1 -- 172.21.15.131:0/3421365881 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff37c003520 con 0x7ff394156750 2024-05-26T22:50:38.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:38.185+0000 7ff393e00640 1 -- 172.21.15.131:0/3421365881 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3369210079 0 0) 0x7ff37c003520 con 0x7ff394156750 2024-05-26T22:50:38.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:38.185+0000 7ff393e00640 1 -- 172.21.15.131:0/3421365881 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff3940cc7f0 msgr2=0x7ff3940ccca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:38.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:38.185+0000 7ff393e00640 1 --2- 172.21.15.131:0/3421365881 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff3940cc7f0 0x7ff3940ccca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:38.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:38.185+0000 7ff393e00640 1 -- 172.21.15.131:0/3421365881 >> v1:172.21.15.187:6789/0 conn(0x7ff39406e8c0 legacy=0x7ff39415d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:38.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:38.185+0000 7ff393e00640 1 -- 172.21.15.131:0/3421365881 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff394160be0 con 0x7ff394156750 2024-05-26T22:50:38.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:38.185+0000 7ff393e00640 1 -- 172.21.15.131:0/3421365881 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7ff388001670 con 0x7ff394156750 2024-05-26T22:50:38.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:38.185+0000 7ff39a600640 1 --2- 172.21.15.131:0/3421365881 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff3940cc7f0 0x7ff3940ccca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-26T22:50:38.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:38.185+0000 7ff393e00640 1 -- 172.21.15.131:0/3421365881 >> v1:172.21.15.131:6789/0 conn(0x7ff394156750 legacy=0x7ff39415fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:38.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:38.185+0000 7ff393e00640 1 --2- 172.21.15.131:0/3421365881 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff37c0039e0 0x7ff37c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:38.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:38.185+0000 7ff393e00640 1 --2- 172.21.15.131:0/3421365881 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff3940cc7f0 0x7ff37c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:38.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:38.185+0000 7ff393e00640 1 --2- 172.21.15.131:0/3421365881 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff37c004d40 0x7ff37c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:38.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:38.185+0000 7ff393e00640 1 -- 172.21.15.131:0/3421365881 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff39415d770 con 0x7ff37c004d40 2024-05-26T22:50:38.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:38.185+0000 7ff393e00640 1 -- 172.21.15.131:0/3421365881 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff37c00bad0 con 0x7ff37c0039e0 2024-05-26T22:50:38.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:38.185+0000 7ff393e00640 1 -- 172.21.15.131:0/3421365881 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff37c00bcd0 con 0x7ff3940cc7f0 2024-05-26T22:50:38.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:38.185+0000 7ff39a600640 1 --2- 172.21.15.131:0/3421365881 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff37c004d40 0x7ff37c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:38.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:38.186+0000 7ff399c00640 1 --2- 172.21.15.131:0/3421365881 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff37c0039e0 0x7ff37c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:38.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:38.186+0000 7ff399200640 1 --2- 172.21.15.131:0/3421365881 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff3940cc7f0 0x7ff37c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:38.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:38.186+0000 7ff39a600640 1 -- 172.21.15.131:0/3421365881 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff3940cc7f0 msgr2=0x7ff37c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:38.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:38.186+0000 7ff39a600640 1 --2- 172.21.15.131:0/3421365881 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff3940cc7f0 0x7ff37c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:38.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:38.186+0000 7ff39a600640 1 -- 172.21.15.131:0/3421365881 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff37c0039e0 msgr2=0x7ff37c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:38.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:38.186+0000 7ff39a600640 1 --2- 172.21.15.131:0/3421365881 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff37c0039e0 0x7ff37c003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:38.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:38.186+0000 7ff39a600640 1 -- 172.21.15.131:0/3421365881 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff37c003520 con 0x7ff37c004d40 2024-05-26T22:50:38.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:38.187+0000 7ff39a600640 1 --2- 172.21.15.131:0/3421365881 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff37c004d40 0x7ff37c0051c0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7ff38c0073b0 tx=0x7ff37c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaa7c6862f19b791 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:38.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:38.187+0000 7ff393e00640 1 -- 172.21.15.131:0/3421365881 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff38c009c80 con 0x7ff37c004d40 2024-05-26T22:50:38.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:38.187+0000 7ff393e00640 1 -- 172.21.15.131:0/3421365881 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff38c02e620 con 0x7ff37c004d40 2024-05-26T22:50:38.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:38.187+0000 7ff393e00640 1 -- 172.21.15.131:0/3421365881 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff38c02e9d0 con 0x7ff37c004d40 2024-05-26T22:50:38.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:38.188+0000 7ff39b000640 1 -- 172.21.15.131:0/3421365881 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff37c004d40 msgr2=0x7ff37c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:38.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:38.188+0000 7ff39b000640 1 --2- 172.21.15.131:0/3421365881 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff37c004d40 0x7ff37c0051c0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7ff38c0073b0 tx=0x7ff37c005830 comp rx=0 tx=0).stop 2024-05-26T22:50:38.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:38.188+0000 7ff39a600640 1 -- 172.21.15.131:0/3421365881 reap_dead start 2024-05-26T22:50:38.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:38.188+0000 7ff39b000640 1 -- 172.21.15.131:0/3421365881 shutdown_connections 2024-05-26T22:50:38.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:38.188+0000 7ff39b000640 1 -- 172.21.15.131:0/3421365881 >> 172.21.15.131:0/3421365881 conn(0x7ff3940cff40 msgr2=0x7ff3940d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:38.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:38.188+0000 7ff39b000640 1 -- 172.21.15.131:0/3421365881 shutdown_connections 2024-05-26T22:50:38.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:38.188+0000 7ff39b000640 1 -- 172.21.15.131:0/3421365881 wait complete. 2024-05-26T22:50:38.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:38.190+0000 7ff39b000640 1 Processor -- start 2024-05-26T22:50:38.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:38.190+0000 7ff39b000640 1 -- start start 2024-05-26T22:50:38.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:38.191+0000 7ff39b000640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff37c004d40 0x7ff394158030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:38.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:38.191+0000 7ff39b000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff3941486d0 0x7ff3940b2290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:38.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:38.191+0000 7ff39b000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff394158d10 0x7ff394159190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:38.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:38.191+0000 7ff39b000640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff394156b30 con 0x7ff3941486d0 2024-05-26T22:50:38.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:38.191+0000 7ff39b000640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff39415cbd0 con 0x7ff37c004d40 2024-05-26T22:50:38.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:38.191+0000 7ff39b000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff3940ccca0 con 0x7ff394158d10 2024-05-26T22:50:38.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:38.191+0000 7ff399200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff3941486d0 0x7ff3940b2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:38.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:38.191+0000 7ff399200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff3941486d0 0x7ff3940b2290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41750/0 (socket says 172.21.15.131:41750) 2024-05-26T22:50:38.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:38.191+0000 7ff399200640 1 -- 172.21.15.131:0/1129616944 learned_addr learned my addr 172.21.15.131:0/1129616944 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:38.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:38.191+0000 7ff39a600640 1 --2- 172.21.15.131:0/1129616944 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff394158d10 0x7ff394159190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:38.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:38.191+0000 7ff399c00640 1 --2- 172.21.15.131:0/1129616944 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff37c004d40 0x7ff394158030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:38.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:38.191+0000 7ff399200640 1 -- 172.21.15.131:0/1129616944 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff394158d10 msgr2=0x7ff394159190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:38.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:38.191+0000 7ff399200640 1 --2- 172.21.15.131:0/1129616944 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff394158d10 0x7ff394159190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:38.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:38.191+0000 7ff399200640 1 -- 172.21.15.131:0/1129616944 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff37c004d40 msgr2=0x7ff394158030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:38.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:38.191+0000 7ff399200640 1 --2- 172.21.15.131:0/1129616944 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff37c004d40 0x7ff394158030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:38.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:38.191+0000 7ff399200640 1 -- 172.21.15.131:0/1129616944 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff394155e00 con 0x7ff3941486d0 2024-05-26T22:50:38.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:38.192+0000 7ff399200640 1 --2- 172.21.15.131:0/1129616944 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff3941486d0 0x7ff3940b2290 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7ff38000e580 tx=0x7ff38000ea40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:38.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:38.192+0000 7ff391600640 1 -- 172.21.15.131:0/1129616944 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3800052e0 con 0x7ff3941486d0 2024-05-26T22:50:38.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:38.192+0000 7ff391600640 1 -- 172.21.15.131:0/1129616944 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff380018070 con 0x7ff3941486d0 2024-05-26T22:50:38.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:38.192+0000 7ff391600640 1 -- 172.21.15.131:0/1129616944 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff380003810 con 0x7ff3941486d0 2024-05-26T22:50:38.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:38.192+0000 7ff39b000640 1 -- 172.21.15.131:0/1129616944 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff394156cb0 con 0x7ff3941486d0 2024-05-26T22:50:38.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:38.192+0000 7ff39b000640 1 -- 172.21.15.131:0/1129616944 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff394157270 con 0x7ff3941486d0 2024-05-26T22:50:38.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:38.193+0000 7ff39b000640 1 -- 172.21.15.131:0/1129616944 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3940694e0 con 0x7ff3941486d0 2024-05-26T22:50:38.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:38.194+0000 7ff391600640 1 -- 172.21.15.131:0/1129616944 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff3800039b0 con 0x7ff3941486d0 2024-05-26T22:50:38.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:38.195+0000 7ff391600640 1 --2- 172.21.15.131:0/1129616944 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ff37406be60 0x7ff37406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:38.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:38.195+0000 7ff391600640 1 -- 172.21.15.131:0/1129616944 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7ff38008abd0 con 0x7ff3941486d0 2024-05-26T22:50:38.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:38.195+0000 7ff399c00640 1 --2- 172.21.15.131:0/1129616944 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ff37406be60 0x7ff37406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:38.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:38.196+0000 7ff399c00640 1 --2- 172.21.15.131:0/1129616944 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ff37406be60 0x7ff37406e320 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7ff388001eb0 tx=0x7ff388000ed0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:38.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:38.200+0000 7ff391600640 1 -- 172.21.15.131:0/1129616944 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7ff380059460 con 0x7ff3941486d0 2024-05-26T22:50:38.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:38.424+0000 7ff39b000640 1 -- 172.21.15.131:0/1129616944 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff3940adc30 con 0x7ff3941486d0 2024-05-26T22:50:38.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:38.425+0000 7ff391600640 1 -- 172.21.15.131:0/1129616944 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+5187 (secure 0 0 0) 0x7ff380058d60 con 0x7ff3941486d0 2024-05-26T22:50:38.425 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:50:38.425 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":29,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4661,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/265040431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6835","nonce":265040431},{"type":"v1","addr":"172.21.15.131:6837","nonce":265040431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24},{"gid":4670,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6834/1027783169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1027783169},{"type":"v1","addr":"172.21.15.187:6834","nonce":1027783169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24},{"gid":4918,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/3429891101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6833","nonce":3429891101},{"type":"v1","addr":"172.21.15.187:6835","nonce":3429891101}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-26T22:50:27.001665+0000","modified":"2024-05-26T22:50:30.097328+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5637},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5637":{"gid":5637,"name":"c","rank":0,"incarnation":25,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6836/650770835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":650770835},{"type":"v1","addr":"172.21.15.131:6836","nonce":650770835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5637,"qdb_cluster":[5637]},"id":3}]} 2024-05-26T22:50:38.425 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 29 2024-05-26T22:50:38.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:38.428+0000 7ff39b000640 1 -- 172.21.15.131:0/1129616944 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ff37406be60 msgr2=0x7ff37406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:38.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:38.428+0000 7ff39b000640 1 --2- 172.21.15.131:0/1129616944 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ff37406be60 0x7ff37406e320 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7ff388001eb0 tx=0x7ff388000ed0 comp rx=0 tx=0).stop 2024-05-26T22:50:38.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:38.428+0000 7ff39b000640 1 -- 172.21.15.131:0/1129616944 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff3941486d0 msgr2=0x7ff3940b2290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:38.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:38.428+0000 7ff39b000640 1 --2- 172.21.15.131:0/1129616944 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff3941486d0 0x7ff3940b2290 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7ff38000e580 tx=0x7ff38000ea40 comp rx=0 tx=0).stop 2024-05-26T22:50:38.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:38.428+0000 7ff39b000640 1 -- 172.21.15.131:0/1129616944 shutdown_connections 2024-05-26T22:50:38.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:38.428+0000 7ff39b000640 1 --2- 172.21.15.131:0/1129616944 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff394158d10 0x7ff394159190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:38.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:38.428+0000 7ff39b000640 1 --2- 172.21.15.131:0/1129616944 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ff37406be60 0x7ff37406e320 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:38.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:38.428+0000 7ff39b000640 1 --2- 172.21.15.131:0/1129616944 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff3941486d0 0x7ff3940b2290 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:38.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:38.428+0000 7ff39b000640 1 --2- 172.21.15.131:0/1129616944 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff37c004d40 0x7ff394158030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:38.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:38.428+0000 7ff39b000640 1 -- 172.21.15.131:0/1129616944 >> 172.21.15.131:0/1129616944 conn(0x7ff3940cff40 msgr2=0x7ff3940d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:38.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:38.428+0000 7ff39b000640 1 -- 172.21.15.131:0/1129616944 shutdown_connections 2024-05-26T22:50:38.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:38.429+0000 7ff39b000640 1 -- 172.21.15.131:0/1129616944 wait complete. 2024-05-26T22:50:38.850 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 29, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-26T22:50:27.001665+0000', 'modified': '2024-05-26T22:50:30.097328+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5637}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5637': {'gid': 5637, 'name': 'c', 'rank': 0, 'incarnation': 25, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.131:6836/650770835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6834', 'nonce': 650770835}, {'type': 'v1', 'addr': '172.21.15.131:6836', 'nonce': 650770835}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5637, 'qdb_cluster': [5637]} 2024-05-26T22:50:38.851 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-26T22:50:38.851 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-05-26T22:50:39.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.061+0000 7f8d95200640 1 Processor -- start 2024-05-26T22:50:39.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.061+0000 7f8d95200640 1 -- start start 2024-05-26T22:50:39.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.062+0000 7f8d95200640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f8d900b5450 0x7f8d900b43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:39.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.063+0000 7f8d95200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f8d9006f420 0x7f8d900cfd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:39.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.063+0000 7f8d95200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8d900cf590 0x7f8d900d0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:39.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.063+0000 7f8d95200640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f8d90160d90 con 0x7f8d9006f420 2024-05-26T22:50:39.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.063+0000 7f8d95200640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f8d900b4900 con 0x7f8d900b5450 2024-05-26T22:50:39.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.063+0000 7f8d95200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f8d90160180 con 0x7f8d900cf590 2024-05-26T22:50:39.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.063+0000 7f8d85400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f8d9006f420 0x7f8d900cfd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:39.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.063+0000 7f8d85400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f8d9006f420 0x7f8d900cfd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41754/0 (socket says 172.21.15.131:41754) 2024-05-26T22:50:39.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.063+0000 7f8d85400640 1 -- 172.21.15.131:0/1606098555 learned_addr learned my addr 172.21.15.131:0/1606098555 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:39.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.063+0000 7f8d85400640 1 -- 172.21.15.131:0/1606098555 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8d900cf590 msgr2=0x7f8d900d0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:39.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.063+0000 7f8d85400640 1 --2- 172.21.15.131:0/1606098555 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8d900cf590 0x7f8d900d0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:39.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.063+0000 7f8d85400640 1 -- 172.21.15.131:0/1606098555 >> v2:172.21.15.187:3300/0 conn(0x7f8d900b5450 msgr2=0x7f8d900b43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:39.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.063+0000 7f8d85400640 1 --2- 172.21.15.131:0/1606098555 >> v2:172.21.15.187:3300/0 conn(0x7f8d900b5450 0x7f8d900b43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:39.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.063+0000 7f8d85400640 1 -- 172.21.15.131:0/1606098555 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d90160460 con 0x7f8d9006f420 2024-05-26T22:50:39.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.064+0000 7f8d85400640 1 --2- 172.21.15.131:0/1606098555 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d9006f420 0x7f8d900cfd50 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f8d900d0c40 tx=0x7f8d7802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9539fcb7cf4b253 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:39.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.065+0000 7f8d8d400640 1 -- 172.21.15.131:0/1606098555 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d78037070 con 0x7f8d9006f420 2024-05-26T22:50:39.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.065+0000 7f8d8d400640 1 -- 172.21.15.131:0/1606098555 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d78032440 con 0x7f8d9006f420 2024-05-26T22:50:39.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.065+0000 7f8d8d400640 1 -- 172.21.15.131:0/1606098555 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d78032810 con 0x7f8d9006f420 2024-05-26T22:50:39.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.065+0000 7f8d95200640 1 -- 172.21.15.131:0/1606098555 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d9006f420 msgr2=0x7f8d900cfd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:39.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.065+0000 7f8d95200640 1 --2- 172.21.15.131:0/1606098555 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d9006f420 0x7f8d900cfd50 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f8d900d0c40 tx=0x7f8d7802a1e0 comp rx=0 tx=0).stop 2024-05-26T22:50:39.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.065+0000 7f8d95200640 1 -- 172.21.15.131:0/1606098555 shutdown_connections 2024-05-26T22:50:39.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.065+0000 7f8d95200640 1 --2- 172.21.15.131:0/1606098555 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8d900cf590 0x7f8d900d0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:39.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.065+0000 7f8d95200640 1 --2- 172.21.15.131:0/1606098555 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d9006f420 0x7f8d900cfd50 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:39.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.065+0000 7f8d95200640 1 --2- 172.21.15.131:0/1606098555 >> v2:172.21.15.187:3300/0 conn(0x7f8d900b5450 0x7f8d900b43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:39.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.065+0000 7f8d95200640 1 -- 172.21.15.131:0/1606098555 >> 172.21.15.131:0/1606098555 conn(0x7f8d9006bc90 msgr2=0x7f8d900b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:39.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.066+0000 7f8d95200640 1 -- 172.21.15.131:0/1606098555 shutdown_connections 2024-05-26T22:50:39.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.066+0000 7f8d95200640 1 -- 172.21.15.131:0/1606098555 wait complete. 2024-05-26T22:50:39.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.067+0000 7f8d95200640 1 Processor -- start 2024-05-26T22:50:39.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.067+0000 7f8d95200640 1 -- start start 2024-05-26T22:50:39.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.068+0000 7f8d95200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8d9006f420 0x7f8d9009bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:39.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.068+0000 7f8d95200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d900b5450 0x7f8d9009c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:39.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.068+0000 7f8d95200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d900cf590 0x7f8d900c9290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:39.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.068+0000 7f8d95200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8d900b5830 con 0x7f8d900b5450 2024-05-26T22:50:39.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.068+0000 7f8d95200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8d90160a20 con 0x7f8d900cf590 2024-05-26T22:50:39.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.068+0000 7f8d95200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f8d9015fde0 con 0x7f8d9006f420 2024-05-26T22:50:39.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.068+0000 7f8d85400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d900b5450 0x7f8d9009c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:39.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.068+0000 7f8d85400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d900b5450 0x7f8d9009c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41760/0 (socket says 172.21.15.131:41760) 2024-05-26T22:50:39.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.068+0000 7f8d85400640 1 -- 172.21.15.131:0/1227714419 learned_addr learned my addr 172.21.15.131:0/1227714419 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:39.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.069+0000 7f8d8e800640 1 --2- 172.21.15.131:0/1227714419 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d900cf590 0x7f8d900c9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:39.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.069+0000 7f8d85400640 1 -- 172.21.15.131:0/1227714419 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8d9006f420 msgr2=0x7f8d9009bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:39.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.069+0000 7f8d8de00640 1 --2- 172.21.15.131:0/1227714419 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8d9006f420 0x7f8d9009bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:39.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.069+0000 7f8d85400640 1 --2- 172.21.15.131:0/1227714419 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8d9006f420 0x7f8d9009bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:39.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.069+0000 7f8d85400640 1 -- 172.21.15.131:0/1227714419 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d900cf590 msgr2=0x7f8d900c9290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:39.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.069+0000 7f8d85400640 1 --2- 172.21.15.131:0/1227714419 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d900cf590 0x7f8d900c9290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:39.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.069+0000 7f8d85400640 1 -- 172.21.15.131:0/1227714419 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d901f8480 con 0x7f8d900b5450 2024-05-26T22:50:39.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.069+0000 7f8d85400640 1 --2- 172.21.15.131:0/1227714419 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d900b5450 0x7f8d9009c970 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f8d78000c00 tx=0x7f8d780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:39.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.069+0000 7f8d86a00640 1 -- 172.21.15.131:0/1227714419 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d78041070 con 0x7f8d900b5450 2024-05-26T22:50:39.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.069+0000 7f8d86a00640 1 -- 172.21.15.131:0/1227714419 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d7803c870 con 0x7f8d900b5450 2024-05-26T22:50:39.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.069+0000 7f8d86a00640 1 -- 172.21.15.131:0/1227714419 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d78037040 con 0x7f8d900b5450 2024-05-26T22:50:39.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.069+0000 7f8d95200640 1 -- 172.21.15.131:0/1227714419 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d9009c000 con 0x7f8d900b5450 2024-05-26T22:50:39.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.070+0000 7f8d95200640 1 -- 172.21.15.131:0/1227714419 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d901f7be0 con 0x7f8d900b5450 2024-05-26T22:50:39.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.072+0000 7f8d86a00640 1 -- 172.21.15.131:0/1227714419 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8d78044070 con 0x7f8d900b5450 2024-05-26T22:50:39.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.072+0000 7f8d86a00640 1 --2- 172.21.15.131:0/1227714419 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f8d6c06be30 0x7f8d6c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:39.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.072+0000 7f8d86a00640 1 -- 172.21.15.131:0/1227714419 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f8d780abc60 con 0x7f8d900b5450 2024-05-26T22:50:39.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.073+0000 7f8d8de00640 1 --2- 172.21.15.131:0/1227714419 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f8d6c06be30 0x7f8d6c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:39.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.073+0000 7f8d8de00640 1 --2- 172.21.15.131:0/1227714419 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f8d6c06be30 0x7f8d6c06e2f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f8d90159c90 tx=0x7f8d80008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:39.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.074+0000 7f8d97ffb740 1 Processor -- start 2024-05-26T22:50:39.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.074+0000 7f8d97ffb740 1 -- start start 2024-05-26T22:50:39.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.075+0000 7f8d97ffb740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2e91c8c70 0x55c2e917b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:39.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.075+0000 7f8d97ffb740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55c2e91abd90 0x55c2e91ab110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:39.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.075+0000 7f8d97ffb740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c2e91ab650 0x55c2e9180460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:39.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.075+0000 7f8d97ffb740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55c2e90e3610 con 0x55c2e91c8c70 2024-05-26T22:50:39.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.075+0000 7f8d97ffb740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55c2e908b5c0 con 0x55c2e91ab650 2024-05-26T22:50:39.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.075+0000 7f8d97ffb740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x55c2e90df5d0 con 0x55c2e91abd90 2024-05-26T22:50:39.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.075+0000 7f8d8de00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2e91c8c70 0x55c2e917b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:39.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.075+0000 7f8d8de00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2e91c8c70 0x55c2e917b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41770/0 (socket says 172.21.15.131:41770) 2024-05-26T22:50:39.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.075+0000 7f8d8de00640 1 -- 172.21.15.131:0/782100383 learned_addr learned my addr 172.21.15.131:0/782100383 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:39.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.075+0000 7f8d8de00640 1 -- 172.21.15.131:0/782100383 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55c2e91abd90 msgr2=0x55c2e91ab110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:39.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.075+0000 7f8d8de00640 1 --2- 172.21.15.131:0/782100383 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55c2e91abd90 0x55c2e91ab110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:39.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.075+0000 7f8d8de00640 1 -- 172.21.15.131:0/782100383 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c2e91ab650 msgr2=0x55c2e9180460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:39.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.075+0000 7f8d8de00640 1 --2- 172.21.15.131:0/782100383 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c2e91ab650 0x55c2e9180460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:39.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.075+0000 7f8d8de00640 1 -- 172.21.15.131:0/782100383 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2e8df72d0 con 0x55c2e91c8c70 2024-05-26T22:50:39.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.076+0000 7f8d8de00640 1 --2- 172.21.15.131:0/782100383 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2e91c8c70 0x55c2e917b890 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f8d8000cac0 tx=0x7f8d8000f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b0cecc64ee716d0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:39.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.076+0000 7f8d7ea00640 1 -- 172.21.15.131:0/782100383 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d80023070 con 0x55c2e91c8c70 2024-05-26T22:50:39.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.076+0000 7f8d7ea00640 1 -- 172.21.15.131:0/782100383 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d80013db0 con 0x55c2e91c8c70 2024-05-26T22:50:39.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.076+0000 7f8d7ea00640 1 -- 172.21.15.131:0/782100383 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d8001e600 con 0x55c2e91c8c70 2024-05-26T22:50:39.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.076+0000 7f8d97ffb740 1 -- 172.21.15.131:0/782100383 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2e91c8c70 msgr2=0x55c2e917b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:39.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.076+0000 7f8d97ffb740 1 --2- 172.21.15.131:0/782100383 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2e91c8c70 0x55c2e917b890 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f8d8000cac0 tx=0x7f8d8000f000 comp rx=0 tx=0).stop 2024-05-26T22:50:39.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.077+0000 7f8d97ffb740 1 -- 172.21.15.131:0/782100383 shutdown_connections 2024-05-26T22:50:39.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.077+0000 7f8d97ffb740 1 --2- 172.21.15.131:0/782100383 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55c2e91abd90 0x55c2e91ab110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:39.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.077+0000 7f8d97ffb740 1 --2- 172.21.15.131:0/782100383 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c2e91ab650 0x55c2e9180460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:39.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.077+0000 7f8d97ffb740 1 --2- 172.21.15.131:0/782100383 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2e91c8c70 0x55c2e917b890 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:39.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.077+0000 7f8d97ffb740 1 -- 172.21.15.131:0/782100383 >> 172.21.15.131:0/782100383 conn(0x55c2e917db00 msgr2=0x55c2e91c6110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:39.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.077+0000 7f8d97ffb740 1 -- 172.21.15.131:0/782100383 shutdown_connections 2024-05-26T22:50:39.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.077+0000 7f8d97ffb740 1 -- 172.21.15.131:0/782100383 wait complete. 2024-05-26T22:50:39.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.077+0000 7f8d97ffb740 1 -- start start 2024-05-26T22:50:39.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.078+0000 7f8d97ffb740 1 Processor -- start 2024-05-26T22:50:39.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.078+0000 7f8d97ffb740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2e91abd90 0x55c2e919bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:39.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.078+0000 7f8d97ffb740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c2e91c8c70 0x55c2e91acc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:39.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.079+0000 7f8d97ffb740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55c2e91ad180 0x55c2e91d5f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:39.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.079+0000 7f8d97ffb740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55c2e917fa20 con 0x55c2e91abd90 2024-05-26T22:50:39.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.079+0000 7f8d97ffb740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55c2e909e910 con 0x55c2e91c8c70 2024-05-26T22:50:39.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.079+0000 7f8d97ffb740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x55c2e917ee10 con 0x55c2e91ad180 2024-05-26T22:50:39.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.079+0000 7f8d97ffb740 10 client.0 _ll_register_callbacks cb 0x55c2e90da410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-26T22:50:39.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.079+0000 7f8d8de00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2e91abd90 0x55c2e919bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:39.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.079+0000 7f8d8de00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2e91abd90 0x55c2e919bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41772/0 (socket says 172.21.15.131:41772) 2024-05-26T22:50:39.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.079+0000 7f8d8de00640 1 -- 172.21.15.131:0/707640250 learned_addr learned my addr 172.21.15.131:0/707640250 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:39.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.079+0000 7f8d85400640 1 --2- 172.21.15.131:0/707640250 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c2e91c8c70 0x55c2e91acc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:39.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.079+0000 7f8d8de00640 1 -- 172.21.15.131:0/707640250 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55c2e91ad180 msgr2=0x55c2e91d5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:39.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.079+0000 7f8d8de00640 1 --2- 172.21.15.131:0/707640250 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55c2e91ad180 0x55c2e91d5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:39.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.079+0000 7f8d8de00640 1 -- 172.21.15.131:0/707640250 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c2e91c8c70 msgr2=0x55c2e91acc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:39.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.079+0000 7f8d8de00640 1 --2- 172.21.15.131:0/707640250 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c2e91c8c70 0x55c2e91acc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:39.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.079+0000 7f8d8de00640 1 -- 172.21.15.131:0/707640250 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2e8ec16b0 con 0x55c2e91abd90 2024-05-26T22:50:39.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.079+0000 7f8d85400640 1 --2- 172.21.15.131:0/707640250 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c2e91c8c70 0x55c2e91acc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:50:39.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.079+0000 7f8d8de00640 1 --2- 172.21.15.131:0/707640250 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2e91abd90 0x55c2e919bb40 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f8d8000a5f0 tx=0x7f8d800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d00f07f5d58295a server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:39.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.079+0000 7f8d73400640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-26T22:50:39.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.080+0000 7f8d73400640 1 -- 172.21.15.131:0/707640250 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d8002d070 con 0x55c2e91abd90 2024-05-26T22:50:39.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.080+0000 7f8d73400640 1 -- 172.21.15.131:0/707640250 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f8d80028930 con 0x55c2e91abd90 2024-05-26T22:50:39.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.080+0000 7f8d97ffb740 1 -- 172.21.15.131:0/707640250 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d80013db0 con 0x55c2e91abd90 2024-05-26T22:50:39.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.080+0000 7f8d73400640 1 -- 172.21.15.131:0/707640250 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8d80038050 con 0x55c2e91abd90 2024-05-26T22:50:39.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.080+0000 7f8d97ffb740 10 client.5823 fetch_fsmap learned FSMap version 29 2024-05-26T22:50:39.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.080+0000 7f8d97ffb740 1 -- 172.21.15.131:0/707640250 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) -- 0x55c2e91adc60 con 0x55c2e91abd90 2024-05-26T22:50:39.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.080+0000 7f8d73400640 1 -- 172.21.15.131:0/707640250 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d80013db0 con 0x55c2e91abd90 2024-05-26T22:50:39.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.080+0000 7f8d73400640 1 -- 172.21.15.131:0/707640250 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 29) ==== 3494+0+0 (secure 0 0 0) 0x7f8d8003b020 con 0x55c2e91abd90 2024-05-26T22:50:39.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.080+0000 7f8d97ffb740 10 client.5823 fetch_fsmap finished waiting for FSMap version 29 2024-05-26T22:50:39.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.080+0000 7f8d97ffb740 10 client.5823 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-26T22:50:39.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.080+0000 7f8d97ffb740 20 client.5823 populate_metadata read hostname 'smithi131' 2024-05-26T22:50:39.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.080+0000 7f8d97ffb740 1 --2- 172.21.15.131:0/707640250 >> [v2:172.21.15.131:6834/650770835,v1:172.21.15.131:6836/650770835] conn(0x55c2e91e1da0 0x55c2e91e4190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:39.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.080+0000 7f8d97ffb740 4 client.5823 mds_command: new command op to 5637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-26T22:50:39.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.080+0000 7f8d97ffb740 1 -- 172.21.15.131:0/707640250 --> [v2:172.21.15.131:6834/650770835,v1:172.21.15.131:6836/650770835] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c2e90f0d40 con 0x55c2e91e1da0 2024-05-26T22:50:39.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.081+0000 7f8d85400640 1 --2- 172.21.15.131:0/707640250 >> [v2:172.21.15.131:6834/650770835,v1:172.21.15.131:6836/650770835] conn(0x55c2e91e1da0 0x55c2e91e4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:39.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.081+0000 7f8d85400640 1 --2- 172.21.15.131:0/707640250 >> [v2:172.21.15.131:6834/650770835,v1:172.21.15.131:6836/650770835] conn(0x55c2e91e1da0 0x55c2e91e4190 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ac5d411b8b542ca server_cookie=9522adfbdb10e51 in_seq=0 out_seq=0 2024-05-26T22:50:39.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.081+0000 7f8d73400640 10 client.5823 ms_handle_connect on v2:172.21.15.131:6834/650770835 2024-05-26T22:50:39.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.082+0000 7f8d73400640 1 -- 172.21.15.131:0/707640250 <== mds.0 v2:172.21.15.131:6834/650770835 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f8d78082e50 con 0x55c2e91e1da0 2024-05-26T22:50:39.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.082+0000 7f8d73400640 10 client.5823 handle_command_reply: tid=0 2024-05-26T22:50:39.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.082+0000 7f8d97ffb740 1 client.5823 shutdown 2024-05-26T22:50:39.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.082+0000 7f8d97ffb740 1 -- 172.21.15.131:0/707640250 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2e91abd90 msgr2=0x55c2e919bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:39.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.082+0000 7f8d97ffb740 1 --2- 172.21.15.131:0/707640250 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2e91abd90 0x55c2e919bb40 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f8d8000a5f0 tx=0x7f8d800046c0 comp rx=0 tx=0).stop 2024-05-26T22:50:39.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.082+0000 7f8d97ffb740 1 -- 172.21.15.131:0/707640250 shutdown_connections 2024-05-26T22:50:39.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.082+0000 7f8d97ffb740 1 --2- 172.21.15.131:0/707640250 >> [v2:172.21.15.131:6834/650770835,v1:172.21.15.131:6836/650770835] conn(0x55c2e91e1da0 0x55c2e91e4190 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:39.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.082+0000 7f8d97ffb740 1 --2- 172.21.15.131:0/707640250 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55c2e91ad180 0x55c2e91d5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:39.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.082+0000 7f8d97ffb740 1 --2- 172.21.15.131:0/707640250 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c2e91c8c70 0x55c2e91acc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:39.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.082+0000 7f8d73400640 0 client.5823 ms_handle_reset on v2:172.21.15.131:6834/650770835 2024-05-26T22:50:39.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.082+0000 7f8d97ffb740 1 --2- 172.21.15.131:0/707640250 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2e91abd90 0x55c2e919bb40 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:39.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.082+0000 7f8d97ffb740 1 -- 172.21.15.131:0/707640250 >> 172.21.15.131:0/707640250 conn(0x55c2e91ab650 msgr2=0x55c2e91c9530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:39.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.083+0000 7f8d97ffb740 1 -- 172.21.15.131:0/707640250 shutdown_connections 2024-05-26T22:50:39.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.083+0000 7f8d97ffb740 1 -- 172.21.15.131:0/707640250 wait complete. 2024-05-26T22:50:39.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.083+0000 7f8d97ffb740 20 client.5823 trim_cache size 0 max 16384 2024-05-26T22:50:39.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.102+0000 7f8d97ffb740 1 Processor -- start 2024-05-26T22:50:39.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.102+0000 7f8d97ffb740 1 -- start start 2024-05-26T22:50:39.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.103+0000 7f8d97ffb740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55c2e91abd90 0x55c2e91ac430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:39.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.103+0000 7f8d97ffb740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c2e91ad180 0x55c2e917e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:39.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.103+0000 7f8d97ffb740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2e91c8c70 0x55c2e9180f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:39.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.103+0000 7f8d97ffb740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8d9009ab90 con 0x55c2e91c8c70 2024-05-26T22:50:39.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.103+0000 7f8d97ffb740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8d90156110 con 0x55c2e91ad180 2024-05-26T22:50:39.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.103+0000 7f8d97ffb740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f8d901f4c30 con 0x55c2e91abd90 2024-05-26T22:50:39.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.103+0000 7f8d8e800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2e91c8c70 0x55c2e9180f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:39.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.103+0000 7f8d8e800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2e91c8c70 0x55c2e9180f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41782/0 (socket says 172.21.15.131:41782) 2024-05-26T22:50:39.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.103+0000 7f8d8e800640 1 -- 172.21.15.131:0/2789529639 learned_addr learned my addr 172.21.15.131:0/2789529639 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:39.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.103+0000 7f8d85400640 1 --2- 172.21.15.131:0/2789529639 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c2e91ad180 0x55c2e917e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:39.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.103+0000 7f8d8e800640 1 -- 172.21.15.131:0/2789529639 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55c2e91abd90 msgr2=0x55c2e91ac430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:39.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.103+0000 7f8d8e800640 1 --2- 172.21.15.131:0/2789529639 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55c2e91abd90 0x55c2e91ac430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:39.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.103+0000 7f8d8e800640 1 -- 172.21.15.131:0/2789529639 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c2e91ad180 msgr2=0x55c2e917e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:39.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.103+0000 7f8d8e800640 1 --2- 172.21.15.131:0/2789529639 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c2e91ad180 0x55c2e917e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:39.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.103+0000 7f8d8e800640 1 -- 172.21.15.131:0/2789529639 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2e91bf8d0 con 0x55c2e91c8c70 2024-05-26T22:50:39.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.104+0000 7f8d8e800640 1 --2- 172.21.15.131:0/2789529639 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2e91c8c70 0x55c2e9180f50 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f8d88008ba0 tx=0x7f8d8800d1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=275ef8917aa6fa3e server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:39.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.104+0000 7f8d7e000640 1 -- 172.21.15.131:0/2789529639 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d88016020 con 0x55c2e91c8c70 2024-05-26T22:50:39.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.104+0000 7f8d7e000640 1 -- 172.21.15.131:0/2789529639 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d8800dd80 con 0x55c2e91c8c70 2024-05-26T22:50:39.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.104+0000 7f8d7e000640 1 -- 172.21.15.131:0/2789529639 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d88012680 con 0x55c2e91c8c70 2024-05-26T22:50:39.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.105+0000 7f8d97ffb740 1 -- 172.21.15.131:0/2789529639 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2e91c8c70 msgr2=0x55c2e9180f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:39.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.105+0000 7f8d97ffb740 1 --2- 172.21.15.131:0/2789529639 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2e91c8c70 0x55c2e9180f50 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f8d88008ba0 tx=0x7f8d8800d1a0 comp rx=0 tx=0).stop 2024-05-26T22:50:39.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.105+0000 7f8d97ffb740 1 -- 172.21.15.131:0/2789529639 shutdown_connections 2024-05-26T22:50:39.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.105+0000 7f8d97ffb740 1 --2- 172.21.15.131:0/2789529639 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2e91c8c70 0x55c2e9180f50 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:39.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.105+0000 7f8d97ffb740 1 --2- 172.21.15.131:0/2789529639 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c2e91ad180 0x55c2e917e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:39.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.105+0000 7f8d97ffb740 1 --2- 172.21.15.131:0/2789529639 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55c2e91abd90 0x55c2e91ac430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:39.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.105+0000 7f8d97ffb740 1 -- 172.21.15.131:0/2789529639 >> 172.21.15.131:0/2789529639 conn(0x55c2e91ab650 msgr2=0x55c2e91c7f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:39.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.105+0000 7f8d97ffb740 1 -- 172.21.15.131:0/2789529639 shutdown_connections 2024-05-26T22:50:39.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.105+0000 7f8d97ffb740 1 -- 172.21.15.131:0/2789529639 wait complete. 2024-05-26T22:50:39.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.105+0000 7f8d97ffb740 1 -- start start 2024-05-26T22:50:39.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.106+0000 7f8d97ffb740 1 Processor -- start 2024-05-26T22:50:39.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.107+0000 7f8d97ffb740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55c2e91abd90 0x55c2e924cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:39.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.107+0000 7f8d97ffb740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2e91ad180 0x55c2e9250b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:39.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.107+0000 7f8d97ffb740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c2e91c8c70 0x55c2e92510d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:39.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.107+0000 7f8d97ffb740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55c2e91ae590 con 0x55c2e91ad180 2024-05-26T22:50:39.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.107+0000 7f8d97ffb740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55c2e917fa20 con 0x55c2e91c8c70 2024-05-26T22:50:39.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.107+0000 7f8d97ffb740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x55c2e91be420 con 0x55c2e91abd90 2024-05-26T22:50:39.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.107+0000 7f8d97ffb740 10 client.0 _ll_register_callbacks cb 0x7f8d80021070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-26T22:50:39.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.107+0000 7f8d85400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2e91ad180 0x55c2e9250b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:39.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.107+0000 7f8d85400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2e91ad180 0x55c2e9250b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41796/0 (socket says 172.21.15.131:41796) 2024-05-26T22:50:39.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.107+0000 7f8d85400640 1 -- 172.21.15.131:0/821141397 learned_addr learned my addr 172.21.15.131:0/821141397 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:39.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.107+0000 7f8d85400640 1 -- 172.21.15.131:0/821141397 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55c2e91abd90 msgr2=0x55c2e924cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:39.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.107+0000 7f8d85400640 1 --2- 172.21.15.131:0/821141397 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55c2e91abd90 0x55c2e924cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:39.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.107+0000 7f8d85400640 1 -- 172.21.15.131:0/821141397 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c2e91c8c70 msgr2=0x55c2e92510d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:39.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.107+0000 7f8d85400640 1 --2- 172.21.15.131:0/821141397 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c2e91c8c70 0x55c2e92510d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:39.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.107+0000 7f8d85400640 1 -- 172.21.15.131:0/821141397 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2e924d590 con 0x55c2e91ad180 2024-05-26T22:50:39.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.107+0000 7f8d85400640 1 --2- 172.21.15.131:0/821141397 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2e91ad180 0x55c2e9250b90 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f8d780aa180 tx=0x7f8d7807e490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bab91b9ab2a59be server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:39.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.107+0000 7f8d73400640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-26T22:50:39.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.108+0000 7f8d73400640 1 -- 172.21.15.131:0/821141397 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d780a8060 con 0x55c2e91ad180 2024-05-26T22:50:39.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.108+0000 7f8d73400640 1 -- 172.21.15.131:0/821141397 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f8d78076690 con 0x55c2e91ad180 2024-05-26T22:50:39.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.108+0000 7f8d97ffb740 1 -- 172.21.15.131:0/821141397 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d8800dd80 con 0x55c2e91ad180 2024-05-26T22:50:39.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.108+0000 7f8d73400640 1 -- 172.21.15.131:0/821141397 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8d78087050 con 0x55c2e91ad180 2024-05-26T22:50:39.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.108+0000 7f8d97ffb740 10 client.5829 fetch_fsmap learned FSMap version 29 2024-05-26T22:50:39.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.108+0000 7f8d97ffb740 1 -- 172.21.15.131:0/821141397 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) -- 0x55c2e91acf10 con 0x55c2e91ad180 2024-05-26T22:50:39.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.108+0000 7f8d73400640 1 -- 172.21.15.131:0/821141397 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d8800dd80 con 0x55c2e91ad180 2024-05-26T22:50:39.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.108+0000 7f8d73400640 1 -- 172.21.15.131:0/821141397 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 29) ==== 3494+0+0 (secure 0 0 0) 0x7f8d7808a020 con 0x55c2e91ad180 2024-05-26T22:50:39.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.108+0000 7f8d97ffb740 10 client.5829 fetch_fsmap finished waiting for FSMap version 29 2024-05-26T22:50:39.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.108+0000 7f8d97ffb740 10 client.5829 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-26T22:50:39.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.108+0000 7f8d97ffb740 20 client.5829 populate_metadata read hostname 'smithi131' 2024-05-26T22:50:39.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.108+0000 7f8d97ffb740 1 --2- 172.21.15.131:0/821141397 >> [v2:172.21.15.131:6834/650770835,v1:172.21.15.131:6836/650770835] conn(0x55c2e91bbaa0 0x55c2e91bbe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:39.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.108+0000 7f8d97ffb740 4 client.5829 mds_command: new command op to 5637 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-26T22:50:39.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.108+0000 7f8d97ffb740 1 -- 172.21.15.131:0/821141397 --> [v2:172.21.15.131:6834/650770835,v1:172.21.15.131:6836/650770835] -- command(tid 0: {"prefix": "status"}) -- 0x55c2e90eb090 con 0x55c2e91bbaa0 2024-05-26T22:50:39.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.108+0000 7f8d8de00640 1 --2- 172.21.15.131:0/821141397 >> [v2:172.21.15.131:6834/650770835,v1:172.21.15.131:6836/650770835] conn(0x55c2e91bbaa0 0x55c2e91bbe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:39.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.109+0000 7f8d73400640 10 client.5829 ms_handle_connect on v2:172.21.15.131:6834/650770835 2024-05-26T22:50:39.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.109+0000 7f8d8de00640 1 --2- 172.21.15.131:0/821141397 >> [v2:172.21.15.131:6834/650770835,v1:172.21.15.131:6836/650770835] conn(0x55c2e91bbaa0 0x55c2e91bbe80 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86bcd3b592ed68b7 server_cookie=7ecae9e67c5df19 in_seq=0 out_seq=0 2024-05-26T22:50:39.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.109+0000 7f8d73400640 1 -- 172.21.15.131:0/821141397 <== mds.0 v2:172.21.15.131:6834/650770835 1 ==== command_reply(tid 0: 0 ) ==== 8+0+319 (crc 0 0 0) 0x55c2e90eb090 con 0x55c2e91bbaa0 2024-05-26T22:50:39.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.109+0000 7f8d73400640 10 client.5829 handle_command_reply: tid=0 2024-05-26T22:50:39.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.109+0000 7f8d97ffb740 1 client.5829 shutdown 2024-05-26T22:50:39.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.109+0000 7f8d97ffb740 1 -- 172.21.15.131:0/821141397 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2e91ad180 msgr2=0x55c2e9250b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:39.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.109+0000 7f8d97ffb740 1 --2- 172.21.15.131:0/821141397 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2e91ad180 0x55c2e9250b90 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f8d780aa180 tx=0x7f8d7807e490 comp rx=0 tx=0).stop 2024-05-26T22:50:39.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.109+0000 7f8d97ffb740 1 -- 172.21.15.131:0/821141397 shutdown_connections 2024-05-26T22:50:39.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.109+0000 7f8d97ffb740 1 --2- 172.21.15.131:0/821141397 >> [v2:172.21.15.131:6834/650770835,v1:172.21.15.131:6836/650770835] conn(0x55c2e91bbaa0 0x55c2e91bbe80 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:39.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.109+0000 7f8d97ffb740 1 --2- 172.21.15.131:0/821141397 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c2e91c8c70 0x55c2e92510d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:39.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.109+0000 7f8d97ffb740 1 --2- 172.21.15.131:0/821141397 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c2e91ad180 0x55c2e9250b90 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:39.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.109+0000 7f8d97ffb740 1 --2- 172.21.15.131:0/821141397 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55c2e91abd90 0x55c2e924cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:39.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.109+0000 7f8d73400640 0 client.5829 ms_handle_reset on v2:172.21.15.131:6834/650770835 2024-05-26T22:50:39.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.109+0000 7f8d97ffb740 1 -- 172.21.15.131:0/821141397 >> 172.21.15.131:0/821141397 conn(0x55c2e91ab650 msgr2=0x55c2e917e120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:39.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.110+0000 7f8d97ffb740 1 -- 172.21.15.131:0/821141397 shutdown_connections 2024-05-26T22:50:39.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.110+0000 7f8d97ffb740 1 -- 172.21.15.131:0/821141397 wait complete. 2024-05-26T22:50:39.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.110+0000 7f8d97ffb740 20 client.5829 trim_cache size 0 max 16384 2024-05-26T22:50:39.114 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-26T22:50:39.114 INFO:teuthology.orchestra.run.smithi131.stdout: "cluster_fsid": "46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f", 2024-05-26T22:50:39.114 INFO:teuthology.orchestra.run.smithi131.stdout: "whoami": 0, 2024-05-26T22:50:39.114 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 5637, 2024-05-26T22:50:39.114 INFO:teuthology.orchestra.run.smithi131.stdout: "want_state": "up:active", 2024-05-26T22:50:39.114 INFO:teuthology.orchestra.run.smithi131.stdout: "state": "up:active", 2024-05-26T22:50:39.114 INFO:teuthology.orchestra.run.smithi131.stdout: "fs_name": "cephfs", 2024-05-26T22:50:39.114 INFO:teuthology.orchestra.run.smithi131.stdout: "rank_uptime": 11.515015169, 2024-05-26T22:50:39.114 INFO:teuthology.orchestra.run.smithi131.stdout: "mdsmap_epoch": 29, 2024-05-26T22:50:39.114 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch": 44, 2024-05-26T22:50:39.114 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch_barrier": 44, 2024-05-26T22:50:39.115 INFO:teuthology.orchestra.run.smithi131.stdout: "uptime": 17.014488427 2024-05-26T22:50:39.115 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-26T22:50:39.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.110+0000 7f8d95200640 1 -- 172.21.15.131:0/1227714419 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f8d6c06be30 msgr2=0x7f8d6c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:39.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.110+0000 7f8d95200640 1 --2- 172.21.15.131:0/1227714419 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f8d6c06be30 0x7f8d6c06e2f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f8d90159c90 tx=0x7f8d80008040 comp rx=0 tx=0).stop 2024-05-26T22:50:39.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.110+0000 7f8d95200640 1 -- 172.21.15.131:0/1227714419 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d900b5450 msgr2=0x7f8d9009c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:39.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.110+0000 7f8d95200640 1 --2- 172.21.15.131:0/1227714419 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d900b5450 0x7f8d9009c970 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f8d78000c00 tx=0x7f8d780046c0 comp rx=0 tx=0).stop 2024-05-26T22:50:39.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.110+0000 7f8d95200640 1 -- 172.21.15.131:0/1227714419 shutdown_connections 2024-05-26T22:50:39.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.110+0000 7f8d95200640 1 --2- 172.21.15.131:0/1227714419 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f8d6c06be30 0x7f8d6c06e2f0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:39.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.110+0000 7f8d95200640 1 --2- 172.21.15.131:0/1227714419 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d900cf590 0x7f8d900c9290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:39.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.110+0000 7f8d95200640 1 --2- 172.21.15.131:0/1227714419 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d900b5450 0x7f8d9009c970 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:39.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.110+0000 7f8d95200640 1 --2- 172.21.15.131:0/1227714419 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8d9006f420 0x7f8d9009bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:39.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.110+0000 7f8d95200640 1 -- 172.21.15.131:0/1227714419 >> 172.21.15.131:0/1227714419 conn(0x7f8d9006bc90 msgr2=0x7f8d9015a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:39.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.111+0000 7f8d95200640 1 -- 172.21.15.131:0/1227714419 shutdown_connections 2024-05-26T22:50:39.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.111+0000 7f8d95200640 1 -- 172.21.15.131:0/1227714419 wait complete. 2024-05-26T22:50:39.126 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-26T22:50:39.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.315+0000 7f5a85000640 1 Processor -- start 2024-05-26T22:50:39.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.315+0000 7f5a85000640 1 -- start start 2024-05-26T22:50:39.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.317+0000 7f5a85000640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f5a8015e580 0x7f5a800d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:39.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.317+0000 7f5a85000640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f5a8015e960 0x7f5a800b3020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:39.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.317+0000 7f5a85000640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a800d1820 con 0x7f5a800b3560 2024-05-26T22:50:39.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.317+0000 7f5a85000640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f5a800d2820 con 0x7f5a8015e960 2024-05-26T22:50:39.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.317+0000 7f5a85000640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f5a800d02d0 con 0x7f5a8015e580 2024-05-26T22:50:39.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.317+0000 7f5a7e800640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f5a800b3560 0x7f5a800d2a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:58712/0 (socket says 172.21.15.131:58712) 2024-05-26T22:50:39.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.317+0000 7f5a7e800640 1 -- 172.21.15.131:0/386139264 learned_addr learned my addr 172.21.15.131:0/386139264 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:50:39.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.318+0000 7f5a7d400640 1 --2- 172.21.15.131:0/386139264 >> v2:172.21.15.131:3300/0 conn(0x7f5a8015e960 0x7f5a800b3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:39.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.318+0000 7f5a7de00640 1 --2- 172.21.15.131:0/386139264 >> v2:172.21.15.187:3300/0 conn(0x7f5a8015e580 0x7f5a800d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:39.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.318+0000 7f5a7ca00640 1 -- 172.21.15.131:0/386139264 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3904440623 0 0) 0x7f5a800d1820 con 0x7f5a800b3560 2024-05-26T22:50:39.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.318+0000 7f5a7ca00640 1 -- 172.21.15.131:0/386139264 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5a60003520 con 0x7f5a800b3560 2024-05-26T22:50:39.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.318+0000 7f5a7d400640 1 -- 172.21.15.131:0/386139264 >> v2:172.21.15.187:3300/0 conn(0x7f5a8015e580 msgr2=0x7f5a800d22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:39.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.318+0000 7f5a7d400640 1 --2- 172.21.15.131:0/386139264 >> v2:172.21.15.187:3300/0 conn(0x7f5a8015e580 0x7f5a800d22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:39.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.318+0000 7f5a7d400640 1 -- 172.21.15.131:0/386139264 >> v1:172.21.15.131:6789/0 conn(0x7f5a800b3560 legacy=0x7f5a800d2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:39.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.318+0000 7f5a7d400640 1 -- 172.21.15.131:0/386139264 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a800d4ce0 con 0x7f5a8015e960 2024-05-26T22:50:39.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.319+0000 7f5a7d400640 1 --2- 172.21.15.131:0/386139264 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a8015e960 0x7f5a800b3020 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f5a80068090 tx=0x7f5a6802a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8f6034cfad0b9e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:39.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.320+0000 7f5a7ca00640 1 -- 172.21.15.131:0/386139264 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a68007740 con 0x7f5a8015e960 2024-05-26T22:50:39.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.320+0000 7f5a7ca00640 1 -- 172.21.15.131:0/386139264 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a680078e0 con 0x7f5a8015e960 2024-05-26T22:50:39.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.320+0000 7f5a7ca00640 1 -- 172.21.15.131:0/386139264 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a68007c90 con 0x7f5a8015e960 2024-05-26T22:50:39.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.320+0000 7f5a85000640 1 -- 172.21.15.131:0/386139264 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a8015e960 msgr2=0x7f5a800b3020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:39.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.320+0000 7f5a85000640 1 --2- 172.21.15.131:0/386139264 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a8015e960 0x7f5a800b3020 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f5a80068090 tx=0x7f5a6802a5f0 comp rx=0 tx=0).stop 2024-05-26T22:50:39.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.320+0000 7f5a85000640 1 -- 172.21.15.131:0/386139264 shutdown_connections 2024-05-26T22:50:39.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.320+0000 7f5a85000640 1 --2- 172.21.15.131:0/386139264 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a8015e960 0x7f5a800b3020 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:39.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.320+0000 7f5a85000640 1 --2- 172.21.15.131:0/386139264 >> v2:172.21.15.187:3300/0 conn(0x7f5a8015e580 0x7f5a800d22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:39.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.320+0000 7f5a85000640 1 -- 172.21.15.131:0/386139264 >> 172.21.15.131:0/386139264 conn(0x7f5a8009d680 msgr2=0x7f5a800d4430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:39.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.320+0000 7f5a85000640 1 -- 172.21.15.131:0/386139264 shutdown_connections 2024-05-26T22:50:39.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.320+0000 7f5a85000640 1 -- 172.21.15.131:0/386139264 wait complete. 2024-05-26T22:50:39.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.322+0000 7f5a85000640 1 Processor -- start 2024-05-26T22:50:39.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.322+0000 7f5a85000640 1 -- start start 2024-05-26T22:50:39.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.323+0000 7f5a85000640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5a800b3560 0x7f5a80152d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:39.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.323+0000 7f5a85000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5a8015e580 0x7f5a800b1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:39.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.323+0000 7f5a85000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a8015e960 0x7f5a800b2500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:39.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.323+0000 7f5a85000640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f5a800d3120 con 0x7f5a8015e960 2024-05-26T22:50:39.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.323+0000 7f5a85000640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5a800d52a0 con 0x7f5a800b3560 2024-05-26T22:50:39.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.323+0000 7f5a85000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f5a800d32a0 con 0x7f5a8015e580 2024-05-26T22:50:39.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.324+0000 7f5a7e800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a8015e960 0x7f5a800b2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:39.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.324+0000 7f5a7e800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a8015e960 0x7f5a800b2500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41804/0 (socket says 172.21.15.131:41804) 2024-05-26T22:50:39.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.324+0000 7f5a7e800640 1 -- 172.21.15.131:0/3168960777 learned_addr learned my addr 172.21.15.131:0/3168960777 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:39.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.324+0000 7f5a7d400640 1 --2- 172.21.15.131:0/3168960777 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5a8015e580 0x7f5a800b1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:39.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.324+0000 7f5a7e800640 1 -- 172.21.15.131:0/3168960777 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5a8015e580 msgr2=0x7f5a800b1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:39.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.324+0000 7f5a7de00640 1 --2- 172.21.15.131:0/3168960777 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5a800b3560 0x7f5a80152d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:39.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.324+0000 7f5a7e800640 1 --2- 172.21.15.131:0/3168960777 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5a8015e580 0x7f5a800b1fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:39.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.324+0000 7f5a7e800640 1 -- 172.21.15.131:0/3168960777 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5a800b3560 msgr2=0x7f5a80152d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:39.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.324+0000 7f5a7e800640 1 --2- 172.21.15.131:0/3168960777 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5a800b3560 0x7f5a80152d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:39.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.324+0000 7f5a7e800640 1 -- 172.21.15.131:0/3168960777 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a800a1920 con 0x7f5a8015e960 2024-05-26T22:50:39.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.324+0000 7f5a7e800640 1 --2- 172.21.15.131:0/3168960777 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a8015e960 0x7f5a800b2500 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f5a7800c390 tx=0x7f5a7800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:39.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.325+0000 7f5a76000640 1 -- 172.21.15.131:0/3168960777 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a78017070 con 0x7f5a8015e960 2024-05-26T22:50:39.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.325+0000 7f5a76000640 1 -- 172.21.15.131:0/3168960777 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a78005ce0 con 0x7f5a8015e960 2024-05-26T22:50:39.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.325+0000 7f5a76000640 1 -- 172.21.15.131:0/3168960777 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a780072d0 con 0x7f5a8015e960 2024-05-26T22:50:39.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.325+0000 7f5a85000640 1 -- 172.21.15.131:0/3168960777 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a800a1c00 con 0x7f5a8015e960 2024-05-26T22:50:39.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.325+0000 7f5a85000640 1 -- 172.21.15.131:0/3168960777 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a8009e300 con 0x7f5a8015e960 2024-05-26T22:50:39.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.326+0000 7f5a85000640 1 -- 172.21.15.131:0/3168960777 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a8009e0f0 con 0x7f5a8015e960 2024-05-26T22:50:39.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.327+0000 7f5a76000640 1 -- 172.21.15.131:0/3168960777 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5a7801a070 con 0x7f5a8015e960 2024-05-26T22:50:39.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.328+0000 7f5a76000640 1 --2- 172.21.15.131:0/3168960777 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f5a5c06be30 0x7f5a5c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:39.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.328+0000 7f5a76000640 1 -- 172.21.15.131:0/3168960777 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f5a7808a190 con 0x7f5a8015e960 2024-05-26T22:50:39.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.329+0000 7f5a7de00640 1 --2- 172.21.15.131:0/3168960777 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f5a5c06be30 0x7f5a5c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:39.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.329+0000 7f5a7de00640 1 --2- 172.21.15.131:0/3168960777 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f5a5c06be30 0x7f5a5c06e2f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f5a801f7ae0 tx=0x7f5a70008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:39.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.333+0000 7f5a76000640 1 -- 172.21.15.131:0/3168960777 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f5a78058450 con 0x7f5a8015e960 2024-05-26T22:50:39.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.554+0000 7f5a85000640 1 -- 172.21.15.131:0/3168960777 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f5a800694e0 con 0x7f5a8015e960 2024-05-26T22:50:39.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.554+0000 7f5a76000640 1 -- 172.21.15.131:0/3168960777 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v29) ==== 53+0+73 (secure 0 0 0) 0x7f5a7805c750 con 0x7f5a8015e960 2024-05-26T22:50:39.554 INFO:teuthology.orchestra.run.smithi131.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-26T22:50:39.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.557+0000 7f5a85000640 1 -- 172.21.15.131:0/3168960777 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f5a5c06be30 msgr2=0x7f5a5c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:39.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.557+0000 7f5a85000640 1 --2- 172.21.15.131:0/3168960777 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f5a5c06be30 0x7f5a5c06e2f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f5a801f7ae0 tx=0x7f5a70008040 comp rx=0 tx=0).stop 2024-05-26T22:50:39.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.557+0000 7f5a85000640 1 -- 172.21.15.131:0/3168960777 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a8015e960 msgr2=0x7f5a800b2500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:39.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.557+0000 7f5a85000640 1 --2- 172.21.15.131:0/3168960777 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a8015e960 0x7f5a800b2500 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f5a7800c390 tx=0x7f5a7800c850 comp rx=0 tx=0).stop 2024-05-26T22:50:39.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.557+0000 7f5a85000640 1 -- 172.21.15.131:0/3168960777 shutdown_connections 2024-05-26T22:50:39.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.557+0000 7f5a85000640 1 --2- 172.21.15.131:0/3168960777 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5a8015e580 0x7f5a800b1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:39.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.557+0000 7f5a85000640 1 --2- 172.21.15.131:0/3168960777 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f5a5c06be30 0x7f5a5c06e2f0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:39.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.557+0000 7f5a85000640 1 --2- 172.21.15.131:0/3168960777 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a8015e960 0x7f5a800b2500 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:39.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.557+0000 7f5a85000640 1 --2- 172.21.15.131:0/3168960777 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5a800b3560 0x7f5a80152d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:39.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.557+0000 7f5a85000640 1 -- 172.21.15.131:0/3168960777 >> 172.21.15.131:0/3168960777 conn(0x7f5a8009d680 msgr2=0x7f5a800cef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:39.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.557+0000 7f5a85000640 1 -- 172.21.15.131:0/3168960777 shutdown_connections 2024-05-26T22:50:39.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39.557+0000 7f5a85000640 1 -- 172.21.15.131:0/3168960777 wait complete. 2024-05-26T22:50:39.575 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-26T22:50:39.575 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-26T22:50:39.575 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi131.front.sepia.ceph.com 2024-05-26T22:50:39.575 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-26T22:50:39.575 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-26T22:50:39.575 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-26T22:50:39.575 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-26T22:50:39.575 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-26T22:50:39.575 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-26T22:50:39.576 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:50:39.576 DEBUG:teuthology.orchestra.run.smithi131:> ip addr 2024-05-26T22:50:39.627 INFO:teuthology.orchestra.run.smithi131.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-26T22:50:39.627 INFO:teuthology.orchestra.run.smithi131.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-26T22:50:39.627 INFO:teuthology.orchestra.run.smithi131.stdout: inet 127.0.0.1/8 scope host lo 2024-05-26T22:50:39.627 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-05-26T22:50:39.627 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 ::1/128 scope host 2024-05-26T22:50:39.628 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-05-26T22:50:39.628 INFO:teuthology.orchestra.run.smithi131.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-26T22:50:39.628 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:d8:fb:18 brd ff:ff:ff:ff:ff:ff 2024-05-26T22:50:39.628 INFO:teuthology.orchestra.run.smithi131.stdout: altname enp7s0f0 2024-05-26T22:50:39.628 INFO:teuthology.orchestra.run.smithi131.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-26T22:50:39.628 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:d8:fb:19 brd ff:ff:ff:ff:ff:ff 2024-05-26T22:50:39.628 INFO:teuthology.orchestra.run.smithi131.stdout: altname enp7s0f1 2024-05-26T22:50:39.628 INFO:teuthology.orchestra.run.smithi131.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-26T22:50:39.628 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:88:72:f8 brd ff:ff:ff:ff:ff:ff 2024-05-26T22:50:39.628 INFO:teuthology.orchestra.run.smithi131.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-26T22:50:39.628 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:88:72:f9 brd ff:ff:ff:ff:ff:ff 2024-05-26T22:50:39.629 INFO:teuthology.orchestra.run.smithi131.stdout: inet 172.21.15.131/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-26T22:50:39.629 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft 42458sec preferred_lft 42458sec 2024-05-26T22:50:39.629 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::ec4:7aff:fe88:72f9/64 scope link 2024-05-26T22:50:39.629 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-05-26T22:50:39.629 INFO:teuthology.orchestra.run.smithi131.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-26T22:50:39.629 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 06:f4:05:86:a4:3e brd ff:ff:ff:ff:ff:ff 2024-05-26T22:50:39.629 INFO:teuthology.orchestra.run.smithi131.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-26T22:50:39.629 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-05-26T22:50:39.629 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::8802:8fff:fe41:fc6d/64 scope link 2024-05-26T22:50:39.629 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-05-26T22:50:39.629 INFO:teuthology.orchestra.run.smithi131.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-26T22:50:39.629 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 06:f4:05:86:a4:3e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-26T22:50:39.629 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::4f4:5ff:fe86:a43e/64 scope link 2024-05-26T22:50:39.630 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-05-26T22:50:39.630 INFO:teuthology.orchestra.run.smithi131.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-26T22:50:39.630 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether ee:6f:35:b2:c0:5b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-26T22:50:39.630 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::ec6f:35ff:feb2:c05b/64 scope link 2024-05-26T22:50:39.630 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-05-26T22:50:39.630 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:50:39.630 DEBUG:teuthology.orchestra.run.smithi131:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-26T22:50:39.703 INFO:teuthology.orchestra.run.smithi131.stdout:1 2024-05-26T22:50:39.707 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:50:39.707 DEBUG:teuthology.orchestra.run.smithi131:> route 2024-05-26T22:50:39.756 INFO:teuthology.orchestra.run.smithi131.stdout:Kernel IP routing table 2024-05-26T22:50:39.757 INFO:teuthology.orchestra.run.smithi131.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-26T22:50:39.757 INFO:teuthology.orchestra.run.smithi131.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-26T22:50:39.757 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-26T22:50:39.757 INFO:teuthology.orchestra.run.smithi131.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-26T22:50:39.757 DEBUG:teuthology.orchestra.run.smithi131:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-26T22:50:39.757 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2024-05-26T22:50:39.757 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-26T22:50:39.757 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-26T22:50:39.757 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-26T22:50:39.757 DEBUG:teuthology.orchestra.run.smithi131:> ') 2024-05-26T22:50:39.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-26T22:50:39.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-26T22:50:39.968 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:50:39.969 DEBUG:teuthology.orchestra.run.smithi131:> ip netns list 2024-05-26T22:50:40.016 INFO:teuthology.orchestra.run.smithi131.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-26T22:50:40.016 INFO:teuthology.orchestra.run.smithi131.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-26T22:50:40.017 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-26T22:50:40.017 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:50:40.018 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link set brx.0 up 2024-05-26T22:50:40.100 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-26T22:50:40.100 DEBUG:teuthology.orchestra.run.smithi131:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-26T22:50:40.153 INFO:teuthology.orchestra.run.smithi131.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-26T22:50:40.154 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-26T22:50:40.154 DEBUG:teuthology.orchestra.run.smithi131:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-26T22:50:40.204 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-05-26T22:50:40.204 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:50:40.205 DEBUG:teuthology.orchestra.run.smithi131:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-26T22:50:40.329 INFO:teuthology.orchestra.run.smithi131.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-26T22:50:40.329 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-26T22:50:40.362 INFO:teuthology.orchestra.run.smithi131.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-26T22:50:40.363 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-26T22:50:40.363 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: resolved to: "172.21.15.131:3300,172.21.15.187:3300,172.21.15.187:3301" 2024-05-26T22:50:40.363 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: trying mount with new device syntax: 0@46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f.cephfs=/ 2024-05-26T22:50:40.363 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.131:3300/172.21.15.187:3300/172.21.15.187:3301" will pass to kernel 2024-05-26T22:50:40.365 INFO:teuthology.orchestra.run.smithi131.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-26T22:50:40.365 INFO:teuthology.orchestra.run.smithi131.stdout: You just mounted a file system that supports labels which does not 2024-05-26T22:50:40.365 INFO:teuthology.orchestra.run.smithi131.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-26T22:50:40.365 INFO:teuthology.orchestra.run.smithi131.stdout: applications will generate AVC messages and not be allowed access to 2024-05-26T22:50:40.365 INFO:teuthology.orchestra.run.smithi131.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-26T22:50:40.369 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-26T22:50:40.369 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:50:40.370 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-26T22:50:40.445 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-26T22:50:40.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:40 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-26T22:50:40.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-26T22:50:40.578 INFO:teuthology.orchestra.run.smithi131.stdout:client5841 2024-05-26T22:50:40.578 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-26T22:50:40.579 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-26T22:50:40.632 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_export_ephemeral_random True 2024-05-26T22:50:40.840 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:40.840+0000 7fa56b000640 1 Processor -- start 2024-05-26T22:50:40.840 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:40.840+0000 7fa56b000640 1 -- start start 2024-05-26T22:50:40.841 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:40.842+0000 7fa56b000640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fa56406dd30 0x7fa56406e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:40.841 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:40.842+0000 7fa56b000640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa56415c1c0 con 0x7fa564155770 2024-05-26T22:50:40.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:40.842+0000 7fa56b000640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa56415c4a0 con 0x7fa5640725a0 2024-05-26T22:50:40.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:40.842+0000 7fa56b000640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fa564071da0 con 0x7fa56406dd30 2024-05-26T22:50:40.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:40.842+0000 7fa563e00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fa564155770 0x7fa56415b700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:58728/0 (socket says 172.21.15.131:58728) 2024-05-26T22:50:40.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:40.842+0000 7fa563e00640 1 -- 172.21.15.131:0/3039611902 learned_addr learned my addr 172.21.15.131:0/3039611902 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:50:40.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:40.842+0000 7fa55b400640 1 --2- 172.21.15.131:0/3039611902 >> v2:172.21.15.187:3300/0 conn(0x7fa56406dd30 0x7fa56406e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:40.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:40.842+0000 7fa563400640 1 -- 172.21.15.131:0/3039611902 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 519934032 0 0) 0x7fa56415c1c0 con 0x7fa564155770 2024-05-26T22:50:40.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:40.843+0000 7fa563400640 1 -- 172.21.15.131:0/3039611902 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa548003520 con 0x7fa564155770 2024-05-26T22:50:40.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:40.843+0000 7fa563400640 1 -- 172.21.15.131:0/3039611902 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2810338934 0 0) 0x7fa548003520 con 0x7fa564155770 2024-05-26T22:50:40.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:40.843+0000 7fa563400640 1 -- 172.21.15.131:0/3039611902 >> v2:172.21.15.187:3300/0 conn(0x7fa56406dd30 msgr2=0x7fa56406e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:40.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:40.843+0000 7fa563400640 1 --2- 172.21.15.131:0/3039611902 >> v2:172.21.15.187:3300/0 conn(0x7fa56406dd30 0x7fa56406e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:40.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:40.843+0000 7fa563400640 1 -- 172.21.15.131:0/3039611902 >> v1:172.21.15.187:6789/0 conn(0x7fa5640725a0 legacy=0x7fa5640729a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:40.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:40.843+0000 7fa563400640 1 -- 172.21.15.131:0/3039611902 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa564072080 con 0x7fa564155770 2024-05-26T22:50:40.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:40.843+0000 7fa563400640 1 -- 172.21.15.131:0/3039611902 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7fa554001670 con 0x7fa564155770 2024-05-26T22:50:40.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:40.843+0000 7fa563400640 1 -- 172.21.15.131:0/3039611902 >> v1:172.21.15.131:6789/0 conn(0x7fa564155770 legacy=0x7fa56415b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:40.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:40.843+0000 7fa563400640 1 --2- 172.21.15.131:0/3039611902 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa5480039b0 0x7fa548003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:40.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:40.843+0000 7fa563400640 1 --2- 172.21.15.131:0/3039611902 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa548004800 0x7fa548004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:40.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:40.843+0000 7fa563400640 1 --2- 172.21.15.131:0/3039611902 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa5480051c0 0x7fa54800dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:40.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:40.843+0000 7fa563400640 1 -- 172.21.15.131:0/3039611902 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa564071da0 con 0x7fa5480039b0 2024-05-26T22:50:40.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:40.843+0000 7fa563400640 1 -- 172.21.15.131:0/3039611902 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa54800e280 con 0x7fa548004800 2024-05-26T22:50:40.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:40.843+0000 7fa563400640 1 -- 172.21.15.131:0/3039611902 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa54800e480 con 0x7fa5480051c0 2024-05-26T22:50:40.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:40.843+0000 7fa563e00640 1 --2- 172.21.15.131:0/3039611902 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa5480039b0 0x7fa548003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:40.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:40.844+0000 7fa563e00640 1 -- 172.21.15.131:0/3039611902 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa5480051c0 msgr2=0x7fa54800dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:40.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:40.844+0000 7fa563e00640 1 --2- 172.21.15.131:0/3039611902 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa5480051c0 0x7fa54800dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:40.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:40.844+0000 7fa563e00640 1 -- 172.21.15.131:0/3039611902 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa548004800 msgr2=0x7fa548004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:40.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:40.844+0000 7fa563e00640 1 --2- 172.21.15.131:0/3039611902 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa548004800 0x7fa548004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:40.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:40.844+0000 7fa563e00640 1 -- 172.21.15.131:0/3039611902 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa548003520 con 0x7fa5480039b0 2024-05-26T22:50:40.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:40.844+0000 7fa569000640 1 -- 172.21.15.131:0/3039611902 reap_dead start 2024-05-26T22:50:40.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:40.845+0000 7fa563e00640 1 --2- 172.21.15.131:0/3039611902 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa5480039b0 0x7fa548003dd0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fa55400c240 tx=0x7fa548005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23b46478a18f5c41 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:40.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:40.845+0000 7fa563400640 1 -- 172.21.15.131:0/3039611902 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa554007cf0 con 0x7fa5480039b0 2024-05-26T22:50:40.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:40.845+0000 7fa563400640 1 -- 172.21.15.131:0/3039611902 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa554005ad0 con 0x7fa5480039b0 2024-05-26T22:50:40.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:40.845+0000 7fa563400640 1 -- 172.21.15.131:0/3039611902 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa554007070 con 0x7fa5480039b0 2024-05-26T22:50:40.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:40.846+0000 7fa56b000640 1 -- 172.21.15.131:0/3039611902 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa5480039b0 msgr2=0x7fa548003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:40.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:40.846+0000 7fa56b000640 1 --2- 172.21.15.131:0/3039611902 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa5480039b0 0x7fa548003dd0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fa55400c240 tx=0x7fa548005830 comp rx=0 tx=0).stop 2024-05-26T22:50:40.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:40.846+0000 7fa56b000640 1 -- 172.21.15.131:0/3039611902 shutdown_connections 2024-05-26T22:50:40.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:40.846+0000 7fa56b000640 1 --2- 172.21.15.131:0/3039611902 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa5480039b0 0x7fa548003dd0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:40.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:40.846+0000 7fa56b000640 1 -- 172.21.15.131:0/3039611902 >> 172.21.15.131:0/3039611902 conn(0x7fa56406bc90 msgr2=0x7fa5640b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:40.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:40.846+0000 7fa56b000640 1 -- 172.21.15.131:0/3039611902 shutdown_connections 2024-05-26T22:50:40.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:40.846+0000 7fa56b000640 1 -- 172.21.15.131:0/3039611902 wait complete. 2024-05-26T22:50:40.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:40.848+0000 7fa56b000640 1 Processor -- start 2024-05-26T22:50:40.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:40.848+0000 7fa56b000640 1 -- start start 2024-05-26T22:50:40.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:40.849+0000 7fa56b000640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa5480039b0 0x7fa5641508d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:40.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:40.849+0000 7fa56b000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa5641480c0 0x7fa564145040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:40.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:40.849+0000 7fa56b000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa564148540 0x7fa564145580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:40.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:40.849+0000 7fa56b000640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa56409de00 con 0x7fa564148540 2024-05-26T22:50:40.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:40.849+0000 7fa56b000640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa56415c8c0 con 0x7fa5480039b0 2024-05-26T22:50:40.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:40.849+0000 7fa56b000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa56409df80 con 0x7fa5641480c0 2024-05-26T22:50:40.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:40.849+0000 7fa569000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa564148540 0x7fa564145580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:40.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:40.849+0000 7fa569000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa564148540 0x7fa564145580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41820/0 (socket says 172.21.15.131:41820) 2024-05-26T22:50:40.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:40.849+0000 7fa569000640 1 -- 172.21.15.131:0/578382875 learned_addr learned my addr 172.21.15.131:0/578382875 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:40.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:40.849+0000 7fa563e00640 1 --2- 172.21.15.131:0/578382875 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa5480039b0 0x7fa5641508d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:40.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:40.849+0000 7fa55b400640 1 --2- 172.21.15.131:0/578382875 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa5641480c0 0x7fa564145040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:40.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:40.849+0000 7fa569000640 1 -- 172.21.15.131:0/578382875 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa5641480c0 msgr2=0x7fa564145040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:40.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:40.850+0000 7fa569000640 1 --2- 172.21.15.131:0/578382875 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa5641480c0 0x7fa564145040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:40.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:40.850+0000 7fa569000640 1 -- 172.21.15.131:0/578382875 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa5480039b0 msgr2=0x7fa5641508d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:40.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:40.850+0000 7fa569000640 1 --2- 172.21.15.131:0/578382875 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa5480039b0 0x7fa5641508d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:40.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:40.850+0000 7fa569000640 1 -- 172.21.15.131:0/578382875 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa564145ac0 con 0x7fa564148540 2024-05-26T22:50:40.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:40.850+0000 7fa55b400640 1 --2- 172.21.15.131:0/578382875 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa5641480c0 0x7fa564145040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:50:40.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:40.850+0000 7fa569000640 1 --2- 172.21.15.131:0/578382875 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa564148540 0x7fa564145580 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fa56406f430 tx=0x7fa55c00cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:40.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:40.850+0000 7fa560c00640 1 -- 172.21.15.131:0/578382875 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa55c017070 con 0x7fa564148540 2024-05-26T22:50:40.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:40.850+0000 7fa560c00640 1 -- 172.21.15.131:0/578382875 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa55c005d60 con 0x7fa564148540 2024-05-26T22:50:40.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:40.850+0000 7fa560c00640 1 -- 172.21.15.131:0/578382875 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa55c00a850 con 0x7fa564148540 2024-05-26T22:50:40.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:40.850+0000 7fa56b000640 1 -- 172.21.15.131:0/578382875 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa564152ed0 con 0x7fa564148540 2024-05-26T22:50:40.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:40.850+0000 7fa56b000640 1 -- 172.21.15.131:0/578382875 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa564153490 con 0x7fa564148540 2024-05-26T22:50:40.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:40.851+0000 7fa56b000640 1 -- 172.21.15.131:0/578382875 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5640695c0 con 0x7fa564148540 2024-05-26T22:50:40.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:40.853+0000 7fa560c00640 1 -- 172.21.15.131:0/578382875 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa55c00a9f0 con 0x7fa564148540 2024-05-26T22:50:40.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:40.853+0000 7fa560c00640 1 --2- 172.21.15.131:0/578382875 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa54006be30 0x7fa54006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:40.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:40.854+0000 7fa560c00640 1 -- 172.21.15.131:0/578382875 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7fa55c089eb0 con 0x7fa564148540 2024-05-26T22:50:40.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:40.854+0000 7fa563e00640 1 --2- 172.21.15.131:0/578382875 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa54006be30 0x7fa54006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:40.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:40.854+0000 7fa563e00640 1 --2- 172.21.15.131:0/578382875 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa54006be30 0x7fa54006e2f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fa554033000 tx=0x7fa554009030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:40.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:40.858+0000 7fa560c00640 1 -- 172.21.15.131:0/578382875 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fa55c0587c0 con 0x7fa564148540 2024-05-26T22:50:41.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.061+0000 7fa56b000640 1 -- 172.21.15.131:0/578382875 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command([{prefix=config set, name=mds_export_ephemeral_random}] v 0) -- 0x7fa5640b57f0 con 0x7fa564148540 2024-05-26T22:50:41.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.066+0000 7fa560c00640 1 -- 172.21.15.131:0/578382875 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_export_ephemeral_random}]=0 v20)=0 v20) ==== 128+0+0 (secure 0 0 0) 0x7fa55c0580c0 con 0x7fa564148540 2024-05-26T22:50:41.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.069+0000 7fa56b000640 1 -- 172.21.15.131:0/578382875 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa54006be30 msgr2=0x7fa54006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:41.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.069+0000 7fa56b000640 1 --2- 172.21.15.131:0/578382875 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa54006be30 0x7fa54006e2f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fa554033000 tx=0x7fa554009030 comp rx=0 tx=0).stop 2024-05-26T22:50:41.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.069+0000 7fa56b000640 1 -- 172.21.15.131:0/578382875 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa564148540 msgr2=0x7fa564145580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:41.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.069+0000 7fa56b000640 1 --2- 172.21.15.131:0/578382875 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa564148540 0x7fa564145580 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fa56406f430 tx=0x7fa55c00cb90 comp rx=0 tx=0).stop 2024-05-26T22:50:41.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.069+0000 7fa56b000640 1 -- 172.21.15.131:0/578382875 shutdown_connections 2024-05-26T22:50:41.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.069+0000 7fa56b000640 1 --2- 172.21.15.131:0/578382875 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa5641480c0 0x7fa564145040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:41.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.069+0000 7fa56b000640 1 --2- 172.21.15.131:0/578382875 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa54006be30 0x7fa54006e2f0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:41.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.069+0000 7fa56b000640 1 --2- 172.21.15.131:0/578382875 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa564148540 0x7fa564145580 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:41.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.069+0000 7fa56b000640 1 --2- 172.21.15.131:0/578382875 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa5480039b0 0x7fa5641508d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:41.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.069+0000 7fa56b000640 1 -- 172.21.15.131:0/578382875 >> 172.21.15.131:0/578382875 conn(0x7fa56406bc90 msgr2=0x7fa5640aef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:41.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.070+0000 7fa56b000640 1 -- 172.21.15.131:0/578382875 shutdown_connections 2024-05-26T22:50:41.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.070+0000 7fa56b000640 1 -- 172.21.15.131:0/578382875 wait complete. 2024-05-26T22:50:41.089 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_export_ephemeral_distributed True 2024-05-26T22:50:41.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.293+0000 7f9708a00640 1 Processor -- start 2024-05-26T22:50:41.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.293+0000 7f9708a00640 1 -- start start 2024-05-26T22:50:41.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.294+0000 7f9708a00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f9704155770 0x7f970415b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:41.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.294+0000 7f9708a00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f970406e440 0x7f97040725a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:41.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.294+0000 7f9708a00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f970415c1c0 con 0x7f970415bc40 2024-05-26T22:50:41.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.294+0000 7f9708a00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f970415c4a0 con 0x7f970406e440 2024-05-26T22:50:41.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.294+0000 7f9708a00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f9704071da0 con 0x7f9704155770 2024-05-26T22:50:41.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.294+0000 7f96f9400640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f970415bc40 0x7f970406dd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:58734/0 (socket says 172.21.15.131:58734) 2024-05-26T22:50:41.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.294+0000 7f9702800640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f970406e440 0x7f97040725a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:41.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.294+0000 7f96f9400640 1 -- 172.21.15.131:0/2630966324 learned_addr learned my addr 172.21.15.131:0/2630966324 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:50:41.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.295+0000 7f9701e00640 1 --2- 172.21.15.131:0/2630966324 >> v2:172.21.15.187:3300/0 conn(0x7f9704155770 0x7f970415b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:41.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.295+0000 7f9702800640 1 -- 172.21.15.131:0/2630966324 >> v2:172.21.15.187:3300/0 conn(0x7f9704155770 msgr2=0x7f970415b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:41.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.295+0000 7f9702800640 1 --2- 172.21.15.131:0/2630966324 >> v2:172.21.15.187:3300/0 conn(0x7f9704155770 0x7f970415b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:41.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.295+0000 7f9702800640 1 -- 172.21.15.131:0/2630966324 >> v1:172.21.15.131:6789/0 conn(0x7f970415bc40 legacy=0x7f970406dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:41.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.295+0000 7f9702800640 1 -- 172.21.15.131:0/2630966324 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9704072080 con 0x7f970406e440 2024-05-26T22:50:41.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.295+0000 7f9701400640 1 -- 172.21.15.131:0/2630966324 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 151919600 0 0) 0x7f970415c1c0 con 0x7f970415bc40 2024-05-26T22:50:41.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.296+0000 7f9702800640 1 --2- 172.21.15.131:0/2630966324 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f970406e440 0x7f97040725a0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f9704071790 tx=0x7f96fc02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31a3af359bf9bc2e server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:41.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.296+0000 7f9701400640 1 -- 172.21.15.131:0/2630966324 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96fc007740 con 0x7f970406e440 2024-05-26T22:50:41.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.296+0000 7f9701400640 1 -- 172.21.15.131:0/2630966324 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96fc0078e0 con 0x7f970406e440 2024-05-26T22:50:41.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.296+0000 7f9701400640 1 -- 172.21.15.131:0/2630966324 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96fc007cb0 con 0x7f970406e440 2024-05-26T22:50:41.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.297+0000 7f9708a00640 1 -- 172.21.15.131:0/2630966324 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f970406e440 msgr2=0x7f97040725a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:41.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.297+0000 7f9708a00640 1 --2- 172.21.15.131:0/2630966324 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f970406e440 0x7f97040725a0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f9704071790 tx=0x7f96fc02a710 comp rx=0 tx=0).stop 2024-05-26T22:50:41.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.297+0000 7f9708a00640 1 -- 172.21.15.131:0/2630966324 shutdown_connections 2024-05-26T22:50:41.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.297+0000 7f9708a00640 1 --2- 172.21.15.131:0/2630966324 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f970406e440 0x7f97040725a0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:41.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.297+0000 7f9708a00640 1 --2- 172.21.15.131:0/2630966324 >> v2:172.21.15.187:3300/0 conn(0x7f9704155770 0x7f970415b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:41.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.297+0000 7f9708a00640 1 -- 172.21.15.131:0/2630966324 >> 172.21.15.131:0/2630966324 conn(0x7f970406bc90 msgr2=0x7f97040b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:41.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.297+0000 7f9708a00640 1 -- 172.21.15.131:0/2630966324 shutdown_connections 2024-05-26T22:50:41.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.297+0000 7f9708a00640 1 -- 172.21.15.131:0/2630966324 wait complete. 2024-05-26T22:50:41.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.299+0000 7f9708a00640 1 Processor -- start 2024-05-26T22:50:41.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.299+0000 7f9708a00640 1 -- start start 2024-05-26T22:50:41.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.300+0000 7f9708a00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f970406e440 0x7f9704148c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:41.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.300+0000 7f9708a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9704155770 0x7f970415a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:41.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.300+0000 7f9708a00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f970415bc40 0x7f970415a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:41.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.300+0000 7f9708a00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f970409de00 con 0x7f9704155770 2024-05-26T22:50:41.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.300+0000 7f9708a00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f970415c8c0 con 0x7f970415bc40 2024-05-26T22:50:41.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.300+0000 7f9708a00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f970409df80 con 0x7f970406e440 2024-05-26T22:50:41.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.300+0000 7f96f9400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9704155770 0x7f970415a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:41.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.300+0000 7f96f9400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9704155770 0x7f970415a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41836/0 (socket says 172.21.15.131:41836) 2024-05-26T22:50:41.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.300+0000 7f96f9400640 1 -- 172.21.15.131:0/2951177172 learned_addr learned my addr 172.21.15.131:0/2951177172 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:41.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.301+0000 7f9702800640 1 --2- 172.21.15.131:0/2951177172 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f970415bc40 0x7f970415a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:41.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.301+0000 7f9701e00640 1 --2- 172.21.15.131:0/2951177172 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f970406e440 0x7f9704148c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:41.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.301+0000 7f96f9400640 1 -- 172.21.15.131:0/2951177172 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f970406e440 msgr2=0x7f9704148c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:41.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.301+0000 7f96f9400640 1 --2- 172.21.15.131:0/2951177172 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f970406e440 0x7f9704148c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:41.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.301+0000 7f96f9400640 1 -- 172.21.15.131:0/2951177172 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f970415bc40 msgr2=0x7f970415a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:41.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.301+0000 7f96f9400640 1 --2- 172.21.15.131:0/2951177172 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f970415bc40 0x7f970415a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:41.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.301+0000 7f96f9400640 1 -- 172.21.15.131:0/2951177172 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97041491a0 con 0x7f9704155770 2024-05-26T22:50:41.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.301+0000 7f9702800640 1 --2- 172.21.15.131:0/2951177172 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f970415bc40 0x7f970415a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:50:41.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.301+0000 7f9701e00640 1 --2- 172.21.15.131:0/2951177172 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f970406e440 0x7f9704148c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:50:41.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.301+0000 7f96f9400640 1 --2- 172.21.15.131:0/2951177172 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9704155770 0x7f970415a1a0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f96ec00c430 tx=0x7f96ec00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:41.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.301+0000 7f96faa00640 1 -- 172.21.15.131:0/2951177172 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96ec0074e0 con 0x7f9704155770 2024-05-26T22:50:41.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.301+0000 7f96faa00640 1 -- 172.21.15.131:0/2951177172 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96ec0047e0 con 0x7f9704155770 2024-05-26T22:50:41.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.301+0000 7f96faa00640 1 -- 172.21.15.131:0/2951177172 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96ec004bd0 con 0x7f9704155770 2024-05-26T22:50:41.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.301+0000 7f9708a00640 1 -- 172.21.15.131:0/2951177172 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9704150150 con 0x7f9704155770 2024-05-26T22:50:41.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.302+0000 7f9708a00640 1 -- 172.21.15.131:0/2951177172 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f970409bb80 con 0x7f9704155770 2024-05-26T22:50:41.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.303+0000 7f9708a00640 1 -- 172.21.15.131:0/2951177172 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9704074e90 con 0x7f9704155770 2024-05-26T22:50:41.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.304+0000 7f96faa00640 1 -- 172.21.15.131:0/2951177172 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f96ec011070 con 0x7f9704155770 2024-05-26T22:50:41.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.304+0000 7f96faa00640 1 --2- 172.21.15.131:0/2951177172 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f96e006be30 0x7f96e006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:41.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.305+0000 7f96faa00640 1 -- 172.21.15.131:0/2951177172 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f96ec08aae0 con 0x7f9704155770 2024-05-26T22:50:41.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.305+0000 7f9701e00640 1 --2- 172.21.15.131:0/2951177172 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f96e006be30 0x7f96e006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:41.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.305+0000 7f9701e00640 1 --2- 172.21.15.131:0/2951177172 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f96e006be30 0x7f96e006e2f0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f96f4006fd0 tx=0x7f96f4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:41.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.309+0000 7f96faa00640 1 -- 172.21.15.131:0/2951177172 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f96ec059480 con 0x7f9704155770 2024-05-26T22:50:41.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.508+0000 7f9708a00640 1 -- 172.21.15.131:0/2951177172 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command([{prefix=config set, name=mds_export_ephemeral_distributed}] v 0) -- 0x7f97040695c0 con 0x7f9704155770 2024-05-26T22:50:41.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.513+0000 7f96faa00640 1 -- 172.21.15.131:0/2951177172 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_export_ephemeral_distributed}]=0 v21)=0 v21) ==== 133+0+0 (secure 0 0 0) 0x7f96ec058d80 con 0x7f9704155770 2024-05-26T22:50:41.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.517+0000 7f9708a00640 1 -- 172.21.15.131:0/2951177172 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f96e006be30 msgr2=0x7f96e006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:41.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.517+0000 7f9708a00640 1 --2- 172.21.15.131:0/2951177172 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f96e006be30 0x7f96e006e2f0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f96f4006fd0 tx=0x7f96f4008040 comp rx=0 tx=0).stop 2024-05-26T22:50:41.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.517+0000 7f9708a00640 1 -- 172.21.15.131:0/2951177172 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9704155770 msgr2=0x7f970415a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:41.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.517+0000 7f9708a00640 1 --2- 172.21.15.131:0/2951177172 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9704155770 0x7f970415a1a0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f96ec00c430 tx=0x7f96ec00c8f0 comp rx=0 tx=0).stop 2024-05-26T22:50:41.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.517+0000 7f9708a00640 1 -- 172.21.15.131:0/2951177172 shutdown_connections 2024-05-26T22:50:41.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.517+0000 7f9708a00640 1 --2- 172.21.15.131:0/2951177172 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f96e006be30 0x7f96e006e2f0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:41.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.517+0000 7f9708a00640 1 --2- 172.21.15.131:0/2951177172 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f970415bc40 0x7f970415a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:41.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.517+0000 7f9708a00640 1 --2- 172.21.15.131:0/2951177172 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9704155770 0x7f970415a1a0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:41.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.517+0000 7f9708a00640 1 --2- 172.21.15.131:0/2951177172 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f970406e440 0x7f9704148c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:41.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.517+0000 7f9708a00640 1 -- 172.21.15.131:0/2951177172 >> 172.21.15.131:0/2951177172 conn(0x7f970406bc90 msgr2=0x7f97040ae440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:41.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.517+0000 7f9708a00640 1 -- 172.21.15.131:0/2951177172 shutdown_connections 2024-05-26T22:50:41.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.517+0000 7f9708a00640 1 -- 172.21.15.131:0/2951177172 wait complete. 2024-05-26T22:50:41.535 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_export_ephemeral_random_max 1.0 2024-05-26T22:50:41.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.740+0000 7faeb6800640 1 Processor -- start 2024-05-26T22:50:42.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.740+0000 7faeb6800640 1 -- start start 2024-05-26T22:50:42.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.742+0000 7faeb6800640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7faeb006d250 0x7faeb006d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:42.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.742+0000 7faeb6800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faeb01400c0 0x7faeb01404f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:42.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.742+0000 7faeb6800640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faeb006db70 con 0x7faeb015b730 2024-05-26T22:50:42.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.742+0000 7faeb6800640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7faeb0071260 con 0x7faeb006d250 2024-05-26T22:50:42.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.742+0000 7faeb6800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7faeb013e820 con 0x7faeb01400c0 2024-05-26T22:50:42.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.742+0000 7faeb5e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faeb01400c0 0x7faeb01404f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:42.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.743+0000 7faeb5e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faeb01400c0 0x7faeb01404f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.131:54914/0 (socket says 172.21.15.131:54914) 2024-05-26T22:50:42.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.743+0000 7faeb5e00640 1 -- 172.21.15.131:0/242297557 learned_addr learned my addr 172.21.15.131:0/242297557 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:42.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.743+0000 7faeb5400640 1 --2- 172.21.15.131:0/242297557 >> v2:172.21.15.187:3300/0 conn(0x7faeb006d250 0x7faeb006d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:42.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.743+0000 7faeb5e00640 1 -- 172.21.15.131:0/242297557 >> v2:172.21.15.187:3300/0 conn(0x7faeb006d250 msgr2=0x7faeb006d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:42.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.743+0000 7faeb5e00640 1 --2- 172.21.15.131:0/242297557 >> v2:172.21.15.187:3300/0 conn(0x7faeb006d250 0x7faeb006d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:42.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.743+0000 7faeb5e00640 1 -- 172.21.15.131:0/242297557 >> v1:172.21.15.187:6789/0 conn(0x7faeb015b730 legacy=0x7faeb015bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:42.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.743+0000 7faeb5e00640 1 -- 172.21.15.131:0/242297557 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faeb015c4a0 con 0x7faeb01400c0 2024-05-26T22:50:42.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.745+0000 7faeb5e00640 1 --2- 172.21.15.131:0/242297557 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faeb01400c0 0x7faeb01404f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7faea8009870 tx=0x7faea802a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=897ef61db6a6959b server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:42.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.745+0000 7faeafe00640 1 -- 172.21.15.131:0/242297557 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faea8037070 con 0x7faeb01400c0 2024-05-26T22:50:42.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.745+0000 7faeafe00640 1 -- 172.21.15.131:0/242297557 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faea8032440 con 0x7faeb01400c0 2024-05-26T22:50:42.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.745+0000 7faeafe00640 1 -- 172.21.15.131:0/242297557 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faea8032810 con 0x7faeb01400c0 2024-05-26T22:50:42.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.746+0000 7faeb6800640 1 -- 172.21.15.131:0/242297557 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faeb01400c0 msgr2=0x7faeb01404f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:42.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.746+0000 7faeb6800640 1 --2- 172.21.15.131:0/242297557 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faeb01400c0 0x7faeb01404f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7faea8009870 tx=0x7faea802a210 comp rx=0 tx=0).stop 2024-05-26T22:50:42.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.746+0000 7faeb6800640 1 -- 172.21.15.131:0/242297557 shutdown_connections 2024-05-26T22:50:42.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.746+0000 7faeb6800640 1 --2- 172.21.15.131:0/242297557 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faeb01400c0 0x7faeb01404f0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:42.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.746+0000 7faeb6800640 1 --2- 172.21.15.131:0/242297557 >> v2:172.21.15.187:3300/0 conn(0x7faeb006d250 0x7faeb006d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:42.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.746+0000 7faeb6800640 1 -- 172.21.15.131:0/242297557 >> 172.21.15.131:0/242297557 conn(0x7faeb00b5660 msgr2=0x7faeb00b5a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:42.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.746+0000 7faeb6800640 1 -- 172.21.15.131:0/242297557 shutdown_connections 2024-05-26T22:50:42.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.746+0000 7faeb6800640 1 -- 172.21.15.131:0/242297557 wait complete. 2024-05-26T22:50:42.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.748+0000 7faeb6800640 1 Processor -- start 2024-05-26T22:50:42.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.748+0000 7faeb6800640 1 -- start start 2024-05-26T22:50:42.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.749+0000 7faeb6800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faeb006d250 0x7faeb01e7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:42.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.749+0000 7faeb6800640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faeb01400c0 0x7faeb0157150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:42.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.749+0000 7faeb6800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faeb015b730 0x7faeb0152aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:42.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.749+0000 7faeb6800640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7faeb0071a70 con 0x7faeb006d250 2024-05-26T22:50:42.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.749+0000 7faeb6800640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7faeb0071790 con 0x7faeb01400c0 2024-05-26T22:50:42.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.749+0000 7faeb6800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7faeb0072fa0 con 0x7faeb015b730 2024-05-26T22:50:42.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.749+0000 7faeb5400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faeb006d250 0x7faeb01e7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:42.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.749+0000 7faeb5400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faeb006d250 0x7faeb01e7ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41850/0 (socket says 172.21.15.131:41850) 2024-05-26T22:50:42.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.749+0000 7faeb5400640 1 -- 172.21.15.131:0/2293515128 learned_addr learned my addr 172.21.15.131:0/2293515128 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:42.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.749+0000 7faeb5e00640 1 --2- 172.21.15.131:0/2293515128 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faeb015b730 0x7faeb0152aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:42.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.749+0000 7faeb4a00640 1 --2- 172.21.15.131:0/2293515128 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faeb01400c0 0x7faeb0157150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:42.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.750+0000 7faeb5400640 1 -- 172.21.15.131:0/2293515128 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faeb015b730 msgr2=0x7faeb0152aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:42.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.750+0000 7faeb5400640 1 --2- 172.21.15.131:0/2293515128 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faeb015b730 0x7faeb0152aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:42.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.750+0000 7faeb5400640 1 -- 172.21.15.131:0/2293515128 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faeb01400c0 msgr2=0x7faeb0157150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:42.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.750+0000 7faeb5400640 1 --2- 172.21.15.131:0/2293515128 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faeb01400c0 0x7faeb0157150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:42.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.750+0000 7faeb5400640 1 -- 172.21.15.131:0/2293515128 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faeb0157c10 con 0x7faeb006d250 2024-05-26T22:50:42.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.750+0000 7faeb5400640 1 --2- 172.21.15.131:0/2293515128 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faeb006d250 0x7faeb01e7ef0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7faea400efc0 tx=0x7faea400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:42.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.750+0000 7faead600640 1 -- 172.21.15.131:0/2293515128 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faea4018020 con 0x7faeb006d250 2024-05-26T22:50:42.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.750+0000 7faead600640 1 -- 172.21.15.131:0/2293515128 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faea4014480 con 0x7faeb006d250 2024-05-26T22:50:42.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.750+0000 7faeb6800640 1 -- 172.21.15.131:0/2293515128 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faeb0157ef0 con 0x7faeb006d250 2024-05-26T22:50:42.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.750+0000 7faead600640 1 -- 172.21.15.131:0/2293515128 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faea4014850 con 0x7faeb006d250 2024-05-26T22:50:42.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.750+0000 7faeb6800640 1 -- 172.21.15.131:0/2293515128 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faeb01584b0 con 0x7faeb006d250 2024-05-26T22:50:42.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.752+0000 7faeb6800640 1 -- 172.21.15.131:0/2293515128 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faeb00694e0 con 0x7faeb006d250 2024-05-26T22:50:42.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.753+0000 7faead600640 1 -- 172.21.15.131:0/2293515128 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7faea4005d10 con 0x7faeb006d250 2024-05-26T22:50:42.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.753+0000 7faead600640 1 --2- 172.21.15.131:0/2293515128 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fae9006be30 0x7fae9006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:42.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.754+0000 7faead600640 1 -- 172.21.15.131:0/2293515128 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7faea408ba00 con 0x7faeb006d250 2024-05-26T22:50:42.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.754+0000 7faeb4a00640 1 --2- 172.21.15.131:0/2293515128 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fae9006be30 0x7fae9006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:42.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.754+0000 7faeb4a00640 1 --2- 172.21.15.131:0/2293515128 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fae9006be30 0x7fae9006e2f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fae9c006fd0 tx=0x7fae9c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:42.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.758+0000 7faead600640 1 -- 172.21.15.131:0/2293515128 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7faea4022070 con 0x7faeb006d250 2024-05-26T22:50:42.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.956+0000 7faeb6800640 1 -- 172.21.15.131:0/2293515128 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command([{prefix=config set, name=mds_export_ephemeral_random_max}] v 0) -- 0x7faeb00af9b0 con 0x7faeb006d250 2024-05-26T22:50:42.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.962+0000 7faead600640 1 -- 172.21.15.131:0/2293515128 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_export_ephemeral_random_max}]=0 v22)=0 v22) ==== 131+0+0 (secure 0 0 0) 0x7faea4010040 con 0x7faeb006d250 2024-05-26T22:50:42.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.965+0000 7faeb6800640 1 -- 172.21.15.131:0/2293515128 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fae9006be30 msgr2=0x7fae9006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:42.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.965+0000 7faeb6800640 1 --2- 172.21.15.131:0/2293515128 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fae9006be30 0x7fae9006e2f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fae9c006fd0 tx=0x7fae9c008040 comp rx=0 tx=0).stop 2024-05-26T22:50:42.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.965+0000 7faeb6800640 1 -- 172.21.15.131:0/2293515128 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faeb006d250 msgr2=0x7faeb01e7ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:42.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.965+0000 7faeb6800640 1 --2- 172.21.15.131:0/2293515128 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faeb006d250 0x7faeb01e7ef0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7faea400efc0 tx=0x7faea400c490 comp rx=0 tx=0).stop 2024-05-26T22:50:42.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.965+0000 7faeb6800640 1 -- 172.21.15.131:0/2293515128 shutdown_connections 2024-05-26T22:50:42.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.965+0000 7faeb6800640 1 --2- 172.21.15.131:0/2293515128 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faeb015b730 0x7faeb0152aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:42.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.965+0000 7faeb6800640 1 --2- 172.21.15.131:0/2293515128 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fae9006be30 0x7fae9006e2f0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:42.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.965+0000 7faeb6800640 1 --2- 172.21.15.131:0/2293515128 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faeb01400c0 0x7faeb0157150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:42.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.966+0000 7faeb6800640 1 --2- 172.21.15.131:0/2293515128 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faeb006d250 0x7faeb01e7ef0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:42.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.966+0000 7faeb6800640 1 -- 172.21.15.131:0/2293515128 >> 172.21.15.131:0/2293515128 conn(0x7faeb00b5660 msgr2=0x7faeb0074400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:42.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.966+0000 7faeb6800640 1 -- 172.21.15.131:0/2293515128 shutdown_connections 2024-05-26T22:50:42.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:41.966+0000 7faeb6800640 1 -- 172.21.15.131:0/2293515128 wait complete. 2024-05-26T22:50:42.074 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-05-26T22:50:42.074 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2024-05-26T22:50:42.074 DEBUG:teuthology.orchestra.run.smithi131:> 2024-05-26T22:50:42.074 DEBUG:teuthology.orchestra.run.smithi131:> # Use up a random number of inode numbers so the ephemeral pinning is not the same every test. 2024-05-26T22:50:42.074 DEBUG:teuthology.orchestra.run.smithi131:> mkdir .inode_number_thrash 2024-05-26T22:50:42.074 DEBUG:teuthology.orchestra.run.smithi131:> count=$((RANDOM % 1024)) 2024-05-26T22:50:42.075 DEBUG:teuthology.orchestra.run.smithi131:> for ((i = 0; i < count; i++)); do touch .inode_number_thrash/$i; done 2024-05-26T22:50:42.075 DEBUG:teuthology.orchestra.run.smithi131:> rm -rf .inode_number_thrash 2024-05-26T22:50:42.075 DEBUG:teuthology.orchestra.run.smithi131:> ') 2024-05-26T22:50:42.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:42 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-26T22:50:44.157 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-26T22:50:44.167 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-26T22:50:46.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-26T22:50:46.088 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2024-05-26T22:50:46.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:46.286+0000 7f2e7da00640 1 Processor -- start 2024-05-26T22:50:46.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:46.286+0000 7f2e7da00640 1 -- start start 2024-05-26T22:50:46.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:46.288+0000 7f2e7da00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2e780d0460 0x7f2e7806d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:46.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:46.288+0000 7f2e7da00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e78160180 con 0x7f2e7806f420 2024-05-26T22:50:46.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:46.288+0000 7f2e7da00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e78160460 con 0x7f2e780b5450 2024-05-26T22:50:46.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:46.288+0000 7f2e7da00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f2e78160d90 con 0x7f2e780d0460 2024-05-26T22:50:46.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:46.288+0000 7f2e6fe00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f2e7806f420 0x7f2e780cfd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:53170/0 (socket says 172.21.15.131:53170) 2024-05-26T22:50:46.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:46.288+0000 7f2e6fe00640 1 -- 172.21.15.131:0/1821939761 learned_addr learned my addr 172.21.15.131:0/1821939761 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:50:46.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:46.289+0000 7f2e77400640 1 --2- 172.21.15.131:0/1821939761 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2e780d0460 0x7f2e7806d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:46.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:46.289+0000 7f2e76000640 1 -- 172.21.15.131:0/1821939761 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3712847566 0 0) 0x7f2e78160180 con 0x7f2e7806f420 2024-05-26T22:50:46.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:46.289+0000 7f2e76000640 1 -- 172.21.15.131:0/1821939761 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2e5c003520 con 0x7f2e7806f420 2024-05-26T22:50:46.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:46.289+0000 7f2e76000640 1 -- 172.21.15.131:0/1821939761 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3248982071 0 0) 0x7f2e5c003520 con 0x7f2e7806f420 2024-05-26T22:50:46.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:46.289+0000 7f2e76000640 1 -- 172.21.15.131:0/1821939761 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2e780d0460 msgr2=0x7f2e7806d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:46.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:46.289+0000 7f2e76000640 1 --2- 172.21.15.131:0/1821939761 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2e780d0460 0x7f2e7806d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:46.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:46.289+0000 7f2e76000640 1 -- 172.21.15.131:0/1821939761 >> v1:172.21.15.187:6789/0 conn(0x7f2e780b5450 legacy=0x7f2e780b43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:46.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:46.289+0000 7f2e76000640 1 -- 172.21.15.131:0/1821939761 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e78160740 con 0x7f2e7806f420 2024-05-26T22:50:46.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:46.289+0000 7f2e76000640 1 -- 172.21.15.131:0/1821939761 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7f2e60001670 con 0x7f2e7806f420 2024-05-26T22:50:46.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:46.289+0000 7f2e76000640 1 -- 172.21.15.131:0/1821939761 >> v1:172.21.15.131:6789/0 conn(0x7f2e7806f420 legacy=0x7f2e780cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:46.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:46.289+0000 7f2e76000640 1 --2- 172.21.15.131:0/1821939761 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2e5c0039e0 0x7f2e5c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:46.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:46.289+0000 7f2e76000640 1 --2- 172.21.15.131:0/1821939761 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2e780d0460 0x7f2e5c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:46.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:46.289+0000 7f2e76000640 1 --2- 172.21.15.131:0/1821939761 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2e5c004d40 0x7f2e5c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:46.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:46.289+0000 7f2e76000640 1 -- 172.21.15.131:0/1821939761 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f2e78160d90 con 0x7f2e5c0039e0 2024-05-26T22:50:46.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:46.289+0000 7f2e76000640 1 -- 172.21.15.131:0/1821939761 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2e5c00bb00 con 0x7f2e5c004d40 2024-05-26T22:50:46.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:46.289+0000 7f2e76000640 1 -- 172.21.15.131:0/1821939761 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f2e5c00bd00 con 0x7f2e780d0460 2024-05-26T22:50:46.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:46.290+0000 7f2e76a00640 1 --2- 172.21.15.131:0/1821939761 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2e5c0039e0 0x7f2e5c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:46.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:46.290+0000 7f2e6fe00640 1 --2- 172.21.15.131:0/1821939761 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2e780d0460 0x7f2e5c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:46.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:46.290+0000 7f2e76a00640 1 -- 172.21.15.131:0/1821939761 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2e780d0460 msgr2=0x7f2e5c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:46.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:46.290+0000 7f2e76a00640 1 --2- 172.21.15.131:0/1821939761 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2e780d0460 0x7f2e5c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:46.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:46.290+0000 7f2e76a00640 1 -- 172.21.15.131:0/1821939761 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2e5c004d40 msgr2=0x7f2e5c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:46.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:46.290+0000 7f2e76a00640 1 --2- 172.21.15.131:0/1821939761 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2e5c004d40 0x7f2e5c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:46.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:46.290+0000 7f2e76a00640 1 -- 172.21.15.131:0/1821939761 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e5c003520 con 0x7f2e5c0039e0 2024-05-26T22:50:46.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:46.290+0000 7f2e6fe00640 1 --2- 172.21.15.131:0/1821939761 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2e780d0460 0x7f2e5c004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:50:46.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:46.291+0000 7f2e76a00640 1 --2- 172.21.15.131:0/1821939761 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2e5c0039e0 0x7f2e5c003e00 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f2e68001750 tx=0x7f2e5c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6425d0504cb2a2d server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:46.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:46.292+0000 7f2e76000640 1 -- 172.21.15.131:0/1821939761 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e68001370 con 0x7f2e5c0039e0 2024-05-26T22:50:46.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:46.292+0000 7f2e76000640 1 -- 172.21.15.131:0/1821939761 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e680052d0 con 0x7f2e5c0039e0 2024-05-26T22:50:46.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:46.292+0000 7f2e76000640 1 -- 172.21.15.131:0/1821939761 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e6802ab20 con 0x7f2e5c0039e0 2024-05-26T22:50:46.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:46.292+0000 7f2e7da00640 1 -- 172.21.15.131:0/1821939761 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2e5c0039e0 msgr2=0x7f2e5c003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:46.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:46.292+0000 7f2e7da00640 1 --2- 172.21.15.131:0/1821939761 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2e5c0039e0 0x7f2e5c003e00 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f2e68001750 tx=0x7f2e5c005830 comp rx=0 tx=0).stop 2024-05-26T22:50:46.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:46.292+0000 7f2e77400640 1 -- 172.21.15.131:0/1821939761 reap_dead start 2024-05-26T22:50:46.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:46.292+0000 7f2e7da00640 1 -- 172.21.15.131:0/1821939761 shutdown_connections 2024-05-26T22:50:46.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:46.292+0000 7f2e7da00640 1 -- 172.21.15.131:0/1821939761 >> 172.21.15.131:0/1821939761 conn(0x7f2e7806bc90 msgr2=0x7f2e780b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:46.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:46.292+0000 7f2e7da00640 1 -- 172.21.15.131:0/1821939761 shutdown_connections 2024-05-26T22:50:46.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:46.292+0000 7f2e7da00640 1 -- 172.21.15.131:0/1821939761 wait complete. 2024-05-26T22:50:46.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:46.294+0000 7f2e7da00640 1 Processor -- start 2024-05-26T22:50:46.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:46.294+0000 7f2e7da00640 1 -- start start 2024-05-26T22:50:46.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:46.295+0000 7f2e7da00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2e5c0039e0 0x7f2e780b1200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:46.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:46.295+0000 7f2e7da00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2e7814fd70 0x7f2e780c5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:46.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:46.295+0000 7f2e7da00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2e7814f7a0 0x7f2e7815ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:46.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:46.295+0000 7f2e7da00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f2e7815fde0 con 0x7f2e5c0039e0 2024-05-26T22:50:46.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:46.295+0000 7f2e7da00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2e780b5830 con 0x7f2e7814f7a0 2024-05-26T22:50:46.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:46.295+0000 7f2e7da00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f2e7815ff60 con 0x7f2e7814fd70 2024-05-26T22:50:46.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:46.296+0000 7f2e76a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2e5c0039e0 0x7f2e780b1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:46.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:46.296+0000 7f2e76a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2e5c0039e0 0x7f2e780b1200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54122/0 (socket says 172.21.15.131:54122) 2024-05-26T22:50:46.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:46.296+0000 7f2e76a00640 1 -- 172.21.15.131:0/3380769788 learned_addr learned my addr 172.21.15.131:0/3380769788 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:46.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:46.296+0000 7f2e76a00640 1 -- 172.21.15.131:0/3380769788 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2e7814fd70 msgr2=0x7f2e780c5840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:46.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:46.296+0000 7f2e76a00640 1 --2- 172.21.15.131:0/3380769788 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2e7814fd70 0x7f2e780c5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:46.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:46.296+0000 7f2e76a00640 1 -- 172.21.15.131:0/3380769788 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2e7814f7a0 msgr2=0x7f2e7815ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:46.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:46.296+0000 7f2e76a00640 1 --2- 172.21.15.131:0/3380769788 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2e7814f7a0 0x7f2e7815ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:46.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:46.296+0000 7f2e76a00640 1 -- 172.21.15.131:0/3380769788 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e780af250 con 0x7f2e5c0039e0 2024-05-26T22:50:46.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:46.296+0000 7f2e76a00640 1 --2- 172.21.15.131:0/3380769788 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2e5c0039e0 0x7f2e780b1200 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f2e68000c00 tx=0x7f2e68001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:46.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:46.297+0000 7f2e6ea00640 1 -- 172.21.15.131:0/3380769788 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e68040020 con 0x7f2e5c0039e0 2024-05-26T22:50:46.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:46.297+0000 7f2e6ea00640 1 -- 172.21.15.131:0/3380769788 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e68005480 con 0x7f2e5c0039e0 2024-05-26T22:50:46.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:46.297+0000 7f2e6ea00640 1 -- 172.21.15.131:0/3380769788 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e68007a30 con 0x7f2e5c0039e0 2024-05-26T22:50:46.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:46.297+0000 7f2e7da00640 1 -- 172.21.15.131:0/3380769788 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e780af530 con 0x7f2e5c0039e0 2024-05-26T22:50:46.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:46.297+0000 7f2e7da00640 1 -- 172.21.15.131:0/3380769788 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e780b0a00 con 0x7f2e5c0039e0 2024-05-26T22:50:46.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:46.298+0000 7f2e7da00640 1 -- 172.21.15.131:0/3380769788 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e780695c0 con 0x7f2e5c0039e0 2024-05-26T22:50:46.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:46.299+0000 7f2e6ea00640 1 -- 172.21.15.131:0/3380769788 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2e68043020 con 0x7f2e5c0039e0 2024-05-26T22:50:46.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:46.300+0000 7f2e6ea00640 1 --2- 172.21.15.131:0/3380769788 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f2e5406be60 0x7f2e5406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:46.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:46.300+0000 7f2e6ea00640 1 -- 172.21.15.131:0/3380769788 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f2e680aace0 con 0x7f2e5c0039e0 2024-05-26T22:50:46.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:46.300+0000 7f2e6fe00640 1 --2- 172.21.15.131:0/3380769788 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f2e5406be60 0x7f2e5406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:46.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:46.301+0000 7f2e6fe00640 1 --2- 172.21.15.131:0/3380769788 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f2e5406be60 0x7f2e5406e320 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f2e60000ff0 tx=0x7f2e60009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:46.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:46.305+0000 7f2e6ea00640 1 -- 172.21.15.131:0/3380769788 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f2e6804a030 con 0x7f2e5c0039e0 2024-05-26T22:50:46.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:46.545+0000 7f2e7da00640 1 -- 172.21.15.131:0/3380769788 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7f2e7815f4b0 con 0x7f2e5c0039e0 2024-05-26T22:50:47.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:47.083+0000 7f2e6ea00640 1 -- 172.21.15.131:0/3380769788 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v30) ==== 105+0+0 (secure 0 0 0) 0x7f2e68079680 con 0x7f2e5c0039e0 2024-05-26T22:50:47.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:47.088+0000 7f2e7da00640 1 -- 172.21.15.131:0/3380769788 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f2e5406be60 msgr2=0x7f2e5406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:47.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:47.088+0000 7f2e7da00640 1 --2- 172.21.15.131:0/3380769788 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f2e5406be60 0x7f2e5406e320 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f2e60000ff0 tx=0x7f2e60009040 comp rx=0 tx=0).stop 2024-05-26T22:50:47.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:47.088+0000 7f2e7da00640 1 -- 172.21.15.131:0/3380769788 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2e5c0039e0 msgr2=0x7f2e780b1200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:47.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:47.088+0000 7f2e7da00640 1 --2- 172.21.15.131:0/3380769788 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2e5c0039e0 0x7f2e780b1200 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f2e68000c00 tx=0x7f2e68001120 comp rx=0 tx=0).stop 2024-05-26T22:50:47.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:47.088+0000 7f2e7da00640 1 -- 172.21.15.131:0/3380769788 shutdown_connections 2024-05-26T22:50:47.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:47.088+0000 7f2e7da00640 1 --2- 172.21.15.131:0/3380769788 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2e7814fd70 0x7f2e780c5840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:47.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:47.088+0000 7f2e7da00640 1 --2- 172.21.15.131:0/3380769788 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f2e5406be60 0x7f2e5406e320 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:47.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:47.088+0000 7f2e7da00640 1 --2- 172.21.15.131:0/3380769788 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2e7814f7a0 0x7f2e7815ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:47.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:47.088+0000 7f2e7da00640 1 --2- 172.21.15.131:0/3380769788 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2e5c0039e0 0x7f2e780b1200 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:47.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:47.088+0000 7f2e7da00640 1 -- 172.21.15.131:0/3380769788 >> 172.21.15.131:0/3380769788 conn(0x7f2e7806bc90 msgr2=0x7f2e7815a420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:47.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:47.088+0000 7f2e7da00640 1 -- 172.21.15.131:0/3380769788 shutdown_connections 2024-05-26T22:50:47.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:47.089+0000 7f2e7da00640 1 -- 172.21.15.131:0/3380769788 wait complete. 2024-05-26T22:50:47.116 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-26T22:50:47.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:47.295+0000 7f188e200640 1 Processor -- start 2024-05-26T22:50:47.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:47.295+0000 7f188e200640 1 -- start start 2024-05-26T22:50:47.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:47.296+0000 7f188e200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f1888155840 0x7f1888071930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:47.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:47.296+0000 7f188e200640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f188815c3d0 0x7f188806ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:47.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:47.297+0000 7f188e200640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18880722a0 con 0x7f188806f490 2024-05-26T22:50:47.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:47.297+0000 7f188e200640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f1888072580 con 0x7f1888155840 2024-05-26T22:50:47.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:47.297+0000 7f188e200640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f1888072860 con 0x7f188815c3d0 2024-05-26T22:50:47.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:47.297+0000 7f1887400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f1888155840 0x7f1888071930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:47.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:47.297+0000 7f1887400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f1888155840 0x7f1888071930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54128/0 (socket says 172.21.15.131:54128) 2024-05-26T22:50:47.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:47.297+0000 7f1887400640 1 -- 172.21.15.131:0/4189436349 learned_addr learned my addr 172.21.15.131:0/4189436349 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:47.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:47.297+0000 7f1886a00640 1 --2- 172.21.15.131:0/4189436349 >> v2:172.21.15.187:3300/0 conn(0x7f188815c3d0 0x7f188806ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:47.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:47.297+0000 7f1887400640 1 -- 172.21.15.131:0/4189436349 >> v2:172.21.15.187:3300/0 conn(0x7f188815c3d0 msgr2=0x7f188806ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:47.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:47.297+0000 7f1887400640 1 --2- 172.21.15.131:0/4189436349 >> v2:172.21.15.187:3300/0 conn(0x7f188815c3d0 0x7f188806ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:47.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:47.297+0000 7f1887400640 1 -- 172.21.15.131:0/4189436349 >> v1:172.21.15.187:6789/0 conn(0x7f188806f490 legacy=0x7f188815ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:47.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:47.298+0000 7f1887400640 1 -- 172.21.15.131:0/4189436349 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1888072b40 con 0x7f1888155840 2024-05-26T22:50:47.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:47.299+0000 7f1887400640 1 --2- 172.21.15.131:0/4189436349 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1888155840 0x7f1888071930 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f1888159280 tx=0x7f187802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16dcef6aaa614f09 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:47.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:47.299+0000 7f1886000640 1 -- 172.21.15.131:0/4189436349 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1878037070 con 0x7f1888155840 2024-05-26T22:50:47.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:47.299+0000 7f1886000640 1 -- 172.21.15.131:0/4189436349 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1878032440 con 0x7f1888155840 2024-05-26T22:50:47.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:47.299+0000 7f1886000640 1 -- 172.21.15.131:0/4189436349 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1878032810 con 0x7f1888155840 2024-05-26T22:50:47.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:47.299+0000 7f188e200640 1 -- 172.21.15.131:0/4189436349 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1888155840 msgr2=0x7f1888071930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:47.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:47.299+0000 7f188e200640 1 --2- 172.21.15.131:0/4189436349 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1888155840 0x7f1888071930 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f1888159280 tx=0x7f187802a230 comp rx=0 tx=0).stop 2024-05-26T22:50:47.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:47.300+0000 7f188e200640 1 -- 172.21.15.131:0/4189436349 shutdown_connections 2024-05-26T22:50:47.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:47.300+0000 7f188e200640 1 --2- 172.21.15.131:0/4189436349 >> v2:172.21.15.187:3300/0 conn(0x7f188815c3d0 0x7f188806ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:47.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:47.300+0000 7f188e200640 1 --2- 172.21.15.131:0/4189436349 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1888155840 0x7f1888071930 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:47.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:47.300+0000 7f188e200640 1 -- 172.21.15.131:0/4189436349 >> 172.21.15.131:0/4189436349 conn(0x7f18880b1e90 msgr2=0x7f18880b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:47.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:47.300+0000 7f188e200640 1 -- 172.21.15.131:0/4189436349 shutdown_connections 2024-05-26T22:50:47.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:47.300+0000 7f188e200640 1 -- 172.21.15.131:0/4189436349 wait complete. 2024-05-26T22:50:47.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:47.301+0000 7f188e200640 1 Processor -- start 2024-05-26T22:50:47.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:47.302+0000 7f188e200640 1 -- start start 2024-05-26T22:50:47.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:47.302+0000 7f188e200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f188806f490 0x7f188814dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:47.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:47.302+0000 7f188e200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1888155840 0x7f188814eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:47.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:47.302+0000 7f188e200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f188815c3d0 0x7f188815d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:47.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:47.302+0000 7f188e200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f1888071ff0 con 0x7f1888155840 2024-05-26T22:50:47.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:47.302+0000 7f188e200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1888071e70 con 0x7f188806f490 2024-05-26T22:50:47.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:47.302+0000 7f188e200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f18880aee50 con 0x7f188815c3d0 2024-05-26T22:50:47.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:47.303+0000 7f1886a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1888155840 0x7f188814eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:47.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:47.303+0000 7f1886a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1888155840 0x7f188814eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54144/0 (socket says 172.21.15.131:54144) 2024-05-26T22:50:47.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:47.303+0000 7f1886a00640 1 -- 172.21.15.131:0/26825120 learned_addr learned my addr 172.21.15.131:0/26825120 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:47.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:47.303+0000 7f1887400640 1 --2- 172.21.15.131:0/26825120 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f188806f490 0x7f188814dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:47.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:47.303+0000 7f1886a00640 1 -- 172.21.15.131:0/26825120 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f188815c3d0 msgr2=0x7f188815d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:47.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:47.303+0000 7f1886a00640 1 --2- 172.21.15.131:0/26825120 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f188815c3d0 0x7f188815d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:47.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:47.303+0000 7f1886a00640 1 -- 172.21.15.131:0/26825120 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f188806f490 msgr2=0x7f188814dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:47.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:47.303+0000 7f1886a00640 1 --2- 172.21.15.131:0/26825120 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f188806f490 0x7f188814dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:47.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:47.303+0000 7f1886a00640 1 -- 172.21.15.131:0/26825120 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18881516a0 con 0x7f1888155840 2024-05-26T22:50:47.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:47.303+0000 7f1886a00640 1 --2- 172.21.15.131:0/26825120 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1888155840 0x7f188814eb80 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f187400d5f0 tx=0x7f187400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:47.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:47.304+0000 7f187f400640 1 -- 172.21.15.131:0/26825120 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18740125f0 con 0x7f1888155840 2024-05-26T22:50:47.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:47.304+0000 7f187f400640 1 -- 172.21.15.131:0/26825120 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1874019070 con 0x7f1888155840 2024-05-26T22:50:47.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:47.304+0000 7f187f400640 1 -- 172.21.15.131:0/26825120 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18740129d0 con 0x7f1888155840 2024-05-26T22:50:47.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:47.304+0000 7f188e200640 1 -- 172.21.15.131:0/26825120 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1888151980 con 0x7f1888155840 2024-05-26T22:50:47.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:47.304+0000 7f188e200640 1 -- 172.21.15.131:0/26825120 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1888151f40 con 0x7f1888155840 2024-05-26T22:50:47.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:47.305+0000 7f188e200640 1 -- 172.21.15.131:0/26825120 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18880694e0 con 0x7f1888155840 2024-05-26T22:50:47.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:47.306+0000 7f187f400640 1 -- 172.21.15.131:0/26825120 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1874012b70 con 0x7f1888155840 2024-05-26T22:50:47.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:47.307+0000 7f187f400640 1 --2- 172.21.15.131:0/26825120 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f186806be60 0x7f186806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:47.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:47.307+0000 7f187f400640 1 -- 172.21.15.131:0/26825120 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f187408b2d0 con 0x7f1888155840 2024-05-26T22:50:47.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:47.307+0000 7f1887400640 1 --2- 172.21.15.131:0/26825120 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f186806be60 0x7f186806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:47.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:47.308+0000 7f1887400640 1 --2- 172.21.15.131:0/26825120 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f186806be60 0x7f186806e320 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f1878034000 tx=0x7f187802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:47.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:47.312+0000 7f187f400640 1 -- 172.21.15.131:0/26825120 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f1874059be0 con 0x7f1888155840 2024-05-26T22:50:47.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:47.548+0000 7f188e200640 1 -- 172.21.15.131:0/26825120 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f18880b0250 con 0x7f1888155840 2024-05-26T22:50:47.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:47.549+0000 7f187f400640 1 -- 172.21.15.131:0/26825120 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+5203 (secure 0 0 0) 0x7f18740594e0 con 0x7f1888155840 2024-05-26T22:50:47.550 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:50:47.550 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":30,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4661,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/265040431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6835","nonce":265040431},{"type":"v1","addr":"172.21.15.131:6837","nonce":265040431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24},{"gid":4670,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6834/1027783169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1027783169},{"type":"v1","addr":"172.21.15.187:6834","nonce":1027783169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-26T22:50:27.001665+0000","modified":"2024-05-26T22:50:47.081182+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1],"up":{"mds_0":5637,"mds_1":4918},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4918":{"gid":4918,"name":"d","rank":1,"incarnation":30,"state":"up:creating","state_seq":1,"addr":"172.21.15.187:6835/3429891101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6833","nonce":3429891101},{"type":"v1","addr":"172.21.15.187:6835","nonce":3429891101}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5637":{"gid":5637,"name":"c","rank":0,"incarnation":25,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6836/650770835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":650770835},{"type":"v1","addr":"172.21.15.131:6836","nonce":650770835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5637,"qdb_cluster":[5637]},"id":3}]} 2024-05-26T22:50:47.551 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 30 2024-05-26T22:50:47.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:47.552+0000 7f188e200640 1 -- 172.21.15.131:0/26825120 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f186806be60 msgr2=0x7f186806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:47.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:47.552+0000 7f188e200640 1 --2- 172.21.15.131:0/26825120 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f186806be60 0x7f186806e320 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f1878034000 tx=0x7f187802e000 comp rx=0 tx=0).stop 2024-05-26T22:50:47.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:47.552+0000 7f188e200640 1 -- 172.21.15.131:0/26825120 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1888155840 msgr2=0x7f188814eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:47.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:47.552+0000 7f188e200640 1 --2- 172.21.15.131:0/26825120 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1888155840 0x7f188814eb80 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f187400d5f0 tx=0x7f187400dab0 comp rx=0 tx=0).stop 2024-05-26T22:50:47.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:47.552+0000 7f188e200640 1 -- 172.21.15.131:0/26825120 shutdown_connections 2024-05-26T22:50:47.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:47.552+0000 7f188e200640 1 --2- 172.21.15.131:0/26825120 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f188815c3d0 0x7f188815d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:47.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:47.552+0000 7f188e200640 1 --2- 172.21.15.131:0/26825120 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f186806be60 0x7f186806e320 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:47.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:47.552+0000 7f188e200640 1 --2- 172.21.15.131:0/26825120 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1888155840 0x7f188814eb80 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:47.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:47.552+0000 7f188e200640 1 --2- 172.21.15.131:0/26825120 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f188806f490 0x7f188814dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:47.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:47.552+0000 7f188e200640 1 -- 172.21.15.131:0/26825120 >> 172.21.15.131:0/26825120 conn(0x7f18880b1e90 msgr2=0x7f188815a030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:47.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:47.552+0000 7f188e200640 1 -- 172.21.15.131:0/26825120 shutdown_connections 2024-05-26T22:50:47.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:47.552+0000 7f188e200640 1 -- 172.21.15.131:0/26825120 wait complete. 2024-05-26T22:50:47.974 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 30, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-26T22:50:27.001665+0000', 'modified': '2024-05-26T22:50:47.081182+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1], 'up': {'mds_0': 5637, 'mds_1': 4918}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4918': {'gid': 4918, 'name': 'd', 'rank': 1, 'incarnation': 30, 'state': 'up:creating', 'state_seq': 1, 'addr': '172.21.15.187:6835/3429891101', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.187:6833', 'nonce': 3429891101}, {'type': 'v1', 'addr': '172.21.15.187:6835', 'nonce': 3429891101}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_5637': {'gid': 5637, 'name': 'c', 'rank': 0, 'incarnation': 25, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.131:6836/650770835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6834', 'nonce': 650770835}, {'type': 'v1', 'addr': '172.21.15.131:6836', 'nonce': 650770835}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5637, 'qdb_cluster': [5637]} 2024-05-26T22:50:47.974 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_4918:up:creating 2024-05-26T22:50:48.974 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-26T22:50:49.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:49.172+0000 7f32b7000640 1 Processor -- start 2024-05-26T22:50:49.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:49.173+0000 7f32b7000640 1 -- start start 2024-05-26T22:50:49.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:49.174+0000 7f32b7000640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f32b015e580 0x7f32b00d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:49.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:49.174+0000 7f32b7000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f32b015e960 0x7f32b00b3020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:49.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:49.174+0000 7f32b7000640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32b00d1820 con 0x7f32b00b3560 2024-05-26T22:50:49.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:49.174+0000 7f32b7000640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f32b00d2820 con 0x7f32b015e580 2024-05-26T22:50:49.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:49.174+0000 7f32b7000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f32b00d02d0 con 0x7f32b015e960 2024-05-26T22:50:49.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:49.174+0000 7f32afe00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f32b015e580 0x7f32b00d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:49.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:49.174+0000 7f32afe00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f32b015e580 0x7f32b00d22e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54146/0 (socket says 172.21.15.131:54146) 2024-05-26T22:50:49.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:49.174+0000 7f32afe00640 1 -- 172.21.15.131:0/3653279151 learned_addr learned my addr 172.21.15.131:0/3653279151 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:49.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:49.175+0000 7f32afe00640 1 -- 172.21.15.131:0/3653279151 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f32b015e960 msgr2=0x7f32b00b3020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:49.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:49.175+0000 7f32af400640 1 --2- 172.21.15.131:0/3653279151 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f32b015e960 0x7f32b00b3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:49.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:49.175+0000 7f32afe00640 1 --2- 172.21.15.131:0/3653279151 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f32b015e960 0x7f32b00b3020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:49.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:49.175+0000 7f32afe00640 1 -- 172.21.15.131:0/3653279151 >> v1:172.21.15.187:6789/0 conn(0x7f32b00b3560 legacy=0x7f32b00d2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:49.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:49.175+0000 7f32afe00640 1 -- 172.21.15.131:0/3653279151 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32b00d4ce0 con 0x7f32b015e580 2024-05-26T22:50:49.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:49.176+0000 7f32afe00640 1 --2- 172.21.15.131:0/3653279151 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f32b015e580 0x7f32b00d22e0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f32a0009870 tx=0x7f32a002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62a26a6d9f6a79a server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:49.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:49.176+0000 7f32aea00640 1 -- 172.21.15.131:0/3653279151 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a0037070 con 0x7f32b015e580 2024-05-26T22:50:49.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:49.176+0000 7f32aea00640 1 -- 172.21.15.131:0/3653279151 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32a0032440 con 0x7f32b015e580 2024-05-26T22:50:49.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:49.176+0000 7f32aea00640 1 -- 172.21.15.131:0/3653279151 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a0032810 con 0x7f32b015e580 2024-05-26T22:50:49.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:49.177+0000 7f32b7000640 1 -- 172.21.15.131:0/3653279151 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f32b015e580 msgr2=0x7f32b00d22e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:49.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:49.177+0000 7f32b7000640 1 --2- 172.21.15.131:0/3653279151 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f32b015e580 0x7f32b00d22e0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f32a0009870 tx=0x7f32a002a180 comp rx=0 tx=0).stop 2024-05-26T22:50:49.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:49.177+0000 7f32b7000640 1 -- 172.21.15.131:0/3653279151 shutdown_connections 2024-05-26T22:50:49.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:49.177+0000 7f32b7000640 1 --2- 172.21.15.131:0/3653279151 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f32b015e960 0x7f32b00b3020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:49.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:49.177+0000 7f32b7000640 1 --2- 172.21.15.131:0/3653279151 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f32b015e580 0x7f32b00d22e0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:49.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:49.177+0000 7f32b7000640 1 -- 172.21.15.131:0/3653279151 >> 172.21.15.131:0/3653279151 conn(0x7f32b009d680 msgr2=0x7f32b00d4430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:49.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:49.177+0000 7f32b7000640 1 -- 172.21.15.131:0/3653279151 shutdown_connections 2024-05-26T22:50:49.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:49.177+0000 7f32b7000640 1 -- 172.21.15.131:0/3653279151 wait complete. 2024-05-26T22:50:49.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:49.179+0000 7f32b7000640 1 Processor -- start 2024-05-26T22:50:49.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:49.179+0000 7f32b7000640 1 -- start start 2024-05-26T22:50:49.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:49.180+0000 7f32b7000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f32b00b3560 0x7f32b0154be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:49.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:49.180+0000 7f32b7000640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f32b015e580 0x7f32b00b9860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:49.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:49.180+0000 7f32b7000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f32b015e960 0x7f32b00cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:49.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:49.180+0000 7f32b7000640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f32b00d3120 con 0x7f32b00b3560 2024-05-26T22:50:49.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:49.180+0000 7f32b7000640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f32b00d52a0 con 0x7f32b015e580 2024-05-26T22:50:49.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:49.180+0000 7f32b7000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f32b00d32a0 con 0x7f32b015e960 2024-05-26T22:50:49.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:49.180+0000 7f32afe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f32b00b3560 0x7f32b0154be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:49.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:49.180+0000 7f32afe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f32b00b3560 0x7f32b0154be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54150/0 (socket says 172.21.15.131:54150) 2024-05-26T22:50:49.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:49.180+0000 7f32afe00640 1 -- 172.21.15.131:0/2242776835 learned_addr learned my addr 172.21.15.131:0/2242776835 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:49.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:49.180+0000 7f32af400640 1 --2- 172.21.15.131:0/2242776835 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f32b015e580 0x7f32b00b9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:49.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:49.181+0000 7f32afe00640 1 -- 172.21.15.131:0/2242776835 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f32b015e960 msgr2=0x7f32b00cd6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:49.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:49.181+0000 7f32b5000640 1 --2- 172.21.15.131:0/2242776835 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f32b015e960 0x7f32b00cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:49.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:49.181+0000 7f32afe00640 1 --2- 172.21.15.131:0/2242776835 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f32b015e960 0x7f32b00cd6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:49.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:49.181+0000 7f32afe00640 1 -- 172.21.15.131:0/2242776835 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f32b015e580 msgr2=0x7f32b00b9860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:49.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:49.181+0000 7f32afe00640 1 --2- 172.21.15.131:0/2242776835 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f32b015e580 0x7f32b00b9860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:49.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:49.181+0000 7f32afe00640 1 -- 172.21.15.131:0/2242776835 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32b0155120 con 0x7f32b00b3560 2024-05-26T22:50:49.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:49.181+0000 7f32af400640 1 --2- 172.21.15.131:0/2242776835 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f32b015e580 0x7f32b00b9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:50:49.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:49.181+0000 7f32afe00640 1 --2- 172.21.15.131:0/2242776835 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f32b00b3560 0x7f32b0154be0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f32a0009870 tx=0x7f32a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:49.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:49.181+0000 7f32a7e00640 1 -- 172.21.15.131:0/2242776835 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a0037040 con 0x7f32b00b3560 2024-05-26T22:50:49.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:49.181+0000 7f32a7e00640 1 -- 172.21.15.131:0/2242776835 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32a0042070 con 0x7f32b00b3560 2024-05-26T22:50:49.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:49.181+0000 7f32b7000640 1 -- 172.21.15.131:0/2242776835 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32b00c7e50 con 0x7f32b00b3560 2024-05-26T22:50:49.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:49.181+0000 7f32a7e00640 1 -- 172.21.15.131:0/2242776835 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a0030bb0 con 0x7f32b00b3560 2024-05-26T22:50:49.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:49.182+0000 7f32b7000640 1 -- 172.21.15.131:0/2242776835 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32b00c8410 con 0x7f32b00b3560 2024-05-26T22:50:49.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:49.183+0000 7f32b7000640 1 -- 172.21.15.131:0/2242776835 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32b00694e0 con 0x7f32b00b3560 2024-05-26T22:50:49.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:49.184+0000 7f32a7e00640 1 -- 172.21.15.131:0/2242776835 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f32a0047460 con 0x7f32b00b3560 2024-05-26T22:50:49.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:49.185+0000 7f32a7e00640 1 --2- 172.21.15.131:0/2242776835 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f329006be60 0x7f329006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:49.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:49.185+0000 7f32a7e00640 1 -- 172.21.15.131:0/2242776835 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f32a00abae0 con 0x7f32b00b3560 2024-05-26T22:50:49.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:49.185+0000 7f32af400640 1 --2- 172.21.15.131:0/2242776835 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f329006be60 0x7f329006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:49.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:49.186+0000 7f32af400640 1 --2- 172.21.15.131:0/2242776835 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f329006be60 0x7f329006e320 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f32b00ce290 tx=0x7f329c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:49.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:49.190+0000 7f32a7e00640 1 -- 172.21.15.131:0/2242776835 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f32b0155120 con 0x7f32b00b3560 2024-05-26T22:50:49.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:49.428+0000 7f32b7000640 1 -- 172.21.15.131:0/2242776835 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f32b00d4210 con 0x7f32b00b3560 2024-05-26T22:50:49.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:49.429+0000 7f32a7e00640 1 -- 172.21.15.131:0/2242776835 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+5225 (secure 0 0 0) 0x7f32a0045360 con 0x7f32b00b3560 2024-05-26T22:50:49.430 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:50:49.430 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":33,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4661,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/265040431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6835","nonce":265040431},{"type":"v1","addr":"172.21.15.131:6837","nonce":265040431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-26T22:50:27.001665+0000","modified":"2024-05-26T22:50:49.114183+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5637,"mds_1":4918,"mds_2":4670},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4670":{"gid":4670,"name":"b","rank":2,"incarnation":32,"state":"up:active","state_seq":8,"addr":"172.21.15.187:6834/1027783169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1027783169},{"type":"v1","addr":"172.21.15.187:6834","nonce":1027783169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4918":{"gid":4918,"name":"d","rank":1,"incarnation":30,"state":"up:active","state_seq":8,"addr":"172.21.15.187:6835/3429891101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6833","nonce":3429891101},{"type":"v1","addr":"172.21.15.187:6835","nonce":3429891101}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5637":{"gid":5637,"name":"c","rank":0,"incarnation":25,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6836/650770835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":650770835},{"type":"v1","addr":"172.21.15.131:6836","nonce":650770835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5637,"qdb_cluster":[4670,4918,5637]},"id":3}]} 2024-05-26T22:50:49.430 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 33 2024-05-26T22:50:49.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:49.432+0000 7f32b7000640 1 -- 172.21.15.131:0/2242776835 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f329006be60 msgr2=0x7f329006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:49.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:49.432+0000 7f32b7000640 1 --2- 172.21.15.131:0/2242776835 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f329006be60 0x7f329006e320 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f32b00ce290 tx=0x7f329c008040 comp rx=0 tx=0).stop 2024-05-26T22:50:49.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:49.432+0000 7f32b7000640 1 -- 172.21.15.131:0/2242776835 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f32b00b3560 msgr2=0x7f32b0154be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:49.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:49.432+0000 7f32b7000640 1 --2- 172.21.15.131:0/2242776835 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f32b00b3560 0x7f32b0154be0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f32a0009870 tx=0x7f32a00046c0 comp rx=0 tx=0).stop 2024-05-26T22:50:49.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:49.432+0000 7f32b7000640 1 -- 172.21.15.131:0/2242776835 shutdown_connections 2024-05-26T22:50:49.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:49.432+0000 7f32b7000640 1 --2- 172.21.15.131:0/2242776835 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f32b015e960 0x7f32b00cd6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:49.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:49.432+0000 7f32b7000640 1 --2- 172.21.15.131:0/2242776835 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f329006be60 0x7f329006e320 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:49.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:49.432+0000 7f32b7000640 1 --2- 172.21.15.131:0/2242776835 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f32b015e580 0x7f32b00b9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:49.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:49.432+0000 7f32b7000640 1 --2- 172.21.15.131:0/2242776835 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f32b00b3560 0x7f32b0154be0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:49.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:49.432+0000 7f32b7000640 1 -- 172.21.15.131:0/2242776835 >> 172.21.15.131:0/2242776835 conn(0x7f32b009d680 msgr2=0x7f32b015cfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:49.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:49.433+0000 7f32b7000640 1 -- 172.21.15.131:0/2242776835 shutdown_connections 2024-05-26T22:50:49.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:49.433+0000 7f32b7000640 1 -- 172.21.15.131:0/2242776835 wait complete. 2024-05-26T22:50:49.855 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 33, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-26T22:50:27.001665+0000', 'modified': '2024-05-26T22:50:49.114183+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 5637, 'mds_1': 4918, 'mds_2': 4670}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4670': {'gid': 4670, 'name': 'b', 'rank': 2, 'incarnation': 32, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.187:6834/1027783169', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.187:6832', 'nonce': 1027783169}, {'type': 'v1', 'addr': '172.21.15.187:6834', 'nonce': 1027783169}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4918': {'gid': 4918, 'name': 'd', 'rank': 1, 'incarnation': 30, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.187:6835/3429891101', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.187:6833', 'nonce': 3429891101}, {'type': 'v1', 'addr': '172.21.15.187:6835', 'nonce': 3429891101}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_5637': {'gid': 5637, 'name': 'c', 'rank': 0, 'incarnation': 25, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.131:6836/650770835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6834', 'nonce': 650770835}, {'type': 'v1', 'addr': '172.21.15.131:6836', 'nonce': 650770835}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5637, 'qdb_cluster': [4670, 4918, 5637]} 2024-05-26T22:50:49.855 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-05-26T22:50:49.855 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-05-26T22:50:50.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.052+0000 7f5bd1000640 1 Processor -- start 2024-05-26T22:50:50.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.052+0000 7f5bd1000640 1 -- start start 2024-05-26T22:50:50.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.054+0000 7f5bd1000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5bcc0b9f00 0x7f5bcc0b5c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:50.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.054+0000 7f5bd1000640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f5bcc0d3c40 0x7f5bcc0b6190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:50.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.054+0000 7f5bd1000640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5bcc0d2e80 con 0x7f5bcc15e3e0 2024-05-26T22:50:50.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.054+0000 7f5bd1000640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f5bcc0cf910 con 0x7f5bcc0d3c40 2024-05-26T22:50:50.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.054+0000 7f5bd1000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f5bcc0d2c30 con 0x7f5bcc0b9f00 2024-05-26T22:50:50.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.054+0000 7f5bcb400640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f5bcc15e3e0 0x7f5bcc0cf010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:53176/0 (socket says 172.21.15.131:53176) 2024-05-26T22:50:50.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.054+0000 7f5bcb400640 1 -- 172.21.15.131:0/1734913164 learned_addr learned my addr 172.21.15.131:0/1734913164 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:50:50.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.054+0000 7f5bcaa00640 1 -- 172.21.15.131:0/1734913164 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3095743517 0 0) 0x7f5bcc0d2e80 con 0x7f5bcc15e3e0 2024-05-26T22:50:50.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.054+0000 7f5bcbe00640 1 --2- 172.21.15.131:0/1734913164 >> v2:172.21.15.187:3300/0 conn(0x7f5bcc0d3c40 0x7f5bcc0b6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:50.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.055+0000 7f5bcaa00640 1 -- 172.21.15.131:0/1734913164 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5bb0003520 con 0x7f5bcc15e3e0 2024-05-26T22:50:50.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.055+0000 7f5bc3e00640 1 --2- 172.21.15.131:0/1734913164 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5bcc0b9f00 0x7f5bcc0b5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:50.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.055+0000 7f5bcaa00640 1 -- 172.21.15.131:0/1734913164 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 347863624 0 0) 0x7f5bb0003520 con 0x7f5bcc15e3e0 2024-05-26T22:50:50.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.055+0000 7f5bcaa00640 1 -- 172.21.15.131:0/1734913164 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5bcc0b9f00 msgr2=0x7f5bcc0b5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:50.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.055+0000 7f5bcaa00640 1 --2- 172.21.15.131:0/1734913164 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5bcc0b9f00 0x7f5bcc0b5c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:50.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.055+0000 7f5bcaa00640 1 -- 172.21.15.131:0/1734913164 >> v2:172.21.15.187:3300/0 conn(0x7f5bcc0d3c40 msgr2=0x7f5bcc0b6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:50.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.055+0000 7f5bcaa00640 1 --2- 172.21.15.131:0/1734913164 >> v2:172.21.15.187:3300/0 conn(0x7f5bcc0d3c40 0x7f5bcc0b6190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:50.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.055+0000 7f5bcaa00640 1 -- 172.21.15.131:0/1734913164 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bcc0d3160 con 0x7f5bcc15e3e0 2024-05-26T22:50:50.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.055+0000 7f5bcaa00640 1 -- 172.21.15.131:0/1734913164 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7f5bbc001670 con 0x7f5bcc15e3e0 2024-05-26T22:50:50.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.055+0000 7f5bcaa00640 1 -- 172.21.15.131:0/1734913164 >> v1:172.21.15.131:6789/0 conn(0x7f5bcc15e3e0 legacy=0x7f5bcc0cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:50.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.055+0000 7f5bcaa00640 1 --2- 172.21.15.131:0/1734913164 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5bcc0b9f00 0x7f5bb00039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:50.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.055+0000 7f5bcaa00640 1 --2- 172.21.15.131:0/1734913164 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5bb0003f80 0x7f5bb0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:50.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.055+0000 7f5bcaa00640 1 --2- 172.21.15.131:0/1734913164 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5bb0004d40 0x7f5bb00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:50.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.055+0000 7f5bcaa00640 1 -- 172.21.15.131:0/1734913164 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f5bcc0d2c30 con 0x7f5bb0004d40 2024-05-26T22:50:50.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.055+0000 7f5bcaa00640 1 -- 172.21.15.131:0/1734913164 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb000b9f0 con 0x7f5bb0003f80 2024-05-26T22:50:50.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.055+0000 7f5bcaa00640 1 -- 172.21.15.131:0/1734913164 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f5bb000bbf0 con 0x7f5bcc0b9f00 2024-05-26T22:50:50.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.055+0000 7f5bcbe00640 1 --2- 172.21.15.131:0/1734913164 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5bb0004d40 0x7f5bb00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:50.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.056+0000 7f5bcb400640 1 --2- 172.21.15.131:0/1734913164 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5bcc0b9f00 0x7f5bb00039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:50.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.056+0000 7f5bcbe00640 1 -- 172.21.15.131:0/1734913164 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5bcc0b9f00 msgr2=0x7f5bb00039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:50.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.056+0000 7f5bcbe00640 1 --2- 172.21.15.131:0/1734913164 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5bcc0b9f00 0x7f5bb00039b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:50.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.056+0000 7f5bcbe00640 1 -- 172.21.15.131:0/1734913164 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5bb0003f80 msgr2=0x7f5bb0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:50.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.056+0000 7f5bcbe00640 1 --2- 172.21.15.131:0/1734913164 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5bb0003f80 0x7f5bb0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:50.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.056+0000 7f5bcbe00640 1 -- 172.21.15.131:0/1734913164 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bb0003520 con 0x7f5bb0004d40 2024-05-26T22:50:50.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.057+0000 7f5bcbe00640 1 --2- 172.21.15.131:0/1734913164 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5bb0004d40 0x7f5bb00051c0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f5bc4009fc0 tx=0x7f5bb0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23a69f5091b0e6a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:50.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.057+0000 7f5bcaa00640 1 -- 172.21.15.131:0/1734913164 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bc4009c10 con 0x7f5bb0004d40 2024-05-26T22:50:50.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.057+0000 7f5bcaa00640 1 -- 172.21.15.131:0/1734913164 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bc4036070 con 0x7f5bb0004d40 2024-05-26T22:50:50.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.057+0000 7f5bcaa00640 1 -- 172.21.15.131:0/1734913164 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bc402d070 con 0x7f5bb0004d40 2024-05-26T22:50:50.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.058+0000 7f5bd1000640 1 -- 172.21.15.131:0/1734913164 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5bb0004d40 msgr2=0x7f5bb00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:50.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.058+0000 7f5bd1000640 1 --2- 172.21.15.131:0/1734913164 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5bb0004d40 0x7f5bb00051c0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f5bc4009fc0 tx=0x7f5bb0005830 comp rx=0 tx=0).stop 2024-05-26T22:50:50.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.058+0000 7f5bcbe00640 1 -- 172.21.15.131:0/1734913164 reap_dead start 2024-05-26T22:50:50.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.058+0000 7f5bd1000640 1 -- 172.21.15.131:0/1734913164 shutdown_connections 2024-05-26T22:50:50.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.058+0000 7f5bd1000640 1 -- 172.21.15.131:0/1734913164 >> 172.21.15.131:0/1734913164 conn(0x7f5bcc073070 msgr2=0x7f5bcc0d0c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:50.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.058+0000 7f5bd1000640 1 -- 172.21.15.131:0/1734913164 shutdown_connections 2024-05-26T22:50:50.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.058+0000 7f5bd1000640 1 -- 172.21.15.131:0/1734913164 wait complete. 2024-05-26T22:50:50.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.060+0000 7f5bd1000640 1 Processor -- start 2024-05-26T22:50:50.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.060+0000 7f5bd1000640 1 -- start start 2024-05-26T22:50:50.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.061+0000 7f5bd1000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5bb0004d40 0x7f5bcc1f6e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:50.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.061+0000 7f5bd1000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5bcc0cc910 0x7f5bcc0b7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:50.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.061+0000 7f5bd1000640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5bcc0ccd90 0x7f5bcc0b8010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:50.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.061+0000 7f5bd1000640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f5bcc15e7c0 con 0x7f5bcc0cc910 2024-05-26T22:50:50.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.061+0000 7f5bd1000640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5bcc0d3720 con 0x7f5bcc0ccd90 2024-05-26T22:50:50.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.061+0000 7f5bd1000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f5bcc0b5c50 con 0x7f5bb0004d40 2024-05-26T22:50:50.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.061+0000 7f5bc3e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5bcc0cc910 0x7f5bcc0b7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:50.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.061+0000 7f5bc3e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5bcc0cc910 0x7f5bcc0b7ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54166/0 (socket says 172.21.15.131:54166) 2024-05-26T22:50:50.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.061+0000 7f5bc3e00640 1 -- 172.21.15.131:0/3620979585 learned_addr learned my addr 172.21.15.131:0/3620979585 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:50.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.061+0000 7f5bcb400640 1 --2- 172.21.15.131:0/3620979585 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5bb0004d40 0x7f5bcc1f6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:50.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.061+0000 7f5bcbe00640 1 --2- 172.21.15.131:0/3620979585 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5bcc0ccd90 0x7f5bcc0b8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:50.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.061+0000 7f5bc3e00640 1 -- 172.21.15.131:0/3620979585 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5bb0004d40 msgr2=0x7f5bcc1f6e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:50.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.061+0000 7f5bc3e00640 1 --2- 172.21.15.131:0/3620979585 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5bb0004d40 0x7f5bcc1f6e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:50.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.061+0000 7f5bc3e00640 1 -- 172.21.15.131:0/3620979585 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5bcc0ccd90 msgr2=0x7f5bcc0b8010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:50.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.061+0000 7f5bc3e00640 1 --2- 172.21.15.131:0/3620979585 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5bcc0ccd90 0x7f5bcc0b8010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:50.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.061+0000 7f5bc3e00640 1 -- 172.21.15.131:0/3620979585 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bcc0cd2f0 con 0x7f5bcc0cc910 2024-05-26T22:50:50.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.062+0000 7f5bc3e00640 1 --2- 172.21.15.131:0/3620979585 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5bcc0cc910 0x7f5bcc0b7ad0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f5bb400d7d0 tx=0x7f5bb400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:50.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.062+0000 7f5bc3400640 1 -- 172.21.15.131:0/3620979585 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb4009640 con 0x7f5bcc0cc910 2024-05-26T22:50:50.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.062+0000 7f5bc3400640 1 -- 172.21.15.131:0/3620979585 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bb4019070 con 0x7f5bcc0cc910 2024-05-26T22:50:50.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.062+0000 7f5bc3400640 1 -- 172.21.15.131:0/3620979585 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb4009a60 con 0x7f5bcc0cc910 2024-05-26T22:50:50.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.062+0000 7f5bd1000640 1 -- 172.21.15.131:0/3620979585 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bcc0cd5d0 con 0x7f5bcc0cc910 2024-05-26T22:50:50.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.062+0000 7f5bd1000640 1 -- 172.21.15.131:0/3620979585 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bcc0cdb90 con 0x7f5bcc0cc910 2024-05-26T22:50:50.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.065+0000 7f5bc3400640 1 -- 172.21.15.131:0/3620979585 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5bb4009c00 con 0x7f5bcc0cc910 2024-05-26T22:50:50.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.066+0000 7f5bc3400640 1 --2- 172.21.15.131:0/3620979585 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f5ba806be60 0x7f5ba806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:50.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.066+0000 7f5bc3400640 1 -- 172.21.15.131:0/3620979585 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f5bb408b1c0 con 0x7f5bcc0cc910 2024-05-26T22:50:50.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.066+0000 7f5bcb400640 1 --2- 172.21.15.131:0/3620979585 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f5ba806be60 0x7f5ba806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:50.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.066+0000 7f5bd53c4740 1 Processor -- start 2024-05-26T22:50:50.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.066+0000 7f5bd53c4740 1 -- start start 2024-05-26T22:50:50.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.067+0000 7f5bcb400640 1 --2- 172.21.15.131:0/3620979585 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f5ba806be60 0x7f5ba806e320 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc0b9030 tx=0x7f5bbc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:50.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.067+0000 7f5bd53c4740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5587d63e1e50 0x5587d6394a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:50.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.067+0000 7f5bd53c4740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5587d63c4f70 0x5587d63c42f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:50.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.067+0000 7f5bd53c4740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5587d63c4830 0x5587d6399640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:50.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.067+0000 7f5bd53c4740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5587d62fc610 con 0x5587d63c4830 2024-05-26T22:50:50.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.067+0000 7f5bd53c4740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5587d62a45c0 con 0x5587d63c4f70 2024-05-26T22:50:50.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.067+0000 7f5bd53c4740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x5587d62f85d0 con 0x5587d63e1e50 2024-05-26T22:50:50.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.068+0000 7f5bcbe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5587d63c4830 0x5587d6399640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:50.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.068+0000 7f5bcbe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5587d63c4830 0x5587d6399640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54170/0 (socket says 172.21.15.131:54170) 2024-05-26T22:50:50.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.068+0000 7f5bcbe00640 1 -- 172.21.15.131:0/4229875315 learned_addr learned my addr 172.21.15.131:0/4229875315 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:50.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.068+0000 7f5bcb400640 1 --2- 172.21.15.131:0/4229875315 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5587d63e1e50 0x5587d6394a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:50.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.068+0000 7f5bc3e00640 1 --2- 172.21.15.131:0/4229875315 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5587d63c4f70 0x5587d63c42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:50.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.068+0000 7f5bcbe00640 1 -- 172.21.15.131:0/4229875315 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5587d63e1e50 msgr2=0x5587d6394a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:50.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.068+0000 7f5bcbe00640 1 --2- 172.21.15.131:0/4229875315 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5587d63e1e50 0x5587d6394a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:50.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.068+0000 7f5bcbe00640 1 -- 172.21.15.131:0/4229875315 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5587d63c4f70 msgr2=0x5587d63c42f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:50.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.068+0000 7f5bcbe00640 1 --2- 172.21.15.131:0/4229875315 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5587d63c4f70 0x5587d63c42f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:50.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.068+0000 7f5bcbe00640 1 -- 172.21.15.131:0/4229875315 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587d60102d0 con 0x5587d63c4830 2024-05-26T22:50:50.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.068+0000 7f5bcbe00640 1 --2- 172.21.15.131:0/4229875315 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5587d63c4830 0x5587d6399640 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f5bc400e200 tx=0x7f5bc402d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb8f3a3d6238120a server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:50.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.069+0000 7f5bbbe00640 1 -- 172.21.15.131:0/4229875315 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bc4003b90 con 0x5587d63c4830 2024-05-26T22:50:50.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.069+0000 7f5bbbe00640 1 -- 172.21.15.131:0/4229875315 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bc4040070 con 0x5587d63c4830 2024-05-26T22:50:50.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.069+0000 7f5bbbe00640 1 -- 172.21.15.131:0/4229875315 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bc4004030 con 0x5587d63c4830 2024-05-26T22:50:50.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.069+0000 7f5bd53c4740 1 -- 172.21.15.131:0/4229875315 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5587d63c4830 msgr2=0x5587d6399640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:50.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.069+0000 7f5bd53c4740 1 --2- 172.21.15.131:0/4229875315 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5587d63c4830 0x5587d6399640 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f5bc400e200 tx=0x7f5bc402d000 comp rx=0 tx=0).stop 2024-05-26T22:50:50.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.069+0000 7f5bd53c4740 1 -- 172.21.15.131:0/4229875315 shutdown_connections 2024-05-26T22:50:50.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.069+0000 7f5bd53c4740 1 --2- 172.21.15.131:0/4229875315 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5587d63c4830 0x5587d6399640 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:50.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.069+0000 7f5bd53c4740 1 --2- 172.21.15.131:0/4229875315 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5587d63c4f70 0x5587d63c42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:50.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.069+0000 7f5bd53c4740 1 --2- 172.21.15.131:0/4229875315 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5587d63e1e50 0x5587d6394a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:50.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.069+0000 7f5bd53c4740 1 -- 172.21.15.131:0/4229875315 >> 172.21.15.131:0/4229875315 conn(0x5587d6396ce0 msgr2=0x5587d63df2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:50.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.070+0000 7f5bd53c4740 1 -- 172.21.15.131:0/4229875315 shutdown_connections 2024-05-26T22:50:50.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.070+0000 7f5bd53c4740 1 -- 172.21.15.131:0/4229875315 wait complete. 2024-05-26T22:50:50.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.070+0000 7f5bd53c4740 1 -- start start 2024-05-26T22:50:50.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.070+0000 7f5bd53c4740 1 Processor -- start 2024-05-26T22:50:50.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.071+0000 7f5bd53c4740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5587d63c4f70 0x5587d63c7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:50.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.071+0000 7f5bd53c4740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5587d63e1e50 0x5587d63e0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:50.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.071+0000 7f5bd53c4740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5587d63c5460 0x5587d63ef1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:50.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.071+0000 7f5bd53c4740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5587d6398c00 con 0x5587d63c4f70 2024-05-26T22:50:50.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.071+0000 7f5bd53c4740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5587d62b7910 con 0x5587d63c5460 2024-05-26T22:50:50.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.071+0000 7f5bd53c4740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x5587d6397ff0 con 0x5587d63e1e50 2024-05-26T22:50:50.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.072+0000 7f5bd53c4740 10 client.0 _ll_register_callbacks cb 0x5587d62f3410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-26T22:50:50.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.072+0000 7f5bcb400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5587d63c4f70 0x5587d63c7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:50.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.072+0000 7f5bcb400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5587d63c4f70 0x5587d63c7ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54178/0 (socket says 172.21.15.131:54178) 2024-05-26T22:50:50.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.072+0000 7f5bcb400640 1 -- 172.21.15.131:0/2556686847 learned_addr learned my addr 172.21.15.131:0/2556686847 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:50.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.072+0000 7f5bc3e00640 1 --2- 172.21.15.131:0/2556686847 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5587d63e1e50 0x5587d63e0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:50.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.072+0000 7f5bcb400640 1 -- 172.21.15.131:0/2556686847 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5587d63e1e50 msgr2=0x5587d63e0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:50.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.072+0000 7f5bcb400640 1 --2- 172.21.15.131:0/2556686847 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5587d63e1e50 0x5587d63e0820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:50.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.072+0000 7f5bcb400640 1 -- 172.21.15.131:0/2556686847 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5587d63c5460 msgr2=0x5587d63ef1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:50.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.072+0000 7f5bcb400640 1 --2- 172.21.15.131:0/2556686847 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5587d63c5460 0x5587d63ef1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:50.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.072+0000 7f5bcb400640 1 -- 172.21.15.131:0/2556686847 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587d60da6b0 con 0x5587d63c4f70 2024-05-26T22:50:50.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.072+0000 7f5bcb400640 1 --2- 172.21.15.131:0/2556686847 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5587d63c4f70 0x5587d63c7ff0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc0072d0 tx=0x7f5bbc0019e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90aa547dfa7b465b server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:50.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.072+0000 7f5bb8c00640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-26T22:50:50.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.073+0000 7f5bb8c00640 1 -- 172.21.15.131:0/2556686847 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bbc013680 con 0x5587d63c4f70 2024-05-26T22:50:50.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.073+0000 7f5bd53c4740 1 -- 172.21.15.131:0/2556686847 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5bc4040070 con 0x5587d63c4f70 2024-05-26T22:50:50.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.073+0000 7f5bb8c00640 1 -- 172.21.15.131:0/2556686847 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f5bbc021a80 con 0x5587d63c4f70 2024-05-26T22:50:50.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.073+0000 7f5bb8c00640 1 -- 172.21.15.131:0/2556686847 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f5bbc021c80 con 0x5587d63c4f70 2024-05-26T22:50:50.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.073+0000 7f5bd53c4740 10 client.5889 fetch_fsmap learned FSMap version 33 2024-05-26T22:50:50.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.073+0000 7f5bd53c4740 1 -- 172.21.15.131:0/2556686847 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) -- 0x5587d63efe90 con 0x5587d63c4f70 2024-05-26T22:50:50.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.073+0000 7f5bb8c00640 1 -- 172.21.15.131:0/2556686847 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bc4040070 con 0x5587d63c4f70 2024-05-26T22:50:50.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.073+0000 7f5bb8c00640 1 -- 172.21.15.131:0/2556686847 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 33) ==== 3534+0+0 (secure 0 0 0) 0x7f5bbc021420 con 0x5587d63c4f70 2024-05-26T22:50:50.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.073+0000 7f5bd53c4740 10 client.5889 fetch_fsmap finished waiting for FSMap version 33 2024-05-26T22:50:50.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.073+0000 7f5bd53c4740 10 client.5889 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-26T22:50:50.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.073+0000 7f5bd53c4740 20 client.5889 populate_metadata read hostname 'smithi131' 2024-05-26T22:50:50.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.073+0000 7f5bd53c4740 1 --2- 172.21.15.131:0/2556686847 >> [v2:172.21.15.187:6832/1027783169,v1:172.21.15.187:6834/1027783169] conn(0x5587d63dd250 0x5587d63f2e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:50.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.073+0000 7f5bd53c4740 4 client.5889 mds_command: new command op to 4670 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-26T22:50:50.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.073+0000 7f5bd53c4740 1 -- 172.21.15.131:0/2556686847 --> [v2:172.21.15.187:6832/1027783169,v1:172.21.15.187:6834/1027783169] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5587d6309d40 con 0x5587d63dd250 2024-05-26T22:50:50.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.074+0000 7f5bc3e00640 1 --2- 172.21.15.131:0/2556686847 >> [v2:172.21.15.187:6832/1027783169,v1:172.21.15.187:6834/1027783169] conn(0x5587d63dd250 0x5587d63f2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:50.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.075+0000 7f5bc3e00640 1 --2- 172.21.15.131:0/2556686847 >> [v2:172.21.15.187:6832/1027783169,v1:172.21.15.187:6834/1027783169] conn(0x5587d63dd250 0x5587d63f2e20 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fac91086e197af67 server_cookie=6c5bee355d79f35c in_seq=0 out_seq=0 2024-05-26T22:50:50.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.075+0000 7f5bb8c00640 10 client.5889 ms_handle_connect on v2:172.21.15.187:6832/1027783169 2024-05-26T22:50:50.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.077+0000 7f5bb8c00640 1 -- 172.21.15.131:0/2556686847 <== mds.2 v2:172.21.15.187:6832/1027783169 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5587d6309d40 con 0x5587d63dd250 2024-05-26T22:50:50.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.077+0000 7f5bb8c00640 10 client.5889 handle_command_reply: tid=0 2024-05-26T22:50:50.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.077+0000 7f5bd53c4740 1 client.5889 shutdown 2024-05-26T22:50:50.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.077+0000 7f5bd53c4740 1 -- 172.21.15.131:0/2556686847 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5587d63c4f70 msgr2=0x5587d63c7ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:50.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.077+0000 7f5bd53c4740 1 --2- 172.21.15.131:0/2556686847 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5587d63c4f70 0x5587d63c7ff0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc0072d0 tx=0x7f5bbc0019e0 comp rx=0 tx=0).stop 2024-05-26T22:50:50.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.077+0000 7f5bd53c4740 1 -- 172.21.15.131:0/2556686847 shutdown_connections 2024-05-26T22:50:50.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.077+0000 7f5bd53c4740 1 --2- 172.21.15.131:0/2556686847 >> [v2:172.21.15.187:6832/1027783169,v1:172.21.15.187:6834/1027783169] conn(0x5587d63dd250 0x5587d63f2e20 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:50.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.077+0000 7f5bd53c4740 1 --2- 172.21.15.131:0/2556686847 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5587d63e1e50 0x5587d63e0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:50.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.077+0000 7f5bd53c4740 1 --2- 172.21.15.131:0/2556686847 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5587d63c5460 0x5587d63ef1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:50.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.077+0000 7f5bd53c4740 1 --2- 172.21.15.131:0/2556686847 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5587d63c4f70 0x5587d63c7ff0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:50.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.077+0000 7f5bd53c4740 1 -- 172.21.15.131:0/2556686847 >> 172.21.15.131:0/2556686847 conn(0x5587d63c4830 msgr2=0x5587d63df2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:50.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.077+0000 7f5bb8c00640 0 client.5889 ms_handle_reset on v2:172.21.15.187:6832/1027783169 2024-05-26T22:50:50.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.077+0000 7f5bd53c4740 1 -- 172.21.15.131:0/2556686847 shutdown_connections 2024-05-26T22:50:50.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.077+0000 7f5bd53c4740 1 -- 172.21.15.131:0/2556686847 wait complete. 2024-05-26T22:50:50.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.078+0000 7f5bd53c4740 20 client.5889 trim_cache size 0 max 16384 2024-05-26T22:50:50.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.097+0000 7f5bd53c4740 1 Processor -- start 2024-05-26T22:50:50.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.097+0000 7f5bd53c4740 1 -- start start 2024-05-26T22:50:50.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.098+0000 7f5bd53c4740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5587d63c4f70 0x5587d63f8e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:50.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.098+0000 7f5bd53c4740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5587d63c5460 0x5587d63f9390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:50.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.098+0000 7f5bd53c4740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5587d63e1e50 0x5587d63bf370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:50.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.098+0000 7f5bd53c4740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f5bcc1fff80 con 0x5587d63c5460 2024-05-26T22:50:50.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.098+0000 7f5bd53c4740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5bcc200650 con 0x5587d63c4f70 2024-05-26T22:50:50.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.098+0000 7f5bd53c4740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f5bcc1f4d40 con 0x5587d63e1e50 2024-05-26T22:50:50.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.099+0000 7f5bc3e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5587d63c5460 0x5587d63f9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:50.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.099+0000 7f5bc3e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5587d63c5460 0x5587d63f9390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54182/0 (socket says 172.21.15.131:54182) 2024-05-26T22:50:50.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.099+0000 7f5bc3e00640 1 -- 172.21.15.131:0/1449226567 learned_addr learned my addr 172.21.15.131:0/1449226567 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:50.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.099+0000 7f5bc3e00640 1 -- 172.21.15.131:0/1449226567 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5587d63e1e50 msgr2=0x5587d63bf370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:50.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.099+0000 7f5bc3e00640 1 --2- 172.21.15.131:0/1449226567 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5587d63e1e50 0x5587d63bf370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:50.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.099+0000 7f5bc3e00640 1 -- 172.21.15.131:0/1449226567 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5587d63c4f70 msgr2=0x5587d63f8e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:50.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.099+0000 7f5bc3e00640 1 --2- 172.21.15.131:0/1449226567 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5587d63c4f70 0x5587d63f8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:50.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.099+0000 7f5bc3e00640 1 -- 172.21.15.131:0/1449226567 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587d6397b90 con 0x5587d63c5460 2024-05-26T22:50:50.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.099+0000 7f5bc3e00640 1 --2- 172.21.15.131:0/1449226567 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5587d63c5460 0x5587d63f9390 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f5bb4055880 tx=0x7f5bb408b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d60fc731f397a45 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:50.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.099+0000 7f5bbb400640 1 -- 172.21.15.131:0/1449226567 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb405d820 con 0x5587d63c5460 2024-05-26T22:50:50.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.099+0000 7f5bbb400640 1 -- 172.21.15.131:0/1449226567 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bb4062050 con 0x5587d63c5460 2024-05-26T22:50:50.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.099+0000 7f5bbb400640 1 -- 172.21.15.131:0/1449226567 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb4057770 con 0x5587d63c5460 2024-05-26T22:50:50.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.100+0000 7f5bd53c4740 1 -- 172.21.15.131:0/1449226567 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5587d63c5460 msgr2=0x5587d63f9390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:50.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.100+0000 7f5bd53c4740 1 --2- 172.21.15.131:0/1449226567 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5587d63c5460 0x5587d63f9390 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f5bb4055880 tx=0x7f5bb408b040 comp rx=0 tx=0).stop 2024-05-26T22:50:50.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.100+0000 7f5bd53c4740 1 -- 172.21.15.131:0/1449226567 shutdown_connections 2024-05-26T22:50:50.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.100+0000 7f5bd53c4740 1 --2- 172.21.15.131:0/1449226567 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5587d63e1e50 0x5587d63bf370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:50.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.100+0000 7f5bd53c4740 1 --2- 172.21.15.131:0/1449226567 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5587d63c5460 0x5587d63f9390 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:50.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.100+0000 7f5bd53c4740 1 --2- 172.21.15.131:0/1449226567 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5587d63c4f70 0x5587d63f8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:50.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.100+0000 7f5bd53c4740 1 -- 172.21.15.131:0/1449226567 >> 172.21.15.131:0/1449226567 conn(0x5587d63c4830 msgr2=0x5587d63e0f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:50.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.100+0000 7f5bd53c4740 1 -- 172.21.15.131:0/1449226567 shutdown_connections 2024-05-26T22:50:50.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.100+0000 7f5bd53c4740 1 -- 172.21.15.131:0/1449226567 wait complete. 2024-05-26T22:50:50.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.101+0000 7f5bd53c4740 1 -- start start 2024-05-26T22:50:50.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.101+0000 7f5bd53c4740 1 Processor -- start 2024-05-26T22:50:50.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.102+0000 7f5bd53c4740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5587d63c4f70 0x5587d645c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:50.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.102+0000 7f5bd53c4740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5587d63c5460 0x5587d63bf250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:50.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.102+0000 7f5bd53c4740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5587d63e1e50 0x5587d63bf790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:50.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.102+0000 7f5bd53c4740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5587d63dd0d0 con 0x5587d63c5460 2024-05-26T22:50:50.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.102+0000 7f5bd53c4740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5587d63efbd0 con 0x5587d63e1e50 2024-05-26T22:50:50.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.102+0000 7f5bd53c4740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x5587d63dd680 con 0x5587d63c4f70 2024-05-26T22:50:50.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.102+0000 7f5bd53c4740 10 client.0 _ll_register_callbacks cb 0x7f5bbc007e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-26T22:50:50.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.102+0000 7f5bc3e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5587d63c5460 0x5587d63bf250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:50.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.102+0000 7f5bc3e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5587d63c5460 0x5587d63bf250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54184/0 (socket says 172.21.15.131:54184) 2024-05-26T22:50:50.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.102+0000 7f5bc3e00640 1 -- 172.21.15.131:0/1626841445 learned_addr learned my addr 172.21.15.131:0/1626841445 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:50.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.102+0000 7f5bcb400640 1 --2- 172.21.15.131:0/1626841445 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5587d63c4f70 0x5587d645c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:50.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.103+0000 7f5bcbe00640 1 --2- 172.21.15.131:0/1626841445 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5587d63e1e50 0x5587d63bf790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:50.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.103+0000 7f5bc3e00640 1 -- 172.21.15.131:0/1626841445 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5587d63c4f70 msgr2=0x5587d645c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:50.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.103+0000 7f5bc3e00640 1 --2- 172.21.15.131:0/1626841445 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5587d63c4f70 0x5587d645c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:50.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.103+0000 7f5bc3e00640 1 -- 172.21.15.131:0/1626841445 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5587d63e1e50 msgr2=0x5587d63bf790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:50.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.103+0000 7f5bc3e00640 1 --2- 172.21.15.131:0/1626841445 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5587d63e1e50 0x5587d63bf790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:50.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.103+0000 7f5bc3e00640 1 -- 172.21.15.131:0/1626841445 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587d63df0f0 con 0x5587d63c5460 2024-05-26T22:50:50.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.103+0000 7f5bcb400640 1 --2- 172.21.15.131:0/1626841445 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5587d63c4f70 0x5587d645c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:50:50.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.103+0000 7f5bc3e00640 1 --2- 172.21.15.131:0/1626841445 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5587d63c5460 0x5587d63bf250 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f5bb4059170 tx=0x7f5bbc00c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c9a5e8142fbf4ed server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:50.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.103+0000 7f5bb8c00640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-26T22:50:50.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.103+0000 7f5bb8c00640 1 -- 172.21.15.131:0/1626841445 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb4076060 con 0x5587d63c5460 2024-05-26T22:50:50.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.103+0000 7f5bb8c00640 1 -- 172.21.15.131:0/1626841445 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f5bb4055ae0 con 0x5587d63c5460 2024-05-26T22:50:50.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.103+0000 7f5bd53c4740 1 -- 172.21.15.131:0/1626841445 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5bb4062050 con 0x5587d63c5460 2024-05-26T22:50:50.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.103+0000 7f5bb8c00640 1 -- 172.21.15.131:0/1626841445 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f5bb405d820 con 0x5587d63c5460 2024-05-26T22:50:50.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.103+0000 7f5bd53c4740 10 client.5895 fetch_fsmap learned FSMap version 33 2024-05-26T22:50:50.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.103+0000 7f5bd53c4740 1 -- 172.21.15.131:0/1626841445 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) -- 0x5587d6402f90 con 0x5587d63c5460 2024-05-26T22:50:50.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.104+0000 7f5bb8c00640 1 -- 172.21.15.131:0/1626841445 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bb4062050 con 0x5587d63c5460 2024-05-26T22:50:50.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.104+0000 7f5bb8c00640 1 -- 172.21.15.131:0/1626841445 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 33) ==== 3534+0+0 (secure 0 0 0) 0x7f5bb4013070 con 0x5587d63c5460 2024-05-26T22:50:50.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.104+0000 7f5bd53c4740 10 client.5895 fetch_fsmap finished waiting for FSMap version 33 2024-05-26T22:50:50.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.104+0000 7f5bd53c4740 10 client.5895 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-26T22:50:50.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.104+0000 7f5bd53c4740 20 client.5895 populate_metadata read hostname 'smithi131' 2024-05-26T22:50:50.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.104+0000 7f5bd53c4740 1 --2- 172.21.15.131:0/1626841445 >> [v2:172.21.15.187:6832/1027783169,v1:172.21.15.187:6834/1027783169] conn(0x5587d63ee920 0x5587d63e2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:50.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.104+0000 7f5bd53c4740 4 client.5895 mds_command: new command op to 4670 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-26T22:50:50.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.104+0000 7f5bd53c4740 1 -- 172.21.15.131:0/1626841445 --> [v2:172.21.15.187:6832/1027783169,v1:172.21.15.187:6834/1027783169] -- command(tid 0: {"prefix": "status"}) -- 0x5587d6304090 con 0x5587d63ee920 2024-05-26T22:50:50.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.104+0000 7f5bcb400640 1 --2- 172.21.15.131:0/1626841445 >> [v2:172.21.15.187:6832/1027783169,v1:172.21.15.187:6834/1027783169] conn(0x5587d63ee920 0x5587d63e2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:50.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.105+0000 7f5bcb400640 1 --2- 172.21.15.131:0/1626841445 >> [v2:172.21.15.187:6832/1027783169,v1:172.21.15.187:6834/1027783169] conn(0x5587d63ee920 0x5587d63e2710 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=770a7a02c5789038 server_cookie=39e2fde2e3894b4b in_seq=0 out_seq=0 2024-05-26T22:50:50.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.105+0000 7f5bb8c00640 10 client.5895 ms_handle_connect on v2:172.21.15.187:6832/1027783169 2024-05-26T22:50:50.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.106+0000 7f5bb8c00640 1 -- 172.21.15.131:0/1626841445 <== mds.2 v2:172.21.15.187:6832/1027783169 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x5587d6304090 con 0x5587d63ee920 2024-05-26T22:50:50.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.106+0000 7f5bb8c00640 10 client.5895 handle_command_reply: tid=0 2024-05-26T22:50:50.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.106+0000 7f5bd53c4740 1 client.5895 shutdown 2024-05-26T22:50:50.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.106+0000 7f5bd53c4740 1 -- 172.21.15.131:0/1626841445 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5587d63c5460 msgr2=0x5587d63bf250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:50.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.106+0000 7f5bd53c4740 1 --2- 172.21.15.131:0/1626841445 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5587d63c5460 0x5587d63bf250 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f5bb4059170 tx=0x7f5bbc00c000 comp rx=0 tx=0).stop 2024-05-26T22:50:50.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.106+0000 7f5bd53c4740 1 -- 172.21.15.131:0/1626841445 shutdown_connections 2024-05-26T22:50:50.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.106+0000 7f5bd53c4740 1 --2- 172.21.15.131:0/1626841445 >> [v2:172.21.15.187:6832/1027783169,v1:172.21.15.187:6834/1027783169] conn(0x5587d63ee920 0x5587d63e2710 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:50.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.106+0000 7f5bd53c4740 1 --2- 172.21.15.131:0/1626841445 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5587d63e1e50 0x5587d63bf790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:50.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.106+0000 7f5bd53c4740 1 --2- 172.21.15.131:0/1626841445 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5587d63c5460 0x5587d63bf250 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:50.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.106+0000 7f5bd53c4740 1 --2- 172.21.15.131:0/1626841445 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5587d63c4f70 0x5587d645c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:50.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.106+0000 7f5bb8c00640 0 client.5895 ms_handle_reset on v2:172.21.15.187:6832/1027783169 2024-05-26T22:50:50.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.106+0000 7f5bd53c4740 1 -- 172.21.15.131:0/1626841445 >> 172.21.15.131:0/1626841445 conn(0x5587d63c4830 msgr2=0x5587d63f8e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:50.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.106+0000 7f5bd53c4740 1 -- 172.21.15.131:0/1626841445 shutdown_connections 2024-05-26T22:50:50.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.106+0000 7f5bd53c4740 1 -- 172.21.15.131:0/1626841445 wait complete. 2024-05-26T22:50:50.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.106+0000 7f5bd53c4740 20 client.5895 trim_cache size 0 max 16384 2024-05-26T22:50:50.109 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-26T22:50:50.109 INFO:teuthology.orchestra.run.smithi131.stdout: "cluster_fsid": "46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f", 2024-05-26T22:50:50.109 INFO:teuthology.orchestra.run.smithi131.stdout: "whoami": 2, 2024-05-26T22:50:50.109 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 4670, 2024-05-26T22:50:50.109 INFO:teuthology.orchestra.run.smithi131.stdout: "want_state": "up:active", 2024-05-26T22:50:50.109 INFO:teuthology.orchestra.run.smithi131.stdout: "state": "up:active", 2024-05-26T22:50:50.109 INFO:teuthology.orchestra.run.smithi131.stdout: "fs_name": "cephfs", 2024-05-26T22:50:50.110 INFO:teuthology.orchestra.run.smithi131.stdout: "rank_uptime": 1.991997091, 2024-05-26T22:50:50.110 INFO:teuthology.orchestra.run.smithi131.stdout: "mdsmap_epoch": 33, 2024-05-26T22:50:50.110 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch": 44, 2024-05-26T22:50:50.110 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch_barrier": 44, 2024-05-26T22:50:50.110 INFO:teuthology.orchestra.run.smithi131.stdout: "uptime": 27.997715230000001 2024-05-26T22:50:50.110 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-26T22:50:50.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.107+0000 7f5bd1000640 1 -- 172.21.15.131:0/3620979585 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f5ba806be60 msgr2=0x7f5ba806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:50.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.107+0000 7f5bd1000640 1 --2- 172.21.15.131:0/3620979585 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f5ba806be60 0x7f5ba806e320 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc0b9030 tx=0x7f5bbc009040 comp rx=0 tx=0).stop 2024-05-26T22:50:50.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.107+0000 7f5bd1000640 1 -- 172.21.15.131:0/3620979585 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5bcc0cc910 msgr2=0x7f5bcc0b7ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:50.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.107+0000 7f5bd1000640 1 --2- 172.21.15.131:0/3620979585 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5bcc0cc910 0x7f5bcc0b7ad0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f5bb400d7d0 tx=0x7f5bb400dc90 comp rx=0 tx=0).stop 2024-05-26T22:50:50.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.107+0000 7f5bd1000640 1 -- 172.21.15.131:0/3620979585 shutdown_connections 2024-05-26T22:50:50.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.107+0000 7f5bd1000640 1 --2- 172.21.15.131:0/3620979585 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f5ba806be60 0x7f5ba806e320 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:50.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.107+0000 7f5bd1000640 1 --2- 172.21.15.131:0/3620979585 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5bcc0ccd90 0x7f5bcc0b8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:50.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.107+0000 7f5bd1000640 1 --2- 172.21.15.131:0/3620979585 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5bcc0cc910 0x7f5bcc0b7ad0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:50.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.107+0000 7f5bd1000640 1 --2- 172.21.15.131:0/3620979585 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5bb0004d40 0x7f5bcc1f6e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:50.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.107+0000 7f5bd1000640 1 -- 172.21.15.131:0/3620979585 >> 172.21.15.131:0/3620979585 conn(0x7f5bcc073070 msgr2=0x7f5bcc0d1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:50.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.107+0000 7f5bd1000640 1 -- 172.21.15.131:0/3620979585 shutdown_connections 2024-05-26T22:50:50.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.107+0000 7f5bd1000640 1 -- 172.21.15.131:0/3620979585 wait complete. 2024-05-26T22:50:50.122 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-05-26T22:50:50.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.297+0000 7f78eaa00640 1 Processor -- start 2024-05-26T22:50:50.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.297+0000 7f78eaa00640 1 -- start start 2024-05-26T22:50:50.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.299+0000 7f78eaa00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f78e415e320 0x7f78e40ae250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:50.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.299+0000 7f78eaa00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f78e40b2f60 0x7f78e40cfe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:50.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.299+0000 7f78eaa00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f78e40d0380 0x7f78e406d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:50.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.299+0000 7f78eaa00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f78e4161fe0 con 0x7f78e40d0380 2024-05-26T22:50:50.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.299+0000 7f78eaa00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f78e40b2440 con 0x7f78e415e320 2024-05-26T22:50:50.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.299+0000 7f78eaa00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f78e4162230 con 0x7f78e40b2f60 2024-05-26T22:50:50.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.299+0000 7f78ea000640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f78e40d0380 0x7f78e406d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:50.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.299+0000 7f78ea000640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f78e40d0380 0x7f78e406d250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54190/0 (socket says 172.21.15.131:54190) 2024-05-26T22:50:50.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.299+0000 7f78ea000640 1 -- 172.21.15.131:0/1878002716 learned_addr learned my addr 172.21.15.131:0/1878002716 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:50.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.300+0000 7f78e9600640 1 --2- 172.21.15.131:0/1878002716 >> v2:172.21.15.187:3300/0 conn(0x7f78e415e320 0x7f78e40ae250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:50.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.300+0000 7f78ea000640 1 -- 172.21.15.131:0/1878002716 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f78e40b2f60 msgr2=0x7f78e40cfe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:50.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.300+0000 7f78e8c00640 1 --2- 172.21.15.131:0/1878002716 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f78e40b2f60 0x7f78e40cfe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:50.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.300+0000 7f78ea000640 1 --2- 172.21.15.131:0/1878002716 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f78e40b2f60 0x7f78e40cfe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:50.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.300+0000 7f78ea000640 1 -- 172.21.15.131:0/1878002716 >> v2:172.21.15.187:3300/0 conn(0x7f78e415e320 msgr2=0x7f78e40ae250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:50.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.300+0000 7f78ea000640 1 --2- 172.21.15.131:0/1878002716 >> v2:172.21.15.187:3300/0 conn(0x7f78e415e320 0x7f78e40ae250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:50.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.300+0000 7f78ea000640 1 -- 172.21.15.131:0/1878002716 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78e4162510 con 0x7f78e40d0380 2024-05-26T22:50:50.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.300+0000 7f78e9600640 1 --2- 172.21.15.131:0/1878002716 >> v2:172.21.15.187:3300/0 conn(0x7f78e415e320 0x7f78e40ae250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:50:50.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.301+0000 7f78ea000640 1 --2- 172.21.15.131:0/1878002716 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f78e40d0380 0x7f78e406d250 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f78e40b2a10 tx=0x7f78dc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43083f4518c3e44d server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:50.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.301+0000 7f78e3e00640 1 -- 172.21.15.131:0/1878002716 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78dc037070 con 0x7f78e40d0380 2024-05-26T22:50:50.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.301+0000 7f78e3e00640 1 -- 172.21.15.131:0/1878002716 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78dc032440 con 0x7f78e40d0380 2024-05-26T22:50:50.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.301+0000 7f78e3e00640 1 -- 172.21.15.131:0/1878002716 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78dc032810 con 0x7f78e40d0380 2024-05-26T22:50:50.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.302+0000 7f78eaa00640 1 -- 172.21.15.131:0/1878002716 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f78e40d0380 msgr2=0x7f78e406d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:50.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.302+0000 7f78eaa00640 1 --2- 172.21.15.131:0/1878002716 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f78e40d0380 0x7f78e406d250 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f78e40b2a10 tx=0x7f78dc02a270 comp rx=0 tx=0).stop 2024-05-26T22:50:50.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.302+0000 7f78eaa00640 1 -- 172.21.15.131:0/1878002716 shutdown_connections 2024-05-26T22:50:50.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.302+0000 7f78eaa00640 1 --2- 172.21.15.131:0/1878002716 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f78e40d0380 0x7f78e406d250 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:50.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.302+0000 7f78eaa00640 1 --2- 172.21.15.131:0/1878002716 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f78e40b2f60 0x7f78e40cfe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:50.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.302+0000 7f78eaa00640 1 --2- 172.21.15.131:0/1878002716 >> v2:172.21.15.187:3300/0 conn(0x7f78e415e320 0x7f78e40ae250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:50.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.302+0000 7f78eaa00640 1 -- 172.21.15.131:0/1878002716 >> 172.21.15.131:0/1878002716 conn(0x7f78e40b5660 msgr2=0x7f78e40b5a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:50.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.302+0000 7f78eaa00640 1 -- 172.21.15.131:0/1878002716 shutdown_connections 2024-05-26T22:50:50.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.302+0000 7f78eaa00640 1 -- 172.21.15.131:0/1878002716 wait complete. 2024-05-26T22:50:50.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.304+0000 7f78eaa00640 1 Processor -- start 2024-05-26T22:50:50.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.304+0000 7f78eaa00640 1 -- start start 2024-05-26T22:50:50.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.305+0000 7f78eaa00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f78e40b2f60 0x7f78e414e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:50.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.306+0000 7f78eaa00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f78e40d0380 0x7f78e414ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:50.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.306+0000 7f78eaa00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f78e415e320 0x7f78e41f7800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:50.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.306+0000 7f78eaa00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f78e4162e40 con 0x7f78e40b2f60 2024-05-26T22:50:50.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.306+0000 7f78eaa00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f78e4162ad0 con 0x7f78e415e320 2024-05-26T22:50:50.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.306+0000 7f78eaa00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f78e40b3570 con 0x7f78e40d0380 2024-05-26T22:50:50.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.306+0000 7f78e9600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f78e40b2f60 0x7f78e414e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:50.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.306+0000 7f78e9600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f78e40b2f60 0x7f78e414e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54200/0 (socket says 172.21.15.131:54200) 2024-05-26T22:50:50.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.306+0000 7f78e9600640 1 -- 172.21.15.131:0/84850340 learned_addr learned my addr 172.21.15.131:0/84850340 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:50.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.306+0000 7f78e8c00640 1 --2- 172.21.15.131:0/84850340 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f78e40d0380 0x7f78e414ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:50.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.306+0000 7f78e9600640 1 -- 172.21.15.131:0/84850340 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f78e40d0380 msgr2=0x7f78e414ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:50.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.306+0000 7f78e9600640 1 --2- 172.21.15.131:0/84850340 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f78e40d0380 0x7f78e414ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:50.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.306+0000 7f78e9600640 1 -- 172.21.15.131:0/84850340 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f78e415e320 msgr2=0x7f78e41f7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:50.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.306+0000 7f78e9600640 1 --2- 172.21.15.131:0/84850340 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f78e415e320 0x7f78e41f7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:50.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.306+0000 7f78e9600640 1 -- 172.21.15.131:0/84850340 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78e41535c0 con 0x7f78e40b2f60 2024-05-26T22:50:50.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.306+0000 7f78e8c00640 1 --2- 172.21.15.131:0/84850340 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f78e40d0380 0x7f78e414ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:50:50.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.307+0000 7f78e9600640 1 --2- 172.21.15.131:0/84850340 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f78e40b2f60 0x7f78e414e750 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f78d400ce00 tx=0x7f78d400a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:50.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.307+0000 7f78e1600640 1 -- 172.21.15.131:0/84850340 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d400f890 con 0x7f78e40b2f60 2024-05-26T22:50:50.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.307+0000 7f78e1600640 1 -- 172.21.15.131:0/84850340 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78d400fa30 con 0x7f78e40b2f60 2024-05-26T22:50:50.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.307+0000 7f78e1600640 1 -- 172.21.15.131:0/84850340 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d400fe40 con 0x7f78e40b2f60 2024-05-26T22:50:50.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.307+0000 7f78eaa00640 1 -- 172.21.15.131:0/84850340 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78e4156200 con 0x7f78e40b2f60 2024-05-26T22:50:50.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.307+0000 7f78eaa00640 1 -- 172.21.15.131:0/84850340 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78e41532e0 con 0x7f78e40b2f60 2024-05-26T22:50:50.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.309+0000 7f78e1600640 1 -- 172.21.15.131:0/84850340 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f78d4018400 con 0x7f78e40b2f60 2024-05-26T22:50:50.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.310+0000 7f78e1600640 1 --2- 172.21.15.131:0/84850340 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f78c406be30 0x7f78c406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:50.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.310+0000 7f78e1600640 1 -- 172.21.15.131:0/84850340 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f78d408bcc0 con 0x7f78e40b2f60 2024-05-26T22:50:50.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.310+0000 7f78e8c00640 1 --2- 172.21.15.131:0/84850340 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f78c406be30 0x7f78c406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:50.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.311+0000 7f78e8c00640 1 --2- 172.21.15.131:0/84850340 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f78c406be30 0x7f78c406e2f0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f78d0006fd0 tx=0x7f78d0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:50.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.311+0000 7f78eefd0740 1 Processor -- start 2024-05-26T22:50:50.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.311+0000 7f78eefd0740 1 -- start start 2024-05-26T22:50:50.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.312+0000 7f78eefd0740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5617f340ee50 0x5617f33c1a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:50.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.312+0000 7f78eefd0740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5617f33f1f70 0x5617f33f12f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:50.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.312+0000 7f78eefd0740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5617f33f1830 0x5617f33c6640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:50.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.312+0000 7f78eefd0740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5617f3329610 con 0x5617f33f1f70 2024-05-26T22:50:50.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.312+0000 7f78eefd0740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5617f32d15c0 con 0x5617f33f1830 2024-05-26T22:50:50.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.312+0000 7f78eefd0740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x5617f33255d0 con 0x5617f340ee50 2024-05-26T22:50:50.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.312+0000 7f78e8c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5617f33f1f70 0x5617f33f12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:50.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.312+0000 7f78e8c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5617f33f1f70 0x5617f33f12f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54206/0 (socket says 172.21.15.131:54206) 2024-05-26T22:50:50.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.312+0000 7f78e8c00640 1 -- 172.21.15.131:0/1515015099 learned_addr learned my addr 172.21.15.131:0/1515015099 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:50.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.313+0000 7f78e9600640 1 --2- 172.21.15.131:0/1515015099 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5617f340ee50 0x5617f33c1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:50.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.313+0000 7f78ea000640 1 --2- 172.21.15.131:0/1515015099 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5617f33f1830 0x5617f33c6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:50.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.313+0000 7f78e8c00640 1 -- 172.21.15.131:0/1515015099 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5617f340ee50 msgr2=0x5617f33c1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:50.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.313+0000 7f78e8c00640 1 --2- 172.21.15.131:0/1515015099 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5617f340ee50 0x5617f33c1a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:50.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.313+0000 7f78e8c00640 1 -- 172.21.15.131:0/1515015099 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5617f33f1830 msgr2=0x5617f33c6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:50.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.313+0000 7f78e8c00640 1 --2- 172.21.15.131:0/1515015099 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5617f33f1830 0x5617f33c6640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:50.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.313+0000 7f78e8c00640 1 -- 172.21.15.131:0/1515015099 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617f303d2d0 con 0x5617f33f1f70 2024-05-26T22:50:50.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.313+0000 7f78e9600640 1 --2- 172.21.15.131:0/1515015099 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5617f340ee50 0x5617f33c1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:50:50.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.313+0000 7f78ea000640 1 --2- 172.21.15.131:0/1515015099 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5617f33f1830 0x5617f33c6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:50:50.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.313+0000 7f78e8c00640 1 --2- 172.21.15.131:0/1515015099 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5617f33f1f70 0x5617f33f12f0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f78d000da00 tx=0x7f78d0010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7269f1b44950f948 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:50.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.313+0000 7f78d9c00640 1 -- 172.21.15.131:0/1515015099 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d001a040 con 0x5617f33f1f70 2024-05-26T22:50:50.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.313+0000 7f78d9c00640 1 -- 172.21.15.131:0/1515015099 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78d0023070 con 0x5617f33f1f70 2024-05-26T22:50:50.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.313+0000 7f78d9c00640 1 -- 172.21.15.131:0/1515015099 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d00149d0 con 0x5617f33f1f70 2024-05-26T22:50:50.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.314+0000 7f78eefd0740 1 -- 172.21.15.131:0/1515015099 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5617f33f1f70 msgr2=0x5617f33f12f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:50.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.314+0000 7f78eefd0740 1 --2- 172.21.15.131:0/1515015099 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5617f33f1f70 0x5617f33f12f0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f78d000da00 tx=0x7f78d0010000 comp rx=0 tx=0).stop 2024-05-26T22:50:50.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.314+0000 7f78eefd0740 1 -- 172.21.15.131:0/1515015099 shutdown_connections 2024-05-26T22:50:50.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.314+0000 7f78eefd0740 1 --2- 172.21.15.131:0/1515015099 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5617f33f1830 0x5617f33c6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:50.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.314+0000 7f78eefd0740 1 --2- 172.21.15.131:0/1515015099 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5617f33f1f70 0x5617f33f12f0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:50.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.314+0000 7f78eefd0740 1 --2- 172.21.15.131:0/1515015099 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5617f340ee50 0x5617f33c1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:50.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.314+0000 7f78eefd0740 1 -- 172.21.15.131:0/1515015099 >> 172.21.15.131:0/1515015099 conn(0x5617f33c3ce0 msgr2=0x5617f340c2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:50.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.314+0000 7f78eefd0740 1 -- 172.21.15.131:0/1515015099 shutdown_connections 2024-05-26T22:50:50.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.314+0000 7f78eefd0740 1 -- 172.21.15.131:0/1515015099 wait complete. 2024-05-26T22:50:50.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.315+0000 7f78eefd0740 1 -- start start 2024-05-26T22:50:50.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.315+0000 7f78eefd0740 1 Processor -- start 2024-05-26T22:50:50.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.316+0000 7f78eefd0740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5617f33f1f70 0x5617f33f4b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:50.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.316+0000 7f78eefd0740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5617f340ee50 0x5617f33db9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:50.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.316+0000 7f78eefd0740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5617f3403020 0x5617f33de1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:50.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.316+0000 7f78eefd0740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5617f33c5c00 con 0x5617f3403020 2024-05-26T22:50:50.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.316+0000 7f78eefd0740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5617f32e4910 con 0x5617f33f1f70 2024-05-26T22:50:50.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.316+0000 7f78eefd0740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x5617f33c4ff0 con 0x5617f340ee50 2024-05-26T22:50:50.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.316+0000 7f78eefd0740 10 client.0 _ll_register_callbacks cb 0x5617f3320410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-26T22:50:50.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.316+0000 7f78ea000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5617f3403020 0x5617f33de1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:50.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.316+0000 7f78ea000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5617f3403020 0x5617f33de1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54214/0 (socket says 172.21.15.131:54214) 2024-05-26T22:50:50.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.316+0000 7f78ea000640 1 -- 172.21.15.131:0/2597998364 learned_addr learned my addr 172.21.15.131:0/2597998364 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:50.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.317+0000 7f78ea000640 1 -- 172.21.15.131:0/2597998364 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5617f340ee50 msgr2=0x5617f33db9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:50.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.317+0000 7f78ea000640 1 --2- 172.21.15.131:0/2597998364 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5617f340ee50 0x5617f33db9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:50.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.317+0000 7f78ea000640 1 -- 172.21.15.131:0/2597998364 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5617f33f1f70 msgr2=0x5617f33f4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:50.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.317+0000 7f78ea000640 1 --2- 172.21.15.131:0/2597998364 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5617f33f1f70 0x5617f33f4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:50.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.317+0000 7f78ea000640 1 -- 172.21.15.131:0/2597998364 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617f31076b0 con 0x5617f3403020 2024-05-26T22:50:50.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.317+0000 7f78ea000640 1 --2- 172.21.15.131:0/2597998364 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5617f3403020 0x5617f33de1f0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f78dc009960 tx=0x7f78dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32ab4a43cc84faee server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:50.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.317+0000 7f78ce000640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-26T22:50:50.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.317+0000 7f78ce000640 1 -- 172.21.15.131:0/2597998364 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78dc042070 con 0x5617f3403020 2024-05-26T22:50:50.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.317+0000 7f78ce000640 1 -- 172.21.15.131:0/2597998364 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f78dc032e00 con 0x5617f3403020 2024-05-26T22:50:50.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.317+0000 7f78eefd0740 1 -- 172.21.15.131:0/2597998364 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78d0023070 con 0x5617f3403020 2024-05-26T22:50:50.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.318+0000 7f78ce000640 1 -- 172.21.15.131:0/2597998364 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f78dc037040 con 0x5617f3403020 2024-05-26T22:50:50.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.318+0000 7f78eefd0740 10 client.5907 fetch_fsmap learned FSMap version 33 2024-05-26T22:50:50.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.318+0000 7f78eefd0740 1 -- 172.21.15.131:0/2597998364 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) -- 0x5617f33dc690 con 0x5617f3403020 2024-05-26T22:50:50.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.318+0000 7f78ce000640 1 -- 172.21.15.131:0/2597998364 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78d0023070 con 0x5617f3403020 2024-05-26T22:50:50.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.318+0000 7f78ce000640 1 -- 172.21.15.131:0/2597998364 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 33) ==== 3534+0+0 (secure 0 0 0) 0x7f78dc03e460 con 0x5617f3403020 2024-05-26T22:50:50.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.318+0000 7f78eefd0740 10 client.5907 fetch_fsmap finished waiting for FSMap version 33 2024-05-26T22:50:50.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.318+0000 7f78eefd0740 10 client.5907 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-26T22:50:50.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.318+0000 7f78eefd0740 20 client.5907 populate_metadata read hostname 'smithi131' 2024-05-26T22:50:50.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.318+0000 7f78eefd0740 1 --2- 172.21.15.131:0/2597998364 >> [v2:172.21.15.187:6833/3429891101,v1:172.21.15.187:6835/3429891101] conn(0x5617f33dd7c0 0x5617f340a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:50.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.318+0000 7f78eefd0740 4 client.5907 mds_command: new command op to 4918 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-26T22:50:50.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.318+0000 7f78eefd0740 1 -- 172.21.15.131:0/2597998364 --> [v2:172.21.15.187:6833/3429891101,v1:172.21.15.187:6835/3429891101] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5617f3336d40 con 0x5617f33dd7c0 2024-05-26T22:50:50.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.319+0000 7f78e9600640 1 --2- 172.21.15.131:0/2597998364 >> [v2:172.21.15.187:6833/3429891101,v1:172.21.15.187:6835/3429891101] conn(0x5617f33dd7c0 0x5617f340a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:50.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.320+0000 7f78e9600640 1 --2- 172.21.15.131:0/2597998364 >> [v2:172.21.15.187:6833/3429891101,v1:172.21.15.187:6835/3429891101] conn(0x5617f33dd7c0 0x5617f340a640 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4bb1942cc028ee9a server_cookie=ebf740c9bba39aec in_seq=0 out_seq=0 2024-05-26T22:50:50.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.320+0000 7f78ce000640 10 client.5907 ms_handle_connect on v2:172.21.15.187:6833/3429891101 2024-05-26T22:50:50.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.322+0000 7f78ce000640 1 -- 172.21.15.131:0/2597998364 <== mds.1 v2:172.21.15.187:6833/3429891101 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5617f3336d40 con 0x5617f33dd7c0 2024-05-26T22:50:50.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.322+0000 7f78ce000640 10 client.5907 handle_command_reply: tid=0 2024-05-26T22:50:50.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.322+0000 7f78eefd0740 1 client.5907 shutdown 2024-05-26T22:50:50.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.322+0000 7f78eefd0740 1 -- 172.21.15.131:0/2597998364 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5617f3403020 msgr2=0x5617f33de1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:50.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.322+0000 7f78eefd0740 1 --2- 172.21.15.131:0/2597998364 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5617f3403020 0x5617f33de1f0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f78dc009960 tx=0x7f78dc0046c0 comp rx=0 tx=0).stop 2024-05-26T22:50:50.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.322+0000 7f78eefd0740 1 -- 172.21.15.131:0/2597998364 shutdown_connections 2024-05-26T22:50:50.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.322+0000 7f78eefd0740 1 --2- 172.21.15.131:0/2597998364 >> [v2:172.21.15.187:6833/3429891101,v1:172.21.15.187:6835/3429891101] conn(0x5617f33dd7c0 0x5617f340a640 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:50.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.322+0000 7f78eefd0740 1 --2- 172.21.15.131:0/2597998364 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5617f340ee50 0x5617f33db9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:50.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.322+0000 7f78eefd0740 1 --2- 172.21.15.131:0/2597998364 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5617f3403020 0x5617f33de1f0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:50.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.322+0000 7f78eefd0740 1 --2- 172.21.15.131:0/2597998364 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5617f33f1f70 0x5617f33f4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:50.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.322+0000 7f78ce000640 0 client.5907 ms_handle_reset on v2:172.21.15.187:6833/3429891101 2024-05-26T22:50:50.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.322+0000 7f78eefd0740 1 -- 172.21.15.131:0/2597998364 >> 172.21.15.131:0/2597998364 conn(0x5617f33f1830 msgr2=0x5617f340c2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:50.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.322+0000 7f78eefd0740 1 -- 172.21.15.131:0/2597998364 shutdown_connections 2024-05-26T22:50:50.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.323+0000 7f78eefd0740 1 -- 172.21.15.131:0/2597998364 wait complete. 2024-05-26T22:50:50.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.323+0000 7f78eefd0740 20 client.5907 trim_cache size 0 max 16384 2024-05-26T22:50:50.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.342+0000 7f78eefd0740 1 Processor -- start 2024-05-26T22:50:50.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.342+0000 7f78eefd0740 1 -- start start 2024-05-26T22:50:50.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.343+0000 7f78eefd0740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5617f33f1f70 0x5617f33c1b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:50.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.343+0000 7f78eefd0740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5617f3403020 0x5617f340e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:50.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.343+0000 7f78eefd0740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5617f340ee50 0x5617f340e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:50.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.343+0000 7f78eefd0740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f78e41ff9c0 con 0x5617f3403020 2024-05-26T22:50:50.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.343+0000 7f78eefd0740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f78e4200050 con 0x5617f33f1f70 2024-05-26T22:50:50.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.343+0000 7f78eefd0740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f78e41fea80 con 0x5617f340ee50 2024-05-26T22:50:50.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.343+0000 7f78e8c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5617f3403020 0x5617f340e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:50.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.343+0000 7f78e8c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5617f3403020 0x5617f340e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54230/0 (socket says 172.21.15.131:54230) 2024-05-26T22:50:50.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.343+0000 7f78e8c00640 1 -- 172.21.15.131:0/818574752 learned_addr learned my addr 172.21.15.131:0/818574752 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:50.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.344+0000 7f78e8c00640 1 -- 172.21.15.131:0/818574752 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5617f340ee50 msgr2=0x5617f340e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:50.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.344+0000 7f78e8c00640 1 --2- 172.21.15.131:0/818574752 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5617f340ee50 0x5617f340e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:50.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.344+0000 7f78e8c00640 1 -- 172.21.15.131:0/818574752 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5617f33f1f70 msgr2=0x5617f33c1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:50.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.344+0000 7f78e8c00640 1 --2- 172.21.15.131:0/818574752 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5617f33f1f70 0x5617f33c1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:50.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.344+0000 7f78e8c00640 1 -- 172.21.15.131:0/818574752 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617f33c4b90 con 0x5617f3403020 2024-05-26T22:50:50.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.344+0000 7f78e8c00640 1 --2- 172.21.15.131:0/818574752 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5617f3403020 0x5617f340e260 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f78d000dec0 tx=0x7f78d0015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1f1fa57a310ddb1 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:50.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.344+0000 7f78d9200640 1 -- 172.21.15.131:0/818574752 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d00147b0 con 0x5617f3403020 2024-05-26T22:50:50.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.344+0000 7f78d9200640 1 -- 172.21.15.131:0/818574752 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78d0023040 con 0x5617f3403020 2024-05-26T22:50:50.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.344+0000 7f78d9200640 1 -- 172.21.15.131:0/818574752 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d0014b70 con 0x5617f3403020 2024-05-26T22:50:50.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.345+0000 7f78eefd0740 1 -- 172.21.15.131:0/818574752 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5617f3403020 msgr2=0x5617f340e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:50.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.345+0000 7f78eefd0740 1 --2- 172.21.15.131:0/818574752 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5617f3403020 0x5617f340e260 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f78d000dec0 tx=0x7f78d0015000 comp rx=0 tx=0).stop 2024-05-26T22:50:50.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.345+0000 7f78eefd0740 1 -- 172.21.15.131:0/818574752 shutdown_connections 2024-05-26T22:50:50.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.345+0000 7f78eefd0740 1 --2- 172.21.15.131:0/818574752 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5617f340ee50 0x5617f340e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:50.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.345+0000 7f78eefd0740 1 --2- 172.21.15.131:0/818574752 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5617f3403020 0x5617f340e260 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:50.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.345+0000 7f78eefd0740 1 --2- 172.21.15.131:0/818574752 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5617f33f1f70 0x5617f33c1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:50.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.345+0000 7f78eefd0740 1 -- 172.21.15.131:0/818574752 >> 172.21.15.131:0/818574752 conn(0x5617f33f1830 msgr2=0x5617f33db7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:50.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.345+0000 7f78eefd0740 1 -- 172.21.15.131:0/818574752 shutdown_connections 2024-05-26T22:50:50.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.345+0000 7f78eefd0740 1 -- 172.21.15.131:0/818574752 wait complete. 2024-05-26T22:50:50.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.346+0000 7f78eefd0740 1 -- start start 2024-05-26T22:50:50.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.346+0000 7f78eefd0740 1 Processor -- start 2024-05-26T22:50:50.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.347+0000 7f78eefd0740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5617f33f1f70 0x5617f340e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:50.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.347+0000 7f78eefd0740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5617f3403020 0x5617f340e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:50.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.347+0000 7f78eefd0740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5617f340ee50 0x5617f348aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:50.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.347+0000 7f78eefd0740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5617f33dd640 con 0x5617f33f1f70 2024-05-26T22:50:50.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.347+0000 7f78eefd0740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5617f33dc970 con 0x5617f340ee50 2024-05-26T22:50:50.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.347+0000 7f78eefd0740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x5617f33ddc90 con 0x5617f3403020 2024-05-26T22:50:50.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.347+0000 7f78eefd0740 10 client.0 _ll_register_callbacks cb 0x7f78dc035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-26T22:50:50.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.347+0000 7f78e9600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5617f33f1f70 0x5617f340e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:50.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.347+0000 7f78e9600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5617f33f1f70 0x5617f340e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54242/0 (socket says 172.21.15.131:54242) 2024-05-26T22:50:50.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.347+0000 7f78e9600640 1 -- 172.21.15.131:0/3099863451 learned_addr learned my addr 172.21.15.131:0/3099863451 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:50.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.347+0000 7f78e9600640 1 -- 172.21.15.131:0/3099863451 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5617f3403020 msgr2=0x5617f340e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:50.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.347+0000 7f78e9600640 1 --2- 172.21.15.131:0/3099863451 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5617f3403020 0x5617f340e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:50.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.347+0000 7f78e9600640 1 -- 172.21.15.131:0/3099863451 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5617f340ee50 msgr2=0x5617f348aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:50.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.347+0000 7f78e9600640 1 --2- 172.21.15.131:0/3099863451 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5617f340ee50 0x5617f348aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:50.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.347+0000 7f78e9600640 1 -- 172.21.15.131:0/3099863451 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617f33c1c30 con 0x5617f33f1f70 2024-05-26T22:50:50.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.348+0000 7f78e9600640 1 --2- 172.21.15.131:0/3099863451 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5617f33f1f70 0x5617f340e260 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f78d4059fa0 tx=0x7f78d405a0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a95437050a8cf69 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:50.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.348+0000 7f78ce000640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-26T22:50:50.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.348+0000 7f78ce000640 1 -- 172.21.15.131:0/3099863451 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d401d040 con 0x5617f33f1f70 2024-05-26T22:50:50.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.348+0000 7f78ce000640 1 -- 172.21.15.131:0/3099863451 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f78d4056680 con 0x5617f33f1f70 2024-05-26T22:50:50.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.348+0000 7f78eefd0740 1 -- 172.21.15.131:0/3099863451 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78d0023040 con 0x5617f33f1f70 2024-05-26T22:50:50.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.348+0000 7f78ce000640 1 -- 172.21.15.131:0/3099863451 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f78d4078e60 con 0x5617f33f1f70 2024-05-26T22:50:50.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.348+0000 7f78eefd0740 10 client.5913 fetch_fsmap learned FSMap version 33 2024-05-26T22:50:50.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.348+0000 7f78eefd0740 1 -- 172.21.15.131:0/3099863451 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) -- 0x5617f341ad30 con 0x5617f33f1f70 2024-05-26T22:50:50.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.348+0000 7f78ce000640 1 -- 172.21.15.131:0/3099863451 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78d0023040 con 0x5617f33f1f70 2024-05-26T22:50:50.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.348+0000 7f78ce000640 1 -- 172.21.15.131:0/3099863451 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 33) ==== 3534+0+0 (secure 0 0 0) 0x7f78d40626b0 con 0x5617f33f1f70 2024-05-26T22:50:50.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.348+0000 7f78eefd0740 10 client.5913 fetch_fsmap finished waiting for FSMap version 33 2024-05-26T22:50:50.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.348+0000 7f78eefd0740 10 client.5913 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-26T22:50:50.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.348+0000 7f78eefd0740 20 client.5913 populate_metadata read hostname 'smithi131' 2024-05-26T22:50:50.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.348+0000 7f78eefd0740 1 --2- 172.21.15.131:0/3099863451 >> [v2:172.21.15.187:6833/3429891101,v1:172.21.15.187:6835/3429891101] conn(0x5617f33ed4a0 0x5617f33ed880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:50.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.348+0000 7f78eefd0740 4 client.5913 mds_command: new command op to 4918 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-26T22:50:50.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.348+0000 7f78eefd0740 1 -- 172.21.15.131:0/3099863451 --> [v2:172.21.15.187:6833/3429891101,v1:172.21.15.187:6835/3429891101] -- command(tid 0: {"prefix": "status"}) -- 0x5617f3331090 con 0x5617f33ed4a0 2024-05-26T22:50:50.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.349+0000 7f78e8c00640 1 --2- 172.21.15.131:0/3099863451 >> [v2:172.21.15.187:6833/3429891101,v1:172.21.15.187:6835/3429891101] conn(0x5617f33ed4a0 0x5617f33ed880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:50.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.350+0000 7f78e8c00640 1 --2- 172.21.15.131:0/3099863451 >> [v2:172.21.15.187:6833/3429891101,v1:172.21.15.187:6835/3429891101] conn(0x5617f33ed4a0 0x5617f33ed880 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cf06edc4e3afeecb server_cookie=8a660e323ac609f5 in_seq=0 out_seq=0 2024-05-26T22:50:50.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.350+0000 7f78ce000640 10 client.5913 ms_handle_connect on v2:172.21.15.187:6833/3429891101 2024-05-26T22:50:50.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.351+0000 7f78ce000640 1 -- 172.21.15.131:0/3099863451 <== mds.1 v2:172.21.15.187:6833/3429891101 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x5617f3331090 con 0x5617f33ed4a0 2024-05-26T22:50:50.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.351+0000 7f78ce000640 10 client.5913 handle_command_reply: tid=0 2024-05-26T22:50:50.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.351+0000 7f78eefd0740 1 client.5913 shutdown 2024-05-26T22:50:50.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.351+0000 7f78eefd0740 1 -- 172.21.15.131:0/3099863451 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5617f33f1f70 msgr2=0x5617f340e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:50.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.351+0000 7f78eefd0740 1 --2- 172.21.15.131:0/3099863451 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5617f33f1f70 0x5617f340e260 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f78d4059fa0 tx=0x7f78d405a0e0 comp rx=0 tx=0).stop 2024-05-26T22:50:50.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.351+0000 7f78eefd0740 1 -- 172.21.15.131:0/3099863451 shutdown_connections 2024-05-26T22:50:50.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.351+0000 7f78eefd0740 1 --2- 172.21.15.131:0/3099863451 >> [v2:172.21.15.187:6833/3429891101,v1:172.21.15.187:6835/3429891101] conn(0x5617f33ed4a0 0x5617f33ed880 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:50.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.351+0000 7f78eefd0740 1 --2- 172.21.15.131:0/3099863451 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5617f3403020 0x5617f340e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:50.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.351+0000 7f78eefd0740 1 --2- 172.21.15.131:0/3099863451 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5617f340ee50 0x5617f348aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:50.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.351+0000 7f78ce000640 0 client.5913 ms_handle_reset on v2:172.21.15.187:6833/3429891101 2024-05-26T22:50:50.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.351+0000 7f78eefd0740 1 --2- 172.21.15.131:0/3099863451 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5617f33f1f70 0x5617f340e260 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:50.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.351+0000 7f78eefd0740 1 -- 172.21.15.131:0/3099863451 >> 172.21.15.131:0/3099863451 conn(0x5617f33f1830 msgr2=0x5617f3403de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:50.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.351+0000 7f78eefd0740 1 -- 172.21.15.131:0/3099863451 shutdown_connections 2024-05-26T22:50:50.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.351+0000 7f78eefd0740 1 -- 172.21.15.131:0/3099863451 wait complete. 2024-05-26T22:50:50.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.351+0000 7f78eefd0740 20 client.5913 trim_cache size 0 max 16384 2024-05-26T22:50:50.354 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-26T22:50:50.354 INFO:teuthology.orchestra.run.smithi131.stdout: "cluster_fsid": "46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f", 2024-05-26T22:50:50.354 INFO:teuthology.orchestra.run.smithi131.stdout: "whoami": 1, 2024-05-26T22:50:50.354 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 4918, 2024-05-26T22:50:50.354 INFO:teuthology.orchestra.run.smithi131.stdout: "want_state": "up:active", 2024-05-26T22:50:50.354 INFO:teuthology.orchestra.run.smithi131.stdout: "state": "up:active", 2024-05-26T22:50:50.354 INFO:teuthology.orchestra.run.smithi131.stdout: "fs_name": "cephfs", 2024-05-26T22:50:50.354 INFO:teuthology.orchestra.run.smithi131.stdout: "rank_uptime": 3.2637911449999999, 2024-05-26T22:50:50.354 INFO:teuthology.orchestra.run.smithi131.stdout: "mdsmap_epoch": 33, 2024-05-26T22:50:50.354 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch": 44, 2024-05-26T22:50:50.354 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch_barrier": 44, 2024-05-26T22:50:50.354 INFO:teuthology.orchestra.run.smithi131.stdout: "uptime": 28.241063566000001 2024-05-26T22:50:50.354 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-26T22:50:50.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.352+0000 7f78eaa00640 1 -- 172.21.15.131:0/84850340 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f78c406be30 msgr2=0x7f78c406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:50.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.352+0000 7f78eaa00640 1 --2- 172.21.15.131:0/84850340 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f78c406be30 0x7f78c406e2f0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f78d0006fd0 tx=0x7f78d0008040 comp rx=0 tx=0).stop 2024-05-26T22:50:50.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.352+0000 7f78eaa00640 1 -- 172.21.15.131:0/84850340 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f78e40b2f60 msgr2=0x7f78e414e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:50.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.352+0000 7f78eaa00640 1 --2- 172.21.15.131:0/84850340 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f78e40b2f60 0x7f78e414e750 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f78d400ce00 tx=0x7f78d400a650 comp rx=0 tx=0).stop 2024-05-26T22:50:50.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.352+0000 7f78eaa00640 1 -- 172.21.15.131:0/84850340 shutdown_connections 2024-05-26T22:50:50.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.352+0000 7f78eaa00640 1 --2- 172.21.15.131:0/84850340 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f78e40d0380 0x7f78e414ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:50.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.352+0000 7f78eaa00640 1 --2- 172.21.15.131:0/84850340 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f78c406be30 0x7f78c406e2f0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:50.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.352+0000 7f78eaa00640 1 --2- 172.21.15.131:0/84850340 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f78e415e320 0x7f78e41f7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:50.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.352+0000 7f78eaa00640 1 --2- 172.21.15.131:0/84850340 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f78e40b2f60 0x7f78e414e750 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:50.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.352+0000 7f78eaa00640 1 -- 172.21.15.131:0/84850340 >> 172.21.15.131:0/84850340 conn(0x7f78e40b5660 msgr2=0x7f78e4071f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:50.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.352+0000 7f78eaa00640 1 -- 172.21.15.131:0/84850340 shutdown_connections 2024-05-26T22:50:50.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.352+0000 7f78eaa00640 1 -- 172.21.15.131:0/84850340 wait complete. 2024-05-26T22:50:50.368 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-05-26T22:50:50.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.555+0000 7fc09f400640 1 Processor -- start 2024-05-26T22:50:50.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.555+0000 7fc09f400640 1 -- start start 2024-05-26T22:50:50.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.557+0000 7fc09f400640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fc098165e30 0x7fc098072540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:50.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.557+0000 7fc09f400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fc0980af230 0x7fc0980cfe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:50.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.557+0000 7fc09f400640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0980733c0 con 0x7fc0980d0390 2024-05-26T22:50:50.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.557+0000 7fc09f400640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fc0980d07e0 con 0x7fc0980af230 2024-05-26T22:50:50.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.557+0000 7fc09f400640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fc098072e90 con 0x7fc098165e30 2024-05-26T22:50:50.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.557+0000 7fc097e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fc0980af230 0x7fc0980cfe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:50.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.558+0000 7fc097e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fc0980af230 0x7fc0980cfe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54258/0 (socket says 172.21.15.131:54258) 2024-05-26T22:50:50.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.558+0000 7fc097e00640 1 -- 172.21.15.131:0/434259586 learned_addr learned my addr 172.21.15.131:0/434259586 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:50.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.558+0000 7fc09ca00640 1 --2- 172.21.15.131:0/434259586 >> v2:172.21.15.187:3300/0 conn(0x7fc098165e30 0x7fc098072540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:50.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.558+0000 7fc097e00640 1 -- 172.21.15.131:0/434259586 >> v2:172.21.15.187:3300/0 conn(0x7fc098165e30 msgr2=0x7fc098072540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:50.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.558+0000 7fc097e00640 1 --2- 172.21.15.131:0/434259586 >> v2:172.21.15.187:3300/0 conn(0x7fc098165e30 0x7fc098072540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:50.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.558+0000 7fc097e00640 1 -- 172.21.15.131:0/434259586 >> v1:172.21.15.187:6789/0 conn(0x7fc0980d0390 legacy=0x7fc098071360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:50.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.558+0000 7fc097e00640 1 -- 172.21.15.131:0/434259586 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0980cf540 con 0x7fc0980af230 2024-05-26T22:50:50.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.559+0000 7fc097e00640 1 --2- 172.21.15.131:0/434259586 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc0980af230 0x7fc0980cfe50 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fc0980d0c40 tx=0x7fc08002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee79a6dfaf9261db server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:50.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.559+0000 7fc097400640 1 -- 172.21.15.131:0/434259586 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc080037070 con 0x7fc0980af230 2024-05-26T22:50:50.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.559+0000 7fc097400640 1 -- 172.21.15.131:0/434259586 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc080032440 con 0x7fc0980af230 2024-05-26T22:50:50.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.559+0000 7fc097400640 1 -- 172.21.15.131:0/434259586 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc080032810 con 0x7fc0980af230 2024-05-26T22:50:50.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.560+0000 7fc09f400640 1 -- 172.21.15.131:0/434259586 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc0980af230 msgr2=0x7fc0980cfe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:50.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.560+0000 7fc09f400640 1 --2- 172.21.15.131:0/434259586 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc0980af230 0x7fc0980cfe50 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fc0980d0c40 tx=0x7fc08002a1a0 comp rx=0 tx=0).stop 2024-05-26T22:50:50.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.560+0000 7fc09f400640 1 -- 172.21.15.131:0/434259586 shutdown_connections 2024-05-26T22:50:50.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.560+0000 7fc09f400640 1 --2- 172.21.15.131:0/434259586 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc0980af230 0x7fc0980cfe50 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:50.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.560+0000 7fc09f400640 1 --2- 172.21.15.131:0/434259586 >> v2:172.21.15.187:3300/0 conn(0x7fc098165e30 0x7fc098072540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:50.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.560+0000 7fc09f400640 1 -- 172.21.15.131:0/434259586 >> 172.21.15.131:0/434259586 conn(0x7fc09806bc90 msgr2=0x7fc0980b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:50.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.560+0000 7fc09f400640 1 -- 172.21.15.131:0/434259586 shutdown_connections 2024-05-26T22:50:50.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.560+0000 7fc09f400640 1 -- 172.21.15.131:0/434259586 wait complete. 2024-05-26T22:50:50.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.562+0000 7fc09f400640 1 Processor -- start 2024-05-26T22:50:50.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.562+0000 7fc09f400640 1 -- start start 2024-05-26T22:50:50.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.563+0000 7fc09f400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc0980af230 0x7fc098160eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:50.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.563+0000 7fc09f400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc0980d0390 0x7fc09814f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:50.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.563+0000 7fc09f400640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc098165e30 0x7fc0981625e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:50.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.563+0000 7fc09f400640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc0980cfac0 con 0x7fc0980d0390 2024-05-26T22:50:50.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.563+0000 7fc09f400640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc0980cf7e0 con 0x7fc098165e30 2024-05-26T22:50:50.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.563+0000 7fc09f400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc0980cece0 con 0x7fc0980af230 2024-05-26T22:50:50.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.563+0000 7fc097e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc0980d0390 0x7fc09814f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:50.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.564+0000 7fc097e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc0980d0390 0x7fc09814f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54272/0 (socket says 172.21.15.131:54272) 2024-05-26T22:50:50.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.564+0000 7fc097e00640 1 -- 172.21.15.131:0/4047150038 learned_addr learned my addr 172.21.15.131:0/4047150038 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:50.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.564+0000 7fc097e00640 1 -- 172.21.15.131:0/4047150038 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc0980af230 msgr2=0x7fc098160eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:50.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.564+0000 7fc09ca00640 1 --2- 172.21.15.131:0/4047150038 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc0980af230 0x7fc098160eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:50.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.564+0000 7fc09d400640 1 --2- 172.21.15.131:0/4047150038 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc098165e30 0x7fc0981625e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:50.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.564+0000 7fc097e00640 1 --2- 172.21.15.131:0/4047150038 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc0980af230 0x7fc098160eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:50.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.564+0000 7fc097e00640 1 -- 172.21.15.131:0/4047150038 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc098165e30 msgr2=0x7fc0981625e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:50.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.564+0000 7fc097e00640 1 --2- 172.21.15.131:0/4047150038 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc098165e30 0x7fc0981625e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:50.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.564+0000 7fc097e00640 1 -- 172.21.15.131:0/4047150038 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc09809d4f0 con 0x7fc0980d0390 2024-05-26T22:50:50.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.564+0000 7fc097e00640 1 --2- 172.21.15.131:0/4047150038 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc0980d0390 0x7fc09814f250 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fc080000c00 tx=0x7fc0800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:50.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.565+0000 7fc094c00640 1 -- 172.21.15.131:0/4047150038 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc080041020 con 0x7fc0980d0390 2024-05-26T22:50:50.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.565+0000 7fc094c00640 1 -- 172.21.15.131:0/4047150038 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0800306c0 con 0x7fc0980d0390 2024-05-26T22:50:50.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.565+0000 7fc094c00640 1 -- 172.21.15.131:0/4047150038 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc080037040 con 0x7fc0980d0390 2024-05-26T22:50:50.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.565+0000 7fc09f400640 1 -- 172.21.15.131:0/4047150038 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc098162b20 con 0x7fc0980d0390 2024-05-26T22:50:50.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.565+0000 7fc09f400640 1 -- 172.21.15.131:0/4047150038 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc098161820 con 0x7fc0980d0390 2024-05-26T22:50:50.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.567+0000 7fc094c00640 1 -- 172.21.15.131:0/4047150038 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc080044020 con 0x7fc0980d0390 2024-05-26T22:50:50.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.568+0000 7fc094c00640 1 --2- 172.21.15.131:0/4047150038 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc07406be30 0x7fc07406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:50.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.568+0000 7fc094c00640 1 -- 172.21.15.131:0/4047150038 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7fc0800ab740 con 0x7fc0980d0390 2024-05-26T22:50:50.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.568+0000 7fc09ca00640 1 --2- 172.21.15.131:0/4047150038 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc07406be30 0x7fc07406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:50.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.569+0000 7fc0a221b740 1 Processor -- start 2024-05-26T22:50:50.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.569+0000 7fc09ca00640 1 --2- 172.21.15.131:0/4047150038 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc07406be30 0x7fc07406e2f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fc088006fd0 tx=0x7fc088008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:50.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.569+0000 7fc0a221b740 1 -- start start 2024-05-26T22:50:50.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.570+0000 7fc0a221b740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55b61b159c70 0x55b61b10c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:50.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.570+0000 7fc0a221b740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b61b13cd90 0x55b61b13c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:50.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.570+0000 7fc0a221b740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b61b13c650 0x55b61b111460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:50.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.570+0000 7fc0a221b740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55b61b074610 con 0x55b61b13cd90 2024-05-26T22:50:50.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.570+0000 7fc0a221b740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55b61b01c5c0 con 0x55b61b13c650 2024-05-26T22:50:50.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.570+0000 7fc0a221b740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x55b61b0705d0 con 0x55b61b159c70 2024-05-26T22:50:50.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.570+0000 7fc097e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b61b13cd90 0x55b61b13c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:50.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.570+0000 7fc097e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b61b13cd90 0x55b61b13c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54276/0 (socket says 172.21.15.131:54276) 2024-05-26T22:50:50.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.570+0000 7fc097e00640 1 -- 172.21.15.131:0/344741450 learned_addr learned my addr 172.21.15.131:0/344741450 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:50.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.570+0000 7fc09ca00640 1 --2- 172.21.15.131:0/344741450 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55b61b159c70 0x55b61b10c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:50.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.570+0000 7fc097e00640 1 -- 172.21.15.131:0/344741450 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55b61b159c70 msgr2=0x55b61b10c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:50.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.570+0000 7fc097e00640 1 --2- 172.21.15.131:0/344741450 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55b61b159c70 0x55b61b10c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:50.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.570+0000 7fc097e00640 1 -- 172.21.15.131:0/344741450 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b61b13c650 msgr2=0x55b61b111460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:50.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.570+0000 7fc097e00640 1 --2- 172.21.15.131:0/344741450 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b61b13c650 0x55b61b111460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:50.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.570+0000 7fc097e00640 1 -- 172.21.15.131:0/344741450 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b61ad882d0 con 0x55b61b13cd90 2024-05-26T22:50:50.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.571+0000 7fc097e00640 1 --2- 172.21.15.131:0/344741450 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b61b13cd90 0x55b61b13c110 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fc0800792b0 tx=0x7fc0800793a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b75bb0baec1a07a server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:50.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.571+0000 7fc08d200640 1 -- 172.21.15.131:0/344741450 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc080079950 con 0x55b61b13cd90 2024-05-26T22:50:50.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.571+0000 7fc08d200640 1 -- 172.21.15.131:0/344741450 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc08007dd00 con 0x55b61b13cd90 2024-05-26T22:50:50.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.571+0000 7fc08d200640 1 -- 172.21.15.131:0/344741450 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc080082530 con 0x55b61b13cd90 2024-05-26T22:50:50.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.571+0000 7fc0a221b740 1 -- 172.21.15.131:0/344741450 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b61b13cd90 msgr2=0x55b61b13c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:50.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.571+0000 7fc0a221b740 1 --2- 172.21.15.131:0/344741450 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b61b13cd90 0x55b61b13c110 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fc0800792b0 tx=0x7fc0800793a0 comp rx=0 tx=0).stop 2024-05-26T22:50:50.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.572+0000 7fc0a221b740 1 -- 172.21.15.131:0/344741450 shutdown_connections 2024-05-26T22:50:50.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.572+0000 7fc0a221b740 1 --2- 172.21.15.131:0/344741450 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b61b13c650 0x55b61b111460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:50.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.572+0000 7fc0a221b740 1 --2- 172.21.15.131:0/344741450 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b61b13cd90 0x55b61b13c110 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:50.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.572+0000 7fc0a221b740 1 --2- 172.21.15.131:0/344741450 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55b61b159c70 0x55b61b10c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:50.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.572+0000 7fc0a221b740 1 -- 172.21.15.131:0/344741450 >> 172.21.15.131:0/344741450 conn(0x55b61b10eb00 msgr2=0x55b61b157110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:50.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.572+0000 7fc0a221b740 1 -- 172.21.15.131:0/344741450 shutdown_connections 2024-05-26T22:50:50.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.572+0000 7fc0a221b740 1 -- 172.21.15.131:0/344741450 wait complete. 2024-05-26T22:50:50.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.572+0000 7fc0a221b740 1 -- start start 2024-05-26T22:50:50.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.573+0000 7fc0a221b740 1 Processor -- start 2024-05-26T22:50:50.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.574+0000 7fc0a221b740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55b61b13cd90 0x55b61b12ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:50.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.574+0000 7fc0a221b740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b61b159c70 0x55b61b13bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:50.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.574+0000 7fc0a221b740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b61b13d280 0x55b61b1587a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:50.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.574+0000 7fc0a221b740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55b61b074610 con 0x55b61b159c70 2024-05-26T22:50:50.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.574+0000 7fc0a221b740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55b61b02f910 con 0x55b61b13d280 2024-05-26T22:50:50.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.574+0000 7fc0a221b740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x55b61b110a20 con 0x55b61b13cd90 2024-05-26T22:50:50.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.574+0000 7fc0a221b740 10 client.0 _ll_register_callbacks cb 0x55b61b06b410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-26T22:50:50.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.574+0000 7fc097e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b61b159c70 0x55b61b13bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:50.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.574+0000 7fc097e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b61b159c70 0x55b61b13bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54288/0 (socket says 172.21.15.131:54288) 2024-05-26T22:50:50.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.574+0000 7fc097e00640 1 -- 172.21.15.131:0/3437659703 learned_addr learned my addr 172.21.15.131:0/3437659703 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:50.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.574+0000 7fc097e00640 1 -- 172.21.15.131:0/3437659703 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55b61b13cd90 msgr2=0x55b61b12ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:50.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.574+0000 7fc097e00640 1 --2- 172.21.15.131:0/3437659703 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55b61b13cd90 0x55b61b12ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:50.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.574+0000 7fc097e00640 1 -- 172.21.15.131:0/3437659703 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b61b13d280 msgr2=0x55b61b1587a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:50.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.574+0000 7fc097e00640 1 --2- 172.21.15.131:0/3437659703 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b61b13d280 0x55b61b1587a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:50.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.574+0000 7fc097e00640 1 -- 172.21.15.131:0/3437659703 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b61ae526b0 con 0x55b61b159c70 2024-05-26T22:50:50.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.575+0000 7fc097e00640 1 --2- 172.21.15.131:0/3437659703 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b61b159c70 0x55b61b13bfe0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fc08007d040 tx=0x7fc080080050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bac6b405abab8d6 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:50.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.575+0000 7fc085600640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-26T22:50:50.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.575+0000 7fc085600640 1 -- 172.21.15.131:0/3437659703 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc080096060 con 0x55b61b159c70 2024-05-26T22:50:50.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.575+0000 7fc085600640 1 -- 172.21.15.131:0/3437659703 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7fc080079bb0 con 0x55b61b159c70 2024-05-26T22:50:50.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.575+0000 7fc0a221b740 1 -- 172.21.15.131:0/3437659703 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc08007dd00 con 0x55b61b159c70 2024-05-26T22:50:50.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.575+0000 7fc085600640 1 -- 172.21.15.131:0/3437659703 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fc080079950 con 0x55b61b159c70 2024-05-26T22:50:50.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.575+0000 7fc0a221b740 10 client.5925 fetch_fsmap learned FSMap version 33 2024-05-26T22:50:50.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.575+0000 7fc0a221b740 1 -- 172.21.15.131:0/3437659703 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) -- 0x55b61ae526b0 con 0x55b61b159c70 2024-05-26T22:50:50.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.575+0000 7fc085600640 1 -- 172.21.15.131:0/3437659703 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc08007dd00 con 0x55b61b159c70 2024-05-26T22:50:50.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.575+0000 7fc085600640 1 -- 172.21.15.131:0/3437659703 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 33) ==== 3534+0+0 (secure 0 0 0) 0x7fc08003a030 con 0x55b61b159c70 2024-05-26T22:50:50.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.575+0000 7fc0a221b740 10 client.5925 fetch_fsmap finished waiting for FSMap version 33 2024-05-26T22:50:50.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.575+0000 7fc0a221b740 10 client.5925 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-26T22:50:50.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.575+0000 7fc0a221b740 20 client.5925 populate_metadata read hostname 'smithi131' 2024-05-26T22:50:50.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.575+0000 7fc0a221b740 1 --2- 172.21.15.131:0/3437659703 >> [v2:172.21.15.131:6834/650770835,v1:172.21.15.131:6836/650770835] conn(0x55b61b13a220 0x55b61b13a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:50.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.575+0000 7fc0a221b740 4 client.5925 mds_command: new command op to 5637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-26T22:50:50.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.575+0000 7fc0a221b740 1 -- 172.21.15.131:0/3437659703 --> [v2:172.21.15.131:6834/650770835,v1:172.21.15.131:6836/650770835] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b61b081d40 con 0x55b61b13a220 2024-05-26T22:50:50.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.576+0000 7fc09ca00640 1 --2- 172.21.15.131:0/3437659703 >> [v2:172.21.15.131:6834/650770835,v1:172.21.15.131:6836/650770835] conn(0x55b61b13a220 0x55b61b13a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:50.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.576+0000 7fc085600640 10 client.5925 ms_handle_connect on v2:172.21.15.131:6834/650770835 2024-05-26T22:50:50.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.576+0000 7fc09ca00640 1 --2- 172.21.15.131:0/3437659703 >> [v2:172.21.15.131:6834/650770835,v1:172.21.15.131:6836/650770835] conn(0x55b61b13a220 0x55b61b13a600 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1fd6dcb8a70edaca server_cookie=9811dd2919585061 in_seq=0 out_seq=0 2024-05-26T22:50:50.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.577+0000 7fc085600640 1 -- 172.21.15.131:0/3437659703 <== mds.0 v2:172.21.15.131:6834/650770835 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b61b081d40 con 0x55b61b13a220 2024-05-26T22:50:50.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.577+0000 7fc085600640 10 client.5925 handle_command_reply: tid=0 2024-05-26T22:50:50.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.577+0000 7fc0a221b740 1 client.5925 shutdown 2024-05-26T22:50:50.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.578+0000 7fc0a221b740 1 -- 172.21.15.131:0/3437659703 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b61b159c70 msgr2=0x55b61b13bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:50.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.578+0000 7fc0a221b740 1 --2- 172.21.15.131:0/3437659703 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b61b159c70 0x55b61b13bfe0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fc08007d040 tx=0x7fc080080050 comp rx=0 tx=0).stop 2024-05-26T22:50:50.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.578+0000 7fc0a221b740 1 -- 172.21.15.131:0/3437659703 shutdown_connections 2024-05-26T22:50:50.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.578+0000 7fc0a221b740 1 --2- 172.21.15.131:0/3437659703 >> [v2:172.21.15.131:6834/650770835,v1:172.21.15.131:6836/650770835] conn(0x55b61b13a220 0x55b61b13a600 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:50.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.578+0000 7fc0a221b740 1 --2- 172.21.15.131:0/3437659703 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b61b13d280 0x55b61b1587a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:50.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.578+0000 7fc0a221b740 1 --2- 172.21.15.131:0/3437659703 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b61b159c70 0x55b61b13bfe0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:50.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.578+0000 7fc0a221b740 1 --2- 172.21.15.131:0/3437659703 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55b61b13cd90 0x55b61b12ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:50.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.578+0000 7fc0a221b740 1 -- 172.21.15.131:0/3437659703 >> 172.21.15.131:0/3437659703 conn(0x55b61b13c650 msgr2=0x55b61b110230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:50.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.578+0000 7fc085600640 0 client.5925 ms_handle_reset on v2:172.21.15.131:6834/650770835 2024-05-26T22:50:50.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.578+0000 7fc0a221b740 1 -- 172.21.15.131:0/3437659703 shutdown_connections 2024-05-26T22:50:50.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.578+0000 7fc0a221b740 1 -- 172.21.15.131:0/3437659703 wait complete. 2024-05-26T22:50:50.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.578+0000 7fc0a221b740 20 client.5925 trim_cache size 0 max 16384 2024-05-26T22:50:50.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.599+0000 7fc0a221b740 1 Processor -- start 2024-05-26T22:50:50.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.599+0000 7fc0a221b740 1 -- start start 2024-05-26T22:50:50.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.600+0000 7fc0a221b740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b61b13cd90 0x55b61b13ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:50.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.600+0000 7fc0a221b740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b61b13d280 0x55b61b10fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:50.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.600+0000 7fc0a221b740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55b61b159c70 0x55b61b1101c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:50.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.600+0000 7fc0a221b740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc098204270 con 0x55b61b13cd90 2024-05-26T22:50:50.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.600+0000 7fc0a221b740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc0982047b0 con 0x55b61b13d280 2024-05-26T22:50:50.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.600+0000 7fc0a221b740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc09809aa00 con 0x55b61b159c70 2024-05-26T22:50:50.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.600+0000 7fc09ca00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b61b13cd90 0x55b61b13ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:50.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.600+0000 7fc09ca00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b61b13cd90 0x55b61b13ffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54298/0 (socket says 172.21.15.131:54298) 2024-05-26T22:50:50.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.600+0000 7fc09ca00640 1 -- 172.21.15.131:0/3730697249 learned_addr learned my addr 172.21.15.131:0/3730697249 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:50.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.601+0000 7fc09ca00640 1 -- 172.21.15.131:0/3730697249 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55b61b159c70 msgr2=0x55b61b1101c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:50.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.601+0000 7fc09ca00640 1 --2- 172.21.15.131:0/3730697249 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55b61b159c70 0x55b61b1101c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:50.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.601+0000 7fc09ca00640 1 -- 172.21.15.131:0/3730697249 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b61b13d280 msgr2=0x55b61b10fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:50.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.601+0000 7fc09ca00640 1 --2- 172.21.15.131:0/3730697249 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b61b13d280 0x55b61b10fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:50.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.601+0000 7fc09ca00640 1 -- 172.21.15.131:0/3730697249 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b61b1542f0 con 0x55b61b13cd90 2024-05-26T22:50:50.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.601+0000 7fc09ca00640 1 --2- 172.21.15.131:0/3730697249 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b61b13cd90 0x55b61b13ffb0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fc08800eb40 tx=0x7fc088011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55241cb0171fb8ce server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:50.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.601+0000 7fc087e00640 1 -- 172.21.15.131:0/3730697249 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc08800f040 con 0x55b61b13cd90 2024-05-26T22:50:50.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.601+0000 7fc087e00640 1 -- 172.21.15.131:0/3730697249 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0880157f0 con 0x55b61b13cd90 2024-05-26T22:50:50.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.601+0000 7fc087e00640 1 -- 172.21.15.131:0/3730697249 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc088015ba0 con 0x55b61b13cd90 2024-05-26T22:50:50.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.602+0000 7fc0a221b740 1 -- 172.21.15.131:0/3730697249 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b61b13cd90 msgr2=0x55b61b13ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:50.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.602+0000 7fc0a221b740 1 --2- 172.21.15.131:0/3730697249 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b61b13cd90 0x55b61b13ffb0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fc08800eb40 tx=0x7fc088011000 comp rx=0 tx=0).stop 2024-05-26T22:50:50.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.602+0000 7fc0a221b740 1 -- 172.21.15.131:0/3730697249 shutdown_connections 2024-05-26T22:50:50.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.602+0000 7fc0a221b740 1 --2- 172.21.15.131:0/3730697249 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55b61b159c70 0x55b61b1101c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:50.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.602+0000 7fc0a221b740 1 --2- 172.21.15.131:0/3730697249 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b61b13d280 0x55b61b10fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:50.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.602+0000 7fc0a221b740 1 --2- 172.21.15.131:0/3730697249 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b61b13cd90 0x55b61b13ffb0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:50.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.602+0000 7fc0a221b740 1 -- 172.21.15.131:0/3730697249 >> 172.21.15.131:0/3730697249 conn(0x55b61b13c650 msgr2=0x55b61b10d2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:50.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.602+0000 7fc0a221b740 1 -- 172.21.15.131:0/3730697249 shutdown_connections 2024-05-26T22:50:50.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.602+0000 7fc0a221b740 1 -- 172.21.15.131:0/3730697249 wait complete. 2024-05-26T22:50:50.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.603+0000 7fc0a221b740 1 -- start start 2024-05-26T22:50:50.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.603+0000 7fc0a221b740 1 Processor -- start 2024-05-26T22:50:50.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.604+0000 7fc0a221b740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b61b13cd90 0x55b61b1d94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:50.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.604+0000 7fc0a221b740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b61b13d280 0x55b61b1389a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:50.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.604+0000 7fc0a221b740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55b61b159c70 0x55b61b156d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:50.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.604+0000 7fc0a221b740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55b61b074610 con 0x55b61b13d280 2024-05-26T22:50:50.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.604+0000 7fc0a221b740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55b61b110a20 con 0x55b61b13cd90 2024-05-26T22:50:50.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.604+0000 7fc0a221b740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x55b61b02f910 con 0x55b61b159c70 2024-05-26T22:50:50.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.604+0000 7fc0a221b740 10 client.0 _ll_register_callbacks cb 0x7fc080090070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-26T22:50:50.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.604+0000 7fc097e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b61b13d280 0x55b61b1389a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:50.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.604+0000 7fc097e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b61b13d280 0x55b61b1389a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54314/0 (socket says 172.21.15.131:54314) 2024-05-26T22:50:50.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.604+0000 7fc097e00640 1 -- 172.21.15.131:0/1803544045 learned_addr learned my addr 172.21.15.131:0/1803544045 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:50.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.605+0000 7fc097e00640 1 -- 172.21.15.131:0/1803544045 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55b61b159c70 msgr2=0x55b61b156d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:50.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.605+0000 7fc097e00640 1 --2- 172.21.15.131:0/1803544045 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55b61b159c70 0x55b61b156d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:50.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.605+0000 7fc097e00640 1 -- 172.21.15.131:0/1803544045 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b61b13cd90 msgr2=0x55b61b1d94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:50.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.605+0000 7fc097e00640 1 --2- 172.21.15.131:0/1803544045 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b61b13cd90 0x55b61b1d94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:50.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.605+0000 7fc097e00640 1 -- 172.21.15.131:0/1803544045 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b61b158af0 con 0x55b61b13d280 2024-05-26T22:50:50.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.605+0000 7fc097e00640 1 --2- 172.21.15.131:0/1803544045 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b61b13d280 0x55b61b1389a0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fc0800792b0 tx=0x7fc080075bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2df4a6457a067c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:50.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.605+0000 7fc085600640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-26T22:50:50.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.605+0000 7fc085600640 1 -- 172.21.15.131:0/1803544045 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0800822a0 con 0x55b61b13d280 2024-05-26T22:50:50.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.605+0000 7fc085600640 1 -- 172.21.15.131:0/1803544045 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7fc080091bb0 con 0x55b61b13d280 2024-05-26T22:50:50.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.605+0000 7fc0a221b740 1 -- 172.21.15.131:0/1803544045 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0880157f0 con 0x55b61b13d280 2024-05-26T22:50:50.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.605+0000 7fc085600640 1 -- 172.21.15.131:0/1803544045 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fc08007dd00 con 0x55b61b13d280 2024-05-26T22:50:50.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.605+0000 7fc0a221b740 10 client.5931 fetch_fsmap learned FSMap version 33 2024-05-26T22:50:50.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.605+0000 7fc0a221b740 1 -- 172.21.15.131:0/1803544045 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) -- 0x55b61b158af0 con 0x55b61b13d280 2024-05-26T22:50:50.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.606+0000 7fc085600640 1 -- 172.21.15.131:0/1803544045 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0880157f0 con 0x55b61b13d280 2024-05-26T22:50:50.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.606+0000 7fc085600640 1 -- 172.21.15.131:0/1803544045 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 33) ==== 3534+0+0 (secure 0 0 0) 0x7fc0800756f0 con 0x55b61b13d280 2024-05-26T22:50:50.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.606+0000 7fc0a221b740 10 client.5931 fetch_fsmap finished waiting for FSMap version 33 2024-05-26T22:50:50.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.606+0000 7fc0a221b740 10 client.5931 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-26T22:50:50.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.606+0000 7fc0a221b740 20 client.5931 populate_metadata read hostname 'smithi131' 2024-05-26T22:50:50.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.606+0000 7fc0a221b740 1 --2- 172.21.15.131:0/1803544045 >> [v2:172.21.15.131:6834/650770835,v1:172.21.15.131:6836/650770835] conn(0x55b61b1d4da0 0x55b61b152d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:50.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.606+0000 7fc0a221b740 4 client.5931 mds_command: new command op to 5637 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-26T22:50:50.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.606+0000 7fc0a221b740 1 -- 172.21.15.131:0/1803544045 --> [v2:172.21.15.131:6834/650770835,v1:172.21.15.131:6836/650770835] -- command(tid 0: {"prefix": "status"}) -- 0x55b61b07c090 con 0x55b61b1d4da0 2024-05-26T22:50:50.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.606+0000 7fc09ca00640 1 --2- 172.21.15.131:0/1803544045 >> [v2:172.21.15.131:6834/650770835,v1:172.21.15.131:6836/650770835] conn(0x55b61b1d4da0 0x55b61b152d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:50.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.607+0000 7fc09ca00640 1 --2- 172.21.15.131:0/1803544045 >> [v2:172.21.15.131:6834/650770835,v1:172.21.15.131:6836/650770835] conn(0x55b61b1d4da0 0x55b61b152d80 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c048117ff5cca313 server_cookie=36b62e728a6ec004 in_seq=0 out_seq=0 2024-05-26T22:50:50.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.607+0000 7fc085600640 10 client.5931 ms_handle_connect on v2:172.21.15.131:6834/650770835 2024-05-26T22:50:50.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.607+0000 7fc085600640 1 -- 172.21.15.131:0/1803544045 <== mds.0 v2:172.21.15.131:6834/650770835 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x55b61b07c090 con 0x55b61b1d4da0 2024-05-26T22:50:50.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.607+0000 7fc085600640 10 client.5931 handle_command_reply: tid=0 2024-05-26T22:50:50.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.607+0000 7fc0a221b740 1 client.5931 shutdown 2024-05-26T22:50:50.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.607+0000 7fc0a221b740 1 -- 172.21.15.131:0/1803544045 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b61b13d280 msgr2=0x55b61b1389a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:50.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.607+0000 7fc0a221b740 1 --2- 172.21.15.131:0/1803544045 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b61b13d280 0x55b61b1389a0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fc0800792b0 tx=0x7fc080075bf0 comp rx=0 tx=0).stop 2024-05-26T22:50:50.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.607+0000 7fc0a221b740 1 -- 172.21.15.131:0/1803544045 shutdown_connections 2024-05-26T22:50:50.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.607+0000 7fc0a221b740 1 --2- 172.21.15.131:0/1803544045 >> [v2:172.21.15.131:6834/650770835,v1:172.21.15.131:6836/650770835] conn(0x55b61b1d4da0 0x55b61b152d80 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:50.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.607+0000 7fc0a221b740 1 --2- 172.21.15.131:0/1803544045 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55b61b159c70 0x55b61b156d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:50.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.607+0000 7fc0a221b740 1 --2- 172.21.15.131:0/1803544045 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b61b13d280 0x55b61b1389a0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:50.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.607+0000 7fc0a221b740 1 --2- 172.21.15.131:0/1803544045 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b61b13cd90 0x55b61b1d94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:50.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.607+0000 7fc085600640 0 client.5931 ms_handle_reset on v2:172.21.15.131:6834/650770835 2024-05-26T22:50:50.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.607+0000 7fc0a221b740 1 -- 172.21.15.131:0/1803544045 >> 172.21.15.131:0/1803544045 conn(0x55b61b13c650 msgr2=0x55b61b111310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:50.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.607+0000 7fc0a221b740 1 -- 172.21.15.131:0/1803544045 shutdown_connections 2024-05-26T22:50:50.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.607+0000 7fc0a221b740 1 -- 172.21.15.131:0/1803544045 wait complete. 2024-05-26T22:50:50.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.607+0000 7fc0a221b740 20 client.5931 trim_cache size 0 max 16384 2024-05-26T22:50:50.611 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-26T22:50:50.611 INFO:teuthology.orchestra.run.smithi131.stdout: "cluster_fsid": "46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f", 2024-05-26T22:50:50.611 INFO:teuthology.orchestra.run.smithi131.stdout: "whoami": 0, 2024-05-26T22:50:50.611 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 5637, 2024-05-26T22:50:50.611 INFO:teuthology.orchestra.run.smithi131.stdout: "want_state": "up:active", 2024-05-26T22:50:50.611 INFO:teuthology.orchestra.run.smithi131.stdout: "state": "up:active", 2024-05-26T22:50:50.611 INFO:teuthology.orchestra.run.smithi131.stdout: "fs_name": "cephfs", 2024-05-26T22:50:50.611 INFO:teuthology.orchestra.run.smithi131.stdout: "rank_uptime": 23.012755899999998, 2024-05-26T22:50:50.611 INFO:teuthology.orchestra.run.smithi131.stdout: "mdsmap_epoch": 33, 2024-05-26T22:50:50.611 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch": 44, 2024-05-26T22:50:50.612 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch_barrier": 44, 2024-05-26T22:50:50.612 INFO:teuthology.orchestra.run.smithi131.stdout: "uptime": 28.512229204 2024-05-26T22:50:50.612 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-26T22:50:50.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.608+0000 7fc09f400640 1 -- 172.21.15.131:0/4047150038 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc07406be30 msgr2=0x7fc07406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:50.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.608+0000 7fc09f400640 1 --2- 172.21.15.131:0/4047150038 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc07406be30 0x7fc07406e2f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fc088006fd0 tx=0x7fc088008040 comp rx=0 tx=0).stop 2024-05-26T22:50:50.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.608+0000 7fc09f400640 1 -- 172.21.15.131:0/4047150038 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc0980d0390 msgr2=0x7fc09814f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:50.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.608+0000 7fc09f400640 1 --2- 172.21.15.131:0/4047150038 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc0980d0390 0x7fc09814f250 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fc080000c00 tx=0x7fc0800046c0 comp rx=0 tx=0).stop 2024-05-26T22:50:50.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.608+0000 7fc09f400640 1 -- 172.21.15.131:0/4047150038 shutdown_connections 2024-05-26T22:50:50.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.608+0000 7fc09f400640 1 --2- 172.21.15.131:0/4047150038 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc07406be30 0x7fc07406e2f0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:50.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.608+0000 7fc09f400640 1 --2- 172.21.15.131:0/4047150038 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc098165e30 0x7fc0981625e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:50.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.608+0000 7fc09f400640 1 --2- 172.21.15.131:0/4047150038 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc0980d0390 0x7fc09814f250 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:50.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.608+0000 7fc09f400640 1 --2- 172.21.15.131:0/4047150038 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc0980af230 0x7fc098160eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:50.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.608+0000 7fc09f400640 1 -- 172.21.15.131:0/4047150038 >> 172.21.15.131:0/4047150038 conn(0x7fc09806bc90 msgr2=0x7fc0980718f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:50.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.608+0000 7fc09f400640 1 -- 172.21.15.131:0/4047150038 shutdown_connections 2024-05-26T22:50:50.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50.608+0000 7fc09f400640 1 -- 172.21.15.131:0/4047150038 wait complete. 2024-05-26T22:50:50.624 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-05-26T22:50:50.625 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-26T22:50:50.625 DEBUG:teuthology.orchestra.run.smithi131:> mkdir -p tree 2024-05-26T22:50:50.625 DEBUG:teuthology.orchestra.run.smithi131:> 2024-05-26T22:50:50.625 DEBUG:teuthology.orchestra.run.smithi131:> setfattr -n ceph.dir.pin.distributed -v 1 tree 2024-05-26T22:50:50.625 DEBUG:teuthology.orchestra.run.smithi131:> 2024-05-26T22:50:50.625 DEBUG:teuthology.orchestra.run.smithi131:> for ((i = 0; i < 100; i++)); do 2024-05-26T22:50:50.625 DEBUG:teuthology.orchestra.run.smithi131:> mkdir -p "tree/$i" 2024-05-26T22:50:50.626 DEBUG:teuthology.orchestra.run.smithi131:> echo file > "tree/$i/file" 2024-05-26T22:50:50.626 DEBUG:teuthology.orchestra.run.smithi131:> done 2024-05-26T22:50:50.626 DEBUG:teuthology.orchestra.run.smithi131:> ') 2024-05-26T22:50:50.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:50 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-26T22:50:50.724 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree 2024-05-26T22:50:50.731 INFO:teuthology.orchestra.run.smithi131.stderr:+ setfattr -n ceph.dir.pin.distributed -v 1 tree 2024-05-26T22:50:50.740 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i = 0 )) 2024-05-26T22:50:50.741 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:50:50.741 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/0 2024-05-26T22:50:50.745 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:50:50.752 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:50:50.752 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:50:50.752 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/1 2024-05-26T22:50:50.758 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:50:50.760 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:50:50.760 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:50:50.760 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/2 2024-05-26T22:50:50.765 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:50:50.767 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:50:50.767 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:50:50.767 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/3 2024-05-26T22:50:50.772 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:50:50.775 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:50:50.775 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:50:50.775 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/4 2024-05-26T22:50:50.780 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:50:50.782 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:50:50.782 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:50:50.782 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/5 2024-05-26T22:50:50.787 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:50:50.789 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:50:50.790 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:50:50.790 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/6 2024-05-26T22:50:50.795 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:50:50.797 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:50:50.797 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:50:50.797 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/7 2024-05-26T22:50:50.803 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:50:50.805 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:50:50.805 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:50:50.805 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/8 2024-05-26T22:50:50.809 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:50:50.810 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:50:50.810 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:50:50.810 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/9 2024-05-26T22:50:50.814 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:50:50.816 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:50:50.816 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:50:50.816 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/10 2024-05-26T22:50:50.820 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:50:50.821 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:50:50.821 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:50:50.821 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/11 2024-05-26T22:50:50.826 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:50:50.827 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:50:50.827 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:50:50.828 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/12 2024-05-26T22:50:50.831 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:50:50.833 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:50:50.833 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:50:50.833 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/13 2024-05-26T22:50:50.839 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:50:50.840 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:50:50.841 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:50:50.841 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/14 2024-05-26T22:50:50.846 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:50:50.848 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:50:50.848 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:50:50.849 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/15 2024-05-26T22:50:50.853 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:50:50.855 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:50:50.855 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:50:50.855 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/16 2024-05-26T22:50:50.860 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:50:50.862 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:50:50.862 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:50:50.862 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/17 2024-05-26T22:50:50.867 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:50:50.869 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:50:50.869 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:50:50.869 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/18 2024-05-26T22:50:50.874 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:50:50.876 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:50:50.876 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:50:50.876 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/19 2024-05-26T22:50:50.881 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:50:50.883 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:50:50.883 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:50:50.883 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/20 2024-05-26T22:50:50.887 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:50:50.890 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:50:50.890 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:50:50.890 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/21 2024-05-26T22:50:50.895 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:50:50.898 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:50:50.898 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:50:50.898 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/22 2024-05-26T22:50:50.905 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:50:50.905 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:50:50.905 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:50:50.905 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/23 2024-05-26T22:50:50.909 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:50:50.911 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:50:50.911 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:50:50.911 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/24 2024-05-26T22:50:50.917 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:50:50.919 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:50:50.920 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:50:50.920 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/25 2024-05-26T22:50:50.926 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:50:50.928 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:50:50.928 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:50:50.929 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/26 2024-05-26T22:50:50.936 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:50:50.938 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:50:50.938 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:50:50.938 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/27 2024-05-26T22:50:50.944 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:50:50.946 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:50:50.946 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:50:50.946 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/28 2024-05-26T22:50:50.951 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:50:50.953 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:50:50.953 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:50:50.953 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/29 2024-05-26T22:50:50.959 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:50:50.960 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:50:50.960 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:50:50.960 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/30 2024-05-26T22:50:50.965 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:50:50.966 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:50:50.966 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:50:50.966 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/31 2024-05-26T22:50:50.971 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:50:50.973 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:50:50.973 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:50:50.973 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/32 2024-05-26T22:50:50.977 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:50:50.978 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:50:50.979 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:50:50.979 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/33 2024-05-26T22:50:50.982 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:50:50.984 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:50:50.984 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:50:50.984 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/34 2024-05-26T22:50:50.988 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:50:50.989 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:50:50.989 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:50:50.989 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/35 2024-05-26T22:50:50.993 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:50:50.994 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:50:50.994 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:50:50.995 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/36 2024-05-26T22:50:50.998 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:50:51.000 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:50:51.000 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:50:51.000 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/37 2024-05-26T22:50:51.004 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:50:51.006 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:50:51.006 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:50:51.006 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/38 2024-05-26T22:50:51.011 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:50:51.013 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:50:51.013 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:50:51.013 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/39 2024-05-26T22:50:51.018 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:50:51.020 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:50:51.020 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:50:51.020 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/40 2024-05-26T22:50:51.025 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:50:51.028 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:50:51.028 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:50:51.028 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/41 2024-05-26T22:50:51.033 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:50:51.035 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:50:51.035 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:50:51.035 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/42 2024-05-26T22:50:51.040 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:50:51.042 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:50:51.042 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:50:51.042 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/43 2024-05-26T22:50:51.047 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:50:51.049 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:50:51.049 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:50:51.049 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/44 2024-05-26T22:50:51.054 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:50:51.057 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:50:51.057 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:50:51.057 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/45 2024-05-26T22:50:51.062 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:50:51.064 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:50:51.065 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:50:51.065 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/46 2024-05-26T22:50:51.070 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:50:51.073 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:50:51.073 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:50:51.073 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/47 2024-05-26T22:50:51.079 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:50:51.081 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:50:51.081 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:50:51.081 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/48 2024-05-26T22:50:51.087 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:50:51.089 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:50:51.089 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:50:51.089 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/49 2024-05-26T22:50:51.095 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:50:51.097 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:50:51.097 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:50:51.097 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/50 2024-05-26T22:50:51.104 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:50:51.106 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:50:51.106 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:50:51.106 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/51 2024-05-26T22:50:51.112 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:50:51.114 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:50:51.114 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:50:51.115 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/52 2024-05-26T22:50:51.120 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:50:51.122 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:50:51.122 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:50:51.122 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/53 2024-05-26T22:50:51.128 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:50:51.130 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:50:51.130 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:50:51.130 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/54 2024-05-26T22:50:51.135 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:50:51.137 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:50:51.137 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:50:51.137 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/55 2024-05-26T22:50:51.141 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:50:51.143 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:50:51.143 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:50:51.143 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/56 2024-05-26T22:50:51.148 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:50:51.149 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:50:51.149 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:50:51.149 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/57 2024-05-26T22:50:51.154 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:50:51.155 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:50:51.155 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:50:51.155 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/58 2024-05-26T22:50:51.159 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:50:51.160 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:50:51.160 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:50:51.160 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/59 2024-05-26T22:50:51.164 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:50:51.165 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:50:51.165 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:50:51.166 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/60 2024-05-26T22:50:51.169 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:50:51.171 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:50:51.171 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:50:51.171 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/61 2024-05-26T22:50:51.177 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:50:51.178 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:50:51.178 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:50:51.178 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/62 2024-05-26T22:50:51.182 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:50:51.184 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:50:51.184 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:50:51.184 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/63 2024-05-26T22:50:51.188 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:50:51.190 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:50:51.190 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:50:51.190 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/64 2024-05-26T22:50:51.194 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:50:51.196 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:50:51.196 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:50:51.196 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/65 2024-05-26T22:50:51.201 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:50:51.202 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:50:51.202 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:50:51.202 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/66 2024-05-26T22:50:51.208 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:50:51.209 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:50:51.209 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:50:51.209 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/67 2024-05-26T22:50:51.213 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:50:51.215 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:50:51.215 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:50:51.215 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/68 2024-05-26T22:50:51.220 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:50:51.221 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:50:51.221 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:50:51.221 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/69 2024-05-26T22:50:51.226 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:50:51.227 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:50:51.227 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:50:51.228 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/70 2024-05-26T22:50:51.232 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:50:51.234 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:50:51.234 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:50:51.234 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/71 2024-05-26T22:50:51.240 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:50:51.241 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:50:51.241 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:50:51.241 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/72 2024-05-26T22:50:51.246 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:50:51.247 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:50:51.247 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:50:51.247 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/73 2024-05-26T22:50:51.252 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:50:51.254 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:50:51.254 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:50:51.255 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/74 2024-05-26T22:50:51.262 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:50:51.264 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:50:51.264 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:50:51.264 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/75 2024-05-26T22:50:51.269 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:50:51.272 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:50:51.272 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:50:51.272 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/76 2024-05-26T22:50:51.276 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:50:51.279 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:50:51.279 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:50:51.279 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/77 2024-05-26T22:50:51.286 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:50:51.286 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:50:51.287 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:50:51.287 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/78 2024-05-26T22:50:51.292 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:50:51.294 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:50:51.294 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:50:51.294 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/79 2024-05-26T22:50:51.299 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:50:51.301 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:50:51.301 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:50:51.301 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/80 2024-05-26T22:50:51.307 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:50:51.309 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:50:51.309 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:50:51.310 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/81 2024-05-26T22:50:51.314 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:50:51.315 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:50:51.315 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:50:51.315 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/82 2024-05-26T22:50:51.321 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:50:51.322 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:50:51.322 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:50:51.322 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/83 2024-05-26T22:50:51.327 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:50:51.329 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:50:51.329 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:50:51.329 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/84 2024-05-26T22:50:51.335 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:50:51.353 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:50:51.353 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:50:51.353 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/85 2024-05-26T22:50:51.359 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:50:51.361 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:50:51.361 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:50:51.361 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/86 2024-05-26T22:50:51.367 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:50:51.369 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:50:51.369 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:50:51.369 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/87 2024-05-26T22:50:51.375 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:50:51.377 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:50:51.377 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:50:51.377 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/88 2024-05-26T22:50:51.383 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:50:51.385 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:50:51.386 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:50:51.386 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/89 2024-05-26T22:50:51.392 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:50:51.394 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:50:51.394 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:50:51.395 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/90 2024-05-26T22:50:51.400 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:50:51.403 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:50:51.403 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:50:51.403 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/91 2024-05-26T22:50:51.408 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:50:51.411 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:50:51.411 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:50:51.411 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/92 2024-05-26T22:50:51.417 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:50:51.419 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:50:51.420 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:50:51.420 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/93 2024-05-26T22:50:51.425 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:50:51.427 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:50:51.427 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:50:51.427 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/94 2024-05-26T22:50:51.432 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:50:51.435 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:50:51.435 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:50:51.435 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/95 2024-05-26T22:50:51.440 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:50:51.443 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:50:51.443 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:50:51.443 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/96 2024-05-26T22:50:51.449 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:50:51.451 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:50:51.451 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:50:51.451 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/97 2024-05-26T22:50:51.457 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:50:51.459 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:50:51.459 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:50:51.459 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/98 2024-05-26T22:50:51.465 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:50:51.467 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:50:51.467 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:50:51.468 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/99 2024-05-26T22:50:51.473 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:50:51.476 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:50:51.476 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:50:51.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-26T22:50:51.490 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json get subtrees 2024-05-26T22:50:51.753 INFO:teuthology.orchestra.run.smithi187.stdout:[{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds2","dirfrag":"0x102","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"2","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-05-26T22:50:51.754 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "2", "dirfrag": "0x102", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds2", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-05-26T22:50:51.754 DEBUG:tasks.cephfs.cephfs_test_case:[ { "is_auth": true, "auth_first": 2, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "~mds2", "dirfrag": "0x102", "snapid_first": 2, "projected_version": "1", "version": "1", "committing_version": "1", "committed_version": "1", "is_rep": false, "dir_auth": "2", "states": [ "auth", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "dirty": 0, "authpin": 0 }, "nref": 2 } } ] 2024-05-26T22:50:51.754 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json get subtrees 2024-05-26T22:50:51.947 INFO:teuthology.orchestra.run.smithi187.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-05-26T22:50:51.948 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-05-26T22:50:51.948 DEBUG:tasks.cephfs.cephfs_test_case:[ { "is_auth": true, "auth_first": 1, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "~mds1", "dirfrag": "0x101", "snapid_first": 2, "projected_version": "1", "version": "1", "committing_version": "1", "committed_version": "1", "is_rep": false, "dir_auth": "1", "states": [ "auth", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "dirty": 0, "authpin": 0 }, "nref": 2 } } ] 2024-05-26T22:50:51.948 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-05-26T22:50:52.206 INFO:teuthology.orchestra.run.smithi131.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"2452","version":"2398","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":27,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1860","version":"1857","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":1,"authpin":1},"nref":4}}] 2024-05-26T22:50:52.207 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 27, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "2452", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "2398" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "~mds0", "pins": { "authpin": 1, "child": 1, "dirty": 1, "subtree": 1 }, "projected_version": "1860", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "1857" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-05-26T22:50:52.207 DEBUG:tasks.cephfs.cephfs_test_case:[ { "is_auth": true, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "", "dirfrag": "0x1", "snapid_first": 2, "projected_version": "2452", "version": "2398", "committing_version": "1", "committed_version": "1", "is_rep": true, "dir_auth": "0", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 27, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "replicated": 0, "dirty": 1, "authpin": 1 }, "nref": 4 } }, { "is_auth": true, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "~mds0", "dirfrag": "0x100", "snapid_first": 2, "projected_version": "1860", "version": "1857", "committing_version": "1", "committed_version": "1", "is_rep": false, "dir_auth": "0", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 1, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "dirty": 1, "authpin": 1 }, "nref": 4 } } ] 2024-05-26T22:50:52.207 INFO:tasks.cephfs.cephfs_test_case:len=0 [] 2024-05-26T22:50:52.515 INFO:tasks.ceph.mds.c.smithi131.stderr:2024-05-26T22:50:52.514+0000 7efef9c00640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-05-26T22:50:57.208 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json get subtrees 2024-05-26T22:50:57.424 INFO:teuthology.orchestra.run.smithi187.stdout:[{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds2","dirfrag":"0x102","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"2","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"2460","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":2},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/tree","dirfrag":"0x100000003f4.100*","snapid_first":2,"projected_version":"426","version":"426","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"2","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/tree","dirfrag":"0x100000003f4.001*","snapid_first":2,"projected_version":"428","version":"428","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"2","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2024-05-26T22:50:57.424 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "2", "dirfrag": "0x102", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds2", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 2 }, "snapid_first": 2, "states": [], "version": "2460" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "401", "committing_version": "401", "dir_auth": "2", "dirfrag": "0x100000003f4.100*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/tree", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "426", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "426" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 2, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "401", "committing_version": "401", "dir_auth": "2", "dirfrag": "0x100000003f4.001*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/tree", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "428", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "428" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 2, "is_auth": true, "random_ephemeral_pin": false } ] 2024-05-26T22:50:57.425 DEBUG:tasks.cephfs.cephfs_test_case:[ { "is_auth": true, "auth_first": 2, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "~mds2", "dirfrag": "0x102", "snapid_first": 2, "projected_version": "1", "version": "1", "committing_version": "1", "committed_version": "1", "is_rep": false, "dir_auth": "2", "states": [ "auth", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "dirty": 0, "authpin": 0 }, "nref": 2 } }, { "is_auth": false, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "", "dirfrag": "0x1", "snapid_first": 2, "projected_version": "0", "version": "2460", "committing_version": "0", "committed_version": "0", "is_rep": true, "dir_auth": "0", "states": [], "is_auth": false, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 2 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1 }, "nref": 2 } }, { "is_auth": true, "auth_first": 2, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 2, "dir": { "path": "/tree", "dirfrag": "0x100000003f4.100*", "snapid_first": 2, "projected_version": "426", "version": "426", "committing_version": "401", "committed_version": "401", "is_rep": false, "dir_auth": "2", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": { "0": 1 } }, "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "child": 1, "frozen": 0, "subtree": 1, "importing": 0, "replicated": 1, "dirty": 1 }, "nref": 4 } }, { "is_auth": true, "auth_first": 2, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 2, "dir": { "path": "/tree", "dirfrag": "0x100000003f4.001*", "snapid_first": 2, "projected_version": "428", "version": "428", "committing_version": "401", "committed_version": "401", "is_rep": false, "dir_auth": "2", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": { "0": 1 } }, "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "child": 1, "frozen": 0, "subtree": 1, "importing": 0, "replicated": 1, "dirty": 1 }, "nref": 4 } } ] 2024-05-26T22:50:57.425 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json get subtrees 2024-05-26T22:50:57.645 INFO:teuthology.orchestra.run.smithi187.stdout:[{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"2460","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":3},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree","dirfrag":"0x100000003f4.111*","snapid_first":2,"projected_version":"416","version":"416","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree","dirfrag":"0x100000003f4.110*","snapid_first":2,"projected_version":"442","version":"442","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree","dirfrag":"0x100000003f4.101*","snapid_first":2,"projected_version":"422","version":"422","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-05-26T22:50:57.645 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 3 }, "snapid_first": 2, "states": [], "version": "2460" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "401", "committing_version": "401", "dir_auth": "1", "dirfrag": "0x100000003f4.111*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/tree", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "416", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "416" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "401", "committing_version": "401", "dir_auth": "1", "dirfrag": "0x100000003f4.110*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/tree", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "442", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "442" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "401", "committing_version": "401", "dir_auth": "1", "dirfrag": "0x100000003f4.101*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/tree", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "422", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "422" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-05-26T22:50:57.646 DEBUG:tasks.cephfs.cephfs_test_case:[ { "is_auth": false, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "", "dirfrag": "0x1", "snapid_first": 2, "projected_version": "0", "version": "2460", "committing_version": "0", "committed_version": "0", "is_rep": true, "dir_auth": "0", "states": [], "is_auth": false, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 3 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1 }, "nref": 2 } }, { "is_auth": true, "auth_first": 1, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 1, "dir": { "path": "/tree", "dirfrag": "0x100000003f4.111*", "snapid_first": 2, "projected_version": "416", "version": "416", "committing_version": "401", "committed_version": "401", "is_rep": false, "dir_auth": "1", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": { "0": 1 } }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "child": 1, "frozen": 0, "subtree": 1, "importing": 0, "replicated": 1, "dirty": 1 }, "nref": 4 } }, { "is_auth": true, "auth_first": 1, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 1, "dir": { "path": "/tree", "dirfrag": "0x100000003f4.110*", "snapid_first": 2, "projected_version": "442", "version": "442", "committing_version": "401", "committed_version": "401", "is_rep": false, "dir_auth": "1", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": { "0": 1 } }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "child": 1, "frozen": 0, "subtree": 1, "importing": 0, "replicated": 1, "dirty": 1 }, "nref": 4 } }, { "is_auth": true, "auth_first": 1, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 1, "dir": { "path": "/tree", "dirfrag": "0x100000003f4.101*", "snapid_first": 2, "projected_version": "422", "version": "422", "committing_version": "401", "committed_version": "401", "is_rep": false, "dir_auth": "1", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": { "0": 1 } }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "child": 1, "frozen": 0, "subtree": 1, "importing": 0, "replicated": 1, "dirty": 1 }, "nref": 4 } }, { "is_auth": true, "auth_first": 1, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "~mds1", "dirfrag": "0x101", "snapid_first": 2, "projected_version": "1", "version": "1", "committing_version": "1", "committed_version": "1", "is_rep": false, "dir_auth": "1", "states": [ "auth", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "dirty": 0, "authpin": 0 }, "nref": 2 } } ] 2024-05-26T22:50:57.646 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-05-26T22:50:57.880 INFO:teuthology.orchestra.run.smithi131.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"2460","version":"2460","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":3,"2":2}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1860","version":"1860","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/tree","dirfrag":"0x100000003f4.000*","snapid_first":2,"projected_version":"426","version":"426","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"frozen":0,"subtree":1,"dirty":1,"waiter":0,"authpin":0},"nref":3}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/tree","dirfrag":"0x100000003f4.001*","snapid_first":2,"projected_version":"428","version":"428","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/tree","dirfrag":"0x100000003f4.010*","snapid_first":2,"projected_version":"426","version":"426","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"frozen":0,"subtree":1,"dirty":1,"waiter":0,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/tree","dirfrag":"0x100000003f4.011*","snapid_first":2,"projected_version":"430","version":"430","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"frozen":0,"subtree":1,"dirty":1,"waiter":0,"authpin":0},"nref":3}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/tree","dirfrag":"0x100000003f4.100*","snapid_first":2,"projected_version":"426","version":"426","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree","dirfrag":"0x100000003f4.101*","snapid_first":2,"projected_version":"422","version":"422","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree","dirfrag":"0x100000003f4.110*","snapid_first":2,"projected_version":"442","version":"442","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree","dirfrag":"0x100000003f4.111*","snapid_first":2,"projected_version":"416","version":"416","committing_version":"401","committed_version":"401","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}}] 2024-05-26T22:50:57.882 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 3, "2": 2 } }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "2460", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "2460" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 1, "subtree": 1 }, "projected_version": "1860", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "1860" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "401", "committing_version": "401", "dir_auth": "0", "dirfrag": "0x100000003f4.000*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "/tree", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "subtree": 1, "waiter": 0 }, "projected_version": "426", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "426" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "401", "committing_version": "401", "dir_auth": "2", "dirfrag": "0x100000003f4.001*", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "428", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "428" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 2, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "401", "committing_version": "401", "dir_auth": "0", "dirfrag": "0x100000003f4.010*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "/tree", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "subtree": 1, "waiter": 0 }, "projected_version": "426", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "426" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "401", "committing_version": "401", "dir_auth": "0", "dirfrag": "0x100000003f4.011*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "/tree", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "subtree": 1, "waiter": 0 }, "projected_version": "430", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "430" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "401", "committing_version": "401", "dir_auth": "2", "dirfrag": "0x100000003f4.100*", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "426", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "426" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 2, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "401", "committing_version": "401", "dir_auth": "1", "dirfrag": "0x100000003f4.101*", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "422", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "422" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "401", "committing_version": "401", "dir_auth": "1", "dirfrag": "0x100000003f4.110*", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "442", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "442" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "401", "committing_version": "401", "dir_auth": "1", "dirfrag": "0x100000003f4.111*", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "416", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "416" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false } ] 2024-05-26T22:50:57.882 DEBUG:tasks.cephfs.cephfs_test_case:[ { "is_auth": true, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "", "dirfrag": "0x1", "snapid_first": 2, "projected_version": "2460", "version": "2460", "committing_version": "1", "committed_version": "1", "is_rep": true, "dir_auth": "0", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": { "1": 3, "2": 2 } }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "replicated": 1, "dirty": 1, "authpin": 0 }, "nref": 4 } }, { "is_auth": true, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "~mds0", "dirfrag": "0x100", "snapid_first": 2, "projected_version": "1860", "version": "1860", "committing_version": "1", "committed_version": "1", "is_rep": false, "dir_auth": "0", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "dirty": 1, "authpin": 0 }, "nref": 3 } }, { "is_auth": true, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 0, "dir": { "path": "/tree", "dirfrag": "0x100000003f4.000*", "snapid_first": 2, "projected_version": "426", "version": "426", "committing_version": "401", "committed_version": "401", "is_rep": false, "dir_auth": "0", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "frozen": 0, "subtree": 1, "dirty": 1, "waiter": 0, "authpin": 0 }, "nref": 3 } }, { "is_auth": false, "auth_first": 2, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 2, "dir": { "path": "/tree", "dirfrag": "0x100000003f4.001*", "snapid_first": 2, "projected_version": "428", "version": "428", "committing_version": "401", "committed_version": "401", "is_rep": false, "dir_auth": "2", "states": [], "is_auth": false, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "request": 0, "child": 0, "frozen": 0, "subtree": 1, "replicated": 0, "dirty": 0, "waiter": 0, "authpin": 0, "tempexporting": 0 }, "nref": 1 } }, { "is_auth": true, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 0, "dir": { "path": "/tree", "dirfrag": "0x100000003f4.010*", "snapid_first": 2, "projected_version": "426", "version": "426", "committing_version": "401", "committed_version": "401", "is_rep": false, "dir_auth": "0", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "frozen": 0, "subtree": 1, "dirty": 1, "waiter": 0, "authpin": 0 }, "nref": 3 } }, { "is_auth": true, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 0, "dir": { "path": "/tree", "dirfrag": "0x100000003f4.011*", "snapid_first": 2, "projected_version": "430", "version": "430", "committing_version": "401", "committed_version": "401", "is_rep": false, "dir_auth": "0", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "frozen": 0, "subtree": 1, "dirty": 1, "waiter": 0, "authpin": 0 }, "nref": 3 } }, { "is_auth": false, "auth_first": 2, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 2, "dir": { "path": "/tree", "dirfrag": "0x100000003f4.100*", "snapid_first": 2, "projected_version": "426", "version": "426", "committing_version": "401", "committed_version": "401", "is_rep": false, "dir_auth": "2", "states": [], "is_auth": false, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "request": 0, "child": 0, "frozen": 0, "subtree": 1, "replicated": 0, "dirty": 0, "waiter": 0, "authpin": 0, "tempexporting": 0 }, "nref": 1 } }, { "is_auth": false, "auth_first": 1, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 1, "dir": { "path": "/tree", "dirfrag": "0x100000003f4.101*", "snapid_first": 2, "projected_version": "422", "version": "422", "committing_version": "401", "committed_version": "401", "is_rep": false, "dir_auth": "1", "states": [], "is_auth": false, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "request": 0, "child": 0, "frozen": 0, "subtree": 1, "replicated": 0, "dirty": 0, "waiter": 0, "authpin": 0, "tempexporting": 0 }, "nref": 1 } }, { "is_auth": false, "auth_first": 1, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 1, "dir": { "path": "/tree", "dirfrag": "0x100000003f4.110*", "snapid_first": 2, "projected_version": "442", "version": "442", "committing_version": "401", "committed_version": "401", "is_rep": false, "dir_auth": "1", "states": [], "is_auth": false, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "request": 0, "child": 0, "frozen": 0, "subtree": 1, "replicated": 0, "dirty": 0, "waiter": 0, "authpin": 0, "tempexporting": 0 }, "nref": 1 } }, { "is_auth": false, "auth_first": 1, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 1, "dir": { "path": "/tree", "dirfrag": "0x100000003f4.111*", "snapid_first": 2, "projected_version": "416", "version": "416", "committing_version": "401", "committed_version": "401", "is_rep": false, "dir_auth": "1", "states": [], "is_auth": false, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "request": 0, "child": 0, "frozen": 0, "subtree": 1, "replicated": 0, "dirty": 0, "waiter": 0, "authpin": 0, "tempexporting": 0 }, "nref": 1 } } ] 2024-05-26T22:50:57.883 INFO:tasks.cephfs.cephfs_test_case:len=8 [ { "is_auth": true, "auth_first": 2, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 2, "dir": { "path": "/tree", "dirfrag": "0x100000003f4.100*", "snapid_first": 2, "projected_version": "426", "version": "426", "committing_version": "401", "committed_version": "401", "is_rep": false, "dir_auth": "2", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": { "0": 1 } }, "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "child": 1, "frozen": 0, "subtree": 1, "importing": 0, "replicated": 1, "dirty": 1 }, "nref": 4 } }, { "is_auth": true, "auth_first": 2, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 2, "dir": { "path": "/tree", "dirfrag": "0x100000003f4.001*", "snapid_first": 2, "projected_version": "428", "version": "428", "committing_version": "401", "committed_version": "401", "is_rep": false, "dir_auth": "2", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": { "0": 1 } }, "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "child": 1, "frozen": 0, "subtree": 1, "importing": 0, "replicated": 1, "dirty": 1 }, "nref": 4 } }, { "is_auth": true, "auth_first": 1, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 1, "dir": { "path": "/tree", "dirfrag": "0x100000003f4.111*", "snapid_first": 2, "projected_version": "416", "version": "416", "committing_version": "401", "committed_version": "401", "is_rep": false, "dir_auth": "1", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": { "0": 1 } }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "child": 1, "frozen": 0, "subtree": 1, "importing": 0, "replicated": 1, "dirty": 1 }, "nref": 4 } }, { "is_auth": true, "auth_first": 1, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 1, "dir": { "path": "/tree", "dirfrag": "0x100000003f4.110*", "snapid_first": 2, "projected_version": "442", "version": "442", "committing_version": "401", "committed_version": "401", "is_rep": false, "dir_auth": "1", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": { "0": 1 } }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "child": 1, "frozen": 0, "subtree": 1, "importing": 0, "replicated": 1, "dirty": 1 }, "nref": 4 } }, { "is_auth": true, "auth_first": 1, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 1, "dir": { "path": "/tree", "dirfrag": "0x100000003f4.101*", "snapid_first": 2, "projected_version": "422", "version": "422", "committing_version": "401", "committed_version": "401", "is_rep": false, "dir_auth": "1", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": { "0": 1 } }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "child": 1, "frozen": 0, "subtree": 1, "importing": 0, "replicated": 1, "dirty": 1 }, "nref": 4 } }, { "is_auth": true, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 0, "dir": { "path": "/tree", "dirfrag": "0x100000003f4.000*", "snapid_first": 2, "projected_version": "426", "version": "426", "committing_version": "401", "committed_version": "401", "is_rep": false, "dir_auth": "0", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "frozen": 0, "subtree": 1, "dirty": 1, "waiter": 0, "authpin": 0 }, "nref": 3 } }, { "is_auth": true, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 0, "dir": { "path": "/tree", "dirfrag": "0x100000003f4.010*", "snapid_first": 2, "projected_version": "426", "version": "426", "committing_version": "401", "committed_version": "401", "is_rep": false, "dir_auth": "0", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "frozen": 0, "subtree": 1, "dirty": 1, "waiter": 0, "authpin": 0 }, "nref": 3 } }, { "is_auth": true, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 0, "dir": { "path": "/tree", "dirfrag": "0x100000003f4.011*", "snapid_first": 2, "projected_version": "430", "version": "430", "committing_version": "401", "committed_version": "401", "is_rep": false, "dir_auth": "0", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "frozen": 0, "subtree": 1, "dirty": 1, "waiter": 0, "authpin": 0 }, "nref": 3 } } ] 2024-05-26T22:50:57.883 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-26T22:50:58.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.077+0000 7f6ba2000640 1 Processor -- start 2024-05-26T22:50:58.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.078+0000 7f6ba2000640 1 -- start start 2024-05-26T22:50:58.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.079+0000 7f6ba2000640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f6b9c0cc7f0 0x7f6b9c0ccc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:58.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.079+0000 7f6ba2000640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b9c160620 con 0x7f6b9c06e8c0 2024-05-26T22:50:58.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.079+0000 7f6ba2000640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b9c160900 con 0x7f6b9c156750 2024-05-26T22:50:58.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.079+0000 7f6ba2000640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f6b9c15d770 con 0x7f6b9c0cc7f0 2024-05-26T22:50:58.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.080+0000 7f6b9be00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f6b9c06e8c0 0x7f6b9c15d060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:44794/0 (socket says 172.21.15.131:44794) 2024-05-26T22:50:58.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.080+0000 7f6b9be00640 1 -- 172.21.15.131:0/2064131411 learned_addr learned my addr 172.21.15.131:0/2064131411 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:50:58.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.080+0000 7f6ba1600640 1 --2- 172.21.15.131:0/2064131411 >> v2:172.21.15.187:3300/0 conn(0x7f6b9c0cc7f0 0x7f6b9c0ccc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:58.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.080+0000 7f6b9b400640 1 -- 172.21.15.131:0/2064131411 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1472424667 0 0) 0x7f6b9c160620 con 0x7f6b9c06e8c0 2024-05-26T22:50:58.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.080+0000 7f6b9b400640 1 -- 172.21.15.131:0/2064131411 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6b84003520 con 0x7f6b9c06e8c0 2024-05-26T22:50:58.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.081+0000 7f6b9b400640 1 -- 172.21.15.131:0/2064131411 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 914732368 0 0) 0x7f6b84003520 con 0x7f6b9c06e8c0 2024-05-26T22:50:58.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.081+0000 7f6b9b400640 1 -- 172.21.15.131:0/2064131411 >> v2:172.21.15.187:3300/0 conn(0x7f6b9c0cc7f0 msgr2=0x7f6b9c0ccc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:58.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.081+0000 7f6b9b400640 1 --2- 172.21.15.131:0/2064131411 >> v2:172.21.15.187:3300/0 conn(0x7f6b9c0cc7f0 0x7f6b9c0ccc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:58.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.081+0000 7f6b9b400640 1 -- 172.21.15.131:0/2064131411 >> v1:172.21.15.187:6789/0 conn(0x7f6b9c156750 legacy=0x7f6b9c15fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:58.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.081+0000 7f6b9b400640 1 -- 172.21.15.131:0/2064131411 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b9c160be0 con 0x7f6b9c06e8c0 2024-05-26T22:50:58.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.081+0000 7f6b9b400640 1 -- 172.21.15.131:0/2064131411 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7f6b88001670 con 0x7f6b9c06e8c0 2024-05-26T22:50:58.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.081+0000 7f6b9b400640 1 -- 172.21.15.131:0/2064131411 >> v1:172.21.15.131:6789/0 conn(0x7f6b9c06e8c0 legacy=0x7f6b9c15d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:58.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.081+0000 7f6b9b400640 1 --2- 172.21.15.131:0/2064131411 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6b84003a00 0x7f6b84003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:58.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.081+0000 7f6b9b400640 1 --2- 172.21.15.131:0/2064131411 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6b84004800 0x7f6b84004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:58.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.081+0000 7f6b9b400640 1 --2- 172.21.15.131:0/2064131411 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6b840051c0 0x7f6b8400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:58.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.081+0000 7f6b9b400640 1 -- 172.21.15.131:0/2064131411 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f6b9c15d770 con 0x7f6b84003a00 2024-05-26T22:50:58.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.081+0000 7f6b9b400640 1 -- 172.21.15.131:0/2064131411 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6b8400c0c0 con 0x7f6b840051c0 2024-05-26T22:50:58.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.081+0000 7f6b9b400640 1 -- 172.21.15.131:0/2064131411 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f6b8400c2c0 con 0x7f6b84004800 2024-05-26T22:50:58.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.081+0000 7f6ba0c00640 1 --2- 172.21.15.131:0/2064131411 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6b84003a00 0x7f6b84003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:58.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.082+0000 7f6b9be00640 1 --2- 172.21.15.131:0/2064131411 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6b84004800 0x7f6b84004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:58.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.082+0000 7f6ba1600640 1 --2- 172.21.15.131:0/2064131411 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6b840051c0 0x7f6b8400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:58.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.082+0000 7f6ba0c00640 1 -- 172.21.15.131:0/2064131411 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6b84004800 msgr2=0x7f6b84004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:58.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.082+0000 7f6ba0c00640 1 --2- 172.21.15.131:0/2064131411 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6b84004800 0x7f6b84004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:58.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.082+0000 7f6ba0c00640 1 -- 172.21.15.131:0/2064131411 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6b840051c0 msgr2=0x7f6b8400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:58.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.082+0000 7f6ba0c00640 1 --2- 172.21.15.131:0/2064131411 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6b840051c0 0x7f6b8400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:58.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.082+0000 7f6ba0c00640 1 -- 172.21.15.131:0/2064131411 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b84003520 con 0x7f6b84003a00 2024-05-26T22:50:58.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.082+0000 7f6ba1600640 1 -- 172.21.15.131:0/2064131411 reap_dead start 2024-05-26T22:50:58.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.083+0000 7f6ba0c00640 1 --2- 172.21.15.131:0/2064131411 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6b84003a00 0x7f6b84003e20 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f6b90001750 tx=0x7f6b84005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73cf33468123a4ea server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:58.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.083+0000 7f6b9b400640 1 -- 172.21.15.131:0/2064131411 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b90001370 con 0x7f6b84003a00 2024-05-26T22:50:58.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.083+0000 7f6b9b400640 1 -- 172.21.15.131:0/2064131411 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b900052d0 con 0x7f6b84003a00 2024-05-26T22:50:58.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.083+0000 7f6b9b400640 1 -- 172.21.15.131:0/2064131411 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b9002abc0 con 0x7f6b84003a00 2024-05-26T22:50:58.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.084+0000 7f6ba2000640 1 -- 172.21.15.131:0/2064131411 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6b84003a00 msgr2=0x7f6b84003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:58.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.084+0000 7f6ba2000640 1 --2- 172.21.15.131:0/2064131411 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6b84003a00 0x7f6b84003e20 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f6b90001750 tx=0x7f6b84005830 comp rx=0 tx=0).stop 2024-05-26T22:50:58.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.084+0000 7f6ba2000640 1 -- 172.21.15.131:0/2064131411 shutdown_connections 2024-05-26T22:50:58.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.084+0000 7f6ba2000640 1 --2- 172.21.15.131:0/2064131411 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6b84003a00 0x7f6b84003e20 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:58.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.084+0000 7f6ba2000640 1 -- 172.21.15.131:0/2064131411 >> 172.21.15.131:0/2064131411 conn(0x7f6b9c0cff40 msgr2=0x7f6b9c0d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:58.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.084+0000 7f6ba2000640 1 -- 172.21.15.131:0/2064131411 shutdown_connections 2024-05-26T22:50:58.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.084+0000 7f6ba2000640 1 -- 172.21.15.131:0/2064131411 wait complete. 2024-05-26T22:50:58.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.086+0000 7f6ba2000640 1 Processor -- start 2024-05-26T22:50:58.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.086+0000 7f6ba2000640 1 -- start start 2024-05-26T22:50:58.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.087+0000 7f6ba2000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6b84003a00 0x7f6b9c14d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:58.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.087+0000 7f6ba2000640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6b9c0b4b20 0x7f6b9c0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:58.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.087+0000 7f6ba2000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6b9c0afcb0 0x7f6b9c0b0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:58.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.087+0000 7f6ba2000640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f6b9c15cbd0 con 0x7f6b84003a00 2024-05-26T22:50:58.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.087+0000 7f6ba2000640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6b9c0cd1a0 con 0x7f6b9c0b4b20 2024-05-26T22:50:58.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.087+0000 7f6ba2000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f6b9c156b30 con 0x7f6b9c0afcb0 2024-05-26T22:50:58.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.087+0000 7f6ba0c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6b84003a00 0x7f6b9c14d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:58.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.087+0000 7f6ba0c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6b84003a00 0x7f6b9c14d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54986/0 (socket says 172.21.15.131:54986) 2024-05-26T22:50:58.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.087+0000 7f6ba0c00640 1 -- 172.21.15.131:0/1132719721 learned_addr learned my addr 172.21.15.131:0/1132719721 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:58.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.088+0000 7f6b9be00640 1 --2- 172.21.15.131:0/1132719721 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6b9c0b4b20 0x7f6b9c0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:58.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.088+0000 7f6ba1600640 1 --2- 172.21.15.131:0/1132719721 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6b9c0afcb0 0x7f6b9c0b0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:58.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.088+0000 7f6ba0c00640 1 -- 172.21.15.131:0/1132719721 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6b9c0afcb0 msgr2=0x7f6b9c0b0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:58.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.088+0000 7f6ba0c00640 1 --2- 172.21.15.131:0/1132719721 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6b9c0afcb0 0x7f6b9c0b0130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:58.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.088+0000 7f6ba0c00640 1 -- 172.21.15.131:0/1132719721 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6b9c0b4b20 msgr2=0x7f6b9c0b4f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:58.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.088+0000 7f6ba0c00640 1 --2- 172.21.15.131:0/1132719721 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6b9c0b4b20 0x7f6b9c0b4f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:58.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.088+0000 7f6ba0c00640 1 -- 172.21.15.131:0/1132719721 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b9c0b0a20 con 0x7f6b84003a00 2024-05-26T22:50:58.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.088+0000 7f6b9be00640 1 --2- 172.21.15.131:0/1132719721 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6b9c0b4b20 0x7f6b9c0b4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:50:58.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.088+0000 7f6ba0c00640 1 --2- 172.21.15.131:0/1132719721 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6b84003a00 0x7f6b9c14d6f0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f6b90000c00 tx=0x7f6b90001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:58.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.088+0000 7f6b98c00640 1 -- 172.21.15.131:0/1132719721 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b90037040 con 0x7f6b84003a00 2024-05-26T22:50:58.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.088+0000 7f6b98c00640 1 -- 172.21.15.131:0/1132719721 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b90005480 con 0x7f6b84003a00 2024-05-26T22:50:58.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.088+0000 7f6b98c00640 1 -- 172.21.15.131:0/1132719721 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b90001370 con 0x7f6b84003a00 2024-05-26T22:50:58.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.089+0000 7f6ba2000640 1 -- 172.21.15.131:0/1132719721 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b9c0b0d00 con 0x7f6b84003a00 2024-05-26T22:50:58.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.089+0000 7f6ba2000640 1 -- 172.21.15.131:0/1132719721 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b9c0b12c0 con 0x7f6b84003a00 2024-05-26T22:50:58.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.090+0000 7f6ba2000640 1 -- 172.21.15.131:0/1132719721 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b9c0b1500 con 0x7f6b84003a00 2024-05-26T22:50:58.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.091+0000 7f6b98c00640 1 -- 172.21.15.131:0/1132719721 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6b90043020 con 0x7f6b84003a00 2024-05-26T22:50:58.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.092+0000 7f6b98c00640 1 --2- 172.21.15.131:0/1132719721 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f6b7c06be60 0x7f6b7c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:58.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.092+0000 7f6b98c00640 1 -- 172.21.15.131:0/1132719721 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f6b900aa5d0 con 0x7f6b84003a00 2024-05-26T22:50:58.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.092+0000 7f6b9be00640 1 --2- 172.21.15.131:0/1132719721 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f6b7c06be60 0x7f6b7c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:58.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.093+0000 7f6b9be00640 1 --2- 172.21.15.131:0/1132719721 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f6b7c06be60 0x7f6b7c06e320 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f6b88000c00 tx=0x7f6b88000ed0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:58.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.097+0000 7f6b98c00640 1 -- 172.21.15.131:0/1132719721 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f6b9c0b1500 con 0x7f6b84003a00 2024-05-26T22:50:58.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.339+0000 7f6ba2000640 1 -- 172.21.15.131:0/1132719721 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6b9c0694e0 con 0x7f6b84003a00 2024-05-26T22:50:58.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.340+0000 7f6b98c00640 1 -- 172.21.15.131:0/1132719721 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+5228 (secure 0 0 0) 0x7f6b90078e50 con 0x7f6b84003a00 2024-05-26T22:50:58.340 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:50:58.340 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":34,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4661,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/265040431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6835","nonce":265040431},{"type":"v1","addr":"172.21.15.131:6837","nonce":265040431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-26T22:50:27.001665+0000","modified":"2024-05-26T22:50:57.103687+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5637,"mds_1":4918,"mds_2":4670},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4670":{"gid":4670,"name":"b","rank":2,"incarnation":32,"state":"up:active","state_seq":8,"addr":"172.21.15.187:6834/1027783169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1027783169},{"type":"v1","addr":"172.21.15.187:6834","nonce":1027783169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4918":{"gid":4918,"name":"d","rank":1,"incarnation":30,"state":"up:active","state_seq":8,"addr":"172.21.15.187:6835/3429891101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6833","nonce":3429891101},{"type":"v1","addr":"172.21.15.187:6835","nonce":3429891101}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5637":{"gid":5637,"name":"c","rank":0,"incarnation":25,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6836/650770835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":650770835},{"type":"v1","addr":"172.21.15.131:6836","nonce":650770835}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5637,"qdb_cluster":[5637,4918,4670]},"id":3}]} 2024-05-26T22:50:58.340 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 34 2024-05-26T22:50:58.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.342+0000 7f6ba2000640 1 -- 172.21.15.131:0/1132719721 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f6b7c06be60 msgr2=0x7f6b7c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:58.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.342+0000 7f6ba2000640 1 --2- 172.21.15.131:0/1132719721 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f6b7c06be60 0x7f6b7c06e320 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f6b88000c00 tx=0x7f6b88000ed0 comp rx=0 tx=0).stop 2024-05-26T22:50:58.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.342+0000 7f6ba2000640 1 -- 172.21.15.131:0/1132719721 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6b84003a00 msgr2=0x7f6b9c14d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:58.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.342+0000 7f6ba2000640 1 --2- 172.21.15.131:0/1132719721 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6b84003a00 0x7f6b9c14d6f0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f6b90000c00 tx=0x7f6b90001120 comp rx=0 tx=0).stop 2024-05-26T22:50:58.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.342+0000 7f6ba2000640 1 -- 172.21.15.131:0/1132719721 shutdown_connections 2024-05-26T22:50:58.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.342+0000 7f6ba2000640 1 --2- 172.21.15.131:0/1132719721 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6b9c0afcb0 0x7f6b9c0b0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:58.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.343+0000 7f6ba2000640 1 --2- 172.21.15.131:0/1132719721 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f6b7c06be60 0x7f6b7c06e320 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:58.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.343+0000 7f6ba2000640 1 --2- 172.21.15.131:0/1132719721 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6b9c0b4b20 0x7f6b9c0b4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:58.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.343+0000 7f6ba2000640 1 --2- 172.21.15.131:0/1132719721 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6b84003a00 0x7f6b9c14d6f0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:58.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.343+0000 7f6ba2000640 1 -- 172.21.15.131:0/1132719721 >> 172.21.15.131:0/1132719721 conn(0x7f6b9c0cff40 msgr2=0x7f6b9c15a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:58.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.343+0000 7f6ba2000640 1 -- 172.21.15.131:0/1132719721 shutdown_connections 2024-05-26T22:50:58.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.343+0000 7f6ba2000640 1 -- 172.21.15.131:0/1132719721 wait complete. 2024-05-26T22:50:58.362 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:2 perf dump 2024-05-26T22:50:58.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.566+0000 7f0441600640 1 Processor -- start 2024-05-26T22:50:58.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.567+0000 7f0441600640 1 -- start start 2024-05-26T22:50:58.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.568+0000 7f0441600640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f043c15c220 0x7f043c06e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:58.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.568+0000 7f0441600640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f043c06e9f0 0x7f043c15d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:58.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.568+0000 7f0441600640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f043c06ee60 con 0x7f043c1554f0 2024-05-26T22:50:58.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.568+0000 7f0441600640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f043c1558d0 con 0x7f043c15c220 2024-05-26T22:50:58.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.568+0000 7f0441600640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f043c0729b0 con 0x7f043c06e9f0 2024-05-26T22:50:58.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.568+0000 7f0433400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f043c15c220 0x7f043c06e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:58.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.568+0000 7f0433400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f043c15c220 0x7f043c06e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54996/0 (socket says 172.21.15.131:54996) 2024-05-26T22:50:58.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.568+0000 7f0433400640 1 -- 172.21.15.131:0/3425114542 learned_addr learned my addr 172.21.15.131:0/3425114542 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:58.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.569+0000 7f0440c00640 1 --2- 172.21.15.131:0/3425114542 >> v2:172.21.15.187:3300/0 conn(0x7f043c06e9f0 0x7f043c15d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:58.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.569+0000 7f0433400640 1 -- 172.21.15.131:0/3425114542 >> v2:172.21.15.187:3300/0 conn(0x7f043c06e9f0 msgr2=0x7f043c15d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:58.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.569+0000 7f0433400640 1 --2- 172.21.15.131:0/3425114542 >> v2:172.21.15.187:3300/0 conn(0x7f043c06e9f0 0x7f043c15d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:58.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.569+0000 7f0433400640 1 -- 172.21.15.131:0/3425114542 >> v1:172.21.15.187:6789/0 conn(0x7f043c1554f0 legacy=0x7f043c071f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:58.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.569+0000 7f0433400640 1 -- 172.21.15.131:0/3425114542 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f043c15c750 con 0x7f043c15c220 2024-05-26T22:50:58.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.569+0000 7f0440c00640 1 --2- 172.21.15.131:0/3425114542 >> v2:172.21.15.187:3300/0 conn(0x7f043c06e9f0 0x7f043c15d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:50:58.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.570+0000 7f0433400640 1 --2- 172.21.15.131:0/3425114542 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f043c15c220 0x7f043c06e4b0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f0424009870 tx=0x7f042402a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26261bbc8736246b server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:58.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.570+0000 7f043b400640 1 -- 172.21.15.131:0/3425114542 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0424037070 con 0x7f043c15c220 2024-05-26T22:50:58.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.570+0000 7f043b400640 1 -- 172.21.15.131:0/3425114542 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0424032440 con 0x7f043c15c220 2024-05-26T22:50:58.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.570+0000 7f043b400640 1 -- 172.21.15.131:0/3425114542 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0424032810 con 0x7f043c15c220 2024-05-26T22:50:58.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.571+0000 7f0441600640 1 -- 172.21.15.131:0/3425114542 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f043c15c220 msgr2=0x7f043c06e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:58.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.571+0000 7f0441600640 1 --2- 172.21.15.131:0/3425114542 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f043c15c220 0x7f043c06e4b0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f0424009870 tx=0x7f042402a250 comp rx=0 tx=0).stop 2024-05-26T22:50:58.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.571+0000 7f0441600640 1 -- 172.21.15.131:0/3425114542 shutdown_connections 2024-05-26T22:50:58.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.571+0000 7f0441600640 1 --2- 172.21.15.131:0/3425114542 >> v2:172.21.15.187:3300/0 conn(0x7f043c06e9f0 0x7f043c15d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:58.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.571+0000 7f0441600640 1 --2- 172.21.15.131:0/3425114542 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f043c15c220 0x7f043c06e4b0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:58.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.571+0000 7f0441600640 1 -- 172.21.15.131:0/3425114542 >> 172.21.15.131:0/3425114542 conn(0x7f043c0cc7f0 msgr2=0x7f043c0ccbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:58.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.571+0000 7f0441600640 1 -- 172.21.15.131:0/3425114542 shutdown_connections 2024-05-26T22:50:58.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.571+0000 7f0441600640 1 -- 172.21.15.131:0/3425114542 wait complete. 2024-05-26T22:50:58.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.573+0000 7f0441600640 1 Processor -- start 2024-05-26T22:50:58.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.573+0000 7f0441600640 1 -- start start 2024-05-26T22:50:58.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.574+0000 7f0441600640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f043c06e9f0 0x7f043c152b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:58.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.574+0000 7f0441600640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f043c1554f0 0x7f043c0a9f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:58.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.574+0000 7f0441600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f043c15c220 0x7f043c0b4b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:58.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.574+0000 7f0441600640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f043c072670 con 0x7f043c15c220 2024-05-26T22:50:58.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.574+0000 7f0441600640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f043c15cd10 con 0x7f043c06e9f0 2024-05-26T22:50:58.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.574+0000 7f0441600640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f043c15d770 con 0x7f043c1554f0 2024-05-26T22:50:58.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.574+0000 7f0440c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f043c15c220 0x7f043c0b4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:58.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.574+0000 7f0440c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f043c15c220 0x7f043c0b4b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55012/0 (socket says 172.21.15.131:55012) 2024-05-26T22:50:58.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.574+0000 7f0440c00640 1 -- 172.21.15.131:0/1564922518 learned_addr learned my addr 172.21.15.131:0/1564922518 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:58.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.574+0000 7f043be00640 1 --2- 172.21.15.131:0/1564922518 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f043c06e9f0 0x7f043c152b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:58.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.574+0000 7f0433400640 1 --2- 172.21.15.131:0/1564922518 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f043c1554f0 0x7f043c0a9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:58.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.574+0000 7f0440c00640 1 -- 172.21.15.131:0/1564922518 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f043c1554f0 msgr2=0x7f043c0a9f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:58.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.574+0000 7f0440c00640 1 --2- 172.21.15.131:0/1564922518 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f043c1554f0 0x7f043c0a9f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:58.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.574+0000 7f0440c00640 1 -- 172.21.15.131:0/1564922518 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f043c06e9f0 msgr2=0x7f043c152b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:58.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.574+0000 7f0440c00640 1 --2- 172.21.15.131:0/1564922518 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f043c06e9f0 0x7f043c152b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:58.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.574+0000 7f0440c00640 1 -- 172.21.15.131:0/1564922518 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f043c1531e0 con 0x7f043c15c220 2024-05-26T22:50:58.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.575+0000 7f043be00640 1 --2- 172.21.15.131:0/1564922518 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f043c06e9f0 0x7f043c152b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:50:58.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.575+0000 7f0440c00640 1 --2- 172.21.15.131:0/1564922518 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f043c15c220 0x7f043c0b4b20 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f043400c7f0 tx=0x7f043400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:58.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.575+0000 7f0438c00640 1 -- 172.21.15.131:0/1564922518 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0434015070 con 0x7f043c15c220 2024-05-26T22:50:58.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.575+0000 7f0438c00640 1 -- 172.21.15.131:0/1564922518 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0434007820 con 0x7f043c15c220 2024-05-26T22:50:58.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.575+0000 7f0438c00640 1 -- 172.21.15.131:0/1564922518 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0434007c50 con 0x7f043c15c220 2024-05-26T22:50:58.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.575+0000 7f0441600640 1 -- 172.21.15.131:0/1564922518 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f043c1534c0 con 0x7f043c15c220 2024-05-26T22:50:58.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.575+0000 7f0441600640 1 -- 172.21.15.131:0/1564922518 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f043c153a80 con 0x7f043c15c220 2024-05-26T22:50:58.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.578+0000 7f0438c00640 1 -- 172.21.15.131:0/1564922518 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0434018070 con 0x7f043c15c220 2024-05-26T22:50:58.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.579+0000 7f0438c00640 1 --2- 172.21.15.131:0/1564922518 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f041806be30 0x7f041806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:58.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.579+0000 7f0438c00640 1 -- 172.21.15.131:0/1564922518 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f043408b030 con 0x7f043c15c220 2024-05-26T22:50:58.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.580+0000 7f043be00640 1 --2- 172.21.15.131:0/1564922518 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f041806be30 0x7f041806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:58.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.580+0000 7f0445b7c740 1 Processor -- start 2024-05-26T22:50:58.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.580+0000 7f0445b7c740 1 -- start start 2024-05-26T22:50:58.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.580+0000 7f043be00640 1 --2- 172.21.15.131:0/1564922518 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f041806be30 0x7f041806e2f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f042c004520 tx=0x7f042c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:58.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.581+0000 7f0445b7c740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55bae6762420 0x55bae6714db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:58.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.581+0000 7f0445b7c740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bae6745540 0x55bae67448c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:58.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.581+0000 7f0445b7c740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bae6744e00 0x55bae6719c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:58.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.581+0000 7f0445b7c740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55bae667d960 con 0x55bae6745540 2024-05-26T22:50:58.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.581+0000 7f0445b7c740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55bae66250d0 con 0x55bae6744e00 2024-05-26T22:50:58.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.581+0000 7f0445b7c740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x55bae6679920 con 0x55bae6762420 2024-05-26T22:50:58.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.581+0000 7f0433400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bae6745540 0x55bae67448c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:58.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.581+0000 7f0433400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bae6745540 0x55bae67448c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55022/0 (socket says 172.21.15.131:55022) 2024-05-26T22:50:58.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.581+0000 7f0433400640 1 -- 172.21.15.131:0/3041566738 learned_addr learned my addr 172.21.15.131:0/3041566738 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:58.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.582+0000 7f0440c00640 1 --2- 172.21.15.131:0/3041566738 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bae6744e00 0x55bae6719c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:58.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.582+0000 7f0433400640 1 -- 172.21.15.131:0/3041566738 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55bae6762420 msgr2=0x55bae6714db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:58.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.582+0000 7f043be00640 1 --2- 172.21.15.131:0/3041566738 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55bae6762420 0x55bae6714db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:58.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.582+0000 7f0433400640 1 --2- 172.21.15.131:0/3041566738 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55bae6762420 0x55bae6714db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:58.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.582+0000 7f0433400640 1 -- 172.21.15.131:0/3041566738 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bae6744e00 msgr2=0x55bae6719c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:58.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.582+0000 7f0433400640 1 --2- 172.21.15.131:0/3041566738 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bae6744e00 0x55bae6719c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:58.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.582+0000 7f0433400640 1 -- 172.21.15.131:0/3041566738 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bae645b7d0 con 0x55bae6745540 2024-05-26T22:50:58.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.582+0000 7f0433400640 1 --2- 172.21.15.131:0/3041566738 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bae6745540 0x55bae67448c0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f042402a290 tx=0x7f042402c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a306c17f9513c1b server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:58.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.583+0000 7f042be00640 1 -- 172.21.15.131:0/3041566738 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0424037040 con 0x55bae6745540 2024-05-26T22:50:58.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.583+0000 7f042be00640 1 -- 172.21.15.131:0/3041566738 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0424042070 con 0x55bae6745540 2024-05-26T22:50:58.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.583+0000 7f042be00640 1 -- 172.21.15.131:0/3041566738 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04240328c0 con 0x55bae6745540 2024-05-26T22:50:58.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.583+0000 7f0445b7c740 1 -- 172.21.15.131:0/3041566738 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bae6745540 msgr2=0x55bae67448c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:58.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.583+0000 7f0445b7c740 1 --2- 172.21.15.131:0/3041566738 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bae6745540 0x55bae67448c0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f042402a290 tx=0x7f042402c000 comp rx=0 tx=0).stop 2024-05-26T22:50:58.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.584+0000 7f0445b7c740 1 -- 172.21.15.131:0/3041566738 shutdown_connections 2024-05-26T22:50:58.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.584+0000 7f0445b7c740 1 --2- 172.21.15.131:0/3041566738 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bae6744e00 0x55bae6719c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:58.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.584+0000 7f0445b7c740 1 --2- 172.21.15.131:0/3041566738 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bae6745540 0x55bae67448c0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:58.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.584+0000 7f0445b7c740 1 --2- 172.21.15.131:0/3041566738 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55bae6762420 0x55bae6714db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:58.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.584+0000 7f0445b7c740 1 -- 172.21.15.131:0/3041566738 >> 172.21.15.131:0/3041566738 conn(0x55bae6717020 msgr2=0x55bae675f8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:58.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.584+0000 7f0445b7c740 1 -- 172.21.15.131:0/3041566738 shutdown_connections 2024-05-26T22:50:58.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.584+0000 7f0445b7c740 1 -- 172.21.15.131:0/3041566738 wait complete. 2024-05-26T22:50:58.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.584+0000 7f0445b7c740 1 -- start start 2024-05-26T22:50:58.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.585+0000 7f0445b7c740 1 Processor -- start 2024-05-26T22:50:58.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.586+0000 7f0445b7c740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bae6745540 0x55bae6761870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:58.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.586+0000 7f0445b7c740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bae6762420 0x55bae6747cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:58.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.586+0000 7f0445b7c740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55bae675e4c0 0x55bae675e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:58.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.586+0000 7f0445b7c740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55bae67191d0 con 0x55bae6745540 2024-05-26T22:50:58.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.586+0000 7f0445b7c740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55bae66384a0 con 0x55bae6762420 2024-05-26T22:50:58.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.586+0000 7f0445b7c740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x55bae67185c0 con 0x55bae675e4c0 2024-05-26T22:50:58.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.586+0000 7f0445b7c740 10 client.0 _ll_register_callbacks cb 0x55bae663fc40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-26T22:50:58.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.586+0000 7f043be00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bae6745540 0x55bae6761870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:58.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.586+0000 7f043be00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bae6745540 0x55bae6761870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55028/0 (socket says 172.21.15.131:55028) 2024-05-26T22:50:58.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.586+0000 7f043be00640 1 -- 172.21.15.131:0/2040671630 learned_addr learned my addr 172.21.15.131:0/2040671630 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:58.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.586+0000 7f0433400640 1 --2- 172.21.15.131:0/2040671630 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bae6762420 0x55bae6747cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:58.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.586+0000 7f043be00640 1 -- 172.21.15.131:0/2040671630 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55bae675e4c0 msgr2=0x55bae675e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:58.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.586+0000 7f0440c00640 1 --2- 172.21.15.131:0/2040671630 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55bae675e4c0 0x55bae675e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:58.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.586+0000 7f043be00640 1 --2- 172.21.15.131:0/2040671630 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55bae675e4c0 0x55bae675e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:58.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.587+0000 7f043be00640 1 -- 172.21.15.131:0/2040671630 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bae6762420 msgr2=0x55bae6747cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:58.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.587+0000 7f043be00640 1 --2- 172.21.15.131:0/2040671630 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bae6762420 0x55bae6747cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:58.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.587+0000 7f043be00640 1 -- 172.21.15.131:0/2040671630 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bae63510f0 con 0x55bae6745540 2024-05-26T22:50:58.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.587+0000 7f043be00640 1 --2- 172.21.15.131:0/2040671630 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bae6745540 0x55bae6761870 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f042c014700 tx=0x7f042c003a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94590b40f281e780 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:58.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.587+0000 7f0428c00640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-26T22:50:58.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.587+0000 7f0428c00640 1 -- 172.21.15.131:0/2040671630 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f042c0128a0 con 0x55bae6745540 2024-05-26T22:50:58.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.587+0000 7f0445b7c740 1 -- 172.21.15.131:0/2040671630 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0424042070 con 0x55bae6745540 2024-05-26T22:50:58.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.587+0000 7f0428c00640 1 -- 172.21.15.131:0/2040671630 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f042c012a40 con 0x55bae6745540 2024-05-26T22:50:58.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.588+0000 7f0428c00640 1 -- 172.21.15.131:0/2040671630 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f042c02c050 con 0x55bae6745540 2024-05-26T22:50:58.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.588+0000 7f0445b7c740 10 client.5949 fetch_fsmap learned FSMap version 34 2024-05-26T22:50:58.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.588+0000 7f0445b7c740 1 -- 172.21.15.131:0/2040671630 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) -- 0x55bae64bbf60 con 0x55bae6745540 2024-05-26T22:50:58.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.588+0000 7f0428c00640 1 -- 172.21.15.131:0/2040671630 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0424042070 con 0x55bae6745540 2024-05-26T22:50:58.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.589+0000 7f0428c00640 1 -- 172.21.15.131:0/2040671630 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 34) ==== 3542+0+0 (secure 0 0 0) 0x7f042c02f020 con 0x55bae6745540 2024-05-26T22:50:58.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.589+0000 7f0445b7c740 10 client.5949 fetch_fsmap finished waiting for FSMap version 34 2024-05-26T22:50:58.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.589+0000 7f0445b7c740 10 client.5949 resolve_mds: resolved 3:2 to role '3:2' aka daemon mds.b 2024-05-26T22:50:58.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.589+0000 7f0445b7c740 20 client.5949 populate_metadata read hostname 'smithi131' 2024-05-26T22:50:58.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.589+0000 7f0445b7c740 1 --2- 172.21.15.131:0/2040671630 >> [v2:172.21.15.187:6832/1027783169,v1:172.21.15.187:6834/1027783169] conn(0x55bae6732ff0 0x55bae6733470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:58.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.589+0000 7f0445b7c740 4 client.5949 mds_command: new command op to 4670 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-26T22:50:58.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.589+0000 7f0445b7c740 1 -- 172.21.15.131:0/2040671630 --> [v2:172.21.15.187:6832/1027783169,v1:172.21.15.187:6834/1027783169] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bae668b090 con 0x55bae6732ff0 2024-05-26T22:50:58.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.590+0000 7f0433400640 1 --2- 172.21.15.131:0/2040671630 >> [v2:172.21.15.187:6832/1027783169,v1:172.21.15.187:6834/1027783169] conn(0x55bae6732ff0 0x55bae6733470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:58.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.590+0000 7f0433400640 1 --2- 172.21.15.131:0/2040671630 >> [v2:172.21.15.187:6832/1027783169,v1:172.21.15.187:6834/1027783169] conn(0x55bae6732ff0 0x55bae6733470 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=12dba229d778b323 server_cookie=fdb7b559d8eb5067 in_seq=0 out_seq=0 2024-05-26T22:50:58.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.590+0000 7f0428c00640 10 client.5949 ms_handle_connect on v2:172.21.15.187:6832/1027783169 2024-05-26T22:50:58.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.592+0000 7f0428c00640 1 -- 172.21.15.131:0/2040671630 <== mds.2 v2:172.21.15.187:6832/1027783169 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55bae668b090 con 0x55bae6732ff0 2024-05-26T22:50:58.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.592+0000 7f0428c00640 10 client.5949 handle_command_reply: tid=0 2024-05-26T22:50:58.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.592+0000 7f0445b7c740 1 client.5949 shutdown 2024-05-26T22:50:58.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.593+0000 7f0445b7c740 1 -- 172.21.15.131:0/2040671630 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bae6745540 msgr2=0x55bae6761870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:58.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.593+0000 7f0445b7c740 1 --2- 172.21.15.131:0/2040671630 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bae6745540 0x55bae6761870 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f042c014700 tx=0x7f042c003a70 comp rx=0 tx=0).stop 2024-05-26T22:50:58.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.593+0000 7f0445b7c740 1 -- 172.21.15.131:0/2040671630 shutdown_connections 2024-05-26T22:50:58.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.593+0000 7f0445b7c740 1 --2- 172.21.15.131:0/2040671630 >> [v2:172.21.15.187:6832/1027783169,v1:172.21.15.187:6834/1027783169] conn(0x55bae6732ff0 0x55bae6733470 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:58.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.593+0000 7f0445b7c740 1 --2- 172.21.15.131:0/2040671630 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55bae675e4c0 0x55bae675e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:58.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.593+0000 7f0445b7c740 1 --2- 172.21.15.131:0/2040671630 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bae6762420 0x55bae6747cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:58.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.593+0000 7f0428c00640 0 client.5949 ms_handle_reset on v2:172.21.15.187:6832/1027783169 2024-05-26T22:50:58.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.593+0000 7f0445b7c740 1 --2- 172.21.15.131:0/2040671630 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bae6745540 0x55bae6761870 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:58.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.593+0000 7f0445b7c740 1 -- 172.21.15.131:0/2040671630 >> 172.21.15.131:0/2040671630 conn(0x55bae6744e00 msgr2=0x55bae6714510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:58.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.593+0000 7f0445b7c740 1 -- 172.21.15.131:0/2040671630 shutdown_connections 2024-05-26T22:50:58.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.593+0000 7f0445b7c740 1 -- 172.21.15.131:0/2040671630 wait complete. 2024-05-26T22:50:58.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.593+0000 7f0445b7c740 20 client.5949 trim_cache size 0 max 16384 2024-05-26T22:50:58.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.611+0000 7f0445b7c740 1 Processor -- start 2024-05-26T22:50:58.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.611+0000 7f0445b7c740 1 -- start start 2024-05-26T22:50:58.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.612+0000 7f0445b7c740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55bae6745540 0x55bae6715360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:58.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.612+0000 7f0445b7c740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bae675e4c0 0x55bae6715af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:58.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.612+0000 7f0445b7c740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bae6762420 0x55bae6716030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:58.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.612+0000 7f0445b7c740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f043c200860 con 0x55bae675e4c0 2024-05-26T22:50:58.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.612+0000 7f0445b7c740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f043c1ffae0 con 0x55bae6762420 2024-05-26T22:50:58.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.612+0000 7f0445b7c740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f043c200130 con 0x55bae6745540 2024-05-26T22:50:58.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.612+0000 7f0433400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bae675e4c0 0x55bae6715af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:58.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.613+0000 7f0433400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bae675e4c0 0x55bae6715af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55042/0 (socket says 172.21.15.131:55042) 2024-05-26T22:50:58.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.613+0000 7f0433400640 1 -- 172.21.15.131:0/3119898552 learned_addr learned my addr 172.21.15.131:0/3119898552 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:58.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.613+0000 7f0433400640 1 -- 172.21.15.131:0/3119898552 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55bae6745540 msgr2=0x55bae6715360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:58.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.613+0000 7f0433400640 1 --2- 172.21.15.131:0/3119898552 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55bae6745540 0x55bae6715360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:58.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.613+0000 7f0433400640 1 -- 172.21.15.131:0/3119898552 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bae6762420 msgr2=0x55bae6716030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:58.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.613+0000 7f0433400640 1 --2- 172.21.15.131:0/3119898552 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bae6762420 0x55bae6716030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:58.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.613+0000 7f0433400640 1 -- 172.21.15.131:0/3119898552 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bae632a1b0 con 0x55bae675e4c0 2024-05-26T22:50:58.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.613+0000 7f0433400640 1 --2- 172.21.15.131:0/3119898552 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bae675e4c0 0x55bae6715af0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f042402a840 tx=0x7f0424008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e346bc90e50abfa server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:58.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.613+0000 7f042b400640 1 -- 172.21.15.131:0/3119898552 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0424037040 con 0x55bae675e4c0 2024-05-26T22:50:58.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.613+0000 7f042b400640 1 -- 172.21.15.131:0/3119898552 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0424042040 con 0x55bae675e4c0 2024-05-26T22:50:58.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.613+0000 7f042b400640 1 -- 172.21.15.131:0/3119898552 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0424032800 con 0x55bae675e4c0 2024-05-26T22:50:58.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.614+0000 7f0445b7c740 1 -- 172.21.15.131:0/3119898552 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bae675e4c0 msgr2=0x55bae6715af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:58.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.614+0000 7f0445b7c740 1 --2- 172.21.15.131:0/3119898552 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bae675e4c0 0x55bae6715af0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f042402a840 tx=0x7f0424008030 comp rx=0 tx=0).stop 2024-05-26T22:50:58.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.614+0000 7f0445b7c740 1 -- 172.21.15.131:0/3119898552 shutdown_connections 2024-05-26T22:50:58.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.614+0000 7f0445b7c740 1 --2- 172.21.15.131:0/3119898552 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bae6762420 0x55bae6716030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:58.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.614+0000 7f0445b7c740 1 --2- 172.21.15.131:0/3119898552 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bae675e4c0 0x55bae6715af0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:58.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.614+0000 7f0445b7c740 1 --2- 172.21.15.131:0/3119898552 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55bae6745540 0x55bae6715360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:58.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.614+0000 7f0445b7c740 1 -- 172.21.15.131:0/3119898552 >> 172.21.15.131:0/3119898552 conn(0x55bae6744e00 msgr2=0x55bae6761320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:58.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.614+0000 7f0445b7c740 1 -- 172.21.15.131:0/3119898552 shutdown_connections 2024-05-26T22:50:58.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.614+0000 7f0445b7c740 1 -- 172.21.15.131:0/3119898552 wait complete. 2024-05-26T22:50:58.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.615+0000 7f0445b7c740 1 -- start start 2024-05-26T22:50:58.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.615+0000 7f0445b7c740 1 Processor -- start 2024-05-26T22:50:58.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.616+0000 7f0445b7c740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55bae6745540 0x55bae67c6750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:58.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.616+0000 7f0445b7c740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bae675e4c0 0x55bae6761170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:58.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.616+0000 7f0445b7c740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bae6762420 0x55bae67616b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:58.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.616+0000 7f0445b7c740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55bae67335c0 con 0x55bae6762420 2024-05-26T22:50:58.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.616+0000 7f0445b7c740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f043c151d00 con 0x55bae675e4c0 2024-05-26T22:50:58.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.616+0000 7f0445b7c740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x55bae6714da0 con 0x55bae6745540 2024-05-26T22:50:58.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.616+0000 7f0445b7c740 10 client.0 _ll_register_callbacks cb 0x7f042c00ddd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-26T22:50:58.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.616+0000 7f0440c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bae6762420 0x55bae67616b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:58.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.616+0000 7f0440c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bae6762420 0x55bae67616b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55054/0 (socket says 172.21.15.131:55054) 2024-05-26T22:50:58.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.616+0000 7f0440c00640 1 -- 172.21.15.131:0/348021489 learned_addr learned my addr 172.21.15.131:0/348021489 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:58.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.617+0000 7f0440c00640 1 -- 172.21.15.131:0/348021489 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55bae6745540 msgr2=0x55bae67c6750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:58.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.617+0000 7f0440c00640 1 --2- 172.21.15.131:0/348021489 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55bae6745540 0x55bae67c6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:58.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.617+0000 7f0440c00640 1 -- 172.21.15.131:0/348021489 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bae675e4c0 msgr2=0x55bae6761170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:58.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.617+0000 7f0440c00640 1 --2- 172.21.15.131:0/348021489 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bae675e4c0 0x55bae6761170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:58.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.617+0000 7f0440c00640 1 -- 172.21.15.131:0/348021489 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bae6713e80 con 0x55bae6762420 2024-05-26T22:50:58.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.617+0000 7f0440c00640 1 --2- 172.21.15.131:0/348021489 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bae6762420 0x55bae67616b0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f0434061990 tx=0x7f0434059420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b4e44dc51ddcf6f server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:58.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.617+0000 7f0428c00640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-26T22:50:58.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.617+0000 7f0428c00640 1 -- 172.21.15.131:0/348021489 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f043401f030 con 0x55bae6762420 2024-05-26T22:50:58.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.617+0000 7f0428c00640 1 -- 172.21.15.131:0/348021489 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f043405d1b0 con 0x55bae6762420 2024-05-26T22:50:58.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.617+0000 7f0445b7c740 1 -- 172.21.15.131:0/348021489 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0424042040 con 0x55bae6762420 2024-05-26T22:50:58.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.618+0000 7f0428c00640 1 -- 172.21.15.131:0/348021489 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f043405d3d0 con 0x55bae6762420 2024-05-26T22:50:58.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.618+0000 7f0445b7c740 10 client.5955 fetch_fsmap learned FSMap version 34 2024-05-26T22:50:58.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.618+0000 7f0445b7c740 1 -- 172.21.15.131:0/348021489 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) -- 0x55bae6741190 con 0x55bae6762420 2024-05-26T22:50:58.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.618+0000 7f0428c00640 1 -- 172.21.15.131:0/348021489 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0424042040 con 0x55bae6762420 2024-05-26T22:50:58.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.618+0000 7f0428c00640 1 -- 172.21.15.131:0/348021489 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 34) ==== 3542+0+0 (secure 0 0 0) 0x7f0434089170 con 0x55bae6762420 2024-05-26T22:50:58.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.618+0000 7f0445b7c740 10 client.5955 fetch_fsmap finished waiting for FSMap version 34 2024-05-26T22:50:58.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.618+0000 7f0445b7c740 10 client.5955 resolve_mds: resolved 3:2 to role '3:2' aka daemon mds.b 2024-05-26T22:50:58.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.618+0000 7f0445b7c740 20 client.5955 populate_metadata read hostname 'smithi131' 2024-05-26T22:50:58.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.618+0000 7f0445b7c740 1 --2- 172.21.15.131:0/348021489 >> [v2:172.21.15.187:6832/1027783169,v1:172.21.15.187:6834/1027783169] conn(0x55bae6760780 0x55bae676dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:58.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.618+0000 7f0445b7c740 4 client.5955 mds_command: new command op to 4670 tid=0 multi_id=0 [{"prefix": "perf dump"}] 2024-05-26T22:50:58.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.618+0000 7f0445b7c740 1 -- 172.21.15.131:0/348021489 --> [v2:172.21.15.187:6832/1027783169,v1:172.21.15.187:6834/1027783169] -- command(tid 0: {"prefix": "perf dump"}) -- 0x55bae66853e0 con 0x55bae6760780 2024-05-26T22:50:58.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.619+0000 7f043be00640 1 --2- 172.21.15.131:0/348021489 >> [v2:172.21.15.187:6832/1027783169,v1:172.21.15.187:6834/1027783169] conn(0x55bae6760780 0x55bae676dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:58.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.620+0000 7f043be00640 1 --2- 172.21.15.131:0/348021489 >> [v2:172.21.15.187:6832/1027783169,v1:172.21.15.187:6834/1027783169] conn(0x55bae6760780 0x55bae676dfc0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=811bbb2182c88c29 server_cookie=f58f43374ad2a290 in_seq=0 out_seq=0 2024-05-26T22:50:58.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.620+0000 7f0428c00640 10 client.5955 ms_handle_connect on v2:172.21.15.187:6832/1027783169 2024-05-26T22:50:58.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.621+0000 7f0428c00640 1 -- 172.21.15.131:0/348021489 <== mds.2 v2:172.21.15.187:6832/1027783169 1 ==== command_reply(tid 0: 0 ) ==== 8+0+17707 (crc 0 0 0) 0x55bae66853e0 con 0x55bae6760780 2024-05-26T22:50:58.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.621+0000 7f0428c00640 10 client.5955 handle_command_reply: tid=0 2024-05-26T22:50:58.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.621+0000 7f0445b7c740 1 client.5955 shutdown 2024-05-26T22:50:58.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.621+0000 7f0445b7c740 1 -- 172.21.15.131:0/348021489 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bae6762420 msgr2=0x55bae67616b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:58.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.621+0000 7f0445b7c740 1 --2- 172.21.15.131:0/348021489 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bae6762420 0x55bae67616b0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f0434061990 tx=0x7f0434059420 comp rx=0 tx=0).stop 2024-05-26T22:50:58.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.622+0000 7f0445b7c740 1 -- 172.21.15.131:0/348021489 shutdown_connections 2024-05-26T22:50:58.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.622+0000 7f0445b7c740 1 --2- 172.21.15.131:0/348021489 >> [v2:172.21.15.187:6832/1027783169,v1:172.21.15.187:6834/1027783169] conn(0x55bae6760780 0x55bae676dfc0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:58.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.622+0000 7f0445b7c740 1 --2- 172.21.15.131:0/348021489 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bae6762420 0x55bae67616b0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:58.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.622+0000 7f0445b7c740 1 --2- 172.21.15.131:0/348021489 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bae675e4c0 0x55bae6761170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:58.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.622+0000 7f0445b7c740 1 --2- 172.21.15.131:0/348021489 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55bae6745540 0x55bae67c6750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:58.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.622+0000 7f0428c00640 0 client.5955 ms_handle_reset on v2:172.21.15.187:6832/1027783169 2024-05-26T22:50:58.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.622+0000 7f0445b7c740 1 -- 172.21.15.131:0/348021489 >> 172.21.15.131:0/348021489 conn(0x55bae6744e00 msgr2=0x55bae6746aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:58.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.622+0000 7f0445b7c740 1 -- 172.21.15.131:0/348021489 shutdown_connections 2024-05-26T22:50:58.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.622+0000 7f0445b7c740 1 -- 172.21.15.131:0/348021489 wait complete. 2024-05-26T22:50:58.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.622+0000 7f0445b7c740 20 client.5955 trim_cache size 0 max 16384 2024-05-26T22:50:58.625 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-26T22:50:58.625 INFO:teuthology.orchestra.run.smithi131.stdout: "AsyncMessenger::Worker-0": { 2024-05-26T22:50:58.625 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_messages": 13, 2024-05-26T22:50:58.625 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages": 14, 2024-05-26T22:50:58.625 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_bytes": 3091, 2024-05-26T22:50:58.625 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_bytes": 60294, 2024-05-26T22:50:58.625 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_created_connections": 12, 2024-05-26T22:50:58.626 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_active_connections": 3, 2024-05-26T22:50:58.626 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_total_time": 0.014699124, 2024-05-26T22:50:58.626 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_send_time": 0.000811602, 2024-05-26T22:50:58.626 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_recv_time": 0.009078602, 2024-05-26T22:50:58.626 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_fast_dispatch_time": 0.000311903, 2024-05-26T22:50:58.626 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages_queue_lat": { 2024-05-26T22:50:58.626 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 13, 2024-05-26T22:50:58.626 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.019228607, 2024-05-26T22:50:58.626 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.001479123 2024-05-26T22:50:58.626 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.626 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_handle_ack_lat": { 2024-05-26T22:50:58.627 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 4, 2024-05-26T22:50:58.627 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000803, 2024-05-26T22:50:58.627 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000200 2024-05-26T22:50:58.627 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.627 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-26T22:50:58.627 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_encrypted_bytes": 0 2024-05-26T22:50:58.627 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.627 INFO:teuthology.orchestra.run.smithi131.stdout: "AsyncMessenger::Worker-1": { 2024-05-26T22:50:58.627 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_messages": 43, 2024-05-26T22:50:58.627 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages": 55, 2024-05-26T22:50:58.627 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_bytes": 79325, 2024-05-26T22:50:58.627 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_bytes": 112958, 2024-05-26T22:50:58.627 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_created_connections": 6, 2024-05-26T22:50:58.627 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_active_connections": 4, 2024-05-26T22:50:58.628 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_total_time": 0.015945818, 2024-05-26T22:50:58.628 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_send_time": 0.004760359, 2024-05-26T22:50:58.628 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_recv_time": 0.011555379, 2024-05-26T22:50:58.628 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_fast_dispatch_time": 0.000327084, 2024-05-26T22:50:58.628 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages_queue_lat": { 2024-05-26T22:50:58.628 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 54, 2024-05-26T22:50:58.628 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.033450214, 2024-05-26T22:50:58.628 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000619448 2024-05-26T22:50:58.628 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.628 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_handle_ack_lat": { 2024-05-26T22:50:58.628 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 58, 2024-05-26T22:50:58.628 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000025968, 2024-05-26T22:50:58.628 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000447 2024-05-26T22:50:58.629 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.629 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-26T22:50:58.629 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_encrypted_bytes": 95680 2024-05-26T22:50:58.629 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.629 INFO:teuthology.orchestra.run.smithi131.stdout: "AsyncMessenger::Worker-2": { 2024-05-26T22:50:58.629 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_messages": 77, 2024-05-26T22:50:58.629 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages": 119, 2024-05-26T22:50:58.629 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_bytes": 122275, 2024-05-26T22:50:58.629 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_bytes": 88326, 2024-05-26T22:50:58.629 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_created_connections": 5, 2024-05-26T22:50:58.629 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_active_connections": 4, 2024-05-26T22:50:58.629 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_total_time": 0.021141412, 2024-05-26T22:50:58.629 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_send_time": 0.005338865, 2024-05-26T22:50:58.629 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_recv_time": 0.020393697, 2024-05-26T22:50:58.630 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_fast_dispatch_time": 0.001059927, 2024-05-26T22:50:58.630 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages_queue_lat": { 2024-05-26T22:50:58.630 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 119, 2024-05-26T22:50:58.630 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.046964694, 2024-05-26T22:50:58.630 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000394661 2024-05-26T22:50:58.630 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.630 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_handle_ack_lat": { 2024-05-26T22:50:58.630 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 2, 2024-05-26T22:50:58.630 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000004605, 2024-05-26T22:50:58.630 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000002302 2024-05-26T22:50:58.630 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.630 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_encrypted_bytes": 117152, 2024-05-26T22:50:58.630 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_encrypted_bytes": 13264 2024-05-26T22:50:58.631 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.631 INFO:teuthology.orchestra.run.smithi131.stdout: "cct": { 2024-05-26T22:50:58.631 INFO:teuthology.orchestra.run.smithi131.stdout: "total_workers": 1, 2024-05-26T22:50:58.631 INFO:teuthology.orchestra.run.smithi131.stdout: "unhealthy_workers": 0 2024-05-26T22:50:58.631 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.631 INFO:teuthology.orchestra.run.smithi131.stdout: "finisher-MDSRank": { 2024-05-26T22:50:58.631 INFO:teuthology.orchestra.run.smithi131.stdout: "queue_len": 0, 2024-05-26T22:50:58.631 INFO:teuthology.orchestra.run.smithi131.stdout: "complete_latency": { 2024-05-26T22:50:58.631 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 50, 2024-05-26T22:50:58.631 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.003761600, 2024-05-26T22:50:58.631 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000075232 2024-05-26T22:50:58.631 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:50:58.631 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.631 INFO:teuthology.orchestra.run.smithi131.stdout: "finisher-PurgeQueue": { 2024-05-26T22:50:58.632 INFO:teuthology.orchestra.run.smithi131.stdout: "queue_len": 0, 2024-05-26T22:50:58.632 INFO:teuthology.orchestra.run.smithi131.stdout: "complete_latency": { 2024-05-26T22:50:58.632 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 2, 2024-05-26T22:50:58.632 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000034801, 2024-05-26T22:50:58.632 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000017400 2024-05-26T22:50:58.632 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:50:58.632 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.632 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": { 2024-05-26T22:50:58.632 INFO:teuthology.orchestra.run.smithi131.stdout: "request": 0, 2024-05-26T22:50:58.632 INFO:teuthology.orchestra.run.smithi131.stdout: "reply": 0, 2024-05-26T22:50:58.632 INFO:teuthology.orchestra.run.smithi131.stdout: "reply_latency": { 2024-05-26T22:50:58.632 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:50:58.632 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:50:58.633 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:50:58.633 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.633 INFO:teuthology.orchestra.run.smithi131.stdout: "slow_reply": 0, 2024-05-26T22:50:58.633 INFO:teuthology.orchestra.run.smithi131.stdout: "forward": 0, 2024-05-26T22:50:58.633 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_fetch_complete": 0, 2024-05-26T22:50:58.633 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_fetch_keys": 0, 2024-05-26T22:50:58.633 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_commit": 11, 2024-05-26T22:50:58.633 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_split": 0, 2024-05-26T22:50:58.633 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_merge": 0, 2024-05-26T22:50:58.633 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes": 10, 2024-05-26T22:50:58.634 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_top": 0, 2024-05-26T22:50:58.634 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_bottom": 10, 2024-05-26T22:50:58.634 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_pin_tail": 0, 2024-05-26T22:50:58.634 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_pinned": 10, 2024-05-26T22:50:58.634 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_expired": 0, 2024-05-26T22:50:58.634 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_with_caps": 0, 2024-05-26T22:50:58.634 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": 0, 2024-05-26T22:50:58.634 INFO:teuthology.orchestra.run.smithi131.stdout: "subtrees": 1, 2024-05-26T22:50:58.634 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse": 2, 2024-05-26T22:50:58.634 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_hit": 0, 2024-05-26T22:50:58.634 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_forward": 0, 2024-05-26T22:50:58.634 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_discover": 2, 2024-05-26T22:50:58.634 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_dir_fetch": 0, 2024-05-26T22:50:58.635 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_remote_ino": 0, 2024-05-26T22:50:58.635 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_lock": 0, 2024-05-26T22:50:58.635 INFO:teuthology.orchestra.run.smithi131.stdout: "load_cent": 0, 2024-05-26T22:50:58.635 INFO:teuthology.orchestra.run.smithi131.stdout: "q": 0, 2024-05-26T22:50:58.635 INFO:teuthology.orchestra.run.smithi131.stdout: "exported": 0, 2024-05-26T22:50:58.635 INFO:teuthology.orchestra.run.smithi131.stdout: "exported_inodes": 0, 2024-05-26T22:50:58.635 INFO:teuthology.orchestra.run.smithi131.stdout: "imported": 2, 2024-05-26T22:50:58.635 INFO:teuthology.orchestra.run.smithi131.stdout: "imported_inodes": 50, 2024-05-26T22:50:58.635 INFO:teuthology.orchestra.run.smithi131.stdout: "openino_dir_fetch": 0, 2024-05-26T22:50:58.635 INFO:teuthology.orchestra.run.smithi131.stdout: "openino_backtrace_fetch": 0, 2024-05-26T22:50:58.635 INFO:teuthology.orchestra.run.smithi131.stdout: "openino_peer_discover": 0, 2024-05-26T22:50:58.635 INFO:teuthology.orchestra.run.smithi131.stdout: "root_rfiles": 100, 2024-05-26T22:50:58.635 INFO:teuthology.orchestra.run.smithi131.stdout: "root_rbytes": 500, 2024-05-26T22:50:58.636 INFO:teuthology.orchestra.run.smithi131.stdout: "root_rsnaps": 0, 2024-05-26T22:50:58.636 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_backtrace_fetch": 0, 2024-05-26T22:50:58.636 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_set_tag": 0, 2024-05-26T22:50:58.636 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_backtrace_repaired": 0, 2024-05-26T22:50:58.636 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_inotable_repaired": 0, 2024-05-26T22:50:58.636 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_dir_inodes": 0, 2024-05-26T22:50:58.636 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_dir_base_inodes": 0, 2024-05-26T22:50:58.636 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_dirfrag_rstats": 0, 2024-05-26T22:50:58.636 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_file_inodes": 0, 2024-05-26T22:50:58.636 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_inode_file_caps": 0, 2024-05-26T22:50:58.636 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_revoke": 0, 2024-05-26T22:50:58.636 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_grant": 0, 2024-05-26T22:50:58.636 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_trunc": 0, 2024-05-26T22:50:58.636 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-26T22:50:58.636 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-26T22:50:58.637 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_caps": 0, 2024-05-26T22:50:58.637 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_caps_dirty": 0, 2024-05-26T22:50:58.637 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_cap_release": 0, 2024-05-26T22:50:58.637 INFO:teuthology.orchestra.run.smithi131.stdout: "process_request_cap_release": 0 2024-05-26T22:50:58.637 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.637 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_cache": { 2024-05-26T22:50:58.637 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_update": 0, 2024-05-26T22:50:58.637 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_update_receipt": 0, 2024-05-26T22:50:58.637 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_try_discover": 0, 2024-05-26T22:50:58.637 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_send_discover": 3, 2024-05-26T22:50:58.637 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_handle_discover": 0, 2024-05-26T22:50:58.637 INFO:teuthology.orchestra.run.smithi131.stdout: "num_strays": 0, 2024-05-26T22:50:58.637 INFO:teuthology.orchestra.run.smithi131.stdout: "num_strays_delayed": 0, 2024-05-26T22:50:58.637 INFO:teuthology.orchestra.run.smithi131.stdout: "num_strays_enqueuing": 0, 2024-05-26T22:50:58.638 INFO:teuthology.orchestra.run.smithi131.stdout: "strays_created": 0, 2024-05-26T22:50:58.638 INFO:teuthology.orchestra.run.smithi131.stdout: "strays_enqueued": 0, 2024-05-26T22:50:58.638 INFO:teuthology.orchestra.run.smithi131.stdout: "strays_reintegrated": 0, 2024-05-26T22:50:58.638 INFO:teuthology.orchestra.run.smithi131.stdout: "strays_migrated": 0, 2024-05-26T22:50:58.638 INFO:teuthology.orchestra.run.smithi131.stdout: "num_recovering_processing": 0, 2024-05-26T22:50:58.638 INFO:teuthology.orchestra.run.smithi131.stdout: "num_recovering_enqueued": 0, 2024-05-26T22:50:58.638 INFO:teuthology.orchestra.run.smithi131.stdout: "num_recovering_prioritized": 0, 2024-05-26T22:50:58.638 INFO:teuthology.orchestra.run.smithi131.stdout: "recovery_started": 0, 2024-05-26T22:50:58.638 INFO:teuthology.orchestra.run.smithi131.stdout: "recovery_completed": 0, 2024-05-26T22:50:58.638 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_quiesce_path": 0, 2024-05-26T22:50:58.638 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_quiesce_inode": 0, 2024-05-26T22:50:58.638 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_enqueue_scrub": 0, 2024-05-26T22:50:58.638 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_exportdir": 0, 2024-05-26T22:50:58.639 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_flush": 0, 2024-05-26T22:50:58.639 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_fragmentdir": 0, 2024-05-26T22:50:58.639 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_fragstats": 0, 2024-05-26T22:50:58.639 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_inodestats": 0 2024-05-26T22:50:58.639 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.639 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_log": { 2024-05-26T22:50:58.639 INFO:teuthology.orchestra.run.smithi131.stdout: "evlrg": 0, 2024-05-26T22:50:58.639 INFO:teuthology.orchestra.run.smithi131.stdout: "evadd": 6, 2024-05-26T22:50:58.640 INFO:teuthology.orchestra.run.smithi131.stdout: "evex": 0, 2024-05-26T22:50:58.640 INFO:teuthology.orchestra.run.smithi131.stdout: "evtrm": 0, 2024-05-26T22:50:58.640 INFO:teuthology.orchestra.run.smithi131.stdout: "ev": 6, 2024-05-26T22:50:58.640 INFO:teuthology.orchestra.run.smithi131.stdout: "evexg": 0, 2024-05-26T22:50:58.640 INFO:teuthology.orchestra.run.smithi131.stdout: "evexd": 0, 2024-05-26T22:50:58.640 INFO:teuthology.orchestra.run.smithi131.stdout: "segadd": 2, 2024-05-26T22:50:58.640 INFO:teuthology.orchestra.run.smithi131.stdout: "segex": 0, 2024-05-26T22:50:58.640 INFO:teuthology.orchestra.run.smithi131.stdout: "segtrm": 0, 2024-05-26T22:50:58.640 INFO:teuthology.orchestra.run.smithi131.stdout: "seg": 2, 2024-05-26T22:50:58.640 INFO:teuthology.orchestra.run.smithi131.stdout: "segmjr": 2, 2024-05-26T22:50:58.640 INFO:teuthology.orchestra.run.smithi131.stdout: "segexg": 0, 2024-05-26T22:50:58.640 INFO:teuthology.orchestra.run.smithi131.stdout: "segexd": 0, 2024-05-26T22:50:58.640 INFO:teuthology.orchestra.run.smithi131.stdout: "expos": 4194304, 2024-05-26T22:50:58.640 INFO:teuthology.orchestra.run.smithi131.stdout: "wrpos": 4236091, 2024-05-26T22:50:58.641 INFO:teuthology.orchestra.run.smithi131.stdout: "rdpos": 0, 2024-05-26T22:50:58.641 INFO:teuthology.orchestra.run.smithi131.stdout: "jlat": { 2024-05-26T22:50:58.641 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 3, 2024-05-26T22:50:58.641 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.017628892, 2024-05-26T22:50:58.641 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.005876297 2024-05-26T22:50:58.641 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.641 INFO:teuthology.orchestra.run.smithi131.stdout: "replayed": 0 2024-05-26T22:50:58.641 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.641 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_mem": { 2024-05-26T22:50:58.641 INFO:teuthology.orchestra.run.smithi131.stdout: "ino": 64, 2024-05-26T22:50:58.641 INFO:teuthology.orchestra.run.smithi131.stdout: "ino+": 64, 2024-05-26T22:50:58.641 INFO:teuthology.orchestra.run.smithi131.stdout: "ino-": 0, 2024-05-26T22:50:58.641 INFO:teuthology.orchestra.run.smithi131.stdout: "dir": 39, 2024-05-26T22:50:58.642 INFO:teuthology.orchestra.run.smithi131.stdout: "dir+": 40, 2024-05-26T22:50:58.642 INFO:teuthology.orchestra.run.smithi131.stdout: "dir-": 1, 2024-05-26T22:50:58.642 INFO:teuthology.orchestra.run.smithi131.stdout: "dn": 61, 2024-05-26T22:50:58.642 INFO:teuthology.orchestra.run.smithi131.stdout: "dn+": 61, 2024-05-26T22:50:58.642 INFO:teuthology.orchestra.run.smithi131.stdout: "dn-": 0, 2024-05-26T22:50:58.642 INFO:teuthology.orchestra.run.smithi131.stdout: "cap": 50, 2024-05-26T22:50:58.642 INFO:teuthology.orchestra.run.smithi131.stdout: "cap+": 50, 2024-05-26T22:50:58.642 INFO:teuthology.orchestra.run.smithi131.stdout: "cap-": 0, 2024-05-26T22:50:58.642 INFO:teuthology.orchestra.run.smithi131.stdout: "rss": 41984, 2024-05-26T22:50:58.642 INFO:teuthology.orchestra.run.smithi131.stdout: "heap": 223596 2024-05-26T22:50:58.642 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.642 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_server": { 2024-05-26T22:50:58.642 INFO:teuthology.orchestra.run.smithi131.stdout: "dispatch_client_request": 0, 2024-05-26T22:50:58.643 INFO:teuthology.orchestra.run.smithi131.stdout: "dispatch_server_request": 0, 2024-05-26T22:50:58.643 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_request": 0, 2024-05-26T22:50:58.643 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_session": 1, 2024-05-26T22:50:58.643 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_peer_request": 0, 2024-05-26T22:50:58.643 INFO:teuthology.orchestra.run.smithi131.stdout: "req_create_latency": { 2024-05-26T22:50:58.643 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:50:58.643 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:50:58.643 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:50:58.643 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.643 INFO:teuthology.orchestra.run.smithi131.stdout: "req_getattr_latency": { 2024-05-26T22:50:58.643 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:50:58.643 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:50:58.643 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:50:58.643 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.644 INFO:teuthology.orchestra.run.smithi131.stdout: "req_getfilelock_latency": { 2024-05-26T22:50:58.644 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:50:58.644 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:50:58.644 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:50:58.644 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.644 INFO:teuthology.orchestra.run.smithi131.stdout: "req_link_latency": { 2024-05-26T22:50:58.644 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:50:58.644 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:50:58.644 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:50:58.645 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.645 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookup_latency": { 2024-05-26T22:50:58.645 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:50:58.645 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:50:58.645 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:50:58.645 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.645 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookuphash_latency": { 2024-05-26T22:50:58.645 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:50:58.645 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:50:58.645 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:50:58.645 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.645 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookupino_latency": { 2024-05-26T22:50:58.645 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:50:58.646 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:50:58.646 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:50:58.646 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.646 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookupname_latency": { 2024-05-26T22:50:58.646 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:50:58.646 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:50:58.646 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:50:58.646 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.646 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookupparent_latency": { 2024-05-26T22:50:58.646 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:50:58.647 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:50:58.647 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:50:58.647 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.647 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookupsnap_latency": { 2024-05-26T22:50:58.647 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:50:58.647 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:50:58.647 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:50:58.647 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.647 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lssnap_latency": { 2024-05-26T22:50:58.647 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:50:58.647 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:50:58.647 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:50:58.648 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.648 INFO:teuthology.orchestra.run.smithi131.stdout: "req_mkdir_latency": { 2024-05-26T22:50:58.648 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:50:58.648 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:50:58.648 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:50:58.648 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.648 INFO:teuthology.orchestra.run.smithi131.stdout: "req_mknod_latency": { 2024-05-26T22:50:58.648 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:50:58.649 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:50:58.649 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:50:58.649 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.649 INFO:teuthology.orchestra.run.smithi131.stdout: "req_mksnap_latency": { 2024-05-26T22:50:58.649 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:50:58.649 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:50:58.649 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:50:58.649 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.649 INFO:teuthology.orchestra.run.smithi131.stdout: "req_open_latency": { 2024-05-26T22:50:58.649 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:50:58.649 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:50:58.649 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:50:58.650 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.650 INFO:teuthology.orchestra.run.smithi131.stdout: "req_readdir_latency": { 2024-05-26T22:50:58.650 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:50:58.650 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:50:58.650 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:50:58.650 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.650 INFO:teuthology.orchestra.run.smithi131.stdout: "req_rename_latency": { 2024-05-26T22:50:58.650 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:50:58.650 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:50:58.650 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:50:58.651 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.651 INFO:teuthology.orchestra.run.smithi131.stdout: "req_renamesnap_latency": { 2024-05-26T22:50:58.651 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:50:58.651 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:50:58.651 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:50:58.651 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.651 INFO:teuthology.orchestra.run.smithi131.stdout: "req_snapdiff_latency": { 2024-05-26T22:50:58.651 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:50:58.651 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:50:58.652 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:50:58.652 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.652 INFO:teuthology.orchestra.run.smithi131.stdout: "req_rmdir_latency": { 2024-05-26T22:50:58.652 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:50:58.652 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:50:58.652 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:50:58.652 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.652 INFO:teuthology.orchestra.run.smithi131.stdout: "req_rmsnap_latency": { 2024-05-26T22:50:58.652 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:50:58.652 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:50:58.653 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:50:58.653 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.653 INFO:teuthology.orchestra.run.smithi131.stdout: "req_rmxattr_latency": { 2024-05-26T22:50:58.653 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:50:58.653 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:50:58.653 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:50:58.653 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.653 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setattr_latency": { 2024-05-26T22:50:58.653 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:50:58.654 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:50:58.654 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:50:58.654 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.654 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setdirlayout_latency": { 2024-05-26T22:50:58.654 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:50:58.654 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:50:58.654 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:50:58.654 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.654 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setfilelock_latency": { 2024-05-26T22:50:58.654 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:50:58.654 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:50:58.655 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:50:58.655 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.655 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setlayout_latency": { 2024-05-26T22:50:58.655 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:50:58.655 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:50:58.655 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:50:58.655 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.655 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setxattr_latency": { 2024-05-26T22:50:58.655 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:50:58.655 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:50:58.655 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:50:58.655 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.655 INFO:teuthology.orchestra.run.smithi131.stdout: "req_symlink_latency": { 2024-05-26T22:50:58.656 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:50:58.656 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:50:58.656 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:50:58.656 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.656 INFO:teuthology.orchestra.run.smithi131.stdout: "req_unlink_latency": { 2024-05-26T22:50:58.656 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:50:58.656 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:50:58.656 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:50:58.656 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.656 INFO:teuthology.orchestra.run.smithi131.stdout: "cap_revoke_eviction": 0, 2024-05-26T22:50:58.656 INFO:teuthology.orchestra.run.smithi131.stdout: "cap_acquisition_throttle": 0, 2024-05-26T22:50:58.656 INFO:teuthology.orchestra.run.smithi131.stdout: "req_getvxattr_latency": { 2024-05-26T22:50:58.656 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:50:58.656 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:50:58.657 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:50:58.657 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:50:58.657 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.657 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_sessions": { 2024-05-26T22:50:58.657 INFO:teuthology.orchestra.run.smithi131.stdout: "session_count": 1, 2024-05-26T22:50:58.657 INFO:teuthology.orchestra.run.smithi131.stdout: "session_add": 1, 2024-05-26T22:50:58.657 INFO:teuthology.orchestra.run.smithi131.stdout: "session_remove": 0, 2024-05-26T22:50:58.657 INFO:teuthology.orchestra.run.smithi131.stdout: "sessions_open": 1, 2024-05-26T22:50:58.657 INFO:teuthology.orchestra.run.smithi131.stdout: "sessions_stale": 0, 2024-05-26T22:50:58.657 INFO:teuthology.orchestra.run.smithi131.stdout: "total_load": 0, 2024-05-26T22:50:58.657 INFO:teuthology.orchestra.run.smithi131.stdout: "average_load": 0, 2024-05-26T22:50:58.657 INFO:teuthology.orchestra.run.smithi131.stdout: "avg_session_uptime": 0, 2024-05-26T22:50:58.657 INFO:teuthology.orchestra.run.smithi131.stdout: "mdthresh_evicted": 0 2024-05-26T22:50:58.657 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.658 INFO:teuthology.orchestra.run.smithi131.stdout: "mempool": { 2024-05-26T22:50:58.658 INFO:teuthology.orchestra.run.smithi131.stdout: "bloom_filter_bytes": 0, 2024-05-26T22:50:58.658 INFO:teuthology.orchestra.run.smithi131.stdout: "bloom_filter_items": 0, 2024-05-26T22:50:58.658 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_alloc_bytes": 0, 2024-05-26T22:50:58.658 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_alloc_items": 0, 2024-05-26T22:50:58.658 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_data_bytes": 0, 2024-05-26T22:50:58.658 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_data_items": 0, 2024-05-26T22:50:58.658 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-26T22:50:58.658 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_onode_items": 0, 2024-05-26T22:50:58.658 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-26T22:50:58.658 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_meta_items": 0, 2024-05-26T22:50:58.658 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_other_bytes": 0, 2024-05-26T22:50:58.658 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_other_items": 0, 2024-05-26T22:50:58.659 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-26T22:50:58.659 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_buffer_items": 0, 2024-05-26T22:50:58.659 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_extent_bytes": 0, 2024-05-26T22:50:58.659 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_extent_items": 0, 2024-05-26T22:50:58.659 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_blob_bytes": 0, 2024-05-26T22:50:58.659 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_blob_items": 0, 2024-05-26T22:50:58.659 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-26T22:50:58.659 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_shared_blob_items": 0, 2024-05-26T22:50:58.659 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-26T22:50:58.659 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_inline_bl_items": 0, 2024-05-26T22:50:58.659 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_fsck_bytes": 0, 2024-05-26T22:50:58.659 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_fsck_items": 0, 2024-05-26T22:50:58.659 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_txc_bytes": 0, 2024-05-26T22:50:58.659 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_txc_items": 0, 2024-05-26T22:50:58.660 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-26T22:50:58.660 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_deferred_items": 0, 2024-05-26T22:50:58.660 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_bytes": 0, 2024-05-26T22:50:58.660 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_items": 0, 2024-05-26T22:50:58.660 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_bytes": 0, 2024-05-26T22:50:58.660 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_items": 0, 2024-05-26T22:50:58.660 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_reader_bytes": 0, 2024-05-26T22:50:58.660 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_reader_items": 0, 2024-05-26T22:50:58.660 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_writer_bytes": 0, 2024-05-26T22:50:58.660 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_writer_items": 0, 2024-05-26T22:50:58.660 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_anon_bytes": 220631, 2024-05-26T22:50:58.660 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_anon_items": 57, 2024-05-26T22:50:58.660 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_meta_bytes": 0, 2024-05-26T22:50:58.660 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_meta_items": 0, 2024-05-26T22:50:58.661 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_bytes": 0, 2024-05-26T22:50:58.661 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_items": 0, 2024-05-26T22:50:58.661 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_mapbl_bytes": 0, 2024-05-26T22:50:58.661 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_mapbl_items": 0, 2024-05-26T22:50:58.661 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_pglog_bytes": 0, 2024-05-26T22:50:58.661 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_pglog_items": 0, 2024-05-26T22:50:58.661 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_bytes": 5312, 2024-05-26T22:50:58.661 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_items": 75, 2024-05-26T22:50:58.661 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_mapping_bytes": 0, 2024-05-26T22:50:58.661 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_mapping_items": 0, 2024-05-26T22:50:58.661 INFO:teuthology.orchestra.run.smithi131.stdout: "pgmap_bytes": 0, 2024-05-26T22:50:58.661 INFO:teuthology.orchestra.run.smithi131.stdout: "pgmap_items": 0, 2024-05-26T22:50:58.661 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_co_bytes": 50664, 2024-05-26T22:50:58.661 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_co_items": 198, 2024-05-26T22:50:58.662 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_1_bytes": 0, 2024-05-26T22:50:58.662 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_1_items": 0, 2024-05-26T22:50:58.662 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_2_bytes": 0, 2024-05-26T22:50:58.662 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_2_items": 0 2024-05-26T22:50:58.662 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.662 INFO:teuthology.orchestra.run.smithi131.stdout: "objecter": { 2024-05-26T22:50:58.662 INFO:teuthology.orchestra.run.smithi131.stdout: "op_active": 0, 2024-05-26T22:50:58.662 INFO:teuthology.orchestra.run.smithi131.stdout: "op_laggy": 0, 2024-05-26T22:50:58.662 INFO:teuthology.orchestra.run.smithi131.stdout: "op_send": 38, 2024-05-26T22:50:58.662 INFO:teuthology.orchestra.run.smithi131.stdout: "op_send_bytes": 54544, 2024-05-26T22:50:58.662 INFO:teuthology.orchestra.run.smithi131.stdout: "op_resend": 29, 2024-05-26T22:50:58.662 INFO:teuthology.orchestra.run.smithi131.stdout: "op_reply": 38, 2024-05-26T22:50:58.662 INFO:teuthology.orchestra.run.smithi131.stdout: "op_latency": { 2024-05-26T22:50:58.663 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 38, 2024-05-26T22:50:58.663 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.503992173, 2024-05-26T22:50:58.663 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.013262951 2024-05-26T22:50:58.663 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.663 INFO:teuthology.orchestra.run.smithi131.stdout: "op_inflight": 0, 2024-05-26T22:50:58.663 INFO:teuthology.orchestra.run.smithi131.stdout: "oplen_avg": { 2024-05-26T22:50:58.663 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 38, 2024-05-26T22:50:58.663 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 61 2024-05-26T22:50:58.663 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.663 INFO:teuthology.orchestra.run.smithi131.stdout: "op": 38, 2024-05-26T22:50:58.663 INFO:teuthology.orchestra.run.smithi131.stdout: "op_r": 0, 2024-05-26T22:50:58.663 INFO:teuthology.orchestra.run.smithi131.stdout: "op_w": 38, 2024-05-26T22:50:58.663 INFO:teuthology.orchestra.run.smithi131.stdout: "op_rmw": 0, 2024-05-26T22:50:58.663 INFO:teuthology.orchestra.run.smithi131.stdout: "op_pg": 0, 2024-05-26T22:50:58.664 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_stat": 0, 2024-05-26T22:50:58.664 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_create": 10, 2024-05-26T22:50:58.664 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_read": 0, 2024-05-26T22:50:58.664 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_write": 3, 2024-05-26T22:50:58.664 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_writefull": 5, 2024-05-26T22:50:58.664 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_writesame": 0, 2024-05-26T22:50:58.664 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_append": 0, 2024-05-26T22:50:58.664 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_zero": 0, 2024-05-26T22:50:58.664 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_truncate": 0, 2024-05-26T22:50:58.664 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_delete": 6, 2024-05-26T22:50:58.664 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_mapext": 0, 2024-05-26T22:50:58.664 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_sparse_read": 0, 2024-05-26T22:50:58.664 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_clonerange": 0, 2024-05-26T22:50:58.665 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_getxattr": 0, 2024-05-26T22:50:58.665 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_setxattr": 20, 2024-05-26T22:50:58.665 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_cmpxattr": 0, 2024-05-26T22:50:58.665 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_rmxattr": 0, 2024-05-26T22:50:58.665 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_resetxattrs": 0, 2024-05-26T22:50:58.665 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_call": 0, 2024-05-26T22:50:58.665 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_watch": 0, 2024-05-26T22:50:58.665 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_notify": 0, 2024-05-26T22:50:58.665 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_src_cmpxattr": 0, 2024-05-26T22:50:58.665 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_pgls": 0, 2024-05-26T22:50:58.665 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_pgls_filter": 0, 2024-05-26T22:50:58.665 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_other": 0, 2024-05-26T22:50:58.665 INFO:teuthology.orchestra.run.smithi131.stdout: "linger_active": 0, 2024-05-26T22:50:58.665 INFO:teuthology.orchestra.run.smithi131.stdout: "linger_send": 0, 2024-05-26T22:50:58.666 INFO:teuthology.orchestra.run.smithi131.stdout: "linger_resend": 0, 2024-05-26T22:50:58.666 INFO:teuthology.orchestra.run.smithi131.stdout: "linger_ping": 0, 2024-05-26T22:50:58.666 INFO:teuthology.orchestra.run.smithi131.stdout: "poolop_active": 0, 2024-05-26T22:50:58.666 INFO:teuthology.orchestra.run.smithi131.stdout: "poolop_send": 0, 2024-05-26T22:50:58.666 INFO:teuthology.orchestra.run.smithi131.stdout: "poolop_resend": 0, 2024-05-26T22:50:58.666 INFO:teuthology.orchestra.run.smithi131.stdout: "poolstat_active": 0, 2024-05-26T22:50:58.666 INFO:teuthology.orchestra.run.smithi131.stdout: "poolstat_send": 0, 2024-05-26T22:50:58.666 INFO:teuthology.orchestra.run.smithi131.stdout: "poolstat_resend": 0, 2024-05-26T22:50:58.666 INFO:teuthology.orchestra.run.smithi131.stdout: "statfs_active": 0, 2024-05-26T22:50:58.666 INFO:teuthology.orchestra.run.smithi131.stdout: "statfs_send": 0, 2024-05-26T22:50:58.666 INFO:teuthology.orchestra.run.smithi131.stdout: "statfs_resend": 0, 2024-05-26T22:50:58.666 INFO:teuthology.orchestra.run.smithi131.stdout: "command_active": 0, 2024-05-26T22:50:58.666 INFO:teuthology.orchestra.run.smithi131.stdout: "command_send": 0, 2024-05-26T22:50:58.666 INFO:teuthology.orchestra.run.smithi131.stdout: "command_resend": 0, 2024-05-26T22:50:58.667 INFO:teuthology.orchestra.run.smithi131.stdout: "map_epoch": 0, 2024-05-26T22:50:58.667 INFO:teuthology.orchestra.run.smithi131.stdout: "map_full": 0, 2024-05-26T22:50:58.667 INFO:teuthology.orchestra.run.smithi131.stdout: "map_inc": 0, 2024-05-26T22:50:58.667 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_sessions": 7, 2024-05-26T22:50:58.667 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_session_open": 7, 2024-05-26T22:50:58.667 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_session_close": 0, 2024-05-26T22:50:58.667 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_laggy": 0, 2024-05-26T22:50:58.667 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_wr": 16, 2024-05-26T22:50:58.667 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_rd": 0, 2024-05-26T22:50:58.667 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_del": 1 2024-05-26T22:50:58.667 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.667 INFO:teuthology.orchestra.run.smithi131.stdout: "oft": { 2024-05-26T22:50:58.667 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_total_objs": 1, 2024-05-26T22:50:58.667 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_total_kv_pairs": 0, 2024-05-26T22:50:58.668 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_total_updates": 0, 2024-05-26T22:50:58.668 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_total_removes": 0 2024-05-26T22:50:58.668 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.668 INFO:teuthology.orchestra.run.smithi131.stdout: "osd-slow-ops": { 2024-05-26T22:50:58.668 INFO:teuthology.orchestra.run.smithi131.stdout: "slow_ops_count": 0 2024-05-26T22:50:58.668 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.668 INFO:teuthology.orchestra.run.smithi131.stdout: "purge_queue": { 2024-05-26T22:50:58.668 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executing_ops": 0, 2024-05-26T22:50:58.668 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executing_ops_high_water": 0, 2024-05-26T22:50:58.668 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executing": 0, 2024-05-26T22:50:58.669 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executing_high_water": 0, 2024-05-26T22:50:58.670 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executed_ops": 0, 2024-05-26T22:50:58.670 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executed": 0, 2024-05-26T22:50:58.670 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_item_in_journal": 0 2024-05-26T22:50:58.670 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.670 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-26T22:50:58.670 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-05-26T22:50:58.670 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 104857600, 2024-05-26T22:50:58.670 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-05-26T22:50:58.670 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 129, 2024-05-26T22:50:58.670 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 188931, 2024-05-26T22:50:58.670 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-05-26T22:50:58.670 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 129, 2024-05-26T22:50:58.670 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 0, 2024-05-26T22:50:58.670 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 0, 2024-05-26T22:50:58.671 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 129, 2024-05-26T22:50:58.671 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 188931, 2024-05-26T22:50:58.671 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-05-26T22:50:58.671 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:50:58.671 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:50:58.671 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:50:58.671 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:50:58.671 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.671 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-objecter_bytes": { 2024-05-26T22:50:58.671 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-05-26T22:50:58.671 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 104857600, 2024-05-26T22:50:58.671 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-05-26T22:50:58.671 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 0, 2024-05-26T22:50:58.672 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 0, 2024-05-26T22:50:58.672 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-05-26T22:50:58.672 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 0, 2024-05-26T22:50:58.672 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 38, 2024-05-26T22:50:58.672 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 54544, 2024-05-26T22:50:58.672 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 38, 2024-05-26T22:50:58.672 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 54544, 2024-05-26T22:50:58.672 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-05-26T22:50:58.672 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:50:58.672 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:50:58.672 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:50:58.672 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:50:58.672 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.672 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-objecter_ops": { 2024-05-26T22:50:58.673 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-05-26T22:50:58.673 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 1024, 2024-05-26T22:50:58.673 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-05-26T22:50:58.673 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 0, 2024-05-26T22:50:58.673 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 0, 2024-05-26T22:50:58.673 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-05-26T22:50:58.673 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 0, 2024-05-26T22:50:58.673 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 38, 2024-05-26T22:50:58.673 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 38, 2024-05-26T22:50:58.673 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 38, 2024-05-26T22:50:58.673 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 38, 2024-05-26T22:50:58.673 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-05-26T22:50:58.673 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:50:58.674 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:50:58.674 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:50:58.674 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:50:58.674 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.674 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-write_buf_throttle": { 2024-05-26T22:50:58.674 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-05-26T22:50:58.674 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 3758096384, 2024-05-26T22:50:58.674 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-05-26T22:50:58.674 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 0, 2024-05-26T22:50:58.674 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 0, 2024-05-26T22:50:58.674 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-05-26T22:50:58.674 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 0, 2024-05-26T22:50:58.674 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 0, 2024-05-26T22:50:58.675 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 0, 2024-05-26T22:50:58.675 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 0, 2024-05-26T22:50:58.675 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 0, 2024-05-26T22:50:58.675 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-05-26T22:50:58.675 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:50:58.675 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:50:58.675 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:50:58.675 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:50:58.675 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.675 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-write_buf_throttle-0x560504ef4500": { 2024-05-26T22:50:58.675 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 122, 2024-05-26T22:50:58.675 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 3758096384, 2024-05-26T22:50:58.675 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-05-26T22:50:58.676 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 6, 2024-05-26T22:50:58.676 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 41787, 2024-05-26T22:50:58.676 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-05-26T22:50:58.676 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 6, 2024-05-26T22:50:58.676 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 0, 2024-05-26T22:50:58.676 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 0, 2024-05-26T22:50:58.676 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 3, 2024-05-26T22:50:58.676 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 41665, 2024-05-26T22:50:58.676 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-05-26T22:50:58.676 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:50:58.676 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:50:58.676 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:50:58.676 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:50:58.676 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:50:58.677 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-26T22:50:58.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.622+0000 7f0441600640 1 -- 172.21.15.131:0/1564922518 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f041806be30 msgr2=0x7f041806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:58.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.622+0000 7f0441600640 1 --2- 172.21.15.131:0/1564922518 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f041806be30 0x7f041806e2f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f042c004520 tx=0x7f042c009040 comp rx=0 tx=0).stop 2024-05-26T22:50:58.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.622+0000 7f0441600640 1 -- 172.21.15.131:0/1564922518 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f043c15c220 msgr2=0x7f043c0b4b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:58.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.622+0000 7f0441600640 1 --2- 172.21.15.131:0/1564922518 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f043c15c220 0x7f043c0b4b20 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f043400c7f0 tx=0x7f043400ccb0 comp rx=0 tx=0).stop 2024-05-26T22:50:58.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.623+0000 7f0441600640 1 -- 172.21.15.131:0/1564922518 shutdown_connections 2024-05-26T22:50:58.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.623+0000 7f0441600640 1 --2- 172.21.15.131:0/1564922518 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f043c1554f0 0x7f043c0a9f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:58.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.623+0000 7f0441600640 1 --2- 172.21.15.131:0/1564922518 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f041806be30 0x7f041806e2f0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:58.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.623+0000 7f0441600640 1 --2- 172.21.15.131:0/1564922518 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f043c15c220 0x7f043c0b4b20 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:58.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.623+0000 7f0441600640 1 --2- 172.21.15.131:0/1564922518 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f043c06e9f0 0x7f043c152b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:58.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.623+0000 7f0441600640 1 -- 172.21.15.131:0/1564922518 >> 172.21.15.131:0/1564922518 conn(0x7f043c0cc7f0 msgr2=0x7f043c06e2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:58.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.623+0000 7f0441600640 1 -- 172.21.15.131:0/1564922518 shutdown_connections 2024-05-26T22:50:58.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.623+0000 7f0441600640 1 -- 172.21.15.131:0/1564922518 wait complete. 2024-05-26T22:50:58.680 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:1 perf dump 2024-05-26T22:50:58.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.889+0000 7f9a3e600640 1 Processor -- start 2024-05-26T22:50:58.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.889+0000 7f9a3e600640 1 -- start start 2024-05-26T22:50:58.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.890+0000 7f9a3e600640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f9a381554f0 0x7f9a38071f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:58.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.890+0000 7f9a3e600640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9a3806ebc0 0x7f9a3815d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:58.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.890+0000 7f9a3e600640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a381558d0 con 0x7f9a380724a0 2024-05-26T22:50:58.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.890+0000 7f9a3e600640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f9a380729b0 con 0x7f9a381554f0 2024-05-26T22:50:58.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.891+0000 7f9a3e600640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9a3815c750 con 0x7f9a3806ebc0 2024-05-26T22:50:58.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.891+0000 7f9a37e00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f9a380724a0 0x7f9a3806e4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:44802/0 (socket says 172.21.15.131:44802) 2024-05-26T22:50:58.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.891+0000 7f9a37e00640 1 -- 172.21.15.131:0/210532152 learned_addr learned my addr 172.21.15.131:0/210532152 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:50:58.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.891+0000 7f9a3d200640 1 --2- 172.21.15.131:0/210532152 >> v2:172.21.15.187:3300/0 conn(0x7f9a381554f0 0x7f9a38071f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:58.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.891+0000 7f9a37400640 1 -- 172.21.15.131:0/210532152 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3406673193 0 0) 0x7f9a24001110 con 0x7f9a380724a0 2024-05-26T22:50:58.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.891+0000 7f9a37400640 1 -- 172.21.15.131:0/210532152 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9a1c003520 con 0x7f9a380724a0 2024-05-26T22:50:58.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.891+0000 7f9a3dc00640 1 --2- 172.21.15.131:0/210532152 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9a3806ebc0 0x7f9a3815d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:58.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.892+0000 7f9a37400640 1 -- 172.21.15.131:0/210532152 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1723738721 0 0) 0x7f9a1c003520 con 0x7f9a380724a0 2024-05-26T22:50:58.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.892+0000 7f9a37400640 1 -- 172.21.15.131:0/210532152 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9a3806ebc0 msgr2=0x7f9a3815d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:58.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.892+0000 7f9a37400640 1 --2- 172.21.15.131:0/210532152 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9a3806ebc0 0x7f9a3815d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:58.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.892+0000 7f9a37400640 1 -- 172.21.15.131:0/210532152 >> v2:172.21.15.187:3300/0 conn(0x7f9a381554f0 msgr2=0x7f9a38071f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:58.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.892+0000 7f9a37400640 1 --2- 172.21.15.131:0/210532152 >> v2:172.21.15.187:3300/0 conn(0x7f9a381554f0 0x7f9a38071f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:58.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.892+0000 7f9a37400640 1 -- 172.21.15.131:0/210532152 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a3815ca30 con 0x7f9a380724a0 2024-05-26T22:50:58.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.892+0000 7f9a37400640 1 -- 172.21.15.131:0/210532152 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7f9a24001840 con 0x7f9a380724a0 2024-05-26T22:50:58.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.892+0000 7f9a37400640 1 -- 172.21.15.131:0/210532152 >> v1:172.21.15.131:6789/0 conn(0x7f9a380724a0 legacy=0x7f9a3806e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:58.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.892+0000 7f9a37400640 1 --2- 172.21.15.131:0/210532152 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9a1c0039b0 0x7f9a1c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:58.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.892+0000 7f9a37400640 1 --2- 172.21.15.131:0/210532152 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9a3806ebc0 0x7f9a1c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:58.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.892+0000 7f9a37400640 1 --2- 172.21.15.131:0/210532152 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9a1c004d40 0x7f9a1c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:58.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.892+0000 7f9a37400640 1 -- 172.21.15.131:0/210532152 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f9a380729b0 con 0x7f9a1c0039b0 2024-05-26T22:50:58.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.892+0000 7f9a37400640 1 -- 172.21.15.131:0/210532152 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9a3815c750 con 0x7f9a1c004d40 2024-05-26T22:50:58.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.892+0000 7f9a37400640 1 -- 172.21.15.131:0/210532152 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9a1c00b9c0 con 0x7f9a3806ebc0 2024-05-26T22:50:58.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.892+0000 7f9a3d200640 1 --2- 172.21.15.131:0/210532152 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9a1c0039b0 0x7f9a1c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:58.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.892+0000 7f9a37e00640 1 --2- 172.21.15.131:0/210532152 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9a3806ebc0 0x7f9a1c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:58.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.892+0000 7f9a3d200640 1 -- 172.21.15.131:0/210532152 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9a3806ebc0 msgr2=0x7f9a1c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:58.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.892+0000 7f9a3d200640 1 --2- 172.21.15.131:0/210532152 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9a3806ebc0 0x7f9a1c004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:58.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.892+0000 7f9a3d200640 1 -- 172.21.15.131:0/210532152 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9a1c004d40 msgr2=0x7f9a1c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:58.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.892+0000 7f9a3d200640 1 --2- 172.21.15.131:0/210532152 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9a1c004d40 0x7f9a1c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:58.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.892+0000 7f9a3d200640 1 -- 172.21.15.131:0/210532152 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a1c003520 con 0x7f9a1c0039b0 2024-05-26T22:50:58.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.893+0000 7f9a3d200640 1 --2- 172.21.15.131:0/210532152 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9a1c0039b0 0x7f9a1c003dd0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f9a28009af0 tx=0x7f9a1c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c79dbc876b16fe8 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:58.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.894+0000 7f9a37400640 1 -- 172.21.15.131:0/210532152 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a28036070 con 0x7f9a1c0039b0 2024-05-26T22:50:58.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.894+0000 7f9a37400640 1 -- 172.21.15.131:0/210532152 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a28031400 con 0x7f9a1c0039b0 2024-05-26T22:50:58.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.894+0000 7f9a37400640 1 -- 172.21.15.131:0/210532152 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a28009690 con 0x7f9a1c0039b0 2024-05-26T22:50:58.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.894+0000 7f9a3e600640 1 -- 172.21.15.131:0/210532152 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9a1c0039b0 msgr2=0x7f9a1c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:58.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.894+0000 7f9a3e600640 1 --2- 172.21.15.131:0/210532152 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9a1c0039b0 0x7f9a1c003dd0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f9a28009af0 tx=0x7f9a1c005830 comp rx=0 tx=0).stop 2024-05-26T22:50:58.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.894+0000 7f9a3dc00640 1 -- 172.21.15.131:0/210532152 reap_dead start 2024-05-26T22:50:58.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.894+0000 7f9a3e600640 1 -- 172.21.15.131:0/210532152 shutdown_connections 2024-05-26T22:50:58.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.894+0000 7f9a3e600640 1 -- 172.21.15.131:0/210532152 >> 172.21.15.131:0/210532152 conn(0x7f9a380cc7f0 msgr2=0x7f9a380ccbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:58.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.895+0000 7f9a3e600640 1 -- 172.21.15.131:0/210532152 shutdown_connections 2024-05-26T22:50:58.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.895+0000 7f9a3e600640 1 -- 172.21.15.131:0/210532152 wait complete. 2024-05-26T22:50:58.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.896+0000 7f9a3e600640 1 Processor -- start 2024-05-26T22:50:58.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.896+0000 7f9a3e600640 1 -- start start 2024-05-26T22:50:58.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.897+0000 7f9a3e600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9a1c0039b0 0x7f9a3813a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:58.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.897+0000 7f9a3e600640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9a3815a880 0x7f9a3813c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:58.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.897+0000 7f9a3e600640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9a381f7d40 0x7f9a381f81c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:58.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.897+0000 7f9a3e600640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f9a3815c3a0 con 0x7f9a1c0039b0 2024-05-26T22:50:58.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.897+0000 7f9a3e600640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9a3815c220 con 0x7f9a3815a880 2024-05-26T22:50:58.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.897+0000 7f9a3e600640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9a3815ace0 con 0x7f9a381f7d40 2024-05-26T22:50:58.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.898+0000 7f9a3d200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9a1c0039b0 0x7f9a3813a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:58.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.898+0000 7f9a3d200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9a1c0039b0 0x7f9a3813a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55082/0 (socket says 172.21.15.131:55082) 2024-05-26T22:50:58.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.898+0000 7f9a3d200640 1 -- 172.21.15.131:0/1645717863 learned_addr learned my addr 172.21.15.131:0/1645717863 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:58.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.898+0000 7f9a3d200640 1 -- 172.21.15.131:0/1645717863 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9a381f7d40 msgr2=0x7f9a381f81c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:58.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.898+0000 7f9a3d200640 1 --2- 172.21.15.131:0/1645717863 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9a381f7d40 0x7f9a381f81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:58.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.898+0000 7f9a3d200640 1 -- 172.21.15.131:0/1645717863 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9a3815a880 msgr2=0x7f9a3813c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:58.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.898+0000 7f9a3d200640 1 --2- 172.21.15.131:0/1645717863 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9a3815a880 0x7f9a3813c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:58.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.898+0000 7f9a3d200640 1 -- 172.21.15.131:0/1645717863 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a380cf290 con 0x7f9a1c0039b0 2024-05-26T22:50:58.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.898+0000 7f9a3d200640 1 --2- 172.21.15.131:0/1645717863 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9a1c0039b0 0x7f9a3813a080 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f9a28034040 tx=0x7f9a28004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:58.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.899+0000 7f9a34c00640 1 -- 172.21.15.131:0/1645717863 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a28041020 con 0x7f9a1c0039b0 2024-05-26T22:50:58.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.899+0000 7f9a34c00640 1 -- 172.21.15.131:0/1645717863 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a2802ade0 con 0x7f9a1c0039b0 2024-05-26T22:50:58.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.899+0000 7f9a3e600640 1 -- 172.21.15.131:0/1645717863 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a3813cf00 con 0x7f9a1c0039b0 2024-05-26T22:50:58.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.899+0000 7f9a34c00640 1 -- 172.21.15.131:0/1645717863 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a28036040 con 0x7f9a1c0039b0 2024-05-26T22:50:58.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.899+0000 7f9a3e600640 1 -- 172.21.15.131:0/1645717863 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a3813d4c0 con 0x7f9a1c0039b0 2024-05-26T22:50:58.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.901+0000 7f9a34c00640 1 -- 172.21.15.131:0/1645717863 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9a28044020 con 0x7f9a1c0039b0 2024-05-26T22:50:58.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.903+0000 7f9a34c00640 1 --2- 172.21.15.131:0/1645717863 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9a1806be30 0x7f9a1806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:58.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.903+0000 7f9a34c00640 1 -- 172.21.15.131:0/1645717863 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f9a280ab410 con 0x7f9a1c0039b0 2024-05-26T22:50:58.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.903+0000 7f9a37e00640 1 --2- 172.21.15.131:0/1645717863 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9a1806be30 0x7f9a1806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:58.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.904+0000 7f9a42b72740 1 Processor -- start 2024-05-26T22:50:58.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.904+0000 7f9a37e00640 1 --2- 172.21.15.131:0/1645717863 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9a1806be30 0x7f9a1806e2f0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f9a24000ff0 tx=0x7f9a24009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:58.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.904+0000 7f9a42b72740 1 -- start start 2024-05-26T22:50:58.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.904+0000 7f9a42b72740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bd564b4420 0x55bd56466db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:58.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.904+0000 7f9a42b72740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55bd56497540 0x55bd564968c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:58.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.904+0000 7f9a42b72740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bd56496e00 0x55bd5646bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:58.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.905+0000 7f9a42b72740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55bd563cf960 con 0x55bd564b4420 2024-05-26T22:50:58.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.905+0000 7f9a42b72740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55bd563770d0 con 0x55bd56496e00 2024-05-26T22:50:58.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.905+0000 7f9a42b72740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x55bd563cb920 con 0x55bd56497540 2024-05-26T22:50:58.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.905+0000 7f9a3d200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bd564b4420 0x55bd56466db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:58.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.905+0000 7f9a3d200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bd564b4420 0x55bd56466db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55090/0 (socket says 172.21.15.131:55090) 2024-05-26T22:50:58.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.905+0000 7f9a3d200640 1 -- 172.21.15.131:0/3279787108 learned_addr learned my addr 172.21.15.131:0/3279787108 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:58.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.905+0000 7f9a3d200640 1 -- 172.21.15.131:0/3279787108 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55bd56497540 msgr2=0x55bd564968c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:58.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.905+0000 7f9a3d200640 1 --2- 172.21.15.131:0/3279787108 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55bd56497540 0x55bd564968c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:58.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.905+0000 7f9a3d200640 1 -- 172.21.15.131:0/3279787108 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bd56496e00 msgr2=0x55bd5646bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:58.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.905+0000 7f9a3d200640 1 --2- 172.21.15.131:0/3279787108 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bd56496e00 0x55bd5646bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:58.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.905+0000 7f9a3d200640 1 -- 172.21.15.131:0/3279787108 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd561ad7d0 con 0x55bd564b4420 2024-05-26T22:50:58.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.905+0000 7f9a3d200640 1 --2- 172.21.15.131:0/3279787108 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bd564b4420 0x55bd56466db0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f9a280a9810 tx=0x7f9a28079160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5183003618fb0bd1 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:58.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.906+0000 7f9a2d600640 1 -- 172.21.15.131:0/3279787108 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a28079740 con 0x55bd564b4420 2024-05-26T22:50:58.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.906+0000 7f9a2d600640 1 -- 172.21.15.131:0/3279787108 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a2807daf0 con 0x55bd564b4420 2024-05-26T22:50:58.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.906+0000 7f9a2d600640 1 -- 172.21.15.131:0/3279787108 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a28082320 con 0x55bd564b4420 2024-05-26T22:50:58.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.906+0000 7f9a42b72740 1 -- 172.21.15.131:0/3279787108 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bd564b4420 msgr2=0x55bd56466db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:58.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.906+0000 7f9a42b72740 1 --2- 172.21.15.131:0/3279787108 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bd564b4420 0x55bd56466db0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f9a280a9810 tx=0x7f9a28079160 comp rx=0 tx=0).stop 2024-05-26T22:50:58.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.906+0000 7f9a42b72740 1 -- 172.21.15.131:0/3279787108 shutdown_connections 2024-05-26T22:50:58.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.906+0000 7f9a42b72740 1 --2- 172.21.15.131:0/3279787108 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55bd56497540 0x55bd564968c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:58.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.906+0000 7f9a42b72740 1 --2- 172.21.15.131:0/3279787108 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bd56496e00 0x55bd5646bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:58.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.906+0000 7f9a42b72740 1 --2- 172.21.15.131:0/3279787108 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bd564b4420 0x55bd56466db0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:58.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.906+0000 7f9a42b72740 1 -- 172.21.15.131:0/3279787108 >> 172.21.15.131:0/3279787108 conn(0x55bd56469020 msgr2=0x55bd564b18c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:58.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.907+0000 7f9a42b72740 1 -- 172.21.15.131:0/3279787108 shutdown_connections 2024-05-26T22:50:58.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.907+0000 7f9a42b72740 1 -- 172.21.15.131:0/3279787108 wait complete. 2024-05-26T22:50:58.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.907+0000 7f9a42b72740 1 -- start start 2024-05-26T22:50:58.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.907+0000 7f9a42b72740 1 Processor -- start 2024-05-26T22:50:58.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.908+0000 7f9a42b72740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55bd56497540 0x55bd56493820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:58.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.908+0000 7f9a42b72740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bd564b4420 0x55bd56499130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:58.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.908+0000 7f9a42b72740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bd56499670 0x55bd564918a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:58.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.908+0000 7f9a42b72740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55bd563cb920 con 0x55bd564b4420 2024-05-26T22:50:58.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.908+0000 7f9a42b72740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55bd5638a4a0 con 0x55bd56499670 2024-05-26T22:50:58.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.908+0000 7f9a42b72740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x55bd563cf960 con 0x55bd56497540 2024-05-26T22:50:58.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.909+0000 7f9a42b72740 10 client.0 _ll_register_callbacks cb 0x55bd56391c40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-26T22:50:58.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.909+0000 7f9a37e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bd564b4420 0x55bd56499130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:58.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.909+0000 7f9a37e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bd564b4420 0x55bd56499130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55098/0 (socket says 172.21.15.131:55098) 2024-05-26T22:50:58.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.909+0000 7f9a37e00640 1 -- 172.21.15.131:0/3063629518 learned_addr learned my addr 172.21.15.131:0/3063629518 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:58.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.909+0000 7f9a3dc00640 1 --2- 172.21.15.131:0/3063629518 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bd56499670 0x55bd564918a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:58.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.909+0000 7f9a3d200640 1 --2- 172.21.15.131:0/3063629518 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55bd56497540 0x55bd56493820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:58.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.909+0000 7f9a37e00640 1 -- 172.21.15.131:0/3063629518 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55bd56497540 msgr2=0x55bd56493820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:58.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.909+0000 7f9a37e00640 1 --2- 172.21.15.131:0/3063629518 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55bd56497540 0x55bd56493820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:58.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.909+0000 7f9a37e00640 1 -- 172.21.15.131:0/3063629518 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bd56499670 msgr2=0x55bd564918a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:58.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.909+0000 7f9a37e00640 1 --2- 172.21.15.131:0/3063629518 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bd56499670 0x55bd564918a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:58.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.909+0000 7f9a37e00640 1 -- 172.21.15.131:0/3063629518 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd560a30f0 con 0x55bd564b4420 2024-05-26T22:50:58.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.909+0000 7f9a37e00640 1 --2- 172.21.15.131:0/3063629518 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bd564b4420 0x55bd56499130 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f9a24007380 tx=0x7f9a24007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8db754e3c771512b server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:58.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.909+0000 7f9a22000640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-26T22:50:58.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.910+0000 7f9a22000640 1 -- 172.21.15.131:0/3063629518 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a24021620 con 0x55bd564b4420 2024-05-26T22:50:58.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.910+0000 7f9a22000640 1 -- 172.21.15.131:0/3063629518 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f9a24012410 con 0x55bd564b4420 2024-05-26T22:50:58.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.910+0000 7f9a42b72740 1 -- 172.21.15.131:0/3063629518 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a2807daf0 con 0x55bd564b4420 2024-05-26T22:50:58.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.910+0000 7f9a22000640 1 -- 172.21.15.131:0/3063629518 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f9a24012610 con 0x55bd564b4420 2024-05-26T22:50:58.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.910+0000 7f9a42b72740 10 client.5967 fetch_fsmap learned FSMap version 34 2024-05-26T22:50:58.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.910+0000 7f9a42b72740 1 -- 172.21.15.131:0/3063629518 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) -- 0x55bd5620df60 con 0x55bd564b4420 2024-05-26T22:50:58.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.910+0000 7f9a22000640 1 -- 172.21.15.131:0/3063629518 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a2807daf0 con 0x55bd564b4420 2024-05-26T22:50:58.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.910+0000 7f9a22000640 1 -- 172.21.15.131:0/3063629518 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 34) ==== 3542+0+0 (secure 0 0 0) 0x7f9a240217c0 con 0x55bd564b4420 2024-05-26T22:50:58.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.910+0000 7f9a42b72740 10 client.5967 fetch_fsmap finished waiting for FSMap version 34 2024-05-26T22:50:58.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.910+0000 7f9a42b72740 10 client.5967 resolve_mds: resolved 3:1 to role '3:1' aka daemon mds.d 2024-05-26T22:50:58.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.911+0000 7f9a42b72740 20 client.5967 populate_metadata read hostname 'smithi131' 2024-05-26T22:50:58.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.911+0000 7f9a42b72740 1 --2- 172.21.15.131:0/3063629518 >> [v2:172.21.15.187:6833/3429891101,v1:172.21.15.187:6835/3429891101] conn(0x55bd56497f50 0x55bd56498330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:58.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.911+0000 7f9a42b72740 4 client.5967 mds_command: new command op to 4918 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-26T22:50:58.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.911+0000 7f9a42b72740 1 -- 172.21.15.131:0/3063629518 --> [v2:172.21.15.187:6833/3429891101,v1:172.21.15.187:6835/3429891101] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bd563dd090 con 0x55bd56497f50 2024-05-26T22:50:58.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.911+0000 7f9a3d200640 1 --2- 172.21.15.131:0/3063629518 >> [v2:172.21.15.187:6833/3429891101,v1:172.21.15.187:6835/3429891101] conn(0x55bd56497f50 0x55bd56498330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:58.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.912+0000 7f9a3d200640 1 --2- 172.21.15.131:0/3063629518 >> [v2:172.21.15.187:6833/3429891101,v1:172.21.15.187:6835/3429891101] conn(0x55bd56497f50 0x55bd56498330 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=81d006fa79b6849c server_cookie=1b6ec2329b649989 in_seq=0 out_seq=0 2024-05-26T22:50:58.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.912+0000 7f9a22000640 10 client.5967 ms_handle_connect on v2:172.21.15.187:6833/3429891101 2024-05-26T22:50:58.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.914+0000 7f9a22000640 1 -- 172.21.15.131:0/3063629518 <== mds.1 v2:172.21.15.187:6833/3429891101 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55bd563dd090 con 0x55bd56497f50 2024-05-26T22:50:58.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.914+0000 7f9a22000640 10 client.5967 handle_command_reply: tid=0 2024-05-26T22:50:58.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.914+0000 7f9a42b72740 1 client.5967 shutdown 2024-05-26T22:50:58.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.914+0000 7f9a42b72740 1 -- 172.21.15.131:0/3063629518 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bd564b4420 msgr2=0x55bd56499130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:58.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.914+0000 7f9a42b72740 1 --2- 172.21.15.131:0/3063629518 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bd564b4420 0x55bd56499130 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f9a24007380 tx=0x7f9a24007780 comp rx=0 tx=0).stop 2024-05-26T22:50:58.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.914+0000 7f9a42b72740 1 -- 172.21.15.131:0/3063629518 shutdown_connections 2024-05-26T22:50:58.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.914+0000 7f9a42b72740 1 --2- 172.21.15.131:0/3063629518 >> [v2:172.21.15.187:6833/3429891101,v1:172.21.15.187:6835/3429891101] conn(0x55bd56497f50 0x55bd56498330 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:58.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.914+0000 7f9a42b72740 1 --2- 172.21.15.131:0/3063629518 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bd56499670 0x55bd564918a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:58.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.914+0000 7f9a42b72740 1 --2- 172.21.15.131:0/3063629518 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bd564b4420 0x55bd56499130 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:58.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.914+0000 7f9a22000640 0 client.5967 ms_handle_reset on v2:172.21.15.187:6833/3429891101 2024-05-26T22:50:58.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.914+0000 7f9a42b72740 1 --2- 172.21.15.131:0/3063629518 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55bd56497540 0x55bd56493820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:58.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.914+0000 7f9a42b72740 1 -- 172.21.15.131:0/3063629518 >> 172.21.15.131:0/3063629518 conn(0x55bd56496e00 msgr2=0x55bd56467af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:58.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.914+0000 7f9a42b72740 1 -- 172.21.15.131:0/3063629518 shutdown_connections 2024-05-26T22:50:58.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.914+0000 7f9a42b72740 1 -- 172.21.15.131:0/3063629518 wait complete. 2024-05-26T22:50:58.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.915+0000 7f9a42b72740 20 client.5967 trim_cache size 0 max 16384 2024-05-26T22:50:58.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.933+0000 7f9a42b72740 1 Processor -- start 2024-05-26T22:50:58.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.933+0000 7f9a42b72740 1 -- start start 2024-05-26T22:50:58.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.934+0000 7f9a42b72740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bd56497540 0x55bd564b0890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:58.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.934+0000 7f9a42b72740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55bd56499670 0x55bd56499fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:58.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.934+0000 7f9a42b72740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bd564b4420 0x55bd5649a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:58.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.934+0000 7f9a42b72740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f9a382013a0 con 0x55bd56497540 2024-05-26T22:50:58.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.934+0000 7f9a42b72740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9a382005a0 con 0x55bd564b4420 2024-05-26T22:50:58.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.934+0000 7f9a42b72740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9a38200c70 con 0x55bd56499670 2024-05-26T22:50:58.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.934+0000 7f9a3d200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bd56497540 0x55bd564b0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:58.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.934+0000 7f9a3d200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bd56497540 0x55bd564b0890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55112/0 (socket says 172.21.15.131:55112) 2024-05-26T22:50:58.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.934+0000 7f9a3d200640 1 -- 172.21.15.131:0/3748147296 learned_addr learned my addr 172.21.15.131:0/3748147296 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:58.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.934+0000 7f9a3d200640 1 -- 172.21.15.131:0/3748147296 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55bd56499670 msgr2=0x55bd56499fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:58.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.934+0000 7f9a3d200640 1 --2- 172.21.15.131:0/3748147296 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55bd56499670 0x55bd56499fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:58.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.934+0000 7f9a3d200640 1 -- 172.21.15.131:0/3748147296 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bd564b4420 msgr2=0x55bd5649a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:58.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.934+0000 7f9a3d200640 1 --2- 172.21.15.131:0/3748147296 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bd564b4420 0x55bd5649a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:58.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.934+0000 7f9a3d200640 1 -- 172.21.15.131:0/3748147296 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd56493b90 con 0x55bd56497540 2024-05-26T22:50:58.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.935+0000 7f9a3d200640 1 --2- 172.21.15.131:0/3748147296 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bd56497540 0x55bd564b0890 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f9a28000c00 tx=0x7f9a2803e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fbeb05cc160bfb server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:58.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.935+0000 7f9a2cc00640 1 -- 172.21.15.131:0/3748147296 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a28079740 con 0x55bd56497540 2024-05-26T22:50:58.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.935+0000 7f9a2cc00640 1 -- 172.21.15.131:0/3748147296 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a28077a40 con 0x55bd56497540 2024-05-26T22:50:58.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.935+0000 7f9a2cc00640 1 -- 172.21.15.131:0/3748147296 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a280ab0b0 con 0x55bd56497540 2024-05-26T22:50:58.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.935+0000 7f9a42b72740 1 -- 172.21.15.131:0/3748147296 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bd56497540 msgr2=0x55bd564b0890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:58.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.935+0000 7f9a42b72740 1 --2- 172.21.15.131:0/3748147296 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bd56497540 0x55bd564b0890 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f9a28000c00 tx=0x7f9a2803e000 comp rx=0 tx=0).stop 2024-05-26T22:50:58.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.935+0000 7f9a42b72740 1 -- 172.21.15.131:0/3748147296 shutdown_connections 2024-05-26T22:50:58.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.935+0000 7f9a42b72740 1 --2- 172.21.15.131:0/3748147296 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55bd56499670 0x55bd56499fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:58.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.935+0000 7f9a42b72740 1 --2- 172.21.15.131:0/3748147296 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bd564b4420 0x55bd5649a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:58.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.935+0000 7f9a42b72740 1 --2- 172.21.15.131:0/3748147296 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bd56497540 0x55bd564b0890 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:58.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.935+0000 7f9a42b72740 1 -- 172.21.15.131:0/3748147296 >> 172.21.15.131:0/3748147296 conn(0x55bd56496e00 msgr2=0x55bd56462460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:58.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.936+0000 7f9a42b72740 1 -- 172.21.15.131:0/3748147296 shutdown_connections 2024-05-26T22:50:58.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.936+0000 7f9a42b72740 1 -- 172.21.15.131:0/3748147296 wait complete. 2024-05-26T22:50:58.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.936+0000 7f9a42b72740 1 -- start start 2024-05-26T22:50:58.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.936+0000 7f9a42b72740 1 Processor -- start 2024-05-26T22:50:58.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.937+0000 7f9a42b72740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bd56497540 0x55bd565371e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:58.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.937+0000 7f9a42b72740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55bd56499670 0x55bd56490080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:58.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.937+0000 7f9a42b72740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bd564b4420 0x55bd564d38c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:58.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.937+0000 7f9a42b72740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55bd563770d0 con 0x55bd564b4420 2024-05-26T22:50:58.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.937+0000 7f9a42b72740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9a3814f9b0 con 0x55bd56497540 2024-05-26T22:50:58.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.937+0000 7f9a42b72740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x55bd5639b4d0 con 0x55bd56499670 2024-05-26T22:50:58.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.937+0000 7f9a42b72740 10 client.0 _ll_register_callbacks cb 0x7f9a240057f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-26T22:50:58.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.938+0000 7f9a3dc00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bd564b4420 0x55bd564d38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:58.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.938+0000 7f9a3dc00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bd564b4420 0x55bd564d38c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55118/0 (socket says 172.21.15.131:55118) 2024-05-26T22:50:58.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.938+0000 7f9a3dc00640 1 -- 172.21.15.131:0/2017099871 learned_addr learned my addr 172.21.15.131:0/2017099871 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:58.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.938+0000 7f9a37e00640 1 --2- 172.21.15.131:0/2017099871 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55bd56499670 0x55bd56490080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:58.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.938+0000 7f9a3d200640 1 --2- 172.21.15.131:0/2017099871 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bd56497540 0x55bd565371e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:58.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.938+0000 7f9a3dc00640 1 -- 172.21.15.131:0/2017099871 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55bd56499670 msgr2=0x55bd56490080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:58.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.938+0000 7f9a3dc00640 1 --2- 172.21.15.131:0/2017099871 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55bd56499670 0x55bd56490080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:58.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.938+0000 7f9a3dc00640 1 -- 172.21.15.131:0/2017099871 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bd56497540 msgr2=0x55bd565371e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:58.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.938+0000 7f9a3dc00640 1 --2- 172.21.15.131:0/2017099871 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bd56497540 0x55bd565371e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:58.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.938+0000 7f9a3dc00640 1 -- 172.21.15.131:0/2017099871 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd564986a0 con 0x55bd564b4420 2024-05-26T22:50:58.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.938+0000 7f9a3dc00640 1 --2- 172.21.15.131:0/2017099871 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bd564b4420 0x55bd564d38c0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f9a380cd810 tx=0x7f9a30007c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cda660659579319 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:58.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.938+0000 7f9a22000640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-26T22:50:58.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.939+0000 7f9a22000640 1 -- 172.21.15.131:0/2017099871 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a30018020 con 0x55bd564b4420 2024-05-26T22:50:58.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.939+0000 7f9a22000640 1 -- 172.21.15.131:0/2017099871 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f9a3000f820 con 0x55bd564b4420 2024-05-26T22:50:58.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.939+0000 7f9a42b72740 1 -- 172.21.15.131:0/2017099871 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a28077a40 con 0x55bd564b4420 2024-05-26T22:50:58.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.939+0000 7f9a22000640 1 -- 172.21.15.131:0/2017099871 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f9a30022050 con 0x55bd564b4420 2024-05-26T22:50:58.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.939+0000 7f9a42b72740 10 client.5973 fetch_fsmap learned FSMap version 34 2024-05-26T22:50:58.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.939+0000 7f9a42b72740 1 -- 172.21.15.131:0/2017099871 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) -- 0x55bd5607c1b0 con 0x55bd564b4420 2024-05-26T22:50:58.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.939+0000 7f9a22000640 1 -- 172.21.15.131:0/2017099871 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a28077a40 con 0x55bd564b4420 2024-05-26T22:50:58.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.939+0000 7f9a22000640 1 -- 172.21.15.131:0/2017099871 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 34) ==== 3542+0+0 (secure 0 0 0) 0x7f9a30025020 con 0x55bd564b4420 2024-05-26T22:50:58.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.939+0000 7f9a42b72740 10 client.5973 fetch_fsmap finished waiting for FSMap version 34 2024-05-26T22:50:58.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.939+0000 7f9a42b72740 10 client.5973 resolve_mds: resolved 3:1 to role '3:1' aka daemon mds.d 2024-05-26T22:50:58.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.939+0000 7f9a42b72740 20 client.5973 populate_metadata read hostname 'smithi131' 2024-05-26T22:50:58.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.939+0000 7f9a42b72740 1 --2- 172.21.15.131:0/2017099871 >> [v2:172.21.15.187:6833/3429891101,v1:172.21.15.187:6835/3429891101] conn(0x55bd564d3e00 0x55bd56492730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:58.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.939+0000 7f9a42b72740 4 client.5973 mds_command: new command op to 4918 tid=0 multi_id=0 [{"prefix": "perf dump"}] 2024-05-26T22:50:58.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.939+0000 7f9a42b72740 1 -- 172.21.15.131:0/2017099871 --> [v2:172.21.15.187:6833/3429891101,v1:172.21.15.187:6835/3429891101] -- command(tid 0: {"prefix": "perf dump"}) -- 0x55bd563d73e0 con 0x55bd564d3e00 2024-05-26T22:50:58.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.940+0000 7f9a3d200640 1 --2- 172.21.15.131:0/2017099871 >> [v2:172.21.15.187:6833/3429891101,v1:172.21.15.187:6835/3429891101] conn(0x55bd564d3e00 0x55bd56492730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:58.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.941+0000 7f9a3d200640 1 --2- 172.21.15.131:0/2017099871 >> [v2:172.21.15.187:6833/3429891101,v1:172.21.15.187:6835/3429891101] conn(0x55bd564d3e00 0x55bd56492730 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=414783fdf7382955 server_cookie=e38420a260e81056 in_seq=0 out_seq=0 2024-05-26T22:50:58.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.941+0000 7f9a22000640 10 client.5973 ms_handle_connect on v2:172.21.15.187:6833/3429891101 2024-05-26T22:50:58.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.942+0000 7f9a22000640 1 -- 172.21.15.131:0/2017099871 <== mds.1 v2:172.21.15.187:6833/3429891101 1 ==== command_reply(tid 0: 0 ) ==== 8+0+17713 (crc 0 0 0) 0x55bd563d73e0 con 0x55bd564d3e00 2024-05-26T22:50:58.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.942+0000 7f9a22000640 10 client.5973 handle_command_reply: tid=0 2024-05-26T22:50:58.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.942+0000 7f9a42b72740 1 client.5973 shutdown 2024-05-26T22:50:58.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.943+0000 7f9a42b72740 1 -- 172.21.15.131:0/2017099871 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bd564b4420 msgr2=0x55bd564d38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:58.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.943+0000 7f9a42b72740 1 --2- 172.21.15.131:0/2017099871 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bd564b4420 0x55bd564d38c0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f9a380cd810 tx=0x7f9a30007c20 comp rx=0 tx=0).stop 2024-05-26T22:50:58.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.943+0000 7f9a42b72740 1 -- 172.21.15.131:0/2017099871 shutdown_connections 2024-05-26T22:50:58.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.943+0000 7f9a42b72740 1 --2- 172.21.15.131:0/2017099871 >> [v2:172.21.15.187:6833/3429891101,v1:172.21.15.187:6835/3429891101] conn(0x55bd564d3e00 0x55bd56492730 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:58.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.943+0000 7f9a42b72740 1 --2- 172.21.15.131:0/2017099871 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55bd56499670 0x55bd56490080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:58.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.943+0000 7f9a42b72740 1 --2- 172.21.15.131:0/2017099871 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bd564b4420 0x55bd564d38c0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:58.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.943+0000 7f9a42b72740 1 --2- 172.21.15.131:0/2017099871 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bd56497540 0x55bd565371e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:58.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.943+0000 7f9a22000640 0 client.5973 ms_handle_reset on v2:172.21.15.187:6833/3429891101 2024-05-26T22:50:58.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.943+0000 7f9a42b72740 1 -- 172.21.15.131:0/2017099871 >> 172.21.15.131:0/2017099871 conn(0x55bd56496e00 msgr2=0x55bd564623a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:58.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.943+0000 7f9a42b72740 1 -- 172.21.15.131:0/2017099871 shutdown_connections 2024-05-26T22:50:58.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.943+0000 7f9a42b72740 1 -- 172.21.15.131:0/2017099871 wait complete. 2024-05-26T22:50:58.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.943+0000 7f9a42b72740 20 client.5973 trim_cache size 0 max 16384 2024-05-26T22:50:58.946 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-26T22:50:58.946 INFO:teuthology.orchestra.run.smithi131.stdout: "AsyncMessenger::Worker-0": { 2024-05-26T22:50:58.946 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_messages": 14, 2024-05-26T22:50:58.946 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages": 13, 2024-05-26T22:50:58.946 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_bytes": 4085, 2024-05-26T22:50:58.946 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_bytes": 31689, 2024-05-26T22:50:58.946 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_created_connections": 9, 2024-05-26T22:50:58.946 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_active_connections": 4, 2024-05-26T22:50:58.946 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_total_time": 0.015199249, 2024-05-26T22:50:58.946 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_send_time": 0.000972971, 2024-05-26T22:50:58.947 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_recv_time": 0.009410690, 2024-05-26T22:50:58.947 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_fast_dispatch_time": 0.000356217, 2024-05-26T22:50:58.947 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages_queue_lat": { 2024-05-26T22:50:58.947 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 12, 2024-05-26T22:50:58.947 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.025062237, 2024-05-26T22:50:58.947 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.002088519 2024-05-26T22:50:58.947 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.947 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_handle_ack_lat": { 2024-05-26T22:50:58.947 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 2, 2024-05-26T22:50:58.947 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000438, 2024-05-26T22:50:58.947 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000219 2024-05-26T22:50:58.947 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.947 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_encrypted_bytes": 1568, 2024-05-26T22:50:58.948 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_encrypted_bytes": 448 2024-05-26T22:50:58.948 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.948 INFO:teuthology.orchestra.run.smithi131.stdout: "AsyncMessenger::Worker-1": { 2024-05-26T22:50:58.948 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_messages": 83, 2024-05-26T22:50:58.948 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages": 149, 2024-05-26T22:50:58.948 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_bytes": 126051, 2024-05-26T22:50:58.948 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_bytes": 149778, 2024-05-26T22:50:58.948 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_created_connections": 8, 2024-05-26T22:50:58.948 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_active_connections": 3, 2024-05-26T22:50:58.948 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_total_time": 0.022422762, 2024-05-26T22:50:58.948 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_send_time": 0.007556041, 2024-05-26T22:50:58.948 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_recv_time": 0.018468844, 2024-05-26T22:50:58.949 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_fast_dispatch_time": 0.001249201, 2024-05-26T22:50:58.949 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages_queue_lat": { 2024-05-26T22:50:58.949 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 148, 2024-05-26T22:50:58.949 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.047022488, 2024-05-26T22:50:58.949 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000317719 2024-05-26T22:50:58.949 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.949 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_handle_ack_lat": { 2024-05-26T22:50:58.949 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 7, 2024-05-26T22:50:58.949 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000006106, 2024-05-26T22:50:58.949 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000872 2024-05-26T22:50:58.949 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.949 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_encrypted_bytes": 120192, 2024-05-26T22:50:58.949 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_encrypted_bytes": 13456 2024-05-26T22:50:58.950 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.950 INFO:teuthology.orchestra.run.smithi131.stdout: "AsyncMessenger::Worker-2": { 2024-05-26T22:50:58.950 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_messages": 53, 2024-05-26T22:50:58.950 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages": 74, 2024-05-26T22:50:58.950 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_bytes": 113682, 2024-05-26T22:50:58.950 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_bytes": 122151, 2024-05-26T22:50:58.950 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_created_connections": 6, 2024-05-26T22:50:58.950 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_active_connections": 4, 2024-05-26T22:50:58.950 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_total_time": 0.018125761, 2024-05-26T22:50:58.950 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_send_time": 0.006143144, 2024-05-26T22:50:58.950 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_recv_time": 0.013812451, 2024-05-26T22:50:58.950 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_fast_dispatch_time": 0.000466651, 2024-05-26T22:50:58.950 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages_queue_lat": { 2024-05-26T22:50:58.951 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 73, 2024-05-26T22:50:58.951 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.043819053, 2024-05-26T22:50:58.951 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000600261 2024-05-26T22:50:58.951 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.951 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_handle_ack_lat": { 2024-05-26T22:50:58.951 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 77, 2024-05-26T22:50:58.951 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000028961, 2024-05-26T22:50:58.951 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000376 2024-05-26T22:50:58.951 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.951 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-26T22:50:58.951 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_encrypted_bytes": 95680 2024-05-26T22:50:58.951 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.951 INFO:teuthology.orchestra.run.smithi131.stdout: "cct": { 2024-05-26T22:50:58.952 INFO:teuthology.orchestra.run.smithi131.stdout: "total_workers": 1, 2024-05-26T22:50:58.952 INFO:teuthology.orchestra.run.smithi131.stdout: "unhealthy_workers": 0 2024-05-26T22:50:58.952 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.952 INFO:teuthology.orchestra.run.smithi131.stdout: "finisher-MDSRank": { 2024-05-26T22:50:58.952 INFO:teuthology.orchestra.run.smithi131.stdout: "queue_len": 0, 2024-05-26T22:50:58.952 INFO:teuthology.orchestra.run.smithi131.stdout: "complete_latency": { 2024-05-26T22:50:58.952 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 52, 2024-05-26T22:50:58.952 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.007747724, 2024-05-26T22:50:58.952 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000148994 2024-05-26T22:50:58.952 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:50:58.952 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.952 INFO:teuthology.orchestra.run.smithi131.stdout: "finisher-PurgeQueue": { 2024-05-26T22:50:58.952 INFO:teuthology.orchestra.run.smithi131.stdout: "queue_len": 0, 2024-05-26T22:50:58.953 INFO:teuthology.orchestra.run.smithi131.stdout: "complete_latency": { 2024-05-26T22:50:58.953 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 2, 2024-05-26T22:50:58.953 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000024412, 2024-05-26T22:50:58.953 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000012206 2024-05-26T22:50:58.953 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:50:58.953 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.953 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": { 2024-05-26T22:50:58.953 INFO:teuthology.orchestra.run.smithi131.stdout: "request": 0, 2024-05-26T22:50:58.953 INFO:teuthology.orchestra.run.smithi131.stdout: "reply": 0, 2024-05-26T22:50:58.954 INFO:teuthology.orchestra.run.smithi131.stdout: "reply_latency": { 2024-05-26T22:50:58.954 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:50:58.954 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:50:58.954 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:50:58.954 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.954 INFO:teuthology.orchestra.run.smithi131.stdout: "slow_reply": 0, 2024-05-26T22:50:58.954 INFO:teuthology.orchestra.run.smithi131.stdout: "forward": 0, 2024-05-26T22:50:58.954 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_fetch_complete": 0, 2024-05-26T22:50:58.954 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_fetch_keys": 0, 2024-05-26T22:50:58.954 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_commit": 11, 2024-05-26T22:50:58.954 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_split": 0, 2024-05-26T22:50:58.954 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_merge": 0, 2024-05-26T22:50:58.954 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes": 10, 2024-05-26T22:50:58.955 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_top": 0, 2024-05-26T22:50:58.955 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_bottom": 10, 2024-05-26T22:50:58.955 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_pin_tail": 0, 2024-05-26T22:50:58.955 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_pinned": 10, 2024-05-26T22:50:58.955 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_expired": 0, 2024-05-26T22:50:58.955 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_with_caps": 0, 2024-05-26T22:50:58.955 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": 0, 2024-05-26T22:50:58.955 INFO:teuthology.orchestra.run.smithi131.stdout: "subtrees": 1, 2024-05-26T22:50:58.955 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse": 3, 2024-05-26T22:50:58.955 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_hit": 0, 2024-05-26T22:50:58.955 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_forward": 0, 2024-05-26T22:50:58.955 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_discover": 3, 2024-05-26T22:50:58.956 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_dir_fetch": 0, 2024-05-26T22:50:58.956 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_remote_ino": 0, 2024-05-26T22:50:58.956 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_lock": 0, 2024-05-26T22:50:58.956 INFO:teuthology.orchestra.run.smithi131.stdout: "load_cent": 0, 2024-05-26T22:50:58.956 INFO:teuthology.orchestra.run.smithi131.stdout: "q": 0, 2024-05-26T22:50:58.956 INFO:teuthology.orchestra.run.smithi131.stdout: "exported": 0, 2024-05-26T22:50:58.956 INFO:teuthology.orchestra.run.smithi131.stdout: "exported_inodes": 0, 2024-05-26T22:50:58.956 INFO:teuthology.orchestra.run.smithi131.stdout: "imported": 3, 2024-05-26T22:50:58.956 INFO:teuthology.orchestra.run.smithi131.stdout: "imported_inodes": 74, 2024-05-26T22:50:58.956 INFO:teuthology.orchestra.run.smithi131.stdout: "openino_dir_fetch": 0, 2024-05-26T22:50:58.956 INFO:teuthology.orchestra.run.smithi131.stdout: "openino_backtrace_fetch": 0, 2024-05-26T22:50:58.956 INFO:teuthology.orchestra.run.smithi131.stdout: "openino_peer_discover": 0, 2024-05-26T22:50:58.957 INFO:teuthology.orchestra.run.smithi131.stdout: "root_rfiles": 100, 2024-05-26T22:50:58.957 INFO:teuthology.orchestra.run.smithi131.stdout: "root_rbytes": 500, 2024-05-26T22:50:58.957 INFO:teuthology.orchestra.run.smithi131.stdout: "root_rsnaps": 0, 2024-05-26T22:50:58.957 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_backtrace_fetch": 0, 2024-05-26T22:50:58.957 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_set_tag": 0, 2024-05-26T22:50:58.957 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_backtrace_repaired": 0, 2024-05-26T22:50:58.957 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_inotable_repaired": 0, 2024-05-26T22:50:58.957 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_dir_inodes": 0, 2024-05-26T22:50:58.957 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_dir_base_inodes": 0, 2024-05-26T22:50:58.957 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_dirfrag_rstats": 0, 2024-05-26T22:50:58.957 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_file_inodes": 0, 2024-05-26T22:50:58.957 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_inode_file_caps": 0, 2024-05-26T22:50:58.957 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_revoke": 0, 2024-05-26T22:50:58.958 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_grant": 0, 2024-05-26T22:50:58.958 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_trunc": 0, 2024-05-26T22:50:58.958 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-26T22:50:58.958 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-26T22:50:58.958 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_caps": 0, 2024-05-26T22:50:58.958 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_caps_dirty": 0, 2024-05-26T22:50:58.958 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_cap_release": 0, 2024-05-26T22:50:58.958 INFO:teuthology.orchestra.run.smithi131.stdout: "process_request_cap_release": 0 2024-05-26T22:50:58.958 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.958 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_cache": { 2024-05-26T22:50:58.958 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_update": 0, 2024-05-26T22:50:58.959 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_update_receipt": 0, 2024-05-26T22:50:58.959 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_try_discover": 0, 2024-05-26T22:50:58.959 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_send_discover": 4, 2024-05-26T22:50:58.959 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_handle_discover": 0, 2024-05-26T22:50:58.959 INFO:teuthology.orchestra.run.smithi131.stdout: "num_strays": 0, 2024-05-26T22:50:58.959 INFO:teuthology.orchestra.run.smithi131.stdout: "num_strays_delayed": 0, 2024-05-26T22:50:58.959 INFO:teuthology.orchestra.run.smithi131.stdout: "num_strays_enqueuing": 0, 2024-05-26T22:50:58.959 INFO:teuthology.orchestra.run.smithi131.stdout: "strays_created": 0, 2024-05-26T22:50:58.959 INFO:teuthology.orchestra.run.smithi131.stdout: "strays_enqueued": 0, 2024-05-26T22:50:58.959 INFO:teuthology.orchestra.run.smithi131.stdout: "strays_reintegrated": 0, 2024-05-26T22:50:58.959 INFO:teuthology.orchestra.run.smithi131.stdout: "strays_migrated": 0, 2024-05-26T22:50:58.960 INFO:teuthology.orchestra.run.smithi131.stdout: "num_recovering_processing": 0, 2024-05-26T22:50:58.960 INFO:teuthology.orchestra.run.smithi131.stdout: "num_recovering_enqueued": 0, 2024-05-26T22:50:58.960 INFO:teuthology.orchestra.run.smithi131.stdout: "num_recovering_prioritized": 0, 2024-05-26T22:50:58.960 INFO:teuthology.orchestra.run.smithi131.stdout: "recovery_started": 0, 2024-05-26T22:50:58.960 INFO:teuthology.orchestra.run.smithi131.stdout: "recovery_completed": 0, 2024-05-26T22:50:58.960 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_quiesce_path": 0, 2024-05-26T22:50:58.960 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_quiesce_inode": 0, 2024-05-26T22:50:58.960 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_enqueue_scrub": 0, 2024-05-26T22:50:58.960 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_exportdir": 0, 2024-05-26T22:50:58.960 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_flush": 0, 2024-05-26T22:50:58.960 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_fragmentdir": 0, 2024-05-26T22:50:58.961 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_fragstats": 0, 2024-05-26T22:50:58.961 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_inodestats": 0 2024-05-26T22:50:58.961 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.961 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_log": { 2024-05-26T22:50:58.961 INFO:teuthology.orchestra.run.smithi131.stdout: "evlrg": 0, 2024-05-26T22:50:58.961 INFO:teuthology.orchestra.run.smithi131.stdout: "evadd": 8, 2024-05-26T22:50:58.961 INFO:teuthology.orchestra.run.smithi131.stdout: "evex": 0, 2024-05-26T22:50:58.961 INFO:teuthology.orchestra.run.smithi131.stdout: "evtrm": 0, 2024-05-26T22:50:58.961 INFO:teuthology.orchestra.run.smithi131.stdout: "ev": 8, 2024-05-26T22:50:58.961 INFO:teuthology.orchestra.run.smithi131.stdout: "evexg": 0, 2024-05-26T22:50:58.961 INFO:teuthology.orchestra.run.smithi131.stdout: "evexd": 0, 2024-05-26T22:50:58.962 INFO:teuthology.orchestra.run.smithi131.stdout: "segadd": 2, 2024-05-26T22:50:58.962 INFO:teuthology.orchestra.run.smithi131.stdout: "segex": 0, 2024-05-26T22:50:58.962 INFO:teuthology.orchestra.run.smithi131.stdout: "segtrm": 0, 2024-05-26T22:50:58.962 INFO:teuthology.orchestra.run.smithi131.stdout: "seg": 2, 2024-05-26T22:50:58.962 INFO:teuthology.orchestra.run.smithi131.stdout: "segmjr": 2, 2024-05-26T22:50:58.962 INFO:teuthology.orchestra.run.smithi131.stdout: "segexg": 0, 2024-05-26T22:50:58.962 INFO:teuthology.orchestra.run.smithi131.stdout: "segexd": 0, 2024-05-26T22:50:58.962 INFO:teuthology.orchestra.run.smithi131.stdout: "expos": 4194304, 2024-05-26T22:50:58.962 INFO:teuthology.orchestra.run.smithi131.stdout: "wrpos": 4255815, 2024-05-26T22:50:58.962 INFO:teuthology.orchestra.run.smithi131.stdout: "rdpos": 0, 2024-05-26T22:50:58.962 INFO:teuthology.orchestra.run.smithi131.stdout: "jlat": { 2024-05-26T22:50:58.963 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 4, 2024-05-26T22:50:58.963 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.023546975, 2024-05-26T22:50:58.963 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.005886743 2024-05-26T22:50:58.963 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.963 INFO:teuthology.orchestra.run.smithi131.stdout: "replayed": 0 2024-05-26T22:50:58.963 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.963 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_mem": { 2024-05-26T22:50:58.963 INFO:teuthology.orchestra.run.smithi131.stdout: "ino": 88, 2024-05-26T22:50:58.963 INFO:teuthology.orchestra.run.smithi131.stdout: "ino+": 88, 2024-05-26T22:50:58.963 INFO:teuthology.orchestra.run.smithi131.stdout: "ino-": 0, 2024-05-26T22:50:58.963 INFO:teuthology.orchestra.run.smithi131.stdout: "dir": 52, 2024-05-26T22:50:58.963 INFO:teuthology.orchestra.run.smithi131.stdout: "dir+": 53, 2024-05-26T22:50:58.963 INFO:teuthology.orchestra.run.smithi131.stdout: "dir-": 1, 2024-05-26T22:50:58.964 INFO:teuthology.orchestra.run.smithi131.stdout: "dn": 85, 2024-05-26T22:50:58.964 INFO:teuthology.orchestra.run.smithi131.stdout: "dn+": 85, 2024-05-26T22:50:58.964 INFO:teuthology.orchestra.run.smithi131.stdout: "dn-": 0, 2024-05-26T22:50:58.964 INFO:teuthology.orchestra.run.smithi131.stdout: "cap": 74, 2024-05-26T22:50:58.964 INFO:teuthology.orchestra.run.smithi131.stdout: "cap+": 74, 2024-05-26T22:50:58.964 INFO:teuthology.orchestra.run.smithi131.stdout: "cap-": 0, 2024-05-26T22:50:58.964 INFO:teuthology.orchestra.run.smithi131.stdout: "rss": 43648, 2024-05-26T22:50:58.964 INFO:teuthology.orchestra.run.smithi131.stdout: "heap": 223596 2024-05-26T22:50:58.964 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.964 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_server": { 2024-05-26T22:50:58.964 INFO:teuthology.orchestra.run.smithi131.stdout: "dispatch_client_request": 0, 2024-05-26T22:50:58.964 INFO:teuthology.orchestra.run.smithi131.stdout: "dispatch_server_request": 0, 2024-05-26T22:50:58.965 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_request": 0, 2024-05-26T22:50:58.965 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_session": 1, 2024-05-26T22:50:58.965 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_peer_request": 0, 2024-05-26T22:50:58.965 INFO:teuthology.orchestra.run.smithi131.stdout: "req_create_latency": { 2024-05-26T22:50:58.965 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:50:58.965 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:50:58.965 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:50:58.965 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.965 INFO:teuthology.orchestra.run.smithi131.stdout: "req_getattr_latency": { 2024-05-26T22:50:58.965 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:50:58.965 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:50:58.965 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:50:58.966 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.966 INFO:teuthology.orchestra.run.smithi131.stdout: "req_getfilelock_latency": { 2024-05-26T22:50:58.966 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:50:58.966 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:50:58.966 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:50:58.966 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.966 INFO:teuthology.orchestra.run.smithi131.stdout: "req_link_latency": { 2024-05-26T22:50:58.966 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:50:58.966 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:50:58.966 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:50:58.966 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.966 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookup_latency": { 2024-05-26T22:50:58.967 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:50:58.967 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:50:58.967 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:50:58.967 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.967 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookuphash_latency": { 2024-05-26T22:50:58.967 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:50:58.967 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:50:58.967 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:50:58.967 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.967 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookupino_latency": { 2024-05-26T22:50:58.967 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:50:58.967 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:50:58.967 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:50:58.967 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.968 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookupname_latency": { 2024-05-26T22:50:58.968 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:50:58.968 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:50:58.968 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:50:58.968 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.968 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookupparent_latency": { 2024-05-26T22:50:58.968 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:50:58.968 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:50:58.968 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:50:58.968 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.968 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookupsnap_latency": { 2024-05-26T22:50:58.968 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:50:58.968 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:50:58.969 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:50:58.969 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.969 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lssnap_latency": { 2024-05-26T22:50:58.969 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:50:58.969 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:50:58.969 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:50:58.969 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.969 INFO:teuthology.orchestra.run.smithi131.stdout: "req_mkdir_latency": { 2024-05-26T22:50:58.969 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:50:58.969 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:50:58.969 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:50:58.969 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.970 INFO:teuthology.orchestra.run.smithi131.stdout: "req_mknod_latency": { 2024-05-26T22:50:58.970 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:50:58.970 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:50:58.970 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:50:58.970 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.970 INFO:teuthology.orchestra.run.smithi131.stdout: "req_mksnap_latency": { 2024-05-26T22:50:58.970 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:50:58.970 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:50:58.970 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:50:58.970 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.970 INFO:teuthology.orchestra.run.smithi131.stdout: "req_open_latency": { 2024-05-26T22:50:58.970 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:50:58.970 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:50:58.971 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:50:58.971 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.971 INFO:teuthology.orchestra.run.smithi131.stdout: "req_readdir_latency": { 2024-05-26T22:50:58.971 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:50:58.971 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:50:58.971 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:50:58.971 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.971 INFO:teuthology.orchestra.run.smithi131.stdout: "req_rename_latency": { 2024-05-26T22:50:58.971 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:50:58.971 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:50:58.971 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:50:58.971 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.971 INFO:teuthology.orchestra.run.smithi131.stdout: "req_renamesnap_latency": { 2024-05-26T22:50:58.971 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:50:58.972 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:50:58.972 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:50:58.972 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.972 INFO:teuthology.orchestra.run.smithi131.stdout: "req_snapdiff_latency": { 2024-05-26T22:50:58.972 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:50:58.972 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:50:58.972 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:50:58.972 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.972 INFO:teuthology.orchestra.run.smithi131.stdout: "req_rmdir_latency": { 2024-05-26T22:50:58.972 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:50:58.972 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:50:58.972 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:50:58.972 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.973 INFO:teuthology.orchestra.run.smithi131.stdout: "req_rmsnap_latency": { 2024-05-26T22:50:58.973 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:50:58.973 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:50:58.973 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:50:58.973 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.973 INFO:teuthology.orchestra.run.smithi131.stdout: "req_rmxattr_latency": { 2024-05-26T22:50:58.973 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:50:58.973 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:50:58.973 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:50:58.973 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.973 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setattr_latency": { 2024-05-26T22:50:58.973 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:50:58.973 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:50:58.973 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:50:58.974 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.974 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setdirlayout_latency": { 2024-05-26T22:50:58.974 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:50:58.974 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:50:58.974 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:50:58.974 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.974 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setfilelock_latency": { 2024-05-26T22:50:58.974 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:50:58.974 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:50:58.974 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:50:58.974 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.974 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setlayout_latency": { 2024-05-26T22:50:58.974 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:50:58.975 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:50:58.975 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:50:58.975 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.975 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setxattr_latency": { 2024-05-26T22:50:58.975 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:50:58.975 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:50:58.975 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:50:58.975 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.975 INFO:teuthology.orchestra.run.smithi131.stdout: "req_symlink_latency": { 2024-05-26T22:50:58.975 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:50:58.975 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:50:58.975 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:50:58.976 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.976 INFO:teuthology.orchestra.run.smithi131.stdout: "req_unlink_latency": { 2024-05-26T22:50:58.976 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:50:58.976 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:50:58.976 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:50:58.976 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.976 INFO:teuthology.orchestra.run.smithi131.stdout: "cap_revoke_eviction": 0, 2024-05-26T22:50:58.976 INFO:teuthology.orchestra.run.smithi131.stdout: "cap_acquisition_throttle": 0, 2024-05-26T22:50:58.976 INFO:teuthology.orchestra.run.smithi131.stdout: "req_getvxattr_latency": { 2024-05-26T22:50:58.976 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:50:58.976 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:50:58.976 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:50:58.976 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:50:58.977 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.977 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_sessions": { 2024-05-26T22:50:58.977 INFO:teuthology.orchestra.run.smithi131.stdout: "session_count": 1, 2024-05-26T22:50:58.977 INFO:teuthology.orchestra.run.smithi131.stdout: "session_add": 1, 2024-05-26T22:50:58.977 INFO:teuthology.orchestra.run.smithi131.stdout: "session_remove": 0, 2024-05-26T22:50:58.977 INFO:teuthology.orchestra.run.smithi131.stdout: "sessions_open": 1, 2024-05-26T22:50:58.977 INFO:teuthology.orchestra.run.smithi131.stdout: "sessions_stale": 0, 2024-05-26T22:50:58.977 INFO:teuthology.orchestra.run.smithi131.stdout: "total_load": 0, 2024-05-26T22:50:58.977 INFO:teuthology.orchestra.run.smithi131.stdout: "average_load": 0, 2024-05-26T22:50:58.977 INFO:teuthology.orchestra.run.smithi131.stdout: "avg_session_uptime": 0, 2024-05-26T22:50:58.977 INFO:teuthology.orchestra.run.smithi131.stdout: "mdthresh_evicted": 0 2024-05-26T22:50:58.977 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.977 INFO:teuthology.orchestra.run.smithi131.stdout: "mempool": { 2024-05-26T22:50:58.978 INFO:teuthology.orchestra.run.smithi131.stdout: "bloom_filter_bytes": 0, 2024-05-26T22:50:58.978 INFO:teuthology.orchestra.run.smithi131.stdout: "bloom_filter_items": 0, 2024-05-26T22:50:58.978 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_alloc_bytes": 0, 2024-05-26T22:50:58.978 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_alloc_items": 0, 2024-05-26T22:50:58.978 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_data_bytes": 0, 2024-05-26T22:50:58.978 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_data_items": 0, 2024-05-26T22:50:58.978 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-26T22:50:58.978 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_onode_items": 0, 2024-05-26T22:50:58.978 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-26T22:50:58.978 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_meta_items": 0, 2024-05-26T22:50:58.978 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_other_bytes": 0, 2024-05-26T22:50:58.978 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_other_items": 0, 2024-05-26T22:50:58.979 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-26T22:50:58.979 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_buffer_items": 0, 2024-05-26T22:50:58.979 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_extent_bytes": 0, 2024-05-26T22:50:58.979 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_extent_items": 0, 2024-05-26T22:50:58.979 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_blob_bytes": 0, 2024-05-26T22:50:58.979 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_blob_items": 0, 2024-05-26T22:50:58.979 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-26T22:50:58.979 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_shared_blob_items": 0, 2024-05-26T22:50:58.979 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-26T22:50:58.979 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_inline_bl_items": 0, 2024-05-26T22:50:58.979 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_fsck_bytes": 0, 2024-05-26T22:50:58.979 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_fsck_items": 0, 2024-05-26T22:50:58.979 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_txc_bytes": 0, 2024-05-26T22:50:58.980 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_txc_items": 0, 2024-05-26T22:50:58.980 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-26T22:50:58.980 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_deferred_items": 0, 2024-05-26T22:50:58.980 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_bytes": 0, 2024-05-26T22:50:58.980 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_items": 0, 2024-05-26T22:50:58.980 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_bytes": 0, 2024-05-26T22:50:58.980 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_items": 0, 2024-05-26T22:50:58.980 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_reader_bytes": 0, 2024-05-26T22:50:58.980 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_reader_items": 0, 2024-05-26T22:50:58.980 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_writer_bytes": 0, 2024-05-26T22:50:58.980 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_writer_items": 0, 2024-05-26T22:50:58.980 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_anon_bytes": 220631, 2024-05-26T22:50:58.980 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_anon_items": 57, 2024-05-26T22:50:58.980 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_meta_bytes": 0, 2024-05-26T22:50:58.981 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_meta_items": 0, 2024-05-26T22:50:58.981 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_bytes": 0, 2024-05-26T22:50:58.981 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_items": 0, 2024-05-26T22:50:58.981 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_mapbl_bytes": 0, 2024-05-26T22:50:58.981 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_mapbl_items": 0, 2024-05-26T22:50:58.981 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_pglog_bytes": 0, 2024-05-26T22:50:58.981 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_pglog_items": 0, 2024-05-26T22:50:58.981 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_bytes": 5312, 2024-05-26T22:50:58.981 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_items": 75, 2024-05-26T22:50:58.981 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_mapping_bytes": 0, 2024-05-26T22:50:58.981 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_mapping_items": 0, 2024-05-26T22:50:58.981 INFO:teuthology.orchestra.run.smithi131.stdout: "pgmap_bytes": 0, 2024-05-26T22:50:58.981 INFO:teuthology.orchestra.run.smithi131.stdout: "pgmap_items": 0, 2024-05-26T22:50:58.982 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_co_bytes": 50664, 2024-05-26T22:50:58.982 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_co_items": 198, 2024-05-26T22:50:58.982 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_1_bytes": 0, 2024-05-26T22:50:58.982 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_1_items": 0, 2024-05-26T22:50:58.982 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_2_bytes": 0, 2024-05-26T22:50:58.982 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_2_items": 0 2024-05-26T22:50:58.982 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.982 INFO:teuthology.orchestra.run.smithi131.stdout: "objecter": { 2024-05-26T22:50:58.982 INFO:teuthology.orchestra.run.smithi131.stdout: "op_active": 0, 2024-05-26T22:50:58.982 INFO:teuthology.orchestra.run.smithi131.stdout: "op_laggy": 0, 2024-05-26T22:50:58.982 INFO:teuthology.orchestra.run.smithi131.stdout: "op_send": 39, 2024-05-26T22:50:58.982 INFO:teuthology.orchestra.run.smithi131.stdout: "op_send_bytes": 75277, 2024-05-26T22:50:58.983 INFO:teuthology.orchestra.run.smithi131.stdout: "op_resend": 33, 2024-05-26T22:50:58.983 INFO:teuthology.orchestra.run.smithi131.stdout: "op_reply": 39, 2024-05-26T22:50:58.983 INFO:teuthology.orchestra.run.smithi131.stdout: "op_latency": { 2024-05-26T22:50:58.983 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 39, 2024-05-26T22:50:58.983 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.417993510, 2024-05-26T22:50:58.983 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.010717782 2024-05-26T22:50:58.983 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.983 INFO:teuthology.orchestra.run.smithi131.stdout: "op_inflight": 0, 2024-05-26T22:50:58.983 INFO:teuthology.orchestra.run.smithi131.stdout: "oplen_avg": { 2024-05-26T22:50:58.983 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 39, 2024-05-26T22:50:58.983 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 62 2024-05-26T22:50:58.983 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.983 INFO:teuthology.orchestra.run.smithi131.stdout: "op": 39, 2024-05-26T22:50:58.983 INFO:teuthology.orchestra.run.smithi131.stdout: "op_r": 0, 2024-05-26T22:50:58.984 INFO:teuthology.orchestra.run.smithi131.stdout: "op_w": 39, 2024-05-26T22:50:58.984 INFO:teuthology.orchestra.run.smithi131.stdout: "op_rmw": 0, 2024-05-26T22:50:58.984 INFO:teuthology.orchestra.run.smithi131.stdout: "op_pg": 0, 2024-05-26T22:50:58.984 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_stat": 0, 2024-05-26T22:50:58.984 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_create": 10, 2024-05-26T22:50:58.984 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_read": 0, 2024-05-26T22:50:58.984 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_write": 4, 2024-05-26T22:50:58.984 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_writefull": 5, 2024-05-26T22:50:58.984 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_writesame": 0, 2024-05-26T22:50:58.984 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_append": 0, 2024-05-26T22:50:58.984 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_zero": 0, 2024-05-26T22:50:58.984 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_truncate": 0, 2024-05-26T22:50:58.984 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_delete": 6, 2024-05-26T22:50:58.984 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_mapext": 0, 2024-05-26T22:50:58.985 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_sparse_read": 0, 2024-05-26T22:50:58.985 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_clonerange": 0, 2024-05-26T22:50:58.985 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_getxattr": 0, 2024-05-26T22:50:58.985 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_setxattr": 20, 2024-05-26T22:50:58.985 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_cmpxattr": 0, 2024-05-26T22:50:58.985 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_rmxattr": 0, 2024-05-26T22:50:58.985 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_resetxattrs": 0, 2024-05-26T22:50:58.985 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_call": 0, 2024-05-26T22:50:58.985 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_watch": 0, 2024-05-26T22:50:58.985 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_notify": 0, 2024-05-26T22:50:58.985 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_src_cmpxattr": 0, 2024-05-26T22:50:58.985 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_pgls": 0, 2024-05-26T22:50:58.985 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_pgls_filter": 0, 2024-05-26T22:50:58.986 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_other": 0, 2024-05-26T22:50:58.986 INFO:teuthology.orchestra.run.smithi131.stdout: "linger_active": 0, 2024-05-26T22:50:58.986 INFO:teuthology.orchestra.run.smithi131.stdout: "linger_send": 0, 2024-05-26T22:50:58.986 INFO:teuthology.orchestra.run.smithi131.stdout: "linger_resend": 0, 2024-05-26T22:50:58.986 INFO:teuthology.orchestra.run.smithi131.stdout: "linger_ping": 0, 2024-05-26T22:50:58.986 INFO:teuthology.orchestra.run.smithi131.stdout: "poolop_active": 0, 2024-05-26T22:50:58.986 INFO:teuthology.orchestra.run.smithi131.stdout: "poolop_send": 0, 2024-05-26T22:50:58.986 INFO:teuthology.orchestra.run.smithi131.stdout: "poolop_resend": 0, 2024-05-26T22:50:58.986 INFO:teuthology.orchestra.run.smithi131.stdout: "poolstat_active": 0, 2024-05-26T22:50:58.986 INFO:teuthology.orchestra.run.smithi131.stdout: "poolstat_send": 0, 2024-05-26T22:50:58.986 INFO:teuthology.orchestra.run.smithi131.stdout: "poolstat_resend": 0, 2024-05-26T22:50:58.986 INFO:teuthology.orchestra.run.smithi131.stdout: "statfs_active": 0, 2024-05-26T22:50:58.987 INFO:teuthology.orchestra.run.smithi131.stdout: "statfs_send": 0, 2024-05-26T22:50:58.987 INFO:teuthology.orchestra.run.smithi131.stdout: "statfs_resend": 0, 2024-05-26T22:50:58.987 INFO:teuthology.orchestra.run.smithi131.stdout: "command_active": 0, 2024-05-26T22:50:58.987 INFO:teuthology.orchestra.run.smithi131.stdout: "command_send": 0, 2024-05-26T22:50:58.987 INFO:teuthology.orchestra.run.smithi131.stdout: "command_resend": 0, 2024-05-26T22:50:58.987 INFO:teuthology.orchestra.run.smithi131.stdout: "map_epoch": 0, 2024-05-26T22:50:58.987 INFO:teuthology.orchestra.run.smithi131.stdout: "map_full": 0, 2024-05-26T22:50:58.987 INFO:teuthology.orchestra.run.smithi131.stdout: "map_inc": 0, 2024-05-26T22:50:58.987 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_sessions": 7, 2024-05-26T22:50:58.987 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_session_open": 7, 2024-05-26T22:50:58.987 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_session_close": 0, 2024-05-26T22:50:58.987 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_laggy": 0, 2024-05-26T22:50:58.987 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_wr": 16, 2024-05-26T22:50:58.988 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_rd": 0, 2024-05-26T22:50:58.988 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_del": 1 2024-05-26T22:50:58.988 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.988 INFO:teuthology.orchestra.run.smithi131.stdout: "oft": { 2024-05-26T22:50:58.988 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_total_objs": 1, 2024-05-26T22:50:58.988 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_total_kv_pairs": 0, 2024-05-26T22:50:58.988 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_total_updates": 0, 2024-05-26T22:50:58.988 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_total_removes": 0 2024-05-26T22:50:58.988 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.988 INFO:teuthology.orchestra.run.smithi131.stdout: "osd-slow-ops": { 2024-05-26T22:50:58.988 INFO:teuthology.orchestra.run.smithi131.stdout: "slow_ops_count": 0 2024-05-26T22:50:58.988 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.988 INFO:teuthology.orchestra.run.smithi131.stdout: "purge_queue": { 2024-05-26T22:50:58.989 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executing_ops": 0, 2024-05-26T22:50:58.989 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executing_ops_high_water": 0, 2024-05-26T22:50:58.989 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executing": 0, 2024-05-26T22:50:58.989 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executing_high_water": 0, 2024-05-26T22:50:58.989 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executed_ops": 0, 2024-05-26T22:50:58.989 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executed": 0, 2024-05-26T22:50:58.989 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_item_in_journal": 0 2024-05-26T22:50:58.989 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.989 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-26T22:50:58.989 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-05-26T22:50:58.989 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 104857600, 2024-05-26T22:50:58.989 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-05-26T22:50:58.990 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 146, 2024-05-26T22:50:58.990 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 226254, 2024-05-26T22:50:58.990 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-05-26T22:50:58.990 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 146, 2024-05-26T22:50:58.990 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 0, 2024-05-26T22:50:58.990 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 0, 2024-05-26T22:50:58.990 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 146, 2024-05-26T22:50:58.990 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 226254, 2024-05-26T22:50:58.990 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-05-26T22:50:58.990 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:50:58.990 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:50:58.990 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:50:58.991 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:50:58.991 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.991 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-objecter_bytes": { 2024-05-26T22:50:58.991 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-05-26T22:50:58.991 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 104857600, 2024-05-26T22:50:58.991 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-05-26T22:50:58.991 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 0, 2024-05-26T22:50:58.991 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 0, 2024-05-26T22:50:58.991 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-05-26T22:50:58.991 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 0, 2024-05-26T22:50:58.991 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 39, 2024-05-26T22:50:58.991 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 75277, 2024-05-26T22:50:58.991 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 39, 2024-05-26T22:50:58.991 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 75277, 2024-05-26T22:50:58.992 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-05-26T22:50:58.992 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:50:58.992 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:50:58.992 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:50:58.992 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:50:58.992 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.992 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-objecter_ops": { 2024-05-26T22:50:58.992 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-05-26T22:50:58.992 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 1024, 2024-05-26T22:50:58.992 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-05-26T22:50:58.992 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 0, 2024-05-26T22:50:58.993 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 0, 2024-05-26T22:50:58.993 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-05-26T22:50:58.993 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 0, 2024-05-26T22:50:58.993 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 39, 2024-05-26T22:50:58.993 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 39, 2024-05-26T22:50:58.993 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 39, 2024-05-26T22:50:58.993 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 39, 2024-05-26T22:50:58.993 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-05-26T22:50:58.993 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:50:58.993 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:50:58.993 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:50:58.993 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:50:58.993 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.994 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-write_buf_throttle": { 2024-05-26T22:50:58.994 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-05-26T22:50:58.994 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 3758096384, 2024-05-26T22:50:58.994 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-05-26T22:50:58.994 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 0, 2024-05-26T22:50:58.994 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 0, 2024-05-26T22:50:58.994 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-05-26T22:50:58.994 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 0, 2024-05-26T22:50:58.994 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 0, 2024-05-26T22:50:58.994 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 0, 2024-05-26T22:50:58.994 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 0, 2024-05-26T22:50:58.995 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 0, 2024-05-26T22:50:58.995 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-05-26T22:50:58.995 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:50:58.995 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:50:58.995 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:50:58.995 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:50:58.995 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:58.995 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-write_buf_throttle-0x559fe4810500": { 2024-05-26T22:50:58.995 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 183, 2024-05-26T22:50:58.995 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 3758096384, 2024-05-26T22:50:58.995 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-05-26T22:50:58.995 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 8, 2024-05-26T22:50:58.995 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 61511, 2024-05-26T22:50:58.996 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-05-26T22:50:58.996 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 8, 2024-05-26T22:50:58.996 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 0, 2024-05-26T22:50:58.996 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 0, 2024-05-26T22:50:58.996 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 4, 2024-05-26T22:50:58.996 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 61328, 2024-05-26T22:50:58.996 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-05-26T22:50:58.996 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:50:58.996 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:50:58.996 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:50:58.996 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:50:58.996 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:50:58.997 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-26T22:50:58.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.944+0000 7f9a3e600640 1 -- 172.21.15.131:0/1645717863 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9a1806be30 msgr2=0x7f9a1806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:58.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.944+0000 7f9a3e600640 1 --2- 172.21.15.131:0/1645717863 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9a1806be30 0x7f9a1806e2f0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f9a24000ff0 tx=0x7f9a24009040 comp rx=0 tx=0).stop 2024-05-26T22:50:58.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.944+0000 7f9a3e600640 1 -- 172.21.15.131:0/1645717863 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9a1c0039b0 msgr2=0x7f9a3813a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:58.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.944+0000 7f9a3e600640 1 --2- 172.21.15.131:0/1645717863 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9a1c0039b0 0x7f9a3813a080 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f9a28034040 tx=0x7f9a28004730 comp rx=0 tx=0).stop 2024-05-26T22:50:58.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.944+0000 7f9a3e600640 1 -- 172.21.15.131:0/1645717863 shutdown_connections 2024-05-26T22:50:58.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.944+0000 7f9a3e600640 1 --2- 172.21.15.131:0/1645717863 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9a381f7d40 0x7f9a381f81c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:58.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.944+0000 7f9a3e600640 1 --2- 172.21.15.131:0/1645717863 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9a1806be30 0x7f9a1806e2f0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:58.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.944+0000 7f9a3e600640 1 --2- 172.21.15.131:0/1645717863 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9a3815a880 0x7f9a3813c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:58.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.944+0000 7f9a3e600640 1 --2- 172.21.15.131:0/1645717863 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9a1c0039b0 0x7f9a3813a080 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:58.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.944+0000 7f9a3e600640 1 -- 172.21.15.131:0/1645717863 >> 172.21.15.131:0/1645717863 conn(0x7f9a380cc7f0 msgr2=0x7f9a380afaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:58.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.944+0000 7f9a3e600640 1 -- 172.21.15.131:0/1645717863 shutdown_connections 2024-05-26T22:50:58.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:58.944+0000 7f9a3e600640 1 -- 172.21.15.131:0/1645717863 wait complete. 2024-05-26T22:50:58.999 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 perf dump 2024-05-26T22:50:59.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.210+0000 7f63e3200640 1 Processor -- start 2024-05-26T22:50:59.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.210+0000 7f63e3200640 1 -- start start 2024-05-26T22:50:59.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.211+0000 7f63e3200640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f63dc0d05d0 0x7f63dc06df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:59.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.211+0000 7f63e3200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f63dc06e450 0x7f63dc0cf320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:59.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.211+0000 7f63e3200640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63dc15fa00 con 0x7f63dc0cf860 2024-05-26T22:50:59.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.212+0000 7f63e3200640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f63dc0cfd10 con 0x7f63dc0d05d0 2024-05-26T22:50:59.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.212+0000 7f63e3200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f63dc0d09b0 con 0x7f63dc06e450 2024-05-26T22:50:59.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.212+0000 7f63e1200640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f63dc0cf860 0x7f63dc0729b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:44808/0 (socket says 172.21.15.131:44808) 2024-05-26T22:50:59.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.212+0000 7f63e1200640 1 -- 172.21.15.131:0/3940688341 learned_addr learned my addr 172.21.15.131:0/3940688341 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:50:59.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.212+0000 7f63d3400640 1 --2- 172.21.15.131:0/3940688341 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f63dc06e450 0x7f63dc0cf320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:59.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.212+0000 7f63dbe00640 1 --2- 172.21.15.131:0/3940688341 >> v2:172.21.15.187:3300/0 conn(0x7f63dc0d05d0 0x7f63dc06df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:59.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.212+0000 7f63db400640 1 -- 172.21.15.131:0/3940688341 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1263548850 0 0) 0x7f63dc15fa00 con 0x7f63dc0cf860 2024-05-26T22:50:59.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.213+0000 7f63db400640 1 -- 172.21.15.131:0/3940688341 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f63c0003520 con 0x7f63dc0cf860 2024-05-26T22:50:59.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.213+0000 7f63db400640 1 -- 172.21.15.131:0/3940688341 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3489142703 0 0) 0x7f63c0003520 con 0x7f63dc0cf860 2024-05-26T22:50:59.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.213+0000 7f63db400640 1 -- 172.21.15.131:0/3940688341 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f63dc06e450 msgr2=0x7f63dc0cf320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:59.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.213+0000 7f63db400640 1 --2- 172.21.15.131:0/3940688341 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f63dc06e450 0x7f63dc0cf320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:59.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.213+0000 7f63db400640 1 -- 172.21.15.131:0/3940688341 >> v2:172.21.15.187:3300/0 conn(0x7f63dc0d05d0 msgr2=0x7f63dc06df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:59.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.213+0000 7f63db400640 1 --2- 172.21.15.131:0/3940688341 >> v2:172.21.15.187:3300/0 conn(0x7f63dc0d05d0 0x7f63dc06df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:59.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.213+0000 7f63db400640 1 -- 172.21.15.131:0/3940688341 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63dc15f720 con 0x7f63dc0cf860 2024-05-26T22:50:59.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.213+0000 7f63db400640 1 -- 172.21.15.131:0/3940688341 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7f63d4001670 con 0x7f63dc0cf860 2024-05-26T22:50:59.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.213+0000 7f63db400640 1 -- 172.21.15.131:0/3940688341 >> v1:172.21.15.131:6789/0 conn(0x7f63dc0cf860 legacy=0x7f63dc0729b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:59.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.213+0000 7f63db400640 1 --2- 172.21.15.131:0/3940688341 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63c00039b0 0x7f63c0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:59.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.213+0000 7f63db400640 1 --2- 172.21.15.131:0/3940688341 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f63dc06e450 0x7f63c0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:59.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.213+0000 7f63db400640 1 --2- 172.21.15.131:0/3940688341 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f63c0004d40 0x7f63c00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:59.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.213+0000 7f63db400640 1 -- 172.21.15.131:0/3940688341 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f63dc0d09b0 con 0x7f63c00039b0 2024-05-26T22:50:59.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.213+0000 7f63db400640 1 -- 172.21.15.131:0/3940688341 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f63c000b9c0 con 0x7f63c0004d40 2024-05-26T22:50:59.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.213+0000 7f63db400640 1 -- 172.21.15.131:0/3940688341 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f63c000bb40 con 0x7f63dc06e450 2024-05-26T22:50:59.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.213+0000 7f63dbe00640 1 --2- 172.21.15.131:0/3940688341 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63c00039b0 0x7f63c0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:59.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.214+0000 7f63d3400640 1 --2- 172.21.15.131:0/3940688341 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f63dc06e450 0x7f63c0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:59.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.214+0000 7f63dbe00640 1 -- 172.21.15.131:0/3940688341 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f63dc06e450 msgr2=0x7f63c0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:59.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.214+0000 7f63dbe00640 1 --2- 172.21.15.131:0/3940688341 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f63dc06e450 0x7f63c0004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:59.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.214+0000 7f63dbe00640 1 -- 172.21.15.131:0/3940688341 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f63c0004d40 msgr2=0x7f63c00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:59.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.214+0000 7f63dbe00640 1 --2- 172.21.15.131:0/3940688341 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f63c0004d40 0x7f63c00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:59.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.214+0000 7f63dbe00640 1 -- 172.21.15.131:0/3940688341 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63c0003520 con 0x7f63c00039b0 2024-05-26T22:50:59.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.215+0000 7f63dbe00640 1 --2- 172.21.15.131:0/3940688341 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63c00039b0 0x7f63c0003dd0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f63cc009fc0 tx=0x7f63c0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4625b2cc11656185 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:59.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.215+0000 7f63db400640 1 -- 172.21.15.131:0/3940688341 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63cc009c10 con 0x7f63c00039b0 2024-05-26T22:50:59.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.215+0000 7f63db400640 1 -- 172.21.15.131:0/3940688341 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63cc036070 con 0x7f63c00039b0 2024-05-26T22:50:59.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.215+0000 7f63db400640 1 -- 172.21.15.131:0/3940688341 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63cc02d070 con 0x7f63c00039b0 2024-05-26T22:50:59.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.216+0000 7f63e3200640 1 -- 172.21.15.131:0/3940688341 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63c00039b0 msgr2=0x7f63c0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:59.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.216+0000 7f63e3200640 1 --2- 172.21.15.131:0/3940688341 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63c00039b0 0x7f63c0003dd0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f63cc009fc0 tx=0x7f63c0005830 comp rx=0 tx=0).stop 2024-05-26T22:50:59.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.216+0000 7f63e1200640 1 -- 172.21.15.131:0/3940688341 reap_dead start 2024-05-26T22:50:59.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.216+0000 7f63e3200640 1 -- 172.21.15.131:0/3940688341 shutdown_connections 2024-05-26T22:50:59.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.216+0000 7f63e3200640 1 -- 172.21.15.131:0/3940688341 >> 172.21.15.131:0/3940688341 conn(0x7f63dc06b940 msgr2=0x7f63dc0b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:59.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.216+0000 7f63e3200640 1 -- 172.21.15.131:0/3940688341 shutdown_connections 2024-05-26T22:50:59.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.216+0000 7f63e3200640 1 -- 172.21.15.131:0/3940688341 wait complete. 2024-05-26T22:50:59.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.218+0000 7f63e3200640 1 Processor -- start 2024-05-26T22:50:59.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.218+0000 7f63e3200640 1 -- start start 2024-05-26T22:50:59.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.219+0000 7f63e3200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63c00039b0 0x7f63dc161890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:59.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.219+0000 7f63e3200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f63dc157410 0x7f63dc1488f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:59.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.219+0000 7f63e3200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f63dc160030 0x7f63dc1604b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:59.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.219+0000 7f63e3200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f63dc06d9f0 con 0x7f63c00039b0 2024-05-26T22:50:59.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.219+0000 7f63e3200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f63dc06d710 con 0x7f63dc160030 2024-05-26T22:50:59.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.219+0000 7f63e3200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f63dc0730c0 con 0x7f63dc157410 2024-05-26T22:50:59.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.219+0000 7f63dbe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63c00039b0 0x7f63dc161890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:59.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.219+0000 7f63dbe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63c00039b0 0x7f63dc161890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55128/0 (socket says 172.21.15.131:55128) 2024-05-26T22:50:59.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.219+0000 7f63dbe00640 1 -- 172.21.15.131:0/1034723795 learned_addr learned my addr 172.21.15.131:0/1034723795 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:59.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.219+0000 7f63d3400640 1 --2- 172.21.15.131:0/1034723795 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f63dc157410 0x7f63dc1488f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:59.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.219+0000 7f63dbe00640 1 -- 172.21.15.131:0/1034723795 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f63dc157410 msgr2=0x7f63dc1488f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:59.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.219+0000 7f63dbe00640 1 --2- 172.21.15.131:0/1034723795 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f63dc157410 0x7f63dc1488f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:59.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.219+0000 7f63dbe00640 1 -- 172.21.15.131:0/1034723795 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f63dc160030 msgr2=0x7f63dc1604b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:59.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.219+0000 7f63e1200640 1 --2- 172.21.15.131:0/1034723795 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f63dc160030 0x7f63dc1604b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:59.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.219+0000 7f63dbe00640 1 --2- 172.21.15.131:0/1034723795 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f63dc160030 0x7f63dc1604b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:59.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.219+0000 7f63dbe00640 1 -- 172.21.15.131:0/1034723795 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63dc156690 con 0x7f63c00039b0 2024-05-26T22:50:59.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.220+0000 7f63dbe00640 1 --2- 172.21.15.131:0/1034723795 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63c00039b0 0x7f63dc161890 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f63cc02af10 tx=0x7f63cc009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:59.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.220+0000 7f63d8c00640 1 -- 172.21.15.131:0/1034723795 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63cc009c10 con 0x7f63c00039b0 2024-05-26T22:50:59.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.220+0000 7f63d8c00640 1 -- 172.21.15.131:0/1034723795 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63cc003b90 con 0x7f63c00039b0 2024-05-26T22:50:59.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.220+0000 7f63d8c00640 1 -- 172.21.15.131:0/1034723795 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63cc004020 con 0x7f63c00039b0 2024-05-26T22:50:59.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.220+0000 7f63e3200640 1 -- 172.21.15.131:0/1034723795 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63dc156930 con 0x7f63c00039b0 2024-05-26T22:50:59.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.220+0000 7f63e3200640 1 -- 172.21.15.131:0/1034723795 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63dc148e30 con 0x7f63c00039b0 2024-05-26T22:50:59.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.223+0000 7f63d8c00640 1 -- 172.21.15.131:0/1034723795 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f63cc005d10 con 0x7f63c00039b0 2024-05-26T22:50:59.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.224+0000 7f63d8c00640 1 --2- 172.21.15.131:0/1034723795 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f63b806be30 0x7f63b806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:59.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.224+0000 7f63d8c00640 1 -- 172.21.15.131:0/1034723795 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f63cc0ac310 con 0x7f63c00039b0 2024-05-26T22:50:59.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.224+0000 7f63d3400640 1 --2- 172.21.15.131:0/1034723795 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f63b806be30 0x7f63b806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:59.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.225+0000 7f63e5f8e740 1 Processor -- start 2024-05-26T22:50:59.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.225+0000 7f63d3400640 1 --2- 172.21.15.131:0/1034723795 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f63b806be30 0x7f63b806e2f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f63dc161610 tx=0x7f63c4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:59.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.225+0000 7f63e5f8e740 1 -- start start 2024-05-26T22:50:59.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.226+0000 7f63e5f8e740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556cc65688b0 0x556cc6568c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:59.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.226+0000 7f63e5f8e740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556cc6565ea0 0x556cc6538bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:59.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.226+0000 7f63e5f8e740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556cc653fe10 0x556cc653cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:59.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.226+0000 7f63e5f8e740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x556cc64a1960 con 0x556cc65688b0 2024-05-26T22:50:59.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.226+0000 7f63e5f8e740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x556cc64490d0 con 0x556cc6565ea0 2024-05-26T22:50:59.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.226+0000 7f63e5f8e740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x556cc649d920 con 0x556cc653fe10 2024-05-26T22:50:59.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.226+0000 7f63dbe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556cc65688b0 0x556cc6568c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:59.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.226+0000 7f63dbe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556cc65688b0 0x556cc6568c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55138/0 (socket says 172.21.15.131:55138) 2024-05-26T22:50:59.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.226+0000 7f63dbe00640 1 -- 172.21.15.131:0/1532131502 learned_addr learned my addr 172.21.15.131:0/1532131502 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:59.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.226+0000 7f63dbe00640 1 -- 172.21.15.131:0/1532131502 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556cc653fe10 msgr2=0x556cc653cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:59.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.226+0000 7f63dbe00640 1 --2- 172.21.15.131:0/1532131502 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556cc653fe10 0x556cc653cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:59.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.227+0000 7f63d3400640 1 --2- 172.21.15.131:0/1532131502 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556cc6565ea0 0x556cc6538bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:59.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.227+0000 7f63dbe00640 1 -- 172.21.15.131:0/1532131502 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556cc6565ea0 msgr2=0x556cc6538bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:59.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.227+0000 7f63dbe00640 1 --2- 172.21.15.131:0/1532131502 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556cc6565ea0 0x556cc6538bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:59.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.227+0000 7f63dbe00640 1 -- 172.21.15.131:0/1532131502 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556cc627f7d0 con 0x556cc65688b0 2024-05-26T22:50:59.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.227+0000 7f63dbe00640 1 --2- 172.21.15.131:0/1532131502 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556cc65688b0 0x556cc6568c90 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f63cc0aa770 tx=0x7f63cc049000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4730aa4bfb376d97 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:59.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.227+0000 7f63d0c00640 1 -- 172.21.15.131:0/1532131502 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63cc04a030 con 0x556cc65688b0 2024-05-26T22:50:59.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.227+0000 7f63d0c00640 1 -- 172.21.15.131:0/1532131502 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63cc036040 con 0x556cc65688b0 2024-05-26T22:50:59.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.227+0000 7f63d0c00640 1 -- 172.21.15.131:0/1532131502 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63cc0832b0 con 0x556cc65688b0 2024-05-26T22:50:59.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.228+0000 7f63e5f8e740 1 -- 172.21.15.131:0/1532131502 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556cc65688b0 msgr2=0x556cc6568c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:59.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.228+0000 7f63e5f8e740 1 --2- 172.21.15.131:0/1532131502 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556cc65688b0 0x556cc6568c90 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f63cc0aa770 tx=0x7f63cc049000 comp rx=0 tx=0).stop 2024-05-26T22:50:59.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.228+0000 7f63e5f8e740 1 -- 172.21.15.131:0/1532131502 shutdown_connections 2024-05-26T22:50:59.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.228+0000 7f63e5f8e740 1 --2- 172.21.15.131:0/1532131502 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556cc653fe10 0x556cc653cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:59.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.228+0000 7f63e5f8e740 1 --2- 172.21.15.131:0/1532131502 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556cc6565ea0 0x556cc6538bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:59.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.228+0000 7f63e5f8e740 1 --2- 172.21.15.131:0/1532131502 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556cc65688b0 0x556cc6568c90 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:59.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.228+0000 7f63e5f8e740 1 -- 172.21.15.131:0/1532131502 >> 172.21.15.131:0/1532131502 conn(0x556cc653ae20 msgr2=0x556cc65836c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:59.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.228+0000 7f63e5f8e740 1 -- 172.21.15.131:0/1532131502 shutdown_connections 2024-05-26T22:50:59.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.228+0000 7f63e5f8e740 1 -- 172.21.15.131:0/1532131502 wait complete. 2024-05-26T22:50:59.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.229+0000 7f63e5f8e740 1 -- start start 2024-05-26T22:50:59.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.229+0000 7f63e5f8e740 1 Processor -- start 2024-05-26T22:50:59.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.230+0000 7f63e5f8e740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556cc6565ea0 0x556cc6562670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:59.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.230+0000 7f63e5f8e740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556cc65688b0 0x556cc6575800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:59.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.230+0000 7f63e5f8e740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556cc6575d40 0x556cc6576810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:59.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.230+0000 7f63e5f8e740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x556cc64490d0 con 0x556cc6565ea0 2024-05-26T22:50:59.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.230+0000 7f63e5f8e740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x556cc645c4a0 con 0x556cc6575d40 2024-05-26T22:50:59.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.230+0000 7f63e5f8e740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x556cc64a1960 con 0x556cc65688b0 2024-05-26T22:50:59.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.230+0000 7f63e5f8e740 10 client.0 _ll_register_callbacks cb 0x556cc6463c40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-26T22:50:59.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.230+0000 7f63dbe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556cc6565ea0 0x556cc6562670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:59.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.230+0000 7f63dbe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556cc6565ea0 0x556cc6562670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55154/0 (socket says 172.21.15.131:55154) 2024-05-26T22:50:59.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.230+0000 7f63dbe00640 1 -- 172.21.15.131:0/4036819065 learned_addr learned my addr 172.21.15.131:0/4036819065 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:59.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.230+0000 7f63dbe00640 1 -- 172.21.15.131:0/4036819065 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556cc65688b0 msgr2=0x556cc6575800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:59.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.230+0000 7f63dbe00640 1 --2- 172.21.15.131:0/4036819065 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556cc65688b0 0x556cc6575800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:59.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.231+0000 7f63dbe00640 1 -- 172.21.15.131:0/4036819065 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556cc6575d40 msgr2=0x556cc6576810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:59.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.231+0000 7f63dbe00640 1 --2- 172.21.15.131:0/4036819065 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556cc6575d40 0x556cc6576810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:59.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.231+0000 7f63dbe00640 1 -- 172.21.15.131:0/4036819065 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556cc61750f0 con 0x556cc6565ea0 2024-05-26T22:50:59.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.231+0000 7f63dbe00640 1 --2- 172.21.15.131:0/4036819065 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556cc6565ea0 0x556cc6562670 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f63cc07a3d0 tx=0x7f63cc0aa700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e475ee2062f4f1ba server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:59.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.231+0000 7f63c9600640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-26T22:50:59.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.231+0000 7f63c9600640 1 -- 172.21.15.131:0/4036819065 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63cc0832b0 con 0x556cc6565ea0 2024-05-26T22:50:59.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.231+0000 7f63c9600640 1 -- 172.21.15.131:0/4036819065 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f63cc076d40 con 0x556cc6565ea0 2024-05-26T22:50:59.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.231+0000 7f63e5f8e740 1 -- 172.21.15.131:0/4036819065 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63cc036040 con 0x556cc6565ea0 2024-05-26T22:50:59.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.231+0000 7f63c9600640 1 -- 172.21.15.131:0/4036819065 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f63cc099050 con 0x556cc6565ea0 2024-05-26T22:50:59.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.231+0000 7f63e5f8e740 10 client.5985 fetch_fsmap learned FSMap version 34 2024-05-26T22:50:59.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.231+0000 7f63e5f8e740 1 -- 172.21.15.131:0/4036819065 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) -- 0x556cc62dff60 con 0x556cc6565ea0 2024-05-26T22:50:59.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.231+0000 7f63c9600640 1 -- 172.21.15.131:0/4036819065 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63cc036040 con 0x556cc6565ea0 2024-05-26T22:50:59.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.232+0000 7f63c9600640 1 -- 172.21.15.131:0/4036819065 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 34) ==== 3542+0+0 (secure 0 0 0) 0x7f63cc04e3a0 con 0x556cc6565ea0 2024-05-26T22:50:59.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.232+0000 7f63e5f8e740 10 client.5985 fetch_fsmap finished waiting for FSMap version 34 2024-05-26T22:50:59.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.232+0000 7f63e5f8e740 10 client.5985 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-05-26T22:50:59.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.232+0000 7f63e5f8e740 20 client.5985 populate_metadata read hostname 'smithi131' 2024-05-26T22:50:59.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.232+0000 7f63e5f8e740 1 --2- 172.21.15.131:0/4036819065 >> [v2:172.21.15.131:6834/650770835,v1:172.21.15.131:6836/650770835] conn(0x556cc656a4d0 0x556cc656a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:59.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.232+0000 7f63e5f8e740 4 client.5985 mds_command: new command op to 5637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-26T22:50:59.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.232+0000 7f63e5f8e740 1 -- 172.21.15.131:0/4036819065 --> [v2:172.21.15.131:6834/650770835,v1:172.21.15.131:6836/650770835] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556cc64af090 con 0x556cc656a4d0 2024-05-26T22:50:59.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.232+0000 7f63d3400640 1 --2- 172.21.15.131:0/4036819065 >> [v2:172.21.15.131:6834/650770835,v1:172.21.15.131:6836/650770835] conn(0x556cc656a4d0 0x556cc656a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:59.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.233+0000 7f63d3400640 1 --2- 172.21.15.131:0/4036819065 >> [v2:172.21.15.131:6834/650770835,v1:172.21.15.131:6836/650770835] conn(0x556cc656a4d0 0x556cc656a960 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4a4675ed86e44f2 server_cookie=b58a9f10e97a7552 in_seq=0 out_seq=0 2024-05-26T22:50:59.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.233+0000 7f63c9600640 10 client.5985 ms_handle_connect on v2:172.21.15.131:6834/650770835 2024-05-26T22:50:59.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.233+0000 7f63c9600640 1 -- 172.21.15.131:0/4036819065 <== mds.0 v2:172.21.15.131:6834/650770835 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556cc64af090 con 0x556cc656a4d0 2024-05-26T22:50:59.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.233+0000 7f63c9600640 10 client.5985 handle_command_reply: tid=0 2024-05-26T22:50:59.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.233+0000 7f63e5f8e740 1 client.5985 shutdown 2024-05-26T22:50:59.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.234+0000 7f63e5f8e740 1 -- 172.21.15.131:0/4036819065 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556cc6565ea0 msgr2=0x556cc6562670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:59.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.234+0000 7f63e5f8e740 1 --2- 172.21.15.131:0/4036819065 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556cc6565ea0 0x556cc6562670 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f63cc07a3d0 tx=0x7f63cc0aa700 comp rx=0 tx=0).stop 2024-05-26T22:50:59.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.234+0000 7f63e5f8e740 1 -- 172.21.15.131:0/4036819065 shutdown_connections 2024-05-26T22:50:59.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.234+0000 7f63e5f8e740 1 --2- 172.21.15.131:0/4036819065 >> [v2:172.21.15.131:6834/650770835,v1:172.21.15.131:6836/650770835] conn(0x556cc656a4d0 0x556cc656a960 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:59.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.234+0000 7f63e5f8e740 1 --2- 172.21.15.131:0/4036819065 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556cc65688b0 0x556cc6575800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:59.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.234+0000 7f63e5f8e740 1 --2- 172.21.15.131:0/4036819065 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556cc6575d40 0x556cc6576810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:59.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.234+0000 7f63e5f8e740 1 --2- 172.21.15.131:0/4036819065 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556cc6565ea0 0x556cc6562670 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:59.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.234+0000 7f63c9600640 0 client.5985 ms_handle_reset on v2:172.21.15.131:6834/650770835 2024-05-26T22:50:59.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.234+0000 7f63e5f8e740 1 -- 172.21.15.131:0/4036819065 >> 172.21.15.131:0/4036819065 conn(0x556cc653fe10 msgr2=0x556cc653e3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:59.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.234+0000 7f63e5f8e740 1 -- 172.21.15.131:0/4036819065 shutdown_connections 2024-05-26T22:50:59.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.234+0000 7f63e5f8e740 1 -- 172.21.15.131:0/4036819065 wait complete. 2024-05-26T22:50:59.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.234+0000 7f63e5f8e740 20 client.5985 trim_cache size 0 max 16384 2024-05-26T22:50:59.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.253+0000 7f63e5f8e740 1 Processor -- start 2024-05-26T22:50:59.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.253+0000 7f63e5f8e740 1 -- start start 2024-05-26T22:50:59.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.254+0000 7f63e5f8e740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556cc6565ea0 0x556cc6581ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:59.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.254+0000 7f63e5f8e740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556cc65688b0 0x556cc659baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:59.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.254+0000 7f63e5f8e740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556cc6575d40 0x556cc65828d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:59.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.254+0000 7f63e5f8e740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f63dc0c77d0 con 0x556cc6565ea0 2024-05-26T22:50:59.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.254+0000 7f63e5f8e740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f63dc0c69d0 con 0x556cc6575d40 2024-05-26T22:50:59.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.254+0000 7f63e5f8e740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f63dc0c70a0 con 0x556cc65688b0 2024-05-26T22:50:59.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.254+0000 7f63dbe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556cc6565ea0 0x556cc6581ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:59.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.254+0000 7f63dbe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556cc6565ea0 0x556cc6581ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55158/0 (socket says 172.21.15.131:55158) 2024-05-26T22:50:59.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.254+0000 7f63dbe00640 1 -- 172.21.15.131:0/3739127966 learned_addr learned my addr 172.21.15.131:0/3739127966 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:59.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.254+0000 7f63dbe00640 1 -- 172.21.15.131:0/3739127966 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556cc65688b0 msgr2=0x556cc659baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:59.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.254+0000 7f63dbe00640 1 --2- 172.21.15.131:0/3739127966 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556cc65688b0 0x556cc659baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:59.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.254+0000 7f63dbe00640 1 -- 172.21.15.131:0/3739127966 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556cc6575d40 msgr2=0x556cc65828d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:59.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.254+0000 7f63dbe00640 1 --2- 172.21.15.131:0/3739127966 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556cc6575d40 0x556cc65828d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:59.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.254+0000 7f63dbe00640 1 -- 172.21.15.131:0/3739127966 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556cc6575b70 con 0x556cc6565ea0 2024-05-26T22:50:59.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.254+0000 7f63dbe00640 1 --2- 172.21.15.131:0/3739127966 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556cc6565ea0 0x556cc6581ee0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f63cc07a100 tx=0x7f63cc084000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49d50466d8d54d1b server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:59.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.255+0000 7f63cbe00640 1 -- 172.21.15.131:0/3739127966 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63cc04a040 con 0x556cc6565ea0 2024-05-26T22:50:59.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.255+0000 7f63cbe00640 1 -- 172.21.15.131:0/3739127966 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63cc07aac0 con 0x556cc6565ea0 2024-05-26T22:50:59.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.255+0000 7f63cbe00640 1 -- 172.21.15.131:0/3739127966 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63cc04e3a0 con 0x556cc6565ea0 2024-05-26T22:50:59.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.255+0000 7f63e5f8e740 1 -- 172.21.15.131:0/3739127966 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556cc6565ea0 msgr2=0x556cc6581ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:59.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.255+0000 7f63e5f8e740 1 --2- 172.21.15.131:0/3739127966 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556cc6565ea0 0x556cc6581ee0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f63cc07a100 tx=0x7f63cc084000 comp rx=0 tx=0).stop 2024-05-26T22:50:59.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.255+0000 7f63e5f8e740 1 -- 172.21.15.131:0/3739127966 shutdown_connections 2024-05-26T22:50:59.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.255+0000 7f63e5f8e740 1 --2- 172.21.15.131:0/3739127966 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556cc65688b0 0x556cc659baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:59.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.255+0000 7f63e5f8e740 1 --2- 172.21.15.131:0/3739127966 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556cc6575d40 0x556cc65828d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:59.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.255+0000 7f63e5f8e740 1 --2- 172.21.15.131:0/3739127966 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556cc6565ea0 0x556cc6581ee0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:59.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.255+0000 7f63e5f8e740 1 -- 172.21.15.131:0/3739127966 >> 172.21.15.131:0/3739127966 conn(0x556cc653fe10 msgr2=0x556cc653b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:59.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.256+0000 7f63e5f8e740 1 -- 172.21.15.131:0/3739127966 shutdown_connections 2024-05-26T22:50:59.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.256+0000 7f63e5f8e740 1 -- 172.21.15.131:0/3739127966 wait complete. 2024-05-26T22:50:59.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.256+0000 7f63e5f8e740 1 -- start start 2024-05-26T22:50:59.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.256+0000 7f63e5f8e740 1 Processor -- start 2024-05-26T22:50:59.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.257+0000 7f63e5f8e740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556cc6565ea0 0x556cc6609660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:59.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.257+0000 7f63e5f8e740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556cc65688b0 0x556cc6560e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:59.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.257+0000 7f63e5f8e740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556cc6575d40 0x556cc6561350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:59.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.257+0000 7f63e5f8e740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x556cc64ae1d0 con 0x556cc6575d40 2024-05-26T22:50:59.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.257+0000 7f63e5f8e740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f63dc154500 con 0x556cc65688b0 2024-05-26T22:50:59.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.257+0000 7f63e5f8e740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x556cc6458090 con 0x556cc6565ea0 2024-05-26T22:50:59.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.257+0000 7f63e5f8e740 10 client.0 _ll_register_callbacks cb 0x7f63cc07a030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-26T22:50:59.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.258+0000 7f63e1200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556cc6575d40 0x556cc6561350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:59.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.258+0000 7f63e1200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556cc6575d40 0x556cc6561350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55162/0 (socket says 172.21.15.131:55162) 2024-05-26T22:50:59.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.258+0000 7f63e1200640 1 -- 172.21.15.131:0/2688923554 learned_addr learned my addr 172.21.15.131:0/2688923554 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:59.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.258+0000 7f63d3400640 1 --2- 172.21.15.131:0/2688923554 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556cc65688b0 0x556cc6560e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:59.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.258+0000 7f63e1200640 1 -- 172.21.15.131:0/2688923554 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556cc6565ea0 msgr2=0x556cc6609660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:59.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.258+0000 7f63e1200640 1 --2- 172.21.15.131:0/2688923554 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556cc6565ea0 0x556cc6609660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:59.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.258+0000 7f63e1200640 1 -- 172.21.15.131:0/2688923554 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556cc65688b0 msgr2=0x556cc6560e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:59.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.258+0000 7f63e1200640 1 --2- 172.21.15.131:0/2688923554 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556cc65688b0 0x556cc6560e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:59.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.258+0000 7f63e1200640 1 -- 172.21.15.131:0/2688923554 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556cc614e1b0 con 0x556cc6575d40 2024-05-26T22:50:59.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.258+0000 7f63d3400640 1 --2- 172.21.15.131:0/2688923554 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556cc65688b0 0x556cc6560e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:50:59.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.258+0000 7f63e1200640 1 --2- 172.21.15.131:0/2688923554 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556cc6575d40 0x556cc6561350 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f63d40011e0 tx=0x7f63d400d5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ecd3dff8e0e5da9 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:59.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.258+0000 7f63c9600640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-26T22:50:59.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.259+0000 7f63c9600640 1 -- 172.21.15.131:0/2688923554 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63d40072d0 con 0x556cc6575d40 2024-05-26T22:50:59.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.259+0000 7f63e5f8e740 1 -- 172.21.15.131:0/2688923554 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63cc07aac0 con 0x556cc6575d40 2024-05-26T22:50:59.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.259+0000 7f63c9600640 1 -- 172.21.15.131:0/2688923554 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f63d4007470 con 0x556cc6575d40 2024-05-26T22:50:59.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.259+0000 7f63c9600640 1 -- 172.21.15.131:0/2688923554 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f63d4007670 con 0x556cc6575d40 2024-05-26T22:50:59.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.259+0000 7f63e5f8e740 10 client.5991 fetch_fsmap learned FSMap version 34 2024-05-26T22:50:59.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.259+0000 7f63e5f8e740 1 -- 172.21.15.131:0/2688923554 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) -- 0x556cc6582ec0 con 0x556cc6575d40 2024-05-26T22:50:59.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.259+0000 7f63c9600640 1 -- 172.21.15.131:0/2688923554 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63cc07aac0 con 0x556cc6575d40 2024-05-26T22:50:59.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.259+0000 7f63c9600640 1 -- 172.21.15.131:0/2688923554 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 34) ==== 3542+0+0 (secure 0 0 0) 0x7f63d401c050 con 0x556cc6575d40 2024-05-26T22:50:59.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.259+0000 7f63e5f8e740 10 client.5991 fetch_fsmap finished waiting for FSMap version 34 2024-05-26T22:50:59.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.259+0000 7f63e5f8e740 10 client.5991 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-05-26T22:50:59.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.259+0000 7f63e5f8e740 20 client.5991 populate_metadata read hostname 'smithi131' 2024-05-26T22:50:59.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.260+0000 7f63e5f8e740 1 --2- 172.21.15.131:0/2688923554 >> [v2:172.21.15.131:6834/650770835,v1:172.21.15.131:6836/650770835] conn(0x556cc6609f90 0x556cc660a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:59.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.260+0000 7f63e5f8e740 4 client.5991 mds_command: new command op to 5637 tid=0 multi_id=0 [{"prefix": "perf dump"}] 2024-05-26T22:50:59.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.260+0000 7f63e5f8e740 1 -- 172.21.15.131:0/2688923554 --> [v2:172.21.15.131:6834/650770835,v1:172.21.15.131:6836/650770835] -- command(tid 0: {"prefix": "perf dump"}) -- 0x556cc64a93e0 con 0x556cc6609f90 2024-05-26T22:50:59.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.260+0000 7f63dbe00640 1 --2- 172.21.15.131:0/2688923554 >> [v2:172.21.15.131:6834/650770835,v1:172.21.15.131:6836/650770835] conn(0x556cc6609f90 0x556cc660a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:59.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.260+0000 7f63dbe00640 1 --2- 172.21.15.131:0/2688923554 >> [v2:172.21.15.131:6834/650770835,v1:172.21.15.131:6836/650770835] conn(0x556cc6609f90 0x556cc660a3a0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=735ffe8c790338b4 server_cookie=e6926d6c1455a3dd in_seq=0 out_seq=0 2024-05-26T22:50:59.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.260+0000 7f63c9600640 10 client.5991 ms_handle_connect on v2:172.21.15.131:6834/650770835 2024-05-26T22:50:59.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.261+0000 7f63c9600640 1 -- 172.21.15.131:0/2688923554 <== mds.0 v2:172.21.15.131:6834/650770835 1 ==== command_reply(tid 0: 0 ) ==== 8+0+17869 (crc 0 0 0) 0x556cc64a93e0 con 0x556cc6609f90 2024-05-26T22:50:59.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.261+0000 7f63c9600640 10 client.5991 handle_command_reply: tid=0 2024-05-26T22:50:59.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.261+0000 7f63e5f8e740 1 client.5991 shutdown 2024-05-26T22:50:59.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.261+0000 7f63e5f8e740 1 -- 172.21.15.131:0/2688923554 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556cc6575d40 msgr2=0x556cc6561350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:59.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.261+0000 7f63e5f8e740 1 --2- 172.21.15.131:0/2688923554 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556cc6575d40 0x556cc6561350 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f63d40011e0 tx=0x7f63d400d5e0 comp rx=0 tx=0).stop 2024-05-26T22:50:59.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.261+0000 7f63e5f8e740 1 -- 172.21.15.131:0/2688923554 shutdown_connections 2024-05-26T22:50:59.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.261+0000 7f63e5f8e740 1 --2- 172.21.15.131:0/2688923554 >> [v2:172.21.15.131:6834/650770835,v1:172.21.15.131:6836/650770835] conn(0x556cc6609f90 0x556cc660a3a0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:59.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.261+0000 7f63e5f8e740 1 --2- 172.21.15.131:0/2688923554 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556cc6575d40 0x556cc6561350 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:59.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.261+0000 7f63e5f8e740 1 --2- 172.21.15.131:0/2688923554 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556cc65688b0 0x556cc6560e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:59.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.261+0000 7f63e5f8e740 1 --2- 172.21.15.131:0/2688923554 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x556cc6565ea0 0x556cc6609660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:59.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.261+0000 7f63e5f8e740 1 -- 172.21.15.131:0/2688923554 >> 172.21.15.131:0/2688923554 conn(0x556cc653fe10 msgr2=0x556cc65648e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:59.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.261+0000 7f63c9600640 0 client.5991 ms_handle_reset on v2:172.21.15.131:6834/650770835 2024-05-26T22:50:59.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.261+0000 7f63e5f8e740 1 -- 172.21.15.131:0/2688923554 shutdown_connections 2024-05-26T22:50:59.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.261+0000 7f63e5f8e740 1 -- 172.21.15.131:0/2688923554 wait complete. 2024-05-26T22:50:59.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.261+0000 7f63e5f8e740 20 client.5991 trim_cache size 0 max 16384 2024-05-26T22:50:59.265 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-26T22:50:59.265 INFO:teuthology.orchestra.run.smithi131.stdout: "AsyncMessenger::Worker-0": { 2024-05-26T22:50:59.265 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_messages": 173, 2024-05-26T22:50:59.265 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages": 180, 2024-05-26T22:50:59.265 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_bytes": 43728, 2024-05-26T22:50:59.265 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_bytes": 4323808, 2024-05-26T22:50:59.265 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_created_connections": 13, 2024-05-26T22:50:59.265 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_active_connections": 8, 2024-05-26T22:50:59.265 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_total_time": 0.060093671, 2024-05-26T22:50:59.266 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_send_time": 0.026523993, 2024-05-26T22:50:59.266 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_recv_time": 0.017338866, 2024-05-26T22:50:59.266 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_fast_dispatch_time": 0.010076734, 2024-05-26T22:50:59.266 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages_queue_lat": { 2024-05-26T22:50:59.266 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 172, 2024-05-26T22:50:59.266 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.210971857, 2024-05-26T22:50:59.266 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.001226580 2024-05-26T22:50:59.266 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:59.266 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_handle_ack_lat": { 2024-05-26T22:50:59.266 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 4, 2024-05-26T22:50:59.266 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000402, 2024-05-26T22:50:59.266 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000100 2024-05-26T22:50:59.266 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:59.267 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-26T22:50:59.267 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_encrypted_bytes": 0 2024-05-26T22:50:59.267 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:59.267 INFO:teuthology.orchestra.run.smithi131.stdout: "AsyncMessenger::Worker-1": { 2024-05-26T22:50:59.267 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_messages": 2098, 2024-05-26T22:50:59.267 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages": 4550, 2024-05-26T22:50:59.267 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_bytes": 793294, 2024-05-26T22:50:59.267 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_bytes": 3099066, 2024-05-26T22:50:59.267 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_created_connections": 9, 2024-05-26T22:50:59.267 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_active_connections": 4, 2024-05-26T22:50:59.267 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_total_time": 0.730884817, 2024-05-26T22:50:59.267 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_send_time": 0.435416429, 2024-05-26T22:50:59.267 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_recv_time": 0.359762608, 2024-05-26T22:50:59.267 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_fast_dispatch_time": 0.008548521, 2024-05-26T22:50:59.268 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages_queue_lat": { 2024-05-26T22:50:59.268 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 4551, 2024-05-26T22:50:59.268 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.351357581, 2024-05-26T22:50:59.268 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000077204 2024-05-26T22:50:59.268 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:59.268 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_handle_ack_lat": { 2024-05-26T22:50:59.268 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 5404, 2024-05-26T22:50:59.268 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.002719521, 2024-05-26T22:50:59.268 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000503 2024-05-26T22:50:59.268 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:59.268 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-26T22:50:59.268 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_encrypted_bytes": 100640 2024-05-26T22:50:59.268 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:59.268 INFO:teuthology.orchestra.run.smithi131.stdout: "AsyncMessenger::Worker-2": { 2024-05-26T22:50:59.269 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_messages": 414, 2024-05-26T22:50:59.269 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages": 395, 2024-05-26T22:50:59.269 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_bytes": 219628, 2024-05-26T22:50:59.269 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_bytes": 216507, 2024-05-26T22:50:59.269 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_created_connections": 6, 2024-05-26T22:50:59.269 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_active_connections": 5, 2024-05-26T22:50:59.269 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_total_time": 0.060999273, 2024-05-26T22:50:59.269 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_send_time": 0.022311301, 2024-05-26T22:50:59.269 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_recv_time": 0.029105881, 2024-05-26T22:50:59.269 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_fast_dispatch_time": 0.012039435, 2024-05-26T22:50:59.269 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages_queue_lat": { 2024-05-26T22:50:59.269 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 398, 2024-05-26T22:50:59.269 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.237840189, 2024-05-26T22:50:59.269 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000597588 2024-05-26T22:50:59.270 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:59.270 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_handle_ack_lat": { 2024-05-26T22:50:59.270 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 62, 2024-05-26T22:50:59.270 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000018229, 2024-05-26T22:50:59.270 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000294 2024-05-26T22:50:59.270 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:59.270 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_encrypted_bytes": 128336, 2024-05-26T22:50:59.270 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_encrypted_bytes": 14896 2024-05-26T22:50:59.270 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:59.270 INFO:teuthology.orchestra.run.smithi131.stdout: "cct": { 2024-05-26T22:50:59.270 INFO:teuthology.orchestra.run.smithi131.stdout: "total_workers": 1, 2024-05-26T22:50:59.270 INFO:teuthology.orchestra.run.smithi131.stdout: "unhealthy_workers": 0 2024-05-26T22:50:59.270 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:59.271 INFO:teuthology.orchestra.run.smithi131.stdout: "finisher-MDSRank": { 2024-05-26T22:50:59.271 INFO:teuthology.orchestra.run.smithi131.stdout: "queue_len": 0, 2024-05-26T22:50:59.271 INFO:teuthology.orchestra.run.smithi131.stdout: "complete_latency": { 2024-05-26T22:50:59.271 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 107, 2024-05-26T22:50:59.271 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.851827073, 2024-05-26T22:50:59.271 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.007961000 2024-05-26T22:50:59.271 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:50:59.271 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:59.271 INFO:teuthology.orchestra.run.smithi131.stdout: "finisher-PurgeQueue": { 2024-05-26T22:50:59.271 INFO:teuthology.orchestra.run.smithi131.stdout: "queue_len": 0, 2024-05-26T22:50:59.271 INFO:teuthology.orchestra.run.smithi131.stdout: "complete_latency": { 2024-05-26T22:50:59.271 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 676, 2024-05-26T22:50:59.271 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.095667067, 2024-05-26T22:50:59.271 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000141519 2024-05-26T22:50:59.272 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:50:59.272 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:59.272 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": { 2024-05-26T22:50:59.272 INFO:teuthology.orchestra.run.smithi131.stdout: "request": 1258, 2024-05-26T22:50:59.272 INFO:teuthology.orchestra.run.smithi131.stdout: "reply": 1258, 2024-05-26T22:50:59.272 INFO:teuthology.orchestra.run.smithi131.stdout: "reply_latency": { 2024-05-26T22:50:59.272 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 1258, 2024-05-26T22:50:59.272 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 2.189444543, 2024-05-26T22:50:59.272 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.001740416 2024-05-26T22:50:59.272 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:59.272 INFO:teuthology.orchestra.run.smithi131.stdout: "slow_reply": 0, 2024-05-26T22:50:59.272 INFO:teuthology.orchestra.run.smithi131.stdout: "forward": 0, 2024-05-26T22:50:59.272 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_fetch_complete": 0, 2024-05-26T22:50:59.273 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_fetch_keys": 0, 2024-05-26T22:50:59.273 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_commit": 21, 2024-05-26T22:50:59.273 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_split": 1, 2024-05-26T22:50:59.273 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_merge": 0, 2024-05-26T22:50:59.273 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes": 213, 2024-05-26T22:50:59.273 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_top": 0, 2024-05-26T22:50:59.273 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_bottom": 213, 2024-05-26T22:50:59.273 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_pin_tail": 0, 2024-05-26T22:50:59.273 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_pinned": 212, 2024-05-26T22:50:59.273 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_expired": 124, 2024-05-26T22:50:59.273 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_with_caps": 202, 2024-05-26T22:50:59.273 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": 202, 2024-05-26T22:50:59.273 INFO:teuthology.orchestra.run.smithi131.stdout: "subtrees": 2, 2024-05-26T22:50:59.273 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse": 1285, 2024-05-26T22:50:59.274 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_hit": 1281, 2024-05-26T22:50:59.274 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_forward": 0, 2024-05-26T22:50:59.274 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_discover": 0, 2024-05-26T22:50:59.274 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_dir_fetch": 0, 2024-05-26T22:50:59.274 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_remote_ino": 0, 2024-05-26T22:50:59.274 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_lock": 0, 2024-05-26T22:50:59.274 INFO:teuthology.orchestra.run.smithi131.stdout: "load_cent": 0, 2024-05-26T22:50:59.274 INFO:teuthology.orchestra.run.smithi131.stdout: "q": 0, 2024-05-26T22:50:59.274 INFO:teuthology.orchestra.run.smithi131.stdout: "exported": 5, 2024-05-26T22:50:59.274 INFO:teuthology.orchestra.run.smithi131.stdout: "exported_inodes": 124, 2024-05-26T22:50:59.274 INFO:teuthology.orchestra.run.smithi131.stdout: "imported": 0, 2024-05-26T22:50:59.274 INFO:teuthology.orchestra.run.smithi131.stdout: "imported_inodes": 0, 2024-05-26T22:50:59.275 INFO:teuthology.orchestra.run.smithi131.stdout: "openino_dir_fetch": 0, 2024-05-26T22:50:59.275 INFO:teuthology.orchestra.run.smithi131.stdout: "openino_backtrace_fetch": 0, 2024-05-26T22:50:59.275 INFO:teuthology.orchestra.run.smithi131.stdout: "openino_peer_discover": 0, 2024-05-26T22:50:59.275 INFO:teuthology.orchestra.run.smithi131.stdout: "root_rfiles": 100, 2024-05-26T22:50:59.275 INFO:teuthology.orchestra.run.smithi131.stdout: "root_rbytes": 500, 2024-05-26T22:50:59.275 INFO:teuthology.orchestra.run.smithi131.stdout: "root_rsnaps": 0, 2024-05-26T22:50:59.275 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_backtrace_fetch": 0, 2024-05-26T22:50:59.275 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_set_tag": 0, 2024-05-26T22:50:59.275 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_backtrace_repaired": 0, 2024-05-26T22:50:59.275 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_inotable_repaired": 0, 2024-05-26T22:50:59.275 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_dir_inodes": 0, 2024-05-26T22:50:59.275 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_dir_base_inodes": 0, 2024-05-26T22:50:59.275 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_dirfrag_rstats": 0, 2024-05-26T22:50:59.275 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_file_inodes": 0, 2024-05-26T22:50:59.276 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_inode_file_caps": 0, 2024-05-26T22:50:59.276 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_revoke": 4, 2024-05-26T22:50:59.276 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_grant": 1127, 2024-05-26T22:50:59.276 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_trunc": 0, 2024-05-26T22:50:59.276 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-26T22:50:59.276 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-26T22:50:59.276 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_caps": 616, 2024-05-26T22:50:59.276 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_caps_dirty": 612, 2024-05-26T22:50:59.276 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_cap_release": 4, 2024-05-26T22:50:59.276 INFO:teuthology.orchestra.run.smithi131.stdout: "process_request_cap_release": 620 2024-05-26T22:50:59.276 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:59.276 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_cache": { 2024-05-26T22:50:59.276 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_update": 0, 2024-05-26T22:50:59.276 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_update_receipt": 0, 2024-05-26T22:50:59.276 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_try_discover": 0, 2024-05-26T22:50:59.277 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_send_discover": 0, 2024-05-26T22:50:59.277 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_handle_discover": 7, 2024-05-26T22:50:59.277 INFO:teuthology.orchestra.run.smithi131.stdout: "num_strays": 0, 2024-05-26T22:50:59.277 INFO:teuthology.orchestra.run.smithi131.stdout: "num_strays_delayed": 0, 2024-05-26T22:50:59.277 INFO:teuthology.orchestra.run.smithi131.stdout: "num_strays_enqueuing": 0, 2024-05-26T22:50:59.277 INFO:teuthology.orchestra.run.smithi131.stdout: "strays_created": 513, 2024-05-26T22:50:59.277 INFO:teuthology.orchestra.run.smithi131.stdout: "strays_enqueued": 513, 2024-05-26T22:50:59.277 INFO:teuthology.orchestra.run.smithi131.stdout: "strays_reintegrated": 0, 2024-05-26T22:50:59.277 INFO:teuthology.orchestra.run.smithi131.stdout: "strays_migrated": 0, 2024-05-26T22:50:59.277 INFO:teuthology.orchestra.run.smithi131.stdout: "num_recovering_processing": 0, 2024-05-26T22:50:59.277 INFO:teuthology.orchestra.run.smithi131.stdout: "num_recovering_enqueued": 0, 2024-05-26T22:50:59.277 INFO:teuthology.orchestra.run.smithi131.stdout: "num_recovering_prioritized": 0, 2024-05-26T22:50:59.277 INFO:teuthology.orchestra.run.smithi131.stdout: "recovery_started": 0, 2024-05-26T22:50:59.278 INFO:teuthology.orchestra.run.smithi131.stdout: "recovery_completed": 0, 2024-05-26T22:50:59.278 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_quiesce_path": 0, 2024-05-26T22:50:59.278 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_quiesce_inode": 0, 2024-05-26T22:50:59.278 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_enqueue_scrub": 0, 2024-05-26T22:50:59.278 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_exportdir": 5, 2024-05-26T22:50:59.278 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_flush": 0, 2024-05-26T22:50:59.278 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_fragmentdir": 1, 2024-05-26T22:50:59.278 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_fragstats": 0, 2024-05-26T22:50:59.278 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_inodestats": 0 2024-05-26T22:50:59.278 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:59.278 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_log": { 2024-05-26T22:50:59.278 INFO:teuthology.orchestra.run.smithi131.stdout: "evlrg": 0, 2024-05-26T22:50:59.278 INFO:teuthology.orchestra.run.smithi131.stdout: "evadd": 2373, 2024-05-26T22:50:59.279 INFO:teuthology.orchestra.run.smithi131.stdout: "evex": 0, 2024-05-26T22:50:59.279 INFO:teuthology.orchestra.run.smithi131.stdout: "evtrm": 0, 2024-05-26T22:50:59.279 INFO:teuthology.orchestra.run.smithi131.stdout: "ev": 2373, 2024-05-26T22:50:59.279 INFO:teuthology.orchestra.run.smithi131.stdout: "evexg": 0, 2024-05-26T22:50:59.279 INFO:teuthology.orchestra.run.smithi131.stdout: "evexd": 0, 2024-05-26T22:50:59.279 INFO:teuthology.orchestra.run.smithi131.stdout: "segadd": 4, 2024-05-26T22:50:59.279 INFO:teuthology.orchestra.run.smithi131.stdout: "segex": 0, 2024-05-26T22:50:59.279 INFO:teuthology.orchestra.run.smithi131.stdout: "segtrm": 0, 2024-05-26T22:50:59.279 INFO:teuthology.orchestra.run.smithi131.stdout: "seg": 4, 2024-05-26T22:50:59.279 INFO:teuthology.orchestra.run.smithi131.stdout: "segmjr": 2, 2024-05-26T22:50:59.279 INFO:teuthology.orchestra.run.smithi131.stdout: "segexg": 0, 2024-05-26T22:50:59.279 INFO:teuthology.orchestra.run.smithi131.stdout: "segexd": 0, 2024-05-26T22:50:59.279 INFO:teuthology.orchestra.run.smithi131.stdout: "expos": 4194304, 2024-05-26T22:50:59.279 INFO:teuthology.orchestra.run.smithi131.stdout: "wrpos": 9042338, 2024-05-26T22:50:59.280 INFO:teuthology.orchestra.run.smithi131.stdout: "rdpos": 0, 2024-05-26T22:50:59.280 INFO:teuthology.orchestra.run.smithi131.stdout: "jlat": { 2024-05-26T22:50:59.280 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 23, 2024-05-26T22:50:59.280 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.265165852, 2024-05-26T22:50:59.280 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.011528950 2024-05-26T22:50:59.280 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:59.280 INFO:teuthology.orchestra.run.smithi131.stdout: "replayed": 0 2024-05-26T22:50:59.280 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:59.280 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_mem": { 2024-05-26T22:50:59.280 INFO:teuthology.orchestra.run.smithi131.stdout: "ino": 90, 2024-05-26T22:50:59.280 INFO:teuthology.orchestra.run.smithi131.stdout: "ino+": 727, 2024-05-26T22:50:59.280 INFO:teuthology.orchestra.run.smithi131.stdout: "ino-": 637, 2024-05-26T22:50:59.280 INFO:teuthology.orchestra.run.smithi131.stdout: "dir": 58, 2024-05-26T22:50:59.280 INFO:teuthology.orchestra.run.smithi131.stdout: "dir+": 122, 2024-05-26T22:50:59.281 INFO:teuthology.orchestra.run.smithi131.stdout: "dir-": 64, 2024-05-26T22:50:59.281 INFO:teuthology.orchestra.run.smithi131.stdout: "dn": 89, 2024-05-26T22:50:59.281 INFO:teuthology.orchestra.run.smithi131.stdout: "dn+": 1238, 2024-05-26T22:50:59.281 INFO:teuthology.orchestra.run.smithi131.stdout: "dn-": 1149, 2024-05-26T22:50:59.281 INFO:teuthology.orchestra.run.smithi131.stdout: "cap": 78, 2024-05-26T22:50:59.281 INFO:teuthology.orchestra.run.smithi131.stdout: "cap+": 715, 2024-05-26T22:50:59.281 INFO:teuthology.orchestra.run.smithi131.stdout: "cap-": 637, 2024-05-26T22:50:59.281 INFO:teuthology.orchestra.run.smithi131.stdout: "rss": 63752, 2024-05-26T22:50:59.281 INFO:teuthology.orchestra.run.smithi131.stdout: "heap": 231788 2024-05-26T22:50:59.281 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:59.281 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_server": { 2024-05-26T22:50:59.281 INFO:teuthology.orchestra.run.smithi131.stdout: "dispatch_client_request": 1265, 2024-05-26T22:50:59.281 INFO:teuthology.orchestra.run.smithi131.stdout: "dispatch_server_request": 0, 2024-05-26T22:50:59.282 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_request": 1258, 2024-05-26T22:50:59.282 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_session": 7, 2024-05-26T22:50:59.282 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_peer_request": 0, 2024-05-26T22:50:59.282 INFO:teuthology.orchestra.run.smithi131.stdout: "req_create_latency": { 2024-05-26T22:50:59.282 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 612, 2024-05-26T22:50:59.282 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.939770929, 2024-05-26T22:50:59.282 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.001535573 2024-05-26T22:50:59.282 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:59.282 INFO:teuthology.orchestra.run.smithi131.stdout: "req_getattr_latency": { 2024-05-26T22:50:59.282 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 26, 2024-05-26T22:50:59.282 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.014781499, 2024-05-26T22:50:59.282 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000568519 2024-05-26T22:50:59.282 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:59.282 INFO:teuthology.orchestra.run.smithi131.stdout: "req_getfilelock_latency": { 2024-05-26T22:50:59.283 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:50:59.283 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:50:59.283 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:50:59.283 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:59.283 INFO:teuthology.orchestra.run.smithi131.stdout: "req_link_latency": { 2024-05-26T22:50:59.283 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:50:59.283 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:50:59.283 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:50:59.283 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:59.283 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookup_latency": { 2024-05-26T22:50:59.283 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 1, 2024-05-26T22:50:59.283 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000227305, 2024-05-26T22:50:59.283 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000227305 2024-05-26T22:50:59.283 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:59.284 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookuphash_latency": { 2024-05-26T22:50:59.284 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:50:59.284 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:50:59.284 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:50:59.284 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:59.284 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookupino_latency": { 2024-05-26T22:50:59.284 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:50:59.284 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:50:59.284 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:50:59.284 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:59.284 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookupname_latency": { 2024-05-26T22:50:59.284 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:50:59.284 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:50:59.285 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:50:59.285 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:59.285 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookupparent_latency": { 2024-05-26T22:50:59.285 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:50:59.285 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:50:59.285 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:50:59.285 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:59.285 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookupsnap_latency": { 2024-05-26T22:50:59.285 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:50:59.285 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:50:59.285 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:50:59.285 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:59.285 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lssnap_latency": { 2024-05-26T22:50:59.285 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:50:59.285 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:50:59.286 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:50:59.286 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:59.286 INFO:teuthology.orchestra.run.smithi131.stdout: "req_mkdir_latency": { 2024-05-26T22:50:59.286 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 102, 2024-05-26T22:50:59.286 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.145482933, 2024-05-26T22:50:59.286 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.001426303 2024-05-26T22:50:59.286 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:59.286 INFO:teuthology.orchestra.run.smithi131.stdout: "req_mknod_latency": { 2024-05-26T22:50:59.286 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:50:59.286 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:50:59.286 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:50:59.286 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:59.286 INFO:teuthology.orchestra.run.smithi131.stdout: "req_mksnap_latency": { 2024-05-26T22:50:59.287 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:50:59.287 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:50:59.287 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:50:59.287 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:59.287 INFO:teuthology.orchestra.run.smithi131.stdout: "req_open_latency": { 2024-05-26T22:50:59.287 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 1, 2024-05-26T22:50:59.290 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000161881, 2024-05-26T22:50:59.290 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000161881 2024-05-26T22:50:59.290 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:59.290 INFO:teuthology.orchestra.run.smithi131.stdout: "req_readdir_latency": { 2024-05-26T22:50:59.290 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 1, 2024-05-26T22:50:59.290 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.014387337, 2024-05-26T22:50:59.290 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.014387337 2024-05-26T22:50:59.290 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:59.290 INFO:teuthology.orchestra.run.smithi131.stdout: "req_rename_latency": { 2024-05-26T22:50:59.290 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:50:59.290 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:50:59.291 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:50:59.291 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:59.291 INFO:teuthology.orchestra.run.smithi131.stdout: "req_renamesnap_latency": { 2024-05-26T22:50:59.291 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:50:59.291 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:50:59.291 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:50:59.291 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:59.291 INFO:teuthology.orchestra.run.smithi131.stdout: "req_snapdiff_latency": { 2024-05-26T22:50:59.291 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:50:59.291 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:50:59.291 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:50:59.291 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:59.291 INFO:teuthology.orchestra.run.smithi131.stdout: "req_rmdir_latency": { 2024-05-26T22:50:59.292 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 1, 2024-05-26T22:50:59.292 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.429350021, 2024-05-26T22:50:59.292 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.429350021 2024-05-26T22:50:59.292 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:59.292 INFO:teuthology.orchestra.run.smithi131.stdout: "req_rmsnap_latency": { 2024-05-26T22:50:59.292 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:50:59.292 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:50:59.292 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:50:59.292 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:59.292 INFO:teuthology.orchestra.run.smithi131.stdout: "req_rmxattr_latency": { 2024-05-26T22:50:59.292 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:50:59.292 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:50:59.292 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:50:59.292 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:59.293 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setattr_latency": { 2024-05-26T22:50:59.293 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 1, 2024-05-26T22:50:59.293 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000403915, 2024-05-26T22:50:59.293 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000403915 2024-05-26T22:50:59.293 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:59.293 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setdirlayout_latency": { 2024-05-26T22:50:59.293 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:50:59.293 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:50:59.293 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:50:59.293 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:59.293 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setfilelock_latency": { 2024-05-26T22:50:59.293 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:50:59.293 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:50:59.294 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:50:59.294 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:59.294 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setlayout_latency": { 2024-05-26T22:50:59.294 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:50:59.294 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:50:59.294 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:50:59.294 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:59.294 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setxattr_latency": { 2024-05-26T22:50:59.294 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 1, 2024-05-26T22:50:59.294 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.001301164, 2024-05-26T22:50:59.294 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.001301164 2024-05-26T22:50:59.294 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:59.294 INFO:teuthology.orchestra.run.smithi131.stdout: "req_symlink_latency": { 2024-05-26T22:50:59.295 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:50:59.295 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:50:59.295 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:50:59.295 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:59.295 INFO:teuthology.orchestra.run.smithi131.stdout: "req_unlink_latency": { 2024-05-26T22:50:59.295 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 512, 2024-05-26T22:50:59.295 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.643577559, 2024-05-26T22:50:59.295 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.001256987 2024-05-26T22:50:59.295 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:59.295 INFO:teuthology.orchestra.run.smithi131.stdout: "cap_revoke_eviction": 0, 2024-05-26T22:50:59.295 INFO:teuthology.orchestra.run.smithi131.stdout: "cap_acquisition_throttle": 0, 2024-05-26T22:50:59.295 INFO:teuthology.orchestra.run.smithi131.stdout: "req_getvxattr_latency": { 2024-05-26T22:50:59.295 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:50:59.295 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:50:59.296 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:50:59.296 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:50:59.296 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:59.296 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_sessions": { 2024-05-26T22:50:59.296 INFO:teuthology.orchestra.run.smithi131.stdout: "session_count": 1, 2024-05-26T22:50:59.296 INFO:teuthology.orchestra.run.smithi131.stdout: "session_add": 1, 2024-05-26T22:50:59.296 INFO:teuthology.orchestra.run.smithi131.stdout: "session_remove": 0, 2024-05-26T22:50:59.296 INFO:teuthology.orchestra.run.smithi131.stdout: "sessions_open": 1, 2024-05-26T22:50:59.296 INFO:teuthology.orchestra.run.smithi131.stdout: "sessions_stale": 0, 2024-05-26T22:50:59.296 INFO:teuthology.orchestra.run.smithi131.stdout: "total_load": 1172, 2024-05-26T22:50:59.296 INFO:teuthology.orchestra.run.smithi131.stdout: "average_load": 1172, 2024-05-26T22:50:59.296 INFO:teuthology.orchestra.run.smithi131.stdout: "avg_session_uptime": 16, 2024-05-26T22:50:59.296 INFO:teuthology.orchestra.run.smithi131.stdout: "mdthresh_evicted": 0 2024-05-26T22:50:59.297 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:59.297 INFO:teuthology.orchestra.run.smithi131.stdout: "mempool": { 2024-05-26T22:50:59.297 INFO:teuthology.orchestra.run.smithi131.stdout: "bloom_filter_bytes": 0, 2024-05-26T22:50:59.297 INFO:teuthology.orchestra.run.smithi131.stdout: "bloom_filter_items": 0, 2024-05-26T22:50:59.297 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_alloc_bytes": 0, 2024-05-26T22:50:59.297 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_alloc_items": 0, 2024-05-26T22:50:59.297 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_data_bytes": 0, 2024-05-26T22:50:59.297 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_data_items": 0, 2024-05-26T22:50:59.297 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-26T22:50:59.297 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_onode_items": 0, 2024-05-26T22:50:59.297 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-26T22:50:59.297 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_meta_items": 0, 2024-05-26T22:50:59.297 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_other_bytes": 0, 2024-05-26T22:50:59.297 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_other_items": 0, 2024-05-26T22:50:59.298 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-26T22:50:59.298 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_buffer_items": 0, 2024-05-26T22:50:59.298 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_extent_bytes": 0, 2024-05-26T22:50:59.298 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_extent_items": 0, 2024-05-26T22:50:59.298 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_blob_bytes": 0, 2024-05-26T22:50:59.298 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_blob_items": 0, 2024-05-26T22:50:59.298 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-26T22:50:59.298 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_shared_blob_items": 0, 2024-05-26T22:50:59.298 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-26T22:50:59.298 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_inline_bl_items": 0, 2024-05-26T22:50:59.298 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_fsck_bytes": 0, 2024-05-26T22:50:59.298 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_fsck_items": 0, 2024-05-26T22:50:59.298 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_txc_bytes": 0, 2024-05-26T22:50:59.299 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_txc_items": 0, 2024-05-26T22:50:59.299 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-26T22:50:59.299 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_deferred_items": 0, 2024-05-26T22:50:59.299 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_bytes": 0, 2024-05-26T22:50:59.299 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_items": 0, 2024-05-26T22:50:59.299 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_bytes": 0, 2024-05-26T22:50:59.299 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_items": 0, 2024-05-26T22:50:59.299 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_reader_bytes": 0, 2024-05-26T22:50:59.299 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_reader_items": 0, 2024-05-26T22:50:59.299 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_writer_bytes": 0, 2024-05-26T22:50:59.299 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_writer_items": 0, 2024-05-26T22:50:59.299 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_anon_bytes": 258041, 2024-05-26T22:50:59.299 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_anon_items": 314, 2024-05-26T22:50:59.300 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_meta_bytes": 1672, 2024-05-26T22:50:59.300 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_meta_items": 19, 2024-05-26T22:50:59.300 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_bytes": 0, 2024-05-26T22:50:59.300 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_items": 0, 2024-05-26T22:50:59.300 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_mapbl_bytes": 0, 2024-05-26T22:50:59.300 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_mapbl_items": 0, 2024-05-26T22:50:59.300 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_pglog_bytes": 0, 2024-05-26T22:50:59.300 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_pglog_items": 0, 2024-05-26T22:50:59.300 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_bytes": 5312, 2024-05-26T22:50:59.300 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_items": 75, 2024-05-26T22:50:59.300 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_mapping_bytes": 0, 2024-05-26T22:50:59.300 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_mapping_items": 0, 2024-05-26T22:50:59.300 INFO:teuthology.orchestra.run.smithi131.stdout: "pgmap_bytes": 0, 2024-05-26T22:50:59.300 INFO:teuthology.orchestra.run.smithi131.stdout: "pgmap_items": 0, 2024-05-26T22:50:59.301 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_co_bytes": 899526, 2024-05-26T22:50:59.301 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_co_items": 8850, 2024-05-26T22:50:59.301 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_1_bytes": 0, 2024-05-26T22:50:59.301 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_1_items": 0, 2024-05-26T22:50:59.301 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_2_bytes": 0, 2024-05-26T22:50:59.301 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_2_items": 0 2024-05-26T22:50:59.301 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:59.301 INFO:teuthology.orchestra.run.smithi131.stdout: "objecter": { 2024-05-26T22:50:59.301 INFO:teuthology.orchestra.run.smithi131.stdout: "op_active": 0, 2024-05-26T22:50:59.301 INFO:teuthology.orchestra.run.smithi131.stdout: "op_laggy": 0, 2024-05-26T22:50:59.301 INFO:teuthology.orchestra.run.smithi131.stdout: "op_send": 617, 2024-05-26T22:50:59.301 INFO:teuthology.orchestra.run.smithi131.stdout: "op_send_bytes": 5107149, 2024-05-26T22:50:59.301 INFO:teuthology.orchestra.run.smithi131.stdout: "op_resend": 37, 2024-05-26T22:50:59.301 INFO:teuthology.orchestra.run.smithi131.stdout: "op_reply": 617, 2024-05-26T22:50:59.302 INFO:teuthology.orchestra.run.smithi131.stdout: "op_latency": { 2024-05-26T22:50:59.302 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 617, 2024-05-26T22:50:59.302 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 5.271888540, 2024-05-26T22:50:59.302 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.008544389 2024-05-26T22:50:59.302 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:59.302 INFO:teuthology.orchestra.run.smithi131.stdout: "op_inflight": 0, 2024-05-26T22:50:59.302 INFO:teuthology.orchestra.run.smithi131.stdout: "oplen_avg": { 2024-05-26T22:50:59.302 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 617, 2024-05-26T22:50:59.302 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 665 2024-05-26T22:50:59.302 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:59.302 INFO:teuthology.orchestra.run.smithi131.stdout: "op": 617, 2024-05-26T22:50:59.302 INFO:teuthology.orchestra.run.smithi131.stdout: "op_r": 6, 2024-05-26T22:50:59.303 INFO:teuthology.orchestra.run.smithi131.stdout: "op_w": 611, 2024-05-26T22:50:59.303 INFO:teuthology.orchestra.run.smithi131.stdout: "op_rmw": 0, 2024-05-26T22:50:59.303 INFO:teuthology.orchestra.run.smithi131.stdout: "op_pg": 0, 2024-05-26T22:50:59.303 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_stat": 0, 2024-05-26T22:50:59.303 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_create": 11, 2024-05-26T22:50:59.303 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_read": 6, 2024-05-26T22:50:59.303 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_write": 29, 2024-05-26T22:50:59.303 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_writefull": 11, 2024-05-26T22:50:59.303 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_writesame": 0, 2024-05-26T22:50:59.303 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_append": 0, 2024-05-26T22:50:59.303 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_zero": 0, 2024-05-26T22:50:59.303 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_truncate": 1, 2024-05-26T22:50:59.303 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_delete": 526, 2024-05-26T22:50:59.304 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_mapext": 0, 2024-05-26T22:50:59.304 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_sparse_read": 0, 2024-05-26T22:50:59.304 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_clonerange": 0, 2024-05-26T22:50:59.304 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_getxattr": 0, 2024-05-26T22:50:59.304 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_setxattr": 22, 2024-05-26T22:50:59.304 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_cmpxattr": 0, 2024-05-26T22:50:59.304 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_rmxattr": 0, 2024-05-26T22:50:59.304 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_resetxattrs": 0, 2024-05-26T22:50:59.304 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_call": 0, 2024-05-26T22:50:59.304 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_watch": 0, 2024-05-26T22:50:59.304 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_notify": 0, 2024-05-26T22:50:59.304 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_src_cmpxattr": 0, 2024-05-26T22:50:59.304 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_pgls": 0, 2024-05-26T22:50:59.304 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_pgls_filter": 0, 2024-05-26T22:50:59.305 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_other": 0, 2024-05-26T22:50:59.305 INFO:teuthology.orchestra.run.smithi131.stdout: "linger_active": 0, 2024-05-26T22:50:59.305 INFO:teuthology.orchestra.run.smithi131.stdout: "linger_send": 0, 2024-05-26T22:50:59.305 INFO:teuthology.orchestra.run.smithi131.stdout: "linger_resend": 0, 2024-05-26T22:50:59.305 INFO:teuthology.orchestra.run.smithi131.stdout: "linger_ping": 0, 2024-05-26T22:50:59.305 INFO:teuthology.orchestra.run.smithi131.stdout: "poolop_active": 0, 2024-05-26T22:50:59.305 INFO:teuthology.orchestra.run.smithi131.stdout: "poolop_send": 0, 2024-05-26T22:50:59.305 INFO:teuthology.orchestra.run.smithi131.stdout: "poolop_resend": 0, 2024-05-26T22:50:59.305 INFO:teuthology.orchestra.run.smithi131.stdout: "poolstat_active": 0, 2024-05-26T22:50:59.305 INFO:teuthology.orchestra.run.smithi131.stdout: "poolstat_send": 0, 2024-05-26T22:50:59.305 INFO:teuthology.orchestra.run.smithi131.stdout: "poolstat_resend": 0, 2024-05-26T22:50:59.305 INFO:teuthology.orchestra.run.smithi131.stdout: "statfs_active": 0, 2024-05-26T22:50:59.305 INFO:teuthology.orchestra.run.smithi131.stdout: "statfs_send": 0, 2024-05-26T22:50:59.305 INFO:teuthology.orchestra.run.smithi131.stdout: "statfs_resend": 0, 2024-05-26T22:50:59.306 INFO:teuthology.orchestra.run.smithi131.stdout: "command_active": 0, 2024-05-26T22:50:59.306 INFO:teuthology.orchestra.run.smithi131.stdout: "command_send": 0, 2024-05-26T22:50:59.306 INFO:teuthology.orchestra.run.smithi131.stdout: "command_resend": 0, 2024-05-26T22:50:59.306 INFO:teuthology.orchestra.run.smithi131.stdout: "map_epoch": 44, 2024-05-26T22:50:59.306 INFO:teuthology.orchestra.run.smithi131.stdout: "map_full": 0, 2024-05-26T22:50:59.306 INFO:teuthology.orchestra.run.smithi131.stdout: "map_inc": 1, 2024-05-26T22:50:59.306 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_sessions": 8, 2024-05-26T22:50:59.306 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_session_open": 8, 2024-05-26T22:50:59.306 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_session_close": 0, 2024-05-26T22:50:59.306 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_laggy": 0, 2024-05-26T22:50:59.306 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_wr": 52, 2024-05-26T22:50:59.306 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_rd": 0, 2024-05-26T22:50:59.306 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_del": 7 2024-05-26T22:50:59.307 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:59.307 INFO:teuthology.orchestra.run.smithi131.stdout: "oft": { 2024-05-26T22:50:59.307 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_total_objs": 1, 2024-05-26T22:50:59.307 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_total_kv_pairs": 0, 2024-05-26T22:50:59.307 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_total_updates": 0, 2024-05-26T22:50:59.307 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_total_removes": 0 2024-05-26T22:50:59.307 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:59.307 INFO:teuthology.orchestra.run.smithi131.stdout: "osd-slow-ops": { 2024-05-26T22:50:59.307 INFO:teuthology.orchestra.run.smithi131.stdout: "slow_ops_count": 0 2024-05-26T22:50:59.307 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:59.307 INFO:teuthology.orchestra.run.smithi131.stdout: "purge_queue": { 2024-05-26T22:50:59.307 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executing_ops": 0, 2024-05-26T22:50:59.307 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executing_ops_high_water": 32, 2024-05-26T22:50:59.308 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executing": 0, 2024-05-26T22:50:59.308 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executing_high_water": 32, 2024-05-26T22:50:59.308 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executed_ops": 513, 2024-05-26T22:50:59.308 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executed": 513, 2024-05-26T22:50:59.308 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_item_in_journal": 0 2024-05-26T22:50:59.308 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:59.308 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-26T22:50:59.308 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-05-26T22:50:59.308 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 104857600, 2024-05-26T22:50:59.308 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-05-26T22:50:59.308 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 2681, 2024-05-26T22:50:59.308 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 810570, 2024-05-26T22:50:59.308 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-05-26T22:50:59.308 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 2681, 2024-05-26T22:50:59.309 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 0, 2024-05-26T22:50:59.309 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 0, 2024-05-26T22:50:59.309 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 2681, 2024-05-26T22:50:59.309 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 810570, 2024-05-26T22:50:59.309 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-05-26T22:50:59.309 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:50:59.309 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:50:59.309 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:50:59.309 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:50:59.309 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:59.309 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-objecter_bytes": { 2024-05-26T22:50:59.309 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-05-26T22:50:59.309 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 104857600, 2024-05-26T22:50:59.309 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-05-26T22:50:59.310 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 0, 2024-05-26T22:50:59.310 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 0, 2024-05-26T22:50:59.310 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-05-26T22:50:59.310 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 0, 2024-05-26T22:50:59.310 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 617, 2024-05-26T22:50:59.310 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 5158962, 2024-05-26T22:50:59.310 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 617, 2024-05-26T22:50:59.310 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 5158962, 2024-05-26T22:50:59.310 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-05-26T22:50:59.310 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:50:59.310 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:50:59.310 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:50:59.310 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:50:59.311 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:59.311 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-objecter_ops": { 2024-05-26T22:50:59.311 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-05-26T22:50:59.311 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 1024, 2024-05-26T22:50:59.311 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-05-26T22:50:59.311 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 0, 2024-05-26T22:50:59.311 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 0, 2024-05-26T22:50:59.311 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-05-26T22:50:59.311 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 0, 2024-05-26T22:50:59.311 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 617, 2024-05-26T22:50:59.311 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 617, 2024-05-26T22:50:59.311 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 617, 2024-05-26T22:50:59.312 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 617, 2024-05-26T22:50:59.312 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-05-26T22:50:59.312 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:50:59.312 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:50:59.312 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:50:59.312 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:50:59.312 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:59.312 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-write_buf_throttle": { 2024-05-26T22:50:59.312 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-05-26T22:50:59.312 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 3758096384, 2024-05-26T22:50:59.312 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-05-26T22:50:59.312 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 513, 2024-05-26T22:50:59.312 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 51813, 2024-05-26T22:50:59.313 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-05-26T22:50:59.313 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 513, 2024-05-26T22:50:59.313 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 0, 2024-05-26T22:50:59.313 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 0, 2024-05-26T22:50:59.313 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 6, 2024-05-26T22:50:59.313 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 51813, 2024-05-26T22:50:59.313 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-05-26T22:50:59.313 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:50:59.313 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:50:59.313 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:50:59.313 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:50:59.313 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:50:59.313 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-write_buf_throttle-0x557025a46500": { 2024-05-26T22:50:59.313 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-05-26T22:50:59.314 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 3758096384, 2024-05-26T22:50:59.314 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-05-26T22:50:59.314 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 2373, 2024-05-26T22:50:59.314 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 4848034, 2024-05-26T22:50:59.314 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-05-26T22:50:59.314 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 2373, 2024-05-26T22:50:59.314 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 0, 2024-05-26T22:50:59.314 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 0, 2024-05-26T22:50:59.314 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 23, 2024-05-26T22:50:59.314 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 4848034, 2024-05-26T22:50:59.314 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-05-26T22:50:59.314 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:50:59.314 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:50:59.315 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:50:59.315 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:50:59.315 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:50:59.315 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-26T22:50:59.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.262+0000 7f63e3200640 1 -- 172.21.15.131:0/1034723795 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f63b806be30 msgr2=0x7f63b806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:59.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.262+0000 7f63e3200640 1 --2- 172.21.15.131:0/1034723795 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f63b806be30 0x7f63b806e2f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f63dc161610 tx=0x7f63c4008040 comp rx=0 tx=0).stop 2024-05-26T22:50:59.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.262+0000 7f63e3200640 1 -- 172.21.15.131:0/1034723795 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63c00039b0 msgr2=0x7f63dc161890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:59.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.262+0000 7f63e3200640 1 --2- 172.21.15.131:0/1034723795 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63c00039b0 0x7f63dc161890 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f63cc02af10 tx=0x7f63cc009750 comp rx=0 tx=0).stop 2024-05-26T22:50:59.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.262+0000 7f63e3200640 1 -- 172.21.15.131:0/1034723795 shutdown_connections 2024-05-26T22:50:59.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.262+0000 7f63e3200640 1 --2- 172.21.15.131:0/1034723795 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f63dc157410 0x7f63dc1488f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:59.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.262+0000 7f63e3200640 1 --2- 172.21.15.131:0/1034723795 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f63b806be30 0x7f63b806e2f0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:59.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.262+0000 7f63e3200640 1 --2- 172.21.15.131:0/1034723795 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f63dc160030 0x7f63dc1604b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:59.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.262+0000 7f63e3200640 1 --2- 172.21.15.131:0/1034723795 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63c00039b0 0x7f63dc161890 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:59.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.262+0000 7f63e3200640 1 -- 172.21.15.131:0/1034723795 >> 172.21.15.131:0/1034723795 conn(0x7f63dc06b940 msgr2=0x7f63dc15dab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:59.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.262+0000 7f63e3200640 1 -- 172.21.15.131:0/1034723795 shutdown_connections 2024-05-26T22:50:59.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.262+0000 7f63e3200640 1 -- 172.21.15.131:0/1034723795 wait complete. 2024-05-26T22:50:59.318 INFO:tasks.cephfs.test_exports:export stats: [(0, 5), (1, 0), (2, 0)] 2024-05-26T22:50:59.318 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail 3:1 2024-05-26T22:50:59.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.518+0000 7f9d21e00640 1 Processor -- start 2024-05-26T22:50:59.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.518+0000 7f9d21e00640 1 -- start start 2024-05-26T22:50:59.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.519+0000 7f9d21e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d1c155770 0x7f9d1c15b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:59.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.519+0000 7f9d21e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f9d1c15bc40 0x7f9d1c06dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:59.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.520+0000 7f9d21e00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f9d1c06e270 0x7f9d1c0ceba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:59.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.520+0000 7f9d21e00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f9d1c06e6a0 con 0x7f9d1c15bc40 2024-05-26T22:50:59.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.520+0000 7f9d21e00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f9d1c15c1c0 con 0x7f9d1c06e270 2024-05-26T22:50:59.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.520+0000 7f9d21e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9d1c15c4a0 con 0x7f9d1c155770 2024-05-26T22:50:59.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.520+0000 7f9d1a000640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f9d1c15bc40 0x7f9d1c06dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:59.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.520+0000 7f9d1a000640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f9d1c15bc40 0x7f9d1c06dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55178/0 (socket says 172.21.15.131:55178) 2024-05-26T22:50:59.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.520+0000 7f9d1a000640 1 -- 172.21.15.131:0/1421691456 learned_addr learned my addr 172.21.15.131:0/1421691456 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:59.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.520+0000 7f9d1b400640 1 --2- 172.21.15.131:0/1421691456 >> v2:172.21.15.187:3300/0 conn(0x7f9d1c06e270 0x7f9d1c0ceba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:59.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.520+0000 7f9d1aa00640 1 --2- 172.21.15.131:0/1421691456 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d1c155770 0x7f9d1c15b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:59.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.520+0000 7f9d1a000640 1 -- 172.21.15.131:0/1421691456 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d1c155770 msgr2=0x7f9d1c15b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:59.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.520+0000 7f9d1a000640 1 --2- 172.21.15.131:0/1421691456 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d1c155770 0x7f9d1c15b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:59.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.520+0000 7f9d1a000640 1 -- 172.21.15.131:0/1421691456 >> v2:172.21.15.187:3300/0 conn(0x7f9d1c06e270 msgr2=0x7f9d1c0ceba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:50:59.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.520+0000 7f9d1a000640 1 --2- 172.21.15.131:0/1421691456 >> v2:172.21.15.187:3300/0 conn(0x7f9d1c06e270 0x7f9d1c0ceba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:59.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.520+0000 7f9d1a000640 1 -- 172.21.15.131:0/1421691456 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d1c071da0 con 0x7f9d1c15bc40 2024-05-26T22:50:59.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.522+0000 7f9d1a000640 1 --2- 172.21.15.131:0/1421691456 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d1c15bc40 0x7f9d1c06dd30 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f9d1c0d0ed0 tx=0x7f9d0402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d57f1e4bc35e403 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:59.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.522+0000 7f9d19600640 1 -- 172.21.15.131:0/1421691456 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d04037070 con 0x7f9d1c15bc40 2024-05-26T22:50:59.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.522+0000 7f9d19600640 1 -- 172.21.15.131:0/1421691456 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d04032440 con 0x7f9d1c15bc40 2024-05-26T22:50:59.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.522+0000 7f9d19600640 1 -- 172.21.15.131:0/1421691456 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d04032810 con 0x7f9d1c15bc40 2024-05-26T22:50:59.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.522+0000 7f9d21e00640 1 -- 172.21.15.131:0/1421691456 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d1c15bc40 msgr2=0x7f9d1c06dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:59.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.522+0000 7f9d21e00640 1 --2- 172.21.15.131:0/1421691456 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d1c15bc40 0x7f9d1c06dd30 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f9d1c0d0ed0 tx=0x7f9d0402a270 comp rx=0 tx=0).stop 2024-05-26T22:50:59.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.523+0000 7f9d21e00640 1 -- 172.21.15.131:0/1421691456 shutdown_connections 2024-05-26T22:50:59.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.523+0000 7f9d21e00640 1 --2- 172.21.15.131:0/1421691456 >> v2:172.21.15.187:3300/0 conn(0x7f9d1c06e270 0x7f9d1c0ceba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:59.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.523+0000 7f9d21e00640 1 --2- 172.21.15.131:0/1421691456 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d1c15bc40 0x7f9d1c06dd30 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:59.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.523+0000 7f9d21e00640 1 --2- 172.21.15.131:0/1421691456 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d1c155770 0x7f9d1c15b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:59.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.523+0000 7f9d21e00640 1 -- 172.21.15.131:0/1421691456 >> 172.21.15.131:0/1421691456 conn(0x7f9d1c06bc90 msgr2=0x7f9d1c0b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:59.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.523+0000 7f9d21e00640 1 -- 172.21.15.131:0/1421691456 shutdown_connections 2024-05-26T22:50:59.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.523+0000 7f9d21e00640 1 -- 172.21.15.131:0/1421691456 wait complete. 2024-05-26T22:50:59.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.524+0000 7f9d21e00640 1 Processor -- start 2024-05-26T22:50:59.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.525+0000 7f9d21e00640 1 -- start start 2024-05-26T22:50:59.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.525+0000 7f9d21e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d1c06e270 0x7f9d1c1f8190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:59.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.525+0000 7f9d21e00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d1c155770 0x7f9d1c0d0000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:59.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.525+0000 7f9d21e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d1c15bc40 0x7f9d1c152500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:59.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.526+0000 7f9d21e00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f9d1c15c8c0 con 0x7f9d1c15bc40 2024-05-26T22:50:59.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.526+0000 7f9d21e00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9d1c155b50 con 0x7f9d1c155770 2024-05-26T22:50:59.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.526+0000 7f9d21e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9d1c09de00 con 0x7f9d1c06e270 2024-05-26T22:50:59.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.526+0000 7f9d1b400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d1c15bc40 0x7f9d1c152500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:59.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.526+0000 7f9d1b400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d1c15bc40 0x7f9d1c152500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55192/0 (socket says 172.21.15.131:55192) 2024-05-26T22:50:59.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.526+0000 7f9d1b400640 1 -- 172.21.15.131:0/202575136 learned_addr learned my addr 172.21.15.131:0/202575136 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:50:59.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.526+0000 7f9d1a000640 1 --2- 172.21.15.131:0/202575136 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d1c155770 0x7f9d1c0d0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:59.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.526+0000 7f9d1b400640 1 -- 172.21.15.131:0/202575136 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d1c06e270 msgr2=0x7f9d1c1f8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:59.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.526+0000 7f9d1aa00640 1 --2- 172.21.15.131:0/202575136 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d1c06e270 0x7f9d1c1f8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:59.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.526+0000 7f9d1b400640 1 --2- 172.21.15.131:0/202575136 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d1c06e270 0x7f9d1c1f8190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:59.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.526+0000 7f9d1b400640 1 -- 172.21.15.131:0/202575136 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d1c155770 msgr2=0x7f9d1c0d0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:59.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.526+0000 7f9d1b400640 1 --2- 172.21.15.131:0/202575136 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d1c155770 0x7f9d1c0d0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:59.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.526+0000 7f9d1b400640 1 -- 172.21.15.131:0/202575136 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d1c14f150 con 0x7f9d1c15bc40 2024-05-26T22:50:59.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.526+0000 7f9d1a000640 1 --2- 172.21.15.131:0/202575136 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d1c155770 0x7f9d1c0d0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:50:59.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.527+0000 7f9d1b400640 1 --2- 172.21.15.131:0/202575136 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d1c15bc40 0x7f9d1c152500 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f9d1400d990 tx=0x7f9d1400de50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:59.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.527+0000 7f9d13400640 1 -- 172.21.15.131:0/202575136 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d1400aa40 con 0x7f9d1c15bc40 2024-05-26T22:50:59.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.527+0000 7f9d13400640 1 -- 172.21.15.131:0/202575136 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d1400abe0 con 0x7f9d1c15bc40 2024-05-26T22:50:59.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.527+0000 7f9d21e00640 1 -- 172.21.15.131:0/202575136 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d1c14f430 con 0x7f9d1c15bc40 2024-05-26T22:50:59.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.527+0000 7f9d13400640 1 -- 172.21.15.131:0/202575136 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d14010470 con 0x7f9d1c15bc40 2024-05-26T22:50:59.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.527+0000 7f9d21e00640 1 -- 172.21.15.131:0/202575136 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d1c14ee50 con 0x7f9d1c15bc40 2024-05-26T22:50:59.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.528+0000 7f9d21e00640 1 -- 172.21.15.131:0/202575136 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d1c074e90 con 0x7f9d1c15bc40 2024-05-26T22:50:59.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.529+0000 7f9d13400640 1 -- 172.21.15.131:0/202575136 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9d14010610 con 0x7f9d1c15bc40 2024-05-26T22:50:59.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.530+0000 7f9d13400640 1 --2- 172.21.15.131:0/202575136 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9cf806be60 0x7f9cf806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:50:59.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.530+0000 7f9d13400640 1 -- 172.21.15.131:0/202575136 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f9d1408b340 con 0x7f9d1c15bc40 2024-05-26T22:50:59.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.530+0000 7f9d1aa00640 1 --2- 172.21.15.131:0/202575136 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9cf806be60 0x7f9cf806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:50:59.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.531+0000 7f9d1aa00640 1 --2- 172.21.15.131:0/202575136 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9cf806be60 0x7f9cf806e320 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c009e90 tx=0x7f9d0c00b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:50:59.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.534+0000 7f9d13400640 1 -- 172.21.15.131:0/202575136 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f9d14015030 con 0x7f9d1c15bc40 2024-05-26T22:50:59.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.821+0000 7f9d21e00640 1 -- 172.21.15.131:0/202575136 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "3:1"} v 0) -- 0x7f9d1c1f7780 con 0x7f9d1c15bc40 2024-05-26T22:50:59.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.825+0000 7f9d13400640 1 -- 172.21.15.131:0/202575136 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "3:1"}]=0 failed mds gid 4918 v35) ==== 97+0+0 (secure 0 0 0) 0x7f9d14059c50 con 0x7f9d1c15bc40 2024-05-26T22:50:59.825 INFO:teuthology.orchestra.run.smithi131.stderr:failed mds gid 4918 2024-05-26T22:50:59.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.829+0000 7f9d21e00640 1 -- 172.21.15.131:0/202575136 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9cf806be60 msgr2=0x7f9cf806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:59.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.829+0000 7f9d21e00640 1 --2- 172.21.15.131:0/202575136 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9cf806be60 0x7f9cf806e320 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c009e90 tx=0x7f9d0c00b040 comp rx=0 tx=0).stop 2024-05-26T22:50:59.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.829+0000 7f9d21e00640 1 -- 172.21.15.131:0/202575136 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d1c15bc40 msgr2=0x7f9d1c152500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:50:59.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.829+0000 7f9d21e00640 1 --2- 172.21.15.131:0/202575136 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d1c15bc40 0x7f9d1c152500 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f9d1400d990 tx=0x7f9d1400de50 comp rx=0 tx=0).stop 2024-05-26T22:50:59.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.829+0000 7f9d21e00640 1 -- 172.21.15.131:0/202575136 shutdown_connections 2024-05-26T22:50:59.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.829+0000 7f9d21e00640 1 --2- 172.21.15.131:0/202575136 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9cf806be60 0x7f9cf806e320 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:59.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.829+0000 7f9d21e00640 1 --2- 172.21.15.131:0/202575136 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d1c15bc40 0x7f9d1c152500 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:59.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.829+0000 7f9d21e00640 1 --2- 172.21.15.131:0/202575136 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d1c155770 0x7f9d1c0d0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:59.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.829+0000 7f9d21e00640 1 --2- 172.21.15.131:0/202575136 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d1c06e270 0x7f9d1c1f8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:50:59.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.829+0000 7f9d21e00640 1 -- 172.21.15.131:0/202575136 >> 172.21.15.131:0/202575136 conn(0x7f9d1c06bc90 msgr2=0x7f9d1c0b1700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:50:59.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.830+0000 7f9d21e00640 1 -- 172.21.15.131:0/202575136 shutdown_connections 2024-05-26T22:50:59.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:50:59.830+0000 7f9d21e00640 1 -- 172.21.15.131:0/202575136 wait complete. 2024-05-26T22:50:59.850 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-26T22:50:59.957 INFO:tasks.ceph.mds.d.smithi187.stdout:starting mds.d at 2024-05-26T22:51:00.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:00.024+0000 7fb1be600640 1 Processor -- start 2024-05-26T22:51:00.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:00.024+0000 7fb1be600640 1 -- start start 2024-05-26T22:51:00.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:00.025+0000 7fb1be600640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fb1b8155840 0x7fb1b8071930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:00.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:00.025+0000 7fb1be600640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fb1b806f660 0x7fb1b80ade40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:00.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:00.026+0000 7fb1be600640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1b80722a0 con 0x7fb1b815c3d0 2024-05-26T22:51:00.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:00.026+0000 7fb1be600640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fb1b8072580 con 0x7fb1b806f660 2024-05-26T22:51:00.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:00.026+0000 7fb1be600640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fb1b8072860 con 0x7fb1b8155840 2024-05-26T22:51:00.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:00.026+0000 7fb1b7e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fb1b806f660 0x7fb1b80ade40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:00.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:00.026+0000 7fb1b6a00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fb1b815c3d0 0x7fb1b806ef50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:44818/0 (socket says 172.21.15.131:44818) 2024-05-26T22:51:00.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:00.026+0000 7fb1b6a00640 1 -- 172.21.15.131:0/2892918053 learned_addr learned my addr 172.21.15.131:0/2892918053 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:51:00.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:00.026+0000 7fb1b7400640 1 --2- 172.21.15.131:0/2892918053 >> v2:172.21.15.187:3300/0 conn(0x7fb1b8155840 0x7fb1b8071930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:00.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:00.026+0000 7fb1b6000640 1 -- 172.21.15.131:0/2892918053 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3298641715 0 0) 0x7fb1b80722a0 con 0x7fb1b815c3d0 2024-05-26T22:51:00.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:00.026+0000 7fb1b6000640 1 -- 172.21.15.131:0/2892918053 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb19c003520 con 0x7fb1b815c3d0 2024-05-26T22:51:00.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:00.027+0000 7fb1b7e00640 1 -- 172.21.15.131:0/2892918053 >> v2:172.21.15.187:3300/0 conn(0x7fb1b8155840 msgr2=0x7fb1b8071930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:00.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:00.027+0000 7fb1b7e00640 1 --2- 172.21.15.131:0/2892918053 >> v2:172.21.15.187:3300/0 conn(0x7fb1b8155840 0x7fb1b8071930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:00.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:00.027+0000 7fb1b7e00640 1 -- 172.21.15.131:0/2892918053 >> v1:172.21.15.131:6789/0 conn(0x7fb1b815c3d0 legacy=0x7fb1b806ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:00.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:00.027+0000 7fb1b7e00640 1 -- 172.21.15.131:0/2892918053 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1b8072b40 con 0x7fb1b806f660 2024-05-26T22:51:00.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:00.028+0000 7fb1b7e00640 1 --2- 172.21.15.131:0/2892918053 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1b806f660 0x7fb1b80ade40 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fb1b806d400 tx=0x7fb1b002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=345878b2e75735bc server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:00.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:00.028+0000 7fb1b6000640 1 -- 172.21.15.131:0/2892918053 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1b0007740 con 0x7fb1b806f660 2024-05-26T22:51:00.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:00.028+0000 7fb1b6000640 1 -- 172.21.15.131:0/2892918053 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1b00078e0 con 0x7fb1b806f660 2024-05-26T22:51:00.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:00.028+0000 7fb1b6000640 1 -- 172.21.15.131:0/2892918053 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1b0007cb0 con 0x7fb1b806f660 2024-05-26T22:51:00.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:00.028+0000 7fb1be600640 1 -- 172.21.15.131:0/2892918053 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1b806f660 msgr2=0x7fb1b80ade40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:00.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:00.028+0000 7fb1be600640 1 --2- 172.21.15.131:0/2892918053 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1b806f660 0x7fb1b80ade40 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fb1b806d400 tx=0x7fb1b002a680 comp rx=0 tx=0).stop 2024-05-26T22:51:00.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:00.029+0000 7fb1be600640 1 -- 172.21.15.131:0/2892918053 shutdown_connections 2024-05-26T22:51:00.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:00.029+0000 7fb1be600640 1 --2- 172.21.15.131:0/2892918053 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1b806f660 0x7fb1b80ade40 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:00.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:00.029+0000 7fb1be600640 1 --2- 172.21.15.131:0/2892918053 >> v2:172.21.15.187:3300/0 conn(0x7fb1b8155840 0x7fb1b8071930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:00.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:00.029+0000 7fb1be600640 1 -- 172.21.15.131:0/2892918053 >> 172.21.15.131:0/2892918053 conn(0x7fb1b80b1e90 msgr2=0x7fb1b80b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:00.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:00.029+0000 7fb1be600640 1 -- 172.21.15.131:0/2892918053 shutdown_connections 2024-05-26T22:51:00.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:00.029+0000 7fb1be600640 1 -- 172.21.15.131:0/2892918053 wait complete. 2024-05-26T22:51:00.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:00.030+0000 7fb1be600640 1 Processor -- start 2024-05-26T22:51:00.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:00.031+0000 7fb1be600640 1 -- start start 2024-05-26T22:51:00.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:00.031+0000 7fb1be600640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb1b806f660 0x7fb1b81584a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:00.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:00.031+0000 7fb1be600640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb1b8155840 0x7fb1b80b05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:00.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:00.031+0000 7fb1be600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1b815c3d0 0x7fb1b814a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:00.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:00.032+0000 7fb1be600640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b8071ff0 con 0x7fb1b815c3d0 2024-05-26T22:51:00.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:00.032+0000 7fb1be600640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b8071e70 con 0x7fb1b806f660 2024-05-26T22:51:00.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:00.032+0000 7fb1be600640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb1b80ae380 con 0x7fb1b8155840 2024-05-26T22:51:00.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:00.032+0000 7fb1b7e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1b815c3d0 0x7fb1b814a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:00.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:00.032+0000 7fb1b7e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1b815c3d0 0x7fb1b814a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55212/0 (socket says 172.21.15.131:55212) 2024-05-26T22:51:00.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:00.032+0000 7fb1b7e00640 1 -- 172.21.15.131:0/20793156 learned_addr learned my addr 172.21.15.131:0/20793156 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:00.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:00.032+0000 7fb1b7400640 1 --2- 172.21.15.131:0/20793156 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb1b806f660 0x7fb1b81584a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:00.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:00.032+0000 7fb1b6a00640 1 --2- 172.21.15.131:0/20793156 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb1b8155840 0x7fb1b80b05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:00.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:00.032+0000 7fb1b7e00640 1 -- 172.21.15.131:0/20793156 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb1b8155840 msgr2=0x7fb1b80b05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:00.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:00.032+0000 7fb1b7e00640 1 --2- 172.21.15.131:0/20793156 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb1b8155840 0x7fb1b80b05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:00.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:00.032+0000 7fb1b7e00640 1 -- 172.21.15.131:0/20793156 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb1b806f660 msgr2=0x7fb1b81584a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:00.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:00.032+0000 7fb1b7e00640 1 --2- 172.21.15.131:0/20793156 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb1b806f660 0x7fb1b81584a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:00.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:00.032+0000 7fb1b7e00640 1 -- 172.21.15.131:0/20793156 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1b815de40 con 0x7fb1b815c3d0 2024-05-26T22:51:00.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:00.033+0000 7fb1b7e00640 1 --2- 172.21.15.131:0/20793156 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1b815c3d0 0x7fb1b814a690 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fb1b0009e50 tx=0x7fb1b0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:00.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:00.033+0000 7fb1af400640 1 -- 172.21.15.131:0/20793156 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1b0040070 con 0x7fb1b815c3d0 2024-05-26T22:51:00.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:00.033+0000 7fb1af400640 1 -- 172.21.15.131:0/20793156 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1b0031bf0 con 0x7fb1b815c3d0 2024-05-26T22:51:00.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:00.033+0000 7fb1af400640 1 -- 172.21.15.131:0/20793156 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1b0009450 con 0x7fb1b815c3d0 2024-05-26T22:51:00.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:00.033+0000 7fb1be600640 1 -- 172.21.15.131:0/20793156 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1b80b0c50 con 0x7fb1b815c3d0 2024-05-26T22:51:00.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:00.033+0000 7fb1be600640 1 -- 172.21.15.131:0/20793156 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1b8154400 con 0x7fb1b815c3d0 2024-05-26T22:51:00.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:00.034+0000 7fb1be600640 1 -- 172.21.15.131:0/20793156 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1b80694e0 con 0x7fb1b815c3d0 2024-05-26T22:51:00.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:00.035+0000 7fb1af400640 1 -- 172.21.15.131:0/20793156 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb1b0039040 con 0x7fb1b815c3d0 2024-05-26T22:51:00.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:00.036+0000 7fb1af400640 1 --2- 172.21.15.131:0/20793156 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb19806be30 0x7fb19806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:00.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:00.036+0000 7fb1af400640 1 -- 172.21.15.131:0/20793156 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5795+0+0 (secure 0 0 0) 0x7fb1b00ac810 con 0x7fb1b815c3d0 2024-05-26T22:51:00.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:00.036+0000 7fb1b7400640 1 --2- 172.21.15.131:0/20793156 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb19806be30 0x7fb19806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:00.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:00.037+0000 7fb1b7400640 1 --2- 172.21.15.131:0/20793156 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb19806be30 0x7fb19806e2f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fb1a8009ad0 tx=0x7fb1a8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:00.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:00.041+0000 7fb1af400640 1 -- 172.21.15.131:0/20793156 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fb1b007b280 con 0x7fb1b815c3d0 2024-05-26T22:51:00.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:00.275+0000 7fb1be600640 1 -- 172.21.15.131:0/20793156 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb1b80d0b10 con 0x7fb1b815c3d0 2024-05-26T22:51:00.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:00.276+0000 7fb1af400640 1 -- 172.21.15.131:0/20793156 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+4397 (secure 0 0 0) 0x7fb1b007ab80 con 0x7fb1b815c3d0 2024-05-26T22:51:00.276 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:51:00.276 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":36,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-26T22:50:27.001665+0000","modified":"2024-05-26T22:50:59.832658+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":45,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5637,"mds_1":4661,"mds_2":4670},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4661":{"gid":4661,"name":"a","rank":1,"incarnation":36,"state":"up:replay","state_seq":1,"addr":"172.21.15.131:6837/265040431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6835","nonce":265040431},{"type":"v1","addr":"172.21.15.131:6837","nonce":265040431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4670":{"gid":4670,"name":"b","rank":2,"incarnation":32,"state":"up:active","state_seq":8,"addr":"172.21.15.187:6834/1027783169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1027783169},{"type":"v1","addr":"172.21.15.187:6834","nonce":1027783169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5637":{"gid":5637,"name":"c","rank":0,"incarnation":25,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6836/650770835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":650770835},{"type":"v1","addr":"172.21.15.131:6836","nonce":650770835}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5637,"qdb_cluster":[5637,4670]},"id":3}]} 2024-05-26T22:51:00.277 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 36 2024-05-26T22:51:00.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:00.279+0000 7fb1be600640 1 -- 172.21.15.131:0/20793156 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb19806be30 msgr2=0x7fb19806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:00.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:00.279+0000 7fb1be600640 1 --2- 172.21.15.131:0/20793156 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb19806be30 0x7fb19806e2f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fb1a8009ad0 tx=0x7fb1a8008040 comp rx=0 tx=0).stop 2024-05-26T22:51:00.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:00.279+0000 7fb1be600640 1 -- 172.21.15.131:0/20793156 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1b815c3d0 msgr2=0x7fb1b814a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:00.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:00.279+0000 7fb1be600640 1 --2- 172.21.15.131:0/20793156 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1b815c3d0 0x7fb1b814a690 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fb1b0009e50 tx=0x7fb1b0004a40 comp rx=0 tx=0).stop 2024-05-26T22:51:00.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:00.279+0000 7fb1be600640 1 -- 172.21.15.131:0/20793156 shutdown_connections 2024-05-26T22:51:00.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:00.279+0000 7fb1be600640 1 --2- 172.21.15.131:0/20793156 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb1b8155840 0x7fb1b80b05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:00.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:00.279+0000 7fb1be600640 1 --2- 172.21.15.131:0/20793156 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb19806be30 0x7fb19806e2f0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:00.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:00.279+0000 7fb1be600640 1 --2- 172.21.15.131:0/20793156 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1b815c3d0 0x7fb1b814a690 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:00.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:00.279+0000 7fb1be600640 1 --2- 172.21.15.131:0/20793156 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb1b806f660 0x7fb1b81584a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:00.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:00.279+0000 7fb1be600640 1 -- 172.21.15.131:0/20793156 >> 172.21.15.131:0/20793156 conn(0x7fb1b80b1e90 msgr2=0x7fb1b80ae680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:00.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:00.279+0000 7fb1be600640 1 -- 172.21.15.131:0/20793156 shutdown_connections 2024-05-26T22:51:00.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:00.279+0000 7fb1be600640 1 -- 172.21.15.131:0/20793156 wait complete. 2024-05-26T22:51:00.699 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 36, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-26T22:50:27.001665+0000', 'modified': '2024-05-26T22:50:59.832658+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 45, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 5637, 'mds_1': 4661, 'mds_2': 4670}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4661': {'gid': 4661, 'name': 'a', 'rank': 1, 'incarnation': 36, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.131:6837/265040431', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6835', 'nonce': 265040431}, {'type': 'v1', 'addr': '172.21.15.131:6837', 'nonce': 265040431}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4670': {'gid': 4670, 'name': 'b', 'rank': 2, 'incarnation': 32, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.187:6834/1027783169', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.187:6832', 'nonce': 1027783169}, {'type': 'v1', 'addr': '172.21.15.187:6834', 'nonce': 1027783169}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_5637': {'gid': 5637, 'name': 'c', 'rank': 0, 'incarnation': 25, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.131:6836/650770835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6834', 'nonce': 650770835}, {'type': 'v1', 'addr': '172.21.15.131:6836', 'nonce': 650770835}]}, 'join_fscid': -1, 'export_targets': [1, 2], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5637, 'qdb_cluster': [5637, 4670]} 2024-05-26T22:51:00.699 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_4661:up:replay 2024-05-26T22:51:01.700 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-26T22:51:01.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:01.896+0000 7f3c25600640 1 Processor -- start 2024-05-26T22:51:01.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:01.896+0000 7f3c25600640 1 -- start start 2024-05-26T22:51:01.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:01.898+0000 7f3c25600640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f3c200d1b30 0x7f3c200d1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:01.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:01.898+0000 7f3c25600640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c200d2450 con 0x7f3c20160080 2024-05-26T22:51:01.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:01.898+0000 7f3c25600640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c200cf940 con 0x7f3c20141300 2024-05-26T22:51:01.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:01.898+0000 7f3c25600640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f3c2013f970 con 0x7f3c200d1b30 2024-05-26T22:51:01.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:01.898+0000 7f3c1e000640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f3c20160080 0x7f3c20160530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:44828/0 (socket says 172.21.15.131:44828) 2024-05-26T22:51:01.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:01.898+0000 7f3c1e000640 1 -- 172.21.15.131:0/63746679 learned_addr learned my addr 172.21.15.131:0/63746679 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:51:01.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:01.899+0000 7f3c1ea00640 1 --2- 172.21.15.131:0/63746679 >> v2:172.21.15.187:3300/0 conn(0x7f3c200d1b30 0x7f3c200d1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:01.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:01.899+0000 7f3c1d600640 1 -- 172.21.15.131:0/63746679 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 353159262 0 0) 0x7f3c200d2450 con 0x7f3c20160080 2024-05-26T22:51:01.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:01.899+0000 7f3c1d600640 1 -- 172.21.15.131:0/63746679 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3c04003520 con 0x7f3c20160080 2024-05-26T22:51:01.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:01.899+0000 7f3c1d600640 1 -- 172.21.15.131:0/63746679 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2265195529 0 0) 0x7f3c04003520 con 0x7f3c20160080 2024-05-26T22:51:01.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:01.899+0000 7f3c1d600640 1 -- 172.21.15.131:0/63746679 >> v2:172.21.15.187:3300/0 conn(0x7f3c200d1b30 msgr2=0x7f3c200d1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:01.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:01.899+0000 7f3c1d600640 1 --2- 172.21.15.131:0/63746679 >> v2:172.21.15.187:3300/0 conn(0x7f3c200d1b30 0x7f3c200d1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:01.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:01.899+0000 7f3c1d600640 1 -- 172.21.15.131:0/63746679 >> v1:172.21.15.187:6789/0 conn(0x7f3c20141300 legacy=0x7f3c20141740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:01.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:01.899+0000 7f3c1d600640 1 -- 172.21.15.131:0/63746679 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c2015c730 con 0x7f3c20160080 2024-05-26T22:51:01.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:01.899+0000 7f3c1d600640 1 -- 172.21.15.131:0/63746679 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7f3c0c001670 con 0x7f3c20160080 2024-05-26T22:51:01.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:01.899+0000 7f3c1d600640 1 -- 172.21.15.131:0/63746679 >> v1:172.21.15.131:6789/0 conn(0x7f3c20160080 legacy=0x7f3c20160530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:01.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:01.899+0000 7f3c1d600640 1 --2- 172.21.15.131:0/63746679 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3c040039b0 0x7f3c04003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:01.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:01.899+0000 7f3c1d600640 1 --2- 172.21.15.131:0/63746679 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3c04004800 0x7f3c04004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:01.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:01.899+0000 7f3c1d600640 1 --2- 172.21.15.131:0/63746679 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3c040051c0 0x7f3c0400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:01.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:01.899+0000 7f3c1d600640 1 -- 172.21.15.131:0/63746679 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f3c0400c030 con 0x7f3c040039b0 2024-05-26T22:51:01.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:01.899+0000 7f3c1d600640 1 -- 172.21.15.131:0/63746679 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3c0400c1e0 con 0x7f3c040051c0 2024-05-26T22:51:01.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:01.899+0000 7f3c1d600640 1 -- 172.21.15.131:0/63746679 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3c0400c3e0 con 0x7f3c04004800 2024-05-26T22:51:01.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:01.900+0000 7f3c1ea00640 1 --2- 172.21.15.131:0/63746679 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3c040039b0 0x7f3c04003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:01.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:01.900+0000 7f3c1ea00640 1 -- 172.21.15.131:0/63746679 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3c04004800 msgr2=0x7f3c04004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:01.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:01.900+0000 7f3c1ea00640 1 --2- 172.21.15.131:0/63746679 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3c04004800 0x7f3c04004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:01.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:01.900+0000 7f3c1ea00640 1 -- 172.21.15.131:0/63746679 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3c040051c0 msgr2=0x7f3c0400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:01.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:01.900+0000 7f3c1ea00640 1 --2- 172.21.15.131:0/63746679 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3c040051c0 0x7f3c0400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:01.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:01.900+0000 7f3c1ea00640 1 -- 172.21.15.131:0/63746679 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c04003520 con 0x7f3c040039b0 2024-05-26T22:51:01.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:01.900+0000 7f3c1f400640 1 -- 172.21.15.131:0/63746679 reap_dead start 2024-05-26T22:51:01.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:01.901+0000 7f3c1ea00640 1 --2- 172.21.15.131:0/63746679 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3c040039b0 0x7f3c04003dd0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f3c10009ac0 tx=0x7f3c04005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f9c12cbe0be31e server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:01.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:01.901+0000 7f3c1d600640 1 -- 172.21.15.131:0/63746679 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c10036070 con 0x7f3c040039b0 2024-05-26T22:51:01.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:01.901+0000 7f3c1d600640 1 -- 172.21.15.131:0/63746679 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c10031400 con 0x7f3c040039b0 2024-05-26T22:51:01.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:01.901+0000 7f3c1d600640 1 -- 172.21.15.131:0/63746679 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c10009690 con 0x7f3c040039b0 2024-05-26T22:51:01.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:01.902+0000 7f3c25600640 1 -- 172.21.15.131:0/63746679 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3c040039b0 msgr2=0x7f3c04003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:01.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:01.902+0000 7f3c25600640 1 --2- 172.21.15.131:0/63746679 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3c040039b0 0x7f3c04003dd0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f3c10009ac0 tx=0x7f3c04005830 comp rx=0 tx=0).stop 2024-05-26T22:51:01.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:01.902+0000 7f3c25600640 1 -- 172.21.15.131:0/63746679 shutdown_connections 2024-05-26T22:51:01.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:01.902+0000 7f3c25600640 1 --2- 172.21.15.131:0/63746679 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3c040039b0 0x7f3c04003dd0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:01.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:01.902+0000 7f3c25600640 1 -- 172.21.15.131:0/63746679 >> 172.21.15.131:0/63746679 conn(0x7f3c200b3cb0 msgr2=0x7f3c200b4090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:01.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:01.902+0000 7f3c25600640 1 -- 172.21.15.131:0/63746679 shutdown_connections 2024-05-26T22:51:01.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:01.902+0000 7f3c25600640 1 -- 172.21.15.131:0/63746679 wait complete. 2024-05-26T22:51:01.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:01.904+0000 7f3c25600640 1 Processor -- start 2024-05-26T22:51:01.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:01.905+0000 7f3c25600640 1 -- start start 2024-05-26T22:51:01.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:01.906+0000 7f3c25600640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3c040039b0 0x7f3c20154e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:01.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:01.906+0000 7f3c25600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3c20155600 0x7f3c2013e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:01.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:01.906+0000 7f3c25600640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3c2015b9f0 0x7f3c2015be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:01.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:01.906+0000 7f3c25600640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f3c2015cfc0 con 0x7f3c20155600 2024-05-26T22:51:01.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:01.906+0000 7f3c25600640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3c2015cce0 con 0x7f3c040039b0 2024-05-26T22:51:01.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:01.906+0000 7f3c25600640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3c2015d2a0 con 0x7f3c2015b9f0 2024-05-26T22:51:01.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:01.906+0000 7f3c1e000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3c20155600 0x7f3c2013e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:01.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:01.907+0000 7f3c1e000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3c20155600 0x7f3c2013e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55228/0 (socket says 172.21.15.131:55228) 2024-05-26T22:51:01.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:01.907+0000 7f3c1e000640 1 -- 172.21.15.131:0/2510522288 learned_addr learned my addr 172.21.15.131:0/2510522288 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:01.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:01.907+0000 7f3c1e000640 1 -- 172.21.15.131:0/2510522288 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3c2015b9f0 msgr2=0x7f3c2015be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:01.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:01.907+0000 7f3c1e000640 1 --2- 172.21.15.131:0/2510522288 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3c2015b9f0 0x7f3c2015be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:01.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:01.907+0000 7f3c1e000640 1 -- 172.21.15.131:0/2510522288 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3c040039b0 msgr2=0x7f3c20154e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:01.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:01.907+0000 7f3c1e000640 1 --2- 172.21.15.131:0/2510522288 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3c040039b0 0x7f3c20154e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:01.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:01.907+0000 7f3c1e000640 1 -- 172.21.15.131:0/2510522288 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c20155dc0 con 0x7f3c20155600 2024-05-26T22:51:01.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:01.907+0000 7f3c1e000640 1 --2- 172.21.15.131:0/2510522288 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3c20155600 0x7f3c2013e940 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c00c700 tx=0x7f3c0c00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:01.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:01.907+0000 7f3c16a00640 1 -- 172.21.15.131:0/2510522288 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c0c018070 con 0x7f3c20155600 2024-05-26T22:51:01.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:01.907+0000 7f3c16a00640 1 -- 172.21.15.131:0/2510522288 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c0c0074e0 con 0x7f3c20155600 2024-05-26T22:51:01.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:01.907+0000 7f3c16a00640 1 -- 172.21.15.131:0/2510522288 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c0c004a20 con 0x7f3c20155600 2024-05-26T22:51:01.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:01.907+0000 7f3c25600640 1 -- 172.21.15.131:0/2510522288 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c201560a0 con 0x7f3c20155600 2024-05-26T22:51:01.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:01.908+0000 7f3c25600640 1 -- 172.21.15.131:0/2510522288 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c20156660 con 0x7f3c20155600 2024-05-26T22:51:01.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:01.909+0000 7f3c25600640 1 -- 172.21.15.131:0/2510522288 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c200694e0 con 0x7f3c20155600 2024-05-26T22:51:01.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:01.910+0000 7f3c16a00640 1 -- 172.21.15.131:0/2510522288 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3c0c01b070 con 0x7f3c20155600 2024-05-26T22:51:01.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:01.910+0000 7f3c16a00640 1 --2- 172.21.15.131:0/2510522288 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f3c0006be30 0x7f3c0006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:01.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:01.911+0000 7f3c16a00640 1 -- 172.21.15.131:0/2510522288 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5795+0+0 (secure 0 0 0) 0x7f3c0c08a420 con 0x7f3c20155600 2024-05-26T22:51:01.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:01.911+0000 7f3c1ea00640 1 --2- 172.21.15.131:0/2510522288 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f3c0006be30 0x7f3c0006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:01.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:01.911+0000 7f3c1ea00640 1 --2- 172.21.15.131:0/2510522288 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f3c0006be30 0x7f3c0006e2f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f3c10033000 tx=0x7f3c1002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:01.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:01.915+0000 7f3c16a00640 1 -- 172.21.15.131:0/2510522288 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f3c0c058d60 con 0x7f3c20155600 2024-05-26T22:51:02.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:02.144+0000 7f3c25600640 1 -- 172.21.15.131:0/2510522288 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3c200c8ef0 con 0x7f3c20155600 2024-05-26T22:51:02.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:02.145+0000 7f3c16a00640 1 -- 172.21.15.131:0/2510522288 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+5228 (secure 0 0 0) 0x7f3c0c058660 con 0x7f3c20155600 2024-05-26T22:51:02.145 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:51:02.145 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":38,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4727,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/3668228040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6833","nonce":3668228040},{"type":"v1","addr":"172.21.15.187:6835","nonce":3668228040}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-26T22:50:27.001665+0000","modified":"2024-05-26T22:51:00.852615+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":45,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5637,"mds_1":4661,"mds_2":4670},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4661":{"gid":4661,"name":"a","rank":1,"incarnation":36,"state":"up:reconnect","state_seq":12,"addr":"172.21.15.131:6837/265040431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6835","nonce":265040431},{"type":"v1","addr":"172.21.15.131:6837","nonce":265040431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4670":{"gid":4670,"name":"b","rank":2,"incarnation":32,"state":"up:active","state_seq":8,"addr":"172.21.15.187:6834/1027783169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1027783169},{"type":"v1","addr":"172.21.15.187:6834","nonce":1027783169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5637":{"gid":5637,"name":"c","rank":0,"incarnation":25,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6836/650770835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":650770835},{"type":"v1","addr":"172.21.15.131:6836","nonce":650770835}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5637,"qdb_cluster":[5637,4670]},"id":3}]} 2024-05-26T22:51:02.145 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 38 2024-05-26T22:51:02.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:02.148+0000 7f3c25600640 1 -- 172.21.15.131:0/2510522288 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f3c0006be30 msgr2=0x7f3c0006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:02.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:02.148+0000 7f3c25600640 1 --2- 172.21.15.131:0/2510522288 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f3c0006be30 0x7f3c0006e2f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f3c10033000 tx=0x7f3c1002e000 comp rx=0 tx=0).stop 2024-05-26T22:51:02.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:02.148+0000 7f3c25600640 1 -- 172.21.15.131:0/2510522288 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3c20155600 msgr2=0x7f3c2013e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:02.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:02.148+0000 7f3c25600640 1 --2- 172.21.15.131:0/2510522288 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3c20155600 0x7f3c2013e940 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c00c700 tx=0x7f3c0c00cbc0 comp rx=0 tx=0).stop 2024-05-26T22:51:02.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:02.148+0000 7f3c25600640 1 -- 172.21.15.131:0/2510522288 shutdown_connections 2024-05-26T22:51:02.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:02.148+0000 7f3c25600640 1 --2- 172.21.15.131:0/2510522288 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3c2015b9f0 0x7f3c2015be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:02.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:02.148+0000 7f3c25600640 1 --2- 172.21.15.131:0/2510522288 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f3c0006be30 0x7f3c0006e2f0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:02.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:02.148+0000 7f3c25600640 1 --2- 172.21.15.131:0/2510522288 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3c20155600 0x7f3c2013e940 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:02.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:02.148+0000 7f3c25600640 1 --2- 172.21.15.131:0/2510522288 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3c040039b0 0x7f3c20154e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:02.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:02.148+0000 7f3c25600640 1 -- 172.21.15.131:0/2510522288 >> 172.21.15.131:0/2510522288 conn(0x7f3c200b3cb0 msgr2=0x7f3c2006fea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:02.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:02.148+0000 7f3c25600640 1 -- 172.21.15.131:0/2510522288 shutdown_connections 2024-05-26T22:51:02.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:02.148+0000 7f3c25600640 1 -- 172.21.15.131:0/2510522288 wait complete. 2024-05-26T22:51:02.570 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 38, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-26T22:50:27.001665+0000', 'modified': '2024-05-26T22:51:00.852615+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 45, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 5637, 'mds_1': 4661, 'mds_2': 4670}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4661': {'gid': 4661, 'name': 'a', 'rank': 1, 'incarnation': 36, 'state': 'up:reconnect', 'state_seq': 12, 'addr': '172.21.15.131:6837/265040431', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6835', 'nonce': 265040431}, {'type': 'v1', 'addr': '172.21.15.131:6837', 'nonce': 265040431}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4670': {'gid': 4670, 'name': 'b', 'rank': 2, 'incarnation': 32, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.187:6834/1027783169', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.187:6832', 'nonce': 1027783169}, {'type': 'v1', 'addr': '172.21.15.187:6834', 'nonce': 1027783169}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_5637': {'gid': 5637, 'name': 'c', 'rank': 0, 'incarnation': 25, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.131:6836/650770835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6834', 'nonce': 650770835}, {'type': 'v1', 'addr': '172.21.15.131:6836', 'nonce': 650770835}]}, 'join_fscid': -1, 'export_targets': [1, 2], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5637, 'qdb_cluster': [5637, 4670]} 2024-05-26T22:51:02.570 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_4661:up:reconnect 2024-05-26T22:51:03.571 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-26T22:51:03.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:03.771+0000 7f75dd800640 1 Processor -- start 2024-05-26T22:51:03.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:03.771+0000 7f75dd800640 1 -- start start 2024-05-26T22:51:03.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:03.773+0000 7f75dd800640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f75d815e540 0x7f75d806b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:03.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:03.773+0000 7f75dd800640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f75d8162190 0x7f75d80b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:03.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:03.773+0000 7f75dd800640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75d80b4680 con 0x7f75d80b3d60 2024-05-26T22:51:03.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:03.773+0000 7f75dd800640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f75d80b41d0 con 0x7f75d815e540 2024-05-26T22:51:03.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:03.773+0000 7f75dd800640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f75d806b850 con 0x7f75d8162190 2024-05-26T22:51:03.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:03.773+0000 7f75d7e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f75d815e540 0x7f75d806b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:03.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:03.773+0000 7f75d7e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f75d815e540 0x7f75d806b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55232/0 (socket says 172.21.15.131:55232) 2024-05-26T22:51:03.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:03.773+0000 7f75d7e00640 1 -- 172.21.15.131:0/1432732564 learned_addr learned my addr 172.21.15.131:0/1432732564 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:03.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:03.773+0000 7f75d7400640 1 --2- 172.21.15.131:0/1432732564 >> v2:172.21.15.187:3300/0 conn(0x7f75d8162190 0x7f75d80b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:03.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:03.773+0000 7f75d7e00640 1 -- 172.21.15.131:0/1432732564 >> v2:172.21.15.187:3300/0 conn(0x7f75d8162190 msgr2=0x7f75d80b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:03.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:03.773+0000 7f75d7e00640 1 --2- 172.21.15.131:0/1432732564 >> v2:172.21.15.187:3300/0 conn(0x7f75d8162190 0x7f75d80b3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:03.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:03.773+0000 7f75d7e00640 1 -- 172.21.15.131:0/1432732564 >> v1:172.21.15.187:6789/0 conn(0x7f75d80b3d60 legacy=0x7f75d806df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:03.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:03.773+0000 7f75d7e00640 1 -- 172.21.15.131:0/1432732564 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75d806ba80 con 0x7f75d815e540 2024-05-26T22:51:03.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:03.773+0000 7f75d7400640 1 --2- 172.21.15.131:0/1432732564 >> v2:172.21.15.187:3300/0 conn(0x7f75d8162190 0x7f75d80b3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:51:03.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:03.774+0000 7f75d7e00640 1 --2- 172.21.15.131:0/1432732564 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75d815e540 0x7f75d806b310 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f75d806d680 tx=0x7f75c802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5729b6a90a88a7db server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:03.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:03.775+0000 7f75d6a00640 1 -- 172.21.15.131:0/1432732564 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c8037070 con 0x7f75d815e540 2024-05-26T22:51:03.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:03.775+0000 7f75d6a00640 1 -- 172.21.15.131:0/1432732564 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75c8032440 con 0x7f75d815e540 2024-05-26T22:51:03.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:03.775+0000 7f75d6a00640 1 -- 172.21.15.131:0/1432732564 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c8032810 con 0x7f75d815e540 2024-05-26T22:51:03.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:03.775+0000 7f75dd800640 1 -- 172.21.15.131:0/1432732564 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75d815e540 msgr2=0x7f75d806b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:03.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:03.775+0000 7f75dd800640 1 --2- 172.21.15.131:0/1432732564 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75d815e540 0x7f75d806b310 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f75d806d680 tx=0x7f75c802a210 comp rx=0 tx=0).stop 2024-05-26T22:51:03.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:03.776+0000 7f75dd800640 1 -- 172.21.15.131:0/1432732564 shutdown_connections 2024-05-26T22:51:03.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:03.776+0000 7f75dd800640 1 --2- 172.21.15.131:0/1432732564 >> v2:172.21.15.187:3300/0 conn(0x7f75d8162190 0x7f75d80b3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:03.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:03.776+0000 7f75dd800640 1 --2- 172.21.15.131:0/1432732564 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75d815e540 0x7f75d806b310 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:03.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:03.776+0000 7f75dd800640 1 -- 172.21.15.131:0/1432732564 >> 172.21.15.131:0/1432732564 conn(0x7f75d80d07b0 msgr2=0x7f75d80d0b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:03.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:03.776+0000 7f75dd800640 1 -- 172.21.15.131:0/1432732564 shutdown_connections 2024-05-26T22:51:03.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:03.776+0000 7f75dd800640 1 -- 172.21.15.131:0/1432732564 wait complete. 2024-05-26T22:51:03.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:03.777+0000 7f75dd800640 1 Processor -- start 2024-05-26T22:51:03.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:03.777+0000 7f75dd800640 1 -- start start 2024-05-26T22:51:03.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:03.778+0000 7f75dd800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f75d80b3d60 0x7f75d8072400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:03.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:03.778+0000 7f75dd800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75d815e540 0x7f75d81484b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:03.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:03.778+0000 7f75dd800640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f75d8162190 0x7f75d8149d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:03.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:03.778+0000 7f75dd800640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f75d806c040 con 0x7f75d815e540 2024-05-26T22:51:03.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:03.778+0000 7f75dd800640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f75d806c320 con 0x7f75d8162190 2024-05-26T22:51:03.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:03.778+0000 7f75dd800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f75d806c600 con 0x7f75d80b3d60 2024-05-26T22:51:03.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:03.779+0000 7f75d7400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75d815e540 0x7f75d81484b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:03.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:03.779+0000 7f75d7400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75d815e540 0x7f75d81484b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55236/0 (socket says 172.21.15.131:55236) 2024-05-26T22:51:03.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:03.779+0000 7f75d7400640 1 -- 172.21.15.131:0/3903409902 learned_addr learned my addr 172.21.15.131:0/3903409902 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:03.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:03.779+0000 7f75d7e00640 1 --2- 172.21.15.131:0/3903409902 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f75d80b3d60 0x7f75d8072400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:03.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:03.779+0000 7f75dce00640 1 --2- 172.21.15.131:0/3903409902 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f75d8162190 0x7f75d8149d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:03.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:03.779+0000 7f75d7400640 1 -- 172.21.15.131:0/3903409902 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f75d80b3d60 msgr2=0x7f75d8072400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:03.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:03.779+0000 7f75d7400640 1 --2- 172.21.15.131:0/3903409902 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f75d80b3d60 0x7f75d8072400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:03.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:03.779+0000 7f75d7400640 1 -- 172.21.15.131:0/3903409902 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f75d8162190 msgr2=0x7f75d8149d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:03.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:03.779+0000 7f75d7400640 1 --2- 172.21.15.131:0/3903409902 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f75d8162190 0x7f75d8149d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:03.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:03.779+0000 7f75d7400640 1 -- 172.21.15.131:0/3903409902 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75d81489f0 con 0x7f75d815e540 2024-05-26T22:51:03.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:03.779+0000 7f75d7400640 1 --2- 172.21.15.131:0/3903409902 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75d815e540 0x7f75d81484b0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f75c400c7f0 tx=0x7f75c400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:03.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:03.780+0000 7f75cfe00640 1 -- 172.21.15.131:0/3903409902 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c4015070 con 0x7f75d815e540 2024-05-26T22:51:03.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:03.780+0000 7f75cfe00640 1 -- 172.21.15.131:0/3903409902 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75c4007820 con 0x7f75d815e540 2024-05-26T22:51:03.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:03.780+0000 7f75dd800640 1 -- 172.21.15.131:0/3903409902 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75d8148c70 con 0x7f75d815e540 2024-05-26T22:51:03.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:03.780+0000 7f75cfe00640 1 -- 172.21.15.131:0/3903409902 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c4007bd0 con 0x7f75d815e540 2024-05-26T22:51:03.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:03.780+0000 7f75dd800640 1 -- 172.21.15.131:0/3903409902 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75d8149230 con 0x7f75d815e540 2024-05-26T22:51:03.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:03.781+0000 7f75dd800640 1 -- 172.21.15.131:0/3903409902 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75d8068f60 con 0x7f75d815e540 2024-05-26T22:51:03.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:03.782+0000 7f75cfe00640 1 -- 172.21.15.131:0/3903409902 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f75c4018070 con 0x7f75d815e540 2024-05-26T22:51:03.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:03.783+0000 7f75cfe00640 1 --2- 172.21.15.131:0/3903409902 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f75b806be30 0x7f75b806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:03.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:03.783+0000 7f75cfe00640 1 -- 172.21.15.131:0/3903409902 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5795+0+0 (secure 0 0 0) 0x7f75c408afc0 con 0x7f75d815e540 2024-05-26T22:51:03.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:03.784+0000 7f75d7e00640 1 --2- 172.21.15.131:0/3903409902 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f75b806be30 0x7f75b806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:03.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:03.784+0000 7f75d7e00640 1 --2- 172.21.15.131:0/3903409902 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f75b806be30 0x7f75b806e2f0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f75c8034000 tx=0x7f75c802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:03.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:03.788+0000 7f75cfe00640 1 -- 172.21.15.131:0/3903409902 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f75d8068f60 con 0x7f75d815e540 2024-05-26T22:51:04.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.028+0000 7f75dd800640 1 -- 172.21.15.131:0/3903409902 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f75d80b48a0 con 0x7f75d815e540 2024-05-26T22:51:04.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.029+0000 7f75cfe00640 1 -- 172.21.15.131:0/3903409902 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+5230 (secure 0 0 0) 0x7f75d80b48a0 con 0x7f75d815e540 2024-05-26T22:51:04.028 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:51:04.028 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":40,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4727,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/3668228040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6833","nonce":3668228040},{"type":"v1","addr":"172.21.15.187:6835","nonce":3668228040}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-26T22:50:27.001665+0000","modified":"2024-05-26T22:51:03.848463+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":45,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5637,"mds_1":4661,"mds_2":4670},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4661":{"gid":4661,"name":"a","rank":1,"incarnation":36,"state":"up:active","state_seq":14,"addr":"172.21.15.131:6837/265040431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6835","nonce":265040431},{"type":"v1","addr":"172.21.15.131:6837","nonce":265040431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4670":{"gid":4670,"name":"b","rank":2,"incarnation":32,"state":"up:active","state_seq":8,"addr":"172.21.15.187:6834/1027783169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1027783169},{"type":"v1","addr":"172.21.15.187:6834","nonce":1027783169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5637":{"gid":5637,"name":"c","rank":0,"incarnation":25,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6836/650770835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":650770835},{"type":"v1","addr":"172.21.15.131:6836","nonce":650770835}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5637,"qdb_cluster":[4661,4670,5637]},"id":3}]} 2024-05-26T22:51:04.029 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 40 2024-05-26T22:51:04.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.031+0000 7f75dd800640 1 -- 172.21.15.131:0/3903409902 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f75b806be30 msgr2=0x7f75b806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:04.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.031+0000 7f75dd800640 1 --2- 172.21.15.131:0/3903409902 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f75b806be30 0x7f75b806e2f0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f75c8034000 tx=0x7f75c802e000 comp rx=0 tx=0).stop 2024-05-26T22:51:04.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.031+0000 7f75dd800640 1 -- 172.21.15.131:0/3903409902 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75d815e540 msgr2=0x7f75d81484b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:04.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.031+0000 7f75dd800640 1 --2- 172.21.15.131:0/3903409902 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75d815e540 0x7f75d81484b0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f75c400c7f0 tx=0x7f75c400ccb0 comp rx=0 tx=0).stop 2024-05-26T22:51:04.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.031+0000 7f75dd800640 1 -- 172.21.15.131:0/3903409902 shutdown_connections 2024-05-26T22:51:04.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.031+0000 7f75dd800640 1 --2- 172.21.15.131:0/3903409902 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f75b806be30 0x7f75b806e2f0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:04.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.031+0000 7f75dd800640 1 --2- 172.21.15.131:0/3903409902 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f75d8162190 0x7f75d8149d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:04.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.031+0000 7f75dd800640 1 --2- 172.21.15.131:0/3903409902 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75d815e540 0x7f75d81484b0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:04.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.031+0000 7f75dd800640 1 --2- 172.21.15.131:0/3903409902 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f75d80b3d60 0x7f75d8072400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:04.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.031+0000 7f75dd800640 1 -- 172.21.15.131:0/3903409902 >> 172.21.15.131:0/3903409902 conn(0x7f75d80d07b0 msgr2=0x7f75d815cda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:04.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.031+0000 7f75dd800640 1 -- 172.21.15.131:0/3903409902 shutdown_connections 2024-05-26T22:51:04.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.032+0000 7f75dd800640 1 -- 172.21.15.131:0/3903409902 wait complete. 2024-05-26T22:51:04.456 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 40, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-26T22:50:27.001665+0000', 'modified': '2024-05-26T22:51:03.848463+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 45, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 5637, 'mds_1': 4661, 'mds_2': 4670}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4661': {'gid': 4661, 'name': 'a', 'rank': 1, 'incarnation': 36, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.131:6837/265040431', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6835', 'nonce': 265040431}, {'type': 'v1', 'addr': '172.21.15.131:6837', 'nonce': 265040431}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4670': {'gid': 4670, 'name': 'b', 'rank': 2, 'incarnation': 32, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.187:6834/1027783169', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.187:6832', 'nonce': 1027783169}, {'type': 'v1', 'addr': '172.21.15.187:6834', 'nonce': 1027783169}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_5637': {'gid': 5637, 'name': 'c', 'rank': 0, 'incarnation': 25, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.131:6836/650770835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6834', 'nonce': 650770835}, {'type': 'v1', 'addr': '172.21.15.131:6836', 'nonce': 650770835}]}, 'join_fscid': -1, 'export_targets': [1, 2], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5637, 'qdb_cluster': [4661, 4670, 5637]} 2024-05-26T22:51:04.456 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-05-26T22:51:04.456 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-05-26T22:51:04.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.650+0000 7f6109000640 1 Processor -- start 2024-05-26T22:51:04.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.650+0000 7f6109000640 1 -- start start 2024-05-26T22:51:04.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.652+0000 7f6109000640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f61040b8d40 0x7f61040b9170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:04.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.652+0000 7f6109000640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61040b79f0 con 0x7f61040b9f20 2024-05-26T22:51:04.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.652+0000 7f6109000640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61040b96b0 con 0x7f610415e340 2024-05-26T22:51:04.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.652+0000 7f6109000640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f61040b88b0 con 0x7f61040b8d40 2024-05-26T22:51:04.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.652+0000 7f60faa00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f61040b9f20 0x7f61040d22b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:52968/0 (socket says 172.21.15.131:52968) 2024-05-26T22:51:04.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.652+0000 7f60faa00640 1 -- 172.21.15.131:0/4099331348 learned_addr learned my addr 172.21.15.131:0/4099331348 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:51:04.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.652+0000 7f6103e00640 1 --2- 172.21.15.131:0/4099331348 >> v2:172.21.15.131:3300/0 conn(0x7f61040b8d40 0x7f61040b9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:04.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.653+0000 7f6102a00640 1 -- 172.21.15.131:0/4099331348 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3141436206 0 0) 0x7f61040b79f0 con 0x7f61040b9f20 2024-05-26T22:51:04.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.653+0000 7f6102a00640 1 -- 172.21.15.131:0/4099331348 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f60e8003520 con 0x7f61040b9f20 2024-05-26T22:51:04.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.653+0000 7f6103e00640 1 -- 172.21.15.131:0/4099331348 >> v1:172.21.15.187:6789/0 conn(0x7f610415e340 legacy=0x7f61040b72e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:04.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.653+0000 7f6103e00640 1 -- 172.21.15.131:0/4099331348 >> v1:172.21.15.131:6789/0 conn(0x7f61040b9f20 legacy=0x7f61040d22b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:04.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.653+0000 7f6103e00640 1 -- 172.21.15.131:0/4099331348 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f610415e720 con 0x7f61040b8d40 2024-05-26T22:51:04.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.654+0000 7f6103e00640 1 --2- 172.21.15.131:0/4099331348 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61040b8d40 0x7f61040b9170 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f60fc009d90 tx=0x7f60fc02a780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c601be7c20fddf70 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:04.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.654+0000 7f6102a00640 1 -- 172.21.15.131:0/4099331348 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60fc037070 con 0x7f61040b8d40 2024-05-26T22:51:04.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.654+0000 7f6102a00640 1 -- 172.21.15.131:0/4099331348 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60fc007740 con 0x7f61040b8d40 2024-05-26T22:51:04.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.654+0000 7f6102a00640 1 -- 172.21.15.131:0/4099331348 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60fc007b10 con 0x7f61040b8d40 2024-05-26T22:51:04.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.655+0000 7f6109000640 1 -- 172.21.15.131:0/4099331348 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61040b8d40 msgr2=0x7f61040b9170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:04.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.655+0000 7f6109000640 1 --2- 172.21.15.131:0/4099331348 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61040b8d40 0x7f61040b9170 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f60fc009d90 tx=0x7f60fc02a780 comp rx=0 tx=0).stop 2024-05-26T22:51:04.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.655+0000 7f6109000640 1 -- 172.21.15.131:0/4099331348 shutdown_connections 2024-05-26T22:51:04.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.655+0000 7f6109000640 1 --2- 172.21.15.131:0/4099331348 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61040b8d40 0x7f61040b9170 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:04.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.655+0000 7f6109000640 1 -- 172.21.15.131:0/4099331348 >> 172.21.15.131:0/4099331348 conn(0x7f6104072e60 msgr2=0x7f6104073270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:04.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.655+0000 7f6109000640 1 -- 172.21.15.131:0/4099331348 shutdown_connections 2024-05-26T22:51:04.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.655+0000 7f6109000640 1 -- 172.21.15.131:0/4099331348 wait complete. 2024-05-26T22:51:04.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.657+0000 7f6109000640 1 Processor -- start 2024-05-26T22:51:04.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.657+0000 7f6109000640 1 -- start start 2024-05-26T22:51:04.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.658+0000 7f6109000640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f61040b8d40 0x7f6104151e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:04.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.658+0000 7f6109000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f61040b9f20 0x7f6104154370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:04.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.658+0000 7f6109000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f610415e340 0x7f6104154d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:04.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.658+0000 7f6109000640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f61040b8260 con 0x7f610415e340 2024-05-26T22:51:04.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.658+0000 7f6109000640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f61040b7f80 con 0x7f61040b8d40 2024-05-26T22:51:04.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.658+0000 7f6109000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f61040b8540 con 0x7f61040b9f20 2024-05-26T22:51:04.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.658+0000 7f6103e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f610415e340 0x7f6104154d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:04.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.658+0000 7f6103e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f610415e340 0x7f6104154d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46672/0 (socket says 172.21.15.131:46672) 2024-05-26T22:51:04.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.658+0000 7f6103e00640 1 -- 172.21.15.131:0/3603652238 learned_addr learned my addr 172.21.15.131:0/3603652238 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:04.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.658+0000 7f6103400640 1 --2- 172.21.15.131:0/3603652238 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f61040b8d40 0x7f6104151e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:04.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.658+0000 7f6103e00640 1 -- 172.21.15.131:0/3603652238 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f61040b9f20 msgr2=0x7f6104154370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:04.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.658+0000 7f60faa00640 1 --2- 172.21.15.131:0/3603652238 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f61040b9f20 0x7f6104154370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:04.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.658+0000 7f6103e00640 1 --2- 172.21.15.131:0/3603652238 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f61040b9f20 0x7f6104154370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:04.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.658+0000 7f6103e00640 1 -- 172.21.15.131:0/3603652238 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f61040b8d40 msgr2=0x7f6104151e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:04.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.658+0000 7f6103e00640 1 --2- 172.21.15.131:0/3603652238 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f61040b8d40 0x7f6104151e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:04.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.659+0000 7f6103e00640 1 -- 172.21.15.131:0/3603652238 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6104153e40 con 0x7f610415e340 2024-05-26T22:51:04.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.659+0000 7f60faa00640 1 --2- 172.21.15.131:0/3603652238 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f61040b9f20 0x7f6104154370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:51:04.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.659+0000 7f6103400640 1 --2- 172.21.15.131:0/3603652238 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f61040b8d40 0x7f6104151e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:51:04.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.659+0000 7f6103e00640 1 --2- 172.21.15.131:0/3603652238 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f610415e340 0x7f6104154d00 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f60fc02ad70 tx=0x7f60fc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:04.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.659+0000 7f60fbe00640 1 -- 172.21.15.131:0/3603652238 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60fc042020 con 0x7f610415e340 2024-05-26T22:51:04.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.659+0000 7f60fbe00640 1 -- 172.21.15.131:0/3603652238 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60fc0327f0 con 0x7f610415e340 2024-05-26T22:51:04.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.659+0000 7f60fbe00640 1 -- 172.21.15.131:0/3603652238 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60fc037040 con 0x7f610415e340 2024-05-26T22:51:04.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.659+0000 7f6109000640 1 -- 172.21.15.131:0/3603652238 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6104152830 con 0x7f610415e340 2024-05-26T22:51:04.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.659+0000 7f6109000640 1 -- 172.21.15.131:0/3603652238 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6104152df0 con 0x7f610415e340 2024-05-26T22:51:04.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.662+0000 7f60fbe00640 1 -- 172.21.15.131:0/3603652238 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f60fc045020 con 0x7f610415e340 2024-05-26T22:51:04.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.663+0000 7f60fbe00640 1 --2- 172.21.15.131:0/3603652238 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f60e006be30 0x7f60e006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:04.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.663+0000 7f610d5b8740 1 Processor -- start 2024-05-26T22:51:04.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.663+0000 7f60fbe00640 1 -- 172.21.15.131:0/3603652238 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5853+0+0 (secure 0 0 0) 0x7f60fc0ac2a0 con 0x7f610415e340 2024-05-26T22:51:04.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.663+0000 7f610d5b8740 1 -- start start 2024-05-26T22:51:04.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.663+0000 7f6103400640 1 --2- 172.21.15.131:0/3603652238 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f60e006be30 0x7f60e006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:04.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.664+0000 7f6103400640 1 --2- 172.21.15.131:0/3603652238 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f60e006be30 0x7f60e006e2f0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f60ec000f60 tx=0x7f60ec009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:04.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.664+0000 7f610d5b8740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560a0f527e50 0x560a0f4daa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:04.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.664+0000 7f610d5b8740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560a0f50af70 0x560a0f50a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:04.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.664+0000 7f610d5b8740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560a0f50a830 0x560a0f4df640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:04.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.664+0000 7f610d5b8740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x560a0f442610 con 0x560a0f50a830 2024-05-26T22:51:04.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.664+0000 7f610d5b8740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x560a0f3ea5c0 con 0x560a0f50af70 2024-05-26T22:51:04.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.664+0000 7f610d5b8740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x560a0f43e5d0 con 0x560a0f527e50 2024-05-26T22:51:04.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.664+0000 7f6103e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560a0f50a830 0x560a0f4df640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:04.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.664+0000 7f6103e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560a0f50a830 0x560a0f4df640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46684/0 (socket says 172.21.15.131:46684) 2024-05-26T22:51:04.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.664+0000 7f6103e00640 1 -- 172.21.15.131:0/3997904860 learned_addr learned my addr 172.21.15.131:0/3997904860 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:04.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.665+0000 7f6103e00640 1 -- 172.21.15.131:0/3997904860 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560a0f527e50 msgr2=0x560a0f4daa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:04.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.665+0000 7f6103e00640 1 --2- 172.21.15.131:0/3997904860 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560a0f527e50 0x560a0f4daa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:04.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.665+0000 7f6103e00640 1 -- 172.21.15.131:0/3997904860 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560a0f50af70 msgr2=0x560a0f50a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:04.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.665+0000 7f6103e00640 1 --2- 172.21.15.131:0/3997904860 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560a0f50af70 0x560a0f50a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:04.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.665+0000 7f6103e00640 1 -- 172.21.15.131:0/3997904860 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a0f1562d0 con 0x560a0f50a830 2024-05-26T22:51:04.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.665+0000 7f6103e00640 1 --2- 172.21.15.131:0/3997904860 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560a0f50a830 0x560a0f4df640 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f60fc07a2d0 tx=0x7f60fc076910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=762b53dc5471d4b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:04.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.665+0000 7f60f3e00640 1 -- 172.21.15.131:0/3997904860 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60fc079f30 con 0x560a0f50a830 2024-05-26T22:51:04.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.665+0000 7f60f3e00640 1 -- 172.21.15.131:0/3997904860 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60fc07acd0 con 0x560a0f50a830 2024-05-26T22:51:04.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.665+0000 7f60f3e00640 1 -- 172.21.15.131:0/3997904860 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60fc07a5d0 con 0x560a0f50a830 2024-05-26T22:51:04.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.666+0000 7f610d5b8740 1 -- 172.21.15.131:0/3997904860 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560a0f50a830 msgr2=0x560a0f4df640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:04.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.666+0000 7f610d5b8740 1 --2- 172.21.15.131:0/3997904860 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560a0f50a830 0x560a0f4df640 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f60fc07a2d0 tx=0x7f60fc076910 comp rx=0 tx=0).stop 2024-05-26T22:51:04.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.666+0000 7f610d5b8740 1 -- 172.21.15.131:0/3997904860 shutdown_connections 2024-05-26T22:51:04.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.666+0000 7f610d5b8740 1 --2- 172.21.15.131:0/3997904860 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560a0f50a830 0x560a0f4df640 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:04.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.666+0000 7f610d5b8740 1 --2- 172.21.15.131:0/3997904860 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560a0f50af70 0x560a0f50a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:04.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.666+0000 7f610d5b8740 1 --2- 172.21.15.131:0/3997904860 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560a0f527e50 0x560a0f4daa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:04.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.666+0000 7f610d5b8740 1 -- 172.21.15.131:0/3997904860 >> 172.21.15.131:0/3997904860 conn(0x560a0f4dcce0 msgr2=0x560a0f5252f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:04.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.666+0000 7f610d5b8740 1 -- 172.21.15.131:0/3997904860 shutdown_connections 2024-05-26T22:51:04.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.666+0000 7f610d5b8740 1 -- 172.21.15.131:0/3997904860 wait complete. 2024-05-26T22:51:04.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.667+0000 7f610d5b8740 1 -- start start 2024-05-26T22:51:04.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.667+0000 7f610d5b8740 1 Processor -- start 2024-05-26T22:51:04.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.669+0000 7f610d5b8740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560a0f50af70 0x560a0f50d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:04.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.669+0000 7f610d5b8740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560a0f527e50 0x560a0f526a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:04.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.669+0000 7f610d5b8740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560a0f526f70 0x560a0f4f84b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:04.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.669+0000 7f610d5b8740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x560a0f4dec00 con 0x560a0f50af70 2024-05-26T22:51:04.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.669+0000 7f610d5b8740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x560a0f3fd910 con 0x560a0f527e50 2024-05-26T22:51:04.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.669+0000 7f610d5b8740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x560a0f4ddff0 con 0x560a0f526f70 2024-05-26T22:51:04.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.669+0000 7f610d5b8740 10 client.0 _ll_register_callbacks cb 0x560a0f439410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-26T22:51:04.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.669+0000 7f6103400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560a0f50af70 0x560a0f50d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:04.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.669+0000 7f6103400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560a0f50af70 0x560a0f50d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46686/0 (socket says 172.21.15.131:46686) 2024-05-26T22:51:04.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.669+0000 7f6103400640 1 -- 172.21.15.131:0/1242476582 learned_addr learned my addr 172.21.15.131:0/1242476582 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:04.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.670+0000 7f6103400640 1 -- 172.21.15.131:0/1242476582 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560a0f526f70 msgr2=0x560a0f4f84b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:04.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.670+0000 7f6103400640 1 --2- 172.21.15.131:0/1242476582 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560a0f526f70 0x560a0f4f84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:04.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.670+0000 7f6103400640 1 -- 172.21.15.131:0/1242476582 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560a0f527e50 msgr2=0x560a0f526a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:04.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.670+0000 7f6103400640 1 --2- 172.21.15.131:0/1242476582 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560a0f527e50 0x560a0f526a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:04.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.670+0000 7f6103400640 1 -- 172.21.15.131:0/1242476582 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a0f2206b0 con 0x560a0f50af70 2024-05-26T22:51:04.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.670+0000 7f6103400640 1 --2- 172.21.15.131:0/1242476582 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560a0f50af70 0x560a0f50d160 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f60ec00dd80 tx=0x7f60ec0147f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4475295b3e08d21f server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:04.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.670+0000 7f60f0c00640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-26T22:51:04.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.670+0000 7f60f0c00640 1 -- 172.21.15.131:0/1242476582 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60ec024060 con 0x560a0f50af70 2024-05-26T22:51:04.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.670+0000 7f60f0c00640 1 -- 172.21.15.131:0/1242476582 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(46..46 src has 1..46) ==== 5853+0+0 (secure 0 0 0) 0x7f60ec01f600 con 0x560a0f50af70 2024-05-26T22:51:04.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.670+0000 7f610d5b8740 1 -- 172.21.15.131:0/1242476582 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60fc07acd0 con 0x560a0f50af70 2024-05-26T22:51:04.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.670+0000 7f60f0c00640 1 -- 172.21.15.131:0/1242476582 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f60ec02a050 con 0x560a0f50af70 2024-05-26T22:51:04.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.671+0000 7f610d5b8740 10 client.6036 fetch_fsmap learned FSMap version 40 2024-05-26T22:51:04.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.671+0000 7f610d5b8740 1 -- 172.21.15.131:0/1242476582 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=2+}) -- 0x560a0f51ccd0 con 0x560a0f50af70 2024-05-26T22:51:04.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.671+0000 7f60f0c00640 1 -- 172.21.15.131:0/1242476582 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60fc07acd0 con 0x560a0f50af70 2024-05-26T22:51:04.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.671+0000 7f60f0c00640 1 -- 172.21.15.131:0/1242476582 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 40) ==== 3542+0+0 (secure 0 0 0) 0x7f60ec02d020 con 0x560a0f50af70 2024-05-26T22:51:04.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.671+0000 7f610d5b8740 10 client.6036 fetch_fsmap finished waiting for FSMap version 40 2024-05-26T22:51:04.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.671+0000 7f610d5b8740 10 client.6036 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-26T22:51:04.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.671+0000 7f610d5b8740 20 client.6036 populate_metadata read hostname 'smithi131' 2024-05-26T22:51:04.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.671+0000 7f610d5b8740 1 --2- 172.21.15.131:0/1242476582 >> [v2:172.21.15.131:6835/265040431,v1:172.21.15.131:6837/265040431] conn(0x560a0f50b460 0x560a0f543c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:04.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.671+0000 7f610d5b8740 4 client.6036 mds_command: new command op to 4661 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-26T22:51:04.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.671+0000 7f610d5b8740 1 -- 172.21.15.131:0/1242476582 --> [v2:172.21.15.131:6835/265040431,v1:172.21.15.131:6837/265040431] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560a0f44fd40 con 0x560a0f50b460 2024-05-26T22:51:04.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.671+0000 7f60faa00640 1 --2- 172.21.15.131:0/1242476582 >> [v2:172.21.15.131:6835/265040431,v1:172.21.15.131:6837/265040431] conn(0x560a0f50b460 0x560a0f543c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:04.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.672+0000 7f60faa00640 1 --2- 172.21.15.131:0/1242476582 >> [v2:172.21.15.131:6835/265040431,v1:172.21.15.131:6837/265040431] conn(0x560a0f50b460 0x560a0f543c60 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5faaf650237a9fb server_cookie=57a5026845ef8c35 in_seq=0 out_seq=0 2024-05-26T22:51:04.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.672+0000 7f60f0c00640 10 client.6036 ms_handle_connect on v2:172.21.15.131:6835/265040431 2024-05-26T22:51:04.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.673+0000 7f60f0c00640 1 -- 172.21.15.131:0/1242476582 <== mds.1 v2:172.21.15.131:6835/265040431 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560a0f44fd40 con 0x560a0f50b460 2024-05-26T22:51:04.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.673+0000 7f60f0c00640 10 client.6036 handle_command_reply: tid=0 2024-05-26T22:51:04.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.673+0000 7f610d5b8740 1 client.6036 shutdown 2024-05-26T22:51:04.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.673+0000 7f610d5b8740 1 -- 172.21.15.131:0/1242476582 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560a0f50af70 msgr2=0x560a0f50d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:04.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.673+0000 7f610d5b8740 1 --2- 172.21.15.131:0/1242476582 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560a0f50af70 0x560a0f50d160 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f60ec00dd80 tx=0x7f60ec0147f0 comp rx=0 tx=0).stop 2024-05-26T22:51:04.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.673+0000 7f610d5b8740 1 -- 172.21.15.131:0/1242476582 shutdown_connections 2024-05-26T22:51:04.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.673+0000 7f610d5b8740 1 --2- 172.21.15.131:0/1242476582 >> [v2:172.21.15.131:6835/265040431,v1:172.21.15.131:6837/265040431] conn(0x560a0f50b460 0x560a0f543c60 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:04.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.673+0000 7f610d5b8740 1 --2- 172.21.15.131:0/1242476582 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560a0f526f70 0x560a0f4f84b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:04.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.673+0000 7f610d5b8740 1 --2- 172.21.15.131:0/1242476582 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560a0f527e50 0x560a0f526a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:04.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.673+0000 7f610d5b8740 1 --2- 172.21.15.131:0/1242476582 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560a0f50af70 0x560a0f50d160 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:04.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.673+0000 7f60f0c00640 0 client.6036 ms_handle_reset on v2:172.21.15.131:6835/265040431 2024-05-26T22:51:04.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.673+0000 7f610d5b8740 1 -- 172.21.15.131:0/1242476582 >> 172.21.15.131:0/1242476582 conn(0x560a0f50a830 msgr2=0x560a0f4db4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:04.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.673+0000 7f610d5b8740 1 -- 172.21.15.131:0/1242476582 shutdown_connections 2024-05-26T22:51:04.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.673+0000 7f610d5b8740 1 -- 172.21.15.131:0/1242476582 wait complete. 2024-05-26T22:51:04.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.674+0000 7f610d5b8740 20 client.6036 trim_cache size 0 max 16384 2024-05-26T22:51:04.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.694+0000 7f610d5b8740 1 Processor -- start 2024-05-26T22:51:04.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.694+0000 7f610d5b8740 1 -- start start 2024-05-26T22:51:04.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.695+0000 7f610d5b8740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560a0f50af70 0x560a0f4e0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:04.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.695+0000 7f610d5b8740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560a0f526f70 0x560a0f4db4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:04.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.695+0000 7f610d5b8740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560a0f527e50 0x560a0f4dba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:04.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.695+0000 7f610d5b8740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f61041ffd20 con 0x560a0f527e50 2024-05-26T22:51:04.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.695+0000 7f610d5b8740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f61042003f0 con 0x560a0f50af70 2024-05-26T22:51:04.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.695+0000 7f610d5b8740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f61041fee10 con 0x560a0f526f70 2024-05-26T22:51:04.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.695+0000 7f6103e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560a0f527e50 0x560a0f4dba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:04.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.695+0000 7f6103e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560a0f527e50 0x560a0f4dba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46688/0 (socket says 172.21.15.131:46688) 2024-05-26T22:51:04.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.696+0000 7f6103e00640 1 -- 172.21.15.131:0/1086283992 learned_addr learned my addr 172.21.15.131:0/1086283992 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:04.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.696+0000 7f60faa00640 1 --2- 172.21.15.131:0/1086283992 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560a0f526f70 0x560a0f4db4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:04.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.696+0000 7f6103400640 1 --2- 172.21.15.131:0/1086283992 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560a0f50af70 0x560a0f4e0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:04.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.696+0000 7f6103e00640 1 -- 172.21.15.131:0/1086283992 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560a0f526f70 msgr2=0x560a0f4db4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:04.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.696+0000 7f6103e00640 1 --2- 172.21.15.131:0/1086283992 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560a0f526f70 0x560a0f4db4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:04.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.696+0000 7f6103e00640 1 -- 172.21.15.131:0/1086283992 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560a0f50af70 msgr2=0x560a0f4e0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:04.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.696+0000 7f6103e00640 1 --2- 172.21.15.131:0/1086283992 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560a0f50af70 0x560a0f4e0480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:04.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.696+0000 7f6103e00640 1 -- 172.21.15.131:0/1086283992 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a0f4db240 con 0x560a0f527e50 2024-05-26T22:51:04.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.696+0000 7f6103e00640 1 --2- 172.21.15.131:0/1086283992 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560a0f527e50 0x560a0f4dba10 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f60fc07a2d0 tx=0x7f60fc03f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c5f3a920e805504 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:04.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.696+0000 7f60f3400640 1 -- 172.21.15.131:0/1086283992 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60fc07acd0 con 0x560a0f527e50 2024-05-26T22:51:04.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.696+0000 7f60f3400640 1 -- 172.21.15.131:0/1086283992 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60fc07e980 con 0x560a0f527e50 2024-05-26T22:51:04.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.696+0000 7f60f3400640 1 -- 172.21.15.131:0/1086283992 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60fc0831b0 con 0x560a0f527e50 2024-05-26T22:51:04.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.697+0000 7f610d5b8740 1 -- 172.21.15.131:0/1086283992 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560a0f527e50 msgr2=0x560a0f4dba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:04.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.697+0000 7f610d5b8740 1 --2- 172.21.15.131:0/1086283992 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560a0f527e50 0x560a0f4dba10 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f60fc07a2d0 tx=0x7f60fc03f000 comp rx=0 tx=0).stop 2024-05-26T22:51:04.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.697+0000 7f610d5b8740 1 -- 172.21.15.131:0/1086283992 shutdown_connections 2024-05-26T22:51:04.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.697+0000 7f610d5b8740 1 --2- 172.21.15.131:0/1086283992 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560a0f526f70 0x560a0f4db4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:04.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.697+0000 7f610d5b8740 1 --2- 172.21.15.131:0/1086283992 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560a0f527e50 0x560a0f4dba10 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:04.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.697+0000 7f610d5b8740 1 --2- 172.21.15.131:0/1086283992 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560a0f50af70 0x560a0f4e0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:04.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.697+0000 7f610d5b8740 1 -- 172.21.15.131:0/1086283992 >> 172.21.15.131:0/1086283992 conn(0x560a0f50a830 msgr2=0x560a0f4de230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:04.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.697+0000 7f610d5b8740 1 -- 172.21.15.131:0/1086283992 shutdown_connections 2024-05-26T22:51:04.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.697+0000 7f610d5b8740 1 -- 172.21.15.131:0/1086283992 wait complete. 2024-05-26T22:51:04.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.698+0000 7f610d5b8740 1 -- start start 2024-05-26T22:51:04.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.698+0000 7f610d5b8740 1 Processor -- start 2024-05-26T22:51:04.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.699+0000 7f610d5b8740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560a0f50af70 0x560a0f4dfb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:04.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.699+0000 7f610d5b8740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560a0f526f70 0x560a0f5a5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:04.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.699+0000 7f610d5b8740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560a0f527e50 0x560a0f5067b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:04.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.699+0000 7f610d5b8740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x560a0f4f8fe0 con 0x560a0f527e50 2024-05-26T22:51:04.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.699+0000 7f610d5b8740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x560a0f51d2e0 con 0x560a0f526f70 2024-05-26T22:51:04.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.699+0000 7f610d5b8740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x560a0f4f92f0 con 0x560a0f50af70 2024-05-26T22:51:04.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.699+0000 7f610d5b8740 10 client.0 _ll_register_callbacks cb 0x7f60ec0013e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-26T22:51:04.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.699+0000 7f6103e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560a0f527e50 0x560a0f5067b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:04.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.699+0000 7f6103e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560a0f527e50 0x560a0f5067b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46702/0 (socket says 172.21.15.131:46702) 2024-05-26T22:51:04.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.699+0000 7f6103e00640 1 -- 172.21.15.131:0/508599212 learned_addr learned my addr 172.21.15.131:0/508599212 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:04.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.699+0000 7f6103400640 1 --2- 172.21.15.131:0/508599212 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560a0f50af70 0x560a0f4dfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:04.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.699+0000 7f6103e00640 1 -- 172.21.15.131:0/508599212 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560a0f50af70 msgr2=0x560a0f4dfb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:04.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.700+0000 7f6103e00640 1 --2- 172.21.15.131:0/508599212 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560a0f50af70 0x560a0f4dfb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:04.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.700+0000 7f6103e00640 1 -- 172.21.15.131:0/508599212 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560a0f526f70 msgr2=0x560a0f5a5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:04.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.700+0000 7f60faa00640 1 --2- 172.21.15.131:0/508599212 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560a0f526f70 0x560a0f5a5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:04.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.700+0000 7f6103e00640 1 --2- 172.21.15.131:0/508599212 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560a0f526f70 0x560a0f5a5760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:04.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.700+0000 7f6103e00640 1 -- 172.21.15.131:0/508599212 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a0f4e0050 con 0x560a0f527e50 2024-05-26T22:51:04.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.700+0000 7f6103e00640 1 --2- 172.21.15.131:0/508599212 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560a0f527e50 0x560a0f5067b0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f60fc007300 tx=0x7f60fc02f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8529aa06500e8bde server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:04.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.700+0000 7f60f0c00640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-26T22:51:04.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.700+0000 7f60f0c00640 1 -- 172.21.15.131:0/508599212 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60fc07acd0 con 0x560a0f527e50 2024-05-26T22:51:04.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.700+0000 7f60f0c00640 1 -- 172.21.15.131:0/508599212 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(46..46 src has 1..46) ==== 5853+0+0 (secure 0 0 0) 0x7f60fc03be00 con 0x560a0f527e50 2024-05-26T22:51:04.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.700+0000 7f610d5b8740 1 -- 172.21.15.131:0/508599212 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60fc07e980 con 0x560a0f527e50 2024-05-26T22:51:04.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.700+0000 7f60f0c00640 1 -- 172.21.15.131:0/508599212 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f60fc09f050 con 0x560a0f527e50 2024-05-26T22:51:04.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.700+0000 7f610d5b8740 10 client.6042 fetch_fsmap learned FSMap version 40 2024-05-26T22:51:04.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.700+0000 7f610d5b8740 1 -- 172.21.15.131:0/508599212 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=2+}) -- 0x560a0f4e0050 con 0x560a0f527e50 2024-05-26T22:51:04.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.701+0000 7f60f0c00640 1 -- 172.21.15.131:0/508599212 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60fc07e980 con 0x560a0f527e50 2024-05-26T22:51:04.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.701+0000 7f60f0c00640 1 -- 172.21.15.131:0/508599212 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 40) ==== 3542+0+0 (secure 0 0 0) 0x7f60fc0a2020 con 0x560a0f527e50 2024-05-26T22:51:04.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.701+0000 7f610d5b8740 10 client.6042 fetch_fsmap finished waiting for FSMap version 40 2024-05-26T22:51:04.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.701+0000 7f610d5b8740 10 client.6042 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-26T22:51:04.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.701+0000 7f610d5b8740 20 client.6042 populate_metadata read hostname 'smithi131' 2024-05-26T22:51:04.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.701+0000 7f610d5b8740 1 --2- 172.21.15.131:0/508599212 >> [v2:172.21.15.131:6835/265040431,v1:172.21.15.131:6837/265040431] conn(0x560a0f5a48f0 0x560a0f506cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:04.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.701+0000 7f610d5b8740 4 client.6042 mds_command: new command op to 4661 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-26T22:51:04.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.701+0000 7f610d5b8740 1 -- 172.21.15.131:0/508599212 --> [v2:172.21.15.131:6835/265040431,v1:172.21.15.131:6837/265040431] -- command(tid 0: {"prefix": "status"}) -- 0x560a0f44a090 con 0x560a0f5a48f0 2024-05-26T22:51:04.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.701+0000 7f6103400640 1 --2- 172.21.15.131:0/508599212 >> [v2:172.21.15.131:6835/265040431,v1:172.21.15.131:6837/265040431] conn(0x560a0f5a48f0 0x560a0f506cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:04.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.702+0000 7f60f0c00640 10 client.6042 ms_handle_connect on v2:172.21.15.131:6835/265040431 2024-05-26T22:51:04.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.702+0000 7f6103400640 1 --2- 172.21.15.131:0/508599212 >> [v2:172.21.15.131:6835/265040431,v1:172.21.15.131:6837/265040431] conn(0x560a0f5a48f0 0x560a0f506cf0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e44fa6c6c1618534 server_cookie=94d9a055e3f6bf16 in_seq=0 out_seq=0 2024-05-26T22:51:04.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.702+0000 7f60f0c00640 1 -- 172.21.15.131:0/508599212 <== mds.1 v2:172.21.15.131:6835/265040431 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x560a0f44a090 con 0x560a0f5a48f0 2024-05-26T22:51:04.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.702+0000 7f60f0c00640 10 client.6042 handle_command_reply: tid=0 2024-05-26T22:51:04.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.702+0000 7f610d5b8740 1 client.6042 shutdown 2024-05-26T22:51:04.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.702+0000 7f610d5b8740 1 -- 172.21.15.131:0/508599212 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560a0f527e50 msgr2=0x560a0f5067b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:04.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.702+0000 7f610d5b8740 1 --2- 172.21.15.131:0/508599212 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560a0f527e50 0x560a0f5067b0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f60fc007300 tx=0x7f60fc02f000 comp rx=0 tx=0).stop 2024-05-26T22:51:04.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.702+0000 7f610d5b8740 1 -- 172.21.15.131:0/508599212 shutdown_connections 2024-05-26T22:51:04.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.702+0000 7f610d5b8740 1 --2- 172.21.15.131:0/508599212 >> [v2:172.21.15.131:6835/265040431,v1:172.21.15.131:6837/265040431] conn(0x560a0f5a48f0 0x560a0f506cf0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:04.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.702+0000 7f610d5b8740 1 --2- 172.21.15.131:0/508599212 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560a0f527e50 0x560a0f5067b0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:04.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.702+0000 7f610d5b8740 1 --2- 172.21.15.131:0/508599212 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560a0f526f70 0x560a0f5a5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:04.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.702+0000 7f610d5b8740 1 --2- 172.21.15.131:0/508599212 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560a0f50af70 0x560a0f4dfb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:04.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.702+0000 7f60f0c00640 0 client.6042 ms_handle_reset on v2:172.21.15.131:6835/265040431 2024-05-26T22:51:04.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.702+0000 7f610d5b8740 1 -- 172.21.15.131:0/508599212 >> 172.21.15.131:0/508599212 conn(0x560a0f50a830 msgr2=0x560a0f4db4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:04.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.702+0000 7f610d5b8740 1 -- 172.21.15.131:0/508599212 shutdown_connections 2024-05-26T22:51:04.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.702+0000 7f610d5b8740 1 -- 172.21.15.131:0/508599212 wait complete. 2024-05-26T22:51:04.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.703+0000 7f610d5b8740 20 client.6042 trim_cache size 0 max 16384 2024-05-26T22:51:04.706 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-26T22:51:04.707 INFO:teuthology.orchestra.run.smithi131.stdout: "cluster_fsid": "46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f", 2024-05-26T22:51:04.707 INFO:teuthology.orchestra.run.smithi131.stdout: "whoami": 1, 2024-05-26T22:51:04.707 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 4661, 2024-05-26T22:51:04.707 INFO:teuthology.orchestra.run.smithi131.stdout: "want_state": "up:active", 2024-05-26T22:51:04.707 INFO:teuthology.orchestra.run.smithi131.stdout: "state": "up:active", 2024-05-26T22:51:04.707 INFO:teuthology.orchestra.run.smithi131.stdout: "fs_name": "cephfs", 2024-05-26T22:51:04.707 INFO:teuthology.orchestra.run.smithi131.stdout: "rank_uptime": 4.8616135409999996, 2024-05-26T22:51:04.707 INFO:teuthology.orchestra.run.smithi131.stdout: "mdsmap_epoch": 40, 2024-05-26T22:51:04.707 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch": 46, 2024-05-26T22:51:04.707 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch_barrier": 45, 2024-05-26T22:51:04.707 INFO:teuthology.orchestra.run.smithi131.stdout: "uptime": 42.607019899999997 2024-05-26T22:51:04.707 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-26T22:51:04.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.703+0000 7f6109000640 1 -- 172.21.15.131:0/3603652238 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f60e006be30 msgr2=0x7f60e006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:04.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.703+0000 7f6109000640 1 --2- 172.21.15.131:0/3603652238 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f60e006be30 0x7f60e006e2f0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f60ec000f60 tx=0x7f60ec009040 comp rx=0 tx=0).stop 2024-05-26T22:51:04.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.703+0000 7f6109000640 1 -- 172.21.15.131:0/3603652238 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f610415e340 msgr2=0x7f6104154d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:04.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.703+0000 7f6109000640 1 --2- 172.21.15.131:0/3603652238 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f610415e340 0x7f6104154d00 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f60fc02ad70 tx=0x7f60fc004a40 comp rx=0 tx=0).stop 2024-05-26T22:51:04.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.703+0000 7f6109000640 1 -- 172.21.15.131:0/3603652238 shutdown_connections 2024-05-26T22:51:04.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.703+0000 7f6109000640 1 --2- 172.21.15.131:0/3603652238 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f61040b9f20 0x7f6104154370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:04.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.703+0000 7f6109000640 1 --2- 172.21.15.131:0/3603652238 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f60e006be30 0x7f60e006e2f0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:04.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.703+0000 7f6109000640 1 --2- 172.21.15.131:0/3603652238 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f610415e340 0x7f6104154d00 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:04.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.703+0000 7f6109000640 1 --2- 172.21.15.131:0/3603652238 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f61040b8d40 0x7f6104151e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:04.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.703+0000 7f6109000640 1 -- 172.21.15.131:0/3603652238 >> 172.21.15.131:0/3603652238 conn(0x7f6104072e60 msgr2=0x7f61040b61f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:04.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.703+0000 7f6109000640 1 -- 172.21.15.131:0/3603652238 shutdown_connections 2024-05-26T22:51:04.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.703+0000 7f6109000640 1 -- 172.21.15.131:0/3603652238 wait complete. 2024-05-26T22:51:04.719 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-05-26T22:51:04.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.901+0000 7fee6f800640 1 Processor -- start 2024-05-26T22:51:04.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.901+0000 7fee6f800640 1 -- start start 2024-05-26T22:51:04.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.903+0000 7fee6f800640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fee680cbef0 0x7fee680cc2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:04.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.903+0000 7fee6f800640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fee680acef0 0x7fee6806d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:04.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.903+0000 7fee6f800640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee680cdca0 con 0x7fee680ad3a0 2024-05-26T22:51:04.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.903+0000 7fee6f800640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fee680cdf80 con 0x7fee680cbef0 2024-05-26T22:51:04.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.903+0000 7fee6f800640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fee680ce260 con 0x7fee680acef0 2024-05-26T22:51:04.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.903+0000 7fee6d800640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fee680ad3a0 0x7fee680cea60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:52982/0 (socket says 172.21.15.131:52982) 2024-05-26T22:51:04.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.903+0000 7fee6ce00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fee680cbef0 0x7fee680cc2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:04.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.903+0000 7fee6d800640 1 -- 172.21.15.131:0/2655634274 learned_addr learned my addr 172.21.15.131:0/2655634274 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:51:04.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.903+0000 7fee5fe00640 1 --2- 172.21.15.131:0/2655634274 >> v2:172.21.15.187:3300/0 conn(0x7fee680acef0 0x7fee6806d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:04.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.904+0000 7fee6ce00640 1 -- 172.21.15.131:0/2655634274 >> v2:172.21.15.187:3300/0 conn(0x7fee680acef0 msgr2=0x7fee6806d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:04.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.904+0000 7fee6ce00640 1 --2- 172.21.15.131:0/2655634274 >> v2:172.21.15.187:3300/0 conn(0x7fee680acef0 0x7fee6806d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:04.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.904+0000 7fee6ce00640 1 -- 172.21.15.131:0/2655634274 >> v1:172.21.15.131:6789/0 conn(0x7fee680ad3a0 legacy=0x7fee680cea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:04.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.904+0000 7fee6ce00640 1 -- 172.21.15.131:0/2655634274 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee680ce540 con 0x7fee680cbef0 2024-05-26T22:51:04.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.904+0000 7fee67e00640 1 -- 172.21.15.131:0/2655634274 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2866363174 0 0) 0x7fee680cdca0 con 0x7fee680ad3a0 2024-05-26T22:51:04.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.905+0000 7fee6ce00640 1 --2- 172.21.15.131:0/2655634274 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee680cbef0 0x7fee680cc2d0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fee680737f0 tx=0x7fee5802b560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c760c51126d406ea server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:04.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.905+0000 7fee67e00640 1 -- 172.21.15.131:0/2655634274 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee5800b040 con 0x7fee680cbef0 2024-05-26T22:51:04.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.905+0000 7fee67e00640 1 -- 172.21.15.131:0/2655634274 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee58030710 con 0x7fee680cbef0 2024-05-26T22:51:04.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.905+0000 7fee67e00640 1 -- 172.21.15.131:0/2655634274 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee58030ae0 con 0x7fee680cbef0 2024-05-26T22:51:04.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.905+0000 7fee6f800640 1 -- 172.21.15.131:0/2655634274 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee680cbef0 msgr2=0x7fee680cc2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:04.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.905+0000 7fee6f800640 1 --2- 172.21.15.131:0/2655634274 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee680cbef0 0x7fee680cc2d0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fee680737f0 tx=0x7fee5802b560 comp rx=0 tx=0).stop 2024-05-26T22:51:04.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.906+0000 7fee6f800640 1 -- 172.21.15.131:0/2655634274 shutdown_connections 2024-05-26T22:51:04.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.906+0000 7fee6f800640 1 --2- 172.21.15.131:0/2655634274 >> v2:172.21.15.187:3300/0 conn(0x7fee680acef0 0x7fee6806d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:04.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.906+0000 7fee6f800640 1 --2- 172.21.15.131:0/2655634274 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee680cbef0 0x7fee680cc2d0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:04.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.906+0000 7fee6f800640 1 -- 172.21.15.131:0/2655634274 >> 172.21.15.131:0/2655634274 conn(0x7fee6806bc90 msgr2=0x7fee680b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:04.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.906+0000 7fee6f800640 1 -- 172.21.15.131:0/2655634274 shutdown_connections 2024-05-26T22:51:04.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.906+0000 7fee6f800640 1 -- 172.21.15.131:0/2655634274 wait complete. 2024-05-26T22:51:04.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.907+0000 7fee6f800640 1 Processor -- start 2024-05-26T22:51:04.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.907+0000 7fee6f800640 1 -- start start 2024-05-26T22:51:04.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.908+0000 7fee6f800640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fee680acef0 0x7fee681f4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:04.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.908+0000 7fee6f800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee680ad3a0 0x7fee680b3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:04.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.908+0000 7fee6f800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fee680cbef0 0x7fee6809c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:04.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.908+0000 7fee6f800640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fee680cf500 con 0x7fee680ad3a0 2024-05-26T22:51:04.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.908+0000 7fee6f800640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fee680cf380 con 0x7fee680acef0 2024-05-26T22:51:04.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.908+0000 7fee6f800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fee6806d860 con 0x7fee680cbef0 2024-05-26T22:51:04.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.909+0000 7fee5fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee680ad3a0 0x7fee680b3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:04.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.909+0000 7fee5fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee680ad3a0 0x7fee680b3850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46712/0 (socket says 172.21.15.131:46712) 2024-05-26T22:51:04.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.909+0000 7fee5fe00640 1 -- 172.21.15.131:0/2037330411 learned_addr learned my addr 172.21.15.131:0/2037330411 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:04.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.909+0000 7fee6ce00640 1 --2- 172.21.15.131:0/2037330411 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fee680acef0 0x7fee681f4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:04.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.909+0000 7fee6d800640 1 --2- 172.21.15.131:0/2037330411 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fee680cbef0 0x7fee6809c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:04.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.909+0000 7fee5fe00640 1 -- 172.21.15.131:0/2037330411 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fee680cbef0 msgr2=0x7fee6809c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:04.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.909+0000 7fee5fe00640 1 --2- 172.21.15.131:0/2037330411 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fee680cbef0 0x7fee6809c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:04.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.909+0000 7fee5fe00640 1 -- 172.21.15.131:0/2037330411 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fee680acef0 msgr2=0x7fee681f4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:04.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.909+0000 7fee5fe00640 1 --2- 172.21.15.131:0/2037330411 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fee680acef0 0x7fee681f4850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:04.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.909+0000 7fee5fe00640 1 -- 172.21.15.131:0/2037330411 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee680b3d90 con 0x7fee680ad3a0 2024-05-26T22:51:04.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.909+0000 7fee6d800640 1 --2- 172.21.15.131:0/2037330411 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fee680cbef0 0x7fee6809c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:51:04.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.909+0000 7fee6ce00640 1 --2- 172.21.15.131:0/2037330411 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fee680acef0 0x7fee681f4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:51:04.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.910+0000 7fee5fe00640 1 --2- 172.21.15.131:0/2037330411 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee680ad3a0 0x7fee680b3850 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fee5000d5f0 tx=0x7fee5000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:04.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.910+0000 7fee65600640 1 -- 172.21.15.131:0/2037330411 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee500125f0 con 0x7fee680ad3a0 2024-05-26T22:51:04.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.910+0000 7fee65600640 1 -- 172.21.15.131:0/2037330411 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee50019070 con 0x7fee680ad3a0 2024-05-26T22:51:04.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.910+0000 7fee6f800640 1 -- 172.21.15.131:0/2037330411 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee680b4060 con 0x7fee680ad3a0 2024-05-26T22:51:04.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.910+0000 7fee65600640 1 -- 172.21.15.131:0/2037330411 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee500129d0 con 0x7fee680ad3a0 2024-05-26T22:51:04.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.910+0000 7fee6f800640 1 -- 172.21.15.131:0/2037330411 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee680b4620 con 0x7fee680ad3a0 2024-05-26T22:51:04.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.912+0000 7fee65600640 1 -- 172.21.15.131:0/2037330411 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fee50012b70 con 0x7fee680ad3a0 2024-05-26T22:51:04.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.914+0000 7fee65600640 1 --2- 172.21.15.131:0/2037330411 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fee4406be60 0x7fee4406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:04.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.914+0000 7fee65600640 1 -- 172.21.15.131:0/2037330411 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5853+0+0 (secure 0 0 0) 0x7fee5008b230 con 0x7fee680ad3a0 2024-05-26T22:51:04.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.914+0000 7fee6ce00640 1 --2- 172.21.15.131:0/2037330411 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fee4406be60 0x7fee4406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:04.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.914+0000 7fee727bc740 1 Processor -- start 2024-05-26T22:51:04.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.914+0000 7fee727bc740 1 -- start start 2024-05-26T22:51:04.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.915+0000 7fee6ce00640 1 --2- 172.21.15.131:0/2037330411 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fee4406be60 0x7fee4406e320 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fee58035000 tx=0x7fee5800c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:04.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.915+0000 7fee727bc740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5592ee662c70 0x5592ee615890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:04.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.915+0000 7fee727bc740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5592ee645d90 0x5592ee645110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:04.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.915+0000 7fee727bc740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5592ee645650 0x5592ee61a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:04.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.915+0000 7fee727bc740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5592ee57d610 con 0x5592ee645650 2024-05-26T22:51:04.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.915+0000 7fee727bc740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5592ee5255c0 con 0x5592ee662c70 2024-05-26T22:51:04.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.915+0000 7fee727bc740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x5592ee5795d0 con 0x5592ee645d90 2024-05-26T22:51:04.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.915+0000 7fee6d800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5592ee645650 0x5592ee61a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:04.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.916+0000 7fee6d800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5592ee645650 0x5592ee61a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46726/0 (socket says 172.21.15.131:46726) 2024-05-26T22:51:04.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.916+0000 7fee6d800640 1 -- 172.21.15.131:0/3151441375 learned_addr learned my addr 172.21.15.131:0/3151441375 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:04.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.916+0000 7fee6d800640 1 -- 172.21.15.131:0/3151441375 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5592ee645d90 msgr2=0x5592ee645110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:04.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.916+0000 7fee6d800640 1 --2- 172.21.15.131:0/3151441375 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5592ee645d90 0x5592ee645110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:04.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.916+0000 7fee6ce00640 1 --2- 172.21.15.131:0/3151441375 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5592ee662c70 0x5592ee615890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:04.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.916+0000 7fee6d800640 1 -- 172.21.15.131:0/3151441375 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5592ee662c70 msgr2=0x5592ee615890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:04.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.916+0000 7fee6d800640 1 --2- 172.21.15.131:0/3151441375 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5592ee662c70 0x5592ee615890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:04.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.916+0000 7fee6d800640 1 -- 172.21.15.131:0/3151441375 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592ee2912d0 con 0x5592ee645650 2024-05-26T22:51:04.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.916+0000 7fee6d800640 1 --2- 172.21.15.131:0/3151441375 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5592ee645650 0x5592ee61a460 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fee60001ef0 tx=0x7fee6000d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb217c739a60d735 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:04.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.916+0000 7fee5d600640 1 -- 172.21.15.131:0/3151441375 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee60004b60 con 0x5592ee645650 2024-05-26T22:51:04.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.917+0000 7fee5d600640 1 -- 172.21.15.131:0/3151441375 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee60004d00 con 0x5592ee645650 2024-05-26T22:51:04.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.917+0000 7fee5d600640 1 -- 172.21.15.131:0/3151441375 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee600072d0 con 0x5592ee645650 2024-05-26T22:51:04.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.917+0000 7fee727bc740 1 -- 172.21.15.131:0/3151441375 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5592ee645650 msgr2=0x5592ee61a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:04.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.917+0000 7fee727bc740 1 --2- 172.21.15.131:0/3151441375 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5592ee645650 0x5592ee61a460 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fee60001ef0 tx=0x7fee6000d520 comp rx=0 tx=0).stop 2024-05-26T22:51:04.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.917+0000 7fee727bc740 1 -- 172.21.15.131:0/3151441375 shutdown_connections 2024-05-26T22:51:04.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.917+0000 7fee727bc740 1 --2- 172.21.15.131:0/3151441375 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5592ee645d90 0x5592ee645110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:04.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.917+0000 7fee727bc740 1 --2- 172.21.15.131:0/3151441375 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5592ee645650 0x5592ee61a460 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:04.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.917+0000 7fee727bc740 1 --2- 172.21.15.131:0/3151441375 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5592ee662c70 0x5592ee615890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:04.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.917+0000 7fee727bc740 1 -- 172.21.15.131:0/3151441375 >> 172.21.15.131:0/3151441375 conn(0x5592ee617b00 msgr2=0x5592ee660110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:04.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.917+0000 7fee727bc740 1 -- 172.21.15.131:0/3151441375 shutdown_connections 2024-05-26T22:51:04.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.917+0000 7fee727bc740 1 -- 172.21.15.131:0/3151441375 wait complete. 2024-05-26T22:51:04.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.918+0000 7fee727bc740 1 -- start start 2024-05-26T22:51:04.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.918+0000 7fee727bc740 1 Processor -- start 2024-05-26T22:51:04.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.919+0000 7fee727bc740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5592ee645d90 0x5592ee635800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:04.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.919+0000 7fee727bc740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5592ee662c70 0x5592ee646920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:04.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.919+0000 7fee727bc740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5592ee635f40 0x5592ee646e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:04.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.919+0000 7fee727bc740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5592ee619a20 con 0x5592ee662c70 2024-05-26T22:51:04.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.919+0000 7fee727bc740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5592ee538910 con 0x5592ee635f40 2024-05-26T22:51:04.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.919+0000 7fee727bc740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x5592ee618e10 con 0x5592ee645d90 2024-05-26T22:51:04.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.919+0000 7fee727bc740 10 client.0 _ll_register_callbacks cb 0x5592ee574410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-26T22:51:04.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.919+0000 7fee5fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5592ee662c70 0x5592ee646920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:04.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.919+0000 7fee5fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5592ee662c70 0x5592ee646920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46734/0 (socket says 172.21.15.131:46734) 2024-05-26T22:51:04.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.920+0000 7fee5fe00640 1 -- 172.21.15.131:0/2628466281 learned_addr learned my addr 172.21.15.131:0/2628466281 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:04.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.920+0000 7fee5fe00640 1 -- 172.21.15.131:0/2628466281 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5592ee645d90 msgr2=0x5592ee635800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:04.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.920+0000 7fee5fe00640 1 --2- 172.21.15.131:0/2628466281 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5592ee645d90 0x5592ee635800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:04.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.920+0000 7fee5fe00640 1 -- 172.21.15.131:0/2628466281 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5592ee635f40 msgr2=0x5592ee646e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:04.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.920+0000 7fee5fe00640 1 --2- 172.21.15.131:0/2628466281 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5592ee635f40 0x5592ee646e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:04.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.920+0000 7fee5fe00640 1 -- 172.21.15.131:0/2628466281 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592ee35b6b0 con 0x5592ee662c70 2024-05-26T22:51:04.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.920+0000 7fee5fe00640 1 --2- 172.21.15.131:0/2628466281 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5592ee662c70 0x5592ee646920 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fee50061b90 tx=0x7fee500595f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2115301167ee941 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:04.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.920+0000 7fee56000640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-26T22:51:04.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.920+0000 7fee56000640 1 -- 172.21.15.131:0/2628466281 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee5008b070 con 0x5592ee662c70 2024-05-26T22:51:04.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.920+0000 7fee56000640 1 -- 172.21.15.131:0/2628466281 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(46..46 src has 1..46) ==== 5853+0+0 (secure 0 0 0) 0x7fee50055b50 con 0x5592ee662c70 2024-05-26T22:51:04.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.920+0000 7fee727bc740 1 -- 172.21.15.131:0/2628466281 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee60004d00 con 0x5592ee662c70 2024-05-26T22:51:04.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.921+0000 7fee56000640 1 -- 172.21.15.131:0/2628466281 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fee50061c30 con 0x5592ee662c70 2024-05-26T22:51:04.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.921+0000 7fee727bc740 10 client.6057 fetch_fsmap learned FSMap version 40 2024-05-26T22:51:04.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.921+0000 7fee727bc740 1 -- 172.21.15.131:0/2628466281 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=2+}) -- 0x5592ee6448b0 con 0x5592ee662c70 2024-05-26T22:51:04.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.921+0000 7fee56000640 1 -- 172.21.15.131:0/2628466281 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee60004d00 con 0x5592ee662c70 2024-05-26T22:51:04.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.921+0000 7fee56000640 1 -- 172.21.15.131:0/2628466281 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 40) ==== 3542+0+0 (secure 0 0 0) 0x7fee50013070 con 0x5592ee662c70 2024-05-26T22:51:04.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.921+0000 7fee727bc740 10 client.6057 fetch_fsmap finished waiting for FSMap version 40 2024-05-26T22:51:04.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.921+0000 7fee727bc740 10 client.6057 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-26T22:51:04.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.921+0000 7fee727bc740 20 client.6057 populate_metadata read hostname 'smithi131' 2024-05-26T22:51:04.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.921+0000 7fee727bc740 1 --2- 172.21.15.131:0/2628466281 >> [v2:172.21.15.187:6832/1027783169,v1:172.21.15.187:6834/1027783169] conn(0x5592ee6740d0 0x5592ee643a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:04.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.921+0000 7fee727bc740 4 client.6057 mds_command: new command op to 4670 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-26T22:51:04.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.921+0000 7fee727bc740 1 -- 172.21.15.131:0/2628466281 --> [v2:172.21.15.187:6832/1027783169,v1:172.21.15.187:6834/1027783169] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5592ee58ad40 con 0x5592ee6740d0 2024-05-26T22:51:04.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.922+0000 7fee6ce00640 1 --2- 172.21.15.131:0/2628466281 >> [v2:172.21.15.187:6832/1027783169,v1:172.21.15.187:6834/1027783169] conn(0x5592ee6740d0 0x5592ee643a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:04.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.923+0000 7fee6ce00640 1 --2- 172.21.15.131:0/2628466281 >> [v2:172.21.15.187:6832/1027783169,v1:172.21.15.187:6834/1027783169] conn(0x5592ee6740d0 0x5592ee643a40 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7ed28509e2f05d39 server_cookie=c7b8d4d450595ea2 in_seq=0 out_seq=0 2024-05-26T22:51:04.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.923+0000 7fee56000640 10 client.6057 ms_handle_connect on v2:172.21.15.187:6832/1027783169 2024-05-26T22:51:04.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.925+0000 7fee56000640 1 -- 172.21.15.131:0/2628466281 <== mds.2 v2:172.21.15.187:6832/1027783169 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5592ee58ad40 con 0x5592ee6740d0 2024-05-26T22:51:04.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.925+0000 7fee56000640 10 client.6057 handle_command_reply: tid=0 2024-05-26T22:51:04.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.925+0000 7fee727bc740 1 client.6057 shutdown 2024-05-26T22:51:04.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.925+0000 7fee727bc740 1 -- 172.21.15.131:0/2628466281 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5592ee662c70 msgr2=0x5592ee646920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:04.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.925+0000 7fee727bc740 1 --2- 172.21.15.131:0/2628466281 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5592ee662c70 0x5592ee646920 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fee50061b90 tx=0x7fee500595f0 comp rx=0 tx=0).stop 2024-05-26T22:51:04.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.925+0000 7fee727bc740 1 -- 172.21.15.131:0/2628466281 shutdown_connections 2024-05-26T22:51:04.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.925+0000 7fee727bc740 1 --2- 172.21.15.131:0/2628466281 >> [v2:172.21.15.187:6832/1027783169,v1:172.21.15.187:6834/1027783169] conn(0x5592ee6740d0 0x5592ee643a40 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:04.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.925+0000 7fee727bc740 1 --2- 172.21.15.131:0/2628466281 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5592ee635f40 0x5592ee646e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:04.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.925+0000 7fee727bc740 1 --2- 172.21.15.131:0/2628466281 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5592ee662c70 0x5592ee646920 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:04.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.925+0000 7fee727bc740 1 --2- 172.21.15.131:0/2628466281 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5592ee645d90 0x5592ee635800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:04.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.925+0000 7fee56000640 0 client.6057 ms_handle_reset on v2:172.21.15.187:6832/1027783169 2024-05-26T22:51:04.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.925+0000 7fee727bc740 1 -- 172.21.15.131:0/2628466281 >> 172.21.15.131:0/2628466281 conn(0x5592ee645650 msgr2=0x5592ee6611e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:04.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.925+0000 7fee727bc740 1 -- 172.21.15.131:0/2628466281 shutdown_connections 2024-05-26T22:51:04.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.925+0000 7fee727bc740 1 -- 172.21.15.131:0/2628466281 wait complete. 2024-05-26T22:51:04.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.925+0000 7fee727bc740 20 client.6057 trim_cache size 0 max 16384 2024-05-26T22:51:04.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.945+0000 7fee727bc740 1 Processor -- start 2024-05-26T22:51:04.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.945+0000 7fee727bc740 1 -- start start 2024-05-26T22:51:04.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.946+0000 7fee727bc740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5592ee635f40 0x5592ee6403f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:04.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.946+0000 7fee727bc740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5592ee645d90 0x5592ee640f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:04.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.946+0000 7fee727bc740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5592ee662c70 0x5592ee684250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:04.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.946+0000 7fee727bc740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fee681ffb70 con 0x5592ee662c70 2024-05-26T22:51:04.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.946+0000 7fee727bc740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fee68200240 con 0x5592ee645d90 2024-05-26T22:51:04.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.946+0000 7fee727bc740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fee68153f50 con 0x5592ee635f40 2024-05-26T22:51:04.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.947+0000 7fee6d800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5592ee662c70 0x5592ee684250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:04.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.947+0000 7fee6d800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5592ee662c70 0x5592ee684250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46742/0 (socket says 172.21.15.131:46742) 2024-05-26T22:51:04.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.947+0000 7fee6d800640 1 -- 172.21.15.131:0/3912988366 learned_addr learned my addr 172.21.15.131:0/3912988366 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:04.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.947+0000 7fee5fe00640 1 --2- 172.21.15.131:0/3912988366 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5592ee645d90 0x5592ee640f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:04.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.947+0000 7fee6ce00640 1 --2- 172.21.15.131:0/3912988366 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5592ee635f40 0x5592ee6403f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:04.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.947+0000 7fee6d800640 1 -- 172.21.15.131:0/3912988366 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5592ee635f40 msgr2=0x5592ee6403f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:04.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.947+0000 7fee6d800640 1 --2- 172.21.15.131:0/3912988366 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5592ee635f40 0x5592ee6403f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:04.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.947+0000 7fee6d800640 1 -- 172.21.15.131:0/3912988366 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5592ee645d90 msgr2=0x5592ee640f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:04.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.947+0000 7fee6d800640 1 --2- 172.21.15.131:0/3912988366 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5592ee645d90 0x5592ee640f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:04.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.947+0000 7fee6d800640 1 -- 172.21.15.131:0/3912988366 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592ee6189b0 con 0x5592ee662c70 2024-05-26T22:51:04.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.947+0000 7fee5fe00640 1 --2- 172.21.15.131:0/3912988366 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5592ee645d90 0x5592ee640f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:51:04.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.947+0000 7fee6d800640 1 --2- 172.21.15.131:0/3912988366 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5592ee662c70 0x5592ee684250 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fee6000d560 tx=0x7fee60010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=107a33384b890a63 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:04.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.948+0000 7fee5cc00640 1 -- 172.21.15.131:0/3912988366 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee60004b60 con 0x5592ee662c70 2024-05-26T22:51:04.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.948+0000 7fee5cc00640 1 -- 172.21.15.131:0/3912988366 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee60025070 con 0x5592ee662c70 2024-05-26T22:51:04.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.948+0000 7fee5cc00640 1 -- 172.21.15.131:0/3912988366 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee60020480 con 0x5592ee662c70 2024-05-26T22:51:04.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.948+0000 7fee727bc740 1 -- 172.21.15.131:0/3912988366 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5592ee662c70 msgr2=0x5592ee684250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:04.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.948+0000 7fee727bc740 1 --2- 172.21.15.131:0/3912988366 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5592ee662c70 0x5592ee684250 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fee6000d560 tx=0x7fee60010000 comp rx=0 tx=0).stop 2024-05-26T22:51:04.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.948+0000 7fee727bc740 1 -- 172.21.15.131:0/3912988366 shutdown_connections 2024-05-26T22:51:04.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.948+0000 7fee727bc740 1 --2- 172.21.15.131:0/3912988366 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5592ee662c70 0x5592ee684250 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:04.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.948+0000 7fee727bc740 1 --2- 172.21.15.131:0/3912988366 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5592ee645d90 0x5592ee640f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:04.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.948+0000 7fee727bc740 1 --2- 172.21.15.131:0/3912988366 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5592ee635f40 0x5592ee6403f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:04.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.948+0000 7fee727bc740 1 -- 172.21.15.131:0/3912988366 >> 172.21.15.131:0/3912988366 conn(0x5592ee645650 msgr2=0x5592ee635610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:04.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.948+0000 7fee727bc740 1 -- 172.21.15.131:0/3912988366 shutdown_connections 2024-05-26T22:51:04.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.949+0000 7fee727bc740 1 -- 172.21.15.131:0/3912988366 wait complete. 2024-05-26T22:51:04.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.949+0000 7fee727bc740 1 -- start start 2024-05-26T22:51:04.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.949+0000 7fee727bc740 1 Processor -- start 2024-05-26T22:51:04.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.950+0000 7fee727bc740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5592ee635f40 0x5592ee684250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:04.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.950+0000 7fee727bc740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5592ee645d90 0x5592ee646610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:04.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.950+0000 7fee727bc740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5592ee662c70 0x5592ee646b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:04.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.950+0000 7fee727bc740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5592ee674540 con 0x5592ee645d90 2024-05-26T22:51:04.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.950+0000 7fee727bc740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5592ee673f50 con 0x5592ee635f40 2024-05-26T22:51:04.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.950+0000 7fee727bc740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x5592ee6479a0 con 0x5592ee662c70 2024-05-26T22:51:04.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.950+0000 7fee727bc740 10 client.0 _ll_register_callbacks cb 0x7fee50055780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-26T22:51:04.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.951+0000 7fee5fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5592ee645d90 0x5592ee646610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:04.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.951+0000 7fee5fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5592ee645d90 0x5592ee646610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46758/0 (socket says 172.21.15.131:46758) 2024-05-26T22:51:04.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.951+0000 7fee5fe00640 1 -- 172.21.15.131:0/3801111858 learned_addr learned my addr 172.21.15.131:0/3801111858 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:04.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.951+0000 7fee6ce00640 1 --2- 172.21.15.131:0/3801111858 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5592ee635f40 0x5592ee684250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:04.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.951+0000 7fee5fe00640 1 -- 172.21.15.131:0/3801111858 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5592ee662c70 msgr2=0x5592ee646b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:04.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.951+0000 7fee5fe00640 1 --2- 172.21.15.131:0/3801111858 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5592ee662c70 0x5592ee646b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:04.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.951+0000 7fee5fe00640 1 -- 172.21.15.131:0/3801111858 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5592ee635f40 msgr2=0x5592ee684250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:04.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.951+0000 7fee5fe00640 1 --2- 172.21.15.131:0/3801111858 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5592ee635f40 0x5592ee684250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:04.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.951+0000 7fee5fe00640 1 -- 172.21.15.131:0/3801111858 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592ee6e2610 con 0x5592ee645d90 2024-05-26T22:51:04.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.951+0000 7fee5fe00640 1 --2- 172.21.15.131:0/3801111858 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5592ee645d90 0x5592ee646610 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fee50057790 tx=0x7fee50020070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0151d1a9402b126 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:04.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.951+0000 7fee56000640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-26T22:51:04.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.951+0000 7fee56000640 1 -- 172.21.15.131:0/3801111858 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee5008b070 con 0x5592ee645d90 2024-05-26T22:51:04.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.951+0000 7fee56000640 1 -- 172.21.15.131:0/3801111858 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(46..46 src has 1..46) ==== 5853+0+0 (secure 0 0 0) 0x7fee500577e0 con 0x5592ee645d90 2024-05-26T22:51:04.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.951+0000 7fee727bc740 1 -- 172.21.15.131:0/3801111858 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee60025070 con 0x5592ee645d90 2024-05-26T22:51:04.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.952+0000 7fee56000640 1 -- 172.21.15.131:0/3801111858 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fee5007f050 con 0x5592ee645d90 2024-05-26T22:51:04.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.952+0000 7fee727bc740 10 client.6063 fetch_fsmap learned FSMap version 40 2024-05-26T22:51:04.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.952+0000 7fee727bc740 1 -- 172.21.15.131:0/3801111858 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=2+}) -- 0x5592ee6e23b0 con 0x5592ee645d90 2024-05-26T22:51:04.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.952+0000 7fee56000640 1 -- 172.21.15.131:0/3801111858 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee60025070 con 0x5592ee645d90 2024-05-26T22:51:04.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.952+0000 7fee56000640 1 -- 172.21.15.131:0/3801111858 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 40) ==== 3542+0+0 (secure 0 0 0) 0x7fee50082020 con 0x5592ee645d90 2024-05-26T22:51:04.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.952+0000 7fee727bc740 10 client.6063 fetch_fsmap finished waiting for FSMap version 40 2024-05-26T22:51:04.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.952+0000 7fee727bc740 10 client.6063 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-26T22:51:04.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.952+0000 7fee727bc740 20 client.6063 populate_metadata read hostname 'smithi131' 2024-05-26T22:51:04.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.952+0000 7fee727bc740 1 --2- 172.21.15.131:0/3801111858 >> [v2:172.21.15.187:6832/1027783169,v1:172.21.15.187:6834/1027783169] conn(0x5592ee6ddcd0 0x5592ee6de0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:04.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.952+0000 7fee727bc740 4 client.6063 mds_command: new command op to 4670 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-26T22:51:04.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.952+0000 7fee727bc740 1 -- 172.21.15.131:0/3801111858 --> [v2:172.21.15.187:6832/1027783169,v1:172.21.15.187:6834/1027783169] -- command(tid 0: {"prefix": "status"}) -- 0x5592ee657f40 con 0x5592ee6ddcd0 2024-05-26T22:51:04.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.953+0000 7fee6ce00640 1 --2- 172.21.15.131:0/3801111858 >> [v2:172.21.15.187:6832/1027783169,v1:172.21.15.187:6834/1027783169] conn(0x5592ee6ddcd0 0x5592ee6de0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:04.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.954+0000 7fee6ce00640 1 --2- 172.21.15.131:0/3801111858 >> [v2:172.21.15.187:6832/1027783169,v1:172.21.15.187:6834/1027783169] conn(0x5592ee6ddcd0 0x5592ee6de0b0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=dd9a8d97a52529bd server_cookie=d7f3f955c52082dd in_seq=0 out_seq=0 2024-05-26T22:51:04.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.954+0000 7fee56000640 10 client.6063 ms_handle_connect on v2:172.21.15.187:6832/1027783169 2024-05-26T22:51:04.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.954+0000 7fee56000640 1 -- 172.21.15.131:0/3801111858 <== mds.2 v2:172.21.15.187:6832/1027783169 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x5592ee657f40 con 0x5592ee6ddcd0 2024-05-26T22:51:04.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.954+0000 7fee56000640 10 client.6063 handle_command_reply: tid=0 2024-05-26T22:51:04.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.954+0000 7fee727bc740 1 client.6063 shutdown 2024-05-26T22:51:04.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.954+0000 7fee727bc740 1 -- 172.21.15.131:0/3801111858 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5592ee645d90 msgr2=0x5592ee646610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:04.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.954+0000 7fee727bc740 1 --2- 172.21.15.131:0/3801111858 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5592ee645d90 0x5592ee646610 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fee50057790 tx=0x7fee50020070 comp rx=0 tx=0).stop 2024-05-26T22:51:04.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.954+0000 7fee727bc740 1 -- 172.21.15.131:0/3801111858 shutdown_connections 2024-05-26T22:51:04.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.954+0000 7fee727bc740 1 --2- 172.21.15.131:0/3801111858 >> [v2:172.21.15.187:6832/1027783169,v1:172.21.15.187:6834/1027783169] conn(0x5592ee6ddcd0 0x5592ee6de0b0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:04.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.954+0000 7fee727bc740 1 --2- 172.21.15.131:0/3801111858 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5592ee662c70 0x5592ee646b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:04.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.954+0000 7fee727bc740 1 --2- 172.21.15.131:0/3801111858 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5592ee645d90 0x5592ee646610 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:04.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.954+0000 7fee727bc740 1 --2- 172.21.15.131:0/3801111858 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5592ee635f40 0x5592ee684250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:04.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.954+0000 7fee727bc740 1 -- 172.21.15.131:0/3801111858 >> 172.21.15.131:0/3801111858 conn(0x5592ee645650 msgr2=0x5592ee6428e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:04.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.954+0000 7fee56000640 0 client.6063 ms_handle_reset on v2:172.21.15.187:6832/1027783169 2024-05-26T22:51:04.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.954+0000 7fee727bc740 1 -- 172.21.15.131:0/3801111858 shutdown_connections 2024-05-26T22:51:04.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.954+0000 7fee727bc740 1 -- 172.21.15.131:0/3801111858 wait complete. 2024-05-26T22:51:04.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.955+0000 7fee727bc740 20 client.6063 trim_cache size 0 max 16384 2024-05-26T22:51:04.958 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-26T22:51:04.958 INFO:teuthology.orchestra.run.smithi131.stdout: "cluster_fsid": "46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f", 2024-05-26T22:51:04.958 INFO:teuthology.orchestra.run.smithi131.stdout: "whoami": 2, 2024-05-26T22:51:04.958 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 4670, 2024-05-26T22:51:04.959 INFO:teuthology.orchestra.run.smithi131.stdout: "want_state": "up:active", 2024-05-26T22:51:04.959 INFO:teuthology.orchestra.run.smithi131.stdout: "state": "up:active", 2024-05-26T22:51:04.959 INFO:teuthology.orchestra.run.smithi131.stdout: "fs_name": "cephfs", 2024-05-26T22:51:04.959 INFO:teuthology.orchestra.run.smithi131.stdout: "rank_uptime": 16.840193994, 2024-05-26T22:51:04.959 INFO:teuthology.orchestra.run.smithi131.stdout: "mdsmap_epoch": 40, 2024-05-26T22:51:04.959 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch": 46, 2024-05-26T22:51:04.959 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch_barrier": 45, 2024-05-26T22:51:04.959 INFO:teuthology.orchestra.run.smithi131.stdout: "uptime": 42.845915222999999 2024-05-26T22:51:04.959 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-26T22:51:04.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.955+0000 7fee6f800640 1 -- 172.21.15.131:0/2037330411 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fee4406be60 msgr2=0x7fee4406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:04.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.955+0000 7fee6f800640 1 --2- 172.21.15.131:0/2037330411 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fee4406be60 0x7fee4406e320 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fee58035000 tx=0x7fee5800c040 comp rx=0 tx=0).stop 2024-05-26T22:51:04.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.955+0000 7fee6f800640 1 -- 172.21.15.131:0/2037330411 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee680ad3a0 msgr2=0x7fee680b3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:04.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.955+0000 7fee6f800640 1 --2- 172.21.15.131:0/2037330411 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee680ad3a0 0x7fee680b3850 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fee5000d5f0 tx=0x7fee5000dab0 comp rx=0 tx=0).stop 2024-05-26T22:51:04.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.955+0000 7fee6f800640 1 -- 172.21.15.131:0/2037330411 shutdown_connections 2024-05-26T22:51:04.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.955+0000 7fee6f800640 1 --2- 172.21.15.131:0/2037330411 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fee680cbef0 0x7fee6809c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:04.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.955+0000 7fee6f800640 1 --2- 172.21.15.131:0/2037330411 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fee4406be60 0x7fee4406e320 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:04.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.955+0000 7fee6f800640 1 --2- 172.21.15.131:0/2037330411 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee680ad3a0 0x7fee680b3850 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:04.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.955+0000 7fee6f800640 1 --2- 172.21.15.131:0/2037330411 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fee680acef0 0x7fee681f4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:04.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.955+0000 7fee6f800640 1 -- 172.21.15.131:0/2037330411 >> 172.21.15.131:0/2037330411 conn(0x7fee6806bc90 msgr2=0x7fee680cd2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:04.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.955+0000 7fee6f800640 1 -- 172.21.15.131:0/2037330411 shutdown_connections 2024-05-26T22:51:04.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:04.955+0000 7fee6f800640 1 -- 172.21.15.131:0/2037330411 wait complete. 2024-05-26T22:51:04.970 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-05-26T22:51:05.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.144+0000 7f366be00640 1 Processor -- start 2024-05-26T22:51:05.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.144+0000 7f366be00640 1 -- start start 2024-05-26T22:51:05.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.145+0000 7f366be00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f366c15bfa0 0x7f366c071500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:05.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.145+0000 7f366be00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f366c1552d0 con 0x7f366c15dc40 2024-05-26T22:51:05.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.146+0000 7f366be00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f366c072230 con 0x7f366c06d960 2024-05-26T22:51:05.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.146+0000 7f366be00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f366c0b3f30 con 0x7f366c15bfa0 2024-05-26T22:51:05.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.146+0000 7f366aa00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f366c15dc40 0x7f366c06d250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:52990/0 (socket says 172.21.15.131:52990) 2024-05-26T22:51:05.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.146+0000 7f366aa00640 1 -- 172.21.15.131:0/214155932 learned_addr learned my addr 172.21.15.131:0/214155932 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:51:05.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.146+0000 7f366a000640 1 -- 172.21.15.131:0/214155932 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2225417072 0 0) 0x7f366c1552d0 con 0x7f366c15dc40 2024-05-26T22:51:05.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.146+0000 7f3662000640 1 --2- 172.21.15.131:0/214155932 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f366c15bfa0 0x7f366c071500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:05.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.146+0000 7f366a000640 1 -- 172.21.15.131:0/214155932 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3650003520 con 0x7f366c15dc40 2024-05-26T22:51:05.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.146+0000 7f366a000640 1 -- 172.21.15.131:0/214155932 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3676863994 0 0) 0x7f3650003520 con 0x7f366c15dc40 2024-05-26T22:51:05.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.147+0000 7f366a000640 1 -- 172.21.15.131:0/214155932 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f366c15bfa0 msgr2=0x7f366c071500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:05.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.147+0000 7f366a000640 1 --2- 172.21.15.131:0/214155932 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f366c15bfa0 0x7f366c071500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:05.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.147+0000 7f366a000640 1 -- 172.21.15.131:0/214155932 >> v1:172.21.15.187:6789/0 conn(0x7f366c06d960 legacy=0x7f366c071a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:05.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.147+0000 7f366a000640 1 -- 172.21.15.131:0/214155932 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f366c072510 con 0x7f366c15dc40 2024-05-26T22:51:05.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.147+0000 7f366a000640 1 -- 172.21.15.131:0/214155932 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7f365c001670 con 0x7f366c15dc40 2024-05-26T22:51:05.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.147+0000 7f366a000640 1 -- 172.21.15.131:0/214155932 >> v1:172.21.15.131:6789/0 conn(0x7f366c15dc40 legacy=0x7f366c06d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:05.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.147+0000 7f366a000640 1 --2- 172.21.15.131:0/214155932 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f36500039e0 0x7f3650003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:05.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.147+0000 7f366a000640 1 --2- 172.21.15.131:0/214155932 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3650004800 0x7f3650004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:05.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.147+0000 7f366a000640 1 --2- 172.21.15.131:0/214155932 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f366c15bfa0 0x7f365000cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:05.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.147+0000 7f366a000640 1 -- 172.21.15.131:0/214155932 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f366c0b3f30 con 0x7f3650004800 2024-05-26T22:51:05.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.147+0000 7f366a000640 1 -- 172.21.15.131:0/214155932 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f36500054f0 con 0x7f36500039e0 2024-05-26T22:51:05.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.147+0000 7f366a000640 1 -- 172.21.15.131:0/214155932 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f365000d010 con 0x7f366c15bfa0 2024-05-26T22:51:05.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.147+0000 7f3662000640 1 --2- 172.21.15.131:0/214155932 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3650004800 0x7f3650004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:05.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.147+0000 7f3662000640 1 -- 172.21.15.131:0/214155932 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f366c15bfa0 msgr2=0x7f365000cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:05.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.147+0000 7f3662000640 1 --2- 172.21.15.131:0/214155932 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f366c15bfa0 0x7f365000cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:05.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.147+0000 7f3662000640 1 -- 172.21.15.131:0/214155932 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f36500039e0 msgr2=0x7f3650003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:05.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.147+0000 7f3662000640 1 --2- 172.21.15.131:0/214155932 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f36500039e0 0x7f3650003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:05.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.147+0000 7f3662000640 1 -- 172.21.15.131:0/214155932 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3650003520 con 0x7f3650004800 2024-05-26T22:51:05.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.148+0000 7f3662000640 1 --2- 172.21.15.131:0/214155932 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3650004800 0x7f3650004c80 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f365400ba00 tx=0x7f3650005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38c3c90b9879bd59 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:05.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.149+0000 7f366a000640 1 -- 172.21.15.131:0/214155932 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3654030070 con 0x7f3650004800 2024-05-26T22:51:05.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.149+0000 7f366a000640 1 -- 172.21.15.131:0/214155932 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f365402ade0 con 0x7f3650004800 2024-05-26T22:51:05.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.149+0000 7f366a000640 1 -- 172.21.15.131:0/214155932 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f365400b600 con 0x7f3650004800 2024-05-26T22:51:05.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.149+0000 7f366be00640 1 -- 172.21.15.131:0/214155932 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3650004800 msgr2=0x7f3650004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:05.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.149+0000 7f366be00640 1 --2- 172.21.15.131:0/214155932 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3650004800 0x7f3650004c80 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f365400ba00 tx=0x7f3650005830 comp rx=0 tx=0).stop 2024-05-26T22:51:05.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.149+0000 7f366b400640 1 -- 172.21.15.131:0/214155932 reap_dead start 2024-05-26T22:51:05.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.149+0000 7f366be00640 1 -- 172.21.15.131:0/214155932 shutdown_connections 2024-05-26T22:51:05.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.149+0000 7f366be00640 1 -- 172.21.15.131:0/214155932 >> 172.21.15.131:0/214155932 conn(0x7f366c0b5660 msgr2=0x7f366c0b5a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:05.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.149+0000 7f366be00640 1 -- 172.21.15.131:0/214155932 shutdown_connections 2024-05-26T22:51:05.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.149+0000 7f366be00640 1 -- 172.21.15.131:0/214155932 wait complete. 2024-05-26T22:51:05.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.151+0000 7f366be00640 1 Processor -- start 2024-05-26T22:51:05.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.151+0000 7f366be00640 1 -- start start 2024-05-26T22:51:05.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.152+0000 7f366be00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3650004800 0x7f366c1f5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:05.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.152+0000 7f366be00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f366c1f4d00 0x7f366c1f6f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:05.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.152+0000 7f366be00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f366c154160 0x7f366c1545e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:05.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.152+0000 7f366be00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f366c15e1a0 con 0x7f366c1f4d00 2024-05-26T22:51:05.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.152+0000 7f366be00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f366c15e020 con 0x7f3650004800 2024-05-26T22:51:05.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.152+0000 7f366be00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f366c071500 con 0x7f366c154160 2024-05-26T22:51:05.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.152+0000 7f3662000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f366c1f4d00 0x7f366c1f6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:05.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.152+0000 7f3662000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f366c1f4d00 0x7f366c1f6f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46788/0 (socket says 172.21.15.131:46788) 2024-05-26T22:51:05.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.152+0000 7f3662000640 1 -- 172.21.15.131:0/3194654633 learned_addr learned my addr 172.21.15.131:0/3194654633 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:05.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.153+0000 7f3662000640 1 -- 172.21.15.131:0/3194654633 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f366c154160 msgr2=0x7f366c1545e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:05.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.153+0000 7f3662000640 1 --2- 172.21.15.131:0/3194654633 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f366c154160 0x7f366c1545e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:05.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.153+0000 7f3662000640 1 -- 172.21.15.131:0/3194654633 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3650004800 msgr2=0x7f366c1f5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:05.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.153+0000 7f3662000640 1 --2- 172.21.15.131:0/3194654633 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3650004800 0x7f366c1f5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:05.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.153+0000 7f3662000640 1 -- 172.21.15.131:0/3194654633 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f366c151a30 con 0x7f366c1f4d00 2024-05-26T22:51:05.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.153+0000 7f3662000640 1 --2- 172.21.15.131:0/3194654633 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f366c1f4d00 0x7f366c1f6f90 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f365400bb20 tx=0x7f36540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:05.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.153+0000 7f3663400640 1 -- 172.21.15.131:0/3194654633 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3654043020 con 0x7f366c1f4d00 2024-05-26T22:51:05.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.153+0000 7f3663400640 1 -- 172.21.15.131:0/3194654633 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f365403f680 con 0x7f366c1f4d00 2024-05-26T22:51:05.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.153+0000 7f366be00640 1 -- 172.21.15.131:0/3194654633 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f366c151d10 con 0x7f366c1f4d00 2024-05-26T22:51:05.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.153+0000 7f3663400640 1 -- 172.21.15.131:0/3194654633 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f365400b600 con 0x7f366c1f4d00 2024-05-26T22:51:05.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.153+0000 7f366be00640 1 -- 172.21.15.131:0/3194654633 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f366c1522d0 con 0x7f366c1f4d00 2024-05-26T22:51:05.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.155+0000 7f3663400640 1 -- 172.21.15.131:0/3194654633 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3654046020 con 0x7f366c1f4d00 2024-05-26T22:51:05.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.156+0000 7f3663400640 1 --2- 172.21.15.131:0/3194654633 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f364806be30 0x7f364806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:05.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.157+0000 7f3663400640 1 -- 172.21.15.131:0/3194654633 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5853+0+0 (secure 0 0 0) 0x7f36540abe40 con 0x7f366c1f4d00 2024-05-26T22:51:05.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.157+0000 7f366aa00640 1 --2- 172.21.15.131:0/3194654633 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f364806be30 0x7f364806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:05.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.157+0000 7f3674973740 1 Processor -- start 2024-05-26T22:51:05.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.157+0000 7f3674973740 1 -- start start 2024-05-26T22:51:05.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.158+0000 7f366aa00640 1 --2- 172.21.15.131:0/3194654633 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f364806be30 0x7f364806e2f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f365c0079b0 tx=0x7f365c001580 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:05.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.158+0000 7f3674973740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5653d84f4e50 0x5653d84a7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:05.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.158+0000 7f3674973740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5653d84d7f70 0x5653d84d72f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:05.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.158+0000 7f3674973740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5653d84d7830 0x5653d84ac640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:05.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.158+0000 7f3674973740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5653d840f610 con 0x5653d84f4e50 2024-05-26T22:51:05.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.158+0000 7f3674973740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5653d83b75c0 con 0x5653d84d7f70 2024-05-26T22:51:05.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.158+0000 7f3674973740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x5653d840b5d0 con 0x5653d84d7830 2024-05-26T22:51:05.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.159+0000 7f366aa00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5653d84f4e50 0x5653d84a7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:05.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.159+0000 7f366aa00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5653d84f4e50 0x5653d84a7a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46800/0 (socket says 172.21.15.131:46800) 2024-05-26T22:51:05.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.159+0000 7f366aa00640 1 -- 172.21.15.131:0/1391963273 learned_addr learned my addr 172.21.15.131:0/1391963273 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:05.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.159+0000 7f366aa00640 1 -- 172.21.15.131:0/1391963273 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5653d84d7830 msgr2=0x5653d84ac640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:05.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.159+0000 7f366aa00640 1 --2- 172.21.15.131:0/1391963273 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5653d84d7830 0x5653d84ac640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:05.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.159+0000 7f366aa00640 1 -- 172.21.15.131:0/1391963273 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5653d84d7f70 msgr2=0x5653d84d72f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:05.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.159+0000 7f366aa00640 1 --2- 172.21.15.131:0/1391963273 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5653d84d7f70 0x5653d84d72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:05.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.159+0000 7f366aa00640 1 -- 172.21.15.131:0/1391963273 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653d81232d0 con 0x5653d84f4e50 2024-05-26T22:51:05.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.159+0000 7f366aa00640 1 --2- 172.21.15.131:0/1391963273 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5653d84f4e50 0x5653d84a7a70 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f365c0074a0 tx=0x7f365c00f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c426ee0e9f159562 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:05.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.159+0000 7f365b400640 1 -- 172.21.15.131:0/1391963273 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f365c005c80 con 0x5653d84f4e50 2024-05-26T22:51:05.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.160+0000 7f365b400640 1 -- 172.21.15.131:0/1391963273 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f365c005e20 con 0x5653d84f4e50 2024-05-26T22:51:05.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.160+0000 7f365b400640 1 -- 172.21.15.131:0/1391963273 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f365c00ce60 con 0x5653d84f4e50 2024-05-26T22:51:05.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.160+0000 7f3674973740 1 -- 172.21.15.131:0/1391963273 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5653d84f4e50 msgr2=0x5653d84a7a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:05.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.160+0000 7f3674973740 1 --2- 172.21.15.131:0/1391963273 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5653d84f4e50 0x5653d84a7a70 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f365c0074a0 tx=0x7f365c00f000 comp rx=0 tx=0).stop 2024-05-26T22:51:05.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.160+0000 7f3674973740 1 -- 172.21.15.131:0/1391963273 shutdown_connections 2024-05-26T22:51:05.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.160+0000 7f3674973740 1 --2- 172.21.15.131:0/1391963273 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5653d84d7830 0x5653d84ac640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:05.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.160+0000 7f3674973740 1 --2- 172.21.15.131:0/1391963273 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5653d84d7f70 0x5653d84d72f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:05.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.160+0000 7f3674973740 1 --2- 172.21.15.131:0/1391963273 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5653d84f4e50 0x5653d84a7a70 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:05.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.160+0000 7f3674973740 1 -- 172.21.15.131:0/1391963273 >> 172.21.15.131:0/1391963273 conn(0x5653d84a9ce0 msgr2=0x5653d84f22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:05.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.160+0000 7f3674973740 1 -- 172.21.15.131:0/1391963273 shutdown_connections 2024-05-26T22:51:05.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.160+0000 7f3674973740 1 -- 172.21.15.131:0/1391963273 wait complete. 2024-05-26T22:51:05.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.161+0000 7f3674973740 1 -- start start 2024-05-26T22:51:05.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.161+0000 7f3674973740 1 Processor -- start 2024-05-26T22:51:05.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.162+0000 7f3674973740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5653d84d7f70 0x5653d84d3290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:05.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.162+0000 7f3674973740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5653d84f4e50 0x5653d84d8e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:05.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.162+0000 7f3674973740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5653d84da590 0x5653d84daa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:05.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.162+0000 7f3674973740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5653d84abc00 con 0x5653d84da590 2024-05-26T22:51:05.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.162+0000 7f3674973740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5653d83ca910 con 0x5653d84d7f70 2024-05-26T22:51:05.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.162+0000 7f3674973740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x5653d84aaff0 con 0x5653d84f4e50 2024-05-26T22:51:05.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.162+0000 7f3674973740 10 client.0 _ll_register_callbacks cb 0x5653d8406410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-26T22:51:05.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.162+0000 7f366b400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5653d84da590 0x5653d84daa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:05.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.163+0000 7f366b400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5653d84da590 0x5653d84daa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46802/0 (socket says 172.21.15.131:46802) 2024-05-26T22:51:05.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.163+0000 7f366b400640 1 -- 172.21.15.131:0/1396706512 learned_addr learned my addr 172.21.15.131:0/1396706512 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:05.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.163+0000 7f366aa00640 1 --2- 172.21.15.131:0/1396706512 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5653d84d7f70 0x5653d84d3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:05.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.163+0000 7f366b400640 1 -- 172.21.15.131:0/1396706512 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5653d84f4e50 msgr2=0x5653d84d8e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:05.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.163+0000 7f366b400640 1 --2- 172.21.15.131:0/1396706512 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5653d84f4e50 0x5653d84d8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:05.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.163+0000 7f366b400640 1 -- 172.21.15.131:0/1396706512 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5653d84d7f70 msgr2=0x5653d84d3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:05.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.163+0000 7f366b400640 1 --2- 172.21.15.131:0/1396706512 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5653d84d7f70 0x5653d84d3290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:05.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.163+0000 7f366b400640 1 -- 172.21.15.131:0/1396706512 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653d81ed6b0 con 0x5653d84da590 2024-05-26T22:51:05.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.163+0000 7f366b400640 1 --2- 172.21.15.131:0/1396706512 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5653d84da590 0x5653d84daa10 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f366c0b3150 tx=0x7f366400c570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2520653b4f2b1d6f server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:05.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.163+0000 7f364fe00640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-26T22:51:05.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.163+0000 7f364fe00640 1 -- 172.21.15.131:0/1396706512 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3664005d40 con 0x5653d84da590 2024-05-26T22:51:05.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.163+0000 7f364fe00640 1 -- 172.21.15.131:0/1396706512 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(46..46 src has 1..46) ==== 5853+0+0 (secure 0 0 0) 0x7f3664014b50 con 0x5653d84da590 2024-05-26T22:51:05.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.163+0000 7f3674973740 1 -- 172.21.15.131:0/1396706512 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f365c005e20 con 0x5653d84da590 2024-05-26T22:51:05.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.164+0000 7f364fe00640 1 -- 172.21.15.131:0/1396706512 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f3664014d50 con 0x5653d84da590 2024-05-26T22:51:05.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.164+0000 7f3674973740 10 client.6075 fetch_fsmap learned FSMap version 40 2024-05-26T22:51:05.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.164+0000 7f3674973740 1 -- 172.21.15.131:0/1396706512 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=2+}) -- 0x5653d84f2390 con 0x5653d84da590 2024-05-26T22:51:05.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.164+0000 7f364fe00640 1 -- 172.21.15.131:0/1396706512 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f365c005e20 con 0x5653d84da590 2024-05-26T22:51:05.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.164+0000 7f364fe00640 1 -- 172.21.15.131:0/1396706512 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 40) ==== 3542+0+0 (secure 0 0 0) 0x7f3664007110 con 0x5653d84da590 2024-05-26T22:51:05.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.164+0000 7f3674973740 10 client.6075 fetch_fsmap finished waiting for FSMap version 40 2024-05-26T22:51:05.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.164+0000 7f3674973740 10 client.6075 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-26T22:51:05.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.164+0000 7f3674973740 20 client.6075 populate_metadata read hostname 'smithi131' 2024-05-26T22:51:05.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.164+0000 7f3674973740 1 --2- 172.21.15.131:0/1396706512 >> [v2:172.21.15.131:6834/650770835,v1:172.21.15.131:6836/650770835] conn(0x5653d8502ca0 0x5653d8510c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:05.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.164+0000 7f3674973740 4 client.6075 mds_command: new command op to 5637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-26T22:51:05.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.164+0000 7f3674973740 1 -- 172.21.15.131:0/1396706512 --> [v2:172.21.15.131:6834/650770835,v1:172.21.15.131:6836/650770835] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5653d841cd40 con 0x5653d8502ca0 2024-05-26T22:51:05.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.164+0000 7f366aa00640 1 --2- 172.21.15.131:0/1396706512 >> [v2:172.21.15.131:6834/650770835,v1:172.21.15.131:6836/650770835] conn(0x5653d8502ca0 0x5653d8510c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:05.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.165+0000 7f364fe00640 10 client.6075 ms_handle_connect on v2:172.21.15.131:6834/650770835 2024-05-26T22:51:05.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.165+0000 7f366aa00640 1 --2- 172.21.15.131:0/1396706512 >> [v2:172.21.15.131:6834/650770835,v1:172.21.15.131:6836/650770835] conn(0x5653d8502ca0 0x5653d8510c50 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=411c139e3028cf0e server_cookie=afea175c23a1963a in_seq=0 out_seq=0 2024-05-26T22:51:05.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.166+0000 7f364fe00640 1 -- 172.21.15.131:0/1396706512 <== mds.0 v2:172.21.15.131:6834/650770835 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5653d841cd40 con 0x5653d8502ca0 2024-05-26T22:51:05.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.166+0000 7f364fe00640 10 client.6075 handle_command_reply: tid=0 2024-05-26T22:51:05.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.166+0000 7f3674973740 1 client.6075 shutdown 2024-05-26T22:51:05.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.166+0000 7f3674973740 1 -- 172.21.15.131:0/1396706512 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5653d84da590 msgr2=0x5653d84daa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:05.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.166+0000 7f3674973740 1 --2- 172.21.15.131:0/1396706512 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5653d84da590 0x5653d84daa10 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f366c0b3150 tx=0x7f366400c570 comp rx=0 tx=0).stop 2024-05-26T22:51:05.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.166+0000 7f3674973740 1 -- 172.21.15.131:0/1396706512 shutdown_connections 2024-05-26T22:51:05.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.166+0000 7f3674973740 1 --2- 172.21.15.131:0/1396706512 >> [v2:172.21.15.131:6834/650770835,v1:172.21.15.131:6836/650770835] conn(0x5653d8502ca0 0x5653d8510c50 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:05.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.166+0000 7f3674973740 1 --2- 172.21.15.131:0/1396706512 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5653d84f4e50 0x5653d84d8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:05.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.166+0000 7f3674973740 1 --2- 172.21.15.131:0/1396706512 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5653d84da590 0x5653d84daa10 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:05.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.166+0000 7f3674973740 1 --2- 172.21.15.131:0/1396706512 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5653d84d7f70 0x5653d84d3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:05.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.166+0000 7f364fe00640 0 client.6075 ms_handle_reset on v2:172.21.15.131:6834/650770835 2024-05-26T22:51:05.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.166+0000 7f3674973740 1 -- 172.21.15.131:0/1396706512 >> 172.21.15.131:0/1396706512 conn(0x5653d84d7830 msgr2=0x5653d84f5710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:05.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.166+0000 7f3674973740 1 -- 172.21.15.131:0/1396706512 shutdown_connections 2024-05-26T22:51:05.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.166+0000 7f3674973740 1 -- 172.21.15.131:0/1396706512 wait complete. 2024-05-26T22:51:05.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.166+0000 7f3674973740 20 client.6075 trim_cache size 0 max 16384 2024-05-26T22:51:05.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.186+0000 7f3674973740 1 Processor -- start 2024-05-26T22:51:05.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.186+0000 7f3674973740 1 -- start start 2024-05-26T22:51:05.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.186+0000 7f3674973740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5653d84d7f70 0x5653d84a78b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:05.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.186+0000 7f3674973740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5653d84da590 0x5653d84f5910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:05.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.186+0000 7f3674973740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5653d84f4e50 0x5653d84f5e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:05.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.187+0000 7f3674973740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f366c2013d0 con 0x5653d84da590 2024-05-26T22:51:05.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.187+0000 7f3674973740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f366c2005d0 con 0x5653d84d7f70 2024-05-26T22:51:05.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.187+0000 7f3674973740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f366c200ca0 con 0x5653d84f4e50 2024-05-26T22:51:05.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.187+0000 7f3662000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5653d84da590 0x5653d84f5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:05.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.187+0000 7f3662000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5653d84da590 0x5653d84f5910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46816/0 (socket says 172.21.15.131:46816) 2024-05-26T22:51:05.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.187+0000 7f3662000640 1 -- 172.21.15.131:0/3602578222 learned_addr learned my addr 172.21.15.131:0/3602578222 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:05.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.187+0000 7f3662000640 1 -- 172.21.15.131:0/3602578222 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5653d84f4e50 msgr2=0x5653d84f5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:05.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.187+0000 7f3662000640 1 --2- 172.21.15.131:0/3602578222 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5653d84f4e50 0x5653d84f5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:05.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.187+0000 7f3662000640 1 -- 172.21.15.131:0/3602578222 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5653d84d7f70 msgr2=0x5653d84a78b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:05.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.187+0000 7f3662000640 1 --2- 172.21.15.131:0/3602578222 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5653d84d7f70 0x5653d84a78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:05.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.187+0000 7f3662000640 1 -- 172.21.15.131:0/3602578222 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653d840b750 con 0x5653d84da590 2024-05-26T22:51:05.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.187+0000 7f3662000640 1 --2- 172.21.15.131:0/3602578222 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5653d84da590 0x5653d84f5910 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f3654082670 tx=0x7f3654079950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdbcf9500c371296 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:05.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.188+0000 7f365aa00640 1 -- 172.21.15.131:0/3602578222 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f365407e2e0 con 0x5653d84da590 2024-05-26T22:51:05.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.188+0000 7f365aa00640 1 -- 172.21.15.131:0/3602578222 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3654082ba0 con 0x5653d84da590 2024-05-26T22:51:05.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.188+0000 7f365aa00640 1 -- 172.21.15.131:0/3602578222 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36540765a0 con 0x5653d84da590 2024-05-26T22:51:05.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.188+0000 7f3674973740 1 -- 172.21.15.131:0/3602578222 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5653d84da590 msgr2=0x5653d84f5910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:05.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.188+0000 7f3674973740 1 --2- 172.21.15.131:0/3602578222 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5653d84da590 0x5653d84f5910 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f3654082670 tx=0x7f3654079950 comp rx=0 tx=0).stop 2024-05-26T22:51:05.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.188+0000 7f3674973740 1 -- 172.21.15.131:0/3602578222 shutdown_connections 2024-05-26T22:51:05.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.188+0000 7f3674973740 1 --2- 172.21.15.131:0/3602578222 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5653d84f4e50 0x5653d84f5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:05.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.188+0000 7f3674973740 1 --2- 172.21.15.131:0/3602578222 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5653d84da590 0x5653d84f5910 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:05.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.188+0000 7f3674973740 1 --2- 172.21.15.131:0/3602578222 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5653d84d7f70 0x5653d84a78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:05.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.188+0000 7f3674973740 1 -- 172.21.15.131:0/3602578222 >> 172.21.15.131:0/3602578222 conn(0x5653d84d7830 msgr2=0x5653d84d30a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:05.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.188+0000 7f3674973740 1 -- 172.21.15.131:0/3602578222 shutdown_connections 2024-05-26T22:51:05.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.188+0000 7f3674973740 1 -- 172.21.15.131:0/3602578222 wait complete. 2024-05-26T22:51:05.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.189+0000 7f3674973740 1 -- start start 2024-05-26T22:51:05.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.189+0000 7f3674973740 1 Processor -- start 2024-05-26T22:51:05.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.190+0000 7f3674973740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5653d84d7f70 0x5653d84f5710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:05.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.190+0000 7f3674973740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5653d84da590 0x5653d84f5c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:05.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.190+0000 7f3674973740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5653d84f4e50 0x5653d84acca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:05.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.190+0000 7f3674973740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5653d84f2910 con 0x5653d84da590 2024-05-26T22:51:05.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.190+0000 7f3674973740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f366c15a280 con 0x5653d84d7f70 2024-05-26T22:51:05.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.190+0000 7f3674973740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x5653d85035e0 con 0x5653d84f4e50 2024-05-26T22:51:05.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.190+0000 7f3674973740 10 client.0 _ll_register_callbacks cb 0x7f3664007ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-26T22:51:05.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.190+0000 7f3662000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5653d84da590 0x5653d84f5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:05.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.191+0000 7f3662000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5653d84da590 0x5653d84f5c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46832/0 (socket says 172.21.15.131:46832) 2024-05-26T22:51:05.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.191+0000 7f3662000640 1 -- 172.21.15.131:0/2410072291 learned_addr learned my addr 172.21.15.131:0/2410072291 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:05.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.191+0000 7f3662000640 1 -- 172.21.15.131:0/2410072291 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5653d84f4e50 msgr2=0x5653d84acca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:05.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.191+0000 7f3662000640 1 --2- 172.21.15.131:0/2410072291 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5653d84f4e50 0x5653d84acca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:05.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.191+0000 7f3662000640 1 -- 172.21.15.131:0/2410072291 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5653d84d7f70 msgr2=0x5653d84f5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:05.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.191+0000 7f3662000640 1 --2- 172.21.15.131:0/2410072291 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5653d84d7f70 0x5653d84f5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:05.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.191+0000 7f3662000640 1 -- 172.21.15.131:0/2410072291 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653d84d4c00 con 0x5653d84da590 2024-05-26T22:51:05.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.191+0000 7f3662000640 1 --2- 172.21.15.131:0/2410072291 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5653d84da590 0x5653d84f5c50 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f365400baf0 tx=0x7f36540799c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12a854c208081ade server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:05.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.191+0000 7f364fe00640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-26T22:51:05.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.191+0000 7f364fe00640 1 -- 172.21.15.131:0/2410072291 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f365407e2e0 con 0x5653d84da590 2024-05-26T22:51:05.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.191+0000 7f364fe00640 1 -- 172.21.15.131:0/2410072291 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(46..46 src has 1..46) ==== 5853+0+0 (secure 0 0 0) 0x7f3654082920 con 0x5653d84da590 2024-05-26T22:51:05.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.191+0000 7f3674973740 1 -- 172.21.15.131:0/2410072291 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3654082ba0 con 0x5653d84da590 2024-05-26T22:51:05.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.192+0000 7f364fe00640 1 -- 172.21.15.131:0/2410072291 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f36540a1050 con 0x5653d84da590 2024-05-26T22:51:05.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.192+0000 7f3674973740 10 client.6081 fetch_fsmap learned FSMap version 40 2024-05-26T22:51:05.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.192+0000 7f3674973740 1 -- 172.21.15.131:0/2410072291 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=2+}) -- 0x5653d84f2160 con 0x5653d84da590 2024-05-26T22:51:05.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.192+0000 7f364fe00640 1 -- 172.21.15.131:0/2410072291 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3654082ba0 con 0x5653d84da590 2024-05-26T22:51:05.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.192+0000 7f364fe00640 1 -- 172.21.15.131:0/2410072291 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 40) ==== 3542+0+0 (secure 0 0 0) 0x7f36540a3ba0 con 0x5653d84da590 2024-05-26T22:51:05.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.192+0000 7f3674973740 10 client.6081 fetch_fsmap finished waiting for FSMap version 40 2024-05-26T22:51:05.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.192+0000 7f3674973740 10 client.6081 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-26T22:51:05.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.192+0000 7f3674973740 20 client.6081 populate_metadata read hostname 'smithi131' 2024-05-26T22:51:05.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.192+0000 7f3674973740 1 --2- 172.21.15.131:0/2410072291 >> [v2:172.21.15.131:6834/650770835,v1:172.21.15.131:6836/650770835] conn(0x5653d84ad250 0x5653d84aba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:05.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.192+0000 7f3674973740 4 client.6081 mds_command: new command op to 5637 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-26T22:51:05.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.192+0000 7f3674973740 1 -- 172.21.15.131:0/2410072291 --> [v2:172.21.15.131:6834/650770835,v1:172.21.15.131:6836/650770835] -- command(tid 0: {"prefix": "status"}) -- 0x5653d84add40 con 0x5653d84ad250 2024-05-26T22:51:05.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.192+0000 7f366aa00640 1 --2- 172.21.15.131:0/2410072291 >> [v2:172.21.15.131:6834/650770835,v1:172.21.15.131:6836/650770835] conn(0x5653d84ad250 0x5653d84aba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:05.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.193+0000 7f366aa00640 1 --2- 172.21.15.131:0/2410072291 >> [v2:172.21.15.131:6834/650770835,v1:172.21.15.131:6836/650770835] conn(0x5653d84ad250 0x5653d84aba80 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=468e9f6660cc9474 server_cookie=1fae50225b65339f in_seq=0 out_seq=0 2024-05-26T22:51:05.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.193+0000 7f364fe00640 10 client.6081 ms_handle_connect on v2:172.21.15.131:6834/650770835 2024-05-26T22:51:05.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.193+0000 7f364fe00640 1 -- 172.21.15.131:0/2410072291 <== mds.0 v2:172.21.15.131:6834/650770835 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x5653d84add40 con 0x5653d84ad250 2024-05-26T22:51:05.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.193+0000 7f364fe00640 10 client.6081 handle_command_reply: tid=0 2024-05-26T22:51:05.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.193+0000 7f3674973740 1 client.6081 shutdown 2024-05-26T22:51:05.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.193+0000 7f3674973740 1 -- 172.21.15.131:0/2410072291 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5653d84da590 msgr2=0x5653d84f5c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:05.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.193+0000 7f3674973740 1 --2- 172.21.15.131:0/2410072291 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5653d84da590 0x5653d84f5c50 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f365400baf0 tx=0x7f36540799c0 comp rx=0 tx=0).stop 2024-05-26T22:51:05.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.193+0000 7f3674973740 1 -- 172.21.15.131:0/2410072291 shutdown_connections 2024-05-26T22:51:05.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.193+0000 7f3674973740 1 --2- 172.21.15.131:0/2410072291 >> [v2:172.21.15.131:6834/650770835,v1:172.21.15.131:6836/650770835] conn(0x5653d84ad250 0x5653d84aba80 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:05.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.193+0000 7f3674973740 1 --2- 172.21.15.131:0/2410072291 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5653d84f4e50 0x5653d84acca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:05.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.193+0000 7f3674973740 1 --2- 172.21.15.131:0/2410072291 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5653d84da590 0x5653d84f5c50 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:05.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.193+0000 7f364fe00640 0 client.6081 ms_handle_reset on v2:172.21.15.131:6834/650770835 2024-05-26T22:51:05.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.193+0000 7f3674973740 1 --2- 172.21.15.131:0/2410072291 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5653d84d7f70 0x5653d84f5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:05.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.193+0000 7f3674973740 1 -- 172.21.15.131:0/2410072291 >> 172.21.15.131:0/2410072291 conn(0x5653d84d7830 msgr2=0x5653d84d9d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:05.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.193+0000 7f3674973740 1 -- 172.21.15.131:0/2410072291 shutdown_connections 2024-05-26T22:51:05.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.193+0000 7f3674973740 1 -- 172.21.15.131:0/2410072291 wait complete. 2024-05-26T22:51:05.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.193+0000 7f3674973740 20 client.6081 trim_cache size 0 max 16384 2024-05-26T22:51:05.199 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-26T22:51:05.199 INFO:teuthology.orchestra.run.smithi131.stdout: "cluster_fsid": "46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f", 2024-05-26T22:51:05.199 INFO:teuthology.orchestra.run.smithi131.stdout: "whoami": 0, 2024-05-26T22:51:05.199 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 5637, 2024-05-26T22:51:05.199 INFO:teuthology.orchestra.run.smithi131.stdout: "want_state": "up:active", 2024-05-26T22:51:05.199 INFO:teuthology.orchestra.run.smithi131.stdout: "state": "up:active", 2024-05-26T22:51:05.199 INFO:teuthology.orchestra.run.smithi131.stdout: "fs_name": "cephfs", 2024-05-26T22:51:05.199 INFO:teuthology.orchestra.run.smithi131.stdout: "rank_uptime": 37.59870944, 2024-05-26T22:51:05.200 INFO:teuthology.orchestra.run.smithi131.stdout: "mdsmap_epoch": 40, 2024-05-26T22:51:05.200 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch": 46, 2024-05-26T22:51:05.200 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch_barrier": 45, 2024-05-26T22:51:05.200 INFO:teuthology.orchestra.run.smithi131.stdout: "uptime": 43.098182643999998 2024-05-26T22:51:05.200 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-26T22:51:05.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.194+0000 7f366be00640 1 -- 172.21.15.131:0/3194654633 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f364806be30 msgr2=0x7f364806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:05.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.194+0000 7f366be00640 1 --2- 172.21.15.131:0/3194654633 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f364806be30 0x7f364806e2f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f365c0079b0 tx=0x7f365c001580 comp rx=0 tx=0).stop 2024-05-26T22:51:05.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.194+0000 7f366be00640 1 -- 172.21.15.131:0/3194654633 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f366c1f4d00 msgr2=0x7f366c1f6f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:05.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.194+0000 7f366be00640 1 --2- 172.21.15.131:0/3194654633 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f366c1f4d00 0x7f366c1f6f90 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f365400bb20 tx=0x7f36540046c0 comp rx=0 tx=0).stop 2024-05-26T22:51:05.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.194+0000 7f366be00640 1 -- 172.21.15.131:0/3194654633 shutdown_connections 2024-05-26T22:51:05.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.194+0000 7f366be00640 1 --2- 172.21.15.131:0/3194654633 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f366c154160 0x7f366c1545e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:05.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.194+0000 7f366be00640 1 --2- 172.21.15.131:0/3194654633 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f364806be30 0x7f364806e2f0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:05.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.194+0000 7f366be00640 1 --2- 172.21.15.131:0/3194654633 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f366c1f4d00 0x7f366c1f6f90 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:05.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.194+0000 7f366be00640 1 --2- 172.21.15.131:0/3194654633 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3650004800 0x7f366c1f5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:05.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.194+0000 7f366be00640 1 -- 172.21.15.131:0/3194654633 >> 172.21.15.131:0/3194654633 conn(0x7f366c0b5660 msgr2=0x7f366c0cfda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:05.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.194+0000 7f366be00640 1 -- 172.21.15.131:0/3194654633 shutdown_connections 2024-05-26T22:51:05.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:05.194+0000 7f366be00640 1 -- 172.21.15.131:0/3194654633 wait complete. 2024-05-26T22:51:07.515 INFO:tasks.ceph.mds.c.smithi131.stderr:2024-05-26T22:51:07.515+0000 7efef9c00640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-05-26T22:51:14.556 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-26T22:51:14.564 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-26T22:51:15.215 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-26T22:51:15.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.424+0000 7fdbd7c00640 1 Processor -- start 2024-05-26T22:51:15.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.424+0000 7fdbd7c00640 1 -- start start 2024-05-26T22:51:15.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.426+0000 7fdbd7c00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fdbd00b0350 0x7fdbd015ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:15.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.426+0000 7fdbd7c00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdbd00aee80 con 0x7fdbd00b3f20 2024-05-26T22:51:15.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.426+0000 7fdbd7c00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdbd00af160 con 0x7fdbd0072470 2024-05-26T22:51:15.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.426+0000 7fdbd7c00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fdbd00af440 con 0x7fdbd00b0350 2024-05-26T22:51:15.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.426+0000 7fdbc7e00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fdbd00b3f20 0x7fdbd00afc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:51302/0 (socket says 172.21.15.131:51302) 2024-05-26T22:51:15.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.426+0000 7fdbc7e00640 1 -- 172.21.15.131:0/2413126493 learned_addr learned my addr 172.21.15.131:0/2413126493 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:51:15.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.427+0000 7fdbd5c00640 1 --2- 172.21.15.131:0/2413126493 >> v2:172.21.15.187:3300/0 conn(0x7fdbd00b0350 0x7fdbd015ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:15.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.427+0000 7fdbcfe00640 1 -- 172.21.15.131:0/2413126493 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3328873147 0 0) 0x7fdbd00aee80 con 0x7fdbd00b3f20 2024-05-26T22:51:15.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.427+0000 7fdbcfe00640 1 -- 172.21.15.131:0/2413126493 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdbb4003520 con 0x7fdbd00b3f20 2024-05-26T22:51:15.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.427+0000 7fdbcfe00640 1 -- 172.21.15.131:0/2413126493 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4270672252 0 0) 0x7fdbb4003520 con 0x7fdbd00b3f20 2024-05-26T22:51:15.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.427+0000 7fdbcfe00640 1 -- 172.21.15.131:0/2413126493 >> v2:172.21.15.187:3300/0 conn(0x7fdbd00b0350 msgr2=0x7fdbd015ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:15.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.427+0000 7fdbcfe00640 1 --2- 172.21.15.131:0/2413126493 >> v2:172.21.15.187:3300/0 conn(0x7fdbd00b0350 0x7fdbd015ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:15.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.427+0000 7fdbcfe00640 1 -- 172.21.15.131:0/2413126493 >> v1:172.21.15.187:6789/0 conn(0x7fdbd0072470 legacy=0x7fdbd0072850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:15.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.427+0000 7fdbcfe00640 1 -- 172.21.15.131:0/2413126493 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbd00af720 con 0x7fdbd00b3f20 2024-05-26T22:51:15.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.427+0000 7fdbcfe00640 1 -- 172.21.15.131:0/2413126493 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7fdbb8001670 con 0x7fdbd00b3f20 2024-05-26T22:51:15.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.427+0000 7fdbcfe00640 1 -- 172.21.15.131:0/2413126493 >> v1:172.21.15.131:6789/0 conn(0x7fdbd00b3f20 legacy=0x7fdbd00afc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:15.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.427+0000 7fdbcfe00640 1 --2- 172.21.15.131:0/2413126493 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdbb40039b0 0x7fdbb4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:15.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.427+0000 7fdbcfe00640 1 --2- 172.21.15.131:0/2413126493 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdbb4004800 0x7fdbb4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:15.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.427+0000 7fdbcfe00640 1 --2- 172.21.15.131:0/2413126493 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdbb40051c0 0x7fdbb400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:15.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.428+0000 7fdbcfe00640 1 -- 172.21.15.131:0/2413126493 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd00af440 con 0x7fdbb4004800 2024-05-26T22:51:15.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.428+0000 7fdbcfe00640 1 -- 172.21.15.131:0/2413126493 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb400c060 con 0x7fdbb40039b0 2024-05-26T22:51:15.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.428+0000 7fdbcfe00640 1 -- 172.21.15.131:0/2413126493 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fdbb400c260 con 0x7fdbb40051c0 2024-05-26T22:51:15.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.428+0000 7fdbc7e00640 1 --2- 172.21.15.131:0/2413126493 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdbb4004800 0x7fdbb4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:15.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.428+0000 7fdbd5200640 1 --2- 172.21.15.131:0/2413126493 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdbb40039b0 0x7fdbb4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:15.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.428+0000 7fdbc7e00640 1 -- 172.21.15.131:0/2413126493 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdbb40051c0 msgr2=0x7fdbb400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:15.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.428+0000 7fdbc7e00640 1 --2- 172.21.15.131:0/2413126493 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdbb40051c0 0x7fdbb400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:15.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.428+0000 7fdbc7e00640 1 -- 172.21.15.131:0/2413126493 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdbb40039b0 msgr2=0x7fdbb4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:15.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.428+0000 7fdbc7e00640 1 --2- 172.21.15.131:0/2413126493 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdbb40039b0 0x7fdbb4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:15.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.428+0000 7fdbc7e00640 1 -- 172.21.15.131:0/2413126493 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbb4003520 con 0x7fdbb4004800 2024-05-26T22:51:15.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.428+0000 7fdbd5200640 1 --2- 172.21.15.131:0/2413126493 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdbb40039b0 0x7fdbb4003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:51:15.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.428+0000 7fdbd5c00640 1 -- 172.21.15.131:0/2413126493 reap_dead start 2024-05-26T22:51:15.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.429+0000 7fdbc7e00640 1 --2- 172.21.15.131:0/2413126493 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdbb4004800 0x7fdbb4004c80 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fdbb8007fb0 tx=0x7fdbb4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f953212b9c1fd9b server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:15.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.429+0000 7fdbcfe00640 1 -- 172.21.15.131:0/2413126493 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbb8007d30 con 0x7fdbb4004800 2024-05-26T22:51:15.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.429+0000 7fdbcfe00640 1 -- 172.21.15.131:0/2413126493 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbb8036070 con 0x7fdbb4004800 2024-05-26T22:51:15.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.429+0000 7fdbcfe00640 1 -- 172.21.15.131:0/2413126493 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbb8005d70 con 0x7fdbb4004800 2024-05-26T22:51:15.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.430+0000 7fdbd7c00640 1 -- 172.21.15.131:0/2413126493 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdbb4004800 msgr2=0x7fdbb4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:15.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.430+0000 7fdbd7c00640 1 --2- 172.21.15.131:0/2413126493 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdbb4004800 0x7fdbb4004c80 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fdbb8007fb0 tx=0x7fdbb4005830 comp rx=0 tx=0).stop 2024-05-26T22:51:15.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.430+0000 7fdbd7c00640 1 -- 172.21.15.131:0/2413126493 shutdown_connections 2024-05-26T22:51:15.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.430+0000 7fdbd7c00640 1 --2- 172.21.15.131:0/2413126493 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdbb4004800 0x7fdbb4004c80 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:15.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.430+0000 7fdbd7c00640 1 -- 172.21.15.131:0/2413126493 >> 172.21.15.131:0/2413126493 conn(0x7fdbd00d0ff0 msgr2=0x7fdbd00d1400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:15.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.430+0000 7fdbd7c00640 1 -- 172.21.15.131:0/2413126493 shutdown_connections 2024-05-26T22:51:15.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.430+0000 7fdbd7c00640 1 -- 172.21.15.131:0/2413126493 wait complete. 2024-05-26T22:51:15.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.432+0000 7fdbd7c00640 1 Processor -- start 2024-05-26T22:51:15.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.432+0000 7fdbd7c00640 1 -- start start 2024-05-26T22:51:15.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.433+0000 7fdbd7c00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdbb4004800 0x7fdbd01e3960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:15.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.433+0000 7fdbd7c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdbd01e5090 0x7fdbd009cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:15.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.433+0000 7fdbd7c00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdbd009d1a0 0x7fdbd009b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:15.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.433+0000 7fdbd7c00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd0155ba0 con 0x7fdbd01e5090 2024-05-26T22:51:15.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.433+0000 7fdbd7c00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd0155a20 con 0x7fdbd009d1a0 2024-05-26T22:51:15.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.433+0000 7fdbd7c00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fdbd00b43d0 con 0x7fdbb4004800 2024-05-26T22:51:15.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.434+0000 7fdbc7e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdbd01e5090 0x7fdbd009cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:15.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.434+0000 7fdbc7e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdbd01e5090 0x7fdbd009cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44698/0 (socket says 172.21.15.131:44698) 2024-05-26T22:51:15.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.434+0000 7fdbc7e00640 1 -- 172.21.15.131:0/4238745374 learned_addr learned my addr 172.21.15.131:0/4238745374 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:15.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.434+0000 7fdbd5200640 1 --2- 172.21.15.131:0/4238745374 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdbb4004800 0x7fdbd01e3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:15.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.434+0000 7fdbc7e00640 1 -- 172.21.15.131:0/4238745374 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdbb4004800 msgr2=0x7fdbd01e3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:15.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.434+0000 7fdbc7e00640 1 --2- 172.21.15.131:0/4238745374 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdbb4004800 0x7fdbd01e3960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:15.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.434+0000 7fdbc7e00640 1 -- 172.21.15.131:0/4238745374 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdbd009d1a0 msgr2=0x7fdbd009b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:15.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.434+0000 7fdbd5c00640 1 --2- 172.21.15.131:0/4238745374 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdbd009d1a0 0x7fdbd009b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:15.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.434+0000 7fdbc7e00640 1 --2- 172.21.15.131:0/4238745374 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdbd009d1a0 0x7fdbd009b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:15.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.434+0000 7fdbc7e00640 1 -- 172.21.15.131:0/4238745374 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbd01e3ff0 con 0x7fdbd01e5090 2024-05-26T22:51:15.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.435+0000 7fdbc7e00640 1 --2- 172.21.15.131:0/4238745374 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdbd01e5090 0x7fdbd009cc60 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fdbb8001970 tx=0x7fdbb8001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:15.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.435+0000 7fdbcd600640 1 -- 172.21.15.131:0/4238745374 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbb8007480 con 0x7fdbd01e5090 2024-05-26T22:51:15.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.435+0000 7fdbcd600640 1 -- 172.21.15.131:0/4238745374 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbb8041070 con 0x7fdbd01e5090 2024-05-26T22:51:15.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.435+0000 7fdbcd600640 1 -- 172.21.15.131:0/4238745374 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbb8000ed0 con 0x7fdbd01e5090 2024-05-26T22:51:15.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.435+0000 7fdbd7c00640 1 -- 172.21.15.131:0/4238745374 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbd01e42d0 con 0x7fdbd01e5090 2024-05-26T22:51:15.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.435+0000 7fdbd7c00640 1 -- 172.21.15.131:0/4238745374 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbd01e4890 con 0x7fdbd01e5090 2024-05-26T22:51:15.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.436+0000 7fdbd7c00640 1 -- 172.21.15.131:0/4238745374 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbd009d5b0 con 0x7fdbd01e5090 2024-05-26T22:51:15.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.437+0000 7fdbcd600640 1 -- 172.21.15.131:0/4238745374 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdbb8031480 con 0x7fdbd01e5090 2024-05-26T22:51:15.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.438+0000 7fdbcd600640 1 --2- 172.21.15.131:0/4238745374 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fdbac06bfd0 0x7fdbac06e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:15.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.438+0000 7fdbcd600640 1 -- 172.21.15.131:0/4238745374 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5853+0+0 (secure 0 0 0) 0x7fdbb80ae330 con 0x7fdbd01e5090 2024-05-26T22:51:15.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.439+0000 7fdbd5200640 1 --2- 172.21.15.131:0/4238745374 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fdbac06bfd0 0x7fdbac06e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:15.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.439+0000 7fdbd5200640 1 --2- 172.21.15.131:0/4238745374 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fdbac06bfd0 0x7fdbac06e490 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fdbc0007fd0 tx=0x7fdbc0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:15.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.443+0000 7fdbcd600640 1 -- 172.21.15.131:0/4238745374 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fdbb807cb20 con 0x7fdbd01e5090 2024-05-26T22:51:15.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.688+0000 7fdbd7c00640 1 -- 172.21.15.131:0/4238745374 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdbd00694e0 con 0x7fdbd01e5090 2024-05-26T22:51:15.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.689+0000 7fdbcd600640 1 -- 172.21.15.131:0/4238745374 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+5230 (secure 0 0 0) 0x7fdbb807c420 con 0x7fdbd01e5090 2024-05-26T22:51:15.688 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:51:15.689 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":40,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4727,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/3668228040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6833","nonce":3668228040},{"type":"v1","addr":"172.21.15.187:6835","nonce":3668228040}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-26T22:50:27.001665+0000","modified":"2024-05-26T22:51:03.848463+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":45,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5637,"mds_1":4661,"mds_2":4670},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4661":{"gid":4661,"name":"a","rank":1,"incarnation":36,"state":"up:active","state_seq":14,"addr":"172.21.15.131:6837/265040431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6835","nonce":265040431},{"type":"v1","addr":"172.21.15.131:6837","nonce":265040431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4670":{"gid":4670,"name":"b","rank":2,"incarnation":32,"state":"up:active","state_seq":8,"addr":"172.21.15.187:6834/1027783169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1027783169},{"type":"v1","addr":"172.21.15.187:6834","nonce":1027783169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5637":{"gid":5637,"name":"c","rank":0,"incarnation":25,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6836/650770835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":650770835},{"type":"v1","addr":"172.21.15.131:6836","nonce":650770835}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5637,"qdb_cluster":[4661,4670,5637]},"id":3}]} 2024-05-26T22:51:15.689 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 40 2024-05-26T22:51:15.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.691+0000 7fdbd7c00640 1 -- 172.21.15.131:0/4238745374 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fdbac06bfd0 msgr2=0x7fdbac06e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:15.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.691+0000 7fdbd7c00640 1 --2- 172.21.15.131:0/4238745374 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fdbac06bfd0 0x7fdbac06e490 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fdbc0007fd0 tx=0x7fdbc0009040 comp rx=0 tx=0).stop 2024-05-26T22:51:15.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.691+0000 7fdbd7c00640 1 -- 172.21.15.131:0/4238745374 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdbd01e5090 msgr2=0x7fdbd009cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:15.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.691+0000 7fdbd7c00640 1 --2- 172.21.15.131:0/4238745374 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdbd01e5090 0x7fdbd009cc60 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fdbb8001970 tx=0x7fdbb8001a40 comp rx=0 tx=0).stop 2024-05-26T22:51:15.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.691+0000 7fdbd7c00640 1 -- 172.21.15.131:0/4238745374 shutdown_connections 2024-05-26T22:51:15.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.691+0000 7fdbd7c00640 1 --2- 172.21.15.131:0/4238745374 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fdbac06bfd0 0x7fdbac06e490 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:15.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.691+0000 7fdbd7c00640 1 --2- 172.21.15.131:0/4238745374 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdbd009d1a0 0x7fdbd009b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:15.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.691+0000 7fdbd7c00640 1 --2- 172.21.15.131:0/4238745374 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdbd01e5090 0x7fdbd009cc60 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:15.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.691+0000 7fdbd7c00640 1 --2- 172.21.15.131:0/4238745374 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdbb4004800 0x7fdbd01e3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:15.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.691+0000 7fdbd7c00640 1 -- 172.21.15.131:0/4238745374 >> 172.21.15.131:0/4238745374 conn(0x7fdbd00d0ff0 msgr2=0x7fdbd006f870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:15.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.692+0000 7fdbd7c00640 1 -- 172.21.15.131:0/4238745374 shutdown_connections 2024-05-26T22:51:15.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.692+0000 7fdbd7c00640 1 -- 172.21.15.131:0/4238745374 wait complete. 2024-05-26T22:51:15.710 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:1 perf dump 2024-05-26T22:51:15.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.903+0000 7feb83200640 1 Processor -- start 2024-05-26T22:51:15.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.904+0000 7feb83200640 1 -- start start 2024-05-26T22:51:15.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.905+0000 7feb83200640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7feb84140e80 0x7feb8406d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:15.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.905+0000 7feb83200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb84141260 0x7feb840cfae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:15.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.905+0000 7feb83200640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb84162d00 con 0x7feb840d0020 2024-05-26T22:51:15.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.905+0000 7feb83200640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7feb840cc600 con 0x7feb84140e80 2024-05-26T22:51:15.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.905+0000 7feb83200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7feb841620f0 con 0x7feb84141260 2024-05-26T22:51:15.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.906+0000 7feb82800640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7feb840d0020 0x7feb8415eac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:51308/0 (socket says 172.21.15.131:51308) 2024-05-26T22:51:15.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.906+0000 7feb82800640 1 -- 172.21.15.131:0/3268894744 learned_addr learned my addr 172.21.15.131:0/3268894744 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:51:15.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.906+0000 7feb81e00640 1 --2- 172.21.15.131:0/3268894744 >> v2:172.21.15.187:3300/0 conn(0x7feb84140e80 0x7feb8406d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:15.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.906+0000 7feb81400640 1 --2- 172.21.15.131:0/3268894744 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb84141260 0x7feb840cfae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:15.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.906+0000 7feb80a00640 1 -- 172.21.15.131:0/3268894744 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1665385286 0 0) 0x7feb84162d00 con 0x7feb840d0020 2024-05-26T22:51:15.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.906+0000 7feb80a00640 1 -- 172.21.15.131:0/3268894744 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feb68003520 con 0x7feb840d0020 2024-05-26T22:51:15.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.906+0000 7feb81e00640 1 -- 172.21.15.131:0/3268894744 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb84141260 msgr2=0x7feb840cfae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:15.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.906+0000 7feb81e00640 1 --2- 172.21.15.131:0/3268894744 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb84141260 0x7feb840cfae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:15.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.906+0000 7feb81e00640 1 -- 172.21.15.131:0/3268894744 >> v1:172.21.15.131:6789/0 conn(0x7feb840d0020 legacy=0x7feb8415eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:15.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.906+0000 7feb81e00640 1 -- 172.21.15.131:0/3268894744 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb841623d0 con 0x7feb84140e80 2024-05-26T22:51:15.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.906+0000 7feb81400640 1 --2- 172.21.15.131:0/3268894744 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb84141260 0x7feb840cfae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-26T22:51:15.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.908+0000 7feb81e00640 1 --2- 172.21.15.131:0/3268894744 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feb84140e80 0x7feb8406d3a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7feb7c0097a0 tx=0x7feb7c02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26e822b088db03ba server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:15.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.908+0000 7feb80a00640 1 -- 172.21.15.131:0/3268894744 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb7c037070 con 0x7feb84140e80 2024-05-26T22:51:15.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.908+0000 7feb80a00640 1 -- 172.21.15.131:0/3268894744 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb7c032440 con 0x7feb84140e80 2024-05-26T22:51:15.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.908+0000 7feb80a00640 1 -- 172.21.15.131:0/3268894744 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb7c0327f0 con 0x7feb84140e80 2024-05-26T22:51:15.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.909+0000 7feb83200640 1 -- 172.21.15.131:0/3268894744 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feb84140e80 msgr2=0x7feb8406d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:15.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.909+0000 7feb83200640 1 --2- 172.21.15.131:0/3268894744 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feb84140e80 0x7feb8406d3a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7feb7c0097a0 tx=0x7feb7c02a180 comp rx=0 tx=0).stop 2024-05-26T22:51:15.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.909+0000 7feb83200640 1 -- 172.21.15.131:0/3268894744 shutdown_connections 2024-05-26T22:51:15.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.909+0000 7feb83200640 1 --2- 172.21.15.131:0/3268894744 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb84141260 0x7feb840cfae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:15.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.909+0000 7feb83200640 1 --2- 172.21.15.131:0/3268894744 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feb84140e80 0x7feb8406d3a0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:15.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.909+0000 7feb83200640 1 -- 172.21.15.131:0/3268894744 >> 172.21.15.131:0/3268894744 conn(0x7feb840cc7f0 msgr2=0x7feb840ccbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:15.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.909+0000 7feb83200640 1 -- 172.21.15.131:0/3268894744 shutdown_connections 2024-05-26T22:51:15.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.909+0000 7feb83200640 1 -- 172.21.15.131:0/3268894744 wait complete. 2024-05-26T22:51:15.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.911+0000 7feb83200640 1 Processor -- start 2024-05-26T22:51:15.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.912+0000 7feb83200640 1 -- start start 2024-05-26T22:51:15.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.913+0000 7feb83200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb840d0020 0x7feb840b2f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:15.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.913+0000 7feb83200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb84140e80 0x7feb841e79f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:15.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.913+0000 7feb83200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feb84141260 0x7feb8415fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:15.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.913+0000 7feb83200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7feb840d0470 con 0x7feb840d0020 2024-05-26T22:51:15.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.913+0000 7feb83200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7feb84162990 con 0x7feb84141260 2024-05-26T22:51:15.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.913+0000 7feb83200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7feb84161da0 con 0x7feb84140e80 2024-05-26T22:51:15.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.913+0000 7feb81e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb840d0020 0x7feb840b2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:15.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.913+0000 7feb81e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb840d0020 0x7feb840b2f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44706/0 (socket says 172.21.15.131:44706) 2024-05-26T22:51:15.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.913+0000 7feb81e00640 1 -- 172.21.15.131:0/4284132583 learned_addr learned my addr 172.21.15.131:0/4284132583 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:15.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.913+0000 7feb81e00640 1 -- 172.21.15.131:0/4284132583 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb84140e80 msgr2=0x7feb841e79f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:15.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.913+0000 7feb81e00640 1 --2- 172.21.15.131:0/4284132583 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb84140e80 0x7feb841e79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:15.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.913+0000 7feb81e00640 1 -- 172.21.15.131:0/4284132583 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feb84141260 msgr2=0x7feb8415fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:15.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.913+0000 7feb82800640 1 --2- 172.21.15.131:0/4284132583 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feb84141260 0x7feb8415fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:15.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.913+0000 7feb81e00640 1 --2- 172.21.15.131:0/4284132583 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feb84141260 0x7feb8415fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:15.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.913+0000 7feb81e00640 1 -- 172.21.15.131:0/4284132583 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb841f41d0 con 0x7feb840d0020 2024-05-26T22:51:15.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.914+0000 7feb81e00640 1 --2- 172.21.15.131:0/4284132583 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb840d0020 0x7feb840b2f60 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7feb7c035040 tx=0x7feb7c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:15.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.914+0000 7feb7a000640 1 -- 172.21.15.131:0/4284132583 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb7c037040 con 0x7feb840d0020 2024-05-26T22:51:15.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.914+0000 7feb7a000640 1 -- 172.21.15.131:0/4284132583 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb7c030500 con 0x7feb840d0020 2024-05-26T22:51:15.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.914+0000 7feb83200640 1 -- 172.21.15.131:0/4284132583 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb8415ed10 con 0x7feb840d0020 2024-05-26T22:51:15.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.914+0000 7feb7a000640 1 -- 172.21.15.131:0/4284132583 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb7c0308d0 con 0x7feb840d0020 2024-05-26T22:51:15.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.914+0000 7feb83200640 1 -- 172.21.15.131:0/4284132583 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb8415f2d0 con 0x7feb840d0020 2024-05-26T22:51:15.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.917+0000 7feb7a000640 1 -- 172.21.15.131:0/4284132583 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7feb7c044020 con 0x7feb840d0020 2024-05-26T22:51:15.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.918+0000 7feb7a000640 1 --2- 172.21.15.131:0/4284132583 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7feb6006be30 0x7feb6006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:15.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.918+0000 7feb7a000640 1 -- 172.21.15.131:0/4284132583 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5853+0+0 (secure 0 0 0) 0x7feb7c0ab220 con 0x7feb840d0020 2024-05-26T22:51:15.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.918+0000 7feb81400640 1 --2- 172.21.15.131:0/4284132583 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7feb6006be30 0x7feb6006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:15.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.918+0000 7feb8ba05740 1 Processor -- start 2024-05-26T22:51:15.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.919+0000 7feb8ba05740 1 -- start start 2024-05-26T22:51:15.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.919+0000 7feb81400640 1 --2- 172.21.15.131:0/4284132583 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7feb6006be30 0x7feb6006e2f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7feb84140950 tx=0x7feb6c0094a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:15.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.919+0000 7feb8ba05740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df491ae420 0x55df49160db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:15.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.920+0000 7feb8ba05740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55df49191540 0x55df491908c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:15.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.920+0000 7feb8ba05740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df49190e00 0x55df49165c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:15.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.920+0000 7feb8ba05740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55df490c9960 con 0x55df49190e00 2024-05-26T22:51:15.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.920+0000 7feb8ba05740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55df490710d0 con 0x55df491ae420 2024-05-26T22:51:15.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.920+0000 7feb8ba05740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x55df490c5920 con 0x55df49191540 2024-05-26T22:51:15.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.920+0000 7feb82800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df49190e00 0x55df49165c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:15.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.920+0000 7feb82800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df49190e00 0x55df49165c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44708/0 (socket says 172.21.15.131:44708) 2024-05-26T22:51:15.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.920+0000 7feb82800640 1 -- 172.21.15.131:0/1649234638 learned_addr learned my addr 172.21.15.131:0/1649234638 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:15.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.920+0000 7feb81400640 1 --2- 172.21.15.131:0/1649234638 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55df49191540 0x55df491908c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:15.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.920+0000 7feb81e00640 1 --2- 172.21.15.131:0/1649234638 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df491ae420 0x55df49160db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:15.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.920+0000 7feb82800640 1 -- 172.21.15.131:0/1649234638 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55df49191540 msgr2=0x55df491908c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:15.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.920+0000 7feb82800640 1 --2- 172.21.15.131:0/1649234638 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55df49191540 0x55df491908c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:15.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.920+0000 7feb82800640 1 -- 172.21.15.131:0/1649234638 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df491ae420 msgr2=0x55df49160db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:15.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.920+0000 7feb82800640 1 --2- 172.21.15.131:0/1649234638 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df491ae420 0x55df49160db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:15.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.920+0000 7feb82800640 1 -- 172.21.15.131:0/1649234638 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df48ea77d0 con 0x55df49190e00 2024-05-26T22:51:15.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.921+0000 7feb82800640 1 --2- 172.21.15.131:0/1649234638 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df49190e00 0x55df49165c10 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7feb740016a0 tx=0x7feb74001b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1bc7b9a33be3b9b server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:15.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.921+0000 7feb72600640 1 -- 172.21.15.131:0/1649234638 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb74018070 con 0x55df49190e00 2024-05-26T22:51:15.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.921+0000 7feb72600640 1 -- 172.21.15.131:0/1649234638 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb74005cc0 con 0x55df49190e00 2024-05-26T22:51:15.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.921+0000 7feb72600640 1 -- 172.21.15.131:0/1649234638 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb740072f0 con 0x55df49190e00 2024-05-26T22:51:15.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.921+0000 7feb8ba05740 1 -- 172.21.15.131:0/1649234638 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df49190e00 msgr2=0x55df49165c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:15.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.921+0000 7feb8ba05740 1 --2- 172.21.15.131:0/1649234638 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df49190e00 0x55df49165c10 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7feb740016a0 tx=0x7feb74001b60 comp rx=0 tx=0).stop 2024-05-26T22:51:15.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.922+0000 7feb8ba05740 1 -- 172.21.15.131:0/1649234638 shutdown_connections 2024-05-26T22:51:15.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.922+0000 7feb8ba05740 1 --2- 172.21.15.131:0/1649234638 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55df49191540 0x55df491908c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:15.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.922+0000 7feb8ba05740 1 --2- 172.21.15.131:0/1649234638 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df49190e00 0x55df49165c10 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:15.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.922+0000 7feb8ba05740 1 --2- 172.21.15.131:0/1649234638 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df491ae420 0x55df49160db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:15.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.922+0000 7feb8ba05740 1 -- 172.21.15.131:0/1649234638 >> 172.21.15.131:0/1649234638 conn(0x55df49163020 msgr2=0x55df491ab8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:15.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.922+0000 7feb8ba05740 1 -- 172.21.15.131:0/1649234638 shutdown_connections 2024-05-26T22:51:15.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.922+0000 7feb8ba05740 1 -- 172.21.15.131:0/1649234638 wait complete. 2024-05-26T22:51:15.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.922+0000 7feb8ba05740 1 -- start start 2024-05-26T22:51:15.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.923+0000 7feb8ba05740 1 Processor -- start 2024-05-26T22:51:15.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.924+0000 7feb8ba05740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df49191540 0x55df4918c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:15.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.924+0000 7feb8ba05740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55df491ae420 0x55df491905a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:15.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.924+0000 7feb8ba05740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df491ac9f0 0x55df491ace70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:15.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.924+0000 7feb8ba05740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55df491651d0 con 0x55df49191540 2024-05-26T22:51:15.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.924+0000 7feb8ba05740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55df490844a0 con 0x55df491ac9f0 2024-05-26T22:51:15.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.924+0000 7feb8ba05740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x55df491645c0 con 0x55df491ae420 2024-05-26T22:51:15.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.924+0000 7feb8ba05740 10 client.0 _ll_register_callbacks cb 0x55df4908bc40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-26T22:51:15.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.924+0000 7feb81e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df49191540 0x55df4918c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:15.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.924+0000 7feb81e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df49191540 0x55df4918c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44712/0 (socket says 172.21.15.131:44712) 2024-05-26T22:51:15.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.924+0000 7feb81e00640 1 -- 172.21.15.131:0/1620025213 learned_addr learned my addr 172.21.15.131:0/1620025213 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:15.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.924+0000 7feb81e00640 1 -- 172.21.15.131:0/1620025213 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55df491ae420 msgr2=0x55df491905a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:15.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.924+0000 7feb81e00640 1 --2- 172.21.15.131:0/1620025213 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55df491ae420 0x55df491905a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:15.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.924+0000 7feb81e00640 1 -- 172.21.15.131:0/1620025213 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df491ac9f0 msgr2=0x55df491ace70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:15.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.924+0000 7feb81e00640 1 --2- 172.21.15.131:0/1620025213 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df491ac9f0 0x55df491ace70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:15.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.924+0000 7feb81e00640 1 -- 172.21.15.131:0/1620025213 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df48d9d0f0 con 0x55df49191540 2024-05-26T22:51:15.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.924+0000 7feb81e00640 1 --2- 172.21.15.131:0/1620025213 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df49191540 0x55df4918c440 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7feb7c0a9590 tx=0x7feb7c078d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=370a0d520db98b75 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:15.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.925+0000 7feb66a00640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-26T22:51:15.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.925+0000 7feb66a00640 1 -- 172.21.15.131:0/1620025213 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb7c07d750 con 0x55df49191540 2024-05-26T22:51:15.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.925+0000 7feb66a00640 1 -- 172.21.15.131:0/1620025213 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(47..47 src has 1..47) ==== 5853+0+0 (secure 0 0 0) 0x7feb7c091170 con 0x55df49191540 2024-05-26T22:51:15.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.925+0000 7feb8ba05740 1 -- 172.21.15.131:0/1620025213 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb74005cc0 con 0x55df49191540 2024-05-26T22:51:15.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.925+0000 7feb66a00640 1 -- 172.21.15.131:0/1620025213 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7feb7c0a9050 con 0x55df49191540 2024-05-26T22:51:15.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.925+0000 7feb8ba05740 10 client.6099 fetch_fsmap learned FSMap version 40 2024-05-26T22:51:15.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.925+0000 7feb8ba05740 1 -- 172.21.15.131:0/1620025213 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=2+}) -- 0x55df48f07f60 con 0x55df49191540 2024-05-26T22:51:15.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.925+0000 7feb66a00640 1 -- 172.21.15.131:0/1620025213 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb74005cc0 con 0x55df49191540 2024-05-26T22:51:15.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.925+0000 7feb66a00640 1 -- 172.21.15.131:0/1620025213 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 40) ==== 3542+0+0 (secure 0 0 0) 0x7feb7c03a070 con 0x55df49191540 2024-05-26T22:51:15.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.925+0000 7feb8ba05740 10 client.6099 fetch_fsmap finished waiting for FSMap version 40 2024-05-26T22:51:15.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.925+0000 7feb8ba05740 10 client.6099 resolve_mds: resolved 3:1 to role '3:1' aka daemon mds.a 2024-05-26T22:51:15.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.925+0000 7feb8ba05740 20 client.6099 populate_metadata read hostname 'smithi131' 2024-05-26T22:51:15.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.925+0000 7feb8ba05740 1 --2- 172.21.15.131:0/1620025213 >> [v2:172.21.15.131:6835/265040431,v1:172.21.15.131:6837/265040431] conn(0x55df491922a0 0x55df49192720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:15.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.925+0000 7feb8ba05740 4 client.6099 mds_command: new command op to 4661 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-26T22:51:15.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.925+0000 7feb8ba05740 1 -- 172.21.15.131:0/1620025213 --> [v2:172.21.15.131:6835/265040431,v1:172.21.15.131:6837/265040431] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55df490d7090 con 0x55df491922a0 2024-05-26T22:51:15.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.926+0000 7feb81400640 1 --2- 172.21.15.131:0/1620025213 >> [v2:172.21.15.131:6835/265040431,v1:172.21.15.131:6837/265040431] conn(0x55df491922a0 0x55df49192720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:15.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.926+0000 7feb81400640 1 --2- 172.21.15.131:0/1620025213 >> [v2:172.21.15.131:6835/265040431,v1:172.21.15.131:6837/265040431] conn(0x55df491922a0 0x55df49192720 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d66a9ee60a250ce9 server_cookie=84a85e4100e8d568 in_seq=0 out_seq=0 2024-05-26T22:51:15.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.926+0000 7feb66a00640 10 client.6099 ms_handle_connect on v2:172.21.15.131:6835/265040431 2024-05-26T22:51:15.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.927+0000 7feb66a00640 1 -- 172.21.15.131:0/1620025213 <== mds.1 v2:172.21.15.131:6835/265040431 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55df490d7090 con 0x55df491922a0 2024-05-26T22:51:15.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.927+0000 7feb66a00640 10 client.6099 handle_command_reply: tid=0 2024-05-26T22:51:15.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.927+0000 7feb8ba05740 1 client.6099 shutdown 2024-05-26T22:51:15.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.927+0000 7feb8ba05740 1 -- 172.21.15.131:0/1620025213 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df49191540 msgr2=0x55df4918c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:15.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.927+0000 7feb8ba05740 1 --2- 172.21.15.131:0/1620025213 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df49191540 0x55df4918c440 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7feb7c0a9590 tx=0x7feb7c078d00 comp rx=0 tx=0).stop 2024-05-26T22:51:15.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.927+0000 7feb8ba05740 1 -- 172.21.15.131:0/1620025213 shutdown_connections 2024-05-26T22:51:15.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.927+0000 7feb8ba05740 1 --2- 172.21.15.131:0/1620025213 >> [v2:172.21.15.131:6835/265040431,v1:172.21.15.131:6837/265040431] conn(0x55df491922a0 0x55df49192720 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:15.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.927+0000 7feb8ba05740 1 --2- 172.21.15.131:0/1620025213 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55df491ae420 0x55df491905a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:15.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.927+0000 7feb8ba05740 1 --2- 172.21.15.131:0/1620025213 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df491ac9f0 0x55df491ace70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:15.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.927+0000 7feb8ba05740 1 --2- 172.21.15.131:0/1620025213 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df49191540 0x55df4918c440 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:15.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.927+0000 7feb66a00640 0 client.6099 ms_handle_reset on v2:172.21.15.131:6835/265040431 2024-05-26T22:51:15.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.927+0000 7feb8ba05740 1 -- 172.21.15.131:0/1620025213 >> 172.21.15.131:0/1620025213 conn(0x55df49190e00 msgr2=0x55df491ab870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:15.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.928+0000 7feb8ba05740 1 -- 172.21.15.131:0/1620025213 shutdown_connections 2024-05-26T22:51:15.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.928+0000 7feb8ba05740 1 -- 172.21.15.131:0/1620025213 wait complete. 2024-05-26T22:51:15.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.928+0000 7feb8ba05740 20 client.6099 trim_cache size 0 max 16384 2024-05-26T22:51:15.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.946+0000 7feb8ba05740 1 Processor -- start 2024-05-26T22:51:15.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.946+0000 7feb8ba05740 1 -- start start 2024-05-26T22:51:15.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.947+0000 7feb8ba05740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df49191540 0x55df491a8740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:15.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.947+0000 7feb8ba05740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55df491ac9f0 0x55df491664c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:15.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.947+0000 7feb8ba05740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df491ae420 0x55df49166a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:15.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.947+0000 7feb8ba05740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7feb842008e0 con 0x55df49191540 2024-05-26T22:51:15.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.947+0000 7feb8ba05740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7feb841ffae0 con 0x55df491ae420 2024-05-26T22:51:15.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.947+0000 7feb8ba05740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7feb842001b0 con 0x55df491ac9f0 2024-05-26T22:51:15.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.947+0000 7feb81e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df49191540 0x55df491a8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:15.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.947+0000 7feb81e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df49191540 0x55df491a8740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44724/0 (socket says 172.21.15.131:44724) 2024-05-26T22:51:15.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.947+0000 7feb81e00640 1 -- 172.21.15.131:0/1597094494 learned_addr learned my addr 172.21.15.131:0/1597094494 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:15.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.947+0000 7feb81e00640 1 -- 172.21.15.131:0/1597094494 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55df491ac9f0 msgr2=0x55df491664c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:15.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.947+0000 7feb81e00640 1 --2- 172.21.15.131:0/1597094494 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55df491ac9f0 0x55df491664c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:15.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.947+0000 7feb81e00640 1 -- 172.21.15.131:0/1597094494 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df491ae420 msgr2=0x55df49166a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:15.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.947+0000 7feb81e00640 1 --2- 172.21.15.131:0/1597094494 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df491ae420 0x55df49166a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:15.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.947+0000 7feb81e00640 1 -- 172.21.15.131:0/1597094494 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df48d761b0 con 0x55df49191540 2024-05-26T22:51:15.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.948+0000 7feb81e00640 1 --2- 172.21.15.131:0/1597094494 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df49191540 0x55df491a8740 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7feb7c081ff0 tx=0x7feb7c008000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f323c4a796157c0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:15.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.948+0000 7feb71c00640 1 -- 172.21.15.131:0/1597094494 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb7c07d750 con 0x55df49191540 2024-05-26T22:51:15.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.948+0000 7feb71c00640 1 -- 172.21.15.131:0/1597094494 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb7c0750f0 con 0x55df49191540 2024-05-26T22:51:15.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.948+0000 7feb71c00640 1 -- 172.21.15.131:0/1597094494 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb7c091170 con 0x55df49191540 2024-05-26T22:51:15.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.948+0000 7feb8ba05740 1 -- 172.21.15.131:0/1597094494 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df49191540 msgr2=0x55df491a8740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:15.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.948+0000 7feb8ba05740 1 --2- 172.21.15.131:0/1597094494 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df49191540 0x55df491a8740 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7feb7c081ff0 tx=0x7feb7c008000 comp rx=0 tx=0).stop 2024-05-26T22:51:15.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.949+0000 7feb8ba05740 1 -- 172.21.15.131:0/1597094494 shutdown_connections 2024-05-26T22:51:15.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.949+0000 7feb8ba05740 1 --2- 172.21.15.131:0/1597094494 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55df491ac9f0 0x55df491664c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:15.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.949+0000 7feb8ba05740 1 --2- 172.21.15.131:0/1597094494 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df491ae420 0x55df49166a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:15.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.949+0000 7feb8ba05740 1 --2- 172.21.15.131:0/1597094494 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df49191540 0x55df491a8740 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:15.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.949+0000 7feb8ba05740 1 -- 172.21.15.131:0/1597094494 >> 172.21.15.131:0/1597094494 conn(0x55df49190e00 msgr2=0x55df49192680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:15.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.949+0000 7feb8ba05740 1 -- 172.21.15.131:0/1597094494 shutdown_connections 2024-05-26T22:51:15.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.949+0000 7feb8ba05740 1 -- 172.21.15.131:0/1597094494 wait complete. 2024-05-26T22:51:15.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.949+0000 7feb8ba05740 1 -- start start 2024-05-26T22:51:15.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.950+0000 7feb8ba05740 1 Processor -- start 2024-05-26T22:51:15.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.951+0000 7feb8ba05740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df49191540 0x55df49231bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:15.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.951+0000 7feb8ba05740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55df491ac9f0 0x55df49164430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:15.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.951+0000 7feb8ba05740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df491ae420 0x55df49164970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:15.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.951+0000 7feb8ba05740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7feb841feb80 con 0x55df49191540 2024-05-26T22:51:15.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.951+0000 7feb8ba05740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55df491615f0 con 0x55df491ae420 2024-05-26T22:51:15.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.951+0000 7feb8ba05740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x55df49193c30 con 0x55df491ac9f0 2024-05-26T22:51:15.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.951+0000 7feb8ba05740 10 client.0 _ll_register_callbacks cb 0x7feb7c009400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-26T22:51:15.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.951+0000 7feb81e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df49191540 0x55df49231bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:15.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.951+0000 7feb81e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df49191540 0x55df49231bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44728/0 (socket says 172.21.15.131:44728) 2024-05-26T22:51:15.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.951+0000 7feb81e00640 1 -- 172.21.15.131:0/2011670235 learned_addr learned my addr 172.21.15.131:0/2011670235 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:15.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.951+0000 7feb81e00640 1 -- 172.21.15.131:0/2011670235 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55df491ac9f0 msgr2=0x55df49164430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:15.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.951+0000 7feb81e00640 1 --2- 172.21.15.131:0/2011670235 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55df491ac9f0 0x55df49164430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:15.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.951+0000 7feb82800640 1 --2- 172.21.15.131:0/2011670235 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df491ae420 0x55df49164970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:15.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.951+0000 7feb81e00640 1 -- 172.21.15.131:0/2011670235 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df491ae420 msgr2=0x55df49164970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:15.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.951+0000 7feb81e00640 1 --2- 172.21.15.131:0/2011670235 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df491ae420 0x55df49164970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:15.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.951+0000 7feb81e00640 1 -- 172.21.15.131:0/2011670235 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df491add40 con 0x55df49191540 2024-05-26T22:51:15.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.952+0000 7feb81e00640 1 --2- 172.21.15.131:0/2011670235 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df49191540 0x55df49231bd0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7feb7c0794b0 tx=0x7feb7c0a94f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18407b1b81a302fa server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:15.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.952+0000 7feb66a00640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-26T22:51:15.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.952+0000 7feb66a00640 1 -- 172.21.15.131:0/2011670235 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb7c07d750 con 0x55df49191540 2024-05-26T22:51:15.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.952+0000 7feb66a00640 1 -- 172.21.15.131:0/2011670235 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(47..47 src has 1..47) ==== 5853+0+0 (secure 0 0 0) 0x7feb7c098370 con 0x55df49191540 2024-05-26T22:51:15.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.952+0000 7feb8ba05740 1 -- 172.21.15.131:0/2011670235 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb7c0750f0 con 0x55df49191540 2024-05-26T22:51:15.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.952+0000 7feb66a00640 1 -- 172.21.15.131:0/2011670235 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7feb7c0a91f0 con 0x55df49191540 2024-05-26T22:51:15.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.952+0000 7feb8ba05740 10 client.6105 fetch_fsmap learned FSMap version 40 2024-05-26T22:51:15.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.952+0000 7feb8ba05740 1 -- 172.21.15.131:0/2011670235 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=2+}) -- 0x55df491add40 con 0x55df49191540 2024-05-26T22:51:15.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.952+0000 7feb66a00640 1 -- 172.21.15.131:0/2011670235 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb7c0750f0 con 0x55df49191540 2024-05-26T22:51:15.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.952+0000 7feb66a00640 1 -- 172.21.15.131:0/2011670235 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 40) ==== 3542+0+0 (secure 0 0 0) 0x7feb7c03a070 con 0x55df49191540 2024-05-26T22:51:15.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.952+0000 7feb8ba05740 10 client.6105 fetch_fsmap finished waiting for FSMap version 40 2024-05-26T22:51:15.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.952+0000 7feb8ba05740 10 client.6105 resolve_mds: resolved 3:1 to role '3:1' aka daemon mds.a 2024-05-26T22:51:15.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.952+0000 7feb8ba05740 20 client.6105 populate_metadata read hostname 'smithi131' 2024-05-26T22:51:15.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.953+0000 7feb8ba05740 1 --2- 172.21.15.131:0/2011670235 >> [v2:172.21.15.131:6835/265040431,v1:172.21.15.131:6837/265040431] conn(0x55df49194b40 0x55df4918cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:15.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.953+0000 7feb8ba05740 4 client.6105 mds_command: new command op to 4661 tid=0 multi_id=0 [{"prefix": "perf dump"}] 2024-05-26T22:51:15.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.953+0000 7feb8ba05740 1 -- 172.21.15.131:0/2011670235 --> [v2:172.21.15.131:6835/265040431,v1:172.21.15.131:6837/265040431] -- command(tid 0: {"prefix": "perf dump"}) -- 0x55df490d13e0 con 0x55df49194b40 2024-05-26T22:51:15.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.953+0000 7feb81400640 1 --2- 172.21.15.131:0/2011670235 >> [v2:172.21.15.131:6835/265040431,v1:172.21.15.131:6837/265040431] conn(0x55df49194b40 0x55df4918cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:15.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.953+0000 7feb81400640 1 --2- 172.21.15.131:0/2011670235 >> [v2:172.21.15.131:6835/265040431,v1:172.21.15.131:6837/265040431] conn(0x55df49194b40 0x55df4918cff0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ff273e56d6426202 server_cookie=c9421097fb99f2de in_seq=0 out_seq=0 2024-05-26T22:51:15.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.953+0000 7feb66a00640 10 client.6105 ms_handle_connect on v2:172.21.15.131:6835/265040431 2024-05-26T22:51:15.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.954+0000 7feb66a00640 1 -- 172.21.15.131:0/2011670235 <== mds.1 v2:172.21.15.131:6835/265040431 1 ==== command_reply(tid 0: 0 ) ==== 8+0+17734 (crc 0 0 0) 0x55df490d13e0 con 0x55df49194b40 2024-05-26T22:51:15.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.954+0000 7feb66a00640 10 client.6105 handle_command_reply: tid=0 2024-05-26T22:51:15.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.954+0000 7feb8ba05740 1 client.6105 shutdown 2024-05-26T22:51:15.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.954+0000 7feb8ba05740 1 -- 172.21.15.131:0/2011670235 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df49191540 msgr2=0x55df49231bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:15.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.954+0000 7feb8ba05740 1 --2- 172.21.15.131:0/2011670235 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df49191540 0x55df49231bd0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7feb7c0794b0 tx=0x7feb7c0a94f0 comp rx=0 tx=0).stop 2024-05-26T22:51:15.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.954+0000 7feb8ba05740 1 -- 172.21.15.131:0/2011670235 shutdown_connections 2024-05-26T22:51:15.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.954+0000 7feb8ba05740 1 --2- 172.21.15.131:0/2011670235 >> [v2:172.21.15.131:6835/265040431,v1:172.21.15.131:6837/265040431] conn(0x55df49194b40 0x55df4918cff0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:15.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.954+0000 7feb8ba05740 1 --2- 172.21.15.131:0/2011670235 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55df491ac9f0 0x55df49164430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:15.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.954+0000 7feb66a00640 0 client.6105 ms_handle_reset on v2:172.21.15.131:6835/265040431 2024-05-26T22:51:15.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.954+0000 7feb8ba05740 1 --2- 172.21.15.131:0/2011670235 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df491ae420 0x55df49164970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:15.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.954+0000 7feb8ba05740 1 --2- 172.21.15.131:0/2011670235 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df49191540 0x55df49231bd0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:15.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.954+0000 7feb8ba05740 1 -- 172.21.15.131:0/2011670235 >> 172.21.15.131:0/2011670235 conn(0x55df49190e00 msgr2=0x55df4918fbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:15.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.954+0000 7feb8ba05740 1 -- 172.21.15.131:0/2011670235 shutdown_connections 2024-05-26T22:51:15.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.954+0000 7feb8ba05740 1 -- 172.21.15.131:0/2011670235 wait complete. 2024-05-26T22:51:15.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.955+0000 7feb8ba05740 20 client.6105 trim_cache size 0 max 16384 2024-05-26T22:51:15.959 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-26T22:51:15.959 INFO:teuthology.orchestra.run.smithi131.stdout: "AsyncMessenger::Worker-0": { 2024-05-26T22:51:15.960 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_messages": 63, 2024-05-26T22:51:15.960 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages": 67, 2024-05-26T22:51:15.960 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_bytes": 90209, 2024-05-26T22:51:15.960 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_bytes": 141702, 2024-05-26T22:51:15.960 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_created_connections": 8, 2024-05-26T22:51:15.960 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_active_connections": 3, 2024-05-26T22:51:15.960 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_total_time": 0.026801960, 2024-05-26T22:51:15.960 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_send_time": 0.008491690, 2024-05-26T22:51:15.960 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_recv_time": 0.011427991, 2024-05-26T22:51:15.960 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_fast_dispatch_time": 0.002358434, 2024-05-26T22:51:15.960 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages_queue_lat": { 2024-05-26T22:51:15.960 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 66, 2024-05-26T22:51:15.961 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.021299067, 2024-05-26T22:51:15.961 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000322713 2024-05-26T22:51:15.961 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:15.961 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_handle_ack_lat": { 2024-05-26T22:51:15.961 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 24, 2024-05-26T22:51:15.961 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000009316, 2024-05-26T22:51:15.961 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000388 2024-05-26T22:51:15.961 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:15.961 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_encrypted_bytes": 1232, 2024-05-26T22:51:15.961 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_encrypted_bytes": 448 2024-05-26T22:51:15.961 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:15.961 INFO:teuthology.orchestra.run.smithi131.stdout: "AsyncMessenger::Worker-1": { 2024-05-26T22:51:15.961 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_messages": 70, 2024-05-26T22:51:15.962 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages": 60, 2024-05-26T22:51:15.962 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_bytes": 138563, 2024-05-26T22:51:15.962 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_bytes": 29130, 2024-05-26T22:51:15.962 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_created_connections": 6, 2024-05-26T22:51:15.962 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_active_connections": 4, 2024-05-26T22:51:15.962 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_total_time": 0.015298461, 2024-05-26T22:51:15.962 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_send_time": 0.003071259, 2024-05-26T22:51:15.962 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_recv_time": 0.014480715, 2024-05-26T22:51:15.962 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_fast_dispatch_time": 0.001288769, 2024-05-26T22:51:15.962 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages_queue_lat": { 2024-05-26T22:51:15.962 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 62, 2024-05-26T22:51:15.962 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.027533013, 2024-05-26T22:51:15.963 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000444080 2024-05-26T22:51:15.963 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:15.963 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_handle_ack_lat": { 2024-05-26T22:51:15.963 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 7, 2024-05-26T22:51:15.963 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000002214, 2024-05-26T22:51:15.963 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000316 2024-05-26T22:51:15.963 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:15.963 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_encrypted_bytes": 119408, 2024-05-26T22:51:15.963 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_encrypted_bytes": 13536 2024-05-26T22:51:15.963 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:15.963 INFO:teuthology.orchestra.run.smithi131.stdout: "AsyncMessenger::Worker-2": { 2024-05-26T22:51:15.963 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_messages": 23, 2024-05-26T22:51:15.963 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages": 38, 2024-05-26T22:51:15.963 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_bytes": 7835, 2024-05-26T22:51:15.964 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_bytes": 150431, 2024-05-26T22:51:15.964 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_created_connections": 11, 2024-05-26T22:51:15.964 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_active_connections": 7, 2024-05-26T22:51:15.964 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_total_time": 0.012143415, 2024-05-26T22:51:15.964 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_send_time": 0.003367323, 2024-05-26T22:51:15.964 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_recv_time": 0.007404658, 2024-05-26T22:51:15.964 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_fast_dispatch_time": 0.000321420, 2024-05-26T22:51:15.964 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages_queue_lat": { 2024-05-26T22:51:15.964 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 35, 2024-05-26T22:51:15.964 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.006444733, 2024-05-26T22:51:15.964 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000184135 2024-05-26T22:51:15.964 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:15.964 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_handle_ack_lat": { 2024-05-26T22:51:15.965 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 9, 2024-05-26T22:51:15.965 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000003226, 2024-05-26T22:51:15.965 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000358 2024-05-26T22:51:15.965 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:15.965 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-26T22:51:15.965 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_encrypted_bytes": 96208 2024-05-26T22:51:15.965 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:15.965 INFO:teuthology.orchestra.run.smithi131.stdout: "cct": { 2024-05-26T22:51:15.965 INFO:teuthology.orchestra.run.smithi131.stdout: "total_workers": 1, 2024-05-26T22:51:15.965 INFO:teuthology.orchestra.run.smithi131.stdout: "unhealthy_workers": 0 2024-05-26T22:51:15.965 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:15.965 INFO:teuthology.orchestra.run.smithi131.stdout: "finisher-MDSRank": { 2024-05-26T22:51:15.965 INFO:teuthology.orchestra.run.smithi131.stdout: "queue_len": 0, 2024-05-26T22:51:15.965 INFO:teuthology.orchestra.run.smithi131.stdout: "complete_latency": { 2024-05-26T22:51:15.966 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 92, 2024-05-26T22:51:15.966 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.038832528, 2024-05-26T22:51:15.966 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000422092 2024-05-26T22:51:15.966 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:51:15.966 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:15.966 INFO:teuthology.orchestra.run.smithi131.stdout: "finisher-PurgeQueue": { 2024-05-26T22:51:15.966 INFO:teuthology.orchestra.run.smithi131.stdout: "queue_len": 0, 2024-05-26T22:51:15.966 INFO:teuthology.orchestra.run.smithi131.stdout: "complete_latency": { 2024-05-26T22:51:15.967 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 4, 2024-05-26T22:51:15.967 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000107641, 2024-05-26T22:51:15.967 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000026910 2024-05-26T22:51:15.967 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:51:15.967 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:15.967 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": { 2024-05-26T22:51:15.967 INFO:teuthology.orchestra.run.smithi131.stdout: "request": 0, 2024-05-26T22:51:15.967 INFO:teuthology.orchestra.run.smithi131.stdout: "reply": 0, 2024-05-26T22:51:15.967 INFO:teuthology.orchestra.run.smithi131.stdout: "reply_latency": { 2024-05-26T22:51:15.967 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:51:15.967 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:51:15.967 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:51:15.967 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:15.968 INFO:teuthology.orchestra.run.smithi131.stdout: "slow_reply": 0, 2024-05-26T22:51:15.968 INFO:teuthology.orchestra.run.smithi131.stdout: "forward": 0, 2024-05-26T22:51:15.968 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_fetch_complete": 11, 2024-05-26T22:51:15.968 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_fetch_keys": 10, 2024-05-26T22:51:15.968 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_commit": 0, 2024-05-26T22:51:15.968 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_split": 0, 2024-05-26T22:51:15.968 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_merge": 0, 2024-05-26T22:51:15.968 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes": 85, 2024-05-26T22:51:15.968 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_top": 7, 2024-05-26T22:51:15.968 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_bottom": 78, 2024-05-26T22:51:15.968 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_pin_tail": 0, 2024-05-26T22:51:15.968 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_pinned": 85, 2024-05-26T22:51:15.968 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_expired": 0, 2024-05-26T22:51:15.968 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_with_caps": 0, 2024-05-26T22:51:15.969 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": 0, 2024-05-26T22:51:15.969 INFO:teuthology.orchestra.run.smithi131.stdout: "subtrees": 5, 2024-05-26T22:51:15.969 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse": 0, 2024-05-26T22:51:15.969 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_hit": 0, 2024-05-26T22:51:15.969 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_forward": 0, 2024-05-26T22:51:15.969 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_discover": 0, 2024-05-26T22:51:15.969 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_dir_fetch": 0, 2024-05-26T22:51:15.969 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_remote_ino": 0, 2024-05-26T22:51:15.969 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_lock": 0, 2024-05-26T22:51:15.969 INFO:teuthology.orchestra.run.smithi131.stdout: "load_cent": 0, 2024-05-26T22:51:15.969 INFO:teuthology.orchestra.run.smithi131.stdout: "q": 0, 2024-05-26T22:51:15.969 INFO:teuthology.orchestra.run.smithi131.stdout: "exported": 0, 2024-05-26T22:51:15.969 INFO:teuthology.orchestra.run.smithi131.stdout: "exported_inodes": 0, 2024-05-26T22:51:15.970 INFO:teuthology.orchestra.run.smithi131.stdout: "imported": 0, 2024-05-26T22:51:15.970 INFO:teuthology.orchestra.run.smithi131.stdout: "imported_inodes": 0, 2024-05-26T22:51:15.970 INFO:teuthology.orchestra.run.smithi131.stdout: "openino_dir_fetch": 0, 2024-05-26T22:51:15.970 INFO:teuthology.orchestra.run.smithi131.stdout: "openino_backtrace_fetch": 0, 2024-05-26T22:51:15.970 INFO:teuthology.orchestra.run.smithi131.stdout: "openino_peer_discover": 0, 2024-05-26T22:51:15.970 INFO:teuthology.orchestra.run.smithi131.stdout: "root_rfiles": 100, 2024-05-26T22:51:15.970 INFO:teuthology.orchestra.run.smithi131.stdout: "root_rbytes": 500, 2024-05-26T22:51:15.970 INFO:teuthology.orchestra.run.smithi131.stdout: "root_rsnaps": 0, 2024-05-26T22:51:15.970 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_backtrace_fetch": 0, 2024-05-26T22:51:15.970 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_set_tag": 0, 2024-05-26T22:51:15.970 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_backtrace_repaired": 0, 2024-05-26T22:51:15.970 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_inotable_repaired": 0, 2024-05-26T22:51:15.970 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_dir_inodes": 0, 2024-05-26T22:51:15.971 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_dir_base_inodes": 0, 2024-05-26T22:51:15.971 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_dirfrag_rstats": 0, 2024-05-26T22:51:15.971 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_file_inodes": 0, 2024-05-26T22:51:15.971 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_inode_file_caps": 0, 2024-05-26T22:51:15.971 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_revoke": 0, 2024-05-26T22:51:15.971 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_grant": 0, 2024-05-26T22:51:15.971 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_trunc": 0, 2024-05-26T22:51:15.971 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-26T22:51:15.971 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-26T22:51:15.971 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_caps": 0, 2024-05-26T22:51:15.971 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_caps_dirty": 0, 2024-05-26T22:51:15.971 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_cap_release": 0, 2024-05-26T22:51:15.972 INFO:teuthology.orchestra.run.smithi131.stdout: "process_request_cap_release": 0 2024-05-26T22:51:15.972 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:15.972 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_cache": { 2024-05-26T22:51:15.972 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_update": 0, 2024-05-26T22:51:15.972 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_update_receipt": 0, 2024-05-26T22:51:15.972 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_try_discover": 0, 2024-05-26T22:51:15.972 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_send_discover": 0, 2024-05-26T22:51:15.972 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_handle_discover": 0, 2024-05-26T22:51:15.972 INFO:teuthology.orchestra.run.smithi131.stdout: "num_strays": 0, 2024-05-26T22:51:15.972 INFO:teuthology.orchestra.run.smithi131.stdout: "num_strays_delayed": 0, 2024-05-26T22:51:15.972 INFO:teuthology.orchestra.run.smithi131.stdout: "num_strays_enqueuing": 0, 2024-05-26T22:51:15.972 INFO:teuthology.orchestra.run.smithi131.stdout: "strays_created": 0, 2024-05-26T22:51:15.972 INFO:teuthology.orchestra.run.smithi131.stdout: "strays_enqueued": 0, 2024-05-26T22:51:15.972 INFO:teuthology.orchestra.run.smithi131.stdout: "strays_reintegrated": 0, 2024-05-26T22:51:15.973 INFO:teuthology.orchestra.run.smithi131.stdout: "strays_migrated": 0, 2024-05-26T22:51:15.973 INFO:teuthology.orchestra.run.smithi131.stdout: "num_recovering_processing": 0, 2024-05-26T22:51:15.973 INFO:teuthology.orchestra.run.smithi131.stdout: "num_recovering_enqueued": 0, 2024-05-26T22:51:15.973 INFO:teuthology.orchestra.run.smithi131.stdout: "num_recovering_prioritized": 0, 2024-05-26T22:51:15.973 INFO:teuthology.orchestra.run.smithi131.stdout: "recovery_started": 25, 2024-05-26T22:51:15.973 INFO:teuthology.orchestra.run.smithi131.stdout: "recovery_completed": 25, 2024-05-26T22:51:15.973 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_quiesce_path": 0, 2024-05-26T22:51:15.973 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_quiesce_inode": 0, 2024-05-26T22:51:15.973 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_enqueue_scrub": 0, 2024-05-26T22:51:15.973 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_exportdir": 0, 2024-05-26T22:51:15.973 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_flush": 0, 2024-05-26T22:51:15.973 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_fragmentdir": 0, 2024-05-26T22:51:15.973 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_fragstats": 0, 2024-05-26T22:51:15.974 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_inodestats": 0 2024-05-26T22:51:15.974 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:15.974 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_log": { 2024-05-26T22:51:15.974 INFO:teuthology.orchestra.run.smithi131.stdout: "evlrg": 0, 2024-05-26T22:51:15.974 INFO:teuthology.orchestra.run.smithi131.stdout: "evadd": 54, 2024-05-26T22:51:15.974 INFO:teuthology.orchestra.run.smithi131.stdout: "evex": 0, 2024-05-26T22:51:15.974 INFO:teuthology.orchestra.run.smithi131.stdout: "evtrm": 0, 2024-05-26T22:51:15.974 INFO:teuthology.orchestra.run.smithi131.stdout: "ev": 59, 2024-05-26T22:51:15.974 INFO:teuthology.orchestra.run.smithi131.stdout: "evexg": 0, 2024-05-26T22:51:15.974 INFO:teuthology.orchestra.run.smithi131.stdout: "evexd": 0, 2024-05-26T22:51:15.974 INFO:teuthology.orchestra.run.smithi131.stdout: "segadd": 0, 2024-05-26T22:51:15.974 INFO:teuthology.orchestra.run.smithi131.stdout: "segex": 0, 2024-05-26T22:51:15.974 INFO:teuthology.orchestra.run.smithi131.stdout: "segtrm": 0, 2024-05-26T22:51:15.975 INFO:teuthology.orchestra.run.smithi131.stdout: "seg": 2, 2024-05-26T22:51:15.975 INFO:teuthology.orchestra.run.smithi131.stdout: "segmjr": 2, 2024-05-26T22:51:15.975 INFO:teuthology.orchestra.run.smithi131.stdout: "segexg": 0, 2024-05-26T22:51:15.975 INFO:teuthology.orchestra.run.smithi131.stdout: "segexd": 0, 2024-05-26T22:51:15.975 INFO:teuthology.orchestra.run.smithi131.stdout: "expos": 4194304, 2024-05-26T22:51:15.975 INFO:teuthology.orchestra.run.smithi131.stdout: "wrpos": 4372106, 2024-05-26T22:51:15.975 INFO:teuthology.orchestra.run.smithi131.stdout: "rdpos": 4243416, 2024-05-26T22:51:15.975 INFO:teuthology.orchestra.run.smithi131.stdout: "jlat": { 2024-05-26T22:51:15.975 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 28, 2024-05-26T22:51:15.976 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.737348135, 2024-05-26T22:51:15.976 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.026333861 2024-05-26T22:51:15.976 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:15.976 INFO:teuthology.orchestra.run.smithi131.stdout: "replayed": 5 2024-05-26T22:51:15.976 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:15.976 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_mem": { 2024-05-26T22:51:15.976 INFO:teuthology.orchestra.run.smithi131.stdout: "ino": 88, 2024-05-26T22:51:15.976 INFO:teuthology.orchestra.run.smithi131.stdout: "ino+": 88, 2024-05-26T22:51:15.976 INFO:teuthology.orchestra.run.smithi131.stdout: "ino-": 0, 2024-05-26T22:51:15.976 INFO:teuthology.orchestra.run.smithi131.stdout: "dir": 52, 2024-05-26T22:51:15.976 INFO:teuthology.orchestra.run.smithi131.stdout: "dir+": 52, 2024-05-26T22:51:15.976 INFO:teuthology.orchestra.run.smithi131.stdout: "dir-": 0, 2024-05-26T22:51:15.976 INFO:teuthology.orchestra.run.smithi131.stdout: "dn": 85, 2024-05-26T22:51:15.977 INFO:teuthology.orchestra.run.smithi131.stdout: "dn+": 85, 2024-05-26T22:51:15.977 INFO:teuthology.orchestra.run.smithi131.stdout: "dn-": 0, 2024-05-26T22:51:15.977 INFO:teuthology.orchestra.run.smithi131.stdout: "cap": 0, 2024-05-26T22:51:15.977 INFO:teuthology.orchestra.run.smithi131.stdout: "cap+": 0, 2024-05-26T22:51:15.977 INFO:teuthology.orchestra.run.smithi131.stdout: "cap-": 0, 2024-05-26T22:51:15.977 INFO:teuthology.orchestra.run.smithi131.stdout: "rss": 48384, 2024-05-26T22:51:15.977 INFO:teuthology.orchestra.run.smithi131.stdout: "heap": 239980 2024-05-26T22:51:15.977 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:15.977 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_server": { 2024-05-26T22:51:15.977 INFO:teuthology.orchestra.run.smithi131.stdout: "dispatch_client_request": 0, 2024-05-26T22:51:15.977 INFO:teuthology.orchestra.run.smithi131.stdout: "dispatch_server_request": 0, 2024-05-26T22:51:15.977 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_request": 0, 2024-05-26T22:51:15.977 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_session": 1, 2024-05-26T22:51:15.978 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_peer_request": 0, 2024-05-26T22:51:15.978 INFO:teuthology.orchestra.run.smithi131.stdout: "req_create_latency": { 2024-05-26T22:51:15.978 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:51:15.978 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:51:15.978 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:51:15.978 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:15.978 INFO:teuthology.orchestra.run.smithi131.stdout: "req_getattr_latency": { 2024-05-26T22:51:15.978 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:51:15.978 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:51:15.978 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:51:15.978 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:15.978 INFO:teuthology.orchestra.run.smithi131.stdout: "req_getfilelock_latency": { 2024-05-26T22:51:15.978 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:51:15.979 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:51:15.979 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:51:15.979 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:15.979 INFO:teuthology.orchestra.run.smithi131.stdout: "req_link_latency": { 2024-05-26T22:51:15.979 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:51:15.979 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:51:15.979 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:51:15.979 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:15.979 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookup_latency": { 2024-05-26T22:51:15.979 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:51:15.979 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:51:15.979 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:51:15.979 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:15.979 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookuphash_latency": { 2024-05-26T22:51:15.980 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:51:15.980 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:51:15.980 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:51:15.980 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:15.980 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookupino_latency": { 2024-05-26T22:51:15.980 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:51:15.980 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:51:15.980 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:51:15.980 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:15.980 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookupname_latency": { 2024-05-26T22:51:15.980 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:51:15.980 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:51:15.980 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:51:15.980 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:15.981 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookupparent_latency": { 2024-05-26T22:51:15.981 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:51:15.981 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:51:15.981 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:51:15.981 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:15.981 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookupsnap_latency": { 2024-05-26T22:51:15.981 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:51:15.981 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:51:15.981 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:51:15.981 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:15.981 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lssnap_latency": { 2024-05-26T22:51:15.981 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:51:15.981 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:51:15.981 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:51:15.982 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:15.982 INFO:teuthology.orchestra.run.smithi131.stdout: "req_mkdir_latency": { 2024-05-26T22:51:15.982 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:51:15.982 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:51:15.982 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:51:15.982 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:15.982 INFO:teuthology.orchestra.run.smithi131.stdout: "req_mknod_latency": { 2024-05-26T22:51:15.982 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:51:15.982 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:51:15.982 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:51:15.982 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:15.982 INFO:teuthology.orchestra.run.smithi131.stdout: "req_mksnap_latency": { 2024-05-26T22:51:15.983 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:51:15.983 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:51:15.983 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:51:15.983 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:15.983 INFO:teuthology.orchestra.run.smithi131.stdout: "req_open_latency": { 2024-05-26T22:51:15.983 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:51:15.983 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:51:15.983 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:51:15.983 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:15.983 INFO:teuthology.orchestra.run.smithi131.stdout: "req_readdir_latency": { 2024-05-26T22:51:15.983 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:51:15.983 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:51:15.983 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:51:15.984 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:15.984 INFO:teuthology.orchestra.run.smithi131.stdout: "req_rename_latency": { 2024-05-26T22:51:15.984 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:51:15.984 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:51:15.984 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:51:15.984 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:15.984 INFO:teuthology.orchestra.run.smithi131.stdout: "req_renamesnap_latency": { 2024-05-26T22:51:15.984 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:51:15.984 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:51:15.984 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:51:15.984 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:15.984 INFO:teuthology.orchestra.run.smithi131.stdout: "req_snapdiff_latency": { 2024-05-26T22:51:15.984 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:51:15.985 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:51:15.985 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:51:15.985 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:15.985 INFO:teuthology.orchestra.run.smithi131.stdout: "req_rmdir_latency": { 2024-05-26T22:51:15.985 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:51:15.985 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:51:15.985 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:51:15.985 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:15.985 INFO:teuthology.orchestra.run.smithi131.stdout: "req_rmsnap_latency": { 2024-05-26T22:51:15.985 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:51:15.985 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:51:15.985 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:51:15.985 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:15.986 INFO:teuthology.orchestra.run.smithi131.stdout: "req_rmxattr_latency": { 2024-05-26T22:51:15.986 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:51:15.986 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:51:15.986 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:51:15.986 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:15.986 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setattr_latency": { 2024-05-26T22:51:15.986 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:51:15.986 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:51:15.986 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:51:15.986 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:15.986 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setdirlayout_latency": { 2024-05-26T22:51:15.986 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:51:15.986 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:51:15.987 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:51:15.987 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:15.987 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setfilelock_latency": { 2024-05-26T22:51:15.987 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:51:15.987 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:51:15.987 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:51:15.987 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:15.987 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setlayout_latency": { 2024-05-26T22:51:15.987 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:51:15.987 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:51:15.987 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:51:15.987 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:15.988 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setxattr_latency": { 2024-05-26T22:51:15.988 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:51:15.988 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:51:15.988 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:51:15.988 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:15.988 INFO:teuthology.orchestra.run.smithi131.stdout: "req_symlink_latency": { 2024-05-26T22:51:15.988 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:51:15.988 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:51:15.988 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:51:15.988 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:15.988 INFO:teuthology.orchestra.run.smithi131.stdout: "req_unlink_latency": { 2024-05-26T22:51:15.988 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:51:15.988 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:51:15.989 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:51:15.989 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:15.989 INFO:teuthology.orchestra.run.smithi131.stdout: "cap_revoke_eviction": 0, 2024-05-26T22:51:15.989 INFO:teuthology.orchestra.run.smithi131.stdout: "cap_acquisition_throttle": 0, 2024-05-26T22:51:15.989 INFO:teuthology.orchestra.run.smithi131.stdout: "req_getvxattr_latency": { 2024-05-26T22:51:15.989 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:51:15.989 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:51:15.989 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:51:15.989 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:51:15.989 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:15.989 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_sessions": { 2024-05-26T22:51:15.989 INFO:teuthology.orchestra.run.smithi131.stdout: "session_count": 1, 2024-05-26T22:51:15.990 INFO:teuthology.orchestra.run.smithi131.stdout: "session_add": 1, 2024-05-26T22:51:15.990 INFO:teuthology.orchestra.run.smithi131.stdout: "session_remove": 0, 2024-05-26T22:51:15.990 INFO:teuthology.orchestra.run.smithi131.stdout: "sessions_open": 1, 2024-05-26T22:51:15.990 INFO:teuthology.orchestra.run.smithi131.stdout: "sessions_stale": 0, 2024-05-26T22:51:15.990 INFO:teuthology.orchestra.run.smithi131.stdout: "total_load": 0, 2024-05-26T22:51:15.990 INFO:teuthology.orchestra.run.smithi131.stdout: "average_load": 0, 2024-05-26T22:51:15.990 INFO:teuthology.orchestra.run.smithi131.stdout: "avg_session_uptime": 800, 2024-05-26T22:51:15.990 INFO:teuthology.orchestra.run.smithi131.stdout: "mdthresh_evicted": 0 2024-05-26T22:51:15.990 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:15.990 INFO:teuthology.orchestra.run.smithi131.stdout: "mempool": { 2024-05-26T22:51:15.990 INFO:teuthology.orchestra.run.smithi131.stdout: "bloom_filter_bytes": 0, 2024-05-26T22:51:15.990 INFO:teuthology.orchestra.run.smithi131.stdout: "bloom_filter_items": 0, 2024-05-26T22:51:15.990 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_alloc_bytes": 0, 2024-05-26T22:51:15.991 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_alloc_items": 0, 2024-05-26T22:51:15.991 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_data_bytes": 0, 2024-05-26T22:51:15.991 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_data_items": 0, 2024-05-26T22:51:15.991 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-26T22:51:15.991 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_onode_items": 0, 2024-05-26T22:51:15.991 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-26T22:51:15.991 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_meta_items": 0, 2024-05-26T22:51:15.991 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_other_bytes": 0, 2024-05-26T22:51:15.991 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_other_items": 0, 2024-05-26T22:51:15.991 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-26T22:51:15.991 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_buffer_items": 0, 2024-05-26T22:51:15.991 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_extent_bytes": 0, 2024-05-26T22:51:15.991 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_extent_items": 0, 2024-05-26T22:51:15.991 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_blob_bytes": 0, 2024-05-26T22:51:15.992 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_blob_items": 0, 2024-05-26T22:51:15.992 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-26T22:51:15.992 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_shared_blob_items": 0, 2024-05-26T22:51:15.992 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-26T22:51:15.992 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_inline_bl_items": 0, 2024-05-26T22:51:15.992 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_fsck_bytes": 0, 2024-05-26T22:51:15.992 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_fsck_items": 0, 2024-05-26T22:51:15.992 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_txc_bytes": 0, 2024-05-26T22:51:15.992 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_txc_items": 0, 2024-05-26T22:51:15.992 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-26T22:51:15.992 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_deferred_items": 0, 2024-05-26T22:51:15.992 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_bytes": 0, 2024-05-26T22:51:15.993 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_items": 0, 2024-05-26T22:51:15.993 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_bytes": 0, 2024-05-26T22:51:15.993 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_items": 0, 2024-05-26T22:51:15.993 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_reader_bytes": 0, 2024-05-26T22:51:15.993 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_reader_items": 0, 2024-05-26T22:51:15.993 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_writer_bytes": 0, 2024-05-26T22:51:15.993 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_writer_items": 0, 2024-05-26T22:51:15.993 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_anon_bytes": 239790, 2024-05-26T22:51:15.993 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_anon_items": 143, 2024-05-26T22:51:15.993 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_meta_bytes": 0, 2024-05-26T22:51:15.993 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_meta_items": 0, 2024-05-26T22:51:15.993 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_bytes": 0, 2024-05-26T22:51:15.993 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_items": 0, 2024-05-26T22:51:15.994 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_mapbl_bytes": 0, 2024-05-26T22:51:15.994 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_mapbl_items": 0, 2024-05-26T22:51:15.994 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_pglog_bytes": 0, 2024-05-26T22:51:15.994 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_pglog_items": 0, 2024-05-26T22:51:15.994 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_bytes": 5736, 2024-05-26T22:51:15.994 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_items": 94, 2024-05-26T22:51:15.994 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_mapping_bytes": 0, 2024-05-26T22:51:15.994 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_mapping_items": 0, 2024-05-26T22:51:15.994 INFO:teuthology.orchestra.run.smithi131.stdout: "pgmap_bytes": 0, 2024-05-26T22:51:15.994 INFO:teuthology.orchestra.run.smithi131.stdout: "pgmap_items": 0, 2024-05-26T22:51:15.994 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_co_bytes": 332891, 2024-05-26T22:51:15.994 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_co_items": 2603, 2024-05-26T22:51:15.994 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_1_bytes": 0, 2024-05-26T22:51:15.995 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_1_items": 0, 2024-05-26T22:51:15.995 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_2_bytes": 0, 2024-05-26T22:51:15.995 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_2_items": 0 2024-05-26T22:51:15.995 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:15.995 INFO:teuthology.orchestra.run.smithi131.stdout: "objecter": { 2024-05-26T22:51:15.995 INFO:teuthology.orchestra.run.smithi131.stdout: "op_active": 0, 2024-05-26T22:51:15.995 INFO:teuthology.orchestra.run.smithi131.stdout: "op_laggy": 0, 2024-05-26T22:51:15.995 INFO:teuthology.orchestra.run.smithi131.stdout: "op_send": 93, 2024-05-26T22:51:15.995 INFO:teuthology.orchestra.run.smithi131.stdout: "op_send_bytes": 117939, 2024-05-26T22:51:15.995 INFO:teuthology.orchestra.run.smithi131.stdout: "op_resend": 0, 2024-05-26T22:51:15.995 INFO:teuthology.orchestra.run.smithi131.stdout: "op_reply": 93, 2024-05-26T22:51:15.995 INFO:teuthology.orchestra.run.smithi131.stdout: "op_latency": { 2024-05-26T22:51:15.996 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 93, 2024-05-26T22:51:15.996 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.912980698, 2024-05-26T22:51:15.996 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.009816996 2024-05-26T22:51:15.996 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:15.996 INFO:teuthology.orchestra.run.smithi131.stdout: "op_inflight": 0, 2024-05-26T22:51:15.996 INFO:teuthology.orchestra.run.smithi131.stdout: "oplen_avg": { 2024-05-26T22:51:15.996 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 93, 2024-05-26T22:51:15.996 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 138 2024-05-26T22:51:15.996 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:15.996 INFO:teuthology.orchestra.run.smithi131.stdout: "op": 93, 2024-05-26T22:51:15.996 INFO:teuthology.orchestra.run.smithi131.stdout: "op_r": 57, 2024-05-26T22:51:15.996 INFO:teuthology.orchestra.run.smithi131.stdout: "op_w": 36, 2024-05-26T22:51:15.997 INFO:teuthology.orchestra.run.smithi131.stdout: "op_rmw": 0, 2024-05-26T22:51:15.997 INFO:teuthology.orchestra.run.smithi131.stdout: "op_pg": 0, 2024-05-26T22:51:15.997 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_stat": 27, 2024-05-26T22:51:15.997 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_create": 0, 2024-05-26T22:51:15.997 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_read": 6, 2024-05-26T22:51:15.997 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_write": 28, 2024-05-26T22:51:15.997 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_writefull": 1, 2024-05-26T22:51:15.997 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_writesame": 0, 2024-05-26T22:51:15.997 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_append": 0, 2024-05-26T22:51:15.997 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_zero": 1, 2024-05-26T22:51:15.997 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_truncate": 0, 2024-05-26T22:51:15.997 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_delete": 5, 2024-05-26T22:51:15.997 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_mapext": 0, 2024-05-26T22:51:15.997 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_sparse_read": 0, 2024-05-26T22:51:15.998 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_clonerange": 0, 2024-05-26T22:51:15.998 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_getxattr": 22, 2024-05-26T22:51:15.998 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_setxattr": 0, 2024-05-26T22:51:15.998 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_cmpxattr": 0, 2024-05-26T22:51:15.998 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_rmxattr": 0, 2024-05-26T22:51:15.998 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_resetxattrs": 0, 2024-05-26T22:51:15.998 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_call": 0, 2024-05-26T22:51:15.998 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_watch": 0, 2024-05-26T22:51:15.998 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_notify": 0, 2024-05-26T22:51:15.998 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_src_cmpxattr": 0, 2024-05-26T22:51:15.998 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_pgls": 0, 2024-05-26T22:51:15.998 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_pgls_filter": 0, 2024-05-26T22:51:15.998 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_other": 0, 2024-05-26T22:51:15.999 INFO:teuthology.orchestra.run.smithi131.stdout: "linger_active": 0, 2024-05-26T22:51:15.999 INFO:teuthology.orchestra.run.smithi131.stdout: "linger_send": 0, 2024-05-26T22:51:15.999 INFO:teuthology.orchestra.run.smithi131.stdout: "linger_resend": 0, 2024-05-26T22:51:15.999 INFO:teuthology.orchestra.run.smithi131.stdout: "linger_ping": 0, 2024-05-26T22:51:15.999 INFO:teuthology.orchestra.run.smithi131.stdout: "poolop_active": 0, 2024-05-26T22:51:15.999 INFO:teuthology.orchestra.run.smithi131.stdout: "poolop_send": 0, 2024-05-26T22:51:15.999 INFO:teuthology.orchestra.run.smithi131.stdout: "poolop_resend": 0, 2024-05-26T22:51:15.999 INFO:teuthology.orchestra.run.smithi131.stdout: "poolstat_active": 0, 2024-05-26T22:51:15.999 INFO:teuthology.orchestra.run.smithi131.stdout: "poolstat_send": 0, 2024-05-26T22:51:15.999 INFO:teuthology.orchestra.run.smithi131.stdout: "poolstat_resend": 0, 2024-05-26T22:51:15.999 INFO:teuthology.orchestra.run.smithi131.stdout: "statfs_active": 0, 2024-05-26T22:51:15.999 INFO:teuthology.orchestra.run.smithi131.stdout: "statfs_send": 0, 2024-05-26T22:51:15.999 INFO:teuthology.orchestra.run.smithi131.stdout: "statfs_resend": 0, 2024-05-26T22:51:16.000 INFO:teuthology.orchestra.run.smithi131.stdout: "command_active": 0, 2024-05-26T22:51:16.000 INFO:teuthology.orchestra.run.smithi131.stdout: "command_send": 0, 2024-05-26T22:51:16.000 INFO:teuthology.orchestra.run.smithi131.stdout: "command_resend": 0, 2024-05-26T22:51:16.000 INFO:teuthology.orchestra.run.smithi131.stdout: "map_epoch": 47, 2024-05-26T22:51:16.000 INFO:teuthology.orchestra.run.smithi131.stdout: "map_full": 0, 2024-05-26T22:51:16.000 INFO:teuthology.orchestra.run.smithi131.stdout: "map_inc": 2, 2024-05-26T22:51:16.000 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_sessions": 8, 2024-05-26T22:51:16.000 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_session_open": 8, 2024-05-26T22:51:16.000 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_session_close": 0, 2024-05-26T22:51:16.000 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_laggy": 0, 2024-05-26T22:51:16.000 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_wr": 1, 2024-05-26T22:51:16.000 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_rd": 46, 2024-05-26T22:51:16.000 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_del": 1 2024-05-26T22:51:16.001 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.001 INFO:teuthology.orchestra.run.smithi131.stdout: "oft": { 2024-05-26T22:51:16.001 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_total_objs": 0, 2024-05-26T22:51:16.001 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_total_kv_pairs": 0, 2024-05-26T22:51:16.001 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_total_updates": 0, 2024-05-26T22:51:16.001 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_total_removes": 0 2024-05-26T22:51:16.001 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.001 INFO:teuthology.orchestra.run.smithi131.stdout: "osd-slow-ops": { 2024-05-26T22:51:16.001 INFO:teuthology.orchestra.run.smithi131.stdout: "slow_ops_count": 0 2024-05-26T22:51:16.001 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.001 INFO:teuthology.orchestra.run.smithi131.stdout: "purge_queue": { 2024-05-26T22:51:16.001 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executing_ops": 0, 2024-05-26T22:51:16.001 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executing_ops_high_water": 0, 2024-05-26T22:51:16.002 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executing": 0, 2024-05-26T22:51:16.002 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executing_high_water": 0, 2024-05-26T22:51:16.002 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executed_ops": 0, 2024-05-26T22:51:16.002 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executed": 0, 2024-05-26T22:51:16.002 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_item_in_journal": 0 2024-05-26T22:51:16.002 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.002 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-26T22:51:16.002 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-05-26T22:51:16.002 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 104857600, 2024-05-26T22:51:16.002 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-05-26T22:51:16.002 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 153, 2024-05-26T22:51:16.002 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 219701, 2024-05-26T22:51:16.002 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-05-26T22:51:16.003 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 153, 2024-05-26T22:51:16.003 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 0, 2024-05-26T22:51:16.003 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 0, 2024-05-26T22:51:16.003 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 153, 2024-05-26T22:51:16.003 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 219701, 2024-05-26T22:51:16.003 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-05-26T22:51:16.003 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:51:16.003 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:51:16.003 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:51:16.003 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:51:16.003 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.003 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-objecter_bytes": { 2024-05-26T22:51:16.003 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-05-26T22:51:16.004 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 104857600, 2024-05-26T22:51:16.004 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-05-26T22:51:16.004 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 0, 2024-05-26T22:51:16.004 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 0, 2024-05-26T22:51:16.004 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-05-26T22:51:16.004 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 0, 2024-05-26T22:51:16.004 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 93, 2024-05-26T22:51:16.004 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 179019, 2024-05-26T22:51:16.004 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 93, 2024-05-26T22:51:16.004 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 179019, 2024-05-26T22:51:16.004 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-05-26T22:51:16.004 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:51:16.004 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:51:16.005 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:51:16.005 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:51:16.005 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.005 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-objecter_ops": { 2024-05-26T22:51:16.005 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-05-26T22:51:16.005 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 1024, 2024-05-26T22:51:16.005 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-05-26T22:51:16.005 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 0, 2024-05-26T22:51:16.005 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 0, 2024-05-26T22:51:16.005 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-05-26T22:51:16.005 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 0, 2024-05-26T22:51:16.005 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 93, 2024-05-26T22:51:16.005 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 93, 2024-05-26T22:51:16.006 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 93, 2024-05-26T22:51:16.006 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 93, 2024-05-26T22:51:16.006 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-05-26T22:51:16.006 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:51:16.006 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:51:16.006 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:51:16.006 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:51:16.006 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.006 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-write_buf_throttle": { 2024-05-26T22:51:16.006 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-05-26T22:51:16.006 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 3758096384, 2024-05-26T22:51:16.006 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-05-26T22:51:16.006 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 0, 2024-05-26T22:51:16.007 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 0, 2024-05-26T22:51:16.007 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-05-26T22:51:16.007 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 0, 2024-05-26T22:51:16.007 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 0, 2024-05-26T22:51:16.007 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 0, 2024-05-26T22:51:16.007 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 0, 2024-05-26T22:51:16.007 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 0, 2024-05-26T22:51:16.007 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-05-26T22:51:16.007 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:51:16.007 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:51:16.007 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:51:16.007 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:51:16.007 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.008 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-write_buf_throttle-0x5567127d2820": { 2024-05-26T22:51:16.008 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-05-26T22:51:16.008 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 3758096384, 2024-05-26T22:51:16.008 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-05-26T22:51:16.008 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 54, 2024-05-26T22:51:16.008 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 116474, 2024-05-26T22:51:16.008 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-05-26T22:51:16.008 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 54, 2024-05-26T22:51:16.008 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 0, 2024-05-26T22:51:16.008 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 0, 2024-05-26T22:51:16.008 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 28, 2024-05-26T22:51:16.008 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 116474, 2024-05-26T22:51:16.008 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-05-26T22:51:16.008 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:51:16.009 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:51:16.009 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:51:16.009 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:51:16.009 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:51:16.009 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-26T22:51:16.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.955+0000 7feb83200640 1 -- 172.21.15.131:0/4284132583 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7feb6006be30 msgr2=0x7feb6006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:16.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.955+0000 7feb83200640 1 --2- 172.21.15.131:0/4284132583 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7feb6006be30 0x7feb6006e2f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7feb84140950 tx=0x7feb6c0094a0 comp rx=0 tx=0).stop 2024-05-26T22:51:16.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.955+0000 7feb83200640 1 -- 172.21.15.131:0/4284132583 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb840d0020 msgr2=0x7feb840b2f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:16.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.955+0000 7feb83200640 1 --2- 172.21.15.131:0/4284132583 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb840d0020 0x7feb840b2f60 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7feb7c035040 tx=0x7feb7c0046c0 comp rx=0 tx=0).stop 2024-05-26T22:51:16.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.955+0000 7feb83200640 1 -- 172.21.15.131:0/4284132583 shutdown_connections 2024-05-26T22:51:16.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.955+0000 7feb83200640 1 --2- 172.21.15.131:0/4284132583 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb84140e80 0x7feb841e79f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:16.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.955+0000 7feb83200640 1 --2- 172.21.15.131:0/4284132583 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7feb6006be30 0x7feb6006e2f0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:16.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.955+0000 7feb83200640 1 --2- 172.21.15.131:0/4284132583 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feb84141260 0x7feb8415fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:16.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.955+0000 7feb83200640 1 --2- 172.21.15.131:0/4284132583 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb840d0020 0x7feb840b2f60 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:16.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.955+0000 7feb83200640 1 -- 172.21.15.131:0/4284132583 >> 172.21.15.131:0/4284132583 conn(0x7feb840cc7f0 msgr2=0x7feb84072600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:16.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.955+0000 7feb83200640 1 -- 172.21.15.131:0/4284132583 shutdown_connections 2024-05-26T22:51:16.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:15.955+0000 7feb83200640 1 -- 172.21.15.131:0/4284132583 wait complete. 2024-05-26T22:51:16.012 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:2 perf dump 2024-05-26T22:51:16.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.219+0000 7fa46ec00640 1 Processor -- start 2024-05-26T22:51:16.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.219+0000 7fa46ec00640 1 -- start start 2024-05-26T22:51:16.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.221+0000 7fa46ec00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fa4681554f0 0x7fa468071f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:16.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.221+0000 7fa46ec00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa4680724a0 0x7fa46806e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:16.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.221+0000 7fa46ec00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fa46815c220 0x7fa46806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:16.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.221+0000 7fa46ec00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fa4681558d0 con 0x7fa4681554f0 2024-05-26T22:51:16.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.221+0000 7fa46ec00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fa4680729b0 con 0x7fa46815c220 2024-05-26T22:51:16.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.221+0000 7fa46ec00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa46815c750 con 0x7fa4680724a0 2024-05-26T22:51:16.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.221+0000 7fa46d800640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fa4681554f0 0x7fa468071f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:16.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.221+0000 7fa46d800640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fa4681554f0 0x7fa468071f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44738/0 (socket says 172.21.15.131:44738) 2024-05-26T22:51:16.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.221+0000 7fa46d800640 1 -- 172.21.15.131:0/83464551 learned_addr learned my addr 172.21.15.131:0/83464551 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:16.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.221+0000 7fa46ce00640 1 --2- 172.21.15.131:0/83464551 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa4680724a0 0x7fa46806e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:16.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.222+0000 7fa46d800640 1 -- 172.21.15.131:0/83464551 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa4680724a0 msgr2=0x7fa46806e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:16.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.222+0000 7fa46d800640 1 --2- 172.21.15.131:0/83464551 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa4680724a0 0x7fa46806e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:16.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.222+0000 7fa46d800640 1 -- 172.21.15.131:0/83464551 >> v2:172.21.15.187:3300/0 conn(0x7fa46815c220 msgr2=0x7fa46806e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:16.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.222+0000 7fa46d800640 1 --2- 172.21.15.131:0/83464551 >> v2:172.21.15.187:3300/0 conn(0x7fa46815c220 0x7fa46806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:16.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.222+0000 7fa46d800640 1 -- 172.21.15.131:0/83464551 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa46815ca30 con 0x7fa4681554f0 2024-05-26T22:51:16.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.223+0000 7fa46d800640 1 --2- 172.21.15.131:0/83464551 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4681554f0 0x7fa468071f60 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fa46806f4c0 tx=0x7fa46002c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8e1be145588eecb server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:16.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.223+0000 7fa467e00640 1 -- 172.21.15.131:0/83464551 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa46000b2e0 con 0x7fa4681554f0 2024-05-26T22:51:16.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.223+0000 7fa467e00640 1 -- 172.21.15.131:0/83464551 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa460009600 con 0x7fa4681554f0 2024-05-26T22:51:16.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.223+0000 7fa467e00640 1 -- 172.21.15.131:0/83464551 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa46000b480 con 0x7fa4681554f0 2024-05-26T22:51:16.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.224+0000 7fa46ec00640 1 -- 172.21.15.131:0/83464551 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4681554f0 msgr2=0x7fa468071f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:16.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.224+0000 7fa46ec00640 1 --2- 172.21.15.131:0/83464551 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4681554f0 0x7fa468071f60 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fa46806f4c0 tx=0x7fa46002c8a0 comp rx=0 tx=0).stop 2024-05-26T22:51:16.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.224+0000 7fa46ec00640 1 -- 172.21.15.131:0/83464551 shutdown_connections 2024-05-26T22:51:16.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.224+0000 7fa46ec00640 1 --2- 172.21.15.131:0/83464551 >> v2:172.21.15.187:3300/0 conn(0x7fa46815c220 0x7fa46806e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:16.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.224+0000 7fa46ec00640 1 --2- 172.21.15.131:0/83464551 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa4680724a0 0x7fa46806e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:16.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.224+0000 7fa46ec00640 1 --2- 172.21.15.131:0/83464551 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4681554f0 0x7fa468071f60 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:16.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.224+0000 7fa46ec00640 1 -- 172.21.15.131:0/83464551 >> 172.21.15.131:0/83464551 conn(0x7fa4680cc7f0 msgr2=0x7fa4680ccbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:16.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.224+0000 7fa46ec00640 1 -- 172.21.15.131:0/83464551 shutdown_connections 2024-05-26T22:51:16.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.224+0000 7fa46ec00640 1 -- 172.21.15.131:0/83464551 wait complete. 2024-05-26T22:51:16.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.226+0000 7fa46ec00640 1 Processor -- start 2024-05-26T22:51:16.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.226+0000 7fa46ec00640 1 -- start start 2024-05-26T22:51:16.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.227+0000 7fa46ec00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa4680724a0 0x7fa46815a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:16.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.227+0000 7fa46ec00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4681554f0 0x7fa4680ce650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:16.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.227+0000 7fa46ec00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa46815c220 0x7fa468150120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:16.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.227+0000 7fa46ec00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa46815ebf0 con 0x7fa4681554f0 2024-05-26T22:51:16.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.227+0000 7fa46ec00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa46815ea70 con 0x7fa46815c220 2024-05-26T22:51:16.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.227+0000 7fa46ec00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa46815ed70 con 0x7fa4680724a0 2024-05-26T22:51:16.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.227+0000 7fa46ce00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4681554f0 0x7fa4680ce650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:16.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.227+0000 7fa46ce00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4681554f0 0x7fa4680ce650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44752/0 (socket says 172.21.15.131:44752) 2024-05-26T22:51:16.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.227+0000 7fa46ce00640 1 -- 172.21.15.131:0/2279583287 learned_addr learned my addr 172.21.15.131:0/2279583287 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:16.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.227+0000 7fa46ce00640 1 -- 172.21.15.131:0/2279583287 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa4680724a0 msgr2=0x7fa46815a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:16.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.227+0000 7fa46ce00640 1 --2- 172.21.15.131:0/2279583287 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa4680724a0 0x7fa46815a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:16.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.227+0000 7fa46ce00640 1 -- 172.21.15.131:0/2279583287 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa46815c220 msgr2=0x7fa468150120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:16.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.227+0000 7fa46e200640 1 --2- 172.21.15.131:0/2279583287 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa46815c220 0x7fa468150120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:16.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.227+0000 7fa46ce00640 1 --2- 172.21.15.131:0/2279583287 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa46815c220 0x7fa468150120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:16.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.227+0000 7fa46ce00640 1 -- 172.21.15.131:0/2279583287 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4680d0690 con 0x7fa4681554f0 2024-05-26T22:51:16.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.227+0000 7fa46e200640 1 --2- 172.21.15.131:0/2279583287 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa46815c220 0x7fa468150120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:51:16.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.228+0000 7fa46ce00640 1 --2- 172.21.15.131:0/2279583287 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4681554f0 0x7fa4680ce650 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fa45400d800 tx=0x7fa45400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:16.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.228+0000 7fa465600640 1 -- 172.21.15.131:0/2279583287 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa45400b6c0 con 0x7fa4681554f0 2024-05-26T22:51:16.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.228+0000 7fa465600640 1 -- 172.21.15.131:0/2279583287 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa454019070 con 0x7fa4681554f0 2024-05-26T22:51:16.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.228+0000 7fa465600640 1 -- 172.21.15.131:0/2279583287 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa45400ba80 con 0x7fa4681554f0 2024-05-26T22:51:16.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.228+0000 7fa46ec00640 1 -- 172.21.15.131:0/2279583287 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4680d0970 con 0x7fa4681554f0 2024-05-26T22:51:16.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.228+0000 7fa46ec00640 1 -- 172.21.15.131:0/2279583287 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4680b3d50 con 0x7fa4681554f0 2024-05-26T22:51:16.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.230+0000 7fa465600640 1 -- 172.21.15.131:0/2279583287 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa45400bc20 con 0x7fa4681554f0 2024-05-26T22:51:16.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.231+0000 7fa465600640 1 --2- 172.21.15.131:0/2279583287 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa44806be30 0x7fa44806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:16.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.231+0000 7fa465600640 1 -- 172.21.15.131:0/2279583287 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5853+0+0 (secure 0 0 0) 0x7fa45408b350 con 0x7fa4681554f0 2024-05-26T22:51:16.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.231+0000 7fa46d800640 1 --2- 172.21.15.131:0/2279583287 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa44806be30 0x7fa44806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:16.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.232+0000 7fa46d800640 1 --2- 172.21.15.131:0/2279583287 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa44806be30 0x7fa44806e2f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fa460036000 tx=0x7fa460031000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:16.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.232+0000 7fa47303d740 1 Processor -- start 2024-05-26T22:51:16.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.232+0000 7fa47303d740 1 -- start start 2024-05-26T22:51:16.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.233+0000 7fa47303d740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55767dc4a420 0x55767dbfcdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:16.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.233+0000 7fa47303d740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55767dc2d540 0x55767dc2c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:16.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.233+0000 7fa47303d740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55767dc2ce00 0x55767dc01c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:16.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.233+0000 7fa47303d740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55767db65960 con 0x55767dc2ce00 2024-05-26T22:51:16.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.233+0000 7fa47303d740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55767db0d0d0 con 0x55767dc4a420 2024-05-26T22:51:16.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.233+0000 7fa47303d740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x55767db61920 con 0x55767dc2d540 2024-05-26T22:51:16.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.233+0000 7fa46e200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55767dc2ce00 0x55767dc01c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:16.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.233+0000 7fa46e200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55767dc2ce00 0x55767dc01c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44768/0 (socket says 172.21.15.131:44768) 2024-05-26T22:51:16.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.233+0000 7fa46e200640 1 -- 172.21.15.131:0/489040860 learned_addr learned my addr 172.21.15.131:0/489040860 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:16.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.234+0000 7fa46d800640 1 --2- 172.21.15.131:0/489040860 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55767dc4a420 0x55767dbfcdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:16.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.234+0000 7fa46e200640 1 -- 172.21.15.131:0/489040860 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55767dc2d540 msgr2=0x55767dc2c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:16.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.234+0000 7fa46ce00640 1 --2- 172.21.15.131:0/489040860 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55767dc2d540 0x55767dc2c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:16.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.234+0000 7fa46e200640 1 --2- 172.21.15.131:0/489040860 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55767dc2d540 0x55767dc2c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:16.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.234+0000 7fa46e200640 1 -- 172.21.15.131:0/489040860 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55767dc4a420 msgr2=0x55767dbfcdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:16.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.234+0000 7fa46e200640 1 --2- 172.21.15.131:0/489040860 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55767dc4a420 0x55767dbfcdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:16.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.234+0000 7fa46e200640 1 -- 172.21.15.131:0/489040860 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55767d9437d0 con 0x55767dc2ce00 2024-05-26T22:51:16.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.234+0000 7fa46d800640 1 --2- 172.21.15.131:0/489040860 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55767dc4a420 0x55767dbfcdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:51:16.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.234+0000 7fa46e200640 1 --2- 172.21.15.131:0/489040860 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55767dc2ce00 0x55767dc01c10 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fa45800a910 tx=0x7fa45800add0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b03564a64216b2c server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:16.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.234+0000 7fa45e000640 1 -- 172.21.15.131:0/489040860 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa458015070 con 0x55767dc2ce00 2024-05-26T22:51:16.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.234+0000 7fa45e000640 1 -- 172.21.15.131:0/489040860 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa45800cc00 con 0x55767dc2ce00 2024-05-26T22:51:16.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.234+0000 7fa45e000640 1 -- 172.21.15.131:0/489040860 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4580113f0 con 0x55767dc2ce00 2024-05-26T22:51:16.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.235+0000 7fa47303d740 1 -- 172.21.15.131:0/489040860 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55767dc2ce00 msgr2=0x55767dc01c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:16.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.235+0000 7fa47303d740 1 --2- 172.21.15.131:0/489040860 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55767dc2ce00 0x55767dc01c10 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fa45800a910 tx=0x7fa45800add0 comp rx=0 tx=0).stop 2024-05-26T22:51:16.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.235+0000 7fa47303d740 1 -- 172.21.15.131:0/489040860 shutdown_connections 2024-05-26T22:51:16.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.235+0000 7fa47303d740 1 --2- 172.21.15.131:0/489040860 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55767dc2d540 0x55767dc2c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:16.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.235+0000 7fa47303d740 1 --2- 172.21.15.131:0/489040860 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55767dc2ce00 0x55767dc01c10 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:16.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.235+0000 7fa47303d740 1 --2- 172.21.15.131:0/489040860 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55767dc4a420 0x55767dbfcdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:16.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.235+0000 7fa47303d740 1 -- 172.21.15.131:0/489040860 >> 172.21.15.131:0/489040860 conn(0x55767dbff020 msgr2=0x55767dc478c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:16.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.235+0000 7fa47303d740 1 -- 172.21.15.131:0/489040860 shutdown_connections 2024-05-26T22:51:16.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.235+0000 7fa47303d740 1 -- 172.21.15.131:0/489040860 wait complete. 2024-05-26T22:51:16.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.236+0000 7fa47303d740 1 -- start start 2024-05-26T22:51:16.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.236+0000 7fa47303d740 1 Processor -- start 2024-05-26T22:51:16.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.237+0000 7fa47303d740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55767dc2d540 0x55767dc45e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:16.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.237+0000 7fa47303d740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55767dc4a420 0x55767dc28490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:16.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.237+0000 7fa47303d740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55767dc01bc0 0x55767dc02020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:16.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.237+0000 7fa47303d740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55767dc011d0 con 0x55767dc2d540 2024-05-26T22:51:16.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.237+0000 7fa47303d740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55767db204a0 con 0x55767dc4a420 2024-05-26T22:51:16.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.237+0000 7fa47303d740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x55767dc005c0 con 0x55767dc01bc0 2024-05-26T22:51:16.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.237+0000 7fa47303d740 10 client.0 _ll_register_callbacks cb 0x55767db27c40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-26T22:51:16.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.238+0000 7fa46d800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55767dc2d540 0x55767dc45e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:16.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.238+0000 7fa46d800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55767dc2d540 0x55767dc45e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44774/0 (socket says 172.21.15.131:44774) 2024-05-26T22:51:16.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.238+0000 7fa46d800640 1 -- 172.21.15.131:0/3836239534 learned_addr learned my addr 172.21.15.131:0/3836239534 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:16.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.238+0000 7fa46ce00640 1 --2- 172.21.15.131:0/3836239534 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55767dc4a420 0x55767dc28490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:16.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.238+0000 7fa46e200640 1 --2- 172.21.15.131:0/3836239534 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55767dc01bc0 0x55767dc02020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:16.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.238+0000 7fa46d800640 1 -- 172.21.15.131:0/3836239534 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55767dc01bc0 msgr2=0x55767dc02020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:16.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.238+0000 7fa46d800640 1 --2- 172.21.15.131:0/3836239534 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55767dc01bc0 0x55767dc02020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:16.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.238+0000 7fa46d800640 1 -- 172.21.15.131:0/3836239534 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55767dc4a420 msgr2=0x55767dc28490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:16.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.238+0000 7fa46d800640 1 --2- 172.21.15.131:0/3836239534 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55767dc4a420 0x55767dc28490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:16.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.238+0000 7fa46d800640 1 -- 172.21.15.131:0/3836239534 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55767d8390f0 con 0x55767dc2d540 2024-05-26T22:51:16.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.238+0000 7fa46d800640 1 --2- 172.21.15.131:0/3836239534 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55767dc2d540 0x55767dc45e80 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fa46002ce90 tx=0x7fa460004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dd51522296923ba server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:16.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.238+0000 7fa452a00640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-26T22:51:16.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.241+0000 7fa452a00640 1 -- 172.21.15.131:0/3836239534 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa460044050 con 0x55767dc2d540 2024-05-26T22:51:16.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.241+0000 7fa452a00640 1 -- 172.21.15.131:0/3836239534 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(47..47 src has 1..47) ==== 5853+0+0 (secure 0 0 0) 0x7fa460033d50 con 0x55767dc2d540 2024-05-26T22:51:16.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.241+0000 7fa47303d740 1 -- 172.21.15.131:0/3836239534 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa45800cc00 con 0x55767dc2d540 2024-05-26T22:51:16.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.241+0000 7fa452a00640 1 -- 172.21.15.131:0/3836239534 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fa460049440 con 0x55767dc2d540 2024-05-26T22:51:16.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.241+0000 7fa47303d740 10 client.6117 fetch_fsmap learned FSMap version 40 2024-05-26T22:51:16.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.241+0000 7fa47303d740 1 -- 172.21.15.131:0/3836239534 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=2+}) -- 0x55767d9a3f60 con 0x55767dc2d540 2024-05-26T22:51:16.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.241+0000 7fa452a00640 1 -- 172.21.15.131:0/3836239534 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa45800cc00 con 0x55767dc2d540 2024-05-26T22:51:16.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.241+0000 7fa452a00640 1 -- 172.21.15.131:0/3836239534 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 40) ==== 3542+0+0 (secure 0 0 0) 0x7fa460008050 con 0x55767dc2d540 2024-05-26T22:51:16.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.241+0000 7fa47303d740 10 client.6117 fetch_fsmap finished waiting for FSMap version 40 2024-05-26T22:51:16.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.241+0000 7fa47303d740 10 client.6117 resolve_mds: resolved 3:2 to role '3:2' aka daemon mds.b 2024-05-26T22:51:16.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.241+0000 7fa47303d740 20 client.6117 populate_metadata read hostname 'smithi131' 2024-05-26T22:51:16.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.241+0000 7fa47303d740 1 --2- 172.21.15.131:0/3836239534 >> [v2:172.21.15.187:6832/1027783169,v1:172.21.15.187:6834/1027783169] conn(0x55767dc2df00 0x55767dc2e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:16.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.241+0000 7fa47303d740 4 client.6117 mds_command: new command op to 4670 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-26T22:51:16.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.241+0000 7fa47303d740 1 -- 172.21.15.131:0/3836239534 --> [v2:172.21.15.187:6832/1027783169,v1:172.21.15.187:6834/1027783169] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55767db73090 con 0x55767dc2df00 2024-05-26T22:51:16.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.242+0000 7fa46ce00640 1 --2- 172.21.15.131:0/3836239534 >> [v2:172.21.15.187:6832/1027783169,v1:172.21.15.187:6834/1027783169] conn(0x55767dc2df00 0x55767dc2e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:16.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.243+0000 7fa46ce00640 1 --2- 172.21.15.131:0/3836239534 >> [v2:172.21.15.187:6832/1027783169,v1:172.21.15.187:6834/1027783169] conn(0x55767dc2df00 0x55767dc2e310 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=187939871b7b10c7 server_cookie=7c915f9165eedc94 in_seq=0 out_seq=0 2024-05-26T22:51:16.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.243+0000 7fa452a00640 10 client.6117 ms_handle_connect on v2:172.21.15.187:6832/1027783169 2024-05-26T22:51:16.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.245+0000 7fa452a00640 1 -- 172.21.15.131:0/3836239534 <== mds.2 v2:172.21.15.187:6832/1027783169 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55767db73090 con 0x55767dc2df00 2024-05-26T22:51:16.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.245+0000 7fa452a00640 10 client.6117 handle_command_reply: tid=0 2024-05-26T22:51:16.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.245+0000 7fa47303d740 1 client.6117 shutdown 2024-05-26T22:51:16.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.245+0000 7fa47303d740 1 -- 172.21.15.131:0/3836239534 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55767dc2d540 msgr2=0x55767dc45e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:16.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.245+0000 7fa47303d740 1 --2- 172.21.15.131:0/3836239534 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55767dc2d540 0x55767dc45e80 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fa46002ce90 tx=0x7fa460004ab0 comp rx=0 tx=0).stop 2024-05-26T22:51:16.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.245+0000 7fa47303d740 1 -- 172.21.15.131:0/3836239534 shutdown_connections 2024-05-26T22:51:16.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.245+0000 7fa47303d740 1 --2- 172.21.15.131:0/3836239534 >> [v2:172.21.15.187:6832/1027783169,v1:172.21.15.187:6834/1027783169] conn(0x55767dc2df00 0x55767dc2e310 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:16.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.245+0000 7fa47303d740 1 --2- 172.21.15.131:0/3836239534 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55767dc01bc0 0x55767dc02020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:16.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.245+0000 7fa452a00640 0 client.6117 ms_handle_reset on v2:172.21.15.187:6832/1027783169 2024-05-26T22:51:16.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.245+0000 7fa47303d740 1 --2- 172.21.15.131:0/3836239534 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55767dc4a420 0x55767dc28490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:16.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.245+0000 7fa47303d740 1 --2- 172.21.15.131:0/3836239534 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55767dc2d540 0x55767dc45e80 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:16.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.245+0000 7fa47303d740 1 -- 172.21.15.131:0/3836239534 >> 172.21.15.131:0/3836239534 conn(0x55767dc2ce00 msgr2=0x55767dc47870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:16.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.245+0000 7fa47303d740 1 -- 172.21.15.131:0/3836239534 shutdown_connections 2024-05-26T22:51:16.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.245+0000 7fa47303d740 1 -- 172.21.15.131:0/3836239534 wait complete. 2024-05-26T22:51:16.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.245+0000 7fa47303d740 20 client.6117 trim_cache size 0 max 16384 2024-05-26T22:51:16.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.263+0000 7fa47303d740 1 Processor -- start 2024-05-26T22:51:16.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.263+0000 7fa47303d740 1 -- start start 2024-05-26T22:51:16.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.264+0000 7fa47303d740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55767dc01bc0 0x55767dc2e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:16.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.264+0000 7fa47303d740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55767dc2d540 0x55767dc6ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:16.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.264+0000 7fa47303d740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55767dc4a420 0x55767dbfc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:16.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.264+0000 7fa47303d740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa468200bf0 con 0x55767dc01bc0 2024-05-26T22:51:16.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.264+0000 7fa47303d740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa4681ffe90 con 0x55767dc2d540 2024-05-26T22:51:16.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.264+0000 7fa47303d740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa4682004c0 con 0x55767dc4a420 2024-05-26T22:51:16.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.264+0000 7fa46d800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55767dc01bc0 0x55767dc2e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:16.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.264+0000 7fa46d800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55767dc01bc0 0x55767dc2e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44786/0 (socket says 172.21.15.131:44786) 2024-05-26T22:51:16.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.265+0000 7fa46d800640 1 -- 172.21.15.131:0/1752683537 learned_addr learned my addr 172.21.15.131:0/1752683537 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:16.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.265+0000 7fa46d800640 1 -- 172.21.15.131:0/1752683537 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55767dc4a420 msgr2=0x55767dbfc6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:16.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.265+0000 7fa46d800640 1 --2- 172.21.15.131:0/1752683537 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55767dc4a420 0x55767dbfc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:16.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.265+0000 7fa46d800640 1 -- 172.21.15.131:0/1752683537 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55767dc2d540 msgr2=0x55767dc6ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:16.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.265+0000 7fa46d800640 1 --2- 172.21.15.131:0/1752683537 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55767dc2d540 0x55767dc6ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:16.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.265+0000 7fa46d800640 1 -- 172.21.15.131:0/1752683537 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55767d8121b0 con 0x55767dc01bc0 2024-05-26T22:51:16.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.265+0000 7fa46d800640 1 --2- 172.21.15.131:0/1752683537 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55767dc01bc0 0x55767dc2e5e0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fa4600337f0 tx=0x7fa460047000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95b8bf74d1629bf1 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:16.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.265+0000 7fa45d600640 1 -- 172.21.15.131:0/1752683537 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa46005f070 con 0x55767dc01bc0 2024-05-26T22:51:16.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.265+0000 7fa45d600640 1 -- 172.21.15.131:0/1752683537 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa46004e070 con 0x55767dc01bc0 2024-05-26T22:51:16.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.265+0000 7fa45d600640 1 -- 172.21.15.131:0/1752683537 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa460044070 con 0x55767dc01bc0 2024-05-26T22:51:16.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.266+0000 7fa47303d740 1 -- 172.21.15.131:0/1752683537 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55767dc01bc0 msgr2=0x55767dc2e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:16.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.266+0000 7fa47303d740 1 --2- 172.21.15.131:0/1752683537 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55767dc01bc0 0x55767dc2e5e0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fa4600337f0 tx=0x7fa460047000 comp rx=0 tx=0).stop 2024-05-26T22:51:16.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.266+0000 7fa47303d740 1 -- 172.21.15.131:0/1752683537 shutdown_connections 2024-05-26T22:51:16.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.266+0000 7fa47303d740 1 --2- 172.21.15.131:0/1752683537 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55767dc4a420 0x55767dbfc6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:16.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.266+0000 7fa47303d740 1 --2- 172.21.15.131:0/1752683537 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55767dc2d540 0x55767dc6ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:16.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.266+0000 7fa47303d740 1 --2- 172.21.15.131:0/1752683537 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55767dc01bc0 0x55767dc2e5e0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:16.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.266+0000 7fa47303d740 1 -- 172.21.15.131:0/1752683537 >> 172.21.15.131:0/1752683537 conn(0x55767dc2ce00 msgr2=0x55767dc577b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:16.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.266+0000 7fa47303d740 1 -- 172.21.15.131:0/1752683537 shutdown_connections 2024-05-26T22:51:16.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.266+0000 7fa47303d740 1 -- 172.21.15.131:0/1752683537 wait complete. 2024-05-26T22:51:16.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.267+0000 7fa47303d740 1 -- start start 2024-05-26T22:51:16.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.267+0000 7fa47303d740 1 Processor -- start 2024-05-26T22:51:16.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.268+0000 7fa47303d740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55767dc01bc0 0x55767dbfc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:16.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.268+0000 7fa47303d740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55767dc2d540 0x55767dbfcbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:16.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.268+0000 7fa47303d740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55767dc4a420 0x55767dc6a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:16.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.268+0000 7fa47303d740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55767dbfdc90 con 0x55767dc2d540 2024-05-26T22:51:16.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.268+0000 7fa47303d740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa4680b4250 con 0x55767dc01bc0 2024-05-26T22:51:16.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.268+0000 7fa47303d740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x55767dc03880 con 0x55767dc4a420 2024-05-26T22:51:16.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.268+0000 7fa47303d740 10 client.0 _ll_register_callbacks cb 0x7fa46000d050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-26T22:51:16.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.268+0000 7fa46ce00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55767dc2d540 0x55767dbfcbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:16.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.268+0000 7fa46ce00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55767dc2d540 0x55767dbfcbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44800/0 (socket says 172.21.15.131:44800) 2024-05-26T22:51:16.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.268+0000 7fa46ce00640 1 -- 172.21.15.131:0/2988391403 learned_addr learned my addr 172.21.15.131:0/2988391403 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:16.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.269+0000 7fa46ce00640 1 -- 172.21.15.131:0/2988391403 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55767dc4a420 msgr2=0x55767dc6a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:16.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.269+0000 7fa46ce00640 1 --2- 172.21.15.131:0/2988391403 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55767dc4a420 0x55767dc6a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:16.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.269+0000 7fa46ce00640 1 -- 172.21.15.131:0/2988391403 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55767dc01bc0 msgr2=0x55767dbfc6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:16.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.269+0000 7fa46ce00640 1 --2- 172.21.15.131:0/2988391403 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55767dc01bc0 0x55767dbfc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:16.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.269+0000 7fa46ce00640 1 -- 172.21.15.131:0/2988391403 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55767dc29000 con 0x55767dc2d540 2024-05-26T22:51:16.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.269+0000 7fa46ce00640 1 --2- 172.21.15.131:0/2988391403 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55767dc2d540 0x55767dbfcbf0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fa4540897f0 tx=0x7fa454055890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a3b28f0bb0f405a server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:16.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.269+0000 7fa452a00640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-26T22:51:16.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.269+0000 7fa452a00640 1 -- 172.21.15.131:0/2988391403 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa454071070 con 0x55767dc2d540 2024-05-26T22:51:16.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.269+0000 7fa452a00640 1 -- 172.21.15.131:0/2988391403 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(47..47 src has 1..47) ==== 5853+0+0 (secure 0 0 0) 0x7fa45408b0b0 con 0x55767dc2d540 2024-05-26T22:51:16.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.269+0000 7fa47303d740 1 -- 172.21.15.131:0/2988391403 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa46004e070 con 0x55767dc2d540 2024-05-26T22:51:16.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.269+0000 7fa452a00640 1 -- 172.21.15.131:0/2988391403 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fa454089420 con 0x55767dc2d540 2024-05-26T22:51:16.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.269+0000 7fa47303d740 10 client.6123 fetch_fsmap learned FSMap version 40 2024-05-26T22:51:16.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.269+0000 7fa47303d740 1 -- 172.21.15.131:0/2988391403 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=2+}) -- 0x55767dc45f90 con 0x55767dc2d540 2024-05-26T22:51:16.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.270+0000 7fa452a00640 1 -- 172.21.15.131:0/2988391403 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa46004e070 con 0x55767dc2d540 2024-05-26T22:51:16.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.270+0000 7fa452a00640 1 -- 172.21.15.131:0/2988391403 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 40) ==== 3542+0+0 (secure 0 0 0) 0x7fa454087020 con 0x55767dc2d540 2024-05-26T22:51:16.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.270+0000 7fa47303d740 10 client.6123 fetch_fsmap finished waiting for FSMap version 40 2024-05-26T22:51:16.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.270+0000 7fa47303d740 10 client.6123 resolve_mds: resolved 3:2 to role '3:2' aka daemon mds.b 2024-05-26T22:51:16.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.270+0000 7fa47303d740 20 client.6123 populate_metadata read hostname 'smithi131' 2024-05-26T22:51:16.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.270+0000 7fa47303d740 1 --2- 172.21.15.131:0/2988391403 >> [v2:172.21.15.187:6832/1027783169,v1:172.21.15.187:6834/1027783169] conn(0x55767dc577d0 0x55767dc6aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:16.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.270+0000 7fa47303d740 4 client.6123 mds_command: new command op to 4670 tid=0 multi_id=0 [{"prefix": "perf dump"}] 2024-05-26T22:51:16.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.270+0000 7fa47303d740 1 -- 172.21.15.131:0/2988391403 --> [v2:172.21.15.187:6832/1027783169,v1:172.21.15.187:6834/1027783169] -- command(tid 0: {"prefix": "perf dump"}) -- 0x55767db6d3e0 con 0x55767dc577d0 2024-05-26T22:51:16.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.270+0000 7fa46d800640 1 --2- 172.21.15.131:0/2988391403 >> [v2:172.21.15.187:6832/1027783169,v1:172.21.15.187:6834/1027783169] conn(0x55767dc577d0 0x55767dc6aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:16.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.271+0000 7fa46d800640 1 --2- 172.21.15.131:0/2988391403 >> [v2:172.21.15.187:6832/1027783169,v1:172.21.15.187:6834/1027783169] conn(0x55767dc577d0 0x55767dc6aea0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d0852bc26be6941a server_cookie=f45d3c2958c06d4a in_seq=0 out_seq=0 2024-05-26T22:51:16.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.271+0000 7fa452a00640 10 client.6123 ms_handle_connect on v2:172.21.15.187:6832/1027783169 2024-05-26T22:51:16.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.273+0000 7fa452a00640 1 -- 172.21.15.131:0/2988391403 <== mds.2 v2:172.21.15.187:6832/1027783169 1 ==== command_reply(tid 0: 0 ) ==== 8+0+17722 (crc 0 0 0) 0x55767db6d3e0 con 0x55767dc577d0 2024-05-26T22:51:16.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.273+0000 7fa452a00640 10 client.6123 handle_command_reply: tid=0 2024-05-26T22:51:16.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.273+0000 7fa47303d740 1 client.6123 shutdown 2024-05-26T22:51:16.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.273+0000 7fa47303d740 1 -- 172.21.15.131:0/2988391403 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55767dc2d540 msgr2=0x55767dbfcbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:16.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.273+0000 7fa47303d740 1 --2- 172.21.15.131:0/2988391403 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55767dc2d540 0x55767dbfcbf0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fa4540897f0 tx=0x7fa454055890 comp rx=0 tx=0).stop 2024-05-26T22:51:16.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.273+0000 7fa47303d740 1 -- 172.21.15.131:0/2988391403 shutdown_connections 2024-05-26T22:51:16.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.273+0000 7fa47303d740 1 --2- 172.21.15.131:0/2988391403 >> [v2:172.21.15.187:6832/1027783169,v1:172.21.15.187:6834/1027783169] conn(0x55767dc577d0 0x55767dc6aea0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:16.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.273+0000 7fa47303d740 1 --2- 172.21.15.131:0/2988391403 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55767dc4a420 0x55767dc6a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:16.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.273+0000 7fa47303d740 1 --2- 172.21.15.131:0/2988391403 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55767dc2d540 0x55767dbfcbf0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:16.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.273+0000 7fa47303d740 1 --2- 172.21.15.131:0/2988391403 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55767dc01bc0 0x55767dbfc6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:16.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.273+0000 7fa47303d740 1 -- 172.21.15.131:0/2988391403 >> 172.21.15.131:0/2988391403 conn(0x55767dc2ce00 msgr2=0x55767dc2e2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:16.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.273+0000 7fa452a00640 0 client.6123 ms_handle_reset on v2:172.21.15.187:6832/1027783169 2024-05-26T22:51:16.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.273+0000 7fa47303d740 1 -- 172.21.15.131:0/2988391403 shutdown_connections 2024-05-26T22:51:16.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.273+0000 7fa47303d740 1 -- 172.21.15.131:0/2988391403 wait complete. 2024-05-26T22:51:16.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.273+0000 7fa47303d740 20 client.6123 trim_cache size 0 max 16384 2024-05-26T22:51:16.276 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-26T22:51:16.276 INFO:teuthology.orchestra.run.smithi131.stdout: "AsyncMessenger::Worker-0": { 2024-05-26T22:51:16.276 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_messages": 20, 2024-05-26T22:51:16.276 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages": 21, 2024-05-26T22:51:16.276 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_bytes": 9268, 2024-05-26T22:51:16.276 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_bytes": 82660, 2024-05-26T22:51:16.276 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_created_connections": 13, 2024-05-26T22:51:16.277 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_active_connections": 2, 2024-05-26T22:51:16.277 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_total_time": 0.018503315, 2024-05-26T22:51:16.277 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_send_time": 0.001476360, 2024-05-26T22:51:16.277 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_recv_time": 0.010972098, 2024-05-26T22:51:16.277 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_fast_dispatch_time": 0.000433302, 2024-05-26T22:51:16.277 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages_queue_lat": { 2024-05-26T22:51:16.277 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 20, 2024-05-26T22:51:16.277 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.025580332, 2024-05-26T22:51:16.277 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.001279016 2024-05-26T22:51:16.277 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.277 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_handle_ack_lat": { 2024-05-26T22:51:16.277 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 12, 2024-05-26T22:51:16.278 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000003184, 2024-05-26T22:51:16.278 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000265 2024-05-26T22:51:16.278 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.278 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-26T22:51:16.278 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_encrypted_bytes": 0 2024-05-26T22:51:16.278 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.278 INFO:teuthology.orchestra.run.smithi131.stdout: "AsyncMessenger::Worker-1": { 2024-05-26T22:51:16.278 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_messages": 57, 2024-05-26T22:51:16.278 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages": 89, 2024-05-26T22:51:16.278 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_bytes": 81141, 2024-05-26T22:51:16.278 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_bytes": 175191, 2024-05-26T22:51:16.278 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_created_connections": 11, 2024-05-26T22:51:16.279 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_active_connections": 5, 2024-05-26T22:51:16.279 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_total_time": 0.025408016, 2024-05-26T22:51:16.279 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_send_time": 0.007604825, 2024-05-26T22:51:16.279 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_recv_time": 0.017358364, 2024-05-26T22:51:16.279 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_fast_dispatch_time": 0.000441099, 2024-05-26T22:51:16.279 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages_queue_lat": { 2024-05-26T22:51:16.279 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 88, 2024-05-26T22:51:16.279 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.036065939, 2024-05-26T22:51:16.279 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000409840 2024-05-26T22:51:16.279 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.279 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_handle_ack_lat": { 2024-05-26T22:51:16.279 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 94, 2024-05-26T22:51:16.280 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000048601, 2024-05-26T22:51:16.280 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000517 2024-05-26T22:51:16.280 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.280 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-26T22:51:16.280 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_encrypted_bytes": 101056 2024-05-26T22:51:16.280 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.280 INFO:teuthology.orchestra.run.smithi131.stdout: "AsyncMessenger::Worker-2": { 2024-05-26T22:51:16.280 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_messages": 95, 2024-05-26T22:51:16.280 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages": 130, 2024-05-26T22:51:16.280 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_bytes": 142170, 2024-05-26T22:51:16.280 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_bytes": 142558, 2024-05-26T22:51:16.280 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_created_connections": 5, 2024-05-26T22:51:16.280 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_active_connections": 4, 2024-05-26T22:51:16.280 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_total_time": 0.025596928, 2024-05-26T22:51:16.281 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_send_time": 0.007451362, 2024-05-26T22:51:16.281 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_recv_time": 0.022584309, 2024-05-26T22:51:16.281 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_fast_dispatch_time": 0.001490889, 2024-05-26T22:51:16.281 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages_queue_lat": { 2024-05-26T22:51:16.281 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 130, 2024-05-26T22:51:16.281 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.048182950, 2024-05-26T22:51:16.281 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000370638 2024-05-26T22:51:16.281 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.281 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_handle_ack_lat": { 2024-05-26T22:51:16.281 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 5, 2024-05-26T22:51:16.281 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000005712, 2024-05-26T22:51:16.281 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000001142 2024-05-26T22:51:16.281 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.282 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_encrypted_bytes": 134992, 2024-05-26T22:51:16.282 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_encrypted_bytes": 16992 2024-05-26T22:51:16.282 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.282 INFO:teuthology.orchestra.run.smithi131.stdout: "cct": { 2024-05-26T22:51:16.282 INFO:teuthology.orchestra.run.smithi131.stdout: "total_workers": 1, 2024-05-26T22:51:16.282 INFO:teuthology.orchestra.run.smithi131.stdout: "unhealthy_workers": 0 2024-05-26T22:51:16.282 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.282 INFO:teuthology.orchestra.run.smithi131.stdout: "finisher-MDSRank": { 2024-05-26T22:51:16.282 INFO:teuthology.orchestra.run.smithi131.stdout: "queue_len": 0, 2024-05-26T22:51:16.282 INFO:teuthology.orchestra.run.smithi131.stdout: "complete_latency": { 2024-05-26T22:51:16.282 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 56, 2024-05-26T22:51:16.282 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.010144349, 2024-05-26T22:51:16.282 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000181149 2024-05-26T22:51:16.283 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:51:16.283 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.283 INFO:teuthology.orchestra.run.smithi131.stdout: "finisher-PurgeQueue": { 2024-05-26T22:51:16.283 INFO:teuthology.orchestra.run.smithi131.stdout: "queue_len": 0, 2024-05-26T22:51:16.283 INFO:teuthology.orchestra.run.smithi131.stdout: "complete_latency": { 2024-05-26T22:51:16.283 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 2, 2024-05-26T22:51:16.283 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000034801, 2024-05-26T22:51:16.283 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000017400 2024-05-26T22:51:16.283 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:51:16.283 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.283 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": { 2024-05-26T22:51:16.283 INFO:teuthology.orchestra.run.smithi131.stdout: "request": 0, 2024-05-26T22:51:16.283 INFO:teuthology.orchestra.run.smithi131.stdout: "reply": 0, 2024-05-26T22:51:16.284 INFO:teuthology.orchestra.run.smithi131.stdout: "reply_latency": { 2024-05-26T22:51:16.284 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:51:16.284 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:51:16.284 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:51:16.284 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.284 INFO:teuthology.orchestra.run.smithi131.stdout: "slow_reply": 0, 2024-05-26T22:51:16.284 INFO:teuthology.orchestra.run.smithi131.stdout: "forward": 0, 2024-05-26T22:51:16.284 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_fetch_complete": 0, 2024-05-26T22:51:16.284 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_fetch_keys": 0, 2024-05-26T22:51:16.284 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_commit": 11, 2024-05-26T22:51:16.284 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_split": 0, 2024-05-26T22:51:16.284 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_merge": 0, 2024-05-26T22:51:16.285 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes": 61, 2024-05-26T22:51:16.285 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_top": 0, 2024-05-26T22:51:16.285 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_bottom": 61, 2024-05-26T22:51:16.285 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_pin_tail": 0, 2024-05-26T22:51:16.285 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_pinned": 61, 2024-05-26T22:51:16.285 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_expired": 0, 2024-05-26T22:51:16.285 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_with_caps": 0, 2024-05-26T22:51:16.285 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": 0, 2024-05-26T22:51:16.285 INFO:teuthology.orchestra.run.smithi131.stdout: "subtrees": 4, 2024-05-26T22:51:16.285 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse": 2, 2024-05-26T22:51:16.285 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_hit": 0, 2024-05-26T22:51:16.285 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_forward": 0, 2024-05-26T22:51:16.285 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_discover": 2, 2024-05-26T22:51:16.286 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_dir_fetch": 0, 2024-05-26T22:51:16.286 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_remote_ino": 0, 2024-05-26T22:51:16.286 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_lock": 0, 2024-05-26T22:51:16.286 INFO:teuthology.orchestra.run.smithi131.stdout: "load_cent": 0, 2024-05-26T22:51:16.286 INFO:teuthology.orchestra.run.smithi131.stdout: "q": 0, 2024-05-26T22:51:16.286 INFO:teuthology.orchestra.run.smithi131.stdout: "exported": 0, 2024-05-26T22:51:16.286 INFO:teuthology.orchestra.run.smithi131.stdout: "exported_inodes": 0, 2024-05-26T22:51:16.286 INFO:teuthology.orchestra.run.smithi131.stdout: "imported": 2, 2024-05-26T22:51:16.286 INFO:teuthology.orchestra.run.smithi131.stdout: "imported_inodes": 50, 2024-05-26T22:51:16.286 INFO:teuthology.orchestra.run.smithi131.stdout: "openino_dir_fetch": 0, 2024-05-26T22:51:16.286 INFO:teuthology.orchestra.run.smithi131.stdout: "openino_backtrace_fetch": 0, 2024-05-26T22:51:16.286 INFO:teuthology.orchestra.run.smithi131.stdout: "openino_peer_discover": 0, 2024-05-26T22:51:16.287 INFO:teuthology.orchestra.run.smithi131.stdout: "root_rfiles": 100, 2024-05-26T22:51:16.287 INFO:teuthology.orchestra.run.smithi131.stdout: "root_rbytes": 500, 2024-05-26T22:51:16.287 INFO:teuthology.orchestra.run.smithi131.stdout: "root_rsnaps": 0, 2024-05-26T22:51:16.287 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_backtrace_fetch": 0, 2024-05-26T22:51:16.287 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_set_tag": 0, 2024-05-26T22:51:16.287 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_backtrace_repaired": 0, 2024-05-26T22:51:16.287 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_inotable_repaired": 0, 2024-05-26T22:51:16.287 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_dir_inodes": 0, 2024-05-26T22:51:16.287 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_dir_base_inodes": 0, 2024-05-26T22:51:16.288 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_dirfrag_rstats": 0, 2024-05-26T22:51:16.288 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_file_inodes": 0, 2024-05-26T22:51:16.288 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_inode_file_caps": 0, 2024-05-26T22:51:16.288 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_revoke": 0, 2024-05-26T22:51:16.288 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_grant": 0, 2024-05-26T22:51:16.288 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_trunc": 0, 2024-05-26T22:51:16.288 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-26T22:51:16.288 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-26T22:51:16.288 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_caps": 0, 2024-05-26T22:51:16.288 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_caps_dirty": 0, 2024-05-26T22:51:16.288 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_cap_release": 1, 2024-05-26T22:51:16.288 INFO:teuthology.orchestra.run.smithi131.stdout: "process_request_cap_release": 0 2024-05-26T22:51:16.289 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.289 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_cache": { 2024-05-26T22:51:16.289 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_update": 0, 2024-05-26T22:51:16.289 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_update_receipt": 0, 2024-05-26T22:51:16.289 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_try_discover": 0, 2024-05-26T22:51:16.289 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_send_discover": 3, 2024-05-26T22:51:16.289 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_handle_discover": 0, 2024-05-26T22:51:16.289 INFO:teuthology.orchestra.run.smithi131.stdout: "num_strays": 0, 2024-05-26T22:51:16.289 INFO:teuthology.orchestra.run.smithi131.stdout: "num_strays_delayed": 0, 2024-05-26T22:51:16.290 INFO:teuthology.orchestra.run.smithi131.stdout: "num_strays_enqueuing": 0, 2024-05-26T22:51:16.290 INFO:teuthology.orchestra.run.smithi131.stdout: "strays_created": 0, 2024-05-26T22:51:16.290 INFO:teuthology.orchestra.run.smithi131.stdout: "strays_enqueued": 0, 2024-05-26T22:51:16.290 INFO:teuthology.orchestra.run.smithi131.stdout: "strays_reintegrated": 0, 2024-05-26T22:51:16.290 INFO:teuthology.orchestra.run.smithi131.stdout: "strays_migrated": 0, 2024-05-26T22:51:16.290 INFO:teuthology.orchestra.run.smithi131.stdout: "num_recovering_processing": 0, 2024-05-26T22:51:16.290 INFO:teuthology.orchestra.run.smithi131.stdout: "num_recovering_enqueued": 0, 2024-05-26T22:51:16.290 INFO:teuthology.orchestra.run.smithi131.stdout: "num_recovering_prioritized": 0, 2024-05-26T22:51:16.290 INFO:teuthology.orchestra.run.smithi131.stdout: "recovery_started": 0, 2024-05-26T22:51:16.290 INFO:teuthology.orchestra.run.smithi131.stdout: "recovery_completed": 0, 2024-05-26T22:51:16.290 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_quiesce_path": 0, 2024-05-26T22:51:16.290 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_quiesce_inode": 0, 2024-05-26T22:51:16.290 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_enqueue_scrub": 0, 2024-05-26T22:51:16.291 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_exportdir": 0, 2024-05-26T22:51:16.291 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_flush": 0, 2024-05-26T22:51:16.291 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_fragmentdir": 0, 2024-05-26T22:51:16.291 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_fragstats": 0, 2024-05-26T22:51:16.291 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_inodestats": 0 2024-05-26T22:51:16.291 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.291 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_log": { 2024-05-26T22:51:16.291 INFO:teuthology.orchestra.run.smithi131.stdout: "evlrg": 0, 2024-05-26T22:51:16.291 INFO:teuthology.orchestra.run.smithi131.stdout: "evadd": 25, 2024-05-26T22:51:16.291 INFO:teuthology.orchestra.run.smithi131.stdout: "evex": 0, 2024-05-26T22:51:16.291 INFO:teuthology.orchestra.run.smithi131.stdout: "evtrm": 0, 2024-05-26T22:51:16.291 INFO:teuthology.orchestra.run.smithi131.stdout: "ev": 25, 2024-05-26T22:51:16.292 INFO:teuthology.orchestra.run.smithi131.stdout: "evexg": 0, 2024-05-26T22:51:16.292 INFO:teuthology.orchestra.run.smithi131.stdout: "evexd": 0, 2024-05-26T22:51:16.292 INFO:teuthology.orchestra.run.smithi131.stdout: "segadd": 2, 2024-05-26T22:51:16.292 INFO:teuthology.orchestra.run.smithi131.stdout: "segex": 0, 2024-05-26T22:51:16.292 INFO:teuthology.orchestra.run.smithi131.stdout: "segtrm": 0, 2024-05-26T22:51:16.292 INFO:teuthology.orchestra.run.smithi131.stdout: "seg": 2, 2024-05-26T22:51:16.292 INFO:teuthology.orchestra.run.smithi131.stdout: "segmjr": 2, 2024-05-26T22:51:16.292 INFO:teuthology.orchestra.run.smithi131.stdout: "segexg": 0, 2024-05-26T22:51:16.292 INFO:teuthology.orchestra.run.smithi131.stdout: "segexd": 0, 2024-05-26T22:51:16.292 INFO:teuthology.orchestra.run.smithi131.stdout: "expos": 4194304, 2024-05-26T22:51:16.293 INFO:teuthology.orchestra.run.smithi131.stdout: "wrpos": 4285719, 2024-05-26T22:51:16.293 INFO:teuthology.orchestra.run.smithi131.stdout: "rdpos": 0, 2024-05-26T22:51:16.293 INFO:teuthology.orchestra.run.smithi131.stdout: "jlat": { 2024-05-26T22:51:16.293 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 5, 2024-05-26T22:51:16.293 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.034663067, 2024-05-26T22:51:16.293 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.006932613 2024-05-26T22:51:16.293 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.293 INFO:teuthology.orchestra.run.smithi131.stdout: "replayed": 0 2024-05-26T22:51:16.293 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.293 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_mem": { 2024-05-26T22:51:16.293 INFO:teuthology.orchestra.run.smithi131.stdout: "ino": 64, 2024-05-26T22:51:16.294 INFO:teuthology.orchestra.run.smithi131.stdout: "ino+": 64, 2024-05-26T22:51:16.294 INFO:teuthology.orchestra.run.smithi131.stdout: "ino-": 0, 2024-05-26T22:51:16.294 INFO:teuthology.orchestra.run.smithi131.stdout: "dir": 39, 2024-05-26T22:51:16.294 INFO:teuthology.orchestra.run.smithi131.stdout: "dir+": 40, 2024-05-26T22:51:16.294 INFO:teuthology.orchestra.run.smithi131.stdout: "dir-": 1, 2024-05-26T22:51:16.294 INFO:teuthology.orchestra.run.smithi131.stdout: "dn": 61, 2024-05-26T22:51:16.294 INFO:teuthology.orchestra.run.smithi131.stdout: "dn+": 61, 2024-05-26T22:51:16.294 INFO:teuthology.orchestra.run.smithi131.stdout: "dn-": 0, 2024-05-26T22:51:16.294 INFO:teuthology.orchestra.run.smithi131.stdout: "cap": 0, 2024-05-26T22:51:16.294 INFO:teuthology.orchestra.run.smithi131.stdout: "cap+": 50, 2024-05-26T22:51:16.294 INFO:teuthology.orchestra.run.smithi131.stdout: "cap-": 50, 2024-05-26T22:51:16.295 INFO:teuthology.orchestra.run.smithi131.stdout: "rss": 46464, 2024-05-26T22:51:16.295 INFO:teuthology.orchestra.run.smithi131.stdout: "heap": 223596 2024-05-26T22:51:16.295 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.295 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_server": { 2024-05-26T22:51:16.295 INFO:teuthology.orchestra.run.smithi131.stdout: "dispatch_client_request": 0, 2024-05-26T22:51:16.295 INFO:teuthology.orchestra.run.smithi131.stdout: "dispatch_server_request": 0, 2024-05-26T22:51:16.295 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_request": 0, 2024-05-26T22:51:16.295 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_session": 2, 2024-05-26T22:51:16.295 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_peer_request": 0, 2024-05-26T22:51:16.296 INFO:teuthology.orchestra.run.smithi131.stdout: "req_create_latency": { 2024-05-26T22:51:16.296 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:51:16.296 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:51:16.296 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:51:16.296 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.296 INFO:teuthology.orchestra.run.smithi131.stdout: "req_getattr_latency": { 2024-05-26T22:51:16.296 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:51:16.296 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:51:16.296 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:51:16.296 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.296 INFO:teuthology.orchestra.run.smithi131.stdout: "req_getfilelock_latency": { 2024-05-26T22:51:16.297 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:51:16.297 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:51:16.297 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:51:16.297 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.297 INFO:teuthology.orchestra.run.smithi131.stdout: "req_link_latency": { 2024-05-26T22:51:16.297 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:51:16.297 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:51:16.297 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:51:16.297 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.297 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookup_latency": { 2024-05-26T22:51:16.298 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:51:16.298 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:51:16.298 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:51:16.298 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.298 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookuphash_latency": { 2024-05-26T22:51:16.298 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:51:16.298 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:51:16.298 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:51:16.298 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.299 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookupino_latency": { 2024-05-26T22:51:16.299 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:51:16.299 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:51:16.299 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:51:16.299 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.299 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookupname_latency": { 2024-05-26T22:51:16.299 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:51:16.299 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:51:16.299 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:51:16.300 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.300 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookupparent_latency": { 2024-05-26T22:51:16.300 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:51:16.300 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:51:16.300 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:51:16.300 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.300 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookupsnap_latency": { 2024-05-26T22:51:16.300 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:51:16.300 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:51:16.300 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:51:16.300 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.301 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lssnap_latency": { 2024-05-26T22:51:16.301 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:51:16.301 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:51:16.301 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:51:16.301 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.301 INFO:teuthology.orchestra.run.smithi131.stdout: "req_mkdir_latency": { 2024-05-26T22:51:16.301 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:51:16.301 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:51:16.301 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:51:16.301 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.301 INFO:teuthology.orchestra.run.smithi131.stdout: "req_mknod_latency": { 2024-05-26T22:51:16.301 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:51:16.301 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:51:16.301 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:51:16.302 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.302 INFO:teuthology.orchestra.run.smithi131.stdout: "req_mksnap_latency": { 2024-05-26T22:51:16.302 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:51:16.302 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:51:16.302 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:51:16.302 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.302 INFO:teuthology.orchestra.run.smithi131.stdout: "req_open_latency": { 2024-05-26T22:51:16.302 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:51:16.302 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:51:16.302 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:51:16.302 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.302 INFO:teuthology.orchestra.run.smithi131.stdout: "req_readdir_latency": { 2024-05-26T22:51:16.303 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:51:16.303 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:51:16.303 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:51:16.303 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.303 INFO:teuthology.orchestra.run.smithi131.stdout: "req_rename_latency": { 2024-05-26T22:51:16.303 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:51:16.303 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:51:16.303 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:51:16.303 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.303 INFO:teuthology.orchestra.run.smithi131.stdout: "req_renamesnap_latency": { 2024-05-26T22:51:16.303 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:51:16.303 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:51:16.304 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:51:16.304 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.304 INFO:teuthology.orchestra.run.smithi131.stdout: "req_snapdiff_latency": { 2024-05-26T22:51:16.304 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:51:16.304 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:51:16.304 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:51:16.304 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.304 INFO:teuthology.orchestra.run.smithi131.stdout: "req_rmdir_latency": { 2024-05-26T22:51:16.304 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:51:16.304 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:51:16.304 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:51:16.304 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.304 INFO:teuthology.orchestra.run.smithi131.stdout: "req_rmsnap_latency": { 2024-05-26T22:51:16.305 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:51:16.305 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:51:16.305 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:51:16.305 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.305 INFO:teuthology.orchestra.run.smithi131.stdout: "req_rmxattr_latency": { 2024-05-26T22:51:16.305 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:51:16.305 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:51:16.305 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:51:16.305 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.305 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setattr_latency": { 2024-05-26T22:51:16.305 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:51:16.305 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:51:16.305 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:51:16.305 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.306 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setdirlayout_latency": { 2024-05-26T22:51:16.306 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:51:16.306 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:51:16.306 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:51:16.306 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.306 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setfilelock_latency": { 2024-05-26T22:51:16.306 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:51:16.306 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:51:16.306 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:51:16.306 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.306 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setlayout_latency": { 2024-05-26T22:51:16.306 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:51:16.307 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:51:16.307 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:51:16.307 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.307 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setxattr_latency": { 2024-05-26T22:51:16.307 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:51:16.307 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:51:16.307 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:51:16.307 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.307 INFO:teuthology.orchestra.run.smithi131.stdout: "req_symlink_latency": { 2024-05-26T22:51:16.307 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:51:16.307 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:51:16.307 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:51:16.307 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.308 INFO:teuthology.orchestra.run.smithi131.stdout: "req_unlink_latency": { 2024-05-26T22:51:16.308 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:51:16.308 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:51:16.308 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:51:16.308 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.308 INFO:teuthology.orchestra.run.smithi131.stdout: "cap_revoke_eviction": 0, 2024-05-26T22:51:16.308 INFO:teuthology.orchestra.run.smithi131.stdout: "cap_acquisition_throttle": 0, 2024-05-26T22:51:16.308 INFO:teuthology.orchestra.run.smithi131.stdout: "req_getvxattr_latency": { 2024-05-26T22:51:16.308 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:51:16.308 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:51:16.308 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:51:16.308 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:51:16.308 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.309 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_sessions": { 2024-05-26T22:51:16.309 INFO:teuthology.orchestra.run.smithi131.stdout: "session_count": 1, 2024-05-26T22:51:16.309 INFO:teuthology.orchestra.run.smithi131.stdout: "session_add": 1, 2024-05-26T22:51:16.309 INFO:teuthology.orchestra.run.smithi131.stdout: "session_remove": 0, 2024-05-26T22:51:16.309 INFO:teuthology.orchestra.run.smithi131.stdout: "sessions_open": 1, 2024-05-26T22:51:16.309 INFO:teuthology.orchestra.run.smithi131.stdout: "sessions_stale": 0, 2024-05-26T22:51:16.309 INFO:teuthology.orchestra.run.smithi131.stdout: "total_load": 0, 2024-05-26T22:51:16.309 INFO:teuthology.orchestra.run.smithi131.stdout: "average_load": 0, 2024-05-26T22:51:16.309 INFO:teuthology.orchestra.run.smithi131.stdout: "avg_session_uptime": 812, 2024-05-26T22:51:16.309 INFO:teuthology.orchestra.run.smithi131.stdout: "mdthresh_evicted": 0 2024-05-26T22:51:16.309 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.309 INFO:teuthology.orchestra.run.smithi131.stdout: "mempool": { 2024-05-26T22:51:16.309 INFO:teuthology.orchestra.run.smithi131.stdout: "bloom_filter_bytes": 0, 2024-05-26T22:51:16.310 INFO:teuthology.orchestra.run.smithi131.stdout: "bloom_filter_items": 0, 2024-05-26T22:51:16.310 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_alloc_bytes": 0, 2024-05-26T22:51:16.310 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_alloc_items": 0, 2024-05-26T22:51:16.310 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_data_bytes": 0, 2024-05-26T22:51:16.310 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_data_items": 0, 2024-05-26T22:51:16.310 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-26T22:51:16.310 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_onode_items": 0, 2024-05-26T22:51:16.310 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-26T22:51:16.310 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_meta_items": 0, 2024-05-26T22:51:16.310 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_other_bytes": 0, 2024-05-26T22:51:16.310 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_other_items": 0, 2024-05-26T22:51:16.310 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-26T22:51:16.310 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_buffer_items": 0, 2024-05-26T22:51:16.310 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_extent_bytes": 0, 2024-05-26T22:51:16.311 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_extent_items": 0, 2024-05-26T22:51:16.311 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_blob_bytes": 0, 2024-05-26T22:51:16.311 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_blob_items": 0, 2024-05-26T22:51:16.311 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-26T22:51:16.311 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_shared_blob_items": 0, 2024-05-26T22:51:16.311 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-26T22:51:16.311 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_inline_bl_items": 0, 2024-05-26T22:51:16.311 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_fsck_bytes": 0, 2024-05-26T22:51:16.311 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_fsck_items": 0, 2024-05-26T22:51:16.311 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_txc_bytes": 0, 2024-05-26T22:51:16.311 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_txc_items": 0, 2024-05-26T22:51:16.311 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-26T22:51:16.312 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_deferred_items": 0, 2024-05-26T22:51:16.312 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_bytes": 0, 2024-05-26T22:51:16.312 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_items": 0, 2024-05-26T22:51:16.312 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_bytes": 0, 2024-05-26T22:51:16.312 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_items": 0, 2024-05-26T22:51:16.312 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_reader_bytes": 0, 2024-05-26T22:51:16.312 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_reader_items": 0, 2024-05-26T22:51:16.312 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_writer_bytes": 0, 2024-05-26T22:51:16.312 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_writer_items": 0, 2024-05-26T22:51:16.312 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_anon_bytes": 479254, 2024-05-26T22:51:16.312 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_anon_items": 177, 2024-05-26T22:51:16.312 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_meta_bytes": 0, 2024-05-26T22:51:16.313 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_meta_items": 0, 2024-05-26T22:51:16.313 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_bytes": 0, 2024-05-26T22:51:16.313 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_items": 0, 2024-05-26T22:51:16.313 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_mapbl_bytes": 0, 2024-05-26T22:51:16.313 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_mapbl_items": 0, 2024-05-26T22:51:16.313 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_pglog_bytes": 0, 2024-05-26T22:51:16.313 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_pglog_items": 0, 2024-05-26T22:51:16.313 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_bytes": 5736, 2024-05-26T22:51:16.313 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_items": 94, 2024-05-26T22:51:16.313 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_mapping_bytes": 0, 2024-05-26T22:51:16.313 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_mapping_items": 0, 2024-05-26T22:51:16.313 INFO:teuthology.orchestra.run.smithi131.stdout: "pgmap_bytes": 0, 2024-05-26T22:51:16.313 INFO:teuthology.orchestra.run.smithi131.stdout: "pgmap_items": 0, 2024-05-26T22:51:16.314 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_co_bytes": 285035, 2024-05-26T22:51:16.314 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_co_items": 2313, 2024-05-26T22:51:16.314 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_1_bytes": 0, 2024-05-26T22:51:16.314 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_1_items": 0, 2024-05-26T22:51:16.314 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_2_bytes": 0, 2024-05-26T22:51:16.314 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_2_items": 0 2024-05-26T22:51:16.314 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.314 INFO:teuthology.orchestra.run.smithi131.stdout: "objecter": { 2024-05-26T22:51:16.314 INFO:teuthology.orchestra.run.smithi131.stdout: "op_active": 0, 2024-05-26T22:51:16.314 INFO:teuthology.orchestra.run.smithi131.stdout: "op_laggy": 0, 2024-05-26T22:51:16.314 INFO:teuthology.orchestra.run.smithi131.stdout: "op_send": 42, 2024-05-26T22:51:16.314 INFO:teuthology.orchestra.run.smithi131.stdout: "op_send_bytes": 105110, 2024-05-26T22:51:16.314 INFO:teuthology.orchestra.run.smithi131.stdout: "op_resend": 29, 2024-05-26T22:51:16.315 INFO:teuthology.orchestra.run.smithi131.stdout: "op_reply": 42, 2024-05-26T22:51:16.315 INFO:teuthology.orchestra.run.smithi131.stdout: "op_latency": { 2024-05-26T22:51:16.315 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 42, 2024-05-26T22:51:16.315 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.533991708, 2024-05-26T22:51:16.315 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.012714088 2024-05-26T22:51:16.315 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.315 INFO:teuthology.orchestra.run.smithi131.stdout: "op_inflight": 0, 2024-05-26T22:51:16.315 INFO:teuthology.orchestra.run.smithi131.stdout: "oplen_avg": { 2024-05-26T22:51:16.315 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 42, 2024-05-26T22:51:16.315 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 66 2024-05-26T22:51:16.315 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.315 INFO:teuthology.orchestra.run.smithi131.stdout: "op": 42, 2024-05-26T22:51:16.315 INFO:teuthology.orchestra.run.smithi131.stdout: "op_r": 0, 2024-05-26T22:51:16.316 INFO:teuthology.orchestra.run.smithi131.stdout: "op_w": 42, 2024-05-26T22:51:16.316 INFO:teuthology.orchestra.run.smithi131.stdout: "op_rmw": 0, 2024-05-26T22:51:16.316 INFO:teuthology.orchestra.run.smithi131.stdout: "op_pg": 0, 2024-05-26T22:51:16.316 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_stat": 0, 2024-05-26T22:51:16.316 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_create": 10, 2024-05-26T22:51:16.316 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_read": 0, 2024-05-26T22:51:16.316 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_write": 5, 2024-05-26T22:51:16.316 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_writefull": 6, 2024-05-26T22:51:16.316 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_writesame": 0, 2024-05-26T22:51:16.316 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_append": 0, 2024-05-26T22:51:16.317 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_zero": 0, 2024-05-26T22:51:16.317 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_truncate": 0, 2024-05-26T22:51:16.317 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_delete": 6, 2024-05-26T22:51:16.317 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_mapext": 0, 2024-05-26T22:51:16.317 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_sparse_read": 0, 2024-05-26T22:51:16.317 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_clonerange": 0, 2024-05-26T22:51:16.317 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_getxattr": 0, 2024-05-26T22:51:16.317 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_setxattr": 20, 2024-05-26T22:51:16.317 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_cmpxattr": 0, 2024-05-26T22:51:16.317 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_rmxattr": 0, 2024-05-26T22:51:16.317 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_resetxattrs": 0, 2024-05-26T22:51:16.317 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_call": 0, 2024-05-26T22:51:16.318 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_watch": 0, 2024-05-26T22:51:16.318 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_notify": 0, 2024-05-26T22:51:16.318 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_src_cmpxattr": 0, 2024-05-26T22:51:16.318 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_pgls": 0, 2024-05-26T22:51:16.318 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_pgls_filter": 0, 2024-05-26T22:51:16.318 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_other": 0, 2024-05-26T22:51:16.318 INFO:teuthology.orchestra.run.smithi131.stdout: "linger_active": 0, 2024-05-26T22:51:16.318 INFO:teuthology.orchestra.run.smithi131.stdout: "linger_send": 0, 2024-05-26T22:51:16.318 INFO:teuthology.orchestra.run.smithi131.stdout: "linger_resend": 0, 2024-05-26T22:51:16.318 INFO:teuthology.orchestra.run.smithi131.stdout: "linger_ping": 0, 2024-05-26T22:51:16.318 INFO:teuthology.orchestra.run.smithi131.stdout: "poolop_active": 0, 2024-05-26T22:51:16.318 INFO:teuthology.orchestra.run.smithi131.stdout: "poolop_send": 0, 2024-05-26T22:51:16.319 INFO:teuthology.orchestra.run.smithi131.stdout: "poolop_resend": 0, 2024-05-26T22:51:16.319 INFO:teuthology.orchestra.run.smithi131.stdout: "poolstat_active": 0, 2024-05-26T22:51:16.319 INFO:teuthology.orchestra.run.smithi131.stdout: "poolstat_send": 0, 2024-05-26T22:51:16.319 INFO:teuthology.orchestra.run.smithi131.stdout: "poolstat_resend": 0, 2024-05-26T22:51:16.319 INFO:teuthology.orchestra.run.smithi131.stdout: "statfs_active": 0, 2024-05-26T22:51:16.319 INFO:teuthology.orchestra.run.smithi131.stdout: "statfs_send": 0, 2024-05-26T22:51:16.319 INFO:teuthology.orchestra.run.smithi131.stdout: "statfs_resend": 0, 2024-05-26T22:51:16.319 INFO:teuthology.orchestra.run.smithi131.stdout: "command_active": 0, 2024-05-26T22:51:16.319 INFO:teuthology.orchestra.run.smithi131.stdout: "command_send": 0, 2024-05-26T22:51:16.319 INFO:teuthology.orchestra.run.smithi131.stdout: "command_resend": 0, 2024-05-26T22:51:16.319 INFO:teuthology.orchestra.run.smithi131.stdout: "map_epoch": 47, 2024-05-26T22:51:16.319 INFO:teuthology.orchestra.run.smithi131.stdout: "map_full": 0, 2024-05-26T22:51:16.320 INFO:teuthology.orchestra.run.smithi131.stdout: "map_inc": 3, 2024-05-26T22:51:16.320 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_sessions": 7, 2024-05-26T22:51:16.320 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_session_open": 7, 2024-05-26T22:51:16.320 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_session_close": 0, 2024-05-26T22:51:16.320 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_laggy": 0, 2024-05-26T22:51:16.320 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_wr": 17, 2024-05-26T22:51:16.320 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_rd": 0, 2024-05-26T22:51:16.320 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_del": 2 2024-05-26T22:51:16.320 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.320 INFO:teuthology.orchestra.run.smithi131.stdout: "oft": { 2024-05-26T22:51:16.320 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_total_objs": 1, 2024-05-26T22:51:16.320 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_total_kv_pairs": 0, 2024-05-26T22:51:16.320 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_total_updates": 0, 2024-05-26T22:51:16.321 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_total_removes": 0 2024-05-26T22:51:16.321 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.321 INFO:teuthology.orchestra.run.smithi131.stdout: "osd-slow-ops": { 2024-05-26T22:51:16.321 INFO:teuthology.orchestra.run.smithi131.stdout: "slow_ops_count": 0 2024-05-26T22:51:16.321 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.321 INFO:teuthology.orchestra.run.smithi131.stdout: "purge_queue": { 2024-05-26T22:51:16.321 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executing_ops": 0, 2024-05-26T22:51:16.321 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executing_ops_high_water": 0, 2024-05-26T22:51:16.321 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executing": 0, 2024-05-26T22:51:16.321 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executing_high_water": 0, 2024-05-26T22:51:16.321 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executed_ops": 0, 2024-05-26T22:51:16.321 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executed": 0, 2024-05-26T22:51:16.321 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_item_in_journal": 0 2024-05-26T22:51:16.322 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.322 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-26T22:51:16.322 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-05-26T22:51:16.322 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 104857600, 2024-05-26T22:51:16.322 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-05-26T22:51:16.322 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 168, 2024-05-26T22:51:16.322 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 212618, 2024-05-26T22:51:16.322 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-05-26T22:51:16.322 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 168, 2024-05-26T22:51:16.322 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 0, 2024-05-26T22:51:16.322 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 0, 2024-05-26T22:51:16.322 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 168, 2024-05-26T22:51:16.323 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 212618, 2024-05-26T22:51:16.323 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-05-26T22:51:16.323 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:51:16.323 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:51:16.323 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:51:16.323 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:51:16.323 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.323 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-objecter_bytes": { 2024-05-26T22:51:16.323 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-05-26T22:51:16.323 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 104857600, 2024-05-26T22:51:16.324 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-05-26T22:51:16.324 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 0, 2024-05-26T22:51:16.324 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 0, 2024-05-26T22:51:16.324 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-05-26T22:51:16.324 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 0, 2024-05-26T22:51:16.324 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 42, 2024-05-26T22:51:16.324 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 105110, 2024-05-26T22:51:16.324 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 42, 2024-05-26T22:51:16.324 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 105110, 2024-05-26T22:51:16.324 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-05-26T22:51:16.324 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:51:16.324 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:51:16.325 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:51:16.325 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:51:16.325 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.325 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-objecter_ops": { 2024-05-26T22:51:16.325 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-05-26T22:51:16.325 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 1024, 2024-05-26T22:51:16.325 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-05-26T22:51:16.325 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 0, 2024-05-26T22:51:16.325 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 0, 2024-05-26T22:51:16.326 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-05-26T22:51:16.326 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 0, 2024-05-26T22:51:16.326 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 42, 2024-05-26T22:51:16.326 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 42, 2024-05-26T22:51:16.326 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 42, 2024-05-26T22:51:16.326 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 42, 2024-05-26T22:51:16.326 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-05-26T22:51:16.326 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:51:16.326 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:51:16.326 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:51:16.326 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:51:16.327 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.327 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-write_buf_throttle": { 2024-05-26T22:51:16.327 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-05-26T22:51:16.327 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 3758096384, 2024-05-26T22:51:16.327 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-05-26T22:51:16.327 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 0, 2024-05-26T22:51:16.327 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 0, 2024-05-26T22:51:16.327 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-05-26T22:51:16.327 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 0, 2024-05-26T22:51:16.327 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 0, 2024-05-26T22:51:16.328 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 0, 2024-05-26T22:51:16.328 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 0, 2024-05-26T22:51:16.328 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 0, 2024-05-26T22:51:16.328 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-05-26T22:51:16.328 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:51:16.328 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:51:16.328 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:51:16.328 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:51:16.328 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.328 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-write_buf_throttle-0x560504ef4500": { 2024-05-26T22:51:16.328 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-05-26T22:51:16.329 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 3758096384, 2024-05-26T22:51:16.329 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-05-26T22:51:16.329 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 25, 2024-05-26T22:51:16.329 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 91415, 2024-05-26T22:51:16.329 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-05-26T22:51:16.329 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 25, 2024-05-26T22:51:16.329 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 0, 2024-05-26T22:51:16.329 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 0, 2024-05-26T22:51:16.329 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 5, 2024-05-26T22:51:16.329 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 91415, 2024-05-26T22:51:16.329 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-05-26T22:51:16.329 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:51:16.330 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:51:16.330 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:51:16.330 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:51:16.330 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:51:16.330 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-26T22:51:16.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.274+0000 7fa46ec00640 1 -- 172.21.15.131:0/2279583287 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa44806be30 msgr2=0x7fa44806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:16.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.274+0000 7fa46ec00640 1 --2- 172.21.15.131:0/2279583287 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa44806be30 0x7fa44806e2f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fa460036000 tx=0x7fa460031000 comp rx=0 tx=0).stop 2024-05-26T22:51:16.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.274+0000 7fa46ec00640 1 -- 172.21.15.131:0/2279583287 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4681554f0 msgr2=0x7fa4680ce650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:16.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.274+0000 7fa46ec00640 1 --2- 172.21.15.131:0/2279583287 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4681554f0 0x7fa4680ce650 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fa45400d800 tx=0x7fa45400dcc0 comp rx=0 tx=0).stop 2024-05-26T22:51:16.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.274+0000 7fa46ec00640 1 -- 172.21.15.131:0/2279583287 shutdown_connections 2024-05-26T22:51:16.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.274+0000 7fa46ec00640 1 --2- 172.21.15.131:0/2279583287 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa44806be30 0x7fa44806e2f0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:16.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.274+0000 7fa46ec00640 1 --2- 172.21.15.131:0/2279583287 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa46815c220 0x7fa468150120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:16.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.274+0000 7fa46ec00640 1 --2- 172.21.15.131:0/2279583287 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4681554f0 0x7fa4680ce650 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:16.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.274+0000 7fa46ec00640 1 --2- 172.21.15.131:0/2279583287 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa4680724a0 0x7fa46815a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:16.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.274+0000 7fa46ec00640 1 -- 172.21.15.131:0/2279583287 >> 172.21.15.131:0/2279583287 conn(0x7fa4680cc7f0 msgr2=0x7fa4680afaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:16.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.274+0000 7fa46ec00640 1 -- 172.21.15.131:0/2279583287 shutdown_connections 2024-05-26T22:51:16.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.274+0000 7fa46ec00640 1 -- 172.21.15.131:0/2279583287 wait complete. 2024-05-26T22:51:16.333 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 perf dump 2024-05-26T22:51:16.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.541+0000 7f8f32c00640 1 Processor -- start 2024-05-26T22:51:16.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.541+0000 7f8f32c00640 1 -- start start 2024-05-26T22:51:16.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.543+0000 7f8f32c00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8f2c0b5450 0x7f8f2c0b43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:16.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.543+0000 7f8f32c00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f8f2c06f420 0x7f8f2c0cfd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:16.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.543+0000 7f8f32c00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f2c0b4900 con 0x7f8f2c0cf590 2024-05-26T22:51:16.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.543+0000 7f8f32c00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f8f2c160d90 con 0x7f8f2c06f420 2024-05-26T22:51:16.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.543+0000 7f8f32c00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f8f2c160180 con 0x7f8f2c0b5450 2024-05-26T22:51:16.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.544+0000 7f8f2b400640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f8f2c06f420 0x7f8f2c0cfd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:16.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.544+0000 7f8f2be00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8f2c0b5450 0x7f8f2c0b43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:16.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.544+0000 7f8f2b400640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f8f2c06f420 0x7f8f2c0cfd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.131:58302/0 (socket says 172.21.15.131:58302) 2024-05-26T22:51:16.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.544+0000 7f8f2b400640 1 -- 172.21.15.131:0/1686597421 learned_addr learned my addr 172.21.15.131:0/1686597421 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:16.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.545+0000 7f8f2be00640 1 -- 172.21.15.131:0/1686597421 >> v2:172.21.15.187:3300/0 conn(0x7f8f2c06f420 msgr2=0x7f8f2c0cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:16.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.545+0000 7f8f2be00640 1 --2- 172.21.15.131:0/1686597421 >> v2:172.21.15.187:3300/0 conn(0x7f8f2c06f420 0x7f8f2c0cfd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:16.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.545+0000 7f8f2be00640 1 -- 172.21.15.131:0/1686597421 >> v1:172.21.15.187:6789/0 conn(0x7f8f2c0cf590 legacy=0x7f8f2c06d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:16.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.545+0000 7f8f2be00640 1 -- 172.21.15.131:0/1686597421 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f2c160460 con 0x7f8f2c0b5450 2024-05-26T22:51:16.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.546+0000 7f8f2be00640 1 --2- 172.21.15.131:0/1686597421 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8f2c0b5450 0x7f8f2c0b43c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c009d90 tx=0x7f8f1c02a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4bfd2fe706b46251 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:16.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.547+0000 7f8f2aa00640 1 -- 172.21.15.131:0/1686597421 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f1c007740 con 0x7f8f2c0b5450 2024-05-26T22:51:16.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.547+0000 7f8f2aa00640 1 -- 172.21.15.131:0/1686597421 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f1c0078e0 con 0x7f8f2c0b5450 2024-05-26T22:51:16.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.547+0000 7f8f2aa00640 1 -- 172.21.15.131:0/1686597421 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f1c007cb0 con 0x7f8f2c0b5450 2024-05-26T22:51:16.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.547+0000 7f8f32c00640 1 -- 172.21.15.131:0/1686597421 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8f2c0b5450 msgr2=0x7f8f2c0b43c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:16.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.547+0000 7f8f32c00640 1 --2- 172.21.15.131:0/1686597421 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8f2c0b5450 0x7f8f2c0b43c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c009d90 tx=0x7f8f1c02a660 comp rx=0 tx=0).stop 2024-05-26T22:51:16.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.548+0000 7f8f32c00640 1 -- 172.21.15.131:0/1686597421 shutdown_connections 2024-05-26T22:51:16.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.548+0000 7f8f32c00640 1 --2- 172.21.15.131:0/1686597421 >> v2:172.21.15.187:3300/0 conn(0x7f8f2c06f420 0x7f8f2c0cfd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:16.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.548+0000 7f8f32c00640 1 --2- 172.21.15.131:0/1686597421 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8f2c0b5450 0x7f8f2c0b43c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:16.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.548+0000 7f8f32c00640 1 -- 172.21.15.131:0/1686597421 >> 172.21.15.131:0/1686597421 conn(0x7f8f2c06bc90 msgr2=0x7f8f2c0b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:16.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.548+0000 7f8f32c00640 1 -- 172.21.15.131:0/1686597421 shutdown_connections 2024-05-26T22:51:16.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.548+0000 7f8f32c00640 1 -- 172.21.15.131:0/1686597421 wait complete. 2024-05-26T22:51:16.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.549+0000 7f8f32c00640 1 Processor -- start 2024-05-26T22:51:16.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.550+0000 7f8f32c00640 1 -- start start 2024-05-26T22:51:16.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.550+0000 7f8f32c00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8f2c06f420 0x7f8f2c1518c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:16.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.550+0000 7f8f32c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f2c0b5450 0x7f8f2c1f20c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:16.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.550+0000 7f8f32c00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8f2c0cf590 0x7f8f2c15d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:16.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.550+0000 7f8f32c00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8f2c0b5830 con 0x7f8f2c0b5450 2024-05-26T22:51:16.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.550+0000 7f8f32c00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8f2c160a20 con 0x7f8f2c06f420 2024-05-26T22:51:16.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.550+0000 7f8f32c00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f8f2c15fde0 con 0x7f8f2c0cf590 2024-05-26T22:51:16.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.551+0000 7f8f2b400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f2c0b5450 0x7f8f2c1f20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:16.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.551+0000 7f8f2b400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f2c0b5450 0x7f8f2c1f20c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44802/0 (socket says 172.21.15.131:44802) 2024-05-26T22:51:16.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.551+0000 7f8f2b400640 1 -- 172.21.15.131:0/713485266 learned_addr learned my addr 172.21.15.131:0/713485266 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:16.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.551+0000 7f8f2be00640 1 --2- 172.21.15.131:0/713485266 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8f2c06f420 0x7f8f2c1518c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:16.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.551+0000 7f8f2b400640 1 -- 172.21.15.131:0/713485266 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8f2c0cf590 msgr2=0x7f8f2c15d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:16.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.551+0000 7f8f2b400640 1 --2- 172.21.15.131:0/713485266 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8f2c0cf590 0x7f8f2c15d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:16.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.551+0000 7f8f2b400640 1 -- 172.21.15.131:0/713485266 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8f2c06f420 msgr2=0x7f8f2c1518c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:16.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.551+0000 7f8f2b400640 1 --2- 172.21.15.131:0/713485266 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8f2c06f420 0x7f8f2c1518c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:16.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.551+0000 7f8f2b400640 1 -- 172.21.15.131:0/713485266 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f2c1f3020 con 0x7f8f2c0b5450 2024-05-26T22:51:16.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.551+0000 7f8f2b400640 1 --2- 172.21.15.131:0/713485266 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f2c0b5450 0x7f8f2c1f20c0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f8f1400e840 tx=0x7f8f1400ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:16.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.552+0000 7f8f23e00640 1 -- 172.21.15.131:0/713485266 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f14017070 con 0x7f8f2c0b5450 2024-05-26T22:51:16.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.552+0000 7f8f23e00640 1 -- 172.21.15.131:0/713485266 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f1400c8e0 con 0x7f8f2c0b5450 2024-05-26T22:51:16.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.552+0000 7f8f23e00640 1 -- 172.21.15.131:0/713485266 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f1400ccd0 con 0x7f8f2c0b5450 2024-05-26T22:51:16.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.552+0000 7f8f32c00640 1 -- 172.21.15.131:0/713485266 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f2c1f2a60 con 0x7f8f2c0b5450 2024-05-26T22:51:16.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.552+0000 7f8f32c00640 1 -- 172.21.15.131:0/713485266 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f2c157280 con 0x7f8f2c0b5450 2024-05-26T22:51:16.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.554+0000 7f8f23e00640 1 -- 172.21.15.131:0/713485266 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8f1401a070 con 0x7f8f2c0b5450 2024-05-26T22:51:16.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.555+0000 7f8f23e00640 1 --2- 172.21.15.131:0/713485266 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f8f0806be30 0x7f8f0806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:16.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.555+0000 7f8f23e00640 1 -- 172.21.15.131:0/713485266 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5853+0+0 (secure 0 0 0) 0x7f8f1408adc0 con 0x7f8f2c0b5450 2024-05-26T22:51:16.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.556+0000 7f8f2be00640 1 --2- 172.21.15.131:0/713485266 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f8f0806be30 0x7f8f0806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:16.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.556+0000 7f8f35a50740 1 Processor -- start 2024-05-26T22:51:16.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.556+0000 7f8f35a50740 1 -- start start 2024-05-26T22:51:16.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.556+0000 7f8f2be00640 1 --2- 172.21.15.131:0/713485266 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f8f0806be30 0x7f8f0806e2f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c034000 tx=0x7f8f1c02f000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:16.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.557+0000 7f8f35a50740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ab1e2d220 0x562ab1ddfbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:16.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.557+0000 7f8f35a50740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562ab1e10340 0x562ab1e0f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:16.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.557+0000 7f8f35a50740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x562ab1e0fc00 0x562ab1de4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:16.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.557+0000 7f8f35a50740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x562ab1d48960 con 0x562ab1e2d220 2024-05-26T22:51:16.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.557+0000 7f8f35a50740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x562ab1cf00d0 con 0x562ab1e10340 2024-05-26T22:51:16.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.557+0000 7f8f35a50740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x562ab1d44920 con 0x562ab1e0fc00 2024-05-26T22:51:16.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.557+0000 7f8f2be00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ab1e2d220 0x562ab1ddfbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:16.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.557+0000 7f8f2be00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ab1e2d220 0x562ab1ddfbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44812/0 (socket says 172.21.15.131:44812) 2024-05-26T22:51:16.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.557+0000 7f8f2be00640 1 -- 172.21.15.131:0/230835214 learned_addr learned my addr 172.21.15.131:0/230835214 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:16.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.557+0000 7f8f2be00640 1 -- 172.21.15.131:0/230835214 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x562ab1e0fc00 msgr2=0x562ab1de4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:16.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.557+0000 7f8f2be00640 1 --2- 172.21.15.131:0/230835214 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x562ab1e0fc00 0x562ab1de4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:16.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.557+0000 7f8f2be00640 1 -- 172.21.15.131:0/230835214 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562ab1e10340 msgr2=0x562ab1e0f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:16.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.557+0000 7f8f2b400640 1 --2- 172.21.15.131:0/230835214 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562ab1e10340 0x562ab1e0f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:16.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.558+0000 7f8f2be00640 1 --2- 172.21.15.131:0/230835214 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562ab1e10340 0x562ab1e0f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:16.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.558+0000 7f8f2be00640 1 -- 172.21.15.131:0/230835214 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ab1b267d0 con 0x562ab1e2d220 2024-05-26T22:51:16.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.558+0000 7f8f2b400640 1 --2- 172.21.15.131:0/230835214 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562ab1e10340 0x562ab1e0f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:51:16.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.558+0000 7f8f2be00640 1 --2- 172.21.15.131:0/230835214 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ab1e2d220 0x562ab1ddfbb0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c02ac50 tx=0x7f8f1c02e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24b34c425bda7a05 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:16.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.558+0000 7f8f1be00640 1 -- 172.21.15.131:0/230835214 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f1c04c070 con 0x562ab1e2d220 2024-05-26T22:51:16.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.558+0000 7f8f1be00640 1 -- 172.21.15.131:0/230835214 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f1c03c6e0 con 0x562ab1e2d220 2024-05-26T22:51:16.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.558+0000 7f8f1be00640 1 -- 172.21.15.131:0/230835214 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f1c03ca90 con 0x562ab1e2d220 2024-05-26T22:51:16.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.559+0000 7f8f35a50740 1 -- 172.21.15.131:0/230835214 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ab1e2d220 msgr2=0x562ab1ddfbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:16.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.559+0000 7f8f35a50740 1 --2- 172.21.15.131:0/230835214 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ab1e2d220 0x562ab1ddfbb0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c02ac50 tx=0x7f8f1c02e000 comp rx=0 tx=0).stop 2024-05-26T22:51:16.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.559+0000 7f8f35a50740 1 -- 172.21.15.131:0/230835214 shutdown_connections 2024-05-26T22:51:16.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.559+0000 7f8f35a50740 1 --2- 172.21.15.131:0/230835214 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x562ab1e0fc00 0x562ab1de4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:16.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.559+0000 7f8f35a50740 1 --2- 172.21.15.131:0/230835214 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562ab1e10340 0x562ab1e0f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:16.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.559+0000 7f8f35a50740 1 --2- 172.21.15.131:0/230835214 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ab1e2d220 0x562ab1ddfbb0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:16.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.559+0000 7f8f35a50740 1 -- 172.21.15.131:0/230835214 >> 172.21.15.131:0/230835214 conn(0x562ab1de1e20 msgr2=0x562ab1e2a6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:16.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.559+0000 7f8f35a50740 1 -- 172.21.15.131:0/230835214 shutdown_connections 2024-05-26T22:51:16.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.559+0000 7f8f35a50740 1 -- 172.21.15.131:0/230835214 wait complete. 2024-05-26T22:51:16.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.560+0000 7f8f35a50740 1 -- start start 2024-05-26T22:51:16.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.560+0000 7f8f35a50740 1 Processor -- start 2024-05-26T22:51:16.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.561+0000 7f8f35a50740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x562ab1e10340 0x562ab1e0b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:16.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.561+0000 7f8f35a50740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562ab1e2d220 0x562ab1e127f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:16.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.561+0000 7f8f35a50740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ab1e3a6b0 0x562ab1de4840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:16.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.561+0000 7f8f35a50740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x562ab1de3fd0 con 0x562ab1e3a6b0 2024-05-26T22:51:16.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.561+0000 7f8f35a50740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x562ab1d034a0 con 0x562ab1e2d220 2024-05-26T22:51:16.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.561+0000 7f8f35a50740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x562ab1de33c0 con 0x562ab1e10340 2024-05-26T22:51:16.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.561+0000 7f8f35a50740 10 client.0 _ll_register_callbacks cb 0x562ab1d0ac40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-26T22:51:16.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.561+0000 7f8f30c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ab1e3a6b0 0x562ab1de4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:16.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.561+0000 7f8f30c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ab1e3a6b0 0x562ab1de4840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44820/0 (socket says 172.21.15.131:44820) 2024-05-26T22:51:16.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.561+0000 7f8f30c00640 1 -- 172.21.15.131:0/3061367349 learned_addr learned my addr 172.21.15.131:0/3061367349 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:16.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.562+0000 7f8f2b400640 1 --2- 172.21.15.131:0/3061367349 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562ab1e2d220 0x562ab1e127f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:16.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.562+0000 7f8f30c00640 1 -- 172.21.15.131:0/3061367349 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x562ab1e10340 msgr2=0x562ab1e0b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:16.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.562+0000 7f8f30c00640 1 --2- 172.21.15.131:0/3061367349 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x562ab1e10340 0x562ab1e0b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:16.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.562+0000 7f8f30c00640 1 -- 172.21.15.131:0/3061367349 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562ab1e2d220 msgr2=0x562ab1e127f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:16.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.562+0000 7f8f30c00640 1 --2- 172.21.15.131:0/3061367349 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562ab1e2d220 0x562ab1e127f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:16.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.562+0000 7f8f30c00640 1 -- 172.21.15.131:0/3061367349 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ab1a1c0f0 con 0x562ab1e3a6b0 2024-05-26T22:51:16.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.562+0000 7f8f2b400640 1 --2- 172.21.15.131:0/3061367349 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562ab1e2d220 0x562ab1e127f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:51:16.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.562+0000 7f8f30c00640 1 --2- 172.21.15.131:0/3061367349 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ab1e3a6b0 0x562ab1de4840 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f8f24001da0 tx=0x7f8f2400c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=309d3f5f84e2e545 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:16.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.562+0000 7f8f18c00640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-26T22:51:16.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.562+0000 7f8f18c00640 1 -- 172.21.15.131:0/3061367349 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f24017060 con 0x562ab1e3a6b0 2024-05-26T22:51:16.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.562+0000 7f8f35a50740 1 -- 172.21.15.131:0/3061367349 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f1c03c6e0 con 0x562ab1e3a6b0 2024-05-26T22:51:16.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.562+0000 7f8f18c00640 1 -- 172.21.15.131:0/3061367349 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(47..47 src has 1..47) ==== 5853+0+0 (secure 0 0 0) 0x7f8f240052d0 con 0x562ab1e3a6b0 2024-05-26T22:51:16.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.563+0000 7f8f18c00640 1 -- 172.21.15.131:0/3061367349 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f8f2401d050 con 0x562ab1e3a6b0 2024-05-26T22:51:16.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.563+0000 7f8f35a50740 10 client.6132 fetch_fsmap learned FSMap version 40 2024-05-26T22:51:16.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.563+0000 7f8f35a50740 1 -- 172.21.15.131:0/3061367349 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=2+}) -- 0x562ab1b86f60 con 0x562ab1e3a6b0 2024-05-26T22:51:16.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.563+0000 7f8f18c00640 1 -- 172.21.15.131:0/3061367349 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f1c03c6e0 con 0x562ab1e3a6b0 2024-05-26T22:51:16.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.563+0000 7f8f18c00640 1 -- 172.21.15.131:0/3061367349 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 40) ==== 3542+0+0 (secure 0 0 0) 0x7f8f24020020 con 0x562ab1e3a6b0 2024-05-26T22:51:16.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.563+0000 7f8f35a50740 10 client.6132 fetch_fsmap finished waiting for FSMap version 40 2024-05-26T22:51:16.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.563+0000 7f8f35a50740 10 client.6132 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-05-26T22:51:16.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.563+0000 7f8f35a50740 20 client.6132 populate_metadata read hostname 'smithi131' 2024-05-26T22:51:16.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.563+0000 7f8f35a50740 1 --2- 172.21.15.131:0/3061367349 >> [v2:172.21.15.131:6834/650770835,v1:172.21.15.131:6836/650770835] conn(0x562ab1e221b0 0x562ab1e0be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:16.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.563+0000 7f8f35a50740 4 client.6132 mds_command: new command op to 5637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-26T22:51:16.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.563+0000 7f8f35a50740 1 -- 172.21.15.131:0/3061367349 --> [v2:172.21.15.131:6834/650770835,v1:172.21.15.131:6836/650770835] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562ab1d56090 con 0x562ab1e221b0 2024-05-26T22:51:16.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.563+0000 7f8f2be00640 1 --2- 172.21.15.131:0/3061367349 >> [v2:172.21.15.131:6834/650770835,v1:172.21.15.131:6836/650770835] conn(0x562ab1e221b0 0x562ab1e0be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:16.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.564+0000 7f8f2be00640 1 --2- 172.21.15.131:0/3061367349 >> [v2:172.21.15.131:6834/650770835,v1:172.21.15.131:6836/650770835] conn(0x562ab1e221b0 0x562ab1e0be20 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92d8cbdee3006da2 server_cookie=398ea3c221089c10 in_seq=0 out_seq=0 2024-05-26T22:51:16.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.564+0000 7f8f18c00640 10 client.6132 ms_handle_connect on v2:172.21.15.131:6834/650770835 2024-05-26T22:51:16.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.565+0000 7f8f18c00640 1 -- 172.21.15.131:0/3061367349 <== mds.0 v2:172.21.15.131:6834/650770835 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562ab1d56090 con 0x562ab1e221b0 2024-05-26T22:51:16.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.565+0000 7f8f18c00640 10 client.6132 handle_command_reply: tid=0 2024-05-26T22:51:16.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.565+0000 7f8f35a50740 1 client.6132 shutdown 2024-05-26T22:51:16.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.565+0000 7f8f35a50740 1 -- 172.21.15.131:0/3061367349 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ab1e3a6b0 msgr2=0x562ab1de4840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:16.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.565+0000 7f8f35a50740 1 --2- 172.21.15.131:0/3061367349 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ab1e3a6b0 0x562ab1de4840 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f8f24001da0 tx=0x7f8f2400c4b0 comp rx=0 tx=0).stop 2024-05-26T22:51:16.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.565+0000 7f8f35a50740 1 -- 172.21.15.131:0/3061367349 shutdown_connections 2024-05-26T22:51:16.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.565+0000 7f8f35a50740 1 --2- 172.21.15.131:0/3061367349 >> [v2:172.21.15.131:6834/650770835,v1:172.21.15.131:6836/650770835] conn(0x562ab1e221b0 0x562ab1e0be20 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:16.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.565+0000 7f8f35a50740 1 --2- 172.21.15.131:0/3061367349 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ab1e3a6b0 0x562ab1de4840 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:16.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.565+0000 7f8f35a50740 1 --2- 172.21.15.131:0/3061367349 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562ab1e2d220 0x562ab1e127f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:16.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.565+0000 7f8f35a50740 1 --2- 172.21.15.131:0/3061367349 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x562ab1e10340 0x562ab1e0b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:16.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.565+0000 7f8f35a50740 1 -- 172.21.15.131:0/3061367349 >> 172.21.15.131:0/3061367349 conn(0x562ab1e0fc00 msgr2=0x562ab1e2c8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:16.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.565+0000 7f8f18c00640 0 client.6132 ms_handle_reset on v2:172.21.15.131:6834/650770835 2024-05-26T22:51:16.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.565+0000 7f8f35a50740 1 -- 172.21.15.131:0/3061367349 shutdown_connections 2024-05-26T22:51:16.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.565+0000 7f8f35a50740 1 -- 172.21.15.131:0/3061367349 wait complete. 2024-05-26T22:51:16.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.565+0000 7f8f35a50740 20 client.6132 trim_cache size 0 max 16384 2024-05-26T22:51:16.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.583+0000 7f8f35a50740 1 Processor -- start 2024-05-26T22:51:16.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.583+0000 7f8f35a50740 1 -- start start 2024-05-26T22:51:16.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.584+0000 7f8f35a50740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562ab1e10340 0x562ab1e2a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:16.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.584+0000 7f8f35a50740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ab1e2d220 0x562ab1e440f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:16.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.584+0000 7f8f35a50740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x562ab1e3a6b0 0x562ab1e44630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:16.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.584+0000 7f8f35a50740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8f2c1fe9c0 con 0x562ab1e2d220 2024-05-26T22:51:16.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.584+0000 7f8f35a50740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8f2c1fdbc0 con 0x562ab1e10340 2024-05-26T22:51:16.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.584+0000 7f8f35a50740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f8f2c1fe290 con 0x562ab1e3a6b0 2024-05-26T22:51:16.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.585+0000 7f8f2b400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ab1e2d220 0x562ab1e440f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:16.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.585+0000 7f8f2b400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ab1e2d220 0x562ab1e440f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44834/0 (socket says 172.21.15.131:44834) 2024-05-26T22:51:16.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.585+0000 7f8f2b400640 1 -- 172.21.15.131:0/3837212238 learned_addr learned my addr 172.21.15.131:0/3837212238 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:16.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.585+0000 7f8f2be00640 1 --2- 172.21.15.131:0/3837212238 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562ab1e10340 0x562ab1e2a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:16.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.585+0000 7f8f30c00640 1 --2- 172.21.15.131:0/3837212238 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x562ab1e3a6b0 0x562ab1e44630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:16.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.585+0000 7f8f2b400640 1 -- 172.21.15.131:0/3837212238 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x562ab1e3a6b0 msgr2=0x562ab1e44630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:16.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.585+0000 7f8f2b400640 1 --2- 172.21.15.131:0/3837212238 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x562ab1e3a6b0 0x562ab1e44630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:16.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.585+0000 7f8f2b400640 1 -- 172.21.15.131:0/3837212238 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562ab1e10340 msgr2=0x562ab1e2a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:16.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.585+0000 7f8f2b400640 1 --2- 172.21.15.131:0/3837212238 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562ab1e10340 0x562ab1e2a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:16.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.585+0000 7f8f2b400640 1 -- 172.21.15.131:0/3837212238 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ab19f51b0 con 0x562ab1e2d220 2024-05-26T22:51:16.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.585+0000 7f8f2b400640 1 --2- 172.21.15.131:0/3837212238 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ab1e2d220 0x562ab1e440f0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f8f14000c00 tx=0x7f8f140596d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d564872da03ec707 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:16.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.586+0000 7f8f1b400640 1 -- 172.21.15.131:0/3837212238 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f14021030 con 0x562ab1e2d220 2024-05-26T22:51:16.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.586+0000 7f8f1b400640 1 -- 172.21.15.131:0/3837212238 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f14071020 con 0x562ab1e2d220 2024-05-26T22:51:16.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.586+0000 7f8f1b400640 1 -- 172.21.15.131:0/3837212238 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f1405d380 con 0x562ab1e2d220 2024-05-26T22:51:16.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.586+0000 7f8f35a50740 1 -- 172.21.15.131:0/3837212238 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ab1e2d220 msgr2=0x562ab1e440f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:16.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.586+0000 7f8f35a50740 1 --2- 172.21.15.131:0/3837212238 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ab1e2d220 0x562ab1e440f0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f8f14000c00 tx=0x7f8f140596d0 comp rx=0 tx=0).stop 2024-05-26T22:51:16.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.586+0000 7f8f35a50740 1 -- 172.21.15.131:0/3837212238 shutdown_connections 2024-05-26T22:51:16.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.586+0000 7f8f35a50740 1 --2- 172.21.15.131:0/3837212238 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x562ab1e3a6b0 0x562ab1e44630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:16.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.586+0000 7f8f35a50740 1 --2- 172.21.15.131:0/3837212238 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ab1e2d220 0x562ab1e440f0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:16.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.586+0000 7f8f35a50740 1 --2- 172.21.15.131:0/3837212238 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562ab1e10340 0x562ab1e2a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:16.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.586+0000 7f8f35a50740 1 -- 172.21.15.131:0/3837212238 >> 172.21.15.131:0/3837212238 conn(0x562ab1e0fc00 msgr2=0x562ab1e12100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:16.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.586+0000 7f8f35a50740 1 -- 172.21.15.131:0/3837212238 shutdown_connections 2024-05-26T22:51:16.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.587+0000 7f8f35a50740 1 -- 172.21.15.131:0/3837212238 wait complete. 2024-05-26T22:51:16.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.587+0000 7f8f35a50740 1 -- start start 2024-05-26T22:51:16.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.587+0000 7f8f35a50740 1 Processor -- start 2024-05-26T22:51:16.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.588+0000 7f8f35a50740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ab1e10340 0x562ab1e2b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:16.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.588+0000 7f8f35a50740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562ab1e2d220 0x562ab1e0a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:16.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.588+0000 7f8f35a50740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x562ab1e3a6b0 0x562ab1ddf500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:16.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.588+0000 7f8f35a50740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x562ab1de5de0 con 0x562ab1e10340 2024-05-26T22:51:16.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.588+0000 7f8f35a50740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8f2c158390 con 0x562ab1e2d220 2024-05-26T22:51:16.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.588+0000 7f8f35a50740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x562ab1e29410 con 0x562ab1e3a6b0 2024-05-26T22:51:16.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.589+0000 7f8f35a50740 10 client.0 _ll_register_callbacks cb 0x7f8f24007c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-26T22:51:16.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.589+0000 7f8f2be00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ab1e10340 0x562ab1e2b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:16.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.589+0000 7f8f2be00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ab1e10340 0x562ab1e2b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44848/0 (socket says 172.21.15.131:44848) 2024-05-26T22:51:16.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.589+0000 7f8f2be00640 1 -- 172.21.15.131:0/815768796 learned_addr learned my addr 172.21.15.131:0/815768796 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:16.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.589+0000 7f8f2be00640 1 -- 172.21.15.131:0/815768796 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x562ab1e3a6b0 msgr2=0x562ab1ddf500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:16.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.589+0000 7f8f2be00640 1 --2- 172.21.15.131:0/815768796 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x562ab1e3a6b0 0x562ab1ddf500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:16.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.589+0000 7f8f2be00640 1 -- 172.21.15.131:0/815768796 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562ab1e2d220 msgr2=0x562ab1e0a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:16.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.589+0000 7f8f2be00640 1 --2- 172.21.15.131:0/815768796 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562ab1e2d220 0x562ab1e0a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:16.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.589+0000 7f8f2be00640 1 -- 172.21.15.131:0/815768796 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ab1e44860 con 0x562ab1e10340 2024-05-26T22:51:16.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.589+0000 7f8f2be00640 1 --2- 172.21.15.131:0/815768796 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ab1e10340 0x562ab1e2b680 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c009d90 tx=0x7f8f1c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47473b5e0df992f server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:16.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.589+0000 7f8f18c00640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-26T22:51:16.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.590+0000 7f8f18c00640 1 -- 172.21.15.131:0/815768796 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f1c04c040 con 0x562ab1e10340 2024-05-26T22:51:16.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.590+0000 7f8f18c00640 1 -- 172.21.15.131:0/815768796 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(47..47 src has 1..47) ==== 5853+0+0 (secure 0 0 0) 0x7f8f1c03cc70 con 0x562ab1e10340 2024-05-26T22:51:16.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.590+0000 7f8f35a50740 1 -- 172.21.15.131:0/815768796 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f14071020 con 0x562ab1e10340 2024-05-26T22:51:16.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.590+0000 7f8f18c00640 1 -- 172.21.15.131:0/815768796 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f8f1c051460 con 0x562ab1e10340 2024-05-26T22:51:16.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.590+0000 7f8f35a50740 10 client.6138 fetch_fsmap learned FSMap version 40 2024-05-26T22:51:16.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.590+0000 7f8f35a50740 1 -- 172.21.15.131:0/815768796 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=2+}) -- 0x562ab1e133b0 con 0x562ab1e10340 2024-05-26T22:51:16.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.590+0000 7f8f18c00640 1 -- 172.21.15.131:0/815768796 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f14071020 con 0x562ab1e10340 2024-05-26T22:51:16.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.590+0000 7f8f18c00640 1 -- 172.21.15.131:0/815768796 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 40) ==== 3542+0+0 (secure 0 0 0) 0x7f8f1c069080 con 0x562ab1e10340 2024-05-26T22:51:16.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.590+0000 7f8f35a50740 10 client.6138 fetch_fsmap finished waiting for FSMap version 40 2024-05-26T22:51:16.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.590+0000 7f8f35a50740 10 client.6138 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-05-26T22:51:16.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.590+0000 7f8f35a50740 20 client.6138 populate_metadata read hostname 'smithi131' 2024-05-26T22:51:16.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.590+0000 7f8f35a50740 1 --2- 172.21.15.131:0/815768796 >> [v2:172.21.15.131:6834/650770835,v1:172.21.15.131:6836/650770835] conn(0x562ab1e0a5a0 0x562ab1ddfa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:16.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.590+0000 7f8f35a50740 4 client.6138 mds_command: new command op to 5637 tid=0 multi_id=0 [{"prefix": "perf dump"}] 2024-05-26T22:51:16.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.590+0000 7f8f35a50740 1 -- 172.21.15.131:0/815768796 --> [v2:172.21.15.131:6834/650770835,v1:172.21.15.131:6836/650770835] -- command(tid 0: {"prefix": "perf dump"}) -- 0x562ab1d503e0 con 0x562ab1e0a5a0 2024-05-26T22:51:16.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.590+0000 7f8f2b400640 1 --2- 172.21.15.131:0/815768796 >> [v2:172.21.15.131:6834/650770835,v1:172.21.15.131:6836/650770835] conn(0x562ab1e0a5a0 0x562ab1ddfa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:16.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.591+0000 7f8f2b400640 1 --2- 172.21.15.131:0/815768796 >> [v2:172.21.15.131:6834/650770835,v1:172.21.15.131:6836/650770835] conn(0x562ab1e0a5a0 0x562ab1ddfa40 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e363a310b7424fa6 server_cookie=ac2e3d2c6eae322b in_seq=0 out_seq=0 2024-05-26T22:51:16.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.591+0000 7f8f18c00640 10 client.6138 ms_handle_connect on v2:172.21.15.131:6834/650770835 2024-05-26T22:51:16.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.592+0000 7f8f18c00640 1 -- 172.21.15.131:0/815768796 <== mds.0 v2:172.21.15.131:6834/650770835 1 ==== command_reply(tid 0: 0 ) ==== 8+0+17864 (crc 0 0 0) 0x562ab1d503e0 con 0x562ab1e0a5a0 2024-05-26T22:51:16.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.592+0000 7f8f18c00640 10 client.6138 handle_command_reply: tid=0 2024-05-26T22:51:16.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.592+0000 7f8f35a50740 1 client.6138 shutdown 2024-05-26T22:51:16.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.592+0000 7f8f35a50740 1 -- 172.21.15.131:0/815768796 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ab1e10340 msgr2=0x562ab1e2b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:16.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.592+0000 7f8f35a50740 1 --2- 172.21.15.131:0/815768796 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ab1e10340 0x562ab1e2b680 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c009d90 tx=0x7f8f1c004a40 comp rx=0 tx=0).stop 2024-05-26T22:51:16.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.592+0000 7f8f35a50740 1 -- 172.21.15.131:0/815768796 shutdown_connections 2024-05-26T22:51:16.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.592+0000 7f8f35a50740 1 --2- 172.21.15.131:0/815768796 >> [v2:172.21.15.131:6834/650770835,v1:172.21.15.131:6836/650770835] conn(0x562ab1e0a5a0 0x562ab1ddfa40 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:16.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.592+0000 7f8f35a50740 1 --2- 172.21.15.131:0/815768796 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x562ab1e3a6b0 0x562ab1ddf500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:16.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.592+0000 7f8f35a50740 1 --2- 172.21.15.131:0/815768796 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562ab1e2d220 0x562ab1e0a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:16.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.592+0000 7f8f35a50740 1 --2- 172.21.15.131:0/815768796 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562ab1e10340 0x562ab1e2b680 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:16.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.592+0000 7f8f18c00640 0 client.6138 ms_handle_reset on v2:172.21.15.131:6834/650770835 2024-05-26T22:51:16.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.592+0000 7f8f35a50740 1 -- 172.21.15.131:0/815768796 >> 172.21.15.131:0/815768796 conn(0x562ab1e0fc00 msgr2=0x562ab1e4e240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:16.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.592+0000 7f8f35a50740 1 -- 172.21.15.131:0/815768796 shutdown_connections 2024-05-26T22:51:16.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.592+0000 7f8f35a50740 1 -- 172.21.15.131:0/815768796 wait complete. 2024-05-26T22:51:16.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.592+0000 7f8f35a50740 20 client.6138 trim_cache size 0 max 16384 2024-05-26T22:51:16.597 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-26T22:51:16.597 INFO:teuthology.orchestra.run.smithi131.stdout: "AsyncMessenger::Worker-0": { 2024-05-26T22:51:16.597 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_messages": 188, 2024-05-26T22:51:16.597 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages": 193, 2024-05-26T22:51:16.597 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_bytes": 51299, 2024-05-26T22:51:16.597 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_bytes": 4350291, 2024-05-26T22:51:16.597 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_created_connections": 14, 2024-05-26T22:51:16.597 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_active_connections": 5, 2024-05-26T22:51:16.597 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_total_time": 0.065923677, 2024-05-26T22:51:16.598 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_send_time": 0.028607012, 2024-05-26T22:51:16.598 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_recv_time": 0.020309562, 2024-05-26T22:51:16.598 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_fast_dispatch_time": 0.010518528, 2024-05-26T22:51:16.598 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages_queue_lat": { 2024-05-26T22:51:16.598 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 185, 2024-05-26T22:51:16.598 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.218477191, 2024-05-26T22:51:16.598 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.001180957 2024-05-26T22:51:16.598 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.598 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_handle_ack_lat": { 2024-05-26T22:51:16.598 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 26, 2024-05-26T22:51:16.598 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000006250, 2024-05-26T22:51:16.599 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000240 2024-05-26T22:51:16.599 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.599 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-26T22:51:16.599 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_encrypted_bytes": 0 2024-05-26T22:51:16.599 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.599 INFO:teuthology.orchestra.run.smithi131.stdout: "AsyncMessenger::Worker-1": { 2024-05-26T22:51:16.599 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_messages": 2124, 2024-05-26T22:51:16.599 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages": 4559, 2024-05-26T22:51:16.599 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_bytes": 804184, 2024-05-26T22:51:16.599 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_bytes": 3221093, 2024-05-26T22:51:16.599 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_created_connections": 13, 2024-05-26T22:51:16.599 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_active_connections": 7, 2024-05-26T22:51:16.600 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_total_time": 0.740265829, 2024-05-26T22:51:16.600 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_send_time": 0.439300540, 2024-05-26T22:51:16.600 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_recv_time": 0.362975574, 2024-05-26T22:51:16.600 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_fast_dispatch_time": 0.010359896, 2024-05-26T22:51:16.600 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages_queue_lat": { 2024-05-26T22:51:16.600 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 4560, 2024-05-26T22:51:16.600 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.352015459, 2024-05-26T22:51:16.600 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000077196 2024-05-26T22:51:16.600 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.600 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_handle_ack_lat": { 2024-05-26T22:51:16.600 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 5429, 2024-05-26T22:51:16.600 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.002725191, 2024-05-26T22:51:16.600 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000501 2024-05-26T22:51:16.601 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.601 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-26T22:51:16.601 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_encrypted_bytes": 106064 2024-05-26T22:51:16.601 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.601 INFO:teuthology.orchestra.run.smithi131.stdout: "AsyncMessenger::Worker-2": { 2024-05-26T22:51:16.601 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_messages": 455, 2024-05-26T22:51:16.601 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages": 413, 2024-05-26T22:51:16.601 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_bytes": 245967, 2024-05-26T22:51:16.601 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_bytes": 221476, 2024-05-26T22:51:16.601 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_created_connections": 6, 2024-05-26T22:51:16.601 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_active_connections": 4, 2024-05-26T22:51:16.601 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_total_time": 0.067079740, 2024-05-26T22:51:16.602 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_send_time": 0.024809698, 2024-05-26T22:51:16.602 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_recv_time": 0.032748258, 2024-05-26T22:51:16.602 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_fast_dispatch_time": 0.012744007, 2024-05-26T22:51:16.602 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages_queue_lat": { 2024-05-26T22:51:16.602 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 416, 2024-05-26T22:51:16.602 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.238932531, 2024-05-26T22:51:16.602 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000574357 2024-05-26T22:51:16.602 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.602 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_handle_ack_lat": { 2024-05-26T22:51:16.602 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 99, 2024-05-26T22:51:16.602 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000057539, 2024-05-26T22:51:16.602 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000581 2024-05-26T22:51:16.603 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.603 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_encrypted_bytes": 146176, 2024-05-26T22:51:16.603 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_encrypted_bytes": 18624 2024-05-26T22:51:16.603 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.603 INFO:teuthology.orchestra.run.smithi131.stdout: "cct": { 2024-05-26T22:51:16.603 INFO:teuthology.orchestra.run.smithi131.stdout: "total_workers": 1, 2024-05-26T22:51:16.603 INFO:teuthology.orchestra.run.smithi131.stdout: "unhealthy_workers": 0 2024-05-26T22:51:16.603 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.603 INFO:teuthology.orchestra.run.smithi131.stdout: "finisher-MDSRank": { 2024-05-26T22:51:16.603 INFO:teuthology.orchestra.run.smithi131.stdout: "queue_len": 0, 2024-05-26T22:51:16.603 INFO:teuthology.orchestra.run.smithi131.stdout: "complete_latency": { 2024-05-26T22:51:16.603 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 111, 2024-05-26T22:51:16.603 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.866540392, 2024-05-26T22:51:16.604 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.007806670 2024-05-26T22:51:16.604 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:51:16.604 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.604 INFO:teuthology.orchestra.run.smithi131.stdout: "finisher-PurgeQueue": { 2024-05-26T22:51:16.604 INFO:teuthology.orchestra.run.smithi131.stdout: "queue_len": 0, 2024-05-26T22:51:16.604 INFO:teuthology.orchestra.run.smithi131.stdout: "complete_latency": { 2024-05-26T22:51:16.604 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 676, 2024-05-26T22:51:16.604 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.095667067, 2024-05-26T22:51:16.604 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000141519 2024-05-26T22:51:16.604 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:51:16.604 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.604 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": { 2024-05-26T22:51:16.604 INFO:teuthology.orchestra.run.smithi131.stdout: "request": 1258, 2024-05-26T22:51:16.605 INFO:teuthology.orchestra.run.smithi131.stdout: "reply": 1258, 2024-05-26T22:51:16.605 INFO:teuthology.orchestra.run.smithi131.stdout: "reply_latency": { 2024-05-26T22:51:16.605 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 1258, 2024-05-26T22:51:16.605 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 2.189444543, 2024-05-26T22:51:16.605 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.001740416 2024-05-26T22:51:16.605 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.605 INFO:teuthology.orchestra.run.smithi131.stdout: "slow_reply": 0, 2024-05-26T22:51:16.605 INFO:teuthology.orchestra.run.smithi131.stdout: "forward": 0, 2024-05-26T22:51:16.605 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_fetch_complete": 0, 2024-05-26T22:51:16.605 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_fetch_keys": 0, 2024-05-26T22:51:16.605 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_commit": 21, 2024-05-26T22:51:16.605 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_split": 1, 2024-05-26T22:51:16.605 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_merge": 0, 2024-05-26T22:51:16.606 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes": 89, 2024-05-26T22:51:16.606 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_top": 0, 2024-05-26T22:51:16.606 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_bottom": 89, 2024-05-26T22:51:16.606 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_pin_tail": 0, 2024-05-26T22:51:16.606 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_pinned": 88, 2024-05-26T22:51:16.606 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_expired": 124, 2024-05-26T22:51:16.606 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_with_caps": 1, 2024-05-26T22:51:16.606 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": 1, 2024-05-26T22:51:16.606 INFO:teuthology.orchestra.run.smithi131.stdout: "subtrees": 10, 2024-05-26T22:51:16.606 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse": 1285, 2024-05-26T22:51:16.606 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_hit": 1281, 2024-05-26T22:51:16.606 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_forward": 0, 2024-05-26T22:51:16.607 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_discover": 0, 2024-05-26T22:51:16.607 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_dir_fetch": 0, 2024-05-26T22:51:16.607 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_remote_ino": 0, 2024-05-26T22:51:16.607 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_lock": 0, 2024-05-26T22:51:16.607 INFO:teuthology.orchestra.run.smithi131.stdout: "load_cent": 0, 2024-05-26T22:51:16.607 INFO:teuthology.orchestra.run.smithi131.stdout: "q": 0, 2024-05-26T22:51:16.607 INFO:teuthology.orchestra.run.smithi131.stdout: "exported": 5, 2024-05-26T22:51:16.607 INFO:teuthology.orchestra.run.smithi131.stdout: "exported_inodes": 124, 2024-05-26T22:51:16.607 INFO:teuthology.orchestra.run.smithi131.stdout: "imported": 0, 2024-05-26T22:51:16.607 INFO:teuthology.orchestra.run.smithi131.stdout: "imported_inodes": 0, 2024-05-26T22:51:16.607 INFO:teuthology.orchestra.run.smithi131.stdout: "openino_dir_fetch": 0, 2024-05-26T22:51:16.607 INFO:teuthology.orchestra.run.smithi131.stdout: "openino_backtrace_fetch": 0, 2024-05-26T22:51:16.607 INFO:teuthology.orchestra.run.smithi131.stdout: "openino_peer_discover": 0, 2024-05-26T22:51:16.608 INFO:teuthology.orchestra.run.smithi131.stdout: "root_rfiles": 100, 2024-05-26T22:51:16.608 INFO:teuthology.orchestra.run.smithi131.stdout: "root_rbytes": 500, 2024-05-26T22:51:16.608 INFO:teuthology.orchestra.run.smithi131.stdout: "root_rsnaps": 0, 2024-05-26T22:51:16.608 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_backtrace_fetch": 0, 2024-05-26T22:51:16.608 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_set_tag": 0, 2024-05-26T22:51:16.608 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_backtrace_repaired": 0, 2024-05-26T22:51:16.608 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_inotable_repaired": 0, 2024-05-26T22:51:16.608 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_dir_inodes": 0, 2024-05-26T22:51:16.608 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_dir_base_inodes": 0, 2024-05-26T22:51:16.608 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_dirfrag_rstats": 0, 2024-05-26T22:51:16.608 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_file_inodes": 0, 2024-05-26T22:51:16.608 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_inode_file_caps": 0, 2024-05-26T22:51:16.608 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_revoke": 4, 2024-05-26T22:51:16.608 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_grant": 1127, 2024-05-26T22:51:16.609 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_trunc": 0, 2024-05-26T22:51:16.609 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-26T22:51:16.609 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-26T22:51:16.609 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_caps": 616, 2024-05-26T22:51:16.609 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_caps_dirty": 612, 2024-05-26T22:51:16.609 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_cap_release": 5, 2024-05-26T22:51:16.609 INFO:teuthology.orchestra.run.smithi131.stdout: "process_request_cap_release": 620 2024-05-26T22:51:16.609 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.609 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_cache": { 2024-05-26T22:51:16.609 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_update": 0, 2024-05-26T22:51:16.609 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_update_receipt": 0, 2024-05-26T22:51:16.609 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_try_discover": 0, 2024-05-26T22:51:16.610 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_send_discover": 0, 2024-05-26T22:51:16.610 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_handle_discover": 7, 2024-05-26T22:51:16.610 INFO:teuthology.orchestra.run.smithi131.stdout: "num_strays": 0, 2024-05-26T22:51:16.610 INFO:teuthology.orchestra.run.smithi131.stdout: "num_strays_delayed": 0, 2024-05-26T22:51:16.610 INFO:teuthology.orchestra.run.smithi131.stdout: "num_strays_enqueuing": 0, 2024-05-26T22:51:16.610 INFO:teuthology.orchestra.run.smithi131.stdout: "strays_created": 513, 2024-05-26T22:51:16.610 INFO:teuthology.orchestra.run.smithi131.stdout: "strays_enqueued": 513, 2024-05-26T22:51:16.610 INFO:teuthology.orchestra.run.smithi131.stdout: "strays_reintegrated": 0, 2024-05-26T22:51:16.610 INFO:teuthology.orchestra.run.smithi131.stdout: "strays_migrated": 0, 2024-05-26T22:51:16.610 INFO:teuthology.orchestra.run.smithi131.stdout: "num_recovering_processing": 0, 2024-05-26T22:51:16.610 INFO:teuthology.orchestra.run.smithi131.stdout: "num_recovering_enqueued": 0, 2024-05-26T22:51:16.610 INFO:teuthology.orchestra.run.smithi131.stdout: "num_recovering_prioritized": 0, 2024-05-26T22:51:16.610 INFO:teuthology.orchestra.run.smithi131.stdout: "recovery_started": 0, 2024-05-26T22:51:16.611 INFO:teuthology.orchestra.run.smithi131.stdout: "recovery_completed": 0, 2024-05-26T22:51:16.611 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_quiesce_path": 0, 2024-05-26T22:51:16.611 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_quiesce_inode": 0, 2024-05-26T22:51:16.611 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_enqueue_scrub": 0, 2024-05-26T22:51:16.611 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_exportdir": 5, 2024-05-26T22:51:16.611 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_flush": 0, 2024-05-26T22:51:16.611 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_fragmentdir": 1, 2024-05-26T22:51:16.611 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_fragstats": 0, 2024-05-26T22:51:16.611 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_inodestats": 0 2024-05-26T22:51:16.611 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.611 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_log": { 2024-05-26T22:51:16.611 INFO:teuthology.orchestra.run.smithi131.stdout: "evlrg": 0, 2024-05-26T22:51:16.612 INFO:teuthology.orchestra.run.smithi131.stdout: "evadd": 2398, 2024-05-26T22:51:16.612 INFO:teuthology.orchestra.run.smithi131.stdout: "evex": 0, 2024-05-26T22:51:16.612 INFO:teuthology.orchestra.run.smithi131.stdout: "evtrm": 0, 2024-05-26T22:51:16.612 INFO:teuthology.orchestra.run.smithi131.stdout: "ev": 2398, 2024-05-26T22:51:16.612 INFO:teuthology.orchestra.run.smithi131.stdout: "evexg": 0, 2024-05-26T22:51:16.612 INFO:teuthology.orchestra.run.smithi131.stdout: "evexd": 0, 2024-05-26T22:51:16.612 INFO:teuthology.orchestra.run.smithi131.stdout: "segadd": 4, 2024-05-26T22:51:16.612 INFO:teuthology.orchestra.run.smithi131.stdout: "segex": 0, 2024-05-26T22:51:16.612 INFO:teuthology.orchestra.run.smithi131.stdout: "segtrm": 0, 2024-05-26T22:51:16.612 INFO:teuthology.orchestra.run.smithi131.stdout: "seg": 4, 2024-05-26T22:51:16.612 INFO:teuthology.orchestra.run.smithi131.stdout: "segmjr": 2, 2024-05-26T22:51:16.613 INFO:teuthology.orchestra.run.smithi131.stdout: "segexg": 0, 2024-05-26T22:51:16.613 INFO:teuthology.orchestra.run.smithi131.stdout: "segexd": 0, 2024-05-26T22:51:16.613 INFO:teuthology.orchestra.run.smithi131.stdout: "expos": 4194304, 2024-05-26T22:51:16.613 INFO:teuthology.orchestra.run.smithi131.stdout: "wrpos": 9109139, 2024-05-26T22:51:16.613 INFO:teuthology.orchestra.run.smithi131.stdout: "rdpos": 0, 2024-05-26T22:51:16.613 INFO:teuthology.orchestra.run.smithi131.stdout: "jlat": { 2024-05-26T22:51:16.613 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 24, 2024-05-26T22:51:16.613 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.275705804, 2024-05-26T22:51:16.613 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.011487741 2024-05-26T22:51:16.613 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.613 INFO:teuthology.orchestra.run.smithi131.stdout: "replayed": 0 2024-05-26T22:51:16.614 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.614 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_mem": { 2024-05-26T22:51:16.614 INFO:teuthology.orchestra.run.smithi131.stdout: "ino": 90, 2024-05-26T22:51:16.614 INFO:teuthology.orchestra.run.smithi131.stdout: "ino+": 727, 2024-05-26T22:51:16.614 INFO:teuthology.orchestra.run.smithi131.stdout: "ino-": 637, 2024-05-26T22:51:16.614 INFO:teuthology.orchestra.run.smithi131.stdout: "dir": 58, 2024-05-26T22:51:16.614 INFO:teuthology.orchestra.run.smithi131.stdout: "dir+": 122, 2024-05-26T22:51:16.614 INFO:teuthology.orchestra.run.smithi131.stdout: "dir-": 64, 2024-05-26T22:51:16.614 INFO:teuthology.orchestra.run.smithi131.stdout: "dn": 89, 2024-05-26T22:51:16.614 INFO:teuthology.orchestra.run.smithi131.stdout: "dn+": 1238, 2024-05-26T22:51:16.614 INFO:teuthology.orchestra.run.smithi131.stdout: "dn-": 1149, 2024-05-26T22:51:16.615 INFO:teuthology.orchestra.run.smithi131.stdout: "cap": 1, 2024-05-26T22:51:16.615 INFO:teuthology.orchestra.run.smithi131.stdout: "cap+": 715, 2024-05-26T22:51:16.615 INFO:teuthology.orchestra.run.smithi131.stdout: "cap-": 714, 2024-05-26T22:51:16.615 INFO:teuthology.orchestra.run.smithi131.stdout: "rss": 63368, 2024-05-26T22:51:16.615 INFO:teuthology.orchestra.run.smithi131.stdout: "heap": 231788 2024-05-26T22:51:16.615 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.615 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_server": { 2024-05-26T22:51:16.615 INFO:teuthology.orchestra.run.smithi131.stdout: "dispatch_client_request": 1265, 2024-05-26T22:51:16.615 INFO:teuthology.orchestra.run.smithi131.stdout: "dispatch_server_request": 0, 2024-05-26T22:51:16.615 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_request": 1258, 2024-05-26T22:51:16.616 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_session": 8, 2024-05-26T22:51:16.616 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_peer_request": 0, 2024-05-26T22:51:16.616 INFO:teuthology.orchestra.run.smithi131.stdout: "req_create_latency": { 2024-05-26T22:51:16.616 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 612, 2024-05-26T22:51:16.616 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.939770929, 2024-05-26T22:51:16.616 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.001535573 2024-05-26T22:51:16.616 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.616 INFO:teuthology.orchestra.run.smithi131.stdout: "req_getattr_latency": { 2024-05-26T22:51:16.616 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 26, 2024-05-26T22:51:16.616 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.014781499, 2024-05-26T22:51:16.616 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000568519 2024-05-26T22:51:16.616 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.616 INFO:teuthology.orchestra.run.smithi131.stdout: "req_getfilelock_latency": { 2024-05-26T22:51:16.617 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:51:16.617 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:51:16.617 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:51:16.617 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.617 INFO:teuthology.orchestra.run.smithi131.stdout: "req_link_latency": { 2024-05-26T22:51:16.617 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:51:16.617 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:51:16.617 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:51:16.617 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.617 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookup_latency": { 2024-05-26T22:51:16.617 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 1, 2024-05-26T22:51:16.618 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000227305, 2024-05-26T22:51:16.618 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000227305 2024-05-26T22:51:16.618 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.618 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookuphash_latency": { 2024-05-26T22:51:16.618 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:51:16.618 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:51:16.618 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:51:16.618 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.618 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookupino_latency": { 2024-05-26T22:51:16.618 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:51:16.619 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:51:16.619 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:51:16.619 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.619 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookupname_latency": { 2024-05-26T22:51:16.619 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:51:16.619 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:51:16.619 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:51:16.619 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.619 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookupparent_latency": { 2024-05-26T22:51:16.619 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:51:16.620 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:51:16.620 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:51:16.620 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.620 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookupsnap_latency": { 2024-05-26T22:51:16.620 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:51:16.620 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:51:16.620 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:51:16.620 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.620 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lssnap_latency": { 2024-05-26T22:51:16.620 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:51:16.620 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:51:16.621 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:51:16.621 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.621 INFO:teuthology.orchestra.run.smithi131.stdout: "req_mkdir_latency": { 2024-05-26T22:51:16.621 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 102, 2024-05-26T22:51:16.621 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.145482933, 2024-05-26T22:51:16.621 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.001426303 2024-05-26T22:51:16.621 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.621 INFO:teuthology.orchestra.run.smithi131.stdout: "req_mknod_latency": { 2024-05-26T22:51:16.621 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:51:16.622 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:51:16.622 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:51:16.622 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.622 INFO:teuthology.orchestra.run.smithi131.stdout: "req_mksnap_latency": { 2024-05-26T22:51:16.622 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:51:16.622 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:51:16.622 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:51:16.622 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.622 INFO:teuthology.orchestra.run.smithi131.stdout: "req_open_latency": { 2024-05-26T22:51:16.623 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 1, 2024-05-26T22:51:16.623 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000161881, 2024-05-26T22:51:16.623 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000161881 2024-05-26T22:51:16.623 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.623 INFO:teuthology.orchestra.run.smithi131.stdout: "req_readdir_latency": { 2024-05-26T22:51:16.623 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 1, 2024-05-26T22:51:16.623 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.014387337, 2024-05-26T22:51:16.623 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.014387337 2024-05-26T22:51:16.623 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.623 INFO:teuthology.orchestra.run.smithi131.stdout: "req_rename_latency": { 2024-05-26T22:51:16.623 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:51:16.624 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:51:16.624 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:51:16.624 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.624 INFO:teuthology.orchestra.run.smithi131.stdout: "req_renamesnap_latency": { 2024-05-26T22:51:16.624 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:51:16.624 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:51:16.624 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:51:16.624 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.624 INFO:teuthology.orchestra.run.smithi131.stdout: "req_snapdiff_latency": { 2024-05-26T22:51:16.624 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:51:16.624 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:51:16.624 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:51:16.624 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.625 INFO:teuthology.orchestra.run.smithi131.stdout: "req_rmdir_latency": { 2024-05-26T22:51:16.625 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 1, 2024-05-26T22:51:16.625 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.429350021, 2024-05-26T22:51:16.625 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.429350021 2024-05-26T22:51:16.625 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.625 INFO:teuthology.orchestra.run.smithi131.stdout: "req_rmsnap_latency": { 2024-05-26T22:51:16.625 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:51:16.625 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:51:16.625 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:51:16.625 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.625 INFO:teuthology.orchestra.run.smithi131.stdout: "req_rmxattr_latency": { 2024-05-26T22:51:16.625 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:51:16.625 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:51:16.625 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:51:16.626 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.626 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setattr_latency": { 2024-05-26T22:51:16.626 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 1, 2024-05-26T22:51:16.626 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000403915, 2024-05-26T22:51:16.626 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000403915 2024-05-26T22:51:16.626 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.626 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setdirlayout_latency": { 2024-05-26T22:51:16.626 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:51:16.626 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:51:16.626 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:51:16.626 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.626 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setfilelock_latency": { 2024-05-26T22:51:16.626 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:51:16.627 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:51:16.627 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:51:16.627 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.627 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setlayout_latency": { 2024-05-26T22:51:16.627 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:51:16.627 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:51:16.627 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:51:16.627 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.627 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setxattr_latency": { 2024-05-26T22:51:16.627 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 1, 2024-05-26T22:51:16.627 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.001301164, 2024-05-26T22:51:16.627 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.001301164 2024-05-26T22:51:16.627 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.628 INFO:teuthology.orchestra.run.smithi131.stdout: "req_symlink_latency": { 2024-05-26T22:51:16.628 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:51:16.628 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:51:16.628 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:51:16.628 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.628 INFO:teuthology.orchestra.run.smithi131.stdout: "req_unlink_latency": { 2024-05-26T22:51:16.628 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 512, 2024-05-26T22:51:16.628 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.643577559, 2024-05-26T22:51:16.628 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.001256987 2024-05-26T22:51:16.628 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.628 INFO:teuthology.orchestra.run.smithi131.stdout: "cap_revoke_eviction": 0, 2024-05-26T22:51:16.628 INFO:teuthology.orchestra.run.smithi131.stdout: "cap_acquisition_throttle": 0, 2024-05-26T22:51:16.628 INFO:teuthology.orchestra.run.smithi131.stdout: "req_getvxattr_latency": { 2024-05-26T22:51:16.628 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:51:16.629 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:51:16.629 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:51:16.629 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:51:16.629 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.629 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_sessions": { 2024-05-26T22:51:16.629 INFO:teuthology.orchestra.run.smithi131.stdout: "session_count": 1, 2024-05-26T22:51:16.629 INFO:teuthology.orchestra.run.smithi131.stdout: "session_add": 1, 2024-05-26T22:51:16.629 INFO:teuthology.orchestra.run.smithi131.stdout: "session_remove": 0, 2024-05-26T22:51:16.629 INFO:teuthology.orchestra.run.smithi131.stdout: "sessions_open": 1, 2024-05-26T22:51:16.629 INFO:teuthology.orchestra.run.smithi131.stdout: "sessions_stale": 0, 2024-05-26T22:51:16.629 INFO:teuthology.orchestra.run.smithi131.stdout: "total_load": 1172, 2024-05-26T22:51:16.629 INFO:teuthology.orchestra.run.smithi131.stdout: "average_load": 1172, 2024-05-26T22:51:16.629 INFO:teuthology.orchestra.run.smithi131.stdout: "avg_session_uptime": 31, 2024-05-26T22:51:16.630 INFO:teuthology.orchestra.run.smithi131.stdout: "mdthresh_evicted": 0 2024-05-26T22:51:16.630 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.630 INFO:teuthology.orchestra.run.smithi131.stdout: "mempool": { 2024-05-26T22:51:16.630 INFO:teuthology.orchestra.run.smithi131.stdout: "bloom_filter_bytes": 0, 2024-05-26T22:51:16.630 INFO:teuthology.orchestra.run.smithi131.stdout: "bloom_filter_items": 0, 2024-05-26T22:51:16.630 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_alloc_bytes": 0, 2024-05-26T22:51:16.630 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_alloc_items": 0, 2024-05-26T22:51:16.630 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_data_bytes": 0, 2024-05-26T22:51:16.630 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_data_items": 0, 2024-05-26T22:51:16.630 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-26T22:51:16.630 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_onode_items": 0, 2024-05-26T22:51:16.630 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-26T22:51:16.630 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_meta_items": 0, 2024-05-26T22:51:16.630 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_other_bytes": 0, 2024-05-26T22:51:16.631 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_other_items": 0, 2024-05-26T22:51:16.631 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-26T22:51:16.631 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_buffer_items": 0, 2024-05-26T22:51:16.631 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_extent_bytes": 0, 2024-05-26T22:51:16.631 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_extent_items": 0, 2024-05-26T22:51:16.631 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_blob_bytes": 0, 2024-05-26T22:51:16.631 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_blob_items": 0, 2024-05-26T22:51:16.631 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-26T22:51:16.631 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_shared_blob_items": 0, 2024-05-26T22:51:16.631 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-26T22:51:16.631 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_inline_bl_items": 0, 2024-05-26T22:51:16.631 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_fsck_bytes": 0, 2024-05-26T22:51:16.631 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_fsck_items": 0, 2024-05-26T22:51:16.632 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_txc_bytes": 0, 2024-05-26T22:51:16.632 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_txc_items": 0, 2024-05-26T22:51:16.632 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-26T22:51:16.632 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_deferred_items": 0, 2024-05-26T22:51:16.632 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_bytes": 0, 2024-05-26T22:51:16.632 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_items": 0, 2024-05-26T22:51:16.632 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_bytes": 0, 2024-05-26T22:51:16.632 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_items": 0, 2024-05-26T22:51:16.632 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_reader_bytes": 0, 2024-05-26T22:51:16.632 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_reader_items": 0, 2024-05-26T22:51:16.632 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_writer_bytes": 0, 2024-05-26T22:51:16.632 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_writer_items": 0, 2024-05-26T22:51:16.632 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_anon_bytes": 565835, 2024-05-26T22:51:16.633 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_anon_items": 267, 2024-05-26T22:51:16.633 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_meta_bytes": 1672, 2024-05-26T22:51:16.633 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_meta_items": 19, 2024-05-26T22:51:16.633 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_bytes": 0, 2024-05-26T22:51:16.633 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_items": 0, 2024-05-26T22:51:16.633 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_mapbl_bytes": 0, 2024-05-26T22:51:16.633 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_mapbl_items": 0, 2024-05-26T22:51:16.633 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_pglog_bytes": 0, 2024-05-26T22:51:16.633 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_pglog_items": 0, 2024-05-26T22:51:16.633 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_bytes": 5736, 2024-05-26T22:51:16.633 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_items": 94, 2024-05-26T22:51:16.633 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_mapping_bytes": 0, 2024-05-26T22:51:16.633 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_mapping_items": 0, 2024-05-26T22:51:16.634 INFO:teuthology.orchestra.run.smithi131.stdout: "pgmap_bytes": 0, 2024-05-26T22:51:16.634 INFO:teuthology.orchestra.run.smithi131.stdout: "pgmap_items": 0, 2024-05-26T22:51:16.634 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_co_bytes": 410154, 2024-05-26T22:51:16.634 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_co_items": 3781, 2024-05-26T22:51:16.634 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_1_bytes": 0, 2024-05-26T22:51:16.634 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_1_items": 0, 2024-05-26T22:51:16.634 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_2_bytes": 0, 2024-05-26T22:51:16.634 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_2_items": 0 2024-05-26T22:51:16.634 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.634 INFO:teuthology.orchestra.run.smithi131.stdout: "objecter": { 2024-05-26T22:51:16.634 INFO:teuthology.orchestra.run.smithi131.stdout: "op_active": 0, 2024-05-26T22:51:16.634 INFO:teuthology.orchestra.run.smithi131.stdout: "op_laggy": 0, 2024-05-26T22:51:16.634 INFO:teuthology.orchestra.run.smithi131.stdout: "op_send": 620, 2024-05-26T22:51:16.635 INFO:teuthology.orchestra.run.smithi131.stdout: "op_send_bytes": 5175094, 2024-05-26T22:51:16.635 INFO:teuthology.orchestra.run.smithi131.stdout: "op_resend": 37, 2024-05-26T22:51:16.635 INFO:teuthology.orchestra.run.smithi131.stdout: "op_reply": 620, 2024-05-26T22:51:16.635 INFO:teuthology.orchestra.run.smithi131.stdout: "op_latency": { 2024-05-26T22:51:16.635 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 620, 2024-05-26T22:51:16.635 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 5.291888119, 2024-05-26T22:51:16.635 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.008535303 2024-05-26T22:51:16.635 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.635 INFO:teuthology.orchestra.run.smithi131.stdout: "op_inflight": 0, 2024-05-26T22:51:16.635 INFO:teuthology.orchestra.run.smithi131.stdout: "oplen_avg": { 2024-05-26T22:51:16.635 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 620, 2024-05-26T22:51:16.635 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 670 2024-05-26T22:51:16.635 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.635 INFO:teuthology.orchestra.run.smithi131.stdout: "op": 620, 2024-05-26T22:51:16.636 INFO:teuthology.orchestra.run.smithi131.stdout: "op_r": 6, 2024-05-26T22:51:16.636 INFO:teuthology.orchestra.run.smithi131.stdout: "op_w": 614, 2024-05-26T22:51:16.636 INFO:teuthology.orchestra.run.smithi131.stdout: "op_rmw": 0, 2024-05-26T22:51:16.636 INFO:teuthology.orchestra.run.smithi131.stdout: "op_pg": 0, 2024-05-26T22:51:16.636 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_stat": 0, 2024-05-26T22:51:16.636 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_create": 11, 2024-05-26T22:51:16.636 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_read": 6, 2024-05-26T22:51:16.636 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_write": 30, 2024-05-26T22:51:16.636 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_writefull": 12, 2024-05-26T22:51:16.636 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_writesame": 0, 2024-05-26T22:51:16.636 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_append": 0, 2024-05-26T22:51:16.636 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_zero": 0, 2024-05-26T22:51:16.636 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_truncate": 1, 2024-05-26T22:51:16.637 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_delete": 526, 2024-05-26T22:51:16.637 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_mapext": 0, 2024-05-26T22:51:16.637 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_sparse_read": 0, 2024-05-26T22:51:16.637 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_clonerange": 0, 2024-05-26T22:51:16.637 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_getxattr": 0, 2024-05-26T22:51:16.637 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_setxattr": 22, 2024-05-26T22:51:16.637 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_cmpxattr": 0, 2024-05-26T22:51:16.637 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_rmxattr": 0, 2024-05-26T22:51:16.637 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_resetxattrs": 0, 2024-05-26T22:51:16.637 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_call": 0, 2024-05-26T22:51:16.637 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_watch": 0, 2024-05-26T22:51:16.637 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_notify": 0, 2024-05-26T22:51:16.637 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_src_cmpxattr": 0, 2024-05-26T22:51:16.638 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_pgls": 0, 2024-05-26T22:51:16.638 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_pgls_filter": 0, 2024-05-26T22:51:16.638 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_other": 0, 2024-05-26T22:51:16.638 INFO:teuthology.orchestra.run.smithi131.stdout: "linger_active": 0, 2024-05-26T22:51:16.638 INFO:teuthology.orchestra.run.smithi131.stdout: "linger_send": 0, 2024-05-26T22:51:16.638 INFO:teuthology.orchestra.run.smithi131.stdout: "linger_resend": 0, 2024-05-26T22:51:16.638 INFO:teuthology.orchestra.run.smithi131.stdout: "linger_ping": 0, 2024-05-26T22:51:16.638 INFO:teuthology.orchestra.run.smithi131.stdout: "poolop_active": 0, 2024-05-26T22:51:16.638 INFO:teuthology.orchestra.run.smithi131.stdout: "poolop_send": 0, 2024-05-26T22:51:16.638 INFO:teuthology.orchestra.run.smithi131.stdout: "poolop_resend": 0, 2024-05-26T22:51:16.638 INFO:teuthology.orchestra.run.smithi131.stdout: "poolstat_active": 0, 2024-05-26T22:51:16.638 INFO:teuthology.orchestra.run.smithi131.stdout: "poolstat_send": 0, 2024-05-26T22:51:16.639 INFO:teuthology.orchestra.run.smithi131.stdout: "poolstat_resend": 0, 2024-05-26T22:51:16.639 INFO:teuthology.orchestra.run.smithi131.stdout: "statfs_active": 0, 2024-05-26T22:51:16.639 INFO:teuthology.orchestra.run.smithi131.stdout: "statfs_send": 0, 2024-05-26T22:51:16.639 INFO:teuthology.orchestra.run.smithi131.stdout: "statfs_resend": 0, 2024-05-26T22:51:16.639 INFO:teuthology.orchestra.run.smithi131.stdout: "command_active": 0, 2024-05-26T22:51:16.639 INFO:teuthology.orchestra.run.smithi131.stdout: "command_send": 0, 2024-05-26T22:51:16.639 INFO:teuthology.orchestra.run.smithi131.stdout: "command_resend": 0, 2024-05-26T22:51:16.639 INFO:teuthology.orchestra.run.smithi131.stdout: "map_epoch": 47, 2024-05-26T22:51:16.639 INFO:teuthology.orchestra.run.smithi131.stdout: "map_full": 0, 2024-05-26T22:51:16.639 INFO:teuthology.orchestra.run.smithi131.stdout: "map_inc": 4, 2024-05-26T22:51:16.639 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_sessions": 8, 2024-05-26T22:51:16.639 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_session_open": 8, 2024-05-26T22:51:16.639 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_session_close": 0, 2024-05-26T22:51:16.640 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_laggy": 0, 2024-05-26T22:51:16.640 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_wr": 54, 2024-05-26T22:51:16.640 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_rd": 0, 2024-05-26T22:51:16.640 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_del": 8 2024-05-26T22:51:16.640 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.640 INFO:teuthology.orchestra.run.smithi131.stdout: "oft": { 2024-05-26T22:51:16.640 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_total_objs": 1, 2024-05-26T22:51:16.640 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_total_kv_pairs": 0, 2024-05-26T22:51:16.640 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_total_updates": 0, 2024-05-26T22:51:16.640 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_total_removes": 0 2024-05-26T22:51:16.640 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.640 INFO:teuthology.orchestra.run.smithi131.stdout: "osd-slow-ops": { 2024-05-26T22:51:16.640 INFO:teuthology.orchestra.run.smithi131.stdout: "slow_ops_count": 0 2024-05-26T22:51:16.641 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.641 INFO:teuthology.orchestra.run.smithi131.stdout: "purge_queue": { 2024-05-26T22:51:16.641 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executing_ops": 0, 2024-05-26T22:51:16.641 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executing_ops_high_water": 32, 2024-05-26T22:51:16.641 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executing": 0, 2024-05-26T22:51:16.641 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executing_high_water": 32, 2024-05-26T22:51:16.641 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executed_ops": 513, 2024-05-26T22:51:16.641 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executed": 513, 2024-05-26T22:51:16.641 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_item_in_journal": 0 2024-05-26T22:51:16.641 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.641 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-26T22:51:16.641 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-05-26T22:51:16.641 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 104857600, 2024-05-26T22:51:16.641 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-05-26T22:51:16.642 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 2763, 2024-05-26T22:51:16.642 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 847299, 2024-05-26T22:51:16.642 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-05-26T22:51:16.642 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 2763, 2024-05-26T22:51:16.642 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 0, 2024-05-26T22:51:16.642 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 0, 2024-05-26T22:51:16.642 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 2763, 2024-05-26T22:51:16.642 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 847299, 2024-05-26T22:51:16.642 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-05-26T22:51:16.642 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:51:16.642 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:51:16.643 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:51:16.643 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:51:16.643 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.643 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-objecter_bytes": { 2024-05-26T22:51:16.643 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-05-26T22:51:16.643 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 104857600, 2024-05-26T22:51:16.643 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-05-26T22:51:16.643 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 0, 2024-05-26T22:51:16.643 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 0, 2024-05-26T22:51:16.643 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-05-26T22:51:16.643 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 0, 2024-05-26T22:51:16.643 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 620, 2024-05-26T22:51:16.643 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 5226907, 2024-05-26T22:51:16.644 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 620, 2024-05-26T22:51:16.644 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 5226907, 2024-05-26T22:51:16.644 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-05-26T22:51:16.644 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:51:16.644 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:51:16.644 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:51:16.644 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:51:16.644 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.644 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-objecter_ops": { 2024-05-26T22:51:16.644 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-05-26T22:51:16.644 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 1024, 2024-05-26T22:51:16.644 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-05-26T22:51:16.644 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 0, 2024-05-26T22:51:16.645 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 0, 2024-05-26T22:51:16.645 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-05-26T22:51:16.645 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 0, 2024-05-26T22:51:16.645 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 620, 2024-05-26T22:51:16.645 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 620, 2024-05-26T22:51:16.645 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 620, 2024-05-26T22:51:16.645 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 620, 2024-05-26T22:51:16.645 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-05-26T22:51:16.646 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:51:16.646 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:51:16.646 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:51:16.646 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:51:16.646 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.646 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-write_buf_throttle": { 2024-05-26T22:51:16.646 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-05-26T22:51:16.646 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 3758096384, 2024-05-26T22:51:16.646 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-05-26T22:51:16.646 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 513, 2024-05-26T22:51:16.646 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 51813, 2024-05-26T22:51:16.646 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-05-26T22:51:16.647 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 513, 2024-05-26T22:51:16.647 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 0, 2024-05-26T22:51:16.647 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 0, 2024-05-26T22:51:16.647 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 6, 2024-05-26T22:51:16.647 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 51813, 2024-05-26T22:51:16.647 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-05-26T22:51:16.647 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:51:16.647 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:51:16.647 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:51:16.648 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:51:16.648 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:16.648 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-write_buf_throttle-0x557025a46500": { 2024-05-26T22:51:16.648 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-05-26T22:51:16.648 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 3758096384, 2024-05-26T22:51:16.648 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-05-26T22:51:16.648 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 2398, 2024-05-26T22:51:16.648 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 4914835, 2024-05-26T22:51:16.648 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-05-26T22:51:16.648 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 2398, 2024-05-26T22:51:16.648 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 0, 2024-05-26T22:51:16.648 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 0, 2024-05-26T22:51:16.648 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 24, 2024-05-26T22:51:16.648 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 4914835, 2024-05-26T22:51:16.649 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-05-26T22:51:16.649 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-05-26T22:51:16.649 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-05-26T22:51:16.649 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-05-26T22:51:16.649 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:51:16.649 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:51:16.649 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-26T22:51:16.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.593+0000 7f8f32c00640 1 -- 172.21.15.131:0/713485266 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f8f0806be30 msgr2=0x7f8f0806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:16.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.593+0000 7f8f32c00640 1 --2- 172.21.15.131:0/713485266 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f8f0806be30 0x7f8f0806e2f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c034000 tx=0x7f8f1c02f000 comp rx=0 tx=0).stop 2024-05-26T22:51:16.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.593+0000 7f8f32c00640 1 -- 172.21.15.131:0/713485266 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f2c0b5450 msgr2=0x7f8f2c1f20c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:16.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.593+0000 7f8f32c00640 1 --2- 172.21.15.131:0/713485266 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f2c0b5450 0x7f8f2c1f20c0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f8f1400e840 tx=0x7f8f1400ed00 comp rx=0 tx=0).stop 2024-05-26T22:51:16.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.593+0000 7f8f32c00640 1 -- 172.21.15.131:0/713485266 shutdown_connections 2024-05-26T22:51:16.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.593+0000 7f8f32c00640 1 --2- 172.21.15.131:0/713485266 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8f2c0cf590 0x7f8f2c15d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:16.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.593+0000 7f8f32c00640 1 --2- 172.21.15.131:0/713485266 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f8f0806be30 0x7f8f0806e2f0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:16.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.593+0000 7f8f32c00640 1 --2- 172.21.15.131:0/713485266 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f2c0b5450 0x7f8f2c1f20c0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:16.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.593+0000 7f8f32c00640 1 --2- 172.21.15.131:0/713485266 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8f2c06f420 0x7f8f2c1518c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:16.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.593+0000 7f8f32c00640 1 -- 172.21.15.131:0/713485266 >> 172.21.15.131:0/713485266 conn(0x7f8f2c06bc90 msgr2=0x7f8f2c15eee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:16.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.593+0000 7f8f32c00640 1 -- 172.21.15.131:0/713485266 shutdown_connections 2024-05-26T22:51:16.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:16.593+0000 7f8f32c00640 1 -- 172.21.15.131:0/713485266 wait complete. 2024-05-26T22:51:16.653 INFO:tasks.cephfs.test_exports:export stats: [(0, 5), (1, 0), (2, 0)] 2024-05-26T22:51:16.653 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-05-26T22:51:16.654 DEBUG:teuthology.orchestra.run.smithi131:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-26T22:51:16.759 DEBUG:teuthology.orchestra.run.smithi187:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-26T22:51:16.872 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-05-26T22:51:16.872 INFO:tasks.cephfs.mount:Terminating background process 2024-05-26T22:51:16.872 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-26T22:51:16.872 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-26T22:51:16.930 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-26T22:51:16.930 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-26T22:51:16.985 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-26T22:51:17.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.183+0000 7f045c800640 1 Processor -- start 2024-05-26T22:51:17.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.183+0000 7f045c800640 1 -- start start 2024-05-26T22:51:17.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.185+0000 7f045c800640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f04540b2f60 0x7f04540d4c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:17.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.185+0000 7f045c800640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04540b37e0 con 0x7f04540d19b0 2024-05-26T22:51:17.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.185+0000 7f045c800640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04540b9cf0 con 0x7f045415e570 2024-05-26T22:51:17.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.185+0000 7f045c800640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f04540d51b0 con 0x7f04540b2f60 2024-05-26T22:51:17.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.185+0000 7f0459400640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f04540d19b0 0x7f04540d2590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:51312/0 (socket says 172.21.15.131:51312) 2024-05-26T22:51:17.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.185+0000 7f045a800640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f04540b2f60 0x7f04540d4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:17.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.185+0000 7f0459400640 1 -- 172.21.15.131:0/2735663316 learned_addr learned my addr 172.21.15.131:0/2735663316 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:51:17.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.186+0000 7f0458a00640 1 -- 172.21.15.131:0/2735663316 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2779543415 0 0) 0x7f0444001110 con 0x7f04540d19b0 2024-05-26T22:51:17.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.186+0000 7f0458a00640 1 -- 172.21.15.131:0/2735663316 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0440003520 con 0x7f04540d19b0 2024-05-26T22:51:17.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.186+0000 7f045a800640 1 -- 172.21.15.131:0/2735663316 >> v1:172.21.15.187:6789/0 conn(0x7f045415e570 legacy=0x7f04540d12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:17.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.186+0000 7f045a800640 1 -- 172.21.15.131:0/2735663316 >> v1:172.21.15.131:6789/0 conn(0x7f04540d19b0 legacy=0x7f04540d2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:17.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.186+0000 7f045a800640 1 -- 172.21.15.131:0/2735663316 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04540b9fd0 con 0x7f04540b2f60 2024-05-26T22:51:17.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.187+0000 7f045a800640 1 --2- 172.21.15.131:0/2735663316 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04540b2f60 0x7f04540d4c70 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f04500097a0 tx=0x7f045002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdfead81be6a7da8 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:17.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.187+0000 7f0458a00640 1 -- 172.21.15.131:0/2735663316 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0450037070 con 0x7f04540b2f60 2024-05-26T22:51:17.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.187+0000 7f0458a00640 1 -- 172.21.15.131:0/2735663316 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0450032440 con 0x7f04540b2f60 2024-05-26T22:51:17.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.187+0000 7f0458a00640 1 -- 172.21.15.131:0/2735663316 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0450032810 con 0x7f04540b2f60 2024-05-26T22:51:17.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.188+0000 7f045c800640 1 -- 172.21.15.131:0/2735663316 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04540b2f60 msgr2=0x7f04540d4c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:17.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.188+0000 7f045c800640 1 --2- 172.21.15.131:0/2735663316 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04540b2f60 0x7f04540d4c70 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f04500097a0 tx=0x7f045002a1e0 comp rx=0 tx=0).stop 2024-05-26T22:51:17.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.188+0000 7f045c800640 1 -- 172.21.15.131:0/2735663316 shutdown_connections 2024-05-26T22:51:17.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.188+0000 7f045c800640 1 --2- 172.21.15.131:0/2735663316 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04540b2f60 0x7f04540d4c70 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:17.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.188+0000 7f045c800640 1 -- 172.21.15.131:0/2735663316 >> 172.21.15.131:0/2735663316 conn(0x7f0454072e90 msgr2=0x7f04540732a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:17.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.188+0000 7f045c800640 1 -- 172.21.15.131:0/2735663316 shutdown_connections 2024-05-26T22:51:17.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.188+0000 7f045c800640 1 -- 172.21.15.131:0/2735663316 wait complete. 2024-05-26T22:51:17.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.190+0000 7f045c800640 1 Processor -- start 2024-05-26T22:51:17.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.190+0000 7f045c800640 1 -- start start 2024-05-26T22:51:17.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.191+0000 7f045c800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04540b2f60 0x7f04540cdb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:17.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.191+0000 7f045c800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04540d19b0 0x7f0454155a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:17.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.191+0000 7f045c800640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f045415e570 0x7f04540cec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:17.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.191+0000 7f045c800640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f04540d1ea0 con 0x7f04540b2f60 2024-05-26T22:51:17.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.191+0000 7f045c800640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f04540d47e0 con 0x7f045415e570 2024-05-26T22:51:17.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.191+0000 7f045c800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f04540d2020 con 0x7f04540d19b0 2024-05-26T22:51:17.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.191+0000 7f0459e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04540b2f60 0x7f04540cdb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:17.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.191+0000 7f0459e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04540b2f60 0x7f04540cdb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44870/0 (socket says 172.21.15.131:44870) 2024-05-26T22:51:17.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.191+0000 7f0459e00640 1 -- 172.21.15.131:0/2750694007 learned_addr learned my addr 172.21.15.131:0/2750694007 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:17.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.191+0000 7f045a800640 1 --2- 172.21.15.131:0/2750694007 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f045415e570 0x7f04540cec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:17.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.191+0000 7f0459e00640 1 -- 172.21.15.131:0/2750694007 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04540d19b0 msgr2=0x7f0454155a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:17.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.191+0000 7f0459e00640 1 --2- 172.21.15.131:0/2750694007 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04540d19b0 0x7f0454155a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:17.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.191+0000 7f0459e00640 1 -- 172.21.15.131:0/2750694007 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f045415e570 msgr2=0x7f04540cec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:17.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.191+0000 7f0459e00640 1 --2- 172.21.15.131:0/2750694007 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f045415e570 0x7f04540cec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:17.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.191+0000 7f0459e00640 1 -- 172.21.15.131:0/2750694007 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04540cca60 con 0x7f04540b2f60 2024-05-26T22:51:17.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.192+0000 7f0459e00640 1 --2- 172.21.15.131:0/2750694007 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04540b2f60 0x7f04540cdb00 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f044c00c7f0 tx=0x7f044c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:17.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.192+0000 7f044a000640 1 -- 172.21.15.131:0/2750694007 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f044c016020 con 0x7f04540b2f60 2024-05-26T22:51:17.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.192+0000 7f044a000640 1 -- 172.21.15.131:0/2750694007 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f044c00aa50 con 0x7f04540b2f60 2024-05-26T22:51:17.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.192+0000 7f045c800640 1 -- 172.21.15.131:0/2750694007 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04540ccd40 con 0x7f04540b2f60 2024-05-26T22:51:17.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.192+0000 7f044a000640 1 -- 172.21.15.131:0/2750694007 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f044c00ae40 con 0x7f04540b2f60 2024-05-26T22:51:17.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.192+0000 7f045c800640 1 -- 172.21.15.131:0/2750694007 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04540cd300 con 0x7f04540b2f60 2024-05-26T22:51:17.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.193+0000 7f045c800640 1 -- 172.21.15.131:0/2750694007 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04540694e0 con 0x7f04540b2f60 2024-05-26T22:51:17.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.194+0000 7f044a000640 1 -- 172.21.15.131:0/2750694007 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f044c019020 con 0x7f04540b2f60 2024-05-26T22:51:17.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.195+0000 7f044a000640 1 --2- 172.21.15.131:0/2750694007 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f043806be30 0x7f043806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:17.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.195+0000 7f044a000640 1 -- 172.21.15.131:0/2750694007 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5853+0+0 (secure 0 0 0) 0x7f044c08af90 con 0x7f04540b2f60 2024-05-26T22:51:17.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.195+0000 7f0459400640 1 --2- 172.21.15.131:0/2750694007 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f043806be30 0x7f043806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:17.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.196+0000 7f0459400640 1 --2- 172.21.15.131:0/2750694007 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f043806be30 0x7f043806e2f0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f0444000f60 tx=0x7f0444001110 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:17.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.200+0000 7f044a000640 1 -- 172.21.15.131:0/2750694007 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f044c020030 con 0x7f04540b2f60 2024-05-26T22:51:17.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.409+0000 7f045c800640 1 -- 172.21.15.131:0/2750694007 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f04540d2e30 con 0x7f04540b2f60 2024-05-26T22:51:17.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.412+0000 7f044a000640 1 -- 172.21.15.131:0/2750694007 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) ==== 74+0+16860 (secure 0 0 0) 0x7f044c0598b0 con 0x7f04540b2f60 2024-05-26T22:51:17.412 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:51:17.412 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":47,"fsid":"46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f","created":"2024-05-26T22:47:57.288905+0000","modified":"2024-05-26T22:51:05.195752+0000","last_up_change":"2024-05-26T22:48:09.602207+0000","last_in_change":"2024-05-26T22:48:00.687341+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-26T22:48:10.463339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-05-26T22:50:24.793406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-05-26T22:50:26.041636+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"814eaf62-a18d-494b-8e27-fd8fb88adea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6801","nonce":2350940051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6803","nonce":2350940051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6807","nonce":2350940051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6805","nonce":2350940051}]},"public_addr":"172.21.15.131:6801/2350940051","cluster_addr":"172.21.15.131:6803/2350940051","heartbeat_back_addr":"172.21.15.131:6807/2350940051","heartbeat_front_addr":"172.21.15.131:6805/2350940051","state":["exists","up"]},{"osd":1,"uuid":"42e66287-f6c6-4e26-af3c-6dfaa3536d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6825","nonce":1525528247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6827","nonce":1525528247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6831","nonce":1525528247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6829","nonce":1525528247}]},"public_addr":"172.21.15.131:6825/1525528247","cluster_addr":"172.21.15.131:6827/1525528247","heartbeat_back_addr":"172.21.15.131:6831/1525528247","heartbeat_front_addr":"172.21.15.131:6829/1525528247","state":["exists","up"]},{"osd":2,"uuid":"0da34673-04a2-4fd0-b57a-0ff2b679b043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6817","nonce":3627842210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6819","nonce":3627842210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6823","nonce":3627842210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6821","nonce":3627842210}]},"public_addr":"172.21.15.131:6817/3627842210","cluster_addr":"172.21.15.131:6819/3627842210","heartbeat_back_addr":"172.21.15.131:6823/3627842210","heartbeat_front_addr":"172.21.15.131:6821/3627842210","state":["exists","up"]},{"osd":3,"uuid":"cf117a27-d6b2-43d3-ae52-b47988bf8eed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6809","nonce":350790591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6811","nonce":350790591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6815","nonce":350790591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6813","nonce":350790591}]},"public_addr":"172.21.15.131:6809/350790591","cluster_addr":"172.21.15.131:6811/350790591","heartbeat_back_addr":"172.21.15.131:6815/350790591","heartbeat_front_addr":"172.21.15.131:6813/350790591","state":["exists","up"]},{"osd":4,"uuid":"a092ccfc-be89-4a84-88d5-0f781fa715eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6816","nonce":244560995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6819","nonce":244560995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6823","nonce":244560995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6821","nonce":244560995}]},"public_addr":"172.21.15.187:6816/244560995","cluster_addr":"172.21.15.187:6819/244560995","heartbeat_back_addr":"172.21.15.187:6823/244560995","heartbeat_front_addr":"172.21.15.187:6821/244560995","state":["exists","up"]},{"osd":5,"uuid":"7513ba69-dd00-4a35-b190-9e8cc3b2bdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6801","nonce":324619398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6803","nonce":324619398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6812","nonce":324619398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6807","nonce":324619398}]},"public_addr":"172.21.15.187:6801/324619398","cluster_addr":"172.21.15.187:6803/324619398","heartbeat_back_addr":"172.21.15.187:6812/324619398","heartbeat_front_addr":"172.21.15.187:6807/324619398","state":["exists","up"]},{"osd":6,"uuid":"50081810-c093-4d2f-a4d7-b7c695c5400d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6806","nonce":1391347786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6809","nonce":1391347786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6818","nonce":1391347786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6813","nonce":1391347786}]},"public_addr":"172.21.15.187:6806/1391347786","cluster_addr":"172.21.15.187:6809/1391347786","heartbeat_back_addr":"172.21.15.187:6818/1391347786","heartbeat_front_addr":"172.21.15.187:6813/1391347786","state":["exists","up"]},{"osd":7,"uuid":"775dae05-54f5-4ad4-bbce-92863c6ed2ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6825","nonce":1220065622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6827","nonce":1220065622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6831","nonce":1220065622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6829","nonce":1220065622}]},"public_addr":"172.21.15.187:6825/1220065622","cluster_addr":"172.21.15.187:6827/1220065622","heartbeat_back_addr":"172.21.15.187:6831/1220065622","heartbeat_front_addr":"172.21.15.187:6829/1220065622","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:05.983520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.299516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.067323+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.307748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.018742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.258775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.010912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.295355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.36","mappings":[{"from":0,"to":5}]},{"pgid":"7.38","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.187:6835/3429891101":"2024-05-27T22:50:59.823337+0000","172.21.15.187:6833/3429891101":"2024-05-27T22:50:59.823337+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-26T22:51:17.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.415+0000 7f045c800640 1 -- 172.21.15.131:0/2750694007 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f043806be30 msgr2=0x7f043806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:17.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.415+0000 7f045c800640 1 --2- 172.21.15.131:0/2750694007 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f043806be30 0x7f043806e2f0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f0444000f60 tx=0x7f0444001110 comp rx=0 tx=0).stop 2024-05-26T22:51:17.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.415+0000 7f045c800640 1 -- 172.21.15.131:0/2750694007 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04540b2f60 msgr2=0x7f04540cdb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:17.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.415+0000 7f045c800640 1 --2- 172.21.15.131:0/2750694007 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04540b2f60 0x7f04540cdb00 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f044c00c7f0 tx=0x7f044c00ccb0 comp rx=0 tx=0).stop 2024-05-26T22:51:17.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.415+0000 7f045c800640 1 -- 172.21.15.131:0/2750694007 shutdown_connections 2024-05-26T22:51:17.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.415+0000 7f045c800640 1 --2- 172.21.15.131:0/2750694007 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04540d19b0 0x7f0454155a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:17.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.415+0000 7f045c800640 1 --2- 172.21.15.131:0/2750694007 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f043806be30 0x7f043806e2f0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:17.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.415+0000 7f045c800640 1 --2- 172.21.15.131:0/2750694007 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f045415e570 0x7f04540cec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:17.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.415+0000 7f045c800640 1 --2- 172.21.15.131:0/2750694007 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04540b2f60 0x7f04540cdb00 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:17.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.415+0000 7f045c800640 1 -- 172.21.15.131:0/2750694007 >> 172.21.15.131:0/2750694007 conn(0x7f0454072e90 msgr2=0x7f04540b52a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:17.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.415+0000 7f045c800640 1 -- 172.21.15.131:0/2750694007 shutdown_connections 2024-05-26T22:51:17.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.415+0000 7f045c800640 1 -- 172.21.15.131:0/2750694007 wait complete. 2024-05-26T22:51:17.433 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-26T22:48:10.463339+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-26T22:50:24.793406+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '43', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2024-05-26T22:50:26.041636+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-26T22:51:17.434 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-26T22:51:17.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.638+0000 7f4ccb200640 1 Processor -- start 2024-05-26T22:51:17.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.638+0000 7f4ccb200640 1 -- start start 2024-05-26T22:51:17.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.640+0000 7f4ccb200640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f4cc415f6e0 0x7f4cc40ad870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:17.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.640+0000 7f4ccb200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4cc406b680 0x7f4cc406bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:17.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.640+0000 7f4ccb200640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4cc40ae0c0 con 0x7f4cc40af580 2024-05-26T22:51:17.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.640+0000 7f4ccb200640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f4cc40ae3a0 con 0x7f4cc415f6e0 2024-05-26T22:51:17.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.640+0000 7f4ccb200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f4cc40ae680 con 0x7f4cc406b680 2024-05-26T22:51:17.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.641+0000 7f4cc9200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4cc406b680 0x7f4cc406bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:17.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.641+0000 7f4cc3400640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f4cc415f6e0 0x7f4cc40ad870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:17.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.641+0000 7f4cc9200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4cc406b680 0x7f4cc406bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.131:57060/0 (socket says 172.21.15.131:57060) 2024-05-26T22:51:17.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.641+0000 7f4cc9200640 1 -- 172.21.15.131:0/1538289939 learned_addr learned my addr 172.21.15.131:0/1538289939 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:17.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.642+0000 7f4cc3400640 1 -- 172.21.15.131:0/1538289939 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4cc406b680 msgr2=0x7f4cc406bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:17.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.642+0000 7f4cc3400640 1 --2- 172.21.15.131:0/1538289939 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4cc406b680 0x7f4cc406bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:17.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.642+0000 7f4cc3400640 1 -- 172.21.15.131:0/1538289939 >> v1:172.21.15.187:6789/0 conn(0x7f4cc40af580 legacy=0x7f4cc40af960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:17.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.642+0000 7f4cc3400640 1 -- 172.21.15.131:0/1538289939 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cc40ae960 con 0x7f4cc415f6e0 2024-05-26T22:51:17.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.643+0000 7f4cc3400640 1 --2- 172.21.15.131:0/1538289939 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4cc415f6e0 0x7f4cc40ad870 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4cb00097d0 tx=0x7f4cb002a0a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4234cd20a83c3767 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:17.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.644+0000 7f4cc2a00640 1 -- 172.21.15.131:0/1538289939 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cb0037070 con 0x7f4cc415f6e0 2024-05-26T22:51:17.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.644+0000 7f4cc2a00640 1 -- 172.21.15.131:0/1538289939 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4cb002adb0 con 0x7f4cc415f6e0 2024-05-26T22:51:17.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.644+0000 7f4cc2a00640 1 -- 172.21.15.131:0/1538289939 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cb0032660 con 0x7f4cc415f6e0 2024-05-26T22:51:17.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.644+0000 7f4ccb200640 1 -- 172.21.15.131:0/1538289939 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4cc415f6e0 msgr2=0x7f4cc40ad870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:17.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.644+0000 7f4ccb200640 1 --2- 172.21.15.131:0/1538289939 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4cc415f6e0 0x7f4cc40ad870 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4cb00097d0 tx=0x7f4cb002a0a0 comp rx=0 tx=0).stop 2024-05-26T22:51:17.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.644+0000 7f4ccb200640 1 -- 172.21.15.131:0/1538289939 shutdown_connections 2024-05-26T22:51:17.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.644+0000 7f4ccb200640 1 --2- 172.21.15.131:0/1538289939 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4cc406b680 0x7f4cc406bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:17.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.644+0000 7f4ccb200640 1 --2- 172.21.15.131:0/1538289939 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4cc415f6e0 0x7f4cc40ad870 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:17.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.644+0000 7f4ccb200640 1 -- 172.21.15.131:0/1538289939 >> 172.21.15.131:0/1538289939 conn(0x7f4cc4072b00 msgr2=0x7f4cc406de50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:17.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.645+0000 7f4ccb200640 1 -- 172.21.15.131:0/1538289939 shutdown_connections 2024-05-26T22:51:17.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.645+0000 7f4ccb200640 1 -- 172.21.15.131:0/1538289939 wait complete. 2024-05-26T22:51:17.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.646+0000 7f4ccb200640 1 Processor -- start 2024-05-26T22:51:17.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.646+0000 7f4ccb200640 1 -- start start 2024-05-26T22:51:17.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.647+0000 7f4ccb200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4cc406b680 0x7f4cc41470c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:17.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.647+0000 7f4ccb200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4cc40af580 0x7f4cc40b09b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:17.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.647+0000 7f4ccb200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4cc415f6e0 0x7f4cc40b0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:17.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.647+0000 7f4ccb200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc40b6a40 con 0x7f4cc406b680 2024-05-26T22:51:17.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.647+0000 7f4ccb200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc40b68c0 con 0x7f4cc40af580 2024-05-26T22:51:17.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.647+0000 7f4ccb200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f4cc406c050 con 0x7f4cc415f6e0 2024-05-26T22:51:17.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.648+0000 7f4cc3e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4cc406b680 0x7f4cc41470c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:17.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.648+0000 7f4cc3e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4cc406b680 0x7f4cc41470c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44878/0 (socket says 172.21.15.131:44878) 2024-05-26T22:51:17.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.648+0000 7f4cc3e00640 1 -- 172.21.15.131:0/3310695216 learned_addr learned my addr 172.21.15.131:0/3310695216 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:17.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.648+0000 7f4cc9200640 1 --2- 172.21.15.131:0/3310695216 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4cc415f6e0 0x7f4cc40b0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:17.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.648+0000 7f4cc3400640 1 --2- 172.21.15.131:0/3310695216 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4cc40af580 0x7f4cc40b09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:17.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.648+0000 7f4cc3e00640 1 -- 172.21.15.131:0/3310695216 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4cc415f6e0 msgr2=0x7f4cc40b0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:17.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.648+0000 7f4cc3e00640 1 --2- 172.21.15.131:0/3310695216 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4cc415f6e0 0x7f4cc40b0ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:17.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.648+0000 7f4cc3e00640 1 -- 172.21.15.131:0/3310695216 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4cc40af580 msgr2=0x7f4cc40b09b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:17.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.648+0000 7f4cc3e00640 1 --2- 172.21.15.131:0/3310695216 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4cc40af580 0x7f4cc40b09b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:17.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.648+0000 7f4cc3e00640 1 -- 172.21.15.131:0/3310695216 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cc40cd200 con 0x7f4cc406b680 2024-05-26T22:51:17.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.648+0000 7f4cc9200640 1 --2- 172.21.15.131:0/3310695216 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4cc415f6e0 0x7f4cc40b0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:51:17.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.649+0000 7f4cc3e00640 1 --2- 172.21.15.131:0/3310695216 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4cc406b680 0x7f4cc41470c0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f4cb400c790 tx=0x7f4cb400cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:17.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.649+0000 7f4cbbe00640 1 -- 172.21.15.131:0/3310695216 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cb4016020 con 0x7f4cc406b680 2024-05-26T22:51:17.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.649+0000 7f4cbbe00640 1 -- 172.21.15.131:0/3310695216 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4cb4006d50 con 0x7f4cc406b680 2024-05-26T22:51:17.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.649+0000 7f4cbbe00640 1 -- 172.21.15.131:0/3310695216 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cb4003bb0 con 0x7f4cc406b680 2024-05-26T22:51:17.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.649+0000 7f4ccb200640 1 -- 172.21.15.131:0/3310695216 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cc40cd4e0 con 0x7f4cc406b680 2024-05-26T22:51:17.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.649+0000 7f4ccb200640 1 -- 172.21.15.131:0/3310695216 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cc40cdaa0 con 0x7f4cc406b680 2024-05-26T22:51:17.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.650+0000 7f4ccb200640 1 -- 172.21.15.131:0/3310695216 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cc40695c0 con 0x7f4cc406b680 2024-05-26T22:51:17.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.651+0000 7f4cbbe00640 1 -- 172.21.15.131:0/3310695216 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4cb400a850 con 0x7f4cc406b680 2024-05-26T22:51:17.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.652+0000 7f4cbbe00640 1 --2- 172.21.15.131:0/3310695216 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f4ca406be30 0x7f4ca406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:17.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.652+0000 7f4cbbe00640 1 -- 172.21.15.131:0/3310695216 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5853+0+0 (secure 0 0 0) 0x7f4cb408ac80 con 0x7f4cc406b680 2024-05-26T22:51:17.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.653+0000 7f4cc3400640 1 --2- 172.21.15.131:0/3310695216 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f4ca406be30 0x7f4ca406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:17.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.653+0000 7f4cc3400640 1 --2- 172.21.15.131:0/3310695216 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f4ca406be30 0x7f4ca406e2f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f4cb0034000 tx=0x7f4cb002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:17.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.657+0000 7f4cbbe00640 1 -- 172.21.15.131:0/3310695216 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f4cb4020020 con 0x7f4cc406b680 2024-05-26T22:51:17.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.852+0000 7f4ccb200640 1 -- 172.21.15.131:0/3310695216 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4cc4067320 con 0x7f4cc406b680 2024-05-26T22:51:17.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.853+0000 7f4cbbe00640 1 -- 172.21.15.131:0/3310695216 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v47) ==== 93+0+10 (secure 0 0 0) 0x7f4cb4019310 con 0x7f4cc406b680 2024-05-26T22:51:17.853 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2024-05-26T22:51:17.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.856+0000 7f4ccb200640 1 -- 172.21.15.131:0/3310695216 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f4ca406be30 msgr2=0x7f4ca406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:17.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.856+0000 7f4ccb200640 1 --2- 172.21.15.131:0/3310695216 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f4ca406be30 0x7f4ca406e2f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f4cb0034000 tx=0x7f4cb002e000 comp rx=0 tx=0).stop 2024-05-26T22:51:17.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.856+0000 7f4ccb200640 1 -- 172.21.15.131:0/3310695216 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4cc406b680 msgr2=0x7f4cc41470c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:17.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.856+0000 7f4ccb200640 1 --2- 172.21.15.131:0/3310695216 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4cc406b680 0x7f4cc41470c0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f4cb400c790 tx=0x7f4cb400cc50 comp rx=0 tx=0).stop 2024-05-26T22:51:17.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.856+0000 7f4ccb200640 1 -- 172.21.15.131:0/3310695216 shutdown_connections 2024-05-26T22:51:17.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.856+0000 7f4ccb200640 1 --2- 172.21.15.131:0/3310695216 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4cc415f6e0 0x7f4cc40b0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:17.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.856+0000 7f4ccb200640 1 --2- 172.21.15.131:0/3310695216 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f4ca406be30 0x7f4ca406e2f0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:17.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.856+0000 7f4ccb200640 1 --2- 172.21.15.131:0/3310695216 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4cc40af580 0x7f4cc40b09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:17.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.856+0000 7f4ccb200640 1 --2- 172.21.15.131:0/3310695216 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4cc406b680 0x7f4cc41470c0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:17.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.856+0000 7f4ccb200640 1 -- 172.21.15.131:0/3310695216 >> 172.21.15.131:0/3310695216 conn(0x7f4cc4072b00 msgr2=0x7f4cc4153f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:17.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.856+0000 7f4ccb200640 1 -- 172.21.15.131:0/3310695216 shutdown_connections 2024-05-26T22:51:17.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:17.856+0000 7f4ccb200640 1 -- 172.21.15.131:0/3310695216 wait complete. 2024-05-26T22:51:17.875 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-26T22:51:18.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.082+0000 7f3005600640 1 Processor -- start 2024-05-26T22:51:18.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.083+0000 7f3005600640 1 -- start start 2024-05-26T22:51:18.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.084+0000 7f3005600640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3000165e30 0x7f3000072540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:18.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.084+0000 7f3005600640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f30000af230 0x7f30000cfe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:18.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.084+0000 7f3005600640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f30000d0390 0x7f3000071360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:18.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.084+0000 7f3005600640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f3000072e90 con 0x7f30000d0390 2024-05-26T22:51:18.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.084+0000 7f3005600640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f30000733c0 con 0x7f30000af230 2024-05-26T22:51:18.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.084+0000 7f3005600640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f30000cf540 con 0x7f3000165e30 2024-05-26T22:51:18.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.085+0000 7f2fff400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f30000d0390 0x7f3000071360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:18.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.085+0000 7f2fff400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f30000d0390 0x7f3000071360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44886/0 (socket says 172.21.15.131:44886) 2024-05-26T22:51:18.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.085+0000 7f2fff400640 1 -- 172.21.15.131:0/254460412 learned_addr learned my addr 172.21.15.131:0/254460412 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:18.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.085+0000 7f2ffea00640 1 --2- 172.21.15.131:0/254460412 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3000165e30 0x7f3000072540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:18.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.085+0000 7f2fff400640 1 -- 172.21.15.131:0/254460412 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3000165e30 msgr2=0x7f3000072540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:18.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.085+0000 7f2fff400640 1 --2- 172.21.15.131:0/254460412 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3000165e30 0x7f3000072540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:18.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.085+0000 7f2fff400640 1 -- 172.21.15.131:0/254460412 >> v2:172.21.15.187:3300/0 conn(0x7f30000af230 msgr2=0x7f30000cfe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:18.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.085+0000 7f2ff7e00640 1 --2- 172.21.15.131:0/254460412 >> v2:172.21.15.187:3300/0 conn(0x7f30000af230 0x7f30000cfe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:18.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.085+0000 7f2fff400640 1 --2- 172.21.15.131:0/254460412 >> v2:172.21.15.187:3300/0 conn(0x7f30000af230 0x7f30000cfe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:18.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.085+0000 7f2fff400640 1 -- 172.21.15.131:0/254460412 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30000730e0 con 0x7f30000d0390 2024-05-26T22:51:18.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.085+0000 7f2ffea00640 1 --2- 172.21.15.131:0/254460412 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3000165e30 0x7f3000072540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:51:18.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.085+0000 7f2ff7e00640 1 --2- 172.21.15.131:0/254460412 >> v2:172.21.15.187:3300/0 conn(0x7f30000af230 0x7f30000cfe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:51:18.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.086+0000 7f2fff400640 1 --2- 172.21.15.131:0/254460412 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f30000d0390 0x7f3000071360 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f30000737f0 tx=0x7f2ff802a600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ced46cd7fd1d6315 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:18.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.087+0000 7f2ffe000640 1 -- 172.21.15.131:0/254460412 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff8036070 con 0x7f30000d0390 2024-05-26T22:51:18.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.087+0000 7f2ffe000640 1 -- 172.21.15.131:0/254460412 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ff8031600 con 0x7f30000d0390 2024-05-26T22:51:18.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.087+0000 7f2ffe000640 1 -- 172.21.15.131:0/254460412 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff80319d0 con 0x7f30000d0390 2024-05-26T22:51:18.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.087+0000 7f3005600640 1 -- 172.21.15.131:0/254460412 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f30000d0390 msgr2=0x7f3000071360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:18.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.087+0000 7f3005600640 1 --2- 172.21.15.131:0/254460412 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f30000d0390 0x7f3000071360 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f30000737f0 tx=0x7f2ff802a600 comp rx=0 tx=0).stop 2024-05-26T22:51:18.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.087+0000 7f3005600640 1 -- 172.21.15.131:0/254460412 shutdown_connections 2024-05-26T22:51:18.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.087+0000 7f3005600640 1 --2- 172.21.15.131:0/254460412 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f30000d0390 0x7f3000071360 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:18.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.087+0000 7f3005600640 1 --2- 172.21.15.131:0/254460412 >> v2:172.21.15.187:3300/0 conn(0x7f30000af230 0x7f30000cfe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:18.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.087+0000 7f3005600640 1 --2- 172.21.15.131:0/254460412 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3000165e30 0x7f3000072540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:18.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.088+0000 7f3005600640 1 -- 172.21.15.131:0/254460412 >> 172.21.15.131:0/254460412 conn(0x7f300006bc90 msgr2=0x7f30000b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:18.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.088+0000 7f3005600640 1 -- 172.21.15.131:0/254460412 shutdown_connections 2024-05-26T22:51:18.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.088+0000 7f3005600640 1 -- 172.21.15.131:0/254460412 wait complete. 2024-05-26T22:51:18.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.089+0000 7f3005600640 1 Processor -- start 2024-05-26T22:51:18.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.089+0000 7f3005600640 1 -- start start 2024-05-26T22:51:18.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.090+0000 7f3005600640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f30000af230 0x7f300009c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:18.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.090+0000 7f3005600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f30000d0390 0x7f3000157360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:18.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.090+0000 7f3005600640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3000165e30 0x7f30000ae570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:18.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.090+0000 7f3005600640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f30000718a0 con 0x7f30000d0390 2024-05-26T22:51:18.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.090+0000 7f3005600640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f30000cfac0 con 0x7f3000165e30 2024-05-26T22:51:18.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.090+0000 7f3005600640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3000071a20 con 0x7f30000af230 2024-05-26T22:51:18.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.091+0000 7f2ff7e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f30000d0390 0x7f3000157360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:18.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.091+0000 7f2ff7e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f30000d0390 0x7f3000157360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44896/0 (socket says 172.21.15.131:44896) 2024-05-26T22:51:18.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.091+0000 7f2ff7e00640 1 -- 172.21.15.131:0/2279573153 learned_addr learned my addr 172.21.15.131:0/2279573153 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:18.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.091+0000 7f2ffea00640 1 --2- 172.21.15.131:0/2279573153 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f30000af230 0x7f300009c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:18.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.091+0000 7f2ff7e00640 1 -- 172.21.15.131:0/2279573153 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f30000af230 msgr2=0x7f300009c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:18.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.091+0000 7f2fff400640 1 --2- 172.21.15.131:0/2279573153 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3000165e30 0x7f30000ae570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:18.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.091+0000 7f2ff7e00640 1 --2- 172.21.15.131:0/2279573153 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f30000af230 0x7f300009c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:18.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.091+0000 7f2ff7e00640 1 -- 172.21.15.131:0/2279573153 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3000165e30 msgr2=0x7f30000ae570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:18.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.091+0000 7f2ff7e00640 1 --2- 172.21.15.131:0/2279573153 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3000165e30 0x7f30000ae570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:18.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.091+0000 7f2ff7e00640 1 -- 172.21.15.131:0/2279573153 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f300009aed0 con 0x7f30000d0390 2024-05-26T22:51:18.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.091+0000 7f2ff7e00640 1 --2- 172.21.15.131:0/2279573153 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f30000d0390 0x7f3000157360 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f2fe800c7c0 tx=0x7f2fe800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:18.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.092+0000 7f2ff6a00640 1 -- 172.21.15.131:0/2279573153 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe8015070 con 0x7f30000d0390 2024-05-26T22:51:18.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.092+0000 7f2ff6a00640 1 -- 172.21.15.131:0/2279573153 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fe8007a30 con 0x7f30000d0390 2024-05-26T22:51:18.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.092+0000 7f2ff6a00640 1 -- 172.21.15.131:0/2279573153 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe8007e00 con 0x7f30000d0390 2024-05-26T22:51:18.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.092+0000 7f3005600640 1 -- 172.21.15.131:0/2279573153 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f300009b1b0 con 0x7f30000d0390 2024-05-26T22:51:18.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.092+0000 7f3005600640 1 -- 172.21.15.131:0/2279573153 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f300009b770 con 0x7f30000d0390 2024-05-26T22:51:18.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.093+0000 7f3005600640 1 -- 172.21.15.131:0/2279573153 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3000068f00 con 0x7f30000d0390 2024-05-26T22:51:18.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.094+0000 7f2ff6a00640 1 -- 172.21.15.131:0/2279573153 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2fe8018070 con 0x7f30000d0390 2024-05-26T22:51:18.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.095+0000 7f2ff6a00640 1 --2- 172.21.15.131:0/2279573153 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f2fdc06be30 0x7f2fdc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:18.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.096+0000 7f2ff6a00640 1 -- 172.21.15.131:0/2279573153 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5853+0+0 (secure 0 0 0) 0x7f2fe808af00 con 0x7f30000d0390 2024-05-26T22:51:18.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.096+0000 7f2ffea00640 1 --2- 172.21.15.131:0/2279573153 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f2fdc06be30 0x7f2fdc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:18.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.096+0000 7f2ffea00640 1 --2- 172.21.15.131:0/2279573153 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f2fdc06be30 0x7f2fdc06e2f0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f2ff000b840 tx=0x7f2ff0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:18.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.100+0000 7f2ff6a00640 1 -- 172.21.15.131:0/2279573153 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f3000068f00 con 0x7f30000d0390 2024-05-26T22:51:18.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.295+0000 7f3005600640 1 -- 172.21.15.131:0/2279573153 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f300009acb0 con 0x7f30000d0390 2024-05-26T22:51:18.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.296+0000 7f2ff6a00640 1 -- 172.21.15.131:0/2279573153 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v47) ==== 104+0+11 (secure 0 0 0) 0x7f2fe8059780 con 0x7f30000d0390 2024-05-26T22:51:18.296 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-05-26T22:51:18.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.299+0000 7f3005600640 1 -- 172.21.15.131:0/2279573153 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f2fdc06be30 msgr2=0x7f2fdc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:18.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.299+0000 7f3005600640 1 --2- 172.21.15.131:0/2279573153 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f2fdc06be30 0x7f2fdc06e2f0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f2ff000b840 tx=0x7f2ff0008040 comp rx=0 tx=0).stop 2024-05-26T22:51:18.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.299+0000 7f3005600640 1 -- 172.21.15.131:0/2279573153 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f30000d0390 msgr2=0x7f3000157360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:18.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.299+0000 7f3005600640 1 --2- 172.21.15.131:0/2279573153 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f30000d0390 0x7f3000157360 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f2fe800c7c0 tx=0x7f2fe800cc80 comp rx=0 tx=0).stop 2024-05-26T22:51:18.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.299+0000 7f3005600640 1 -- 172.21.15.131:0/2279573153 shutdown_connections 2024-05-26T22:51:18.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.299+0000 7f3005600640 1 --2- 172.21.15.131:0/2279573153 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f2fdc06be30 0x7f2fdc06e2f0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:18.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.299+0000 7f3005600640 1 --2- 172.21.15.131:0/2279573153 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3000165e30 0x7f30000ae570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:18.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.299+0000 7f3005600640 1 --2- 172.21.15.131:0/2279573153 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f30000d0390 0x7f3000157360 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:18.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.299+0000 7f3005600640 1 --2- 172.21.15.131:0/2279573153 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f30000af230 0x7f300009c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:18.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.299+0000 7f3005600640 1 -- 172.21.15.131:0/2279573153 >> 172.21.15.131:0/2279573153 conn(0x7f300006bc90 msgr2=0x7f30000afb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:18.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.300+0000 7f3005600640 1 -- 172.21.15.131:0/2279573153 shutdown_connections 2024-05-26T22:51:18.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.300+0000 7f3005600640 1 -- 172.21.15.131:0/2279573153 wait complete. 2024-05-26T22:51:18.317 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-26T22:51:18.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.509+0000 7f851f200640 1 Processor -- start 2024-05-26T22:51:18.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.509+0000 7f851f200640 1 -- start start 2024-05-26T22:51:18.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.510+0000 7f851f200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f852015dc40 0x7f852006d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:18.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.510+0000 7f851f200640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f8520071c10 0x7f852006d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:18.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.510+0000 7f851f200640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85201552d0 con 0x7f852015bfa0 2024-05-26T22:51:18.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.510+0000 7f851f200640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f85200b3f30 con 0x7f852015dc40 2024-05-26T22:51:18.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.510+0000 7f851f200640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f8520072230 con 0x7f8520071c10 2024-05-26T22:51:18.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.511+0000 7f851de00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f852015dc40 0x7f852006d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:18.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.511+0000 7f851de00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f852015dc40 0x7f852006d250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44898/0 (socket says 172.21.15.131:44898) 2024-05-26T22:51:18.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.511+0000 7f851de00640 1 -- 172.21.15.131:0/1956391987 learned_addr learned my addr 172.21.15.131:0/1956391987 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:18.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.511+0000 7f851de00640 1 -- 172.21.15.131:0/1956391987 >> v2:172.21.15.187:3300/0 conn(0x7f8520071c10 msgr2=0x7f852006d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:18.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.511+0000 7f851de00640 1 --2- 172.21.15.131:0/1956391987 >> v2:172.21.15.187:3300/0 conn(0x7f8520071c10 0x7f852006d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:18.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.511+0000 7f851de00640 1 -- 172.21.15.131:0/1956391987 >> v1:172.21.15.187:6789/0 conn(0x7f852015bfa0 legacy=0x7f8520071500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:18.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.511+0000 7f851de00640 1 -- 172.21.15.131:0/1956391987 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8520072510 con 0x7f852015dc40 2024-05-26T22:51:18.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.512+0000 7f851de00640 1 --2- 172.21.15.131:0/1956391987 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f852015dc40 0x7f852006d250 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f8520072960 tx=0x7f851002b5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b40a971b76658651 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:18.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.512+0000 7f851ca00640 1 -- 172.21.15.131:0/1956391987 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f851000b040 con 0x7f852015dc40 2024-05-26T22:51:18.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.513+0000 7f851ca00640 1 -- 172.21.15.131:0/1956391987 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8510030710 con 0x7f852015dc40 2024-05-26T22:51:18.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.513+0000 7f851ca00640 1 -- 172.21.15.131:0/1956391987 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8510030ae0 con 0x7f852015dc40 2024-05-26T22:51:18.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.513+0000 7f851f200640 1 -- 172.21.15.131:0/1956391987 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f852015dc40 msgr2=0x7f852006d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:18.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.513+0000 7f851f200640 1 --2- 172.21.15.131:0/1956391987 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f852015dc40 0x7f852006d250 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f8520072960 tx=0x7f851002b5e0 comp rx=0 tx=0).stop 2024-05-26T22:51:18.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.513+0000 7f851f200640 1 -- 172.21.15.131:0/1956391987 shutdown_connections 2024-05-26T22:51:18.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.513+0000 7f851f200640 1 --2- 172.21.15.131:0/1956391987 >> v2:172.21.15.187:3300/0 conn(0x7f8520071c10 0x7f852006d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:18.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.513+0000 7f851f200640 1 --2- 172.21.15.131:0/1956391987 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f852015dc40 0x7f852006d250 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:18.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.513+0000 7f851f200640 1 -- 172.21.15.131:0/1956391987 >> 172.21.15.131:0/1956391987 conn(0x7f85200b5660 msgr2=0x7f85200b5a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:18.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.513+0000 7f851f200640 1 -- 172.21.15.131:0/1956391987 shutdown_connections 2024-05-26T22:51:18.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.513+0000 7f851f200640 1 -- 172.21.15.131:0/1956391987 wait complete. 2024-05-26T22:51:18.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.515+0000 7f851f200640 1 Processor -- start 2024-05-26T22:51:18.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.515+0000 7f851f200640 1 -- start start 2024-05-26T22:51:18.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.516+0000 7f851f200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8520071c10 0x7f8520149c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:18.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.516+0000 7f851f200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f852015bfa0 0x7f85200afb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:18.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.516+0000 7f851f200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f852015dc40 0x7f85200b0040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:18.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.516+0000 7f851f200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f852015e1a0 con 0x7f8520071c10 2024-05-26T22:51:18.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.516+0000 7f851f200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f852015e020 con 0x7f852015bfa0 2024-05-26T22:51:18.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.516+0000 7f851f200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f85200b2080 con 0x7f852015dc40 2024-05-26T22:51:18.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.516+0000 7f851de00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8520071c10 0x7f8520149c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:18.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.516+0000 7f851de00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8520071c10 0x7f8520149c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44900/0 (socket says 172.21.15.131:44900) 2024-05-26T22:51:18.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.516+0000 7f851de00640 1 -- 172.21.15.131:0/3759284286 learned_addr learned my addr 172.21.15.131:0/3759284286 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:18.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.517+0000 7f851e800640 1 --2- 172.21.15.131:0/3759284286 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f852015dc40 0x7f85200b0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:18.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.517+0000 7f851d400640 1 --2- 172.21.15.131:0/3759284286 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f852015bfa0 0x7f85200afb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:18.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.517+0000 7f851de00640 1 -- 172.21.15.131:0/3759284286 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f852015dc40 msgr2=0x7f85200b0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:18.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.517+0000 7f851de00640 1 --2- 172.21.15.131:0/3759284286 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f852015dc40 0x7f85200b0040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:18.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.517+0000 7f851de00640 1 -- 172.21.15.131:0/3759284286 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f852015bfa0 msgr2=0x7f85200afb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:18.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.517+0000 7f851de00640 1 --2- 172.21.15.131:0/3759284286 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f852015bfa0 0x7f85200afb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:18.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.517+0000 7f851de00640 1 -- 172.21.15.131:0/3759284286 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f852014a140 con 0x7f8520071c10 2024-05-26T22:51:18.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.517+0000 7f851d400640 1 --2- 172.21.15.131:0/3759284286 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f852015bfa0 0x7f85200afb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:51:18.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.517+0000 7f851de00640 1 --2- 172.21.15.131:0/3759284286 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8520071c10 0x7f8520149c00 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f851000a1c0 tx=0x7f85100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:18.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.518+0000 7f8516000640 1 -- 172.21.15.131:0/3759284286 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f851003f070 con 0x7f8520071c10 2024-05-26T22:51:18.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.518+0000 7f8516000640 1 -- 172.21.15.131:0/3759284286 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8510030e30 con 0x7f8520071c10 2024-05-26T22:51:18.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.518+0000 7f8516000640 1 -- 172.21.15.131:0/3759284286 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f851003b430 con 0x7f8520071c10 2024-05-26T22:51:18.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.518+0000 7f851f200640 1 -- 172.21.15.131:0/3759284286 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f852014ae00 con 0x7f8520071c10 2024-05-26T22:51:18.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.518+0000 7f851f200640 1 -- 172.21.15.131:0/3759284286 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f852014b3c0 con 0x7f8520071c10 2024-05-26T22:51:18.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.519+0000 7f851f200640 1 -- 172.21.15.131:0/3759284286 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85200694e0 con 0x7f8520071c10 2024-05-26T22:51:18.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.520+0000 7f8516000640 1 -- 172.21.15.131:0/3759284286 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8510042070 con 0x7f8520071c10 2024-05-26T22:51:18.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.521+0000 7f8516000640 1 --2- 172.21.15.131:0/3759284286 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f850006be30 0x7f850006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:18.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.521+0000 7f8516000640 1 -- 172.21.15.131:0/3759284286 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5853+0+0 (secure 0 0 0) 0x7f85100aba20 con 0x7f8520071c10 2024-05-26T22:51:18.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.521+0000 7f851d400640 1 --2- 172.21.15.131:0/3759284286 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f850006be30 0x7f850006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:18.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.522+0000 7f851d400640 1 --2- 172.21.15.131:0/3759284286 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f850006be30 0x7f850006e2f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f850c007fd0 tx=0x7f850c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:18.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.526+0000 7f8516000640 1 -- 172.21.15.131:0/3759284286 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f851007a330 con 0x7f8520071c10 2024-05-26T22:51:18.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.724+0000 7f851f200640 1 -- 172.21.15.131:0/3759284286 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f85200b35d0 con 0x7f8520071c10 2024-05-26T22:51:18.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.725+0000 7f8516000640 1 -- 172.21.15.131:0/3759284286 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v47) ==== 100+0+11 (secure 0 0 0) 0x7f8510079c30 con 0x7f8520071c10 2024-05-26T22:51:18.724 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 64 2024-05-26T22:51:18.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.728+0000 7f851f200640 1 -- 172.21.15.131:0/3759284286 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f850006be30 msgr2=0x7f850006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:18.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.728+0000 7f851f200640 1 --2- 172.21.15.131:0/3759284286 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f850006be30 0x7f850006e2f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f850c007fd0 tx=0x7f850c009040 comp rx=0 tx=0).stop 2024-05-26T22:51:18.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.728+0000 7f851f200640 1 -- 172.21.15.131:0/3759284286 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8520071c10 msgr2=0x7f8520149c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:18.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.728+0000 7f851f200640 1 --2- 172.21.15.131:0/3759284286 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8520071c10 0x7f8520149c00 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f851000a1c0 tx=0x7f85100046c0 comp rx=0 tx=0).stop 2024-05-26T22:51:18.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.728+0000 7f851f200640 1 -- 172.21.15.131:0/3759284286 shutdown_connections 2024-05-26T22:51:18.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.728+0000 7f851f200640 1 --2- 172.21.15.131:0/3759284286 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f852015dc40 0x7f85200b0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:18.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.728+0000 7f851f200640 1 --2- 172.21.15.131:0/3759284286 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f850006be30 0x7f850006e2f0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:18.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.728+0000 7f851f200640 1 --2- 172.21.15.131:0/3759284286 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f852015bfa0 0x7f85200afb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:18.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.728+0000 7f851f200640 1 --2- 172.21.15.131:0/3759284286 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8520071c10 0x7f8520149c00 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:18.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.728+0000 7f851f200640 1 -- 172.21.15.131:0/3759284286 >> 172.21.15.131:0/3759284286 conn(0x7f85200b5660 msgr2=0x7f852006d980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:18.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.729+0000 7f851f200640 1 -- 172.21.15.131:0/3759284286 shutdown_connections 2024-05-26T22:51:18.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.729+0000 7f851f200640 1 -- 172.21.15.131:0/3759284286 wait complete. 2024-05-26T22:51:18.746 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-26T22:51:18.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.938+0000 7f7a17200640 1 Processor -- start 2024-05-26T22:51:18.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.938+0000 7f7a17200640 1 -- start start 2024-05-26T22:51:18.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.940+0000 7f7a17200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7a180712d0 0x7f7a1806d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:18.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.940+0000 7f7a17200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f7a1806d790 0x7f7a180cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:18.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.940+0000 7f7a17200640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a18072d60 con 0x7f7a18145a10 2024-05-26T22:51:18.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.940+0000 7f7a17200640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f7a1815c760 con 0x7f7a1806d790 2024-05-26T22:51:18.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.940+0000 7f7a17200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f7a18072a80 con 0x7f7a180712d0 2024-05-26T22:51:18.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.940+0000 7f7a15e00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f7a18145a10 0x7f7a18072180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:51320/0 (socket says 172.21.15.131:51320) 2024-05-26T22:51:18.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.940+0000 7f7a15e00640 1 -- 172.21.15.131:0/3477210402 learned_addr learned my addr 172.21.15.131:0/3477210402 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:51:18.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.940+0000 7f7a16800640 1 --2- 172.21.15.131:0/3477210402 >> v2:172.21.15.131:3300/0 conn(0x7f7a1806d790 0x7f7a180cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:18.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.941+0000 7f7a0fe00640 1 --2- 172.21.15.131:0/3477210402 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7a180712d0 0x7f7a1806d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:18.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.941+0000 7f7a15400640 1 -- 172.21.15.131:0/3477210402 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 9493388 0 0) 0x7f7a18072d60 con 0x7f7a18145a10 2024-05-26T22:51:18.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.941+0000 7f7a15400640 1 -- 172.21.15.131:0/3477210402 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f79fc003520 con 0x7f7a18145a10 2024-05-26T22:51:18.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.941+0000 7f7a16800640 1 -- 172.21.15.131:0/3477210402 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7a180712d0 msgr2=0x7f7a1806d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:18.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.941+0000 7f7a16800640 1 --2- 172.21.15.131:0/3477210402 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7a180712d0 0x7f7a1806d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:18.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.941+0000 7f7a16800640 1 -- 172.21.15.131:0/3477210402 >> v1:172.21.15.131:6789/0 conn(0x7f7a18145a10 legacy=0x7f7a18072180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:18.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.941+0000 7f7a16800640 1 -- 172.21.15.131:0/3477210402 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a180ceab0 con 0x7f7a1806d790 2024-05-26T22:51:18.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.942+0000 7f7a16800640 1 --2- 172.21.15.131:0/3477210402 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7a1806d790 0x7f7a180cf590 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f7a10009d90 tx=0x7f7a1002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56e3abb81eb02ae9 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:18.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.942+0000 7f7a15400640 1 -- 172.21.15.131:0/3477210402 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a10007740 con 0x7f7a1806d790 2024-05-26T22:51:18.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.942+0000 7f7a15400640 1 -- 172.21.15.131:0/3477210402 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a100078e0 con 0x7f7a1806d790 2024-05-26T22:51:18.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.942+0000 7f7a15400640 1 -- 172.21.15.131:0/3477210402 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a10007cb0 con 0x7f7a1806d790 2024-05-26T22:51:18.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.943+0000 7f7a17200640 1 -- 172.21.15.131:0/3477210402 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7a1806d790 msgr2=0x7f7a180cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:18.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.943+0000 7f7a17200640 1 --2- 172.21.15.131:0/3477210402 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7a1806d790 0x7f7a180cf590 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f7a10009d90 tx=0x7f7a1002a660 comp rx=0 tx=0).stop 2024-05-26T22:51:18.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.943+0000 7f7a17200640 1 -- 172.21.15.131:0/3477210402 shutdown_connections 2024-05-26T22:51:18.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.943+0000 7f7a17200640 1 --2- 172.21.15.131:0/3477210402 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7a1806d790 0x7f7a180cf590 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:18.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.943+0000 7f7a17200640 1 --2- 172.21.15.131:0/3477210402 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7a180712d0 0x7f7a1806d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:18.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.943+0000 7f7a17200640 1 -- 172.21.15.131:0/3477210402 >> 172.21.15.131:0/3477210402 conn(0x7f7a180b5660 msgr2=0x7f7a180b5a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:18.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.943+0000 7f7a17200640 1 -- 172.21.15.131:0/3477210402 shutdown_connections 2024-05-26T22:51:18.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.943+0000 7f7a17200640 1 -- 172.21.15.131:0/3477210402 wait complete. 2024-05-26T22:51:18.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.944+0000 7f7a17200640 1 Processor -- start 2024-05-26T22:51:18.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.945+0000 7f7a17200640 1 -- start start 2024-05-26T22:51:18.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.945+0000 7f7a17200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7a1806d790 0x7f7a1814dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:18.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.945+0000 7f7a17200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7a180712d0 0x7f7a1814e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:18.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.945+0000 7f7a17200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7a18145a10 0x7f7a1815a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:18.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.945+0000 7f7a17200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f7a18145df0 con 0x7f7a18145a10 2024-05-26T22:51:18.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.945+0000 7f7a17200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7a180cf070 con 0x7f7a1806d790 2024-05-26T22:51:18.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.945+0000 7f7a17200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f7a180cfad0 con 0x7f7a180712d0 2024-05-26T22:51:18.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.946+0000 7f7a16800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7a18145a10 0x7f7a1815a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:18.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.946+0000 7f7a16800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7a18145a10 0x7f7a1815a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44920/0 (socket says 172.21.15.131:44920) 2024-05-26T22:51:18.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.946+0000 7f7a16800640 1 -- 172.21.15.131:0/1785573538 learned_addr learned my addr 172.21.15.131:0/1785573538 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:18.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.946+0000 7f7a0fe00640 1 --2- 172.21.15.131:0/1785573538 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7a180712d0 0x7f7a1814e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:18.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.946+0000 7f7a15e00640 1 --2- 172.21.15.131:0/1785573538 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7a1806d790 0x7f7a1814dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:18.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.946+0000 7f7a16800640 1 -- 172.21.15.131:0/1785573538 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7a180712d0 msgr2=0x7f7a1814e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:18.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.946+0000 7f7a16800640 1 --2- 172.21.15.131:0/1785573538 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7a180712d0 0x7f7a1814e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:18.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.946+0000 7f7a16800640 1 -- 172.21.15.131:0/1785573538 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7a1806d790 msgr2=0x7f7a1814dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:18.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.946+0000 7f7a16800640 1 --2- 172.21.15.131:0/1785573538 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7a1806d790 0x7f7a1814dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:18.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.946+0000 7f7a16800640 1 -- 172.21.15.131:0/1785573538 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a180a98b0 con 0x7f7a18145a10 2024-05-26T22:51:18.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.946+0000 7f7a16800640 1 --2- 172.21.15.131:0/1785573538 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7a18145a10 0x7f7a1815a590 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f7a10009eb0 tx=0x7f7a10004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:18.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.947+0000 7f7a0e000640 1 -- 172.21.15.131:0/1785573538 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a10040070 con 0x7f7a18145a10 2024-05-26T22:51:18.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.947+0000 7f7a0e000640 1 -- 172.21.15.131:0/1785573538 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a10031bc0 con 0x7f7a18145a10 2024-05-26T22:51:18.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.947+0000 7f7a0e000640 1 -- 172.21.15.131:0/1785573538 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a10009450 con 0x7f7a18145a10 2024-05-26T22:51:18.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.947+0000 7f7a17200640 1 -- 172.21.15.131:0/1785573538 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a1814b5a0 con 0x7f7a18145a10 2024-05-26T22:51:18.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.947+0000 7f7a17200640 1 -- 172.21.15.131:0/1785573538 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a1814bb60 con 0x7f7a18145a10 2024-05-26T22:51:18.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.948+0000 7f7a17200640 1 -- 172.21.15.131:0/1785573538 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a180694e0 con 0x7f7a18145a10 2024-05-26T22:51:18.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.949+0000 7f7a0e000640 1 -- 172.21.15.131:0/1785573538 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7a10039040 con 0x7f7a18145a10 2024-05-26T22:51:18.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.950+0000 7f7a0e000640 1 --2- 172.21.15.131:0/1785573538 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f79f406be30 0x7f79f406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:18.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.950+0000 7f7a0e000640 1 -- 172.21.15.131:0/1785573538 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5853+0+0 (secure 0 0 0) 0x7f7a100ac870 con 0x7f7a18145a10 2024-05-26T22:51:18.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.950+0000 7f7a15e00640 1 --2- 172.21.15.131:0/1785573538 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f79f406be30 0x7f79f406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:18.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.951+0000 7f7a15e00640 1 --2- 172.21.15.131:0/1785573538 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f79f406be30 0x7f79f406e2f0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f7a180af570 tx=0x7f7a08009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:18.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:18.954+0000 7f7a0e000640 1 -- 172.21.15.131:0/1785573538 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f7a1003ae60 con 0x7f7a18145a10 2024-05-26T22:51:19.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:19.164+0000 7f7a17200640 1 -- 172.21.15.131:0/1785573538 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f7a1806dc10 con 0x7f7a18145a10 2024-05-26T22:51:19.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:19.165+0000 7f7a0e000640 1 -- 172.21.15.131:0/1785573538 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v47) ==== 80+0+124 (secure 0 0 0) 0x7f7a1007b0f0 con 0x7f7a18145a10 2024-05-26T22:51:19.165 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.15.187:6835/3429891101 2024-05-27T22:50:59.823337+0000 2024-05-26T22:51:19.165 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.15.187:6833/3429891101 2024-05-27T22:50:59.823337+0000 2024-05-26T22:51:19.165 INFO:teuthology.orchestra.run.smithi131.stderr:listed 2 entries 2024-05-26T22:51:19.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:19.167+0000 7f7a17200640 1 -- 172.21.15.131:0/1785573538 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f79f406be30 msgr2=0x7f79f406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:19.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:19.167+0000 7f7a17200640 1 --2- 172.21.15.131:0/1785573538 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f79f406be30 0x7f79f406e2f0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f7a180af570 tx=0x7f7a08009040 comp rx=0 tx=0).stop 2024-05-26T22:51:19.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:19.167+0000 7f7a17200640 1 -- 172.21.15.131:0/1785573538 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7a18145a10 msgr2=0x7f7a1815a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:19.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:19.167+0000 7f7a17200640 1 --2- 172.21.15.131:0/1785573538 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7a18145a10 0x7f7a1815a590 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f7a10009eb0 tx=0x7f7a10004a40 comp rx=0 tx=0).stop 2024-05-26T22:51:19.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:19.168+0000 7f7a17200640 1 -- 172.21.15.131:0/1785573538 shutdown_connections 2024-05-26T22:51:19.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:19.168+0000 7f7a17200640 1 --2- 172.21.15.131:0/1785573538 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7a180712d0 0x7f7a1814e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:19.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:19.168+0000 7f7a17200640 1 --2- 172.21.15.131:0/1785573538 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f79f406be30 0x7f79f406e2f0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:19.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:19.168+0000 7f7a17200640 1 --2- 172.21.15.131:0/1785573538 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7a18145a10 0x7f7a1815a590 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:19.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:19.168+0000 7f7a17200640 1 --2- 172.21.15.131:0/1785573538 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7a1806d790 0x7f7a1814dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:19.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:19.168+0000 7f7a17200640 1 -- 172.21.15.131:0/1785573538 >> 172.21.15.131:0/1785573538 conn(0x7f7a180b5660 msgr2=0x7f7a180b0e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:19.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:19.168+0000 7f7a17200640 1 -- 172.21.15.131:0/1785573538 shutdown_connections 2024-05-26T22:51:19.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:19.168+0000 7f7a17200640 1 -- 172.21.15.131:0/1785573538 wait complete. 2024-05-26T22:51:19.187 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-26T22:51:19.187 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:51:19.187 DEBUG:teuthology.orchestra.run.smithi131:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-26T22:51:19.303 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi131.front.sepia.ceph.com 2024-05-26T22:51:19.303 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:51:19.303 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-26T22:51:19.364 INFO:tasks.cephfs.mount:Terminating background process 2024-05-26T22:51:19.364 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-05-26T22:51:19.364 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-26T22:51:19.405 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-26T22:51:19.405 INFO:tasks.cephfs.mount:Terminating background process 2024-05-26T22:51:19.405 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-26T22:51:19.406 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-26T22:51:19.446 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-26T22:51:19.446 INFO:tasks.cephfs.mount:Terminating background process 2024-05-26T22:51:19.446 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-05-26T22:51:19.446 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-26T22:51:19.494 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-26T22:51:19.495 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-26T22:51:19.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:19.692+0000 7f5ecac00640 1 Processor -- start 2024-05-26T22:51:19.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:19.692+0000 7f5ecac00640 1 -- start start 2024-05-26T22:51:19.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:19.693+0000 7f5ecac00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f5ec406e8c0 0x7f5ec415d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:19.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:19.693+0000 7f5ecac00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ec4160620 con 0x7f5ec40cc7f0 2024-05-26T22:51:19.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:19.693+0000 7f5ecac00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ec4160900 con 0x7f5ec4156750 2024-05-26T22:51:19.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:19.694+0000 7f5ecac00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f5ec415cbd0 con 0x7f5ec406e8c0 2024-05-26T22:51:19.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:19.694+0000 7f5ec8e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f5ec406e8c0 0x7f5ec415d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:19.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:19.694+0000 7f5ec8e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f5ec406e8c0 0x7f5ec415d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44926/0 (socket says 172.21.15.131:44926) 2024-05-26T22:51:19.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:19.694+0000 7f5eca200640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f5ec40cc7f0 0x7f5ec40cdc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:51324/0 (socket says 172.21.15.131:51324) 2024-05-26T22:51:19.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:19.694+0000 7f5ec8e00640 1 -- 172.21.15.131:0/772945566 learned_addr learned my addr 172.21.15.131:0/772945566 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:19.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:19.694+0000 7f5ec8e00640 1 -- 172.21.15.131:0/772945566 >> v1:172.21.15.187:6789/0 conn(0x7f5ec4156750 legacy=0x7f5ec415fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:19.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:19.694+0000 7f5ec8e00640 1 -- 172.21.15.131:0/772945566 >> v1:172.21.15.131:6789/0 conn(0x7f5ec40cc7f0 legacy=0x7f5ec40cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:19.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:19.694+0000 7f5ec8e00640 1 -- 172.21.15.131:0/772945566 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ec4160be0 con 0x7f5ec406e8c0 2024-05-26T22:51:19.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:19.694+0000 7f5ec3e00640 1 -- 172.21.15.131:0/772945566 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3040478496 0 0) 0x7f5ec4160620 con 0x7f5ec40cc7f0 2024-05-26T22:51:19.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:19.695+0000 7f5ec8e00640 1 --2- 172.21.15.131:0/772945566 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ec406e8c0 0x7f5ec415d060 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f5ec415cd50 tx=0x7f5eb002a2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2b46563b80ae387 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:19.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:19.696+0000 7f5ec3e00640 1 -- 172.21.15.131:0/772945566 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb0037070 con 0x7f5ec406e8c0 2024-05-26T22:51:19.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:19.696+0000 7f5ec3e00640 1 -- 172.21.15.131:0/772945566 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5eb0032640 con 0x7f5ec406e8c0 2024-05-26T22:51:19.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:19.696+0000 7f5ec3e00640 1 -- 172.21.15.131:0/772945566 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb0032a10 con 0x7f5ec406e8c0 2024-05-26T22:51:19.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:19.696+0000 7f5ecac00640 1 -- 172.21.15.131:0/772945566 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ec406e8c0 msgr2=0x7f5ec415d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:19.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:19.696+0000 7f5ecac00640 1 --2- 172.21.15.131:0/772945566 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ec406e8c0 0x7f5ec415d060 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f5ec415cd50 tx=0x7f5eb002a2e0 comp rx=0 tx=0).stop 2024-05-26T22:51:19.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:19.697+0000 7f5ecac00640 1 -- 172.21.15.131:0/772945566 shutdown_connections 2024-05-26T22:51:19.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:19.697+0000 7f5ecac00640 1 --2- 172.21.15.131:0/772945566 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ec406e8c0 0x7f5ec415d060 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:19.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:19.697+0000 7f5ecac00640 1 -- 172.21.15.131:0/772945566 >> 172.21.15.131:0/772945566 conn(0x7f5ec40cff40 msgr2=0x7f5ec40d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:19.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:19.697+0000 7f5ecac00640 1 -- 172.21.15.131:0/772945566 shutdown_connections 2024-05-26T22:51:19.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:19.697+0000 7f5ecac00640 1 -- 172.21.15.131:0/772945566 wait complete. 2024-05-26T22:51:19.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:19.699+0000 7f5ecac00640 1 Processor -- start 2024-05-26T22:51:19.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:19.699+0000 7f5ecac00640 1 -- start start 2024-05-26T22:51:19.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:19.700+0000 7f5ecac00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5ec406e8c0 0x7f5ec415db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:19.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:19.700+0000 7f5ecac00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ec40cc7f0 0x7f5ec41f1da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:19.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:19.700+0000 7f5ecac00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5ec4156750 0x7f5ec41432e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:19.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:19.700+0000 7f5ecac00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec415d5a0 con 0x7f5ec40cc7f0 2024-05-26T22:51:19.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:19.700+0000 7f5ecac00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec4156b30 con 0x7f5ec406e8c0 2024-05-26T22:51:19.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:19.700+0000 7f5ecac00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f5ec415d720 con 0x7f5ec4156750 2024-05-26T22:51:19.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:19.700+0000 7f5ec8e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ec40cc7f0 0x7f5ec41f1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:19.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:19.700+0000 7f5ec8e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ec40cc7f0 0x7f5ec41f1da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44934/0 (socket says 172.21.15.131:44934) 2024-05-26T22:51:19.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:19.701+0000 7f5ec8e00640 1 -- 172.21.15.131:0/3870347587 learned_addr learned my addr 172.21.15.131:0/3870347587 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:19.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:19.701+0000 7f5ec8e00640 1 -- 172.21.15.131:0/3870347587 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5ec4156750 msgr2=0x7f5ec41432e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:19.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:19.701+0000 7f5ec8e00640 1 --2- 172.21.15.131:0/3870347587 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5ec4156750 0x7f5ec41432e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:19.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:19.701+0000 7f5ec8e00640 1 -- 172.21.15.131:0/3870347587 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5ec406e8c0 msgr2=0x7f5ec415db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:19.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:19.701+0000 7f5ec8e00640 1 --2- 172.21.15.131:0/3870347587 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5ec406e8c0 0x7f5ec415db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:19.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:19.701+0000 7f5ec8e00640 1 -- 172.21.15.131:0/3870347587 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ec41f2820 con 0x7f5ec40cc7f0 2024-05-26T22:51:19.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:19.701+0000 7f5ec8e00640 1 --2- 172.21.15.131:0/3870347587 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ec40cc7f0 0x7f5ec41f1da0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f5eb0000c00 tx=0x7f5eb00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:19.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:19.702+0000 7f5ec1600640 1 -- 172.21.15.131:0/3870347587 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb0041020 con 0x7f5ec40cc7f0 2024-05-26T22:51:19.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:19.702+0000 7f5ec1600640 1 -- 172.21.15.131:0/3870347587 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5eb0030970 con 0x7f5ec40cc7f0 2024-05-26T22:51:19.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:19.702+0000 7f5ec1600640 1 -- 172.21.15.131:0/3870347587 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb0037040 con 0x7f5ec40cc7f0 2024-05-26T22:51:19.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:19.702+0000 7f5ecac00640 1 -- 172.21.15.131:0/3870347587 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ec41f2b00 con 0x7f5ec40cc7f0 2024-05-26T22:51:19.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:19.702+0000 7f5ecac00640 1 -- 172.21.15.131:0/3870347587 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ec40aff00 con 0x7f5ec40cc7f0 2024-05-26T22:51:19.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:19.703+0000 7f5ecac00640 1 -- 172.21.15.131:0/3870347587 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ec40694e0 con 0x7f5ec40cc7f0 2024-05-26T22:51:19.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:19.704+0000 7f5ec1600640 1 -- 172.21.15.131:0/3870347587 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5eb0044020 con 0x7f5ec40cc7f0 2024-05-26T22:51:19.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:19.705+0000 7f5ec1600640 1 --2- 172.21.15.131:0/3870347587 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f5ea406be30 0x7f5ea406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:19.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:19.705+0000 7f5ec1600640 1 -- 172.21.15.131:0/3870347587 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5853+0+0 (secure 0 0 0) 0x7f5eb00ab940 con 0x7f5ec40cc7f0 2024-05-26T22:51:19.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:19.705+0000 7f5ec9800640 1 --2- 172.21.15.131:0/3870347587 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f5ea406be30 0x7f5ea406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:19.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:19.706+0000 7f5ec9800640 1 --2- 172.21.15.131:0/3870347587 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f5ea406be30 0x7f5ea406e2f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f5eb8000f60 tx=0x7f5eb80079b0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:19.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:19.710+0000 7f5ec1600640 1 -- 172.21.15.131:0/3870347587 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f5eb004b030 con 0x7f5ec40cc7f0 2024-05-26T22:51:19.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:19.945+0000 7f5ecac00640 1 -- 172.21.15.131:0/3870347587 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5ec415a3f0 con 0x7f5ec40cc7f0 2024-05-26T22:51:19.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:19.950+0000 7f5ec1600640 1 -- 172.21.15.131:0/3870347587 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+5230 (secure 0 0 0) 0x7f5eb007a250 con 0x7f5ec40cc7f0 2024-05-26T22:51:19.949 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:51:19.949 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":40,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4727,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/3668228040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6833","nonce":3668228040},{"type":"v1","addr":"172.21.15.187:6835","nonce":3668228040}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-26T22:50:27.001665+0000","modified":"2024-05-26T22:51:03.848463+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":45,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5637,"mds_1":4661,"mds_2":4670},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4661":{"gid":4661,"name":"a","rank":1,"incarnation":36,"state":"up:active","state_seq":14,"addr":"172.21.15.131:6837/265040431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6835","nonce":265040431},{"type":"v1","addr":"172.21.15.131:6837","nonce":265040431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4670":{"gid":4670,"name":"b","rank":2,"incarnation":32,"state":"up:active","state_seq":8,"addr":"172.21.15.187:6834/1027783169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1027783169},{"type":"v1","addr":"172.21.15.187:6834","nonce":1027783169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5637":{"gid":5637,"name":"c","rank":0,"incarnation":25,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6836/650770835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":650770835},{"type":"v1","addr":"172.21.15.131:6836","nonce":650770835}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5637,"qdb_cluster":[4661,4670,5637]},"id":3}]} 2024-05-26T22:51:19.950 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 40 2024-05-26T22:51:19.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:19.951+0000 7f5ecac00640 1 -- 172.21.15.131:0/3870347587 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f5ea406be30 msgr2=0x7f5ea406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:19.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:19.951+0000 7f5ecac00640 1 --2- 172.21.15.131:0/3870347587 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f5ea406be30 0x7f5ea406e2f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f5eb8000f60 tx=0x7f5eb80079b0 comp rx=0 tx=0).stop 2024-05-26T22:51:19.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:19.951+0000 7f5ecac00640 1 -- 172.21.15.131:0/3870347587 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ec40cc7f0 msgr2=0x7f5ec41f1da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:19.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:19.951+0000 7f5ecac00640 1 --2- 172.21.15.131:0/3870347587 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ec40cc7f0 0x7f5ec41f1da0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f5eb0000c00 tx=0x7f5eb00046c0 comp rx=0 tx=0).stop 2024-05-26T22:51:19.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:19.951+0000 7f5ecac00640 1 -- 172.21.15.131:0/3870347587 shutdown_connections 2024-05-26T22:51:19.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:19.951+0000 7f5ecac00640 1 --2- 172.21.15.131:0/3870347587 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5ec4156750 0x7f5ec41432e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:19.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:19.951+0000 7f5ecac00640 1 --2- 172.21.15.131:0/3870347587 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f5ea406be30 0x7f5ea406e2f0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:19.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:19.951+0000 7f5ecac00640 1 --2- 172.21.15.131:0/3870347587 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ec40cc7f0 0x7f5ec41f1da0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:19.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:19.951+0000 7f5ecac00640 1 --2- 172.21.15.131:0/3870347587 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5ec406e8c0 0x7f5ec415db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:19.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:19.951+0000 7f5ecac00640 1 -- 172.21.15.131:0/3870347587 >> 172.21.15.131:0/3870347587 conn(0x7f5ec40cff40 msgr2=0x7f5ec40cf0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:19.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:19.952+0000 7f5ecac00640 1 -- 172.21.15.131:0/3870347587 shutdown_connections 2024-05-26T22:51:19.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:19.952+0000 7f5ecac00640 1 -- 172.21.15.131:0/3870347587 wait complete. 2024-05-26T22:51:19.971 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-26T22:51:20.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.169+0000 7f9fdc800640 1 Processor -- start 2024-05-26T22:51:20.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.169+0000 7f9fdc800640 1 -- start start 2024-05-26T22:51:20.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.171+0000 7f9fdc800640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f9fd4155840 0x7f9fd4071930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:20.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.171+0000 7f9fdc800640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9fd40722a0 con 0x7f9fd415c3d0 2024-05-26T22:51:20.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.171+0000 7f9fdc800640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9fd4072580 con 0x7f9fd406f660 2024-05-26T22:51:20.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.171+0000 7f9fdc800640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f9fd4072860 con 0x7f9fd4155840 2024-05-26T22:51:20.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.171+0000 7f9fd9400640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f9fd415c3d0 0x7f9fd406ef50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:51336/0 (socket says 172.21.15.131:51336) 2024-05-26T22:51:20.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.171+0000 7f9fd9400640 1 -- 172.21.15.131:0/310781362 learned_addr learned my addr 172.21.15.131:0/310781362 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:51:20.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.171+0000 7f9fd9e00640 1 --2- 172.21.15.131:0/310781362 >> v2:172.21.15.131:3300/0 conn(0x7f9fd4155840 0x7f9fd4071930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:20.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.172+0000 7f9fd9e00640 1 -- 172.21.15.131:0/310781362 >> v1:172.21.15.187:6789/0 conn(0x7f9fd406f660 legacy=0x7f9fd40ade40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:20.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.172+0000 7f9fd9e00640 1 -- 172.21.15.131:0/310781362 >> v1:172.21.15.131:6789/0 conn(0x7f9fd415c3d0 legacy=0x7f9fd406ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:20.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.172+0000 7f9fd9e00640 1 -- 172.21.15.131:0/310781362 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fd4072b40 con 0x7f9fd4155840 2024-05-26T22:51:20.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.173+0000 7f9fd9e00640 1 --2- 172.21.15.131:0/310781362 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9fd4155840 0x7f9fd4071930 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f9fc80097a0 tx=0x7f9fc802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f40daf268c99211a server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:20.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.173+0000 7f9fd8a00640 1 -- 172.21.15.131:0/310781362 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fc8037070 con 0x7f9fd4155840 2024-05-26T22:51:20.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.173+0000 7f9fd8a00640 1 -- 172.21.15.131:0/310781362 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fc8032440 con 0x7f9fd4155840 2024-05-26T22:51:20.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.173+0000 7f9fd8a00640 1 -- 172.21.15.131:0/310781362 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fc8032810 con 0x7f9fd4155840 2024-05-26T22:51:20.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.174+0000 7f9fdc800640 1 -- 172.21.15.131:0/310781362 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9fd4155840 msgr2=0x7f9fd4071930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:20.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.174+0000 7f9fdc800640 1 --2- 172.21.15.131:0/310781362 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9fd4155840 0x7f9fd4071930 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f9fc80097a0 tx=0x7f9fc802a1e0 comp rx=0 tx=0).stop 2024-05-26T22:51:20.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.174+0000 7f9fdc800640 1 -- 172.21.15.131:0/310781362 shutdown_connections 2024-05-26T22:51:20.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.174+0000 7f9fdc800640 1 --2- 172.21.15.131:0/310781362 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9fd4155840 0x7f9fd4071930 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:20.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.174+0000 7f9fdc800640 1 -- 172.21.15.131:0/310781362 >> 172.21.15.131:0/310781362 conn(0x7f9fd40b1e90 msgr2=0x7f9fd40b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:20.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.174+0000 7f9fdc800640 1 -- 172.21.15.131:0/310781362 shutdown_connections 2024-05-26T22:51:20.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.174+0000 7f9fdc800640 1 -- 172.21.15.131:0/310781362 wait complete. 2024-05-26T22:51:20.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.175+0000 7f9fdc800640 1 Processor -- start 2024-05-26T22:51:20.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.176+0000 7f9fdc800640 1 -- start start 2024-05-26T22:51:20.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.177+0000 7f9fdc800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9fd406f660 0x7f9fd4155c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:20.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.177+0000 7f9fdc800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9fd4155840 0x7f9fd41e4330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:20.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.177+0000 7f9fdc800640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9fd415c3d0 0x7f9fd41572e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:20.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.177+0000 7f9fdc800640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f9fd4071ff0 con 0x7f9fd406f660 2024-05-26T22:51:20.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.177+0000 7f9fdc800640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9fd4071e70 con 0x7f9fd415c3d0 2024-05-26T22:51:20.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.177+0000 7f9fdc800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9fd40ae550 con 0x7f9fd4155840 2024-05-26T22:51:20.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.177+0000 7f9fd9e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9fd406f660 0x7f9fd4155c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:20.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.177+0000 7f9fd9e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9fd406f660 0x7f9fd4155c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44954/0 (socket says 172.21.15.131:44954) 2024-05-26T22:51:20.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.177+0000 7f9fd9e00640 1 -- 172.21.15.131:0/3554108235 learned_addr learned my addr 172.21.15.131:0/3554108235 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:20.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.177+0000 7f9fd9e00640 1 -- 172.21.15.131:0/3554108235 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9fd4155840 msgr2=0x7f9fd41e4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:20.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.177+0000 7f9fd9e00640 1 --2- 172.21.15.131:0/3554108235 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9fd4155840 0x7f9fd41e4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:20.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.177+0000 7f9fd9e00640 1 -- 172.21.15.131:0/3554108235 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9fd415c3d0 msgr2=0x7f9fd41572e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:20.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.177+0000 7f9fd9e00640 1 --2- 172.21.15.131:0/3554108235 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9fd415c3d0 0x7f9fd41572e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:20.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.177+0000 7f9fd9e00640 1 -- 172.21.15.131:0/3554108235 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fd41e4870 con 0x7f9fd406f660 2024-05-26T22:51:20.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.178+0000 7f9fd9e00640 1 --2- 172.21.15.131:0/3554108235 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9fd406f660 0x7f9fd4155c20 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f9fc8000c00 tx=0x7f9fc80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:20.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.178+0000 7f9fce000640 1 -- 172.21.15.131:0/3554108235 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fc8041070 con 0x7f9fd406f660 2024-05-26T22:51:20.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.178+0000 7f9fce000640 1 -- 172.21.15.131:0/3554108235 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fc803c930 con 0x7f9fd406f660 2024-05-26T22:51:20.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.178+0000 7f9fce000640 1 -- 172.21.15.131:0/3554108235 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fc8037040 con 0x7f9fd406f660 2024-05-26T22:51:20.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.178+0000 7f9fdc800640 1 -- 172.21.15.131:0/3554108235 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fd4156240 con 0x7f9fd406f660 2024-05-26T22:51:20.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.178+0000 7f9fdc800640 1 -- 172.21.15.131:0/3554108235 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fd4156800 con 0x7f9fd406f660 2024-05-26T22:51:20.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.179+0000 7f9fdc800640 1 -- 172.21.15.131:0/3554108235 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fd40694e0 con 0x7f9fd406f660 2024-05-26T22:51:20.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.180+0000 7f9fce000640 1 -- 172.21.15.131:0/3554108235 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9fc8044070 con 0x7f9fd406f660 2024-05-26T22:51:20.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.181+0000 7f9fce000640 1 --2- 172.21.15.131:0/3554108235 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9fb806be60 0x7f9fb806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:20.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.181+0000 7f9fce000640 1 -- 172.21.15.131:0/3554108235 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5853+0+0 (secure 0 0 0) 0x7f9fc80abc60 con 0x7f9fd406f660 2024-05-26T22:51:20.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.181+0000 7f9fd9400640 1 --2- 172.21.15.131:0/3554108235 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9fb806be60 0x7f9fb806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:20.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.182+0000 7f9fd9400640 1 --2- 172.21.15.131:0/3554108235 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9fb806be60 0x7f9fb806e320 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f9fc4000f60 tx=0x7f9fc4007a80 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:20.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.186+0000 7f9fce000640 1 -- 172.21.15.131:0/3554108235 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f9fc807a600 con 0x7f9fd406f660 2024-05-26T22:51:20.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.387+0000 7f9fdc800640 1 -- 172.21.15.131:0/3554108235 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9fd406d730 con 0x7f9fd406f660 2024-05-26T22:51:20.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.390+0000 7f9fce000640 1 -- 172.21.15.131:0/3554108235 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) ==== 74+0+16860 (secure 0 0 0) 0x7f9fc8079f00 con 0x7f9fd406f660 2024-05-26T22:51:20.391 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:51:20.391 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":47,"fsid":"46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f","created":"2024-05-26T22:47:57.288905+0000","modified":"2024-05-26T22:51:05.195752+0000","last_up_change":"2024-05-26T22:48:09.602207+0000","last_in_change":"2024-05-26T22:48:00.687341+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-26T22:48:10.463339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-05-26T22:50:24.793406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-05-26T22:50:26.041636+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"814eaf62-a18d-494b-8e27-fd8fb88adea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6801","nonce":2350940051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6803","nonce":2350940051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6807","nonce":2350940051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6805","nonce":2350940051}]},"public_addr":"172.21.15.131:6801/2350940051","cluster_addr":"172.21.15.131:6803/2350940051","heartbeat_back_addr":"172.21.15.131:6807/2350940051","heartbeat_front_addr":"172.21.15.131:6805/2350940051","state":["exists","up"]},{"osd":1,"uuid":"42e66287-f6c6-4e26-af3c-6dfaa3536d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6825","nonce":1525528247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6827","nonce":1525528247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6831","nonce":1525528247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6829","nonce":1525528247}]},"public_addr":"172.21.15.131:6825/1525528247","cluster_addr":"172.21.15.131:6827/1525528247","heartbeat_back_addr":"172.21.15.131:6831/1525528247","heartbeat_front_addr":"172.21.15.131:6829/1525528247","state":["exists","up"]},{"osd":2,"uuid":"0da34673-04a2-4fd0-b57a-0ff2b679b043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6817","nonce":3627842210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6819","nonce":3627842210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6823","nonce":3627842210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6821","nonce":3627842210}]},"public_addr":"172.21.15.131:6817/3627842210","cluster_addr":"172.21.15.131:6819/3627842210","heartbeat_back_addr":"172.21.15.131:6823/3627842210","heartbeat_front_addr":"172.21.15.131:6821/3627842210","state":["exists","up"]},{"osd":3,"uuid":"cf117a27-d6b2-43d3-ae52-b47988bf8eed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6809","nonce":350790591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6811","nonce":350790591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6815","nonce":350790591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6813","nonce":350790591}]},"public_addr":"172.21.15.131:6809/350790591","cluster_addr":"172.21.15.131:6811/350790591","heartbeat_back_addr":"172.21.15.131:6815/350790591","heartbeat_front_addr":"172.21.15.131:6813/350790591","state":["exists","up"]},{"osd":4,"uuid":"a092ccfc-be89-4a84-88d5-0f781fa715eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6816","nonce":244560995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6819","nonce":244560995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6823","nonce":244560995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6821","nonce":244560995}]},"public_addr":"172.21.15.187:6816/244560995","cluster_addr":"172.21.15.187:6819/244560995","heartbeat_back_addr":"172.21.15.187:6823/244560995","heartbeat_front_addr":"172.21.15.187:6821/244560995","state":["exists","up"]},{"osd":5,"uuid":"7513ba69-dd00-4a35-b190-9e8cc3b2bdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6801","nonce":324619398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6803","nonce":324619398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6812","nonce":324619398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6807","nonce":324619398}]},"public_addr":"172.21.15.187:6801/324619398","cluster_addr":"172.21.15.187:6803/324619398","heartbeat_back_addr":"172.21.15.187:6812/324619398","heartbeat_front_addr":"172.21.15.187:6807/324619398","state":["exists","up"]},{"osd":6,"uuid":"50081810-c093-4d2f-a4d7-b7c695c5400d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6806","nonce":1391347786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6809","nonce":1391347786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6818","nonce":1391347786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6813","nonce":1391347786}]},"public_addr":"172.21.15.187:6806/1391347786","cluster_addr":"172.21.15.187:6809/1391347786","heartbeat_back_addr":"172.21.15.187:6818/1391347786","heartbeat_front_addr":"172.21.15.187:6813/1391347786","state":["exists","up"]},{"osd":7,"uuid":"775dae05-54f5-4ad4-bbce-92863c6ed2ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6825","nonce":1220065622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6827","nonce":1220065622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6831","nonce":1220065622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6829","nonce":1220065622}]},"public_addr":"172.21.15.187:6825/1220065622","cluster_addr":"172.21.15.187:6827/1220065622","heartbeat_back_addr":"172.21.15.187:6831/1220065622","heartbeat_front_addr":"172.21.15.187:6829/1220065622","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:05.983520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.299516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.067323+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.307748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.018742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.258775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.010912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.295355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.36","mappings":[{"from":0,"to":5}]},{"pgid":"7.38","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.187:6835/3429891101":"2024-05-27T22:50:59.823337+0000","172.21.15.187:6833/3429891101":"2024-05-27T22:50:59.823337+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-26T22:51:20.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.393+0000 7f9fdc800640 1 -- 172.21.15.131:0/3554108235 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9fb806be60 msgr2=0x7f9fb806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:20.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.393+0000 7f9fdc800640 1 --2- 172.21.15.131:0/3554108235 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9fb806be60 0x7f9fb806e320 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f9fc4000f60 tx=0x7f9fc4007a80 comp rx=0 tx=0).stop 2024-05-26T22:51:20.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.393+0000 7f9fdc800640 1 -- 172.21.15.131:0/3554108235 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9fd406f660 msgr2=0x7f9fd4155c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:20.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.393+0000 7f9fdc800640 1 --2- 172.21.15.131:0/3554108235 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9fd406f660 0x7f9fd4155c20 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f9fc8000c00 tx=0x7f9fc80046c0 comp rx=0 tx=0).stop 2024-05-26T22:51:20.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.393+0000 7f9fdc800640 1 -- 172.21.15.131:0/3554108235 shutdown_connections 2024-05-26T22:51:20.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.393+0000 7f9fdc800640 1 --2- 172.21.15.131:0/3554108235 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9fd4155840 0x7f9fd41e4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:20.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.393+0000 7f9fdc800640 1 --2- 172.21.15.131:0/3554108235 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9fb806be60 0x7f9fb806e320 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:20.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.393+0000 7f9fdc800640 1 --2- 172.21.15.131:0/3554108235 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9fd415c3d0 0x7f9fd41572e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:20.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.393+0000 7f9fdc800640 1 --2- 172.21.15.131:0/3554108235 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9fd406f660 0x7f9fd4155c20 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:20.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.393+0000 7f9fdc800640 1 -- 172.21.15.131:0/3554108235 >> 172.21.15.131:0/3554108235 conn(0x7f9fd40b1e90 msgr2=0x7f9fd415cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:20.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.393+0000 7f9fdc800640 1 -- 172.21.15.131:0/3554108235 shutdown_connections 2024-05-26T22:51:20.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.393+0000 7f9fdc800640 1 -- 172.21.15.131:0/3554108235 wait complete. 2024-05-26T22:51:20.412 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-26T22:48:10.463339+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-26T22:50:24.793406+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '43', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2024-05-26T22:50:26.041636+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-26T22:51:20.413 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-26T22:51:20.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.612+0000 7feb07e00640 1 Processor -- start 2024-05-26T22:51:20.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.612+0000 7feb07e00640 1 -- start start 2024-05-26T22:51:20.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.614+0000 7feb07e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb0806d960 0x7feb08071c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:20.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.614+0000 7feb07e00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb081552d0 con 0x7feb0815dc40 2024-05-26T22:51:20.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.614+0000 7feb07e00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb08072230 con 0x7feb0815bfa0 2024-05-26T22:51:20.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.614+0000 7feb07e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7feb080b3f30 con 0x7feb0806d960 2024-05-26T22:51:20.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.614+0000 7feb06a00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7feb0815dc40 0x7feb0806d250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:51346/0 (socket says 172.21.15.131:51346) 2024-05-26T22:51:20.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.614+0000 7feb06a00640 1 -- 172.21.15.131:0/1978230112 learned_addr learned my addr 172.21.15.131:0/1978230112 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:51:20.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.614+0000 7feb06000640 1 -- 172.21.15.131:0/1978230112 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1669723577 0 0) 0x7feb081552d0 con 0x7feb0815dc40 2024-05-26T22:51:20.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.614+0000 7feb06000640 1 -- 172.21.15.131:0/1978230112 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feaec003520 con 0x7feb0815dc40 2024-05-26T22:51:20.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.614+0000 7feb07400640 1 --2- 172.21.15.131:0/1978230112 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb0806d960 0x7feb08071c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:20.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.615+0000 7feb06000640 1 -- 172.21.15.131:0/1978230112 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2167679847 0 0) 0x7feaec003520 con 0x7feb0815dc40 2024-05-26T22:51:20.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.615+0000 7feb06000640 1 -- 172.21.15.131:0/1978230112 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb0806d960 msgr2=0x7feb08071c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:20.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.615+0000 7feb06000640 1 --2- 172.21.15.131:0/1978230112 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb0806d960 0x7feb08071c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:20.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.615+0000 7feb06000640 1 -- 172.21.15.131:0/1978230112 >> v1:172.21.15.187:6789/0 conn(0x7feb0815bfa0 legacy=0x7feb08071500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:20.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.615+0000 7feb06000640 1 -- 172.21.15.131:0/1978230112 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb08072510 con 0x7feb0815dc40 2024-05-26T22:51:20.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.615+0000 7feb06000640 1 -- 172.21.15.131:0/1978230112 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7feaf8001670 con 0x7feb0815dc40 2024-05-26T22:51:20.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.615+0000 7feb06000640 1 -- 172.21.15.131:0/1978230112 >> v1:172.21.15.131:6789/0 conn(0x7feb0815dc40 legacy=0x7feb0806d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:20.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.615+0000 7feb06000640 1 --2- 172.21.15.131:0/1978230112 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feaec0039e0 0x7feaec003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:20.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.615+0000 7feb06000640 1 --2- 172.21.15.131:0/1978230112 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb0806d960 0x7feaec004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:20.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.615+0000 7feb06000640 1 --2- 172.21.15.131:0/1978230112 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feaec004d40 0x7feaec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:20.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.615+0000 7feb06000640 1 -- 172.21.15.131:0/1978230112 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7feb080b3f30 con 0x7feaec0039e0 2024-05-26T22:51:20.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.615+0000 7feb06000640 1 -- 172.21.15.131:0/1978230112 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7feaec00bc70 con 0x7feaec004d40 2024-05-26T22:51:20.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.615+0000 7feb06000640 1 -- 172.21.15.131:0/1978230112 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7feaec00be70 con 0x7feb0806d960 2024-05-26T22:51:20.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.615+0000 7feb06a00640 1 --2- 172.21.15.131:0/1978230112 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feaec0039e0 0x7feaec003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:20.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.616+0000 7feb06a00640 1 -- 172.21.15.131:0/1978230112 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb0806d960 msgr2=0x7feaec004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:20.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.616+0000 7feb06a00640 1 --2- 172.21.15.131:0/1978230112 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb0806d960 0x7feaec004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:20.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.616+0000 7feb06a00640 1 -- 172.21.15.131:0/1978230112 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feaec004d40 msgr2=0x7feaec0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:20.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.616+0000 7feb06a00640 1 --2- 172.21.15.131:0/1978230112 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feaec004d40 0x7feaec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:20.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.616+0000 7feb06a00640 1 -- 172.21.15.131:0/1978230112 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaec003520 con 0x7feaec0039e0 2024-05-26T22:51:20.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.617+0000 7feb06a00640 1 --2- 172.21.15.131:0/1978230112 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feaec0039e0 0x7feaec003e00 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7feaf800c0e0 tx=0x7feaec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b812998eba1abde server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:20.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.617+0000 7feb06000640 1 -- 172.21.15.131:0/1978230112 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf8007c20 con 0x7feaec0039e0 2024-05-26T22:51:20.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.617+0000 7feb06000640 1 -- 172.21.15.131:0/1978230112 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feaf8005af0 con 0x7feaec0039e0 2024-05-26T22:51:20.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.617+0000 7feb06000640 1 -- 172.21.15.131:0/1978230112 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf8007070 con 0x7feaec0039e0 2024-05-26T22:51:20.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.617+0000 7feb07e00640 1 -- 172.21.15.131:0/1978230112 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feaec0039e0 msgr2=0x7feaec003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:20.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.617+0000 7feb07e00640 1 --2- 172.21.15.131:0/1978230112 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feaec0039e0 0x7feaec003e00 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7feaf800c0e0 tx=0x7feaec005830 comp rx=0 tx=0).stop 2024-05-26T22:51:20.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.617+0000 7feb07400640 1 -- 172.21.15.131:0/1978230112 reap_dead start 2024-05-26T22:51:20.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.618+0000 7feb07e00640 1 -- 172.21.15.131:0/1978230112 shutdown_connections 2024-05-26T22:51:20.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.618+0000 7feb07e00640 1 -- 172.21.15.131:0/1978230112 >> 172.21.15.131:0/1978230112 conn(0x7feb080b5660 msgr2=0x7feb080b5a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:20.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.618+0000 7feb07e00640 1 -- 172.21.15.131:0/1978230112 shutdown_connections 2024-05-26T22:51:20.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.618+0000 7feb07e00640 1 -- 172.21.15.131:0/1978230112 wait complete. 2024-05-26T22:51:20.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.619+0000 7feb07e00640 1 Processor -- start 2024-05-26T22:51:20.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.619+0000 7feb07e00640 1 -- start start 2024-05-26T22:51:20.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.620+0000 7feb07e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feaec0039e0 0x7feb080ccec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:20.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.620+0000 7feb07e00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feb08147840 0x7feb0813a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:20.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.620+0000 7feb07e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb0813fc60 0x7feb081596e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:20.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.620+0000 7feb07e00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7feb0815e1a0 con 0x7feb0813fc60 2024-05-26T22:51:20.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.620+0000 7feb07e00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7feb0815e020 con 0x7feb08147840 2024-05-26T22:51:20.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.620+0000 7feb07e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7feb08071c10 con 0x7feaec0039e0 2024-05-26T22:51:20.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.621+0000 7feb07400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb0813fc60 0x7feb081596e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:20.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.621+0000 7feb07400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb0813fc60 0x7feb081596e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44964/0 (socket says 172.21.15.131:44964) 2024-05-26T22:51:20.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.621+0000 7feb07400640 1 -- 172.21.15.131:0/1235237234 learned_addr learned my addr 172.21.15.131:0/1235237234 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:20.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.621+0000 7feb07400640 1 -- 172.21.15.131:0/1235237234 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feaec0039e0 msgr2=0x7feb080ccec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:20.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.621+0000 7feb06a00640 1 --2- 172.21.15.131:0/1235237234 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feaec0039e0 0x7feb080ccec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:20.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.621+0000 7feb07400640 1 --2- 172.21.15.131:0/1235237234 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feaec0039e0 0x7feb080ccec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:20.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.621+0000 7feb07400640 1 -- 172.21.15.131:0/1235237234 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feb08147840 msgr2=0x7feb0813a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:20.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.621+0000 7feafe000640 1 --2- 172.21.15.131:0/1235237234 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feb08147840 0x7feb0813a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:20.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.621+0000 7feb07400640 1 --2- 172.21.15.131:0/1235237234 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feb08147840 0x7feb0813a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:20.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.621+0000 7feb07400640 1 -- 172.21.15.131:0/1235237234 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb081408b0 con 0x7feb0813fc60 2024-05-26T22:51:20.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.621+0000 7feb07400640 1 --2- 172.21.15.131:0/1235237234 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb0813fc60 0x7feb081596e0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7feb0806e340 tx=0x7feb0000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:20.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.622+0000 7feaff400640 1 -- 172.21.15.131:0/1235237234 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb00007e40 con 0x7feb0813fc60 2024-05-26T22:51:20.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.622+0000 7feaff400640 1 -- 172.21.15.131:0/1235237234 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb00019070 con 0x7feb0813fc60 2024-05-26T22:51:20.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.622+0000 7feaff400640 1 -- 172.21.15.131:0/1235237234 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb00014660 con 0x7feb0813fc60 2024-05-26T22:51:20.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.622+0000 7feb07e00640 1 -- 172.21.15.131:0/1235237234 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb081405d0 con 0x7feb0813fc60 2024-05-26T22:51:20.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.622+0000 7feb07e00640 1 -- 172.21.15.131:0/1235237234 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb081693b0 con 0x7feb0813fc60 2024-05-26T22:51:20.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.623+0000 7feb07e00640 1 -- 172.21.15.131:0/1235237234 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb080694e0 con 0x7feb0813fc60 2024-05-26T22:51:20.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.624+0000 7feaff400640 1 -- 172.21.15.131:0/1235237234 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7feb00014800 con 0x7feb0813fc60 2024-05-26T22:51:20.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.625+0000 7feaff400640 1 --2- 172.21.15.131:0/1235237234 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7feae406be60 0x7feae406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:20.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.625+0000 7feaff400640 1 -- 172.21.15.131:0/1235237234 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5853+0+0 (secure 0 0 0) 0x7feb0008b2f0 con 0x7feb0813fc60 2024-05-26T22:51:20.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.625+0000 7feb06a00640 1 --2- 172.21.15.131:0/1235237234 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7feae406be60 0x7feae406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:20.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.626+0000 7feb06a00640 1 --2- 172.21.15.131:0/1235237234 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7feae406be60 0x7feae406e320 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7feaf8032000 tx=0x7feaf8009030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:20.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.629+0000 7feaff400640 1 -- 172.21.15.131:0/1235237234 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7feb081408b0 con 0x7feb0813fc60 2024-05-26T22:51:20.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.829+0000 7feb07e00640 1 -- 172.21.15.131:0/1235237234 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7feb08168670 con 0x7feb0813fc60 2024-05-26T22:51:20.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.830+0000 7feaff400640 1 -- 172.21.15.131:0/1235237234 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v47) ==== 93+0+10 (secure 0 0 0) 0x7feb00059c00 con 0x7feb0813fc60 2024-05-26T22:51:20.830 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2024-05-26T22:51:20.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.832+0000 7feb07e00640 1 -- 172.21.15.131:0/1235237234 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7feae406be60 msgr2=0x7feae406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:20.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.832+0000 7feb07e00640 1 --2- 172.21.15.131:0/1235237234 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7feae406be60 0x7feae406e320 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7feaf8032000 tx=0x7feaf8009030 comp rx=0 tx=0).stop 2024-05-26T22:51:20.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.833+0000 7feb07e00640 1 -- 172.21.15.131:0/1235237234 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb0813fc60 msgr2=0x7feb081596e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:20.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.833+0000 7feb07e00640 1 --2- 172.21.15.131:0/1235237234 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb0813fc60 0x7feb081596e0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7feb0806e340 tx=0x7feb0000e4b0 comp rx=0 tx=0).stop 2024-05-26T22:51:20.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.833+0000 7feb07e00640 1 -- 172.21.15.131:0/1235237234 shutdown_connections 2024-05-26T22:51:20.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.833+0000 7feb07e00640 1 --2- 172.21.15.131:0/1235237234 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7feae406be60 0x7feae406e320 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:20.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.833+0000 7feb07e00640 1 --2- 172.21.15.131:0/1235237234 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb0813fc60 0x7feb081596e0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:20.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.833+0000 7feb07e00640 1 --2- 172.21.15.131:0/1235237234 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feb08147840 0x7feb0813a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:20.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.833+0000 7feb07e00640 1 --2- 172.21.15.131:0/1235237234 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feaec0039e0 0x7feb080ccec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:20.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.833+0000 7feb07e00640 1 -- 172.21.15.131:0/1235237234 >> 172.21.15.131:0/1235237234 conn(0x7feb080b5660 msgr2=0x7feb0806d230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:20.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.833+0000 7feb07e00640 1 -- 172.21.15.131:0/1235237234 shutdown_connections 2024-05-26T22:51:20.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:20.833+0000 7feb07e00640 1 -- 172.21.15.131:0/1235237234 wait complete. 2024-05-26T22:51:20.851 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-26T22:51:21.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.054+0000 7fc4fa400640 1 Processor -- start 2024-05-26T22:51:21.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.054+0000 7fc4fa400640 1 -- start start 2024-05-26T22:51:21.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.055+0000 7fc4fa400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc4f4145a10 0x7fc4f4072180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:21.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.055+0000 7fc4fa400640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fc4f406d960 0x7fc4f40cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:21.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.055+0000 7fc4fa400640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4f4072d60 con 0x7fc4f40712d0 2024-05-26T22:51:21.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.055+0000 7fc4fa400640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fc4f415c760 con 0x7fc4f406d960 2024-05-26T22:51:21.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.055+0000 7fc4fa400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc4f4072a80 con 0x7fc4f4145a10 2024-05-26T22:51:21.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.056+0000 7fc4f3e00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fc4f40712d0 0x7fc4f406d250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:51350/0 (socket says 172.21.15.131:51350) 2024-05-26T22:51:21.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.056+0000 7fc4f3e00640 1 -- 172.21.15.131:0/2598111823 learned_addr learned my addr 172.21.15.131:0/2598111823 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:51:21.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.056+0000 7fc4f9000640 1 --2- 172.21.15.131:0/2598111823 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc4f4145a10 0x7fc4f4072180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:21.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.056+0000 7fc4f9a00640 1 --2- 172.21.15.131:0/2598111823 >> v2:172.21.15.187:3300/0 conn(0x7fc4f406d960 0x7fc4f40cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:21.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.056+0000 7fc4f3400640 1 -- 172.21.15.131:0/2598111823 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2125987702 0 0) 0x7fc4f4072d60 con 0x7fc4f40712d0 2024-05-26T22:51:21.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.056+0000 7fc4f3400640 1 -- 172.21.15.131:0/2598111823 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc4dc003520 con 0x7fc4f40712d0 2024-05-26T22:51:21.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.057+0000 7fc4f3400640 1 -- 172.21.15.131:0/2598111823 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3074028455 0 0) 0x7fc4dc003520 con 0x7fc4f40712d0 2024-05-26T22:51:21.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.057+0000 7fc4f3400640 1 -- 172.21.15.131:0/2598111823 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc4f4145a10 msgr2=0x7fc4f4072180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:21.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.057+0000 7fc4f3400640 1 --2- 172.21.15.131:0/2598111823 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc4f4145a10 0x7fc4f4072180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:21.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.057+0000 7fc4f3400640 1 -- 172.21.15.131:0/2598111823 >> v2:172.21.15.187:3300/0 conn(0x7fc4f406d960 msgr2=0x7fc4f40cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:21.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.057+0000 7fc4f3400640 1 --2- 172.21.15.131:0/2598111823 >> v2:172.21.15.187:3300/0 conn(0x7fc4f406d960 0x7fc4f40cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:21.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.057+0000 7fc4f3400640 1 -- 172.21.15.131:0/2598111823 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4f40ceab0 con 0x7fc4f40712d0 2024-05-26T22:51:21.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.057+0000 7fc4f3400640 1 -- 172.21.15.131:0/2598111823 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7fc4e0001670 con 0x7fc4f40712d0 2024-05-26T22:51:21.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.057+0000 7fc4f3400640 1 -- 172.21.15.131:0/2598111823 >> v1:172.21.15.131:6789/0 conn(0x7fc4f40712d0 legacy=0x7fc4f406d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:21.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.057+0000 7fc4f3400640 1 --2- 172.21.15.131:0/2598111823 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc4f4145a10 0x7fc4dc0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:21.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.057+0000 7fc4f3400640 1 --2- 172.21.15.131:0/2598111823 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc4dc003f80 0x7fc4dc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:21.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.057+0000 7fc4f3400640 1 --2- 172.21.15.131:0/2598111823 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc4dc004d40 0x7fc4dc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:21.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.057+0000 7fc4f3400640 1 -- 172.21.15.131:0/2598111823 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f415c760 con 0x7fc4dc004d40 2024-05-26T22:51:21.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.057+0000 7fc4f3400640 1 -- 172.21.15.131:0/2598111823 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f4072a80 con 0x7fc4dc003f80 2024-05-26T22:51:21.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.057+0000 7fc4f3400640 1 -- 172.21.15.131:0/2598111823 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc4dc00b9c0 con 0x7fc4f4145a10 2024-05-26T22:51:21.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.057+0000 7fc4f9a00640 1 --2- 172.21.15.131:0/2598111823 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc4dc004d40 0x7fc4dc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:21.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.057+0000 7fc4f9000640 1 --2- 172.21.15.131:0/2598111823 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc4f4145a10 0x7fc4dc0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:21.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.057+0000 7fc4f9a00640 1 -- 172.21.15.131:0/2598111823 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc4f4145a10 msgr2=0x7fc4dc0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:21.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.057+0000 7fc4f9a00640 1 --2- 172.21.15.131:0/2598111823 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc4f4145a10 0x7fc4dc0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:21.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.057+0000 7fc4f9a00640 1 -- 172.21.15.131:0/2598111823 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc4dc003f80 msgr2=0x7fc4dc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:21.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.057+0000 7fc4f9a00640 1 --2- 172.21.15.131:0/2598111823 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc4dc003f80 0x7fc4dc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:21.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.058+0000 7fc4f9a00640 1 -- 172.21.15.131:0/2598111823 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4dc003520 con 0x7fc4dc004d40 2024-05-26T22:51:21.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.058+0000 7fc4f9a00640 1 --2- 172.21.15.131:0/2598111823 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc4dc004d40 0x7fc4dc0051c0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fc4ec009f90 tx=0x7fc4dc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58e44ad28448ba20 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:21.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.059+0000 7fc4f3400640 1 -- 172.21.15.131:0/2598111823 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4ec009c10 con 0x7fc4dc004d40 2024-05-26T22:51:21.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.059+0000 7fc4f3400640 1 -- 172.21.15.131:0/2598111823 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4ec036070 con 0x7fc4dc004d40 2024-05-26T22:51:21.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.059+0000 7fc4f3400640 1 -- 172.21.15.131:0/2598111823 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4ec031880 con 0x7fc4dc004d40 2024-05-26T22:51:21.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.059+0000 7fc4fa400640 1 -- 172.21.15.131:0/2598111823 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc4dc004d40 msgr2=0x7fc4dc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:21.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.059+0000 7fc4fa400640 1 --2- 172.21.15.131:0/2598111823 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc4dc004d40 0x7fc4dc0051c0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fc4ec009f90 tx=0x7fc4dc005830 comp rx=0 tx=0).stop 2024-05-26T22:51:21.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.059+0000 7fc4f9a00640 1 -- 172.21.15.131:0/2598111823 reap_dead start 2024-05-26T22:51:21.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.059+0000 7fc4fa400640 1 -- 172.21.15.131:0/2598111823 shutdown_connections 2024-05-26T22:51:21.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.059+0000 7fc4fa400640 1 -- 172.21.15.131:0/2598111823 >> 172.21.15.131:0/2598111823 conn(0x7fc4f40b5660 msgr2=0x7fc4f40b5a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:21.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.059+0000 7fc4fa400640 1 -- 172.21.15.131:0/2598111823 shutdown_connections 2024-05-26T22:51:21.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.059+0000 7fc4fa400640 1 -- 172.21.15.131:0/2598111823 wait complete. 2024-05-26T22:51:21.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.061+0000 7fc4fa400640 1 Processor -- start 2024-05-26T22:51:21.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.061+0000 7fc4fa400640 1 -- start start 2024-05-26T22:51:21.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.062+0000 7fc4fa400640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc4dc004d40 0x7fc4f41f1620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:21.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.062+0000 7fc4fa400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc4f413be20 0x7fc4f413c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:21.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.062+0000 7fc4fa400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc4f415edd0 0x7fc4f415f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:21.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.062+0000 7fc4fa400640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f4145df0 con 0x7fc4f413be20 2024-05-26T22:51:21.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.062+0000 7fc4fa400640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f40cf070 con 0x7fc4dc004d40 2024-05-26T22:51:21.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.062+0000 7fc4fa400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc4f4072180 con 0x7fc4f415edd0 2024-05-26T22:51:21.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.063+0000 7fc4f3e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc4f413be20 0x7fc4f413c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:21.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.063+0000 7fc4f3e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc4f413be20 0x7fc4f413c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44968/0 (socket says 172.21.15.131:44968) 2024-05-26T22:51:21.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.063+0000 7fc4f3e00640 1 -- 172.21.15.131:0/3646929999 learned_addr learned my addr 172.21.15.131:0/3646929999 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:21.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.063+0000 7fc4f9000640 1 --2- 172.21.15.131:0/3646929999 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc4dc004d40 0x7fc4f41f1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:21.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.063+0000 7fc4f9a00640 1 --2- 172.21.15.131:0/3646929999 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc4f415edd0 0x7fc4f415f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:21.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.063+0000 7fc4f3e00640 1 -- 172.21.15.131:0/3646929999 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc4f415edd0 msgr2=0x7fc4f415f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:21.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.063+0000 7fc4f3e00640 1 --2- 172.21.15.131:0/3646929999 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc4f415edd0 0x7fc4f415f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:21.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.063+0000 7fc4f3e00640 1 -- 172.21.15.131:0/3646929999 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc4dc004d40 msgr2=0x7fc4f41f1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:21.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.063+0000 7fc4f3e00640 1 --2- 172.21.15.131:0/3646929999 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc4dc004d40 0x7fc4f41f1620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:21.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.063+0000 7fc4f3e00640 1 -- 172.21.15.131:0/3646929999 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4f415f7c0 con 0x7fc4f413be20 2024-05-26T22:51:21.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.063+0000 7fc4f9000640 1 --2- 172.21.15.131:0/3646929999 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc4dc004d40 0x7fc4f41f1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:51:21.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.063+0000 7fc4f3e00640 1 --2- 172.21.15.131:0/3646929999 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc4f413be20 0x7fc4f413c2a0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fc4e000c730 tx=0x7fc4e000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:21.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.064+0000 7fc4f0c00640 1 -- 172.21.15.131:0/3646929999 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e0018070 con 0x7fc4f413be20 2024-05-26T22:51:21.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.064+0000 7fc4f0c00640 1 -- 172.21.15.131:0/3646929999 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4e00072f0 con 0x7fc4f413be20 2024-05-26T22:51:21.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.064+0000 7fc4fa400640 1 -- 172.21.15.131:0/3646929999 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4f41f5360 con 0x7fc4f413be20 2024-05-26T22:51:21.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.064+0000 7fc4f0c00640 1 -- 172.21.15.131:0/3646929999 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e00047e0 con 0x7fc4f413be20 2024-05-26T22:51:21.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.064+0000 7fc4fa400640 1 -- 172.21.15.131:0/3646929999 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4f41f5920 con 0x7fc4f413be20 2024-05-26T22:51:21.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.065+0000 7fc4fa400640 1 -- 172.21.15.131:0/3646929999 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4f40694e0 con 0x7fc4f413be20 2024-05-26T22:51:21.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.066+0000 7fc4f0c00640 1 -- 172.21.15.131:0/3646929999 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc4e001b070 con 0x7fc4f413be20 2024-05-26T22:51:21.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.066+0000 7fc4f0c00640 1 --2- 172.21.15.131:0/3646929999 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc4d406be60 0x7fc4d406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:21.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.067+0000 7fc4f0c00640 1 -- 172.21.15.131:0/3646929999 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5853+0+0 (secure 0 0 0) 0x7fc4e008a600 con 0x7fc4f413be20 2024-05-26T22:51:21.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.067+0000 7fc4f9000640 1 --2- 172.21.15.131:0/3646929999 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc4d406be60 0x7fc4d406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:21.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.068+0000 7fc4f9000640 1 --2- 172.21.15.131:0/3646929999 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc4d406be60 0x7fc4d406e320 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fc4f40b6120 tx=0x7fc4e8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:21.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.071+0000 7fc4f0c00640 1 -- 172.21.15.131:0/3646929999 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fc4e0058e80 con 0x7fc4f413be20 2024-05-26T22:51:21.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.254+0000 7fc4fa400640 1 -- 172.21.15.131:0/3646929999 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc4f41590f0 con 0x7fc4f413be20 2024-05-26T22:51:21.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.255+0000 7fc4f0c00640 1 -- 172.21.15.131:0/3646929999 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v47) ==== 104+0+11 (secure 0 0 0) 0x7fc4e0088e20 con 0x7fc4f413be20 2024-05-26T22:51:21.255 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-05-26T22:51:21.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.257+0000 7fc4fa400640 1 -- 172.21.15.131:0/3646929999 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc4d406be60 msgr2=0x7fc4d406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:21.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.257+0000 7fc4fa400640 1 --2- 172.21.15.131:0/3646929999 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc4d406be60 0x7fc4d406e320 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fc4f40b6120 tx=0x7fc4e8008040 comp rx=0 tx=0).stop 2024-05-26T22:51:21.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.257+0000 7fc4fa400640 1 -- 172.21.15.131:0/3646929999 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc4f413be20 msgr2=0x7fc4f413c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:21.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.257+0000 7fc4fa400640 1 --2- 172.21.15.131:0/3646929999 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc4f413be20 0x7fc4f413c2a0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fc4e000c730 tx=0x7fc4e000cbf0 comp rx=0 tx=0).stop 2024-05-26T22:51:21.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.258+0000 7fc4fa400640 1 -- 172.21.15.131:0/3646929999 shutdown_connections 2024-05-26T22:51:21.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.258+0000 7fc4fa400640 1 --2- 172.21.15.131:0/3646929999 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc4f415edd0 0x7fc4f415f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:21.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.258+0000 7fc4fa400640 1 --2- 172.21.15.131:0/3646929999 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc4d406be60 0x7fc4d406e320 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:21.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.258+0000 7fc4fa400640 1 --2- 172.21.15.131:0/3646929999 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc4f413be20 0x7fc4f413c2a0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:21.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.258+0000 7fc4fa400640 1 --2- 172.21.15.131:0/3646929999 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc4dc004d40 0x7fc4f41f1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:21.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.258+0000 7fc4fa400640 1 -- 172.21.15.131:0/3646929999 >> 172.21.15.131:0/3646929999 conn(0x7fc4f40b5660 msgr2=0x7fc4f40b07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:21.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.258+0000 7fc4fa400640 1 -- 172.21.15.131:0/3646929999 shutdown_connections 2024-05-26T22:51:21.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.258+0000 7fc4fa400640 1 -- 172.21.15.131:0/3646929999 wait complete. 2024-05-26T22:51:21.277 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-26T22:51:21.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.469+0000 7f2f26800640 1 Processor -- start 2024-05-26T22:51:21.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.469+0000 7f2f26800640 1 -- start start 2024-05-26T22:51:21.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.471+0000 7f2f26800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2f200712d0 0x7f2f2006d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:21.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.471+0000 7f2f26800640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f2f2006d790 0x7f2f200cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:21.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.471+0000 7f2f26800640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f20072d60 con 0x7f2f20145a10 2024-05-26T22:51:21.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.471+0000 7f2f26800640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f2f2015c760 con 0x7f2f2006d790 2024-05-26T22:51:21.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.471+0000 7f2f26800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f2f20072a80 con 0x7f2f200712d0 2024-05-26T22:51:21.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.472+0000 7f2f25e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f2f2006d790 0x7f2f200cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:21.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.472+0000 7f2f25e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f2f2006d790 0x7f2f200cf590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44976/0 (socket says 172.21.15.131:44976) 2024-05-26T22:51:21.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.472+0000 7f2f25e00640 1 -- 172.21.15.131:0/655092823 learned_addr learned my addr 172.21.15.131:0/655092823 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:21.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.472+0000 7f2f24a00640 1 --2- 172.21.15.131:0/655092823 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2f200712d0 0x7f2f2006d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:21.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.472+0000 7f2f25e00640 1 -- 172.21.15.131:0/655092823 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2f200712d0 msgr2=0x7f2f2006d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:21.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.472+0000 7f2f25e00640 1 --2- 172.21.15.131:0/655092823 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2f200712d0 0x7f2f2006d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:21.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.472+0000 7f2f25e00640 1 -- 172.21.15.131:0/655092823 >> v1:172.21.15.187:6789/0 conn(0x7f2f20145a10 legacy=0x7f2f20072180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:21.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.472+0000 7f2f25e00640 1 -- 172.21.15.131:0/655092823 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f200ceab0 con 0x7f2f2006d790 2024-05-26T22:51:21.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.474+0000 7f2f25e00640 1 --2- 172.21.15.131:0/655092823 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2f2006d790 0x7f2f200cf590 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f2f18009e50 tx=0x7f2f1802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=859ef44676f5047c server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:21.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.474+0000 7f2f1fe00640 1 -- 172.21.15.131:0/655092823 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f18036070 con 0x7f2f2006d790 2024-05-26T22:51:21.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.474+0000 7f2f1fe00640 1 -- 172.21.15.131:0/655092823 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f18031600 con 0x7f2f2006d790 2024-05-26T22:51:21.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.474+0000 7f2f1fe00640 1 -- 172.21.15.131:0/655092823 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f180319d0 con 0x7f2f2006d790 2024-05-26T22:51:21.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.475+0000 7f2f26800640 1 -- 172.21.15.131:0/655092823 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2f2006d790 msgr2=0x7f2f200cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:21.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.475+0000 7f2f26800640 1 --2- 172.21.15.131:0/655092823 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2f2006d790 0x7f2f200cf590 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f2f18009e50 tx=0x7f2f1802a710 comp rx=0 tx=0).stop 2024-05-26T22:51:21.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.475+0000 7f2f26800640 1 -- 172.21.15.131:0/655092823 shutdown_connections 2024-05-26T22:51:21.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.475+0000 7f2f26800640 1 --2- 172.21.15.131:0/655092823 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2f2006d790 0x7f2f200cf590 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:21.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.475+0000 7f2f26800640 1 --2- 172.21.15.131:0/655092823 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2f200712d0 0x7f2f2006d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:21.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.475+0000 7f2f26800640 1 -- 172.21.15.131:0/655092823 >> 172.21.15.131:0/655092823 conn(0x7f2f200b5660 msgr2=0x7f2f200b5a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:21.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.475+0000 7f2f26800640 1 -- 172.21.15.131:0/655092823 shutdown_connections 2024-05-26T22:51:21.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.475+0000 7f2f26800640 1 -- 172.21.15.131:0/655092823 wait complete. 2024-05-26T22:51:21.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.477+0000 7f2f26800640 1 Processor -- start 2024-05-26T22:51:21.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.477+0000 7f2f26800640 1 -- start start 2024-05-26T22:51:21.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.478+0000 7f2f26800640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f2006d790 0x7f2f2015ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:21.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.478+0000 7f2f26800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2f200712d0 0x7f2f2013fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:21.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.479+0000 7f2f26800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2f20145a10 0x7f2f2015e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:21.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.479+0000 7f2f26800640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f2f20145df0 con 0x7f2f20145a10 2024-05-26T22:51:21.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.479+0000 7f2f26800640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2f200cf070 con 0x7f2f2006d790 2024-05-26T22:51:21.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.479+0000 7f2f26800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f2f200cfad0 con 0x7f2f200712d0 2024-05-26T22:51:21.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.479+0000 7f2f25e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2f20145a10 0x7f2f2015e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:21.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.479+0000 7f2f25e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2f20145a10 0x7f2f2015e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44992/0 (socket says 172.21.15.131:44992) 2024-05-26T22:51:21.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.479+0000 7f2f25e00640 1 -- 172.21.15.131:0/170420509 learned_addr learned my addr 172.21.15.131:0/170420509 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:21.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.479+0000 7f2f24a00640 1 --2- 172.21.15.131:0/170420509 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2f200712d0 0x7f2f2013fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:21.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.479+0000 7f2f25400640 1 --2- 172.21.15.131:0/170420509 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f2006d790 0x7f2f2015ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:21.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.479+0000 7f2f25e00640 1 -- 172.21.15.131:0/170420509 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2f200712d0 msgr2=0x7f2f2013fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:21.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.479+0000 7f2f25e00640 1 --2- 172.21.15.131:0/170420509 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2f200712d0 0x7f2f2013fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:21.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.479+0000 7f2f25e00640 1 -- 172.21.15.131:0/170420509 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f2006d790 msgr2=0x7f2f2015ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:21.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.479+0000 7f2f25e00640 1 --2- 172.21.15.131:0/170420509 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f2006d790 0x7f2f2015ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:21.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.479+0000 7f2f25e00640 1 -- 172.21.15.131:0/170420509 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f2015bf40 con 0x7f2f20145a10 2024-05-26T22:51:21.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.480+0000 7f2f25e00640 1 --2- 172.21.15.131:0/170420509 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2f20145a10 0x7f2f2015e8f0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f2f18034040 tx=0x7f2f18004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:21.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.480+0000 7f2f1d600640 1 -- 172.21.15.131:0/170420509 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f1803f020 con 0x7f2f20145a10 2024-05-26T22:51:21.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.480+0000 7f2f1d600640 1 -- 172.21.15.131:0/170420509 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f18037040 con 0x7f2f20145a10 2024-05-26T22:51:21.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.480+0000 7f2f26800640 1 -- 172.21.15.131:0/170420509 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f201f75d0 con 0x7f2f20145a10 2024-05-26T22:51:21.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.480+0000 7f2f1d600640 1 -- 172.21.15.131:0/170420509 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f18036070 con 0x7f2f20145a10 2024-05-26T22:51:21.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.480+0000 7f2f26800640 1 -- 172.21.15.131:0/170420509 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f201f7b90 con 0x7f2f20145a10 2024-05-26T22:51:21.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.481+0000 7f2f26800640 1 -- 172.21.15.131:0/170420509 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f201f8250 con 0x7f2f20145a10 2024-05-26T22:51:21.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.482+0000 7f2f1d600640 1 -- 172.21.15.131:0/170420509 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2f18042020 con 0x7f2f20145a10 2024-05-26T22:51:21.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.483+0000 7f2f1d600640 1 --2- 172.21.15.131:0/170420509 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f2f0006be60 0x7f2f0006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:21.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.483+0000 7f2f1d600640 1 -- 172.21.15.131:0/170420509 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5853+0+0 (secure 0 0 0) 0x7f2f180ab360 con 0x7f2f20145a10 2024-05-26T22:51:21.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.483+0000 7f2f25400640 1 --2- 172.21.15.131:0/170420509 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f2f0006be60 0x7f2f0006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:21.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.484+0000 7f2f25400640 1 --2- 172.21.15.131:0/170420509 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f2f0006be60 0x7f2f0006e320 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f2f14007fd0 tx=0x7f2f14009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:21.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.488+0000 7f2f1d600640 1 -- 172.21.15.131:0/170420509 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f2f201f8250 con 0x7f2f20145a10 2024-05-26T22:51:21.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.694+0000 7f2f26800640 1 -- 172.21.15.131:0/170420509 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f2f200694e0 con 0x7f2f20145a10 2024-05-26T22:51:21.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.695+0000 7f2f1d600640 1 -- 172.21.15.131:0/170420509 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v47) ==== 100+0+11 (secure 0 0 0) 0x7f2f18049030 con 0x7f2f20145a10 2024-05-26T22:51:21.694 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 64 2024-05-26T22:51:21.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.697+0000 7f2f26800640 1 -- 172.21.15.131:0/170420509 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f2f0006be60 msgr2=0x7f2f0006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:21.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.697+0000 7f2f26800640 1 --2- 172.21.15.131:0/170420509 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f2f0006be60 0x7f2f0006e320 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f2f14007fd0 tx=0x7f2f14009040 comp rx=0 tx=0).stop 2024-05-26T22:51:21.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.697+0000 7f2f26800640 1 -- 172.21.15.131:0/170420509 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2f20145a10 msgr2=0x7f2f2015e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:21.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.697+0000 7f2f26800640 1 --2- 172.21.15.131:0/170420509 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2f20145a10 0x7f2f2015e8f0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f2f18034040 tx=0x7f2f18004a40 comp rx=0 tx=0).stop 2024-05-26T22:51:21.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.697+0000 7f2f26800640 1 -- 172.21.15.131:0/170420509 shutdown_connections 2024-05-26T22:51:21.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.697+0000 7f2f26800640 1 --2- 172.21.15.131:0/170420509 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2f200712d0 0x7f2f2013fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:21.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.697+0000 7f2f26800640 1 --2- 172.21.15.131:0/170420509 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f2f0006be60 0x7f2f0006e320 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:21.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.697+0000 7f2f26800640 1 --2- 172.21.15.131:0/170420509 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2f20145a10 0x7f2f2015e8f0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:21.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.697+0000 7f2f26800640 1 --2- 172.21.15.131:0/170420509 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f2006d790 0x7f2f2015ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:21.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.697+0000 7f2f26800640 1 -- 172.21.15.131:0/170420509 >> 172.21.15.131:0/170420509 conn(0x7f2f200b5660 msgr2=0x7f2f200af7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:21.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.697+0000 7f2f26800640 1 -- 172.21.15.131:0/170420509 shutdown_connections 2024-05-26T22:51:21.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.698+0000 7f2f26800640 1 -- 172.21.15.131:0/170420509 wait complete. 2024-05-26T22:51:21.717 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-26T22:51:21.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.914+0000 7f95d6e00640 1 Processor -- start 2024-05-26T22:51:21.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.914+0000 7f95d6e00640 1 -- start start 2024-05-26T22:51:21.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.916+0000 7f95d6e00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f95d006ee20 0x7f95d00b3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:21.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.916+0000 7f95d6e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f95d00ad250 0x7f95d00af790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:21.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.916+0000 7f95d6e00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95d015cb20 con 0x7f95d006f200 2024-05-26T22:51:21.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.916+0000 7f95d6e00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f95d015ce00 con 0x7f95d00ad250 2024-05-26T22:51:21.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.916+0000 7f95d6e00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f95d015d0e0 con 0x7f95d006ee20 2024-05-26T22:51:21.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.916+0000 7f95cf400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f95d00ad250 0x7f95d00af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:21.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.916+0000 7f95cf400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f95d00ad250 0x7f95d00af790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45008/0 (socket says 172.21.15.131:45008) 2024-05-26T22:51:21.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.916+0000 7f95cf400640 1 -- 172.21.15.131:0/577820322 learned_addr learned my addr 172.21.15.131:0/577820322 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:21.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.917+0000 7f95cfe00640 1 --2- 172.21.15.131:0/577820322 >> v2:172.21.15.187:3300/0 conn(0x7f95d006ee20 0x7f95d00b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:21.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.917+0000 7f95cf400640 1 -- 172.21.15.131:0/577820322 >> v2:172.21.15.187:3300/0 conn(0x7f95d006ee20 msgr2=0x7f95d00b3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:21.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.917+0000 7f95cf400640 1 --2- 172.21.15.131:0/577820322 >> v2:172.21.15.187:3300/0 conn(0x7f95d006ee20 0x7f95d00b3190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:21.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.917+0000 7f95cf400640 1 -- 172.21.15.131:0/577820322 >> v1:172.21.15.187:6789/0 conn(0x7f95d006f200 legacy=0x7f95d01562c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:21.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.917+0000 7f95cf400640 1 -- 172.21.15.131:0/577820322 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95d015d3c0 con 0x7f95d00ad250 2024-05-26T22:51:21.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.918+0000 7f95cf400640 1 --2- 172.21.15.131:0/577820322 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f95d00ad250 0x7f95d00af790 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f95d0071090 tx=0x7f95bc02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26340e15ae41bd33 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:21.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.918+0000 7f95cea00640 1 -- 172.21.15.131:0/577820322 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95bc037070 con 0x7f95d00ad250 2024-05-26T22:51:21.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.918+0000 7f95cea00640 1 -- 172.21.15.131:0/577820322 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95bc032440 con 0x7f95d00ad250 2024-05-26T22:51:21.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.918+0000 7f95cea00640 1 -- 172.21.15.131:0/577820322 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95bc032810 con 0x7f95d00ad250 2024-05-26T22:51:21.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.919+0000 7f95d6e00640 1 -- 172.21.15.131:0/577820322 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f95d00ad250 msgr2=0x7f95d00af790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:21.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.919+0000 7f95d6e00640 1 --2- 172.21.15.131:0/577820322 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f95d00ad250 0x7f95d00af790 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f95d0071090 tx=0x7f95bc02a1a0 comp rx=0 tx=0).stop 2024-05-26T22:51:21.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.919+0000 7f95d6e00640 1 -- 172.21.15.131:0/577820322 shutdown_connections 2024-05-26T22:51:21.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.919+0000 7f95d6e00640 1 --2- 172.21.15.131:0/577820322 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f95d00ad250 0x7f95d00af790 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:21.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.919+0000 7f95d6e00640 1 --2- 172.21.15.131:0/577820322 >> v2:172.21.15.187:3300/0 conn(0x7f95d006ee20 0x7f95d00b3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:21.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.919+0000 7f95d6e00640 1 -- 172.21.15.131:0/577820322 >> 172.21.15.131:0/577820322 conn(0x7f95d00b1e90 msgr2=0x7f95d00b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:21.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.919+0000 7f95d6e00640 1 -- 172.21.15.131:0/577820322 shutdown_connections 2024-05-26T22:51:21.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.919+0000 7f95d6e00640 1 -- 172.21.15.131:0/577820322 wait complete. 2024-05-26T22:51:21.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.921+0000 7f95d6e00640 1 Processor -- start 2024-05-26T22:51:21.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.921+0000 7f95d6e00640 1 -- start start 2024-05-26T22:51:21.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.922+0000 7f95d6e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f95d006ee20 0x7f95d0071b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:21.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.922+0000 7f95d6e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f95d006f200 0x7f95d0073210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:21.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.922+0000 7f95d6e00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f95d00ad250 0x7f95d0151080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:21.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.922+0000 7f95d6e00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f95d015c900 con 0x7f95d006f200 2024-05-26T22:51:21.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.922+0000 7f95d6e00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f95d015c780 con 0x7f95d00ad250 2024-05-26T22:51:21.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.922+0000 7f95d6e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f95d01569d0 con 0x7f95d006ee20 2024-05-26T22:51:21.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.922+0000 7f95cf400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f95d006f200 0x7f95d0073210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:21.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.922+0000 7f95cf400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f95d006f200 0x7f95d0073210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45012/0 (socket says 172.21.15.131:45012) 2024-05-26T22:51:21.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.922+0000 7f95cf400640 1 -- 172.21.15.131:0/3764063512 learned_addr learned my addr 172.21.15.131:0/3764063512 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:21.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.922+0000 7f95d4e00640 1 --2- 172.21.15.131:0/3764063512 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f95d00ad250 0x7f95d0151080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:21.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.922+0000 7f95cf400640 1 -- 172.21.15.131:0/3764063512 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f95d006ee20 msgr2=0x7f95d0071b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:21.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.922+0000 7f95cf400640 1 --2- 172.21.15.131:0/3764063512 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f95d006ee20 0x7f95d0071b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:21.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.922+0000 7f95cf400640 1 -- 172.21.15.131:0/3764063512 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f95d00ad250 msgr2=0x7f95d0151080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:21.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.922+0000 7f95cf400640 1 --2- 172.21.15.131:0/3764063512 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f95d00ad250 0x7f95d0151080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:21.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.922+0000 7f95cf400640 1 -- 172.21.15.131:0/3764063512 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95d0072170 con 0x7f95d006f200 2024-05-26T22:51:21.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.923+0000 7f95cf400640 1 --2- 172.21.15.131:0/3764063512 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f95d006f200 0x7f95d0073210 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f95bc000c00 tx=0x7f95bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:21.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.923+0000 7f95c7e00640 1 -- 172.21.15.131:0/3764063512 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95bc041020 con 0x7f95d006f200 2024-05-26T22:51:21.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.923+0000 7f95c7e00640 1 -- 172.21.15.131:0/3764063512 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95bc030970 con 0x7f95d006f200 2024-05-26T22:51:21.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.923+0000 7f95c7e00640 1 -- 172.21.15.131:0/3764063512 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95bc037040 con 0x7f95d006f200 2024-05-26T22:51:21.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.923+0000 7f95d6e00640 1 -- 172.21.15.131:0/3764063512 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95d0072450 con 0x7f95d006f200 2024-05-26T22:51:21.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.923+0000 7f95d6e00640 1 -- 172.21.15.131:0/3764063512 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95d0072a10 con 0x7f95d006f200 2024-05-26T22:51:21.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.924+0000 7f95d6e00640 1 -- 172.21.15.131:0/3764063512 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95d00694e0 con 0x7f95d006f200 2024-05-26T22:51:21.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.925+0000 7f95c7e00640 1 -- 172.21.15.131:0/3764063512 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f95bc044020 con 0x7f95d006f200 2024-05-26T22:51:21.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.926+0000 7f95c7e00640 1 --2- 172.21.15.131:0/3764063512 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f95b006be30 0x7f95b006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:21.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.927+0000 7f95c7e00640 1 -- 172.21.15.131:0/3764063512 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5853+0+0 (secure 0 0 0) 0x7f95bc0ab9d0 con 0x7f95d006f200 2024-05-26T22:51:21.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.927+0000 7f95cfe00640 1 --2- 172.21.15.131:0/3764063512 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f95b006be30 0x7f95b006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:21.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.927+0000 7f95cfe00640 1 --2- 172.21.15.131:0/3764063512 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f95b006be30 0x7f95b006e2f0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f95c0006fd0 tx=0x7f95c0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:21.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:21.931+0000 7f95c7e00640 1 -- 172.21.15.131:0/3764063512 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f95bc04b030 con 0x7f95d006f200 2024-05-26T22:51:22.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.172+0000 7f95d6e00640 1 -- 172.21.15.131:0/3764063512 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f95d00b49c0 con 0x7f95d006f200 2024-05-26T22:51:22.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.174+0000 7f95c7e00640 1 -- 172.21.15.131:0/3764063512 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+5230 (secure 0 0 0) 0x7f95bc07a250 con 0x7f95d006f200 2024-05-26T22:51:22.174 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:51:22.174 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":40,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4727,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/3668228040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6833","nonce":3668228040},{"type":"v1","addr":"172.21.15.187:6835","nonce":3668228040}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-26T22:50:27.001665+0000","modified":"2024-05-26T22:51:03.848463+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":45,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5637,"mds_1":4661,"mds_2":4670},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4661":{"gid":4661,"name":"a","rank":1,"incarnation":36,"state":"up:active","state_seq":14,"addr":"172.21.15.131:6837/265040431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6835","nonce":265040431},{"type":"v1","addr":"172.21.15.131:6837","nonce":265040431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4670":{"gid":4670,"name":"b","rank":2,"incarnation":32,"state":"up:active","state_seq":8,"addr":"172.21.15.187:6834/1027783169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1027783169},{"type":"v1","addr":"172.21.15.187:6834","nonce":1027783169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5637":{"gid":5637,"name":"c","rank":0,"incarnation":25,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6836/650770835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":650770835},{"type":"v1","addr":"172.21.15.131:6836","nonce":650770835}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5637,"qdb_cluster":[4661,4670,5637]},"id":3}]} 2024-05-26T22:51:22.174 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 40 2024-05-26T22:51:22.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.177+0000 7f95d6e00640 1 -- 172.21.15.131:0/3764063512 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f95b006be30 msgr2=0x7f95b006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:22.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.177+0000 7f95d6e00640 1 --2- 172.21.15.131:0/3764063512 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f95b006be30 0x7f95b006e2f0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f95c0006fd0 tx=0x7f95c0008040 comp rx=0 tx=0).stop 2024-05-26T22:51:22.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.177+0000 7f95d6e00640 1 -- 172.21.15.131:0/3764063512 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f95d006f200 msgr2=0x7f95d0073210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:22.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.177+0000 7f95d6e00640 1 --2- 172.21.15.131:0/3764063512 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f95d006f200 0x7f95d0073210 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f95bc000c00 tx=0x7f95bc0046c0 comp rx=0 tx=0).stop 2024-05-26T22:51:22.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.177+0000 7f95d6e00640 1 -- 172.21.15.131:0/3764063512 shutdown_connections 2024-05-26T22:51:22.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.177+0000 7f95d6e00640 1 --2- 172.21.15.131:0/3764063512 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f95b006be30 0x7f95b006e2f0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:22.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.177+0000 7f95d6e00640 1 --2- 172.21.15.131:0/3764063512 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f95d00ad250 0x7f95d0151080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:22.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.177+0000 7f95d6e00640 1 --2- 172.21.15.131:0/3764063512 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f95d006f200 0x7f95d0073210 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:22.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.177+0000 7f95d6e00640 1 --2- 172.21.15.131:0/3764063512 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f95d006ee20 0x7f95d0071b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:22.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.177+0000 7f95d6e00640 1 -- 172.21.15.131:0/3764063512 >> 172.21.15.131:0/3764063512 conn(0x7f95d00b1e90 msgr2=0x7f95d015bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:22.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.177+0000 7f95d6e00640 1 -- 172.21.15.131:0/3764063512 shutdown_connections 2024-05-26T22:51:22.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.177+0000 7f95d6e00640 1 -- 172.21.15.131:0/3764063512 wait complete. 2024-05-26T22:51:22.195 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-26T22:51:22.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.395+0000 7f5a77200640 1 Processor -- start 2024-05-26T22:51:22.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.395+0000 7f5a77200640 1 -- start start 2024-05-26T22:51:22.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.396+0000 7f5a77200640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f5a7806ebc0 0x7f5a780cedf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:22.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.396+0000 7f5a77200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5a78071b70 0x7f5a780b3d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:22.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.396+0000 7f5a77200640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a780722a0 con 0x7f5a78144340 2024-05-26T22:51:22.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.396+0000 7f5a77200640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f5a78072eb0 con 0x7f5a7806ebc0 2024-05-26T22:51:22.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.396+0000 7f5a77200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f5a78072580 con 0x7f5a78071b70 2024-05-26T22:51:22.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.396+0000 7f5a75e00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f5a78144340 0x7f5a7806e4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:51354/0 (socket says 172.21.15.131:51354) 2024-05-26T22:51:22.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.396+0000 7f5a75e00640 1 -- 172.21.15.131:0/816305687 learned_addr learned my addr 172.21.15.131:0/816305687 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:51:22.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.396+0000 7f5a74a00640 1 -- 172.21.15.131:0/816305687 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1430458659 0 0) 0x7f5a780722a0 con 0x7f5a78144340 2024-05-26T22:51:22.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.396+0000 7f5a74a00640 1 -- 172.21.15.131:0/816305687 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5a5c005730 con 0x7f5a78144340 2024-05-26T22:51:22.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.396+0000 7f5a76800640 1 --2- 172.21.15.131:0/816305687 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5a78071b70 0x7f5a780b3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:22.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.396+0000 7f5a75400640 1 --2- 172.21.15.131:0/816305687 >> v2:172.21.15.187:3300/0 conn(0x7f5a7806ebc0 0x7f5a780cedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:22.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.397+0000 7f5a74a00640 1 -- 172.21.15.131:0/816305687 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1292900102 0 0) 0x7f5a5c005730 con 0x7f5a78144340 2024-05-26T22:51:22.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.397+0000 7f5a74a00640 1 -- 172.21.15.131:0/816305687 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5a78071b70 msgr2=0x7f5a780b3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:22.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.397+0000 7f5a74a00640 1 --2- 172.21.15.131:0/816305687 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5a78071b70 0x7f5a780b3d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:22.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.397+0000 7f5a74a00640 1 -- 172.21.15.131:0/816305687 >> v2:172.21.15.187:3300/0 conn(0x7f5a7806ebc0 msgr2=0x7f5a780cedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:22.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.397+0000 7f5a74a00640 1 --2- 172.21.15.131:0/816305687 >> v2:172.21.15.187:3300/0 conn(0x7f5a7806ebc0 0x7f5a780cedf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:22.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.397+0000 7f5a74a00640 1 -- 172.21.15.131:0/816305687 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a78072860 con 0x7f5a78144340 2024-05-26T22:51:22.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.397+0000 7f5a74a00640 1 -- 172.21.15.131:0/816305687 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7f5a68001670 con 0x7f5a78144340 2024-05-26T22:51:22.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.397+0000 7f5a74a00640 1 -- 172.21.15.131:0/816305687 >> v1:172.21.15.131:6789/0 conn(0x7f5a78144340 legacy=0x7f5a7806e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:22.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.397+0000 7f5a74a00640 1 --2- 172.21.15.131:0/816305687 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5a78071b70 0x7f5a5c005bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:22.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.397+0000 7f5a74a00640 1 --2- 172.21.15.131:0/816305687 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a5c0061f0 0x7f5a5c006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:22.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.397+0000 7f5a74a00640 1 --2- 172.21.15.131:0/816305687 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5a5c006f50 0x7f5a5c0073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:22.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.397+0000 7f5a74a00640 1 -- 172.21.15.131:0/816305687 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f5a78072580 con 0x7f5a5c0061f0 2024-05-26T22:51:22.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.397+0000 7f5a74a00640 1 -- 172.21.15.131:0/816305687 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5a5c00dc80 con 0x7f5a5c006f50 2024-05-26T22:51:22.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.397+0000 7f5a74a00640 1 -- 172.21.15.131:0/816305687 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f5a5c00de50 con 0x7f5a78071b70 2024-05-26T22:51:22.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.397+0000 7f5a75400640 1 --2- 172.21.15.131:0/816305687 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a5c0061f0 0x7f5a5c006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:22.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.397+0000 7f5a75e00640 1 --2- 172.21.15.131:0/816305687 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5a78071b70 0x7f5a5c005bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:22.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.398+0000 7f5a76800640 1 --2- 172.21.15.131:0/816305687 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5a5c006f50 0x7f5a5c0073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:22.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.398+0000 7f5a75400640 1 -- 172.21.15.131:0/816305687 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5a78071b70 msgr2=0x7f5a5c005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:22.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.398+0000 7f5a75400640 1 --2- 172.21.15.131:0/816305687 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5a78071b70 0x7f5a5c005bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:22.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.398+0000 7f5a75400640 1 -- 172.21.15.131:0/816305687 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5a5c006f50 msgr2=0x7f5a5c0073d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:22.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.398+0000 7f5a75400640 1 --2- 172.21.15.131:0/816305687 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5a5c006f50 0x7f5a5c0073d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:22.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.398+0000 7f5a75400640 1 -- 172.21.15.131:0/816305687 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a5c005730 con 0x7f5a5c0061f0 2024-05-26T22:51:22.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.399+0000 7f5a75400640 1 --2- 172.21.15.131:0/816305687 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a5c0061f0 0x7f5a5c006a10 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f5a6400ba30 tx=0x7f5a5c007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e414363a45c14c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:22.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.399+0000 7f5a74a00640 1 -- 172.21.15.131:0/816305687 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a64030070 con 0x7f5a5c0061f0 2024-05-26T22:51:22.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.399+0000 7f5a74a00640 1 -- 172.21.15.131:0/816305687 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a6402ae10 con 0x7f5a5c0061f0 2024-05-26T22:51:22.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.399+0000 7f5a74a00640 1 -- 172.21.15.131:0/816305687 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a6400b600 con 0x7f5a5c0061f0 2024-05-26T22:51:22.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.400+0000 7f5a77200640 1 -- 172.21.15.131:0/816305687 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a5c0061f0 msgr2=0x7f5a5c006a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:22.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.400+0000 7f5a77200640 1 --2- 172.21.15.131:0/816305687 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a5c0061f0 0x7f5a5c006a10 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f5a6400ba30 tx=0x7f5a5c007a40 comp rx=0 tx=0).stop 2024-05-26T22:51:22.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.400+0000 7f5a76800640 1 -- 172.21.15.131:0/816305687 reap_dead start 2024-05-26T22:51:22.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.400+0000 7f5a77200640 1 -- 172.21.15.131:0/816305687 shutdown_connections 2024-05-26T22:51:22.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.400+0000 7f5a77200640 1 -- 172.21.15.131:0/816305687 >> 172.21.15.131:0/816305687 conn(0x7f5a780cff40 msgr2=0x7f5a780d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:22.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.400+0000 7f5a77200640 1 -- 172.21.15.131:0/816305687 shutdown_connections 2024-05-26T22:51:22.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.400+0000 7f5a77200640 1 -- 172.21.15.131:0/816305687 wait complete. 2024-05-26T22:51:22.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.402+0000 7f5a77200640 1 Processor -- start 2024-05-26T22:51:22.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.402+0000 7f5a77200640 1 -- start start 2024-05-26T22:51:22.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.403+0000 7f5a77200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5a5c0061f0 0x7f5a7815a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:22.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.403+0000 7f5a77200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a78139ff0 0x7f5a78142240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:22.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.403+0000 7f5a77200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5a78142780 0x7f5a7813d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:22.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.403+0000 7f5a77200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f5a7806d3a0 con 0x7f5a78139ff0 2024-05-26T22:51:22.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.403+0000 7f5a77200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5a78071fa0 con 0x7f5a78142780 2024-05-26T22:51:22.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.403+0000 7f5a77200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f5a7806d520 con 0x7f5a5c0061f0 2024-05-26T22:51:22.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.403+0000 7f5a75400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a78139ff0 0x7f5a78142240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:22.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.403+0000 7f5a75400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a78139ff0 0x7f5a78142240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45030/0 (socket says 172.21.15.131:45030) 2024-05-26T22:51:22.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.403+0000 7f5a75400640 1 -- 172.21.15.131:0/121973725 learned_addr learned my addr 172.21.15.131:0/121973725 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:22.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.403+0000 7f5a75e00640 1 --2- 172.21.15.131:0/121973725 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5a5c0061f0 0x7f5a7815a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:22.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.403+0000 7f5a76800640 1 --2- 172.21.15.131:0/121973725 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5a78142780 0x7f5a7813d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:22.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.403+0000 7f5a75400640 1 -- 172.21.15.131:0/121973725 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5a5c0061f0 msgr2=0x7f5a7815a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:22.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.403+0000 7f5a75400640 1 --2- 172.21.15.131:0/121973725 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5a5c0061f0 0x7f5a7815a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:22.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.403+0000 7f5a75400640 1 -- 172.21.15.131:0/121973725 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5a78142780 msgr2=0x7f5a7813d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:22.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.403+0000 7f5a75400640 1 --2- 172.21.15.131:0/121973725 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5a78142780 0x7f5a7813d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:22.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.403+0000 7f5a75400640 1 -- 172.21.15.131:0/121973725 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a78139230 con 0x7f5a78139ff0 2024-05-26T22:51:22.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.403+0000 7f5a76800640 1 --2- 172.21.15.131:0/121973725 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5a78142780 0x7f5a7813d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:51:22.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.404+0000 7f5a75400640 1 --2- 172.21.15.131:0/121973725 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a78139ff0 0x7f5a78142240 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f5a6402a900 tx=0x7f5a64004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:22.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.404+0000 7f5a6e000640 1 -- 172.21.15.131:0/121973725 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a64043030 con 0x7f5a78139ff0 2024-05-26T22:51:22.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.404+0000 7f5a6e000640 1 -- 172.21.15.131:0/121973725 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a64034e50 con 0x7f5a78139ff0 2024-05-26T22:51:22.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.404+0000 7f5a6e000640 1 -- 172.21.15.131:0/121973725 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a64004020 con 0x7f5a78139ff0 2024-05-26T22:51:22.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.404+0000 7f5a77200640 1 -- 172.21.15.131:0/121973725 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a78139510 con 0x7f5a78139ff0 2024-05-26T22:51:22.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.404+0000 7f5a77200640 1 -- 172.21.15.131:0/121973725 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a78139ad0 con 0x7f5a78139ff0 2024-05-26T22:51:22.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.405+0000 7f5a77200640 1 -- 172.21.15.131:0/121973725 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a780694e0 con 0x7f5a78139ff0 2024-05-26T22:51:22.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.407+0000 7f5a6e000640 1 -- 172.21.15.131:0/121973725 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5a64046070 con 0x7f5a78139ff0 2024-05-26T22:51:22.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.408+0000 7f5a6e000640 1 --2- 172.21.15.131:0/121973725 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f5a5806be30 0x7f5a5806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:22.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.408+0000 7f5a6e000640 1 -- 172.21.15.131:0/121973725 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5853+0+0 (secure 0 0 0) 0x7f5a640ab9c0 con 0x7f5a78139ff0 2024-05-26T22:51:22.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.408+0000 7f5a75e00640 1 --2- 172.21.15.131:0/121973725 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f5a5806be30 0x7f5a5806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:22.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.409+0000 7f5a75e00640 1 --2- 172.21.15.131:0/121973725 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f5a5806be30 0x7f5a5806e2f0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f5a68001eb0 tx=0x7f5a68009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:22.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.413+0000 7f5a6e000640 1 -- 172.21.15.131:0/121973725 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f5a640ade20 con 0x7f5a78139ff0 2024-05-26T22:51:22.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.615+0000 7f5a77200640 1 -- 172.21.15.131:0/121973725 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5a78142b60 con 0x7f5a78139ff0 2024-05-26T22:51:22.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.617+0000 7f5a6e000640 1 -- 172.21.15.131:0/121973725 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) ==== 74+0+16860 (secure 0 0 0) 0x7f5a6403c3a0 con 0x7f5a78139ff0 2024-05-26T22:51:22.617 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:51:22.617 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":47,"fsid":"46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f","created":"2024-05-26T22:47:57.288905+0000","modified":"2024-05-26T22:51:05.195752+0000","last_up_change":"2024-05-26T22:48:09.602207+0000","last_in_change":"2024-05-26T22:48:00.687341+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-26T22:48:10.463339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-05-26T22:50:24.793406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-05-26T22:50:26.041636+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"814eaf62-a18d-494b-8e27-fd8fb88adea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6801","nonce":2350940051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6803","nonce":2350940051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6807","nonce":2350940051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6805","nonce":2350940051}]},"public_addr":"172.21.15.131:6801/2350940051","cluster_addr":"172.21.15.131:6803/2350940051","heartbeat_back_addr":"172.21.15.131:6807/2350940051","heartbeat_front_addr":"172.21.15.131:6805/2350940051","state":["exists","up"]},{"osd":1,"uuid":"42e66287-f6c6-4e26-af3c-6dfaa3536d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6825","nonce":1525528247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6827","nonce":1525528247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6831","nonce":1525528247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6829","nonce":1525528247}]},"public_addr":"172.21.15.131:6825/1525528247","cluster_addr":"172.21.15.131:6827/1525528247","heartbeat_back_addr":"172.21.15.131:6831/1525528247","heartbeat_front_addr":"172.21.15.131:6829/1525528247","state":["exists","up"]},{"osd":2,"uuid":"0da34673-04a2-4fd0-b57a-0ff2b679b043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6817","nonce":3627842210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6819","nonce":3627842210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6823","nonce":3627842210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6821","nonce":3627842210}]},"public_addr":"172.21.15.131:6817/3627842210","cluster_addr":"172.21.15.131:6819/3627842210","heartbeat_back_addr":"172.21.15.131:6823/3627842210","heartbeat_front_addr":"172.21.15.131:6821/3627842210","state":["exists","up"]},{"osd":3,"uuid":"cf117a27-d6b2-43d3-ae52-b47988bf8eed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6809","nonce":350790591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6811","nonce":350790591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6815","nonce":350790591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6813","nonce":350790591}]},"public_addr":"172.21.15.131:6809/350790591","cluster_addr":"172.21.15.131:6811/350790591","heartbeat_back_addr":"172.21.15.131:6815/350790591","heartbeat_front_addr":"172.21.15.131:6813/350790591","state":["exists","up"]},{"osd":4,"uuid":"a092ccfc-be89-4a84-88d5-0f781fa715eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6816","nonce":244560995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6819","nonce":244560995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6823","nonce":244560995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6821","nonce":244560995}]},"public_addr":"172.21.15.187:6816/244560995","cluster_addr":"172.21.15.187:6819/244560995","heartbeat_back_addr":"172.21.15.187:6823/244560995","heartbeat_front_addr":"172.21.15.187:6821/244560995","state":["exists","up"]},{"osd":5,"uuid":"7513ba69-dd00-4a35-b190-9e8cc3b2bdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6801","nonce":324619398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6803","nonce":324619398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6812","nonce":324619398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6807","nonce":324619398}]},"public_addr":"172.21.15.187:6801/324619398","cluster_addr":"172.21.15.187:6803/324619398","heartbeat_back_addr":"172.21.15.187:6812/324619398","heartbeat_front_addr":"172.21.15.187:6807/324619398","state":["exists","up"]},{"osd":6,"uuid":"50081810-c093-4d2f-a4d7-b7c695c5400d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6806","nonce":1391347786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6809","nonce":1391347786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6818","nonce":1391347786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6813","nonce":1391347786}]},"public_addr":"172.21.15.187:6806/1391347786","cluster_addr":"172.21.15.187:6809/1391347786","heartbeat_back_addr":"172.21.15.187:6818/1391347786","heartbeat_front_addr":"172.21.15.187:6813/1391347786","state":["exists","up"]},{"osd":7,"uuid":"775dae05-54f5-4ad4-bbce-92863c6ed2ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6825","nonce":1220065622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6827","nonce":1220065622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6831","nonce":1220065622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6829","nonce":1220065622}]},"public_addr":"172.21.15.187:6825/1220065622","cluster_addr":"172.21.15.187:6827/1220065622","heartbeat_back_addr":"172.21.15.187:6831/1220065622","heartbeat_front_addr":"172.21.15.187:6829/1220065622","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:05.983520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.299516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.067323+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.307748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.018742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.258775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.010912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.295355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.36","mappings":[{"from":0,"to":5}]},{"pgid":"7.38","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.187:6835/3429891101":"2024-05-27T22:50:59.823337+0000","172.21.15.187:6833/3429891101":"2024-05-27T22:50:59.823337+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-26T22:51:22.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.620+0000 7f5a77200640 1 -- 172.21.15.131:0/121973725 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f5a5806be30 msgr2=0x7f5a5806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:22.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.620+0000 7f5a77200640 1 --2- 172.21.15.131:0/121973725 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f5a5806be30 0x7f5a5806e2f0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f5a68001eb0 tx=0x7f5a68009040 comp rx=0 tx=0).stop 2024-05-26T22:51:22.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.620+0000 7f5a77200640 1 -- 172.21.15.131:0/121973725 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a78139ff0 msgr2=0x7f5a78142240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:22.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.620+0000 7f5a77200640 1 --2- 172.21.15.131:0/121973725 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a78139ff0 0x7f5a78142240 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f5a6402a900 tx=0x7f5a64004730 comp rx=0 tx=0).stop 2024-05-26T22:51:22.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.620+0000 7f5a77200640 1 -- 172.21.15.131:0/121973725 shutdown_connections 2024-05-26T22:51:22.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.620+0000 7f5a77200640 1 --2- 172.21.15.131:0/121973725 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f5a5806be30 0x7f5a5806e2f0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:22.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.620+0000 7f5a77200640 1 --2- 172.21.15.131:0/121973725 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5a78142780 0x7f5a7813d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:22.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.620+0000 7f5a77200640 1 --2- 172.21.15.131:0/121973725 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a78139ff0 0x7f5a78142240 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:22.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.620+0000 7f5a77200640 1 --2- 172.21.15.131:0/121973725 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5a5c0061f0 0x7f5a7815a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:22.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.620+0000 7f5a77200640 1 -- 172.21.15.131:0/121973725 >> 172.21.15.131:0/121973725 conn(0x7f5a780cff40 msgr2=0x7f5a78071420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:22.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.620+0000 7f5a77200640 1 -- 172.21.15.131:0/121973725 shutdown_connections 2024-05-26T22:51:22.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.620+0000 7f5a77200640 1 -- 172.21.15.131:0/121973725 wait complete. 2024-05-26T22:51:22.640 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-26T22:51:22.640 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-26T22:51:22.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.837+0000 7fafe5c00640 1 Processor -- start 2024-05-26T22:51:22.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.837+0000 7fafe5c00640 1 -- start start 2024-05-26T22:51:22.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.839+0000 7fafe5c00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fafe00b5b90 0x7fafe00d23d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:22.840 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.839+0000 7fafe5c00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fafe0161de0 0x7fafe00b5f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:22.840 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.839+0000 7fafe5c00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fafe00b64b0 con 0x7fafe00b9990 2024-05-26T22:51:22.840 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.839+0000 7fafe5c00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fafe015e970 con 0x7fafe0161de0 2024-05-26T22:51:22.840 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.839+0000 7fafe5c00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fafe00d2910 con 0x7fafe00b5b90 2024-05-26T22:51:22.840 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.839+0000 7fafd7400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fafe0161de0 0x7fafe00b5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:22.840 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.839+0000 7fafd7400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fafe0161de0 0x7fafe00b5f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45038/0 (socket says 172.21.15.131:45038) 2024-05-26T22:51:22.840 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.839+0000 7fafd7400640 1 -- 172.21.15.131:0/298746519 learned_addr learned my addr 172.21.15.131:0/298746519 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:22.840 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.839+0000 7fafd7400640 1 -- 172.21.15.131:0/298746519 >> v2:172.21.15.187:3300/0 conn(0x7fafe00b5b90 msgr2=0x7fafe00d23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:22.840 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.839+0000 7fafd7400640 1 --2- 172.21.15.131:0/298746519 >> v2:172.21.15.187:3300/0 conn(0x7fafe00b5b90 0x7fafe00d23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:22.840 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.839+0000 7fafd7400640 1 -- 172.21.15.131:0/298746519 >> v1:172.21.15.131:6789/0 conn(0x7fafe00b9990 legacy=0x7fafe015ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:22.841 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.839+0000 7fafd7400640 1 -- 172.21.15.131:0/298746519 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafe0162330 con 0x7fafe0161de0 2024-05-26T22:51:22.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.840+0000 7fafd7400640 1 --2- 172.21.15.131:0/298746519 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fafe0161de0 0x7fafe00b5f70 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fafe015eaf0 tx=0x7fafc802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52ae37b40cde9170 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:22.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.840+0000 7fafdf400640 1 -- 172.21.15.131:0/298746519 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc8037070 con 0x7fafe0161de0 2024-05-26T22:51:22.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.840+0000 7fafdf400640 1 -- 172.21.15.131:0/298746519 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafc8032440 con 0x7fafe0161de0 2024-05-26T22:51:22.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.840+0000 7fafdf400640 1 -- 172.21.15.131:0/298746519 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc8032810 con 0x7fafe0161de0 2024-05-26T22:51:22.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.841+0000 7fafe5c00640 1 -- 172.21.15.131:0/298746519 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fafe0161de0 msgr2=0x7fafe00b5f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:22.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.841+0000 7fafe5c00640 1 --2- 172.21.15.131:0/298746519 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fafe0161de0 0x7fafe00b5f70 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fafe015eaf0 tx=0x7fafc802a1e0 comp rx=0 tx=0).stop 2024-05-26T22:51:22.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.841+0000 7fafe5c00640 1 -- 172.21.15.131:0/298746519 shutdown_connections 2024-05-26T22:51:22.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.841+0000 7fafe5c00640 1 --2- 172.21.15.131:0/298746519 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fafe0161de0 0x7fafe00b5f70 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:22.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.841+0000 7fafe5c00640 1 --2- 172.21.15.131:0/298746519 >> v2:172.21.15.187:3300/0 conn(0x7fafe00b5b90 0x7fafe00d23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:22.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.841+0000 7fafe5c00640 1 -- 172.21.15.131:0/298746519 >> 172.21.15.131:0/298746519 conn(0x7fafe0073070 msgr2=0x7fafe00d0b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:22.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.841+0000 7fafe5c00640 1 -- 172.21.15.131:0/298746519 shutdown_connections 2024-05-26T22:51:22.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.841+0000 7fafe5c00640 1 -- 172.21.15.131:0/298746519 wait complete. 2024-05-26T22:51:22.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.843+0000 7fafe5c00640 1 Processor -- start 2024-05-26T22:51:22.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.843+0000 7fafe5c00640 1 -- start start 2024-05-26T22:51:22.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.844+0000 7fafe5c00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fafe00b5b90 0x7fafe01e8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:22.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.844+0000 7fafe5c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fafe00b9990 0x7fafe01f1170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:22.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.844+0000 7fafe5c00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fafe0161de0 0x7fafe01e95e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:22.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.844+0000 7fafe5c00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fafe0162bd0 con 0x7fafe00b9990 2024-05-26T22:51:22.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.844+0000 7fafe5c00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fafe01628f0 con 0x7fafe0161de0 2024-05-26T22:51:22.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.844+0000 7fafe5c00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fafe00b9e00 con 0x7fafe00b5b90 2024-05-26T22:51:22.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.844+0000 7fafd7400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fafe00b9990 0x7fafe01f1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:22.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.844+0000 7fafd7400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fafe00b9990 0x7fafe01f1170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45042/0 (socket says 172.21.15.131:45042) 2024-05-26T22:51:22.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.844+0000 7fafd7400640 1 -- 172.21.15.131:0/2040744997 learned_addr learned my addr 172.21.15.131:0/2040744997 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:22.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.845+0000 7fafdfe00640 1 --2- 172.21.15.131:0/2040744997 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fafe00b5b90 0x7fafe01e8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:22.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.845+0000 7fafd7400640 1 -- 172.21.15.131:0/2040744997 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fafe00b5b90 msgr2=0x7fafe01e8ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:22.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.845+0000 7fafe5200640 1 --2- 172.21.15.131:0/2040744997 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fafe0161de0 0x7fafe01e95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:22.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.845+0000 7fafd7400640 1 --2- 172.21.15.131:0/2040744997 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fafe00b5b90 0x7fafe01e8ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:22.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.845+0000 7fafd7400640 1 -- 172.21.15.131:0/2040744997 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fafe0161de0 msgr2=0x7fafe01e95e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:22.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.845+0000 7fafd7400640 1 --2- 172.21.15.131:0/2040744997 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fafe0161de0 0x7fafe01e95e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:22.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.845+0000 7fafd7400640 1 -- 172.21.15.131:0/2040744997 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafe01f7600 con 0x7fafe00b9990 2024-05-26T22:51:22.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.845+0000 7fafd7400640 1 --2- 172.21.15.131:0/2040744997 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fafe00b9990 0x7fafe01f1170 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fafc802a220 tx=0x7fafc80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:22.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.845+0000 7fafdcc00640 1 -- 172.21.15.131:0/2040744997 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc8041020 con 0x7fafe00b9990 2024-05-26T22:51:22.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.845+0000 7fafdcc00640 1 -- 172.21.15.131:0/2040744997 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafc8030990 con 0x7fafe00b9990 2024-05-26T22:51:22.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.845+0000 7fafdcc00640 1 -- 172.21.15.131:0/2040744997 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc8037040 con 0x7fafe00b9990 2024-05-26T22:51:22.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.845+0000 7fafe5c00640 1 -- 172.21.15.131:0/2040744997 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafe01f78e0 con 0x7fafe00b9990 2024-05-26T22:51:22.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.846+0000 7fafe5c00640 1 -- 172.21.15.131:0/2040744997 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafe01f7ea0 con 0x7fafe00b9990 2024-05-26T22:51:22.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.847+0000 7fafe5c00640 1 -- 172.21.15.131:0/2040744997 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafe00694e0 con 0x7fafe00b9990 2024-05-26T22:51:22.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.848+0000 7fafdcc00640 1 -- 172.21.15.131:0/2040744997 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fafc8044020 con 0x7fafe00b9990 2024-05-26T22:51:22.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.848+0000 7fafdcc00640 1 --2- 172.21.15.131:0/2040744997 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fafbc06be30 0x7fafbc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:22.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.849+0000 7fafdcc00640 1 -- 172.21.15.131:0/2040744997 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5853+0+0 (secure 0 0 0) 0x7fafc80ab9c0 con 0x7fafe00b9990 2024-05-26T22:51:22.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.849+0000 7fafdfe00640 1 --2- 172.21.15.131:0/2040744997 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fafbc06be30 0x7fafbc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:22.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.849+0000 7fafdfe00640 1 --2- 172.21.15.131:0/2040744997 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fafbc06be30 0x7fafbc06e2f0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fafe0161a60 tx=0x7fafd0007310 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:22.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:22.853+0000 7fafdcc00640 1 -- 172.21.15.131:0/2040744997 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fafc804b030 con 0x7fafe00b9990 2024-05-26T22:51:23.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.089+0000 7fafe5c00640 1 -- 172.21.15.131:0/2040744997 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fafe01f3c30 con 0x7fafe00b9990 2024-05-26T22:51:23.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.091+0000 7fafdcc00640 1 -- 172.21.15.131:0/2040744997 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+5230 (secure 0 0 0) 0x7fafc807a240 con 0x7fafe00b9990 2024-05-26T22:51:23.091 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:51:23.091 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":40,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4727,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/3668228040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6833","nonce":3668228040},{"type":"v1","addr":"172.21.15.187:6835","nonce":3668228040}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-26T22:50:27.001665+0000","modified":"2024-05-26T22:51:03.848463+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":45,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5637,"mds_1":4661,"mds_2":4670},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4661":{"gid":4661,"name":"a","rank":1,"incarnation":36,"state":"up:active","state_seq":14,"addr":"172.21.15.131:6837/265040431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6835","nonce":265040431},{"type":"v1","addr":"172.21.15.131:6837","nonce":265040431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4670":{"gid":4670,"name":"b","rank":2,"incarnation":32,"state":"up:active","state_seq":8,"addr":"172.21.15.187:6834/1027783169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1027783169},{"type":"v1","addr":"172.21.15.187:6834","nonce":1027783169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5637":{"gid":5637,"name":"c","rank":0,"incarnation":25,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6836/650770835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":650770835},{"type":"v1","addr":"172.21.15.131:6836","nonce":650770835}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5637,"qdb_cluster":[4661,4670,5637]},"id":3}]} 2024-05-26T22:51:23.092 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 40 2024-05-26T22:51:23.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.093+0000 7fafe5c00640 1 -- 172.21.15.131:0/2040744997 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fafbc06be30 msgr2=0x7fafbc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:23.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.093+0000 7fafe5c00640 1 --2- 172.21.15.131:0/2040744997 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fafbc06be30 0x7fafbc06e2f0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fafe0161a60 tx=0x7fafd0007310 comp rx=0 tx=0).stop 2024-05-26T22:51:23.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.093+0000 7fafe5c00640 1 -- 172.21.15.131:0/2040744997 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fafe00b9990 msgr2=0x7fafe01f1170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:23.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.093+0000 7fafe5c00640 1 --2- 172.21.15.131:0/2040744997 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fafe00b9990 0x7fafe01f1170 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fafc802a220 tx=0x7fafc80046c0 comp rx=0 tx=0).stop 2024-05-26T22:51:23.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.094+0000 7fafe5c00640 1 -- 172.21.15.131:0/2040744997 shutdown_connections 2024-05-26T22:51:23.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.094+0000 7fafe5c00640 1 --2- 172.21.15.131:0/2040744997 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fafbc06be30 0x7fafbc06e2f0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:23.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.094+0000 7fafe5c00640 1 --2- 172.21.15.131:0/2040744997 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fafe0161de0 0x7fafe01e95e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:23.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.094+0000 7fafe5c00640 1 --2- 172.21.15.131:0/2040744997 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fafe00b9990 0x7fafe01f1170 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:23.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.094+0000 7fafe5c00640 1 --2- 172.21.15.131:0/2040744997 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fafe00b5b90 0x7fafe01e8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:23.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.094+0000 7fafe5c00640 1 -- 172.21.15.131:0/2040744997 >> 172.21.15.131:0/2040744997 conn(0x7fafe0073070 msgr2=0x7fafe015c9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:23.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.094+0000 7fafe5c00640 1 -- 172.21.15.131:0/2040744997 shutdown_connections 2024-05-26T22:51:23.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.094+0000 7fafe5c00640 1 -- 172.21.15.131:0/2040744997 wait complete. 2024-05-26T22:51:23.113 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-26T22:51:23.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.322+0000 7f1bf0a00640 1 Processor -- start 2024-05-26T22:51:23.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.322+0000 7f1bf0a00640 1 -- start start 2024-05-26T22:51:23.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.324+0000 7f1bf0a00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f1bec158c00 0x7f1bec06d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:23.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.324+0000 7f1bf0a00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f1bec0b38a0 0x7f1bec15fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:23.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.324+0000 7f1bf0a00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1bec0b3e50 con 0x7f1bec06daf0 2024-05-26T22:51:23.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.324+0000 7f1bf0a00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f1bec06df50 con 0x7f1bec0b38a0 2024-05-26T22:51:23.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.324+0000 7f1bf0a00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f1bec0b4130 con 0x7f1bec158c00 2024-05-26T22:51:23.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.324+0000 7f1bea800640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f1bec0b38a0 0x7f1bec15fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:23.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.324+0000 7f1bea800640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f1bec0b38a0 0x7f1bec15fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45048/0 (socket says 172.21.15.131:45048) 2024-05-26T22:51:23.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.324+0000 7f1bea800640 1 -- 172.21.15.131:0/1252019059 learned_addr learned my addr 172.21.15.131:0/1252019059 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:23.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.325+0000 7f1be9e00640 1 --2- 172.21.15.131:0/1252019059 >> v2:172.21.15.187:3300/0 conn(0x7f1bec158c00 0x7f1bec06d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:23.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.325+0000 7f1bea800640 1 -- 172.21.15.131:0/1252019059 >> v2:172.21.15.187:3300/0 conn(0x7f1bec158c00 msgr2=0x7f1bec06d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:23.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.325+0000 7f1bea800640 1 --2- 172.21.15.131:0/1252019059 >> v2:172.21.15.187:3300/0 conn(0x7f1bec158c00 0x7f1bec06d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:23.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.325+0000 7f1bea800640 1 -- 172.21.15.131:0/1252019059 >> v1:172.21.15.131:6789/0 conn(0x7f1bec06daf0 legacy=0x7f1bec0b3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:23.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.325+0000 7f1bea800640 1 -- 172.21.15.131:0/1252019059 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bec0b4410 con 0x7f1bec0b38a0 2024-05-26T22:51:23.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.325+0000 7f1be9e00640 1 --2- 172.21.15.131:0/1252019059 >> v2:172.21.15.187:3300/0 conn(0x7f1bec158c00 0x7f1bec06d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:51:23.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.326+0000 7f1bea800640 1 --2- 172.21.15.131:0/1252019059 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1bec0b38a0 0x7f1bec15fc50 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f1bec06c1f0 tx=0x7f1be402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7fedc58ba7058b7 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:23.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.326+0000 7f1be8a00640 1 -- 172.21.15.131:0/1252019059 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1be4036070 con 0x7f1bec0b38a0 2024-05-26T22:51:23.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.326+0000 7f1be8a00640 1 -- 172.21.15.131:0/1252019059 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1be4031600 con 0x7f1bec0b38a0 2024-05-26T22:51:23.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.326+0000 7f1be8a00640 1 -- 172.21.15.131:0/1252019059 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1be40319d0 con 0x7f1bec0b38a0 2024-05-26T22:51:23.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.327+0000 7f1bf0a00640 1 -- 172.21.15.131:0/1252019059 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1bec0b38a0 msgr2=0x7f1bec15fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:23.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.327+0000 7f1bf0a00640 1 --2- 172.21.15.131:0/1252019059 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1bec0b38a0 0x7f1bec15fc50 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f1bec06c1f0 tx=0x7f1be402a6a0 comp rx=0 tx=0).stop 2024-05-26T22:51:23.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.327+0000 7f1bf0a00640 1 -- 172.21.15.131:0/1252019059 shutdown_connections 2024-05-26T22:51:23.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.327+0000 7f1bf0a00640 1 --2- 172.21.15.131:0/1252019059 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1bec0b38a0 0x7f1bec15fc50 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:23.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.327+0000 7f1bf0a00640 1 --2- 172.21.15.131:0/1252019059 >> v2:172.21.15.187:3300/0 conn(0x7f1bec158c00 0x7f1bec06d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:23.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.327+0000 7f1bf0a00640 1 -- 172.21.15.131:0/1252019059 >> 172.21.15.131:0/1252019059 conn(0x7f1bec0b1e90 msgr2=0x7f1bec0b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:23.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.327+0000 7f1bf0a00640 1 -- 172.21.15.131:0/1252019059 shutdown_connections 2024-05-26T22:51:23.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.327+0000 7f1bf0a00640 1 -- 172.21.15.131:0/1252019059 wait complete. 2024-05-26T22:51:23.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.329+0000 7f1bf0a00640 1 Processor -- start 2024-05-26T22:51:23.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.329+0000 7f1bf0a00640 1 -- start start 2024-05-26T22:51:23.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.330+0000 7f1bf0a00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1bec06daf0 0x7f1bec0af550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:23.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.330+0000 7f1bf0a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1bec0b38a0 0x7f1bec14d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:23.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.330+0000 7f1bf0a00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1bec158c00 0x7f1bec0ae2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:23.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.330+0000 7f1bf0a00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f1bec160190 con 0x7f1bec0b38a0 2024-05-26T22:51:23.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.330+0000 7f1bf0a00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1bec0b49d0 con 0x7f1bec06daf0 2024-05-26T22:51:23.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.330+0000 7f1bf0a00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1bec160310 con 0x7f1bec158c00 2024-05-26T22:51:23.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.330+0000 7f1be9400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1bec0b38a0 0x7f1bec14d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:23.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.330+0000 7f1be9400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1bec0b38a0 0x7f1bec14d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45056/0 (socket says 172.21.15.131:45056) 2024-05-26T22:51:23.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.330+0000 7f1be9400640 1 -- 172.21.15.131:0/2010872298 learned_addr learned my addr 172.21.15.131:0/2010872298 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:23.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.331+0000 7f1bea800640 1 --2- 172.21.15.131:0/2010872298 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1bec158c00 0x7f1bec0ae2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:23.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.331+0000 7f1be9e00640 1 --2- 172.21.15.131:0/2010872298 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1bec06daf0 0x7f1bec0af550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:23.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.331+0000 7f1be9400640 1 -- 172.21.15.131:0/2010872298 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1bec158c00 msgr2=0x7f1bec0ae2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:23.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.331+0000 7f1be9400640 1 --2- 172.21.15.131:0/2010872298 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1bec158c00 0x7f1bec0ae2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:23.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.331+0000 7f1be9400640 1 -- 172.21.15.131:0/2010872298 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1bec06daf0 msgr2=0x7f1bec0af550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:23.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.331+0000 7f1be9400640 1 --2- 172.21.15.131:0/2010872298 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1bec06daf0 0x7f1bec0af550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:23.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.331+0000 7f1be9400640 1 -- 172.21.15.131:0/2010872298 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bec1591a0 con 0x7f1bec0b38a0 2024-05-26T22:51:23.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.331+0000 7f1be9e00640 1 --2- 172.21.15.131:0/2010872298 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1bec06daf0 0x7f1bec0af550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:51:23.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.331+0000 7f1be9400640 1 --2- 172.21.15.131:0/2010872298 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1bec0b38a0 0x7f1bec14d470 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f1bd800c430 tx=0x7f1bd800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:23.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.331+0000 7f1be2000640 1 -- 172.21.15.131:0/2010872298 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd80052d0 con 0x7f1bec0b38a0 2024-05-26T22:51:23.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.331+0000 7f1be2000640 1 -- 172.21.15.131:0/2010872298 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bd8005490 con 0x7f1bec0b38a0 2024-05-26T22:51:23.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.331+0000 7f1bf0a00640 1 -- 172.21.15.131:0/2010872298 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bec159440 con 0x7f1bec0b38a0 2024-05-26T22:51:23.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.331+0000 7f1be2000640 1 -- 172.21.15.131:0/2010872298 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd8007260 con 0x7f1bec0b38a0 2024-05-26T22:51:23.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.332+0000 7f1bf0a00640 1 -- 172.21.15.131:0/2010872298 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bec159a00 con 0x7f1bec0b38a0 2024-05-26T22:51:23.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.333+0000 7f1bf0a00640 1 -- 172.21.15.131:0/2010872298 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bec159c00 con 0x7f1bec0b38a0 2024-05-26T22:51:23.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.334+0000 7f1be2000640 1 -- 172.21.15.131:0/2010872298 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1bd8007400 con 0x7f1bec0b38a0 2024-05-26T22:51:23.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.336+0000 7f1be2000640 1 --2- 172.21.15.131:0/2010872298 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f1bcc06be30 0x7f1bcc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:23.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.336+0000 7f1be2000640 1 -- 172.21.15.131:0/2010872298 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5853+0+0 (secure 0 0 0) 0x7f1bd808a420 con 0x7f1bec0b38a0 2024-05-26T22:51:23.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.336+0000 7f1be9e00640 1 --2- 172.21.15.131:0/2010872298 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f1bcc06be30 0x7f1bcc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:23.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.337+0000 7f1be9e00640 1 --2- 172.21.15.131:0/2010872298 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f1bcc06be30 0x7f1bcc06e2f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc006fd0 tx=0x7f1bdc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:23.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.340+0000 7f1be2000640 1 -- 172.21.15.131:0/2010872298 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f1bec159c00 con 0x7f1bec0b38a0 2024-05-26T22:51:23.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.578+0000 7f1bf0a00640 1 -- 172.21.15.131:0/2010872298 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1bec0cf9c0 con 0x7f1bec0b38a0 2024-05-26T22:51:23.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.580+0000 7f1be2000640 1 -- 172.21.15.131:0/2010872298 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+5230 (secure 0 0 0) 0x7f1bd8058d30 con 0x7f1bec0b38a0 2024-05-26T22:51:23.580 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:51:23.580 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":40,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4727,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/3668228040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6833","nonce":3668228040},{"type":"v1","addr":"172.21.15.187:6835","nonce":3668228040}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-26T22:50:27.001665+0000","modified":"2024-05-26T22:51:03.848463+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":45,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5637,"mds_1":4661,"mds_2":4670},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4661":{"gid":4661,"name":"a","rank":1,"incarnation":36,"state":"up:active","state_seq":14,"addr":"172.21.15.131:6837/265040431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6835","nonce":265040431},{"type":"v1","addr":"172.21.15.131:6837","nonce":265040431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4670":{"gid":4670,"name":"b","rank":2,"incarnation":32,"state":"up:active","state_seq":8,"addr":"172.21.15.187:6834/1027783169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1027783169},{"type":"v1","addr":"172.21.15.187:6834","nonce":1027783169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5637":{"gid":5637,"name":"c","rank":0,"incarnation":25,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6836/650770835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":650770835},{"type":"v1","addr":"172.21.15.131:6836","nonce":650770835}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5637,"qdb_cluster":[4661,4670,5637]},"id":3}]} 2024-05-26T22:51:23.580 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 40 2024-05-26T22:51:23.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.582+0000 7f1bf0a00640 1 -- 172.21.15.131:0/2010872298 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f1bcc06be30 msgr2=0x7f1bcc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:23.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.582+0000 7f1bf0a00640 1 --2- 172.21.15.131:0/2010872298 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f1bcc06be30 0x7f1bcc06e2f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc006fd0 tx=0x7f1bdc008040 comp rx=0 tx=0).stop 2024-05-26T22:51:23.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.582+0000 7f1bf0a00640 1 -- 172.21.15.131:0/2010872298 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1bec0b38a0 msgr2=0x7f1bec14d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:23.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.582+0000 7f1bf0a00640 1 --2- 172.21.15.131:0/2010872298 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1bec0b38a0 0x7f1bec14d470 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f1bd800c430 tx=0x7f1bd800c8f0 comp rx=0 tx=0).stop 2024-05-26T22:51:23.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.582+0000 7f1bf0a00640 1 -- 172.21.15.131:0/2010872298 shutdown_connections 2024-05-26T22:51:23.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.582+0000 7f1bf0a00640 1 --2- 172.21.15.131:0/2010872298 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1bec158c00 0x7f1bec0ae2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:23.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.582+0000 7f1bf0a00640 1 --2- 172.21.15.131:0/2010872298 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f1bcc06be30 0x7f1bcc06e2f0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:23.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.582+0000 7f1bf0a00640 1 --2- 172.21.15.131:0/2010872298 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1bec0b38a0 0x7f1bec14d470 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:23.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.582+0000 7f1bf0a00640 1 --2- 172.21.15.131:0/2010872298 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1bec06daf0 0x7f1bec0af550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:23.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.582+0000 7f1bf0a00640 1 -- 172.21.15.131:0/2010872298 >> 172.21.15.131:0/2010872298 conn(0x7f1bec0b1e90 msgr2=0x7f1bec15db20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:23.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.583+0000 7f1bf0a00640 1 -- 172.21.15.131:0/2010872298 shutdown_connections 2024-05-26T22:51:23.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.583+0000 7f1bf0a00640 1 -- 172.21.15.131:0/2010872298 wait complete. 2024-05-26T22:51:23.601 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-26T22:51:23.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.810+0000 7f9c8bc00640 1 Processor -- start 2024-05-26T22:51:23.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.810+0000 7f9c8bc00640 1 -- start start 2024-05-26T22:51:23.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.812+0000 7f9c8bc00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f9c840ad250 0x7f9c840b3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:23.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.812+0000 7f9c8bc00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f9c841569d0 0x7f9c840af790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:23.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.812+0000 7f9c8bc00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c8415cb20 con 0x7f9c8406ee20 2024-05-26T22:51:23.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.812+0000 7f9c8bc00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f9c8415ce00 con 0x7f9c840ad250 2024-05-26T22:51:23.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.812+0000 7f9c8bc00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f9c8415d0e0 con 0x7f9c841569d0 2024-05-26T22:51:23.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.812+0000 7f9c83e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f9c840ad250 0x7f9c840b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:23.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.812+0000 7f9c83e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f9c840ad250 0x7f9c840b3190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45058/0 (socket says 172.21.15.131:45058) 2024-05-26T22:51:23.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.812+0000 7f9c89200640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f9c8406ee20 0x7f9c841562c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:51386/0 (socket says 172.21.15.131:51386) 2024-05-26T22:51:23.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.812+0000 7f9c83e00640 1 -- 172.21.15.131:0/1984639881 learned_addr learned my addr 172.21.15.131:0/1984639881 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:23.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.812+0000 7f9c83e00640 1 -- 172.21.15.131:0/1984639881 >> v2:172.21.15.187:3300/0 conn(0x7f9c841569d0 msgr2=0x7f9c840af790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:23.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.812+0000 7f9c83e00640 1 --2- 172.21.15.131:0/1984639881 >> v2:172.21.15.187:3300/0 conn(0x7f9c841569d0 0x7f9c840af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:23.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.813+0000 7f9c83e00640 1 -- 172.21.15.131:0/1984639881 >> v1:172.21.15.131:6789/0 conn(0x7f9c8406ee20 legacy=0x7f9c841562c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:23.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.813+0000 7f9c83e00640 1 -- 172.21.15.131:0/1984639881 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c8415d3c0 con 0x7f9c840ad250 2024-05-26T22:51:23.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.813+0000 7f9c83400640 1 -- 172.21.15.131:0/1984639881 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 422568533 0 0) 0x7f9c8415cb20 con 0x7f9c8406ee20 2024-05-26T22:51:23.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.814+0000 7f9c83e00640 1 --2- 172.21.15.131:0/1984639881 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c840ad250 0x7f9c840b3190 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f9c840adf50 tx=0x7f9c7002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=137407a37f73355e server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:23.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.814+0000 7f9c83400640 1 -- 172.21.15.131:0/1984639881 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c70037070 con 0x7f9c840ad250 2024-05-26T22:51:23.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.814+0000 7f9c83400640 1 -- 172.21.15.131:0/1984639881 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c70032440 con 0x7f9c840ad250 2024-05-26T22:51:23.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.814+0000 7f9c83400640 1 -- 172.21.15.131:0/1984639881 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c70032810 con 0x7f9c840ad250 2024-05-26T22:51:23.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.814+0000 7f9c8bc00640 1 -- 172.21.15.131:0/1984639881 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c840ad250 msgr2=0x7f9c840b3190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:23.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.814+0000 7f9c8bc00640 1 --2- 172.21.15.131:0/1984639881 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c840ad250 0x7f9c840b3190 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f9c840adf50 tx=0x7f9c7002a1e0 comp rx=0 tx=0).stop 2024-05-26T22:51:23.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.815+0000 7f9c8bc00640 1 -- 172.21.15.131:0/1984639881 shutdown_connections 2024-05-26T22:51:23.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.815+0000 7f9c8bc00640 1 --2- 172.21.15.131:0/1984639881 >> v2:172.21.15.187:3300/0 conn(0x7f9c841569d0 0x7f9c840af790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:23.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.815+0000 7f9c8bc00640 1 --2- 172.21.15.131:0/1984639881 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c840ad250 0x7f9c840b3190 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:23.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.815+0000 7f9c8bc00640 1 -- 172.21.15.131:0/1984639881 >> 172.21.15.131:0/1984639881 conn(0x7f9c840b1e90 msgr2=0x7f9c840b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:23.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.815+0000 7f9c8bc00640 1 -- 172.21.15.131:0/1984639881 shutdown_connections 2024-05-26T22:51:23.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.815+0000 7f9c8bc00640 1 -- 172.21.15.131:0/1984639881 wait complete. 2024-05-26T22:51:23.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.816+0000 7f9c8bc00640 1 Processor -- start 2024-05-26T22:51:23.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.817+0000 7f9c8bc00640 1 -- start start 2024-05-26T22:51:23.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.818+0000 7f9c8bc00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9c8406ee20 0x7f9c8415e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:23.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.818+0000 7f9c8bc00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9c840ad250 0x7f9c841e5d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:23.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.818+0000 7f9c8bc00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c841569d0 0x7f9c84146e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:23.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.818+0000 7f9c8bc00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f9c8415c900 con 0x7f9c841569d0 2024-05-26T22:51:23.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.818+0000 7f9c8bc00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9c8415c780 con 0x7f9c840ad250 2024-05-26T22:51:23.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.818+0000 7f9c8bc00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9c8406f200 con 0x7f9c8406ee20 2024-05-26T22:51:23.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.818+0000 7f9c89c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c841569d0 0x7f9c84146e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:23.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.818+0000 7f9c89c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c841569d0 0x7f9c84146e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45064/0 (socket says 172.21.15.131:45064) 2024-05-26T22:51:23.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.818+0000 7f9c89c00640 1 -- 172.21.15.131:0/360915086 learned_addr learned my addr 172.21.15.131:0/360915086 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:23.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.818+0000 7f9c83e00640 1 --2- 172.21.15.131:0/360915086 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9c840ad250 0x7f9c841e5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:23.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.818+0000 7f9c89c00640 1 -- 172.21.15.131:0/360915086 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9c8406ee20 msgr2=0x7f9c8415e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:23.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.818+0000 7f9c89c00640 1 --2- 172.21.15.131:0/360915086 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9c8406ee20 0x7f9c8415e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:23.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.818+0000 7f9c89c00640 1 -- 172.21.15.131:0/360915086 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9c840ad250 msgr2=0x7f9c841e5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:23.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.818+0000 7f9c89c00640 1 --2- 172.21.15.131:0/360915086 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9c840ad250 0x7f9c841e5d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:23.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.818+0000 7f9c89c00640 1 -- 172.21.15.131:0/360915086 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c841e66e0 con 0x7f9c841569d0 2024-05-26T22:51:23.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.819+0000 7f9c89c00640 1 --2- 172.21.15.131:0/360915086 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c841569d0 0x7f9c84146e80 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c00ba10 tx=0x7f9c7c00bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:23.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.819+0000 7f9c80c00640 1 -- 172.21.15.131:0/360915086 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c7c019070 con 0x7f9c841569d0 2024-05-26T22:51:23.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.819+0000 7f9c80c00640 1 -- 172.21.15.131:0/360915086 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c7c00cb40 con 0x7f9c841569d0 2024-05-26T22:51:23.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.819+0000 7f9c80c00640 1 -- 172.21.15.131:0/360915086 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c7c014440 con 0x7f9c841569d0 2024-05-26T22:51:23.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.819+0000 7f9c8bc00640 1 -- 172.21.15.131:0/360915086 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c841e69c0 con 0x7f9c841569d0 2024-05-26T22:51:23.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.819+0000 7f9c8bc00640 1 -- 172.21.15.131:0/360915086 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c841e7260 con 0x7f9c841569d0 2024-05-26T22:51:23.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.820+0000 7f9c8bc00640 1 -- 172.21.15.131:0/360915086 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c841e6db0 con 0x7f9c841569d0 2024-05-26T22:51:23.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.821+0000 7f9c80c00640 1 -- 172.21.15.131:0/360915086 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9c7c01c070 con 0x7f9c841569d0 2024-05-26T22:51:23.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.822+0000 7f9c80c00640 1 --2- 172.21.15.131:0/360915086 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9c6406be30 0x7f9c6406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:23.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.822+0000 7f9c80c00640 1 -- 172.21.15.131:0/360915086 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5853+0+0 (secure 0 0 0) 0x7f9c7c08abe0 con 0x7f9c841569d0 2024-05-26T22:51:23.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.822+0000 7f9c89200640 1 --2- 172.21.15.131:0/360915086 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9c6406be30 0x7f9c6406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:23.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.823+0000 7f9c89200640 1 --2- 172.21.15.131:0/360915086 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9c6406be30 0x7f9c6406e2f0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f9c78007fd0 tx=0x7f9c78007920 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:23.826 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:23.827+0000 7f9c80c00640 1 -- 172.21.15.131:0/360915086 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f9c7c0594f0 con 0x7f9c841569d0 2024-05-26T22:51:24.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.034+0000 7f9c8bc00640 1 -- 172.21.15.131:0/360915086 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9c840694e0 con 0x7f9c841569d0 2024-05-26T22:51:24.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.037+0000 7f9c80c00640 1 -- 172.21.15.131:0/360915086 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) ==== 74+0+16860 (secure 0 0 0) 0x7f9c7c058df0 con 0x7f9c841569d0 2024-05-26T22:51:24.037 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:51:24.037 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":47,"fsid":"46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f","created":"2024-05-26T22:47:57.288905+0000","modified":"2024-05-26T22:51:05.195752+0000","last_up_change":"2024-05-26T22:48:09.602207+0000","last_in_change":"2024-05-26T22:48:00.687341+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-26T22:48:10.463339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-05-26T22:50:24.793406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-05-26T22:50:26.041636+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"814eaf62-a18d-494b-8e27-fd8fb88adea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6801","nonce":2350940051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6803","nonce":2350940051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6807","nonce":2350940051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6805","nonce":2350940051}]},"public_addr":"172.21.15.131:6801/2350940051","cluster_addr":"172.21.15.131:6803/2350940051","heartbeat_back_addr":"172.21.15.131:6807/2350940051","heartbeat_front_addr":"172.21.15.131:6805/2350940051","state":["exists","up"]},{"osd":1,"uuid":"42e66287-f6c6-4e26-af3c-6dfaa3536d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6825","nonce":1525528247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6827","nonce":1525528247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6831","nonce":1525528247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6829","nonce":1525528247}]},"public_addr":"172.21.15.131:6825/1525528247","cluster_addr":"172.21.15.131:6827/1525528247","heartbeat_back_addr":"172.21.15.131:6831/1525528247","heartbeat_front_addr":"172.21.15.131:6829/1525528247","state":["exists","up"]},{"osd":2,"uuid":"0da34673-04a2-4fd0-b57a-0ff2b679b043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6817","nonce":3627842210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6819","nonce":3627842210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6823","nonce":3627842210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6821","nonce":3627842210}]},"public_addr":"172.21.15.131:6817/3627842210","cluster_addr":"172.21.15.131:6819/3627842210","heartbeat_back_addr":"172.21.15.131:6823/3627842210","heartbeat_front_addr":"172.21.15.131:6821/3627842210","state":["exists","up"]},{"osd":3,"uuid":"cf117a27-d6b2-43d3-ae52-b47988bf8eed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6809","nonce":350790591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6811","nonce":350790591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6815","nonce":350790591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6813","nonce":350790591}]},"public_addr":"172.21.15.131:6809/350790591","cluster_addr":"172.21.15.131:6811/350790591","heartbeat_back_addr":"172.21.15.131:6815/350790591","heartbeat_front_addr":"172.21.15.131:6813/350790591","state":["exists","up"]},{"osd":4,"uuid":"a092ccfc-be89-4a84-88d5-0f781fa715eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6816","nonce":244560995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6819","nonce":244560995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6823","nonce":244560995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6821","nonce":244560995}]},"public_addr":"172.21.15.187:6816/244560995","cluster_addr":"172.21.15.187:6819/244560995","heartbeat_back_addr":"172.21.15.187:6823/244560995","heartbeat_front_addr":"172.21.15.187:6821/244560995","state":["exists","up"]},{"osd":5,"uuid":"7513ba69-dd00-4a35-b190-9e8cc3b2bdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6801","nonce":324619398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6803","nonce":324619398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6812","nonce":324619398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6807","nonce":324619398}]},"public_addr":"172.21.15.187:6801/324619398","cluster_addr":"172.21.15.187:6803/324619398","heartbeat_back_addr":"172.21.15.187:6812/324619398","heartbeat_front_addr":"172.21.15.187:6807/324619398","state":["exists","up"]},{"osd":6,"uuid":"50081810-c093-4d2f-a4d7-b7c695c5400d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6806","nonce":1391347786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6809","nonce":1391347786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6818","nonce":1391347786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6813","nonce":1391347786}]},"public_addr":"172.21.15.187:6806/1391347786","cluster_addr":"172.21.15.187:6809/1391347786","heartbeat_back_addr":"172.21.15.187:6818/1391347786","heartbeat_front_addr":"172.21.15.187:6813/1391347786","state":["exists","up"]},{"osd":7,"uuid":"775dae05-54f5-4ad4-bbce-92863c6ed2ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6825","nonce":1220065622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6827","nonce":1220065622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6831","nonce":1220065622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6829","nonce":1220065622}]},"public_addr":"172.21.15.187:6825/1220065622","cluster_addr":"172.21.15.187:6827/1220065622","heartbeat_back_addr":"172.21.15.187:6831/1220065622","heartbeat_front_addr":"172.21.15.187:6829/1220065622","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:05.983520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.299516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.067323+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.307748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.018742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.258775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.010912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.295355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.36","mappings":[{"from":0,"to":5}]},{"pgid":"7.38","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.187:6835/3429891101":"2024-05-27T22:50:59.823337+0000","172.21.15.187:6833/3429891101":"2024-05-27T22:50:59.823337+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-26T22:51:24.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.040+0000 7f9c8bc00640 1 -- 172.21.15.131:0/360915086 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9c6406be30 msgr2=0x7f9c6406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:24.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.040+0000 7f9c8bc00640 1 --2- 172.21.15.131:0/360915086 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9c6406be30 0x7f9c6406e2f0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f9c78007fd0 tx=0x7f9c78007920 comp rx=0 tx=0).stop 2024-05-26T22:51:24.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.040+0000 7f9c8bc00640 1 -- 172.21.15.131:0/360915086 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c841569d0 msgr2=0x7f9c84146e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:24.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.040+0000 7f9c8bc00640 1 --2- 172.21.15.131:0/360915086 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c841569d0 0x7f9c84146e80 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c00ba10 tx=0x7f9c7c00bed0 comp rx=0 tx=0).stop 2024-05-26T22:51:24.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.040+0000 7f9c8bc00640 1 -- 172.21.15.131:0/360915086 shutdown_connections 2024-05-26T22:51:24.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.040+0000 7f9c8bc00640 1 --2- 172.21.15.131:0/360915086 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9c6406be30 0x7f9c6406e2f0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:24.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.040+0000 7f9c8bc00640 1 --2- 172.21.15.131:0/360915086 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c841569d0 0x7f9c84146e80 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:24.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.040+0000 7f9c8bc00640 1 --2- 172.21.15.131:0/360915086 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9c840ad250 0x7f9c841e5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:24.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.040+0000 7f9c8bc00640 1 --2- 172.21.15.131:0/360915086 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9c8406ee20 0x7f9c8415e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:24.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.040+0000 7f9c8bc00640 1 -- 172.21.15.131:0/360915086 >> 172.21.15.131:0/360915086 conn(0x7f9c840b1e90 msgr2=0x7f9c8406d170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:24.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.040+0000 7f9c8bc00640 1 -- 172.21.15.131:0/360915086 shutdown_connections 2024-05-26T22:51:24.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.040+0000 7f9c8bc00640 1 -- 172.21.15.131:0/360915086 wait complete. 2024-05-26T22:51:24.059 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-26T22:51:24.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.254+0000 7f987be00640 1 Processor -- start 2024-05-26T22:51:24.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.255+0000 7f987be00640 1 -- start start 2024-05-26T22:51:24.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.256+0000 7f987be00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f987406f180 0x7f9874072e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:24.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.256+0000 7f987be00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9874072540 0x7f9874161ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:24.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.256+0000 7f987be00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98740af8e0 con 0x7f98740cfe50 2024-05-26T22:51:24.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.256+0000 7f987be00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f98740cfb30 con 0x7f987406f180 2024-05-26T22:51:24.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.256+0000 7f987be00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f98740cf200 con 0x7f9874072540 2024-05-26T22:51:24.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.257+0000 7f9879400640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f987406f180 0x7f9874072e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:24.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.257+0000 7f9879400640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f987406f180 0x7f9874072e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.131:58620/0 (socket says 172.21.15.131:58620) 2024-05-26T22:51:24.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.257+0000 7f9879400640 1 -- 172.21.15.131:0/855576749 learned_addr learned my addr 172.21.15.131:0/855576749 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:24.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.257+0000 7f9878a00640 1 --2- 172.21.15.131:0/855576749 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9874072540 0x7f9874161ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:24.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.258+0000 7f9879400640 1 -- 172.21.15.131:0/855576749 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9874072540 msgr2=0x7f9874161ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:24.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.258+0000 7f9879400640 1 --2- 172.21.15.131:0/855576749 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9874072540 0x7f9874161ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:24.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.258+0000 7f9879400640 1 -- 172.21.15.131:0/855576749 >> v1:172.21.15.187:6789/0 conn(0x7f98740cfe50 legacy=0x7f98740715a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:24.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.258+0000 7f9879400640 1 -- 172.21.15.131:0/855576749 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98740cf4e0 con 0x7f987406f180 2024-05-26T22:51:24.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.259+0000 7f9879400640 1 --2- 172.21.15.131:0/855576749 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f987406f180 0x7f9874072e90 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9868009870 tx=0x7f986802a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=991ee58d92682c02 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:24.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.260+0000 7f9873e00640 1 -- 172.21.15.131:0/855576749 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9868037070 con 0x7f987406f180 2024-05-26T22:51:24.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.260+0000 7f9873e00640 1 -- 172.21.15.131:0/855576749 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9868032440 con 0x7f987406f180 2024-05-26T22:51:24.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.260+0000 7f9873e00640 1 -- 172.21.15.131:0/855576749 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9868032810 con 0x7f987406f180 2024-05-26T22:51:24.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.261+0000 7f987be00640 1 -- 172.21.15.131:0/855576749 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f987406f180 msgr2=0x7f9874072e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:24.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.261+0000 7f987be00640 1 --2- 172.21.15.131:0/855576749 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f987406f180 0x7f9874072e90 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9868009870 tx=0x7f986802a270 comp rx=0 tx=0).stop 2024-05-26T22:51:24.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.261+0000 7f987be00640 1 -- 172.21.15.131:0/855576749 shutdown_connections 2024-05-26T22:51:24.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.261+0000 7f987be00640 1 --2- 172.21.15.131:0/855576749 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9874072540 0x7f9874161ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:24.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.261+0000 7f987be00640 1 --2- 172.21.15.131:0/855576749 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f987406f180 0x7f9874072e90 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:24.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.261+0000 7f987be00640 1 -- 172.21.15.131:0/855576749 >> 172.21.15.131:0/855576749 conn(0x7f987406bc90 msgr2=0x7f98740b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:24.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.261+0000 7f987be00640 1 -- 172.21.15.131:0/855576749 shutdown_connections 2024-05-26T22:51:24.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.261+0000 7f987be00640 1 -- 172.21.15.131:0/855576749 wait complete. 2024-05-26T22:51:24.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.263+0000 7f987be00640 1 Processor -- start 2024-05-26T22:51:24.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.263+0000 7f987be00640 1 -- start start 2024-05-26T22:51:24.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.264+0000 7f987be00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f987406f180 0x7f98740cbc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:24.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.264+0000 7f987be00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9874072540 0x7f98740ae800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:24.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.264+0000 7f987be00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f98740cfe50 0x7f98740aed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:24.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.264+0000 7f987be00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f987406f6e0 con 0x7f98740cfe50 2024-05-26T22:51:24.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.264+0000 7f987be00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f987406f560 con 0x7f9874072540 2024-05-26T22:51:24.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.264+0000 7f987be00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f98740d02e0 con 0x7f987406f180 2024-05-26T22:51:24.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.264+0000 7f9879e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f98740cfe50 0x7f98740aed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:24.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.264+0000 7f9879e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f98740cfe50 0x7f98740aed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45068/0 (socket says 172.21.15.131:45068) 2024-05-26T22:51:24.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.264+0000 7f9879e00640 1 -- 172.21.15.131:0/4224758235 learned_addr learned my addr 172.21.15.131:0/4224758235 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:24.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.265+0000 7f9879e00640 1 -- 172.21.15.131:0/4224758235 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f987406f180 msgr2=0x7f98740cbc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:24.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.265+0000 7f9878a00640 1 --2- 172.21.15.131:0/4224758235 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9874072540 0x7f98740ae800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:24.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.265+0000 7f9879e00640 1 --2- 172.21.15.131:0/4224758235 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f987406f180 0x7f98740cbc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:24.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.265+0000 7f9879e00640 1 -- 172.21.15.131:0/4224758235 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9874072540 msgr2=0x7f98740ae800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:24.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.265+0000 7f9879e00640 1 --2- 172.21.15.131:0/4224758235 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9874072540 0x7f98740ae800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:24.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.265+0000 7f9879e00640 1 -- 172.21.15.131:0/4224758235 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9874161820 con 0x7f98740cfe50 2024-05-26T22:51:24.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.265+0000 7f9879e00640 1 --2- 172.21.15.131:0/4224758235 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f98740cfe50 0x7f98740aed40 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f986c00c6b0 tx=0x7f986c00cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:24.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.265+0000 7f9871600640 1 -- 172.21.15.131:0/4224758235 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f986c003b70 con 0x7f98740cfe50 2024-05-26T22:51:24.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.265+0000 7f9871600640 1 -- 172.21.15.131:0/4224758235 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f986c003d30 con 0x7f98740cfe50 2024-05-26T22:51:24.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.265+0000 7f987be00640 1 -- 172.21.15.131:0/4224758235 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98740c9d40 con 0x7f98740cfe50 2024-05-26T22:51:24.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.265+0000 7f9871600640 1 -- 172.21.15.131:0/4224758235 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f986c005310 con 0x7f98740cfe50 2024-05-26T22:51:24.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.266+0000 7f987be00640 1 -- 172.21.15.131:0/4224758235 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98740cc1c0 con 0x7f98740cfe50 2024-05-26T22:51:24.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.267+0000 7f987be00640 1 -- 172.21.15.131:0/4224758235 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98740695c0 con 0x7f98740cfe50 2024-05-26T22:51:24.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.268+0000 7f9871600640 1 -- 172.21.15.131:0/4224758235 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f986c0054b0 con 0x7f98740cfe50 2024-05-26T22:51:24.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.268+0000 7f9871600640 1 --2- 172.21.15.131:0/4224758235 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f985406be60 0x7f985406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:24.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.269+0000 7f9871600640 1 -- 172.21.15.131:0/4224758235 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5853+0+0 (secure 0 0 0) 0x7f986c08ad50 con 0x7f98740cfe50 2024-05-26T22:51:24.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.269+0000 7f9879400640 1 --2- 172.21.15.131:0/4224758235 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f985406be60 0x7f985406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:24.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.270+0000 7f9879400640 1 --2- 172.21.15.131:0/4224758235 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f985406be60 0x7f985406e320 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f9868034000 tx=0x7f986802c000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:24.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.273+0000 7f9871600640 1 -- 172.21.15.131:0/4224758235 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f986c059660 con 0x7f98740cfe50 2024-05-26T22:51:24.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.516+0000 7f987be00640 1 -- 172.21.15.131:0/4224758235 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f98740b0f60 con 0x7f98740cfe50 2024-05-26T22:51:24.516 INFO:tasks.ceph.mon.a.smithi131.stderr:2024-05-26T22:51:24.517+0000 7fcf0d200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-26T22:51:24.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.520+0000 7f9871600640 1 -- 172.21.15.131:0/4224758235 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v41) ==== 161+0+0 (secure 0 0 0) 0x7f986c058f60 con 0x7f98740cfe50 2024-05-26T22:51:24.520 INFO:teuthology.orchestra.run.smithi131.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-26T22:51:24.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.524+0000 7f987be00640 1 -- 172.21.15.131:0/4224758235 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f985406be60 msgr2=0x7f985406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:24.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.524+0000 7f987be00640 1 --2- 172.21.15.131:0/4224758235 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f985406be60 0x7f985406e320 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f9868034000 tx=0x7f986802c000 comp rx=0 tx=0).stop 2024-05-26T22:51:24.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.524+0000 7f987be00640 1 -- 172.21.15.131:0/4224758235 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f98740cfe50 msgr2=0x7f98740aed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:24.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.524+0000 7f987be00640 1 --2- 172.21.15.131:0/4224758235 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f98740cfe50 0x7f98740aed40 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f986c00c6b0 tx=0x7f986c00cb70 comp rx=0 tx=0).stop 2024-05-26T22:51:24.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.525+0000 7f987be00640 1 -- 172.21.15.131:0/4224758235 shutdown_connections 2024-05-26T22:51:24.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.525+0000 7f987be00640 1 --2- 172.21.15.131:0/4224758235 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f985406be60 0x7f985406e320 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:24.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.525+0000 7f987be00640 1 --2- 172.21.15.131:0/4224758235 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f98740cfe50 0x7f98740aed40 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:24.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.525+0000 7f987be00640 1 --2- 172.21.15.131:0/4224758235 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9874072540 0x7f98740ae800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:24.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.525+0000 7f987be00640 1 --2- 172.21.15.131:0/4224758235 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f987406f180 0x7f98740cbc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:24.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.525+0000 7f987be00640 1 -- 172.21.15.131:0/4224758235 >> 172.21.15.131:0/4224758235 conn(0x7f987406bc90 msgr2=0x7f98740ad740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:24.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.525+0000 7f987be00640 1 -- 172.21.15.131:0/4224758235 shutdown_connections 2024-05-26T22:51:24.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.525+0000 7f987be00640 1 -- 172.21.15.131:0/4224758235 wait complete. 2024-05-26T22:51:24.541 INFO:tasks.ceph.mds.c.smithi131.stderr: -5995> 2024-05-26T22:51:07.515+0000 7efef9c00640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-05-26T22:51:24.545 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-26T22:51:24.606 INFO:tasks.ceph.mds.c.smithi131.stdout:starting mds.c at 2024-05-26T22:51:24.615 INFO:tasks.ceph.mds.a.smithi131.stdout:starting mds.a at 2024-05-26T22:51:24.650 INFO:tasks.ceph.mds.b.smithi187.stdout:starting mds.b at 2024-05-26T22:51:24.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.726+0000 7ffa6f800640 1 Processor -- start 2024-05-26T22:51:24.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.726+0000 7ffa6f800640 1 -- start start 2024-05-26T22:51:24.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.727+0000 7ffa6f800640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7ffa680b5450 0x7ffa680b43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:24.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.727+0000 7ffa6f800640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7ffa680d0460 0x7ffa6806d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:24.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.727+0000 7ffa6f800640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa680b4900 con 0x7ffa6806f420 2024-05-26T22:51:24.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.727+0000 7ffa6f800640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7ffa68160d90 con 0x7ffa680d0460 2024-05-26T22:51:24.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.727+0000 7ffa6f800640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7ffa68160180 con 0x7ffa680b5450 2024-05-26T22:51:24.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.728+0000 7ffa6d800640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7ffa680d0460 0x7ffa6806d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:24.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.728+0000 7ffa5fe00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7ffa6806f420 0x7ffa680cfd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:43032/0 (socket says 172.21.15.131:43032) 2024-05-26T22:51:24.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.728+0000 7ffa5fe00640 1 -- 172.21.15.131:0/4002050382 learned_addr learned my addr 172.21.15.131:0/4002050382 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:51:24.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.728+0000 7ffa6ce00640 1 --2- 172.21.15.131:0/4002050382 >> v2:172.21.15.187:3300/0 conn(0x7ffa680b5450 0x7ffa680b43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:24.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.728+0000 7ffa6d800640 1 -- 172.21.15.131:0/4002050382 >> v2:172.21.15.187:3300/0 conn(0x7ffa680b5450 msgr2=0x7ffa680b43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:24.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.728+0000 7ffa6d800640 1 --2- 172.21.15.131:0/4002050382 >> v2:172.21.15.187:3300/0 conn(0x7ffa680b5450 0x7ffa680b43c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:24.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.728+0000 7ffa6d800640 1 -- 172.21.15.131:0/4002050382 >> v1:172.21.15.131:6789/0 conn(0x7ffa6806f420 legacy=0x7ffa680cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:24.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.728+0000 7ffa6d800640 1 -- 172.21.15.131:0/4002050382 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa68160460 con 0x7ffa680d0460 2024-05-26T22:51:24.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.728+0000 7ffa67e00640 1 -- 172.21.15.131:0/4002050382 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1031230206 0 0) 0x7ffa680b4900 con 0x7ffa6806f420 2024-05-26T22:51:24.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.729+0000 7ffa6d800640 1 --2- 172.21.15.131:0/4002050382 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa680d0460 0x7ffa6806d600 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7ffa60009d90 tx=0x7ffa6002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8492008f08b4bf2 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:24.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.730+0000 7ffa67e00640 1 -- 172.21.15.131:0/4002050382 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa60007740 con 0x7ffa680d0460 2024-05-26T22:51:24.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.730+0000 7ffa67e00640 1 -- 172.21.15.131:0/4002050382 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa600078e0 con 0x7ffa680d0460 2024-05-26T22:51:24.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.730+0000 7ffa67e00640 1 -- 172.21.15.131:0/4002050382 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa60007cb0 con 0x7ffa680d0460 2024-05-26T22:51:24.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.730+0000 7ffa6f800640 1 -- 172.21.15.131:0/4002050382 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa680d0460 msgr2=0x7ffa6806d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:24.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.730+0000 7ffa6f800640 1 --2- 172.21.15.131:0/4002050382 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa680d0460 0x7ffa6806d600 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7ffa60009d90 tx=0x7ffa6002a6a0 comp rx=0 tx=0).stop 2024-05-26T22:51:24.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.730+0000 7ffa6f800640 1 -- 172.21.15.131:0/4002050382 shutdown_connections 2024-05-26T22:51:24.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.730+0000 7ffa6f800640 1 --2- 172.21.15.131:0/4002050382 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa680d0460 0x7ffa6806d600 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:24.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.730+0000 7ffa6f800640 1 --2- 172.21.15.131:0/4002050382 >> v2:172.21.15.187:3300/0 conn(0x7ffa680b5450 0x7ffa680b43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:24.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.730+0000 7ffa6f800640 1 -- 172.21.15.131:0/4002050382 >> 172.21.15.131:0/4002050382 conn(0x7ffa6806bc90 msgr2=0x7ffa680b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:24.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.730+0000 7ffa6f800640 1 -- 172.21.15.131:0/4002050382 shutdown_connections 2024-05-26T22:51:24.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.730+0000 7ffa6f800640 1 -- 172.21.15.131:0/4002050382 wait complete. 2024-05-26T22:51:24.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.732+0000 7ffa6f800640 1 Processor -- start 2024-05-26T22:51:24.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.732+0000 7ffa6f800640 1 -- start start 2024-05-26T22:51:24.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.733+0000 7ffa6f800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffa6806f420 0x7ffa681489d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:24.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.733+0000 7ffa6f800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa680b5450 0x7ffa68156630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:24.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.733+0000 7ffa6f800640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffa680d0460 0x7ffa68156b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:24.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.733+0000 7ffa6f800640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ffa680b5830 con 0x7ffa680b5450 2024-05-26T22:51:24.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.733+0000 7ffa6f800640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ffa68160a20 con 0x7ffa680d0460 2024-05-26T22:51:24.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.733+0000 7ffa6f800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ffa6815fde0 con 0x7ffa6806f420 2024-05-26T22:51:24.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.733+0000 7ffa5fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa680b5450 0x7ffa68156630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:24.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.734+0000 7ffa5fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa680b5450 0x7ffa68156630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48156/0 (socket says 172.21.15.131:48156) 2024-05-26T22:51:24.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.734+0000 7ffa5fe00640 1 -- 172.21.15.131:0/2667562640 learned_addr learned my addr 172.21.15.131:0/2667562640 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:24.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.734+0000 7ffa5fe00640 1 -- 172.21.15.131:0/2667562640 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffa6806f420 msgr2=0x7ffa681489d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:24.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.734+0000 7ffa5fe00640 1 --2- 172.21.15.131:0/2667562640 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffa6806f420 0x7ffa681489d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:24.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.734+0000 7ffa5fe00640 1 -- 172.21.15.131:0/2667562640 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffa680d0460 msgr2=0x7ffa68156b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:24.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.734+0000 7ffa6d800640 1 --2- 172.21.15.131:0/2667562640 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffa680d0460 0x7ffa68156b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:24.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.734+0000 7ffa5fe00640 1 --2- 172.21.15.131:0/2667562640 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffa680d0460 0x7ffa68156b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:24.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.734+0000 7ffa5fe00640 1 -- 172.21.15.131:0/2667562640 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa680ce610 con 0x7ffa680b5450 2024-05-26T22:51:24.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.734+0000 7ffa5fe00640 1 --2- 172.21.15.131:0/2667562640 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa680b5450 0x7ffa68156630 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7ffa5800c430 tx=0x7ffa5800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:24.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.734+0000 7ffa65600640 1 -- 172.21.15.131:0/2667562640 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa580074e0 con 0x7ffa680b5450 2024-05-26T22:51:24.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.735+0000 7ffa65600640 1 -- 172.21.15.131:0/2667562640 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa580047e0 con 0x7ffa680b5450 2024-05-26T22:51:24.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.735+0000 7ffa65600640 1 -- 172.21.15.131:0/2667562640 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa58004bd0 con 0x7ffa680b5450 2024-05-26T22:51:24.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.735+0000 7ffa6f800640 1 -- 172.21.15.131:0/2667562640 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa68154ad0 con 0x7ffa680b5450 2024-05-26T22:51:24.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.735+0000 7ffa6f800640 1 -- 172.21.15.131:0/2667562640 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa68155050 con 0x7ffa680b5450 2024-05-26T22:51:24.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.736+0000 7ffa6f800640 1 -- 172.21.15.131:0/2667562640 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa68074e90 con 0x7ffa680b5450 2024-05-26T22:51:24.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.737+0000 7ffa65600640 1 -- 172.21.15.131:0/2667562640 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ffa58011070 con 0x7ffa680b5450 2024-05-26T22:51:24.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.737+0000 7ffa65600640 1 --2- 172.21.15.131:0/2667562640 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ffa4406be30 0x7ffa4406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:24.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.738+0000 7ffa65600640 1 -- 172.21.15.131:0/2667562640 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 6111+0+0 (secure 0 0 0) 0x7ffa5808ad30 con 0x7ffa680b5450 2024-05-26T22:51:24.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.738+0000 7ffa6ce00640 1 --2- 172.21.15.131:0/2667562640 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ffa4406be30 0x7ffa4406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:24.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.739+0000 7ffa6ce00640 1 --2- 172.21.15.131:0/2667562640 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ffa4406be30 0x7ffa4406e2f0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7ffa500098f0 tx=0x7ffa50008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:24.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.742+0000 7ffa65600640 1 -- 172.21.15.131:0/2667562640 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7ffa580594b0 con 0x7ffa680b5450 2024-05-26T22:51:24.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:24.971+0000 7ffa6f800640 1 -- 172.21.15.131:0/2667562640 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7ffa680695c0 con 0x7ffa680b5450 2024-05-26T22:51:25.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:25.523+0000 7ffa65600640 1 -- 172.21.15.131:0/2667562640 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v42) ==== 104+0+0 (secure 0 0 0) 0x7ffa58058db0 con 0x7ffa680b5450 2024-05-26T22:51:25.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:25.528+0000 7ffa6f800640 1 -- 172.21.15.131:0/2667562640 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ffa4406be30 msgr2=0x7ffa4406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:25.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:25.528+0000 7ffa6f800640 1 --2- 172.21.15.131:0/2667562640 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ffa4406be30 0x7ffa4406e2f0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7ffa500098f0 tx=0x7ffa50008040 comp rx=0 tx=0).stop 2024-05-26T22:51:25.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:25.528+0000 7ffa6f800640 1 -- 172.21.15.131:0/2667562640 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa680b5450 msgr2=0x7ffa68156630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:25.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:25.528+0000 7ffa6f800640 1 --2- 172.21.15.131:0/2667562640 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa680b5450 0x7ffa68156630 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7ffa5800c430 tx=0x7ffa5800c8f0 comp rx=0 tx=0).stop 2024-05-26T22:51:25.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:25.528+0000 7ffa6f800640 1 -- 172.21.15.131:0/2667562640 shutdown_connections 2024-05-26T22:51:25.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:25.528+0000 7ffa6f800640 1 --2- 172.21.15.131:0/2667562640 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ffa4406be30 0x7ffa4406e2f0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:25.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:25.528+0000 7ffa6f800640 1 --2- 172.21.15.131:0/2667562640 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffa680d0460 0x7ffa68156b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:25.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:25.528+0000 7ffa6f800640 1 --2- 172.21.15.131:0/2667562640 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa680b5450 0x7ffa68156630 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:25.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:25.528+0000 7ffa6f800640 1 --2- 172.21.15.131:0/2667562640 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffa6806f420 0x7ffa681489d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:25.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:25.528+0000 7ffa6f800640 1 -- 172.21.15.131:0/2667562640 >> 172.21.15.131:0/2667562640 conn(0x7ffa6806bc90 msgr2=0x7ffa6806e6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:25.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:25.528+0000 7ffa6f800640 1 -- 172.21.15.131:0/2667562640 shutdown_connections 2024-05-26T22:51:25.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:25.528+0000 7ffa6f800640 1 -- 172.21.15.131:0/2667562640 wait complete. 2024-05-26T22:51:25.550 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-26T22:51:25.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:25.751+0000 7f0d6ae00640 1 Processor -- start 2024-05-26T22:51:25.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:25.751+0000 7f0d6ae00640 1 -- start start 2024-05-26T22:51:25.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:25.752+0000 7f0d6ae00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f0d640cee10 0x7f0d640b9d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:25.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:25.752+0000 7f0d6ae00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0d640cf1f0 0x7f0d640b7d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:25.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:25.752+0000 7f0d6ae00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d640b9670 con 0x7f0d640b5bc0 2024-05-26T22:51:25.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:25.752+0000 7f0d6ae00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f0d640b85d0 con 0x7f0d640cee10 2024-05-26T22:51:25.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:25.752+0000 7f0d6ae00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f0d640b88b0 con 0x7f0d640cf1f0 2024-05-26T22:51:25.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:25.753+0000 7f0d69a00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f0d640cee10 0x7f0d640b9d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:25.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:25.753+0000 7f0d6a400640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f0d640b5bc0 0x7f0d640b6050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:43034/0 (socket says 172.21.15.131:43034) 2024-05-26T22:51:25.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:25.753+0000 7f0d6a400640 1 -- 172.21.15.131:0/3160304920 learned_addr learned my addr 172.21.15.131:0/3160304920 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:51:25.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:25.753+0000 7f0d69000640 1 --2- 172.21.15.131:0/3160304920 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0d640cf1f0 0x7f0d640b7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:25.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:25.753+0000 7f0d69a00640 1 -- 172.21.15.131:0/3160304920 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0d640cf1f0 msgr2=0x7f0d640b7d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:25.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:25.753+0000 7f0d69a00640 1 --2- 172.21.15.131:0/3160304920 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0d640cf1f0 0x7f0d640b7d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:25.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:25.753+0000 7f0d69a00640 1 -- 172.21.15.131:0/3160304920 >> v1:172.21.15.131:6789/0 conn(0x7f0d640b5bc0 legacy=0x7f0d640b6050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:25.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:25.753+0000 7f0d69a00640 1 -- 172.21.15.131:0/3160304920 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d640b8b90 con 0x7f0d640cee10 2024-05-26T22:51:25.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:25.753+0000 7f0d63e00640 1 -- 172.21.15.131:0/3160304920 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1529683992 0 0) 0x7f0d5c001490 con 0x7f0d640b5bc0 2024-05-26T22:51:25.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:25.754+0000 7f0d69a00640 1 --2- 172.21.15.131:0/3160304920 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0d640cee10 0x7f0d640b9d60 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f0d580097a0 tx=0x7f0d5802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca75a758baa8093e server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:25.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:25.754+0000 7f0d63e00640 1 -- 172.21.15.131:0/3160304920 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d58037070 con 0x7f0d640cee10 2024-05-26T22:51:25.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:25.754+0000 7f0d63e00640 1 -- 172.21.15.131:0/3160304920 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d58032440 con 0x7f0d640cee10 2024-05-26T22:51:25.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:25.754+0000 7f0d63e00640 1 -- 172.21.15.131:0/3160304920 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d58032810 con 0x7f0d640cee10 2024-05-26T22:51:25.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:25.755+0000 7f0d6ae00640 1 -- 172.21.15.131:0/3160304920 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0d640cee10 msgr2=0x7f0d640b9d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:25.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:25.755+0000 7f0d6ae00640 1 --2- 172.21.15.131:0/3160304920 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0d640cee10 0x7f0d640b9d60 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f0d580097a0 tx=0x7f0d5802a180 comp rx=0 tx=0).stop 2024-05-26T22:51:25.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:25.755+0000 7f0d6ae00640 1 -- 172.21.15.131:0/3160304920 shutdown_connections 2024-05-26T22:51:25.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:25.755+0000 7f0d6ae00640 1 --2- 172.21.15.131:0/3160304920 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0d640cf1f0 0x7f0d640b7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:25.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:25.755+0000 7f0d6ae00640 1 --2- 172.21.15.131:0/3160304920 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0d640cee10 0x7f0d640b9d60 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:25.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:25.755+0000 7f0d6ae00640 1 -- 172.21.15.131:0/3160304920 >> 172.21.15.131:0/3160304920 conn(0x7f0d64072e60 msgr2=0x7f0d64073270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:25.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:25.755+0000 7f0d6ae00640 1 -- 172.21.15.131:0/3160304920 shutdown_connections 2024-05-26T22:51:25.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:25.755+0000 7f0d6ae00640 1 -- 172.21.15.131:0/3160304920 wait complete. 2024-05-26T22:51:25.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:25.757+0000 7f0d6ae00640 1 Processor -- start 2024-05-26T22:51:25.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:25.757+0000 7f0d6ae00640 1 -- start start 2024-05-26T22:51:25.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:25.758+0000 7f0d6ae00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0d640b5bc0 0x7f0d641f4280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:25.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:25.758+0000 7f0d6ae00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0d640cee10 0x7f0d640cbc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:25.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:25.758+0000 7f0d6ae00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0d640cf1f0 0x7f0d640cd760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:25.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:25.758+0000 7f0d6ae00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f0d640b82c0 con 0x7f0d640cee10 2024-05-26T22:51:25.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:25.758+0000 7f0d6ae00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0d640b9150 con 0x7f0d640cf1f0 2024-05-26T22:51:25.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:25.758+0000 7f0d6ae00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f0d640cf650 con 0x7f0d640b5bc0 2024-05-26T22:51:25.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:25.758+0000 7f0d69000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0d640cee10 0x7f0d640cbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:25.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:25.758+0000 7f0d69000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0d640cee10 0x7f0d640cbc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48172/0 (socket says 172.21.15.131:48172) 2024-05-26T22:51:25.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:25.758+0000 7f0d69000640 1 -- 172.21.15.131:0/1717853923 learned_addr learned my addr 172.21.15.131:0/1717853923 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:25.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:25.759+0000 7f0d69a00640 1 --2- 172.21.15.131:0/1717853923 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0d640b5bc0 0x7f0d641f4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:25.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:25.759+0000 7f0d69000640 1 -- 172.21.15.131:0/1717853923 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0d640b5bc0 msgr2=0x7f0d641f4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:25.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:25.759+0000 7f0d69000640 1 --2- 172.21.15.131:0/1717853923 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0d640b5bc0 0x7f0d641f4280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:25.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:25.759+0000 7f0d6a400640 1 --2- 172.21.15.131:0/1717853923 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0d640cf1f0 0x7f0d640cd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:25.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:25.759+0000 7f0d69000640 1 -- 172.21.15.131:0/1717853923 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0d640cf1f0 msgr2=0x7f0d640cd760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:25.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:25.759+0000 7f0d69000640 1 --2- 172.21.15.131:0/1717853923 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0d640cf1f0 0x7f0d640cd760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:25.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:25.759+0000 7f0d69000640 1 -- 172.21.15.131:0/1717853923 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d640c7cd0 con 0x7f0d640cee10 2024-05-26T22:51:25.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:25.759+0000 7f0d69a00640 1 --2- 172.21.15.131:0/1717853923 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0d640b5bc0 0x7f0d641f4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:51:25.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:25.759+0000 7f0d69000640 1 --2- 172.21.15.131:0/1717853923 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0d640cee10 0x7f0d640cbc20 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f0d5000d830 tx=0x7f0d5000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:25.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:25.759+0000 7f0d61600640 1 -- 172.21.15.131:0/1717853923 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d5000b6c0 con 0x7f0d640cee10 2024-05-26T22:51:25.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:25.759+0000 7f0d61600640 1 -- 172.21.15.131:0/1717853923 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d50019070 con 0x7f0d640cee10 2024-05-26T22:51:25.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:25.759+0000 7f0d61600640 1 -- 172.21.15.131:0/1717853923 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d5000ba80 con 0x7f0d640cee10 2024-05-26T22:51:25.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:25.759+0000 7f0d6ae00640 1 -- 172.21.15.131:0/1717853923 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d640c7fb0 con 0x7f0d640cee10 2024-05-26T22:51:25.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:25.760+0000 7f0d6ae00640 1 -- 172.21.15.131:0/1717853923 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d640c8570 con 0x7f0d640cee10 2024-05-26T22:51:25.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:25.761+0000 7f0d6ae00640 1 -- 172.21.15.131:0/1717853923 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d640694e0 con 0x7f0d640cee10 2024-05-26T22:51:25.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:25.762+0000 7f0d61600640 1 -- 172.21.15.131:0/1717853923 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0d5001e460 con 0x7f0d640cee10 2024-05-26T22:51:25.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:25.763+0000 7f0d61600640 1 --2- 172.21.15.131:0/1717853923 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f0d4406be30 0x7f0d4406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:25.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:25.763+0000 7f0d61600640 1 -- 172.21.15.131:0/1717853923 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 6111+0+0 (secure 0 0 0) 0x7f0d5008b760 con 0x7f0d640cee10 2024-05-26T22:51:25.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:25.763+0000 7f0d69a00640 1 --2- 172.21.15.131:0/1717853923 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f0d4406be30 0x7f0d4406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:25.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:25.764+0000 7f0d69a00640 1 --2- 172.21.15.131:0/1717853923 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f0d4406be30 0x7f0d4406e2f0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f0d58034000 tx=0x7f0d5802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:25.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:25.768+0000 7f0d61600640 1 -- 172.21.15.131:0/1717853923 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f0d50059ee0 con 0x7f0d640cee10 2024-05-26T22:51:25.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:25.966+0000 7f0d6ae00640 1 -- 172.21.15.131:0/1717853923 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f0d640b3880 con 0x7f0d640cee10 2024-05-26T22:51:26.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:26.527+0000 7f0d61600640 1 -- 172.21.15.131:0/1717853923 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v49) ==== 181+0+0 (secure 0 0 0) 0x7f0d500597e0 con 0x7f0d640cee10 2024-05-26T22:51:26.528 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_metadata' removed 2024-05-26T22:51:26.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:26.531+0000 7f0d6ae00640 1 -- 172.21.15.131:0/1717853923 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f0d4406be30 msgr2=0x7f0d4406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:26.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:26.531+0000 7f0d6ae00640 1 --2- 172.21.15.131:0/1717853923 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f0d4406be30 0x7f0d4406e2f0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f0d58034000 tx=0x7f0d5802e000 comp rx=0 tx=0).stop 2024-05-26T22:51:26.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:26.531+0000 7f0d6ae00640 1 -- 172.21.15.131:0/1717853923 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0d640cee10 msgr2=0x7f0d640cbc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:26.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:26.531+0000 7f0d6ae00640 1 --2- 172.21.15.131:0/1717853923 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0d640cee10 0x7f0d640cbc20 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f0d5000d830 tx=0x7f0d5000dcf0 comp rx=0 tx=0).stop 2024-05-26T22:51:26.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:26.531+0000 7f0d6ae00640 1 -- 172.21.15.131:0/1717853923 shutdown_connections 2024-05-26T22:51:26.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:26.531+0000 7f0d6ae00640 1 --2- 172.21.15.131:0/1717853923 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f0d4406be30 0x7f0d4406e2f0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:26.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:26.531+0000 7f0d6ae00640 1 --2- 172.21.15.131:0/1717853923 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0d640cf1f0 0x7f0d640cd760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:26.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:26.531+0000 7f0d6ae00640 1 --2- 172.21.15.131:0/1717853923 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0d640cee10 0x7f0d640cbc20 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:26.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:26.531+0000 7f0d6ae00640 1 --2- 172.21.15.131:0/1717853923 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0d640b5bc0 0x7f0d641f4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:26.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:26.531+0000 7f0d6ae00640 1 -- 172.21.15.131:0/1717853923 >> 172.21.15.131:0/1717853923 conn(0x7f0d64072e60 msgr2=0x7f0d640b3aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:26.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:26.531+0000 7f0d6ae00640 1 -- 172.21.15.131:0/1717853923 shutdown_connections 2024-05-26T22:51:26.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:26.531+0000 7f0d6ae00640 1 -- 172.21.15.131:0/1717853923 wait complete. 2024-05-26T22:51:26.555 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-26T22:51:26.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:26.752+0000 7f6580200640 1 Processor -- start 2024-05-26T22:51:26.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:26.752+0000 7f6580200640 1 -- start start 2024-05-26T22:51:26.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:26.754+0000 7f6580200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f65780d0a70 0x7f6578072250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:26.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:26.754+0000 7f6580200640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65780ccb70 con 0x7f657806e860 2024-05-26T22:51:26.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:26.754+0000 7f6580200640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65780cce50 con 0x7f65780cf8f0 2024-05-26T22:51:26.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:26.754+0000 7f6580200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f65780cd130 con 0x7f65780d0a70 2024-05-26T22:51:26.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:26.755+0000 7f657e200640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f657806e860 0x7f657806ecf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:43036/0 (socket says 172.21.15.131:43036) 2024-05-26T22:51:26.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:26.755+0000 7f657e200640 1 -- 172.21.15.131:0/2638656314 learned_addr learned my addr 172.21.15.131:0/2638656314 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:51:26.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:26.755+0000 7f657ce00640 1 --2- 172.21.15.131:0/2638656314 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f65780d0a70 0x7f6578072250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:26.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:26.755+0000 7f6577e00640 1 -- 172.21.15.131:0/2638656314 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2766256293 0 0) 0x7f65780ccb70 con 0x7f657806e860 2024-05-26T22:51:26.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:26.755+0000 7f6577e00640 1 -- 172.21.15.131:0/2638656314 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6560003520 con 0x7f657806e860 2024-05-26T22:51:26.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:26.755+0000 7f6577e00640 1 -- 172.21.15.131:0/2638656314 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4042532454 0 0) 0x7f6560003520 con 0x7f657806e860 2024-05-26T22:51:26.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:26.755+0000 7f6577e00640 1 -- 172.21.15.131:0/2638656314 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f65780d0a70 msgr2=0x7f6578072250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:26.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:26.755+0000 7f6577e00640 1 --2- 172.21.15.131:0/2638656314 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f65780d0a70 0x7f6578072250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:26.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:26.755+0000 7f6577e00640 1 -- 172.21.15.131:0/2638656314 >> v1:172.21.15.187:6789/0 conn(0x7f65780cf8f0 legacy=0x7f65780d0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:26.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:26.755+0000 7f6577e00640 1 -- 172.21.15.131:0/2638656314 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65780cd410 con 0x7f657806e860 2024-05-26T22:51:26.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:26.755+0000 7f6577e00640 1 -- 172.21.15.131:0/2638656314 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7f65700019f0 con 0x7f657806e860 2024-05-26T22:51:26.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:26.756+0000 7f6577e00640 1 -- 172.21.15.131:0/2638656314 >> v1:172.21.15.131:6789/0 conn(0x7f657806e860 legacy=0x7f657806ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:26.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:26.756+0000 7f6577e00640 1 --2- 172.21.15.131:0/2638656314 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f65600039e0 0x7f6560003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:26.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:26.756+0000 7f6577e00640 1 --2- 172.21.15.131:0/2638656314 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6560004800 0x7f6560004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:26.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:26.756+0000 7f6577e00640 1 --2- 172.21.15.131:0/2638656314 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f65780d0a70 0x7f65600051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:26.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:26.756+0000 7f6577e00640 1 -- 172.21.15.131:0/2638656314 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f65780cd130 con 0x7f65600039e0 2024-05-26T22:51:26.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:26.756+0000 7f6577e00640 1 -- 172.21.15.131:0/2638656314 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f656000baa0 con 0x7f6560004800 2024-05-26T22:51:26.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:26.756+0000 7f6577e00640 1 -- 172.21.15.131:0/2638656314 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f656000bca0 con 0x7f65780d0a70 2024-05-26T22:51:26.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:26.756+0000 7f657d800640 1 --2- 172.21.15.131:0/2638656314 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f65600039e0 0x7f6560003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:26.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:26.756+0000 7f657e200640 1 --2- 172.21.15.131:0/2638656314 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f65780d0a70 0x7f65600051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:26.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:26.756+0000 7f657ce00640 1 --2- 172.21.15.131:0/2638656314 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6560004800 0x7f6560004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:26.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:26.756+0000 7f657d800640 1 -- 172.21.15.131:0/2638656314 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f65780d0a70 msgr2=0x7f65600051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:26.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:26.756+0000 7f657d800640 1 --2- 172.21.15.131:0/2638656314 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f65780d0a70 0x7f65600051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:26.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:26.756+0000 7f657d800640 1 -- 172.21.15.131:0/2638656314 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6560004800 msgr2=0x7f6560004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:26.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:26.756+0000 7f657d800640 1 --2- 172.21.15.131:0/2638656314 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6560004800 0x7f6560004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:26.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:26.756+0000 7f657d800640 1 -- 172.21.15.131:0/2638656314 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6560003520 con 0x7f65600039e0 2024-05-26T22:51:26.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:26.756+0000 7f657e200640 1 --2- 172.21.15.131:0/2638656314 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f65780d0a70 0x7f65600051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:51:26.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:26.757+0000 7f657d800640 1 --2- 172.21.15.131:0/2638656314 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f65600039e0 0x7f6560003e00 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f656c001720 tx=0x7f6560005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=add0b8100c88241b server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:26.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:26.758+0000 7f6577e00640 1 -- 172.21.15.131:0/2638656314 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f656c001370 con 0x7f65600039e0 2024-05-26T22:51:26.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:26.758+0000 7f6577e00640 1 -- 172.21.15.131:0/2638656314 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f656c0052d0 con 0x7f65600039e0 2024-05-26T22:51:26.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:26.758+0000 7f6577e00640 1 -- 172.21.15.131:0/2638656314 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f656c02ab90 con 0x7f65600039e0 2024-05-26T22:51:26.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:26.758+0000 7f6580200640 1 -- 172.21.15.131:0/2638656314 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f65600039e0 msgr2=0x7f6560003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:26.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:26.758+0000 7f6580200640 1 --2- 172.21.15.131:0/2638656314 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f65600039e0 0x7f6560003e00 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f656c001720 tx=0x7f6560005830 comp rx=0 tx=0).stop 2024-05-26T22:51:26.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:26.758+0000 7f657e200640 1 -- 172.21.15.131:0/2638656314 reap_dead start 2024-05-26T22:51:26.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:26.758+0000 7f6580200640 1 -- 172.21.15.131:0/2638656314 shutdown_connections 2024-05-26T22:51:26.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:26.758+0000 7f6580200640 1 -- 172.21.15.131:0/2638656314 >> 172.21.15.131:0/2638656314 conn(0x7f65780b2110 msgr2=0x7f65780b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:26.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:26.758+0000 7f6580200640 1 -- 172.21.15.131:0/2638656314 shutdown_connections 2024-05-26T22:51:26.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:26.758+0000 7f6580200640 1 -- 172.21.15.131:0/2638656314 wait complete. 2024-05-26T22:51:26.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:26.760+0000 7f6580200640 1 Processor -- start 2024-05-26T22:51:26.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:26.760+0000 7f6580200640 1 -- start start 2024-05-26T22:51:26.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:26.761+0000 7f6580200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f65600039e0 0x7f657814fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:26.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:26.761+0000 7f6580200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6578151720 0x7f6578151ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:26.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:26.761+0000 7f6580200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f65780b3e30 0x7f65780b42b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:26.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:26.761+0000 7f6580200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f65780723d0 con 0x7f6578151720 2024-05-26T22:51:26.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:26.761+0000 7f6580200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6578072250 con 0x7f65600039e0 2024-05-26T22:51:26.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:26.761+0000 7f6580200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f6578072550 con 0x7f65780b3e30 2024-05-26T22:51:26.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:26.761+0000 7f657ce00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6578151720 0x7f6578151ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:26.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:26.762+0000 7f657ce00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6578151720 0x7f6578151ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48190/0 (socket says 172.21.15.131:48190) 2024-05-26T22:51:26.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:26.762+0000 7f657ce00640 1 -- 172.21.15.131:0/1698054246 learned_addr learned my addr 172.21.15.131:0/1698054246 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:26.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:26.762+0000 7f657d800640 1 --2- 172.21.15.131:0/1698054246 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f65600039e0 0x7f657814fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:26.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:26.762+0000 7f657e200640 1 --2- 172.21.15.131:0/1698054246 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f65780b3e30 0x7f65780b42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:26.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:26.762+0000 7f657ce00640 1 -- 172.21.15.131:0/1698054246 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f65780b3e30 msgr2=0x7f65780b42b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:26.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:26.762+0000 7f657ce00640 1 --2- 172.21.15.131:0/1698054246 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f65780b3e30 0x7f65780b42b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:26.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:26.762+0000 7f657ce00640 1 -- 172.21.15.131:0/1698054246 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f65600039e0 msgr2=0x7f657814fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:26.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:26.762+0000 7f657ce00640 1 --2- 172.21.15.131:0/1698054246 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f65600039e0 0x7f657814fe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:26.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:26.762+0000 7f657ce00640 1 -- 172.21.15.131:0/1698054246 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6578152270 con 0x7f6578151720 2024-05-26T22:51:26.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:26.762+0000 7f657ce00640 1 --2- 172.21.15.131:0/1698054246 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6578151720 0x7f6578151ba0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f6564010890 tx=0x7f6564010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:26.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:26.762+0000 7f6575600640 1 -- 172.21.15.131:0/1698054246 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f656400f040 con 0x7f6578151720 2024-05-26T22:51:26.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:26.763+0000 7f6575600640 1 -- 172.21.15.131:0/1698054246 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f656400eb00 con 0x7f6578151720 2024-05-26T22:51:26.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:26.763+0000 7f6575600640 1 -- 172.21.15.131:0/1698054246 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f656401c400 con 0x7f6578151720 2024-05-26T22:51:26.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:26.763+0000 7f6580200640 1 -- 172.21.15.131:0/1698054246 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6578152550 con 0x7f6578151720 2024-05-26T22:51:26.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:26.763+0000 7f6580200640 1 -- 172.21.15.131:0/1698054246 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6578158d00 con 0x7f6578151720 2024-05-26T22:51:26.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:26.764+0000 7f6580200640 1 -- 172.21.15.131:0/1698054246 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6578068440 con 0x7f6578151720 2024-05-26T22:51:26.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:26.765+0000 7f6575600640 1 -- 172.21.15.131:0/1698054246 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f656401c5a0 con 0x7f6578151720 2024-05-26T22:51:26.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:26.766+0000 7f6575600640 1 --2- 172.21.15.131:0/1698054246 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f655806be30 0x7f655806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:26.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:26.766+0000 7f6575600640 1 -- 172.21.15.131:0/1698054246 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5673+0+0 (secure 0 0 0) 0x7f656408b310 con 0x7f6578151720 2024-05-26T22:51:26.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:26.766+0000 7f657d800640 1 --2- 172.21.15.131:0/1698054246 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f655806be30 0x7f655806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:26.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:26.767+0000 7f657d800640 1 --2- 172.21.15.131:0/1698054246 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f655806be30 0x7f655806e2f0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f656c032000 tx=0x7f656c02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:26.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:26.770+0000 7f6575600640 1 -- 172.21.15.131:0/1698054246 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f6564018030 con 0x7f6578151720 2024-05-26T22:51:26.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:26.970+0000 7f6580200640 1 -- 172.21.15.131:0/1698054246 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f65780ce970 con 0x7f6578151720 2024-05-26T22:51:27.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:27.538+0000 7f6575600640 1 -- 172.21.15.131:0/1698054246 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v50) ==== 169+0+0 (secure 0 0 0) 0x7f6564059680 con 0x7f6578151720 2024-05-26T22:51:27.538 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_data' removed 2024-05-26T22:51:27.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:27.543+0000 7f6580200640 1 -- 172.21.15.131:0/1698054246 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f655806be30 msgr2=0x7f655806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:27.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:27.543+0000 7f6580200640 1 --2- 172.21.15.131:0/1698054246 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f655806be30 0x7f655806e2f0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f656c032000 tx=0x7f656c02e000 comp rx=0 tx=0).stop 2024-05-26T22:51:27.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:27.543+0000 7f6580200640 1 -- 172.21.15.131:0/1698054246 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6578151720 msgr2=0x7f6578151ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:27.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:27.543+0000 7f6580200640 1 --2- 172.21.15.131:0/1698054246 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6578151720 0x7f6578151ba0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f6564010890 tx=0x7f6564010d50 comp rx=0 tx=0).stop 2024-05-26T22:51:27.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:27.544+0000 7f6580200640 1 -- 172.21.15.131:0/1698054246 shutdown_connections 2024-05-26T22:51:27.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:27.544+0000 7f6580200640 1 --2- 172.21.15.131:0/1698054246 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f65780b3e30 0x7f65780b42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:27.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:27.544+0000 7f6580200640 1 --2- 172.21.15.131:0/1698054246 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f655806be30 0x7f655806e2f0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:27.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:27.544+0000 7f6580200640 1 --2- 172.21.15.131:0/1698054246 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6578151720 0x7f6578151ba0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:27.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:27.544+0000 7f6580200640 1 --2- 172.21.15.131:0/1698054246 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f65600039e0 0x7f657814fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:27.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:27.544+0000 7f6580200640 1 -- 172.21.15.131:0/1698054246 >> 172.21.15.131:0/1698054246 conn(0x7f65780b2110 msgr2=0x7f65780d0330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:27.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:27.548+0000 7f6580200640 1 -- 172.21.15.131:0/1698054246 shutdown_connections 2024-05-26T22:51:27.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:27.548+0000 7f6580200640 1 -- 172.21.15.131:0/1698054246 wait complete. 2024-05-26T22:51:27.575 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_export_ephemeral_random_max 2024-05-26T22:51:27.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:27.761+0000 7fef6b000640 1 Processor -- start 2024-05-26T22:51:27.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:27.762+0000 7fef6b000640 1 -- start start 2024-05-26T22:51:27.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:27.763+0000 7fef6b000640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fef64156660 0x7fef640cfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:27.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:27.764+0000 7fef6b000640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fef6415cf00 0x7fef64144ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:27.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:27.764+0000 7fef6b000640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef6415fcc0 con 0x7fef640d01e0 2024-05-26T22:51:27.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:27.764+0000 7fef6b000640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fef6415d310 con 0x7fef6415cf00 2024-05-26T22:51:27.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:27.764+0000 7fef6b000640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fef6415ffa0 con 0x7fef64156660 2024-05-26T22:51:27.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:27.764+0000 7fef6a600640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fef6415cf00 0x7fef64144ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:27.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:27.764+0000 7fef6a600640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fef6415cf00 0x7fef64144ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48194/0 (socket says 172.21.15.131:48194) 2024-05-26T22:51:27.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:27.764+0000 7fef6a600640 1 -- 172.21.15.131:0/2614937565 learned_addr learned my addr 172.21.15.131:0/2614937565 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:27.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:27.764+0000 7fef69c00640 1 --2- 172.21.15.131:0/2614937565 >> v2:172.21.15.187:3300/0 conn(0x7fef64156660 0x7fef640cfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:27.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:27.765+0000 7fef6a600640 1 -- 172.21.15.131:0/2614937565 >> v2:172.21.15.187:3300/0 conn(0x7fef64156660 msgr2=0x7fef640cfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:27.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:27.765+0000 7fef6a600640 1 --2- 172.21.15.131:0/2614937565 >> v2:172.21.15.187:3300/0 conn(0x7fef64156660 0x7fef640cfca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:27.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:27.765+0000 7fef6a600640 1 -- 172.21.15.131:0/2614937565 >> v1:172.21.15.187:6789/0 conn(0x7fef640d01e0 legacy=0x7fef6415c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:27.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:27.765+0000 7fef6a600640 1 -- 172.21.15.131:0/2614937565 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef64160280 con 0x7fef6415cf00 2024-05-26T22:51:27.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:27.766+0000 7fef6a600640 1 --2- 172.21.15.131:0/2614937565 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fef6415cf00 0x7fef64144ec0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fef640cd790 tx=0x7fef5c02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a658e168910c5987 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:27.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:27.766+0000 7fef63e00640 1 -- 172.21.15.131:0/2614937565 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef5c036070 con 0x7fef6415cf00 2024-05-26T22:51:27.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:27.766+0000 7fef63e00640 1 -- 172.21.15.131:0/2614937565 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef5c031600 con 0x7fef6415cf00 2024-05-26T22:51:27.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:27.766+0000 7fef63e00640 1 -- 172.21.15.131:0/2614937565 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef5c0319d0 con 0x7fef6415cf00 2024-05-26T22:51:27.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:27.767+0000 7fef6b000640 1 -- 172.21.15.131:0/2614937565 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fef6415cf00 msgr2=0x7fef64144ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:27.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:27.767+0000 7fef6b000640 1 --2- 172.21.15.131:0/2614937565 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fef6415cf00 0x7fef64144ec0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fef640cd790 tx=0x7fef5c02a6f0 comp rx=0 tx=0).stop 2024-05-26T22:51:27.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:27.767+0000 7fef6b000640 1 -- 172.21.15.131:0/2614937565 shutdown_connections 2024-05-26T22:51:27.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:27.767+0000 7fef6b000640 1 --2- 172.21.15.131:0/2614937565 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fef6415cf00 0x7fef64144ec0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:27.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:27.767+0000 7fef6b000640 1 --2- 172.21.15.131:0/2614937565 >> v2:172.21.15.187:3300/0 conn(0x7fef64156660 0x7fef640cfca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:27.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:27.767+0000 7fef6b000640 1 -- 172.21.15.131:0/2614937565 >> 172.21.15.131:0/2614937565 conn(0x7fef640cc7f0 msgr2=0x7fef640ccbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:27.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:27.767+0000 7fef6b000640 1 -- 172.21.15.131:0/2614937565 shutdown_connections 2024-05-26T22:51:27.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:27.767+0000 7fef6b000640 1 -- 172.21.15.131:0/2614937565 wait complete. 2024-05-26T22:51:27.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:27.768+0000 7fef6b000640 1 Processor -- start 2024-05-26T22:51:27.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:27.769+0000 7fef6b000640 1 -- start start 2024-05-26T22:51:27.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:27.770+0000 7fef6b000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fef640d01e0 0x7fef6414f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:27.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:27.770+0000 7fef6b000640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fef64156660 0x7fef64141700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:27.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:27.770+0000 7fef6b000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fef6415cf00 0x7fef6414dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:27.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:27.770+0000 7fef6b000640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fef64156a40 con 0x7fef640d01e0 2024-05-26T22:51:27.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:27.770+0000 7fef6b000640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fef6415d540 con 0x7fef64156660 2024-05-26T22:51:27.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:27.770+0000 7fef6b000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fef64160a80 con 0x7fef6415cf00 2024-05-26T22:51:27.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:27.770+0000 7fef69c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fef640d01e0 0x7fef6414f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:27.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:27.770+0000 7fef69c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fef640d01e0 0x7fef6414f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48204/0 (socket says 172.21.15.131:48204) 2024-05-26T22:51:27.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:27.770+0000 7fef69c00640 1 -- 172.21.15.131:0/2265390550 learned_addr learned my addr 172.21.15.131:0/2265390550 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:27.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:27.770+0000 7fef6a600640 1 --2- 172.21.15.131:0/2265390550 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fef6415cf00 0x7fef6414dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:27.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:27.770+0000 7fef69c00640 1 -- 172.21.15.131:0/2265390550 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fef6415cf00 msgr2=0x7fef6414dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:27.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:27.770+0000 7fef69200640 1 --2- 172.21.15.131:0/2265390550 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fef64156660 0x7fef64141700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:27.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:27.770+0000 7fef69c00640 1 --2- 172.21.15.131:0/2265390550 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fef6415cf00 0x7fef6414dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:27.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:27.770+0000 7fef69c00640 1 -- 172.21.15.131:0/2265390550 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fef64156660 msgr2=0x7fef64141700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:27.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:27.770+0000 7fef69c00640 1 --2- 172.21.15.131:0/2265390550 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fef64156660 0x7fef64141700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:27.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:27.770+0000 7fef69c00640 1 -- 172.21.15.131:0/2265390550 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef6414e7c0 con 0x7fef640d01e0 2024-05-26T22:51:27.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:27.770+0000 7fef6a600640 1 --2- 172.21.15.131:0/2265390550 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fef6415cf00 0x7fef6414dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:51:27.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:27.770+0000 7fef69200640 1 --2- 172.21.15.131:0/2265390550 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fef64156660 0x7fef64141700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:51:27.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:27.771+0000 7fef69c00640 1 --2- 172.21.15.131:0/2265390550 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fef640d01e0 0x7fef6414f860 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fef5800d830 tx=0x7fef5800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:27.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:27.771+0000 7fef61600640 1 -- 172.21.15.131:0/2265390550 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef5800b6c0 con 0x7fef640d01e0 2024-05-26T22:51:27.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:27.771+0000 7fef61600640 1 -- 172.21.15.131:0/2265390550 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef58019070 con 0x7fef640d01e0 2024-05-26T22:51:27.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:27.771+0000 7fef61600640 1 -- 172.21.15.131:0/2265390550 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef5800ba80 con 0x7fef640d01e0 2024-05-26T22:51:27.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:27.771+0000 7fef6b000640 1 -- 172.21.15.131:0/2265390550 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef6414eaa0 con 0x7fef640d01e0 2024-05-26T22:51:27.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:27.771+0000 7fef6b000640 1 -- 172.21.15.131:0/2265390550 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef6414f060 con 0x7fef640d01e0 2024-05-26T22:51:27.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:27.772+0000 7fef6b000640 1 -- 172.21.15.131:0/2265390550 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef640694e0 con 0x7fef640d01e0 2024-05-26T22:51:27.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:27.773+0000 7fef61600640 1 -- 172.21.15.131:0/2265390550 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fef5801e460 con 0x7fef640d01e0 2024-05-26T22:51:27.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:27.774+0000 7fef61600640 1 --2- 172.21.15.131:0/2265390550 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fef4406be30 0x7fef4406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:27.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:27.774+0000 7fef61600640 1 -- 172.21.15.131:0/2265390550 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5201+0+0 (secure 0 0 0) 0x7fef5808b3f0 con 0x7fef640d01e0 2024-05-26T22:51:27.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:27.775+0000 7fef69200640 1 --2- 172.21.15.131:0/2265390550 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fef4406be30 0x7fef4406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:27.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:27.775+0000 7fef69200640 1 --2- 172.21.15.131:0/2265390550 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fef4406be30 0x7fef4406e2f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fef641547b0 tx=0x7fef50008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:27.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:27.779+0000 7fef61600640 1 -- 172.21.15.131:0/2265390550 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fef58059ef0 con 0x7fef640d01e0 2024-05-26T22:51:27.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:27.989+0000 7fef6b000640 1 -- 172.21.15.131:0/2265390550 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_export_ephemeral_random_max"} v 0) -- 0x7fef6406ede0 con 0x7fef640d01e0 2024-05-26T22:51:27.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:27.997+0000 7fef61600640 1 -- 172.21.15.131:0/2265390550 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_export_ephemeral_random_max"}]=0 v23) ==== 114+0+0 (secure 0 0 0) 0x7fef580597f0 con 0x7fef640d01e0 2024-05-26T22:51:28.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.000+0000 7fef6b000640 1 -- 172.21.15.131:0/2265390550 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fef4406be30 msgr2=0x7fef4406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:28.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.000+0000 7fef6b000640 1 --2- 172.21.15.131:0/2265390550 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fef4406be30 0x7fef4406e2f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fef641547b0 tx=0x7fef50008040 comp rx=0 tx=0).stop 2024-05-26T22:51:28.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.000+0000 7fef6b000640 1 -- 172.21.15.131:0/2265390550 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fef640d01e0 msgr2=0x7fef6414f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:28.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.000+0000 7fef6b000640 1 --2- 172.21.15.131:0/2265390550 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fef640d01e0 0x7fef6414f860 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fef5800d830 tx=0x7fef5800dcf0 comp rx=0 tx=0).stop 2024-05-26T22:51:28.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.000+0000 7fef6b000640 1 -- 172.21.15.131:0/2265390550 shutdown_connections 2024-05-26T22:51:28.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.000+0000 7fef6b000640 1 --2- 172.21.15.131:0/2265390550 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fef6415cf00 0x7fef6414dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:28.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.000+0000 7fef6b000640 1 --2- 172.21.15.131:0/2265390550 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fef4406be30 0x7fef4406e2f0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:28.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.000+0000 7fef6b000640 1 --2- 172.21.15.131:0/2265390550 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fef64156660 0x7fef64141700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:28.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.000+0000 7fef6b000640 1 --2- 172.21.15.131:0/2265390550 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fef640d01e0 0x7fef6414f860 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:28.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.000+0000 7fef6b000640 1 -- 172.21.15.131:0/2265390550 >> 172.21.15.131:0/2265390550 conn(0x7fef640cc7f0 msgr2=0x7fef64071250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:28.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.000+0000 7fef6b000640 1 -- 172.21.15.131:0/2265390550 shutdown_connections 2024-05-26T22:51:28.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.000+0000 7fef6b000640 1 -- 172.21.15.131:0/2265390550 wait complete. 2024-05-26T22:51:28.019 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_export_ephemeral_random 2024-05-26T22:51:28.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.226+0000 7f9f50a00640 1 Processor -- start 2024-05-26T22:51:28.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.227+0000 7f9f50a00640 1 -- start start 2024-05-26T22:51:28.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.228+0000 7f9f50a00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f9f4c15ec80 0x7f9f4c0cedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:28.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.228+0000 7f9f50a00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f4c15f140 con 0x7f9f4c1556e0 2024-05-26T22:51:28.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.228+0000 7f9f50a00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f4c15f420 con 0x7f9f4c0cf2f0 2024-05-26T22:51:28.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.228+0000 7f9f50a00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f9f4c0cf760 con 0x7f9f4c15ec80 2024-05-26T22:51:28.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.229+0000 7f9f49e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f9f4c15ec80 0x7f9f4c0cedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:28.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.229+0000 7f9f4a800640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f9f4c1556e0 0x7f9f4c15c0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:43050/0 (socket says 172.21.15.131:43050) 2024-05-26T22:51:28.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.229+0000 7f9f4a800640 1 -- 172.21.15.131:0/4142971465 learned_addr learned my addr 172.21.15.131:0/4142971465 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:51:28.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.229+0000 7f9f49e00640 1 -- 172.21.15.131:0/4142971465 >> v1:172.21.15.187:6789/0 conn(0x7f9f4c0cf2f0 legacy=0x7f9f4c143f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:28.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.229+0000 7f9f49e00640 1 -- 172.21.15.131:0/4142971465 >> v1:172.21.15.131:6789/0 conn(0x7f9f4c1556e0 legacy=0x7f9f4c15c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:28.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.229+0000 7f9f49e00640 1 -- 172.21.15.131:0/4142971465 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f4c15f700 con 0x7f9f4c15ec80 2024-05-26T22:51:28.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.229+0000 7f9f49400640 1 -- 172.21.15.131:0/4142971465 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1466364145 0 0) 0x7f9f4c15f140 con 0x7f9f4c1556e0 2024-05-26T22:51:28.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.230+0000 7f9f49e00640 1 --2- 172.21.15.131:0/4142971465 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f4c15ec80 0x7f9f4c0cedb0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f9f380097a0 tx=0x7f9f3802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df115b41dcba9c61 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:28.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.230+0000 7f9f49400640 1 -- 172.21.15.131:0/4142971465 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f38037070 con 0x7f9f4c15ec80 2024-05-26T22:51:28.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.230+0000 7f9f49400640 1 -- 172.21.15.131:0/4142971465 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f38032440 con 0x7f9f4c15ec80 2024-05-26T22:51:28.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.230+0000 7f9f49400640 1 -- 172.21.15.131:0/4142971465 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f38032810 con 0x7f9f4c15ec80 2024-05-26T22:51:28.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.231+0000 7f9f50a00640 1 -- 172.21.15.131:0/4142971465 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f4c15ec80 msgr2=0x7f9f4c0cedb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:28.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.231+0000 7f9f50a00640 1 --2- 172.21.15.131:0/4142971465 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f4c15ec80 0x7f9f4c0cedb0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f9f380097a0 tx=0x7f9f3802a1e0 comp rx=0 tx=0).stop 2024-05-26T22:51:28.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.231+0000 7f9f50a00640 1 -- 172.21.15.131:0/4142971465 shutdown_connections 2024-05-26T22:51:28.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.231+0000 7f9f50a00640 1 --2- 172.21.15.131:0/4142971465 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f4c15ec80 0x7f9f4c0cedb0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:28.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.231+0000 7f9f50a00640 1 -- 172.21.15.131:0/4142971465 >> 172.21.15.131:0/4142971465 conn(0x7f9f4c0cc860 msgr2=0x7f9f4c0ccc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:28.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.231+0000 7f9f50a00640 1 -- 172.21.15.131:0/4142971465 shutdown_connections 2024-05-26T22:51:28.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.231+0000 7f9f50a00640 1 -- 172.21.15.131:0/4142971465 wait complete. 2024-05-26T22:51:28.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.233+0000 7f9f50a00640 1 Processor -- start 2024-05-26T22:51:28.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.233+0000 7f9f50a00640 1 -- start start 2024-05-26T22:51:28.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.234+0000 7f9f50a00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9f4c0cf2f0 0x7f9f4c13e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:28.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.234+0000 7f9f50a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f4c1556e0 0x7f9f4c1500e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:28.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.234+0000 7f9f50a00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9f4c15ec80 0x7f9f4c150620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:28.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.234+0000 7f9f50a00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f9f4c155ac0 con 0x7f9f4c1556e0 2024-05-26T22:51:28.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.234+0000 7f9f50a00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9f4c072510 con 0x7f9f4c15ec80 2024-05-26T22:51:28.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.234+0000 7f9f50a00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9f4c072690 con 0x7f9f4c0cf2f0 2024-05-26T22:51:28.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.234+0000 7f9f49e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f4c1556e0 0x7f9f4c1500e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:28.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.234+0000 7f9f49e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f4c1556e0 0x7f9f4c1500e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48220/0 (socket says 172.21.15.131:48220) 2024-05-26T22:51:28.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.234+0000 7f9f49e00640 1 -- 172.21.15.131:0/3363965858 learned_addr learned my addr 172.21.15.131:0/3363965858 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:28.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.235+0000 7f9f4a800640 1 --2- 172.21.15.131:0/3363965858 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9f4c0cf2f0 0x7f9f4c13e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:28.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.235+0000 7f9f4b200640 1 --2- 172.21.15.131:0/3363965858 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9f4c15ec80 0x7f9f4c150620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:28.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.235+0000 7f9f49e00640 1 -- 172.21.15.131:0/3363965858 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9f4c0cf2f0 msgr2=0x7f9f4c13e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:28.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.235+0000 7f9f49e00640 1 --2- 172.21.15.131:0/3363965858 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9f4c0cf2f0 0x7f9f4c13e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:28.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.235+0000 7f9f49e00640 1 -- 172.21.15.131:0/3363965858 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9f4c15ec80 msgr2=0x7f9f4c150620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:28.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.235+0000 7f9f49e00640 1 --2- 172.21.15.131:0/3363965858 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9f4c15ec80 0x7f9f4c150620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:28.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.235+0000 7f9f49e00640 1 -- 172.21.15.131:0/3363965858 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f4c141980 con 0x7f9f4c1556e0 2024-05-26T22:51:28.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.235+0000 7f9f49e00640 1 --2- 172.21.15.131:0/3363965858 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f4c1556e0 0x7f9f4c1500e0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f9f380097a0 tx=0x7f9f380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:28.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.235+0000 7f9f42a00640 1 -- 172.21.15.131:0/3363965858 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f38037040 con 0x7f9f4c1556e0 2024-05-26T22:51:28.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.235+0000 7f9f42a00640 1 -- 172.21.15.131:0/3363965858 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f38004250 con 0x7f9f4c1556e0 2024-05-26T22:51:28.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.235+0000 7f9f42a00640 1 -- 172.21.15.131:0/3363965858 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f38032e20 con 0x7f9f4c1556e0 2024-05-26T22:51:28.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.236+0000 7f9f50a00640 1 -- 172.21.15.131:0/3363965858 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f4c1408e0 con 0x7f9f4c1556e0 2024-05-26T22:51:28.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.236+0000 7f9f50a00640 1 -- 172.21.15.131:0/3363965858 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f4c140ea0 con 0x7f9f4c1556e0 2024-05-26T22:51:28.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.237+0000 7f9f50a00640 1 -- 172.21.15.131:0/3363965858 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f4c0694e0 con 0x7f9f4c1556e0 2024-05-26T22:51:28.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.238+0000 7f9f42a00640 1 -- 172.21.15.131:0/3363965858 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9f38030770 con 0x7f9f4c1556e0 2024-05-26T22:51:28.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.239+0000 7f9f42a00640 1 --2- 172.21.15.131:0/3363965858 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9f2c06be30 0x7f9f2c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:28.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.239+0000 7f9f42a00640 1 -- 172.21.15.131:0/3363965858 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5201+0+0 (secure 0 0 0) 0x7f9f380abfa0 con 0x7f9f4c1556e0 2024-05-26T22:51:28.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.239+0000 7f9f4a800640 1 --2- 172.21.15.131:0/3363965858 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9f2c06be30 0x7f9f2c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:28.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.240+0000 7f9f4a800640 1 --2- 172.21.15.131:0/3363965858 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9f2c06be30 0x7f9f2c06e2f0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c000f60 tx=0x7f9f3c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:28.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.244+0000 7f9f42a00640 1 -- 172.21.15.131:0/3363965858 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f9f4c141980 con 0x7f9f4c1556e0 2024-05-26T22:51:28.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.448+0000 7f9f50a00640 1 -- 172.21.15.131:0/3363965858 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_export_ephemeral_random"} v 0) -- 0x7f9f4c06e080 con 0x7f9f4c1556e0 2024-05-26T22:51:28.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.453+0000 7f9f42a00640 1 -- 172.21.15.131:0/3363965858 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_export_ephemeral_random"}]=0 v24) ==== 110+0+0 (secure 0 0 0) 0x7f9f3807ac50 con 0x7f9f4c1556e0 2024-05-26T22:51:28.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.456+0000 7f9f50a00640 1 -- 172.21.15.131:0/3363965858 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9f2c06be30 msgr2=0x7f9f2c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:28.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.456+0000 7f9f50a00640 1 --2- 172.21.15.131:0/3363965858 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9f2c06be30 0x7f9f2c06e2f0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c000f60 tx=0x7f9f3c009040 comp rx=0 tx=0).stop 2024-05-26T22:51:28.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.457+0000 7f9f50a00640 1 -- 172.21.15.131:0/3363965858 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f4c1556e0 msgr2=0x7f9f4c1500e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:28.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.457+0000 7f9f50a00640 1 --2- 172.21.15.131:0/3363965858 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f4c1556e0 0x7f9f4c1500e0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f9f380097a0 tx=0x7f9f380046c0 comp rx=0 tx=0).stop 2024-05-26T22:51:28.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.457+0000 7f9f50a00640 1 -- 172.21.15.131:0/3363965858 shutdown_connections 2024-05-26T22:51:28.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.457+0000 7f9f50a00640 1 --2- 172.21.15.131:0/3363965858 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9f2c06be30 0x7f9f2c06e2f0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:28.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.457+0000 7f9f50a00640 1 --2- 172.21.15.131:0/3363965858 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9f4c15ec80 0x7f9f4c150620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:28.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.457+0000 7f9f50a00640 1 --2- 172.21.15.131:0/3363965858 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f4c1556e0 0x7f9f4c1500e0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:28.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.457+0000 7f9f50a00640 1 --2- 172.21.15.131:0/3363965858 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9f4c0cf2f0 0x7f9f4c13e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:28.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.457+0000 7f9f50a00640 1 -- 172.21.15.131:0/3363965858 >> 172.21.15.131:0/3363965858 conn(0x7f9f4c0cc860 msgr2=0x7f9f4c15ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:28.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.457+0000 7f9f50a00640 1 -- 172.21.15.131:0/3363965858 shutdown_connections 2024-05-26T22:51:28.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.457+0000 7f9f50a00640 1 -- 172.21.15.131:0/3363965858 wait complete. 2024-05-26T22:51:28.475 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_export_ephemeral_distributed 2024-05-26T22:51:28.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.683+0000 7fd166800640 1 Processor -- start 2024-05-26T22:51:28.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.683+0000 7fd166800640 1 -- start start 2024-05-26T22:51:28.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.685+0000 7fd166800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd16006da30 0x7fd1600cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:28.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.685+0000 7fd166800640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1600cc780 con 0x7fd1600cc2d0 2024-05-26T22:51:28.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.685+0000 7fd166800640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1600cdca0 con 0x7fd1600cbef0 2024-05-26T22:51:28.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.685+0000 7fd166800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd1600ad600 con 0x7fd16006da30 2024-05-26T22:51:28.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.685+0000 7fd15ea00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fd1600cc2d0 0x7fd16006d320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:43052/0 (socket says 172.21.15.131:43052) 2024-05-26T22:51:28.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.685+0000 7fd15ea00640 1 -- 172.21.15.131:0/1610489107 learned_addr learned my addr 172.21.15.131:0/1610489107 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:51:28.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.686+0000 7fd15fe00640 1 --2- 172.21.15.131:0/1610489107 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd16006da30 0x7fd1600cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:28.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.686+0000 7fd15e000640 1 -- 172.21.15.131:0/1610489107 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1275942036 0 0) 0x7fd14c001110 con 0x7fd1600cc2d0 2024-05-26T22:51:28.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.686+0000 7fd15e000640 1 -- 172.21.15.131:0/1610489107 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd144003520 con 0x7fd1600cc2d0 2024-05-26T22:51:28.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.686+0000 7fd15e000640 1 -- 172.21.15.131:0/1610489107 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4191130987 0 0) 0x7fd144003520 con 0x7fd1600cc2d0 2024-05-26T22:51:28.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.686+0000 7fd15e000640 1 -- 172.21.15.131:0/1610489107 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd16006da30 msgr2=0x7fd1600cea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:28.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.686+0000 7fd15e000640 1 --2- 172.21.15.131:0/1610489107 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd16006da30 0x7fd1600cea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:28.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.686+0000 7fd15e000640 1 -- 172.21.15.131:0/1610489107 >> v1:172.21.15.187:6789/0 conn(0x7fd1600cbef0 legacy=0x7fd1600acef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:28.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.686+0000 7fd15e000640 1 -- 172.21.15.131:0/1610489107 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1600cdf80 con 0x7fd1600cc2d0 2024-05-26T22:51:28.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.686+0000 7fd15e000640 1 -- 172.21.15.131:0/1610489107 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7fd14c001840 con 0x7fd1600cc2d0 2024-05-26T22:51:28.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.686+0000 7fd15e000640 1 -- 172.21.15.131:0/1610489107 >> v1:172.21.15.131:6789/0 conn(0x7fd1600cc2d0 legacy=0x7fd16006d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:28.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.686+0000 7fd15e000640 1 --2- 172.21.15.131:0/1610489107 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd1440039e0 0x7fd144003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:28.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.687+0000 7fd15e000640 1 --2- 172.21.15.131:0/1610489107 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd144004800 0x7fd144004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:28.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.687+0000 7fd15e000640 1 --2- 172.21.15.131:0/1610489107 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd16006da30 0x7fd1440051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:28.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.687+0000 7fd15e000640 1 -- 172.21.15.131:0/1610489107 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fd1600ad600 con 0x7fd144004800 2024-05-26T22:51:28.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.687+0000 7fd15e000640 1 -- 172.21.15.131:0/1610489107 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd14400baa0 con 0x7fd1440039e0 2024-05-26T22:51:28.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.687+0000 7fd15e000640 1 -- 172.21.15.131:0/1610489107 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd14400bca0 con 0x7fd16006da30 2024-05-26T22:51:28.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.687+0000 7fd15ea00640 1 --2- 172.21.15.131:0/1610489107 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd144004800 0x7fd144004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:28.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.687+0000 7fd15fe00640 1 --2- 172.21.15.131:0/1610489107 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd16006da30 0x7fd1440051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:28.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.687+0000 7fd15f400640 1 --2- 172.21.15.131:0/1610489107 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd1440039e0 0x7fd144003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:28.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.687+0000 7fd15ea00640 1 -- 172.21.15.131:0/1610489107 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd16006da30 msgr2=0x7fd1440051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:28.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.687+0000 7fd15ea00640 1 --2- 172.21.15.131:0/1610489107 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd16006da30 0x7fd1440051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:28.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.687+0000 7fd15ea00640 1 -- 172.21.15.131:0/1610489107 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd1440039e0 msgr2=0x7fd144003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:28.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.687+0000 7fd15ea00640 1 --2- 172.21.15.131:0/1610489107 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd1440039e0 0x7fd144003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:28.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.687+0000 7fd15ea00640 1 -- 172.21.15.131:0/1610489107 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd144003520 con 0x7fd144004800 2024-05-26T22:51:28.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.689+0000 7fd15ea00640 1 --2- 172.21.15.131:0/1610489107 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd144004800 0x7fd144004c80 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fd14c00c490 tx=0x7fd144005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c6c53f166459996 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:28.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.689+0000 7fd15e000640 1 -- 172.21.15.131:0/1610489107 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd14c00c0e0 con 0x7fd144004800 2024-05-26T22:51:28.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.689+0000 7fd15e000640 1 -- 172.21.15.131:0/1610489107 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd14c034070 con 0x7fd144004800 2024-05-26T22:51:28.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.689+0000 7fd15e000640 1 -- 172.21.15.131:0/1610489107 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd14c005d30 con 0x7fd144004800 2024-05-26T22:51:28.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.690+0000 7fd166800640 1 -- 172.21.15.131:0/1610489107 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd144004800 msgr2=0x7fd144004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:28.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.690+0000 7fd166800640 1 --2- 172.21.15.131:0/1610489107 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd144004800 0x7fd144004c80 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fd14c00c490 tx=0x7fd144005830 comp rx=0 tx=0).stop 2024-05-26T22:51:28.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.690+0000 7fd15fe00640 1 -- 172.21.15.131:0/1610489107 reap_dead start 2024-05-26T22:51:28.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.690+0000 7fd166800640 1 -- 172.21.15.131:0/1610489107 shutdown_connections 2024-05-26T22:51:28.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.690+0000 7fd166800640 1 -- 172.21.15.131:0/1610489107 >> 172.21.15.131:0/1610489107 conn(0x7fd16006bc90 msgr2=0x7fd1600b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:28.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.690+0000 7fd166800640 1 -- 172.21.15.131:0/1610489107 shutdown_connections 2024-05-26T22:51:28.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.690+0000 7fd166800640 1 -- 172.21.15.131:0/1610489107 wait complete. 2024-05-26T22:51:28.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.692+0000 7fd166800640 1 Processor -- start 2024-05-26T22:51:28.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.692+0000 7fd166800640 1 -- start start 2024-05-26T22:51:28.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.693+0000 7fd166800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd144004800 0x7fd160144660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:28.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.693+0000 7fd166800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd160154ae0 0x7fd160159730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:28.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.693+0000 7fd166800640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd1600b2c30 0x7fd160159c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:28.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.693+0000 7fd166800640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fd1600cfec0 con 0x7fd144004800 2024-05-26T22:51:28.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.693+0000 7fd166800640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd1600ce540 con 0x7fd1600b2c30 2024-05-26T22:51:28.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.693+0000 7fd166800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd1600d0040 con 0x7fd160154ae0 2024-05-26T22:51:28.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.693+0000 7fd15f400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd144004800 0x7fd160144660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:28.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.693+0000 7fd15f400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd144004800 0x7fd160144660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48236/0 (socket says 172.21.15.131:48236) 2024-05-26T22:51:28.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.693+0000 7fd15f400640 1 -- 172.21.15.131:0/2308793385 learned_addr learned my addr 172.21.15.131:0/2308793385 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:28.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.693+0000 7fd15ea00640 1 --2- 172.21.15.131:0/2308793385 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd160154ae0 0x7fd160159730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:28.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.693+0000 7fd15f400640 1 -- 172.21.15.131:0/2308793385 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd160154ae0 msgr2=0x7fd160159730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:28.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.693+0000 7fd15f400640 1 --2- 172.21.15.131:0/2308793385 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd160154ae0 0x7fd160159730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:28.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.693+0000 7fd15f400640 1 -- 172.21.15.131:0/2308793385 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd1600b2c30 msgr2=0x7fd160159c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:28.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.693+0000 7fd15fe00640 1 --2- 172.21.15.131:0/2308793385 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd1600b2c30 0x7fd160159c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:28.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.693+0000 7fd15f400640 1 --2- 172.21.15.131:0/2308793385 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd1600b2c30 0x7fd160159c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:28.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.693+0000 7fd15f400640 1 -- 172.21.15.131:0/2308793385 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd160155050 con 0x7fd144004800 2024-05-26T22:51:28.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.694+0000 7fd15ea00640 1 --2- 172.21.15.131:0/2308793385 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd160154ae0 0x7fd160159730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:51:28.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.694+0000 7fd15fe00640 1 --2- 172.21.15.131:0/2308793385 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd1600b2c30 0x7fd160159c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:51:28.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.694+0000 7fd15f400640 1 --2- 172.21.15.131:0/2308793385 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd144004800 0x7fd160144660 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fd15000e440 tx=0x7fd15000e900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:28.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.694+0000 7fd157400640 1 -- 172.21.15.131:0/2308793385 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1500052e0 con 0x7fd144004800 2024-05-26T22:51:28.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.694+0000 7fd157400640 1 -- 172.21.15.131:0/2308793385 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd150018070 con 0x7fd144004800 2024-05-26T22:51:28.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.694+0000 7fd157400640 1 -- 172.21.15.131:0/2308793385 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd150003810 con 0x7fd144004800 2024-05-26T22:51:28.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.694+0000 7fd166800640 1 -- 172.21.15.131:0/2308793385 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd16016a7a0 con 0x7fd144004800 2024-05-26T22:51:28.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.694+0000 7fd166800640 1 -- 172.21.15.131:0/2308793385 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd16016ad60 con 0x7fd144004800 2024-05-26T22:51:28.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.696+0000 7fd166800640 1 -- 172.21.15.131:0/2308793385 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1600695c0 con 0x7fd144004800 2024-05-26T22:51:28.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.696+0000 7fd157400640 1 -- 172.21.15.131:0/2308793385 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd1500039b0 con 0x7fd144004800 2024-05-26T22:51:28.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.698+0000 7fd157400640 1 --2- 172.21.15.131:0/2308793385 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fd14006be60 0x7fd14006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:28.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.698+0000 7fd157400640 1 -- 172.21.15.131:0/2308793385 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5201+0+0 (secure 0 0 0) 0x7fd15008a8a0 con 0x7fd144004800 2024-05-26T22:51:28.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.698+0000 7fd15ea00640 1 --2- 172.21.15.131:0/2308793385 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fd14006be60 0x7fd14006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:28.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.699+0000 7fd15ea00640 1 --2- 172.21.15.131:0/2308793385 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fd14006be60 0x7fd14006e320 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fd14c033000 tx=0x7fd14c009030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:28.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.703+0000 7fd157400640 1 -- 172.21.15.131:0/2308793385 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fd1500593a0 con 0x7fd144004800 2024-05-26T22:51:28.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.900+0000 7fd166800640 1 -- 172.21.15.131:0/2308793385 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_export_ephemeral_distributed"} v 0) -- 0x7fd16014b020 con 0x7fd144004800 2024-05-26T22:51:28.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.907+0000 7fd157400640 1 -- 172.21.15.131:0/2308793385 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_export_ephemeral_distributed"}]=0 v25) ==== 115+0+0 (secure 0 0 0) 0x7fd150058ca0 con 0x7fd144004800 2024-05-26T22:51:28.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.910+0000 7fd166800640 1 -- 172.21.15.131:0/2308793385 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fd14006be60 msgr2=0x7fd14006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:28.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.910+0000 7fd166800640 1 --2- 172.21.15.131:0/2308793385 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fd14006be60 0x7fd14006e320 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fd14c033000 tx=0x7fd14c009030 comp rx=0 tx=0).stop 2024-05-26T22:51:28.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.910+0000 7fd166800640 1 -- 172.21.15.131:0/2308793385 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd144004800 msgr2=0x7fd160144660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:28.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.910+0000 7fd166800640 1 --2- 172.21.15.131:0/2308793385 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd144004800 0x7fd160144660 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fd15000e440 tx=0x7fd15000e900 comp rx=0 tx=0).stop 2024-05-26T22:51:28.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.910+0000 7fd166800640 1 -- 172.21.15.131:0/2308793385 shutdown_connections 2024-05-26T22:51:28.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.910+0000 7fd166800640 1 --2- 172.21.15.131:0/2308793385 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd160154ae0 0x7fd160159730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:28.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.910+0000 7fd166800640 1 --2- 172.21.15.131:0/2308793385 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fd14006be60 0x7fd14006e320 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:28.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.910+0000 7fd166800640 1 --2- 172.21.15.131:0/2308793385 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd1600b2c30 0x7fd160159c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:28.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.910+0000 7fd166800640 1 --2- 172.21.15.131:0/2308793385 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd144004800 0x7fd160144660 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:28.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.910+0000 7fd166800640 1 -- 172.21.15.131:0/2308793385 >> 172.21.15.131:0/2308793385 conn(0x7fd16006bc90 msgr2=0x7fd1600adc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:28.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.910+0000 7fd166800640 1 -- 172.21.15.131:0/2308793385 shutdown_connections 2024-05-26T22:51:28.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:28.910+0000 7fd166800640 1 -- 172.21.15.131:0/2308793385 wait complete. 2024-05-26T22:51:28.928 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-26T22:51:29.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.133+0000 7f340c600640 1 Processor -- start 2024-05-26T22:51:29.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.134+0000 7f340c600640 1 -- start start 2024-05-26T22:51:29.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.135+0000 7f340c600640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f34040b5450 0x7f34040b43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:29.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.135+0000 7f340c600640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f340406f420 0x7f34040cfd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:29.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.135+0000 7f340c600640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34040b4900 con 0x7f34040cf590 2024-05-26T22:51:29.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.135+0000 7f340c600640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f3404160d90 con 0x7f340406f420 2024-05-26T22:51:29.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.135+0000 7f340c600640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f3404160180 con 0x7f34040b5450 2024-05-26T22:51:29.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.136+0000 7f3409200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f340406f420 0x7f34040cfd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:29.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.136+0000 7f340a600640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f34040cf590 0x7f340406d600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:43060/0 (socket says 172.21.15.131:43060) 2024-05-26T22:51:29.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.136+0000 7f340a600640 1 -- 172.21.15.131:0/2754166354 learned_addr learned my addr 172.21.15.131:0/2754166354 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:51:29.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.136+0000 7f3409c00640 1 --2- 172.21.15.131:0/2754166354 >> v2:172.21.15.187:3300/0 conn(0x7f34040b5450 0x7f34040b43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:29.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.136+0000 7f3409200640 1 -- 172.21.15.131:0/2754166354 >> v2:172.21.15.187:3300/0 conn(0x7f34040b5450 msgr2=0x7f34040b43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:29.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.136+0000 7f3409200640 1 --2- 172.21.15.131:0/2754166354 >> v2:172.21.15.187:3300/0 conn(0x7f34040b5450 0x7f34040b43c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:29.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.136+0000 7f3409200640 1 -- 172.21.15.131:0/2754166354 >> v1:172.21.15.131:6789/0 conn(0x7f34040cf590 legacy=0x7f340406d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:29.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.136+0000 7f3409200640 1 -- 172.21.15.131:0/2754166354 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3404160460 con 0x7f340406f420 2024-05-26T22:51:29.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.137+0000 7f3409200640 1 --2- 172.21.15.131:0/2754166354 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f340406f420 0x7f34040cfd50 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f34040d0c40 tx=0x7f33f002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f316c6c06c4dc0a1 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:29.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.137+0000 7f3403e00640 1 -- 172.21.15.131:0/2754166354 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33f0037070 con 0x7f340406f420 2024-05-26T22:51:29.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.138+0000 7f3403e00640 1 -- 172.21.15.131:0/2754166354 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33f0032440 con 0x7f340406f420 2024-05-26T22:51:29.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.138+0000 7f3403e00640 1 -- 172.21.15.131:0/2754166354 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33f0032810 con 0x7f340406f420 2024-05-26T22:51:29.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.138+0000 7f340c600640 1 -- 172.21.15.131:0/2754166354 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f340406f420 msgr2=0x7f34040cfd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:29.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.138+0000 7f340c600640 1 --2- 172.21.15.131:0/2754166354 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f340406f420 0x7f34040cfd50 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f34040d0c40 tx=0x7f33f002a1a0 comp rx=0 tx=0).stop 2024-05-26T22:51:29.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.138+0000 7f340c600640 1 -- 172.21.15.131:0/2754166354 shutdown_connections 2024-05-26T22:51:29.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.138+0000 7f340c600640 1 --2- 172.21.15.131:0/2754166354 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f340406f420 0x7f34040cfd50 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:29.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.138+0000 7f340c600640 1 --2- 172.21.15.131:0/2754166354 >> v2:172.21.15.187:3300/0 conn(0x7f34040b5450 0x7f34040b43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:29.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.138+0000 7f340c600640 1 -- 172.21.15.131:0/2754166354 >> 172.21.15.131:0/2754166354 conn(0x7f340406bc90 msgr2=0x7f34040b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:29.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.138+0000 7f340c600640 1 -- 172.21.15.131:0/2754166354 shutdown_connections 2024-05-26T22:51:29.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.138+0000 7f340c600640 1 -- 172.21.15.131:0/2754166354 wait complete. 2024-05-26T22:51:29.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.140+0000 7f340c600640 1 Processor -- start 2024-05-26T22:51:29.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.140+0000 7f340c600640 1 -- start start 2024-05-26T22:51:29.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.141+0000 7f340c600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f340406f420 0x7f3404146660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:29.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.141+0000 7f340c600640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f34040b5450 0x7f3404158700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:29.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.141+0000 7f340c600640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f34040cf590 0x7f3404158c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:29.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.141+0000 7f340c600640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f34040b5830 con 0x7f340406f420 2024-05-26T22:51:29.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.141+0000 7f340c600640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3404160a20 con 0x7f34040b5450 2024-05-26T22:51:29.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.141+0000 7f340c600640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f340415fde0 con 0x7f34040cf590 2024-05-26T22:51:29.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.141+0000 7f3409c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f340406f420 0x7f3404146660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:29.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.141+0000 7f3409c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f340406f420 0x7f3404146660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48252/0 (socket says 172.21.15.131:48252) 2024-05-26T22:51:29.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.141+0000 7f3409c00640 1 -- 172.21.15.131:0/7844086 learned_addr learned my addr 172.21.15.131:0/7844086 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:29.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.142+0000 7f3409c00640 1 -- 172.21.15.131:0/7844086 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f34040cf590 msgr2=0x7f3404158c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:29.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.142+0000 7f3409c00640 1 --2- 172.21.15.131:0/7844086 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f34040cf590 0x7f3404158c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:29.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.142+0000 7f3409c00640 1 -- 172.21.15.131:0/7844086 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f34040b5450 msgr2=0x7f3404158700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:29.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.142+0000 7f3409200640 1 --2- 172.21.15.131:0/7844086 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f34040b5450 0x7f3404158700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:29.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.142+0000 7f3409c00640 1 --2- 172.21.15.131:0/7844086 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f34040b5450 0x7f3404158700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:29.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.142+0000 7f3409c00640 1 -- 172.21.15.131:0/7844086 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f340409d200 con 0x7f340406f420 2024-05-26T22:51:29.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.142+0000 7f3409c00640 1 --2- 172.21.15.131:0/7844086 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f340406f420 0x7f3404146660 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f33f800d800 tx=0x7f33f800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:29.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.142+0000 7f3401600640 1 -- 172.21.15.131:0/7844086 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33f800b6c0 con 0x7f340406f420 2024-05-26T22:51:29.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.142+0000 7f3401600640 1 -- 172.21.15.131:0/7844086 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33f8019070 con 0x7f340406f420 2024-05-26T22:51:29.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.142+0000 7f340c600640 1 -- 172.21.15.131:0/7844086 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f340409d4e0 con 0x7f340406f420 2024-05-26T22:51:29.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.142+0000 7f3401600640 1 -- 172.21.15.131:0/7844086 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33f800ba80 con 0x7f340406f420 2024-05-26T22:51:29.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.142+0000 7f340c600640 1 -- 172.21.15.131:0/7844086 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f340415c7f0 con 0x7f340406f420 2024-05-26T22:51:29.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.144+0000 7f340c600640 1 -- 172.21.15.131:0/7844086 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3404074e90 con 0x7f340406f420 2024-05-26T22:51:29.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.145+0000 7f3401600640 1 -- 172.21.15.131:0/7844086 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f33f800bc20 con 0x7f340406f420 2024-05-26T22:51:29.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.146+0000 7f3401600640 1 --2- 172.21.15.131:0/7844086 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f33e406be30 0x7f33e406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:29.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.146+0000 7f3401600640 1 -- 172.21.15.131:0/7844086 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5201+0+0 (secure 0 0 0) 0x7f33f808b230 con 0x7f340406f420 2024-05-26T22:51:29.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.146+0000 7f3409200640 1 --2- 172.21.15.131:0/7844086 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f33e406be30 0x7f33e406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:29.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.147+0000 7f3409200640 1 --2- 172.21.15.131:0/7844086 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f33e406be30 0x7f33e406e2f0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f33f0034000 tx=0x7f33f002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:29.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.150+0000 7f3401600640 1 -- 172.21.15.131:0/7844086 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f33f8059ca0 con 0x7f340406f420 2024-05-26T22:51:29.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.349+0000 7f340c600640 1 -- 172.21.15.131:0/7844086 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f34040695c0 con 0x7f340406f420 2024-05-26T22:51:29.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.355+0000 7f3401600640 1 -- 172.21.15.131:0/7844086 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v26) ==== 104+0+0 (secure 0 0 0) 0x7f33f80595a0 con 0x7f340406f420 2024-05-26T22:51:29.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.357+0000 7f340c600640 1 -- 172.21.15.131:0/7844086 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f33e406be30 msgr2=0x7f33e406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:29.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.357+0000 7f340c600640 1 --2- 172.21.15.131:0/7844086 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f33e406be30 0x7f33e406e2f0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f33f0034000 tx=0x7f33f002e000 comp rx=0 tx=0).stop 2024-05-26T22:51:29.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.357+0000 7f340c600640 1 -- 172.21.15.131:0/7844086 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f340406f420 msgr2=0x7f3404146660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:29.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.357+0000 7f340c600640 1 --2- 172.21.15.131:0/7844086 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f340406f420 0x7f3404146660 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f33f800d800 tx=0x7f33f800dcc0 comp rx=0 tx=0).stop 2024-05-26T22:51:29.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.357+0000 7f340c600640 1 -- 172.21.15.131:0/7844086 shutdown_connections 2024-05-26T22:51:29.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.358+0000 7f340c600640 1 --2- 172.21.15.131:0/7844086 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f34040cf590 0x7f3404158c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:29.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.358+0000 7f340c600640 1 --2- 172.21.15.131:0/7844086 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f33e406be30 0x7f33e406e2f0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:29.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.358+0000 7f340c600640 1 --2- 172.21.15.131:0/7844086 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f34040b5450 0x7f3404158700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:29.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.358+0000 7f340c600640 1 --2- 172.21.15.131:0/7844086 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f340406f420 0x7f3404146660 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:29.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.358+0000 7f340c600640 1 -- 172.21.15.131:0/7844086 >> 172.21.15.131:0/7844086 conn(0x7f340406bc90 msgr2=0x7f34040af290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:29.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.358+0000 7f340c600640 1 -- 172.21.15.131:0/7844086 shutdown_connections 2024-05-26T22:51:29.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.358+0000 7f340c600640 1 -- 172.21.15.131:0/7844086 wait complete. 2024-05-26T22:51:29.376 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_exports.TestEphemeralPins.test_ephemeral_pin_dist_failover' 2024-05-26T22:51:29.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.576+0000 7f3c0fe00640 1 Processor -- start 2024-05-26T22:51:29.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.576+0000 7f3c0fe00640 1 -- start start 2024-05-26T22:51:29.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.578+0000 7f3c0fe00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3c1006e8c0 0x7f3c1015d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:29.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.578+0000 7f3c0fe00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f3c100cc7f0 0x7f3c100ccc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:29.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.578+0000 7f3c0fe00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c10160620 con 0x7f3c10156750 2024-05-26T22:51:29.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.578+0000 7f3c0fe00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f3c100cd1a0 con 0x7f3c100cc7f0 2024-05-26T22:51:29.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.578+0000 7f3c0fe00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3c1015cbd0 con 0x7f3c1006e8c0 2024-05-26T22:51:29.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.578+0000 7f3c0ea00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f3c10156750 0x7f3c1015fd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:43072/0 (socket says 172.21.15.131:43072) 2024-05-26T22:51:29.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.578+0000 7f3c0ea00640 1 -- 172.21.15.131:0/2372903015 learned_addr learned my addr 172.21.15.131:0/2372903015 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:51:29.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.578+0000 7f3c0e000640 1 -- 172.21.15.131:0/2372903015 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 862847081 0 0) 0x7f3c10160620 con 0x7f3c10156750 2024-05-26T22:51:29.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.578+0000 7f3c0f400640 1 --2- 172.21.15.131:0/2372903015 >> v2:172.21.15.187:3300/0 conn(0x7f3c100cc7f0 0x7f3c100ccc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:29.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.578+0000 7f3c06000640 1 --2- 172.21.15.131:0/2372903015 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3c1006e8c0 0x7f3c1015d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:29.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.578+0000 7f3c0e000640 1 -- 172.21.15.131:0/2372903015 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3bf4003520 con 0x7f3c10156750 2024-05-26T22:51:29.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.579+0000 7f3c0e000640 1 -- 172.21.15.131:0/2372903015 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1944614777 0 0) 0x7f3bf4003520 con 0x7f3c10156750 2024-05-26T22:51:29.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.579+0000 7f3c0e000640 1 -- 172.21.15.131:0/2372903015 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3c1006e8c0 msgr2=0x7f3c1015d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:29.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.579+0000 7f3c0e000640 1 --2- 172.21.15.131:0/2372903015 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3c1006e8c0 0x7f3c1015d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:29.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.579+0000 7f3c0e000640 1 -- 172.21.15.131:0/2372903015 >> v2:172.21.15.187:3300/0 conn(0x7f3c100cc7f0 msgr2=0x7f3c100ccc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:29.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.579+0000 7f3c0e000640 1 --2- 172.21.15.131:0/2372903015 >> v2:172.21.15.187:3300/0 conn(0x7f3c100cc7f0 0x7f3c100ccc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:29.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.579+0000 7f3c0e000640 1 -- 172.21.15.131:0/2372903015 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c10160900 con 0x7f3c10156750 2024-05-26T22:51:29.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.579+0000 7f3c0e000640 1 -- 172.21.15.131:0/2372903015 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7f3c00001670 con 0x7f3c10156750 2024-05-26T22:51:29.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.579+0000 7f3c0e000640 1 -- 172.21.15.131:0/2372903015 >> v1:172.21.15.131:6789/0 conn(0x7f3c10156750 legacy=0x7f3c1015fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:29.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.579+0000 7f3c0e000640 1 --2- 172.21.15.131:0/2372903015 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3bf40039b0 0x7f3bf4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:29.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.579+0000 7f3c0e000640 1 --2- 172.21.15.131:0/2372903015 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3bf4004800 0x7f3bf4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:29.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.579+0000 7f3c0e000640 1 --2- 172.21.15.131:0/2372903015 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3c1006e8c0 0x7f3bf40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:29.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.579+0000 7f3c0e000640 1 -- 172.21.15.131:0/2372903015 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f3c100cd1a0 con 0x7f3bf4004800 2024-05-26T22:51:29.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.579+0000 7f3c0e000640 1 -- 172.21.15.131:0/2372903015 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3c1015cbd0 con 0x7f3bf40039b0 2024-05-26T22:51:29.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.579+0000 7f3c0e000640 1 -- 172.21.15.131:0/2372903015 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3bf400b9c0 con 0x7f3c1006e8c0 2024-05-26T22:51:29.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.579+0000 7f3c06000640 1 --2- 172.21.15.131:0/2372903015 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3bf4004800 0x7f3bf4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:29.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.579+0000 7f3c0f400640 1 --2- 172.21.15.131:0/2372903015 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3c1006e8c0 0x7f3bf40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:29.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.579+0000 7f3c0ea00640 1 --2- 172.21.15.131:0/2372903015 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3bf40039b0 0x7f3bf4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:29.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.580+0000 7f3c06000640 1 -- 172.21.15.131:0/2372903015 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3c1006e8c0 msgr2=0x7f3bf40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:29.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.580+0000 7f3c06000640 1 --2- 172.21.15.131:0/2372903015 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3c1006e8c0 0x7f3bf40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:29.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.580+0000 7f3c06000640 1 -- 172.21.15.131:0/2372903015 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3bf40039b0 msgr2=0x7f3bf4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:29.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.580+0000 7f3c06000640 1 --2- 172.21.15.131:0/2372903015 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3bf40039b0 0x7f3bf4003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:29.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.580+0000 7f3c06000640 1 -- 172.21.15.131:0/2372903015 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bf4003520 con 0x7f3bf4004800 2024-05-26T22:51:29.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.580+0000 7f3c0f400640 1 --2- 172.21.15.131:0/2372903015 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3c1006e8c0 0x7f3bf40051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:51:29.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.581+0000 7f3c06000640 1 --2- 172.21.15.131:0/2372903015 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3bf4004800 0x7f3bf4004c80 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f3bf800ba30 tx=0x7f3bf4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=590952ef8223e9a8 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:29.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.581+0000 7f3c0e000640 1 -- 172.21.15.131:0/2372903015 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bf8030070 con 0x7f3bf4004800 2024-05-26T22:51:29.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.581+0000 7f3c0e000640 1 -- 172.21.15.131:0/2372903015 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3bf802ade0 con 0x7f3bf4004800 2024-05-26T22:51:29.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.581+0000 7f3c0e000640 1 -- 172.21.15.131:0/2372903015 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bf800b600 con 0x7f3bf4004800 2024-05-26T22:51:29.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.581+0000 7f3c0fe00640 1 -- 172.21.15.131:0/2372903015 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3bf4004800 msgr2=0x7f3bf4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:29.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.581+0000 7f3c0fe00640 1 --2- 172.21.15.131:0/2372903015 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3bf4004800 0x7f3bf4004c80 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f3bf800ba30 tx=0x7f3bf4005830 comp rx=0 tx=0).stop 2024-05-26T22:51:29.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.581+0000 7f3c0f400640 1 -- 172.21.15.131:0/2372903015 reap_dead start 2024-05-26T22:51:29.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.582+0000 7f3c0fe00640 1 -- 172.21.15.131:0/2372903015 shutdown_connections 2024-05-26T22:51:29.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.582+0000 7f3c0fe00640 1 -- 172.21.15.131:0/2372903015 >> 172.21.15.131:0/2372903015 conn(0x7f3c100cff40 msgr2=0x7f3c100d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:29.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.582+0000 7f3c0fe00640 1 -- 172.21.15.131:0/2372903015 shutdown_connections 2024-05-26T22:51:29.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.582+0000 7f3c0fe00640 1 -- 172.21.15.131:0/2372903015 wait complete. 2024-05-26T22:51:29.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.583+0000 7f3c0fe00640 1 Processor -- start 2024-05-26T22:51:29.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.583+0000 7f3c0fe00640 1 -- start start 2024-05-26T22:51:29.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.584+0000 7f3c0fe00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3bf4004800 0x7f3c100a9460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:29.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.584+0000 7f3c0fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3c10143830 0x7f3c101430c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:29.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.584+0000 7f3c0fe00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3c1015b0c0 0x7f3c1015b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:29.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.584+0000 7f3c0fe00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f3c10156b30 con 0x7f3c10143830 2024-05-26T22:51:29.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.585+0000 7f3c0fe00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3c1015c860 con 0x7f3c1015b0c0 2024-05-26T22:51:29.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.585+0000 7f3c0fe00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3c1015d060 con 0x7f3bf4004800 2024-05-26T22:51:29.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.585+0000 7f3c06000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3c10143830 0x7f3c101430c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:29.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.585+0000 7f3c06000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3c10143830 0x7f3c101430c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48282/0 (socket says 172.21.15.131:48282) 2024-05-26T22:51:29.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.585+0000 7f3c06000640 1 -- 172.21.15.131:0/1306027088 learned_addr learned my addr 172.21.15.131:0/1306027088 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:29.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.585+0000 7f3c0f400640 1 --2- 172.21.15.131:0/1306027088 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3c1015b0c0 0x7f3c1015b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:29.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.585+0000 7f3c06000640 1 -- 172.21.15.131:0/1306027088 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3bf4004800 msgr2=0x7f3c100a9460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:29.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.585+0000 7f3c06000640 1 --2- 172.21.15.131:0/1306027088 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3bf4004800 0x7f3c100a9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:29.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.585+0000 7f3c06000640 1 -- 172.21.15.131:0/1306027088 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3c1015b0c0 msgr2=0x7f3c1015b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:29.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.585+0000 7f3c06000640 1 --2- 172.21.15.131:0/1306027088 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3c1015b0c0 0x7f3c1015b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:29.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.585+0000 7f3c06000640 1 -- 172.21.15.131:0/1306027088 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c10143600 con 0x7f3c10143830 2024-05-26T22:51:29.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.586+0000 7f3c06000640 1 --2- 172.21.15.131:0/1306027088 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3c10143830 0x7f3c101430c0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f3bf800bb20 tx=0x7f3bf80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:29.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.586+0000 7f3c07400640 1 -- 172.21.15.131:0/1306027088 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bf8043030 con 0x7f3c10143830 2024-05-26T22:51:29.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.586+0000 7f3c07400640 1 -- 172.21.15.131:0/1306027088 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3bf802abe0 con 0x7f3c10143830 2024-05-26T22:51:29.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.586+0000 7f3c07400640 1 -- 172.21.15.131:0/1306027088 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bf800b600 con 0x7f3c10143830 2024-05-26T22:51:29.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.586+0000 7f3c0fe00640 1 -- 172.21.15.131:0/1306027088 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c100a9bd0 con 0x7f3c10143830 2024-05-26T22:51:29.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.586+0000 7f3c0fe00640 1 -- 172.21.15.131:0/1306027088 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c100aed10 con 0x7f3c10143830 2024-05-26T22:51:29.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.587+0000 7f3c0fe00640 1 -- 172.21.15.131:0/1306027088 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c1015ba80 con 0x7f3c10143830 2024-05-26T22:51:29.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.588+0000 7f3c07400640 1 -- 172.21.15.131:0/1306027088 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3bf8046070 con 0x7f3c10143830 2024-05-26T22:51:29.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.589+0000 7f3c07400640 1 --2- 172.21.15.131:0/1306027088 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f3bec06be60 0x7f3bec06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:29.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.589+0000 7f3c07400640 1 -- 172.21.15.131:0/1306027088 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5201+0+0 (secure 0 0 0) 0x7f3bf80ab5c0 con 0x7f3c10143830 2024-05-26T22:51:29.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.589+0000 7f3c0ea00640 1 --2- 172.21.15.131:0/1306027088 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f3bec06be60 0x7f3bec06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:29.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.590+0000 7f3c0ea00640 1 --2- 172.21.15.131:0/1306027088 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f3bec06be60 0x7f3bec06e320 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f3c000078d0 tx=0x7f3c00009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:29.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.593+0000 7f3c07400640 1 -- 172.21.15.131:0/1306027088 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f3bf807a0d0 con 0x7f3c10143830 2024-05-26T22:51:29.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:29.866+0000 7f3c0fe00640 1 -- 172.21.15.131:0/1306027088 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_exports.TestEphemeralPins.test_ephemeral_pin_dist_failover"]} v 0) -- 0x7f3c100694e0 con 0x7f3c10143830 2024-05-26T22:51:30.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:30.555+0000 7f3c07400640 1 -- 172.21.15.131:0/1306027088 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_exports.TestEphemeralPins.test_ephemeral_pin_dist_failover"]}]=0 v178) ==== 155+0+0 (secure 0 0 0) 0x7f3bf80799d0 con 0x7f3c10143830 2024-05-26T22:51:30.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:30.560+0000 7f3c0fe00640 1 -- 172.21.15.131:0/1306027088 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f3bec06be60 msgr2=0x7f3bec06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:30.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:30.560+0000 7f3c0fe00640 1 --2- 172.21.15.131:0/1306027088 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f3bec06be60 0x7f3bec06e320 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f3c000078d0 tx=0x7f3c00009040 comp rx=0 tx=0).stop 2024-05-26T22:51:30.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:30.560+0000 7f3c0fe00640 1 -- 172.21.15.131:0/1306027088 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3c10143830 msgr2=0x7f3c101430c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:30.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:30.560+0000 7f3c0fe00640 1 --2- 172.21.15.131:0/1306027088 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3c10143830 0x7f3c101430c0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f3bf800bb20 tx=0x7f3bf80046c0 comp rx=0 tx=0).stop 2024-05-26T22:51:30.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:30.560+0000 7f3c0fe00640 1 -- 172.21.15.131:0/1306027088 shutdown_connections 2024-05-26T22:51:30.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:30.560+0000 7f3c0fe00640 1 --2- 172.21.15.131:0/1306027088 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f3bec06be60 0x7f3bec06e320 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:30.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:30.560+0000 7f3c0fe00640 1 --2- 172.21.15.131:0/1306027088 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3c1015b0c0 0x7f3c1015b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:30.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:30.560+0000 7f3c0fe00640 1 --2- 172.21.15.131:0/1306027088 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3c10143830 0x7f3c101430c0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:30.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:30.560+0000 7f3c0fe00640 1 --2- 172.21.15.131:0/1306027088 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3bf4004800 0x7f3c100a9460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:30.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:30.560+0000 7f3c0fe00640 1 -- 172.21.15.131:0/1306027088 >> 172.21.15.131:0/1306027088 conn(0x7f3c100cff40 msgr2=0x7f3c100ce770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:30.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:30.560+0000 7f3c0fe00640 1 -- 172.21.15.131:0/1306027088 shutdown_connections 2024-05-26T22:51:30.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:30.560+0000 7f3c0fe00640 1 -- 172.21.15.131:0/1306027088 wait complete. 2024-05-26T22:51:30.582 INFO:tasks.cephfs_test_runner:That MDS failover does not cause unnecessary migrations. ... ok 2024-05-26T22:51:30.582 INFO:tasks.cephfs_test_runner:Starting test: test_ephemeral_pin_dist_off (tasks.cephfs.test_exports.TestEphemeralPins) That turning off ephemeral distributed pin merges subtrees. 2024-05-26T22:51:30.582 INFO:tasks.cephfs_test_runner:test_ephemeral_pin_dist_off (tasks.cephfs.test_exports.TestEphemeralPins) 2024-05-26T22:51:30.583 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-26T22:51:30.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:30.785+0000 7f82d4c00640 1 Processor -- start 2024-05-26T22:51:30.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:30.786+0000 7f82d4c00640 1 -- start start 2024-05-26T22:51:30.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:30.787+0000 7f82d4c00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f82d0155840 0x7f82d0071930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:30.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:30.787+0000 7f82d4c00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f82d006f660 0x7f82d00ade40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:30.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:30.787+0000 7f82d4c00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82d00722a0 con 0x7f82d015c3d0 2024-05-26T22:51:30.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:30.787+0000 7f82d4c00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f82d0072580 con 0x7f82d0155840 2024-05-26T22:51:30.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:30.787+0000 7f82d4c00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f82d0072860 con 0x7f82d006f660 2024-05-26T22:51:30.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:30.787+0000 7f82c5400640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f82d015c3d0 0x7f82d006ef50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:43088/0 (socket says 172.21.15.131:43088) 2024-05-26T22:51:30.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:30.787+0000 7f82c5400640 1 -- 172.21.15.131:0/712981269 learned_addr learned my addr 172.21.15.131:0/712981269 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:51:30.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:30.788+0000 7f82cd400640 1 -- 172.21.15.131:0/712981269 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2418819999 0 0) 0x7f82d00722a0 con 0x7f82d015c3d0 2024-05-26T22:51:30.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:30.788+0000 7f82cd400640 1 -- 172.21.15.131:0/712981269 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f82b4005730 con 0x7f82d015c3d0 2024-05-26T22:51:30.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:30.788+0000 7f82cde00640 1 --2- 172.21.15.131:0/712981269 >> v2:172.21.15.187:3300/0 conn(0x7f82d0155840 0x7f82d0071930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:30.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:30.788+0000 7f82ce800640 1 --2- 172.21.15.131:0/712981269 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f82d006f660 0x7f82d00ade40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:30.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:30.788+0000 7f82cd400640 1 -- 172.21.15.131:0/712981269 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1461796817 0 0) 0x7f82b4005730 con 0x7f82d015c3d0 2024-05-26T22:51:30.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:30.788+0000 7f82cd400640 1 -- 172.21.15.131:0/712981269 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f82d006f660 msgr2=0x7f82d00ade40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:30.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:30.788+0000 7f82cd400640 1 --2- 172.21.15.131:0/712981269 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f82d006f660 0x7f82d00ade40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:30.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:30.788+0000 7f82cd400640 1 -- 172.21.15.131:0/712981269 >> v2:172.21.15.187:3300/0 conn(0x7f82d0155840 msgr2=0x7f82d0071930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:30.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:30.788+0000 7f82cd400640 1 --2- 172.21.15.131:0/712981269 >> v2:172.21.15.187:3300/0 conn(0x7f82d0155840 0x7f82d0071930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:30.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:30.788+0000 7f82cd400640 1 -- 172.21.15.131:0/712981269 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82d0072b40 con 0x7f82d015c3d0 2024-05-26T22:51:30.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:30.788+0000 7f82cd400640 1 -- 172.21.15.131:0/712981269 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7f82b8001670 con 0x7f82d015c3d0 2024-05-26T22:51:30.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:30.788+0000 7f82cde00640 1 --2- 172.21.15.131:0/712981269 >> v2:172.21.15.187:3300/0 conn(0x7f82d0155840 0x7f82d0071930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-26T22:51:30.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:30.788+0000 7f82cd400640 1 -- 172.21.15.131:0/712981269 >> v1:172.21.15.131:6789/0 conn(0x7f82d015c3d0 legacy=0x7f82d006ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:30.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:30.788+0000 7f82cd400640 1 --2- 172.21.15.131:0/712981269 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f82b4005bc0 0x7f82b4005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:30.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:30.788+0000 7f82cd400640 1 --2- 172.21.15.131:0/712981269 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f82d006f660 0x7f82b4006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:30.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:30.788+0000 7f82cd400640 1 --2- 172.21.15.131:0/712981269 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f82b4006f50 0x7f82b40073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:30.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:30.788+0000 7f82cd400640 1 -- 172.21.15.131:0/712981269 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f82d0072580 con 0x7f82b4005bc0 2024-05-26T22:51:30.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:30.788+0000 7f82cd400640 1 -- 172.21.15.131:0/712981269 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f82d0072860 con 0x7f82b4006f50 2024-05-26T22:51:30.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:30.788+0000 7f82cd400640 1 -- 172.21.15.131:0/712981269 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f82b400dd30 con 0x7f82d006f660 2024-05-26T22:51:30.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:30.789+0000 7f82cde00640 1 --2- 172.21.15.131:0/712981269 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f82b4005bc0 0x7f82b4005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:30.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:30.789+0000 7f82ce800640 1 --2- 172.21.15.131:0/712981269 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f82b4006f50 0x7f82b40073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:30.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:30.789+0000 7f82cde00640 1 -- 172.21.15.131:0/712981269 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f82d006f660 msgr2=0x7f82b4006a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:30.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:30.789+0000 7f82cde00640 1 --2- 172.21.15.131:0/712981269 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f82d006f660 0x7f82b4006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:30.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:30.789+0000 7f82cde00640 1 -- 172.21.15.131:0/712981269 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f82b4006f50 msgr2=0x7f82b40073d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:30.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:30.789+0000 7f82cde00640 1 --2- 172.21.15.131:0/712981269 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f82b4006f50 0x7f82b40073d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:30.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:30.789+0000 7f82cde00640 1 -- 172.21.15.131:0/712981269 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82b4005730 con 0x7f82b4005bc0 2024-05-26T22:51:30.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:30.790+0000 7f82cde00640 1 --2- 172.21.15.131:0/712981269 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f82b4005bc0 0x7f82b4005fe0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f82c00098a0 tx=0x7f82b4007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e348d76a794d7f8f server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:30.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:30.790+0000 7f82cd400640 1 -- 172.21.15.131:0/712981269 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c000f040 con 0x7f82b4005bc0 2024-05-26T22:51:30.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:30.790+0000 7f82cd400640 1 -- 172.21.15.131:0/712981269 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82c002ad20 con 0x7f82b4005bc0 2024-05-26T22:51:30.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:30.790+0000 7f82cd400640 1 -- 172.21.15.131:0/712981269 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c0004250 con 0x7f82b4005bc0 2024-05-26T22:51:30.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:30.791+0000 7f82d4c00640 1 -- 172.21.15.131:0/712981269 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f82b4005bc0 msgr2=0x7f82b4005fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:30.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:30.791+0000 7f82d4c00640 1 --2- 172.21.15.131:0/712981269 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f82b4005bc0 0x7f82b4005fe0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f82c00098a0 tx=0x7f82b4007a40 comp rx=0 tx=0).stop 2024-05-26T22:51:30.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:30.791+0000 7f82ce800640 1 -- 172.21.15.131:0/712981269 reap_dead start 2024-05-26T22:51:30.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:30.791+0000 7f82d4c00640 1 -- 172.21.15.131:0/712981269 shutdown_connections 2024-05-26T22:51:30.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:30.791+0000 7f82d4c00640 1 -- 172.21.15.131:0/712981269 >> 172.21.15.131:0/712981269 conn(0x7f82d00b1e90 msgr2=0x7f82d00b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:30.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:30.791+0000 7f82d4c00640 1 -- 172.21.15.131:0/712981269 shutdown_connections 2024-05-26T22:51:30.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:30.791+0000 7f82d4c00640 1 -- 172.21.15.131:0/712981269 wait complete. 2024-05-26T22:51:30.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:30.793+0000 7f82d4c00640 1 Processor -- start 2024-05-26T22:51:30.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:30.793+0000 7f82d4c00640 1 -- start start 2024-05-26T22:51:30.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:30.794+0000 7f82d4c00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f82b4005bc0 0x7f82d009d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:30.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:30.794+0000 7f82d4c00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f82d00cf6c0 0x7f82d01546c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:30.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:30.794+0000 7f82d4c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f82d01f74d0 0x7f82d00add60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:30.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:30.794+0000 7f82d4c00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f82d0071ff0 con 0x7f82d01f74d0 2024-05-26T22:51:30.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:30.794+0000 7f82d4c00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f82d0071e70 con 0x7f82d00cf6c0 2024-05-26T22:51:30.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:30.794+0000 7f82d4c00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f82d006de40 con 0x7f82b4005bc0 2024-05-26T22:51:30.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:30.794+0000 7f82ce800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f82d01f74d0 0x7f82d00add60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:30.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:30.794+0000 7f82ce800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f82d01f74d0 0x7f82d00add60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48292/0 (socket says 172.21.15.131:48292) 2024-05-26T22:51:30.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:30.794+0000 7f82ce800640 1 -- 172.21.15.131:0/2084257460 learned_addr learned my addr 172.21.15.131:0/2084257460 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:30.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:30.794+0000 7f82cde00640 1 --2- 172.21.15.131:0/2084257460 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f82b4005bc0 0x7f82d009d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:30.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:30.794+0000 7f82c5400640 1 --2- 172.21.15.131:0/2084257460 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f82d00cf6c0 0x7f82d01546c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:30.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:30.794+0000 7f82ce800640 1 -- 172.21.15.131:0/2084257460 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f82b4005bc0 msgr2=0x7f82d009d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:30.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:30.794+0000 7f82ce800640 1 --2- 172.21.15.131:0/2084257460 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f82b4005bc0 0x7f82d009d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:30.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:30.794+0000 7f82ce800640 1 -- 172.21.15.131:0/2084257460 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f82d00cf6c0 msgr2=0x7f82d01546c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:30.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:30.794+0000 7f82ce800640 1 --2- 172.21.15.131:0/2084257460 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f82d00cf6c0 0x7f82d01546c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:30.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:30.794+0000 7f82ce800640 1 -- 172.21.15.131:0/2084257460 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82d01f8370 con 0x7f82d01f74d0 2024-05-26T22:51:30.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:30.794+0000 7f82c5400640 1 --2- 172.21.15.131:0/2084257460 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f82d00cf6c0 0x7f82d01546c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:51:30.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:30.795+0000 7f82ce800640 1 --2- 172.21.15.131:0/2084257460 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f82d01f74d0 0x7f82d00add60 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f82d0070db0 tx=0x7f82c800e510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:30.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:30.795+0000 7f82c6a00640 1 -- 172.21.15.131:0/2084257460 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c800ed90 con 0x7f82d01f74d0 2024-05-26T22:51:30.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:30.795+0000 7f82c6a00640 1 -- 172.21.15.131:0/2084257460 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82c801a070 con 0x7f82d01f74d0 2024-05-26T22:51:30.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:30.795+0000 7f82d4c00640 1 -- 172.21.15.131:0/2084257460 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82d0154c00 con 0x7f82d01f74d0 2024-05-26T22:51:30.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:30.795+0000 7f82c6a00640 1 -- 172.21.15.131:0/2084257460 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c8015650 con 0x7f82d01f74d0 2024-05-26T22:51:30.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:30.795+0000 7f82d4c00640 1 -- 172.21.15.131:0/2084257460 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82d01f8090 con 0x7f82d01f74d0 2024-05-26T22:51:30.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:30.796+0000 7f82d4c00640 1 -- 172.21.15.131:0/2084257460 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82d015b110 con 0x7f82d01f74d0 2024-05-26T22:51:30.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:30.797+0000 7f82c6a00640 1 -- 172.21.15.131:0/2084257460 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f82c8012030 con 0x7f82d01f74d0 2024-05-26T22:51:30.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:30.798+0000 7f82c6a00640 1 --2- 172.21.15.131:0/2084257460 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f82ac06be60 0x7f82ac06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:30.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:30.798+0000 7f82c6a00640 1 -- 172.21.15.131:0/2084257460 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5201+0+0 (secure 0 0 0) 0x7f82c808b330 con 0x7f82d01f74d0 2024-05-26T22:51:30.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:30.798+0000 7f82cde00640 1 --2- 172.21.15.131:0/2084257460 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f82ac06be60 0x7f82ac06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:30.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:30.799+0000 7f82cde00640 1 --2- 172.21.15.131:0/2084257460 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f82ac06be60 0x7f82ac06e320 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f82c0033000 tx=0x7f82c002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:30.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:30.803+0000 7f82c6a00640 1 -- 172.21.15.131:0/2084257460 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f82d015b110 con 0x7f82d01f74d0 2024-05-26T22:51:31.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.003+0000 7f82d4c00640 1 -- 172.21.15.131:0/2084257460 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f82d00694e0 con 0x7f82d01f74d0 2024-05-26T22:51:31.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.005+0000 7f82c6a00640 1 -- 172.21.15.131:0/2084257460 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) ==== 74+0+13438 (secure 0 0 0) 0x7f82c8010040 con 0x7f82d01f74d0 2024-05-26T22:51:31.005 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:51:31.005 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":50,"fsid":"46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f","created":"2024-05-26T22:47:57.288905+0000","modified":"2024-05-26T22:51:27.528941+0000","last_up_change":"2024-05-26T22:48:09.602207+0000","last_in_change":"2024-05-26T22:48:00.687341+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-26T22:48:10.463339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"814eaf62-a18d-494b-8e27-fd8fb88adea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6801","nonce":2350940051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6803","nonce":2350940051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6807","nonce":2350940051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6805","nonce":2350940051}]},"public_addr":"172.21.15.131:6801/2350940051","cluster_addr":"172.21.15.131:6803/2350940051","heartbeat_back_addr":"172.21.15.131:6807/2350940051","heartbeat_front_addr":"172.21.15.131:6805/2350940051","state":["exists","up"]},{"osd":1,"uuid":"42e66287-f6c6-4e26-af3c-6dfaa3536d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6825","nonce":1525528247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6827","nonce":1525528247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6831","nonce":1525528247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6829","nonce":1525528247}]},"public_addr":"172.21.15.131:6825/1525528247","cluster_addr":"172.21.15.131:6827/1525528247","heartbeat_back_addr":"172.21.15.131:6831/1525528247","heartbeat_front_addr":"172.21.15.131:6829/1525528247","state":["exists","up"]},{"osd":2,"uuid":"0da34673-04a2-4fd0-b57a-0ff2b679b043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6817","nonce":3627842210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6819","nonce":3627842210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6823","nonce":3627842210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6821","nonce":3627842210}]},"public_addr":"172.21.15.131:6817/3627842210","cluster_addr":"172.21.15.131:6819/3627842210","heartbeat_back_addr":"172.21.15.131:6823/3627842210","heartbeat_front_addr":"172.21.15.131:6821/3627842210","state":["exists","up"]},{"osd":3,"uuid":"cf117a27-d6b2-43d3-ae52-b47988bf8eed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6809","nonce":350790591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6811","nonce":350790591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6815","nonce":350790591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6813","nonce":350790591}]},"public_addr":"172.21.15.131:6809/350790591","cluster_addr":"172.21.15.131:6811/350790591","heartbeat_back_addr":"172.21.15.131:6815/350790591","heartbeat_front_addr":"172.21.15.131:6813/350790591","state":["exists","up"]},{"osd":4,"uuid":"a092ccfc-be89-4a84-88d5-0f781fa715eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6816","nonce":244560995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6819","nonce":244560995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6823","nonce":244560995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6821","nonce":244560995}]},"public_addr":"172.21.15.187:6816/244560995","cluster_addr":"172.21.15.187:6819/244560995","heartbeat_back_addr":"172.21.15.187:6823/244560995","heartbeat_front_addr":"172.21.15.187:6821/244560995","state":["exists","up"]},{"osd":5,"uuid":"7513ba69-dd00-4a35-b190-9e8cc3b2bdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6801","nonce":324619398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6803","nonce":324619398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6812","nonce":324619398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6807","nonce":324619398}]},"public_addr":"172.21.15.187:6801/324619398","cluster_addr":"172.21.15.187:6803/324619398","heartbeat_back_addr":"172.21.15.187:6812/324619398","heartbeat_front_addr":"172.21.15.187:6807/324619398","state":["exists","up"]},{"osd":6,"uuid":"50081810-c093-4d2f-a4d7-b7c695c5400d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6806","nonce":1391347786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6809","nonce":1391347786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6818","nonce":1391347786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6813","nonce":1391347786}]},"public_addr":"172.21.15.187:6806/1391347786","cluster_addr":"172.21.15.187:6809/1391347786","heartbeat_back_addr":"172.21.15.187:6818/1391347786","heartbeat_front_addr":"172.21.15.187:6813/1391347786","state":["exists","up"]},{"osd":7,"uuid":"775dae05-54f5-4ad4-bbce-92863c6ed2ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6825","nonce":1220065622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6827","nonce":1220065622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6831","nonce":1220065622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6829","nonce":1220065622}]},"public_addr":"172.21.15.187:6825/1220065622","cluster_addr":"172.21.15.187:6827/1220065622","heartbeat_back_addr":"172.21.15.187:6831/1220065622","heartbeat_front_addr":"172.21.15.187:6829/1220065622","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:05.983520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.299516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.067323+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.307748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.018742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.258775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.010912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.295355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:6836/650770835":"2024-05-27T22:51:24.518119+0000","172.21.15.131:6834/650770835":"2024-05-27T22:51:24.518119+0000","172.21.15.187:6834/1027783169":"2024-05-27T22:51:24.518112+0000","172.21.15.131:6837/265040431":"2024-05-27T22:51:24.518094+0000","172.21.15.187:6832/1027783169":"2024-05-27T22:51:24.518112+0000","172.21.15.187:6835/3429891101":"2024-05-27T22:50:59.823337+0000","172.21.15.131:6835/265040431":"2024-05-27T22:51:24.518094+0000","172.21.15.187:6833/3429891101":"2024-05-27T22:50:59.823337+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-26T22:51:31.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.008+0000 7f82d4c00640 1 -- 172.21.15.131:0/2084257460 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f82ac06be60 msgr2=0x7f82ac06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:31.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.008+0000 7f82d4c00640 1 --2- 172.21.15.131:0/2084257460 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f82ac06be60 0x7f82ac06e320 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f82c0033000 tx=0x7f82c002e000 comp rx=0 tx=0).stop 2024-05-26T22:51:31.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.008+0000 7f82d4c00640 1 -- 172.21.15.131:0/2084257460 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f82d01f74d0 msgr2=0x7f82d00add60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:31.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.008+0000 7f82d4c00640 1 --2- 172.21.15.131:0/2084257460 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f82d01f74d0 0x7f82d00add60 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f82d0070db0 tx=0x7f82c800e510 comp rx=0 tx=0).stop 2024-05-26T22:51:31.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.008+0000 7f82d4c00640 1 -- 172.21.15.131:0/2084257460 shutdown_connections 2024-05-26T22:51:31.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.008+0000 7f82d4c00640 1 --2- 172.21.15.131:0/2084257460 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f82ac06be60 0x7f82ac06e320 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:31.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.008+0000 7f82d4c00640 1 --2- 172.21.15.131:0/2084257460 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f82d01f74d0 0x7f82d00add60 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:31.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.008+0000 7f82d4c00640 1 --2- 172.21.15.131:0/2084257460 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f82d00cf6c0 0x7f82d01546c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:31.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.008+0000 7f82d4c00640 1 --2- 172.21.15.131:0/2084257460 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f82b4005bc0 0x7f82d009d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:31.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.008+0000 7f82d4c00640 1 -- 172.21.15.131:0/2084257460 >> 172.21.15.131:0/2084257460 conn(0x7f82d00b1e90 msgr2=0x7f82d00afe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:31.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.008+0000 7f82d4c00640 1 -- 172.21.15.131:0/2084257460 shutdown_connections 2024-05-26T22:51:31.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.008+0000 7f82d4c00640 1 -- 172.21.15.131:0/2084257460 wait complete. 2024-05-26T22:51:31.028 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-26T22:48:10.463339+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-26T22:51:31.028 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-26T22:51:31.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.230+0000 7f7d3f200640 1 Processor -- start 2024-05-26T22:51:31.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.230+0000 7f7d3f200640 1 -- start start 2024-05-26T22:51:31.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.231+0000 7f7d3f200640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f7d4015dc40 0x7f7d4006d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:31.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.231+0000 7f7d3f200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f7d40071c10 0x7f7d4006d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:31.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.231+0000 7f7d3f200640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d401552d0 con 0x7f7d4015bfa0 2024-05-26T22:51:31.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.231+0000 7f7d3f200640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f7d400b3f30 con 0x7f7d40071c10 2024-05-26T22:51:31.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.231+0000 7f7d3f200640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f7d40072230 con 0x7f7d4015dc40 2024-05-26T22:51:31.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.232+0000 7f7d3e800640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f7d40071c10 0x7f7d4006d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:31.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.232+0000 7f7d3e800640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f7d40071c10 0x7f7d4006d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48298/0 (socket says 172.21.15.131:48298) 2024-05-26T22:51:31.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.232+0000 7f7d3e800640 1 -- 172.21.15.131:0/2129933740 learned_addr learned my addr 172.21.15.131:0/2129933740 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:31.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.232+0000 7f7d3de00640 1 --2- 172.21.15.131:0/2129933740 >> v2:172.21.15.187:3300/0 conn(0x7f7d4015dc40 0x7f7d4006d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:31.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.232+0000 7f7d3e800640 1 -- 172.21.15.131:0/2129933740 >> v2:172.21.15.187:3300/0 conn(0x7f7d4015dc40 msgr2=0x7f7d4006d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:31.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.232+0000 7f7d3e800640 1 --2- 172.21.15.131:0/2129933740 >> v2:172.21.15.187:3300/0 conn(0x7f7d4015dc40 0x7f7d4006d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:31.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.232+0000 7f7d3e800640 1 -- 172.21.15.131:0/2129933740 >> v1:172.21.15.187:6789/0 conn(0x7f7d4015bfa0 legacy=0x7f7d40071500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:31.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.232+0000 7f7d3e800640 1 -- 172.21.15.131:0/2129933740 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d40072510 con 0x7f7d40071c10 2024-05-26T22:51:31.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.232+0000 7f7d3de00640 1 --2- 172.21.15.131:0/2129933740 >> v2:172.21.15.187:3300/0 conn(0x7f7d4015dc40 0x7f7d4006d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:51:31.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.233+0000 7f7d3e800640 1 --2- 172.21.15.131:0/2129933740 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d40071c10 0x7f7d4006d790 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f7d4006f3c0 tx=0x7f7d3802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82a185da6a31908d server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:31.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.234+0000 7f7d3ca00640 1 -- 172.21.15.131:0/2129933740 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d38036070 con 0x7f7d40071c10 2024-05-26T22:51:31.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.234+0000 7f7d3ca00640 1 -- 172.21.15.131:0/2129933740 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d38031600 con 0x7f7d40071c10 2024-05-26T22:51:31.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.234+0000 7f7d3ca00640 1 -- 172.21.15.131:0/2129933740 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d380319d0 con 0x7f7d40071c10 2024-05-26T22:51:31.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.234+0000 7f7d3f200640 1 -- 172.21.15.131:0/2129933740 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d40071c10 msgr2=0x7f7d4006d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:31.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.234+0000 7f7d3f200640 1 --2- 172.21.15.131:0/2129933740 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d40071c10 0x7f7d4006d790 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f7d4006f3c0 tx=0x7f7d3802a6f0 comp rx=0 tx=0).stop 2024-05-26T22:51:31.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.234+0000 7f7d3f200640 1 -- 172.21.15.131:0/2129933740 shutdown_connections 2024-05-26T22:51:31.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.234+0000 7f7d3f200640 1 --2- 172.21.15.131:0/2129933740 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d40071c10 0x7f7d4006d790 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:31.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.234+0000 7f7d3f200640 1 --2- 172.21.15.131:0/2129933740 >> v2:172.21.15.187:3300/0 conn(0x7f7d4015dc40 0x7f7d4006d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:31.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.234+0000 7f7d3f200640 1 -- 172.21.15.131:0/2129933740 >> 172.21.15.131:0/2129933740 conn(0x7f7d400b5660 msgr2=0x7f7d400b5a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:31.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.234+0000 7f7d3f200640 1 -- 172.21.15.131:0/2129933740 shutdown_connections 2024-05-26T22:51:31.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.234+0000 7f7d3f200640 1 -- 172.21.15.131:0/2129933740 wait complete. 2024-05-26T22:51:31.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.236+0000 7f7d3f200640 1 Processor -- start 2024-05-26T22:51:31.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.236+0000 7f7d3f200640 1 -- start start 2024-05-26T22:51:31.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.237+0000 7f7d3f200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7d40071c10 0x7f7d401482f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:31.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.237+0000 7f7d3f200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d4015bfa0 0x7f7d40146510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:31.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.237+0000 7f7d3f200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d4015dc40 0x7f7d4013f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:31.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.237+0000 7f7d3f200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f7d4015e1a0 con 0x7f7d4015bfa0 2024-05-26T22:51:31.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.237+0000 7f7d3f200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7d4015e020 con 0x7f7d4015dc40 2024-05-26T22:51:31.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.237+0000 7f7d3f200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f7d400b2080 con 0x7f7d40071c10 2024-05-26T22:51:31.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.237+0000 7f7d3d400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d4015bfa0 0x7f7d40146510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:31.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.237+0000 7f7d3d400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d4015bfa0 0x7f7d40146510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48300/0 (socket says 172.21.15.131:48300) 2024-05-26T22:51:31.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.237+0000 7f7d3d400640 1 -- 172.21.15.131:0/2814747425 learned_addr learned my addr 172.21.15.131:0/2814747425 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:31.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.238+0000 7f7d3de00640 1 --2- 172.21.15.131:0/2814747425 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7d40071c10 0x7f7d401482f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:31.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.238+0000 7f7d3d400640 1 -- 172.21.15.131:0/2814747425 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7d40071c10 msgr2=0x7f7d401482f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:31.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.238+0000 7f7d3e800640 1 --2- 172.21.15.131:0/2814747425 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d4015dc40 0x7f7d4013f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:31.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.238+0000 7f7d3d400640 1 --2- 172.21.15.131:0/2814747425 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7d40071c10 0x7f7d401482f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:31.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.238+0000 7f7d3d400640 1 -- 172.21.15.131:0/2814747425 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d4015dc40 msgr2=0x7f7d4013f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:31.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.238+0000 7f7d3d400640 1 --2- 172.21.15.131:0/2814747425 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d4015dc40 0x7f7d4013f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:31.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.238+0000 7f7d3d400640 1 -- 172.21.15.131:0/2814747425 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d4013f790 con 0x7f7d4015bfa0 2024-05-26T22:51:31.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.238+0000 7f7d3d400640 1 --2- 172.21.15.131:0/2814747425 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d4015bfa0 0x7f7d40146510 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c00c440 tx=0x7f7d2c00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:31.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.238+0000 7f7d36000640 1 -- 172.21.15.131:0/2814747425 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d2c017070 con 0x7f7d4015bfa0 2024-05-26T22:51:31.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.238+0000 7f7d36000640 1 -- 172.21.15.131:0/2814747425 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d2c0035f0 con 0x7f7d4015bfa0 2024-05-26T22:51:31.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.238+0000 7f7d3f200640 1 -- 172.21.15.131:0/2814747425 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d40147800 con 0x7f7d4015bfa0 2024-05-26T22:51:31.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.238+0000 7f7d36000640 1 -- 172.21.15.131:0/2814747425 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d2c0039e0 con 0x7f7d4015bfa0 2024-05-26T22:51:31.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.239+0000 7f7d3f200640 1 -- 172.21.15.131:0/2814747425 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d40146f60 con 0x7f7d4015bfa0 2024-05-26T22:51:31.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.240+0000 7f7d3f200640 1 -- 172.21.15.131:0/2814747425 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d400694e0 con 0x7f7d4015bfa0 2024-05-26T22:51:31.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.241+0000 7f7d36000640 1 -- 172.21.15.131:0/2814747425 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7d2c01a070 con 0x7f7d4015bfa0 2024-05-26T22:51:31.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.243+0000 7f7d36000640 1 --2- 172.21.15.131:0/2814747425 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f7d2006be30 0x7f7d2006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:31.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.243+0000 7f7d36000640 1 -- 172.21.15.131:0/2814747425 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5201+0+0 (secure 0 0 0) 0x7f7d2c089ff0 con 0x7f7d4015bfa0 2024-05-26T22:51:31.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.243+0000 7f7d3de00640 1 --2- 172.21.15.131:0/2814747425 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f7d2006be30 0x7f7d2006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:31.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.244+0000 7f7d3de00640 1 --2- 172.21.15.131:0/2814747425 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f7d2006be30 0x7f7d2006e2f0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f7d3000b840 tx=0x7f7d30008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:31.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.247+0000 7f7d36000640 1 -- 172.21.15.131:0/2814747425 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f7d2c058b90 con 0x7f7d4015bfa0 2024-05-26T22:51:31.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.448+0000 7f7d3f200640 1 -- 172.21.15.131:0/2814747425 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7d40072bd0 con 0x7f7d4015bfa0 2024-05-26T22:51:31.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.449+0000 7f7d36000640 1 -- 172.21.15.131:0/2814747425 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v50) ==== 93+0+10 (secure 0 0 0) 0x7f7d2c0010c0 con 0x7f7d4015bfa0 2024-05-26T22:51:31.449 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2024-05-26T22:51:31.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.452+0000 7f7d3f200640 1 -- 172.21.15.131:0/2814747425 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f7d2006be30 msgr2=0x7f7d2006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:31.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.452+0000 7f7d3f200640 1 --2- 172.21.15.131:0/2814747425 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f7d2006be30 0x7f7d2006e2f0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f7d3000b840 tx=0x7f7d30008040 comp rx=0 tx=0).stop 2024-05-26T22:51:31.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.452+0000 7f7d3f200640 1 -- 172.21.15.131:0/2814747425 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d4015bfa0 msgr2=0x7f7d40146510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:31.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.452+0000 7f7d3f200640 1 --2- 172.21.15.131:0/2814747425 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d4015bfa0 0x7f7d40146510 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c00c440 tx=0x7f7d2c00c900 comp rx=0 tx=0).stop 2024-05-26T22:51:31.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.452+0000 7f7d3f200640 1 -- 172.21.15.131:0/2814747425 shutdown_connections 2024-05-26T22:51:31.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.452+0000 7f7d3f200640 1 --2- 172.21.15.131:0/2814747425 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f7d2006be30 0x7f7d2006e2f0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:31.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.452+0000 7f7d3f200640 1 --2- 172.21.15.131:0/2814747425 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d4015dc40 0x7f7d4013f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:31.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.452+0000 7f7d3f200640 1 --2- 172.21.15.131:0/2814747425 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d4015bfa0 0x7f7d40146510 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:31.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.452+0000 7f7d3f200640 1 --2- 172.21.15.131:0/2814747425 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7d40071c10 0x7f7d401482f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:31.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.452+0000 7f7d3f200640 1 -- 172.21.15.131:0/2814747425 >> 172.21.15.131:0/2814747425 conn(0x7f7d400b5660 msgr2=0x7f7d4006d230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:31.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.452+0000 7f7d3f200640 1 -- 172.21.15.131:0/2814747425 shutdown_connections 2024-05-26T22:51:31.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.452+0000 7f7d3f200640 1 -- 172.21.15.131:0/2814747425 wait complete. 2024-05-26T22:51:31.472 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_exports.TestEphemeralPins.test_ephemeral_pin_dist_off' 2024-05-26T22:51:31.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.679+0000 7fbc13800640 1 Processor -- start 2024-05-26T22:51:31.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.680+0000 7fbc13800640 1 -- start start 2024-05-26T22:51:31.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.681+0000 7fbc13800640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fbc0c158c00 0x7fbc0c06d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:31.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.681+0000 7fbc13800640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fbc0c06daf0 0x7fbc0c0b3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:31.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.681+0000 7fbc13800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbc0c0b36d0 0x7fbc0c161fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:31.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.681+0000 7fbc13800640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fbc0c06df50 con 0x7fbc0c158c00 2024-05-26T22:51:31.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.681+0000 7fbc13800640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fbc0c0b3e50 con 0x7fbc0c06daf0 2024-05-26T22:51:31.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.681+0000 7fbc13800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fbc0c0b4130 con 0x7fbc0c0b36d0 2024-05-26T22:51:31.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.682+0000 7fbc10e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fbc0c158c00 0x7fbc0c06d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:31.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.682+0000 7fbc10e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fbc0c158c00 0x7fbc0c06d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48308/0 (socket says 172.21.15.131:48308) 2024-05-26T22:51:31.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.682+0000 7fbc10e00640 1 -- 172.21.15.131:0/3681188675 learned_addr learned my addr 172.21.15.131:0/3681188675 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:31.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.682+0000 7fbc11800640 1 --2- 172.21.15.131:0/3681188675 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbc0c0b36d0 0x7fbc0c161fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:31.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.682+0000 7fbc10e00640 1 -- 172.21.15.131:0/3681188675 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbc0c0b36d0 msgr2=0x7fbc0c161fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:31.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.682+0000 7fbc0be00640 1 --2- 172.21.15.131:0/3681188675 >> v2:172.21.15.187:3300/0 conn(0x7fbc0c06daf0 0x7fbc0c0b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:31.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.682+0000 7fbc10e00640 1 --2- 172.21.15.131:0/3681188675 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbc0c0b36d0 0x7fbc0c161fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:31.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.682+0000 7fbc10e00640 1 -- 172.21.15.131:0/3681188675 >> v2:172.21.15.187:3300/0 conn(0x7fbc0c06daf0 msgr2=0x7fbc0c0b3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:31.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.682+0000 7fbc10e00640 1 --2- 172.21.15.131:0/3681188675 >> v2:172.21.15.187:3300/0 conn(0x7fbc0c06daf0 0x7fbc0c0b3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:31.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.682+0000 7fbc10e00640 1 -- 172.21.15.131:0/3681188675 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc0c0b4410 con 0x7fbc0c158c00 2024-05-26T22:51:31.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.682+0000 7fbc0be00640 1 --2- 172.21.15.131:0/3681188675 >> v2:172.21.15.187:3300/0 conn(0x7fbc0c06daf0 0x7fbc0c0b3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:51:31.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.683+0000 7fbc10e00640 1 --2- 172.21.15.131:0/3681188675 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc0c158c00 0x7fbc0c06d5b0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c071090 tx=0x7fbc0002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f788a2cd78d207f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:31.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.684+0000 7fbc0b400640 1 -- 172.21.15.131:0/3681188675 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc00037070 con 0x7fbc0c158c00 2024-05-26T22:51:31.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.684+0000 7fbc0b400640 1 -- 172.21.15.131:0/3681188675 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc00032440 con 0x7fbc0c158c00 2024-05-26T22:51:31.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.684+0000 7fbc0b400640 1 -- 172.21.15.131:0/3681188675 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc00032810 con 0x7fbc0c158c00 2024-05-26T22:51:31.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.684+0000 7fbc13800640 1 -- 172.21.15.131:0/3681188675 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc0c158c00 msgr2=0x7fbc0c06d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:31.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.684+0000 7fbc13800640 1 --2- 172.21.15.131:0/3681188675 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc0c158c00 0x7fbc0c06d5b0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c071090 tx=0x7fbc0002a140 comp rx=0 tx=0).stop 2024-05-26T22:51:31.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.684+0000 7fbc13800640 1 -- 172.21.15.131:0/3681188675 shutdown_connections 2024-05-26T22:51:31.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.684+0000 7fbc13800640 1 --2- 172.21.15.131:0/3681188675 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbc0c0b36d0 0x7fbc0c161fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:31.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.684+0000 7fbc13800640 1 --2- 172.21.15.131:0/3681188675 >> v2:172.21.15.187:3300/0 conn(0x7fbc0c06daf0 0x7fbc0c0b3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:31.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.684+0000 7fbc13800640 1 --2- 172.21.15.131:0/3681188675 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc0c158c00 0x7fbc0c06d5b0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:31.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.684+0000 7fbc13800640 1 -- 172.21.15.131:0/3681188675 >> 172.21.15.131:0/3681188675 conn(0x7fbc0c0b1e90 msgr2=0x7fbc0c0b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:31.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.685+0000 7fbc13800640 1 -- 172.21.15.131:0/3681188675 shutdown_connections 2024-05-26T22:51:31.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.685+0000 7fbc13800640 1 -- 172.21.15.131:0/3681188675 wait complete. 2024-05-26T22:51:31.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.686+0000 7fbc13800640 1 Processor -- start 2024-05-26T22:51:31.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.686+0000 7fbc13800640 1 -- start start 2024-05-26T22:51:31.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.687+0000 7fbc13800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbc0c06daf0 0x7fbc0c145990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:31.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.687+0000 7fbc13800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc0c0b36d0 0x7fbc0c146980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:31.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.687+0000 7fbc13800640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbc0c158c00 0x7fbc0c099500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:31.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.687+0000 7fbc13800640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fbc0c0b3b60 con 0x7fbc0c0b36d0 2024-05-26T22:51:31.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.687+0000 7fbc13800640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fbc0c0b49d0 con 0x7fbc0c158c00 2024-05-26T22:51:31.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.687+0000 7fbc13800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fbc0c1624e0 con 0x7fbc0c06daf0 2024-05-26T22:51:31.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.688+0000 7fbc0be00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc0c0b36d0 0x7fbc0c146980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:31.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.688+0000 7fbc0be00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc0c0b36d0 0x7fbc0c146980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48312/0 (socket says 172.21.15.131:48312) 2024-05-26T22:51:31.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.688+0000 7fbc0be00640 1 -- 172.21.15.131:0/2792696242 learned_addr learned my addr 172.21.15.131:0/2792696242 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:31.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.688+0000 7fbc11800640 1 --2- 172.21.15.131:0/2792696242 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbc0c158c00 0x7fbc0c099500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:31.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.688+0000 7fbc10e00640 1 --2- 172.21.15.131:0/2792696242 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbc0c06daf0 0x7fbc0c145990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:31.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.688+0000 7fbc0be00640 1 -- 172.21.15.131:0/2792696242 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbc0c06daf0 msgr2=0x7fbc0c145990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:31.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.688+0000 7fbc0be00640 1 --2- 172.21.15.131:0/2792696242 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbc0c06daf0 0x7fbc0c145990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:31.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.688+0000 7fbc0be00640 1 -- 172.21.15.131:0/2792696242 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbc0c158c00 msgr2=0x7fbc0c099500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:31.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.688+0000 7fbc0be00640 1 --2- 172.21.15.131:0/2792696242 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbc0c158c00 0x7fbc0c099500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:31.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.688+0000 7fbc0be00640 1 -- 172.21.15.131:0/2792696242 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc0c0ad170 con 0x7fbc0c0b36d0 2024-05-26T22:51:31.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.689+0000 7fbc0be00640 1 --2- 172.21.15.131:0/2792696242 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc0c0b36d0 0x7fbc0c146980 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fbbf800c820 tx=0x7fbbf800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:31.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.689+0000 7fbc08c00640 1 -- 172.21.15.131:0/2792696242 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbf8015070 con 0x7fbc0c0b36d0 2024-05-26T22:51:31.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.689+0000 7fbc08c00640 1 -- 172.21.15.131:0/2792696242 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbbf8007a30 con 0x7fbc0c0b36d0 2024-05-26T22:51:31.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.689+0000 7fbc08c00640 1 -- 172.21.15.131:0/2792696242 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbf8007e00 con 0x7fbc0c0b36d0 2024-05-26T22:51:31.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.689+0000 7fbc13800640 1 -- 172.21.15.131:0/2792696242 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc0c1471e0 con 0x7fbc0c0b36d0 2024-05-26T22:51:31.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.689+0000 7fbc13800640 1 -- 172.21.15.131:0/2792696242 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc0c1477a0 con 0x7fbc0c0b36d0 2024-05-26T22:51:31.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.690+0000 7fbc13800640 1 -- 172.21.15.131:0/2792696242 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc0c0694e0 con 0x7fbc0c0b36d0 2024-05-26T22:51:31.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.691+0000 7fbc08c00640 1 -- 172.21.15.131:0/2792696242 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbbf8018070 con 0x7fbc0c0b36d0 2024-05-26T22:51:31.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.692+0000 7fbc08c00640 1 --2- 172.21.15.131:0/2792696242 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fbbec06be30 0x7fbbec06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:31.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.692+0000 7fbc08c00640 1 -- 172.21.15.131:0/2792696242 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5201+0+0 (secure 0 0 0) 0x7fbbf808ac20 con 0x7fbc0c0b36d0 2024-05-26T22:51:31.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.692+0000 7fbc10e00640 1 --2- 172.21.15.131:0/2792696242 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fbbec06be30 0x7fbbec06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:31.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.693+0000 7fbc10e00640 1 --2- 172.21.15.131:0/2792696242 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fbbec06be30 0x7fbbec06e2f0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fbc00034000 tx=0x7fbc0002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:31.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.697+0000 7fbc08c00640 1 -- 172.21.15.131:0/2792696242 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fbbf80597b0 con 0x7fbc0c0b36d0 2024-05-26T22:51:31.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:31.965+0000 7fbc13800640 1 -- 172.21.15.131:0/2792696242 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_exports.TestEphemeralPins.test_ephemeral_pin_dist_off"]} v 0) -- 0x7fbc0c0d0c20 con 0x7fbc0c0b36d0 2024-05-26T22:51:32.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:32.568+0000 7fbc08c00640 1 -- 172.21.15.131:0/2792696242 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_exports.TestEphemeralPins.test_ephemeral_pin_dist_off"]}]=0 v180) ==== 153+0+0 (secure 0 0 0) 0x7fbbf80590b0 con 0x7fbc0c0b36d0 2024-05-26T22:51:32.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:32.572+0000 7fbc13800640 1 -- 172.21.15.131:0/2792696242 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fbbec06be30 msgr2=0x7fbbec06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:32.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:32.572+0000 7fbc13800640 1 --2- 172.21.15.131:0/2792696242 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fbbec06be30 0x7fbbec06e2f0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fbc00034000 tx=0x7fbc0002e000 comp rx=0 tx=0).stop 2024-05-26T22:51:32.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:32.572+0000 7fbc13800640 1 -- 172.21.15.131:0/2792696242 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc0c0b36d0 msgr2=0x7fbc0c146980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:32.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:32.572+0000 7fbc13800640 1 --2- 172.21.15.131:0/2792696242 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc0c0b36d0 0x7fbc0c146980 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fbbf800c820 tx=0x7fbbf800cc20 comp rx=0 tx=0).stop 2024-05-26T22:51:32.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:32.572+0000 7fbc13800640 1 -- 172.21.15.131:0/2792696242 shutdown_connections 2024-05-26T22:51:32.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:32.572+0000 7fbc13800640 1 --2- 172.21.15.131:0/2792696242 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fbbec06be30 0x7fbbec06e2f0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:32.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:32.572+0000 7fbc13800640 1 --2- 172.21.15.131:0/2792696242 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbc0c158c00 0x7fbc0c099500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:32.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:32.572+0000 7fbc13800640 1 --2- 172.21.15.131:0/2792696242 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc0c0b36d0 0x7fbc0c146980 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:32.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:32.572+0000 7fbc13800640 1 --2- 172.21.15.131:0/2792696242 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbc0c06daf0 0x7fbc0c145990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:32.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:32.572+0000 7fbc13800640 1 -- 172.21.15.131:0/2792696242 >> 172.21.15.131:0/2792696242 conn(0x7fbc0c0b1e90 msgr2=0x7fbc0c15ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:32.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:32.572+0000 7fbc13800640 1 -- 172.21.15.131:0/2792696242 shutdown_connections 2024-05-26T22:51:32.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:32.572+0000 7fbc13800640 1 -- 172.21.15.131:0/2792696242 wait complete. 2024-05-26T22:51:32.593 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-26T22:51:32.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:32.800+0000 7f8504e00640 1 Processor -- start 2024-05-26T22:51:32.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:32.800+0000 7f8504e00640 1 -- start start 2024-05-26T22:51:32.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:32.801+0000 7f8504e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f8500155770 0x7f850015b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:32.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:32.801+0000 7f8504e00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f850015c1c0 con 0x7f850015bc40 2024-05-26T22:51:32.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:32.801+0000 7f8504e00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f850015c4a0 con 0x7f850006e440 2024-05-26T22:51:32.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:32.801+0000 7f8504e00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f8500071da0 con 0x7f8500155770 2024-05-26T22:51:32.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:32.802+0000 7f84fde00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f8500155770 0x7f850015b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:32.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:32.802+0000 7f84fd400640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f850015bc40 0x7f850006dd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:43102/0 (socket says 172.21.15.131:43102) 2024-05-26T22:51:32.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:32.802+0000 7f84fd400640 1 -- 172.21.15.131:0/430764921 learned_addr learned my addr 172.21.15.131:0/430764921 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:51:32.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:32.802+0000 7f84fde00640 1 -- 172.21.15.131:0/430764921 >> v1:172.21.15.187:6789/0 conn(0x7f850006e440 legacy=0x7f85000725a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:32.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:32.802+0000 7f84fde00640 1 -- 172.21.15.131:0/430764921 >> v1:172.21.15.131:6789/0 conn(0x7f850015bc40 legacy=0x7f850006dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:32.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:32.802+0000 7f84fde00640 1 -- 172.21.15.131:0/430764921 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8500072080 con 0x7f8500155770 2024-05-26T22:51:32.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:32.803+0000 7f84fde00640 1 --2- 172.21.15.131:0/430764921 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8500155770 0x7f850015b700 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f84f0009d90 tx=0x7f84f002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58de7b7a8980c2aa server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:32.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:32.804+0000 7f84fca00640 1 -- 172.21.15.131:0/430764921 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f0007740 con 0x7f8500155770 2024-05-26T22:51:32.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:32.804+0000 7f84fca00640 1 -- 172.21.15.131:0/430764921 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84f00078e0 con 0x7f8500155770 2024-05-26T22:51:32.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:32.804+0000 7f84fca00640 1 -- 172.21.15.131:0/430764921 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f0007cb0 con 0x7f8500155770 2024-05-26T22:51:32.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:32.804+0000 7f8504e00640 1 -- 172.21.15.131:0/430764921 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8500155770 msgr2=0x7f850015b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:32.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:32.804+0000 7f8504e00640 1 --2- 172.21.15.131:0/430764921 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8500155770 0x7f850015b700 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f84f0009d90 tx=0x7f84f002a730 comp rx=0 tx=0).stop 2024-05-26T22:51:32.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:32.804+0000 7f8504e00640 1 -- 172.21.15.131:0/430764921 shutdown_connections 2024-05-26T22:51:32.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:32.804+0000 7f8504e00640 1 --2- 172.21.15.131:0/430764921 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8500155770 0x7f850015b700 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:32.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:32.804+0000 7f8504e00640 1 -- 172.21.15.131:0/430764921 >> 172.21.15.131:0/430764921 conn(0x7f850006bc90 msgr2=0x7f85000b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:32.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:32.804+0000 7f8504e00640 1 -- 172.21.15.131:0/430764921 shutdown_connections 2024-05-26T22:51:32.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:32.804+0000 7f8504e00640 1 -- 172.21.15.131:0/430764921 wait complete. 2024-05-26T22:51:32.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:32.806+0000 7f8504e00640 1 Processor -- start 2024-05-26T22:51:32.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:32.806+0000 7f8504e00640 1 -- start start 2024-05-26T22:51:32.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:32.807+0000 7f8504e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f850006e440 0x7f8500143470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:32.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:32.807+0000 7f8504e00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8500155770 0x7f8500142b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:32.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:32.807+0000 7f8504e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f850015bc40 0x7f850009aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:32.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:32.807+0000 7f8504e00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f850009de00 con 0x7f850015bc40 2024-05-26T22:51:32.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:32.807+0000 7f8504e00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f850015c8c0 con 0x7f8500155770 2024-05-26T22:51:32.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:32.807+0000 7f8504e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f850009df80 con 0x7f850006e440 2024-05-26T22:51:32.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:32.807+0000 7f84fe800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f850015bc40 0x7f850009aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:32.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:32.808+0000 7f84fe800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f850015bc40 0x7f850009aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48338/0 (socket says 172.21.15.131:48338) 2024-05-26T22:51:32.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:32.808+0000 7f84fe800640 1 -- 172.21.15.131:0/3711890728 learned_addr learned my addr 172.21.15.131:0/3711890728 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:32.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:32.808+0000 7f84fe800640 1 -- 172.21.15.131:0/3711890728 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f850006e440 msgr2=0x7f8500143470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:32.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:32.808+0000 7f84fe800640 1 --2- 172.21.15.131:0/3711890728 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f850006e440 0x7f8500143470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:32.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:32.808+0000 7f84fd400640 1 --2- 172.21.15.131:0/3711890728 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8500155770 0x7f8500142b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:32.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:32.808+0000 7f84fe800640 1 -- 172.21.15.131:0/3711890728 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8500155770 msgr2=0x7f8500142b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:32.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:32.808+0000 7f84fe800640 1 --2- 172.21.15.131:0/3711890728 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8500155770 0x7f8500142b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:32.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:32.808+0000 7f84fe800640 1 -- 172.21.15.131:0/3711890728 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8500151d00 con 0x7f850015bc40 2024-05-26T22:51:32.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:32.808+0000 7f84fe800640 1 --2- 172.21.15.131:0/3711890728 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f850015bc40 0x7f850009aa00 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f84f800c440 tx=0x7f84f800c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:32.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:32.808+0000 7f84f6000640 1 -- 172.21.15.131:0/3711890728 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f8011790 con 0x7f850015bc40 2024-05-26T22:51:32.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:32.809+0000 7f84f6000640 1 -- 172.21.15.131:0/3711890728 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84f8017070 con 0x7f850015bc40 2024-05-26T22:51:32.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:32.809+0000 7f84f6000640 1 -- 172.21.15.131:0/3711890728 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f8011b70 con 0x7f850015bc40 2024-05-26T22:51:32.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:32.809+0000 7f8504e00640 1 -- 172.21.15.131:0/3711890728 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8500151fe0 con 0x7f850015bc40 2024-05-26T22:51:32.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:32.809+0000 7f8504e00640 1 -- 172.21.15.131:0/3711890728 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85001525a0 con 0x7f850015bc40 2024-05-26T22:51:32.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:32.810+0000 7f8504e00640 1 -- 172.21.15.131:0/3711890728 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85000695c0 con 0x7f850015bc40 2024-05-26T22:51:32.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:32.811+0000 7f84f6000640 1 -- 172.21.15.131:0/3711890728 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f84f8011d10 con 0x7f850015bc40 2024-05-26T22:51:32.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:32.812+0000 7f84f6000640 1 --2- 172.21.15.131:0/3711890728 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f84dc06be30 0x7f84dc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:32.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:32.812+0000 7f84f6000640 1 -- 172.21.15.131:0/3711890728 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5201+0+0 (secure 0 0 0) 0x7f84f808ab50 con 0x7f850015bc40 2024-05-26T22:51:32.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:32.812+0000 7f84fde00640 1 --2- 172.21.15.131:0/3711890728 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f84dc06be30 0x7f84dc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:32.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:32.813+0000 7f84fde00640 1 --2- 172.21.15.131:0/3711890728 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f84dc06be30 0x7f84dc06e2f0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f84f0034000 tx=0x7f84f002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:32.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:32.817+0000 7f84f6000640 1 -- 172.21.15.131:0/3711890728 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f84f80595c0 con 0x7f850015bc40 2024-05-26T22:51:33.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:33.001+0000 7f8504e00640 1 -- 172.21.15.131:0/3711890728 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f850009af40 con 0x7f850015bc40 2024-05-26T22:51:33.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:33.008+0000 7f84f6000640 1 -- 172.21.15.131:0/3711890728 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v27)=0 v27) ==== 122+0+0 (secure 0 0 0) 0x7f84f8058ec0 con 0x7f850015bc40 2024-05-26T22:51:33.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:33.011+0000 7f8504e00640 1 -- 172.21.15.131:0/3711890728 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f84dc06be30 msgr2=0x7f84dc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:33.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:33.011+0000 7f8504e00640 1 --2- 172.21.15.131:0/3711890728 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f84dc06be30 0x7f84dc06e2f0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f84f0034000 tx=0x7f84f002e000 comp rx=0 tx=0).stop 2024-05-26T22:51:33.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:33.011+0000 7f8504e00640 1 -- 172.21.15.131:0/3711890728 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f850015bc40 msgr2=0x7f850009aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:33.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:33.011+0000 7f8504e00640 1 --2- 172.21.15.131:0/3711890728 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f850015bc40 0x7f850009aa00 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f84f800c440 tx=0x7f84f800c900 comp rx=0 tx=0).stop 2024-05-26T22:51:33.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:33.011+0000 7f8504e00640 1 -- 172.21.15.131:0/3711890728 shutdown_connections 2024-05-26T22:51:33.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:33.011+0000 7f8504e00640 1 --2- 172.21.15.131:0/3711890728 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f84dc06be30 0x7f84dc06e2f0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:33.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:33.011+0000 7f8504e00640 1 --2- 172.21.15.131:0/3711890728 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f850015bc40 0x7f850009aa00 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:33.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:33.011+0000 7f8504e00640 1 --2- 172.21.15.131:0/3711890728 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8500155770 0x7f8500142b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:33.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:33.011+0000 7f8504e00640 1 --2- 172.21.15.131:0/3711890728 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f850006e440 0x7f8500143470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:33.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:33.011+0000 7f8504e00640 1 -- 172.21.15.131:0/3711890728 >> 172.21.15.131:0/3711890728 conn(0x7f850006bc90 msgr2=0x7f85000cdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:33.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:33.011+0000 7f8504e00640 1 -- 172.21.15.131:0/3711890728 shutdown_connections 2024-05-26T22:51:33.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:33.011+0000 7f8504e00640 1 -- 172.21.15.131:0/3711890728 wait complete. 2024-05-26T22:51:33.027 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-05-26T22:51:33.028 DEBUG:teuthology.orchestra.run.smithi131:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-26T22:51:33.133 DEBUG:teuthology.orchestra.run.smithi187:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-26T22:51:33.233 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-05-26T22:51:33.233 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-26T22:51:33.233 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-26T22:51:33.288 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-26T22:51:33.288 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-05-26T22:51:33.288 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-26T22:51:33.347 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-26T22:51:33.347 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-26T22:51:33.347 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-26T22:51:33.404 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-26T22:51:33.404 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-05-26T22:51:33.404 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-26T22:51:33.461 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-26T22:51:33.461 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-26T22:51:33.461 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-26T22:51:33.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:33.668+0000 7f2647e00640 1 Processor -- start 2024-05-26T22:51:33.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:33.668+0000 7f2647e00640 1 -- start start 2024-05-26T22:51:33.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:33.669+0000 7f2647e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f2648156750 0x7f264815fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:33.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:33.669+0000 7f2647e00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f264806e8c0 0x7f264815d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:33.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:33.669+0000 7f2647e00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2648160620 con 0x7f26480cc7f0 2024-05-26T22:51:33.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:33.669+0000 7f2647e00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f264815cbd0 con 0x7f2648156750 2024-05-26T22:51:33.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:33.669+0000 7f2647e00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f2648160900 con 0x7f264806e8c0 2024-05-26T22:51:33.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:33.670+0000 7f2647400640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f26480cc7f0 0x7f26480cdc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:43110/0 (socket says 172.21.15.131:43110) 2024-05-26T22:51:33.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:33.670+0000 7f2647400640 1 -- 172.21.15.131:0/3491696504 learned_addr learned my addr 172.21.15.131:0/3491696504 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:51:33.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:33.670+0000 7f2646a00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f2648156750 0x7f264815fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:33.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:33.670+0000 7f2646a00640 1 -- 172.21.15.131:0/3491696504 >> v2:172.21.15.187:3300/0 conn(0x7f264806e8c0 msgr2=0x7f264815d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:33.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:33.670+0000 7f263fe00640 1 --2- 172.21.15.131:0/3491696504 >> v2:172.21.15.187:3300/0 conn(0x7f264806e8c0 0x7f264815d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:33.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:33.670+0000 7f2646000640 1 -- 172.21.15.131:0/3491696504 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 54119316 0 0) 0x7f2648160620 con 0x7f26480cc7f0 2024-05-26T22:51:33.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:33.670+0000 7f2646a00640 1 --2- 172.21.15.131:0/3491696504 >> v2:172.21.15.187:3300/0 conn(0x7f264806e8c0 0x7f264815d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:33.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:33.670+0000 7f2646a00640 1 -- 172.21.15.131:0/3491696504 >> v1:172.21.15.131:6789/0 conn(0x7f26480cc7f0 legacy=0x7f26480cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:33.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:33.670+0000 7f2646a00640 1 -- 172.21.15.131:0/3491696504 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2648160be0 con 0x7f2648156750 2024-05-26T22:51:33.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:33.671+0000 7f2646a00640 1 --2- 172.21.15.131:0/3491696504 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2648156750 0x7f264815fd40 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f2648160e40 tx=0x7f263802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdb7bd265cea306c server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:33.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:33.671+0000 7f2646000640 1 -- 172.21.15.131:0/3491696504 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2638037070 con 0x7f2648156750 2024-05-26T22:51:33.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:33.671+0000 7f2646000640 1 -- 172.21.15.131:0/3491696504 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2638032440 con 0x7f2648156750 2024-05-26T22:51:33.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:33.671+0000 7f2646000640 1 -- 172.21.15.131:0/3491696504 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2638032810 con 0x7f2648156750 2024-05-26T22:51:33.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:33.672+0000 7f2647e00640 1 -- 172.21.15.131:0/3491696504 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2648156750 msgr2=0x7f264815fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:33.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:33.672+0000 7f2647e00640 1 --2- 172.21.15.131:0/3491696504 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2648156750 0x7f264815fd40 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f2648160e40 tx=0x7f263802a1a0 comp rx=0 tx=0).stop 2024-05-26T22:51:33.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:33.672+0000 7f2647e00640 1 -- 172.21.15.131:0/3491696504 shutdown_connections 2024-05-26T22:51:33.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:33.672+0000 7f2647e00640 1 --2- 172.21.15.131:0/3491696504 >> v2:172.21.15.187:3300/0 conn(0x7f264806e8c0 0x7f264815d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:33.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:33.672+0000 7f2647e00640 1 --2- 172.21.15.131:0/3491696504 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2648156750 0x7f264815fd40 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:33.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:33.672+0000 7f2647e00640 1 -- 172.21.15.131:0/3491696504 >> 172.21.15.131:0/3491696504 conn(0x7f26480cff40 msgr2=0x7f26480d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:33.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:33.672+0000 7f2647e00640 1 -- 172.21.15.131:0/3491696504 shutdown_connections 2024-05-26T22:51:33.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:33.672+0000 7f2647e00640 1 -- 172.21.15.131:0/3491696504 wait complete. 2024-05-26T22:51:33.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:33.674+0000 7f2647e00640 1 Processor -- start 2024-05-26T22:51:33.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:33.674+0000 7f2647e00640 1 -- start start 2024-05-26T22:51:33.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:33.675+0000 7f2647e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f264806e8c0 0x7f26481578d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:33.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:33.675+0000 7f2647e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f26480cc7f0 0x7f26481443d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:33.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:33.675+0000 7f2647e00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2648156750 0x7f2648144910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:33.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:33.675+0000 7f2647e00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f2648160280 con 0x7f264806e8c0 2024-05-26T22:51:33.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:33.675+0000 7f2647e00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2648156b30 con 0x7f2648156750 2024-05-26T22:51:33.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:33.675+0000 7f2647e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f2648160400 con 0x7f26480cc7f0 2024-05-26T22:51:33.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:33.675+0000 7f2646a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f264806e8c0 0x7f26481578d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:33.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:33.675+0000 7f2646a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f264806e8c0 0x7f26481578d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48368/0 (socket says 172.21.15.131:48368) 2024-05-26T22:51:33.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:33.675+0000 7f2646a00640 1 -- 172.21.15.131:0/812949677 learned_addr learned my addr 172.21.15.131:0/812949677 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:33.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:33.676+0000 7f2646a00640 1 -- 172.21.15.131:0/812949677 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f26480cc7f0 msgr2=0x7f26481443d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:33.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:33.676+0000 7f2646a00640 1 --2- 172.21.15.131:0/812949677 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f26480cc7f0 0x7f26481443d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:33.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:33.676+0000 7f2646a00640 1 -- 172.21.15.131:0/812949677 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2648156750 msgr2=0x7f2648144910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:33.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:33.676+0000 7f2646a00640 1 --2- 172.21.15.131:0/812949677 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2648156750 0x7f2648144910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:33.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:33.676+0000 7f2646a00640 1 -- 172.21.15.131:0/812949677 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2648071620 con 0x7f264806e8c0 2024-05-26T22:51:33.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:33.676+0000 7f2646a00640 1 --2- 172.21.15.131:0/812949677 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f264806e8c0 0x7f26481578d0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f2638000c00 tx=0x7f26380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:33.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:33.676+0000 7f263ea00640 1 -- 172.21.15.131:0/812949677 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2638037040 con 0x7f264806e8c0 2024-05-26T22:51:33.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:33.676+0000 7f263ea00640 1 -- 172.21.15.131:0/812949677 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2638042070 con 0x7f264806e8c0 2024-05-26T22:51:33.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:33.676+0000 7f263ea00640 1 -- 172.21.15.131:0/812949677 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2638030a50 con 0x7f264806e8c0 2024-05-26T22:51:33.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:33.676+0000 7f2647e00640 1 -- 172.21.15.131:0/812949677 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26480aefb0 con 0x7f264806e8c0 2024-05-26T22:51:33.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:33.676+0000 7f2647e00640 1 -- 172.21.15.131:0/812949677 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2648071bc0 con 0x7f264806e8c0 2024-05-26T22:51:33.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:33.677+0000 7f2647e00640 1 -- 172.21.15.131:0/812949677 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26480694e0 con 0x7f264806e8c0 2024-05-26T22:51:33.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:33.678+0000 7f263ea00640 1 -- 172.21.15.131:0/812949677 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2638030bf0 con 0x7f264806e8c0 2024-05-26T22:51:33.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:33.679+0000 7f263ea00640 1 --2- 172.21.15.131:0/812949677 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f262406be60 0x7f262406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:33.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:33.679+0000 7f263ea00640 1 -- 172.21.15.131:0/812949677 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5201+0+0 (secure 0 0 0) 0x7f26380ab630 con 0x7f264806e8c0 2024-05-26T22:51:33.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:33.680+0000 7f263fe00640 1 --2- 172.21.15.131:0/812949677 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f262406be60 0x7f262406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:33.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:33.680+0000 7f263fe00640 1 --2- 172.21.15.131:0/812949677 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f262406be60 0x7f262406e320 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f2630006fd0 tx=0x7f2630008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:33.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:33.684+0000 7f263ea00640 1 -- 172.21.15.131:0/812949677 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f2638045360 con 0x7f264806e8c0 2024-05-26T22:51:33.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:33.919+0000 7f2647e00640 1 -- 172.21.15.131:0/812949677 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f264815f020 con 0x7f264806e8c0 2024-05-26T22:51:33.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:33.920+0000 7f263ea00640 1 -- 172.21.15.131:0/812949677 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+3878 (secure 0 0 0) 0x7f263807a260 con 0x7f264806e8c0 2024-05-26T22:51:33.919 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:51:33.920 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":42,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4727,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/3668228040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6833","nonce":3668228040},{"type":"v1","addr":"172.21.15.187:6835","nonce":3668228040}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":4793,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6834/296490019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":296490019},{"type":"v1","addr":"172.21.15.187:6834","nonce":296490019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":6261,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/1945237906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1945237906},{"type":"v1","addr":"172.21.15.131:6835","nonce":1945237906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":6264,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/3019892676","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":3019892676},{"type":"v1","addr":"172.21.15.131:6837","nonce":3019892676}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42}],"filesystems":[]} 2024-05-26T22:51:33.920 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 42 2024-05-26T22:51:33.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:33.922+0000 7f2647e00640 1 -- 172.21.15.131:0/812949677 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f262406be60 msgr2=0x7f262406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:33.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:33.922+0000 7f2647e00640 1 --2- 172.21.15.131:0/812949677 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f262406be60 0x7f262406e320 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f2630006fd0 tx=0x7f2630008040 comp rx=0 tx=0).stop 2024-05-26T22:51:33.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:33.922+0000 7f2647e00640 1 -- 172.21.15.131:0/812949677 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f264806e8c0 msgr2=0x7f26481578d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:33.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:33.922+0000 7f2647e00640 1 --2- 172.21.15.131:0/812949677 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f264806e8c0 0x7f26481578d0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f2638000c00 tx=0x7f26380046c0 comp rx=0 tx=0).stop 2024-05-26T22:51:33.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:33.922+0000 7f2647e00640 1 -- 172.21.15.131:0/812949677 shutdown_connections 2024-05-26T22:51:33.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:33.922+0000 7f2647e00640 1 --2- 172.21.15.131:0/812949677 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f26480cc7f0 0x7f26481443d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:33.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:33.922+0000 7f2647e00640 1 --2- 172.21.15.131:0/812949677 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f262406be60 0x7f262406e320 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:33.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:33.922+0000 7f2647e00640 1 --2- 172.21.15.131:0/812949677 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2648156750 0x7f2648144910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:33.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:33.922+0000 7f2647e00640 1 --2- 172.21.15.131:0/812949677 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f264806e8c0 0x7f26481578d0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:33.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:33.922+0000 7f2647e00640 1 -- 172.21.15.131:0/812949677 >> 172.21.15.131:0/812949677 conn(0x7f26480cff40 msgr2=0x7f264815d900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:33.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:33.922+0000 7f2647e00640 1 -- 172.21.15.131:0/812949677 shutdown_connections 2024-05-26T22:51:33.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:33.922+0000 7f2647e00640 1 -- 172.21.15.131:0/812949677 wait complete. 2024-05-26T22:51:33.942 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-26T22:51:33.943 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-26T22:51:33.943 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-26T22:51:33.943 INFO:teuthology.orchestra.run:waiting for 300 2024-05-26T22:51:33.943 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-26T22:51:33.943 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-26T22:51:33.944 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-26T22:51:33.944 INFO:teuthology.orchestra.run:waiting for 300 2024-05-26T22:51:33.944 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-26T22:51:33.944 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-26T22:51:33.944 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-26T22:51:33.944 INFO:teuthology.orchestra.run:waiting for 300 2024-05-26T22:51:33.944 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-26T22:51:33.944 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-26T22:51:33.945 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-26T22:51:33.945 INFO:teuthology.orchestra.run:waiting for 300 2024-05-26T22:51:33.962 INFO:tasks.ceph.mds.c:Stopped 2024-05-26T22:51:33.962 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-26T22:51:33.964 INFO:tasks.ceph.mds.a:Stopped 2024-05-26T22:51:33.964 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-26T22:51:33.969 INFO:tasks.ceph.mds.b:Stopped 2024-05-26T22:51:33.969 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-26T22:51:33.973 INFO:tasks.ceph.mds.d:Stopped 2024-05-26T22:51:33.973 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-26T22:51:34.008 INFO:tasks.ceph.mds.c:Started 2024-05-26T22:51:34.009 DEBUG:teuthology.parallel:result is None 2024-05-26T22:51:34.013 INFO:tasks.ceph.mds.a:Started 2024-05-26T22:51:34.013 DEBUG:teuthology.parallel:result is None 2024-05-26T22:51:34.020 INFO:tasks.ceph.mds.b:Started 2024-05-26T22:51:34.020 DEBUG:teuthology.parallel:result is None 2024-05-26T22:51:34.026 INFO:tasks.ceph.mds.d:Started 2024-05-26T22:51:34.027 DEBUG:teuthology.parallel:result is None 2024-05-26T22:51:34.027 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-26T22:51:34.149 INFO:tasks.ceph.mds.c.smithi131.stdout:starting mds.c at 2024-05-26T22:51:34.150 INFO:tasks.ceph.mds.a.smithi131.stdout:starting mds.a at 2024-05-26T22:51:34.168 INFO:tasks.ceph.mds.d.smithi187.stdout:starting mds.d at 2024-05-26T22:51:34.172 INFO:tasks.ceph.mds.b.smithi187.stdout:starting mds.b at 2024-05-26T22:51:34.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.195+0000 7f8421e00640 1 Processor -- start 2024-05-26T22:51:34.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.195+0000 7f8421e00640 1 -- start start 2024-05-26T22:51:34.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.196+0000 7f8421e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f841c155770 0x7f841c15b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:34.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.196+0000 7f8421e00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f841c15bc40 0x7f841c06dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:34.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.196+0000 7f8421e00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f841c06e680 con 0x7f841c06e270 2024-05-26T22:51:34.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.196+0000 7f8421e00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f841c0735b0 con 0x7f841c155770 2024-05-26T22:51:34.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.196+0000 7f8421e00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f841c15c1c0 con 0x7f841c15bc40 2024-05-26T22:51:34.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.196+0000 7f841aa00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f841c155770 0x7f841c15b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:34.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.196+0000 7f841aa00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f841c155770 0x7f841c15b700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48408/0 (socket says 172.21.15.131:48408) 2024-05-26T22:51:34.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.196+0000 7f841aa00640 1 -- 172.21.15.131:0/1844362019 learned_addr learned my addr 172.21.15.131:0/1844362019 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:34.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.196+0000 7f841a000640 1 --2- 172.21.15.131:0/1844362019 >> v2:172.21.15.187:3300/0 conn(0x7f841c15bc40 0x7f841c06dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:34.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.197+0000 7f841aa00640 1 -- 172.21.15.131:0/1844362019 >> v2:172.21.15.187:3300/0 conn(0x7f841c15bc40 msgr2=0x7f841c06dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:34.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.197+0000 7f841aa00640 1 --2- 172.21.15.131:0/1844362019 >> v2:172.21.15.187:3300/0 conn(0x7f841c15bc40 0x7f841c06dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:34.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.197+0000 7f841aa00640 1 -- 172.21.15.131:0/1844362019 >> v1:172.21.15.187:6789/0 conn(0x7f841c06e270 legacy=0x7f841c0ceba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:34.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.197+0000 7f841aa00640 1 -- 172.21.15.131:0/1844362019 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f841c15c4a0 con 0x7f841c155770 2024-05-26T22:51:34.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.198+0000 7f841aa00640 1 --2- 172.21.15.131:0/1844362019 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f841c155770 0x7f841c15b700 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f841c15c0d0 tx=0x7f840c02a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b08673a7d40f28d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:34.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.198+0000 7f8419600640 1 -- 172.21.15.131:0/1844362019 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f840c037070 con 0x7f841c155770 2024-05-26T22:51:34.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.198+0000 7f8419600640 1 -- 172.21.15.131:0/1844362019 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f840c02ae50 con 0x7f841c155770 2024-05-26T22:51:34.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.198+0000 7f8419600640 1 -- 172.21.15.131:0/1844362019 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f840c032860 con 0x7f841c155770 2024-05-26T22:51:34.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.198+0000 7f8421e00640 1 -- 172.21.15.131:0/1844362019 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f841c155770 msgr2=0x7f841c15b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:34.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.198+0000 7f8421e00640 1 --2- 172.21.15.131:0/1844362019 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f841c155770 0x7f841c15b700 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f841c15c0d0 tx=0x7f840c02a300 comp rx=0 tx=0).stop 2024-05-26T22:51:34.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.199+0000 7f8421e00640 1 -- 172.21.15.131:0/1844362019 shutdown_connections 2024-05-26T22:51:34.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.199+0000 7f8421e00640 1 --2- 172.21.15.131:0/1844362019 >> v2:172.21.15.187:3300/0 conn(0x7f841c15bc40 0x7f841c06dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:34.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.199+0000 7f8421e00640 1 --2- 172.21.15.131:0/1844362019 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f841c155770 0x7f841c15b700 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:34.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.199+0000 7f8421e00640 1 -- 172.21.15.131:0/1844362019 >> 172.21.15.131:0/1844362019 conn(0x7f841c06bc90 msgr2=0x7f841c0b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:34.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.199+0000 7f8421e00640 1 -- 172.21.15.131:0/1844362019 shutdown_connections 2024-05-26T22:51:34.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.199+0000 7f8421e00640 1 -- 172.21.15.131:0/1844362019 wait complete. 2024-05-26T22:51:34.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.200+0000 7f8421e00640 1 Processor -- start 2024-05-26T22:51:34.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.200+0000 7f8421e00640 1 -- start start 2024-05-26T22:51:34.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.201+0000 7f8421e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f841c06e270 0x7f841c1f8190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:34.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.201+0000 7f8421e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f841c155770 0x7f841c15a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:34.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.201+0000 7f8421e00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f841c15bc40 0x7f841c0cc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:34.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.201+0000 7f8421e00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f841c155b50 con 0x7f841c06e270 2024-05-26T22:51:34.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.201+0000 7f8421e00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f841c072080 con 0x7f841c15bc40 2024-05-26T22:51:34.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.201+0000 7f8421e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f841c15c8c0 con 0x7f841c155770 2024-05-26T22:51:34.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.201+0000 7f841aa00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f841c06e270 0x7f841c1f8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:34.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.202+0000 7f841aa00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f841c06e270 0x7f841c1f8190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48416/0 (socket says 172.21.15.131:48416) 2024-05-26T22:51:34.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.202+0000 7f841aa00640 1 -- 172.21.15.131:0/2886134338 learned_addr learned my addr 172.21.15.131:0/2886134338 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:34.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.202+0000 7f841a000640 1 --2- 172.21.15.131:0/2886134338 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f841c155770 0x7f841c15a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:34.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.202+0000 7f841b400640 1 --2- 172.21.15.131:0/2886134338 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f841c15bc40 0x7f841c0cc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:34.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.202+0000 7f841aa00640 1 -- 172.21.15.131:0/2886134338 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f841c155770 msgr2=0x7f841c15a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:34.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.202+0000 7f841aa00640 1 --2- 172.21.15.131:0/2886134338 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f841c155770 0x7f841c15a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:34.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.202+0000 7f841aa00640 1 -- 172.21.15.131:0/2886134338 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f841c15bc40 msgr2=0x7f841c0cc6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:34.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.202+0000 7f841aa00640 1 --2- 172.21.15.131:0/2886134338 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f841c15bc40 0x7f841c0cc6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:34.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.202+0000 7f841aa00640 1 -- 172.21.15.131:0/2886134338 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f841c143830 con 0x7f841c06e270 2024-05-26T22:51:34.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.202+0000 7f841b400640 1 --2- 172.21.15.131:0/2886134338 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f841c15bc40 0x7f841c0cc6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-26T22:51:34.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.202+0000 7f841aa00640 1 --2- 172.21.15.131:0/2886134338 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f841c06e270 0x7f841c1f8190 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f840c02a340 tx=0x7f840c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:34.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.202+0000 7f8413400640 1 -- 172.21.15.131:0/2886134338 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f840c041070 con 0x7f841c06e270 2024-05-26T22:51:34.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.203+0000 7f8413400640 1 -- 172.21.15.131:0/2886134338 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f840c03c680 con 0x7f841c06e270 2024-05-26T22:51:34.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.203+0000 7f8413400640 1 -- 172.21.15.131:0/2886134338 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f840c037040 con 0x7f841c06e270 2024-05-26T22:51:34.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.203+0000 7f8421e00640 1 -- 172.21.15.131:0/2886134338 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f841c143b10 con 0x7f841c06e270 2024-05-26T22:51:34.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.203+0000 7f8421e00640 1 -- 172.21.15.131:0/2886134338 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f841c1440d0 con 0x7f841c06e270 2024-05-26T22:51:34.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.204+0000 7f8421e00640 1 -- 172.21.15.131:0/2886134338 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f841c074e90 con 0x7f841c06e270 2024-05-26T22:51:34.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.205+0000 7f8413400640 1 -- 172.21.15.131:0/2886134338 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f840c044070 con 0x7f841c06e270 2024-05-26T22:51:34.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.206+0000 7f8413400640 1 --2- 172.21.15.131:0/2886134338 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f83fc06be30 0x7f83fc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:34.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.206+0000 7f8413400640 1 -- 172.21.15.131:0/2886134338 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5201+0+0 (secure 0 0 0) 0x7f840c0ab490 con 0x7f841c06e270 2024-05-26T22:51:34.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.206+0000 7f841a000640 1 --2- 172.21.15.131:0/2886134338 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f83fc06be30 0x7f83fc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:34.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.207+0000 7f841a000640 1 --2- 172.21.15.131:0/2886134338 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f83fc06be30 0x7f83fc06e2f0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f841c09ab20 tx=0x7f8408009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:34.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.211+0000 7f8413400640 1 -- 172.21.15.131:0/2886134338 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f840c079a60 con 0x7f841c06e270 2024-05-26T22:51:34.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.421+0000 7f8421e00640 1 -- 172.21.15.131:0/2886134338 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f841c0695c0 con 0x7f841c06e270 2024-05-26T22:51:34.598 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.598+0000 7f8413400640 1 -- 172.21.15.131:0/2886134338 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v52) ==== 97+0+0 (secure 0 0 0) 0x7f840c07dd60 con 0x7f841c06e270 2024-05-26T22:51:34.598 INFO:teuthology.orchestra.run.smithi131.stderr: removed all blocklist entries 2024-05-26T22:51:34.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.602+0000 7f8421e00640 1 -- 172.21.15.131:0/2886134338 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f83fc06be30 msgr2=0x7f83fc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:34.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.602+0000 7f8421e00640 1 --2- 172.21.15.131:0/2886134338 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f83fc06be30 0x7f83fc06e2f0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f841c09ab20 tx=0x7f8408009190 comp rx=0 tx=0).stop 2024-05-26T22:51:34.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.602+0000 7f8421e00640 1 -- 172.21.15.131:0/2886134338 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f841c06e270 msgr2=0x7f841c1f8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:34.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.602+0000 7f8421e00640 1 --2- 172.21.15.131:0/2886134338 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f841c06e270 0x7f841c1f8190 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f840c02a340 tx=0x7f840c0046c0 comp rx=0 tx=0).stop 2024-05-26T22:51:34.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.603+0000 7f8421e00640 1 -- 172.21.15.131:0/2886134338 shutdown_connections 2024-05-26T22:51:34.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.603+0000 7f8421e00640 1 --2- 172.21.15.131:0/2886134338 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f841c155770 0x7f841c15a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:34.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.603+0000 7f8421e00640 1 --2- 172.21.15.131:0/2886134338 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f83fc06be30 0x7f83fc06e2f0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:34.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.603+0000 7f8421e00640 1 --2- 172.21.15.131:0/2886134338 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f841c15bc40 0x7f841c0cc6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:34.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.603+0000 7f8421e00640 1 --2- 172.21.15.131:0/2886134338 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f841c06e270 0x7f841c1f8190 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:34.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.603+0000 7f8421e00640 1 -- 172.21.15.131:0/2886134338 >> 172.21.15.131:0/2886134338 conn(0x7f841c06bc90 msgr2=0x7f841c0b1600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:34.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.603+0000 7f8421e00640 1 -- 172.21.15.131:0/2886134338 shutdown_connections 2024-05-26T22:51:34.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.603+0000 7f8421e00640 1 -- 172.21.15.131:0/2886134338 wait complete. 2024-05-26T22:51:34.626 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-26T22:51:34.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.820+0000 7f52d0a00640 1 Processor -- start 2024-05-26T22:51:34.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.820+0000 7f52d0a00640 1 -- start start 2024-05-26T22:51:34.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.822+0000 7f52d0a00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f52cc06e8c0 0x7f52cc15d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:34.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.822+0000 7f52d0a00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f52cc0cc7f0 0x7f52cc0ccca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:34.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.822+0000 7f52d0a00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52cc160620 con 0x7f52cc156750 2024-05-26T22:51:34.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.822+0000 7f52d0a00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f52cc15cbd0 con 0x7f52cc06e8c0 2024-05-26T22:51:34.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.822+0000 7f52d0a00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f52cc160900 con 0x7f52cc0cc7f0 2024-05-26T22:51:34.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.822+0000 7f52c1e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f52cc06e8c0 0x7f52cc15d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:34.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.822+0000 7f52c1e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f52cc06e8c0 0x7f52cc15d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33370/0 (socket says 172.21.15.131:33370) 2024-05-26T22:51:34.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.822+0000 7f52c1e00640 1 -- 172.21.15.131:0/3447667904 learned_addr learned my addr 172.21.15.131:0/3447667904 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:34.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.823+0000 7f52c1e00640 1 -- 172.21.15.131:0/3447667904 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f52cc0cc7f0 msgr2=0x7f52cc0ccca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:34.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.822+0000 7f52cb200640 1 --2- 172.21.15.131:0/3447667904 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f52cc0cc7f0 0x7f52cc0ccca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:34.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.823+0000 7f52c1e00640 1 --2- 172.21.15.131:0/3447667904 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f52cc0cc7f0 0x7f52cc0ccca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:34.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.823+0000 7f52c1e00640 1 -- 172.21.15.131:0/3447667904 >> v1:172.21.15.187:6789/0 conn(0x7f52cc156750 legacy=0x7f52cc15fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:34.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.823+0000 7f52c1e00640 1 -- 172.21.15.131:0/3447667904 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52cc160be0 con 0x7f52cc06e8c0 2024-05-26T22:51:34.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.823+0000 7f52cb200640 1 --2- 172.21.15.131:0/3447667904 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f52cc0cc7f0 0x7f52cc0ccca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:51:34.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.824+0000 7f52c1e00640 1 --2- 172.21.15.131:0/3447667904 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f52cc06e8c0 0x7f52cc15d060 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f52b4009870 tx=0x7f52b402a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=636de8287b938371 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:34.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.824+0000 7f52c9e00640 1 -- 172.21.15.131:0/3447667904 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b4037070 con 0x7f52cc06e8c0 2024-05-26T22:51:34.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.824+0000 7f52c9e00640 1 -- 172.21.15.131:0/3447667904 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52b4032440 con 0x7f52cc06e8c0 2024-05-26T22:51:34.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.824+0000 7f52c9e00640 1 -- 172.21.15.131:0/3447667904 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b4032810 con 0x7f52cc06e8c0 2024-05-26T22:51:34.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.825+0000 7f52d0a00640 1 -- 172.21.15.131:0/3447667904 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f52cc06e8c0 msgr2=0x7f52cc15d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:34.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.825+0000 7f52d0a00640 1 --2- 172.21.15.131:0/3447667904 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f52cc06e8c0 0x7f52cc15d060 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f52b4009870 tx=0x7f52b402a200 comp rx=0 tx=0).stop 2024-05-26T22:51:34.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.825+0000 7f52d0a00640 1 -- 172.21.15.131:0/3447667904 shutdown_connections 2024-05-26T22:51:34.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.825+0000 7f52d0a00640 1 --2- 172.21.15.131:0/3447667904 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f52cc0cc7f0 0x7f52cc0ccca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:34.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.825+0000 7f52d0a00640 1 --2- 172.21.15.131:0/3447667904 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f52cc06e8c0 0x7f52cc15d060 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:34.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.825+0000 7f52d0a00640 1 -- 172.21.15.131:0/3447667904 >> 172.21.15.131:0/3447667904 conn(0x7f52cc0cff40 msgr2=0x7f52cc0d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:34.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.825+0000 7f52d0a00640 1 -- 172.21.15.131:0/3447667904 shutdown_connections 2024-05-26T22:51:34.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.825+0000 7f52d0a00640 1 -- 172.21.15.131:0/3447667904 wait complete. 2024-05-26T22:51:34.826 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.827+0000 7f52d0a00640 1 Processor -- start 2024-05-26T22:51:34.826 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.827+0000 7f52d0a00640 1 -- start start 2024-05-26T22:51:34.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.828+0000 7f52d0a00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f52cc06e8c0 0x7f52cc0ae280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:34.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.828+0000 7f52d0a00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f52cc0cc7f0 0x7f52cc0ae7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:34.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.828+0000 7f52d0a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f52cc156750 0x7f52cc071c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:34.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.828+0000 7f52d0a00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f52cc15d5a0 con 0x7f52cc156750 2024-05-26T22:51:34.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.828+0000 7f52d0a00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f52cc156b30 con 0x7f52cc06e8c0 2024-05-26T22:51:34.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.828+0000 7f52d0a00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f52cc15d720 con 0x7f52cc0cc7f0 2024-05-26T22:51:34.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.828+0000 7f52cb200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f52cc156750 0x7f52cc071c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:34.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.828+0000 7f52cb200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f52cc156750 0x7f52cc071c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33372/0 (socket says 172.21.15.131:33372) 2024-05-26T22:51:34.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.828+0000 7f52cb200640 1 -- 172.21.15.131:0/2339286269 learned_addr learned my addr 172.21.15.131:0/2339286269 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:34.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.828+0000 7f52c1e00640 1 --2- 172.21.15.131:0/2339286269 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f52cc0cc7f0 0x7f52cc0ae7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:34.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.828+0000 7f52ca800640 1 --2- 172.21.15.131:0/2339286269 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f52cc06e8c0 0x7f52cc0ae280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:34.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.829+0000 7f52cb200640 1 -- 172.21.15.131:0/2339286269 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f52cc0cc7f0 msgr2=0x7f52cc0ae7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:34.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.829+0000 7f52cb200640 1 --2- 172.21.15.131:0/2339286269 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f52cc0cc7f0 0x7f52cc0ae7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:34.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.829+0000 7f52cb200640 1 -- 172.21.15.131:0/2339286269 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f52cc06e8c0 msgr2=0x7f52cc0ae280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:34.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.829+0000 7f52cb200640 1 --2- 172.21.15.131:0/2339286269 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f52cc06e8c0 0x7f52cc0ae280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:34.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.829+0000 7f52cb200640 1 -- 172.21.15.131:0/2339286269 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52cc0afb90 con 0x7f52cc156750 2024-05-26T22:51:34.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.829+0000 7f52cb200640 1 --2- 172.21.15.131:0/2339286269 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f52cc156750 0x7f52cc071c30 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f52c400cef0 tx=0x7f52c400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:34.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.829+0000 7f52c3400640 1 -- 172.21.15.131:0/2339286269 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52c401a070 con 0x7f52cc156750 2024-05-26T22:51:34.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.829+0000 7f52c3400640 1 -- 172.21.15.131:0/2339286269 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52c400d860 con 0x7f52cc156750 2024-05-26T22:51:34.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.829+0000 7f52d0a00640 1 -- 172.21.15.131:0/2339286269 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52cc141980 con 0x7f52cc156750 2024-05-26T22:51:34.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.829+0000 7f52c3400640 1 -- 172.21.15.131:0/2339286269 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52c400dc50 con 0x7f52cc156750 2024-05-26T22:51:34.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.829+0000 7f52d0a00640 1 -- 172.21.15.131:0/2339286269 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52cc0af0d0 con 0x7f52cc156750 2024-05-26T22:51:34.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.831+0000 7f52d0a00640 1 -- 172.21.15.131:0/2339286269 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52cc1e6280 con 0x7f52cc156750 2024-05-26T22:51:34.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.831+0000 7f52c3400640 1 -- 172.21.15.131:0/2339286269 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f52c401d070 con 0x7f52cc156750 2024-05-26T22:51:34.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.832+0000 7f52c3400640 1 --2- 172.21.15.131:0/2339286269 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f52a806be30 0x7f52a806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:34.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.832+0000 7f52c3400640 1 -- 172.21.15.131:0/2339286269 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 4857+0+0 (secure 0 0 0) 0x7f52c408b490 con 0x7f52cc156750 2024-05-26T22:51:34.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.833+0000 7f52ca800640 1 --2- 172.21.15.131:0/2339286269 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f52a806be30 0x7f52a806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:34.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.833+0000 7f52ca800640 1 --2- 172.21.15.131:0/2339286269 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f52a806be30 0x7f52a806e2f0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f52bc006fd0 tx=0x7f52bc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:34.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:34.837+0000 7f52c3400640 1 -- 172.21.15.131:0/2339286269 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f52cc1e6280 con 0x7f52cc156750 2024-05-26T22:51:35.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.104+0000 7f52d0a00640 1 -- 172.21.15.131:0/2339286269 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f52cc0694e0 con 0x7f52cc156750 2024-05-26T22:51:35.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.105+0000 7f52c3400640 1 -- 172.21.15.131:0/2339286269 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) ==== 80+0+5304 (secure 0 0 0) 0x7f52c405a1b0 con 0x7f52cc156750 2024-05-26T22:51:35.105 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:51:35.105 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-26T22:51:35.105 INFO:teuthology.orchestra.run.smithi131.stdout: "auth_dump": [ 2024-05-26T22:51:35.105 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:51:35.105 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mds.a", 2024-05-26T22:51:35.106 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD1u1NmTQf1DhAAdvEt7wP0IgKhNHOqjVAllA==", 2024-05-26T22:51:35.106 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:51:35.106 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-05-26T22:51:35.106 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2024-05-26T22:51:35.106 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2024-05-26T22:51:35.106 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-05-26T22:51:35.106 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:51:35.106 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:35.106 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:51:35.106 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mds.b", 2024-05-26T22:51:35.106 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD1u1NmHNZ8KxAAPgQ/ojbl+EDfPhqrkg7LPQ==", 2024-05-26T22:51:35.106 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:51:35.107 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-05-26T22:51:35.107 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2024-05-26T22:51:35.107 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2024-05-26T22:51:35.107 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-05-26T22:51:35.107 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:51:35.107 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:35.107 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:51:35.107 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mds.c", 2024-05-26T22:51:35.107 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD1u1NmFjCHHRAAqix3Ujfsq2tnSU5+d/pSYA==", 2024-05-26T22:51:35.107 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:51:35.107 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-05-26T22:51:35.107 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2024-05-26T22:51:35.107 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2024-05-26T22:51:35.107 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-05-26T22:51:35.108 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:51:35.108 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:35.108 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:51:35.108 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mds.d", 2024-05-26T22:51:35.108 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD1u1NmM4LPORAAhtQtr+Vfpf7vdVwjSW6xtA==", 2024-05-26T22:51:35.108 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:51:35.108 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-05-26T22:51:35.108 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2024-05-26T22:51:35.108 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2024-05-26T22:51:35.108 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-05-26T22:51:35.108 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:51:35.109 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:35.109 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:51:35.109 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.0", 2024-05-26T22:51:35.109 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD5u1Nm+w1LNhAAsJLQkiKSh3KkMymlrZkcrw==", 2024-05-26T22:51:35.109 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:51:35.109 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-05-26T22:51:35.109 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-05-26T22:51:35.109 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-05-26T22:51:35.109 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:51:35.109 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:35.109 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:51:35.109 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.1", 2024-05-26T22:51:35.109 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD9u1Nm4c8RFBAAcn9HQ+izML9rujJ/2/Q/BQ==", 2024-05-26T22:51:35.109 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:51:35.110 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-05-26T22:51:35.110 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-05-26T22:51:35.110 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-05-26T22:51:35.110 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:51:35.110 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:35.110 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:51:35.110 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.2", 2024-05-26T22:51:35.110 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAAvFNmR5wmLRAAYEBAXPBZFq67iARZnwF3iw==", 2024-05-26T22:51:35.110 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:51:35.110 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-05-26T22:51:35.110 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-05-26T22:51:35.110 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-05-26T22:51:35.110 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:51:35.111 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:35.111 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:51:35.111 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.3", 2024-05-26T22:51:35.111 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAEvFNmJ4G5CxAAnTbP8DinhmuvGNo1VBVzjQ==", 2024-05-26T22:51:35.111 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:51:35.111 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-05-26T22:51:35.111 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-05-26T22:51:35.112 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-05-26T22:51:35.112 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:51:35.112 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:35.112 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:51:35.112 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.4", 2024-05-26T22:51:35.112 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAKvFNmcw5rGhAAvnivP6kt3vcN9UPm/svKqw==", 2024-05-26T22:51:35.112 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:51:35.112 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-05-26T22:51:35.112 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-05-26T22:51:35.112 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-05-26T22:51:35.113 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:51:35.113 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:35.113 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:51:35.113 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.5", 2024-05-26T22:51:35.113 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQANvFNmTqlzNhAAGfrSvNNHYKGOBzRXo0TEdw==", 2024-05-26T22:51:35.113 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:51:35.113 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-05-26T22:51:35.113 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-05-26T22:51:35.114 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-05-26T22:51:35.114 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:51:35.114 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:35.114 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:51:35.114 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.6", 2024-05-26T22:51:35.114 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQARvFNmpzd6FRAAgi2RYlQPvfHNe0iP4bJrwg==", 2024-05-26T22:51:35.115 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:51:35.115 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-05-26T22:51:35.115 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-05-26T22:51:35.115 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-05-26T22:51:35.115 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:51:35.115 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:35.115 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:51:35.115 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.7", 2024-05-26T22:51:35.116 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAUvFNmAkC8LhAAsmkKF4eloxdpCi/mrDtzqA==", 2024-05-26T22:51:35.116 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:51:35.116 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-05-26T22:51:35.117 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-05-26T22:51:35.117 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-05-26T22:51:35.117 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:51:35.117 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:35.117 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:51:35.117 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.0", 2024-05-26T22:51:35.117 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD2u1NmSGVWChAAH/wq3iaUs6ZELRhLKhmURQ==", 2024-05-26T22:51:35.117 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:51:35.117 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-05-26T22:51:35.118 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow r", 2024-05-26T22:51:35.118 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-05-26T22:51:35.118 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:51:35.118 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:35.118 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:51:35.118 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.1", 2024-05-26T22:51:35.118 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD2u1NmnA9CHRAA2XMmZQxysKweHNykiQUEeA==", 2024-05-26T22:51:35.118 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:51:35.119 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-05-26T22:51:35.119 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow r", 2024-05-26T22:51:35.119 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow rw", 2024-05-26T22:51:35.119 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow rwx" 2024-05-26T22:51:35.119 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:51:35.119 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:35.119 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:51:35.119 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.2", 2024-05-26T22:51:35.119 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD2u1NmGEI9ExAAvJn1LRwmALkD7zh/AP07vQ==", 2024-05-26T22:51:35.120 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:51:35.120 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-05-26T22:51:35.120 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow r", 2024-05-26T22:51:35.120 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow rw", 2024-05-26T22:51:35.120 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow rwx" 2024-05-26T22:51:35.120 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:51:35.120 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:35.120 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:51:35.120 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.3", 2024-05-26T22:51:35.121 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD2u1NmIHOtJhAAJCUm0kGqWu4rWVLsDkW6Eg==", 2024-05-26T22:51:35.121 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:51:35.121 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-05-26T22:51:35.121 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow r", 2024-05-26T22:51:35.121 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow rw", 2024-05-26T22:51:35.121 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow rwx" 2024-05-26T22:51:35.121 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:51:35.121 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:35.121 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:51:35.122 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.admin", 2024-05-26T22:51:35.122 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD0u1NmLRGoCBAAhDeVJV14MTAIdUte067jig==", 2024-05-26T22:51:35.122 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:51:35.122 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow *", 2024-05-26T22:51:35.122 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2024-05-26T22:51:35.122 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2024-05-26T22:51:35.122 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-05-26T22:51:35.122 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:51:35.122 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:35.123 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:51:35.123 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mgr.x", 2024-05-26T22:51:35.123 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD1u1NmaWhyBBAAlU7Lq6HMjkCPu0kztJIAXQ==", 2024-05-26T22:51:35.123 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:51:35.123 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow *", 2024-05-26T22:51:35.123 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile mgr", 2024-05-26T22:51:35.123 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-05-26T22:51:35.123 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:51:35.123 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:51:35.124 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:51:35.124 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mgr.y", 2024-05-26T22:51:35.124 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD0u1NmX1RhNhAAo5mEOGzUyrdwFlB/XOXKnw==", 2024-05-26T22:51:35.124 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:51:35.124 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow *", 2024-05-26T22:51:35.124 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile mgr", 2024-05-26T22:51:35.124 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-05-26T22:51:35.124 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:51:35.125 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:51:35.125 INFO:teuthology.orchestra.run.smithi131.stdout: ] 2024-05-26T22:51:35.125 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-26T22:51:35.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.108+0000 7f52d0a00640 1 -- 172.21.15.131:0/2339286269 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f52a806be30 msgr2=0x7f52a806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:35.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.108+0000 7f52d0a00640 1 --2- 172.21.15.131:0/2339286269 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f52a806be30 0x7f52a806e2f0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f52bc006fd0 tx=0x7f52bc008040 comp rx=0 tx=0).stop 2024-05-26T22:51:35.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.108+0000 7f52d0a00640 1 -- 172.21.15.131:0/2339286269 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f52cc156750 msgr2=0x7f52cc071c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:35.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.108+0000 7f52d0a00640 1 --2- 172.21.15.131:0/2339286269 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f52cc156750 0x7f52cc071c30 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f52c400cef0 tx=0x7f52c400a710 comp rx=0 tx=0).stop 2024-05-26T22:51:35.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.108+0000 7f52d0a00640 1 -- 172.21.15.131:0/2339286269 shutdown_connections 2024-05-26T22:51:35.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.108+0000 7f52d0a00640 1 --2- 172.21.15.131:0/2339286269 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f52cc0cc7f0 0x7f52cc0ae7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:35.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.108+0000 7f52d0a00640 1 --2- 172.21.15.131:0/2339286269 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f52a806be30 0x7f52a806e2f0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:35.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.108+0000 7f52d0a00640 1 --2- 172.21.15.131:0/2339286269 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f52cc156750 0x7f52cc071c30 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:35.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.108+0000 7f52d0a00640 1 --2- 172.21.15.131:0/2339286269 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f52cc06e8c0 0x7f52cc0ae280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:35.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.108+0000 7f52d0a00640 1 -- 172.21.15.131:0/2339286269 >> 172.21.15.131:0/2339286269 conn(0x7f52cc0cff40 msgr2=0x7f52cc0b4bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:35.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.108+0000 7f52d0a00640 1 -- 172.21.15.131:0/2339286269 shutdown_connections 2024-05-26T22:51:35.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.108+0000 7f52d0a00640 1 -- 172.21.15.131:0/2339286269 wait complete. 2024-05-26T22:51:35.130 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-26T22:51:35.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.325+0000 7ff08f200640 1 Processor -- start 2024-05-26T22:51:35.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.325+0000 7ff08f200640 1 -- start start 2024-05-26T22:51:35.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.326+0000 7ff08f200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff08806e8c0 0x7ff08815d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:35.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.327+0000 7ff08f200640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff088160620 con 0x7ff0880cc7f0 2024-05-26T22:51:35.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.327+0000 7ff08f200640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff088160900 con 0x7ff088156750 2024-05-26T22:51:35.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.327+0000 7ff08f200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff08815cbd0 con 0x7ff08806e8c0 2024-05-26T22:51:35.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.327+0000 7ff08e800640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7ff0880cc7f0 0x7ff0880cdc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:35736/0 (socket says 172.21.15.131:35736) 2024-05-26T22:51:35.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.327+0000 7ff08e800640 1 -- 172.21.15.131:0/3554036115 learned_addr learned my addr 172.21.15.131:0/3554036115 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:51:35.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.327+0000 7ff08d400640 1 --2- 172.21.15.131:0/3554036115 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff08806e8c0 0x7ff08815d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:35.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.327+0000 7ff08ca00640 1 -- 172.21.15.131:0/3554036115 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 996550629 0 0) 0x7ff088160620 con 0x7ff0880cc7f0 2024-05-26T22:51:35.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.327+0000 7ff08ca00640 1 -- 172.21.15.131:0/3554036115 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff070003520 con 0x7ff0880cc7f0 2024-05-26T22:51:35.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.328+0000 7ff08ca00640 1 -- 172.21.15.131:0/3554036115 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2007854503 0 0) 0x7ff070003520 con 0x7ff0880cc7f0 2024-05-26T22:51:35.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.328+0000 7ff08ca00640 1 -- 172.21.15.131:0/3554036115 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff08806e8c0 msgr2=0x7ff08815d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:35.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.328+0000 7ff08ca00640 1 --2- 172.21.15.131:0/3554036115 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff08806e8c0 0x7ff08815d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:35.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.328+0000 7ff08ca00640 1 -- 172.21.15.131:0/3554036115 >> v1:172.21.15.187:6789/0 conn(0x7ff088156750 legacy=0x7ff08815fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:35.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.328+0000 7ff08ca00640 1 -- 172.21.15.131:0/3554036115 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff088160be0 con 0x7ff0880cc7f0 2024-05-26T22:51:35.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.328+0000 7ff08ca00640 1 -- 172.21.15.131:0/3554036115 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7ff0840019f0 con 0x7ff0880cc7f0 2024-05-26T22:51:35.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.328+0000 7ff08ca00640 1 -- 172.21.15.131:0/3554036115 >> v1:172.21.15.131:6789/0 conn(0x7ff0880cc7f0 legacy=0x7ff0880cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:35.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.328+0000 7ff08ca00640 1 --2- 172.21.15.131:0/3554036115 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff0700039e0 0x7ff070003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:35.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.328+0000 7ff08ca00640 1 --2- 172.21.15.131:0/3554036115 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff08806e8c0 0x7ff070004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:35.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.328+0000 7ff08ca00640 1 --2- 172.21.15.131:0/3554036115 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff070004d40 0x7ff0700051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:35.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.328+0000 7ff08ca00640 1 -- 172.21.15.131:0/3554036115 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff08815cbd0 con 0x7ff0700039e0 2024-05-26T22:51:35.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.328+0000 7ff08ca00640 1 -- 172.21.15.131:0/3554036115 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff07000baa0 con 0x7ff070004d40 2024-05-26T22:51:35.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.328+0000 7ff08ca00640 1 -- 172.21.15.131:0/3554036115 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff07000bca0 con 0x7ff08806e8c0 2024-05-26T22:51:35.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.328+0000 7ff08de00640 1 --2- 172.21.15.131:0/3554036115 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff0700039e0 0x7ff070003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:35.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.329+0000 7ff08de00640 1 -- 172.21.15.131:0/3554036115 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff08806e8c0 msgr2=0x7ff070004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:35.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.329+0000 7ff08d400640 1 --2- 172.21.15.131:0/3554036115 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff08806e8c0 0x7ff070004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:35.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.329+0000 7ff08de00640 1 --2- 172.21.15.131:0/3554036115 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff08806e8c0 0x7ff070004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:35.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.329+0000 7ff08e800640 1 --2- 172.21.15.131:0/3554036115 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff070004d40 0x7ff0700051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:35.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.329+0000 7ff08de00640 1 -- 172.21.15.131:0/3554036115 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff070004d40 msgr2=0x7ff0700051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:35.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.329+0000 7ff08de00640 1 --2- 172.21.15.131:0/3554036115 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff070004d40 0x7ff0700051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:35.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.329+0000 7ff08de00640 1 -- 172.21.15.131:0/3554036115 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff070003520 con 0x7ff0700039e0 2024-05-26T22:51:35.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.329+0000 7ff08e800640 1 --2- 172.21.15.131:0/3554036115 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff070004d40 0x7ff0700051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:51:35.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.330+0000 7ff08de00640 1 --2- 172.21.15.131:0/3554036115 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff0700039e0 0x7ff070003e00 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7ff07c001720 tx=0x7ff070005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e609a1b26c8ad332 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:35.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.330+0000 7ff08ca00640 1 -- 172.21.15.131:0/3554036115 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff07c001370 con 0x7ff0700039e0 2024-05-26T22:51:35.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.330+0000 7ff08ca00640 1 -- 172.21.15.131:0/3554036115 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff07c0052d0 con 0x7ff0700039e0 2024-05-26T22:51:35.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.330+0000 7ff08ca00640 1 -- 172.21.15.131:0/3554036115 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff07c02abc0 con 0x7ff0700039e0 2024-05-26T22:51:35.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.331+0000 7ff08f200640 1 -- 172.21.15.131:0/3554036115 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff0700039e0 msgr2=0x7ff070003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:35.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.331+0000 7ff08f200640 1 --2- 172.21.15.131:0/3554036115 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff0700039e0 0x7ff070003e00 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7ff07c001720 tx=0x7ff070005830 comp rx=0 tx=0).stop 2024-05-26T22:51:35.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.331+0000 7ff08e800640 1 -- 172.21.15.131:0/3554036115 reap_dead start 2024-05-26T22:51:35.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.331+0000 7ff08f200640 1 -- 172.21.15.131:0/3554036115 shutdown_connections 2024-05-26T22:51:35.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.331+0000 7ff08f200640 1 -- 172.21.15.131:0/3554036115 >> 172.21.15.131:0/3554036115 conn(0x7ff0880cff40 msgr2=0x7ff0880d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:35.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.331+0000 7ff08f200640 1 -- 172.21.15.131:0/3554036115 shutdown_connections 2024-05-26T22:51:35.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.331+0000 7ff08f200640 1 -- 172.21.15.131:0/3554036115 wait complete. 2024-05-26T22:51:35.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.333+0000 7ff08f200640 1 Processor -- start 2024-05-26T22:51:35.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.333+0000 7ff08f200640 1 -- start start 2024-05-26T22:51:35.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.334+0000 7ff08f200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff0700039e0 0x7ff088072980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:35.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.334+0000 7ff08f200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff0881e7770 0x7ff0881f8110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:35.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.334+0000 7ff08f200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff0880b4b20 0x7ff0880b4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:35.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.334+0000 7ff08f200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff0880ce370 con 0x7ff0700039e0 2024-05-26T22:51:35.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.334+0000 7ff08f200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff088156b30 con 0x7ff0881e7770 2024-05-26T22:51:35.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.334+0000 7ff08f200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff0880ce4f0 con 0x7ff0880b4b20 2024-05-26T22:51:35.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.334+0000 7ff08de00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff0700039e0 0x7ff088072980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:35.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.334+0000 7ff08de00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff0700039e0 0x7ff088072980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33388/0 (socket says 172.21.15.131:33388) 2024-05-26T22:51:35.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.334+0000 7ff08de00640 1 -- 172.21.15.131:0/1074908343 learned_addr learned my addr 172.21.15.131:0/1074908343 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:35.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.334+0000 7ff08e800640 1 --2- 172.21.15.131:0/1074908343 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff0880b4b20 0x7ff0880b4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:35.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.334+0000 7ff08d400640 1 --2- 172.21.15.131:0/1074908343 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff0881e7770 0x7ff0881f8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:35.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.334+0000 7ff08de00640 1 -- 172.21.15.131:0/1074908343 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff0880b4b20 msgr2=0x7ff0880b4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:35.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.334+0000 7ff08de00640 1 --2- 172.21.15.131:0/1074908343 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff0880b4b20 0x7ff0880b4fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:35.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.334+0000 7ff08de00640 1 -- 172.21.15.131:0/1074908343 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff0881e7770 msgr2=0x7ff0881f8110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:35.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.334+0000 7ff08de00640 1 --2- 172.21.15.131:0/1074908343 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff0881e7770 0x7ff0881f8110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:35.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.334+0000 7ff08de00640 1 -- 172.21.15.131:0/1074908343 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff088072f60 con 0x7ff0700039e0 2024-05-26T22:51:35.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.335+0000 7ff08de00640 1 --2- 172.21.15.131:0/1074908343 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff0700039e0 0x7ff088072980 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7ff07c001840 tx=0x7ff07c001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:35.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.335+0000 7ff082000640 1 -- 172.21.15.131:0/1074908343 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff07c03f020 con 0x7ff0700039e0 2024-05-26T22:51:35.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.335+0000 7ff082000640 1 -- 172.21.15.131:0/1074908343 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff07c02be50 con 0x7ff0700039e0 2024-05-26T22:51:35.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.335+0000 7ff082000640 1 -- 172.21.15.131:0/1074908343 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff07c007a30 con 0x7ff0700039e0 2024-05-26T22:51:35.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.335+0000 7ff08f200640 1 -- 172.21.15.131:0/1074908343 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0881e6c90 con 0x7ff0700039e0 2024-05-26T22:51:35.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.335+0000 7ff08f200640 1 -- 172.21.15.131:0/1074908343 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0881e7250 con 0x7ff0700039e0 2024-05-26T22:51:35.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.336+0000 7ff08f200640 1 -- 172.21.15.131:0/1074908343 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0880694e0 con 0x7ff0700039e0 2024-05-26T22:51:35.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.337+0000 7ff082000640 1 -- 172.21.15.131:0/1074908343 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff07c042020 con 0x7ff0700039e0 2024-05-26T22:51:35.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.338+0000 7ff082000640 1 --2- 172.21.15.131:0/1074908343 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ff06c06be30 0x7ff06c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:35.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.338+0000 7ff082000640 1 -- 172.21.15.131:0/1074908343 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 4857+0+0 (secure 0 0 0) 0x7ff07c0aa730 con 0x7ff0700039e0 2024-05-26T22:51:35.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.338+0000 7ff08d400640 1 --2- 172.21.15.131:0/1074908343 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ff06c06be30 0x7ff06c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:35.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.339+0000 7ff08d400640 1 --2- 172.21.15.131:0/1074908343 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ff06c06be30 0x7ff06c06e2f0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7ff0880af4a0 tx=0x7ff078008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:35.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.343+0000 7ff082000640 1 -- 172.21.15.131:0/1074908343 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7ff07c049030 con 0x7ff0700039e0 2024-05-26T22:51:35.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.538+0000 7ff08f200640 1 -- 172.21.15.131:0/1074908343 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff0880cfb10 con 0x7ff0700039e0 2024-05-26T22:51:35.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.540+0000 7ff082000640 1 -- 172.21.15.131:0/1074908343 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) ==== 74+0+12915 (secure 0 0 0) 0x7ff07c079390 con 0x7ff0700039e0 2024-05-26T22:51:35.540 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:51:35.540 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":54,"fsid":"46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f","created":"2024-05-26T22:47:57.288905+0000","modified":"2024-05-26T22:51:34.611371+0000","last_up_change":"2024-05-26T22:48:09.602207+0000","last_in_change":"2024-05-26T22:48:00.687341+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-26T22:48:10.463339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"814eaf62-a18d-494b-8e27-fd8fb88adea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6801","nonce":2350940051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6803","nonce":2350940051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6807","nonce":2350940051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6805","nonce":2350940051}]},"public_addr":"172.21.15.131:6801/2350940051","cluster_addr":"172.21.15.131:6803/2350940051","heartbeat_back_addr":"172.21.15.131:6807/2350940051","heartbeat_front_addr":"172.21.15.131:6805/2350940051","state":["exists","up"]},{"osd":1,"uuid":"42e66287-f6c6-4e26-af3c-6dfaa3536d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6825","nonce":1525528247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6827","nonce":1525528247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6831","nonce":1525528247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6829","nonce":1525528247}]},"public_addr":"172.21.15.131:6825/1525528247","cluster_addr":"172.21.15.131:6827/1525528247","heartbeat_back_addr":"172.21.15.131:6831/1525528247","heartbeat_front_addr":"172.21.15.131:6829/1525528247","state":["exists","up"]},{"osd":2,"uuid":"0da34673-04a2-4fd0-b57a-0ff2b679b043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6817","nonce":3627842210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6819","nonce":3627842210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6823","nonce":3627842210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6821","nonce":3627842210}]},"public_addr":"172.21.15.131:6817/3627842210","cluster_addr":"172.21.15.131:6819/3627842210","heartbeat_back_addr":"172.21.15.131:6823/3627842210","heartbeat_front_addr":"172.21.15.131:6821/3627842210","state":["exists","up"]},{"osd":3,"uuid":"cf117a27-d6b2-43d3-ae52-b47988bf8eed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6809","nonce":350790591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6811","nonce":350790591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6815","nonce":350790591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6813","nonce":350790591}]},"public_addr":"172.21.15.131:6809/350790591","cluster_addr":"172.21.15.131:6811/350790591","heartbeat_back_addr":"172.21.15.131:6815/350790591","heartbeat_front_addr":"172.21.15.131:6813/350790591","state":["exists","up"]},{"osd":4,"uuid":"a092ccfc-be89-4a84-88d5-0f781fa715eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6816","nonce":244560995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6819","nonce":244560995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6823","nonce":244560995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6821","nonce":244560995}]},"public_addr":"172.21.15.187:6816/244560995","cluster_addr":"172.21.15.187:6819/244560995","heartbeat_back_addr":"172.21.15.187:6823/244560995","heartbeat_front_addr":"172.21.15.187:6821/244560995","state":["exists","up"]},{"osd":5,"uuid":"7513ba69-dd00-4a35-b190-9e8cc3b2bdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6801","nonce":324619398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6803","nonce":324619398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6812","nonce":324619398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6807","nonce":324619398}]},"public_addr":"172.21.15.187:6801/324619398","cluster_addr":"172.21.15.187:6803/324619398","heartbeat_back_addr":"172.21.15.187:6812/324619398","heartbeat_front_addr":"172.21.15.187:6807/324619398","state":["exists","up"]},{"osd":6,"uuid":"50081810-c093-4d2f-a4d7-b7c695c5400d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6806","nonce":1391347786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6809","nonce":1391347786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6818","nonce":1391347786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6813","nonce":1391347786}]},"public_addr":"172.21.15.187:6806/1391347786","cluster_addr":"172.21.15.187:6809/1391347786","heartbeat_back_addr":"172.21.15.187:6818/1391347786","heartbeat_front_addr":"172.21.15.187:6813/1391347786","state":["exists","up"]},{"osd":7,"uuid":"775dae05-54f5-4ad4-bbce-92863c6ed2ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6825","nonce":1220065622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6827","nonce":1220065622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6831","nonce":1220065622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6829","nonce":1220065622}]},"public_addr":"172.21.15.187:6825/1220065622","cluster_addr":"172.21.15.187:6827/1220065622","heartbeat_back_addr":"172.21.15.187:6831/1220065622","heartbeat_front_addr":"172.21.15.187:6829/1220065622","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:05.983520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.299516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.067323+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.307748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.018742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.258775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.010912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.295355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-26T22:51:35.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.542+0000 7ff08f200640 1 -- 172.21.15.131:0/1074908343 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ff06c06be30 msgr2=0x7ff06c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:35.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.542+0000 7ff08f200640 1 --2- 172.21.15.131:0/1074908343 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ff06c06be30 0x7ff06c06e2f0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7ff0880af4a0 tx=0x7ff078008040 comp rx=0 tx=0).stop 2024-05-26T22:51:35.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.542+0000 7ff08f200640 1 -- 172.21.15.131:0/1074908343 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff0700039e0 msgr2=0x7ff088072980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:35.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.542+0000 7ff08f200640 1 --2- 172.21.15.131:0/1074908343 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff0700039e0 0x7ff088072980 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7ff07c001840 tx=0x7ff07c001120 comp rx=0 tx=0).stop 2024-05-26T22:51:35.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.543+0000 7ff08f200640 1 -- 172.21.15.131:0/1074908343 shutdown_connections 2024-05-26T22:51:35.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.543+0000 7ff08f200640 1 --2- 172.21.15.131:0/1074908343 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff0880b4b20 0x7ff0880b4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:35.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.543+0000 7ff08f200640 1 --2- 172.21.15.131:0/1074908343 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ff06c06be30 0x7ff06c06e2f0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:35.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.543+0000 7ff08f200640 1 --2- 172.21.15.131:0/1074908343 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff0881e7770 0x7ff0881f8110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:35.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.543+0000 7ff08f200640 1 --2- 172.21.15.131:0/1074908343 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff0700039e0 0x7ff088072980 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:35.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.543+0000 7ff08f200640 1 -- 172.21.15.131:0/1074908343 >> 172.21.15.131:0/1074908343 conn(0x7ff0880cff40 msgr2=0x7ff0880cf370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:35.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.543+0000 7ff08f200640 1 -- 172.21.15.131:0/1074908343 shutdown_connections 2024-05-26T22:51:35.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.543+0000 7ff08f200640 1 -- 172.21.15.131:0/1074908343 wait complete. 2024-05-26T22:51:35.560 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-26T22:48:10.463339+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-26T22:51:35.560 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-26T22:51:35.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.751+0000 7fcd96600640 1 Processor -- start 2024-05-26T22:51:35.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.751+0000 7fcd96600640 1 -- start start 2024-05-26T22:51:35.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.753+0000 7fcd96600640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fcd90145a10 0x7fcd90072180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:35.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.753+0000 7fcd96600640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcd9006d960 0x7fcd900cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:35.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.753+0000 7fcd96600640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd90072d60 con 0x7fcd900712d0 2024-05-26T22:51:35.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.753+0000 7fcd96600640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fcd9015c760 con 0x7fcd90145a10 2024-05-26T22:51:35.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.753+0000 7fcd96600640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fcd90072a80 con 0x7fcd9006d960 2024-05-26T22:51:35.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.753+0000 7fcd95200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fcd90145a10 0x7fcd90072180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:35.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.753+0000 7fcd95200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fcd90145a10 0x7fcd90072180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33394/0 (socket says 172.21.15.131:33394) 2024-05-26T22:51:35.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.753+0000 7fcd95200640 1 -- 172.21.15.131:0/640050236 learned_addr learned my addr 172.21.15.131:0/640050236 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:35.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.754+0000 7fcd95c00640 1 --2- 172.21.15.131:0/640050236 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcd9006d960 0x7fcd900cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:35.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.754+0000 7fcd95200640 1 -- 172.21.15.131:0/640050236 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcd9006d960 msgr2=0x7fcd900cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:35.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.754+0000 7fcd95200640 1 --2- 172.21.15.131:0/640050236 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcd9006d960 0x7fcd900cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:35.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.754+0000 7fcd95200640 1 -- 172.21.15.131:0/640050236 >> v1:172.21.15.187:6789/0 conn(0x7fcd900712d0 legacy=0x7fcd9006d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:35.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.754+0000 7fcd95200640 1 -- 172.21.15.131:0/640050236 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd900ceab0 con 0x7fcd90145a10 2024-05-26T22:51:35.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.754+0000 7fcd95c00640 1 --2- 172.21.15.131:0/640050236 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcd9006d960 0x7fcd900cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:51:35.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.755+0000 7fcd95200640 1 --2- 172.21.15.131:0/640050236 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd90145a10 0x7fcd90072180 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fcd80009e50 tx=0x7fcd8002a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=318846210d47dea5 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:35.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.755+0000 7fcd8f400640 1 -- 172.21.15.131:0/640050236 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd80036070 con 0x7fcd90145a10 2024-05-26T22:51:35.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.755+0000 7fcd8f400640 1 -- 172.21.15.131:0/640050236 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd80031600 con 0x7fcd90145a10 2024-05-26T22:51:35.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.755+0000 7fcd8f400640 1 -- 172.21.15.131:0/640050236 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd800319d0 con 0x7fcd90145a10 2024-05-26T22:51:35.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.756+0000 7fcd96600640 1 -- 172.21.15.131:0/640050236 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd90145a10 msgr2=0x7fcd90072180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:35.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.756+0000 7fcd96600640 1 --2- 172.21.15.131:0/640050236 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd90145a10 0x7fcd90072180 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fcd80009e50 tx=0x7fcd8002a6d0 comp rx=0 tx=0).stop 2024-05-26T22:51:35.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.756+0000 7fcd96600640 1 -- 172.21.15.131:0/640050236 shutdown_connections 2024-05-26T22:51:35.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.756+0000 7fcd96600640 1 --2- 172.21.15.131:0/640050236 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcd9006d960 0x7fcd900cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:35.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.756+0000 7fcd96600640 1 --2- 172.21.15.131:0/640050236 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd90145a10 0x7fcd90072180 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:35.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.756+0000 7fcd96600640 1 -- 172.21.15.131:0/640050236 >> 172.21.15.131:0/640050236 conn(0x7fcd900b5660 msgr2=0x7fcd900b5a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:35.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.756+0000 7fcd96600640 1 -- 172.21.15.131:0/640050236 shutdown_connections 2024-05-26T22:51:35.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.756+0000 7fcd96600640 1 -- 172.21.15.131:0/640050236 wait complete. 2024-05-26T22:51:35.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.758+0000 7fcd96600640 1 Processor -- start 2024-05-26T22:51:35.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.758+0000 7fcd96600640 1 -- start start 2024-05-26T22:51:35.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.759+0000 7fcd96600640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcd9006d960 0x7fcd90140b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:35.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.759+0000 7fcd96600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd900712d0 0x7fcd90143970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:35.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.759+0000 7fcd96600640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcd90145a10 0x7fcd900ae920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:35.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.759+0000 7fcd96600640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fcd90145df0 con 0x7fcd900712d0 2024-05-26T22:51:35.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.759+0000 7fcd96600640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fcd900cf070 con 0x7fcd90145a10 2024-05-26T22:51:35.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.759+0000 7fcd96600640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fcd900726c0 con 0x7fcd9006d960 2024-05-26T22:51:35.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.759+0000 7fcd8fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd900712d0 0x7fcd90143970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:35.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.759+0000 7fcd8fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd900712d0 0x7fcd90143970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33404/0 (socket says 172.21.15.131:33404) 2024-05-26T22:51:35.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.759+0000 7fcd8fe00640 1 -- 172.21.15.131:0/2754311883 learned_addr learned my addr 172.21.15.131:0/2754311883 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:35.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.759+0000 7fcd95200640 1 --2- 172.21.15.131:0/2754311883 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcd9006d960 0x7fcd90140b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:35.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.759+0000 7fcd95c00640 1 --2- 172.21.15.131:0/2754311883 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcd90145a10 0x7fcd900ae920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:35.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.759+0000 7fcd8fe00640 1 -- 172.21.15.131:0/2754311883 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcd9006d960 msgr2=0x7fcd90140b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:35.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.759+0000 7fcd8fe00640 1 --2- 172.21.15.131:0/2754311883 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcd9006d960 0x7fcd90140b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:35.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.760+0000 7fcd8fe00640 1 -- 172.21.15.131:0/2754311883 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcd90145a10 msgr2=0x7fcd900ae920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:35.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.760+0000 7fcd8fe00640 1 --2- 172.21.15.131:0/2754311883 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcd90145a10 0x7fcd900ae920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:35.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.760+0000 7fcd8fe00640 1 -- 172.21.15.131:0/2754311883 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd901f42f0 con 0x7fcd900712d0 2024-05-26T22:51:35.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.760+0000 7fcd8fe00640 1 --2- 172.21.15.131:0/2754311883 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd900712d0 0x7fcd90143970 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fcd7800c400 tx=0x7fcd7800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:35.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.760+0000 7fcd8cc00640 1 -- 172.21.15.131:0/2754311883 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd78011790 con 0x7fcd900712d0 2024-05-26T22:51:35.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.760+0000 7fcd8cc00640 1 -- 172.21.15.131:0/2754311883 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd78017070 con 0x7fcd900712d0 2024-05-26T22:51:35.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.760+0000 7fcd8cc00640 1 -- 172.21.15.131:0/2754311883 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd78011bb0 con 0x7fcd900712d0 2024-05-26T22:51:35.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.760+0000 7fcd96600640 1 -- 172.21.15.131:0/2754311883 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd901f45d0 con 0x7fcd900712d0 2024-05-26T22:51:35.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.760+0000 7fcd96600640 1 -- 172.21.15.131:0/2754311883 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd901f4b90 con 0x7fcd900712d0 2024-05-26T22:51:35.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.761+0000 7fcd96600640 1 -- 172.21.15.131:0/2754311883 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd900b3180 con 0x7fcd900712d0 2024-05-26T22:51:35.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.762+0000 7fcd8cc00640 1 -- 172.21.15.131:0/2754311883 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcd7801c460 con 0x7fcd900712d0 2024-05-26T22:51:35.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.763+0000 7fcd8cc00640 1 --2- 172.21.15.131:0/2754311883 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fcd6c06be30 0x7fcd6c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:35.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.763+0000 7fcd8cc00640 1 -- 172.21.15.131:0/2754311883 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 4857+0+0 (secure 0 0 0) 0x7fcd7808a990 con 0x7fcd900712d0 2024-05-26T22:51:35.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.763+0000 7fcd95200640 1 --2- 172.21.15.131:0/2754311883 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fcd6c06be30 0x7fcd6c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:35.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.764+0000 7fcd95200640 1 --2- 172.21.15.131:0/2754311883 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fcd6c06be30 0x7fcd6c06e2f0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fcd80033000 tx=0x7fcd80009000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:35.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.767+0000 7fcd8cc00640 1 -- 172.21.15.131:0/2754311883 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fcd78059680 con 0x7fcd900712d0 2024-05-26T22:51:35.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.950+0000 7fcd96600640 1 -- 172.21.15.131:0/2754311883 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fcd9006d3d0 con 0x7fcd900712d0 2024-05-26T22:51:35.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.951+0000 7fcd8cc00640 1 -- 172.21.15.131:0/2754311883 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v54) ==== 93+0+10 (secure 0 0 0) 0x7fcd78058f80 con 0x7fcd900712d0 2024-05-26T22:51:35.951 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2024-05-26T22:51:35.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.953+0000 7fcd96600640 1 -- 172.21.15.131:0/2754311883 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fcd6c06be30 msgr2=0x7fcd6c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:35.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.953+0000 7fcd96600640 1 --2- 172.21.15.131:0/2754311883 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fcd6c06be30 0x7fcd6c06e2f0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fcd80033000 tx=0x7fcd80009000 comp rx=0 tx=0).stop 2024-05-26T22:51:35.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.953+0000 7fcd96600640 1 -- 172.21.15.131:0/2754311883 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd900712d0 msgr2=0x7fcd90143970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:35.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.953+0000 7fcd96600640 1 --2- 172.21.15.131:0/2754311883 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd900712d0 0x7fcd90143970 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fcd7800c400 tx=0x7fcd7800c8c0 comp rx=0 tx=0).stop 2024-05-26T22:51:35.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.953+0000 7fcd96600640 1 -- 172.21.15.131:0/2754311883 shutdown_connections 2024-05-26T22:51:35.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.953+0000 7fcd96600640 1 --2- 172.21.15.131:0/2754311883 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fcd6c06be30 0x7fcd6c06e2f0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:35.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.953+0000 7fcd96600640 1 --2- 172.21.15.131:0/2754311883 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcd90145a10 0x7fcd900ae920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:35.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.953+0000 7fcd96600640 1 --2- 172.21.15.131:0/2754311883 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd900712d0 0x7fcd90143970 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:35.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.953+0000 7fcd96600640 1 --2- 172.21.15.131:0/2754311883 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcd9006d960 0x7fcd90140b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:35.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.953+0000 7fcd96600640 1 -- 172.21.15.131:0/2754311883 >> 172.21.15.131:0/2754311883 conn(0x7fcd900b5660 msgr2=0x7fcd9015f550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:35.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.954+0000 7fcd96600640 1 -- 172.21.15.131:0/2754311883 shutdown_connections 2024-05-26T22:51:35.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:35.954+0000 7fcd96600640 1 -- 172.21.15.131:0/2754311883 wait complete. 2024-05-26T22:51:35.972 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-26T22:51:36.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.158+0000 7fcd4de00640 1 Processor -- start 2024-05-26T22:51:36.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.158+0000 7fcd4de00640 1 -- start start 2024-05-26T22:51:36.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.159+0000 7fcd4de00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fcd48144340 0x7fcd480cedf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:36.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.159+0000 7fcd4de00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcd48071b70 0x7fcd4806e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:36.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.159+0000 7fcd4de00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fcd4806e9f0 0x7fcd4815ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:36.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.159+0000 7fcd4de00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fcd4806ee60 con 0x7fcd48144340 2024-05-26T22:51:36.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.159+0000 7fcd4de00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fcd48072eb0 con 0x7fcd4806e9f0 2024-05-26T22:51:36.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.159+0000 7fcd4de00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fcd48071fd0 con 0x7fcd48071b70 2024-05-26T22:51:36.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.160+0000 7fcd4ca00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fcd48144340 0x7fcd480cedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:36.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.160+0000 7fcd4ca00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fcd48144340 0x7fcd480cedf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33406/0 (socket says 172.21.15.131:33406) 2024-05-26T22:51:36.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.160+0000 7fcd4ca00640 1 -- 172.21.15.131:0/1211768441 learned_addr learned my addr 172.21.15.131:0/1211768441 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:36.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.160+0000 7fcd4ca00640 1 -- 172.21.15.131:0/1211768441 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcd48071b70 msgr2=0x7fcd4806e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:36.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.160+0000 7fcd4ca00640 1 --2- 172.21.15.131:0/1211768441 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcd48071b70 0x7fcd4806e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:36.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.160+0000 7fcd4ca00640 1 -- 172.21.15.131:0/1211768441 >> v2:172.21.15.187:3300/0 conn(0x7fcd4806e9f0 msgr2=0x7fcd4815ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:36.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.160+0000 7fcd4d400640 1 --2- 172.21.15.131:0/1211768441 >> v2:172.21.15.187:3300/0 conn(0x7fcd4806e9f0 0x7fcd4815ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:36.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.160+0000 7fcd4ca00640 1 --2- 172.21.15.131:0/1211768441 >> v2:172.21.15.187:3300/0 conn(0x7fcd4806e9f0 0x7fcd4815ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:36.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.160+0000 7fcd4ca00640 1 -- 172.21.15.131:0/1211768441 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd480722a0 con 0x7fcd48144340 2024-05-26T22:51:36.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.161+0000 7fcd4ca00640 1 --2- 172.21.15.131:0/1211768441 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd48144340 0x7fcd480cedf0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fcd480cfcf0 tx=0x7fcd3c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e669b41331b784f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:36.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.162+0000 7fcd47400640 1 -- 172.21.15.131:0/1211768441 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd3c037070 con 0x7fcd48144340 2024-05-26T22:51:36.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.162+0000 7fcd47400640 1 -- 172.21.15.131:0/1211768441 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd3c032440 con 0x7fcd48144340 2024-05-26T22:51:36.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.162+0000 7fcd47400640 1 -- 172.21.15.131:0/1211768441 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd3c032810 con 0x7fcd48144340 2024-05-26T22:51:36.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.162+0000 7fcd4de00640 1 -- 172.21.15.131:0/1211768441 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd48144340 msgr2=0x7fcd480cedf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:36.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.162+0000 7fcd4de00640 1 --2- 172.21.15.131:0/1211768441 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd48144340 0x7fcd480cedf0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fcd480cfcf0 tx=0x7fcd3c02a1e0 comp rx=0 tx=0).stop 2024-05-26T22:51:36.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.162+0000 7fcd4de00640 1 -- 172.21.15.131:0/1211768441 shutdown_connections 2024-05-26T22:51:36.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.162+0000 7fcd4de00640 1 --2- 172.21.15.131:0/1211768441 >> v2:172.21.15.187:3300/0 conn(0x7fcd4806e9f0 0x7fcd4815ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:36.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.162+0000 7fcd4de00640 1 --2- 172.21.15.131:0/1211768441 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcd48071b70 0x7fcd4806e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:36.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.162+0000 7fcd4de00640 1 --2- 172.21.15.131:0/1211768441 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd48144340 0x7fcd480cedf0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:36.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.162+0000 7fcd4de00640 1 -- 172.21.15.131:0/1211768441 >> 172.21.15.131:0/1211768441 conn(0x7fcd480cff40 msgr2=0x7fcd480d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:36.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.162+0000 7fcd4de00640 1 -- 172.21.15.131:0/1211768441 shutdown_connections 2024-05-26T22:51:36.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.162+0000 7fcd4de00640 1 -- 172.21.15.131:0/1211768441 wait complete. 2024-05-26T22:51:36.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.164+0000 7fcd4de00640 1 Processor -- start 2024-05-26T22:51:36.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.164+0000 7fcd4de00640 1 -- start start 2024-05-26T22:51:36.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.165+0000 7fcd4de00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcd4806e9f0 0x7fcd4815d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:36.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.165+0000 7fcd4de00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcd48071b70 0x7fcd48141e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:36.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.165+0000 7fcd4de00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd48144340 0x7fcd48142360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:36.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.165+0000 7fcd4de00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fcd48072b40 con 0x7fcd48144340 2024-05-26T22:51:36.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.165+0000 7fcd4de00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fcd48072860 con 0x7fcd48071b70 2024-05-26T22:51:36.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.165+0000 7fcd4de00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fcd4815f170 con 0x7fcd4806e9f0 2024-05-26T22:51:36.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.166+0000 7fcd4d400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd48144340 0x7fcd48142360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:36.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.166+0000 7fcd4d400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd48144340 0x7fcd48142360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33408/0 (socket says 172.21.15.131:33408) 2024-05-26T22:51:36.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.166+0000 7fcd4d400640 1 -- 172.21.15.131:0/1651086551 learned_addr learned my addr 172.21.15.131:0/1651086551 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:36.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.166+0000 7fcd47e00640 1 --2- 172.21.15.131:0/1651086551 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcd48071b70 0x7fcd48141e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:36.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.166+0000 7fcd4ca00640 1 --2- 172.21.15.131:0/1651086551 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcd4806e9f0 0x7fcd4815d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:36.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.166+0000 7fcd4d400640 1 -- 172.21.15.131:0/1651086551 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcd4806e9f0 msgr2=0x7fcd4815d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:36.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.166+0000 7fcd4d400640 1 --2- 172.21.15.131:0/1651086551 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcd4806e9f0 0x7fcd4815d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:36.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.166+0000 7fcd4d400640 1 -- 172.21.15.131:0/1651086551 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcd48071b70 msgr2=0x7fcd48141e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:36.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.166+0000 7fcd4d400640 1 --2- 172.21.15.131:0/1651086551 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcd48071b70 0x7fcd48141e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:36.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.166+0000 7fcd4d400640 1 -- 172.21.15.131:0/1651086551 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd4815db20 con 0x7fcd48144340 2024-05-26T22:51:36.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.166+0000 7fcd4d400640 1 --2- 172.21.15.131:0/1651086551 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd48144340 0x7fcd48142360 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fcd4000cda0 tx=0x7fcd4000a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:36.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.167+0000 7fcd44c00640 1 -- 172.21.15.131:0/1651086551 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd40003b90 con 0x7fcd48144340 2024-05-26T22:51:36.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.167+0000 7fcd44c00640 1 -- 172.21.15.131:0/1651086551 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd40003d30 con 0x7fcd48144340 2024-05-26T22:51:36.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.167+0000 7fcd44c00640 1 -- 172.21.15.131:0/1651086551 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd40004240 con 0x7fcd48144340 2024-05-26T22:51:36.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.167+0000 7fcd4de00640 1 -- 172.21.15.131:0/1651086551 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd4815de00 con 0x7fcd48144340 2024-05-26T22:51:36.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.167+0000 7fcd4de00640 1 -- 172.21.15.131:0/1651086551 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd4815e3c0 con 0x7fcd48144340 2024-05-26T22:51:36.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.168+0000 7fcd4de00640 1 -- 172.21.15.131:0/1651086551 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd480694e0 con 0x7fcd48144340 2024-05-26T22:51:36.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.169+0000 7fcd44c00640 1 -- 172.21.15.131:0/1651086551 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcd400043e0 con 0x7fcd48144340 2024-05-26T22:51:36.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.170+0000 7fcd44c00640 1 --2- 172.21.15.131:0/1651086551 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fcd2806be30 0x7fcd2806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:36.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.170+0000 7fcd44c00640 1 -- 172.21.15.131:0/1651086551 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 4857+0+0 (secure 0 0 0) 0x7fcd4008b680 con 0x7fcd48144340 2024-05-26T22:51:36.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.170+0000 7fcd4ca00640 1 --2- 172.21.15.131:0/1651086551 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fcd2806be30 0x7fcd2806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:36.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.171+0000 7fcd4ca00640 1 --2- 172.21.15.131:0/1651086551 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fcd2806be30 0x7fcd2806e2f0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c034000 tx=0x7fcd3c02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:36.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.174+0000 7fcd44c00640 1 -- 172.21.15.131:0/1651086551 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fcd4001d340 con 0x7fcd48144340 2024-05-26T22:51:36.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.376+0000 7fcd4de00640 1 -- 172.21.15.131:0/1651086551 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fcd480b5950 con 0x7fcd48144340 2024-05-26T22:51:36.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.377+0000 7fcd44c00640 1 -- 172.21.15.131:0/1651086551 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v54) ==== 84+0+65 (secure 0 0 0) 0x7fcd40060070 con 0x7fcd48144340 2024-05-26T22:51:36.377 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:51:36.377 INFO:teuthology.orchestra.run.smithi131.stdout:[ 2024-05-26T22:51:36.377 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:51:36.377 INFO:teuthology.orchestra.run.smithi131.stdout: "poolnum": 1, 2024-05-26T22:51:36.377 INFO:teuthology.orchestra.run.smithi131.stdout: "poolname": ".mgr" 2024-05-26T22:51:36.377 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:51:36.377 INFO:teuthology.orchestra.run.smithi131.stdout:] 2024-05-26T22:51:36.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.380+0000 7fcd4de00640 1 -- 172.21.15.131:0/1651086551 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fcd2806be30 msgr2=0x7fcd2806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:36.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.380+0000 7fcd4de00640 1 --2- 172.21.15.131:0/1651086551 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fcd2806be30 0x7fcd2806e2f0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c034000 tx=0x7fcd3c02e000 comp rx=0 tx=0).stop 2024-05-26T22:51:36.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.380+0000 7fcd4de00640 1 -- 172.21.15.131:0/1651086551 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd48144340 msgr2=0x7fcd48142360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:36.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.380+0000 7fcd4de00640 1 --2- 172.21.15.131:0/1651086551 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd48144340 0x7fcd48142360 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fcd4000cda0 tx=0x7fcd4000a660 comp rx=0 tx=0).stop 2024-05-26T22:51:36.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.380+0000 7fcd4de00640 1 -- 172.21.15.131:0/1651086551 shutdown_connections 2024-05-26T22:51:36.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.380+0000 7fcd4de00640 1 --2- 172.21.15.131:0/1651086551 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fcd2806be30 0x7fcd2806e2f0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:36.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.380+0000 7fcd4de00640 1 --2- 172.21.15.131:0/1651086551 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd48144340 0x7fcd48142360 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:36.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.380+0000 7fcd4de00640 1 --2- 172.21.15.131:0/1651086551 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcd48071b70 0x7fcd48141e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:36.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.380+0000 7fcd4de00640 1 --2- 172.21.15.131:0/1651086551 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcd4806e9f0 0x7fcd4815d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:36.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.380+0000 7fcd4de00640 1 -- 172.21.15.131:0/1651086551 >> 172.21.15.131:0/1651086551 conn(0x7fcd480cff40 msgr2=0x7fcd480b45b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:36.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.380+0000 7fcd4de00640 1 -- 172.21.15.131:0/1651086551 shutdown_connections 2024-05-26T22:51:36.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.380+0000 7fcd4de00640 1 -- 172.21.15.131:0/1651086551 wait complete. 2024-05-26T22:51:36.398 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-26T22:51:36.398 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-26T22:51:36.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.605+0000 7f8ea9800640 1 Processor -- start 2024-05-26T22:51:36.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.605+0000 7f8ea9800640 1 -- start start 2024-05-26T22:51:36.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.607+0000 7f8ea9800640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f8ea415fba0 0x7f8ea406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:36.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.607+0000 7f8ea9800640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ea4145c60 con 0x7f8ea4145880 2024-05-26T22:51:36.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.607+0000 7f8ea9800640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ea41605b0 con 0x7f8ea4160000 2024-05-26T22:51:36.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.607+0000 7f8ea9800640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f8ea415c560 con 0x7f8ea415fba0 2024-05-26T22:51:36.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.608+0000 7f8ea3e00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f8ea4145880 0x7f8ea415cfa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:35738/0 (socket says 172.21.15.131:35738) 2024-05-26T22:51:36.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.608+0000 7f8ea3e00640 1 -- 172.21.15.131:0/99133008 learned_addr learned my addr 172.21.15.131:0/99133008 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:51:36.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.608+0000 7f8e9b400640 1 --2- 172.21.15.131:0/99133008 >> v2:172.21.15.131:3300/0 conn(0x7f8ea415fba0 0x7f8ea406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:36.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.608+0000 7f8e9b400640 1 -- 172.21.15.131:0/99133008 >> v1:172.21.15.187:6789/0 conn(0x7f8ea4160000 legacy=0x7f8ea40b37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:36.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.608+0000 7f8e9b400640 1 -- 172.21.15.131:0/99133008 >> v1:172.21.15.131:6789/0 conn(0x7f8ea4145880 legacy=0x7f8ea415cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:36.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.608+0000 7f8e9b400640 1 -- 172.21.15.131:0/99133008 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ea4160890 con 0x7f8ea415fba0 2024-05-26T22:51:36.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.609+0000 7f8e9b400640 1 --2- 172.21.15.131:0/99133008 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ea415fba0 0x7f8ea406d4e0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f8e8c0097a0 tx=0x7f8e8c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19db570236edca66 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:36.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.611+0000 7f8ea3400640 1 -- 172.21.15.131:0/99133008 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e8c037070 con 0x7f8ea415fba0 2024-05-26T22:51:36.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.611+0000 7f8ea3400640 1 -- 172.21.15.131:0/99133008 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e8c032440 con 0x7f8ea415fba0 2024-05-26T22:51:36.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.611+0000 7f8ea3400640 1 -- 172.21.15.131:0/99133008 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e8c032810 con 0x7f8ea415fba0 2024-05-26T22:51:36.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.611+0000 7f8ea9800640 1 -- 172.21.15.131:0/99133008 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ea415fba0 msgr2=0x7f8ea406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:36.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.611+0000 7f8ea9800640 1 --2- 172.21.15.131:0/99133008 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ea415fba0 0x7f8ea406d4e0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f8e8c0097a0 tx=0x7f8e8c02a1e0 comp rx=0 tx=0).stop 2024-05-26T22:51:36.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.611+0000 7f8ea9800640 1 -- 172.21.15.131:0/99133008 shutdown_connections 2024-05-26T22:51:36.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.611+0000 7f8ea9800640 1 --2- 172.21.15.131:0/99133008 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ea415fba0 0x7f8ea406d4e0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:36.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.611+0000 7f8ea9800640 1 -- 172.21.15.131:0/99133008 >> 172.21.15.131:0/99133008 conn(0x7f8ea40b5410 msgr2=0x7f8ea40b57f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:36.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.612+0000 7f8ea9800640 1 -- 172.21.15.131:0/99133008 shutdown_connections 2024-05-26T22:51:36.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.612+0000 7f8ea9800640 1 -- 172.21.15.131:0/99133008 wait complete. 2024-05-26T22:51:36.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.613+0000 7f8ea9800640 1 Processor -- start 2024-05-26T22:51:36.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.613+0000 7f8ea9800640 1 -- start start 2024-05-26T22:51:36.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.614+0000 7f8ea9800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ea4145880 0x7f8ea41f4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:36.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.614+0000 7f8ea9800640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ea415fba0 0x7f8ea4139410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:36.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.614+0000 7f8ea9800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8ea4160000 0x7f8ea4139950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:36.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.614+0000 7f8ea9800640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8ea40b1d10 con 0x7f8ea4145880 2024-05-26T22:51:36.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.614+0000 7f8ea9800640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8ea415cb10 con 0x7f8ea415fba0 2024-05-26T22:51:36.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.614+0000 7f8ea9800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f8ea40b1e90 con 0x7f8ea4160000 2024-05-26T22:51:36.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.615+0000 7f8ea3e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ea4145880 0x7f8ea41f4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:36.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.615+0000 7f8ea3e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ea4145880 0x7f8ea41f4ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33430/0 (socket says 172.21.15.131:33430) 2024-05-26T22:51:36.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.615+0000 7f8ea3e00640 1 -- 172.21.15.131:0/3494242838 learned_addr learned my addr 172.21.15.131:0/3494242838 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:36.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.615+0000 7f8ea3e00640 1 -- 172.21.15.131:0/3494242838 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8ea4160000 msgr2=0x7f8ea4139950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:36.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.615+0000 7f8ea3e00640 1 --2- 172.21.15.131:0/3494242838 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8ea4160000 0x7f8ea4139950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:36.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.615+0000 7f8ea3e00640 1 -- 172.21.15.131:0/3494242838 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ea415fba0 msgr2=0x7f8ea4139410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:36.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.615+0000 7f8ea3e00640 1 --2- 172.21.15.131:0/3494242838 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ea415fba0 0x7f8ea4139410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:36.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.615+0000 7f8ea3e00640 1 -- 172.21.15.131:0/3494242838 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ea41f53e0 con 0x7f8ea4145880 2024-05-26T22:51:36.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.615+0000 7f8ea3e00640 1 --2- 172.21.15.131:0/3494242838 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ea4145880 0x7f8ea41f4ea0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f8e9400c3c0 tx=0x7f8e9400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:36.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.616+0000 7f8ea0c00640 1 -- 172.21.15.131:0/3494242838 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e94017070 con 0x7f8ea4145880 2024-05-26T22:51:36.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.616+0000 7f8ea0c00640 1 -- 172.21.15.131:0/3494242838 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e94005ce0 con 0x7f8ea4145880 2024-05-26T22:51:36.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.616+0000 7f8ea9800640 1 -- 172.21.15.131:0/3494242838 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ea4144850 con 0x7f8ea4145880 2024-05-26T22:51:36.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.616+0000 7f8ea0c00640 1 -- 172.21.15.131:0/3494242838 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e940072f0 con 0x7f8ea4145880 2024-05-26T22:51:36.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.616+0000 7f8ea9800640 1 -- 172.21.15.131:0/3494242838 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ea41427c0 con 0x7f8ea4145880 2024-05-26T22:51:36.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.617+0000 7f8ea9800640 1 -- 172.21.15.131:0/3494242838 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ea413b620 con 0x7f8ea4145880 2024-05-26T22:51:36.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.618+0000 7f8ea0c00640 1 -- 172.21.15.131:0/3494242838 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8e9401a070 con 0x7f8ea4145880 2024-05-26T22:51:36.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.619+0000 7f8ea0c00640 1 --2- 172.21.15.131:0/3494242838 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f8e8006be30 0x7f8e8006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:36.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.619+0000 7f8ea0c00640 1 -- 172.21.15.131:0/3494242838 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 4857+0+0 (secure 0 0 0) 0x7f8e94089e20 con 0x7f8ea4145880 2024-05-26T22:51:36.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.619+0000 7f8e9b400640 1 --2- 172.21.15.131:0/3494242838 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f8e8006be30 0x7f8e8006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:36.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.620+0000 7f8e9b400640 1 --2- 172.21.15.131:0/3494242838 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f8e8006be30 0x7f8e8006e2f0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f8e8c034000 tx=0x7f8e8c02c000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:36.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.623+0000 7f8ea0c00640 1 -- 172.21.15.131:0/3494242838 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f8e94058bb0 con 0x7f8ea4145880 2024-05-26T22:51:36.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:36.823+0000 7f8ea9800640 1 -- 172.21.15.131:0/3494242838 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f8ea40694e0 con 0x7f8ea4145880 2024-05-26T22:51:37.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:37.622+0000 7f8ea0c00640 1 -- 172.21.15.131:0/3494242838 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v55) ==== 138+0+0 (secure 0 0 0) 0x7f8e940584b0 con 0x7f8ea4145880 2024-05-26T22:51:37.622 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_metadata' created 2024-05-26T22:51:37.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:37.625+0000 7f8ea9800640 1 -- 172.21.15.131:0/3494242838 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f8e8006be30 msgr2=0x7f8e8006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:37.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:37.626+0000 7f8ea9800640 1 --2- 172.21.15.131:0/3494242838 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f8e8006be30 0x7f8e8006e2f0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f8e8c034000 tx=0x7f8e8c02c000 comp rx=0 tx=0).stop 2024-05-26T22:51:37.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:37.626+0000 7f8ea9800640 1 -- 172.21.15.131:0/3494242838 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ea4145880 msgr2=0x7f8ea41f4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:37.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:37.626+0000 7f8ea9800640 1 --2- 172.21.15.131:0/3494242838 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ea4145880 0x7f8ea41f4ea0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f8e9400c3c0 tx=0x7f8e9400c880 comp rx=0 tx=0).stop 2024-05-26T22:51:37.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:37.626+0000 7f8ea9800640 1 -- 172.21.15.131:0/3494242838 shutdown_connections 2024-05-26T22:51:37.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:37.626+0000 7f8ea9800640 1 --2- 172.21.15.131:0/3494242838 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8ea4160000 0x7f8ea4139950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:37.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:37.626+0000 7f8ea9800640 1 --2- 172.21.15.131:0/3494242838 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f8e8006be30 0x7f8e8006e2f0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:37.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:37.626+0000 7f8ea9800640 1 --2- 172.21.15.131:0/3494242838 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ea415fba0 0x7f8ea4139410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:37.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:37.626+0000 7f8ea9800640 1 --2- 172.21.15.131:0/3494242838 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ea4145880 0x7f8ea41f4ea0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:37.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:37.626+0000 7f8ea9800640 1 -- 172.21.15.131:0/3494242838 >> 172.21.15.131:0/3494242838 conn(0x7f8ea40b5410 msgr2=0x7f8ea40b10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:37.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:37.626+0000 7f8ea9800640 1 -- 172.21.15.131:0/3494242838 shutdown_connections 2024-05-26T22:51:37.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:37.626+0000 7f8ea9800640 1 -- 172.21.15.131:0/3494242838 wait complete. 2024-05-26T22:51:37.653 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-26T22:51:37.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:37.848+0000 7f59c6800640 1 Processor -- start 2024-05-26T22:51:37.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:37.848+0000 7f59c6800640 1 -- start start 2024-05-26T22:51:37.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:37.850+0000 7f59c6800640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f59c0145a10 0x7f59c0072180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:37.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:37.850+0000 7f59c6800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f59c006d960 0x7f59c00cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:37.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:37.850+0000 7f59c6800640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59c0072d60 con 0x7f59c00712d0 2024-05-26T22:51:37.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:37.850+0000 7f59c6800640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f59c015c760 con 0x7f59c0145a10 2024-05-26T22:51:37.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:37.850+0000 7f59c6800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f59c0072a80 con 0x7f59c006d960 2024-05-26T22:51:37.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:37.850+0000 7f59c5400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f59c0145a10 0x7f59c0072180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:37.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:37.850+0000 7f59c5400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f59c0145a10 0x7f59c0072180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33440/0 (socket says 172.21.15.131:33440) 2024-05-26T22:51:37.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:37.850+0000 7f59c5400640 1 -- 172.21.15.131:0/2749944329 learned_addr learned my addr 172.21.15.131:0/2749944329 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:37.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:37.851+0000 7f59c5e00640 1 --2- 172.21.15.131:0/2749944329 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f59c006d960 0x7f59c00cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:37.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:37.851+0000 7f59c5400640 1 -- 172.21.15.131:0/2749944329 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f59c006d960 msgr2=0x7f59c00cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:37.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:37.851+0000 7f59c5400640 1 --2- 172.21.15.131:0/2749944329 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f59c006d960 0x7f59c00cf590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:37.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:37.851+0000 7f59c5400640 1 -- 172.21.15.131:0/2749944329 >> v1:172.21.15.131:6789/0 conn(0x7f59c00712d0 legacy=0x7f59c006d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:37.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:37.851+0000 7f59c5400640 1 -- 172.21.15.131:0/2749944329 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59c00ceab0 con 0x7f59c0145a10 2024-05-26T22:51:37.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:37.852+0000 7f59c5400640 1 --2- 172.21.15.131:0/2749944329 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59c0145a10 0x7f59c0072180 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f59b4009870 tx=0x7f59b402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6375f68e1d6c564 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:37.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:37.853+0000 7f59bfe00640 1 -- 172.21.15.131:0/2749944329 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b4037070 con 0x7f59c0145a10 2024-05-26T22:51:37.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:37.853+0000 7f59bfe00640 1 -- 172.21.15.131:0/2749944329 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59b4032440 con 0x7f59c0145a10 2024-05-26T22:51:37.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:37.853+0000 7f59bfe00640 1 -- 172.21.15.131:0/2749944329 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b4032810 con 0x7f59c0145a10 2024-05-26T22:51:37.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:37.853+0000 7f59c6800640 1 -- 172.21.15.131:0/2749944329 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59c0145a10 msgr2=0x7f59c0072180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:37.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:37.853+0000 7f59c6800640 1 --2- 172.21.15.131:0/2749944329 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59c0145a10 0x7f59c0072180 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f59b4009870 tx=0x7f59b402a270 comp rx=0 tx=0).stop 2024-05-26T22:51:37.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:37.853+0000 7f59c6800640 1 -- 172.21.15.131:0/2749944329 shutdown_connections 2024-05-26T22:51:37.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:37.853+0000 7f59c6800640 1 --2- 172.21.15.131:0/2749944329 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f59c006d960 0x7f59c00cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:37.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:37.853+0000 7f59c6800640 1 --2- 172.21.15.131:0/2749944329 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59c0145a10 0x7f59c0072180 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:37.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:37.853+0000 7f59c6800640 1 -- 172.21.15.131:0/2749944329 >> 172.21.15.131:0/2749944329 conn(0x7f59c00b5660 msgr2=0x7f59c00b5a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:37.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:37.853+0000 7f59c6800640 1 -- 172.21.15.131:0/2749944329 shutdown_connections 2024-05-26T22:51:37.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:37.853+0000 7f59c6800640 1 -- 172.21.15.131:0/2749944329 wait complete. 2024-05-26T22:51:37.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:37.855+0000 7f59c6800640 1 Processor -- start 2024-05-26T22:51:37.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:37.855+0000 7f59c6800640 1 -- start start 2024-05-26T22:51:37.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:37.856+0000 7f59c6800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59c006d960 0x7f59c01f39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:37.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:37.856+0000 7f59c6800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f59c00712d0 0x7f59c014b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:37.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:37.856+0000 7f59c6800640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f59c0145a10 0x7f59c014b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:37.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:37.856+0000 7f59c6800640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f59c0145df0 con 0x7f59c006d960 2024-05-26T22:51:37.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:37.856+0000 7f59c6800640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f59c00cf070 con 0x7f59c0145a10 2024-05-26T22:51:37.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:37.856+0000 7f59c6800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f59c00726c0 con 0x7f59c00712d0 2024-05-26T22:51:37.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:37.857+0000 7f59c5400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59c006d960 0x7f59c01f39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:37.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:37.857+0000 7f59c5400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59c006d960 0x7f59c01f39c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33448/0 (socket says 172.21.15.131:33448) 2024-05-26T22:51:37.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:37.857+0000 7f59c5400640 1 -- 172.21.15.131:0/999940304 learned_addr learned my addr 172.21.15.131:0/999940304 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:37.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:37.857+0000 7f59c5400640 1 -- 172.21.15.131:0/999940304 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f59c00712d0 msgr2=0x7f59c014b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:37.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:37.857+0000 7f59c5400640 1 --2- 172.21.15.131:0/999940304 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f59c00712d0 0x7f59c014b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:37.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:37.857+0000 7f59c5400640 1 -- 172.21.15.131:0/999940304 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f59c0145a10 msgr2=0x7f59c014b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:37.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:37.857+0000 7f59c5400640 1 --2- 172.21.15.131:0/999940304 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f59c0145a10 0x7f59c014b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:37.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:37.857+0000 7f59c5400640 1 -- 172.21.15.131:0/999940304 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59c015f350 con 0x7f59c006d960 2024-05-26T22:51:37.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:37.857+0000 7f59c5400640 1 --2- 172.21.15.131:0/999940304 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59c006d960 0x7f59c01f39c0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f59b4000c00 tx=0x7f59b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:37.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:37.858+0000 7f59bd600640 1 -- 172.21.15.131:0/999940304 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b4037040 con 0x7f59c006d960 2024-05-26T22:51:37.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:37.858+0000 7f59bd600640 1 -- 172.21.15.131:0/999940304 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59b4042070 con 0x7f59c006d960 2024-05-26T22:51:37.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:37.858+0000 7f59bd600640 1 -- 172.21.15.131:0/999940304 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b4030bd0 con 0x7f59c006d960 2024-05-26T22:51:37.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:37.858+0000 7f59c6800640 1 -- 172.21.15.131:0/999940304 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59c0149280 con 0x7f59c006d960 2024-05-26T22:51:37.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:37.858+0000 7f59c6800640 1 -- 172.21.15.131:0/999940304 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59c0149840 con 0x7f59c006d960 2024-05-26T22:51:37.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:37.859+0000 7f59c6800640 1 -- 172.21.15.131:0/999940304 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59c01490a0 con 0x7f59c006d960 2024-05-26T22:51:37.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:37.860+0000 7f59bd600640 1 -- 172.21.15.131:0/999940304 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f59b4047460 con 0x7f59c006d960 2024-05-26T22:51:37.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:37.861+0000 7f59bd600640 1 --2- 172.21.15.131:0/999940304 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f59a006be80 0x7f59a006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:37.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:37.861+0000 7f59bd600640 1 -- 172.21.15.131:0/999940304 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5227+0+0 (secure 0 0 0) 0x7f59b40ab800 con 0x7f59c006d960 2024-05-26T22:51:37.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:37.861+0000 7f59c4a00640 1 --2- 172.21.15.131:0/999940304 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f59a006be80 0x7f59a006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:37.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:37.862+0000 7f59c4a00640 1 --2- 172.21.15.131:0/999940304 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f59a006be80 0x7f59a006e340 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f59ac000f60 tx=0x7f59ac0079d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:37.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:37.866+0000 7f59bd600640 1 -- 172.21.15.131:0/999940304 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f59c01490a0 con 0x7f59c006d960 2024-05-26T22:51:38.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:38.065+0000 7f59c6800640 1 -- 172.21.15.131:0/999940304 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f59c00694e0 con 0x7f59c006d960 2024-05-26T22:51:38.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:38.637+0000 7f59bd600640 1 -- 172.21.15.131:0/999940304 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v56) ==== 170+0+0 (secure 0 0 0) 0x7f59b4045360 con 0x7f59c006d960 2024-05-26T22:51:38.640 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_data' created 2024-05-26T22:51:38.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:38.646+0000 7f59c6800640 1 -- 172.21.15.131:0/999940304 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f59a006be80 msgr2=0x7f59a006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:38.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:38.646+0000 7f59c6800640 1 --2- 172.21.15.131:0/999940304 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f59a006be80 0x7f59a006e340 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f59ac000f60 tx=0x7f59ac0079d0 comp rx=0 tx=0).stop 2024-05-26T22:51:38.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:38.646+0000 7f59c6800640 1 -- 172.21.15.131:0/999940304 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59c006d960 msgr2=0x7f59c01f39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:38.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:38.646+0000 7f59c6800640 1 --2- 172.21.15.131:0/999940304 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59c006d960 0x7f59c01f39c0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f59b4000c00 tx=0x7f59b40046c0 comp rx=0 tx=0).stop 2024-05-26T22:51:38.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:38.647+0000 7f59c6800640 1 -- 172.21.15.131:0/999940304 shutdown_connections 2024-05-26T22:51:38.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:38.647+0000 7f59c6800640 1 --2- 172.21.15.131:0/999940304 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f59c00712d0 0x7f59c014b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:38.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:38.647+0000 7f59c6800640 1 --2- 172.21.15.131:0/999940304 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f59a006be80 0x7f59a006e340 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:38.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:38.647+0000 7f59c6800640 1 --2- 172.21.15.131:0/999940304 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f59c0145a10 0x7f59c014b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:38.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:38.647+0000 7f59c6800640 1 --2- 172.21.15.131:0/999940304 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59c006d960 0x7f59c01f39c0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:38.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:38.647+0000 7f59c6800640 1 -- 172.21.15.131:0/999940304 >> 172.21.15.131:0/999940304 conn(0x7f59c00b5660 msgr2=0x7f59c00b5a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:38.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:38.650+0000 7f59c6800640 1 -- 172.21.15.131:0/999940304 shutdown_connections 2024-05-26T22:51:38.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:38.650+0000 7f59c6800640 1 -- 172.21.15.131:0/999940304 wait complete. 2024-05-26T22:51:38.682 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-26T22:51:38.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:38.864+0000 7fee66400640 1 Processor -- start 2024-05-26T22:51:38.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:38.864+0000 7fee66400640 1 -- start start 2024-05-26T22:51:38.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:38.866+0000 7fee66400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fee600b5450 0x7fee600b43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:38.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:38.866+0000 7fee66400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fee600d0460 0x7fee6006d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:38.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:38.866+0000 7fee66400640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee600b4900 con 0x7fee6006f420 2024-05-26T22:51:38.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:38.866+0000 7fee66400640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fee60160d90 con 0x7fee600d0460 2024-05-26T22:51:38.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:38.866+0000 7fee66400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fee60160180 con 0x7fee600b5450 2024-05-26T22:51:38.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:38.866+0000 7fee5fe00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fee600d0460 0x7fee6006d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:38.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:38.866+0000 7fee5fe00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fee600d0460 0x7fee6006d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33452/0 (socket says 172.21.15.131:33452) 2024-05-26T22:51:38.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:38.866+0000 7fee56a00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fee6006f420 0x7fee600cfd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:35746/0 (socket says 172.21.15.131:35746) 2024-05-26T22:51:38.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:38.866+0000 7fee5fe00640 1 -- 172.21.15.131:0/2174607021 learned_addr learned my addr 172.21.15.131:0/2174607021 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:38.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:38.866+0000 7fee5f400640 1 --2- 172.21.15.131:0/2174607021 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fee600b5450 0x7fee600b43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:38.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:38.867+0000 7fee5fe00640 1 -- 172.21.15.131:0/2174607021 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fee600b5450 msgr2=0x7fee600b43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:38.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:38.867+0000 7fee5fe00640 1 --2- 172.21.15.131:0/2174607021 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fee600b5450 0x7fee600b43c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:38.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:38.867+0000 7fee5fe00640 1 -- 172.21.15.131:0/2174607021 >> v1:172.21.15.131:6789/0 conn(0x7fee6006f420 legacy=0x7fee600cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:38.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:38.867+0000 7fee5fe00640 1 -- 172.21.15.131:0/2174607021 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee60160460 con 0x7fee600d0460 2024-05-26T22:51:38.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:38.867+0000 7fee5ea00640 1 -- 172.21.15.131:0/2174607021 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2845200911 0 0) 0x7fee600b4900 con 0x7fee6006f420 2024-05-26T22:51:38.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:38.868+0000 7fee5fe00640 1 --2- 172.21.15.131:0/2174607021 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee600d0460 0x7fee6006d600 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fee58009e50 tx=0x7fee5802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d46eb90a10c5f22 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:38.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:38.868+0000 7fee5ea00640 1 -- 172.21.15.131:0/2174607021 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee58036070 con 0x7fee600d0460 2024-05-26T22:51:38.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:38.868+0000 7fee5ea00640 1 -- 172.21.15.131:0/2174607021 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee58031600 con 0x7fee600d0460 2024-05-26T22:51:38.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:38.868+0000 7fee5ea00640 1 -- 172.21.15.131:0/2174607021 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee580319d0 con 0x7fee600d0460 2024-05-26T22:51:38.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:38.868+0000 7fee66400640 1 -- 172.21.15.131:0/2174607021 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee600d0460 msgr2=0x7fee6006d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:38.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:38.868+0000 7fee66400640 1 --2- 172.21.15.131:0/2174607021 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee600d0460 0x7fee6006d600 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fee58009e50 tx=0x7fee5802a710 comp rx=0 tx=0).stop 2024-05-26T22:51:38.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:38.869+0000 7fee66400640 1 -- 172.21.15.131:0/2174607021 shutdown_connections 2024-05-26T22:51:38.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:38.869+0000 7fee66400640 1 --2- 172.21.15.131:0/2174607021 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee600d0460 0x7fee6006d600 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:38.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:38.869+0000 7fee66400640 1 --2- 172.21.15.131:0/2174607021 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fee600b5450 0x7fee600b43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:38.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:38.869+0000 7fee66400640 1 -- 172.21.15.131:0/2174607021 >> 172.21.15.131:0/2174607021 conn(0x7fee6006bc90 msgr2=0x7fee600b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:38.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:38.869+0000 7fee66400640 1 -- 172.21.15.131:0/2174607021 shutdown_connections 2024-05-26T22:51:38.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:38.869+0000 7fee66400640 1 -- 172.21.15.131:0/2174607021 wait complete. 2024-05-26T22:51:38.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:38.871+0000 7fee66400640 1 Processor -- start 2024-05-26T22:51:38.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:38.871+0000 7fee66400640 1 -- start start 2024-05-26T22:51:38.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:38.872+0000 7fee66400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee6006f420 0x7fee600c4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:38.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:38.872+0000 7fee66400640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fee600b5450 0x7fee600c6de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:38.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:38.872+0000 7fee66400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fee600d0460 0x7fee600c7320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:38.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:38.872+0000 7fee66400640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fee600b5830 con 0x7fee6006f420 2024-05-26T22:51:38.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:38.872+0000 7fee66400640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fee60160a20 con 0x7fee600b5450 2024-05-26T22:51:38.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:38.872+0000 7fee66400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fee6015fde0 con 0x7fee600d0460 2024-05-26T22:51:38.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:38.872+0000 7fee5f400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee6006f420 0x7fee600c4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:38.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:38.872+0000 7fee5f400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee6006f420 0x7fee600c4e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33466/0 (socket says 172.21.15.131:33466) 2024-05-26T22:51:38.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:38.872+0000 7fee5f400640 1 -- 172.21.15.131:0/368963200 learned_addr learned my addr 172.21.15.131:0/368963200 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:38.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:38.872+0000 7fee5fe00640 1 --2- 172.21.15.131:0/368963200 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fee600d0460 0x7fee600c7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:38.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:38.872+0000 7fee56a00640 1 --2- 172.21.15.131:0/368963200 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fee600b5450 0x7fee600c6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:38.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:38.872+0000 7fee5f400640 1 -- 172.21.15.131:0/368963200 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fee600d0460 msgr2=0x7fee600c7320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:38.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:38.872+0000 7fee5f400640 1 --2- 172.21.15.131:0/368963200 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fee600d0460 0x7fee600c7320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:38.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:38.872+0000 7fee5f400640 1 -- 172.21.15.131:0/368963200 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fee600b5450 msgr2=0x7fee600c6de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:38.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:38.872+0000 7fee5f400640 1 --2- 172.21.15.131:0/368963200 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fee600b5450 0x7fee600c6de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:38.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:38.872+0000 7fee5f400640 1 -- 172.21.15.131:0/368963200 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee600c56d0 con 0x7fee6006f420 2024-05-26T22:51:38.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:38.873+0000 7fee5f400640 1 --2- 172.21.15.131:0/368963200 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee6006f420 0x7fee600c4e80 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fee5000d800 tx=0x7fee5000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:38.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:38.873+0000 7fee57e00640 1 -- 172.21.15.131:0/368963200 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee5000b6c0 con 0x7fee6006f420 2024-05-26T22:51:38.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:38.873+0000 7fee57e00640 1 -- 172.21.15.131:0/368963200 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee50019070 con 0x7fee6006f420 2024-05-26T22:51:38.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:38.873+0000 7fee57e00640 1 -- 172.21.15.131:0/368963200 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee5000ba80 con 0x7fee6006f420 2024-05-26T22:51:38.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:38.873+0000 7fee66400640 1 -- 172.21.15.131:0/368963200 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee600c59b0 con 0x7fee6006f420 2024-05-26T22:51:38.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:38.873+0000 7fee66400640 1 -- 172.21.15.131:0/368963200 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee600c5f70 con 0x7fee6006f420 2024-05-26T22:51:38.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:38.874+0000 7fee66400640 1 -- 172.21.15.131:0/368963200 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee60074e90 con 0x7fee6006f420 2024-05-26T22:51:38.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:38.876+0000 7fee57e00640 1 -- 172.21.15.131:0/368963200 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fee5000bc20 con 0x7fee6006f420 2024-05-26T22:51:38.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:38.876+0000 7fee57e00640 1 --2- 172.21.15.131:0/368963200 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fee3c06be30 0x7fee3c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:38.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:38.877+0000 7fee57e00640 1 -- 172.21.15.131:0/368963200 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5609+0+0 (secure 0 0 0) 0x7fee5008b2e0 con 0x7fee6006f420 2024-05-26T22:51:38.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:38.877+0000 7fee56a00640 1 --2- 172.21.15.131:0/368963200 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fee3c06be30 0x7fee3c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:38.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:38.877+0000 7fee56a00640 1 --2- 172.21.15.131:0/368963200 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fee3c06be30 0x7fee3c06e2f0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fee60159b00 tx=0x7fee48009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:38.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:38.882+0000 7fee57e00640 1 -- 172.21.15.131:0/368963200 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fee50059ce0 con 0x7fee6006f420 2024-05-26T22:51:39.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:39.112+0000 7fee66400640 1 -- 172.21.15.131:0/368963200 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fee600cee90 con 0x7fee6006f420 2024-05-26T22:51:39.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:39.664+0000 7fee57e00640 1 -- 172.21.15.131:0/368963200 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 8 and data pool 9 v45) ==== 172+0+0 (secure 0 0 0) 0x7fee500595e0 con 0x7fee6006f420 2024-05-26T22:51:39.666 INFO:teuthology.orchestra.run.smithi131.stderr:new fs with metadata pool 8 and data pool 9 2024-05-26T22:51:39.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:39.674+0000 7fee66400640 1 -- 172.21.15.131:0/368963200 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fee3c06be30 msgr2=0x7fee3c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:39.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:39.674+0000 7fee66400640 1 --2- 172.21.15.131:0/368963200 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fee3c06be30 0x7fee3c06e2f0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fee60159b00 tx=0x7fee48009040 comp rx=0 tx=0).stop 2024-05-26T22:51:39.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:39.674+0000 7fee66400640 1 -- 172.21.15.131:0/368963200 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee6006f420 msgr2=0x7fee600c4e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:39.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:39.674+0000 7fee66400640 1 --2- 172.21.15.131:0/368963200 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee6006f420 0x7fee600c4e80 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fee5000d800 tx=0x7fee5000dcc0 comp rx=0 tx=0).stop 2024-05-26T22:51:39.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:39.675+0000 7fee66400640 1 -- 172.21.15.131:0/368963200 shutdown_connections 2024-05-26T22:51:39.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:39.675+0000 7fee66400640 1 --2- 172.21.15.131:0/368963200 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fee600d0460 0x7fee600c7320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:39.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:39.675+0000 7fee66400640 1 --2- 172.21.15.131:0/368963200 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fee3c06be30 0x7fee3c06e2f0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:39.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:39.675+0000 7fee66400640 1 --2- 172.21.15.131:0/368963200 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fee600b5450 0x7fee600c6de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:39.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:39.676+0000 7fee66400640 1 --2- 172.21.15.131:0/368963200 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee6006f420 0x7fee600c4e80 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:39.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:39.676+0000 7fee66400640 1 -- 172.21.15.131:0/368963200 >> 172.21.15.131:0/368963200 conn(0x7fee6006bc90 msgr2=0x7fee60071a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:39.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:39.677+0000 7fee66400640 1 -- 172.21.15.131:0/368963200 shutdown_connections 2024-05-26T22:51:39.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:39.678+0000 7fee66400640 1 -- 172.21.15.131:0/368963200 wait complete. 2024-05-26T22:51:39.709 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-26T22:51:39.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:39.871+0000 7f1f24000640 1 Processor -- start 2024-05-26T22:51:39.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:39.872+0000 7f1f24000640 1 -- start start 2024-05-26T22:51:39.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:39.873+0000 7f1f24000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f1c0ad250 0x7f1f1c0b3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:39.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:39.873+0000 7f1f24000640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f1f1c1569d0 0x7f1f1c0af790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:39.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:39.873+0000 7f1f24000640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f1c15cb20 con 0x7f1f1c06ee20 2024-05-26T22:51:39.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:39.873+0000 7f1f24000640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f1f1c15ce00 con 0x7f1f1c1569d0 2024-05-26T22:51:39.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:39.873+0000 7f1f24000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1f1c15d0e0 con 0x7f1f1c0ad250 2024-05-26T22:51:39.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:39.873+0000 7f1f22000640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f1f1c1569d0 0x7f1f1c0af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:39.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:39.873+0000 7f1f22000640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f1f1c1569d0 0x7f1f1c0af790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33476/0 (socket says 172.21.15.131:33476) 2024-05-26T22:51:39.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:39.873+0000 7f1f22000640 1 -- 172.21.15.131:0/3314796902 learned_addr learned my addr 172.21.15.131:0/3314796902 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:39.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:39.873+0000 7f1f20c00640 1 --2- 172.21.15.131:0/3314796902 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f1c0ad250 0x7f1f1c0b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:39.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:39.874+0000 7f1f22000640 1 -- 172.21.15.131:0/3314796902 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f1c0ad250 msgr2=0x7f1f1c0b3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:39.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:39.874+0000 7f1f22000640 1 --2- 172.21.15.131:0/3314796902 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f1c0ad250 0x7f1f1c0b3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:39.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:39.874+0000 7f1f22000640 1 -- 172.21.15.131:0/3314796902 >> v1:172.21.15.187:6789/0 conn(0x7f1f1c06ee20 legacy=0x7f1f1c1562c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:39.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:39.874+0000 7f1f22000640 1 -- 172.21.15.131:0/3314796902 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f1c15d3c0 con 0x7f1f1c1569d0 2024-05-26T22:51:39.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:39.875+0000 7f1f22000640 1 --2- 172.21.15.131:0/3314796902 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f1c1569d0 0x7f1f1c0af790 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f1f14009e80 tx=0x7f1f1402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d18462eccee746e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:39.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:39.875+0000 7f1f1be00640 1 -- 172.21.15.131:0/3314796902 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f14036070 con 0x7f1f1c1569d0 2024-05-26T22:51:39.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:39.875+0000 7f1f1be00640 1 -- 172.21.15.131:0/3314796902 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f14031600 con 0x7f1f1c1569d0 2024-05-26T22:51:39.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:39.875+0000 7f1f1be00640 1 -- 172.21.15.131:0/3314796902 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f140319d0 con 0x7f1f1c1569d0 2024-05-26T22:51:39.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:39.875+0000 7f1f24000640 1 -- 172.21.15.131:0/3314796902 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f1c1569d0 msgr2=0x7f1f1c0af790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:39.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:39.875+0000 7f1f24000640 1 --2- 172.21.15.131:0/3314796902 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f1c1569d0 0x7f1f1c0af790 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f1f14009e80 tx=0x7f1f1402a6f0 comp rx=0 tx=0).stop 2024-05-26T22:51:39.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:39.876+0000 7f1f24000640 1 -- 172.21.15.131:0/3314796902 shutdown_connections 2024-05-26T22:51:39.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:39.876+0000 7f1f24000640 1 --2- 172.21.15.131:0/3314796902 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f1c1569d0 0x7f1f1c0af790 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:39.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:39.876+0000 7f1f24000640 1 --2- 172.21.15.131:0/3314796902 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f1c0ad250 0x7f1f1c0b3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:39.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:39.876+0000 7f1f24000640 1 -- 172.21.15.131:0/3314796902 >> 172.21.15.131:0/3314796902 conn(0x7f1f1c0b1e90 msgr2=0x7f1f1c0b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:39.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:39.876+0000 7f1f24000640 1 -- 172.21.15.131:0/3314796902 shutdown_connections 2024-05-26T22:51:39.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:39.876+0000 7f1f24000640 1 -- 172.21.15.131:0/3314796902 wait complete. 2024-05-26T22:51:39.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:39.877+0000 7f1f24000640 1 Processor -- start 2024-05-26T22:51:39.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:39.878+0000 7f1f24000640 1 -- start start 2024-05-26T22:51:39.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:39.878+0000 7f1f24000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f1c06ee20 0x7f1f1c1f4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:39.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:39.878+0000 7f1f24000640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1f1c0ad250 0x7f1f1c0af600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:39.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:39.878+0000 7f1f24000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f1c1569d0 0x7f1f1c1e6f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:39.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:39.878+0000 7f1f24000640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f1f1c15c900 con 0x7f1f1c1569d0 2024-05-26T22:51:39.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:39.878+0000 7f1f24000640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1f1c15c780 con 0x7f1f1c0ad250 2024-05-26T22:51:39.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:39.879+0000 7f1f24000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1f1c06f200 con 0x7f1f1c06ee20 2024-05-26T22:51:39.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:39.879+0000 7f1f22000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f1c1569d0 0x7f1f1c1e6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:39.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:39.879+0000 7f1f22000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f1c1569d0 0x7f1f1c1e6f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33492/0 (socket says 172.21.15.131:33492) 2024-05-26T22:51:39.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:39.879+0000 7f1f22000640 1 -- 172.21.15.131:0/218969047 learned_addr learned my addr 172.21.15.131:0/218969047 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:39.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:39.879+0000 7f1f20c00640 1 --2- 172.21.15.131:0/218969047 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1f1c0ad250 0x7f1f1c0af600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:39.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:39.879+0000 7f1f21600640 1 --2- 172.21.15.131:0/218969047 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f1c06ee20 0x7f1f1c1f4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:39.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:39.879+0000 7f1f22000640 1 -- 172.21.15.131:0/218969047 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f1c06ee20 msgr2=0x7f1f1c1f4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:39.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:39.879+0000 7f1f22000640 1 --2- 172.21.15.131:0/218969047 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f1c06ee20 0x7f1f1c1f4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:39.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:39.879+0000 7f1f22000640 1 -- 172.21.15.131:0/218969047 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1f1c0ad250 msgr2=0x7f1f1c0af600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:39.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:39.879+0000 7f1f22000640 1 --2- 172.21.15.131:0/218969047 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1f1c0ad250 0x7f1f1c0af600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:39.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:39.879+0000 7f1f22000640 1 -- 172.21.15.131:0/218969047 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f1c0afb40 con 0x7f1f1c1569d0 2024-05-26T22:51:39.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:39.879+0000 7f1f21600640 1 --2- 172.21.15.131:0/218969047 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f1c06ee20 0x7f1f1c1f4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:51:39.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:39.879+0000 7f1f20c00640 1 --2- 172.21.15.131:0/218969047 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1f1c0ad250 0x7f1f1c0af600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:51:39.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:39.880+0000 7f1f22000640 1 --2- 172.21.15.131:0/218969047 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f1c1569d0 0x7f1f1c1e6f40 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f1f14034040 tx=0x7f1f14004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:39.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:39.880+0000 7f1f19600640 1 -- 172.21.15.131:0/218969047 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f14036050 con 0x7f1f1c1569d0 2024-05-26T22:51:39.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:39.880+0000 7f1f19600640 1 -- 172.21.15.131:0/218969047 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f14042070 con 0x7f1f1c1569d0 2024-05-26T22:51:39.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:39.880+0000 7f1f24000640 1 -- 172.21.15.131:0/218969047 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f1c0afdf0 con 0x7f1f1c1569d0 2024-05-26T22:51:39.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:39.880+0000 7f1f19600640 1 -- 172.21.15.131:0/218969047 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f14009510 con 0x7f1f1c1569d0 2024-05-26T22:51:39.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:39.880+0000 7f1f24000640 1 -- 172.21.15.131:0/218969047 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f1c0b03b0 con 0x7f1f1c1569d0 2024-05-26T22:51:39.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:39.881+0000 7f1f24000640 1 -- 172.21.15.131:0/218969047 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f1c0694e0 con 0x7f1f1c1569d0 2024-05-26T22:51:39.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:39.882+0000 7f1f19600640 1 -- 172.21.15.131:0/218969047 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1f14007bb0 con 0x7f1f1c1569d0 2024-05-26T22:51:39.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:39.883+0000 7f1f19600640 1 --2- 172.21.15.131:0/218969047 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f1efc06be30 0x7f1efc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:39.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:39.883+0000 7f1f19600640 1 -- 172.21.15.131:0/218969047 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f1f140abda0 con 0x7f1f1c1569d0 2024-05-26T22:51:39.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:39.883+0000 7f1f21600640 1 --2- 172.21.15.131:0/218969047 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f1efc06be30 0x7f1efc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:39.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:39.884+0000 7f1f21600640 1 --2- 172.21.15.131:0/218969047 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f1efc06be30 0x7f1efc06e2f0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c152e40 tx=0x7f1f10009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:39.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:39.888+0000 7f1f19600640 1 -- 172.21.15.131:0/218969047 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f1f1407a6b0 con 0x7f1f1c1569d0 2024-05-26T22:51:40.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.076+0000 7f1f24000640 1 -- 172.21.15.131:0/218969047 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1f1c06c590 con 0x7f1f1c1569d0 2024-05-26T22:51:40.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.078+0000 7f1f19600640 1 -- 172.21.15.131:0/218969047 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) ==== 74+0+16702 (secure 0 0 0) 0x7f1f14079fb0 con 0x7f1f1c1569d0 2024-05-26T22:51:40.078 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:51:40.078 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":57,"fsid":"46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f","created":"2024-05-26T22:47:57.288905+0000","modified":"2024-05-26T22:51:39.113990+0000","last_up_change":"2024-05-26T22:48:09.602207+0000","last_in_change":"2024-05-26T22:48:00.687341+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-26T22:48:10.463339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-05-26T22:51:36.873056+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-05-26T22:51:38.120229+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"814eaf62-a18d-494b-8e27-fd8fb88adea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6801","nonce":2350940051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6803","nonce":2350940051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6807","nonce":2350940051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6805","nonce":2350940051}]},"public_addr":"172.21.15.131:6801/2350940051","cluster_addr":"172.21.15.131:6803/2350940051","heartbeat_back_addr":"172.21.15.131:6807/2350940051","heartbeat_front_addr":"172.21.15.131:6805/2350940051","state":["exists","up"]},{"osd":1,"uuid":"42e66287-f6c6-4e26-af3c-6dfaa3536d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6825","nonce":1525528247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6827","nonce":1525528247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6831","nonce":1525528247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6829","nonce":1525528247}]},"public_addr":"172.21.15.131:6825/1525528247","cluster_addr":"172.21.15.131:6827/1525528247","heartbeat_back_addr":"172.21.15.131:6831/1525528247","heartbeat_front_addr":"172.21.15.131:6829/1525528247","state":["exists","up"]},{"osd":2,"uuid":"0da34673-04a2-4fd0-b57a-0ff2b679b043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6817","nonce":3627842210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6819","nonce":3627842210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6823","nonce":3627842210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6821","nonce":3627842210}]},"public_addr":"172.21.15.131:6817/3627842210","cluster_addr":"172.21.15.131:6819/3627842210","heartbeat_back_addr":"172.21.15.131:6823/3627842210","heartbeat_front_addr":"172.21.15.131:6821/3627842210","state":["exists","up"]},{"osd":3,"uuid":"cf117a27-d6b2-43d3-ae52-b47988bf8eed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6809","nonce":350790591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6811","nonce":350790591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6815","nonce":350790591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6813","nonce":350790591}]},"public_addr":"172.21.15.131:6809/350790591","cluster_addr":"172.21.15.131:6811/350790591","heartbeat_back_addr":"172.21.15.131:6815/350790591","heartbeat_front_addr":"172.21.15.131:6813/350790591","state":["exists","up"]},{"osd":4,"uuid":"a092ccfc-be89-4a84-88d5-0f781fa715eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6816","nonce":244560995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6819","nonce":244560995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6823","nonce":244560995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6821","nonce":244560995}]},"public_addr":"172.21.15.187:6816/244560995","cluster_addr":"172.21.15.187:6819/244560995","heartbeat_back_addr":"172.21.15.187:6823/244560995","heartbeat_front_addr":"172.21.15.187:6821/244560995","state":["exists","up"]},{"osd":5,"uuid":"7513ba69-dd00-4a35-b190-9e8cc3b2bdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6801","nonce":324619398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6803","nonce":324619398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6812","nonce":324619398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6807","nonce":324619398}]},"public_addr":"172.21.15.187:6801/324619398","cluster_addr":"172.21.15.187:6803/324619398","heartbeat_back_addr":"172.21.15.187:6812/324619398","heartbeat_front_addr":"172.21.15.187:6807/324619398","state":["exists","up"]},{"osd":6,"uuid":"50081810-c093-4d2f-a4d7-b7c695c5400d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6806","nonce":1391347786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6809","nonce":1391347786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6818","nonce":1391347786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6813","nonce":1391347786}]},"public_addr":"172.21.15.187:6806/1391347786","cluster_addr":"172.21.15.187:6809/1391347786","heartbeat_back_addr":"172.21.15.187:6818/1391347786","heartbeat_front_addr":"172.21.15.187:6813/1391347786","state":["exists","up"]},{"osd":7,"uuid":"775dae05-54f5-4ad4-bbce-92863c6ed2ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6825","nonce":1220065622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6827","nonce":1220065622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6831","nonce":1220065622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6829","nonce":1220065622}]},"public_addr":"172.21.15.187:6825/1220065622","cluster_addr":"172.21.15.187:6827/1220065622","heartbeat_back_addr":"172.21.15.187:6831/1220065622","heartbeat_front_addr":"172.21.15.187:6829/1220065622","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:05.983520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.299516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.067323+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.307748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.018742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.258775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.010912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.295355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-26T22:51:40.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.080+0000 7f1f24000640 1 -- 172.21.15.131:0/218969047 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f1efc06be30 msgr2=0x7f1efc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:40.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.080+0000 7f1f24000640 1 --2- 172.21.15.131:0/218969047 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f1efc06be30 0x7f1efc06e2f0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c152e40 tx=0x7f1f10009040 comp rx=0 tx=0).stop 2024-05-26T22:51:40.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.080+0000 7f1f24000640 1 -- 172.21.15.131:0/218969047 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f1c1569d0 msgr2=0x7f1f1c1e6f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:40.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.080+0000 7f1f24000640 1 --2- 172.21.15.131:0/218969047 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f1c1569d0 0x7f1f1c1e6f40 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f1f14034040 tx=0x7f1f14004a40 comp rx=0 tx=0).stop 2024-05-26T22:51:40.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.081+0000 7f1f24000640 1 -- 172.21.15.131:0/218969047 shutdown_connections 2024-05-26T22:51:40.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.081+0000 7f1f24000640 1 --2- 172.21.15.131:0/218969047 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f1efc06be30 0x7f1efc06e2f0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:40.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.081+0000 7f1f24000640 1 --2- 172.21.15.131:0/218969047 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f1c1569d0 0x7f1f1c1e6f40 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:40.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.081+0000 7f1f24000640 1 --2- 172.21.15.131:0/218969047 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1f1c0ad250 0x7f1f1c0af600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:40.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.081+0000 7f1f24000640 1 --2- 172.21.15.131:0/218969047 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f1c06ee20 0x7f1f1c1f4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:40.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.081+0000 7f1f24000640 1 -- 172.21.15.131:0/218969047 >> 172.21.15.131:0/218969047 conn(0x7f1f1c0b1e90 msgr2=0x7f1f1c06d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:40.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.081+0000 7f1f24000640 1 -- 172.21.15.131:0/218969047 shutdown_connections 2024-05-26T22:51:40.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.081+0000 7f1f24000640 1 -- 172.21.15.131:0/218969047 wait complete. 2024-05-26T22:51:40.099 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-26T22:51:40.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.295+0000 7f96f9800640 1 Processor -- start 2024-05-26T22:51:40.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.295+0000 7f96f9800640 1 -- start start 2024-05-26T22:51:40.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.297+0000 7f96f9800640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f96f406f600 0x7f96f406d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:40.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.297+0000 7f96f9800640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96f40ad170 con 0x7f96f4162780 2024-05-26T22:51:40.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.297+0000 7f96f9800640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96f40b4510 con 0x7f96f406daf0 2024-05-26T22:51:40.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.297+0000 7f96f9800640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f96f406df60 con 0x7f96f406f600 2024-05-26T22:51:40.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.297+0000 7f96ebe00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f96f406f600 0x7f96f406d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:40.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.297+0000 7f96f2a00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f96f4162780 0x7f96f40b3ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:35752/0 (socket says 172.21.15.131:35752) 2024-05-26T22:51:40.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.297+0000 7f96f2a00640 1 -- 172.21.15.131:0/4189800226 learned_addr learned my addr 172.21.15.131:0/4189800226 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:51:40.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.297+0000 7f96ebe00640 1 -- 172.21.15.131:0/4189800226 >> v1:172.21.15.187:6789/0 conn(0x7f96f406daf0 legacy=0x7f96f40ad950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:40.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.297+0000 7f96ebe00640 1 -- 172.21.15.131:0/4189800226 >> v1:172.21.15.131:6789/0 conn(0x7f96f4162780 legacy=0x7f96f40b3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:40.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.298+0000 7f96ebe00640 1 -- 172.21.15.131:0/4189800226 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96f40b47f0 con 0x7f96f406f600 2024-05-26T22:51:40.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.297+0000 7f96f2000640 1 -- 172.21.15.131:0/4189800226 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 237659027 0 0) 0x7f96f40ad170 con 0x7f96f4162780 2024-05-26T22:51:40.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.298+0000 7f96ebe00640 1 --2- 172.21.15.131:0/4189800226 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96f406f600 0x7f96f406d5b0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f96dc0097a0 tx=0x7f96dc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40b7fe94dc76dd68 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:40.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.299+0000 7f96f2000640 1 -- 172.21.15.131:0/4189800226 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96dc037070 con 0x7f96f406f600 2024-05-26T22:51:40.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.299+0000 7f96f2000640 1 -- 172.21.15.131:0/4189800226 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96dc032440 con 0x7f96f406f600 2024-05-26T22:51:40.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.299+0000 7f96f2000640 1 -- 172.21.15.131:0/4189800226 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96dc032810 con 0x7f96f406f600 2024-05-26T22:51:40.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.299+0000 7f96f9800640 1 -- 172.21.15.131:0/4189800226 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96f406f600 msgr2=0x7f96f406d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:40.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.299+0000 7f96f9800640 1 --2- 172.21.15.131:0/4189800226 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96f406f600 0x7f96f406d5b0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f96dc0097a0 tx=0x7f96dc02a1e0 comp rx=0 tx=0).stop 2024-05-26T22:51:40.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.299+0000 7f96f9800640 1 -- 172.21.15.131:0/4189800226 shutdown_connections 2024-05-26T22:51:40.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.299+0000 7f96f9800640 1 --2- 172.21.15.131:0/4189800226 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96f406f600 0x7f96f406d5b0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:40.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.299+0000 7f96f9800640 1 -- 172.21.15.131:0/4189800226 >> 172.21.15.131:0/4189800226 conn(0x7f96f40b1e90 msgr2=0x7f96f40b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:40.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.300+0000 7f96f9800640 1 -- 172.21.15.131:0/4189800226 shutdown_connections 2024-05-26T22:51:40.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.300+0000 7f96f9800640 1 -- 172.21.15.131:0/4189800226 wait complete. 2024-05-26T22:51:40.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.301+0000 7f96f9800640 1 Processor -- start 2024-05-26T22:51:40.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.301+0000 7f96f9800640 1 -- start start 2024-05-26T22:51:40.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.302+0000 7f96f9800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f96f406daf0 0x7f96f4159620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:40.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.302+0000 7f96f9800640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f96f406f600 0x7f96f4159b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:40.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.302+0000 7f96f9800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96f4162780 0x7f96f41f8790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:40.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.302+0000 7f96f9800640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f96f4162b60 con 0x7f96f4162780 2024-05-26T22:51:40.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.302+0000 7f96f9800640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f96f40b42b0 con 0x7f96f406f600 2024-05-26T22:51:40.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.302+0000 7f96f9800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f96f40ae060 con 0x7f96f406daf0 2024-05-26T22:51:40.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.303+0000 7f96f3400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96f4162780 0x7f96f41f8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:40.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.303+0000 7f96f3400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96f4162780 0x7f96f41f8790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33508/0 (socket says 172.21.15.131:33508) 2024-05-26T22:51:40.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.303+0000 7f96f3400640 1 -- 172.21.15.131:0/881146421 learned_addr learned my addr 172.21.15.131:0/881146421 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:40.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.303+0000 7f96f2a00640 1 --2- 172.21.15.131:0/881146421 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f96f406daf0 0x7f96f4159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:40.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.303+0000 7f96ebe00640 1 --2- 172.21.15.131:0/881146421 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f96f406f600 0x7f96f4159b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:40.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.303+0000 7f96f3400640 1 -- 172.21.15.131:0/881146421 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f96f406daf0 msgr2=0x7f96f4159620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:40.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.303+0000 7f96f3400640 1 --2- 172.21.15.131:0/881146421 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f96f406daf0 0x7f96f4159620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:40.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.303+0000 7f96f3400640 1 -- 172.21.15.131:0/881146421 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f96f406f600 msgr2=0x7f96f4159b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:40.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.303+0000 7f96f3400640 1 --2- 172.21.15.131:0/881146421 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f96f406f600 0x7f96f4159b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:40.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.303+0000 7f96f3400640 1 -- 172.21.15.131:0/881146421 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96f41579d0 con 0x7f96f4162780 2024-05-26T22:51:40.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.303+0000 7f96f3400640 1 --2- 172.21.15.131:0/881146421 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96f4162780 0x7f96f41f8790 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f96ec00c7b0 tx=0x7f96ec00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:40.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.303+0000 7f96eaa00640 1 -- 172.21.15.131:0/881146421 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96ec017070 con 0x7f96f4162780 2024-05-26T22:51:40.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.304+0000 7f96eaa00640 1 -- 172.21.15.131:0/881146421 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96ec003da0 con 0x7f96f4162780 2024-05-26T22:51:40.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.304+0000 7f96eaa00640 1 -- 172.21.15.131:0/881146421 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96ec0052b0 con 0x7f96f4162780 2024-05-26T22:51:40.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.304+0000 7f96f9800640 1 -- 172.21.15.131:0/881146421 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96f4157cb0 con 0x7f96f4162780 2024-05-26T22:51:40.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.304+0000 7f96f9800640 1 -- 172.21.15.131:0/881146421 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96f4158270 con 0x7f96f4162780 2024-05-26T22:51:40.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.305+0000 7f96f9800640 1 -- 172.21.15.131:0/881146421 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96f40694e0 con 0x7f96f4162780 2024-05-26T22:51:40.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.306+0000 7f96eaa00640 1 -- 172.21.15.131:0/881146421 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f96ec01a070 con 0x7f96f4162780 2024-05-26T22:51:40.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.306+0000 7f96eaa00640 1 --2- 172.21.15.131:0/881146421 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f96d006be30 0x7f96d006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:40.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.307+0000 7f96eaa00640 1 -- 172.21.15.131:0/881146421 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f96ec08a510 con 0x7f96f4162780 2024-05-26T22:51:40.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.307+0000 7f96f2a00640 1 --2- 172.21.15.131:0/881146421 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f96d006be30 0x7f96d006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:40.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.307+0000 7f96f2a00640 1 --2- 172.21.15.131:0/881146421 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f96d006be30 0x7f96d006e2f0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f96e4000f60 tx=0x7f96e4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:40.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.311+0000 7f96eaa00640 1 -- 172.21.15.131:0/881146421 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f96ec058eb0 con 0x7f96f4162780 2024-05-26T22:51:40.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.498+0000 7f96f9800640 1 -- 172.21.15.131:0/881146421 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f96f40af6a0 con 0x7f96f4162780 2024-05-26T22:51:40.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.500+0000 7f96eaa00640 1 -- 172.21.15.131:0/881146421 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) ==== 74+0+16702 (secure 0 0 0) 0x7f96ec05f080 con 0x7f96f4162780 2024-05-26T22:51:40.500 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:51:40.500 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":57,"fsid":"46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f","created":"2024-05-26T22:47:57.288905+0000","modified":"2024-05-26T22:51:39.113990+0000","last_up_change":"2024-05-26T22:48:09.602207+0000","last_in_change":"2024-05-26T22:48:00.687341+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-26T22:48:10.463339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-05-26T22:51:36.873056+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-05-26T22:51:38.120229+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"814eaf62-a18d-494b-8e27-fd8fb88adea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6801","nonce":2350940051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6803","nonce":2350940051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6807","nonce":2350940051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6805","nonce":2350940051}]},"public_addr":"172.21.15.131:6801/2350940051","cluster_addr":"172.21.15.131:6803/2350940051","heartbeat_back_addr":"172.21.15.131:6807/2350940051","heartbeat_front_addr":"172.21.15.131:6805/2350940051","state":["exists","up"]},{"osd":1,"uuid":"42e66287-f6c6-4e26-af3c-6dfaa3536d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6825","nonce":1525528247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6827","nonce":1525528247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6831","nonce":1525528247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6829","nonce":1525528247}]},"public_addr":"172.21.15.131:6825/1525528247","cluster_addr":"172.21.15.131:6827/1525528247","heartbeat_back_addr":"172.21.15.131:6831/1525528247","heartbeat_front_addr":"172.21.15.131:6829/1525528247","state":["exists","up"]},{"osd":2,"uuid":"0da34673-04a2-4fd0-b57a-0ff2b679b043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6817","nonce":3627842210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6819","nonce":3627842210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6823","nonce":3627842210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6821","nonce":3627842210}]},"public_addr":"172.21.15.131:6817/3627842210","cluster_addr":"172.21.15.131:6819/3627842210","heartbeat_back_addr":"172.21.15.131:6823/3627842210","heartbeat_front_addr":"172.21.15.131:6821/3627842210","state":["exists","up"]},{"osd":3,"uuid":"cf117a27-d6b2-43d3-ae52-b47988bf8eed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6809","nonce":350790591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6811","nonce":350790591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6815","nonce":350790591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6813","nonce":350790591}]},"public_addr":"172.21.15.131:6809/350790591","cluster_addr":"172.21.15.131:6811/350790591","heartbeat_back_addr":"172.21.15.131:6815/350790591","heartbeat_front_addr":"172.21.15.131:6813/350790591","state":["exists","up"]},{"osd":4,"uuid":"a092ccfc-be89-4a84-88d5-0f781fa715eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6816","nonce":244560995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6819","nonce":244560995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6823","nonce":244560995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6821","nonce":244560995}]},"public_addr":"172.21.15.187:6816/244560995","cluster_addr":"172.21.15.187:6819/244560995","heartbeat_back_addr":"172.21.15.187:6823/244560995","heartbeat_front_addr":"172.21.15.187:6821/244560995","state":["exists","up"]},{"osd":5,"uuid":"7513ba69-dd00-4a35-b190-9e8cc3b2bdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6801","nonce":324619398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6803","nonce":324619398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6812","nonce":324619398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6807","nonce":324619398}]},"public_addr":"172.21.15.187:6801/324619398","cluster_addr":"172.21.15.187:6803/324619398","heartbeat_back_addr":"172.21.15.187:6812/324619398","heartbeat_front_addr":"172.21.15.187:6807/324619398","state":["exists","up"]},{"osd":6,"uuid":"50081810-c093-4d2f-a4d7-b7c695c5400d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6806","nonce":1391347786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6809","nonce":1391347786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6818","nonce":1391347786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6813","nonce":1391347786}]},"public_addr":"172.21.15.187:6806/1391347786","cluster_addr":"172.21.15.187:6809/1391347786","heartbeat_back_addr":"172.21.15.187:6818/1391347786","heartbeat_front_addr":"172.21.15.187:6813/1391347786","state":["exists","up"]},{"osd":7,"uuid":"775dae05-54f5-4ad4-bbce-92863c6ed2ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6825","nonce":1220065622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6827","nonce":1220065622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6831","nonce":1220065622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6829","nonce":1220065622}]},"public_addr":"172.21.15.187:6825/1220065622","cluster_addr":"172.21.15.187:6827/1220065622","heartbeat_back_addr":"172.21.15.187:6831/1220065622","heartbeat_front_addr":"172.21.15.187:6829/1220065622","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:05.983520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.299516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.067323+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.307748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.018742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.258775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.010912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.295355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-26T22:51:40.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.503+0000 7f96f9800640 1 -- 172.21.15.131:0/881146421 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f96d006be30 msgr2=0x7f96d006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:40.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.503+0000 7f96f9800640 1 --2- 172.21.15.131:0/881146421 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f96d006be30 0x7f96d006e2f0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f96e4000f60 tx=0x7f96e4009040 comp rx=0 tx=0).stop 2024-05-26T22:51:40.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.503+0000 7f96f9800640 1 -- 172.21.15.131:0/881146421 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96f4162780 msgr2=0x7f96f41f8790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:40.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.503+0000 7f96f9800640 1 --2- 172.21.15.131:0/881146421 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96f4162780 0x7f96f41f8790 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f96ec00c7b0 tx=0x7f96ec00cc70 comp rx=0 tx=0).stop 2024-05-26T22:51:40.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.503+0000 7f96f9800640 1 -- 172.21.15.131:0/881146421 shutdown_connections 2024-05-26T22:51:40.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.503+0000 7f96f9800640 1 --2- 172.21.15.131:0/881146421 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f96d006be30 0x7f96d006e2f0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:40.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.503+0000 7f96f9800640 1 --2- 172.21.15.131:0/881146421 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96f4162780 0x7f96f41f8790 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:40.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.503+0000 7f96f9800640 1 --2- 172.21.15.131:0/881146421 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f96f406f600 0x7f96f4159b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:40.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.503+0000 7f96f9800640 1 --2- 172.21.15.131:0/881146421 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f96f406daf0 0x7f96f4159620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:40.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.503+0000 7f96f9800640 1 -- 172.21.15.131:0/881146421 >> 172.21.15.131:0/881146421 conn(0x7f96f40b1e90 msgr2=0x7f96f40b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:40.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.504+0000 7f96f9800640 1 -- 172.21.15.131:0/881146421 shutdown_connections 2024-05-26T22:51:40.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.504+0000 7f96f9800640 1 -- 172.21.15.131:0/881146421 wait complete. 2024-05-26T22:51:40.521 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-26T22:51:40.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.718+0000 7f2032800640 1 Processor -- start 2024-05-26T22:51:40.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.718+0000 7f2032800640 1 -- start start 2024-05-26T22:51:40.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.719+0000 7f2032800640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f202c15e340 0x7f202c0b72e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:40.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.719+0000 7f2032800640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f202c0b7820 0x7f202c0d22b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:40.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.720+0000 7f2032800640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f202c15e720 con 0x7f202c0d27f0 2024-05-26T22:51:40.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.720+0000 7f2032800640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f202c0b88b0 con 0x7f202c0b7820 2024-05-26T22:51:40.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.720+0000 7f2032800640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f202c0b7ca0 con 0x7f202c15e340 2024-05-26T22:51:40.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.720+0000 7f2030a00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f202c0b7820 0x7f202c0d22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:40.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.720+0000 7f2031e00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f202c0d27f0 0x7f202c0b8d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:35762/0 (socket says 172.21.15.131:35762) 2024-05-26T22:51:40.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.720+0000 7f2031e00640 1 -- 172.21.15.131:0/652568594 learned_addr learned my addr 172.21.15.131:0/652568594 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:51:40.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.720+0000 7f2031400640 1 --2- 172.21.15.131:0/652568594 >> v2:172.21.15.187:3300/0 conn(0x7f202c15e340 0x7f202c0b72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:40.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.720+0000 7f2030a00640 1 -- 172.21.15.131:0/652568594 >> v2:172.21.15.187:3300/0 conn(0x7f202c15e340 msgr2=0x7f202c0b72e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:40.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.720+0000 7f2030a00640 1 --2- 172.21.15.131:0/652568594 >> v2:172.21.15.187:3300/0 conn(0x7f202c15e340 0x7f202c0b72e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:40.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.720+0000 7f2030a00640 1 -- 172.21.15.131:0/652568594 >> v1:172.21.15.131:6789/0 conn(0x7f202c0d27f0 legacy=0x7f202c0b8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:40.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.720+0000 7f2030a00640 1 -- 172.21.15.131:0/652568594 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f202c0b7f80 con 0x7f202c0b7820 2024-05-26T22:51:40.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.720+0000 7f202be00640 1 -- 172.21.15.131:0/652568594 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1021677282 0 0) 0x7f202c15e720 con 0x7f202c0d27f0 2024-05-26T22:51:40.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.721+0000 7f2030a00640 1 --2- 172.21.15.131:0/652568594 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f202c0b7820 0x7f202c0d22b0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f202c0d14c0 tx=0x7f201802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a736c1685aced8c5 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:40.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.722+0000 7f202be00640 1 -- 172.21.15.131:0/652568594 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2018007740 con 0x7f202c0b7820 2024-05-26T22:51:40.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.722+0000 7f202be00640 1 -- 172.21.15.131:0/652568594 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20180078e0 con 0x7f202c0b7820 2024-05-26T22:51:40.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.722+0000 7f202be00640 1 -- 172.21.15.131:0/652568594 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2018007cb0 con 0x7f202c0b7820 2024-05-26T22:51:40.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.722+0000 7f2032800640 1 -- 172.21.15.131:0/652568594 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f202c0b7820 msgr2=0x7f202c0d22b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:40.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.722+0000 7f2032800640 1 --2- 172.21.15.131:0/652568594 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f202c0b7820 0x7f202c0d22b0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f202c0d14c0 tx=0x7f201802a680 comp rx=0 tx=0).stop 2024-05-26T22:51:40.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.722+0000 7f2032800640 1 -- 172.21.15.131:0/652568594 shutdown_connections 2024-05-26T22:51:40.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.722+0000 7f2032800640 1 --2- 172.21.15.131:0/652568594 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f202c0b7820 0x7f202c0d22b0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:40.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.722+0000 7f2032800640 1 --2- 172.21.15.131:0/652568594 >> v2:172.21.15.187:3300/0 conn(0x7f202c15e340 0x7f202c0b72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:40.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.722+0000 7f2032800640 1 -- 172.21.15.131:0/652568594 >> 172.21.15.131:0/652568594 conn(0x7f202c072e60 msgr2=0x7f202c073270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:40.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.722+0000 7f2032800640 1 -- 172.21.15.131:0/652568594 shutdown_connections 2024-05-26T22:51:40.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.722+0000 7f2032800640 1 -- 172.21.15.131:0/652568594 wait complete. 2024-05-26T22:51:40.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.724+0000 7f2032800640 1 Processor -- start 2024-05-26T22:51:40.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.724+0000 7f2032800640 1 -- start start 2024-05-26T22:51:40.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.725+0000 7f2032800640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f202c0b7820 0x7f202c15b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:40.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.725+0000 7f2032800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f202c0d27f0 0x7f202c153750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:40.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.725+0000 7f2032800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f202c15e340 0x7f202c0d3390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:40.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.725+0000 7f2032800640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f202c0b9450 con 0x7f202c15e340 2024-05-26T22:51:40.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.725+0000 7f2032800640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f202c0b8540 con 0x7f202c0b7820 2024-05-26T22:51:40.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.725+0000 7f2032800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f202c0b95d0 con 0x7f202c0d27f0 2024-05-26T22:51:40.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.725+0000 7f2031e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f202c15e340 0x7f202c0d3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:40.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.725+0000 7f2031e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f202c15e340 0x7f202c0d3390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33518/0 (socket says 172.21.15.131:33518) 2024-05-26T22:51:40.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.725+0000 7f2031e00640 1 -- 172.21.15.131:0/4178365850 learned_addr learned my addr 172.21.15.131:0/4178365850 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:40.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.726+0000 7f2031400640 1 --2- 172.21.15.131:0/4178365850 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f202c0b7820 0x7f202c15b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:40.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.726+0000 7f2030a00640 1 --2- 172.21.15.131:0/4178365850 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f202c0d27f0 0x7f202c153750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:40.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.726+0000 7f2031e00640 1 -- 172.21.15.131:0/4178365850 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f202c0d27f0 msgr2=0x7f202c153750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:40.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.726+0000 7f2031e00640 1 --2- 172.21.15.131:0/4178365850 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f202c0d27f0 0x7f202c153750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:40.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.726+0000 7f2031e00640 1 -- 172.21.15.131:0/4178365850 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f202c0b7820 msgr2=0x7f202c15b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:40.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.726+0000 7f2031e00640 1 --2- 172.21.15.131:0/4178365850 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f202c0b7820 0x7f202c15b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:40.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.726+0000 7f2031e00640 1 -- 172.21.15.131:0/4178365850 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f202c15cc80 con 0x7f202c15e340 2024-05-26T22:51:40.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.726+0000 7f2031e00640 1 --2- 172.21.15.131:0/4178365850 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f202c15e340 0x7f202c0d3390 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f202400c4b0 tx=0x7f202400c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:40.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.726+0000 7f2029600640 1 -- 172.21.15.131:0/4178365850 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2024018070 con 0x7f202c15e340 2024-05-26T22:51:40.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.727+0000 7f2029600640 1 -- 172.21.15.131:0/4178365850 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20240074e0 con 0x7f202c15e340 2024-05-26T22:51:40.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.727+0000 7f2032800640 1 -- 172.21.15.131:0/4178365850 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f202c153c90 con 0x7f202c15e340 2024-05-26T22:51:40.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.727+0000 7f2029600640 1 -- 172.21.15.131:0/4178365850 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2024004a20 con 0x7f202c15e340 2024-05-26T22:51:40.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.727+0000 7f2032800640 1 -- 172.21.15.131:0/4178365850 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f202c15d200 con 0x7f202c15e340 2024-05-26T22:51:40.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.728+0000 7f2032800640 1 -- 172.21.15.131:0/4178365850 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f202c0694e0 con 0x7f202c15e340 2024-05-26T22:51:40.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.728+0000 7f2029600640 1 -- 172.21.15.131:0/4178365850 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f202401b070 con 0x7f202c15e340 2024-05-26T22:51:40.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.729+0000 7f2029600640 1 --2- 172.21.15.131:0/4178365850 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f200c06be30 0x7f200c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:40.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.729+0000 7f2029600640 1 -- 172.21.15.131:0/4178365850 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f202408a0d0 con 0x7f202c15e340 2024-05-26T22:51:40.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.730+0000 7f2031400640 1 --2- 172.21.15.131:0/4178365850 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f200c06be30 0x7f200c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:40.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.730+0000 7f2031400640 1 --2- 172.21.15.131:0/4178365850 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f200c06be30 0x7f200c06e2f0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f201c006fd0 tx=0x7f201c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:40.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.734+0000 7f2029600640 1 -- 172.21.15.131:0/4178365850 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f20240589e0 con 0x7f202c15e340 2024-05-26T22:51:40.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:40.975+0000 7f2032800640 1 -- 172.21.15.131:0/4178365850 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f202c0c8630 con 0x7f202c15e340 2024-05-26T22:51:41.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:41.677+0000 7f2029600640 1 -- 172.21.15.131:0/4178365850 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v48) ==== 118+0+0 (secure 0 0 0) 0x7f20240582e0 con 0x7f202c15e340 2024-05-26T22:51:41.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:41.681+0000 7f2032800640 1 -- 172.21.15.131:0/4178365850 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f200c06be30 msgr2=0x7f200c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:41.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:41.681+0000 7f2032800640 1 --2- 172.21.15.131:0/4178365850 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f200c06be30 0x7f200c06e2f0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f201c006fd0 tx=0x7f201c008040 comp rx=0 tx=0).stop 2024-05-26T22:51:41.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:41.681+0000 7f2032800640 1 -- 172.21.15.131:0/4178365850 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f202c15e340 msgr2=0x7f202c0d3390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:41.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:41.681+0000 7f2032800640 1 --2- 172.21.15.131:0/4178365850 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f202c15e340 0x7f202c0d3390 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f202400c4b0 tx=0x7f202400c970 comp rx=0 tx=0).stop 2024-05-26T22:51:41.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:41.681+0000 7f2032800640 1 -- 172.21.15.131:0/4178365850 shutdown_connections 2024-05-26T22:51:41.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:41.681+0000 7f2032800640 1 --2- 172.21.15.131:0/4178365850 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f202c0d27f0 0x7f202c153750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:41.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:41.681+0000 7f2032800640 1 --2- 172.21.15.131:0/4178365850 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f200c06be30 0x7f200c06e2f0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:41.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:41.681+0000 7f2032800640 1 --2- 172.21.15.131:0/4178365850 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f202c15e340 0x7f202c0d3390 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:41.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:41.681+0000 7f2032800640 1 --2- 172.21.15.131:0/4178365850 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f202c0b7820 0x7f202c15b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:41.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:41.681+0000 7f2032800640 1 -- 172.21.15.131:0/4178365850 >> 172.21.15.131:0/4178365850 conn(0x7f202c072e60 msgr2=0x7f202c0c9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:41.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:41.682+0000 7f2032800640 1 -- 172.21.15.131:0/4178365850 shutdown_connections 2024-05-26T22:51:41.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:41.682+0000 7f2032800640 1 -- 172.21.15.131:0/4178365850 wait complete. 2024-05-26T22:51:41.702 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-26T22:51:41.706 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-26T22:51:41.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:41.914+0000 7f01c0a00640 1 Processor -- start 2024-05-26T22:51:41.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:41.914+0000 7f01c0a00640 1 -- start start 2024-05-26T22:51:41.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:41.916+0000 7f01c0a00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f01bc06d250 0x7f01bc06d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:41.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:41.916+0000 7f01c0a00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01bc06db70 con 0x7f01bc1400c0 2024-05-26T22:51:41.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:41.916+0000 7f01c0a00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01bc13e820 con 0x7f01bc15b730 2024-05-26T22:51:41.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:41.916+0000 7f01c0a00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f01bc071260 con 0x7f01bc06d250 2024-05-26T22:51:41.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:41.916+0000 7f01bb200640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f01bc1400c0 0x7f01bc1404f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:35764/0 (socket says 172.21.15.131:35764) 2024-05-26T22:51:41.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:41.916+0000 7f01bb200640 1 -- 172.21.15.131:0/3444333390 learned_addr learned my addr 172.21.15.131:0/3444333390 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:51:41.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:41.916+0000 7f01ba800640 1 --2- 172.21.15.131:0/3444333390 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f01bc06d250 0x7f01bc06d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:41.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:41.917+0000 7f01b9400640 1 -- 172.21.15.131:0/3444333390 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4068196269 0 0) 0x7f01bc06db70 con 0x7f01bc1400c0 2024-05-26T22:51:41.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:41.917+0000 7f01b9400640 1 -- 172.21.15.131:0/3444333390 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f019c005730 con 0x7f01bc1400c0 2024-05-26T22:51:41.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:41.917+0000 7f01b9400640 1 -- 172.21.15.131:0/3444333390 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1866041297 0 0) 0x7f019c005730 con 0x7f01bc1400c0 2024-05-26T22:51:41.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:41.917+0000 7f01b9400640 1 -- 172.21.15.131:0/3444333390 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f01bc06d250 msgr2=0x7f01bc06d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:41.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:41.917+0000 7f01b9400640 1 --2- 172.21.15.131:0/3444333390 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f01bc06d250 0x7f01bc06d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:41.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:41.917+0000 7f01b9400640 1 -- 172.21.15.131:0/3444333390 >> v1:172.21.15.187:6789/0 conn(0x7f01bc15b730 legacy=0x7f01bc15bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:41.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:41.917+0000 7f01b9400640 1 -- 172.21.15.131:0/3444333390 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01bc15c4a0 con 0x7f01bc1400c0 2024-05-26T22:51:41.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:41.917+0000 7f01b9400640 1 -- 172.21.15.131:0/3444333390 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7f01b4001670 con 0x7f01bc1400c0 2024-05-26T22:51:41.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:41.917+0000 7f01b9400640 1 -- 172.21.15.131:0/3444333390 >> v1:172.21.15.131:6789/0 conn(0x7f01bc1400c0 legacy=0x7f01bc1404f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:41.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:41.917+0000 7f01b9400640 1 --2- 172.21.15.131:0/3444333390 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f019c005bf0 0x7f019c006010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:41.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:41.917+0000 7f01b9400640 1 --2- 172.21.15.131:0/3444333390 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f019c006a10 0x7f019c006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:41.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:41.917+0000 7f01b9400640 1 --2- 172.21.15.131:0/3444333390 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f01bc06d250 0x7f019c007460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:41.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:41.917+0000 7f01b9400640 1 -- 172.21.15.131:0/3444333390 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f01bc071260 con 0x7f019c006a10 2024-05-26T22:51:41.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:41.917+0000 7f01b9400640 1 -- 172.21.15.131:0/3444333390 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f019c00dd60 con 0x7f019c005bf0 2024-05-26T22:51:41.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:41.917+0000 7f01b9400640 1 -- 172.21.15.131:0/3444333390 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f019c00df60 con 0x7f01bc06d250 2024-05-26T22:51:41.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:41.917+0000 7f01b9e00640 1 --2- 172.21.15.131:0/3444333390 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f019c006a10 0x7f019c006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:41.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:41.918+0000 7f01bb200640 1 --2- 172.21.15.131:0/3444333390 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f01bc06d250 0x7f019c007460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:41.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:41.918+0000 7f01ba800640 1 --2- 172.21.15.131:0/3444333390 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f019c005bf0 0x7f019c006010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:41.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:41.918+0000 7f01b9e00640 1 -- 172.21.15.131:0/3444333390 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f01bc06d250 msgr2=0x7f019c007460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:41.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:41.918+0000 7f01b9e00640 1 --2- 172.21.15.131:0/3444333390 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f01bc06d250 0x7f019c007460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:41.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:41.918+0000 7f01b9e00640 1 -- 172.21.15.131:0/3444333390 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f019c005bf0 msgr2=0x7f019c006010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:41.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:41.918+0000 7f01b9e00640 1 --2- 172.21.15.131:0/3444333390 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f019c005bf0 0x7f019c006010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:41.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:41.918+0000 7f01b9e00640 1 -- 172.21.15.131:0/3444333390 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f019c005730 con 0x7f019c006a10 2024-05-26T22:51:41.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:41.919+0000 7f01b9e00640 1 --2- 172.21.15.131:0/3444333390 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f019c006a10 0x7f019c006e90 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f01a4001e50 tx=0x7f019c007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e41b90900af535be server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:41.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:41.920+0000 7f01b9400640 1 -- 172.21.15.131:0/3444333390 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a4001a70 con 0x7f019c006a10 2024-05-26T22:51:41.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:41.920+0000 7f01b9400640 1 -- 172.21.15.131:0/3444333390 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01a4003970 con 0x7f019c006a10 2024-05-26T22:51:41.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:41.920+0000 7f01b9400640 1 -- 172.21.15.131:0/3444333390 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a4003d60 con 0x7f019c006a10 2024-05-26T22:51:41.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:41.920+0000 7f01c0a00640 1 -- 172.21.15.131:0/3444333390 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f019c006a10 msgr2=0x7f019c006e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:41.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:41.920+0000 7f01c0a00640 1 --2- 172.21.15.131:0/3444333390 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f019c006a10 0x7f019c006e90 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f01a4001e50 tx=0x7f019c007a40 comp rx=0 tx=0).stop 2024-05-26T22:51:41.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:41.920+0000 7f01bb200640 1 -- 172.21.15.131:0/3444333390 reap_dead start 2024-05-26T22:51:41.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:41.920+0000 7f01c0a00640 1 -- 172.21.15.131:0/3444333390 shutdown_connections 2024-05-26T22:51:41.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:41.920+0000 7f01c0a00640 1 -- 172.21.15.131:0/3444333390 >> 172.21.15.131:0/3444333390 conn(0x7f01bc0b5660 msgr2=0x7f01bc0b5a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:41.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:41.920+0000 7f01c0a00640 1 -- 172.21.15.131:0/3444333390 shutdown_connections 2024-05-26T22:51:41.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:41.920+0000 7f01c0a00640 1 -- 172.21.15.131:0/3444333390 wait complete. 2024-05-26T22:51:41.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:41.922+0000 7f01c0a00640 1 Processor -- start 2024-05-26T22:51:41.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:41.922+0000 7f01c0a00640 1 -- start start 2024-05-26T22:51:41.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:41.923+0000 7f01c0a00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f019c006a10 0x7f01bc152570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:41.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:41.923+0000 7f01c0a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f01bc131470 0x7f01bc1532f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:41.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:41.923+0000 7f01c0a00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f01bc1334c0 0x7f01bc155d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:41.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:41.923+0000 7f01c0a00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f01bc071a70 con 0x7f01bc131470 2024-05-26T22:51:41.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:41.923+0000 7f01c0a00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f01bc071790 con 0x7f01bc1334c0 2024-05-26T22:51:41.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:41.923+0000 7f01c0a00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f01bc072fa0 con 0x7f019c006a10 2024-05-26T22:51:41.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:41.923+0000 7f01b9e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f01bc131470 0x7f01bc1532f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:41.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:41.923+0000 7f01b9e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f01bc131470 0x7f01bc1532f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33534/0 (socket says 172.21.15.131:33534) 2024-05-26T22:51:41.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:41.923+0000 7f01b9e00640 1 -- 172.21.15.131:0/1735973999 learned_addr learned my addr 172.21.15.131:0/1735973999 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:41.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:41.924+0000 7f01b9e00640 1 -- 172.21.15.131:0/1735973999 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f019c006a10 msgr2=0x7f01bc152570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:41.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:41.924+0000 7f01b9e00640 1 --2- 172.21.15.131:0/1735973999 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f019c006a10 0x7f01bc152570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:41.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:41.924+0000 7f01b9e00640 1 -- 172.21.15.131:0/1735973999 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f01bc1334c0 msgr2=0x7f01bc155d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:41.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:41.924+0000 7f01b9e00640 1 --2- 172.21.15.131:0/1735973999 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f01bc1334c0 0x7f01bc155d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:41.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:41.924+0000 7f01b9e00640 1 -- 172.21.15.131:0/1735973999 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01bc157c50 con 0x7f01bc131470 2024-05-26T22:51:41.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:41.924+0000 7f01b9e00640 1 --2- 172.21.15.131:0/1735973999 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f01bc131470 0x7f01bc1532f0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f01a402a920 tx=0x7f01a40014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:41.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:41.924+0000 7f01b2a00640 1 -- 172.21.15.131:0/1735973999 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a4042070 con 0x7f01bc131470 2024-05-26T22:51:41.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:41.924+0000 7f01b2a00640 1 -- 172.21.15.131:0/1735973999 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01a4006d50 con 0x7f01bc131470 2024-05-26T22:51:41.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:41.924+0000 7f01b2a00640 1 -- 172.21.15.131:0/1735973999 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a4037040 con 0x7f01bc131470 2024-05-26T22:51:41.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:41.924+0000 7f01c0a00640 1 -- 172.21.15.131:0/1735973999 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01bc133990 con 0x7f01bc131470 2024-05-26T22:51:41.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:41.925+0000 7f01c0a00640 1 -- 172.21.15.131:0/1735973999 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01bc133f50 con 0x7f01bc131470 2024-05-26T22:51:41.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:41.926+0000 7f01c0a00640 1 -- 172.21.15.131:0/1735973999 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01bc0694e0 con 0x7f01bc131470 2024-05-26T22:51:41.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:41.926+0000 7f01b2a00640 1 -- 172.21.15.131:0/1735973999 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f01a4045070 con 0x7f01bc131470 2024-05-26T22:51:41.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:41.927+0000 7f01b2a00640 1 --2- 172.21.15.131:0/1735973999 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f019806be30 0x7f019806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:41.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:41.927+0000 7f01b2a00640 1 -- 172.21.15.131:0/1735973999 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f01a40aaaf0 con 0x7f01bc131470 2024-05-26T22:51:41.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:41.928+0000 7f01ba800640 1 --2- 172.21.15.131:0/1735973999 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f019806be30 0x7f019806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:41.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:41.928+0000 7f01ba800640 1 --2- 172.21.15.131:0/1735973999 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f019806be30 0x7f019806e2f0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f01bc0aea30 tx=0x7f01ac00b2e0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:41.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:41.932+0000 7f01b2a00640 1 -- 172.21.15.131:0/1735973999 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f01bc157c50 con 0x7f01bc131470 2024-05-26T22:51:42.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:42.151+0000 7f01c0a00640 1 -- 172.21.15.131:0/1735973999 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f01bc0af9b0 con 0x7f01bc131470 2024-05-26T22:51:42.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:42.693+0000 7f01b2a00640 1 -- 172.21.15.131:0/1735973999 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v49) ==== 122+0+0 (secure 0 0 0) 0x7f01a4079400 con 0x7f01bc131470 2024-05-26T22:51:42.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:42.697+0000 7f01c0a00640 1 -- 172.21.15.131:0/1735973999 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f019806be30 msgr2=0x7f019806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:42.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:42.697+0000 7f01c0a00640 1 --2- 172.21.15.131:0/1735973999 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f019806be30 0x7f019806e2f0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f01bc0aea30 tx=0x7f01ac00b2e0 comp rx=0 tx=0).stop 2024-05-26T22:51:42.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:42.697+0000 7f01c0a00640 1 -- 172.21.15.131:0/1735973999 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f01bc131470 msgr2=0x7f01bc1532f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:42.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:42.697+0000 7f01c0a00640 1 --2- 172.21.15.131:0/1735973999 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f01bc131470 0x7f01bc1532f0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f01a402a920 tx=0x7f01a40014a0 comp rx=0 tx=0).stop 2024-05-26T22:51:42.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:42.697+0000 7f01c0a00640 1 -- 172.21.15.131:0/1735973999 shutdown_connections 2024-05-26T22:51:42.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:42.697+0000 7f01c0a00640 1 --2- 172.21.15.131:0/1735973999 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f019806be30 0x7f019806e2f0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:42.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:42.697+0000 7f01c0a00640 1 --2- 172.21.15.131:0/1735973999 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f01bc1334c0 0x7f01bc155d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:42.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:42.697+0000 7f01c0a00640 1 --2- 172.21.15.131:0/1735973999 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f01bc131470 0x7f01bc1532f0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:42.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:42.697+0000 7f01c0a00640 1 --2- 172.21.15.131:0/1735973999 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f019c006a10 0x7f01bc152570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:42.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:42.697+0000 7f01c0a00640 1 -- 172.21.15.131:0/1735973999 >> 172.21.15.131:0/1735973999 conn(0x7f01bc0b5660 msgr2=0x7f01bc153f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:42.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:42.697+0000 7f01c0a00640 1 -- 172.21.15.131:0/1735973999 shutdown_connections 2024-05-26T22:51:42.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:42.697+0000 7f01c0a00640 1 -- 172.21.15.131:0/1735973999 wait complete. 2024-05-26T22:51:42.717 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-26T22:51:42.717 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-26T22:51:42.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:42.918+0000 7fefe2400640 1 Processor -- start 2024-05-26T22:51:42.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:42.918+0000 7fefe2400640 1 -- start start 2024-05-26T22:51:42.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:42.919+0000 7fefe2400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fefdc06f660 0x7fefdc165e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:42.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:42.919+0000 7fefe2400640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fefdc0722a0 con 0x7fefdc155840 2024-05-26T22:51:42.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:42.919+0000 7fefe2400640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fefdc072580 con 0x7fefdc15c3d0 2024-05-26T22:51:42.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:42.919+0000 7fefe2400640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fefdc072040 con 0x7fefdc06f660 2024-05-26T22:51:42.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:42.919+0000 7fefdb400640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fefdc155840 0x7fefdc071930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:35766/0 (socket says 172.21.15.131:35766) 2024-05-26T22:51:42.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:42.919+0000 7fefdb400640 1 -- 172.21.15.131:0/473937877 learned_addr learned my addr 172.21.15.131:0/473937877 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:51:42.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:42.920+0000 7fefdbe00640 1 --2- 172.21.15.131:0/473937877 >> v2:172.21.15.131:3300/0 conn(0x7fefdc06f660 0x7fefdc165e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:42.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:42.920+0000 7fefda000640 1 -- 172.21.15.131:0/473937877 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 246274493 0 0) 0x7fefdc0722a0 con 0x7fefdc155840 2024-05-26T22:51:42.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:42.920+0000 7fefda000640 1 -- 172.21.15.131:0/473937877 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fefc0003520 con 0x7fefdc155840 2024-05-26T22:51:42.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:42.920+0000 7fefdbe00640 1 -- 172.21.15.131:0/473937877 >> v1:172.21.15.187:6789/0 conn(0x7fefdc15c3d0 legacy=0x7fefdc06ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:42.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:42.920+0000 7fefdbe00640 1 -- 172.21.15.131:0/473937877 >> v1:172.21.15.131:6789/0 conn(0x7fefdc155840 legacy=0x7fefdc071930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:42.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:42.920+0000 7fefdbe00640 1 -- 172.21.15.131:0/473937877 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefdc072860 con 0x7fefdc06f660 2024-05-26T22:51:42.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:42.921+0000 7fefdbe00640 1 --2- 172.21.15.131:0/473937877 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fefdc06f660 0x7fefdc165e00 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fefd4009d90 tx=0x7fefd402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d32507cafbb48ac0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:42.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:42.921+0000 7fefda000640 1 -- 172.21.15.131:0/473937877 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefd4007740 con 0x7fefdc06f660 2024-05-26T22:51:42.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:42.921+0000 7fefda000640 1 -- 172.21.15.131:0/473937877 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefd40078e0 con 0x7fefdc06f660 2024-05-26T22:51:42.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:42.921+0000 7fefda000640 1 -- 172.21.15.131:0/473937877 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefd4007c90 con 0x7fefdc06f660 2024-05-26T22:51:42.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:42.922+0000 7fefe2400640 1 -- 172.21.15.131:0/473937877 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fefdc06f660 msgr2=0x7fefdc165e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:42.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:42.922+0000 7fefe2400640 1 --2- 172.21.15.131:0/473937877 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fefdc06f660 0x7fefdc165e00 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fefd4009d90 tx=0x7fefd402a6a0 comp rx=0 tx=0).stop 2024-05-26T22:51:42.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:42.922+0000 7fefe2400640 1 -- 172.21.15.131:0/473937877 shutdown_connections 2024-05-26T22:51:42.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:42.922+0000 7fefe2400640 1 --2- 172.21.15.131:0/473937877 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fefdc06f660 0x7fefdc165e00 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:42.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:42.922+0000 7fefe2400640 1 -- 172.21.15.131:0/473937877 >> 172.21.15.131:0/473937877 conn(0x7fefdc0b1e90 msgr2=0x7fefdc0b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:42.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:42.922+0000 7fefe2400640 1 -- 172.21.15.131:0/473937877 shutdown_connections 2024-05-26T22:51:42.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:42.922+0000 7fefe2400640 1 -- 172.21.15.131:0/473937877 wait complete. 2024-05-26T22:51:42.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:42.924+0000 7fefe2400640 1 Processor -- start 2024-05-26T22:51:42.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:42.924+0000 7fefe2400640 1 -- start start 2024-05-26T22:51:42.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:42.925+0000 7fefe2400640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fefdc06f660 0x7fefdc143470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:42.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:42.925+0000 7fefe2400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fefdc155840 0x7fefdc0afe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:42.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:42.925+0000 7fefe2400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fefdc15c3d0 0x7fefdc0b0340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:42.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:42.925+0000 7fefe2400640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fefdc0aee50 con 0x7fefdc15c3d0 2024-05-26T22:51:42.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:42.925+0000 7fefe2400640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fefdc072e20 con 0x7fefdc06f660 2024-05-26T22:51:42.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:42.925+0000 7fefe2400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fefdc0aefd0 con 0x7fefdc155840 2024-05-26T22:51:42.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:42.925+0000 7fefdbe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fefdc15c3d0 0x7fefdc0b0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:42.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:42.925+0000 7fefdbe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fefdc15c3d0 0x7fefdc0b0340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33546/0 (socket says 172.21.15.131:33546) 2024-05-26T22:51:42.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:42.925+0000 7fefdbe00640 1 -- 172.21.15.131:0/4242433698 learned_addr learned my addr 172.21.15.131:0/4242433698 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:42.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:42.925+0000 7fefdb400640 1 --2- 172.21.15.131:0/4242433698 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fefdc06f660 0x7fefdc143470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:42.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:42.925+0000 7fefdaa00640 1 --2- 172.21.15.131:0/4242433698 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fefdc155840 0x7fefdc0afe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:42.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:42.925+0000 7fefdbe00640 1 -- 172.21.15.131:0/4242433698 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fefdc155840 msgr2=0x7fefdc0afe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:42.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:42.926+0000 7fefdbe00640 1 --2- 172.21.15.131:0/4242433698 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fefdc155840 0x7fefdc0afe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:42.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:42.926+0000 7fefdbe00640 1 -- 172.21.15.131:0/4242433698 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fefdc06f660 msgr2=0x7fefdc143470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:42.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:42.926+0000 7fefdbe00640 1 --2- 172.21.15.131:0/4242433698 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fefdc06f660 0x7fefdc143470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:42.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:42.926+0000 7fefdbe00640 1 -- 172.21.15.131:0/4242433698 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefdc0b2d90 con 0x7fefdc15c3d0 2024-05-26T22:51:42.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:42.926+0000 7fefdbe00640 1 --2- 172.21.15.131:0/4242433698 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fefdc15c3d0 0x7fefdc0b0340 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fefd4009eb0 tx=0x7fefd4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:42.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:42.926+0000 7fefd3400640 1 -- 172.21.15.131:0/4242433698 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefd403f070 con 0x7fefdc15c3d0 2024-05-26T22:51:42.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:42.926+0000 7fefd3400640 1 -- 172.21.15.131:0/4242433698 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefd4031bc0 con 0x7fefdc15c3d0 2024-05-26T22:51:42.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:42.926+0000 7fefe2400640 1 -- 172.21.15.131:0/4242433698 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefdc0b3070 con 0x7fefdc15c3d0 2024-05-26T22:51:42.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:42.926+0000 7fefd3400640 1 -- 172.21.15.131:0/4242433698 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefd4009450 con 0x7fefdc15c3d0 2024-05-26T22:51:42.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:42.926+0000 7fefe2400640 1 -- 172.21.15.131:0/4242433698 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefdc0b3630 con 0x7fefdc15c3d0 2024-05-26T22:51:42.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:42.927+0000 7fefe2400640 1 -- 172.21.15.131:0/4242433698 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefdc06f480 con 0x7fefdc15c3d0 2024-05-26T22:51:42.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:42.928+0000 7fefd3400640 1 -- 172.21.15.131:0/4242433698 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fefd4043410 con 0x7fefdc15c3d0 2024-05-26T22:51:42.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:42.929+0000 7fefd3400640 1 --2- 172.21.15.131:0/4242433698 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fefb806be30 0x7fefb806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:42.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:42.929+0000 7fefd3400640 1 -- 172.21.15.131:0/4242433698 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fefd40ac320 con 0x7fefdc15c3d0 2024-05-26T22:51:42.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:42.930+0000 7fefdb400640 1 --2- 172.21.15.131:0/4242433698 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fefb806be30 0x7fefb806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:42.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:42.930+0000 7fefdb400640 1 --2- 172.21.15.131:0/4242433698 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fefb806be30 0x7fefb806e2f0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fefcc005880 tx=0x7fefcc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:42.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:42.934+0000 7fefd3400640 1 -- 172.21.15.131:0/4242433698 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fefdc06f480 con 0x7fefdc15c3d0 2024-05-26T22:51:43.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.171+0000 7fefe2400640 1 -- 172.21.15.131:0/4242433698 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fefdc0694e0 con 0x7fefdc15c3d0 2024-05-26T22:51:43.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.173+0000 7fefd3400640 1 -- 172.21.15.131:0/4242433698 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) ==== 94+0+5190 (secure 0 0 0) 0x7fefd407ac30 con 0x7fefdc15c3d0 2024-05-26T22:51:43.172 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:51:43.173 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":49,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4832,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/1327386245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":1327386245},{"type":"v1","addr":"172.21.15.187:6835","nonce":1327386245}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":44},{"gid":6378,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6836/2737657374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6835","nonce":2737657374},{"type":"v1","addr":"172.21.15.131:6836","nonce":2737657374}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43},{"gid":6381,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/640544945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":640544945},{"type":"v1","addr":"172.21.15.131:6837","nonce":640544945}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-26T22:51:39.114644+0000","modified":"2024-05-26T22:51:42.152944+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6390},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6390":{"gid":6390,"name":"d","rank":0,"incarnation":45,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6833/3300872633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3300872633},{"type":"v1","addr":"172.21.15.187:6833","nonce":3300872633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6390,"qdb_cluster":[6390]},"id":4}]} 2024-05-26T22:51:43.173 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 49 2024-05-26T22:51:43.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.175+0000 7fefe2400640 1 -- 172.21.15.131:0/4242433698 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fefb806be30 msgr2=0x7fefb806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:43.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.175+0000 7fefe2400640 1 --2- 172.21.15.131:0/4242433698 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fefb806be30 0x7fefb806e2f0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fefcc005880 tx=0x7fefcc009040 comp rx=0 tx=0).stop 2024-05-26T22:51:43.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.175+0000 7fefe2400640 1 -- 172.21.15.131:0/4242433698 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fefdc15c3d0 msgr2=0x7fefdc0b0340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:43.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.175+0000 7fefe2400640 1 --2- 172.21.15.131:0/4242433698 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fefdc15c3d0 0x7fefdc0b0340 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fefd4009eb0 tx=0x7fefd4004a40 comp rx=0 tx=0).stop 2024-05-26T22:51:43.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.176+0000 7fefe2400640 1 -- 172.21.15.131:0/4242433698 shutdown_connections 2024-05-26T22:51:43.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.176+0000 7fefe2400640 1 --2- 172.21.15.131:0/4242433698 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fefdc155840 0x7fefdc0afe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:43.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.176+0000 7fefe2400640 1 --2- 172.21.15.131:0/4242433698 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fefb806be30 0x7fefb806e2f0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:43.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.176+0000 7fefe2400640 1 --2- 172.21.15.131:0/4242433698 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fefdc15c3d0 0x7fefdc0b0340 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:43.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.176+0000 7fefe2400640 1 --2- 172.21.15.131:0/4242433698 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fefdc06f660 0x7fefdc143470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:43.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.176+0000 7fefe2400640 1 -- 172.21.15.131:0/4242433698 >> 172.21.15.131:0/4242433698 conn(0x7fefdc0b1e90 msgr2=0x7fefdc06d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:43.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.176+0000 7fefe2400640 1 -- 172.21.15.131:0/4242433698 shutdown_connections 2024-05-26T22:51:43.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.176+0000 7fefe2400640 1 -- 172.21.15.131:0/4242433698 wait complete. 2024-05-26T22:51:43.194 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-26T22:51:43.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.393+0000 7fcca5c00640 1 Processor -- start 2024-05-26T22:51:43.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.393+0000 7fcca5c00640 1 -- start start 2024-05-26T22:51:43.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.394+0000 7fcca5c00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fcca0144340 0x7fcca00cedf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:43.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.394+0000 7fcca5c00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fcca0071b70 0x7fcca006e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:43.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.394+0000 7fcca5c00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcca0072000 con 0x7fcca006e9f0 2024-05-26T22:51:43.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.394+0000 7fcca5c00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fcca0072eb0 con 0x7fcca0144340 2024-05-26T22:51:43.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.394+0000 7fcca5c00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fcca006ee20 con 0x7fcca0071b70 2024-05-26T22:51:43.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.395+0000 7fcc9fe00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fcca0144340 0x7fcca00cedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:43.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.395+0000 7fcc9fe00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fcca0144340 0x7fcca00cedf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33556/0 (socket says 172.21.15.131:33556) 2024-05-26T22:51:43.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.395+0000 7fcc9fe00640 1 -- 172.21.15.131:0/136431373 learned_addr learned my addr 172.21.15.131:0/136431373 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:43.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.395+0000 7fcc9fe00640 1 -- 172.21.15.131:0/136431373 >> v2:172.21.15.187:3300/0 conn(0x7fcca0071b70 msgr2=0x7fcca006e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:43.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.395+0000 7fcc97400640 1 --2- 172.21.15.131:0/136431373 >> v2:172.21.15.187:3300/0 conn(0x7fcca0071b70 0x7fcca006e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:43.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.395+0000 7fcc9fe00640 1 --2- 172.21.15.131:0/136431373 >> v2:172.21.15.187:3300/0 conn(0x7fcca0071b70 0x7fcca006e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:43.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.395+0000 7fcc9fe00640 1 -- 172.21.15.131:0/136431373 >> v1:172.21.15.187:6789/0 conn(0x7fcca006e9f0 legacy=0x7fcca015ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:43.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.395+0000 7fcc9fe00640 1 -- 172.21.15.131:0/136431373 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcca00722a0 con 0x7fcca0144340 2024-05-26T22:51:43.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.395+0000 7fcc97400640 1 --2- 172.21.15.131:0/136431373 >> v2:172.21.15.187:3300/0 conn(0x7fcca0071b70 0x7fcca006e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:51:43.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.396+0000 7fcc9fe00640 1 --2- 172.21.15.131:0/136431373 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcca0144340 0x7fcca00cedf0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fcca00cfc40 tx=0x7fcc9002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8041037ad7950002 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:43.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.397+0000 7fcc9f400640 1 -- 172.21.15.131:0/136431373 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc90037070 con 0x7fcca0144340 2024-05-26T22:51:43.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.397+0000 7fcc9f400640 1 -- 172.21.15.131:0/136431373 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc90032440 con 0x7fcca0144340 2024-05-26T22:51:43.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.397+0000 7fcc9f400640 1 -- 172.21.15.131:0/136431373 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc90032810 con 0x7fcca0144340 2024-05-26T22:51:43.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.397+0000 7fcca5c00640 1 -- 172.21.15.131:0/136431373 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcca0144340 msgr2=0x7fcca00cedf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:43.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.397+0000 7fcca5c00640 1 --2- 172.21.15.131:0/136431373 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcca0144340 0x7fcca00cedf0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fcca00cfc40 tx=0x7fcc9002a230 comp rx=0 tx=0).stop 2024-05-26T22:51:43.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.397+0000 7fcca5c00640 1 -- 172.21.15.131:0/136431373 shutdown_connections 2024-05-26T22:51:43.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.397+0000 7fcca5c00640 1 --2- 172.21.15.131:0/136431373 >> v2:172.21.15.187:3300/0 conn(0x7fcca0071b70 0x7fcca006e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:43.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.397+0000 7fcca5c00640 1 --2- 172.21.15.131:0/136431373 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcca0144340 0x7fcca00cedf0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:43.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.397+0000 7fcca5c00640 1 -- 172.21.15.131:0/136431373 >> 172.21.15.131:0/136431373 conn(0x7fcca00cff40 msgr2=0x7fcca00d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:43.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.397+0000 7fcca5c00640 1 -- 172.21.15.131:0/136431373 shutdown_connections 2024-05-26T22:51:43.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.398+0000 7fcca5c00640 1 -- 172.21.15.131:0/136431373 wait complete. 2024-05-26T22:51:43.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.399+0000 7fcca5c00640 1 Processor -- start 2024-05-26T22:51:43.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.399+0000 7fcca5c00640 1 -- start start 2024-05-26T22:51:43.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.400+0000 7fcca5c00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcca006e9f0 0x7fcca01583d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:43.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.400+0000 7fcca5c00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcca0071b70 0x7fcca013c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:43.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.400+0000 7fcca5c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcca0144340 0x7fcca013c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:43.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.400+0000 7fcca5c00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fcca0072b40 con 0x7fcca0144340 2024-05-26T22:51:43.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.400+0000 7fcca5c00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fcca0072860 con 0x7fcca006e9f0 2024-05-26T22:51:43.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.400+0000 7fcca5c00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fcca00b4d30 con 0x7fcca0071b70 2024-05-26T22:51:43.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.401+0000 7fcca5200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcca0144340 0x7fcca013c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:43.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.401+0000 7fcca5200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcca0144340 0x7fcca013c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33558/0 (socket says 172.21.15.131:33558) 2024-05-26T22:51:43.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.401+0000 7fcca5200640 1 -- 172.21.15.131:0/2141304352 learned_addr learned my addr 172.21.15.131:0/2141304352 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:43.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.401+0000 7fcc9fe00640 1 --2- 172.21.15.131:0/2141304352 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcca006e9f0 0x7fcca01583d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:43.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.401+0000 7fcca5200640 1 -- 172.21.15.131:0/2141304352 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcca0071b70 msgr2=0x7fcca013c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:43.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.401+0000 7fcca5200640 1 --2- 172.21.15.131:0/2141304352 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcca0071b70 0x7fcca013c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:43.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.401+0000 7fcca5200640 1 -- 172.21.15.131:0/2141304352 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcca006e9f0 msgr2=0x7fcca01583d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:43.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.401+0000 7fcca5200640 1 --2- 172.21.15.131:0/2141304352 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcca006e9f0 0x7fcca01583d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:43.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.401+0000 7fcca5200640 1 -- 172.21.15.131:0/2141304352 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcca01f55b0 con 0x7fcca0144340 2024-05-26T22:51:43.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.401+0000 7fcca5200640 1 --2- 172.21.15.131:0/2141304352 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcca0144340 0x7fcca013c920 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fcc9800c7b0 tx=0x7fcc9800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:43.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.402+0000 7fcc9cc00640 1 -- 172.21.15.131:0/2141304352 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc98017070 con 0x7fcca0144340 2024-05-26T22:51:43.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.402+0000 7fcc9cc00640 1 -- 172.21.15.131:0/2141304352 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc98003da0 con 0x7fcca0144340 2024-05-26T22:51:43.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.402+0000 7fcca5c00640 1 -- 172.21.15.131:0/2141304352 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcca01f5890 con 0x7fcca0144340 2024-05-26T22:51:43.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.402+0000 7fcc9cc00640 1 -- 172.21.15.131:0/2141304352 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc980052b0 con 0x7fcca0144340 2024-05-26T22:51:43.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.402+0000 7fcca5c00640 1 -- 172.21.15.131:0/2141304352 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcca01f5e50 con 0x7fcca0144340 2024-05-26T22:51:43.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.403+0000 7fcca5c00640 1 -- 172.21.15.131:0/2141304352 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcca0074c10 con 0x7fcca0144340 2024-05-26T22:51:43.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.404+0000 7fcc9cc00640 1 -- 172.21.15.131:0/2141304352 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcc9801a070 con 0x7fcca0144340 2024-05-26T22:51:43.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.404+0000 7fcc9cc00640 1 --2- 172.21.15.131:0/2141304352 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fcc7c06be30 0x7fcc7c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:43.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.405+0000 7fcc9cc00640 1 -- 172.21.15.131:0/2141304352 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fcc9808a550 con 0x7fcca0144340 2024-05-26T22:51:43.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.405+0000 7fcc9fe00640 1 --2- 172.21.15.131:0/2141304352 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fcc7c06be30 0x7fcc7c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:43.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.405+0000 7fcc9fe00640 1 --2- 172.21.15.131:0/2141304352 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fcc7c06be30 0x7fcc7c06e2f0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fcc90034000 tx=0x7fcc9002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:43.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.409+0000 7fcc9cc00640 1 -- 172.21.15.131:0/2141304352 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fcc98058e60 con 0x7fcca0144340 2024-05-26T22:51:43.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.594+0000 7fcca5c00640 1 -- 172.21.15.131:0/2141304352 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcca00694e0 con 0x7fcca0144340 2024-05-26T22:51:43.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.596+0000 7fcc9cc00640 1 -- 172.21.15.131:0/2141304352 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) ==== 74+0+16689 (secure 0 0 0) 0x7fcc98058760 con 0x7fcca0144340 2024-05-26T22:51:43.596 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:51:43.596 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":58,"fsid":"46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f","created":"2024-05-26T22:47:57.288905+0000","modified":"2024-05-26T22:51:40.663750+0000","last_up_change":"2024-05-26T22:48:09.602207+0000","last_in_change":"2024-05-26T22:48:00.687341+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-26T22:48:10.463339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-05-26T22:51:36.873056+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-05-26T22:51:38.120229+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"814eaf62-a18d-494b-8e27-fd8fb88adea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6801","nonce":2350940051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6803","nonce":2350940051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6807","nonce":2350940051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6805","nonce":2350940051}]},"public_addr":"172.21.15.131:6801/2350940051","cluster_addr":"172.21.15.131:6803/2350940051","heartbeat_back_addr":"172.21.15.131:6807/2350940051","heartbeat_front_addr":"172.21.15.131:6805/2350940051","state":["exists","up"]},{"osd":1,"uuid":"42e66287-f6c6-4e26-af3c-6dfaa3536d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6825","nonce":1525528247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6827","nonce":1525528247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6831","nonce":1525528247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6829","nonce":1525528247}]},"public_addr":"172.21.15.131:6825/1525528247","cluster_addr":"172.21.15.131:6827/1525528247","heartbeat_back_addr":"172.21.15.131:6831/1525528247","heartbeat_front_addr":"172.21.15.131:6829/1525528247","state":["exists","up"]},{"osd":2,"uuid":"0da34673-04a2-4fd0-b57a-0ff2b679b043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6817","nonce":3627842210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6819","nonce":3627842210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6823","nonce":3627842210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6821","nonce":3627842210}]},"public_addr":"172.21.15.131:6817/3627842210","cluster_addr":"172.21.15.131:6819/3627842210","heartbeat_back_addr":"172.21.15.131:6823/3627842210","heartbeat_front_addr":"172.21.15.131:6821/3627842210","state":["exists","up"]},{"osd":3,"uuid":"cf117a27-d6b2-43d3-ae52-b47988bf8eed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6809","nonce":350790591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6811","nonce":350790591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6815","nonce":350790591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6813","nonce":350790591}]},"public_addr":"172.21.15.131:6809/350790591","cluster_addr":"172.21.15.131:6811/350790591","heartbeat_back_addr":"172.21.15.131:6815/350790591","heartbeat_front_addr":"172.21.15.131:6813/350790591","state":["exists","up"]},{"osd":4,"uuid":"a092ccfc-be89-4a84-88d5-0f781fa715eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6816","nonce":244560995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6819","nonce":244560995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6823","nonce":244560995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6821","nonce":244560995}]},"public_addr":"172.21.15.187:6816/244560995","cluster_addr":"172.21.15.187:6819/244560995","heartbeat_back_addr":"172.21.15.187:6823/244560995","heartbeat_front_addr":"172.21.15.187:6821/244560995","state":["exists","up"]},{"osd":5,"uuid":"7513ba69-dd00-4a35-b190-9e8cc3b2bdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6801","nonce":324619398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6803","nonce":324619398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6812","nonce":324619398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6807","nonce":324619398}]},"public_addr":"172.21.15.187:6801/324619398","cluster_addr":"172.21.15.187:6803/324619398","heartbeat_back_addr":"172.21.15.187:6812/324619398","heartbeat_front_addr":"172.21.15.187:6807/324619398","state":["exists","up"]},{"osd":6,"uuid":"50081810-c093-4d2f-a4d7-b7c695c5400d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6806","nonce":1391347786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6809","nonce":1391347786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6818","nonce":1391347786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6813","nonce":1391347786}]},"public_addr":"172.21.15.187:6806/1391347786","cluster_addr":"172.21.15.187:6809/1391347786","heartbeat_back_addr":"172.21.15.187:6818/1391347786","heartbeat_front_addr":"172.21.15.187:6813/1391347786","state":["exists","up"]},{"osd":7,"uuid":"775dae05-54f5-4ad4-bbce-92863c6ed2ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6825","nonce":1220065622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6827","nonce":1220065622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6831","nonce":1220065622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6829","nonce":1220065622}]},"public_addr":"172.21.15.187:6825/1220065622","cluster_addr":"172.21.15.187:6827/1220065622","heartbeat_back_addr":"172.21.15.187:6831/1220065622","heartbeat_front_addr":"172.21.15.187:6829/1220065622","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:05.983520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.299516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.067323+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.307748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.018742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.258775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.010912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.295355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-26T22:51:43.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.599+0000 7fcca5c00640 1 -- 172.21.15.131:0/2141304352 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fcc7c06be30 msgr2=0x7fcc7c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:43.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.599+0000 7fcca5c00640 1 --2- 172.21.15.131:0/2141304352 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fcc7c06be30 0x7fcc7c06e2f0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fcc90034000 tx=0x7fcc9002e000 comp rx=0 tx=0).stop 2024-05-26T22:51:43.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.599+0000 7fcca5c00640 1 -- 172.21.15.131:0/2141304352 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcca0144340 msgr2=0x7fcca013c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:43.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.599+0000 7fcca5c00640 1 --2- 172.21.15.131:0/2141304352 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcca0144340 0x7fcca013c920 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fcc9800c7b0 tx=0x7fcc9800cc70 comp rx=0 tx=0).stop 2024-05-26T22:51:43.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.599+0000 7fcca5c00640 1 -- 172.21.15.131:0/2141304352 shutdown_connections 2024-05-26T22:51:43.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.599+0000 7fcca5c00640 1 --2- 172.21.15.131:0/2141304352 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcca0071b70 0x7fcca013c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:43.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.599+0000 7fcca5c00640 1 --2- 172.21.15.131:0/2141304352 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fcc7c06be30 0x7fcc7c06e2f0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:43.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.599+0000 7fcca5c00640 1 --2- 172.21.15.131:0/2141304352 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcca0144340 0x7fcca013c920 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:43.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.599+0000 7fcca5c00640 1 --2- 172.21.15.131:0/2141304352 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcca006e9f0 0x7fcca01583d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:43.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.599+0000 7fcca5c00640 1 -- 172.21.15.131:0/2141304352 >> 172.21.15.131:0/2141304352 conn(0x7fcca00cff40 msgr2=0x7fcca015db10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:43.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.599+0000 7fcca5c00640 1 -- 172.21.15.131:0/2141304352 shutdown_connections 2024-05-26T22:51:43.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.599+0000 7fcca5c00640 1 -- 172.21.15.131:0/2141304352 wait complete. 2024-05-26T22:51:43.617 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-26T22:51:43.617 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-26T22:51:43.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.805+0000 7f922a200640 1 Processor -- start 2024-05-26T22:51:43.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.805+0000 7f922a200640 1 -- start start 2024-05-26T22:51:43.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.807+0000 7f922a200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f922414c090 0x7f92240b2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:43.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.807+0000 7f922a200640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f922406df90 0x7f9224071160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:43.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.807+0000 7f922a200640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9224071990 con 0x7f92240cc760 2024-05-26T22:51:43.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.807+0000 7f922a200640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f9224071f50 con 0x7f922414c090 2024-05-26T22:51:43.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.807+0000 7f922a200640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f92240ce150 con 0x7f922406df90 2024-05-26T22:51:43.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.807+0000 7f9228e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f922414c090 0x7f92240b2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:43.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.807+0000 7f9228e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f922414c090 0x7f92240b2e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33566/0 (socket says 172.21.15.131:33566) 2024-05-26T22:51:43.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.807+0000 7f9228e00640 1 -- 172.21.15.131:0/3092725018 learned_addr learned my addr 172.21.15.131:0/3092725018 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:43.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.808+0000 7f921be00640 1 --2- 172.21.15.131:0/3092725018 >> v2:172.21.15.187:3300/0 conn(0x7f922406df90 0x7f9224071160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:43.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.808+0000 7f9228e00640 1 -- 172.21.15.131:0/3092725018 >> v2:172.21.15.187:3300/0 conn(0x7f922406df90 msgr2=0x7f9224071160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:43.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.808+0000 7f9228e00640 1 --2- 172.21.15.131:0/3092725018 >> v2:172.21.15.187:3300/0 conn(0x7f922406df90 0x7f9224071160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:43.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.808+0000 7f9228e00640 1 -- 172.21.15.131:0/3092725018 >> v1:172.21.15.187:6789/0 conn(0x7f92240cc760 legacy=0x7f92240ce5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:43.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.808+0000 7f9228e00640 1 -- 172.21.15.131:0/3092725018 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9224071c70 con 0x7f922414c090 2024-05-26T22:51:43.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.809+0000 7f9228e00640 1 --2- 172.21.15.131:0/3092725018 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f922414c090 0x7f92240b2e10 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f922406e4a0 tx=0x7f921402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1186c2d206a61a55 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:43.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.810+0000 7f9223e00640 1 -- 172.21.15.131:0/3092725018 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9214037070 con 0x7f922414c090 2024-05-26T22:51:43.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.810+0000 7f9223e00640 1 -- 172.21.15.131:0/3092725018 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9214032440 con 0x7f922414c090 2024-05-26T22:51:43.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.810+0000 7f9223e00640 1 -- 172.21.15.131:0/3092725018 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9214032810 con 0x7f922414c090 2024-05-26T22:51:43.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.810+0000 7f922a200640 1 -- 172.21.15.131:0/3092725018 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f922414c090 msgr2=0x7f92240b2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:43.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.810+0000 7f922a200640 1 --2- 172.21.15.131:0/3092725018 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f922414c090 0x7f92240b2e10 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f922406e4a0 tx=0x7f921402a1a0 comp rx=0 tx=0).stop 2024-05-26T22:51:43.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.810+0000 7f922a200640 1 -- 172.21.15.131:0/3092725018 shutdown_connections 2024-05-26T22:51:43.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.810+0000 7f922a200640 1 --2- 172.21.15.131:0/3092725018 >> v2:172.21.15.187:3300/0 conn(0x7f922406df90 0x7f9224071160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:43.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.810+0000 7f922a200640 1 --2- 172.21.15.131:0/3092725018 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f922414c090 0x7f92240b2e10 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:43.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.810+0000 7f922a200640 1 -- 172.21.15.131:0/3092725018 >> 172.21.15.131:0/3092725018 conn(0x7f92240cff40 msgr2=0x7f92240d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:43.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.810+0000 7f922a200640 1 -- 172.21.15.131:0/3092725018 shutdown_connections 2024-05-26T22:51:43.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.810+0000 7f922a200640 1 -- 172.21.15.131:0/3092725018 wait complete. 2024-05-26T22:51:43.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.812+0000 7f922a200640 1 Processor -- start 2024-05-26T22:51:43.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.812+0000 7f922a200640 1 -- start start 2024-05-26T22:51:43.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.813+0000 7f922a200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f922406df90 0x7f9224147b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:43.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.813+0000 7f922a200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f92240cc760 0x7f92240a9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:43.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.813+0000 7f922a200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f922414c090 0x7f9224158b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:43.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.813+0000 7f922a200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f922414c470 con 0x7f92240cc760 2024-05-26T22:51:43.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.813+0000 7f922a200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f92240716a0 con 0x7f922406df90 2024-05-26T22:51:43.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.813+0000 7f922a200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f922414c5f0 con 0x7f922414c090 2024-05-26T22:51:43.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.813+0000 7f921be00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f92240cc760 0x7f92240a9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:43.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.813+0000 7f921be00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f92240cc760 0x7f92240a9ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33578/0 (socket says 172.21.15.131:33578) 2024-05-26T22:51:43.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.813+0000 7f921be00640 1 -- 172.21.15.131:0/853179543 learned_addr learned my addr 172.21.15.131:0/853179543 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:43.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.814+0000 7f921be00640 1 -- 172.21.15.131:0/853179543 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f922414c090 msgr2=0x7f9224158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:43.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.814+0000 7f9228e00640 1 --2- 172.21.15.131:0/853179543 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f922406df90 0x7f9224147b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:43.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.814+0000 7f921be00640 1 --2- 172.21.15.131:0/853179543 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f922414c090 0x7f9224158b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:43.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.814+0000 7f921be00640 1 -- 172.21.15.131:0/853179543 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f922406df90 msgr2=0x7f9224147b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:43.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.814+0000 7f921be00640 1 --2- 172.21.15.131:0/853179543 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f922406df90 0x7f9224147b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:43.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.814+0000 7f921be00640 1 -- 172.21.15.131:0/853179543 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92240b1b80 con 0x7f92240cc760 2024-05-26T22:51:43.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.814+0000 7f921be00640 1 --2- 172.21.15.131:0/853179543 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f92240cc760 0x7f92240a9ee0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f920c00d800 tx=0x7f920c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:43.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.814+0000 7f9221600640 1 -- 172.21.15.131:0/853179543 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f920c00b660 con 0x7f92240cc760 2024-05-26T22:51:43.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.814+0000 7f9221600640 1 -- 172.21.15.131:0/853179543 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f920c019070 con 0x7f92240cc760 2024-05-26T22:51:43.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.814+0000 7f922a200640 1 -- 172.21.15.131:0/853179543 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92240b1e60 con 0x7f92240cc760 2024-05-26T22:51:43.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.814+0000 7f9221600640 1 -- 172.21.15.131:0/853179543 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f920c00ba60 con 0x7f92240cc760 2024-05-26T22:51:43.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.814+0000 7f922a200640 1 -- 172.21.15.131:0/853179543 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92240b2420 con 0x7f92240cc760 2024-05-26T22:51:43.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.815+0000 7f922a200640 1 -- 172.21.15.131:0/853179543 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92240694e0 con 0x7f92240cc760 2024-05-26T22:51:43.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.816+0000 7f9221600640 1 -- 172.21.15.131:0/853179543 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f920c00bc00 con 0x7f92240cc760 2024-05-26T22:51:43.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.817+0000 7f9221600640 1 --2- 172.21.15.131:0/853179543 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f920006be30 0x7f920006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:43.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.817+0000 7f9221600640 1 -- 172.21.15.131:0/853179543 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f920c08b410 con 0x7f92240cc760 2024-05-26T22:51:43.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.817+0000 7f9228e00640 1 --2- 172.21.15.131:0/853179543 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f920006be30 0x7f920006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:43.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.818+0000 7f9228e00640 1 --2- 172.21.15.131:0/853179543 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f920006be30 0x7f920006e2f0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f9214034000 tx=0x7f921402e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:43.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:43.822+0000 7f9221600640 1 -- 172.21.15.131:0/853179543 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f920c059c90 con 0x7f92240cc760 2024-05-26T22:51:44.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.015+0000 7f922a200640 1 -- 172.21.15.131:0/853179543 --> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f92240b3f40 con 0x7f920006be30 2024-05-26T22:51:44.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.034+0000 7f9221600640 1 -- 172.21.15.131:0/853179543 <== mgr.4104 v2:172.21.15.131:6832/3484832947 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251887 (secure 0 0 0) 0x7f92240b3f40 con 0x7f920006be30 2024-05-26T22:51:44.037 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:51:44.039 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-05-26T22:51:44.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.040+0000 7f922a200640 1 -- 172.21.15.131:0/853179543 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f920006be30 msgr2=0x7f920006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:44.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.040+0000 7f922a200640 1 --2- 172.21.15.131:0/853179543 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f920006be30 0x7f920006e2f0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f9214034000 tx=0x7f921402e000 comp rx=0 tx=0).stop 2024-05-26T22:51:44.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.040+0000 7f922a200640 1 -- 172.21.15.131:0/853179543 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f92240cc760 msgr2=0x7f92240a9ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:44.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.040+0000 7f922a200640 1 --2- 172.21.15.131:0/853179543 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f92240cc760 0x7f92240a9ee0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f920c00d800 tx=0x7f920c00dcc0 comp rx=0 tx=0).stop 2024-05-26T22:51:44.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.040+0000 7f922a200640 1 -- 172.21.15.131:0/853179543 shutdown_connections 2024-05-26T22:51:44.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.040+0000 7f922a200640 1 --2- 172.21.15.131:0/853179543 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f922414c090 0x7f9224158b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:44.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.040+0000 7f922a200640 1 --2- 172.21.15.131:0/853179543 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f920006be30 0x7f920006e2f0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:44.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.040+0000 7f922a200640 1 --2- 172.21.15.131:0/853179543 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f92240cc760 0x7f92240a9ee0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:44.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.040+0000 7f922a200640 1 --2- 172.21.15.131:0/853179543 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f922406df90 0x7f9224147b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:44.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.040+0000 7f922a200640 1 -- 172.21.15.131:0/853179543 >> 172.21.15.131:0/853179543 conn(0x7f92240cff40 msgr2=0x7f92240b3390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:44.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.040+0000 7f922a200640 1 -- 172.21.15.131:0/853179543 shutdown_connections 2024-05-26T22:51:44.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.040+0000 7f922a200640 1 -- 172.21.15.131:0/853179543 wait complete. 2024-05-26T22:51:44.058 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":157,"stamp":"2024-05-26T22:51:42.450687+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":124,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":255992,"kb_used_data":6520,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754718728,"statfs":{"total":773094113280,"available":772831977472,"internally_reserved":0,"allocated":6676480,"data_stored":3477033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":38,"apply_latency_ms":38,"commit_latency_ns":38000000,"apply_latency_ns":38000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.825541"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.688105+0000","last_change":"2024-05-26T22:51:39.681899+0000","last_active":"2024-05-26T22:51:39.688105+0000","last_peered":"2024-05-26T22:51:39.688105+0000","last_clean":"2024-05-26T22:51:39.688105+0000","last_became_active":"2024-05-26T22:51:39.681619+0000","last_became_peered":"2024-05-26T22:51:39.681619+0000","last_unstale":"2024-05-26T22:51:39.688105+0000","last_undegraded":"2024-05-26T22:51:39.688105+0000","last_fullsized":"2024-05-26T22:51:39.688105+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:30:12.496898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.699114+0000","last_change":"2024-05-26T22:51:39.686789+0000","last_active":"2024-05-26T22:51:39.699114+0000","last_peered":"2024-05-26T22:51:39.699114+0000","last_clean":"2024-05-26T22:51:39.699114+0000","last_became_active":"2024-05-26T22:51:39.686599+0000","last_became_peered":"2024-05-26T22:51:39.686599+0000","last_unstale":"2024-05-26T22:51:39.699114+0000","last_undegraded":"2024-05-26T22:51:39.699114+0000","last_fullsized":"2024-05-26T22:51:39.699114+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:47:57.346804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.693955+0000","last_change":"2024-05-26T22:51:39.686405+0000","last_active":"2024-05-26T22:51:39.693955+0000","last_peered":"2024-05-26T22:51:39.693955+0000","last_clean":"2024-05-26T22:51:39.693955+0000","last_became_active":"2024-05-26T22:51:39.685726+0000","last_became_peered":"2024-05-26T22:51:39.685726+0000","last_unstale":"2024-05-26T22:51:39.693955+0000","last_undegraded":"2024-05-26T22:51:39.693955+0000","last_fullsized":"2024-05-26T22:51:39.693955+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:31:36.673510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.682768+0000","last_change":"2024-05-26T22:51:39.679886+0000","last_active":"2024-05-26T22:51:39.682768+0000","last_peered":"2024-05-26T22:51:39.682768+0000","last_clean":"2024-05-26T22:51:39.682768+0000","last_became_active":"2024-05-26T22:51:39.679621+0000","last_became_peered":"2024-05-26T22:51:39.679621+0000","last_unstale":"2024-05-26T22:51:39.682768+0000","last_undegraded":"2024-05-26T22:51:39.682768+0000","last_fullsized":"2024-05-26T22:51:39.682768+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:50:28.022562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.685516+0000","last_change":"2024-05-26T22:51:39.676186+0000","last_active":"2024-05-26T22:51:39.685516+0000","last_peered":"2024-05-26T22:51:39.685516+0000","last_clean":"2024-05-26T22:51:39.685516+0000","last_became_active":"2024-05-26T22:51:39.675391+0000","last_became_peered":"2024-05-26T22:51:39.675391+0000","last_unstale":"2024-05-26T22:51:39.685516+0000","last_undegraded":"2024-05-26T22:51:39.685516+0000","last_fullsized":"2024-05-26T22:51:39.685516+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:49:52.321601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.684636+0000","last_change":"2024-05-26T22:51:39.679359+0000","last_active":"2024-05-26T22:51:39.684636+0000","last_peered":"2024-05-26T22:51:39.684636+0000","last_clean":"2024-05-26T22:51:39.684636+0000","last_became_active":"2024-05-26T22:51:39.679082+0000","last_became_peered":"2024-05-26T22:51:39.679082+0000","last_unstale":"2024-05-26T22:51:39.684636+0000","last_undegraded":"2024-05-26T22:51:39.684636+0000","last_fullsized":"2024-05-26T22:51:39.684636+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:53:31.658157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.686989+0000","last_change":"2024-05-26T22:51:39.682487+0000","last_active":"2024-05-26T22:51:39.686989+0000","last_peered":"2024-05-26T22:51:39.686989+0000","last_clean":"2024-05-26T22:51:39.686989+0000","last_became_active":"2024-05-26T22:51:39.681735+0000","last_became_peered":"2024-05-26T22:51:39.681735+0000","last_unstale":"2024-05-26T22:51:39.686989+0000","last_undegraded":"2024-05-26T22:51:39.686989+0000","last_fullsized":"2024-05-26T22:51:39.686989+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:28:52.199885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.702484+0000","last_change":"2024-05-26T22:51:39.699147+0000","last_active":"2024-05-26T22:51:39.702484+0000","last_peered":"2024-05-26T22:51:39.702484+0000","last_clean":"2024-05-26T22:51:39.702484+0000","last_became_active":"2024-05-26T22:51:39.698718+0000","last_became_peered":"2024-05-26T22:51:39.698718+0000","last_unstale":"2024-05-26T22:51:39.702484+0000","last_undegraded":"2024-05-26T22:51:39.702484+0000","last_fullsized":"2024-05-26T22:51:39.702484+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:33:27.761464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.692318+0000","last_change":"2024-05-26T22:51:39.686596+0000","last_active":"2024-05-26T22:51:39.692318+0000","last_peered":"2024-05-26T22:51:39.692318+0000","last_clean":"2024-05-26T22:51:39.692318+0000","last_became_active":"2024-05-26T22:51:39.686257+0000","last_became_peered":"2024-05-26T22:51:39.686257+0000","last_unstale":"2024-05-26T22:51:39.692318+0000","last_undegraded":"2024-05-26T22:51:39.692318+0000","last_fullsized":"2024-05-26T22:51:39.692318+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:31:09.888922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.668932+0000","last_change":"2024-05-26T22:51:39.661444+0000","last_active":"2024-05-26T22:51:39.668932+0000","last_peered":"2024-05-26T22:51:39.668932+0000","last_clean":"2024-05-26T22:51:39.668932+0000","last_became_active":"2024-05-26T22:51:39.660595+0000","last_became_peered":"2024-05-26T22:51:39.660595+0000","last_unstale":"2024-05-26T22:51:39.668932+0000","last_undegraded":"2024-05-26T22:51:39.668932+0000","last_fullsized":"2024-05-26T22:51:39.668932+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:48:04.727497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.688703+0000","last_change":"2024-05-26T22:51:39.684510+0000","last_active":"2024-05-26T22:51:39.688703+0000","last_peered":"2024-05-26T22:51:39.688703+0000","last_clean":"2024-05-26T22:51:39.688703+0000","last_became_active":"2024-05-26T22:51:39.684185+0000","last_became_peered":"2024-05-26T22:51:39.684185+0000","last_unstale":"2024-05-26T22:51:39.688703+0000","last_undegraded":"2024-05-26T22:51:39.688703+0000","last_fullsized":"2024-05-26T22:51:39.688703+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:53:24.309742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.687919+0000","last_change":"2024-05-26T22:51:39.682601+0000","last_active":"2024-05-26T22:51:39.687919+0000","last_peered":"2024-05-26T22:51:39.687919+0000","last_clean":"2024-05-26T22:51:39.687919+0000","last_became_active":"2024-05-26T22:51:39.682027+0000","last_became_peered":"2024-05-26T22:51:39.682027+0000","last_unstale":"2024-05-26T22:51:39.687919+0000","last_undegraded":"2024-05-26T22:51:39.687919+0000","last_fullsized":"2024-05-26T22:51:39.687919+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:33:52.360239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.681500+0000","last_change":"2024-05-26T22:51:39.675281+0000","last_active":"2024-05-26T22:51:39.681500+0000","last_peered":"2024-05-26T22:51:39.681500+0000","last_clean":"2024-05-26T22:51:39.681500+0000","last_became_active":"2024-05-26T22:51:39.674925+0000","last_became_peered":"2024-05-26T22:51:39.674925+0000","last_unstale":"2024-05-26T22:51:39.681500+0000","last_undegraded":"2024-05-26T22:51:39.681500+0000","last_fullsized":"2024-05-26T22:51:39.681500+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:48:04.727497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.687360+0000","last_change":"2024-05-26T22:51:39.682232+0000","last_active":"2024-05-26T22:51:39.687360+0000","last_peered":"2024-05-26T22:51:39.687360+0000","last_clean":"2024-05-26T22:51:39.687360+0000","last_became_active":"2024-05-26T22:51:39.681888+0000","last_became_peered":"2024-05-26T22:51:39.681888+0000","last_unstale":"2024-05-26T22:51:39.687360+0000","last_undegraded":"2024-05-26T22:51:39.687360+0000","last_fullsized":"2024-05-26T22:51:39.687360+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:31:58.182736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.694674+0000","last_change":"2024-05-26T22:51:39.688223+0000","last_active":"2024-05-26T22:51:39.694674+0000","last_peered":"2024-05-26T22:51:39.694674+0000","last_clean":"2024-05-26T22:51:39.694674+0000","last_became_active":"2024-05-26T22:51:39.687937+0000","last_became_peered":"2024-05-26T22:51:39.687937+0000","last_unstale":"2024-05-26T22:51:39.694674+0000","last_undegraded":"2024-05-26T22:51:39.694674+0000","last_fullsized":"2024-05-26T22:51:39.694674+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:28:58.750510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.692586+0000","last_change":"2024-05-26T22:51:39.685935+0000","last_active":"2024-05-26T22:51:39.692586+0000","last_peered":"2024-05-26T22:51:39.692586+0000","last_clean":"2024-05-26T22:51:39.692586+0000","last_became_active":"2024-05-26T22:51:39.685672+0000","last_became_peered":"2024-05-26T22:51:39.685672+0000","last_unstale":"2024-05-26T22:51:39.692586+0000","last_undegraded":"2024-05-26T22:51:39.692586+0000","last_fullsized":"2024-05-26T22:51:39.692586+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:31:53.601456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.696416+0000","last_change":"2024-05-26T22:51:39.683113+0000","last_active":"2024-05-26T22:51:39.696416+0000","last_peered":"2024-05-26T22:51:39.696416+0000","last_clean":"2024-05-26T22:51:39.696416+0000","last_became_active":"2024-05-26T22:51:39.682798+0000","last_became_peered":"2024-05-26T22:51:39.682798+0000","last_unstale":"2024-05-26T22:51:39.696416+0000","last_undegraded":"2024-05-26T22:51:39.696416+0000","last_fullsized":"2024-05-26T22:51:39.696416+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:27:47.119685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.697355+0000","last_change":"2024-05-26T22:51:39.693360+0000","last_active":"2024-05-26T22:51:39.697355+0000","last_peered":"2024-05-26T22:51:39.697355+0000","last_clean":"2024-05-26T22:51:39.697355+0000","last_became_active":"2024-05-26T22:51:39.693037+0000","last_became_peered":"2024-05-26T22:51:39.693037+0000","last_unstale":"2024-05-26T22:51:39.697355+0000","last_undegraded":"2024-05-26T22:51:39.697355+0000","last_fullsized":"2024-05-26T22:51:39.697355+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T23:25:48.753320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.690297+0000","last_change":"2024-05-26T22:51:39.683068+0000","last_active":"2024-05-26T22:51:39.690297+0000","last_peered":"2024-05-26T22:51:39.690297+0000","last_clean":"2024-05-26T22:51:39.690297+0000","last_became_active":"2024-05-26T22:51:39.682205+0000","last_became_peered":"2024-05-26T22:51:39.682205+0000","last_unstale":"2024-05-26T22:51:39.690297+0000","last_undegraded":"2024-05-26T22:51:39.690297+0000","last_fullsized":"2024-05-26T22:51:39.690297+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:29:54.618801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.681593+0000","last_change":"2024-05-26T22:51:39.678024+0000","last_active":"2024-05-26T22:51:39.681593+0000","last_peered":"2024-05-26T22:51:39.681593+0000","last_clean":"2024-05-26T22:51:39.681593+0000","last_became_active":"2024-05-26T22:51:39.677715+0000","last_became_peered":"2024-05-26T22:51:39.677715+0000","last_unstale":"2024-05-26T22:51:39.681593+0000","last_undegraded":"2024-05-26T22:51:39.681593+0000","last_fullsized":"2024-05-26T22:51:39.681593+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:56:06.462281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.691858+0000","last_change":"2024-05-26T22:51:39.683919+0000","last_active":"2024-05-26T22:51:39.691858+0000","last_peered":"2024-05-26T22:51:39.691858+0000","last_clean":"2024-05-26T22:51:39.691858+0000","last_became_active":"2024-05-26T22:51:39.683658+0000","last_became_peered":"2024-05-26T22:51:39.683658+0000","last_unstale":"2024-05-26T22:51:39.691858+0000","last_undegraded":"2024-05-26T22:51:39.691858+0000","last_fullsized":"2024-05-26T22:51:39.691858+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:32:59.337859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.696470+0000","last_change":"2024-05-26T22:51:39.680787+0000","last_active":"2024-05-26T22:51:39.696470+0000","last_peered":"2024-05-26T22:51:39.696470+0000","last_clean":"2024-05-26T22:51:39.696470+0000","last_became_active":"2024-05-26T22:51:39.680463+0000","last_became_peered":"2024-05-26T22:51:39.680463+0000","last_unstale":"2024-05-26T22:51:39.696470+0000","last_undegraded":"2024-05-26T22:51:39.696470+0000","last_fullsized":"2024-05-26T22:51:39.696470+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:42:24.864056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.685953+0000","last_change":"2024-05-26T22:51:39.678261+0000","last_active":"2024-05-26T22:51:39.685953+0000","last_peered":"2024-05-26T22:51:39.685953+0000","last_clean":"2024-05-26T22:51:39.685953+0000","last_became_active":"2024-05-26T22:51:39.677799+0000","last_became_peered":"2024-05-26T22:51:39.677799+0000","last_unstale":"2024-05-26T22:51:39.685953+0000","last_undegraded":"2024-05-26T22:51:39.685953+0000","last_fullsized":"2024-05-26T22:51:39.685953+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:50:37.362604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.680921+0000","last_change":"2024-05-26T22:51:39.676125+0000","last_active":"2024-05-26T22:51:39.680921+0000","last_peered":"2024-05-26T22:51:39.680921+0000","last_clean":"2024-05-26T22:51:39.680921+0000","last_became_active":"2024-05-26T22:51:39.675652+0000","last_became_peered":"2024-05-26T22:51:39.675652+0000","last_unstale":"2024-05-26T22:51:39.680921+0000","last_undegraded":"2024-05-26T22:51:39.680921+0000","last_fullsized":"2024-05-26T22:51:39.680921+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:55:10.612046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.688081+0000","last_change":"2024-05-26T22:51:39.677788+0000","last_active":"2024-05-26T22:51:39.688081+0000","last_peered":"2024-05-26T22:51:39.688081+0000","last_clean":"2024-05-26T22:51:39.688081+0000","last_became_active":"2024-05-26T22:51:39.677212+0000","last_became_peered":"2024-05-26T22:51:39.677212+0000","last_unstale":"2024-05-26T22:51:39.688081+0000","last_undegraded":"2024-05-26T22:51:39.688081+0000","last_fullsized":"2024-05-26T22:51:39.688081+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:53:40.267842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.687416+0000","last_change":"2024-05-26T22:51:39.682252+0000","last_active":"2024-05-26T22:51:39.687416+0000","last_peered":"2024-05-26T22:51:39.687416+0000","last_clean":"2024-05-26T22:51:39.687416+0000","last_became_active":"2024-05-26T22:51:39.681898+0000","last_became_peered":"2024-05-26T22:51:39.681898+0000","last_unstale":"2024-05-26T22:51:39.687416+0000","last_undegraded":"2024-05-26T22:51:39.687416+0000","last_fullsized":"2024-05-26T22:51:39.687416+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:08:00.991256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.692076+0000","last_change":"2024-05-26T22:51:39.685562+0000","last_active":"2024-05-26T22:51:39.692076+0000","last_peered":"2024-05-26T22:51:39.692076+0000","last_clean":"2024-05-26T22:51:39.692076+0000","last_became_active":"2024-05-26T22:51:39.685285+0000","last_became_peered":"2024-05-26T22:51:39.685285+0000","last_unstale":"2024-05-26T22:51:39.692076+0000","last_undegraded":"2024-05-26T22:51:39.692076+0000","last_fullsized":"2024-05-26T22:51:39.692076+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:52:34.474139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.687893+0000","last_change":"2024-05-26T22:51:39.675557+0000","last_active":"2024-05-26T22:51:39.687893+0000","last_peered":"2024-05-26T22:51:39.687893+0000","last_clean":"2024-05-26T22:51:39.687893+0000","last_became_active":"2024-05-26T22:51:39.675258+0000","last_became_peered":"2024-05-26T22:51:39.675258+0000","last_unstale":"2024-05-26T22:51:39.687893+0000","last_undegraded":"2024-05-26T22:51:39.687893+0000","last_fullsized":"2024-05-26T22:51:39.687893+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:15:22.527455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.702776+0000","last_change":"2024-05-26T22:51:39.695156+0000","last_active":"2024-05-26T22:51:39.702776+0000","last_peered":"2024-05-26T22:51:39.702776+0000","last_clean":"2024-05-26T22:51:39.702776+0000","last_became_active":"2024-05-26T22:51:39.689092+0000","last_became_peered":"2024-05-26T22:51:39.689092+0000","last_unstale":"2024-05-26T22:51:39.702776+0000","last_undegraded":"2024-05-26T22:51:39.702776+0000","last_fullsized":"2024-05-26T22:51:39.702776+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:44:31.960643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.693819+0000","last_change":"2024-05-26T22:51:39.689894+0000","last_active":"2024-05-26T22:51:39.693819+0000","last_peered":"2024-05-26T22:51:39.693819+0000","last_clean":"2024-05-26T22:51:39.693819+0000","last_became_active":"2024-05-26T22:51:39.689200+0000","last_became_peered":"2024-05-26T22:51:39.689200+0000","last_unstale":"2024-05-26T22:51:39.693819+0000","last_undegraded":"2024-05-26T22:51:39.693819+0000","last_fullsized":"2024-05-26T22:51:39.693819+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:33:52.360239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.688141+0000","last_change":"2024-05-26T22:51:39.678227+0000","last_active":"2024-05-26T22:51:39.688141+0000","last_peered":"2024-05-26T22:51:39.688141+0000","last_clean":"2024-05-26T22:51:39.688141+0000","last_became_active":"2024-05-26T22:51:39.674765+0000","last_became_peered":"2024-05-26T22:51:39.674765+0000","last_unstale":"2024-05-26T22:51:39.688141+0000","last_undegraded":"2024-05-26T22:51:39.688141+0000","last_fullsized":"2024-05-26T22:51:39.688141+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:07:20.949312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.677932+0000","last_change":"2024-05-26T22:51:39.672133+0000","last_active":"2024-05-26T22:51:39.677932+0000","last_peered":"2024-05-26T22:51:39.677932+0000","last_clean":"2024-05-26T22:51:39.677932+0000","last_became_active":"2024-05-26T22:51:39.671831+0000","last_became_peered":"2024-05-26T22:51:39.671831+0000","last_unstale":"2024-05-26T22:51:39.677932+0000","last_undegraded":"2024-05-26T22:51:39.677932+0000","last_fullsized":"2024-05-26T22:51:39.677932+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:50:28.022562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.677956+0000","last_change":"2024-05-26T22:51:39.672094+0000","last_active":"2024-05-26T22:51:39.677956+0000","last_peered":"2024-05-26T22:51:39.677956+0000","last_clean":"2024-05-26T22:51:39.677956+0000","last_became_active":"2024-05-26T22:51:39.671686+0000","last_became_peered":"2024-05-26T22:51:39.671686+0000","last_unstale":"2024-05-26T22:51:39.677956+0000","last_undegraded":"2024-05-26T22:51:39.677956+0000","last_fullsized":"2024-05-26T22:51:39.677956+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:19:31.293296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"57'4","reported_seq":16,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.705509+0000","last_change":"2024-05-26T22:51:38.676687+0000","last_active":"2024-05-26T22:51:39.705509+0000","last_peered":"2024-05-26T22:51:39.705509+0000","last_clean":"2024-05-26T22:51:39.705509+0000","last_became_active":"2024-05-26T22:51:38.676052+0000","last_became_peered":"2024-05-26T22:51:38.676052+0000","last_unstale":"2024-05-26T22:51:39.705509+0000","last_undegraded":"2024-05-26T22:51:39.705509+0000","last_fullsized":"2024-05-26T22:51:39.705509+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_clean_scrub_stamp":"2024-05-26T22:51:37.608737+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:54:07.054820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.682019+0000","last_change":"2024-05-26T22:51:39.670939+0000","last_active":"2024-05-26T22:51:39.682019+0000","last_peered":"2024-05-26T22:51:39.682019+0000","last_clean":"2024-05-26T22:51:39.682019+0000","last_became_active":"2024-05-26T22:51:39.670665+0000","last_became_peered":"2024-05-26T22:51:39.670665+0000","last_unstale":"2024-05-26T22:51:39.682019+0000","last_undegraded":"2024-05-26T22:51:39.682019+0000","last_fullsized":"2024-05-26T22:51:39.682019+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:55:10.612046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"57'1","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.704272+0000","last_change":"2024-05-26T22:51:38.676783+0000","last_active":"2024-05-26T22:51:39.704272+0000","last_peered":"2024-05-26T22:51:39.704272+0000","last_clean":"2024-05-26T22:51:39.704272+0000","last_became_active":"2024-05-26T22:51:38.676466+0000","last_became_peered":"2024-05-26T22:51:38.676466+0000","last_unstale":"2024-05-26T22:51:39.704272+0000","last_undegraded":"2024-05-26T22:51:39.704272+0000","last_fullsized":"2024-05-26T22:51:39.704272+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_clean_scrub_stamp":"2024-05-26T22:51:37.608737+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:31:52.586288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.691914+0000","last_change":"2024-05-26T22:51:39.686450+0000","last_active":"2024-05-26T22:51:39.691914+0000","last_peered":"2024-05-26T22:51:39.691914+0000","last_clean":"2024-05-26T22:51:39.691914+0000","last_became_active":"2024-05-26T22:51:39.681303+0000","last_became_peered":"2024-05-26T22:51:39.681303+0000","last_unstale":"2024-05-26T22:51:39.691914+0000","last_undegraded":"2024-05-26T22:51:39.691914+0000","last_fullsized":"2024-05-26T22:51:39.691914+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:56:06.462281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"57'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.702326+0000","last_change":"2024-05-26T22:51:38.672736+0000","last_active":"2024-05-26T22:51:39.702326+0000","last_peered":"2024-05-26T22:51:39.702326+0000","last_clean":"2024-05-26T22:51:39.702326+0000","last_became_active":"2024-05-26T22:51:38.672443+0000","last_became_peered":"2024-05-26T22:51:38.672443+0000","last_unstale":"2024-05-26T22:51:39.702326+0000","last_undegraded":"2024-05-26T22:51:39.702326+0000","last_fullsized":"2024-05-26T22:51:39.702326+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_clean_scrub_stamp":"2024-05-26T22:51:37.608737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:31:35.658342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.692676+0000","last_change":"2024-05-26T22:51:39.686321+0000","last_active":"2024-05-26T22:51:39.692676+0000","last_peered":"2024-05-26T22:51:39.692676+0000","last_clean":"2024-05-26T22:51:39.692676+0000","last_became_active":"2024-05-26T22:51:39.684865+0000","last_became_peered":"2024-05-26T22:51:39.684865+0000","last_unstale":"2024-05-26T22:51:39.692676+0000","last_undegraded":"2024-05-26T22:51:39.692676+0000","last_fullsized":"2024-05-26T22:51:39.692676+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:22:14.856913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"57'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.696476+0000","last_change":"2024-05-26T22:51:38.680442+0000","last_active":"2024-05-26T22:51:39.696476+0000","last_peered":"2024-05-26T22:51:39.696476+0000","last_clean":"2024-05-26T22:51:39.696476+0000","last_became_active":"2024-05-26T22:51:38.680133+0000","last_became_peered":"2024-05-26T22:51:38.680133+0000","last_unstale":"2024-05-26T22:51:39.696476+0000","last_undegraded":"2024-05-26T22:51:39.696476+0000","last_fullsized":"2024-05-26T22:51:39.696476+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_clean_scrub_stamp":"2024-05-26T22:51:37.608737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:50:36.347437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.701171+0000","last_change":"2024-05-26T22:51:39.695805+0000","last_active":"2024-05-26T22:51:39.701171+0000","last_peered":"2024-05-26T22:51:39.701171+0000","last_clean":"2024-05-26T22:51:39.701171+0000","last_became_active":"2024-05-26T22:51:39.695515+0000","last_became_peered":"2024-05-26T22:51:39.695515+0000","last_unstale":"2024-05-26T22:51:39.701171+0000","last_undegraded":"2024-05-26T22:51:39.701171+0000","last_fullsized":"2024-05-26T22:51:39.701171+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:25:17.473409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"57'1","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.696821+0000","last_change":"2024-05-26T22:51:38.668602+0000","last_active":"2024-05-26T22:51:39.696821+0000","last_peered":"2024-05-26T22:51:39.696821+0000","last_clean":"2024-05-26T22:51:39.696821+0000","last_became_active":"2024-05-26T22:51:38.668096+0000","last_became_peered":"2024-05-26T22:51:38.668096+0000","last_unstale":"2024-05-26T22:51:39.696821+0000","last_undegraded":"2024-05-26T22:51:39.696821+0000","last_fullsized":"2024-05-26T22:51:39.696821+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_clean_scrub_stamp":"2024-05-26T22:51:37.608737+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:27:29.941288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.702754+0000","last_change":"2024-05-26T22:51:39.694227+0000","last_active":"2024-05-26T22:51:39.702754+0000","last_peered":"2024-05-26T22:51:39.702754+0000","last_clean":"2024-05-26T22:51:39.702754+0000","last_became_active":"2024-05-26T22:51:39.693893+0000","last_became_peered":"2024-05-26T22:51:39.693893+0000","last_unstale":"2024-05-26T22:51:39.702754+0000","last_undegraded":"2024-05-26T22:51:39.702754+0000","last_fullsized":"2024-05-26T22:51:39.702754+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:08:25.990269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"57'1","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.698841+0000","last_change":"2024-05-26T22:51:38.671258+0000","last_active":"2024-05-26T22:51:39.698841+0000","last_peered":"2024-05-26T22:51:39.698841+0000","last_clean":"2024-05-26T22:51:39.698841+0000","last_became_active":"2024-05-26T22:51:38.670971+0000","last_became_peered":"2024-05-26T22:51:38.670971+0000","last_unstale":"2024-05-26T22:51:39.698841+0000","last_undegraded":"2024-05-26T22:51:39.698841+0000","last_fullsized":"2024-05-26T22:51:39.698841+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_clean_scrub_stamp":"2024-05-26T22:51:37.608737+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:50:36.347437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.680892+0000","last_change":"2024-05-26T22:51:39.674354+0000","last_active":"2024-05-26T22:51:39.680892+0000","last_peered":"2024-05-26T22:51:39.680892+0000","last_clean":"2024-05-26T22:51:39.680892+0000","last_became_active":"2024-05-26T22:51:39.674004+0000","last_became_peered":"2024-05-26T22:51:39.674004+0000","last_unstale":"2024-05-26T22:51:39.680892+0000","last_undegraded":"2024-05-26T22:51:39.680892+0000","last_fullsized":"2024-05-26T22:51:39.680892+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T23:30:00.567203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.662472+0000","last_change":"2024-05-26T22:51:38.676905+0000","last_active":"2024-05-26T22:51:39.662472+0000","last_peered":"2024-05-26T22:51:39.662472+0000","last_clean":"2024-05-26T22:51:39.662472+0000","last_became_active":"2024-05-26T22:51:38.676347+0000","last_became_peered":"2024-05-26T22:51:38.676347+0000","last_unstale":"2024-05-26T22:51:39.662472+0000","last_undegraded":"2024-05-26T22:51:39.662472+0000","last_fullsized":"2024-05-26T22:51:39.662472+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_clean_scrub_stamp":"2024-05-26T22:51:37.608737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:56:05.395225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.693021+0000","last_change":"2024-05-26T22:51:39.684604+0000","last_active":"2024-05-26T22:51:39.693021+0000","last_peered":"2024-05-26T22:51:39.693021+0000","last_clean":"2024-05-26T22:51:39.693021+0000","last_became_active":"2024-05-26T22:51:39.682802+0000","last_became_peered":"2024-05-26T22:51:39.682802+0000","last_unstale":"2024-05-26T22:51:39.693021+0000","last_undegraded":"2024-05-26T22:51:39.693021+0000","last_fullsized":"2024-05-26T22:51:39.693021+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:18:17.196608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.669722+0000","last_change":"2024-05-26T22:51:38.675793+0000","last_active":"2024-05-26T22:51:39.669722+0000","last_peered":"2024-05-26T22:51:39.669722+0000","last_clean":"2024-05-26T22:51:39.669722+0000","last_became_active":"2024-05-26T22:51:38.674745+0000","last_became_peered":"2024-05-26T22:51:38.674745+0000","last_unstale":"2024-05-26T22:51:39.669722+0000","last_undegraded":"2024-05-26T22:51:39.669722+0000","last_fullsized":"2024-05-26T22:51:39.669722+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_clean_scrub_stamp":"2024-05-26T22:51:37.608737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:11:47.615881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.687141+0000","last_change":"2024-05-26T22:51:39.683593+0000","last_active":"2024-05-26T22:51:39.687141+0000","last_peered":"2024-05-26T22:51:39.687141+0000","last_clean":"2024-05-26T22:51:39.687141+0000","last_became_active":"2024-05-26T22:51:39.681458+0000","last_became_peered":"2024-05-26T22:51:39.681458+0000","last_unstale":"2024-05-26T22:51:39.687141+0000","last_undegraded":"2024-05-26T22:51:39.687141+0000","last_fullsized":"2024-05-26T22:51:39.687141+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:21:52.907223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.677231+0000","last_change":"2024-05-26T22:51:38.678768+0000","last_active":"2024-05-26T22:51:39.677231+0000","last_peered":"2024-05-26T22:51:39.677231+0000","last_clean":"2024-05-26T22:51:39.677231+0000","last_became_active":"2024-05-26T22:51:38.678131+0000","last_became_peered":"2024-05-26T22:51:38.678131+0000","last_unstale":"2024-05-26T22:51:39.677231+0000","last_undegraded":"2024-05-26T22:51:39.677231+0000","last_fullsized":"2024-05-26T22:51:39.677231+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_clean_scrub_stamp":"2024-05-26T22:51:37.608737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:10:38.410360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.693024+0000","last_change":"2024-05-26T22:51:39.685794+0000","last_active":"2024-05-26T22:51:39.693024+0000","last_peered":"2024-05-26T22:51:39.693024+0000","last_clean":"2024-05-26T22:51:39.693024+0000","last_became_active":"2024-05-26T22:51:39.685510+0000","last_became_peered":"2024-05-26T22:51:39.685510+0000","last_unstale":"2024-05-26T22:51:39.693024+0000","last_undegraded":"2024-05-26T22:51:39.693024+0000","last_fullsized":"2024-05-26T22:51:39.693024+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:54:08.069987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.676650+0000","last_change":"2024-05-26T22:51:38.655494+0000","last_active":"2024-05-26T22:51:39.676650+0000","last_peered":"2024-05-26T22:51:39.676650+0000","last_clean":"2024-05-26T22:51:39.676650+0000","last_became_active":"2024-05-26T22:51:38.655095+0000","last_became_peered":"2024-05-26T22:51:38.655095+0000","last_unstale":"2024-05-26T22:51:39.676650+0000","last_undegraded":"2024-05-26T22:51:39.676650+0000","last_fullsized":"2024-05-26T22:51:39.676650+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_clean_scrub_stamp":"2024-05-26T22:51:37.608737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:22:13.841746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.684555+0000","last_change":"2024-05-26T22:51:39.682477+0000","last_active":"2024-05-26T22:51:39.684555+0000","last_peered":"2024-05-26T22:51:39.684555+0000","last_clean":"2024-05-26T22:51:39.684555+0000","last_became_active":"2024-05-26T22:51:39.682200+0000","last_became_peered":"2024-05-26T22:51:39.682200+0000","last_unstale":"2024-05-26T22:51:39.684555+0000","last_undegraded":"2024-05-26T22:51:39.684555+0000","last_fullsized":"2024-05-26T22:51:39.684555+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:11:48.631048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"57'3","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.702223+0000","last_change":"2024-05-26T22:51:38.678350+0000","last_active":"2024-05-26T22:51:39.702223+0000","last_peered":"2024-05-26T22:51:39.702223+0000","last_clean":"2024-05-26T22:51:39.702223+0000","last_became_active":"2024-05-26T22:51:38.677993+0000","last_became_peered":"2024-05-26T22:51:38.677993+0000","last_unstale":"2024-05-26T22:51:39.702223+0000","last_undegraded":"2024-05-26T22:51:39.702223+0000","last_fullsized":"2024-05-26T22:51:39.702223+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_clean_scrub_stamp":"2024-05-26T22:51:37.608737+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T23:59:47.896772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.693983+0000","last_change":"2024-05-26T22:51:39.686519+0000","last_active":"2024-05-26T22:51:39.693983+0000","last_peered":"2024-05-26T22:51:39.693983+0000","last_clean":"2024-05-26T22:51:39.693983+0000","last_became_active":"2024-05-26T22:51:39.686043+0000","last_became_peered":"2024-05-26T22:51:39.686043+0000","last_unstale":"2024-05-26T22:51:39.693983+0000","last_undegraded":"2024-05-26T22:51:39.693983+0000","last_fullsized":"2024-05-26T22:51:39.693983+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:19:31.293296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"57'6","reported_seq":18,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.705195+0000","last_change":"2024-05-26T22:51:38.674489+0000","last_active":"2024-05-26T22:51:39.705195+0000","last_peered":"2024-05-26T22:51:39.705195+0000","last_clean":"2024-05-26T22:51:39.705195+0000","last_became_active":"2024-05-26T22:51:38.672180+0000","last_became_peered":"2024-05-26T22:51:38.672180+0000","last_unstale":"2024-05-26T22:51:39.705195+0000","last_undegraded":"2024-05-26T22:51:39.705195+0000","last_fullsized":"2024-05-26T22:51:39.705195+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_clean_scrub_stamp":"2024-05-26T22:51:37.608737+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:50:35.189821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.690880+0000","last_change":"2024-05-26T22:51:39.685191+0000","last_active":"2024-05-26T22:51:39.690880+0000","last_peered":"2024-05-26T22:51:39.690880+0000","last_clean":"2024-05-26T22:51:39.690880+0000","last_became_active":"2024-05-26T22:51:39.684917+0000","last_became_peered":"2024-05-26T22:51:39.684917+0000","last_unstale":"2024-05-26T22:51:39.690880+0000","last_undegraded":"2024-05-26T22:51:39.690880+0000","last_fullsized":"2024-05-26T22:51:39.690880+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:53:31.658157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"57'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.704976+0000","last_change":"2024-05-26T22:51:38.663472+0000","last_active":"2024-05-26T22:51:39.704976+0000","last_peered":"2024-05-26T22:51:39.704976+0000","last_clean":"2024-05-26T22:51:39.704976+0000","last_became_active":"2024-05-26T22:51:38.663084+0000","last_became_peered":"2024-05-26T22:51:38.663084+0000","last_unstale":"2024-05-26T22:51:39.704976+0000","last_undegraded":"2024-05-26T22:51:39.704976+0000","last_fullsized":"2024-05-26T22:51:39.704976+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_clean_scrub_stamp":"2024-05-26T22:51:37.608737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T23:25:47.738153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.692999+0000","last_change":"2024-05-26T22:51:39.686638+0000","last_active":"2024-05-26T22:51:39.692999+0000","last_peered":"2024-05-26T22:51:39.692999+0000","last_clean":"2024-05-26T22:51:39.692999+0000","last_became_active":"2024-05-26T22:51:39.685858+0000","last_became_peered":"2024-05-26T22:51:39.685858+0000","last_unstale":"2024-05-26T22:51:39.692999+0000","last_undegraded":"2024-05-26T22:51:39.692999+0000","last_fullsized":"2024-05-26T22:51:39.692999+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:50:28.022562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.650524+0000","last_change":"2024-05-26T22:51:38.674401+0000","last_active":"2024-05-26T22:51:39.650524+0000","last_peered":"2024-05-26T22:51:39.650524+0000","last_clean":"2024-05-26T22:51:39.650524+0000","last_became_active":"2024-05-26T22:51:38.673184+0000","last_became_peered":"2024-05-26T22:51:38.673184+0000","last_unstale":"2024-05-26T22:51:39.650524+0000","last_undegraded":"2024-05-26T22:51:39.650524+0000","last_fullsized":"2024-05-26T22:51:39.650524+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_clean_scrub_stamp":"2024-05-26T22:51:37.608737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:09:25.370801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.687046+0000","last_change":"2024-05-26T22:51:39.683074+0000","last_active":"2024-05-26T22:51:39.687046+0000","last_peered":"2024-05-26T22:51:39.687046+0000","last_clean":"2024-05-26T22:51:39.687046+0000","last_became_active":"2024-05-26T22:51:39.680613+0000","last_became_peered":"2024-05-26T22:51:39.680613+0000","last_unstale":"2024-05-26T22:51:39.687046+0000","last_undegraded":"2024-05-26T22:51:39.687046+0000","last_fullsized":"2024-05-26T22:51:39.687046+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:21:18.774120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.662469+0000","last_change":"2024-05-26T22:51:38.677513+0000","last_active":"2024-05-26T22:51:39.662469+0000","last_peered":"2024-05-26T22:51:39.662469+0000","last_clean":"2024-05-26T22:51:39.662469+0000","last_became_active":"2024-05-26T22:51:38.677223+0000","last_became_peered":"2024-05-26T22:51:38.677223+0000","last_unstale":"2024-05-26T22:51:39.662469+0000","last_undegraded":"2024-05-26T22:51:39.662469+0000","last_fullsized":"2024-05-26T22:51:39.662469+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_clean_scrub_stamp":"2024-05-26T22:51:37.608737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T23:59:47.896772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.685978+0000","last_change":"2024-05-26T22:51:39.679663+0000","last_active":"2024-05-26T22:51:39.685978+0000","last_peered":"2024-05-26T22:51:39.685978+0000","last_clean":"2024-05-26T22:51:39.685978+0000","last_became_active":"2024-05-26T22:51:39.678171+0000","last_became_peered":"2024-05-26T22:51:39.678171+0000","last_unstale":"2024-05-26T22:51:39.685978+0000","last_undegraded":"2024-05-26T22:51:39.685978+0000","last_fullsized":"2024-05-26T22:51:39.685978+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:22:56.866177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"57'8","reported_seq":20,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.704953+0000","last_change":"2024-05-26T22:51:38.679123+0000","last_active":"2024-05-26T22:51:39.704953+0000","last_peered":"2024-05-26T22:51:39.704953+0000","last_clean":"2024-05-26T22:51:39.704953+0000","last_became_active":"2024-05-26T22:51:38.678677+0000","last_became_peered":"2024-05-26T22:51:38.678677+0000","last_unstale":"2024-05-26T22:51:39.704953+0000","last_undegraded":"2024-05-26T22:51:39.704953+0000","last_fullsized":"2024-05-26T22:51:39.704953+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_clean_scrub_stamp":"2024-05-26T22:51:37.608737+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:56:05.395225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.693030+0000","last_change":"2024-05-26T22:51:39.688473+0000","last_active":"2024-05-26T22:51:39.693030+0000","last_peered":"2024-05-26T22:51:39.693030+0000","last_clean":"2024-05-26T22:51:39.693030+0000","last_became_active":"2024-05-26T22:51:39.685899+0000","last_became_peered":"2024-05-26T22:51:39.685899+0000","last_unstale":"2024-05-26T22:51:39.693030+0000","last_undegraded":"2024-05-26T22:51:39.693030+0000","last_fullsized":"2024-05-26T22:51:39.693030+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:08:00.991256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"57'1","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.704908+0000","last_change":"2024-05-26T22:51:38.662805+0000","last_active":"2024-05-26T22:51:39.704908+0000","last_peered":"2024-05-26T22:51:39.704908+0000","last_clean":"2024-05-26T22:51:39.704908+0000","last_became_active":"2024-05-26T22:51:38.662504+0000","last_became_peered":"2024-05-26T22:51:38.662504+0000","last_unstale":"2024-05-26T22:51:39.704908+0000","last_undegraded":"2024-05-26T22:51:39.704908+0000","last_fullsized":"2024-05-26T22:51:39.704908+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_clean_scrub_stamp":"2024-05-26T22:51:37.608737+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:28:48.523435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.683284+0000","last_change":"2024-05-26T22:51:39.679084+0000","last_active":"2024-05-26T22:51:39.683284+0000","last_peered":"2024-05-26T22:51:39.683284+0000","last_clean":"2024-05-26T22:51:39.683284+0000","last_became_active":"2024-05-26T22:51:39.678447+0000","last_became_peered":"2024-05-26T22:51:39.678447+0000","last_unstale":"2024-05-26T22:51:39.683284+0000","last_undegraded":"2024-05-26T22:51:39.683284+0000","last_fullsized":"2024-05-26T22:51:39.683284+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:19:31.293296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"57'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.699771+0000","last_change":"2024-05-26T22:51:38.664104+0000","last_active":"2024-05-26T22:51:39.699771+0000","last_peered":"2024-05-26T22:51:39.699771+0000","last_clean":"2024-05-26T22:51:39.699771+0000","last_became_active":"2024-05-26T22:51:38.663821+0000","last_became_peered":"2024-05-26T22:51:38.663821+0000","last_unstale":"2024-05-26T22:51:39.699771+0000","last_undegraded":"2024-05-26T22:51:39.699771+0000","last_fullsized":"2024-05-26T22:51:39.699771+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_clean_scrub_stamp":"2024-05-26T22:51:37.608737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:32:58.322692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.701440+0000","last_change":"2024-05-26T22:51:39.697587+0000","last_active":"2024-05-26T22:51:39.701440+0000","last_peered":"2024-05-26T22:51:39.701440+0000","last_clean":"2024-05-26T22:51:39.701440+0000","last_became_active":"2024-05-26T22:51:39.697215+0000","last_became_peered":"2024-05-26T22:51:39.697215+0000","last_unstale":"2024-05-26T22:51:39.701440+0000","last_undegraded":"2024-05-26T22:51:39.701440+0000","last_fullsized":"2024-05-26T22:51:39.701440+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:48:59.747131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"57'1","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.698228+0000","last_change":"2024-05-26T22:51:38.680443+0000","last_active":"2024-05-26T22:51:39.698228+0000","last_peered":"2024-05-26T22:51:39.698228+0000","last_clean":"2024-05-26T22:51:39.698228+0000","last_became_active":"2024-05-26T22:51:38.680110+0000","last_became_peered":"2024-05-26T22:51:38.680110+0000","last_unstale":"2024-05-26T22:51:39.698228+0000","last_undegraded":"2024-05-26T22:51:39.698228+0000","last_fullsized":"2024-05-26T22:51:39.698228+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_clean_scrub_stamp":"2024-05-26T22:51:37.608737+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:32:58.322692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.694001+0000","last_change":"2024-05-26T22:51:39.690545+0000","last_active":"2024-05-26T22:51:39.694001+0000","last_peered":"2024-05-26T22:51:39.694001+0000","last_clean":"2024-05-26T22:51:39.694001+0000","last_became_active":"2024-05-26T22:51:39.690244+0000","last_became_peered":"2024-05-26T22:51:39.690244+0000","last_unstale":"2024-05-26T22:51:39.694001+0000","last_undegraded":"2024-05-26T22:51:39.694001+0000","last_fullsized":"2024-05-26T22:51:39.694001+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:53:31.658157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.662098+0000","last_change":"2024-05-26T22:51:38.669746+0000","last_active":"2024-05-26T22:51:39.662098+0000","last_peered":"2024-05-26T22:51:39.662098+0000","last_clean":"2024-05-26T22:51:39.662098+0000","last_became_active":"2024-05-26T22:51:38.667884+0000","last_became_peered":"2024-05-26T22:51:38.667884+0000","last_unstale":"2024-05-26T22:51:39.662098+0000","last_undegraded":"2024-05-26T22:51:39.662098+0000","last_fullsized":"2024-05-26T22:51:39.662098+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_clean_scrub_stamp":"2024-05-26T22:51:37.608737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:22:55.851009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.680474+0000","last_change":"2024-05-26T22:51:39.674860+0000","last_active":"2024-05-26T22:51:39.680474+0000","last_peered":"2024-05-26T22:51:39.680474+0000","last_clean":"2024-05-26T22:51:39.680474+0000","last_became_active":"2024-05-26T22:51:39.674521+0000","last_became_peered":"2024-05-26T22:51:39.674521+0000","last_unstale":"2024-05-26T22:51:39.680474+0000","last_undegraded":"2024-05-26T22:51:39.680474+0000","last_fullsized":"2024-05-26T22:51:39.680474+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:31:36.673510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"57'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.707850+0000","last_change":"2024-05-26T22:51:38.669282+0000","last_active":"2024-05-26T22:51:39.707850+0000","last_peered":"2024-05-26T22:51:39.707850+0000","last_clean":"2024-05-26T22:51:39.707850+0000","last_became_active":"2024-05-26T22:51:38.668954+0000","last_became_peered":"2024-05-26T22:51:38.668954+0000","last_unstale":"2024-05-26T22:51:39.707850+0000","last_undegraded":"2024-05-26T22:51:39.707850+0000","last_fullsized":"2024-05-26T22:51:39.707850+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_clean_scrub_stamp":"2024-05-26T22:51:37.608737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:52:33.458972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.686243+0000","last_change":"2024-05-26T22:51:39.680673+0000","last_active":"2024-05-26T22:51:39.686243+0000","last_peered":"2024-05-26T22:51:39.686243+0000","last_clean":"2024-05-26T22:51:39.686243+0000","last_became_active":"2024-05-26T22:51:39.680407+0000","last_became_peered":"2024-05-26T22:51:39.680407+0000","last_unstale":"2024-05-26T22:51:39.686243+0000","last_undegraded":"2024-05-26T22:51:39.686243+0000","last_fullsized":"2024-05-26T22:51:39.686243+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T23:25:48.753320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.657275+0000","last_change":"2024-05-26T22:51:38.669083+0000","last_active":"2024-05-26T22:51:39.657275+0000","last_peered":"2024-05-26T22:51:39.657275+0000","last_clean":"2024-05-26T22:51:39.657275+0000","last_became_active":"2024-05-26T22:51:38.668811+0000","last_became_peered":"2024-05-26T22:51:38.668811+0000","last_unstale":"2024-05-26T22:51:39.657275+0000","last_undegraded":"2024-05-26T22:51:39.657275+0000","last_fullsized":"2024-05-26T22:51:39.657275+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_clean_scrub_stamp":"2024-05-26T22:51:37.608737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:10:19.722657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.680490+0000","last_change":"2024-05-26T22:51:39.675832+0000","last_active":"2024-05-26T22:51:39.680490+0000","last_peered":"2024-05-26T22:51:39.680490+0000","last_clean":"2024-05-26T22:51:39.680490+0000","last_became_active":"2024-05-26T22:51:39.675480+0000","last_became_peered":"2024-05-26T22:51:39.675480+0000","last_unstale":"2024-05-26T22:51:39.680490+0000","last_undegraded":"2024-05-26T22:51:39.680490+0000","last_fullsized":"2024-05-26T22:51:39.680490+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:08:00.991256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.662500+0000","last_change":"2024-05-26T22:51:38.673393+0000","last_active":"2024-05-26T22:51:39.662500+0000","last_peered":"2024-05-26T22:51:39.662500+0000","last_clean":"2024-05-26T22:51:39.662500+0000","last_became_active":"2024-05-26T22:51:38.673070+0000","last_became_peered":"2024-05-26T22:51:38.673070+0000","last_unstale":"2024-05-26T22:51:39.662500+0000","last_undegraded":"2024-05-26T22:51:39.662500+0000","last_fullsized":"2024-05-26T22:51:39.662500+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_clean_scrub_stamp":"2024-05-26T22:51:37.608737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:10:38.410360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.680408+0000","last_change":"2024-05-26T22:51:39.674633+0000","last_active":"2024-05-26T22:51:39.680408+0000","last_peered":"2024-05-26T22:51:39.680408+0000","last_clean":"2024-05-26T22:51:39.680408+0000","last_became_active":"2024-05-26T22:51:39.674316+0000","last_became_peered":"2024-05-26T22:51:39.674316+0000","last_unstale":"2024-05-26T22:51:39.680408+0000","last_undegraded":"2024-05-26T22:51:39.680408+0000","last_fullsized":"2024-05-26T22:51:39.680408+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:54:08.069987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"57'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.701985+0000","last_change":"2024-05-26T22:51:38.667102+0000","last_active":"2024-05-26T22:51:39.701985+0000","last_peered":"2024-05-26T22:51:39.701985+0000","last_clean":"2024-05-26T22:51:39.701985+0000","last_became_active":"2024-05-26T22:51:38.666792+0000","last_became_peered":"2024-05-26T22:51:38.666792+0000","last_unstale":"2024-05-26T22:51:39.701985+0000","last_undegraded":"2024-05-26T22:51:39.701985+0000","last_fullsized":"2024-05-26T22:51:39.701985+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_clean_scrub_stamp":"2024-05-26T22:51:37.608737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:09:25.370801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.680416+0000","last_change":"2024-05-26T22:51:39.674265+0000","last_active":"2024-05-26T22:51:39.680416+0000","last_peered":"2024-05-26T22:51:39.680416+0000","last_clean":"2024-05-26T22:51:39.680416+0000","last_became_active":"2024-05-26T22:51:39.673905+0000","last_became_peered":"2024-05-26T22:51:39.673905+0000","last_unstale":"2024-05-26T22:51:39.680416+0000","last_undegraded":"2024-05-26T22:51:39.680416+0000","last_fullsized":"2024-05-26T22:51:39.680416+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:48:18.385788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.668597+0000","last_change":"2024-05-26T22:51:38.667697+0000","last_active":"2024-05-26T22:51:39.668597+0000","last_peered":"2024-05-26T22:51:39.668597+0000","last_clean":"2024-05-26T22:51:39.668597+0000","last_became_active":"2024-05-26T22:51:38.666882+0000","last_became_peered":"2024-05-26T22:51:38.666882+0000","last_unstale":"2024-05-26T22:51:39.668597+0000","last_undegraded":"2024-05-26T22:51:39.668597+0000","last_fullsized":"2024-05-26T22:51:39.668597+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_clean_scrub_stamp":"2024-05-26T22:51:37.608737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:48:17.370621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.680888+0000","last_change":"2024-05-26T22:51:39.672768+0000","last_active":"2024-05-26T22:51:39.680888+0000","last_peered":"2024-05-26T22:51:39.680888+0000","last_clean":"2024-05-26T22:51:39.680888+0000","last_became_active":"2024-05-26T22:51:39.672501+0000","last_became_peered":"2024-05-26T22:51:39.672501+0000","last_unstale":"2024-05-26T22:51:39.680888+0000","last_undegraded":"2024-05-26T22:51:39.680888+0000","last_fullsized":"2024-05-26T22:51:39.680888+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:48:34.561071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.650660+0000","last_change":"2024-05-26T22:51:38.676020+0000","last_active":"2024-05-26T22:51:39.650660+0000","last_peered":"2024-05-26T22:51:39.650660+0000","last_clean":"2024-05-26T22:51:39.650660+0000","last_became_active":"2024-05-26T22:51:38.675705+0000","last_became_peered":"2024-05-26T22:51:38.675705+0000","last_unstale":"2024-05-26T22:51:39.650660+0000","last_undegraded":"2024-05-26T22:51:39.650660+0000","last_fullsized":"2024-05-26T22:51:39.650660+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_clean_scrub_stamp":"2024-05-26T22:51:37.608737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:10:19.722657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.684805+0000","last_change":"2024-05-26T22:51:39.677822+0000","last_active":"2024-05-26T22:51:39.684805+0000","last_peered":"2024-05-26T22:51:39.684805+0000","last_clean":"2024-05-26T22:51:39.684805+0000","last_became_active":"2024-05-26T22:51:39.677458+0000","last_became_peered":"2024-05-26T22:51:39.677458+0000","last_unstale":"2024-05-26T22:51:39.684805+0000","last_undegraded":"2024-05-26T22:51:39.684805+0000","last_fullsized":"2024-05-26T22:51:39.684805+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:33:52.360239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.670789+0000","last_change":"2024-05-26T22:51:38.679591+0000","last_active":"2024-05-26T22:51:39.670789+0000","last_peered":"2024-05-26T22:51:39.670789+0000","last_clean":"2024-05-26T22:51:39.670789+0000","last_became_active":"2024-05-26T22:51:38.679262+0000","last_became_peered":"2024-05-26T22:51:38.679262+0000","last_unstale":"2024-05-26T22:51:39.670789+0000","last_undegraded":"2024-05-26T22:51:39.670789+0000","last_fullsized":"2024-05-26T22:51:39.670789+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_clean_scrub_stamp":"2024-05-26T22:51:37.608737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:22:55.851009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.701198+0000","last_change":"2024-05-26T22:51:39.696587+0000","last_active":"2024-05-26T22:51:39.701198+0000","last_peered":"2024-05-26T22:51:39.701198+0000","last_clean":"2024-05-26T22:51:39.701198+0000","last_became_active":"2024-05-26T22:51:39.696269+0000","last_became_peered":"2024-05-26T22:51:39.696269+0000","last_unstale":"2024-05-26T22:51:39.701198+0000","last_undegraded":"2024-05-26T22:51:39.701198+0000","last_fullsized":"2024-05-26T22:51:39.701198+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:22:19.684542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.653783+0000","last_change":"2024-05-26T22:51:38.675053+0000","last_active":"2024-05-26T22:51:39.653783+0000","last_peered":"2024-05-26T22:51:39.653783+0000","last_clean":"2024-05-26T22:51:39.653783+0000","last_became_active":"2024-05-26T22:51:38.674504+0000","last_became_peered":"2024-05-26T22:51:38.674504+0000","last_unstale":"2024-05-26T22:51:39.653783+0000","last_undegraded":"2024-05-26T22:51:39.653783+0000","last_fullsized":"2024-05-26T22:51:39.653783+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_clean_scrub_stamp":"2024-05-26T22:51:37.608737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:54:41.244332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.691889+0000","last_change":"2024-05-26T22:51:39.684412+0000","last_active":"2024-05-26T22:51:39.691889+0000","last_peered":"2024-05-26T22:51:39.691889+0000","last_clean":"2024-05-26T22:51:39.691889+0000","last_became_active":"2024-05-26T22:51:39.683555+0000","last_became_peered":"2024-05-26T22:51:39.683555+0000","last_unstale":"2024-05-26T22:51:39.691889+0000","last_undegraded":"2024-05-26T22:51:39.691889+0000","last_fullsized":"2024-05-26T22:51:39.691889+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:48:18.385788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.668501+0000","last_change":"2024-05-26T22:51:38.672448+0000","last_active":"2024-05-26T22:51:39.668501+0000","last_peered":"2024-05-26T22:51:39.668501+0000","last_clean":"2024-05-26T22:51:39.668501+0000","last_became_active":"2024-05-26T22:51:38.672116+0000","last_became_peered":"2024-05-26T22:51:38.672116+0000","last_unstale":"2024-05-26T22:51:39.668501+0000","last_undegraded":"2024-05-26T22:51:39.668501+0000","last_fullsized":"2024-05-26T22:51:39.668501+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_clean_scrub_stamp":"2024-05-26T22:51:37.608737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:53:23.294575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.670200+0000","last_change":"2024-05-26T22:51:39.659441+0000","last_active":"2024-05-26T22:51:39.670200+0000","last_peered":"2024-05-26T22:51:39.670200+0000","last_clean":"2024-05-26T22:51:39.670200+0000","last_became_active":"2024-05-26T22:51:39.658590+0000","last_became_peered":"2024-05-26T22:51:39.658590+0000","last_unstale":"2024-05-26T22:51:39.670200+0000","last_undegraded":"2024-05-26T22:51:39.670200+0000","last_fullsized":"2024-05-26T22:51:39.670200+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T23:30:00.567203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.652829+0000","last_change":"2024-05-26T22:51:38.671647+0000","last_active":"2024-05-26T22:51:39.652829+0000","last_peered":"2024-05-26T22:51:39.652829+0000","last_clean":"2024-05-26T22:51:39.652829+0000","last_became_active":"2024-05-26T22:51:38.671341+0000","last_became_peered":"2024-05-26T22:51:38.671341+0000","last_unstale":"2024-05-26T22:51:39.652829+0000","last_undegraded":"2024-05-26T22:51:39.652829+0000","last_fullsized":"2024-05-26T22:51:39.652829+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_clean_scrub_stamp":"2024-05-26T22:51:37.608737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:05:58.675660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.692982+0000","last_change":"2024-05-26T22:51:39.683047+0000","last_active":"2024-05-26T22:51:39.692982+0000","last_peered":"2024-05-26T22:51:39.692982+0000","last_clean":"2024-05-26T22:51:39.692982+0000","last_became_active":"2024-05-26T22:51:39.682464+0000","last_became_peered":"2024-05-26T22:51:39.682464+0000","last_unstale":"2024-05-26T22:51:39.692982+0000","last_undegraded":"2024-05-26T22:51:39.692982+0000","last_fullsized":"2024-05-26T22:51:39.692982+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T23:57:52.148119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.684774+0000","last_change":"2024-05-26T22:51:38.670209+0000","last_active":"2024-05-26T22:51:39.684774+0000","last_peered":"2024-05-26T22:51:39.684774+0000","last_clean":"2024-05-26T22:51:39.684774+0000","last_became_active":"2024-05-26T22:51:38.669932+0000","last_became_peered":"2024-05-26T22:51:38.669932+0000","last_unstale":"2024-05-26T22:51:39.684774+0000","last_undegraded":"2024-05-26T22:51:39.684774+0000","last_fullsized":"2024-05-26T22:51:39.684774+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_clean_scrub_stamp":"2024-05-26T22:51:37.608737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:38:10.691783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.699284+0000","last_change":"2024-05-26T22:51:39.686302+0000","last_active":"2024-05-26T22:51:39.699284+0000","last_peered":"2024-05-26T22:51:39.699284+0000","last_clean":"2024-05-26T22:51:39.699284+0000","last_became_active":"2024-05-26T22:51:39.686044+0000","last_became_peered":"2024-05-26T22:51:39.686044+0000","last_unstale":"2024-05-26T22:51:39.699284+0000","last_undegraded":"2024-05-26T22:51:39.699284+0000","last_fullsized":"2024-05-26T22:51:39.699284+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:05:00.695645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.650132+0000","last_change":"2024-05-26T22:51:38.673169+0000","last_active":"2024-05-26T22:51:39.650132+0000","last_peered":"2024-05-26T22:51:39.650132+0000","last_clean":"2024-05-26T22:51:39.650132+0000","last_became_active":"2024-05-26T22:51:38.672105+0000","last_became_peered":"2024-05-26T22:51:38.672105+0000","last_unstale":"2024-05-26T22:51:39.650132+0000","last_undegraded":"2024-05-26T22:51:39.650132+0000","last_fullsized":"2024-05-26T22:51:39.650132+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_clean_scrub_stamp":"2024-05-26T22:51:37.608737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:54:41.244332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":238,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.667362+0000","last_change":"2024-05-26T22:48:12.068917+0000","last_active":"2024-05-26T22:51:39.667362+0000","last_peered":"2024-05-26T22:51:39.667362+0000","last_clean":"2024-05-26T22:51:39.667362+0000","last_became_active":"2024-05-26T22:48:12.065934+0000","last_became_peered":"2024-05-26T22:48:12.065934+0000","last_unstale":"2024-05-26T22:51:39.667362+0000","last_undegraded":"2024-05-26T22:51:39.667362+0000","last_fullsized":"2024-05-26T22:51:39.667362+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:10.628122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:10.628122+0000","last_clean_scrub_stamp":"2024-05-26T22:48:10.628122+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:14:58.895911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574893,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32436,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339404,"statfs":{"total":96636764160,"available":96603549696,"internally_reserved":0,"allocated":1277952,"data_stored":877885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574893,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31852,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339988,"statfs":{"total":96636764160,"available":96604147712,"internally_reserved":0,"allocated":684032,"data_stored":287517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574894,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31860,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339980,"statfs":{"total":96636764160,"available":96604139520,"internally_reserved":0,"allocated":692224,"data_stored":288169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574893,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31852,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339988,"statfs":{"total":96636764160,"available":96604147712,"internally_reserved":0,"allocated":684032,"data_stored":287517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574893,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31840,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340000,"statfs":{"total":96636764160,"available":96604160000,"internally_reserved":0,"allocated":671744,"data_stored":280742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574893,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31852,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339988,"statfs":{"total":96636764160,"available":96604147712,"internally_reserved":0,"allocated":684032,"data_stored":287517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574893,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31856,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339984,"statfs":{"total":96636764160,"available":96604143616,"internally_reserved":0,"allocated":688128,"data_stored":287607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574893,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32444,"kb_used_data":1264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339396,"statfs":{"total":96636764160,"available":96603541504,"internally_reserved":0,"allocated":1294336,"data_stored":880079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-26T22:51:44.062 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-26T22:51:44.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.303+0000 7fc54d600640 1 Processor -- start 2024-05-26T22:51:44.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.304+0000 7fc54d600640 1 -- start start 2024-05-26T22:51:44.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.305+0000 7fc54d600640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fc54806ee20 0x7fc5480b3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:44.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.305+0000 7fc54d600640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc5481569d0 0x7fc5480af790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:44.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.305+0000 7fc54d600640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc54815cb20 con 0x7fc5480ad250 2024-05-26T22:51:44.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.305+0000 7fc54d600640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fc54815ce00 con 0x7fc54806ee20 2024-05-26T22:51:44.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.305+0000 7fc54d600640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc54815d0e0 con 0x7fc5481569d0 2024-05-26T22:51:44.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.305+0000 7fc546000640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fc5480ad250 0x7fc5481562c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:37864/0 (socket says 172.21.15.131:37864) 2024-05-26T22:51:44.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.305+0000 7fc546000640 1 -- 172.21.15.131:0/2394058676 learned_addr learned my addr 172.21.15.131:0/2394058676 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:51:44.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.305+0000 7fc546a00640 1 --2- 172.21.15.131:0/2394058676 >> v2:172.21.15.131:3300/0 conn(0x7fc54806ee20 0x7fc5480b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:44.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.306+0000 7fc546a00640 1 -- 172.21.15.131:0/2394058676 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc5481569d0 msgr2=0x7fc5480af790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:44.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.306+0000 7fc547400640 1 --2- 172.21.15.131:0/2394058676 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc5481569d0 0x7fc5480af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:44.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.306+0000 7fc545600640 1 -- 172.21.15.131:0/2394058676 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 619307589 0 0) 0x7fc54815cb20 con 0x7fc5480ad250 2024-05-26T22:51:44.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.306+0000 7fc546a00640 1 --2- 172.21.15.131:0/2394058676 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc5481569d0 0x7fc5480af790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:44.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.306+0000 7fc546a00640 1 -- 172.21.15.131:0/2394058676 >> v1:172.21.15.131:6789/0 conn(0x7fc5480ad250 legacy=0x7fc5481562c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:44.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.306+0000 7fc546a00640 1 -- 172.21.15.131:0/2394058676 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc54815d3c0 con 0x7fc54806ee20 2024-05-26T22:51:44.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.306+0000 7fc547400640 1 --2- 172.21.15.131:0/2394058676 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc5481569d0 0x7fc5480af790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:51:44.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.307+0000 7fc546a00640 1 --2- 172.21.15.131:0/2394058676 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc54806ee20 0x7fc5480b3190 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fc5380097a0 tx=0x7fc53802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f408ea8a1a9c3658 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:44.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.307+0000 7fc545600640 1 -- 172.21.15.131:0/2394058676 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc538037070 con 0x7fc54806ee20 2024-05-26T22:51:44.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.308+0000 7fc545600640 1 -- 172.21.15.131:0/2394058676 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc538032440 con 0x7fc54806ee20 2024-05-26T22:51:44.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.308+0000 7fc545600640 1 -- 172.21.15.131:0/2394058676 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc538032810 con 0x7fc54806ee20 2024-05-26T22:51:44.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.308+0000 7fc54d600640 1 -- 172.21.15.131:0/2394058676 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc54806ee20 msgr2=0x7fc5480b3190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:44.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.308+0000 7fc54d600640 1 --2- 172.21.15.131:0/2394058676 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc54806ee20 0x7fc5480b3190 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fc5380097a0 tx=0x7fc53802a180 comp rx=0 tx=0).stop 2024-05-26T22:51:44.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.308+0000 7fc54d600640 1 -- 172.21.15.131:0/2394058676 shutdown_connections 2024-05-26T22:51:44.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.308+0000 7fc54d600640 1 --2- 172.21.15.131:0/2394058676 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc5481569d0 0x7fc5480af790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:44.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.308+0000 7fc54d600640 1 --2- 172.21.15.131:0/2394058676 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc54806ee20 0x7fc5480b3190 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:44.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.308+0000 7fc54d600640 1 -- 172.21.15.131:0/2394058676 >> 172.21.15.131:0/2394058676 conn(0x7fc5480b1e90 msgr2=0x7fc5480b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:44.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.308+0000 7fc54d600640 1 -- 172.21.15.131:0/2394058676 shutdown_connections 2024-05-26T22:51:44.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.308+0000 7fc54d600640 1 -- 172.21.15.131:0/2394058676 wait complete. 2024-05-26T22:51:44.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.310+0000 7fc54d600640 1 Processor -- start 2024-05-26T22:51:44.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.310+0000 7fc54d600640 1 -- start start 2024-05-26T22:51:44.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.311+0000 7fc54d600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc54806ee20 0x7fc548154590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:44.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.311+0000 7fc54d600640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc5480ad250 0x7fc5481f3b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:44.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.311+0000 7fc54d600640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc5481569d0 0x7fc548151340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:44.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.311+0000 7fc54d600640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc54815c900 con 0x7fc54806ee20 2024-05-26T22:51:44.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.311+0000 7fc54d600640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc54815c780 con 0x7fc5481569d0 2024-05-26T22:51:44.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.311+0000 7fc54d600640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc54806f200 con 0x7fc5480ad250 2024-05-26T22:51:44.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.311+0000 7fc546a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc54806ee20 0x7fc548154590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:44.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.311+0000 7fc546a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc54806ee20 0x7fc548154590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47718/0 (socket says 172.21.15.131:47718) 2024-05-26T22:51:44.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.311+0000 7fc546a00640 1 -- 172.21.15.131:0/1405141838 learned_addr learned my addr 172.21.15.131:0/1405141838 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:44.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.312+0000 7fc546a00640 1 -- 172.21.15.131:0/1405141838 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc5480ad250 msgr2=0x7fc5481f3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:44.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.312+0000 7fc546a00640 1 --2- 172.21.15.131:0/1405141838 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc5480ad250 0x7fc5481f3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:44.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.312+0000 7fc546a00640 1 -- 172.21.15.131:0/1405141838 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc5481569d0 msgr2=0x7fc548151340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:44.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.312+0000 7fc546a00640 1 --2- 172.21.15.131:0/1405141838 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc5481569d0 0x7fc548151340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:44.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.312+0000 7fc546a00640 1 -- 172.21.15.131:0/1405141838 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc548154ad0 con 0x7fc54806ee20 2024-05-26T22:51:44.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.312+0000 7fc546a00640 1 --2- 172.21.15.131:0/1405141838 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc54806ee20 0x7fc548154590 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fc538000c00 tx=0x7fc5380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:44.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.312+0000 7fc53ea00640 1 -- 172.21.15.131:0/1405141838 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc538037040 con 0x7fc54806ee20 2024-05-26T22:51:44.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.312+0000 7fc53ea00640 1 -- 172.21.15.131:0/1405141838 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc538042070 con 0x7fc54806ee20 2024-05-26T22:51:44.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.312+0000 7fc54d600640 1 -- 172.21.15.131:0/1405141838 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5481534f0 con 0x7fc54806ee20 2024-05-26T22:51:44.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.312+0000 7fc53ea00640 1 -- 172.21.15.131:0/1405141838 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc538030a70 con 0x7fc54806ee20 2024-05-26T22:51:44.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.313+0000 7fc54d600640 1 -- 172.21.15.131:0/1405141838 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc548153ab0 con 0x7fc54806ee20 2024-05-26T22:51:44.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.314+0000 7fc54d600640 1 -- 172.21.15.131:0/1405141838 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5480ae2d0 con 0x7fc54806ee20 2024-05-26T22:51:44.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.315+0000 7fc53ea00640 1 -- 172.21.15.131:0/1405141838 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc538030c10 con 0x7fc54806ee20 2024-05-26T22:51:44.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.315+0000 7fc53ea00640 1 --2- 172.21.15.131:0/1405141838 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc52406be30 0x7fc52406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:44.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.316+0000 7fc53ea00640 1 -- 172.21.15.131:0/1405141838 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fc5380aba50 con 0x7fc54806ee20 2024-05-26T22:51:44.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.316+0000 7fc546000640 1 --2- 172.21.15.131:0/1405141838 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc52406be30 0x7fc52406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:44.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.316+0000 7fc546000640 1 --2- 172.21.15.131:0/1405141838 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc52406be30 0x7fc52406e2f0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fc5300011e0 tx=0x7fc530007920 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:44.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.320+0000 7fc53ea00640 1 -- 172.21.15.131:0/1405141838 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fc5480ae2d0 con 0x7fc54806ee20 2024-05-26T22:51:44.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.510+0000 7fc54d600640 1 -- 172.21.15.131:0/1405141838 --> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc5480694e0 con 0x7fc52406be30 2024-05-26T22:51:44.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.528+0000 7fc53ea00640 1 -- 172.21.15.131:0/1405141838 <== mgr.4104 v2:172.21.15.131:6832/3484832947 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251887 (secure 0 0 0) 0x7fc530053800 con 0x7fc52406be30 2024-05-26T22:51:44.531 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:51:44.532 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-05-26T22:51:44.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.532+0000 7fc54d600640 1 -- 172.21.15.131:0/1405141838 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc52406be30 msgr2=0x7fc52406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:44.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.532+0000 7fc54d600640 1 --2- 172.21.15.131:0/1405141838 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc52406be30 0x7fc52406e2f0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fc5300011e0 tx=0x7fc530007920 comp rx=0 tx=0).stop 2024-05-26T22:51:44.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.532+0000 7fc54d600640 1 -- 172.21.15.131:0/1405141838 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc54806ee20 msgr2=0x7fc548154590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:44.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.532+0000 7fc54d600640 1 --2- 172.21.15.131:0/1405141838 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc54806ee20 0x7fc548154590 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fc538000c00 tx=0x7fc5380046c0 comp rx=0 tx=0).stop 2024-05-26T22:51:44.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.533+0000 7fc54d600640 1 -- 172.21.15.131:0/1405141838 shutdown_connections 2024-05-26T22:51:44.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.533+0000 7fc54d600640 1 --2- 172.21.15.131:0/1405141838 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc5480ad250 0x7fc5481f3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:44.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.533+0000 7fc54d600640 1 --2- 172.21.15.131:0/1405141838 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc52406be30 0x7fc52406e2f0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:44.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.533+0000 7fc54d600640 1 --2- 172.21.15.131:0/1405141838 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc5481569d0 0x7fc548151340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:44.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.533+0000 7fc54d600640 1 --2- 172.21.15.131:0/1405141838 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc54806ee20 0x7fc548154590 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:44.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.533+0000 7fc54d600640 1 -- 172.21.15.131:0/1405141838 >> 172.21.15.131:0/1405141838 conn(0x7fc5480b1e90 msgr2=0x7fc5480b2480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:44.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.533+0000 7fc54d600640 1 -- 172.21.15.131:0/1405141838 shutdown_connections 2024-05-26T22:51:44.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.533+0000 7fc54d600640 1 -- 172.21.15.131:0/1405141838 wait complete. 2024-05-26T22:51:44.550 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":158,"stamp":"2024-05-26T22:51:44.451221+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":124,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":255992,"kb_used_data":6520,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754718728,"statfs":{"total":773094113280,"available":772831977472,"internally_reserved":0,"allocated":6676480,"data_stored":3477033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":38,"apply_latency_ms":38,"commit_latency_ns":38000000,"apply_latency_ns":38000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001766"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.688105+0000","last_change":"2024-05-26T22:51:39.681899+0000","last_active":"2024-05-26T22:51:39.688105+0000","last_peered":"2024-05-26T22:51:39.688105+0000","last_clean":"2024-05-26T22:51:39.688105+0000","last_became_active":"2024-05-26T22:51:39.681619+0000","last_became_peered":"2024-05-26T22:51:39.681619+0000","last_unstale":"2024-05-26T22:51:39.688105+0000","last_undegraded":"2024-05-26T22:51:39.688105+0000","last_fullsized":"2024-05-26T22:51:39.688105+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:30:12.496898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.699114+0000","last_change":"2024-05-26T22:51:39.686789+0000","last_active":"2024-05-26T22:51:39.699114+0000","last_peered":"2024-05-26T22:51:39.699114+0000","last_clean":"2024-05-26T22:51:39.699114+0000","last_became_active":"2024-05-26T22:51:39.686599+0000","last_became_peered":"2024-05-26T22:51:39.686599+0000","last_unstale":"2024-05-26T22:51:39.699114+0000","last_undegraded":"2024-05-26T22:51:39.699114+0000","last_fullsized":"2024-05-26T22:51:39.699114+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:47:57.346804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.693955+0000","last_change":"2024-05-26T22:51:39.686405+0000","last_active":"2024-05-26T22:51:39.693955+0000","last_peered":"2024-05-26T22:51:39.693955+0000","last_clean":"2024-05-26T22:51:39.693955+0000","last_became_active":"2024-05-26T22:51:39.685726+0000","last_became_peered":"2024-05-26T22:51:39.685726+0000","last_unstale":"2024-05-26T22:51:39.693955+0000","last_undegraded":"2024-05-26T22:51:39.693955+0000","last_fullsized":"2024-05-26T22:51:39.693955+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:31:36.673510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.682768+0000","last_change":"2024-05-26T22:51:39.679886+0000","last_active":"2024-05-26T22:51:39.682768+0000","last_peered":"2024-05-26T22:51:39.682768+0000","last_clean":"2024-05-26T22:51:39.682768+0000","last_became_active":"2024-05-26T22:51:39.679621+0000","last_became_peered":"2024-05-26T22:51:39.679621+0000","last_unstale":"2024-05-26T22:51:39.682768+0000","last_undegraded":"2024-05-26T22:51:39.682768+0000","last_fullsized":"2024-05-26T22:51:39.682768+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:50:28.022562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.685516+0000","last_change":"2024-05-26T22:51:39.676186+0000","last_active":"2024-05-26T22:51:39.685516+0000","last_peered":"2024-05-26T22:51:39.685516+0000","last_clean":"2024-05-26T22:51:39.685516+0000","last_became_active":"2024-05-26T22:51:39.675391+0000","last_became_peered":"2024-05-26T22:51:39.675391+0000","last_unstale":"2024-05-26T22:51:39.685516+0000","last_undegraded":"2024-05-26T22:51:39.685516+0000","last_fullsized":"2024-05-26T22:51:39.685516+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:49:52.321601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.684636+0000","last_change":"2024-05-26T22:51:39.679359+0000","last_active":"2024-05-26T22:51:39.684636+0000","last_peered":"2024-05-26T22:51:39.684636+0000","last_clean":"2024-05-26T22:51:39.684636+0000","last_became_active":"2024-05-26T22:51:39.679082+0000","last_became_peered":"2024-05-26T22:51:39.679082+0000","last_unstale":"2024-05-26T22:51:39.684636+0000","last_undegraded":"2024-05-26T22:51:39.684636+0000","last_fullsized":"2024-05-26T22:51:39.684636+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:53:31.658157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.686989+0000","last_change":"2024-05-26T22:51:39.682487+0000","last_active":"2024-05-26T22:51:39.686989+0000","last_peered":"2024-05-26T22:51:39.686989+0000","last_clean":"2024-05-26T22:51:39.686989+0000","last_became_active":"2024-05-26T22:51:39.681735+0000","last_became_peered":"2024-05-26T22:51:39.681735+0000","last_unstale":"2024-05-26T22:51:39.686989+0000","last_undegraded":"2024-05-26T22:51:39.686989+0000","last_fullsized":"2024-05-26T22:51:39.686989+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:28:52.199885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.702484+0000","last_change":"2024-05-26T22:51:39.699147+0000","last_active":"2024-05-26T22:51:39.702484+0000","last_peered":"2024-05-26T22:51:39.702484+0000","last_clean":"2024-05-26T22:51:39.702484+0000","last_became_active":"2024-05-26T22:51:39.698718+0000","last_became_peered":"2024-05-26T22:51:39.698718+0000","last_unstale":"2024-05-26T22:51:39.702484+0000","last_undegraded":"2024-05-26T22:51:39.702484+0000","last_fullsized":"2024-05-26T22:51:39.702484+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:33:27.761464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.692318+0000","last_change":"2024-05-26T22:51:39.686596+0000","last_active":"2024-05-26T22:51:39.692318+0000","last_peered":"2024-05-26T22:51:39.692318+0000","last_clean":"2024-05-26T22:51:39.692318+0000","last_became_active":"2024-05-26T22:51:39.686257+0000","last_became_peered":"2024-05-26T22:51:39.686257+0000","last_unstale":"2024-05-26T22:51:39.692318+0000","last_undegraded":"2024-05-26T22:51:39.692318+0000","last_fullsized":"2024-05-26T22:51:39.692318+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:31:09.888922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.668932+0000","last_change":"2024-05-26T22:51:39.661444+0000","last_active":"2024-05-26T22:51:39.668932+0000","last_peered":"2024-05-26T22:51:39.668932+0000","last_clean":"2024-05-26T22:51:39.668932+0000","last_became_active":"2024-05-26T22:51:39.660595+0000","last_became_peered":"2024-05-26T22:51:39.660595+0000","last_unstale":"2024-05-26T22:51:39.668932+0000","last_undegraded":"2024-05-26T22:51:39.668932+0000","last_fullsized":"2024-05-26T22:51:39.668932+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:48:04.727497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.688703+0000","last_change":"2024-05-26T22:51:39.684510+0000","last_active":"2024-05-26T22:51:39.688703+0000","last_peered":"2024-05-26T22:51:39.688703+0000","last_clean":"2024-05-26T22:51:39.688703+0000","last_became_active":"2024-05-26T22:51:39.684185+0000","last_became_peered":"2024-05-26T22:51:39.684185+0000","last_unstale":"2024-05-26T22:51:39.688703+0000","last_undegraded":"2024-05-26T22:51:39.688703+0000","last_fullsized":"2024-05-26T22:51:39.688703+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:53:24.309742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.687919+0000","last_change":"2024-05-26T22:51:39.682601+0000","last_active":"2024-05-26T22:51:39.687919+0000","last_peered":"2024-05-26T22:51:39.687919+0000","last_clean":"2024-05-26T22:51:39.687919+0000","last_became_active":"2024-05-26T22:51:39.682027+0000","last_became_peered":"2024-05-26T22:51:39.682027+0000","last_unstale":"2024-05-26T22:51:39.687919+0000","last_undegraded":"2024-05-26T22:51:39.687919+0000","last_fullsized":"2024-05-26T22:51:39.687919+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:33:52.360239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.681500+0000","last_change":"2024-05-26T22:51:39.675281+0000","last_active":"2024-05-26T22:51:39.681500+0000","last_peered":"2024-05-26T22:51:39.681500+0000","last_clean":"2024-05-26T22:51:39.681500+0000","last_became_active":"2024-05-26T22:51:39.674925+0000","last_became_peered":"2024-05-26T22:51:39.674925+0000","last_unstale":"2024-05-26T22:51:39.681500+0000","last_undegraded":"2024-05-26T22:51:39.681500+0000","last_fullsized":"2024-05-26T22:51:39.681500+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:48:04.727497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.687360+0000","last_change":"2024-05-26T22:51:39.682232+0000","last_active":"2024-05-26T22:51:39.687360+0000","last_peered":"2024-05-26T22:51:39.687360+0000","last_clean":"2024-05-26T22:51:39.687360+0000","last_became_active":"2024-05-26T22:51:39.681888+0000","last_became_peered":"2024-05-26T22:51:39.681888+0000","last_unstale":"2024-05-26T22:51:39.687360+0000","last_undegraded":"2024-05-26T22:51:39.687360+0000","last_fullsized":"2024-05-26T22:51:39.687360+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:31:58.182736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.694674+0000","last_change":"2024-05-26T22:51:39.688223+0000","last_active":"2024-05-26T22:51:39.694674+0000","last_peered":"2024-05-26T22:51:39.694674+0000","last_clean":"2024-05-26T22:51:39.694674+0000","last_became_active":"2024-05-26T22:51:39.687937+0000","last_became_peered":"2024-05-26T22:51:39.687937+0000","last_unstale":"2024-05-26T22:51:39.694674+0000","last_undegraded":"2024-05-26T22:51:39.694674+0000","last_fullsized":"2024-05-26T22:51:39.694674+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:28:58.750510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.692586+0000","last_change":"2024-05-26T22:51:39.685935+0000","last_active":"2024-05-26T22:51:39.692586+0000","last_peered":"2024-05-26T22:51:39.692586+0000","last_clean":"2024-05-26T22:51:39.692586+0000","last_became_active":"2024-05-26T22:51:39.685672+0000","last_became_peered":"2024-05-26T22:51:39.685672+0000","last_unstale":"2024-05-26T22:51:39.692586+0000","last_undegraded":"2024-05-26T22:51:39.692586+0000","last_fullsized":"2024-05-26T22:51:39.692586+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:31:53.601456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.696416+0000","last_change":"2024-05-26T22:51:39.683113+0000","last_active":"2024-05-26T22:51:39.696416+0000","last_peered":"2024-05-26T22:51:39.696416+0000","last_clean":"2024-05-26T22:51:39.696416+0000","last_became_active":"2024-05-26T22:51:39.682798+0000","last_became_peered":"2024-05-26T22:51:39.682798+0000","last_unstale":"2024-05-26T22:51:39.696416+0000","last_undegraded":"2024-05-26T22:51:39.696416+0000","last_fullsized":"2024-05-26T22:51:39.696416+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:27:47.119685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.697355+0000","last_change":"2024-05-26T22:51:39.693360+0000","last_active":"2024-05-26T22:51:39.697355+0000","last_peered":"2024-05-26T22:51:39.697355+0000","last_clean":"2024-05-26T22:51:39.697355+0000","last_became_active":"2024-05-26T22:51:39.693037+0000","last_became_peered":"2024-05-26T22:51:39.693037+0000","last_unstale":"2024-05-26T22:51:39.697355+0000","last_undegraded":"2024-05-26T22:51:39.697355+0000","last_fullsized":"2024-05-26T22:51:39.697355+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T23:25:48.753320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.690297+0000","last_change":"2024-05-26T22:51:39.683068+0000","last_active":"2024-05-26T22:51:39.690297+0000","last_peered":"2024-05-26T22:51:39.690297+0000","last_clean":"2024-05-26T22:51:39.690297+0000","last_became_active":"2024-05-26T22:51:39.682205+0000","last_became_peered":"2024-05-26T22:51:39.682205+0000","last_unstale":"2024-05-26T22:51:39.690297+0000","last_undegraded":"2024-05-26T22:51:39.690297+0000","last_fullsized":"2024-05-26T22:51:39.690297+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:29:54.618801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.681593+0000","last_change":"2024-05-26T22:51:39.678024+0000","last_active":"2024-05-26T22:51:39.681593+0000","last_peered":"2024-05-26T22:51:39.681593+0000","last_clean":"2024-05-26T22:51:39.681593+0000","last_became_active":"2024-05-26T22:51:39.677715+0000","last_became_peered":"2024-05-26T22:51:39.677715+0000","last_unstale":"2024-05-26T22:51:39.681593+0000","last_undegraded":"2024-05-26T22:51:39.681593+0000","last_fullsized":"2024-05-26T22:51:39.681593+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:56:06.462281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.691858+0000","last_change":"2024-05-26T22:51:39.683919+0000","last_active":"2024-05-26T22:51:39.691858+0000","last_peered":"2024-05-26T22:51:39.691858+0000","last_clean":"2024-05-26T22:51:39.691858+0000","last_became_active":"2024-05-26T22:51:39.683658+0000","last_became_peered":"2024-05-26T22:51:39.683658+0000","last_unstale":"2024-05-26T22:51:39.691858+0000","last_undegraded":"2024-05-26T22:51:39.691858+0000","last_fullsized":"2024-05-26T22:51:39.691858+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:32:59.337859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.696470+0000","last_change":"2024-05-26T22:51:39.680787+0000","last_active":"2024-05-26T22:51:39.696470+0000","last_peered":"2024-05-26T22:51:39.696470+0000","last_clean":"2024-05-26T22:51:39.696470+0000","last_became_active":"2024-05-26T22:51:39.680463+0000","last_became_peered":"2024-05-26T22:51:39.680463+0000","last_unstale":"2024-05-26T22:51:39.696470+0000","last_undegraded":"2024-05-26T22:51:39.696470+0000","last_fullsized":"2024-05-26T22:51:39.696470+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:42:24.864056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.685953+0000","last_change":"2024-05-26T22:51:39.678261+0000","last_active":"2024-05-26T22:51:39.685953+0000","last_peered":"2024-05-26T22:51:39.685953+0000","last_clean":"2024-05-26T22:51:39.685953+0000","last_became_active":"2024-05-26T22:51:39.677799+0000","last_became_peered":"2024-05-26T22:51:39.677799+0000","last_unstale":"2024-05-26T22:51:39.685953+0000","last_undegraded":"2024-05-26T22:51:39.685953+0000","last_fullsized":"2024-05-26T22:51:39.685953+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:50:37.362604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.680921+0000","last_change":"2024-05-26T22:51:39.676125+0000","last_active":"2024-05-26T22:51:39.680921+0000","last_peered":"2024-05-26T22:51:39.680921+0000","last_clean":"2024-05-26T22:51:39.680921+0000","last_became_active":"2024-05-26T22:51:39.675652+0000","last_became_peered":"2024-05-26T22:51:39.675652+0000","last_unstale":"2024-05-26T22:51:39.680921+0000","last_undegraded":"2024-05-26T22:51:39.680921+0000","last_fullsized":"2024-05-26T22:51:39.680921+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:55:10.612046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.688081+0000","last_change":"2024-05-26T22:51:39.677788+0000","last_active":"2024-05-26T22:51:39.688081+0000","last_peered":"2024-05-26T22:51:39.688081+0000","last_clean":"2024-05-26T22:51:39.688081+0000","last_became_active":"2024-05-26T22:51:39.677212+0000","last_became_peered":"2024-05-26T22:51:39.677212+0000","last_unstale":"2024-05-26T22:51:39.688081+0000","last_undegraded":"2024-05-26T22:51:39.688081+0000","last_fullsized":"2024-05-26T22:51:39.688081+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:53:40.267842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.687416+0000","last_change":"2024-05-26T22:51:39.682252+0000","last_active":"2024-05-26T22:51:39.687416+0000","last_peered":"2024-05-26T22:51:39.687416+0000","last_clean":"2024-05-26T22:51:39.687416+0000","last_became_active":"2024-05-26T22:51:39.681898+0000","last_became_peered":"2024-05-26T22:51:39.681898+0000","last_unstale":"2024-05-26T22:51:39.687416+0000","last_undegraded":"2024-05-26T22:51:39.687416+0000","last_fullsized":"2024-05-26T22:51:39.687416+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:08:00.991256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.692076+0000","last_change":"2024-05-26T22:51:39.685562+0000","last_active":"2024-05-26T22:51:39.692076+0000","last_peered":"2024-05-26T22:51:39.692076+0000","last_clean":"2024-05-26T22:51:39.692076+0000","last_became_active":"2024-05-26T22:51:39.685285+0000","last_became_peered":"2024-05-26T22:51:39.685285+0000","last_unstale":"2024-05-26T22:51:39.692076+0000","last_undegraded":"2024-05-26T22:51:39.692076+0000","last_fullsized":"2024-05-26T22:51:39.692076+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:52:34.474139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.687893+0000","last_change":"2024-05-26T22:51:39.675557+0000","last_active":"2024-05-26T22:51:39.687893+0000","last_peered":"2024-05-26T22:51:39.687893+0000","last_clean":"2024-05-26T22:51:39.687893+0000","last_became_active":"2024-05-26T22:51:39.675258+0000","last_became_peered":"2024-05-26T22:51:39.675258+0000","last_unstale":"2024-05-26T22:51:39.687893+0000","last_undegraded":"2024-05-26T22:51:39.687893+0000","last_fullsized":"2024-05-26T22:51:39.687893+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:15:22.527455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.702776+0000","last_change":"2024-05-26T22:51:39.695156+0000","last_active":"2024-05-26T22:51:39.702776+0000","last_peered":"2024-05-26T22:51:39.702776+0000","last_clean":"2024-05-26T22:51:39.702776+0000","last_became_active":"2024-05-26T22:51:39.689092+0000","last_became_peered":"2024-05-26T22:51:39.689092+0000","last_unstale":"2024-05-26T22:51:39.702776+0000","last_undegraded":"2024-05-26T22:51:39.702776+0000","last_fullsized":"2024-05-26T22:51:39.702776+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:44:31.960643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.693819+0000","last_change":"2024-05-26T22:51:39.689894+0000","last_active":"2024-05-26T22:51:39.693819+0000","last_peered":"2024-05-26T22:51:39.693819+0000","last_clean":"2024-05-26T22:51:39.693819+0000","last_became_active":"2024-05-26T22:51:39.689200+0000","last_became_peered":"2024-05-26T22:51:39.689200+0000","last_unstale":"2024-05-26T22:51:39.693819+0000","last_undegraded":"2024-05-26T22:51:39.693819+0000","last_fullsized":"2024-05-26T22:51:39.693819+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:33:52.360239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.688141+0000","last_change":"2024-05-26T22:51:39.678227+0000","last_active":"2024-05-26T22:51:39.688141+0000","last_peered":"2024-05-26T22:51:39.688141+0000","last_clean":"2024-05-26T22:51:39.688141+0000","last_became_active":"2024-05-26T22:51:39.674765+0000","last_became_peered":"2024-05-26T22:51:39.674765+0000","last_unstale":"2024-05-26T22:51:39.688141+0000","last_undegraded":"2024-05-26T22:51:39.688141+0000","last_fullsized":"2024-05-26T22:51:39.688141+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:07:20.949312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.677932+0000","last_change":"2024-05-26T22:51:39.672133+0000","last_active":"2024-05-26T22:51:39.677932+0000","last_peered":"2024-05-26T22:51:39.677932+0000","last_clean":"2024-05-26T22:51:39.677932+0000","last_became_active":"2024-05-26T22:51:39.671831+0000","last_became_peered":"2024-05-26T22:51:39.671831+0000","last_unstale":"2024-05-26T22:51:39.677932+0000","last_undegraded":"2024-05-26T22:51:39.677932+0000","last_fullsized":"2024-05-26T22:51:39.677932+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:50:28.022562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.677956+0000","last_change":"2024-05-26T22:51:39.672094+0000","last_active":"2024-05-26T22:51:39.677956+0000","last_peered":"2024-05-26T22:51:39.677956+0000","last_clean":"2024-05-26T22:51:39.677956+0000","last_became_active":"2024-05-26T22:51:39.671686+0000","last_became_peered":"2024-05-26T22:51:39.671686+0000","last_unstale":"2024-05-26T22:51:39.677956+0000","last_undegraded":"2024-05-26T22:51:39.677956+0000","last_fullsized":"2024-05-26T22:51:39.677956+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:19:31.293296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"57'4","reported_seq":16,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.705509+0000","last_change":"2024-05-26T22:51:38.676687+0000","last_active":"2024-05-26T22:51:39.705509+0000","last_peered":"2024-05-26T22:51:39.705509+0000","last_clean":"2024-05-26T22:51:39.705509+0000","last_became_active":"2024-05-26T22:51:38.676052+0000","last_became_peered":"2024-05-26T22:51:38.676052+0000","last_unstale":"2024-05-26T22:51:39.705509+0000","last_undegraded":"2024-05-26T22:51:39.705509+0000","last_fullsized":"2024-05-26T22:51:39.705509+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_clean_scrub_stamp":"2024-05-26T22:51:37.608737+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:54:07.054820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.682019+0000","last_change":"2024-05-26T22:51:39.670939+0000","last_active":"2024-05-26T22:51:39.682019+0000","last_peered":"2024-05-26T22:51:39.682019+0000","last_clean":"2024-05-26T22:51:39.682019+0000","last_became_active":"2024-05-26T22:51:39.670665+0000","last_became_peered":"2024-05-26T22:51:39.670665+0000","last_unstale":"2024-05-26T22:51:39.682019+0000","last_undegraded":"2024-05-26T22:51:39.682019+0000","last_fullsized":"2024-05-26T22:51:39.682019+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:55:10.612046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"57'1","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.704272+0000","last_change":"2024-05-26T22:51:38.676783+0000","last_active":"2024-05-26T22:51:39.704272+0000","last_peered":"2024-05-26T22:51:39.704272+0000","last_clean":"2024-05-26T22:51:39.704272+0000","last_became_active":"2024-05-26T22:51:38.676466+0000","last_became_peered":"2024-05-26T22:51:38.676466+0000","last_unstale":"2024-05-26T22:51:39.704272+0000","last_undegraded":"2024-05-26T22:51:39.704272+0000","last_fullsized":"2024-05-26T22:51:39.704272+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_clean_scrub_stamp":"2024-05-26T22:51:37.608737+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:31:52.586288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.691914+0000","last_change":"2024-05-26T22:51:39.686450+0000","last_active":"2024-05-26T22:51:39.691914+0000","last_peered":"2024-05-26T22:51:39.691914+0000","last_clean":"2024-05-26T22:51:39.691914+0000","last_became_active":"2024-05-26T22:51:39.681303+0000","last_became_peered":"2024-05-26T22:51:39.681303+0000","last_unstale":"2024-05-26T22:51:39.691914+0000","last_undegraded":"2024-05-26T22:51:39.691914+0000","last_fullsized":"2024-05-26T22:51:39.691914+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:56:06.462281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"57'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.702326+0000","last_change":"2024-05-26T22:51:38.672736+0000","last_active":"2024-05-26T22:51:39.702326+0000","last_peered":"2024-05-26T22:51:39.702326+0000","last_clean":"2024-05-26T22:51:39.702326+0000","last_became_active":"2024-05-26T22:51:38.672443+0000","last_became_peered":"2024-05-26T22:51:38.672443+0000","last_unstale":"2024-05-26T22:51:39.702326+0000","last_undegraded":"2024-05-26T22:51:39.702326+0000","last_fullsized":"2024-05-26T22:51:39.702326+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_clean_scrub_stamp":"2024-05-26T22:51:37.608737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:31:35.658342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.692676+0000","last_change":"2024-05-26T22:51:39.686321+0000","last_active":"2024-05-26T22:51:39.692676+0000","last_peered":"2024-05-26T22:51:39.692676+0000","last_clean":"2024-05-26T22:51:39.692676+0000","last_became_active":"2024-05-26T22:51:39.684865+0000","last_became_peered":"2024-05-26T22:51:39.684865+0000","last_unstale":"2024-05-26T22:51:39.692676+0000","last_undegraded":"2024-05-26T22:51:39.692676+0000","last_fullsized":"2024-05-26T22:51:39.692676+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:22:14.856913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"57'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.696476+0000","last_change":"2024-05-26T22:51:38.680442+0000","last_active":"2024-05-26T22:51:39.696476+0000","last_peered":"2024-05-26T22:51:39.696476+0000","last_clean":"2024-05-26T22:51:39.696476+0000","last_became_active":"2024-05-26T22:51:38.680133+0000","last_became_peered":"2024-05-26T22:51:38.680133+0000","last_unstale":"2024-05-26T22:51:39.696476+0000","last_undegraded":"2024-05-26T22:51:39.696476+0000","last_fullsized":"2024-05-26T22:51:39.696476+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_clean_scrub_stamp":"2024-05-26T22:51:37.608737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:50:36.347437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.701171+0000","last_change":"2024-05-26T22:51:39.695805+0000","last_active":"2024-05-26T22:51:39.701171+0000","last_peered":"2024-05-26T22:51:39.701171+0000","last_clean":"2024-05-26T22:51:39.701171+0000","last_became_active":"2024-05-26T22:51:39.695515+0000","last_became_peered":"2024-05-26T22:51:39.695515+0000","last_unstale":"2024-05-26T22:51:39.701171+0000","last_undegraded":"2024-05-26T22:51:39.701171+0000","last_fullsized":"2024-05-26T22:51:39.701171+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:25:17.473409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"57'1","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.696821+0000","last_change":"2024-05-26T22:51:38.668602+0000","last_active":"2024-05-26T22:51:39.696821+0000","last_peered":"2024-05-26T22:51:39.696821+0000","last_clean":"2024-05-26T22:51:39.696821+0000","last_became_active":"2024-05-26T22:51:38.668096+0000","last_became_peered":"2024-05-26T22:51:38.668096+0000","last_unstale":"2024-05-26T22:51:39.696821+0000","last_undegraded":"2024-05-26T22:51:39.696821+0000","last_fullsized":"2024-05-26T22:51:39.696821+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_clean_scrub_stamp":"2024-05-26T22:51:37.608737+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:27:29.941288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.702754+0000","last_change":"2024-05-26T22:51:39.694227+0000","last_active":"2024-05-26T22:51:39.702754+0000","last_peered":"2024-05-26T22:51:39.702754+0000","last_clean":"2024-05-26T22:51:39.702754+0000","last_became_active":"2024-05-26T22:51:39.693893+0000","last_became_peered":"2024-05-26T22:51:39.693893+0000","last_unstale":"2024-05-26T22:51:39.702754+0000","last_undegraded":"2024-05-26T22:51:39.702754+0000","last_fullsized":"2024-05-26T22:51:39.702754+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:08:25.990269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"57'1","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.698841+0000","last_change":"2024-05-26T22:51:38.671258+0000","last_active":"2024-05-26T22:51:39.698841+0000","last_peered":"2024-05-26T22:51:39.698841+0000","last_clean":"2024-05-26T22:51:39.698841+0000","last_became_active":"2024-05-26T22:51:38.670971+0000","last_became_peered":"2024-05-26T22:51:38.670971+0000","last_unstale":"2024-05-26T22:51:39.698841+0000","last_undegraded":"2024-05-26T22:51:39.698841+0000","last_fullsized":"2024-05-26T22:51:39.698841+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_clean_scrub_stamp":"2024-05-26T22:51:37.608737+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:50:36.347437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.680892+0000","last_change":"2024-05-26T22:51:39.674354+0000","last_active":"2024-05-26T22:51:39.680892+0000","last_peered":"2024-05-26T22:51:39.680892+0000","last_clean":"2024-05-26T22:51:39.680892+0000","last_became_active":"2024-05-26T22:51:39.674004+0000","last_became_peered":"2024-05-26T22:51:39.674004+0000","last_unstale":"2024-05-26T22:51:39.680892+0000","last_undegraded":"2024-05-26T22:51:39.680892+0000","last_fullsized":"2024-05-26T22:51:39.680892+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T23:30:00.567203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.662472+0000","last_change":"2024-05-26T22:51:38.676905+0000","last_active":"2024-05-26T22:51:39.662472+0000","last_peered":"2024-05-26T22:51:39.662472+0000","last_clean":"2024-05-26T22:51:39.662472+0000","last_became_active":"2024-05-26T22:51:38.676347+0000","last_became_peered":"2024-05-26T22:51:38.676347+0000","last_unstale":"2024-05-26T22:51:39.662472+0000","last_undegraded":"2024-05-26T22:51:39.662472+0000","last_fullsized":"2024-05-26T22:51:39.662472+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_clean_scrub_stamp":"2024-05-26T22:51:37.608737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:56:05.395225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.693021+0000","last_change":"2024-05-26T22:51:39.684604+0000","last_active":"2024-05-26T22:51:39.693021+0000","last_peered":"2024-05-26T22:51:39.693021+0000","last_clean":"2024-05-26T22:51:39.693021+0000","last_became_active":"2024-05-26T22:51:39.682802+0000","last_became_peered":"2024-05-26T22:51:39.682802+0000","last_unstale":"2024-05-26T22:51:39.693021+0000","last_undegraded":"2024-05-26T22:51:39.693021+0000","last_fullsized":"2024-05-26T22:51:39.693021+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:18:17.196608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.669722+0000","last_change":"2024-05-26T22:51:38.675793+0000","last_active":"2024-05-26T22:51:39.669722+0000","last_peered":"2024-05-26T22:51:39.669722+0000","last_clean":"2024-05-26T22:51:39.669722+0000","last_became_active":"2024-05-26T22:51:38.674745+0000","last_became_peered":"2024-05-26T22:51:38.674745+0000","last_unstale":"2024-05-26T22:51:39.669722+0000","last_undegraded":"2024-05-26T22:51:39.669722+0000","last_fullsized":"2024-05-26T22:51:39.669722+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_clean_scrub_stamp":"2024-05-26T22:51:37.608737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:11:47.615881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.687141+0000","last_change":"2024-05-26T22:51:39.683593+0000","last_active":"2024-05-26T22:51:39.687141+0000","last_peered":"2024-05-26T22:51:39.687141+0000","last_clean":"2024-05-26T22:51:39.687141+0000","last_became_active":"2024-05-26T22:51:39.681458+0000","last_became_peered":"2024-05-26T22:51:39.681458+0000","last_unstale":"2024-05-26T22:51:39.687141+0000","last_undegraded":"2024-05-26T22:51:39.687141+0000","last_fullsized":"2024-05-26T22:51:39.687141+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:21:52.907223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.677231+0000","last_change":"2024-05-26T22:51:38.678768+0000","last_active":"2024-05-26T22:51:39.677231+0000","last_peered":"2024-05-26T22:51:39.677231+0000","last_clean":"2024-05-26T22:51:39.677231+0000","last_became_active":"2024-05-26T22:51:38.678131+0000","last_became_peered":"2024-05-26T22:51:38.678131+0000","last_unstale":"2024-05-26T22:51:39.677231+0000","last_undegraded":"2024-05-26T22:51:39.677231+0000","last_fullsized":"2024-05-26T22:51:39.677231+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_clean_scrub_stamp":"2024-05-26T22:51:37.608737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:10:38.410360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.693024+0000","last_change":"2024-05-26T22:51:39.685794+0000","last_active":"2024-05-26T22:51:39.693024+0000","last_peered":"2024-05-26T22:51:39.693024+0000","last_clean":"2024-05-26T22:51:39.693024+0000","last_became_active":"2024-05-26T22:51:39.685510+0000","last_became_peered":"2024-05-26T22:51:39.685510+0000","last_unstale":"2024-05-26T22:51:39.693024+0000","last_undegraded":"2024-05-26T22:51:39.693024+0000","last_fullsized":"2024-05-26T22:51:39.693024+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:54:08.069987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.676650+0000","last_change":"2024-05-26T22:51:38.655494+0000","last_active":"2024-05-26T22:51:39.676650+0000","last_peered":"2024-05-26T22:51:39.676650+0000","last_clean":"2024-05-26T22:51:39.676650+0000","last_became_active":"2024-05-26T22:51:38.655095+0000","last_became_peered":"2024-05-26T22:51:38.655095+0000","last_unstale":"2024-05-26T22:51:39.676650+0000","last_undegraded":"2024-05-26T22:51:39.676650+0000","last_fullsized":"2024-05-26T22:51:39.676650+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_clean_scrub_stamp":"2024-05-26T22:51:37.608737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:22:13.841746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.684555+0000","last_change":"2024-05-26T22:51:39.682477+0000","last_active":"2024-05-26T22:51:39.684555+0000","last_peered":"2024-05-26T22:51:39.684555+0000","last_clean":"2024-05-26T22:51:39.684555+0000","last_became_active":"2024-05-26T22:51:39.682200+0000","last_became_peered":"2024-05-26T22:51:39.682200+0000","last_unstale":"2024-05-26T22:51:39.684555+0000","last_undegraded":"2024-05-26T22:51:39.684555+0000","last_fullsized":"2024-05-26T22:51:39.684555+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:11:48.631048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"57'3","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.702223+0000","last_change":"2024-05-26T22:51:38.678350+0000","last_active":"2024-05-26T22:51:39.702223+0000","last_peered":"2024-05-26T22:51:39.702223+0000","last_clean":"2024-05-26T22:51:39.702223+0000","last_became_active":"2024-05-26T22:51:38.677993+0000","last_became_peered":"2024-05-26T22:51:38.677993+0000","last_unstale":"2024-05-26T22:51:39.702223+0000","last_undegraded":"2024-05-26T22:51:39.702223+0000","last_fullsized":"2024-05-26T22:51:39.702223+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_clean_scrub_stamp":"2024-05-26T22:51:37.608737+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T23:59:47.896772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.693983+0000","last_change":"2024-05-26T22:51:39.686519+0000","last_active":"2024-05-26T22:51:39.693983+0000","last_peered":"2024-05-26T22:51:39.693983+0000","last_clean":"2024-05-26T22:51:39.693983+0000","last_became_active":"2024-05-26T22:51:39.686043+0000","last_became_peered":"2024-05-26T22:51:39.686043+0000","last_unstale":"2024-05-26T22:51:39.693983+0000","last_undegraded":"2024-05-26T22:51:39.693983+0000","last_fullsized":"2024-05-26T22:51:39.693983+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:19:31.293296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"57'6","reported_seq":18,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.705195+0000","last_change":"2024-05-26T22:51:38.674489+0000","last_active":"2024-05-26T22:51:39.705195+0000","last_peered":"2024-05-26T22:51:39.705195+0000","last_clean":"2024-05-26T22:51:39.705195+0000","last_became_active":"2024-05-26T22:51:38.672180+0000","last_became_peered":"2024-05-26T22:51:38.672180+0000","last_unstale":"2024-05-26T22:51:39.705195+0000","last_undegraded":"2024-05-26T22:51:39.705195+0000","last_fullsized":"2024-05-26T22:51:39.705195+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_clean_scrub_stamp":"2024-05-26T22:51:37.608737+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:50:35.189821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.690880+0000","last_change":"2024-05-26T22:51:39.685191+0000","last_active":"2024-05-26T22:51:39.690880+0000","last_peered":"2024-05-26T22:51:39.690880+0000","last_clean":"2024-05-26T22:51:39.690880+0000","last_became_active":"2024-05-26T22:51:39.684917+0000","last_became_peered":"2024-05-26T22:51:39.684917+0000","last_unstale":"2024-05-26T22:51:39.690880+0000","last_undegraded":"2024-05-26T22:51:39.690880+0000","last_fullsized":"2024-05-26T22:51:39.690880+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:53:31.658157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"57'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.704976+0000","last_change":"2024-05-26T22:51:38.663472+0000","last_active":"2024-05-26T22:51:39.704976+0000","last_peered":"2024-05-26T22:51:39.704976+0000","last_clean":"2024-05-26T22:51:39.704976+0000","last_became_active":"2024-05-26T22:51:38.663084+0000","last_became_peered":"2024-05-26T22:51:38.663084+0000","last_unstale":"2024-05-26T22:51:39.704976+0000","last_undegraded":"2024-05-26T22:51:39.704976+0000","last_fullsized":"2024-05-26T22:51:39.704976+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_clean_scrub_stamp":"2024-05-26T22:51:37.608737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T23:25:47.738153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.692999+0000","last_change":"2024-05-26T22:51:39.686638+0000","last_active":"2024-05-26T22:51:39.692999+0000","last_peered":"2024-05-26T22:51:39.692999+0000","last_clean":"2024-05-26T22:51:39.692999+0000","last_became_active":"2024-05-26T22:51:39.685858+0000","last_became_peered":"2024-05-26T22:51:39.685858+0000","last_unstale":"2024-05-26T22:51:39.692999+0000","last_undegraded":"2024-05-26T22:51:39.692999+0000","last_fullsized":"2024-05-26T22:51:39.692999+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:50:28.022562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.650524+0000","last_change":"2024-05-26T22:51:38.674401+0000","last_active":"2024-05-26T22:51:39.650524+0000","last_peered":"2024-05-26T22:51:39.650524+0000","last_clean":"2024-05-26T22:51:39.650524+0000","last_became_active":"2024-05-26T22:51:38.673184+0000","last_became_peered":"2024-05-26T22:51:38.673184+0000","last_unstale":"2024-05-26T22:51:39.650524+0000","last_undegraded":"2024-05-26T22:51:39.650524+0000","last_fullsized":"2024-05-26T22:51:39.650524+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_clean_scrub_stamp":"2024-05-26T22:51:37.608737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:09:25.370801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.687046+0000","last_change":"2024-05-26T22:51:39.683074+0000","last_active":"2024-05-26T22:51:39.687046+0000","last_peered":"2024-05-26T22:51:39.687046+0000","last_clean":"2024-05-26T22:51:39.687046+0000","last_became_active":"2024-05-26T22:51:39.680613+0000","last_became_peered":"2024-05-26T22:51:39.680613+0000","last_unstale":"2024-05-26T22:51:39.687046+0000","last_undegraded":"2024-05-26T22:51:39.687046+0000","last_fullsized":"2024-05-26T22:51:39.687046+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:21:18.774120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.662469+0000","last_change":"2024-05-26T22:51:38.677513+0000","last_active":"2024-05-26T22:51:39.662469+0000","last_peered":"2024-05-26T22:51:39.662469+0000","last_clean":"2024-05-26T22:51:39.662469+0000","last_became_active":"2024-05-26T22:51:38.677223+0000","last_became_peered":"2024-05-26T22:51:38.677223+0000","last_unstale":"2024-05-26T22:51:39.662469+0000","last_undegraded":"2024-05-26T22:51:39.662469+0000","last_fullsized":"2024-05-26T22:51:39.662469+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_clean_scrub_stamp":"2024-05-26T22:51:37.608737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T23:59:47.896772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.685978+0000","last_change":"2024-05-26T22:51:39.679663+0000","last_active":"2024-05-26T22:51:39.685978+0000","last_peered":"2024-05-26T22:51:39.685978+0000","last_clean":"2024-05-26T22:51:39.685978+0000","last_became_active":"2024-05-26T22:51:39.678171+0000","last_became_peered":"2024-05-26T22:51:39.678171+0000","last_unstale":"2024-05-26T22:51:39.685978+0000","last_undegraded":"2024-05-26T22:51:39.685978+0000","last_fullsized":"2024-05-26T22:51:39.685978+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:22:56.866177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"57'8","reported_seq":20,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.704953+0000","last_change":"2024-05-26T22:51:38.679123+0000","last_active":"2024-05-26T22:51:39.704953+0000","last_peered":"2024-05-26T22:51:39.704953+0000","last_clean":"2024-05-26T22:51:39.704953+0000","last_became_active":"2024-05-26T22:51:38.678677+0000","last_became_peered":"2024-05-26T22:51:38.678677+0000","last_unstale":"2024-05-26T22:51:39.704953+0000","last_undegraded":"2024-05-26T22:51:39.704953+0000","last_fullsized":"2024-05-26T22:51:39.704953+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_clean_scrub_stamp":"2024-05-26T22:51:37.608737+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:56:05.395225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.693030+0000","last_change":"2024-05-26T22:51:39.688473+0000","last_active":"2024-05-26T22:51:39.693030+0000","last_peered":"2024-05-26T22:51:39.693030+0000","last_clean":"2024-05-26T22:51:39.693030+0000","last_became_active":"2024-05-26T22:51:39.685899+0000","last_became_peered":"2024-05-26T22:51:39.685899+0000","last_unstale":"2024-05-26T22:51:39.693030+0000","last_undegraded":"2024-05-26T22:51:39.693030+0000","last_fullsized":"2024-05-26T22:51:39.693030+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:08:00.991256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"57'1","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.704908+0000","last_change":"2024-05-26T22:51:38.662805+0000","last_active":"2024-05-26T22:51:39.704908+0000","last_peered":"2024-05-26T22:51:39.704908+0000","last_clean":"2024-05-26T22:51:39.704908+0000","last_became_active":"2024-05-26T22:51:38.662504+0000","last_became_peered":"2024-05-26T22:51:38.662504+0000","last_unstale":"2024-05-26T22:51:39.704908+0000","last_undegraded":"2024-05-26T22:51:39.704908+0000","last_fullsized":"2024-05-26T22:51:39.704908+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_clean_scrub_stamp":"2024-05-26T22:51:37.608737+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:28:48.523435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.683284+0000","last_change":"2024-05-26T22:51:39.679084+0000","last_active":"2024-05-26T22:51:39.683284+0000","last_peered":"2024-05-26T22:51:39.683284+0000","last_clean":"2024-05-26T22:51:39.683284+0000","last_became_active":"2024-05-26T22:51:39.678447+0000","last_became_peered":"2024-05-26T22:51:39.678447+0000","last_unstale":"2024-05-26T22:51:39.683284+0000","last_undegraded":"2024-05-26T22:51:39.683284+0000","last_fullsized":"2024-05-26T22:51:39.683284+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:19:31.293296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"57'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.699771+0000","last_change":"2024-05-26T22:51:38.664104+0000","last_active":"2024-05-26T22:51:39.699771+0000","last_peered":"2024-05-26T22:51:39.699771+0000","last_clean":"2024-05-26T22:51:39.699771+0000","last_became_active":"2024-05-26T22:51:38.663821+0000","last_became_peered":"2024-05-26T22:51:38.663821+0000","last_unstale":"2024-05-26T22:51:39.699771+0000","last_undegraded":"2024-05-26T22:51:39.699771+0000","last_fullsized":"2024-05-26T22:51:39.699771+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_clean_scrub_stamp":"2024-05-26T22:51:37.608737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:32:58.322692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.701440+0000","last_change":"2024-05-26T22:51:39.697587+0000","last_active":"2024-05-26T22:51:39.701440+0000","last_peered":"2024-05-26T22:51:39.701440+0000","last_clean":"2024-05-26T22:51:39.701440+0000","last_became_active":"2024-05-26T22:51:39.697215+0000","last_became_peered":"2024-05-26T22:51:39.697215+0000","last_unstale":"2024-05-26T22:51:39.701440+0000","last_undegraded":"2024-05-26T22:51:39.701440+0000","last_fullsized":"2024-05-26T22:51:39.701440+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:48:59.747131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"57'1","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.698228+0000","last_change":"2024-05-26T22:51:38.680443+0000","last_active":"2024-05-26T22:51:39.698228+0000","last_peered":"2024-05-26T22:51:39.698228+0000","last_clean":"2024-05-26T22:51:39.698228+0000","last_became_active":"2024-05-26T22:51:38.680110+0000","last_became_peered":"2024-05-26T22:51:38.680110+0000","last_unstale":"2024-05-26T22:51:39.698228+0000","last_undegraded":"2024-05-26T22:51:39.698228+0000","last_fullsized":"2024-05-26T22:51:39.698228+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_clean_scrub_stamp":"2024-05-26T22:51:37.608737+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:32:58.322692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.694001+0000","last_change":"2024-05-26T22:51:39.690545+0000","last_active":"2024-05-26T22:51:39.694001+0000","last_peered":"2024-05-26T22:51:39.694001+0000","last_clean":"2024-05-26T22:51:39.694001+0000","last_became_active":"2024-05-26T22:51:39.690244+0000","last_became_peered":"2024-05-26T22:51:39.690244+0000","last_unstale":"2024-05-26T22:51:39.694001+0000","last_undegraded":"2024-05-26T22:51:39.694001+0000","last_fullsized":"2024-05-26T22:51:39.694001+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:53:31.658157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.662098+0000","last_change":"2024-05-26T22:51:38.669746+0000","last_active":"2024-05-26T22:51:39.662098+0000","last_peered":"2024-05-26T22:51:39.662098+0000","last_clean":"2024-05-26T22:51:39.662098+0000","last_became_active":"2024-05-26T22:51:38.667884+0000","last_became_peered":"2024-05-26T22:51:38.667884+0000","last_unstale":"2024-05-26T22:51:39.662098+0000","last_undegraded":"2024-05-26T22:51:39.662098+0000","last_fullsized":"2024-05-26T22:51:39.662098+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_clean_scrub_stamp":"2024-05-26T22:51:37.608737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:22:55.851009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.680474+0000","last_change":"2024-05-26T22:51:39.674860+0000","last_active":"2024-05-26T22:51:39.680474+0000","last_peered":"2024-05-26T22:51:39.680474+0000","last_clean":"2024-05-26T22:51:39.680474+0000","last_became_active":"2024-05-26T22:51:39.674521+0000","last_became_peered":"2024-05-26T22:51:39.674521+0000","last_unstale":"2024-05-26T22:51:39.680474+0000","last_undegraded":"2024-05-26T22:51:39.680474+0000","last_fullsized":"2024-05-26T22:51:39.680474+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:31:36.673510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"57'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.707850+0000","last_change":"2024-05-26T22:51:38.669282+0000","last_active":"2024-05-26T22:51:39.707850+0000","last_peered":"2024-05-26T22:51:39.707850+0000","last_clean":"2024-05-26T22:51:39.707850+0000","last_became_active":"2024-05-26T22:51:38.668954+0000","last_became_peered":"2024-05-26T22:51:38.668954+0000","last_unstale":"2024-05-26T22:51:39.707850+0000","last_undegraded":"2024-05-26T22:51:39.707850+0000","last_fullsized":"2024-05-26T22:51:39.707850+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_clean_scrub_stamp":"2024-05-26T22:51:37.608737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:52:33.458972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.686243+0000","last_change":"2024-05-26T22:51:39.680673+0000","last_active":"2024-05-26T22:51:39.686243+0000","last_peered":"2024-05-26T22:51:39.686243+0000","last_clean":"2024-05-26T22:51:39.686243+0000","last_became_active":"2024-05-26T22:51:39.680407+0000","last_became_peered":"2024-05-26T22:51:39.680407+0000","last_unstale":"2024-05-26T22:51:39.686243+0000","last_undegraded":"2024-05-26T22:51:39.686243+0000","last_fullsized":"2024-05-26T22:51:39.686243+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T23:25:48.753320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.657275+0000","last_change":"2024-05-26T22:51:38.669083+0000","last_active":"2024-05-26T22:51:39.657275+0000","last_peered":"2024-05-26T22:51:39.657275+0000","last_clean":"2024-05-26T22:51:39.657275+0000","last_became_active":"2024-05-26T22:51:38.668811+0000","last_became_peered":"2024-05-26T22:51:38.668811+0000","last_unstale":"2024-05-26T22:51:39.657275+0000","last_undegraded":"2024-05-26T22:51:39.657275+0000","last_fullsized":"2024-05-26T22:51:39.657275+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_clean_scrub_stamp":"2024-05-26T22:51:37.608737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:10:19.722657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.680490+0000","last_change":"2024-05-26T22:51:39.675832+0000","last_active":"2024-05-26T22:51:39.680490+0000","last_peered":"2024-05-26T22:51:39.680490+0000","last_clean":"2024-05-26T22:51:39.680490+0000","last_became_active":"2024-05-26T22:51:39.675480+0000","last_became_peered":"2024-05-26T22:51:39.675480+0000","last_unstale":"2024-05-26T22:51:39.680490+0000","last_undegraded":"2024-05-26T22:51:39.680490+0000","last_fullsized":"2024-05-26T22:51:39.680490+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:08:00.991256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.662500+0000","last_change":"2024-05-26T22:51:38.673393+0000","last_active":"2024-05-26T22:51:39.662500+0000","last_peered":"2024-05-26T22:51:39.662500+0000","last_clean":"2024-05-26T22:51:39.662500+0000","last_became_active":"2024-05-26T22:51:38.673070+0000","last_became_peered":"2024-05-26T22:51:38.673070+0000","last_unstale":"2024-05-26T22:51:39.662500+0000","last_undegraded":"2024-05-26T22:51:39.662500+0000","last_fullsized":"2024-05-26T22:51:39.662500+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_clean_scrub_stamp":"2024-05-26T22:51:37.608737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:10:38.410360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.680408+0000","last_change":"2024-05-26T22:51:39.674633+0000","last_active":"2024-05-26T22:51:39.680408+0000","last_peered":"2024-05-26T22:51:39.680408+0000","last_clean":"2024-05-26T22:51:39.680408+0000","last_became_active":"2024-05-26T22:51:39.674316+0000","last_became_peered":"2024-05-26T22:51:39.674316+0000","last_unstale":"2024-05-26T22:51:39.680408+0000","last_undegraded":"2024-05-26T22:51:39.680408+0000","last_fullsized":"2024-05-26T22:51:39.680408+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:54:08.069987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"57'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.701985+0000","last_change":"2024-05-26T22:51:38.667102+0000","last_active":"2024-05-26T22:51:39.701985+0000","last_peered":"2024-05-26T22:51:39.701985+0000","last_clean":"2024-05-26T22:51:39.701985+0000","last_became_active":"2024-05-26T22:51:38.666792+0000","last_became_peered":"2024-05-26T22:51:38.666792+0000","last_unstale":"2024-05-26T22:51:39.701985+0000","last_undegraded":"2024-05-26T22:51:39.701985+0000","last_fullsized":"2024-05-26T22:51:39.701985+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_clean_scrub_stamp":"2024-05-26T22:51:37.608737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:09:25.370801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.680416+0000","last_change":"2024-05-26T22:51:39.674265+0000","last_active":"2024-05-26T22:51:39.680416+0000","last_peered":"2024-05-26T22:51:39.680416+0000","last_clean":"2024-05-26T22:51:39.680416+0000","last_became_active":"2024-05-26T22:51:39.673905+0000","last_became_peered":"2024-05-26T22:51:39.673905+0000","last_unstale":"2024-05-26T22:51:39.680416+0000","last_undegraded":"2024-05-26T22:51:39.680416+0000","last_fullsized":"2024-05-26T22:51:39.680416+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:48:18.385788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.668597+0000","last_change":"2024-05-26T22:51:38.667697+0000","last_active":"2024-05-26T22:51:39.668597+0000","last_peered":"2024-05-26T22:51:39.668597+0000","last_clean":"2024-05-26T22:51:39.668597+0000","last_became_active":"2024-05-26T22:51:38.666882+0000","last_became_peered":"2024-05-26T22:51:38.666882+0000","last_unstale":"2024-05-26T22:51:39.668597+0000","last_undegraded":"2024-05-26T22:51:39.668597+0000","last_fullsized":"2024-05-26T22:51:39.668597+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_clean_scrub_stamp":"2024-05-26T22:51:37.608737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:48:17.370621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.680888+0000","last_change":"2024-05-26T22:51:39.672768+0000","last_active":"2024-05-26T22:51:39.680888+0000","last_peered":"2024-05-26T22:51:39.680888+0000","last_clean":"2024-05-26T22:51:39.680888+0000","last_became_active":"2024-05-26T22:51:39.672501+0000","last_became_peered":"2024-05-26T22:51:39.672501+0000","last_unstale":"2024-05-26T22:51:39.680888+0000","last_undegraded":"2024-05-26T22:51:39.680888+0000","last_fullsized":"2024-05-26T22:51:39.680888+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:48:34.561071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.650660+0000","last_change":"2024-05-26T22:51:38.676020+0000","last_active":"2024-05-26T22:51:39.650660+0000","last_peered":"2024-05-26T22:51:39.650660+0000","last_clean":"2024-05-26T22:51:39.650660+0000","last_became_active":"2024-05-26T22:51:38.675705+0000","last_became_peered":"2024-05-26T22:51:38.675705+0000","last_unstale":"2024-05-26T22:51:39.650660+0000","last_undegraded":"2024-05-26T22:51:39.650660+0000","last_fullsized":"2024-05-26T22:51:39.650660+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_clean_scrub_stamp":"2024-05-26T22:51:37.608737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:10:19.722657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.684805+0000","last_change":"2024-05-26T22:51:39.677822+0000","last_active":"2024-05-26T22:51:39.684805+0000","last_peered":"2024-05-26T22:51:39.684805+0000","last_clean":"2024-05-26T22:51:39.684805+0000","last_became_active":"2024-05-26T22:51:39.677458+0000","last_became_peered":"2024-05-26T22:51:39.677458+0000","last_unstale":"2024-05-26T22:51:39.684805+0000","last_undegraded":"2024-05-26T22:51:39.684805+0000","last_fullsized":"2024-05-26T22:51:39.684805+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:33:52.360239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.670789+0000","last_change":"2024-05-26T22:51:38.679591+0000","last_active":"2024-05-26T22:51:39.670789+0000","last_peered":"2024-05-26T22:51:39.670789+0000","last_clean":"2024-05-26T22:51:39.670789+0000","last_became_active":"2024-05-26T22:51:38.679262+0000","last_became_peered":"2024-05-26T22:51:38.679262+0000","last_unstale":"2024-05-26T22:51:39.670789+0000","last_undegraded":"2024-05-26T22:51:39.670789+0000","last_fullsized":"2024-05-26T22:51:39.670789+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_clean_scrub_stamp":"2024-05-26T22:51:37.608737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:22:55.851009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.701198+0000","last_change":"2024-05-26T22:51:39.696587+0000","last_active":"2024-05-26T22:51:39.701198+0000","last_peered":"2024-05-26T22:51:39.701198+0000","last_clean":"2024-05-26T22:51:39.701198+0000","last_became_active":"2024-05-26T22:51:39.696269+0000","last_became_peered":"2024-05-26T22:51:39.696269+0000","last_unstale":"2024-05-26T22:51:39.701198+0000","last_undegraded":"2024-05-26T22:51:39.701198+0000","last_fullsized":"2024-05-26T22:51:39.701198+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:22:19.684542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.653783+0000","last_change":"2024-05-26T22:51:38.675053+0000","last_active":"2024-05-26T22:51:39.653783+0000","last_peered":"2024-05-26T22:51:39.653783+0000","last_clean":"2024-05-26T22:51:39.653783+0000","last_became_active":"2024-05-26T22:51:38.674504+0000","last_became_peered":"2024-05-26T22:51:38.674504+0000","last_unstale":"2024-05-26T22:51:39.653783+0000","last_undegraded":"2024-05-26T22:51:39.653783+0000","last_fullsized":"2024-05-26T22:51:39.653783+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_clean_scrub_stamp":"2024-05-26T22:51:37.608737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:54:41.244332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.691889+0000","last_change":"2024-05-26T22:51:39.684412+0000","last_active":"2024-05-26T22:51:39.691889+0000","last_peered":"2024-05-26T22:51:39.691889+0000","last_clean":"2024-05-26T22:51:39.691889+0000","last_became_active":"2024-05-26T22:51:39.683555+0000","last_became_peered":"2024-05-26T22:51:39.683555+0000","last_unstale":"2024-05-26T22:51:39.691889+0000","last_undegraded":"2024-05-26T22:51:39.691889+0000","last_fullsized":"2024-05-26T22:51:39.691889+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:48:18.385788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.668501+0000","last_change":"2024-05-26T22:51:38.672448+0000","last_active":"2024-05-26T22:51:39.668501+0000","last_peered":"2024-05-26T22:51:39.668501+0000","last_clean":"2024-05-26T22:51:39.668501+0000","last_became_active":"2024-05-26T22:51:38.672116+0000","last_became_peered":"2024-05-26T22:51:38.672116+0000","last_unstale":"2024-05-26T22:51:39.668501+0000","last_undegraded":"2024-05-26T22:51:39.668501+0000","last_fullsized":"2024-05-26T22:51:39.668501+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_clean_scrub_stamp":"2024-05-26T22:51:37.608737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:53:23.294575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.670200+0000","last_change":"2024-05-26T22:51:39.659441+0000","last_active":"2024-05-26T22:51:39.670200+0000","last_peered":"2024-05-26T22:51:39.670200+0000","last_clean":"2024-05-26T22:51:39.670200+0000","last_became_active":"2024-05-26T22:51:39.658590+0000","last_became_peered":"2024-05-26T22:51:39.658590+0000","last_unstale":"2024-05-26T22:51:39.670200+0000","last_undegraded":"2024-05-26T22:51:39.670200+0000","last_fullsized":"2024-05-26T22:51:39.670200+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T23:30:00.567203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.652829+0000","last_change":"2024-05-26T22:51:38.671647+0000","last_active":"2024-05-26T22:51:39.652829+0000","last_peered":"2024-05-26T22:51:39.652829+0000","last_clean":"2024-05-26T22:51:39.652829+0000","last_became_active":"2024-05-26T22:51:38.671341+0000","last_became_peered":"2024-05-26T22:51:38.671341+0000","last_unstale":"2024-05-26T22:51:39.652829+0000","last_undegraded":"2024-05-26T22:51:39.652829+0000","last_fullsized":"2024-05-26T22:51:39.652829+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_clean_scrub_stamp":"2024-05-26T22:51:37.608737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:05:58.675660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.692982+0000","last_change":"2024-05-26T22:51:39.683047+0000","last_active":"2024-05-26T22:51:39.692982+0000","last_peered":"2024-05-26T22:51:39.692982+0000","last_clean":"2024-05-26T22:51:39.692982+0000","last_became_active":"2024-05-26T22:51:39.682464+0000","last_became_peered":"2024-05-26T22:51:39.682464+0000","last_unstale":"2024-05-26T22:51:39.692982+0000","last_undegraded":"2024-05-26T22:51:39.692982+0000","last_fullsized":"2024-05-26T22:51:39.692982+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T23:57:52.148119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.684774+0000","last_change":"2024-05-26T22:51:38.670209+0000","last_active":"2024-05-26T22:51:39.684774+0000","last_peered":"2024-05-26T22:51:39.684774+0000","last_clean":"2024-05-26T22:51:39.684774+0000","last_became_active":"2024-05-26T22:51:38.669932+0000","last_became_peered":"2024-05-26T22:51:38.669932+0000","last_unstale":"2024-05-26T22:51:39.684774+0000","last_undegraded":"2024-05-26T22:51:39.684774+0000","last_fullsized":"2024-05-26T22:51:39.684774+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_clean_scrub_stamp":"2024-05-26T22:51:37.608737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:38:10.691783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.699284+0000","last_change":"2024-05-26T22:51:39.686302+0000","last_active":"2024-05-26T22:51:39.699284+0000","last_peered":"2024-05-26T22:51:39.699284+0000","last_clean":"2024-05-26T22:51:39.699284+0000","last_became_active":"2024-05-26T22:51:39.686044+0000","last_became_peered":"2024-05-26T22:51:39.686044+0000","last_unstale":"2024-05-26T22:51:39.699284+0000","last_undegraded":"2024-05-26T22:51:39.699284+0000","last_fullsized":"2024-05-26T22:51:39.699284+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:38.623904+0000","last_clean_scrub_stamp":"2024-05-26T22:51:38.623904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:05:00.695645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.650132+0000","last_change":"2024-05-26T22:51:38.673169+0000","last_active":"2024-05-26T22:51:39.650132+0000","last_peered":"2024-05-26T22:51:39.650132+0000","last_clean":"2024-05-26T22:51:39.650132+0000","last_became_active":"2024-05-26T22:51:38.672105+0000","last_became_peered":"2024-05-26T22:51:38.672105+0000","last_unstale":"2024-05-26T22:51:39.650132+0000","last_undegraded":"2024-05-26T22:51:39.650132+0000","last_fullsized":"2024-05-26T22:51:39.650132+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:51:37.608737+0000","last_clean_scrub_stamp":"2024-05-26T22:51:37.608737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:54:41.244332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":238,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-26T22:51:39.667362+0000","last_change":"2024-05-26T22:48:12.068917+0000","last_active":"2024-05-26T22:51:39.667362+0000","last_peered":"2024-05-26T22:51:39.667362+0000","last_clean":"2024-05-26T22:51:39.667362+0000","last_became_active":"2024-05-26T22:48:12.065934+0000","last_became_peered":"2024-05-26T22:48:12.065934+0000","last_unstale":"2024-05-26T22:51:39.667362+0000","last_undegraded":"2024-05-26T22:51:39.667362+0000","last_fullsized":"2024-05-26T22:51:39.667362+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:10.628122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:10.628122+0000","last_clean_scrub_stamp":"2024-05-26T22:48:10.628122+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:14:58.895911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574893,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32436,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339404,"statfs":{"total":96636764160,"available":96603549696,"internally_reserved":0,"allocated":1277952,"data_stored":877885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574893,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31852,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339988,"statfs":{"total":96636764160,"available":96604147712,"internally_reserved":0,"allocated":684032,"data_stored":287517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574894,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31860,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339980,"statfs":{"total":96636764160,"available":96604139520,"internally_reserved":0,"allocated":692224,"data_stored":288169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574893,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31852,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339988,"statfs":{"total":96636764160,"available":96604147712,"internally_reserved":0,"allocated":684032,"data_stored":287517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574893,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31840,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340000,"statfs":{"total":96636764160,"available":96604160000,"internally_reserved":0,"allocated":671744,"data_stored":280742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574893,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31852,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339988,"statfs":{"total":96636764160,"available":96604147712,"internally_reserved":0,"allocated":684032,"data_stored":287517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574893,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31856,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339984,"statfs":{"total":96636764160,"available":96604143616,"internally_reserved":0,"allocated":688128,"data_stored":287607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574893,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32444,"kb_used_data":1264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339396,"statfs":{"total":96636764160,"available":96603541504,"internally_reserved":0,"allocated":1294336,"data_stored":880079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-26T22:51:44.554 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-26T22:51:44.554 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-26T22:51:44.701 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-26T22:51:44.709 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-26T22:51:44.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.807+0000 7f67f3800640 1 Processor -- start 2024-05-26T22:51:44.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.807+0000 7f67f3800640 1 -- start start 2024-05-26T22:51:44.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.808+0000 7f67f3800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f67ec15e4e0 0x7f67ec0b9c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:44.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.808+0000 7f67f3800640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f67ec0d2330 0x7f67ec0cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:44.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.808+0000 7f67f3800640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67ec0d28f0 con 0x7f67ec0cf550 2024-05-26T22:51:44.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.808+0000 7f67f3800640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f67ec0d2bd0 con 0x7f67ec0d2330 2024-05-26T22:51:44.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.808+0000 7f67f3800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f67ec0d2eb0 con 0x7f67ec15e4e0 2024-05-26T22:51:44.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.809+0000 7f67ebe00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f67ec0d2330 0x7f67ec0cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:44.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.809+0000 7f67f1800640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f67ec0cf550 0x7f67ec0d3c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:37878/0 (socket says 172.21.15.131:37878) 2024-05-26T22:51:44.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.809+0000 7f67f1800640 1 -- 172.21.15.131:0/1084875924 learned_addr learned my addr 172.21.15.131:0/1084875924 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:51:44.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.809+0000 7f67f0e00640 1 --2- 172.21.15.131:0/1084875924 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f67ec15e4e0 0x7f67ec0b9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:44.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.809+0000 7f67ebe00640 1 -- 172.21.15.131:0/1084875924 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f67ec15e4e0 msgr2=0x7f67ec0b9c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:44.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.809+0000 7f67ebe00640 1 --2- 172.21.15.131:0/1084875924 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f67ec15e4e0 0x7f67ec0b9c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:44.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.809+0000 7f67ebe00640 1 -- 172.21.15.131:0/1084875924 >> v1:172.21.15.131:6789/0 conn(0x7f67ec0cf550 legacy=0x7f67ec0d3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:44.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.809+0000 7f67ebe00640 1 -- 172.21.15.131:0/1084875924 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67ec0d3190 con 0x7f67ec0d2330 2024-05-26T22:51:44.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.809+0000 7f67eb400640 1 -- 172.21.15.131:0/1084875924 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1075370157 0 0) 0x7f67ec0d28f0 con 0x7f67ec0cf550 2024-05-26T22:51:44.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.810+0000 7f67ebe00640 1 --2- 172.21.15.131:0/1084875924 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f67ec0d2330 0x7f67ec0cf010 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f67d8009d90 tx=0x7f67d802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af477e574053cb21 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:44.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.811+0000 7f67eb400640 1 -- 172.21.15.131:0/1084875924 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d8007740 con 0x7f67ec0d2330 2024-05-26T22:51:44.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.811+0000 7f67eb400640 1 -- 172.21.15.131:0/1084875924 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67d80078e0 con 0x7f67ec0d2330 2024-05-26T22:51:44.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.811+0000 7f67eb400640 1 -- 172.21.15.131:0/1084875924 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d8007cb0 con 0x7f67ec0d2330 2024-05-26T22:51:44.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.811+0000 7f67f3800640 1 -- 172.21.15.131:0/1084875924 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f67ec0d2330 msgr2=0x7f67ec0cf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:44.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.811+0000 7f67f3800640 1 --2- 172.21.15.131:0/1084875924 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f67ec0d2330 0x7f67ec0cf010 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f67d8009d90 tx=0x7f67d802a660 comp rx=0 tx=0).stop 2024-05-26T22:51:44.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.811+0000 7f67f3800640 1 -- 172.21.15.131:0/1084875924 shutdown_connections 2024-05-26T22:51:44.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.811+0000 7f67f3800640 1 --2- 172.21.15.131:0/1084875924 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f67ec0d2330 0x7f67ec0cf010 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:44.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.811+0000 7f67f3800640 1 --2- 172.21.15.131:0/1084875924 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f67ec15e4e0 0x7f67ec0b9c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:44.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.811+0000 7f67f3800640 1 -- 172.21.15.131:0/1084875924 >> 172.21.15.131:0/1084875924 conn(0x7f67ec072e90 msgr2=0x7f67ec0732a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:44.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.811+0000 7f67f3800640 1 -- 172.21.15.131:0/1084875924 shutdown_connections 2024-05-26T22:51:44.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.812+0000 7f67f3800640 1 -- 172.21.15.131:0/1084875924 wait complete. 2024-05-26T22:51:44.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.813+0000 7f67f3800640 1 Processor -- start 2024-05-26T22:51:44.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.813+0000 7f67f3800640 1 -- start start 2024-05-26T22:51:44.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.814+0000 7f67f3800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f67ec0cf550 0x7f67ec152440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:44.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.814+0000 7f67f3800640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f67ec0d2330 0x7f67ec152980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:44.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.814+0000 7f67f3800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f67ec15e4e0 0x7f67ec0c8280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:44.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.814+0000 7f67f3800640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f67ec0d3990 con 0x7f67ec15e4e0 2024-05-26T22:51:44.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.814+0000 7f67f3800640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f67ec15e8c0 con 0x7f67ec0d2330 2024-05-26T22:51:44.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.814+0000 7f67f3800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f67ec0ba1c0 con 0x7f67ec0cf550 2024-05-26T22:51:44.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.815+0000 7f67f1800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f67ec15e4e0 0x7f67ec0c8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:44.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.815+0000 7f67f1800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f67ec15e4e0 0x7f67ec0c8280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47734/0 (socket says 172.21.15.131:47734) 2024-05-26T22:51:44.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.815+0000 7f67f1800640 1 -- 172.21.15.131:0/1110635319 learned_addr learned my addr 172.21.15.131:0/1110635319 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:44.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.815+0000 7f67f0e00640 1 --2- 172.21.15.131:0/1110635319 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f67ec0cf550 0x7f67ec152440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:44.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.815+0000 7f67ebe00640 1 --2- 172.21.15.131:0/1110635319 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f67ec0d2330 0x7f67ec152980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:44.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.815+0000 7f67f1800640 1 -- 172.21.15.131:0/1110635319 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f67ec0cf550 msgr2=0x7f67ec152440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:44.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.815+0000 7f67f1800640 1 --2- 172.21.15.131:0/1110635319 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f67ec0cf550 0x7f67ec152440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:44.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.815+0000 7f67f1800640 1 -- 172.21.15.131:0/1110635319 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f67ec0d2330 msgr2=0x7f67ec152980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:44.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.815+0000 7f67f1800640 1 --2- 172.21.15.131:0/1110635319 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f67ec0d2330 0x7f67ec152980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:44.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.815+0000 7f67f1800640 1 -- 172.21.15.131:0/1110635319 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67ec150430 con 0x7f67ec15e4e0 2024-05-26T22:51:44.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.815+0000 7f67f1800640 1 --2- 172.21.15.131:0/1110635319 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f67ec15e4e0 0x7f67ec0c8280 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f67e400c410 tx=0x7f67e400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:44.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.816+0000 7f67e8c00640 1 -- 172.21.15.131:0/1110635319 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e40074e0 con 0x7f67ec15e4e0 2024-05-26T22:51:44.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.816+0000 7f67e8c00640 1 -- 172.21.15.131:0/1110635319 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67e40047e0 con 0x7f67ec15e4e0 2024-05-26T22:51:44.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.816+0000 7f67e8c00640 1 -- 172.21.15.131:0/1110635319 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e4004bd0 con 0x7f67ec15e4e0 2024-05-26T22:51:44.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.816+0000 7f67f3800640 1 -- 172.21.15.131:0/1110635319 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67ec0c9190 con 0x7f67ec15e4e0 2024-05-26T22:51:44.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.816+0000 7f67f3800640 1 -- 172.21.15.131:0/1110635319 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67ec0c9750 con 0x7f67ec15e4e0 2024-05-26T22:51:44.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.817+0000 7f67f3800640 1 -- 172.21.15.131:0/1110635319 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67ec0695c0 con 0x7f67ec15e4e0 2024-05-26T22:51:44.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.818+0000 7f67e8c00640 1 -- 172.21.15.131:0/1110635319 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f67e4011070 con 0x7f67ec15e4e0 2024-05-26T22:51:44.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.819+0000 7f67e8c00640 1 --2- 172.21.15.131:0/1110635319 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f67cc06be30 0x7f67cc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:44.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.819+0000 7f67e8c00640 1 -- 172.21.15.131:0/1110635319 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f67e408aa60 con 0x7f67ec15e4e0 2024-05-26T22:51:44.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.819+0000 7f67f0e00640 1 --2- 172.21.15.131:0/1110635319 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f67cc06be30 0x7f67cc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:44.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.820+0000 7f67f0e00640 1 --2- 172.21.15.131:0/1110635319 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f67cc06be30 0x7f67cc06e2f0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f67dc006fd0 tx=0x7f67dc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:44.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:44.823+0000 7f67e8c00640 1 -- 172.21.15.131:0/1110635319 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f67e4059370 con 0x7f67ec15e4e0 2024-05-26T22:51:45.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:45.070+0000 7f67f3800640 1 -- 172.21.15.131:0/1110635319 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f67ec0ca860 con 0x7f67ec15e4e0 2024-05-26T22:51:45.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:45.076+0000 7f67e8c00640 1 -- 172.21.15.131:0/1110635319 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) ==== 190+0+147 (secure 0 0 0) 0x7f67e405f020 con 0x7f67ec15e4e0 2024-05-26T22:51:45.076 INFO:teuthology.orchestra.run.smithi131.stdout:[client.0] 2024-05-26T22:51:45.076 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQD2u1NmSGVWChAAH/wq3iaUs6ZELRhLKhmURQ== 2024-05-26T22:51:45.076 INFO:teuthology.orchestra.run.smithi131.stdout: caps mds = "allow" 2024-05-26T22:51:45.076 INFO:teuthology.orchestra.run.smithi131.stdout: caps mon = "allow r" 2024-05-26T22:51:45.076 INFO:teuthology.orchestra.run.smithi131.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-05-26T22:51:45.076 INFO:teuthology.orchestra.run.smithi131.stderr:updated caps for client.0 2024-05-26T22:51:45.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:45.078+0000 7f67f3800640 1 -- 172.21.15.131:0/1110635319 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f67cc06be30 msgr2=0x7f67cc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:45.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:45.078+0000 7f67f3800640 1 --2- 172.21.15.131:0/1110635319 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f67cc06be30 0x7f67cc06e2f0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f67dc006fd0 tx=0x7f67dc008040 comp rx=0 tx=0).stop 2024-05-26T22:51:45.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:45.079+0000 7f67f3800640 1 -- 172.21.15.131:0/1110635319 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f67ec15e4e0 msgr2=0x7f67ec0c8280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:45.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:45.079+0000 7f67f3800640 1 --2- 172.21.15.131:0/1110635319 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f67ec15e4e0 0x7f67ec0c8280 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f67e400c410 tx=0x7f67e400c8d0 comp rx=0 tx=0).stop 2024-05-26T22:51:45.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:45.079+0000 7f67f3800640 1 -- 172.21.15.131:0/1110635319 shutdown_connections 2024-05-26T22:51:45.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:45.079+0000 7f67f3800640 1 --2- 172.21.15.131:0/1110635319 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f67cc06be30 0x7f67cc06e2f0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:45.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:45.079+0000 7f67f3800640 1 --2- 172.21.15.131:0/1110635319 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f67ec15e4e0 0x7f67ec0c8280 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:45.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:45.079+0000 7f67f3800640 1 --2- 172.21.15.131:0/1110635319 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f67ec0d2330 0x7f67ec152980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:45.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:45.079+0000 7f67f3800640 1 --2- 172.21.15.131:0/1110635319 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f67ec0cf550 0x7f67ec152440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:45.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:45.079+0000 7f67f3800640 1 -- 172.21.15.131:0/1110635319 >> 172.21.15.131:0/1110635319 conn(0x7f67ec072e90 msgr2=0x7f67ec0b3cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:45.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:45.079+0000 7f67f3800640 1 -- 172.21.15.131:0/1110635319 shutdown_connections 2024-05-26T22:51:45.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:45.079+0000 7f67f3800640 1 -- 172.21.15.131:0/1110635319 wait complete. 2024-05-26T22:51:45.098 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-26T22:51:45.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:45.343+0000 7f5192a00640 1 Processor -- start 2024-05-26T22:51:45.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:45.343+0000 7f5192a00640 1 -- start start 2024-05-26T22:51:45.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:45.344+0000 7f5192a00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f518c156750 0x7f518c15fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:45.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:45.344+0000 7f5192a00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f518c160620 con 0x7f518c0cc7f0 2024-05-26T22:51:45.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:45.344+0000 7f5192a00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f518c160900 con 0x7f518c06e8c0 2024-05-26T22:51:45.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:45.344+0000 7f5192a00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f518c15d770 con 0x7f518c156750 2024-05-26T22:51:45.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:45.345+0000 7f5192000640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f518c0cc7f0 0x7f518c0cdc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:37892/0 (socket says 172.21.15.131:37892) 2024-05-26T22:51:45.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:45.345+0000 7f5192000640 1 -- 172.21.15.131:0/1411641049 learned_addr learned my addr 172.21.15.131:0/1411641049 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:51:45.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:45.345+0000 7f5191600640 1 --2- 172.21.15.131:0/1411641049 >> v2:172.21.15.187:3300/0 conn(0x7f518c156750 0x7f518c15fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:45.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:45.345+0000 7f518be00640 1 -- 172.21.15.131:0/1411641049 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3725972002 0 0) 0x7f518c160620 con 0x7f518c0cc7f0 2024-05-26T22:51:45.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:45.345+0000 7f518be00640 1 -- 172.21.15.131:0/1411641049 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5170003520 con 0x7f518c0cc7f0 2024-05-26T22:51:45.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:45.345+0000 7f518be00640 1 -- 172.21.15.131:0/1411641049 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2240132600 0 0) 0x7f5170003520 con 0x7f518c0cc7f0 2024-05-26T22:51:45.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:45.345+0000 7f518be00640 1 -- 172.21.15.131:0/1411641049 >> v2:172.21.15.187:3300/0 conn(0x7f518c156750 msgr2=0x7f518c15fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:45.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:45.345+0000 7f518be00640 1 --2- 172.21.15.131:0/1411641049 >> v2:172.21.15.187:3300/0 conn(0x7f518c156750 0x7f518c15fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:45.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:45.346+0000 7f518be00640 1 -- 172.21.15.131:0/1411641049 >> v1:172.21.15.187:6789/0 conn(0x7f518c06e8c0 legacy=0x7f518c15d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:45.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:45.346+0000 7f518be00640 1 -- 172.21.15.131:0/1411641049 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f518c160be0 con 0x7f518c0cc7f0 2024-05-26T22:51:45.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:45.346+0000 7f518be00640 1 -- 172.21.15.131:0/1411641049 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7f5184001670 con 0x7f518c0cc7f0 2024-05-26T22:51:45.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:45.346+0000 7f518be00640 1 -- 172.21.15.131:0/1411641049 >> v1:172.21.15.131:6789/0 conn(0x7f518c0cc7f0 legacy=0x7f518c0cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:45.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:45.346+0000 7f518be00640 1 --2- 172.21.15.131:0/1411641049 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5170003a00 0x7f5170003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:45.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:45.346+0000 7f518be00640 1 --2- 172.21.15.131:0/1411641049 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5170004800 0x7f5170004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:45.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:45.346+0000 7f518be00640 1 --2- 172.21.15.131:0/1411641049 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f51700051c0 0x7f517000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:45.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:45.346+0000 7f518be00640 1 -- 172.21.15.131:0/1411641049 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f518c15d770 con 0x7f5170003a00 2024-05-26T22:51:45.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:45.346+0000 7f518be00640 1 -- 172.21.15.131:0/1411641049 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f517000c0c0 con 0x7f5170004800 2024-05-26T22:51:45.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:45.346+0000 7f518be00640 1 -- 172.21.15.131:0/1411641049 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f517000c2c0 con 0x7f51700051c0 2024-05-26T22:51:45.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:45.346+0000 7f5191600640 1 --2- 172.21.15.131:0/1411641049 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5170003a00 0x7f5170003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:45.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:45.346+0000 7f5190c00640 1 --2- 172.21.15.131:0/1411641049 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5170004800 0x7f5170004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:45.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:45.346+0000 7f5191600640 1 -- 172.21.15.131:0/1411641049 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f51700051c0 msgr2=0x7f517000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:45.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:45.346+0000 7f5191600640 1 --2- 172.21.15.131:0/1411641049 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f51700051c0 0x7f517000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:45.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:45.346+0000 7f5191600640 1 -- 172.21.15.131:0/1411641049 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5170004800 msgr2=0x7f5170004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:45.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:45.346+0000 7f5191600640 1 --2- 172.21.15.131:0/1411641049 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5170004800 0x7f5170004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:45.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:45.346+0000 7f5191600640 1 -- 172.21.15.131:0/1411641049 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5170003520 con 0x7f5170003a00 2024-05-26T22:51:45.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:45.346+0000 7f5192000640 1 -- 172.21.15.131:0/1411641049 reap_dead start 2024-05-26T22:51:45.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:45.347+0000 7f5191600640 1 --2- 172.21.15.131:0/1411641049 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5170003a00 0x7f5170003e20 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f517c009fc0 tx=0x7f5170005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=510dc702fad3699b server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:45.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:45.348+0000 7f518be00640 1 -- 172.21.15.131:0/1411641049 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f517c034020 con 0x7f5170003a00 2024-05-26T22:51:45.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:45.348+0000 7f518be00640 1 -- 172.21.15.131:0/1411641049 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f517c0305f0 con 0x7f5170003a00 2024-05-26T22:51:45.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:45.348+0000 7f518be00640 1 -- 172.21.15.131:0/1411641049 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f517c009c10 con 0x7f5170003a00 2024-05-26T22:51:45.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:45.348+0000 7f5192a00640 1 -- 172.21.15.131:0/1411641049 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5170003a00 msgr2=0x7f5170003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:45.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:45.348+0000 7f5192a00640 1 --2- 172.21.15.131:0/1411641049 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5170003a00 0x7f5170003e20 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f517c009fc0 tx=0x7f5170005830 comp rx=0 tx=0).stop 2024-05-26T22:51:45.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:45.348+0000 7f5192a00640 1 -- 172.21.15.131:0/1411641049 shutdown_connections 2024-05-26T22:51:45.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:45.348+0000 7f5192a00640 1 --2- 172.21.15.131:0/1411641049 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5170003a00 0x7f5170003e20 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:45.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:45.348+0000 7f5192a00640 1 -- 172.21.15.131:0/1411641049 >> 172.21.15.131:0/1411641049 conn(0x7f518c0cff40 msgr2=0x7f518c0d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:45.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:45.349+0000 7f5192a00640 1 -- 172.21.15.131:0/1411641049 shutdown_connections 2024-05-26T22:51:45.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:45.349+0000 7f5192a00640 1 -- 172.21.15.131:0/1411641049 wait complete. 2024-05-26T22:51:45.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:45.350+0000 7f5192a00640 1 Processor -- start 2024-05-26T22:51:45.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:45.350+0000 7f5192a00640 1 -- start start 2024-05-26T22:51:45.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:45.351+0000 7f5192a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5170003a00 0x7f518c0afa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:45.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:45.351+0000 7f5192a00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f518c145120 0x7f518c1f7a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:45.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:45.351+0000 7f5192a00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f518c1f7fa0 0x7f518c157bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:45.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:45.351+0000 7f5192a00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f518c156b30 con 0x7f5170003a00 2024-05-26T22:51:45.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:45.351+0000 7f5192a00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f518c15cbd0 con 0x7f518c145120 2024-05-26T22:51:45.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:45.351+0000 7f5192a00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f518c0ccc60 con 0x7f518c1f7fa0 2024-05-26T22:51:45.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:45.351+0000 7f5191600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5170003a00 0x7f518c0afa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:45.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:45.352+0000 7f5191600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5170003a00 0x7f518c0afa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47746/0 (socket says 172.21.15.131:47746) 2024-05-26T22:51:45.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:45.352+0000 7f5191600640 1 -- 172.21.15.131:0/4075439418 learned_addr learned my addr 172.21.15.131:0/4075439418 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:45.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:45.352+0000 7f5191600640 1 -- 172.21.15.131:0/4075439418 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f518c1f7fa0 msgr2=0x7f518c157bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:45.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:45.352+0000 7f5191600640 1 --2- 172.21.15.131:0/4075439418 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f518c1f7fa0 0x7f518c157bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:45.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:45.352+0000 7f5191600640 1 -- 172.21.15.131:0/4075439418 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f518c145120 msgr2=0x7f518c1f7a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:45.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:45.352+0000 7f5191600640 1 --2- 172.21.15.131:0/4075439418 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f518c145120 0x7f518c1f7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:45.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:45.352+0000 7f5191600640 1 -- 172.21.15.131:0/4075439418 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f518c1458f0 con 0x7f5170003a00 2024-05-26T22:51:45.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:45.352+0000 7f5191600640 1 --2- 172.21.15.131:0/4075439418 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5170003a00 0x7f518c0afa80 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f517c033040 tx=0x7f517c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:45.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:45.352+0000 7f5189600640 1 -- 172.21.15.131:0/4075439418 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f517c040070 con 0x7f5170003a00 2024-05-26T22:51:45.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:45.353+0000 7f5189600640 1 -- 172.21.15.131:0/4075439418 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f517c003990 con 0x7f5170003a00 2024-05-26T22:51:45.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:45.353+0000 7f5189600640 1 -- 172.21.15.131:0/4075439418 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f517c034040 con 0x7f5170003a00 2024-05-26T22:51:45.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:45.353+0000 7f5192a00640 1 -- 172.21.15.131:0/4075439418 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f518c145bd0 con 0x7f5170003a00 2024-05-26T22:51:45.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:45.353+0000 7f5192a00640 1 -- 172.21.15.131:0/4075439418 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f518c146190 con 0x7f5170003a00 2024-05-26T22:51:45.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:45.354+0000 7f5192a00640 1 -- 172.21.15.131:0/4075439418 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f518c1f83c0 con 0x7f5170003a00 2024-05-26T22:51:45.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:45.355+0000 7f5189600640 1 -- 172.21.15.131:0/4075439418 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f517c037020 con 0x7f5170003a00 2024-05-26T22:51:45.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:45.356+0000 7f5189600640 1 --2- 172.21.15.131:0/4075439418 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f516c06be30 0x7f516c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:45.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:45.356+0000 7f5189600640 1 -- 172.21.15.131:0/4075439418 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f517c0ab940 con 0x7f5170003a00 2024-05-26T22:51:45.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:45.356+0000 7f5190c00640 1 --2- 172.21.15.131:0/4075439418 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f516c06be30 0x7f516c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:45.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:45.357+0000 7f5190c00640 1 --2- 172.21.15.131:0/4075439418 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f516c06be30 0x7f516c06e2f0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f5178000c00 tx=0x7f5178009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:45.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:45.360+0000 7f5189600640 1 -- 172.21.15.131:0/4075439418 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f517c07a2e0 con 0x7f5170003a00 2024-05-26T22:51:45.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:45.604+0000 7f5192a00640 1 -- 172.21.15.131:0/4075439418 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f518c0694e0 con 0x7f5170003a00 2024-05-26T22:51:45.604 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:45.605+0000 7f5189600640 1 -- 172.21.15.131:0/4075439418 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) ==== 94+0+5190 (secure 0 0 0) 0x7f517c079be0 con 0x7f5170003a00 2024-05-26T22:51:45.605 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:51:45.605 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":49,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4832,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/1327386245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":1327386245},{"type":"v1","addr":"172.21.15.187:6835","nonce":1327386245}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":44},{"gid":6378,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6836/2737657374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6835","nonce":2737657374},{"type":"v1","addr":"172.21.15.131:6836","nonce":2737657374}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43},{"gid":6381,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/640544945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":640544945},{"type":"v1","addr":"172.21.15.131:6837","nonce":640544945}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-26T22:51:39.114644+0000","modified":"2024-05-26T22:51:42.152944+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6390},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6390":{"gid":6390,"name":"d","rank":0,"incarnation":45,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6833/3300872633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3300872633},{"type":"v1","addr":"172.21.15.187:6833","nonce":3300872633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6390,"qdb_cluster":[6390]},"id":4}]} 2024-05-26T22:51:45.605 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 49 2024-05-26T22:51:45.607 INFO:tasks.ceph.mds.d.smithi187.stderr:2024-05-26T22:51:45.607+0000 7fbf69e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-26T22:51:45.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:45.607+0000 7f5192a00640 1 -- 172.21.15.131:0/4075439418 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f516c06be30 msgr2=0x7f516c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:45.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:45.607+0000 7f5192a00640 1 --2- 172.21.15.131:0/4075439418 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f516c06be30 0x7f516c06e2f0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f5178000c00 tx=0x7f5178009040 comp rx=0 tx=0).stop 2024-05-26T22:51:45.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:45.607+0000 7f5192a00640 1 -- 172.21.15.131:0/4075439418 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5170003a00 msgr2=0x7f518c0afa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:45.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:45.608+0000 7f5192a00640 1 --2- 172.21.15.131:0/4075439418 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5170003a00 0x7f518c0afa80 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f517c033040 tx=0x7f517c009750 comp rx=0 tx=0).stop 2024-05-26T22:51:45.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:45.608+0000 7f5192a00640 1 -- 172.21.15.131:0/4075439418 shutdown_connections 2024-05-26T22:51:45.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:45.608+0000 7f5192a00640 1 --2- 172.21.15.131:0/4075439418 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f518c1f7fa0 0x7f518c157bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:45.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:45.608+0000 7f5192a00640 1 --2- 172.21.15.131:0/4075439418 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f516c06be30 0x7f516c06e2f0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:45.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:45.608+0000 7f5192a00640 1 --2- 172.21.15.131:0/4075439418 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f518c145120 0x7f518c1f7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:45.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:45.608+0000 7f5192a00640 1 --2- 172.21.15.131:0/4075439418 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5170003a00 0x7f518c0afa80 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:45.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:45.608+0000 7f5192a00640 1 -- 172.21.15.131:0/4075439418 >> 172.21.15.131:0/4075439418 conn(0x7f518c0cff40 msgr2=0x7f518c0b4b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:45.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:45.608+0000 7f5192a00640 1 -- 172.21.15.131:0/4075439418 shutdown_connections 2024-05-26T22:51:45.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:45.608+0000 7f5192a00640 1 -- 172.21.15.131:0/4075439418 wait complete. 2024-05-26T22:51:46.029 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 49, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-26T22:51:39.114644+0000', 'modified': '2024-05-26T22:51:42.152944+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6390}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6390': {'gid': 6390, 'name': 'd', 'rank': 0, 'incarnation': 45, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.187:6833/3300872633', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.187:6832', 'nonce': 3300872633}, {'type': 'v1', 'addr': '172.21.15.187:6833', 'nonce': 3300872633}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6390, 'qdb_cluster': [6390]} 2024-05-26T22:51:46.029 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-26T22:51:46.030 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-05-26T22:51:46.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.244+0000 7f345fe00640 1 Processor -- start 2024-05-26T22:51:46.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.244+0000 7f345fe00640 1 -- start start 2024-05-26T22:51:46.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.245+0000 7f345fe00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f3460145a10 0x7f3460072180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:46.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.245+0000 7f345fe00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f34600712d0 0x7f346006d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:46.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.245+0000 7f345fe00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3460072d60 con 0x7f346006d790 2024-05-26T22:51:46.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.245+0000 7f345fe00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f346015c760 con 0x7f3460145a10 2024-05-26T22:51:46.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.245+0000 7f345fe00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3460072a80 con 0x7f34600712d0 2024-05-26T22:51:46.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.246+0000 7f345ea00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f3460145a10 0x7f3460072180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:46.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.246+0000 7f345ea00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f3460145a10 0x7f3460072180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47756/0 (socket says 172.21.15.131:47756) 2024-05-26T22:51:46.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.246+0000 7f345ea00640 1 -- 172.21.15.131:0/3270514539 learned_addr learned my addr 172.21.15.131:0/3270514539 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:46.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.246+0000 7f3456000640 1 --2- 172.21.15.131:0/3270514539 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f34600712d0 0x7f346006d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:46.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.246+0000 7f345ea00640 1 -- 172.21.15.131:0/3270514539 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f34600712d0 msgr2=0x7f346006d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:46.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.246+0000 7f345ea00640 1 --2- 172.21.15.131:0/3270514539 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f34600712d0 0x7f346006d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:46.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.246+0000 7f345ea00640 1 -- 172.21.15.131:0/3270514539 >> v1:172.21.15.187:6789/0 conn(0x7f346006d790 legacy=0x7f346015fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:46.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.246+0000 7f345ea00640 1 -- 172.21.15.131:0/3270514539 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34600ceab0 con 0x7f3460145a10 2024-05-26T22:51:46.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.246+0000 7f3456000640 1 --2- 172.21.15.131:0/3270514539 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f34600712d0 0x7f346006d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:51:46.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.247+0000 7f345ea00640 1 --2- 172.21.15.131:0/3270514539 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3460145a10 0x7f3460072180 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f3450009e50 tx=0x7f345002a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79f73fe8472e380f server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:46.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.247+0000 7f345e000640 1 -- 172.21.15.131:0/3270514539 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3450036070 con 0x7f3460145a10 2024-05-26T22:51:46.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.248+0000 7f345e000640 1 -- 172.21.15.131:0/3270514539 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3450031600 con 0x7f3460145a10 2024-05-26T22:51:46.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.248+0000 7f345e000640 1 -- 172.21.15.131:0/3270514539 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34500319d0 con 0x7f3460145a10 2024-05-26T22:51:46.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.248+0000 7f345fe00640 1 -- 172.21.15.131:0/3270514539 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3460145a10 msgr2=0x7f3460072180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:46.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.248+0000 7f345fe00640 1 --2- 172.21.15.131:0/3270514539 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3460145a10 0x7f3460072180 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f3450009e50 tx=0x7f345002a6d0 comp rx=0 tx=0).stop 2024-05-26T22:51:46.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.248+0000 7f345fe00640 1 -- 172.21.15.131:0/3270514539 shutdown_connections 2024-05-26T22:51:46.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.248+0000 7f345fe00640 1 --2- 172.21.15.131:0/3270514539 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f34600712d0 0x7f346006d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:46.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.248+0000 7f345fe00640 1 --2- 172.21.15.131:0/3270514539 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3460145a10 0x7f3460072180 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:46.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.248+0000 7f345fe00640 1 -- 172.21.15.131:0/3270514539 >> 172.21.15.131:0/3270514539 conn(0x7f34600b5660 msgr2=0x7f34600b5a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:46.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.248+0000 7f345fe00640 1 -- 172.21.15.131:0/3270514539 shutdown_connections 2024-05-26T22:51:46.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.248+0000 7f345fe00640 1 -- 172.21.15.131:0/3270514539 wait complete. 2024-05-26T22:51:46.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.250+0000 7f345fe00640 1 Processor -- start 2024-05-26T22:51:46.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.250+0000 7f345fe00640 1 -- start start 2024-05-26T22:51:46.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.251+0000 7f345fe00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f346006d790 0x7f34601f3280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:46.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.251+0000 7f345fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f34600712d0 0x7f34601f7060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:46.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.251+0000 7f345fe00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3460145a10 0x7f346015fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:46.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.251+0000 7f345fe00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f3460145df0 con 0x7f34600712d0 2024-05-26T22:51:46.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.251+0000 7f345fe00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f34600cf070 con 0x7f346006d790 2024-05-26T22:51:46.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.251+0000 7f345fe00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f34600726c0 con 0x7f3460145a10 2024-05-26T22:51:46.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.252+0000 7f3456000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f34600712d0 0x7f34601f7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:46.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.252+0000 7f3456000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f34600712d0 0x7f34601f7060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47760/0 (socket says 172.21.15.131:47760) 2024-05-26T22:51:46.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.252+0000 7f3456000640 1 -- 172.21.15.131:0/3544541450 learned_addr learned my addr 172.21.15.131:0/3544541450 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:46.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.252+0000 7f345ea00640 1 --2- 172.21.15.131:0/3544541450 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f346006d790 0x7f34601f3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:46.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.252+0000 7f345f400640 1 --2- 172.21.15.131:0/3544541450 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3460145a10 0x7f346015fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:46.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.252+0000 7f3456000640 1 -- 172.21.15.131:0/3544541450 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3460145a10 msgr2=0x7f346015fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:46.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.252+0000 7f3456000640 1 --2- 172.21.15.131:0/3544541450 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3460145a10 0x7f346015fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:46.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.252+0000 7f3456000640 1 -- 172.21.15.131:0/3544541450 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f346006d790 msgr2=0x7f34601f3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:46.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.252+0000 7f3456000640 1 --2- 172.21.15.131:0/3544541450 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f346006d790 0x7f34601f3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:46.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.252+0000 7f3456000640 1 -- 172.21.15.131:0/3544541450 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34601f7b20 con 0x7f34600712d0 2024-05-26T22:51:46.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.252+0000 7f345ea00640 1 --2- 172.21.15.131:0/3544541450 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f346006d790 0x7f34601f3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:51:46.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.252+0000 7f3456000640 1 --2- 172.21.15.131:0/3544541450 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f34600712d0 0x7f34601f7060 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f344800c7f0 tx=0x7f344800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:46.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.253+0000 7f3457400640 1 -- 172.21.15.131:0/3544541450 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3448015070 con 0x7f34600712d0 2024-05-26T22:51:46.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.253+0000 7f3457400640 1 -- 172.21.15.131:0/3544541450 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3448007a30 con 0x7f34600712d0 2024-05-26T22:51:46.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.253+0000 7f3457400640 1 -- 172.21.15.131:0/3544541450 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3448007e00 con 0x7f34600712d0 2024-05-26T22:51:46.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.253+0000 7f345fe00640 1 -- 172.21.15.131:0/3544541450 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34601f7e00 con 0x7f34600712d0 2024-05-26T22:51:46.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.253+0000 7f345fe00640 1 -- 172.21.15.131:0/3544541450 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3460141ef0 con 0x7f34600712d0 2024-05-26T22:51:46.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.256+0000 7f3457400640 1 -- 172.21.15.131:0/3544541450 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3448018070 con 0x7f34600712d0 2024-05-26T22:51:46.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.256+0000 7f3457400640 1 --2- 172.21.15.131:0/3544541450 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f343c06be30 0x7f343c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:46.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.257+0000 7f3457400640 1 -- 172.21.15.131:0/3544541450 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f344808ae50 con 0x7f34600712d0 2024-05-26T22:51:46.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.257+0000 7f345ea00640 1 --2- 172.21.15.131:0/3544541450 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f343c06be30 0x7f343c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:46.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.257+0000 7f3468a82740 1 Processor -- start 2024-05-26T22:51:46.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.257+0000 7f3468a82740 1 -- start start 2024-05-26T22:51:46.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.257+0000 7f345ea00640 1 --2- 172.21.15.131:0/3544541450 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f343c06be30 0x7f343c06e2f0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f3450033000 tx=0x7f3450009000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:46.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.258+0000 7f3468a82740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bb59347e50 0x55bb592faa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:46.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.258+0000 7f3468a82740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bb5932af70 0x55bb5932a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:46.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.258+0000 7f3468a82740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55bb5932a830 0x55bb592ff640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:46.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.258+0000 7f3468a82740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55bb59262610 con 0x55bb5932af70 2024-05-26T22:51:46.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.258+0000 7f3468a82740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55bb5920a5c0 con 0x55bb59347e50 2024-05-26T22:51:46.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.258+0000 7f3468a82740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x55bb5925e5d0 con 0x55bb5932a830 2024-05-26T22:51:46.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.258+0000 7f3456000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bb5932af70 0x55bb5932a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:46.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.258+0000 7f3456000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bb5932af70 0x55bb5932a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47764/0 (socket says 172.21.15.131:47764) 2024-05-26T22:51:46.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.258+0000 7f3456000640 1 -- 172.21.15.131:0/3094943905 learned_addr learned my addr 172.21.15.131:0/3094943905 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:46.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.259+0000 7f345ea00640 1 --2- 172.21.15.131:0/3094943905 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bb59347e50 0x55bb592faa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:46.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.259+0000 7f345f400640 1 --2- 172.21.15.131:0/3094943905 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55bb5932a830 0x55bb592ff640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:46.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.259+0000 7f3456000640 1 -- 172.21.15.131:0/3094943905 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55bb5932a830 msgr2=0x55bb592ff640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:46.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.259+0000 7f3456000640 1 --2- 172.21.15.131:0/3094943905 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55bb5932a830 0x55bb592ff640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:46.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.259+0000 7f3456000640 1 -- 172.21.15.131:0/3094943905 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bb59347e50 msgr2=0x55bb592faa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:46.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.259+0000 7f3456000640 1 --2- 172.21.15.131:0/3094943905 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bb59347e50 0x55bb592faa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:46.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.259+0000 7f3456000640 1 -- 172.21.15.131:0/3094943905 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb58f762d0 con 0x55bb5932af70 2024-05-26T22:51:46.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.259+0000 7f345ea00640 1 --2- 172.21.15.131:0/3094943905 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bb59347e50 0x55bb592faa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:51:46.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.259+0000 7f345f400640 1 --2- 172.21.15.131:0/3094943905 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55bb5932a830 0x55bb592ff640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:51:46.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.259+0000 7f3456000640 1 --2- 172.21.15.131:0/3094943905 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bb5932af70 0x55bb5932a2f0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f34480893a0 tx=0x7f3448059100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6c6043da96cc330 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:46.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.259+0000 7f344f400640 1 -- 172.21.15.131:0/3094943905 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f344801f040 con 0x55bb5932af70 2024-05-26T22:51:46.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.259+0000 7f344f400640 1 -- 172.21.15.131:0/3094943905 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3448071120 con 0x55bb5932af70 2024-05-26T22:51:46.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.259+0000 7f344f400640 1 -- 172.21.15.131:0/3094943905 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f344805d170 con 0x55bb5932af70 2024-05-26T22:51:46.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.260+0000 7f3468a82740 1 -- 172.21.15.131:0/3094943905 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bb5932af70 msgr2=0x55bb5932a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:46.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.260+0000 7f3468a82740 1 --2- 172.21.15.131:0/3094943905 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bb5932af70 0x55bb5932a2f0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f34480893a0 tx=0x7f3448059100 comp rx=0 tx=0).stop 2024-05-26T22:51:46.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.260+0000 7f3468a82740 1 -- 172.21.15.131:0/3094943905 shutdown_connections 2024-05-26T22:51:46.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.260+0000 7f3468a82740 1 --2- 172.21.15.131:0/3094943905 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55bb5932a830 0x55bb592ff640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:46.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.260+0000 7f3468a82740 1 --2- 172.21.15.131:0/3094943905 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bb5932af70 0x55bb5932a2f0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:46.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.260+0000 7f3468a82740 1 --2- 172.21.15.131:0/3094943905 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bb59347e50 0x55bb592faa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:46.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.260+0000 7f3468a82740 1 -- 172.21.15.131:0/3094943905 >> 172.21.15.131:0/3094943905 conn(0x55bb592fcce0 msgr2=0x55bb593452f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:46.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.260+0000 7f3468a82740 1 -- 172.21.15.131:0/3094943905 shutdown_connections 2024-05-26T22:51:46.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.260+0000 7f3468a82740 1 -- 172.21.15.131:0/3094943905 wait complete. 2024-05-26T22:51:46.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.261+0000 7f3468a82740 1 -- start start 2024-05-26T22:51:46.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.261+0000 7f3468a82740 1 Processor -- start 2024-05-26T22:51:46.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.262+0000 7f3468a82740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55bb5932af70 0x55bb5932cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:46.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.262+0000 7f3468a82740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bb59347e50 0x55bb59326190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:46.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.262+0000 7f3468a82740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bb5932e1d0 0x55bb5932b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:46.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.262+0000 7f3468a82740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55bb592fec00 con 0x55bb5932e1d0 2024-05-26T22:51:46.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.262+0000 7f3468a82740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55bb5921d910 con 0x55bb59347e50 2024-05-26T22:51:46.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.262+0000 7f3468a82740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x55bb592fdff0 con 0x55bb5932af70 2024-05-26T22:51:46.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.262+0000 7f3468a82740 10 client.0 _ll_register_callbacks cb 0x55bb59259410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-26T22:51:46.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.262+0000 7f345f400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bb5932e1d0 0x55bb5932b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:46.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.263+0000 7f345f400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bb5932e1d0 0x55bb5932b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47768/0 (socket says 172.21.15.131:47768) 2024-05-26T22:51:46.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.263+0000 7f345f400640 1 -- 172.21.15.131:0/1053680078 learned_addr learned my addr 172.21.15.131:0/1053680078 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:46.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.263+0000 7f345ea00640 1 --2- 172.21.15.131:0/1053680078 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55bb5932af70 0x55bb5932cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:46.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.263+0000 7f345f400640 1 -- 172.21.15.131:0/1053680078 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55bb5932af70 msgr2=0x55bb5932cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:46.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.263+0000 7f3456000640 1 --2- 172.21.15.131:0/1053680078 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bb59347e50 0x55bb59326190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:46.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.263+0000 7f345f400640 1 --2- 172.21.15.131:0/1053680078 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55bb5932af70 0x55bb5932cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:46.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.263+0000 7f345f400640 1 -- 172.21.15.131:0/1053680078 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bb59347e50 msgr2=0x55bb59326190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:46.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.263+0000 7f345f400640 1 --2- 172.21.15.131:0/1053680078 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bb59347e50 0x55bb59326190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:46.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.263+0000 7f345f400640 1 -- 172.21.15.131:0/1053680078 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb590406b0 con 0x55bb5932e1d0 2024-05-26T22:51:46.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.263+0000 7f345ea00640 1 --2- 172.21.15.131:0/1053680078 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55bb5932af70 0x55bb5932cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:51:46.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.263+0000 7f345f400640 1 --2- 172.21.15.131:0/1053680078 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bb5932e1d0 0x55bb5932b460 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f3458004460 tx=0x7f345800e590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e554ea99be9c1a3e server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:46.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.263+0000 7f3443e00640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-26T22:51:46.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.263+0000 7f3443e00640 1 -- 172.21.15.131:0/1053680078 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3458005230 con 0x55bb5932e1d0 2024-05-26T22:51:46.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.264+0000 7f3443e00640 1 -- 172.21.15.131:0/1053680078 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f34580053d0 con 0x55bb5932e1d0 2024-05-26T22:51:46.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.264+0000 7f3468a82740 1 -- 172.21.15.131:0/1053680078 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3448071120 con 0x55bb5932e1d0 2024-05-26T22:51:46.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.264+0000 7f3443e00640 1 -- 172.21.15.131:0/1053680078 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f3458020050 con 0x55bb5932e1d0 2024-05-26T22:51:46.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.264+0000 7f3468a82740 10 client.6531 fetch_fsmap learned FSMap version 49 2024-05-26T22:51:46.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.264+0000 7f3468a82740 1 -- 172.21.15.131:0/1053680078 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=2+}) -- 0x55bb5932bf50 con 0x55bb5932e1d0 2024-05-26T22:51:46.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.264+0000 7f3443e00640 1 -- 172.21.15.131:0/1053680078 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3448071120 con 0x55bb5932e1d0 2024-05-26T22:51:46.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.264+0000 7f3443e00640 1 -- 172.21.15.131:0/1053680078 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 49) ==== 3494+0+0 (secure 0 0 0) 0x7f3458023020 con 0x55bb5932e1d0 2024-05-26T22:51:46.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.264+0000 7f3468a82740 10 client.6531 fetch_fsmap finished waiting for FSMap version 49 2024-05-26T22:51:46.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.264+0000 7f3468a82740 10 client.6531 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-26T22:51:46.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.264+0000 7f3468a82740 20 client.6531 populate_metadata read hostname 'smithi131' 2024-05-26T22:51:46.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.264+0000 7f3468a82740 1 --2- 172.21.15.131:0/1053680078 >> [v2:172.21.15.187:6832/3300872633,v1:172.21.15.187:6833/3300872633] conn(0x55bb5933c7b0 0x55bb593640c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:46.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.264+0000 7f3468a82740 4 client.6531 mds_command: new command op to 6390 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-26T22:51:46.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.264+0000 7f3468a82740 1 -- 172.21.15.131:0/1053680078 --> [v2:172.21.15.187:6832/3300872633,v1:172.21.15.187:6833/3300872633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bb5926fd40 con 0x55bb5933c7b0 2024-05-26T22:51:46.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.265+0000 7f345ea00640 1 --2- 172.21.15.131:0/1053680078 >> [v2:172.21.15.187:6832/3300872633,v1:172.21.15.187:6833/3300872633] conn(0x55bb5933c7b0 0x55bb593640c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:46.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.266+0000 7f3443e00640 10 client.6531 ms_handle_connect on v2:172.21.15.187:6832/3300872633 2024-05-26T22:51:46.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.266+0000 7f345ea00640 1 --2- 172.21.15.131:0/1053680078 >> [v2:172.21.15.187:6832/3300872633,v1:172.21.15.187:6833/3300872633] conn(0x55bb5933c7b0 0x55bb593640c0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2eac8652c69c002b server_cookie=b1003a4047e29ae5 in_seq=0 out_seq=0 2024-05-26T22:51:46.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.268+0000 7f3443e00640 1 -- 172.21.15.131:0/1053680078 <== mds.0 v2:172.21.15.187:6832/3300872633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55bb5926fd40 con 0x55bb5933c7b0 2024-05-26T22:51:46.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.268+0000 7f3443e00640 10 client.6531 handle_command_reply: tid=0 2024-05-26T22:51:46.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.268+0000 7f3468a82740 1 client.6531 shutdown 2024-05-26T22:51:46.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.269+0000 7f3468a82740 1 -- 172.21.15.131:0/1053680078 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bb5932e1d0 msgr2=0x55bb5932b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:46.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.269+0000 7f3468a82740 1 --2- 172.21.15.131:0/1053680078 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bb5932e1d0 0x55bb5932b460 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f3458004460 tx=0x7f345800e590 comp rx=0 tx=0).stop 2024-05-26T22:51:46.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.269+0000 7f3468a82740 1 -- 172.21.15.131:0/1053680078 shutdown_connections 2024-05-26T22:51:46.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.269+0000 7f3468a82740 1 --2- 172.21.15.131:0/1053680078 >> [v2:172.21.15.187:6832/3300872633,v1:172.21.15.187:6833/3300872633] conn(0x55bb5933c7b0 0x55bb593640c0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:46.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.269+0000 7f3468a82740 1 --2- 172.21.15.131:0/1053680078 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bb5932e1d0 0x55bb5932b460 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:46.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.269+0000 7f3443e00640 0 client.6531 ms_handle_reset on v2:172.21.15.187:6832/3300872633 2024-05-26T22:51:46.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.269+0000 7f3468a82740 1 --2- 172.21.15.131:0/1053680078 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bb59347e50 0x55bb59326190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:46.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.269+0000 7f3468a82740 1 --2- 172.21.15.131:0/1053680078 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55bb5932af70 0x55bb5932cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:46.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.269+0000 7f3468a82740 1 -- 172.21.15.131:0/1053680078 >> 172.21.15.131:0/1053680078 conn(0x55bb5932a830 msgr2=0x55bb59348710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:46.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.269+0000 7f3468a82740 1 -- 172.21.15.131:0/1053680078 shutdown_connections 2024-05-26T22:51:46.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.269+0000 7f3468a82740 1 -- 172.21.15.131:0/1053680078 wait complete. 2024-05-26T22:51:46.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.269+0000 7f3468a82740 20 client.6531 trim_cache size 0 max 16384 2024-05-26T22:51:46.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.289+0000 7f3468a82740 1 Processor -- start 2024-05-26T22:51:46.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.289+0000 7f3468a82740 1 -- start start 2024-05-26T22:51:46.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.290+0000 7f3468a82740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bb5932af70 0x55bb59354e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:46.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.290+0000 7f3468a82740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55bb5932e1d0 0x55bb5932d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:46.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.290+0000 7f3468a82740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bb59347e50 0x55bb5932daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:46.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.290+0000 7f3468a82740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f34601ff6c0 con 0x55bb5932af70 2024-05-26T22:51:46.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.290+0000 7f3468a82740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3460142f60 con 0x55bb59347e50 2024-05-26T22:51:46.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.290+0000 7f3468a82740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f34601435b0 con 0x55bb5932e1d0 2024-05-26T22:51:46.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.290+0000 7f345ea00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bb5932af70 0x55bb59354e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:46.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.290+0000 7f345ea00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bb5932af70 0x55bb59354e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47778/0 (socket says 172.21.15.131:47778) 2024-05-26T22:51:46.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.290+0000 7f345ea00640 1 -- 172.21.15.131:0/2778942215 learned_addr learned my addr 172.21.15.131:0/2778942215 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:46.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.291+0000 7f345ea00640 1 -- 172.21.15.131:0/2778942215 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55bb5932e1d0 msgr2=0x55bb5932d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:46.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.291+0000 7f345ea00640 1 --2- 172.21.15.131:0/2778942215 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55bb5932e1d0 0x55bb5932d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:46.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.291+0000 7f345ea00640 1 -- 172.21.15.131:0/2778942215 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bb59347e50 msgr2=0x55bb5932daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:46.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.291+0000 7f345ea00640 1 --2- 172.21.15.131:0/2778942215 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bb59347e50 0x55bb5932daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:46.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.291+0000 7f345ea00640 1 -- 172.21.15.131:0/2778942215 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb592fbff0 con 0x55bb5932af70 2024-05-26T22:51:46.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.291+0000 7f345ea00640 1 --2- 172.21.15.131:0/2778942215 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bb5932af70 0x55bb59354e90 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f3450000f80 tx=0x7f345002e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c15cb6d956613c8e server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:46.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.291+0000 7f344ea00640 1 -- 172.21.15.131:0/2778942215 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3450036040 con 0x55bb5932af70 2024-05-26T22:51:46.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.291+0000 7f344ea00640 1 -- 172.21.15.131:0/2778942215 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f345003cc40 con 0x55bb5932af70 2024-05-26T22:51:46.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.291+0000 7f344ea00640 1 -- 172.21.15.131:0/2778942215 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f345004d150 con 0x55bb5932af70 2024-05-26T22:51:46.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.292+0000 7f3468a82740 1 -- 172.21.15.131:0/2778942215 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bb5932af70 msgr2=0x55bb59354e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:46.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.292+0000 7f3468a82740 1 --2- 172.21.15.131:0/2778942215 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bb5932af70 0x55bb59354e90 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f3450000f80 tx=0x7f345002e000 comp rx=0 tx=0).stop 2024-05-26T22:51:46.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.292+0000 7f3468a82740 1 -- 172.21.15.131:0/2778942215 shutdown_connections 2024-05-26T22:51:46.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.292+0000 7f3468a82740 1 --2- 172.21.15.131:0/2778942215 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55bb5932e1d0 0x55bb5932d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:46.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.292+0000 7f3468a82740 1 --2- 172.21.15.131:0/2778942215 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bb59347e50 0x55bb5932daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:46.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.292+0000 7f3468a82740 1 --2- 172.21.15.131:0/2778942215 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bb5932af70 0x55bb59354e90 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:46.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.292+0000 7f3468a82740 1 -- 172.21.15.131:0/2778942215 >> 172.21.15.131:0/2778942215 conn(0x55bb5932a830 msgr2=0x55bb59300000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:46.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.292+0000 7f3468a82740 1 -- 172.21.15.131:0/2778942215 shutdown_connections 2024-05-26T22:51:46.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.292+0000 7f3468a82740 1 -- 172.21.15.131:0/2778942215 wait complete. 2024-05-26T22:51:46.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.293+0000 7f3468a82740 1 -- start start 2024-05-26T22:51:46.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.293+0000 7f3468a82740 1 Processor -- start 2024-05-26T22:51:46.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.294+0000 7f3468a82740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bb5932af70 0x55bb593bda80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:46.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.294+0000 7f3468a82740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bb5932e1d0 0x55bb5932d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:46.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.294+0000 7f3468a82740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55bb59347e50 0x55bb5932da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:46.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.294+0000 7f3468a82740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55bb5932c840 con 0x55bb5932e1d0 2024-05-26T22:51:46.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.294+0000 7f3468a82740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55bb5932c230 con 0x55bb5932af70 2024-05-26T22:51:46.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.294+0000 7f3468a82740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x55bb59355e90 con 0x55bb59347e50 2024-05-26T22:51:46.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.294+0000 7f3468a82740 10 client.0 _ll_register_callbacks cb 0x7f3458007c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-26T22:51:46.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.294+0000 7f3456000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bb5932e1d0 0x55bb5932d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:46.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.294+0000 7f3456000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bb5932e1d0 0x55bb5932d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47784/0 (socket says 172.21.15.131:47784) 2024-05-26T22:51:46.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.294+0000 7f3456000640 1 -- 172.21.15.131:0/462232241 learned_addr learned my addr 172.21.15.131:0/462232241 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:46.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.294+0000 7f345ea00640 1 --2- 172.21.15.131:0/462232241 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bb5932af70 0x55bb593bda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:46.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.295+0000 7f3456000640 1 -- 172.21.15.131:0/462232241 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55bb59347e50 msgr2=0x55bb5932da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:46.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.295+0000 7f3456000640 1 --2- 172.21.15.131:0/462232241 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55bb59347e50 0x55bb5932da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:46.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.295+0000 7f3456000640 1 -- 172.21.15.131:0/462232241 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bb5932af70 msgr2=0x55bb593bda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:46.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.295+0000 7f3456000640 1 --2- 172.21.15.131:0/462232241 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bb5932af70 0x55bb593bda80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:46.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.295+0000 7f3456000640 1 -- 172.21.15.131:0/462232241 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb593273c0 con 0x55bb5932e1d0 2024-05-26T22:51:46.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.295+0000 7f3456000640 1 --2- 172.21.15.131:0/462232241 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bb5932e1d0 0x55bb5932d520 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f344800a120 tx=0x7f3448089250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bf257b916f3b3a1 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:46.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.295+0000 7f3443e00640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-26T22:51:46.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.295+0000 7f3443e00640 1 -- 172.21.15.131:0/462232241 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f344801f040 con 0x55bb5932e1d0 2024-05-26T22:51:46.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.295+0000 7f3443e00640 1 -- 172.21.15.131:0/462232241 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f3448058550 con 0x55bb5932e1d0 2024-05-26T22:51:46.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.295+0000 7f3468a82740 1 -- 172.21.15.131:0/462232241 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f345003cc40 con 0x55bb5932e1d0 2024-05-26T22:51:46.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.295+0000 7f3443e00640 1 -- 172.21.15.131:0/462232241 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f344800ce40 con 0x55bb5932e1d0 2024-05-26T22:51:46.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.295+0000 7f3468a82740 10 client.6537 fetch_fsmap learned FSMap version 49 2024-05-26T22:51:46.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.295+0000 7f3468a82740 1 -- 172.21.15.131:0/462232241 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=2+}) -- 0x55bb593c4b40 con 0x55bb5932e1d0 2024-05-26T22:51:46.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.295+0000 7f3443e00640 1 -- 172.21.15.131:0/462232241 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f345003cc40 con 0x55bb5932e1d0 2024-05-26T22:51:46.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.296+0000 7f3443e00640 1 -- 172.21.15.131:0/462232241 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 49) ==== 3494+0+0 (secure 0 0 0) 0x7f3448082ce0 con 0x55bb5932e1d0 2024-05-26T22:51:46.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.296+0000 7f3468a82740 10 client.6537 fetch_fsmap finished waiting for FSMap version 49 2024-05-26T22:51:46.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.296+0000 7f3468a82740 10 client.6537 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-26T22:51:46.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.296+0000 7f3468a82740 20 client.6537 populate_metadata read hostname 'smithi131' 2024-05-26T22:51:46.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.296+0000 7f3468a82740 1 --2- 172.21.15.131:0/462232241 >> [v2:172.21.15.187:6832/3300872633,v1:172.21.15.187:6833/3300872633] conn(0x55bb5935eda0 0x55bb5935f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:46.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.296+0000 7f3468a82740 4 client.6537 mds_command: new command op to 6390 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-26T22:51:46.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.296+0000 7f3468a82740 1 -- 172.21.15.131:0/462232241 --> [v2:172.21.15.187:6832/3300872633,v1:172.21.15.187:6833/3300872633] -- command(tid 0: {"prefix": "status"}) -- 0x55bb5926a090 con 0x55bb5935eda0 2024-05-26T22:51:46.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.296+0000 7f345ea00640 1 --2- 172.21.15.131:0/462232241 >> [v2:172.21.15.187:6832/3300872633,v1:172.21.15.187:6833/3300872633] conn(0x55bb5935eda0 0x55bb5935f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:46.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.297+0000 7f345ea00640 1 --2- 172.21.15.131:0/462232241 >> [v2:172.21.15.187:6832/3300872633,v1:172.21.15.187:6833/3300872633] conn(0x55bb5935eda0 0x55bb5935f180 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd412af98dd72996 server_cookie=d0d131bf68f2922 in_seq=0 out_seq=0 2024-05-26T22:51:46.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.297+0000 7f3443e00640 10 client.6537 ms_handle_connect on v2:172.21.15.187:6832/3300872633 2024-05-26T22:51:46.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.298+0000 7f3443e00640 1 -- 172.21.15.131:0/462232241 <== mds.0 v2:172.21.15.187:6832/3300872633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55bb5926a090 con 0x55bb5935eda0 2024-05-26T22:51:46.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.298+0000 7f3443e00640 10 client.6537 handle_command_reply: tid=0 2024-05-26T22:51:46.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.298+0000 7f3468a82740 1 client.6537 shutdown 2024-05-26T22:51:46.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.298+0000 7f3468a82740 1 -- 172.21.15.131:0/462232241 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bb5932e1d0 msgr2=0x55bb5932d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:46.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.298+0000 7f3468a82740 1 --2- 172.21.15.131:0/462232241 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bb5932e1d0 0x55bb5932d520 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f344800a120 tx=0x7f3448089250 comp rx=0 tx=0).stop 2024-05-26T22:51:46.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.298+0000 7f3468a82740 1 -- 172.21.15.131:0/462232241 shutdown_connections 2024-05-26T22:51:46.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.298+0000 7f3468a82740 1 --2- 172.21.15.131:0/462232241 >> [v2:172.21.15.187:6832/3300872633,v1:172.21.15.187:6833/3300872633] conn(0x55bb5935eda0 0x55bb5935f180 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:46.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.298+0000 7f3468a82740 1 --2- 172.21.15.131:0/462232241 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55bb59347e50 0x55bb5932da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:46.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.298+0000 7f3443e00640 0 client.6537 ms_handle_reset on v2:172.21.15.187:6832/3300872633 2024-05-26T22:51:46.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.298+0000 7f3468a82740 1 --2- 172.21.15.131:0/462232241 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bb5932e1d0 0x55bb5932d520 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:46.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.298+0000 7f3468a82740 1 --2- 172.21.15.131:0/462232241 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bb5932af70 0x55bb593bda80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:46.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.298+0000 7f3468a82740 1 -- 172.21.15.131:0/462232241 >> 172.21.15.131:0/462232241 conn(0x55bb5932a830 msgr2=0x55bb59326080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:46.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.298+0000 7f3468a82740 1 -- 172.21.15.131:0/462232241 shutdown_connections 2024-05-26T22:51:46.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.298+0000 7f3468a82740 1 -- 172.21.15.131:0/462232241 wait complete. 2024-05-26T22:51:46.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.298+0000 7f3468a82740 20 client.6537 trim_cache size 0 max 16384 2024-05-26T22:51:46.303 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-26T22:51:46.303 INFO:teuthology.orchestra.run.smithi131.stdout: "cluster_fsid": "46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f", 2024-05-26T22:51:46.303 INFO:teuthology.orchestra.run.smithi131.stdout: "whoami": 0, 2024-05-26T22:51:46.303 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 6390, 2024-05-26T22:51:46.303 INFO:teuthology.orchestra.run.smithi131.stdout: "want_state": "up:active", 2024-05-26T22:51:46.303 INFO:teuthology.orchestra.run.smithi131.stdout: "state": "up:active", 2024-05-26T22:51:46.303 INFO:teuthology.orchestra.run.smithi131.stdout: "fs_name": "cephfs", 2024-05-26T22:51:46.303 INFO:teuthology.orchestra.run.smithi131.stdout: "rank_uptime": 6.6299650469999998, 2024-05-26T22:51:46.303 INFO:teuthology.orchestra.run.smithi131.stdout: "mdsmap_epoch": 49, 2024-05-26T22:51:46.304 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch": 58, 2024-05-26T22:51:46.304 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch_barrier": 58, 2024-05-26T22:51:46.304 INFO:teuthology.orchestra.run.smithi131.stdout: "uptime": 12.124845607999999 2024-05-26T22:51:46.304 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-26T22:51:46.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.299+0000 7f345fe00640 1 -- 172.21.15.131:0/3544541450 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f343c06be30 msgr2=0x7f343c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:46.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.299+0000 7f345fe00640 1 --2- 172.21.15.131:0/3544541450 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f343c06be30 0x7f343c06e2f0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f3450033000 tx=0x7f3450009000 comp rx=0 tx=0).stop 2024-05-26T22:51:46.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.299+0000 7f345fe00640 1 -- 172.21.15.131:0/3544541450 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f34600712d0 msgr2=0x7f34601f7060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:46.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.299+0000 7f345fe00640 1 --2- 172.21.15.131:0/3544541450 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f34600712d0 0x7f34601f7060 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f344800c7f0 tx=0x7f344800cbf0 comp rx=0 tx=0).stop 2024-05-26T22:51:46.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.299+0000 7f345fe00640 1 -- 172.21.15.131:0/3544541450 shutdown_connections 2024-05-26T22:51:46.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.299+0000 7f345fe00640 1 --2- 172.21.15.131:0/3544541450 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3460145a10 0x7f346015fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:46.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.299+0000 7f345fe00640 1 --2- 172.21.15.131:0/3544541450 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f343c06be30 0x7f343c06e2f0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:46.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.299+0000 7f345fe00640 1 --2- 172.21.15.131:0/3544541450 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f34600712d0 0x7f34601f7060 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:46.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.299+0000 7f345fe00640 1 --2- 172.21.15.131:0/3544541450 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f346006d790 0x7f34601f3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:46.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.299+0000 7f345fe00640 1 -- 172.21.15.131:0/3544541450 >> 172.21.15.131:0/3544541450 conn(0x7f34600b5660 msgr2=0x7f3460071ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:46.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.299+0000 7f345fe00640 1 -- 172.21.15.131:0/3544541450 shutdown_connections 2024-05-26T22:51:46.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.299+0000 7f345fe00640 1 -- 172.21.15.131:0/3544541450 wait complete. 2024-05-26T22:51:46.317 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-26T22:51:46.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.506+0000 7fc98e400640 1 Processor -- start 2024-05-26T22:51:46.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.507+0000 7fc98e400640 1 -- start start 2024-05-26T22:51:46.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.508+0000 7fc98e400640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fc98814c090 0x7fc9880b2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:46.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.508+0000 7fc98e400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc98806df90 0x7fc988071160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:46.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.508+0000 7fc98e400640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc988071990 con 0x7fc9880cc760 2024-05-26T22:51:46.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.508+0000 7fc98e400640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fc988071f50 con 0x7fc98814c090 2024-05-26T22:51:46.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.508+0000 7fc98e400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc9880ce150 con 0x7fc98806df90 2024-05-26T22:51:46.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.508+0000 7fc98da00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fc9880cc760 0x7fc9880ce5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:37894/0 (socket says 172.21.15.131:37894) 2024-05-26T22:51:46.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.508+0000 7fc98da00640 1 -- 172.21.15.131:0/4068287042 learned_addr learned my addr 172.21.15.131:0/4068287042 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:51:46.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.509+0000 7fc97fe00640 1 --2- 172.21.15.131:0/4068287042 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc98806df90 0x7fc988071160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:46.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.509+0000 7fc98d000640 1 --2- 172.21.15.131:0/4068287042 >> v2:172.21.15.187:3300/0 conn(0x7fc98814c090 0x7fc9880b2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:46.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.509+0000 7fc987e00640 1 -- 172.21.15.131:0/4068287042 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 245880860 0 0) 0x7fc988071990 con 0x7fc9880cc760 2024-05-26T22:51:46.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.509+0000 7fc987e00640 1 -- 172.21.15.131:0/4068287042 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc96c003520 con 0x7fc9880cc760 2024-05-26T22:51:46.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.509+0000 7fc987e00640 1 -- 172.21.15.131:0/4068287042 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3682557598 0 0) 0x7fc96c003520 con 0x7fc9880cc760 2024-05-26T22:51:46.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.509+0000 7fc987e00640 1 -- 172.21.15.131:0/4068287042 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc98806df90 msgr2=0x7fc988071160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:46.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.509+0000 7fc987e00640 1 --2- 172.21.15.131:0/4068287042 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc98806df90 0x7fc988071160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:46.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.509+0000 7fc987e00640 1 -- 172.21.15.131:0/4068287042 >> v2:172.21.15.187:3300/0 conn(0x7fc98814c090 msgr2=0x7fc9880b2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:46.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.509+0000 7fc987e00640 1 --2- 172.21.15.131:0/4068287042 >> v2:172.21.15.187:3300/0 conn(0x7fc98814c090 0x7fc9880b2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:46.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.509+0000 7fc987e00640 1 -- 172.21.15.131:0/4068287042 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc988071c70 con 0x7fc9880cc760 2024-05-26T22:51:46.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.509+0000 7fc987e00640 1 -- 172.21.15.131:0/4068287042 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7fc9800019f0 con 0x7fc9880cc760 2024-05-26T22:51:46.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.509+0000 7fc987e00640 1 -- 172.21.15.131:0/4068287042 >> v1:172.21.15.131:6789/0 conn(0x7fc9880cc760 legacy=0x7fc9880ce5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:46.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.510+0000 7fc987e00640 1 --2- 172.21.15.131:0/4068287042 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc98806df90 0x7fc96c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:46.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.510+0000 7fc987e00640 1 --2- 172.21.15.131:0/4068287042 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc96c003f30 0x7fc96c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:46.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.510+0000 7fc987e00640 1 --2- 172.21.15.131:0/4068287042 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc96c004d40 0x7fc96c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:46.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.510+0000 7fc987e00640 1 -- 172.21.15.131:0/4068287042 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc988071f50 con 0x7fc96c003f30 2024-05-26T22:51:46.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.510+0000 7fc987e00640 1 -- 172.21.15.131:0/4068287042 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc9880ce150 con 0x7fc96c004d40 2024-05-26T22:51:46.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.510+0000 7fc987e00640 1 -- 172.21.15.131:0/4068287042 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc96c00bac0 con 0x7fc98806df90 2024-05-26T22:51:46.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.510+0000 7fc97fe00640 1 --2- 172.21.15.131:0/4068287042 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc96c003f30 0x7fc96c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:46.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.510+0000 7fc98da00640 1 --2- 172.21.15.131:0/4068287042 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc96c004d40 0x7fc96c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:46.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.510+0000 7fc98d000640 1 --2- 172.21.15.131:0/4068287042 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc98806df90 0x7fc96c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:46.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.510+0000 7fc97fe00640 1 -- 172.21.15.131:0/4068287042 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc98806df90 msgr2=0x7fc96c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:46.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.510+0000 7fc97fe00640 1 --2- 172.21.15.131:0/4068287042 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc98806df90 0x7fc96c0039b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:46.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.510+0000 7fc97fe00640 1 -- 172.21.15.131:0/4068287042 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc96c004d40 msgr2=0x7fc96c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:46.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.510+0000 7fc97fe00640 1 --2- 172.21.15.131:0/4068287042 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc96c004d40 0x7fc96c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:46.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.510+0000 7fc97fe00640 1 -- 172.21.15.131:0/4068287042 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc96c003520 con 0x7fc96c003f30 2024-05-26T22:51:46.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.510+0000 7fc98da00640 1 --2- 172.21.15.131:0/4068287042 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc96c004d40 0x7fc96c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:51:46.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.511+0000 7fc97fe00640 1 --2- 172.21.15.131:0/4068287042 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc96c003f30 0x7fc96c004800 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fc97000ba60 tx=0x7fc96c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e97569ee91420a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:46.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.511+0000 7fc987e00640 1 -- 172.21.15.131:0/4068287042 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc970030070 con 0x7fc96c003f30 2024-05-26T22:51:46.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.511+0000 7fc987e00640 1 -- 172.21.15.131:0/4068287042 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc97002ae10 con 0x7fc96c003f30 2024-05-26T22:51:46.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.512+0000 7fc987e00640 1 -- 172.21.15.131:0/4068287042 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc97000b600 con 0x7fc96c003f30 2024-05-26T22:51:46.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.512+0000 7fc98e400640 1 -- 172.21.15.131:0/4068287042 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc96c003f30 msgr2=0x7fc96c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:46.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.512+0000 7fc98e400640 1 --2- 172.21.15.131:0/4068287042 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc96c003f30 0x7fc96c004800 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fc97000ba60 tx=0x7fc96c005830 comp rx=0 tx=0).stop 2024-05-26T22:51:46.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.512+0000 7fc98da00640 1 -- 172.21.15.131:0/4068287042 reap_dead start 2024-05-26T22:51:46.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.512+0000 7fc98e400640 1 -- 172.21.15.131:0/4068287042 shutdown_connections 2024-05-26T22:51:46.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.512+0000 7fc98e400640 1 -- 172.21.15.131:0/4068287042 >> 172.21.15.131:0/4068287042 conn(0x7fc9880cff40 msgr2=0x7fc9880d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:46.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.512+0000 7fc98e400640 1 -- 172.21.15.131:0/4068287042 shutdown_connections 2024-05-26T22:51:46.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.512+0000 7fc98e400640 1 -- 172.21.15.131:0/4068287042 wait complete. 2024-05-26T22:51:46.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.514+0000 7fc98e400640 1 Processor -- start 2024-05-26T22:51:46.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.514+0000 7fc98e400640 1 -- start start 2024-05-26T22:51:46.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.515+0000 7fc98e400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc96c003f30 0x7fc988154040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:46.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.515+0000 7fc98e400640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc9880b2080 0x7fc988164f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:46.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.515+0000 7fc98e400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc9880b2500 0x7fc988165440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:46.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.515+0000 7fc98e400640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc98814c5f0 con 0x7fc96c003f30 2024-05-26T22:51:46.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.515+0000 7fc98e400640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc98814c470 con 0x7fc9880b2080 2024-05-26T22:51:46.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.515+0000 7fc98e400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc9880ccbf0 con 0x7fc9880b2500 2024-05-26T22:51:46.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.515+0000 7fc98d000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc96c003f30 0x7fc988154040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:46.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.515+0000 7fc98d000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc96c003f30 0x7fc988154040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47798/0 (socket says 172.21.15.131:47798) 2024-05-26T22:51:46.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.515+0000 7fc98d000640 1 -- 172.21.15.131:0/230738743 learned_addr learned my addr 172.21.15.131:0/230738743 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:46.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.516+0000 7fc98d000640 1 -- 172.21.15.131:0/230738743 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc9880b2500 msgr2=0x7fc988165440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:46.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.516+0000 7fc98d000640 1 --2- 172.21.15.131:0/230738743 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc9880b2500 0x7fc988165440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:46.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.516+0000 7fc98d000640 1 -- 172.21.15.131:0/230738743 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc9880b2080 msgr2=0x7fc988164f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:46.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.516+0000 7fc97fe00640 1 --2- 172.21.15.131:0/230738743 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc9880b2080 0x7fc988164f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:46.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.516+0000 7fc98d000640 1 --2- 172.21.15.131:0/230738743 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc9880b2080 0x7fc988164f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:46.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.516+0000 7fc98d000640 1 -- 172.21.15.131:0/230738743 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc988154580 con 0x7fc96c003f30 2024-05-26T22:51:46.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.516+0000 7fc98d000640 1 --2- 172.21.15.131:0/230738743 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc96c003f30 0x7fc988154040 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fc978010940 tx=0x7fc978010e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:46.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.516+0000 7fc985600640 1 -- 172.21.15.131:0/230738743 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc97800f040 con 0x7fc96c003f30 2024-05-26T22:51:46.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.516+0000 7fc985600640 1 -- 172.21.15.131:0/230738743 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc97800eb00 con 0x7fc96c003f30 2024-05-26T22:51:46.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.516+0000 7fc985600640 1 -- 172.21.15.131:0/230738743 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc97801c400 con 0x7fc96c003f30 2024-05-26T22:51:46.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.516+0000 7fc98e400640 1 -- 172.21.15.131:0/230738743 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc988142210 con 0x7fc96c003f30 2024-05-26T22:51:46.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.516+0000 7fc98e400640 1 -- 172.21.15.131:0/230738743 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9881427d0 con 0x7fc96c003f30 2024-05-26T22:51:46.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.517+0000 7fc98e400640 1 -- 172.21.15.131:0/230738743 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9880694e0 con 0x7fc96c003f30 2024-05-26T22:51:46.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.518+0000 7fc985600640 1 -- 172.21.15.131:0/230738743 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc97801c5a0 con 0x7fc96c003f30 2024-05-26T22:51:46.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.519+0000 7fc985600640 1 --2- 172.21.15.131:0/230738743 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc96406be30 0x7fc96406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:46.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.519+0000 7fc985600640 1 -- 172.21.15.131:0/230738743 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fc97808b5b0 con 0x7fc96c003f30 2024-05-26T22:51:46.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.519+0000 7fc97fe00640 1 --2- 172.21.15.131:0/230738743 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc96406be30 0x7fc96406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:46.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.520+0000 7fc97fe00640 1 --2- 172.21.15.131:0/230738743 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc96406be30 0x7fc96406e2f0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fc97000ba30 tx=0x7fc970030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:46.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.524+0000 7fc985600640 1 -- 172.21.15.131:0/230738743 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fc978018030 con 0x7fc96c003f30 2024-05-26T22:51:46.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.764+0000 7fc98e400640 1 -- 172.21.15.131:0/230738743 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fc98806eb90 con 0x7fc96c003f30 2024-05-26T22:51:46.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.764+0000 7fc985600640 1 -- 172.21.15.131:0/230738743 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v49) ==== 53+0+73 (secure 0 0 0) 0x7fc978059900 con 0x7fc96c003f30 2024-05-26T22:51:46.764 INFO:teuthology.orchestra.run.smithi131.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-26T22:51:46.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.767+0000 7fc98e400640 1 -- 172.21.15.131:0/230738743 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc96406be30 msgr2=0x7fc96406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:46.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.767+0000 7fc98e400640 1 --2- 172.21.15.131:0/230738743 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc96406be30 0x7fc96406e2f0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fc97000ba30 tx=0x7fc970030000 comp rx=0 tx=0).stop 2024-05-26T22:51:46.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.767+0000 7fc98e400640 1 -- 172.21.15.131:0/230738743 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc96c003f30 msgr2=0x7fc988154040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:46.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.767+0000 7fc98e400640 1 --2- 172.21.15.131:0/230738743 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc96c003f30 0x7fc988154040 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fc978010940 tx=0x7fc978010e00 comp rx=0 tx=0).stop 2024-05-26T22:51:46.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.767+0000 7fc98e400640 1 -- 172.21.15.131:0/230738743 shutdown_connections 2024-05-26T22:51:46.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.767+0000 7fc98e400640 1 --2- 172.21.15.131:0/230738743 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc9880b2500 0x7fc988165440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:46.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.767+0000 7fc98e400640 1 --2- 172.21.15.131:0/230738743 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc96406be30 0x7fc96406e2f0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:46.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.767+0000 7fc98e400640 1 --2- 172.21.15.131:0/230738743 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc9880b2080 0x7fc988164f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:46.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.767+0000 7fc98e400640 1 --2- 172.21.15.131:0/230738743 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc96c003f30 0x7fc988154040 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:46.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.767+0000 7fc98e400640 1 -- 172.21.15.131:0/230738743 >> 172.21.15.131:0/230738743 conn(0x7fc9880cff40 msgr2=0x7fc9880b32c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:46.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.767+0000 7fc98e400640 1 -- 172.21.15.131:0/230738743 shutdown_connections 2024-05-26T22:51:46.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:46.767+0000 7fc98e400640 1 -- 172.21.15.131:0/230738743 wait complete. 2024-05-26T22:51:46.785 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-26T22:51:46.786 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-26T22:51:46.786 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi131.front.sepia.ceph.com 2024-05-26T22:51:46.786 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-26T22:51:46.786 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-26T22:51:46.786 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-26T22:51:46.786 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-26T22:51:46.786 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-26T22:51:46.786 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-26T22:51:46.786 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:51:46.786 DEBUG:teuthology.orchestra.run.smithi131:> ip addr 2024-05-26T22:51:46.834 INFO:teuthology.orchestra.run.smithi131.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-26T22:51:46.834 INFO:teuthology.orchestra.run.smithi131.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-26T22:51:46.834 INFO:teuthology.orchestra.run.smithi131.stdout: inet 127.0.0.1/8 scope host lo 2024-05-26T22:51:46.834 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-05-26T22:51:46.834 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 ::1/128 scope host 2024-05-26T22:51:46.834 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-05-26T22:51:46.835 INFO:teuthology.orchestra.run.smithi131.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-26T22:51:46.835 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:d8:fb:18 brd ff:ff:ff:ff:ff:ff 2024-05-26T22:51:46.835 INFO:teuthology.orchestra.run.smithi131.stdout: altname enp7s0f0 2024-05-26T22:51:46.835 INFO:teuthology.orchestra.run.smithi131.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-26T22:51:46.835 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:d8:fb:19 brd ff:ff:ff:ff:ff:ff 2024-05-26T22:51:46.835 INFO:teuthology.orchestra.run.smithi131.stdout: altname enp7s0f1 2024-05-26T22:51:46.835 INFO:teuthology.orchestra.run.smithi131.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-26T22:51:46.835 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:88:72:f8 brd ff:ff:ff:ff:ff:ff 2024-05-26T22:51:46.835 INFO:teuthology.orchestra.run.smithi131.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-26T22:51:46.835 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:88:72:f9 brd ff:ff:ff:ff:ff:ff 2024-05-26T22:51:46.835 INFO:teuthology.orchestra.run.smithi131.stdout: inet 172.21.15.131/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-26T22:51:46.835 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft 42390sec preferred_lft 42390sec 2024-05-26T22:51:46.836 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::ec4:7aff:fe88:72f9/64 scope link 2024-05-26T22:51:46.836 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-05-26T22:51:46.836 INFO:teuthology.orchestra.run.smithi131.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-26T22:51:46.836 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 06:f4:05:86:a4:3e brd ff:ff:ff:ff:ff:ff 2024-05-26T22:51:46.836 INFO:teuthology.orchestra.run.smithi131.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-26T22:51:46.836 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-05-26T22:51:46.836 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::8802:8fff:fe41:fc6d/64 scope link 2024-05-26T22:51:46.836 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-05-26T22:51:46.836 INFO:teuthology.orchestra.run.smithi131.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-26T22:51:46.836 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 06:f4:05:86:a4:3e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-26T22:51:46.836 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::4f4:5ff:fe86:a43e/64 scope link 2024-05-26T22:51:46.836 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-05-26T22:51:46.836 INFO:teuthology.orchestra.run.smithi131.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-26T22:51:46.836 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether ee:6f:35:b2:c0:5b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-26T22:51:46.837 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::ec6f:35ff:feb2:c05b/64 scope link 2024-05-26T22:51:46.837 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-05-26T22:51:46.838 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:51:46.838 DEBUG:teuthology.orchestra.run.smithi131:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-26T22:51:46.905 INFO:teuthology.orchestra.run.smithi131.stdout:1 2024-05-26T22:51:46.908 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:51:46.908 DEBUG:teuthology.orchestra.run.smithi131:> route 2024-05-26T22:51:46.951 INFO:teuthology.orchestra.run.smithi131.stdout:Kernel IP routing table 2024-05-26T22:51:46.952 INFO:teuthology.orchestra.run.smithi131.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-26T22:51:46.952 INFO:teuthology.orchestra.run.smithi131.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-26T22:51:46.952 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-26T22:51:46.952 INFO:teuthology.orchestra.run.smithi131.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-26T22:51:46.952 DEBUG:teuthology.orchestra.run.smithi131:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-26T22:51:46.952 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2024-05-26T22:51:46.952 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-26T22:51:46.952 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-26T22:51:46.953 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-26T22:51:46.953 DEBUG:teuthology.orchestra.run.smithi131:> ') 2024-05-26T22:51:47.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:47 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-26T22:51:47.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-26T22:51:47.204 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:51:47.204 DEBUG:teuthology.orchestra.run.smithi131:> ip netns list 2024-05-26T22:51:47.253 INFO:teuthology.orchestra.run.smithi131.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-26T22:51:47.253 INFO:teuthology.orchestra.run.smithi131.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-26T22:51:47.253 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-26T22:51:47.254 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:51:47.254 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link set brx.0 up 2024-05-26T22:51:47.331 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-26T22:51:47.331 DEBUG:teuthology.orchestra.run.smithi131:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-26T22:51:47.383 INFO:teuthology.orchestra.run.smithi131.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-26T22:51:47.384 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-26T22:51:47.384 DEBUG:teuthology.orchestra.run.smithi131:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-26T22:51:47.433 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-05-26T22:51:47.433 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:51:47.433 DEBUG:teuthology.orchestra.run.smithi131:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-26T22:51:47.557 INFO:teuthology.orchestra.run.smithi131.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-26T22:51:47.557 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-26T22:51:47.591 INFO:teuthology.orchestra.run.smithi131.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-26T22:51:47.591 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-26T22:51:47.591 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: resolved to: "172.21.15.131:3300,172.21.15.187:3300,172.21.15.187:3301" 2024-05-26T22:51:47.591 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: trying mount with new device syntax: 0@46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f.cephfs=/ 2024-05-26T22:51:47.592 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.131:3300/172.21.15.187:3300/172.21.15.187:3301" will pass to kernel 2024-05-26T22:51:47.593 INFO:teuthology.orchestra.run.smithi131.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-26T22:51:47.593 INFO:teuthology.orchestra.run.smithi131.stdout: You just mounted a file system that supports labels which does not 2024-05-26T22:51:47.593 INFO:teuthology.orchestra.run.smithi131.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-26T22:51:47.593 INFO:teuthology.orchestra.run.smithi131.stdout: applications will generate AVC messages and not be allowed access to 2024-05-26T22:51:47.593 INFO:teuthology.orchestra.run.smithi131.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-26T22:51:47.598 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-26T22:51:47.598 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:51:47.598 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-26T22:51:47.672 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-26T22:51:47.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:47 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-26T22:51:47.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-26T22:51:47.810 INFO:teuthology.orchestra.run.smithi131.stdout:client5209 2024-05-26T22:51:47.810 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-26T22:51:47.810 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-26T22:51:47.863 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_export_ephemeral_random True 2024-05-26T22:51:48.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.071+0000 7f51da600640 1 Processor -- start 2024-05-26T22:51:48.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.072+0000 7f51da600640 1 -- start start 2024-05-26T22:51:48.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.073+0000 7f51da600640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f51d4145a10 0x7f51d4072180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:48.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.073+0000 7f51da600640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f51d40712d0 0x7f51d406d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:48.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.073+0000 7f51da600640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51d4072d60 con 0x7f51d406d790 2024-05-26T22:51:48.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.073+0000 7f51da600640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f51d415c760 con 0x7f51d4145a10 2024-05-26T22:51:48.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.073+0000 7f51da600640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f51d4072a80 con 0x7f51d40712d0 2024-05-26T22:51:48.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.074+0000 7f51d9200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f51d4145a10 0x7f51d4072180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:48.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.074+0000 7f51d9200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f51d4145a10 0x7f51d4072180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47814/0 (socket says 172.21.15.131:47814) 2024-05-26T22:51:48.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.074+0000 7f51d9200640 1 -- 172.21.15.131:0/2631085817 learned_addr learned my addr 172.21.15.131:0/2631085817 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:48.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.074+0000 7f51cbe00640 1 --2- 172.21.15.131:0/2631085817 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f51d40712d0 0x7f51d406d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:48.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.074+0000 7f51d9200640 1 -- 172.21.15.131:0/2631085817 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f51d40712d0 msgr2=0x7f51d406d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:48.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.074+0000 7f51d9200640 1 --2- 172.21.15.131:0/2631085817 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f51d40712d0 0x7f51d406d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:48.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.074+0000 7f51d9200640 1 -- 172.21.15.131:0/2631085817 >> v1:172.21.15.187:6789/0 conn(0x7f51d406d790 legacy=0x7f51d415fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:48.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.074+0000 7f51d9200640 1 -- 172.21.15.131:0/2631085817 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51d40ceab0 con 0x7f51d4145a10 2024-05-26T22:51:48.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.074+0000 7f51cbe00640 1 --2- 172.21.15.131:0/2631085817 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f51d40712d0 0x7f51d406d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:51:48.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.075+0000 7f51d9200640 1 --2- 172.21.15.131:0/2631085817 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f51d4145a10 0x7f51d4072180 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f51c4009870 tx=0x7f51c402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88ba5496df2ae2a server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:48.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.076+0000 7f51d3e00640 1 -- 172.21.15.131:0/2631085817 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51c4037070 con 0x7f51d4145a10 2024-05-26T22:51:48.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.076+0000 7f51d3e00640 1 -- 172.21.15.131:0/2631085817 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51c4032440 con 0x7f51d4145a10 2024-05-26T22:51:48.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.076+0000 7f51d3e00640 1 -- 172.21.15.131:0/2631085817 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51c4032810 con 0x7f51d4145a10 2024-05-26T22:51:48.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.076+0000 7f51da600640 1 -- 172.21.15.131:0/2631085817 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f51d4145a10 msgr2=0x7f51d4072180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:48.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.076+0000 7f51da600640 1 --2- 172.21.15.131:0/2631085817 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f51d4145a10 0x7f51d4072180 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f51c4009870 tx=0x7f51c402a210 comp rx=0 tx=0).stop 2024-05-26T22:51:48.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.076+0000 7f51da600640 1 -- 172.21.15.131:0/2631085817 shutdown_connections 2024-05-26T22:51:48.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.076+0000 7f51da600640 1 --2- 172.21.15.131:0/2631085817 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f51d40712d0 0x7f51d406d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:48.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.076+0000 7f51da600640 1 --2- 172.21.15.131:0/2631085817 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f51d4145a10 0x7f51d4072180 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:48.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.076+0000 7f51da600640 1 -- 172.21.15.131:0/2631085817 >> 172.21.15.131:0/2631085817 conn(0x7f51d40b5660 msgr2=0x7f51d40b5a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:48.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.077+0000 7f51da600640 1 -- 172.21.15.131:0/2631085817 shutdown_connections 2024-05-26T22:51:48.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.077+0000 7f51da600640 1 -- 172.21.15.131:0/2631085817 wait complete. 2024-05-26T22:51:48.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.078+0000 7f51da600640 1 Processor -- start 2024-05-26T22:51:48.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.078+0000 7f51da600640 1 -- start start 2024-05-26T22:51:48.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.079+0000 7f51da600640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f51d406d790 0x7f51d4141a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:48.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.079+0000 7f51da600640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f51d40712d0 0x7f51d415ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:48.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.079+0000 7f51da600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f51d4145a10 0x7f51d413b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:48.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.079+0000 7f51da600640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f51d4145df0 con 0x7f51d4145a10 2024-05-26T22:51:48.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.079+0000 7f51da600640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f51d40cf070 con 0x7f51d40712d0 2024-05-26T22:51:48.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.079+0000 7f51da600640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f51d40726c0 con 0x7f51d406d790 2024-05-26T22:51:48.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.079+0000 7f51d9c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f51d4145a10 0x7f51d413b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:48.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.080+0000 7f51d9c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f51d4145a10 0x7f51d413b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47830/0 (socket says 172.21.15.131:47830) 2024-05-26T22:51:48.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.080+0000 7f51d9c00640 1 -- 172.21.15.131:0/451511156 learned_addr learned my addr 172.21.15.131:0/451511156 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:48.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.080+0000 7f51d9200640 1 --2- 172.21.15.131:0/451511156 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f51d406d790 0x7f51d4141a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:48.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.080+0000 7f51cbe00640 1 --2- 172.21.15.131:0/451511156 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f51d40712d0 0x7f51d415ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:48.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.080+0000 7f51d9c00640 1 -- 172.21.15.131:0/451511156 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f51d406d790 msgr2=0x7f51d4141a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:48.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.080+0000 7f51d9c00640 1 --2- 172.21.15.131:0/451511156 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f51d406d790 0x7f51d4141a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:48.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.080+0000 7f51d9c00640 1 -- 172.21.15.131:0/451511156 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f51d40712d0 msgr2=0x7f51d415ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:48.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.080+0000 7f51d9c00640 1 --2- 172.21.15.131:0/451511156 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f51d40712d0 0x7f51d415ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:48.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.080+0000 7f51d9c00640 1 -- 172.21.15.131:0/451511156 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51d4141f40 con 0x7f51d4145a10 2024-05-26T22:51:48.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.080+0000 7f51d9200640 1 --2- 172.21.15.131:0/451511156 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f51d406d790 0x7f51d4141a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:51:48.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.080+0000 7f51cbe00640 1 --2- 172.21.15.131:0/451511156 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f51d40712d0 0x7f51d415ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:51:48.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.080+0000 7f51d9c00640 1 --2- 172.21.15.131:0/451511156 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f51d4145a10 0x7f51d413b480 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f51cc00c7b0 tx=0x7f51cc00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:48.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.081+0000 7f51d1600640 1 -- 172.21.15.131:0/451511156 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51cc004b60 con 0x7f51d4145a10 2024-05-26T22:51:48.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.081+0000 7f51d1600640 1 -- 172.21.15.131:0/451511156 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51cc004d00 con 0x7f51d4145a10 2024-05-26T22:51:48.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.081+0000 7f51d1600640 1 -- 172.21.15.131:0/451511156 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51cc00aa90 con 0x7f51d4145a10 2024-05-26T22:51:48.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.081+0000 7f51da600640 1 -- 172.21.15.131:0/451511156 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51d41421e0 con 0x7f51d4145a10 2024-05-26T22:51:48.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.081+0000 7f51da600640 1 -- 172.21.15.131:0/451511156 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51d41427a0 con 0x7f51d4145a10 2024-05-26T22:51:48.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.082+0000 7f51da600640 1 -- 172.21.15.131:0/451511156 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51d40694e0 con 0x7f51d4145a10 2024-05-26T22:51:48.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.083+0000 7f51d1600640 1 -- 172.21.15.131:0/451511156 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f51cc00ac30 con 0x7f51d4145a10 2024-05-26T22:51:48.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.084+0000 7f51d1600640 1 --2- 172.21.15.131:0/451511156 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f51b006be30 0x7f51b006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:48.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.084+0000 7f51d1600640 1 -- 172.21.15.131:0/451511156 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f51cc08b2c0 con 0x7f51d4145a10 2024-05-26T22:51:48.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.084+0000 7f51d9200640 1 --2- 172.21.15.131:0/451511156 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f51b006be30 0x7f51b006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:48.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.085+0000 7f51d9200640 1 --2- 172.21.15.131:0/451511156 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f51b006be30 0x7f51b006e2f0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f51c4034000 tx=0x7f51c402e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:48.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.089+0000 7f51d1600640 1 -- 172.21.15.131:0/451511156 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f51cc020070 con 0x7f51d4145a10 2024-05-26T22:51:48.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.292+0000 7f51da600640 1 -- 172.21.15.131:0/451511156 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command([{prefix=config set, name=mds_export_ephemeral_random}] v 0) -- 0x7f51d40cf9f0 con 0x7f51d4145a10 2024-05-26T22:51:48.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.298+0000 7f51d1600640 1 -- 172.21.15.131:0/451511156 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_export_ephemeral_random}]=0 v28)=0 v28) ==== 128+0+0 (secure 0 0 0) 0x7f51cc059b40 con 0x7f51d4145a10 2024-05-26T22:51:48.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.301+0000 7f51da600640 1 -- 172.21.15.131:0/451511156 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f51b006be30 msgr2=0x7f51b006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:48.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.301+0000 7f51da600640 1 --2- 172.21.15.131:0/451511156 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f51b006be30 0x7f51b006e2f0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f51c4034000 tx=0x7f51c402e000 comp rx=0 tx=0).stop 2024-05-26T22:51:48.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.301+0000 7f51da600640 1 -- 172.21.15.131:0/451511156 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f51d4145a10 msgr2=0x7f51d413b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:48.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.301+0000 7f51da600640 1 --2- 172.21.15.131:0/451511156 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f51d4145a10 0x7f51d413b480 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f51cc00c7b0 tx=0x7f51cc00cc70 comp rx=0 tx=0).stop 2024-05-26T22:51:48.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.301+0000 7f51da600640 1 -- 172.21.15.131:0/451511156 shutdown_connections 2024-05-26T22:51:48.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.301+0000 7f51da600640 1 --2- 172.21.15.131:0/451511156 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f51b006be30 0x7f51b006e2f0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:48.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.301+0000 7f51da600640 1 --2- 172.21.15.131:0/451511156 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f51d4145a10 0x7f51d413b480 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:48.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.301+0000 7f51da600640 1 --2- 172.21.15.131:0/451511156 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f51d40712d0 0x7f51d415ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:48.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.301+0000 7f51da600640 1 --2- 172.21.15.131:0/451511156 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f51d406d790 0x7f51d4141a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:48.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.301+0000 7f51da600640 1 -- 172.21.15.131:0/451511156 >> 172.21.15.131:0/451511156 conn(0x7f51d40b5660 msgr2=0x7f51d4071ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:48.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.301+0000 7f51da600640 1 -- 172.21.15.131:0/451511156 shutdown_connections 2024-05-26T22:51:48.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.302+0000 7f51da600640 1 -- 172.21.15.131:0/451511156 wait complete. 2024-05-26T22:51:48.320 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_export_ephemeral_distributed True 2024-05-26T22:51:48.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.515+0000 7fb8ce400640 1 Processor -- start 2024-05-26T22:51:48.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.516+0000 7fb8ce400640 1 -- start start 2024-05-26T22:51:48.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.517+0000 7fb8ce400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fb8c815e320 0x7fb8c80ae250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:48.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.517+0000 7fb8ce400640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fb8c80b2f60 0x7fb8c80cfe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:48.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.517+0000 7fb8ce400640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8c80b2440 con 0x7fb8c80d0380 2024-05-26T22:51:48.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.517+0000 7fb8ce400640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fb8c80725e0 con 0x7fb8c815e320 2024-05-26T22:51:48.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.517+0000 7fb8ce400640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fb8c8161fe0 con 0x7fb8c80b2f60 2024-05-26T22:51:48.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.518+0000 7fb8cd000640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fb8c815e320 0x7fb8c80ae250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:48.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.518+0000 7fb8cd000640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fb8c815e320 0x7fb8c80ae250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47844/0 (socket says 172.21.15.131:47844) 2024-05-26T22:51:48.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.518+0000 7fb8cd000640 1 -- 172.21.15.131:0/986251883 learned_addr learned my addr 172.21.15.131:0/986251883 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:48.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.518+0000 7fb8cd000640 1 -- 172.21.15.131:0/986251883 >> v2:172.21.15.187:3300/0 conn(0x7fb8c80b2f60 msgr2=0x7fb8c80cfe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:48.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.518+0000 7fb8cd000640 1 --2- 172.21.15.131:0/986251883 >> v2:172.21.15.187:3300/0 conn(0x7fb8c80b2f60 0x7fb8c80cfe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:48.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.518+0000 7fb8cd000640 1 -- 172.21.15.131:0/986251883 >> v1:172.21.15.131:6789/0 conn(0x7fb8c80d0380 legacy=0x7fb8c806d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:48.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.518+0000 7fb8cd000640 1 -- 172.21.15.131:0/986251883 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8c8162230 con 0x7fb8c815e320 2024-05-26T22:51:48.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.519+0000 7fb8cd000640 1 --2- 172.21.15.131:0/986251883 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb8c815e320 0x7fb8c80ae250 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fb8c80b2970 tx=0x7fb8b002a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c888e19a8306a459 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:48.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.519+0000 7fb8c7e00640 1 -- 172.21.15.131:0/986251883 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b0037070 con 0x7fb8c815e320 2024-05-26T22:51:48.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.519+0000 7fb8c7e00640 1 -- 172.21.15.131:0/986251883 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8b002ae50 con 0x7fb8c815e320 2024-05-26T22:51:48.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.519+0000 7fb8c7e00640 1 -- 172.21.15.131:0/986251883 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b0032860 con 0x7fb8c815e320 2024-05-26T22:51:48.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.520+0000 7fb8ce400640 1 -- 172.21.15.131:0/986251883 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb8c815e320 msgr2=0x7fb8c80ae250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:48.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.520+0000 7fb8ce400640 1 --2- 172.21.15.131:0/986251883 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb8c815e320 0x7fb8c80ae250 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fb8c80b2970 tx=0x7fb8b002a300 comp rx=0 tx=0).stop 2024-05-26T22:51:48.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.520+0000 7fb8ce400640 1 -- 172.21.15.131:0/986251883 shutdown_connections 2024-05-26T22:51:48.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.520+0000 7fb8ce400640 1 --2- 172.21.15.131:0/986251883 >> v2:172.21.15.187:3300/0 conn(0x7fb8c80b2f60 0x7fb8c80cfe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:48.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.520+0000 7fb8ce400640 1 --2- 172.21.15.131:0/986251883 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb8c815e320 0x7fb8c80ae250 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:48.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.520+0000 7fb8ce400640 1 -- 172.21.15.131:0/986251883 >> 172.21.15.131:0/986251883 conn(0x7fb8c80b5660 msgr2=0x7fb8c80b5a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:48.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.520+0000 7fb8ce400640 1 -- 172.21.15.131:0/986251883 shutdown_connections 2024-05-26T22:51:48.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.520+0000 7fb8ce400640 1 -- 172.21.15.131:0/986251883 wait complete. 2024-05-26T22:51:48.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.523+0000 7fb8ce400640 1 Processor -- start 2024-05-26T22:51:48.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.523+0000 7fb8ce400640 1 -- start start 2024-05-26T22:51:48.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.524+0000 7fb8ce400640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb8c80b2f60 0x7fb8c81531b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:48.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.524+0000 7fb8ce400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb8c80d0380 0x7fb8c815bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:48.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.524+0000 7fb8ce400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb8c815e320 0x7fb8c8149b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:48.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.524+0000 7fb8ce400640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fb8c8162ad0 con 0x7fb8c815e320 2024-05-26T22:51:48.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.524+0000 7fb8ce400640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb8c81627f0 con 0x7fb8c80b2f60 2024-05-26T22:51:48.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.524+0000 7fb8ce400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb8c8162e40 con 0x7fb8c80d0380 2024-05-26T22:51:48.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.524+0000 7fb8cda00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb8c815e320 0x7fb8c8149b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:48.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.524+0000 7fb8cda00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb8c815e320 0x7fb8c8149b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47848/0 (socket says 172.21.15.131:47848) 2024-05-26T22:51:48.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.524+0000 7fb8cda00640 1 -- 172.21.15.131:0/3238436854 learned_addr learned my addr 172.21.15.131:0/3238436854 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:48.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.525+0000 7fb8cd000640 1 --2- 172.21.15.131:0/3238436854 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb8c80b2f60 0x7fb8c81531b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:48.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.525+0000 7fb8cda00640 1 -- 172.21.15.131:0/3238436854 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb8c80d0380 msgr2=0x7fb8c815bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:48.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.525+0000 7fb8bfe00640 1 --2- 172.21.15.131:0/3238436854 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb8c80d0380 0x7fb8c815bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:48.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.525+0000 7fb8cda00640 1 --2- 172.21.15.131:0/3238436854 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb8c80d0380 0x7fb8c815bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:48.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.525+0000 7fb8cda00640 1 -- 172.21.15.131:0/3238436854 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb8c80b2f60 msgr2=0x7fb8c81531b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:48.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.525+0000 7fb8cda00640 1 --2- 172.21.15.131:0/3238436854 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb8c80b2f60 0x7fb8c81531b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:48.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.525+0000 7fb8cda00640 1 -- 172.21.15.131:0/3238436854 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8c8156020 con 0x7fb8c815e320 2024-05-26T22:51:48.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.525+0000 7fb8cd000640 1 --2- 172.21.15.131:0/3238436854 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb8c80b2f60 0x7fb8c81531b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:51:48.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.525+0000 7fb8cda00640 1 --2- 172.21.15.131:0/3238436854 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb8c815e320 0x7fb8c8149b90 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fb8c000c7a0 tx=0x7fb8c000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:48.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.525+0000 7fb8c5600640 1 -- 172.21.15.131:0/3238436854 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c0003b70 con 0x7fb8c815e320 2024-05-26T22:51:48.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.526+0000 7fb8c5600640 1 -- 172.21.15.131:0/3238436854 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8c0003d30 con 0x7fb8c815e320 2024-05-26T22:51:48.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.526+0000 7fb8c5600640 1 -- 172.21.15.131:0/3238436854 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c000aa90 con 0x7fb8c815e320 2024-05-26T22:51:48.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.526+0000 7fb8ce400640 1 -- 172.21.15.131:0/3238436854 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8c8156300 con 0x7fb8c815e320 2024-05-26T22:51:48.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.526+0000 7fb8ce400640 1 -- 172.21.15.131:0/3238436854 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8c815b610 con 0x7fb8c815e320 2024-05-26T22:51:48.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.527+0000 7fb8ce400640 1 -- 172.21.15.131:0/3238436854 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8c80694e0 con 0x7fb8c815e320 2024-05-26T22:51:48.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.528+0000 7fb8c5600640 1 -- 172.21.15.131:0/3238436854 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb8c000ac30 con 0x7fb8c815e320 2024-05-26T22:51:48.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.529+0000 7fb8c5600640 1 --2- 172.21.15.131:0/3238436854 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb8a406be30 0x7fb8a406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:48.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.529+0000 7fb8c5600640 1 -- 172.21.15.131:0/3238436854 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fb8c008ad60 con 0x7fb8c815e320 2024-05-26T22:51:48.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.529+0000 7fb8cd000640 1 --2- 172.21.15.131:0/3238436854 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb8a406be30 0x7fb8a406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:48.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.530+0000 7fb8cd000640 1 --2- 172.21.15.131:0/3238436854 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb8a406be30 0x7fb8a406e2f0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fb8b0034000 tx=0x7fb8b002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:48.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.534+0000 7fb8c5600640 1 -- 172.21.15.131:0/3238436854 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fb8c00597a0 con 0x7fb8c815e320 2024-05-26T22:51:48.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.730+0000 7fb8ce400640 1 -- 172.21.15.131:0/3238436854 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command([{prefix=config set, name=mds_export_ephemeral_distributed}] v 0) -- 0x7fb8c81564d0 con 0x7fb8c815e320 2024-05-26T22:51:48.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.735+0000 7fb8c5600640 1 -- 172.21.15.131:0/3238436854 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_export_ephemeral_distributed}]=0 v29)=0 v29) ==== 133+0+0 (secure 0 0 0) 0x7fb8c00590a0 con 0x7fb8c815e320 2024-05-26T22:51:48.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.739+0000 7fb8ce400640 1 -- 172.21.15.131:0/3238436854 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb8a406be30 msgr2=0x7fb8a406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:48.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.739+0000 7fb8ce400640 1 --2- 172.21.15.131:0/3238436854 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb8a406be30 0x7fb8a406e2f0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fb8b0034000 tx=0x7fb8b002e000 comp rx=0 tx=0).stop 2024-05-26T22:51:48.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.739+0000 7fb8ce400640 1 -- 172.21.15.131:0/3238436854 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb8c815e320 msgr2=0x7fb8c8149b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:48.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.739+0000 7fb8ce400640 1 --2- 172.21.15.131:0/3238436854 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb8c815e320 0x7fb8c8149b90 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fb8c000c7a0 tx=0x7fb8c000cc60 comp rx=0 tx=0).stop 2024-05-26T22:51:48.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.739+0000 7fb8ce400640 1 -- 172.21.15.131:0/3238436854 shutdown_connections 2024-05-26T22:51:48.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.739+0000 7fb8ce400640 1 --2- 172.21.15.131:0/3238436854 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb8c80d0380 0x7fb8c815bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:48.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.739+0000 7fb8ce400640 1 --2- 172.21.15.131:0/3238436854 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb8a406be30 0x7fb8a406e2f0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:48.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.739+0000 7fb8ce400640 1 --2- 172.21.15.131:0/3238436854 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb8c815e320 0x7fb8c8149b90 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:48.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.739+0000 7fb8ce400640 1 --2- 172.21.15.131:0/3238436854 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb8c80b2f60 0x7fb8c81531b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:48.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.739+0000 7fb8ce400640 1 -- 172.21.15.131:0/3238436854 >> 172.21.15.131:0/3238436854 conn(0x7fb8c80b5660 msgr2=0x7fb8c80afad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:48.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.739+0000 7fb8ce400640 1 -- 172.21.15.131:0/3238436854 shutdown_connections 2024-05-26T22:51:48.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.739+0000 7fb8ce400640 1 -- 172.21.15.131:0/3238436854 wait complete. 2024-05-26T22:51:48.759 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_export_ephemeral_random_max 1.0 2024-05-26T22:51:48.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.959+0000 7fd0e3400640 1 Processor -- start 2024-05-26T22:51:49.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.959+0000 7fd0e3400640 1 -- start start 2024-05-26T22:51:49.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.960+0000 7fd0e3400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd0dc0d0460 0x7fd0dc06d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:49.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.960+0000 7fd0e3400640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0dc160180 con 0x7fd0dc06f420 2024-05-26T22:51:49.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.960+0000 7fd0e3400640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0dc160460 con 0x7fd0dc0b5450 2024-05-26T22:51:49.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.960+0000 7fd0e3400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd0dc160d90 con 0x7fd0dc0d0460 2024-05-26T22:51:49.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.961+0000 7fd0d3e00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fd0dc06f420 0x7fd0dc0cfd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:37908/0 (socket says 172.21.15.131:37908) 2024-05-26T22:51:49.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.961+0000 7fd0d3e00640 1 -- 172.21.15.131:0/1114704219 learned_addr learned my addr 172.21.15.131:0/1114704219 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:51:49.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.961+0000 7fd0e1400640 1 --2- 172.21.15.131:0/1114704219 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd0dc0d0460 0x7fd0dc06d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:49.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.961+0000 7fd0dbe00640 1 -- 172.21.15.131:0/1114704219 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1081978881 0 0) 0x7fd0dc160180 con 0x7fd0dc06f420 2024-05-26T22:51:49.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.961+0000 7fd0dbe00640 1 -- 172.21.15.131:0/1114704219 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd0c0003520 con 0x7fd0dc06f420 2024-05-26T22:51:49.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.962+0000 7fd0dbe00640 1 -- 172.21.15.131:0/1114704219 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 892305943 0 0) 0x7fd0c0003520 con 0x7fd0dc06f420 2024-05-26T22:51:49.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.962+0000 7fd0dbe00640 1 -- 172.21.15.131:0/1114704219 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd0dc0d0460 msgr2=0x7fd0dc06d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:49.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.962+0000 7fd0dbe00640 1 --2- 172.21.15.131:0/1114704219 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd0dc0d0460 0x7fd0dc06d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:49.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.962+0000 7fd0dbe00640 1 -- 172.21.15.131:0/1114704219 >> v1:172.21.15.187:6789/0 conn(0x7fd0dc0b5450 legacy=0x7fd0dc0b43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:49.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.962+0000 7fd0dbe00640 1 -- 172.21.15.131:0/1114704219 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0dc160740 con 0x7fd0dc06f420 2024-05-26T22:51:49.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.962+0000 7fd0dbe00640 1 -- 172.21.15.131:0/1114704219 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7fd0c4001670 con 0x7fd0dc06f420 2024-05-26T22:51:49.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.962+0000 7fd0dbe00640 1 -- 172.21.15.131:0/1114704219 >> v1:172.21.15.131:6789/0 conn(0x7fd0dc06f420 legacy=0x7fd0dc0cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:49.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.962+0000 7fd0dbe00640 1 --2- 172.21.15.131:0/1114704219 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd0dc0d0460 0x7fd0c00039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:49.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.962+0000 7fd0dbe00640 1 --2- 172.21.15.131:0/1114704219 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd0c0003f60 0x7fd0c0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:49.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.962+0000 7fd0dbe00640 1 --2- 172.21.15.131:0/1114704219 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd0c0004d40 0x7fd0c00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:49.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.962+0000 7fd0dbe00640 1 -- 172.21.15.131:0/1114704219 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fd0dc160d90 con 0x7fd0c0004d40 2024-05-26T22:51:49.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.962+0000 7fd0dbe00640 1 -- 172.21.15.131:0/1114704219 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd0c000baa0 con 0x7fd0c0003f60 2024-05-26T22:51:49.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.962+0000 7fd0dbe00640 1 -- 172.21.15.131:0/1114704219 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd0c000bca0 con 0x7fd0dc0d0460 2024-05-26T22:51:49.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.962+0000 7fd0e1400640 1 --2- 172.21.15.131:0/1114704219 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd0c0004d40 0x7fd0c00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:49.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.962+0000 7fd0e1400640 1 -- 172.21.15.131:0/1114704219 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd0dc0d0460 msgr2=0x7fd0c00039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:49.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.962+0000 7fd0e1400640 1 --2- 172.21.15.131:0/1114704219 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd0dc0d0460 0x7fd0c00039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:49.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.962+0000 7fd0e1400640 1 -- 172.21.15.131:0/1114704219 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd0c0003f60 msgr2=0x7fd0c0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:49.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.962+0000 7fd0d3e00640 1 --2- 172.21.15.131:0/1114704219 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd0c0003f60 0x7fd0c0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:49.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.962+0000 7fd0e1400640 1 --2- 172.21.15.131:0/1114704219 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd0c0003f60 0x7fd0c0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:49.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.963+0000 7fd0e1400640 1 -- 172.21.15.131:0/1114704219 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0c0003520 con 0x7fd0c0004d40 2024-05-26T22:51:49.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.963+0000 7fd0d3e00640 1 --2- 172.21.15.131:0/1114704219 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd0c0003f60 0x7fd0c0004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:51:49.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.963+0000 7fd0e1400640 1 --2- 172.21.15.131:0/1114704219 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd0c0004d40 0x7fd0c00051c0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fd0d4009fc0 tx=0x7fd0c0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c66913641f4f22ea server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:49.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.964+0000 7fd0dbe00640 1 -- 172.21.15.131:0/1114704219 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0d4009c10 con 0x7fd0c0004d40 2024-05-26T22:51:49.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.964+0000 7fd0dbe00640 1 -- 172.21.15.131:0/1114704219 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0d4036070 con 0x7fd0c0004d40 2024-05-26T22:51:49.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.964+0000 7fd0dbe00640 1 -- 172.21.15.131:0/1114704219 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0d402d070 con 0x7fd0c0004d40 2024-05-26T22:51:49.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.964+0000 7fd0e3400640 1 -- 172.21.15.131:0/1114704219 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd0c0004d40 msgr2=0x7fd0c00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:49.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.964+0000 7fd0e3400640 1 --2- 172.21.15.131:0/1114704219 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd0c0004d40 0x7fd0c00051c0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fd0d4009fc0 tx=0x7fd0c0005830 comp rx=0 tx=0).stop 2024-05-26T22:51:49.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.964+0000 7fd0e1400640 1 -- 172.21.15.131:0/1114704219 reap_dead start 2024-05-26T22:51:49.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.964+0000 7fd0e3400640 1 -- 172.21.15.131:0/1114704219 shutdown_connections 2024-05-26T22:51:49.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.964+0000 7fd0e3400640 1 -- 172.21.15.131:0/1114704219 >> 172.21.15.131:0/1114704219 conn(0x7fd0dc06bc90 msgr2=0x7fd0dc0b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:49.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.965+0000 7fd0e3400640 1 -- 172.21.15.131:0/1114704219 shutdown_connections 2024-05-26T22:51:49.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.965+0000 7fd0e3400640 1 -- 172.21.15.131:0/1114704219 wait complete. 2024-05-26T22:51:49.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.966+0000 7fd0e3400640 1 Processor -- start 2024-05-26T22:51:49.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.966+0000 7fd0e3400640 1 -- start start 2024-05-26T22:51:49.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.967+0000 7fd0e3400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd0c0004d40 0x7fd0dc156640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:49.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.967+0000 7fd0e3400640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd0dc071360 0x7fd0dc06e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:49.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.967+0000 7fd0e3400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd0dc14ffd0 0x7fd0dc150450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:49.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.967+0000 7fd0e3400640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fd0dc15fde0 con 0x7fd0dc14ffd0 2024-05-26T22:51:49.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.967+0000 7fd0e3400640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd0dc0b5830 con 0x7fd0dc071360 2024-05-26T22:51:49.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.967+0000 7fd0e3400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd0dc15ff60 con 0x7fd0c0004d40 2024-05-26T22:51:49.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.968+0000 7fd0e1400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd0dc14ffd0 0x7fd0dc150450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:49.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.968+0000 7fd0e1400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd0dc14ffd0 0x7fd0dc150450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47868/0 (socket says 172.21.15.131:47868) 2024-05-26T22:51:49.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.968+0000 7fd0e1400640 1 -- 172.21.15.131:0/2610433979 learned_addr learned my addr 172.21.15.131:0/2610433979 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:49.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.968+0000 7fd0e0a00640 1 --2- 172.21.15.131:0/2610433979 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd0c0004d40 0x7fd0dc156640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:49.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.968+0000 7fd0d3e00640 1 --2- 172.21.15.131:0/2610433979 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd0dc071360 0x7fd0dc06e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:49.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.968+0000 7fd0e1400640 1 -- 172.21.15.131:0/2610433979 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd0c0004d40 msgr2=0x7fd0dc156640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:49.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.968+0000 7fd0e1400640 1 --2- 172.21.15.131:0/2610433979 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd0c0004d40 0x7fd0dc156640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:49.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.968+0000 7fd0e1400640 1 -- 172.21.15.131:0/2610433979 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd0dc071360 msgr2=0x7fd0dc06e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:49.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.968+0000 7fd0e1400640 1 --2- 172.21.15.131:0/2610433979 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd0dc071360 0x7fd0dc06e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:49.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.968+0000 7fd0e1400640 1 -- 172.21.15.131:0/2610433979 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0dc071b20 con 0x7fd0dc14ffd0 2024-05-26T22:51:49.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.969+0000 7fd0e1400640 1 --2- 172.21.15.131:0/2610433979 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd0dc14ffd0 0x7fd0dc150450 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fd0d400e230 tx=0x7fd0d4009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:49.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.969+0000 7fd0d9600640 1 -- 172.21.15.131:0/2610433979 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0d4009c10 con 0x7fd0dc14ffd0 2024-05-26T22:51:49.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.969+0000 7fd0d9600640 1 -- 172.21.15.131:0/2610433979 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0d4003990 con 0x7fd0dc14ffd0 2024-05-26T22:51:49.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.969+0000 7fd0e3400640 1 -- 172.21.15.131:0/2610433979 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0dc071e00 con 0x7fd0dc14ffd0 2024-05-26T22:51:49.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.969+0000 7fd0d9600640 1 -- 172.21.15.131:0/2610433979 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0d4003d60 con 0x7fd0dc14ffd0 2024-05-26T22:51:49.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.969+0000 7fd0e3400640 1 -- 172.21.15.131:0/2610433979 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0dc0723c0 con 0x7fd0dc14ffd0 2024-05-26T22:51:49.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.970+0000 7fd0e3400640 1 -- 172.21.15.131:0/2610433979 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0dc074e90 con 0x7fd0dc14ffd0 2024-05-26T22:51:49.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.971+0000 7fd0d9600640 1 -- 172.21.15.131:0/2610433979 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd0d4005d10 con 0x7fd0dc14ffd0 2024-05-26T22:51:49.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.972+0000 7fd0d9600640 1 --2- 172.21.15.131:0/2610433979 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fd0b806be60 0x7fd0b806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:49.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.972+0000 7fd0d9600640 1 -- 172.21.15.131:0/2610433979 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fd0d40ac2a0 con 0x7fd0dc14ffd0 2024-05-26T22:51:49.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.972+0000 7fd0e0a00640 1 --2- 172.21.15.131:0/2610433979 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fd0b806be60 0x7fd0b806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:49.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.973+0000 7fd0e0a00640 1 --2- 172.21.15.131:0/2610433979 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fd0b806be60 0x7fd0b806e320 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc001680 tx=0x7fd0cc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:49.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:48.976+0000 7fd0d9600640 1 -- 172.21.15.131:0/2610433979 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fd0d404a060 con 0x7fd0dc14ffd0 2024-05-26T22:51:49.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:49.171+0000 7fd0e3400640 1 -- 172.21.15.131:0/2610433979 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command([{prefix=config set, name=mds_export_ephemeral_random_max}] v 0) -- 0x7fd0dc0cee90 con 0x7fd0dc14ffd0 2024-05-26T22:51:49.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:49.179+0000 7fd0d9600640 1 -- 172.21.15.131:0/2610433979 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_export_ephemeral_random_max}]=0 v30)=0 v30) ==== 131+0+0 (secure 0 0 0) 0x7fd0d407abb0 con 0x7fd0dc14ffd0 2024-05-26T22:51:49.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:49.181+0000 7fd0e3400640 1 -- 172.21.15.131:0/2610433979 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fd0b806be60 msgr2=0x7fd0b806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:49.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:49.181+0000 7fd0e3400640 1 --2- 172.21.15.131:0/2610433979 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fd0b806be60 0x7fd0b806e320 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc001680 tx=0x7fd0cc009040 comp rx=0 tx=0).stop 2024-05-26T22:51:49.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:49.181+0000 7fd0e3400640 1 -- 172.21.15.131:0/2610433979 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd0dc14ffd0 msgr2=0x7fd0dc150450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:49.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:49.181+0000 7fd0e3400640 1 --2- 172.21.15.131:0/2610433979 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd0dc14ffd0 0x7fd0dc150450 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fd0d400e230 tx=0x7fd0d4009750 comp rx=0 tx=0).stop 2024-05-26T22:51:49.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:49.182+0000 7fd0e3400640 1 -- 172.21.15.131:0/2610433979 shutdown_connections 2024-05-26T22:51:49.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:49.182+0000 7fd0e3400640 1 --2- 172.21.15.131:0/2610433979 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fd0b806be60 0x7fd0b806e320 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:49.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:49.182+0000 7fd0e3400640 1 --2- 172.21.15.131:0/2610433979 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd0dc14ffd0 0x7fd0dc150450 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:49.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:49.182+0000 7fd0e3400640 1 --2- 172.21.15.131:0/2610433979 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd0dc071360 0x7fd0dc06e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:49.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:49.182+0000 7fd0e3400640 1 --2- 172.21.15.131:0/2610433979 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd0c0004d40 0x7fd0dc156640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:49.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:49.182+0000 7fd0e3400640 1 -- 172.21.15.131:0/2610433979 >> 172.21.15.131:0/2610433979 conn(0x7fd0dc06bc90 msgr2=0x7fd0dc0cc100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:49.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:49.182+0000 7fd0e3400640 1 -- 172.21.15.131:0/2610433979 shutdown_connections 2024-05-26T22:51:49.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:49.182+0000 7fd0e3400640 1 -- 172.21.15.131:0/2610433979 wait complete. 2024-05-26T22:51:49.200 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-05-26T22:51:49.200 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2024-05-26T22:51:49.201 DEBUG:teuthology.orchestra.run.smithi131:> 2024-05-26T22:51:49.201 DEBUG:teuthology.orchestra.run.smithi131:> # Use up a random number of inode numbers so the ephemeral pinning is not the same every test. 2024-05-26T22:51:49.201 DEBUG:teuthology.orchestra.run.smithi131:> mkdir .inode_number_thrash 2024-05-26T22:51:49.201 DEBUG:teuthology.orchestra.run.smithi131:> count=$((RANDOM % 1024)) 2024-05-26T22:51:49.201 DEBUG:teuthology.orchestra.run.smithi131:> for ((i = 0; i < count; i++)); do touch .inode_number_thrash/$i; done 2024-05-26T22:51:49.201 DEBUG:teuthology.orchestra.run.smithi131:> rm -rf .inode_number_thrash 2024-05-26T22:51:49.201 DEBUG:teuthology.orchestra.run.smithi131:> ') 2024-05-26T22:51:49.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:49 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-26T22:51:58.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-26T22:51:58.290 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2024-05-26T22:51:58.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:58.497+0000 7efc2b200640 1 Processor -- start 2024-05-26T22:51:58.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:58.498+0000 7efc2b200640 1 -- start start 2024-05-26T22:51:58.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:58.499+0000 7efc2b200640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7efc2406f420 0x7efc240cfd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:58.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:58.500+0000 7efc2b200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efc240cf590 0x7efc240d0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:58.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:58.500+0000 7efc2b200640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc24160180 con 0x7efc240b5450 2024-05-26T22:51:58.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:58.500+0000 7efc2b200640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7efc24160d90 con 0x7efc2406f420 2024-05-26T22:51:58.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:58.500+0000 7efc2b200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7efc24160460 con 0x7efc240cf590 2024-05-26T22:51:58.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:58.500+0000 7efc23400640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7efc2406f420 0x7efc240cfd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:58.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:58.501+0000 7efc29200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efc240cf590 0x7efc240d0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:58.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:58.501+0000 7efc23e00640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7efc240b5450 0x7efc240b43c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.131:42158/0 (socket says 172.21.15.131:42158) 2024-05-26T22:51:58.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:58.501+0000 7efc23e00640 1 -- 172.21.15.131:0/1042945959 learned_addr learned my addr 172.21.15.131:0/1042945959 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:51:58.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:58.502+0000 7efc23400640 1 -- 172.21.15.131:0/1042945959 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efc240cf590 msgr2=0x7efc240d0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:58.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:58.502+0000 7efc23400640 1 --2- 172.21.15.131:0/1042945959 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efc240cf590 0x7efc240d0290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:58.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:58.502+0000 7efc23400640 1 -- 172.21.15.131:0/1042945959 >> v1:172.21.15.187:6789/0 conn(0x7efc240b5450 legacy=0x7efc240b43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:58.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:58.502+0000 7efc23400640 1 -- 172.21.15.131:0/1042945959 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc24160740 con 0x7efc2406f420 2024-05-26T22:51:58.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:58.503+0000 7efc23400640 1 --2- 172.21.15.131:0/1042945959 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efc2406f420 0x7efc240cfd50 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7efc0c0097a0 tx=0x7efc0c02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8eef1731aa2e21 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:58.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:58.503+0000 7efc22a00640 1 -- 172.21.15.131:0/1042945959 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc0c037070 con 0x7efc2406f420 2024-05-26T22:51:58.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:58.503+0000 7efc22a00640 1 -- 172.21.15.131:0/1042945959 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc0c032440 con 0x7efc2406f420 2024-05-26T22:51:58.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:58.503+0000 7efc22a00640 1 -- 172.21.15.131:0/1042945959 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc0c032810 con 0x7efc2406f420 2024-05-26T22:51:58.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:58.504+0000 7efc2b200640 1 -- 172.21.15.131:0/1042945959 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efc2406f420 msgr2=0x7efc240cfd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:58.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:58.504+0000 7efc2b200640 1 --2- 172.21.15.131:0/1042945959 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efc2406f420 0x7efc240cfd50 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7efc0c0097a0 tx=0x7efc0c02a180 comp rx=0 tx=0).stop 2024-05-26T22:51:58.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:58.504+0000 7efc2b200640 1 -- 172.21.15.131:0/1042945959 shutdown_connections 2024-05-26T22:51:58.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:58.504+0000 7efc2b200640 1 --2- 172.21.15.131:0/1042945959 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efc240cf590 0x7efc240d0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:58.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:58.504+0000 7efc2b200640 1 --2- 172.21.15.131:0/1042945959 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efc2406f420 0x7efc240cfd50 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:58.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:58.504+0000 7efc2b200640 1 -- 172.21.15.131:0/1042945959 >> 172.21.15.131:0/1042945959 conn(0x7efc2406bc90 msgr2=0x7efc240b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:58.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:58.504+0000 7efc2b200640 1 -- 172.21.15.131:0/1042945959 shutdown_connections 2024-05-26T22:51:58.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:58.504+0000 7efc2b200640 1 -- 172.21.15.131:0/1042945959 wait complete. 2024-05-26T22:51:58.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:58.506+0000 7efc2b200640 1 Processor -- start 2024-05-26T22:51:58.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:58.506+0000 7efc2b200640 1 -- start start 2024-05-26T22:51:58.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:58.508+0000 7efc2b200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efc2406f420 0x7efc24072610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:58.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:58.508+0000 7efc2b200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efc240b5450 0x7efc240c9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:58.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:58.508+0000 7efc2b200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efc240cf590 0x7efc2415a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:58.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:58.508+0000 7efc2b200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7efc2415fde0 con 0x7efc240b5450 2024-05-26T22:51:58.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:58.508+0000 7efc2b200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7efc240b5830 con 0x7efc2406f420 2024-05-26T22:51:58.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:58.508+0000 7efc2b200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7efc2415ff60 con 0x7efc240cf590 2024-05-26T22:51:58.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:58.508+0000 7efc23400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efc240b5450 0x7efc240c9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:58.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:58.508+0000 7efc23400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efc240b5450 0x7efc240c9ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50920/0 (socket says 172.21.15.131:50920) 2024-05-26T22:51:58.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:58.508+0000 7efc23400640 1 -- 172.21.15.131:0/199642242 learned_addr learned my addr 172.21.15.131:0/199642242 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:58.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:58.508+0000 7efc29200640 1 --2- 172.21.15.131:0/199642242 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efc240cf590 0x7efc2415a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:58.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:58.508+0000 7efc23e00640 1 --2- 172.21.15.131:0/199642242 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efc2406f420 0x7efc24072610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:58.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:58.508+0000 7efc23400640 1 -- 172.21.15.131:0/199642242 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efc240cf590 msgr2=0x7efc2415a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:58.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:58.508+0000 7efc23400640 1 --2- 172.21.15.131:0/199642242 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efc240cf590 0x7efc2415a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:58.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:58.508+0000 7efc23400640 1 -- 172.21.15.131:0/199642242 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efc2406f420 msgr2=0x7efc24072610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:58.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:58.508+0000 7efc23400640 1 --2- 172.21.15.131:0/199642242 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efc2406f420 0x7efc24072610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:58.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:58.508+0000 7efc23400640 1 -- 172.21.15.131:0/199642242 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc2409ac70 con 0x7efc240b5450 2024-05-26T22:51:58.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:58.508+0000 7efc23e00640 1 --2- 172.21.15.131:0/199642242 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efc2406f420 0x7efc24072610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:51:58.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:58.509+0000 7efc23400640 1 --2- 172.21.15.131:0/199642242 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efc240b5450 0x7efc240c9ca0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7efc0c0098c0 tx=0x7efc0c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:58.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:58.509+0000 7efc1be00640 1 -- 172.21.15.131:0/199642242 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc0c037040 con 0x7efc240b5450 2024-05-26T22:51:58.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:58.509+0000 7efc1be00640 1 -- 172.21.15.131:0/199642242 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc0c030500 con 0x7efc240b5450 2024-05-26T22:51:58.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:58.509+0000 7efc2b200640 1 -- 172.21.15.131:0/199642242 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc240cacb0 con 0x7efc240b5450 2024-05-26T22:51:58.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:58.509+0000 7efc1be00640 1 -- 172.21.15.131:0/199642242 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc0c0308d0 con 0x7efc240b5450 2024-05-26T22:51:58.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:58.509+0000 7efc2b200640 1 -- 172.21.15.131:0/199642242 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc240ca6f0 con 0x7efc240b5450 2024-05-26T22:51:58.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:58.510+0000 7efc2b200640 1 -- 172.21.15.131:0/199642242 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc24074e90 con 0x7efc240b5450 2024-05-26T22:51:58.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:58.511+0000 7efc1be00640 1 -- 172.21.15.131:0/199642242 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7efc0c044020 con 0x7efc240b5450 2024-05-26T22:51:58.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:58.512+0000 7efc1be00640 1 --2- 172.21.15.131:0/199642242 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7efc0006be30 0x7efc0006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:58.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:58.512+0000 7efc1be00640 1 -- 172.21.15.131:0/199642242 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7efc0c0ab160 con 0x7efc240b5450 2024-05-26T22:51:58.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:58.513+0000 7efc23e00640 1 --2- 172.21.15.131:0/199642242 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7efc0006be30 0x7efc0006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:58.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:58.513+0000 7efc23e00640 1 --2- 172.21.15.131:0/199642242 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7efc0006be30 0x7efc0006e2f0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7efc14005c00 tx=0x7efc14009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:58.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:58.520+0000 7efc1be00640 1 -- 172.21.15.131:0/199642242 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7efc0c079b00 con 0x7efc240b5450 2024-05-26T22:51:58.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:58.759+0000 7efc2b200640 1 -- 172.21.15.131:0/199642242 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7efc24073350 con 0x7efc240b5450 2024-05-26T22:51:59.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:59.346+0000 7efc1be00640 1 -- 172.21.15.131:0/199642242 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v50) ==== 105+0+0 (secure 0 0 0) 0x7efc0c079400 con 0x7efc240b5450 2024-05-26T22:51:59.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:59.351+0000 7efc2b200640 1 -- 172.21.15.131:0/199642242 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7efc0006be30 msgr2=0x7efc0006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:59.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:59.351+0000 7efc2b200640 1 --2- 172.21.15.131:0/199642242 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7efc0006be30 0x7efc0006e2f0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7efc14005c00 tx=0x7efc14009040 comp rx=0 tx=0).stop 2024-05-26T22:51:59.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:59.351+0000 7efc2b200640 1 -- 172.21.15.131:0/199642242 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efc240b5450 msgr2=0x7efc240c9ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:59.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:59.351+0000 7efc2b200640 1 --2- 172.21.15.131:0/199642242 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efc240b5450 0x7efc240c9ca0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7efc0c0098c0 tx=0x7efc0c0046c0 comp rx=0 tx=0).stop 2024-05-26T22:51:59.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:59.351+0000 7efc2b200640 1 -- 172.21.15.131:0/199642242 shutdown_connections 2024-05-26T22:51:59.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:59.351+0000 7efc2b200640 1 --2- 172.21.15.131:0/199642242 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efc240cf590 0x7efc2415a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:59.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:59.351+0000 7efc2b200640 1 --2- 172.21.15.131:0/199642242 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7efc0006be30 0x7efc0006e2f0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:59.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:59.351+0000 7efc2b200640 1 --2- 172.21.15.131:0/199642242 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efc240b5450 0x7efc240c9ca0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:59.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:59.351+0000 7efc2b200640 1 --2- 172.21.15.131:0/199642242 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efc2406f420 0x7efc24072610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:59.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:59.351+0000 7efc2b200640 1 -- 172.21.15.131:0/199642242 >> 172.21.15.131:0/199642242 conn(0x7efc2406bc90 msgr2=0x7efc240b1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:59.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:59.352+0000 7efc2b200640 1 -- 172.21.15.131:0/199642242 shutdown_connections 2024-05-26T22:51:59.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:59.352+0000 7efc2b200640 1 -- 172.21.15.131:0/199642242 wait complete. 2024-05-26T22:51:59.378 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-26T22:51:59.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:59.553+0000 7f4a9d400640 1 Processor -- start 2024-05-26T22:51:59.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:59.553+0000 7f4a9d400640 1 -- start start 2024-05-26T22:51:59.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:59.554+0000 7f4a9d400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f4a9806e8c0 0x7f4a9815d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:59.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:59.554+0000 7f4a9d400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4a980cc7f0 0x7f4a980ccca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:59.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:59.554+0000 7f4a9d400640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a98160620 con 0x7f4a98156750 2024-05-26T22:51:59.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:59.554+0000 7f4a9d400640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f4a9815cbd0 con 0x7f4a9806e8c0 2024-05-26T22:51:59.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:59.554+0000 7f4a9d400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f4a98160900 con 0x7f4a980cc7f0 2024-05-26T22:51:59.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:59.555+0000 7f4a8fe00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f4a9806e8c0 0x7f4a9815d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:59.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:59.555+0000 7f4a97e00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f4a98156750 0x7f4a9815fd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:56018/0 (socket says 172.21.15.131:56018) 2024-05-26T22:51:59.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:59.555+0000 7f4a97e00640 1 -- 172.21.15.131:0/4174823861 learned_addr learned my addr 172.21.15.131:0/4174823861 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:51:59.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:59.555+0000 7f4a9ca00640 1 --2- 172.21.15.131:0/4174823861 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4a980cc7f0 0x7f4a980ccca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:59.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:59.555+0000 7f4a8fe00640 1 -- 172.21.15.131:0/4174823861 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4a980cc7f0 msgr2=0x7f4a980ccca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:51:59.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:59.555+0000 7f4a8fe00640 1 --2- 172.21.15.131:0/4174823861 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4a980cc7f0 0x7f4a980ccca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:59.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:59.555+0000 7f4a8fe00640 1 -- 172.21.15.131:0/4174823861 >> v1:172.21.15.131:6789/0 conn(0x7f4a98156750 legacy=0x7f4a9815fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:59.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:59.555+0000 7f4a8fe00640 1 -- 172.21.15.131:0/4174823861 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a98160be0 con 0x7f4a9806e8c0 2024-05-26T22:51:59.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:59.555+0000 7f4a97400640 1 -- 172.21.15.131:0/4174823861 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2491378724 0 0) 0x7f4a98160620 con 0x7f4a98156750 2024-05-26T22:51:59.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:59.555+0000 7f4a9ca00640 1 --2- 172.21.15.131:0/4174823861 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4a980cc7f0 0x7f4a980ccca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:51:59.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:59.556+0000 7f4a8fe00640 1 --2- 172.21.15.131:0/4174823861 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a9806e8c0 0x7f4a9815d060 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f4a800097a0 tx=0x7f4a8002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc322020acf14cd6 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:59.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:59.557+0000 7f4a97400640 1 -- 172.21.15.131:0/4174823861 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a80037070 con 0x7f4a9806e8c0 2024-05-26T22:51:59.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:59.557+0000 7f4a97400640 1 -- 172.21.15.131:0/4174823861 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a80032440 con 0x7f4a9806e8c0 2024-05-26T22:51:59.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:59.557+0000 7f4a97400640 1 -- 172.21.15.131:0/4174823861 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a80032810 con 0x7f4a9806e8c0 2024-05-26T22:51:59.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:59.557+0000 7f4a9d400640 1 -- 172.21.15.131:0/4174823861 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a9806e8c0 msgr2=0x7f4a9815d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:59.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:59.557+0000 7f4a9d400640 1 --2- 172.21.15.131:0/4174823861 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a9806e8c0 0x7f4a9815d060 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f4a800097a0 tx=0x7f4a8002a180 comp rx=0 tx=0).stop 2024-05-26T22:51:59.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:59.557+0000 7f4a9d400640 1 -- 172.21.15.131:0/4174823861 shutdown_connections 2024-05-26T22:51:59.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:59.557+0000 7f4a9d400640 1 --2- 172.21.15.131:0/4174823861 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4a980cc7f0 0x7f4a980ccca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:59.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:59.557+0000 7f4a9d400640 1 --2- 172.21.15.131:0/4174823861 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a9806e8c0 0x7f4a9815d060 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:59.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:59.557+0000 7f4a9d400640 1 -- 172.21.15.131:0/4174823861 >> 172.21.15.131:0/4174823861 conn(0x7f4a980cff40 msgr2=0x7f4a980d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:59.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:59.557+0000 7f4a9d400640 1 -- 172.21.15.131:0/4174823861 shutdown_connections 2024-05-26T22:51:59.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:59.557+0000 7f4a9d400640 1 -- 172.21.15.131:0/4174823861 wait complete. 2024-05-26T22:51:59.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:59.559+0000 7f4a9d400640 1 Processor -- start 2024-05-26T22:51:59.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:59.559+0000 7f4a9d400640 1 -- start start 2024-05-26T22:51:59.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:59.560+0000 7f4a9d400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a9806e8c0 0x7f4a980714b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:59.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:59.560+0000 7f4a9d400640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4a980cc7f0 0x7f4a98157d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:59.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:59.560+0000 7f4a9d400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4a98156750 0x7f4a98159520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:59.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:59.560+0000 7f4a9d400640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f4a9815d5a0 con 0x7f4a9806e8c0 2024-05-26T22:51:59.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:59.560+0000 7f4a9d400640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4a98156b30 con 0x7f4a980cc7f0 2024-05-26T22:51:59.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:59.560+0000 7f4a9d400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f4a9815d720 con 0x7f4a98156750 2024-05-26T22:51:59.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:59.560+0000 7f4a97e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a9806e8c0 0x7f4a980714b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:59.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:59.560+0000 7f4a97e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a9806e8c0 0x7f4a980714b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50942/0 (socket says 172.21.15.131:50942) 2024-05-26T22:51:59.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:59.560+0000 7f4a97e00640 1 -- 172.21.15.131:0/2286609848 learned_addr learned my addr 172.21.15.131:0/2286609848 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:51:59.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:59.561+0000 7f4a97e00640 1 -- 172.21.15.131:0/2286609848 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4a98156750 msgr2=0x7f4a98159520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:59.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:59.561+0000 7f4a97e00640 1 --2- 172.21.15.131:0/2286609848 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4a98156750 0x7f4a98159520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:59.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:59.561+0000 7f4a97e00640 1 -- 172.21.15.131:0/2286609848 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4a980cc7f0 msgr2=0x7f4a98157d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:59.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:59.561+0000 7f4a97e00640 1 --2- 172.21.15.131:0/2286609848 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4a980cc7f0 0x7f4a98157d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:59.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:59.561+0000 7f4a97e00640 1 -- 172.21.15.131:0/2286609848 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a98158480 con 0x7f4a9806e8c0 2024-05-26T22:51:59.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:59.561+0000 7f4a97e00640 1 --2- 172.21.15.131:0/2286609848 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a9806e8c0 0x7f4a980714b0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f4a8800c460 tx=0x7f4a8800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:59.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:59.561+0000 7f4a94c00640 1 -- 172.21.15.131:0/2286609848 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a88018070 con 0x7f4a9806e8c0 2024-05-26T22:51:59.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:59.561+0000 7f4a94c00640 1 -- 172.21.15.131:0/2286609848 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a88007500 con 0x7f4a9806e8c0 2024-05-26T22:51:59.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:59.561+0000 7f4a94c00640 1 -- 172.21.15.131:0/2286609848 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a88004a20 con 0x7f4a9806e8c0 2024-05-26T22:51:59.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:59.561+0000 7f4a9d400640 1 -- 172.21.15.131:0/2286609848 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a98158760 con 0x7f4a9806e8c0 2024-05-26T22:51:59.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:59.561+0000 7f4a9d400640 1 -- 172.21.15.131:0/2286609848 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a98158d20 con 0x7f4a9806e8c0 2024-05-26T22:51:59.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:59.562+0000 7f4a9d400640 1 -- 172.21.15.131:0/2286609848 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a980694e0 con 0x7f4a9806e8c0 2024-05-26T22:51:59.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:59.563+0000 7f4a94c00640 1 -- 172.21.15.131:0/2286609848 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4a8801b070 con 0x7f4a9806e8c0 2024-05-26T22:51:59.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:59.564+0000 7f4a94c00640 1 --2- 172.21.15.131:0/2286609848 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f4a7406be30 0x7f4a7406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:51:59.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:59.564+0000 7f4a94c00640 1 -- 172.21.15.131:0/2286609848 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f4a8808a080 con 0x7f4a9806e8c0 2024-05-26T22:51:59.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:59.565+0000 7f4a8fe00640 1 --2- 172.21.15.131:0/2286609848 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f4a7406be30 0x7f4a7406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:51:59.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:59.565+0000 7f4a8fe00640 1 --2- 172.21.15.131:0/2286609848 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f4a7406be30 0x7f4a7406e2f0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f4a80034000 tx=0x7f4a8002c000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:51:59.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:59.569+0000 7f4a94c00640 1 -- 172.21.15.131:0/2286609848 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f4a88058ac0 con 0x7f4a9806e8c0 2024-05-26T22:51:59.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:59.806+0000 7f4a9d400640 1 -- 172.21.15.131:0/2286609848 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4a9806e650 con 0x7f4a9806e8c0 2024-05-26T22:51:59.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:59.807+0000 7f4a94c00640 1 -- 172.21.15.131:0/2286609848 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) ==== 94+0+5206 (secure 0 0 0) 0x7f4a880583c0 con 0x7f4a9806e8c0 2024-05-26T22:51:59.807 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:51:59.808 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":51,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4832,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/1327386245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":1327386245},{"type":"v1","addr":"172.21.15.187:6835","nonce":1327386245}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":44},{"gid":6378,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6836/2737657374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6835","nonce":2737657374},{"type":"v1","addr":"172.21.15.131:6836","nonce":2737657374}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-26T22:51:39.114644+0000","modified":"2024-05-26T22:51:59.352815+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1],"up":{"mds_0":6390,"mds_1":6381},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6381":{"gid":6381,"name":"c","rank":1,"incarnation":51,"state":"up:creating","state_seq":1,"addr":"172.21.15.131:6837/640544945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":640544945},{"type":"v1","addr":"172.21.15.131:6837","nonce":640544945}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6390":{"gid":6390,"name":"d","rank":0,"incarnation":45,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6833/3300872633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3300872633},{"type":"v1","addr":"172.21.15.187:6833","nonce":3300872633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6390,"qdb_cluster":[6390]},"id":4}]} 2024-05-26T22:51:59.808 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 51 2024-05-26T22:51:59.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:59.810+0000 7f4a9d400640 1 -- 172.21.15.131:0/2286609848 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f4a7406be30 msgr2=0x7f4a7406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:59.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:59.810+0000 7f4a9d400640 1 --2- 172.21.15.131:0/2286609848 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f4a7406be30 0x7f4a7406e2f0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f4a80034000 tx=0x7f4a8002c000 comp rx=0 tx=0).stop 2024-05-26T22:51:59.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:59.810+0000 7f4a9d400640 1 -- 172.21.15.131:0/2286609848 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a9806e8c0 msgr2=0x7f4a980714b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:51:59.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:59.810+0000 7f4a9d400640 1 --2- 172.21.15.131:0/2286609848 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a9806e8c0 0x7f4a980714b0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f4a8800c460 tx=0x7f4a8800c920 comp rx=0 tx=0).stop 2024-05-26T22:51:59.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:59.810+0000 7f4a9d400640 1 -- 172.21.15.131:0/2286609848 shutdown_connections 2024-05-26T22:51:59.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:59.810+0000 7f4a9d400640 1 --2- 172.21.15.131:0/2286609848 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4a98156750 0x7f4a98159520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:59.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:59.810+0000 7f4a9d400640 1 --2- 172.21.15.131:0/2286609848 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f4a7406be30 0x7f4a7406e2f0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:59.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:59.810+0000 7f4a9d400640 1 --2- 172.21.15.131:0/2286609848 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4a980cc7f0 0x7f4a98157d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:59.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:59.810+0000 7f4a9d400640 1 --2- 172.21.15.131:0/2286609848 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a9806e8c0 0x7f4a980714b0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:51:59.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:59.810+0000 7f4a9d400640 1 -- 172.21.15.131:0/2286609848 >> 172.21.15.131:0/2286609848 conn(0x7f4a980cff40 msgr2=0x7f4a980cf0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:51:59.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:59.811+0000 7f4a9d400640 1 -- 172.21.15.131:0/2286609848 shutdown_connections 2024-05-26T22:51:59.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:51:59.811+0000 7f4a9d400640 1 -- 172.21.15.131:0/2286609848 wait complete. 2024-05-26T22:52:00.231 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 51, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-26T22:51:39.114644+0000', 'modified': '2024-05-26T22:51:59.352815+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1], 'up': {'mds_0': 6390, 'mds_1': 6381}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6381': {'gid': 6381, 'name': 'c', 'rank': 1, 'incarnation': 51, 'state': 'up:creating', 'state_seq': 1, 'addr': '172.21.15.131:6837/640544945', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6834', 'nonce': 640544945}, {'type': 'v1', 'addr': '172.21.15.131:6837', 'nonce': 640544945}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_6390': {'gid': 6390, 'name': 'd', 'rank': 0, 'incarnation': 45, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.187:6833/3300872633', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.187:6832', 'nonce': 3300872633}, {'type': 'v1', 'addr': '172.21.15.187:6833', 'nonce': 3300872633}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6390, 'qdb_cluster': [6390]} 2024-05-26T22:52:00.231 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6381:up:creating 2024-05-26T22:52:00.608 INFO:tasks.ceph.mds.d.smithi187.stderr:2024-05-26T22:52:00.608+0000 7fbf69e00640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-05-26T22:52:01.232 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-26T22:52:01.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:01.445+0000 7f025b200640 1 Processor -- start 2024-05-26T22:52:01.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:01.445+0000 7f025b200640 1 -- start start 2024-05-26T22:52:01.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:01.447+0000 7f025b200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f025c14c090 0x7f025c0b2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:01.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:01.447+0000 7f025b200640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f025c06df90 0x7f025c071160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:01.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:01.447+0000 7f025b200640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f025c071990 con 0x7f025c0cc760 2024-05-26T22:52:01.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:01.447+0000 7f025b200640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f025c071f50 con 0x7f025c14c090 2024-05-26T22:52:01.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:01.447+0000 7f025b200640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f025c0ce150 con 0x7f025c06df90 2024-05-26T22:52:01.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:01.447+0000 7f0259e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f025c14c090 0x7f025c0b2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:01.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:01.447+0000 7f0259e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f025c14c090 0x7f025c0b2e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50950/0 (socket says 172.21.15.131:50950) 2024-05-26T22:52:01.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:01.447+0000 7f025a800640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f025c0cc760 0x7f025c0ce5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:56024/0 (socket says 172.21.15.131:56024) 2024-05-26T22:52:01.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:01.447+0000 7f0259e00640 1 -- 172.21.15.131:0/3291048006 learned_addr learned my addr 172.21.15.131:0/3291048006 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:52:01.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:01.448+0000 7f0259400640 1 --2- 172.21.15.131:0/3291048006 >> v2:172.21.15.187:3300/0 conn(0x7f025c06df90 0x7f025c071160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:01.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:01.448+0000 7f0259e00640 1 -- 172.21.15.131:0/3291048006 >> v2:172.21.15.187:3300/0 conn(0x7f025c06df90 msgr2=0x7f025c071160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:01.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:01.448+0000 7f0259e00640 1 --2- 172.21.15.131:0/3291048006 >> v2:172.21.15.187:3300/0 conn(0x7f025c06df90 0x7f025c071160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:01.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:01.448+0000 7f0259e00640 1 -- 172.21.15.131:0/3291048006 >> v1:172.21.15.131:6789/0 conn(0x7f025c0cc760 legacy=0x7f025c0ce5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:01.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:01.448+0000 7f0259e00640 1 -- 172.21.15.131:0/3291048006 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f025c071c70 con 0x7f025c14c090 2024-05-26T22:52:01.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:01.448+0000 7f0258a00640 1 -- 172.21.15.131:0/3291048006 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1138979993 0 0) 0x7f025c071990 con 0x7f025c0cc760 2024-05-26T22:52:01.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:01.449+0000 7f0259e00640 1 --2- 172.21.15.131:0/3291048006 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f025c14c090 0x7f025c0b2e10 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f025c06e4a0 tx=0x7f024c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34e8a336d6c50c78 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:01.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:01.449+0000 7f0258a00640 1 -- 172.21.15.131:0/3291048006 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f024c037070 con 0x7f025c14c090 2024-05-26T22:52:01.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:01.449+0000 7f0258a00640 1 -- 172.21.15.131:0/3291048006 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f024c032440 con 0x7f025c14c090 2024-05-26T22:52:01.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:01.449+0000 7f0258a00640 1 -- 172.21.15.131:0/3291048006 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f024c032810 con 0x7f025c14c090 2024-05-26T22:52:01.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:01.450+0000 7f025b200640 1 -- 172.21.15.131:0/3291048006 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f025c14c090 msgr2=0x7f025c0b2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:01.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:01.450+0000 7f025b200640 1 --2- 172.21.15.131:0/3291048006 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f025c14c090 0x7f025c0b2e10 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f025c06e4a0 tx=0x7f024c02a1e0 comp rx=0 tx=0).stop 2024-05-26T22:52:01.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:01.450+0000 7f025b200640 1 -- 172.21.15.131:0/3291048006 shutdown_connections 2024-05-26T22:52:01.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:01.450+0000 7f025b200640 1 --2- 172.21.15.131:0/3291048006 >> v2:172.21.15.187:3300/0 conn(0x7f025c06df90 0x7f025c071160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:01.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:01.450+0000 7f025b200640 1 --2- 172.21.15.131:0/3291048006 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f025c14c090 0x7f025c0b2e10 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:01.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:01.450+0000 7f025b200640 1 -- 172.21.15.131:0/3291048006 >> 172.21.15.131:0/3291048006 conn(0x7f025c0cff40 msgr2=0x7f025c0d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:52:01.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:01.450+0000 7f025b200640 1 -- 172.21.15.131:0/3291048006 shutdown_connections 2024-05-26T22:52:01.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:01.450+0000 7f025b200640 1 -- 172.21.15.131:0/3291048006 wait complete. 2024-05-26T22:52:01.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:01.452+0000 7f025b200640 1 Processor -- start 2024-05-26T22:52:01.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:01.452+0000 7f025b200640 1 -- start start 2024-05-26T22:52:01.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:01.453+0000 7f025b200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f025c06df90 0x7f025c0a9460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:01.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:01.453+0000 7f025b200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f025c0cc760 0x7f025c1457c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:01.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:01.453+0000 7f025b200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f025c14c090 0x7f025c1476d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:01.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:01.453+0000 7f025b200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f025c14c470 con 0x7f025c06df90 2024-05-26T22:52:01.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:01.453+0000 7f025b200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f025c0716a0 con 0x7f025c0cc760 2024-05-26T22:52:01.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:01.453+0000 7f025b200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f025c14c5f0 con 0x7f025c14c090 2024-05-26T22:52:01.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:01.453+0000 7f0259e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f025c06df90 0x7f025c0a9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:01.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:01.453+0000 7f0259e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f025c06df90 0x7f025c0a9460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50956/0 (socket says 172.21.15.131:50956) 2024-05-26T22:52:01.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:01.453+0000 7f0259e00640 1 -- 172.21.15.131:0/809383210 learned_addr learned my addr 172.21.15.131:0/809383210 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:52:01.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:01.453+0000 7f0259400640 1 --2- 172.21.15.131:0/809383210 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f025c0cc760 0x7f025c1457c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:01.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:01.453+0000 7f0259e00640 1 -- 172.21.15.131:0/809383210 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f025c14c090 msgr2=0x7f025c1476d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:01.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:01.453+0000 7f0259e00640 1 --2- 172.21.15.131:0/809383210 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f025c14c090 0x7f025c1476d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:01.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:01.453+0000 7f0259e00640 1 -- 172.21.15.131:0/809383210 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f025c0cc760 msgr2=0x7f025c1457c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:01.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:01.453+0000 7f0259e00640 1 --2- 172.21.15.131:0/809383210 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f025c0cc760 0x7f025c1457c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:01.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:01.453+0000 7f0259e00640 1 -- 172.21.15.131:0/809383210 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f025c0b07c0 con 0x7f025c06df90 2024-05-26T22:52:01.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:01.454+0000 7f0259e00640 1 --2- 172.21.15.131:0/809383210 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f025c06df90 0x7f025c0a9460 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f024c0090b0 tx=0x7f024c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:01.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:01.454+0000 7f0252000640 1 -- 172.21.15.131:0/809383210 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f024c037040 con 0x7f025c06df90 2024-05-26T22:52:01.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:01.454+0000 7f0252000640 1 -- 172.21.15.131:0/809383210 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f024c004230 con 0x7f025c06df90 2024-05-26T22:52:01.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:01.454+0000 7f0252000640 1 -- 172.21.15.131:0/809383210 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f024c003610 con 0x7f025c06df90 2024-05-26T22:52:01.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:01.454+0000 7f025b200640 1 -- 172.21.15.131:0/809383210 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f025c0b0aa0 con 0x7f025c06df90 2024-05-26T22:52:01.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:01.454+0000 7f025b200640 1 -- 172.21.15.131:0/809383210 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f025c0b1060 con 0x7f025c06df90 2024-05-26T22:52:01.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:01.455+0000 7f025b200640 1 -- 172.21.15.131:0/809383210 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f025c0694e0 con 0x7f025c06df90 2024-05-26T22:52:01.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:01.456+0000 7f0252000640 1 -- 172.21.15.131:0/809383210 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f024c0037b0 con 0x7f025c06df90 2024-05-26T22:52:01.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:01.457+0000 7f0252000640 1 --2- 172.21.15.131:0/809383210 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f023806be60 0x7f023806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:01.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:01.458+0000 7f0252000640 1 -- 172.21.15.131:0/809383210 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f024c0abc30 con 0x7f025c06df90 2024-05-26T22:52:01.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:01.458+0000 7f0259400640 1 --2- 172.21.15.131:0/809383210 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f023806be60 0x7f023806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:01.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:01.458+0000 7f0259400640 1 --2- 172.21.15.131:0/809383210 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f023806be60 0x7f023806e320 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f025c0b23c0 tx=0x7f0244003990 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:01.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:01.462+0000 7f0252000640 1 -- 172.21.15.131:0/809383210 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f024c07a670 con 0x7f025c06df90 2024-05-26T22:52:01.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:01.701+0000 7f025b200640 1 -- 172.21.15.131:0/809383210 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f025c148c80 con 0x7f025c06df90 2024-05-26T22:52:01.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:01.703+0000 7f0252000640 1 -- 172.21.15.131:0/809383210 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) ==== 94+0+5228 (secure 0 0 0) 0x7f024c079f70 con 0x7f025c06df90 2024-05-26T22:52:01.703 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:52:01.703 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":54,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4832,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/1327386245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":1327386245},{"type":"v1","addr":"172.21.15.187:6835","nonce":1327386245}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-26T22:51:39.114644+0000","modified":"2024-05-26T22:52:01.367610+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6390,"mds_1":6381,"mds_2":6378},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6378":{"gid":6378,"name":"a","rank":2,"incarnation":53,"state":"up:active","state_seq":8,"addr":"172.21.15.131:6836/2737657374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6835","nonce":2737657374},{"type":"v1","addr":"172.21.15.131:6836","nonce":2737657374}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6381":{"gid":6381,"name":"c","rank":1,"incarnation":51,"state":"up:active","state_seq":8,"addr":"172.21.15.131:6837/640544945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":640544945},{"type":"v1","addr":"172.21.15.131:6837","nonce":640544945}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6390":{"gid":6390,"name":"d","rank":0,"incarnation":45,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6833/3300872633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3300872633},{"type":"v1","addr":"172.21.15.187:6833","nonce":3300872633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6390,"qdb_cluster":[6378,6381,6390]},"id":4}]} 2024-05-26T22:52:01.703 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 54 2024-05-26T22:52:01.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:01.705+0000 7f025b200640 1 -- 172.21.15.131:0/809383210 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f023806be60 msgr2=0x7f023806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:01.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:01.705+0000 7f025b200640 1 --2- 172.21.15.131:0/809383210 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f023806be60 0x7f023806e320 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f025c0b23c0 tx=0x7f0244003990 comp rx=0 tx=0).stop 2024-05-26T22:52:01.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:01.705+0000 7f025b200640 1 -- 172.21.15.131:0/809383210 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f025c06df90 msgr2=0x7f025c0a9460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:01.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:01.705+0000 7f025b200640 1 --2- 172.21.15.131:0/809383210 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f025c06df90 0x7f025c0a9460 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f024c0090b0 tx=0x7f024c0046c0 comp rx=0 tx=0).stop 2024-05-26T22:52:01.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:01.705+0000 7f025b200640 1 -- 172.21.15.131:0/809383210 shutdown_connections 2024-05-26T22:52:01.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:01.705+0000 7f025b200640 1 --2- 172.21.15.131:0/809383210 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f025c14c090 0x7f025c1476d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:01.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:01.705+0000 7f025b200640 1 --2- 172.21.15.131:0/809383210 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f023806be60 0x7f023806e320 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:01.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:01.705+0000 7f025b200640 1 --2- 172.21.15.131:0/809383210 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f025c0cc760 0x7f025c1457c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:01.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:01.705+0000 7f025b200640 1 --2- 172.21.15.131:0/809383210 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f025c06df90 0x7f025c0a9460 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:01.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:01.705+0000 7f025b200640 1 -- 172.21.15.131:0/809383210 >> 172.21.15.131:0/809383210 conn(0x7f025c0cff40 msgr2=0x7f025c06d3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:52:01.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:01.706+0000 7f025b200640 1 -- 172.21.15.131:0/809383210 shutdown_connections 2024-05-26T22:52:01.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:01.706+0000 7f025b200640 1 -- 172.21.15.131:0/809383210 wait complete. 2024-05-26T22:52:02.126 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 54, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-26T22:51:39.114644+0000', 'modified': '2024-05-26T22:52:01.367610+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 6390, 'mds_1': 6381, 'mds_2': 6378}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6378': {'gid': 6378, 'name': 'a', 'rank': 2, 'incarnation': 53, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.131:6836/2737657374', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6835', 'nonce': 2737657374}, {'type': 'v1', 'addr': '172.21.15.131:6836', 'nonce': 2737657374}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_6381': {'gid': 6381, 'name': 'c', 'rank': 1, 'incarnation': 51, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.131:6837/640544945', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6834', 'nonce': 640544945}, {'type': 'v1', 'addr': '172.21.15.131:6837', 'nonce': 640544945}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_6390': {'gid': 6390, 'name': 'd', 'rank': 0, 'incarnation': 45, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.187:6833/3300872633', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.187:6832', 'nonce': 3300872633}, {'type': 'v1', 'addr': '172.21.15.187:6833', 'nonce': 3300872633}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6390, 'qdb_cluster': [6378, 6381, 6390]} 2024-05-26T22:52:02.127 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-05-26T22:52:02.127 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-05-26T22:52:02.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.331+0000 7f96f3200640 1 Processor -- start 2024-05-26T22:52:02.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.332+0000 7f96f3200640 1 -- start start 2024-05-26T22:52:02.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.333+0000 7f96f3200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f96ec0b5450 0x7f96ec0b43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:02.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.333+0000 7f96f3200640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f96ec0d0460 0x7f96ec06d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:02.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.333+0000 7f96f3200640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96ec0b4900 con 0x7f96ec06f420 2024-05-26T22:52:02.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.333+0000 7f96f3200640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f96ec160d90 con 0x7f96ec0b5450 2024-05-26T22:52:02.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.333+0000 7f96f3200640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f96ec160180 con 0x7f96ec0d0460 2024-05-26T22:52:02.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.333+0000 7f96ebe00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f96ec0b5450 0x7f96ec0b43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:02.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.334+0000 7f96eb400640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f96ec06f420 0x7f96ec0cfd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:56036/0 (socket says 172.21.15.131:56036) 2024-05-26T22:52:02.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.334+0000 7f96eb400640 1 -- 172.21.15.131:0/906650194 learned_addr learned my addr 172.21.15.131:0/906650194 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:52:02.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.334+0000 7f96f1200640 1 --2- 172.21.15.131:0/906650194 >> v2:172.21.15.187:3300/0 conn(0x7f96ec0d0460 0x7f96ec06d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:02.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.334+0000 7f96ebe00640 1 -- 172.21.15.131:0/906650194 >> v2:172.21.15.187:3300/0 conn(0x7f96ec0d0460 msgr2=0x7f96ec06d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:02.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.334+0000 7f96ebe00640 1 --2- 172.21.15.131:0/906650194 >> v2:172.21.15.187:3300/0 conn(0x7f96ec0d0460 0x7f96ec06d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:02.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.334+0000 7f96ebe00640 1 -- 172.21.15.131:0/906650194 >> v1:172.21.15.131:6789/0 conn(0x7f96ec06f420 legacy=0x7f96ec0cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:02.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.334+0000 7f96ebe00640 1 -- 172.21.15.131:0/906650194 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96ec160460 con 0x7f96ec0b5450 2024-05-26T22:52:02.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.335+0000 7f96ebe00640 1 --2- 172.21.15.131:0/906650194 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96ec0b5450 0x7f96ec0b43c0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f96dc0097a0 tx=0x7f96dc02a1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97b79099ca1101af server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:02.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.335+0000 7f96eaa00640 1 -- 172.21.15.131:0/906650194 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96dc037070 con 0x7f96ec0b5450 2024-05-26T22:52:02.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.336+0000 7f96eaa00640 1 -- 172.21.15.131:0/906650194 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96dc032440 con 0x7f96ec0b5450 2024-05-26T22:52:02.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.336+0000 7f96eaa00640 1 -- 172.21.15.131:0/906650194 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96dc032810 con 0x7f96ec0b5450 2024-05-26T22:52:02.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.336+0000 7f96f3200640 1 -- 172.21.15.131:0/906650194 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96ec0b5450 msgr2=0x7f96ec0b43c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:02.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.336+0000 7f96f3200640 1 --2- 172.21.15.131:0/906650194 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96ec0b5450 0x7f96ec0b43c0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f96dc0097a0 tx=0x7f96dc02a1d0 comp rx=0 tx=0).stop 2024-05-26T22:52:02.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.336+0000 7f96f3200640 1 -- 172.21.15.131:0/906650194 shutdown_connections 2024-05-26T22:52:02.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.336+0000 7f96f3200640 1 --2- 172.21.15.131:0/906650194 >> v2:172.21.15.187:3300/0 conn(0x7f96ec0d0460 0x7f96ec06d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:02.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.336+0000 7f96f3200640 1 --2- 172.21.15.131:0/906650194 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96ec0b5450 0x7f96ec0b43c0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:02.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.336+0000 7f96f3200640 1 -- 172.21.15.131:0/906650194 >> 172.21.15.131:0/906650194 conn(0x7f96ec06bc90 msgr2=0x7f96ec0b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:52:02.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.336+0000 7f96f3200640 1 -- 172.21.15.131:0/906650194 shutdown_connections 2024-05-26T22:52:02.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.336+0000 7f96f3200640 1 -- 172.21.15.131:0/906650194 wait complete. 2024-05-26T22:52:02.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.338+0000 7f96f3200640 1 Processor -- start 2024-05-26T22:52:02.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.338+0000 7f96f3200640 1 -- start start 2024-05-26T22:52:02.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.339+0000 7f96f3200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96ec06f420 0x7f96ec14f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:02.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.339+0000 7f96f3200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f96ec0b5450 0x7f96ec0ad160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:02.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.339+0000 7f96f3200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f96ec0d0460 0x7f96ec09ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:02.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.339+0000 7f96f3200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f96ec0b5830 con 0x7f96ec06f420 2024-05-26T22:52:02.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.339+0000 7f96f3200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f96ec160a20 con 0x7f96ec0b5450 2024-05-26T22:52:02.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.339+0000 7f96f3200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f96ec15fde0 con 0x7f96ec0d0460 2024-05-26T22:52:02.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.339+0000 7f96ebe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96ec06f420 0x7f96ec14f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:02.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.339+0000 7f96ebe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96ec06f420 0x7f96ec14f960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50972/0 (socket says 172.21.15.131:50972) 2024-05-26T22:52:02.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.340+0000 7f96ebe00640 1 -- 172.21.15.131:0/3482100822 learned_addr learned my addr 172.21.15.131:0/3482100822 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:52:02.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.340+0000 7f96eb400640 1 --2- 172.21.15.131:0/3482100822 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f96ec0b5450 0x7f96ec0ad160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:02.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.340+0000 7f96f1200640 1 --2- 172.21.15.131:0/3482100822 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f96ec0d0460 0x7f96ec09ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:02.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.340+0000 7f96ebe00640 1 -- 172.21.15.131:0/3482100822 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f96ec0d0460 msgr2=0x7f96ec09ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:02.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.340+0000 7f96ebe00640 1 --2- 172.21.15.131:0/3482100822 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f96ec0d0460 0x7f96ec09ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:02.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.340+0000 7f96ebe00640 1 -- 172.21.15.131:0/3482100822 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f96ec0b5450 msgr2=0x7f96ec0ad160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:02.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.340+0000 7f96ebe00640 1 --2- 172.21.15.131:0/3482100822 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f96ec0b5450 0x7f96ec0ad160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:02.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.340+0000 7f96ebe00640 1 -- 172.21.15.131:0/3482100822 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96ec0adb90 con 0x7f96ec06f420 2024-05-26T22:52:02.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.340+0000 7f96f1200640 1 --2- 172.21.15.131:0/3482100822 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f96ec0d0460 0x7f96ec09ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:52:02.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.340+0000 7f96ebe00640 1 --2- 172.21.15.131:0/3482100822 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96ec06f420 0x7f96ec14f960 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f96dc035040 tx=0x7f96dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:02.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.341+0000 7f96e3e00640 1 -- 172.21.15.131:0/3482100822 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96dc037040 con 0x7f96ec06f420 2024-05-26T22:52:02.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.341+0000 7f96e3e00640 1 -- 172.21.15.131:0/3482100822 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96dc030770 con 0x7f96ec06f420 2024-05-26T22:52:02.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.341+0000 7f96e3e00640 1 -- 172.21.15.131:0/3482100822 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96dc030b40 con 0x7f96ec06f420 2024-05-26T22:52:02.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.341+0000 7f96f3200640 1 -- 172.21.15.131:0/3482100822 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96ec0ade70 con 0x7f96ec06f420 2024-05-26T22:52:02.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.341+0000 7f96f3200640 1 -- 172.21.15.131:0/3482100822 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96ec0ae430 con 0x7f96ec06f420 2024-05-26T22:52:02.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.343+0000 7f96e3e00640 1 -- 172.21.15.131:0/3482100822 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f96dc044020 con 0x7f96ec06f420 2024-05-26T22:52:02.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.344+0000 7f96e3e00640 1 --2- 172.21.15.131:0/3482100822 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f96c806be30 0x7f96c806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:02.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.344+0000 7f96e3e00640 1 -- 172.21.15.131:0/3482100822 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f96dc0ab440 con 0x7f96ec06f420 2024-05-26T22:52:02.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.344+0000 7f96eb400640 1 --2- 172.21.15.131:0/3482100822 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f96c806be30 0x7f96c806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:02.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.344+0000 7f96f6014740 1 Processor -- start 2024-05-26T22:52:02.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.345+0000 7f96f6014740 1 -- start start 2024-05-26T22:52:02.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.345+0000 7f96eb400640 1 --2- 172.21.15.131:0/3482100822 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f96c806be30 0x7f96c806e2f0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f96ec1502e0 tx=0x7f96d4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:02.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.345+0000 7f96f6014740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55971d419c70 0x55971d3cc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:02.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.345+0000 7f96f6014740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55971d3fcd90 0x55971d3fc110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:02.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.345+0000 7f96f6014740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55971d3fc650 0x55971d3d1460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:02.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.346+0000 7f96f6014740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55971d334610 con 0x55971d419c70 2024-05-26T22:52:02.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.346+0000 7f96f6014740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55971d2dc5c0 con 0x55971d3fc650 2024-05-26T22:52:02.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.346+0000 7f96f6014740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x55971d3305d0 con 0x55971d3fcd90 2024-05-26T22:52:02.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.346+0000 7f96ebe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55971d419c70 0x55971d3cc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:02.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.346+0000 7f96ebe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55971d419c70 0x55971d3cc890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50976/0 (socket says 172.21.15.131:50976) 2024-05-26T22:52:02.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.346+0000 7f96ebe00640 1 -- 172.21.15.131:0/2725850277 learned_addr learned my addr 172.21.15.131:0/2725850277 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:52:02.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.346+0000 7f96eb400640 1 --2- 172.21.15.131:0/2725850277 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55971d3fcd90 0x55971d3fc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:02.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.346+0000 7f96ebe00640 1 -- 172.21.15.131:0/2725850277 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55971d3fcd90 msgr2=0x55971d3fc110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:02.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.346+0000 7f96ebe00640 1 --2- 172.21.15.131:0/2725850277 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55971d3fcd90 0x55971d3fc110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:02.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.346+0000 7f96ebe00640 1 -- 172.21.15.131:0/2725850277 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55971d3fc650 msgr2=0x55971d3d1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:02.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.346+0000 7f96f1200640 1 --2- 172.21.15.131:0/2725850277 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55971d3fc650 0x55971d3d1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:02.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.346+0000 7f96ebe00640 1 --2- 172.21.15.131:0/2725850277 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55971d3fc650 0x55971d3d1460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:02.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.346+0000 7f96ebe00640 1 -- 172.21.15.131:0/2725850277 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55971d0482d0 con 0x55971d419c70 2024-05-26T22:52:02.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.346+0000 7f96eb400640 1 --2- 172.21.15.131:0/2725850277 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55971d3fcd90 0x55971d3fc110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:52:02.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.346+0000 7f96f1200640 1 --2- 172.21.15.131:0/2725850277 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55971d3fc650 0x55971d3d1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:52:02.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.346+0000 7f96ebe00640 1 --2- 172.21.15.131:0/2725850277 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55971d419c70 0x55971d3cc890 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f96dc0800d0 tx=0x7f96dc02a780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0136f44b791ef48 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:02.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.347+0000 7f96e0c00640 1 -- 172.21.15.131:0/2725850277 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96dc079650 con 0x55971d419c70 2024-05-26T22:52:02.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.347+0000 7f96e0c00640 1 -- 172.21.15.131:0/2725850277 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96dc07da00 con 0x55971d419c70 2024-05-26T22:52:02.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.347+0000 7f96e0c00640 1 -- 172.21.15.131:0/2725850277 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96dc082230 con 0x55971d419c70 2024-05-26T22:52:02.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.347+0000 7f96f6014740 1 -- 172.21.15.131:0/2725850277 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55971d419c70 msgr2=0x55971d3cc890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:02.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.347+0000 7f96f6014740 1 --2- 172.21.15.131:0/2725850277 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55971d419c70 0x55971d3cc890 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f96dc0800d0 tx=0x7f96dc02a780 comp rx=0 tx=0).stop 2024-05-26T22:52:02.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.348+0000 7f96f6014740 1 -- 172.21.15.131:0/2725850277 shutdown_connections 2024-05-26T22:52:02.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.348+0000 7f96f6014740 1 --2- 172.21.15.131:0/2725850277 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55971d3fcd90 0x55971d3fc110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:02.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.348+0000 7f96f6014740 1 --2- 172.21.15.131:0/2725850277 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55971d3fc650 0x55971d3d1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:02.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.348+0000 7f96f6014740 1 --2- 172.21.15.131:0/2725850277 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55971d419c70 0x55971d3cc890 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:02.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.348+0000 7f96f6014740 1 -- 172.21.15.131:0/2725850277 >> 172.21.15.131:0/2725850277 conn(0x55971d3ceb00 msgr2=0x55971d417110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:52:02.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.348+0000 7f96f6014740 1 -- 172.21.15.131:0/2725850277 shutdown_connections 2024-05-26T22:52:02.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.348+0000 7f96f6014740 1 -- 172.21.15.131:0/2725850277 wait complete. 2024-05-26T22:52:02.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.348+0000 7f96f6014740 1 -- start start 2024-05-26T22:52:02.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.349+0000 7f96f6014740 1 Processor -- start 2024-05-26T22:52:02.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.350+0000 7f96f6014740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55971d3fcd90 0x55971d41a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:02.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.350+0000 7f96f6014740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55971d419c70 0x55971d41aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:02.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.350+0000 7f96f6014740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55971d426fe0 0x55971d427460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:02.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.350+0000 7f96f6014740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55971d334610 con 0x55971d3fcd90 2024-05-26T22:52:02.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.350+0000 7f96f6014740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55971d2ef910 con 0x55971d419c70 2024-05-26T22:52:02.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.350+0000 7f96f6014740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x55971d3d0a20 con 0x55971d426fe0 2024-05-26T22:52:02.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.350+0000 7f96f6014740 10 client.0 _ll_register_callbacks cb 0x55971d32b410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-26T22:52:02.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.350+0000 7f96ebe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55971d3fcd90 0x55971d41a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:02.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.350+0000 7f96ebe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55971d3fcd90 0x55971d41a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50980/0 (socket says 172.21.15.131:50980) 2024-05-26T22:52:02.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.350+0000 7f96ebe00640 1 -- 172.21.15.131:0/1065897133 learned_addr learned my addr 172.21.15.131:0/1065897133 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:52:02.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.350+0000 7f96f1200640 1 --2- 172.21.15.131:0/1065897133 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55971d426fe0 0x55971d427460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:02.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.350+0000 7f96ebe00640 1 -- 172.21.15.131:0/1065897133 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55971d426fe0 msgr2=0x55971d427460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:02.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.350+0000 7f96eb400640 1 --2- 172.21.15.131:0/1065897133 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55971d419c70 0x55971d41aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:02.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.350+0000 7f96ebe00640 1 --2- 172.21.15.131:0/1065897133 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55971d426fe0 0x55971d427460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:02.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.350+0000 7f96ebe00640 1 -- 172.21.15.131:0/1065897133 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55971d419c70 msgr2=0x55971d41aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:02.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.350+0000 7f96ebe00640 1 --2- 172.21.15.131:0/1065897133 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55971d419c70 0x55971d41aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:02.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.350+0000 7f96ebe00640 1 -- 172.21.15.131:0/1065897133 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55971d1126b0 con 0x55971d3fcd90 2024-05-26T22:52:02.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.351+0000 7f96ebe00640 1 --2- 172.21.15.131:0/1065897133 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55971d3fcd90 0x55971d41a530 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f96dc0795e0 tx=0x7f96dc009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80c0360c933a54b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:02.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.351+0000 7f96d9600640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-26T22:52:02.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.351+0000 7f96d9600640 1 -- 172.21.15.131:0/1065897133 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96dc082230 con 0x55971d3fcd90 2024-05-26T22:52:02.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.351+0000 7f96d9600640 1 -- 172.21.15.131:0/1065897133 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f96dc077950 con 0x55971d3fcd90 2024-05-26T22:52:02.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.351+0000 7f96f6014740 1 -- 172.21.15.131:0/1065897133 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96dc07da00 con 0x55971d3fcd90 2024-05-26T22:52:02.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.351+0000 7f96d9600640 1 -- 172.21.15.131:0/1065897133 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f96dc09d050 con 0x55971d3fcd90 2024-05-26T22:52:02.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.351+0000 7f96f6014740 10 client.6600 fetch_fsmap learned FSMap version 54 2024-05-26T22:52:02.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.351+0000 7f96f6014740 1 -- 172.21.15.131:0/1065897133 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) -- 0x55971d40f4f0 con 0x55971d3fcd90 2024-05-26T22:52:02.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.351+0000 7f96d9600640 1 -- 172.21.15.131:0/1065897133 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96dc07da00 con 0x55971d3fcd90 2024-05-26T22:52:02.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.351+0000 7f96d9600640 1 -- 172.21.15.131:0/1065897133 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 54) ==== 3534+0+0 (secure 0 0 0) 0x7f96dc03a030 con 0x55971d3fcd90 2024-05-26T22:52:02.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.351+0000 7f96f6014740 10 client.6600 fetch_fsmap finished waiting for FSMap version 54 2024-05-26T22:52:02.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.351+0000 7f96f6014740 10 client.6600 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-26T22:52:02.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.351+0000 7f96f6014740 20 client.6600 populate_metadata read hostname 'smithi131' 2024-05-26T22:52:02.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.351+0000 7f96f6014740 1 --2- 172.21.15.131:0/1065897133 >> [v2:172.21.15.131:6835/2737657374,v1:172.21.15.131:6836/2737657374] conn(0x55971d4279a0 0x55971d3fec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:02.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.351+0000 7f96f6014740 4 client.6600 mds_command: new command op to 6378 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-26T22:52:02.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.351+0000 7f96f6014740 1 -- 172.21.15.131:0/1065897133 --> [v2:172.21.15.131:6835/2737657374,v1:172.21.15.131:6836/2737657374] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55971d341d40 con 0x55971d4279a0 2024-05-26T22:52:02.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.352+0000 7f96eb400640 1 --2- 172.21.15.131:0/1065897133 >> [v2:172.21.15.131:6835/2737657374,v1:172.21.15.131:6836/2737657374] conn(0x55971d4279a0 0x55971d3fec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:02.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.352+0000 7f96d9600640 10 client.6600 ms_handle_connect on v2:172.21.15.131:6835/2737657374 2024-05-26T22:52:02.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.352+0000 7f96eb400640 1 --2- 172.21.15.131:0/1065897133 >> [v2:172.21.15.131:6835/2737657374,v1:172.21.15.131:6836/2737657374] conn(0x55971d4279a0 0x55971d3fec60 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b480f2d9a98e45b6 server_cookie=3182a171a8607564 in_seq=0 out_seq=0 2024-05-26T22:52:02.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.353+0000 7f96d9600640 1 -- 172.21.15.131:0/1065897133 <== mds.2 v2:172.21.15.131:6835/2737657374 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55971d341d40 con 0x55971d4279a0 2024-05-26T22:52:02.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.353+0000 7f96d9600640 10 client.6600 handle_command_reply: tid=0 2024-05-26T22:52:02.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.353+0000 7f96f6014740 1 client.6600 shutdown 2024-05-26T22:52:02.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.353+0000 7f96f6014740 1 -- 172.21.15.131:0/1065897133 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55971d3fcd90 msgr2=0x55971d41a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:02.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.353+0000 7f96f6014740 1 --2- 172.21.15.131:0/1065897133 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55971d3fcd90 0x55971d41a530 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f96dc0795e0 tx=0x7f96dc009190 comp rx=0 tx=0).stop 2024-05-26T22:52:02.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.353+0000 7f96f6014740 1 -- 172.21.15.131:0/1065897133 shutdown_connections 2024-05-26T22:52:02.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.353+0000 7f96f6014740 1 --2- 172.21.15.131:0/1065897133 >> [v2:172.21.15.131:6835/2737657374,v1:172.21.15.131:6836/2737657374] conn(0x55971d4279a0 0x55971d3fec60 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:02.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.353+0000 7f96f6014740 1 --2- 172.21.15.131:0/1065897133 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55971d426fe0 0x55971d427460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:02.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.353+0000 7f96f6014740 1 --2- 172.21.15.131:0/1065897133 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55971d419c70 0x55971d41aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:02.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.353+0000 7f96f6014740 1 --2- 172.21.15.131:0/1065897133 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55971d3fcd90 0x55971d41a530 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:02.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.353+0000 7f96d9600640 0 client.6600 ms_handle_reset on v2:172.21.15.131:6835/2737657374 2024-05-26T22:52:02.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.353+0000 7f96f6014740 1 -- 172.21.15.131:0/1065897133 >> 172.21.15.131:0/1065897133 conn(0x55971d3fc650 msgr2=0x55971d419260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:52:02.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.354+0000 7f96f6014740 1 -- 172.21.15.131:0/1065897133 shutdown_connections 2024-05-26T22:52:02.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.354+0000 7f96f6014740 1 -- 172.21.15.131:0/1065897133 wait complete. 2024-05-26T22:52:02.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.354+0000 7f96f6014740 20 client.6600 trim_cache size 0 max 16384 2024-05-26T22:52:02.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.374+0000 7f96f6014740 1 Processor -- start 2024-05-26T22:52:02.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.374+0000 7f96f6014740 1 -- start start 2024-05-26T22:52:02.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.375+0000 7f96f6014740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55971d3fcd90 0x55971d417330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:02.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.375+0000 7f96f6014740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55971d419c70 0x55971d3d20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:02.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.375+0000 7f96f6014740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55971d426fe0 0x55971d419260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:02.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.375+0000 7f96f6014740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f96ec1ffeb0 con 0x55971d419c70 2024-05-26T22:52:02.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.375+0000 7f96f6014740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f96ec200580 con 0x55971d426fe0 2024-05-26T22:52:02.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.375+0000 7f96f6014740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f96ec0cb2e0 con 0x55971d3fcd90 2024-05-26T22:52:02.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.376+0000 7f96eb400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55971d419c70 0x55971d3d20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:02.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.376+0000 7f96eb400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55971d419c70 0x55971d3d20f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50992/0 (socket says 172.21.15.131:50992) 2024-05-26T22:52:02.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.376+0000 7f96eb400640 1 -- 172.21.15.131:0/634728833 learned_addr learned my addr 172.21.15.131:0/634728833 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:52:02.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.376+0000 7f96eb400640 1 -- 172.21.15.131:0/634728833 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55971d3fcd90 msgr2=0x55971d417330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:02.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.376+0000 7f96eb400640 1 --2- 172.21.15.131:0/634728833 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55971d3fcd90 0x55971d417330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:02.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.376+0000 7f96eb400640 1 -- 172.21.15.131:0/634728833 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55971d426fe0 msgr2=0x55971d419260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:02.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.376+0000 7f96eb400640 1 --2- 172.21.15.131:0/634728833 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55971d426fe0 0x55971d419260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:02.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.376+0000 7f96eb400640 1 -- 172.21.15.131:0/634728833 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55971d41b200 con 0x55971d419c70 2024-05-26T22:52:02.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.376+0000 7f96eb400640 1 --2- 172.21.15.131:0/634728833 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55971d419c70 0x55971d3d20f0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f96d4000f60 tx=0x7f96d4012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26453bbc7d95648b server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:02.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.377+0000 7f96dbe00640 1 -- 172.21.15.131:0/634728833 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96d4005ce0 con 0x55971d419c70 2024-05-26T22:52:02.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.377+0000 7f96dbe00640 1 -- 172.21.15.131:0/634728833 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96d4029020 con 0x55971d419c70 2024-05-26T22:52:02.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.377+0000 7f96dbe00640 1 -- 172.21.15.131:0/634728833 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96d4007560 con 0x55971d419c70 2024-05-26T22:52:02.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.377+0000 7f96f6014740 1 -- 172.21.15.131:0/634728833 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55971d419c70 msgr2=0x55971d3d20f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:02.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.377+0000 7f96f6014740 1 --2- 172.21.15.131:0/634728833 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55971d419c70 0x55971d3d20f0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f96d4000f60 tx=0x7f96d4012000 comp rx=0 tx=0).stop 2024-05-26T22:52:02.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.377+0000 7f96f6014740 1 -- 172.21.15.131:0/634728833 shutdown_connections 2024-05-26T22:52:02.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.377+0000 7f96f6014740 1 --2- 172.21.15.131:0/634728833 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55971d426fe0 0x55971d419260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:02.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.377+0000 7f96f6014740 1 --2- 172.21.15.131:0/634728833 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55971d419c70 0x55971d3d20f0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:02.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.377+0000 7f96f6014740 1 --2- 172.21.15.131:0/634728833 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55971d3fcd90 0x55971d417330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:02.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.377+0000 7f96f6014740 1 -- 172.21.15.131:0/634728833 >> 172.21.15.131:0/634728833 conn(0x55971d3fc650 msgr2=0x55971d3f7620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:52:02.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.377+0000 7f96f6014740 1 -- 172.21.15.131:0/634728833 shutdown_connections 2024-05-26T22:52:02.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.378+0000 7f96f6014740 1 -- 172.21.15.131:0/634728833 wait complete. 2024-05-26T22:52:02.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.378+0000 7f96f6014740 1 -- start start 2024-05-26T22:52:02.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.378+0000 7f96f6014740 1 Processor -- start 2024-05-26T22:52:02.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.379+0000 7f96f6014740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55971d3fcd90 0x55971d494b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:02.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.379+0000 7f96f6014740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55971d419c70 0x55971d3f8700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:02.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.379+0000 7f96f6014740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55971d426fe0 0x55971d43aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:02.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.379+0000 7f96f6014740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55971d334610 con 0x55971d3fcd90 2024-05-26T22:52:02.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.379+0000 7f96f6014740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55971d3d0a20 con 0x55971d419c70 2024-05-26T22:52:02.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.379+0000 7f96f6014740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x55971d2ef910 con 0x55971d426fe0 2024-05-26T22:52:02.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.379+0000 7f96f6014740 10 client.0 _ll_register_callbacks cb 0x7f96dc090070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-26T22:52:02.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.379+0000 7f96ebe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55971d3fcd90 0x55971d494b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:02.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.380+0000 7f96ebe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55971d3fcd90 0x55971d494b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50996/0 (socket says 172.21.15.131:50996) 2024-05-26T22:52:02.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.380+0000 7f96ebe00640 1 -- 172.21.15.131:0/3312650836 learned_addr learned my addr 172.21.15.131:0/3312650836 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:52:02.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.380+0000 7f96ebe00640 1 -- 172.21.15.131:0/3312650836 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55971d426fe0 msgr2=0x55971d43aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:02.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.380+0000 7f96ebe00640 1 --2- 172.21.15.131:0/3312650836 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55971d426fe0 0x55971d43aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:02.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.380+0000 7f96ebe00640 1 -- 172.21.15.131:0/3312650836 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55971d419c70 msgr2=0x55971d3f8700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:02.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.380+0000 7f96ebe00640 1 --2- 172.21.15.131:0/3312650836 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55971d419c70 0x55971d3f8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:02.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.380+0000 7f96ebe00640 1 -- 172.21.15.131:0/3312650836 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55971d426240 con 0x55971d3fcd90 2024-05-26T22:52:02.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.380+0000 7f96ebe00640 1 --2- 172.21.15.131:0/3312650836 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55971d3fcd90 0x55971d494b10 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f96dc079350 tx=0x7f96dc080040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4c4cd61d5fef8e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:02.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.380+0000 7f96d9600640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-26T22:52:02.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.380+0000 7f96d9600640 1 -- 172.21.15.131:0/3312650836 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96dc082230 con 0x55971d3fcd90 2024-05-26T22:52:02.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.380+0000 7f96d9600640 1 -- 172.21.15.131:0/3312650836 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f96dc081fa0 con 0x55971d3fcd90 2024-05-26T22:52:02.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.380+0000 7f96f6014740 1 -- 172.21.15.131:0/3312650836 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96d4029020 con 0x55971d3fcd90 2024-05-26T22:52:02.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.381+0000 7f96d9600640 1 -- 172.21.15.131:0/3312650836 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f96dc087050 con 0x55971d3fcd90 2024-05-26T22:52:02.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.381+0000 7f96f6014740 10 client.6606 fetch_fsmap learned FSMap version 54 2024-05-26T22:52:02.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.381+0000 7f96f6014740 1 -- 172.21.15.131:0/3312650836 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) -- 0x55971d3d0ba0 con 0x55971d3fcd90 2024-05-26T22:52:02.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.381+0000 7f96d9600640 1 -- 172.21.15.131:0/3312650836 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96d4029020 con 0x55971d3fcd90 2024-05-26T22:52:02.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.381+0000 7f96d9600640 1 -- 172.21.15.131:0/3312650836 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 54) ==== 3534+0+0 (secure 0 0 0) 0x7f96dc08a020 con 0x55971d3fcd90 2024-05-26T22:52:02.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.381+0000 7f96f6014740 10 client.6606 fetch_fsmap finished waiting for FSMap version 54 2024-05-26T22:52:02.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.381+0000 7f96f6014740 10 client.6606 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-26T22:52:02.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.381+0000 7f96f6014740 20 client.6606 populate_metadata read hostname 'smithi131' 2024-05-26T22:52:02.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.381+0000 7f96f6014740 1 --2- 172.21.15.131:0/3312650836 >> [v2:172.21.15.131:6835/2737657374,v1:172.21.15.131:6836/2737657374] conn(0x55971d3ff120 0x55971d4398c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:02.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.381+0000 7f96f6014740 4 client.6606 mds_command: new command op to 6378 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-26T22:52:02.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.381+0000 7f96f6014740 1 -- 172.21.15.131:0/3312650836 --> [v2:172.21.15.131:6835/2737657374,v1:172.21.15.131:6836/2737657374] -- command(tid 0: {"prefix": "status"}) -- 0x55971d411060 con 0x55971d3ff120 2024-05-26T22:52:02.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.381+0000 7f96eb400640 1 --2- 172.21.15.131:0/3312650836 >> [v2:172.21.15.131:6835/2737657374,v1:172.21.15.131:6836/2737657374] conn(0x55971d3ff120 0x55971d4398c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:02.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.382+0000 7f96d9600640 10 client.6606 ms_handle_connect on v2:172.21.15.131:6835/2737657374 2024-05-26T22:52:02.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.382+0000 7f96eb400640 1 --2- 172.21.15.131:0/3312650836 >> [v2:172.21.15.131:6835/2737657374,v1:172.21.15.131:6836/2737657374] conn(0x55971d3ff120 0x55971d4398c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=16344d182e922b03 server_cookie=7a764cbf6f79f3b7 in_seq=0 out_seq=0 2024-05-26T22:52:02.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.382+0000 7f96d9600640 1 -- 172.21.15.131:0/3312650836 <== mds.2 v2:172.21.15.131:6835/2737657374 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55971d411060 con 0x55971d3ff120 2024-05-26T22:52:02.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.382+0000 7f96d9600640 10 client.6606 handle_command_reply: tid=0 2024-05-26T22:52:02.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.382+0000 7f96f6014740 1 client.6606 shutdown 2024-05-26T22:52:02.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.382+0000 7f96f6014740 1 -- 172.21.15.131:0/3312650836 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55971d3fcd90 msgr2=0x55971d494b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:02.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.382+0000 7f96f6014740 1 --2- 172.21.15.131:0/3312650836 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55971d3fcd90 0x55971d494b10 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f96dc079350 tx=0x7f96dc080040 comp rx=0 tx=0).stop 2024-05-26T22:52:02.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.382+0000 7f96f6014740 1 -- 172.21.15.131:0/3312650836 shutdown_connections 2024-05-26T22:52:02.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.382+0000 7f96f6014740 1 --2- 172.21.15.131:0/3312650836 >> [v2:172.21.15.131:6835/2737657374,v1:172.21.15.131:6836/2737657374] conn(0x55971d3ff120 0x55971d4398c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:02.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.382+0000 7f96f6014740 1 --2- 172.21.15.131:0/3312650836 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55971d426fe0 0x55971d43aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:02.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.382+0000 7f96f6014740 1 --2- 172.21.15.131:0/3312650836 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55971d419c70 0x55971d3f8700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:02.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.382+0000 7f96d9600640 0 client.6606 ms_handle_reset on v2:172.21.15.131:6835/2737657374 2024-05-26T22:52:02.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.382+0000 7f96f6014740 1 --2- 172.21.15.131:0/3312650836 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55971d3fcd90 0x55971d494b10 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:02.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.382+0000 7f96f6014740 1 -- 172.21.15.131:0/3312650836 >> 172.21.15.131:0/3312650836 conn(0x55971d3fc650 msgr2=0x55971d3cb920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:52:02.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.383+0000 7f96f6014740 1 -- 172.21.15.131:0/3312650836 shutdown_connections 2024-05-26T22:52:02.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.383+0000 7f96f6014740 1 -- 172.21.15.131:0/3312650836 wait complete. 2024-05-26T22:52:02.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.383+0000 7f96f6014740 20 client.6606 trim_cache size 0 max 16384 2024-05-26T22:52:02.386 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-26T22:52:02.386 INFO:teuthology.orchestra.run.smithi131.stdout: "cluster_fsid": "46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f", 2024-05-26T22:52:02.386 INFO:teuthology.orchestra.run.smithi131.stdout: "whoami": 2, 2024-05-26T22:52:02.386 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 6378, 2024-05-26T22:52:02.387 INFO:teuthology.orchestra.run.smithi131.stdout: "want_state": "up:active", 2024-05-26T22:52:02.387 INFO:teuthology.orchestra.run.smithi131.stdout: "state": "up:active", 2024-05-26T22:52:02.387 INFO:teuthology.orchestra.run.smithi131.stdout: "fs_name": "cephfs", 2024-05-26T22:52:02.387 INFO:teuthology.orchestra.run.smithi131.stdout: "rank_uptime": 2.0142785189999999, 2024-05-26T22:52:02.387 INFO:teuthology.orchestra.run.smithi131.stdout: "mdsmap_epoch": 54, 2024-05-26T22:52:02.387 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch": 58, 2024-05-26T22:52:02.387 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch_barrier": 58, 2024-05-26T22:52:02.387 INFO:teuthology.orchestra.run.smithi131.stdout: "uptime": 28.228637725999999 2024-05-26T22:52:02.387 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-26T22:52:02.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.383+0000 7f96f3200640 1 -- 172.21.15.131:0/3482100822 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f96c806be30 msgr2=0x7f96c806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:02.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.383+0000 7f96f3200640 1 --2- 172.21.15.131:0/3482100822 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f96c806be30 0x7f96c806e2f0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f96ec1502e0 tx=0x7f96d4009040 comp rx=0 tx=0).stop 2024-05-26T22:52:02.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.383+0000 7f96f3200640 1 -- 172.21.15.131:0/3482100822 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96ec06f420 msgr2=0x7f96ec14f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:02.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.383+0000 7f96f3200640 1 --2- 172.21.15.131:0/3482100822 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96ec06f420 0x7f96ec14f960 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f96dc035040 tx=0x7f96dc0046c0 comp rx=0 tx=0).stop 2024-05-26T22:52:02.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.383+0000 7f96f3200640 1 -- 172.21.15.131:0/3482100822 shutdown_connections 2024-05-26T22:52:02.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.383+0000 7f96f3200640 1 --2- 172.21.15.131:0/3482100822 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f96ec0d0460 0x7f96ec09ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:02.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.383+0000 7f96f3200640 1 --2- 172.21.15.131:0/3482100822 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f96c806be30 0x7f96c806e2f0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:02.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.383+0000 7f96f3200640 1 --2- 172.21.15.131:0/3482100822 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f96ec0b5450 0x7f96ec0ad160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:02.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.383+0000 7f96f3200640 1 --2- 172.21.15.131:0/3482100822 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96ec06f420 0x7f96ec14f960 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:02.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.383+0000 7f96f3200640 1 -- 172.21.15.131:0/3482100822 >> 172.21.15.131:0/3482100822 conn(0x7f96ec06bc90 msgr2=0x7f96ec15a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:52:02.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.384+0000 7f96f3200640 1 -- 172.21.15.131:0/3482100822 shutdown_connections 2024-05-26T22:52:02.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.384+0000 7f96f3200640 1 -- 172.21.15.131:0/3482100822 wait complete. 2024-05-26T22:52:02.401 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-05-26T22:52:02.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.585+0000 7fe2fc600640 1 Processor -- start 2024-05-26T22:52:02.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.585+0000 7fe2fc600640 1 -- start start 2024-05-26T22:52:02.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.587+0000 7fe2fc600640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fe2f40b5450 0x7fe2f40b43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:02.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.587+0000 7fe2fc600640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fe2f406f420 0x7fe2f40cfd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:02.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.587+0000 7fe2fc600640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2f40b4900 con 0x7fe2f40cf590 2024-05-26T22:52:02.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.587+0000 7fe2fc600640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fe2f4160d90 con 0x7fe2f406f420 2024-05-26T22:52:02.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.587+0000 7fe2fc600640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fe2f4160180 con 0x7fe2f40b5450 2024-05-26T22:52:02.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.587+0000 7fe2f9200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fe2f406f420 0x7fe2f40cfd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:02.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.587+0000 7fe2f9200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fe2f406f420 0x7fe2f40cfd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51004/0 (socket says 172.21.15.131:51004) 2024-05-26T22:52:02.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.587+0000 7fe2f9200640 1 -- 172.21.15.131:0/1685993212 learned_addr learned my addr 172.21.15.131:0/1685993212 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:52:02.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.587+0000 7fe2f9c00640 1 --2- 172.21.15.131:0/1685993212 >> v2:172.21.15.187:3300/0 conn(0x7fe2f40b5450 0x7fe2f40b43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:02.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.588+0000 7fe2f9200640 1 -- 172.21.15.131:0/1685993212 >> v2:172.21.15.187:3300/0 conn(0x7fe2f40b5450 msgr2=0x7fe2f40b43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:02.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.588+0000 7fe2f9200640 1 --2- 172.21.15.131:0/1685993212 >> v2:172.21.15.187:3300/0 conn(0x7fe2f40b5450 0x7fe2f40b43c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:02.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.588+0000 7fe2f9200640 1 -- 172.21.15.131:0/1685993212 >> v1:172.21.15.187:6789/0 conn(0x7fe2f40cf590 legacy=0x7fe2f406d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:02.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.588+0000 7fe2f9200640 1 -- 172.21.15.131:0/1685993212 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2f4160460 con 0x7fe2f406f420 2024-05-26T22:52:02.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.589+0000 7fe2f9200640 1 --2- 172.21.15.131:0/1685993212 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2f406f420 0x7fe2f40cfd50 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fe2f40d0c40 tx=0x7fe2e002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9276b1ed935e58ee server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:02.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.589+0000 7fe2f3e00640 1 -- 172.21.15.131:0/1685993212 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2e0037070 con 0x7fe2f406f420 2024-05-26T22:52:02.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.589+0000 7fe2f3e00640 1 -- 172.21.15.131:0/1685993212 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2e0032440 con 0x7fe2f406f420 2024-05-26T22:52:02.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.589+0000 7fe2f3e00640 1 -- 172.21.15.131:0/1685993212 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2e0032810 con 0x7fe2f406f420 2024-05-26T22:52:02.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.590+0000 7fe2fc600640 1 -- 172.21.15.131:0/1685993212 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2f406f420 msgr2=0x7fe2f40cfd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:02.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.590+0000 7fe2fc600640 1 --2- 172.21.15.131:0/1685993212 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2f406f420 0x7fe2f40cfd50 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fe2f40d0c40 tx=0x7fe2e002a230 comp rx=0 tx=0).stop 2024-05-26T22:52:02.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.590+0000 7fe2fc600640 1 -- 172.21.15.131:0/1685993212 shutdown_connections 2024-05-26T22:52:02.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.590+0000 7fe2fc600640 1 --2- 172.21.15.131:0/1685993212 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2f406f420 0x7fe2f40cfd50 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:02.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.590+0000 7fe2fc600640 1 --2- 172.21.15.131:0/1685993212 >> v2:172.21.15.187:3300/0 conn(0x7fe2f40b5450 0x7fe2f40b43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:02.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.590+0000 7fe2fc600640 1 -- 172.21.15.131:0/1685993212 >> 172.21.15.131:0/1685993212 conn(0x7fe2f406bc90 msgr2=0x7fe2f40b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:52:02.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.590+0000 7fe2fc600640 1 -- 172.21.15.131:0/1685993212 shutdown_connections 2024-05-26T22:52:02.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.590+0000 7fe2fc600640 1 -- 172.21.15.131:0/1685993212 wait complete. 2024-05-26T22:52:02.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.592+0000 7fe2fc600640 1 Processor -- start 2024-05-26T22:52:02.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.592+0000 7fe2fc600640 1 -- start start 2024-05-26T22:52:02.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.593+0000 7fe2fc600640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe2f406f420 0x7fe2f41595f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:02.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.594+0000 7fe2fc600640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2f40b5450 0x7fe2f4157150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:02.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.594+0000 7fe2fc600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2f40cf590 0x7fe2f4151100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:02.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.594+0000 7fe2fc600640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe2f40b5830 con 0x7fe2f40cf590 2024-05-26T22:52:02.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.594+0000 7fe2fc600640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe2f4160a20 con 0x7fe2f406f420 2024-05-26T22:52:02.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.594+0000 7fe2fc600640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe2f415fde0 con 0x7fe2f40b5450 2024-05-26T22:52:02.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.594+0000 7fe2fa600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2f40cf590 0x7fe2f4151100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:02.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.594+0000 7fe2fa600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2f40cf590 0x7fe2f4151100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51010/0 (socket says 172.21.15.131:51010) 2024-05-26T22:52:02.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.594+0000 7fe2fa600640 1 -- 172.21.15.131:0/1847352241 learned_addr learned my addr 172.21.15.131:0/1847352241 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:52:02.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.594+0000 7fe2f9c00640 1 --2- 172.21.15.131:0/1847352241 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe2f406f420 0x7fe2f41595f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:02.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.594+0000 7fe2f9200640 1 --2- 172.21.15.131:0/1847352241 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2f40b5450 0x7fe2f4157150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:02.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.594+0000 7fe2fa600640 1 -- 172.21.15.131:0/1847352241 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2f40b5450 msgr2=0x7fe2f4157150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:02.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.594+0000 7fe2fa600640 1 --2- 172.21.15.131:0/1847352241 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2f40b5450 0x7fe2f4157150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:02.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.594+0000 7fe2fa600640 1 -- 172.21.15.131:0/1847352241 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe2f406f420 msgr2=0x7fe2f41595f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:02.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.594+0000 7fe2fa600640 1 --2- 172.21.15.131:0/1847352241 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe2f406f420 0x7fe2f41595f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:02.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.594+0000 7fe2fa600640 1 -- 172.21.15.131:0/1847352241 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2f41f48c0 con 0x7fe2f40cf590 2024-05-26T22:52:02.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.594+0000 7fe2f9200640 1 --2- 172.21.15.131:0/1847352241 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2f40b5450 0x7fe2f4157150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:52:02.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.595+0000 7fe2fa600640 1 --2- 172.21.15.131:0/1847352241 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2f40cf590 0x7fe2f4151100 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec00c780 tx=0x7fe2ec00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:02.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.595+0000 7fe2f1600640 1 -- 172.21.15.131:0/1847352241 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2ec004b60 con 0x7fe2f40cf590 2024-05-26T22:52:02.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.595+0000 7fe2f1600640 1 -- 172.21.15.131:0/1847352241 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2ec004d00 con 0x7fe2f40cf590 2024-05-26T22:52:02.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.595+0000 7fe2f1600640 1 -- 172.21.15.131:0/1847352241 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2ec00aa90 con 0x7fe2f40cf590 2024-05-26T22:52:02.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.595+0000 7fe2fc600640 1 -- 172.21.15.131:0/1847352241 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2f41f4ba0 con 0x7fe2f40cf590 2024-05-26T22:52:02.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.595+0000 7fe2fc600640 1 -- 172.21.15.131:0/1847352241 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2f41f5160 con 0x7fe2f40cf590 2024-05-26T22:52:02.597 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.597+0000 7fe2f1600640 1 -- 172.21.15.131:0/1847352241 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe2ec00ac30 con 0x7fe2f40cf590 2024-05-26T22:52:02.597 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.598+0000 7fe2f1600640 1 --2- 172.21.15.131:0/1847352241 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fe2d406be30 0x7fe2d406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:02.598 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.598+0000 7fe2f1600640 1 -- 172.21.15.131:0/1847352241 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fe2ec08b200 con 0x7fe2f40cf590 2024-05-26T22:52:02.598 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.598+0000 7fe2f9c00640 1 --2- 172.21.15.131:0/1847352241 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fe2d406be30 0x7fe2d406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:02.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.599+0000 7fe2f9c00640 1 --2- 172.21.15.131:0/1847352241 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fe2d406be30 0x7fe2d406e2f0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fe2e8006fd0 tx=0x7fe2e8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:02.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.599+0000 7fe2ff3ec740 1 Processor -- start 2024-05-26T22:52:02.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.599+0000 7fe2ff3ec740 1 -- start start 2024-05-26T22:52:02.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.600+0000 7fe2ff3ec740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dc98dedc70 0x55dc98da0890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:02.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.600+0000 7fe2ff3ec740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55dc98dd0d90 0x55dc98dd0110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:02.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.600+0000 7fe2ff3ec740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dc98dd0650 0x55dc98da5460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:02.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.600+0000 7fe2ff3ec740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55dc98d08610 con 0x55dc98dd0650 2024-05-26T22:52:02.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.600+0000 7fe2ff3ec740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55dc98cb05c0 con 0x55dc98dedc70 2024-05-26T22:52:02.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.600+0000 7fe2ff3ec740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x55dc98d045d0 con 0x55dc98dd0d90 2024-05-26T22:52:02.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.601+0000 7fe2fa600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dc98dd0650 0x55dc98da5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:02.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.601+0000 7fe2fa600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dc98dd0650 0x55dc98da5460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51016/0 (socket says 172.21.15.131:51016) 2024-05-26T22:52:02.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.601+0000 7fe2fa600640 1 -- 172.21.15.131:0/1195097264 learned_addr learned my addr 172.21.15.131:0/1195097264 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:52:02.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.601+0000 7fe2f9c00640 1 --2- 172.21.15.131:0/1195097264 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dc98dedc70 0x55dc98da0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:02.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.601+0000 7fe2f9200640 1 --2- 172.21.15.131:0/1195097264 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55dc98dd0d90 0x55dc98dd0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:02.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.601+0000 7fe2fa600640 1 -- 172.21.15.131:0/1195097264 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55dc98dd0d90 msgr2=0x55dc98dd0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:02.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.601+0000 7fe2fa600640 1 --2- 172.21.15.131:0/1195097264 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55dc98dd0d90 0x55dc98dd0110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:02.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.601+0000 7fe2fa600640 1 -- 172.21.15.131:0/1195097264 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dc98dedc70 msgr2=0x55dc98da0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:02.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.601+0000 7fe2fa600640 1 --2- 172.21.15.131:0/1195097264 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dc98dedc70 0x55dc98da0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:02.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.601+0000 7fe2fa600640 1 -- 172.21.15.131:0/1195097264 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc98a1c2d0 con 0x55dc98dd0650 2024-05-26T22:52:02.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.601+0000 7fe2fa600640 1 --2- 172.21.15.131:0/1195097264 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dc98dd0650 0x55dc98da5460 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec059020 tx=0x7fe2ec058e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=850b904f603277d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:02.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.602+0000 7fe2e6000640 1 -- 172.21.15.131:0/1195097264 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2ec071030 con 0x55dc98dd0650 2024-05-26T22:52:02.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.602+0000 7fe2e6000640 1 -- 172.21.15.131:0/1195097264 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2ec05d7c0 con 0x55dc98dd0650 2024-05-26T22:52:02.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.602+0000 7fe2e6000640 1 -- 172.21.15.131:0/1195097264 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2ec061ff0 con 0x55dc98dd0650 2024-05-26T22:52:02.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.602+0000 7fe2ff3ec740 1 -- 172.21.15.131:0/1195097264 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dc98dd0650 msgr2=0x55dc98da5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:02.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.602+0000 7fe2ff3ec740 1 --2- 172.21.15.131:0/1195097264 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dc98dd0650 0x55dc98da5460 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec059020 tx=0x7fe2ec058e30 comp rx=0 tx=0).stop 2024-05-26T22:52:02.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.602+0000 7fe2ff3ec740 1 -- 172.21.15.131:0/1195097264 shutdown_connections 2024-05-26T22:52:02.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.602+0000 7fe2ff3ec740 1 --2- 172.21.15.131:0/1195097264 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55dc98dd0d90 0x55dc98dd0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:02.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.602+0000 7fe2ff3ec740 1 --2- 172.21.15.131:0/1195097264 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dc98dd0650 0x55dc98da5460 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:02.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.602+0000 7fe2ff3ec740 1 --2- 172.21.15.131:0/1195097264 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dc98dedc70 0x55dc98da0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:02.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.602+0000 7fe2ff3ec740 1 -- 172.21.15.131:0/1195097264 >> 172.21.15.131:0/1195097264 conn(0x55dc98da2b00 msgr2=0x55dc98deb110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:52:02.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.602+0000 7fe2ff3ec740 1 -- 172.21.15.131:0/1195097264 shutdown_connections 2024-05-26T22:52:02.604 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.602+0000 7fe2ff3ec740 1 -- 172.21.15.131:0/1195097264 wait complete. 2024-05-26T22:52:02.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.603+0000 7fe2ff3ec740 1 -- start start 2024-05-26T22:52:02.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.603+0000 7fe2ff3ec740 1 Processor -- start 2024-05-26T22:52:02.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.604+0000 7fe2ff3ec740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55dc98dd0d90 0x55dc98dd2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:02.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.604+0000 7fe2ff3ec740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dc98dedc70 0x55dc98da4e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:02.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.604+0000 7fe2ff3ec740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dc98dc11f0 0x55dc98da5360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:02.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.604+0000 7fe2ff3ec740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55dc98d08610 con 0x55dc98dedc70 2024-05-26T22:52:02.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.604+0000 7fe2ff3ec740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55dc98cc3910 con 0x55dc98dc11f0 2024-05-26T22:52:02.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.604+0000 7fe2ff3ec740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x55dc98da4a20 con 0x55dc98dd0d90 2024-05-26T22:52:02.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.604+0000 7fe2ff3ec740 10 client.0 _ll_register_callbacks cb 0x55dc98cff410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-26T22:52:02.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.605+0000 7fe2f9200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dc98dedc70 0x55dc98da4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:02.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.605+0000 7fe2f9200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dc98dedc70 0x55dc98da4e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51026/0 (socket says 172.21.15.131:51026) 2024-05-26T22:52:02.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.605+0000 7fe2f9200640 1 -- 172.21.15.131:0/4048098709 learned_addr learned my addr 172.21.15.131:0/4048098709 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:52:02.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.605+0000 7fe2fa600640 1 --2- 172.21.15.131:0/4048098709 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dc98dc11f0 0x55dc98da5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:02.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.605+0000 7fe2f9c00640 1 --2- 172.21.15.131:0/4048098709 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55dc98dd0d90 0x55dc98dd2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:02.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.605+0000 7fe2f9200640 1 -- 172.21.15.131:0/4048098709 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55dc98dd0d90 msgr2=0x55dc98dd2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:02.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.605+0000 7fe2f9200640 1 --2- 172.21.15.131:0/4048098709 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55dc98dd0d90 0x55dc98dd2bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:02.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.605+0000 7fe2f9200640 1 -- 172.21.15.131:0/4048098709 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dc98dc11f0 msgr2=0x55dc98da5360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:02.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.605+0000 7fe2f9200640 1 --2- 172.21.15.131:0/4048098709 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dc98dc11f0 0x55dc98da5360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:02.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.605+0000 7fe2f9200640 1 -- 172.21.15.131:0/4048098709 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc98ae66b0 con 0x55dc98dedc70 2024-05-26T22:52:02.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.605+0000 7fe2f9200640 1 --2- 172.21.15.131:0/4048098709 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dc98dedc70 0x55dc98da4e20 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fe2e002a270 tx=0x7fe2e0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7fa0d1702fd4724 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:02.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.605+0000 7fe2daa00640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-26T22:52:02.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.606+0000 7fe2daa00640 1 -- 172.21.15.131:0/4048098709 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2e0037040 con 0x55dc98dedc70 2024-05-26T22:52:02.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.606+0000 7fe2daa00640 1 -- 172.21.15.131:0/4048098709 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fe2e003e660 con 0x55dc98dedc70 2024-05-26T22:52:02.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.606+0000 7fe2ff3ec740 1 -- 172.21.15.131:0/4048098709 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2ec05d7c0 con 0x55dc98dedc70 2024-05-26T22:52:02.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.606+0000 7fe2daa00640 1 -- 172.21.15.131:0/4048098709 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fe2e003e860 con 0x55dc98dedc70 2024-05-26T22:52:02.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.606+0000 7fe2ff3ec740 10 client.6618 fetch_fsmap learned FSMap version 54 2024-05-26T22:52:02.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.606+0000 7fe2ff3ec740 1 -- 172.21.15.131:0/4048098709 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) -- 0x55dc98dd1e00 con 0x55dc98dedc70 2024-05-26T22:52:02.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.606+0000 7fe2daa00640 1 -- 172.21.15.131:0/4048098709 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2ec05d7c0 con 0x55dc98dedc70 2024-05-26T22:52:02.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.606+0000 7fe2daa00640 1 -- 172.21.15.131:0/4048098709 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 54) ==== 3534+0+0 (secure 0 0 0) 0x7fe2e0032ce0 con 0x55dc98dedc70 2024-05-26T22:52:02.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.606+0000 7fe2ff3ec740 10 client.6618 fetch_fsmap finished waiting for FSMap version 54 2024-05-26T22:52:02.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.606+0000 7fe2ff3ec740 10 client.6618 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-26T22:52:02.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.606+0000 7fe2ff3ec740 20 client.6618 populate_metadata read hostname 'smithi131' 2024-05-26T22:52:02.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.606+0000 7fe2ff3ec740 1 --2- 172.21.15.131:0/4048098709 >> [v2:172.21.15.131:6834/640544945,v1:172.21.15.131:6837/640544945] conn(0x55dc98dbe9d0 0x55dc98de94c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:02.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.606+0000 7fe2ff3ec740 4 client.6618 mds_command: new command op to 6381 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-26T22:52:02.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.606+0000 7fe2ff3ec740 1 -- 172.21.15.131:0/4048098709 --> [v2:172.21.15.131:6834/640544945,v1:172.21.15.131:6837/640544945] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dc98d15d40 con 0x55dc98dbe9d0 2024-05-26T22:52:02.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.607+0000 7fe2f9c00640 1 --2- 172.21.15.131:0/4048098709 >> [v2:172.21.15.131:6834/640544945,v1:172.21.15.131:6837/640544945] conn(0x55dc98dbe9d0 0x55dc98de94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:02.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.607+0000 7fe2f9c00640 1 --2- 172.21.15.131:0/4048098709 >> [v2:172.21.15.131:6834/640544945,v1:172.21.15.131:6837/640544945] conn(0x55dc98dbe9d0 0x55dc98de94c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a9f32f45117eb476 server_cookie=ae58ddfbdf4adedf in_seq=0 out_seq=0 2024-05-26T22:52:02.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.607+0000 7fe2daa00640 10 client.6618 ms_handle_connect on v2:172.21.15.131:6834/640544945 2024-05-26T22:52:02.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.608+0000 7fe2daa00640 1 -- 172.21.15.131:0/4048098709 <== mds.1 v2:172.21.15.131:6834/640544945 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55dc98d15d40 con 0x55dc98dbe9d0 2024-05-26T22:52:02.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.608+0000 7fe2daa00640 10 client.6618 handle_command_reply: tid=0 2024-05-26T22:52:02.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.608+0000 7fe2ff3ec740 1 client.6618 shutdown 2024-05-26T22:52:02.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.608+0000 7fe2ff3ec740 1 -- 172.21.15.131:0/4048098709 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dc98dedc70 msgr2=0x55dc98da4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:02.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.608+0000 7fe2ff3ec740 1 --2- 172.21.15.131:0/4048098709 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dc98dedc70 0x55dc98da4e20 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fe2e002a270 tx=0x7fe2e0004730 comp rx=0 tx=0).stop 2024-05-26T22:52:02.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.608+0000 7fe2ff3ec740 1 -- 172.21.15.131:0/4048098709 shutdown_connections 2024-05-26T22:52:02.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.608+0000 7fe2ff3ec740 1 --2- 172.21.15.131:0/4048098709 >> [v2:172.21.15.131:6834/640544945,v1:172.21.15.131:6837/640544945] conn(0x55dc98dbe9d0 0x55dc98de94c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:02.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.608+0000 7fe2ff3ec740 1 --2- 172.21.15.131:0/4048098709 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dc98dc11f0 0x55dc98da5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:02.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.608+0000 7fe2ff3ec740 1 --2- 172.21.15.131:0/4048098709 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dc98dedc70 0x55dc98da4e20 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:02.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.608+0000 7fe2ff3ec740 1 --2- 172.21.15.131:0/4048098709 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55dc98dd0d90 0x55dc98dd2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:02.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.608+0000 7fe2daa00640 0 client.6618 ms_handle_reset on v2:172.21.15.131:6834/640544945 2024-05-26T22:52:02.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.608+0000 7fe2ff3ec740 1 -- 172.21.15.131:0/4048098709 >> 172.21.15.131:0/4048098709 conn(0x55dc98dd0650 msgr2=0x55dc98da12f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:52:02.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.609+0000 7fe2ff3ec740 1 -- 172.21.15.131:0/4048098709 shutdown_connections 2024-05-26T22:52:02.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.609+0000 7fe2ff3ec740 1 -- 172.21.15.131:0/4048098709 wait complete. 2024-05-26T22:52:02.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.609+0000 7fe2ff3ec740 20 client.6618 trim_cache size 0 max 16384 2024-05-26T22:52:02.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.628+0000 7fe2ff3ec740 1 Processor -- start 2024-05-26T22:52:02.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.628+0000 7fe2ff3ec740 1 -- start start 2024-05-26T22:52:02.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.629+0000 7fe2ff3ec740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dc98dc11f0 0x55dc98de2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:02.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.629+0000 7fe2ff3ec740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55dc98dd0d90 0x55dc98de1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:02.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.629+0000 7fe2ff3ec740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dc98dedc70 0x55dc98e0f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:02.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.629+0000 7fe2ff3ec740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe2f40720c0 con 0x55dc98dc11f0 2024-05-26T22:52:02.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.629+0000 7fe2ff3ec740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe2f4071360 con 0x55dc98dedc70 2024-05-26T22:52:02.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.629+0000 7fe2ff3ec740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe2f40719f0 con 0x55dc98dd0d90 2024-05-26T22:52:02.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.630+0000 7fe2f9c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dc98dc11f0 0x55dc98de2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:02.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.630+0000 7fe2f9c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dc98dc11f0 0x55dc98de2090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51030/0 (socket says 172.21.15.131:51030) 2024-05-26T22:52:02.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.630+0000 7fe2f9c00640 1 -- 172.21.15.131:0/2628475294 learned_addr learned my addr 172.21.15.131:0/2628475294 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:52:02.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.630+0000 7fe2f9c00640 1 -- 172.21.15.131:0/2628475294 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55dc98dd0d90 msgr2=0x55dc98de1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:02.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.630+0000 7fe2f9c00640 1 --2- 172.21.15.131:0/2628475294 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55dc98dd0d90 0x55dc98de1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:02.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.630+0000 7fe2f9c00640 1 -- 172.21.15.131:0/2628475294 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dc98dedc70 msgr2=0x55dc98e0f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:02.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.630+0000 7fe2f9c00640 1 --2- 172.21.15.131:0/2628475294 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dc98dedc70 0x55dc98e0f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:02.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.630+0000 7fe2f9c00640 1 -- 172.21.15.131:0/2628475294 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc98dd3fc0 con 0x55dc98dc11f0 2024-05-26T22:52:02.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.630+0000 7fe2f9c00640 1 --2- 172.21.15.131:0/2628475294 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dc98dc11f0 0x55dc98de2090 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fe2e800b270 tx=0x7fe2e8015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d92d673b82f6b8f5 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:02.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.630+0000 7fe2e5600640 1 -- 172.21.15.131:0/2628475294 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2e801b070 con 0x55dc98dc11f0 2024-05-26T22:52:02.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.631+0000 7fe2e5600640 1 -- 172.21.15.131:0/2628475294 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2e801c040 con 0x55dc98dc11f0 2024-05-26T22:52:02.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.631+0000 7fe2e5600640 1 -- 172.21.15.131:0/2628475294 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2e8014e60 con 0x55dc98dc11f0 2024-05-26T22:52:02.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.631+0000 7fe2ff3ec740 1 -- 172.21.15.131:0/2628475294 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dc98dc11f0 msgr2=0x55dc98de2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:02.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.631+0000 7fe2ff3ec740 1 --2- 172.21.15.131:0/2628475294 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dc98dc11f0 0x55dc98de2090 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fe2e800b270 tx=0x7fe2e8015000 comp rx=0 tx=0).stop 2024-05-26T22:52:02.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.631+0000 7fe2ff3ec740 1 -- 172.21.15.131:0/2628475294 shutdown_connections 2024-05-26T22:52:02.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.631+0000 7fe2ff3ec740 1 --2- 172.21.15.131:0/2628475294 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55dc98dd0d90 0x55dc98de1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:02.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.631+0000 7fe2ff3ec740 1 --2- 172.21.15.131:0/2628475294 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dc98dedc70 0x55dc98e0f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:02.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.631+0000 7fe2ff3ec740 1 --2- 172.21.15.131:0/2628475294 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dc98dc11f0 0x55dc98de2090 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:02.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.631+0000 7fe2ff3ec740 1 -- 172.21.15.131:0/2628475294 >> 172.21.15.131:0/2628475294 conn(0x55dc98dd0650 msgr2=0x55dc98dcd670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:52:02.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.631+0000 7fe2ff3ec740 1 -- 172.21.15.131:0/2628475294 shutdown_connections 2024-05-26T22:52:02.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.631+0000 7fe2ff3ec740 1 -- 172.21.15.131:0/2628475294 wait complete. 2024-05-26T22:52:02.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.632+0000 7fe2ff3ec740 1 -- start start 2024-05-26T22:52:02.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.632+0000 7fe2ff3ec740 1 Processor -- start 2024-05-26T22:52:02.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.633+0000 7fe2ff3ec740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55dc98dc11f0 0x55dc98e693b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:02.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.633+0000 7fe2ff3ec740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dc98dd0d90 0x55dc98de0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:02.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.633+0000 7fe2ff3ec740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dc98dedc70 0x55dc98de0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:02.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.633+0000 7fe2ff3ec740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55dc98dbf400 con 0x55dc98dedc70 2024-05-26T22:52:02.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.633+0000 7fe2ff3ec740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe2f41f7ca0 con 0x55dc98dd0d90 2024-05-26T22:52:02.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.633+0000 7fe2ff3ec740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x55dc98dbee40 con 0x55dc98dc11f0 2024-05-26T22:52:02.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.633+0000 7fe2ff3ec740 10 client.0 _ll_register_callbacks cb 0x7fe2e0035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-26T22:52:02.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.633+0000 7fe2fa600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dc98dedc70 0x55dc98de0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:02.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.633+0000 7fe2fa600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dc98dedc70 0x55dc98de0fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51040/0 (socket says 172.21.15.131:51040) 2024-05-26T22:52:02.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.633+0000 7fe2fa600640 1 -- 172.21.15.131:0/1870292375 learned_addr learned my addr 172.21.15.131:0/1870292375 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:52:02.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.634+0000 7fe2fa600640 1 -- 172.21.15.131:0/1870292375 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55dc98dc11f0 msgr2=0x55dc98e693b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:02.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.634+0000 7fe2fa600640 1 --2- 172.21.15.131:0/1870292375 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55dc98dc11f0 0x55dc98e693b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:02.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.634+0000 7fe2fa600640 1 -- 172.21.15.131:0/1870292375 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dc98dd0d90 msgr2=0x55dc98de0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:02.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.634+0000 7fe2fa600640 1 --2- 172.21.15.131:0/1870292375 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dc98dd0d90 0x55dc98de0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:02.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.634+0000 7fe2fa600640 1 -- 172.21.15.131:0/1870292375 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc98de9720 con 0x55dc98dedc70 2024-05-26T22:52:02.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.634+0000 7fe2fa600640 1 --2- 172.21.15.131:0/1870292375 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dc98dedc70 0x55dc98de0fe0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec055820 tx=0x7fe2ec007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4aa6e5f2fc230bd server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:02.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.634+0000 7fe2daa00640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-26T22:52:02.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.634+0000 7fe2daa00640 1 -- 172.21.15.131:0/1870292375 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2ec077060 con 0x55dc98dedc70 2024-05-26T22:52:02.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.634+0000 7fe2daa00640 1 -- 172.21.15.131:0/1870292375 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fe2ec0892d0 con 0x55dc98dedc70 2024-05-26T22:52:02.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.634+0000 7fe2ff3ec740 1 -- 172.21.15.131:0/1870292375 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2e801c040 con 0x55dc98dedc70 2024-05-26T22:52:02.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.634+0000 7fe2daa00640 1 -- 172.21.15.131:0/1870292375 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fe2ec079e60 con 0x55dc98dedc70 2024-05-26T22:52:02.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.634+0000 7fe2ff3ec740 10 client.6624 fetch_fsmap learned FSMap version 54 2024-05-26T22:52:02.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.634+0000 7fe2ff3ec740 1 -- 172.21.15.131:0/1870292375 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) -- 0x55dc98dfd880 con 0x55dc98dedc70 2024-05-26T22:52:02.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.634+0000 7fe2daa00640 1 -- 172.21.15.131:0/1870292375 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2e801c040 con 0x55dc98dedc70 2024-05-26T22:52:02.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.635+0000 7fe2daa00640 1 -- 172.21.15.131:0/1870292375 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 54) ==== 3534+0+0 (secure 0 0 0) 0x7fe2ec055a80 con 0x55dc98dedc70 2024-05-26T22:52:02.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.635+0000 7fe2ff3ec740 10 client.6624 fetch_fsmap finished waiting for FSMap version 54 2024-05-26T22:52:02.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.635+0000 7fe2ff3ec740 10 client.6624 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-26T22:52:02.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.635+0000 7fe2ff3ec740 20 client.6624 populate_metadata read hostname 'smithi131' 2024-05-26T22:52:02.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.635+0000 7fe2ff3ec740 1 --2- 172.21.15.131:0/1870292375 >> [v2:172.21.15.131:6834/640544945,v1:172.21.15.131:6837/640544945] conn(0x55dc98dfa420 0x55dc98dee530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:02.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.635+0000 7fe2ff3ec740 4 client.6624 mds_command: new command op to 6381 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-26T22:52:02.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.635+0000 7fe2ff3ec740 1 -- 172.21.15.131:0/1870292375 --> [v2:172.21.15.131:6834/640544945,v1:172.21.15.131:6837/640544945] -- command(tid 0: {"prefix": "status"}) -- 0x55dc98d10090 con 0x55dc98dfa420 2024-05-26T22:52:02.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.635+0000 7fe2f9c00640 1 --2- 172.21.15.131:0/1870292375 >> [v2:172.21.15.131:6834/640544945,v1:172.21.15.131:6837/640544945] conn(0x55dc98dfa420 0x55dc98dee530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:02.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.636+0000 7fe2f9c00640 1 --2- 172.21.15.131:0/1870292375 >> [v2:172.21.15.131:6834/640544945,v1:172.21.15.131:6837/640544945] conn(0x55dc98dfa420 0x55dc98dee530 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fc110d4516bc3b49 server_cookie=f43902f5c50289c2 in_seq=0 out_seq=0 2024-05-26T22:52:02.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.636+0000 7fe2daa00640 10 client.6624 ms_handle_connect on v2:172.21.15.131:6834/640544945 2024-05-26T22:52:02.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.636+0000 7fe2daa00640 1 -- 172.21.15.131:0/1870292375 <== mds.1 v2:172.21.15.131:6834/640544945 1 ==== command_reply(tid 0: 0 ) ==== 8+0+318 (crc 0 0 0) 0x55dc98d10090 con 0x55dc98dfa420 2024-05-26T22:52:02.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.636+0000 7fe2daa00640 10 client.6624 handle_command_reply: tid=0 2024-05-26T22:52:02.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.636+0000 7fe2ff3ec740 1 client.6624 shutdown 2024-05-26T22:52:02.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.636+0000 7fe2ff3ec740 1 -- 172.21.15.131:0/1870292375 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dc98dedc70 msgr2=0x55dc98de0fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:02.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.636+0000 7fe2ff3ec740 1 --2- 172.21.15.131:0/1870292375 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dc98dedc70 0x55dc98de0fe0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec055820 tx=0x7fe2ec007f50 comp rx=0 tx=0).stop 2024-05-26T22:52:02.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.636+0000 7fe2ff3ec740 1 -- 172.21.15.131:0/1870292375 shutdown_connections 2024-05-26T22:52:02.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.636+0000 7fe2ff3ec740 1 --2- 172.21.15.131:0/1870292375 >> [v2:172.21.15.131:6834/640544945,v1:172.21.15.131:6837/640544945] conn(0x55dc98dfa420 0x55dc98dee530 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:02.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.636+0000 7fe2ff3ec740 1 --2- 172.21.15.131:0/1870292375 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dc98dedc70 0x55dc98de0fe0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:02.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.636+0000 7fe2ff3ec740 1 --2- 172.21.15.131:0/1870292375 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dc98dd0d90 0x55dc98de0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:02.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.636+0000 7fe2ff3ec740 1 --2- 172.21.15.131:0/1870292375 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55dc98dc11f0 0x55dc98e693b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:02.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.636+0000 7fe2daa00640 0 client.6624 ms_handle_reset on v2:172.21.15.131:6834/640544945 2024-05-26T22:52:02.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.636+0000 7fe2ff3ec740 1 -- 172.21.15.131:0/1870292375 >> 172.21.15.131:0/1870292375 conn(0x55dc98dd0650 msgr2=0x55dc98deb100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:52:02.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.636+0000 7fe2ff3ec740 1 -- 172.21.15.131:0/1870292375 shutdown_connections 2024-05-26T22:52:02.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.636+0000 7fe2ff3ec740 1 -- 172.21.15.131:0/1870292375 wait complete. 2024-05-26T22:52:02.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.636+0000 7fe2ff3ec740 20 client.6624 trim_cache size 0 max 16384 2024-05-26T22:52:02.640 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-26T22:52:02.640 INFO:teuthology.orchestra.run.smithi131.stdout: "cluster_fsid": "46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f", 2024-05-26T22:52:02.640 INFO:teuthology.orchestra.run.smithi131.stdout: "whoami": 1, 2024-05-26T22:52:02.640 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 6381, 2024-05-26T22:52:02.640 INFO:teuthology.orchestra.run.smithi131.stdout: "want_state": "up:active", 2024-05-26T22:52:02.640 INFO:teuthology.orchestra.run.smithi131.stdout: "state": "up:active", 2024-05-26T22:52:02.640 INFO:teuthology.orchestra.run.smithi131.stdout: "fs_name": "cephfs", 2024-05-26T22:52:02.640 INFO:teuthology.orchestra.run.smithi131.stdout: "rank_uptime": 3.280284462, 2024-05-26T22:52:02.640 INFO:teuthology.orchestra.run.smithi131.stdout: "mdsmap_epoch": 54, 2024-05-26T22:52:02.640 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch": 58, 2024-05-26T22:52:02.640 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch_barrier": 58, 2024-05-26T22:52:02.641 INFO:teuthology.orchestra.run.smithi131.stdout: "uptime": 28.482113667 2024-05-26T22:52:02.641 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-26T22:52:02.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.637+0000 7fe2fc600640 1 -- 172.21.15.131:0/1847352241 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fe2d406be30 msgr2=0x7fe2d406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:02.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.637+0000 7fe2fc600640 1 --2- 172.21.15.131:0/1847352241 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fe2d406be30 0x7fe2d406e2f0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fe2e8006fd0 tx=0x7fe2e8008040 comp rx=0 tx=0).stop 2024-05-26T22:52:02.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.637+0000 7fe2fc600640 1 -- 172.21.15.131:0/1847352241 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2f40cf590 msgr2=0x7fe2f4151100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:02.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.637+0000 7fe2fc600640 1 --2- 172.21.15.131:0/1847352241 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2f40cf590 0x7fe2f4151100 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec00c780 tx=0x7fe2ec00cc40 comp rx=0 tx=0).stop 2024-05-26T22:52:02.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.637+0000 7fe2fc600640 1 -- 172.21.15.131:0/1847352241 shutdown_connections 2024-05-26T22:52:02.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.637+0000 7fe2fc600640 1 --2- 172.21.15.131:0/1847352241 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2f40b5450 0x7fe2f4157150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:02.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.637+0000 7fe2fc600640 1 --2- 172.21.15.131:0/1847352241 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fe2d406be30 0x7fe2d406e2f0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:02.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.637+0000 7fe2fc600640 1 --2- 172.21.15.131:0/1847352241 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2f40cf590 0x7fe2f4151100 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:02.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.637+0000 7fe2fc600640 1 --2- 172.21.15.131:0/1847352241 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe2f406f420 0x7fe2f41595f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:02.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.637+0000 7fe2fc600640 1 -- 172.21.15.131:0/1847352241 >> 172.21.15.131:0/1847352241 conn(0x7fe2f406bc90 msgr2=0x7fe2f40af400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:52:02.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.637+0000 7fe2fc600640 1 -- 172.21.15.131:0/1847352241 shutdown_connections 2024-05-26T22:52:02.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.637+0000 7fe2fc600640 1 -- 172.21.15.131:0/1847352241 wait complete. 2024-05-26T22:52:02.654 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-05-26T22:52:02.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.850+0000 7f65cf200640 1 Processor -- start 2024-05-26T22:52:02.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.850+0000 7f65cf200640 1 -- start start 2024-05-26T22:52:02.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.852+0000 7f65cf200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f65d015b730 0x7f65d015bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:02.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.852+0000 7f65cf200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f65d01400c0 0x7f65d0140510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:02.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.852+0000 7f65cf200640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65d013e820 con 0x7f65d006d250 2024-05-26T22:52:02.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.852+0000 7f65cf200640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f65d0071260 con 0x7f65d01400c0 2024-05-26T22:52:02.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.852+0000 7f65cf200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f65d015c4a0 con 0x7f65d015b730 2024-05-26T22:52:02.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.852+0000 7f65cde00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f65d006d250 0x7f65d006d630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:56040/0 (socket says 172.21.15.131:56040) 2024-05-26T22:52:02.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.852+0000 7f65cde00640 1 -- 172.21.15.131:0/2132491159 learned_addr learned my addr 172.21.15.131:0/2132491159 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:52:02.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.852+0000 7f65ce800640 1 --2- 172.21.15.131:0/2132491159 >> v2:172.21.15.131:3300/0 conn(0x7f65d01400c0 0x7f65d0140510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:02.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.852+0000 7f65c7e00640 1 --2- 172.21.15.131:0/2132491159 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f65d015b730 0x7f65d015bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:02.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.853+0000 7f65ce800640 1 -- 172.21.15.131:0/2132491159 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f65d015b730 msgr2=0x7f65d015bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:02.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.853+0000 7f65ce800640 1 --2- 172.21.15.131:0/2132491159 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f65d015b730 0x7f65d015bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:02.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.853+0000 7f65ce800640 1 -- 172.21.15.131:0/2132491159 >> v1:172.21.15.131:6789/0 conn(0x7f65d006d250 legacy=0x7f65d006d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:02.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.853+0000 7f65cd400640 1 -- 172.21.15.131:0/2132491159 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4245096381 0 0) 0x7f65d013e820 con 0x7f65d006d250 2024-05-26T22:52:02.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.853+0000 7f65ce800640 1 -- 172.21.15.131:0/2132491159 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65d00714b0 con 0x7f65d01400c0 2024-05-26T22:52:02.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.854+0000 7f65ce800640 1 --2- 172.21.15.131:0/2132491159 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f65d01400c0 0x7f65d0140510 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f65d00713e0 tx=0x7f65c802a640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54feb91bea5c33e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:02.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.854+0000 7f65cd400640 1 -- 172.21.15.131:0/2132491159 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c802ae60 con 0x7f65d01400c0 2024-05-26T22:52:02.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.855+0000 7f65cd400640 1 -- 172.21.15.131:0/2132491159 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65c8007740 con 0x7f65d01400c0 2024-05-26T22:52:02.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.855+0000 7f65cd400640 1 -- 172.21.15.131:0/2132491159 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c8007b10 con 0x7f65d01400c0 2024-05-26T22:52:02.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.855+0000 7f65cf200640 1 -- 172.21.15.131:0/2132491159 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f65d01400c0 msgr2=0x7f65d0140510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:02.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.855+0000 7f65cf200640 1 --2- 172.21.15.131:0/2132491159 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f65d01400c0 0x7f65d0140510 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f65d00713e0 tx=0x7f65c802a640 comp rx=0 tx=0).stop 2024-05-26T22:52:02.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.855+0000 7f65cf200640 1 -- 172.21.15.131:0/2132491159 shutdown_connections 2024-05-26T22:52:02.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.855+0000 7f65cf200640 1 --2- 172.21.15.131:0/2132491159 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f65d01400c0 0x7f65d0140510 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:02.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.855+0000 7f65cf200640 1 --2- 172.21.15.131:0/2132491159 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f65d015b730 0x7f65d015bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:02.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.855+0000 7f65cf200640 1 -- 172.21.15.131:0/2132491159 >> 172.21.15.131:0/2132491159 conn(0x7f65d00b5660 msgr2=0x7f65d00b5a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:52:02.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.855+0000 7f65cf200640 1 -- 172.21.15.131:0/2132491159 shutdown_connections 2024-05-26T22:52:02.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.855+0000 7f65cf200640 1 -- 172.21.15.131:0/2132491159 wait complete. 2024-05-26T22:52:02.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.857+0000 7f65cf200640 1 Processor -- start 2024-05-26T22:52:02.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.857+0000 7f65cf200640 1 -- start start 2024-05-26T22:52:02.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.858+0000 7f65cf200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f65d006d250 0x7f65d013ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:02.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.858+0000 7f65cf200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f65d01400c0 0x7f65d0153f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:02.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.858+0000 7f65cf200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f65d015b730 0x7f65d015a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:02.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.858+0000 7f65cf200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f65d015c0d0 con 0x7f65d006d250 2024-05-26T22:52:02.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.858+0000 7f65cf200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f65d0071a70 con 0x7f65d01400c0 2024-05-26T22:52:02.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.858+0000 7f65cf200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f65d015c250 con 0x7f65d015b730 2024-05-26T22:52:02.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.858+0000 7f65cde00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f65d006d250 0x7f65d013ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:02.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.858+0000 7f65cde00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f65d006d250 0x7f65d013ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51048/0 (socket says 172.21.15.131:51048) 2024-05-26T22:52:02.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.858+0000 7f65cde00640 1 -- 172.21.15.131:0/1494176862 learned_addr learned my addr 172.21.15.131:0/1494176862 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:52:02.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.859+0000 7f65cde00640 1 -- 172.21.15.131:0/1494176862 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f65d015b730 msgr2=0x7f65d015a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:02.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.859+0000 7f65cde00640 1 --2- 172.21.15.131:0/1494176862 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f65d015b730 0x7f65d015a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:02.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.859+0000 7f65cde00640 1 -- 172.21.15.131:0/1494176862 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f65d01400c0 msgr2=0x7f65d0153f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:02.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.859+0000 7f65cde00640 1 --2- 172.21.15.131:0/1494176862 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f65d01400c0 0x7f65d0153f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:02.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.859+0000 7f65cde00640 1 -- 172.21.15.131:0/1494176862 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65d013b880 con 0x7f65d006d250 2024-05-26T22:52:02.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.859+0000 7f65cde00640 1 --2- 172.21.15.131:0/1494176862 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f65d006d250 0x7f65d013ad50 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f65c000c430 tx=0x7f65c000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:02.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.859+0000 7f65c6000640 1 -- 172.21.15.131:0/1494176862 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c00074e0 con 0x7f65d006d250 2024-05-26T22:52:02.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.859+0000 7f65c6000640 1 -- 172.21.15.131:0/1494176862 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65c00047e0 con 0x7f65d006d250 2024-05-26T22:52:02.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.859+0000 7f65cf200640 1 -- 172.21.15.131:0/1494176862 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65d013bb60 con 0x7f65d006d250 2024-05-26T22:52:02.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.859+0000 7f65c6000640 1 -- 172.21.15.131:0/1494176862 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c0004bd0 con 0x7f65d006d250 2024-05-26T22:52:02.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.860+0000 7f65cf200640 1 -- 172.21.15.131:0/1494176862 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65d0134bb0 con 0x7f65d006d250 2024-05-26T22:52:02.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.863+0000 7f65c6000640 1 -- 172.21.15.131:0/1494176862 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f65c0011070 con 0x7f65d006d250 2024-05-26T22:52:02.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.864+0000 7f65c6000640 1 --2- 172.21.15.131:0/1494176862 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f65ac06be30 0x7f65ac06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:02.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.864+0000 7f65c6000640 1 -- 172.21.15.131:0/1494176862 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f65c008ac00 con 0x7f65d006d250 2024-05-26T22:52:02.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.864+0000 7f65d8023740 1 Processor -- start 2024-05-26T22:52:02.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.864+0000 7f65c7e00640 1 --2- 172.21.15.131:0/1494176862 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f65ac06be30 0x7f65ac06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:02.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.864+0000 7f65d8023740 1 -- start start 2024-05-26T22:52:02.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.865+0000 7f65c7e00640 1 --2- 172.21.15.131:0/1494176862 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f65ac06be30 0x7f65ac06e2f0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f65d015bc00 tx=0x7f65b80095a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:02.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.865+0000 7f65d8023740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5619d6e33e50 0x5619d6de6a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:02.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.865+0000 7f65d8023740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5619d6e16f70 0x5619d6e162f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:02.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.865+0000 7f65d8023740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5619d6e16830 0x5619d6deb640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:02.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.865+0000 7f65d8023740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5619d6d4e610 con 0x5619d6e16830 2024-05-26T22:52:02.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.865+0000 7f65d8023740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5619d6cf65c0 con 0x5619d6e16f70 2024-05-26T22:52:02.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.865+0000 7f65d8023740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x5619d6d4a5d0 con 0x5619d6e33e50 2024-05-26T22:52:02.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.865+0000 7f65ce800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5619d6e16830 0x5619d6deb640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:02.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.865+0000 7f65ce800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5619d6e16830 0x5619d6deb640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51056/0 (socket says 172.21.15.131:51056) 2024-05-26T22:52:02.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.865+0000 7f65ce800640 1 -- 172.21.15.131:0/2492053588 learned_addr learned my addr 172.21.15.131:0/2492053588 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:52:02.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.866+0000 7f65cde00640 1 --2- 172.21.15.131:0/2492053588 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5619d6e33e50 0x5619d6de6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:02.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.866+0000 7f65ce800640 1 -- 172.21.15.131:0/2492053588 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5619d6e33e50 msgr2=0x5619d6de6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:02.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.866+0000 7f65ce800640 1 --2- 172.21.15.131:0/2492053588 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5619d6e33e50 0x5619d6de6a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:02.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.866+0000 7f65ce800640 1 -- 172.21.15.131:0/2492053588 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5619d6e16f70 msgr2=0x5619d6e162f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:02.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.866+0000 7f65ce800640 1 --2- 172.21.15.131:0/2492053588 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5619d6e16f70 0x5619d6e162f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:02.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.866+0000 7f65ce800640 1 -- 172.21.15.131:0/2492053588 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619d6a622d0 con 0x5619d6e16830 2024-05-26T22:52:02.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.866+0000 7f65ce800640 1 --2- 172.21.15.131:0/2492053588 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5619d6e16830 0x5619d6deb640 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f65c80072e0 tx=0x7f65c802c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41f47123a68f2903 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:02.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.866+0000 7f65bea00640 1 -- 172.21.15.131:0/2492053588 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c8040070 con 0x5619d6e16830 2024-05-26T22:52:02.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.866+0000 7f65bea00640 1 -- 172.21.15.131:0/2492053588 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65c8007540 con 0x5619d6e16830 2024-05-26T22:52:02.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.866+0000 7f65bea00640 1 -- 172.21.15.131:0/2492053588 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c80078f0 con 0x5619d6e16830 2024-05-26T22:52:02.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.867+0000 7f65d8023740 1 -- 172.21.15.131:0/2492053588 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5619d6e16830 msgr2=0x5619d6deb640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:02.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.867+0000 7f65d8023740 1 --2- 172.21.15.131:0/2492053588 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5619d6e16830 0x5619d6deb640 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f65c80072e0 tx=0x7f65c802c000 comp rx=0 tx=0).stop 2024-05-26T22:52:02.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.867+0000 7f65d8023740 1 -- 172.21.15.131:0/2492053588 shutdown_connections 2024-05-26T22:52:02.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.867+0000 7f65d8023740 1 --2- 172.21.15.131:0/2492053588 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5619d6e16830 0x5619d6deb640 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:02.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.867+0000 7f65d8023740 1 --2- 172.21.15.131:0/2492053588 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5619d6e16f70 0x5619d6e162f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:02.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.867+0000 7f65d8023740 1 --2- 172.21.15.131:0/2492053588 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5619d6e33e50 0x5619d6de6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:02.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.867+0000 7f65d8023740 1 -- 172.21.15.131:0/2492053588 >> 172.21.15.131:0/2492053588 conn(0x5619d6de8ce0 msgr2=0x5619d6e312f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:52:02.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.867+0000 7f65d8023740 1 -- 172.21.15.131:0/2492053588 shutdown_connections 2024-05-26T22:52:02.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.867+0000 7f65d8023740 1 -- 172.21.15.131:0/2492053588 wait complete. 2024-05-26T22:52:02.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.868+0000 7f65d8023740 1 -- start start 2024-05-26T22:52:02.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.868+0000 7f65d8023740 1 Processor -- start 2024-05-26T22:52:02.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.869+0000 7f65d8023740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5619d6e16f70 0x5619d6e03db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:02.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.869+0000 7f65d8023740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5619d6e33e50 0x5619d6e44830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:02.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.869+0000 7f65d8023740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5619d6e45520 0x5619d6e44d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:02.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.869+0000 7f65d8023740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5619d6deac00 con 0x5619d6e45520 2024-05-26T22:52:02.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.869+0000 7f65d8023740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5619d6d09910 con 0x5619d6e33e50 2024-05-26T22:52:02.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.869+0000 7f65d8023740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x5619d6de9ff0 con 0x5619d6e16f70 2024-05-26T22:52:02.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.869+0000 7f65d8023740 10 client.0 _ll_register_callbacks cb 0x5619d6d45410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-26T22:52:02.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.869+0000 7f65ce800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5619d6e45520 0x5619d6e44d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:02.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.869+0000 7f65ce800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5619d6e45520 0x5619d6e44d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51070/0 (socket says 172.21.15.131:51070) 2024-05-26T22:52:02.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.869+0000 7f65ce800640 1 -- 172.21.15.131:0/1571905142 learned_addr learned my addr 172.21.15.131:0/1571905142 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:52:02.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.869+0000 7f65ce800640 1 -- 172.21.15.131:0/1571905142 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5619d6e16f70 msgr2=0x5619d6e03db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:02.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.869+0000 7f65ce800640 1 --2- 172.21.15.131:0/1571905142 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5619d6e16f70 0x5619d6e03db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:02.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.870+0000 7f65ce800640 1 -- 172.21.15.131:0/1571905142 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5619d6e33e50 msgr2=0x5619d6e44830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:02.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.870+0000 7f65ce800640 1 --2- 172.21.15.131:0/1571905142 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5619d6e33e50 0x5619d6e44830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:02.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.870+0000 7f65ce800640 1 -- 172.21.15.131:0/1571905142 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619d6b2c6b0 con 0x5619d6e45520 2024-05-26T22:52:02.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.870+0000 7f65ce800640 1 --2- 172.21.15.131:0/1571905142 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5619d6e45520 0x5619d6e44d70 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f65c8009450 tx=0x7f65c8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44d455308cf9beb8 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:02.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.870+0000 7f65b3400640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-26T22:52:02.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.870+0000 7f65b3400640 1 -- 172.21.15.131:0/1571905142 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c8040050 con 0x5619d6e45520 2024-05-26T22:52:02.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.870+0000 7f65b3400640 1 -- 172.21.15.131:0/1571905142 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f65c8007900 con 0x5619d6e45520 2024-05-26T22:52:02.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.870+0000 7f65d8023740 1 -- 172.21.15.131:0/1571905142 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65c8007540 con 0x5619d6e45520 2024-05-26T22:52:02.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.870+0000 7f65b3400640 1 -- 172.21.15.131:0/1571905142 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f65c8058050 con 0x5619d6e45520 2024-05-26T22:52:02.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.870+0000 7f65d8023740 10 client.6639 fetch_fsmap learned FSMap version 54 2024-05-26T22:52:02.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.870+0000 7f65d8023740 1 -- 172.21.15.131:0/1571905142 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) -- 0x5619d6e17600 con 0x5619d6e45520 2024-05-26T22:52:02.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.871+0000 7f65b3400640 1 -- 172.21.15.131:0/1571905142 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65c8007540 con 0x5619d6e45520 2024-05-26T22:52:02.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.871+0000 7f65b3400640 1 -- 172.21.15.131:0/1571905142 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 54) ==== 3534+0+0 (secure 0 0 0) 0x7f65c805b020 con 0x5619d6e45520 2024-05-26T22:52:02.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.871+0000 7f65d8023740 10 client.6639 fetch_fsmap finished waiting for FSMap version 54 2024-05-26T22:52:02.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.871+0000 7f65d8023740 10 client.6639 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-26T22:52:02.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.871+0000 7f65d8023740 20 client.6639 populate_metadata read hostname 'smithi131' 2024-05-26T22:52:02.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.871+0000 7f65d8023740 1 --2- 172.21.15.131:0/1571905142 >> [v2:172.21.15.187:6832/3300872633,v1:172.21.15.187:6833/3300872633] conn(0x5619d6e14ea0 0x5619d6e15280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:02.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.871+0000 7f65d8023740 4 client.6639 mds_command: new command op to 6390 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-26T22:52:02.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.871+0000 7f65d8023740 1 -- 172.21.15.131:0/1571905142 --> [v2:172.21.15.187:6832/3300872633,v1:172.21.15.187:6833/3300872633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5619d6d5bd40 con 0x5619d6e14ea0 2024-05-26T22:52:02.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.871+0000 7f65cde00640 1 --2- 172.21.15.131:0/1571905142 >> [v2:172.21.15.187:6832/3300872633,v1:172.21.15.187:6833/3300872633] conn(0x5619d6e14ea0 0x5619d6e15280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:02.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.872+0000 7f65cde00640 1 --2- 172.21.15.131:0/1571905142 >> [v2:172.21.15.187:6832/3300872633,v1:172.21.15.187:6833/3300872633] conn(0x5619d6e14ea0 0x5619d6e15280 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=711110466a44841a server_cookie=e11094040a8c8e27 in_seq=0 out_seq=0 2024-05-26T22:52:02.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.872+0000 7f65b3400640 10 client.6639 ms_handle_connect on v2:172.21.15.187:6832/3300872633 2024-05-26T22:52:02.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.874+0000 7f65b3400640 1 -- 172.21.15.131:0/1571905142 <== mds.0 v2:172.21.15.187:6832/3300872633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5619d6d5bd40 con 0x5619d6e14ea0 2024-05-26T22:52:02.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.874+0000 7f65b3400640 10 client.6639 handle_command_reply: tid=0 2024-05-26T22:52:02.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.874+0000 7f65d8023740 1 client.6639 shutdown 2024-05-26T22:52:02.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.874+0000 7f65d8023740 1 -- 172.21.15.131:0/1571905142 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5619d6e45520 msgr2=0x5619d6e44d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:02.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.874+0000 7f65d8023740 1 --2- 172.21.15.131:0/1571905142 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5619d6e45520 0x5619d6e44d70 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f65c8009450 tx=0x7f65c8004ab0 comp rx=0 tx=0).stop 2024-05-26T22:52:02.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.874+0000 7f65d8023740 1 -- 172.21.15.131:0/1571905142 shutdown_connections 2024-05-26T22:52:02.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.874+0000 7f65d8023740 1 --2- 172.21.15.131:0/1571905142 >> [v2:172.21.15.187:6832/3300872633,v1:172.21.15.187:6833/3300872633] conn(0x5619d6e14ea0 0x5619d6e15280 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:02.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.874+0000 7f65d8023740 1 --2- 172.21.15.131:0/1571905142 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5619d6e45520 0x5619d6e44d70 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:02.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.874+0000 7f65d8023740 1 --2- 172.21.15.131:0/1571905142 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5619d6e33e50 0x5619d6e44830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:02.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.874+0000 7f65b3400640 0 client.6639 ms_handle_reset on v2:172.21.15.187:6832/3300872633 2024-05-26T22:52:02.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.874+0000 7f65d8023740 1 --2- 172.21.15.131:0/1571905142 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5619d6e16f70 0x5619d6e03db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:02.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.874+0000 7f65d8023740 1 -- 172.21.15.131:0/1571905142 >> 172.21.15.131:0/1571905142 conn(0x5619d6e16830 msgr2=0x5619d6de74d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:52:02.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.875+0000 7f65d8023740 1 -- 172.21.15.131:0/1571905142 shutdown_connections 2024-05-26T22:52:02.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.875+0000 7f65d8023740 1 -- 172.21.15.131:0/1571905142 wait complete. 2024-05-26T22:52:02.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.875+0000 7f65d8023740 20 client.6639 trim_cache size 0 max 16384 2024-05-26T22:52:02.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.895+0000 7f65d8023740 1 Processor -- start 2024-05-26T22:52:02.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.895+0000 7f65d8023740 1 -- start start 2024-05-26T22:52:02.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.895+0000 7f65d8023740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5619d6e16f70 0x5619d6e551e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:02.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.895+0000 7f65d8023740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5619d6e33e50 0x5619d6e4aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:02.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.895+0000 7f65d8023740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5619d6e45520 0x5619d6e447c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:02.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.896+0000 7f65d8023740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f65d0200650 con 0x5619d6e45520 2024-05-26T22:52:02.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.896+0000 7f65d8023740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f65d01ff8b0 con 0x5619d6e16f70 2024-05-26T22:52:02.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.896+0000 7f65d8023740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f65d01fff80 con 0x5619d6e33e50 2024-05-26T22:52:02.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.896+0000 7f65ce800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5619d6e45520 0x5619d6e447c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:02.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.896+0000 7f65ce800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5619d6e45520 0x5619d6e447c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51086/0 (socket says 172.21.15.131:51086) 2024-05-26T22:52:02.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.896+0000 7f65ce800640 1 -- 172.21.15.131:0/1556681337 learned_addr learned my addr 172.21.15.131:0/1556681337 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:52:02.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.896+0000 7f65cde00640 1 --2- 172.21.15.131:0/1556681337 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5619d6e16f70 0x5619d6e551e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:02.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.896+0000 7f65c7e00640 1 --2- 172.21.15.131:0/1556681337 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5619d6e33e50 0x5619d6e4aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:02.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.896+0000 7f65ce800640 1 -- 172.21.15.131:0/1556681337 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5619d6e33e50 msgr2=0x5619d6e4aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:02.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.896+0000 7f65ce800640 1 --2- 172.21.15.131:0/1556681337 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5619d6e33e50 0x5619d6e4aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:02.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.896+0000 7f65ce800640 1 -- 172.21.15.131:0/1556681337 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5619d6e16f70 msgr2=0x5619d6e551e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:02.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.896+0000 7f65ce800640 1 --2- 172.21.15.131:0/1556681337 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5619d6e16f70 0x5619d6e551e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:02.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.896+0000 7f65ce800640 1 -- 172.21.15.131:0/1556681337 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619d6de7240 con 0x5619d6e45520 2024-05-26T22:52:02.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.897+0000 7f65ce800640 1 --2- 172.21.15.131:0/1556681337 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5619d6e45520 0x5619d6e447c0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f65c8007300 tx=0x7f65c8047070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aab0b2ab4283e2b4 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:02.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.897+0000 7f65be000640 1 -- 172.21.15.131:0/1556681337 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c8040070 con 0x5619d6e45520 2024-05-26T22:52:02.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.897+0000 7f65be000640 1 -- 172.21.15.131:0/1556681337 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65c8007900 con 0x5619d6e45520 2024-05-26T22:52:02.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.897+0000 7f65be000640 1 -- 172.21.15.131:0/1556681337 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c8004040 con 0x5619d6e45520 2024-05-26T22:52:02.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.897+0000 7f65d8023740 1 -- 172.21.15.131:0/1556681337 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5619d6e45520 msgr2=0x5619d6e447c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:02.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.897+0000 7f65d8023740 1 --2- 172.21.15.131:0/1556681337 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5619d6e45520 0x5619d6e447c0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f65c8007300 tx=0x7f65c8047070 comp rx=0 tx=0).stop 2024-05-26T22:52:02.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.898+0000 7f65d8023740 1 -- 172.21.15.131:0/1556681337 shutdown_connections 2024-05-26T22:52:02.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.898+0000 7f65d8023740 1 --2- 172.21.15.131:0/1556681337 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5619d6e33e50 0x5619d6e4aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:02.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.898+0000 7f65d8023740 1 --2- 172.21.15.131:0/1556681337 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5619d6e45520 0x5619d6e447c0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:02.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.898+0000 7f65d8023740 1 --2- 172.21.15.131:0/1556681337 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5619d6e16f70 0x5619d6e551e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:02.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.898+0000 7f65d8023740 1 -- 172.21.15.131:0/1556681337 >> 172.21.15.131:0/1556681337 conn(0x5619d6e16830 msgr2=0x5619d6de74d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:52:02.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.898+0000 7f65d8023740 1 -- 172.21.15.131:0/1556681337 shutdown_connections 2024-05-26T22:52:02.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.898+0000 7f65d8023740 1 -- 172.21.15.131:0/1556681337 wait complete. 2024-05-26T22:52:02.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.898+0000 7f65d8023740 1 -- start start 2024-05-26T22:52:02.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.898+0000 7f65d8023740 1 Processor -- start 2024-05-26T22:52:02.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.899+0000 7f65d8023740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5619d6e16f70 0x5619d6e280a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:02.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.899+0000 7f65d8023740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5619d6e33e50 0x5619d6de64f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:02.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.899+0000 7f65d8023740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5619d6e45520 0x5619d6e19ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:02.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.899+0000 7f65d8023740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5619d6deac00 con 0x5619d6e16f70 2024-05-26T22:52:02.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.899+0000 7f65d8023740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f65d01fe8f0 con 0x5619d6e33e50 2024-05-26T22:52:02.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.899+0000 7f65d8023740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x5619d6e17bc0 con 0x5619d6e45520 2024-05-26T22:52:02.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.899+0000 7f65d8023740 10 client.0 _ll_register_callbacks cb 0x7f65c8009c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-26T22:52:02.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.900+0000 7f65cde00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5619d6e16f70 0x5619d6e280a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:02.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.900+0000 7f65cde00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5619d6e16f70 0x5619d6e280a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51090/0 (socket says 172.21.15.131:51090) 2024-05-26T22:52:02.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.900+0000 7f65cde00640 1 -- 172.21.15.131:0/2961460545 learned_addr learned my addr 172.21.15.131:0/2961460545 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:52:02.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.900+0000 7f65ce800640 1 --2- 172.21.15.131:0/2961460545 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5619d6e45520 0x5619d6e19ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:02.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.900+0000 7f65c7e00640 1 --2- 172.21.15.131:0/2961460545 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5619d6e33e50 0x5619d6de64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:02.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.900+0000 7f65cde00640 1 -- 172.21.15.131:0/2961460545 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5619d6e45520 msgr2=0x5619d6e19ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:02.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.900+0000 7f65cde00640 1 --2- 172.21.15.131:0/2961460545 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5619d6e45520 0x5619d6e19ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:02.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.900+0000 7f65cde00640 1 -- 172.21.15.131:0/2961460545 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5619d6e33e50 msgr2=0x5619d6de64f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:02.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.900+0000 7f65cde00640 1 --2- 172.21.15.131:0/2961460545 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5619d6e33e50 0x5619d6de64f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:02.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.900+0000 7f65cde00640 1 -- 172.21.15.131:0/2961460545 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619d6e54b00 con 0x5619d6e16f70 2024-05-26T22:52:02.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.900+0000 7f65cde00640 1 --2- 172.21.15.131:0/2961460545 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5619d6e16f70 0x5619d6e280a0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f65c0058d10 tx=0x7f65c0055120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93aaaca205af2fd7 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:02.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.900+0000 7f65b3400640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-26T22:52:02.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.901+0000 7f65b3400640 1 -- 172.21.15.131:0/2961460545 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c0077060 con 0x5619d6e16f70 2024-05-26T22:52:02.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.901+0000 7f65b3400640 1 -- 172.21.15.131:0/2961460545 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f65c0004d70 con 0x5619d6e16f70 2024-05-26T22:52:02.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.901+0000 7f65d8023740 1 -- 172.21.15.131:0/2961460545 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65c8007900 con 0x5619d6e16f70 2024-05-26T22:52:02.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.901+0000 7f65b3400640 1 -- 172.21.15.131:0/2961460545 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f65c007e050 con 0x5619d6e16f70 2024-05-26T22:52:02.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.901+0000 7f65d8023740 10 client.6645 fetch_fsmap learned FSMap version 54 2024-05-26T22:52:02.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.901+0000 7f65d8023740 1 -- 172.21.15.131:0/2961460545 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) -- 0x5619d6de74d0 con 0x5619d6e16f70 2024-05-26T22:52:02.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.901+0000 7f65b3400640 1 -- 172.21.15.131:0/2961460545 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65c8007900 con 0x5619d6e16f70 2024-05-26T22:52:02.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.901+0000 7f65b3400640 1 -- 172.21.15.131:0/2961460545 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 54) ==== 3534+0+0 (secure 0 0 0) 0x7f65c0081020 con 0x5619d6e16f70 2024-05-26T22:52:02.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.901+0000 7f65d8023740 10 client.6645 fetch_fsmap finished waiting for FSMap version 54 2024-05-26T22:52:02.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.901+0000 7f65d8023740 10 client.6645 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-26T22:52:02.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.901+0000 7f65d8023740 20 client.6645 populate_metadata read hostname 'smithi131' 2024-05-26T22:52:02.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.901+0000 7f65d8023740 1 --2- 172.21.15.131:0/2961460545 >> [v2:172.21.15.187:6832/3300872633,v1:172.21.15.187:6833/3300872633] conn(0x5619d6e319a0 0x5619d6e31d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:02.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.901+0000 7f65d8023740 4 client.6645 mds_command: new command op to 6390 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-26T22:52:02.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.901+0000 7f65d8023740 1 -- 172.21.15.131:0/2961460545 --> [v2:172.21.15.187:6832/3300872633,v1:172.21.15.187:6833/3300872633] -- command(tid 0: {"prefix": "status"}) -- 0x5619d6d56090 con 0x5619d6e319a0 2024-05-26T22:52:02.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.902+0000 7f65c7e00640 1 --2- 172.21.15.131:0/2961460545 >> [v2:172.21.15.187:6832/3300872633,v1:172.21.15.187:6833/3300872633] conn(0x5619d6e319a0 0x5619d6e31d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:02.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.903+0000 7f65c7e00640 1 --2- 172.21.15.131:0/2961460545 >> [v2:172.21.15.187:6832/3300872633,v1:172.21.15.187:6833/3300872633] conn(0x5619d6e319a0 0x5619d6e31d80 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b73717cf271ed99b server_cookie=775dc2b1fbbd2023 in_seq=0 out_seq=0 2024-05-26T22:52:02.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.903+0000 7f65b3400640 10 client.6645 ms_handle_connect on v2:172.21.15.187:6832/3300872633 2024-05-26T22:52:02.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.903+0000 7f65b3400640 1 -- 172.21.15.131:0/2961460545 <== mds.0 v2:172.21.15.187:6832/3300872633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x5619d6d56090 con 0x5619d6e319a0 2024-05-26T22:52:02.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.903+0000 7f65b3400640 10 client.6645 handle_command_reply: tid=0 2024-05-26T22:52:02.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.903+0000 7f65d8023740 1 client.6645 shutdown 2024-05-26T22:52:02.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.903+0000 7f65d8023740 1 -- 172.21.15.131:0/2961460545 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5619d6e16f70 msgr2=0x5619d6e280a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:02.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.903+0000 7f65d8023740 1 --2- 172.21.15.131:0/2961460545 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5619d6e16f70 0x5619d6e280a0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f65c0058d10 tx=0x7f65c0055120 comp rx=0 tx=0).stop 2024-05-26T22:52:02.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.904+0000 7f65d8023740 1 -- 172.21.15.131:0/2961460545 shutdown_connections 2024-05-26T22:52:02.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.904+0000 7f65d8023740 1 --2- 172.21.15.131:0/2961460545 >> [v2:172.21.15.187:6832/3300872633,v1:172.21.15.187:6833/3300872633] conn(0x5619d6e319a0 0x5619d6e31d80 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:02.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.904+0000 7f65d8023740 1 --2- 172.21.15.131:0/2961460545 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5619d6e45520 0x5619d6e19ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:02.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.904+0000 7f65d8023740 1 --2- 172.21.15.131:0/2961460545 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5619d6e33e50 0x5619d6de64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:02.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.904+0000 7f65d8023740 1 --2- 172.21.15.131:0/2961460545 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5619d6e16f70 0x5619d6e280a0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:02.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.904+0000 7f65b3400640 0 client.6645 ms_handle_reset on v2:172.21.15.187:6832/3300872633 2024-05-26T22:52:02.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.904+0000 7f65d8023740 1 -- 172.21.15.131:0/2961460545 >> 172.21.15.131:0/2961460545 conn(0x5619d6e16830 msgr2=0x5619d6e55130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:52:02.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.904+0000 7f65d8023740 1 -- 172.21.15.131:0/2961460545 shutdown_connections 2024-05-26T22:52:02.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.904+0000 7f65d8023740 1 -- 172.21.15.131:0/2961460545 wait complete. 2024-05-26T22:52:02.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.904+0000 7f65d8023740 20 client.6645 trim_cache size 0 max 16384 2024-05-26T22:52:02.907 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-26T22:52:02.907 INFO:teuthology.orchestra.run.smithi131.stdout: "cluster_fsid": "46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f", 2024-05-26T22:52:02.907 INFO:teuthology.orchestra.run.smithi131.stdout: "whoami": 0, 2024-05-26T22:52:02.907 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 6390, 2024-05-26T22:52:02.907 INFO:teuthology.orchestra.run.smithi131.stdout: "want_state": "up:active", 2024-05-26T22:52:02.908 INFO:teuthology.orchestra.run.smithi131.stdout: "state": "up:active", 2024-05-26T22:52:02.908 INFO:teuthology.orchestra.run.smithi131.stdout: "fs_name": "cephfs", 2024-05-26T22:52:02.908 INFO:teuthology.orchestra.run.smithi131.stdout: "rank_uptime": 23.235454626999999, 2024-05-26T22:52:02.908 INFO:teuthology.orchestra.run.smithi131.stdout: "mdsmap_epoch": 54, 2024-05-26T22:52:02.908 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch": 58, 2024-05-26T22:52:02.908 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch_barrier": 58, 2024-05-26T22:52:02.908 INFO:teuthology.orchestra.run.smithi131.stdout: "uptime": 28.730328387 2024-05-26T22:52:02.908 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-26T22:52:02.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.904+0000 7f65cf200640 1 -- 172.21.15.131:0/1494176862 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f65ac06be30 msgr2=0x7f65ac06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:02.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.904+0000 7f65cf200640 1 --2- 172.21.15.131:0/1494176862 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f65ac06be30 0x7f65ac06e2f0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f65d015bc00 tx=0x7f65b80095a0 comp rx=0 tx=0).stop 2024-05-26T22:52:02.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.904+0000 7f65cf200640 1 -- 172.21.15.131:0/1494176862 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f65d006d250 msgr2=0x7f65d013ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:02.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.904+0000 7f65cf200640 1 --2- 172.21.15.131:0/1494176862 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f65d006d250 0x7f65d013ad50 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f65c000c430 tx=0x7f65c000c8f0 comp rx=0 tx=0).stop 2024-05-26T22:52:02.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.904+0000 7f65cf200640 1 -- 172.21.15.131:0/1494176862 shutdown_connections 2024-05-26T22:52:02.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.904+0000 7f65cf200640 1 --2- 172.21.15.131:0/1494176862 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f65d015b730 0x7f65d015a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:02.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.905+0000 7f65cf200640 1 --2- 172.21.15.131:0/1494176862 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f65ac06be30 0x7f65ac06e2f0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:02.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.905+0000 7f65cf200640 1 --2- 172.21.15.131:0/1494176862 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f65d01400c0 0x7f65d0153f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:02.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.905+0000 7f65cf200640 1 --2- 172.21.15.131:0/1494176862 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f65d006d250 0x7f65d013ad50 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:02.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.905+0000 7f65cf200640 1 -- 172.21.15.131:0/1494176862 >> 172.21.15.131:0/1494176862 conn(0x7f65d00b5660 msgr2=0x7f65d006d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:52:02.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.905+0000 7f65cf200640 1 -- 172.21.15.131:0/1494176862 shutdown_connections 2024-05-26T22:52:02.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:02.905+0000 7f65cf200640 1 -- 172.21.15.131:0/1494176862 wait complete. 2024-05-26T22:52:02.922 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-05-26T22:52:02.922 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-26T22:52:02.922 DEBUG:teuthology.orchestra.run.smithi131:> mkdir -p tree 2024-05-26T22:52:02.922 DEBUG:teuthology.orchestra.run.smithi131:> 2024-05-26T22:52:02.922 DEBUG:teuthology.orchestra.run.smithi131:> setfattr -n ceph.dir.pin.distributed -v 1 tree 2024-05-26T22:52:02.923 DEBUG:teuthology.orchestra.run.smithi131:> 2024-05-26T22:52:02.923 DEBUG:teuthology.orchestra.run.smithi131:> for ((i = 0; i < 100; i++)); do 2024-05-26T22:52:02.923 DEBUG:teuthology.orchestra.run.smithi131:> mkdir -p "tree/$i" 2024-05-26T22:52:02.923 DEBUG:teuthology.orchestra.run.smithi131:> echo file > "tree/$i/file" 2024-05-26T22:52:02.923 DEBUG:teuthology.orchestra.run.smithi131:> done 2024-05-26T22:52:02.923 DEBUG:teuthology.orchestra.run.smithi131:> ') 2024-05-26T22:52:03.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:03 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-26T22:52:03.025 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree 2024-05-26T22:52:03.034 INFO:teuthology.orchestra.run.smithi131.stderr:+ setfattr -n ceph.dir.pin.distributed -v 1 tree 2024-05-26T22:52:03.045 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i = 0 )) 2024-05-26T22:52:03.045 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:52:03.045 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/0 2024-05-26T22:52:03.049 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:52:03.058 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:52:03.058 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:52:03.058 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/1 2024-05-26T22:52:03.062 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:52:03.065 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:52:03.065 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:52:03.065 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/2 2024-05-26T22:52:03.070 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:52:03.073 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:52:03.073 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:52:03.073 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/3 2024-05-26T22:52:03.078 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:52:03.081 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:52:03.081 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:52:03.081 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/4 2024-05-26T22:52:03.086 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:52:03.089 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:52:03.089 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:52:03.089 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/5 2024-05-26T22:52:03.095 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:52:03.097 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:52:03.097 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:52:03.097 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/6 2024-05-26T22:52:03.103 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:52:03.105 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:52:03.105 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:52:03.106 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/7 2024-05-26T22:52:03.112 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:52:03.114 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:52:03.114 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:52:03.115 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/8 2024-05-26T22:52:03.120 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:52:03.122 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:52:03.122 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:52:03.123 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/9 2024-05-26T22:52:03.128 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:52:03.130 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:52:03.130 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:52:03.130 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/10 2024-05-26T22:52:03.136 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:52:03.138 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:52:03.138 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:52:03.139 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/11 2024-05-26T22:52:03.144 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:52:03.147 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:52:03.147 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:52:03.147 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/12 2024-05-26T22:52:03.153 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:52:03.155 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:52:03.155 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:52:03.155 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/13 2024-05-26T22:52:03.161 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:52:03.163 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:52:03.163 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:52:03.163 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/14 2024-05-26T22:52:03.170 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:52:03.172 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:52:03.172 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:52:03.173 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/15 2024-05-26T22:52:03.178 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:52:03.181 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:52:03.181 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:52:03.181 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/16 2024-05-26T22:52:03.186 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:52:03.189 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:52:03.189 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:52:03.189 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/17 2024-05-26T22:52:03.195 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:52:03.198 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:52:03.198 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:52:03.198 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/18 2024-05-26T22:52:03.204 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:52:03.207 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:52:03.207 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:52:03.207 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/19 2024-05-26T22:52:03.213 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:52:03.216 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:52:03.216 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:52:03.216 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/20 2024-05-26T22:52:03.221 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:52:03.223 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:52:03.223 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:52:03.223 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/21 2024-05-26T22:52:03.229 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:52:03.232 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:52:03.232 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:52:03.232 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/22 2024-05-26T22:52:03.238 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:52:03.240 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:52:03.241 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:52:03.241 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/23 2024-05-26T22:52:03.246 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:52:03.249 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:52:03.249 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:52:03.249 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/24 2024-05-26T22:52:03.255 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:52:03.257 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:52:03.258 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:52:03.258 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/25 2024-05-26T22:52:03.264 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:52:03.266 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:52:03.266 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:52:03.266 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/26 2024-05-26T22:52:03.272 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:52:03.274 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:52:03.275 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:52:03.275 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/27 2024-05-26T22:52:03.280 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:52:03.282 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:52:03.282 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:52:03.283 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/28 2024-05-26T22:52:03.288 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:52:03.291 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:52:03.291 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:52:03.291 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/29 2024-05-26T22:52:03.297 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:52:03.299 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:52:03.300 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:52:03.300 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/30 2024-05-26T22:52:03.305 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:52:03.308 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:52:03.308 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:52:03.308 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/31 2024-05-26T22:52:03.315 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:52:03.317 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:52:03.317 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:52:03.318 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/32 2024-05-26T22:52:03.323 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:52:03.326 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:52:03.326 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:52:03.326 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/33 2024-05-26T22:52:03.332 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:52:03.334 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:52:03.334 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:52:03.334 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/34 2024-05-26T22:52:03.340 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:52:03.343 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:52:03.343 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:52:03.343 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/35 2024-05-26T22:52:03.349 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:52:03.352 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:52:03.352 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:52:03.352 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/36 2024-05-26T22:52:03.358 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:52:03.361 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:52:03.361 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:52:03.361 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/37 2024-05-26T22:52:03.367 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:52:03.369 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:52:03.369 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:52:03.370 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/38 2024-05-26T22:52:03.376 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:52:03.380 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:52:03.380 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:52:03.380 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/39 2024-05-26T22:52:03.386 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:52:03.389 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:52:03.389 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:52:03.389 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/40 2024-05-26T22:52:03.395 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:52:03.398 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:52:03.398 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:52:03.398 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/41 2024-05-26T22:52:03.403 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:52:03.407 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:52:03.407 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:52:03.407 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/42 2024-05-26T22:52:03.413 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:52:03.416 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:52:03.416 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:52:03.416 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/43 2024-05-26T22:52:03.421 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:52:03.424 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:52:03.424 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:52:03.424 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/44 2024-05-26T22:52:03.430 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:52:03.432 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:52:03.432 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:52:03.432 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/45 2024-05-26T22:52:03.438 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:52:03.441 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:52:03.441 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:52:03.441 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/46 2024-05-26T22:52:03.447 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:52:03.450 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:52:03.450 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:52:03.450 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/47 2024-05-26T22:52:03.456 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:52:03.459 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:52:03.459 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:52:03.459 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/48 2024-05-26T22:52:03.465 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:52:03.467 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:52:03.467 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:52:03.468 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/49 2024-05-26T22:52:03.474 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:52:03.476 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:52:03.476 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:52:03.476 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/50 2024-05-26T22:52:03.483 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:52:03.485 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:52:03.485 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:52:03.485 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/51 2024-05-26T22:52:03.491 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:52:03.494 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:52:03.494 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:52:03.494 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/52 2024-05-26T22:52:03.500 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:52:03.502 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:52:03.502 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:52:03.502 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/53 2024-05-26T22:52:03.508 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:52:03.511 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:52:03.511 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:52:03.511 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/54 2024-05-26T22:52:03.517 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:52:03.519 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:52:03.519 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:52:03.520 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/55 2024-05-26T22:52:03.526 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:52:03.528 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:52:03.529 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:52:03.529 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/56 2024-05-26T22:52:03.534 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:52:03.537 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:52:03.537 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:52:03.537 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/57 2024-05-26T22:52:03.543 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:52:03.545 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:52:03.545 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:52:03.546 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/58 2024-05-26T22:52:03.551 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:52:03.554 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:52:03.554 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:52:03.554 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/59 2024-05-26T22:52:03.560 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:52:03.562 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:52:03.562 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:52:03.562 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/60 2024-05-26T22:52:03.568 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:52:03.570 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:52:03.570 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:52:03.570 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/61 2024-05-26T22:52:03.575 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:52:03.578 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:52:03.578 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:52:03.578 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/62 2024-05-26T22:52:03.584 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:52:03.586 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:52:03.586 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:52:03.586 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/63 2024-05-26T22:52:03.593 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:52:03.595 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:52:03.595 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:52:03.595 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/64 2024-05-26T22:52:03.601 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:52:03.604 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:52:03.604 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:52:03.604 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/65 2024-05-26T22:52:03.610 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:52:03.612 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:52:03.612 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:52:03.612 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/66 2024-05-26T22:52:03.618 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:52:03.621 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:52:03.621 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:52:03.621 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/67 2024-05-26T22:52:03.627 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:52:03.629 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:52:03.629 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:52:03.630 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/68 2024-05-26T22:52:03.635 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:52:03.637 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:52:03.637 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:52:03.638 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/69 2024-05-26T22:52:03.644 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:52:03.646 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:52:03.646 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:52:03.646 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/70 2024-05-26T22:52:03.654 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:52:03.656 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:52:03.657 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:52:03.657 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/71 2024-05-26T22:52:03.662 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:52:03.665 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:52:03.665 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:52:03.665 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/72 2024-05-26T22:52:03.671 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:52:03.675 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:52:03.676 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:52:03.676 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/73 2024-05-26T22:52:03.684 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:52:03.688 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:52:03.688 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:52:03.688 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/74 2024-05-26T22:52:03.694 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:52:03.697 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:52:03.697 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:52:03.697 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/75 2024-05-26T22:52:03.702 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:52:03.705 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:52:03.705 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:52:03.705 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/76 2024-05-26T22:52:03.711 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:52:03.713 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:52:03.713 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:52:03.714 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/77 2024-05-26T22:52:03.719 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:52:03.721 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:52:03.722 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:52:03.722 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/78 2024-05-26T22:52:03.727 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:52:03.730 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:52:03.730 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:52:03.730 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/79 2024-05-26T22:52:03.736 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:52:03.739 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:52:03.739 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:52:03.740 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/80 2024-05-26T22:52:03.746 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:52:03.750 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:52:03.750 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:52:03.750 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/81 2024-05-26T22:52:03.757 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:52:03.760 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:52:03.760 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:52:03.760 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/82 2024-05-26T22:52:03.767 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:52:03.770 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:52:03.770 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:52:03.771 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/83 2024-05-26T22:52:03.776 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:52:03.778 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:52:03.779 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:52:03.779 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/84 2024-05-26T22:52:03.784 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:52:03.787 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:52:03.788 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:52:03.788 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/85 2024-05-26T22:52:03.793 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:52:03.796 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:52:03.796 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:52:03.796 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/86 2024-05-26T22:52:03.804 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:52:03.807 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:52:03.807 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:52:03.808 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/87 2024-05-26T22:52:03.813 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:52:03.816 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:52:03.816 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:52:03.816 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/88 2024-05-26T22:52:03.822 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:52:03.824 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:52:03.824 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:52:03.824 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/89 2024-05-26T22:52:03.830 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:52:03.833 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:52:03.833 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:52:03.833 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/90 2024-05-26T22:52:03.839 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:52:03.841 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:52:03.841 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:52:03.841 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/91 2024-05-26T22:52:03.847 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:52:03.850 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:52:03.850 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:52:03.850 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/92 2024-05-26T22:52:03.856 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:52:03.858 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:52:03.858 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:52:03.858 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/93 2024-05-26T22:52:03.864 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:52:03.867 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:52:03.867 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:52:03.867 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/94 2024-05-26T22:52:03.874 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:52:03.877 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:52:03.877 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:52:03.877 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/95 2024-05-26T22:52:03.883 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:52:03.885 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:52:03.886 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:52:03.886 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/96 2024-05-26T22:52:03.891 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:52:03.894 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:52:03.894 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:52:03.894 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/97 2024-05-26T22:52:03.900 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:52:03.903 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:52:03.903 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:52:03.903 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/98 2024-05-26T22:52:03.908 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:52:03.911 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:52:03.911 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:52:03.911 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/99 2024-05-26T22:52:03.917 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:52:03.920 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:52:03.920 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:52:03.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-26T22:52:03.933 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-05-26T22:52:04.160 INFO:teuthology.orchestra.run.smithi131.stdout:[{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds2","dirfrag":"0x102","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"2","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-05-26T22:52:04.161 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "2", "dirfrag": "0x102", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds2", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-05-26T22:52:04.161 DEBUG:tasks.cephfs.cephfs_test_case:[ { "is_auth": true, "auth_first": 2, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "~mds2", "dirfrag": "0x102", "snapid_first": 2, "projected_version": "1", "version": "1", "committing_version": "1", "committed_version": "1", "is_rep": false, "dir_auth": "2", "states": [ "auth", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "dirty": 0, "authpin": 0 }, "nref": 2 } } ] 2024-05-26T22:52:04.161 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-05-26T22:52:04.384 INFO:teuthology.orchestra.run.smithi131.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-05-26T22:52:04.384 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-05-26T22:52:04.384 DEBUG:tasks.cephfs.cephfs_test_case:[ { "is_auth": true, "auth_first": 1, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "~mds1", "dirfrag": "0x101", "snapid_first": 2, "projected_version": "1", "version": "1", "committing_version": "1", "committed_version": "1", "is_rep": false, "dir_auth": "1", "states": [ "auth", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "dirty": 0, "authpin": 0 }, "nref": 2 } } ] 2024-05-26T22:52:04.385 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json get subtrees 2024-05-26T22:52:04.613 INFO:teuthology.orchestra.run.smithi187.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"4000","version":"4000","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"3514","version":"3514","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":1,"authpin":0},"nref":3}}] 2024-05-26T22:52:04.614 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "4000", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "4000" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 1, "subtree": 1 }, "projected_version": "3514", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "3514" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-05-26T22:52:04.614 DEBUG:tasks.cephfs.cephfs_test_case:[ { "is_auth": true, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "", "dirfrag": "0x1", "snapid_first": 2, "projected_version": "4000", "version": "4000", "committing_version": "1", "committed_version": "1", "is_rep": true, "dir_auth": "0", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "replicated": 0, "dirty": 1, "authpin": 0 }, "nref": 3 } }, { "is_auth": true, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "~mds0", "dirfrag": "0x100", "snapid_first": 2, "projected_version": "3514", "version": "3514", "committing_version": "1", "committed_version": "1", "is_rep": false, "dir_auth": "0", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "dirty": 1, "authpin": 0 }, "nref": 3 } } ] 2024-05-26T22:52:04.615 INFO:tasks.cephfs.cephfs_test_case:len=0 [] 2024-05-26T22:52:05.609 INFO:tasks.ceph.mds.d.smithi187.stderr:2024-05-26T22:52:05.609+0000 7fbf69e00640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-05-26T22:52:09.616 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-05-26T22:52:09.844 INFO:teuthology.orchestra.run.smithi131.stdout:[{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds2","dirfrag":"0x102","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"2","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-05-26T22:52:09.845 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "2", "dirfrag": "0x102", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds2", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-05-26T22:52:09.845 DEBUG:tasks.cephfs.cephfs_test_case:[ { "is_auth": true, "auth_first": 2, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "~mds2", "dirfrag": "0x102", "snapid_first": 2, "projected_version": "1", "version": "1", "committing_version": "1", "committed_version": "1", "is_rep": false, "dir_auth": "2", "states": [ "auth", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "dirty": 0, "authpin": 0 }, "nref": 2 } } ] 2024-05-26T22:52:09.846 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-05-26T22:52:10.070 INFO:teuthology.orchestra.run.smithi131.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-05-26T22:52:10.071 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-05-26T22:52:10.071 DEBUG:tasks.cephfs.cephfs_test_case:[ { "is_auth": true, "auth_first": 1, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "~mds1", "dirfrag": "0x101", "snapid_first": 2, "projected_version": "1", "version": "1", "committing_version": "1", "committed_version": "1", "is_rep": false, "dir_auth": "1", "states": [ "auth", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "dirty": 0, "authpin": 0 }, "nref": 2 } } ] 2024-05-26T22:52:10.072 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json get subtrees 2024-05-26T22:52:10.302 INFO:teuthology.orchestra.run.smithi187.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"4008","version":"4008","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"3514","version":"3514","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":1,"authpin":0},"nref":3}}] 2024-05-26T22:52:10.302 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "4008", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "4008" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 1, "subtree": 1 }, "projected_version": "3514", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "3514" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-05-26T22:52:10.303 DEBUG:tasks.cephfs.cephfs_test_case:[ { "is_auth": true, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "", "dirfrag": "0x1", "snapid_first": 2, "projected_version": "4008", "version": "4008", "committing_version": "1", "committed_version": "1", "is_rep": true, "dir_auth": "0", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "replicated": 0, "dirty": 1, "authpin": 0 }, "nref": 3 } }, { "is_auth": true, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "~mds0", "dirfrag": "0x100", "snapid_first": 2, "projected_version": "3514", "version": "3514", "committing_version": "1", "committed_version": "1", "is_rep": false, "dir_auth": "0", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "dirty": 1, "authpin": 0 }, "nref": 3 } } ] 2024-05-26T22:52:10.303 INFO:tasks.cephfs.cephfs_test_case:len=0 [] 2024-05-26T22:52:14.823 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-26T22:52:14.832 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-26T22:52:15.304 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-05-26T22:52:15.534 INFO:teuthology.orchestra.run.smithi131.stdout:[{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/tree","dirfrag":"0x10000000584.111*","snapid_first":2,"projected_version":"602","version":"602","committing_version":"602","committed_version":"602","is_rep":false,"dir_auth":"2","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1},"nref":3}},{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds2","dirfrag":"0x102","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"2","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"4008","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":2},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/tree","dirfrag":"0x10000000584.000*","snapid_first":2,"projected_version":"602","version":"602","committing_version":"602","committed_version":"602","is_rep":false,"dir_auth":"2","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1},"nref":3}}] 2024-05-26T22:52:15.536 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "602", "committing_version": "602", "dir_auth": "2", "dirfrag": "0x10000000584.111*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "/tree", "pins": { "child": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "602", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "602" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 2, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "2", "dirfrag": "0x102", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds2", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 2 }, "snapid_first": 2, "states": [], "version": "4008" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "602", "committing_version": "602", "dir_auth": "2", "dirfrag": "0x10000000584.000*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "/tree", "pins": { "child": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "602", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "602" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 2, "is_auth": true, "random_ephemeral_pin": false } ] 2024-05-26T22:52:15.536 DEBUG:tasks.cephfs.cephfs_test_case:[ { "is_auth": true, "auth_first": 2, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 2, "dir": { "path": "/tree", "dirfrag": "0x10000000584.111*", "snapid_first": 2, "projected_version": "602", "version": "602", "committing_version": "602", "committed_version": "602", "is_rep": false, "dir_auth": "2", "states": [ "auth", "complete" ], "is_auth": true, "auth_state": { "replicas": { "0": 1 } }, "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "child": 1, "frozen": 0, "subtree": 1, "importing": 0, "replicated": 1 }, "nref": 3 } }, { "is_auth": true, "auth_first": 2, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "~mds2", "dirfrag": "0x102", "snapid_first": 2, "projected_version": "1", "version": "1", "committing_version": "1", "committed_version": "1", "is_rep": false, "dir_auth": "2", "states": [ "auth", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "dirty": 0, "authpin": 0 }, "nref": 2 } }, { "is_auth": false, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "", "dirfrag": "0x1", "snapid_first": 2, "projected_version": "0", "version": "4008", "committing_version": "0", "committed_version": "0", "is_rep": true, "dir_auth": "0", "states": [], "is_auth": false, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 2 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1 }, "nref": 2 } }, { "is_auth": true, "auth_first": 2, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 2, "dir": { "path": "/tree", "dirfrag": "0x10000000584.000*", "snapid_first": 2, "projected_version": "602", "version": "602", "committing_version": "602", "committed_version": "602", "is_rep": false, "dir_auth": "2", "states": [ "auth", "complete" ], "is_auth": true, "auth_state": { "replicas": { "0": 1 } }, "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "child": 1, "frozen": 0, "subtree": 1, "importing": 0, "replicated": 1 }, "nref": 3 } } ] 2024-05-26T22:52:15.536 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-05-26T22:52:15.762 INFO:teuthology.orchestra.run.smithi131.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree","dirfrag":"0x10000000584.110*","snapid_first":2,"projected_version":"602","version":"602","committing_version":"602","committed_version":"602","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1},"nref":3}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"4008","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":3},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree","dirfrag":"0x10000000584.101*","snapid_first":2,"projected_version":"602","version":"602","committing_version":"602","committed_version":"602","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1},"nref":3}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree","dirfrag":"0x10000000584.100*","snapid_first":2,"projected_version":"602","version":"602","committing_version":"602","committed_version":"602","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1},"nref":3}}] 2024-05-26T22:52:15.763 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "602", "committing_version": "602", "dir_auth": "1", "dirfrag": "0x10000000584.110*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "/tree", "pins": { "child": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "602", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "602" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 3 }, "snapid_first": 2, "states": [], "version": "4008" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "602", "committing_version": "602", "dir_auth": "1", "dirfrag": "0x10000000584.101*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "/tree", "pins": { "child": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "602", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "602" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "602", "committing_version": "602", "dir_auth": "1", "dirfrag": "0x10000000584.100*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "/tree", "pins": { "child": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "602", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "602" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-05-26T22:52:15.764 DEBUG:tasks.cephfs.cephfs_test_case:[ { "is_auth": true, "auth_first": 1, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 1, "dir": { "path": "/tree", "dirfrag": "0x10000000584.110*", "snapid_first": 2, "projected_version": "602", "version": "602", "committing_version": "602", "committed_version": "602", "is_rep": false, "dir_auth": "1", "states": [ "auth", "complete" ], "is_auth": true, "auth_state": { "replicas": { "0": 1 } }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "child": 1, "frozen": 0, "subtree": 1, "importing": 0, "replicated": 1 }, "nref": 3 } }, { "is_auth": true, "auth_first": 1, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "~mds1", "dirfrag": "0x101", "snapid_first": 2, "projected_version": "1", "version": "1", "committing_version": "1", "committed_version": "1", "is_rep": false, "dir_auth": "1", "states": [ "auth", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "dirty": 0, "authpin": 0 }, "nref": 2 } }, { "is_auth": false, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "", "dirfrag": "0x1", "snapid_first": 2, "projected_version": "0", "version": "4008", "committing_version": "0", "committed_version": "0", "is_rep": true, "dir_auth": "0", "states": [], "is_auth": false, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 3 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1 }, "nref": 2 } }, { "is_auth": true, "auth_first": 1, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 1, "dir": { "path": "/tree", "dirfrag": "0x10000000584.101*", "snapid_first": 2, "projected_version": "602", "version": "602", "committing_version": "602", "committed_version": "602", "is_rep": false, "dir_auth": "1", "states": [ "auth", "complete" ], "is_auth": true, "auth_state": { "replicas": { "0": 1 } }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "child": 1, "frozen": 0, "subtree": 1, "importing": 0, "replicated": 1 }, "nref": 3 } }, { "is_auth": true, "auth_first": 1, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 1, "dir": { "path": "/tree", "dirfrag": "0x10000000584.100*", "snapid_first": 2, "projected_version": "602", "version": "602", "committing_version": "602", "committed_version": "602", "is_rep": false, "dir_auth": "1", "states": [ "auth", "complete" ], "is_auth": true, "auth_state": { "replicas": { "0": 1 } }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "child": 1, "frozen": 0, "subtree": 1, "importing": 0, "replicated": 1 }, "nref": 3 } } ] 2024-05-26T22:52:15.764 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json get subtrees 2024-05-26T22:52:15.992 INFO:teuthology.orchestra.run.smithi187.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"4008","version":"4008","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":3,"2":2}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"3514","version":"3514","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":1,"authpin":0},"nref":3}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree","dirfrag":"0x10000000584.110*","snapid_first":2,"projected_version":"602","version":"602","committing_version":"602","committed_version":"602","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/tree","dirfrag":"0x10000000584.111*","snapid_first":2,"projected_version":"602","version":"602","committing_version":"602","committed_version":"602","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree","dirfrag":"0x10000000584.100*","snapid_first":2,"projected_version":"602","version":"602","committing_version":"602","committed_version":"602","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/tree","dirfrag":"0x10000000584.011*","snapid_first":2,"projected_version":"602","version":"602","committing_version":"602","committed_version":"602","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"frozen":0,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/tree","dirfrag":"0x10000000584.010*","snapid_first":2,"projected_version":"602","version":"602","committing_version":"602","committed_version":"602","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"frozen":0,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/tree","dirfrag":"0x10000000584.001*","snapid_first":2,"projected_version":"602","version":"602","committing_version":"602","committed_version":"602","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"frozen":0,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/tree","dirfrag":"0x10000000584.000*","snapid_first":2,"projected_version":"602","version":"602","committing_version":"602","committed_version":"602","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree","dirfrag":"0x10000000584.101*","snapid_first":2,"projected_version":"602","version":"602","committing_version":"602","committed_version":"602","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}}] 2024-05-26T22:52:15.993 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 3, "2": 2 } }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "4008", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "4008" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 1, "subtree": 1 }, "projected_version": "3514", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "3514" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "602", "committing_version": "602", "dir_auth": "1", "dirfrag": "0x10000000584.110*", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "602", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "602" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "602", "committing_version": "602", "dir_auth": "2", "dirfrag": "0x10000000584.111*", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "602", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "602" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 2, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "602", "committing_version": "602", "dir_auth": "1", "dirfrag": "0x10000000584.100*", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "602", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "602" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "602", "committing_version": "602", "dir_auth": "0", "dirfrag": "0x10000000584.011*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/tree", "pins": { "authpin": 0, "child": 1, "dirty": 0, "frozen": 0, "subtree": 1 }, "projected_version": "602", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "602" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "602", "committing_version": "602", "dir_auth": "0", "dirfrag": "0x10000000584.010*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/tree", "pins": { "authpin": 0, "child": 1, "dirty": 0, "frozen": 0, "subtree": 1 }, "projected_version": "602", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "602" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "602", "committing_version": "602", "dir_auth": "0", "dirfrag": "0x10000000584.001*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/tree", "pins": { "authpin": 0, "child": 1, "dirty": 0, "frozen": 0, "subtree": 1 }, "projected_version": "602", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "602" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "602", "committing_version": "602", "dir_auth": "2", "dirfrag": "0x10000000584.000*", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "602", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "602" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 2, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "602", "committing_version": "602", "dir_auth": "1", "dirfrag": "0x10000000584.101*", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "602", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "602" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false } ] 2024-05-26T22:52:15.994 DEBUG:tasks.cephfs.cephfs_test_case:[ { "is_auth": true, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "", "dirfrag": "0x1", "snapid_first": 2, "projected_version": "4008", "version": "4008", "committing_version": "1", "committed_version": "1", "is_rep": true, "dir_auth": "0", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": { "1": 3, "2": 2 } }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "replicated": 1, "dirty": 1, "authpin": 0 }, "nref": 4 } }, { "is_auth": true, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "~mds0", "dirfrag": "0x100", "snapid_first": 2, "projected_version": "3514", "version": "3514", "committing_version": "1", "committed_version": "1", "is_rep": false, "dir_auth": "0", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "dirty": 1, "authpin": 0 }, "nref": 3 } }, { "is_auth": false, "auth_first": 1, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 1, "dir": { "path": "/tree", "dirfrag": "0x10000000584.110*", "snapid_first": 2, "projected_version": "602", "version": "602", "committing_version": "602", "committed_version": "602", "is_rep": false, "dir_auth": "1", "states": [], "is_auth": false, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "request": 0, "child": 0, "frozen": 0, "subtree": 1, "replicated": 0, "dirty": 0, "waiter": 0, "authpin": 0, "tempexporting": 0 }, "nref": 1 } }, { "is_auth": false, "auth_first": 2, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 2, "dir": { "path": "/tree", "dirfrag": "0x10000000584.111*", "snapid_first": 2, "projected_version": "602", "version": "602", "committing_version": "602", "committed_version": "602", "is_rep": false, "dir_auth": "2", "states": [], "is_auth": false, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "request": 0, "child": 0, "frozen": 0, "subtree": 1, "replicated": 0, "dirty": 0, "waiter": 0, "authpin": 0, "tempexporting": 0 }, "nref": 1 } }, { "is_auth": false, "auth_first": 1, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 1, "dir": { "path": "/tree", "dirfrag": "0x10000000584.100*", "snapid_first": 2, "projected_version": "602", "version": "602", "committing_version": "602", "committed_version": "602", "is_rep": false, "dir_auth": "1", "states": [], "is_auth": false, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "request": 0, "child": 0, "frozen": 0, "subtree": 1, "replicated": 0, "dirty": 0, "waiter": 0, "authpin": 0, "tempexporting": 0 }, "nref": 1 } }, { "is_auth": true, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 0, "dir": { "path": "/tree", "dirfrag": "0x10000000584.011*", "snapid_first": 2, "projected_version": "602", "version": "602", "committing_version": "602", "committed_version": "602", "is_rep": false, "dir_auth": "0", "states": [ "auth", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "frozen": 0, "subtree": 1, "dirty": 0, "authpin": 0 }, "nref": 2 } }, { "is_auth": true, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 0, "dir": { "path": "/tree", "dirfrag": "0x10000000584.010*", "snapid_first": 2, "projected_version": "602", "version": "602", "committing_version": "602", "committed_version": "602", "is_rep": false, "dir_auth": "0", "states": [ "auth", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "frozen": 0, "subtree": 1, "dirty": 0, "authpin": 0 }, "nref": 2 } }, { "is_auth": true, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 0, "dir": { "path": "/tree", "dirfrag": "0x10000000584.001*", "snapid_first": 2, "projected_version": "602", "version": "602", "committing_version": "602", "committed_version": "602", "is_rep": false, "dir_auth": "0", "states": [ "auth", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "frozen": 0, "subtree": 1, "dirty": 0, "authpin": 0 }, "nref": 2 } }, { "is_auth": false, "auth_first": 2, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 2, "dir": { "path": "/tree", "dirfrag": "0x10000000584.000*", "snapid_first": 2, "projected_version": "602", "version": "602", "committing_version": "602", "committed_version": "602", "is_rep": false, "dir_auth": "2", "states": [], "is_auth": false, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "request": 0, "child": 0, "frozen": 0, "subtree": 1, "replicated": 0, "dirty": 0, "waiter": 0, "authpin": 0, "tempexporting": 0 }, "nref": 1 } }, { "is_auth": false, "auth_first": 1, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 1, "dir": { "path": "/tree", "dirfrag": "0x10000000584.101*", "snapid_first": 2, "projected_version": "602", "version": "602", "committing_version": "602", "committed_version": "602", "is_rep": false, "dir_auth": "1", "states": [], "is_auth": false, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "request": 0, "child": 0, "frozen": 0, "subtree": 1, "replicated": 0, "dirty": 0, "waiter": 0, "authpin": 0, "tempexporting": 0 }, "nref": 1 } } ] 2024-05-26T22:52:15.994 INFO:tasks.cephfs.cephfs_test_case:len=8 [ { "is_auth": true, "auth_first": 2, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 2, "dir": { "path": "/tree", "dirfrag": "0x10000000584.111*", "snapid_first": 2, "projected_version": "602", "version": "602", "committing_version": "602", "committed_version": "602", "is_rep": false, "dir_auth": "2", "states": [ "auth", "complete" ], "is_auth": true, "auth_state": { "replicas": { "0": 1 } }, "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "child": 1, "frozen": 0, "subtree": 1, "importing": 0, "replicated": 1 }, "nref": 3 } }, { "is_auth": true, "auth_first": 2, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 2, "dir": { "path": "/tree", "dirfrag": "0x10000000584.000*", "snapid_first": 2, "projected_version": "602", "version": "602", "committing_version": "602", "committed_version": "602", "is_rep": false, "dir_auth": "2", "states": [ "auth", "complete" ], "is_auth": true, "auth_state": { "replicas": { "0": 1 } }, "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "child": 1, "frozen": 0, "subtree": 1, "importing": 0, "replicated": 1 }, "nref": 3 } }, { "is_auth": true, "auth_first": 1, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 1, "dir": { "path": "/tree", "dirfrag": "0x10000000584.110*", "snapid_first": 2, "projected_version": "602", "version": "602", "committing_version": "602", "committed_version": "602", "is_rep": false, "dir_auth": "1", "states": [ "auth", "complete" ], "is_auth": true, "auth_state": { "replicas": { "0": 1 } }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "child": 1, "frozen": 0, "subtree": 1, "importing": 0, "replicated": 1 }, "nref": 3 } }, { "is_auth": true, "auth_first": 1, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 1, "dir": { "path": "/tree", "dirfrag": "0x10000000584.101*", "snapid_first": 2, "projected_version": "602", "version": "602", "committing_version": "602", "committed_version": "602", "is_rep": false, "dir_auth": "1", "states": [ "auth", "complete" ], "is_auth": true, "auth_state": { "replicas": { "0": 1 } }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "child": 1, "frozen": 0, "subtree": 1, "importing": 0, "replicated": 1 }, "nref": 3 } }, { "is_auth": true, "auth_first": 1, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 1, "dir": { "path": "/tree", "dirfrag": "0x10000000584.100*", "snapid_first": 2, "projected_version": "602", "version": "602", "committing_version": "602", "committed_version": "602", "is_rep": false, "dir_auth": "1", "states": [ "auth", "complete" ], "is_auth": true, "auth_state": { "replicas": { "0": 1 } }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "child": 1, "frozen": 0, "subtree": 1, "importing": 0, "replicated": 1 }, "nref": 3 } }, { "is_auth": true, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 0, "dir": { "path": "/tree", "dirfrag": "0x10000000584.011*", "snapid_first": 2, "projected_version": "602", "version": "602", "committing_version": "602", "committed_version": "602", "is_rep": false, "dir_auth": "0", "states": [ "auth", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "frozen": 0, "subtree": 1, "dirty": 0, "authpin": 0 }, "nref": 2 } }, { "is_auth": true, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 0, "dir": { "path": "/tree", "dirfrag": "0x10000000584.010*", "snapid_first": 2, "projected_version": "602", "version": "602", "committing_version": "602", "committed_version": "602", "is_rep": false, "dir_auth": "0", "states": [ "auth", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "frozen": 0, "subtree": 1, "dirty": 0, "authpin": 0 }, "nref": 2 } }, { "is_auth": true, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 0, "dir": { "path": "/tree", "dirfrag": "0x10000000584.001*", "snapid_first": 2, "projected_version": "602", "version": "602", "committing_version": "602", "committed_version": "602", "is_rep": false, "dir_auth": "0", "states": [ "auth", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "frozen": 0, "subtree": 1, "dirty": 0, "authpin": 0 }, "nref": 2 } } ] 2024-05-26T22:52:15.995 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:52:15.995 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin.distributed -v 0 tree) 2024-05-26T22:52:31.057 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json get subtrees 2024-05-26T22:52:31.287 INFO:teuthology.orchestra.run.smithi187.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"4014","version":"4014","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":3,"2":2}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"3514","version":"3514","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":1,"authpin":0},"nref":3}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"/tree","dirfrag":"0x10000000584.110*","snapid_first":2,"projected_version":"602","version":"602","committing_version":"602","committed_version":"602","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"/tree","dirfrag":"0x10000000584.111*","snapid_first":2,"projected_version":"602","version":"602","committing_version":"602","committed_version":"602","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"/tree","dirfrag":"0x10000000584.100*","snapid_first":2,"projected_version":"602","version":"602","committing_version":"602","committed_version":"602","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"/tree","dirfrag":"0x10000000584.000*","snapid_first":2,"projected_version":"602","version":"602","committing_version":"602","committed_version":"602","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"/tree","dirfrag":"0x10000000584.101*","snapid_first":2,"projected_version":"602","version":"602","committing_version":"602","committed_version":"602","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}}] 2024-05-26T22:52:31.288 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 3, "2": 2 } }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "4014", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "4014" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 1, "subtree": 1 }, "projected_version": "3514", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "3514" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "602", "committing_version": "602", "dir_auth": "1", "dirfrag": "0x10000000584.110*", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "602", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "602" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "602", "committing_version": "602", "dir_auth": "2", "dirfrag": "0x10000000584.111*", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "602", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "602" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "602", "committing_version": "602", "dir_auth": "1", "dirfrag": "0x10000000584.100*", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "602", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "602" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "602", "committing_version": "602", "dir_auth": "2", "dirfrag": "0x10000000584.000*", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "602", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "602" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "602", "committing_version": "602", "dir_auth": "1", "dirfrag": "0x10000000584.101*", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "602", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "602" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false } ] 2024-05-26T22:52:31.289 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-05-26T22:52:31.289 DEBUG:teuthology.orchestra.run.smithi131:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-26T22:52:31.400 DEBUG:teuthology.orchestra.run.smithi187:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-26T22:52:31.506 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-05-26T22:52:31.506 INFO:tasks.cephfs.mount:Terminating background process 2024-05-26T22:52:31.506 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-26T22:52:31.506 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-26T22:52:31.564 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-26T22:52:31.564 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-26T22:52:31.618 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-26T22:52:31.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:31.819+0000 7ff804c00640 1 Processor -- start 2024-05-26T22:52:31.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:31.819+0000 7ff804c00640 1 -- start start 2024-05-26T22:52:31.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:31.821+0000 7ff804c00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7ff800155840 0x7ff800071930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:31.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:31.821+0000 7ff804c00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff80015c3d0 0x7ff80006ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:31.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:31.821+0000 7ff804c00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7ff80006f490 0x7ff80015ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:31.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:31.821+0000 7ff804c00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7ff8000722a0 con 0x7ff800155840 2024-05-26T22:52:31.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:31.821+0000 7ff804c00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7ff800072580 con 0x7ff80006f490 2024-05-26T22:52:31.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:31.821+0000 7ff804c00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff800072860 con 0x7ff80015c3d0 2024-05-26T22:52:31.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:31.821+0000 7ff7fde00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7ff800155840 0x7ff800071930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:31.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:31.821+0000 7ff7fde00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7ff800155840 0x7ff800071930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33742/0 (socket says 172.21.15.131:33742) 2024-05-26T22:52:31.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:31.821+0000 7ff7fde00640 1 -- 172.21.15.131:0/3823616286 learned_addr learned my addr 172.21.15.131:0/3823616286 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:52:31.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:31.821+0000 7ff7fde00640 1 -- 172.21.15.131:0/3823616286 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff80015c3d0 msgr2=0x7ff80006ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:31.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:31.821+0000 7ff7fde00640 1 --2- 172.21.15.131:0/3823616286 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff80015c3d0 0x7ff80006ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:31.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:31.821+0000 7ff7fde00640 1 -- 172.21.15.131:0/3823616286 >> v2:172.21.15.187:3300/0 conn(0x7ff80006f490 msgr2=0x7ff80015ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:31.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:31.821+0000 7ff7fde00640 1 --2- 172.21.15.131:0/3823616286 >> v2:172.21.15.187:3300/0 conn(0x7ff80006f490 0x7ff80015ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:31.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:31.821+0000 7ff7fde00640 1 -- 172.21.15.131:0/3823616286 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff800072b40 con 0x7ff800155840 2024-05-26T22:52:31.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:31.822+0000 7ff7fde00640 1 --2- 172.21.15.131:0/3823616286 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff800155840 0x7ff800071930 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7ff80015ebc0 tx=0x7ff7f002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce9c2e3b24d0f279 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:31.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:31.823+0000 7ff7fca00640 1 -- 172.21.15.131:0/3823616286 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7f0037070 con 0x7ff800155840 2024-05-26T22:52:31.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:31.823+0000 7ff7fca00640 1 -- 172.21.15.131:0/3823616286 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7f0032440 con 0x7ff800155840 2024-05-26T22:52:31.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:31.823+0000 7ff7fca00640 1 -- 172.21.15.131:0/3823616286 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7f0032810 con 0x7ff800155840 2024-05-26T22:52:31.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:31.823+0000 7ff804c00640 1 -- 172.21.15.131:0/3823616286 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff800155840 msgr2=0x7ff800071930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:31.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:31.823+0000 7ff804c00640 1 --2- 172.21.15.131:0/3823616286 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff800155840 0x7ff800071930 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7ff80015ebc0 tx=0x7ff7f002a1e0 comp rx=0 tx=0).stop 2024-05-26T22:52:31.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:31.824+0000 7ff804c00640 1 -- 172.21.15.131:0/3823616286 shutdown_connections 2024-05-26T22:52:31.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:31.824+0000 7ff804c00640 1 --2- 172.21.15.131:0/3823616286 >> v2:172.21.15.187:3300/0 conn(0x7ff80006f490 0x7ff80015ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:31.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:31.824+0000 7ff804c00640 1 --2- 172.21.15.131:0/3823616286 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff80015c3d0 0x7ff80006ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:31.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:31.824+0000 7ff804c00640 1 --2- 172.21.15.131:0/3823616286 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff800155840 0x7ff800071930 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:31.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:31.824+0000 7ff804c00640 1 -- 172.21.15.131:0/3823616286 >> 172.21.15.131:0/3823616286 conn(0x7ff8000b1e90 msgr2=0x7ff8000b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:52:31.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:31.824+0000 7ff804c00640 1 -- 172.21.15.131:0/3823616286 shutdown_connections 2024-05-26T22:52:31.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:31.824+0000 7ff804c00640 1 -- 172.21.15.131:0/3823616286 wait complete. 2024-05-26T22:52:31.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:31.825+0000 7ff804c00640 1 Processor -- start 2024-05-26T22:52:31.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:31.826+0000 7ff804c00640 1 -- start start 2024-05-26T22:52:31.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:31.826+0000 7ff804c00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff80006f490 0x7ff8001f3be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:31.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:31.826+0000 7ff804c00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff800155840 0x7ff800153680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:31.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:31.826+0000 7ff804c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff80015c3d0 0x7ff800153bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:31.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:31.826+0000 7ff804c00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff800071ff0 con 0x7ff80015c3d0 2024-05-26T22:52:31.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:31.826+0000 7ff804c00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff800071e70 con 0x7ff800155840 2024-05-26T22:52:31.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:31.826+0000 7ff804c00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff80015f1c0 con 0x7ff80006f490 2024-05-26T22:52:31.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:31.827+0000 7ff7fe800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff80015c3d0 0x7ff800153bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:31.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:31.827+0000 7ff7fe800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff80015c3d0 0x7ff800153bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33748/0 (socket says 172.21.15.131:33748) 2024-05-26T22:52:31.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:31.827+0000 7ff7fe800640 1 -- 172.21.15.131:0/1381951180 learned_addr learned my addr 172.21.15.131:0/1381951180 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:52:31.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:31.827+0000 7ff7fde00640 1 --2- 172.21.15.131:0/1381951180 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff80006f490 0x7ff8001f3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:31.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:31.827+0000 7ff7fd400640 1 --2- 172.21.15.131:0/1381951180 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff800155840 0x7ff800153680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:31.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:31.827+0000 7ff7fe800640 1 -- 172.21.15.131:0/1381951180 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff80006f490 msgr2=0x7ff8001f3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:31.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:31.827+0000 7ff7fe800640 1 --2- 172.21.15.131:0/1381951180 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff80006f490 0x7ff8001f3be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:31.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:31.827+0000 7ff7fe800640 1 -- 172.21.15.131:0/1381951180 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff800155840 msgr2=0x7ff800153680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:31.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:31.827+0000 7ff7fe800640 1 --2- 172.21.15.131:0/1381951180 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff800155840 0x7ff800153680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:31.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:31.827+0000 7ff7fe800640 1 -- 172.21.15.131:0/1381951180 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8001f4120 con 0x7ff80015c3d0 2024-05-26T22:52:31.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:31.828+0000 7ff7fe800640 1 --2- 172.21.15.131:0/1381951180 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff80015c3d0 0x7ff800153bc0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7ff7f800b970 tx=0x7ff7f800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:31.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:31.828+0000 7ff7f6000640 1 -- 172.21.15.131:0/1381951180 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7f8019070 con 0x7ff80015c3d0 2024-05-26T22:52:31.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:31.828+0000 7ff7f6000640 1 -- 172.21.15.131:0/1381951180 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7f800cb60 con 0x7ff80015c3d0 2024-05-26T22:52:31.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:31.828+0000 7ff7f6000640 1 -- 172.21.15.131:0/1381951180 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7f8014440 con 0x7ff80015c3d0 2024-05-26T22:52:31.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:31.828+0000 7ff804c00640 1 -- 172.21.15.131:0/1381951180 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff800157fb0 con 0x7ff80015c3d0 2024-05-26T22:52:31.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:31.828+0000 7ff804c00640 1 -- 172.21.15.131:0/1381951180 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff800158570 con 0x7ff80015c3d0 2024-05-26T22:52:31.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:31.829+0000 7ff804c00640 1 -- 172.21.15.131:0/1381951180 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8000694e0 con 0x7ff80015c3d0 2024-05-26T22:52:31.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:31.830+0000 7ff7f6000640 1 -- 172.21.15.131:0/1381951180 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff7f801c070 con 0x7ff80015c3d0 2024-05-26T22:52:31.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:31.831+0000 7ff7f6000640 1 --2- 172.21.15.131:0/1381951180 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ff7e006be30 0x7ff7e006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:31.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:31.831+0000 7ff7f6000640 1 -- 172.21.15.131:0/1381951180 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7ff7f808ac60 con 0x7ff80015c3d0 2024-05-26T22:52:31.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:31.831+0000 7ff7fde00640 1 --2- 172.21.15.131:0/1381951180 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ff7e006be30 0x7ff7e006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:31.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:31.832+0000 7ff7fde00640 1 --2- 172.21.15.131:0/1381951180 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ff7e006be30 0x7ff7e006e2f0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7ff7f0034000 tx=0x7ff7f002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:31.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:31.836+0000 7ff7f6000640 1 -- 172.21.15.131:0/1381951180 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7ff7f8059570 con 0x7ff80015c3d0 2024-05-26T22:52:32.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.033+0000 7ff804c00640 1 -- 172.21.15.131:0/1381951180 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff8000aec20 con 0x7ff80015c3d0 2024-05-26T22:52:32.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.036+0000 7ff7f6000640 1 -- 172.21.15.131:0/1381951180 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) ==== 74+0+16689 (secure 0 0 0) 0x7ff7f8058e70 con 0x7ff80015c3d0 2024-05-26T22:52:32.036 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:52:32.036 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":58,"fsid":"46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f","created":"2024-05-26T22:47:57.288905+0000","modified":"2024-05-26T22:51:40.663750+0000","last_up_change":"2024-05-26T22:48:09.602207+0000","last_in_change":"2024-05-26T22:48:00.687341+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-26T22:48:10.463339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-05-26T22:51:36.873056+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-05-26T22:51:38.120229+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"814eaf62-a18d-494b-8e27-fd8fb88adea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6801","nonce":2350940051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6803","nonce":2350940051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6807","nonce":2350940051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6805","nonce":2350940051}]},"public_addr":"172.21.15.131:6801/2350940051","cluster_addr":"172.21.15.131:6803/2350940051","heartbeat_back_addr":"172.21.15.131:6807/2350940051","heartbeat_front_addr":"172.21.15.131:6805/2350940051","state":["exists","up"]},{"osd":1,"uuid":"42e66287-f6c6-4e26-af3c-6dfaa3536d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6825","nonce":1525528247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6827","nonce":1525528247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6831","nonce":1525528247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6829","nonce":1525528247}]},"public_addr":"172.21.15.131:6825/1525528247","cluster_addr":"172.21.15.131:6827/1525528247","heartbeat_back_addr":"172.21.15.131:6831/1525528247","heartbeat_front_addr":"172.21.15.131:6829/1525528247","state":["exists","up"]},{"osd":2,"uuid":"0da34673-04a2-4fd0-b57a-0ff2b679b043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6817","nonce":3627842210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6819","nonce":3627842210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6823","nonce":3627842210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6821","nonce":3627842210}]},"public_addr":"172.21.15.131:6817/3627842210","cluster_addr":"172.21.15.131:6819/3627842210","heartbeat_back_addr":"172.21.15.131:6823/3627842210","heartbeat_front_addr":"172.21.15.131:6821/3627842210","state":["exists","up"]},{"osd":3,"uuid":"cf117a27-d6b2-43d3-ae52-b47988bf8eed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6809","nonce":350790591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6811","nonce":350790591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6815","nonce":350790591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6813","nonce":350790591}]},"public_addr":"172.21.15.131:6809/350790591","cluster_addr":"172.21.15.131:6811/350790591","heartbeat_back_addr":"172.21.15.131:6815/350790591","heartbeat_front_addr":"172.21.15.131:6813/350790591","state":["exists","up"]},{"osd":4,"uuid":"a092ccfc-be89-4a84-88d5-0f781fa715eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6816","nonce":244560995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6819","nonce":244560995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6823","nonce":244560995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6821","nonce":244560995}]},"public_addr":"172.21.15.187:6816/244560995","cluster_addr":"172.21.15.187:6819/244560995","heartbeat_back_addr":"172.21.15.187:6823/244560995","heartbeat_front_addr":"172.21.15.187:6821/244560995","state":["exists","up"]},{"osd":5,"uuid":"7513ba69-dd00-4a35-b190-9e8cc3b2bdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6801","nonce":324619398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6803","nonce":324619398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6812","nonce":324619398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6807","nonce":324619398}]},"public_addr":"172.21.15.187:6801/324619398","cluster_addr":"172.21.15.187:6803/324619398","heartbeat_back_addr":"172.21.15.187:6812/324619398","heartbeat_front_addr":"172.21.15.187:6807/324619398","state":["exists","up"]},{"osd":6,"uuid":"50081810-c093-4d2f-a4d7-b7c695c5400d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6806","nonce":1391347786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6809","nonce":1391347786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6818","nonce":1391347786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6813","nonce":1391347786}]},"public_addr":"172.21.15.187:6806/1391347786","cluster_addr":"172.21.15.187:6809/1391347786","heartbeat_back_addr":"172.21.15.187:6818/1391347786","heartbeat_front_addr":"172.21.15.187:6813/1391347786","state":["exists","up"]},{"osd":7,"uuid":"775dae05-54f5-4ad4-bbce-92863c6ed2ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6825","nonce":1220065622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6827","nonce":1220065622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6831","nonce":1220065622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6829","nonce":1220065622}]},"public_addr":"172.21.15.187:6825/1220065622","cluster_addr":"172.21.15.187:6827/1220065622","heartbeat_back_addr":"172.21.15.187:6831/1220065622","heartbeat_front_addr":"172.21.15.187:6829/1220065622","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:05.983520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.299516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.067323+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.307748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.018742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.258775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.010912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.295355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-26T22:52:32.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.038+0000 7ff804c00640 1 -- 172.21.15.131:0/1381951180 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ff7e006be30 msgr2=0x7ff7e006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:32.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.038+0000 7ff804c00640 1 --2- 172.21.15.131:0/1381951180 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ff7e006be30 0x7ff7e006e2f0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7ff7f0034000 tx=0x7ff7f002e000 comp rx=0 tx=0).stop 2024-05-26T22:52:32.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.038+0000 7ff804c00640 1 -- 172.21.15.131:0/1381951180 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff80015c3d0 msgr2=0x7ff800153bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:32.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.038+0000 7ff804c00640 1 --2- 172.21.15.131:0/1381951180 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff80015c3d0 0x7ff800153bc0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7ff7f800b970 tx=0x7ff7f800be30 comp rx=0 tx=0).stop 2024-05-26T22:52:32.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.038+0000 7ff804c00640 1 -- 172.21.15.131:0/1381951180 shutdown_connections 2024-05-26T22:52:32.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.039+0000 7ff804c00640 1 --2- 172.21.15.131:0/1381951180 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ff7e006be30 0x7ff7e006e2f0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:32.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.039+0000 7ff804c00640 1 --2- 172.21.15.131:0/1381951180 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff80015c3d0 0x7ff800153bc0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:32.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.039+0000 7ff804c00640 1 --2- 172.21.15.131:0/1381951180 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff800155840 0x7ff800153680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:32.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.039+0000 7ff804c00640 1 --2- 172.21.15.131:0/1381951180 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff80006f490 0x7ff8001f3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:32.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.039+0000 7ff804c00640 1 -- 172.21.15.131:0/1381951180 >> 172.21.15.131:0/1381951180 conn(0x7ff8000b1e90 msgr2=0x7ff8000add60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:52:32.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.039+0000 7ff804c00640 1 -- 172.21.15.131:0/1381951180 shutdown_connections 2024-05-26T22:52:32.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.039+0000 7ff804c00640 1 -- 172.21.15.131:0/1381951180 wait complete. 2024-05-26T22:52:32.058 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-26T22:48:10.463339+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-26T22:51:36.873056+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '57', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-05-26T22:51:38.120229+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '58', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-26T22:52:32.058 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-26T22:52:32.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.256+0000 7fb6da000640 1 Processor -- start 2024-05-26T22:52:32.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.257+0000 7fb6da000640 1 -- start start 2024-05-26T22:52:32.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.258+0000 7fb6da000640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fb6d406e090 0x7fb6d40b4110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:32.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.258+0000 7fb6da000640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fb6d415f290 0x7fb6d40d0a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:32.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.258+0000 7fb6da000640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6d40714c0 con 0x7fb6d40b4650 2024-05-26T22:52:32.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.258+0000 7fb6da000640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fb6d406e470 con 0x7fb6d415f290 2024-05-26T22:52:32.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.258+0000 7fb6da000640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fb6d415f760 con 0x7fb6d406e090 2024-05-26T22:52:32.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.259+0000 7fb6d3e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fb6d415f290 0x7fb6d40d0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:32.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.259+0000 7fb6d3e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fb6d415f290 0x7fb6d40d0a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33760/0 (socket says 172.21.15.131:33760) 2024-05-26T22:52:32.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.259+0000 7fb6d3e00640 1 -- 172.21.15.131:0/1235673300 learned_addr learned my addr 172.21.15.131:0/1235673300 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:52:32.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.259+0000 7fb6d3400640 1 --2- 172.21.15.131:0/1235673300 >> v2:172.21.15.187:3300/0 conn(0x7fb6d406e090 0x7fb6d40b4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:32.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.259+0000 7fb6d3e00640 1 -- 172.21.15.131:0/1235673300 >> v2:172.21.15.187:3300/0 conn(0x7fb6d406e090 msgr2=0x7fb6d40b4110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:32.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.259+0000 7fb6d3e00640 1 --2- 172.21.15.131:0/1235673300 >> v2:172.21.15.187:3300/0 conn(0x7fb6d406e090 0x7fb6d40b4110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:32.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.259+0000 7fb6d3e00640 1 -- 172.21.15.131:0/1235673300 >> v1:172.21.15.187:6789/0 conn(0x7fb6d40b4650 legacy=0x7fb6d415eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:32.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.259+0000 7fb6d3e00640 1 -- 172.21.15.131:0/1235673300 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6d415fa40 con 0x7fb6d415f290 2024-05-26T22:52:32.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.261+0000 7fb6d3e00640 1 --2- 172.21.15.131:0/1235673300 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6d415f290 0x7fb6d40d0a80 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fb6d40ced20 tx=0x7fb6cc02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3416f76f4f424d76 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:32.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.261+0000 7fb6d2000640 1 -- 172.21.15.131:0/1235673300 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6cc036070 con 0x7fb6d415f290 2024-05-26T22:52:32.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.261+0000 7fb6d2000640 1 -- 172.21.15.131:0/1235673300 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6cc031600 con 0x7fb6d415f290 2024-05-26T22:52:32.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.261+0000 7fb6d2000640 1 -- 172.21.15.131:0/1235673300 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6cc0319d0 con 0x7fb6d415f290 2024-05-26T22:52:32.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.261+0000 7fb6da000640 1 -- 172.21.15.131:0/1235673300 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6d415f290 msgr2=0x7fb6d40d0a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:32.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.261+0000 7fb6da000640 1 --2- 172.21.15.131:0/1235673300 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6d415f290 0x7fb6d40d0a80 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fb6d40ced20 tx=0x7fb6cc02a660 comp rx=0 tx=0).stop 2024-05-26T22:52:32.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.262+0000 7fb6da000640 1 -- 172.21.15.131:0/1235673300 shutdown_connections 2024-05-26T22:52:32.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.262+0000 7fb6da000640 1 --2- 172.21.15.131:0/1235673300 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6d415f290 0x7fb6d40d0a80 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:32.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.262+0000 7fb6da000640 1 --2- 172.21.15.131:0/1235673300 >> v2:172.21.15.187:3300/0 conn(0x7fb6d406e090 0x7fb6d40b4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:32.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.262+0000 7fb6da000640 1 -- 172.21.15.131:0/1235673300 >> 172.21.15.131:0/1235673300 conn(0x7fb6d4067520 msgr2=0x7fb6d4067930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:52:32.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.262+0000 7fb6da000640 1 -- 172.21.15.131:0/1235673300 shutdown_connections 2024-05-26T22:52:32.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.262+0000 7fb6da000640 1 -- 172.21.15.131:0/1235673300 wait complete. 2024-05-26T22:52:32.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.263+0000 7fb6da000640 1 Processor -- start 2024-05-26T22:52:32.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.264+0000 7fb6da000640 1 -- start start 2024-05-26T22:52:32.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.264+0000 7fb6da000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb6d406e090 0x7fb6d41463e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:32.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.265+0000 7fb6da000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6d40b4650 0x7fb6d40b33d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:32.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.265+0000 7fb6da000640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb6d415f290 0x7fb6d40cd540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:32.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.265+0000 7fb6da000640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d40d0fc0 con 0x7fb6d40b4650 2024-05-26T22:52:32.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.265+0000 7fb6da000640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d40d0280 con 0x7fb6d415f290 2024-05-26T22:52:32.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.265+0000 7fb6da000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb6d40d1140 con 0x7fb6d406e090 2024-05-26T22:52:32.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.265+0000 7fb6d2a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6d40b4650 0x7fb6d40b33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:32.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.265+0000 7fb6d2a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6d40b4650 0x7fb6d40b33d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33764/0 (socket says 172.21.15.131:33764) 2024-05-26T22:52:32.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.265+0000 7fb6d2a00640 1 -- 172.21.15.131:0/2299736689 learned_addr learned my addr 172.21.15.131:0/2299736689 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:52:32.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.265+0000 7fb6d3400640 1 --2- 172.21.15.131:0/2299736689 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb6d406e090 0x7fb6d41463e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:32.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.265+0000 7fb6d3e00640 1 --2- 172.21.15.131:0/2299736689 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb6d415f290 0x7fb6d40cd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:32.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.265+0000 7fb6d2a00640 1 -- 172.21.15.131:0/2299736689 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb6d406e090 msgr2=0x7fb6d41463e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:32.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.265+0000 7fb6d2a00640 1 --2- 172.21.15.131:0/2299736689 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb6d406e090 0x7fb6d41463e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:32.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.265+0000 7fb6d2a00640 1 -- 172.21.15.131:0/2299736689 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb6d415f290 msgr2=0x7fb6d40cd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:32.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.265+0000 7fb6d2a00640 1 --2- 172.21.15.131:0/2299736689 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb6d415f290 0x7fb6d40cd540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:32.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.265+0000 7fb6d2a00640 1 -- 172.21.15.131:0/2299736689 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6d41e8170 con 0x7fb6d40b4650 2024-05-26T22:52:32.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.266+0000 7fb6d2a00640 1 --2- 172.21.15.131:0/2299736689 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6d40b4650 0x7fb6d40b33d0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fb6c000c460 tx=0x7fb6c000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:32.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.266+0000 7fb6cb400640 1 -- 172.21.15.131:0/2299736689 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c0016020 con 0x7fb6d40b4650 2024-05-26T22:52:32.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.266+0000 7fb6cb400640 1 -- 172.21.15.131:0/2299736689 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6c0012870 con 0x7fb6d40b4650 2024-05-26T22:52:32.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.266+0000 7fb6da000640 1 -- 172.21.15.131:0/2299736689 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6d41e8450 con 0x7fb6d40b4650 2024-05-26T22:52:32.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.266+0000 7fb6cb400640 1 -- 172.21.15.131:0/2299736689 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c0012c60 con 0x7fb6d40b4650 2024-05-26T22:52:32.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.266+0000 7fb6da000640 1 -- 172.21.15.131:0/2299736689 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6d41e8a10 con 0x7fb6d40b4650 2024-05-26T22:52:32.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.267+0000 7fb6da000640 1 -- 172.21.15.131:0/2299736689 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6d40695c0 con 0x7fb6d40b4650 2024-05-26T22:52:32.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.268+0000 7fb6cb400640 1 -- 172.21.15.131:0/2299736689 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb6c0019020 con 0x7fb6d40b4650 2024-05-26T22:52:32.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.269+0000 7fb6cb400640 1 --2- 172.21.15.131:0/2299736689 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb6b406be80 0x7fb6b406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:32.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.269+0000 7fb6cb400640 1 -- 172.21.15.131:0/2299736689 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fb6c008ac00 con 0x7fb6d40b4650 2024-05-26T22:52:32.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.269+0000 7fb6d3400640 1 --2- 172.21.15.131:0/2299736689 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb6b406be80 0x7fb6b406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:32.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.270+0000 7fb6d3400640 1 --2- 172.21.15.131:0/2299736689 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb6b406be80 0x7fb6b406e340 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fb6c4009a10 tx=0x7fb6c4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:32.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.274+0000 7fb6cb400640 1 -- 172.21.15.131:0/2299736689 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fb6c0020030 con 0x7fb6d40b4650 2024-05-26T22:52:32.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.476+0000 7fb6da000640 1 -- 172.21.15.131:0/2299736689 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb6d406e630 con 0x7fb6d40b4650 2024-05-26T22:52:32.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.477+0000 7fb6cb400640 1 -- 172.21.15.131:0/2299736689 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v58) ==== 93+0+10 (secure 0 0 0) 0x7fb6c0059500 con 0x7fb6d40b4650 2024-05-26T22:52:32.477 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2024-05-26T22:52:32.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.480+0000 7fb6da000640 1 -- 172.21.15.131:0/2299736689 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb6b406be80 msgr2=0x7fb6b406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:32.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.480+0000 7fb6da000640 1 --2- 172.21.15.131:0/2299736689 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb6b406be80 0x7fb6b406e340 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fb6c4009a10 tx=0x7fb6c4008040 comp rx=0 tx=0).stop 2024-05-26T22:52:32.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.480+0000 7fb6da000640 1 -- 172.21.15.131:0/2299736689 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6d40b4650 msgr2=0x7fb6d40b33d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:32.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.480+0000 7fb6da000640 1 --2- 172.21.15.131:0/2299736689 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6d40b4650 0x7fb6d40b33d0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fb6c000c460 tx=0x7fb6c000c920 comp rx=0 tx=0).stop 2024-05-26T22:52:32.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.481+0000 7fb6da000640 1 -- 172.21.15.131:0/2299736689 shutdown_connections 2024-05-26T22:52:32.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.481+0000 7fb6da000640 1 --2- 172.21.15.131:0/2299736689 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb6b406be80 0x7fb6b406e340 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:32.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.481+0000 7fb6da000640 1 --2- 172.21.15.131:0/2299736689 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb6d415f290 0x7fb6d40cd540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:32.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.481+0000 7fb6da000640 1 --2- 172.21.15.131:0/2299736689 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6d40b4650 0x7fb6d40b33d0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:32.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.481+0000 7fb6da000640 1 --2- 172.21.15.131:0/2299736689 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb6d406e090 0x7fb6d41463e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:32.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.481+0000 7fb6da000640 1 -- 172.21.15.131:0/2299736689 >> 172.21.15.131:0/2299736689 conn(0x7fb6d4067520 msgr2=0x7fb6d415fd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:52:32.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.481+0000 7fb6da000640 1 -- 172.21.15.131:0/2299736689 shutdown_connections 2024-05-26T22:52:32.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.481+0000 7fb6da000640 1 -- 172.21.15.131:0/2299736689 wait complete. 2024-05-26T22:52:32.499 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-26T22:52:32.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.712+0000 7fa4b1c00640 1 Processor -- start 2024-05-26T22:52:32.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.712+0000 7fa4b1c00640 1 -- start start 2024-05-26T22:52:32.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.714+0000 7fa4b1c00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fa4ac0b4370 0x7fa4ac15fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:32.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.714+0000 7fa4b1c00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fa4ac13f1b0 0x7fa4ac160110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:32.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.714+0000 7fa4b1c00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4ac160b00 con 0x7fa4ac0b4750 2024-05-26T22:52:32.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.714+0000 7fa4b1c00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fa4ac160820 con 0x7fa4ac0b4370 2024-05-26T22:52:32.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.714+0000 7fa4b1c00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fa4ac13f7f0 con 0x7fa4ac13f1b0 2024-05-26T22:52:32.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.714+0000 7fa4abe00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fa4ac0b4370 0x7fa4ac15fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:32.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.715+0000 7fa4abe00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fa4ac0b4370 0x7fa4ac15fbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33778/0 (socket says 172.21.15.131:33778) 2024-05-26T22:52:32.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.715+0000 7fa4abe00640 1 -- 172.21.15.131:0/967176809 learned_addr learned my addr 172.21.15.131:0/967176809 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:52:32.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.715+0000 7fa4ab400640 1 --1- 172.21.15.131:0/967176809 >> v1:172.21.15.131:6789/0 conn(0x7fa4ac0b4750 0x7fa4ac13eaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:48842/0 (socket says 172.21.15.131:48842) 2024-05-26T22:52:32.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.715+0000 7fa4b1200640 1 --2- 172.21.15.131:0/967176809 >> v2:172.21.15.187:3300/0 conn(0x7fa4ac13f1b0 0x7fa4ac160110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:32.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.715+0000 7fa4abe00640 1 -- 172.21.15.131:0/967176809 >> v2:172.21.15.187:3300/0 conn(0x7fa4ac13f1b0 msgr2=0x7fa4ac160110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:32.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.715+0000 7fa4abe00640 1 --2- 172.21.15.131:0/967176809 >> v2:172.21.15.187:3300/0 conn(0x7fa4ac13f1b0 0x7fa4ac160110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:32.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.715+0000 7fa4aaa00640 1 -- 172.21.15.131:0/967176809 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3848358739 0 0) 0x7fa4ac160b00 con 0x7fa4ac0b4750 2024-05-26T22:52:32.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.715+0000 7fa4abe00640 1 -- 172.21.15.131:0/967176809 >> v1:172.21.15.131:6789/0 conn(0x7fa4ac0b4750 legacy=0x7fa4ac13eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:32.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.715+0000 7fa4abe00640 1 -- 172.21.15.131:0/967176809 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4ac13fad0 con 0x7fa4ac0b4370 2024-05-26T22:52:32.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.715+0000 7fa4b1200640 1 --2- 172.21.15.131:0/967176809 >> v2:172.21.15.187:3300/0 conn(0x7fa4ac13f1b0 0x7fa4ac160110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:52:32.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.716+0000 7fa4abe00640 1 --2- 172.21.15.131:0/967176809 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4ac0b4370 0x7fa4ac15fbd0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac0d01f0 tx=0x7fa49c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e10a5acd5978df3a server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:32.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.716+0000 7fa4aaa00640 1 -- 172.21.15.131:0/967176809 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa49c037070 con 0x7fa4ac0b4370 2024-05-26T22:52:32.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.717+0000 7fa4aaa00640 1 -- 172.21.15.131:0/967176809 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa49c032440 con 0x7fa4ac0b4370 2024-05-26T22:52:32.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.717+0000 7fa4aaa00640 1 -- 172.21.15.131:0/967176809 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa49c032810 con 0x7fa4ac0b4370 2024-05-26T22:52:32.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.717+0000 7fa4b1c00640 1 -- 172.21.15.131:0/967176809 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4ac0b4370 msgr2=0x7fa4ac15fbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:32.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.717+0000 7fa4b1c00640 1 --2- 172.21.15.131:0/967176809 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4ac0b4370 0x7fa4ac15fbd0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac0d01f0 tx=0x7fa49c02a270 comp rx=0 tx=0).stop 2024-05-26T22:52:32.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.717+0000 7fa4b1c00640 1 -- 172.21.15.131:0/967176809 shutdown_connections 2024-05-26T22:52:32.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.717+0000 7fa4b1c00640 1 --2- 172.21.15.131:0/967176809 >> v2:172.21.15.187:3300/0 conn(0x7fa4ac13f1b0 0x7fa4ac160110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:32.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.717+0000 7fa4b1c00640 1 --2- 172.21.15.131:0/967176809 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4ac0b4370 0x7fa4ac15fbd0 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:32.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.717+0000 7fa4b1c00640 1 -- 172.21.15.131:0/967176809 >> 172.21.15.131:0/967176809 conn(0x7fa4ac0672e0 msgr2=0x7fa4ac0676f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:52:32.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.717+0000 7fa4b1c00640 1 -- 172.21.15.131:0/967176809 shutdown_connections 2024-05-26T22:52:32.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.717+0000 7fa4b1c00640 1 -- 172.21.15.131:0/967176809 wait complete. 2024-05-26T22:52:32.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.719+0000 7fa4b1c00640 1 Processor -- start 2024-05-26T22:52:32.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.719+0000 7fa4b1c00640 1 -- start start 2024-05-26T22:52:32.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.720+0000 7fa4b1c00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa4ac0b4370 0x7fa4ac15ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:32.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.720+0000 7fa4b1c00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa4ac0b4750 0x7fa4ac06cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:32.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.720+0000 7fa4b1c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4ac13f1b0 0x7fa4ac1f6f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:32.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.720+0000 7fa4b1c00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ac140450 con 0x7fa4ac13f1b0 2024-05-26T22:52:32.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.720+0000 7fa4b1c00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ac1402d0 con 0x7fa4ac0b4370 2024-05-26T22:52:32.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.720+0000 7fa4b1c00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa4ac1405d0 con 0x7fa4ac0b4750 2024-05-26T22:52:32.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.720+0000 7fa4b1200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4ac13f1b0 0x7fa4ac1f6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:32.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.720+0000 7fa4b1200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4ac13f1b0 0x7fa4ac1f6f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33788/0 (socket says 172.21.15.131:33788) 2024-05-26T22:52:32.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.720+0000 7fa4b1200640 1 -- 172.21.15.131:0/1729872391 learned_addr learned my addr 172.21.15.131:0/1729872391 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:52:32.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.721+0000 7fa4abe00640 1 --2- 172.21.15.131:0/1729872391 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa4ac0b4370 0x7fa4ac15ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:32.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.721+0000 7fa4b1200640 1 -- 172.21.15.131:0/1729872391 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa4ac0b4750 msgr2=0x7fa4ac06cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:32.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.721+0000 7fa4b1200640 1 --2- 172.21.15.131:0/1729872391 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa4ac0b4750 0x7fa4ac06cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:32.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.721+0000 7fa4b1200640 1 -- 172.21.15.131:0/1729872391 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa4ac0b4370 msgr2=0x7fa4ac15ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:32.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.721+0000 7fa4b1200640 1 --2- 172.21.15.131:0/1729872391 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa4ac0b4370 0x7fa4ac15ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:32.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.721+0000 7fa4b1200640 1 -- 172.21.15.131:0/1729872391 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4ac06d4e0 con 0x7fa4ac13f1b0 2024-05-26T22:52:32.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.721+0000 7fa4b1200640 1 --2- 172.21.15.131:0/1729872391 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4ac13f1b0 0x7fa4ac1f6f70 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fa4a400c7c0 tx=0x7fa4a400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:32.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.721+0000 7fa4a3e00640 1 -- 172.21.15.131:0/1729872391 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a4015020 con 0x7fa4ac13f1b0 2024-05-26T22:52:32.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.721+0000 7fa4a3e00640 1 -- 172.21.15.131:0/1729872391 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4a4004230 con 0x7fa4ac13f1b0 2024-05-26T22:52:32.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.721+0000 7fa4b1c00640 1 -- 172.21.15.131:0/1729872391 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4ac06d780 con 0x7fa4ac13f1b0 2024-05-26T22:52:32.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.721+0000 7fa4a3e00640 1 -- 172.21.15.131:0/1729872391 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a4007620 con 0x7fa4ac13f1b0 2024-05-26T22:52:32.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.722+0000 7fa4b1c00640 1 -- 172.21.15.131:0/1729872391 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4ac06dd40 con 0x7fa4ac13f1b0 2024-05-26T22:52:32.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.723+0000 7fa4b1c00640 1 -- 172.21.15.131:0/1729872391 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4ac0694e0 con 0x7fa4ac13f1b0 2024-05-26T22:52:32.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.724+0000 7fa4a3e00640 1 -- 172.21.15.131:0/1729872391 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa4a4018020 con 0x7fa4ac13f1b0 2024-05-26T22:52:32.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.725+0000 7fa4a3e00640 1 --2- 172.21.15.131:0/1729872391 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa48806be30 0x7fa48806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:32.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.725+0000 7fa4a3e00640 1 -- 172.21.15.131:0/1729872391 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fa4a408af80 con 0x7fa4ac13f1b0 2024-05-26T22:52:32.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.725+0000 7fa4abe00640 1 --2- 172.21.15.131:0/1729872391 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa48806be30 0x7fa48806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:32.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.726+0000 7fa4abe00640 1 --2- 172.21.15.131:0/1729872391 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa48806be30 0x7fa48806e2f0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fa49c034000 tx=0x7fa49c02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:32.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.729+0000 7fa4a3e00640 1 -- 172.21.15.131:0/1729872391 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fa4a401f020 con 0x7fa4ac13f1b0 2024-05-26T22:52:32.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.927+0000 7fa4b1c00640 1 -- 172.21.15.131:0/1729872391 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa4ac1f41e0 con 0x7fa4ac13f1b0 2024-05-26T22:52:32.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.928+0000 7fa4a3e00640 1 -- 172.21.15.131:0/1729872391 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v58) ==== 104+0+11 (secure 0 0 0) 0x7fa4a4059950 con 0x7fa4ac13f1b0 2024-05-26T22:52:32.927 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-05-26T22:52:32.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.930+0000 7fa4b1c00640 1 -- 172.21.15.131:0/1729872391 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa48806be30 msgr2=0x7fa48806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:32.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.930+0000 7fa4b1c00640 1 --2- 172.21.15.131:0/1729872391 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa48806be30 0x7fa48806e2f0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fa49c034000 tx=0x7fa49c02e000 comp rx=0 tx=0).stop 2024-05-26T22:52:32.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.930+0000 7fa4b1c00640 1 -- 172.21.15.131:0/1729872391 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4ac13f1b0 msgr2=0x7fa4ac1f6f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:32.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.930+0000 7fa4b1c00640 1 --2- 172.21.15.131:0/1729872391 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4ac13f1b0 0x7fa4ac1f6f70 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fa4a400c7c0 tx=0x7fa4a400cc80 comp rx=0 tx=0).stop 2024-05-26T22:52:32.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.931+0000 7fa4b1c00640 1 -- 172.21.15.131:0/1729872391 shutdown_connections 2024-05-26T22:52:32.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.931+0000 7fa4b1c00640 1 --2- 172.21.15.131:0/1729872391 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa4ac0b4750 0x7fa4ac06cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:32.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.931+0000 7fa4b1c00640 1 --2- 172.21.15.131:0/1729872391 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa48806be30 0x7fa48806e2f0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:32.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.931+0000 7fa4b1c00640 1 --2- 172.21.15.131:0/1729872391 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4ac13f1b0 0x7fa4ac1f6f70 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:32.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.931+0000 7fa4b1c00640 1 --2- 172.21.15.131:0/1729872391 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa4ac0b4370 0x7fa4ac15ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:32.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.931+0000 7fa4b1c00640 1 -- 172.21.15.131:0/1729872391 >> 172.21.15.131:0/1729872391 conn(0x7fa4ac0672e0 msgr2=0x7fa4ac15b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:52:32.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.931+0000 7fa4b1c00640 1 -- 172.21.15.131:0/1729872391 shutdown_connections 2024-05-26T22:52:32.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:32.931+0000 7fa4b1c00640 1 -- 172.21.15.131:0/1729872391 wait complete. 2024-05-26T22:52:32.950 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-26T22:52:33.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.156+0000 7f110d000640 1 Processor -- start 2024-05-26T22:52:33.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.156+0000 7f110d000640 1 -- start start 2024-05-26T22:52:33.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.158+0000 7f110d000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f110815bfa0 0x7f1108071500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:33.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.158+0000 7f110d000640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f110806d960 0x7f1108071a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:33.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.158+0000 7f110d000640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11081552d0 con 0x7f110815dc40 2024-05-26T22:52:33.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.158+0000 7f110d000640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f11080b3f30 con 0x7f110806d960 2024-05-26T22:52:33.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.158+0000 7f110d000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1108071f80 con 0x7f110815bfa0 2024-05-26T22:52:33.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.158+0000 7f1107e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f110806d960 0x7f1108071a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:33.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.158+0000 7f1107400640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f110815dc40 0x7f110806d250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:48854/0 (socket says 172.21.15.131:48854) 2024-05-26T22:52:33.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.158+0000 7f1107400640 1 -- 172.21.15.131:0/2370372064 learned_addr learned my addr 172.21.15.131:0/2370372064 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:52:33.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.159+0000 7f1107e00640 1 -- 172.21.15.131:0/2370372064 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f110815bfa0 msgr2=0x7f1108071500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:33.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.159+0000 7f1106a00640 1 --2- 172.21.15.131:0/2370372064 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f110815bfa0 0x7f1108071500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:33.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.159+0000 7f1107e00640 1 --2- 172.21.15.131:0/2370372064 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f110815bfa0 0x7f1108071500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:33.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.159+0000 7f1107e00640 1 -- 172.21.15.131:0/2370372064 >> v1:172.21.15.131:6789/0 conn(0x7f110815dc40 legacy=0x7f110806d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:33.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.159+0000 7f1107e00640 1 -- 172.21.15.131:0/2370372064 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1108072230 con 0x7f110806d960 2024-05-26T22:52:33.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.160+0000 7f1107e00640 1 --2- 172.21.15.131:0/2370372064 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f110806d960 0x7f1108071a40 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f1100009d90 tx=0x7f110002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78b41053886936b1 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:33.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.160+0000 7f1106000640 1 -- 172.21.15.131:0/2370372064 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1100007740 con 0x7f110806d960 2024-05-26T22:52:33.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.160+0000 7f1106000640 1 -- 172.21.15.131:0/2370372064 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11000078e0 con 0x7f110806d960 2024-05-26T22:52:33.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.160+0000 7f1106000640 1 -- 172.21.15.131:0/2370372064 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1100007cb0 con 0x7f110806d960 2024-05-26T22:52:33.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.161+0000 7f110d000640 1 -- 172.21.15.131:0/2370372064 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f110806d960 msgr2=0x7f1108071a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:33.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.161+0000 7f110d000640 1 --2- 172.21.15.131:0/2370372064 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f110806d960 0x7f1108071a40 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f1100009d90 tx=0x7f110002a660 comp rx=0 tx=0).stop 2024-05-26T22:52:33.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.161+0000 7f110d000640 1 -- 172.21.15.131:0/2370372064 shutdown_connections 2024-05-26T22:52:33.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.161+0000 7f110d000640 1 --2- 172.21.15.131:0/2370372064 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f110806d960 0x7f1108071a40 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:33.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.161+0000 7f110d000640 1 --2- 172.21.15.131:0/2370372064 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f110815bfa0 0x7f1108071500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:33.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.161+0000 7f110d000640 1 -- 172.21.15.131:0/2370372064 >> 172.21.15.131:0/2370372064 conn(0x7f11080b5660 msgr2=0x7f11080b5a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:52:33.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.161+0000 7f110d000640 1 -- 172.21.15.131:0/2370372064 shutdown_connections 2024-05-26T22:52:33.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.161+0000 7f110d000640 1 -- 172.21.15.131:0/2370372064 wait complete. 2024-05-26T22:52:33.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.163+0000 7f110d000640 1 Processor -- start 2024-05-26T22:52:33.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.163+0000 7f110d000640 1 -- start start 2024-05-26T22:52:33.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.164+0000 7f110d000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f110806d960 0x7f110814db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:33.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.164+0000 7f110d000640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f110815bfa0 0x7f1108147900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:33.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.164+0000 7f110d000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f110815dc40 0x7f1108147e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:33.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.164+0000 7f110d000640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f110815e020 con 0x7f110815dc40 2024-05-26T22:52:33.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.164+0000 7f110d000640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f11080b3bc0 con 0x7f110815bfa0 2024-05-26T22:52:33.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.164+0000 7f110d000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f110815e1a0 con 0x7f110806d960 2024-05-26T22:52:33.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.164+0000 7f1107e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f110815dc40 0x7f1108147e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:33.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.164+0000 7f1107e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f110815dc40 0x7f1108147e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33806/0 (socket says 172.21.15.131:33806) 2024-05-26T22:52:33.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.164+0000 7f1107e00640 1 -- 172.21.15.131:0/2103799867 learned_addr learned my addr 172.21.15.131:0/2103799867 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:52:33.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.164+0000 7f1106a00640 1 --2- 172.21.15.131:0/2103799867 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f110815bfa0 0x7f1108147900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:33.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.164+0000 7f1107400640 1 --2- 172.21.15.131:0/2103799867 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f110806d960 0x7f110814db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:33.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.164+0000 7f1107e00640 1 -- 172.21.15.131:0/2103799867 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f110806d960 msgr2=0x7f110814db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:33.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.165+0000 7f1107e00640 1 --2- 172.21.15.131:0/2103799867 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f110806d960 0x7f110814db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:33.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.165+0000 7f1107e00640 1 -- 172.21.15.131:0/2103799867 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f110815bfa0 msgr2=0x7f1108147900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:33.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.165+0000 7f1107e00640 1 --2- 172.21.15.131:0/2103799867 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f110815bfa0 0x7f1108147900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:33.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.165+0000 7f1107e00640 1 -- 172.21.15.131:0/2103799867 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11081484a0 con 0x7f110815dc40 2024-05-26T22:52:33.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.165+0000 7f1106a00640 1 --2- 172.21.15.131:0/2103799867 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f110815bfa0 0x7f1108147900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:52:33.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.165+0000 7f1107400640 1 --2- 172.21.15.131:0/2103799867 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f110806d960 0x7f110814db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:52:33.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.165+0000 7f1107e00640 1 --2- 172.21.15.131:0/2103799867 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f110815dc40 0x7f1108147e40 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f1100009eb0 tx=0x7f1100004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:33.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.165+0000 7f10ff400640 1 -- 172.21.15.131:0/2103799867 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1100040070 con 0x7f110815dc40 2024-05-26T22:52:33.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.165+0000 7f10ff400640 1 -- 172.21.15.131:0/2103799867 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1100031c90 con 0x7f110815dc40 2024-05-26T22:52:33.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.165+0000 7f110d000640 1 -- 172.21.15.131:0/2103799867 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f110814d340 con 0x7f110815dc40 2024-05-26T22:52:33.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.165+0000 7f10ff400640 1 -- 172.21.15.131:0/2103799867 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1100045680 con 0x7f110815dc40 2024-05-26T22:52:33.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.165+0000 7f110d000640 1 -- 172.21.15.131:0/2103799867 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1108148780 con 0x7f110815dc40 2024-05-26T22:52:33.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.166+0000 7f110d000640 1 -- 172.21.15.131:0/2103799867 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11080694e0 con 0x7f110815dc40 2024-05-26T22:52:33.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.168+0000 7f10ff400640 1 -- 172.21.15.131:0/2103799867 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1100039040 con 0x7f110815dc40 2024-05-26T22:52:33.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.169+0000 7f10ff400640 1 --2- 172.21.15.131:0/2103799867 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f10e806be30 0x7f10e806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:33.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.169+0000 7f10ff400640 1 -- 172.21.15.131:0/2103799867 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f11000ac880 con 0x7f110815dc40 2024-05-26T22:52:33.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.169+0000 7f1107400640 1 --2- 172.21.15.131:0/2103799867 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f10e806be30 0x7f10e806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:33.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.170+0000 7f1107400640 1 --2- 172.21.15.131:0/2103799867 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f10e806be30 0x7f10e806e2f0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f11081492c0 tx=0x7f10f8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:33.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.174+0000 7f10ff400640 1 -- 172.21.15.131:0/2103799867 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f1100043360 con 0x7f110815dc40 2024-05-26T22:52:33.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.371+0000 7f110d000640 1 -- 172.21.15.131:0/2103799867 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f11080b35d0 con 0x7f110815dc40 2024-05-26T22:52:33.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.372+0000 7f10ff400640 1 -- 172.21.15.131:0/2103799867 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v58) ==== 100+0+11 (secure 0 0 0) 0x7f110007b2e0 con 0x7f110815dc40 2024-05-26T22:52:33.372 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 64 2024-05-26T22:52:33.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.375+0000 7f110d000640 1 -- 172.21.15.131:0/2103799867 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f10e806be30 msgr2=0x7f10e806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:33.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.375+0000 7f110d000640 1 --2- 172.21.15.131:0/2103799867 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f10e806be30 0x7f10e806e2f0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f11081492c0 tx=0x7f10f8009040 comp rx=0 tx=0).stop 2024-05-26T22:52:33.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.375+0000 7f110d000640 1 -- 172.21.15.131:0/2103799867 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f110815dc40 msgr2=0x7f1108147e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:33.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.375+0000 7f110d000640 1 --2- 172.21.15.131:0/2103799867 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f110815dc40 0x7f1108147e40 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f1100009eb0 tx=0x7f1100004a40 comp rx=0 tx=0).stop 2024-05-26T22:52:33.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.375+0000 7f110d000640 1 -- 172.21.15.131:0/2103799867 shutdown_connections 2024-05-26T22:52:33.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.375+0000 7f110d000640 1 --2- 172.21.15.131:0/2103799867 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f10e806be30 0x7f10e806e2f0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:33.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.375+0000 7f110d000640 1 --2- 172.21.15.131:0/2103799867 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f110815dc40 0x7f1108147e40 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:33.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.375+0000 7f110d000640 1 --2- 172.21.15.131:0/2103799867 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f110815bfa0 0x7f1108147900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:33.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.375+0000 7f110d000640 1 --2- 172.21.15.131:0/2103799867 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f110806d960 0x7f110814db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:33.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.375+0000 7f110d000640 1 -- 172.21.15.131:0/2103799867 >> 172.21.15.131:0/2103799867 conn(0x7f11080b5660 msgr2=0x7f11080afc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:52:33.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.375+0000 7f110d000640 1 -- 172.21.15.131:0/2103799867 shutdown_connections 2024-05-26T22:52:33.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.375+0000 7f110d000640 1 -- 172.21.15.131:0/2103799867 wait complete. 2024-05-26T22:52:33.393 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-26T22:52:33.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.595+0000 7f39bfa00640 1 Processor -- start 2024-05-26T22:52:33.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.595+0000 7f39bfa00640 1 -- start start 2024-05-26T22:52:33.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.596+0000 7f39bfa00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f39b80b5450 0x7f39b80b43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:33.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.596+0000 7f39bfa00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f39b80d0460 0x7f39b806d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:33.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.596+0000 7f39bfa00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39b80b4900 con 0x7f39b806f420 2024-05-26T22:52:33.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.596+0000 7f39bfa00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f39b8160d90 con 0x7f39b80d0460 2024-05-26T22:52:33.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.596+0000 7f39bfa00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f39b8160180 con 0x7f39b80b5450 2024-05-26T22:52:33.597 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.597+0000 7f39afe00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f39b806f420 0x7f39b80cfd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:48868/0 (socket says 172.21.15.131:48868) 2024-05-26T22:52:33.597 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.597+0000 7f39afe00640 1 -- 172.21.15.131:0/2480236586 learned_addr learned my addr 172.21.15.131:0/2480236586 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:52:33.597 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.597+0000 7f39bd000640 1 --2- 172.21.15.131:0/2480236586 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f39b80b5450 0x7f39b80b43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:33.597 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.597+0000 7f39bda00640 1 --2- 172.21.15.131:0/2480236586 >> v2:172.21.15.187:3300/0 conn(0x7f39b80d0460 0x7f39b806d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:33.597 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.597+0000 7f39b7e00640 1 -- 172.21.15.131:0/2480236586 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1439027755 0 0) 0x7f39b80b4900 con 0x7f39b806f420 2024-05-26T22:52:33.597 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.597+0000 7f39b7e00640 1 -- 172.21.15.131:0/2480236586 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3998003520 con 0x7f39b806f420 2024-05-26T22:52:33.598 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.598+0000 7f39bd000640 1 -- 172.21.15.131:0/2480236586 >> v2:172.21.15.187:3300/0 conn(0x7f39b80d0460 msgr2=0x7f39b806d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:33.598 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.598+0000 7f39bd000640 1 --2- 172.21.15.131:0/2480236586 >> v2:172.21.15.187:3300/0 conn(0x7f39b80d0460 0x7f39b806d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:33.598 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.598+0000 7f39bd000640 1 -- 172.21.15.131:0/2480236586 >> v1:172.21.15.131:6789/0 conn(0x7f39b806f420 legacy=0x7f39b80cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:33.598 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.598+0000 7f39bd000640 1 -- 172.21.15.131:0/2480236586 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39b8160460 con 0x7f39b80b5450 2024-05-26T22:52:33.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.599+0000 7f39bd000640 1 --2- 172.21.15.131:0/2480236586 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f39b80b5450 0x7f39b80b43c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f39a00097a0 tx=0x7f39a002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb83b0889769f023 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:33.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.608+0000 7f39b7e00640 1 -- 172.21.15.131:0/2480236586 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39a0037070 con 0x7f39b80b5450 2024-05-26T22:52:33.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.608+0000 7f39b7e00640 1 -- 172.21.15.131:0/2480236586 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39a0032440 con 0x7f39b80b5450 2024-05-26T22:52:33.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.608+0000 7f39b7e00640 1 -- 172.21.15.131:0/2480236586 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39a0032810 con 0x7f39b80b5450 2024-05-26T22:52:33.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.608+0000 7f39bfa00640 1 -- 172.21.15.131:0/2480236586 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f39b80b5450 msgr2=0x7f39b80b43c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:33.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.608+0000 7f39bfa00640 1 --2- 172.21.15.131:0/2480236586 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f39b80b5450 0x7f39b80b43c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f39a00097a0 tx=0x7f39a002a180 comp rx=0 tx=0).stop 2024-05-26T22:52:33.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.609+0000 7f39bfa00640 1 -- 172.21.15.131:0/2480236586 shutdown_connections 2024-05-26T22:52:33.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.609+0000 7f39bfa00640 1 --2- 172.21.15.131:0/2480236586 >> v2:172.21.15.187:3300/0 conn(0x7f39b80d0460 0x7f39b806d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:33.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.609+0000 7f39bfa00640 1 --2- 172.21.15.131:0/2480236586 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f39b80b5450 0x7f39b80b43c0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:33.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.609+0000 7f39bfa00640 1 -- 172.21.15.131:0/2480236586 >> 172.21.15.131:0/2480236586 conn(0x7f39b806bc90 msgr2=0x7f39b80b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:52:33.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.609+0000 7f39bfa00640 1 -- 172.21.15.131:0/2480236586 shutdown_connections 2024-05-26T22:52:33.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.609+0000 7f39bfa00640 1 -- 172.21.15.131:0/2480236586 wait complete. 2024-05-26T22:52:33.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.611+0000 7f39bfa00640 1 Processor -- start 2024-05-26T22:52:33.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.611+0000 7f39bfa00640 1 -- start start 2024-05-26T22:52:33.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.612+0000 7f39bfa00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f39b806f420 0x7f39b809d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:33.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.613+0000 7f39bfa00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f39b80b5450 0x7f39b809b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:33.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.613+0000 7f39bfa00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f39b80d0460 0x7f39b81467a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:33.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.613+0000 7f39bfa00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f39b80b5830 con 0x7f39b806f420 2024-05-26T22:52:33.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.613+0000 7f39bfa00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f39b8160a20 con 0x7f39b80d0460 2024-05-26T22:52:33.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.613+0000 7f39bfa00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f39b815fde0 con 0x7f39b80b5450 2024-05-26T22:52:33.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.613+0000 7f39bd000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f39b806f420 0x7f39b809d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:33.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.613+0000 7f39bd000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f39b806f420 0x7f39b809d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33810/0 (socket says 172.21.15.131:33810) 2024-05-26T22:52:33.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.613+0000 7f39bd000640 1 -- 172.21.15.131:0/372005573 learned_addr learned my addr 172.21.15.131:0/372005573 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:52:33.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.613+0000 7f39afe00640 1 --2- 172.21.15.131:0/372005573 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f39b80b5450 0x7f39b809b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:33.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.613+0000 7f39bd000640 1 -- 172.21.15.131:0/372005573 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f39b80b5450 msgr2=0x7f39b809b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:33.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.613+0000 7f39bd000640 1 --2- 172.21.15.131:0/372005573 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f39b80b5450 0x7f39b809b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:33.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.613+0000 7f39bd000640 1 -- 172.21.15.131:0/372005573 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f39b80d0460 msgr2=0x7f39b81467a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:33.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.613+0000 7f39bd000640 1 --2- 172.21.15.131:0/372005573 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f39b80d0460 0x7f39b81467a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:33.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.613+0000 7f39bd000640 1 -- 172.21.15.131:0/372005573 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39b809bbf0 con 0x7f39b806f420 2024-05-26T22:52:33.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.613+0000 7f39afe00640 1 --2- 172.21.15.131:0/372005573 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f39b80b5450 0x7f39b809b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:52:33.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.614+0000 7f39bd000640 1 --2- 172.21.15.131:0/372005573 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f39b806f420 0x7f39b809d530 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f39a0035040 tx=0x7f39a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:33.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.614+0000 7f39b5600640 1 -- 172.21.15.131:0/372005573 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39a0041030 con 0x7f39b806f420 2024-05-26T22:52:33.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.614+0000 7f39b5600640 1 -- 172.21.15.131:0/372005573 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39a003d4e0 con 0x7f39b806f420 2024-05-26T22:52:33.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.614+0000 7f39b5600640 1 -- 172.21.15.131:0/372005573 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39a0037040 con 0x7f39b806f420 2024-05-26T22:52:33.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.614+0000 7f39bfa00640 1 -- 172.21.15.131:0/372005573 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39b809bed0 con 0x7f39b806f420 2024-05-26T22:52:33.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.614+0000 7f39bfa00640 1 -- 172.21.15.131:0/372005573 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39b809b870 con 0x7f39b806f420 2024-05-26T22:52:33.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.615+0000 7f39bfa00640 1 -- 172.21.15.131:0/372005573 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39b8074e90 con 0x7f39b806f420 2024-05-26T22:52:33.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.617+0000 7f39b5600640 1 -- 172.21.15.131:0/372005573 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f39a0044070 con 0x7f39b806f420 2024-05-26T22:52:33.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.618+0000 7f39b5600640 1 --2- 172.21.15.131:0/372005573 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f399406be30 0x7f399406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:33.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.618+0000 7f39b5600640 1 -- 172.21.15.131:0/372005573 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f39a00ab520 con 0x7f39b806f420 2024-05-26T22:52:33.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.618+0000 7f39afe00640 1 --2- 172.21.15.131:0/372005573 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f399406be30 0x7f399406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:33.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.619+0000 7f39afe00640 1 --2- 172.21.15.131:0/372005573 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f399406be30 0x7f399406e2f0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f39b81478b0 tx=0x7f39a8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:33.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.623+0000 7f39b5600640 1 -- 172.21.15.131:0/372005573 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f39a0044e40 con 0x7f39b806f420 2024-05-26T22:52:33.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.832+0000 7f39bfa00640 1 -- 172.21.15.131:0/372005573 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f39b80695c0 con 0x7f39b806f420 2024-05-26T22:52:33.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.833+0000 7f39b5600640 1 -- 172.21.15.131:0/372005573 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v58) ==== 80+0+0 (secure 0 0 0) 0x7f39a0079f60 con 0x7f39b806f420 2024-05-26T22:52:33.832 INFO:teuthology.orchestra.run.smithi131.stderr:listed 0 entries 2024-05-26T22:52:33.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.835+0000 7f39bfa00640 1 -- 172.21.15.131:0/372005573 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f399406be30 msgr2=0x7f399406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:33.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.835+0000 7f39bfa00640 1 --2- 172.21.15.131:0/372005573 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f399406be30 0x7f399406e2f0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f39b81478b0 tx=0x7f39a8009040 comp rx=0 tx=0).stop 2024-05-26T22:52:33.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.835+0000 7f39bfa00640 1 -- 172.21.15.131:0/372005573 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f39b806f420 msgr2=0x7f39b809d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:33.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.835+0000 7f39bfa00640 1 --2- 172.21.15.131:0/372005573 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f39b806f420 0x7f39b809d530 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f39a0035040 tx=0x7f39a00046c0 comp rx=0 tx=0).stop 2024-05-26T22:52:33.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.835+0000 7f39bfa00640 1 -- 172.21.15.131:0/372005573 shutdown_connections 2024-05-26T22:52:33.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.835+0000 7f39bfa00640 1 --2- 172.21.15.131:0/372005573 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f39b80b5450 0x7f39b809b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:33.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.835+0000 7f39bfa00640 1 --2- 172.21.15.131:0/372005573 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f399406be30 0x7f399406e2f0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:33.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.835+0000 7f39bfa00640 1 --2- 172.21.15.131:0/372005573 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f39b80d0460 0x7f39b81467a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:33.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.835+0000 7f39bfa00640 1 --2- 172.21.15.131:0/372005573 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f39b806f420 0x7f39b809d530 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:33.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.835+0000 7f39bfa00640 1 -- 172.21.15.131:0/372005573 >> 172.21.15.131:0/372005573 conn(0x7f39b806bc90 msgr2=0x7f39b806e6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:52:33.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.836+0000 7f39bfa00640 1 -- 172.21.15.131:0/372005573 shutdown_connections 2024-05-26T22:52:33.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:33.836+0000 7f39bfa00640 1 -- 172.21.15.131:0/372005573 wait complete. 2024-05-26T22:52:33.853 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-26T22:52:33.853 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:52:33.853 DEBUG:teuthology.orchestra.run.smithi131:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-26T22:52:34.005 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi131.front.sepia.ceph.com 2024-05-26T22:52:34.005 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:52:34.005 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-26T22:52:34.044 INFO:tasks.cephfs.mount:Terminating background process 2024-05-26T22:52:34.044 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-05-26T22:52:34.044 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-26T22:52:34.083 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-26T22:52:34.084 INFO:tasks.cephfs.mount:Terminating background process 2024-05-26T22:52:34.084 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-26T22:52:34.084 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-26T22:52:34.128 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-26T22:52:34.129 INFO:tasks.cephfs.mount:Terminating background process 2024-05-26T22:52:34.129 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-05-26T22:52:34.129 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-26T22:52:34.178 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-26T22:52:34.178 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-26T22:52:34.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.381+0000 7f7381600640 1 Processor -- start 2024-05-26T22:52:34.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.381+0000 7f7381600640 1 -- start start 2024-05-26T22:52:34.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.383+0000 7f7381600640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f737c144340 0x7f737c0cedf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:34.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.383+0000 7f7381600640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f737c071b70 0x7f737c06e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:34.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.383+0000 7f7381600640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f737c06e9f0 0x7f737c15ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:34.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.383+0000 7f7381600640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f737c06ee60 con 0x7f737c144340 2024-05-26T22:52:34.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.383+0000 7f7381600640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f737c072eb0 con 0x7f737c06e9f0 2024-05-26T22:52:34.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.383+0000 7f7381600640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f737c071fd0 con 0x7f737c071b70 2024-05-26T22:52:34.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.383+0000 7f737be00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f737c144340 0x7f737c0cedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:34.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.383+0000 7f737be00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f737c144340 0x7f737c0cedf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49666/0 (socket says 172.21.15.131:49666) 2024-05-26T22:52:34.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.383+0000 7f737be00640 1 -- 172.21.15.131:0/2295836508 learned_addr learned my addr 172.21.15.131:0/2295836508 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:52:34.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.384+0000 7f737be00640 1 -- 172.21.15.131:0/2295836508 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f737c071b70 msgr2=0x7f737c06e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:34.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.384+0000 7f737be00640 1 --2- 172.21.15.131:0/2295836508 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f737c071b70 0x7f737c06e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:34.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.384+0000 7f737be00640 1 -- 172.21.15.131:0/2295836508 >> v2:172.21.15.187:3300/0 conn(0x7f737c06e9f0 msgr2=0x7f737c15ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:34.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.384+0000 7f737be00640 1 --2- 172.21.15.131:0/2295836508 >> v2:172.21.15.187:3300/0 conn(0x7f737c06e9f0 0x7f737c15ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:34.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.384+0000 7f737be00640 1 -- 172.21.15.131:0/2295836508 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f737c0722a0 con 0x7f737c144340 2024-05-26T22:52:34.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.385+0000 7f737be00640 1 --2- 172.21.15.131:0/2295836508 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f737c144340 0x7f737c0cedf0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f737c0cfcf0 tx=0x7f736c02c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67f740709373e4e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:34.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.385+0000 7f737aa00640 1 -- 172.21.15.131:0/2295836508 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f736c038070 con 0x7f737c144340 2024-05-26T22:52:34.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.385+0000 7f737aa00640 1 -- 172.21.15.131:0/2295836508 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f736c031710 con 0x7f737c144340 2024-05-26T22:52:34.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.385+0000 7f737aa00640 1 -- 172.21.15.131:0/2295836508 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f736c031ae0 con 0x7f737c144340 2024-05-26T22:52:34.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.386+0000 7f7381600640 1 -- 172.21.15.131:0/2295836508 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f737c144340 msgr2=0x7f737c0cedf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:34.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.386+0000 7f7381600640 1 --2- 172.21.15.131:0/2295836508 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f737c144340 0x7f737c0cedf0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f737c0cfcf0 tx=0x7f736c02c3f0 comp rx=0 tx=0).stop 2024-05-26T22:52:34.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.386+0000 7f7381600640 1 -- 172.21.15.131:0/2295836508 shutdown_connections 2024-05-26T22:52:34.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.386+0000 7f7381600640 1 --2- 172.21.15.131:0/2295836508 >> v2:172.21.15.187:3300/0 conn(0x7f737c06e9f0 0x7f737c15ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:34.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.386+0000 7f7381600640 1 --2- 172.21.15.131:0/2295836508 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f737c071b70 0x7f737c06e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:34.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.386+0000 7f7381600640 1 --2- 172.21.15.131:0/2295836508 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f737c144340 0x7f737c0cedf0 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:34.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.386+0000 7f7381600640 1 -- 172.21.15.131:0/2295836508 >> 172.21.15.131:0/2295836508 conn(0x7f737c0cff40 msgr2=0x7f737c0d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:52:34.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.386+0000 7f7381600640 1 -- 172.21.15.131:0/2295836508 shutdown_connections 2024-05-26T22:52:34.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.386+0000 7f7381600640 1 -- 172.21.15.131:0/2295836508 wait complete. 2024-05-26T22:52:34.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.387+0000 7f7381600640 1 Processor -- start 2024-05-26T22:52:34.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.388+0000 7f7381600640 1 -- start start 2024-05-26T22:52:34.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.389+0000 7f7381600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f737c06e9f0 0x7f737c15b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:34.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.389+0000 7f7381600640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f737c071b70 0x7f737c13e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:34.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.389+0000 7f7381600640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f737c144340 0x7f737c0ce940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:34.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.389+0000 7f7381600640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f737c072b40 con 0x7f737c06e9f0 2024-05-26T22:52:34.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.389+0000 7f7381600640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f737c072860 con 0x7f737c071b70 2024-05-26T22:52:34.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.389+0000 7f7381600640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f737c15f170 con 0x7f737c144340 2024-05-26T22:52:34.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.389+0000 7f737be00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f737c06e9f0 0x7f737c15b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:34.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.389+0000 7f737be00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f737c06e9f0 0x7f737c15b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49682/0 (socket says 172.21.15.131:49682) 2024-05-26T22:52:34.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.389+0000 7f737be00640 1 -- 172.21.15.131:0/1623466436 learned_addr learned my addr 172.21.15.131:0/1623466436 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:52:34.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.389+0000 7f737b400640 1 --2- 172.21.15.131:0/1623466436 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f737c071b70 0x7f737c13e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:34.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.389+0000 7f7380c00640 1 --2- 172.21.15.131:0/1623466436 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f737c144340 0x7f737c0ce940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:34.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.389+0000 7f737be00640 1 -- 172.21.15.131:0/1623466436 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f737c144340 msgr2=0x7f737c0ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:34.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.389+0000 7f737be00640 1 --2- 172.21.15.131:0/1623466436 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f737c144340 0x7f737c0ce940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:34.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.389+0000 7f737be00640 1 -- 172.21.15.131:0/1623466436 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f737c071b70 msgr2=0x7f737c13e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:34.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.389+0000 7f737be00640 1 --2- 172.21.15.131:0/1623466436 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f737c071b70 0x7f737c13e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:34.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.389+0000 7f737be00640 1 -- 172.21.15.131:0/1623466436 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f737c15bea0 con 0x7f737c06e9f0 2024-05-26T22:52:34.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.389+0000 7f737b400640 1 --2- 172.21.15.131:0/1623466436 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f737c071b70 0x7f737c13e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:52:34.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.389+0000 7f7380c00640 1 --2- 172.21.15.131:0/1623466436 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f737c144340 0x7f737c0ce940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:52:34.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.390+0000 7f737be00640 1 --2- 172.21.15.131:0/1623466436 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f737c06e9f0 0x7f737c15b670 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f736c02c430 tx=0x7f736c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:34.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.390+0000 7f7373e00640 1 -- 172.21.15.131:0/1623466436 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f736c043070 con 0x7f737c06e9f0 2024-05-26T22:52:34.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.390+0000 7f7373e00640 1 -- 172.21.15.131:0/1623466436 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f736c033990 con 0x7f737c06e9f0 2024-05-26T22:52:34.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.390+0000 7f7373e00640 1 -- 172.21.15.131:0/1623466436 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f736c038040 con 0x7f737c06e9f0 2024-05-26T22:52:34.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.390+0000 7f7381600640 1 -- 172.21.15.131:0/1623466436 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f737c15c180 con 0x7f737c06e9f0 2024-05-26T22:52:34.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.390+0000 7f7381600640 1 -- 172.21.15.131:0/1623466436 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f737c15c740 con 0x7f737c06e9f0 2024-05-26T22:52:34.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.391+0000 7f7381600640 1 -- 172.21.15.131:0/1623466436 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f737c0694e0 con 0x7f737c06e9f0 2024-05-26T22:52:34.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.392+0000 7f7373e00640 1 -- 172.21.15.131:0/1623466436 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f736c046070 con 0x7f737c06e9f0 2024-05-26T22:52:34.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.393+0000 7f7373e00640 1 --2- 172.21.15.131:0/1623466436 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f735c06be30 0x7f735c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:34.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.393+0000 7f7373e00640 1 -- 172.21.15.131:0/1623466436 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f736c0ade30 con 0x7f737c06e9f0 2024-05-26T22:52:34.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.393+0000 7f737b400640 1 --2- 172.21.15.131:0/1623466436 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f735c06be30 0x7f735c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:34.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.394+0000 7f737b400640 1 --2- 172.21.15.131:0/1623466436 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f735c06be30 0x7f735c06e2f0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f7368006fd0 tx=0x7f7368008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:34.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.398+0000 7f7373e00640 1 -- 172.21.15.131:0/1623466436 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f736c07c900 con 0x7f737c06e9f0 2024-05-26T22:52:34.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.636+0000 7f7381600640 1 -- 172.21.15.131:0/1623466436 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f737c073670 con 0x7f737c06e9f0 2024-05-26T22:52:34.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.636+0000 7f7373e00640 1 -- 172.21.15.131:0/1623466436 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+5231 (secure 0 0 0) 0x7f736c07c200 con 0x7f737c06e9f0 2024-05-26T22:52:34.636 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:52:34.636 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":55,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4832,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/1327386245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":1327386245},{"type":"v1","addr":"172.21.15.187:6835","nonce":1327386245}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-26T22:51:39.114644+0000","modified":"2024-05-26T22:52:14.181234+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6390,"mds_1":6381,"mds_2":6378},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6378":{"gid":6378,"name":"a","rank":2,"incarnation":53,"state":"up:active","state_seq":8,"addr":"172.21.15.131:6836/2737657374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6835","nonce":2737657374},{"type":"v1","addr":"172.21.15.131:6836","nonce":2737657374}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6381":{"gid":6381,"name":"c","rank":1,"incarnation":51,"state":"up:active","state_seq":8,"addr":"172.21.15.131:6837/640544945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":640544945},{"type":"v1","addr":"172.21.15.131:6837","nonce":640544945}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6390":{"gid":6390,"name":"d","rank":0,"incarnation":45,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6833/3300872633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3300872633},{"type":"v1","addr":"172.21.15.187:6833","nonce":3300872633}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6390,"qdb_cluster":[6390,6381,6378]},"id":4}]} 2024-05-26T22:52:34.637 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 55 2024-05-26T22:52:34.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.639+0000 7f7381600640 1 -- 172.21.15.131:0/1623466436 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f735c06be30 msgr2=0x7f735c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:34.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.639+0000 7f7381600640 1 --2- 172.21.15.131:0/1623466436 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f735c06be30 0x7f735c06e2f0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f7368006fd0 tx=0x7f7368008040 comp rx=0 tx=0).stop 2024-05-26T22:52:34.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.639+0000 7f7381600640 1 -- 172.21.15.131:0/1623466436 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f737c06e9f0 msgr2=0x7f737c15b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:34.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.639+0000 7f7381600640 1 --2- 172.21.15.131:0/1623466436 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f737c06e9f0 0x7f737c15b670 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f736c02c430 tx=0x7f736c0046c0 comp rx=0 tx=0).stop 2024-05-26T22:52:34.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.639+0000 7f7381600640 1 -- 172.21.15.131:0/1623466436 shutdown_connections 2024-05-26T22:52:34.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.639+0000 7f7381600640 1 --2- 172.21.15.131:0/1623466436 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f737c144340 0x7f737c0ce940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:34.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.639+0000 7f7381600640 1 --2- 172.21.15.131:0/1623466436 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f735c06be30 0x7f735c06e2f0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:34.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.639+0000 7f7381600640 1 --2- 172.21.15.131:0/1623466436 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f737c071b70 0x7f737c13e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:34.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.639+0000 7f7381600640 1 --2- 172.21.15.131:0/1623466436 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f737c06e9f0 0x7f737c15b670 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:34.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.639+0000 7f7381600640 1 -- 172.21.15.131:0/1623466436 >> 172.21.15.131:0/1623466436 conn(0x7f737c0cff40 msgr2=0x7f737c0b45b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:52:34.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.639+0000 7f7381600640 1 -- 172.21.15.131:0/1623466436 shutdown_connections 2024-05-26T22:52:34.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.639+0000 7f7381600640 1 -- 172.21.15.131:0/1623466436 wait complete. 2024-05-26T22:52:34.658 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-26T22:52:34.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.862+0000 7fe6aa000640 1 Processor -- start 2024-05-26T22:52:34.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.862+0000 7fe6aa000640 1 -- start start 2024-05-26T22:52:34.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.864+0000 7fe6aa000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe6a4158c00 0x7fe6a406d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:34.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.864+0000 7fe6aa000640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6a40b3e50 con 0x7fe6a40b38a0 2024-05-26T22:52:34.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.864+0000 7fe6aa000640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6a40b4130 con 0x7fe6a406daf0 2024-05-26T22:52:34.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.864+0000 7fe6aa000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe6a406df50 con 0x7fe6a4158c00 2024-05-26T22:52:34.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.864+0000 7fe6a3e00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fe6a40b38a0 0x7fe6a415fc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:52034/0 (socket says 172.21.15.131:52034) 2024-05-26T22:52:34.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.864+0000 7fe6a3e00640 1 -- 172.21.15.131:0/3682591603 learned_addr learned my addr 172.21.15.131:0/3682591603 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:52:34.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.864+0000 7fe6a3400640 1 --2- 172.21.15.131:0/3682591603 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe6a4158c00 0x7fe6a406d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:34.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.865+0000 7fe6a2000640 1 -- 172.21.15.131:0/3682591603 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 57826886 0 0) 0x7fe6a40b3e50 con 0x7fe6a40b38a0 2024-05-26T22:52:34.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.865+0000 7fe6a2000640 1 -- 172.21.15.131:0/3682591603 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe688003520 con 0x7fe6a40b38a0 2024-05-26T22:52:34.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.865+0000 7fe6a2000640 1 -- 172.21.15.131:0/3682591603 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1961672398 0 0) 0x7fe688003520 con 0x7fe6a40b38a0 2024-05-26T22:52:34.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.865+0000 7fe6a2000640 1 -- 172.21.15.131:0/3682591603 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe6a4158c00 msgr2=0x7fe6a406d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:34.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.865+0000 7fe6a2000640 1 --2- 172.21.15.131:0/3682591603 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe6a4158c00 0x7fe6a406d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:34.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.865+0000 7fe6a2000640 1 -- 172.21.15.131:0/3682591603 >> v1:172.21.15.187:6789/0 conn(0x7fe6a406daf0 legacy=0x7fe6a40b3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:34.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.865+0000 7fe6a2000640 1 -- 172.21.15.131:0/3682591603 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6a40b4410 con 0x7fe6a40b38a0 2024-05-26T22:52:34.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.865+0000 7fe6a2000640 1 -- 172.21.15.131:0/3682591603 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7fe69c0019f0 con 0x7fe6a40b38a0 2024-05-26T22:52:34.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.865+0000 7fe6a2000640 1 -- 172.21.15.131:0/3682591603 >> v1:172.21.15.131:6789/0 conn(0x7fe6a40b38a0 legacy=0x7fe6a415fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:34.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.865+0000 7fe6a2000640 1 --2- 172.21.15.131:0/3682591603 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe6880039e0 0x7fe688003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:34.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.865+0000 7fe6a2000640 1 --2- 172.21.15.131:0/3682591603 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe688004800 0x7fe688004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:34.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.865+0000 7fe6a2000640 1 --2- 172.21.15.131:0/3682591603 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe6a4158c00 0x7fe68800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:34.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.865+0000 7fe6a2000640 1 -- 172.21.15.131:0/3682591603 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe6880054f0 con 0x7fe688004800 2024-05-26T22:52:34.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.865+0000 7fe6a2000640 1 -- 172.21.15.131:0/3682591603 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe68800bf00 con 0x7fe6880039e0 2024-05-26T22:52:34.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.865+0000 7fe6a2000640 1 -- 172.21.15.131:0/3682591603 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe68800c080 con 0x7fe6a4158c00 2024-05-26T22:52:34.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.865+0000 7fe6a2a00640 1 --2- 172.21.15.131:0/3682591603 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe688004800 0x7fe688004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:34.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.866+0000 7fe6a3400640 1 --2- 172.21.15.131:0/3682591603 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe6880039e0 0x7fe688003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:34.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.866+0000 7fe6a3e00640 1 --2- 172.21.15.131:0/3682591603 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe6a4158c00 0x7fe68800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:34.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.866+0000 7fe6a2a00640 1 -- 172.21.15.131:0/3682591603 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe6a4158c00 msgr2=0x7fe68800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:34.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.866+0000 7fe6a2a00640 1 --2- 172.21.15.131:0/3682591603 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe6a4158c00 0x7fe68800b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:34.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.866+0000 7fe6a2a00640 1 -- 172.21.15.131:0/3682591603 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe6880039e0 msgr2=0x7fe688003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:34.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.866+0000 7fe6a2a00640 1 --2- 172.21.15.131:0/3682591603 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe6880039e0 0x7fe688003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:34.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.866+0000 7fe6a2a00640 1 -- 172.21.15.131:0/3682591603 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe688003520 con 0x7fe688004800 2024-05-26T22:52:34.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.866+0000 7fe6a3400640 1 --2- 172.21.15.131:0/3682591603 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe6880039e0 0x7fe688003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:52:34.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.867+0000 7fe6a2a00640 1 --2- 172.21.15.131:0/3682591603 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe688004800 0x7fe688004c80 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fe690001750 tx=0x7fe688005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8752370faff32f54 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:34.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.867+0000 7fe6a2000640 1 -- 172.21.15.131:0/3682591603 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe690001370 con 0x7fe688004800 2024-05-26T22:52:34.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.867+0000 7fe6a2000640 1 -- 172.21.15.131:0/3682591603 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6900052d0 con 0x7fe688004800 2024-05-26T22:52:34.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.867+0000 7fe6a2000640 1 -- 172.21.15.131:0/3682591603 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe69002abe0 con 0x7fe688004800 2024-05-26T22:52:34.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.868+0000 7fe6aa000640 1 -- 172.21.15.131:0/3682591603 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe688004800 msgr2=0x7fe688004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:34.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.868+0000 7fe6aa000640 1 --2- 172.21.15.131:0/3682591603 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe688004800 0x7fe688004c80 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fe690001750 tx=0x7fe688005830 comp rx=0 tx=0).stop 2024-05-26T22:52:34.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.868+0000 7fe6a3e00640 1 -- 172.21.15.131:0/3682591603 reap_dead start 2024-05-26T22:52:34.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.868+0000 7fe6aa000640 1 -- 172.21.15.131:0/3682591603 shutdown_connections 2024-05-26T22:52:34.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.868+0000 7fe6aa000640 1 -- 172.21.15.131:0/3682591603 >> 172.21.15.131:0/3682591603 conn(0x7fe6a40b1e90 msgr2=0x7fe6a40b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:52:34.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.868+0000 7fe6aa000640 1 -- 172.21.15.131:0/3682591603 shutdown_connections 2024-05-26T22:52:34.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.868+0000 7fe6aa000640 1 -- 172.21.15.131:0/3682591603 wait complete. 2024-05-26T22:52:34.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.870+0000 7fe6aa000640 1 Processor -- start 2024-05-26T22:52:34.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.870+0000 7fe6aa000640 1 -- start start 2024-05-26T22:52:34.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.871+0000 7fe6aa000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe688004800 0x7fe6a4156bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:34.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.871+0000 7fe6aa000640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe6a40ae890 0x7fe6a40aed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:34.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.871+0000 7fe6aa000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe6a415b140 0x7fe6a415b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:34.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.871+0000 7fe6aa000640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a406d5b0 con 0x7fe688004800 2024-05-26T22:52:34.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.871+0000 7fe6aa000640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a40b49d0 con 0x7fe6a40ae890 2024-05-26T22:52:34.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.871+0000 7fe6aa000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe6a406d730 con 0x7fe6a415b140 2024-05-26T22:52:34.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.871+0000 7fe6a3400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe688004800 0x7fe6a4156bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:34.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.871+0000 7fe6a3400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe688004800 0x7fe6a4156bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49698/0 (socket says 172.21.15.131:49698) 2024-05-26T22:52:34.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.871+0000 7fe6a3400640 1 -- 172.21.15.131:0/780522392 learned_addr learned my addr 172.21.15.131:0/780522392 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:52:34.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.871+0000 7fe6a3400640 1 -- 172.21.15.131:0/780522392 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe6a415b140 msgr2=0x7fe6a415b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:34.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.871+0000 7fe6a3400640 1 --2- 172.21.15.131:0/780522392 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe6a415b140 0x7fe6a415b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:34.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.871+0000 7fe6a2a00640 1 --2- 172.21.15.131:0/780522392 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe6a40ae890 0x7fe6a40aed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:34.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.871+0000 7fe6a3400640 1 -- 172.21.15.131:0/780522392 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe6a40ae890 msgr2=0x7fe6a40aed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:34.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.871+0000 7fe6a3400640 1 --2- 172.21.15.131:0/780522392 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe6a40ae890 0x7fe6a40aed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:34.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.871+0000 7fe6a3400640 1 -- 172.21.15.131:0/780522392 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6a409a420 con 0x7fe688004800 2024-05-26T22:52:34.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.872+0000 7fe6a3400640 1 --2- 172.21.15.131:0/780522392 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe688004800 0x7fe6a4156bb0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fe69400e810 tx=0x7fe69400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:34.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.872+0000 7fe69b400640 1 -- 172.21.15.131:0/780522392 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe694017070 con 0x7fe688004800 2024-05-26T22:52:34.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.872+0000 7fe69b400640 1 -- 172.21.15.131:0/780522392 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe69400c6d0 con 0x7fe688004800 2024-05-26T22:52:34.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.872+0000 7fe69b400640 1 -- 172.21.15.131:0/780522392 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe69400f040 con 0x7fe688004800 2024-05-26T22:52:34.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.872+0000 7fe6aa000640 1 -- 172.21.15.131:0/780522392 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6a415beb0 con 0x7fe688004800 2024-05-26T22:52:34.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.872+0000 7fe6aa000640 1 -- 172.21.15.131:0/780522392 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6a409a9a0 con 0x7fe688004800 2024-05-26T22:52:34.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.873+0000 7fe6aa000640 1 -- 172.21.15.131:0/780522392 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6a40694e0 con 0x7fe688004800 2024-05-26T22:52:34.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.874+0000 7fe69b400640 1 -- 172.21.15.131:0/780522392 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe69401a070 con 0x7fe688004800 2024-05-26T22:52:34.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.875+0000 7fe69b400640 1 --2- 172.21.15.131:0/780522392 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fe68406be60 0x7fe68406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:34.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.875+0000 7fe69b400640 1 -- 172.21.15.131:0/780522392 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fe69408aaa0 con 0x7fe688004800 2024-05-26T22:52:34.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.876+0000 7fe6a2a00640 1 --2- 172.21.15.131:0/780522392 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fe68406be60 0x7fe68406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:34.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.876+0000 7fe6a2a00640 1 --2- 172.21.15.131:0/780522392 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fe68406be60 0x7fe68406e320 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fe690032000 tx=0x7fe69002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:34.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:34.879+0000 7fe69b400640 1 -- 172.21.15.131:0/780522392 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fe6940593b0 con 0x7fe688004800 2024-05-26T22:52:35.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.078+0000 7fe6aa000640 1 -- 172.21.15.131:0/780522392 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe6a4162b10 con 0x7fe688004800 2024-05-26T22:52:35.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.082+0000 7fe69b400640 1 -- 172.21.15.131:0/780522392 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) ==== 74+0+16689 (secure 0 0 0) 0x7fe694058cb0 con 0x7fe688004800 2024-05-26T22:52:35.082 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:52:35.082 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":58,"fsid":"46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f","created":"2024-05-26T22:47:57.288905+0000","modified":"2024-05-26T22:51:40.663750+0000","last_up_change":"2024-05-26T22:48:09.602207+0000","last_in_change":"2024-05-26T22:48:00.687341+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-26T22:48:10.463339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-05-26T22:51:36.873056+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-05-26T22:51:38.120229+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"814eaf62-a18d-494b-8e27-fd8fb88adea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6801","nonce":2350940051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6803","nonce":2350940051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6807","nonce":2350940051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6805","nonce":2350940051}]},"public_addr":"172.21.15.131:6801/2350940051","cluster_addr":"172.21.15.131:6803/2350940051","heartbeat_back_addr":"172.21.15.131:6807/2350940051","heartbeat_front_addr":"172.21.15.131:6805/2350940051","state":["exists","up"]},{"osd":1,"uuid":"42e66287-f6c6-4e26-af3c-6dfaa3536d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6825","nonce":1525528247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6827","nonce":1525528247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6831","nonce":1525528247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6829","nonce":1525528247}]},"public_addr":"172.21.15.131:6825/1525528247","cluster_addr":"172.21.15.131:6827/1525528247","heartbeat_back_addr":"172.21.15.131:6831/1525528247","heartbeat_front_addr":"172.21.15.131:6829/1525528247","state":["exists","up"]},{"osd":2,"uuid":"0da34673-04a2-4fd0-b57a-0ff2b679b043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6817","nonce":3627842210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6819","nonce":3627842210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6823","nonce":3627842210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6821","nonce":3627842210}]},"public_addr":"172.21.15.131:6817/3627842210","cluster_addr":"172.21.15.131:6819/3627842210","heartbeat_back_addr":"172.21.15.131:6823/3627842210","heartbeat_front_addr":"172.21.15.131:6821/3627842210","state":["exists","up"]},{"osd":3,"uuid":"cf117a27-d6b2-43d3-ae52-b47988bf8eed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6809","nonce":350790591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6811","nonce":350790591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6815","nonce":350790591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6813","nonce":350790591}]},"public_addr":"172.21.15.131:6809/350790591","cluster_addr":"172.21.15.131:6811/350790591","heartbeat_back_addr":"172.21.15.131:6815/350790591","heartbeat_front_addr":"172.21.15.131:6813/350790591","state":["exists","up"]},{"osd":4,"uuid":"a092ccfc-be89-4a84-88d5-0f781fa715eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6816","nonce":244560995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6819","nonce":244560995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6823","nonce":244560995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6821","nonce":244560995}]},"public_addr":"172.21.15.187:6816/244560995","cluster_addr":"172.21.15.187:6819/244560995","heartbeat_back_addr":"172.21.15.187:6823/244560995","heartbeat_front_addr":"172.21.15.187:6821/244560995","state":["exists","up"]},{"osd":5,"uuid":"7513ba69-dd00-4a35-b190-9e8cc3b2bdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6801","nonce":324619398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6803","nonce":324619398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6812","nonce":324619398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6807","nonce":324619398}]},"public_addr":"172.21.15.187:6801/324619398","cluster_addr":"172.21.15.187:6803/324619398","heartbeat_back_addr":"172.21.15.187:6812/324619398","heartbeat_front_addr":"172.21.15.187:6807/324619398","state":["exists","up"]},{"osd":6,"uuid":"50081810-c093-4d2f-a4d7-b7c695c5400d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6806","nonce":1391347786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6809","nonce":1391347786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6818","nonce":1391347786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6813","nonce":1391347786}]},"public_addr":"172.21.15.187:6806/1391347786","cluster_addr":"172.21.15.187:6809/1391347786","heartbeat_back_addr":"172.21.15.187:6818/1391347786","heartbeat_front_addr":"172.21.15.187:6813/1391347786","state":["exists","up"]},{"osd":7,"uuid":"775dae05-54f5-4ad4-bbce-92863c6ed2ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6825","nonce":1220065622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6827","nonce":1220065622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6831","nonce":1220065622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6829","nonce":1220065622}]},"public_addr":"172.21.15.187:6825/1220065622","cluster_addr":"172.21.15.187:6827/1220065622","heartbeat_back_addr":"172.21.15.187:6831/1220065622","heartbeat_front_addr":"172.21.15.187:6829/1220065622","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:05.983520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.299516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.067323+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.307748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.018742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.258775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.010912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.295355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-26T22:52:35.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.084+0000 7fe6aa000640 1 -- 172.21.15.131:0/780522392 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fe68406be60 msgr2=0x7fe68406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:35.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.084+0000 7fe6aa000640 1 --2- 172.21.15.131:0/780522392 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fe68406be60 0x7fe68406e320 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fe690032000 tx=0x7fe69002e000 comp rx=0 tx=0).stop 2024-05-26T22:52:35.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.084+0000 7fe6aa000640 1 -- 172.21.15.131:0/780522392 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe688004800 msgr2=0x7fe6a4156bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:35.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.084+0000 7fe6aa000640 1 --2- 172.21.15.131:0/780522392 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe688004800 0x7fe6a4156bb0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fe69400e810 tx=0x7fe69400ecd0 comp rx=0 tx=0).stop 2024-05-26T22:52:35.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.085+0000 7fe6aa000640 1 -- 172.21.15.131:0/780522392 shutdown_connections 2024-05-26T22:52:35.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.085+0000 7fe6aa000640 1 --2- 172.21.15.131:0/780522392 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe6a415b140 0x7fe6a415b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:35.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.085+0000 7fe6aa000640 1 --2- 172.21.15.131:0/780522392 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fe68406be60 0x7fe68406e320 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:35.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.085+0000 7fe6aa000640 1 --2- 172.21.15.131:0/780522392 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe6a40ae890 0x7fe6a40aed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:35.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.085+0000 7fe6aa000640 1 --2- 172.21.15.131:0/780522392 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe688004800 0x7fe6a4156bb0 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:35.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.085+0000 7fe6aa000640 1 -- 172.21.15.131:0/780522392 >> 172.21.15.131:0/780522392 conn(0x7fe6a40b1e90 msgr2=0x7fe6a415f510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:52:35.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.085+0000 7fe6aa000640 1 -- 172.21.15.131:0/780522392 shutdown_connections 2024-05-26T22:52:35.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.085+0000 7fe6aa000640 1 -- 172.21.15.131:0/780522392 wait complete. 2024-05-26T22:52:35.104 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-26T22:48:10.463339+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-26T22:51:36.873056+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '57', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-05-26T22:51:38.120229+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '58', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-26T22:52:35.105 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-26T22:52:35.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.314+0000 7f7b4c200640 1 Processor -- start 2024-05-26T22:52:35.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.314+0000 7f7b4c200640 1 -- start start 2024-05-26T22:52:35.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.316+0000 7f7b4c200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7b44155770 0x7f7b4415b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:35.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.316+0000 7f7b4c200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f7b4415bc40 0x7f7b4406dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:35.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.316+0000 7f7b4c200640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b440735b0 con 0x7f7b4406e270 2024-05-26T22:52:35.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.316+0000 7f7b4c200640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f7b4406e6a0 con 0x7f7b4415bc40 2024-05-26T22:52:35.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.316+0000 7f7b4c200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f7b4415c1c0 con 0x7f7b44155770 2024-05-26T22:52:35.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.316+0000 7f7b48e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f7b4415bc40 0x7f7b4406dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:35.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.316+0000 7f7b48e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f7b4415bc40 0x7f7b4406dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49700/0 (socket says 172.21.15.131:49700) 2024-05-26T22:52:35.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.316+0000 7f7b48e00640 1 -- 172.21.15.131:0/4198960376 learned_addr learned my addr 172.21.15.131:0/4198960376 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:52:35.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.317+0000 7f7b48e00640 1 -- 172.21.15.131:0/4198960376 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7b44155770 msgr2=0x7f7b4415b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:35.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.317+0000 7f7b49800640 1 --2- 172.21.15.131:0/4198960376 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7b44155770 0x7f7b4415b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:35.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.317+0000 7f7b48e00640 1 --2- 172.21.15.131:0/4198960376 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7b44155770 0x7f7b4415b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:35.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.317+0000 7f7b48e00640 1 -- 172.21.15.131:0/4198960376 >> v1:172.21.15.187:6789/0 conn(0x7f7b4406e270 legacy=0x7f7b440ceba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:35.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.317+0000 7f7b48e00640 1 -- 172.21.15.131:0/4198960376 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b4415c4a0 con 0x7f7b4415bc40 2024-05-26T22:52:35.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.317+0000 7f7b49800640 1 --2- 172.21.15.131:0/4198960376 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7b44155770 0x7f7b4415b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:52:35.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.318+0000 7f7b48e00640 1 --2- 172.21.15.131:0/4198960376 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b4415bc40 0x7f7b4406dd30 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f7b440cfd60 tx=0x7f7b3002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce3185d9f1d2a302 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:35.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.318+0000 7f7b43e00640 1 -- 172.21.15.131:0/4198960376 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b30037070 con 0x7f7b4415bc40 2024-05-26T22:52:35.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.319+0000 7f7b43e00640 1 -- 172.21.15.131:0/4198960376 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b30032440 con 0x7f7b4415bc40 2024-05-26T22:52:35.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.319+0000 7f7b43e00640 1 -- 172.21.15.131:0/4198960376 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b30032810 con 0x7f7b4415bc40 2024-05-26T22:52:35.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.319+0000 7f7b4c200640 1 -- 172.21.15.131:0/4198960376 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b4415bc40 msgr2=0x7f7b4406dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:35.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.319+0000 7f7b4c200640 1 --2- 172.21.15.131:0/4198960376 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b4415bc40 0x7f7b4406dd30 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f7b440cfd60 tx=0x7f7b3002a270 comp rx=0 tx=0).stop 2024-05-26T22:52:35.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.319+0000 7f7b4c200640 1 -- 172.21.15.131:0/4198960376 shutdown_connections 2024-05-26T22:52:35.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.319+0000 7f7b4c200640 1 --2- 172.21.15.131:0/4198960376 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b4415bc40 0x7f7b4406dd30 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:35.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.319+0000 7f7b4c200640 1 --2- 172.21.15.131:0/4198960376 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7b44155770 0x7f7b4415b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:35.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.319+0000 7f7b4c200640 1 -- 172.21.15.131:0/4198960376 >> 172.21.15.131:0/4198960376 conn(0x7f7b4406bc90 msgr2=0x7f7b440b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:52:35.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.319+0000 7f7b4c200640 1 -- 172.21.15.131:0/4198960376 shutdown_connections 2024-05-26T22:52:35.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.319+0000 7f7b4c200640 1 -- 172.21.15.131:0/4198960376 wait complete. 2024-05-26T22:52:35.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.321+0000 7f7b4c200640 1 Processor -- start 2024-05-26T22:52:35.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.321+0000 7f7b4c200640 1 -- start start 2024-05-26T22:52:35.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.322+0000 7f7b4c200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7b4406e270 0x7f7b4409aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:35.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.322+0000 7f7b4c200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7b44155770 0x7f7b4415aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:35.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.322+0000 7f7b4c200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b4415bc40 0x7f7b441f7070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:35.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.322+0000 7f7b4c200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f7b44155b50 con 0x7f7b4415bc40 2024-05-26T22:52:35.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.322+0000 7f7b4c200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7b44072080 con 0x7f7b4406e270 2024-05-26T22:52:35.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.322+0000 7f7b4c200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f7b4415c8c0 con 0x7f7b44155770 2024-05-26T22:52:35.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.322+0000 7f7b4a200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b4415bc40 0x7f7b441f7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:35.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.322+0000 7f7b4a200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b4415bc40 0x7f7b441f7070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49714/0 (socket says 172.21.15.131:49714) 2024-05-26T22:52:35.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.322+0000 7f7b4a200640 1 -- 172.21.15.131:0/3226423698 learned_addr learned my addr 172.21.15.131:0/3226423698 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:52:35.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.323+0000 7f7b48e00640 1 --2- 172.21.15.131:0/3226423698 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7b44155770 0x7f7b4415aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:35.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.323+0000 7f7b49800640 1 --2- 172.21.15.131:0/3226423698 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7b4406e270 0x7f7b4409aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:35.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.323+0000 7f7b4a200640 1 -- 172.21.15.131:0/3226423698 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7b44155770 msgr2=0x7f7b4415aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:35.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.323+0000 7f7b4a200640 1 --2- 172.21.15.131:0/3226423698 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7b44155770 0x7f7b4415aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:35.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.323+0000 7f7b4a200640 1 -- 172.21.15.131:0/3226423698 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7b4406e270 msgr2=0x7f7b4409aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:35.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.323+0000 7f7b4a200640 1 --2- 172.21.15.131:0/3226423698 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7b4406e270 0x7f7b4409aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:35.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.323+0000 7f7b4a200640 1 -- 172.21.15.131:0/3226423698 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b441f75b0 con 0x7f7b4415bc40 2024-05-26T22:52:35.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.323+0000 7f7b49800640 1 --2- 172.21.15.131:0/3226423698 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7b4406e270 0x7f7b4409aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:52:35.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.323+0000 7f7b4a200640 1 --2- 172.21.15.131:0/3226423698 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b4415bc40 0x7f7b441f7070 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f7b3c00c780 tx=0x7f7b3c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:35.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.324+0000 7f7b41600640 1 -- 172.21.15.131:0/3226423698 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b3c004b60 con 0x7f7b4415bc40 2024-05-26T22:52:35.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.324+0000 7f7b41600640 1 -- 172.21.15.131:0/3226423698 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b3c004d00 con 0x7f7b4415bc40 2024-05-26T22:52:35.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.324+0000 7f7b41600640 1 -- 172.21.15.131:0/3226423698 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b3c00aa90 con 0x7f7b4415bc40 2024-05-26T22:52:35.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.324+0000 7f7b4c200640 1 -- 172.21.15.131:0/3226423698 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b4415b1f0 con 0x7f7b4415bc40 2024-05-26T22:52:35.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.324+0000 7f7b4c200640 1 -- 172.21.15.131:0/3226423698 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b4415b7b0 con 0x7f7b4415bc40 2024-05-26T22:52:35.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.325+0000 7f7b4c200640 1 -- 172.21.15.131:0/3226423698 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b44074e90 con 0x7f7b4415bc40 2024-05-26T22:52:35.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.326+0000 7f7b41600640 1 -- 172.21.15.131:0/3226423698 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7b3c00ac30 con 0x7f7b4415bc40 2024-05-26T22:52:35.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.327+0000 7f7b41600640 1 --2- 172.21.15.131:0/3226423698 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f7b2406be30 0x7f7b2406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:35.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.327+0000 7f7b41600640 1 -- 172.21.15.131:0/3226423698 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f7b3c08b100 con 0x7f7b4415bc40 2024-05-26T22:52:35.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.327+0000 7f7b49800640 1 --2- 172.21.15.131:0/3226423698 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f7b2406be30 0x7f7b2406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:35.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.328+0000 7f7b49800640 1 --2- 172.21.15.131:0/3226423698 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f7b2406be30 0x7f7b2406e2f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f7b38006fd0 tx=0x7f7b38008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:35.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.331+0000 7f7b41600640 1 -- 172.21.15.131:0/3226423698 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f7b3c020070 con 0x7f7b4415bc40 2024-05-26T22:52:35.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.534+0000 7f7b4c200640 1 -- 172.21.15.131:0/3226423698 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7b440aff80 con 0x7f7b4415bc40 2024-05-26T22:52:35.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.535+0000 7f7b41600640 1 -- 172.21.15.131:0/3226423698 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v58) ==== 93+0+10 (secure 0 0 0) 0x7f7b440aff80 con 0x7f7b4415bc40 2024-05-26T22:52:35.534 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2024-05-26T22:52:35.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.537+0000 7f7b4c200640 1 -- 172.21.15.131:0/3226423698 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f7b2406be30 msgr2=0x7f7b2406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:35.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.537+0000 7f7b4c200640 1 --2- 172.21.15.131:0/3226423698 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f7b2406be30 0x7f7b2406e2f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f7b38006fd0 tx=0x7f7b38008040 comp rx=0 tx=0).stop 2024-05-26T22:52:35.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.537+0000 7f7b4c200640 1 -- 172.21.15.131:0/3226423698 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b4415bc40 msgr2=0x7f7b441f7070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:35.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.537+0000 7f7b4c200640 1 --2- 172.21.15.131:0/3226423698 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b4415bc40 0x7f7b441f7070 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f7b3c00c780 tx=0x7f7b3c00cc40 comp rx=0 tx=0).stop 2024-05-26T22:52:35.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.538+0000 7f7b4c200640 1 -- 172.21.15.131:0/3226423698 shutdown_connections 2024-05-26T22:52:35.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.538+0000 7f7b4c200640 1 --2- 172.21.15.131:0/3226423698 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7b44155770 0x7f7b4415aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:35.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.538+0000 7f7b4c200640 1 --2- 172.21.15.131:0/3226423698 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f7b2406be30 0x7f7b2406e2f0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:35.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.538+0000 7f7b4c200640 1 --2- 172.21.15.131:0/3226423698 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b4415bc40 0x7f7b441f7070 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:35.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.538+0000 7f7b4c200640 1 --2- 172.21.15.131:0/3226423698 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7b4406e270 0x7f7b4409aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:35.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.538+0000 7f7b4c200640 1 -- 172.21.15.131:0/3226423698 >> 172.21.15.131:0/3226423698 conn(0x7f7b4406bc90 msgr2=0x7f7b440cc6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:52:35.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.538+0000 7f7b4c200640 1 -- 172.21.15.131:0/3226423698 shutdown_connections 2024-05-26T22:52:35.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.538+0000 7f7b4c200640 1 -- 172.21.15.131:0/3226423698 wait complete. 2024-05-26T22:52:35.556 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-26T22:52:35.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.769+0000 7fb3d7000640 1 Processor -- start 2024-05-26T22:52:35.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.769+0000 7fb3d7000640 1 -- start start 2024-05-26T22:52:35.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.771+0000 7fb3d7000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb3d015f6e0 0x7fb3d00ad870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:35.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.771+0000 7fb3d7000640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3d00ae0c0 con 0x7fb3d00af580 2024-05-26T22:52:35.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.771+0000 7fb3d7000640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3d00ae3a0 con 0x7fb3d006b680 2024-05-26T22:52:35.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.771+0000 7fb3d7000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb3d00ae680 con 0x7fb3d015f6e0 2024-05-26T22:52:35.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.771+0000 7fb3cfe00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fb3d00af580 0x7fb3d00af960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:52050/0 (socket says 172.21.15.131:52050) 2024-05-26T22:52:35.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.771+0000 7fb3cfe00640 1 -- 172.21.15.131:0/931660354 learned_addr learned my addr 172.21.15.131:0/931660354 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:52:35.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.772+0000 7fb3cf400640 1 -- 172.21.15.131:0/931660354 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 68586363 0 0) 0x7fb3d00ae0c0 con 0x7fb3d00af580 2024-05-26T22:52:35.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.772+0000 7fb3cf400640 1 -- 172.21.15.131:0/931660354 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb3b4003520 con 0x7fb3d00af580 2024-05-26T22:52:35.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.772+0000 7fb3c7400640 1 --2- 172.21.15.131:0/931660354 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb3d015f6e0 0x7fb3d00ad870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:35.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.772+0000 7fb3cf400640 1 -- 172.21.15.131:0/931660354 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 526250214 0 0) 0x7fb3b4003520 con 0x7fb3d00af580 2024-05-26T22:52:35.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.772+0000 7fb3cf400640 1 -- 172.21.15.131:0/931660354 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb3d015f6e0 msgr2=0x7fb3d00ad870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:35.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.772+0000 7fb3cf400640 1 --2- 172.21.15.131:0/931660354 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb3d015f6e0 0x7fb3d00ad870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:35.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.772+0000 7fb3cf400640 1 -- 172.21.15.131:0/931660354 >> v1:172.21.15.187:6789/0 conn(0x7fb3d006b680 legacy=0x7fb3d006bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:35.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.772+0000 7fb3cf400640 1 -- 172.21.15.131:0/931660354 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3d00ae960 con 0x7fb3d00af580 2024-05-26T22:52:35.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.772+0000 7fb3cf400640 1 -- 172.21.15.131:0/931660354 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7fb3c0001670 con 0x7fb3d00af580 2024-05-26T22:52:35.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.772+0000 7fb3cf400640 1 -- 172.21.15.131:0/931660354 >> v1:172.21.15.131:6789/0 conn(0x7fb3d00af580 legacy=0x7fb3d00af960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:35.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.772+0000 7fb3cf400640 1 --2- 172.21.15.131:0/931660354 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb3d015f6e0 0x7fb3b40039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:35.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.772+0000 7fb3cf400640 1 --2- 172.21.15.131:0/931660354 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb3b4004040 0x7fb3b4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:35.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.772+0000 7fb3cf400640 1 --2- 172.21.15.131:0/931660354 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb3b4004db0 0x7fb3b4005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:35.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.772+0000 7fb3cf400640 1 -- 172.21.15.131:0/931660354 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d00ae680 con 0x7fb3b4004db0 2024-05-26T22:52:35.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.772+0000 7fb3cf400640 1 -- 172.21.15.131:0/931660354 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b400bcc0 con 0x7fb3b4004040 2024-05-26T22:52:35.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.772+0000 7fb3cf400640 1 -- 172.21.15.131:0/931660354 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb3b400bec0 con 0x7fb3d015f6e0 2024-05-26T22:52:35.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.773+0000 7fb3d5000640 1 --2- 172.21.15.131:0/931660354 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb3b4004db0 0x7fb3b4005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:35.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.773+0000 7fb3cfe00640 1 --2- 172.21.15.131:0/931660354 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb3d015f6e0 0x7fb3b40039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:35.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.773+0000 7fb3c7400640 1 --2- 172.21.15.131:0/931660354 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb3b4004040 0x7fb3b4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:35.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.773+0000 7fb3d5000640 1 -- 172.21.15.131:0/931660354 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb3d015f6e0 msgr2=0x7fb3b40039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:35.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.773+0000 7fb3d5000640 1 --2- 172.21.15.131:0/931660354 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb3d015f6e0 0x7fb3b40039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:35.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.773+0000 7fb3d5000640 1 -- 172.21.15.131:0/931660354 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb3b4004040 msgr2=0x7fb3b4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:35.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.773+0000 7fb3d5000640 1 --2- 172.21.15.131:0/931660354 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb3b4004040 0x7fb3b4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:35.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.773+0000 7fb3d5000640 1 -- 172.21.15.131:0/931660354 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3b4003520 con 0x7fb3b4004db0 2024-05-26T22:52:35.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.773+0000 7fb3c7400640 1 --2- 172.21.15.131:0/931660354 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb3b4004040 0x7fb3b4004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:52:35.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.774+0000 7fb3d5000640 1 --2- 172.21.15.131:0/931660354 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb3b4004db0 0x7fb3b4005230 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fb3c8003db0 tx=0x7fb3b4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8850cb9dbf8ecb65 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:35.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.774+0000 7fb3cf400640 1 -- 172.21.15.131:0/931660354 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3c8037070 con 0x7fb3b4004db0 2024-05-26T22:52:35.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.774+0000 7fb3cf400640 1 -- 172.21.15.131:0/931660354 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3c802aa00 con 0x7fb3b4004db0 2024-05-26T22:52:35.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.774+0000 7fb3cf400640 1 -- 172.21.15.131:0/931660354 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3c8003980 con 0x7fb3b4004db0 2024-05-26T22:52:35.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.775+0000 7fb3d7000640 1 -- 172.21.15.131:0/931660354 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb3b4004db0 msgr2=0x7fb3b4005230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:35.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.775+0000 7fb3d7000640 1 --2- 172.21.15.131:0/931660354 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb3b4004db0 0x7fb3b4005230 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fb3c8003db0 tx=0x7fb3b4005830 comp rx=0 tx=0).stop 2024-05-26T22:52:35.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.775+0000 7fb3d5000640 1 -- 172.21.15.131:0/931660354 reap_dead start 2024-05-26T22:52:35.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.775+0000 7fb3d7000640 1 -- 172.21.15.131:0/931660354 shutdown_connections 2024-05-26T22:52:35.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.775+0000 7fb3d7000640 1 -- 172.21.15.131:0/931660354 >> 172.21.15.131:0/931660354 conn(0x7fb3d0072b00 msgr2=0x7fb3d006de50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:52:35.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.775+0000 7fb3d7000640 1 -- 172.21.15.131:0/931660354 shutdown_connections 2024-05-26T22:52:35.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.775+0000 7fb3d7000640 1 -- 172.21.15.131:0/931660354 wait complete. 2024-05-26T22:52:35.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.777+0000 7fb3d7000640 1 Processor -- start 2024-05-26T22:52:35.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.777+0000 7fb3d7000640 1 -- start start 2024-05-26T22:52:35.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.778+0000 7fb3d7000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb3b4004db0 0x7fb3d00ce060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:35.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.778+0000 7fb3d7000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb3d013fd70 0x7fb3d01401f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:35.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.778+0000 7fb3d7000640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb3d01f4570 0x7fb3d01f49f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:35.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.778+0000 7fb3d7000640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d00ad9f0 con 0x7fb3d013fd70 2024-05-26T22:52:35.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.778+0000 7fb3d7000640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d00ad870 con 0x7fb3d01f4570 2024-05-26T22:52:35.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.778+0000 7fb3d7000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb3d00adb70 con 0x7fb3b4004db0 2024-05-26T22:52:35.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.778+0000 7fb3c7400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb3d013fd70 0x7fb3d01401f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:35.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.778+0000 7fb3c7400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb3d013fd70 0x7fb3d01401f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49732/0 (socket says 172.21.15.131:49732) 2024-05-26T22:52:35.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.778+0000 7fb3c7400640 1 -- 172.21.15.131:0/916710918 learned_addr learned my addr 172.21.15.131:0/916710918 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:52:35.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.779+0000 7fb3cfe00640 1 --2- 172.21.15.131:0/916710918 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb3b4004db0 0x7fb3d00ce060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:35.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.779+0000 7fb3c7400640 1 -- 172.21.15.131:0/916710918 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb3b4004db0 msgr2=0x7fb3d00ce060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:35.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.779+0000 7fb3c7400640 1 --2- 172.21.15.131:0/916710918 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb3b4004db0 0x7fb3d00ce060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:35.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.779+0000 7fb3c7400640 1 -- 172.21.15.131:0/916710918 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb3d01f4570 msgr2=0x7fb3d01f49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:35.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.779+0000 7fb3c7400640 1 --2- 172.21.15.131:0/916710918 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb3d01f4570 0x7fb3d01f49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:35.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.779+0000 7fb3c7400640 1 -- 172.21.15.131:0/916710918 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3d00cd2b0 con 0x7fb3d013fd70 2024-05-26T22:52:35.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.779+0000 7fb3c7400640 1 --2- 172.21.15.131:0/916710918 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb3d013fd70 0x7fb3d01401f0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fb3b8009fd0 tx=0x7fb3b800f5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:35.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.779+0000 7fb3ccc00640 1 -- 172.21.15.131:0/916710918 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b8018020 con 0x7fb3d013fd70 2024-05-26T22:52:35.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.779+0000 7fb3ccc00640 1 -- 172.21.15.131:0/916710918 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3b8014610 con 0x7fb3d013fd70 2024-05-26T22:52:35.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.779+0000 7fb3ccc00640 1 -- 172.21.15.131:0/916710918 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b8014a00 con 0x7fb3d013fd70 2024-05-26T22:52:35.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.779+0000 7fb3d7000640 1 -- 172.21.15.131:0/916710918 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3d00b14d0 con 0x7fb3d013fd70 2024-05-26T22:52:35.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.779+0000 7fb3d7000640 1 -- 172.21.15.131:0/916710918 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3d00b1a90 con 0x7fb3d013fd70 2024-05-26T22:52:35.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.781+0000 7fb3d7000640 1 -- 172.21.15.131:0/916710918 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3d00695c0 con 0x7fb3d013fd70 2024-05-26T22:52:35.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.781+0000 7fb3ccc00640 1 -- 172.21.15.131:0/916710918 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb3b801b020 con 0x7fb3d013fd70 2024-05-26T22:52:35.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.782+0000 7fb3ccc00640 1 --2- 172.21.15.131:0/916710918 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb3ac06be30 0x7fb3ac06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:35.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.783+0000 7fb3ccc00640 1 -- 172.21.15.131:0/916710918 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fb3b808b700 con 0x7fb3d013fd70 2024-05-26T22:52:35.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.783+0000 7fb3cfe00640 1 --2- 172.21.15.131:0/916710918 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb3ac06be30 0x7fb3ac06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:35.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.783+0000 7fb3cfe00640 1 --2- 172.21.15.131:0/916710918 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb3ac06be30 0x7fb3ac06e2f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fb3c0001eb0 tx=0x7fb3c0000fb0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:35.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.787+0000 7fb3ccc00640 1 -- 172.21.15.131:0/916710918 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fb3b8022030 con 0x7fb3d013fd70 2024-05-26T22:52:35.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.984+0000 7fb3d7000640 1 -- 172.21.15.131:0/916710918 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb3d0067320 con 0x7fb3d013fd70 2024-05-26T22:52:35.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.985+0000 7fb3ccc00640 1 -- 172.21.15.131:0/916710918 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v58) ==== 104+0+11 (secure 0 0 0) 0x7fb3b805a0b0 con 0x7fb3d013fd70 2024-05-26T22:52:35.985 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-05-26T22:52:35.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.988+0000 7fb3d7000640 1 -- 172.21.15.131:0/916710918 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb3ac06be30 msgr2=0x7fb3ac06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:35.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.988+0000 7fb3d7000640 1 --2- 172.21.15.131:0/916710918 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb3ac06be30 0x7fb3ac06e2f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fb3c0001eb0 tx=0x7fb3c0000fb0 comp rx=0 tx=0).stop 2024-05-26T22:52:35.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.988+0000 7fb3d7000640 1 -- 172.21.15.131:0/916710918 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb3d013fd70 msgr2=0x7fb3d01401f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:35.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.988+0000 7fb3d7000640 1 --2- 172.21.15.131:0/916710918 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb3d013fd70 0x7fb3d01401f0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fb3b8009fd0 tx=0x7fb3b800f5e0 comp rx=0 tx=0).stop 2024-05-26T22:52:35.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.988+0000 7fb3d7000640 1 -- 172.21.15.131:0/916710918 shutdown_connections 2024-05-26T22:52:35.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.988+0000 7fb3d7000640 1 --2- 172.21.15.131:0/916710918 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb3ac06be30 0x7fb3ac06e2f0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:35.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.988+0000 7fb3d7000640 1 --2- 172.21.15.131:0/916710918 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb3d01f4570 0x7fb3d01f49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:35.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.988+0000 7fb3d7000640 1 --2- 172.21.15.131:0/916710918 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb3d013fd70 0x7fb3d01401f0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:35.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.988+0000 7fb3d7000640 1 --2- 172.21.15.131:0/916710918 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb3b4004db0 0x7fb3d00ce060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:35.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.988+0000 7fb3d7000640 1 -- 172.21.15.131:0/916710918 >> 172.21.15.131:0/916710918 conn(0x7fb3d0072b00 msgr2=0x7fb3d00b44e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:52:35.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.989+0000 7fb3d7000640 1 -- 172.21.15.131:0/916710918 shutdown_connections 2024-05-26T22:52:35.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:35.989+0000 7fb3d7000640 1 -- 172.21.15.131:0/916710918 wait complete. 2024-05-26T22:52:36.008 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-26T22:52:36.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.212+0000 7ffa89800640 1 Processor -- start 2024-05-26T22:52:36.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.212+0000 7ffa89800640 1 -- start start 2024-05-26T22:52:36.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.214+0000 7ffa89800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffa8406d250 0x7ffa8406d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:36.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.214+0000 7ffa89800640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7ffa841400c0 0x7ffa841404f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:36.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.214+0000 7ffa89800640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa8406db70 con 0x7ffa8415b730 2024-05-26T22:52:36.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.214+0000 7ffa89800640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7ffa84071260 con 0x7ffa841400c0 2024-05-26T22:52:36.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.214+0000 7ffa89800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ffa8413e820 con 0x7ffa8406d250 2024-05-26T22:52:36.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.214+0000 7ffa83400640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7ffa8415b730 0x7ffa8415bb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:52062/0 (socket says 172.21.15.131:52062) 2024-05-26T22:52:36.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.214+0000 7ffa83400640 1 -- 172.21.15.131:0/2148057052 learned_addr learned my addr 172.21.15.131:0/2148057052 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:52:36.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.215+0000 7ffa88e00640 1 --2- 172.21.15.131:0/2148057052 >> v2:172.21.15.187:3300/0 conn(0x7ffa841400c0 0x7ffa841404f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:36.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.215+0000 7ffa82a00640 1 -- 172.21.15.131:0/2148057052 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 383288751 0 0) 0x7ffa8406db70 con 0x7ffa8415b730 2024-05-26T22:52:36.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.215+0000 7ffa83e00640 1 --2- 172.21.15.131:0/2148057052 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffa8406d250 0x7ffa8406d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:36.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.215+0000 7ffa82a00640 1 -- 172.21.15.131:0/2148057052 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffa68003520 con 0x7ffa8415b730 2024-05-26T22:52:36.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.215+0000 7ffa82a00640 1 -- 172.21.15.131:0/2148057052 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 862609412 0 0) 0x7ffa68003520 con 0x7ffa8415b730 2024-05-26T22:52:36.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.215+0000 7ffa82a00640 1 -- 172.21.15.131:0/2148057052 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffa8406d250 msgr2=0x7ffa8406d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:36.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.215+0000 7ffa82a00640 1 --2- 172.21.15.131:0/2148057052 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffa8406d250 0x7ffa8406d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:36.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.215+0000 7ffa82a00640 1 -- 172.21.15.131:0/2148057052 >> v2:172.21.15.187:3300/0 conn(0x7ffa841400c0 msgr2=0x7ffa841404f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:36.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.215+0000 7ffa82a00640 1 --2- 172.21.15.131:0/2148057052 >> v2:172.21.15.187:3300/0 conn(0x7ffa841400c0 0x7ffa841404f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:36.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.215+0000 7ffa82a00640 1 -- 172.21.15.131:0/2148057052 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa8415c4a0 con 0x7ffa8415b730 2024-05-26T22:52:36.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.215+0000 7ffa82a00640 1 -- 172.21.15.131:0/2148057052 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7ffa70001670 con 0x7ffa8415b730 2024-05-26T22:52:36.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.215+0000 7ffa83e00640 1 --2- 172.21.15.131:0/2148057052 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffa8406d250 0x7ffa8406d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-26T22:52:36.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.215+0000 7ffa82a00640 1 -- 172.21.15.131:0/2148057052 >> v1:172.21.15.131:6789/0 conn(0x7ffa8415b730 legacy=0x7ffa8415bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:36.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.215+0000 7ffa82a00640 1 --2- 172.21.15.131:0/2148057052 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffa8406d250 0x7ffa680039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:36.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.216+0000 7ffa82a00640 1 --2- 172.21.15.131:0/2148057052 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffa68003f40 0x7ffa68004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:36.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.216+0000 7ffa82a00640 1 --2- 172.21.15.131:0/2148057052 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa68004d40 0x7ffa680051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:36.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.216+0000 7ffa82a00640 1 -- 172.21.15.131:0/2148057052 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ffa84071260 con 0x7ffa68004d40 2024-05-26T22:52:36.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.216+0000 7ffa82a00640 1 -- 172.21.15.131:0/2148057052 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ffa8413e820 con 0x7ffa68003f40 2024-05-26T22:52:36.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.216+0000 7ffa82a00640 1 -- 172.21.15.131:0/2148057052 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ffa6800dc50 con 0x7ffa8406d250 2024-05-26T22:52:36.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.216+0000 7ffa88e00640 1 --2- 172.21.15.131:0/2148057052 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa68004d40 0x7ffa680051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:36.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.216+0000 7ffa83400640 1 --2- 172.21.15.131:0/2148057052 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffa68003f40 0x7ffa68004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:36.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.216+0000 7ffa83e00640 1 --2- 172.21.15.131:0/2148057052 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffa8406d250 0x7ffa680039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:36.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.216+0000 7ffa88e00640 1 -- 172.21.15.131:0/2148057052 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffa8406d250 msgr2=0x7ffa680039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:36.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.216+0000 7ffa88e00640 1 --2- 172.21.15.131:0/2148057052 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffa8406d250 0x7ffa680039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:36.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.216+0000 7ffa88e00640 1 -- 172.21.15.131:0/2148057052 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffa68003f40 msgr2=0x7ffa68004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:36.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.216+0000 7ffa88e00640 1 --2- 172.21.15.131:0/2148057052 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffa68003f40 0x7ffa68004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:36.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.216+0000 7ffa88e00640 1 -- 172.21.15.131:0/2148057052 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa68003520 con 0x7ffa68004d40 2024-05-26T22:52:36.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.217+0000 7ffa88e00640 1 --2- 172.21.15.131:0/2148057052 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa68004d40 0x7ffa680051c0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c009fc0 tx=0x7ffa68005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=321583affc9bd78c server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:36.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.218+0000 7ffa82a00640 1 -- 172.21.15.131:0/2148057052 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa7c009c10 con 0x7ffa68004d40 2024-05-26T22:52:36.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.218+0000 7ffa82a00640 1 -- 172.21.15.131:0/2148057052 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa7c036070 con 0x7ffa68004d40 2024-05-26T22:52:36.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.218+0000 7ffa82a00640 1 -- 172.21.15.131:0/2148057052 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa7c02d070 con 0x7ffa68004d40 2024-05-26T22:52:36.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.218+0000 7ffa89800640 1 -- 172.21.15.131:0/2148057052 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa68004d40 msgr2=0x7ffa680051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:36.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.218+0000 7ffa89800640 1 --2- 172.21.15.131:0/2148057052 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa68004d40 0x7ffa680051c0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c009fc0 tx=0x7ffa68005830 comp rx=0 tx=0).stop 2024-05-26T22:52:36.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.218+0000 7ffa88e00640 1 -- 172.21.15.131:0/2148057052 reap_dead start 2024-05-26T22:52:36.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.218+0000 7ffa89800640 1 -- 172.21.15.131:0/2148057052 shutdown_connections 2024-05-26T22:52:36.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.218+0000 7ffa89800640 1 -- 172.21.15.131:0/2148057052 >> 172.21.15.131:0/2148057052 conn(0x7ffa840b5660 msgr2=0x7ffa840b5a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:52:36.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.218+0000 7ffa89800640 1 -- 172.21.15.131:0/2148057052 shutdown_connections 2024-05-26T22:52:36.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.218+0000 7ffa89800640 1 -- 172.21.15.131:0/2148057052 wait complete. 2024-05-26T22:52:36.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.220+0000 7ffa89800640 1 Processor -- start 2024-05-26T22:52:36.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.220+0000 7ffa89800640 1 -- start start 2024-05-26T22:52:36.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.221+0000 7ffa89800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffa68004d40 0x7ffa8413da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:36.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.221+0000 7ffa89800640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffa841f0af0 0x7ffa84142590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:36.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.221+0000 7ffa89800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa841f0f70 0x7ffa84142ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:36.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.221+0000 7ffa89800640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ffa84071a70 con 0x7ffa841f0f70 2024-05-26T22:52:36.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.221+0000 7ffa89800640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ffa84071790 con 0x7ffa841f0af0 2024-05-26T22:52:36.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.221+0000 7ffa89800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ffa84072fa0 con 0x7ffa68004d40 2024-05-26T22:52:36.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.221+0000 7ffa88e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa841f0f70 0x7ffa84142ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:36.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.221+0000 7ffa88e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa841f0f70 0x7ffa84142ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49750/0 (socket says 172.21.15.131:49750) 2024-05-26T22:52:36.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.221+0000 7ffa88e00640 1 -- 172.21.15.131:0/2917490971 learned_addr learned my addr 172.21.15.131:0/2917490971 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:52:36.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.222+0000 7ffa88e00640 1 -- 172.21.15.131:0/2917490971 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffa68004d40 msgr2=0x7ffa8413da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:36.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.222+0000 7ffa88e00640 1 --2- 172.21.15.131:0/2917490971 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffa68004d40 0x7ffa8413da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:36.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.222+0000 7ffa83400640 1 --2- 172.21.15.131:0/2917490971 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffa841f0af0 0x7ffa84142590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:36.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.222+0000 7ffa88e00640 1 -- 172.21.15.131:0/2917490971 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffa841f0af0 msgr2=0x7ffa84142590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:36.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.222+0000 7ffa88e00640 1 --2- 172.21.15.131:0/2917490971 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffa841f0af0 0x7ffa84142590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:36.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.222+0000 7ffa88e00640 1 -- 172.21.15.131:0/2917490971 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa841414f0 con 0x7ffa841f0f70 2024-05-26T22:52:36.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.222+0000 7ffa88e00640 1 --2- 172.21.15.131:0/2917490971 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa841f0f70 0x7ffa84142ad0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c00e230 tx=0x7ffa7c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:36.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.222+0000 7ffa7be00640 1 -- 172.21.15.131:0/2917490971 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa7c009c10 con 0x7ffa841f0f70 2024-05-26T22:52:36.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.223+0000 7ffa7be00640 1 -- 172.21.15.131:0/2917490971 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa7c003990 con 0x7ffa841f0f70 2024-05-26T22:52:36.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.223+0000 7ffa89800640 1 -- 172.21.15.131:0/2917490971 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa841417d0 con 0x7ffa841f0f70 2024-05-26T22:52:36.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.223+0000 7ffa7be00640 1 -- 172.21.15.131:0/2917490971 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa7c003d80 con 0x7ffa841f0f70 2024-05-26T22:52:36.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.223+0000 7ffa89800640 1 -- 172.21.15.131:0/2917490971 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa84141d90 con 0x7ffa841f0f70 2024-05-26T22:52:36.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.224+0000 7ffa89800640 1 -- 172.21.15.131:0/2917490971 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa840694e0 con 0x7ffa841f0f70 2024-05-26T22:52:36.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.225+0000 7ffa7be00640 1 -- 172.21.15.131:0/2917490971 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ffa7c005d10 con 0x7ffa841f0f70 2024-05-26T22:52:36.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.226+0000 7ffa7be00640 1 --2- 172.21.15.131:0/2917490971 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ffa6406be00 0x7ffa6406e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:36.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.226+0000 7ffa7be00640 1 -- 172.21.15.131:0/2917490971 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7ffa7c0ac1c0 con 0x7ffa841f0f70 2024-05-26T22:52:36.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.226+0000 7ffa83e00640 1 --2- 172.21.15.131:0/2917490971 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ffa6406be00 0x7ffa6406e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:36.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.227+0000 7ffa83e00640 1 --2- 172.21.15.131:0/2917490971 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ffa6406be00 0x7ffa6406e2c0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7ffa74006f80 tx=0x7ffa74006e30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:36.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.230+0000 7ffa7be00640 1 -- 172.21.15.131:0/2917490971 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7ffa7c07aad0 con 0x7ffa841f0f70 2024-05-26T22:52:36.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.426+0000 7ffa89800640 1 -- 172.21.15.131:0/2917490971 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ffa840b5db0 con 0x7ffa841f0f70 2024-05-26T22:52:36.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.427+0000 7ffa7be00640 1 -- 172.21.15.131:0/2917490971 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v58) ==== 100+0+11 (secure 0 0 0) 0x7ffa7c07a3d0 con 0x7ffa841f0f70 2024-05-26T22:52:36.426 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 64 2024-05-26T22:52:36.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.429+0000 7ffa89800640 1 -- 172.21.15.131:0/2917490971 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ffa6406be00 msgr2=0x7ffa6406e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:36.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.429+0000 7ffa89800640 1 --2- 172.21.15.131:0/2917490971 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ffa6406be00 0x7ffa6406e2c0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7ffa74006f80 tx=0x7ffa74006e30 comp rx=0 tx=0).stop 2024-05-26T22:52:36.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.429+0000 7ffa89800640 1 -- 172.21.15.131:0/2917490971 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa841f0f70 msgr2=0x7ffa84142ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:36.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.429+0000 7ffa89800640 1 --2- 172.21.15.131:0/2917490971 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa841f0f70 0x7ffa84142ad0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c00e230 tx=0x7ffa7c009750 comp rx=0 tx=0).stop 2024-05-26T22:52:36.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.430+0000 7ffa89800640 1 -- 172.21.15.131:0/2917490971 shutdown_connections 2024-05-26T22:52:36.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.430+0000 7ffa89800640 1 --2- 172.21.15.131:0/2917490971 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ffa6406be00 0x7ffa6406e2c0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:36.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.430+0000 7ffa89800640 1 --2- 172.21.15.131:0/2917490971 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa841f0f70 0x7ffa84142ad0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:36.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.430+0000 7ffa89800640 1 --2- 172.21.15.131:0/2917490971 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffa841f0af0 0x7ffa84142590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:36.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.430+0000 7ffa89800640 1 --2- 172.21.15.131:0/2917490971 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffa68004d40 0x7ffa8413da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:36.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.430+0000 7ffa89800640 1 -- 172.21.15.131:0/2917490971 >> 172.21.15.131:0/2917490971 conn(0x7ffa840b5660 msgr2=0x7ffa84155580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:52:36.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.430+0000 7ffa89800640 1 -- 172.21.15.131:0/2917490971 shutdown_connections 2024-05-26T22:52:36.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.430+0000 7ffa89800640 1 -- 172.21.15.131:0/2917490971 wait complete. 2024-05-26T22:52:36.449 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-26T22:52:36.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.642+0000 7f164da00640 1 Processor -- start 2024-05-26T22:52:36.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.643+0000 7f164da00640 1 -- start start 2024-05-26T22:52:36.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.644+0000 7f164da00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f16480ad450 0x7f164806eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:36.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.644+0000 7f164da00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f16480ad8e0 0x7f164806f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:36.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.644+0000 7f164da00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16480b5190 con 0x7f1648071d80 2024-05-26T22:52:36.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.644+0000 7f164da00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f164806dee0 con 0x7f16480ad450 2024-05-26T22:52:36.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.644+0000 7f164da00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f164806e1c0 con 0x7f16480ad8e0 2024-05-26T22:52:36.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.645+0000 7f1646a00640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f1648071d80 0x7f1648072160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.131:39010/0 (socket says 172.21.15.131:39010) 2024-05-26T22:52:36.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.645+0000 7f1646a00640 1 -- 172.21.15.131:0/3162595755 learned_addr learned my addr 172.21.15.131:0/3162595755 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:52:36.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.645+0000 7f1647400640 1 --2- 172.21.15.131:0/3162595755 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f16480ad8e0 0x7f164806f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:36.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.645+0000 7f163fe00640 1 --2- 172.21.15.131:0/3162595755 >> v2:172.21.15.187:3300/0 conn(0x7f16480ad450 0x7f164806eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:36.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.645+0000 7f1646000640 1 -- 172.21.15.131:0/3162595755 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4101697234 0 0) 0x7f16480b5190 con 0x7f1648071d80 2024-05-26T22:52:36.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.646+0000 7f1646000640 1 -- 172.21.15.131:0/3162595755 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f162c003520 con 0x7f1648071d80 2024-05-26T22:52:36.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.646+0000 7f1647400640 1 -- 172.21.15.131:0/3162595755 >> v2:172.21.15.187:3300/0 conn(0x7f16480ad450 msgr2=0x7f164806eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:36.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.646+0000 7f1647400640 1 --2- 172.21.15.131:0/3162595755 >> v2:172.21.15.187:3300/0 conn(0x7f16480ad450 0x7f164806eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:36.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.646+0000 7f1647400640 1 -- 172.21.15.131:0/3162595755 >> v1:172.21.15.187:6789/0 conn(0x7f1648071d80 legacy=0x7f1648072160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:36.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.646+0000 7f1647400640 1 -- 172.21.15.131:0/3162595755 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f164806e4a0 con 0x7f16480ad8e0 2024-05-26T22:52:36.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.647+0000 7f1647400640 1 --2- 172.21.15.131:0/3162595755 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f16480ad8e0 0x7f164806f1e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1640009d90 tx=0x7f164002a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11bacda4b73af2e3 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:36.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.648+0000 7f1646000640 1 -- 172.21.15.131:0/3162595755 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1640031600 con 0x7f16480ad8e0 2024-05-26T22:52:36.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.648+0000 7f1646000640 1 -- 172.21.15.131:0/3162595755 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16400317a0 con 0x7f16480ad8e0 2024-05-26T22:52:36.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.648+0000 7f1646000640 1 -- 172.21.15.131:0/3162595755 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1640031b50 con 0x7f16480ad8e0 2024-05-26T22:52:36.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.648+0000 7f164da00640 1 -- 172.21.15.131:0/3162595755 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f16480ad8e0 msgr2=0x7f164806f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:36.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.648+0000 7f164da00640 1 --2- 172.21.15.131:0/3162595755 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f16480ad8e0 0x7f164806f1e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1640009d90 tx=0x7f164002a5d0 comp rx=0 tx=0).stop 2024-05-26T22:52:36.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.648+0000 7f164da00640 1 -- 172.21.15.131:0/3162595755 shutdown_connections 2024-05-26T22:52:36.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.648+0000 7f164da00640 1 --2- 172.21.15.131:0/3162595755 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f16480ad8e0 0x7f164806f1e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:36.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.648+0000 7f164da00640 1 --2- 172.21.15.131:0/3162595755 >> v2:172.21.15.187:3300/0 conn(0x7f16480ad450 0x7f164806eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:36.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.648+0000 7f164da00640 1 -- 172.21.15.131:0/3162595755 >> 172.21.15.131:0/3162595755 conn(0x7f16480b1e90 msgr2=0x7f16480b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:52:36.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.649+0000 7f164da00640 1 -- 172.21.15.131:0/3162595755 shutdown_connections 2024-05-26T22:52:36.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.649+0000 7f164da00640 1 -- 172.21.15.131:0/3162595755 wait complete. 2024-05-26T22:52:36.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.650+0000 7f164da00640 1 Processor -- start 2024-05-26T22:52:36.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.651+0000 7f164da00640 1 -- start start 2024-05-26T22:52:36.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.652+0000 7f164da00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1648071d80 0x7f1648146070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:36.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.652+0000 7f164da00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f16480ad450 0x7f1648099850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:36.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.652+0000 7f164da00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1648154b10 0x7f1648156040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:36.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.652+0000 7f164da00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f164815e020 con 0x7f16480ad450 2024-05-26T22:52:36.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.652+0000 7f164da00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f164806f720 con 0x7f1648154b10 2024-05-26T22:52:36.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.652+0000 7f164da00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f164815e1a0 con 0x7f1648071d80 2024-05-26T22:52:36.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.652+0000 7f163fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f16480ad450 0x7f1648099850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:36.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.652+0000 7f163fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f16480ad450 0x7f1648099850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49756/0 (socket says 172.21.15.131:49756) 2024-05-26T22:52:36.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.652+0000 7f163fe00640 1 -- 172.21.15.131:0/349188148 learned_addr learned my addr 172.21.15.131:0/349188148 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:52:36.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.652+0000 7f1647400640 1 --2- 172.21.15.131:0/349188148 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1648154b10 0x7f1648156040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:36.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.652+0000 7f1646a00640 1 --2- 172.21.15.131:0/349188148 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1648071d80 0x7f1648146070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:36.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.653+0000 7f163fe00640 1 -- 172.21.15.131:0/349188148 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1648071d80 msgr2=0x7f1648146070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:36.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.653+0000 7f163fe00640 1 --2- 172.21.15.131:0/349188148 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1648071d80 0x7f1648146070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:36.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.653+0000 7f163fe00640 1 -- 172.21.15.131:0/349188148 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1648154b10 msgr2=0x7f1648156040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:36.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.653+0000 7f163fe00640 1 --2- 172.21.15.131:0/349188148 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1648154b10 0x7f1648156040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:36.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.653+0000 7f163fe00640 1 -- 172.21.15.131:0/349188148 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1648146980 con 0x7f16480ad450 2024-05-26T22:52:36.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.653+0000 7f163fe00640 1 --2- 172.21.15.131:0/349188148 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f16480ad450 0x7f1648099850 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f163000e9e0 tx=0x7f163000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:36.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.653+0000 7f163ea00640 1 -- 172.21.15.131:0/349188148 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1630017070 con 0x7f16480ad450 2024-05-26T22:52:36.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.653+0000 7f163ea00640 1 -- 172.21.15.131:0/349188148 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f163000c9d0 con 0x7f16480ad450 2024-05-26T22:52:36.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.653+0000 7f164da00640 1 -- 172.21.15.131:0/349188148 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1648154fa0 con 0x7f16480ad450 2024-05-26T22:52:36.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.653+0000 7f163ea00640 1 -- 172.21.15.131:0/349188148 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f163000cdc0 con 0x7f16480ad450 2024-05-26T22:52:36.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.653+0000 7f164da00640 1 -- 172.21.15.131:0/349188148 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1648155560 con 0x7f16480ad450 2024-05-26T22:52:36.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.655+0000 7f164da00640 1 -- 172.21.15.131:0/349188148 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16480694e0 con 0x7f16480ad450 2024-05-26T22:52:36.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.655+0000 7f163ea00640 1 -- 172.21.15.131:0/349188148 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f163001a070 con 0x7f16480ad450 2024-05-26T22:52:36.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.656+0000 7f163ea00640 1 --2- 172.21.15.131:0/349188148 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f162406be30 0x7f162406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:36.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.656+0000 7f163ea00640 1 -- 172.21.15.131:0/349188148 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f163008afd0 con 0x7f16480ad450 2024-05-26T22:52:36.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.657+0000 7f1646a00640 1 --2- 172.21.15.131:0/349188148 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f162406be30 0x7f162406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:36.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.657+0000 7f1646a00640 1 --2- 172.21.15.131:0/349188148 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f162406be30 0x7f162406e2f0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f1638007fd0 tx=0x7f1638009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:36.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.661+0000 7f163ea00640 1 -- 172.21.15.131:0/349188148 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f16300598e0 con 0x7f16480ad450 2024-05-26T22:52:36.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.879+0000 7f164da00640 1 -- 172.21.15.131:0/349188148 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1648147930 con 0x7f16480ad450 2024-05-26T22:52:36.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.880+0000 7f163ea00640 1 -- 172.21.15.131:0/349188148 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+5231 (secure 0 0 0) 0x7f16300591e0 con 0x7f16480ad450 2024-05-26T22:52:36.880 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:52:36.880 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":55,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4832,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/1327386245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":1327386245},{"type":"v1","addr":"172.21.15.187:6835","nonce":1327386245}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-26T22:51:39.114644+0000","modified":"2024-05-26T22:52:14.181234+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6390,"mds_1":6381,"mds_2":6378},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6378":{"gid":6378,"name":"a","rank":2,"incarnation":53,"state":"up:active","state_seq":8,"addr":"172.21.15.131:6836/2737657374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6835","nonce":2737657374},{"type":"v1","addr":"172.21.15.131:6836","nonce":2737657374}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6381":{"gid":6381,"name":"c","rank":1,"incarnation":51,"state":"up:active","state_seq":8,"addr":"172.21.15.131:6837/640544945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":640544945},{"type":"v1","addr":"172.21.15.131:6837","nonce":640544945}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6390":{"gid":6390,"name":"d","rank":0,"incarnation":45,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6833/3300872633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3300872633},{"type":"v1","addr":"172.21.15.187:6833","nonce":3300872633}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6390,"qdb_cluster":[6390,6381,6378]},"id":4}]} 2024-05-26T22:52:36.881 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 55 2024-05-26T22:52:36.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.883+0000 7f164da00640 1 -- 172.21.15.131:0/349188148 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f162406be30 msgr2=0x7f162406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:36.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.883+0000 7f164da00640 1 --2- 172.21.15.131:0/349188148 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f162406be30 0x7f162406e2f0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f1638007fd0 tx=0x7f1638009040 comp rx=0 tx=0).stop 2024-05-26T22:52:36.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.883+0000 7f164da00640 1 -- 172.21.15.131:0/349188148 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f16480ad450 msgr2=0x7f1648099850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:36.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.883+0000 7f164da00640 1 --2- 172.21.15.131:0/349188148 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f16480ad450 0x7f1648099850 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f163000e9e0 tx=0x7f163000eea0 comp rx=0 tx=0).stop 2024-05-26T22:52:36.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.883+0000 7f164da00640 1 -- 172.21.15.131:0/349188148 shutdown_connections 2024-05-26T22:52:36.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.883+0000 7f164da00640 1 --2- 172.21.15.131:0/349188148 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f162406be30 0x7f162406e2f0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:36.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.883+0000 7f164da00640 1 --2- 172.21.15.131:0/349188148 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1648154b10 0x7f1648156040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:36.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.883+0000 7f164da00640 1 --2- 172.21.15.131:0/349188148 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f16480ad450 0x7f1648099850 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:36.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.883+0000 7f164da00640 1 --2- 172.21.15.131:0/349188148 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1648071d80 0x7f1648146070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:36.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.883+0000 7f164da00640 1 -- 172.21.15.131:0/349188148 >> 172.21.15.131:0/349188148 conn(0x7f16480b1e90 msgr2=0x7f1648150df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:52:36.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.883+0000 7f164da00640 1 -- 172.21.15.131:0/349188148 shutdown_connections 2024-05-26T22:52:36.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:36.883+0000 7f164da00640 1 -- 172.21.15.131:0/349188148 wait complete. 2024-05-26T22:52:36.901 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-26T22:52:37.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.099+0000 7f24c2c00640 1 Processor -- start 2024-05-26T22:52:37.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.099+0000 7f24c2c00640 1 -- start start 2024-05-26T22:52:37.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.101+0000 7f24c2c00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f24bc0d3830 0x7f24bc0b7160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:37.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.101+0000 7f24c2c00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24bc15e730 con 0x7f24bc0b5be0 2024-05-26T22:52:37.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.101+0000 7f24c2c00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24bc0cf4f0 con 0x7f24bc0cef10 2024-05-26T22:52:37.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.101+0000 7f24c2c00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f24bc0d2d50 con 0x7f24bc0d3830 2024-05-26T22:52:37.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.101+0000 7f24c1800640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f24bc0b5be0 0x7f24bc0b5fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:52064/0 (socket says 172.21.15.131:52064) 2024-05-26T22:52:37.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.101+0000 7f24c1800640 1 -- 172.21.15.131:0/3796995508 learned_addr learned my addr 172.21.15.131:0/3796995508 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:52:37.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.101+0000 7f24c2200640 1 --2- 172.21.15.131:0/3796995508 >> v2:172.21.15.187:3300/0 conn(0x7f24bc0d3830 0x7f24bc0b7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:37.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.101+0000 7f24bbe00640 1 -- 172.21.15.131:0/3796995508 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 504347310 0 0) 0x7f24bc15e730 con 0x7f24bc0b5be0 2024-05-26T22:52:37.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.101+0000 7f24bbe00640 1 -- 172.21.15.131:0/3796995508 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f24a4003520 con 0x7f24bc0b5be0 2024-05-26T22:52:37.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.102+0000 7f24bbe00640 1 -- 172.21.15.131:0/3796995508 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3421574407 0 0) 0x7f24a4003520 con 0x7f24bc0b5be0 2024-05-26T22:52:37.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.102+0000 7f24bbe00640 1 -- 172.21.15.131:0/3796995508 >> v2:172.21.15.187:3300/0 conn(0x7f24bc0d3830 msgr2=0x7f24bc0b7160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:37.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.102+0000 7f24bbe00640 1 --2- 172.21.15.131:0/3796995508 >> v2:172.21.15.187:3300/0 conn(0x7f24bc0d3830 0x7f24bc0b7160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:37.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.102+0000 7f24bbe00640 1 -- 172.21.15.131:0/3796995508 >> v1:172.21.15.187:6789/0 conn(0x7f24bc0cef10 legacy=0x7f24bc0b9ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:37.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.102+0000 7f24bbe00640 1 -- 172.21.15.131:0/3796995508 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24bc0d3030 con 0x7f24bc0b5be0 2024-05-26T22:52:37.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.102+0000 7f24bbe00640 1 -- 172.21.15.131:0/3796995508 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7f24b00019f0 con 0x7f24bc0b5be0 2024-05-26T22:52:37.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.102+0000 7f24bbe00640 1 -- 172.21.15.131:0/3796995508 >> v1:172.21.15.131:6789/0 conn(0x7f24bc0b5be0 legacy=0x7f24bc0b5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:37.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.102+0000 7f24bbe00640 1 --2- 172.21.15.131:0/3796995508 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f24a40039b0 0x7f24a4003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:37.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.102+0000 7f24bbe00640 1 --2- 172.21.15.131:0/3796995508 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f24a4004800 0x7f24a4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:37.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.102+0000 7f24bbe00640 1 --2- 172.21.15.131:0/3796995508 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f24a40051c0 0x7f24a400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:37.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.102+0000 7f24bbe00640 1 -- 172.21.15.131:0/3796995508 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f24bc0d2d50 con 0x7f24a4004800 2024-05-26T22:52:37.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.102+0000 7f24bbe00640 1 -- 172.21.15.131:0/3796995508 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f24a400c0d0 con 0x7f24a40051c0 2024-05-26T22:52:37.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.102+0000 7f24bbe00640 1 -- 172.21.15.131:0/3796995508 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f24a400c2d0 con 0x7f24a40039b0 2024-05-26T22:52:37.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.102+0000 7f24c0e00640 1 --2- 172.21.15.131:0/3796995508 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f24a4004800 0x7f24a4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:37.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.102+0000 7f24c2200640 1 --2- 172.21.15.131:0/3796995508 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f24a40051c0 0x7f24a400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:37.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.102+0000 7f24c1800640 1 --2- 172.21.15.131:0/3796995508 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f24a40039b0 0x7f24a4003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:37.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.103+0000 7f24c0e00640 1 -- 172.21.15.131:0/3796995508 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f24a40039b0 msgr2=0x7f24a4003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:37.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.103+0000 7f24c0e00640 1 --2- 172.21.15.131:0/3796995508 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f24a40039b0 0x7f24a4003db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:37.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.103+0000 7f24c0e00640 1 -- 172.21.15.131:0/3796995508 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f24a40051c0 msgr2=0x7f24a400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:37.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.103+0000 7f24c0e00640 1 --2- 172.21.15.131:0/3796995508 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f24a40051c0 0x7f24a400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:37.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.103+0000 7f24c0e00640 1 -- 172.21.15.131:0/3796995508 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24a4003520 con 0x7f24a4004800 2024-05-26T22:52:37.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.103+0000 7f24c2200640 1 -- 172.21.15.131:0/3796995508 reap_dead start 2024-05-26T22:52:37.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.104+0000 7f24c0e00640 1 --2- 172.21.15.131:0/3796995508 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f24a4004800 0x7f24a4004c80 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f24a8003db0 tx=0x7f24a4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a22fa7b0c4987e12 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:37.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.104+0000 7f24bbe00640 1 -- 172.21.15.131:0/3796995508 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a8037070 con 0x7f24a4004800 2024-05-26T22:52:37.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.104+0000 7f24bbe00640 1 -- 172.21.15.131:0/3796995508 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24a802a9f0 con 0x7f24a4004800 2024-05-26T22:52:37.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.104+0000 7f24bbe00640 1 -- 172.21.15.131:0/3796995508 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a8003980 con 0x7f24a4004800 2024-05-26T22:52:37.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.105+0000 7f24c2c00640 1 -- 172.21.15.131:0/3796995508 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f24a4004800 msgr2=0x7f24a4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:37.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.105+0000 7f24c2c00640 1 --2- 172.21.15.131:0/3796995508 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f24a4004800 0x7f24a4004c80 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f24a8003db0 tx=0x7f24a4005830 comp rx=0 tx=0).stop 2024-05-26T22:52:37.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.105+0000 7f24c2c00640 1 -- 172.21.15.131:0/3796995508 shutdown_connections 2024-05-26T22:52:37.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.105+0000 7f24c2c00640 1 --2- 172.21.15.131:0/3796995508 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f24a4004800 0x7f24a4004c80 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:37.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.105+0000 7f24c2c00640 1 -- 172.21.15.131:0/3796995508 >> 172.21.15.131:0/3796995508 conn(0x7f24bc073070 msgr2=0x7f24bc0d4160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:52:37.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.105+0000 7f24c2c00640 1 -- 172.21.15.131:0/3796995508 shutdown_connections 2024-05-26T22:52:37.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.105+0000 7f24c2c00640 1 -- 172.21.15.131:0/3796995508 wait complete. 2024-05-26T22:52:37.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.106+0000 7f24c2c00640 1 Processor -- start 2024-05-26T22:52:37.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.107+0000 7f24c2c00640 1 -- start start 2024-05-26T22:52:37.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.107+0000 7f24c2c00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f24a4004800 0x7f24bc161310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:37.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.107+0000 7f24c2c00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f24bc1f46b0 0x7f24bc156d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:37.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.107+0000 7f24c2c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f24bc15f020 0x7f24bc15f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:37.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.107+0000 7f24c2c00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f24bc0d3e20 con 0x7f24bc15f020 2024-05-26T22:52:37.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.107+0000 7f24c2c00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f24bc0d3ca0 con 0x7f24bc1f46b0 2024-05-26T22:52:37.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.107+0000 7f24c2c00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f24bc0d3fa0 con 0x7f24a4004800 2024-05-26T22:52:37.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.108+0000 7f24c2200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f24bc15f020 0x7f24bc15f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:37.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.108+0000 7f24c2200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f24bc15f020 0x7f24bc15f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49776/0 (socket says 172.21.15.131:49776) 2024-05-26T22:52:37.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.108+0000 7f24c2200640 1 -- 172.21.15.131:0/3735118917 learned_addr learned my addr 172.21.15.131:0/3735118917 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:52:37.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.108+0000 7f24c1800640 1 --2- 172.21.15.131:0/3735118917 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f24a4004800 0x7f24bc161310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:37.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.108+0000 7f24c2200640 1 -- 172.21.15.131:0/3735118917 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f24a4004800 msgr2=0x7f24bc161310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:37.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.108+0000 7f24c2200640 1 --2- 172.21.15.131:0/3735118917 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f24a4004800 0x7f24bc161310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:37.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.108+0000 7f24c2200640 1 -- 172.21.15.131:0/3735118917 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f24bc1f46b0 msgr2=0x7f24bc156d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:37.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.108+0000 7f24c2200640 1 --2- 172.21.15.131:0/3735118917 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f24bc1f46b0 0x7f24bc156d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:37.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.108+0000 7f24c2200640 1 -- 172.21.15.131:0/3735118917 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24bc1f4b30 con 0x7f24bc15f020 2024-05-26T22:52:37.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.108+0000 7f24c2200640 1 --2- 172.21.15.131:0/3735118917 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f24bc15f020 0x7f24bc15f4a0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f24b4006fd0 tx=0x7f24b400f8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:37.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.109+0000 7f24b9600640 1 -- 172.21.15.131:0/3735118917 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b4004440 con 0x7f24bc15f020 2024-05-26T22:52:37.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.109+0000 7f24b9600640 1 -- 172.21.15.131:0/3735118917 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24b4019070 con 0x7f24bc15f020 2024-05-26T22:52:37.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.109+0000 7f24b9600640 1 -- 172.21.15.131:0/3735118917 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b4003850 con 0x7f24bc15f020 2024-05-26T22:52:37.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.109+0000 7f24c2c00640 1 -- 172.21.15.131:0/3735118917 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24bc0ca6a0 con 0x7f24bc15f020 2024-05-26T22:52:37.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.109+0000 7f24c2c00640 1 -- 172.21.15.131:0/3735118917 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24bc1f5140 con 0x7f24bc15f020 2024-05-26T22:52:37.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.110+0000 7f24c2c00640 1 -- 172.21.15.131:0/3735118917 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24bc0694e0 con 0x7f24bc15f020 2024-05-26T22:52:37.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.111+0000 7f24b9600640 1 -- 172.21.15.131:0/3735118917 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f24b4012070 con 0x7f24bc15f020 2024-05-26T22:52:37.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.112+0000 7f24b9600640 1 --2- 172.21.15.131:0/3735118917 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f249c06be30 0x7f249c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:37.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.112+0000 7f24b9600640 1 -- 172.21.15.131:0/3735118917 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f24b408a8a0 con 0x7f24bc15f020 2024-05-26T22:52:37.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.112+0000 7f24c1800640 1 --2- 172.21.15.131:0/3735118917 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f249c06be30 0x7f249c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:37.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.113+0000 7f24c1800640 1 --2- 172.21.15.131:0/3735118917 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f249c06be30 0x7f249c06e2f0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f24b00013d0 tx=0x7f24b0001510 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:37.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.117+0000 7f24b9600640 1 -- 172.21.15.131:0/3735118917 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f24b4059240 con 0x7f24bc15f020 2024-05-26T22:52:37.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.299+0000 7f24c2c00640 1 -- 172.21.15.131:0/3735118917 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f24bc0b38a0 con 0x7f24bc15f020 2024-05-26T22:52:37.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.301+0000 7f24b9600640 1 -- 172.21.15.131:0/3735118917 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) ==== 74+0+16689 (secure 0 0 0) 0x7f24b4058b40 con 0x7f24bc15f020 2024-05-26T22:52:37.301 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:52:37.302 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":58,"fsid":"46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f","created":"2024-05-26T22:47:57.288905+0000","modified":"2024-05-26T22:51:40.663750+0000","last_up_change":"2024-05-26T22:48:09.602207+0000","last_in_change":"2024-05-26T22:48:00.687341+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-26T22:48:10.463339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-05-26T22:51:36.873056+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-05-26T22:51:38.120229+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"814eaf62-a18d-494b-8e27-fd8fb88adea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6801","nonce":2350940051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6803","nonce":2350940051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6807","nonce":2350940051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6805","nonce":2350940051}]},"public_addr":"172.21.15.131:6801/2350940051","cluster_addr":"172.21.15.131:6803/2350940051","heartbeat_back_addr":"172.21.15.131:6807/2350940051","heartbeat_front_addr":"172.21.15.131:6805/2350940051","state":["exists","up"]},{"osd":1,"uuid":"42e66287-f6c6-4e26-af3c-6dfaa3536d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6825","nonce":1525528247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6827","nonce":1525528247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6831","nonce":1525528247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6829","nonce":1525528247}]},"public_addr":"172.21.15.131:6825/1525528247","cluster_addr":"172.21.15.131:6827/1525528247","heartbeat_back_addr":"172.21.15.131:6831/1525528247","heartbeat_front_addr":"172.21.15.131:6829/1525528247","state":["exists","up"]},{"osd":2,"uuid":"0da34673-04a2-4fd0-b57a-0ff2b679b043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6817","nonce":3627842210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6819","nonce":3627842210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6823","nonce":3627842210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6821","nonce":3627842210}]},"public_addr":"172.21.15.131:6817/3627842210","cluster_addr":"172.21.15.131:6819/3627842210","heartbeat_back_addr":"172.21.15.131:6823/3627842210","heartbeat_front_addr":"172.21.15.131:6821/3627842210","state":["exists","up"]},{"osd":3,"uuid":"cf117a27-d6b2-43d3-ae52-b47988bf8eed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6809","nonce":350790591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6811","nonce":350790591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6815","nonce":350790591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6813","nonce":350790591}]},"public_addr":"172.21.15.131:6809/350790591","cluster_addr":"172.21.15.131:6811/350790591","heartbeat_back_addr":"172.21.15.131:6815/350790591","heartbeat_front_addr":"172.21.15.131:6813/350790591","state":["exists","up"]},{"osd":4,"uuid":"a092ccfc-be89-4a84-88d5-0f781fa715eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6816","nonce":244560995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6819","nonce":244560995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6823","nonce":244560995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6821","nonce":244560995}]},"public_addr":"172.21.15.187:6816/244560995","cluster_addr":"172.21.15.187:6819/244560995","heartbeat_back_addr":"172.21.15.187:6823/244560995","heartbeat_front_addr":"172.21.15.187:6821/244560995","state":["exists","up"]},{"osd":5,"uuid":"7513ba69-dd00-4a35-b190-9e8cc3b2bdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6801","nonce":324619398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6803","nonce":324619398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6812","nonce":324619398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6807","nonce":324619398}]},"public_addr":"172.21.15.187:6801/324619398","cluster_addr":"172.21.15.187:6803/324619398","heartbeat_back_addr":"172.21.15.187:6812/324619398","heartbeat_front_addr":"172.21.15.187:6807/324619398","state":["exists","up"]},{"osd":6,"uuid":"50081810-c093-4d2f-a4d7-b7c695c5400d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6806","nonce":1391347786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6809","nonce":1391347786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6818","nonce":1391347786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6813","nonce":1391347786}]},"public_addr":"172.21.15.187:6806/1391347786","cluster_addr":"172.21.15.187:6809/1391347786","heartbeat_back_addr":"172.21.15.187:6818/1391347786","heartbeat_front_addr":"172.21.15.187:6813/1391347786","state":["exists","up"]},{"osd":7,"uuid":"775dae05-54f5-4ad4-bbce-92863c6ed2ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6825","nonce":1220065622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6827","nonce":1220065622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6831","nonce":1220065622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6829","nonce":1220065622}]},"public_addr":"172.21.15.187:6825/1220065622","cluster_addr":"172.21.15.187:6827/1220065622","heartbeat_back_addr":"172.21.15.187:6831/1220065622","heartbeat_front_addr":"172.21.15.187:6829/1220065622","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:05.983520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.299516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.067323+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.307748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.018742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.258775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.010912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.295355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-26T22:52:37.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.303+0000 7f24c2c00640 1 -- 172.21.15.131:0/3735118917 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f249c06be30 msgr2=0x7f249c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:37.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.304+0000 7f24c2c00640 1 --2- 172.21.15.131:0/3735118917 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f249c06be30 0x7f249c06e2f0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f24b00013d0 tx=0x7f24b0001510 comp rx=0 tx=0).stop 2024-05-26T22:52:37.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.304+0000 7f24c2c00640 1 -- 172.21.15.131:0/3735118917 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f24bc15f020 msgr2=0x7f24bc15f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:37.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.304+0000 7f24c2c00640 1 --2- 172.21.15.131:0/3735118917 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f24bc15f020 0x7f24bc15f4a0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f24b4006fd0 tx=0x7f24b400f8a0 comp rx=0 tx=0).stop 2024-05-26T22:52:37.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.304+0000 7f24c2c00640 1 -- 172.21.15.131:0/3735118917 shutdown_connections 2024-05-26T22:52:37.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.304+0000 7f24c2c00640 1 --2- 172.21.15.131:0/3735118917 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f249c06be30 0x7f249c06e2f0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:37.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.304+0000 7f24c2c00640 1 --2- 172.21.15.131:0/3735118917 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f24bc15f020 0x7f24bc15f4a0 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:37.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.304+0000 7f24c2c00640 1 --2- 172.21.15.131:0/3735118917 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f24bc1f46b0 0x7f24bc156d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:37.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.304+0000 7f24c2c00640 1 --2- 172.21.15.131:0/3735118917 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f24a4004800 0x7f24bc161310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:37.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.304+0000 7f24c2c00640 1 -- 172.21.15.131:0/3735118917 >> 172.21.15.131:0/3735118917 conn(0x7f24bc073070 msgr2=0x7f24bc0d0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:52:37.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.304+0000 7f24c2c00640 1 -- 172.21.15.131:0/3735118917 shutdown_connections 2024-05-26T22:52:37.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.304+0000 7f24c2c00640 1 -- 172.21.15.131:0/3735118917 wait complete. 2024-05-26T22:52:37.321 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-26T22:52:37.321 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-26T22:52:37.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.521+0000 7fe572400640 1 Processor -- start 2024-05-26T22:52:37.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.521+0000 7fe572400640 1 -- start start 2024-05-26T22:52:37.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.522+0000 7fe572400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fe56c158c00 0x7fe56c06d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:37.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.522+0000 7fe572400640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe56c0b3e50 con 0x7fe56c06daf0 2024-05-26T22:52:37.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.522+0000 7fe572400640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe56c0b4130 con 0x7fe56c0b38a0 2024-05-26T22:52:37.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.522+0000 7fe572400640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fe56c06df50 con 0x7fe56c158c00 2024-05-26T22:52:37.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.523+0000 7fe563e00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fe56c06daf0 0x7fe56c0b3190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:52080/0 (socket says 172.21.15.131:52080) 2024-05-26T22:52:37.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.523+0000 7fe563e00640 1 -- 172.21.15.131:0/1172041599 learned_addr learned my addr 172.21.15.131:0/1172041599 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:52:37.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.523+0000 7fe56b400640 1 --2- 172.21.15.131:0/1172041599 >> v2:172.21.15.131:3300/0 conn(0x7fe56c158c00 0x7fe56c06d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:37.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.523+0000 7fe56aa00640 1 -- 172.21.15.131:0/1172041599 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2159331234 0 0) 0x7fe56c0b3e50 con 0x7fe56c06daf0 2024-05-26T22:52:37.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.523+0000 7fe56aa00640 1 -- 172.21.15.131:0/1172041599 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe550003520 con 0x7fe56c06daf0 2024-05-26T22:52:37.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.523+0000 7fe56b400640 1 -- 172.21.15.131:0/1172041599 >> v1:172.21.15.187:6789/0 conn(0x7fe56c0b38a0 legacy=0x7fe56c15fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:37.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.523+0000 7fe56b400640 1 -- 172.21.15.131:0/1172041599 >> v1:172.21.15.131:6789/0 conn(0x7fe56c06daf0 legacy=0x7fe56c0b3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:37.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.523+0000 7fe56b400640 1 -- 172.21.15.131:0/1172041599 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe56c0b4410 con 0x7fe56c158c00 2024-05-26T22:52:37.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.525+0000 7fe56b400640 1 --2- 172.21.15.131:0/1172041599 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe56c158c00 0x7fe56c06d5b0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fe55c0097a0 tx=0x7fe55c02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27c6f6314f7ee943 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:37.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.525+0000 7fe56aa00640 1 -- 172.21.15.131:0/1172041599 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe55c037070 con 0x7fe56c158c00 2024-05-26T22:52:37.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.525+0000 7fe56aa00640 1 -- 172.21.15.131:0/1172041599 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe55c02ae50 con 0x7fe56c158c00 2024-05-26T22:52:37.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.525+0000 7fe56aa00640 1 -- 172.21.15.131:0/1172041599 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe55c032640 con 0x7fe56c158c00 2024-05-26T22:52:37.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.525+0000 7fe572400640 1 -- 172.21.15.131:0/1172041599 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe56c158c00 msgr2=0x7fe56c06d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:37.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.525+0000 7fe572400640 1 --2- 172.21.15.131:0/1172041599 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe56c158c00 0x7fe56c06d5b0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fe55c0097a0 tx=0x7fe55c02a140 comp rx=0 tx=0).stop 2024-05-26T22:52:37.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.526+0000 7fe572400640 1 -- 172.21.15.131:0/1172041599 shutdown_connections 2024-05-26T22:52:37.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.526+0000 7fe572400640 1 --2- 172.21.15.131:0/1172041599 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe56c158c00 0x7fe56c06d5b0 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:37.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.526+0000 7fe572400640 1 -- 172.21.15.131:0/1172041599 >> 172.21.15.131:0/1172041599 conn(0x7fe56c0b1e90 msgr2=0x7fe56c0b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:52:37.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.526+0000 7fe572400640 1 -- 172.21.15.131:0/1172041599 shutdown_connections 2024-05-26T22:52:37.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.526+0000 7fe572400640 1 -- 172.21.15.131:0/1172041599 wait complete. 2024-05-26T22:52:37.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.527+0000 7fe572400640 1 Processor -- start 2024-05-26T22:52:37.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.528+0000 7fe572400640 1 -- start start 2024-05-26T22:52:37.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.528+0000 7fe572400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe56c06daf0 0x7fe56c09b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:37.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.529+0000 7fe572400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe56c0b38a0 0x7fe56c099a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:37.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.529+0000 7fe572400640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe56c158c00 0x7fe56c15bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:37.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.529+0000 7fe572400640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe56c160360 con 0x7fe56c06daf0 2024-05-26T22:52:37.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.529+0000 7fe572400640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe56c0b49d0 con 0x7fe56c158c00 2024-05-26T22:52:37.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.529+0000 7fe572400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe56c1604e0 con 0x7fe56c0b38a0 2024-05-26T22:52:37.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.529+0000 7fe56b400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe56c06daf0 0x7fe56c09b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:37.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.529+0000 7fe56b400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe56c06daf0 0x7fe56c09b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49798/0 (socket says 172.21.15.131:49798) 2024-05-26T22:52:37.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.529+0000 7fe56b400640 1 -- 172.21.15.131:0/4232543868 learned_addr learned my addr 172.21.15.131:0/4232543868 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:52:37.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.529+0000 7fe56b400640 1 -- 172.21.15.131:0/4232543868 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe56c0b38a0 msgr2=0x7fe56c099a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:37.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.529+0000 7fe56b400640 1 --2- 172.21.15.131:0/4232543868 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe56c0b38a0 0x7fe56c099a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:37.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.529+0000 7fe56b400640 1 -- 172.21.15.131:0/4232543868 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe56c158c00 msgr2=0x7fe56c15bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:37.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.529+0000 7fe56b400640 1 --2- 172.21.15.131:0/4232543868 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe56c158c00 0x7fe56c15bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:37.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.529+0000 7fe56b400640 1 -- 172.21.15.131:0/4232543868 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe56c0af920 con 0x7fe56c06daf0 2024-05-26T22:52:37.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.529+0000 7fe56b400640 1 --2- 172.21.15.131:0/4232543868 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe56c06daf0 0x7fe56c09b9f0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fe55c02a180 tx=0x7fe55c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:37.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.530+0000 7fe563400640 1 -- 172.21.15.131:0/4232543868 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe55c041070 con 0x7fe56c06daf0 2024-05-26T22:52:37.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.530+0000 7fe563400640 1 -- 172.21.15.131:0/4232543868 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe55c03c670 con 0x7fe56c06daf0 2024-05-26T22:52:37.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.530+0000 7fe563400640 1 -- 172.21.15.131:0/4232543868 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe55c037040 con 0x7fe56c06daf0 2024-05-26T22:52:37.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.530+0000 7fe572400640 1 -- 172.21.15.131:0/4232543868 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe56c0af640 con 0x7fe56c06daf0 2024-05-26T22:52:37.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.530+0000 7fe572400640 1 -- 172.21.15.131:0/4232543868 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe56c159e90 con 0x7fe56c06daf0 2024-05-26T22:52:37.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.531+0000 7fe572400640 1 -- 172.21.15.131:0/4232543868 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe56c0694e0 con 0x7fe56c06daf0 2024-05-26T22:52:37.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.532+0000 7fe563400640 1 -- 172.21.15.131:0/4232543868 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe55c044070 con 0x7fe56c06daf0 2024-05-26T22:52:37.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.533+0000 7fe563400640 1 --2- 172.21.15.131:0/4232543868 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fe54806be30 0x7fe54806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:37.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.533+0000 7fe563400640 1 -- 172.21.15.131:0/4232543868 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fe55c0ab520 con 0x7fe56c06daf0 2024-05-26T22:52:37.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.533+0000 7fe563e00640 1 --2- 172.21.15.131:0/4232543868 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fe54806be30 0x7fe54806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:37.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.534+0000 7fe563e00640 1 --2- 172.21.15.131:0/4232543868 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fe54806be30 0x7fe54806e2f0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fe554000f60 tx=0x7fe5540078f0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:37.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.537+0000 7fe563400640 1 -- 172.21.15.131:0/4232543868 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fe55c079ec0 con 0x7fe56c06daf0 2024-05-26T22:52:37.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.775+0000 7fe572400640 1 -- 172.21.15.131:0/4232543868 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe56c15b950 con 0x7fe56c06daf0 2024-05-26T22:52:37.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.776+0000 7fe563400640 1 -- 172.21.15.131:0/4232543868 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+5231 (secure 0 0 0) 0x7fe55c0797c0 con 0x7fe56c06daf0 2024-05-26T22:52:37.776 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:52:37.776 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":55,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4832,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/1327386245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":1327386245},{"type":"v1","addr":"172.21.15.187:6835","nonce":1327386245}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-26T22:51:39.114644+0000","modified":"2024-05-26T22:52:14.181234+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6390,"mds_1":6381,"mds_2":6378},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6378":{"gid":6378,"name":"a","rank":2,"incarnation":53,"state":"up:active","state_seq":8,"addr":"172.21.15.131:6836/2737657374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6835","nonce":2737657374},{"type":"v1","addr":"172.21.15.131:6836","nonce":2737657374}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6381":{"gid":6381,"name":"c","rank":1,"incarnation":51,"state":"up:active","state_seq":8,"addr":"172.21.15.131:6837/640544945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":640544945},{"type":"v1","addr":"172.21.15.131:6837","nonce":640544945}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6390":{"gid":6390,"name":"d","rank":0,"incarnation":45,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6833/3300872633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3300872633},{"type":"v1","addr":"172.21.15.187:6833","nonce":3300872633}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6390,"qdb_cluster":[6390,6381,6378]},"id":4}]} 2024-05-26T22:52:37.776 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 55 2024-05-26T22:52:37.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.779+0000 7fe572400640 1 -- 172.21.15.131:0/4232543868 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fe54806be30 msgr2=0x7fe54806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:37.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.779+0000 7fe572400640 1 --2- 172.21.15.131:0/4232543868 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fe54806be30 0x7fe54806e2f0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fe554000f60 tx=0x7fe5540078f0 comp rx=0 tx=0).stop 2024-05-26T22:52:37.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.779+0000 7fe572400640 1 -- 172.21.15.131:0/4232543868 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe56c06daf0 msgr2=0x7fe56c09b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:37.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.779+0000 7fe572400640 1 --2- 172.21.15.131:0/4232543868 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe56c06daf0 0x7fe56c09b9f0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fe55c02a180 tx=0x7fe55c0046c0 comp rx=0 tx=0).stop 2024-05-26T22:52:37.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.779+0000 7fe572400640 1 -- 172.21.15.131:0/4232543868 shutdown_connections 2024-05-26T22:52:37.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.779+0000 7fe572400640 1 --2- 172.21.15.131:0/4232543868 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe56c0b38a0 0x7fe56c099a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:37.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.779+0000 7fe572400640 1 --2- 172.21.15.131:0/4232543868 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fe54806be30 0x7fe54806e2f0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:37.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.779+0000 7fe572400640 1 --2- 172.21.15.131:0/4232543868 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe56c158c00 0x7fe56c15bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:37.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.779+0000 7fe572400640 1 --2- 172.21.15.131:0/4232543868 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe56c06daf0 0x7fe56c09b9f0 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:37.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.779+0000 7fe572400640 1 -- 172.21.15.131:0/4232543868 >> 172.21.15.131:0/4232543868 conn(0x7fe56c0b1e90 msgr2=0x7fe56c15e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:52:37.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.779+0000 7fe572400640 1 -- 172.21.15.131:0/4232543868 shutdown_connections 2024-05-26T22:52:37.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.779+0000 7fe572400640 1 -- 172.21.15.131:0/4232543868 wait complete. 2024-05-26T22:52:37.798 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-26T22:52:37.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.997+0000 7fd16a600640 1 Processor -- start 2024-05-26T22:52:37.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.997+0000 7fd16a600640 1 -- start start 2024-05-26T22:52:37.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.999+0000 7fd16a600640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fd16406e8c0 0x7fd16415d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:37.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.999+0000 7fd16a600640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd1640cc7f0 0x7fd1640ccca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:37.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.999+0000 7fd16a600640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd164160620 con 0x7fd164156750 2024-05-26T22:52:37.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.999+0000 7fd16a600640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fd16415cbd0 con 0x7fd16406e8c0 2024-05-26T22:52:37.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.999+0000 7fd16a600640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd164160900 con 0x7fd1640cc7f0 2024-05-26T22:52:37.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.999+0000 7fd169200640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fd164156750 0x7fd16415fd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:52092/0 (socket says 172.21.15.131:52092) 2024-05-26T22:52:37.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.999+0000 7fd169200640 1 -- 172.21.15.131:0/20478629 learned_addr learned my addr 172.21.15.131:0/20478629 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:52:38.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.999+0000 7fd169c00640 1 --2- 172.21.15.131:0/20478629 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd1640cc7f0 0x7fd1640ccca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:38.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:37.999+0000 7fd163e00640 1 --2- 172.21.15.131:0/20478629 >> v2:172.21.15.187:3300/0 conn(0x7fd16406e8c0 0x7fd16415d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:38.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.000+0000 7fd163400640 1 -- 172.21.15.131:0/20478629 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 387977523 0 0) 0x7fd164160620 con 0x7fd164156750 2024-05-26T22:52:38.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.000+0000 7fd163400640 1 -- 172.21.15.131:0/20478629 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd148003520 con 0x7fd164156750 2024-05-26T22:52:38.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.000+0000 7fd163400640 1 -- 172.21.15.131:0/20478629 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1670179661 0 0) 0x7fd148003520 con 0x7fd164156750 2024-05-26T22:52:38.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.000+0000 7fd163400640 1 -- 172.21.15.131:0/20478629 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd1640cc7f0 msgr2=0x7fd1640ccca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:38.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.000+0000 7fd163400640 1 --2- 172.21.15.131:0/20478629 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd1640cc7f0 0x7fd1640ccca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:38.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.000+0000 7fd163400640 1 -- 172.21.15.131:0/20478629 >> v2:172.21.15.187:3300/0 conn(0x7fd16406e8c0 msgr2=0x7fd16415d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:38.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.000+0000 7fd163400640 1 --2- 172.21.15.131:0/20478629 >> v2:172.21.15.187:3300/0 conn(0x7fd16406e8c0 0x7fd16415d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:38.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.000+0000 7fd163400640 1 -- 172.21.15.131:0/20478629 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd164160be0 con 0x7fd164156750 2024-05-26T22:52:38.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.000+0000 7fd163400640 1 -- 172.21.15.131:0/20478629 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7fd15c0019f0 con 0x7fd164156750 2024-05-26T22:52:38.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.000+0000 7fd163400640 1 -- 172.21.15.131:0/20478629 >> v1:172.21.15.131:6789/0 conn(0x7fd164156750 legacy=0x7fd16415fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:38.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.000+0000 7fd163400640 1 --2- 172.21.15.131:0/20478629 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd1480039b0 0x7fd148003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:38.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.000+0000 7fd163400640 1 --2- 172.21.15.131:0/20478629 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd148004800 0x7fd148004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:38.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.000+0000 7fd163400640 1 --2- 172.21.15.131:0/20478629 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd1640cc7f0 0x7fd1480051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:38.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.000+0000 7fd163400640 1 -- 172.21.15.131:0/20478629 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fd16415cbd0 con 0x7fd1480039b0 2024-05-26T22:52:38.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.000+0000 7fd163400640 1 -- 172.21.15.131:0/20478629 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd164160900 con 0x7fd148004800 2024-05-26T22:52:38.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.000+0000 7fd163400640 1 -- 172.21.15.131:0/20478629 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd14800fdd0 con 0x7fd1640cc7f0 2024-05-26T22:52:38.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.001+0000 7fd169200640 1 --2- 172.21.15.131:0/20478629 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd1480039b0 0x7fd148003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:38.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.001+0000 7fd163e00640 1 --2- 172.21.15.131:0/20478629 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd148004800 0x7fd148004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:38.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.001+0000 7fd169c00640 1 --2- 172.21.15.131:0/20478629 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd1640cc7f0 0x7fd1480051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:38.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.001+0000 7fd169200640 1 -- 172.21.15.131:0/20478629 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd1640cc7f0 msgr2=0x7fd1480051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:38.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.001+0000 7fd169200640 1 --2- 172.21.15.131:0/20478629 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd1640cc7f0 0x7fd1480051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:38.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.001+0000 7fd169200640 1 -- 172.21.15.131:0/20478629 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd148004800 msgr2=0x7fd148004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:38.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.001+0000 7fd169200640 1 --2- 172.21.15.131:0/20478629 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd148004800 0x7fd148004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:38.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.001+0000 7fd169200640 1 -- 172.21.15.131:0/20478629 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd148003520 con 0x7fd1480039b0 2024-05-26T22:52:38.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.001+0000 7fd163e00640 1 --2- 172.21.15.131:0/20478629 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd148004800 0x7fd148004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-26T22:52:38.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.002+0000 7fd169200640 1 --2- 172.21.15.131:0/20478629 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd1480039b0 0x7fd148003dd0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fd15c00a7b0 tx=0x7fd148005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4ef9510068ba11f server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:38.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.003+0000 7fd163400640 1 -- 172.21.15.131:0/20478629 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd15c00a3d0 con 0x7fd1480039b0 2024-05-26T22:52:38.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.003+0000 7fd163400640 1 -- 172.21.15.131:0/20478629 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd15c02f070 con 0x7fd1480039b0 2024-05-26T22:52:38.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.003+0000 7fd163400640 1 -- 172.21.15.131:0/20478629 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd15c02bc10 con 0x7fd1480039b0 2024-05-26T22:52:38.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.003+0000 7fd16a600640 1 -- 172.21.15.131:0/20478629 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd1480039b0 msgr2=0x7fd148003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:38.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.003+0000 7fd16a600640 1 --2- 172.21.15.131:0/20478629 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd1480039b0 0x7fd148003dd0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fd15c00a7b0 tx=0x7fd148005830 comp rx=0 tx=0).stop 2024-05-26T22:52:38.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.003+0000 7fd169c00640 1 -- 172.21.15.131:0/20478629 reap_dead start 2024-05-26T22:52:38.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.003+0000 7fd16a600640 1 -- 172.21.15.131:0/20478629 shutdown_connections 2024-05-26T22:52:38.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.003+0000 7fd16a600640 1 -- 172.21.15.131:0/20478629 >> 172.21.15.131:0/20478629 conn(0x7fd1640cff40 msgr2=0x7fd1640d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:52:38.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.004+0000 7fd16a600640 1 -- 172.21.15.131:0/20478629 shutdown_connections 2024-05-26T22:52:38.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.004+0000 7fd16a600640 1 -- 172.21.15.131:0/20478629 wait complete. 2024-05-26T22:52:38.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.005+0000 7fd16a600640 1 Processor -- start 2024-05-26T22:52:38.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.006+0000 7fd16a600640 1 -- start start 2024-05-26T22:52:38.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.006+0000 7fd16a600640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd1480039b0 0x7fd164142540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:38.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.006+0000 7fd16a600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd1640b26b0 0x7fd164158650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:38.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.006+0000 7fd16a600640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd164156cb0 0x7fd164144090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:38.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.006+0000 7fd16a600640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fd16415d5a0 con 0x7fd1640b26b0 2024-05-26T22:52:38.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.006+0000 7fd16a600640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd164156b30 con 0x7fd164156cb0 2024-05-26T22:52:38.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.006+0000 7fd16a600640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd16415d720 con 0x7fd1480039b0 2024-05-26T22:52:38.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.007+0000 7fd163e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd1640b26b0 0x7fd164158650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:38.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.007+0000 7fd163e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd1640b26b0 0x7fd164158650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49814/0 (socket says 172.21.15.131:49814) 2024-05-26T22:52:38.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.007+0000 7fd163e00640 1 -- 172.21.15.131:0/661537134 learned_addr learned my addr 172.21.15.131:0/661537134 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:52:38.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.007+0000 7fd169c00640 1 --2- 172.21.15.131:0/661537134 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd164156cb0 0x7fd164144090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:38.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.007+0000 7fd169200640 1 --2- 172.21.15.131:0/661537134 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd1480039b0 0x7fd164142540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:38.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.007+0000 7fd163e00640 1 -- 172.21.15.131:0/661537134 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd1480039b0 msgr2=0x7fd164142540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:38.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.007+0000 7fd163e00640 1 --2- 172.21.15.131:0/661537134 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd1480039b0 0x7fd164142540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:38.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.007+0000 7fd163e00640 1 -- 172.21.15.131:0/661537134 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd164156cb0 msgr2=0x7fd164144090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:38.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.007+0000 7fd163e00640 1 --2- 172.21.15.131:0/661537134 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd164156cb0 0x7fd164144090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:38.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.007+0000 7fd163e00640 1 -- 172.21.15.131:0/661537134 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1641575b0 con 0x7fd1640b26b0 2024-05-26T22:52:38.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.007+0000 7fd169c00640 1 --2- 172.21.15.131:0/661537134 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd164156cb0 0x7fd164144090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:52:38.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.007+0000 7fd169200640 1 --2- 172.21.15.131:0/661537134 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd1480039b0 0x7fd164142540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:52:38.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.008+0000 7fd163e00640 1 --2- 172.21.15.131:0/661537134 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd1640b26b0 0x7fd164158650 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fd14c0115d0 tx=0x7fd14c011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:38.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.008+0000 7fd160c00640 1 -- 172.21.15.131:0/661537134 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd14c00f8f0 con 0x7fd1640b26b0 2024-05-26T22:52:38.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.008+0000 7fd160c00640 1 -- 172.21.15.131:0/661537134 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd14c00fa90 con 0x7fd1640b26b0 2024-05-26T22:52:38.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.008+0000 7fd16a600640 1 -- 172.21.15.131:0/661537134 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd164157890 con 0x7fd1640b26b0 2024-05-26T22:52:38.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.008+0000 7fd160c00640 1 -- 172.21.15.131:0/661537134 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd14c00fe60 con 0x7fd1640b26b0 2024-05-26T22:52:38.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.008+0000 7fd16a600640 1 -- 172.21.15.131:0/661537134 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd164157e50 con 0x7fd1640b26b0 2024-05-26T22:52:38.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.009+0000 7fd16a600640 1 -- 172.21.15.131:0/661537134 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1640694e0 con 0x7fd1640b26b0 2024-05-26T22:52:38.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.010+0000 7fd160c00640 1 -- 172.21.15.131:0/661537134 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd14c015460 con 0x7fd1640b26b0 2024-05-26T22:52:38.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.011+0000 7fd160c00640 1 --2- 172.21.15.131:0/661537134 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fd14006be30 0x7fd14006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:38.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.011+0000 7fd160c00640 1 -- 172.21.15.131:0/661537134 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fd14c08b7b0 con 0x7fd1640b26b0 2024-05-26T22:52:38.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.011+0000 7fd169200640 1 --2- 172.21.15.131:0/661537134 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fd14006be30 0x7fd14006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:38.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.012+0000 7fd169200640 1 --2- 172.21.15.131:0/661537134 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fd14006be30 0x7fd14006e2f0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fd15c033000 tx=0x7fd15c007000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:38.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.016+0000 7fd160c00640 1 -- 172.21.15.131:0/661537134 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fd14c019030 con 0x7fd1640b26b0 2024-05-26T22:52:38.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.252+0000 7fd16a600640 1 -- 172.21.15.131:0/661537134 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd164155ea0 con 0x7fd1640b26b0 2024-05-26T22:52:38.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.254+0000 7fd160c00640 1 -- 172.21.15.131:0/661537134 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+5231 (secure 0 0 0) 0x7fd14c05a0c0 con 0x7fd1640b26b0 2024-05-26T22:52:38.254 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:52:38.254 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":55,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4832,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/1327386245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":1327386245},{"type":"v1","addr":"172.21.15.187:6835","nonce":1327386245}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-26T22:51:39.114644+0000","modified":"2024-05-26T22:52:14.181234+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6390,"mds_1":6381,"mds_2":6378},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6378":{"gid":6378,"name":"a","rank":2,"incarnation":53,"state":"up:active","state_seq":8,"addr":"172.21.15.131:6836/2737657374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6835","nonce":2737657374},{"type":"v1","addr":"172.21.15.131:6836","nonce":2737657374}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6381":{"gid":6381,"name":"c","rank":1,"incarnation":51,"state":"up:active","state_seq":8,"addr":"172.21.15.131:6837/640544945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":640544945},{"type":"v1","addr":"172.21.15.131:6837","nonce":640544945}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6390":{"gid":6390,"name":"d","rank":0,"incarnation":45,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6833/3300872633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3300872633},{"type":"v1","addr":"172.21.15.187:6833","nonce":3300872633}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6390,"qdb_cluster":[6390,6381,6378]},"id":4}]} 2024-05-26T22:52:38.254 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 55 2024-05-26T22:52:38.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.257+0000 7fd16a600640 1 -- 172.21.15.131:0/661537134 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fd14006be30 msgr2=0x7fd14006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:38.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.257+0000 7fd16a600640 1 --2- 172.21.15.131:0/661537134 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fd14006be30 0x7fd14006e2f0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fd15c033000 tx=0x7fd15c007000 comp rx=0 tx=0).stop 2024-05-26T22:52:38.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.257+0000 7fd16a600640 1 -- 172.21.15.131:0/661537134 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd1640b26b0 msgr2=0x7fd164158650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:38.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.257+0000 7fd16a600640 1 --2- 172.21.15.131:0/661537134 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd1640b26b0 0x7fd164158650 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fd14c0115d0 tx=0x7fd14c011a90 comp rx=0 tx=0).stop 2024-05-26T22:52:38.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.257+0000 7fd16a600640 1 -- 172.21.15.131:0/661537134 shutdown_connections 2024-05-26T22:52:38.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.257+0000 7fd16a600640 1 --2- 172.21.15.131:0/661537134 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fd14006be30 0x7fd14006e2f0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:38.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.257+0000 7fd16a600640 1 --2- 172.21.15.131:0/661537134 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd164156cb0 0x7fd164144090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:38.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.257+0000 7fd16a600640 1 --2- 172.21.15.131:0/661537134 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd1640b26b0 0x7fd164158650 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:38.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.257+0000 7fd16a600640 1 --2- 172.21.15.131:0/661537134 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd1480039b0 0x7fd164142540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:38.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.257+0000 7fd16a600640 1 -- 172.21.15.131:0/661537134 >> 172.21.15.131:0/661537134 conn(0x7fd1640cff40 msgr2=0x7fd1640ce890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:52:38.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.257+0000 7fd16a600640 1 -- 172.21.15.131:0/661537134 shutdown_connections 2024-05-26T22:52:38.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.257+0000 7fd16a600640 1 -- 172.21.15.131:0/661537134 wait complete. 2024-05-26T22:52:38.275 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-26T22:52:38.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.468+0000 7f20db200640 1 Processor -- start 2024-05-26T22:52:38.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.468+0000 7f20db200640 1 -- start start 2024-05-26T22:52:38.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.470+0000 7f20db200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f20d4156750 0x7f20d415fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:38.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.470+0000 7f20db200640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f20d406e8c0 0x7f20d415d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:38.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.470+0000 7f20db200640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20d4160620 con 0x7f20d40cc7f0 2024-05-26T22:52:38.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.470+0000 7f20db200640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f20d415cbd0 con 0x7f20d4156750 2024-05-26T22:52:38.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.470+0000 7f20db200640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f20d4160900 con 0x7f20d406e8c0 2024-05-26T22:52:38.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.470+0000 7f20d9e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f20d4156750 0x7f20d415fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:38.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.470+0000 7f20da800640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f20d40cc7f0 0x7f20d40cdc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:52102/0 (socket says 172.21.15.131:52102) 2024-05-26T22:52:38.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.470+0000 7f20d9e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f20d4156750 0x7f20d415fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49822/0 (socket says 172.21.15.131:49822) 2024-05-26T22:52:38.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.470+0000 7f20da800640 1 -- 172.21.15.131:0/3069901964 learned_addr learned my addr 172.21.15.131:0/3069901964 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:52:38.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.471+0000 7f20d9400640 1 --2- 172.21.15.131:0/3069901964 >> v2:172.21.15.187:3300/0 conn(0x7f20d406e8c0 0x7f20d415d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:38.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.471+0000 7f20d9e00640 1 -- 172.21.15.131:0/3069901964 >> v2:172.21.15.187:3300/0 conn(0x7f20d406e8c0 msgr2=0x7f20d415d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:38.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.471+0000 7f20d9e00640 1 --2- 172.21.15.131:0/3069901964 >> v2:172.21.15.187:3300/0 conn(0x7f20d406e8c0 0x7f20d415d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:38.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.471+0000 7f20d9e00640 1 -- 172.21.15.131:0/3069901964 >> v1:172.21.15.131:6789/0 conn(0x7f20d40cc7f0 legacy=0x7f20d40cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:38.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.471+0000 7f20d9e00640 1 -- 172.21.15.131:0/3069901964 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20d4160be0 con 0x7f20d4156750 2024-05-26T22:52:38.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.472+0000 7f20d9e00640 1 --2- 172.21.15.131:0/3069901964 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f20d4156750 0x7f20d415fd40 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f20d4160e40 tx=0x7f20c802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f80900324dd368b server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:38.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.472+0000 7f20d8a00640 1 -- 172.21.15.131:0/3069901964 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c8037070 con 0x7f20d4156750 2024-05-26T22:52:38.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.472+0000 7f20d8a00640 1 -- 172.21.15.131:0/3069901964 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20c8032440 con 0x7f20d4156750 2024-05-26T22:52:38.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.472+0000 7f20d8a00640 1 -- 172.21.15.131:0/3069901964 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c8032810 con 0x7f20d4156750 2024-05-26T22:52:38.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.473+0000 7f20db200640 1 -- 172.21.15.131:0/3069901964 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f20d4156750 msgr2=0x7f20d415fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:38.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.473+0000 7f20db200640 1 --2- 172.21.15.131:0/3069901964 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f20d4156750 0x7f20d415fd40 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f20d4160e40 tx=0x7f20c802a1a0 comp rx=0 tx=0).stop 2024-05-26T22:52:38.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.473+0000 7f20db200640 1 -- 172.21.15.131:0/3069901964 shutdown_connections 2024-05-26T22:52:38.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.473+0000 7f20db200640 1 --2- 172.21.15.131:0/3069901964 >> v2:172.21.15.187:3300/0 conn(0x7f20d406e8c0 0x7f20d415d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:38.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.473+0000 7f20db200640 1 --2- 172.21.15.131:0/3069901964 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f20d4156750 0x7f20d415fd40 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:38.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.473+0000 7f20db200640 1 -- 172.21.15.131:0/3069901964 >> 172.21.15.131:0/3069901964 conn(0x7f20d40cff40 msgr2=0x7f20d40d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:52:38.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.473+0000 7f20db200640 1 -- 172.21.15.131:0/3069901964 shutdown_connections 2024-05-26T22:52:38.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.473+0000 7f20db200640 1 -- 172.21.15.131:0/3069901964 wait complete. 2024-05-26T22:52:38.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.475+0000 7f20db200640 1 Processor -- start 2024-05-26T22:52:38.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.475+0000 7f20db200640 1 -- start start 2024-05-26T22:52:38.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.476+0000 7f20db200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f20d406e8c0 0x7f20d41f7b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:38.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.476+0000 7f20db200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f20d40cc7f0 0x7f20d414cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:38.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.476+0000 7f20db200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f20d4156750 0x7f20d41f3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:38.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.476+0000 7f20db200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f20d4160280 con 0x7f20d40cc7f0 2024-05-26T22:52:38.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.476+0000 7f20db200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f20d4156b30 con 0x7f20d4156750 2024-05-26T22:52:38.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.476+0000 7f20db200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f20d4160400 con 0x7f20d406e8c0 2024-05-26T22:52:38.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.476+0000 7f20d9400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f20d40cc7f0 0x7f20d414cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:38.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.476+0000 7f20d9400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f20d40cc7f0 0x7f20d414cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49836/0 (socket says 172.21.15.131:49836) 2024-05-26T22:52:38.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.476+0000 7f20d9400640 1 -- 172.21.15.131:0/200286607 learned_addr learned my addr 172.21.15.131:0/200286607 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:52:38.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.476+0000 7f20da800640 1 --2- 172.21.15.131:0/200286607 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f20d4156750 0x7f20d41f3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:38.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.476+0000 7f20d9e00640 1 --2- 172.21.15.131:0/200286607 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f20d406e8c0 0x7f20d41f7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:38.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.476+0000 7f20d9400640 1 -- 172.21.15.131:0/200286607 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f20d406e8c0 msgr2=0x7f20d41f7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:38.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.476+0000 7f20d9400640 1 --2- 172.21.15.131:0/200286607 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f20d406e8c0 0x7f20d41f7b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:38.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.476+0000 7f20d9400640 1 -- 172.21.15.131:0/200286607 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f20d4156750 msgr2=0x7f20d41f3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:38.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.476+0000 7f20d9400640 1 --2- 172.21.15.131:0/200286607 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f20d4156750 0x7f20d41f3c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:38.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.476+0000 7f20d9400640 1 -- 172.21.15.131:0/200286607 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20d41f4660 con 0x7f20d40cc7f0 2024-05-26T22:52:38.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.477+0000 7f20d9400640 1 --2- 172.21.15.131:0/200286607 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f20d40cc7f0 0x7f20d414cff0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f20c400c820 tx=0x7f20c400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:38.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.477+0000 7f20ce000640 1 -- 172.21.15.131:0/200286607 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c4015070 con 0x7f20d40cc7f0 2024-05-26T22:52:38.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.477+0000 7f20ce000640 1 -- 172.21.15.131:0/200286607 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20c4004240 con 0x7f20d40cc7f0 2024-05-26T22:52:38.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.477+0000 7f20db200640 1 -- 172.21.15.131:0/200286607 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20d41f4940 con 0x7f20d40cc7f0 2024-05-26T22:52:38.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.477+0000 7f20ce000640 1 -- 172.21.15.131:0/200286607 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c400ce10 con 0x7f20d40cc7f0 2024-05-26T22:52:38.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.477+0000 7f20db200640 1 -- 172.21.15.131:0/200286607 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20d41f80c0 con 0x7f20d40cc7f0 2024-05-26T22:52:38.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.478+0000 7f20db200640 1 -- 172.21.15.131:0/200286607 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20d40694e0 con 0x7f20d40cc7f0 2024-05-26T22:52:38.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.479+0000 7f20ce000640 1 -- 172.21.15.131:0/200286607 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f20c4007830 con 0x7f20d40cc7f0 2024-05-26T22:52:38.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.481+0000 7f20ce000640 1 --2- 172.21.15.131:0/200286607 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f20b806be30 0x7f20b806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:38.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.481+0000 7f20ce000640 1 -- 172.21.15.131:0/200286607 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f20c408ad60 con 0x7f20d40cc7f0 2024-05-26T22:52:38.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.481+0000 7f20d9e00640 1 --2- 172.21.15.131:0/200286607 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f20b806be30 0x7f20b806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:38.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.482+0000 7f20d9e00640 1 --2- 172.21.15.131:0/200286607 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f20b806be30 0x7f20b806e2f0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f20c8034000 tx=0x7f20c802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:38.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.485+0000 7f20ce000640 1 -- 172.21.15.131:0/200286607 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f20c4059700 con 0x7f20d40cc7f0 2024-05-26T22:52:38.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.673+0000 7f20db200640 1 -- 172.21.15.131:0/200286607 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f20d41f4b10 con 0x7f20d40cc7f0 2024-05-26T22:52:38.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.675+0000 7f20ce000640 1 -- 172.21.15.131:0/200286607 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) ==== 74+0+16689 (secure 0 0 0) 0x7f20c4059000 con 0x7f20d40cc7f0 2024-05-26T22:52:38.675 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:52:38.675 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":58,"fsid":"46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f","created":"2024-05-26T22:47:57.288905+0000","modified":"2024-05-26T22:51:40.663750+0000","last_up_change":"2024-05-26T22:48:09.602207+0000","last_in_change":"2024-05-26T22:48:00.687341+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-26T22:48:10.463339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-05-26T22:51:36.873056+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-05-26T22:51:38.120229+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"814eaf62-a18d-494b-8e27-fd8fb88adea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6801","nonce":2350940051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6803","nonce":2350940051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6807","nonce":2350940051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6805","nonce":2350940051}]},"public_addr":"172.21.15.131:6801/2350940051","cluster_addr":"172.21.15.131:6803/2350940051","heartbeat_back_addr":"172.21.15.131:6807/2350940051","heartbeat_front_addr":"172.21.15.131:6805/2350940051","state":["exists","up"]},{"osd":1,"uuid":"42e66287-f6c6-4e26-af3c-6dfaa3536d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6825","nonce":1525528247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6827","nonce":1525528247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6831","nonce":1525528247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6829","nonce":1525528247}]},"public_addr":"172.21.15.131:6825/1525528247","cluster_addr":"172.21.15.131:6827/1525528247","heartbeat_back_addr":"172.21.15.131:6831/1525528247","heartbeat_front_addr":"172.21.15.131:6829/1525528247","state":["exists","up"]},{"osd":2,"uuid":"0da34673-04a2-4fd0-b57a-0ff2b679b043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6817","nonce":3627842210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6819","nonce":3627842210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6823","nonce":3627842210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6821","nonce":3627842210}]},"public_addr":"172.21.15.131:6817/3627842210","cluster_addr":"172.21.15.131:6819/3627842210","heartbeat_back_addr":"172.21.15.131:6823/3627842210","heartbeat_front_addr":"172.21.15.131:6821/3627842210","state":["exists","up"]},{"osd":3,"uuid":"cf117a27-d6b2-43d3-ae52-b47988bf8eed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6809","nonce":350790591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6811","nonce":350790591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6815","nonce":350790591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6813","nonce":350790591}]},"public_addr":"172.21.15.131:6809/350790591","cluster_addr":"172.21.15.131:6811/350790591","heartbeat_back_addr":"172.21.15.131:6815/350790591","heartbeat_front_addr":"172.21.15.131:6813/350790591","state":["exists","up"]},{"osd":4,"uuid":"a092ccfc-be89-4a84-88d5-0f781fa715eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6816","nonce":244560995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6819","nonce":244560995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6823","nonce":244560995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6821","nonce":244560995}]},"public_addr":"172.21.15.187:6816/244560995","cluster_addr":"172.21.15.187:6819/244560995","heartbeat_back_addr":"172.21.15.187:6823/244560995","heartbeat_front_addr":"172.21.15.187:6821/244560995","state":["exists","up"]},{"osd":5,"uuid":"7513ba69-dd00-4a35-b190-9e8cc3b2bdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6801","nonce":324619398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6803","nonce":324619398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6812","nonce":324619398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6807","nonce":324619398}]},"public_addr":"172.21.15.187:6801/324619398","cluster_addr":"172.21.15.187:6803/324619398","heartbeat_back_addr":"172.21.15.187:6812/324619398","heartbeat_front_addr":"172.21.15.187:6807/324619398","state":["exists","up"]},{"osd":6,"uuid":"50081810-c093-4d2f-a4d7-b7c695c5400d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6806","nonce":1391347786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6809","nonce":1391347786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6818","nonce":1391347786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6813","nonce":1391347786}]},"public_addr":"172.21.15.187:6806/1391347786","cluster_addr":"172.21.15.187:6809/1391347786","heartbeat_back_addr":"172.21.15.187:6818/1391347786","heartbeat_front_addr":"172.21.15.187:6813/1391347786","state":["exists","up"]},{"osd":7,"uuid":"775dae05-54f5-4ad4-bbce-92863c6ed2ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6825","nonce":1220065622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6827","nonce":1220065622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6831","nonce":1220065622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6829","nonce":1220065622}]},"public_addr":"172.21.15.187:6825/1220065622","cluster_addr":"172.21.15.187:6827/1220065622","heartbeat_back_addr":"172.21.15.187:6831/1220065622","heartbeat_front_addr":"172.21.15.187:6829/1220065622","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:05.983520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.299516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.067323+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.307748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.018742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.258775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.010912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.295355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-26T22:52:38.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.678+0000 7f20db200640 1 -- 172.21.15.131:0/200286607 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f20b806be30 msgr2=0x7f20b806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:38.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.678+0000 7f20db200640 1 --2- 172.21.15.131:0/200286607 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f20b806be30 0x7f20b806e2f0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f20c8034000 tx=0x7f20c802e000 comp rx=0 tx=0).stop 2024-05-26T22:52:38.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.678+0000 7f20db200640 1 -- 172.21.15.131:0/200286607 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f20d40cc7f0 msgr2=0x7f20d414cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:38.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.678+0000 7f20db200640 1 --2- 172.21.15.131:0/200286607 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f20d40cc7f0 0x7f20d414cff0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f20c400c820 tx=0x7f20c400cce0 comp rx=0 tx=0).stop 2024-05-26T22:52:38.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.678+0000 7f20db200640 1 -- 172.21.15.131:0/200286607 shutdown_connections 2024-05-26T22:52:38.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.678+0000 7f20db200640 1 --2- 172.21.15.131:0/200286607 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f20b806be30 0x7f20b806e2f0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:38.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.678+0000 7f20db200640 1 --2- 172.21.15.131:0/200286607 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f20d4156750 0x7f20d41f3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:38.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.678+0000 7f20db200640 1 --2- 172.21.15.131:0/200286607 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f20d40cc7f0 0x7f20d414cff0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:38.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.678+0000 7f20db200640 1 --2- 172.21.15.131:0/200286607 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f20d406e8c0 0x7f20d41f7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:38.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.678+0000 7f20db200640 1 -- 172.21.15.131:0/200286607 >> 172.21.15.131:0/200286607 conn(0x7f20d40cff40 msgr2=0x7f20d415f070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:52:38.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.678+0000 7f20db200640 1 -- 172.21.15.131:0/200286607 shutdown_connections 2024-05-26T22:52:38.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.678+0000 7f20db200640 1 -- 172.21.15.131:0/200286607 wait complete. 2024-05-26T22:52:38.696 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-26T22:52:38.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.887+0000 7f786de00640 1 Processor -- start 2024-05-26T22:52:38.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.887+0000 7f786de00640 1 -- start start 2024-05-26T22:52:38.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.889+0000 7f786de00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f7868145a10 0x7f7868072180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:38.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.889+0000 7f786de00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7868072d60 con 0x7f78680712d0 2024-05-26T22:52:38.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.889+0000 7f786de00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7868072a80 con 0x7f786806d960 2024-05-26T22:52:38.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.889+0000 7f786de00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f786815c760 con 0x7f7868145a10 2024-05-26T22:52:38.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.889+0000 7f785fe00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f78680712d0 0x7f786806d250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:52114/0 (socket says 172.21.15.131:52114) 2024-05-26T22:52:38.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.889+0000 7f785fe00640 1 -- 172.21.15.131:0/3024816986 learned_addr learned my addr 172.21.15.131:0/3024816986 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:52:38.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.890+0000 7f786ca00640 1 --2- 172.21.15.131:0/3024816986 >> v2:172.21.15.187:3300/0 conn(0x7f7868145a10 0x7f7868072180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:38.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.890+0000 7f7867e00640 1 -- 172.21.15.131:0/3024816986 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1844323544 0 0) 0x7f7868072d60 con 0x7f78680712d0 2024-05-26T22:52:38.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.890+0000 7f7867e00640 1 -- 172.21.15.131:0/3024816986 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f784c003520 con 0x7f78680712d0 2024-05-26T22:52:38.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.890+0000 7f7867e00640 1 -- 172.21.15.131:0/3024816986 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2298998491 0 0) 0x7f784c003520 con 0x7f78680712d0 2024-05-26T22:52:38.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.890+0000 7f7867e00640 1 -- 172.21.15.131:0/3024816986 >> v2:172.21.15.187:3300/0 conn(0x7f7868145a10 msgr2=0x7f7868072180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:38.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.890+0000 7f7867e00640 1 --2- 172.21.15.131:0/3024816986 >> v2:172.21.15.187:3300/0 conn(0x7f7868145a10 0x7f7868072180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:38.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.890+0000 7f7867e00640 1 -- 172.21.15.131:0/3024816986 >> v1:172.21.15.187:6789/0 conn(0x7f786806d960 legacy=0x7f78680cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:38.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.890+0000 7f7867e00640 1 -- 172.21.15.131:0/3024816986 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78680ceab0 con 0x7f78680712d0 2024-05-26T22:52:38.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.890+0000 7f7867e00640 1 -- 172.21.15.131:0/3024816986 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7f7850001670 con 0x7f78680712d0 2024-05-26T22:52:38.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.890+0000 7f7867e00640 1 -- 172.21.15.131:0/3024816986 >> v1:172.21.15.131:6789/0 conn(0x7f78680712d0 legacy=0x7f786806d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:38.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.890+0000 7f7867e00640 1 --2- 172.21.15.131:0/3024816986 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f784c0039b0 0x7f784c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:38.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.890+0000 7f7867e00640 1 --2- 172.21.15.131:0/3024816986 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f784c004800 0x7f784c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:38.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.890+0000 7f7867e00640 1 --2- 172.21.15.131:0/3024816986 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f784c0051c0 0x7f784c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:38.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.890+0000 7f7867e00640 1 -- 172.21.15.131:0/3024816986 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f786815c760 con 0x7f784c004800 2024-05-26T22:52:38.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.890+0000 7f7867e00640 1 -- 172.21.15.131:0/3024816986 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f784c00c060 con 0x7f784c0039b0 2024-05-26T22:52:38.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.890+0000 7f7867e00640 1 -- 172.21.15.131:0/3024816986 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f784c00c260 con 0x7f784c0051c0 2024-05-26T22:52:38.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.891+0000 7f785fe00640 1 --2- 172.21.15.131:0/3024816986 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f784c004800 0x7f784c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:38.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.891+0000 7f785fe00640 1 -- 172.21.15.131:0/3024816986 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f784c0051c0 msgr2=0x7f784c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:38.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.891+0000 7f786ca00640 1 --2- 172.21.15.131:0/3024816986 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f784c0039b0 0x7f784c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:38.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.891+0000 7f786d400640 1 --2- 172.21.15.131:0/3024816986 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f784c0051c0 0x7f784c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:38.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.891+0000 7f785fe00640 1 --2- 172.21.15.131:0/3024816986 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f784c0051c0 0x7f784c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:38.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.891+0000 7f785fe00640 1 -- 172.21.15.131:0/3024816986 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f784c0039b0 msgr2=0x7f784c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:38.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.891+0000 7f785fe00640 1 --2- 172.21.15.131:0/3024816986 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f784c0039b0 0x7f784c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:38.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.891+0000 7f785fe00640 1 -- 172.21.15.131:0/3024816986 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f784c003520 con 0x7f784c004800 2024-05-26T22:52:38.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.891+0000 7f786ca00640 1 --2- 172.21.15.131:0/3024816986 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f784c0039b0 0x7f784c003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:52:38.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.891+0000 7f786d400640 1 --2- 172.21.15.131:0/3024816986 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f784c0051c0 0x7f784c00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:52:38.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.891+0000 7f786d400640 1 -- 172.21.15.131:0/3024816986 reap_dead start 2024-05-26T22:52:38.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.892+0000 7f785fe00640 1 --2- 172.21.15.131:0/3024816986 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f784c004800 0x7f784c004c80 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f785000c210 tx=0x7f784c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55c3392430ad909f server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:38.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.892+0000 7f7867e00640 1 -- 172.21.15.131:0/3024816986 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7850007d10 con 0x7f784c004800 2024-05-26T22:52:38.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.892+0000 7f7867e00640 1 -- 172.21.15.131:0/3024816986 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78500058d0 con 0x7f784c004800 2024-05-26T22:52:38.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.892+0000 7f7867e00640 1 -- 172.21.15.131:0/3024816986 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7850005d40 con 0x7f784c004800 2024-05-26T22:52:38.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.893+0000 7f786de00640 1 -- 172.21.15.131:0/3024816986 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f784c004800 msgr2=0x7f784c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:38.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.893+0000 7f786de00640 1 --2- 172.21.15.131:0/3024816986 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f784c004800 0x7f784c004c80 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f785000c210 tx=0x7f784c005830 comp rx=0 tx=0).stop 2024-05-26T22:52:38.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.893+0000 7f786de00640 1 -- 172.21.15.131:0/3024816986 shutdown_connections 2024-05-26T22:52:38.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.893+0000 7f786de00640 1 --2- 172.21.15.131:0/3024816986 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f784c004800 0x7f784c004c80 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:38.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.893+0000 7f786de00640 1 -- 172.21.15.131:0/3024816986 >> 172.21.15.131:0/3024816986 conn(0x7f78680b5660 msgr2=0x7f78680b5a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:52:38.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.893+0000 7f786de00640 1 -- 172.21.15.131:0/3024816986 shutdown_connections 2024-05-26T22:52:38.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.893+0000 7f786de00640 1 -- 172.21.15.131:0/3024816986 wait complete. 2024-05-26T22:52:38.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.895+0000 7f786de00640 1 Processor -- start 2024-05-26T22:52:38.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.895+0000 7f786de00640 1 -- start start 2024-05-26T22:52:38.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.896+0000 7f786de00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f784c004800 0x7f786815bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:38.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.896+0000 7f786de00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f786813d6f0 0x7f78680b3880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:38.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.896+0000 7f786de00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f78680b3dc0 0x7f7868143e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:38.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.896+0000 7f786de00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f7868145df0 con 0x7f784c004800 2024-05-26T22:52:38.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.896+0000 7f786de00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f78680cf070 con 0x7f78680b3dc0 2024-05-26T22:52:38.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.896+0000 7f786de00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f786806d250 con 0x7f786813d6f0 2024-05-26T22:52:38.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.896+0000 7f786ca00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f784c004800 0x7f786815bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:38.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.896+0000 7f786ca00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f784c004800 0x7f786815bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49856/0 (socket says 172.21.15.131:49856) 2024-05-26T22:52:38.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.896+0000 7f786ca00640 1 -- 172.21.15.131:0/1522705068 learned_addr learned my addr 172.21.15.131:0/1522705068 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:52:38.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.897+0000 7f786d400640 1 --2- 172.21.15.131:0/1522705068 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f78680b3dc0 0x7f7868143e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:38.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.897+0000 7f786ca00640 1 -- 172.21.15.131:0/1522705068 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f786813d6f0 msgr2=0x7f78680b3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:38.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.897+0000 7f785fe00640 1 --2- 172.21.15.131:0/1522705068 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f786813d6f0 0x7f78680b3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:38.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.897+0000 7f786ca00640 1 --2- 172.21.15.131:0/1522705068 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f786813d6f0 0x7f78680b3880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:38.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.897+0000 7f786ca00640 1 -- 172.21.15.131:0/1522705068 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f78680b3dc0 msgr2=0x7f7868143e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:38.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.897+0000 7f786ca00640 1 --2- 172.21.15.131:0/1522705068 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f78680b3dc0 0x7f7868143e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:38.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.897+0000 7f786ca00640 1 -- 172.21.15.131:0/1522705068 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78681418b0 con 0x7f784c004800 2024-05-26T22:52:38.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.897+0000 7f786ca00640 1 --2- 172.21.15.131:0/1522705068 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f784c004800 0x7f786815bab0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f7858010a00 tx=0x7f7858010ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:38.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.897+0000 7f7865600640 1 -- 172.21.15.131:0/1522705068 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f785800f040 con 0x7f784c004800 2024-05-26T22:52:38.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.897+0000 7f7865600640 1 -- 172.21.15.131:0/1522705068 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f785800ebc0 con 0x7f784c004800 2024-05-26T22:52:38.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.897+0000 7f786de00640 1 -- 172.21.15.131:0/1522705068 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7868141b90 con 0x7f784c004800 2024-05-26T22:52:38.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.897+0000 7f7865600640 1 -- 172.21.15.131:0/1522705068 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f785801c400 con 0x7f784c004800 2024-05-26T22:52:38.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.897+0000 7f786de00640 1 -- 172.21.15.131:0/1522705068 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7868142150 con 0x7f784c004800 2024-05-26T22:52:38.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.899+0000 7f786de00640 1 -- 172.21.15.131:0/1522705068 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78681427a0 con 0x7f784c004800 2024-05-26T22:52:38.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.899+0000 7f7865600640 1 -- 172.21.15.131:0/1522705068 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f785801c5a0 con 0x7f784c004800 2024-05-26T22:52:38.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.900+0000 7f7865600640 1 --2- 172.21.15.131:0/1522705068 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f784406be60 0x7f784406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:38.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.900+0000 7f7865600640 1 -- 172.21.15.131:0/1522705068 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f785808b510 con 0x7f784c004800 2024-05-26T22:52:38.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.901+0000 7f785fe00640 1 --2- 172.21.15.131:0/1522705068 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f784406be60 0x7f784406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:38.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.901+0000 7f785fe00640 1 --2- 172.21.15.131:0/1522705068 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f784406be60 0x7f784406e320 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f7850033000 tx=0x7f7850035040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:38.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:38.905+0000 7f7865600640 1 -- 172.21.15.131:0/1522705068 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f7858018030 con 0x7f784c004800 2024-05-26T22:52:39.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:39.141+0000 7f786de00640 1 -- 172.21.15.131:0/1522705068 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f78680694e0 con 0x7f784c004800 2024-05-26T22:52:39.142 INFO:tasks.ceph.mon.a.smithi131.stderr:2024-05-26T22:52:39.142+0000 7fcf0d200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-26T22:52:39.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:39.146+0000 7f7865600640 1 -- 172.21.15.131:0/1522705068 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v56) ==== 161+0+0 (secure 0 0 0) 0x7f7858059eb0 con 0x7f784c004800 2024-05-26T22:52:39.145 INFO:teuthology.orchestra.run.smithi131.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-26T22:52:39.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:39.149+0000 7f786de00640 1 -- 172.21.15.131:0/1522705068 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f784406be60 msgr2=0x7f784406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:39.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:39.149+0000 7f786de00640 1 --2- 172.21.15.131:0/1522705068 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f784406be60 0x7f784406e320 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f7850033000 tx=0x7f7850035040 comp rx=0 tx=0).stop 2024-05-26T22:52:39.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:39.149+0000 7f786de00640 1 -- 172.21.15.131:0/1522705068 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f784c004800 msgr2=0x7f786815bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:39.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:39.149+0000 7f786de00640 1 --2- 172.21.15.131:0/1522705068 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f784c004800 0x7f786815bab0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f7858010a00 tx=0x7f7858010ec0 comp rx=0 tx=0).stop 2024-05-26T22:52:39.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:39.149+0000 7f786de00640 1 -- 172.21.15.131:0/1522705068 shutdown_connections 2024-05-26T22:52:39.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:39.149+0000 7f786de00640 1 --2- 172.21.15.131:0/1522705068 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f786813d6f0 0x7f78680b3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:39.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:39.150+0000 7f786de00640 1 --2- 172.21.15.131:0/1522705068 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f784406be60 0x7f784406e320 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:39.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:39.150+0000 7f786de00640 1 --2- 172.21.15.131:0/1522705068 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f78680b3dc0 0x7f7868143e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:39.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:39.150+0000 7f786de00640 1 --2- 172.21.15.131:0/1522705068 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f784c004800 0x7f786815bab0 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:39.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:39.150+0000 7f786de00640 1 -- 172.21.15.131:0/1522705068 >> 172.21.15.131:0/1522705068 conn(0x7f78680b5660 msgr2=0x7f78680b5a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:52:39.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:39.153+0000 7f786de00640 1 -- 172.21.15.131:0/1522705068 shutdown_connections 2024-05-26T22:52:39.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:39.154+0000 7f786de00640 1 -- 172.21.15.131:0/1522705068 wait complete. 2024-05-26T22:52:39.174 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-26T22:52:39.228 INFO:tasks.ceph.mds.a.smithi131.stdout:starting mds.a at 2024-05-26T22:52:39.240 INFO:tasks.ceph.mds.c.smithi131.stdout:starting mds.c at 2024-05-26T22:52:39.283 INFO:tasks.ceph.mds.d.smithi187.stdout:starting mds.d at 2024-05-26T22:52:39.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:39.355+0000 7fbba4000640 1 Processor -- start 2024-05-26T22:52:39.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:39.355+0000 7fbba4000640 1 -- start start 2024-05-26T22:52:39.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:39.357+0000 7fbba4000640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fbb9c145c20 0x7fbb9c0d16b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:39.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:39.357+0000 7fbba4000640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fbb9c0b5460 0x7fbb9c0b58b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:39.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:39.357+0000 7fbba4000640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb9c071730 con 0x7fbb9c0d1bf0 2024-05-26T22:52:39.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:39.357+0000 7fbba4000640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fbb9c071a10 con 0x7fbb9c0b5460 2024-05-26T22:52:39.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:39.357+0000 7fbba4000640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fbb9c0d0eb0 con 0x7fbb9c145c20 2024-05-26T22:52:39.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:39.357+0000 7fbba2000640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fbb9c0b5460 0x7fbb9c0b58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:39.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:39.357+0000 7fbba0c00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fbb9c0d1bf0 0x7fbb9c070c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:52136/0 (socket says 172.21.15.131:52136) 2024-05-26T22:52:39.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:39.357+0000 7fbba0c00640 1 -- 172.21.15.131:0/1609032507 learned_addr learned my addr 172.21.15.131:0/1609032507 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:52:39.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:39.357+0000 7fbba1600640 1 --2- 172.21.15.131:0/1609032507 >> v2:172.21.15.187:3300/0 conn(0x7fbb9c145c20 0x7fbb9c0d16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:39.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:39.358+0000 7fbba2000640 1 -- 172.21.15.131:0/1609032507 >> v2:172.21.15.187:3300/0 conn(0x7fbb9c145c20 msgr2=0x7fbb9c0d16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:39.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:39.358+0000 7fbba2000640 1 --2- 172.21.15.131:0/1609032507 >> v2:172.21.15.187:3300/0 conn(0x7fbb9c145c20 0x7fbb9c0d16b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:39.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:39.358+0000 7fbba2000640 1 -- 172.21.15.131:0/1609032507 >> v1:172.21.15.131:6789/0 conn(0x7fbb9c0d1bf0 legacy=0x7fbb9c070c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:39.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:39.358+0000 7fbba2000640 1 -- 172.21.15.131:0/1609032507 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb9c0d1190 con 0x7fbb9c0b5460 2024-05-26T22:52:39.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:39.358+0000 7fbb9be00640 1 -- 172.21.15.131:0/1609032507 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1164219293 0 0) 0x7fbb9c071730 con 0x7fbb9c0d1bf0 2024-05-26T22:52:39.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:39.359+0000 7fbba2000640 1 --2- 172.21.15.131:0/1609032507 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb9c0b5460 0x7fbb9c0b58b0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c0d2090 tx=0x7fbb9402a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=512cea59f2e925e1 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:39.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:39.359+0000 7fbb9be00640 1 -- 172.21.15.131:0/1609032507 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb94007740 con 0x7fbb9c0b5460 2024-05-26T22:52:39.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:39.359+0000 7fbb9be00640 1 -- 172.21.15.131:0/1609032507 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb940078e0 con 0x7fbb9c0b5460 2024-05-26T22:52:39.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:39.359+0000 7fbb9be00640 1 -- 172.21.15.131:0/1609032507 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb94007cb0 con 0x7fbb9c0b5460 2024-05-26T22:52:39.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:39.360+0000 7fbba4000640 1 -- 172.21.15.131:0/1609032507 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb9c0b5460 msgr2=0x7fbb9c0b58b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:39.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:39.360+0000 7fbba4000640 1 --2- 172.21.15.131:0/1609032507 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb9c0b5460 0x7fbb9c0b58b0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c0d2090 tx=0x7fbb9402a5f0 comp rx=0 tx=0).stop 2024-05-26T22:52:39.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:39.360+0000 7fbba4000640 1 -- 172.21.15.131:0/1609032507 shutdown_connections 2024-05-26T22:52:39.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:39.360+0000 7fbba4000640 1 --2- 172.21.15.131:0/1609032507 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb9c0b5460 0x7fbb9c0b58b0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:39.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:39.360+0000 7fbba4000640 1 --2- 172.21.15.131:0/1609032507 >> v2:172.21.15.187:3300/0 conn(0x7fbb9c145c20 0x7fbb9c0d16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:39.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:39.360+0000 7fbba4000640 1 -- 172.21.15.131:0/1609032507 >> 172.21.15.131:0/1609032507 conn(0x7fbb9c06f6d0 msgr2=0x7fbb9c06fae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:52:39.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:39.360+0000 7fbba4000640 1 -- 172.21.15.131:0/1609032507 shutdown_connections 2024-05-26T22:52:39.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:39.360+0000 7fbba4000640 1 -- 172.21.15.131:0/1609032507 wait complete. 2024-05-26T22:52:39.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:39.362+0000 7fbba4000640 1 Processor -- start 2024-05-26T22:52:39.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:39.362+0000 7fbba4000640 1 -- start start 2024-05-26T22:52:39.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:39.363+0000 7fbba4000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb9c0b5460 0x7fbb9c0cda00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:39.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:39.363+0000 7fbba4000640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb9c145c20 0x7fbb9c0b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:39.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:39.363+0000 7fbba4000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbb9c06ddd0 0x7fbb9c15b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:39.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:39.363+0000 7fbba4000640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fbb9c0b1950 con 0x7fbb9c0b5460 2024-05-26T22:52:39.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:39.363+0000 7fbba4000640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fbb9c146000 con 0x7fbb9c145c20 2024-05-26T22:52:39.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:39.363+0000 7fbba4000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fbb9c071380 con 0x7fbb9c06ddd0 2024-05-26T22:52:39.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:39.363+0000 7fbba1600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb9c0b5460 0x7fbb9c0cda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:39.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:39.363+0000 7fbba1600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb9c0b5460 0x7fbb9c0cda00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49904/0 (socket says 172.21.15.131:49904) 2024-05-26T22:52:39.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:39.363+0000 7fbba1600640 1 -- 172.21.15.131:0/1303954413 learned_addr learned my addr 172.21.15.131:0/1303954413 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:52:39.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:39.363+0000 7fbba1600640 1 -- 172.21.15.131:0/1303954413 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbb9c06ddd0 msgr2=0x7fbb9c15b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:39.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:39.363+0000 7fbba0c00640 1 --2- 172.21.15.131:0/1303954413 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb9c145c20 0x7fbb9c0b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:39.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:39.363+0000 7fbba1600640 1 --2- 172.21.15.131:0/1303954413 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbb9c06ddd0 0x7fbb9c15b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:39.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:39.363+0000 7fbba1600640 1 -- 172.21.15.131:0/1303954413 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb9c145c20 msgr2=0x7fbb9c0b5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:39.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:39.363+0000 7fbba1600640 1 --2- 172.21.15.131:0/1303954413 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb9c145c20 0x7fbb9c0b5ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:39.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:39.363+0000 7fbba1600640 1 -- 172.21.15.131:0/1303954413 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb9c1f75b0 con 0x7fbb9c0b5460 2024-05-26T22:52:39.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:39.364+0000 7fbba1600640 1 --2- 172.21.15.131:0/1303954413 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb9c0b5460 0x7fbb9c0cda00 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fbb9000e9e0 tx=0x7fbb9000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:39.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:39.364+0000 7fbb99600640 1 -- 172.21.15.131:0/1303954413 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb90017070 con 0x7fbb9c0b5460 2024-05-26T22:52:39.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:39.364+0000 7fbb99600640 1 -- 172.21.15.131:0/1303954413 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb9000c9d0 con 0x7fbb9c0b5460 2024-05-26T22:52:39.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:39.364+0000 7fbba4000640 1 -- 172.21.15.131:0/1303954413 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb9c1f7890 con 0x7fbb9c0b5460 2024-05-26T22:52:39.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:39.364+0000 7fbb99600640 1 -- 172.21.15.131:0/1303954413 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb9000cdc0 con 0x7fbb9c0b5460 2024-05-26T22:52:39.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:39.364+0000 7fbba4000640 1 -- 172.21.15.131:0/1303954413 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb9c1f7e50 con 0x7fbb9c0b5460 2024-05-26T22:52:39.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:39.365+0000 7fbba4000640 1 -- 172.21.15.131:0/1303954413 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb9c0695c0 con 0x7fbb9c0b5460 2024-05-26T22:52:39.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:39.366+0000 7fbb99600640 1 -- 172.21.15.131:0/1303954413 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbb9001a070 con 0x7fbb9c0b5460 2024-05-26T22:52:39.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:39.367+0000 7fbb99600640 1 --2- 172.21.15.131:0/1303954413 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fbb7c06be30 0x7fbb7c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:39.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:39.367+0000 7fbb99600640 1 -- 172.21.15.131:0/1303954413 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5967+0+0 (secure 0 0 0) 0x7fbb9008afa0 con 0x7fbb9c0b5460 2024-05-26T22:52:39.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:39.367+0000 7fbba0c00640 1 --2- 172.21.15.131:0/1303954413 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fbb7c06be30 0x7fbb7c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:39.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:39.368+0000 7fbba0c00640 1 --2- 172.21.15.131:0/1303954413 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fbb7c06be30 0x7fbb7c06e2f0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fbb88007fd0 tx=0x7fbb88009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:39.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:39.372+0000 7fbb99600640 1 -- 172.21.15.131:0/1303954413 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fbb90059840 con 0x7fbb9c0b5460 2024-05-26T22:52:39.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:39.609+0000 7fbba4000640 1 -- 172.21.15.131:0/1303954413 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fbb9c0b7160 con 0x7fbb9c0b5460 2024-05-26T22:52:40.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:40.149+0000 7fbb99600640 1 -- 172.21.15.131:0/1303954413 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v57) ==== 104+0+0 (secure 0 0 0) 0x7fbb90059140 con 0x7fbb9c0b5460 2024-05-26T22:52:40.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:40.152+0000 7fbba4000640 1 -- 172.21.15.131:0/1303954413 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fbb7c06be30 msgr2=0x7fbb7c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:40.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:40.152+0000 7fbba4000640 1 --2- 172.21.15.131:0/1303954413 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fbb7c06be30 0x7fbb7c06e2f0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fbb88007fd0 tx=0x7fbb88009040 comp rx=0 tx=0).stop 2024-05-26T22:52:40.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:40.153+0000 7fbba4000640 1 -- 172.21.15.131:0/1303954413 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb9c0b5460 msgr2=0x7fbb9c0cda00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:40.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:40.153+0000 7fbba4000640 1 --2- 172.21.15.131:0/1303954413 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb9c0b5460 0x7fbb9c0cda00 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fbb9000e9e0 tx=0x7fbb9000eea0 comp rx=0 tx=0).stop 2024-05-26T22:52:40.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:40.153+0000 7fbba4000640 1 -- 172.21.15.131:0/1303954413 shutdown_connections 2024-05-26T22:52:40.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:40.153+0000 7fbba4000640 1 --2- 172.21.15.131:0/1303954413 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbb9c06ddd0 0x7fbb9c15b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:40.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:40.153+0000 7fbba4000640 1 --2- 172.21.15.131:0/1303954413 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fbb7c06be30 0x7fbb7c06e2f0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:40.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:40.153+0000 7fbba4000640 1 --2- 172.21.15.131:0/1303954413 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb9c145c20 0x7fbb9c0b5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:40.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:40.153+0000 7fbba4000640 1 --2- 172.21.15.131:0/1303954413 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb9c0b5460 0x7fbb9c0cda00 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:40.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:40.153+0000 7fbba4000640 1 -- 172.21.15.131:0/1303954413 >> 172.21.15.131:0/1303954413 conn(0x7fbb9c06f6d0 msgr2=0x7fbb9c15c8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:52:40.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:40.153+0000 7fbba4000640 1 -- 172.21.15.131:0/1303954413 shutdown_connections 2024-05-26T22:52:40.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:40.153+0000 7fbba4000640 1 -- 172.21.15.131:0/1303954413 wait complete. 2024-05-26T22:52:40.174 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-26T22:52:40.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:40.371+0000 7f9ec5200640 1 Processor -- start 2024-05-26T22:52:40.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:40.371+0000 7f9ec5200640 1 -- start start 2024-05-26T22:52:40.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:40.373+0000 7f9ec5200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9ec015fba0 0x7f9ec006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:40.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:40.373+0000 7f9ec5200640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ec0145c60 con 0x7f9ec0160000 2024-05-26T22:52:40.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:40.373+0000 7f9ec5200640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ec01605b0 con 0x7f9ec0145880 2024-05-26T22:52:40.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:40.373+0000 7f9ec5200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9ec015c560 con 0x7f9ec015fba0 2024-05-26T22:52:40.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:40.373+0000 7f9ebfe00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f9ec0160000 0x7f9ec00b37f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:52152/0 (socket says 172.21.15.131:52152) 2024-05-26T22:52:40.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:40.373+0000 7f9ebfe00640 1 -- 172.21.15.131:0/1190307374 learned_addr learned my addr 172.21.15.131:0/1190307374 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:52:40.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:40.373+0000 7f9ebe000640 1 -- 172.21.15.131:0/1190307374 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1815681260 0 0) 0x7f9ec0145c60 con 0x7f9ec0160000 2024-05-26T22:52:40.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:40.373+0000 7f9ebe000640 1 -- 172.21.15.131:0/1190307374 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9ea4003520 con 0x7f9ec0160000 2024-05-26T22:52:40.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:40.373+0000 7f9ebea00640 1 --2- 172.21.15.131:0/1190307374 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9ec015fba0 0x7f9ec006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:40.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:40.374+0000 7f9ebe000640 1 -- 172.21.15.131:0/1190307374 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2027003113 0 0) 0x7f9ea4003520 con 0x7f9ec0160000 2024-05-26T22:52:40.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:40.374+0000 7f9ebe000640 1 -- 172.21.15.131:0/1190307374 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9ec015fba0 msgr2=0x7f9ec006d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:40.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:40.374+0000 7f9ebe000640 1 --2- 172.21.15.131:0/1190307374 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9ec015fba0 0x7f9ec006d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:40.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:40.374+0000 7f9ebe000640 1 -- 172.21.15.131:0/1190307374 >> v1:172.21.15.187:6789/0 conn(0x7f9ec0145880 legacy=0x7f9ec015cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:40.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:40.374+0000 7f9ebe000640 1 -- 172.21.15.131:0/1190307374 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ec0160890 con 0x7f9ec0160000 2024-05-26T22:52:40.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:40.374+0000 7f9ebe000640 1 -- 172.21.15.131:0/1190307374 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7f9eb80019f0 con 0x7f9ec0160000 2024-05-26T22:52:40.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:40.374+0000 7f9ebe000640 1 -- 172.21.15.131:0/1190307374 >> v1:172.21.15.131:6789/0 conn(0x7f9ec0160000 legacy=0x7f9ec00b37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:40.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:40.374+0000 7f9ebe000640 1 --2- 172.21.15.131:0/1190307374 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ea40039e0 0x7f9ea4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:40.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:40.374+0000 7f9ebe000640 1 --2- 172.21.15.131:0/1190307374 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9ec015fba0 0x7f9ea4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:40.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:40.374+0000 7f9ebe000640 1 --2- 172.21.15.131:0/1190307374 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9ea4004d40 0x7f9ea40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:40.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:40.374+0000 7f9ebe000640 1 -- 172.21.15.131:0/1190307374 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f9ec015c560 con 0x7f9ea40039e0 2024-05-26T22:52:40.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:40.374+0000 7f9ebe000640 1 -- 172.21.15.131:0/1190307374 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9ea400bb50 con 0x7f9ea4004d40 2024-05-26T22:52:40.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:40.374+0000 7f9ebe000640 1 -- 172.21.15.131:0/1190307374 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9ea400bd20 con 0x7f9ec015fba0 2024-05-26T22:52:40.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:40.374+0000 7f9ebf400640 1 --2- 172.21.15.131:0/1190307374 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ea40039e0 0x7f9ea4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:40.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:40.375+0000 7f9ebea00640 1 --2- 172.21.15.131:0/1190307374 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9ec015fba0 0x7f9ea4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:40.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:40.375+0000 7f9ebfe00640 1 --2- 172.21.15.131:0/1190307374 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9ea4004d40 0x7f9ea40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:40.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:40.375+0000 7f9ebf400640 1 -- 172.21.15.131:0/1190307374 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9ec015fba0 msgr2=0x7f9ea4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:40.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:40.375+0000 7f9ebf400640 1 --2- 172.21.15.131:0/1190307374 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9ec015fba0 0x7f9ea4004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:40.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:40.375+0000 7f9ebf400640 1 -- 172.21.15.131:0/1190307374 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9ea4004d40 msgr2=0x7f9ea40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:40.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:40.375+0000 7f9ebf400640 1 --2- 172.21.15.131:0/1190307374 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9ea4004d40 0x7f9ea40051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:40.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:40.375+0000 7f9ebf400640 1 -- 172.21.15.131:0/1190307374 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ea4003520 con 0x7f9ea40039e0 2024-05-26T22:52:40.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:40.376+0000 7f9ebf400640 1 --2- 172.21.15.131:0/1190307374 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ea40039e0 0x7f9ea4003e00 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f9eb0001780 tx=0x7f9ea4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa11ab3b53bec458 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:40.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:40.376+0000 7f9ebe000640 1 -- 172.21.15.131:0/1190307374 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eb0001370 con 0x7f9ea40039e0 2024-05-26T22:52:40.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:40.376+0000 7f9ebe000640 1 -- 172.21.15.131:0/1190307374 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9eb00052d0 con 0x7f9ea40039e0 2024-05-26T22:52:40.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:40.376+0000 7f9ebe000640 1 -- 172.21.15.131:0/1190307374 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eb002ac00 con 0x7f9ea40039e0 2024-05-26T22:52:40.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:40.377+0000 7f9ec5200640 1 -- 172.21.15.131:0/1190307374 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ea40039e0 msgr2=0x7f9ea4003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:40.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:40.377+0000 7f9ec5200640 1 --2- 172.21.15.131:0/1190307374 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ea40039e0 0x7f9ea4003e00 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f9eb0001780 tx=0x7f9ea4005830 comp rx=0 tx=0).stop 2024-05-26T22:52:40.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:40.377+0000 7f9ebfe00640 1 -- 172.21.15.131:0/1190307374 reap_dead start 2024-05-26T22:52:40.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:40.377+0000 7f9ec5200640 1 -- 172.21.15.131:0/1190307374 shutdown_connections 2024-05-26T22:52:40.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:40.377+0000 7f9ec5200640 1 -- 172.21.15.131:0/1190307374 >> 172.21.15.131:0/1190307374 conn(0x7f9ec00b5410 msgr2=0x7f9ec00b57f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:52:40.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:40.377+0000 7f9ec5200640 1 -- 172.21.15.131:0/1190307374 shutdown_connections 2024-05-26T22:52:40.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:40.377+0000 7f9ec5200640 1 -- 172.21.15.131:0/1190307374 wait complete. 2024-05-26T22:52:40.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:40.379+0000 7f9ec5200640 1 Processor -- start 2024-05-26T22:52:40.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:40.379+0000 7f9ec5200640 1 -- start start 2024-05-26T22:52:40.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:40.380+0000 7f9ec5200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ea40039e0 0x7f9ec0136620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:40.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:40.380+0000 7f9ec5200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9ec015aa50 0x7f9ec015aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:40.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:40.380+0000 7f9ec5200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9ec00ae9f0 0x7f9ec00aee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:40.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:40.380+0000 7f9ec5200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f9ec00b1d10 con 0x7f9ea40039e0 2024-05-26T22:52:40.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:40.380+0000 7f9ec5200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9ec015cb10 con 0x7f9ec015aa50 2024-05-26T22:52:40.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:40.380+0000 7f9ec5200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9ec00b1e90 con 0x7f9ec00ae9f0 2024-05-26T22:52:40.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:40.380+0000 7f9ebf400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ea40039e0 0x7f9ec0136620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:40.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:40.380+0000 7f9ebf400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ea40039e0 0x7f9ec0136620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49912/0 (socket says 172.21.15.131:49912) 2024-05-26T22:52:40.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:40.380+0000 7f9ebf400640 1 -- 172.21.15.131:0/2762395621 learned_addr learned my addr 172.21.15.131:0/2762395621 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:52:40.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:40.380+0000 7f9ebea00640 1 --2- 172.21.15.131:0/2762395621 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9ec015aa50 0x7f9ec015aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:40.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:40.380+0000 7f9ebfe00640 1 --2- 172.21.15.131:0/2762395621 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9ec00ae9f0 0x7f9ec00aee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:40.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:40.380+0000 7f9ebf400640 1 -- 172.21.15.131:0/2762395621 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9ec00ae9f0 msgr2=0x7f9ec00aee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:40.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:40.380+0000 7f9ebf400640 1 --2- 172.21.15.131:0/2762395621 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9ec00ae9f0 0x7f9ec00aee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:40.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:40.380+0000 7f9ebf400640 1 -- 172.21.15.131:0/2762395621 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9ec015aa50 msgr2=0x7f9ec015aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:40.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:40.380+0000 7f9ebf400640 1 --2- 172.21.15.131:0/2762395621 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9ec015aa50 0x7f9ec015aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:40.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:40.380+0000 7f9ebf400640 1 -- 172.21.15.131:0/2762395621 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ec015b510 con 0x7f9ea40039e0 2024-05-26T22:52:40.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:40.381+0000 7f9ebf400640 1 --2- 172.21.15.131:0/2762395621 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ea40039e0 0x7f9ec0136620 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f9eb0008fd0 tx=0x7f9eb0001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:40.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:40.381+0000 7f9eb7400640 1 -- 172.21.15.131:0/2762395621 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eb0040020 con 0x7f9ea40039e0 2024-05-26T22:52:40.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:40.381+0000 7f9eb7400640 1 -- 172.21.15.131:0/2762395621 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9eb0007680 con 0x7f9ea40039e0 2024-05-26T22:52:40.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:40.381+0000 7f9eb7400640 1 -- 172.21.15.131:0/2762395621 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eb0007a30 con 0x7f9ea40039e0 2024-05-26T22:52:40.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:40.381+0000 7f9ec5200640 1 -- 172.21.15.131:0/2762395621 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ec015b7f0 con 0x7f9ea40039e0 2024-05-26T22:52:40.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:40.381+0000 7f9ec5200640 1 -- 172.21.15.131:0/2762395621 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ec013e010 con 0x7f9ea40039e0 2024-05-26T22:52:40.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:40.382+0000 7f9ec5200640 1 -- 172.21.15.131:0/2762395621 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ec00694e0 con 0x7f9ea40039e0 2024-05-26T22:52:40.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:40.383+0000 7f9eb7400640 1 -- 172.21.15.131:0/2762395621 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9eb0043020 con 0x7f9ea40039e0 2024-05-26T22:52:40.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:40.384+0000 7f9eb7400640 1 --2- 172.21.15.131:0/2762395621 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9ea006be60 0x7f9ea006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:40.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:40.384+0000 7f9eb7400640 1 -- 172.21.15.131:0/2762395621 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5967+0+0 (secure 0 0 0) 0x7f9eb00aaf30 con 0x7f9ea40039e0 2024-05-26T22:52:40.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:40.385+0000 7f9ebea00640 1 --2- 172.21.15.131:0/2762395621 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9ea006be60 0x7f9ea006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:40.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:40.385+0000 7f9ebea00640 1 --2- 172.21.15.131:0/2762395621 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9ea006be60 0x7f9ea006e320 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f9ec00b0ce0 tx=0x7f9eac008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:40.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:40.389+0000 7f9eb7400640 1 -- 172.21.15.131:0/2762395621 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f9eb004a030 con 0x7f9ea40039e0 2024-05-26T22:52:40.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:40.585+0000 7f9ec5200640 1 -- 172.21.15.131:0/2762395621 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f9ec00b38b0 con 0x7f9ea40039e0 2024-05-26T22:52:41.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:41.160+0000 7f9eb7400640 1 -- 172.21.15.131:0/2762395621 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v60) ==== 181+0+0 (secure 0 0 0) 0x7f9eb00796b0 con 0x7f9ea40039e0 2024-05-26T22:52:41.160 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_metadata' removed 2024-05-26T22:52:41.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:41.163+0000 7f9ec5200640 1 -- 172.21.15.131:0/2762395621 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9ea006be60 msgr2=0x7f9ea006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:41.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:41.163+0000 7f9ec5200640 1 --2- 172.21.15.131:0/2762395621 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9ea006be60 0x7f9ea006e320 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f9ec00b0ce0 tx=0x7f9eac008040 comp rx=0 tx=0).stop 2024-05-26T22:52:41.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:41.163+0000 7f9ec5200640 1 -- 172.21.15.131:0/2762395621 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ea40039e0 msgr2=0x7f9ec0136620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:41.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:41.163+0000 7f9ec5200640 1 --2- 172.21.15.131:0/2762395621 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ea40039e0 0x7f9ec0136620 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f9eb0008fd0 tx=0x7f9eb0001120 comp rx=0 tx=0).stop 2024-05-26T22:52:41.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:41.164+0000 7f9ec5200640 1 -- 172.21.15.131:0/2762395621 shutdown_connections 2024-05-26T22:52:41.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:41.164+0000 7f9ec5200640 1 --2- 172.21.15.131:0/2762395621 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9ec00ae9f0 0x7f9ec00aee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:41.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:41.164+0000 7f9ec5200640 1 --2- 172.21.15.131:0/2762395621 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9ea006be60 0x7f9ea006e320 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:41.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:41.164+0000 7f9ec5200640 1 --2- 172.21.15.131:0/2762395621 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9ec015aa50 0x7f9ec015aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:41.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:41.164+0000 7f9ec5200640 1 --2- 172.21.15.131:0/2762395621 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ea40039e0 0x7f9ec0136620 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:41.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:41.164+0000 7f9ec5200640 1 -- 172.21.15.131:0/2762395621 >> 172.21.15.131:0/2762395621 conn(0x7f9ec00b5410 msgr2=0x7f9ec015cfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:52:41.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:41.165+0000 7f9ec5200640 1 -- 172.21.15.131:0/2762395621 shutdown_connections 2024-05-26T22:52:41.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:41.165+0000 7f9ec5200640 1 -- 172.21.15.131:0/2762395621 wait complete. 2024-05-26T22:52:41.187 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-26T22:52:41.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:41.386+0000 7f68d6000640 1 Processor -- start 2024-05-26T22:52:41.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:41.386+0000 7f68d6000640 1 -- start start 2024-05-26T22:52:41.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:41.388+0000 7f68d6000640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f68d0145880 0x7f68d015cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:41.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:41.388+0000 7f68d6000640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f68d015fba0 0x7f68d006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:41.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:41.388+0000 7f68d6000640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68d015d4e0 con 0x7f68d0160020 2024-05-26T22:52:41.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:41.388+0000 7f68d6000640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f68d015c560 con 0x7f68d0145880 2024-05-26T22:52:41.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:41.388+0000 7f68d6000640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f68d015cb10 con 0x7f68d015fba0 2024-05-26T22:52:41.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:41.388+0000 7f68d5600640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f68d0160020 0x7f68d00b37f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:52164/0 (socket says 172.21.15.131:52164) 2024-05-26T22:52:41.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:41.388+0000 7f68d5600640 1 -- 172.21.15.131:0/1124351159 learned_addr learned my addr 172.21.15.131:0/1124351159 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:52:41.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:41.388+0000 7f68d4c00640 1 --2- 172.21.15.131:0/1124351159 >> v2:172.21.15.131:3300/0 conn(0x7f68d0145880 0x7f68d015cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:41.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:41.389+0000 7f68cfe00640 1 --2- 172.21.15.131:0/1124351159 >> v2:172.21.15.187:3300/0 conn(0x7f68d015fba0 0x7f68d006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:41.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:41.389+0000 7f68cf400640 1 -- 172.21.15.131:0/1124351159 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3037584796 0 0) 0x7f68d015d4e0 con 0x7f68d0160020 2024-05-26T22:52:41.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:41.389+0000 7f68cf400640 1 -- 172.21.15.131:0/1124351159 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f68b8003520 con 0x7f68d0160020 2024-05-26T22:52:41.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:41.389+0000 7f68d4c00640 1 -- 172.21.15.131:0/1124351159 >> v2:172.21.15.187:3300/0 conn(0x7f68d015fba0 msgr2=0x7f68d006d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:41.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:41.389+0000 7f68d4c00640 1 --2- 172.21.15.131:0/1124351159 >> v2:172.21.15.187:3300/0 conn(0x7f68d015fba0 0x7f68d006d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:41.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:41.389+0000 7f68d4c00640 1 -- 172.21.15.131:0/1124351159 >> v1:172.21.15.131:6789/0 conn(0x7f68d0160020 legacy=0x7f68d00b37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:41.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:41.389+0000 7f68d4c00640 1 -- 172.21.15.131:0/1124351159 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68d0145c60 con 0x7f68d0145880 2024-05-26T22:52:41.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:41.390+0000 7f68d4c00640 1 --2- 172.21.15.131:0/1124351159 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f68d0145880 0x7f68d015cfa0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f68d00ce7d0 tx=0x7f68c402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bdc52cbcc0eb133 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:41.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:41.391+0000 7f68cf400640 1 -- 172.21.15.131:0/1124351159 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68c4037070 con 0x7f68d0145880 2024-05-26T22:52:41.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:41.391+0000 7f68cf400640 1 -- 172.21.15.131:0/1124351159 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68c4032440 con 0x7f68d0145880 2024-05-26T22:52:41.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:41.391+0000 7f68cf400640 1 -- 172.21.15.131:0/1124351159 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68c4032810 con 0x7f68d0145880 2024-05-26T22:52:41.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:41.391+0000 7f68d6000640 1 -- 172.21.15.131:0/1124351159 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f68d0145880 msgr2=0x7f68d015cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:41.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:41.391+0000 7f68d6000640 1 --2- 172.21.15.131:0/1124351159 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f68d0145880 0x7f68d015cfa0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f68d00ce7d0 tx=0x7f68c402a1a0 comp rx=0 tx=0).stop 2024-05-26T22:52:41.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:41.391+0000 7f68d6000640 1 -- 172.21.15.131:0/1124351159 shutdown_connections 2024-05-26T22:52:41.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:41.391+0000 7f68d6000640 1 --2- 172.21.15.131:0/1124351159 >> v2:172.21.15.187:3300/0 conn(0x7f68d015fba0 0x7f68d006d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:41.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:41.391+0000 7f68d6000640 1 --2- 172.21.15.131:0/1124351159 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f68d0145880 0x7f68d015cfa0 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:41.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:41.391+0000 7f68d6000640 1 -- 172.21.15.131:0/1124351159 >> 172.21.15.131:0/1124351159 conn(0x7f68d00b5410 msgr2=0x7f68d00b57f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:52:41.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:41.391+0000 7f68d6000640 1 -- 172.21.15.131:0/1124351159 shutdown_connections 2024-05-26T22:52:41.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:41.391+0000 7f68d6000640 1 -- 172.21.15.131:0/1124351159 wait complete. 2024-05-26T22:52:41.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:41.393+0000 7f68d6000640 1 Processor -- start 2024-05-26T22:52:41.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:41.393+0000 7f68d6000640 1 -- start start 2024-05-26T22:52:41.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:41.394+0000 7f68d6000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f68d0145880 0x7f68d00aeb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:41.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:41.394+0000 7f68d6000640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f68d015fba0 0x7f68d01531e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:41.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:41.394+0000 7f68d6000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f68d0160020 0x7f68d014faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:41.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:41.394+0000 7f68d6000640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f68d015c7a0 con 0x7f68d0145880 2024-05-26T22:52:41.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:41.394+0000 7f68d6000640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f68d0160890 con 0x7f68d015fba0 2024-05-26T22:52:41.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:41.394+0000 7f68d6000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f68d00b1d10 con 0x7f68d0160020 2024-05-26T22:52:41.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:41.394+0000 7f68d4c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f68d0145880 0x7f68d00aeb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:41.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:41.394+0000 7f68d4c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f68d0145880 0x7f68d00aeb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49920/0 (socket says 172.21.15.131:49920) 2024-05-26T22:52:41.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:41.394+0000 7f68d4c00640 1 -- 172.21.15.131:0/2578055862 learned_addr learned my addr 172.21.15.131:0/2578055862 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:52:41.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:41.395+0000 7f68d4c00640 1 -- 172.21.15.131:0/2578055862 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f68d0160020 msgr2=0x7f68d014faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:41.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:41.395+0000 7f68d4c00640 1 --2- 172.21.15.131:0/2578055862 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f68d0160020 0x7f68d014faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:41.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:41.395+0000 7f68d4c00640 1 -- 172.21.15.131:0/2578055862 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f68d015fba0 msgr2=0x7f68d01531e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:41.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:41.395+0000 7f68d4c00640 1 --2- 172.21.15.131:0/2578055862 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f68d015fba0 0x7f68d01531e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:41.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:41.395+0000 7f68d4c00640 1 -- 172.21.15.131:0/2578055862 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68d00ae3b0 con 0x7f68d0145880 2024-05-26T22:52:41.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:41.395+0000 7f68d4c00640 1 --2- 172.21.15.131:0/2578055862 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f68d0145880 0x7f68d00aeb70 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f68c4009890 tx=0x7f68c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:41.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:41.395+0000 7f68ccc00640 1 -- 172.21.15.131:0/2578055862 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68c4037040 con 0x7f68d0145880 2024-05-26T22:52:41.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:41.395+0000 7f68ccc00640 1 -- 172.21.15.131:0/2578055862 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68c4042070 con 0x7f68d0145880 2024-05-26T22:52:41.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:41.395+0000 7f68d6000640 1 -- 172.21.15.131:0/2578055862 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68d0139340 con 0x7f68d0145880 2024-05-26T22:52:41.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:41.395+0000 7f68ccc00640 1 -- 172.21.15.131:0/2578055862 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68c4030940 con 0x7f68d0145880 2024-05-26T22:52:41.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:41.395+0000 7f68d6000640 1 -- 172.21.15.131:0/2578055862 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68d00ae650 con 0x7f68d0145880 2024-05-26T22:52:41.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:41.397+0000 7f68d6000640 1 -- 172.21.15.131:0/2578055862 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68d00694e0 con 0x7f68d0145880 2024-05-26T22:52:41.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:41.397+0000 7f68ccc00640 1 -- 172.21.15.131:0/2578055862 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f68c4030ae0 con 0x7f68d0145880 2024-05-26T22:52:41.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:41.398+0000 7f68ccc00640 1 --2- 172.21.15.131:0/2578055862 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f68b406be30 0x7f68b406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:41.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:41.398+0000 7f68ccc00640 1 -- 172.21.15.131:0/2578055862 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5529+0+0 (secure 0 0 0) 0x7f68c40ab760 con 0x7f68d0145880 2024-05-26T22:52:41.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:41.399+0000 7f68cfe00640 1 --2- 172.21.15.131:0/2578055862 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f68b406be30 0x7f68b406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:41.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:41.399+0000 7f68cfe00640 1 --2- 172.21.15.131:0/2578055862 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f68b406be30 0x7f68b406e2f0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f68c0006fd0 tx=0x7f68c0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:41.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:41.403+0000 7f68ccc00640 1 -- 172.21.15.131:0/2578055862 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f68c407a1b0 con 0x7f68d0145880 2024-05-26T22:52:41.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:41.600+0000 7f68d6000640 1 -- 172.21.15.131:0/2578055862 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f68d00b45f0 con 0x7f68d0145880 2024-05-26T22:52:42.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.172+0000 7f68ccc00640 1 -- 172.21.15.131:0/2578055862 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v61) ==== 169+0+0 (secure 0 0 0) 0x7f68c4079ab0 con 0x7f68d0145880 2024-05-26T22:52:42.172 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_data' removed 2024-05-26T22:52:42.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.181+0000 7f68d6000640 1 -- 172.21.15.131:0/2578055862 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f68b406be30 msgr2=0x7f68b406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:42.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.181+0000 7f68d6000640 1 --2- 172.21.15.131:0/2578055862 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f68b406be30 0x7f68b406e2f0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f68c0006fd0 tx=0x7f68c0008040 comp rx=0 tx=0).stop 2024-05-26T22:52:42.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.181+0000 7f68d6000640 1 -- 172.21.15.131:0/2578055862 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f68d0145880 msgr2=0x7f68d00aeb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:42.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.181+0000 7f68d6000640 1 --2- 172.21.15.131:0/2578055862 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f68d0145880 0x7f68d00aeb70 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f68c4009890 tx=0x7f68c40046c0 comp rx=0 tx=0).stop 2024-05-26T22:52:42.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.182+0000 7f68d6000640 1 -- 172.21.15.131:0/2578055862 shutdown_connections 2024-05-26T22:52:42.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.182+0000 7f68d6000640 1 --2- 172.21.15.131:0/2578055862 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f68d0160020 0x7f68d014faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:42.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.182+0000 7f68d6000640 1 --2- 172.21.15.131:0/2578055862 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f68b406be30 0x7f68b406e2f0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:42.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.182+0000 7f68d6000640 1 --2- 172.21.15.131:0/2578055862 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f68d015fba0 0x7f68d01531e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:42.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.182+0000 7f68d6000640 1 --2- 172.21.15.131:0/2578055862 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f68d0145880 0x7f68d00aeb70 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:42.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.182+0000 7f68d6000640 1 -- 172.21.15.131:0/2578055862 >> 172.21.15.131:0/2578055862 conn(0x7f68d00b5410 msgr2=0x7f68d006d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:52:42.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.183+0000 7f68d6000640 1 -- 172.21.15.131:0/2578055862 shutdown_connections 2024-05-26T22:52:42.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.183+0000 7f68d6000640 1 -- 172.21.15.131:0/2578055862 wait complete. 2024-05-26T22:52:42.207 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_export_ephemeral_random_max 2024-05-26T22:52:42.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.395+0000 7f023f600640 1 Processor -- start 2024-05-26T22:52:42.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.395+0000 7f023f600640 1 -- start start 2024-05-26T22:52:42.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.396+0000 7f023f600640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f023815e4e0 0x7f02380b9c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:42.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.396+0000 7f023f600640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f02380cf720 0x7f02380b7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:42.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.396+0000 7f023f600640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02380d28f0 con 0x7f02380d2330 2024-05-26T22:52:42.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.396+0000 7f023f600640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f02380d2bd0 con 0x7f023815e4e0 2024-05-26T22:52:42.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.396+0000 7f023f600640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f02380d2eb0 con 0x7f02380cf720 2024-05-26T22:52:42.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.397+0000 7f023cc00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f023815e4e0 0x7f02380b9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:42.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.397+0000 7f023cc00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f023815e4e0 0x7f02380b9c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49934/0 (socket says 172.21.15.131:49934) 2024-05-26T22:52:42.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.397+0000 7f023cc00640 1 -- 172.21.15.131:0/253544639 learned_addr learned my addr 172.21.15.131:0/253544639 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:52:42.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.397+0000 7f023d600640 1 --2- 172.21.15.131:0/253544639 >> v2:172.21.15.187:3300/0 conn(0x7f02380cf720 0x7f02380b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:42.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.397+0000 7f023cc00640 1 -- 172.21.15.131:0/253544639 >> v2:172.21.15.187:3300/0 conn(0x7f02380cf720 msgr2=0x7f02380b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:42.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.397+0000 7f023cc00640 1 --2- 172.21.15.131:0/253544639 >> v2:172.21.15.187:3300/0 conn(0x7f02380cf720 0x7f02380b7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:42.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.397+0000 7f023cc00640 1 -- 172.21.15.131:0/253544639 >> v1:172.21.15.187:6789/0 conn(0x7f02380d2330 legacy=0x7f02380cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:42.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.397+0000 7f023cc00640 1 -- 172.21.15.131:0/253544639 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02380d3190 con 0x7f023815e4e0 2024-05-26T22:52:42.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.397+0000 7f023d600640 1 --2- 172.21.15.131:0/253544639 >> v2:172.21.15.187:3300/0 conn(0x7f02380cf720 0x7f02380b7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:52:42.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.398+0000 7f023cc00640 1 --2- 172.21.15.131:0/253544639 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f023815e4e0 0x7f02380b9c80 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f0228009870 tx=0x7f022802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b939c55c1acf9dd server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:42.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.399+0000 7f0237400640 1 -- 172.21.15.131:0/253544639 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0228037070 con 0x7f023815e4e0 2024-05-26T22:52:42.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.399+0000 7f0237400640 1 -- 172.21.15.131:0/253544639 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0228032440 con 0x7f023815e4e0 2024-05-26T22:52:42.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.399+0000 7f0237400640 1 -- 172.21.15.131:0/253544639 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0228032810 con 0x7f023815e4e0 2024-05-26T22:52:42.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.399+0000 7f023f600640 1 -- 172.21.15.131:0/253544639 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f023815e4e0 msgr2=0x7f02380b9c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:42.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.399+0000 7f023f600640 1 --2- 172.21.15.131:0/253544639 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f023815e4e0 0x7f02380b9c80 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f0228009870 tx=0x7f022802a1c0 comp rx=0 tx=0).stop 2024-05-26T22:52:42.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.399+0000 7f023f600640 1 -- 172.21.15.131:0/253544639 shutdown_connections 2024-05-26T22:52:42.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.399+0000 7f023f600640 1 --2- 172.21.15.131:0/253544639 >> v2:172.21.15.187:3300/0 conn(0x7f02380cf720 0x7f02380b7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:42.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.399+0000 7f023f600640 1 --2- 172.21.15.131:0/253544639 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f023815e4e0 0x7f02380b9c80 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:42.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.399+0000 7f023f600640 1 -- 172.21.15.131:0/253544639 >> 172.21.15.131:0/253544639 conn(0x7f0238072e90 msgr2=0x7f02380732a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:52:42.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.399+0000 7f023f600640 1 -- 172.21.15.131:0/253544639 shutdown_connections 2024-05-26T22:52:42.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.399+0000 7f023f600640 1 -- 172.21.15.131:0/253544639 wait complete. 2024-05-26T22:52:42.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.401+0000 7f023f600640 1 Processor -- start 2024-05-26T22:52:42.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.401+0000 7f023f600640 1 -- start start 2024-05-26T22:52:42.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.402+0000 7f023f600640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f02380cf720 0x7f02380ca660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:42.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.402+0000 7f023f600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f02380d2330 0x7f023814d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:42.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.402+0000 7f023f600640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f023815e4e0 0x7f02380cb410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:42.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.402+0000 7f023f600640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f02380d3990 con 0x7f02380d2330 2024-05-26T22:52:42.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.402+0000 7f023f600640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f023815e8c0 con 0x7f02380cf720 2024-05-26T22:52:42.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.402+0000 7f023f600640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f02380ba1c0 con 0x7f023815e4e0 2024-05-26T22:52:42.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.403+0000 7f0237e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f02380d2330 0x7f023814d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:42.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.403+0000 7f0237e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f02380d2330 0x7f023814d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49942/0 (socket says 172.21.15.131:49942) 2024-05-26T22:52:42.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.403+0000 7f0237e00640 1 -- 172.21.15.131:0/3907452165 learned_addr learned my addr 172.21.15.131:0/3907452165 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:52:42.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.403+0000 7f023cc00640 1 --2- 172.21.15.131:0/3907452165 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f02380cf720 0x7f02380ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:42.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.403+0000 7f0237e00640 1 -- 172.21.15.131:0/3907452165 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f023815e4e0 msgr2=0x7f02380cb410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:42.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.403+0000 7f0237e00640 1 --2- 172.21.15.131:0/3907452165 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f023815e4e0 0x7f02380cb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:42.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.403+0000 7f0237e00640 1 -- 172.21.15.131:0/3907452165 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f02380cf720 msgr2=0x7f02380ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:42.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.403+0000 7f0237e00640 1 --2- 172.21.15.131:0/3907452165 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f02380cf720 0x7f02380ca660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:42.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.403+0000 7f0237e00640 1 -- 172.21.15.131:0/3907452165 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02380c95c0 con 0x7f02380d2330 2024-05-26T22:52:42.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.403+0000 7f0237e00640 1 --2- 172.21.15.131:0/3907452165 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f02380d2330 0x7f023814d990 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f022000b410 tx=0x7f022000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:42.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.404+0000 7f0234c00640 1 -- 172.21.15.131:0/3907452165 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0220015020 con 0x7f02380d2330 2024-05-26T22:52:42.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.404+0000 7f0234c00640 1 -- 172.21.15.131:0/3907452165 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0220011830 con 0x7f02380d2330 2024-05-26T22:52:42.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.404+0000 7f0234c00640 1 -- 172.21.15.131:0/3907452165 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0220011c20 con 0x7f02380d2330 2024-05-26T22:52:42.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.404+0000 7f023f600640 1 -- 172.21.15.131:0/3907452165 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02380c98a0 con 0x7f02380d2330 2024-05-26T22:52:42.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.404+0000 7f023f600640 1 -- 172.21.15.131:0/3907452165 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02380c9e60 con 0x7f02380d2330 2024-05-26T22:52:42.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.405+0000 7f023f600640 1 -- 172.21.15.131:0/3907452165 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02380695c0 con 0x7f02380d2330 2024-05-26T22:52:42.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.406+0000 7f0234c00640 1 -- 172.21.15.131:0/3907452165 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0220018020 con 0x7f02380d2330 2024-05-26T22:52:42.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.407+0000 7f0234c00640 1 --2- 172.21.15.131:0/3907452165 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f021406be30 0x7f021406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:42.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.408+0000 7f0234c00640 1 -- 172.21.15.131:0/3907452165 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5115+0+0 (secure 0 0 0) 0x7f022008ac30 con 0x7f02380d2330 2024-05-26T22:52:42.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.408+0000 7f023cc00640 1 --2- 172.21.15.131:0/3907452165 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f021406be30 0x7f021406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:42.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.408+0000 7f023cc00640 1 --2- 172.21.15.131:0/3907452165 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f021406be30 0x7f021406e2f0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f0228034000 tx=0x7f022802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:42.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.413+0000 7f0234c00640 1 -- 172.21.15.131:0/3907452165 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f022001f020 con 0x7f02380d2330 2024-05-26T22:52:42.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.621+0000 7f023f600640 1 -- 172.21.15.131:0/3907452165 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_export_ephemeral_random_max"} v 0) -- 0x7f0238068650 con 0x7f02380d2330 2024-05-26T22:52:42.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.627+0000 7f0234c00640 1 -- 172.21.15.131:0/3907452165 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_export_ephemeral_random_max"}]=0 v31) ==== 114+0+0 (secure 0 0 0) 0x7f02200172c0 con 0x7f02380d2330 2024-05-26T22:52:42.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.630+0000 7f023f600640 1 -- 172.21.15.131:0/3907452165 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f021406be30 msgr2=0x7f021406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:42.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.630+0000 7f023f600640 1 --2- 172.21.15.131:0/3907452165 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f021406be30 0x7f021406e2f0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f0228034000 tx=0x7f022802e000 comp rx=0 tx=0).stop 2024-05-26T22:52:42.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.630+0000 7f023f600640 1 -- 172.21.15.131:0/3907452165 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f02380d2330 msgr2=0x7f023814d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:42.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.631+0000 7f023f600640 1 --2- 172.21.15.131:0/3907452165 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f02380d2330 0x7f023814d990 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f022000b410 tx=0x7f022000b8d0 comp rx=0 tx=0).stop 2024-05-26T22:52:42.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.631+0000 7f023f600640 1 -- 172.21.15.131:0/3907452165 shutdown_connections 2024-05-26T22:52:42.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.631+0000 7f023f600640 1 --2- 172.21.15.131:0/3907452165 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f023815e4e0 0x7f02380cb410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:42.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.631+0000 7f023f600640 1 --2- 172.21.15.131:0/3907452165 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f021406be30 0x7f021406e2f0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:42.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.631+0000 7f023f600640 1 --2- 172.21.15.131:0/3907452165 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f02380d2330 0x7f023814d990 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:42.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.631+0000 7f023f600640 1 --2- 172.21.15.131:0/3907452165 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f02380cf720 0x7f02380ca660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:42.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.631+0000 7f023f600640 1 -- 172.21.15.131:0/3907452165 >> 172.21.15.131:0/3907452165 conn(0x7f0238072e90 msgr2=0x7f02380b5660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:52:42.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.631+0000 7f023f600640 1 -- 172.21.15.131:0/3907452165 shutdown_connections 2024-05-26T22:52:42.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.631+0000 7f023f600640 1 -- 172.21.15.131:0/3907452165 wait complete. 2024-05-26T22:52:42.650 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_export_ephemeral_random 2024-05-26T22:52:42.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.855+0000 7f3f4fc00640 1 Processor -- start 2024-05-26T22:52:42.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.856+0000 7f3f4fc00640 1 -- start start 2024-05-26T22:52:42.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.857+0000 7f3f4fc00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f4806f180 0x7f3f48072e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:42.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.857+0000 7f3f4fc00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f3f48072540 0x7f3f48161ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:42.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.857+0000 7f3f4fc00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f3f480cfe50 0x7f3f480d02e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:42.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.857+0000 7f3f4fc00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f3f480cfb30 con 0x7f3f480cfe50 2024-05-26T22:52:42.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.857+0000 7f3f4fc00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f3f480af8e0 con 0x7f3f48072540 2024-05-26T22:52:42.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.857+0000 7f3f4fc00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3f480cf200 con 0x7f3f4806f180 2024-05-26T22:52:42.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.858+0000 7f3f4dc00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f3f480cfe50 0x7f3f480d02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:42.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.858+0000 7f3f4dc00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f3f480cfe50 0x7f3f480d02e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49956/0 (socket says 172.21.15.131:49956) 2024-05-26T22:52:42.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.858+0000 7f3f4dc00640 1 -- 172.21.15.131:0/413498992 learned_addr learned my addr 172.21.15.131:0/413498992 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:52:42.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.858+0000 7f3f4d200640 1 --2- 172.21.15.131:0/413498992 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f4806f180 0x7f3f48072e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:42.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.858+0000 7f3f3fe00640 1 --2- 172.21.15.131:0/413498992 >> v2:172.21.15.187:3300/0 conn(0x7f3f48072540 0x7f3f48161ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:42.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.858+0000 7f3f4dc00640 1 -- 172.21.15.131:0/413498992 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f4806f180 msgr2=0x7f3f48072e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:42.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.858+0000 7f3f4dc00640 1 --2- 172.21.15.131:0/413498992 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f4806f180 0x7f3f48072e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:42.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.858+0000 7f3f4dc00640 1 -- 172.21.15.131:0/413498992 >> v2:172.21.15.187:3300/0 conn(0x7f3f48072540 msgr2=0x7f3f48161ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:42.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.858+0000 7f3f4dc00640 1 --2- 172.21.15.131:0/413498992 >> v2:172.21.15.187:3300/0 conn(0x7f3f48072540 0x7f3f48161ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:42.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.858+0000 7f3f4dc00640 1 -- 172.21.15.131:0/413498992 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f480cf4e0 con 0x7f3f480cfe50 2024-05-26T22:52:42.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.858+0000 7f3f3fe00640 1 --2- 172.21.15.131:0/413498992 >> v2:172.21.15.187:3300/0 conn(0x7f3f48072540 0x7f3f48161ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:52:42.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.859+0000 7f3f4dc00640 1 --2- 172.21.15.131:0/413498992 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f480cfe50 0x7f3f480d02e0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f3f480737f0 tx=0x7f3f4002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a691698f8f0faa07 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:42.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.860+0000 7f3f47e00640 1 -- 172.21.15.131:0/413498992 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f40036070 con 0x7f3f480cfe50 2024-05-26T22:52:42.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.860+0000 7f3f47e00640 1 -- 172.21.15.131:0/413498992 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f40031600 con 0x7f3f480cfe50 2024-05-26T22:52:42.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.860+0000 7f3f47e00640 1 -- 172.21.15.131:0/413498992 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f400319d0 con 0x7f3f480cfe50 2024-05-26T22:52:42.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.861+0000 7f3f4fc00640 1 -- 172.21.15.131:0/413498992 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f480cfe50 msgr2=0x7f3f480d02e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:42.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.861+0000 7f3f4fc00640 1 --2- 172.21.15.131:0/413498992 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f480cfe50 0x7f3f480d02e0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f3f480737f0 tx=0x7f3f4002a6a0 comp rx=0 tx=0).stop 2024-05-26T22:52:42.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.861+0000 7f3f4fc00640 1 -- 172.21.15.131:0/413498992 shutdown_connections 2024-05-26T22:52:42.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.861+0000 7f3f4fc00640 1 --2- 172.21.15.131:0/413498992 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f480cfe50 0x7f3f480d02e0 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:42.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.861+0000 7f3f4fc00640 1 --2- 172.21.15.131:0/413498992 >> v2:172.21.15.187:3300/0 conn(0x7f3f48072540 0x7f3f48161ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:42.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.861+0000 7f3f4fc00640 1 --2- 172.21.15.131:0/413498992 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f4806f180 0x7f3f48072e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:42.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.861+0000 7f3f4fc00640 1 -- 172.21.15.131:0/413498992 >> 172.21.15.131:0/413498992 conn(0x7f3f4806bc90 msgr2=0x7f3f480b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:52:42.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.861+0000 7f3f4fc00640 1 -- 172.21.15.131:0/413498992 shutdown_connections 2024-05-26T22:52:42.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.861+0000 7f3f4fc00640 1 -- 172.21.15.131:0/413498992 wait complete. 2024-05-26T22:52:42.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.863+0000 7f3f4fc00640 1 Processor -- start 2024-05-26T22:52:42.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.863+0000 7f3f4fc00640 1 -- start start 2024-05-26T22:52:42.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.864+0000 7f3f4fc00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3f4806f180 0x7f3f4814dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:42.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.864+0000 7f3f4fc00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f48072540 0x7f3f480ae830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:42.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.864+0000 7f3f4fc00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f480cfe50 0x7f3f4809aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:42.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.864+0000 7f3f4fc00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f3f4806f6e0 con 0x7f3f48072540 2024-05-26T22:52:42.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.864+0000 7f3f4fc00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3f4806f560 con 0x7f3f4806f180 2024-05-26T22:52:42.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.864+0000 7f3f4fc00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3f480715a0 con 0x7f3f480cfe50 2024-05-26T22:52:42.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.864+0000 7f3f3fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f48072540 0x7f3f480ae830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:42.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.864+0000 7f3f3fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f48072540 0x7f3f480ae830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49960/0 (socket says 172.21.15.131:49960) 2024-05-26T22:52:42.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.864+0000 7f3f3fe00640 1 -- 172.21.15.131:0/1167907986 learned_addr learned my addr 172.21.15.131:0/1167907986 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:52:42.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.865+0000 7f3f3fe00640 1 -- 172.21.15.131:0/1167907986 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f480cfe50 msgr2=0x7f3f4809aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:42.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.865+0000 7f3f3fe00640 1 --2- 172.21.15.131:0/1167907986 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f480cfe50 0x7f3f4809aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:42.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.865+0000 7f3f3fe00640 1 -- 172.21.15.131:0/1167907986 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3f4806f180 msgr2=0x7f3f4814dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:42.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.865+0000 7f3f3fe00640 1 --2- 172.21.15.131:0/1167907986 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3f4806f180 0x7f3f4814dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:42.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.865+0000 7f3f3fe00640 1 -- 172.21.15.131:0/1167907986 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f4814d1e0 con 0x7f3f48072540 2024-05-26T22:52:42.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.865+0000 7f3f3fe00640 1 --2- 172.21.15.131:0/1167907986 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f48072540 0x7f3f480ae830 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f3f3000c820 tx=0x7f3f3000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:42.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.865+0000 7f3f45600640 1 -- 172.21.15.131:0/1167907986 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f30015070 con 0x7f3f48072540 2024-05-26T22:52:42.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.865+0000 7f3f45600640 1 -- 172.21.15.131:0/1167907986 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f30007a30 con 0x7f3f48072540 2024-05-26T22:52:42.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.865+0000 7f3f45600640 1 -- 172.21.15.131:0/1167907986 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f30007e00 con 0x7f3f48072540 2024-05-26T22:52:42.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.865+0000 7f3f4fc00640 1 -- 172.21.15.131:0/1167907986 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f4809b300 con 0x7f3f48072540 2024-05-26T22:52:42.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.866+0000 7f3f4fc00640 1 -- 172.21.15.131:0/1167907986 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f4814d480 con 0x7f3f48072540 2024-05-26T22:52:42.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.867+0000 7f3f4fc00640 1 -- 172.21.15.131:0/1167907986 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f48074e90 con 0x7f3f48072540 2024-05-26T22:52:42.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.868+0000 7f3f45600640 1 -- 172.21.15.131:0/1167907986 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3f30018070 con 0x7f3f48072540 2024-05-26T22:52:42.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.868+0000 7f3f45600640 1 --2- 172.21.15.131:0/1167907986 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f3f2406be30 0x7f3f2406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:42.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.869+0000 7f3f45600640 1 -- 172.21.15.131:0/1167907986 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5115+0+0 (secure 0 0 0) 0x7f3f3008ab90 con 0x7f3f48072540 2024-05-26T22:52:42.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.869+0000 7f3f4d200640 1 --2- 172.21.15.131:0/1167907986 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f3f2406be30 0x7f3f2406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:42.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.869+0000 7f3f4d200640 1 --2- 172.21.15.131:0/1167907986 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f3f2406be30 0x7f3f2406e2f0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f3f38009870 tx=0x7f3f38008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:42.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:42.873+0000 7f3f45600640 1 -- 172.21.15.131:0/1167907986 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f3f30059780 con 0x7f3f48072540 2024-05-26T22:52:43.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.072+0000 7f3f4fc00640 1 -- 172.21.15.131:0/1167907986 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_export_ephemeral_random"} v 0) -- 0x7f3f480695c0 con 0x7f3f48072540 2024-05-26T22:52:43.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.080+0000 7f3f45600640 1 -- 172.21.15.131:0/1167907986 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_export_ephemeral_random"}]=0 v32) ==== 110+0+0 (secure 0 0 0) 0x7f3f30059080 con 0x7f3f48072540 2024-05-26T22:52:43.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.083+0000 7f3f4fc00640 1 -- 172.21.15.131:0/1167907986 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f3f2406be30 msgr2=0x7f3f2406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:43.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.083+0000 7f3f4fc00640 1 --2- 172.21.15.131:0/1167907986 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f3f2406be30 0x7f3f2406e2f0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f3f38009870 tx=0x7f3f38008040 comp rx=0 tx=0).stop 2024-05-26T22:52:43.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.083+0000 7f3f4fc00640 1 -- 172.21.15.131:0/1167907986 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f48072540 msgr2=0x7f3f480ae830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:43.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.083+0000 7f3f4fc00640 1 --2- 172.21.15.131:0/1167907986 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f48072540 0x7f3f480ae830 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f3f3000c820 tx=0x7f3f3000cc20 comp rx=0 tx=0).stop 2024-05-26T22:52:43.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.083+0000 7f3f4fc00640 1 -- 172.21.15.131:0/1167907986 shutdown_connections 2024-05-26T22:52:43.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.083+0000 7f3f4fc00640 1 --2- 172.21.15.131:0/1167907986 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f480cfe50 0x7f3f4809aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:43.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.083+0000 7f3f4fc00640 1 --2- 172.21.15.131:0/1167907986 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f3f2406be30 0x7f3f2406e2f0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:43.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.083+0000 7f3f4fc00640 1 --2- 172.21.15.131:0/1167907986 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f48072540 0x7f3f480ae830 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:43.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.083+0000 7f3f4fc00640 1 --2- 172.21.15.131:0/1167907986 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3f4806f180 0x7f3f4814dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:43.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.083+0000 7f3f4fc00640 1 -- 172.21.15.131:0/1167907986 >> 172.21.15.131:0/1167907986 conn(0x7f3f4806bc90 msgr2=0x7f3f4815dcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:52:43.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.083+0000 7f3f4fc00640 1 -- 172.21.15.131:0/1167907986 shutdown_connections 2024-05-26T22:52:43.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.083+0000 7f3f4fc00640 1 -- 172.21.15.131:0/1167907986 wait complete. 2024-05-26T22:52:43.101 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_export_ephemeral_distributed 2024-05-26T22:52:43.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.299+0000 7fee38a00640 1 Processor -- start 2024-05-26T22:52:43.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.300+0000 7fee38a00640 1 -- start start 2024-05-26T22:52:43.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.301+0000 7fee38a00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fee3415c220 0x7fee3406e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:43.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.301+0000 7fee38a00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fee3406e9f0 0x7fee3415d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:43.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.301+0000 7fee38a00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee341558d0 con 0x7fee341554f0 2024-05-26T22:52:43.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.301+0000 7fee38a00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fee3406ee80 con 0x7fee3406e9f0 2024-05-26T22:52:43.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.301+0000 7fee38a00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fee340729b0 con 0x7fee3415c220 2024-05-26T22:52:43.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.301+0000 7fee33400640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fee341554f0 0x7fee34071f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:52174/0 (socket says 172.21.15.131:52174) 2024-05-26T22:52:43.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.301+0000 7fee33400640 1 -- 172.21.15.131:0/2691445324 learned_addr learned my addr 172.21.15.131:0/2691445324 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:52:43.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.302+0000 7fee33e00640 1 --2- 172.21.15.131:0/2691445324 >> v2:172.21.15.131:3300/0 conn(0x7fee3406e9f0 0x7fee3415d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:43.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.302+0000 7fee2be00640 1 --2- 172.21.15.131:0/2691445324 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fee3415c220 0x7fee3406e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:43.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.302+0000 7fee32a00640 1 -- 172.21.15.131:0/2691445324 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 879799614 0 0) 0x7fee24001110 con 0x7fee341554f0 2024-05-26T22:52:43.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.302+0000 7fee32a00640 1 -- 172.21.15.131:0/2691445324 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fee18003520 con 0x7fee341554f0 2024-05-26T22:52:43.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.302+0000 7fee33e00640 1 -- 172.21.15.131:0/2691445324 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fee3415c220 msgr2=0x7fee3406e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:43.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.302+0000 7fee33e00640 1 --2- 172.21.15.131:0/2691445324 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fee3415c220 0x7fee3406e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:43.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.302+0000 7fee33e00640 1 -- 172.21.15.131:0/2691445324 >> v1:172.21.15.131:6789/0 conn(0x7fee341554f0 legacy=0x7fee34071f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:43.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.302+0000 7fee33e00640 1 -- 172.21.15.131:0/2691445324 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee3415c750 con 0x7fee3406e9f0 2024-05-26T22:52:43.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.302+0000 7fee2be00640 1 --2- 172.21.15.131:0/2691445324 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fee3415c220 0x7fee3406e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:52:43.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.303+0000 7fee33e00640 1 --2- 172.21.15.131:0/2691445324 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee3406e9f0 0x7fee3415d230 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fee2c009d90 tx=0x7fee2c02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=121969a73c699523 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:43.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.304+0000 7fee32a00640 1 -- 172.21.15.131:0/2691445324 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee2c007740 con 0x7fee3406e9f0 2024-05-26T22:52:43.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.304+0000 7fee32a00640 1 -- 172.21.15.131:0/2691445324 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee2c0078e0 con 0x7fee3406e9f0 2024-05-26T22:52:43.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.304+0000 7fee32a00640 1 -- 172.21.15.131:0/2691445324 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee2c007cb0 con 0x7fee3406e9f0 2024-05-26T22:52:43.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.304+0000 7fee38a00640 1 -- 172.21.15.131:0/2691445324 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee3406e9f0 msgr2=0x7fee3415d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:43.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.304+0000 7fee38a00640 1 --2- 172.21.15.131:0/2691445324 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee3406e9f0 0x7fee3415d230 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fee2c009d90 tx=0x7fee2c02a660 comp rx=0 tx=0).stop 2024-05-26T22:52:43.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.304+0000 7fee38a00640 1 -- 172.21.15.131:0/2691445324 shutdown_connections 2024-05-26T22:52:43.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.304+0000 7fee38a00640 1 --2- 172.21.15.131:0/2691445324 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee3406e9f0 0x7fee3415d230 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:43.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.304+0000 7fee38a00640 1 --2- 172.21.15.131:0/2691445324 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fee3415c220 0x7fee3406e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:43.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.304+0000 7fee38a00640 1 -- 172.21.15.131:0/2691445324 >> 172.21.15.131:0/2691445324 conn(0x7fee340cc7f0 msgr2=0x7fee340ccbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:52:43.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.304+0000 7fee38a00640 1 -- 172.21.15.131:0/2691445324 shutdown_connections 2024-05-26T22:52:43.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.304+0000 7fee38a00640 1 -- 172.21.15.131:0/2691445324 wait complete. 2024-05-26T22:52:43.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.306+0000 7fee38a00640 1 Processor -- start 2024-05-26T22:52:43.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.306+0000 7fee38a00640 1 -- start start 2024-05-26T22:52:43.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.307+0000 7fee38a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee3406e9f0 0x7fee340aec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:43.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.307+0000 7fee38a00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fee341554f0 0x7fee3414b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:43.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.307+0000 7fee38a00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fee3415c220 0x7fee341542b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:43.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.307+0000 7fee38a00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fee34072670 con 0x7fee3406e9f0 2024-05-26T22:52:43.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.307+0000 7fee38a00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fee3415cd10 con 0x7fee341554f0 2024-05-26T22:52:43.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.307+0000 7fee38a00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fee3415d770 con 0x7fee3415c220 2024-05-26T22:52:43.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.308+0000 7fee33400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee3406e9f0 0x7fee340aec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:43.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.308+0000 7fee33400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee3406e9f0 0x7fee340aec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49972/0 (socket says 172.21.15.131:49972) 2024-05-26T22:52:43.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.308+0000 7fee33400640 1 -- 172.21.15.131:0/4127674756 learned_addr learned my addr 172.21.15.131:0/4127674756 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:52:43.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.308+0000 7fee33e00640 1 --2- 172.21.15.131:0/4127674756 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fee3415c220 0x7fee341542b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:43.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.308+0000 7fee2be00640 1 --2- 172.21.15.131:0/4127674756 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fee341554f0 0x7fee3414b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:43.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.308+0000 7fee33400640 1 -- 172.21.15.131:0/4127674756 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fee3415c220 msgr2=0x7fee341542b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:43.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.308+0000 7fee33400640 1 --2- 172.21.15.131:0/4127674756 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fee3415c220 0x7fee341542b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:43.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.308+0000 7fee33400640 1 -- 172.21.15.131:0/4127674756 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fee341554f0 msgr2=0x7fee3414b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:43.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.308+0000 7fee33400640 1 --2- 172.21.15.131:0/4127674756 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fee341554f0 0x7fee3414b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:43.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.308+0000 7fee33400640 1 -- 172.21.15.131:0/4127674756 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee340af180 con 0x7fee3406e9f0 2024-05-26T22:52:43.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.308+0000 7fee33e00640 1 --2- 172.21.15.131:0/4127674756 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fee3415c220 0x7fee341542b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:52:43.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.308+0000 7fee2be00640 1 --2- 172.21.15.131:0/4127674756 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fee341554f0 0x7fee3414b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:52:43.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.308+0000 7fee33400640 1 --2- 172.21.15.131:0/4127674756 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee3406e9f0 0x7fee340aec40 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fee2400c730 tx=0x7fee2400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:43.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.309+0000 7fee2b400640 1 -- 172.21.15.131:0/4127674756 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee24017070 con 0x7fee3406e9f0 2024-05-26T22:52:43.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.309+0000 7fee2b400640 1 -- 172.21.15.131:0/4127674756 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee24005ad0 con 0x7fee3406e9f0 2024-05-26T22:52:43.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.309+0000 7fee38a00640 1 -- 172.21.15.131:0/4127674756 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee340b3820 con 0x7fee3406e9f0 2024-05-26T22:52:43.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.309+0000 7fee2b400640 1 -- 172.21.15.131:0/4127674756 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee24007e20 con 0x7fee3406e9f0 2024-05-26T22:52:43.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.309+0000 7fee38a00640 1 -- 172.21.15.131:0/4127674756 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee3414fe30 con 0x7fee3406e9f0 2024-05-26T22:52:43.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.310+0000 7fee38a00640 1 -- 172.21.15.131:0/4127674756 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee340694e0 con 0x7fee3406e9f0 2024-05-26T22:52:43.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.312+0000 7fee2b400640 1 -- 172.21.15.131:0/4127674756 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fee2401a070 con 0x7fee3406e9f0 2024-05-26T22:52:43.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.313+0000 7fee2b400640 1 --2- 172.21.15.131:0/4127674756 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fee1006be30 0x7fee1006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:43.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.313+0000 7fee2b400640 1 -- 172.21.15.131:0/4127674756 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5115+0+0 (secure 0 0 0) 0x7fee24089fb0 con 0x7fee3406e9f0 2024-05-26T22:52:43.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.313+0000 7fee2be00640 1 --2- 172.21.15.131:0/4127674756 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fee1006be30 0x7fee1006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:43.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.314+0000 7fee2be00640 1 --2- 172.21.15.131:0/4127674756 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fee1006be30 0x7fee1006e2f0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fee1c006fd0 tx=0x7fee1c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:43.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.319+0000 7fee2b400640 1 -- 172.21.15.131:0/4127674756 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fee341558d0 con 0x7fee3406e9f0 2024-05-26T22:52:43.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.519+0000 7fee38a00640 1 -- 172.21.15.131:0/4127674756 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_export_ephemeral_distributed"} v 0) -- 0x7fee340ce430 con 0x7fee3406e9f0 2024-05-26T22:52:43.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.524+0000 7fee2b400640 1 -- 172.21.15.131:0/4127674756 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_export_ephemeral_distributed"}]=0 v33) ==== 115+0+0 (secure 0 0 0) 0x7fee24058c40 con 0x7fee3406e9f0 2024-05-26T22:52:43.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.527+0000 7fee38a00640 1 -- 172.21.15.131:0/4127674756 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fee1006be30 msgr2=0x7fee1006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:43.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.527+0000 7fee38a00640 1 --2- 172.21.15.131:0/4127674756 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fee1006be30 0x7fee1006e2f0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fee1c006fd0 tx=0x7fee1c008040 comp rx=0 tx=0).stop 2024-05-26T22:52:43.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.527+0000 7fee38a00640 1 -- 172.21.15.131:0/4127674756 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee3406e9f0 msgr2=0x7fee340aec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:43.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.527+0000 7fee38a00640 1 --2- 172.21.15.131:0/4127674756 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee3406e9f0 0x7fee340aec40 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fee2400c730 tx=0x7fee2400cbf0 comp rx=0 tx=0).stop 2024-05-26T22:52:43.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.527+0000 7fee38a00640 1 -- 172.21.15.131:0/4127674756 shutdown_connections 2024-05-26T22:52:43.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.527+0000 7fee38a00640 1 --2- 172.21.15.131:0/4127674756 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fee3415c220 0x7fee341542b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:43.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.527+0000 7fee38a00640 1 --2- 172.21.15.131:0/4127674756 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fee1006be30 0x7fee1006e2f0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:43.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.527+0000 7fee38a00640 1 --2- 172.21.15.131:0/4127674756 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fee341554f0 0x7fee3414b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:43.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.527+0000 7fee38a00640 1 --2- 172.21.15.131:0/4127674756 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee3406e9f0 0x7fee340aec40 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:43.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.527+0000 7fee38a00640 1 -- 172.21.15.131:0/4127674756 >> 172.21.15.131:0/4127674756 conn(0x7fee340cc7f0 msgr2=0x7fee34159ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:52:43.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.527+0000 7fee38a00640 1 -- 172.21.15.131:0/4127674756 shutdown_connections 2024-05-26T22:52:43.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.528+0000 7fee38a00640 1 -- 172.21.15.131:0/4127674756 wait complete. 2024-05-26T22:52:43.546 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-26T22:52:43.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.746+0000 7f7354a00640 1 Processor -- start 2024-05-26T22:52:43.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.746+0000 7f7354a00640 1 -- start start 2024-05-26T22:52:43.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.748+0000 7f7354a00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f735015e5b0 0x7f73500ced40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:43.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.748+0000 7f7354a00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f73500b3430 0x7f73500cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:43.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.748+0000 7f7354a00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73500d2780 con 0x7f73500cf7c0 2024-05-26T22:52:43.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.748+0000 7f7354a00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f73500d2a60 con 0x7f73500b3430 2024-05-26T22:52:43.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.748+0000 7f7354a00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f73500d2d40 con 0x7f735015e5b0 2024-05-26T22:52:43.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.748+0000 7f734f200640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f73500cf7c0 0x7f73500d09e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:52186/0 (socket says 172.21.15.131:52186) 2024-05-26T22:52:43.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.748+0000 7f734f200640 1 -- 172.21.15.131:0/1997927166 learned_addr learned my addr 172.21.15.131:0/1997927166 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:52:43.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.749+0000 7f734e800640 1 --2- 172.21.15.131:0/1997927166 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f735015e5b0 0x7f73500ced40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:43.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.749+0000 7f734d400640 1 -- 172.21.15.131:0/1997927166 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 12951653 0 0) 0x7f73500d2780 con 0x7f73500cf7c0 2024-05-26T22:52:43.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.749+0000 7f734d400640 1 -- 172.21.15.131:0/1997927166 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7334003520 con 0x7f73500cf7c0 2024-05-26T22:52:43.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.749+0000 7f734de00640 1 --2- 172.21.15.131:0/1997927166 >> v2:172.21.15.187:3300/0 conn(0x7f73500b3430 0x7f73500cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:43.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.749+0000 7f734d400640 1 -- 172.21.15.131:0/1997927166 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3488318214 0 0) 0x7f7334003520 con 0x7f73500cf7c0 2024-05-26T22:52:43.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.749+0000 7f734d400640 1 -- 172.21.15.131:0/1997927166 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f735015e5b0 msgr2=0x7f73500ced40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:43.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.749+0000 7f734d400640 1 --2- 172.21.15.131:0/1997927166 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f735015e5b0 0x7f73500ced40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:43.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.749+0000 7f734d400640 1 -- 172.21.15.131:0/1997927166 >> v2:172.21.15.187:3300/0 conn(0x7f73500b3430 msgr2=0x7f73500cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:43.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.749+0000 7f734d400640 1 --2- 172.21.15.131:0/1997927166 >> v2:172.21.15.187:3300/0 conn(0x7f73500b3430 0x7f73500cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:43.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.749+0000 7f734d400640 1 -- 172.21.15.131:0/1997927166 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73500d3020 con 0x7f73500cf7c0 2024-05-26T22:52:43.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.749+0000 7f734d400640 1 -- 172.21.15.131:0/1997927166 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7f7348001670 con 0x7f73500cf7c0 2024-05-26T22:52:43.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.749+0000 7f734d400640 1 -- 172.21.15.131:0/1997927166 >> v1:172.21.15.131:6789/0 conn(0x7f73500cf7c0 legacy=0x7f73500d09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:43.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.749+0000 7f734d400640 1 --2- 172.21.15.131:0/1997927166 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f73340039b0 0x7f7334003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:43.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.750+0000 7f734d400640 1 --2- 172.21.15.131:0/1997927166 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7334004800 0x7f7334004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:43.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.750+0000 7f734d400640 1 --2- 172.21.15.131:0/1997927166 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7334005180 0x7f733400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:43.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.750+0000 7f734d400640 1 -- 172.21.15.131:0/1997927166 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f73500d2a60 con 0x7f7334005180 2024-05-26T22:52:43.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.750+0000 7f734d400640 1 -- 172.21.15.131:0/1997927166 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f73500d2d40 con 0x7f73340039b0 2024-05-26T22:52:43.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.750+0000 7f734d400640 1 -- 172.21.15.131:0/1997927166 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f733400bf00 con 0x7f7334004800 2024-05-26T22:52:43.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.750+0000 7f734f200640 1 --2- 172.21.15.131:0/1997927166 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7334005180 0x7f733400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:43.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.750+0000 7f734de00640 1 --2- 172.21.15.131:0/1997927166 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7334004800 0x7f7334004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:43.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.750+0000 7f734e800640 1 --2- 172.21.15.131:0/1997927166 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f73340039b0 0x7f7334003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:43.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.750+0000 7f734f200640 1 -- 172.21.15.131:0/1997927166 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7334004800 msgr2=0x7f7334004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:43.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.750+0000 7f734f200640 1 --2- 172.21.15.131:0/1997927166 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7334004800 0x7f7334004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:43.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.750+0000 7f734f200640 1 -- 172.21.15.131:0/1997927166 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f73340039b0 msgr2=0x7f7334003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:43.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.750+0000 7f734f200640 1 --2- 172.21.15.131:0/1997927166 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f73340039b0 0x7f7334003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:43.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.750+0000 7f734f200640 1 -- 172.21.15.131:0/1997927166 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7334003520 con 0x7f7334005180 2024-05-26T22:52:43.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.750+0000 7f734e800640 1 --2- 172.21.15.131:0/1997927166 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f73340039b0 0x7f7334003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:52:43.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.751+0000 7f734f200640 1 --2- 172.21.15.131:0/1997927166 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7334005180 0x7f733400b9c0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f734800c190 tx=0x7f7334005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c8d175853bd64cc server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:43.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.752+0000 7f734d400640 1 -- 172.21.15.131:0/1997927166 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7348007c80 con 0x7f7334005180 2024-05-26T22:52:43.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.752+0000 7f734d400640 1 -- 172.21.15.131:0/1997927166 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7348005ad0 con 0x7f7334005180 2024-05-26T22:52:43.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.752+0000 7f734d400640 1 -- 172.21.15.131:0/1997927166 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7348007070 con 0x7f7334005180 2024-05-26T22:52:43.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.752+0000 7f7354a00640 1 -- 172.21.15.131:0/1997927166 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7334005180 msgr2=0x7f733400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:43.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.752+0000 7f7354a00640 1 --2- 172.21.15.131:0/1997927166 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7334005180 0x7f733400b9c0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f734800c190 tx=0x7f7334005830 comp rx=0 tx=0).stop 2024-05-26T22:52:43.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.752+0000 7f734f200640 1 -- 172.21.15.131:0/1997927166 reap_dead start 2024-05-26T22:52:43.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.752+0000 7f7354a00640 1 -- 172.21.15.131:0/1997927166 shutdown_connections 2024-05-26T22:52:43.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.752+0000 7f7354a00640 1 -- 172.21.15.131:0/1997927166 >> 172.21.15.131:0/1997927166 conn(0x7f735009d4d0 msgr2=0x7f73500d41a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:52:43.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.752+0000 7f7354a00640 1 -- 172.21.15.131:0/1997927166 shutdown_connections 2024-05-26T22:52:43.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.752+0000 7f7354a00640 1 -- 172.21.15.131:0/1997927166 wait complete. 2024-05-26T22:52:43.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.754+0000 7f7354a00640 1 Processor -- start 2024-05-26T22:52:43.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.754+0000 7f7354a00640 1 -- start start 2024-05-26T22:52:43.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.755+0000 7f7354a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7334005180 0x7f73500cd2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:43.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.755+0000 7f7354a00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f73500c7c30 0x7f7350152d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:43.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.755+0000 7f7354a00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f73500c9e60 0x7f73500ca2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:43.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.755+0000 7f7354a00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f73500b5030 con 0x7f7334005180 2024-05-26T22:52:43.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.755+0000 7f7354a00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f73500cfc10 con 0x7f73500c9e60 2024-05-26T22:52:43.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.755+0000 7f7354a00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f73500b51b0 con 0x7f73500c7c30 2024-05-26T22:52:43.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.756+0000 7f734e800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7334005180 0x7f73500cd2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:43.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.756+0000 7f734e800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7334005180 0x7f73500cd2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49994/0 (socket says 172.21.15.131:49994) 2024-05-26T22:52:43.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.756+0000 7f734e800640 1 -- 172.21.15.131:0/1403508990 learned_addr learned my addr 172.21.15.131:0/1403508990 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:52:43.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.756+0000 7f734e800640 1 -- 172.21.15.131:0/1403508990 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f73500c7c30 msgr2=0x7f7350152d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:43.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.756+0000 7f734e800640 1 --2- 172.21.15.131:0/1403508990 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f73500c7c30 0x7f7350152d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:43.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.756+0000 7f734e800640 1 -- 172.21.15.131:0/1403508990 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f73500c9e60 msgr2=0x7f73500ca2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:43.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.756+0000 7f734e800640 1 --2- 172.21.15.131:0/1403508990 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f73500c9e60 0x7f73500ca2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:43.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.756+0000 7f734e800640 1 -- 172.21.15.131:0/1403508990 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73500c84f0 con 0x7f7334005180 2024-05-26T22:52:43.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.756+0000 7f734e800640 1 --2- 172.21.15.131:0/1403508990 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7334005180 0x7f73500cd2e0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f73380108e0 tx=0x7f7338010da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:43.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.757+0000 7f7346a00640 1 -- 172.21.15.131:0/1403508990 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f733800f040 con 0x7f7334005180 2024-05-26T22:52:43.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.757+0000 7f7346a00640 1 -- 172.21.15.131:0/1403508990 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f733800eb40 con 0x7f7334005180 2024-05-26T22:52:43.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.757+0000 7f7346a00640 1 -- 172.21.15.131:0/1403508990 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f733801c400 con 0x7f7334005180 2024-05-26T22:52:43.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.757+0000 7f7354a00640 1 -- 172.21.15.131:0/1403508990 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73500c87d0 con 0x7f7334005180 2024-05-26T22:52:43.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.757+0000 7f7354a00640 1 -- 172.21.15.131:0/1403508990 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73500c8d90 con 0x7f7334005180 2024-05-26T22:52:43.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.758+0000 7f7354a00640 1 -- 172.21.15.131:0/1403508990 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73500674d0 con 0x7f7334005180 2024-05-26T22:52:43.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.759+0000 7f7346a00640 1 -- 172.21.15.131:0/1403508990 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f733801c5a0 con 0x7f7334005180 2024-05-26T22:52:43.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.760+0000 7f7346a00640 1 --2- 172.21.15.131:0/1403508990 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f732c06be30 0x7f732c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:43.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.760+0000 7f7346a00640 1 -- 172.21.15.131:0/1403508990 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5115+0+0 (secure 0 0 0) 0x7f733808b380 con 0x7f7334005180 2024-05-26T22:52:43.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.760+0000 7f734de00640 1 --2- 172.21.15.131:0/1403508990 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f732c06be30 0x7f732c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:43.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.761+0000 7f734de00640 1 --2- 172.21.15.131:0/1403508990 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f732c06be30 0x7f732c06e2f0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f735006df70 tx=0x7f73400039b0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:43.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.764+0000 7f7346a00640 1 -- 172.21.15.131:0/1403508990 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f73500674d0 con 0x7f7334005180 2024-05-26T22:52:43.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.963+0000 7f7354a00640 1 -- 172.21.15.131:0/1403508990 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f73500694e0 con 0x7f7334005180 2024-05-26T22:52:43.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.970+0000 7f7346a00640 1 -- 172.21.15.131:0/1403508990 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v34) ==== 104+0+0 (secure 0 0 0) 0x7f7338018030 con 0x7f7334005180 2024-05-26T22:52:43.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.972+0000 7f7354a00640 1 -- 172.21.15.131:0/1403508990 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f732c06be30 msgr2=0x7f732c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:43.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.972+0000 7f7354a00640 1 --2- 172.21.15.131:0/1403508990 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f732c06be30 0x7f732c06e2f0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f735006df70 tx=0x7f73400039b0 comp rx=0 tx=0).stop 2024-05-26T22:52:43.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.972+0000 7f7354a00640 1 -- 172.21.15.131:0/1403508990 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7334005180 msgr2=0x7f73500cd2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:43.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.972+0000 7f7354a00640 1 --2- 172.21.15.131:0/1403508990 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7334005180 0x7f73500cd2e0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f73380108e0 tx=0x7f7338010da0 comp rx=0 tx=0).stop 2024-05-26T22:52:43.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.973+0000 7f7354a00640 1 -- 172.21.15.131:0/1403508990 shutdown_connections 2024-05-26T22:52:43.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.973+0000 7f7354a00640 1 --2- 172.21.15.131:0/1403508990 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f73500c7c30 0x7f7350152d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:43.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.973+0000 7f7354a00640 1 --2- 172.21.15.131:0/1403508990 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f732c06be30 0x7f732c06e2f0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:43.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.973+0000 7f7354a00640 1 --2- 172.21.15.131:0/1403508990 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f73500c9e60 0x7f73500ca2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:43.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.973+0000 7f7354a00640 1 --2- 172.21.15.131:0/1403508990 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7334005180 0x7f73500cd2e0 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:43.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.973+0000 7f7354a00640 1 -- 172.21.15.131:0/1403508990 >> 172.21.15.131:0/1403508990 conn(0x7f735009d4d0 msgr2=0x7f73500d0ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:52:43.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.973+0000 7f7354a00640 1 -- 172.21.15.131:0/1403508990 shutdown_connections 2024-05-26T22:52:43.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:43.973+0000 7f7354a00640 1 -- 172.21.15.131:0/1403508990 wait complete. 2024-05-26T22:52:43.990 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_exports.TestEphemeralPins.test_ephemeral_pin_dist_off' 2024-05-26T22:52:44.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:44.189+0000 7f1003e00640 1 Processor -- start 2024-05-26T22:52:44.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:44.190+0000 7f1003e00640 1 -- start start 2024-05-26T22:52:44.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:44.191+0000 7f1003e00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f100415e540 0x7f100406b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:44.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:44.191+0000 7f1003e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f10040b3f30 0x7f10040ad870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:44.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:44.191+0000 7f1003e00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10040b4680 con 0x7f1004162190 2024-05-26T22:52:44.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:44.191+0000 7f1003e00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f100406b850 con 0x7f10040b3f30 2024-05-26T22:52:44.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:44.191+0000 7f1003e00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f100406ba80 con 0x7f100415e540 2024-05-26T22:52:44.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:44.192+0000 7f1002000640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f1004162190 0x7f10040b3820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:52196/0 (socket says 172.21.15.131:52196) 2024-05-26T22:52:44.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:44.192+0000 7f1002000640 1 -- 172.21.15.131:0/778684968 learned_addr learned my addr 172.21.15.131:0/778684968 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:52:44.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:44.192+0000 7f1003400640 1 --2- 172.21.15.131:0/778684968 >> v2:172.21.15.131:3300/0 conn(0x7f10040b3f30 0x7f10040ad870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:44.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:44.192+0000 7f1002a00640 1 --2- 172.21.15.131:0/778684968 >> v2:172.21.15.187:3300/0 conn(0x7f100415e540 0x7f100406b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:44.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:44.192+0000 7f1001600640 1 -- 172.21.15.131:0/778684968 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2485024776 0 0) 0x7f10040b4680 con 0x7f1004162190 2024-05-26T22:52:44.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:44.192+0000 7f1001600640 1 -- 172.21.15.131:0/778684968 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0fe8003520 con 0x7f1004162190 2024-05-26T22:52:44.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:44.192+0000 7f1003400640 1 -- 172.21.15.131:0/778684968 >> v2:172.21.15.187:3300/0 conn(0x7f100415e540 msgr2=0x7f100406b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:44.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:44.192+0000 7f1003400640 1 --2- 172.21.15.131:0/778684968 >> v2:172.21.15.187:3300/0 conn(0x7f100415e540 0x7f100406b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:44.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:44.192+0000 7f1001600640 1 -- 172.21.15.131:0/778684968 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2851378698 0 0) 0x7f0fe8003520 con 0x7f1004162190 2024-05-26T22:52:44.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:44.192+0000 7f1003400640 1 -- 172.21.15.131:0/778684968 >> v1:172.21.15.131:6789/0 conn(0x7f1004162190 legacy=0x7f10040b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:44.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:44.192+0000 7f1003400640 1 -- 172.21.15.131:0/778684968 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f100406bd60 con 0x7f10040b3f30 2024-05-26T22:52:44.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:44.193+0000 7f1003400640 1 --2- 172.21.15.131:0/778684968 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f10040b3f30 0x7f10040ad870 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f100406d680 tx=0x7f0ffc02a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2347c59becde58ef server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:44.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:44.194+0000 7f1001600640 1 -- 172.21.15.131:0/778684968 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ffc037070 con 0x7f10040b3f30 2024-05-26T22:52:44.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:44.194+0000 7f1001600640 1 -- 172.21.15.131:0/778684968 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ffc02abc0 con 0x7f10040b3f30 2024-05-26T22:52:44.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:44.194+0000 7f1001600640 1 -- 172.21.15.131:0/778684968 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fec0019f0 con 0x7f10040b3f30 2024-05-26T22:52:44.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:44.194+0000 7f1003e00640 1 -- 172.21.15.131:0/778684968 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f10040b3f30 msgr2=0x7f10040ad870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:44.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:44.194+0000 7f1003e00640 1 --2- 172.21.15.131:0/778684968 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f10040b3f30 0x7f10040ad870 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f100406d680 tx=0x7f0ffc02a130 comp rx=0 tx=0).stop 2024-05-26T22:52:44.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:44.194+0000 7f1003e00640 1 -- 172.21.15.131:0/778684968 shutdown_connections 2024-05-26T22:52:44.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:44.194+0000 7f1003e00640 1 --2- 172.21.15.131:0/778684968 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f10040b3f30 0x7f10040ad870 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:44.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:44.194+0000 7f1003e00640 1 --2- 172.21.15.131:0/778684968 >> v2:172.21.15.187:3300/0 conn(0x7f100415e540 0x7f100406b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:44.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:44.194+0000 7f1003e00640 1 -- 172.21.15.131:0/778684968 >> 172.21.15.131:0/778684968 conn(0x7f10040d07b0 msgr2=0x7f10040d0b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:52:44.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:44.194+0000 7f1003e00640 1 -- 172.21.15.131:0/778684968 shutdown_connections 2024-05-26T22:52:44.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:44.194+0000 7f1003e00640 1 -- 172.21.15.131:0/778684968 wait complete. 2024-05-26T22:52:44.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:44.196+0000 7f1003e00640 1 Processor -- start 2024-05-26T22:52:44.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:44.196+0000 7f1003e00640 1 -- start start 2024-05-26T22:52:44.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:44.197+0000 7f1003e00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f10040b3f30 0x7f10041f1c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:44.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:44.197+0000 7f1003e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f100415e540 0x7f1004159460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:44.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:44.197+0000 7f1003e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1004162190 0x7f10041f4a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:44.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:44.197+0000 7f1003e00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f100406c600 con 0x7f1004162190 2024-05-26T22:52:44.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:44.197+0000 7f1003e00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f100406c320 con 0x7f10040b3f30 2024-05-26T22:52:44.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:44.197+0000 7f1003e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f100415e920 con 0x7f100415e540 2024-05-26T22:52:44.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:44.197+0000 7f1003400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1004162190 0x7f10041f4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:44.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:44.197+0000 7f1003400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1004162190 0x7f10041f4a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50016/0 (socket says 172.21.15.131:50016) 2024-05-26T22:52:44.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:44.197+0000 7f1003400640 1 -- 172.21.15.131:0/3485062085 learned_addr learned my addr 172.21.15.131:0/3485062085 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:52:44.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:44.198+0000 7f1003400640 1 -- 172.21.15.131:0/3485062085 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f100415e540 msgr2=0x7f1004159460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:44.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:44.198+0000 7f1003400640 1 --2- 172.21.15.131:0/3485062085 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f100415e540 0x7f1004159460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:44.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:44.198+0000 7f1003400640 1 -- 172.21.15.131:0/3485062085 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f10040b3f30 msgr2=0x7f10041f1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:44.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:44.198+0000 7f1003400640 1 --2- 172.21.15.131:0/3485062085 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f10040b3f30 0x7f10041f1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:44.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:44.198+0000 7f1003400640 1 -- 172.21.15.131:0/3485062085 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10041f4fd0 con 0x7f1004162190 2024-05-26T22:52:44.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:44.198+0000 7f1003400640 1 --2- 172.21.15.131:0/3485062085 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1004162190 0x7f10041f4a90 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f0ffc009890 tx=0x7f0ffc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:44.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:44.198+0000 7f0ffaa00640 1 -- 172.21.15.131:0/3485062085 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ffc041020 con 0x7f1004162190 2024-05-26T22:52:44.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:44.198+0000 7f0ffaa00640 1 -- 172.21.15.131:0/3485062085 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ffc03d620 con 0x7f1004162190 2024-05-26T22:52:44.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:44.198+0000 7f1003e00640 1 -- 172.21.15.131:0/3485062085 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10041f5670 con 0x7f1004162190 2024-05-26T22:52:44.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:44.198+0000 7f0ffaa00640 1 -- 172.21.15.131:0/3485062085 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ffc037040 con 0x7f1004162190 2024-05-26T22:52:44.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:44.199+0000 7f1003e00640 1 -- 172.21.15.131:0/3485062085 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10041f5c30 con 0x7f1004162190 2024-05-26T22:52:44.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:44.200+0000 7f1003e00640 1 -- 172.21.15.131:0/3485062085 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10040694e0 con 0x7f1004162190 2024-05-26T22:52:44.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:44.201+0000 7f0ffaa00640 1 -- 172.21.15.131:0/3485062085 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0ffc044020 con 0x7f1004162190 2024-05-26T22:52:44.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:44.202+0000 7f0ffaa00640 1 --2- 172.21.15.131:0/3485062085 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f0fe006be30 0x7f0fe006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:44.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:44.202+0000 7f0ffaa00640 1 -- 172.21.15.131:0/3485062085 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5115+0+0 (secure 0 0 0) 0x7f0ffc0ab160 con 0x7f1004162190 2024-05-26T22:52:44.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:44.202+0000 7f1002a00640 1 --2- 172.21.15.131:0/3485062085 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f0fe006be30 0x7f0fe006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:44.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:44.203+0000 7f1002a00640 1 --2- 172.21.15.131:0/3485062085 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f0fe006be30 0x7f0fe006e2f0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f0ff4006a90 tx=0x7f0ff4006a20 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:44.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:44.206+0000 7f0ffaa00640 1 -- 172.21.15.131:0/3485062085 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f0ffc04b030 con 0x7f1004162190 2024-05-26T22:52:44.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:44.470+0000 7f1003e00640 1 -- 172.21.15.131:0/3485062085 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_exports.TestEphemeralPins.test_ephemeral_pin_dist_off"]} v 0) -- 0x7f10040674f0 con 0x7f1004162190 2024-05-26T22:52:44.955 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-26T22:52:44.964 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-26T22:52:45.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.196+0000 7f0ffaa00640 1 -- 172.21.15.131:0/3485062085 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_exports.TestEphemeralPins.test_ephemeral_pin_dist_off"]}]=0 v232) ==== 150+0+0 (secure 0 0 0) 0x7f10041f4fd0 con 0x7f1004162190 2024-05-26T22:52:45.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.200+0000 7f1003e00640 1 -- 172.21.15.131:0/3485062085 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f0fe006be30 msgr2=0x7f0fe006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:45.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.200+0000 7f1003e00640 1 --2- 172.21.15.131:0/3485062085 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f0fe006be30 0x7f0fe006e2f0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f0ff4006a90 tx=0x7f0ff4006a20 comp rx=0 tx=0).stop 2024-05-26T22:52:45.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.200+0000 7f1003e00640 1 -- 172.21.15.131:0/3485062085 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1004162190 msgr2=0x7f10041f4a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:45.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.200+0000 7f1003e00640 1 --2- 172.21.15.131:0/3485062085 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1004162190 0x7f10041f4a90 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f0ffc009890 tx=0x7f0ffc0046c0 comp rx=0 tx=0).stop 2024-05-26T22:52:45.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.200+0000 7f1003e00640 1 -- 172.21.15.131:0/3485062085 shutdown_connections 2024-05-26T22:52:45.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.200+0000 7f1003e00640 1 --2- 172.21.15.131:0/3485062085 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f100415e540 0x7f1004159460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:45.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.200+0000 7f1003e00640 1 --2- 172.21.15.131:0/3485062085 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f0fe006be30 0x7f0fe006e2f0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:45.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.200+0000 7f1003e00640 1 --2- 172.21.15.131:0/3485062085 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1004162190 0x7f10041f4a90 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:45.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.200+0000 7f1003e00640 1 --2- 172.21.15.131:0/3485062085 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f10040b3f30 0x7f10041f1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:45.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.200+0000 7f1003e00640 1 -- 172.21.15.131:0/3485062085 >> 172.21.15.131:0/3485062085 conn(0x7f10040d07b0 msgr2=0x7f100415d680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:52:45.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.200+0000 7f1003e00640 1 -- 172.21.15.131:0/3485062085 shutdown_connections 2024-05-26T22:52:45.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.200+0000 7f1003e00640 1 -- 172.21.15.131:0/3485062085 wait complete. 2024-05-26T22:52:45.221 INFO:tasks.cephfs_test_runner:That turning off ephemeral distributed pin merges subtrees. ... ok 2024-05-26T22:52:45.221 INFO:tasks.cephfs_test_runner:Starting test: test_ephemeral_pin_dist_override (tasks.cephfs.test_exports.TestEphemeralPins) That an ephemeral distributed pin overrides a normal export pin. 2024-05-26T22:52:45.221 INFO:tasks.cephfs_test_runner:test_ephemeral_pin_dist_override (tasks.cephfs.test_exports.TestEphemeralPins) 2024-05-26T22:52:45.222 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-26T22:52:45.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.422+0000 7f7401600640 1 Processor -- start 2024-05-26T22:52:45.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.422+0000 7f7401600640 1 -- start start 2024-05-26T22:52:45.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.423+0000 7f7401600640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f73fc144800 0x7f73fc15fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:45.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.423+0000 7f7401600640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73fc15f530 con 0x7f73fc0ad870 2024-05-26T22:52:45.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.423+0000 7f7401600640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73fc15f7b0 con 0x7f73fc15f100 2024-05-26T22:52:45.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.423+0000 7f7401600640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f73fc15ffd0 con 0x7f73fc144800 2024-05-26T22:52:45.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.424+0000 7f73f3400640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f73fc0ad870 0x7f73fc15e9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:58298/0 (socket says 172.21.15.131:58298) 2024-05-26T22:52:45.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.424+0000 7f73f3400640 1 -- 172.21.15.131:0/1488699135 learned_addr learned my addr 172.21.15.131:0/1488699135 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:52:45.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.424+0000 7f73fbe00640 1 --2- 172.21.15.131:0/1488699135 >> v2:172.21.15.187:3300/0 conn(0x7f73fc144800 0x7f73fc15fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:45.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.424+0000 7f73fb400640 1 -- 172.21.15.131:0/1488699135 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1660561567 0 0) 0x7f73e4001110 con 0x7f73fc0ad870 2024-05-26T22:52:45.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.424+0000 7f73fb400640 1 -- 172.21.15.131:0/1488699135 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f73e0003520 con 0x7f73fc0ad870 2024-05-26T22:52:45.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.425+0000 7f73fb400640 1 -- 172.21.15.131:0/1488699135 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2489774681 0 0) 0x7f73e0003520 con 0x7f73fc0ad870 2024-05-26T22:52:45.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.425+0000 7f73fb400640 1 -- 172.21.15.131:0/1488699135 >> v2:172.21.15.187:3300/0 conn(0x7f73fc144800 msgr2=0x7f73fc15fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:45.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.425+0000 7f73fb400640 1 --2- 172.21.15.131:0/1488699135 >> v2:172.21.15.187:3300/0 conn(0x7f73fc144800 0x7f73fc15fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:45.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.425+0000 7f73fb400640 1 -- 172.21.15.131:0/1488699135 >> v1:172.21.15.187:6789/0 conn(0x7f73fc15f100 legacy=0x7f73fc0ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:45.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.425+0000 7f73fb400640 1 -- 172.21.15.131:0/1488699135 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73fc0cde40 con 0x7f73fc0ad870 2024-05-26T22:52:45.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.425+0000 7f73fb400640 1 -- 172.21.15.131:0/1488699135 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7f73e4001840 con 0x7f73fc0ad870 2024-05-26T22:52:45.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.425+0000 7f73fb400640 1 -- 172.21.15.131:0/1488699135 >> v1:172.21.15.131:6789/0 conn(0x7f73fc0ad870 legacy=0x7f73fc15e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:45.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.425+0000 7f73fb400640 1 --2- 172.21.15.131:0/1488699135 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f73e00039b0 0x7f73e0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:45.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.425+0000 7f73fb400640 1 --2- 172.21.15.131:0/1488699135 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f73e0004800 0x7f73e0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:45.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.425+0000 7f73fb400640 1 --2- 172.21.15.131:0/1488699135 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f73e00051c0 0x7f73e000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:45.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.425+0000 7f73fb400640 1 -- 172.21.15.131:0/1488699135 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f73fc15ffd0 con 0x7f73e0004800 2024-05-26T22:52:45.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.425+0000 7f73fb400640 1 -- 172.21.15.131:0/1488699135 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f73e000c070 con 0x7f73e00051c0 2024-05-26T22:52:45.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.425+0000 7f73fb400640 1 -- 172.21.15.131:0/1488699135 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f73e000c270 con 0x7f73e00039b0 2024-05-26T22:52:45.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.425+0000 7f73f3400640 1 --2- 172.21.15.131:0/1488699135 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f73e0004800 0x7f73e0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:45.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.425+0000 7f73fbe00640 1 --2- 172.21.15.131:0/1488699135 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f73e00039b0 0x7f73e0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:45.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.425+0000 7f7400c00640 1 --2- 172.21.15.131:0/1488699135 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f73e00051c0 0x7f73e000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:45.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.426+0000 7f73f3400640 1 -- 172.21.15.131:0/1488699135 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f73e00039b0 msgr2=0x7f73e0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:45.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.426+0000 7f73f3400640 1 --2- 172.21.15.131:0/1488699135 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f73e00039b0 0x7f73e0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:45.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.426+0000 7f73f3400640 1 -- 172.21.15.131:0/1488699135 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f73e00051c0 msgr2=0x7f73e000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:45.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.426+0000 7f73f3400640 1 --2- 172.21.15.131:0/1488699135 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f73e00051c0 0x7f73e000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:45.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.426+0000 7f73f3400640 1 -- 172.21.15.131:0/1488699135 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73e0003520 con 0x7f73e0004800 2024-05-26T22:52:45.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.426+0000 7f7400c00640 1 -- 172.21.15.131:0/1488699135 reap_dead start 2024-05-26T22:52:45.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.427+0000 7f73f3400640 1 --2- 172.21.15.131:0/1488699135 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f73e0004800 0x7f73e0004c80 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f73e400c4c0 tx=0x7f73e0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4848403eddc3de4e server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:45.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.427+0000 7f73fb400640 1 -- 172.21.15.131:0/1488699135 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e400c0e0 con 0x7f73e0004800 2024-05-26T22:52:45.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.427+0000 7f73fb400640 1 -- 172.21.15.131:0/1488699135 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73e4034070 con 0x7f73e0004800 2024-05-26T22:52:45.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.427+0000 7f73fb400640 1 -- 172.21.15.131:0/1488699135 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e4005d70 con 0x7f73e0004800 2024-05-26T22:52:45.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.427+0000 7f7401600640 1 -- 172.21.15.131:0/1488699135 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f73e0004800 msgr2=0x7f73e0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:45.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.427+0000 7f7401600640 1 --2- 172.21.15.131:0/1488699135 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f73e0004800 0x7f73e0004c80 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f73e400c4c0 tx=0x7f73e0005830 comp rx=0 tx=0).stop 2024-05-26T22:52:45.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.427+0000 7f7401600640 1 -- 172.21.15.131:0/1488699135 shutdown_connections 2024-05-26T22:52:45.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.427+0000 7f7401600640 1 --2- 172.21.15.131:0/1488699135 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f73e0004800 0x7f73e0004c80 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:45.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.427+0000 7f7401600640 1 -- 172.21.15.131:0/1488699135 >> 172.21.15.131:0/1488699135 conn(0x7f73fc0cff40 msgr2=0x7f73fc0d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:52:45.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.427+0000 7f7401600640 1 -- 172.21.15.131:0/1488699135 shutdown_connections 2024-05-26T22:52:45.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.428+0000 7f7401600640 1 -- 172.21.15.131:0/1488699135 wait complete. 2024-05-26T22:52:45.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.429+0000 7f7401600640 1 Processor -- start 2024-05-26T22:52:45.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.429+0000 7f7401600640 1 -- start start 2024-05-26T22:52:45.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.430+0000 7f7401600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f73e0004800 0x7f73fc139a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:45.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.430+0000 7f7401600640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f73fc13ac60 0x7f73fc13b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:45.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.430+0000 7f7401600640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f73fc13cbe0 0x7f73fc13d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:45.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.430+0000 7f7401600640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f73fc144be0 con 0x7f73e0004800 2024-05-26T22:52:45.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.430+0000 7f7401600640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f73fc0ce400 con 0x7f73fc13cbe0 2024-05-26T22:52:45.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.430+0000 7f7401600640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f73fc15fa90 con 0x7f73fc13ac60 2024-05-26T22:52:45.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.431+0000 7f73fbe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f73e0004800 0x7f73fc139a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:45.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.431+0000 7f73fbe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f73e0004800 0x7f73fc139a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39058/0 (socket says 172.21.15.131:39058) 2024-05-26T22:52:45.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.431+0000 7f73fbe00640 1 -- 172.21.15.131:0/2189381775 learned_addr learned my addr 172.21.15.131:0/2189381775 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:52:45.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.431+0000 7f7400c00640 1 --2- 172.21.15.131:0/2189381775 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f73fc13cbe0 0x7f73fc13d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:45.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.431+0000 7f73f3400640 1 --2- 172.21.15.131:0/2189381775 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f73fc13ac60 0x7f73fc13b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:45.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.431+0000 7f73fbe00640 1 -- 172.21.15.131:0/2189381775 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f73fc13ac60 msgr2=0x7f73fc13b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:45.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.431+0000 7f73fbe00640 1 --2- 172.21.15.131:0/2189381775 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f73fc13ac60 0x7f73fc13b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:45.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.431+0000 7f73fbe00640 1 -- 172.21.15.131:0/2189381775 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f73fc13cbe0 msgr2=0x7f73fc13d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:45.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.431+0000 7f73fbe00640 1 --2- 172.21.15.131:0/2189381775 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f73fc13cbe0 0x7f73fc13d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:45.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.431+0000 7f73fbe00640 1 -- 172.21.15.131:0/2189381775 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73fc13d5a0 con 0x7f73e0004800 2024-05-26T22:52:45.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.431+0000 7f73fbe00640 1 --2- 172.21.15.131:0/2189381775 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f73e0004800 0x7f73fc139a00 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f73ec0114e0 tx=0x7f73ec0119a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:45.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.432+0000 7f73f8c00640 1 -- 172.21.15.131:0/2189381775 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73ec015870 con 0x7f73e0004800 2024-05-26T22:52:45.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.432+0000 7f73f8c00640 1 -- 172.21.15.131:0/2189381775 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73ec015a10 con 0x7f73e0004800 2024-05-26T22:52:45.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.432+0000 7f7401600640 1 -- 172.21.15.131:0/2189381775 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73fc13d850 con 0x7f73e0004800 2024-05-26T22:52:45.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.432+0000 7f73f8c00640 1 -- 172.21.15.131:0/2189381775 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73ec015dc0 con 0x7f73e0004800 2024-05-26T22:52:45.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.432+0000 7f7401600640 1 -- 172.21.15.131:0/2189381775 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73fc13de10 con 0x7f73e0004800 2024-05-26T22:52:45.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.433+0000 7f7401600640 1 -- 172.21.15.131:0/2189381775 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73fc0694e0 con 0x7f73e0004800 2024-05-26T22:52:45.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.434+0000 7f73f8c00640 1 -- 172.21.15.131:0/2189381775 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f73ec013400 con 0x7f73e0004800 2024-05-26T22:52:45.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.435+0000 7f73f8c00640 1 --2- 172.21.15.131:0/2189381775 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f73d806be30 0x7f73d806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:45.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.435+0000 7f73f8c00640 1 -- 172.21.15.131:0/2189381775 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5115+0+0 (secure 0 0 0) 0x7f73ec08b580 con 0x7f73e0004800 2024-05-26T22:52:45.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.435+0000 7f73f3400640 1 --2- 172.21.15.131:0/2189381775 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f73d806be30 0x7f73d806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:45.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.436+0000 7f73f3400640 1 --2- 172.21.15.131:0/2189381775 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f73d806be30 0x7f73d806e2f0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f73e4033000 tx=0x7f73e402e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:45.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.439+0000 7f73f8c00640 1 -- 172.21.15.131:0/2189381775 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f73ec019030 con 0x7f73e0004800 2024-05-26T22:52:45.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.625+0000 7f7401600640 1 -- 172.21.15.131:0/2189381775 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f73fc06eb90 con 0x7f73e0004800 2024-05-26T22:52:45.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.626+0000 7f73f8c00640 1 -- 172.21.15.131:0/2189381775 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) ==== 74+0+13308 (secure 0 0 0) 0x7f73ec05a0e0 con 0x7f73e0004800 2024-05-26T22:52:45.627 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:52:45.627 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":61,"fsid":"46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f","created":"2024-05-26T22:47:57.288905+0000","modified":"2024-05-26T22:52:42.161393+0000","last_up_change":"2024-05-26T22:48:09.602207+0000","last_in_change":"2024-05-26T22:48:00.687341+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-26T22:48:10.463339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"814eaf62-a18d-494b-8e27-fd8fb88adea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6801","nonce":2350940051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6803","nonce":2350940051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6807","nonce":2350940051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6805","nonce":2350940051}]},"public_addr":"172.21.15.131:6801/2350940051","cluster_addr":"172.21.15.131:6803/2350940051","heartbeat_back_addr":"172.21.15.131:6807/2350940051","heartbeat_front_addr":"172.21.15.131:6805/2350940051","state":["exists","up"]},{"osd":1,"uuid":"42e66287-f6c6-4e26-af3c-6dfaa3536d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6825","nonce":1525528247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6827","nonce":1525528247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6831","nonce":1525528247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6829","nonce":1525528247}]},"public_addr":"172.21.15.131:6825/1525528247","cluster_addr":"172.21.15.131:6827/1525528247","heartbeat_back_addr":"172.21.15.131:6831/1525528247","heartbeat_front_addr":"172.21.15.131:6829/1525528247","state":["exists","up"]},{"osd":2,"uuid":"0da34673-04a2-4fd0-b57a-0ff2b679b043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6817","nonce":3627842210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6819","nonce":3627842210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6823","nonce":3627842210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6821","nonce":3627842210}]},"public_addr":"172.21.15.131:6817/3627842210","cluster_addr":"172.21.15.131:6819/3627842210","heartbeat_back_addr":"172.21.15.131:6823/3627842210","heartbeat_front_addr":"172.21.15.131:6821/3627842210","state":["exists","up"]},{"osd":3,"uuid":"cf117a27-d6b2-43d3-ae52-b47988bf8eed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6809","nonce":350790591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6811","nonce":350790591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6815","nonce":350790591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6813","nonce":350790591}]},"public_addr":"172.21.15.131:6809/350790591","cluster_addr":"172.21.15.131:6811/350790591","heartbeat_back_addr":"172.21.15.131:6815/350790591","heartbeat_front_addr":"172.21.15.131:6813/350790591","state":["exists","up"]},{"osd":4,"uuid":"a092ccfc-be89-4a84-88d5-0f781fa715eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6816","nonce":244560995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6819","nonce":244560995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6823","nonce":244560995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6821","nonce":244560995}]},"public_addr":"172.21.15.187:6816/244560995","cluster_addr":"172.21.15.187:6819/244560995","heartbeat_back_addr":"172.21.15.187:6823/244560995","heartbeat_front_addr":"172.21.15.187:6821/244560995","state":["exists","up"]},{"osd":5,"uuid":"7513ba69-dd00-4a35-b190-9e8cc3b2bdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6801","nonce":324619398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6803","nonce":324619398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6812","nonce":324619398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6807","nonce":324619398}]},"public_addr":"172.21.15.187:6801/324619398","cluster_addr":"172.21.15.187:6803/324619398","heartbeat_back_addr":"172.21.15.187:6812/324619398","heartbeat_front_addr":"172.21.15.187:6807/324619398","state":["exists","up"]},{"osd":6,"uuid":"50081810-c093-4d2f-a4d7-b7c695c5400d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6806","nonce":1391347786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6809","nonce":1391347786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6818","nonce":1391347786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6813","nonce":1391347786}]},"public_addr":"172.21.15.187:6806/1391347786","cluster_addr":"172.21.15.187:6809/1391347786","heartbeat_back_addr":"172.21.15.187:6818/1391347786","heartbeat_front_addr":"172.21.15.187:6813/1391347786","state":["exists","up"]},{"osd":7,"uuid":"775dae05-54f5-4ad4-bbce-92863c6ed2ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6825","nonce":1220065622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6827","nonce":1220065622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6831","nonce":1220065622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6829","nonce":1220065622}]},"public_addr":"172.21.15.187:6825/1220065622","cluster_addr":"172.21.15.187:6827/1220065622","heartbeat_back_addr":"172.21.15.187:6831/1220065622","heartbeat_front_addr":"172.21.15.187:6829/1220065622","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:05.983520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.299516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.067323+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.307748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.018742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.258775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.010912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.295355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.187:6833/3300872633":"2024-05-27T22:52:39.143366+0000","172.21.15.187:6832/3300872633":"2024-05-27T22:52:39.143366+0000","172.21.15.131:6837/640544945":"2024-05-27T22:52:39.143362+0000","172.21.15.131:6834/640544945":"2024-05-27T22:52:39.143362+0000","172.21.15.131:6836/2737657374":"2024-05-27T22:52:39.143349+0000","172.21.15.131:6835/2737657374":"2024-05-27T22:52:39.143349+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-26T22:52:45.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.629+0000 7f7401600640 1 -- 172.21.15.131:0/2189381775 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f73d806be30 msgr2=0x7f73d806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:45.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.629+0000 7f7401600640 1 --2- 172.21.15.131:0/2189381775 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f73d806be30 0x7f73d806e2f0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f73e4033000 tx=0x7f73e402e000 comp rx=0 tx=0).stop 2024-05-26T22:52:45.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.629+0000 7f7401600640 1 -- 172.21.15.131:0/2189381775 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f73e0004800 msgr2=0x7f73fc139a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:45.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.629+0000 7f7401600640 1 --2- 172.21.15.131:0/2189381775 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f73e0004800 0x7f73fc139a00 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f73ec0114e0 tx=0x7f73ec0119a0 comp rx=0 tx=0).stop 2024-05-26T22:52:45.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.629+0000 7f7401600640 1 -- 172.21.15.131:0/2189381775 shutdown_connections 2024-05-26T22:52:45.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.629+0000 7f7401600640 1 --2- 172.21.15.131:0/2189381775 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f73fc13ac60 0x7f73fc13b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:45.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.629+0000 7f7401600640 1 --2- 172.21.15.131:0/2189381775 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f73d806be30 0x7f73d806e2f0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:45.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.629+0000 7f7401600640 1 --2- 172.21.15.131:0/2189381775 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f73fc13cbe0 0x7f73fc13d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:45.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.629+0000 7f7401600640 1 --2- 172.21.15.131:0/2189381775 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f73e0004800 0x7f73fc139a00 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:45.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.629+0000 7f7401600640 1 -- 172.21.15.131:0/2189381775 >> 172.21.15.131:0/2189381775 conn(0x7f73fc0cff40 msgr2=0x7f73fc072160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:52:45.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.630+0000 7f7401600640 1 -- 172.21.15.131:0/2189381775 shutdown_connections 2024-05-26T22:52:45.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.630+0000 7f7401600640 1 -- 172.21.15.131:0/2189381775 wait complete. 2024-05-26T22:52:45.648 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-26T22:48:10.463339+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-26T22:52:45.648 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-26T22:52:45.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.848+0000 7f9600c00640 1 Processor -- start 2024-05-26T22:52:45.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.848+0000 7f9600c00640 1 -- start start 2024-05-26T22:52:45.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.849+0000 7f9600c00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f95fc06dd30 0x7f95fc06e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:45.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.849+0000 7f9600c00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f95fc0725a0 0x7f95fc0729a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:45.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.849+0000 7f9600c00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95fc15c1c0 con 0x7f95fc155770 2024-05-26T22:52:45.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.849+0000 7f9600c00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f95fc15c4a0 con 0x7f95fc06dd30 2024-05-26T22:52:45.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.849+0000 7f9600c00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f95fc071da0 con 0x7f95fc0725a0 2024-05-26T22:52:45.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.850+0000 7f95f9400640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f95fc06dd30 0x7f95fc06e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:45.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.850+0000 7f95fa800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f95fc0725a0 0x7f95fc0729a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:45.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.850+0000 7f95f9400640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f95fc06dd30 0x7f95fc06e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.131:53162/0 (socket says 172.21.15.131:53162) 2024-05-26T22:52:45.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.850+0000 7f95fa800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f95fc0725a0 0x7f95fc0729a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.131:41746/0 (socket says 172.21.15.131:41746) 2024-05-26T22:52:45.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.850+0000 7f95f9400640 1 -- 172.21.15.131:0/1359135741 learned_addr learned my addr 172.21.15.131:0/1359135741 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:52:45.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.851+0000 7f95fa800640 1 -- 172.21.15.131:0/1359135741 >> v2:172.21.15.187:3300/0 conn(0x7f95fc06dd30 msgr2=0x7f95fc06e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:45.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.851+0000 7f95fa800640 1 --2- 172.21.15.131:0/1359135741 >> v2:172.21.15.187:3300/0 conn(0x7f95fc06dd30 0x7f95fc06e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:45.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.851+0000 7f95fa800640 1 -- 172.21.15.131:0/1359135741 >> v1:172.21.15.187:6789/0 conn(0x7f95fc155770 legacy=0x7f95fc15b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:45.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.851+0000 7f95fa800640 1 -- 172.21.15.131:0/1359135741 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95fc072080 con 0x7f95fc0725a0 2024-05-26T22:52:45.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.851+0000 7f95f9400640 1 --2- 172.21.15.131:0/1359135741 >> v2:172.21.15.187:3300/0 conn(0x7f95fc06dd30 0x7f95fc06e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-26T22:52:45.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.852+0000 7f95fa800640 1 --2- 172.21.15.131:0/1359135741 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f95fc0725a0 0x7f95fc0729a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f95f4009d90 tx=0x7f95f402a6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a8f01c83b42357b server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:45.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.853+0000 7f95f8a00640 1 -- 172.21.15.131:0/1359135741 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95f4007740 con 0x7f95fc0725a0 2024-05-26T22:52:45.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.853+0000 7f95f8a00640 1 -- 172.21.15.131:0/1359135741 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95f40078e0 con 0x7f95fc0725a0 2024-05-26T22:52:45.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.853+0000 7f95f8a00640 1 -- 172.21.15.131:0/1359135741 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95f4007cb0 con 0x7f95fc0725a0 2024-05-26T22:52:45.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.853+0000 7f9600c00640 1 -- 172.21.15.131:0/1359135741 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f95fc0725a0 msgr2=0x7f95fc0729a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:45.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.853+0000 7f9600c00640 1 --2- 172.21.15.131:0/1359135741 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f95fc0725a0 0x7f95fc0729a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f95f4009d90 tx=0x7f95f402a6f0 comp rx=0 tx=0).stop 2024-05-26T22:52:45.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.853+0000 7f9600c00640 1 -- 172.21.15.131:0/1359135741 shutdown_connections 2024-05-26T22:52:45.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.853+0000 7f9600c00640 1 --2- 172.21.15.131:0/1359135741 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f95fc0725a0 0x7f95fc0729a0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:45.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.853+0000 7f9600c00640 1 --2- 172.21.15.131:0/1359135741 >> v2:172.21.15.187:3300/0 conn(0x7f95fc06dd30 0x7f95fc06e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:45.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.853+0000 7f9600c00640 1 -- 172.21.15.131:0/1359135741 >> 172.21.15.131:0/1359135741 conn(0x7f95fc06bc90 msgr2=0x7f95fc0b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:52:45.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.853+0000 7f9600c00640 1 -- 172.21.15.131:0/1359135741 shutdown_connections 2024-05-26T22:52:45.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.853+0000 7f9600c00640 1 -- 172.21.15.131:0/1359135741 wait complete. 2024-05-26T22:52:45.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.855+0000 7f9600c00640 1 Processor -- start 2024-05-26T22:52:45.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.855+0000 7f9600c00640 1 -- start start 2024-05-26T22:52:45.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.856+0000 7f9600c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f95fc06dd30 0x7f95fc141b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:45.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.856+0000 7f9600c00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f95fc0725a0 0x7f95fc168620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:45.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.856+0000 7f9600c00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f95fc155770 0x7f95fc15a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:45.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.856+0000 7f9600c00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f95fc15be10 con 0x7f95fc06dd30 2024-05-26T22:52:45.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.856+0000 7f9600c00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f95fc15c8c0 con 0x7f95fc0725a0 2024-05-26T22:52:45.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.856+0000 7f9600c00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f95fc0b01e0 con 0x7f95fc155770 2024-05-26T22:52:45.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.856+0000 7f95f9e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f95fc06dd30 0x7f95fc141b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:45.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.856+0000 7f95f9e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f95fc06dd30 0x7f95fc141b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39070/0 (socket says 172.21.15.131:39070) 2024-05-26T22:52:45.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.856+0000 7f95f9e00640 1 -- 172.21.15.131:0/1946942450 learned_addr learned my addr 172.21.15.131:0/1946942450 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:52:45.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.857+0000 7f95fa800640 1 --2- 172.21.15.131:0/1946942450 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f95fc155770 0x7f95fc15a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:45.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.857+0000 7f95f9400640 1 --2- 172.21.15.131:0/1946942450 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f95fc0725a0 0x7f95fc168620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:45.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.857+0000 7f95f9e00640 1 -- 172.21.15.131:0/1946942450 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f95fc155770 msgr2=0x7f95fc15a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:45.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.857+0000 7f95f9e00640 1 --2- 172.21.15.131:0/1946942450 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f95fc155770 0x7f95fc15a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:45.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.857+0000 7f95f9e00640 1 -- 172.21.15.131:0/1946942450 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f95fc0725a0 msgr2=0x7f95fc168620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:45.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.857+0000 7f95f9e00640 1 --2- 172.21.15.131:0/1946942450 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f95fc0725a0 0x7f95fc168620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:45.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.857+0000 7f95f9e00640 1 -- 172.21.15.131:0/1946942450 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95fc0b04c0 con 0x7f95fc06dd30 2024-05-26T22:52:45.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.857+0000 7f95fa800640 1 --2- 172.21.15.131:0/1946942450 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f95fc155770 0x7f95fc15a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:52:45.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.857+0000 7f95f9400640 1 --2- 172.21.15.131:0/1946942450 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f95fc0725a0 0x7f95fc168620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:52:45.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.857+0000 7f95f9e00640 1 --2- 172.21.15.131:0/1946942450 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f95fc06dd30 0x7f95fc141b80 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f95ec00c400 tx=0x7f95ec00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:45.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.857+0000 7f95f2000640 1 -- 172.21.15.131:0/1946942450 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95ec0052d0 con 0x7f95fc06dd30 2024-05-26T22:52:45.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.858+0000 7f95f2000640 1 -- 172.21.15.131:0/1946942450 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95ec005490 con 0x7f95fc06dd30 2024-05-26T22:52:45.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.858+0000 7f95f2000640 1 -- 172.21.15.131:0/1946942450 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95ec007260 con 0x7f95fc06dd30 2024-05-26T22:52:45.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.858+0000 7f9600c00640 1 -- 172.21.15.131:0/1946942450 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95fc0b07a0 con 0x7f95fc06dd30 2024-05-26T22:52:45.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.858+0000 7f9600c00640 1 -- 172.21.15.131:0/1946942450 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95fc0b0d60 con 0x7f95fc06dd30 2024-05-26T22:52:45.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.859+0000 7f9600c00640 1 -- 172.21.15.131:0/1946942450 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95fc074e90 con 0x7f95fc06dd30 2024-05-26T22:52:45.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.860+0000 7f95f2000640 1 -- 172.21.15.131:0/1946942450 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f95ec007400 con 0x7f95fc06dd30 2024-05-26T22:52:45.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.861+0000 7f95f2000640 1 --2- 172.21.15.131:0/1946942450 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f95dc06be30 0x7f95dc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:45.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.861+0000 7f95f2000640 1 -- 172.21.15.131:0/1946942450 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5115+0+0 (secure 0 0 0) 0x7f95ec08a240 con 0x7f95fc06dd30 2024-05-26T22:52:45.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.861+0000 7f95f9400640 1 --2- 172.21.15.131:0/1946942450 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f95dc06be30 0x7f95dc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:45.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.862+0000 7f95f9400640 1 --2- 172.21.15.131:0/1946942450 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f95dc06be30 0x7f95dc06e2f0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f95fc15ad50 tx=0x7f95e8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:45.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:45.866+0000 7f95f2000640 1 -- 172.21.15.131:0/1946942450 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f95ec058da0 con 0x7f95fc06dd30 2024-05-26T22:52:46.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:46.064+0000 7f9600c00640 1 -- 172.21.15.131:0/1946942450 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f95fc0695c0 con 0x7f95fc06dd30 2024-05-26T22:52:46.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:46.065+0000 7f95f2000640 1 -- 172.21.15.131:0/1946942450 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v61) ==== 93+0+10 (secure 0 0 0) 0x7f95ec0586a0 con 0x7f95fc06dd30 2024-05-26T22:52:46.065 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2024-05-26T22:52:46.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:46.068+0000 7f9600c00640 1 -- 172.21.15.131:0/1946942450 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f95dc06be30 msgr2=0x7f95dc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:46.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:46.068+0000 7f9600c00640 1 --2- 172.21.15.131:0/1946942450 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f95dc06be30 0x7f95dc06e2f0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f95fc15ad50 tx=0x7f95e8008040 comp rx=0 tx=0).stop 2024-05-26T22:52:46.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:46.068+0000 7f9600c00640 1 -- 172.21.15.131:0/1946942450 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f95fc06dd30 msgr2=0x7f95fc141b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:46.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:46.068+0000 7f9600c00640 1 --2- 172.21.15.131:0/1946942450 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f95fc06dd30 0x7f95fc141b80 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f95ec00c400 tx=0x7f95ec00c8c0 comp rx=0 tx=0).stop 2024-05-26T22:52:46.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:46.068+0000 7f9600c00640 1 -- 172.21.15.131:0/1946942450 shutdown_connections 2024-05-26T22:52:46.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:46.068+0000 7f9600c00640 1 --2- 172.21.15.131:0/1946942450 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f95fc155770 0x7f95fc15a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:46.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:46.068+0000 7f9600c00640 1 --2- 172.21.15.131:0/1946942450 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f95dc06be30 0x7f95dc06e2f0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:46.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:46.068+0000 7f9600c00640 1 --2- 172.21.15.131:0/1946942450 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f95fc0725a0 0x7f95fc168620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:46.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:46.068+0000 7f9600c00640 1 --2- 172.21.15.131:0/1946942450 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f95fc06dd30 0x7f95fc141b80 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:46.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:46.068+0000 7f9600c00640 1 -- 172.21.15.131:0/1946942450 >> 172.21.15.131:0/1946942450 conn(0x7f95fc06bc90 msgr2=0x7f95fc0b4d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:52:46.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:46.068+0000 7f9600c00640 1 -- 172.21.15.131:0/1946942450 shutdown_connections 2024-05-26T22:52:46.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:46.068+0000 7f9600c00640 1 -- 172.21.15.131:0/1946942450 wait complete. 2024-05-26T22:52:46.087 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_exports.TestEphemeralPins.test_ephemeral_pin_dist_override' 2024-05-26T22:52:46.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:46.271+0000 7f2f4b200640 1 Processor -- start 2024-05-26T22:52:46.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:46.271+0000 7f2f4b200640 1 -- start start 2024-05-26T22:52:46.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:46.273+0000 7f2f4b200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f2f4406df90 0x7f2f440b2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:46.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:46.273+0000 7f2f4b200640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f44071990 con 0x7f2f440cc760 2024-05-26T22:52:46.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:46.273+0000 7f2f4b200640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f44071c70 con 0x7f2f4414c090 2024-05-26T22:52:46.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:46.273+0000 7f2f4b200640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f2f44071f50 con 0x7f2f4406df90 2024-05-26T22:52:46.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:46.273+0000 7f2f4a800640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f2f440cc760 0x7f2f440ce5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:58306/0 (socket says 172.21.15.131:58306) 2024-05-26T22:52:46.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:46.273+0000 7f2f4a800640 1 -- 172.21.15.131:0/1337212190 learned_addr learned my addr 172.21.15.131:0/1337212190 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:52:46.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:46.273+0000 7f2f49400640 1 --2- 172.21.15.131:0/1337212190 >> v2:172.21.15.131:3300/0 conn(0x7f2f4406df90 0x7f2f440b2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:46.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:46.274+0000 7f2f48a00640 1 -- 172.21.15.131:0/1337212190 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1332936190 0 0) 0x7f2f44071990 con 0x7f2f440cc760 2024-05-26T22:52:46.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:46.274+0000 7f2f48a00640 1 -- 172.21.15.131:0/1337212190 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2f30003520 con 0x7f2f440cc760 2024-05-26T22:52:46.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:46.274+0000 7f2f49400640 1 -- 172.21.15.131:0/1337212190 >> v1:172.21.15.187:6789/0 conn(0x7f2f4414c090 legacy=0x7f2f44071160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:46.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:46.274+0000 7f2f49400640 1 -- 172.21.15.131:0/1337212190 >> v1:172.21.15.131:6789/0 conn(0x7f2f440cc760 legacy=0x7f2f440ce5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:46.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:46.274+0000 7f2f49400640 1 -- 172.21.15.131:0/1337212190 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f440cdde0 con 0x7f2f4406df90 2024-05-26T22:52:46.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:46.275+0000 7f2f49400640 1 --2- 172.21.15.131:0/1337212190 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2f4406df90 0x7f2f440b2e10 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f2f340097a0 tx=0x7f2f3402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=554d99fa5e3b1002 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:46.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:46.275+0000 7f2f48a00640 1 -- 172.21.15.131:0/1337212190 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f34037070 con 0x7f2f4406df90 2024-05-26T22:52:46.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:46.275+0000 7f2f48a00640 1 -- 172.21.15.131:0/1337212190 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f34032440 con 0x7f2f4406df90 2024-05-26T22:52:46.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:46.275+0000 7f2f48a00640 1 -- 172.21.15.131:0/1337212190 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f34032810 con 0x7f2f4406df90 2024-05-26T22:52:46.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:46.276+0000 7f2f4b200640 1 -- 172.21.15.131:0/1337212190 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2f4406df90 msgr2=0x7f2f440b2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:46.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:46.276+0000 7f2f4b200640 1 --2- 172.21.15.131:0/1337212190 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2f4406df90 0x7f2f440b2e10 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f2f340097a0 tx=0x7f2f3402a1a0 comp rx=0 tx=0).stop 2024-05-26T22:52:46.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:46.276+0000 7f2f4b200640 1 -- 172.21.15.131:0/1337212190 shutdown_connections 2024-05-26T22:52:46.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:46.276+0000 7f2f4b200640 1 --2- 172.21.15.131:0/1337212190 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2f4406df90 0x7f2f440b2e10 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:46.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:46.276+0000 7f2f4b200640 1 -- 172.21.15.131:0/1337212190 >> 172.21.15.131:0/1337212190 conn(0x7f2f440cff40 msgr2=0x7f2f440d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:52:46.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:46.276+0000 7f2f4b200640 1 -- 172.21.15.131:0/1337212190 shutdown_connections 2024-05-26T22:52:46.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:46.276+0000 7f2f4b200640 1 -- 172.21.15.131:0/1337212190 wait complete. 2024-05-26T22:52:46.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:46.278+0000 7f2f4b200640 1 Processor -- start 2024-05-26T22:52:46.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:46.278+0000 7f2f4b200640 1 -- start start 2024-05-26T22:52:46.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:46.279+0000 7f2f4b200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2f4406df90 0x7f2f44143a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:46.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:46.279+0000 7f2f4b200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f440cc760 0x7f2f441fb430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:46.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:46.279+0000 7f2f4b200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2f4414c090 0x7f2f441fb970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:46.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:46.279+0000 7f2f4b200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f2f440ccbf0 con 0x7f2f4406df90 2024-05-26T22:52:46.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:46.279+0000 7f2f4b200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2f440ce150 con 0x7f2f440cc760 2024-05-26T22:52:46.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:46.279+0000 7f2f4b200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f2f440ccd70 con 0x7f2f4414c090 2024-05-26T22:52:46.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:46.279+0000 7f2f49e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2f4406df90 0x7f2f44143a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:46.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:46.279+0000 7f2f49e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2f4406df90 0x7f2f44143a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39086/0 (socket says 172.21.15.131:39086) 2024-05-26T22:52:46.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:46.279+0000 7f2f49e00640 1 -- 172.21.15.131:0/336865776 learned_addr learned my addr 172.21.15.131:0/336865776 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:52:46.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:46.280+0000 7f2f49400640 1 --2- 172.21.15.131:0/336865776 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f440cc760 0x7f2f441fb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:46.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:46.280+0000 7f2f4a800640 1 --2- 172.21.15.131:0/336865776 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2f4414c090 0x7f2f441fb970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:46.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:46.280+0000 7f2f49e00640 1 -- 172.21.15.131:0/336865776 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2f4414c090 msgr2=0x7f2f441fb970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:46.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:46.280+0000 7f2f49e00640 1 --2- 172.21.15.131:0/336865776 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2f4414c090 0x7f2f441fb970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:46.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:46.280+0000 7f2f49e00640 1 -- 172.21.15.131:0/336865776 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f440cc760 msgr2=0x7f2f441fb430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:46.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:46.280+0000 7f2f49e00640 1 --2- 172.21.15.131:0/336865776 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f440cc760 0x7f2f441fb430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:46.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:46.280+0000 7f2f49e00640 1 -- 172.21.15.131:0/336865776 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f441fa390 con 0x7f2f4406df90 2024-05-26T22:52:46.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:46.280+0000 7f2f49400640 1 --2- 172.21.15.131:0/336865776 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f440cc760 0x7f2f441fb430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-26T22:52:46.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:46.280+0000 7f2f49e00640 1 --2- 172.21.15.131:0/336865776 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2f4406df90 0x7f2f44143a00 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c00c730 tx=0x7f2f3c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:46.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:46.281+0000 7f2f3a000640 1 -- 172.21.15.131:0/336865776 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f3c016020 con 0x7f2f4406df90 2024-05-26T22:52:46.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:46.281+0000 7f2f3a000640 1 -- 172.21.15.131:0/336865776 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f3c006d50 con 0x7f2f4406df90 2024-05-26T22:52:46.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:46.281+0000 7f2f3a000640 1 -- 172.21.15.131:0/336865776 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f3c003bb0 con 0x7f2f4406df90 2024-05-26T22:52:46.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:46.281+0000 7f2f4b200640 1 -- 172.21.15.131:0/336865776 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f441fa670 con 0x7f2f4406df90 2024-05-26T22:52:46.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:46.281+0000 7f2f4b200640 1 -- 172.21.15.131:0/336865776 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f441faf10 con 0x7f2f4406df90 2024-05-26T22:52:46.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:46.282+0000 7f2f4b200640 1 -- 172.21.15.131:0/336865776 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f441fa840 con 0x7f2f4406df90 2024-05-26T22:52:46.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:46.283+0000 7f2f3a000640 1 -- 172.21.15.131:0/336865776 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2f3c00a850 con 0x7f2f4406df90 2024-05-26T22:52:46.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:46.284+0000 7f2f3a000640 1 --2- 172.21.15.131:0/336865776 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f2f2806be30 0x7f2f2806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:46.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:46.284+0000 7f2f3a000640 1 -- 172.21.15.131:0/336865776 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5115+0+0 (secure 0 0 0) 0x7f2f3c08a8f0 con 0x7f2f4406df90 2024-05-26T22:52:46.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:46.285+0000 7f2f49400640 1 --2- 172.21.15.131:0/336865776 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f2f2806be30 0x7f2f2806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:46.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:46.285+0000 7f2f49400640 1 --2- 172.21.15.131:0/336865776 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f2f2806be30 0x7f2f2806e2f0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f2f34034000 tx=0x7f2f3402e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:46.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:46.289+0000 7f2f3a000640 1 -- 172.21.15.131:0/336865776 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f2f3c020020 con 0x7f2f4406df90 2024-05-26T22:52:46.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:46.562+0000 7f2f4b200640 1 -- 172.21.15.131:0/336865776 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_exports.TestEphemeralPins.test_ephemeral_pin_dist_override"]} v 0) -- 0x7f2f4406dd60 con 0x7f2f4406df90 2024-05-26T22:52:47.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:47.209+0000 7f2f3a000640 1 -- 172.21.15.131:0/336865776 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_exports.TestEphemeralPins.test_ephemeral_pin_dist_override"]}]=0 v234) ==== 158+0+0 (secure 0 0 0) 0x7f2f4406dd60 con 0x7f2f4406df90 2024-05-26T22:52:47.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:47.213+0000 7f2f4b200640 1 -- 172.21.15.131:0/336865776 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f2f2806be30 msgr2=0x7f2f2806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:47.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:47.213+0000 7f2f4b200640 1 --2- 172.21.15.131:0/336865776 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f2f2806be30 0x7f2f2806e2f0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f2f34034000 tx=0x7f2f3402e000 comp rx=0 tx=0).stop 2024-05-26T22:52:47.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:47.213+0000 7f2f4b200640 1 -- 172.21.15.131:0/336865776 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2f4406df90 msgr2=0x7f2f44143a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:47.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:47.213+0000 7f2f4b200640 1 --2- 172.21.15.131:0/336865776 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2f4406df90 0x7f2f44143a00 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c00c730 tx=0x7f2f3c00cbf0 comp rx=0 tx=0).stop 2024-05-26T22:52:47.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:47.213+0000 7f2f4b200640 1 -- 172.21.15.131:0/336865776 shutdown_connections 2024-05-26T22:52:47.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:47.213+0000 7f2f4b200640 1 --2- 172.21.15.131:0/336865776 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2f4414c090 0x7f2f441fb970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:47.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:47.213+0000 7f2f4b200640 1 --2- 172.21.15.131:0/336865776 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f2f2806be30 0x7f2f2806e2f0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:47.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:47.213+0000 7f2f4b200640 1 --2- 172.21.15.131:0/336865776 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f440cc760 0x7f2f441fb430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:47.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:47.213+0000 7f2f4b200640 1 --2- 172.21.15.131:0/336865776 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2f4406df90 0x7f2f44143a00 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:47.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:47.213+0000 7f2f4b200640 1 -- 172.21.15.131:0/336865776 >> 172.21.15.131:0/336865776 conn(0x7f2f440cff40 msgr2=0x7f2f44162d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:52:47.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:47.214+0000 7f2f4b200640 1 -- 172.21.15.131:0/336865776 shutdown_connections 2024-05-26T22:52:47.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:47.214+0000 7f2f4b200640 1 -- 172.21.15.131:0/336865776 wait complete. 2024-05-26T22:52:47.234 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-26T22:52:47.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:47.440+0000 7ffbe2c00640 1 Processor -- start 2024-05-26T22:52:47.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:47.440+0000 7ffbe2c00640 1 -- start start 2024-05-26T22:52:47.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:47.442+0000 7ffbe2c00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7ffbdc0b6220 0x7ffbdc0b6600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:47.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:47.442+0000 7ffbe2c00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffbdc0cf710 0x7ffbdc0b98b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:47.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:47.442+0000 7ffbe2c00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffbdc0b6b40 con 0x7ffbdc0cf2b0 2024-05-26T22:52:47.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:47.442+0000 7ffbe2c00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7ffbdc0b7eb0 con 0x7ffbdc0b6220 2024-05-26T22:52:47.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:47.442+0000 7ffbe2c00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ffbdc0b8d10 con 0x7ffbdc0cf710 2024-05-26T22:52:47.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:47.442+0000 7ffbe1800640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7ffbdc0b6220 0x7ffbdc0b6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:47.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:47.442+0000 7ffbe2200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffbdc0cf710 0x7ffbdc0b98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:47.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:47.442+0000 7ffbe1800640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7ffbdc0b6220 0x7ffbdc0b6600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.131:53206/0 (socket says 172.21.15.131:53206) 2024-05-26T22:52:47.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:47.442+0000 7ffbe1800640 1 -- 172.21.15.131:0/3024365541 learned_addr learned my addr 172.21.15.131:0/3024365541 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:52:47.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:47.443+0000 7ffbe2200640 1 -- 172.21.15.131:0/3024365541 >> v2:172.21.15.187:3300/0 conn(0x7ffbdc0b6220 msgr2=0x7ffbdc0b6600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:47.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:47.443+0000 7ffbe2200640 1 --2- 172.21.15.131:0/3024365541 >> v2:172.21.15.187:3300/0 conn(0x7ffbdc0b6220 0x7ffbdc0b6600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:47.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:47.443+0000 7ffbe2200640 1 -- 172.21.15.131:0/3024365541 >> v1:172.21.15.187:6789/0 conn(0x7ffbdc0cf2b0 legacy=0x7ffbdc0b91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:47.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:47.443+0000 7ffbe2200640 1 -- 172.21.15.131:0/3024365541 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbdc0b8100 con 0x7ffbdc0cf710 2024-05-26T22:52:47.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:47.443+0000 7ffbe1800640 1 --2- 172.21.15.131:0/3024365541 >> v2:172.21.15.187:3300/0 conn(0x7ffbdc0b6220 0x7ffbdc0b6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-26T22:52:47.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:47.445+0000 7ffbe2200640 1 --2- 172.21.15.131:0/3024365541 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffbdc0cf710 0x7ffbdc0b98b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ffbd40097a0 tx=0x7ffbd402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=afe92ffeacf9dcb2 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:47.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:47.445+0000 7ffbdbe00640 1 -- 172.21.15.131:0/3024365541 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbd4037070 con 0x7ffbdc0cf710 2024-05-26T22:52:47.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:47.446+0000 7ffbdbe00640 1 -- 172.21.15.131:0/3024365541 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffbd4032440 con 0x7ffbdc0cf710 2024-05-26T22:52:47.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:47.446+0000 7ffbdbe00640 1 -- 172.21.15.131:0/3024365541 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbd4032810 con 0x7ffbdc0cf710 2024-05-26T22:52:47.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:47.446+0000 7ffbe2c00640 1 -- 172.21.15.131:0/3024365541 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffbdc0cf710 msgr2=0x7ffbdc0b98b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:47.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:47.446+0000 7ffbe2c00640 1 --2- 172.21.15.131:0/3024365541 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffbdc0cf710 0x7ffbdc0b98b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ffbd40097a0 tx=0x7ffbd402a180 comp rx=0 tx=0).stop 2024-05-26T22:52:47.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:47.446+0000 7ffbe2c00640 1 -- 172.21.15.131:0/3024365541 shutdown_connections 2024-05-26T22:52:47.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:47.446+0000 7ffbe2c00640 1 --2- 172.21.15.131:0/3024365541 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffbdc0cf710 0x7ffbdc0b98b0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:47.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:47.446+0000 7ffbe2c00640 1 --2- 172.21.15.131:0/3024365541 >> v2:172.21.15.187:3300/0 conn(0x7ffbdc0b6220 0x7ffbdc0b6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:47.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:47.446+0000 7ffbe2c00640 1 -- 172.21.15.131:0/3024365541 >> 172.21.15.131:0/3024365541 conn(0x7ffbdc072e60 msgr2=0x7ffbdc073270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:52:47.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:47.446+0000 7ffbe2c00640 1 -- 172.21.15.131:0/3024365541 shutdown_connections 2024-05-26T22:52:47.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:47.446+0000 7ffbe2c00640 1 -- 172.21.15.131:0/3024365541 wait complete. 2024-05-26T22:52:47.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:47.448+0000 7ffbe2c00640 1 Processor -- start 2024-05-26T22:52:47.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:47.448+0000 7ffbe2c00640 1 -- start start 2024-05-26T22:52:47.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:47.449+0000 7ffbe2c00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffbdc0b6220 0x7ffbdc0cbb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:47.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:47.449+0000 7ffbe2c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffbdc0cf2b0 0x7ffbdc159020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:47.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:47.449+0000 7ffbe2c00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffbdc0cf710 0x7ffbdc159560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:47.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:47.449+0000 7ffbe2c00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ffbdc0b89a0 con 0x7ffbdc0cf2b0 2024-05-26T22:52:47.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:47.449+0000 7ffbe2c00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ffbdc0b86c0 con 0x7ffbdc0cf710 2024-05-26T22:52:47.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:47.449+0000 7ffbe2c00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ffbdc15e680 con 0x7ffbdc0b6220 2024-05-26T22:52:47.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:47.450+0000 7ffbe0e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffbdc0cf2b0 0x7ffbdc159020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:47.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:47.450+0000 7ffbe0e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffbdc0cf2b0 0x7ffbdc159020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39102/0 (socket says 172.21.15.131:39102) 2024-05-26T22:52:47.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:47.450+0000 7ffbe0e00640 1 -- 172.21.15.131:0/1045411689 learned_addr learned my addr 172.21.15.131:0/1045411689 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:52:47.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:47.450+0000 7ffbe1800640 1 --2- 172.21.15.131:0/1045411689 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffbdc0b6220 0x7ffbdc0cbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:47.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:47.450+0000 7ffbe2200640 1 --2- 172.21.15.131:0/1045411689 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffbdc0cf710 0x7ffbdc159560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:47.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:47.450+0000 7ffbe0e00640 1 -- 172.21.15.131:0/1045411689 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffbdc0b6220 msgr2=0x7ffbdc0cbb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:47.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:47.450+0000 7ffbe0e00640 1 --2- 172.21.15.131:0/1045411689 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffbdc0b6220 0x7ffbdc0cbb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:47.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:47.450+0000 7ffbe0e00640 1 -- 172.21.15.131:0/1045411689 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffbdc0cf710 msgr2=0x7ffbdc159560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:47.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:47.450+0000 7ffbe0e00640 1 --2- 172.21.15.131:0/1045411689 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffbdc0cf710 0x7ffbdc159560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:47.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:47.450+0000 7ffbe0e00640 1 -- 172.21.15.131:0/1045411689 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbdc0cc5e0 con 0x7ffbdc0cf2b0 2024-05-26T22:52:47.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:47.450+0000 7ffbe0e00640 1 --2- 172.21.15.131:0/1045411689 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffbdc0cf2b0 0x7ffbdc159020 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7ffbc800c400 tx=0x7ffbc800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:47.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:47.451+0000 7ffbd9600640 1 -- 172.21.15.131:0/1045411689 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc80052d0 con 0x7ffbdc0cf2b0 2024-05-26T22:52:47.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:47.451+0000 7ffbd9600640 1 -- 172.21.15.131:0/1045411689 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffbc8005490 con 0x7ffbdc0cf2b0 2024-05-26T22:52:47.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:47.451+0000 7ffbd9600640 1 -- 172.21.15.131:0/1045411689 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc8007260 con 0x7ffbdc0cf2b0 2024-05-26T22:52:47.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:47.451+0000 7ffbe2c00640 1 -- 172.21.15.131:0/1045411689 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbdc0cc8c0 con 0x7ffbdc0cf2b0 2024-05-26T22:52:47.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:47.451+0000 7ffbe2c00640 1 -- 172.21.15.131:0/1045411689 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbdc0cce80 con 0x7ffbdc0cf2b0 2024-05-26T22:52:47.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:47.452+0000 7ffbe2c00640 1 -- 172.21.15.131:0/1045411689 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbdc0694e0 con 0x7ffbdc0cf2b0 2024-05-26T22:52:47.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:47.453+0000 7ffbd9600640 1 -- 172.21.15.131:0/1045411689 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ffbc8007400 con 0x7ffbdc0cf2b0 2024-05-26T22:52:47.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:47.454+0000 7ffbd9600640 1 --2- 172.21.15.131:0/1045411689 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ffbbc06be30 0x7ffbbc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:47.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:47.454+0000 7ffbd9600640 1 -- 172.21.15.131:0/1045411689 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5115+0+0 (secure 0 0 0) 0x7ffbc808a2c0 con 0x7ffbdc0cf2b0 2024-05-26T22:52:47.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:47.454+0000 7ffbe1800640 1 --2- 172.21.15.131:0/1045411689 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ffbbc06be30 0x7ffbbc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:47.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:47.455+0000 7ffbe1800640 1 --2- 172.21.15.131:0/1045411689 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ffbbc06be30 0x7ffbbc06e2f0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc009240 tx=0x7ffbcc00b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:47.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:47.459+0000 7ffbd9600640 1 -- 172.21.15.131:0/1045411689 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7ffbc8058da0 con 0x7ffbdc0cf2b0 2024-05-26T22:52:47.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:47.661+0000 7ffbe2c00640 1 -- 172.21.15.131:0/1045411689 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7ffbdc0d2a80 con 0x7ffbdc0cf2b0 2024-05-26T22:52:47.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:47.666+0000 7ffbd9600640 1 -- 172.21.15.131:0/1045411689 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v35)=0 v35) ==== 122+0+0 (secure 0 0 0) 0x7ffbc80586a0 con 0x7ffbdc0cf2b0 2024-05-26T22:52:47.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:47.669+0000 7ffbe2c00640 1 -- 172.21.15.131:0/1045411689 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ffbbc06be30 msgr2=0x7ffbbc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:47.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:47.669+0000 7ffbe2c00640 1 --2- 172.21.15.131:0/1045411689 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ffbbc06be30 0x7ffbbc06e2f0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc009240 tx=0x7ffbcc00b040 comp rx=0 tx=0).stop 2024-05-26T22:52:47.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:47.669+0000 7ffbe2c00640 1 -- 172.21.15.131:0/1045411689 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffbdc0cf2b0 msgr2=0x7ffbdc159020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:47.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:47.669+0000 7ffbe2c00640 1 --2- 172.21.15.131:0/1045411689 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffbdc0cf2b0 0x7ffbdc159020 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7ffbc800c400 tx=0x7ffbc800c8c0 comp rx=0 tx=0).stop 2024-05-26T22:52:47.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:47.669+0000 7ffbe2c00640 1 -- 172.21.15.131:0/1045411689 shutdown_connections 2024-05-26T22:52:47.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:47.669+0000 7ffbe2c00640 1 --2- 172.21.15.131:0/1045411689 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ffbbc06be30 0x7ffbbc06e2f0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:47.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:47.669+0000 7ffbe2c00640 1 --2- 172.21.15.131:0/1045411689 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffbdc0cf710 0x7ffbdc159560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:47.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:47.669+0000 7ffbe2c00640 1 --2- 172.21.15.131:0/1045411689 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffbdc0cf2b0 0x7ffbdc159020 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:47.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:47.669+0000 7ffbe2c00640 1 --2- 172.21.15.131:0/1045411689 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffbdc0b6220 0x7ffbdc0cbb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:47.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:47.669+0000 7ffbe2c00640 1 -- 172.21.15.131:0/1045411689 >> 172.21.15.131:0/1045411689 conn(0x7ffbdc072e60 msgr2=0x7ffbdc0d39c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:52:47.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:47.670+0000 7ffbe2c00640 1 -- 172.21.15.131:0/1045411689 shutdown_connections 2024-05-26T22:52:47.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:47.670+0000 7ffbe2c00640 1 -- 172.21.15.131:0/1045411689 wait complete. 2024-05-26T22:52:47.688 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-05-26T22:52:47.688 DEBUG:teuthology.orchestra.run.smithi131:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-26T22:52:47.794 DEBUG:teuthology.orchestra.run.smithi187:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-26T22:52:47.916 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-05-26T22:52:47.917 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-26T22:52:47.917 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-26T22:52:47.972 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-26T22:52:47.973 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-05-26T22:52:47.973 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-26T22:52:48.030 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-26T22:52:48.030 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-26T22:52:48.030 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-26T22:52:48.088 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-26T22:52:48.088 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-05-26T22:52:48.088 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-26T22:52:48.144 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-26T22:52:48.144 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-26T22:52:48.145 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-26T22:52:48.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.356+0000 7efe02e00640 1 Processor -- start 2024-05-26T22:52:48.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.356+0000 7efe02e00640 1 -- start start 2024-05-26T22:52:48.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.357+0000 7efe02e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efdfc0ad250 0x7efdfc0b3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:48.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.357+0000 7efe02e00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efdfc15cb20 con 0x7efdfc1569d0 2024-05-26T22:52:48.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.358+0000 7efe02e00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efdfc15ce00 con 0x7efdfc06ee20 2024-05-26T22:52:48.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.358+0000 7efe02e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7efdfc15d0e0 con 0x7efdfc0ad250 2024-05-26T22:52:48.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.358+0000 7efe00e00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7efdfc1569d0 0x7efdfc160010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:58308/0 (socket says 172.21.15.131:58308) 2024-05-26T22:52:48.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.358+0000 7efe00e00640 1 -- 172.21.15.131:0/2315619902 learned_addr learned my addr 172.21.15.131:0/2315619902 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:52:48.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.358+0000 7efdfb400640 1 --2- 172.21.15.131:0/2315619902 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efdfc0ad250 0x7efdfc0b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:48.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.359+0000 7efdfaa00640 1 -- 172.21.15.131:0/2315619902 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1315019092 0 0) 0x7efdfc15cb20 con 0x7efdfc1569d0 2024-05-26T22:52:48.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.359+0000 7efdfaa00640 1 -- 172.21.15.131:0/2315619902 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efde0003520 con 0x7efdfc1569d0 2024-05-26T22:52:48.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.359+0000 7efdfaa00640 1 -- 172.21.15.131:0/2315619902 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 825075248 0 0) 0x7efde0003520 con 0x7efdfc1569d0 2024-05-26T22:52:48.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.359+0000 7efdfaa00640 1 -- 172.21.15.131:0/2315619902 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efdfc0ad250 msgr2=0x7efdfc0b3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:48.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.359+0000 7efdfaa00640 1 --2- 172.21.15.131:0/2315619902 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efdfc0ad250 0x7efdfc0b3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:48.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.359+0000 7efdfaa00640 1 -- 172.21.15.131:0/2315619902 >> v1:172.21.15.187:6789/0 conn(0x7efdfc06ee20 legacy=0x7efdfc1562c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:48.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.359+0000 7efdfaa00640 1 -- 172.21.15.131:0/2315619902 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdfc15d3c0 con 0x7efdfc1569d0 2024-05-26T22:52:48.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.359+0000 7efdfaa00640 1 -- 172.21.15.131:0/2315619902 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7efdf4001670 con 0x7efdfc1569d0 2024-05-26T22:52:48.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.359+0000 7efdfaa00640 1 -- 172.21.15.131:0/2315619902 >> v1:172.21.15.131:6789/0 conn(0x7efdfc1569d0 legacy=0x7efdfc160010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:48.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.359+0000 7efdfaa00640 1 --2- 172.21.15.131:0/2315619902 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efde00039e0 0x7efde0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:48.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.359+0000 7efdfaa00640 1 --2- 172.21.15.131:0/2315619902 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efdfc0ad250 0x7efde0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:48.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.359+0000 7efdfaa00640 1 --2- 172.21.15.131:0/2315619902 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efde0004d40 0x7efde00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:48.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.359+0000 7efdfaa00640 1 -- 172.21.15.131:0/2315619902 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7efdfc15d0e0 con 0x7efde0004d40 2024-05-26T22:52:48.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.359+0000 7efdfaa00640 1 -- 172.21.15.131:0/2315619902 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7efde000bb50 con 0x7efde00039e0 2024-05-26T22:52:48.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.359+0000 7efdfaa00640 1 -- 172.21.15.131:0/2315619902 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7efde000bd50 con 0x7efdfc0ad250 2024-05-26T22:52:48.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.360+0000 7efe00e00640 1 --2- 172.21.15.131:0/2315619902 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efde0004d40 0x7efde00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:48.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.360+0000 7efdfbe00640 1 --2- 172.21.15.131:0/2315619902 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efde00039e0 0x7efde0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:48.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.360+0000 7efdfb400640 1 --2- 172.21.15.131:0/2315619902 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efdfc0ad250 0x7efde0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:48.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.360+0000 7efe00e00640 1 -- 172.21.15.131:0/2315619902 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efdfc0ad250 msgr2=0x7efde0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:48.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.360+0000 7efe00e00640 1 --2- 172.21.15.131:0/2315619902 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efdfc0ad250 0x7efde0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:48.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.360+0000 7efe00e00640 1 -- 172.21.15.131:0/2315619902 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efde00039e0 msgr2=0x7efde0003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:48.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.360+0000 7efe00e00640 1 --2- 172.21.15.131:0/2315619902 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efde00039e0 0x7efde0003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:48.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.360+0000 7efe00e00640 1 -- 172.21.15.131:0/2315619902 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efde0003520 con 0x7efde0004d40 2024-05-26T22:52:48.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.361+0000 7efe00e00640 1 --2- 172.21.15.131:0/2315619902 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efde0004d40 0x7efde00051c0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7efdf400c240 tx=0x7efde0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d15fd93211dcc5b5 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:48.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.361+0000 7efdfaa00640 1 -- 172.21.15.131:0/2315619902 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf4007d10 con 0x7efde0004d40 2024-05-26T22:52:48.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.362+0000 7efdfaa00640 1 -- 172.21.15.131:0/2315619902 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdf4005ad0 con 0x7efde0004d40 2024-05-26T22:52:48.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.362+0000 7efdfaa00640 1 -- 172.21.15.131:0/2315619902 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf4007070 con 0x7efde0004d40 2024-05-26T22:52:48.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.362+0000 7efe02e00640 1 -- 172.21.15.131:0/2315619902 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efde0004d40 msgr2=0x7efde00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:48.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.362+0000 7efe02e00640 1 --2- 172.21.15.131:0/2315619902 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efde0004d40 0x7efde00051c0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7efdf400c240 tx=0x7efde0005830 comp rx=0 tx=0).stop 2024-05-26T22:52:48.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.362+0000 7efe00e00640 1 -- 172.21.15.131:0/2315619902 reap_dead start 2024-05-26T22:52:48.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.362+0000 7efe02e00640 1 -- 172.21.15.131:0/2315619902 shutdown_connections 2024-05-26T22:52:48.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.362+0000 7efe02e00640 1 -- 172.21.15.131:0/2315619902 >> 172.21.15.131:0/2315619902 conn(0x7efdfc0b1e90 msgr2=0x7efdfc0b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:52:48.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.362+0000 7efe02e00640 1 -- 172.21.15.131:0/2315619902 shutdown_connections 2024-05-26T22:52:48.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.362+0000 7efe02e00640 1 -- 172.21.15.131:0/2315619902 wait complete. 2024-05-26T22:52:48.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.364+0000 7efe02e00640 1 Processor -- start 2024-05-26T22:52:48.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.364+0000 7efe02e00640 1 -- start start 2024-05-26T22:52:48.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.365+0000 7efe02e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efdfc09cf60 0x7efdfc151d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:48.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.365+0000 7efe02e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efdfc1f2320 0x7efdfc152250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:48.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.365+0000 7efe02e00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efdfc09d340 0x7efdfc09ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:48.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.365+0000 7efe02e00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7efdfc15c900 con 0x7efdfc09cf60 2024-05-26T22:52:48.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.365+0000 7efe02e00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7efdfc15c780 con 0x7efdfc09d340 2024-05-26T22:52:48.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.365+0000 7efe02e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7efdfc06f200 con 0x7efdfc1f2320 2024-05-26T22:52:48.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.365+0000 7efdfbe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efdfc09cf60 0x7efdfc151d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:48.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.366+0000 7efdfbe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efdfc09cf60 0x7efdfc151d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39122/0 (socket says 172.21.15.131:39122) 2024-05-26T22:52:48.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.366+0000 7efdfbe00640 1 -- 172.21.15.131:0/1794227134 learned_addr learned my addr 172.21.15.131:0/1794227134 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:52:48.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.366+0000 7efdfbe00640 1 -- 172.21.15.131:0/1794227134 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efdfc1f2320 msgr2=0x7efdfc152250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:48.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.366+0000 7efdfbe00640 1 --2- 172.21.15.131:0/1794227134 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efdfc1f2320 0x7efdfc152250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:48.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.366+0000 7efdfbe00640 1 -- 172.21.15.131:0/1794227134 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efdfc09d340 msgr2=0x7efdfc09ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:48.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.366+0000 7efdfbe00640 1 --2- 172.21.15.131:0/1794227134 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efdfc09d340 0x7efdfc09ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:48.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.366+0000 7efdfbe00640 1 -- 172.21.15.131:0/1794227134 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdfc09bb80 con 0x7efdfc09cf60 2024-05-26T22:52:48.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.366+0000 7efdfbe00640 1 --2- 172.21.15.131:0/1794227134 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efdfc09cf60 0x7efdfc151d10 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7efdec00e8f0 tx=0x7efdec00edb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:48.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.366+0000 7efdf3e00640 1 -- 172.21.15.131:0/1794227134 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdec003b80 con 0x7efdfc09cf60 2024-05-26T22:52:48.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.367+0000 7efdf3e00640 1 -- 172.21.15.131:0/1794227134 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdec018070 con 0x7efdfc09cf60 2024-05-26T22:52:48.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.367+0000 7efdf3e00640 1 -- 172.21.15.131:0/1794227134 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdec0050d0 con 0x7efdfc09cf60 2024-05-26T22:52:48.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.367+0000 7efe02e00640 1 -- 172.21.15.131:0/1794227134 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdfc1e4d30 con 0x7efdfc09cf60 2024-05-26T22:52:48.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.367+0000 7efe02e00640 1 -- 172.21.15.131:0/1794227134 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdfc1e52f0 con 0x7efdfc09cf60 2024-05-26T22:52:48.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.368+0000 7efe02e00640 1 -- 172.21.15.131:0/1794227134 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdfc0694e0 con 0x7efdfc09cf60 2024-05-26T22:52:48.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.369+0000 7efdf3e00640 1 -- 172.21.15.131:0/1794227134 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7efdec005270 con 0x7efdfc09cf60 2024-05-26T22:52:48.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.370+0000 7efdf3e00640 1 --2- 172.21.15.131:0/1794227134 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7efdd806be60 0x7efdd806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:48.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.370+0000 7efdf3e00640 1 -- 172.21.15.131:0/1794227134 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5115+0+0 (secure 0 0 0) 0x7efdec08ace0 con 0x7efdfc09cf60 2024-05-26T22:52:48.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.370+0000 7efdfb400640 1 --2- 172.21.15.131:0/1794227134 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7efdd806be60 0x7efdd806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:48.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.371+0000 7efdfb400640 1 --2- 172.21.15.131:0/1794227134 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7efdd806be60 0x7efdd806e320 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7efdfc09bde0 tx=0x7efde400b2e0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:48.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.374+0000 7efdf3e00640 1 -- 172.21.15.131:0/1794227134 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7efdec01a310 con 0x7efdfc09cf60 2024-05-26T22:52:48.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.612+0000 7efe02e00640 1 -- 172.21.15.131:0/1794227134 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efdfc09e210 con 0x7efdfc09cf60 2024-05-26T22:52:48.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.613+0000 7efdf3e00640 1 -- 172.21.15.131:0/1794227134 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) ==== 94+0+3878 (secure 0 0 0) 0x7efdec059840 con 0x7efdfc09cf60 2024-05-26T22:52:48.613 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:52:48.613 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":57,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4832,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/1327386245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":1327386245},{"type":"v1","addr":"172.21.15.187:6835","nonce":1327386245}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":44},{"gid":4913,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/755245158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":755245158},{"type":"v1","addr":"172.21.15.187:6833","nonce":755245158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":6756,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/2564693219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":2564693219},{"type":"v1","addr":"172.21.15.131:6835","nonce":2564693219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":6759,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/2105702067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":2105702067},{"type":"v1","addr":"172.21.15.131:6837","nonce":2105702067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57}],"filesystems":[]} 2024-05-26T22:52:48.613 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 57 2024-05-26T22:52:48.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.616+0000 7efe02e00640 1 -- 172.21.15.131:0/1794227134 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7efdd806be60 msgr2=0x7efdd806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:48.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.616+0000 7efe02e00640 1 --2- 172.21.15.131:0/1794227134 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7efdd806be60 0x7efdd806e320 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7efdfc09bde0 tx=0x7efde400b2e0 comp rx=0 tx=0).stop 2024-05-26T22:52:48.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.616+0000 7efe02e00640 1 -- 172.21.15.131:0/1794227134 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efdfc09cf60 msgr2=0x7efdfc151d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:48.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.616+0000 7efe02e00640 1 --2- 172.21.15.131:0/1794227134 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efdfc09cf60 0x7efdfc151d10 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7efdec00e8f0 tx=0x7efdec00edb0 comp rx=0 tx=0).stop 2024-05-26T22:52:48.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.616+0000 7efe02e00640 1 -- 172.21.15.131:0/1794227134 shutdown_connections 2024-05-26T22:52:48.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.616+0000 7efe02e00640 1 --2- 172.21.15.131:0/1794227134 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efdfc1f2320 0x7efdfc152250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:48.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.616+0000 7efe02e00640 1 --2- 172.21.15.131:0/1794227134 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7efdd806be60 0x7efdd806e320 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:48.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.616+0000 7efe02e00640 1 --2- 172.21.15.131:0/1794227134 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efdfc09d340 0x7efdfc09ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:48.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.616+0000 7efe02e00640 1 --2- 172.21.15.131:0/1794227134 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efdfc09cf60 0x7efdfc151d10 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:48.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.616+0000 7efe02e00640 1 -- 172.21.15.131:0/1794227134 >> 172.21.15.131:0/1794227134 conn(0x7efdfc0b1e90 msgr2=0x7efdfc15ea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:52:48.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.617+0000 7efe02e00640 1 -- 172.21.15.131:0/1794227134 shutdown_connections 2024-05-26T22:52:48.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.617+0000 7efe02e00640 1 -- 172.21.15.131:0/1794227134 wait complete. 2024-05-26T22:52:48.635 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-26T22:52:48.635 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-26T22:52:48.636 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-26T22:52:48.636 INFO:teuthology.orchestra.run:waiting for 300 2024-05-26T22:52:48.636 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-26T22:52:48.636 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-26T22:52:48.636 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-26T22:52:48.636 INFO:teuthology.orchestra.run:waiting for 300 2024-05-26T22:52:48.636 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-26T22:52:48.637 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-26T22:52:48.637 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-26T22:52:48.637 INFO:teuthology.orchestra.run:waiting for 300 2024-05-26T22:52:48.637 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-26T22:52:48.637 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-26T22:52:48.637 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-26T22:52:48.637 INFO:teuthology.orchestra.run:waiting for 300 2024-05-26T22:52:48.655 INFO:tasks.ceph.mds.c:Stopped 2024-05-26T22:52:48.655 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-26T22:52:48.657 INFO:tasks.ceph.mds.a:Stopped 2024-05-26T22:52:48.657 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-26T22:52:48.661 INFO:tasks.ceph.mds.d:Stopped 2024-05-26T22:52:48.661 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-26T22:52:48.665 INFO:tasks.ceph.mds.b:Stopped 2024-05-26T22:52:48.665 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-26T22:52:48.704 INFO:tasks.ceph.mds.c:Started 2024-05-26T22:52:48.705 DEBUG:teuthology.parallel:result is None 2024-05-26T22:52:48.710 INFO:tasks.ceph.mds.a:Started 2024-05-26T22:52:48.711 DEBUG:teuthology.parallel:result is None 2024-05-26T22:52:48.712 INFO:tasks.ceph.mds.d:Started 2024-05-26T22:52:48.713 DEBUG:teuthology.parallel:result is None 2024-05-26T22:52:48.719 INFO:tasks.ceph.mds.b:Started 2024-05-26T22:52:48.719 DEBUG:teuthology.parallel:result is None 2024-05-26T22:52:48.719 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-26T22:52:48.850 INFO:tasks.ceph.mds.a.smithi131.stdout:starting mds.a at 2024-05-26T22:52:48.851 INFO:tasks.ceph.mds.c.smithi131.stdout:starting mds.c at 2024-05-26T22:52:48.875 INFO:tasks.ceph.mds.d.smithi187.stdout:starting mds.d at 2024-05-26T22:52:48.875 INFO:tasks.ceph.mds.b.smithi187.stdout:starting mds.b at 2024-05-26T22:52:48.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.893+0000 7f9c04200640 1 Processor -- start 2024-05-26T22:52:48.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.893+0000 7f9c04200640 1 -- start start 2024-05-26T22:52:48.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.894+0000 7f9c04200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9bfc0d1070 0x7f9bfc0d14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:48.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.894+0000 7f9c04200640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f9bfc15d520 0x7f9bfc15d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:48.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.894+0000 7f9c04200640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9bfc06bd90 con 0x7f9bfc145160 2024-05-26T22:52:48.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.894+0000 7f9c04200640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f9bfc0d1a10 con 0x7f9bfc15d520 2024-05-26T22:52:48.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.894+0000 7f9c04200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9bfc15cac0 con 0x7f9bfc0d1070 2024-05-26T22:52:48.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.894+0000 7f9c01800640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f9bfc145160 0x7f9bfc06b310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:58334/0 (socket says 172.21.15.131:58334) 2024-05-26T22:52:48.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.894+0000 7f9c01800640 1 -- 172.21.15.131:0/1121246060 learned_addr learned my addr 172.21.15.131:0/1121246060 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:52:48.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.894+0000 7f9c02200640 1 --2- 172.21.15.131:0/1121246060 >> v2:172.21.15.187:3300/0 conn(0x7f9bfc15d520 0x7f9bfc15d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:48.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.894+0000 7f9c00e00640 1 --2- 172.21.15.131:0/1121246060 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9bfc0d1070 0x7f9bfc0d14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:48.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.895+0000 7f9bfbe00640 1 -- 172.21.15.131:0/1121246060 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2212239953 0 0) 0x7f9bfc06bd90 con 0x7f9bfc145160 2024-05-26T22:52:48.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.895+0000 7f9bfbe00640 1 -- 172.21.15.131:0/1121246060 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9be4003520 con 0x7f9bfc145160 2024-05-26T22:52:48.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.895+0000 7f9bfbe00640 1 -- 172.21.15.131:0/1121246060 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2100053294 0 0) 0x7f9be4003520 con 0x7f9bfc145160 2024-05-26T22:52:48.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.895+0000 7f9c02200640 1 -- 172.21.15.131:0/1121246060 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9bfc0d1070 msgr2=0x7f9bfc0d14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:48.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.895+0000 7f9c02200640 1 --2- 172.21.15.131:0/1121246060 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9bfc0d1070 0x7f9bfc0d14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:48.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.895+0000 7f9c02200640 1 -- 172.21.15.131:0/1121246060 >> v1:172.21.15.131:6789/0 conn(0x7f9bfc145160 legacy=0x7f9bfc06b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:48.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.895+0000 7f9c02200640 1 -- 172.21.15.131:0/1121246060 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bfc06c070 con 0x7f9bfc15d520 2024-05-26T22:52:48.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.895+0000 7f9c00e00640 1 --2- 172.21.15.131:0/1121246060 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9bfc0d1070 0x7f9bfc0d14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-26T22:52:48.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.896+0000 7f9c02200640 1 --2- 172.21.15.131:0/1121246060 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9bfc15d520 0x7f9bfc15d990 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f9bf4009d90 tx=0x7f9bf402a5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95f54abfde2a40dd server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:48.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.896+0000 7f9bfbe00640 1 -- 172.21.15.131:0/1121246060 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf0001670 con 0x7f9bfc15d520 2024-05-26T22:52:48.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.896+0000 7f9bfbe00640 1 -- 172.21.15.131:0/1121246060 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9bf4007540 con 0x7f9bfc15d520 2024-05-26T22:52:48.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.896+0000 7f9bfbe00640 1 -- 172.21.15.131:0/1121246060 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf4007910 con 0x7f9bfc15d520 2024-05-26T22:52:48.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.897+0000 7f9c04200640 1 -- 172.21.15.131:0/1121246060 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9bfc15d520 msgr2=0x7f9bfc15d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:48.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.897+0000 7f9c04200640 1 --2- 172.21.15.131:0/1121246060 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9bfc15d520 0x7f9bfc15d990 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f9bf4009d90 tx=0x7f9bf402a5f0 comp rx=0 tx=0).stop 2024-05-26T22:52:48.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.897+0000 7f9c04200640 1 -- 172.21.15.131:0/1121246060 shutdown_connections 2024-05-26T22:52:48.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.897+0000 7f9c04200640 1 --2- 172.21.15.131:0/1121246060 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9bfc15d520 0x7f9bfc15d990 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:48.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.897+0000 7f9c04200640 1 --2- 172.21.15.131:0/1121246060 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9bfc0d1070 0x7f9bfc0d14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:48.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.897+0000 7f9c04200640 1 -- 172.21.15.131:0/1121246060 >> 172.21.15.131:0/1121246060 conn(0x7f9bfc0b65b0 msgr2=0x7f9bfc0b69c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:52:48.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.897+0000 7f9c04200640 1 -- 172.21.15.131:0/1121246060 shutdown_connections 2024-05-26T22:52:48.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.897+0000 7f9c04200640 1 -- 172.21.15.131:0/1121246060 wait complete. 2024-05-26T22:52:48.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.899+0000 7f9c04200640 1 Processor -- start 2024-05-26T22:52:48.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.899+0000 7f9c04200640 1 -- start start 2024-05-26T22:52:48.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.900+0000 7f9c04200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9bfc0d1070 0x7f9bfc15b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:48.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.900+0000 7f9c04200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9bfc145160 0x7f9bfc1f7450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:48.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.900+0000 7f9c04200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9bfc15d520 0x7f9bfc1f7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:48.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.900+0000 7f9c04200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f9bfc06ba20 con 0x7f9bfc145160 2024-05-26T22:52:48.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.900+0000 7f9c04200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9bfc145540 con 0x7f9bfc0d1070 2024-05-26T22:52:48.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.900+0000 7f9c04200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9bfc0d1d70 con 0x7f9bfc15d520 2024-05-26T22:52:48.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.900+0000 7f9c00e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9bfc145160 0x7f9bfc1f7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:48.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.900+0000 7f9c00e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9bfc145160 0x7f9bfc1f7450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39142/0 (socket says 172.21.15.131:39142) 2024-05-26T22:52:48.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.900+0000 7f9c00e00640 1 -- 172.21.15.131:0/2489783082 learned_addr learned my addr 172.21.15.131:0/2489783082 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:52:48.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.900+0000 7f9c02200640 1 --2- 172.21.15.131:0/2489783082 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9bfc15d520 0x7f9bfc1f7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:48.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.900+0000 7f9c01800640 1 --2- 172.21.15.131:0/2489783082 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9bfc0d1070 0x7f9bfc15b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:48.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.900+0000 7f9c00e00640 1 -- 172.21.15.131:0/2489783082 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9bfc15d520 msgr2=0x7f9bfc1f7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:48.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.900+0000 7f9c00e00640 1 --2- 172.21.15.131:0/2489783082 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9bfc15d520 0x7f9bfc1f7990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:48.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.900+0000 7f9c00e00640 1 -- 172.21.15.131:0/2489783082 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9bfc0d1070 msgr2=0x7f9bfc15b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:48.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.900+0000 7f9c00e00640 1 --2- 172.21.15.131:0/2489783082 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9bfc0d1070 0x7f9bfc15b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:48.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.900+0000 7f9c00e00640 1 -- 172.21.15.131:0/2489783082 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bfc1f82d0 con 0x7f9bfc145160 2024-05-26T22:52:48.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.900+0000 7f9c02200640 1 --2- 172.21.15.131:0/2489783082 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9bfc15d520 0x7f9bfc1f7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:52:48.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.900+0000 7f9c01800640 1 --2- 172.21.15.131:0/2489783082 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9bfc0d1070 0x7f9bfc15b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:52:48.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.901+0000 7f9c00e00640 1 --2- 172.21.15.131:0/2489783082 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9bfc145160 0x7f9bfc1f7450 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f9be800ebe0 tx=0x7f9be800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:48.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.901+0000 7f9bf9600640 1 -- 172.21.15.131:0/2489783082 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be8017070 con 0x7f9bfc145160 2024-05-26T22:52:48.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.901+0000 7f9bf9600640 1 -- 172.21.15.131:0/2489783082 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9be800cc30 con 0x7f9bfc145160 2024-05-26T22:52:48.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.901+0000 7f9bf9600640 1 -- 172.21.15.131:0/2489783082 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be8013470 con 0x7f9bfc145160 2024-05-26T22:52:48.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.901+0000 7f9c04200640 1 -- 172.21.15.131:0/2489783082 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bfc1f7ff0 con 0x7f9bfc145160 2024-05-26T22:52:48.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.901+0000 7f9c04200640 1 -- 172.21.15.131:0/2489783082 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bfc0b0a70 con 0x7f9bfc145160 2024-05-26T22:52:48.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.902+0000 7f9c04200640 1 -- 172.21.15.131:0/2489783082 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bfc068b50 con 0x7f9bfc145160 2024-05-26T22:52:48.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.903+0000 7f9bf9600640 1 -- 172.21.15.131:0/2489783082 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9be801a070 con 0x7f9bfc145160 2024-05-26T22:52:48.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.904+0000 7f9bf9600640 1 --2- 172.21.15.131:0/2489783082 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9bdc06be60 0x7f9bdc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:48.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.904+0000 7f9bf9600640 1 -- 172.21.15.131:0/2489783082 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5115+0+0 (secure 0 0 0) 0x7f9be808ab60 con 0x7f9bfc145160 2024-05-26T22:52:48.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.904+0000 7f9c01800640 1 --2- 172.21.15.131:0/2489783082 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9bdc06be60 0x7f9bdc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:48.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.905+0000 7f9c01800640 1 --2- 172.21.15.131:0/2489783082 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9bdc06be60 0x7f9bdc06e320 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f9bf0005880 tx=0x7f9bf0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:48.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:48.908+0000 7f9bf9600640 1 -- 172.21.15.131:0/2489783082 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f9bfc068b50 con 0x7f9bfc145160 2024-05-26T22:52:49.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.112+0000 7f9c04200640 1 -- 172.21.15.131:0/2489783082 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f9bfc0695c0 con 0x7f9bfc145160 2024-05-26T22:52:49.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.234+0000 7f9bf9600640 1 -- 172.21.15.131:0/2489783082 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v63) ==== 97+0+0 (secure 0 0 0) 0x7f9be8059750 con 0x7f9bfc145160 2024-05-26T22:52:49.234 INFO:teuthology.orchestra.run.smithi131.stderr: removed all blocklist entries 2024-05-26T22:52:49.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.238+0000 7f9c04200640 1 -- 172.21.15.131:0/2489783082 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9bdc06be60 msgr2=0x7f9bdc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:49.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.238+0000 7f9c04200640 1 --2- 172.21.15.131:0/2489783082 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9bdc06be60 0x7f9bdc06e320 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f9bf0005880 tx=0x7f9bf0009040 comp rx=0 tx=0).stop 2024-05-26T22:52:49.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.238+0000 7f9c04200640 1 -- 172.21.15.131:0/2489783082 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9bfc145160 msgr2=0x7f9bfc1f7450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:49.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.238+0000 7f9c04200640 1 --2- 172.21.15.131:0/2489783082 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9bfc145160 0x7f9bfc1f7450 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f9be800ebe0 tx=0x7f9be800c520 comp rx=0 tx=0).stop 2024-05-26T22:52:49.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.239+0000 7f9c04200640 1 -- 172.21.15.131:0/2489783082 shutdown_connections 2024-05-26T22:52:49.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.239+0000 7f9c04200640 1 --2- 172.21.15.131:0/2489783082 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9bfc15d520 0x7f9bfc1f7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:49.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.239+0000 7f9c04200640 1 --2- 172.21.15.131:0/2489783082 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9bdc06be60 0x7f9bdc06e320 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:49.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.239+0000 7f9c04200640 1 --2- 172.21.15.131:0/2489783082 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9bfc145160 0x7f9bfc1f7450 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:49.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.239+0000 7f9c04200640 1 --2- 172.21.15.131:0/2489783082 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9bfc0d1070 0x7f9bfc15b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:49.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.239+0000 7f9c04200640 1 -- 172.21.15.131:0/2489783082 >> 172.21.15.131:0/2489783082 conn(0x7f9bfc0b65b0 msgr2=0x7f9bfc142f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:52:49.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.239+0000 7f9c04200640 1 -- 172.21.15.131:0/2489783082 shutdown_connections 2024-05-26T22:52:49.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.239+0000 7f9c04200640 1 -- 172.21.15.131:0/2489783082 wait complete. 2024-05-26T22:52:49.259 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-26T22:52:49.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.448+0000 7fc288600640 1 Processor -- start 2024-05-26T22:52:49.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.448+0000 7fc288600640 1 -- start start 2024-05-26T22:52:49.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.450+0000 7fc288600640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc280155840 0x7fc280071930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:49.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.450+0000 7fc288600640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fc28015c3d0 0x7fc28006ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:49.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.450+0000 7fc288600640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fc28006f490 0x7fc28015ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:49.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.450+0000 7fc288600640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fc2800722a0 con 0x7fc28006f490 2024-05-26T22:52:49.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.450+0000 7fc288600640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fc280072580 con 0x7fc28015c3d0 2024-05-26T22:52:49.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.450+0000 7fc288600640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc280072860 con 0x7fc280155840 2024-05-26T22:52:49.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.450+0000 7fc286600640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fc28006f490 0x7fc28015ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:49.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.450+0000 7fc286600640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fc28006f490 0x7fc28015ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39152/0 (socket says 172.21.15.131:39152) 2024-05-26T22:52:49.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.450+0000 7fc286600640 1 -- 172.21.15.131:0/3738418611 learned_addr learned my addr 172.21.15.131:0/3738418611 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:52:49.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.450+0000 7fc286600640 1 -- 172.21.15.131:0/3738418611 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc280155840 msgr2=0x7fc280071930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:49.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.450+0000 7fc286600640 1 --2- 172.21.15.131:0/3738418611 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc280155840 0x7fc280071930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:49.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.450+0000 7fc286600640 1 -- 172.21.15.131:0/3738418611 >> v2:172.21.15.187:3300/0 conn(0x7fc28015c3d0 msgr2=0x7fc28006ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:49.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.450+0000 7fc286600640 1 --2- 172.21.15.131:0/3738418611 >> v2:172.21.15.187:3300/0 conn(0x7fc28015c3d0 0x7fc28006ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:49.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.450+0000 7fc286600640 1 -- 172.21.15.131:0/3738418611 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc280072b40 con 0x7fc28006f490 2024-05-26T22:52:49.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.451+0000 7fc286600640 1 --2- 172.21.15.131:0/3738418611 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc28006f490 0x7fc28015ec80 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fc28006c250 tx=0x7fc27802a690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=179f1e1e4068aa33 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:49.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.452+0000 7fc27fe00640 1 -- 172.21.15.131:0/3738418611 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc278036070 con 0x7fc28006f490 2024-05-26T22:52:49.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.452+0000 7fc27fe00640 1 -- 172.21.15.131:0/3738418611 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc278031600 con 0x7fc28006f490 2024-05-26T22:52:49.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.452+0000 7fc27fe00640 1 -- 172.21.15.131:0/3738418611 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2780319d0 con 0x7fc28006f490 2024-05-26T22:52:49.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.452+0000 7fc288600640 1 -- 172.21.15.131:0/3738418611 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc28006f490 msgr2=0x7fc28015ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:49.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.452+0000 7fc288600640 1 --2- 172.21.15.131:0/3738418611 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc28006f490 0x7fc28015ec80 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fc28006c250 tx=0x7fc27802a690 comp rx=0 tx=0).stop 2024-05-26T22:52:49.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.452+0000 7fc288600640 1 -- 172.21.15.131:0/3738418611 shutdown_connections 2024-05-26T22:52:49.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.452+0000 7fc288600640 1 --2- 172.21.15.131:0/3738418611 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc28006f490 0x7fc28015ec80 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:49.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.452+0000 7fc288600640 1 --2- 172.21.15.131:0/3738418611 >> v2:172.21.15.187:3300/0 conn(0x7fc28015c3d0 0x7fc28006ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:49.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.452+0000 7fc288600640 1 --2- 172.21.15.131:0/3738418611 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc280155840 0x7fc280071930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:49.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.452+0000 7fc288600640 1 -- 172.21.15.131:0/3738418611 >> 172.21.15.131:0/3738418611 conn(0x7fc2800b1e90 msgr2=0x7fc2800b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:52:49.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.453+0000 7fc288600640 1 -- 172.21.15.131:0/3738418611 shutdown_connections 2024-05-26T22:52:49.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.453+0000 7fc288600640 1 -- 172.21.15.131:0/3738418611 wait complete. 2024-05-26T22:52:49.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.454+0000 7fc288600640 1 Processor -- start 2024-05-26T22:52:49.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.454+0000 7fc288600640 1 -- start start 2024-05-26T22:52:49.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.455+0000 7fc288600640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc28006f490 0x7fc28015dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:49.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.455+0000 7fc288600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc280155840 0x7fc2800b3030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:49.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.455+0000 7fc288600640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc28015c3d0 0x7fc2800b3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:49.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.455+0000 7fc288600640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc280071ff0 con 0x7fc280155840 2024-05-26T22:52:49.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.455+0000 7fc288600640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc280071e70 con 0x7fc28015c3d0 2024-05-26T22:52:49.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.455+0000 7fc288600640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc28015f1c0 con 0x7fc28006f490 2024-05-26T22:52:49.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.456+0000 7fc285200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc280155840 0x7fc2800b3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:49.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.456+0000 7fc285200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc280155840 0x7fc2800b3030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39160/0 (socket says 172.21.15.131:39160) 2024-05-26T22:52:49.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.456+0000 7fc285200640 1 -- 172.21.15.131:0/3860633623 learned_addr learned my addr 172.21.15.131:0/3860633623 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:52:49.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.456+0000 7fc286600640 1 --2- 172.21.15.131:0/3860633623 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc28015c3d0 0x7fc2800b3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:49.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.456+0000 7fc285c00640 1 --2- 172.21.15.131:0/3860633623 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc28006f490 0x7fc28015dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:49.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.456+0000 7fc285200640 1 -- 172.21.15.131:0/3860633623 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc28006f490 msgr2=0x7fc28015dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:49.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.456+0000 7fc285200640 1 --2- 172.21.15.131:0/3860633623 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc28006f490 0x7fc28015dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:49.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.456+0000 7fc285200640 1 -- 172.21.15.131:0/3860633623 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc28015c3d0 msgr2=0x7fc2800b3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:49.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.456+0000 7fc285200640 1 --2- 172.21.15.131:0/3860633623 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc28015c3d0 0x7fc2800b3570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:49.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.456+0000 7fc285200640 1 -- 172.21.15.131:0/3860633623 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2800b3da0 con 0x7fc280155840 2024-05-26T22:52:49.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.456+0000 7fc285200640 1 --2- 172.21.15.131:0/3860633623 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc280155840 0x7fc2800b3030 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fc26c00b410 tx=0x7fc26c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:49.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.457+0000 7fc27d600640 1 -- 172.21.15.131:0/3860633623 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc26c015020 con 0x7fc280155840 2024-05-26T22:52:49.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.457+0000 7fc27d600640 1 -- 172.21.15.131:0/3860633623 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc26c011850 con 0x7fc280155840 2024-05-26T22:52:49.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.457+0000 7fc27d600640 1 -- 172.21.15.131:0/3860633623 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc26c011c40 con 0x7fc280155840 2024-05-26T22:52:49.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.457+0000 7fc288600640 1 -- 172.21.15.131:0/3860633623 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2800b4080 con 0x7fc280155840 2024-05-26T22:52:49.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.457+0000 7fc288600640 1 -- 172.21.15.131:0/3860633623 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2800b3ab0 con 0x7fc280155840 2024-05-26T22:52:49.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.458+0000 7fc288600640 1 -- 172.21.15.131:0/3860633623 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2800694e0 con 0x7fc280155840 2024-05-26T22:52:49.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.459+0000 7fc27d600640 1 -- 172.21.15.131:0/3860633623 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc26c018020 con 0x7fc280155840 2024-05-26T22:52:49.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.460+0000 7fc27d600640 1 --2- 172.21.15.131:0/3860633623 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc26006be30 0x7fc26006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:49.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.460+0000 7fc27d600640 1 -- 172.21.15.131:0/3860633623 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 4857+0+0 (secure 0 0 0) 0x7fc26c08ab50 con 0x7fc280155840 2024-05-26T22:52:49.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.460+0000 7fc285c00640 1 --2- 172.21.15.131:0/3860633623 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc26006be30 0x7fc26006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:49.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.461+0000 7fc285c00640 1 --2- 172.21.15.131:0/3860633623 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc26006be30 0x7fc26006e2f0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fc27400a810 tx=0x7fc274008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:49.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.464+0000 7fc27d600640 1 -- 172.21.15.131:0/3860633623 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fc26c01f020 con 0x7fc280155840 2024-05-26T22:52:49.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.778+0000 7fc288600640 1 -- 172.21.15.131:0/3860633623 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fc2800b4b90 con 0x7fc280155840 2024-05-26T22:52:49.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.779+0000 7fc27d600640 1 -- 172.21.15.131:0/3860633623 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v15) ==== 80+0+5304 (secure 0 0 0) 0x7fc26c0172c0 con 0x7fc280155840 2024-05-26T22:52:49.779 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:52:49.779 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-26T22:52:49.779 INFO:teuthology.orchestra.run.smithi131.stdout: "auth_dump": [ 2024-05-26T22:52:49.779 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:52:49.779 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mds.a", 2024-05-26T22:52:49.779 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD1u1NmTQf1DhAAdvEt7wP0IgKhNHOqjVAllA==", 2024-05-26T22:52:49.779 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:52:49.779 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-05-26T22:52:49.779 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2024-05-26T22:52:49.779 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2024-05-26T22:52:49.779 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-05-26T22:52:49.779 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:52:49.780 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:52:49.780 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:52:49.780 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mds.b", 2024-05-26T22:52:49.780 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD1u1NmHNZ8KxAAPgQ/ojbl+EDfPhqrkg7LPQ==", 2024-05-26T22:52:49.780 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:52:49.780 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-05-26T22:52:49.780 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2024-05-26T22:52:49.780 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2024-05-26T22:52:49.780 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-05-26T22:52:49.780 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:52:49.780 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:52:49.780 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:52:49.780 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mds.c", 2024-05-26T22:52:49.781 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD1u1NmFjCHHRAAqix3Ujfsq2tnSU5+d/pSYA==", 2024-05-26T22:52:49.781 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:52:49.781 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-05-26T22:52:49.781 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2024-05-26T22:52:49.781 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2024-05-26T22:52:49.781 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-05-26T22:52:49.781 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:52:49.781 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:52:49.781 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:52:49.781 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mds.d", 2024-05-26T22:52:49.781 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD1u1NmM4LPORAAhtQtr+Vfpf7vdVwjSW6xtA==", 2024-05-26T22:52:49.781 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:52:49.782 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-05-26T22:52:49.782 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2024-05-26T22:52:49.782 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2024-05-26T22:52:49.782 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-05-26T22:52:49.782 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:52:49.782 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:52:49.782 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:52:49.782 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.0", 2024-05-26T22:52:49.782 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD5u1Nm+w1LNhAAsJLQkiKSh3KkMymlrZkcrw==", 2024-05-26T22:52:49.782 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:52:49.782 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-05-26T22:52:49.782 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-05-26T22:52:49.782 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-05-26T22:52:49.782 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:52:49.783 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:52:49.783 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:52:49.783 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.1", 2024-05-26T22:52:49.783 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD9u1Nm4c8RFBAAcn9HQ+izML9rujJ/2/Q/BQ==", 2024-05-26T22:52:49.783 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:52:49.783 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-05-26T22:52:49.783 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-05-26T22:52:49.783 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-05-26T22:52:49.783 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:52:49.783 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:52:49.783 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:52:49.783 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.2", 2024-05-26T22:52:49.783 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAAvFNmR5wmLRAAYEBAXPBZFq67iARZnwF3iw==", 2024-05-26T22:52:49.784 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:52:49.784 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-05-26T22:52:49.784 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-05-26T22:52:49.784 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-05-26T22:52:49.784 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:52:49.784 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:52:49.784 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:52:49.784 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.3", 2024-05-26T22:52:49.784 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAEvFNmJ4G5CxAAnTbP8DinhmuvGNo1VBVzjQ==", 2024-05-26T22:52:49.784 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:52:49.784 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-05-26T22:52:49.784 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-05-26T22:52:49.784 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-05-26T22:52:49.784 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:52:49.784 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:52:49.785 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:52:49.785 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.4", 2024-05-26T22:52:49.785 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAKvFNmcw5rGhAAvnivP6kt3vcN9UPm/svKqw==", 2024-05-26T22:52:49.785 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:52:49.785 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-05-26T22:52:49.785 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-05-26T22:52:49.785 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-05-26T22:52:49.785 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:52:49.785 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:52:49.785 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:52:49.785 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.5", 2024-05-26T22:52:49.785 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQANvFNmTqlzNhAAGfrSvNNHYKGOBzRXo0TEdw==", 2024-05-26T22:52:49.785 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:52:49.785 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-05-26T22:52:49.786 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-05-26T22:52:49.786 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-05-26T22:52:49.786 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:52:49.786 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:52:49.786 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:52:49.786 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.6", 2024-05-26T22:52:49.786 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQARvFNmpzd6FRAAgi2RYlQPvfHNe0iP4bJrwg==", 2024-05-26T22:52:49.786 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:52:49.786 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-05-26T22:52:49.786 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-05-26T22:52:49.786 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-05-26T22:52:49.786 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:52:49.787 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:52:49.787 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:52:49.787 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.7", 2024-05-26T22:52:49.787 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAUvFNmAkC8LhAAsmkKF4eloxdpCi/mrDtzqA==", 2024-05-26T22:52:49.787 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:52:49.787 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-05-26T22:52:49.787 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-05-26T22:52:49.787 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-05-26T22:52:49.787 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:52:49.787 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:52:49.787 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:52:49.787 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.0", 2024-05-26T22:52:49.787 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD2u1NmSGVWChAAH/wq3iaUs6ZELRhLKhmURQ==", 2024-05-26T22:52:49.787 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:52:49.788 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-05-26T22:52:49.788 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow r", 2024-05-26T22:52:49.788 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-05-26T22:52:49.788 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:52:49.788 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:52:49.788 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:52:49.788 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.1", 2024-05-26T22:52:49.788 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD2u1NmnA9CHRAA2XMmZQxysKweHNykiQUEeA==", 2024-05-26T22:52:49.788 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:52:49.788 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-05-26T22:52:49.788 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow r", 2024-05-26T22:52:49.788 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow rw", 2024-05-26T22:52:49.788 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow rwx" 2024-05-26T22:52:49.788 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:52:49.789 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:52:49.789 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:52:49.789 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.2", 2024-05-26T22:52:49.789 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD2u1NmGEI9ExAAvJn1LRwmALkD7zh/AP07vQ==", 2024-05-26T22:52:49.789 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:52:49.789 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-05-26T22:52:49.789 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow r", 2024-05-26T22:52:49.789 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow rw", 2024-05-26T22:52:49.789 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow rwx" 2024-05-26T22:52:49.789 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:52:49.789 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:52:49.789 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:52:49.789 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.3", 2024-05-26T22:52:49.789 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD2u1NmIHOtJhAAJCUm0kGqWu4rWVLsDkW6Eg==", 2024-05-26T22:52:49.790 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:52:49.790 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-05-26T22:52:49.790 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow r", 2024-05-26T22:52:49.790 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow rw", 2024-05-26T22:52:49.790 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow rwx" 2024-05-26T22:52:49.790 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:52:49.790 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:52:49.790 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:52:49.790 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.admin", 2024-05-26T22:52:49.790 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD0u1NmLRGoCBAAhDeVJV14MTAIdUte067jig==", 2024-05-26T22:52:49.790 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:52:49.790 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow *", 2024-05-26T22:52:49.790 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2024-05-26T22:52:49.790 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2024-05-26T22:52:49.791 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-05-26T22:52:49.791 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:52:49.791 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:52:49.791 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:52:49.791 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mgr.x", 2024-05-26T22:52:49.791 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD1u1NmaWhyBBAAlU7Lq6HMjkCPu0kztJIAXQ==", 2024-05-26T22:52:49.791 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:52:49.791 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow *", 2024-05-26T22:52:49.791 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile mgr", 2024-05-26T22:52:49.791 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-05-26T22:52:49.791 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:52:49.791 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:52:49.791 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:52:49.791 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mgr.y", 2024-05-26T22:52:49.792 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD0u1NmX1RhNhAAo5mEOGzUyrdwFlB/XOXKnw==", 2024-05-26T22:52:49.792 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:52:49.792 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow *", 2024-05-26T22:52:49.792 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile mgr", 2024-05-26T22:52:49.792 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-05-26T22:52:49.792 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:52:49.792 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:52:49.792 INFO:teuthology.orchestra.run.smithi131.stdout: ] 2024-05-26T22:52:49.792 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-26T22:52:49.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.781+0000 7fc288600640 1 -- 172.21.15.131:0/3860633623 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc26006be30 msgr2=0x7fc26006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:49.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.781+0000 7fc288600640 1 --2- 172.21.15.131:0/3860633623 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc26006be30 0x7fc26006e2f0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fc27400a810 tx=0x7fc274008040 comp rx=0 tx=0).stop 2024-05-26T22:52:49.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.781+0000 7fc288600640 1 -- 172.21.15.131:0/3860633623 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc280155840 msgr2=0x7fc2800b3030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:49.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.781+0000 7fc288600640 1 --2- 172.21.15.131:0/3860633623 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc280155840 0x7fc2800b3030 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fc26c00b410 tx=0x7fc26c00b8d0 comp rx=0 tx=0).stop 2024-05-26T22:52:49.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.781+0000 7fc288600640 1 -- 172.21.15.131:0/3860633623 shutdown_connections 2024-05-26T22:52:49.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.781+0000 7fc288600640 1 --2- 172.21.15.131:0/3860633623 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc26006be30 0x7fc26006e2f0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:49.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.781+0000 7fc288600640 1 --2- 172.21.15.131:0/3860633623 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc28015c3d0 0x7fc2800b3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:49.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.781+0000 7fc288600640 1 --2- 172.21.15.131:0/3860633623 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc280155840 0x7fc2800b3030 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:49.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.781+0000 7fc288600640 1 --2- 172.21.15.131:0/3860633623 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc28006f490 0x7fc28015dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:49.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.781+0000 7fc288600640 1 -- 172.21.15.131:0/3860633623 >> 172.21.15.131:0/3860633623 conn(0x7fc2800b1e90 msgr2=0x7fc28006e870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:52:49.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.782+0000 7fc288600640 1 -- 172.21.15.131:0/3860633623 shutdown_connections 2024-05-26T22:52:49.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.782+0000 7fc288600640 1 -- 172.21.15.131:0/3860633623 wait complete. 2024-05-26T22:52:49.800 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-26T22:52:49.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.982+0000 7ff728a00640 1 Processor -- start 2024-05-26T22:52:49.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.982+0000 7ff728a00640 1 -- start start 2024-05-26T22:52:49.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.984+0000 7ff728a00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff72406e8c0 0x7ff72415d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:49.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.984+0000 7ff728a00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff724160620 con 0x7ff7240cc7f0 2024-05-26T22:52:49.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.984+0000 7ff728a00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff724160900 con 0x7ff724156750 2024-05-26T22:52:49.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.984+0000 7ff728a00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff72415cbd0 con 0x7ff72406e8c0 2024-05-26T22:52:49.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.984+0000 7ff723e00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7ff7240cc7f0 0x7ff7240cdc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:58336/0 (socket says 172.21.15.131:58336) 2024-05-26T22:52:49.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.985+0000 7ff723e00640 1 -- 172.21.15.131:0/2518538036 learned_addr learned my addr 172.21.15.131:0/2518538036 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:52:49.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.985+0000 7ff71be00640 1 --2- 172.21.15.131:0/2518538036 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff72406e8c0 0x7ff72415d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:49.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.985+0000 7ff722a00640 1 -- 172.21.15.131:0/2518538036 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2687966313 0 0) 0x7ff724160620 con 0x7ff7240cc7f0 2024-05-26T22:52:49.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.985+0000 7ff722a00640 1 -- 172.21.15.131:0/2518538036 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff708003520 con 0x7ff7240cc7f0 2024-05-26T22:52:49.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.985+0000 7ff722a00640 1 -- 172.21.15.131:0/2518538036 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2301420113 0 0) 0x7ff708003520 con 0x7ff7240cc7f0 2024-05-26T22:52:49.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.985+0000 7ff722a00640 1 -- 172.21.15.131:0/2518538036 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff72406e8c0 msgr2=0x7ff72415d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:49.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.985+0000 7ff722a00640 1 --2- 172.21.15.131:0/2518538036 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff72406e8c0 0x7ff72415d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:49.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.985+0000 7ff722a00640 1 -- 172.21.15.131:0/2518538036 >> v1:172.21.15.187:6789/0 conn(0x7ff724156750 legacy=0x7ff72415fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:49.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.985+0000 7ff722a00640 1 -- 172.21.15.131:0/2518538036 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff724160be0 con 0x7ff7240cc7f0 2024-05-26T22:52:49.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.985+0000 7ff722a00640 1 -- 172.21.15.131:0/2518538036 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7ff71c0019f0 con 0x7ff7240cc7f0 2024-05-26T22:52:49.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.985+0000 7ff722a00640 1 -- 172.21.15.131:0/2518538036 >> v1:172.21.15.131:6789/0 conn(0x7ff7240cc7f0 legacy=0x7ff7240cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:49.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.985+0000 7ff722a00640 1 --2- 172.21.15.131:0/2518538036 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7080039e0 0x7ff708003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:49.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.986+0000 7ff722a00640 1 --2- 172.21.15.131:0/2518538036 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff708004800 0x7ff708004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:49.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.986+0000 7ff722a00640 1 --2- 172.21.15.131:0/2518538036 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff72406e8c0 0x7ff708005250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:49.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.986+0000 7ff722a00640 1 -- 172.21.15.131:0/2518538036 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff72415cbd0 con 0x7ff7080039e0 2024-05-26T22:52:49.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.986+0000 7ff722a00640 1 -- 172.21.15.131:0/2518538036 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff70800bb50 con 0x7ff708004800 2024-05-26T22:52:49.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.986+0000 7ff722a00640 1 -- 172.21.15.131:0/2518538036 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff70800bd50 con 0x7ff72406e8c0 2024-05-26T22:52:49.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.986+0000 7ff723400640 1 --2- 172.21.15.131:0/2518538036 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7080039e0 0x7ff708003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:49.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.986+0000 7ff71be00640 1 --2- 172.21.15.131:0/2518538036 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff708004800 0x7ff708004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:49.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.986+0000 7ff723400640 1 -- 172.21.15.131:0/2518538036 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff72406e8c0 msgr2=0x7ff708005250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:49.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.986+0000 7ff723400640 1 --2- 172.21.15.131:0/2518538036 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff72406e8c0 0x7ff708005250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:49.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.986+0000 7ff723400640 1 -- 172.21.15.131:0/2518538036 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff708004800 msgr2=0x7ff708004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:49.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.986+0000 7ff723400640 1 --2- 172.21.15.131:0/2518538036 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff708004800 0x7ff708004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:49.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.986+0000 7ff723400640 1 -- 172.21.15.131:0/2518538036 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff708003520 con 0x7ff7080039e0 2024-05-26T22:52:49.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.987+0000 7ff723400640 1 --2- 172.21.15.131:0/2518538036 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7080039e0 0x7ff708003e00 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7ff714001750 tx=0x7ff708005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9bc3763a0c34cae server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:49.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.987+0000 7ff722a00640 1 -- 172.21.15.131:0/2518538036 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff714001370 con 0x7ff7080039e0 2024-05-26T22:52:49.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.987+0000 7ff722a00640 1 -- 172.21.15.131:0/2518538036 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7140052d0 con 0x7ff7080039e0 2024-05-26T22:52:49.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.987+0000 7ff722a00640 1 -- 172.21.15.131:0/2518538036 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff71402abe0 con 0x7ff7080039e0 2024-05-26T22:52:49.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.988+0000 7ff728a00640 1 -- 172.21.15.131:0/2518538036 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7080039e0 msgr2=0x7ff708003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:49.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.988+0000 7ff728a00640 1 --2- 172.21.15.131:0/2518538036 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7080039e0 0x7ff708003e00 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7ff714001750 tx=0x7ff708005830 comp rx=0 tx=0).stop 2024-05-26T22:52:49.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.988+0000 7ff723e00640 1 -- 172.21.15.131:0/2518538036 reap_dead start 2024-05-26T22:52:49.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.988+0000 7ff728a00640 1 -- 172.21.15.131:0/2518538036 shutdown_connections 2024-05-26T22:52:49.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.988+0000 7ff728a00640 1 -- 172.21.15.131:0/2518538036 >> 172.21.15.131:0/2518538036 conn(0x7ff7240cff40 msgr2=0x7ff7240d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:52:49.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.988+0000 7ff728a00640 1 -- 172.21.15.131:0/2518538036 shutdown_connections 2024-05-26T22:52:49.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.988+0000 7ff728a00640 1 -- 172.21.15.131:0/2518538036 wait complete. 2024-05-26T22:52:49.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.990+0000 7ff728a00640 1 Processor -- start 2024-05-26T22:52:49.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.990+0000 7ff728a00640 1 -- start start 2024-05-26T22:52:49.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.991+0000 7ff728a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7080039e0 0x7ff72414b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:49.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.991+0000 7ff728a00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff724155f20 0x7ff724156cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:49.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.991+0000 7ff728a00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff72415b630 0x7ff72415bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:49.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.991+0000 7ff728a00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff7240ce370 con 0x7ff7080039e0 2024-05-26T22:52:49.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.991+0000 7ff728a00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff724156b30 con 0x7ff72415b630 2024-05-26T22:52:49.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.991+0000 7ff728a00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff7240ce4f0 con 0x7ff724155f20 2024-05-26T22:52:49.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.991+0000 7ff723400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7080039e0 0x7ff72414b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:49.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.991+0000 7ff723400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7080039e0 0x7ff72414b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39174/0 (socket says 172.21.15.131:39174) 2024-05-26T22:52:49.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.991+0000 7ff723400640 1 -- 172.21.15.131:0/59742415 learned_addr learned my addr 172.21.15.131:0/59742415 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:52:49.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.991+0000 7ff723400640 1 -- 172.21.15.131:0/59742415 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff724155f20 msgr2=0x7ff724156cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:49.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.991+0000 7ff723400640 1 --2- 172.21.15.131:0/59742415 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff724155f20 0x7ff724156cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:49.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.991+0000 7ff723400640 1 -- 172.21.15.131:0/59742415 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff72415b630 msgr2=0x7ff72415bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:49.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.991+0000 7ff723400640 1 --2- 172.21.15.131:0/59742415 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff72415b630 0x7ff72415bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:49.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.991+0000 7ff723400640 1 -- 172.21.15.131:0/59742415 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7241575b0 con 0x7ff7080039e0 2024-05-26T22:52:49.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.992+0000 7ff723400640 1 --2- 172.21.15.131:0/59742415 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7080039e0 0x7ff72414b800 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7ff714001870 tx=0x7ff714001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:49.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.992+0000 7ff71b400640 1 -- 172.21.15.131:0/59742415 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff714040020 con 0x7ff7080039e0 2024-05-26T22:52:49.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.992+0000 7ff71b400640 1 -- 172.21.15.131:0/59742415 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff714007680 con 0x7ff7080039e0 2024-05-26T22:52:49.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.992+0000 7ff71b400640 1 -- 172.21.15.131:0/59742415 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff714007a30 con 0x7ff7080039e0 2024-05-26T22:52:49.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.992+0000 7ff728a00640 1 -- 172.21.15.131:0/59742415 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff724157890 con 0x7ff7080039e0 2024-05-26T22:52:49.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.992+0000 7ff728a00640 1 -- 172.21.15.131:0/59742415 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff724157e50 con 0x7ff7080039e0 2024-05-26T22:52:49.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.993+0000 7ff728a00640 1 -- 172.21.15.131:0/59742415 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7240694e0 con 0x7ff7080039e0 2024-05-26T22:52:49.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.994+0000 7ff71b400640 1 -- 172.21.15.131:0/59742415 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff714043020 con 0x7ff7080039e0 2024-05-26T22:52:49.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.995+0000 7ff71b400640 1 --2- 172.21.15.131:0/59742415 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ff70006be30 0x7ff70006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:49.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.995+0000 7ff71b400640 1 -- 172.21.15.131:0/59742415 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 4857+0+0 (secure 0 0 0) 0x7ff7140aab80 con 0x7ff7080039e0 2024-05-26T22:52:49.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.995+0000 7ff71be00640 1 --2- 172.21.15.131:0/59742415 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ff70006be30 0x7ff70006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:49.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.996+0000 7ff71be00640 1 --2- 172.21.15.131:0/59742415 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ff70006be30 0x7ff70006e2f0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7ff70c006f00 tx=0x7ff70c006e10 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:49.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:49.999+0000 7ff71b400640 1 -- 172.21.15.131:0/59742415 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7ff71404a030 con 0x7ff7080039e0 2024-05-26T22:52:50.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.198+0000 7ff728a00640 1 -- 172.21.15.131:0/59742415 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff7240ced60 con 0x7ff7080039e0 2024-05-26T22:52:50.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.199+0000 7ff71b400640 1 -- 172.21.15.131:0/59742415 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) ==== 74+0+12915 (secure 0 0 0) 0x7ff7140797e0 con 0x7ff7080039e0 2024-05-26T22:52:50.199 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:52:50.199 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":65,"fsid":"46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f","created":"2024-05-26T22:47:57.288905+0000","modified":"2024-05-26T22:52:49.246552+0000","last_up_change":"2024-05-26T22:48:09.602207+0000","last_in_change":"2024-05-26T22:48:00.687341+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-26T22:48:10.463339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"814eaf62-a18d-494b-8e27-fd8fb88adea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6801","nonce":2350940051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6803","nonce":2350940051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6807","nonce":2350940051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6805","nonce":2350940051}]},"public_addr":"172.21.15.131:6801/2350940051","cluster_addr":"172.21.15.131:6803/2350940051","heartbeat_back_addr":"172.21.15.131:6807/2350940051","heartbeat_front_addr":"172.21.15.131:6805/2350940051","state":["exists","up"]},{"osd":1,"uuid":"42e66287-f6c6-4e26-af3c-6dfaa3536d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6825","nonce":1525528247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6827","nonce":1525528247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6831","nonce":1525528247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6829","nonce":1525528247}]},"public_addr":"172.21.15.131:6825/1525528247","cluster_addr":"172.21.15.131:6827/1525528247","heartbeat_back_addr":"172.21.15.131:6831/1525528247","heartbeat_front_addr":"172.21.15.131:6829/1525528247","state":["exists","up"]},{"osd":2,"uuid":"0da34673-04a2-4fd0-b57a-0ff2b679b043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6817","nonce":3627842210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6819","nonce":3627842210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6823","nonce":3627842210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6821","nonce":3627842210}]},"public_addr":"172.21.15.131:6817/3627842210","cluster_addr":"172.21.15.131:6819/3627842210","heartbeat_back_addr":"172.21.15.131:6823/3627842210","heartbeat_front_addr":"172.21.15.131:6821/3627842210","state":["exists","up"]},{"osd":3,"uuid":"cf117a27-d6b2-43d3-ae52-b47988bf8eed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6809","nonce":350790591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6811","nonce":350790591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6815","nonce":350790591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6813","nonce":350790591}]},"public_addr":"172.21.15.131:6809/350790591","cluster_addr":"172.21.15.131:6811/350790591","heartbeat_back_addr":"172.21.15.131:6815/350790591","heartbeat_front_addr":"172.21.15.131:6813/350790591","state":["exists","up"]},{"osd":4,"uuid":"a092ccfc-be89-4a84-88d5-0f781fa715eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6816","nonce":244560995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6819","nonce":244560995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6823","nonce":244560995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6821","nonce":244560995}]},"public_addr":"172.21.15.187:6816/244560995","cluster_addr":"172.21.15.187:6819/244560995","heartbeat_back_addr":"172.21.15.187:6823/244560995","heartbeat_front_addr":"172.21.15.187:6821/244560995","state":["exists","up"]},{"osd":5,"uuid":"7513ba69-dd00-4a35-b190-9e8cc3b2bdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6801","nonce":324619398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6803","nonce":324619398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6812","nonce":324619398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6807","nonce":324619398}]},"public_addr":"172.21.15.187:6801/324619398","cluster_addr":"172.21.15.187:6803/324619398","heartbeat_back_addr":"172.21.15.187:6812/324619398","heartbeat_front_addr":"172.21.15.187:6807/324619398","state":["exists","up"]},{"osd":6,"uuid":"50081810-c093-4d2f-a4d7-b7c695c5400d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6806","nonce":1391347786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6809","nonce":1391347786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6818","nonce":1391347786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6813","nonce":1391347786}]},"public_addr":"172.21.15.187:6806/1391347786","cluster_addr":"172.21.15.187:6809/1391347786","heartbeat_back_addr":"172.21.15.187:6818/1391347786","heartbeat_front_addr":"172.21.15.187:6813/1391347786","state":["exists","up"]},{"osd":7,"uuid":"775dae05-54f5-4ad4-bbce-92863c6ed2ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6825","nonce":1220065622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6827","nonce":1220065622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6831","nonce":1220065622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6829","nonce":1220065622}]},"public_addr":"172.21.15.187:6825/1220065622","cluster_addr":"172.21.15.187:6827/1220065622","heartbeat_back_addr":"172.21.15.187:6831/1220065622","heartbeat_front_addr":"172.21.15.187:6829/1220065622","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:05.983520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.299516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.067323+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.307748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.018742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.258775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.010912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.295355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-26T22:52:50.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.202+0000 7ff728a00640 1 -- 172.21.15.131:0/59742415 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ff70006be30 msgr2=0x7ff70006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:50.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.202+0000 7ff728a00640 1 --2- 172.21.15.131:0/59742415 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ff70006be30 0x7ff70006e2f0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7ff70c006f00 tx=0x7ff70c006e10 comp rx=0 tx=0).stop 2024-05-26T22:52:50.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.202+0000 7ff728a00640 1 -- 172.21.15.131:0/59742415 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7080039e0 msgr2=0x7ff72414b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:50.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.202+0000 7ff728a00640 1 --2- 172.21.15.131:0/59742415 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7080039e0 0x7ff72414b800 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7ff714001870 tx=0x7ff714001120 comp rx=0 tx=0).stop 2024-05-26T22:52:50.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.202+0000 7ff728a00640 1 -- 172.21.15.131:0/59742415 shutdown_connections 2024-05-26T22:52:50.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.202+0000 7ff728a00640 1 --2- 172.21.15.131:0/59742415 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff724155f20 0x7ff724156cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:50.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.202+0000 7ff728a00640 1 --2- 172.21.15.131:0/59742415 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ff70006be30 0x7ff70006e2f0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:50.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.202+0000 7ff728a00640 1 --2- 172.21.15.131:0/59742415 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff72415b630 0x7ff72415bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:50.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.202+0000 7ff728a00640 1 --2- 172.21.15.131:0/59742415 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7080039e0 0x7ff72414b800 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:50.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.202+0000 7ff728a00640 1 -- 172.21.15.131:0/59742415 >> 172.21.15.131:0/59742415 conn(0x7ff7240cff40 msgr2=0x7ff7240cf400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:52:50.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.202+0000 7ff728a00640 1 -- 172.21.15.131:0/59742415 shutdown_connections 2024-05-26T22:52:50.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.202+0000 7ff728a00640 1 -- 172.21.15.131:0/59742415 wait complete. 2024-05-26T22:52:50.221 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-26T22:48:10.463339+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-26T22:52:50.221 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-26T22:52:50.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.414+0000 7f5776800640 1 Processor -- start 2024-05-26T22:52:50.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.415+0000 7f5776800640 1 -- start start 2024-05-26T22:52:50.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.416+0000 7f5776800640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f577006f180 0x7f5770072e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:50.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.416+0000 7f5776800640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f57700d0560 0x7f5770161ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:50.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.416+0000 7f5776800640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57700af8e0 con 0x7f5770072540 2024-05-26T22:52:50.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.416+0000 7f5776800640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f57700cfb30 con 0x7f577006f180 2024-05-26T22:52:50.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.416+0000 7f5776800640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f57700cf200 con 0x7f57700d0560 2024-05-26T22:52:50.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.416+0000 7f576f400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f577006f180 0x7f5770072e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:50.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.417+0000 7f576ea00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f5770072540 0x7f57700cfe50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:58352/0 (socket says 172.21.15.131:58352) 2024-05-26T22:52:50.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.417+0000 7f576ea00640 1 -- 172.21.15.131:0/797222807 learned_addr learned my addr 172.21.15.131:0/797222807 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:52:50.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.417+0000 7f576fe00640 1 --2- 172.21.15.131:0/797222807 >> v2:172.21.15.187:3300/0 conn(0x7f57700d0560 0x7f5770161ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:50.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.417+0000 7f576f400640 1 -- 172.21.15.131:0/797222807 >> v2:172.21.15.187:3300/0 conn(0x7f57700d0560 msgr2=0x7f5770161ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:50.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.417+0000 7f576f400640 1 --2- 172.21.15.131:0/797222807 >> v2:172.21.15.187:3300/0 conn(0x7f57700d0560 0x7f5770161ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:50.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.417+0000 7f576f400640 1 -- 172.21.15.131:0/797222807 >> v1:172.21.15.131:6789/0 conn(0x7f5770072540 legacy=0x7f57700cfe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:50.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.417+0000 7f576f400640 1 -- 172.21.15.131:0/797222807 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57700cf4e0 con 0x7f577006f180 2024-05-26T22:52:50.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.417+0000 7f576fe00640 1 --2- 172.21.15.131:0/797222807 >> v2:172.21.15.187:3300/0 conn(0x7f57700d0560 0x7f5770161ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:52:50.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.417+0000 7f576e000640 1 -- 172.21.15.131:0/797222807 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3402171168 0 0) 0x7f57700af8e0 con 0x7f5770072540 2024-05-26T22:52:50.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.418+0000 7f576f400640 1 --2- 172.21.15.131:0/797222807 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f577006f180 0x7f5770072e90 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f57700cfcb0 tx=0x7f576002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70abdf6885055a1 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:50.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.419+0000 7f576e000640 1 -- 172.21.15.131:0/797222807 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5760037070 con 0x7f577006f180 2024-05-26T22:52:50.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.419+0000 7f576e000640 1 -- 172.21.15.131:0/797222807 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5760032440 con 0x7f577006f180 2024-05-26T22:52:50.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.419+0000 7f576e000640 1 -- 172.21.15.131:0/797222807 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5760032810 con 0x7f577006f180 2024-05-26T22:52:50.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.419+0000 7f5776800640 1 -- 172.21.15.131:0/797222807 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f577006f180 msgr2=0x7f5770072e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:50.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.419+0000 7f5776800640 1 --2- 172.21.15.131:0/797222807 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f577006f180 0x7f5770072e90 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f57700cfcb0 tx=0x7f576002a1a0 comp rx=0 tx=0).stop 2024-05-26T22:52:50.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.420+0000 7f5776800640 1 -- 172.21.15.131:0/797222807 shutdown_connections 2024-05-26T22:52:50.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.420+0000 7f5776800640 1 --2- 172.21.15.131:0/797222807 >> v2:172.21.15.187:3300/0 conn(0x7f57700d0560 0x7f5770161ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:50.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.420+0000 7f5776800640 1 --2- 172.21.15.131:0/797222807 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f577006f180 0x7f5770072e90 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:50.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.420+0000 7f5776800640 1 -- 172.21.15.131:0/797222807 >> 172.21.15.131:0/797222807 conn(0x7f577006bc90 msgr2=0x7f57700b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:52:50.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.420+0000 7f5776800640 1 -- 172.21.15.131:0/797222807 shutdown_connections 2024-05-26T22:52:50.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.420+0000 7f5776800640 1 -- 172.21.15.131:0/797222807 wait complete. 2024-05-26T22:52:50.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.421+0000 7f5776800640 1 Processor -- start 2024-05-26T22:52:50.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.422+0000 7f5776800640 1 -- start start 2024-05-26T22:52:50.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.422+0000 7f5776800640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f577006f180 0x7f57700ada90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:50.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.422+0000 7f5776800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5770072540 0x7f5770159b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:50.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.422+0000 7f5776800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f57700d0560 0x7f577015a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:50.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.422+0000 7f5776800640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f577006f6e0 con 0x7f5770072540 2024-05-26T22:52:50.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.422+0000 7f5776800640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f577006f560 con 0x7f577006f180 2024-05-26T22:52:50.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.422+0000 7f5776800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f57700715a0 con 0x7f57700d0560 2024-05-26T22:52:50.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.423+0000 7f576ea00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5770072540 0x7f5770159b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:50.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.423+0000 7f576ea00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5770072540 0x7f5770159b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39194/0 (socket says 172.21.15.131:39194) 2024-05-26T22:52:50.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.423+0000 7f576ea00640 1 -- 172.21.15.131:0/2030887598 learned_addr learned my addr 172.21.15.131:0/2030887598 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:52:50.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.423+0000 7f576f400640 1 --2- 172.21.15.131:0/2030887598 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f577006f180 0x7f57700ada90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:50.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.423+0000 7f576fe00640 1 --2- 172.21.15.131:0/2030887598 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f57700d0560 0x7f577015a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:50.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.423+0000 7f576ea00640 1 -- 172.21.15.131:0/2030887598 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f57700d0560 msgr2=0x7f577015a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:50.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.423+0000 7f576ea00640 1 --2- 172.21.15.131:0/2030887598 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f57700d0560 0x7f577015a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:50.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.423+0000 7f576ea00640 1 -- 172.21.15.131:0/2030887598 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f577006f180 msgr2=0x7f57700ada90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:50.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.423+0000 7f576ea00640 1 --2- 172.21.15.131:0/2030887598 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f577006f180 0x7f57700ada90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:50.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.423+0000 7f576ea00640 1 -- 172.21.15.131:0/2030887598 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f577015a6a0 con 0x7f5770072540 2024-05-26T22:52:50.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.423+0000 7f576f400640 1 --2- 172.21.15.131:0/2030887598 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f577006f180 0x7f57700ada90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:52:50.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.423+0000 7f576fe00640 1 --2- 172.21.15.131:0/2030887598 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f57700d0560 0x7f577015a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:52:50.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.424+0000 7f576ea00640 1 --2- 172.21.15.131:0/2030887598 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5770072540 0x7f5770159b30 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f575800c500 tx=0x7f575800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:50.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.424+0000 7f5767400640 1 -- 172.21.15.131:0/2030887598 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5758018070 con 0x7f5770072540 2024-05-26T22:52:50.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.424+0000 7f5767400640 1 -- 172.21.15.131:0/2030887598 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5758007580 con 0x7f5770072540 2024-05-26T22:52:50.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.424+0000 7f5767400640 1 -- 172.21.15.131:0/2030887598 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5758004a20 con 0x7f5770072540 2024-05-26T22:52:50.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.424+0000 7f5776800640 1 -- 172.21.15.131:0/2030887598 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f577015a980 con 0x7f5770072540 2024-05-26T22:52:50.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.424+0000 7f5776800640 1 -- 172.21.15.131:0/2030887598 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f577015b220 con 0x7f5770072540 2024-05-26T22:52:50.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.425+0000 7f5776800640 1 -- 172.21.15.131:0/2030887598 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5770074e90 con 0x7f5770072540 2024-05-26T22:52:50.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.426+0000 7f5767400640 1 -- 172.21.15.131:0/2030887598 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5758004bc0 con 0x7f5770072540 2024-05-26T22:52:50.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.427+0000 7f5767400640 1 --2- 172.21.15.131:0/2030887598 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f574c06be30 0x7f574c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:50.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.427+0000 7f5767400640 1 -- 172.21.15.131:0/2030887598 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 4857+0+0 (secure 0 0 0) 0x7f5758089ed0 con 0x7f5770072540 2024-05-26T22:52:50.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.427+0000 7f576f400640 1 --2- 172.21.15.131:0/2030887598 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f574c06be30 0x7f574c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:50.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.428+0000 7f576f400640 1 --2- 172.21.15.131:0/2030887598 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f574c06be30 0x7f574c06e2f0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f5760034000 tx=0x7f576002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:50.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.431+0000 7f5767400640 1 -- 172.21.15.131:0/2030887598 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f5758058b30 con 0x7f5770072540 2024-05-26T22:52:50.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.615+0000 7f5776800640 1 -- 172.21.15.131:0/2030887598 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f577015b420 con 0x7f5770072540 2024-05-26T22:52:50.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.616+0000 7f5767400640 1 -- 172.21.15.131:0/2030887598 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v65) ==== 93+0+10 (secure 0 0 0) 0x7f577015b420 con 0x7f5770072540 2024-05-26T22:52:50.616 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2024-05-26T22:52:50.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.619+0000 7f5776800640 1 -- 172.21.15.131:0/2030887598 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f574c06be30 msgr2=0x7f574c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:50.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.619+0000 7f5776800640 1 --2- 172.21.15.131:0/2030887598 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f574c06be30 0x7f574c06e2f0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f5760034000 tx=0x7f576002e000 comp rx=0 tx=0).stop 2024-05-26T22:52:50.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.619+0000 7f5776800640 1 -- 172.21.15.131:0/2030887598 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5770072540 msgr2=0x7f5770159b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:50.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.619+0000 7f5776800640 1 --2- 172.21.15.131:0/2030887598 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5770072540 0x7f5770159b30 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f575800c500 tx=0x7f575800c9c0 comp rx=0 tx=0).stop 2024-05-26T22:52:50.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.619+0000 7f5776800640 1 -- 172.21.15.131:0/2030887598 shutdown_connections 2024-05-26T22:52:50.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.619+0000 7f5776800640 1 --2- 172.21.15.131:0/2030887598 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f57700d0560 0x7f577015a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:50.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.619+0000 7f5776800640 1 --2- 172.21.15.131:0/2030887598 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f574c06be30 0x7f574c06e2f0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:50.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.619+0000 7f5776800640 1 --2- 172.21.15.131:0/2030887598 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5770072540 0x7f5770159b30 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:50.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.619+0000 7f5776800640 1 --2- 172.21.15.131:0/2030887598 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f577006f180 0x7f57700ada90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:50.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.619+0000 7f5776800640 1 -- 172.21.15.131:0/2030887598 >> 172.21.15.131:0/2030887598 conn(0x7f577006bc90 msgr2=0x7f57700b5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:52:50.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.619+0000 7f5776800640 1 -- 172.21.15.131:0/2030887598 shutdown_connections 2024-05-26T22:52:50.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.619+0000 7f5776800640 1 -- 172.21.15.131:0/2030887598 wait complete. 2024-05-26T22:52:50.637 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-26T22:52:50.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.827+0000 7f84d5400640 1 Processor -- start 2024-05-26T22:52:50.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.827+0000 7f84d5400640 1 -- start start 2024-05-26T22:52:50.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.829+0000 7f84d5400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f84d00cc7f0 0x7f84d00ccc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:50.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.829+0000 7f84d5400640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84d0160620 con 0x7f84d0156750 2024-05-26T22:52:50.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.829+0000 7f84d5400640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84d0160900 con 0x7f84d006e8c0 2024-05-26T22:52:50.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.829+0000 7f84d5400640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f84d015d770 con 0x7f84d00cc7f0 2024-05-26T22:52:50.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.829+0000 7f84cfe00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f84d0156750 0x7f84d015fd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:58362/0 (socket says 172.21.15.131:58362) 2024-05-26T22:52:50.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.829+0000 7f84cfe00640 1 -- 172.21.15.131:0/3939096940 learned_addr learned my addr 172.21.15.131:0/3939096940 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:52:50.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.829+0000 7f84d4a00640 1 --2- 172.21.15.131:0/3939096940 >> v2:172.21.15.131:3300/0 conn(0x7f84d00cc7f0 0x7f84d00ccc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:50.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.829+0000 7f84d4a00640 1 -- 172.21.15.131:0/3939096940 >> v1:172.21.15.187:6789/0 conn(0x7f84d006e8c0 legacy=0x7f84d015d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:50.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.830+0000 7f84d4a00640 1 -- 172.21.15.131:0/3939096940 >> v1:172.21.15.131:6789/0 conn(0x7f84d0156750 legacy=0x7f84d015fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:50.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.830+0000 7f84d4a00640 1 -- 172.21.15.131:0/3939096940 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84d0160be0 con 0x7f84d00cc7f0 2024-05-26T22:52:50.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.830+0000 7f84cea00640 1 -- 172.21.15.131:0/3939096940 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3336850908 0 0) 0x7f84d0160620 con 0x7f84d0156750 2024-05-26T22:52:50.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.830+0000 7f84d4a00640 1 --2- 172.21.15.131:0/3939096940 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84d00cc7f0 0x7f84d00ccc60 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f84c80097a0 tx=0x7f84c802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5952a2c19b2db44d server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:50.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.831+0000 7f84cea00640 1 -- 172.21.15.131:0/3939096940 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84c8037070 con 0x7f84d00cc7f0 2024-05-26T22:52:50.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.831+0000 7f84cea00640 1 -- 172.21.15.131:0/3939096940 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84c8032440 con 0x7f84d00cc7f0 2024-05-26T22:52:50.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.831+0000 7f84cea00640 1 -- 172.21.15.131:0/3939096940 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84c8032810 con 0x7f84d00cc7f0 2024-05-26T22:52:50.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.831+0000 7f84d5400640 1 -- 172.21.15.131:0/3939096940 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84d00cc7f0 msgr2=0x7f84d00ccc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:50.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.831+0000 7f84d5400640 1 --2- 172.21.15.131:0/3939096940 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84d00cc7f0 0x7f84d00ccc60 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f84c80097a0 tx=0x7f84c802a1e0 comp rx=0 tx=0).stop 2024-05-26T22:52:50.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.831+0000 7f84d5400640 1 -- 172.21.15.131:0/3939096940 shutdown_connections 2024-05-26T22:52:50.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.831+0000 7f84d5400640 1 --2- 172.21.15.131:0/3939096940 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84d00cc7f0 0x7f84d00ccc60 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:50.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.831+0000 7f84d5400640 1 -- 172.21.15.131:0/3939096940 >> 172.21.15.131:0/3939096940 conn(0x7f84d00cff40 msgr2=0x7f84d00d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:52:50.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.831+0000 7f84d5400640 1 -- 172.21.15.131:0/3939096940 shutdown_connections 2024-05-26T22:52:50.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.831+0000 7f84d5400640 1 -- 172.21.15.131:0/3939096940 wait complete. 2024-05-26T22:52:50.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.833+0000 7f84d5400640 1 Processor -- start 2024-05-26T22:52:50.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.833+0000 7f84d5400640 1 -- start start 2024-05-26T22:52:50.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.834+0000 7f84d5400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f84d006e8c0 0x7f84d012f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:50.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.834+0000 7f84d5400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84d00cc7f0 0x7f84d0130a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:50.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.834+0000 7f84d5400640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f84d0156750 0x7f84d0155930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:50.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.834+0000 7f84d5400640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f84d015cbd0 con 0x7f84d00cc7f0 2024-05-26T22:52:50.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.834+0000 7f84d5400640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f84d00cd1a0 con 0x7f84d0156750 2024-05-26T22:52:50.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.834+0000 7f84d5400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f84d0156b30 con 0x7f84d006e8c0 2024-05-26T22:52:50.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.834+0000 7f84cf400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84d00cc7f0 0x7f84d0130a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:50.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.834+0000 7f84cf400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84d00cc7f0 0x7f84d0130a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39222/0 (socket says 172.21.15.131:39222) 2024-05-26T22:52:50.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.834+0000 7f84cf400640 1 -- 172.21.15.131:0/3138175603 learned_addr learned my addr 172.21.15.131:0/3138175603 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:52:50.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.835+0000 7f84cf400640 1 -- 172.21.15.131:0/3138175603 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f84d006e8c0 msgr2=0x7f84d012f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:50.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.835+0000 7f84cf400640 1 --2- 172.21.15.131:0/3138175603 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f84d006e8c0 0x7f84d012f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:50.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.835+0000 7f84cf400640 1 -- 172.21.15.131:0/3138175603 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f84d0156750 msgr2=0x7f84d0155930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:50.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.835+0000 7f84cf400640 1 --2- 172.21.15.131:0/3138175603 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f84d0156750 0x7f84d0155930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:50.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.835+0000 7f84cf400640 1 -- 172.21.15.131:0/3138175603 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84d012f9a0 con 0x7f84d00cc7f0 2024-05-26T22:52:50.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.835+0000 7f84cf400640 1 --2- 172.21.15.131:0/3138175603 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84d00cc7f0 0x7f84d0130a40 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f84b800c7c0 tx=0x7f84b800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:50.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.835+0000 7f84c7e00640 1 -- 172.21.15.131:0/3138175603 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b8016020 con 0x7f84d00cc7f0 2024-05-26T22:52:50.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.835+0000 7f84c7e00640 1 -- 172.21.15.131:0/3138175603 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84b800aa50 con 0x7f84d00cc7f0 2024-05-26T22:52:50.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.835+0000 7f84c7e00640 1 -- 172.21.15.131:0/3138175603 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b800ae40 con 0x7f84d00cc7f0 2024-05-26T22:52:50.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.835+0000 7f84d5400640 1 -- 172.21.15.131:0/3138175603 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84d012fc80 con 0x7f84d00cc7f0 2024-05-26T22:52:50.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.835+0000 7f84d5400640 1 -- 172.21.15.131:0/3138175603 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84d0130240 con 0x7f84d00cc7f0 2024-05-26T22:52:50.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.836+0000 7f84d5400640 1 -- 172.21.15.131:0/3138175603 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84d00694e0 con 0x7f84d00cc7f0 2024-05-26T22:52:50.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.837+0000 7f84c7e00640 1 -- 172.21.15.131:0/3138175603 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f84b8019020 con 0x7f84d00cc7f0 2024-05-26T22:52:50.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.838+0000 7f84c7e00640 1 --2- 172.21.15.131:0/3138175603 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f84ac06be30 0x7f84ac06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:50.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.838+0000 7f84c7e00640 1 -- 172.21.15.131:0/3138175603 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 4857+0+0 (secure 0 0 0) 0x7f84b808ae70 con 0x7f84d00cc7f0 2024-05-26T22:52:50.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.839+0000 7f84cfe00640 1 --2- 172.21.15.131:0/3138175603 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f84ac06be30 0x7f84ac06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:50.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.839+0000 7f84cfe00640 1 --2- 172.21.15.131:0/3138175603 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f84ac06be30 0x7f84ac06e2f0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f84c0007fd0 tx=0x7f84c0007960 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:50.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:50.844+0000 7f84c7e00640 1 -- 172.21.15.131:0/3138175603 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f84b8020020 con 0x7f84d00cc7f0 2024-05-26T22:52:51.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:51.039+0000 7f84d5400640 1 -- 172.21.15.131:0/3138175603 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f84d015b330 con 0x7f84d00cc7f0 2024-05-26T22:52:51.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:51.039+0000 7f84c7e00640 1 -- 172.21.15.131:0/3138175603 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v65) ==== 84+0+65 (secure 0 0 0) 0x7f84b8059c00 con 0x7f84d00cc7f0 2024-05-26T22:52:51.039 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:52:51.039 INFO:teuthology.orchestra.run.smithi131.stdout:[ 2024-05-26T22:52:51.039 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:52:51.039 INFO:teuthology.orchestra.run.smithi131.stdout: "poolnum": 1, 2024-05-26T22:52:51.040 INFO:teuthology.orchestra.run.smithi131.stdout: "poolname": ".mgr" 2024-05-26T22:52:51.040 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:52:51.040 INFO:teuthology.orchestra.run.smithi131.stdout:] 2024-05-26T22:52:51.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:51.042+0000 7f84d5400640 1 -- 172.21.15.131:0/3138175603 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f84ac06be30 msgr2=0x7f84ac06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:51.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:51.042+0000 7f84d5400640 1 --2- 172.21.15.131:0/3138175603 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f84ac06be30 0x7f84ac06e2f0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f84c0007fd0 tx=0x7f84c0007960 comp rx=0 tx=0).stop 2024-05-26T22:52:51.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:51.042+0000 7f84d5400640 1 -- 172.21.15.131:0/3138175603 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84d00cc7f0 msgr2=0x7f84d0130a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:51.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:51.042+0000 7f84d5400640 1 --2- 172.21.15.131:0/3138175603 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84d00cc7f0 0x7f84d0130a40 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f84b800c7c0 tx=0x7f84b800cc80 comp rx=0 tx=0).stop 2024-05-26T22:52:51.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:51.042+0000 7f84d5400640 1 -- 172.21.15.131:0/3138175603 shutdown_connections 2024-05-26T22:52:51.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:51.042+0000 7f84d5400640 1 --2- 172.21.15.131:0/3138175603 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f84ac06be30 0x7f84ac06e2f0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:51.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:51.042+0000 7f84d5400640 1 --2- 172.21.15.131:0/3138175603 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f84d0156750 0x7f84d0155930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:51.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:51.042+0000 7f84d5400640 1 --2- 172.21.15.131:0/3138175603 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84d00cc7f0 0x7f84d0130a40 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:51.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:51.042+0000 7f84d5400640 1 --2- 172.21.15.131:0/3138175603 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f84d006e8c0 0x7f84d012f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:51.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:51.042+0000 7f84d5400640 1 -- 172.21.15.131:0/3138175603 >> 172.21.15.131:0/3138175603 conn(0x7f84d00cff40 msgr2=0x7f84d00cdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:52:51.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:51.043+0000 7f84d5400640 1 -- 172.21.15.131:0/3138175603 shutdown_connections 2024-05-26T22:52:51.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:51.043+0000 7f84d5400640 1 -- 172.21.15.131:0/3138175603 wait complete. 2024-05-26T22:52:51.062 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-26T22:52:51.063 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-26T22:52:51.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:51.265+0000 7f8373c00640 1 Processor -- start 2024-05-26T22:52:51.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:51.265+0000 7f8373c00640 1 -- start start 2024-05-26T22:52:51.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:51.267+0000 7f8373c00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f836c0d1b70 0x7f836c166da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:51.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:51.267+0000 7f8373c00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f836c1672e0 con 0x7f836c0d0780 2024-05-26T22:52:51.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:51.267+0000 7f8373c00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f836c0ced50 con 0x7f836c06b310 2024-05-26T22:52:51.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:51.267+0000 7f8373c00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f836c0cf030 con 0x7f836c0d1b70 2024-05-26T22:52:51.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:51.268+0000 7f8363e00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f836c0d1b70 0x7f836c166da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:51.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:51.268+0000 7f8363e00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f836c0d1b70 0x7f836c166da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.131:53340/0 (socket says 172.21.15.131:53340) 2024-05-26T22:52:51.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:51.268+0000 7f8363e00640 1 -- 172.21.15.131:0/2638974370 learned_addr learned my addr 172.21.15.131:0/2638974370 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:52:51.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:51.268+0000 7f8371c00640 1 --1- 172.21.15.131:0/2638974370 >> v1:172.21.15.131:6789/0 conn(0x7f836c0d0780 0x7f836c06e400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:58372/0 (socket says 172.21.15.131:58372) 2024-05-26T22:52:51.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:51.268+0000 7f836be00640 1 -- 172.21.15.131:0/2638974370 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2124882249 0 0) 0x7f836c1672e0 con 0x7f836c0d0780 2024-05-26T22:52:51.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:51.268+0000 7f836be00640 1 -- 172.21.15.131:0/2638974370 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8350003520 con 0x7f836c0d0780 2024-05-26T22:52:51.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:51.269+0000 7f8363e00640 1 -- 172.21.15.131:0/2638974370 >> v1:172.21.15.187:6789/0 conn(0x7f836c06b310 legacy=0x7f836c06b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:51.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:51.269+0000 7f8363e00640 1 -- 172.21.15.131:0/2638974370 >> v1:172.21.15.131:6789/0 conn(0x7f836c0d0780 legacy=0x7f836c06e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:51.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:51.269+0000 7f8363e00640 1 -- 172.21.15.131:0/2638974370 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f836c1665a0 con 0x7f836c0d1b70 2024-05-26T22:52:51.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:51.270+0000 7f8363e00640 1 --2- 172.21.15.131:0/2638974370 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f836c0d1b70 0x7f836c166da0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f8354009870 tx=0x7f835402a1e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44ae6fea9db2a655 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:51.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:51.270+0000 7f836be00640 1 -- 172.21.15.131:0/2638974370 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8354037070 con 0x7f836c0d1b70 2024-05-26T22:52:51.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:51.270+0000 7f836be00640 1 -- 172.21.15.131:0/2638974370 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8354032440 con 0x7f836c0d1b70 2024-05-26T22:52:51.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:51.270+0000 7f836be00640 1 -- 172.21.15.131:0/2638974370 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83540327f0 con 0x7f836c0d1b70 2024-05-26T22:52:51.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:51.271+0000 7f8373c00640 1 -- 172.21.15.131:0/2638974370 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f836c0d1b70 msgr2=0x7f836c166da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:51.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:51.271+0000 7f8373c00640 1 --2- 172.21.15.131:0/2638974370 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f836c0d1b70 0x7f836c166da0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f8354009870 tx=0x7f835402a1e0 comp rx=0 tx=0).stop 2024-05-26T22:52:51.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:51.271+0000 7f8373c00640 1 -- 172.21.15.131:0/2638974370 shutdown_connections 2024-05-26T22:52:51.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:51.271+0000 7f8373c00640 1 --2- 172.21.15.131:0/2638974370 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f836c0d1b70 0x7f836c166da0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:51.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:51.271+0000 7f8373c00640 1 -- 172.21.15.131:0/2638974370 >> 172.21.15.131:0/2638974370 conn(0x7f836c0d1050 msgr2=0x7f836c0d1460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:52:51.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:51.271+0000 7f8373c00640 1 -- 172.21.15.131:0/2638974370 shutdown_connections 2024-05-26T22:52:51.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:51.271+0000 7f8373c00640 1 -- 172.21.15.131:0/2638974370 wait complete. 2024-05-26T22:52:51.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:51.273+0000 7f8373c00640 1 Processor -- start 2024-05-26T22:52:51.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:51.273+0000 7f8373c00640 1 -- start start 2024-05-26T22:52:51.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:51.274+0000 7f8373c00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f836c06b310 0x7f836c1f8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:51.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:51.274+0000 7f8373c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f836c0d0780 0x7f836c0b4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:51.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:51.274+0000 7f8373c00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f836c0d1b70 0x7f836c0b5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:51.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:51.274+0000 7f8373c00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f836c0d0d90 con 0x7f836c0d0780 2024-05-26T22:52:51.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:51.274+0000 7f8373c00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f836c0d0c10 con 0x7f836c06b310 2024-05-26T22:52:51.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:51.274+0000 7f8373c00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f836c162b50 con 0x7f836c0d1b70 2024-05-26T22:52:51.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:51.274+0000 7f8363e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f836c0d0780 0x7f836c0b4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:51.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:51.274+0000 7f8363e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f836c0d0780 0x7f836c0b4290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39238/0 (socket says 172.21.15.131:39238) 2024-05-26T22:52:51.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:51.274+0000 7f8363e00640 1 -- 172.21.15.131:0/3906374930 learned_addr learned my addr 172.21.15.131:0/3906374930 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:52:51.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:51.274+0000 7f8371c00640 1 --2- 172.21.15.131:0/3906374930 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f836c0d1b70 0x7f836c0b5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:51.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:51.274+0000 7f8371200640 1 --2- 172.21.15.131:0/3906374930 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f836c06b310 0x7f836c1f8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:51.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:51.274+0000 7f8363e00640 1 -- 172.21.15.131:0/3906374930 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f836c0d1b70 msgr2=0x7f836c0b5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:51.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:51.274+0000 7f8363e00640 1 --2- 172.21.15.131:0/3906374930 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f836c0d1b70 0x7f836c0b5ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:51.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:51.274+0000 7f8363e00640 1 -- 172.21.15.131:0/3906374930 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f836c06b310 msgr2=0x7f836c1f8ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:51.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:51.274+0000 7f8363e00640 1 --2- 172.21.15.131:0/3906374930 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f836c06b310 0x7f836c1f8ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:51.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:51.274+0000 7f8363e00640 1 -- 172.21.15.131:0/3906374930 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f836c0b6b10 con 0x7f836c0d0780 2024-05-26T22:52:51.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:51.274+0000 7f8371c00640 1 --2- 172.21.15.131:0/3906374930 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f836c0d1b70 0x7f836c0b5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:52:51.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:51.274+0000 7f8371200640 1 --2- 172.21.15.131:0/3906374930 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f836c06b310 0x7f836c1f8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:52:51.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:51.275+0000 7f8363e00640 1 --2- 172.21.15.131:0/3906374930 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f836c0d0780 0x7f836c0b4290 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f8354009870 tx=0x7f83540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:51.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:51.275+0000 7f8369600640 1 -- 172.21.15.131:0/3906374930 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8354037040 con 0x7f836c0d0780 2024-05-26T22:52:51.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:51.275+0000 7f8369600640 1 -- 172.21.15.131:0/3906374930 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8354004230 con 0x7f836c0d0780 2024-05-26T22:52:51.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:51.275+0000 7f8373c00640 1 -- 172.21.15.131:0/3906374930 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f836c0b6830 con 0x7f836c0d0780 2024-05-26T22:52:51.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:51.275+0000 7f8369600640 1 -- 172.21.15.131:0/3906374930 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f835403c670 con 0x7f836c0d0780 2024-05-26T22:52:51.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:51.275+0000 7f8373c00640 1 -- 172.21.15.131:0/3906374930 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f836c1f86e0 con 0x7f836c0d0780 2024-05-26T22:52:51.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:51.276+0000 7f8373c00640 1 -- 172.21.15.131:0/3906374930 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f836c068b50 con 0x7f836c0d0780 2024-05-26T22:52:51.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:51.277+0000 7f8369600640 1 -- 172.21.15.131:0/3906374930 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f835403a070 con 0x7f836c0d0780 2024-05-26T22:52:51.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:51.278+0000 7f8369600640 1 --2- 172.21.15.131:0/3906374930 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f834806be30 0x7f834806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:51.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:51.278+0000 7f8369600640 1 -- 172.21.15.131:0/3906374930 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 4857+0+0 (secure 0 0 0) 0x7f83540ab4d0 con 0x7f836c0d0780 2024-05-26T22:52:51.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:51.279+0000 7f8371200640 1 --2- 172.21.15.131:0/3906374930 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f834806be30 0x7f834806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:51.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:51.279+0000 7f8371200640 1 --2- 172.21.15.131:0/3906374930 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f834806be30 0x7f834806e2f0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f836c0b2950 tx=0x7f835c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:51.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:51.283+0000 7f8369600640 1 -- 172.21.15.131:0/3906374930 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f8354041020 con 0x7f836c0d0780 2024-05-26T22:52:51.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:51.485+0000 7f8373c00640 1 -- 172.21.15.131:0/3906374930 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f836c06c080 con 0x7f836c0d0780 2024-05-26T22:52:52.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:52.264+0000 7f8369600640 1 -- 172.21.15.131:0/3906374930 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v66) ==== 138+0+0 (secure 0 0 0) 0x7f835407a130 con 0x7f836c0d0780 2024-05-26T22:52:52.264 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_metadata' created 2024-05-26T22:52:52.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:52.268+0000 7f8373c00640 1 -- 172.21.15.131:0/3906374930 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f834806be30 msgr2=0x7f834806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:52.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:52.268+0000 7f8373c00640 1 --2- 172.21.15.131:0/3906374930 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f834806be30 0x7f834806e2f0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f836c0b2950 tx=0x7f835c009040 comp rx=0 tx=0).stop 2024-05-26T22:52:52.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:52.268+0000 7f8373c00640 1 -- 172.21.15.131:0/3906374930 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f836c0d0780 msgr2=0x7f836c0b4290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:52.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:52.268+0000 7f8373c00640 1 --2- 172.21.15.131:0/3906374930 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f836c0d0780 0x7f836c0b4290 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f8354009870 tx=0x7f83540046c0 comp rx=0 tx=0).stop 2024-05-26T22:52:52.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:52.268+0000 7f8373c00640 1 -- 172.21.15.131:0/3906374930 shutdown_connections 2024-05-26T22:52:52.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:52.268+0000 7f8373c00640 1 --2- 172.21.15.131:0/3906374930 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f836c0d1b70 0x7f836c0b5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:52.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:52.268+0000 7f8373c00640 1 --2- 172.21.15.131:0/3906374930 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f834806be30 0x7f834806e2f0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:52.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:52.268+0000 7f8373c00640 1 --2- 172.21.15.131:0/3906374930 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f836c0d0780 0x7f836c0b4290 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:52.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:52.268+0000 7f8373c00640 1 --2- 172.21.15.131:0/3906374930 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f836c06b310 0x7f836c1f8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:52.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:52.268+0000 7f8373c00640 1 -- 172.21.15.131:0/3906374930 >> 172.21.15.131:0/3906374930 conn(0x7f836c0d1050 msgr2=0x7f836c0d1430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:52:52.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:52.268+0000 7f8373c00640 1 -- 172.21.15.131:0/3906374930 shutdown_connections 2024-05-26T22:52:52.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:52.268+0000 7f8373c00640 1 -- 172.21.15.131:0/3906374930 wait complete. 2024-05-26T22:52:52.293 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-26T22:52:52.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:52.492+0000 7fe1ee800640 1 Processor -- start 2024-05-26T22:52:52.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:52.492+0000 7fe1ee800640 1 -- start start 2024-05-26T22:52:52.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:52.494+0000 7fe1ee800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe1e80cbef0 0x7fe1e80cc2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:52.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:52.494+0000 7fe1ee800640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fe1e80acef0 0x7fe1e806d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:52.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:52.494+0000 7fe1ee800640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1e80cdca0 con 0x7fe1e80ad370 2024-05-26T22:52:52.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:52.494+0000 7fe1ee800640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fe1e80cdf80 con 0x7fe1e80acef0 2024-05-26T22:52:52.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:52.494+0000 7fe1ee800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe1e80ce260 con 0x7fe1e80cbef0 2024-05-26T22:52:52.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:52.495+0000 7fe1e7e00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fe1e80ad370 0x7fe1e80cea60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:58386/0 (socket says 172.21.15.131:58386) 2024-05-26T22:52:52.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:52.495+0000 7fe1e7e00640 1 -- 172.21.15.131:0/3570341719 learned_addr learned my addr 172.21.15.131:0/3570341719 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:52:52.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:52.495+0000 7fe1dea00640 1 --2- 172.21.15.131:0/3570341719 >> v2:172.21.15.187:3300/0 conn(0x7fe1e80acef0 0x7fe1e806d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:52.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:52.495+0000 7fe1e6a00640 1 -- 172.21.15.131:0/3570341719 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3316717851 0 0) 0x7fe1e80cdca0 con 0x7fe1e80ad370 2024-05-26T22:52:52.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:52.495+0000 7fe1e6a00640 1 -- 172.21.15.131:0/3570341719 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe1cc003520 con 0x7fe1e80ad370 2024-05-26T22:52:52.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:52.495+0000 7fe1e6a00640 1 -- 172.21.15.131:0/3570341719 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1100974248 0 0) 0x7fe1cc003520 con 0x7fe1e80ad370 2024-05-26T22:52:52.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:52.495+0000 7fe1e6a00640 1 -- 172.21.15.131:0/3570341719 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe1e80cbef0 msgr2=0x7fe1e80cc2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:52.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:52.495+0000 7fe1e6a00640 1 --2- 172.21.15.131:0/3570341719 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe1e80cbef0 0x7fe1e80cc2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:52.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:52.495+0000 7fe1e6a00640 1 -- 172.21.15.131:0/3570341719 >> v2:172.21.15.187:3300/0 conn(0x7fe1e80acef0 msgr2=0x7fe1e806d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:52.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:52.495+0000 7fe1e6a00640 1 --2- 172.21.15.131:0/3570341719 >> v2:172.21.15.187:3300/0 conn(0x7fe1e80acef0 0x7fe1e806d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:52.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:52.495+0000 7fe1e6a00640 1 -- 172.21.15.131:0/3570341719 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1e80ce540 con 0x7fe1e80ad370 2024-05-26T22:52:52.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:52.495+0000 7fe1e6a00640 1 -- 172.21.15.131:0/3570341719 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7fe1e0001670 con 0x7fe1e80ad370 2024-05-26T22:52:52.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:52.495+0000 7fe1e6a00640 1 -- 172.21.15.131:0/3570341719 >> v1:172.21.15.131:6789/0 conn(0x7fe1e80ad370 legacy=0x7fe1e80cea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:52.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:52.496+0000 7fe1e6a00640 1 --2- 172.21.15.131:0/3570341719 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe1e80cbef0 0x7fe1cc0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:52.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:52.496+0000 7fe1e6a00640 1 --2- 172.21.15.131:0/3570341719 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe1cc003f30 0x7fe1cc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:52.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:52.496+0000 7fe1e6a00640 1 --2- 172.21.15.131:0/3570341719 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe1cc004d40 0x7fe1cc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:52.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:52.496+0000 7fe1e6a00640 1 -- 172.21.15.131:0/3570341719 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe1e80cdf80 con 0x7fe1cc003f30 2024-05-26T22:52:52.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:52.496+0000 7fe1e6a00640 1 -- 172.21.15.131:0/3570341719 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe1e80ce260 con 0x7fe1cc004d40 2024-05-26T22:52:52.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:52.496+0000 7fe1e6a00640 1 -- 172.21.15.131:0/3570341719 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe1cc00baf0 con 0x7fe1e80cbef0 2024-05-26T22:52:52.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:52.496+0000 7fe1dea00640 1 --2- 172.21.15.131:0/3570341719 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe1cc003f30 0x7fe1cc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:52.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:52.496+0000 7fe1e7e00640 1 --2- 172.21.15.131:0/3570341719 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe1cc004d40 0x7fe1cc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:52.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:52.496+0000 7fe1dea00640 1 -- 172.21.15.131:0/3570341719 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe1e80cbef0 msgr2=0x7fe1cc0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:52.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:52.496+0000 7fe1dea00640 1 --2- 172.21.15.131:0/3570341719 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe1e80cbef0 0x7fe1cc0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:52.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:52.496+0000 7fe1dea00640 1 -- 172.21.15.131:0/3570341719 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe1cc004d40 msgr2=0x7fe1cc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:52.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:52.496+0000 7fe1dea00640 1 --2- 172.21.15.131:0/3570341719 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe1cc004d40 0x7fe1cc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:52.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:52.496+0000 7fe1dea00640 1 -- 172.21.15.131:0/3570341719 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1cc003520 con 0x7fe1cc003f30 2024-05-26T22:52:52.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:52.496+0000 7fe1e7e00640 1 --2- 172.21.15.131:0/3570341719 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe1cc004d40 0x7fe1cc0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:52:52.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:52.498+0000 7fe1dea00640 1 --2- 172.21.15.131:0/3570341719 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe1cc003f30 0x7fe1cc004800 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fe1d000f270 tx=0x7fe1cc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da8ae16bbd82dde1 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:52.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:52.498+0000 7fe1e6a00640 1 -- 172.21.15.131:0/3570341719 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1d0034020 con 0x7fe1cc003f30 2024-05-26T22:52:52.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:52.498+0000 7fe1e6a00640 1 -- 172.21.15.131:0/3570341719 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1d0008440 con 0x7fe1cc003f30 2024-05-26T22:52:52.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:52.498+0000 7fe1e6a00640 1 -- 172.21.15.131:0/3570341719 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1d000ad20 con 0x7fe1cc003f30 2024-05-26T22:52:52.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:52.498+0000 7fe1ee800640 1 -- 172.21.15.131:0/3570341719 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe1cc003f30 msgr2=0x7fe1cc004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:52.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:52.498+0000 7fe1ee800640 1 --2- 172.21.15.131:0/3570341719 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe1cc003f30 0x7fe1cc004800 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fe1d000f270 tx=0x7fe1cc005830 comp rx=0 tx=0).stop 2024-05-26T22:52:52.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:52.498+0000 7fe1e7e00640 1 -- 172.21.15.131:0/3570341719 reap_dead start 2024-05-26T22:52:52.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:52.499+0000 7fe1ee800640 1 -- 172.21.15.131:0/3570341719 shutdown_connections 2024-05-26T22:52:52.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:52.499+0000 7fe1ee800640 1 -- 172.21.15.131:0/3570341719 >> 172.21.15.131:0/3570341719 conn(0x7fe1e806bc90 msgr2=0x7fe1e80b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:52:52.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:52.499+0000 7fe1ee800640 1 -- 172.21.15.131:0/3570341719 shutdown_connections 2024-05-26T22:52:52.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:52.499+0000 7fe1ee800640 1 -- 172.21.15.131:0/3570341719 wait complete. 2024-05-26T22:52:52.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:52.501+0000 7fe1ee800640 1 Processor -- start 2024-05-26T22:52:52.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:52.501+0000 7fe1ee800640 1 -- start start 2024-05-26T22:52:52.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:52.502+0000 7fe1ee800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe1cc003f30 0x7fe1e8159730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:52.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:52.502+0000 7fe1ee800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe1e815d1e0 0x7fe1e80ad9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:52.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:52.502+0000 7fe1ee800640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe1e80b4530 0x7fe1e81f4630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:52.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:52.502+0000 7fe1ee800640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe1e80cf500 con 0x7fe1e815d1e0 2024-05-26T22:52:52.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:52.502+0000 7fe1ee800640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe1e80cf380 con 0x7fe1e80b4530 2024-05-26T22:52:52.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:52.502+0000 7fe1ee800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe1e806d860 con 0x7fe1cc003f30 2024-05-26T22:52:52.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:52.502+0000 7fe1dea00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe1e815d1e0 0x7fe1e80ad9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:52.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:52.502+0000 7fe1dea00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe1e815d1e0 0x7fe1e80ad9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39254/0 (socket says 172.21.15.131:39254) 2024-05-26T22:52:52.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:52.502+0000 7fe1dea00640 1 -- 172.21.15.131:0/1807491105 learned_addr learned my addr 172.21.15.131:0/1807491105 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:52:52.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:52.503+0000 7fe1e7e00640 1 --2- 172.21.15.131:0/1807491105 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe1e80b4530 0x7fe1e81f4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:52.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:52.503+0000 7fe1e7400640 1 --2- 172.21.15.131:0/1807491105 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe1cc003f30 0x7fe1e8159730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:52.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:52.503+0000 7fe1dea00640 1 -- 172.21.15.131:0/1807491105 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe1cc003f30 msgr2=0x7fe1e8159730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:52.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:52.503+0000 7fe1dea00640 1 --2- 172.21.15.131:0/1807491105 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe1cc003f30 0x7fe1e8159730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:52.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:52.503+0000 7fe1dea00640 1 -- 172.21.15.131:0/1807491105 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe1e80b4530 msgr2=0x7fe1e81f4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:52.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:52.503+0000 7fe1dea00640 1 --2- 172.21.15.131:0/1807491105 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe1e80b4530 0x7fe1e81f4630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:52.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:52.503+0000 7fe1dea00640 1 -- 172.21.15.131:0/1807491105 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1e815a440 con 0x7fe1e815d1e0 2024-05-26T22:52:52.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:52.503+0000 7fe1e7400640 1 --2- 172.21.15.131:0/1807491105 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe1cc003f30 0x7fe1e8159730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:52:52.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:52.503+0000 7fe1dea00640 1 --2- 172.21.15.131:0/1807491105 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe1e815d1e0 0x7fe1e80ad9f0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fe1d000f390 tx=0x7fe1d000a860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:52.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:52.503+0000 7fe1dfe00640 1 -- 172.21.15.131:0/1807491105 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1d0034040 con 0x7fe1e815d1e0 2024-05-26T22:52:52.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:52.503+0000 7fe1dfe00640 1 -- 172.21.15.131:0/1807491105 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1d0003990 con 0x7fe1e815d1e0 2024-05-26T22:52:52.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:52.503+0000 7fe1ee800640 1 -- 172.21.15.131:0/1807491105 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1e80b3d70 con 0x7fe1e815d1e0 2024-05-26T22:52:52.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:52.503+0000 7fe1dfe00640 1 -- 172.21.15.131:0/1807491105 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1d0003d80 con 0x7fe1e815d1e0 2024-05-26T22:52:52.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:52.504+0000 7fe1ee800640 1 -- 172.21.15.131:0/1807491105 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1e815a160 con 0x7fe1e815d1e0 2024-05-26T22:52:52.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:52.505+0000 7fe1ee800640 1 -- 172.21.15.131:0/1807491105 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1e8074e90 con 0x7fe1e815d1e0 2024-05-26T22:52:52.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:52.506+0000 7fe1dfe00640 1 -- 172.21.15.131:0/1807491105 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe1d0005d10 con 0x7fe1e815d1e0 2024-05-26T22:52:52.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:52.507+0000 7fe1dfe00640 1 --2- 172.21.15.131:0/1807491105 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fe1c406be30 0x7fe1c406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:52.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:52.507+0000 7fe1dfe00640 1 -- 172.21.15.131:0/1807491105 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5227+0+0 (secure 0 0 0) 0x7fe1d00ac1e0 con 0x7fe1e815d1e0 2024-05-26T22:52:52.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:52.507+0000 7fe1e7400640 1 --2- 172.21.15.131:0/1807491105 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fe1c406be30 0x7fe1c406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:52.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:52.508+0000 7fe1e7400640 1 --2- 172.21.15.131:0/1807491105 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fe1c406be30 0x7fe1c406e2f0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fe1d8006fd0 tx=0x7fe1d8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:52.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:52.511+0000 7fe1dfe00640 1 -- 172.21.15.131:0/1807491105 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fe1d0051020 con 0x7fe1e815d1e0 2024-05-26T22:52:52.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:52.714+0000 7fe1ee800640 1 -- 172.21.15.131:0/1807491105 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fe1e80695c0 con 0x7fe1e815d1e0 2024-05-26T22:52:53.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:53.280+0000 7fe1dfe00640 1 -- 172.21.15.131:0/1807491105 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v67) ==== 170+0+0 (secure 0 0 0) 0x7fe1d007ade0 con 0x7fe1e815d1e0 2024-05-26T22:52:53.280 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_data' created 2024-05-26T22:52:53.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:53.284+0000 7fe1ee800640 1 -- 172.21.15.131:0/1807491105 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fe1c406be30 msgr2=0x7fe1c406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:53.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:53.284+0000 7fe1ee800640 1 --2- 172.21.15.131:0/1807491105 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fe1c406be30 0x7fe1c406e2f0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fe1d8006fd0 tx=0x7fe1d8008040 comp rx=0 tx=0).stop 2024-05-26T22:52:53.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:53.284+0000 7fe1ee800640 1 -- 172.21.15.131:0/1807491105 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe1e815d1e0 msgr2=0x7fe1e80ad9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:53.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:53.284+0000 7fe1ee800640 1 --2- 172.21.15.131:0/1807491105 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe1e815d1e0 0x7fe1e80ad9f0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fe1d000f390 tx=0x7fe1d000a860 comp rx=0 tx=0).stop 2024-05-26T22:52:53.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:53.284+0000 7fe1ee800640 1 -- 172.21.15.131:0/1807491105 shutdown_connections 2024-05-26T22:52:53.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:53.284+0000 7fe1ee800640 1 --2- 172.21.15.131:0/1807491105 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fe1c406be30 0x7fe1c406e2f0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:53.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:53.284+0000 7fe1ee800640 1 --2- 172.21.15.131:0/1807491105 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe1e80b4530 0x7fe1e81f4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:53.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:53.284+0000 7fe1ee800640 1 --2- 172.21.15.131:0/1807491105 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe1e815d1e0 0x7fe1e80ad9f0 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:53.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:53.284+0000 7fe1ee800640 1 --2- 172.21.15.131:0/1807491105 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe1cc003f30 0x7fe1e8159730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:53.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:53.284+0000 7fe1ee800640 1 -- 172.21.15.131:0/1807491105 >> 172.21.15.131:0/1807491105 conn(0x7fe1e806bc90 msgr2=0x7fe1e8154ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:52:53.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:53.284+0000 7fe1ee800640 1 -- 172.21.15.131:0/1807491105 shutdown_connections 2024-05-26T22:52:53.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:53.284+0000 7fe1ee800640 1 -- 172.21.15.131:0/1807491105 wait complete. 2024-05-26T22:52:53.322 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-26T22:52:53.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:53.500+0000 7f7d14a00640 1 Processor -- start 2024-05-26T22:52:53.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:53.500+0000 7f7d14a00640 1 -- start start 2024-05-26T22:52:53.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:53.501+0000 7f7d14a00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f7d100d0560 0x7f7d10072540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:53.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:53.501+0000 7f7d14a00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f7d100af230 0x7f7d100cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:53.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:53.501+0000 7f7d14a00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d100733c0 con 0x7f7d10165e30 2024-05-26T22:52:53.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:53.501+0000 7f7d14a00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f7d10072e90 con 0x7f7d100d0560 2024-05-26T22:52:53.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:53.501+0000 7f7d14a00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f7d100cf540 con 0x7f7d100af230 2024-05-26T22:52:53.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:53.502+0000 7f7d05400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f7d100d0560 0x7f7d10072540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:53.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:53.502+0000 7f7d05400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f7d100d0560 0x7f7d10072540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39256/0 (socket says 172.21.15.131:39256) 2024-05-26T22:52:53.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:53.502+0000 7f7d05400640 1 -- 172.21.15.131:0/3715590330 learned_addr learned my addr 172.21.15.131:0/3715590330 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:52:53.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:53.502+0000 7f7d0e800640 1 --2- 172.21.15.131:0/3715590330 >> v2:172.21.15.187:3300/0 conn(0x7f7d100af230 0x7f7d100cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:53.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:53.502+0000 7f7d05400640 1 -- 172.21.15.131:0/3715590330 >> v2:172.21.15.187:3300/0 conn(0x7f7d100af230 msgr2=0x7f7d100cdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:53.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:53.502+0000 7f7d05400640 1 --2- 172.21.15.131:0/3715590330 >> v2:172.21.15.187:3300/0 conn(0x7f7d100af230 0x7f7d100cdcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:53.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:53.502+0000 7f7d05400640 1 -- 172.21.15.131:0/3715590330 >> v1:172.21.15.187:6789/0 conn(0x7f7d10165e30 legacy=0x7f7d100cfe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:53.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:53.502+0000 7f7d05400640 1 -- 172.21.15.131:0/3715590330 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d100730e0 con 0x7f7d100d0560 2024-05-26T22:52:53.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:53.503+0000 7f7d05400640 1 --2- 172.21.15.131:0/3715590330 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d100d0560 0x7f7d10072540 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f7d100d0fa0 tx=0x7f7cf802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=850f7d44639655af server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:53.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:53.504+0000 7f7d0d400640 1 -- 172.21.15.131:0/3715590330 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cf8037070 con 0x7f7d100d0560 2024-05-26T22:52:53.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:53.504+0000 7f7d0d400640 1 -- 172.21.15.131:0/3715590330 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cf8032440 con 0x7f7d100d0560 2024-05-26T22:52:53.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:53.504+0000 7f7d0d400640 1 -- 172.21.15.131:0/3715590330 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cf8032810 con 0x7f7d100d0560 2024-05-26T22:52:53.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:53.504+0000 7f7d14a00640 1 -- 172.21.15.131:0/3715590330 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d100d0560 msgr2=0x7f7d10072540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:53.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:53.504+0000 7f7d14a00640 1 --2- 172.21.15.131:0/3715590330 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d100d0560 0x7f7d10072540 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f7d100d0fa0 tx=0x7f7cf802a1a0 comp rx=0 tx=0).stop 2024-05-26T22:52:53.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:53.504+0000 7f7d14a00640 1 -- 172.21.15.131:0/3715590330 shutdown_connections 2024-05-26T22:52:53.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:53.504+0000 7f7d14a00640 1 --2- 172.21.15.131:0/3715590330 >> v2:172.21.15.187:3300/0 conn(0x7f7d100af230 0x7f7d100cdcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:53.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:53.504+0000 7f7d14a00640 1 --2- 172.21.15.131:0/3715590330 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d100d0560 0x7f7d10072540 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:53.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:53.504+0000 7f7d14a00640 1 -- 172.21.15.131:0/3715590330 >> 172.21.15.131:0/3715590330 conn(0x7f7d1006bc90 msgr2=0x7f7d100b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:52:53.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:53.505+0000 7f7d14a00640 1 -- 172.21.15.131:0/3715590330 shutdown_connections 2024-05-26T22:52:53.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:53.505+0000 7f7d14a00640 1 -- 172.21.15.131:0/3715590330 wait complete. 2024-05-26T22:52:53.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:53.506+0000 7f7d14a00640 1 Processor -- start 2024-05-26T22:52:53.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:53.506+0000 7f7d14a00640 1 -- start start 2024-05-26T22:52:53.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:53.507+0000 7f7d14a00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7d100af230 0x7f7d101fc8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:53.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:53.507+0000 7f7d14a00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d100d0560 0x7f7d1014f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:53.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:53.507+0000 7f7d14a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d10165e30 0x7f7d100b3d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:53.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:53.507+0000 7f7d14a00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f7d100ce210 con 0x7f7d10165e30 2024-05-26T22:52:53.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:53.507+0000 7f7d14a00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7d100cfac0 con 0x7f7d100d0560 2024-05-26T22:52:53.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:53.507+0000 7f7d14a00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f7d100ce390 con 0x7f7d100af230 2024-05-26T22:52:53.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:53.508+0000 7f7d0e800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d10165e30 0x7f7d100b3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:53.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:53.508+0000 7f7d0e800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d10165e30 0x7f7d100b3d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39270/0 (socket says 172.21.15.131:39270) 2024-05-26T22:52:53.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:53.508+0000 7f7d0e800640 1 -- 172.21.15.131:0/2480692747 learned_addr learned my addr 172.21.15.131:0/2480692747 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:52:53.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:53.508+0000 7f7d0de00640 1 --2- 172.21.15.131:0/2480692747 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7d100af230 0x7f7d101fc8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:53.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:53.508+0000 7f7d05400640 1 --2- 172.21.15.131:0/2480692747 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d100d0560 0x7f7d1014f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:53.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:53.508+0000 7f7d0e800640 1 -- 172.21.15.131:0/2480692747 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7d100af230 msgr2=0x7f7d101fc8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:53.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:53.508+0000 7f7d0e800640 1 --2- 172.21.15.131:0/2480692747 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7d100af230 0x7f7d101fc8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:53.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:53.508+0000 7f7d0e800640 1 -- 172.21.15.131:0/2480692747 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d100d0560 msgr2=0x7f7d1014f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:53.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:53.508+0000 7f7d0e800640 1 --2- 172.21.15.131:0/2480692747 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d100d0560 0x7f7d1014f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:53.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:53.508+0000 7f7d0e800640 1 -- 172.21.15.131:0/2480692747 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d1009c500 con 0x7f7d10165e30 2024-05-26T22:52:53.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:53.508+0000 7f7d05400640 1 --2- 172.21.15.131:0/2480692747 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d100d0560 0x7f7d1014f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:52:53.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:53.508+0000 7f7d0de00640 1 --2- 172.21.15.131:0/2480692747 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7d100af230 0x7f7d101fc8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-26T22:52:53.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:53.509+0000 7f7d0e800640 1 --2- 172.21.15.131:0/2480692747 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d10165e30 0x7f7d100b3d50 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f7d08007e70 tx=0x7f7d0800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:53.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:53.509+0000 7f7d06a00640 1 -- 172.21.15.131:0/2480692747 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d08014620 con 0x7f7d10165e30 2024-05-26T22:52:53.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:53.509+0000 7f7d06a00640 1 -- 172.21.15.131:0/2480692747 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d08019070 con 0x7f7d10165e30 2024-05-26T22:52:53.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:53.509+0000 7f7d06a00640 1 -- 172.21.15.131:0/2480692747 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d08014a20 con 0x7f7d10165e30 2024-05-26T22:52:53.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:53.509+0000 7f7d14a00640 1 -- 172.21.15.131:0/2480692747 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d10160fe0 con 0x7f7d10165e30 2024-05-26T22:52:53.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:53.509+0000 7f7d14a00640 1 -- 172.21.15.131:0/2480692747 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d1015ba10 con 0x7f7d10165e30 2024-05-26T22:52:53.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:53.510+0000 7f7d14a00640 1 -- 172.21.15.131:0/2480692747 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d10074e90 con 0x7f7d10165e30 2024-05-26T22:52:53.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:53.512+0000 7f7d06a00640 1 -- 172.21.15.131:0/2480692747 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7d08014bc0 con 0x7f7d10165e30 2024-05-26T22:52:53.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:53.512+0000 7f7d06a00640 1 --2- 172.21.15.131:0/2480692747 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f7cec06be60 0x7f7cec06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:53.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:53.513+0000 7f7d06a00640 1 -- 172.21.15.131:0/2480692747 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5609+0+0 (secure 0 0 0) 0x7f7d0808b780 con 0x7f7d10165e30 2024-05-26T22:52:53.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:53.513+0000 7f7d0de00640 1 --2- 172.21.15.131:0/2480692747 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f7cec06be60 0x7f7cec06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:53.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:53.514+0000 7f7d0de00640 1 --2- 172.21.15.131:0/2480692747 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f7cec06be60 0x7f7cec06e320 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f7d00006fd0 tx=0x7f7d00008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:53.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:53.518+0000 7f7d06a00640 1 -- 172.21.15.131:0/2480692747 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f7d0805a180 con 0x7f7d10165e30 2024-05-26T22:52:53.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:53.753+0000 7f7d14a00640 1 -- 172.21.15.131:0/2480692747 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f7d100695c0 con 0x7f7d10165e30 2024-05-26T22:52:54.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.303+0000 7f7d06a00640 1 -- 172.21.15.131:0/2480692747 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 10 and data pool 11 v60) ==== 174+0+0 (secure 0 0 0) 0x7f7d08059a80 con 0x7f7d10165e30 2024-05-26T22:52:54.305 INFO:teuthology.orchestra.run.smithi131.stderr:new fs with metadata pool 10 and data pool 11 2024-05-26T22:52:54.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.310+0000 7f7d14a00640 1 -- 172.21.15.131:0/2480692747 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f7cec06be60 msgr2=0x7f7cec06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:54.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.310+0000 7f7d14a00640 1 --2- 172.21.15.131:0/2480692747 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f7cec06be60 0x7f7cec06e320 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f7d00006fd0 tx=0x7f7d00008040 comp rx=0 tx=0).stop 2024-05-26T22:52:54.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.310+0000 7f7d14a00640 1 -- 172.21.15.131:0/2480692747 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d10165e30 msgr2=0x7f7d100b3d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:54.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.310+0000 7f7d14a00640 1 --2- 172.21.15.131:0/2480692747 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d10165e30 0x7f7d100b3d50 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f7d08007e70 tx=0x7f7d0800e4b0 comp rx=0 tx=0).stop 2024-05-26T22:52:54.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.311+0000 7f7d14a00640 1 -- 172.21.15.131:0/2480692747 shutdown_connections 2024-05-26T22:52:54.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.311+0000 7f7d14a00640 1 --2- 172.21.15.131:0/2480692747 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f7cec06be60 0x7f7cec06e320 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:54.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.311+0000 7f7d14a00640 1 --2- 172.21.15.131:0/2480692747 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d10165e30 0x7f7d100b3d50 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:54.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.311+0000 7f7d14a00640 1 --2- 172.21.15.131:0/2480692747 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d100d0560 0x7f7d1014f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:54.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.311+0000 7f7d14a00640 1 --2- 172.21.15.131:0/2480692747 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7d100af230 0x7f7d101fc8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:54.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.311+0000 7f7d14a00640 1 -- 172.21.15.131:0/2480692747 >> 172.21.15.131:0/2480692747 conn(0x7f7d1006bc90 msgr2=0x7f7d10164610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:52:54.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.312+0000 7f7d14a00640 1 -- 172.21.15.131:0/2480692747 shutdown_connections 2024-05-26T22:52:54.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.312+0000 7f7d14a00640 1 -- 172.21.15.131:0/2480692747 wait complete. 2024-05-26T22:52:54.343 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-26T22:52:54.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.511+0000 7f2dca800640 1 Processor -- start 2024-05-26T22:52:54.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.511+0000 7f2dca800640 1 -- start start 2024-05-26T22:52:54.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.513+0000 7f2dca800640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f2dc415e320 0x7f2dc415e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:54.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.513+0000 7f2dca800640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f2dc4071500 0x7f2dc40ce2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:54.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.513+0000 7f2dca800640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2dc415ec40 con 0x7f2dc4150f70 2024-05-26T22:52:54.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.513+0000 7f2dca800640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f2dc40cfbe0 con 0x7f2dc4071500 2024-05-26T22:52:54.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.513+0000 7f2dca800640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f2dc40ceb40 con 0x7f2dc415e320 2024-05-26T22:52:54.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.513+0000 7f2dc8a00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f2dc4071500 0x7f2dc40ce2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:54.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.513+0000 7f2dc8a00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f2dc4071500 0x7f2dc40ce2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48330/0 (socket says 172.21.15.131:48330) 2024-05-26T22:52:54.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.513+0000 7f2dc8a00640 1 -- 172.21.15.131:0/2623691860 learned_addr learned my addr 172.21.15.131:0/2623691860 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:52:54.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.514+0000 7f2dc9400640 1 --2- 172.21.15.131:0/2623691860 >> v2:172.21.15.187:3300/0 conn(0x7f2dc415e320 0x7f2dc415e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:54.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.514+0000 7f2dc8a00640 1 -- 172.21.15.131:0/2623691860 >> v2:172.21.15.187:3300/0 conn(0x7f2dc415e320 msgr2=0x7f2dc415e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:54.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.514+0000 7f2dc8a00640 1 --2- 172.21.15.131:0/2623691860 >> v2:172.21.15.187:3300/0 conn(0x7f2dc415e320 0x7f2dc415e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:54.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.514+0000 7f2dc8a00640 1 -- 172.21.15.131:0/2623691860 >> v1:172.21.15.131:6789/0 conn(0x7f2dc4150f70 legacy=0x7f2dc41513a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:54.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.514+0000 7f2dc8a00640 1 -- 172.21.15.131:0/2623691860 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dc40cee20 con 0x7f2dc4071500 2024-05-26T22:52:54.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.515+0000 7f2dc8a00640 1 --2- 172.21.15.131:0/2623691860 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2dc4071500 0x7f2dc40ce2f0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f2dc406efc0 tx=0x7f2db002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aea37bc7239b5cd9 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:54.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.515+0000 7f2dc3e00640 1 -- 172.21.15.131:0/2623691860 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2db0037070 con 0x7f2dc4071500 2024-05-26T22:52:54.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.515+0000 7f2dc3e00640 1 -- 172.21.15.131:0/2623691860 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2db0032440 con 0x7f2dc4071500 2024-05-26T22:52:54.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.515+0000 7f2dc3e00640 1 -- 172.21.15.131:0/2623691860 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2db0032810 con 0x7f2dc4071500 2024-05-26T22:52:54.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.515+0000 7f2dca800640 1 -- 172.21.15.131:0/2623691860 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2dc4071500 msgr2=0x7f2dc40ce2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:54.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.515+0000 7f2dca800640 1 --2- 172.21.15.131:0/2623691860 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2dc4071500 0x7f2dc40ce2f0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f2dc406efc0 tx=0x7f2db002a270 comp rx=0 tx=0).stop 2024-05-26T22:52:54.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.516+0000 7f2dca800640 1 -- 172.21.15.131:0/2623691860 shutdown_connections 2024-05-26T22:52:54.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.516+0000 7f2dca800640 1 --2- 172.21.15.131:0/2623691860 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2dc4071500 0x7f2dc40ce2f0 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:54.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.516+0000 7f2dca800640 1 --2- 172.21.15.131:0/2623691860 >> v2:172.21.15.187:3300/0 conn(0x7f2dc415e320 0x7f2dc415e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:54.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.516+0000 7f2dca800640 1 -- 172.21.15.131:0/2623691860 >> 172.21.15.131:0/2623691860 conn(0x7f2dc40cff40 msgr2=0x7f2dc40d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:52:54.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.516+0000 7f2dca800640 1 -- 172.21.15.131:0/2623691860 shutdown_connections 2024-05-26T22:52:54.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.516+0000 7f2dca800640 1 -- 172.21.15.131:0/2623691860 wait complete. 2024-05-26T22:52:54.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.517+0000 7f2dca800640 1 Processor -- start 2024-05-26T22:52:54.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.517+0000 7f2dca800640 1 -- start start 2024-05-26T22:52:54.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.518+0000 7f2dca800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2dc4071500 0x7f2dc413a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:54.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.518+0000 7f2dca800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2dc4150f70 0x7f2dc406dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:54.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.518+0000 7f2dca800640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2dc415e320 0x7f2dc406e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:54.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.518+0000 7f2dca800640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f2dc40cf6c0 con 0x7f2dc4071500 2024-05-26T22:52:54.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.518+0000 7f2dca800640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2dc40cf3e0 con 0x7f2dc415e320 2024-05-26T22:52:54.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.518+0000 7f2dca800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f2dc40ce830 con 0x7f2dc4150f70 2024-05-26T22:52:54.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.519+0000 7f2dc9400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2dc4071500 0x7f2dc413a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:54.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.519+0000 7f2dc9400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2dc4071500 0x7f2dc413a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48342/0 (socket says 172.21.15.131:48342) 2024-05-26T22:52:54.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.519+0000 7f2dc9400640 1 -- 172.21.15.131:0/4185381309 learned_addr learned my addr 172.21.15.131:0/4185381309 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:52:54.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.519+0000 7f2dc9400640 1 -- 172.21.15.131:0/4185381309 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2dc4150f70 msgr2=0x7f2dc406dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:54.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.519+0000 7f2dc8a00640 1 --2- 172.21.15.131:0/4185381309 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2dc4150f70 0x7f2dc406dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:54.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.519+0000 7f2dc9400640 1 --2- 172.21.15.131:0/4185381309 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2dc4150f70 0x7f2dc406dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:54.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.519+0000 7f2dc9400640 1 -- 172.21.15.131:0/4185381309 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2dc415e320 msgr2=0x7f2dc406e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:54.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.519+0000 7f2dc9e00640 1 --2- 172.21.15.131:0/4185381309 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2dc415e320 0x7f2dc406e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:54.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.519+0000 7f2dc9400640 1 --2- 172.21.15.131:0/4185381309 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2dc415e320 0x7f2dc406e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:54.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.519+0000 7f2dc9400640 1 -- 172.21.15.131:0/4185381309 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dc41521f0 con 0x7f2dc4071500 2024-05-26T22:52:54.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.519+0000 7f2dc9400640 1 --2- 172.21.15.131:0/4185381309 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2dc4071500 0x7f2dc413a4b0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f2db800c7f0 tx=0x7f2db800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:54.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.520+0000 7f2dc1600640 1 -- 172.21.15.131:0/4185381309 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2db8015070 con 0x7f2dc4071500 2024-05-26T22:52:54.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.520+0000 7f2dc1600640 1 -- 172.21.15.131:0/4185381309 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2db8004220 con 0x7f2dc4071500 2024-05-26T22:52:54.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.520+0000 7f2dc1600640 1 -- 172.21.15.131:0/4185381309 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2db8007620 con 0x7f2dc4071500 2024-05-26T22:52:54.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.520+0000 7f2dca800640 1 -- 172.21.15.131:0/4185381309 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2dc4153650 con 0x7f2dc4071500 2024-05-26T22:52:54.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.520+0000 7f2dca800640 1 -- 172.21.15.131:0/4185381309 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2dc4153bd0 con 0x7f2dc4071500 2024-05-26T22:52:54.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.521+0000 7f2dca800640 1 -- 172.21.15.131:0/4185381309 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2dc40694e0 con 0x7f2dc4071500 2024-05-26T22:52:54.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.522+0000 7f2dc1600640 1 -- 172.21.15.131:0/4185381309 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2db80079d0 con 0x7f2dc4071500 2024-05-26T22:52:54.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.523+0000 7f2dc1600640 1 --2- 172.21.15.131:0/4185381309 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f2da406be30 0x7f2da406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:54.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.523+0000 7f2dc1600640 1 -- 172.21.15.131:0/4185381309 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5709+0+0 (secure 0 0 0) 0x7f2db808b1a0 con 0x7f2dc4071500 2024-05-26T22:52:54.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.523+0000 7f2dc8a00640 1 --2- 172.21.15.131:0/4185381309 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f2da406be30 0x7f2da406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:54.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.524+0000 7f2dc8a00640 1 --2- 172.21.15.131:0/4185381309 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f2da406be30 0x7f2da406e2f0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f2db0034000 tx=0x7f2db002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:54.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.529+0000 7f2dc1600640 1 -- 172.21.15.131:0/4185381309 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f2dc41521f0 con 0x7f2dc4071500 2024-05-26T22:52:54.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.727+0000 7f2dca800640 1 -- 172.21.15.131:0/4185381309 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2dc4149690 con 0x7f2dc4071500 2024-05-26T22:52:54.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.730+0000 7f2dc1600640 1 -- 172.21.15.131:0/4185381309 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v68) ==== 74+0+16705 (secure 0 0 0) 0x7f2db8059be0 con 0x7f2dc4071500 2024-05-26T22:52:54.730 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:52:54.730 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":68,"fsid":"46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f","created":"2024-05-26T22:47:57.288905+0000","modified":"2024-05-26T22:52:53.755035+0000","last_up_change":"2024-05-26T22:48:09.602207+0000","last_in_change":"2024-05-26T22:48:00.687341+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-26T22:48:10.463339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-05-26T22:52:51.536251+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-05-26T22:52:52.768524+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"814eaf62-a18d-494b-8e27-fd8fb88adea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6801","nonce":2350940051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6803","nonce":2350940051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6807","nonce":2350940051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6805","nonce":2350940051}]},"public_addr":"172.21.15.131:6801/2350940051","cluster_addr":"172.21.15.131:6803/2350940051","heartbeat_back_addr":"172.21.15.131:6807/2350940051","heartbeat_front_addr":"172.21.15.131:6805/2350940051","state":["exists","up"]},{"osd":1,"uuid":"42e66287-f6c6-4e26-af3c-6dfaa3536d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6825","nonce":1525528247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6827","nonce":1525528247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6831","nonce":1525528247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6829","nonce":1525528247}]},"public_addr":"172.21.15.131:6825/1525528247","cluster_addr":"172.21.15.131:6827/1525528247","heartbeat_back_addr":"172.21.15.131:6831/1525528247","heartbeat_front_addr":"172.21.15.131:6829/1525528247","state":["exists","up"]},{"osd":2,"uuid":"0da34673-04a2-4fd0-b57a-0ff2b679b043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6817","nonce":3627842210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6819","nonce":3627842210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6823","nonce":3627842210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6821","nonce":3627842210}]},"public_addr":"172.21.15.131:6817/3627842210","cluster_addr":"172.21.15.131:6819/3627842210","heartbeat_back_addr":"172.21.15.131:6823/3627842210","heartbeat_front_addr":"172.21.15.131:6821/3627842210","state":["exists","up"]},{"osd":3,"uuid":"cf117a27-d6b2-43d3-ae52-b47988bf8eed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6809","nonce":350790591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6811","nonce":350790591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6815","nonce":350790591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6813","nonce":350790591}]},"public_addr":"172.21.15.131:6809/350790591","cluster_addr":"172.21.15.131:6811/350790591","heartbeat_back_addr":"172.21.15.131:6815/350790591","heartbeat_front_addr":"172.21.15.131:6813/350790591","state":["exists","up"]},{"osd":4,"uuid":"a092ccfc-be89-4a84-88d5-0f781fa715eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6816","nonce":244560995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6819","nonce":244560995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6823","nonce":244560995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6821","nonce":244560995}]},"public_addr":"172.21.15.187:6816/244560995","cluster_addr":"172.21.15.187:6819/244560995","heartbeat_back_addr":"172.21.15.187:6823/244560995","heartbeat_front_addr":"172.21.15.187:6821/244560995","state":["exists","up"]},{"osd":5,"uuid":"7513ba69-dd00-4a35-b190-9e8cc3b2bdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6801","nonce":324619398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6803","nonce":324619398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6812","nonce":324619398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6807","nonce":324619398}]},"public_addr":"172.21.15.187:6801/324619398","cluster_addr":"172.21.15.187:6803/324619398","heartbeat_back_addr":"172.21.15.187:6812/324619398","heartbeat_front_addr":"172.21.15.187:6807/324619398","state":["exists","up"]},{"osd":6,"uuid":"50081810-c093-4d2f-a4d7-b7c695c5400d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6806","nonce":1391347786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6809","nonce":1391347786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6818","nonce":1391347786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6813","nonce":1391347786}]},"public_addr":"172.21.15.187:6806/1391347786","cluster_addr":"172.21.15.187:6809/1391347786","heartbeat_back_addr":"172.21.15.187:6818/1391347786","heartbeat_front_addr":"172.21.15.187:6813/1391347786","state":["exists","up"]},{"osd":7,"uuid":"775dae05-54f5-4ad4-bbce-92863c6ed2ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6825","nonce":1220065622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6827","nonce":1220065622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6831","nonce":1220065622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6829","nonce":1220065622}]},"public_addr":"172.21.15.187:6825/1220065622","cluster_addr":"172.21.15.187:6827/1220065622","heartbeat_back_addr":"172.21.15.187:6831/1220065622","heartbeat_front_addr":"172.21.15.187:6829/1220065622","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:05.983520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.299516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.067323+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.307748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.018742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.258775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.010912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.295355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-26T22:52:54.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.733+0000 7f2dca800640 1 -- 172.21.15.131:0/4185381309 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f2da406be30 msgr2=0x7f2da406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:54.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.733+0000 7f2dca800640 1 --2- 172.21.15.131:0/4185381309 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f2da406be30 0x7f2da406e2f0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f2db0034000 tx=0x7f2db002e000 comp rx=0 tx=0).stop 2024-05-26T22:52:54.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.733+0000 7f2dca800640 1 -- 172.21.15.131:0/4185381309 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2dc4071500 msgr2=0x7f2dc413a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:54.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.733+0000 7f2dca800640 1 --2- 172.21.15.131:0/4185381309 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2dc4071500 0x7f2dc413a4b0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f2db800c7f0 tx=0x7f2db800ccb0 comp rx=0 tx=0).stop 2024-05-26T22:52:54.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.733+0000 7f2dca800640 1 -- 172.21.15.131:0/4185381309 shutdown_connections 2024-05-26T22:52:54.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.733+0000 7f2dca800640 1 --2- 172.21.15.131:0/4185381309 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2dc4150f70 0x7f2dc406dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:54.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.733+0000 7f2dca800640 1 --2- 172.21.15.131:0/4185381309 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f2da406be30 0x7f2da406e2f0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:54.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.733+0000 7f2dca800640 1 --2- 172.21.15.131:0/4185381309 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2dc415e320 0x7f2dc406e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:54.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.733+0000 7f2dca800640 1 --2- 172.21.15.131:0/4185381309 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2dc4071500 0x7f2dc413a4b0 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:54.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.733+0000 7f2dca800640 1 -- 172.21.15.131:0/4185381309 >> 172.21.15.131:0/4185381309 conn(0x7f2dc40cff40 msgr2=0x7f2dc40b02a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:52:54.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.733+0000 7f2dca800640 1 -- 172.21.15.131:0/4185381309 shutdown_connections 2024-05-26T22:52:54.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.733+0000 7f2dca800640 1 -- 172.21.15.131:0/4185381309 wait complete. 2024-05-26T22:52:54.752 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-26T22:52:54.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.948+0000 7f688ee00640 1 Processor -- start 2024-05-26T22:52:54.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.948+0000 7f688ee00640 1 -- start start 2024-05-26T22:52:54.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.950+0000 7f688ee00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f688806ebc0 0x7f68880cedf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:54.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.950+0000 7f688ee00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f6888071b70 0x7f68880b3d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:54.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.950+0000 7f688ee00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68880722a0 con 0x7f6888144340 2024-05-26T22:52:54.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.950+0000 7f688ee00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f6888072eb0 con 0x7f688806ebc0 2024-05-26T22:52:54.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.950+0000 7f688ee00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f6888072580 con 0x7f6888071b70 2024-05-26T22:52:54.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.950+0000 7f688da00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f6888144340 0x7f688806e4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:42234/0 (socket says 172.21.15.131:42234) 2024-05-26T22:52:54.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.950+0000 7f688da00640 1 -- 172.21.15.131:0/1019308446 learned_addr learned my addr 172.21.15.131:0/1019308446 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:52:54.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.950+0000 7f688d000640 1 --2- 172.21.15.131:0/1019308446 >> v2:172.21.15.131:3300/0 conn(0x7f688806ebc0 0x7f68880cedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:54.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.950+0000 7f688e400640 1 --2- 172.21.15.131:0/1019308446 >> v2:172.21.15.187:3300/0 conn(0x7f6888071b70 0x7f68880b3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:54.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.950+0000 7f6887e00640 1 -- 172.21.15.131:0/1019308446 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2297346914 0 0) 0x7f68880722a0 con 0x7f6888144340 2024-05-26T22:52:54.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.951+0000 7f6887e00640 1 -- 172.21.15.131:0/1019308446 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f686c003520 con 0x7f6888144340 2024-05-26T22:52:54.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.951+0000 7f688d000640 1 -- 172.21.15.131:0/1019308446 >> v2:172.21.15.187:3300/0 conn(0x7f6888071b70 msgr2=0x7f68880b3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:54.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.951+0000 7f688d000640 1 --2- 172.21.15.131:0/1019308446 >> v2:172.21.15.187:3300/0 conn(0x7f6888071b70 0x7f68880b3d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:54.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.951+0000 7f688d000640 1 -- 172.21.15.131:0/1019308446 >> v1:172.21.15.131:6789/0 conn(0x7f6888144340 legacy=0x7f688806e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:54.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.951+0000 7f688d000640 1 -- 172.21.15.131:0/1019308446 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6888072860 con 0x7f688806ebc0 2024-05-26T22:52:54.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.952+0000 7f688d000640 1 --2- 172.21.15.131:0/1019308446 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f688806ebc0 0x7f68880cedf0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f68880afa30 tx=0x7f687402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f986b76610ce0792 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:54.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.952+0000 7f6887e00640 1 -- 172.21.15.131:0/1019308446 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6874037070 con 0x7f688806ebc0 2024-05-26T22:52:54.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.952+0000 7f6887e00640 1 -- 172.21.15.131:0/1019308446 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6874032440 con 0x7f688806ebc0 2024-05-26T22:52:54.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.952+0000 7f6887e00640 1 -- 172.21.15.131:0/1019308446 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6874032810 con 0x7f688806ebc0 2024-05-26T22:52:54.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.952+0000 7f688ee00640 1 -- 172.21.15.131:0/1019308446 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f688806ebc0 msgr2=0x7f68880cedf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:54.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.952+0000 7f688ee00640 1 --2- 172.21.15.131:0/1019308446 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f688806ebc0 0x7f68880cedf0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f68880afa30 tx=0x7f687402a1e0 comp rx=0 tx=0).stop 2024-05-26T22:52:54.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.952+0000 7f688ee00640 1 -- 172.21.15.131:0/1019308446 shutdown_connections 2024-05-26T22:52:54.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.952+0000 7f688ee00640 1 --2- 172.21.15.131:0/1019308446 >> v2:172.21.15.187:3300/0 conn(0x7f6888071b70 0x7f68880b3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:54.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.952+0000 7f688ee00640 1 --2- 172.21.15.131:0/1019308446 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f688806ebc0 0x7f68880cedf0 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:54.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.952+0000 7f688ee00640 1 -- 172.21.15.131:0/1019308446 >> 172.21.15.131:0/1019308446 conn(0x7f68880cff40 msgr2=0x7f68880d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:52:54.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.953+0000 7f688ee00640 1 -- 172.21.15.131:0/1019308446 shutdown_connections 2024-05-26T22:52:54.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.953+0000 7f688ee00640 1 -- 172.21.15.131:0/1019308446 wait complete. 2024-05-26T22:52:54.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.954+0000 7f688ee00640 1 Processor -- start 2024-05-26T22:52:54.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.954+0000 7f688ee00640 1 -- start start 2024-05-26T22:52:54.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.955+0000 7f688ee00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f688806ebc0 0x7f6888158250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:54.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.955+0000 7f688ee00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6888071b70 0x7f6888142d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:54.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.955+0000 7f688ee00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6888144340 0x7f6888143240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:54.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.955+0000 7f688ee00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f68880b4260 con 0x7f6888144340 2024-05-26T22:52:54.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.955+0000 7f688ee00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6888071f80 con 0x7f688806ebc0 2024-05-26T22:52:54.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.955+0000 7f688ee00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f68880b43e0 con 0x7f6888071b70 2024-05-26T22:52:54.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.955+0000 7f688e400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6888144340 0x7f6888143240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:54.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.956+0000 7f688e400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6888144340 0x7f6888143240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48368/0 (socket says 172.21.15.131:48368) 2024-05-26T22:52:54.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.956+0000 7f688e400640 1 -- 172.21.15.131:0/1002255045 learned_addr learned my addr 172.21.15.131:0/1002255045 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:52:54.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.956+0000 7f688da00640 1 --2- 172.21.15.131:0/1002255045 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f688806ebc0 0x7f6888158250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:54.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.956+0000 7f688d000640 1 --2- 172.21.15.131:0/1002255045 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6888071b70 0x7f6888142d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:54.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.956+0000 7f688e400640 1 -- 172.21.15.131:0/1002255045 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6888071b70 msgr2=0x7f6888142d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:54.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.956+0000 7f688e400640 1 --2- 172.21.15.131:0/1002255045 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6888071b70 0x7f6888142d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:54.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.956+0000 7f688e400640 1 -- 172.21.15.131:0/1002255045 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f688806ebc0 msgr2=0x7f6888158250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:54.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.956+0000 7f688e400640 1 --2- 172.21.15.131:0/1002255045 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f688806ebc0 0x7f6888158250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:54.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.956+0000 7f688e400640 1 -- 172.21.15.131:0/1002255045 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68881366c0 con 0x7f6888144340 2024-05-26T22:52:54.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.956+0000 7f688d000640 1 --2- 172.21.15.131:0/1002255045 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6888071b70 0x7f6888142d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:52:54.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.956+0000 7f688e400640 1 --2- 172.21.15.131:0/1002255045 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6888144340 0x7f6888143240 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f688000d830 tx=0x7f688000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:54.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.956+0000 7f6885600640 1 -- 172.21.15.131:0/1002255045 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f688000b6c0 con 0x7f6888144340 2024-05-26T22:52:54.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.957+0000 7f6885600640 1 -- 172.21.15.131:0/1002255045 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6880019070 con 0x7f6888144340 2024-05-26T22:52:54.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.957+0000 7f688ee00640 1 -- 172.21.15.131:0/1002255045 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68881378f0 con 0x7f6888144340 2024-05-26T22:52:54.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.957+0000 7f6885600640 1 -- 172.21.15.131:0/1002255045 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f688000bac0 con 0x7f6888144340 2024-05-26T22:52:54.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.957+0000 7f688ee00640 1 -- 172.21.15.131:0/1002255045 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6888137eb0 con 0x7f6888144340 2024-05-26T22:52:54.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.958+0000 7f688ee00640 1 -- 172.21.15.131:0/1002255045 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68880694e0 con 0x7f6888144340 2024-05-26T22:52:54.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.959+0000 7f6885600640 1 -- 172.21.15.131:0/1002255045 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f688001e460 con 0x7f6888144340 2024-05-26T22:52:54.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.960+0000 7f6885600640 1 --2- 172.21.15.131:0/1002255045 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f686806be30 0x7f686806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:54.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.960+0000 7f6885600640 1 -- 172.21.15.131:0/1002255045 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5709+0+0 (secure 0 0 0) 0x7f688008b5a0 con 0x7f6888144340 2024-05-26T22:52:54.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.960+0000 7f688da00640 1 --2- 172.21.15.131:0/1002255045 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f686806be30 0x7f686806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:54.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.961+0000 7f688da00640 1 --2- 172.21.15.131:0/1002255045 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f686806be30 0x7f686806e2f0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f6878001580 tx=0x7f6878009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:54.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:54.964+0000 7f6885600640 1 -- 172.21.15.131:0/1002255045 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f6880059eb0 con 0x7f6888144340 2024-05-26T22:52:55.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:55.162+0000 7f688ee00640 1 -- 172.21.15.131:0/1002255045 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6888139f60 con 0x7f6888144340 2024-05-26T22:52:55.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:55.163+0000 7f6885600640 1 -- 172.21.15.131:0/1002255045 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v68) ==== 74+0+16705 (secure 0 0 0) 0x7f68800597b0 con 0x7f6888144340 2024-05-26T22:52:55.163 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:52:55.164 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":68,"fsid":"46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f","created":"2024-05-26T22:47:57.288905+0000","modified":"2024-05-26T22:52:53.755035+0000","last_up_change":"2024-05-26T22:48:09.602207+0000","last_in_change":"2024-05-26T22:48:00.687341+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-26T22:48:10.463339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-05-26T22:52:51.536251+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-05-26T22:52:52.768524+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"814eaf62-a18d-494b-8e27-fd8fb88adea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6801","nonce":2350940051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6803","nonce":2350940051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6807","nonce":2350940051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6805","nonce":2350940051}]},"public_addr":"172.21.15.131:6801/2350940051","cluster_addr":"172.21.15.131:6803/2350940051","heartbeat_back_addr":"172.21.15.131:6807/2350940051","heartbeat_front_addr":"172.21.15.131:6805/2350940051","state":["exists","up"]},{"osd":1,"uuid":"42e66287-f6c6-4e26-af3c-6dfaa3536d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6825","nonce":1525528247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6827","nonce":1525528247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6831","nonce":1525528247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6829","nonce":1525528247}]},"public_addr":"172.21.15.131:6825/1525528247","cluster_addr":"172.21.15.131:6827/1525528247","heartbeat_back_addr":"172.21.15.131:6831/1525528247","heartbeat_front_addr":"172.21.15.131:6829/1525528247","state":["exists","up"]},{"osd":2,"uuid":"0da34673-04a2-4fd0-b57a-0ff2b679b043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6817","nonce":3627842210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6819","nonce":3627842210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6823","nonce":3627842210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6821","nonce":3627842210}]},"public_addr":"172.21.15.131:6817/3627842210","cluster_addr":"172.21.15.131:6819/3627842210","heartbeat_back_addr":"172.21.15.131:6823/3627842210","heartbeat_front_addr":"172.21.15.131:6821/3627842210","state":["exists","up"]},{"osd":3,"uuid":"cf117a27-d6b2-43d3-ae52-b47988bf8eed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6809","nonce":350790591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6811","nonce":350790591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6815","nonce":350790591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6813","nonce":350790591}]},"public_addr":"172.21.15.131:6809/350790591","cluster_addr":"172.21.15.131:6811/350790591","heartbeat_back_addr":"172.21.15.131:6815/350790591","heartbeat_front_addr":"172.21.15.131:6813/350790591","state":["exists","up"]},{"osd":4,"uuid":"a092ccfc-be89-4a84-88d5-0f781fa715eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6816","nonce":244560995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6819","nonce":244560995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6823","nonce":244560995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6821","nonce":244560995}]},"public_addr":"172.21.15.187:6816/244560995","cluster_addr":"172.21.15.187:6819/244560995","heartbeat_back_addr":"172.21.15.187:6823/244560995","heartbeat_front_addr":"172.21.15.187:6821/244560995","state":["exists","up"]},{"osd":5,"uuid":"7513ba69-dd00-4a35-b190-9e8cc3b2bdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6801","nonce":324619398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6803","nonce":324619398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6812","nonce":324619398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6807","nonce":324619398}]},"public_addr":"172.21.15.187:6801/324619398","cluster_addr":"172.21.15.187:6803/324619398","heartbeat_back_addr":"172.21.15.187:6812/324619398","heartbeat_front_addr":"172.21.15.187:6807/324619398","state":["exists","up"]},{"osd":6,"uuid":"50081810-c093-4d2f-a4d7-b7c695c5400d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6806","nonce":1391347786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6809","nonce":1391347786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6818","nonce":1391347786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6813","nonce":1391347786}]},"public_addr":"172.21.15.187:6806/1391347786","cluster_addr":"172.21.15.187:6809/1391347786","heartbeat_back_addr":"172.21.15.187:6818/1391347786","heartbeat_front_addr":"172.21.15.187:6813/1391347786","state":["exists","up"]},{"osd":7,"uuid":"775dae05-54f5-4ad4-bbce-92863c6ed2ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6825","nonce":1220065622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6827","nonce":1220065622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6831","nonce":1220065622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6829","nonce":1220065622}]},"public_addr":"172.21.15.187:6825/1220065622","cluster_addr":"172.21.15.187:6827/1220065622","heartbeat_back_addr":"172.21.15.187:6831/1220065622","heartbeat_front_addr":"172.21.15.187:6829/1220065622","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:05.983520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.299516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.067323+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.307748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.018742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.258775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.010912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.295355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-26T22:52:55.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:55.166+0000 7f688ee00640 1 -- 172.21.15.131:0/1002255045 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f686806be30 msgr2=0x7f686806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:55.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:55.166+0000 7f688ee00640 1 --2- 172.21.15.131:0/1002255045 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f686806be30 0x7f686806e2f0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f6878001580 tx=0x7f6878009040 comp rx=0 tx=0).stop 2024-05-26T22:52:55.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:55.166+0000 7f688ee00640 1 -- 172.21.15.131:0/1002255045 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6888144340 msgr2=0x7f6888143240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:55.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:55.166+0000 7f688ee00640 1 --2- 172.21.15.131:0/1002255045 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6888144340 0x7f6888143240 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f688000d830 tx=0x7f688000dcf0 comp rx=0 tx=0).stop 2024-05-26T22:52:55.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:55.166+0000 7f688ee00640 1 -- 172.21.15.131:0/1002255045 shutdown_connections 2024-05-26T22:52:55.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:55.166+0000 7f688ee00640 1 --2- 172.21.15.131:0/1002255045 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6888071b70 0x7f6888142d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:55.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:55.166+0000 7f688ee00640 1 --2- 172.21.15.131:0/1002255045 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f686806be30 0x7f686806e2f0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:55.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:55.166+0000 7f688ee00640 1 --2- 172.21.15.131:0/1002255045 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6888144340 0x7f6888143240 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:55.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:55.166+0000 7f688ee00640 1 --2- 172.21.15.131:0/1002255045 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f688806ebc0 0x7f6888158250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:55.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:55.166+0000 7f688ee00640 1 -- 172.21.15.131:0/1002255045 >> 172.21.15.131:0/1002255045 conn(0x7f68880cff40 msgr2=0x7f688813d750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:52:55.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:55.166+0000 7f688ee00640 1 -- 172.21.15.131:0/1002255045 shutdown_connections 2024-05-26T22:52:55.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:55.166+0000 7f688ee00640 1 -- 172.21.15.131:0/1002255045 wait complete. 2024-05-26T22:52:55.186 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-26T22:52:55.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:55.392+0000 7feaee200640 1 Processor -- start 2024-05-26T22:52:55.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:55.392+0000 7feaee200640 1 -- start start 2024-05-26T22:52:55.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:55.393+0000 7feaee200640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7feae806dd30 0x7feae806e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:55.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:55.393+0000 7feaee200640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feae815c1c0 con 0x7feae8155770 2024-05-26T22:52:55.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:55.394+0000 7feaee200640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feae815c4a0 con 0x7feae80725a0 2024-05-26T22:52:55.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:55.394+0000 7feaee200640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7feae8071da0 con 0x7feae806dd30 2024-05-26T22:52:55.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:55.394+0000 7feae7400640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7feae8155770 0x7feae815b700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:42244/0 (socket says 172.21.15.131:42244) 2024-05-26T22:52:55.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:55.394+0000 7feae7400640 1 -- 172.21.15.131:0/195305218 learned_addr learned my addr 172.21.15.131:0/195305218 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:52:55.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:55.394+0000 7feae6000640 1 -- 172.21.15.131:0/195305218 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1667160609 0 0) 0x7feae815c1c0 con 0x7feae8155770 2024-05-26T22:52:55.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:55.394+0000 7feae6000640 1 -- 172.21.15.131:0/195305218 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feacc003520 con 0x7feae8155770 2024-05-26T22:52:55.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:55.394+0000 7feae6a00640 1 --2- 172.21.15.131:0/195305218 >> v2:172.21.15.187:3300/0 conn(0x7feae806dd30 0x7feae806e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:55.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:55.395+0000 7feae6000640 1 -- 172.21.15.131:0/195305218 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4023197853 0 0) 0x7feacc003520 con 0x7feae8155770 2024-05-26T22:52:55.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:55.395+0000 7feae6000640 1 -- 172.21.15.131:0/195305218 >> v2:172.21.15.187:3300/0 conn(0x7feae806dd30 msgr2=0x7feae806e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:55.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:55.395+0000 7feae6000640 1 --2- 172.21.15.131:0/195305218 >> v2:172.21.15.187:3300/0 conn(0x7feae806dd30 0x7feae806e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:55.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:55.395+0000 7feae6000640 1 -- 172.21.15.131:0/195305218 >> v1:172.21.15.187:6789/0 conn(0x7feae80725a0 legacy=0x7feae80729a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:55.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:55.395+0000 7feae6000640 1 -- 172.21.15.131:0/195305218 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feae8072080 con 0x7feae8155770 2024-05-26T22:52:55.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:55.395+0000 7feae6000640 1 -- 172.21.15.131:0/195305218 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7fead8001670 con 0x7feae8155770 2024-05-26T22:52:55.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:55.395+0000 7feae6000640 1 -- 172.21.15.131:0/195305218 >> v1:172.21.15.131:6789/0 conn(0x7feae8155770 legacy=0x7feae815b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:55.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:55.395+0000 7feae6000640 1 --2- 172.21.15.131:0/195305218 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feacc0039b0 0x7feacc003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:55.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:55.395+0000 7feae6000640 1 --2- 172.21.15.131:0/195305218 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feacc004800 0x7feacc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:55.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:55.395+0000 7feae6000640 1 --2- 172.21.15.131:0/195305218 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feacc0051c0 0x7feacc00dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:55.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:55.395+0000 7feae6000640 1 -- 172.21.15.131:0/195305218 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7feae8071da0 con 0x7feacc0051c0 2024-05-26T22:52:55.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:55.395+0000 7feae6000640 1 -- 172.21.15.131:0/195305218 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7feacc00e2e0 con 0x7feacc0039b0 2024-05-26T22:52:55.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:55.395+0000 7feae6000640 1 -- 172.21.15.131:0/195305218 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7feacc00e4e0 con 0x7feacc004800 2024-05-26T22:52:55.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:55.395+0000 7feae7e00640 1 --2- 172.21.15.131:0/195305218 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feacc0051c0 0x7feacc00dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:55.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:55.395+0000 7feae7400640 1 --2- 172.21.15.131:0/195305218 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feacc0039b0 0x7feacc003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:55.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:55.395+0000 7feae6a00640 1 --2- 172.21.15.131:0/195305218 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feacc004800 0x7feacc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:55.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:55.395+0000 7feae7e00640 1 -- 172.21.15.131:0/195305218 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feacc004800 msgr2=0x7feacc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:55.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:55.395+0000 7feae7e00640 1 --2- 172.21.15.131:0/195305218 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feacc004800 0x7feacc004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:55.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:55.395+0000 7feae7e00640 1 -- 172.21.15.131:0/195305218 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feacc0039b0 msgr2=0x7feacc003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:55.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:55.395+0000 7feae7e00640 1 --2- 172.21.15.131:0/195305218 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feacc0039b0 0x7feacc003db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:55.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:55.395+0000 7feae7e00640 1 -- 172.21.15.131:0/195305218 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feacc003520 con 0x7feacc0051c0 2024-05-26T22:52:55.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:55.396+0000 7feae7e00640 1 -- 172.21.15.131:0/195305218 reap_dead start 2024-05-26T22:52:55.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:55.396+0000 7feae7e00640 1 --2- 172.21.15.131:0/195305218 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feacc0051c0 0x7feacc00dbd0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7feae00044b0 tx=0x7feacc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4d4c45565fbec04 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:55.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:55.397+0000 7feae6000640 1 -- 172.21.15.131:0/195305218 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feae0004080 con 0x7feacc0051c0 2024-05-26T22:52:55.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:55.397+0000 7feae6000640 1 -- 172.21.15.131:0/195305218 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feae002ada0 con 0x7feacc0051c0 2024-05-26T22:52:55.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:55.397+0000 7feae6000640 1 -- 172.21.15.131:0/195305218 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feae0006d90 con 0x7feacc0051c0 2024-05-26T22:52:55.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:55.397+0000 7feaee200640 1 -- 172.21.15.131:0/195305218 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feacc0051c0 msgr2=0x7feacc00dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:55.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:55.397+0000 7feaee200640 1 --2- 172.21.15.131:0/195305218 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feacc0051c0 0x7feacc00dbd0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7feae00044b0 tx=0x7feacc005830 comp rx=0 tx=0).stop 2024-05-26T22:52:55.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:55.397+0000 7feaee200640 1 -- 172.21.15.131:0/195305218 shutdown_connections 2024-05-26T22:52:55.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:55.397+0000 7feaee200640 1 --2- 172.21.15.131:0/195305218 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feacc0051c0 0x7feacc00dbd0 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:55.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:55.397+0000 7feaee200640 1 -- 172.21.15.131:0/195305218 >> 172.21.15.131:0/195305218 conn(0x7feae806bc90 msgr2=0x7feae80b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:52:55.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:55.397+0000 7feaee200640 1 -- 172.21.15.131:0/195305218 shutdown_connections 2024-05-26T22:52:55.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:55.397+0000 7feaee200640 1 -- 172.21.15.131:0/195305218 wait complete. 2024-05-26T22:52:55.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:55.399+0000 7feaee200640 1 Processor -- start 2024-05-26T22:52:55.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:55.399+0000 7feaee200640 1 -- start start 2024-05-26T22:52:55.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:55.400+0000 7feaee200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feacc0051c0 0x7feae8152cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:55.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:55.400+0000 7feaee200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feae81f7e10 0x7feae81f3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:55.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:55.400+0000 7feaee200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feae81f4310 0x7feae815a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:55.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:55.400+0000 7feaee200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7feae809de00 con 0x7feae81f4310 2024-05-26T22:52:55.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:55.400+0000 7feaee200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7feae815c8c0 con 0x7feacc0051c0 2024-05-26T22:52:55.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:55.400+0000 7feaee200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7feae809df80 con 0x7feae81f7e10 2024-05-26T22:52:55.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:55.400+0000 7feae7e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feae81f4310 0x7feae815a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:55.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:55.400+0000 7feae7e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feae81f4310 0x7feae815a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48386/0 (socket says 172.21.15.131:48386) 2024-05-26T22:52:55.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:55.401+0000 7feae7e00640 1 -- 172.21.15.131:0/3545788267 learned_addr learned my addr 172.21.15.131:0/3545788267 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:52:55.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:55.401+0000 7feae7400640 1 --2- 172.21.15.131:0/3545788267 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feacc0051c0 0x7feae8152cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:55.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:55.401+0000 7feae6a00640 1 --2- 172.21.15.131:0/3545788267 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feae81f7e10 0x7feae81f3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:55.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:55.401+0000 7feae7e00640 1 -- 172.21.15.131:0/3545788267 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feae81f7e10 msgr2=0x7feae81f3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:55.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:55.401+0000 7feae7e00640 1 --2- 172.21.15.131:0/3545788267 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feae81f7e10 0x7feae81f3dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:55.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:55.401+0000 7feae7e00640 1 -- 172.21.15.131:0/3545788267 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feacc0051c0 msgr2=0x7feae8152cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:55.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:55.401+0000 7feae7e00640 1 --2- 172.21.15.131:0/3545788267 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feacc0051c0 0x7feae8152cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:55.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:55.401+0000 7feae7e00640 1 -- 172.21.15.131:0/3545788267 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feae81f8290 con 0x7feae81f4310 2024-05-26T22:52:55.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:55.401+0000 7feae7e00640 1 --2- 172.21.15.131:0/3545788267 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feae81f4310 0x7feae815a1d0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7feae00045d0 tx=0x7feae0003a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:55.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:55.402+0000 7feadf400640 1 -- 172.21.15.131:0/3545788267 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feae0040030 con 0x7feae81f4310 2024-05-26T22:52:55.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:55.402+0000 7feadf400640 1 -- 172.21.15.131:0/3545788267 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feae00075e0 con 0x7feae81f4310 2024-05-26T22:52:55.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:55.402+0000 7feadf400640 1 -- 172.21.15.131:0/3545788267 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feae0007df0 con 0x7feae81f4310 2024-05-26T22:52:55.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:55.402+0000 7feaee200640 1 -- 172.21.15.131:0/3545788267 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feae81f4890 con 0x7feae81f4310 2024-05-26T22:52:55.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:55.402+0000 7feaee200640 1 -- 172.21.15.131:0/3545788267 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feae81f7610 con 0x7feae81f4310 2024-05-26T22:52:55.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:55.403+0000 7feaee200640 1 -- 172.21.15.131:0/3545788267 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feae8074e90 con 0x7feae81f4310 2024-05-26T22:52:55.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:55.404+0000 7feadf400640 1 -- 172.21.15.131:0/3545788267 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7feae0043070 con 0x7feae81f4310 2024-05-26T22:52:55.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:55.405+0000 7feadf400640 1 --2- 172.21.15.131:0/3545788267 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7feac806be60 0x7feac806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:55.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:55.405+0000 7feadf400640 1 -- 172.21.15.131:0/3545788267 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7feae00aac10 con 0x7feae81f4310 2024-05-26T22:52:55.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:55.405+0000 7feae7400640 1 --2- 172.21.15.131:0/3545788267 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7feac806be60 0x7feac806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:55.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:55.406+0000 7feae7400640 1 --2- 172.21.15.131:0/3545788267 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7feac806be60 0x7feac806e320 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fead8001050 tx=0x7fead8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:55.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:55.410+0000 7feadf400640 1 -- 172.21.15.131:0/3545788267 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7feae00795b0 con 0x7feae81f4310 2024-05-26T22:52:55.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:55.637+0000 7feaee200640 1 -- 172.21.15.131:0/3545788267 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7feae81e45d0 con 0x7feae81f4310 2024-05-26T22:52:56.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:56.317+0000 7feadf400640 1 -- 172.21.15.131:0/3545788267 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v63) ==== 118+0+0 (secure 0 0 0) 0x7feae0078eb0 con 0x7feae81f4310 2024-05-26T22:52:56.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:56.321+0000 7feaee200640 1 -- 172.21.15.131:0/3545788267 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7feac806be60 msgr2=0x7feac806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:56.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:56.321+0000 7feaee200640 1 --2- 172.21.15.131:0/3545788267 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7feac806be60 0x7feac806e320 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fead8001050 tx=0x7fead8009040 comp rx=0 tx=0).stop 2024-05-26T22:52:56.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:56.321+0000 7feaee200640 1 -- 172.21.15.131:0/3545788267 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feae81f4310 msgr2=0x7feae815a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:56.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:56.321+0000 7feaee200640 1 --2- 172.21.15.131:0/3545788267 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feae81f4310 0x7feae815a1d0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7feae00045d0 tx=0x7feae0003a70 comp rx=0 tx=0).stop 2024-05-26T22:52:56.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:56.322+0000 7feaee200640 1 -- 172.21.15.131:0/3545788267 shutdown_connections 2024-05-26T22:52:56.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:56.322+0000 7feaee200640 1 --2- 172.21.15.131:0/3545788267 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feae81f7e10 0x7feae81f3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:56.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:56.322+0000 7feaee200640 1 --2- 172.21.15.131:0/3545788267 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7feac806be60 0x7feac806e320 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:56.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:56.322+0000 7feaee200640 1 --2- 172.21.15.131:0/3545788267 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feae81f4310 0x7feae815a1d0 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:56.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:56.322+0000 7feaee200640 1 --2- 172.21.15.131:0/3545788267 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feacc0051c0 0x7feae8152cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:56.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:56.322+0000 7feaee200640 1 -- 172.21.15.131:0/3545788267 >> 172.21.15.131:0/3545788267 conn(0x7feae806bc90 msgr2=0x7feae80cc660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:52:56.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:56.322+0000 7feaee200640 1 -- 172.21.15.131:0/3545788267 shutdown_connections 2024-05-26T22:52:56.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:56.322+0000 7feaee200640 1 -- 172.21.15.131:0/3545788267 wait complete. 2024-05-26T22:52:56.345 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-26T22:52:56.346 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-26T22:52:56.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:56.553+0000 7f1890a00640 1 Processor -- start 2024-05-26T22:52:56.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:56.553+0000 7f1890a00640 1 -- start start 2024-05-26T22:52:56.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:56.553+0000 7f1890a00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f188c0b9f00 0x7f188c0b5c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:56.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:56.554+0000 7f1890a00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f188c0d2e80 con 0x7f188c0d3c40 2024-05-26T22:52:56.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:56.554+0000 7f1890a00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f188c0d3160 con 0x7f188c15e3e0 2024-05-26T22:52:56.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:56.554+0000 7f1890a00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f188c0cf910 con 0x7f188c0b9f00 2024-05-26T22:52:56.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:56.554+0000 7f188be00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f188c0d3c40 0x7f188c0b7260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:42252/0 (socket says 172.21.15.131:42252) 2024-05-26T22:52:56.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:56.554+0000 7f188be00640 1 -- 172.21.15.131:0/3622266134 learned_addr learned my addr 172.21.15.131:0/3622266134 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:52:56.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:56.554+0000 7f1882a00640 1 --2- 172.21.15.131:0/3622266134 >> v2:172.21.15.187:3300/0 conn(0x7f188c0b9f00 0x7f188c0b5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:56.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:56.554+0000 7f188aa00640 1 -- 172.21.15.131:0/3622266134 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3400601753 0 0) 0x7f188c0d2e80 con 0x7f188c0d3c40 2024-05-26T22:52:56.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:56.554+0000 7f188aa00640 1 -- 172.21.15.131:0/3622266134 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1870003520 con 0x7f188c0d3c40 2024-05-26T22:52:56.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:56.555+0000 7f188aa00640 1 -- 172.21.15.131:0/3622266134 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1510350906 0 0) 0x7f1870003520 con 0x7f188c0d3c40 2024-05-26T22:52:56.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:56.555+0000 7f188aa00640 1 -- 172.21.15.131:0/3622266134 >> v2:172.21.15.187:3300/0 conn(0x7f188c0b9f00 msgr2=0x7f188c0b5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:56.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:56.555+0000 7f188aa00640 1 --2- 172.21.15.131:0/3622266134 >> v2:172.21.15.187:3300/0 conn(0x7f188c0b9f00 0x7f188c0b5c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:56.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:56.555+0000 7f188aa00640 1 -- 172.21.15.131:0/3622266134 >> v1:172.21.15.187:6789/0 conn(0x7f188c15e3e0 legacy=0x7f188c0cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:56.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:56.555+0000 7f188aa00640 1 -- 172.21.15.131:0/3622266134 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f188c0d3440 con 0x7f188c0d3c40 2024-05-26T22:52:56.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:56.555+0000 7f188aa00640 1 -- 172.21.15.131:0/3622266134 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7f18840019f0 con 0x7f188c0d3c40 2024-05-26T22:52:56.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:56.555+0000 7f188aa00640 1 -- 172.21.15.131:0/3622266134 >> v1:172.21.15.131:6789/0 conn(0x7f188c0d3c40 legacy=0x7f188c0b7260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:56.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:56.555+0000 7f188aa00640 1 --2- 172.21.15.131:0/3622266134 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f18700039b0 0x7f1870003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:56.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:56.555+0000 7f188aa00640 1 --2- 172.21.15.131:0/3622266134 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1870004800 0x7f1870004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:56.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:56.555+0000 7f188aa00640 1 --2- 172.21.15.131:0/3622266134 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18700051c0 0x7f187000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:56.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:56.555+0000 7f188aa00640 1 -- 172.21.15.131:0/3622266134 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f187000c030 con 0x7f18700051c0 2024-05-26T22:52:56.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:56.555+0000 7f188aa00640 1 -- 172.21.15.131:0/3622266134 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f187000c1e0 con 0x7f18700039b0 2024-05-26T22:52:56.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:56.555+0000 7f188aa00640 1 -- 172.21.15.131:0/3622266134 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f187000c3e0 con 0x7f1870004800 2024-05-26T22:52:56.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:56.555+0000 7f188be00640 1 --2- 172.21.15.131:0/3622266134 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18700051c0 0x7f187000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:56.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:56.555+0000 7f188b400640 1 --2- 172.21.15.131:0/3622266134 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f18700039b0 0x7f1870003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:56.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:56.556+0000 7f1882a00640 1 --2- 172.21.15.131:0/3622266134 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1870004800 0x7f1870004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:56.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:56.556+0000 7f188be00640 1 -- 172.21.15.131:0/3622266134 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1870004800 msgr2=0x7f1870004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:56.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:56.556+0000 7f188be00640 1 --2- 172.21.15.131:0/3622266134 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1870004800 0x7f1870004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:56.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:56.556+0000 7f188be00640 1 -- 172.21.15.131:0/3622266134 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f18700039b0 msgr2=0x7f1870003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:56.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:56.556+0000 7f188be00640 1 --2- 172.21.15.131:0/3622266134 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f18700039b0 0x7f1870003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:56.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:56.556+0000 7f188be00640 1 -- 172.21.15.131:0/3622266134 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1870003520 con 0x7f18700051c0 2024-05-26T22:52:56.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:56.556+0000 7f1882a00640 1 --2- 172.21.15.131:0/3622266134 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1870004800 0x7f1870004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:52:56.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:56.556+0000 7f188be00640 1 -- 172.21.15.131:0/3622266134 reap_dead start 2024-05-26T22:52:56.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:56.557+0000 7f188be00640 1 --2- 172.21.15.131:0/3622266134 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18700051c0 0x7f187000b9c0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f188400a840 tx=0x7f1870005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6b7d9b96591a4bc server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:56.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:56.557+0000 7f188aa00640 1 -- 172.21.15.131:0/3622266134 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f188400a460 con 0x7f18700051c0 2024-05-26T22:52:56.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:56.557+0000 7f188aa00640 1 -- 172.21.15.131:0/3622266134 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f188402f070 con 0x7f18700051c0 2024-05-26T22:52:56.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:56.557+0000 7f188aa00640 1 -- 172.21.15.131:0/3622266134 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f188402bcb0 con 0x7f18700051c0 2024-05-26T22:52:56.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:56.557+0000 7f1890a00640 1 -- 172.21.15.131:0/3622266134 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18700051c0 msgr2=0x7f187000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:56.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:56.557+0000 7f1890a00640 1 --2- 172.21.15.131:0/3622266134 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18700051c0 0x7f187000b9c0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f188400a840 tx=0x7f1870005830 comp rx=0 tx=0).stop 2024-05-26T22:52:56.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:56.558+0000 7f1890a00640 1 -- 172.21.15.131:0/3622266134 shutdown_connections 2024-05-26T22:52:56.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:56.558+0000 7f1890a00640 1 --2- 172.21.15.131:0/3622266134 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18700051c0 0x7f187000b9c0 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:56.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:56.558+0000 7f1890a00640 1 -- 172.21.15.131:0/3622266134 >> 172.21.15.131:0/3622266134 conn(0x7f188c073070 msgr2=0x7f188c0d0c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:52:56.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:56.558+0000 7f1890a00640 1 -- 172.21.15.131:0/3622266134 shutdown_connections 2024-05-26T22:52:56.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:56.558+0000 7f1890a00640 1 -- 172.21.15.131:0/3622266134 wait complete. 2024-05-26T22:52:56.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:56.559+0000 7f1890a00640 1 Processor -- start 2024-05-26T22:52:56.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:56.559+0000 7f1890a00640 1 -- start start 2024-05-26T22:52:56.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:56.560+0000 7f1890a00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f18700051c0 0x7f188c15be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:56.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:56.560+0000 7f1890a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f188c0abd10 0x7f188c15c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:56.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:56.560+0000 7f1890a00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f188c1f7080 0x7f188c1f7530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:56.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:56.560+0000 7f1890a00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f188c15e7c0 con 0x7f188c0abd10 2024-05-26T22:52:56.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:56.560+0000 7f1890a00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f188c0d2c30 con 0x7f188c1f7080 2024-05-26T22:52:56.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:56.560+0000 7f1890a00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f188c0cf010 con 0x7f18700051c0 2024-05-26T22:52:56.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:56.561+0000 7f1882a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f188c0abd10 0x7f188c15c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:56.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:56.561+0000 7f1882a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f188c0abd10 0x7f188c15c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48414/0 (socket says 172.21.15.131:48414) 2024-05-26T22:52:56.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:56.561+0000 7f1882a00640 1 -- 172.21.15.131:0/1403311225 learned_addr learned my addr 172.21.15.131:0/1403311225 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:52:56.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:56.561+0000 7f188b400640 1 --2- 172.21.15.131:0/1403311225 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f18700051c0 0x7f188c15be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:56.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:56.561+0000 7f188be00640 1 --2- 172.21.15.131:0/1403311225 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f188c1f7080 0x7f188c1f7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:56.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:56.561+0000 7f1882a00640 1 -- 172.21.15.131:0/1403311225 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f18700051c0 msgr2=0x7f188c15be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:56.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:56.561+0000 7f1882a00640 1 --2- 172.21.15.131:0/1403311225 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f18700051c0 0x7f188c15be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:56.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:56.561+0000 7f1882a00640 1 -- 172.21.15.131:0/1403311225 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f188c1f7080 msgr2=0x7f188c1f7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:56.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:56.561+0000 7f1882a00640 1 --2- 172.21.15.131:0/1403311225 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f188c1f7080 0x7f188c1f7530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:56.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:56.561+0000 7f1882a00640 1 -- 172.21.15.131:0/1403311225 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f188c0aac70 con 0x7f188c0abd10 2024-05-26T22:52:56.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:56.561+0000 7f1882a00640 1 --2- 172.21.15.131:0/1403311225 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f188c0abd10 0x7f188c15c340 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f18740108e0 tx=0x7f1874010da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:56.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:56.562+0000 7f1883e00640 1 -- 172.21.15.131:0/1403311225 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f187400f040 con 0x7f188c0abd10 2024-05-26T22:52:56.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:56.562+0000 7f1883e00640 1 -- 172.21.15.131:0/1403311225 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f187400ec20 con 0x7f188c0abd10 2024-05-26T22:52:56.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:56.562+0000 7f1883e00640 1 -- 172.21.15.131:0/1403311225 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f187401c400 con 0x7f188c0abd10 2024-05-26T22:52:56.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:56.562+0000 7f1890a00640 1 -- 172.21.15.131:0/1403311225 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f188c0aaf50 con 0x7f188c0abd10 2024-05-26T22:52:56.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:56.562+0000 7f1890a00640 1 -- 172.21.15.131:0/1403311225 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f188c0ab510 con 0x7f188c0abd10 2024-05-26T22:52:56.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:56.563+0000 7f1890a00640 1 -- 172.21.15.131:0/1403311225 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f188c0694e0 con 0x7f188c0abd10 2024-05-26T22:52:56.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:56.564+0000 7f1883e00640 1 -- 172.21.15.131:0/1403311225 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f187401c5a0 con 0x7f188c0abd10 2024-05-26T22:52:56.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:56.565+0000 7f1883e00640 1 --2- 172.21.15.131:0/1403311225 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f186806be30 0x7f186806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:56.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:56.565+0000 7f1883e00640 1 -- 172.21.15.131:0/1403311225 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7f187408b690 con 0x7f188c0abd10 2024-05-26T22:52:56.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:56.565+0000 7f188b400640 1 --2- 172.21.15.131:0/1403311225 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f186806be30 0x7f186806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:56.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:56.566+0000 7f188b400640 1 --2- 172.21.15.131:0/1403311225 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f186806be30 0x7f186806e2f0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f187c007fd0 tx=0x7f187c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:56.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:56.569+0000 7f1883e00640 1 -- 172.21.15.131:0/1403311225 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f1874018030 con 0x7f188c0abd10 2024-05-26T22:52:56.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:56.807+0000 7f1890a00640 1 -- 172.21.15.131:0/1403311225 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f188c15d570 con 0x7f188c0abd10 2024-05-26T22:52:57.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:57.328+0000 7f1883e00640 1 -- 172.21.15.131:0/1403311225 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v64) ==== 122+0+0 (secure 0 0 0) 0x7f187405a030 con 0x7f188c0abd10 2024-05-26T22:52:57.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:57.331+0000 7f1890a00640 1 -- 172.21.15.131:0/1403311225 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f186806be30 msgr2=0x7f186806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:57.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:57.331+0000 7f1890a00640 1 --2- 172.21.15.131:0/1403311225 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f186806be30 0x7f186806e2f0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f187c007fd0 tx=0x7f187c009040 comp rx=0 tx=0).stop 2024-05-26T22:52:57.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:57.331+0000 7f1890a00640 1 -- 172.21.15.131:0/1403311225 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f188c0abd10 msgr2=0x7f188c15c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:57.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:57.331+0000 7f1890a00640 1 --2- 172.21.15.131:0/1403311225 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f188c0abd10 0x7f188c15c340 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f18740108e0 tx=0x7f1874010da0 comp rx=0 tx=0).stop 2024-05-26T22:52:57.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:57.331+0000 7f1890a00640 1 -- 172.21.15.131:0/1403311225 shutdown_connections 2024-05-26T22:52:57.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:57.331+0000 7f1890a00640 1 --2- 172.21.15.131:0/1403311225 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f186806be30 0x7f186806e2f0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:57.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:57.331+0000 7f1890a00640 1 --2- 172.21.15.131:0/1403311225 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f188c1f7080 0x7f188c1f7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:57.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:57.331+0000 7f1890a00640 1 --2- 172.21.15.131:0/1403311225 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f188c0abd10 0x7f188c15c340 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:57.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:57.331+0000 7f1890a00640 1 --2- 172.21.15.131:0/1403311225 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f18700051c0 0x7f188c15be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:57.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:57.331+0000 7f1890a00640 1 -- 172.21.15.131:0/1403311225 >> 172.21.15.131:0/1403311225 conn(0x7f188c073070 msgr2=0x7f188c0b7220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:52:57.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:57.332+0000 7f1890a00640 1 -- 172.21.15.131:0/1403311225 shutdown_connections 2024-05-26T22:52:57.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:57.332+0000 7f1890a00640 1 -- 172.21.15.131:0/1403311225 wait complete. 2024-05-26T22:52:57.352 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-26T22:52:57.352 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-26T22:52:57.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:57.554+0000 7f52b6000640 1 Processor -- start 2024-05-26T22:52:57.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:57.554+0000 7f52b6000640 1 -- start start 2024-05-26T22:52:57.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:57.555+0000 7f52b6000640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f52b0155840 0x7f52b0071930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:57.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:57.556+0000 7f52b6000640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f52b015c3d0 0x7f52b006ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:57.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:57.556+0000 7f52b6000640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52b00722a0 con 0x7f52b006f490 2024-05-26T22:52:57.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:57.556+0000 7f52b6000640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f52b0072580 con 0x7f52b015c3d0 2024-05-26T22:52:57.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:57.556+0000 7f52b6000640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f52b0072860 con 0x7f52b0155840 2024-05-26T22:52:57.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:57.556+0000 7f52a7e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f52b015c3d0 0x7f52b006ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:57.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:57.556+0000 7f52a7e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f52b015c3d0 0x7f52b006ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48428/0 (socket says 172.21.15.131:48428) 2024-05-26T22:52:57.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:57.556+0000 7f52a7e00640 1 -- 172.21.15.131:0/2337366231 learned_addr learned my addr 172.21.15.131:0/2337366231 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:52:57.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:57.556+0000 7f52a7e00640 1 -- 172.21.15.131:0/2337366231 >> v2:172.21.15.187:3300/0 conn(0x7f52b0155840 msgr2=0x7f52b0071930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:57.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:57.556+0000 7f52a7e00640 1 --2- 172.21.15.131:0/2337366231 >> v2:172.21.15.187:3300/0 conn(0x7f52b0155840 0x7f52b0071930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:57.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:57.556+0000 7f52a7e00640 1 -- 172.21.15.131:0/2337366231 >> v1:172.21.15.187:6789/0 conn(0x7f52b006f490 legacy=0x7f52b015ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:57.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:57.556+0000 7f52a7e00640 1 -- 172.21.15.131:0/2337366231 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52b0072b40 con 0x7f52b015c3d0 2024-05-26T22:52:57.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:57.556+0000 7f52a7e00640 1 --2- 172.21.15.131:0/2337366231 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f52b015c3d0 0x7f52b006ef50 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f52b006d400 tx=0x7f529802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca60fe4322292872 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:57.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:57.557+0000 7f52aea00640 1 -- 172.21.15.131:0/2337366231 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5298037070 con 0x7f52b015c3d0 2024-05-26T22:52:57.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:57.557+0000 7f52aea00640 1 -- 172.21.15.131:0/2337366231 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5298032440 con 0x7f52b015c3d0 2024-05-26T22:52:57.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:57.557+0000 7f52aea00640 1 -- 172.21.15.131:0/2337366231 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5298032810 con 0x7f52b015c3d0 2024-05-26T22:52:57.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:57.557+0000 7f52b6000640 1 -- 172.21.15.131:0/2337366231 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f52b015c3d0 msgr2=0x7f52b006ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:57.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:57.557+0000 7f52b6000640 1 --2- 172.21.15.131:0/2337366231 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f52b015c3d0 0x7f52b006ef50 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f52b006d400 tx=0x7f529802a230 comp rx=0 tx=0).stop 2024-05-26T22:52:57.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:57.557+0000 7f52b6000640 1 -- 172.21.15.131:0/2337366231 shutdown_connections 2024-05-26T22:52:57.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:57.557+0000 7f52b6000640 1 --2- 172.21.15.131:0/2337366231 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f52b015c3d0 0x7f52b006ef50 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:57.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:57.557+0000 7f52b6000640 1 --2- 172.21.15.131:0/2337366231 >> v2:172.21.15.187:3300/0 conn(0x7f52b0155840 0x7f52b0071930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:57.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:57.557+0000 7f52b6000640 1 -- 172.21.15.131:0/2337366231 >> 172.21.15.131:0/2337366231 conn(0x7f52b00b1e90 msgr2=0x7f52b00b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:52:57.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:57.557+0000 7f52b6000640 1 -- 172.21.15.131:0/2337366231 shutdown_connections 2024-05-26T22:52:57.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:57.557+0000 7f52b6000640 1 -- 172.21.15.131:0/2337366231 wait complete. 2024-05-26T22:52:57.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:57.559+0000 7f52b6000640 1 Processor -- start 2024-05-26T22:52:57.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:57.560+0000 7f52b6000640 1 -- start start 2024-05-26T22:52:57.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:57.561+0000 7f52b6000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f52b006f490 0x7f52b01437d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:57.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:57.561+0000 7f52b6000640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f52b0155840 0x7f52b0143d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:57.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:57.561+0000 7f52b6000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f52b015c3d0 0x7f52b01f5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:57.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:57.561+0000 7f52b6000640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f52b0071ff0 con 0x7f52b015c3d0 2024-05-26T22:52:57.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:57.561+0000 7f52b6000640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f52b0071e70 con 0x7f52b0155840 2024-05-26T22:52:57.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:57.561+0000 7f52b6000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f52b00aee50 con 0x7f52b006f490 2024-05-26T22:52:57.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:57.561+0000 7f52afe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f52b015c3d0 0x7f52b01f5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:57.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:57.561+0000 7f52afe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f52b015c3d0 0x7f52b01f5250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48440/0 (socket says 172.21.15.131:48440) 2024-05-26T22:52:57.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:57.561+0000 7f52afe00640 1 -- 172.21.15.131:0/1087683807 learned_addr learned my addr 172.21.15.131:0/1087683807 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:52:57.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:57.561+0000 7f52a7e00640 1 --2- 172.21.15.131:0/1087683807 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f52b0155840 0x7f52b0143d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:57.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:57.561+0000 7f52af400640 1 --2- 172.21.15.131:0/1087683807 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f52b006f490 0x7f52b01437d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:57.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:57.561+0000 7f52afe00640 1 -- 172.21.15.131:0/1087683807 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f52b006f490 msgr2=0x7f52b01437d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:57.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:57.561+0000 7f52afe00640 1 --2- 172.21.15.131:0/1087683807 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f52b006f490 0x7f52b01437d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:57.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:57.561+0000 7f52afe00640 1 -- 172.21.15.131:0/1087683807 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f52b0155840 msgr2=0x7f52b0143d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:57.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:57.561+0000 7f52afe00640 1 --2- 172.21.15.131:0/1087683807 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f52b0155840 0x7f52b0143d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:57.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:57.561+0000 7f52afe00640 1 -- 172.21.15.131:0/1087683807 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52b00cf6b0 con 0x7f52b015c3d0 2024-05-26T22:52:57.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:57.562+0000 7f52afe00640 1 --2- 172.21.15.131:0/1087683807 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f52b015c3d0 0x7f52b01f5250 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f52a800c7b0 tx=0x7f52a800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:57.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:57.562+0000 7f52a7400640 1 -- 172.21.15.131:0/1087683807 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a8017070 con 0x7f52b015c3d0 2024-05-26T22:52:57.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:57.562+0000 7f52a7400640 1 -- 172.21.15.131:0/1087683807 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52a8003da0 con 0x7f52b015c3d0 2024-05-26T22:52:57.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:57.562+0000 7f52a7400640 1 -- 172.21.15.131:0/1087683807 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a80052b0 con 0x7f52b015c3d0 2024-05-26T22:52:57.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:57.562+0000 7f52b6000640 1 -- 172.21.15.131:0/1087683807 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52b01418c0 con 0x7f52b015c3d0 2024-05-26T22:52:57.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:57.562+0000 7f52b6000640 1 -- 172.21.15.131:0/1087683807 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52b01f5790 con 0x7f52b015c3d0 2024-05-26T22:52:57.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:57.563+0000 7f52b6000640 1 -- 172.21.15.131:0/1087683807 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52b00694e0 con 0x7f52b015c3d0 2024-05-26T22:52:57.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:57.564+0000 7f52a7400640 1 -- 172.21.15.131:0/1087683807 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f52a8005450 con 0x7f52b015c3d0 2024-05-26T22:52:57.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:57.565+0000 7f52a7400640 1 --2- 172.21.15.131:0/1087683807 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f528c06be30 0x7f528c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:57.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:57.565+0000 7f52a7400640 1 -- 172.21.15.131:0/1087683807 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7f52a808a3c0 con 0x7f52b015c3d0 2024-05-26T22:52:57.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:57.565+0000 7f52af400640 1 --2- 172.21.15.131:0/1087683807 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f528c06be30 0x7f528c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:57.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:57.566+0000 7f52af400640 1 --2- 172.21.15.131:0/1087683807 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f528c06be30 0x7f528c06e2f0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f52a0009890 tx=0x7f52a0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:57.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:57.571+0000 7f52a7400640 1 -- 172.21.15.131:0/1087683807 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f52a8058d60 con 0x7f52b015c3d0 2024-05-26T22:52:57.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:57.805+0000 7f52b6000640 1 -- 172.21.15.131:0/1087683807 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f52b015bc60 con 0x7f52b015c3d0 2024-05-26T22:52:57.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:57.806+0000 7f52a7400640 1 -- 172.21.15.131:0/1087683807 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) ==== 94+0+5195 (secure 0 0 0) 0x7f52a8058660 con 0x7f52b015c3d0 2024-05-26T22:52:57.807 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:52:57.807 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":64,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5359,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/4204396845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6833","nonce":4204396845},{"type":"v1","addr":"172.21.15.187:6835","nonce":4204396845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59},{"gid":6861,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6836/4223534499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":4223534499},{"type":"v1","addr":"172.21.15.131:6836","nonce":4223534499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58},{"gid":6864,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/1418741551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6835","nonce":1418741551},{"type":"v1","addr":"172.21.15.131:6837","nonce":1418741551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-26T22:52:53.755842+0000","modified":"2024-05-26T22:52:56.809807+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6876},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6876":{"gid":6876,"name":"d","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6834/3680999672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3680999672},{"type":"v1","addr":"172.21.15.187:6834","nonce":3680999672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6876,"qdb_cluster":[6876]},"id":5}]} 2024-05-26T22:52:57.807 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 64 2024-05-26T22:52:57.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:57.808+0000 7f52b6000640 1 -- 172.21.15.131:0/1087683807 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f528c06be30 msgr2=0x7f528c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:57.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:57.808+0000 7f52b6000640 1 --2- 172.21.15.131:0/1087683807 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f528c06be30 0x7f528c06e2f0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f52a0009890 tx=0x7f52a0008040 comp rx=0 tx=0).stop 2024-05-26T22:52:57.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:57.808+0000 7f52b6000640 1 -- 172.21.15.131:0/1087683807 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f52b015c3d0 msgr2=0x7f52b01f5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:57.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:57.808+0000 7f52b6000640 1 --2- 172.21.15.131:0/1087683807 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f52b015c3d0 0x7f52b01f5250 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f52a800c7b0 tx=0x7f52a800cc70 comp rx=0 tx=0).stop 2024-05-26T22:52:57.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:57.809+0000 7f52b6000640 1 -- 172.21.15.131:0/1087683807 shutdown_connections 2024-05-26T22:52:57.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:57.809+0000 7f52b6000640 1 --2- 172.21.15.131:0/1087683807 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f528c06be30 0x7f528c06e2f0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:57.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:57.809+0000 7f52b6000640 1 --2- 172.21.15.131:0/1087683807 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f52b015c3d0 0x7f52b01f5250 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:57.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:57.809+0000 7f52b6000640 1 --2- 172.21.15.131:0/1087683807 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f52b0155840 0x7f52b0143d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:57.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:57.809+0000 7f52b6000640 1 --2- 172.21.15.131:0/1087683807 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f52b006f490 0x7f52b01437d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:57.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:57.809+0000 7f52b6000640 1 -- 172.21.15.131:0/1087683807 >> 172.21.15.131:0/1087683807 conn(0x7f52b00b1e90 msgr2=0x7f52b00afe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:52:57.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:57.809+0000 7f52b6000640 1 -- 172.21.15.131:0/1087683807 shutdown_connections 2024-05-26T22:52:57.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:57.809+0000 7f52b6000640 1 -- 172.21.15.131:0/1087683807 wait complete. 2024-05-26T22:52:57.828 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-26T22:52:58.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.021+0000 7f86c2c00640 1 Processor -- start 2024-05-26T22:52:58.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.021+0000 7f86c2c00640 1 -- start start 2024-05-26T22:52:58.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.022+0000 7f86c2c00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f86bc156750 0x7f86bc15fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:58.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.022+0000 7f86c2c00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f86bc06e8c0 0x7f86bc15d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:58.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.022+0000 7f86c2c00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86bc160620 con 0x7f86bc0cc7f0 2024-05-26T22:52:58.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.022+0000 7f86c2c00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f86bc15cbd0 con 0x7f86bc06e8c0 2024-05-26T22:52:58.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.022+0000 7f86c2c00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f86bc160900 con 0x7f86bc156750 2024-05-26T22:52:58.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.022+0000 7f86c0e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f86bc06e8c0 0x7f86bc15d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:58.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.023+0000 7f86c0e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f86bc06e8c0 0x7f86bc15d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48448/0 (socket says 172.21.15.131:48448) 2024-05-26T22:52:58.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.023+0000 7f86c0e00640 1 -- 172.21.15.131:0/4111168235 learned_addr learned my addr 172.21.15.131:0/4111168235 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:52:58.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.023+0000 7f86c2200640 1 --1- 172.21.15.131:0/4111168235 >> v1:172.21.15.131:6789/0 conn(0x7f86bc0cc7f0 0x7f86bc0cdc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:42262/0 (socket says 172.21.15.131:42262) 2024-05-26T22:52:58.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.023+0000 7f86c1800640 1 --2- 172.21.15.131:0/4111168235 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f86bc156750 0x7f86bc15fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:58.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.023+0000 7f86c0e00640 1 -- 172.21.15.131:0/4111168235 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f86bc156750 msgr2=0x7f86bc15fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:58.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.023+0000 7f86c0e00640 1 --2- 172.21.15.131:0/4111168235 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f86bc156750 0x7f86bc15fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:58.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.023+0000 7f86c0e00640 1 -- 172.21.15.131:0/4111168235 >> v1:172.21.15.131:6789/0 conn(0x7f86bc0cc7f0 legacy=0x7f86bc0cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:58.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.023+0000 7f86c0e00640 1 -- 172.21.15.131:0/4111168235 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86bc160be0 con 0x7f86bc06e8c0 2024-05-26T22:52:58.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.023+0000 7f86c1800640 1 --2- 172.21.15.131:0/4111168235 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f86bc156750 0x7f86bc15fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:52:58.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.024+0000 7f86c0e00640 1 --2- 172.21.15.131:0/4111168235 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f86bc06e8c0 0x7f86bc15d060 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f86a8009870 tx=0x7f86a802a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2f98615cbc07713 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:58.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.024+0000 7f86bbe00640 1 -- 172.21.15.131:0/4111168235 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a8037070 con 0x7f86bc06e8c0 2024-05-26T22:52:58.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.024+0000 7f86bbe00640 1 -- 172.21.15.131:0/4111168235 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86a8032440 con 0x7f86bc06e8c0 2024-05-26T22:52:58.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.024+0000 7f86bbe00640 1 -- 172.21.15.131:0/4111168235 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a8032810 con 0x7f86bc06e8c0 2024-05-26T22:52:58.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.025+0000 7f86c2c00640 1 -- 172.21.15.131:0/4111168235 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f86bc06e8c0 msgr2=0x7f86bc15d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:58.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.025+0000 7f86c2c00640 1 --2- 172.21.15.131:0/4111168235 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f86bc06e8c0 0x7f86bc15d060 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f86a8009870 tx=0x7f86a802a250 comp rx=0 tx=0).stop 2024-05-26T22:52:58.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.025+0000 7f86c2c00640 1 -- 172.21.15.131:0/4111168235 shutdown_connections 2024-05-26T22:52:58.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.025+0000 7f86c2c00640 1 --2- 172.21.15.131:0/4111168235 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f86bc06e8c0 0x7f86bc15d060 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:58.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.025+0000 7f86c2c00640 1 --2- 172.21.15.131:0/4111168235 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f86bc156750 0x7f86bc15fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:58.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.025+0000 7f86c2c00640 1 -- 172.21.15.131:0/4111168235 >> 172.21.15.131:0/4111168235 conn(0x7f86bc0cff40 msgr2=0x7f86bc0d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:52:58.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.025+0000 7f86c2c00640 1 -- 172.21.15.131:0/4111168235 shutdown_connections 2024-05-26T22:52:58.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.026+0000 7f86c2c00640 1 -- 172.21.15.131:0/4111168235 wait complete. 2024-05-26T22:52:58.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.027+0000 7f86c2c00640 1 Processor -- start 2024-05-26T22:52:58.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.027+0000 7f86c2c00640 1 -- start start 2024-05-26T22:52:58.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.028+0000 7f86c2c00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f86bc06e8c0 0x7f86bc1f3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:58.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.028+0000 7f86c2c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f86bc0cc7f0 0x7f86bc1e83c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:58.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.028+0000 7f86c2c00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f86bc156750 0x7f86bc1e8900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:58.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.028+0000 7f86c2c00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f86bc160280 con 0x7f86bc0cc7f0 2024-05-26T22:52:58.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.028+0000 7f86c2c00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f86bc156b30 con 0x7f86bc06e8c0 2024-05-26T22:52:58.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.028+0000 7f86c2c00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f86bc160400 con 0x7f86bc156750 2024-05-26T22:52:58.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.029+0000 7f86c0e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f86bc0cc7f0 0x7f86bc1e83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:58.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.029+0000 7f86c0e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f86bc0cc7f0 0x7f86bc1e83c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48454/0 (socket says 172.21.15.131:48454) 2024-05-26T22:52:58.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.029+0000 7f86c0e00640 1 -- 172.21.15.131:0/2075639053 learned_addr learned my addr 172.21.15.131:0/2075639053 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:52:58.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.029+0000 7f86c0e00640 1 -- 172.21.15.131:0/2075639053 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f86bc156750 msgr2=0x7f86bc1e8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:58.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.029+0000 7f86c0e00640 1 --2- 172.21.15.131:0/2075639053 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f86bc156750 0x7f86bc1e8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:58.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.029+0000 7f86c0e00640 1 -- 172.21.15.131:0/2075639053 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f86bc06e8c0 msgr2=0x7f86bc1f3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:58.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.029+0000 7f86c1800640 1 --2- 172.21.15.131:0/2075639053 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f86bc06e8c0 0x7f86bc1f3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:58.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.029+0000 7f86c0e00640 1 --2- 172.21.15.131:0/2075639053 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f86bc06e8c0 0x7f86bc1f3b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:58.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.029+0000 7f86c0e00640 1 -- 172.21.15.131:0/2075639053 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86bc0b2210 con 0x7f86bc0cc7f0 2024-05-26T22:52:58.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.030+0000 7f86c0e00640 1 --2- 172.21.15.131:0/2075639053 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f86bc0cc7f0 0x7f86bc1e83c0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f86a8000c00 tx=0x7f86a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:58.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.030+0000 7f86b9600640 1 -- 172.21.15.131:0/2075639053 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a8037040 con 0x7f86bc0cc7f0 2024-05-26T22:52:58.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.030+0000 7f86b9600640 1 -- 172.21.15.131:0/2075639053 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86a8042070 con 0x7f86bc0cc7f0 2024-05-26T22:52:58.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.030+0000 7f86c2c00640 1 -- 172.21.15.131:0/2075639053 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86bc0b24f0 con 0x7f86bc0cc7f0 2024-05-26T22:52:58.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.030+0000 7f86b9600640 1 -- 172.21.15.131:0/2075639053 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a8030bd0 con 0x7f86bc0cc7f0 2024-05-26T22:52:58.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.030+0000 7f86c2c00640 1 -- 172.21.15.131:0/2075639053 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86bc1308e0 con 0x7f86bc0cc7f0 2024-05-26T22:52:58.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.031+0000 7f86c2c00640 1 -- 172.21.15.131:0/2075639053 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86bc0694e0 con 0x7f86bc0cc7f0 2024-05-26T22:52:58.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.032+0000 7f86b9600640 1 -- 172.21.15.131:0/2075639053 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f86a8047460 con 0x7f86bc0cc7f0 2024-05-26T22:52:58.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.033+0000 7f86b9600640 1 --2- 172.21.15.131:0/2075639053 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f869c06be60 0x7f869c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:58.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.033+0000 7f86b9600640 1 -- 172.21.15.131:0/2075639053 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7f86a80abbb0 con 0x7f86bc0cc7f0 2024-05-26T22:52:58.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.033+0000 7f86c1800640 1 --2- 172.21.15.131:0/2075639053 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f869c06be60 0x7f869c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:58.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.034+0000 7f86c1800640 1 --2- 172.21.15.131:0/2075639053 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f869c06be60 0x7f869c06e320 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f86b0008850 tx=0x7f86b0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:58.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.037+0000 7f86b9600640 1 -- 172.21.15.131:0/2075639053 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f86a8045360 con 0x7f86bc0cc7f0 2024-05-26T22:52:58.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.236+0000 7f86c2c00640 1 -- 172.21.15.131:0/2075639053 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f86bc0ce350 con 0x7f86bc0cc7f0 2024-05-26T22:52:58.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.239+0000 7f86b9600640 1 -- 172.21.15.131:0/2075639053 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v69) ==== 74+0+16692 (secure 0 0 0) 0x7f86a807a430 con 0x7f86bc0cc7f0 2024-05-26T22:52:58.239 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:52:58.239 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":69,"fsid":"46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f","created":"2024-05-26T22:47:57.288905+0000","modified":"2024-05-26T22:52:55.304822+0000","last_up_change":"2024-05-26T22:48:09.602207+0000","last_in_change":"2024-05-26T22:48:00.687341+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-26T22:48:10.463339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-05-26T22:52:51.536251+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-05-26T22:52:52.768524+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"814eaf62-a18d-494b-8e27-fd8fb88adea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6801","nonce":2350940051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6803","nonce":2350940051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6807","nonce":2350940051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6805","nonce":2350940051}]},"public_addr":"172.21.15.131:6801/2350940051","cluster_addr":"172.21.15.131:6803/2350940051","heartbeat_back_addr":"172.21.15.131:6807/2350940051","heartbeat_front_addr":"172.21.15.131:6805/2350940051","state":["exists","up"]},{"osd":1,"uuid":"42e66287-f6c6-4e26-af3c-6dfaa3536d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6825","nonce":1525528247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6827","nonce":1525528247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6831","nonce":1525528247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6829","nonce":1525528247}]},"public_addr":"172.21.15.131:6825/1525528247","cluster_addr":"172.21.15.131:6827/1525528247","heartbeat_back_addr":"172.21.15.131:6831/1525528247","heartbeat_front_addr":"172.21.15.131:6829/1525528247","state":["exists","up"]},{"osd":2,"uuid":"0da34673-04a2-4fd0-b57a-0ff2b679b043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6817","nonce":3627842210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6819","nonce":3627842210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6823","nonce":3627842210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6821","nonce":3627842210}]},"public_addr":"172.21.15.131:6817/3627842210","cluster_addr":"172.21.15.131:6819/3627842210","heartbeat_back_addr":"172.21.15.131:6823/3627842210","heartbeat_front_addr":"172.21.15.131:6821/3627842210","state":["exists","up"]},{"osd":3,"uuid":"cf117a27-d6b2-43d3-ae52-b47988bf8eed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6809","nonce":350790591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6811","nonce":350790591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6815","nonce":350790591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6813","nonce":350790591}]},"public_addr":"172.21.15.131:6809/350790591","cluster_addr":"172.21.15.131:6811/350790591","heartbeat_back_addr":"172.21.15.131:6815/350790591","heartbeat_front_addr":"172.21.15.131:6813/350790591","state":["exists","up"]},{"osd":4,"uuid":"a092ccfc-be89-4a84-88d5-0f781fa715eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6816","nonce":244560995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6819","nonce":244560995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6823","nonce":244560995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6821","nonce":244560995}]},"public_addr":"172.21.15.187:6816/244560995","cluster_addr":"172.21.15.187:6819/244560995","heartbeat_back_addr":"172.21.15.187:6823/244560995","heartbeat_front_addr":"172.21.15.187:6821/244560995","state":["exists","up"]},{"osd":5,"uuid":"7513ba69-dd00-4a35-b190-9e8cc3b2bdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6801","nonce":324619398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6803","nonce":324619398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6812","nonce":324619398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6807","nonce":324619398}]},"public_addr":"172.21.15.187:6801/324619398","cluster_addr":"172.21.15.187:6803/324619398","heartbeat_back_addr":"172.21.15.187:6812/324619398","heartbeat_front_addr":"172.21.15.187:6807/324619398","state":["exists","up"]},{"osd":6,"uuid":"50081810-c093-4d2f-a4d7-b7c695c5400d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6806","nonce":1391347786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6809","nonce":1391347786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6818","nonce":1391347786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6813","nonce":1391347786}]},"public_addr":"172.21.15.187:6806/1391347786","cluster_addr":"172.21.15.187:6809/1391347786","heartbeat_back_addr":"172.21.15.187:6818/1391347786","heartbeat_front_addr":"172.21.15.187:6813/1391347786","state":["exists","up"]},{"osd":7,"uuid":"775dae05-54f5-4ad4-bbce-92863c6ed2ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6825","nonce":1220065622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6827","nonce":1220065622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6831","nonce":1220065622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6829","nonce":1220065622}]},"public_addr":"172.21.15.187:6825/1220065622","cluster_addr":"172.21.15.187:6827/1220065622","heartbeat_back_addr":"172.21.15.187:6831/1220065622","heartbeat_front_addr":"172.21.15.187:6829/1220065622","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:05.983520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.299516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.067323+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.307748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.018742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.258775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.010912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.295355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-26T22:52:58.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.241+0000 7f86c2c00640 1 -- 172.21.15.131:0/2075639053 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f869c06be60 msgr2=0x7f869c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:58.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.241+0000 7f86c2c00640 1 --2- 172.21.15.131:0/2075639053 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f869c06be60 0x7f869c06e320 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f86b0008850 tx=0x7f86b0008040 comp rx=0 tx=0).stop 2024-05-26T22:52:58.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.241+0000 7f86c2c00640 1 -- 172.21.15.131:0/2075639053 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f86bc0cc7f0 msgr2=0x7f86bc1e83c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:58.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.241+0000 7f86c2c00640 1 --2- 172.21.15.131:0/2075639053 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f86bc0cc7f0 0x7f86bc1e83c0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f86a8000c00 tx=0x7f86a80046c0 comp rx=0 tx=0).stop 2024-05-26T22:52:58.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.241+0000 7f86c2c00640 1 -- 172.21.15.131:0/2075639053 shutdown_connections 2024-05-26T22:52:58.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.241+0000 7f86c2c00640 1 --2- 172.21.15.131:0/2075639053 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f86bc156750 0x7f86bc1e8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:58.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.241+0000 7f86c2c00640 1 --2- 172.21.15.131:0/2075639053 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f869c06be60 0x7f869c06e320 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:58.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.241+0000 7f86c2c00640 1 --2- 172.21.15.131:0/2075639053 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f86bc0cc7f0 0x7f86bc1e83c0 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:58.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.241+0000 7f86c2c00640 1 --2- 172.21.15.131:0/2075639053 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f86bc06e8c0 0x7f86bc1f3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:58.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.241+0000 7f86c2c00640 1 -- 172.21.15.131:0/2075639053 >> 172.21.15.131:0/2075639053 conn(0x7f86bc0cff40 msgr2=0x7f86bc0717e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:52:58.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.242+0000 7f86c2c00640 1 -- 172.21.15.131:0/2075639053 shutdown_connections 2024-05-26T22:52:58.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.242+0000 7f86c2c00640 1 -- 172.21.15.131:0/2075639053 wait complete. 2024-05-26T22:52:58.259 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-26T22:52:58.259 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-26T22:52:58.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.460+0000 7f298e600640 1 Processor -- start 2024-05-26T22:52:58.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.460+0000 7f298e600640 1 -- start start 2024-05-26T22:52:58.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.462+0000 7f298e600640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f2988156750 0x7f298815fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:58.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.462+0000 7f298e600640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f29880cc7f0 0x7f29880ccc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:58.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.462+0000 7f298e600640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2988160620 con 0x7f298806e8c0 2024-05-26T22:52:58.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.462+0000 7f298e600640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f298815d770 con 0x7f2988156750 2024-05-26T22:52:58.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.462+0000 7f298e600640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f29880cd1a0 con 0x7f29880cc7f0 2024-05-26T22:52:58.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.462+0000 7f2987e00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f298806e8c0 0x7f298815d060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:42278/0 (socket says 172.21.15.131:42278) 2024-05-26T22:52:58.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.462+0000 7f298d200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f2988156750 0x7f298815fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:58.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.462+0000 7f2987e00640 1 -- 172.21.15.131:0/3421514985 learned_addr learned my addr 172.21.15.131:0/3421514985 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:52:58.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.462+0000 7f298dc00640 1 --2- 172.21.15.131:0/3421514985 >> v2:172.21.15.187:3300/0 conn(0x7f29880cc7f0 0x7f29880ccc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:58.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.463+0000 7f2987400640 1 -- 172.21.15.131:0/3421514985 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2563930242 0 0) 0x7f2988160620 con 0x7f298806e8c0 2024-05-26T22:52:58.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.463+0000 7f2987400640 1 -- 172.21.15.131:0/3421514985 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2970003520 con 0x7f298806e8c0 2024-05-26T22:52:58.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.463+0000 7f298d200640 1 -- 172.21.15.131:0/3421514985 >> v2:172.21.15.187:3300/0 conn(0x7f29880cc7f0 msgr2=0x7f29880ccc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:58.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.463+0000 7f298d200640 1 --2- 172.21.15.131:0/3421514985 >> v2:172.21.15.187:3300/0 conn(0x7f29880cc7f0 0x7f29880ccc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:58.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.463+0000 7f298d200640 1 -- 172.21.15.131:0/3421514985 >> v1:172.21.15.131:6789/0 conn(0x7f298806e8c0 legacy=0x7f298815d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:58.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.463+0000 7f298d200640 1 -- 172.21.15.131:0/3421514985 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2988160900 con 0x7f2988156750 2024-05-26T22:52:58.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.464+0000 7f298d200640 1 --2- 172.21.15.131:0/3421514985 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2988156750 0x7f298815fd40 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f298815a320 tx=0x7f297c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17f1502be7c80aee server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:58.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.464+0000 7f2987400640 1 -- 172.21.15.131:0/3421514985 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f297c037070 con 0x7f2988156750 2024-05-26T22:52:58.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.464+0000 7f2987400640 1 -- 172.21.15.131:0/3421514985 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f297c032440 con 0x7f2988156750 2024-05-26T22:52:58.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.464+0000 7f2987400640 1 -- 172.21.15.131:0/3421514985 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f297c032810 con 0x7f2988156750 2024-05-26T22:52:58.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.465+0000 7f298e600640 1 -- 172.21.15.131:0/3421514985 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2988156750 msgr2=0x7f298815fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:58.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.465+0000 7f298e600640 1 --2- 172.21.15.131:0/3421514985 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2988156750 0x7f298815fd40 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f298815a320 tx=0x7f297c02a1a0 comp rx=0 tx=0).stop 2024-05-26T22:52:58.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.465+0000 7f298e600640 1 -- 172.21.15.131:0/3421514985 shutdown_connections 2024-05-26T22:52:58.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.465+0000 7f298e600640 1 --2- 172.21.15.131:0/3421514985 >> v2:172.21.15.187:3300/0 conn(0x7f29880cc7f0 0x7f29880ccc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:58.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.465+0000 7f298e600640 1 --2- 172.21.15.131:0/3421514985 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2988156750 0x7f298815fd40 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:58.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.465+0000 7f298e600640 1 -- 172.21.15.131:0/3421514985 >> 172.21.15.131:0/3421514985 conn(0x7f29880cff40 msgr2=0x7f29880d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:52:58.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.465+0000 7f298e600640 1 -- 172.21.15.131:0/3421514985 shutdown_connections 2024-05-26T22:52:58.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.465+0000 7f298e600640 1 -- 172.21.15.131:0/3421514985 wait complete. 2024-05-26T22:52:58.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.466+0000 7f298e600640 1 Processor -- start 2024-05-26T22:52:58.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.467+0000 7f298e600640 1 -- start start 2024-05-26T22:52:58.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.467+0000 7f298e600640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f298806e8c0 0x7f298815b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:58.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.467+0000 7f298e600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f29880cc7f0 0x7f298813bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:58.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.467+0000 7f298e600640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2988156750 0x7f298813c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:58.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.467+0000 7f298e600640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f298815c860 con 0x7f29880cc7f0 2024-05-26T22:52:58.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.467+0000 7f298e600640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f298815cbd0 con 0x7f2988156750 2024-05-26T22:52:58.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.467+0000 7f298e600640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f2988156b30 con 0x7f298806e8c0 2024-05-26T22:52:58.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.468+0000 7f2987e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f29880cc7f0 0x7f298813bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:58.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.468+0000 7f2987e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f29880cc7f0 0x7f298813bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48476/0 (socket says 172.21.15.131:48476) 2024-05-26T22:52:58.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.468+0000 7f2987e00640 1 -- 172.21.15.131:0/2271146011 learned_addr learned my addr 172.21.15.131:0/2271146011 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:52:58.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.468+0000 7f298dc00640 1 --2- 172.21.15.131:0/2271146011 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2988156750 0x7f298813c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:58.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.468+0000 7f2987e00640 1 -- 172.21.15.131:0/2271146011 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f298806e8c0 msgr2=0x7f298815b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:58.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.468+0000 7f2987e00640 1 --2- 172.21.15.131:0/2271146011 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f298806e8c0 0x7f298815b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:58.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.468+0000 7f2987e00640 1 -- 172.21.15.131:0/2271146011 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2988156750 msgr2=0x7f298813c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:58.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.468+0000 7f2987e00640 1 --2- 172.21.15.131:0/2271146011 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2988156750 0x7f298813c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:58.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.468+0000 7f2987e00640 1 -- 172.21.15.131:0/2271146011 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29881f80f0 con 0x7f29880cc7f0 2024-05-26T22:52:58.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.468+0000 7f2987e00640 1 --2- 172.21.15.131:0/2271146011 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f29880cc7f0 0x7f298813bb40 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f297400c460 tx=0x7f297400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:58.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.469+0000 7f2984c00640 1 -- 172.21.15.131:0/2271146011 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2974017070 con 0x7f29880cc7f0 2024-05-26T22:52:58.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.469+0000 7f2984c00640 1 -- 172.21.15.131:0/2271146011 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2974005ce0 con 0x7f29880cc7f0 2024-05-26T22:52:58.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.469+0000 7f2984c00640 1 -- 172.21.15.131:0/2271146011 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2974007290 con 0x7f29880cc7f0 2024-05-26T22:52:58.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.469+0000 7f298e600640 1 -- 172.21.15.131:0/2271146011 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29881f7850 con 0x7f29880cc7f0 2024-05-26T22:52:58.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.469+0000 7f298e600640 1 -- 172.21.15.131:0/2271146011 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29881f7e10 con 0x7f29880cc7f0 2024-05-26T22:52:58.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.470+0000 7f298e600640 1 -- 172.21.15.131:0/2271146011 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29881f72e0 con 0x7f29880cc7f0 2024-05-26T22:52:58.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.471+0000 7f2984c00640 1 -- 172.21.15.131:0/2271146011 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f297401a070 con 0x7f29880cc7f0 2024-05-26T22:52:58.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.472+0000 7f2984c00640 1 --2- 172.21.15.131:0/2271146011 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f296806be30 0x7f296806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:58.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.472+0000 7f2984c00640 1 -- 172.21.15.131:0/2271146011 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7f297408a1d0 con 0x7f29880cc7f0 2024-05-26T22:52:58.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.472+0000 7f298d200640 1 --2- 172.21.15.131:0/2271146011 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f296806be30 0x7f296806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:58.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.473+0000 7f298d200640 1 --2- 172.21.15.131:0/2271146011 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f296806be30 0x7f296806e2f0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f297c034000 tx=0x7f297c02c000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:58.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.476+0000 7f2984c00640 1 -- 172.21.15.131:0/2271146011 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f2974058b70 con 0x7f29880cc7f0 2024-05-26T22:52:58.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.657+0000 7f298e600640 1 -- 172.21.15.131:0/2271146011 --> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f29880694e0 con 0x7f296806be30 2024-05-26T22:52:58.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.674+0000 7f2984c00640 1 -- 172.21.15.131:0/2271146011 <== mgr.4104 v2:172.21.15.131:6832/3484832947 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252064 (secure 0 0 0) 0x7f29880694e0 con 0x7f296806be30 2024-05-26T22:52:58.678 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:52:58.680 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-05-26T22:52:58.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.680+0000 7f298e600640 1 -- 172.21.15.131:0/2271146011 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f296806be30 msgr2=0x7f296806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:58.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.680+0000 7f298e600640 1 --2- 172.21.15.131:0/2271146011 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f296806be30 0x7f296806e2f0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f297c034000 tx=0x7f297c02c000 comp rx=0 tx=0).stop 2024-05-26T22:52:58.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.680+0000 7f298e600640 1 -- 172.21.15.131:0/2271146011 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f29880cc7f0 msgr2=0x7f298813bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:58.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.680+0000 7f298e600640 1 --2- 172.21.15.131:0/2271146011 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f29880cc7f0 0x7f298813bb40 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f297400c460 tx=0x7f297400c920 comp rx=0 tx=0).stop 2024-05-26T22:52:58.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.680+0000 7f298e600640 1 -- 172.21.15.131:0/2271146011 shutdown_connections 2024-05-26T22:52:58.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.680+0000 7f298e600640 1 --2- 172.21.15.131:0/2271146011 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f296806be30 0x7f296806e2f0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:58.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.680+0000 7f298e600640 1 --2- 172.21.15.131:0/2271146011 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2988156750 0x7f298813c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:58.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.680+0000 7f298e600640 1 --2- 172.21.15.131:0/2271146011 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f29880cc7f0 0x7f298813bb40 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:58.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.680+0000 7f298e600640 1 --2- 172.21.15.131:0/2271146011 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f298806e8c0 0x7f298815b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:58.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.680+0000 7f298e600640 1 -- 172.21.15.131:0/2271146011 >> 172.21.15.131:0/2271146011 conn(0x7f29880cff40 msgr2=0x7f29880cf0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:52:58.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.681+0000 7f298e600640 1 -- 172.21.15.131:0/2271146011 shutdown_connections 2024-05-26T22:52:58.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.681+0000 7f298e600640 1 -- 172.21.15.131:0/2271146011 wait complete. 2024-05-26T22:52:58.698 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":206,"stamp":"2024-05-26T22:52:58.476472+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":93,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":257192,"kb_used_data":7712,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754717528,"statfs":{"total":773094113280,"available":772830748672,"internally_reserved":0,"allocated":7897088,"data_stored":4011892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001923"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.423385+0000","last_change":"2024-05-26T22:52:54.333360+0000","last_active":"2024-05-26T22:52:55.423385+0000","last_peered":"2024-05-26T22:52:55.423385+0000","last_clean":"2024-05-26T22:52:55.423385+0000","last_became_active":"2024-05-26T22:52:54.333050+0000","last_became_peered":"2024-05-26T22:52:54.333050+0000","last_unstale":"2024-05-26T22:52:55.423385+0000","last_undegraded":"2024-05-26T22:52:55.423385+0000","last_fullsized":"2024-05-26T22:52:55.423385+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T22:55:42.930355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.320822+0000","last_change":"2024-05-26T22:52:54.336338+0000","last_active":"2024-05-26T22:52:55.320822+0000","last_peered":"2024-05-26T22:52:55.320822+0000","last_clean":"2024-05-26T22:52:55.320822+0000","last_became_active":"2024-05-26T22:52:54.335884+0000","last_became_peered":"2024-05-26T22:52:54.335884+0000","last_unstale":"2024-05-26T22:52:55.320822+0000","last_undegraded":"2024-05-26T22:52:55.320822+0000","last_fullsized":"2024-05-26T22:52:55.320822+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:10:56.957932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.322331+0000","last_change":"2024-05-26T22:52:54.319682+0000","last_active":"2024-05-26T22:52:55.322331+0000","last_peered":"2024-05-26T22:52:55.322331+0000","last_clean":"2024-05-26T22:52:55.322331+0000","last_became_active":"2024-05-26T22:52:54.319265+0000","last_became_peered":"2024-05-26T22:52:54.319265+0000","last_unstale":"2024-05-26T22:52:55.322331+0000","last_undegraded":"2024-05-26T22:52:55.322331+0000","last_fullsized":"2024-05-26T22:52:55.322331+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T23:24:03.220832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.418078+0000","last_change":"2024-05-26T22:52:54.329529+0000","last_active":"2024-05-26T22:52:55.418078+0000","last_peered":"2024-05-26T22:52:55.418078+0000","last_clean":"2024-05-26T22:52:55.418078+0000","last_became_active":"2024-05-26T22:52:54.329181+0000","last_became_peered":"2024-05-26T22:52:54.329181+0000","last_unstale":"2024-05-26T22:52:55.418078+0000","last_undegraded":"2024-05-26T22:52:55.418078+0000","last_fullsized":"2024-05-26T22:52:55.418078+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:13:45.559399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.320076+0000","last_change":"2024-05-26T22:52:54.325242+0000","last_active":"2024-05-26T22:52:55.320076+0000","last_peered":"2024-05-26T22:52:55.320076+0000","last_clean":"2024-05-26T22:52:55.320076+0000","last_became_active":"2024-05-26T22:52:54.324610+0000","last_became_peered":"2024-05-26T22:52:54.324610+0000","last_unstale":"2024-05-26T22:52:55.320076+0000","last_undegraded":"2024-05-26T22:52:55.320076+0000","last_fullsized":"2024-05-26T22:52:55.320076+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:09:15.633550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.319277+0000","last_change":"2024-05-26T22:52:54.332798+0000","last_active":"2024-05-26T22:52:55.319277+0000","last_peered":"2024-05-26T22:52:55.319277+0000","last_clean":"2024-05-26T22:52:55.319277+0000","last_became_active":"2024-05-26T22:52:54.330758+0000","last_became_peered":"2024-05-26T22:52:54.330758+0000","last_unstale":"2024-05-26T22:52:55.319277+0000","last_undegraded":"2024-05-26T22:52:55.319277+0000","last_fullsized":"2024-05-26T22:52:55.319277+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:02:46.912953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.324794+0000","last_change":"2024-05-26T22:52:54.321771+0000","last_active":"2024-05-26T22:52:55.324794+0000","last_peered":"2024-05-26T22:52:55.324794+0000","last_clean":"2024-05-26T22:52:55.324794+0000","last_became_active":"2024-05-26T22:52:54.321452+0000","last_became_peered":"2024-05-26T22:52:54.321452+0000","last_unstale":"2024-05-26T22:52:55.324794+0000","last_undegraded":"2024-05-26T22:52:55.324794+0000","last_fullsized":"2024-05-26T22:52:55.324794+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:16:08.984694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.319002+0000","last_change":"2024-05-26T22:52:54.313423+0000","last_active":"2024-05-26T22:52:55.319002+0000","last_peered":"2024-05-26T22:52:55.319002+0000","last_clean":"2024-05-26T22:52:55.319002+0000","last_became_active":"2024-05-26T22:52:54.312774+0000","last_became_peered":"2024-05-26T22:52:54.312774+0000","last_unstale":"2024-05-26T22:52:55.319002+0000","last_undegraded":"2024-05-26T22:52:55.319002+0000","last_fullsized":"2024-05-26T22:52:55.319002+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:15:48.861927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.323409+0000","last_change":"2024-05-26T22:52:54.311650+0000","last_active":"2024-05-26T22:52:55.323409+0000","last_peered":"2024-05-26T22:52:55.323409+0000","last_clean":"2024-05-26T22:52:55.323409+0000","last_became_active":"2024-05-26T22:52:54.311145+0000","last_became_peered":"2024-05-26T22:52:54.311145+0000","last_unstale":"2024-05-26T22:52:55.323409+0000","last_undegraded":"2024-05-26T22:52:55.323409+0000","last_fullsized":"2024-05-26T22:52:55.323409+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:23:29.499207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.420707+0000","last_change":"2024-05-26T22:52:54.324499+0000","last_active":"2024-05-26T22:52:55.420707+0000","last_peered":"2024-05-26T22:52:55.420707+0000","last_clean":"2024-05-26T22:52:55.420707+0000","last_became_active":"2024-05-26T22:52:54.324109+0000","last_became_peered":"2024-05-26T22:52:54.324109+0000","last_unstale":"2024-05-26T22:52:55.420707+0000","last_undegraded":"2024-05-26T22:52:55.420707+0000","last_fullsized":"2024-05-26T22:52:55.420707+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:54:09.591587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.423353+0000","last_change":"2024-05-26T22:52:54.327991+0000","last_active":"2024-05-26T22:52:55.423353+0000","last_peered":"2024-05-26T22:52:55.423353+0000","last_clean":"2024-05-26T22:52:55.423353+0000","last_became_active":"2024-05-26T22:52:54.325134+0000","last_became_peered":"2024-05-26T22:52:54.325134+0000","last_unstale":"2024-05-26T22:52:55.423353+0000","last_undegraded":"2024-05-26T22:52:55.423353+0000","last_fullsized":"2024-05-26T22:52:55.423353+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:02:22.457006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.418087+0000","last_change":"2024-05-26T22:52:54.323082+0000","last_active":"2024-05-26T22:52:55.418087+0000","last_peered":"2024-05-26T22:52:55.418087+0000","last_clean":"2024-05-26T22:52:55.418087+0000","last_became_active":"2024-05-26T22:52:54.322776+0000","last_became_peered":"2024-05-26T22:52:54.322776+0000","last_unstale":"2024-05-26T22:52:55.418087+0000","last_undegraded":"2024-05-26T22:52:55.418087+0000","last_fullsized":"2024-05-26T22:52:55.418087+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:07:19.490731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.329516+0000","last_change":"2024-05-26T22:52:54.323520+0000","last_active":"2024-05-26T22:52:55.329516+0000","last_peered":"2024-05-26T22:52:55.329516+0000","last_clean":"2024-05-26T22:52:55.329516+0000","last_became_active":"2024-05-26T22:52:54.323229+0000","last_became_peered":"2024-05-26T22:52:54.323229+0000","last_unstale":"2024-05-26T22:52:55.329516+0000","last_undegraded":"2024-05-26T22:52:55.329516+0000","last_fullsized":"2024-05-26T22:52:55.329516+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:07:21.524347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.321086+0000","last_change":"2024-05-26T22:52:54.331143+0000","last_active":"2024-05-26T22:52:55.321086+0000","last_peered":"2024-05-26T22:52:55.321086+0000","last_clean":"2024-05-26T22:52:55.321086+0000","last_became_active":"2024-05-26T22:52:54.330479+0000","last_became_peered":"2024-05-26T22:52:54.330479+0000","last_unstale":"2024-05-26T22:52:55.321086+0000","last_undegraded":"2024-05-26T22:52:55.321086+0000","last_fullsized":"2024-05-26T22:52:55.321086+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:35:07.002533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.329252+0000","last_change":"2024-05-26T22:52:54.316704+0000","last_active":"2024-05-26T22:52:55.329252+0000","last_peered":"2024-05-26T22:52:55.329252+0000","last_clean":"2024-05-26T22:52:55.329252+0000","last_became_active":"2024-05-26T22:52:54.316016+0000","last_became_peered":"2024-05-26T22:52:54.316016+0000","last_unstale":"2024-05-26T22:52:55.329252+0000","last_undegraded":"2024-05-26T22:52:55.329252+0000","last_fullsized":"2024-05-26T22:52:55.329252+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:31:19.880810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.420048+0000","last_change":"2024-05-26T22:52:54.322929+0000","last_active":"2024-05-26T22:52:55.420048+0000","last_peered":"2024-05-26T22:52:55.420048+0000","last_clean":"2024-05-26T22:52:55.420048+0000","last_became_active":"2024-05-26T22:52:54.322597+0000","last_became_peered":"2024-05-26T22:52:54.322597+0000","last_unstale":"2024-05-26T22:52:55.420048+0000","last_undegraded":"2024-05-26T22:52:55.420048+0000","last_fullsized":"2024-05-26T22:52:55.420048+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T23:07:36.508541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.323618+0000","last_change":"2024-05-26T22:52:54.311423+0000","last_active":"2024-05-26T22:52:55.323618+0000","last_peered":"2024-05-26T22:52:55.323618+0000","last_clean":"2024-05-26T22:52:55.323618+0000","last_became_active":"2024-05-26T22:52:54.310514+0000","last_became_peered":"2024-05-26T22:52:54.310514+0000","last_unstale":"2024-05-26T22:52:55.323618+0000","last_undegraded":"2024-05-26T22:52:55.323618+0000","last_fullsized":"2024-05-26T22:52:55.323618+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:57:21.104575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.418349+0000","last_change":"2024-05-26T22:52:54.329535+0000","last_active":"2024-05-26T22:52:55.418349+0000","last_peered":"2024-05-26T22:52:55.418349+0000","last_clean":"2024-05-26T22:52:55.418349+0000","last_became_active":"2024-05-26T22:52:54.329155+0000","last_became_peered":"2024-05-26T22:52:54.329155+0000","last_unstale":"2024-05-26T22:52:55.418349+0000","last_undegraded":"2024-05-26T22:52:55.418349+0000","last_fullsized":"2024-05-26T22:52:55.418349+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T23:55:59.295714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.320127+0000","last_change":"2024-05-26T22:52:54.314530+0000","last_active":"2024-05-26T22:52:55.320127+0000","last_peered":"2024-05-26T22:52:55.320127+0000","last_clean":"2024-05-26T22:52:55.320127+0000","last_became_active":"2024-05-26T22:52:54.314005+0000","last_became_peered":"2024-05-26T22:52:54.314005+0000","last_unstale":"2024-05-26T22:52:55.320127+0000","last_undegraded":"2024-05-26T22:52:55.320127+0000","last_fullsized":"2024-05-26T22:52:55.320127+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:38:46.157530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.329806+0000","last_change":"2024-05-26T22:52:54.330936+0000","last_active":"2024-05-26T22:52:55.329806+0000","last_peered":"2024-05-26T22:52:55.329806+0000","last_clean":"2024-05-26T22:52:55.329806+0000","last_became_active":"2024-05-26T22:52:54.330650+0000","last_became_peered":"2024-05-26T22:52:54.330650+0000","last_unstale":"2024-05-26T22:52:55.329806+0000","last_undegraded":"2024-05-26T22:52:55.329806+0000","last_fullsized":"2024-05-26T22:52:55.329806+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:02:22.457006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.420480+0000","last_change":"2024-05-26T22:52:54.324464+0000","last_active":"2024-05-26T22:52:55.420480+0000","last_peered":"2024-05-26T22:52:55.420480+0000","last_clean":"2024-05-26T22:52:55.420480+0000","last_became_active":"2024-05-26T22:52:54.324177+0000","last_became_peered":"2024-05-26T22:52:54.324177+0000","last_unstale":"2024-05-26T22:52:55.420480+0000","last_undegraded":"2024-05-26T22:52:55.420480+0000","last_fullsized":"2024-05-26T22:52:55.420480+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:42:27.434611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.420184+0000","last_change":"2024-05-26T22:52:54.319660+0000","last_active":"2024-05-26T22:52:55.420184+0000","last_peered":"2024-05-26T22:52:55.420184+0000","last_clean":"2024-05-26T22:52:55.420184+0000","last_became_active":"2024-05-26T22:52:54.319374+0000","last_became_peered":"2024-05-26T22:52:54.319374+0000","last_unstale":"2024-05-26T22:52:55.420184+0000","last_undegraded":"2024-05-26T22:52:55.420184+0000","last_fullsized":"2024-05-26T22:52:55.420184+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:02:46.912953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.422342+0000","last_change":"2024-05-26T22:52:54.336467+0000","last_active":"2024-05-26T22:52:55.422342+0000","last_peered":"2024-05-26T22:52:55.422342+0000","last_clean":"2024-05-26T22:52:55.422342+0000","last_became_active":"2024-05-26T22:52:54.336117+0000","last_became_peered":"2024-05-26T22:52:54.336117+0000","last_unstale":"2024-05-26T22:52:55.422342+0000","last_undegraded":"2024-05-26T22:52:55.422342+0000","last_fullsized":"2024-05-26T22:52:55.422342+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:06:23.444659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.422588+0000","last_change":"2024-05-26T22:52:54.342506+0000","last_active":"2024-05-26T22:52:55.422588+0000","last_peered":"2024-05-26T22:52:55.422588+0000","last_clean":"2024-05-26T22:52:55.422588+0000","last_became_active":"2024-05-26T22:52:54.341921+0000","last_became_peered":"2024-05-26T22:52:54.341921+0000","last_unstale":"2024-05-26T22:52:55.422588+0000","last_undegraded":"2024-05-26T22:52:55.422588+0000","last_fullsized":"2024-05-26T22:52:55.422588+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:54:14.144555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.420726+0000","last_change":"2024-05-26T22:52:54.336439+0000","last_active":"2024-05-26T22:52:55.420726+0000","last_peered":"2024-05-26T22:52:55.420726+0000","last_clean":"2024-05-26T22:52:55.420726+0000","last_became_active":"2024-05-26T22:52:54.336072+0000","last_became_peered":"2024-05-26T22:52:54.336072+0000","last_unstale":"2024-05-26T22:52:55.420726+0000","last_undegraded":"2024-05-26T22:52:55.420726+0000","last_fullsized":"2024-05-26T22:52:55.420726+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:16:08.984694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.325203+0000","last_change":"2024-05-26T22:52:54.338757+0000","last_active":"2024-05-26T22:52:55.325203+0000","last_peered":"2024-05-26T22:52:55.325203+0000","last_clean":"2024-05-26T22:52:55.325203+0000","last_became_active":"2024-05-26T22:52:54.338265+0000","last_became_peered":"2024-05-26T22:52:54.338265+0000","last_unstale":"2024-05-26T22:52:55.325203+0000","last_undegraded":"2024-05-26T22:52:55.325203+0000","last_fullsized":"2024-05-26T22:52:55.325203+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:12:25.007015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.322659+0000","last_change":"2024-05-26T22:52:54.336337+0000","last_active":"2024-05-26T22:52:55.322659+0000","last_peered":"2024-05-26T22:52:55.322659+0000","last_clean":"2024-05-26T22:52:55.322659+0000","last_became_active":"2024-05-26T22:52:54.336105+0000","last_became_peered":"2024-05-26T22:52:54.336105+0000","last_unstale":"2024-05-26T22:52:55.322659+0000","last_undegraded":"2024-05-26T22:52:55.322659+0000","last_fullsized":"2024-05-26T22:52:55.322659+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:46:56.093316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.420922+0000","last_change":"2024-05-26T22:52:54.327417+0000","last_active":"2024-05-26T22:52:55.420922+0000","last_peered":"2024-05-26T22:52:55.420922+0000","last_clean":"2024-05-26T22:52:55.420922+0000","last_became_active":"2024-05-26T22:52:54.324876+0000","last_became_peered":"2024-05-26T22:52:54.324876+0000","last_unstale":"2024-05-26T22:52:55.420922+0000","last_undegraded":"2024-05-26T22:52:55.420922+0000","last_fullsized":"2024-05-26T22:52:55.420922+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T23:09:02.321433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.324148+0000","last_change":"2024-05-26T22:52:54.330638+0000","last_active":"2024-05-26T22:52:55.324148+0000","last_peered":"2024-05-26T22:52:55.324148+0000","last_clean":"2024-05-26T22:52:55.324148+0000","last_became_active":"2024-05-26T22:52:54.330365+0000","last_became_peered":"2024-05-26T22:52:54.330365+0000","last_unstale":"2024-05-26T22:52:55.324148+0000","last_undegraded":"2024-05-26T22:52:55.324148+0000","last_fullsized":"2024-05-26T22:52:55.324148+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:26:58.318248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.325068+0000","last_change":"2024-05-26T22:52:54.326077+0000","last_active":"2024-05-26T22:52:55.325068+0000","last_peered":"2024-05-26T22:52:55.325068+0000","last_clean":"2024-05-26T22:52:55.325068+0000","last_became_active":"2024-05-26T22:52:54.325512+0000","last_became_peered":"2024-05-26T22:52:54.325512+0000","last_unstale":"2024-05-26T22:52:55.325068+0000","last_undegraded":"2024-05-26T22:52:55.325068+0000","last_fullsized":"2024-05-26T22:52:55.325068+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T23:09:02.321433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.324975+0000","last_change":"2024-05-26T22:52:54.328874+0000","last_active":"2024-05-26T22:52:55.324975+0000","last_peered":"2024-05-26T22:52:55.324975+0000","last_clean":"2024-05-26T22:52:55.324975+0000","last_became_active":"2024-05-26T22:52:54.326585+0000","last_became_peered":"2024-05-26T22:52:54.326585+0000","last_unstale":"2024-05-26T22:52:55.324975+0000","last_undegraded":"2024-05-26T22:52:55.324975+0000","last_fullsized":"2024-05-26T22:52:55.324975+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:10:56.957932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.319000+0000","last_change":"2024-05-26T22:52:54.315875+0000","last_active":"2024-05-26T22:52:55.319000+0000","last_peered":"2024-05-26T22:52:55.319000+0000","last_clean":"2024-05-26T22:52:55.319000+0000","last_became_active":"2024-05-26T22:52:54.308338+0000","last_became_peered":"2024-05-26T22:52:54.308338+0000","last_unstale":"2024-05-26T22:52:55.319000+0000","last_undegraded":"2024-05-26T22:52:55.319000+0000","last_fullsized":"2024-05-26T22:52:55.319000+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T23:07:36.508541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.423330+0000","last_change":"2024-05-26T22:52:54.342602+0000","last_active":"2024-05-26T22:52:55.423330+0000","last_peered":"2024-05-26T22:52:55.423330+0000","last_clean":"2024-05-26T22:52:55.423330+0000","last_became_active":"2024-05-26T22:52:54.335864+0000","last_became_peered":"2024-05-26T22:52:54.335864+0000","last_unstale":"2024-05-26T22:52:55.423330+0000","last_undegraded":"2024-05-26T22:52:55.423330+0000","last_fullsized":"2024-05-26T22:52:55.423330+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:07:21.524347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"68'4","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.423868+0000","last_change":"2024-05-26T22:52:53.320118+0000","last_active":"2024-05-26T22:52:55.423868+0000","last_peered":"2024-05-26T22:52:55.423868+0000","last_clean":"2024-05-26T22:52:55.423868+0000","last_became_active":"2024-05-26T22:52:53.319776+0000","last_became_peered":"2024-05-26T22:52:53.319776+0000","last_unstale":"2024-05-26T22:52:55.423868+0000","last_undegraded":"2024-05-26T22:52:55.423868+0000","last_fullsized":"2024-05-26T22:52:55.423868+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_clean_scrub_stamp":"2024-05-26T22:52:52.250515+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:01:21.968664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.323819+0000","last_change":"2024-05-26T22:52:54.312192+0000","last_active":"2024-05-26T22:52:55.323819+0000","last_peered":"2024-05-26T22:52:55.323819+0000","last_clean":"2024-05-26T22:52:55.323819+0000","last_became_active":"2024-05-26T22:52:54.310169+0000","last_became_peered":"2024-05-26T22:52:54.310169+0000","last_unstale":"2024-05-26T22:52:55.323819+0000","last_undegraded":"2024-05-26T22:52:55.323819+0000","last_fullsized":"2024-05-26T22:52:55.323819+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:30:04.180895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"68'1","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.330064+0000","last_change":"2024-05-26T22:52:53.311353+0000","last_active":"2024-05-26T22:52:55.330064+0000","last_peered":"2024-05-26T22:52:55.330064+0000","last_clean":"2024-05-26T22:52:55.330064+0000","last_became_active":"2024-05-26T22:52:53.311100+0000","last_became_peered":"2024-05-26T22:52:53.311100+0000","last_unstale":"2024-05-26T22:52:55.330064+0000","last_undegraded":"2024-05-26T22:52:55.330064+0000","last_fullsized":"2024-05-26T22:52:55.330064+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_clean_scrub_stamp":"2024-05-26T22:52:52.250515+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:54:08.575904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.319834+0000","last_change":"2024-05-26T22:52:54.327613+0000","last_active":"2024-05-26T22:52:55.319834+0000","last_peered":"2024-05-26T22:52:55.319834+0000","last_clean":"2024-05-26T22:52:55.319834+0000","last_became_active":"2024-05-26T22:52:54.322032+0000","last_became_peered":"2024-05-26T22:52:54.322032+0000","last_unstale":"2024-05-26T22:52:55.319834+0000","last_undegraded":"2024-05-26T22:52:55.319834+0000","last_fullsized":"2024-05-26T22:52:55.319834+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:54:09.591587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"68'2","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.420768+0000","last_change":"2024-05-26T22:52:53.322733+0000","last_active":"2024-05-26T22:52:55.420768+0000","last_peered":"2024-05-26T22:52:55.420768+0000","last_clean":"2024-05-26T22:52:55.420768+0000","last_became_active":"2024-05-26T22:52:53.321828+0000","last_became_peered":"2024-05-26T22:52:53.321828+0000","last_unstale":"2024-05-26T22:52:55.420768+0000","last_undegraded":"2024-05-26T22:52:55.420768+0000","last_fullsized":"2024-05-26T22:52:55.420768+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_clean_scrub_stamp":"2024-05-26T22:52:52.250515+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:13:44.543717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.423286+0000","last_change":"2024-05-26T22:52:54.326633+0000","last_active":"2024-05-26T22:52:55.423286+0000","last_peered":"2024-05-26T22:52:55.423286+0000","last_clean":"2024-05-26T22:52:55.423286+0000","last_became_active":"2024-05-26T22:52:54.326331+0000","last_became_peered":"2024-05-26T22:52:54.326331+0000","last_unstale":"2024-05-26T22:52:55.423286+0000","last_undegraded":"2024-05-26T22:52:55.423286+0000","last_fullsized":"2024-05-26T22:52:55.423286+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:02:43.306586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"68'2","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.319986+0000","last_change":"2024-05-26T22:52:53.311637+0000","last_active":"2024-05-26T22:52:55.319986+0000","last_peered":"2024-05-26T22:52:55.319986+0000","last_clean":"2024-05-26T22:52:55.319986+0000","last_became_active":"2024-05-26T22:52:53.311349+0000","last_became_peered":"2024-05-26T22:52:53.311349+0000","last_unstale":"2024-05-26T22:52:55.319986+0000","last_undegraded":"2024-05-26T22:52:55.319986+0000","last_fullsized":"2024-05-26T22:52:55.319986+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_clean_scrub_stamp":"2024-05-26T22:52:52.250515+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:34:41.388075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.324967+0000","last_change":"2024-05-26T22:52:54.330214+0000","last_active":"2024-05-26T22:52:55.324967+0000","last_peered":"2024-05-26T22:52:55.324967+0000","last_clean":"2024-05-26T22:52:55.324967+0000","last_became_active":"2024-05-26T22:52:54.329873+0000","last_became_peered":"2024-05-26T22:52:54.329873+0000","last_unstale":"2024-05-26T22:52:55.324967+0000","last_undegraded":"2024-05-26T22:52:55.324967+0000","last_fullsized":"2024-05-26T22:52:55.324967+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:38:46.157530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"68'1","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.324314+0000","last_change":"2024-05-26T22:52:53.315524+0000","last_active":"2024-05-26T22:52:55.324314+0000","last_peered":"2024-05-26T22:52:55.324314+0000","last_clean":"2024-05-26T22:52:55.324314+0000","last_became_active":"2024-05-26T22:52:53.315217+0000","last_became_peered":"2024-05-26T22:52:53.315217+0000","last_unstale":"2024-05-26T22:52:55.324314+0000","last_undegraded":"2024-05-26T22:52:55.324314+0000","last_fullsized":"2024-05-26T22:52:55.324314+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_clean_scrub_stamp":"2024-05-26T22:52:52.250515+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:37:23.559093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.420916+0000","last_change":"2024-05-26T22:52:54.328875+0000","last_active":"2024-05-26T22:52:55.420916+0000","last_peered":"2024-05-26T22:52:55.420916+0000","last_clean":"2024-05-26T22:52:55.420916+0000","last_became_active":"2024-05-26T22:52:54.328287+0000","last_became_peered":"2024-05-26T22:52:54.328287+0000","last_unstale":"2024-05-26T22:52:55.420916+0000","last_undegraded":"2024-05-26T22:52:55.420916+0000","last_fullsized":"2024-05-26T22:52:55.420916+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:10:56.957932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"68'1","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.319324+0000","last_change":"2024-05-26T22:52:53.309674+0000","last_active":"2024-05-26T22:52:55.319324+0000","last_peered":"2024-05-26T22:52:55.319324+0000","last_clean":"2024-05-26T22:52:55.319324+0000","last_became_active":"2024-05-26T22:52:53.308165+0000","last_became_peered":"2024-05-26T22:52:53.308165+0000","last_unstale":"2024-05-26T22:52:55.319324+0000","last_undegraded":"2024-05-26T22:52:55.319324+0000","last_fullsized":"2024-05-26T22:52:55.319324+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_clean_scrub_stamp":"2024-05-26T22:52:52.250515+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:49:12.112441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.319766+0000","last_change":"2024-05-26T22:52:54.328030+0000","last_active":"2024-05-26T22:52:55.319766+0000","last_peered":"2024-05-26T22:52:55.319766+0000","last_clean":"2024-05-26T22:52:55.319766+0000","last_became_active":"2024-05-26T22:52:54.327705+0000","last_became_peered":"2024-05-26T22:52:54.327705+0000","last_unstale":"2024-05-26T22:52:55.319766+0000","last_undegraded":"2024-05-26T22:52:55.319766+0000","last_fullsized":"2024-05-26T22:52:55.319766+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:51:42.664856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.417932+0000","last_change":"2024-05-26T22:52:53.316531+0000","last_active":"2024-05-26T22:52:55.417932+0000","last_peered":"2024-05-26T22:52:55.417932+0000","last_clean":"2024-05-26T22:52:55.417932+0000","last_became_active":"2024-05-26T22:52:53.316205+0000","last_became_peered":"2024-05-26T22:52:53.316205+0000","last_unstale":"2024-05-26T22:52:55.417932+0000","last_undegraded":"2024-05-26T22:52:55.417932+0000","last_fullsized":"2024-05-26T22:52:55.417932+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_clean_scrub_stamp":"2024-05-26T22:52:52.250515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:23:33.311153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.329473+0000","last_change":"2024-05-26T22:52:54.334339+0000","last_active":"2024-05-26T22:52:55.329473+0000","last_peered":"2024-05-26T22:52:55.329473+0000","last_clean":"2024-05-26T22:52:55.329473+0000","last_became_active":"2024-05-26T22:52:54.333870+0000","last_became_peered":"2024-05-26T22:52:54.333870+0000","last_unstale":"2024-05-26T22:52:55.329473+0000","last_undegraded":"2024-05-26T22:52:55.329473+0000","last_fullsized":"2024-05-26T22:52:55.329473+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:02:43.306586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.319768+0000","last_change":"2024-05-26T22:52:53.310821+0000","last_active":"2024-05-26T22:52:55.319768+0000","last_peered":"2024-05-26T22:52:55.319768+0000","last_clean":"2024-05-26T22:52:55.319768+0000","last_became_active":"2024-05-26T22:52:53.310373+0000","last_became_peered":"2024-05-26T22:52:53.310373+0000","last_unstale":"2024-05-26T22:52:55.319768+0000","last_undegraded":"2024-05-26T22:52:55.319768+0000","last_fullsized":"2024-05-26T22:52:55.319768+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_clean_scrub_stamp":"2024-05-26T22:52:52.250515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:32:23.515533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.325456+0000","last_change":"2024-05-26T22:52:54.335108+0000","last_active":"2024-05-26T22:52:55.325456+0000","last_peered":"2024-05-26T22:52:55.325456+0000","last_clean":"2024-05-26T22:52:55.325456+0000","last_became_active":"2024-05-26T22:52:54.334602+0000","last_became_peered":"2024-05-26T22:52:54.334602+0000","last_unstale":"2024-05-26T22:52:55.325456+0000","last_undegraded":"2024-05-26T22:52:55.325456+0000","last_fullsized":"2024-05-26T22:52:55.325456+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:54:14.144555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.420978+0000","last_change":"2024-05-26T22:52:53.320502+0000","last_active":"2024-05-26T22:52:55.420978+0000","last_peered":"2024-05-26T22:52:55.420978+0000","last_clean":"2024-05-26T22:52:55.420978+0000","last_became_active":"2024-05-26T22:52:53.320028+0000","last_became_peered":"2024-05-26T22:52:53.320028+0000","last_unstale":"2024-05-26T22:52:55.420978+0000","last_undegraded":"2024-05-26T22:52:55.420978+0000","last_fullsized":"2024-05-26T22:52:55.420978+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_clean_scrub_stamp":"2024-05-26T22:52:52.250515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T23:55:58.280032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.423570+0000","last_change":"2024-05-26T22:52:54.336216+0000","last_active":"2024-05-26T22:52:55.423570+0000","last_peered":"2024-05-26T22:52:55.423570+0000","last_clean":"2024-05-26T22:52:55.423570+0000","last_became_active":"2024-05-26T22:52:54.335891+0000","last_became_peered":"2024-05-26T22:52:54.335891+0000","last_unstale":"2024-05-26T22:52:55.423570+0000","last_undegraded":"2024-05-26T22:52:55.423570+0000","last_fullsized":"2024-05-26T22:52:55.423570+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:22:58.350242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.320749+0000","last_change":"2024-05-26T22:52:53.318985+0000","last_active":"2024-05-26T22:52:55.320749+0000","last_peered":"2024-05-26T22:52:55.320749+0000","last_clean":"2024-05-26T22:52:55.320749+0000","last_became_active":"2024-05-26T22:52:53.318620+0000","last_became_peered":"2024-05-26T22:52:53.318620+0000","last_unstale":"2024-05-26T22:52:55.320749+0000","last_undegraded":"2024-05-26T22:52:55.320749+0000","last_fullsized":"2024-05-26T22:52:55.320749+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_clean_scrub_stamp":"2024-05-26T22:52:52.250515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:03:52.876316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.319940+0000","last_change":"2024-05-26T22:52:54.335411+0000","last_active":"2024-05-26T22:52:55.319940+0000","last_peered":"2024-05-26T22:52:55.319940+0000","last_clean":"2024-05-26T22:52:55.319940+0000","last_became_active":"2024-05-26T22:52:54.335093+0000","last_became_peered":"2024-05-26T22:52:54.335093+0000","last_unstale":"2024-05-26T22:52:55.319940+0000","last_undegraded":"2024-05-26T22:52:55.319940+0000","last_fullsized":"2024-05-26T22:52:55.319940+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:16:08.984694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"68'3","reported_seq":17,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.322769+0000","last_change":"2024-05-26T22:52:53.311847+0000","last_active":"2024-05-26T22:52:55.322769+0000","last_peered":"2024-05-26T22:52:55.322769+0000","last_clean":"2024-05-26T22:52:55.322769+0000","last_became_active":"2024-05-26T22:52:53.311508+0000","last_became_peered":"2024-05-26T22:52:53.311508+0000","last_unstale":"2024-05-26T22:52:55.322769+0000","last_undegraded":"2024-05-26T22:52:55.322769+0000","last_fullsized":"2024-05-26T22:52:55.322769+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_clean_scrub_stamp":"2024-05-26T22:52:52.250515+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:40:45.060296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.321245+0000","last_change":"2024-05-26T22:52:54.317305+0000","last_active":"2024-05-26T22:52:55.321245+0000","last_peered":"2024-05-26T22:52:55.321245+0000","last_clean":"2024-05-26T22:52:55.321245+0000","last_became_active":"2024-05-26T22:52:54.316091+0000","last_became_peered":"2024-05-26T22:52:54.316091+0000","last_unstale":"2024-05-26T22:52:55.321245+0000","last_undegraded":"2024-05-26T22:52:55.321245+0000","last_fullsized":"2024-05-26T22:52:55.321245+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:54:46.300450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"68'6","reported_seq":20,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.418428+0000","last_change":"2024-05-26T22:52:53.310828+0000","last_active":"2024-05-26T22:52:55.418428+0000","last_peered":"2024-05-26T22:52:55.418428+0000","last_clean":"2024-05-26T22:52:55.418428+0000","last_became_active":"2024-05-26T22:52:53.310528+0000","last_became_peered":"2024-05-26T22:52:53.310528+0000","last_unstale":"2024-05-26T22:52:55.418428+0000","last_undegraded":"2024-05-26T22:52:55.418428+0000","last_fullsized":"2024-05-26T22:52:55.418428+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_clean_scrub_stamp":"2024-05-26T22:52:52.250515+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:50:13.373742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.422281+0000","last_change":"2024-05-26T22:52:54.342773+0000","last_active":"2024-05-26T22:52:55.422281+0000","last_peered":"2024-05-26T22:52:55.422281+0000","last_clean":"2024-05-26T22:52:55.422281+0000","last_became_active":"2024-05-26T22:52:54.333675+0000","last_became_peered":"2024-05-26T22:52:54.333675+0000","last_unstale":"2024-05-26T22:52:55.422281+0000","last_undegraded":"2024-05-26T22:52:55.422281+0000","last_fullsized":"2024-05-26T22:52:55.422281+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:41:38.018569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"68'2","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.418085+0000","last_change":"2024-05-26T22:52:53.304451+0000","last_active":"2024-05-26T22:52:55.418085+0000","last_peered":"2024-05-26T22:52:55.418085+0000","last_clean":"2024-05-26T22:52:55.418085+0000","last_became_active":"2024-05-26T22:52:53.304007+0000","last_became_peered":"2024-05-26T22:52:53.304007+0000","last_unstale":"2024-05-26T22:52:55.418085+0000","last_undegraded":"2024-05-26T22:52:55.418085+0000","last_fullsized":"2024-05-26T22:52:55.418085+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_clean_scrub_stamp":"2024-05-26T22:52:52.250515+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:42:34.112776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.325229+0000","last_change":"2024-05-26T22:52:54.331458+0000","last_active":"2024-05-26T22:52:55.325229+0000","last_peered":"2024-05-26T22:52:55.325229+0000","last_clean":"2024-05-26T22:52:55.325229+0000","last_became_active":"2024-05-26T22:52:54.331166+0000","last_became_peered":"2024-05-26T22:52:54.331166+0000","last_unstale":"2024-05-26T22:52:55.325229+0000","last_undegraded":"2024-05-26T22:52:55.325229+0000","last_fullsized":"2024-05-26T22:52:55.325229+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:41:38.018569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.319449+0000","last_change":"2024-05-26T22:52:53.312297+0000","last_active":"2024-05-26T22:52:55.319449+0000","last_peered":"2024-05-26T22:52:55.319449+0000","last_clean":"2024-05-26T22:52:55.319449+0000","last_became_active":"2024-05-26T22:52:53.311821+0000","last_became_peered":"2024-05-26T22:52:53.311821+0000","last_unstale":"2024-05-26T22:52:55.319449+0000","last_undegraded":"2024-05-26T22:52:55.319449+0000","last_fullsized":"2024-05-26T22:52:55.319449+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_clean_scrub_stamp":"2024-05-26T22:52:52.250515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:08:29.076391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.328976+0000","last_change":"2024-05-26T22:52:54.324508+0000","last_active":"2024-05-26T22:52:55.328976+0000","last_peered":"2024-05-26T22:52:55.328976+0000","last_clean":"2024-05-26T22:52:55.328976+0000","last_became_active":"2024-05-26T22:52:54.324201+0000","last_became_peered":"2024-05-26T22:52:54.324201+0000","last_unstale":"2024-05-26T22:52:55.328976+0000","last_undegraded":"2024-05-26T22:52:55.328976+0000","last_fullsized":"2024-05-26T22:52:55.328976+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:54:14.144555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.324739+0000","last_change":"2024-05-26T22:52:53.314647+0000","last_active":"2024-05-26T22:52:55.324739+0000","last_peered":"2024-05-26T22:52:55.324739+0000","last_clean":"2024-05-26T22:52:55.324739+0000","last_became_active":"2024-05-26T22:52:53.314189+0000","last_became_peered":"2024-05-26T22:52:53.314189+0000","last_unstale":"2024-05-26T22:52:55.324739+0000","last_undegraded":"2024-05-26T22:52:55.324739+0000","last_fullsized":"2024-05-26T22:52:55.324739+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_clean_scrub_stamp":"2024-05-26T22:52:52.250515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T22:54:25.373084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.322202+0000","last_change":"2024-05-26T22:52:54.313588+0000","last_active":"2024-05-26T22:52:55.322202+0000","last_peered":"2024-05-26T22:52:55.322202+0000","last_clean":"2024-05-26T22:52:55.322202+0000","last_became_active":"2024-05-26T22:52:54.308097+0000","last_became_peered":"2024-05-26T22:52:54.308097+0000","last_unstale":"2024-05-26T22:52:55.322202+0000","last_undegraded":"2024-05-26T22:52:55.322202+0000","last_fullsized":"2024-05-26T22:52:55.322202+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T23:27:03.395614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"68'8","reported_seq":22,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.420383+0000","last_change":"2024-05-26T22:52:53.317131+0000","last_active":"2024-05-26T22:52:55.420383+0000","last_peered":"2024-05-26T22:52:55.420383+0000","last_clean":"2024-05-26T22:52:55.420383+0000","last_became_active":"2024-05-26T22:52:53.316753+0000","last_became_peered":"2024-05-26T22:52:53.316753+0000","last_unstale":"2024-05-26T22:52:55.420383+0000","last_undegraded":"2024-05-26T22:52:55.420383+0000","last_fullsized":"2024-05-26T22:52:55.420383+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_clean_scrub_stamp":"2024-05-26T22:52:52.250515+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:07:18.475048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.420849+0000","last_change":"2024-05-26T22:52:54.325743+0000","last_active":"2024-05-26T22:52:55.420849+0000","last_peered":"2024-05-26T22:52:55.420849+0000","last_clean":"2024-05-26T22:52:55.420849+0000","last_became_active":"2024-05-26T22:52:54.325446+0000","last_became_peered":"2024-05-26T22:52:54.325446+0000","last_unstale":"2024-05-26T22:52:55.420849+0000","last_undegraded":"2024-05-26T22:52:55.420849+0000","last_fullsized":"2024-05-26T22:52:55.420849+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:38:46.157530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"68'1","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.320875+0000","last_change":"2024-05-26T22:52:53.314351+0000","last_active":"2024-05-26T22:52:55.320875+0000","last_peered":"2024-05-26T22:52:55.320875+0000","last_clean":"2024-05-26T22:52:55.320875+0000","last_became_active":"2024-05-26T22:52:53.314071+0000","last_became_peered":"2024-05-26T22:52:53.314071+0000","last_unstale":"2024-05-26T22:52:55.320875+0000","last_undegraded":"2024-05-26T22:52:55.320875+0000","last_fullsized":"2024-05-26T22:52:55.320875+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_clean_scrub_stamp":"2024-05-26T22:52:52.250515+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:42:34.112776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.423758+0000","last_change":"2024-05-26T22:52:54.334664+0000","last_active":"2024-05-26T22:52:55.423758+0000","last_peered":"2024-05-26T22:52:55.423758+0000","last_clean":"2024-05-26T22:52:55.423758+0000","last_became_active":"2024-05-26T22:52:54.334313+0000","last_became_peered":"2024-05-26T22:52:54.334313+0000","last_unstale":"2024-05-26T22:52:55.423758+0000","last_undegraded":"2024-05-26T22:52:55.423758+0000","last_fullsized":"2024-05-26T22:52:55.423758+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:16:35.223557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"68'2","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.420998+0000","last_change":"2024-05-26T22:52:53.320585+0000","last_active":"2024-05-26T22:52:55.420998+0000","last_peered":"2024-05-26T22:52:55.420998+0000","last_clean":"2024-05-26T22:52:55.420998+0000","last_became_active":"2024-05-26T22:52:53.319296+0000","last_became_peered":"2024-05-26T22:52:53.319296+0000","last_unstale":"2024-05-26T22:52:55.420998+0000","last_undegraded":"2024-05-26T22:52:55.420998+0000","last_fullsized":"2024-05-26T22:52:55.420998+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_clean_scrub_stamp":"2024-05-26T22:52:52.250515+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:20:15.243688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.418324+0000","last_change":"2024-05-26T22:52:54.322591+0000","last_active":"2024-05-26T22:52:55.418324+0000","last_peered":"2024-05-26T22:52:55.418324+0000","last_clean":"2024-05-26T22:52:55.418324+0000","last_became_active":"2024-05-26T22:52:54.322307+0000","last_became_peered":"2024-05-26T22:52:54.322307+0000","last_unstale":"2024-05-26T22:52:55.418324+0000","last_undegraded":"2024-05-26T22:52:55.418324+0000","last_fullsized":"2024-05-26T22:52:55.418324+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:20:16.259371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"68'1","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.320401+0000","last_change":"2024-05-26T22:52:53.317832+0000","last_active":"2024-05-26T22:52:55.320401+0000","last_peered":"2024-05-26T22:52:55.320401+0000","last_clean":"2024-05-26T22:52:55.320401+0000","last_became_active":"2024-05-26T22:52:53.317432+0000","last_became_peered":"2024-05-26T22:52:53.317432+0000","last_unstale":"2024-05-26T22:52:55.320401+0000","last_undegraded":"2024-05-26T22:52:55.320401+0000","last_fullsized":"2024-05-26T22:52:55.320401+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_clean_scrub_stamp":"2024-05-26T22:52:52.250515+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:23:54.826711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.322736+0000","last_change":"2024-05-26T22:52:54.326502+0000","last_active":"2024-05-26T22:52:55.322736+0000","last_peered":"2024-05-26T22:52:55.322736+0000","last_clean":"2024-05-26T22:52:55.322736+0000","last_became_active":"2024-05-26T22:52:54.326190+0000","last_became_peered":"2024-05-26T22:52:54.326190+0000","last_unstale":"2024-05-26T22:52:55.322736+0000","last_undegraded":"2024-05-26T22:52:55.322736+0000","last_fullsized":"2024-05-26T22:52:55.322736+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:33:08.243749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.422477+0000","last_change":"2024-05-26T22:52:53.322332+0000","last_active":"2024-05-26T22:52:55.422477+0000","last_peered":"2024-05-26T22:52:55.422477+0000","last_clean":"2024-05-26T22:52:55.422477+0000","last_became_active":"2024-05-26T22:52:53.321697+0000","last_became_peered":"2024-05-26T22:52:53.321697+0000","last_unstale":"2024-05-26T22:52:55.422477+0000","last_undegraded":"2024-05-26T22:52:55.422477+0000","last_fullsized":"2024-05-26T22:52:55.422477+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_clean_scrub_stamp":"2024-05-26T22:52:52.250515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:25:01.606093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.324403+0000","last_change":"2024-05-26T22:52:54.320200+0000","last_active":"2024-05-26T22:52:55.324403+0000","last_peered":"2024-05-26T22:52:55.324403+0000","last_clean":"2024-05-26T22:52:55.324403+0000","last_became_active":"2024-05-26T22:52:54.319640+0000","last_became_peered":"2024-05-26T22:52:54.319640+0000","last_unstale":"2024-05-26T22:52:55.324403+0000","last_undegraded":"2024-05-26T22:52:55.324403+0000","last_fullsized":"2024-05-26T22:52:55.324403+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T23:57:12.586083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.319525+0000","last_change":"2024-05-26T22:52:53.308496+0000","last_active":"2024-05-26T22:52:55.319525+0000","last_peered":"2024-05-26T22:52:55.319525+0000","last_clean":"2024-05-26T22:52:55.319525+0000","last_became_active":"2024-05-26T22:52:53.307650+0000","last_became_peered":"2024-05-26T22:52:53.307650+0000","last_unstale":"2024-05-26T22:52:55.319525+0000","last_undegraded":"2024-05-26T22:52:55.319525+0000","last_fullsized":"2024-05-26T22:52:55.319525+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_clean_scrub_stamp":"2024-05-26T22:52:52.250515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:45:51.874305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.324169+0000","last_change":"2024-05-26T22:52:54.318210+0000","last_active":"2024-05-26T22:52:55.324169+0000","last_peered":"2024-05-26T22:52:55.324169+0000","last_clean":"2024-05-26T22:52:55.324169+0000","last_became_active":"2024-05-26T22:52:54.317916+0000","last_became_peered":"2024-05-26T22:52:54.317916+0000","last_unstale":"2024-05-26T22:52:55.324169+0000","last_undegraded":"2024-05-26T22:52:55.324169+0000","last_fullsized":"2024-05-26T22:52:55.324169+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T23:48:12.418685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.324990+0000","last_change":"2024-05-26T22:52:53.312501+0000","last_active":"2024-05-26T22:52:55.324990+0000","last_peered":"2024-05-26T22:52:55.324990+0000","last_clean":"2024-05-26T22:52:55.324990+0000","last_became_active":"2024-05-26T22:52:53.312214+0000","last_became_peered":"2024-05-26T22:52:53.312214+0000","last_unstale":"2024-05-26T22:52:55.324990+0000","last_undegraded":"2024-05-26T22:52:55.324990+0000","last_fullsized":"2024-05-26T22:52:55.324990+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_clean_scrub_stamp":"2024-05-26T22:52:52.250515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:56:54.130267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.323942+0000","last_change":"2024-05-26T22:52:54.305899+0000","last_active":"2024-05-26T22:52:55.323942+0000","last_peered":"2024-05-26T22:52:55.323942+0000","last_clean":"2024-05-26T22:52:55.323942+0000","last_became_active":"2024-05-26T22:52:54.305336+0000","last_became_peered":"2024-05-26T22:52:54.305336+0000","last_unstale":"2024-05-26T22:52:55.323942+0000","last_undegraded":"2024-05-26T22:52:55.323942+0000","last_fullsized":"2024-05-26T22:52:55.323942+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:33:56.867587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.328947+0000","last_change":"2024-05-26T22:52:53.311245+0000","last_active":"2024-05-26T22:52:55.328947+0000","last_peered":"2024-05-26T22:52:55.328947+0000","last_clean":"2024-05-26T22:52:55.328947+0000","last_became_active":"2024-05-26T22:52:53.310959+0000","last_became_peered":"2024-05-26T22:52:53.310959+0000","last_unstale":"2024-05-26T22:52:55.328947+0000","last_undegraded":"2024-05-26T22:52:55.328947+0000","last_fullsized":"2024-05-26T22:52:55.328947+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_clean_scrub_stamp":"2024-05-26T22:52:52.250515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:16:07.969012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.329984+0000","last_change":"2024-05-26T22:52:54.318907+0000","last_active":"2024-05-26T22:52:55.329984+0000","last_peered":"2024-05-26T22:52:55.329984+0000","last_clean":"2024-05-26T22:52:55.329984+0000","last_became_active":"2024-05-26T22:52:54.318626+0000","last_became_peered":"2024-05-26T22:52:54.318626+0000","last_unstale":"2024-05-26T22:52:55.329984+0000","last_undegraded":"2024-05-26T22:52:55.329984+0000","last_fullsized":"2024-05-26T22:52:55.329984+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T22:55:42.930355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"68'2","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.320893+0000","last_change":"2024-05-26T22:52:53.306902+0000","last_active":"2024-05-26T22:52:55.320893+0000","last_peered":"2024-05-26T22:52:55.320893+0000","last_clean":"2024-05-26T22:52:55.320893+0000","last_became_active":"2024-05-26T22:52:53.306509+0000","last_became_peered":"2024-05-26T22:52:53.306509+0000","last_unstale":"2024-05-26T22:52:55.320893+0000","last_undegraded":"2024-05-26T22:52:55.320893+0000","last_fullsized":"2024-05-26T22:52:55.320893+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_clean_scrub_stamp":"2024-05-26T22:52:52.250515+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:05:59.630328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.325223+0000","last_change":"2024-05-26T22:52:54.326978+0000","last_active":"2024-05-26T22:52:55.325223+0000","last_peered":"2024-05-26T22:52:55.325223+0000","last_clean":"2024-05-26T22:52:55.325223+0000","last_became_active":"2024-05-26T22:52:54.324166+0000","last_became_peered":"2024-05-26T22:52:54.324166+0000","last_unstale":"2024-05-26T22:52:55.325223+0000","last_undegraded":"2024-05-26T22:52:55.325223+0000","last_fullsized":"2024-05-26T22:52:55.325223+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:06:23.444659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"69'1","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.328217+0000","last_change":"2024-05-26T22:52:53.311315+0000","last_active":"2024-05-26T22:52:55.328217+0000","last_peered":"2024-05-26T22:52:55.328217+0000","last_clean":"2024-05-26T22:52:55.328217+0000","last_became_active":"2024-05-26T22:52:53.310978+0000","last_became_peered":"2024-05-26T22:52:53.310978+0000","last_unstale":"2024-05-26T22:52:55.328217+0000","last_undegraded":"2024-05-26T22:52:55.328217+0000","last_fullsized":"2024-05-26T22:52:55.328217+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_clean_scrub_stamp":"2024-05-26T22:52:52.250515+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:49:26.486812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.422767+0000","last_change":"2024-05-26T22:52:54.340034+0000","last_active":"2024-05-26T22:52:55.422767+0000","last_peered":"2024-05-26T22:52:55.422767+0000","last_clean":"2024-05-26T22:52:55.422767+0000","last_became_active":"2024-05-26T22:52:54.336325+0000","last_became_peered":"2024-05-26T22:52:54.336325+0000","last_unstale":"2024-05-26T22:52:55.422767+0000","last_undegraded":"2024-05-26T22:52:55.422767+0000","last_fullsized":"2024-05-26T22:52:55.422767+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:31:19.880810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.321599+0000","last_change":"2024-05-26T22:52:53.310841+0000","last_active":"2024-05-26T22:52:55.321599+0000","last_peered":"2024-05-26T22:52:55.321599+0000","last_clean":"2024-05-26T22:52:55.321599+0000","last_became_active":"2024-05-26T22:52:53.310346+0000","last_became_peered":"2024-05-26T22:52:53.310346+0000","last_unstale":"2024-05-26T22:52:55.321599+0000","last_undegraded":"2024-05-26T22:52:55.321599+0000","last_fullsized":"2024-05-26T22:52:55.321599+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_clean_scrub_stamp":"2024-05-26T22:52:52.250515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:06:45.944808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.423725+0000","last_change":"2024-05-26T22:52:54.337947+0000","last_active":"2024-05-26T22:52:55.423725+0000","last_peered":"2024-05-26T22:52:55.423725+0000","last_clean":"2024-05-26T22:52:55.423725+0000","last_became_active":"2024-05-26T22:52:54.336813+0000","last_became_peered":"2024-05-26T22:52:54.336813+0000","last_unstale":"2024-05-26T22:52:55.423725+0000","last_undegraded":"2024-05-26T22:52:55.423725+0000","last_fullsized":"2024-05-26T22:52:55.423725+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:13:23.082826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.329736+0000","last_change":"2024-05-26T22:52:53.316773+0000","last_active":"2024-05-26T22:52:55.329736+0000","last_peered":"2024-05-26T22:52:55.329736+0000","last_clean":"2024-05-26T22:52:55.329736+0000","last_became_active":"2024-05-26T22:52:53.316482+0000","last_became_peered":"2024-05-26T22:52:53.316482+0000","last_unstale":"2024-05-26T22:52:55.329736+0000","last_undegraded":"2024-05-26T22:52:55.329736+0000","last_fullsized":"2024-05-26T22:52:55.329736+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_clean_scrub_stamp":"2024-05-26T22:52:52.250515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:42:26.418928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.322272+0000","last_change":"2024-05-26T22:52:54.310732+0000","last_active":"2024-05-26T22:52:55.322272+0000","last_peered":"2024-05-26T22:52:55.322272+0000","last_clean":"2024-05-26T22:52:55.322272+0000","last_became_active":"2024-05-26T22:52:54.308845+0000","last_became_peered":"2024-05-26T22:52:54.308845+0000","last_unstale":"2024-05-26T22:52:55.322272+0000","last_undegraded":"2024-05-26T22:52:55.322272+0000","last_fullsized":"2024-05-26T22:52:55.322272+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:53:49.116433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.320462+0000","last_change":"2024-05-26T22:52:53.317077+0000","last_active":"2024-05-26T22:52:55.320462+0000","last_peered":"2024-05-26T22:52:55.320462+0000","last_clean":"2024-05-26T22:52:55.320462+0000","last_became_active":"2024-05-26T22:52:53.316702+0000","last_became_peered":"2024-05-26T22:52:53.316702+0000","last_unstale":"2024-05-26T22:52:55.320462+0000","last_undegraded":"2024-05-26T22:52:55.320462+0000","last_fullsized":"2024-05-26T22:52:55.320462+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_clean_scrub_stamp":"2024-05-26T22:52:52.250515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:30:12.377121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.319280+0000","last_change":"2024-05-26T22:52:54.323551+0000","last_active":"2024-05-26T22:52:55.319280+0000","last_peered":"2024-05-26T22:52:55.319280+0000","last_clean":"2024-05-26T22:52:55.319280+0000","last_became_active":"2024-05-26T22:52:54.323142+0000","last_became_peered":"2024-05-26T22:52:54.323142+0000","last_unstale":"2024-05-26T22:52:55.319280+0000","last_undegraded":"2024-05-26T22:52:55.319280+0000","last_fullsized":"2024-05-26T22:52:55.319280+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:42:27.434611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.417934+0000","last_change":"2024-05-26T22:52:53.319126+0000","last_active":"2024-05-26T22:52:55.417934+0000","last_peered":"2024-05-26T22:52:55.417934+0000","last_clean":"2024-05-26T22:52:55.417934+0000","last_became_active":"2024-05-26T22:52:53.318832+0000","last_became_peered":"2024-05-26T22:52:53.318832+0000","last_unstale":"2024-05-26T22:52:55.417934+0000","last_undegraded":"2024-05-26T22:52:55.417934+0000","last_fullsized":"2024-05-26T22:52:55.417934+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_clean_scrub_stamp":"2024-05-26T22:52:52.250515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:34:41.388075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.418053+0000","last_change":"2024-05-26T22:52:54.320025+0000","last_active":"2024-05-26T22:52:55.418053+0000","last_peered":"2024-05-26T22:52:55.418053+0000","last_clean":"2024-05-26T22:52:55.418053+0000","last_became_active":"2024-05-26T22:52:54.319106+0000","last_became_peered":"2024-05-26T22:52:54.319106+0000","last_unstale":"2024-05-26T22:52:55.418053+0000","last_undegraded":"2024-05-26T22:52:55.418053+0000","last_fullsized":"2024-05-26T22:52:55.418053+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:03:53.891999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.420039+0000","last_change":"2024-05-26T22:52:53.322311+0000","last_active":"2024-05-26T22:52:55.420039+0000","last_peered":"2024-05-26T22:52:55.420039+0000","last_clean":"2024-05-26T22:52:55.420039+0000","last_became_active":"2024-05-26T22:52:53.321876+0000","last_became_peered":"2024-05-26T22:52:53.321876+0000","last_unstale":"2024-05-26T22:52:55.420039+0000","last_undegraded":"2024-05-26T22:52:55.420039+0000","last_fullsized":"2024-05-26T22:52:55.420039+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_clean_scrub_stamp":"2024-05-26T22:52:52.250515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:03:52.876316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.318869+0000","last_change":"2024-05-26T22:52:54.310194+0000","last_active":"2024-05-26T22:52:55.318869+0000","last_peered":"2024-05-26T22:52:55.318869+0000","last_clean":"2024-05-26T22:52:55.318869+0000","last_became_active":"2024-05-26T22:52:54.309341+0000","last_became_peered":"2024-05-26T22:52:54.309341+0000","last_unstale":"2024-05-26T22:52:55.318869+0000","last_undegraded":"2024-05-26T22:52:55.318869+0000","last_fullsized":"2024-05-26T22:52:55.318869+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:26:38.417192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.319168+0000","last_change":"2024-05-26T22:52:53.313078+0000","last_active":"2024-05-26T22:52:55.319168+0000","last_peered":"2024-05-26T22:52:55.319168+0000","last_clean":"2024-05-26T22:52:55.319168+0000","last_became_active":"2024-05-26T22:52:53.312819+0000","last_became_peered":"2024-05-26T22:52:53.312819+0000","last_unstale":"2024-05-26T22:52:55.319168+0000","last_undegraded":"2024-05-26T22:52:55.319168+0000","last_fullsized":"2024-05-26T22:52:55.319168+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_clean_scrub_stamp":"2024-05-26T22:52:52.250515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:50:13.373742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.322456+0000","last_change":"2024-05-26T22:52:54.317258+0000","last_active":"2024-05-26T22:52:55.322456+0000","last_peered":"2024-05-26T22:52:55.322456+0000","last_clean":"2024-05-26T22:52:55.322456+0000","last_became_active":"2024-05-26T22:52:54.316945+0000","last_became_peered":"2024-05-26T22:52:54.316945+0000","last_unstale":"2024-05-26T22:52:55.322456+0000","last_undegraded":"2024-05-26T22:52:55.322456+0000","last_fullsized":"2024-05-26T22:52:55.322456+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:37:24.149835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"68'2","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.319487+0000","last_change":"2024-05-26T22:52:53.307081+0000","last_active":"2024-05-26T22:52:55.319487+0000","last_peered":"2024-05-26T22:52:55.319487+0000","last_clean":"2024-05-26T22:52:55.319487+0000","last_became_active":"2024-05-26T22:52:53.306701+0000","last_became_peered":"2024-05-26T22:52:53.306701+0000","last_unstale":"2024-05-26T22:52:55.319487+0000","last_undegraded":"2024-05-26T22:52:55.319487+0000","last_fullsized":"2024-05-26T22:52:55.319487+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_clean_scrub_stamp":"2024-05-26T22:52:52.250515+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:13:44.543717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":261,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.421918+0000","last_change":"2024-05-26T22:48:12.068917+0000","last_active":"2024-05-26T22:52:55.421918+0000","last_peered":"2024-05-26T22:52:55.421918+0000","last_clean":"2024-05-26T22:52:55.421918+0000","last_became_active":"2024-05-26T22:48:12.065934+0000","last_became_peered":"2024-05-26T22:48:12.065934+0000","last_unstale":"2024-05-26T22:52:55.421918+0000","last_undegraded":"2024-05-26T22:52:55.421918+0000","last_fullsized":"2024-05-26T22:52:55.421918+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:10.628122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:10.628122+0000","last_clean_scrub_stamp":"2024-05-26T22:48:10.628122+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:14:58.895911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574908,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32584,"kb_used_data":1396,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339256,"statfs":{"total":96636764160,"available":96603398144,"internally_reserved":0,"allocated":1429504,"data_stored":942701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574908,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32000,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339840,"statfs":{"total":96636764160,"available":96603996160,"internally_reserved":0,"allocated":835584,"data_stored":351809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574909,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31996,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339844,"statfs":{"total":96636764160,"available":96604000256,"internally_reserved":0,"allocated":831488,"data_stored":351807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574908,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31996,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339844,"statfs":{"total":96636764160,"available":96604000256,"internally_reserved":0,"allocated":831488,"data_stored":351787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574908,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32008,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339832,"statfs":{"total":96636764160,"available":96603987968,"internally_reserved":0,"allocated":847872,"data_stored":358129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574908,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32020,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339820,"statfs":{"total":96636764160,"available":96603975680,"internally_reserved":0,"allocated":851968,"data_stored":359629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574908,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32012,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339828,"statfs":{"total":96636764160,"available":96603983872,"internally_reserved":0,"allocated":847872,"data_stored":353945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574908,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32576,"kb_used_data":1388,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339264,"statfs":{"total":96636764160,"available":96603406336,"internally_reserved":0,"allocated":1421312,"data_stored":942085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-26T22:52:58.702 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-26T22:52:58.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.940+0000 7f6fcbc00640 1 Processor -- start 2024-05-26T22:52:58.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.940+0000 7f6fcbc00640 1 -- start start 2024-05-26T22:52:58.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.941+0000 7f6fcbc00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f6fc415c3d0 0x7f6fc406ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:58.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.942+0000 7f6fcbc00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6fc406f490 0x7f6fc40ade40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:58.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.942+0000 7f6fcbc00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6fc40722a0 con 0x7f6fc4155840 2024-05-26T22:52:58.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.942+0000 7f6fcbc00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f6fc4072040 con 0x7f6fc415c3d0 2024-05-26T22:52:58.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.942+0000 7f6fcbc00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f6fc4072580 con 0x7f6fc406f490 2024-05-26T22:52:58.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.942+0000 7f6fc9200640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f6fc4155840 0x7f6fc4071930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:42286/0 (socket says 172.21.15.131:42286) 2024-05-26T22:52:58.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.942+0000 7f6fc9200640 1 -- 172.21.15.131:0/933798193 learned_addr learned my addr 172.21.15.131:0/933798193 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:52:58.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.942+0000 7f6fc3e00640 1 --2- 172.21.15.131:0/933798193 >> v2:172.21.15.131:3300/0 conn(0x7f6fc415c3d0 0x7f6fc406ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:58.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.942+0000 7f6fc3400640 1 -- 172.21.15.131:0/933798193 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 800572348 0 0) 0x7f6fc40722a0 con 0x7f6fc4155840 2024-05-26T22:52:58.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.942+0000 7f6fc3400640 1 -- 172.21.15.131:0/933798193 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6fa8003520 con 0x7f6fc4155840 2024-05-26T22:52:58.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.942+0000 7f6fc3e00640 1 -- 172.21.15.131:0/933798193 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6fc406f490 msgr2=0x7f6fc40ade40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:58.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.942+0000 7f6fc3e00640 1 --2- 172.21.15.131:0/933798193 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6fc406f490 0x7f6fc40ade40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:58.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.942+0000 7f6fc3e00640 1 -- 172.21.15.131:0/933798193 >> v1:172.21.15.131:6789/0 conn(0x7f6fc4155840 legacy=0x7f6fc4071930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:58.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.942+0000 7f6fc3e00640 1 -- 172.21.15.131:0/933798193 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fc4072860 con 0x7f6fc415c3d0 2024-05-26T22:52:58.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.943+0000 7f6fc3e00640 1 --2- 172.21.15.131:0/933798193 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6fc415c3d0 0x7f6fc406ef50 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f6fac0097a0 tx=0x7f6fac02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5962c8516345905 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:58.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.944+0000 7f6fc3400640 1 -- 172.21.15.131:0/933798193 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fac037070 con 0x7f6fc415c3d0 2024-05-26T22:52:58.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.944+0000 7f6fc3400640 1 -- 172.21.15.131:0/933798193 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fac032440 con 0x7f6fc415c3d0 2024-05-26T22:52:58.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.944+0000 7f6fc3400640 1 -- 172.21.15.131:0/933798193 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fac032810 con 0x7f6fc415c3d0 2024-05-26T22:52:58.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.944+0000 7f6fcbc00640 1 -- 172.21.15.131:0/933798193 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6fc415c3d0 msgr2=0x7f6fc406ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:58.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.944+0000 7f6fcbc00640 1 --2- 172.21.15.131:0/933798193 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6fc415c3d0 0x7f6fc406ef50 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f6fac0097a0 tx=0x7f6fac02a180 comp rx=0 tx=0).stop 2024-05-26T22:52:58.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.944+0000 7f6fcbc00640 1 -- 172.21.15.131:0/933798193 shutdown_connections 2024-05-26T22:52:58.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.944+0000 7f6fcbc00640 1 --2- 172.21.15.131:0/933798193 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6fc406f490 0x7f6fc40ade40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:58.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.944+0000 7f6fcbc00640 1 --2- 172.21.15.131:0/933798193 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6fc415c3d0 0x7f6fc406ef50 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:58.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.944+0000 7f6fcbc00640 1 -- 172.21.15.131:0/933798193 >> 172.21.15.131:0/933798193 conn(0x7f6fc40b1e90 msgr2=0x7f6fc40b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:52:58.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.944+0000 7f6fcbc00640 1 -- 172.21.15.131:0/933798193 shutdown_connections 2024-05-26T22:52:58.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.944+0000 7f6fcbc00640 1 -- 172.21.15.131:0/933798193 wait complete. 2024-05-26T22:52:58.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.946+0000 7f6fcbc00640 1 Processor -- start 2024-05-26T22:52:58.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.947+0000 7f6fcbc00640 1 -- start start 2024-05-26T22:52:58.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.948+0000 7f6fcbc00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6fc406f490 0x7f6fc40ce740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:58.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.948+0000 7f6fcbc00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6fc4155840 0x7f6fc41f7a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:58.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.948+0000 7f6fcbc00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6fc415c3d0 0x7f6fc4151500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:58.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.948+0000 7f6fcbc00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f6fc40ae380 con 0x7f6fc415c3d0 2024-05-26T22:52:58.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.948+0000 7f6fcbc00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6fc4072e20 con 0x7f6fc406f490 2024-05-26T22:52:58.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.948+0000 7f6fcbc00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f6fc40ae500 con 0x7f6fc4155840 2024-05-26T22:52:58.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.948+0000 7f6fc9c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6fc415c3d0 0x7f6fc4151500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:58.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.948+0000 7f6fc9c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6fc415c3d0 0x7f6fc4151500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48486/0 (socket says 172.21.15.131:48486) 2024-05-26T22:52:58.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.948+0000 7f6fc9c00640 1 -- 172.21.15.131:0/1426005676 learned_addr learned my addr 172.21.15.131:0/1426005676 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:52:58.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.948+0000 7f6fc3e00640 1 --2- 172.21.15.131:0/1426005676 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6fc4155840 0x7f6fc41f7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:58.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.949+0000 7f6fc9200640 1 --2- 172.21.15.131:0/1426005676 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6fc406f490 0x7f6fc40ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:58.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.949+0000 7f6fc9c00640 1 -- 172.21.15.131:0/1426005676 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6fc4155840 msgr2=0x7f6fc41f7a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:58.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.949+0000 7f6fc9c00640 1 --2- 172.21.15.131:0/1426005676 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6fc4155840 0x7f6fc41f7a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:58.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.949+0000 7f6fc9c00640 1 -- 172.21.15.131:0/1426005676 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6fc406f490 msgr2=0x7f6fc40ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:58.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.949+0000 7f6fc9c00640 1 --2- 172.21.15.131:0/1426005676 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6fc406f490 0x7f6fc40ce740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:58.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.949+0000 7f6fc9c00640 1 -- 172.21.15.131:0/1426005676 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fc40cee00 con 0x7f6fc415c3d0 2024-05-26T22:52:58.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.949+0000 7f6fc9c00640 1 --2- 172.21.15.131:0/1426005676 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6fc415c3d0 0x7f6fc4151500 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc009fd0 tx=0x7f6fbc00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:58.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.949+0000 7f6fc0c00640 1 -- 172.21.15.131:0/1426005676 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fbc011830 con 0x7f6fc415c3d0 2024-05-26T22:52:58.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.949+0000 7f6fc0c00640 1 -- 172.21.15.131:0/1426005676 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fbc016070 con 0x7f6fc415c3d0 2024-05-26T22:52:58.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.949+0000 7f6fc0c00640 1 -- 172.21.15.131:0/1426005676 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fbc011bf0 con 0x7f6fc415c3d0 2024-05-26T22:52:58.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.949+0000 7f6fcbc00640 1 -- 172.21.15.131:0/1426005676 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fc40cf0e0 con 0x7f6fc415c3d0 2024-05-26T22:52:58.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.949+0000 7f6fcbc00640 1 -- 172.21.15.131:0/1426005676 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fc40cf6a0 con 0x7f6fc415c3d0 2024-05-26T22:52:58.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.950+0000 7f6fcbc00640 1 -- 172.21.15.131:0/1426005676 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fc41f3a00 con 0x7f6fc415c3d0 2024-05-26T22:52:58.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.952+0000 7f6fc0c00640 1 -- 172.21.15.131:0/1426005676 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6fbc01b460 con 0x7f6fc415c3d0 2024-05-26T22:52:58.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.952+0000 7f6fc0c00640 1 --2- 172.21.15.131:0/1426005676 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f6fa006be30 0x7f6fa006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:58.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.953+0000 7f6fc0c00640 1 -- 172.21.15.131:0/1426005676 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7f6fbc08ac00 con 0x7f6fc415c3d0 2024-05-26T22:52:58.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.953+0000 7f6fc9200640 1 --2- 172.21.15.131:0/1426005676 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f6fa006be30 0x7f6fa006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:58.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.953+0000 7f6fc9200640 1 --2- 172.21.15.131:0/1426005676 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f6fa006be30 0x7f6fa006e2f0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f6fb4001560 tx=0x7f6fb4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:58.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:58.957+0000 7f6fc0c00640 1 -- 172.21.15.131:0/1426005676 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f6fbc0595a0 con 0x7f6fc415c3d0 2024-05-26T22:52:59.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.149+0000 7f6fcbc00640 1 -- 172.21.15.131:0/1426005676 --> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6fc40694e0 con 0x7f6fa006be30 2024-05-26T22:52:59.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.156+0000 7f6fc0c00640 1 -- 172.21.15.131:0/1426005676 <== mgr.4104 v2:172.21.15.131:6832/3484832947 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252064 (secure 0 0 0) 0x7f6fc40694e0 con 0x7f6fa006be30 2024-05-26T22:52:59.160 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:52:59.162 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-05-26T22:52:59.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.162+0000 7f6fcbc00640 1 -- 172.21.15.131:0/1426005676 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f6fa006be30 msgr2=0x7f6fa006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:59.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.162+0000 7f6fcbc00640 1 --2- 172.21.15.131:0/1426005676 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f6fa006be30 0x7f6fa006e2f0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f6fb4001560 tx=0x7f6fb4009040 comp rx=0 tx=0).stop 2024-05-26T22:52:59.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.162+0000 7f6fcbc00640 1 -- 172.21.15.131:0/1426005676 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6fc415c3d0 msgr2=0x7f6fc4151500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:59.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.162+0000 7f6fcbc00640 1 --2- 172.21.15.131:0/1426005676 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6fc415c3d0 0x7f6fc4151500 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc009fd0 tx=0x7f6fbc00b8e0 comp rx=0 tx=0).stop 2024-05-26T22:52:59.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.163+0000 7f6fcbc00640 1 -- 172.21.15.131:0/1426005676 shutdown_connections 2024-05-26T22:52:59.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.163+0000 7f6fcbc00640 1 --2- 172.21.15.131:0/1426005676 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6fc4155840 0x7f6fc41f7a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:59.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.163+0000 7f6fcbc00640 1 --2- 172.21.15.131:0/1426005676 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f6fa006be30 0x7f6fa006e2f0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:59.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.163+0000 7f6fcbc00640 1 --2- 172.21.15.131:0/1426005676 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6fc415c3d0 0x7f6fc4151500 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:59.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.163+0000 7f6fcbc00640 1 --2- 172.21.15.131:0/1426005676 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6fc406f490 0x7f6fc40ce740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:59.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.163+0000 7f6fcbc00640 1 -- 172.21.15.131:0/1426005676 >> 172.21.15.131:0/1426005676 conn(0x7f6fc40b1e90 msgr2=0x7f6fc40afe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:52:59.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.163+0000 7f6fcbc00640 1 -- 172.21.15.131:0/1426005676 shutdown_connections 2024-05-26T22:52:59.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.163+0000 7f6fcbc00640 1 -- 172.21.15.131:0/1426005676 wait complete. 2024-05-26T22:52:59.182 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":206,"stamp":"2024-05-26T22:52:58.476472+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":93,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":257192,"kb_used_data":7712,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754717528,"statfs":{"total":773094113280,"available":772830748672,"internally_reserved":0,"allocated":7897088,"data_stored":4011892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001923"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.423385+0000","last_change":"2024-05-26T22:52:54.333360+0000","last_active":"2024-05-26T22:52:55.423385+0000","last_peered":"2024-05-26T22:52:55.423385+0000","last_clean":"2024-05-26T22:52:55.423385+0000","last_became_active":"2024-05-26T22:52:54.333050+0000","last_became_peered":"2024-05-26T22:52:54.333050+0000","last_unstale":"2024-05-26T22:52:55.423385+0000","last_undegraded":"2024-05-26T22:52:55.423385+0000","last_fullsized":"2024-05-26T22:52:55.423385+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T22:55:42.930355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.320822+0000","last_change":"2024-05-26T22:52:54.336338+0000","last_active":"2024-05-26T22:52:55.320822+0000","last_peered":"2024-05-26T22:52:55.320822+0000","last_clean":"2024-05-26T22:52:55.320822+0000","last_became_active":"2024-05-26T22:52:54.335884+0000","last_became_peered":"2024-05-26T22:52:54.335884+0000","last_unstale":"2024-05-26T22:52:55.320822+0000","last_undegraded":"2024-05-26T22:52:55.320822+0000","last_fullsized":"2024-05-26T22:52:55.320822+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:10:56.957932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.322331+0000","last_change":"2024-05-26T22:52:54.319682+0000","last_active":"2024-05-26T22:52:55.322331+0000","last_peered":"2024-05-26T22:52:55.322331+0000","last_clean":"2024-05-26T22:52:55.322331+0000","last_became_active":"2024-05-26T22:52:54.319265+0000","last_became_peered":"2024-05-26T22:52:54.319265+0000","last_unstale":"2024-05-26T22:52:55.322331+0000","last_undegraded":"2024-05-26T22:52:55.322331+0000","last_fullsized":"2024-05-26T22:52:55.322331+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T23:24:03.220832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.418078+0000","last_change":"2024-05-26T22:52:54.329529+0000","last_active":"2024-05-26T22:52:55.418078+0000","last_peered":"2024-05-26T22:52:55.418078+0000","last_clean":"2024-05-26T22:52:55.418078+0000","last_became_active":"2024-05-26T22:52:54.329181+0000","last_became_peered":"2024-05-26T22:52:54.329181+0000","last_unstale":"2024-05-26T22:52:55.418078+0000","last_undegraded":"2024-05-26T22:52:55.418078+0000","last_fullsized":"2024-05-26T22:52:55.418078+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:13:45.559399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.320076+0000","last_change":"2024-05-26T22:52:54.325242+0000","last_active":"2024-05-26T22:52:55.320076+0000","last_peered":"2024-05-26T22:52:55.320076+0000","last_clean":"2024-05-26T22:52:55.320076+0000","last_became_active":"2024-05-26T22:52:54.324610+0000","last_became_peered":"2024-05-26T22:52:54.324610+0000","last_unstale":"2024-05-26T22:52:55.320076+0000","last_undegraded":"2024-05-26T22:52:55.320076+0000","last_fullsized":"2024-05-26T22:52:55.320076+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:09:15.633550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.319277+0000","last_change":"2024-05-26T22:52:54.332798+0000","last_active":"2024-05-26T22:52:55.319277+0000","last_peered":"2024-05-26T22:52:55.319277+0000","last_clean":"2024-05-26T22:52:55.319277+0000","last_became_active":"2024-05-26T22:52:54.330758+0000","last_became_peered":"2024-05-26T22:52:54.330758+0000","last_unstale":"2024-05-26T22:52:55.319277+0000","last_undegraded":"2024-05-26T22:52:55.319277+0000","last_fullsized":"2024-05-26T22:52:55.319277+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:02:46.912953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.324794+0000","last_change":"2024-05-26T22:52:54.321771+0000","last_active":"2024-05-26T22:52:55.324794+0000","last_peered":"2024-05-26T22:52:55.324794+0000","last_clean":"2024-05-26T22:52:55.324794+0000","last_became_active":"2024-05-26T22:52:54.321452+0000","last_became_peered":"2024-05-26T22:52:54.321452+0000","last_unstale":"2024-05-26T22:52:55.324794+0000","last_undegraded":"2024-05-26T22:52:55.324794+0000","last_fullsized":"2024-05-26T22:52:55.324794+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:16:08.984694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.319002+0000","last_change":"2024-05-26T22:52:54.313423+0000","last_active":"2024-05-26T22:52:55.319002+0000","last_peered":"2024-05-26T22:52:55.319002+0000","last_clean":"2024-05-26T22:52:55.319002+0000","last_became_active":"2024-05-26T22:52:54.312774+0000","last_became_peered":"2024-05-26T22:52:54.312774+0000","last_unstale":"2024-05-26T22:52:55.319002+0000","last_undegraded":"2024-05-26T22:52:55.319002+0000","last_fullsized":"2024-05-26T22:52:55.319002+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:15:48.861927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.323409+0000","last_change":"2024-05-26T22:52:54.311650+0000","last_active":"2024-05-26T22:52:55.323409+0000","last_peered":"2024-05-26T22:52:55.323409+0000","last_clean":"2024-05-26T22:52:55.323409+0000","last_became_active":"2024-05-26T22:52:54.311145+0000","last_became_peered":"2024-05-26T22:52:54.311145+0000","last_unstale":"2024-05-26T22:52:55.323409+0000","last_undegraded":"2024-05-26T22:52:55.323409+0000","last_fullsized":"2024-05-26T22:52:55.323409+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:23:29.499207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.420707+0000","last_change":"2024-05-26T22:52:54.324499+0000","last_active":"2024-05-26T22:52:55.420707+0000","last_peered":"2024-05-26T22:52:55.420707+0000","last_clean":"2024-05-26T22:52:55.420707+0000","last_became_active":"2024-05-26T22:52:54.324109+0000","last_became_peered":"2024-05-26T22:52:54.324109+0000","last_unstale":"2024-05-26T22:52:55.420707+0000","last_undegraded":"2024-05-26T22:52:55.420707+0000","last_fullsized":"2024-05-26T22:52:55.420707+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:54:09.591587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.423353+0000","last_change":"2024-05-26T22:52:54.327991+0000","last_active":"2024-05-26T22:52:55.423353+0000","last_peered":"2024-05-26T22:52:55.423353+0000","last_clean":"2024-05-26T22:52:55.423353+0000","last_became_active":"2024-05-26T22:52:54.325134+0000","last_became_peered":"2024-05-26T22:52:54.325134+0000","last_unstale":"2024-05-26T22:52:55.423353+0000","last_undegraded":"2024-05-26T22:52:55.423353+0000","last_fullsized":"2024-05-26T22:52:55.423353+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:02:22.457006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.418087+0000","last_change":"2024-05-26T22:52:54.323082+0000","last_active":"2024-05-26T22:52:55.418087+0000","last_peered":"2024-05-26T22:52:55.418087+0000","last_clean":"2024-05-26T22:52:55.418087+0000","last_became_active":"2024-05-26T22:52:54.322776+0000","last_became_peered":"2024-05-26T22:52:54.322776+0000","last_unstale":"2024-05-26T22:52:55.418087+0000","last_undegraded":"2024-05-26T22:52:55.418087+0000","last_fullsized":"2024-05-26T22:52:55.418087+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:07:19.490731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.329516+0000","last_change":"2024-05-26T22:52:54.323520+0000","last_active":"2024-05-26T22:52:55.329516+0000","last_peered":"2024-05-26T22:52:55.329516+0000","last_clean":"2024-05-26T22:52:55.329516+0000","last_became_active":"2024-05-26T22:52:54.323229+0000","last_became_peered":"2024-05-26T22:52:54.323229+0000","last_unstale":"2024-05-26T22:52:55.329516+0000","last_undegraded":"2024-05-26T22:52:55.329516+0000","last_fullsized":"2024-05-26T22:52:55.329516+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:07:21.524347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.321086+0000","last_change":"2024-05-26T22:52:54.331143+0000","last_active":"2024-05-26T22:52:55.321086+0000","last_peered":"2024-05-26T22:52:55.321086+0000","last_clean":"2024-05-26T22:52:55.321086+0000","last_became_active":"2024-05-26T22:52:54.330479+0000","last_became_peered":"2024-05-26T22:52:54.330479+0000","last_unstale":"2024-05-26T22:52:55.321086+0000","last_undegraded":"2024-05-26T22:52:55.321086+0000","last_fullsized":"2024-05-26T22:52:55.321086+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:35:07.002533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.329252+0000","last_change":"2024-05-26T22:52:54.316704+0000","last_active":"2024-05-26T22:52:55.329252+0000","last_peered":"2024-05-26T22:52:55.329252+0000","last_clean":"2024-05-26T22:52:55.329252+0000","last_became_active":"2024-05-26T22:52:54.316016+0000","last_became_peered":"2024-05-26T22:52:54.316016+0000","last_unstale":"2024-05-26T22:52:55.329252+0000","last_undegraded":"2024-05-26T22:52:55.329252+0000","last_fullsized":"2024-05-26T22:52:55.329252+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:31:19.880810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.420048+0000","last_change":"2024-05-26T22:52:54.322929+0000","last_active":"2024-05-26T22:52:55.420048+0000","last_peered":"2024-05-26T22:52:55.420048+0000","last_clean":"2024-05-26T22:52:55.420048+0000","last_became_active":"2024-05-26T22:52:54.322597+0000","last_became_peered":"2024-05-26T22:52:54.322597+0000","last_unstale":"2024-05-26T22:52:55.420048+0000","last_undegraded":"2024-05-26T22:52:55.420048+0000","last_fullsized":"2024-05-26T22:52:55.420048+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T23:07:36.508541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.323618+0000","last_change":"2024-05-26T22:52:54.311423+0000","last_active":"2024-05-26T22:52:55.323618+0000","last_peered":"2024-05-26T22:52:55.323618+0000","last_clean":"2024-05-26T22:52:55.323618+0000","last_became_active":"2024-05-26T22:52:54.310514+0000","last_became_peered":"2024-05-26T22:52:54.310514+0000","last_unstale":"2024-05-26T22:52:55.323618+0000","last_undegraded":"2024-05-26T22:52:55.323618+0000","last_fullsized":"2024-05-26T22:52:55.323618+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:57:21.104575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.418349+0000","last_change":"2024-05-26T22:52:54.329535+0000","last_active":"2024-05-26T22:52:55.418349+0000","last_peered":"2024-05-26T22:52:55.418349+0000","last_clean":"2024-05-26T22:52:55.418349+0000","last_became_active":"2024-05-26T22:52:54.329155+0000","last_became_peered":"2024-05-26T22:52:54.329155+0000","last_unstale":"2024-05-26T22:52:55.418349+0000","last_undegraded":"2024-05-26T22:52:55.418349+0000","last_fullsized":"2024-05-26T22:52:55.418349+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T23:55:59.295714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.320127+0000","last_change":"2024-05-26T22:52:54.314530+0000","last_active":"2024-05-26T22:52:55.320127+0000","last_peered":"2024-05-26T22:52:55.320127+0000","last_clean":"2024-05-26T22:52:55.320127+0000","last_became_active":"2024-05-26T22:52:54.314005+0000","last_became_peered":"2024-05-26T22:52:54.314005+0000","last_unstale":"2024-05-26T22:52:55.320127+0000","last_undegraded":"2024-05-26T22:52:55.320127+0000","last_fullsized":"2024-05-26T22:52:55.320127+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:38:46.157530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.329806+0000","last_change":"2024-05-26T22:52:54.330936+0000","last_active":"2024-05-26T22:52:55.329806+0000","last_peered":"2024-05-26T22:52:55.329806+0000","last_clean":"2024-05-26T22:52:55.329806+0000","last_became_active":"2024-05-26T22:52:54.330650+0000","last_became_peered":"2024-05-26T22:52:54.330650+0000","last_unstale":"2024-05-26T22:52:55.329806+0000","last_undegraded":"2024-05-26T22:52:55.329806+0000","last_fullsized":"2024-05-26T22:52:55.329806+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:02:22.457006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.420480+0000","last_change":"2024-05-26T22:52:54.324464+0000","last_active":"2024-05-26T22:52:55.420480+0000","last_peered":"2024-05-26T22:52:55.420480+0000","last_clean":"2024-05-26T22:52:55.420480+0000","last_became_active":"2024-05-26T22:52:54.324177+0000","last_became_peered":"2024-05-26T22:52:54.324177+0000","last_unstale":"2024-05-26T22:52:55.420480+0000","last_undegraded":"2024-05-26T22:52:55.420480+0000","last_fullsized":"2024-05-26T22:52:55.420480+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:42:27.434611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.420184+0000","last_change":"2024-05-26T22:52:54.319660+0000","last_active":"2024-05-26T22:52:55.420184+0000","last_peered":"2024-05-26T22:52:55.420184+0000","last_clean":"2024-05-26T22:52:55.420184+0000","last_became_active":"2024-05-26T22:52:54.319374+0000","last_became_peered":"2024-05-26T22:52:54.319374+0000","last_unstale":"2024-05-26T22:52:55.420184+0000","last_undegraded":"2024-05-26T22:52:55.420184+0000","last_fullsized":"2024-05-26T22:52:55.420184+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:02:46.912953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.422342+0000","last_change":"2024-05-26T22:52:54.336467+0000","last_active":"2024-05-26T22:52:55.422342+0000","last_peered":"2024-05-26T22:52:55.422342+0000","last_clean":"2024-05-26T22:52:55.422342+0000","last_became_active":"2024-05-26T22:52:54.336117+0000","last_became_peered":"2024-05-26T22:52:54.336117+0000","last_unstale":"2024-05-26T22:52:55.422342+0000","last_undegraded":"2024-05-26T22:52:55.422342+0000","last_fullsized":"2024-05-26T22:52:55.422342+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:06:23.444659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.422588+0000","last_change":"2024-05-26T22:52:54.342506+0000","last_active":"2024-05-26T22:52:55.422588+0000","last_peered":"2024-05-26T22:52:55.422588+0000","last_clean":"2024-05-26T22:52:55.422588+0000","last_became_active":"2024-05-26T22:52:54.341921+0000","last_became_peered":"2024-05-26T22:52:54.341921+0000","last_unstale":"2024-05-26T22:52:55.422588+0000","last_undegraded":"2024-05-26T22:52:55.422588+0000","last_fullsized":"2024-05-26T22:52:55.422588+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:54:14.144555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.420726+0000","last_change":"2024-05-26T22:52:54.336439+0000","last_active":"2024-05-26T22:52:55.420726+0000","last_peered":"2024-05-26T22:52:55.420726+0000","last_clean":"2024-05-26T22:52:55.420726+0000","last_became_active":"2024-05-26T22:52:54.336072+0000","last_became_peered":"2024-05-26T22:52:54.336072+0000","last_unstale":"2024-05-26T22:52:55.420726+0000","last_undegraded":"2024-05-26T22:52:55.420726+0000","last_fullsized":"2024-05-26T22:52:55.420726+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:16:08.984694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.325203+0000","last_change":"2024-05-26T22:52:54.338757+0000","last_active":"2024-05-26T22:52:55.325203+0000","last_peered":"2024-05-26T22:52:55.325203+0000","last_clean":"2024-05-26T22:52:55.325203+0000","last_became_active":"2024-05-26T22:52:54.338265+0000","last_became_peered":"2024-05-26T22:52:54.338265+0000","last_unstale":"2024-05-26T22:52:55.325203+0000","last_undegraded":"2024-05-26T22:52:55.325203+0000","last_fullsized":"2024-05-26T22:52:55.325203+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:12:25.007015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.322659+0000","last_change":"2024-05-26T22:52:54.336337+0000","last_active":"2024-05-26T22:52:55.322659+0000","last_peered":"2024-05-26T22:52:55.322659+0000","last_clean":"2024-05-26T22:52:55.322659+0000","last_became_active":"2024-05-26T22:52:54.336105+0000","last_became_peered":"2024-05-26T22:52:54.336105+0000","last_unstale":"2024-05-26T22:52:55.322659+0000","last_undegraded":"2024-05-26T22:52:55.322659+0000","last_fullsized":"2024-05-26T22:52:55.322659+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:46:56.093316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.420922+0000","last_change":"2024-05-26T22:52:54.327417+0000","last_active":"2024-05-26T22:52:55.420922+0000","last_peered":"2024-05-26T22:52:55.420922+0000","last_clean":"2024-05-26T22:52:55.420922+0000","last_became_active":"2024-05-26T22:52:54.324876+0000","last_became_peered":"2024-05-26T22:52:54.324876+0000","last_unstale":"2024-05-26T22:52:55.420922+0000","last_undegraded":"2024-05-26T22:52:55.420922+0000","last_fullsized":"2024-05-26T22:52:55.420922+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T23:09:02.321433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.324148+0000","last_change":"2024-05-26T22:52:54.330638+0000","last_active":"2024-05-26T22:52:55.324148+0000","last_peered":"2024-05-26T22:52:55.324148+0000","last_clean":"2024-05-26T22:52:55.324148+0000","last_became_active":"2024-05-26T22:52:54.330365+0000","last_became_peered":"2024-05-26T22:52:54.330365+0000","last_unstale":"2024-05-26T22:52:55.324148+0000","last_undegraded":"2024-05-26T22:52:55.324148+0000","last_fullsized":"2024-05-26T22:52:55.324148+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:26:58.318248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.325068+0000","last_change":"2024-05-26T22:52:54.326077+0000","last_active":"2024-05-26T22:52:55.325068+0000","last_peered":"2024-05-26T22:52:55.325068+0000","last_clean":"2024-05-26T22:52:55.325068+0000","last_became_active":"2024-05-26T22:52:54.325512+0000","last_became_peered":"2024-05-26T22:52:54.325512+0000","last_unstale":"2024-05-26T22:52:55.325068+0000","last_undegraded":"2024-05-26T22:52:55.325068+0000","last_fullsized":"2024-05-26T22:52:55.325068+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T23:09:02.321433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.324975+0000","last_change":"2024-05-26T22:52:54.328874+0000","last_active":"2024-05-26T22:52:55.324975+0000","last_peered":"2024-05-26T22:52:55.324975+0000","last_clean":"2024-05-26T22:52:55.324975+0000","last_became_active":"2024-05-26T22:52:54.326585+0000","last_became_peered":"2024-05-26T22:52:54.326585+0000","last_unstale":"2024-05-26T22:52:55.324975+0000","last_undegraded":"2024-05-26T22:52:55.324975+0000","last_fullsized":"2024-05-26T22:52:55.324975+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:10:56.957932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.319000+0000","last_change":"2024-05-26T22:52:54.315875+0000","last_active":"2024-05-26T22:52:55.319000+0000","last_peered":"2024-05-26T22:52:55.319000+0000","last_clean":"2024-05-26T22:52:55.319000+0000","last_became_active":"2024-05-26T22:52:54.308338+0000","last_became_peered":"2024-05-26T22:52:54.308338+0000","last_unstale":"2024-05-26T22:52:55.319000+0000","last_undegraded":"2024-05-26T22:52:55.319000+0000","last_fullsized":"2024-05-26T22:52:55.319000+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T23:07:36.508541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.423330+0000","last_change":"2024-05-26T22:52:54.342602+0000","last_active":"2024-05-26T22:52:55.423330+0000","last_peered":"2024-05-26T22:52:55.423330+0000","last_clean":"2024-05-26T22:52:55.423330+0000","last_became_active":"2024-05-26T22:52:54.335864+0000","last_became_peered":"2024-05-26T22:52:54.335864+0000","last_unstale":"2024-05-26T22:52:55.423330+0000","last_undegraded":"2024-05-26T22:52:55.423330+0000","last_fullsized":"2024-05-26T22:52:55.423330+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:07:21.524347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"68'4","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.423868+0000","last_change":"2024-05-26T22:52:53.320118+0000","last_active":"2024-05-26T22:52:55.423868+0000","last_peered":"2024-05-26T22:52:55.423868+0000","last_clean":"2024-05-26T22:52:55.423868+0000","last_became_active":"2024-05-26T22:52:53.319776+0000","last_became_peered":"2024-05-26T22:52:53.319776+0000","last_unstale":"2024-05-26T22:52:55.423868+0000","last_undegraded":"2024-05-26T22:52:55.423868+0000","last_fullsized":"2024-05-26T22:52:55.423868+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_clean_scrub_stamp":"2024-05-26T22:52:52.250515+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:01:21.968664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.323819+0000","last_change":"2024-05-26T22:52:54.312192+0000","last_active":"2024-05-26T22:52:55.323819+0000","last_peered":"2024-05-26T22:52:55.323819+0000","last_clean":"2024-05-26T22:52:55.323819+0000","last_became_active":"2024-05-26T22:52:54.310169+0000","last_became_peered":"2024-05-26T22:52:54.310169+0000","last_unstale":"2024-05-26T22:52:55.323819+0000","last_undegraded":"2024-05-26T22:52:55.323819+0000","last_fullsized":"2024-05-26T22:52:55.323819+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:30:04.180895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"68'1","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.330064+0000","last_change":"2024-05-26T22:52:53.311353+0000","last_active":"2024-05-26T22:52:55.330064+0000","last_peered":"2024-05-26T22:52:55.330064+0000","last_clean":"2024-05-26T22:52:55.330064+0000","last_became_active":"2024-05-26T22:52:53.311100+0000","last_became_peered":"2024-05-26T22:52:53.311100+0000","last_unstale":"2024-05-26T22:52:55.330064+0000","last_undegraded":"2024-05-26T22:52:55.330064+0000","last_fullsized":"2024-05-26T22:52:55.330064+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_clean_scrub_stamp":"2024-05-26T22:52:52.250515+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:54:08.575904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.319834+0000","last_change":"2024-05-26T22:52:54.327613+0000","last_active":"2024-05-26T22:52:55.319834+0000","last_peered":"2024-05-26T22:52:55.319834+0000","last_clean":"2024-05-26T22:52:55.319834+0000","last_became_active":"2024-05-26T22:52:54.322032+0000","last_became_peered":"2024-05-26T22:52:54.322032+0000","last_unstale":"2024-05-26T22:52:55.319834+0000","last_undegraded":"2024-05-26T22:52:55.319834+0000","last_fullsized":"2024-05-26T22:52:55.319834+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:54:09.591587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"68'2","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.420768+0000","last_change":"2024-05-26T22:52:53.322733+0000","last_active":"2024-05-26T22:52:55.420768+0000","last_peered":"2024-05-26T22:52:55.420768+0000","last_clean":"2024-05-26T22:52:55.420768+0000","last_became_active":"2024-05-26T22:52:53.321828+0000","last_became_peered":"2024-05-26T22:52:53.321828+0000","last_unstale":"2024-05-26T22:52:55.420768+0000","last_undegraded":"2024-05-26T22:52:55.420768+0000","last_fullsized":"2024-05-26T22:52:55.420768+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_clean_scrub_stamp":"2024-05-26T22:52:52.250515+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:13:44.543717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.423286+0000","last_change":"2024-05-26T22:52:54.326633+0000","last_active":"2024-05-26T22:52:55.423286+0000","last_peered":"2024-05-26T22:52:55.423286+0000","last_clean":"2024-05-26T22:52:55.423286+0000","last_became_active":"2024-05-26T22:52:54.326331+0000","last_became_peered":"2024-05-26T22:52:54.326331+0000","last_unstale":"2024-05-26T22:52:55.423286+0000","last_undegraded":"2024-05-26T22:52:55.423286+0000","last_fullsized":"2024-05-26T22:52:55.423286+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:02:43.306586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"68'2","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.319986+0000","last_change":"2024-05-26T22:52:53.311637+0000","last_active":"2024-05-26T22:52:55.319986+0000","last_peered":"2024-05-26T22:52:55.319986+0000","last_clean":"2024-05-26T22:52:55.319986+0000","last_became_active":"2024-05-26T22:52:53.311349+0000","last_became_peered":"2024-05-26T22:52:53.311349+0000","last_unstale":"2024-05-26T22:52:55.319986+0000","last_undegraded":"2024-05-26T22:52:55.319986+0000","last_fullsized":"2024-05-26T22:52:55.319986+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_clean_scrub_stamp":"2024-05-26T22:52:52.250515+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:34:41.388075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.324967+0000","last_change":"2024-05-26T22:52:54.330214+0000","last_active":"2024-05-26T22:52:55.324967+0000","last_peered":"2024-05-26T22:52:55.324967+0000","last_clean":"2024-05-26T22:52:55.324967+0000","last_became_active":"2024-05-26T22:52:54.329873+0000","last_became_peered":"2024-05-26T22:52:54.329873+0000","last_unstale":"2024-05-26T22:52:55.324967+0000","last_undegraded":"2024-05-26T22:52:55.324967+0000","last_fullsized":"2024-05-26T22:52:55.324967+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:38:46.157530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"68'1","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.324314+0000","last_change":"2024-05-26T22:52:53.315524+0000","last_active":"2024-05-26T22:52:55.324314+0000","last_peered":"2024-05-26T22:52:55.324314+0000","last_clean":"2024-05-26T22:52:55.324314+0000","last_became_active":"2024-05-26T22:52:53.315217+0000","last_became_peered":"2024-05-26T22:52:53.315217+0000","last_unstale":"2024-05-26T22:52:55.324314+0000","last_undegraded":"2024-05-26T22:52:55.324314+0000","last_fullsized":"2024-05-26T22:52:55.324314+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_clean_scrub_stamp":"2024-05-26T22:52:52.250515+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:37:23.559093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.420916+0000","last_change":"2024-05-26T22:52:54.328875+0000","last_active":"2024-05-26T22:52:55.420916+0000","last_peered":"2024-05-26T22:52:55.420916+0000","last_clean":"2024-05-26T22:52:55.420916+0000","last_became_active":"2024-05-26T22:52:54.328287+0000","last_became_peered":"2024-05-26T22:52:54.328287+0000","last_unstale":"2024-05-26T22:52:55.420916+0000","last_undegraded":"2024-05-26T22:52:55.420916+0000","last_fullsized":"2024-05-26T22:52:55.420916+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:10:56.957932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"68'1","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.319324+0000","last_change":"2024-05-26T22:52:53.309674+0000","last_active":"2024-05-26T22:52:55.319324+0000","last_peered":"2024-05-26T22:52:55.319324+0000","last_clean":"2024-05-26T22:52:55.319324+0000","last_became_active":"2024-05-26T22:52:53.308165+0000","last_became_peered":"2024-05-26T22:52:53.308165+0000","last_unstale":"2024-05-26T22:52:55.319324+0000","last_undegraded":"2024-05-26T22:52:55.319324+0000","last_fullsized":"2024-05-26T22:52:55.319324+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_clean_scrub_stamp":"2024-05-26T22:52:52.250515+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:49:12.112441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.319766+0000","last_change":"2024-05-26T22:52:54.328030+0000","last_active":"2024-05-26T22:52:55.319766+0000","last_peered":"2024-05-26T22:52:55.319766+0000","last_clean":"2024-05-26T22:52:55.319766+0000","last_became_active":"2024-05-26T22:52:54.327705+0000","last_became_peered":"2024-05-26T22:52:54.327705+0000","last_unstale":"2024-05-26T22:52:55.319766+0000","last_undegraded":"2024-05-26T22:52:55.319766+0000","last_fullsized":"2024-05-26T22:52:55.319766+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:51:42.664856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.417932+0000","last_change":"2024-05-26T22:52:53.316531+0000","last_active":"2024-05-26T22:52:55.417932+0000","last_peered":"2024-05-26T22:52:55.417932+0000","last_clean":"2024-05-26T22:52:55.417932+0000","last_became_active":"2024-05-26T22:52:53.316205+0000","last_became_peered":"2024-05-26T22:52:53.316205+0000","last_unstale":"2024-05-26T22:52:55.417932+0000","last_undegraded":"2024-05-26T22:52:55.417932+0000","last_fullsized":"2024-05-26T22:52:55.417932+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_clean_scrub_stamp":"2024-05-26T22:52:52.250515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:23:33.311153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.329473+0000","last_change":"2024-05-26T22:52:54.334339+0000","last_active":"2024-05-26T22:52:55.329473+0000","last_peered":"2024-05-26T22:52:55.329473+0000","last_clean":"2024-05-26T22:52:55.329473+0000","last_became_active":"2024-05-26T22:52:54.333870+0000","last_became_peered":"2024-05-26T22:52:54.333870+0000","last_unstale":"2024-05-26T22:52:55.329473+0000","last_undegraded":"2024-05-26T22:52:55.329473+0000","last_fullsized":"2024-05-26T22:52:55.329473+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:02:43.306586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.319768+0000","last_change":"2024-05-26T22:52:53.310821+0000","last_active":"2024-05-26T22:52:55.319768+0000","last_peered":"2024-05-26T22:52:55.319768+0000","last_clean":"2024-05-26T22:52:55.319768+0000","last_became_active":"2024-05-26T22:52:53.310373+0000","last_became_peered":"2024-05-26T22:52:53.310373+0000","last_unstale":"2024-05-26T22:52:55.319768+0000","last_undegraded":"2024-05-26T22:52:55.319768+0000","last_fullsized":"2024-05-26T22:52:55.319768+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_clean_scrub_stamp":"2024-05-26T22:52:52.250515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:32:23.515533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.325456+0000","last_change":"2024-05-26T22:52:54.335108+0000","last_active":"2024-05-26T22:52:55.325456+0000","last_peered":"2024-05-26T22:52:55.325456+0000","last_clean":"2024-05-26T22:52:55.325456+0000","last_became_active":"2024-05-26T22:52:54.334602+0000","last_became_peered":"2024-05-26T22:52:54.334602+0000","last_unstale":"2024-05-26T22:52:55.325456+0000","last_undegraded":"2024-05-26T22:52:55.325456+0000","last_fullsized":"2024-05-26T22:52:55.325456+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:54:14.144555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.420978+0000","last_change":"2024-05-26T22:52:53.320502+0000","last_active":"2024-05-26T22:52:55.420978+0000","last_peered":"2024-05-26T22:52:55.420978+0000","last_clean":"2024-05-26T22:52:55.420978+0000","last_became_active":"2024-05-26T22:52:53.320028+0000","last_became_peered":"2024-05-26T22:52:53.320028+0000","last_unstale":"2024-05-26T22:52:55.420978+0000","last_undegraded":"2024-05-26T22:52:55.420978+0000","last_fullsized":"2024-05-26T22:52:55.420978+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_clean_scrub_stamp":"2024-05-26T22:52:52.250515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T23:55:58.280032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.423570+0000","last_change":"2024-05-26T22:52:54.336216+0000","last_active":"2024-05-26T22:52:55.423570+0000","last_peered":"2024-05-26T22:52:55.423570+0000","last_clean":"2024-05-26T22:52:55.423570+0000","last_became_active":"2024-05-26T22:52:54.335891+0000","last_became_peered":"2024-05-26T22:52:54.335891+0000","last_unstale":"2024-05-26T22:52:55.423570+0000","last_undegraded":"2024-05-26T22:52:55.423570+0000","last_fullsized":"2024-05-26T22:52:55.423570+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:22:58.350242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.320749+0000","last_change":"2024-05-26T22:52:53.318985+0000","last_active":"2024-05-26T22:52:55.320749+0000","last_peered":"2024-05-26T22:52:55.320749+0000","last_clean":"2024-05-26T22:52:55.320749+0000","last_became_active":"2024-05-26T22:52:53.318620+0000","last_became_peered":"2024-05-26T22:52:53.318620+0000","last_unstale":"2024-05-26T22:52:55.320749+0000","last_undegraded":"2024-05-26T22:52:55.320749+0000","last_fullsized":"2024-05-26T22:52:55.320749+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_clean_scrub_stamp":"2024-05-26T22:52:52.250515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:03:52.876316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.319940+0000","last_change":"2024-05-26T22:52:54.335411+0000","last_active":"2024-05-26T22:52:55.319940+0000","last_peered":"2024-05-26T22:52:55.319940+0000","last_clean":"2024-05-26T22:52:55.319940+0000","last_became_active":"2024-05-26T22:52:54.335093+0000","last_became_peered":"2024-05-26T22:52:54.335093+0000","last_unstale":"2024-05-26T22:52:55.319940+0000","last_undegraded":"2024-05-26T22:52:55.319940+0000","last_fullsized":"2024-05-26T22:52:55.319940+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:16:08.984694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"68'3","reported_seq":17,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.322769+0000","last_change":"2024-05-26T22:52:53.311847+0000","last_active":"2024-05-26T22:52:55.322769+0000","last_peered":"2024-05-26T22:52:55.322769+0000","last_clean":"2024-05-26T22:52:55.322769+0000","last_became_active":"2024-05-26T22:52:53.311508+0000","last_became_peered":"2024-05-26T22:52:53.311508+0000","last_unstale":"2024-05-26T22:52:55.322769+0000","last_undegraded":"2024-05-26T22:52:55.322769+0000","last_fullsized":"2024-05-26T22:52:55.322769+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_clean_scrub_stamp":"2024-05-26T22:52:52.250515+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:40:45.060296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.321245+0000","last_change":"2024-05-26T22:52:54.317305+0000","last_active":"2024-05-26T22:52:55.321245+0000","last_peered":"2024-05-26T22:52:55.321245+0000","last_clean":"2024-05-26T22:52:55.321245+0000","last_became_active":"2024-05-26T22:52:54.316091+0000","last_became_peered":"2024-05-26T22:52:54.316091+0000","last_unstale":"2024-05-26T22:52:55.321245+0000","last_undegraded":"2024-05-26T22:52:55.321245+0000","last_fullsized":"2024-05-26T22:52:55.321245+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:54:46.300450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"68'6","reported_seq":20,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.418428+0000","last_change":"2024-05-26T22:52:53.310828+0000","last_active":"2024-05-26T22:52:55.418428+0000","last_peered":"2024-05-26T22:52:55.418428+0000","last_clean":"2024-05-26T22:52:55.418428+0000","last_became_active":"2024-05-26T22:52:53.310528+0000","last_became_peered":"2024-05-26T22:52:53.310528+0000","last_unstale":"2024-05-26T22:52:55.418428+0000","last_undegraded":"2024-05-26T22:52:55.418428+0000","last_fullsized":"2024-05-26T22:52:55.418428+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_clean_scrub_stamp":"2024-05-26T22:52:52.250515+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:50:13.373742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.422281+0000","last_change":"2024-05-26T22:52:54.342773+0000","last_active":"2024-05-26T22:52:55.422281+0000","last_peered":"2024-05-26T22:52:55.422281+0000","last_clean":"2024-05-26T22:52:55.422281+0000","last_became_active":"2024-05-26T22:52:54.333675+0000","last_became_peered":"2024-05-26T22:52:54.333675+0000","last_unstale":"2024-05-26T22:52:55.422281+0000","last_undegraded":"2024-05-26T22:52:55.422281+0000","last_fullsized":"2024-05-26T22:52:55.422281+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:41:38.018569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"68'2","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.418085+0000","last_change":"2024-05-26T22:52:53.304451+0000","last_active":"2024-05-26T22:52:55.418085+0000","last_peered":"2024-05-26T22:52:55.418085+0000","last_clean":"2024-05-26T22:52:55.418085+0000","last_became_active":"2024-05-26T22:52:53.304007+0000","last_became_peered":"2024-05-26T22:52:53.304007+0000","last_unstale":"2024-05-26T22:52:55.418085+0000","last_undegraded":"2024-05-26T22:52:55.418085+0000","last_fullsized":"2024-05-26T22:52:55.418085+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_clean_scrub_stamp":"2024-05-26T22:52:52.250515+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:42:34.112776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.325229+0000","last_change":"2024-05-26T22:52:54.331458+0000","last_active":"2024-05-26T22:52:55.325229+0000","last_peered":"2024-05-26T22:52:55.325229+0000","last_clean":"2024-05-26T22:52:55.325229+0000","last_became_active":"2024-05-26T22:52:54.331166+0000","last_became_peered":"2024-05-26T22:52:54.331166+0000","last_unstale":"2024-05-26T22:52:55.325229+0000","last_undegraded":"2024-05-26T22:52:55.325229+0000","last_fullsized":"2024-05-26T22:52:55.325229+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:41:38.018569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.319449+0000","last_change":"2024-05-26T22:52:53.312297+0000","last_active":"2024-05-26T22:52:55.319449+0000","last_peered":"2024-05-26T22:52:55.319449+0000","last_clean":"2024-05-26T22:52:55.319449+0000","last_became_active":"2024-05-26T22:52:53.311821+0000","last_became_peered":"2024-05-26T22:52:53.311821+0000","last_unstale":"2024-05-26T22:52:55.319449+0000","last_undegraded":"2024-05-26T22:52:55.319449+0000","last_fullsized":"2024-05-26T22:52:55.319449+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_clean_scrub_stamp":"2024-05-26T22:52:52.250515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:08:29.076391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.328976+0000","last_change":"2024-05-26T22:52:54.324508+0000","last_active":"2024-05-26T22:52:55.328976+0000","last_peered":"2024-05-26T22:52:55.328976+0000","last_clean":"2024-05-26T22:52:55.328976+0000","last_became_active":"2024-05-26T22:52:54.324201+0000","last_became_peered":"2024-05-26T22:52:54.324201+0000","last_unstale":"2024-05-26T22:52:55.328976+0000","last_undegraded":"2024-05-26T22:52:55.328976+0000","last_fullsized":"2024-05-26T22:52:55.328976+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:54:14.144555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.324739+0000","last_change":"2024-05-26T22:52:53.314647+0000","last_active":"2024-05-26T22:52:55.324739+0000","last_peered":"2024-05-26T22:52:55.324739+0000","last_clean":"2024-05-26T22:52:55.324739+0000","last_became_active":"2024-05-26T22:52:53.314189+0000","last_became_peered":"2024-05-26T22:52:53.314189+0000","last_unstale":"2024-05-26T22:52:55.324739+0000","last_undegraded":"2024-05-26T22:52:55.324739+0000","last_fullsized":"2024-05-26T22:52:55.324739+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_clean_scrub_stamp":"2024-05-26T22:52:52.250515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T22:54:25.373084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.322202+0000","last_change":"2024-05-26T22:52:54.313588+0000","last_active":"2024-05-26T22:52:55.322202+0000","last_peered":"2024-05-26T22:52:55.322202+0000","last_clean":"2024-05-26T22:52:55.322202+0000","last_became_active":"2024-05-26T22:52:54.308097+0000","last_became_peered":"2024-05-26T22:52:54.308097+0000","last_unstale":"2024-05-26T22:52:55.322202+0000","last_undegraded":"2024-05-26T22:52:55.322202+0000","last_fullsized":"2024-05-26T22:52:55.322202+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T23:27:03.395614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"68'8","reported_seq":22,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.420383+0000","last_change":"2024-05-26T22:52:53.317131+0000","last_active":"2024-05-26T22:52:55.420383+0000","last_peered":"2024-05-26T22:52:55.420383+0000","last_clean":"2024-05-26T22:52:55.420383+0000","last_became_active":"2024-05-26T22:52:53.316753+0000","last_became_peered":"2024-05-26T22:52:53.316753+0000","last_unstale":"2024-05-26T22:52:55.420383+0000","last_undegraded":"2024-05-26T22:52:55.420383+0000","last_fullsized":"2024-05-26T22:52:55.420383+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_clean_scrub_stamp":"2024-05-26T22:52:52.250515+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:07:18.475048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.420849+0000","last_change":"2024-05-26T22:52:54.325743+0000","last_active":"2024-05-26T22:52:55.420849+0000","last_peered":"2024-05-26T22:52:55.420849+0000","last_clean":"2024-05-26T22:52:55.420849+0000","last_became_active":"2024-05-26T22:52:54.325446+0000","last_became_peered":"2024-05-26T22:52:54.325446+0000","last_unstale":"2024-05-26T22:52:55.420849+0000","last_undegraded":"2024-05-26T22:52:55.420849+0000","last_fullsized":"2024-05-26T22:52:55.420849+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:38:46.157530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"68'1","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.320875+0000","last_change":"2024-05-26T22:52:53.314351+0000","last_active":"2024-05-26T22:52:55.320875+0000","last_peered":"2024-05-26T22:52:55.320875+0000","last_clean":"2024-05-26T22:52:55.320875+0000","last_became_active":"2024-05-26T22:52:53.314071+0000","last_became_peered":"2024-05-26T22:52:53.314071+0000","last_unstale":"2024-05-26T22:52:55.320875+0000","last_undegraded":"2024-05-26T22:52:55.320875+0000","last_fullsized":"2024-05-26T22:52:55.320875+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_clean_scrub_stamp":"2024-05-26T22:52:52.250515+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:42:34.112776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.423758+0000","last_change":"2024-05-26T22:52:54.334664+0000","last_active":"2024-05-26T22:52:55.423758+0000","last_peered":"2024-05-26T22:52:55.423758+0000","last_clean":"2024-05-26T22:52:55.423758+0000","last_became_active":"2024-05-26T22:52:54.334313+0000","last_became_peered":"2024-05-26T22:52:54.334313+0000","last_unstale":"2024-05-26T22:52:55.423758+0000","last_undegraded":"2024-05-26T22:52:55.423758+0000","last_fullsized":"2024-05-26T22:52:55.423758+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:16:35.223557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"68'2","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.420998+0000","last_change":"2024-05-26T22:52:53.320585+0000","last_active":"2024-05-26T22:52:55.420998+0000","last_peered":"2024-05-26T22:52:55.420998+0000","last_clean":"2024-05-26T22:52:55.420998+0000","last_became_active":"2024-05-26T22:52:53.319296+0000","last_became_peered":"2024-05-26T22:52:53.319296+0000","last_unstale":"2024-05-26T22:52:55.420998+0000","last_undegraded":"2024-05-26T22:52:55.420998+0000","last_fullsized":"2024-05-26T22:52:55.420998+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_clean_scrub_stamp":"2024-05-26T22:52:52.250515+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:20:15.243688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.418324+0000","last_change":"2024-05-26T22:52:54.322591+0000","last_active":"2024-05-26T22:52:55.418324+0000","last_peered":"2024-05-26T22:52:55.418324+0000","last_clean":"2024-05-26T22:52:55.418324+0000","last_became_active":"2024-05-26T22:52:54.322307+0000","last_became_peered":"2024-05-26T22:52:54.322307+0000","last_unstale":"2024-05-26T22:52:55.418324+0000","last_undegraded":"2024-05-26T22:52:55.418324+0000","last_fullsized":"2024-05-26T22:52:55.418324+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:20:16.259371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"68'1","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.320401+0000","last_change":"2024-05-26T22:52:53.317832+0000","last_active":"2024-05-26T22:52:55.320401+0000","last_peered":"2024-05-26T22:52:55.320401+0000","last_clean":"2024-05-26T22:52:55.320401+0000","last_became_active":"2024-05-26T22:52:53.317432+0000","last_became_peered":"2024-05-26T22:52:53.317432+0000","last_unstale":"2024-05-26T22:52:55.320401+0000","last_undegraded":"2024-05-26T22:52:55.320401+0000","last_fullsized":"2024-05-26T22:52:55.320401+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_clean_scrub_stamp":"2024-05-26T22:52:52.250515+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:23:54.826711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.322736+0000","last_change":"2024-05-26T22:52:54.326502+0000","last_active":"2024-05-26T22:52:55.322736+0000","last_peered":"2024-05-26T22:52:55.322736+0000","last_clean":"2024-05-26T22:52:55.322736+0000","last_became_active":"2024-05-26T22:52:54.326190+0000","last_became_peered":"2024-05-26T22:52:54.326190+0000","last_unstale":"2024-05-26T22:52:55.322736+0000","last_undegraded":"2024-05-26T22:52:55.322736+0000","last_fullsized":"2024-05-26T22:52:55.322736+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:33:08.243749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.422477+0000","last_change":"2024-05-26T22:52:53.322332+0000","last_active":"2024-05-26T22:52:55.422477+0000","last_peered":"2024-05-26T22:52:55.422477+0000","last_clean":"2024-05-26T22:52:55.422477+0000","last_became_active":"2024-05-26T22:52:53.321697+0000","last_became_peered":"2024-05-26T22:52:53.321697+0000","last_unstale":"2024-05-26T22:52:55.422477+0000","last_undegraded":"2024-05-26T22:52:55.422477+0000","last_fullsized":"2024-05-26T22:52:55.422477+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_clean_scrub_stamp":"2024-05-26T22:52:52.250515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:25:01.606093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.324403+0000","last_change":"2024-05-26T22:52:54.320200+0000","last_active":"2024-05-26T22:52:55.324403+0000","last_peered":"2024-05-26T22:52:55.324403+0000","last_clean":"2024-05-26T22:52:55.324403+0000","last_became_active":"2024-05-26T22:52:54.319640+0000","last_became_peered":"2024-05-26T22:52:54.319640+0000","last_unstale":"2024-05-26T22:52:55.324403+0000","last_undegraded":"2024-05-26T22:52:55.324403+0000","last_fullsized":"2024-05-26T22:52:55.324403+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T23:57:12.586083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.319525+0000","last_change":"2024-05-26T22:52:53.308496+0000","last_active":"2024-05-26T22:52:55.319525+0000","last_peered":"2024-05-26T22:52:55.319525+0000","last_clean":"2024-05-26T22:52:55.319525+0000","last_became_active":"2024-05-26T22:52:53.307650+0000","last_became_peered":"2024-05-26T22:52:53.307650+0000","last_unstale":"2024-05-26T22:52:55.319525+0000","last_undegraded":"2024-05-26T22:52:55.319525+0000","last_fullsized":"2024-05-26T22:52:55.319525+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_clean_scrub_stamp":"2024-05-26T22:52:52.250515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:45:51.874305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.324169+0000","last_change":"2024-05-26T22:52:54.318210+0000","last_active":"2024-05-26T22:52:55.324169+0000","last_peered":"2024-05-26T22:52:55.324169+0000","last_clean":"2024-05-26T22:52:55.324169+0000","last_became_active":"2024-05-26T22:52:54.317916+0000","last_became_peered":"2024-05-26T22:52:54.317916+0000","last_unstale":"2024-05-26T22:52:55.324169+0000","last_undegraded":"2024-05-26T22:52:55.324169+0000","last_fullsized":"2024-05-26T22:52:55.324169+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T23:48:12.418685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.324990+0000","last_change":"2024-05-26T22:52:53.312501+0000","last_active":"2024-05-26T22:52:55.324990+0000","last_peered":"2024-05-26T22:52:55.324990+0000","last_clean":"2024-05-26T22:52:55.324990+0000","last_became_active":"2024-05-26T22:52:53.312214+0000","last_became_peered":"2024-05-26T22:52:53.312214+0000","last_unstale":"2024-05-26T22:52:55.324990+0000","last_undegraded":"2024-05-26T22:52:55.324990+0000","last_fullsized":"2024-05-26T22:52:55.324990+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_clean_scrub_stamp":"2024-05-26T22:52:52.250515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:56:54.130267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.323942+0000","last_change":"2024-05-26T22:52:54.305899+0000","last_active":"2024-05-26T22:52:55.323942+0000","last_peered":"2024-05-26T22:52:55.323942+0000","last_clean":"2024-05-26T22:52:55.323942+0000","last_became_active":"2024-05-26T22:52:54.305336+0000","last_became_peered":"2024-05-26T22:52:54.305336+0000","last_unstale":"2024-05-26T22:52:55.323942+0000","last_undegraded":"2024-05-26T22:52:55.323942+0000","last_fullsized":"2024-05-26T22:52:55.323942+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:33:56.867587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.328947+0000","last_change":"2024-05-26T22:52:53.311245+0000","last_active":"2024-05-26T22:52:55.328947+0000","last_peered":"2024-05-26T22:52:55.328947+0000","last_clean":"2024-05-26T22:52:55.328947+0000","last_became_active":"2024-05-26T22:52:53.310959+0000","last_became_peered":"2024-05-26T22:52:53.310959+0000","last_unstale":"2024-05-26T22:52:55.328947+0000","last_undegraded":"2024-05-26T22:52:55.328947+0000","last_fullsized":"2024-05-26T22:52:55.328947+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_clean_scrub_stamp":"2024-05-26T22:52:52.250515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:16:07.969012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.329984+0000","last_change":"2024-05-26T22:52:54.318907+0000","last_active":"2024-05-26T22:52:55.329984+0000","last_peered":"2024-05-26T22:52:55.329984+0000","last_clean":"2024-05-26T22:52:55.329984+0000","last_became_active":"2024-05-26T22:52:54.318626+0000","last_became_peered":"2024-05-26T22:52:54.318626+0000","last_unstale":"2024-05-26T22:52:55.329984+0000","last_undegraded":"2024-05-26T22:52:55.329984+0000","last_fullsized":"2024-05-26T22:52:55.329984+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T22:55:42.930355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"68'2","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.320893+0000","last_change":"2024-05-26T22:52:53.306902+0000","last_active":"2024-05-26T22:52:55.320893+0000","last_peered":"2024-05-26T22:52:55.320893+0000","last_clean":"2024-05-26T22:52:55.320893+0000","last_became_active":"2024-05-26T22:52:53.306509+0000","last_became_peered":"2024-05-26T22:52:53.306509+0000","last_unstale":"2024-05-26T22:52:55.320893+0000","last_undegraded":"2024-05-26T22:52:55.320893+0000","last_fullsized":"2024-05-26T22:52:55.320893+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_clean_scrub_stamp":"2024-05-26T22:52:52.250515+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:05:59.630328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.325223+0000","last_change":"2024-05-26T22:52:54.326978+0000","last_active":"2024-05-26T22:52:55.325223+0000","last_peered":"2024-05-26T22:52:55.325223+0000","last_clean":"2024-05-26T22:52:55.325223+0000","last_became_active":"2024-05-26T22:52:54.324166+0000","last_became_peered":"2024-05-26T22:52:54.324166+0000","last_unstale":"2024-05-26T22:52:55.325223+0000","last_undegraded":"2024-05-26T22:52:55.325223+0000","last_fullsized":"2024-05-26T22:52:55.325223+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:06:23.444659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"69'1","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.328217+0000","last_change":"2024-05-26T22:52:53.311315+0000","last_active":"2024-05-26T22:52:55.328217+0000","last_peered":"2024-05-26T22:52:55.328217+0000","last_clean":"2024-05-26T22:52:55.328217+0000","last_became_active":"2024-05-26T22:52:53.310978+0000","last_became_peered":"2024-05-26T22:52:53.310978+0000","last_unstale":"2024-05-26T22:52:55.328217+0000","last_undegraded":"2024-05-26T22:52:55.328217+0000","last_fullsized":"2024-05-26T22:52:55.328217+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_clean_scrub_stamp":"2024-05-26T22:52:52.250515+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:49:26.486812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.422767+0000","last_change":"2024-05-26T22:52:54.340034+0000","last_active":"2024-05-26T22:52:55.422767+0000","last_peered":"2024-05-26T22:52:55.422767+0000","last_clean":"2024-05-26T22:52:55.422767+0000","last_became_active":"2024-05-26T22:52:54.336325+0000","last_became_peered":"2024-05-26T22:52:54.336325+0000","last_unstale":"2024-05-26T22:52:55.422767+0000","last_undegraded":"2024-05-26T22:52:55.422767+0000","last_fullsized":"2024-05-26T22:52:55.422767+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:31:19.880810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.321599+0000","last_change":"2024-05-26T22:52:53.310841+0000","last_active":"2024-05-26T22:52:55.321599+0000","last_peered":"2024-05-26T22:52:55.321599+0000","last_clean":"2024-05-26T22:52:55.321599+0000","last_became_active":"2024-05-26T22:52:53.310346+0000","last_became_peered":"2024-05-26T22:52:53.310346+0000","last_unstale":"2024-05-26T22:52:55.321599+0000","last_undegraded":"2024-05-26T22:52:55.321599+0000","last_fullsized":"2024-05-26T22:52:55.321599+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_clean_scrub_stamp":"2024-05-26T22:52:52.250515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:06:45.944808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.423725+0000","last_change":"2024-05-26T22:52:54.337947+0000","last_active":"2024-05-26T22:52:55.423725+0000","last_peered":"2024-05-26T22:52:55.423725+0000","last_clean":"2024-05-26T22:52:55.423725+0000","last_became_active":"2024-05-26T22:52:54.336813+0000","last_became_peered":"2024-05-26T22:52:54.336813+0000","last_unstale":"2024-05-26T22:52:55.423725+0000","last_undegraded":"2024-05-26T22:52:55.423725+0000","last_fullsized":"2024-05-26T22:52:55.423725+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:13:23.082826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.329736+0000","last_change":"2024-05-26T22:52:53.316773+0000","last_active":"2024-05-26T22:52:55.329736+0000","last_peered":"2024-05-26T22:52:55.329736+0000","last_clean":"2024-05-26T22:52:55.329736+0000","last_became_active":"2024-05-26T22:52:53.316482+0000","last_became_peered":"2024-05-26T22:52:53.316482+0000","last_unstale":"2024-05-26T22:52:55.329736+0000","last_undegraded":"2024-05-26T22:52:55.329736+0000","last_fullsized":"2024-05-26T22:52:55.329736+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_clean_scrub_stamp":"2024-05-26T22:52:52.250515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:42:26.418928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.322272+0000","last_change":"2024-05-26T22:52:54.310732+0000","last_active":"2024-05-26T22:52:55.322272+0000","last_peered":"2024-05-26T22:52:55.322272+0000","last_clean":"2024-05-26T22:52:55.322272+0000","last_became_active":"2024-05-26T22:52:54.308845+0000","last_became_peered":"2024-05-26T22:52:54.308845+0000","last_unstale":"2024-05-26T22:52:55.322272+0000","last_undegraded":"2024-05-26T22:52:55.322272+0000","last_fullsized":"2024-05-26T22:52:55.322272+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:53:49.116433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.320462+0000","last_change":"2024-05-26T22:52:53.317077+0000","last_active":"2024-05-26T22:52:55.320462+0000","last_peered":"2024-05-26T22:52:55.320462+0000","last_clean":"2024-05-26T22:52:55.320462+0000","last_became_active":"2024-05-26T22:52:53.316702+0000","last_became_peered":"2024-05-26T22:52:53.316702+0000","last_unstale":"2024-05-26T22:52:55.320462+0000","last_undegraded":"2024-05-26T22:52:55.320462+0000","last_fullsized":"2024-05-26T22:52:55.320462+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_clean_scrub_stamp":"2024-05-26T22:52:52.250515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:30:12.377121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.319280+0000","last_change":"2024-05-26T22:52:54.323551+0000","last_active":"2024-05-26T22:52:55.319280+0000","last_peered":"2024-05-26T22:52:55.319280+0000","last_clean":"2024-05-26T22:52:55.319280+0000","last_became_active":"2024-05-26T22:52:54.323142+0000","last_became_peered":"2024-05-26T22:52:54.323142+0000","last_unstale":"2024-05-26T22:52:55.319280+0000","last_undegraded":"2024-05-26T22:52:55.319280+0000","last_fullsized":"2024-05-26T22:52:55.319280+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:42:27.434611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.417934+0000","last_change":"2024-05-26T22:52:53.319126+0000","last_active":"2024-05-26T22:52:55.417934+0000","last_peered":"2024-05-26T22:52:55.417934+0000","last_clean":"2024-05-26T22:52:55.417934+0000","last_became_active":"2024-05-26T22:52:53.318832+0000","last_became_peered":"2024-05-26T22:52:53.318832+0000","last_unstale":"2024-05-26T22:52:55.417934+0000","last_undegraded":"2024-05-26T22:52:55.417934+0000","last_fullsized":"2024-05-26T22:52:55.417934+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_clean_scrub_stamp":"2024-05-26T22:52:52.250515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:34:41.388075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.418053+0000","last_change":"2024-05-26T22:52:54.320025+0000","last_active":"2024-05-26T22:52:55.418053+0000","last_peered":"2024-05-26T22:52:55.418053+0000","last_clean":"2024-05-26T22:52:55.418053+0000","last_became_active":"2024-05-26T22:52:54.319106+0000","last_became_peered":"2024-05-26T22:52:54.319106+0000","last_unstale":"2024-05-26T22:52:55.418053+0000","last_undegraded":"2024-05-26T22:52:55.418053+0000","last_fullsized":"2024-05-26T22:52:55.418053+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:03:53.891999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.420039+0000","last_change":"2024-05-26T22:52:53.322311+0000","last_active":"2024-05-26T22:52:55.420039+0000","last_peered":"2024-05-26T22:52:55.420039+0000","last_clean":"2024-05-26T22:52:55.420039+0000","last_became_active":"2024-05-26T22:52:53.321876+0000","last_became_peered":"2024-05-26T22:52:53.321876+0000","last_unstale":"2024-05-26T22:52:55.420039+0000","last_undegraded":"2024-05-26T22:52:55.420039+0000","last_fullsized":"2024-05-26T22:52:55.420039+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_clean_scrub_stamp":"2024-05-26T22:52:52.250515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:03:52.876316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.318869+0000","last_change":"2024-05-26T22:52:54.310194+0000","last_active":"2024-05-26T22:52:55.318869+0000","last_peered":"2024-05-26T22:52:55.318869+0000","last_clean":"2024-05-26T22:52:55.318869+0000","last_became_active":"2024-05-26T22:52:54.309341+0000","last_became_peered":"2024-05-26T22:52:54.309341+0000","last_unstale":"2024-05-26T22:52:55.318869+0000","last_undegraded":"2024-05-26T22:52:55.318869+0000","last_fullsized":"2024-05-26T22:52:55.318869+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:26:38.417192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.319168+0000","last_change":"2024-05-26T22:52:53.313078+0000","last_active":"2024-05-26T22:52:55.319168+0000","last_peered":"2024-05-26T22:52:55.319168+0000","last_clean":"2024-05-26T22:52:55.319168+0000","last_became_active":"2024-05-26T22:52:53.312819+0000","last_became_peered":"2024-05-26T22:52:53.312819+0000","last_unstale":"2024-05-26T22:52:55.319168+0000","last_undegraded":"2024-05-26T22:52:55.319168+0000","last_fullsized":"2024-05-26T22:52:55.319168+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_clean_scrub_stamp":"2024-05-26T22:52:52.250515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:50:13.373742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.322456+0000","last_change":"2024-05-26T22:52:54.317258+0000","last_active":"2024-05-26T22:52:55.322456+0000","last_peered":"2024-05-26T22:52:55.322456+0000","last_clean":"2024-05-26T22:52:55.322456+0000","last_became_active":"2024-05-26T22:52:54.316945+0000","last_became_peered":"2024-05-26T22:52:54.316945+0000","last_unstale":"2024-05-26T22:52:55.322456+0000","last_undegraded":"2024-05-26T22:52:55.322456+0000","last_fullsized":"2024-05-26T22:52:55.322456+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:53.266198+0000","last_clean_scrub_stamp":"2024-05-26T22:52:53.266198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:37:24.149835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"68'2","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.319487+0000","last_change":"2024-05-26T22:52:53.307081+0000","last_active":"2024-05-26T22:52:55.319487+0000","last_peered":"2024-05-26T22:52:55.319487+0000","last_clean":"2024-05-26T22:52:55.319487+0000","last_became_active":"2024-05-26T22:52:53.306701+0000","last_became_peered":"2024-05-26T22:52:53.306701+0000","last_unstale":"2024-05-26T22:52:55.319487+0000","last_undegraded":"2024-05-26T22:52:55.319487+0000","last_fullsized":"2024-05-26T22:52:55.319487+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:52:52.250515+0000","last_clean_scrub_stamp":"2024-05-26T22:52:52.250515+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:13:44.543717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":261,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-05-26T22:52:55.421918+0000","last_change":"2024-05-26T22:48:12.068917+0000","last_active":"2024-05-26T22:52:55.421918+0000","last_peered":"2024-05-26T22:52:55.421918+0000","last_clean":"2024-05-26T22:52:55.421918+0000","last_became_active":"2024-05-26T22:48:12.065934+0000","last_became_peered":"2024-05-26T22:48:12.065934+0000","last_unstale":"2024-05-26T22:52:55.421918+0000","last_undegraded":"2024-05-26T22:52:55.421918+0000","last_fullsized":"2024-05-26T22:52:55.421918+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:10.628122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:10.628122+0000","last_clean_scrub_stamp":"2024-05-26T22:48:10.628122+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:14:58.895911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574908,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32584,"kb_used_data":1396,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339256,"statfs":{"total":96636764160,"available":96603398144,"internally_reserved":0,"allocated":1429504,"data_stored":942701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574908,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32000,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339840,"statfs":{"total":96636764160,"available":96603996160,"internally_reserved":0,"allocated":835584,"data_stored":351809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574909,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31996,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339844,"statfs":{"total":96636764160,"available":96604000256,"internally_reserved":0,"allocated":831488,"data_stored":351807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574908,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31996,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339844,"statfs":{"total":96636764160,"available":96604000256,"internally_reserved":0,"allocated":831488,"data_stored":351787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574908,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32008,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339832,"statfs":{"total":96636764160,"available":96603987968,"internally_reserved":0,"allocated":847872,"data_stored":358129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574908,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32020,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339820,"statfs":{"total":96636764160,"available":96603975680,"internally_reserved":0,"allocated":851968,"data_stored":359629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574908,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32012,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339828,"statfs":{"total":96636764160,"available":96603983872,"internally_reserved":0,"allocated":847872,"data_stored":353945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574908,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32576,"kb_used_data":1388,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339264,"statfs":{"total":96636764160,"available":96603406336,"internally_reserved":0,"allocated":1421312,"data_stored":942085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-26T22:52:59.186 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-26T22:52:59.186 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-26T22:52:59.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.435+0000 7f0aa4a00640 1 Processor -- start 2024-05-26T22:52:59.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.435+0000 7f0aa4a00640 1 -- start start 2024-05-26T22:52:59.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.437+0000 7f0aa4a00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f0aa0145a10 0x7f0aa0072180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:59.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.437+0000 7f0aa4a00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0aa0072d60 con 0x7f0aa006d960 2024-05-26T22:52:59.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.437+0000 7f0aa4a00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0aa0072a80 con 0x7f0aa00712d0 2024-05-26T22:52:59.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.437+0000 7f0aa4a00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f0aa015c760 con 0x7f0aa0145a10 2024-05-26T22:52:59.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.437+0000 7f0a9e800640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f0aa0145a10 0x7f0aa0072180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:59.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.437+0000 7f0a9f200640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f0aa006d960 0x7f0aa00cf590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:42300/0 (socket says 172.21.15.131:42300) 2024-05-26T22:52:59.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.437+0000 7f0a9f200640 1 -- 172.21.15.131:0/4132068733 learned_addr learned my addr 172.21.15.131:0/4132068733 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:52:59.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.438+0000 7f0a9e800640 1 -- 172.21.15.131:0/4132068733 >> v1:172.21.15.187:6789/0 conn(0x7f0aa00712d0 legacy=0x7f0aa006d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:59.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.438+0000 7f0a9e800640 1 -- 172.21.15.131:0/4132068733 >> v1:172.21.15.131:6789/0 conn(0x7f0aa006d960 legacy=0x7f0aa00cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:59.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.438+0000 7f0a9e800640 1 -- 172.21.15.131:0/4132068733 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0aa00ceab0 con 0x7f0aa0145a10 2024-05-26T22:52:59.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.439+0000 7f0a9e800640 1 --2- 172.21.15.131:0/4132068733 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0aa0145a10 0x7f0aa0072180 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f0a900097a0 tx=0x7f0a9002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf0ec5fe6999dbf2 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:59.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.439+0000 7f0a9de00640 1 -- 172.21.15.131:0/4132068733 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a90037070 con 0x7f0aa0145a10 2024-05-26T22:52:59.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.439+0000 7f0a9de00640 1 -- 172.21.15.131:0/4132068733 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a90032440 con 0x7f0aa0145a10 2024-05-26T22:52:59.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.439+0000 7f0a9de00640 1 -- 172.21.15.131:0/4132068733 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a90032810 con 0x7f0aa0145a10 2024-05-26T22:52:59.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.440+0000 7f0aa4a00640 1 -- 172.21.15.131:0/4132068733 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0aa0145a10 msgr2=0x7f0aa0072180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:59.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.440+0000 7f0aa4a00640 1 --2- 172.21.15.131:0/4132068733 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0aa0145a10 0x7f0aa0072180 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f0a900097a0 tx=0x7f0a9002a1e0 comp rx=0 tx=0).stop 2024-05-26T22:52:59.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.440+0000 7f0aa4a00640 1 -- 172.21.15.131:0/4132068733 shutdown_connections 2024-05-26T22:52:59.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.440+0000 7f0aa4a00640 1 --2- 172.21.15.131:0/4132068733 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0aa0145a10 0x7f0aa0072180 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:59.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.440+0000 7f0aa4a00640 1 -- 172.21.15.131:0/4132068733 >> 172.21.15.131:0/4132068733 conn(0x7f0aa00b5660 msgr2=0x7f0aa00b5a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:52:59.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.440+0000 7f0aa4a00640 1 -- 172.21.15.131:0/4132068733 shutdown_connections 2024-05-26T22:52:59.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.440+0000 7f0aa4a00640 1 -- 172.21.15.131:0/4132068733 wait complete. 2024-05-26T22:52:59.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.442+0000 7f0aa4a00640 1 Processor -- start 2024-05-26T22:52:59.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.442+0000 7f0aa4a00640 1 -- start start 2024-05-26T22:52:59.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.443+0000 7f0aa4a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0aa006d960 0x7f0aa00ac810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:59.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.443+0000 7f0aa4a00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0aa00712d0 0x7f0aa00b4540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:59.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.443+0000 7f0aa4a00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0aa0145a10 0x7f0aa014f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:59.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.443+0000 7f0aa4a00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f0aa0145df0 con 0x7f0aa006d960 2024-05-26T22:52:59.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.443+0000 7f0aa4a00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0aa00cf070 con 0x7f0aa00712d0 2024-05-26T22:52:59.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.443+0000 7f0aa4a00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f0aa00726c0 con 0x7f0aa0145a10 2024-05-26T22:52:59.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.443+0000 7f0a9e800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0aa006d960 0x7f0aa00ac810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:59.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.443+0000 7f0a9e800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0aa006d960 0x7f0aa00ac810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48494/0 (socket says 172.21.15.131:48494) 2024-05-26T22:52:59.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.443+0000 7f0a9e800640 1 -- 172.21.15.131:0/4287253315 learned_addr learned my addr 172.21.15.131:0/4287253315 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:52:59.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.444+0000 7f0a95e00640 1 --2- 172.21.15.131:0/4287253315 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0aa00712d0 0x7f0aa00b4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:59.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.444+0000 7f0a9e800640 1 -- 172.21.15.131:0/4287253315 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0aa0145a10 msgr2=0x7f0aa014f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:59.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.444+0000 7f0a9e800640 1 --2- 172.21.15.131:0/4287253315 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0aa0145a10 0x7f0aa014f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:59.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.444+0000 7f0a9e800640 1 -- 172.21.15.131:0/4287253315 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0aa00712d0 msgr2=0x7f0aa00b4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:59.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.444+0000 7f0a9e800640 1 --2- 172.21.15.131:0/4287253315 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0aa00712d0 0x7f0aa00b4540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:59.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.444+0000 7f0a9e800640 1 -- 172.21.15.131:0/4287253315 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0aa00c9630 con 0x7f0aa006d960 2024-05-26T22:52:59.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.444+0000 7f0a9e800640 1 --2- 172.21.15.131:0/4287253315 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0aa006d960 0x7f0aa00ac810 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f0a900097a0 tx=0x7f0a900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:59.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.444+0000 7f0a97400640 1 -- 172.21.15.131:0/4287253315 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a90041070 con 0x7f0aa006d960 2024-05-26T22:52:59.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.445+0000 7f0a97400640 1 -- 172.21.15.131:0/4287253315 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a9003c870 con 0x7f0aa006d960 2024-05-26T22:52:59.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.445+0000 7f0a97400640 1 -- 172.21.15.131:0/4287253315 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a90037040 con 0x7f0aa006d960 2024-05-26T22:52:59.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.445+0000 7f0aa4a00640 1 -- 172.21.15.131:0/4287253315 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0aa00c9910 con 0x7f0aa006d960 2024-05-26T22:52:59.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.445+0000 7f0aa4a00640 1 -- 172.21.15.131:0/4287253315 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0aa014db40 con 0x7f0aa006d960 2024-05-26T22:52:59.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.446+0000 7f0aa4a00640 1 -- 172.21.15.131:0/4287253315 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0aa014d650 con 0x7f0aa006d960 2024-05-26T22:52:59.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.447+0000 7f0a97400640 1 -- 172.21.15.131:0/4287253315 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0a90044070 con 0x7f0aa006d960 2024-05-26T22:52:59.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.448+0000 7f0a97400640 1 --2- 172.21.15.131:0/4287253315 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f0a7c06be30 0x7f0a7c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:59.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.448+0000 7f0a97400640 1 -- 172.21.15.131:0/4287253315 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7f0a900abb80 con 0x7f0aa006d960 2024-05-26T22:52:59.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.448+0000 7f0a95e00640 1 --2- 172.21.15.131:0/4287253315 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f0a7c06be30 0x7f0a7c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:59.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.449+0000 7f0a95e00640 1 --2- 172.21.15.131:0/4287253315 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f0a7c06be30 0x7f0a7c06e2f0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f0aa015b6b0 tx=0x7f0a88009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:59.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.452+0000 7f0a97400640 1 -- 172.21.15.131:0/4287253315 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f0a9007a5b0 con 0x7f0aa006d960 2024-05-26T22:52:59.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.724+0000 7f0aa4a00640 1 -- 172.21.15.131:0/4287253315 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f0aa00694e0 con 0x7f0aa006d960 2024-05-26T22:52:59.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.727+0000 7f0a97400640 1 -- 172.21.15.131:0/4287253315 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v16) ==== 190+0+147 (secure 0 0 0) 0x7f0a90079eb0 con 0x7f0aa006d960 2024-05-26T22:52:59.727 INFO:teuthology.orchestra.run.smithi131.stdout:[client.0] 2024-05-26T22:52:59.728 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQD2u1NmSGVWChAAH/wq3iaUs6ZELRhLKhmURQ== 2024-05-26T22:52:59.728 INFO:teuthology.orchestra.run.smithi131.stdout: caps mds = "allow" 2024-05-26T22:52:59.728 INFO:teuthology.orchestra.run.smithi131.stdout: caps mon = "allow r" 2024-05-26T22:52:59.728 INFO:teuthology.orchestra.run.smithi131.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-05-26T22:52:59.728 INFO:teuthology.orchestra.run.smithi131.stderr:updated caps for client.0 2024-05-26T22:52:59.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.730+0000 7f0aa4a00640 1 -- 172.21.15.131:0/4287253315 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f0a7c06be30 msgr2=0x7f0a7c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:59.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.730+0000 7f0aa4a00640 1 --2- 172.21.15.131:0/4287253315 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f0a7c06be30 0x7f0a7c06e2f0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f0aa015b6b0 tx=0x7f0a88009040 comp rx=0 tx=0).stop 2024-05-26T22:52:59.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.730+0000 7f0aa4a00640 1 -- 172.21.15.131:0/4287253315 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0aa006d960 msgr2=0x7f0aa00ac810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:59.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.730+0000 7f0aa4a00640 1 --2- 172.21.15.131:0/4287253315 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0aa006d960 0x7f0aa00ac810 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f0a900097a0 tx=0x7f0a900046c0 comp rx=0 tx=0).stop 2024-05-26T22:52:59.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.730+0000 7f0aa4a00640 1 -- 172.21.15.131:0/4287253315 shutdown_connections 2024-05-26T22:52:59.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.730+0000 7f0aa4a00640 1 --2- 172.21.15.131:0/4287253315 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0aa0145a10 0x7f0aa014f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:59.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.730+0000 7f0aa4a00640 1 --2- 172.21.15.131:0/4287253315 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f0a7c06be30 0x7f0a7c06e2f0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:59.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.730+0000 7f0aa4a00640 1 --2- 172.21.15.131:0/4287253315 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0aa00712d0 0x7f0aa00b4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:59.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.730+0000 7f0aa4a00640 1 --2- 172.21.15.131:0/4287253315 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0aa006d960 0x7f0aa00ac810 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:59.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.730+0000 7f0aa4a00640 1 -- 172.21.15.131:0/4287253315 >> 172.21.15.131:0/4287253315 conn(0x7f0aa00b5660 msgr2=0x7f0aa015e9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:52:59.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.731+0000 7f0aa4a00640 1 -- 172.21.15.131:0/4287253315 shutdown_connections 2024-05-26T22:52:59.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.731+0000 7f0aa4a00640 1 -- 172.21.15.131:0/4287253315 wait complete. 2024-05-26T22:52:59.751 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-26T22:52:59.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.957+0000 7f24e1400640 1 Processor -- start 2024-05-26T22:52:59.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.958+0000 7f24e1400640 1 -- start start 2024-05-26T22:52:59.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.959+0000 7f24e1400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f24dc06ee20 0x7f24dc0b3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:59.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.959+0000 7f24e1400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f24dc1569d0 0x7f24dc0af790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:59.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.959+0000 7f24e1400640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24dc15cb20 con 0x7f24dc0ad250 2024-05-26T22:52:59.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.959+0000 7f24e1400640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f24dc15ce00 con 0x7f24dc06ee20 2024-05-26T22:52:59.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.959+0000 7f24e1400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f24dc15d0e0 con 0x7f24dc1569d0 2024-05-26T22:52:59.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.959+0000 7f24da800640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f24dc06ee20 0x7f24dc0b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:59.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.959+0000 7f24d1e00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f24dc0ad250 0x7f24dc1562c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:42316/0 (socket says 172.21.15.131:42316) 2024-05-26T22:52:59.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.959+0000 7f24d1e00640 1 -- 172.21.15.131:0/1040129633 learned_addr learned my addr 172.21.15.131:0/1040129633 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:52:59.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.960+0000 7f24db200640 1 --2- 172.21.15.131:0/1040129633 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f24dc1569d0 0x7f24dc0af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:59.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.960+0000 7f24da800640 1 -- 172.21.15.131:0/1040129633 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f24dc1569d0 msgr2=0x7f24dc0af790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:52:59.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.960+0000 7f24da800640 1 --2- 172.21.15.131:0/1040129633 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f24dc1569d0 0x7f24dc0af790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:59.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.960+0000 7f24da800640 1 -- 172.21.15.131:0/1040129633 >> v1:172.21.15.131:6789/0 conn(0x7f24dc0ad250 legacy=0x7f24dc1562c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:59.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.960+0000 7f24da800640 1 -- 172.21.15.131:0/1040129633 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24dc15d3c0 con 0x7f24dc06ee20 2024-05-26T22:52:59.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.960+0000 7f24d9e00640 1 -- 172.21.15.131:0/1040129633 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3577708285 0 0) 0x7f24dc15cb20 con 0x7f24dc0ad250 2024-05-26T22:52:59.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.961+0000 7f24da800640 1 --2- 172.21.15.131:0/1040129633 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f24dc06ee20 0x7f24dc0b3190 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f24cc0097a0 tx=0x7f24cc02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62a7c40addd1b7f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:59.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.961+0000 7f24d9e00640 1 -- 172.21.15.131:0/1040129633 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24cc037070 con 0x7f24dc06ee20 2024-05-26T22:52:59.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.962+0000 7f24d9e00640 1 -- 172.21.15.131:0/1040129633 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24cc032440 con 0x7f24dc06ee20 2024-05-26T22:52:59.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.962+0000 7f24d9e00640 1 -- 172.21.15.131:0/1040129633 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24cc032810 con 0x7f24dc06ee20 2024-05-26T22:52:59.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.962+0000 7f24e1400640 1 -- 172.21.15.131:0/1040129633 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f24dc06ee20 msgr2=0x7f24dc0b3190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:59.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.962+0000 7f24e1400640 1 --2- 172.21.15.131:0/1040129633 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f24dc06ee20 0x7f24dc0b3190 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f24cc0097a0 tx=0x7f24cc02a180 comp rx=0 tx=0).stop 2024-05-26T22:52:59.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.962+0000 7f24e1400640 1 -- 172.21.15.131:0/1040129633 shutdown_connections 2024-05-26T22:52:59.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.962+0000 7f24e1400640 1 --2- 172.21.15.131:0/1040129633 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f24dc1569d0 0x7f24dc0af790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:59.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.962+0000 7f24e1400640 1 --2- 172.21.15.131:0/1040129633 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f24dc06ee20 0x7f24dc0b3190 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:59.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.962+0000 7f24e1400640 1 -- 172.21.15.131:0/1040129633 >> 172.21.15.131:0/1040129633 conn(0x7f24dc0b1e90 msgr2=0x7f24dc0b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:52:59.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.962+0000 7f24e1400640 1 -- 172.21.15.131:0/1040129633 shutdown_connections 2024-05-26T22:52:59.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.962+0000 7f24e1400640 1 -- 172.21.15.131:0/1040129633 wait complete. 2024-05-26T22:52:59.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.964+0000 7f24e1400640 1 Processor -- start 2024-05-26T22:52:59.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.964+0000 7f24e1400640 1 -- start start 2024-05-26T22:52:59.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.965+0000 7f24e1400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f24dc06ee20 0x7f24dc071590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:59.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.965+0000 7f24e1400640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f24dc0ad250 0x7f24dc143ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:59.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.965+0000 7f24e1400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f24dc1569d0 0x7f24dc09dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:59.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.965+0000 7f24e1400640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f24dc15c900 con 0x7f24dc06ee20 2024-05-26T22:52:59.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.965+0000 7f24e1400640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f24dc15c780 con 0x7f24dc0ad250 2024-05-26T22:52:59.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.965+0000 7f24e1400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f24dc06f200 con 0x7f24dc1569d0 2024-05-26T22:52:59.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.965+0000 7f24da800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f24dc06ee20 0x7f24dc071590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:59.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.965+0000 7f24da800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f24dc06ee20 0x7f24dc071590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48516/0 (socket says 172.21.15.131:48516) 2024-05-26T22:52:59.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.965+0000 7f24da800640 1 -- 172.21.15.131:0/1432626641 learned_addr learned my addr 172.21.15.131:0/1432626641 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:52:59.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.966+0000 7f24da800640 1 -- 172.21.15.131:0/1432626641 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f24dc1569d0 msgr2=0x7f24dc09dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:59.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.966+0000 7f24da800640 1 --2- 172.21.15.131:0/1432626641 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f24dc1569d0 0x7f24dc09dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:59.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.966+0000 7f24da800640 1 -- 172.21.15.131:0/1432626641 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f24dc0ad250 msgr2=0x7f24dc143ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:52:59.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.966+0000 7f24da800640 1 --2- 172.21.15.131:0/1432626641 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f24dc0ad250 0x7f24dc143ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:52:59.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.966+0000 7f24da800640 1 -- 172.21.15.131:0/1432626641 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24dc0724f0 con 0x7f24dc06ee20 2024-05-26T22:52:59.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.966+0000 7f24da800640 1 --2- 172.21.15.131:0/1432626641 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f24dc06ee20 0x7f24dc071590 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f24cc000c00 tx=0x7f24cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:59.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.966+0000 7f24d3400640 1 -- 172.21.15.131:0/1432626641 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24cc037040 con 0x7f24dc06ee20 2024-05-26T22:52:59.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.966+0000 7f24d3400640 1 -- 172.21.15.131:0/1432626641 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24cc042070 con 0x7f24dc06ee20 2024-05-26T22:52:59.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.966+0000 7f24d3400640 1 -- 172.21.15.131:0/1432626641 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24cc030bb0 con 0x7f24dc06ee20 2024-05-26T22:52:59.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.966+0000 7f24e1400640 1 -- 172.21.15.131:0/1432626641 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24dc071c50 con 0x7f24dc06ee20 2024-05-26T22:52:59.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.967+0000 7f24e1400640 1 -- 172.21.15.131:0/1432626641 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24dc072210 con 0x7f24dc06ee20 2024-05-26T22:52:59.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.968+0000 7f24e1400640 1 -- 172.21.15.131:0/1432626641 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24dc0694e0 con 0x7f24dc06ee20 2024-05-26T22:52:59.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.969+0000 7f24d3400640 1 -- 172.21.15.131:0/1432626641 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f24cc047460 con 0x7f24dc06ee20 2024-05-26T22:52:59.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.970+0000 7f24d3400640 1 --2- 172.21.15.131:0/1432626641 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f24b806be60 0x7f24b806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:52:59.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.970+0000 7f24d3400640 1 -- 172.21.15.131:0/1432626641 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7f24cc0ab9f0 con 0x7f24dc06ee20 2024-05-26T22:52:59.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.970+0000 7f24d1e00640 1 --2- 172.21.15.131:0/1432626641 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f24b806be60 0x7f24b806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:52:59.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.971+0000 7f24d1e00640 1 --2- 172.21.15.131:0/1432626641 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f24b806be60 0x7f24b806e320 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f24c4007fd0 tx=0x7f24c4007920 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:52:59.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:52:59.974+0000 7f24d3400640 1 -- 172.21.15.131:0/1432626641 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f24cc032e20 con 0x7f24dc06ee20 2024-05-26T22:53:00.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.207+0000 7f24e1400640 1 -- 172.21.15.131:0/1432626641 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f24dc155b20 con 0x7f24dc06ee20 2024-05-26T22:53:00.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.208+0000 7f24d3400640 1 -- 172.21.15.131:0/1432626641 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) ==== 94+0+5195 (secure 0 0 0) 0x7f24cc07a430 con 0x7f24dc06ee20 2024-05-26T22:53:00.208 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:53:00.208 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":64,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5359,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/4204396845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6833","nonce":4204396845},{"type":"v1","addr":"172.21.15.187:6835","nonce":4204396845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59},{"gid":6861,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6836/4223534499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":4223534499},{"type":"v1","addr":"172.21.15.131:6836","nonce":4223534499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58},{"gid":6864,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/1418741551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6835","nonce":1418741551},{"type":"v1","addr":"172.21.15.131:6837","nonce":1418741551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-26T22:52:53.755842+0000","modified":"2024-05-26T22:52:56.809807+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6876},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6876":{"gid":6876,"name":"d","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6834/3680999672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3680999672},{"type":"v1","addr":"172.21.15.187:6834","nonce":3680999672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6876,"qdb_cluster":[6876]},"id":5}]} 2024-05-26T22:53:00.208 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 64 2024-05-26T22:53:00.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.211+0000 7f24e1400640 1 -- 172.21.15.131:0/1432626641 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f24b806be60 msgr2=0x7f24b806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:00.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.211+0000 7f24e1400640 1 --2- 172.21.15.131:0/1432626641 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f24b806be60 0x7f24b806e320 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f24c4007fd0 tx=0x7f24c4007920 comp rx=0 tx=0).stop 2024-05-26T22:53:00.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.211+0000 7f24e1400640 1 -- 172.21.15.131:0/1432626641 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f24dc06ee20 msgr2=0x7f24dc071590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:00.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.211+0000 7f24e1400640 1 --2- 172.21.15.131:0/1432626641 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f24dc06ee20 0x7f24dc071590 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f24cc000c00 tx=0x7f24cc0046c0 comp rx=0 tx=0).stop 2024-05-26T22:53:00.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.211+0000 7f24e1400640 1 -- 172.21.15.131:0/1432626641 shutdown_connections 2024-05-26T22:53:00.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.211+0000 7f24e1400640 1 --2- 172.21.15.131:0/1432626641 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f24dc1569d0 0x7f24dc09dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:00.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.211+0000 7f24e1400640 1 --2- 172.21.15.131:0/1432626641 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f24b806be60 0x7f24b806e320 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:00.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.211+0000 7f24e1400640 1 --2- 172.21.15.131:0/1432626641 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f24dc0ad250 0x7f24dc143ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:00.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.211+0000 7f24e1400640 1 --2- 172.21.15.131:0/1432626641 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f24dc06ee20 0x7f24dc071590 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:00.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.211+0000 7f24e1400640 1 -- 172.21.15.131:0/1432626641 >> 172.21.15.131:0/1432626641 conn(0x7f24dc0b1e90 msgr2=0x7f24dc15ee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:00.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.211+0000 7f24e1400640 1 -- 172.21.15.131:0/1432626641 shutdown_connections 2024-05-26T22:53:00.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.211+0000 7f24e1400640 1 -- 172.21.15.131:0/1432626641 wait complete. 2024-05-26T22:53:00.251 INFO:tasks.ceph.mds.d.smithi187.stderr:2024-05-26T22:53:00.251+0000 7f64e0a00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-26T22:53:00.631 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 64, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-26T22:52:53.755842+0000', 'modified': '2024-05-26T22:52:56.809807+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6876}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6876': {'gid': 6876, 'name': 'd', 'rank': 0, 'incarnation': 60, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.187:6834/3680999672', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.187:6832', 'nonce': 3680999672}, {'type': 'v1', 'addr': '172.21.15.187:6834', 'nonce': 3680999672}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6876, 'qdb_cluster': [6876]} 2024-05-26T22:53:00.631 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-26T22:53:00.632 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-05-26T22:53:00.843 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.843+0000 7f1e1c000640 1 Processor -- start 2024-05-26T22:53:00.843 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.843+0000 7f1e1c000640 1 -- start start 2024-05-26T22:53:00.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.845+0000 7f1e1c000640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f1e14155770 0x7f1e1415b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:00.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.845+0000 7f1e1c000640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e1415c1c0 con 0x7f1e1415bc40 2024-05-26T22:53:00.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.845+0000 7f1e1c000640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e1415c4a0 con 0x7f1e1406e440 2024-05-26T22:53:00.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.845+0000 7f1e1c000640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f1e14071da0 con 0x7f1e14155770 2024-05-26T22:53:00.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.845+0000 7f1e18c00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f1e1415bc40 0x7f1e1406dd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:42324/0 (socket says 172.21.15.131:42324) 2024-05-26T22:53:00.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.845+0000 7f1e18c00640 1 -- 172.21.15.131:0/2815260984 learned_addr learned my addr 172.21.15.131:0/2815260984 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:53:00.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.846+0000 7f1e13e00640 1 -- 172.21.15.131:0/2815260984 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3821131796 0 0) 0x7f1e1415c1c0 con 0x7f1e1415bc40 2024-05-26T22:53:00.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.846+0000 7f1e13e00640 1 -- 172.21.15.131:0/2815260984 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1dfc005730 con 0x7f1e1415bc40 2024-05-26T22:53:00.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.846+0000 7f1e19600640 1 --2- 172.21.15.131:0/2815260984 >> v2:172.21.15.187:3300/0 conn(0x7f1e14155770 0x7f1e1415b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:00.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.846+0000 7f1e13e00640 1 -- 172.21.15.131:0/2815260984 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 537830894 0 0) 0x7f1dfc005730 con 0x7f1e1415bc40 2024-05-26T22:53:00.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.846+0000 7f1e13e00640 1 -- 172.21.15.131:0/2815260984 >> v2:172.21.15.187:3300/0 conn(0x7f1e14155770 msgr2=0x7f1e1415b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:00.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.846+0000 7f1e13e00640 1 --2- 172.21.15.131:0/2815260984 >> v2:172.21.15.187:3300/0 conn(0x7f1e14155770 0x7f1e1415b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:00.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.846+0000 7f1e13e00640 1 -- 172.21.15.131:0/2815260984 >> v1:172.21.15.187:6789/0 conn(0x7f1e1406e440 legacy=0x7f1e140725a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:00.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.846+0000 7f1e13e00640 1 -- 172.21.15.131:0/2815260984 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e14072080 con 0x7f1e1415bc40 2024-05-26T22:53:00.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.846+0000 7f1e13e00640 1 -- 172.21.15.131:0/2815260984 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7f1e00001670 con 0x7f1e1415bc40 2024-05-26T22:53:00.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.846+0000 7f1e13e00640 1 -- 172.21.15.131:0/2815260984 >> v1:172.21.15.131:6789/0 conn(0x7f1e1415bc40 legacy=0x7f1e1406dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:00.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.846+0000 7f1e13e00640 1 --2- 172.21.15.131:0/2815260984 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1dfc005bc0 0x7f1dfc005fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:00.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.846+0000 7f1e13e00640 1 --2- 172.21.15.131:0/2815260984 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1dfc006a10 0x7f1dfc006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:00.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.846+0000 7f1e13e00640 1 --2- 172.21.15.131:0/2815260984 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1dfc0073d0 0x7f1dfc00dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:00.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.846+0000 7f1e13e00640 1 -- 172.21.15.131:0/2815260984 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f1e14071da0 con 0x7f1dfc005bc0 2024-05-26T22:53:00.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.846+0000 7f1e13e00640 1 -- 172.21.15.131:0/2815260984 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1dfc00e2e0 con 0x7f1dfc006a10 2024-05-26T22:53:00.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.846+0000 7f1e13e00640 1 -- 172.21.15.131:0/2815260984 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1dfc00e4e0 con 0x7f1dfc0073d0 2024-05-26T22:53:00.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.847+0000 7f1e19600640 1 --2- 172.21.15.131:0/2815260984 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1dfc005bc0 0x7f1dfc005fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:00.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.847+0000 7f1e19600640 1 -- 172.21.15.131:0/2815260984 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1dfc0073d0 msgr2=0x7f1dfc00dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:00.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.847+0000 7f1e19600640 1 --2- 172.21.15.131:0/2815260984 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1dfc0073d0 0x7f1dfc00dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:00.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.847+0000 7f1e19600640 1 -- 172.21.15.131:0/2815260984 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1dfc006a10 msgr2=0x7f1dfc006e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:00.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.847+0000 7f1e19600640 1 --2- 172.21.15.131:0/2815260984 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1dfc006a10 0x7f1dfc006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:00.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.847+0000 7f1e19600640 1 -- 172.21.15.131:0/2815260984 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dfc005730 con 0x7f1dfc005bc0 2024-05-26T22:53:00.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.847+0000 7f1e1a000640 1 -- 172.21.15.131:0/2815260984 reap_dead start 2024-05-26T22:53:00.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.848+0000 7f1e19600640 1 --2- 172.21.15.131:0/2815260984 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1dfc005bc0 0x7f1dfc005fc0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f1e0800ba30 tx=0x7f1dfc007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afa91fecd23c42e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:00.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.848+0000 7f1e13e00640 1 -- 172.21.15.131:0/2815260984 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e08030070 con 0x7f1dfc005bc0 2024-05-26T22:53:00.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.848+0000 7f1e13e00640 1 -- 172.21.15.131:0/2815260984 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e0802ade0 con 0x7f1dfc005bc0 2024-05-26T22:53:00.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.848+0000 7f1e13e00640 1 -- 172.21.15.131:0/2815260984 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e0800b600 con 0x7f1dfc005bc0 2024-05-26T22:53:00.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.849+0000 7f1e1c000640 1 -- 172.21.15.131:0/2815260984 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1dfc005bc0 msgr2=0x7f1dfc005fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:00.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.849+0000 7f1e1c000640 1 --2- 172.21.15.131:0/2815260984 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1dfc005bc0 0x7f1dfc005fc0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f1e0800ba30 tx=0x7f1dfc007a40 comp rx=0 tx=0).stop 2024-05-26T22:53:00.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.849+0000 7f1e1c000640 1 -- 172.21.15.131:0/2815260984 shutdown_connections 2024-05-26T22:53:00.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.849+0000 7f1e1c000640 1 --2- 172.21.15.131:0/2815260984 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1dfc005bc0 0x7f1dfc005fc0 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:00.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.849+0000 7f1e1c000640 1 -- 172.21.15.131:0/2815260984 >> 172.21.15.131:0/2815260984 conn(0x7f1e1406bc90 msgr2=0x7f1e140b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:00.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.849+0000 7f1e1c000640 1 -- 172.21.15.131:0/2815260984 shutdown_connections 2024-05-26T22:53:00.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.849+0000 7f1e1c000640 1 -- 172.21.15.131:0/2815260984 wait complete. 2024-05-26T22:53:00.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.851+0000 7f1e1c000640 1 Processor -- start 2024-05-26T22:53:00.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.851+0000 7f1e1c000640 1 -- start start 2024-05-26T22:53:00.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.852+0000 7f1e1c000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1dfc005bc0 0x7f1e14158c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:00.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.852+0000 7f1e1c000640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1e140cd5b0 0x7f1e141539a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:00.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.852+0000 7f1e1c000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1e14156a70 0x7f1e1409c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:00.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.852+0000 7f1e1c000640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f1e1409de00 con 0x7f1dfc005bc0 2024-05-26T22:53:00.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.852+0000 7f1e1c000640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1e1415c8c0 con 0x7f1e140cd5b0 2024-05-26T22:53:00.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.852+0000 7f1e1c000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1e1409df80 con 0x7f1e14156a70 2024-05-26T22:53:00.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.853+0000 7f1e19600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1dfc005bc0 0x7f1e14158c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:00.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.853+0000 7f1e19600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1dfc005bc0 0x7f1e14158c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48540/0 (socket says 172.21.15.131:48540) 2024-05-26T22:53:00.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.853+0000 7f1e19600640 1 -- 172.21.15.131:0/2936073237 learned_addr learned my addr 172.21.15.131:0/2936073237 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:53:00.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.853+0000 7f1e19600640 1 -- 172.21.15.131:0/2936073237 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1e14156a70 msgr2=0x7f1e1409c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:00.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.853+0000 7f1e19600640 1 --2- 172.21.15.131:0/2936073237 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1e14156a70 0x7f1e1409c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:00.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.853+0000 7f1e19600640 1 -- 172.21.15.131:0/2936073237 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1e140cd5b0 msgr2=0x7f1e141539a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:00.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.853+0000 7f1e19600640 1 --2- 172.21.15.131:0/2936073237 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1e140cd5b0 0x7f1e141539a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:00.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.853+0000 7f1e19600640 1 -- 172.21.15.131:0/2936073237 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e14153ee0 con 0x7f1dfc005bc0 2024-05-26T22:53:00.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.853+0000 7f1e19600640 1 --2- 172.21.15.131:0/2936073237 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1dfc005bc0 0x7f1e14158c60 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f1e0800bb50 tx=0x7f1e080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:00.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.854+0000 7f1e11600640 1 -- 172.21.15.131:0/2936073237 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e08043030 con 0x7f1dfc005bc0 2024-05-26T22:53:00.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.854+0000 7f1e11600640 1 -- 172.21.15.131:0/2936073237 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e0802abe0 con 0x7f1dfc005bc0 2024-05-26T22:53:00.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.854+0000 7f1e11600640 1 -- 172.21.15.131:0/2936073237 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e0800b600 con 0x7f1dfc005bc0 2024-05-26T22:53:00.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.854+0000 7f1e1c000640 1 -- 172.21.15.131:0/2936073237 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e14157430 con 0x7f1dfc005bc0 2024-05-26T22:53:00.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.854+0000 7f1e1c000640 1 -- 172.21.15.131:0/2936073237 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e140ce1a0 con 0x7f1dfc005bc0 2024-05-26T22:53:00.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.857+0000 7f1e11600640 1 -- 172.21.15.131:0/2936073237 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1e08046070 con 0x7f1dfc005bc0 2024-05-26T22:53:00.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.858+0000 7f1e11600640 1 --2- 172.21.15.131:0/2936073237 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f1df406be30 0x7f1df406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:00.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.858+0000 7f1e11600640 1 -- 172.21.15.131:0/2936073237 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7f1e080ab820 con 0x7f1dfc005bc0 2024-05-26T22:53:00.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.858+0000 7f1e18c00640 1 --2- 172.21.15.131:0/2936073237 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f1df406be30 0x7f1df406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:00.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.858+0000 7f1e1ee53740 1 Processor -- start 2024-05-26T22:53:00.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.858+0000 7f1e1ee53740 1 -- start start 2024-05-26T22:53:00.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.859+0000 7f1e18c00640 1 --2- 172.21.15.131:0/2936073237 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f1df406be30 0x7f1df406e2f0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f1e14154a50 tx=0x7f1e000014e0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:00.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.859+0000 7f1e1ee53740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5598d2350c70 0x5598d2303890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:00.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.859+0000 7f1e1ee53740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5598d2333d90 0x5598d2333110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:00.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.859+0000 7f1e1ee53740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5598d2333650 0x5598d2308460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:00.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.859+0000 7f1e1ee53740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5598d226b610 con 0x5598d2333650 2024-05-26T22:53:00.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.859+0000 7f1e1ee53740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5598d22135c0 con 0x5598d2333d90 2024-05-26T22:53:00.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.859+0000 7f1e1ee53740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x5598d22675d0 con 0x5598d2350c70 2024-05-26T22:53:00.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.860+0000 7f1e1a000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5598d2333650 0x5598d2308460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:00.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.860+0000 7f1e1a000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5598d2333650 0x5598d2308460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48554/0 (socket says 172.21.15.131:48554) 2024-05-26T22:53:00.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.860+0000 7f1e1a000640 1 -- 172.21.15.131:0/3096626742 learned_addr learned my addr 172.21.15.131:0/3096626742 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:53:00.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.860+0000 7f1e18c00640 1 --2- 172.21.15.131:0/3096626742 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5598d2333d90 0x5598d2333110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:00.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.860+0000 7f1e1a000640 1 -- 172.21.15.131:0/3096626742 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5598d2350c70 msgr2=0x5598d2303890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:00.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.860+0000 7f1e1a000640 1 --2- 172.21.15.131:0/3096626742 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5598d2350c70 0x5598d2303890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:00.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.860+0000 7f1e1a000640 1 -- 172.21.15.131:0/3096626742 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5598d2333d90 msgr2=0x5598d2333110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:00.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.860+0000 7f1e1a000640 1 --2- 172.21.15.131:0/3096626742 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5598d2333d90 0x5598d2333110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:00.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.860+0000 7f1e1a000640 1 -- 172.21.15.131:0/3096626742 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598d1f7f2d0 con 0x5598d2333650 2024-05-26T22:53:00.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.860+0000 7f1e1a000640 1 --2- 172.21.15.131:0/3096626742 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5598d2333650 0x5598d2308460 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c004200 tx=0x7f1e0c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f532a13fc9482c server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:00.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.861+0000 7f1e05c00640 1 -- 172.21.15.131:0/3096626742 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e0c00c6c0 con 0x5598d2333650 2024-05-26T22:53:00.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.861+0000 7f1e05c00640 1 -- 172.21.15.131:0/3096626742 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e0c00c860 con 0x5598d2333650 2024-05-26T22:53:00.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.861+0000 7f1e05c00640 1 -- 172.21.15.131:0/3096626742 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e0c00cc50 con 0x5598d2333650 2024-05-26T22:53:00.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.861+0000 7f1e1ee53740 1 -- 172.21.15.131:0/3096626742 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5598d2333650 msgr2=0x5598d2308460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:00.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.861+0000 7f1e1ee53740 1 --2- 172.21.15.131:0/3096626742 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5598d2333650 0x5598d2308460 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c004200 tx=0x7f1e0c0046c0 comp rx=0 tx=0).stop 2024-05-26T22:53:00.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.861+0000 7f1e1ee53740 1 -- 172.21.15.131:0/3096626742 shutdown_connections 2024-05-26T22:53:00.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.861+0000 7f1e1ee53740 1 --2- 172.21.15.131:0/3096626742 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5598d2333650 0x5598d2308460 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:00.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.861+0000 7f1e1ee53740 1 --2- 172.21.15.131:0/3096626742 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5598d2333d90 0x5598d2333110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:00.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.861+0000 7f1e1ee53740 1 --2- 172.21.15.131:0/3096626742 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5598d2350c70 0x5598d2303890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:00.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.861+0000 7f1e1ee53740 1 -- 172.21.15.131:0/3096626742 >> 172.21.15.131:0/3096626742 conn(0x5598d2305b00 msgr2=0x5598d234e110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:00.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.861+0000 7f1e1ee53740 1 -- 172.21.15.131:0/3096626742 shutdown_connections 2024-05-26T22:53:00.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.861+0000 7f1e1ee53740 1 -- 172.21.15.131:0/3096626742 wait complete. 2024-05-26T22:53:00.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.862+0000 7f1e1ee53740 1 -- start start 2024-05-26T22:53:00.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.862+0000 7f1e1ee53740 1 Processor -- start 2024-05-26T22:53:00.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.863+0000 7f1e1ee53740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5598d2333d90 0x5598d2324e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:00.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.863+0000 7f1e1ee53740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5598d2350c70 0x5598d23342f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:00.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.863+0000 7f1e1ee53740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5598d2334830 0x5598d234e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:00.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.863+0000 7f1e1ee53740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5598d2307a20 con 0x5598d2350c70 2024-05-26T22:53:00.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.863+0000 7f1e1ee53740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5598d2226910 con 0x5598d2333d90 2024-05-26T22:53:00.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.863+0000 7f1e1ee53740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x5598d2306e10 con 0x5598d2334830 2024-05-26T22:53:00.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.863+0000 7f1e1ee53740 10 client.0 _ll_register_callbacks cb 0x5598d2262410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-26T22:53:00.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.863+0000 7f1e18c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5598d2350c70 0x5598d23342f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:00.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.864+0000 7f1e18c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5598d2350c70 0x5598d23342f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48562/0 (socket says 172.21.15.131:48562) 2024-05-26T22:53:00.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.864+0000 7f1e18c00640 1 -- 172.21.15.131:0/3806599889 learned_addr learned my addr 172.21.15.131:0/3806599889 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:53:00.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.864+0000 7f1e19600640 1 --2- 172.21.15.131:0/3806599889 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5598d2333d90 0x5598d2324e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:00.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.864+0000 7f1e18c00640 1 -- 172.21.15.131:0/3806599889 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5598d2334830 msgr2=0x5598d234e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:00.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.864+0000 7f1e18c00640 1 --2- 172.21.15.131:0/3806599889 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5598d2334830 0x5598d234e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:00.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.864+0000 7f1e18c00640 1 -- 172.21.15.131:0/3806599889 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5598d2333d90 msgr2=0x5598d2324e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:00.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.864+0000 7f1e18c00640 1 --2- 172.21.15.131:0/3806599889 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5598d2333d90 0x5598d2324e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:00.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.864+0000 7f1e18c00640 1 -- 172.21.15.131:0/3806599889 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598d20496b0 con 0x5598d2350c70 2024-05-26T22:53:00.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.864+0000 7f1e18c00640 1 --2- 172.21.15.131:0/3806599889 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5598d2350c70 0x5598d23342f0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f1e00005ad0 tx=0x7f1e1415c670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ef157e511bf513d server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:00.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.864+0000 7f1dfa000640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-26T22:53:00.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.864+0000 7f1dfa000640 1 -- 172.21.15.131:0/3806599889 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e00012640 con 0x5598d2350c70 2024-05-26T22:53:00.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.865+0000 7f1e1ee53740 1 -- 172.21.15.131:0/3806599889 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e0c00c860 con 0x5598d2350c70 2024-05-26T22:53:00.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.865+0000 7f1dfa000640 1 -- 172.21.15.131:0/3806599889 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7f1e00022ad0 con 0x5598d2350c70 2024-05-26T22:53:00.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.865+0000 7f1dfa000640 1 -- 172.21.15.131:0/3806599889 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f1e00022cd0 con 0x5598d2350c70 2024-05-26T22:53:00.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.865+0000 7f1e1ee53740 10 client.7026 fetch_fsmap learned FSMap version 64 2024-05-26T22:53:00.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.865+0000 7f1e1ee53740 1 -- 172.21.15.131:0/3806599889 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=2+}) -- 0x5598d234a2a0 con 0x5598d2350c70 2024-05-26T22:53:00.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.865+0000 7f1dfa000640 1 -- 172.21.15.131:0/3806599889 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e0c00c860 con 0x5598d2350c70 2024-05-26T22:53:00.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.865+0000 7f1dfa000640 1 -- 172.21.15.131:0/3806599889 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 64) ==== 3494+0+0 (secure 0 0 0) 0x7f1e000127e0 con 0x5598d2350c70 2024-05-26T22:53:00.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.865+0000 7f1e1ee53740 10 client.7026 fetch_fsmap finished waiting for FSMap version 64 2024-05-26T22:53:00.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.865+0000 7f1e1ee53740 10 client.7026 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-26T22:53:00.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.865+0000 7f1e1ee53740 20 client.7026 populate_metadata read hostname 'smithi131' 2024-05-26T22:53:00.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.865+0000 7f1e1ee53740 1 --2- 172.21.15.131:0/3806599889 >> [v2:172.21.15.187:6832/3680999672,v1:172.21.15.187:6834/3680999672] conn(0x5598d2332c40 0x5598d234b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:00.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.865+0000 7f1e1ee53740 4 client.7026 mds_command: new command op to 6876 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-26T22:53:00.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.865+0000 7f1e1ee53740 1 -- 172.21.15.131:0/3806599889 --> [v2:172.21.15.187:6832/3680999672,v1:172.21.15.187:6834/3680999672] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5598d2278d40 con 0x5598d2332c40 2024-05-26T22:53:00.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.866+0000 7f1e19600640 1 --2- 172.21.15.131:0/3806599889 >> [v2:172.21.15.187:6832/3680999672,v1:172.21.15.187:6834/3680999672] conn(0x5598d2332c40 0x5598d234b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:00.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.867+0000 7f1e19600640 1 --2- 172.21.15.131:0/3806599889 >> [v2:172.21.15.187:6832/3680999672,v1:172.21.15.187:6834/3680999672] conn(0x5598d2332c40 0x5598d234b240 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4370956e9b040937 server_cookie=9f24f0f7e7fefe79 in_seq=0 out_seq=0 2024-05-26T22:53:00.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.867+0000 7f1dfa000640 10 client.7026 ms_handle_connect on v2:172.21.15.187:6832/3680999672 2024-05-26T22:53:00.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.870+0000 7f1dfa000640 1 -- 172.21.15.131:0/3806599889 <== mds.0 v2:172.21.15.187:6832/3680999672 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5598d2278d40 con 0x5598d2332c40 2024-05-26T22:53:00.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.870+0000 7f1dfa000640 10 client.7026 handle_command_reply: tid=0 2024-05-26T22:53:00.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.870+0000 7f1e1ee53740 1 client.7026 shutdown 2024-05-26T22:53:00.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.870+0000 7f1e1ee53740 1 -- 172.21.15.131:0/3806599889 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5598d2350c70 msgr2=0x5598d23342f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:00.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.870+0000 7f1e1ee53740 1 --2- 172.21.15.131:0/3806599889 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5598d2350c70 0x5598d23342f0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f1e00005ad0 tx=0x7f1e1415c670 comp rx=0 tx=0).stop 2024-05-26T22:53:00.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.870+0000 7f1e1ee53740 1 -- 172.21.15.131:0/3806599889 shutdown_connections 2024-05-26T22:53:00.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.870+0000 7f1e1ee53740 1 --2- 172.21.15.131:0/3806599889 >> [v2:172.21.15.187:6832/3680999672,v1:172.21.15.187:6834/3680999672] conn(0x5598d2332c40 0x5598d234b240 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:00.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.870+0000 7f1e1ee53740 1 --2- 172.21.15.131:0/3806599889 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5598d2334830 0x5598d234e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:00.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.870+0000 7f1e1ee53740 1 --2- 172.21.15.131:0/3806599889 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5598d2350c70 0x5598d23342f0 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:00.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.870+0000 7f1e1ee53740 1 --2- 172.21.15.131:0/3806599889 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5598d2333d90 0x5598d2324e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:00.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.870+0000 7f1dfa000640 0 client.7026 ms_handle_reset on v2:172.21.15.187:6832/3680999672 2024-05-26T22:53:00.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.870+0000 7f1e1ee53740 1 -- 172.21.15.131:0/3806599889 >> 172.21.15.131:0/3806599889 conn(0x5598d2333650 msgr2=0x5598d23042f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:00.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.870+0000 7f1e1ee53740 1 -- 172.21.15.131:0/3806599889 shutdown_connections 2024-05-26T22:53:00.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.870+0000 7f1e1ee53740 1 -- 172.21.15.131:0/3806599889 wait complete. 2024-05-26T22:53:00.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.870+0000 7f1e1ee53740 20 client.7026 trim_cache size 0 max 16384 2024-05-26T22:53:00.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.890+0000 7f1e1ee53740 1 Processor -- start 2024-05-26T22:53:00.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.890+0000 7f1e1ee53740 1 -- start start 2024-05-26T22:53:00.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.891+0000 7f1e1ee53740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5598d2333d90 0x5598d2303350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:00.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.891+0000 7f1e1ee53740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5598d2334830 0x5598d2303890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:00.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.891+0000 7f1e1ee53740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5598d2350c70 0x5598d2361840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:00.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.891+0000 7f1e1ee53740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f1e141fbbc0 con 0x5598d2350c70 2024-05-26T22:53:00.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.891+0000 7f1e1ee53740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1e141faf70 con 0x5598d2334830 2024-05-26T22:53:00.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.891+0000 7f1e1ee53740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1e141fb570 con 0x5598d2333d90 2024-05-26T22:53:00.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.891+0000 7f1e1a000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5598d2350c70 0x5598d2361840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:00.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.891+0000 7f1e1a000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5598d2350c70 0x5598d2361840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48578/0 (socket says 172.21.15.131:48578) 2024-05-26T22:53:00.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.891+0000 7f1e1a000640 1 -- 172.21.15.131:0/583523294 learned_addr learned my addr 172.21.15.131:0/583523294 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:53:00.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.892+0000 7f1e18c00640 1 --2- 172.21.15.131:0/583523294 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5598d2334830 0x5598d2303890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:00.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.892+0000 7f1e19600640 1 --2- 172.21.15.131:0/583523294 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5598d2333d90 0x5598d2303350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:00.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.892+0000 7f1e1a000640 1 -- 172.21.15.131:0/583523294 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5598d2333d90 msgr2=0x5598d2303350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:00.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.892+0000 7f1e1a000640 1 --2- 172.21.15.131:0/583523294 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5598d2333d90 0x5598d2303350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:00.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.892+0000 7f1e1a000640 1 -- 172.21.15.131:0/583523294 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5598d2334830 msgr2=0x5598d2303890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:00.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.892+0000 7f1e1a000640 1 --2- 172.21.15.131:0/583523294 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5598d2334830 0x5598d2303890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:00.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.892+0000 7f1e1a000640 1 -- 172.21.15.131:0/583523294 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598d2334d30 con 0x5598d2350c70 2024-05-26T22:53:00.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.892+0000 7f1e18c00640 1 --2- 172.21.15.131:0/583523294 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5598d2334830 0x5598d2303890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:53:00.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.892+0000 7f1e1a000640 1 --2- 172.21.15.131:0/583523294 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5598d2350c70 0x5598d2361840 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c010000 tx=0x7f1e0c009000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6abc55b93a7971a server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:00.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.892+0000 7f1e05200640 1 -- 172.21.15.131:0/583523294 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e0c00c940 con 0x5598d2350c70 2024-05-26T22:53:00.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.892+0000 7f1e05200640 1 -- 172.21.15.131:0/583523294 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e0c00cae0 con 0x5598d2350c70 2024-05-26T22:53:00.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.892+0000 7f1e05200640 1 -- 172.21.15.131:0/583523294 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e0c013410 con 0x5598d2350c70 2024-05-26T22:53:00.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.893+0000 7f1e1ee53740 1 -- 172.21.15.131:0/583523294 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5598d2350c70 msgr2=0x5598d2361840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:00.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.893+0000 7f1e1ee53740 1 --2- 172.21.15.131:0/583523294 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5598d2350c70 0x5598d2361840 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c010000 tx=0x7f1e0c009000 comp rx=0 tx=0).stop 2024-05-26T22:53:00.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.893+0000 7f1e1ee53740 1 -- 172.21.15.131:0/583523294 shutdown_connections 2024-05-26T22:53:00.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.893+0000 7f1e1ee53740 1 --2- 172.21.15.131:0/583523294 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5598d2350c70 0x5598d2361840 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:00.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.893+0000 7f1e1ee53740 1 --2- 172.21.15.131:0/583523294 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5598d2334830 0x5598d2303890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:00.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.893+0000 7f1e1ee53740 1 --2- 172.21.15.131:0/583523294 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5598d2333d90 0x5598d2303350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:00.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.893+0000 7f1e1ee53740 1 -- 172.21.15.131:0/583523294 >> 172.21.15.131:0/583523294 conn(0x5598d2333650 msgr2=0x5598d234df10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:00.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.893+0000 7f1e1ee53740 1 -- 172.21.15.131:0/583523294 shutdown_connections 2024-05-26T22:53:00.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.893+0000 7f1e1ee53740 1 -- 172.21.15.131:0/583523294 wait complete. 2024-05-26T22:53:00.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.894+0000 7f1e1ee53740 1 -- start start 2024-05-26T22:53:00.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.894+0000 7f1e1ee53740 1 Processor -- start 2024-05-26T22:53:00.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.895+0000 7f1e1ee53740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5598d2333d90 0x5598d2308890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:00.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.895+0000 7f1e1ee53740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5598d2334830 0x5598d23318b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:00.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.895+0000 7f1e1ee53740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5598d2350c70 0x5598d2330000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:00.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.895+0000 7f1e1ee53740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5598d2332ac0 con 0x5598d2333d90 2024-05-26T22:53:00.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.895+0000 7f1e1ee53740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5598d234b840 con 0x5598d2350c70 2024-05-26T22:53:00.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.895+0000 7f1e1ee53740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x5598d2333100 con 0x5598d2334830 2024-05-26T22:53:00.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.895+0000 7f1e1ee53740 10 client.0 _ll_register_callbacks cb 0x7f1e00001710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-26T22:53:00.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.895+0000 7f1e19600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5598d2333d90 0x5598d2308890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:00.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.895+0000 7f1e19600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5598d2333d90 0x5598d2308890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48586/0 (socket says 172.21.15.131:48586) 2024-05-26T22:53:00.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.895+0000 7f1e19600640 1 -- 172.21.15.131:0/32478288 learned_addr learned my addr 172.21.15.131:0/32478288 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:53:00.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.896+0000 7f1e18c00640 1 --2- 172.21.15.131:0/32478288 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5598d2334830 0x5598d23318b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:00.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.896+0000 7f1e19600640 1 -- 172.21.15.131:0/32478288 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5598d2334830 msgr2=0x5598d23318b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:00.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.896+0000 7f1e1a000640 1 --2- 172.21.15.131:0/32478288 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5598d2350c70 0x5598d2330000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:00.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.896+0000 7f1e19600640 1 --2- 172.21.15.131:0/32478288 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5598d2334830 0x5598d23318b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:00.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.896+0000 7f1e19600640 1 -- 172.21.15.131:0/32478288 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5598d2350c70 msgr2=0x5598d2330000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:00.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.896+0000 7f1e19600640 1 --2- 172.21.15.131:0/32478288 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5598d2350c70 0x5598d2330000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:00.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.896+0000 7f1e19600640 1 -- 172.21.15.131:0/32478288 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598d2331df0 con 0x5598d2333d90 2024-05-26T22:53:00.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.896+0000 7f1e19600640 1 --2- 172.21.15.131:0/32478288 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5598d2333d90 0x5598d2308890 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f1e080797d0 tx=0x7f1e0807df40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5600b2fb37fe52d server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:00.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.896+0000 7f1dfa000640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-26T22:53:00.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.896+0000 7f1dfa000640 1 -- 172.21.15.131:0/32478288 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e080ab040 con 0x5598d2333d90 2024-05-26T22:53:00.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.896+0000 7f1dfa000640 1 -- 172.21.15.131:0/32478288 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7f1e0802ad90 con 0x5598d2333d90 2024-05-26T22:53:00.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.896+0000 7f1e1ee53740 1 -- 172.21.15.131:0/32478288 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e0c00cae0 con 0x5598d2333d90 2024-05-26T22:53:00.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.896+0000 7f1dfa000640 1 -- 172.21.15.131:0/32478288 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f1e08077dd0 con 0x5598d2333d90 2024-05-26T22:53:00.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.896+0000 7f1e1ee53740 10 client.7032 fetch_fsmap learned FSMap version 64 2024-05-26T22:53:00.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.896+0000 7f1e1ee53740 1 -- 172.21.15.131:0/32478288 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=2+}) -- 0x5598d2330540 con 0x5598d2333d90 2024-05-26T22:53:00.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.897+0000 7f1dfa000640 1 -- 172.21.15.131:0/32478288 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e0c00cae0 con 0x5598d2333d90 2024-05-26T22:53:00.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.897+0000 7f1dfa000640 1 -- 172.21.15.131:0/32478288 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 64) ==== 3494+0+0 (secure 0 0 0) 0x7f1e08050060 con 0x5598d2333d90 2024-05-26T22:53:00.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.897+0000 7f1e1ee53740 10 client.7032 fetch_fsmap finished waiting for FSMap version 64 2024-05-26T22:53:00.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.897+0000 7f1e1ee53740 10 client.7032 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-26T22:53:00.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.897+0000 7f1e1ee53740 20 client.7032 populate_metadata read hostname 'smithi131' 2024-05-26T22:53:00.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.897+0000 7f1e1ee53740 1 --2- 172.21.15.131:0/32478288 >> [v2:172.21.15.187:6832/3680999672,v1:172.21.15.187:6834/3680999672] conn(0x5598d2374ed0 0x5598d23752b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:00.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.897+0000 7f1e1ee53740 4 client.7032 mds_command: new command op to 6876 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-26T22:53:00.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.897+0000 7f1e1ee53740 1 -- 172.21.15.131:0/32478288 --> [v2:172.21.15.187:6832/3680999672,v1:172.21.15.187:6834/3680999672] -- command(tid 0: {"prefix": "status"}) -- 0x5598d2335fc0 con 0x5598d2374ed0 2024-05-26T22:53:00.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.897+0000 7f1e18c00640 1 --2- 172.21.15.131:0/32478288 >> [v2:172.21.15.187:6832/3680999672,v1:172.21.15.187:6834/3680999672] conn(0x5598d2374ed0 0x5598d23752b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:00.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.898+0000 7f1e18c00640 1 --2- 172.21.15.131:0/32478288 >> [v2:172.21.15.187:6832/3680999672,v1:172.21.15.187:6834/3680999672] conn(0x5598d2374ed0 0x5598d23752b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=abf3c20070bf1337 server_cookie=c441d19ee94e0d1d in_seq=0 out_seq=0 2024-05-26T22:53:00.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.898+0000 7f1dfa000640 10 client.7032 ms_handle_connect on v2:172.21.15.187:6832/3680999672 2024-05-26T22:53:00.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.899+0000 7f1dfa000640 1 -- 172.21.15.131:0/32478288 <== mds.0 v2:172.21.15.187:6832/3680999672 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x5598d2335fc0 con 0x5598d2374ed0 2024-05-26T22:53:00.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.899+0000 7f1dfa000640 10 client.7032 handle_command_reply: tid=0 2024-05-26T22:53:00.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.899+0000 7f1e1ee53740 1 client.7032 shutdown 2024-05-26T22:53:00.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.899+0000 7f1e1ee53740 1 -- 172.21.15.131:0/32478288 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5598d2333d90 msgr2=0x5598d2308890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:00.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.899+0000 7f1e1ee53740 1 --2- 172.21.15.131:0/32478288 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5598d2333d90 0x5598d2308890 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f1e080797d0 tx=0x7f1e0807df40 comp rx=0 tx=0).stop 2024-05-26T22:53:00.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.899+0000 7f1e1ee53740 1 -- 172.21.15.131:0/32478288 shutdown_connections 2024-05-26T22:53:00.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.899+0000 7f1e1ee53740 1 --2- 172.21.15.131:0/32478288 >> [v2:172.21.15.187:6832/3680999672,v1:172.21.15.187:6834/3680999672] conn(0x5598d2374ed0 0x5598d23752b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:00.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.899+0000 7f1e1ee53740 1 --2- 172.21.15.131:0/32478288 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5598d2334830 0x5598d23318b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:00.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.899+0000 7f1e1ee53740 1 --2- 172.21.15.131:0/32478288 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5598d2350c70 0x5598d2330000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:00.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.899+0000 7f1dfa000640 0 client.7032 ms_handle_reset on v2:172.21.15.187:6832/3680999672 2024-05-26T22:53:00.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.899+0000 7f1e1ee53740 1 --2- 172.21.15.131:0/32478288 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5598d2333d90 0x5598d2308890 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:00.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.899+0000 7f1e1ee53740 1 -- 172.21.15.131:0/32478288 >> 172.21.15.131:0/32478288 conn(0x5598d2333650 msgr2=0x5598d23032b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:00.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.899+0000 7f1e1ee53740 1 -- 172.21.15.131:0/32478288 shutdown_connections 2024-05-26T22:53:00.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.899+0000 7f1e1ee53740 1 -- 172.21.15.131:0/32478288 wait complete. 2024-05-26T22:53:00.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.899+0000 7f1e1ee53740 20 client.7032 trim_cache size 0 max 16384 2024-05-26T22:53:00.903 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-26T22:53:00.903 INFO:teuthology.orchestra.run.smithi131.stdout: "cluster_fsid": "46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f", 2024-05-26T22:53:00.903 INFO:teuthology.orchestra.run.smithi131.stdout: "whoami": 0, 2024-05-26T22:53:00.903 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 6876, 2024-05-26T22:53:00.903 INFO:teuthology.orchestra.run.smithi131.stdout: "want_state": "up:active", 2024-05-26T22:53:00.903 INFO:teuthology.orchestra.run.smithi131.stdout: "state": "up:active", 2024-05-26T22:53:00.903 INFO:teuthology.orchestra.run.smithi131.stdout: "fs_name": "cephfs", 2024-05-26T22:53:00.903 INFO:teuthology.orchestra.run.smithi131.stdout: "rank_uptime": 6.5920616550000002, 2024-05-26T22:53:00.903 INFO:teuthology.orchestra.run.smithi131.stdout: "mdsmap_epoch": 64, 2024-05-26T22:53:00.903 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch": 69, 2024-05-26T22:53:00.904 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch_barrier": 69, 2024-05-26T22:53:00.904 INFO:teuthology.orchestra.run.smithi131.stdout: "uptime": 12.019742013 2024-05-26T22:53:00.904 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-26T22:53:00.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.900+0000 7f1e1c000640 1 -- 172.21.15.131:0/2936073237 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f1df406be30 msgr2=0x7f1df406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:00.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.900+0000 7f1e1c000640 1 --2- 172.21.15.131:0/2936073237 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f1df406be30 0x7f1df406e2f0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f1e14154a50 tx=0x7f1e000014e0 comp rx=0 tx=0).stop 2024-05-26T22:53:00.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.900+0000 7f1e1c000640 1 -- 172.21.15.131:0/2936073237 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1dfc005bc0 msgr2=0x7f1e14158c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:00.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.900+0000 7f1e1c000640 1 --2- 172.21.15.131:0/2936073237 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1dfc005bc0 0x7f1e14158c60 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f1e0800bb50 tx=0x7f1e080046c0 comp rx=0 tx=0).stop 2024-05-26T22:53:00.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.900+0000 7f1e1c000640 1 -- 172.21.15.131:0/2936073237 shutdown_connections 2024-05-26T22:53:00.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.900+0000 7f1e1c000640 1 --2- 172.21.15.131:0/2936073237 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1e14156a70 0x7f1e1409c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:00.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.900+0000 7f1e1c000640 1 --2- 172.21.15.131:0/2936073237 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f1df406be30 0x7f1df406e2f0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:00.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.900+0000 7f1e1c000640 1 --2- 172.21.15.131:0/2936073237 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1e140cd5b0 0x7f1e141539a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:00.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.900+0000 7f1e1c000640 1 --2- 172.21.15.131:0/2936073237 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1dfc005bc0 0x7f1e14158c60 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:00.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.900+0000 7f1e1c000640 1 -- 172.21.15.131:0/2936073237 >> 172.21.15.131:0/2936073237 conn(0x7f1e1406bc90 msgr2=0x7f1e140afb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:00.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.900+0000 7f1e1c000640 1 -- 172.21.15.131:0/2936073237 shutdown_connections 2024-05-26T22:53:00.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:00.900+0000 7f1e1c000640 1 -- 172.21.15.131:0/2936073237 wait complete. 2024-05-26T22:53:00.918 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-26T22:53:01.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:01.112+0000 7f2f8ee00640 1 Processor -- start 2024-05-26T22:53:01.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:01.113+0000 7f2f8ee00640 1 -- start start 2024-05-26T22:53:01.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:01.114+0000 7f2f8ee00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2f88156750 0x7f2f8815fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:01.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:01.114+0000 7f2f8ee00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f2f8806e8c0 0x7f2f8815d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:01.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:01.114+0000 7f2f8ee00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f2f880cc7f0 0x7f2f880ccc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:01.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:01.114+0000 7f2f8ee00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f2f880cd1a0 con 0x7f2f880cc7f0 2024-05-26T22:53:01.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:01.114+0000 7f2f8ee00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f2f8815cbd0 con 0x7f2f8806e8c0 2024-05-26T22:53:01.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:01.114+0000 7f2f8ee00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f2f88160620 con 0x7f2f88156750 2024-05-26T22:53:01.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:01.115+0000 7f2f8e400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f2f880cc7f0 0x7f2f880ccc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:01.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:01.115+0000 7f2f8e400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f2f880cc7f0 0x7f2f880ccc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48588/0 (socket says 172.21.15.131:48588) 2024-05-26T22:53:01.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:01.115+0000 7f2f8e400640 1 -- 172.21.15.131:0/4269744430 learned_addr learned my addr 172.21.15.131:0/4269744430 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:53:01.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:01.115+0000 7f2f8e400640 1 -- 172.21.15.131:0/4269744430 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2f88156750 msgr2=0x7f2f8815fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:01.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:01.115+0000 7f2f8e400640 1 --2- 172.21.15.131:0/4269744430 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2f88156750 0x7f2f8815fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:01.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:01.115+0000 7f2f8e400640 1 -- 172.21.15.131:0/4269744430 >> v2:172.21.15.187:3300/0 conn(0x7f2f8806e8c0 msgr2=0x7f2f8815d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:01.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:01.115+0000 7f2f8e400640 1 --2- 172.21.15.131:0/4269744430 >> v2:172.21.15.187:3300/0 conn(0x7f2f8806e8c0 0x7f2f8815d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:01.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:01.115+0000 7f2f8e400640 1 -- 172.21.15.131:0/4269744430 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f88160900 con 0x7f2f880cc7f0 2024-05-26T22:53:01.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:01.116+0000 7f2f8e400640 1 --2- 172.21.15.131:0/4269744430 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2f880cc7f0 0x7f2f880ccc60 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f2f880738c0 tx=0x7f2f8002a690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36099b330932b078 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:01.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:01.116+0000 7f2f87e00640 1 -- 172.21.15.131:0/4269744430 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f80036070 con 0x7f2f880cc7f0 2024-05-26T22:53:01.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:01.116+0000 7f2f87e00640 1 -- 172.21.15.131:0/4269744430 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f80031600 con 0x7f2f880cc7f0 2024-05-26T22:53:01.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:01.116+0000 7f2f87e00640 1 -- 172.21.15.131:0/4269744430 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f800319d0 con 0x7f2f880cc7f0 2024-05-26T22:53:01.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:01.117+0000 7f2f8ee00640 1 -- 172.21.15.131:0/4269744430 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2f880cc7f0 msgr2=0x7f2f880ccc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:01.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:01.117+0000 7f2f8ee00640 1 --2- 172.21.15.131:0/4269744430 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2f880cc7f0 0x7f2f880ccc60 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f2f880738c0 tx=0x7f2f8002a690 comp rx=0 tx=0).stop 2024-05-26T22:53:01.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:01.117+0000 7f2f8ee00640 1 -- 172.21.15.131:0/4269744430 shutdown_connections 2024-05-26T22:53:01.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:01.117+0000 7f2f8ee00640 1 --2- 172.21.15.131:0/4269744430 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2f880cc7f0 0x7f2f880ccc60 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:01.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:01.117+0000 7f2f8ee00640 1 --2- 172.21.15.131:0/4269744430 >> v2:172.21.15.187:3300/0 conn(0x7f2f8806e8c0 0x7f2f8815d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:01.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:01.117+0000 7f2f8ee00640 1 --2- 172.21.15.131:0/4269744430 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2f88156750 0x7f2f8815fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:01.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:01.117+0000 7f2f8ee00640 1 -- 172.21.15.131:0/4269744430 >> 172.21.15.131:0/4269744430 conn(0x7f2f880cff40 msgr2=0x7f2f880d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:01.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:01.117+0000 7f2f8ee00640 1 -- 172.21.15.131:0/4269744430 shutdown_connections 2024-05-26T22:53:01.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:01.117+0000 7f2f8ee00640 1 -- 172.21.15.131:0/4269744430 wait complete. 2024-05-26T22:53:01.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:01.119+0000 7f2f8ee00640 1 Processor -- start 2024-05-26T22:53:01.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:01.119+0000 7f2f8ee00640 1 -- start start 2024-05-26T22:53:01.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:01.120+0000 7f2f8ee00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2f8806e8c0 0x7f2f881559e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:01.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:01.120+0000 7f2f8ee00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2f880cc7f0 0x7f2f880b2440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:01.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:01.120+0000 7f2f8ee00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f88156750 0x7f2f880afa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:01.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:01.120+0000 7f2f8ee00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f2f88156b30 con 0x7f2f880cc7f0 2024-05-26T22:53:01.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:01.120+0000 7f2f8ee00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2f8815c860 con 0x7f2f88156750 2024-05-26T22:53:01.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:01.120+0000 7f2f8ee00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f2f88160280 con 0x7f2f8806e8c0 2024-05-26T22:53:01.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:01.120+0000 7f2f8d000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2f880cc7f0 0x7f2f880b2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:01.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:01.120+0000 7f2f8d000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2f880cc7f0 0x7f2f880b2440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48602/0 (socket says 172.21.15.131:48602) 2024-05-26T22:53:01.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:01.120+0000 7f2f8d000640 1 -- 172.21.15.131:0/878191864 learned_addr learned my addr 172.21.15.131:0/878191864 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:53:01.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:01.121+0000 7f2f8d000640 1 -- 172.21.15.131:0/878191864 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2f8806e8c0 msgr2=0x7f2f881559e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:01.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:01.121+0000 7f2f8e400640 1 --2- 172.21.15.131:0/878191864 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f88156750 0x7f2f880afa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:01.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:01.121+0000 7f2f8da00640 1 --2- 172.21.15.131:0/878191864 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2f8806e8c0 0x7f2f881559e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:01.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:01.121+0000 7f2f8d000640 1 --2- 172.21.15.131:0/878191864 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2f8806e8c0 0x7f2f881559e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:01.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:01.121+0000 7f2f8d000640 1 -- 172.21.15.131:0/878191864 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f88156750 msgr2=0x7f2f880afa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:01.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:01.121+0000 7f2f8d000640 1 --2- 172.21.15.131:0/878191864 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f88156750 0x7f2f880afa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:01.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:01.121+0000 7f2f8d000640 1 -- 172.21.15.131:0/878191864 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f880affc0 con 0x7f2f880cc7f0 2024-05-26T22:53:01.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:01.121+0000 7f2f8d000640 1 --2- 172.21.15.131:0/878191864 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2f880cc7f0 0x7f2f880b2440 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f2f7400b4a0 tx=0x7f2f7400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:01.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:01.121+0000 7f2f85600640 1 -- 172.21.15.131:0/878191864 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f74014070 con 0x7f2f880cc7f0 2024-05-26T22:53:01.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:01.121+0000 7f2f85600640 1 -- 172.21.15.131:0/878191864 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f74010610 con 0x7f2f880cc7f0 2024-05-26T22:53:01.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:01.121+0000 7f2f85600640 1 -- 172.21.15.131:0/878191864 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f74010a00 con 0x7f2f880cc7f0 2024-05-26T22:53:01.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:01.121+0000 7f2f8ee00640 1 -- 172.21.15.131:0/878191864 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f880b0240 con 0x7f2f880cc7f0 2024-05-26T22:53:01.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:01.121+0000 7f2f8ee00640 1 -- 172.21.15.131:0/878191864 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f880b0800 con 0x7f2f880cc7f0 2024-05-26T22:53:01.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:01.122+0000 7f2f8ee00640 1 -- 172.21.15.131:0/878191864 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f88155f20 con 0x7f2f880cc7f0 2024-05-26T22:53:01.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:01.123+0000 7f2f85600640 1 -- 172.21.15.131:0/878191864 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2f74017070 con 0x7f2f880cc7f0 2024-05-26T22:53:01.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:01.124+0000 7f2f85600640 1 --2- 172.21.15.131:0/878191864 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f2f6806be30 0x7f2f6806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:01.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:01.124+0000 7f2f85600640 1 -- 172.21.15.131:0/878191864 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7f2f7408a9a0 con 0x7f2f880cc7f0 2024-05-26T22:53:01.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:01.124+0000 7f2f8da00640 1 --2- 172.21.15.131:0/878191864 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f2f6806be30 0x7f2f6806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:01.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:01.125+0000 7f2f8da00640 1 --2- 172.21.15.131:0/878191864 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f2f6806be30 0x7f2f6806e2f0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c00a770 tx=0x7f2f7c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:01.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:01.129+0000 7f2f85600640 1 -- 172.21.15.131:0/878191864 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f2f880affc0 con 0x7f2f880cc7f0 2024-05-26T22:53:01.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:01.371+0000 7f2f8ee00640 1 -- 172.21.15.131:0/878191864 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f2f880694e0 con 0x7f2f880cc7f0 2024-05-26T22:53:01.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:01.372+0000 7f2f85600640 1 -- 172.21.15.131:0/878191864 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v64) ==== 53+0+73 (secure 0 0 0) 0x7f2f740592b0 con 0x7f2f880cc7f0 2024-05-26T22:53:01.372 INFO:teuthology.orchestra.run.smithi131.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-26T22:53:01.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:01.374+0000 7f2f8ee00640 1 -- 172.21.15.131:0/878191864 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f2f6806be30 msgr2=0x7f2f6806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:01.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:01.374+0000 7f2f8ee00640 1 --2- 172.21.15.131:0/878191864 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f2f6806be30 0x7f2f6806e2f0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c00a770 tx=0x7f2f7c008040 comp rx=0 tx=0).stop 2024-05-26T22:53:01.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:01.374+0000 7f2f8ee00640 1 -- 172.21.15.131:0/878191864 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2f880cc7f0 msgr2=0x7f2f880b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:01.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:01.374+0000 7f2f8ee00640 1 --2- 172.21.15.131:0/878191864 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2f880cc7f0 0x7f2f880b2440 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f2f7400b4a0 tx=0x7f2f7400b960 comp rx=0 tx=0).stop 2024-05-26T22:53:01.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:01.374+0000 7f2f8ee00640 1 -- 172.21.15.131:0/878191864 shutdown_connections 2024-05-26T22:53:01.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:01.374+0000 7f2f8ee00640 1 --2- 172.21.15.131:0/878191864 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f2f6806be30 0x7f2f6806e2f0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:01.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:01.375+0000 7f2f8ee00640 1 --2- 172.21.15.131:0/878191864 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f88156750 0x7f2f880afa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:01.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:01.375+0000 7f2f8ee00640 1 --2- 172.21.15.131:0/878191864 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2f880cc7f0 0x7f2f880b2440 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:01.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:01.375+0000 7f2f8ee00640 1 --2- 172.21.15.131:0/878191864 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2f8806e8c0 0x7f2f881559e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:01.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:01.375+0000 7f2f8ee00640 1 -- 172.21.15.131:0/878191864 >> 172.21.15.131:0/878191864 conn(0x7f2f880cff40 msgr2=0x7f2f8815eba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:01.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:01.375+0000 7f2f8ee00640 1 -- 172.21.15.131:0/878191864 shutdown_connections 2024-05-26T22:53:01.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:01.375+0000 7f2f8ee00640 1 -- 172.21.15.131:0/878191864 wait complete. 2024-05-26T22:53:01.395 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-26T22:53:01.395 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-26T22:53:01.395 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi131.front.sepia.ceph.com 2024-05-26T22:53:01.395 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-26T22:53:01.395 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-26T22:53:01.395 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-26T22:53:01.396 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-26T22:53:01.396 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-26T22:53:01.396 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-26T22:53:01.396 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:53:01.397 DEBUG:teuthology.orchestra.run.smithi131:> ip addr 2024-05-26T22:53:01.449 INFO:teuthology.orchestra.run.smithi131.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-26T22:53:01.450 INFO:teuthology.orchestra.run.smithi131.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-26T22:53:01.450 INFO:teuthology.orchestra.run.smithi131.stdout: inet 127.0.0.1/8 scope host lo 2024-05-26T22:53:01.450 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-05-26T22:53:01.450 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 ::1/128 scope host 2024-05-26T22:53:01.450 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-05-26T22:53:01.450 INFO:teuthology.orchestra.run.smithi131.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-26T22:53:01.450 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:d8:fb:18 brd ff:ff:ff:ff:ff:ff 2024-05-26T22:53:01.450 INFO:teuthology.orchestra.run.smithi131.stdout: altname enp7s0f0 2024-05-26T22:53:01.450 INFO:teuthology.orchestra.run.smithi131.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-26T22:53:01.450 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:d8:fb:19 brd ff:ff:ff:ff:ff:ff 2024-05-26T22:53:01.450 INFO:teuthology.orchestra.run.smithi131.stdout: altname enp7s0f1 2024-05-26T22:53:01.450 INFO:teuthology.orchestra.run.smithi131.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-26T22:53:01.451 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:88:72:f8 brd ff:ff:ff:ff:ff:ff 2024-05-26T22:53:01.451 INFO:teuthology.orchestra.run.smithi131.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-26T22:53:01.451 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:88:72:f9 brd ff:ff:ff:ff:ff:ff 2024-05-26T22:53:01.451 INFO:teuthology.orchestra.run.smithi131.stdout: inet 172.21.15.131/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-26T22:53:01.451 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft 42316sec preferred_lft 42316sec 2024-05-26T22:53:01.451 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::ec4:7aff:fe88:72f9/64 scope link 2024-05-26T22:53:01.451 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-05-26T22:53:01.451 INFO:teuthology.orchestra.run.smithi131.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-26T22:53:01.451 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 06:f4:05:86:a4:3e brd ff:ff:ff:ff:ff:ff 2024-05-26T22:53:01.451 INFO:teuthology.orchestra.run.smithi131.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-26T22:53:01.451 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-05-26T22:53:01.451 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::8802:8fff:fe41:fc6d/64 scope link 2024-05-26T22:53:01.451 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-05-26T22:53:01.452 INFO:teuthology.orchestra.run.smithi131.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-26T22:53:01.452 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 06:f4:05:86:a4:3e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-26T22:53:01.452 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::4f4:5ff:fe86:a43e/64 scope link 2024-05-26T22:53:01.452 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-05-26T22:53:01.452 INFO:teuthology.orchestra.run.smithi131.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-26T22:53:01.452 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether ee:6f:35:b2:c0:5b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-26T22:53:01.452 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::ec6f:35ff:feb2:c05b/64 scope link 2024-05-26T22:53:01.452 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-05-26T22:53:01.453 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:53:01.454 DEBUG:teuthology.orchestra.run.smithi131:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-26T22:53:01.534 INFO:teuthology.orchestra.run.smithi131.stdout:1 2024-05-26T22:53:01.538 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:53:01.538 DEBUG:teuthology.orchestra.run.smithi131:> route 2024-05-26T22:53:01.590 INFO:teuthology.orchestra.run.smithi131.stdout:Kernel IP routing table 2024-05-26T22:53:01.590 INFO:teuthology.orchestra.run.smithi131.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-26T22:53:01.590 INFO:teuthology.orchestra.run.smithi131.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-26T22:53:01.590 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-26T22:53:01.590 INFO:teuthology.orchestra.run.smithi131.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-26T22:53:01.591 DEBUG:teuthology.orchestra.run.smithi131:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-26T22:53:01.591 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2024-05-26T22:53:01.591 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-26T22:53:01.591 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-26T22:53:01.591 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-26T22:53:01.591 DEBUG:teuthology.orchestra.run.smithi131:> ') 2024-05-26T22:53:01.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:01 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-26T22:53:01.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-26T22:53:01.800 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:53:01.800 DEBUG:teuthology.orchestra.run.smithi131:> ip netns list 2024-05-26T22:53:01.849 INFO:teuthology.orchestra.run.smithi131.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-26T22:53:01.849 INFO:teuthology.orchestra.run.smithi131.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-26T22:53:01.850 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-26T22:53:01.850 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:53:01.850 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link set brx.0 up 2024-05-26T22:53:01.934 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-26T22:53:02.267 DEBUG:teuthology.orchestra.run.smithi131:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-26T22:53:02.328 INFO:teuthology.orchestra.run.smithi131.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-26T22:53:02.328 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-26T22:53:02.328 DEBUG:teuthology.orchestra.run.smithi131:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-26T22:53:02.377 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-05-26T22:53:02.378 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:53:02.378 DEBUG:teuthology.orchestra.run.smithi131:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-26T22:53:02.504 INFO:teuthology.orchestra.run.smithi131.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-26T22:53:02.504 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-26T22:53:02.538 INFO:teuthology.orchestra.run.smithi131.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-26T22:53:02.538 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-26T22:53:02.539 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: resolved to: "172.21.15.131:3300,172.21.15.187:3300,172.21.15.187:3301" 2024-05-26T22:53:02.539 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: trying mount with new device syntax: 0@46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f.cephfs=/ 2024-05-26T22:53:02.539 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.131:3300/172.21.15.187:3300/172.21.15.187:3301" will pass to kernel 2024-05-26T22:53:02.543 INFO:teuthology.orchestra.run.smithi131.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-26T22:53:02.543 INFO:teuthology.orchestra.run.smithi131.stdout: You just mounted a file system that supports labels which does not 2024-05-26T22:53:02.543 INFO:teuthology.orchestra.run.smithi131.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-26T22:53:02.543 INFO:teuthology.orchestra.run.smithi131.stdout: applications will generate AVC messages and not be allowed access to 2024-05-26T22:53:02.543 INFO:teuthology.orchestra.run.smithi131.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-26T22:53:02.549 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-26T22:53:02.549 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:53:02.549 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-26T22:53:02.622 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-26T22:53:02.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:02 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-26T22:53:02.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-26T22:53:02.764 INFO:teuthology.orchestra.run.smithi131.stdout:client7041 2024-05-26T22:53:02.765 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-26T22:53:02.765 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-26T22:53:02.819 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_export_ephemeral_random True 2024-05-26T22:53:03.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.026+0000 7f43d4200640 1 Processor -- start 2024-05-26T22:53:03.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.027+0000 7f43d4200640 1 -- start start 2024-05-26T22:53:03.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.028+0000 7f43d4200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f43cc0d0420 0x7f43cc15c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:03.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.028+0000 7f43d4200640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43cc06e690 con 0x7f43cc06db10 2024-05-26T22:53:03.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.028+0000 7f43d4200640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43cc15f090 con 0x7f43cc0d0040 2024-05-26T22:53:03.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.028+0000 7f43d4200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f43cc15ee40 con 0x7f43cc0d0420 2024-05-26T22:53:03.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.029+0000 7f43d2200640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f43cc06db10 0x7f43cc06df80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:42340/0 (socket says 172.21.15.131:42340) 2024-05-26T22:53:03.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.029+0000 7f43d2200640 1 -- 172.21.15.131:0/3634979802 learned_addr learned my addr 172.21.15.131:0/3634979802 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:53:03.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.029+0000 7f43d0e00640 1 --2- 172.21.15.131:0/3634979802 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f43cc0d0420 0x7f43cc15c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:03.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.029+0000 7f43cbe00640 1 -- 172.21.15.131:0/3634979802 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2928182240 0 0) 0x7f43cc06e690 con 0x7f43cc06db10 2024-05-26T22:53:03.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.029+0000 7f43cbe00640 1 -- 172.21.15.131:0/3634979802 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f43b4005730 con 0x7f43cc06db10 2024-05-26T22:53:03.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.029+0000 7f43cbe00640 1 -- 172.21.15.131:0/3634979802 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1605106725 0 0) 0x7f43b4005730 con 0x7f43cc06db10 2024-05-26T22:53:03.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.030+0000 7f43cbe00640 1 -- 172.21.15.131:0/3634979802 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f43cc0d0420 msgr2=0x7f43cc15c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:03.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.030+0000 7f43cbe00640 1 --2- 172.21.15.131:0/3634979802 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f43cc0d0420 0x7f43cc15c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:03.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.030+0000 7f43cbe00640 1 -- 172.21.15.131:0/3634979802 >> v1:172.21.15.187:6789/0 conn(0x7f43cc0d0040 legacy=0x7f43cc15bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:03.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.030+0000 7f43cbe00640 1 -- 172.21.15.131:0/3634979802 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43cc15f370 con 0x7f43cc06db10 2024-05-26T22:53:03.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.030+0000 7f43cbe00640 1 -- 172.21.15.131:0/3634979802 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7f43c40019f0 con 0x7f43cc06db10 2024-05-26T22:53:03.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.030+0000 7f43cbe00640 1 -- 172.21.15.131:0/3634979802 >> v1:172.21.15.131:6789/0 conn(0x7f43cc06db10 legacy=0x7f43cc06df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:03.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.030+0000 7f43cbe00640 1 --2- 172.21.15.131:0/3634979802 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f43b4005bf0 0x7f43b4006010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:03.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.030+0000 7f43cbe00640 1 --2- 172.21.15.131:0/3634979802 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f43cc0d0420 0x7f43b4006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:03.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.030+0000 7f43cbe00640 1 --2- 172.21.15.131:0/3634979802 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f43b4006f50 0x7f43b40073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:03.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.030+0000 7f43cbe00640 1 -- 172.21.15.131:0/3634979802 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f43cc15ee40 con 0x7f43b4006f50 2024-05-26T22:53:03.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.030+0000 7f43cbe00640 1 -- 172.21.15.131:0/3634979802 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f43b400dd30 con 0x7f43b4005bf0 2024-05-26T22:53:03.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.030+0000 7f43cbe00640 1 -- 172.21.15.131:0/3634979802 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f43b400df30 con 0x7f43cc0d0420 2024-05-26T22:53:03.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.030+0000 7f43d2200640 1 --2- 172.21.15.131:0/3634979802 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f43b4006f50 0x7f43b40073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:03.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.030+0000 7f43d1800640 1 --2- 172.21.15.131:0/3634979802 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f43b4005bf0 0x7f43b4006010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:03.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.030+0000 7f43d0e00640 1 --2- 172.21.15.131:0/3634979802 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f43cc0d0420 0x7f43b4006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:03.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.031+0000 7f43d2200640 1 -- 172.21.15.131:0/3634979802 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f43cc0d0420 msgr2=0x7f43b4006a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:03.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.031+0000 7f43d2200640 1 --2- 172.21.15.131:0/3634979802 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f43cc0d0420 0x7f43b4006a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:03.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.031+0000 7f43d2200640 1 -- 172.21.15.131:0/3634979802 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f43b4005bf0 msgr2=0x7f43b4006010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:03.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.031+0000 7f43d2200640 1 --2- 172.21.15.131:0/3634979802 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f43b4005bf0 0x7f43b4006010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:03.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.031+0000 7f43d2200640 1 -- 172.21.15.131:0/3634979802 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43b4005730 con 0x7f43b4006f50 2024-05-26T22:53:03.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.031+0000 7f43d1800640 1 --2- 172.21.15.131:0/3634979802 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f43b4005bf0 0x7f43b4006010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-26T22:53:03.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.032+0000 7f43d2200640 1 --2- 172.21.15.131:0/3634979802 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f43b4006f50 0x7f43b40073d0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f43c400a860 tx=0x7f43b4007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3251637c319d923 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:03.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.032+0000 7f43cbe00640 1 -- 172.21.15.131:0/3634979802 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c400a480 con 0x7f43b4006f50 2024-05-26T22:53:03.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.032+0000 7f43cbe00640 1 -- 172.21.15.131:0/3634979802 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43c402f070 con 0x7f43b4006f50 2024-05-26T22:53:03.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.032+0000 7f43cbe00640 1 -- 172.21.15.131:0/3634979802 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c402bd00 con 0x7f43b4006f50 2024-05-26T22:53:03.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.033+0000 7f43d4200640 1 -- 172.21.15.131:0/3634979802 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f43b4006f50 msgr2=0x7f43b40073d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:03.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.033+0000 7f43d4200640 1 --2- 172.21.15.131:0/3634979802 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f43b4006f50 0x7f43b40073d0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f43c400a860 tx=0x7f43b4007a40 comp rx=0 tx=0).stop 2024-05-26T22:53:03.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.033+0000 7f43d2200640 1 -- 172.21.15.131:0/3634979802 reap_dead start 2024-05-26T22:53:03.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.033+0000 7f43d4200640 1 -- 172.21.15.131:0/3634979802 shutdown_connections 2024-05-26T22:53:03.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.033+0000 7f43d4200640 1 -- 172.21.15.131:0/3634979802 >> 172.21.15.131:0/3634979802 conn(0x7f43cc06bc90 msgr2=0x7f43cc0b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:03.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.033+0000 7f43d4200640 1 -- 172.21.15.131:0/3634979802 shutdown_connections 2024-05-26T22:53:03.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.033+0000 7f43d4200640 1 -- 172.21.15.131:0/3634979802 wait complete. 2024-05-26T22:53:03.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.035+0000 7f43d4200640 1 Processor -- start 2024-05-26T22:53:03.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.035+0000 7f43d4200640 1 -- start start 2024-05-26T22:53:03.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.036+0000 7f43d4200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f43b4006f50 0x7f43cc14cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:03.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.036+0000 7f43d4200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f43cc156610 0x7f43cc15db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:03.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.036+0000 7f43d4200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f43cc156a70 0x7f43cc15e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:03.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.036+0000 7f43d4200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f43cc072980 con 0x7f43cc156610 2024-05-26T22:53:03.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.036+0000 7f43d4200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f43cc15f930 con 0x7f43b4006f50 2024-05-26T22:53:03.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.036+0000 7f43d4200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f43cc072b00 con 0x7f43cc156a70 2024-05-26T22:53:03.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.036+0000 7f43d0e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f43cc156610 0x7f43cc15db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:03.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.036+0000 7f43d0e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f43cc156610 0x7f43cc15db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48616/0 (socket says 172.21.15.131:48616) 2024-05-26T22:53:03.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.036+0000 7f43d0e00640 1 -- 172.21.15.131:0/372396936 learned_addr learned my addr 172.21.15.131:0/372396936 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:53:03.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.037+0000 7f43d0e00640 1 -- 172.21.15.131:0/372396936 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f43cc156a70 msgr2=0x7f43cc15e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:03.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.037+0000 7f43d0e00640 1 --2- 172.21.15.131:0/372396936 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f43cc156a70 0x7f43cc15e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:03.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.037+0000 7f43d0e00640 1 -- 172.21.15.131:0/372396936 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f43b4006f50 msgr2=0x7f43cc14cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:03.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.037+0000 7f43d0e00640 1 --2- 172.21.15.131:0/372396936 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f43b4006f50 0x7f43cc14cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:03.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.037+0000 7f43d0e00640 1 -- 172.21.15.131:0/372396936 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43cc0b41e0 con 0x7f43cc156610 2024-05-26T22:53:03.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.037+0000 7f43d0e00640 1 --2- 172.21.15.131:0/372396936 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f43cc156610 0x7f43cc15db10 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f43b8011630 tx=0x7f43b8011af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:03.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.037+0000 7f43c9600640 1 -- 172.21.15.131:0/372396936 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b8015870 con 0x7f43cc156610 2024-05-26T22:53:03.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.037+0000 7f43c9600640 1 -- 172.21.15.131:0/372396936 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43b8015a10 con 0x7f43cc156610 2024-05-26T22:53:03.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.037+0000 7f43c9600640 1 -- 172.21.15.131:0/372396936 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b8015dc0 con 0x7f43cc156610 2024-05-26T22:53:03.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.037+0000 7f43d4200640 1 -- 172.21.15.131:0/372396936 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43cc1f5090 con 0x7f43cc156610 2024-05-26T22:53:03.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.038+0000 7f43d4200640 1 -- 172.21.15.131:0/372396936 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43cc0b4460 con 0x7f43cc156610 2024-05-26T22:53:03.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.039+0000 7f43d4200640 1 -- 172.21.15.131:0/372396936 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43cc074e90 con 0x7f43cc156610 2024-05-26T22:53:03.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.040+0000 7f43c9600640 1 -- 172.21.15.131:0/372396936 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f43b8013430 con 0x7f43cc156610 2024-05-26T22:53:03.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.041+0000 7f43c9600640 1 --2- 172.21.15.131:0/372396936 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f43ac06be60 0x7f43ac06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:03.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.041+0000 7f43c9600640 1 -- 172.21.15.131:0/372396936 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7f43b808b720 con 0x7f43cc156610 2024-05-26T22:53:03.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.041+0000 7f43d1800640 1 --2- 172.21.15.131:0/372396936 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f43ac06be60 0x7f43ac06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:03.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.042+0000 7f43d1800640 1 --2- 172.21.15.131:0/372396936 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f43ac06be60 0x7f43ac06e320 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f43c0001550 tx=0x7f43c0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:03.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.045+0000 7f43c9600640 1 -- 172.21.15.131:0/372396936 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f43cc0b41e0 con 0x7f43cc156610 2024-05-26T22:53:03.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.268+0000 7f43d4200640 1 -- 172.21.15.131:0/372396936 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command([{prefix=config set, name=mds_export_ephemeral_random}] v 0) -- 0x7f43cc0695c0 con 0x7f43cc156610 2024-05-26T22:53:03.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.275+0000 7f43c9600640 1 -- 172.21.15.131:0/372396936 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_export_ephemeral_random}]=0 v36)=0 v36) ==== 128+0+0 (secure 0 0 0) 0x7f43b8019030 con 0x7f43cc156610 2024-05-26T22:53:03.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.279+0000 7f43d4200640 1 -- 172.21.15.131:0/372396936 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f43ac06be60 msgr2=0x7f43ac06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:03.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.279+0000 7f43d4200640 1 --2- 172.21.15.131:0/372396936 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f43ac06be60 0x7f43ac06e320 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f43c0001550 tx=0x7f43c0009040 comp rx=0 tx=0).stop 2024-05-26T22:53:03.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.279+0000 7f43d4200640 1 -- 172.21.15.131:0/372396936 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f43cc156610 msgr2=0x7f43cc15db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:03.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.279+0000 7f43d4200640 1 --2- 172.21.15.131:0/372396936 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f43cc156610 0x7f43cc15db10 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f43b8011630 tx=0x7f43b8011af0 comp rx=0 tx=0).stop 2024-05-26T22:53:03.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.279+0000 7f43d4200640 1 -- 172.21.15.131:0/372396936 shutdown_connections 2024-05-26T22:53:03.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.279+0000 7f43d4200640 1 --2- 172.21.15.131:0/372396936 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f43cc156a70 0x7f43cc15e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:03.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.279+0000 7f43d4200640 1 --2- 172.21.15.131:0/372396936 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f43ac06be60 0x7f43ac06e320 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:03.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.279+0000 7f43d4200640 1 --2- 172.21.15.131:0/372396936 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f43cc156610 0x7f43cc15db10 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:03.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.279+0000 7f43d4200640 1 --2- 172.21.15.131:0/372396936 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f43b4006f50 0x7f43cc14cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:03.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.279+0000 7f43d4200640 1 -- 172.21.15.131:0/372396936 >> 172.21.15.131:0/372396936 conn(0x7f43cc06bc90 msgr2=0x7f43cc0b05e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:03.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.279+0000 7f43d4200640 1 -- 172.21.15.131:0/372396936 shutdown_connections 2024-05-26T22:53:03.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.279+0000 7f43d4200640 1 -- 172.21.15.131:0/372396936 wait complete. 2024-05-26T22:53:03.299 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_export_ephemeral_distributed True 2024-05-26T22:53:03.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.498+0000 7f58c5c00640 1 Processor -- start 2024-05-26T22:53:03.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.499+0000 7f58c5c00640 1 -- start start 2024-05-26T22:53:03.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.500+0000 7f58c5c00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f58c006f180 0x7f58c0072e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:03.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.500+0000 7f58c5c00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f58c00d0560 0x7f58c0161ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:03.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.500+0000 7f58c5c00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58c00af8e0 con 0x7f58c0072540 2024-05-26T22:53:03.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.500+0000 7f58c5c00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f58c00cfb30 con 0x7f58c00d0560 2024-05-26T22:53:03.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.500+0000 7f58c5c00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f58c00cf200 con 0x7f58c006f180 2024-05-26T22:53:03.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.501+0000 7f58bf400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f58c00d0560 0x7f58c0161ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:03.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.501+0000 7f58bf400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f58c00d0560 0x7f58c0161ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48628/0 (socket says 172.21.15.131:48628) 2024-05-26T22:53:03.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.501+0000 7f58bf400640 1 -- 172.21.15.131:0/2743191012 learned_addr learned my addr 172.21.15.131:0/2743191012 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:53:03.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.501+0000 7f58bea00640 1 --2- 172.21.15.131:0/2743191012 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f58c006f180 0x7f58c0072e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:03.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.501+0000 7f58bf400640 1 -- 172.21.15.131:0/2743191012 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f58c006f180 msgr2=0x7f58c0072e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:03.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.501+0000 7f58bf400640 1 --2- 172.21.15.131:0/2743191012 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f58c006f180 0x7f58c0072e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:03.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.501+0000 7f58bf400640 1 -- 172.21.15.131:0/2743191012 >> v1:172.21.15.131:6789/0 conn(0x7f58c0072540 legacy=0x7f58c00cfe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:03.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.501+0000 7f58bf400640 1 -- 172.21.15.131:0/2743191012 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58c00cf4e0 con 0x7f58c00d0560 2024-05-26T22:53:03.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.502+0000 7f58bf400640 1 --2- 172.21.15.131:0/2743191012 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f58c00d0560 0x7f58c0161ff0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f58b8009e50 tx=0x7f58b802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14c2a4ff1ab285a8 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:03.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.503+0000 7f58be000640 1 -- 172.21.15.131:0/2743191012 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58b8036070 con 0x7f58c00d0560 2024-05-26T22:53:03.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.503+0000 7f58be000640 1 -- 172.21.15.131:0/2743191012 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58b8031600 con 0x7f58c00d0560 2024-05-26T22:53:03.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.503+0000 7f58be000640 1 -- 172.21.15.131:0/2743191012 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58b80319d0 con 0x7f58c00d0560 2024-05-26T22:53:03.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.503+0000 7f58c5c00640 1 -- 172.21.15.131:0/2743191012 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f58c00d0560 msgr2=0x7f58c0161ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:03.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.503+0000 7f58c5c00640 1 --2- 172.21.15.131:0/2743191012 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f58c00d0560 0x7f58c0161ff0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f58b8009e50 tx=0x7f58b802a710 comp rx=0 tx=0).stop 2024-05-26T22:53:03.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.504+0000 7f58c5c00640 1 -- 172.21.15.131:0/2743191012 shutdown_connections 2024-05-26T22:53:03.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.504+0000 7f58c5c00640 1 --2- 172.21.15.131:0/2743191012 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f58c00d0560 0x7f58c0161ff0 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:03.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.504+0000 7f58c5c00640 1 --2- 172.21.15.131:0/2743191012 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f58c006f180 0x7f58c0072e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:03.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.504+0000 7f58c5c00640 1 -- 172.21.15.131:0/2743191012 >> 172.21.15.131:0/2743191012 conn(0x7f58c006bc90 msgr2=0x7f58c00b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:03.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.504+0000 7f58c5c00640 1 -- 172.21.15.131:0/2743191012 shutdown_connections 2024-05-26T22:53:03.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.504+0000 7f58c5c00640 1 -- 172.21.15.131:0/2743191012 wait complete. 2024-05-26T22:53:03.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.506+0000 7f58c5c00640 1 Processor -- start 2024-05-26T22:53:03.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.507+0000 7f58c5c00640 1 -- start start 2024-05-26T22:53:03.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.508+0000 7f58c5c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f58c006f180 0x7f58c014da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:03.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.508+0000 7f58c5c00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f58c0072540 0x7f58c009b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:03.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.508+0000 7f58c5c00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f58c00d0560 0x7f58c014e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:03.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.508+0000 7f58c5c00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f58c006f6e0 con 0x7f58c006f180 2024-05-26T22:53:03.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.508+0000 7f58c5c00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f58c006f560 con 0x7f58c0072540 2024-05-26T22:53:03.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.508+0000 7f58c5c00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f58c00715a0 con 0x7f58c00d0560 2024-05-26T22:53:03.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.508+0000 7f58bea00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f58c006f180 0x7f58c014da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:03.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.509+0000 7f58bea00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f58c006f180 0x7f58c014da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48632/0 (socket says 172.21.15.131:48632) 2024-05-26T22:53:03.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.509+0000 7f58bea00640 1 -- 172.21.15.131:0/4147107320 learned_addr learned my addr 172.21.15.131:0/4147107320 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:53:03.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.509+0000 7f58bf400640 1 --2- 172.21.15.131:0/4147107320 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f58c00d0560 0x7f58c014e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:03.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.509+0000 7f58bea00640 1 -- 172.21.15.131:0/4147107320 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f58c00d0560 msgr2=0x7f58c014e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:03.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.509+0000 7f58bea00640 1 --2- 172.21.15.131:0/4147107320 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f58c00d0560 0x7f58c014e1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:03.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.509+0000 7f58b7e00640 1 --2- 172.21.15.131:0/4147107320 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f58c0072540 0x7f58c009b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:03.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.509+0000 7f58bea00640 1 -- 172.21.15.131:0/4147107320 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f58c0072540 msgr2=0x7f58c009b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:03.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.509+0000 7f58bea00640 1 --2- 172.21.15.131:0/4147107320 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f58c0072540 0x7f58c009b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:03.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.509+0000 7f58bea00640 1 -- 172.21.15.131:0/4147107320 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58c009b950 con 0x7f58c006f180 2024-05-26T22:53:03.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.509+0000 7f58bea00640 1 --2- 172.21.15.131:0/4147107320 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f58c006f180 0x7f58c014da40 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f58b000d800 tx=0x7f58b000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:03.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.509+0000 7f58b6a00640 1 -- 172.21.15.131:0/4147107320 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58b000b6c0 con 0x7f58c006f180 2024-05-26T22:53:03.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.510+0000 7f58b6a00640 1 -- 172.21.15.131:0/4147107320 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58b0019070 con 0x7f58c006f180 2024-05-26T22:53:03.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.510+0000 7f58b6a00640 1 -- 172.21.15.131:0/4147107320 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58b000ba80 con 0x7f58c006f180 2024-05-26T22:53:03.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.510+0000 7f58c5c00640 1 -- 172.21.15.131:0/4147107320 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58c009bbf0 con 0x7f58c006f180 2024-05-26T22:53:03.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.510+0000 7f58c5c00640 1 -- 172.21.15.131:0/4147107320 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58c00ad750 con 0x7f58c006f180 2024-05-26T22:53:03.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.511+0000 7f58c5c00640 1 -- 172.21.15.131:0/4147107320 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58c0074e90 con 0x7f58c006f180 2024-05-26T22:53:03.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.512+0000 7f58b6a00640 1 -- 172.21.15.131:0/4147107320 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f58b000bc20 con 0x7f58c006f180 2024-05-26T22:53:03.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.513+0000 7f58b6a00640 1 --2- 172.21.15.131:0/4147107320 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f589c06be60 0x7f589c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:03.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.513+0000 7f58b6a00640 1 -- 172.21.15.131:0/4147107320 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7f58b008b370 con 0x7f58c006f180 2024-05-26T22:53:03.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.513+0000 7f58b7e00640 1 --2- 172.21.15.131:0/4147107320 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f589c06be60 0x7f589c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:03.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.514+0000 7f58b7e00640 1 --2- 172.21.15.131:0/4147107320 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f589c06be60 0x7f589c06e320 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f58c00b57b0 tx=0x7f58a8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:03.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.518+0000 7f58b6a00640 1 -- 172.21.15.131:0/4147107320 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f58b0059c80 con 0x7f58c006f180 2024-05-26T22:53:03.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.722+0000 7f58c5c00640 1 -- 172.21.15.131:0/4147107320 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command([{prefix=config set, name=mds_export_ephemeral_distributed}] v 0) -- 0x7f58c00b57e0 con 0x7f58c006f180 2024-05-26T22:53:03.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.728+0000 7f58b6a00640 1 -- 172.21.15.131:0/4147107320 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_export_ephemeral_distributed}]=0 v37)=0 v37) ==== 133+0+0 (secure 0 0 0) 0x7f58b0059580 con 0x7f58c006f180 2024-05-26T22:53:03.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.731+0000 7f58c5c00640 1 -- 172.21.15.131:0/4147107320 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f589c06be60 msgr2=0x7f589c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:03.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.731+0000 7f58c5c00640 1 --2- 172.21.15.131:0/4147107320 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f589c06be60 0x7f589c06e320 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f58c00b57b0 tx=0x7f58a8009040 comp rx=0 tx=0).stop 2024-05-26T22:53:03.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.731+0000 7f58c5c00640 1 -- 172.21.15.131:0/4147107320 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f58c006f180 msgr2=0x7f58c014da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:03.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.731+0000 7f58c5c00640 1 --2- 172.21.15.131:0/4147107320 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f58c006f180 0x7f58c014da40 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f58b000d800 tx=0x7f58b000dcc0 comp rx=0 tx=0).stop 2024-05-26T22:53:03.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.731+0000 7f58c5c00640 1 -- 172.21.15.131:0/4147107320 shutdown_connections 2024-05-26T22:53:03.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.731+0000 7f58c5c00640 1 --2- 172.21.15.131:0/4147107320 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f58c00d0560 0x7f58c014e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:03.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.731+0000 7f58c5c00640 1 --2- 172.21.15.131:0/4147107320 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f589c06be60 0x7f589c06e320 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:03.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.731+0000 7f58c5c00640 1 --2- 172.21.15.131:0/4147107320 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f58c0072540 0x7f58c009b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:03.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.731+0000 7f58c5c00640 1 --2- 172.21.15.131:0/4147107320 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f58c006f180 0x7f58c014da40 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:03.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.731+0000 7f58c5c00640 1 -- 172.21.15.131:0/4147107320 >> 172.21.15.131:0/4147107320 conn(0x7f58c006bc90 msgr2=0x7f58c00b0b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:03.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.731+0000 7f58c5c00640 1 -- 172.21.15.131:0/4147107320 shutdown_connections 2024-05-26T22:53:03.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.731+0000 7f58c5c00640 1 -- 172.21.15.131:0/4147107320 wait complete. 2024-05-26T22:53:03.751 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_export_ephemeral_random_max 1.0 2024-05-26T22:53:03.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.943+0000 7f0d8bc00640 1 Processor -- start 2024-05-26T22:53:03.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.943+0000 7f0d8bc00640 1 -- start start 2024-05-26T22:53:03.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.944+0000 7f0d8bc00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f0d8406f420 0x7f0d840cfd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:03.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.944+0000 7f0d8bc00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0d840cf590 0x7f0d840d0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:03.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.944+0000 7f0d8bc00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d84160180 con 0x7f0d840b5450 2024-05-26T22:53:03.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.944+0000 7f0d8bc00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f0d84160d90 con 0x7f0d8406f420 2024-05-26T22:53:03.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.944+0000 7f0d8bc00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f0d84160460 con 0x7f0d840cf590 2024-05-26T22:53:03.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.945+0000 7f0d83e00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f0d8406f420 0x7f0d840cfd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:03.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.945+0000 7f0d83e00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f0d8406f420 0x7f0d840cfd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.131:53950/0 (socket says 172.21.15.131:53950) 2024-05-26T22:53:03.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.945+0000 7f0d83e00640 1 -- 172.21.15.131:0/2345291007 learned_addr learned my addr 172.21.15.131:0/2345291007 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:53:03.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.945+0000 7f0d89c00640 1 --2- 172.21.15.131:0/2345291007 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0d840cf590 0x7f0d840d0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:03.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.946+0000 7f0d83e00640 1 -- 172.21.15.131:0/2345291007 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0d840cf590 msgr2=0x7f0d840d0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:03.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.946+0000 7f0d83e00640 1 --2- 172.21.15.131:0/2345291007 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0d840cf590 0x7f0d840d0290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:03.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.946+0000 7f0d83e00640 1 -- 172.21.15.131:0/2345291007 >> v1:172.21.15.187:6789/0 conn(0x7f0d840b5450 legacy=0x7f0d840b43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:03.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.946+0000 7f0d83e00640 1 -- 172.21.15.131:0/2345291007 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d84160740 con 0x7f0d8406f420 2024-05-26T22:53:03.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.947+0000 7f0d83e00640 1 --2- 172.21.15.131:0/2345291007 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0d8406f420 0x7f0d840cfd50 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f0d70009870 tx=0x7f0d7002a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7faf49639b3bf32 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:03.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.947+0000 7f0d83400640 1 -- 172.21.15.131:0/2345291007 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d70037070 con 0x7f0d8406f420 2024-05-26T22:53:03.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.947+0000 7f0d83400640 1 -- 172.21.15.131:0/2345291007 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d70032440 con 0x7f0d8406f420 2024-05-26T22:53:03.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.947+0000 7f0d83400640 1 -- 172.21.15.131:0/2345291007 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d70032810 con 0x7f0d8406f420 2024-05-26T22:53:03.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.948+0000 7f0d8bc00640 1 -- 172.21.15.131:0/2345291007 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0d8406f420 msgr2=0x7f0d840cfd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:03.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.948+0000 7f0d8bc00640 1 --2- 172.21.15.131:0/2345291007 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0d8406f420 0x7f0d840cfd50 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f0d70009870 tx=0x7f0d7002a270 comp rx=0 tx=0).stop 2024-05-26T22:53:03.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.948+0000 7f0d8bc00640 1 -- 172.21.15.131:0/2345291007 shutdown_connections 2024-05-26T22:53:03.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.948+0000 7f0d8bc00640 1 --2- 172.21.15.131:0/2345291007 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0d840cf590 0x7f0d840d0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:03.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.948+0000 7f0d8bc00640 1 --2- 172.21.15.131:0/2345291007 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0d8406f420 0x7f0d840cfd50 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:03.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.948+0000 7f0d8bc00640 1 -- 172.21.15.131:0/2345291007 >> 172.21.15.131:0/2345291007 conn(0x7f0d8406bc90 msgr2=0x7f0d840b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:03.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.948+0000 7f0d8bc00640 1 -- 172.21.15.131:0/2345291007 shutdown_connections 2024-05-26T22:53:03.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.948+0000 7f0d8bc00640 1 -- 172.21.15.131:0/2345291007 wait complete. 2024-05-26T22:53:03.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.950+0000 7f0d8bc00640 1 Processor -- start 2024-05-26T22:53:03.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.950+0000 7f0d8bc00640 1 -- start start 2024-05-26T22:53:03.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.951+0000 7f0d8bc00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0d8406f420 0x7f0d8409cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:03.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.951+0000 7f0d8bc00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0d840b5450 0x7f0d84072ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:03.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.951+0000 7f0d8bc00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0d840cf590 0x7f0d84153390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:03.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.951+0000 7f0d8bc00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f0d8415fde0 con 0x7f0d8406f420 2024-05-26T22:53:03.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.951+0000 7f0d8bc00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0d840b5830 con 0x7f0d840cf590 2024-05-26T22:53:03.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.951+0000 7f0d8bc00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f0d8415ff60 con 0x7f0d840b5450 2024-05-26T22:53:03.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.951+0000 7f0d89200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0d8406f420 0x7f0d8409cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:03.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.951+0000 7f0d89200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0d8406f420 0x7f0d8409cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48648/0 (socket says 172.21.15.131:48648) 2024-05-26T22:53:03.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.951+0000 7f0d89200640 1 -- 172.21.15.131:0/3594222737 learned_addr learned my addr 172.21.15.131:0/3594222737 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:53:03.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.952+0000 7f0d89c00640 1 --2- 172.21.15.131:0/3594222737 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0d840cf590 0x7f0d84153390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:03.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.952+0000 7f0d83e00640 1 --2- 172.21.15.131:0/3594222737 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0d840b5450 0x7f0d84072ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:03.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.952+0000 7f0d89200640 1 -- 172.21.15.131:0/3594222737 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0d840b5450 msgr2=0x7f0d84072ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:03.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.952+0000 7f0d89200640 1 --2- 172.21.15.131:0/3594222737 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0d840b5450 0x7f0d84072ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:03.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.952+0000 7f0d89200640 1 -- 172.21.15.131:0/3594222737 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0d840cf590 msgr2=0x7f0d84153390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:03.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.952+0000 7f0d89200640 1 --2- 172.21.15.131:0/3594222737 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0d840cf590 0x7f0d84153390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:03.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.952+0000 7f0d89200640 1 -- 172.21.15.131:0/3594222737 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d841538d0 con 0x7f0d8406f420 2024-05-26T22:53:03.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.952+0000 7f0d89200640 1 --2- 172.21.15.131:0/3594222737 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0d8406f420 0x7f0d8409cd60 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f0d7800c400 tx=0x7f0d7800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:03.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.952+0000 7f0d80c00640 1 -- 172.21.15.131:0/3594222737 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d780052d0 con 0x7f0d8406f420 2024-05-26T22:53:03.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.952+0000 7f0d80c00640 1 -- 172.21.15.131:0/3594222737 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d78005490 con 0x7f0d8406f420 2024-05-26T22:53:03.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.952+0000 7f0d80c00640 1 -- 172.21.15.131:0/3594222737 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d78007260 con 0x7f0d8406f420 2024-05-26T22:53:03.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.952+0000 7f0d8bc00640 1 -- 172.21.15.131:0/3594222737 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d8409d460 con 0x7f0d8406f420 2024-05-26T22:53:03.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.953+0000 7f0d8bc00640 1 -- 172.21.15.131:0/3594222737 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d8409af30 con 0x7f0d8406f420 2024-05-26T22:53:03.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.954+0000 7f0d8bc00640 1 -- 172.21.15.131:0/3594222737 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d84074e90 con 0x7f0d8406f420 2024-05-26T22:53:03.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.955+0000 7f0d80c00640 1 -- 172.21.15.131:0/3594222737 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0d78007400 con 0x7f0d8406f420 2024-05-26T22:53:03.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.956+0000 7f0d80c00640 1 --2- 172.21.15.131:0/3594222737 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f0d6406be30 0x7f0d6406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:03.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.956+0000 7f0d80c00640 1 -- 172.21.15.131:0/3594222737 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7f0d7808a490 con 0x7f0d8406f420 2024-05-26T22:53:03.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.956+0000 7f0d83e00640 1 --2- 172.21.15.131:0/3594222737 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f0d6406be30 0x7f0d6406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:03.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.957+0000 7f0d83e00640 1 --2- 172.21.15.131:0/3594222737 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f0d6406be30 0x7f0d6406e2f0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f0d70034000 tx=0x7f0d7002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:03.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:03.964+0000 7f0d80c00640 1 -- 172.21.15.131:0/3594222737 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f0d78058da0 con 0x7f0d8406f420 2024-05-26T22:53:04.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:04.163+0000 7f0d8bc00640 1 -- 172.21.15.131:0/3594222737 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command([{prefix=config set, name=mds_export_ephemeral_random_max}] v 0) -- 0x7f0d840695c0 con 0x7f0d8406f420 2024-05-26T22:53:04.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:04.168+0000 7f0d80c00640 1 -- 172.21.15.131:0/3594222737 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_export_ephemeral_random_max}]=0 v38)=0 v38) ==== 131+0+0 (secure 0 0 0) 0x7f0d780586a0 con 0x7f0d8406f420 2024-05-26T22:53:04.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:04.171+0000 7f0d8bc00640 1 -- 172.21.15.131:0/3594222737 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f0d6406be30 msgr2=0x7f0d6406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:04.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:04.171+0000 7f0d8bc00640 1 --2- 172.21.15.131:0/3594222737 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f0d6406be30 0x7f0d6406e2f0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f0d70034000 tx=0x7f0d7002e000 comp rx=0 tx=0).stop 2024-05-26T22:53:04.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:04.171+0000 7f0d8bc00640 1 -- 172.21.15.131:0/3594222737 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0d8406f420 msgr2=0x7f0d8409cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:04.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:04.171+0000 7f0d8bc00640 1 --2- 172.21.15.131:0/3594222737 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0d8406f420 0x7f0d8409cd60 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f0d7800c400 tx=0x7f0d7800c8c0 comp rx=0 tx=0).stop 2024-05-26T22:53:04.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:04.171+0000 7f0d8bc00640 1 -- 172.21.15.131:0/3594222737 shutdown_connections 2024-05-26T22:53:04.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:04.171+0000 7f0d8bc00640 1 --2- 172.21.15.131:0/3594222737 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0d840b5450 0x7f0d84072ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:04.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:04.171+0000 7f0d8bc00640 1 --2- 172.21.15.131:0/3594222737 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f0d6406be30 0x7f0d6406e2f0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:04.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:04.171+0000 7f0d8bc00640 1 --2- 172.21.15.131:0/3594222737 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0d840cf590 0x7f0d84153390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:04.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:04.171+0000 7f0d8bc00640 1 --2- 172.21.15.131:0/3594222737 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0d8406f420 0x7f0d8409cd60 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:04.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:04.171+0000 7f0d8bc00640 1 -- 172.21.15.131:0/3594222737 >> 172.21.15.131:0/3594222737 conn(0x7f0d8406bc90 msgr2=0x7f0d840b1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:04.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:04.171+0000 7f0d8bc00640 1 -- 172.21.15.131:0/3594222737 shutdown_connections 2024-05-26T22:53:04.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:04.171+0000 7f0d8bc00640 1 -- 172.21.15.131:0/3594222737 wait complete. 2024-05-26T22:53:04.187 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-05-26T22:53:04.187 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2024-05-26T22:53:04.187 DEBUG:teuthology.orchestra.run.smithi131:> 2024-05-26T22:53:04.187 DEBUG:teuthology.orchestra.run.smithi131:> # Use up a random number of inode numbers so the ephemeral pinning is not the same every test. 2024-05-26T22:53:04.187 DEBUG:teuthology.orchestra.run.smithi131:> mkdir .inode_number_thrash 2024-05-26T22:53:04.188 DEBUG:teuthology.orchestra.run.smithi131:> count=$((RANDOM % 1024)) 2024-05-26T22:53:04.188 DEBUG:teuthology.orchestra.run.smithi131:> for ((i = 0; i < count; i++)); do touch .inode_number_thrash/$i; done 2024-05-26T22:53:04.188 DEBUG:teuthology.orchestra.run.smithi131:> rm -rf .inode_number_thrash 2024-05-26T22:53:04.188 DEBUG:teuthology.orchestra.run.smithi131:> ') 2024-05-26T22:53:04.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:04 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-26T22:53:06.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-26T22:53:06.512 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2024-05-26T22:53:06.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:06.723+0000 7fdfdac00640 1 Processor -- start 2024-05-26T22:53:06.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:06.723+0000 7fdfdac00640 1 -- start start 2024-05-26T22:53:06.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:06.725+0000 7fdfdac00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fdfd40cf190 0x7fdfd4070860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:06.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:06.725+0000 7fdfdac00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdfd4072fe0 0x7fdfd40d0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:06.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:06.725+0000 7fdfdac00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfd415f9f0 con 0x7fdfd41449f0 2024-05-26T22:53:06.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:06.725+0000 7fdfdac00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fdfd40b09b0 con 0x7fdfd40cf190 2024-05-26T22:53:06.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:06.725+0000 7fdfdac00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fdfd40735a0 con 0x7fdfd4072fe0 2024-05-26T22:53:06.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:06.725+0000 7fdfd9800640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fdfd41449f0 0x7fdfd40cea80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:53180/0 (socket says 172.21.15.131:53180) 2024-05-26T22:53:06.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:06.725+0000 7fdfd9800640 1 -- 172.21.15.131:0/4123716042 learned_addr learned my addr 172.21.15.131:0/4123716042 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:53:06.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:06.725+0000 7fdfd8e00640 1 --2- 172.21.15.131:0/4123716042 >> v2:172.21.15.131:3300/0 conn(0x7fdfd40cf190 0x7fdfd4070860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:06.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:06.726+0000 7fdfd3e00640 1 -- 172.21.15.131:0/4123716042 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1695743574 0 0) 0x7fdfd415f9f0 con 0x7fdfd41449f0 2024-05-26T22:53:06.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:06.726+0000 7fdfd3e00640 1 -- 172.21.15.131:0/4123716042 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdfbc003520 con 0x7fdfd41449f0 2024-05-26T22:53:06.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:06.726+0000 7fdfda200640 1 --2- 172.21.15.131:0/4123716042 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdfd4072fe0 0x7fdfd40d0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:06.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:06.726+0000 7fdfd8e00640 1 -- 172.21.15.131:0/4123716042 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdfd4072fe0 msgr2=0x7fdfd40d0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:06.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:06.726+0000 7fdfd8e00640 1 --2- 172.21.15.131:0/4123716042 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdfd4072fe0 0x7fdfd40d0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:06.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:06.726+0000 7fdfd8e00640 1 -- 172.21.15.131:0/4123716042 >> v1:172.21.15.131:6789/0 conn(0x7fdfd41449f0 legacy=0x7fdfd40cea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:06.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:06.726+0000 7fdfd8e00640 1 -- 172.21.15.131:0/4123716042 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfd415ec30 con 0x7fdfd40cf190 2024-05-26T22:53:06.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:06.726+0000 7fdfda200640 1 --2- 172.21.15.131:0/4123716042 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdfd4072fe0 0x7fdfd40d0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:53:06.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:06.727+0000 7fdfd8e00640 1 --2- 172.21.15.131:0/4123716042 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdfd40cf190 0x7fdfd4070860 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fdfc00097a0 tx=0x7fdfc002a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13f00622168581ee server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:06.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:06.727+0000 7fdfd3e00640 1 -- 172.21.15.131:0/4123716042 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc0037070 con 0x7fdfd40cf190 2024-05-26T22:53:06.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:06.727+0000 7fdfd3e00640 1 -- 172.21.15.131:0/4123716042 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfc002adb0 con 0x7fdfd40cf190 2024-05-26T22:53:06.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:06.727+0000 7fdfd3e00640 1 -- 172.21.15.131:0/4123716042 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc0032660 con 0x7fdfd40cf190 2024-05-26T22:53:06.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:06.728+0000 7fdfdac00640 1 -- 172.21.15.131:0/4123716042 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdfd40cf190 msgr2=0x7fdfd4070860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:06.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:06.728+0000 7fdfdac00640 1 --2- 172.21.15.131:0/4123716042 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdfd40cf190 0x7fdfd4070860 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fdfc00097a0 tx=0x7fdfc002a060 comp rx=0 tx=0).stop 2024-05-26T22:53:06.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:06.728+0000 7fdfdac00640 1 -- 172.21.15.131:0/4123716042 shutdown_connections 2024-05-26T22:53:06.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:06.728+0000 7fdfdac00640 1 --2- 172.21.15.131:0/4123716042 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdfd4072fe0 0x7fdfd40d0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:06.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:06.728+0000 7fdfdac00640 1 --2- 172.21.15.131:0/4123716042 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdfd40cf190 0x7fdfd4070860 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:06.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:06.728+0000 7fdfdac00640 1 -- 172.21.15.131:0/4123716042 >> 172.21.15.131:0/4123716042 conn(0x7fdfd406e4d0 msgr2=0x7fdfd406e8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:06.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:06.728+0000 7fdfdac00640 1 -- 172.21.15.131:0/4123716042 shutdown_connections 2024-05-26T22:53:06.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:06.728+0000 7fdfdac00640 1 -- 172.21.15.131:0/4123716042 wait complete. 2024-05-26T22:53:06.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:06.730+0000 7fdfdac00640 1 Processor -- start 2024-05-26T22:53:06.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:06.730+0000 7fdfdac00640 1 -- start start 2024-05-26T22:53:06.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:06.731+0000 7fdfdac00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdfd4072fe0 0x7fdfd413a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:06.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:06.731+0000 7fdfdac00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdfd40cf190 0x7fdfd415d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:06.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:06.731+0000 7fdfdac00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdfd41449f0 0x7fdfd4159c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:06.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:06.731+0000 7fdfdac00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd415f4d0 con 0x7fdfd41449f0 2024-05-26T22:53:06.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:06.731+0000 7fdfdac00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd415f1f0 con 0x7fdfd40cf190 2024-05-26T22:53:06.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:06.731+0000 7fdfdac00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fdfd4144dd0 con 0x7fdfd4072fe0 2024-05-26T22:53:06.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:06.731+0000 7fdfda200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdfd41449f0 0x7fdfd4159c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:06.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:06.731+0000 7fdfda200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdfd41449f0 0x7fdfd4159c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40630/0 (socket says 172.21.15.131:40630) 2024-05-26T22:53:06.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:06.731+0000 7fdfda200640 1 -- 172.21.15.131:0/621527311 learned_addr learned my addr 172.21.15.131:0/621527311 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:53:06.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:06.731+0000 7fdfd8e00640 1 --2- 172.21.15.131:0/621527311 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdfd40cf190 0x7fdfd415d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:06.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:06.731+0000 7fdfd9800640 1 --2- 172.21.15.131:0/621527311 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdfd4072fe0 0x7fdfd413a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:06.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:06.732+0000 7fdfda200640 1 -- 172.21.15.131:0/621527311 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdfd4072fe0 msgr2=0x7fdfd413a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:06.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:06.732+0000 7fdfda200640 1 --2- 172.21.15.131:0/621527311 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdfd4072fe0 0x7fdfd413a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:06.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:06.732+0000 7fdfda200640 1 -- 172.21.15.131:0/621527311 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdfd40cf190 msgr2=0x7fdfd415d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:06.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:06.732+0000 7fdfda200640 1 --2- 172.21.15.131:0/621527311 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdfd40cf190 0x7fdfd415d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:06.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:06.732+0000 7fdfda200640 1 -- 172.21.15.131:0/621527311 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfd413ad90 con 0x7fdfd41449f0 2024-05-26T22:53:06.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:06.732+0000 7fdfd8e00640 1 --2- 172.21.15.131:0/621527311 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdfd40cf190 0x7fdfd415d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:53:06.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:06.732+0000 7fdfd9800640 1 --2- 172.21.15.131:0/621527311 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdfd4072fe0 0x7fdfd413a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:53:06.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:06.732+0000 7fdfda200640 1 --2- 172.21.15.131:0/621527311 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdfd41449f0 0x7fdfd4159c10 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc00cef0 tx=0x7fdfcc00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:06.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:06.732+0000 7fdfd1600640 1 -- 172.21.15.131:0/621527311 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfcc01a070 con 0x7fdfd41449f0 2024-05-26T22:53:06.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:06.732+0000 7fdfd1600640 1 -- 172.21.15.131:0/621527311 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfcc00d860 con 0x7fdfd41449f0 2024-05-26T22:53:06.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:06.732+0000 7fdfd1600640 1 -- 172.21.15.131:0/621527311 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfcc00dc50 con 0x7fdfd41449f0 2024-05-26T22:53:06.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:06.732+0000 7fdfdac00640 1 -- 172.21.15.131:0/621527311 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfd415cae0 con 0x7fdfd41449f0 2024-05-26T22:53:06.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:06.732+0000 7fdfdac00640 1 -- 172.21.15.131:0/621527311 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfd415d0a0 con 0x7fdfd41449f0 2024-05-26T22:53:06.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:06.734+0000 7fdfdac00640 1 -- 172.21.15.131:0/621527311 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfd40694e0 con 0x7fdfd41449f0 2024-05-26T22:53:06.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:06.734+0000 7fdfd1600640 1 -- 172.21.15.131:0/621527311 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdfcc01d070 con 0x7fdfd41449f0 2024-05-26T22:53:06.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:06.735+0000 7fdfd1600640 1 --2- 172.21.15.131:0/621527311 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fdfb406be30 0x7fdfb406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:06.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:06.735+0000 7fdfd1600640 1 -- 172.21.15.131:0/621527311 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5767+0+0 (secure 0 0 0) 0x7fdfcc08b7e0 con 0x7fdfd41449f0 2024-05-26T22:53:06.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:06.736+0000 7fdfd9800640 1 --2- 172.21.15.131:0/621527311 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fdfb406be30 0x7fdfb406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:06.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:06.736+0000 7fdfd9800640 1 --2- 172.21.15.131:0/621527311 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fdfb406be30 0x7fdfb406e2f0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fdfc8005880 tx=0x7fdfc8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:06.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:06.741+0000 7fdfd1600640 1 -- 172.21.15.131:0/621527311 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fdfcc05a170 con 0x7fdfd41449f0 2024-05-26T22:53:06.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:06.978+0000 7fdfdac00640 1 -- 172.21.15.131:0/621527311 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7fdfd40728e0 con 0x7fdfd41449f0 2024-05-26T22:53:07.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:07.123+0000 7fdfd1600640 1 -- 172.21.15.131:0/621527311 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v65) ==== 105+0+0 (secure 0 0 0) 0x7fdfcc059a70 con 0x7fdfd41449f0 2024-05-26T22:53:07.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:07.126+0000 7fdfdac00640 1 -- 172.21.15.131:0/621527311 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fdfb406be30 msgr2=0x7fdfb406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:07.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:07.126+0000 7fdfdac00640 1 --2- 172.21.15.131:0/621527311 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fdfb406be30 0x7fdfb406e2f0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fdfc8005880 tx=0x7fdfc8009040 comp rx=0 tx=0).stop 2024-05-26T22:53:07.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:07.126+0000 7fdfdac00640 1 -- 172.21.15.131:0/621527311 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdfd41449f0 msgr2=0x7fdfd4159c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:07.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:07.126+0000 7fdfdac00640 1 --2- 172.21.15.131:0/621527311 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdfd41449f0 0x7fdfd4159c10 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc00cef0 tx=0x7fdfcc00a710 comp rx=0 tx=0).stop 2024-05-26T22:53:07.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:07.126+0000 7fdfdac00640 1 -- 172.21.15.131:0/621527311 shutdown_connections 2024-05-26T22:53:07.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:07.126+0000 7fdfdac00640 1 --2- 172.21.15.131:0/621527311 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fdfb406be30 0x7fdfb406e2f0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:07.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:07.126+0000 7fdfdac00640 1 --2- 172.21.15.131:0/621527311 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdfd41449f0 0x7fdfd4159c10 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:07.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:07.126+0000 7fdfdac00640 1 --2- 172.21.15.131:0/621527311 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdfd40cf190 0x7fdfd415d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:07.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:07.126+0000 7fdfdac00640 1 --2- 172.21.15.131:0/621527311 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdfd4072fe0 0x7fdfd413a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:07.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:07.126+0000 7fdfdac00640 1 -- 172.21.15.131:0/621527311 >> 172.21.15.131:0/621527311 conn(0x7fdfd406e4d0 msgr2=0x7fdfd40ce510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:07.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:07.127+0000 7fdfdac00640 1 -- 172.21.15.131:0/621527311 shutdown_connections 2024-05-26T22:53:07.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:07.127+0000 7fdfdac00640 1 -- 172.21.15.131:0/621527311 wait complete. 2024-05-26T22:53:07.153 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-26T22:53:07.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:07.333+0000 7f26e1800640 1 Processor -- start 2024-05-26T22:53:07.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:07.333+0000 7f26e1800640 1 -- start start 2024-05-26T22:53:07.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:07.334+0000 7f26e1800640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f26dc071d80 0x7f26dc072160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:07.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:07.335+0000 7f26e1800640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26dc0726a0 con 0x7f26dc0ad450 2024-05-26T22:53:07.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:07.335+0000 7f26e1800640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26dc0b5190 con 0x7f26dc0ad8e0 2024-05-26T22:53:07.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:07.335+0000 7f26e1800640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f26dc06dee0 con 0x7f26dc071d80 2024-05-26T22:53:07.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:07.335+0000 7f26daa00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f26dc071d80 0x7f26dc072160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:07.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:07.335+0000 7f26daa00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f26dc071d80 0x7f26dc072160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40638/0 (socket says 172.21.15.131:40638) 2024-05-26T22:53:07.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:07.335+0000 7f26daa00640 1 -- 172.21.15.131:0/2976214818 learned_addr learned my addr 172.21.15.131:0/2976214818 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:53:07.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:07.335+0000 7f26d2000640 1 --1- 172.21.15.131:0/2976214818 >> v1:172.21.15.131:6789/0 conn(0x7f26dc0ad450 0x7f26dc06eca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:53182/0 (socket says 172.21.15.131:53182) 2024-05-26T22:53:07.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:07.335+0000 7f26daa00640 1 -- 172.21.15.131:0/2976214818 >> v1:172.21.15.187:6789/0 conn(0x7f26dc0ad8e0 legacy=0x7f26dc0b43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:07.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:07.335+0000 7f26daa00640 1 -- 172.21.15.131:0/2976214818 >> v1:172.21.15.131:6789/0 conn(0x7f26dc0ad450 legacy=0x7f26dc06eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:07.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:07.335+0000 7f26daa00640 1 -- 172.21.15.131:0/2976214818 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26dc06e1c0 con 0x7f26dc071d80 2024-05-26T22:53:07.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:07.336+0000 7f26daa00640 1 --2- 172.21.15.131:0/2976214818 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f26dc071d80 0x7f26dc072160 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f26cc00aa10 tx=0x7f26cc02b5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec1296ffc0a828a4 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:07.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:07.337+0000 7f26da000640 1 -- 172.21.15.131:0/2976214818 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26cc00b040 con 0x7f26dc071d80 2024-05-26T22:53:07.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:07.337+0000 7f26da000640 1 -- 172.21.15.131:0/2976214818 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26cc030710 con 0x7f26dc071d80 2024-05-26T22:53:07.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:07.337+0000 7f26da000640 1 -- 172.21.15.131:0/2976214818 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26cc030ae0 con 0x7f26dc071d80 2024-05-26T22:53:07.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:07.337+0000 7f26e1800640 1 -- 172.21.15.131:0/2976214818 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f26dc071d80 msgr2=0x7f26dc072160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:07.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:07.337+0000 7f26e1800640 1 --2- 172.21.15.131:0/2976214818 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f26dc071d80 0x7f26dc072160 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f26cc00aa10 tx=0x7f26cc02b5e0 comp rx=0 tx=0).stop 2024-05-26T22:53:07.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:07.338+0000 7f26e1800640 1 -- 172.21.15.131:0/2976214818 shutdown_connections 2024-05-26T22:53:07.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:07.338+0000 7f26e1800640 1 --2- 172.21.15.131:0/2976214818 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f26dc071d80 0x7f26dc072160 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:07.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:07.338+0000 7f26e1800640 1 -- 172.21.15.131:0/2976214818 >> 172.21.15.131:0/2976214818 conn(0x7f26dc0b1e90 msgr2=0x7f26dc0b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:07.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:07.338+0000 7f26e1800640 1 -- 172.21.15.131:0/2976214818 shutdown_connections 2024-05-26T22:53:07.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:07.338+0000 7f26e1800640 1 -- 172.21.15.131:0/2976214818 wait complete. 2024-05-26T22:53:07.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:07.340+0000 7f26e1800640 1 Processor -- start 2024-05-26T22:53:07.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:07.340+0000 7f26e1800640 1 -- start start 2024-05-26T22:53:07.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:07.341+0000 7f26e1800640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f26dc071d80 0x7f26dc15cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:07.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:07.341+0000 7f26e1800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f26dc0ad450 0x7f26dc15c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:07.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:07.341+0000 7f26e1800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f26dc1e3060 0x7f26dc1f7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:07.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:07.341+0000 7f26e1800640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f26dc15e020 con 0x7f26dc0ad450 2024-05-26T22:53:07.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:07.341+0000 7f26e1800640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f26dc06e780 con 0x7f26dc071d80 2024-05-26T22:53:07.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:07.341+0000 7f26e1800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f26dc15e1a0 con 0x7f26dc1e3060 2024-05-26T22:53:07.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:07.342+0000 7f26d2000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f26dc0ad450 0x7f26dc15c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:07.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:07.342+0000 7f26d2000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f26dc0ad450 0x7f26dc15c230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40650/0 (socket says 172.21.15.131:40650) 2024-05-26T22:53:07.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:07.342+0000 7f26d2000640 1 -- 172.21.15.131:0/1976692042 learned_addr learned my addr 172.21.15.131:0/1976692042 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:53:07.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:07.342+0000 7f26daa00640 1 --2- 172.21.15.131:0/1976692042 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f26dc071d80 0x7f26dc15cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:07.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:07.342+0000 7f26db400640 1 --2- 172.21.15.131:0/1976692042 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f26dc1e3060 0x7f26dc1f7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:07.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:07.342+0000 7f26d2000640 1 -- 172.21.15.131:0/1976692042 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f26dc1e3060 msgr2=0x7f26dc1f7ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:07.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:07.342+0000 7f26d2000640 1 --2- 172.21.15.131:0/1976692042 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f26dc1e3060 0x7f26dc1f7ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:07.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:07.342+0000 7f26d2000640 1 -- 172.21.15.131:0/1976692042 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f26dc071d80 msgr2=0x7f26dc15cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:07.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:07.342+0000 7f26d2000640 1 --2- 172.21.15.131:0/1976692042 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f26dc071d80 0x7f26dc15cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:07.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:07.342+0000 7f26d2000640 1 -- 172.21.15.131:0/1976692042 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26dc0ce8b0 con 0x7f26dc0ad450 2024-05-26T22:53:07.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:07.342+0000 7f26daa00640 1 --2- 172.21.15.131:0/1976692042 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f26dc071d80 0x7f26dc15cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-26T22:53:07.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:07.343+0000 7f26d2000640 1 --2- 172.21.15.131:0/1976692042 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f26dc0ad450 0x7f26dc15c230 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f26c400c400 tx=0x7f26c400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:07.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:07.343+0000 7f26d3400640 1 -- 172.21.15.131:0/1976692042 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c40074e0 con 0x7f26dc0ad450 2024-05-26T22:53:07.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:07.343+0000 7f26d3400640 1 -- 172.21.15.131:0/1976692042 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26c4007680 con 0x7f26dc0ad450 2024-05-26T22:53:07.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:07.343+0000 7f26d3400640 1 -- 172.21.15.131:0/1976692042 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c4004a00 con 0x7f26dc0ad450 2024-05-26T22:53:07.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:07.343+0000 7f26e1800640 1 -- 172.21.15.131:0/1976692042 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26dc0ceb90 con 0x7f26dc0ad450 2024-05-26T22:53:07.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:07.343+0000 7f26e1800640 1 -- 172.21.15.131:0/1976692042 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26dc0cf150 con 0x7f26dc0ad450 2024-05-26T22:53:07.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:07.344+0000 7f26e1800640 1 -- 172.21.15.131:0/1976692042 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26dc0694e0 con 0x7f26dc0ad450 2024-05-26T22:53:07.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:07.345+0000 7f26d3400640 1 -- 172.21.15.131:0/1976692042 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f26c401a020 con 0x7f26dc0ad450 2024-05-26T22:53:07.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:07.346+0000 7f26d3400640 1 --2- 172.21.15.131:0/1976692042 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f26b806be30 0x7f26b806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:07.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:07.346+0000 7f26d3400640 1 -- 172.21.15.131:0/1976692042 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5767+0+0 (secure 0 0 0) 0x7f26c408a710 con 0x7f26dc0ad450 2024-05-26T22:53:07.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:07.346+0000 7f26daa00640 1 --2- 172.21.15.131:0/1976692042 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f26b806be30 0x7f26b806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:07.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:07.347+0000 7f26daa00640 1 --2- 172.21.15.131:0/1976692042 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f26b806be30 0x7f26b806e2f0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f26cc035000 tx=0x7f26cc00c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:07.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:07.351+0000 7f26d3400640 1 -- 172.21.15.131:0/1976692042 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f26c4059070 con 0x7f26dc0ad450 2024-05-26T22:53:07.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:07.609+0000 7f26e1800640 1 -- 172.21.15.131:0/1976692042 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f26dc0b4540 con 0x7f26dc0ad450 2024-05-26T22:53:07.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:07.610+0000 7f26d3400640 1 -- 172.21.15.131:0/1976692042 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) ==== 94+0+5211 (secure 0 0 0) 0x7f26c4058970 con 0x7f26dc0ad450 2024-05-26T22:53:07.611 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:53:07.611 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":65,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5359,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/4204396845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6833","nonce":4204396845},{"type":"v1","addr":"172.21.15.187:6835","nonce":4204396845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59},{"gid":6861,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6836/4223534499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":4223534499},{"type":"v1","addr":"172.21.15.131:6836","nonce":4223534499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-26T22:52:53.755842+0000","modified":"2024-05-26T22:53:07.120270+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1],"up":{"mds_0":6876,"mds_1":6864},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6864":{"gid":6864,"name":"c","rank":1,"incarnation":65,"state":"up:creating","state_seq":1,"addr":"172.21.15.131:6837/1418741551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6835","nonce":1418741551},{"type":"v1","addr":"172.21.15.131:6837","nonce":1418741551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6876":{"gid":6876,"name":"d","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6834/3680999672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3680999672},{"type":"v1","addr":"172.21.15.187:6834","nonce":3680999672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6876,"qdb_cluster":[6876]},"id":5}]} 2024-05-26T22:53:07.611 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 65 2024-05-26T22:53:07.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:07.613+0000 7f26e1800640 1 -- 172.21.15.131:0/1976692042 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f26b806be30 msgr2=0x7f26b806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:07.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:07.613+0000 7f26e1800640 1 --2- 172.21.15.131:0/1976692042 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f26b806be30 0x7f26b806e2f0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f26cc035000 tx=0x7f26cc00c040 comp rx=0 tx=0).stop 2024-05-26T22:53:07.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:07.613+0000 7f26e1800640 1 -- 172.21.15.131:0/1976692042 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f26dc0ad450 msgr2=0x7f26dc15c230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:07.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:07.613+0000 7f26e1800640 1 --2- 172.21.15.131:0/1976692042 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f26dc0ad450 0x7f26dc15c230 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f26c400c400 tx=0x7f26c400c8c0 comp rx=0 tx=0).stop 2024-05-26T22:53:07.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:07.613+0000 7f26e1800640 1 -- 172.21.15.131:0/1976692042 shutdown_connections 2024-05-26T22:53:07.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:07.613+0000 7f26e1800640 1 --2- 172.21.15.131:0/1976692042 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f26dc1e3060 0x7f26dc1f7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:07.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:07.613+0000 7f26e1800640 1 --2- 172.21.15.131:0/1976692042 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f26b806be30 0x7f26b806e2f0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:07.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:07.613+0000 7f26e1800640 1 --2- 172.21.15.131:0/1976692042 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f26dc0ad450 0x7f26dc15c230 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:07.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:07.613+0000 7f26e1800640 1 --2- 172.21.15.131:0/1976692042 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f26dc071d80 0x7f26dc15cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:07.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:07.613+0000 7f26e1800640 1 -- 172.21.15.131:0/1976692042 >> 172.21.15.131:0/1976692042 conn(0x7f26dc0b1e90 msgr2=0x7f26dc0b13b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:07.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:07.613+0000 7f26e1800640 1 -- 172.21.15.131:0/1976692042 shutdown_connections 2024-05-26T22:53:07.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:07.613+0000 7f26e1800640 1 -- 172.21.15.131:0/1976692042 wait complete. 2024-05-26T22:53:08.034 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 65, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-26T22:52:53.755842+0000', 'modified': '2024-05-26T22:53:07.120270+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1], 'up': {'mds_0': 6876, 'mds_1': 6864}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6864': {'gid': 6864, 'name': 'c', 'rank': 1, 'incarnation': 65, 'state': 'up:creating', 'state_seq': 1, 'addr': '172.21.15.131:6837/1418741551', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6835', 'nonce': 1418741551}, {'type': 'v1', 'addr': '172.21.15.131:6837', 'nonce': 1418741551}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_6876': {'gid': 6876, 'name': 'd', 'rank': 0, 'incarnation': 60, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.187:6834/3680999672', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.187:6832', 'nonce': 3680999672}, {'type': 'v1', 'addr': '172.21.15.187:6834', 'nonce': 3680999672}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6876, 'qdb_cluster': [6876]} 2024-05-26T22:53:08.035 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6864:up:creating 2024-05-26T22:53:09.036 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-26T22:53:09.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:09.248+0000 7f3a51200640 1 Processor -- start 2024-05-26T22:53:09.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:09.248+0000 7f3a51200640 1 -- start start 2024-05-26T22:53:09.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:09.250+0000 7f3a51200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a4c144340 0x7f3a4c0cedf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:09.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:09.250+0000 7f3a51200640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f3a4c071b70 0x7f3a4c06e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:09.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:09.250+0000 7f3a51200640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a4c071fd0 con 0x7f3a4c06e9f0 2024-05-26T22:53:09.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:09.250+0000 7f3a51200640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f3a4c06ee60 con 0x7f3a4c071b70 2024-05-26T22:53:09.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:09.250+0000 7f3a51200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3a4c072eb0 con 0x7f3a4c144340 2024-05-26T22:53:09.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:09.250+0000 7f3a4aa00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f3a4c071b70 0x7f3a4c06e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:09.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:09.250+0000 7f3a4b400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a4c144340 0x7f3a4c0cedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:09.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:09.250+0000 7f3a4aa00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f3a4c071b70 0x7f3a4c06e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.131:51102/0 (socket says 172.21.15.131:51102) 2024-05-26T22:53:09.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:09.250+0000 7f3a4aa00640 1 -- 172.21.15.131:0/344148474 learned_addr learned my addr 172.21.15.131:0/344148474 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:53:09.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:09.251+0000 7f3a4b400640 1 -- 172.21.15.131:0/344148474 >> v2:172.21.15.187:3300/0 conn(0x7f3a4c071b70 msgr2=0x7f3a4c06e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:09.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:09.251+0000 7f3a4b400640 1 --2- 172.21.15.131:0/344148474 >> v2:172.21.15.187:3300/0 conn(0x7f3a4c071b70 0x7f3a4c06e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:09.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:09.251+0000 7f3a4b400640 1 -- 172.21.15.131:0/344148474 >> v1:172.21.15.187:6789/0 conn(0x7f3a4c06e9f0 legacy=0x7f3a4c15ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:09.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:09.251+0000 7f3a4b400640 1 -- 172.21.15.131:0/344148474 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a4c0722a0 con 0x7f3a4c144340 2024-05-26T22:53:09.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:09.251+0000 7f3a4aa00640 1 --2- 172.21.15.131:0/344148474 >> v2:172.21.15.187:3300/0 conn(0x7f3a4c071b70 0x7f3a4c06e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-26T22:53:09.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:09.252+0000 7f3a4b400640 1 --2- 172.21.15.131:0/344148474 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a4c144340 0x7f3a4c0cedf0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c0097a0 tx=0x7f3a3c02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=805f723af30d43a1 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:09.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:09.253+0000 7f3a4a000640 1 -- 172.21.15.131:0/344148474 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a3c037070 con 0x7f3a4c144340 2024-05-26T22:53:09.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:09.253+0000 7f3a4a000640 1 -- 172.21.15.131:0/344148474 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a3c032440 con 0x7f3a4c144340 2024-05-26T22:53:09.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:09.253+0000 7f3a4a000640 1 -- 172.21.15.131:0/344148474 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a3c032810 con 0x7f3a4c144340 2024-05-26T22:53:09.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:09.254+0000 7f3a51200640 1 -- 172.21.15.131:0/344148474 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a4c144340 msgr2=0x7f3a4c0cedf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:09.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:09.254+0000 7f3a51200640 1 --2- 172.21.15.131:0/344148474 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a4c144340 0x7f3a4c0cedf0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c0097a0 tx=0x7f3a3c02a180 comp rx=0 tx=0).stop 2024-05-26T22:53:09.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:09.254+0000 7f3a51200640 1 -- 172.21.15.131:0/344148474 shutdown_connections 2024-05-26T22:53:09.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:09.254+0000 7f3a51200640 1 --2- 172.21.15.131:0/344148474 >> v2:172.21.15.187:3300/0 conn(0x7f3a4c071b70 0x7f3a4c06e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:09.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:09.254+0000 7f3a51200640 1 --2- 172.21.15.131:0/344148474 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a4c144340 0x7f3a4c0cedf0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:09.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:09.254+0000 7f3a51200640 1 -- 172.21.15.131:0/344148474 >> 172.21.15.131:0/344148474 conn(0x7f3a4c0cff40 msgr2=0x7f3a4c0d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:09.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:09.254+0000 7f3a51200640 1 -- 172.21.15.131:0/344148474 shutdown_connections 2024-05-26T22:53:09.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:09.254+0000 7f3a51200640 1 -- 172.21.15.131:0/344148474 wait complete. 2024-05-26T22:53:09.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:09.256+0000 7f3a51200640 1 Processor -- start 2024-05-26T22:53:09.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:09.256+0000 7f3a51200640 1 -- start start 2024-05-26T22:53:09.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:09.257+0000 7f3a51200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a4c06e9f0 0x7f3a4c15a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:09.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:09.257+0000 7f3a51200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a4c071b70 0x7f3a4c0ad870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:09.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:09.257+0000 7f3a51200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a4c144340 0x7f3a4c0cd1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:09.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:09.257+0000 7f3a51200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f3a4c072b40 con 0x7f3a4c06e9f0 2024-05-26T22:53:09.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:09.257+0000 7f3a51200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3a4c072860 con 0x7f3a4c144340 2024-05-26T22:53:09.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:09.257+0000 7f3a51200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3a4c0b4d30 con 0x7f3a4c071b70 2024-05-26T22:53:09.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:09.257+0000 7f3a4b400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a4c06e9f0 0x7f3a4c15a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:09.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:09.258+0000 7f3a4b400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a4c06e9f0 0x7f3a4c15a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40666/0 (socket says 172.21.15.131:40666) 2024-05-26T22:53:09.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:09.258+0000 7f3a4b400640 1 -- 172.21.15.131:0/1721578382 learned_addr learned my addr 172.21.15.131:0/1721578382 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:53:09.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:09.258+0000 7f3a4aa00640 1 --2- 172.21.15.131:0/1721578382 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a4c071b70 0x7f3a4c0ad870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:09.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:09.258+0000 7f3a4be00640 1 --2- 172.21.15.131:0/1721578382 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a4c144340 0x7f3a4c0cd1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:09.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:09.258+0000 7f3a4b400640 1 -- 172.21.15.131:0/1721578382 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a4c071b70 msgr2=0x7f3a4c0ad870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:09.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:09.258+0000 7f3a4b400640 1 --2- 172.21.15.131:0/1721578382 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a4c071b70 0x7f3a4c0ad870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:09.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:09.258+0000 7f3a4b400640 1 -- 172.21.15.131:0/1721578382 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a4c144340 msgr2=0x7f3a4c0cd1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:09.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:09.258+0000 7f3a4b400640 1 --2- 172.21.15.131:0/1721578382 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a4c144340 0x7f3a4c0cd1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:09.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:09.258+0000 7f3a4b400640 1 -- 172.21.15.131:0/1721578382 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a4c13ab20 con 0x7f3a4c06e9f0 2024-05-26T22:53:09.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:09.258+0000 7f3a4aa00640 1 --2- 172.21.15.131:0/1721578382 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a4c071b70 0x7f3a4c0ad870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:53:09.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:09.258+0000 7f3a4be00640 1 --2- 172.21.15.131:0/1721578382 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a4c144340 0x7f3a4c0cd1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:53:09.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:09.258+0000 7f3a4b400640 1 --2- 172.21.15.131:0/1721578382 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a4c06e9f0 0x7f3a4c15a780 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c0098c0 tx=0x7f3a3c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:09.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:09.259+0000 7f3a43400640 1 -- 172.21.15.131:0/1721578382 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a3c037040 con 0x7f3a4c06e9f0 2024-05-26T22:53:09.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:09.259+0000 7f3a43400640 1 -- 172.21.15.131:0/1721578382 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a3c030500 con 0x7f3a4c06e9f0 2024-05-26T22:53:09.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:09.259+0000 7f3a43400640 1 -- 172.21.15.131:0/1721578382 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a3c0308d0 con 0x7f3a4c06e9f0 2024-05-26T22:53:09.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:09.259+0000 7f3a51200640 1 -- 172.21.15.131:0/1721578382 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a4c13ae00 con 0x7f3a4c06e9f0 2024-05-26T22:53:09.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:09.259+0000 7f3a51200640 1 -- 172.21.15.131:0/1721578382 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a4c13cbe0 con 0x7f3a4c06e9f0 2024-05-26T22:53:09.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:09.260+0000 7f3a51200640 1 -- 172.21.15.131:0/1721578382 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a4c0694e0 con 0x7f3a4c06e9f0 2024-05-26T22:53:09.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:09.261+0000 7f3a43400640 1 -- 172.21.15.131:0/1721578382 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3a3c044020 con 0x7f3a4c06e9f0 2024-05-26T22:53:09.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:09.262+0000 7f3a43400640 1 --2- 172.21.15.131:0/1721578382 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f3a2c06be30 0x7f3a2c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:09.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:09.262+0000 7f3a43400640 1 -- 172.21.15.131:0/1721578382 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5767+0+0 (secure 0 0 0) 0x7f3a3c0ab170 con 0x7f3a4c06e9f0 2024-05-26T22:53:09.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:09.263+0000 7f3a4aa00640 1 --2- 172.21.15.131:0/1721578382 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f3a2c06be30 0x7f3a2c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:09.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:09.263+0000 7f3a4aa00640 1 --2- 172.21.15.131:0/1721578382 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f3a2c06be30 0x7f3a2c06e2f0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f3a4c068880 tx=0x7f3a38008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:09.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:09.267+0000 7f3a43400640 1 -- 172.21.15.131:0/1721578382 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f3a3c079ad0 con 0x7f3a4c06e9f0 2024-05-26T22:53:09.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:09.504+0000 7f3a51200640 1 -- 172.21.15.131:0/1721578382 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3a4c0688d0 con 0x7f3a4c06e9f0 2024-05-26T22:53:09.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:09.505+0000 7f3a43400640 1 -- 172.21.15.131:0/1721578382 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) ==== 94+0+5233 (secure 0 0 0) 0x7f3a3c0793d0 con 0x7f3a4c06e9f0 2024-05-26T22:53:09.505 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:53:09.505 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":68,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5359,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/4204396845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6833","nonce":4204396845},{"type":"v1","addr":"172.21.15.187:6835","nonce":4204396845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-26T22:52:53.755842+0000","modified":"2024-05-26T22:53:09.135520+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6876,"mds_1":6864,"mds_2":6861},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6861":{"gid":6861,"name":"a","rank":2,"incarnation":67,"state":"up:active","state_seq":6,"addr":"172.21.15.131:6836/4223534499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":4223534499},{"type":"v1","addr":"172.21.15.131:6836","nonce":4223534499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6864":{"gid":6864,"name":"c","rank":1,"incarnation":65,"state":"up:active","state_seq":6,"addr":"172.21.15.131:6837/1418741551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6835","nonce":1418741551},{"type":"v1","addr":"172.21.15.131:6837","nonce":1418741551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6876":{"gid":6876,"name":"d","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6834/3680999672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3680999672},{"type":"v1","addr":"172.21.15.187:6834","nonce":3680999672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6876,"qdb_cluster":[6861,6864,6876]},"id":5}]} 2024-05-26T22:53:09.506 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 68 2024-05-26T22:53:09.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:09.508+0000 7f3a51200640 1 -- 172.21.15.131:0/1721578382 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f3a2c06be30 msgr2=0x7f3a2c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:09.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:09.508+0000 7f3a51200640 1 --2- 172.21.15.131:0/1721578382 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f3a2c06be30 0x7f3a2c06e2f0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f3a4c068880 tx=0x7f3a38008040 comp rx=0 tx=0).stop 2024-05-26T22:53:09.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:09.508+0000 7f3a51200640 1 -- 172.21.15.131:0/1721578382 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a4c06e9f0 msgr2=0x7f3a4c15a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:09.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:09.508+0000 7f3a51200640 1 --2- 172.21.15.131:0/1721578382 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a4c06e9f0 0x7f3a4c15a780 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c0098c0 tx=0x7f3a3c0046c0 comp rx=0 tx=0).stop 2024-05-26T22:53:09.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:09.508+0000 7f3a51200640 1 -- 172.21.15.131:0/1721578382 shutdown_connections 2024-05-26T22:53:09.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:09.508+0000 7f3a51200640 1 --2- 172.21.15.131:0/1721578382 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3a4c071b70 0x7f3a4c0ad870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:09.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:09.508+0000 7f3a51200640 1 --2- 172.21.15.131:0/1721578382 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f3a2c06be30 0x7f3a2c06e2f0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:09.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:09.508+0000 7f3a51200640 1 --2- 172.21.15.131:0/1721578382 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a4c144340 0x7f3a4c0cd1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:09.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:09.508+0000 7f3a51200640 1 --2- 172.21.15.131:0/1721578382 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a4c06e9f0 0x7f3a4c15a780 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:09.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:09.508+0000 7f3a51200640 1 -- 172.21.15.131:0/1721578382 >> 172.21.15.131:0/1721578382 conn(0x7f3a4c0cff40 msgr2=0x7f3a4c0d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:09.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:09.508+0000 7f3a51200640 1 -- 172.21.15.131:0/1721578382 shutdown_connections 2024-05-26T22:53:09.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:09.508+0000 7f3a51200640 1 -- 172.21.15.131:0/1721578382 wait complete. 2024-05-26T22:53:09.930 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 68, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-26T22:52:53.755842+0000', 'modified': '2024-05-26T22:53:09.135520+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 6876, 'mds_1': 6864, 'mds_2': 6861}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6861': {'gid': 6861, 'name': 'a', 'rank': 2, 'incarnation': 67, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.131:6836/4223534499', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6834', 'nonce': 4223534499}, {'type': 'v1', 'addr': '172.21.15.131:6836', 'nonce': 4223534499}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_6864': {'gid': 6864, 'name': 'c', 'rank': 1, 'incarnation': 65, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.131:6837/1418741551', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6835', 'nonce': 1418741551}, {'type': 'v1', 'addr': '172.21.15.131:6837', 'nonce': 1418741551}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_6876': {'gid': 6876, 'name': 'd', 'rank': 0, 'incarnation': 60, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.187:6834/3680999672', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.187:6832', 'nonce': 3680999672}, {'type': 'v1', 'addr': '172.21.15.187:6834', 'nonce': 3680999672}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6876, 'qdb_cluster': [6861, 6864, 6876]} 2024-05-26T22:53:09.930 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-05-26T22:53:09.930 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-05-26T22:53:10.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.149+0000 7f060ec00640 1 Processor -- start 2024-05-26T22:53:10.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.149+0000 7f060ec00640 1 -- start start 2024-05-26T22:53:10.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.151+0000 7f060ec00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f06080b9f20 0x7f06080d22b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:10.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.151+0000 7f060ec00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f06080d27f0 0x7f06080b8d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:10.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.151+0000 7f060ec00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06080b79f0 con 0x7f060815e340 2024-05-26T22:53:10.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.151+0000 7f060ec00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f06080b88b0 con 0x7f06080d27f0 2024-05-26T22:53:10.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.151+0000 7f060ec00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f060815e720 con 0x7f06080b9f20 2024-05-26T22:53:10.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.151+0000 7f060d800640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f060815e340 0x7f06080b72e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:53186/0 (socket says 172.21.15.131:53186) 2024-05-26T22:53:10.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.151+0000 7f060e200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f06080d27f0 0x7f06080b8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:10.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.151+0000 7f060d800640 1 -- 172.21.15.131:0/1769828224 learned_addr learned my addr 172.21.15.131:0/1769828224 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:53:10.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.151+0000 7f060ce00640 1 --2- 172.21.15.131:0/1769828224 >> v2:172.21.15.187:3300/0 conn(0x7f06080b9f20 0x7f06080d22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:10.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.152+0000 7f060e200640 1 -- 172.21.15.131:0/1769828224 >> v2:172.21.15.187:3300/0 conn(0x7f06080b9f20 msgr2=0x7f06080d22b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:10.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.152+0000 7f060e200640 1 --2- 172.21.15.131:0/1769828224 >> v2:172.21.15.187:3300/0 conn(0x7f06080b9f20 0x7f06080d22b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:10.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.152+0000 7f060e200640 1 -- 172.21.15.131:0/1769828224 >> v1:172.21.15.131:6789/0 conn(0x7f060815e340 legacy=0x7f06080b72e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:10.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.152+0000 7f060e200640 1 -- 172.21.15.131:0/1769828224 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06080b7ca0 con 0x7f06080d27f0 2024-05-26T22:53:10.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.152+0000 7f0607e00640 1 -- 172.21.15.131:0/1769828224 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2684393079 0 0) 0x7f06080b79f0 con 0x7f060815e340 2024-05-26T22:53:10.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.153+0000 7f060e200640 1 --2- 172.21.15.131:0/1769828224 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f06080d27f0 0x7f06080b8d40 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f06080d1ac0 tx=0x7f060002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5f1a4e03f3b5237 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:10.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.153+0000 7f0607e00640 1 -- 172.21.15.131:0/1769828224 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0600037070 con 0x7f06080d27f0 2024-05-26T22:53:10.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.153+0000 7f0607e00640 1 -- 172.21.15.131:0/1769828224 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0600032440 con 0x7f06080d27f0 2024-05-26T22:53:10.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.153+0000 7f0607e00640 1 -- 172.21.15.131:0/1769828224 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0600032810 con 0x7f06080d27f0 2024-05-26T22:53:10.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.154+0000 7f060ec00640 1 -- 172.21.15.131:0/1769828224 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f06080d27f0 msgr2=0x7f06080b8d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:10.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.154+0000 7f060ec00640 1 --2- 172.21.15.131:0/1769828224 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f06080d27f0 0x7f06080b8d40 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f06080d1ac0 tx=0x7f060002a230 comp rx=0 tx=0).stop 2024-05-26T22:53:10.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.154+0000 7f060ec00640 1 -- 172.21.15.131:0/1769828224 shutdown_connections 2024-05-26T22:53:10.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.154+0000 7f060ec00640 1 --2- 172.21.15.131:0/1769828224 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f06080d27f0 0x7f06080b8d40 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:10.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.154+0000 7f060ec00640 1 --2- 172.21.15.131:0/1769828224 >> v2:172.21.15.187:3300/0 conn(0x7f06080b9f20 0x7f06080d22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:10.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.154+0000 7f060ec00640 1 -- 172.21.15.131:0/1769828224 >> 172.21.15.131:0/1769828224 conn(0x7f0608072e60 msgr2=0x7f0608073270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:10.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.154+0000 7f060ec00640 1 -- 172.21.15.131:0/1769828224 shutdown_connections 2024-05-26T22:53:10.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.154+0000 7f060ec00640 1 -- 172.21.15.131:0/1769828224 wait complete. 2024-05-26T22:53:10.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.156+0000 7f060ec00640 1 Processor -- start 2024-05-26T22:53:10.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.156+0000 7f060ec00640 1 -- start start 2024-05-26T22:53:10.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.157+0000 7f060ec00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f06080b9f20 0x7f0608144750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:10.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.157+0000 7f060ec00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f06080d27f0 0x7f0608154140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:10.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.157+0000 7f060ec00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f060815e340 0x7f0608154680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:10.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.157+0000 7f060ec00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f06080b8540 con 0x7f060815e340 2024-05-26T22:53:10.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.157+0000 7f060ec00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f06080b8260 con 0x7f06080b9f20 2024-05-26T22:53:10.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.157+0000 7f060ec00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f06080b9280 con 0x7f06080d27f0 2024-05-26T22:53:10.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.157+0000 7f060e200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f060815e340 0x7f0608154680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:10.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.157+0000 7f060e200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f060815e340 0x7f0608154680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40688/0 (socket says 172.21.15.131:40688) 2024-05-26T22:53:10.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.157+0000 7f060e200640 1 -- 172.21.15.131:0/292371707 learned_addr learned my addr 172.21.15.131:0/292371707 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:53:10.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.157+0000 7f060ce00640 1 --2- 172.21.15.131:0/292371707 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f06080d27f0 0x7f0608154140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:10.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.157+0000 7f060d800640 1 --2- 172.21.15.131:0/292371707 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f06080b9f20 0x7f0608144750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:10.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.158+0000 7f060e200640 1 -- 172.21.15.131:0/292371707 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f06080d27f0 msgr2=0x7f0608154140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:10.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.158+0000 7f060e200640 1 --2- 172.21.15.131:0/292371707 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f06080d27f0 0x7f0608154140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:10.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.158+0000 7f060e200640 1 -- 172.21.15.131:0/292371707 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f06080b9f20 msgr2=0x7f0608144750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:10.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.158+0000 7f060e200640 1 --2- 172.21.15.131:0/292371707 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f06080b9f20 0x7f0608144750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:10.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.158+0000 7f060e200640 1 -- 172.21.15.131:0/292371707 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0608144c90 con 0x7f060815e340 2024-05-26T22:53:10.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.158+0000 7f060e200640 1 --2- 172.21.15.131:0/292371707 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f060815e340 0x7f0608154680 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f0600009890 tx=0x7f06000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:10.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.158+0000 7f0605600640 1 -- 172.21.15.131:0/292371707 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0600037040 con 0x7f060815e340 2024-05-26T22:53:10.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.158+0000 7f0605600640 1 -- 172.21.15.131:0/292371707 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0600042070 con 0x7f060815e340 2024-05-26T22:53:10.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.158+0000 7f060ec00640 1 -- 172.21.15.131:0/292371707 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06080c8df0 con 0x7f060815e340 2024-05-26T22:53:10.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.158+0000 7f0605600640 1 -- 172.21.15.131:0/292371707 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06000308a0 con 0x7f060815e340 2024-05-26T22:53:10.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.158+0000 7f060ec00640 1 -- 172.21.15.131:0/292371707 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06080c9370 con 0x7f060815e340 2024-05-26T22:53:10.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.160+0000 7f0605600640 1 -- 172.21.15.131:0/292371707 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0600030a40 con 0x7f060815e340 2024-05-26T22:53:10.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.161+0000 7f0605600640 1 --2- 172.21.15.131:0/292371707 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f05e806be30 0x7f05e806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:10.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.161+0000 7f0605600640 1 -- 172.21.15.131:0/292371707 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5767+0+0 (secure 0 0 0) 0x7f06000ab7d0 con 0x7f060815e340 2024-05-26T22:53:10.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.162+0000 7f060d800640 1 --2- 172.21.15.131:0/292371707 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f05e806be30 0x7f05e806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:10.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.162+0000 7f0612ff1740 1 Processor -- start 2024-05-26T22:53:10.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.162+0000 7f060d800640 1 --2- 172.21.15.131:0/292371707 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f05e806be30 0x7f05e806e2f0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f05fc007ed0 tx=0x7f05fc007e60 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:10.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.162+0000 7f0612ff1740 1 -- start start 2024-05-26T22:53:10.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.163+0000 7f0612ff1740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560306be4e50 0x560306b97a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:10.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.163+0000 7f0612ff1740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560306bc7f70 0x560306bc72f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:10.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.163+0000 7f0612ff1740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560306bc7830 0x560306b9c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:10.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.163+0000 7f0612ff1740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x560306aff610 con 0x560306be4e50 2024-05-26T22:53:10.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.163+0000 7f0612ff1740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x560306aa75c0 con 0x560306bc7830 2024-05-26T22:53:10.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.163+0000 7f0612ff1740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x560306afb5d0 con 0x560306bc7f70 2024-05-26T22:53:10.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.163+0000 7f060d800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560306be4e50 0x560306b97a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:10.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.163+0000 7f060d800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560306be4e50 0x560306b97a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40702/0 (socket says 172.21.15.131:40702) 2024-05-26T22:53:10.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.163+0000 7f060d800640 1 -- 172.21.15.131:0/2096712542 learned_addr learned my addr 172.21.15.131:0/2096712542 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:53:10.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.164+0000 7f060ce00640 1 --2- 172.21.15.131:0/2096712542 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560306bc7f70 0x560306bc72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:10.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.164+0000 7f060d800640 1 -- 172.21.15.131:0/2096712542 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560306bc7f70 msgr2=0x560306bc72f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:10.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.164+0000 7f060d800640 1 --2- 172.21.15.131:0/2096712542 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560306bc7f70 0x560306bc72f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:10.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.164+0000 7f060d800640 1 -- 172.21.15.131:0/2096712542 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560306bc7830 msgr2=0x560306b9c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:10.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.164+0000 7f060d800640 1 --2- 172.21.15.131:0/2096712542 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560306bc7830 0x560306b9c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:10.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.164+0000 7f060d800640 1 -- 172.21.15.131:0/2096712542 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603068132d0 con 0x560306be4e50 2024-05-26T22:53:10.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.164+0000 7f060d800640 1 --2- 172.21.15.131:0/2096712542 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560306be4e50 0x560306b97a70 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f05fc001d50 tx=0x7f05fc012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9e553d376323119 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:10.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.164+0000 7f05fa000640 1 -- 172.21.15.131:0/2096712542 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05fc0159c0 con 0x560306be4e50 2024-05-26T22:53:10.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.164+0000 7f05fa000640 1 -- 172.21.15.131:0/2096712542 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05fc015b60 con 0x560306be4e50 2024-05-26T22:53:10.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.164+0000 7f05fa000640 1 -- 172.21.15.131:0/2096712542 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05fc021450 con 0x560306be4e50 2024-05-26T22:53:10.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.165+0000 7f0612ff1740 1 -- 172.21.15.131:0/2096712542 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560306be4e50 msgr2=0x560306b97a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:10.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.165+0000 7f0612ff1740 1 --2- 172.21.15.131:0/2096712542 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560306be4e50 0x560306b97a70 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f05fc001d50 tx=0x7f05fc012000 comp rx=0 tx=0).stop 2024-05-26T22:53:10.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.165+0000 7f0612ff1740 1 -- 172.21.15.131:0/2096712542 shutdown_connections 2024-05-26T22:53:10.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.165+0000 7f0612ff1740 1 --2- 172.21.15.131:0/2096712542 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560306bc7f70 0x560306bc72f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:10.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.165+0000 7f0612ff1740 1 --2- 172.21.15.131:0/2096712542 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560306bc7830 0x560306b9c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:10.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.165+0000 7f0612ff1740 1 --2- 172.21.15.131:0/2096712542 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560306be4e50 0x560306b97a70 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:10.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.165+0000 7f0612ff1740 1 -- 172.21.15.131:0/2096712542 >> 172.21.15.131:0/2096712542 conn(0x560306b99ce0 msgr2=0x560306be22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:10.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.165+0000 7f0612ff1740 1 -- 172.21.15.131:0/2096712542 shutdown_connections 2024-05-26T22:53:10.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.165+0000 7f0612ff1740 1 -- 172.21.15.131:0/2096712542 wait complete. 2024-05-26T22:53:10.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.166+0000 7f0612ff1740 1 -- start start 2024-05-26T22:53:10.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.166+0000 7f0612ff1740 1 Processor -- start 2024-05-26T22:53:10.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.167+0000 7f0612ff1740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560306bc7f70 0x560306bb54b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:10.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.167+0000 7f0612ff1740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560306be4e50 0x560306bd8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:10.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.167+0000 7f0612ff1740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560306bb80d0 0x560306be3420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:10.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.167+0000 7f0612ff1740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x560306b9bc00 con 0x560306bc7f70 2024-05-26T22:53:10.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.167+0000 7f0612ff1740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x560306aba910 con 0x560306bb80d0 2024-05-26T22:53:10.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.167+0000 7f0612ff1740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x560306b9aff0 con 0x560306be4e50 2024-05-26T22:53:10.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.167+0000 7f0612ff1740 10 client.0 _ll_register_callbacks cb 0x560306af6410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-26T22:53:10.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.167+0000 7f060d800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560306bc7f70 0x560306bb54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:10.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.167+0000 7f060d800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560306bc7f70 0x560306bb54b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40706/0 (socket says 172.21.15.131:40706) 2024-05-26T22:53:10.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.167+0000 7f060d800640 1 -- 172.21.15.131:0/3348012663 learned_addr learned my addr 172.21.15.131:0/3348012663 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:53:10.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.168+0000 7f060ce00640 1 --2- 172.21.15.131:0/3348012663 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560306be4e50 0x560306bd8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:10.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.168+0000 7f060e200640 1 --2- 172.21.15.131:0/3348012663 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560306bb80d0 0x560306be3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:10.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.168+0000 7f060d800640 1 -- 172.21.15.131:0/3348012663 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560306be4e50 msgr2=0x560306bd8fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:10.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.168+0000 7f060d800640 1 --2- 172.21.15.131:0/3348012663 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560306be4e50 0x560306bd8fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:10.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.168+0000 7f060d800640 1 -- 172.21.15.131:0/3348012663 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560306bb80d0 msgr2=0x560306be3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:10.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.168+0000 7f060d800640 1 --2- 172.21.15.131:0/3348012663 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560306bb80d0 0x560306be3420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:10.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.168+0000 7f060d800640 1 -- 172.21.15.131:0/3348012663 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603068dd6b0 con 0x560306bc7f70 2024-05-26T22:53:10.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.168+0000 7f060d800640 1 --2- 172.21.15.131:0/3348012663 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560306bc7f70 0x560306bb54b0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f05fc015810 tx=0x7f05fc001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2cf46d1c12ab34f server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:10.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.168+0000 7f05eea00640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-26T22:53:10.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.168+0000 7f05eea00640 1 -- 172.21.15.131:0/3348012663 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05fc021660 con 0x560306bc7f70 2024-05-26T22:53:10.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.169+0000 7f05eea00640 1 -- 172.21.15.131:0/3348012663 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(71..71 src has 1..71) ==== 5767+0+0 (secure 0 0 0) 0x7f05fc010070 con 0x560306bc7f70 2024-05-26T22:53:10.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.169+0000 7f0612ff1740 1 -- 172.21.15.131:0/3348012663 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05fc015b60 con 0x560306bc7f70 2024-05-26T22:53:10.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.169+0000 7f05eea00640 1 -- 172.21.15.131:0/3348012663 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7f05fc034050 con 0x560306bc7f70 2024-05-26T22:53:10.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.169+0000 7f0612ff1740 10 client.7095 fetch_fsmap learned FSMap version 68 2024-05-26T22:53:10.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.169+0000 7f0612ff1740 1 -- 172.21.15.131:0/3348012663 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) -- 0x560306bda130 con 0x560306bc7f70 2024-05-26T22:53:10.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.169+0000 7f05eea00640 1 -- 172.21.15.131:0/3348012663 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05fc015b60 con 0x560306bc7f70 2024-05-26T22:53:10.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.169+0000 7f05eea00640 1 -- 172.21.15.131:0/3348012663 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 68) ==== 3534+0+0 (secure 0 0 0) 0x7f05fc037020 con 0x560306bc7f70 2024-05-26T22:53:10.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.169+0000 7f0612ff1740 10 client.7095 fetch_fsmap finished waiting for FSMap version 68 2024-05-26T22:53:10.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.169+0000 7f0612ff1740 10 client.7095 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-26T22:53:10.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.169+0000 7f0612ff1740 20 client.7095 populate_metadata read hostname 'smithi131' 2024-05-26T22:53:10.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.169+0000 7f0612ff1740 1 --2- 172.21.15.131:0/3348012663 >> [v2:172.21.15.131:6834/4223534499,v1:172.21.15.131:6836/4223534499] conn(0x560306bdaba0 0x560306bc8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:10.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.169+0000 7f0612ff1740 4 client.7095 mds_command: new command op to 6861 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-26T22:53:10.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.169+0000 7f0612ff1740 1 -- 172.21.15.131:0/3348012663 --> [v2:172.21.15.131:6834/4223534499,v1:172.21.15.131:6836/4223534499] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560306b0cd40 con 0x560306bdaba0 2024-05-26T22:53:10.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.170+0000 7f060ce00640 1 --2- 172.21.15.131:0/3348012663 >> [v2:172.21.15.131:6834/4223534499,v1:172.21.15.131:6836/4223534499] conn(0x560306bdaba0 0x560306bc8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:10.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.170+0000 7f060ce00640 1 --2- 172.21.15.131:0/3348012663 >> [v2:172.21.15.131:6834/4223534499,v1:172.21.15.131:6836/4223534499] conn(0x560306bdaba0 0x560306bc8bb0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6ae6335c32d4a27d server_cookie=3a6f529ddb757ae9 in_seq=0 out_seq=0 2024-05-26T22:53:10.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.170+0000 7f05eea00640 10 client.7095 ms_handle_connect on v2:172.21.15.131:6834/4223534499 2024-05-26T22:53:10.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.171+0000 7f05eea00640 1 -- 172.21.15.131:0/3348012663 <== mds.2 v2:172.21.15.131:6834/4223534499 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560306b0cd40 con 0x560306bdaba0 2024-05-26T22:53:10.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.171+0000 7f05eea00640 10 client.7095 handle_command_reply: tid=0 2024-05-26T22:53:10.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.171+0000 7f0612ff1740 1 client.7095 shutdown 2024-05-26T22:53:10.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.171+0000 7f0612ff1740 1 -- 172.21.15.131:0/3348012663 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560306bc7f70 msgr2=0x560306bb54b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:10.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.171+0000 7f0612ff1740 1 --2- 172.21.15.131:0/3348012663 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560306bc7f70 0x560306bb54b0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f05fc015810 tx=0x7f05fc001250 comp rx=0 tx=0).stop 2024-05-26T22:53:10.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.171+0000 7f0612ff1740 1 -- 172.21.15.131:0/3348012663 shutdown_connections 2024-05-26T22:53:10.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.171+0000 7f0612ff1740 1 --2- 172.21.15.131:0/3348012663 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560306be4e50 0x560306bd8fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:10.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.171+0000 7f0612ff1740 1 --2- 172.21.15.131:0/3348012663 >> [v2:172.21.15.131:6834/4223534499,v1:172.21.15.131:6836/4223534499] conn(0x560306bdaba0 0x560306bc8bb0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:10.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.171+0000 7f0612ff1740 1 --2- 172.21.15.131:0/3348012663 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560306bb80d0 0x560306be3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:10.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.171+0000 7f0612ff1740 1 --2- 172.21.15.131:0/3348012663 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560306bc7f70 0x560306bb54b0 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:10.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.171+0000 7f05eea00640 0 client.7095 ms_handle_reset on v2:172.21.15.131:6834/4223534499 2024-05-26T22:53:10.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.171+0000 7f0612ff1740 1 -- 172.21.15.131:0/3348012663 >> 172.21.15.131:0/3348012663 conn(0x560306bc7830 msgr2=0x560306bc4af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:10.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.172+0000 7f0612ff1740 1 -- 172.21.15.131:0/3348012663 shutdown_connections 2024-05-26T22:53:10.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.172+0000 7f0612ff1740 1 -- 172.21.15.131:0/3348012663 wait complete. 2024-05-26T22:53:10.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.172+0000 7f0612ff1740 20 client.7095 trim_cache size 0 max 16384 2024-05-26T22:53:10.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.192+0000 7f0612ff1740 1 Processor -- start 2024-05-26T22:53:10.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.192+0000 7f0612ff1740 1 -- start start 2024-05-26T22:53:10.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.193+0000 7f0612ff1740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560306bb80d0 0x560306b97700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:10.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.193+0000 7f0612ff1740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560306bc7f70 0x560306b9d000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:10.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.193+0000 7f0612ff1740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560306be4e50 0x560306b9d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:10.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.193+0000 7f0612ff1740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f06081ff9d0 con 0x560306bb80d0 2024-05-26T22:53:10.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.193+0000 7f0612ff1740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f06082000a0 con 0x560306be4e50 2024-05-26T22:53:10.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.193+0000 7f0612ff1740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f06081f81a0 con 0x560306bc7f70 2024-05-26T22:53:10.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.193+0000 7f060d800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560306bb80d0 0x560306b97700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:10.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.193+0000 7f060d800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560306bb80d0 0x560306b97700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40708/0 (socket says 172.21.15.131:40708) 2024-05-26T22:53:10.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.193+0000 7f060d800640 1 -- 172.21.15.131:0/3495046084 learned_addr learned my addr 172.21.15.131:0/3495046084 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:53:10.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.193+0000 7f060ce00640 1 --2- 172.21.15.131:0/3495046084 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560306bc7f70 0x560306b9d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:10.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.193+0000 7f060d800640 1 -- 172.21.15.131:0/3495046084 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560306bc7f70 msgr2=0x560306b9d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:10.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.193+0000 7f060d800640 1 --2- 172.21.15.131:0/3495046084 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560306bc7f70 0x560306b9d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:10.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.193+0000 7f060d800640 1 -- 172.21.15.131:0/3495046084 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560306be4e50 msgr2=0x560306b9d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:10.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.193+0000 7f060e200640 1 --2- 172.21.15.131:0/3495046084 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560306be4e50 0x560306b9d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:10.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.193+0000 7f060d800640 1 --2- 172.21.15.131:0/3495046084 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560306be4e50 0x560306b9d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:10.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.193+0000 7f060d800640 1 -- 172.21.15.131:0/3495046084 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560306bc9f50 con 0x560306bb80d0 2024-05-26T22:53:10.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.193+0000 7f060ce00640 1 --2- 172.21.15.131:0/3495046084 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560306bc7f70 0x560306b9d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:53:10.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.193+0000 7f060e200640 1 --2- 172.21.15.131:0/3495046084 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560306be4e50 0x560306b9d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:53:10.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.194+0000 7f060d800640 1 --2- 172.21.15.131:0/3495046084 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560306bb80d0 0x560306b97700 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f05fc005fe0 tx=0x7f05fc027000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2df706b5b1d78369 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:10.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.194+0000 7f05f9600640 1 -- 172.21.15.131:0/3495046084 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05fc005c30 con 0x560306bb80d0 2024-05-26T22:53:10.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.194+0000 7f05f9600640 1 -- 172.21.15.131:0/3495046084 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05fc021660 con 0x560306bb80d0 2024-05-26T22:53:10.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.194+0000 7f05f9600640 1 -- 172.21.15.131:0/3495046084 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05fc015d10 con 0x560306bb80d0 2024-05-26T22:53:10.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.195+0000 7f0612ff1740 1 -- 172.21.15.131:0/3495046084 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560306bb80d0 msgr2=0x560306b97700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:10.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.195+0000 7f0612ff1740 1 --2- 172.21.15.131:0/3495046084 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560306bb80d0 0x560306b97700 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f05fc005fe0 tx=0x7f05fc027000 comp rx=0 tx=0).stop 2024-05-26T22:53:10.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.195+0000 7f0612ff1740 1 -- 172.21.15.131:0/3495046084 shutdown_connections 2024-05-26T22:53:10.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.195+0000 7f0612ff1740 1 --2- 172.21.15.131:0/3495046084 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560306bc7f70 0x560306b9d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:10.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.195+0000 7f0612ff1740 1 --2- 172.21.15.131:0/3495046084 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560306be4e50 0x560306b9d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:10.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.195+0000 7f0612ff1740 1 --2- 172.21.15.131:0/3495046084 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560306bb80d0 0x560306b97700 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:10.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.195+0000 7f0612ff1740 1 -- 172.21.15.131:0/3495046084 >> 172.21.15.131:0/3495046084 conn(0x560306bc7830 msgr2=0x560306bc8bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:10.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.195+0000 7f0612ff1740 1 -- 172.21.15.131:0/3495046084 shutdown_connections 2024-05-26T22:53:10.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.195+0000 7f0612ff1740 1 -- 172.21.15.131:0/3495046084 wait complete. 2024-05-26T22:53:10.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.195+0000 7f0612ff1740 1 -- start start 2024-05-26T22:53:10.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.196+0000 7f0612ff1740 1 Processor -- start 2024-05-26T22:53:10.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.197+0000 7f0612ff1740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560306bb80d0 0x560306b9d000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:10.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.197+0000 7f0612ff1740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560306bc7f70 0x560306b9d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:10.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.197+0000 7f0612ff1740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560306be4e50 0x560306be3e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:10.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.197+0000 7f0612ff1740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x560306bdaa20 con 0x560306bc7f70 2024-05-26T22:53:10.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.197+0000 7f0612ff1740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x560306bb5a80 con 0x560306be4e50 2024-05-26T22:53:10.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.197+0000 7f0612ff1740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x560306bdafb0 con 0x560306bb80d0 2024-05-26T22:53:10.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.197+0000 7f0612ff1740 10 client.0 _ll_register_callbacks cb 0x7f05fc001bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-26T22:53:10.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.197+0000 7f060ce00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560306bc7f70 0x560306b9d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:10.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.197+0000 7f060ce00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560306bc7f70 0x560306b9d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40710/0 (socket says 172.21.15.131:40710) 2024-05-26T22:53:10.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.197+0000 7f060ce00640 1 -- 172.21.15.131:0/2358091475 learned_addr learned my addr 172.21.15.131:0/2358091475 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:53:10.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.197+0000 7f060d800640 1 --2- 172.21.15.131:0/2358091475 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560306bb80d0 0x560306b9d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:10.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.197+0000 7f060ce00640 1 -- 172.21.15.131:0/2358091475 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560306bb80d0 msgr2=0x560306b9d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:10.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.197+0000 7f060e200640 1 --2- 172.21.15.131:0/2358091475 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560306be4e50 0x560306be3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:10.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.197+0000 7f060ce00640 1 --2- 172.21.15.131:0/2358091475 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560306bb80d0 0x560306b9d000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:10.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.197+0000 7f060ce00640 1 -- 172.21.15.131:0/2358091475 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560306be4e50 msgr2=0x560306be3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:10.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.197+0000 7f060ce00640 1 --2- 172.21.15.131:0/2358091475 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560306be4e50 0x560306be3e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:10.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.197+0000 7f060ce00640 1 -- 172.21.15.131:0/2358091475 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560306c60b90 con 0x560306bc7f70 2024-05-26T22:53:10.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.198+0000 7f060ce00640 1 --2- 172.21.15.131:0/2358091475 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560306bc7f70 0x560306b9d540 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x560306bd9aa0 tx=0x7f05f4009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a34fbcf202bf28a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:10.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.198+0000 7f05eea00640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-26T22:53:10.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.198+0000 7f05eea00640 1 -- 172.21.15.131:0/2358091475 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f4023070 con 0x560306bc7f70 2024-05-26T22:53:10.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.198+0000 7f05eea00640 1 -- 172.21.15.131:0/2358091475 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(71..71 src has 1..71) ==== 5767+0+0 (secure 0 0 0) 0x7f05f401f230 con 0x560306bc7f70 2024-05-26T22:53:10.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.198+0000 7f0612ff1740 1 -- 172.21.15.131:0/2358091475 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05fc021660 con 0x560306bc7f70 2024-05-26T22:53:10.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.198+0000 7f05eea00640 1 -- 172.21.15.131:0/2358091475 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7f05f402e050 con 0x560306bc7f70 2024-05-26T22:53:10.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.198+0000 7f0612ff1740 10 client.7101 fetch_fsmap learned FSMap version 68 2024-05-26T22:53:10.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.198+0000 7f0612ff1740 1 -- 172.21.15.131:0/2358091475 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) -- 0x560306bda300 con 0x560306bc7f70 2024-05-26T22:53:10.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.198+0000 7f05eea00640 1 -- 172.21.15.131:0/2358091475 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05fc021660 con 0x560306bc7f70 2024-05-26T22:53:10.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.198+0000 7f05eea00640 1 -- 172.21.15.131:0/2358091475 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 68) ==== 3534+0+0 (secure 0 0 0) 0x7f05f4031020 con 0x560306bc7f70 2024-05-26T22:53:10.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.198+0000 7f0612ff1740 10 client.7101 fetch_fsmap finished waiting for FSMap version 68 2024-05-26T22:53:10.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.198+0000 7f0612ff1740 10 client.7101 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-26T22:53:10.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.198+0000 7f0612ff1740 20 client.7101 populate_metadata read hostname 'smithi131' 2024-05-26T22:53:10.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.199+0000 7f0612ff1740 1 --2- 172.21.15.131:0/2358091475 >> [v2:172.21.15.131:6834/4223534499,v1:172.21.15.131:6836/4223534499] conn(0x560306be07b0 0x560306be0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:10.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.199+0000 7f0612ff1740 4 client.7101 mds_command: new command op to 6861 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-26T22:53:10.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.199+0000 7f0612ff1740 1 -- 172.21.15.131:0/2358091475 --> [v2:172.21.15.131:6834/4223534499,v1:172.21.15.131:6836/4223534499] -- command(tid 0: {"prefix": "status"}) -- 0x560306b07090 con 0x560306be07b0 2024-05-26T22:53:10.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.199+0000 7f060d800640 1 --2- 172.21.15.131:0/2358091475 >> [v2:172.21.15.131:6834/4223534499,v1:172.21.15.131:6836/4223534499] conn(0x560306be07b0 0x560306be0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:10.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.199+0000 7f05eea00640 10 client.7101 ms_handle_connect on v2:172.21.15.131:6834/4223534499 2024-05-26T22:53:10.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.199+0000 7f060d800640 1 --2- 172.21.15.131:0/2358091475 >> [v2:172.21.15.131:6834/4223534499,v1:172.21.15.131:6836/4223534499] conn(0x560306be07b0 0x560306be0b90 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e1808f85f8a1171 server_cookie=2fb034cdf3f561c3 in_seq=0 out_seq=0 2024-05-26T22:53:10.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.200+0000 7f05eea00640 1 -- 172.21.15.131:0/2358091475 <== mds.2 v2:172.21.15.131:6834/4223534499 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x560306b07090 con 0x560306be07b0 2024-05-26T22:53:10.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.200+0000 7f05eea00640 10 client.7101 handle_command_reply: tid=0 2024-05-26T22:53:10.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.200+0000 7f0612ff1740 1 client.7101 shutdown 2024-05-26T22:53:10.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.200+0000 7f0612ff1740 1 -- 172.21.15.131:0/2358091475 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560306bc7f70 msgr2=0x560306b9d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:10.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.200+0000 7f0612ff1740 1 --2- 172.21.15.131:0/2358091475 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560306bc7f70 0x560306b9d540 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x560306bd9aa0 tx=0x7f05f4009190 comp rx=0 tx=0).stop 2024-05-26T22:53:10.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.200+0000 7f0612ff1740 1 -- 172.21.15.131:0/2358091475 shutdown_connections 2024-05-26T22:53:10.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.200+0000 7f0612ff1740 1 --2- 172.21.15.131:0/2358091475 >> [v2:172.21.15.131:6834/4223534499,v1:172.21.15.131:6836/4223534499] conn(0x560306be07b0 0x560306be0b90 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:10.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.200+0000 7f0612ff1740 1 --2- 172.21.15.131:0/2358091475 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560306be4e50 0x560306be3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:10.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.200+0000 7f0612ff1740 1 --2- 172.21.15.131:0/2358091475 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560306bc7f70 0x560306b9d540 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:10.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.200+0000 7f05eea00640 0 client.7101 ms_handle_reset on v2:172.21.15.131:6834/4223534499 2024-05-26T22:53:10.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.200+0000 7f0612ff1740 1 --2- 172.21.15.131:0/2358091475 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x560306bb80d0 0x560306b9d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:10.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.200+0000 7f0612ff1740 1 -- 172.21.15.131:0/2358091475 >> 172.21.15.131:0/2358091475 conn(0x560306bc7830 msgr2=0x560306bb5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:10.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.200+0000 7f0612ff1740 1 -- 172.21.15.131:0/2358091475 shutdown_connections 2024-05-26T22:53:10.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.200+0000 7f0612ff1740 1 -- 172.21.15.131:0/2358091475 wait complete. 2024-05-26T22:53:10.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.200+0000 7f0612ff1740 20 client.7101 trim_cache size 0 max 16384 2024-05-26T22:53:10.209 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-26T22:53:10.209 INFO:teuthology.orchestra.run.smithi131.stdout: "cluster_fsid": "46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f", 2024-05-26T22:53:10.209 INFO:teuthology.orchestra.run.smithi131.stdout: "whoami": 2, 2024-05-26T22:53:10.209 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 6861, 2024-05-26T22:53:10.209 INFO:teuthology.orchestra.run.smithi131.stdout: "want_state": "up:active", 2024-05-26T22:53:10.209 INFO:teuthology.orchestra.run.smithi131.stdout: "state": "up:active", 2024-05-26T22:53:10.209 INFO:teuthology.orchestra.run.smithi131.stdout: "fs_name": "cephfs", 2024-05-26T22:53:10.209 INFO:teuthology.orchestra.run.smithi131.stdout: "rank_uptime": 2.0640857430000001, 2024-05-26T22:53:10.209 INFO:teuthology.orchestra.run.smithi131.stdout: "mdsmap_epoch": 68, 2024-05-26T22:53:10.209 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch": 71, 2024-05-26T22:53:10.209 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch_barrier": 71, 2024-05-26T22:53:10.210 INFO:teuthology.orchestra.run.smithi131.stdout: "uptime": 21.345660946999999 2024-05-26T22:53:10.210 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-26T22:53:10.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.201+0000 7f060ec00640 1 -- 172.21.15.131:0/292371707 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f05e806be30 msgr2=0x7f05e806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:10.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.201+0000 7f060ec00640 1 --2- 172.21.15.131:0/292371707 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f05e806be30 0x7f05e806e2f0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f05fc007ed0 tx=0x7f05fc007e60 comp rx=0 tx=0).stop 2024-05-26T22:53:10.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.201+0000 7f060ec00640 1 -- 172.21.15.131:0/292371707 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f060815e340 msgr2=0x7f0608154680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:10.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.201+0000 7f060ec00640 1 --2- 172.21.15.131:0/292371707 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f060815e340 0x7f0608154680 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f0600009890 tx=0x7f06000046c0 comp rx=0 tx=0).stop 2024-05-26T22:53:10.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.201+0000 7f060ec00640 1 -- 172.21.15.131:0/292371707 shutdown_connections 2024-05-26T22:53:10.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.201+0000 7f060ec00640 1 --2- 172.21.15.131:0/292371707 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f06080d27f0 0x7f0608154140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:10.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.201+0000 7f060ec00640 1 --2- 172.21.15.131:0/292371707 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f05e806be30 0x7f05e806e2f0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:10.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.201+0000 7f060ec00640 1 --2- 172.21.15.131:0/292371707 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f060815e340 0x7f0608154680 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:10.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.201+0000 7f060ec00640 1 --2- 172.21.15.131:0/292371707 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f06080b9f20 0x7f0608144750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:10.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.201+0000 7f060ec00640 1 -- 172.21.15.131:0/292371707 >> 172.21.15.131:0/292371707 conn(0x7f0608072e60 msgr2=0x7f06080ceb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:10.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.201+0000 7f060ec00640 1 -- 172.21.15.131:0/292371707 shutdown_connections 2024-05-26T22:53:10.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.201+0000 7f060ec00640 1 -- 172.21.15.131:0/292371707 wait complete. 2024-05-26T22:53:10.217 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-05-26T22:53:10.252 INFO:tasks.ceph.mds.d.smithi187.stderr:2024-05-26T22:53:10.252+0000 7f64e0a00640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-05-26T22:53:10.252 INFO:tasks.ceph.mds.d.smithi187.stderr:2024-05-26T22:53:10.252+0000 7f64e0a00640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-05-26T22:53:10.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.407+0000 7f20f2800640 1 Processor -- start 2024-05-26T22:53:10.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.407+0000 7f20f2800640 1 -- start start 2024-05-26T22:53:10.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.409+0000 7f20f2800640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f20ec0712d0 0x7f20ec06d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:10.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.409+0000 7f20f2800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f20ec06d790 0x7f20ec0cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:10.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.409+0000 7f20f2800640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20ec072d60 con 0x7f20ec145a10 2024-05-26T22:53:10.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.409+0000 7f20f2800640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f20ec06dc00 con 0x7f20ec0712d0 2024-05-26T22:53:10.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.409+0000 7f20f2800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f20ec15c760 con 0x7f20ec06d790 2024-05-26T22:53:10.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.409+0000 7f20f1400640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f20ec145a10 0x7f20ec072180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:53200/0 (socket says 172.21.15.131:53200) 2024-05-26T22:53:10.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.409+0000 7f20f1400640 1 -- 172.21.15.131:0/4142990759 learned_addr learned my addr 172.21.15.131:0/4142990759 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:53:10.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.410+0000 7f20f1e00640 1 --2- 172.21.15.131:0/4142990759 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f20ec06d790 0x7f20ec0cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:10.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.410+0000 7f20f0a00640 1 --2- 172.21.15.131:0/4142990759 >> v2:172.21.15.187:3300/0 conn(0x7f20ec0712d0 0x7f20ec06d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:10.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.410+0000 7f20ebe00640 1 -- 172.21.15.131:0/4142990759 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 750798230 0 0) 0x7f20ec072d60 con 0x7f20ec145a10 2024-05-26T22:53:10.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.410+0000 7f20ebe00640 1 -- 172.21.15.131:0/4142990759 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f20d4003520 con 0x7f20ec145a10 2024-05-26T22:53:10.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.410+0000 7f20ebe00640 1 -- 172.21.15.131:0/4142990759 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4100400108 0 0) 0x7f20d4003520 con 0x7f20ec145a10 2024-05-26T22:53:10.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.410+0000 7f20ebe00640 1 -- 172.21.15.131:0/4142990759 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f20ec06d790 msgr2=0x7f20ec0cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:10.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.410+0000 7f20ebe00640 1 --2- 172.21.15.131:0/4142990759 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f20ec06d790 0x7f20ec0cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:10.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.410+0000 7f20ebe00640 1 -- 172.21.15.131:0/4142990759 >> v2:172.21.15.187:3300/0 conn(0x7f20ec0712d0 msgr2=0x7f20ec06d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:10.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.410+0000 7f20ebe00640 1 --2- 172.21.15.131:0/4142990759 >> v2:172.21.15.187:3300/0 conn(0x7f20ec0712d0 0x7f20ec06d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:10.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.410+0000 7f20ebe00640 1 -- 172.21.15.131:0/4142990759 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20ec072a80 con 0x7f20ec145a10 2024-05-26T22:53:10.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.410+0000 7f20ebe00640 1 -- 172.21.15.131:0/4142990759 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7f20e0001670 con 0x7f20ec145a10 2024-05-26T22:53:10.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.410+0000 7f20ebe00640 1 -- 172.21.15.131:0/4142990759 >> v1:172.21.15.131:6789/0 conn(0x7f20ec145a10 legacy=0x7f20ec072180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:10.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.410+0000 7f20ebe00640 1 --2- 172.21.15.131:0/4142990759 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f20d40039b0 0x7f20d4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:10.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.410+0000 7f20ebe00640 1 --2- 172.21.15.131:0/4142990759 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f20ec06d790 0x7f20d4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:10.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.410+0000 7f20ebe00640 1 --2- 172.21.15.131:0/4142990759 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f20d4004d40 0x7f20d40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:10.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.410+0000 7f20ebe00640 1 -- 172.21.15.131:0/4142990759 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f20ec06dc00 con 0x7f20d4004d40 2024-05-26T22:53:10.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.410+0000 7f20ebe00640 1 -- 172.21.15.131:0/4142990759 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f20ec15c760 con 0x7f20d40039b0 2024-05-26T22:53:10.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.410+0000 7f20ebe00640 1 -- 172.21.15.131:0/4142990759 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f20d400b9c0 con 0x7f20ec06d790 2024-05-26T22:53:10.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.411+0000 7f20f1e00640 1 --2- 172.21.15.131:0/4142990759 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f20d4004d40 0x7f20d40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:10.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.411+0000 7f20f1400640 1 --2- 172.21.15.131:0/4142990759 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f20d40039b0 0x7f20d4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:10.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.411+0000 7f20f0a00640 1 --2- 172.21.15.131:0/4142990759 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f20ec06d790 0x7f20d4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:10.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.411+0000 7f20f1e00640 1 -- 172.21.15.131:0/4142990759 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f20ec06d790 msgr2=0x7f20d4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:10.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.411+0000 7f20f1e00640 1 --2- 172.21.15.131:0/4142990759 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f20ec06d790 0x7f20d4004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:10.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.411+0000 7f20f1e00640 1 -- 172.21.15.131:0/4142990759 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f20d40039b0 msgr2=0x7f20d4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:10.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.411+0000 7f20f1e00640 1 --2- 172.21.15.131:0/4142990759 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f20d40039b0 0x7f20d4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:10.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.411+0000 7f20f1e00640 1 -- 172.21.15.131:0/4142990759 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20d4003520 con 0x7f20d4004d40 2024-05-26T22:53:10.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.411+0000 7f20f1400640 1 --2- 172.21.15.131:0/4142990759 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f20d40039b0 0x7f20d4003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:53:10.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.412+0000 7f20f1e00640 1 --2- 172.21.15.131:0/4142990759 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f20d4004d40 0x7f20d40051c0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f20e4009fc0 tx=0x7f20d4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e44e4aad0edbd954 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:10.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.412+0000 7f20ebe00640 1 -- 172.21.15.131:0/4142990759 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e4009c10 con 0x7f20d4004d40 2024-05-26T22:53:10.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.412+0000 7f20ebe00640 1 -- 172.21.15.131:0/4142990759 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20e4036070 con 0x7f20d4004d40 2024-05-26T22:53:10.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.412+0000 7f20ebe00640 1 -- 172.21.15.131:0/4142990759 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e4031880 con 0x7f20d4004d40 2024-05-26T22:53:10.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.413+0000 7f20f2800640 1 -- 172.21.15.131:0/4142990759 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f20d4004d40 msgr2=0x7f20d40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:10.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.413+0000 7f20f2800640 1 --2- 172.21.15.131:0/4142990759 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f20d4004d40 0x7f20d40051c0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f20e4009fc0 tx=0x7f20d4005830 comp rx=0 tx=0).stop 2024-05-26T22:53:10.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.413+0000 7f20f1e00640 1 -- 172.21.15.131:0/4142990759 reap_dead start 2024-05-26T22:53:10.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.413+0000 7f20f2800640 1 -- 172.21.15.131:0/4142990759 shutdown_connections 2024-05-26T22:53:10.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.413+0000 7f20f2800640 1 -- 172.21.15.131:0/4142990759 >> 172.21.15.131:0/4142990759 conn(0x7f20ec0b5660 msgr2=0x7f20ec0b5a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:10.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.413+0000 7f20f2800640 1 -- 172.21.15.131:0/4142990759 shutdown_connections 2024-05-26T22:53:10.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.413+0000 7f20f2800640 1 -- 172.21.15.131:0/4142990759 wait complete. 2024-05-26T22:53:10.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.415+0000 7f20f2800640 1 Processor -- start 2024-05-26T22:53:10.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.415+0000 7f20f2800640 1 -- start start 2024-05-26T22:53:10.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.416+0000 7f20f2800640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f20d4004d40 0x7f20ec144c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:10.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.416+0000 7f20f2800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f20ec13fa80 0x7f20ec1599c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:10.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.416+0000 7f20f2800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f20ec0cf490 0x7f20ec0cf920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:10.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.416+0000 7f20f2800640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f20ec0cf070 con 0x7f20ec13fa80 2024-05-26T22:53:10.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.416+0000 7f20f2800640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f20ec0ced90 con 0x7f20d4004d40 2024-05-26T22:53:10.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.416+0000 7f20f2800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f20ec145df0 con 0x7f20ec0cf490 2024-05-26T22:53:10.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.416+0000 7f20f0a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f20ec13fa80 0x7f20ec1599c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:10.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.416+0000 7f20f0a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f20ec13fa80 0x7f20ec1599c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40732/0 (socket says 172.21.15.131:40732) 2024-05-26T22:53:10.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.416+0000 7f20f0a00640 1 -- 172.21.15.131:0/82515542 learned_addr learned my addr 172.21.15.131:0/82515542 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:53:10.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.416+0000 7f20f1400640 1 --2- 172.21.15.131:0/82515542 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f20d4004d40 0x7f20ec144c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:10.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.417+0000 7f20f0a00640 1 -- 172.21.15.131:0/82515542 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f20ec0cf490 msgr2=0x7f20ec0cf920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:10.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.417+0000 7f20f0a00640 1 --2- 172.21.15.131:0/82515542 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f20ec0cf490 0x7f20ec0cf920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:10.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.417+0000 7f20f0a00640 1 -- 172.21.15.131:0/82515542 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f20d4004d40 msgr2=0x7f20ec144c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:10.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.417+0000 7f20f0a00640 1 --2- 172.21.15.131:0/82515542 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f20d4004d40 0x7f20ec144c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:10.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.417+0000 7f20f0a00640 1 -- 172.21.15.131:0/82515542 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20ec13ffb0 con 0x7f20ec13fa80 2024-05-26T22:53:10.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.417+0000 7f20f0a00640 1 --2- 172.21.15.131:0/82515542 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f20ec13fa80 0x7f20ec1599c0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f20d80108c0 tx=0x7f20d8010d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:10.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.417+0000 7f20e9600640 1 -- 172.21.15.131:0/82515542 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d800f040 con 0x7f20ec13fa80 2024-05-26T22:53:10.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.417+0000 7f20e9600640 1 -- 172.21.15.131:0/82515542 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20d800eae0 con 0x7f20ec13fa80 2024-05-26T22:53:10.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.417+0000 7f20f2800640 1 -- 172.21.15.131:0/82515542 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20ec140430 con 0x7f20ec13fa80 2024-05-26T22:53:10.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.417+0000 7f20e9600640 1 -- 172.21.15.131:0/82515542 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d801c400 con 0x7f20ec13fa80 2024-05-26T22:53:10.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.417+0000 7f20f2800640 1 -- 172.21.15.131:0/82515542 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20ec13c5a0 con 0x7f20ec13fa80 2024-05-26T22:53:10.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.419+0000 7f20e9600640 1 -- 172.21.15.131:0/82515542 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f20d801c5a0 con 0x7f20ec13fa80 2024-05-26T22:53:10.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.420+0000 7f20e9600640 1 --2- 172.21.15.131:0/82515542 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f20cc06be60 0x7f20cc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:10.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.420+0000 7f20e9600640 1 -- 172.21.15.131:0/82515542 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5767+0+0 (secure 0 0 0) 0x7f20d808b4e0 con 0x7f20ec13fa80 2024-05-26T22:53:10.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.421+0000 7f20f1400640 1 --2- 172.21.15.131:0/82515542 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f20cc06be60 0x7f20cc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:10.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.421+0000 7f20f6c68740 1 Processor -- start 2024-05-26T22:53:10.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.421+0000 7f20f6c68740 1 -- start start 2024-05-26T22:53:10.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.421+0000 7f20f1400640 1 --2- 172.21.15.131:0/82515542 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f20cc06be60 0x7f20cc06e320 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f20ec0b0280 tx=0x7f20e000e040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:10.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.422+0000 7f20f6c68740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559dae58be50 0x559dae53ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:10.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.422+0000 7f20f6c68740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559dae56ef70 0x559dae56e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:10.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.422+0000 7f20f6c68740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559dae56e830 0x559dae543640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:10.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.422+0000 7f20f6c68740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x559dae4a6610 con 0x559dae56e830 2024-05-26T22:53:10.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.422+0000 7f20f6c68740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x559dae44e5c0 con 0x559dae58be50 2024-05-26T22:53:10.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.422+0000 7f20f6c68740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x559dae4a25d0 con 0x559dae56ef70 2024-05-26T22:53:10.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.422+0000 7f20f1e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559dae56e830 0x559dae543640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:10.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.422+0000 7f20f1e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559dae56e830 0x559dae543640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40746/0 (socket says 172.21.15.131:40746) 2024-05-26T22:53:10.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.422+0000 7f20f1e00640 1 -- 172.21.15.131:0/3418661642 learned_addr learned my addr 172.21.15.131:0/3418661642 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:53:10.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.423+0000 7f20f1e00640 1 -- 172.21.15.131:0/3418661642 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559dae56ef70 msgr2=0x559dae56e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:10.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.423+0000 7f20f1e00640 1 --2- 172.21.15.131:0/3418661642 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559dae56ef70 0x559dae56e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:10.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.423+0000 7f20f1e00640 1 -- 172.21.15.131:0/3418661642 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559dae58be50 msgr2=0x559dae53ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:10.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.423+0000 7f20f1e00640 1 --2- 172.21.15.131:0/3418661642 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559dae58be50 0x559dae53ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:10.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.423+0000 7f20f1e00640 1 -- 172.21.15.131:0/3418661642 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559dae1ba2d0 con 0x559dae56e830 2024-05-26T22:53:10.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.423+0000 7f20f1e00640 1 --2- 172.21.15.131:0/3418661642 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559dae56e830 0x559dae543640 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f20e400e200 tx=0x7f20e4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40f2b6aef9a56328 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:10.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.423+0000 7f20ddc00640 1 -- 172.21.15.131:0/3418661642 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e4003b90 con 0x559dae56e830 2024-05-26T22:53:10.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.423+0000 7f20ddc00640 1 -- 172.21.15.131:0/3418661642 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20e4036070 con 0x559dae56e830 2024-05-26T22:53:10.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.423+0000 7f20ddc00640 1 -- 172.21.15.131:0/3418661642 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e4004030 con 0x559dae56e830 2024-05-26T22:53:10.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.424+0000 7f20f6c68740 1 -- 172.21.15.131:0/3418661642 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559dae56e830 msgr2=0x559dae543640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:10.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.424+0000 7f20f6c68740 1 --2- 172.21.15.131:0/3418661642 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559dae56e830 0x559dae543640 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f20e400e200 tx=0x7f20e4004a40 comp rx=0 tx=0).stop 2024-05-26T22:53:10.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.424+0000 7f20f6c68740 1 -- 172.21.15.131:0/3418661642 shutdown_connections 2024-05-26T22:53:10.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.424+0000 7f20f6c68740 1 --2- 172.21.15.131:0/3418661642 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559dae56ef70 0x559dae56e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:10.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.424+0000 7f20f6c68740 1 --2- 172.21.15.131:0/3418661642 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559dae56e830 0x559dae543640 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:10.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.424+0000 7f20f6c68740 1 --2- 172.21.15.131:0/3418661642 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559dae58be50 0x559dae53ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:10.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.424+0000 7f20f6c68740 1 -- 172.21.15.131:0/3418661642 >> 172.21.15.131:0/3418661642 conn(0x559dae540ce0 msgr2=0x559dae5892f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:10.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.424+0000 7f20f6c68740 1 -- 172.21.15.131:0/3418661642 shutdown_connections 2024-05-26T22:53:10.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.424+0000 7f20f6c68740 1 -- 172.21.15.131:0/3418661642 wait complete. 2024-05-26T22:53:10.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.425+0000 7f20f6c68740 1 -- start start 2024-05-26T22:53:10.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.425+0000 7f20f6c68740 1 Processor -- start 2024-05-26T22:53:10.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.426+0000 7f20f6c68740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559dae56ef70 0x559dae586240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:10.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.426+0000 7f20f6c68740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559dae58be50 0x559dae58b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:10.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.426+0000 7f20f6c68740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559dae5995b0 0x559dae569a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:10.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.426+0000 7f20f6c68740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x559dae542c00 con 0x559dae56ef70 2024-05-26T22:53:10.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.426+0000 7f20f6c68740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x559dae461910 con 0x559dae58be50 2024-05-26T22:53:10.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.426+0000 7f20f6c68740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x559dae541ff0 con 0x559dae5995b0 2024-05-26T22:53:10.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.426+0000 7f20f6c68740 10 client.0 _ll_register_callbacks cb 0x559dae49d410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-26T22:53:10.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.426+0000 7f20f1400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559dae56ef70 0x559dae586240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:10.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.426+0000 7f20f1400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559dae56ef70 0x559dae586240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40762/0 (socket says 172.21.15.131:40762) 2024-05-26T22:53:10.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.426+0000 7f20f1400640 1 -- 172.21.15.131:0/932799810 learned_addr learned my addr 172.21.15.131:0/932799810 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:53:10.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.426+0000 7f20f0a00640 1 --2- 172.21.15.131:0/932799810 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559dae58be50 0x559dae58b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:10.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.426+0000 7f20f1400640 1 -- 172.21.15.131:0/932799810 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559dae5995b0 msgr2=0x559dae569a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:10.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.426+0000 7f20f1400640 1 --2- 172.21.15.131:0/932799810 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559dae5995b0 0x559dae569a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:10.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.426+0000 7f20f1400640 1 -- 172.21.15.131:0/932799810 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559dae58be50 msgr2=0x559dae58b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:10.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.426+0000 7f20f1400640 1 --2- 172.21.15.131:0/932799810 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559dae58be50 0x559dae58b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:10.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.426+0000 7f20f1400640 1 -- 172.21.15.131:0/932799810 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559dae2846b0 con 0x559dae56ef70 2024-05-26T22:53:10.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.427+0000 7f20f1400640 1 --2- 172.21.15.131:0/932799810 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559dae56ef70 0x559dae586240 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f20e0005da0 tx=0x7f20e0007420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=587ddc5cfc26ab40 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:10.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.427+0000 7f20d2000640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-26T22:53:10.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.427+0000 7f20d2000640 1 -- 172.21.15.131:0/932799810 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e0007750 con 0x559dae56ef70 2024-05-26T22:53:10.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.427+0000 7f20d2000640 1 -- 172.21.15.131:0/932799810 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(71..71 src has 1..71) ==== 5767+0+0 (secure 0 0 0) 0x7f20e0027b10 con 0x559dae56ef70 2024-05-26T22:53:10.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.427+0000 7f20f6c68740 1 -- 172.21.15.131:0/932799810 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20e4036070 con 0x559dae56ef70 2024-05-26T22:53:10.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.427+0000 7f20d2000640 1 -- 172.21.15.131:0/932799810 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7f20e0027d10 con 0x559dae56ef70 2024-05-26T22:53:10.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.427+0000 7f20f6c68740 10 client.7113 fetch_fsmap learned FSMap version 68 2024-05-26T22:53:10.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.427+0000 7f20f6c68740 1 -- 172.21.15.131:0/932799810 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) -- 0x559dae570150 con 0x559dae56ef70 2024-05-26T22:53:10.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.428+0000 7f20d2000640 1 -- 172.21.15.131:0/932799810 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20e4036070 con 0x559dae56ef70 2024-05-26T22:53:10.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.428+0000 7f20d2000640 1 -- 172.21.15.131:0/932799810 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 68) ==== 3534+0+0 (secure 0 0 0) 0x7f20e0019930 con 0x559dae56ef70 2024-05-26T22:53:10.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.428+0000 7f20f6c68740 10 client.7113 fetch_fsmap finished waiting for FSMap version 68 2024-05-26T22:53:10.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.428+0000 7f20f6c68740 10 client.7113 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-26T22:53:10.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.428+0000 7f20f6c68740 20 client.7113 populate_metadata read hostname 'smithi131' 2024-05-26T22:53:10.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.428+0000 7f20f6c68740 1 --2- 172.21.15.131:0/932799810 >> [v2:172.21.15.131:6835/1418741551,v1:172.21.15.131:6837/1418741551] conn(0x559dae56cab0 0x559dae5a7e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:10.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.428+0000 7f20f6c68740 4 client.7113 mds_command: new command op to 6864 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-26T22:53:10.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.428+0000 7f20f6c68740 1 -- 172.21.15.131:0/932799810 --> [v2:172.21.15.131:6835/1418741551,v1:172.21.15.131:6837/1418741551] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559dae4b3d40 con 0x559dae56cab0 2024-05-26T22:53:10.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.428+0000 7f20f0a00640 1 --2- 172.21.15.131:0/932799810 >> [v2:172.21.15.131:6835/1418741551,v1:172.21.15.131:6837/1418741551] conn(0x559dae56cab0 0x559dae5a7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:10.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.428+0000 7f20f0a00640 1 --2- 172.21.15.131:0/932799810 >> [v2:172.21.15.131:6835/1418741551,v1:172.21.15.131:6837/1418741551] conn(0x559dae56cab0 0x559dae5a7e70 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=dd35e76987035464 server_cookie=5af420f53770cd1 in_seq=0 out_seq=0 2024-05-26T22:53:10.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.428+0000 7f20d2000640 10 client.7113 ms_handle_connect on v2:172.21.15.131:6835/1418741551 2024-05-26T22:53:10.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.429+0000 7f20d2000640 1 -- 172.21.15.131:0/932799810 <== mds.1 v2:172.21.15.131:6835/1418741551 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559dae4b3d40 con 0x559dae56cab0 2024-05-26T22:53:10.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.429+0000 7f20d2000640 10 client.7113 handle_command_reply: tid=0 2024-05-26T22:53:10.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.429+0000 7f20f6c68740 1 client.7113 shutdown 2024-05-26T22:53:10.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.430+0000 7f20f6c68740 1 -- 172.21.15.131:0/932799810 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559dae56ef70 msgr2=0x559dae586240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:10.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.430+0000 7f20f6c68740 1 --2- 172.21.15.131:0/932799810 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559dae56ef70 0x559dae586240 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f20e0005da0 tx=0x7f20e0007420 comp rx=0 tx=0).stop 2024-05-26T22:53:10.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.430+0000 7f20f6c68740 1 -- 172.21.15.131:0/932799810 shutdown_connections 2024-05-26T22:53:10.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.430+0000 7f20f6c68740 1 --2- 172.21.15.131:0/932799810 >> [v2:172.21.15.131:6835/1418741551,v1:172.21.15.131:6837/1418741551] conn(0x559dae56cab0 0x559dae5a7e70 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:10.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.430+0000 7f20f6c68740 1 --2- 172.21.15.131:0/932799810 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559dae5995b0 0x559dae569a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:10.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.430+0000 7f20f6c68740 1 --2- 172.21.15.131:0/932799810 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559dae58be50 0x559dae58b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:10.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.430+0000 7f20f6c68740 1 --2- 172.21.15.131:0/932799810 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559dae56ef70 0x559dae586240 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:10.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.430+0000 7f20d2000640 0 client.7113 ms_handle_reset on v2:172.21.15.131:6835/1418741551 2024-05-26T22:53:10.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.430+0000 7f20f6c68740 1 -- 172.21.15.131:0/932799810 >> 172.21.15.131:0/932799810 conn(0x559dae56e830 msgr2=0x559dae5892a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:10.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.430+0000 7f20f6c68740 1 -- 172.21.15.131:0/932799810 shutdown_connections 2024-05-26T22:53:10.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.430+0000 7f20f6c68740 1 -- 172.21.15.131:0/932799810 wait complete. 2024-05-26T22:53:10.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.430+0000 7f20f6c68740 20 client.7113 trim_cache size 0 max 16384 2024-05-26T22:53:10.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.450+0000 7f20f6c68740 1 Processor -- start 2024-05-26T22:53:10.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.450+0000 7f20f6c68740 1 -- start start 2024-05-26T22:53:10.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.451+0000 7f20f6c68740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559dae56ef70 0x559dae5676c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:10.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.451+0000 7f20f6c68740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559dae58be50 0x559dae567c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:10.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.451+0000 7f20f6c68740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559dae5995b0 0x559dae56a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:10.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.451+0000 7f20f6c68740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f20ec1fbeb0 con 0x559dae56ef70 2024-05-26T22:53:10.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.451+0000 7f20f6c68740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f20ec1fc4d0 con 0x559dae5995b0 2024-05-26T22:53:10.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.451+0000 7f20f6c68740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f20ec13da10 con 0x559dae58be50 2024-05-26T22:53:10.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.451+0000 7f20f1400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559dae56ef70 0x559dae5676c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:10.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.451+0000 7f20f1400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559dae56ef70 0x559dae5676c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40764/0 (socket says 172.21.15.131:40764) 2024-05-26T22:53:10.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.451+0000 7f20f1400640 1 -- 172.21.15.131:0/3675847363 learned_addr learned my addr 172.21.15.131:0/3675847363 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:53:10.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.451+0000 7f20f1400640 1 -- 172.21.15.131:0/3675847363 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559dae58be50 msgr2=0x559dae567c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:10.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.451+0000 7f20f1400640 1 --2- 172.21.15.131:0/3675847363 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559dae58be50 0x559dae567c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:10.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.451+0000 7f20f1400640 1 -- 172.21.15.131:0/3675847363 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559dae5995b0 msgr2=0x559dae56a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:10.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.451+0000 7f20f1400640 1 --2- 172.21.15.131:0/3675847363 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559dae5995b0 0x559dae56a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:10.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.451+0000 7f20f1400640 1 -- 172.21.15.131:0/3675847363 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559dae541b80 con 0x559dae56ef70 2024-05-26T22:53:10.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.452+0000 7f20f1400640 1 --2- 172.21.15.131:0/3675847363 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559dae56ef70 0x559dae5676c0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f20e0019a90 tx=0x7f20e0021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3197559246896a04 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:10.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.452+0000 7f20dd200640 1 -- 172.21.15.131:0/3675847363 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e0034030 con 0x559dae56ef70 2024-05-26T22:53:10.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.452+0000 7f20dd200640 1 -- 172.21.15.131:0/3675847363 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20e0007550 con 0x559dae56ef70 2024-05-26T22:53:10.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.452+0000 7f20dd200640 1 -- 172.21.15.131:0/3675847363 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e002f420 con 0x559dae56ef70 2024-05-26T22:53:10.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.452+0000 7f20f6c68740 1 -- 172.21.15.131:0/3675847363 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559dae56ef70 msgr2=0x559dae5676c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:10.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.452+0000 7f20f6c68740 1 --2- 172.21.15.131:0/3675847363 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559dae56ef70 0x559dae5676c0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f20e0019a90 tx=0x7f20e0021000 comp rx=0 tx=0).stop 2024-05-26T22:53:10.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.453+0000 7f20f6c68740 1 -- 172.21.15.131:0/3675847363 shutdown_connections 2024-05-26T22:53:10.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.453+0000 7f20f6c68740 1 --2- 172.21.15.131:0/3675847363 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559dae58be50 0x559dae567c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:10.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.453+0000 7f20f6c68740 1 --2- 172.21.15.131:0/3675847363 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559dae5995b0 0x559dae56a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:10.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.453+0000 7f20f6c68740 1 --2- 172.21.15.131:0/3675847363 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559dae56ef70 0x559dae5676c0 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:10.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.453+0000 7f20f6c68740 1 -- 172.21.15.131:0/3675847363 >> 172.21.15.131:0/3675847363 conn(0x559dae56e830 msgr2=0x559dae58af50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:10.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.453+0000 7f20f6c68740 1 -- 172.21.15.131:0/3675847363 shutdown_connections 2024-05-26T22:53:10.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.453+0000 7f20f6c68740 1 -- 172.21.15.131:0/3675847363 wait complete. 2024-05-26T22:53:10.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.453+0000 7f20f6c68740 1 -- start start 2024-05-26T22:53:10.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.454+0000 7f20f6c68740 1 Processor -- start 2024-05-26T22:53:10.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.454+0000 7f20f6c68740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559dae56ef70 0x559dae580950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:10.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.454+0000 7f20f6c68740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559dae58be50 0x559dae566350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:10.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.454+0000 7f20f6c68740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559dae5995b0 0x559dae5889c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:10.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.454+0000 7f20f6c68740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x559dae56e410 con 0x559dae56ef70 2024-05-26T22:53:10.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.455+0000 7f20f6c68740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x559dae570430 con 0x559dae58be50 2024-05-26T22:53:10.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.455+0000 7f20f6c68740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x559dae571180 con 0x559dae5995b0 2024-05-26T22:53:10.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.455+0000 7f20f6c68740 10 client.0 _ll_register_callbacks cb 0x7f20e0001db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-26T22:53:10.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.455+0000 7f20f1400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559dae56ef70 0x559dae580950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:10.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.455+0000 7f20f1400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559dae56ef70 0x559dae580950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40770/0 (socket says 172.21.15.131:40770) 2024-05-26T22:53:10.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.455+0000 7f20f1400640 1 -- 172.21.15.131:0/3753199517 learned_addr learned my addr 172.21.15.131:0/3753199517 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:53:10.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.455+0000 7f20f1400640 1 -- 172.21.15.131:0/3753199517 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559dae5995b0 msgr2=0x559dae5889c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:10.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.455+0000 7f20f1400640 1 --2- 172.21.15.131:0/3753199517 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559dae5995b0 0x559dae5889c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:10.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.455+0000 7f20f1400640 1 -- 172.21.15.131:0/3753199517 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559dae58be50 msgr2=0x559dae566350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:10.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.455+0000 7f20f1400640 1 --2- 172.21.15.131:0/3753199517 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559dae58be50 0x559dae566350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:10.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.455+0000 7f20f1400640 1 -- 172.21.15.131:0/3753199517 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559dae607c10 con 0x559dae56ef70 2024-05-26T22:53:10.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.455+0000 7f20f1400640 1 --2- 172.21.15.131:0/3753199517 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559dae56ef70 0x559dae580950 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f20e0019a90 tx=0x7f20e0010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6b38cf2b10a633 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:10.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.455+0000 7f20d2000640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-26T22:53:10.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.456+0000 7f20d2000640 1 -- 172.21.15.131:0/3753199517 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e0039640 con 0x559dae56ef70 2024-05-26T22:53:10.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.456+0000 7f20d2000640 1 -- 172.21.15.131:0/3753199517 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(71..71 src has 1..71) ==== 5767+0+0 (secure 0 0 0) 0x7f20e00397e0 con 0x559dae56ef70 2024-05-26T22:53:10.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.456+0000 7f20f6c68740 1 -- 172.21.15.131:0/3753199517 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20e0007550 con 0x559dae56ef70 2024-05-26T22:53:10.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.456+0000 7f20d2000640 1 -- 172.21.15.131:0/3753199517 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7f20e0047050 con 0x559dae56ef70 2024-05-26T22:53:10.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.456+0000 7f20f6c68740 10 client.7119 fetch_fsmap learned FSMap version 68 2024-05-26T22:53:10.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.456+0000 7f20f6c68740 1 -- 172.21.15.131:0/3753199517 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) -- 0x559dae566b40 con 0x559dae56ef70 2024-05-26T22:53:10.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.456+0000 7f20d2000640 1 -- 172.21.15.131:0/3753199517 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20e0007550 con 0x559dae56ef70 2024-05-26T22:53:10.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.456+0000 7f20d2000640 1 -- 172.21.15.131:0/3753199517 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 68) ==== 3534+0+0 (secure 0 0 0) 0x7f20e004a020 con 0x559dae56ef70 2024-05-26T22:53:10.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.456+0000 7f20f6c68740 10 client.7119 fetch_fsmap finished waiting for FSMap version 68 2024-05-26T22:53:10.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.456+0000 7f20f6c68740 10 client.7119 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-26T22:53:10.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.456+0000 7f20f6c68740 20 client.7119 populate_metadata read hostname 'smithi131' 2024-05-26T22:53:10.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.456+0000 7f20f6c68740 1 --2- 172.21.15.131:0/3753199517 >> [v2:172.21.15.131:6835/1418741551,v1:172.21.15.131:6837/1418741551] conn(0x559dae58c820 0x559dae58cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:10.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.456+0000 7f20f6c68740 4 client.7119 mds_command: new command op to 6864 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-26T22:53:10.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.456+0000 7f20f6c68740 1 -- 172.21.15.131:0/3753199517 --> [v2:172.21.15.131:6835/1418741551,v1:172.21.15.131:6837/1418741551] -- command(tid 0: {"prefix": "status"}) -- 0x559dae4ae090 con 0x559dae58c820 2024-05-26T22:53:10.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.457+0000 7f20f0a00640 1 --2- 172.21.15.131:0/3753199517 >> [v2:172.21.15.131:6835/1418741551,v1:172.21.15.131:6837/1418741551] conn(0x559dae58c820 0x559dae58cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:10.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.457+0000 7f20f0a00640 1 --2- 172.21.15.131:0/3753199517 >> [v2:172.21.15.131:6835/1418741551,v1:172.21.15.131:6837/1418741551] conn(0x559dae58c820 0x559dae58cc00 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c152c600bd42d46f server_cookie=1f5182ebc38efdff in_seq=0 out_seq=0 2024-05-26T22:53:10.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.457+0000 7f20d2000640 10 client.7119 ms_handle_connect on v2:172.21.15.131:6835/1418741551 2024-05-26T22:53:10.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.457+0000 7f20d2000640 1 -- 172.21.15.131:0/3753199517 <== mds.1 v2:172.21.15.131:6835/1418741551 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x559dae4ae090 con 0x559dae58c820 2024-05-26T22:53:10.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.457+0000 7f20d2000640 10 client.7119 handle_command_reply: tid=0 2024-05-26T22:53:10.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.457+0000 7f20f6c68740 1 client.7119 shutdown 2024-05-26T22:53:10.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.457+0000 7f20f6c68740 1 -- 172.21.15.131:0/3753199517 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559dae56ef70 msgr2=0x559dae580950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:10.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.457+0000 7f20f6c68740 1 --2- 172.21.15.131:0/3753199517 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559dae56ef70 0x559dae580950 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f20e0019a90 tx=0x7f20e0010000 comp rx=0 tx=0).stop 2024-05-26T22:53:10.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.458+0000 7f20f6c68740 1 -- 172.21.15.131:0/3753199517 shutdown_connections 2024-05-26T22:53:10.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.458+0000 7f20f6c68740 1 --2- 172.21.15.131:0/3753199517 >> [v2:172.21.15.131:6835/1418741551,v1:172.21.15.131:6837/1418741551] conn(0x559dae58c820 0x559dae58cc00 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:10.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.458+0000 7f20f6c68740 1 --2- 172.21.15.131:0/3753199517 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x559dae5995b0 0x559dae5889c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:10.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.458+0000 7f20f6c68740 1 --2- 172.21.15.131:0/3753199517 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559dae58be50 0x559dae566350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:10.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.458+0000 7f20f6c68740 1 --2- 172.21.15.131:0/3753199517 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559dae56ef70 0x559dae580950 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:10.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.458+0000 7f20f6c68740 1 -- 172.21.15.131:0/3753199517 >> 172.21.15.131:0/3753199517 conn(0x559dae56e830 msgr2=0x559dae5676c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:10.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.458+0000 7f20d2000640 0 client.7119 ms_handle_reset on v2:172.21.15.131:6835/1418741551 2024-05-26T22:53:10.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.458+0000 7f20f6c68740 1 -- 172.21.15.131:0/3753199517 shutdown_connections 2024-05-26T22:53:10.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.458+0000 7f20f6c68740 1 -- 172.21.15.131:0/3753199517 wait complete. 2024-05-26T22:53:10.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.458+0000 7f20f6c68740 20 client.7119 trim_cache size 0 max 16384 2024-05-26T22:53:10.462 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-26T22:53:10.462 INFO:teuthology.orchestra.run.smithi131.stdout: "cluster_fsid": "46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f", 2024-05-26T22:53:10.462 INFO:teuthology.orchestra.run.smithi131.stdout: "whoami": 1, 2024-05-26T22:53:10.462 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 6864, 2024-05-26T22:53:10.462 INFO:teuthology.orchestra.run.smithi131.stdout: "want_state": "up:active", 2024-05-26T22:53:10.462 INFO:teuthology.orchestra.run.smithi131.stdout: "state": "up:active", 2024-05-26T22:53:10.462 INFO:teuthology.orchestra.run.smithi131.stdout: "fs_name": "cephfs", 2024-05-26T22:53:10.462 INFO:teuthology.orchestra.run.smithi131.stdout: "rank_uptime": 3.3327089079999999, 2024-05-26T22:53:10.462 INFO:teuthology.orchestra.run.smithi131.stdout: "mdsmap_epoch": 68, 2024-05-26T22:53:10.462 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch": 71, 2024-05-26T22:53:10.462 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch_barrier": 71, 2024-05-26T22:53:10.462 INFO:teuthology.orchestra.run.smithi131.stdout: "uptime": 21.600837145 2024-05-26T22:53:10.463 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-26T22:53:10.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.458+0000 7f20f2800640 1 -- 172.21.15.131:0/82515542 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f20cc06be60 msgr2=0x7f20cc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:10.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.458+0000 7f20f2800640 1 --2- 172.21.15.131:0/82515542 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f20cc06be60 0x7f20cc06e320 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f20ec0b0280 tx=0x7f20e000e040 comp rx=0 tx=0).stop 2024-05-26T22:53:10.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.458+0000 7f20f2800640 1 -- 172.21.15.131:0/82515542 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f20ec13fa80 msgr2=0x7f20ec1599c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:10.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.458+0000 7f20f2800640 1 --2- 172.21.15.131:0/82515542 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f20ec13fa80 0x7f20ec1599c0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f20d80108c0 tx=0x7f20d8010d80 comp rx=0 tx=0).stop 2024-05-26T22:53:10.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.459+0000 7f20f2800640 1 -- 172.21.15.131:0/82515542 shutdown_connections 2024-05-26T22:53:10.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.459+0000 7f20f2800640 1 --2- 172.21.15.131:0/82515542 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f20ec0cf490 0x7f20ec0cf920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:10.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.459+0000 7f20f2800640 1 --2- 172.21.15.131:0/82515542 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f20cc06be60 0x7f20cc06e320 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:10.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.459+0000 7f20f2800640 1 --2- 172.21.15.131:0/82515542 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f20ec13fa80 0x7f20ec1599c0 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:10.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.459+0000 7f20f2800640 1 --2- 172.21.15.131:0/82515542 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f20d4004d40 0x7f20ec144c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:10.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.459+0000 7f20f2800640 1 -- 172.21.15.131:0/82515542 >> 172.21.15.131:0/82515542 conn(0x7f20ec0b5660 msgr2=0x7f20ec0b0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:10.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.459+0000 7f20f2800640 1 -- 172.21.15.131:0/82515542 shutdown_connections 2024-05-26T22:53:10.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.459+0000 7f20f2800640 1 -- 172.21.15.131:0/82515542 wait complete. 2024-05-26T22:53:10.476 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-05-26T22:53:10.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.672+0000 7f55dec00640 1 Processor -- start 2024-05-26T22:53:10.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.673+0000 7f55dec00640 1 -- start start 2024-05-26T22:53:10.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.674+0000 7f55dec00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f55d806f180 0x7f55d8072e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:10.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.674+0000 7f55dec00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55d8072540 0x7f55d8161ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:10.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.674+0000 7f55dec00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55d80af8e0 con 0x7f55d80cfe50 2024-05-26T22:53:10.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.674+0000 7f55dec00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f55d80cfb30 con 0x7f55d806f180 2024-05-26T22:53:10.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.674+0000 7f55dec00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f55d80cf200 con 0x7f55d8072540 2024-05-26T22:53:10.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.674+0000 7f55dcc00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f55d80cfe50 0x7f55d80715a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:53208/0 (socket says 172.21.15.131:53208) 2024-05-26T22:53:10.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.674+0000 7f55dcc00640 1 -- 172.21.15.131:0/3696643716 learned_addr learned my addr 172.21.15.131:0/3696643716 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:53:10.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.674+0000 7f55d7e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f55d806f180 0x7f55d8072e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:10.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.675+0000 7f55cf400640 1 --2- 172.21.15.131:0/3696643716 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55d8072540 0x7f55d8161ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:10.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.675+0000 7f55d7400640 1 -- 172.21.15.131:0/3696643716 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4032530257 0 0) 0x7f55d80af8e0 con 0x7f55d80cfe50 2024-05-26T22:53:10.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.675+0000 7f55d7e00640 1 -- 172.21.15.131:0/3696643716 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55d8072540 msgr2=0x7f55d8161ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:10.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.675+0000 7f55d7e00640 1 --2- 172.21.15.131:0/3696643716 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55d8072540 0x7f55d8161ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:10.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.675+0000 7f55d7e00640 1 -- 172.21.15.131:0/3696643716 >> v1:172.21.15.131:6789/0 conn(0x7f55d80cfe50 legacy=0x7f55d80715a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:10.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.675+0000 7f55d7e00640 1 -- 172.21.15.131:0/3696643716 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55d80cf4e0 con 0x7f55d806f180 2024-05-26T22:53:10.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.675+0000 7f55cf400640 1 --2- 172.21.15.131:0/3696643716 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55d8072540 0x7f55d8161ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:53:10.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.676+0000 7f55d7e00640 1 --2- 172.21.15.131:0/3696643716 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55d806f180 0x7f55d8072e90 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f55c80097a0 tx=0x7f55c802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4bcf1468e182c2c server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:10.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.676+0000 7f55d7400640 1 -- 172.21.15.131:0/3696643716 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c8037070 con 0x7f55d806f180 2024-05-26T22:53:10.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.676+0000 7f55d7400640 1 -- 172.21.15.131:0/3696643716 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55c8032440 con 0x7f55d806f180 2024-05-26T22:53:10.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.676+0000 7f55d7400640 1 -- 172.21.15.131:0/3696643716 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c8032810 con 0x7f55d806f180 2024-05-26T22:53:10.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.677+0000 7f55dec00640 1 -- 172.21.15.131:0/3696643716 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55d806f180 msgr2=0x7f55d8072e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:10.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.677+0000 7f55dec00640 1 --2- 172.21.15.131:0/3696643716 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55d806f180 0x7f55d8072e90 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f55c80097a0 tx=0x7f55c802a180 comp rx=0 tx=0).stop 2024-05-26T22:53:10.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.677+0000 7f55dec00640 1 -- 172.21.15.131:0/3696643716 shutdown_connections 2024-05-26T22:53:10.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.677+0000 7f55dec00640 1 --2- 172.21.15.131:0/3696643716 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55d8072540 0x7f55d8161ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:10.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.677+0000 7f55dec00640 1 --2- 172.21.15.131:0/3696643716 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55d806f180 0x7f55d8072e90 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:10.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.677+0000 7f55dec00640 1 -- 172.21.15.131:0/3696643716 >> 172.21.15.131:0/3696643716 conn(0x7f55d806bc90 msgr2=0x7f55d80b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:10.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.677+0000 7f55dec00640 1 -- 172.21.15.131:0/3696643716 shutdown_connections 2024-05-26T22:53:10.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.677+0000 7f55dec00640 1 -- 172.21.15.131:0/3696643716 wait complete. 2024-05-26T22:53:10.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.679+0000 7f55dec00640 1 Processor -- start 2024-05-26T22:53:10.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.679+0000 7f55dec00640 1 -- start start 2024-05-26T22:53:10.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.680+0000 7f55dec00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f55d806f180 0x7f55d81518c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:10.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.680+0000 7f55dec00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55d8072540 0x7f55d80ce720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:10.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.680+0000 7f55dec00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55d80cfe50 0x7f55d8160c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:10.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.680+0000 7f55dec00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f55d806f6e0 con 0x7f55d80cfe50 2024-05-26T22:53:10.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.680+0000 7f55dec00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f55d806f560 con 0x7f55d806f180 2024-05-26T22:53:10.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.680+0000 7f55dec00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f55d80d02e0 con 0x7f55d8072540 2024-05-26T22:53:10.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.680+0000 7f55dcc00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55d80cfe50 0x7f55d8160c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:10.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.680+0000 7f55dcc00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55d80cfe50 0x7f55d8160c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40796/0 (socket says 172.21.15.131:40796) 2024-05-26T22:53:10.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.680+0000 7f55dcc00640 1 -- 172.21.15.131:0/3868925746 learned_addr learned my addr 172.21.15.131:0/3868925746 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:53:10.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.681+0000 7f55d7e00640 1 --2- 172.21.15.131:0/3868925746 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f55d806f180 0x7f55d81518c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:10.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.681+0000 7f55cf400640 1 --2- 172.21.15.131:0/3868925746 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55d8072540 0x7f55d80ce720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:10.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.681+0000 7f55dcc00640 1 -- 172.21.15.131:0/3868925746 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55d8072540 msgr2=0x7f55d80ce720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:10.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.681+0000 7f55dcc00640 1 --2- 172.21.15.131:0/3868925746 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55d8072540 0x7f55d80ce720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:10.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.681+0000 7f55dcc00640 1 -- 172.21.15.131:0/3868925746 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f55d806f180 msgr2=0x7f55d81518c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:10.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.681+0000 7f55dcc00640 1 --2- 172.21.15.131:0/3868925746 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f55d806f180 0x7f55d81518c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:10.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.681+0000 7f55dcc00640 1 -- 172.21.15.131:0/3868925746 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55d80c97c0 con 0x7f55d80cfe50 2024-05-26T22:53:10.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.681+0000 7f55dcc00640 1 --2- 172.21.15.131:0/3868925746 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55d80cfe50 0x7f55d8160c10 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f55d000c730 tx=0x7f55d000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:10.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.681+0000 7f55d4c00640 1 -- 172.21.15.131:0/3868925746 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d0018070 con 0x7f55d80cfe50 2024-05-26T22:53:10.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.681+0000 7f55d4c00640 1 -- 172.21.15.131:0/3868925746 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55d000ce40 con 0x7f55d80cfe50 2024-05-26T22:53:10.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.681+0000 7f55d4c00640 1 -- 172.21.15.131:0/3868925746 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d000a860 con 0x7f55d80cfe50 2024-05-26T22:53:10.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.681+0000 7f55dec00640 1 -- 172.21.15.131:0/3868925746 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55d80c9aa0 con 0x7f55d80cfe50 2024-05-26T22:53:10.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.681+0000 7f55dec00640 1 -- 172.21.15.131:0/3868925746 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55d80ca060 con 0x7f55d80cfe50 2024-05-26T22:53:10.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.684+0000 7f55d4c00640 1 -- 172.21.15.131:0/3868925746 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f55d000aa00 con 0x7f55d80cfe50 2024-05-26T22:53:10.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.685+0000 7f55d4c00640 1 --2- 172.21.15.131:0/3868925746 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f55b406be30 0x7f55b406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:10.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.685+0000 7f55d4c00640 1 -- 172.21.15.131:0/3868925746 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5767+0+0 (secure 0 0 0) 0x7f55d008a530 con 0x7f55d80cfe50 2024-05-26T22:53:10.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.685+0000 7f55d7e00640 1 --2- 172.21.15.131:0/3868925746 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f55b406be30 0x7f55b406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:10.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.686+0000 7f55e19f5740 1 Processor -- start 2024-05-26T22:53:10.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.686+0000 7f55e19f5740 1 -- start start 2024-05-26T22:53:10.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.686+0000 7f55d7e00640 1 --2- 172.21.15.131:0/3868925746 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f55b406be30 0x7f55b406e2f0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f55c8034000 tx=0x7f55c802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:10.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.686+0000 7f55e19f5740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a2d5d44c70 0x55a2d5cf7890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:10.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.686+0000 7f55e19f5740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a2d5d27d90 0x55a2d5d27110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:10.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.687+0000 7f55e19f5740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a2d5d27650 0x55a2d5cfc460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:10.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.687+0000 7f55e19f5740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55a2d5c5f610 con 0x55a2d5d27d90 2024-05-26T22:53:10.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.687+0000 7f55e19f5740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55a2d5c075c0 con 0x55a2d5d44c70 2024-05-26T22:53:10.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.687+0000 7f55e19f5740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x55a2d5c5b5d0 con 0x55a2d5d27650 2024-05-26T22:53:10.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.687+0000 7f55cf400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a2d5d27d90 0x55a2d5d27110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:10.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.687+0000 7f55cf400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a2d5d27d90 0x55a2d5d27110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40806/0 (socket says 172.21.15.131:40806) 2024-05-26T22:53:10.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.687+0000 7f55cf400640 1 -- 172.21.15.131:0/3745199763 learned_addr learned my addr 172.21.15.131:0/3745199763 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:53:10.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.687+0000 7f55d7e00640 1 --2- 172.21.15.131:0/3745199763 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a2d5d44c70 0x55a2d5cf7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:10.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.687+0000 7f55dcc00640 1 --2- 172.21.15.131:0/3745199763 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a2d5d27650 0x55a2d5cfc460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:10.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.687+0000 7f55cf400640 1 -- 172.21.15.131:0/3745199763 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a2d5d27650 msgr2=0x55a2d5cfc460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:10.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.687+0000 7f55cf400640 1 --2- 172.21.15.131:0/3745199763 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a2d5d27650 0x55a2d5cfc460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:10.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.687+0000 7f55cf400640 1 -- 172.21.15.131:0/3745199763 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a2d5d44c70 msgr2=0x55a2d5cf7890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:10.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.687+0000 7f55cf400640 1 --2- 172.21.15.131:0/3745199763 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a2d5d44c70 0x55a2d5cf7890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:10.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.687+0000 7f55cf400640 1 -- 172.21.15.131:0/3745199763 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2d59732d0 con 0x55a2d5d27d90 2024-05-26T22:53:10.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.687+0000 7f55dcc00640 1 --2- 172.21.15.131:0/3745199763 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a2d5d27650 0x55a2d5cfc460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:53:10.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.688+0000 7f55cf400640 1 --2- 172.21.15.131:0/3745199763 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a2d5d27d90 0x55a2d5d27110 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f55c000b980 tx=0x7f55c000be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6e4f815492623a2 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:10.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.688+0000 7f55c7e00640 1 -- 172.21.15.131:0/3745199763 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c0016070 con 0x55a2d5d27d90 2024-05-26T22:53:10.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.688+0000 7f55c7e00640 1 -- 172.21.15.131:0/3745199763 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55c000dc70 con 0x55a2d5d27d90 2024-05-26T22:53:10.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.688+0000 7f55c7e00640 1 -- 172.21.15.131:0/3745199763 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c0012460 con 0x55a2d5d27d90 2024-05-26T22:53:10.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.688+0000 7f55e19f5740 1 -- 172.21.15.131:0/3745199763 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a2d5d27d90 msgr2=0x55a2d5d27110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:10.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.688+0000 7f55e19f5740 1 --2- 172.21.15.131:0/3745199763 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a2d5d27d90 0x55a2d5d27110 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f55c000b980 tx=0x7f55c000be40 comp rx=0 tx=0).stop 2024-05-26T22:53:10.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.689+0000 7f55e19f5740 1 -- 172.21.15.131:0/3745199763 shutdown_connections 2024-05-26T22:53:10.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.689+0000 7f55e19f5740 1 --2- 172.21.15.131:0/3745199763 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a2d5d27650 0x55a2d5cfc460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:10.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.689+0000 7f55e19f5740 1 --2- 172.21.15.131:0/3745199763 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a2d5d27d90 0x55a2d5d27110 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:10.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.689+0000 7f55e19f5740 1 --2- 172.21.15.131:0/3745199763 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a2d5d44c70 0x55a2d5cf7890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:10.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.689+0000 7f55e19f5740 1 -- 172.21.15.131:0/3745199763 >> 172.21.15.131:0/3745199763 conn(0x55a2d5cf9b00 msgr2=0x55a2d5d42110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:10.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.689+0000 7f55e19f5740 1 -- 172.21.15.131:0/3745199763 shutdown_connections 2024-05-26T22:53:10.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.689+0000 7f55e19f5740 1 -- 172.21.15.131:0/3745199763 wait complete. 2024-05-26T22:53:10.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.689+0000 7f55e19f5740 1 -- start start 2024-05-26T22:53:10.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.690+0000 7f55e19f5740 1 Processor -- start 2024-05-26T22:53:10.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.690+0000 7f55e19f5740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a2d5d27d90 0x55a2d5d23140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:10.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.690+0000 7f55e19f5740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a2d5d44c70 0x55a2d5d2a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:10.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.690+0000 7f55e19f5740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a2d5d2a710 0x55a2d5d26030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:10.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.690+0000 7f55e19f5740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55a2d5cfba20 con 0x55a2d5d2a710 2024-05-26T22:53:10.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.690+0000 7f55e19f5740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55a2d5c1a910 con 0x55a2d5d44c70 2024-05-26T22:53:10.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.690+0000 7f55e19f5740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x55a2d5cfae10 con 0x55a2d5d27d90 2024-05-26T22:53:10.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.691+0000 7f55e19f5740 10 client.0 _ll_register_callbacks cb 0x55a2d5c56410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-26T22:53:10.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.691+0000 7f55dcc00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a2d5d2a710 0x55a2d5d26030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:10.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.691+0000 7f55dcc00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a2d5d2a710 0x55a2d5d26030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40822/0 (socket says 172.21.15.131:40822) 2024-05-26T22:53:10.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.691+0000 7f55dcc00640 1 -- 172.21.15.131:0/2828536254 learned_addr learned my addr 172.21.15.131:0/2828536254 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:53:10.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.691+0000 7f55cf400640 1 --2- 172.21.15.131:0/2828536254 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a2d5d44c70 0x55a2d5d2a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:10.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.691+0000 7f55d7e00640 1 --2- 172.21.15.131:0/2828536254 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a2d5d27d90 0x55a2d5d23140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:10.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.691+0000 7f55dcc00640 1 -- 172.21.15.131:0/2828536254 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a2d5d27d90 msgr2=0x55a2d5d23140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:10.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.691+0000 7f55dcc00640 1 --2- 172.21.15.131:0/2828536254 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a2d5d27d90 0x55a2d5d23140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:10.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.691+0000 7f55dcc00640 1 -- 172.21.15.131:0/2828536254 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a2d5d44c70 msgr2=0x55a2d5d2a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:10.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.691+0000 7f55dcc00640 1 --2- 172.21.15.131:0/2828536254 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a2d5d44c70 0x55a2d5d2a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:10.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.691+0000 7f55dcc00640 1 -- 172.21.15.131:0/2828536254 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2d5a3d6b0 con 0x55a2d5d2a710 2024-05-26T22:53:10.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.691+0000 7f55d7e00640 1 --2- 172.21.15.131:0/2828536254 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a2d5d27d90 0x55a2d5d23140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:53:10.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.691+0000 7f55dcc00640 1 --2- 172.21.15.131:0/2828536254 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a2d5d2a710 0x55a2d5d26030 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f55d000c700 tx=0x7f55d0058f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbedc4603e3b977a server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:10.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.691+0000 7f55c4c00640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-26T22:53:10.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.692+0000 7f55c4c00640 1 -- 172.21.15.131:0/2828536254 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d008a0c0 con 0x55a2d5d2a710 2024-05-26T22:53:10.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.692+0000 7f55c4c00640 1 -- 172.21.15.131:0/2828536254 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(71..71 src has 1..71) ==== 5767+0+0 (secure 0 0 0) 0x7f55d008a260 con 0x55a2d5d2a710 2024-05-26T22:53:10.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.692+0000 7f55e19f5740 1 -- 172.21.15.131:0/2828536254 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55c000dc70 con 0x55a2d5d2a710 2024-05-26T22:53:10.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.692+0000 7f55c4c00640 1 -- 172.21.15.131:0/2828536254 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7f55d0088050 con 0x55a2d5d2a710 2024-05-26T22:53:10.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.692+0000 7f55e19f5740 10 client.7134 fetch_fsmap learned FSMap version 68 2024-05-26T22:53:10.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.692+0000 7f55e19f5740 1 -- 172.21.15.131:0/2828536254 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) -- 0x55a2d5d42ae0 con 0x55a2d5d2a710 2024-05-26T22:53:10.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.692+0000 7f55c4c00640 1 -- 172.21.15.131:0/2828536254 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55c000dc70 con 0x55a2d5d2a710 2024-05-26T22:53:10.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.692+0000 7f55c4c00640 1 -- 172.21.15.131:0/2828536254 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 68) ==== 3534+0+0 (secure 0 0 0) 0x7f55d006cb90 con 0x55a2d5d2a710 2024-05-26T22:53:10.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.692+0000 7f55e19f5740 10 client.7134 fetch_fsmap finished waiting for FSMap version 68 2024-05-26T22:53:10.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.692+0000 7f55e19f5740 10 client.7134 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-26T22:53:10.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.692+0000 7f55e19f5740 20 client.7134 populate_metadata read hostname 'smithi131' 2024-05-26T22:53:10.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.692+0000 7f55e19f5740 1 --2- 172.21.15.131:0/2828536254 >> [v2:172.21.15.187:6832/3680999672,v1:172.21.15.187:6834/3680999672] conn(0x55a2d5d288d0 0x55a2d5d55e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:10.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.692+0000 7f55e19f5740 4 client.7134 mds_command: new command op to 6876 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-26T22:53:10.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.692+0000 7f55e19f5740 1 -- 172.21.15.131:0/2828536254 --> [v2:172.21.15.187:6832/3680999672,v1:172.21.15.187:6834/3680999672] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a2d5c6cd40 con 0x55a2d5d288d0 2024-05-26T22:53:10.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.693+0000 7f55d7e00640 1 --2- 172.21.15.131:0/2828536254 >> [v2:172.21.15.187:6832/3680999672,v1:172.21.15.187:6834/3680999672] conn(0x55a2d5d288d0 0x55a2d5d55e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:10.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.694+0000 7f55d7e00640 1 --2- 172.21.15.131:0/2828536254 >> [v2:172.21.15.187:6832/3680999672,v1:172.21.15.187:6834/3680999672] conn(0x55a2d5d288d0 0x55a2d5d55e10 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17e7c7c50439e81e server_cookie=9c95fb77030b018f in_seq=0 out_seq=0 2024-05-26T22:53:10.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.694+0000 7f55c4c00640 10 client.7134 ms_handle_connect on v2:172.21.15.187:6832/3680999672 2024-05-26T22:53:10.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.696+0000 7f55c4c00640 1 -- 172.21.15.131:0/2828536254 <== mds.0 v2:172.21.15.187:6832/3680999672 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a2d5c6cd40 con 0x55a2d5d288d0 2024-05-26T22:53:10.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.696+0000 7f55c4c00640 10 client.7134 handle_command_reply: tid=0 2024-05-26T22:53:10.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.696+0000 7f55e19f5740 1 client.7134 shutdown 2024-05-26T22:53:10.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.696+0000 7f55e19f5740 1 -- 172.21.15.131:0/2828536254 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a2d5d2a710 msgr2=0x55a2d5d26030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:10.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.696+0000 7f55e19f5740 1 --2- 172.21.15.131:0/2828536254 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a2d5d2a710 0x55a2d5d26030 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f55d000c700 tx=0x7f55d0058f40 comp rx=0 tx=0).stop 2024-05-26T22:53:10.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.696+0000 7f55e19f5740 1 -- 172.21.15.131:0/2828536254 shutdown_connections 2024-05-26T22:53:10.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.696+0000 7f55e19f5740 1 --2- 172.21.15.131:0/2828536254 >> [v2:172.21.15.187:6832/3680999672,v1:172.21.15.187:6834/3680999672] conn(0x55a2d5d288d0 0x55a2d5d55e10 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:10.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.696+0000 7f55e19f5740 1 --2- 172.21.15.131:0/2828536254 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a2d5d2a710 0x55a2d5d26030 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:10.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.696+0000 7f55e19f5740 1 --2- 172.21.15.131:0/2828536254 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a2d5d44c70 0x55a2d5d2a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:10.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.696+0000 7f55e19f5740 1 --2- 172.21.15.131:0/2828536254 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a2d5d27d90 0x55a2d5d23140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:10.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.696+0000 7f55c4c00640 0 client.7134 ms_handle_reset on v2:172.21.15.187:6832/3680999672 2024-05-26T22:53:10.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.696+0000 7f55e19f5740 1 -- 172.21.15.131:0/2828536254 >> 172.21.15.131:0/2828536254 conn(0x55a2d5d27650 msgr2=0x55a2d5cfbe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:10.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.696+0000 7f55e19f5740 1 -- 172.21.15.131:0/2828536254 shutdown_connections 2024-05-26T22:53:10.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.696+0000 7f55e19f5740 1 -- 172.21.15.131:0/2828536254 wait complete. 2024-05-26T22:53:10.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.696+0000 7f55e19f5740 20 client.7134 trim_cache size 0 max 16384 2024-05-26T22:53:10.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.715+0000 7f55e19f5740 1 Processor -- start 2024-05-26T22:53:10.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.716+0000 7f55e19f5740 1 -- start start 2024-05-26T22:53:10.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.716+0000 7f55e19f5740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a2d5d27d90 0x55a2d5d51b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:10.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.716+0000 7f55e19f5740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a2d5d2a710 0x55a2d5d26f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:10.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.716+0000 7f55e19f5740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a2d5d44c70 0x55a2d5d54a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:10.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.716+0000 7f55e19f5740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f55d81ffc60 con 0x55a2d5d44c70 2024-05-26T22:53:10.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.716+0000 7f55e19f5740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f55d82001f0 con 0x55a2d5d27d90 2024-05-26T22:53:10.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.716+0000 7f55e19f5740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f55d80cd550 con 0x55a2d5d2a710 2024-05-26T22:53:10.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.717+0000 7f55dcc00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a2d5d44c70 0x55a2d5d54a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:10.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.717+0000 7f55dcc00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a2d5d44c70 0x55a2d5d54a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40826/0 (socket says 172.21.15.131:40826) 2024-05-26T22:53:10.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.717+0000 7f55dcc00640 1 -- 172.21.15.131:0/450805165 learned_addr learned my addr 172.21.15.131:0/450805165 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:53:10.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.717+0000 7f55d7e00640 1 --2- 172.21.15.131:0/450805165 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a2d5d27d90 0x55a2d5d51b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:10.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.717+0000 7f55dcc00640 1 -- 172.21.15.131:0/450805165 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a2d5d2a710 msgr2=0x55a2d5d26f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:10.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.717+0000 7f55cf400640 1 --2- 172.21.15.131:0/450805165 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a2d5d2a710 0x55a2d5d26f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:10.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.717+0000 7f55dcc00640 1 --2- 172.21.15.131:0/450805165 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a2d5d2a710 0x55a2d5d26f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:10.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.717+0000 7f55dcc00640 1 -- 172.21.15.131:0/450805165 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a2d5d27d90 msgr2=0x55a2d5d51b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:10.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.717+0000 7f55dcc00640 1 --2- 172.21.15.131:0/450805165 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a2d5d27d90 0x55a2d5d51b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:10.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.717+0000 7f55dcc00640 1 -- 172.21.15.131:0/450805165 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2d5c6d470 con 0x55a2d5d44c70 2024-05-26T22:53:10.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.717+0000 7f55dcc00640 1 --2- 172.21.15.131:0/450805165 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a2d5d44c70 0x55a2d5d54a20 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f55d00569b0 tx=0x7f55d008a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef88a8492ba1c7f0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:10.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.718+0000 7f55c7400640 1 -- 172.21.15.131:0/450805165 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d008a0c0 con 0x55a2d5d44c70 2024-05-26T22:53:10.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.718+0000 7f55c7400640 1 -- 172.21.15.131:0/450805165 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55d008a2a0 con 0x55a2d5d44c70 2024-05-26T22:53:10.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.718+0000 7f55c7400640 1 -- 172.21.15.131:0/450805165 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d0057d90 con 0x55a2d5d44c70 2024-05-26T22:53:10.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.718+0000 7f55e19f5740 1 -- 172.21.15.131:0/450805165 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a2d5d44c70 msgr2=0x55a2d5d54a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:10.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.718+0000 7f55e19f5740 1 --2- 172.21.15.131:0/450805165 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a2d5d44c70 0x55a2d5d54a20 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f55d00569b0 tx=0x7f55d008a050 comp rx=0 tx=0).stop 2024-05-26T22:53:10.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.718+0000 7f55e19f5740 1 -- 172.21.15.131:0/450805165 shutdown_connections 2024-05-26T22:53:10.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.718+0000 7f55e19f5740 1 --2- 172.21.15.131:0/450805165 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a2d5d2a710 0x55a2d5d26f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:10.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.718+0000 7f55e19f5740 1 --2- 172.21.15.131:0/450805165 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a2d5d44c70 0x55a2d5d54a20 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:10.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.718+0000 7f55e19f5740 1 --2- 172.21.15.131:0/450805165 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a2d5d27d90 0x55a2d5d51b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:10.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.718+0000 7f55e19f5740 1 -- 172.21.15.131:0/450805165 >> 172.21.15.131:0/450805165 conn(0x55a2d5d27650 msgr2=0x55a2d5d250d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:10.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.718+0000 7f55e19f5740 1 -- 172.21.15.131:0/450805165 shutdown_connections 2024-05-26T22:53:10.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.718+0000 7f55e19f5740 1 -- 172.21.15.131:0/450805165 wait complete. 2024-05-26T22:53:10.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.719+0000 7f55e19f5740 1 -- start start 2024-05-26T22:53:10.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.719+0000 7f55e19f5740 1 Processor -- start 2024-05-26T22:53:10.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.720+0000 7f55e19f5740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a2d5d27d90 0x55a2d5dc0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:10.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.720+0000 7f55e19f5740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a2d5d2a710 0x55a2d5cfbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:10.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.720+0000 7f55e19f5740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a2d5d44c70 0x55a2d5d29b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:10.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.720+0000 7f55e19f5740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55a2d5d28750 con 0x55a2d5d27d90 2024-05-26T22:53:10.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.720+0000 7f55e19f5740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55a2d5d42dc0 con 0x55a2d5d2a710 2024-05-26T22:53:10.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.720+0000 7f55e19f5740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x55a2d5d28da0 con 0x55a2d5d44c70 2024-05-26T22:53:10.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.720+0000 7f55e19f5740 10 client.0 _ll_register_callbacks cb 0x7f55d00549a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-26T22:53:10.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.720+0000 7f55d7e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a2d5d27d90 0x55a2d5dc0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:10.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.720+0000 7f55d7e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a2d5d27d90 0x55a2d5dc0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40840/0 (socket says 172.21.15.131:40840) 2024-05-26T22:53:10.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.720+0000 7f55d7e00640 1 -- 172.21.15.131:0/839573543 learned_addr learned my addr 172.21.15.131:0/839573543 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:53:10.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.721+0000 7f55d7e00640 1 -- 172.21.15.131:0/839573543 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a2d5d44c70 msgr2=0x55a2d5d29b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:10.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.721+0000 7f55d7e00640 1 --2- 172.21.15.131:0/839573543 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a2d5d44c70 0x55a2d5d29b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:10.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.721+0000 7f55d7e00640 1 -- 172.21.15.131:0/839573543 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a2d5d2a710 msgr2=0x55a2d5cfbcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:10.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.721+0000 7f55d7e00640 1 --2- 172.21.15.131:0/839573543 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a2d5d2a710 0x55a2d5cfbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:10.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.721+0000 7f55d7e00640 1 -- 172.21.15.131:0/839573543 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2d5d388c0 con 0x55a2d5d27d90 2024-05-26T22:53:10.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.721+0000 7f55d7e00640 1 --2- 172.21.15.131:0/839573543 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a2d5d27d90 0x55a2d5dc0bd0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f55c80097a0 tx=0x7f55c8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b0f3cb59e7a129d server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:10.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.721+0000 7f55c4c00640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-26T22:53:10.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.721+0000 7f55c4c00640 1 -- 172.21.15.131:0/839573543 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c8051070 con 0x55a2d5d27d90 2024-05-26T22:53:10.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.721+0000 7f55c4c00640 1 -- 172.21.15.131:0/839573543 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(71..71 src has 1..71) ==== 5767+0+0 (secure 0 0 0) 0x7f55c8004230 con 0x55a2d5d27d90 2024-05-26T22:53:10.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.721+0000 7f55e19f5740 1 -- 172.21.15.131:0/839573543 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55d008a2a0 con 0x55a2d5d27d90 2024-05-26T22:53:10.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.722+0000 7f55c4c00640 1 -- 172.21.15.131:0/839573543 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7f55c8058050 con 0x55a2d5d27d90 2024-05-26T22:53:10.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.722+0000 7f55e19f5740 10 client.7140 fetch_fsmap learned FSMap version 68 2024-05-26T22:53:10.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.722+0000 7f55e19f5740 1 -- 172.21.15.131:0/839573543 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) -- 0x55a2d5cfb740 con 0x55a2d5d27d90 2024-05-26T22:53:10.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.722+0000 7f55c4c00640 1 -- 172.21.15.131:0/839573543 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55d008a2a0 con 0x55a2d5d27d90 2024-05-26T22:53:10.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.722+0000 7f55c4c00640 1 -- 172.21.15.131:0/839573543 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 68) ==== 3534+0+0 (secure 0 0 0) 0x7f55c805b020 con 0x55a2d5d27d90 2024-05-26T22:53:10.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.722+0000 7f55e19f5740 10 client.7140 fetch_fsmap finished waiting for FSMap version 68 2024-05-26T22:53:10.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.722+0000 7f55e19f5740 10 client.7140 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-26T22:53:10.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.722+0000 7f55e19f5740 20 client.7140 populate_metadata read hostname 'smithi131' 2024-05-26T22:53:10.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.722+0000 7f55e19f5740 1 --2- 172.21.15.131:0/839573543 >> [v2:172.21.15.187:6832/3680999672,v1:172.21.15.187:6834/3680999672] conn(0x55a2d5d45530 0x55a2d5d45910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:10.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.722+0000 7f55e19f5740 4 client.7140 mds_command: new command op to 6876 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-26T22:53:10.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.722+0000 7f55e19f5740 1 -- 172.21.15.131:0/839573543 --> [v2:172.21.15.187:6832/3680999672,v1:172.21.15.187:6834/3680999672] -- command(tid 0: {"prefix": "status"}) -- 0x55a2d5cf8f50 con 0x55a2d5d45530 2024-05-26T22:53:10.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.723+0000 7f55cf400640 1 --2- 172.21.15.131:0/839573543 >> [v2:172.21.15.187:6832/3680999672,v1:172.21.15.187:6834/3680999672] conn(0x55a2d5d45530 0x55a2d5d45910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:10.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.724+0000 7f55cf400640 1 --2- 172.21.15.131:0/839573543 >> [v2:172.21.15.187:6832/3680999672,v1:172.21.15.187:6834/3680999672] conn(0x55a2d5d45530 0x55a2d5d45910 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c19b412e6957d9f server_cookie=6afb275071b944c0 in_seq=0 out_seq=0 2024-05-26T22:53:10.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.724+0000 7f55c4c00640 10 client.7140 ms_handle_connect on v2:172.21.15.187:6832/3680999672 2024-05-26T22:53:10.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.724+0000 7f55c4c00640 1 -- 172.21.15.131:0/839573543 <== mds.0 v2:172.21.15.187:6832/3680999672 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x55a2d5cf8f50 con 0x55a2d5d45530 2024-05-26T22:53:10.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.724+0000 7f55c4c00640 10 client.7140 handle_command_reply: tid=0 2024-05-26T22:53:10.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.724+0000 7f55e19f5740 1 client.7140 shutdown 2024-05-26T22:53:10.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.724+0000 7f55e19f5740 1 -- 172.21.15.131:0/839573543 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a2d5d27d90 msgr2=0x55a2d5dc0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:10.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.724+0000 7f55e19f5740 1 --2- 172.21.15.131:0/839573543 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a2d5d27d90 0x55a2d5dc0bd0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f55c80097a0 tx=0x7f55c8004730 comp rx=0 tx=0).stop 2024-05-26T22:53:10.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.724+0000 7f55e19f5740 1 -- 172.21.15.131:0/839573543 shutdown_connections 2024-05-26T22:53:10.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.724+0000 7f55e19f5740 1 --2- 172.21.15.131:0/839573543 >> [v2:172.21.15.187:6832/3680999672,v1:172.21.15.187:6834/3680999672] conn(0x55a2d5d45530 0x55a2d5d45910 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:10.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.724+0000 7f55e19f5740 1 --2- 172.21.15.131:0/839573543 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a2d5d44c70 0x55a2d5d29b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:10.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.724+0000 7f55e19f5740 1 --2- 172.21.15.131:0/839573543 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a2d5d2a710 0x55a2d5cfbcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:10.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.724+0000 7f55c4c00640 0 client.7140 ms_handle_reset on v2:172.21.15.187:6832/3680999672 2024-05-26T22:53:10.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.724+0000 7f55e19f5740 1 --2- 172.21.15.131:0/839573543 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a2d5d27d90 0x55a2d5dc0bd0 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:10.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.724+0000 7f55e19f5740 1 -- 172.21.15.131:0/839573543 >> 172.21.15.131:0/839573543 conn(0x55a2d5d27650 msgr2=0x55a2d5cf6920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:10.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.724+0000 7f55e19f5740 1 -- 172.21.15.131:0/839573543 shutdown_connections 2024-05-26T22:53:10.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.724+0000 7f55e19f5740 1 -- 172.21.15.131:0/839573543 wait complete. 2024-05-26T22:53:10.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.725+0000 7f55e19f5740 20 client.7140 trim_cache size 0 max 16384 2024-05-26T22:53:10.729 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-26T22:53:10.729 INFO:teuthology.orchestra.run.smithi131.stdout: "cluster_fsid": "46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f", 2024-05-26T22:53:10.729 INFO:teuthology.orchestra.run.smithi131.stdout: "whoami": 0, 2024-05-26T22:53:10.729 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 6876, 2024-05-26T22:53:10.729 INFO:teuthology.orchestra.run.smithi131.stdout: "want_state": "up:active", 2024-05-26T22:53:10.729 INFO:teuthology.orchestra.run.smithi131.stdout: "state": "up:active", 2024-05-26T22:53:10.729 INFO:teuthology.orchestra.run.smithi131.stdout: "fs_name": "cephfs", 2024-05-26T22:53:10.729 INFO:teuthology.orchestra.run.smithi131.stdout: "rank_uptime": 16.417245849, 2024-05-26T22:53:10.729 INFO:teuthology.orchestra.run.smithi131.stdout: "mdsmap_epoch": 68, 2024-05-26T22:53:10.730 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch": 71, 2024-05-26T22:53:10.730 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch_barrier": 71, 2024-05-26T22:53:10.730 INFO:teuthology.orchestra.run.smithi131.stdout: "uptime": 21.844927609999999 2024-05-26T22:53:10.730 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-26T22:53:10.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.725+0000 7f55dec00640 1 -- 172.21.15.131:0/3868925746 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f55b406be30 msgr2=0x7f55b406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:10.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.725+0000 7f55dec00640 1 --2- 172.21.15.131:0/3868925746 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f55b406be30 0x7f55b406e2f0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f55c8034000 tx=0x7f55c802e000 comp rx=0 tx=0).stop 2024-05-26T22:53:10.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.725+0000 7f55dec00640 1 -- 172.21.15.131:0/3868925746 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55d80cfe50 msgr2=0x7f55d8160c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:10.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.725+0000 7f55dec00640 1 --2- 172.21.15.131:0/3868925746 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55d80cfe50 0x7f55d8160c10 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f55d000c730 tx=0x7f55d000cbf0 comp rx=0 tx=0).stop 2024-05-26T22:53:10.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.725+0000 7f55dec00640 1 -- 172.21.15.131:0/3868925746 shutdown_connections 2024-05-26T22:53:10.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.725+0000 7f55dec00640 1 --2- 172.21.15.131:0/3868925746 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55d8072540 0x7f55d80ce720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:10.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.725+0000 7f55dec00640 1 --2- 172.21.15.131:0/3868925746 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f55b406be30 0x7f55b406e2f0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:10.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.725+0000 7f55dec00640 1 --2- 172.21.15.131:0/3868925746 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55d80cfe50 0x7f55d8160c10 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:10.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.725+0000 7f55dec00640 1 --2- 172.21.15.131:0/3868925746 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f55d806f180 0x7f55d81518c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:10.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.725+0000 7f55dec00640 1 -- 172.21.15.131:0/3868925746 >> 172.21.15.131:0/3868925746 conn(0x7f55d806bc90 msgr2=0x7f55d815c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:10.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.725+0000 7f55dec00640 1 -- 172.21.15.131:0/3868925746 shutdown_connections 2024-05-26T22:53:10.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10.725+0000 7f55dec00640 1 -- 172.21.15.131:0/3868925746 wait complete. 2024-05-26T22:53:10.742 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-05-26T22:53:10.742 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-26T22:53:10.742 DEBUG:teuthology.orchestra.run.smithi131:> mkdir -p tree 2024-05-26T22:53:10.742 DEBUG:teuthology.orchestra.run.smithi131:> 2024-05-26T22:53:10.742 DEBUG:teuthology.orchestra.run.smithi131:> setfattr -n ceph.dir.pin.distributed -v 1 tree 2024-05-26T22:53:10.742 DEBUG:teuthology.orchestra.run.smithi131:> 2024-05-26T22:53:10.742 DEBUG:teuthology.orchestra.run.smithi131:> for ((i = 0; i < 100; i++)); do 2024-05-26T22:53:10.742 DEBUG:teuthology.orchestra.run.smithi131:> mkdir -p "tree/$i" 2024-05-26T22:53:10.742 DEBUG:teuthology.orchestra.run.smithi131:> echo file > "tree/$i/file" 2024-05-26T22:53:10.742 DEBUG:teuthology.orchestra.run.smithi131:> done 2024-05-26T22:53:10.743 DEBUG:teuthology.orchestra.run.smithi131:> ') 2024-05-26T22:53:10.840 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:10 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-26T22:53:10.842 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree 2024-05-26T22:53:10.848 INFO:teuthology.orchestra.run.smithi131.stderr:+ setfattr -n ceph.dir.pin.distributed -v 1 tree 2024-05-26T22:53:10.858 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i = 0 )) 2024-05-26T22:53:10.858 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:53:10.858 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/0 2024-05-26T22:53:10.863 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:53:10.868 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:53:10.868 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:53:10.868 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/1 2024-05-26T22:53:10.873 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:53:10.876 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:53:10.876 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:53:10.876 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/2 2024-05-26T22:53:10.880 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:53:10.883 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:53:10.883 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:53:10.883 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/3 2024-05-26T22:53:10.887 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:53:10.890 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:53:10.890 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:53:10.890 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/4 2024-05-26T22:53:10.894 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:53:10.897 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:53:10.897 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:53:10.897 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/5 2024-05-26T22:53:10.902 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:53:10.905 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:53:10.905 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:53:10.905 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/6 2024-05-26T22:53:10.910 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:53:10.912 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:53:10.912 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:53:10.912 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/7 2024-05-26T22:53:10.918 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:53:10.920 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:53:10.920 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:53:10.920 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/8 2024-05-26T22:53:10.925 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:53:10.927 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:53:10.927 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:53:10.927 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/9 2024-05-26T22:53:10.933 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:53:10.935 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:53:10.935 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:53:10.935 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/10 2024-05-26T22:53:10.940 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:53:10.943 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:53:10.943 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:53:10.943 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/11 2024-05-26T22:53:10.951 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:53:10.955 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:53:10.955 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:53:10.955 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/12 2024-05-26T22:53:10.961 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:53:10.965 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:53:10.965 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:53:10.965 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/13 2024-05-26T22:53:10.972 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:53:10.975 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:53:10.975 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:53:10.975 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/14 2024-05-26T22:53:10.982 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:53:10.985 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:53:10.986 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:53:10.986 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/15 2024-05-26T22:53:10.993 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:53:10.996 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:53:10.996 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:53:10.996 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/16 2024-05-26T22:53:11.003 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:53:11.009 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:53:11.009 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:53:11.009 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/17 2024-05-26T22:53:11.014 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:53:11.017 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:53:11.017 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:53:11.017 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/18 2024-05-26T22:53:11.025 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:53:11.028 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:53:11.028 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:53:11.028 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/19 2024-05-26T22:53:11.035 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:53:11.038 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:53:11.038 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:53:11.038 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/20 2024-05-26T22:53:11.045 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:53:11.049 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:53:11.049 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:53:11.049 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/21 2024-05-26T22:53:11.056 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:53:11.060 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:53:11.060 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:53:11.060 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/22 2024-05-26T22:53:11.067 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:53:11.070 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:53:11.070 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:53:11.070 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/23 2024-05-26T22:53:11.078 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:53:11.081 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:53:11.081 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:53:11.081 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/24 2024-05-26T22:53:11.088 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:53:11.092 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:53:11.093 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:53:11.093 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/25 2024-05-26T22:53:11.099 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:53:11.102 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:53:11.102 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:53:11.103 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/26 2024-05-26T22:53:11.109 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:53:11.113 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:53:11.113 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:53:11.113 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/27 2024-05-26T22:53:11.120 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:53:11.123 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:53:11.123 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:53:11.124 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/28 2024-05-26T22:53:11.130 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:53:11.134 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:53:11.134 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:53:11.134 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/29 2024-05-26T22:53:11.141 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:53:11.144 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:53:11.144 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:53:11.144 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/30 2024-05-26T22:53:11.152 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:53:11.155 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:53:11.156 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:53:11.156 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/31 2024-05-26T22:53:11.163 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:53:11.166 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:53:11.166 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:53:11.166 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/32 2024-05-26T22:53:11.173 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:53:11.177 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:53:11.177 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:53:11.177 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/33 2024-05-26T22:53:11.184 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:53:11.187 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:53:11.187 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:53:11.187 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/34 2024-05-26T22:53:11.194 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:53:11.198 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:53:11.198 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:53:11.198 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/35 2024-05-26T22:53:11.205 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:53:11.208 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:53:11.209 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:53:11.209 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/36 2024-05-26T22:53:11.216 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:53:11.219 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:53:11.219 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:53:11.220 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/37 2024-05-26T22:53:11.226 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:53:11.230 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:53:11.230 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:53:11.230 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/38 2024-05-26T22:53:11.237 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:53:11.241 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:53:11.241 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:53:11.241 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/39 2024-05-26T22:53:11.248 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:53:11.251 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:53:11.252 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:53:11.252 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/40 2024-05-26T22:53:11.259 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:53:11.262 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:53:11.262 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:53:11.263 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/41 2024-05-26T22:53:11.269 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:53:11.273 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:53:11.273 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:53:11.273 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/42 2024-05-26T22:53:11.281 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:53:11.284 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:53:11.284 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:53:11.284 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/43 2024-05-26T22:53:11.291 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:53:11.295 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:53:11.295 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:53:11.295 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/44 2024-05-26T22:53:11.302 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:53:11.305 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:53:11.306 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:53:11.306 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/45 2024-05-26T22:53:11.313 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:53:11.316 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:53:11.316 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:53:11.316 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/46 2024-05-26T22:53:11.324 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:53:11.327 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:53:11.327 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:53:11.327 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/47 2024-05-26T22:53:11.334 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:53:11.338 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:53:11.338 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:53:11.338 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/48 2024-05-26T22:53:11.345 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:53:11.348 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:53:11.348 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:53:11.348 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/49 2024-05-26T22:53:11.356 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:53:11.359 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:53:11.359 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:53:11.359 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/50 2024-05-26T22:53:11.367 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:53:11.369 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:53:11.369 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:53:11.369 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/51 2024-05-26T22:53:11.375 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:53:11.377 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:53:11.377 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:53:11.378 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/52 2024-05-26T22:53:11.383 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:53:11.385 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:53:11.385 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:53:11.385 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/53 2024-05-26T22:53:11.392 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:53:11.394 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:53:11.394 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:53:11.394 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/54 2024-05-26T22:53:11.400 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:53:11.402 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:53:11.403 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:53:11.403 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/55 2024-05-26T22:53:11.409 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:53:11.411 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:53:11.411 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:53:11.412 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/56 2024-05-26T22:53:11.417 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:53:11.420 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:53:11.420 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:53:11.420 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/57 2024-05-26T22:53:11.426 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:53:11.430 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:53:11.430 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:53:11.430 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/58 2024-05-26T22:53:11.434 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:53:11.437 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:53:11.437 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:53:11.437 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/59 2024-05-26T22:53:11.443 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:53:11.445 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:53:11.445 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:53:11.446 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/60 2024-05-26T22:53:11.452 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:53:11.455 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:53:11.455 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:53:11.455 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/61 2024-05-26T22:53:11.461 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:53:11.463 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:53:11.463 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:53:11.463 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/62 2024-05-26T22:53:11.469 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:53:11.471 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:53:11.471 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:53:11.472 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/63 2024-05-26T22:53:11.477 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:53:11.480 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:53:11.480 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:53:11.480 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/64 2024-05-26T22:53:11.486 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:53:11.489 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:53:11.489 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:53:11.489 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/65 2024-05-26T22:53:11.494 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:53:11.497 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:53:11.497 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:53:11.497 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/66 2024-05-26T22:53:11.503 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:53:11.505 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:53:11.505 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:53:11.505 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/67 2024-05-26T22:53:11.511 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:53:11.514 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:53:11.514 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:53:11.514 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/68 2024-05-26T22:53:11.519 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:53:11.521 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:53:11.522 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:53:11.522 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/69 2024-05-26T22:53:11.527 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:53:11.529 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:53:11.529 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:53:11.529 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/70 2024-05-26T22:53:11.535 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:53:11.537 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:53:11.537 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:53:11.537 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/71 2024-05-26T22:53:11.544 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:53:11.546 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:53:11.546 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:53:11.546 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/72 2024-05-26T22:53:11.552 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:53:11.555 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:53:11.555 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:53:11.555 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/73 2024-05-26T22:53:11.561 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:53:11.564 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:53:11.564 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:53:11.564 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/74 2024-05-26T22:53:11.570 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:53:11.572 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:53:11.572 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:53:11.572 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/75 2024-05-26T22:53:11.578 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:53:11.581 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:53:11.581 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:53:11.581 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/76 2024-05-26T22:53:11.587 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:53:11.590 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:53:11.590 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:53:11.590 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/77 2024-05-26T22:53:11.596 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:53:11.598 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:53:11.598 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:53:11.599 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/78 2024-05-26T22:53:11.605 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:53:11.609 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:53:11.609 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:53:11.609 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/79 2024-05-26T22:53:11.616 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:53:11.618 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:53:11.618 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:53:11.619 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/80 2024-05-26T22:53:11.625 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:53:11.628 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:53:11.628 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:53:11.628 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/81 2024-05-26T22:53:11.634 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:53:11.637 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:53:11.637 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:53:11.637 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/82 2024-05-26T22:53:11.643 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:53:11.646 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:53:11.646 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:53:11.646 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/83 2024-05-26T22:53:11.652 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:53:11.654 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:53:11.655 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:53:11.655 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/84 2024-05-26T22:53:11.660 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:53:11.663 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:53:11.663 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:53:11.663 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/85 2024-05-26T22:53:11.669 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:53:11.671 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:53:11.671 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:53:11.672 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/86 2024-05-26T22:53:11.677 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:53:11.680 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:53:11.680 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:53:11.680 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/87 2024-05-26T22:53:11.686 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:53:11.688 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:53:11.688 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:53:11.689 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/88 2024-05-26T22:53:11.694 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:53:11.697 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:53:11.697 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:53:11.697 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/89 2024-05-26T22:53:11.703 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:53:11.705 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:53:11.705 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:53:11.705 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/90 2024-05-26T22:53:11.711 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:53:11.714 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:53:11.714 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:53:11.714 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/91 2024-05-26T22:53:11.719 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:53:11.722 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:53:11.722 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:53:11.722 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/92 2024-05-26T22:53:11.728 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:53:11.731 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:53:11.731 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:53:11.731 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/93 2024-05-26T22:53:11.737 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:53:11.739 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:53:11.739 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:53:11.739 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/94 2024-05-26T22:53:11.745 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:53:11.748 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:53:11.748 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:53:11.748 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/95 2024-05-26T22:53:11.754 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:53:11.756 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:53:11.756 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:53:11.756 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/96 2024-05-26T22:53:11.763 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:53:11.765 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:53:11.765 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:53:11.766 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/97 2024-05-26T22:53:11.771 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:53:11.773 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:53:11.773 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:53:11.773 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/98 2024-05-26T22:53:11.778 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:53:11.781 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:53:11.781 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:53:11.781 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/99 2024-05-26T22:53:11.786 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:53:11.789 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:53:11.789 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:53:11.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-26T22:53:11.802 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-05-26T22:53:12.030 INFO:teuthology.orchestra.run.smithi131.stdout:[{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds2","dirfrag":"0x102","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"2","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-05-26T22:53:12.031 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "2", "dirfrag": "0x102", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds2", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-05-26T22:53:12.031 DEBUG:tasks.cephfs.cephfs_test_case:[ { "is_auth": true, "auth_first": 2, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "~mds2", "dirfrag": "0x102", "snapid_first": 2, "projected_version": "1", "version": "1", "committing_version": "1", "committed_version": "1", "is_rep": false, "dir_auth": "2", "states": [ "auth", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "dirty": 0, "authpin": 0 }, "nref": 2 } } ] 2024-05-26T22:53:12.031 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-05-26T22:53:12.257 INFO:teuthology.orchestra.run.smithi131.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-05-26T22:53:12.257 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-05-26T22:53:12.258 DEBUG:tasks.cephfs.cephfs_test_case:[ { "is_auth": true, "auth_first": 1, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "~mds1", "dirfrag": "0x101", "snapid_first": 2, "projected_version": "1", "version": "1", "committing_version": "1", "committed_version": "1", "is_rep": false, "dir_auth": "1", "states": [ "auth", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "dirty": 0, "authpin": 0 }, "nref": 2 } } ] 2024-05-26T22:53:12.258 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json get subtrees 2024-05-26T22:53:12.488 INFO:teuthology.orchestra.run.smithi187.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"1255","version":"1053","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":101,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"997","version":"997","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":1,"authpin":0},"nref":3}}] 2024-05-26T22:53:12.489 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 101, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "1255", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "1053" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 1, "subtree": 1 }, "projected_version": "997", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "997" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-05-26T22:53:12.489 DEBUG:tasks.cephfs.cephfs_test_case:[ { "is_auth": true, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "", "dirfrag": "0x1", "snapid_first": 2, "projected_version": "1255", "version": "1053", "committing_version": "1", "committed_version": "1", "is_rep": true, "dir_auth": "0", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 101, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "replicated": 0, "dirty": 1, "authpin": 1 }, "nref": 4 } }, { "is_auth": true, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "~mds0", "dirfrag": "0x100", "snapid_first": 2, "projected_version": "997", "version": "997", "committing_version": "1", "committed_version": "1", "is_rep": false, "dir_auth": "0", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "dirty": 1, "authpin": 0 }, "nref": 3 } } ] 2024-05-26T22:53:12.489 INFO:tasks.cephfs.cephfs_test_case:len=0 [] 2024-05-26T22:53:15.100 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-26T22:53:15.109 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-26T22:53:17.490 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-05-26T22:53:17.720 INFO:teuthology.orchestra.run.smithi131.stdout:[{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds2","dirfrag":"0x102","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"2","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-05-26T22:53:17.720 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "2", "dirfrag": "0x102", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds2", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-05-26T22:53:17.721 DEBUG:tasks.cephfs.cephfs_test_case:[ { "is_auth": true, "auth_first": 2, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "~mds2", "dirfrag": "0x102", "snapid_first": 2, "projected_version": "1", "version": "1", "committing_version": "1", "committed_version": "1", "is_rep": false, "dir_auth": "2", "states": [ "auth", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "dirty": 0, "authpin": 0 }, "nref": 2 } } ] 2024-05-26T22:53:17.721 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-05-26T22:53:17.944 INFO:teuthology.orchestra.run.smithi131.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-05-26T22:53:17.945 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-05-26T22:53:17.945 DEBUG:tasks.cephfs.cephfs_test_case:[ { "is_auth": true, "auth_first": 1, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "~mds1", "dirfrag": "0x101", "snapid_first": 2, "projected_version": "1", "version": "1", "committing_version": "1", "committed_version": "1", "is_rep": false, "dir_auth": "1", "states": [ "auth", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "dirty": 0, "authpin": 0 }, "nref": 2 } } ] 2024-05-26T22:53:17.945 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json get subtrees 2024-05-26T22:53:18.175 INFO:teuthology.orchestra.run.smithi187.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"1258","version":"1258","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"999","version":"999","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":1,"authpin":0},"nref":3}}] 2024-05-26T22:53:18.175 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "1258", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "1258" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 1, "subtree": 1 }, "projected_version": "999", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "999" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-05-26T22:53:18.176 DEBUG:tasks.cephfs.cephfs_test_case:[ { "is_auth": true, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "", "dirfrag": "0x1", "snapid_first": 2, "projected_version": "1258", "version": "1258", "committing_version": "1", "committed_version": "1", "is_rep": true, "dir_auth": "0", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "replicated": 0, "dirty": 1, "authpin": 0 }, "nref": 3 } }, { "is_auth": true, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "~mds0", "dirfrag": "0x100", "snapid_first": 2, "projected_version": "999", "version": "999", "committing_version": "1", "committed_version": "1", "is_rep": false, "dir_auth": "0", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "dirty": 1, "authpin": 0 }, "nref": 3 } } ] 2024-05-26T22:53:18.176 INFO:tasks.cephfs.cephfs_test_case:len=0 [] 2024-05-26T22:53:23.177 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-05-26T22:53:23.411 INFO:teuthology.orchestra.run.smithi131.stdout:[{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds2","dirfrag":"0x102","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"2","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-05-26T22:53:23.412 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "2", "dirfrag": "0x102", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds2", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-05-26T22:53:23.412 DEBUG:tasks.cephfs.cephfs_test_case:[ { "is_auth": true, "auth_first": 2, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "~mds2", "dirfrag": "0x102", "snapid_first": 2, "projected_version": "1", "version": "1", "committing_version": "1", "committed_version": "1", "is_rep": false, "dir_auth": "2", "states": [ "auth", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "dirty": 0, "authpin": 0 }, "nref": 2 } } ] 2024-05-26T22:53:23.412 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-05-26T22:53:23.636 INFO:teuthology.orchestra.run.smithi131.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-05-26T22:53:23.636 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-05-26T22:53:23.636 DEBUG:tasks.cephfs.cephfs_test_case:[ { "is_auth": true, "auth_first": 1, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "~mds1", "dirfrag": "0x101", "snapid_first": 2, "projected_version": "1", "version": "1", "committing_version": "1", "committed_version": "1", "is_rep": false, "dir_auth": "1", "states": [ "auth", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "dirty": 0, "authpin": 0 }, "nref": 2 } } ] 2024-05-26T22:53:23.636 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json get subtrees 2024-05-26T22:53:23.861 INFO:teuthology.orchestra.run.smithi187.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"1262","version":"1262","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"999","version":"999","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":1,"authpin":0},"nref":3}}] 2024-05-26T22:53:23.862 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "1262", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "1262" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 1, "subtree": 1 }, "projected_version": "999", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "999" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-05-26T22:53:23.862 DEBUG:tasks.cephfs.cephfs_test_case:[ { "is_auth": true, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "", "dirfrag": "0x1", "snapid_first": 2, "projected_version": "1262", "version": "1262", "committing_version": "1", "committed_version": "1", "is_rep": true, "dir_auth": "0", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "replicated": 0, "dirty": 1, "authpin": 0 }, "nref": 3 } }, { "is_auth": true, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "~mds0", "dirfrag": "0x100", "snapid_first": 2, "projected_version": "999", "version": "999", "committing_version": "1", "committed_version": "1", "is_rep": false, "dir_auth": "0", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "dirty": 1, "authpin": 0 }, "nref": 3 } } ] 2024-05-26T22:53:23.862 INFO:tasks.cephfs.cephfs_test_case:len=0 [] 2024-05-26T22:53:28.863 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-05-26T22:53:29.091 INFO:teuthology.orchestra.run.smithi131.stdout:[{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds2","dirfrag":"0x102","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"2","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"1262","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":3},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/tree","dirfrag":"0x100000002f4.101*","snapid_first":2,"projected_version":"602","version":"602","committing_version":"602","committed_version":"602","is_rep":false,"dir_auth":"2","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1},"nref":3}},{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/tree","dirfrag":"0x100000002f4.010*","snapid_first":2,"projected_version":"602","version":"602","committing_version":"602","committed_version":"602","is_rep":false,"dir_auth":"2","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1},"nref":3}},{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/tree","dirfrag":"0x100000002f4.001*","snapid_first":2,"projected_version":"602","version":"602","committing_version":"602","committed_version":"602","is_rep":false,"dir_auth":"2","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1},"nref":3}}] 2024-05-26T22:53:29.092 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "2", "dirfrag": "0x102", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds2", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 3 }, "snapid_first": 2, "states": [], "version": "1262" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "602", "committing_version": "602", "dir_auth": "2", "dirfrag": "0x100000002f4.101*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "/tree", "pins": { "child": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "602", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "602" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 2, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "602", "committing_version": "602", "dir_auth": "2", "dirfrag": "0x100000002f4.010*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "/tree", "pins": { "child": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "602", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "602" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 2, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "602", "committing_version": "602", "dir_auth": "2", "dirfrag": "0x100000002f4.001*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "/tree", "pins": { "child": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "602", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "602" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 2, "is_auth": true, "random_ephemeral_pin": false } ] 2024-05-26T22:53:29.092 DEBUG:tasks.cephfs.cephfs_test_case:[ { "is_auth": true, "auth_first": 2, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "~mds2", "dirfrag": "0x102", "snapid_first": 2, "projected_version": "1", "version": "1", "committing_version": "1", "committed_version": "1", "is_rep": false, "dir_auth": "2", "states": [ "auth", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "dirty": 0, "authpin": 0 }, "nref": 2 } }, { "is_auth": false, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "", "dirfrag": "0x1", "snapid_first": 2, "projected_version": "0", "version": "1262", "committing_version": "0", "committed_version": "0", "is_rep": true, "dir_auth": "0", "states": [], "is_auth": false, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 3 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1 }, "nref": 2 } }, { "is_auth": true, "auth_first": 2, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 2, "dir": { "path": "/tree", "dirfrag": "0x100000002f4.101*", "snapid_first": 2, "projected_version": "602", "version": "602", "committing_version": "602", "committed_version": "602", "is_rep": false, "dir_auth": "2", "states": [ "auth", "complete" ], "is_auth": true, "auth_state": { "replicas": { "0": 1 } }, "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "child": 1, "frozen": 0, "subtree": 1, "importing": 0, "replicated": 1 }, "nref": 3 } }, { "is_auth": true, "auth_first": 2, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 2, "dir": { "path": "/tree", "dirfrag": "0x100000002f4.010*", "snapid_first": 2, "projected_version": "602", "version": "602", "committing_version": "602", "committed_version": "602", "is_rep": false, "dir_auth": "2", "states": [ "auth", "complete" ], "is_auth": true, "auth_state": { "replicas": { "0": 1 } }, "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "child": 1, "frozen": 0, "subtree": 1, "importing": 0, "replicated": 1 }, "nref": 3 } }, { "is_auth": true, "auth_first": 2, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 2, "dir": { "path": "/tree", "dirfrag": "0x100000002f4.001*", "snapid_first": 2, "projected_version": "602", "version": "602", "committing_version": "602", "committed_version": "602", "is_rep": false, "dir_auth": "2", "states": [ "auth", "complete" ], "is_auth": true, "auth_state": { "replicas": { "0": 1 } }, "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "child": 1, "frozen": 0, "subtree": 1, "importing": 0, "replicated": 1 }, "nref": 3 } } ] 2024-05-26T22:53:29.092 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-05-26T22:53:29.318 INFO:teuthology.orchestra.run.smithi131.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"1262","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":3},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree","dirfrag":"0x100000002f4.110*","snapid_first":2,"projected_version":"602","version":"602","committing_version":"602","committed_version":"602","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1},"nref":3}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree","dirfrag":"0x100000002f4.100*","snapid_first":2,"projected_version":"602","version":"602","committing_version":"602","committed_version":"602","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1},"nref":3}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree","dirfrag":"0x100000002f4.011*","snapid_first":2,"projected_version":"602","version":"602","committing_version":"602","committed_version":"602","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1},"nref":3}}] 2024-05-26T22:53:29.319 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 3 }, "snapid_first": 2, "states": [], "version": "1262" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "602", "committing_version": "602", "dir_auth": "1", "dirfrag": "0x100000002f4.110*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "/tree", "pins": { "child": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "602", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "602" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "602", "committing_version": "602", "dir_auth": "1", "dirfrag": "0x100000002f4.100*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "/tree", "pins": { "child": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "602", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "602" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "602", "committing_version": "602", "dir_auth": "1", "dirfrag": "0x100000002f4.011*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "/tree", "pins": { "child": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "602", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "602" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-05-26T22:53:29.319 DEBUG:tasks.cephfs.cephfs_test_case:[ { "is_auth": true, "auth_first": 1, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "~mds1", "dirfrag": "0x101", "snapid_first": 2, "projected_version": "1", "version": "1", "committing_version": "1", "committed_version": "1", "is_rep": false, "dir_auth": "1", "states": [ "auth", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "dirty": 0, "authpin": 0 }, "nref": 2 } }, { "is_auth": false, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "", "dirfrag": "0x1", "snapid_first": 2, "projected_version": "0", "version": "1262", "committing_version": "0", "committed_version": "0", "is_rep": true, "dir_auth": "0", "states": [], "is_auth": false, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 3 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1 }, "nref": 2 } }, { "is_auth": true, "auth_first": 1, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 1, "dir": { "path": "/tree", "dirfrag": "0x100000002f4.110*", "snapid_first": 2, "projected_version": "602", "version": "602", "committing_version": "602", "committed_version": "602", "is_rep": false, "dir_auth": "1", "states": [ "auth", "complete" ], "is_auth": true, "auth_state": { "replicas": { "0": 1 } }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "child": 1, "frozen": 0, "subtree": 1, "importing": 0, "replicated": 1 }, "nref": 3 } }, { "is_auth": true, "auth_first": 1, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 1, "dir": { "path": "/tree", "dirfrag": "0x100000002f4.100*", "snapid_first": 2, "projected_version": "602", "version": "602", "committing_version": "602", "committed_version": "602", "is_rep": false, "dir_auth": "1", "states": [ "auth", "complete" ], "is_auth": true, "auth_state": { "replicas": { "0": 1 } }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "child": 1, "frozen": 0, "subtree": 1, "importing": 0, "replicated": 1 }, "nref": 3 } }, { "is_auth": true, "auth_first": 1, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 1, "dir": { "path": "/tree", "dirfrag": "0x100000002f4.011*", "snapid_first": 2, "projected_version": "602", "version": "602", "committing_version": "602", "committed_version": "602", "is_rep": false, "dir_auth": "1", "states": [ "auth", "complete" ], "is_auth": true, "auth_state": { "replicas": { "0": 1 } }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "child": 1, "frozen": 0, "subtree": 1, "importing": 0, "replicated": 1 }, "nref": 3 } } ] 2024-05-26T22:53:29.320 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json get subtrees 2024-05-26T22:53:29.551 INFO:teuthology.orchestra.run.smithi187.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"1262","version":"1262","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":3,"2":3}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"999","version":"999","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/tree","dirfrag":"0x100000002f4.000*","snapid_first":2,"projected_version":"602","version":"602","committing_version":"602","committed_version":"602","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"frozen":0,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/tree","dirfrag":"0x100000002f4.001*","snapid_first":2,"projected_version":"602","version":"602","committing_version":"602","committed_version":"602","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/tree","dirfrag":"0x100000002f4.010*","snapid_first":2,"projected_version":"602","version":"602","committing_version":"602","committed_version":"602","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree","dirfrag":"0x100000002f4.011*","snapid_first":2,"projected_version":"602","version":"602","committing_version":"602","committed_version":"602","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree","dirfrag":"0x100000002f4.100*","snapid_first":2,"projected_version":"602","version":"602","committing_version":"602","committed_version":"602","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/tree","dirfrag":"0x100000002f4.101*","snapid_first":2,"projected_version":"602","version":"602","committing_version":"602","committed_version":"602","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree","dirfrag":"0x100000002f4.110*","snapid_first":2,"projected_version":"602","version":"602","committing_version":"602","committed_version":"602","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/tree","dirfrag":"0x100000002f4.111*","snapid_first":2,"projected_version":"602","version":"602","committing_version":"602","committed_version":"602","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"frozen":0,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-05-26T22:53:29.552 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 3, "2": 3 } }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "1262", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "1262" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 1, "subtree": 1 }, "projected_version": "999", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "999" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "602", "committing_version": "602", "dir_auth": "0", "dirfrag": "0x100000002f4.000*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/tree", "pins": { "authpin": 0, "child": 1, "dirty": 0, "frozen": 0, "subtree": 1 }, "projected_version": "602", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "602" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "602", "committing_version": "602", "dir_auth": "2", "dirfrag": "0x100000002f4.001*", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "602", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "602" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 2, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "602", "committing_version": "602", "dir_auth": "2", "dirfrag": "0x100000002f4.010*", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "602", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "602" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 2, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "602", "committing_version": "602", "dir_auth": "1", "dirfrag": "0x100000002f4.011*", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "602", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "602" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "602", "committing_version": "602", "dir_auth": "1", "dirfrag": "0x100000002f4.100*", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "602", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "602" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "602", "committing_version": "602", "dir_auth": "2", "dirfrag": "0x100000002f4.101*", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "602", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "602" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 2, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "602", "committing_version": "602", "dir_auth": "1", "dirfrag": "0x100000002f4.110*", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "602", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "602" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "602", "committing_version": "602", "dir_auth": "0", "dirfrag": "0x100000002f4.111*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/tree", "pins": { "authpin": 0, "child": 1, "dirty": 0, "frozen": 0, "subtree": 1 }, "projected_version": "602", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "602" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false } ] 2024-05-26T22:53:29.553 DEBUG:tasks.cephfs.cephfs_test_case:[ { "is_auth": true, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "", "dirfrag": "0x1", "snapid_first": 2, "projected_version": "1262", "version": "1262", "committing_version": "1", "committed_version": "1", "is_rep": true, "dir_auth": "0", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": { "1": 3, "2": 3 } }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "replicated": 1, "dirty": 1, "authpin": 0 }, "nref": 4 } }, { "is_auth": true, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "~mds0", "dirfrag": "0x100", "snapid_first": 2, "projected_version": "999", "version": "999", "committing_version": "1", "committed_version": "1", "is_rep": false, "dir_auth": "0", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "dirty": 1, "authpin": 0 }, "nref": 3 } }, { "is_auth": true, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 0, "dir": { "path": "/tree", "dirfrag": "0x100000002f4.000*", "snapid_first": 2, "projected_version": "602", "version": "602", "committing_version": "602", "committed_version": "602", "is_rep": false, "dir_auth": "0", "states": [ "auth", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "frozen": 0, "subtree": 1, "dirty": 0, "authpin": 0 }, "nref": 2 } }, { "is_auth": false, "auth_first": 2, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 2, "dir": { "path": "/tree", "dirfrag": "0x100000002f4.001*", "snapid_first": 2, "projected_version": "602", "version": "602", "committing_version": "602", "committed_version": "602", "is_rep": false, "dir_auth": "2", "states": [], "is_auth": false, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "request": 0, "child": 0, "frozen": 0, "subtree": 1, "replicated": 0, "dirty": 0, "waiter": 0, "authpin": 0, "tempexporting": 0 }, "nref": 1 } }, { "is_auth": false, "auth_first": 2, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 2, "dir": { "path": "/tree", "dirfrag": "0x100000002f4.010*", "snapid_first": 2, "projected_version": "602", "version": "602", "committing_version": "602", "committed_version": "602", "is_rep": false, "dir_auth": "2", "states": [], "is_auth": false, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "request": 0, "child": 0, "frozen": 0, "subtree": 1, "replicated": 0, "dirty": 0, "waiter": 0, "authpin": 0, "tempexporting": 0 }, "nref": 1 } }, { "is_auth": false, "auth_first": 1, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 1, "dir": { "path": "/tree", "dirfrag": "0x100000002f4.011*", "snapid_first": 2, "projected_version": "602", "version": "602", "committing_version": "602", "committed_version": "602", "is_rep": false, "dir_auth": "1", "states": [], "is_auth": false, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "request": 0, "child": 0, "frozen": 0, "subtree": 1, "replicated": 0, "dirty": 0, "waiter": 0, "authpin": 0, "tempexporting": 0 }, "nref": 1 } }, { "is_auth": false, "auth_first": 1, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 1, "dir": { "path": "/tree", "dirfrag": "0x100000002f4.100*", "snapid_first": 2, "projected_version": "602", "version": "602", "committing_version": "602", "committed_version": "602", "is_rep": false, "dir_auth": "1", "states": [], "is_auth": false, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "request": 0, "child": 0, "frozen": 0, "subtree": 1, "replicated": 0, "dirty": 0, "waiter": 0, "authpin": 0, "tempexporting": 0 }, "nref": 1 } }, { "is_auth": false, "auth_first": 2, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 2, "dir": { "path": "/tree", "dirfrag": "0x100000002f4.101*", "snapid_first": 2, "projected_version": "602", "version": "602", "committing_version": "602", "committed_version": "602", "is_rep": false, "dir_auth": "2", "states": [], "is_auth": false, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "request": 0, "child": 0, "frozen": 0, "subtree": 1, "replicated": 0, "dirty": 0, "waiter": 0, "authpin": 0, "tempexporting": 0 }, "nref": 1 } }, { "is_auth": false, "auth_first": 1, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 1, "dir": { "path": "/tree", "dirfrag": "0x100000002f4.110*", "snapid_first": 2, "projected_version": "602", "version": "602", "committing_version": "602", "committed_version": "602", "is_rep": false, "dir_auth": "1", "states": [], "is_auth": false, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "request": 0, "child": 0, "frozen": 0, "subtree": 1, "replicated": 0, "dirty": 0, "waiter": 0, "authpin": 0, "tempexporting": 0 }, "nref": 1 } }, { "is_auth": true, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 0, "dir": { "path": "/tree", "dirfrag": "0x100000002f4.111*", "snapid_first": 2, "projected_version": "602", "version": "602", "committing_version": "602", "committed_version": "602", "is_rep": false, "dir_auth": "0", "states": [ "auth", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "frozen": 0, "subtree": 1, "dirty": 0, "authpin": 0 }, "nref": 2 } } ] 2024-05-26T22:53:29.553 INFO:tasks.cephfs.cephfs_test_case:len=8 [ { "is_auth": true, "auth_first": 2, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 2, "dir": { "path": "/tree", "dirfrag": "0x100000002f4.101*", "snapid_first": 2, "projected_version": "602", "version": "602", "committing_version": "602", "committed_version": "602", "is_rep": false, "dir_auth": "2", "states": [ "auth", "complete" ], "is_auth": true, "auth_state": { "replicas": { "0": 1 } }, "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "child": 1, "frozen": 0, "subtree": 1, "importing": 0, "replicated": 1 }, "nref": 3 } }, { "is_auth": true, "auth_first": 2, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 2, "dir": { "path": "/tree", "dirfrag": "0x100000002f4.010*", "snapid_first": 2, "projected_version": "602", "version": "602", "committing_version": "602", "committed_version": "602", "is_rep": false, "dir_auth": "2", "states": [ "auth", "complete" ], "is_auth": true, "auth_state": { "replicas": { "0": 1 } }, "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "child": 1, "frozen": 0, "subtree": 1, "importing": 0, "replicated": 1 }, "nref": 3 } }, { "is_auth": true, "auth_first": 2, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 2, "dir": { "path": "/tree", "dirfrag": "0x100000002f4.001*", "snapid_first": 2, "projected_version": "602", "version": "602", "committing_version": "602", "committed_version": "602", "is_rep": false, "dir_auth": "2", "states": [ "auth", "complete" ], "is_auth": true, "auth_state": { "replicas": { "0": 1 } }, "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "child": 1, "frozen": 0, "subtree": 1, "importing": 0, "replicated": 1 }, "nref": 3 } }, { "is_auth": true, "auth_first": 1, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 1, "dir": { "path": "/tree", "dirfrag": "0x100000002f4.110*", "snapid_first": 2, "projected_version": "602", "version": "602", "committing_version": "602", "committed_version": "602", "is_rep": false, "dir_auth": "1", "states": [ "auth", "complete" ], "is_auth": true, "auth_state": { "replicas": { "0": 1 } }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "child": 1, "frozen": 0, "subtree": 1, "importing": 0, "replicated": 1 }, "nref": 3 } }, { "is_auth": true, "auth_first": 1, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 1, "dir": { "path": "/tree", "dirfrag": "0x100000002f4.100*", "snapid_first": 2, "projected_version": "602", "version": "602", "committing_version": "602", "committed_version": "602", "is_rep": false, "dir_auth": "1", "states": [ "auth", "complete" ], "is_auth": true, "auth_state": { "replicas": { "0": 1 } }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "child": 1, "frozen": 0, "subtree": 1, "importing": 0, "replicated": 1 }, "nref": 3 } }, { "is_auth": true, "auth_first": 1, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 1, "dir": { "path": "/tree", "dirfrag": "0x100000002f4.011*", "snapid_first": 2, "projected_version": "602", "version": "602", "committing_version": "602", "committed_version": "602", "is_rep": false, "dir_auth": "1", "states": [ "auth", "complete" ], "is_auth": true, "auth_state": { "replicas": { "0": 1 } }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "child": 1, "frozen": 0, "subtree": 1, "importing": 0, "replicated": 1 }, "nref": 3 } }, { "is_auth": true, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 0, "dir": { "path": "/tree", "dirfrag": "0x100000002f4.000*", "snapid_first": 2, "projected_version": "602", "version": "602", "committing_version": "602", "committed_version": "602", "is_rep": false, "dir_auth": "0", "states": [ "auth", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "frozen": 0, "subtree": 1, "dirty": 0, "authpin": 0 }, "nref": 2 } }, { "is_auth": true, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 0, "dir": { "path": "/tree", "dirfrag": "0x100000002f4.111*", "snapid_first": 2, "projected_version": "602", "version": "602", "committing_version": "602", "committed_version": "602", "is_rep": false, "dir_auth": "0", "states": [ "auth", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "frozen": 0, "subtree": 1, "dirty": 0, "authpin": 0 }, "nref": 2 } } ] 2024-05-26T22:53:29.553 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-05-26T22:53:29.554 DEBUG:teuthology.orchestra.run.smithi131:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-26T22:53:29.661 DEBUG:teuthology.orchestra.run.smithi187:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-26T22:53:29.773 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-05-26T22:53:29.773 INFO:tasks.cephfs.mount:Terminating background process 2024-05-26T22:53:29.773 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-26T22:53:29.774 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-26T22:53:29.832 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-26T22:53:29.832 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-26T22:53:29.885 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-26T22:53:30.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.085+0000 7fb95ce00640 1 Processor -- start 2024-05-26T22:53:30.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.086+0000 7fb95ce00640 1 -- start start 2024-05-26T22:53:30.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.087+0000 7fb95ce00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fb958144800 0x7fb95815fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:30.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.087+0000 7fb95ce00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb9580ad870 0x7fb9580adcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:30.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.087+0000 7fb95ce00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fb95815e9f0 0x7fb95815ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:30.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.087+0000 7fb95ce00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fb9580ae210 con 0x7fb95815e9f0 2024-05-26T22:53:30.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.087+0000 7fb95ce00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fb95815ffd0 con 0x7fb958144800 2024-05-26T22:53:30.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.087+0000 7fb95ce00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb95815f7b0 con 0x7fb9580ad870 2024-05-26T22:53:30.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.088+0000 7fb957e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fb95815e9f0 0x7fb95815ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:30.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.088+0000 7fb957400640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fb958144800 0x7fb95815fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:30.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.088+0000 7fb957e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fb95815e9f0 0x7fb95815ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48962/0 (socket says 172.21.15.131:48962) 2024-05-26T22:53:30.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.088+0000 7fb94fe00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb9580ad870 0x7fb9580adcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:30.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.088+0000 7fb957e00640 1 -- 172.21.15.131:0/3429844959 learned_addr learned my addr 172.21.15.131:0/3429844959 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:53:30.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.089+0000 7fb957e00640 1 -- 172.21.15.131:0/3429844959 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb9580ad870 msgr2=0x7fb9580adcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:30.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.089+0000 7fb957e00640 1 --2- 172.21.15.131:0/3429844959 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb9580ad870 0x7fb9580adcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:30.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.089+0000 7fb957e00640 1 -- 172.21.15.131:0/3429844959 >> v2:172.21.15.187:3300/0 conn(0x7fb958144800 msgr2=0x7fb95815fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:30.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.089+0000 7fb957e00640 1 --2- 172.21.15.131:0/3429844959 >> v2:172.21.15.187:3300/0 conn(0x7fb958144800 0x7fb95815fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:30.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.089+0000 7fb957e00640 1 -- 172.21.15.131:0/3429844959 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9580cde40 con 0x7fb95815e9f0 2024-05-26T22:53:30.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.090+0000 7fb957e00640 1 --2- 172.21.15.131:0/3429844959 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb95815e9f0 0x7fb95815ee80 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fb95806e8c0 tx=0x7fb95002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae177e46d86b954a server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:30.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.090+0000 7fb956a00640 1 -- 172.21.15.131:0/3429844959 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb950036070 con 0x7fb95815e9f0 2024-05-26T22:53:30.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.090+0000 7fb956a00640 1 -- 172.21.15.131:0/3429844959 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb950031600 con 0x7fb95815e9f0 2024-05-26T22:53:30.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.090+0000 7fb956a00640 1 -- 172.21.15.131:0/3429844959 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9500319d0 con 0x7fb95815e9f0 2024-05-26T22:53:30.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.091+0000 7fb95ce00640 1 -- 172.21.15.131:0/3429844959 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb95815e9f0 msgr2=0x7fb95815ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:30.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.091+0000 7fb95ce00640 1 --2- 172.21.15.131:0/3429844959 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb95815e9f0 0x7fb95815ee80 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fb95806e8c0 tx=0x7fb95002a730 comp rx=0 tx=0).stop 2024-05-26T22:53:30.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.091+0000 7fb95ce00640 1 -- 172.21.15.131:0/3429844959 shutdown_connections 2024-05-26T22:53:30.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.091+0000 7fb95ce00640 1 --2- 172.21.15.131:0/3429844959 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb95815e9f0 0x7fb95815ee80 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:30.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.091+0000 7fb95ce00640 1 --2- 172.21.15.131:0/3429844959 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb9580ad870 0x7fb9580adcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:30.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.091+0000 7fb95ce00640 1 --2- 172.21.15.131:0/3429844959 >> v2:172.21.15.187:3300/0 conn(0x7fb958144800 0x7fb95815fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:30.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.091+0000 7fb95ce00640 1 -- 172.21.15.131:0/3429844959 >> 172.21.15.131:0/3429844959 conn(0x7fb9580cff40 msgr2=0x7fb9580d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:30.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.091+0000 7fb95ce00640 1 -- 172.21.15.131:0/3429844959 shutdown_connections 2024-05-26T22:53:30.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.091+0000 7fb95ce00640 1 -- 172.21.15.131:0/3429844959 wait complete. 2024-05-26T22:53:30.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.093+0000 7fb95ce00640 1 Processor -- start 2024-05-26T22:53:30.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.093+0000 7fb95ce00640 1 -- start start 2024-05-26T22:53:30.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.094+0000 7fb95ce00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb9580ad870 0x7fb9581f3510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:30.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.094+0000 7fb95ce00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb958144800 0x7fb95812f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:30.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.094+0000 7fb95ce00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb95815e9f0 0x7fb958071e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:30.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.094+0000 7fb95ce00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fb958144be0 con 0x7fb958144800 2024-05-26T22:53:30.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.094+0000 7fb95ce00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb9580ce400 con 0x7fb9580ad870 2024-05-26T22:53:30.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.094+0000 7fb95ce00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb95815f3c0 con 0x7fb95815e9f0 2024-05-26T22:53:30.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.094+0000 7fb94fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb958144800 0x7fb95812f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:30.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.094+0000 7fb94fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb958144800 0x7fb95812f3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48968/0 (socket says 172.21.15.131:48968) 2024-05-26T22:53:30.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.094+0000 7fb94fe00640 1 -- 172.21.15.131:0/551778030 learned_addr learned my addr 172.21.15.131:0/551778030 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:53:30.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.094+0000 7fb957400640 1 --2- 172.21.15.131:0/551778030 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb9580ad870 0x7fb9581f3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:30.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.094+0000 7fb94fe00640 1 -- 172.21.15.131:0/551778030 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb95815e9f0 msgr2=0x7fb958071e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:30.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.094+0000 7fb94fe00640 1 --2- 172.21.15.131:0/551778030 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb95815e9f0 0x7fb958071e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:30.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.094+0000 7fb94fe00640 1 -- 172.21.15.131:0/551778030 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb9580ad870 msgr2=0x7fb9581f3510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:30.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.094+0000 7fb94fe00640 1 --2- 172.21.15.131:0/551778030 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb9580ad870 0x7fb9581f3510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:30.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.094+0000 7fb94fe00640 1 -- 172.21.15.131:0/551778030 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb95812fe90 con 0x7fb958144800 2024-05-26T22:53:30.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.094+0000 7fb957400640 1 --2- 172.21.15.131:0/551778030 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb9580ad870 0x7fb9581f3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:53:30.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.095+0000 7fb94fe00640 1 --2- 172.21.15.131:0/551778030 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb958144800 0x7fb95812f3d0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fb94000e890 tx=0x7fb94000ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:30.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.095+0000 7fb94f400640 1 -- 172.21.15.131:0/551778030 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb940017070 con 0x7fb958144800 2024-05-26T22:53:30.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.095+0000 7fb94f400640 1 -- 172.21.15.131:0/551778030 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb94000c940 con 0x7fb958144800 2024-05-26T22:53:30.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.095+0000 7fb94f400640 1 -- 172.21.15.131:0/551778030 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb94000cd10 con 0x7fb958144800 2024-05-26T22:53:30.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.095+0000 7fb95ce00640 1 -- 172.21.15.131:0/551778030 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb958130170 con 0x7fb958144800 2024-05-26T22:53:30.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.095+0000 7fb95ce00640 1 -- 172.21.15.131:0/551778030 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb958130730 con 0x7fb958144800 2024-05-26T22:53:30.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.096+0000 7fb95ce00640 1 -- 172.21.15.131:0/551778030 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9580694e0 con 0x7fb958144800 2024-05-26T22:53:30.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.097+0000 7fb94f400640 1 -- 172.21.15.131:0/551778030 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb94001a070 con 0x7fb958144800 2024-05-26T22:53:30.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.098+0000 7fb94f400640 1 --2- 172.21.15.131:0/551778030 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb93406be60 0x7fb93406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:30.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.098+0000 7fb94f400640 1 -- 172.21.15.131:0/551778030 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5767+0+0 (secure 0 0 0) 0x7fb94008ae60 con 0x7fb958144800 2024-05-26T22:53:30.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.098+0000 7fb957400640 1 --2- 172.21.15.131:0/551778030 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb93406be60 0x7fb93406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:30.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.099+0000 7fb957400640 1 --2- 172.21.15.131:0/551778030 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb93406be60 0x7fb93406e320 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fb948009ac0 tx=0x7fb948008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:30.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.103+0000 7fb94f400640 1 -- 172.21.15.131:0/551778030 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fb9400597c0 con 0x7fb958144800 2024-05-26T22:53:30.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.305+0000 7fb95ce00640 1 -- 172.21.15.131:0/551778030 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb95806eb90 con 0x7fb958144800 2024-05-26T22:53:30.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.307+0000 7fb94f400640 1 -- 172.21.15.131:0/551778030 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) ==== 74+0+16789 (secure 0 0 0) 0x7fb9400590c0 con 0x7fb958144800 2024-05-26T22:53:30.307 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:53:30.307 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":71,"fsid":"46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f","created":"2024-05-26T22:47:57.288905+0000","modified":"2024-05-26T22:53:06.170553+0000","last_up_change":"2024-05-26T22:48:09.602207+0000","last_in_change":"2024-05-26T22:48:00.687341+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-26T22:48:10.463339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-05-26T22:52:51.536251+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-05-26T22:52:52.768524+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"814eaf62-a18d-494b-8e27-fd8fb88adea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6801","nonce":2350940051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6803","nonce":2350940051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6807","nonce":2350940051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6805","nonce":2350940051}]},"public_addr":"172.21.15.131:6801/2350940051","cluster_addr":"172.21.15.131:6803/2350940051","heartbeat_back_addr":"172.21.15.131:6807/2350940051","heartbeat_front_addr":"172.21.15.131:6805/2350940051","state":["exists","up"]},{"osd":1,"uuid":"42e66287-f6c6-4e26-af3c-6dfaa3536d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6825","nonce":1525528247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6827","nonce":1525528247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6831","nonce":1525528247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6829","nonce":1525528247}]},"public_addr":"172.21.15.131:6825/1525528247","cluster_addr":"172.21.15.131:6827/1525528247","heartbeat_back_addr":"172.21.15.131:6831/1525528247","heartbeat_front_addr":"172.21.15.131:6829/1525528247","state":["exists","up"]},{"osd":2,"uuid":"0da34673-04a2-4fd0-b57a-0ff2b679b043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6817","nonce":3627842210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6819","nonce":3627842210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6823","nonce":3627842210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6821","nonce":3627842210}]},"public_addr":"172.21.15.131:6817/3627842210","cluster_addr":"172.21.15.131:6819/3627842210","heartbeat_back_addr":"172.21.15.131:6823/3627842210","heartbeat_front_addr":"172.21.15.131:6821/3627842210","state":["exists","up"]},{"osd":3,"uuid":"cf117a27-d6b2-43d3-ae52-b47988bf8eed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6809","nonce":350790591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6811","nonce":350790591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6815","nonce":350790591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6813","nonce":350790591}]},"public_addr":"172.21.15.131:6809/350790591","cluster_addr":"172.21.15.131:6811/350790591","heartbeat_back_addr":"172.21.15.131:6815/350790591","heartbeat_front_addr":"172.21.15.131:6813/350790591","state":["exists","up"]},{"osd":4,"uuid":"a092ccfc-be89-4a84-88d5-0f781fa715eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6816","nonce":244560995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6819","nonce":244560995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6823","nonce":244560995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6821","nonce":244560995}]},"public_addr":"172.21.15.187:6816/244560995","cluster_addr":"172.21.15.187:6819/244560995","heartbeat_back_addr":"172.21.15.187:6823/244560995","heartbeat_front_addr":"172.21.15.187:6821/244560995","state":["exists","up"]},{"osd":5,"uuid":"7513ba69-dd00-4a35-b190-9e8cc3b2bdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6801","nonce":324619398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6803","nonce":324619398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6812","nonce":324619398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6807","nonce":324619398}]},"public_addr":"172.21.15.187:6801/324619398","cluster_addr":"172.21.15.187:6803/324619398","heartbeat_back_addr":"172.21.15.187:6812/324619398","heartbeat_front_addr":"172.21.15.187:6807/324619398","state":["exists","up"]},{"osd":6,"uuid":"50081810-c093-4d2f-a4d7-b7c695c5400d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6806","nonce":1391347786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6809","nonce":1391347786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6818","nonce":1391347786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6813","nonce":1391347786}]},"public_addr":"172.21.15.187:6806/1391347786","cluster_addr":"172.21.15.187:6809/1391347786","heartbeat_back_addr":"172.21.15.187:6818/1391347786","heartbeat_front_addr":"172.21.15.187:6813/1391347786","state":["exists","up"]},{"osd":7,"uuid":"775dae05-54f5-4ad4-bbce-92863c6ed2ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6825","nonce":1220065622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6827","nonce":1220065622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6831","nonce":1220065622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6829","nonce":1220065622}]},"public_addr":"172.21.15.187:6825/1220065622","cluster_addr":"172.21.15.187:6827/1220065622","heartbeat_back_addr":"172.21.15.187:6831/1220065622","heartbeat_front_addr":"172.21.15.187:6829/1220065622","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:05.983520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.299516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.067323+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.307748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.018742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.258775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.010912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.295355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.a","mappings":[{"from":6,"to":0}]},{"pgid":"11.22","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-26T22:53:30.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.310+0000 7fb95ce00640 1 -- 172.21.15.131:0/551778030 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb93406be60 msgr2=0x7fb93406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:30.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.310+0000 7fb95ce00640 1 --2- 172.21.15.131:0/551778030 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb93406be60 0x7fb93406e320 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fb948009ac0 tx=0x7fb948008040 comp rx=0 tx=0).stop 2024-05-26T22:53:30.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.310+0000 7fb95ce00640 1 -- 172.21.15.131:0/551778030 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb958144800 msgr2=0x7fb95812f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:30.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.310+0000 7fb95ce00640 1 --2- 172.21.15.131:0/551778030 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb958144800 0x7fb95812f3d0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fb94000e890 tx=0x7fb94000ed50 comp rx=0 tx=0).stop 2024-05-26T22:53:30.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.310+0000 7fb95ce00640 1 -- 172.21.15.131:0/551778030 shutdown_connections 2024-05-26T22:53:30.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.310+0000 7fb95ce00640 1 --2- 172.21.15.131:0/551778030 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb95815e9f0 0x7fb958071e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:30.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.310+0000 7fb95ce00640 1 --2- 172.21.15.131:0/551778030 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb93406be60 0x7fb93406e320 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:30.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.310+0000 7fb95ce00640 1 --2- 172.21.15.131:0/551778030 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb958144800 0x7fb95812f3d0 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:30.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.310+0000 7fb95ce00640 1 --2- 172.21.15.131:0/551778030 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb9580ad870 0x7fb9581f3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:30.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.310+0000 7fb95ce00640 1 -- 172.21.15.131:0/551778030 >> 172.21.15.131:0/551778030 conn(0x7fb9580cff40 msgr2=0x7fb9580d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:30.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.310+0000 7fb95ce00640 1 -- 172.21.15.131:0/551778030 shutdown_connections 2024-05-26T22:53:30.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.310+0000 7fb95ce00640 1 -- 172.21.15.131:0/551778030 wait complete. 2024-05-26T22:53:30.329 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-26T22:48:10.463339+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-26T22:52:51.536251+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '68', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-05-26T22:52:52.768524+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '69', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-26T22:53:30.329 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-26T22:53:30.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.531+0000 7f8bc7800640 1 Processor -- start 2024-05-26T22:53:30.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.532+0000 7f8bc7800640 1 -- start start 2024-05-26T22:53:30.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.533+0000 7f8bc7800640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f8bc006f420 0x7f8bc00cfd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:30.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.533+0000 7f8bc7800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8bc00cf590 0x7f8bc00d0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:30.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.533+0000 7f8bc7800640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8bc0160180 con 0x7f8bc00b5450 2024-05-26T22:53:30.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.533+0000 7f8bc7800640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f8bc0160d90 con 0x7f8bc006f420 2024-05-26T22:53:30.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.533+0000 7f8bc7800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f8bc0160460 con 0x7f8bc00cf590 2024-05-26T22:53:30.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.533+0000 7f8bc4e00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f8bc00b5450 0x7f8bc00b43c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:58574/0 (socket says 172.21.15.131:58574) 2024-05-26T22:53:30.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.533+0000 7f8bc4e00640 1 -- 172.21.15.131:0/1456923641 learned_addr learned my addr 172.21.15.131:0/1456923641 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:53:30.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.534+0000 7f8bbf400640 1 -- 172.21.15.131:0/1456923641 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1560788514 0 0) 0x7f8bc0160180 con 0x7f8bc00b5450 2024-05-26T22:53:30.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.534+0000 7f8bbf400640 1 -- 172.21.15.131:0/1456923641 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8ba4003520 con 0x7f8bc00b5450 2024-05-26T22:53:30.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.534+0000 7f8bbfe00640 1 --2- 172.21.15.131:0/1456923641 >> v2:172.21.15.187:3300/0 conn(0x7f8bc006f420 0x7f8bc00cfd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:30.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.534+0000 7f8bc5800640 1 --2- 172.21.15.131:0/1456923641 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8bc00cf590 0x7f8bc00d0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:30.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.534+0000 7f8bbf400640 1 -- 172.21.15.131:0/1456923641 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1116366345 0 0) 0x7f8ba4003520 con 0x7f8bc00b5450 2024-05-26T22:53:30.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.534+0000 7f8bbf400640 1 -- 172.21.15.131:0/1456923641 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8bc00cf590 msgr2=0x7f8bc00d0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:30.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.534+0000 7f8bbf400640 1 --2- 172.21.15.131:0/1456923641 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8bc00cf590 0x7f8bc00d0290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:30.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.534+0000 7f8bbf400640 1 -- 172.21.15.131:0/1456923641 >> v2:172.21.15.187:3300/0 conn(0x7f8bc006f420 msgr2=0x7f8bc00cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:30.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.534+0000 7f8bbf400640 1 --2- 172.21.15.131:0/1456923641 >> v2:172.21.15.187:3300/0 conn(0x7f8bc006f420 0x7f8bc00cfd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:30.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.534+0000 7f8bbf400640 1 -- 172.21.15.131:0/1456923641 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bc0160740 con 0x7f8bc00b5450 2024-05-26T22:53:30.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.534+0000 7f8bbf400640 1 -- 172.21.15.131:0/1456923641 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7f8bb00019f0 con 0x7f8bc00b5450 2024-05-26T22:53:30.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.534+0000 7f8bbf400640 1 -- 172.21.15.131:0/1456923641 >> v1:172.21.15.131:6789/0 conn(0x7f8bc00b5450 legacy=0x7f8bc00b43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:30.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.534+0000 7f8bbf400640 1 --2- 172.21.15.131:0/1456923641 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ba40039b0 0x7f8ba4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:30.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.534+0000 7f8bbf400640 1 --2- 172.21.15.131:0/1456923641 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ba4004800 0x7f8ba4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:30.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.534+0000 7f8bbf400640 1 --2- 172.21.15.131:0/1456923641 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8bc00cf590 0x7f8ba40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:30.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.534+0000 7f8bbf400640 1 -- 172.21.15.131:0/1456923641 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc0160d90 con 0x7f8ba4004800 2024-05-26T22:53:30.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.534+0000 7f8bbf400640 1 -- 172.21.15.131:0/1456923641 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc0160460 con 0x7f8ba40039b0 2024-05-26T22:53:30.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.534+0000 7f8bbf400640 1 -- 172.21.15.131:0/1456923641 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f8ba400baf0 con 0x7f8bc00cf590 2024-05-26T22:53:30.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.535+0000 7f8bbfe00640 1 --2- 172.21.15.131:0/1456923641 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ba4004800 0x7f8ba4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:30.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.535+0000 7f8bc4e00640 1 --2- 172.21.15.131:0/1456923641 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ba40039b0 0x7f8ba4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:30.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.535+0000 7f8bc5800640 1 --2- 172.21.15.131:0/1456923641 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8bc00cf590 0x7f8ba40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:30.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.535+0000 7f8bbfe00640 1 -- 172.21.15.131:0/1456923641 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8bc00cf590 msgr2=0x7f8ba40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:30.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.535+0000 7f8bbfe00640 1 --2- 172.21.15.131:0/1456923641 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8bc00cf590 0x7f8ba40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:30.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.535+0000 7f8bbfe00640 1 -- 172.21.15.131:0/1456923641 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ba40039b0 msgr2=0x7f8ba4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:30.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.535+0000 7f8bbfe00640 1 --2- 172.21.15.131:0/1456923641 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ba40039b0 0x7f8ba4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:30.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.535+0000 7f8bbfe00640 1 -- 172.21.15.131:0/1456923641 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ba4003520 con 0x7f8ba4004800 2024-05-26T22:53:30.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.535+0000 7f8bc4e00640 1 --2- 172.21.15.131:0/1456923641 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ba40039b0 0x7f8ba4003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:53:30.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.535+0000 7f8bc5800640 1 --2- 172.21.15.131:0/1456923641 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8bc00cf590 0x7f8ba40051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:53:30.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.536+0000 7f8bbfe00640 1 --2- 172.21.15.131:0/1456923641 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ba4004800 0x7f8ba4004c80 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f8ba800ba00 tx=0x7f8ba4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31a3eda23bfd8eb1 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:30.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.536+0000 7f8bbf400640 1 -- 172.21.15.131:0/1456923641 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ba8030070 con 0x7f8ba4004800 2024-05-26T22:53:30.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.537+0000 7f8bbf400640 1 -- 172.21.15.131:0/1456923641 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ba802adb0 con 0x7f8ba4004800 2024-05-26T22:53:30.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.537+0000 7f8bbf400640 1 -- 172.21.15.131:0/1456923641 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ba800b600 con 0x7f8ba4004800 2024-05-26T22:53:30.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.537+0000 7f8bc7800640 1 -- 172.21.15.131:0/1456923641 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ba4004800 msgr2=0x7f8ba4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:30.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.537+0000 7f8bc7800640 1 --2- 172.21.15.131:0/1456923641 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ba4004800 0x7f8ba4004c80 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f8ba800ba00 tx=0x7f8ba4005830 comp rx=0 tx=0).stop 2024-05-26T22:53:30.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.537+0000 7f8bc5800640 1 -- 172.21.15.131:0/1456923641 reap_dead start 2024-05-26T22:53:30.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.537+0000 7f8bc7800640 1 -- 172.21.15.131:0/1456923641 shutdown_connections 2024-05-26T22:53:30.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.537+0000 7f8bc7800640 1 -- 172.21.15.131:0/1456923641 >> 172.21.15.131:0/1456923641 conn(0x7f8bc006bc90 msgr2=0x7f8bc00b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:30.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.537+0000 7f8bc7800640 1 -- 172.21.15.131:0/1456923641 shutdown_connections 2024-05-26T22:53:30.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.537+0000 7f8bc7800640 1 -- 172.21.15.131:0/1456923641 wait complete. 2024-05-26T22:53:30.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.539+0000 7f8bc7800640 1 Processor -- start 2024-05-26T22:53:30.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.539+0000 7f8bc7800640 1 -- start start 2024-05-26T22:53:30.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.540+0000 7f8bc7800640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ba4004800 0x7f8bc00cc940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:30.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.540+0000 7f8bc7800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8bc01f2a90 0x7f8bc00cce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:30.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.540+0000 7f8bc7800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8bc0156200 0x7f8bc0156680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:30.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.540+0000 7f8bc7800640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc015fde0 con 0x7f8bc01f2a90 2024-05-26T22:53:30.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.540+0000 7f8bc7800640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc00b5830 con 0x7f8ba4004800 2024-05-26T22:53:30.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.540+0000 7f8bc7800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f8bc015ff60 con 0x7f8bc0156200 2024-05-26T22:53:30.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.540+0000 7f8bbfe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8bc01f2a90 0x7f8bc00cce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:30.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.540+0000 7f8bbfe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8bc01f2a90 0x7f8bc00cce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:48992/0 (socket says 172.21.15.131:48992) 2024-05-26T22:53:30.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.540+0000 7f8bbfe00640 1 -- 172.21.15.131:0/1369523324 learned_addr learned my addr 172.21.15.131:0/1369523324 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:53:30.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.541+0000 7f8bbfe00640 1 -- 172.21.15.131:0/1369523324 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8bc0156200 msgr2=0x7f8bc0156680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:30.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.541+0000 7f8bc4e00640 1 --2- 172.21.15.131:0/1369523324 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ba4004800 0x7f8bc00cc940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:30.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.541+0000 7f8bbfe00640 1 --2- 172.21.15.131:0/1369523324 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8bc0156200 0x7f8bc0156680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:30.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.541+0000 7f8bbfe00640 1 -- 172.21.15.131:0/1369523324 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ba4004800 msgr2=0x7f8bc00cc940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:30.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.541+0000 7f8bbfe00640 1 --2- 172.21.15.131:0/1369523324 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ba4004800 0x7f8bc00cc940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:30.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.541+0000 7f8bbfe00640 1 -- 172.21.15.131:0/1369523324 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bc0156f30 con 0x7f8bc01f2a90 2024-05-26T22:53:30.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.541+0000 7f8bbfe00640 1 --2- 172.21.15.131:0/1369523324 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8bc01f2a90 0x7f8bc00cce80 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f8ba800bb20 tx=0x7f8ba8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:30.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.541+0000 7f8bbcc00640 1 -- 172.21.15.131:0/1369523324 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ba8043020 con 0x7f8bc01f2a90 2024-05-26T22:53:30.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.541+0000 7f8bbcc00640 1 -- 172.21.15.131:0/1369523324 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ba803f680 con 0x7f8bc01f2a90 2024-05-26T22:53:30.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.541+0000 7f8bbcc00640 1 -- 172.21.15.131:0/1369523324 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ba800b600 con 0x7f8bc01f2a90 2024-05-26T22:53:30.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.541+0000 7f8bc7800640 1 -- 172.21.15.131:0/1369523324 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bc0157210 con 0x7f8bc01f2a90 2024-05-26T22:53:30.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.542+0000 7f8bc7800640 1 -- 172.21.15.131:0/1369523324 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bc01577d0 con 0x7f8bc01f2a90 2024-05-26T22:53:30.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.543+0000 7f8bc7800640 1 -- 172.21.15.131:0/1369523324 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bc0074e90 con 0x7f8bc01f2a90 2024-05-26T22:53:30.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.543+0000 7f8bbcc00640 1 -- 172.21.15.131:0/1369523324 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8ba8046020 con 0x7f8bc01f2a90 2024-05-26T22:53:30.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.544+0000 7f8bbcc00640 1 --2- 172.21.15.131:0/1369523324 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f8b9c06be30 0x7f8b9c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:30.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.545+0000 7f8bbcc00640 1 -- 172.21.15.131:0/1369523324 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5767+0+0 (secure 0 0 0) 0x7f8ba80abbb0 con 0x7f8bc01f2a90 2024-05-26T22:53:30.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.545+0000 7f8bc4e00640 1 --2- 172.21.15.131:0/1369523324 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f8b9c06be30 0x7f8b9c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:30.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.546+0000 7f8bc4e00640 1 --2- 172.21.15.131:0/1369523324 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f8b9c06be30 0x7f8b9c06e2f0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f8bb0001b60 tx=0x7f8bb0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:30.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.549+0000 7f8bbcc00640 1 -- 172.21.15.131:0/1369523324 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f8ba804d030 con 0x7f8bc01f2a90 2024-05-26T22:53:30.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.747+0000 7f8bc7800640 1 -- 172.21.15.131:0/1369523324 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8bc00cee90 con 0x7f8bc01f2a90 2024-05-26T22:53:30.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.748+0000 7f8bbcc00640 1 -- 172.21.15.131:0/1369523324 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v71) ==== 93+0+10 (secure 0 0 0) 0x7f8ba807a480 con 0x7f8bc01f2a90 2024-05-26T22:53:30.747 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2024-05-26T22:53:30.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.750+0000 7f8bc7800640 1 -- 172.21.15.131:0/1369523324 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f8b9c06be30 msgr2=0x7f8b9c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:30.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.750+0000 7f8bc7800640 1 --2- 172.21.15.131:0/1369523324 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f8b9c06be30 0x7f8b9c06e2f0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f8bb0001b60 tx=0x7f8bb0009040 comp rx=0 tx=0).stop 2024-05-26T22:53:30.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.750+0000 7f8bc7800640 1 -- 172.21.15.131:0/1369523324 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8bc01f2a90 msgr2=0x7f8bc00cce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:30.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.750+0000 7f8bc7800640 1 --2- 172.21.15.131:0/1369523324 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8bc01f2a90 0x7f8bc00cce80 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f8ba800bb20 tx=0x7f8ba8004730 comp rx=0 tx=0).stop 2024-05-26T22:53:30.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.750+0000 7f8bc7800640 1 -- 172.21.15.131:0/1369523324 shutdown_connections 2024-05-26T22:53:30.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.750+0000 7f8bc7800640 1 --2- 172.21.15.131:0/1369523324 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8bc0156200 0x7f8bc0156680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:30.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.750+0000 7f8bc7800640 1 --2- 172.21.15.131:0/1369523324 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f8b9c06be30 0x7f8b9c06e2f0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:30.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.750+0000 7f8bc7800640 1 --2- 172.21.15.131:0/1369523324 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8bc01f2a90 0x7f8bc00cce80 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:30.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.750+0000 7f8bc7800640 1 --2- 172.21.15.131:0/1369523324 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ba4004800 0x7f8bc00cc940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:30.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.750+0000 7f8bc7800640 1 -- 172.21.15.131:0/1369523324 >> 172.21.15.131:0/1369523324 conn(0x7f8bc006bc90 msgr2=0x7f8bc00b0ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:30.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.751+0000 7f8bc7800640 1 -- 172.21.15.131:0/1369523324 shutdown_connections 2024-05-26T22:53:30.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.751+0000 7f8bc7800640 1 -- 172.21.15.131:0/1369523324 wait complete. 2024-05-26T22:53:30.770 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-26T22:53:30.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.965+0000 7f79c5600640 1 Processor -- start 2024-05-26T22:53:30.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.965+0000 7f79c5600640 1 -- start start 2024-05-26T22:53:30.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.967+0000 7f79c5600640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f79c00af580 0x7f79c015f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:30.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.967+0000 7f79c5600640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f79c006b680 0x7f79c006bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:30.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.967+0000 7f79c5600640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79c00ae0c0 con 0x7f79c00af960 2024-05-26T22:53:30.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.967+0000 7f79c5600640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f79c00ae3a0 con 0x7f79c006b680 2024-05-26T22:53:30.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.967+0000 7f79c5600640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f79c00ae680 con 0x7f79c00af580 2024-05-26T22:53:30.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.967+0000 7f79bf400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f79c006b680 0x7f79c006bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:30.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.967+0000 7f79bf400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f79c006b680 0x7f79c006bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49002/0 (socket says 172.21.15.131:49002) 2024-05-26T22:53:30.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.967+0000 7f79bf400640 1 -- 172.21.15.131:0/178963731 learned_addr learned my addr 172.21.15.131:0/178963731 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:53:30.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.967+0000 7f79bea00640 1 --2- 172.21.15.131:0/178963731 >> v2:172.21.15.187:3300/0 conn(0x7f79c00af580 0x7f79c015f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:30.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.968+0000 7f79bf400640 1 -- 172.21.15.131:0/178963731 >> v2:172.21.15.187:3300/0 conn(0x7f79c00af580 msgr2=0x7f79c015f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:30.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.968+0000 7f79bf400640 1 --2- 172.21.15.131:0/178963731 >> v2:172.21.15.187:3300/0 conn(0x7f79c00af580 0x7f79c015f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:30.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.968+0000 7f79bf400640 1 -- 172.21.15.131:0/178963731 >> v1:172.21.15.187:6789/0 conn(0x7f79c00af960 legacy=0x7f79c00ad870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:30.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.968+0000 7f79bf400640 1 -- 172.21.15.131:0/178963731 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79c00ae960 con 0x7f79c006b680 2024-05-26T22:53:30.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.969+0000 7f79bf400640 1 --2- 172.21.15.131:0/178963731 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f79c006b680 0x7f79c006bb10 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f79c0070750 tx=0x7f79b802a150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7a7b88ee9f05681 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:30.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.969+0000 7f79bd600640 1 -- 172.21.15.131:0/178963731 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b8037070 con 0x7f79c006b680 2024-05-26T22:53:30.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.969+0000 7f79bd600640 1 -- 172.21.15.131:0/178963731 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79b8032440 con 0x7f79c006b680 2024-05-26T22:53:30.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.969+0000 7f79bd600640 1 -- 172.21.15.131:0/178963731 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b8032810 con 0x7f79c006b680 2024-05-26T22:53:30.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.970+0000 7f79c5600640 1 -- 172.21.15.131:0/178963731 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f79c006b680 msgr2=0x7f79c006bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:30.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.970+0000 7f79c5600640 1 --2- 172.21.15.131:0/178963731 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f79c006b680 0x7f79c006bb10 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f79c0070750 tx=0x7f79b802a150 comp rx=0 tx=0).stop 2024-05-26T22:53:30.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.970+0000 7f79c5600640 1 -- 172.21.15.131:0/178963731 shutdown_connections 2024-05-26T22:53:30.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.970+0000 7f79c5600640 1 --2- 172.21.15.131:0/178963731 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f79c006b680 0x7f79c006bb10 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:30.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.970+0000 7f79c5600640 1 --2- 172.21.15.131:0/178963731 >> v2:172.21.15.187:3300/0 conn(0x7f79c00af580 0x7f79c015f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:30.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.970+0000 7f79c5600640 1 -- 172.21.15.131:0/178963731 >> 172.21.15.131:0/178963731 conn(0x7f79c0072b00 msgr2=0x7f79c006de50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:30.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.970+0000 7f79c5600640 1 -- 172.21.15.131:0/178963731 shutdown_connections 2024-05-26T22:53:30.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.970+0000 7f79c5600640 1 -- 172.21.15.131:0/178963731 wait complete. 2024-05-26T22:53:30.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.971+0000 7f79c5600640 1 Processor -- start 2024-05-26T22:53:30.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.972+0000 7f79c5600640 1 -- start start 2024-05-26T22:53:30.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.973+0000 7f79c5600640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f79c006b680 0x7f79c0139ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:30.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.973+0000 7f79c5600640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f79c00af580 0x7f79c014b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:30.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.973+0000 7f79c5600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f79c00af960 0x7f79c01f7800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:30.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.973+0000 7f79c5600640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f79c00b6a40 con 0x7f79c00af960 2024-05-26T22:53:30.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.973+0000 7f79c5600640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f79c00b68c0 con 0x7f79c00af580 2024-05-26T22:53:30.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.973+0000 7f79c5600640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f79c006c050 con 0x7f79c006b680 2024-05-26T22:53:30.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.973+0000 7f79bf400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f79c00af960 0x7f79c01f7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:30.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.973+0000 7f79bf400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f79c00af960 0x7f79c01f7800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49008/0 (socket says 172.21.15.131:49008) 2024-05-26T22:53:30.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.973+0000 7f79bf400640 1 -- 172.21.15.131:0/144798454 learned_addr learned my addr 172.21.15.131:0/144798454 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:53:30.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.973+0000 7f79bea00640 1 --2- 172.21.15.131:0/144798454 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f79c006b680 0x7f79c0139ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:30.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.973+0000 7f79bf400640 1 -- 172.21.15.131:0/144798454 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f79c006b680 msgr2=0x7f79c0139ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:30.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.973+0000 7f79be000640 1 --2- 172.21.15.131:0/144798454 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f79c00af580 0x7f79c014b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:30.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.973+0000 7f79bf400640 1 --2- 172.21.15.131:0/144798454 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f79c006b680 0x7f79c0139ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:30.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.973+0000 7f79bf400640 1 -- 172.21.15.131:0/144798454 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f79c00af580 msgr2=0x7f79c014b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:30.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.973+0000 7f79bf400640 1 --2- 172.21.15.131:0/144798454 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f79c00af580 0x7f79c014b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:30.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.973+0000 7f79bf400640 1 -- 172.21.15.131:0/144798454 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79c01f8170 con 0x7f79c00af960 2024-05-26T22:53:30.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.973+0000 7f79be000640 1 --2- 172.21.15.131:0/144798454 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f79c00af580 0x7f79c014b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:53:30.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.974+0000 7f79bf400640 1 --2- 172.21.15.131:0/144798454 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f79c00af960 0x7f79c01f7800 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f79b802a190 tx=0x7f79b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:30.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.974+0000 7f79b6a00640 1 -- 172.21.15.131:0/144798454 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b8041070 con 0x7f79c00af960 2024-05-26T22:53:30.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.974+0000 7f79b6a00640 1 -- 172.21.15.131:0/144798454 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79b803c870 con 0x7f79c00af960 2024-05-26T22:53:30.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.974+0000 7f79b6a00640 1 -- 172.21.15.131:0/144798454 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b8037040 con 0x7f79c00af960 2024-05-26T22:53:30.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.974+0000 7f79c5600640 1 -- 172.21.15.131:0/144798454 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79c01f8450 con 0x7f79c00af960 2024-05-26T22:53:30.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.974+0000 7f79c5600640 1 -- 172.21.15.131:0/144798454 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79c013a530 con 0x7f79c00af960 2024-05-26T22:53:30.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.975+0000 7f79c5600640 1 -- 172.21.15.131:0/144798454 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79c0074890 con 0x7f79c00af960 2024-05-26T22:53:30.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.976+0000 7f79b6a00640 1 -- 172.21.15.131:0/144798454 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f79b8044070 con 0x7f79c00af960 2024-05-26T22:53:30.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.978+0000 7f79b6a00640 1 --2- 172.21.15.131:0/144798454 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f799c06be30 0x7f799c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:30.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.978+0000 7f79b6a00640 1 -- 172.21.15.131:0/144798454 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5767+0+0 (secure 0 0 0) 0x7f79b80abb70 con 0x7f79c00af960 2024-05-26T22:53:30.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.978+0000 7f79bea00640 1 --2- 172.21.15.131:0/144798454 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f799c06be30 0x7f799c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:30.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.979+0000 7f79bea00640 1 --2- 172.21.15.131:0/144798454 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f799c06be30 0x7f799c06e2f0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f79b0006fd0 tx=0x7f79b0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:30.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:30.982+0000 7f79b6a00640 1 -- 172.21.15.131:0/144798454 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f79c0074890 con 0x7f79c00af960 2024-05-26T22:53:31.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.185+0000 7f79c5600640 1 -- 172.21.15.131:0/144798454 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f79c0067320 con 0x7f79c00af960 2024-05-26T22:53:31.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.186+0000 7f79b6a00640 1 -- 172.21.15.131:0/144798454 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v71) ==== 104+0+11 (secure 0 0 0) 0x7f79b807a5f0 con 0x7f79c00af960 2024-05-26T22:53:31.186 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-05-26T22:53:31.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.189+0000 7f79c5600640 1 -- 172.21.15.131:0/144798454 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f799c06be30 msgr2=0x7f799c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:31.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.189+0000 7f79c5600640 1 --2- 172.21.15.131:0/144798454 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f799c06be30 0x7f799c06e2f0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f79b0006fd0 tx=0x7f79b0008040 comp rx=0 tx=0).stop 2024-05-26T22:53:31.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.189+0000 7f79c5600640 1 -- 172.21.15.131:0/144798454 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f79c00af960 msgr2=0x7f79c01f7800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:31.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.189+0000 7f79c5600640 1 --2- 172.21.15.131:0/144798454 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f79c00af960 0x7f79c01f7800 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f79b802a190 tx=0x7f79b80046c0 comp rx=0 tx=0).stop 2024-05-26T22:53:31.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.189+0000 7f79c5600640 1 -- 172.21.15.131:0/144798454 shutdown_connections 2024-05-26T22:53:31.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.189+0000 7f79c5600640 1 --2- 172.21.15.131:0/144798454 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f799c06be30 0x7f799c06e2f0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:31.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.189+0000 7f79c5600640 1 --2- 172.21.15.131:0/144798454 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f79c00af960 0x7f79c01f7800 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:31.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.189+0000 7f79c5600640 1 --2- 172.21.15.131:0/144798454 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f79c00af580 0x7f79c014b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:31.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.189+0000 7f79c5600640 1 --2- 172.21.15.131:0/144798454 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f79c006b680 0x7f79c0139ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:31.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.189+0000 7f79c5600640 1 -- 172.21.15.131:0/144798454 >> 172.21.15.131:0/144798454 conn(0x7f79c0072b00 msgr2=0x7f79c015fc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:31.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.189+0000 7f79c5600640 1 -- 172.21.15.131:0/144798454 shutdown_connections 2024-05-26T22:53:31.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.189+0000 7f79c5600640 1 -- 172.21.15.131:0/144798454 wait complete. 2024-05-26T22:53:31.207 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-26T22:53:31.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.405+0000 7f9b94800640 1 Processor -- start 2024-05-26T22:53:31.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.405+0000 7f9b94800640 1 -- start start 2024-05-26T22:53:31.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.406+0000 7f9b94800640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f9b8c155770 0x7f9b8c15b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:31.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.407+0000 7f9b94800640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f9b8c06e440 0x7f9b8c0725a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:31.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.407+0000 7f9b94800640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b8c15c1c0 con 0x7f9b8c15bc40 2024-05-26T22:53:31.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.407+0000 7f9b94800640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f9b8c15c4a0 con 0x7f9b8c06e440 2024-05-26T22:53:31.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.407+0000 7f9b94800640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f9b8c071da0 con 0x7f9b8c155770 2024-05-26T22:53:31.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.407+0000 7f9b92800640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f9b8c06e440 0x7f9b8c0725a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:31.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.407+0000 7f9b92800640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f9b8c06e440 0x7f9b8c0725a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49024/0 (socket says 172.21.15.131:49024) 2024-05-26T22:53:31.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.407+0000 7f9b92800640 1 -- 172.21.15.131:0/604699155 learned_addr learned my addr 172.21.15.131:0/604699155 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:53:31.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.407+0000 7f9b92800640 1 -- 172.21.15.131:0/604699155 >> v2:172.21.15.187:3300/0 conn(0x7f9b8c155770 msgr2=0x7f9b8c15b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:31.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.407+0000 7f9b92800640 1 --2- 172.21.15.131:0/604699155 >> v2:172.21.15.187:3300/0 conn(0x7f9b8c155770 0x7f9b8c15b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:31.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.407+0000 7f9b92800640 1 -- 172.21.15.131:0/604699155 >> v1:172.21.15.187:6789/0 conn(0x7f9b8c15bc40 legacy=0x7f9b8c06dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:31.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.407+0000 7f9b92800640 1 -- 172.21.15.131:0/604699155 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b8c072080 con 0x7f9b8c06e440 2024-05-26T22:53:31.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.408+0000 7f9b92800640 1 --2- 172.21.15.131:0/604699155 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b8c06e440 0x7f9b8c0725a0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c071790 tx=0x7f9b8802c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d601b2c0bb1bb47 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:31.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.409+0000 7f9b90a00640 1 -- 172.21.15.131:0/604699155 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b8802e040 con 0x7f9b8c06e440 2024-05-26T22:53:31.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.409+0000 7f9b90a00640 1 -- 172.21.15.131:0/604699155 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b88037070 con 0x7f9b8c06e440 2024-05-26T22:53:31.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.409+0000 7f9b90a00640 1 -- 172.21.15.131:0/604699155 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b88009600 con 0x7f9b8c06e440 2024-05-26T22:53:31.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.409+0000 7f9b94800640 1 -- 172.21.15.131:0/604699155 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b8c06e440 msgr2=0x7f9b8c0725a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:31.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.409+0000 7f9b94800640 1 --2- 172.21.15.131:0/604699155 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b8c06e440 0x7f9b8c0725a0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c071790 tx=0x7f9b8802c990 comp rx=0 tx=0).stop 2024-05-26T22:53:31.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.409+0000 7f9b94800640 1 -- 172.21.15.131:0/604699155 shutdown_connections 2024-05-26T22:53:31.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.409+0000 7f9b94800640 1 --2- 172.21.15.131:0/604699155 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b8c06e440 0x7f9b8c0725a0 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:31.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.409+0000 7f9b94800640 1 --2- 172.21.15.131:0/604699155 >> v2:172.21.15.187:3300/0 conn(0x7f9b8c155770 0x7f9b8c15b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:31.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.409+0000 7f9b94800640 1 -- 172.21.15.131:0/604699155 >> 172.21.15.131:0/604699155 conn(0x7f9b8c06bc90 msgr2=0x7f9b8c0b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:31.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.409+0000 7f9b94800640 1 -- 172.21.15.131:0/604699155 shutdown_connections 2024-05-26T22:53:31.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.409+0000 7f9b94800640 1 -- 172.21.15.131:0/604699155 wait complete. 2024-05-26T22:53:31.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.411+0000 7f9b94800640 1 Processor -- start 2024-05-26T22:53:31.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.411+0000 7f9b94800640 1 -- start start 2024-05-26T22:53:31.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.412+0000 7f9b94800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b8c06e440 0x7f9b8c1e5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:31.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.412+0000 7f9b94800640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9b8c155770 0x7f9b8c152200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:31.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.412+0000 7f9b94800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9b8c15bc40 0x7f9b8c152740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:31.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.412+0000 7f9b94800640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f9b8c09de00 con 0x7f9b8c06e440 2024-05-26T22:53:31.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.412+0000 7f9b94800640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9b8c15c8c0 con 0x7f9b8c155770 2024-05-26T22:53:31.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.412+0000 7f9b94800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9b8c09df80 con 0x7f9b8c15bc40 2024-05-26T22:53:31.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.412+0000 7f9b91e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b8c06e440 0x7f9b8c1e5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:31.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.412+0000 7f9b91e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b8c06e440 0x7f9b8c1e5dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49028/0 (socket says 172.21.15.131:49028) 2024-05-26T22:53:31.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.412+0000 7f9b91e00640 1 -- 172.21.15.131:0/4181474380 learned_addr learned my addr 172.21.15.131:0/4181474380 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:53:31.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.413+0000 7f9b91e00640 1 -- 172.21.15.131:0/4181474380 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9b8c15bc40 msgr2=0x7f9b8c152740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:31.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.413+0000 7f9b91e00640 1 --2- 172.21.15.131:0/4181474380 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9b8c15bc40 0x7f9b8c152740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:31.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.413+0000 7f9b91e00640 1 -- 172.21.15.131:0/4181474380 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9b8c155770 msgr2=0x7f9b8c152200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:31.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.413+0000 7f9b91e00640 1 --2- 172.21.15.131:0/4181474380 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9b8c155770 0x7f9b8c152200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:31.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.413+0000 7f9b91e00640 1 -- 172.21.15.131:0/4181474380 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b8c1467b0 con 0x7f9b8c06e440 2024-05-26T22:53:31.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.413+0000 7f9b91e00640 1 --2- 172.21.15.131:0/4181474380 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b8c06e440 0x7f9b8c1e5dd0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f9b8000b4a0 tx=0x7f9b8000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:31.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.413+0000 7f9b86000640 1 -- 172.21.15.131:0/4181474380 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b80014070 con 0x7f9b8c06e440 2024-05-26T22:53:31.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.413+0000 7f9b86000640 1 -- 172.21.15.131:0/4181474380 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b80010610 con 0x7f9b8c06e440 2024-05-26T22:53:31.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.413+0000 7f9b94800640 1 -- 172.21.15.131:0/4181474380 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b8c1459f0 con 0x7f9b8c06e440 2024-05-26T22:53:31.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.413+0000 7f9b86000640 1 -- 172.21.15.131:0/4181474380 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b80010a00 con 0x7f9b8c06e440 2024-05-26T22:53:31.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.413+0000 7f9b94800640 1 -- 172.21.15.131:0/4181474380 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b8c145fb0 con 0x7f9b8c06e440 2024-05-26T22:53:31.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.415+0000 7f9b94800640 1 -- 172.21.15.131:0/4181474380 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b8c074e90 con 0x7f9b8c06e440 2024-05-26T22:53:31.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.415+0000 7f9b86000640 1 -- 172.21.15.131:0/4181474380 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9b80017070 con 0x7f9b8c06e440 2024-05-26T22:53:31.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.416+0000 7f9b86000640 1 --2- 172.21.15.131:0/4181474380 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9b7006be30 0x7f9b7006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:31.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.416+0000 7f9b86000640 1 -- 172.21.15.131:0/4181474380 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5767+0+0 (secure 0 0 0) 0x7f9b8008a960 con 0x7f9b8c06e440 2024-05-26T22:53:31.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.417+0000 7f9b91400640 1 --2- 172.21.15.131:0/4181474380 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9b7006be30 0x7f9b7006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:31.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.417+0000 7f9b91400640 1 --2- 172.21.15.131:0/4181474380 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9b7006be30 0x7f9b7006e2f0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c0b2150 tx=0x7f9b7c006ba0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:31.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.421+0000 7f9b86000640 1 -- 172.21.15.131:0/4181474380 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f9b80059230 con 0x7f9b8c06e440 2024-05-26T22:53:31.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.619+0000 7f9b94800640 1 -- 172.21.15.131:0/4181474380 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f9b8c0695c0 con 0x7f9b8c06e440 2024-05-26T22:53:31.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.620+0000 7f9b86000640 1 -- 172.21.15.131:0/4181474380 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v71) ==== 100+0+11 (secure 0 0 0) 0x7f9b80058b30 con 0x7f9b8c06e440 2024-05-26T22:53:31.620 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 64 2024-05-26T22:53:31.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.622+0000 7f9b94800640 1 -- 172.21.15.131:0/4181474380 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9b7006be30 msgr2=0x7f9b7006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:31.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.622+0000 7f9b94800640 1 --2- 172.21.15.131:0/4181474380 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9b7006be30 0x7f9b7006e2f0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c0b2150 tx=0x7f9b7c006ba0 comp rx=0 tx=0).stop 2024-05-26T22:53:31.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.622+0000 7f9b94800640 1 -- 172.21.15.131:0/4181474380 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b8c06e440 msgr2=0x7f9b8c1e5dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:31.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.622+0000 7f9b94800640 1 --2- 172.21.15.131:0/4181474380 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b8c06e440 0x7f9b8c1e5dd0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f9b8000b4a0 tx=0x7f9b8000b960 comp rx=0 tx=0).stop 2024-05-26T22:53:31.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.622+0000 7f9b94800640 1 -- 172.21.15.131:0/4181474380 shutdown_connections 2024-05-26T22:53:31.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.622+0000 7f9b94800640 1 --2- 172.21.15.131:0/4181474380 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9b8c15bc40 0x7f9b8c152740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:31.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.622+0000 7f9b94800640 1 --2- 172.21.15.131:0/4181474380 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9b7006be30 0x7f9b7006e2f0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:31.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.622+0000 7f9b94800640 1 --2- 172.21.15.131:0/4181474380 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9b8c155770 0x7f9b8c152200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:31.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.622+0000 7f9b94800640 1 --2- 172.21.15.131:0/4181474380 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b8c06e440 0x7f9b8c1e5dd0 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:31.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.622+0000 7f9b94800640 1 -- 172.21.15.131:0/4181474380 >> 172.21.15.131:0/4181474380 conn(0x7f9b8c06bc90 msgr2=0x7f9b8c0cfd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:31.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.623+0000 7f9b94800640 1 -- 172.21.15.131:0/4181474380 shutdown_connections 2024-05-26T22:53:31.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.623+0000 7f9b94800640 1 -- 172.21.15.131:0/4181474380 wait complete. 2024-05-26T22:53:31.642 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-26T22:53:31.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.839+0000 7f9a9d200640 1 Processor -- start 2024-05-26T22:53:31.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.840+0000 7f9a9d200640 1 -- start start 2024-05-26T22:53:31.841 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.841+0000 7f9a9d200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f9a9806ba20 0x7f9a981628f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:31.841 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.841+0000 7f9a9d200640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a98070560 con 0x7f9a98147b60 2024-05-26T22:53:31.841 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.841+0000 7f9a9d200640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a980ce700 con 0x7f9a980700d0 2024-05-26T22:53:31.841 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.841+0000 7f9a9d200640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f9a980cd940 con 0x7f9a9806ba20 2024-05-26T22:53:31.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.842+0000 7f9a96a00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f9a9806ba20 0x7f9a981628f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:31.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.842+0000 7f9a97400640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f9a98147b60 0x7f9a9806b310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:58584/0 (socket says 172.21.15.131:58584) 2024-05-26T22:53:31.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.842+0000 7f9a96a00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f9a9806ba20 0x7f9a981628f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49038/0 (socket says 172.21.15.131:49038) 2024-05-26T22:53:31.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.842+0000 7f9a97400640 1 -- 172.21.15.131:0/3750754940 learned_addr learned my addr 172.21.15.131:0/3750754940 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:53:31.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.842+0000 7f9a96a00640 1 -- 172.21.15.131:0/3750754940 >> v1:172.21.15.187:6789/0 conn(0x7f9a980700d0 legacy=0x7f9a980cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:31.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.842+0000 7f9a96000640 1 -- 172.21.15.131:0/3750754940 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1428816329 0 0) 0x7f9a88001490 con 0x7f9a98147b60 2024-05-26T22:53:31.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.842+0000 7f9a96a00640 1 -- 172.21.15.131:0/3750754940 >> v1:172.21.15.131:6789/0 conn(0x7f9a98147b60 legacy=0x7f9a9806b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:31.843 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.842+0000 7f9a96a00640 1 -- 172.21.15.131:0/3750754940 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a980cdc20 con 0x7f9a9806ba20 2024-05-26T22:53:31.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.843+0000 7f9a96a00640 1 --2- 172.21.15.131:0/3750754940 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9a9806ba20 0x7f9a981628f0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f9a800097a0 tx=0x7f9a8002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a5ee2b3ad720dec server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:31.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.843+0000 7f9a96000640 1 -- 172.21.15.131:0/3750754940 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a80037070 con 0x7f9a9806ba20 2024-05-26T22:53:31.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.844+0000 7f9a96000640 1 -- 172.21.15.131:0/3750754940 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a80032440 con 0x7f9a9806ba20 2024-05-26T22:53:31.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.844+0000 7f9a96000640 1 -- 172.21.15.131:0/3750754940 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a80032810 con 0x7f9a9806ba20 2024-05-26T22:53:31.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.844+0000 7f9a9d200640 1 -- 172.21.15.131:0/3750754940 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9a9806ba20 msgr2=0x7f9a981628f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:31.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.844+0000 7f9a9d200640 1 --2- 172.21.15.131:0/3750754940 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9a9806ba20 0x7f9a981628f0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f9a800097a0 tx=0x7f9a8002a1e0 comp rx=0 tx=0).stop 2024-05-26T22:53:31.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.844+0000 7f9a9d200640 1 -- 172.21.15.131:0/3750754940 shutdown_connections 2024-05-26T22:53:31.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.844+0000 7f9a9d200640 1 --2- 172.21.15.131:0/3750754940 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9a9806ba20 0x7f9a981628f0 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:31.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.844+0000 7f9a9d200640 1 -- 172.21.15.131:0/3750754940 >> 172.21.15.131:0/3750754940 conn(0x7f9a9806e4d0 msgr2=0x7f9a9806e8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:31.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.844+0000 7f9a9d200640 1 -- 172.21.15.131:0/3750754940 shutdown_connections 2024-05-26T22:53:31.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.844+0000 7f9a9d200640 1 -- 172.21.15.131:0/3750754940 wait complete. 2024-05-26T22:53:31.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.846+0000 7f9a9d200640 1 Processor -- start 2024-05-26T22:53:31.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.846+0000 7f9a9d200640 1 -- start start 2024-05-26T22:53:31.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.847+0000 7f9a9d200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9a9806ba20 0x7f9a98150810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:31.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.847+0000 7f9a9d200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9a980700d0 0x7f9a981f3370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:31.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.847+0000 7f9a9d200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9a98147b60 0x7f9a981f38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:31.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.847+0000 7f9a9d200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f9a98147f40 con 0x7f9a9806ba20 2024-05-26T22:53:31.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.847+0000 7f9a9d200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9a980ce1e0 con 0x7f9a98147b60 2024-05-26T22:53:31.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.847+0000 7f9a9d200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9a980707e0 con 0x7f9a980700d0 2024-05-26T22:53:31.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.847+0000 7f9a97400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9a9806ba20 0x7f9a98150810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:31.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.847+0000 7f9a97400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9a9806ba20 0x7f9a98150810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49042/0 (socket says 172.21.15.131:49042) 2024-05-26T22:53:31.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.847+0000 7f9a97400640 1 -- 172.21.15.131:0/767208515 learned_addr learned my addr 172.21.15.131:0/767208515 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:53:31.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.848+0000 7f9a97400640 1 -- 172.21.15.131:0/767208515 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9a980700d0 msgr2=0x7f9a981f3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:31.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.848+0000 7f9a97400640 1 --2- 172.21.15.131:0/767208515 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9a980700d0 0x7f9a981f3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:31.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.848+0000 7f9a97e00640 1 --2- 172.21.15.131:0/767208515 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9a98147b60 0x7f9a981f38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:31.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.848+0000 7f9a97400640 1 -- 172.21.15.131:0/767208515 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9a98147b60 msgr2=0x7f9a981f38b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:31.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.848+0000 7f9a97400640 1 --2- 172.21.15.131:0/767208515 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9a98147b60 0x7f9a981f38b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:31.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.848+0000 7f9a97400640 1 -- 172.21.15.131:0/767208515 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a9813dd30 con 0x7f9a9806ba20 2024-05-26T22:53:31.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.848+0000 7f9a97400640 1 --2- 172.21.15.131:0/767208515 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9a9806ba20 0x7f9a98150810 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f9a8800c7c0 tx=0x7f9a8800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:31.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.848+0000 7f9a8f400640 1 -- 172.21.15.131:0/767208515 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a88018070 con 0x7f9a9806ba20 2024-05-26T22:53:31.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.848+0000 7f9a8f400640 1 -- 172.21.15.131:0/767208515 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a8800a5e0 con 0x7f9a9806ba20 2024-05-26T22:53:31.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.848+0000 7f9a8f400640 1 -- 172.21.15.131:0/767208515 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a8800a9d0 con 0x7f9a9806ba20 2024-05-26T22:53:31.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.848+0000 7f9a9d200640 1 -- 172.21.15.131:0/767208515 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a9813e010 con 0x7f9a9806ba20 2024-05-26T22:53:31.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.848+0000 7f9a9d200640 1 -- 172.21.15.131:0/767208515 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a981e82e0 con 0x7f9a9806ba20 2024-05-26T22:53:31.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.849+0000 7f9a9d200640 1 -- 172.21.15.131:0/767208515 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a981e7e30 con 0x7f9a9806ba20 2024-05-26T22:53:31.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.850+0000 7f9a8f400640 1 -- 172.21.15.131:0/767208515 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9a8801b070 con 0x7f9a9806ba20 2024-05-26T22:53:31.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.851+0000 7f9a8f400640 1 --2- 172.21.15.131:0/767208515 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9a7406be30 0x7f9a7406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:31.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.851+0000 7f9a8f400640 1 -- 172.21.15.131:0/767208515 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5767+0+0 (secure 0 0 0) 0x7f9a8808a580 con 0x7f9a9806ba20 2024-05-26T22:53:31.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.852+0000 7f9a96a00640 1 --2- 172.21.15.131:0/767208515 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9a7406be30 0x7f9a7406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:31.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.852+0000 7f9a96a00640 1 --2- 172.21.15.131:0/767208515 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9a7406be30 0x7f9a7406e2f0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f9a80034000 tx=0x7f9a8002c000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:31.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:31.856+0000 7f9a8f400640 1 -- 172.21.15.131:0/767208515 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f9a98070560 con 0x7f9a9806ba20 2024-05-26T22:53:32.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:32.064+0000 7f9a9d200640 1 -- 172.21.15.131:0/767208515 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f9a980d1490 con 0x7f9a9806ba20 2024-05-26T22:53:32.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:32.065+0000 7f9a8f400640 1 -- 172.21.15.131:0/767208515 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v71) ==== 80+0+0 (secure 0 0 0) 0x7f9a88058e50 con 0x7f9a9806ba20 2024-05-26T22:53:32.065 INFO:teuthology.orchestra.run.smithi131.stderr:listed 0 entries 2024-05-26T22:53:32.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:32.068+0000 7f9a9d200640 1 -- 172.21.15.131:0/767208515 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9a7406be30 msgr2=0x7f9a7406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:32.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:32.068+0000 7f9a9d200640 1 --2- 172.21.15.131:0/767208515 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9a7406be30 0x7f9a7406e2f0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f9a80034000 tx=0x7f9a8002c000 comp rx=0 tx=0).stop 2024-05-26T22:53:32.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:32.068+0000 7f9a9d200640 1 -- 172.21.15.131:0/767208515 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9a9806ba20 msgr2=0x7f9a98150810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:32.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:32.068+0000 7f9a9d200640 1 --2- 172.21.15.131:0/767208515 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9a9806ba20 0x7f9a98150810 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f9a8800c7c0 tx=0x7f9a8800cc80 comp rx=0 tx=0).stop 2024-05-26T22:53:32.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:32.069+0000 7f9a9d200640 1 -- 172.21.15.131:0/767208515 shutdown_connections 2024-05-26T22:53:32.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:32.069+0000 7f9a9d200640 1 --2- 172.21.15.131:0/767208515 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9a980700d0 0x7f9a981f3370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:32.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:32.069+0000 7f9a9d200640 1 --2- 172.21.15.131:0/767208515 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9a7406be30 0x7f9a7406e2f0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:32.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:32.069+0000 7f9a9d200640 1 --2- 172.21.15.131:0/767208515 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9a98147b60 0x7f9a981f38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:32.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:32.069+0000 7f9a9d200640 1 --2- 172.21.15.131:0/767208515 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9a9806ba20 0x7f9a98150810 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:32.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:32.069+0000 7f9a9d200640 1 -- 172.21.15.131:0/767208515 >> 172.21.15.131:0/767208515 conn(0x7f9a9806e4d0 msgr2=0x7f9a980d07a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:32.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:32.069+0000 7f9a9d200640 1 -- 172.21.15.131:0/767208515 shutdown_connections 2024-05-26T22:53:32.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:32.069+0000 7f9a9d200640 1 -- 172.21.15.131:0/767208515 wait complete. 2024-05-26T22:53:32.086 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-26T22:53:32.086 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:53:32.087 DEBUG:teuthology.orchestra.run.smithi131:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-26T22:53:32.201 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi131.front.sepia.ceph.com 2024-05-26T22:53:32.201 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:53:32.201 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-26T22:53:32.247 INFO:tasks.cephfs.mount:Terminating background process 2024-05-26T22:53:32.247 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-05-26T22:53:32.247 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-26T22:53:32.285 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-26T22:53:32.286 INFO:tasks.cephfs.mount:Terminating background process 2024-05-26T22:53:32.286 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-26T22:53:32.286 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-26T22:53:32.324 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-26T22:53:32.324 INFO:tasks.cephfs.mount:Terminating background process 2024-05-26T22:53:32.324 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-05-26T22:53:32.325 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-26T22:53:32.368 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-26T22:53:32.368 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-26T22:53:32.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:32.555+0000 7f8825a00640 1 Processor -- start 2024-05-26T22:53:32.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:32.555+0000 7f8825a00640 1 -- start start 2024-05-26T22:53:32.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:32.557+0000 7f8825a00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f882006ee20 0x7f88200b3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:32.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:32.557+0000 7f8825a00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f882015cb20 con 0x7f88201569d0 2024-05-26T22:53:32.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:32.557+0000 7f8825a00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f882015ce00 con 0x7f88200ad250 2024-05-26T22:53:32.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:32.557+0000 7f8825a00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f882015d0e0 con 0x7f882006ee20 2024-05-26T22:53:32.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:32.557+0000 7f881f400640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f88201569d0 0x7f8820160010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:58596/0 (socket says 172.21.15.131:58596) 2024-05-26T22:53:32.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:32.557+0000 7f881f400640 1 -- 172.21.15.131:0/3711581028 learned_addr learned my addr 172.21.15.131:0/3711581028 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:53:32.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:32.557+0000 7f881d600640 1 -- 172.21.15.131:0/3711581028 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 48247178 0 0) 0x7f882015cb20 con 0x7f88201569d0 2024-05-26T22:53:32.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:32.557+0000 7f881ea00640 1 --2- 172.21.15.131:0/3711581028 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f882006ee20 0x7f88200b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:32.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:32.557+0000 7f881d600640 1 -- 172.21.15.131:0/3711581028 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8804003520 con 0x7f88201569d0 2024-05-26T22:53:32.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:32.558+0000 7f881d600640 1 -- 172.21.15.131:0/3711581028 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 507732468 0 0) 0x7f8804003520 con 0x7f88201569d0 2024-05-26T22:53:32.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:32.558+0000 7f881d600640 1 -- 172.21.15.131:0/3711581028 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f882006ee20 msgr2=0x7f88200b3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:32.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:32.558+0000 7f881d600640 1 --2- 172.21.15.131:0/3711581028 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f882006ee20 0x7f88200b3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:32.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:32.558+0000 7f881d600640 1 -- 172.21.15.131:0/3711581028 >> v1:172.21.15.187:6789/0 conn(0x7f88200ad250 legacy=0x7f88201562c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:32.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:32.558+0000 7f881d600640 1 -- 172.21.15.131:0/3711581028 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f882015d3c0 con 0x7f88201569d0 2024-05-26T22:53:32.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:32.558+0000 7f881d600640 1 -- 172.21.15.131:0/3711581028 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7f88180019f0 con 0x7f88201569d0 2024-05-26T22:53:32.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:32.558+0000 7f881d600640 1 -- 172.21.15.131:0/3711581028 >> v1:172.21.15.131:6789/0 conn(0x7f88201569d0 legacy=0x7f8820160010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:32.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:32.558+0000 7f881d600640 1 --2- 172.21.15.131:0/3711581028 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f88040039e0 0x7f8804003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:32.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:32.558+0000 7f881d600640 1 --2- 172.21.15.131:0/3711581028 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8804004800 0x7f8804004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:32.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:32.558+0000 7f881d600640 1 --2- 172.21.15.131:0/3711581028 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f882006ee20 0x7f88040051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:32.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:32.558+0000 7f881d600640 1 -- 172.21.15.131:0/3711581028 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f882015d0e0 con 0x7f8804004800 2024-05-26T22:53:32.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:32.558+0000 7f881d600640 1 -- 172.21.15.131:0/3711581028 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f880400ba70 con 0x7f88040039e0 2024-05-26T22:53:32.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:32.558+0000 7f881d600640 1 -- 172.21.15.131:0/3711581028 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f880400bc70 con 0x7f882006ee20 2024-05-26T22:53:32.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:32.558+0000 7f881e000640 1 --2- 172.21.15.131:0/3711581028 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8804004800 0x7f8804004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:32.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:32.559+0000 7f881ea00640 1 --2- 172.21.15.131:0/3711581028 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f88040039e0 0x7f8804003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:32.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:32.559+0000 7f881f400640 1 --2- 172.21.15.131:0/3711581028 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f882006ee20 0x7f88040051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:32.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:32.559+0000 7f881e000640 1 -- 172.21.15.131:0/3711581028 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f882006ee20 msgr2=0x7f88040051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:32.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:32.559+0000 7f881e000640 1 --2- 172.21.15.131:0/3711581028 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f882006ee20 0x7f88040051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:32.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:32.559+0000 7f881e000640 1 -- 172.21.15.131:0/3711581028 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f88040039e0 msgr2=0x7f8804003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:32.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:32.559+0000 7f881e000640 1 --2- 172.21.15.131:0/3711581028 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f88040039e0 0x7f8804003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:32.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:32.559+0000 7f881e000640 1 -- 172.21.15.131:0/3711581028 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8804003520 con 0x7f8804004800 2024-05-26T22:53:32.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:32.559+0000 7f881f400640 1 --2- 172.21.15.131:0/3711581028 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f882006ee20 0x7f88040051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:53:32.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:32.559+0000 7f881ea00640 1 --2- 172.21.15.131:0/3711581028 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f88040039e0 0x7f8804003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:53:32.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:32.560+0000 7f881e000640 1 --2- 172.21.15.131:0/3711581028 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8804004800 0x7f8804004c80 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f880c001720 tx=0x7f8804005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a39cf2539b617ba7 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:32.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:32.560+0000 7f881d600640 1 -- 172.21.15.131:0/3711581028 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f880c001370 con 0x7f8804004800 2024-05-26T22:53:32.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:32.560+0000 7f881d600640 1 -- 172.21.15.131:0/3711581028 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f880c0052d0 con 0x7f8804004800 2024-05-26T22:53:32.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:32.560+0000 7f881d600640 1 -- 172.21.15.131:0/3711581028 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f880c02ab90 con 0x7f8804004800 2024-05-26T22:53:32.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:32.561+0000 7f8825a00640 1 -- 172.21.15.131:0/3711581028 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8804004800 msgr2=0x7f8804004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:32.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:32.561+0000 7f8825a00640 1 --2- 172.21.15.131:0/3711581028 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8804004800 0x7f8804004c80 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f880c001720 tx=0x7f8804005830 comp rx=0 tx=0).stop 2024-05-26T22:53:32.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:32.561+0000 7f881f400640 1 -- 172.21.15.131:0/3711581028 reap_dead start 2024-05-26T22:53:32.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:32.561+0000 7f8825a00640 1 -- 172.21.15.131:0/3711581028 shutdown_connections 2024-05-26T22:53:32.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:32.561+0000 7f8825a00640 1 -- 172.21.15.131:0/3711581028 >> 172.21.15.131:0/3711581028 conn(0x7f88200b1e90 msgr2=0x7f88200b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:32.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:32.561+0000 7f8825a00640 1 -- 172.21.15.131:0/3711581028 shutdown_connections 2024-05-26T22:53:32.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:32.561+0000 7f8825a00640 1 -- 172.21.15.131:0/3711581028 wait complete. 2024-05-26T22:53:32.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:32.563+0000 7f8825a00640 1 Processor -- start 2024-05-26T22:53:32.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:32.563+0000 7f8825a00640 1 -- start start 2024-05-26T22:53:32.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:32.563+0000 7f8825a00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8804004800 0x7f88201f3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:32.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:32.564+0000 7f8825a00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8820150460 0x7f88201508e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:32.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:32.564+0000 7f8825a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f88201542c0 0x7f8820154770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:32.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:32.564+0000 7f8825a00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f882015c900 con 0x7f88201542c0 2024-05-26T22:53:32.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:32.564+0000 7f8825a00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f882015c780 con 0x7f8820150460 2024-05-26T22:53:32.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:32.564+0000 7f8825a00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f882006f200 con 0x7f8804004800 2024-05-26T22:53:32.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:32.564+0000 7f881f400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f88201542c0 0x7f8820154770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:32.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:32.564+0000 7f881f400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f88201542c0 0x7f8820154770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49054/0 (socket says 172.21.15.131:49054) 2024-05-26T22:53:32.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:32.564+0000 7f881f400640 1 -- 172.21.15.131:0/1009467782 learned_addr learned my addr 172.21.15.131:0/1009467782 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:53:32.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:32.564+0000 7f881ea00640 1 --2- 172.21.15.131:0/1009467782 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8804004800 0x7f88201f3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:32.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:32.564+0000 7f881e000640 1 --2- 172.21.15.131:0/1009467782 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8820150460 0x7f88201508e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:32.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:32.564+0000 7f881f400640 1 -- 172.21.15.131:0/1009467782 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8804004800 msgr2=0x7f88201f3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:32.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:32.564+0000 7f881f400640 1 --2- 172.21.15.131:0/1009467782 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8804004800 0x7f88201f3ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:32.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:32.564+0000 7f881f400640 1 -- 172.21.15.131:0/1009467782 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8820150460 msgr2=0x7f88201508e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:32.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:32.564+0000 7f881f400640 1 --2- 172.21.15.131:0/1009467782 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8820150460 0x7f88201508e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:32.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:32.564+0000 7f881f400640 1 -- 172.21.15.131:0/1009467782 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8820150f20 con 0x7f88201542c0 2024-05-26T22:53:32.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:32.565+0000 7f881f400640 1 --2- 172.21.15.131:0/1009467782 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f88201542c0 0x7f8820154770 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f8820156db0 tx=0x7f881800b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:32.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:32.565+0000 7f8816a00640 1 -- 172.21.15.131:0/1009467782 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8818004d60 con 0x7f88201542c0 2024-05-26T22:53:32.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:32.565+0000 7f8816a00640 1 -- 172.21.15.131:0/1009467782 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f881800bb80 con 0x7f88201542c0 2024-05-26T22:53:32.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:32.565+0000 7f8816a00640 1 -- 172.21.15.131:0/1009467782 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8818007110 con 0x7f88201542c0 2024-05-26T22:53:32.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:32.565+0000 7f8825a00640 1 -- 172.21.15.131:0/1009467782 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8820151200 con 0x7f88201542c0 2024-05-26T22:53:32.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:32.565+0000 7f8825a00640 1 -- 172.21.15.131:0/1009467782 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88201517c0 con 0x7f88201542c0 2024-05-26T22:53:32.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:32.566+0000 7f8825a00640 1 -- 172.21.15.131:0/1009467782 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88200694e0 con 0x7f88201542c0 2024-05-26T22:53:32.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:32.567+0000 7f8816a00640 1 -- 172.21.15.131:0/1009467782 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f881801b060 con 0x7f88201542c0 2024-05-26T22:53:32.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:32.568+0000 7f8816a00640 1 --2- 172.21.15.131:0/1009467782 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f880006be60 0x7f880006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:32.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:32.568+0000 7f8816a00640 1 -- 172.21.15.131:0/1009467782 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5767+0+0 (secure 0 0 0) 0x7f881808aaa0 con 0x7f88201542c0 2024-05-26T22:53:32.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:32.568+0000 7f881ea00640 1 --2- 172.21.15.131:0/1009467782 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f880006be60 0x7f880006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:32.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:32.569+0000 7f881ea00640 1 --2- 172.21.15.131:0/1009467782 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f880006be60 0x7f880006e320 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f88201f7cf0 tx=0x7f8810008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:32.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:32.573+0000 7f8816a00640 1 -- 172.21.15.131:0/1009467782 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f881800f230 con 0x7f88201542c0 2024-05-26T22:53:32.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:32.805+0000 7f8825a00640 1 -- 172.21.15.131:0/1009467782 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f88200b3ee0 con 0x7f88201542c0 2024-05-26T22:53:32.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:32.807+0000 7f8816a00640 1 -- 172.21.15.131:0/1009467782 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) ==== 94+0+5236 (secure 0 0 0) 0x7f8818059400 con 0x7f88201542c0 2024-05-26T22:53:32.807 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:53:32.807 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":69,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5359,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/4204396845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6833","nonce":4204396845},{"type":"v1","addr":"172.21.15.187:6835","nonce":4204396845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-26T22:52:53.755842+0000","modified":"2024-05-26T22:53:23.885672+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6876,"mds_1":6864,"mds_2":6861},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6861":{"gid":6861,"name":"a","rank":2,"incarnation":67,"state":"up:active","state_seq":6,"addr":"172.21.15.131:6836/4223534499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":4223534499},{"type":"v1","addr":"172.21.15.131:6836","nonce":4223534499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6864":{"gid":6864,"name":"c","rank":1,"incarnation":65,"state":"up:active","state_seq":6,"addr":"172.21.15.131:6837/1418741551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6835","nonce":1418741551},{"type":"v1","addr":"172.21.15.131:6837","nonce":1418741551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6876":{"gid":6876,"name":"d","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6834/3680999672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3680999672},{"type":"v1","addr":"172.21.15.187:6834","nonce":3680999672}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6876,"qdb_cluster":[6876,6864,6861]},"id":5}]} 2024-05-26T22:53:32.807 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 69 2024-05-26T22:53:32.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:32.809+0000 7f8825a00640 1 -- 172.21.15.131:0/1009467782 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f880006be60 msgr2=0x7f880006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:32.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:32.809+0000 7f8825a00640 1 --2- 172.21.15.131:0/1009467782 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f880006be60 0x7f880006e320 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f88201f7cf0 tx=0x7f8810008040 comp rx=0 tx=0).stop 2024-05-26T22:53:32.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:32.809+0000 7f8825a00640 1 -- 172.21.15.131:0/1009467782 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f88201542c0 msgr2=0x7f8820154770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:32.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:32.809+0000 7f8825a00640 1 --2- 172.21.15.131:0/1009467782 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f88201542c0 0x7f8820154770 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f8820156db0 tx=0x7f881800b4a0 comp rx=0 tx=0).stop 2024-05-26T22:53:32.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:32.810+0000 7f8825a00640 1 -- 172.21.15.131:0/1009467782 shutdown_connections 2024-05-26T22:53:32.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:32.810+0000 7f8825a00640 1 --2- 172.21.15.131:0/1009467782 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f880006be60 0x7f880006e320 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:32.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:32.810+0000 7f8825a00640 1 --2- 172.21.15.131:0/1009467782 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f88201542c0 0x7f8820154770 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:32.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:32.810+0000 7f8825a00640 1 --2- 172.21.15.131:0/1009467782 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8820150460 0x7f88201508e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:32.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:32.810+0000 7f8825a00640 1 --2- 172.21.15.131:0/1009467782 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8804004800 0x7f88201f3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:32.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:32.810+0000 7f8825a00640 1 -- 172.21.15.131:0/1009467782 >> 172.21.15.131:0/1009467782 conn(0x7f88200b1e90 msgr2=0x7f88200b46e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:32.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:32.810+0000 7f8825a00640 1 -- 172.21.15.131:0/1009467782 shutdown_connections 2024-05-26T22:53:32.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:32.810+0000 7f8825a00640 1 -- 172.21.15.131:0/1009467782 wait complete. 2024-05-26T22:53:32.827 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-26T22:53:33.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.021+0000 7f5bc2000640 1 Processor -- start 2024-05-26T22:53:33.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.021+0000 7f5bc2000640 1 -- start start 2024-05-26T22:53:33.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.022+0000 7f5bc2000640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f5bbc156750 0x7f5bbc15fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:33.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.022+0000 7f5bc2000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5bbc06e8c0 0x7f5bbc15d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:33.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.022+0000 7f5bc2000640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f5bbc0cc7f0 0x7f5bbc0ccc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:33.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.022+0000 7f5bc2000640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f5bbc0cd1a0 con 0x7f5bbc0cc7f0 2024-05-26T22:53:33.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.022+0000 7f5bc2000640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f5bbc15cbd0 con 0x7f5bbc156750 2024-05-26T22:53:33.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.022+0000 7f5bc2000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f5bbc160620 con 0x7f5bbc06e8c0 2024-05-26T22:53:33.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.023+0000 7f5bc1600640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f5bbc0cc7f0 0x7f5bbc0ccc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:33.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.023+0000 7f5bc1600640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f5bbc0cc7f0 0x7f5bbc0ccc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49056/0 (socket says 172.21.15.131:49056) 2024-05-26T22:53:33.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.023+0000 7f5bc1600640 1 -- 172.21.15.131:0/3382783122 learned_addr learned my addr 172.21.15.131:0/3382783122 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:53:33.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.023+0000 7f5bc0c00640 1 --2- 172.21.15.131:0/3382783122 >> v2:172.21.15.187:3300/0 conn(0x7f5bbc156750 0x7f5bbc15fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:33.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.023+0000 7f5bb3e00640 1 --2- 172.21.15.131:0/3382783122 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5bbc06e8c0 0x7f5bbc15d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:33.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.023+0000 7f5bc1600640 1 -- 172.21.15.131:0/3382783122 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5bbc06e8c0 msgr2=0x7f5bbc15d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:33.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.023+0000 7f5bc1600640 1 --2- 172.21.15.131:0/3382783122 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5bbc06e8c0 0x7f5bbc15d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:33.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.023+0000 7f5bc1600640 1 -- 172.21.15.131:0/3382783122 >> v2:172.21.15.187:3300/0 conn(0x7f5bbc156750 msgr2=0x7f5bbc15fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:33.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.023+0000 7f5bc1600640 1 --2- 172.21.15.131:0/3382783122 >> v2:172.21.15.187:3300/0 conn(0x7f5bbc156750 0x7f5bbc15fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:33.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.023+0000 7f5bc1600640 1 -- 172.21.15.131:0/3382783122 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bbc160900 con 0x7f5bbc0cc7f0 2024-05-26T22:53:33.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.023+0000 7f5bc0c00640 1 --2- 172.21.15.131:0/3382783122 >> v2:172.21.15.187:3300/0 conn(0x7f5bbc156750 0x7f5bbc15fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:53:33.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.024+0000 7f5bc1600640 1 --2- 172.21.15.131:0/3382783122 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5bbc0cc7f0 0x7f5bbc0ccc60 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc0cdb10 tx=0x7f5bb402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6c4953f0d35bd96 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:33.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.024+0000 7f5bbbe00640 1 -- 172.21.15.131:0/3382783122 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb4036070 con 0x7f5bbc0cc7f0 2024-05-26T22:53:33.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.025+0000 7f5bbbe00640 1 -- 172.21.15.131:0/3382783122 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bb4031600 con 0x7f5bbc0cc7f0 2024-05-26T22:53:33.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.025+0000 7f5bbbe00640 1 -- 172.21.15.131:0/3382783122 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb40319d0 con 0x7f5bbc0cc7f0 2024-05-26T22:53:33.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.025+0000 7f5bc2000640 1 -- 172.21.15.131:0/3382783122 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5bbc0cc7f0 msgr2=0x7f5bbc0ccc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:33.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.025+0000 7f5bc2000640 1 --2- 172.21.15.131:0/3382783122 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5bbc0cc7f0 0x7f5bbc0ccc60 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc0cdb10 tx=0x7f5bb402a6a0 comp rx=0 tx=0).stop 2024-05-26T22:53:33.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.025+0000 7f5bc2000640 1 -- 172.21.15.131:0/3382783122 shutdown_connections 2024-05-26T22:53:33.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.025+0000 7f5bc2000640 1 --2- 172.21.15.131:0/3382783122 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5bbc0cc7f0 0x7f5bbc0ccc60 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:33.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.025+0000 7f5bc2000640 1 --2- 172.21.15.131:0/3382783122 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5bbc06e8c0 0x7f5bbc15d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:33.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.025+0000 7f5bc2000640 1 --2- 172.21.15.131:0/3382783122 >> v2:172.21.15.187:3300/0 conn(0x7f5bbc156750 0x7f5bbc15fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:33.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.025+0000 7f5bc2000640 1 -- 172.21.15.131:0/3382783122 >> 172.21.15.131:0/3382783122 conn(0x7f5bbc0cff40 msgr2=0x7f5bbc0d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:33.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.025+0000 7f5bc2000640 1 -- 172.21.15.131:0/3382783122 shutdown_connections 2024-05-26T22:53:33.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.025+0000 7f5bc2000640 1 -- 172.21.15.131:0/3382783122 wait complete. 2024-05-26T22:53:33.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.027+0000 7f5bc2000640 1 Processor -- start 2024-05-26T22:53:33.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.027+0000 7f5bc2000640 1 -- start start 2024-05-26T22:53:33.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.028+0000 7f5bc2000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5bbc06e8c0 0x7f5bbc0aeec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:33.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.028+0000 7f5bc2000640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5bbc0cc7f0 0x7f5bbc13d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:33.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.028+0000 7f5bc2000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5bbc156750 0x7f5bbc158bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:33.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.028+0000 7f5bc2000640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f5bbc156b30 con 0x7f5bbc156750 2024-05-26T22:53:33.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.028+0000 7f5bc2000640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5bbc15c860 con 0x7f5bbc0cc7f0 2024-05-26T22:53:33.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.028+0000 7f5bc2000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f5bbc160280 con 0x7f5bbc06e8c0 2024-05-26T22:53:33.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.028+0000 7f5bc1600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5bbc156750 0x7f5bbc158bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:33.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.028+0000 7f5bc1600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5bbc156750 0x7f5bbc158bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49062/0 (socket says 172.21.15.131:49062) 2024-05-26T22:53:33.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.028+0000 7f5bc1600640 1 -- 172.21.15.131:0/2799287354 learned_addr learned my addr 172.21.15.131:0/2799287354 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:53:33.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.029+0000 7f5bb3e00640 1 --2- 172.21.15.131:0/2799287354 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5bbc0cc7f0 0x7f5bbc13d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:33.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.029+0000 7f5bc0c00640 1 --2- 172.21.15.131:0/2799287354 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5bbc06e8c0 0x7f5bbc0aeec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:33.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.029+0000 7f5bc1600640 1 -- 172.21.15.131:0/2799287354 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5bbc06e8c0 msgr2=0x7f5bbc0aeec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:33.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.029+0000 7f5bc1600640 1 --2- 172.21.15.131:0/2799287354 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5bbc06e8c0 0x7f5bbc0aeec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:33.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.029+0000 7f5bc1600640 1 -- 172.21.15.131:0/2799287354 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5bbc0cc7f0 msgr2=0x7f5bbc13d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:33.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.029+0000 7f5bc1600640 1 --2- 172.21.15.131:0/2799287354 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5bbc0cc7f0 0x7f5bbc13d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:33.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.029+0000 7f5bc1600640 1 -- 172.21.15.131:0/2799287354 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bbc13dac0 con 0x7f5bbc156750 2024-05-26T22:53:33.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.029+0000 7f5bc1600640 1 --2- 172.21.15.131:0/2799287354 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5bbc156750 0x7f5bbc158bf0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f5bb4009f40 tx=0x7f5bb4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:33.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.029+0000 7f5bb9600640 1 -- 172.21.15.131:0/2799287354 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb4036050 con 0x7f5bbc156750 2024-05-26T22:53:33.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.030+0000 7f5bb9600640 1 -- 172.21.15.131:0/2799287354 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bb4042070 con 0x7f5bbc156750 2024-05-26T22:53:33.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.030+0000 7f5bb9600640 1 -- 172.21.15.131:0/2799287354 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb4009510 con 0x7f5bbc156750 2024-05-26T22:53:33.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.030+0000 7f5bc2000640 1 -- 172.21.15.131:0/2799287354 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bbc159130 con 0x7f5bbc156750 2024-05-26T22:53:33.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.030+0000 7f5bc2000640 1 -- 172.21.15.131:0/2799287354 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bbc1596d0 con 0x7f5bbc156750 2024-05-26T22:53:33.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.031+0000 7f5bc2000640 1 -- 172.21.15.131:0/2799287354 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bbc0694e0 con 0x7f5bbc156750 2024-05-26T22:53:33.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.032+0000 7f5bb9600640 1 -- 172.21.15.131:0/2799287354 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5bb4007bd0 con 0x7f5bbc156750 2024-05-26T22:53:33.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.033+0000 7f5bb9600640 1 --2- 172.21.15.131:0/2799287354 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f5b9806be30 0x7f5b9806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:33.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.033+0000 7f5bb9600640 1 -- 172.21.15.131:0/2799287354 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5767+0+0 (secure 0 0 0) 0x7f5bb40abea0 con 0x7f5bbc156750 2024-05-26T22:53:33.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.033+0000 7f5bc0c00640 1 --2- 172.21.15.131:0/2799287354 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f5b9806be30 0x7f5b9806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:33.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.034+0000 7f5bc0c00640 1 --2- 172.21.15.131:0/2799287354 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f5b9806be30 0x7f5b9806e2f0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f5bac00a8d0 tx=0x7f5bac008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:33.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.037+0000 7f5bb9600640 1 -- 172.21.15.131:0/2799287354 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f5bb4045360 con 0x7f5bbc156750 2024-05-26T22:53:33.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.239+0000 7f5bc2000640 1 -- 172.21.15.131:0/2799287354 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5bbc06c320 con 0x7f5bbc156750 2024-05-26T22:53:33.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.242+0000 7f5bb9600640 1 -- 172.21.15.131:0/2799287354 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) ==== 74+0+16789 (secure 0 0 0) 0x7f5bb407a6e0 con 0x7f5bbc156750 2024-05-26T22:53:33.734 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:53:33.735 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":71,"fsid":"46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f","created":"2024-05-26T22:47:57.288905+0000","modified":"2024-05-26T22:53:06.170553+0000","last_up_change":"2024-05-26T22:48:09.602207+0000","last_in_change":"2024-05-26T22:48:00.687341+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-26T22:48:10.463339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-05-26T22:52:51.536251+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-05-26T22:52:52.768524+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"814eaf62-a18d-494b-8e27-fd8fb88adea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6801","nonce":2350940051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6803","nonce":2350940051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6807","nonce":2350940051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6805","nonce":2350940051}]},"public_addr":"172.21.15.131:6801/2350940051","cluster_addr":"172.21.15.131:6803/2350940051","heartbeat_back_addr":"172.21.15.131:6807/2350940051","heartbeat_front_addr":"172.21.15.131:6805/2350940051","state":["exists","up"]},{"osd":1,"uuid":"42e66287-f6c6-4e26-af3c-6dfaa3536d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6825","nonce":1525528247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6827","nonce":1525528247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6831","nonce":1525528247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6829","nonce":1525528247}]},"public_addr":"172.21.15.131:6825/1525528247","cluster_addr":"172.21.15.131:6827/1525528247","heartbeat_back_addr":"172.21.15.131:6831/1525528247","heartbeat_front_addr":"172.21.15.131:6829/1525528247","state":["exists","up"]},{"osd":2,"uuid":"0da34673-04a2-4fd0-b57a-0ff2b679b043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6817","nonce":3627842210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6819","nonce":3627842210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6823","nonce":3627842210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6821","nonce":3627842210}]},"public_addr":"172.21.15.131:6817/3627842210","cluster_addr":"172.21.15.131:6819/3627842210","heartbeat_back_addr":"172.21.15.131:6823/3627842210","heartbeat_front_addr":"172.21.15.131:6821/3627842210","state":["exists","up"]},{"osd":3,"uuid":"cf117a27-d6b2-43d3-ae52-b47988bf8eed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6809","nonce":350790591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6811","nonce":350790591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6815","nonce":350790591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6813","nonce":350790591}]},"public_addr":"172.21.15.131:6809/350790591","cluster_addr":"172.21.15.131:6811/350790591","heartbeat_back_addr":"172.21.15.131:6815/350790591","heartbeat_front_addr":"172.21.15.131:6813/350790591","state":["exists","up"]},{"osd":4,"uuid":"a092ccfc-be89-4a84-88d5-0f781fa715eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6816","nonce":244560995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6819","nonce":244560995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6823","nonce":244560995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6821","nonce":244560995}]},"public_addr":"172.21.15.187:6816/244560995","cluster_addr":"172.21.15.187:6819/244560995","heartbeat_back_addr":"172.21.15.187:6823/244560995","heartbeat_front_addr":"172.21.15.187:6821/244560995","state":["exists","up"]},{"osd":5,"uuid":"7513ba69-dd00-4a35-b190-9e8cc3b2bdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6801","nonce":324619398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6803","nonce":324619398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6812","nonce":324619398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6807","nonce":324619398}]},"public_addr":"172.21.15.187:6801/324619398","cluster_addr":"172.21.15.187:6803/324619398","heartbeat_back_addr":"172.21.15.187:6812/324619398","heartbeat_front_addr":"172.21.15.187:6807/324619398","state":["exists","up"]},{"osd":6,"uuid":"50081810-c093-4d2f-a4d7-b7c695c5400d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6806","nonce":1391347786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6809","nonce":1391347786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6818","nonce":1391347786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6813","nonce":1391347786}]},"public_addr":"172.21.15.187:6806/1391347786","cluster_addr":"172.21.15.187:6809/1391347786","heartbeat_back_addr":"172.21.15.187:6818/1391347786","heartbeat_front_addr":"172.21.15.187:6813/1391347786","state":["exists","up"]},{"osd":7,"uuid":"775dae05-54f5-4ad4-bbce-92863c6ed2ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6825","nonce":1220065622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6827","nonce":1220065622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6831","nonce":1220065622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6829","nonce":1220065622}]},"public_addr":"172.21.15.187:6825/1220065622","cluster_addr":"172.21.15.187:6827/1220065622","heartbeat_back_addr":"172.21.15.187:6831/1220065622","heartbeat_front_addr":"172.21.15.187:6829/1220065622","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:05.983520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.299516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.067323+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.307748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.018742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.258775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.010912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.295355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.a","mappings":[{"from":6,"to":0}]},{"pgid":"11.22","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-26T22:53:33.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.244+0000 7f5bc2000640 1 -- 172.21.15.131:0/2799287354 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f5b9806be30 msgr2=0x7f5b9806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:33.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.244+0000 7f5bc2000640 1 --2- 172.21.15.131:0/2799287354 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f5b9806be30 0x7f5b9806e2f0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f5bac00a8d0 tx=0x7f5bac008040 comp rx=0 tx=0).stop 2024-05-26T22:53:33.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.245+0000 7f5bc2000640 1 -- 172.21.15.131:0/2799287354 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5bbc156750 msgr2=0x7f5bbc158bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:33.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.245+0000 7f5bc2000640 1 --2- 172.21.15.131:0/2799287354 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5bbc156750 0x7f5bbc158bf0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f5bb4009f40 tx=0x7f5bb4004a40 comp rx=0 tx=0).stop 2024-05-26T22:53:33.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.245+0000 7f5bc2000640 1 -- 172.21.15.131:0/2799287354 shutdown_connections 2024-05-26T22:53:33.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.245+0000 7f5bc2000640 1 --2- 172.21.15.131:0/2799287354 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f5b9806be30 0x7f5b9806e2f0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:33.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.245+0000 7f5bc2000640 1 --2- 172.21.15.131:0/2799287354 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5bbc156750 0x7f5bbc158bf0 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:33.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.245+0000 7f5bc2000640 1 --2- 172.21.15.131:0/2799287354 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5bbc0cc7f0 0x7f5bbc13d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:33.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.245+0000 7f5bc2000640 1 --2- 172.21.15.131:0/2799287354 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5bbc06e8c0 0x7f5bbc0aeec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:33.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.245+0000 7f5bc2000640 1 -- 172.21.15.131:0/2799287354 >> 172.21.15.131:0/2799287354 conn(0x7f5bbc0cff40 msgr2=0x7f5bbc0d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:33.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.245+0000 7f5bc2000640 1 -- 172.21.15.131:0/2799287354 shutdown_connections 2024-05-26T22:53:33.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.245+0000 7f5bc2000640 1 -- 172.21.15.131:0/2799287354 wait complete. 2024-05-26T22:53:33.737 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-26T22:48:10.463339+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-26T22:52:51.536251+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '68', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-05-26T22:52:52.768524+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '69', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-26T22:53:33.738 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-26T22:53:33.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.952+0000 7f5943e00640 1 Processor -- start 2024-05-26T22:53:33.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.952+0000 7f5943e00640 1 -- start start 2024-05-26T22:53:33.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.954+0000 7f5943e00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f594415b730 0x7f594415bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:33.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.954+0000 7f5943e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f59441400c0 0x7f59441404f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:33.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.954+0000 7f5943e00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f594413e820 con 0x7f594406d250 2024-05-26T22:53:33.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.954+0000 7f5943e00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f5944071260 con 0x7f594415b730 2024-05-26T22:53:33.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.954+0000 7f5943e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f594415c4a0 con 0x7f59441400c0 2024-05-26T22:53:33.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.954+0000 7f5942a00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f594406d250 0x7f594406d630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:58602/0 (socket says 172.21.15.131:58602) 2024-05-26T22:53:33.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.954+0000 7f5942a00640 1 -- 172.21.15.131:0/2225450501 learned_addr learned my addr 172.21.15.131:0/2225450501 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:53:33.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.954+0000 7f5943400640 1 --2- 172.21.15.131:0/2225450501 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f59441400c0 0x7f59441404f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:33.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.954+0000 7f5942000640 1 --2- 172.21.15.131:0/2225450501 >> v2:172.21.15.187:3300/0 conn(0x7f594415b730 0x7f594415bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:33.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.955+0000 7f5941600640 1 -- 172.21.15.131:0/2225450501 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3153281123 0 0) 0x7f594413e820 con 0x7f594406d250 2024-05-26T22:53:33.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.955+0000 7f5941600640 1 -- 172.21.15.131:0/2225450501 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5928003520 con 0x7f594406d250 2024-05-26T22:53:33.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.955+0000 7f5941600640 1 -- 172.21.15.131:0/2225450501 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3724336324 0 0) 0x7f5928003520 con 0x7f594406d250 2024-05-26T22:53:33.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.955+0000 7f5941600640 1 -- 172.21.15.131:0/2225450501 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f59441400c0 msgr2=0x7f59441404f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:33.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.955+0000 7f5941600640 1 --2- 172.21.15.131:0/2225450501 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f59441400c0 0x7f59441404f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:33.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.955+0000 7f5941600640 1 -- 172.21.15.131:0/2225450501 >> v2:172.21.15.187:3300/0 conn(0x7f594415b730 msgr2=0x7f594415bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:33.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.955+0000 7f5941600640 1 --2- 172.21.15.131:0/2225450501 >> v2:172.21.15.187:3300/0 conn(0x7f594415b730 0x7f594415bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:33.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.955+0000 7f5941600640 1 -- 172.21.15.131:0/2225450501 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59440714b0 con 0x7f594406d250 2024-05-26T22:53:33.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.955+0000 7f5941600640 1 -- 172.21.15.131:0/2225450501 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7f5934001670 con 0x7f594406d250 2024-05-26T22:53:33.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.955+0000 7f5941600640 1 -- 172.21.15.131:0/2225450501 >> v1:172.21.15.131:6789/0 conn(0x7f594406d250 legacy=0x7f594406d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:33.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.955+0000 7f5941600640 1 --2- 172.21.15.131:0/2225450501 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59280039b0 0x7f5928003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:33.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.955+0000 7f5941600640 1 --2- 172.21.15.131:0/2225450501 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5928004800 0x7f5928004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:33.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.955+0000 7f5941600640 1 --2- 172.21.15.131:0/2225450501 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f59280051c0 0x7f592800dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:33.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.955+0000 7f5941600640 1 -- 172.21.15.131:0/2225450501 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f5944071260 con 0x7f59280039b0 2024-05-26T22:53:33.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.955+0000 7f5941600640 1 -- 172.21.15.131:0/2225450501 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f594415c4a0 con 0x7f5928004800 2024-05-26T22:53:33.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.955+0000 7f5941600640 1 -- 172.21.15.131:0/2225450501 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f592800e110 con 0x7f59280051c0 2024-05-26T22:53:33.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.955+0000 7f5943400640 1 --2- 172.21.15.131:0/2225450501 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f59441400c0 0x7f59441404f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-26T22:53:33.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.956+0000 7f5942a00640 1 --2- 172.21.15.131:0/2225450501 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59280039b0 0x7f5928003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:33.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.956+0000 7f5942000640 1 --2- 172.21.15.131:0/2225450501 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5928004800 0x7f5928004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:33.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.956+0000 7f5942a00640 1 -- 172.21.15.131:0/2225450501 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f59280051c0 msgr2=0x7f592800dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:33.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.956+0000 7f5943400640 1 --2- 172.21.15.131:0/2225450501 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f59280051c0 0x7f592800dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:33.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.956+0000 7f5942a00640 1 --2- 172.21.15.131:0/2225450501 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f59280051c0 0x7f592800dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:33.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.956+0000 7f5942a00640 1 -- 172.21.15.131:0/2225450501 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5928004800 msgr2=0x7f5928004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:33.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.956+0000 7f5942a00640 1 --2- 172.21.15.131:0/2225450501 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5928004800 0x7f5928004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:33.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.956+0000 7f5942a00640 1 -- 172.21.15.131:0/2225450501 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5928003520 con 0x7f59280039b0 2024-05-26T22:53:33.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.956+0000 7f5942000640 1 --2- 172.21.15.131:0/2225450501 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5928004800 0x7f5928004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:53:33.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.957+0000 7f5942a00640 1 --2- 172.21.15.131:0/2225450501 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59280039b0 0x7f5928003dd0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f593400c2c0 tx=0x7f5928005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea9059bd2ba96a5f server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:33.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.957+0000 7f5941600640 1 -- 172.21.15.131:0/2225450501 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5934007d50 con 0x7f59280039b0 2024-05-26T22:53:33.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.957+0000 7f5941600640 1 -- 172.21.15.131:0/2225450501 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f593402e070 con 0x7f59280039b0 2024-05-26T22:53:33.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.957+0000 7f5941600640 1 -- 172.21.15.131:0/2225450501 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5934005d30 con 0x7f59280039b0 2024-05-26T22:53:33.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.958+0000 7f5943e00640 1 -- 172.21.15.131:0/2225450501 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59280039b0 msgr2=0x7f5928003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:33.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.958+0000 7f5943e00640 1 --2- 172.21.15.131:0/2225450501 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59280039b0 0x7f5928003dd0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f593400c2c0 tx=0x7f5928005830 comp rx=0 tx=0).stop 2024-05-26T22:53:33.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.958+0000 7f5943400640 1 -- 172.21.15.131:0/2225450501 reap_dead start 2024-05-26T22:53:33.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.958+0000 7f5943e00640 1 -- 172.21.15.131:0/2225450501 shutdown_connections 2024-05-26T22:53:33.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.958+0000 7f5943e00640 1 -- 172.21.15.131:0/2225450501 >> 172.21.15.131:0/2225450501 conn(0x7f59440b5660 msgr2=0x7f59440b5a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:33.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.958+0000 7f5943e00640 1 -- 172.21.15.131:0/2225450501 shutdown_connections 2024-05-26T22:53:33.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.958+0000 7f5943e00640 1 -- 172.21.15.131:0/2225450501 wait complete. 2024-05-26T22:53:33.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.960+0000 7f5943e00640 1 Processor -- start 2024-05-26T22:53:33.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.960+0000 7f5943e00640 1 -- start start 2024-05-26T22:53:33.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.961+0000 7f5943e00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f59280039b0 0x7f59440d0470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:33.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.961+0000 7f5943e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5944155d50 0x7f59441376b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:33.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.961+0000 7f5943e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59441579d0 0x7f5944157e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:33.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.961+0000 7f5943e00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f594415c100 con 0x7f59441579d0 2024-05-26T22:53:33.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.961+0000 7f5943e00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5944071a70 con 0x7f59280039b0 2024-05-26T22:53:33.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.961+0000 7f5943e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f594415c280 con 0x7f5944155d50 2024-05-26T22:53:33.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.961+0000 7f5943400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59441579d0 0x7f5944157e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:33.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.961+0000 7f5943400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59441579d0 0x7f5944157e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49076/0 (socket says 172.21.15.131:49076) 2024-05-26T22:53:33.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.961+0000 7f5943400640 1 -- 172.21.15.131:0/2703099110 learned_addr learned my addr 172.21.15.131:0/2703099110 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:53:33.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.961+0000 7f5942a00640 1 --2- 172.21.15.131:0/2703099110 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f59280039b0 0x7f59440d0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:33.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.962+0000 7f5943400640 1 -- 172.21.15.131:0/2703099110 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5944155d50 msgr2=0x7f59441376b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:33.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.962+0000 7f5943400640 1 --2- 172.21.15.131:0/2703099110 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5944155d50 0x7f59441376b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:33.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.962+0000 7f5943400640 1 -- 172.21.15.131:0/2703099110 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f59280039b0 msgr2=0x7f59440d0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:33.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.962+0000 7f5943400640 1 --2- 172.21.15.131:0/2703099110 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f59280039b0 0x7f59440d0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:33.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.962+0000 7f5943400640 1 -- 172.21.15.131:0/2703099110 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5944137bf0 con 0x7f59441579d0 2024-05-26T22:53:33.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.962+0000 7f5943400640 1 --2- 172.21.15.131:0/2703099110 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59441579d0 0x7f5944157e50 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f59441404a0 tx=0x7f593c00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:33.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.962+0000 7f593aa00640 1 -- 172.21.15.131:0/2703099110 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f593c01c070 con 0x7f59441579d0 2024-05-26T22:53:33.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.962+0000 7f593aa00640 1 -- 172.21.15.131:0/2703099110 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f593c017400 con 0x7f59441579d0 2024-05-26T22:53:33.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.962+0000 7f5943e00640 1 -- 172.21.15.131:0/2703099110 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59441562c0 con 0x7f59441579d0 2024-05-26T22:53:33.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.962+0000 7f593aa00640 1 -- 172.21.15.131:0/2703099110 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f593c0177f0 con 0x7f59441579d0 2024-05-26T22:53:33.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.962+0000 7f5943e00640 1 -- 172.21.15.131:0/2703099110 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59440ce990 con 0x7f59441579d0 2024-05-26T22:53:33.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.964+0000 7f5943e00640 1 -- 172.21.15.131:0/2703099110 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59440694e0 con 0x7f59441579d0 2024-05-26T22:53:33.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.965+0000 7f593aa00640 1 -- 172.21.15.131:0/2703099110 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f593c01f070 con 0x7f59441579d0 2024-05-26T22:53:33.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.966+0000 7f593aa00640 1 --2- 172.21.15.131:0/2703099110 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f592406be30 0x7f592406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:33.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.966+0000 7f593aa00640 1 -- 172.21.15.131:0/2703099110 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5767+0+0 (secure 0 0 0) 0x7f593c08b080 con 0x7f59441579d0 2024-05-26T22:53:33.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.966+0000 7f5942a00640 1 --2- 172.21.15.131:0/2703099110 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f592406be30 0x7f592406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:33.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.967+0000 7f5942a00640 1 --2- 172.21.15.131:0/2703099110 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f592406be30 0x7f592406e2f0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f5934033000 tx=0x7f5934009030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:33.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:33.970+0000 7f593aa00640 1 -- 172.21.15.131:0/2703099110 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f593c012070 con 0x7f59441579d0 2024-05-26T22:53:34.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.168+0000 7f5943e00640 1 -- 172.21.15.131:0/2703099110 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f59440af9b0 con 0x7f59441579d0 2024-05-26T22:53:34.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.169+0000 7f593aa00640 1 -- 172.21.15.131:0/2703099110 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v71) ==== 93+0+10 (secure 0 0 0) 0x7f593c0599f0 con 0x7f59441579d0 2024-05-26T22:53:34.169 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2024-05-26T22:53:34.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.172+0000 7f5943e00640 1 -- 172.21.15.131:0/2703099110 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f592406be30 msgr2=0x7f592406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:34.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.172+0000 7f5943e00640 1 --2- 172.21.15.131:0/2703099110 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f592406be30 0x7f592406e2f0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f5934033000 tx=0x7f5934009030 comp rx=0 tx=0).stop 2024-05-26T22:53:34.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.172+0000 7f5943e00640 1 -- 172.21.15.131:0/2703099110 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59441579d0 msgr2=0x7f5944157e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:34.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.172+0000 7f5943e00640 1 --2- 172.21.15.131:0/2703099110 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59441579d0 0x7f5944157e50 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f59441404a0 tx=0x7f593c00f790 comp rx=0 tx=0).stop 2024-05-26T22:53:34.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.172+0000 7f5943e00640 1 -- 172.21.15.131:0/2703099110 shutdown_connections 2024-05-26T22:53:34.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.172+0000 7f5943e00640 1 --2- 172.21.15.131:0/2703099110 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5944155d50 0x7f59441376b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:34.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.172+0000 7f5943e00640 1 --2- 172.21.15.131:0/2703099110 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f592406be30 0x7f592406e2f0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:34.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.172+0000 7f5943e00640 1 --2- 172.21.15.131:0/2703099110 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59441579d0 0x7f5944157e50 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:34.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.172+0000 7f5943e00640 1 --2- 172.21.15.131:0/2703099110 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f59280039b0 0x7f59440d0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:34.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.172+0000 7f5943e00640 1 -- 172.21.15.131:0/2703099110 >> 172.21.15.131:0/2703099110 conn(0x7f59440b5660 msgr2=0x7f59440ad870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:34.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.172+0000 7f5943e00640 1 -- 172.21.15.131:0/2703099110 shutdown_connections 2024-05-26T22:53:34.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.172+0000 7f5943e00640 1 -- 172.21.15.131:0/2703099110 wait complete. 2024-05-26T22:53:34.190 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-26T22:53:34.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.401+0000 7f1e87200640 1 Processor -- start 2024-05-26T22:53:34.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.401+0000 7f1e87200640 1 -- start start 2024-05-26T22:53:34.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.403+0000 7f1e87200640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f1e8806d250 0x7f1e8806d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:34.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.403+0000 7f1e87200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f1e8815b730 0x7f1e8815bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:34.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.403+0000 7f1e87200640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e8806db70 con 0x7f1e88071f90 2024-05-26T22:53:34.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.403+0000 7f1e87200640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f1e88071260 con 0x7f1e8815b730 2024-05-26T22:53:34.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.403+0000 7f1e87200640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f1e8813e820 con 0x7f1e8806d250 2024-05-26T22:53:34.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.403+0000 7f1e85400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f1e8815b730 0x7f1e8815bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:34.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.403+0000 7f1e86800640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f1e88071f90 0x7f1e880723c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:47670/0 (socket says 172.21.15.131:47670) 2024-05-26T22:53:34.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.403+0000 7f1e86800640 1 -- 172.21.15.131:0/2504853992 learned_addr learned my addr 172.21.15.131:0/2504853992 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:53:34.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.403+0000 7f1e85400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f1e8815b730 0x7f1e8815bbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50894/0 (socket says 172.21.15.131:50894) 2024-05-26T22:53:34.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.403+0000 7f1e85e00640 1 --2- 172.21.15.131:0/2504853992 >> v2:172.21.15.187:3300/0 conn(0x7f1e8806d250 0x7f1e8806d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:34.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.403+0000 7f1e85400640 1 -- 172.21.15.131:0/2504853992 >> v2:172.21.15.187:3300/0 conn(0x7f1e8806d250 msgr2=0x7f1e8806d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:34.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.404+0000 7f1e85400640 1 --2- 172.21.15.131:0/2504853992 >> v2:172.21.15.187:3300/0 conn(0x7f1e8806d250 0x7f1e8806d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:34.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.403+0000 7f1e84a00640 1 -- 172.21.15.131:0/2504853992 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3750246253 0 0) 0x7f1e8806db70 con 0x7f1e88071f90 2024-05-26T22:53:34.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.404+0000 7f1e85400640 1 -- 172.21.15.131:0/2504853992 >> v1:172.21.15.131:6789/0 conn(0x7f1e88071f90 legacy=0x7f1e880723c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:34.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.404+0000 7f1e85400640 1 -- 172.21.15.131:0/2504853992 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e8815c4a0 con 0x7f1e8815b730 2024-05-26T22:53:34.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.404+0000 7f1e85e00640 1 --2- 172.21.15.131:0/2504853992 >> v2:172.21.15.187:3300/0 conn(0x7f1e8806d250 0x7f1e8806d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:53:34.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.405+0000 7f1e85400640 1 --2- 172.21.15.131:0/2504853992 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1e8815b730 0x7f1e8815bbc0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f1e8806f3c0 tx=0x7f1e7402c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9acda18b791d7f28 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:34.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.405+0000 7f1e84a00640 1 -- 172.21.15.131:0/2504853992 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e74032770 con 0x7f1e8815b730 2024-05-26T22:53:34.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.405+0000 7f1e84a00640 1 -- 172.21.15.131:0/2504853992 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e74038070 con 0x7f1e8815b730 2024-05-26T22:53:34.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.405+0000 7f1e84a00640 1 -- 172.21.15.131:0/2504853992 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e74032b30 con 0x7f1e8815b730 2024-05-26T22:53:34.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.406+0000 7f1e87200640 1 -- 172.21.15.131:0/2504853992 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1e8815b730 msgr2=0x7f1e8815bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:34.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.406+0000 7f1e87200640 1 --2- 172.21.15.131:0/2504853992 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1e8815b730 0x7f1e8815bbc0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f1e8806f3c0 tx=0x7f1e7402c870 comp rx=0 tx=0).stop 2024-05-26T22:53:34.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.406+0000 7f1e87200640 1 -- 172.21.15.131:0/2504853992 shutdown_connections 2024-05-26T22:53:34.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.406+0000 7f1e87200640 1 --2- 172.21.15.131:0/2504853992 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1e8815b730 0x7f1e8815bbc0 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:34.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.406+0000 7f1e87200640 1 --2- 172.21.15.131:0/2504853992 >> v2:172.21.15.187:3300/0 conn(0x7f1e8806d250 0x7f1e8806d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:34.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.406+0000 7f1e87200640 1 -- 172.21.15.131:0/2504853992 >> 172.21.15.131:0/2504853992 conn(0x7f1e880b5660 msgr2=0x7f1e880b5a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:34.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.406+0000 7f1e87200640 1 -- 172.21.15.131:0/2504853992 shutdown_connections 2024-05-26T22:53:34.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.406+0000 7f1e87200640 1 -- 172.21.15.131:0/2504853992 wait complete. 2024-05-26T22:53:34.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.407+0000 7f1e87200640 1 Processor -- start 2024-05-26T22:53:34.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.408+0000 7f1e87200640 1 -- start start 2024-05-26T22:53:34.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.408+0000 7f1e87200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1e8806d250 0x7f1e88155cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:34.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.408+0000 7f1e87200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1e88071f90 0x7f1e88157490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:34.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.409+0000 7f1e87200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1e8815b730 0x7f1e880aec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:34.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.409+0000 7f1e87200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f1e88071a70 con 0x7f1e8815b730 2024-05-26T22:53:34.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.409+0000 7f1e87200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1e88071790 con 0x7f1e88071f90 2024-05-26T22:53:34.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.409+0000 7f1e87200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1e8815c100 con 0x7f1e8806d250 2024-05-26T22:53:34.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.409+0000 7f1e86800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1e8815b730 0x7f1e880aec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:34.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.409+0000 7f1e86800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1e8815b730 0x7f1e880aec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50906/0 (socket says 172.21.15.131:50906) 2024-05-26T22:53:34.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.409+0000 7f1e86800640 1 -- 172.21.15.131:0/783867274 learned_addr learned my addr 172.21.15.131:0/783867274 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:53:34.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.409+0000 7f1e85400640 1 --2- 172.21.15.131:0/783867274 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1e88071f90 0x7f1e88157490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:34.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.409+0000 7f1e85e00640 1 --2- 172.21.15.131:0/783867274 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1e8806d250 0x7f1e88155cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:34.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.409+0000 7f1e86800640 1 -- 172.21.15.131:0/783867274 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1e8806d250 msgr2=0x7f1e88155cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:34.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.409+0000 7f1e86800640 1 --2- 172.21.15.131:0/783867274 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1e8806d250 0x7f1e88155cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:34.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.409+0000 7f1e86800640 1 -- 172.21.15.131:0/783867274 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1e88071f90 msgr2=0x7f1e88157490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:34.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.409+0000 7f1e86800640 1 --2- 172.21.15.131:0/783867274 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1e88071f90 0x7f1e88157490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:34.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.409+0000 7f1e86800640 1 -- 172.21.15.131:0/783867274 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e881563f0 con 0x7f1e8815b730 2024-05-26T22:53:34.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.410+0000 7f1e86800640 1 --2- 172.21.15.131:0/783867274 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1e8815b730 0x7f1e880aec00 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f1e8000c7a0 tx=0x7f1e8000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:34.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.410+0000 7f1e7e000640 1 -- 172.21.15.131:0/783867274 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e80018070 con 0x7f1e8815b730 2024-05-26T22:53:34.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.410+0000 7f1e7e000640 1 -- 172.21.15.131:0/783867274 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e8000a640 con 0x7f1e8815b730 2024-05-26T22:53:34.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.410+0000 7f1e87200640 1 -- 172.21.15.131:0/783867274 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e881566d0 con 0x7f1e8815b730 2024-05-26T22:53:34.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.410+0000 7f1e7e000640 1 -- 172.21.15.131:0/783867274 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e8000aab0 con 0x7f1e8815b730 2024-05-26T22:53:34.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.410+0000 7f1e87200640 1 -- 172.21.15.131:0/783867274 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e88156c90 con 0x7f1e8815b730 2024-05-26T22:53:34.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.411+0000 7f1e87200640 1 -- 172.21.15.131:0/783867274 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e880694e0 con 0x7f1e8815b730 2024-05-26T22:53:34.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.412+0000 7f1e7e000640 1 -- 172.21.15.131:0/783867274 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1e8001b070 con 0x7f1e8815b730 2024-05-26T22:53:34.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.413+0000 7f1e7e000640 1 --2- 172.21.15.131:0/783867274 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f1e6806be30 0x7f1e6806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:34.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.413+0000 7f1e7e000640 1 -- 172.21.15.131:0/783867274 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5767+0+0 (secure 0 0 0) 0x7f1e8008a500 con 0x7f1e8815b730 2024-05-26T22:53:34.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.413+0000 7f1e85e00640 1 --2- 172.21.15.131:0/783867274 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f1e6806be30 0x7f1e6806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:34.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.414+0000 7f1e85e00640 1 --2- 172.21.15.131:0/783867274 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f1e6806be30 0x7f1e6806e2f0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f1e7800a990 tx=0x7f1e78008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:34.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.418+0000 7f1e7e000640 1 -- 172.21.15.131:0/783867274 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f1e80058e60 con 0x7f1e8815b730 2024-05-26T22:53:34.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.616+0000 7f1e87200640 1 -- 172.21.15.131:0/783867274 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f1e880af9b0 con 0x7f1e8815b730 2024-05-26T22:53:34.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.617+0000 7f1e7e000640 1 -- 172.21.15.131:0/783867274 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v71) ==== 104+0+11 (secure 0 0 0) 0x7f1e80058760 con 0x7f1e8815b730 2024-05-26T22:53:34.617 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-05-26T22:53:34.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.620+0000 7f1e87200640 1 -- 172.21.15.131:0/783867274 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f1e6806be30 msgr2=0x7f1e6806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:34.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.620+0000 7f1e87200640 1 --2- 172.21.15.131:0/783867274 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f1e6806be30 0x7f1e6806e2f0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f1e7800a990 tx=0x7f1e78008040 comp rx=0 tx=0).stop 2024-05-26T22:53:34.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.620+0000 7f1e87200640 1 -- 172.21.15.131:0/783867274 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1e8815b730 msgr2=0x7f1e880aec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:34.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.620+0000 7f1e87200640 1 --2- 172.21.15.131:0/783867274 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1e8815b730 0x7f1e880aec00 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f1e8000c7a0 tx=0x7f1e8000cc60 comp rx=0 tx=0).stop 2024-05-26T22:53:34.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.620+0000 7f1e87200640 1 -- 172.21.15.131:0/783867274 shutdown_connections 2024-05-26T22:53:34.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.620+0000 7f1e87200640 1 --2- 172.21.15.131:0/783867274 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f1e6806be30 0x7f1e6806e2f0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:34.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.620+0000 7f1e87200640 1 --2- 172.21.15.131:0/783867274 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1e8815b730 0x7f1e880aec00 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:34.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.620+0000 7f1e87200640 1 --2- 172.21.15.131:0/783867274 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1e88071f90 0x7f1e88157490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:34.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.620+0000 7f1e87200640 1 --2- 172.21.15.131:0/783867274 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1e8806d250 0x7f1e88155cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:34.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.620+0000 7f1e87200640 1 -- 172.21.15.131:0/783867274 >> 172.21.15.131:0/783867274 conn(0x7f1e880b5660 msgr2=0x7f1e880b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:34.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.620+0000 7f1e87200640 1 -- 172.21.15.131:0/783867274 shutdown_connections 2024-05-26T22:53:34.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.620+0000 7f1e87200640 1 -- 172.21.15.131:0/783867274 wait complete. 2024-05-26T22:53:34.639 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-26T22:53:34.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.847+0000 7ffa83800640 1 Processor -- start 2024-05-26T22:53:34.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.848+0000 7ffa83800640 1 -- start start 2024-05-26T22:53:34.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.849+0000 7ffa83800640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7ffa7c15e2f0 0x7ffa7c0ae810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:34.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.849+0000 7ffa83800640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7ffa7c15e6d0 0x7ffa7c0b3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:34.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.849+0000 7ffa83800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffa7c0aed50 0x7ffa7c0b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:34.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.849+0000 7ffa83800640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7ffa7c0b4540 con 0x7ffa7c15e2f0 2024-05-26T22:53:34.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.849+0000 7ffa83800640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7ffa7c0b4820 con 0x7ffa7c15e6d0 2024-05-26T22:53:34.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.849+0000 7ffa83800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ffa7c0b4b00 con 0x7ffa7c0aed50 2024-05-26T22:53:34.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.850+0000 7ffa80e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7ffa7c15e2f0 0x7ffa7c0ae810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:34.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.850+0000 7ffa80e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7ffa7c15e2f0 0x7ffa7c0ae810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50916/0 (socket says 172.21.15.131:50916) 2024-05-26T22:53:34.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.850+0000 7ffa80e00640 1 -- 172.21.15.131:0/1086800318 learned_addr learned my addr 172.21.15.131:0/1086800318 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:53:34.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.850+0000 7ffa73e00640 1 --2- 172.21.15.131:0/1086800318 >> v2:172.21.15.187:3300/0 conn(0x7ffa7c15e6d0 0x7ffa7c0b3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:34.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.850+0000 7ffa81800640 1 --2- 172.21.15.131:0/1086800318 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffa7c0aed50 0x7ffa7c0b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:34.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.850+0000 7ffa80e00640 1 -- 172.21.15.131:0/1086800318 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffa7c0aed50 msgr2=0x7ffa7c0b4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:34.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.850+0000 7ffa80e00640 1 --2- 172.21.15.131:0/1086800318 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffa7c0aed50 0x7ffa7c0b4d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:34.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.850+0000 7ffa80e00640 1 -- 172.21.15.131:0/1086800318 >> v2:172.21.15.187:3300/0 conn(0x7ffa7c15e6d0 msgr2=0x7ffa7c0b3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:34.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.850+0000 7ffa80e00640 1 --2- 172.21.15.131:0/1086800318 >> v2:172.21.15.187:3300/0 conn(0x7ffa7c15e6d0 0x7ffa7c0b3ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:34.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.850+0000 7ffa80e00640 1 -- 172.21.15.131:0/1086800318 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa7c06df70 con 0x7ffa7c15e2f0 2024-05-26T22:53:34.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.850+0000 7ffa81800640 1 --2- 172.21.15.131:0/1086800318 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffa7c0aed50 0x7ffa7c0b4d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:53:34.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.851+0000 7ffa80e00640 1 --2- 172.21.15.131:0/1086800318 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa7c15e2f0 0x7ffa7c0ae810 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c0ae350 tx=0x7ffa6c02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77a6f40585231d11 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:34.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.852+0000 7ffa7be00640 1 -- 172.21.15.131:0/1086800318 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa6c037070 con 0x7ffa7c15e2f0 2024-05-26T22:53:34.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.852+0000 7ffa7be00640 1 -- 172.21.15.131:0/1086800318 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa6c032440 con 0x7ffa7c15e2f0 2024-05-26T22:53:34.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.852+0000 7ffa7be00640 1 -- 172.21.15.131:0/1086800318 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa6c032810 con 0x7ffa7c15e2f0 2024-05-26T22:53:34.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.852+0000 7ffa83800640 1 -- 172.21.15.131:0/1086800318 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa7c15e2f0 msgr2=0x7ffa7c0ae810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:34.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.852+0000 7ffa83800640 1 --2- 172.21.15.131:0/1086800318 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa7c15e2f0 0x7ffa7c0ae810 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c0ae350 tx=0x7ffa6c02a140 comp rx=0 tx=0).stop 2024-05-26T22:53:34.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.852+0000 7ffa83800640 1 -- 172.21.15.131:0/1086800318 shutdown_connections 2024-05-26T22:53:34.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.852+0000 7ffa83800640 1 --2- 172.21.15.131:0/1086800318 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffa7c0aed50 0x7ffa7c0b4d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:34.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.852+0000 7ffa83800640 1 --2- 172.21.15.131:0/1086800318 >> v2:172.21.15.187:3300/0 conn(0x7ffa7c15e6d0 0x7ffa7c0b3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:34.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.852+0000 7ffa83800640 1 --2- 172.21.15.131:0/1086800318 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa7c15e2f0 0x7ffa7c0ae810 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:34.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.852+0000 7ffa83800640 1 -- 172.21.15.131:0/1086800318 >> 172.21.15.131:0/1086800318 conn(0x7ffa7c0b27c0 msgr2=0x7ffa7c0b2bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:34.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.852+0000 7ffa83800640 1 -- 172.21.15.131:0/1086800318 shutdown_connections 2024-05-26T22:53:34.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.852+0000 7ffa83800640 1 -- 172.21.15.131:0/1086800318 wait complete. 2024-05-26T22:53:34.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.854+0000 7ffa83800640 1 Processor -- start 2024-05-26T22:53:34.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.854+0000 7ffa83800640 1 -- start start 2024-05-26T22:53:34.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.855+0000 7ffa83800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa7c0aed50 0x7ffa7c14e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:34.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.855+0000 7ffa83800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffa7c15e2f0 0x7ffa7c15c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:34.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.855+0000 7ffa83800640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffa7c15e6d0 0x7ffa7c1443c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:34.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.855+0000 7ffa83800640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ffa7c0b4000 con 0x7ffa7c0aed50 2024-05-26T22:53:34.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.855+0000 7ffa83800640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ffa7c0b5b30 con 0x7ffa7c15e6d0 2024-05-26T22:53:34.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.855+0000 7ffa83800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ffa7c0b4180 con 0x7ffa7c15e2f0 2024-05-26T22:53:34.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.855+0000 7ffa80e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa7c0aed50 0x7ffa7c14e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:34.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.856+0000 7ffa80e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa7c0aed50 0x7ffa7c14e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50924/0 (socket says 172.21.15.131:50924) 2024-05-26T22:53:34.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.856+0000 7ffa80e00640 1 -- 172.21.15.131:0/3529618460 learned_addr learned my addr 172.21.15.131:0/3529618460 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:53:34.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.856+0000 7ffa80e00640 1 -- 172.21.15.131:0/3529618460 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffa7c15e2f0 msgr2=0x7ffa7c15c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:34.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.856+0000 7ffa80e00640 1 --2- 172.21.15.131:0/3529618460 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffa7c15e2f0 0x7ffa7c15c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:34.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.856+0000 7ffa81800640 1 --2- 172.21.15.131:0/3529618460 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffa7c15e6d0 0x7ffa7c1443c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:34.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.856+0000 7ffa80e00640 1 -- 172.21.15.131:0/3529618460 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffa7c15e6d0 msgr2=0x7ffa7c1443c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:34.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.856+0000 7ffa80e00640 1 --2- 172.21.15.131:0/3529618460 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffa7c15e6d0 0x7ffa7c1443c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:34.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.856+0000 7ffa80e00640 1 -- 172.21.15.131:0/3529618460 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa7c15bdf0 con 0x7ffa7c0aed50 2024-05-26T22:53:34.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.856+0000 7ffa80e00640 1 --2- 172.21.15.131:0/3529618460 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa7c0aed50 0x7ffa7c14e2b0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c000c00 tx=0x7ffa6c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:34.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.857+0000 7ffa79600640 1 -- 172.21.15.131:0/3529618460 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa6c041020 con 0x7ffa7c0aed50 2024-05-26T22:53:34.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.857+0000 7ffa79600640 1 -- 172.21.15.131:0/3529618460 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa6c0306c0 con 0x7ffa7c0aed50 2024-05-26T22:53:34.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.857+0000 7ffa79600640 1 -- 172.21.15.131:0/3529618460 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa6c037040 con 0x7ffa7c0aed50 2024-05-26T22:53:34.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.857+0000 7ffa83800640 1 -- 172.21.15.131:0/3529618460 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa7c15c0d0 con 0x7ffa7c0aed50 2024-05-26T22:53:34.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.857+0000 7ffa83800640 1 -- 172.21.15.131:0/3529618460 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa7c1500a0 con 0x7ffa7c0aed50 2024-05-26T22:53:34.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.858+0000 7ffa83800640 1 -- 172.21.15.131:0/3529618460 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa7c067910 con 0x7ffa7c0aed50 2024-05-26T22:53:34.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.859+0000 7ffa79600640 1 -- 172.21.15.131:0/3529618460 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ffa6c044020 con 0x7ffa7c0aed50 2024-05-26T22:53:34.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.860+0000 7ffa79600640 1 --2- 172.21.15.131:0/3529618460 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ffa5806be60 0x7ffa5806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:34.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.860+0000 7ffa79600640 1 -- 172.21.15.131:0/3529618460 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5767+0+0 (secure 0 0 0) 0x7ffa6c0ab7b0 con 0x7ffa7c0aed50 2024-05-26T22:53:34.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.860+0000 7ffa73e00640 1 --2- 172.21.15.131:0/3529618460 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ffa5806be60 0x7ffa5806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:34.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.861+0000 7ffa73e00640 1 --2- 172.21.15.131:0/3529618460 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ffa5806be60 0x7ffa5806e320 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7ffa64006fd0 tx=0x7ffa64008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:34.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:34.865+0000 7ffa79600640 1 -- 172.21.15.131:0/3529618460 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7ffa6c04b030 con 0x7ffa7c0aed50 2024-05-26T22:53:35.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.061+0000 7ffa83800640 1 -- 172.21.15.131:0/3529618460 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ffa7c14f690 con 0x7ffa7c0aed50 2024-05-26T22:53:35.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.062+0000 7ffa79600640 1 -- 172.21.15.131:0/3529618460 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v71) ==== 100+0+11 (secure 0 0 0) 0x7ffa6c07a080 con 0x7ffa7c0aed50 2024-05-26T22:53:35.062 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 64 2024-05-26T22:53:35.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.064+0000 7ffa83800640 1 -- 172.21.15.131:0/3529618460 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ffa5806be60 msgr2=0x7ffa5806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:35.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.064+0000 7ffa83800640 1 --2- 172.21.15.131:0/3529618460 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ffa5806be60 0x7ffa5806e320 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7ffa64006fd0 tx=0x7ffa64008040 comp rx=0 tx=0).stop 2024-05-26T22:53:35.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.064+0000 7ffa83800640 1 -- 172.21.15.131:0/3529618460 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa7c0aed50 msgr2=0x7ffa7c14e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:35.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.064+0000 7ffa83800640 1 --2- 172.21.15.131:0/3529618460 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa7c0aed50 0x7ffa7c14e2b0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c000c00 tx=0x7ffa6c0046c0 comp rx=0 tx=0).stop 2024-05-26T22:53:35.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.065+0000 7ffa83800640 1 -- 172.21.15.131:0/3529618460 shutdown_connections 2024-05-26T22:53:35.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.065+0000 7ffa83800640 1 --2- 172.21.15.131:0/3529618460 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffa7c15e2f0 0x7ffa7c15c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:35.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.065+0000 7ffa83800640 1 --2- 172.21.15.131:0/3529618460 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ffa5806be60 0x7ffa5806e320 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:35.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.065+0000 7ffa83800640 1 --2- 172.21.15.131:0/3529618460 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffa7c15e6d0 0x7ffa7c1443c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:35.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.065+0000 7ffa83800640 1 --2- 172.21.15.131:0/3529618460 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa7c0aed50 0x7ffa7c14e2b0 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:35.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.065+0000 7ffa83800640 1 -- 172.21.15.131:0/3529618460 >> 172.21.15.131:0/3529618460 conn(0x7ffa7c0b27c0 msgr2=0x7ffa7c0b1e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:35.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.065+0000 7ffa83800640 1 -- 172.21.15.131:0/3529618460 shutdown_connections 2024-05-26T22:53:35.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.065+0000 7ffa83800640 1 -- 172.21.15.131:0/3529618460 wait complete. 2024-05-26T22:53:35.084 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-26T22:53:35.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.283+0000 7f109da00640 1 Processor -- start 2024-05-26T22:53:35.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.283+0000 7f109da00640 1 -- start start 2024-05-26T22:53:35.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.285+0000 7f109da00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f109806ee20 0x7f10980b3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:35.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.285+0000 7f109da00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f10980ad250 0x7f10980af790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:35.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.285+0000 7f109da00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f109815cb20 con 0x7f109806f200 2024-05-26T22:53:35.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.285+0000 7f109da00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f109815ce00 con 0x7f109806ee20 2024-05-26T22:53:35.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.285+0000 7f109da00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f109815d0e0 con 0x7f10980ad250 2024-05-26T22:53:35.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.285+0000 7f1096a00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f109806ee20 0x7f10980b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:35.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.285+0000 7f1096a00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f109806ee20 0x7f10980b3190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50938/0 (socket says 172.21.15.131:50938) 2024-05-26T22:53:35.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.285+0000 7f1096a00640 1 -- 172.21.15.131:0/3471603670 learned_addr learned my addr 172.21.15.131:0/3471603670 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:53:35.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.285+0000 7f1096a00640 1 -- 172.21.15.131:0/3471603670 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f10980ad250 msgr2=0x7f10980af790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:35.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.285+0000 7f1096000640 1 --2- 172.21.15.131:0/3471603670 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f10980ad250 0x7f10980af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:35.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.285+0000 7f1096a00640 1 --2- 172.21.15.131:0/3471603670 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f10980ad250 0x7f10980af790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:35.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.285+0000 7f1096a00640 1 -- 172.21.15.131:0/3471603670 >> v1:172.21.15.187:6789/0 conn(0x7f109806f200 legacy=0x7f10981562c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:35.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.285+0000 7f1096a00640 1 -- 172.21.15.131:0/3471603670 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f109815d3c0 con 0x7f109806ee20 2024-05-26T22:53:35.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.286+0000 7f1096a00640 1 --2- 172.21.15.131:0/3471603670 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f109806ee20 0x7f10980b3190 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f1088009870 tx=0x7f108802a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b10917e2d3fe29c server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:35.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.287+0000 7f1095600640 1 -- 172.21.15.131:0/3471603670 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1088037070 con 0x7f109806ee20 2024-05-26T22:53:35.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.287+0000 7f1095600640 1 -- 172.21.15.131:0/3471603670 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1088032440 con 0x7f109806ee20 2024-05-26T22:53:35.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.287+0000 7f1095600640 1 -- 172.21.15.131:0/3471603670 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1088032810 con 0x7f109806ee20 2024-05-26T22:53:35.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.287+0000 7f109da00640 1 -- 172.21.15.131:0/3471603670 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f109806ee20 msgr2=0x7f10980b3190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:35.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.287+0000 7f109da00640 1 --2- 172.21.15.131:0/3471603670 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f109806ee20 0x7f10980b3190 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f1088009870 tx=0x7f108802a200 comp rx=0 tx=0).stop 2024-05-26T22:53:35.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.287+0000 7f109da00640 1 -- 172.21.15.131:0/3471603670 shutdown_connections 2024-05-26T22:53:35.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.287+0000 7f109da00640 1 --2- 172.21.15.131:0/3471603670 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f10980ad250 0x7f10980af790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:35.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.287+0000 7f109da00640 1 --2- 172.21.15.131:0/3471603670 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f109806ee20 0x7f10980b3190 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:35.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.287+0000 7f109da00640 1 -- 172.21.15.131:0/3471603670 >> 172.21.15.131:0/3471603670 conn(0x7f10980b1e90 msgr2=0x7f10980b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:35.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.288+0000 7f109da00640 1 -- 172.21.15.131:0/3471603670 shutdown_connections 2024-05-26T22:53:35.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.288+0000 7f109da00640 1 -- 172.21.15.131:0/3471603670 wait complete. 2024-05-26T22:53:35.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.289+0000 7f109da00640 1 Processor -- start 2024-05-26T22:53:35.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.289+0000 7f109da00640 1 -- start start 2024-05-26T22:53:35.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.290+0000 7f109da00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f109806ee20 0x7f10981442c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:35.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.290+0000 7f109da00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f109806f200 0x7f10980b1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:35.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.290+0000 7f109da00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f10980ad250 0x7f10980b1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:35.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.290+0000 7f109da00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f109815c900 con 0x7f109806f200 2024-05-26T22:53:35.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.290+0000 7f109da00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f109815c780 con 0x7f10980ad250 2024-05-26T22:53:35.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.290+0000 7f109da00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f10981569d0 con 0x7f109806ee20 2024-05-26T22:53:35.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.290+0000 7f1096000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f109806f200 0x7f10980b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:35.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.291+0000 7f1096000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f109806f200 0x7f10980b1350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50944/0 (socket says 172.21.15.131:50944) 2024-05-26T22:53:35.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.291+0000 7f1096000640 1 -- 172.21.15.131:0/2871172420 learned_addr learned my addr 172.21.15.131:0/2871172420 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:53:35.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.291+0000 7f1096a00640 1 --2- 172.21.15.131:0/2871172420 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f109806ee20 0x7f10981442c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:35.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.291+0000 7f1096000640 1 -- 172.21.15.131:0/2871172420 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f109806ee20 msgr2=0x7f10981442c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:35.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.291+0000 7f1096000640 1 --2- 172.21.15.131:0/2871172420 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f109806ee20 0x7f10981442c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:35.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.291+0000 7f1096000640 1 -- 172.21.15.131:0/2871172420 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f10980ad250 msgr2=0x7f10980b1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:35.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.291+0000 7f1096000640 1 --2- 172.21.15.131:0/2871172420 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f10980ad250 0x7f10980b1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:35.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.291+0000 7f1096000640 1 -- 172.21.15.131:0/2871172420 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1098151d20 con 0x7f109806f200 2024-05-26T22:53:35.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.291+0000 7f1096000640 1 --2- 172.21.15.131:0/2871172420 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f109806f200 0x7f10980b1350 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f108400c820 tx=0x7f108400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:35.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.292+0000 7f108ea00640 1 -- 172.21.15.131:0/2871172420 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1084015070 con 0x7f109806f200 2024-05-26T22:53:35.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.292+0000 7f108ea00640 1 -- 172.21.15.131:0/2871172420 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1084004020 con 0x7f109806f200 2024-05-26T22:53:35.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.292+0000 7f108ea00640 1 -- 172.21.15.131:0/2871172420 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10840043f0 con 0x7f109806f200 2024-05-26T22:53:35.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.292+0000 7f109da00640 1 -- 172.21.15.131:0/2871172420 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1098151fe0 con 0x7f109806f200 2024-05-26T22:53:35.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.292+0000 7f109da00640 1 -- 172.21.15.131:0/2871172420 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10981525a0 con 0x7f109806f200 2024-05-26T22:53:35.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.293+0000 7f109da00640 1 -- 172.21.15.131:0/2871172420 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10980694e0 con 0x7f109806f200 2024-05-26T22:53:35.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.294+0000 7f108ea00640 1 -- 172.21.15.131:0/2871172420 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1084007830 con 0x7f109806f200 2024-05-26T22:53:35.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.295+0000 7f108ea00640 1 --2- 172.21.15.131:0/2871172420 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f107806be30 0x7f107806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:35.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.295+0000 7f108ea00640 1 -- 172.21.15.131:0/2871172420 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5767+0+0 (secure 0 0 0) 0x7f108408ad10 con 0x7f109806f200 2024-05-26T22:53:35.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.295+0000 7f1096a00640 1 --2- 172.21.15.131:0/2871172420 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f107806be30 0x7f107806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:35.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.296+0000 7f1096a00640 1 --2- 172.21.15.131:0/2871172420 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f107806be30 0x7f107806e2f0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f1088034000 tx=0x7f108802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:35.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.299+0000 7f108ea00640 1 -- 172.21.15.131:0/2871172420 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f1084090020 con 0x7f109806f200 2024-05-26T22:53:35.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.538+0000 7f109da00640 1 -- 172.21.15.131:0/2871172420 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f109806c590 con 0x7f109806f200 2024-05-26T22:53:35.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.539+0000 7f108ea00640 1 -- 172.21.15.131:0/2871172420 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) ==== 94+0+5236 (secure 0 0 0) 0x7f10840595e0 con 0x7f109806f200 2024-05-26T22:53:35.539 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:53:35.539 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":69,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5359,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/4204396845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6833","nonce":4204396845},{"type":"v1","addr":"172.21.15.187:6835","nonce":4204396845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-26T22:52:53.755842+0000","modified":"2024-05-26T22:53:23.885672+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6876,"mds_1":6864,"mds_2":6861},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6861":{"gid":6861,"name":"a","rank":2,"incarnation":67,"state":"up:active","state_seq":6,"addr":"172.21.15.131:6836/4223534499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":4223534499},{"type":"v1","addr":"172.21.15.131:6836","nonce":4223534499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6864":{"gid":6864,"name":"c","rank":1,"incarnation":65,"state":"up:active","state_seq":6,"addr":"172.21.15.131:6837/1418741551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6835","nonce":1418741551},{"type":"v1","addr":"172.21.15.131:6837","nonce":1418741551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6876":{"gid":6876,"name":"d","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6834/3680999672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3680999672},{"type":"v1","addr":"172.21.15.187:6834","nonce":3680999672}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6876,"qdb_cluster":[6876,6864,6861]},"id":5}]} 2024-05-26T22:53:35.540 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 69 2024-05-26T22:53:35.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.542+0000 7f109da00640 1 -- 172.21.15.131:0/2871172420 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f107806be30 msgr2=0x7f107806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:35.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.542+0000 7f109da00640 1 --2- 172.21.15.131:0/2871172420 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f107806be30 0x7f107806e2f0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f1088034000 tx=0x7f108802e000 comp rx=0 tx=0).stop 2024-05-26T22:53:35.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.542+0000 7f109da00640 1 -- 172.21.15.131:0/2871172420 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f109806f200 msgr2=0x7f10980b1350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:35.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.542+0000 7f109da00640 1 --2- 172.21.15.131:0/2871172420 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f109806f200 0x7f10980b1350 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f108400c820 tx=0x7f108400cce0 comp rx=0 tx=0).stop 2024-05-26T22:53:35.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.542+0000 7f109da00640 1 -- 172.21.15.131:0/2871172420 shutdown_connections 2024-05-26T22:53:35.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.542+0000 7f109da00640 1 --2- 172.21.15.131:0/2871172420 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f107806be30 0x7f107806e2f0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:35.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.543+0000 7f109da00640 1 --2- 172.21.15.131:0/2871172420 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f10980ad250 0x7f10980b1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:35.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.543+0000 7f109da00640 1 --2- 172.21.15.131:0/2871172420 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f109806f200 0x7f10980b1350 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:35.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.543+0000 7f109da00640 1 --2- 172.21.15.131:0/2871172420 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f109806ee20 0x7f10981442c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:35.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.543+0000 7f109da00640 1 -- 172.21.15.131:0/2871172420 >> 172.21.15.131:0/2871172420 conn(0x7f10980b1e90 msgr2=0x7f109815b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:35.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.543+0000 7f109da00640 1 -- 172.21.15.131:0/2871172420 shutdown_connections 2024-05-26T22:53:35.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.543+0000 7f109da00640 1 -- 172.21.15.131:0/2871172420 wait complete. 2024-05-26T22:53:35.562 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-26T22:53:35.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.762+0000 7f6b1e400640 1 Processor -- start 2024-05-26T22:53:35.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.762+0000 7f6b1e400640 1 -- start start 2024-05-26T22:53:35.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.764+0000 7f6b1e400640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f6b18156750 0x7f6b1815fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:35.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.764+0000 7f6b1e400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6b180cc7f0 0x7f6b180ccca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:35.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.764+0000 7f6b1e400640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b18160620 con 0x7f6b1806e8c0 2024-05-26T22:53:35.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.764+0000 7f6b1e400640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f6b1815d770 con 0x7f6b18156750 2024-05-26T22:53:35.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.764+0000 7f6b1e400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f6b1815cbd0 con 0x7f6b180cc7f0 2024-05-26T22:53:35.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.764+0000 7f6b1da00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6b180cc7f0 0x7f6b180ccca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:35.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.764+0000 7f6b1da00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6b180cc7f0 0x7f6b180ccca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.131:37778/0 (socket says 172.21.15.131:37778) 2024-05-26T22:53:35.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.764+0000 7f6b1da00640 1 -- 172.21.15.131:0/3155345005 learned_addr learned my addr 172.21.15.131:0/3155345005 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:53:35.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.764+0000 7f6b1d000640 1 --2- 172.21.15.131:0/3155345005 >> v2:172.21.15.187:3300/0 conn(0x7f6b18156750 0x7f6b1815fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:35.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.765+0000 7f6b1da00640 1 -- 172.21.15.131:0/3155345005 >> v2:172.21.15.187:3300/0 conn(0x7f6b18156750 msgr2=0x7f6b1815fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:35.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.765+0000 7f6b1da00640 1 --2- 172.21.15.131:0/3155345005 >> v2:172.21.15.187:3300/0 conn(0x7f6b18156750 0x7f6b1815fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:35.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.765+0000 7f6b1da00640 1 -- 172.21.15.131:0/3155345005 >> v1:172.21.15.187:6789/0 conn(0x7f6b1806e8c0 legacy=0x7f6b1815d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:35.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.765+0000 7f6b1da00640 1 -- 172.21.15.131:0/3155345005 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b18160900 con 0x7f6b180cc7f0 2024-05-26T22:53:35.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.766+0000 7f6b1da00640 1 --2- 172.21.15.131:0/3155345005 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6b180cc7f0 0x7f6b180ccca0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6b10009e50 tx=0x7f6b1002a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1672e302c760808c server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:35.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.767+0000 7f6b17e00640 1 -- 172.21.15.131:0/3155345005 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b10036070 con 0x7f6b180cc7f0 2024-05-26T22:53:35.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.767+0000 7f6b17e00640 1 -- 172.21.15.131:0/3155345005 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b10031600 con 0x7f6b180cc7f0 2024-05-26T22:53:35.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.767+0000 7f6b17e00640 1 -- 172.21.15.131:0/3155345005 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b100319d0 con 0x7f6b180cc7f0 2024-05-26T22:53:35.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.768+0000 7f6b1e400640 1 -- 172.21.15.131:0/3155345005 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6b180cc7f0 msgr2=0x7f6b180ccca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:35.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.768+0000 7f6b1e400640 1 --2- 172.21.15.131:0/3155345005 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6b180cc7f0 0x7f6b180ccca0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6b10009e50 tx=0x7f6b1002a6d0 comp rx=0 tx=0).stop 2024-05-26T22:53:35.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.768+0000 7f6b1e400640 1 -- 172.21.15.131:0/3155345005 shutdown_connections 2024-05-26T22:53:35.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.768+0000 7f6b1e400640 1 --2- 172.21.15.131:0/3155345005 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6b180cc7f0 0x7f6b180ccca0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:35.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.768+0000 7f6b1e400640 1 --2- 172.21.15.131:0/3155345005 >> v2:172.21.15.187:3300/0 conn(0x7f6b18156750 0x7f6b1815fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:35.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.768+0000 7f6b1e400640 1 -- 172.21.15.131:0/3155345005 >> 172.21.15.131:0/3155345005 conn(0x7f6b180cff40 msgr2=0x7f6b180d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:35.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.768+0000 7f6b1e400640 1 -- 172.21.15.131:0/3155345005 shutdown_connections 2024-05-26T22:53:35.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.768+0000 7f6b1e400640 1 -- 172.21.15.131:0/3155345005 wait complete. 2024-05-26T22:53:35.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.770+0000 7f6b1e400640 1 Processor -- start 2024-05-26T22:53:35.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.770+0000 7f6b1e400640 1 -- start start 2024-05-26T22:53:35.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.771+0000 7f6b1e400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6b1806e8c0 0x7f6b18140da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:35.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.771+0000 7f6b1e400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6b180cc7f0 0x7f6b18159630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:35.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.771+0000 7f6b1e400640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6b18156750 0x7f6b181e67f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:35.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.771+0000 7f6b1e400640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f6b18156b30 con 0x7f6b1806e8c0 2024-05-26T22:53:35.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.771+0000 7f6b1e400640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6b1815c860 con 0x7f6b18156750 2024-05-26T22:53:35.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.771+0000 7f6b1e400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f6b18160280 con 0x7f6b180cc7f0 2024-05-26T22:53:35.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.771+0000 7f6b1d000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6b1806e8c0 0x7f6b18140da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:35.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.771+0000 7f6b1d000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6b1806e8c0 0x7f6b18140da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50954/0 (socket says 172.21.15.131:50954) 2024-05-26T22:53:35.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.771+0000 7f6b1d000640 1 -- 172.21.15.131:0/2575979333 learned_addr learned my addr 172.21.15.131:0/2575979333 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:53:35.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.771+0000 7f6b0fe00640 1 --2- 172.21.15.131:0/2575979333 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6b180cc7f0 0x7f6b18159630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:35.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.771+0000 7f6b1da00640 1 --2- 172.21.15.131:0/2575979333 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6b18156750 0x7f6b181e67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:35.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.772+0000 7f6b1d000640 1 -- 172.21.15.131:0/2575979333 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6b180cc7f0 msgr2=0x7f6b18159630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:35.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.772+0000 7f6b1d000640 1 --2- 172.21.15.131:0/2575979333 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6b180cc7f0 0x7f6b18159630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:35.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.772+0000 7f6b1d000640 1 -- 172.21.15.131:0/2575979333 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6b18156750 msgr2=0x7f6b181e67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:35.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.772+0000 7f6b1d000640 1 --2- 172.21.15.131:0/2575979333 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6b18156750 0x7f6b181e67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:35.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.772+0000 7f6b1d000640 1 -- 172.21.15.131:0/2575979333 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b180ad9f0 con 0x7f6b1806e8c0 2024-05-26T22:53:35.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.772+0000 7f6b1da00640 1 --2- 172.21.15.131:0/2575979333 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6b18156750 0x7f6b181e67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-26T22:53:35.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.772+0000 7f6b1d000640 1 --2- 172.21.15.131:0/2575979333 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6b1806e8c0 0x7f6b18140da0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f6b0800e870 tx=0x7f6b0800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:35.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.772+0000 7f6b15600640 1 -- 172.21.15.131:0/2575979333 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b08017070 con 0x7f6b1806e8c0 2024-05-26T22:53:35.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.772+0000 7f6b15600640 1 -- 172.21.15.131:0/2575979333 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b0800c940 con 0x7f6b1806e8c0 2024-05-26T22:53:35.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.772+0000 7f6b15600640 1 -- 172.21.15.131:0/2575979333 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b0800cd30 con 0x7f6b1806e8c0 2024-05-26T22:53:35.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.772+0000 7f6b1e400640 1 -- 172.21.15.131:0/2575979333 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b181e6f40 con 0x7f6b1806e8c0 2024-05-26T22:53:35.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.772+0000 7f6b1e400640 1 -- 172.21.15.131:0/2575979333 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b18156cb0 con 0x7f6b1806e8c0 2024-05-26T22:53:35.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.774+0000 7f6b1e400640 1 -- 172.21.15.131:0/2575979333 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b180694e0 con 0x7f6b1806e8c0 2024-05-26T22:53:35.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.774+0000 7f6b15600640 1 -- 172.21.15.131:0/2575979333 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6b0801a070 con 0x7f6b1806e8c0 2024-05-26T22:53:35.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.775+0000 7f6b15600640 1 --2- 172.21.15.131:0/2575979333 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f6af406be30 0x7f6af406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:35.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.775+0000 7f6b15600640 1 -- 172.21.15.131:0/2575979333 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5767+0+0 (secure 0 0 0) 0x7f6b0808ac10 con 0x7f6b1806e8c0 2024-05-26T22:53:35.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.776+0000 7f6b0fe00640 1 --2- 172.21.15.131:0/2575979333 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f6af406be30 0x7f6af406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:35.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.776+0000 7f6b0fe00640 1 --2- 172.21.15.131:0/2575979333 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f6af406be30 0x7f6af406e2f0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f6b180b2dd0 tx=0x7f6b00007e30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:35.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.780+0000 7f6b15600640 1 -- 172.21.15.131:0/2575979333 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f6b08058fc0 con 0x7f6b1806e8c0 2024-05-26T22:53:35.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.978+0000 7f6b1e400640 1 -- 172.21.15.131:0/2575979333 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6b18073150 con 0x7f6b1806e8c0 2024-05-26T22:53:35.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.981+0000 7f6b15600640 1 -- 172.21.15.131:0/2575979333 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) ==== 74+0+16789 (secure 0 0 0) 0x7f6b0805d2c0 con 0x7f6b1806e8c0 2024-05-26T22:53:35.981 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:53:35.982 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":71,"fsid":"46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f","created":"2024-05-26T22:47:57.288905+0000","modified":"2024-05-26T22:53:06.170553+0000","last_up_change":"2024-05-26T22:48:09.602207+0000","last_in_change":"2024-05-26T22:48:00.687341+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-26T22:48:10.463339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-05-26T22:52:51.536251+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-05-26T22:52:52.768524+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"814eaf62-a18d-494b-8e27-fd8fb88adea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6801","nonce":2350940051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6803","nonce":2350940051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6807","nonce":2350940051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6805","nonce":2350940051}]},"public_addr":"172.21.15.131:6801/2350940051","cluster_addr":"172.21.15.131:6803/2350940051","heartbeat_back_addr":"172.21.15.131:6807/2350940051","heartbeat_front_addr":"172.21.15.131:6805/2350940051","state":["exists","up"]},{"osd":1,"uuid":"42e66287-f6c6-4e26-af3c-6dfaa3536d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6825","nonce":1525528247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6827","nonce":1525528247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6831","nonce":1525528247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6829","nonce":1525528247}]},"public_addr":"172.21.15.131:6825/1525528247","cluster_addr":"172.21.15.131:6827/1525528247","heartbeat_back_addr":"172.21.15.131:6831/1525528247","heartbeat_front_addr":"172.21.15.131:6829/1525528247","state":["exists","up"]},{"osd":2,"uuid":"0da34673-04a2-4fd0-b57a-0ff2b679b043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6817","nonce":3627842210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6819","nonce":3627842210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6823","nonce":3627842210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6821","nonce":3627842210}]},"public_addr":"172.21.15.131:6817/3627842210","cluster_addr":"172.21.15.131:6819/3627842210","heartbeat_back_addr":"172.21.15.131:6823/3627842210","heartbeat_front_addr":"172.21.15.131:6821/3627842210","state":["exists","up"]},{"osd":3,"uuid":"cf117a27-d6b2-43d3-ae52-b47988bf8eed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6809","nonce":350790591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6811","nonce":350790591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6815","nonce":350790591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6813","nonce":350790591}]},"public_addr":"172.21.15.131:6809/350790591","cluster_addr":"172.21.15.131:6811/350790591","heartbeat_back_addr":"172.21.15.131:6815/350790591","heartbeat_front_addr":"172.21.15.131:6813/350790591","state":["exists","up"]},{"osd":4,"uuid":"a092ccfc-be89-4a84-88d5-0f781fa715eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6816","nonce":244560995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6819","nonce":244560995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6823","nonce":244560995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6821","nonce":244560995}]},"public_addr":"172.21.15.187:6816/244560995","cluster_addr":"172.21.15.187:6819/244560995","heartbeat_back_addr":"172.21.15.187:6823/244560995","heartbeat_front_addr":"172.21.15.187:6821/244560995","state":["exists","up"]},{"osd":5,"uuid":"7513ba69-dd00-4a35-b190-9e8cc3b2bdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6801","nonce":324619398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6803","nonce":324619398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6812","nonce":324619398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6807","nonce":324619398}]},"public_addr":"172.21.15.187:6801/324619398","cluster_addr":"172.21.15.187:6803/324619398","heartbeat_back_addr":"172.21.15.187:6812/324619398","heartbeat_front_addr":"172.21.15.187:6807/324619398","state":["exists","up"]},{"osd":6,"uuid":"50081810-c093-4d2f-a4d7-b7c695c5400d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6806","nonce":1391347786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6809","nonce":1391347786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6818","nonce":1391347786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6813","nonce":1391347786}]},"public_addr":"172.21.15.187:6806/1391347786","cluster_addr":"172.21.15.187:6809/1391347786","heartbeat_back_addr":"172.21.15.187:6818/1391347786","heartbeat_front_addr":"172.21.15.187:6813/1391347786","state":["exists","up"]},{"osd":7,"uuid":"775dae05-54f5-4ad4-bbce-92863c6ed2ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6825","nonce":1220065622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6827","nonce":1220065622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6831","nonce":1220065622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6829","nonce":1220065622}]},"public_addr":"172.21.15.187:6825/1220065622","cluster_addr":"172.21.15.187:6827/1220065622","heartbeat_back_addr":"172.21.15.187:6831/1220065622","heartbeat_front_addr":"172.21.15.187:6829/1220065622","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:05.983520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.299516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.067323+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.307748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.018742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.258775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.010912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.295355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.a","mappings":[{"from":6,"to":0}]},{"pgid":"11.22","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-26T22:53:35.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.984+0000 7f6b1e400640 1 -- 172.21.15.131:0/2575979333 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f6af406be30 msgr2=0x7f6af406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:35.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.984+0000 7f6b1e400640 1 --2- 172.21.15.131:0/2575979333 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f6af406be30 0x7f6af406e2f0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f6b180b2dd0 tx=0x7f6b00007e30 comp rx=0 tx=0).stop 2024-05-26T22:53:35.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.984+0000 7f6b1e400640 1 -- 172.21.15.131:0/2575979333 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6b1806e8c0 msgr2=0x7f6b18140da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:35.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.984+0000 7f6b1e400640 1 --2- 172.21.15.131:0/2575979333 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6b1806e8c0 0x7f6b18140da0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f6b0800e870 tx=0x7f6b0800ed30 comp rx=0 tx=0).stop 2024-05-26T22:53:35.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.984+0000 7f6b1e400640 1 -- 172.21.15.131:0/2575979333 shutdown_connections 2024-05-26T22:53:35.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.984+0000 7f6b1e400640 1 --2- 172.21.15.131:0/2575979333 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6b180cc7f0 0x7f6b18159630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:35.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.984+0000 7f6b1e400640 1 --2- 172.21.15.131:0/2575979333 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f6af406be30 0x7f6af406e2f0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:35.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.984+0000 7f6b1e400640 1 --2- 172.21.15.131:0/2575979333 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6b18156750 0x7f6b181e67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:35.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.984+0000 7f6b1e400640 1 --2- 172.21.15.131:0/2575979333 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6b1806e8c0 0x7f6b18140da0 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:35.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.984+0000 7f6b1e400640 1 -- 172.21.15.131:0/2575979333 >> 172.21.15.131:0/2575979333 conn(0x7f6b180cff40 msgr2=0x7f6b1815d060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:35.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.984+0000 7f6b1e400640 1 -- 172.21.15.131:0/2575979333 shutdown_connections 2024-05-26T22:53:35.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:35.984+0000 7f6b1e400640 1 -- 172.21.15.131:0/2575979333 wait complete. 2024-05-26T22:53:36.004 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-26T22:53:36.004 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-26T22:53:36.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.208+0000 7f0f56200640 1 Processor -- start 2024-05-26T22:53:36.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.208+0000 7f0f56200640 1 -- start start 2024-05-26T22:53:36.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.210+0000 7f0f56200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f0f5006e8c0 0x7f0f5015d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:36.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.210+0000 7f0f56200640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f50160620 con 0x7f0f50156750 2024-05-26T22:53:36.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.210+0000 7f0f56200640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f50160900 con 0x7f0f500cc7f0 2024-05-26T22:53:36.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.210+0000 7f0f56200640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f0f5015cbd0 con 0x7f0f5006e8c0 2024-05-26T22:53:36.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.210+0000 7f0f4fe00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f0f5006e8c0 0x7f0f5015d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:36.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.210+0000 7f0f4fe00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f0f5006e8c0 0x7f0f5015d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50964/0 (socket says 172.21.15.131:50964) 2024-05-26T22:53:36.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.210+0000 7f0f4fe00640 1 -- 172.21.15.131:0/3110209616 learned_addr learned my addr 172.21.15.131:0/3110209616 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:53:36.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.210+0000 7f0f54e00640 1 --1- 172.21.15.131:0/3110209616 >> v1:172.21.15.131:6789/0 conn(0x7f0f50156750 0x7f0f5015fd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:47686/0 (socket says 172.21.15.131:47686) 2024-05-26T22:53:36.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.211+0000 7f0f4fe00640 1 -- 172.21.15.131:0/3110209616 >> v1:172.21.15.187:6789/0 conn(0x7f0f500cc7f0 legacy=0x7f0f500cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:36.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.211+0000 7f0f4fe00640 1 -- 172.21.15.131:0/3110209616 >> v1:172.21.15.131:6789/0 conn(0x7f0f50156750 legacy=0x7f0f5015fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:36.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.211+0000 7f0f4fe00640 1 -- 172.21.15.131:0/3110209616 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f50160be0 con 0x7f0f5006e8c0 2024-05-26T22:53:36.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.211+0000 7f0f4f400640 1 -- 172.21.15.131:0/3110209616 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3180931854 0 0) 0x7f0f50160620 con 0x7f0f50156750 2024-05-26T22:53:36.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.212+0000 7f0f4fe00640 1 --2- 172.21.15.131:0/3110209616 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0f5006e8c0 0x7f0f5015d060 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f0f5015cd50 tx=0x7f0f3c02a290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c121fa4e36ef324 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:36.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.212+0000 7f0f4f400640 1 -- 172.21.15.131:0/3110209616 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f3c037070 con 0x7f0f5006e8c0 2024-05-26T22:53:36.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.212+0000 7f0f4f400640 1 -- 172.21.15.131:0/3110209616 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f3c032440 con 0x7f0f5006e8c0 2024-05-26T22:53:36.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.212+0000 7f0f4f400640 1 -- 172.21.15.131:0/3110209616 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f3c032810 con 0x7f0f5006e8c0 2024-05-26T22:53:36.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.213+0000 7f0f56200640 1 -- 172.21.15.131:0/3110209616 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0f5006e8c0 msgr2=0x7f0f5015d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:36.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.213+0000 7f0f56200640 1 --2- 172.21.15.131:0/3110209616 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0f5006e8c0 0x7f0f5015d060 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f0f5015cd50 tx=0x7f0f3c02a290 comp rx=0 tx=0).stop 2024-05-26T22:53:36.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.213+0000 7f0f56200640 1 -- 172.21.15.131:0/3110209616 shutdown_connections 2024-05-26T22:53:36.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.213+0000 7f0f56200640 1 --2- 172.21.15.131:0/3110209616 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0f5006e8c0 0x7f0f5015d060 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:36.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.213+0000 7f0f56200640 1 -- 172.21.15.131:0/3110209616 >> 172.21.15.131:0/3110209616 conn(0x7f0f500cff40 msgr2=0x7f0f500d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:36.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.213+0000 7f0f56200640 1 -- 172.21.15.131:0/3110209616 shutdown_connections 2024-05-26T22:53:36.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.213+0000 7f0f56200640 1 -- 172.21.15.131:0/3110209616 wait complete. 2024-05-26T22:53:36.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.215+0000 7f0f56200640 1 Processor -- start 2024-05-26T22:53:36.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.215+0000 7f0f56200640 1 -- start start 2024-05-26T22:53:36.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.216+0000 7f0f56200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0f5006e8c0 0x7f0f5014fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:36.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.216+0000 7f0f56200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0f500cc7f0 0x7f0f50152e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:36.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.216+0000 7f0f56200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0f50156750 0x7f0f501509c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:36.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.216+0000 7f0f56200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f0f5015d5a0 con 0x7f0f500cc7f0 2024-05-26T22:53:36.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.216+0000 7f0f56200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0f50156b30 con 0x7f0f50156750 2024-05-26T22:53:36.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.216+0000 7f0f56200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f0f5015d720 con 0x7f0f5006e8c0 2024-05-26T22:53:36.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.216+0000 7f0f4fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0f500cc7f0 0x7f0f50152e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:36.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.216+0000 7f0f4fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0f500cc7f0 0x7f0f50152e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50978/0 (socket says 172.21.15.131:50978) 2024-05-26T22:53:36.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.216+0000 7f0f4fe00640 1 -- 172.21.15.131:0/3179478236 learned_addr learned my addr 172.21.15.131:0/3179478236 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:53:36.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.216+0000 7f0f55800640 1 --2- 172.21.15.131:0/3179478236 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0f50156750 0x7f0f501509c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:36.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.216+0000 7f0f4fe00640 1 -- 172.21.15.131:0/3179478236 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0f5006e8c0 msgr2=0x7f0f5014fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:36.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.216+0000 7f0f4fe00640 1 --2- 172.21.15.131:0/3179478236 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0f5006e8c0 0x7f0f5014fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:36.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.216+0000 7f0f4fe00640 1 -- 172.21.15.131:0/3179478236 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0f50156750 msgr2=0x7f0f501509c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:36.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.216+0000 7f0f4fe00640 1 --2- 172.21.15.131:0/3179478236 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0f50156750 0x7f0f501509c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:36.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.216+0000 7f0f4fe00640 1 -- 172.21.15.131:0/3179478236 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f50150f00 con 0x7f0f500cc7f0 2024-05-26T22:53:36.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.217+0000 7f0f4fe00640 1 --2- 172.21.15.131:0/3179478236 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0f500cc7f0 0x7f0f50152e40 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c000c00 tx=0x7f0f3c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:36.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.217+0000 7f0f4cc00640 1 -- 172.21.15.131:0/3179478236 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f3c041070 con 0x7f0f500cc7f0 2024-05-26T22:53:36.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.217+0000 7f0f4cc00640 1 -- 172.21.15.131:0/3179478236 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f3c03c870 con 0x7f0f500cc7f0 2024-05-26T22:53:36.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.217+0000 7f0f4cc00640 1 -- 172.21.15.131:0/3179478236 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f3c037040 con 0x7f0f500cc7f0 2024-05-26T22:53:36.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.217+0000 7f0f56200640 1 -- 172.21.15.131:0/3179478236 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f50157540 con 0x7f0f500cc7f0 2024-05-26T22:53:36.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.217+0000 7f0f56200640 1 -- 172.21.15.131:0/3179478236 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f50157b00 con 0x7f0f500cc7f0 2024-05-26T22:53:36.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.218+0000 7f0f56200640 1 -- 172.21.15.131:0/3179478236 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f500694e0 con 0x7f0f500cc7f0 2024-05-26T22:53:36.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.219+0000 7f0f4cc00640 1 -- 172.21.15.131:0/3179478236 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0f3c044070 con 0x7f0f500cc7f0 2024-05-26T22:53:36.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.220+0000 7f0f4cc00640 1 --2- 172.21.15.131:0/3179478236 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f0f3006be30 0x7f0f3006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:36.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.220+0000 7f0f4cc00640 1 -- 172.21.15.131:0/3179478236 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5767+0+0 (secure 0 0 0) 0x7f0f3c0abc30 con 0x7f0f500cc7f0 2024-05-26T22:53:36.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.220+0000 7f0f54e00640 1 --2- 172.21.15.131:0/3179478236 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f0f3006be30 0x7f0f3006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:36.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.221+0000 7f0f54e00640 1 --2- 172.21.15.131:0/3179478236 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f0f3006be30 0x7f0f3006e2f0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f0f44007fd0 tx=0x7f0f44007960 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:36.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.224+0000 7f0f4cc00640 1 -- 172.21.15.131:0/3179478236 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f0f3c07a6b0 con 0x7f0f500cc7f0 2024-05-26T22:53:36.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.446+0000 7f0f56200640 1 -- 172.21.15.131:0/3179478236 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0f50159c30 con 0x7f0f500cc7f0 2024-05-26T22:53:36.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.447+0000 7f0f4cc00640 1 -- 172.21.15.131:0/3179478236 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) ==== 94+0+5236 (secure 0 0 0) 0x7f0f3c079fb0 con 0x7f0f500cc7f0 2024-05-26T22:53:36.447 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:53:36.447 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":69,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5359,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/4204396845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6833","nonce":4204396845},{"type":"v1","addr":"172.21.15.187:6835","nonce":4204396845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-26T22:52:53.755842+0000","modified":"2024-05-26T22:53:23.885672+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6876,"mds_1":6864,"mds_2":6861},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6861":{"gid":6861,"name":"a","rank":2,"incarnation":67,"state":"up:active","state_seq":6,"addr":"172.21.15.131:6836/4223534499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":4223534499},{"type":"v1","addr":"172.21.15.131:6836","nonce":4223534499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6864":{"gid":6864,"name":"c","rank":1,"incarnation":65,"state":"up:active","state_seq":6,"addr":"172.21.15.131:6837/1418741551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6835","nonce":1418741551},{"type":"v1","addr":"172.21.15.131:6837","nonce":1418741551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6876":{"gid":6876,"name":"d","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6834/3680999672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3680999672},{"type":"v1","addr":"172.21.15.187:6834","nonce":3680999672}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6876,"qdb_cluster":[6876,6864,6861]},"id":5}]} 2024-05-26T22:53:36.448 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 69 2024-05-26T22:53:36.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.450+0000 7f0f56200640 1 -- 172.21.15.131:0/3179478236 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f0f3006be30 msgr2=0x7f0f3006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:36.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.450+0000 7f0f56200640 1 --2- 172.21.15.131:0/3179478236 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f0f3006be30 0x7f0f3006e2f0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f0f44007fd0 tx=0x7f0f44007960 comp rx=0 tx=0).stop 2024-05-26T22:53:36.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.450+0000 7f0f56200640 1 -- 172.21.15.131:0/3179478236 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0f500cc7f0 msgr2=0x7f0f50152e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:36.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.450+0000 7f0f56200640 1 --2- 172.21.15.131:0/3179478236 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0f500cc7f0 0x7f0f50152e40 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c000c00 tx=0x7f0f3c0046c0 comp rx=0 tx=0).stop 2024-05-26T22:53:36.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.450+0000 7f0f56200640 1 -- 172.21.15.131:0/3179478236 shutdown_connections 2024-05-26T22:53:36.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.450+0000 7f0f56200640 1 --2- 172.21.15.131:0/3179478236 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f0f3006be30 0x7f0f3006e2f0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:36.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.450+0000 7f0f56200640 1 --2- 172.21.15.131:0/3179478236 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0f50156750 0x7f0f501509c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:36.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.450+0000 7f0f56200640 1 --2- 172.21.15.131:0/3179478236 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0f500cc7f0 0x7f0f50152e40 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:36.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.450+0000 7f0f56200640 1 --2- 172.21.15.131:0/3179478236 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0f5006e8c0 0x7f0f5014fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:36.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.450+0000 7f0f56200640 1 -- 172.21.15.131:0/3179478236 >> 172.21.15.131:0/3179478236 conn(0x7f0f500cff40 msgr2=0x7f0f5015a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:36.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.450+0000 7f0f56200640 1 -- 172.21.15.131:0/3179478236 shutdown_connections 2024-05-26T22:53:36.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.450+0000 7f0f56200640 1 -- 172.21.15.131:0/3179478236 wait complete. 2024-05-26T22:53:36.468 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-26T22:53:36.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.669+0000 7f94f9800640 1 Processor -- start 2024-05-26T22:53:36.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.669+0000 7f94f9800640 1 -- start start 2024-05-26T22:53:36.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.670+0000 7f94f9800640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f94f406ee20 0x7f94f40b3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:36.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.670+0000 7f94f9800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f94f40ad250 0x7f94f40af790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:36.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.670+0000 7f94f9800640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94f415cb20 con 0x7f94f406f200 2024-05-26T22:53:36.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.670+0000 7f94f9800640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f94f415ce00 con 0x7f94f406ee20 2024-05-26T22:53:36.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.670+0000 7f94f9800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f94f415d0e0 con 0x7f94f40ad250 2024-05-26T22:53:36.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.671+0000 7f94f2a00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f94f406ee20 0x7f94f40b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:36.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.671+0000 7f94f2a00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f94f406ee20 0x7f94f40b3190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50982/0 (socket says 172.21.15.131:50982) 2024-05-26T22:53:36.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.671+0000 7f94f2a00640 1 -- 172.21.15.131:0/2305366441 learned_addr learned my addr 172.21.15.131:0/2305366441 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:53:36.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.671+0000 7f94f2a00640 1 -- 172.21.15.131:0/2305366441 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f94f40ad250 msgr2=0x7f94f40af790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:36.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.671+0000 7f94f2a00640 1 --2- 172.21.15.131:0/2305366441 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f94f40ad250 0x7f94f40af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:36.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.671+0000 7f94f2a00640 1 -- 172.21.15.131:0/2305366441 >> v1:172.21.15.187:6789/0 conn(0x7f94f406f200 legacy=0x7f94f41562c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:36.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.671+0000 7f94f2a00640 1 -- 172.21.15.131:0/2305366441 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94f415d3c0 con 0x7f94f406ee20 2024-05-26T22:53:36.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.672+0000 7f94f2a00640 1 --2- 172.21.15.131:0/2305366441 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f94f406ee20 0x7f94f40b3190 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f94e4009870 tx=0x7f94e402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13539187d9a0feca server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:36.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.672+0000 7f94f2000640 1 -- 172.21.15.131:0/2305366441 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94e4037070 con 0x7f94f406ee20 2024-05-26T22:53:36.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.672+0000 7f94f2000640 1 -- 172.21.15.131:0/2305366441 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94e4032440 con 0x7f94f406ee20 2024-05-26T22:53:36.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.673+0000 7f94f2000640 1 -- 172.21.15.131:0/2305366441 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94e4032810 con 0x7f94f406ee20 2024-05-26T22:53:36.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.673+0000 7f94f9800640 1 -- 172.21.15.131:0/2305366441 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f94f406ee20 msgr2=0x7f94f40b3190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:36.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.673+0000 7f94f9800640 1 --2- 172.21.15.131:0/2305366441 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f94f406ee20 0x7f94f40b3190 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f94e4009870 tx=0x7f94e402a210 comp rx=0 tx=0).stop 2024-05-26T22:53:36.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.673+0000 7f94f9800640 1 -- 172.21.15.131:0/2305366441 shutdown_connections 2024-05-26T22:53:36.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.673+0000 7f94f9800640 1 --2- 172.21.15.131:0/2305366441 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f94f40ad250 0x7f94f40af790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:36.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.673+0000 7f94f9800640 1 --2- 172.21.15.131:0/2305366441 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f94f406ee20 0x7f94f40b3190 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:36.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.673+0000 7f94f9800640 1 -- 172.21.15.131:0/2305366441 >> 172.21.15.131:0/2305366441 conn(0x7f94f40b1e90 msgr2=0x7f94f40b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:36.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.673+0000 7f94f9800640 1 -- 172.21.15.131:0/2305366441 shutdown_connections 2024-05-26T22:53:36.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.673+0000 7f94f9800640 1 -- 172.21.15.131:0/2305366441 wait complete. 2024-05-26T22:53:36.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.675+0000 7f94f9800640 1 Processor -- start 2024-05-26T22:53:36.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.675+0000 7f94f9800640 1 -- start start 2024-05-26T22:53:36.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.676+0000 7f94f9800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f94f406ee20 0x7f94f41ee350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:36.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.676+0000 7f94f9800640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f94f406f200 0x7f94f4072880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:36.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.676+0000 7f94f9800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f94f40ad250 0x7f94f4072dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:36.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.676+0000 7f94f9800640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f94f415c900 con 0x7f94f40ad250 2024-05-26T22:53:36.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.676+0000 7f94f9800640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f94f415c780 con 0x7f94f406f200 2024-05-26T22:53:36.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.676+0000 7f94f9800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f94f41569d0 con 0x7f94f406ee20 2024-05-26T22:53:36.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.677+0000 7f94f3400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f94f40ad250 0x7f94f4072dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:36.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.677+0000 7f94f3400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f94f40ad250 0x7f94f4072dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50990/0 (socket says 172.21.15.131:50990) 2024-05-26T22:53:36.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.677+0000 7f94f3400640 1 -- 172.21.15.131:0/850388265 learned_addr learned my addr 172.21.15.131:0/850388265 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:53:36.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.677+0000 7f94ea000640 1 --2- 172.21.15.131:0/850388265 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f94f406f200 0x7f94f4072880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:36.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.677+0000 7f94f2a00640 1 --2- 172.21.15.131:0/850388265 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f94f406ee20 0x7f94f41ee350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:36.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.677+0000 7f94f3400640 1 -- 172.21.15.131:0/850388265 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f94f406ee20 msgr2=0x7f94f41ee350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:36.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.677+0000 7f94f3400640 1 --2- 172.21.15.131:0/850388265 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f94f406ee20 0x7f94f41ee350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:36.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.677+0000 7f94f3400640 1 -- 172.21.15.131:0/850388265 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f94f406f200 msgr2=0x7f94f4072880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:36.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.677+0000 7f94f3400640 1 --2- 172.21.15.131:0/850388265 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f94f406f200 0x7f94f4072880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:36.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.677+0000 7f94f3400640 1 -- 172.21.15.131:0/850388265 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94f41f7cc0 con 0x7f94f40ad250 2024-05-26T22:53:36.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.677+0000 7f94f3400640 1 --2- 172.21.15.131:0/850388265 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f94f40ad250 0x7f94f4072dc0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f94ec00c780 tx=0x7f94ec00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:36.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.678+0000 7f94eb400640 1 -- 172.21.15.131:0/850388265 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94ec017070 con 0x7f94f40ad250 2024-05-26T22:53:36.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.678+0000 7f94eb400640 1 -- 172.21.15.131:0/850388265 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94ec003da0 con 0x7f94f40ad250 2024-05-26T22:53:36.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.678+0000 7f94eb400640 1 -- 172.21.15.131:0/850388265 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94ec0052b0 con 0x7f94f40ad250 2024-05-26T22:53:36.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.678+0000 7f94f9800640 1 -- 172.21.15.131:0/850388265 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94f41f7f80 con 0x7f94f40ad250 2024-05-26T22:53:36.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.678+0000 7f94f9800640 1 -- 172.21.15.131:0/850388265 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94f41f8540 con 0x7f94f40ad250 2024-05-26T22:53:36.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.679+0000 7f94f9800640 1 -- 172.21.15.131:0/850388265 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94f40694e0 con 0x7f94f40ad250 2024-05-26T22:53:36.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.680+0000 7f94eb400640 1 -- 172.21.15.131:0/850388265 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f94ec005450 con 0x7f94f40ad250 2024-05-26T22:53:36.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.681+0000 7f94eb400640 1 --2- 172.21.15.131:0/850388265 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f94d006be60 0x7f94d006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:36.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.681+0000 7f94eb400640 1 -- 172.21.15.131:0/850388265 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5767+0+0 (secure 0 0 0) 0x7f94ec08a4b0 con 0x7f94f40ad250 2024-05-26T22:53:36.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.681+0000 7f94f2a00640 1 --2- 172.21.15.131:0/850388265 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f94d006be60 0x7f94d006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:36.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.682+0000 7f94f2a00640 1 --2- 172.21.15.131:0/850388265 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f94d006be60 0x7f94d006e320 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f94e4034000 tx=0x7f94e402e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:36.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.685+0000 7f94eb400640 1 -- 172.21.15.131:0/850388265 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f94ec058d80 con 0x7f94f40ad250 2024-05-26T22:53:36.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.922+0000 7f94f9800640 1 -- 172.21.15.131:0/850388265 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f94f40b3720 con 0x7f94f40ad250 2024-05-26T22:53:36.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.923+0000 7f94eb400640 1 -- 172.21.15.131:0/850388265 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) ==== 94+0+5236 (secure 0 0 0) 0x7f94ec058680 con 0x7f94f40ad250 2024-05-26T22:53:36.923 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:53:36.923 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":69,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5359,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/4204396845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6833","nonce":4204396845},{"type":"v1","addr":"172.21.15.187:6835","nonce":4204396845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-26T22:52:53.755842+0000","modified":"2024-05-26T22:53:23.885672+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6876,"mds_1":6864,"mds_2":6861},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6861":{"gid":6861,"name":"a","rank":2,"incarnation":67,"state":"up:active","state_seq":6,"addr":"172.21.15.131:6836/4223534499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":4223534499},{"type":"v1","addr":"172.21.15.131:6836","nonce":4223534499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6864":{"gid":6864,"name":"c","rank":1,"incarnation":65,"state":"up:active","state_seq":6,"addr":"172.21.15.131:6837/1418741551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6835","nonce":1418741551},{"type":"v1","addr":"172.21.15.131:6837","nonce":1418741551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6876":{"gid":6876,"name":"d","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6834/3680999672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3680999672},{"type":"v1","addr":"172.21.15.187:6834","nonce":3680999672}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6876,"qdb_cluster":[6876,6864,6861]},"id":5}]} 2024-05-26T22:53:36.923 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 69 2024-05-26T22:53:36.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.925+0000 7f94f9800640 1 -- 172.21.15.131:0/850388265 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f94d006be60 msgr2=0x7f94d006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:36.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.925+0000 7f94f9800640 1 --2- 172.21.15.131:0/850388265 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f94d006be60 0x7f94d006e320 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f94e4034000 tx=0x7f94e402e000 comp rx=0 tx=0).stop 2024-05-26T22:53:36.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.925+0000 7f94f9800640 1 -- 172.21.15.131:0/850388265 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f94f40ad250 msgr2=0x7f94f4072dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:36.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.925+0000 7f94f9800640 1 --2- 172.21.15.131:0/850388265 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f94f40ad250 0x7f94f4072dc0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f94ec00c780 tx=0x7f94ec00cc40 comp rx=0 tx=0).stop 2024-05-26T22:53:36.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.925+0000 7f94f9800640 1 -- 172.21.15.131:0/850388265 shutdown_connections 2024-05-26T22:53:36.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.925+0000 7f94f9800640 1 --2- 172.21.15.131:0/850388265 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f94d006be60 0x7f94d006e320 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:36.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.925+0000 7f94f9800640 1 --2- 172.21.15.131:0/850388265 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f94f40ad250 0x7f94f4072dc0 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:36.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.925+0000 7f94f9800640 1 --2- 172.21.15.131:0/850388265 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f94f406f200 0x7f94f4072880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:36.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.925+0000 7f94f9800640 1 --2- 172.21.15.131:0/850388265 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f94f406ee20 0x7f94f41ee350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:36.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.925+0000 7f94f9800640 1 -- 172.21.15.131:0/850388265 >> 172.21.15.131:0/850388265 conn(0x7f94f40b1e90 msgr2=0x7f94f415d990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:36.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.926+0000 7f94f9800640 1 -- 172.21.15.131:0/850388265 shutdown_connections 2024-05-26T22:53:36.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:36.926+0000 7f94f9800640 1 -- 172.21.15.131:0/850388265 wait complete. 2024-05-26T22:53:36.945 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-26T22:53:37.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.148+0000 7fb035a00640 1 Processor -- start 2024-05-26T22:53:37.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.149+0000 7fb035a00640 1 -- start start 2024-05-26T22:53:37.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.150+0000 7fb035a00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fb03015e570 0x7fb0300d4c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:37.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.150+0000 7fb035a00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fb0300d19b0 0x7fb0300d2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:37.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.150+0000 7fb035a00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0300b37e0 con 0x7fb0300b2f60 2024-05-26T22:53:37.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.150+0000 7fb035a00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fb0300d51b0 con 0x7fb0300d19b0 2024-05-26T22:53:37.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.150+0000 7fb035a00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fb0300d47e0 con 0x7fb03015e570 2024-05-26T22:53:37.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.150+0000 7fb02f400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fb0300d19b0 0x7fb0300d2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:37.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.150+0000 7fb02e000640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fb0300b2f60 0x7fb0300d12a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:47690/0 (socket says 172.21.15.131:47690) 2024-05-26T22:53:37.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.150+0000 7fb02e000640 1 -- 172.21.15.131:0/1840216282 learned_addr learned my addr 172.21.15.131:0/1840216282 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:53:37.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.151+0000 7fb02ea00640 1 --2- 172.21.15.131:0/1840216282 >> v2:172.21.15.187:3300/0 conn(0x7fb03015e570 0x7fb0300d4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:37.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.151+0000 7fb02f400640 1 -- 172.21.15.131:0/1840216282 >> v2:172.21.15.187:3300/0 conn(0x7fb03015e570 msgr2=0x7fb0300d4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:37.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.151+0000 7fb02f400640 1 --2- 172.21.15.131:0/1840216282 >> v2:172.21.15.187:3300/0 conn(0x7fb03015e570 0x7fb0300d4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:37.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.151+0000 7fb02f400640 1 -- 172.21.15.131:0/1840216282 >> v1:172.21.15.131:6789/0 conn(0x7fb0300b2f60 legacy=0x7fb0300d12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:37.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.151+0000 7fb02f400640 1 -- 172.21.15.131:0/1840216282 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0300b9cf0 con 0x7fb0300d19b0 2024-05-26T22:53:37.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.151+0000 7fb02d600640 1 -- 172.21.15.131:0/1840216282 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2099407854 0 0) 0x7fb01c001490 con 0x7fb0300b2f60 2024-05-26T22:53:37.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.152+0000 7fb02f400640 1 --2- 172.21.15.131:0/1840216282 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb0300d19b0 0x7fb0300d2590 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7fb0300b6a90 tx=0x7fb02802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9d3d02afac2e8d2 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:37.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.152+0000 7fb02d600640 1 -- 172.21.15.131:0/1840216282 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb028037070 con 0x7fb0300d19b0 2024-05-26T22:53:37.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.152+0000 7fb02d600640 1 -- 172.21.15.131:0/1840216282 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb028032440 con 0x7fb0300d19b0 2024-05-26T22:53:37.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.152+0000 7fb02d600640 1 -- 172.21.15.131:0/1840216282 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb028032810 con 0x7fb0300d19b0 2024-05-26T22:53:37.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.153+0000 7fb035a00640 1 -- 172.21.15.131:0/1840216282 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb0300d19b0 msgr2=0x7fb0300d2590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:37.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.153+0000 7fb035a00640 1 --2- 172.21.15.131:0/1840216282 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb0300d19b0 0x7fb0300d2590 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7fb0300b6a90 tx=0x7fb02802a1a0 comp rx=0 tx=0).stop 2024-05-26T22:53:37.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.153+0000 7fb035a00640 1 -- 172.21.15.131:0/1840216282 shutdown_connections 2024-05-26T22:53:37.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.153+0000 7fb035a00640 1 --2- 172.21.15.131:0/1840216282 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb0300d19b0 0x7fb0300d2590 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:37.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.153+0000 7fb035a00640 1 --2- 172.21.15.131:0/1840216282 >> v2:172.21.15.187:3300/0 conn(0x7fb03015e570 0x7fb0300d4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:37.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.153+0000 7fb035a00640 1 -- 172.21.15.131:0/1840216282 >> 172.21.15.131:0/1840216282 conn(0x7fb030072e90 msgr2=0x7fb0300732a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:37.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.153+0000 7fb035a00640 1 -- 172.21.15.131:0/1840216282 shutdown_connections 2024-05-26T22:53:37.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.153+0000 7fb035a00640 1 -- 172.21.15.131:0/1840216282 wait complete. 2024-05-26T22:53:37.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.155+0000 7fb035a00640 1 Processor -- start 2024-05-26T22:53:37.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.155+0000 7fb035a00640 1 -- start start 2024-05-26T22:53:37.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.156+0000 7fb035a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb0300b2f60 0x7fb0300ca7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:37.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.156+0000 7fb035a00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb0300d19b0 0x7fb03014c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:37.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.156+0000 7fb035a00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb03015e570 0x7fb03015e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:37.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.156+0000 7fb035a00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fb0300d1ea0 con 0x7fb0300b2f60 2024-05-26T22:53:37.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.156+0000 7fb035a00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb0300d4470 con 0x7fb0300d19b0 2024-05-26T22:53:37.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.156+0000 7fb035a00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb0300d2020 con 0x7fb03015e570 2024-05-26T22:53:37.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.156+0000 7fb02ea00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb0300b2f60 0x7fb0300ca7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:37.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.156+0000 7fb02ea00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb0300b2f60 0x7fb0300ca7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51000/0 (socket says 172.21.15.131:51000) 2024-05-26T22:53:37.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.156+0000 7fb02ea00640 1 -- 172.21.15.131:0/2342220939 learned_addr learned my addr 172.21.15.131:0/2342220939 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:53:37.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.156+0000 7fb02e000640 1 --2- 172.21.15.131:0/2342220939 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb0300d19b0 0x7fb03014c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:37.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.156+0000 7fb02ea00640 1 -- 172.21.15.131:0/2342220939 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb03015e570 msgr2=0x7fb03015e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:37.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.156+0000 7fb02ea00640 1 --2- 172.21.15.131:0/2342220939 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb03015e570 0x7fb03015e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:37.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.157+0000 7fb02ea00640 1 -- 172.21.15.131:0/2342220939 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb0300d19b0 msgr2=0x7fb03014c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:37.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.157+0000 7fb02ea00640 1 --2- 172.21.15.131:0/2342220939 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb0300d19b0 0x7fb03014c010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:37.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.157+0000 7fb02ea00640 1 -- 172.21.15.131:0/2342220939 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb03015f2c0 con 0x7fb0300b2f60 2024-05-26T22:53:37.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.157+0000 7fb02ea00640 1 --2- 172.21.15.131:0/2342220939 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb0300b2f60 0x7fb0300ca7e0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7fb02000d800 tx=0x7fb02000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:37.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.157+0000 7fb026a00640 1 -- 172.21.15.131:0/2342220939 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb02000b6c0 con 0x7fb0300b2f60 2024-05-26T22:53:37.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.157+0000 7fb026a00640 1 -- 172.21.15.131:0/2342220939 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb020019070 con 0x7fb0300b2f60 2024-05-26T22:53:37.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.157+0000 7fb035a00640 1 -- 172.21.15.131:0/2342220939 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb03015f5a0 con 0x7fb0300b2f60 2024-05-26T22:53:37.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.157+0000 7fb026a00640 1 -- 172.21.15.131:0/2342220939 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb02000ba80 con 0x7fb0300b2f60 2024-05-26T22:53:37.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.157+0000 7fb035a00640 1 -- 172.21.15.131:0/2342220939 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb03015fb60 con 0x7fb0300b2f60 2024-05-26T22:53:37.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.158+0000 7fb035a00640 1 -- 172.21.15.131:0/2342220939 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0300694e0 con 0x7fb0300b2f60 2024-05-26T22:53:37.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.159+0000 7fb026a00640 1 -- 172.21.15.131:0/2342220939 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb02000bc20 con 0x7fb0300b2f60 2024-05-26T22:53:37.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.160+0000 7fb026a00640 1 --2- 172.21.15.131:0/2342220939 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb01006be30 0x7fb01006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:37.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.160+0000 7fb026a00640 1 -- 172.21.15.131:0/2342220939 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5767+0+0 (secure 0 0 0) 0x7fb02008b380 con 0x7fb0300b2f60 2024-05-26T22:53:37.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.160+0000 7fb02e000640 1 --2- 172.21.15.131:0/2342220939 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb01006be30 0x7fb01006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:37.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.161+0000 7fb02e000640 1 --2- 172.21.15.131:0/2342220939 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb01006be30 0x7fb01006e2f0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fb030162c00 tx=0x7fb01c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:37.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.165+0000 7fb026a00640 1 -- 172.21.15.131:0/2342220939 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fb020059ce0 con 0x7fb0300b2f60 2024-05-26T22:53:37.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.364+0000 7fb035a00640 1 -- 172.21.15.131:0/2342220939 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb0300b4a70 con 0x7fb0300b2f60 2024-05-26T22:53:37.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.367+0000 7fb026a00640 1 -- 172.21.15.131:0/2342220939 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) ==== 74+0+16789 (secure 0 0 0) 0x7fb0200595e0 con 0x7fb0300b2f60 2024-05-26T22:53:37.367 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:53:37.367 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":71,"fsid":"46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f","created":"2024-05-26T22:47:57.288905+0000","modified":"2024-05-26T22:53:06.170553+0000","last_up_change":"2024-05-26T22:48:09.602207+0000","last_in_change":"2024-05-26T22:48:00.687341+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-26T22:48:10.463339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-05-26T22:52:51.536251+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-05-26T22:52:52.768524+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"814eaf62-a18d-494b-8e27-fd8fb88adea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6801","nonce":2350940051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6803","nonce":2350940051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6807","nonce":2350940051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6805","nonce":2350940051}]},"public_addr":"172.21.15.131:6801/2350940051","cluster_addr":"172.21.15.131:6803/2350940051","heartbeat_back_addr":"172.21.15.131:6807/2350940051","heartbeat_front_addr":"172.21.15.131:6805/2350940051","state":["exists","up"]},{"osd":1,"uuid":"42e66287-f6c6-4e26-af3c-6dfaa3536d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6825","nonce":1525528247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6827","nonce":1525528247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6831","nonce":1525528247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6829","nonce":1525528247}]},"public_addr":"172.21.15.131:6825/1525528247","cluster_addr":"172.21.15.131:6827/1525528247","heartbeat_back_addr":"172.21.15.131:6831/1525528247","heartbeat_front_addr":"172.21.15.131:6829/1525528247","state":["exists","up"]},{"osd":2,"uuid":"0da34673-04a2-4fd0-b57a-0ff2b679b043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6817","nonce":3627842210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6819","nonce":3627842210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6823","nonce":3627842210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6821","nonce":3627842210}]},"public_addr":"172.21.15.131:6817/3627842210","cluster_addr":"172.21.15.131:6819/3627842210","heartbeat_back_addr":"172.21.15.131:6823/3627842210","heartbeat_front_addr":"172.21.15.131:6821/3627842210","state":["exists","up"]},{"osd":3,"uuid":"cf117a27-d6b2-43d3-ae52-b47988bf8eed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6809","nonce":350790591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6811","nonce":350790591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6815","nonce":350790591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6813","nonce":350790591}]},"public_addr":"172.21.15.131:6809/350790591","cluster_addr":"172.21.15.131:6811/350790591","heartbeat_back_addr":"172.21.15.131:6815/350790591","heartbeat_front_addr":"172.21.15.131:6813/350790591","state":["exists","up"]},{"osd":4,"uuid":"a092ccfc-be89-4a84-88d5-0f781fa715eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6816","nonce":244560995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6819","nonce":244560995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6823","nonce":244560995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6821","nonce":244560995}]},"public_addr":"172.21.15.187:6816/244560995","cluster_addr":"172.21.15.187:6819/244560995","heartbeat_back_addr":"172.21.15.187:6823/244560995","heartbeat_front_addr":"172.21.15.187:6821/244560995","state":["exists","up"]},{"osd":5,"uuid":"7513ba69-dd00-4a35-b190-9e8cc3b2bdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6801","nonce":324619398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6803","nonce":324619398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6812","nonce":324619398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6807","nonce":324619398}]},"public_addr":"172.21.15.187:6801/324619398","cluster_addr":"172.21.15.187:6803/324619398","heartbeat_back_addr":"172.21.15.187:6812/324619398","heartbeat_front_addr":"172.21.15.187:6807/324619398","state":["exists","up"]},{"osd":6,"uuid":"50081810-c093-4d2f-a4d7-b7c695c5400d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6806","nonce":1391347786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6809","nonce":1391347786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6818","nonce":1391347786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6813","nonce":1391347786}]},"public_addr":"172.21.15.187:6806/1391347786","cluster_addr":"172.21.15.187:6809/1391347786","heartbeat_back_addr":"172.21.15.187:6818/1391347786","heartbeat_front_addr":"172.21.15.187:6813/1391347786","state":["exists","up"]},{"osd":7,"uuid":"775dae05-54f5-4ad4-bbce-92863c6ed2ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6825","nonce":1220065622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6827","nonce":1220065622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6831","nonce":1220065622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6829","nonce":1220065622}]},"public_addr":"172.21.15.187:6825/1220065622","cluster_addr":"172.21.15.187:6827/1220065622","heartbeat_back_addr":"172.21.15.187:6831/1220065622","heartbeat_front_addr":"172.21.15.187:6829/1220065622","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:05.983520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.299516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.067323+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.307748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.018742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.258775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.010912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.295355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.a","mappings":[{"from":6,"to":0}]},{"pgid":"11.22","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-26T22:53:37.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.370+0000 7fb035a00640 1 -- 172.21.15.131:0/2342220939 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb01006be30 msgr2=0x7fb01006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:37.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.370+0000 7fb035a00640 1 --2- 172.21.15.131:0/2342220939 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb01006be30 0x7fb01006e2f0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fb030162c00 tx=0x7fb01c009040 comp rx=0 tx=0).stop 2024-05-26T22:53:37.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.370+0000 7fb035a00640 1 -- 172.21.15.131:0/2342220939 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb0300b2f60 msgr2=0x7fb0300ca7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:37.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.370+0000 7fb035a00640 1 --2- 172.21.15.131:0/2342220939 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb0300b2f60 0x7fb0300ca7e0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7fb02000d800 tx=0x7fb02000dcc0 comp rx=0 tx=0).stop 2024-05-26T22:53:37.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.370+0000 7fb035a00640 1 -- 172.21.15.131:0/2342220939 shutdown_connections 2024-05-26T22:53:37.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.370+0000 7fb035a00640 1 --2- 172.21.15.131:0/2342220939 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb03015e570 0x7fb03015e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:37.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.370+0000 7fb035a00640 1 --2- 172.21.15.131:0/2342220939 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb01006be30 0x7fb01006e2f0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:37.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.370+0000 7fb035a00640 1 --2- 172.21.15.131:0/2342220939 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb0300d19b0 0x7fb03014c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:37.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.370+0000 7fb035a00640 1 --2- 172.21.15.131:0/2342220939 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb0300b2f60 0x7fb0300ca7e0 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:37.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.370+0000 7fb035a00640 1 -- 172.21.15.131:0/2342220939 >> 172.21.15.131:0/2342220939 conn(0x7fb030072e90 msgr2=0x7fb030073270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:37.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.370+0000 7fb035a00640 1 -- 172.21.15.131:0/2342220939 shutdown_connections 2024-05-26T22:53:37.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.370+0000 7fb035a00640 1 -- 172.21.15.131:0/2342220939 wait complete. 2024-05-26T22:53:37.389 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-26T22:53:37.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.585+0000 7f3677e00640 1 Processor -- start 2024-05-26T22:53:37.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.585+0000 7f3677e00640 1 -- start start 2024-05-26T22:53:37.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.587+0000 7f3677e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f36780b9f20 0x7f36780d22b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:37.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.587+0000 7f3677e00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f36780d27f0 0x7f36780b8d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:37.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.587+0000 7f3677e00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36780b79f0 con 0x7f367815e340 2024-05-26T22:53:37.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.587+0000 7f3677e00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f36780b88b0 con 0x7f36780b9f20 2024-05-26T22:53:37.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.587+0000 7f3677e00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f367815e720 con 0x7f36780d27f0 2024-05-26T22:53:37.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.587+0000 7f3676000640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f36780b9f20 0x7f36780d22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:37.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.587+0000 7f3676000640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f36780b9f20 0x7f36780d22b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51008/0 (socket says 172.21.15.131:51008) 2024-05-26T22:53:37.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.587+0000 7f3676000640 1 -- 172.21.15.131:0/519612608 learned_addr learned my addr 172.21.15.131:0/519612608 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:53:37.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.587+0000 7f3676000640 1 -- 172.21.15.131:0/519612608 >> v2:172.21.15.187:3300/0 conn(0x7f36780d27f0 msgr2=0x7f36780b8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:37.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.587+0000 7f3676000640 1 --2- 172.21.15.131:0/519612608 >> v2:172.21.15.187:3300/0 conn(0x7f36780d27f0 0x7f36780b8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:37.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.587+0000 7f3676000640 1 -- 172.21.15.131:0/519612608 >> v1:172.21.15.187:6789/0 conn(0x7f367815e340 legacy=0x7f36780b72e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:37.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.587+0000 7f3676000640 1 -- 172.21.15.131:0/519612608 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36780b7ca0 con 0x7f36780b9f20 2024-05-26T22:53:37.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.589+0000 7f3676000640 1 --2- 172.21.15.131:0/519612608 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f36780b9f20 0x7f36780d22b0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f36780b7230 tx=0x7f366402a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e4fcf9c9a51c851 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:37.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.589+0000 7f3675600640 1 -- 172.21.15.131:0/519612608 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3664037070 con 0x7f36780b9f20 2024-05-26T22:53:37.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.589+0000 7f3675600640 1 -- 172.21.15.131:0/519612608 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3664032440 con 0x7f36780b9f20 2024-05-26T22:53:37.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.589+0000 7f3675600640 1 -- 172.21.15.131:0/519612608 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3664032810 con 0x7f36780b9f20 2024-05-26T22:53:37.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.589+0000 7f3677e00640 1 -- 172.21.15.131:0/519612608 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f36780b9f20 msgr2=0x7f36780d22b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:37.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.589+0000 7f3677e00640 1 --2- 172.21.15.131:0/519612608 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f36780b9f20 0x7f36780d22b0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f36780b7230 tx=0x7f366402a2c0 comp rx=0 tx=0).stop 2024-05-26T22:53:37.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.590+0000 7f3677e00640 1 -- 172.21.15.131:0/519612608 shutdown_connections 2024-05-26T22:53:37.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.590+0000 7f3677e00640 1 --2- 172.21.15.131:0/519612608 >> v2:172.21.15.187:3300/0 conn(0x7f36780d27f0 0x7f36780b8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:37.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.590+0000 7f3677e00640 1 --2- 172.21.15.131:0/519612608 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f36780b9f20 0x7f36780d22b0 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:37.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.590+0000 7f3677e00640 1 -- 172.21.15.131:0/519612608 >> 172.21.15.131:0/519612608 conn(0x7f3678072e60 msgr2=0x7f3678073270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:37.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.590+0000 7f3677e00640 1 -- 172.21.15.131:0/519612608 shutdown_connections 2024-05-26T22:53:37.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.590+0000 7f3677e00640 1 -- 172.21.15.131:0/519612608 wait complete. 2024-05-26T22:53:37.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.591+0000 7f3677e00640 1 Processor -- start 2024-05-26T22:53:37.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.591+0000 7f3677e00640 1 -- start start 2024-05-26T22:53:37.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.592+0000 7f3677e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f36780b9f20 0x7f3678154590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:37.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.592+0000 7f3677e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f36780d27f0 0x7f36780ca470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:37.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.592+0000 7f3677e00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f367815e340 0x7f36781503a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:37.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.592+0000 7f3677e00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f36780b8540 con 0x7f36780d27f0 2024-05-26T22:53:37.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.592+0000 7f3677e00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f36780b8260 con 0x7f367815e340 2024-05-26T22:53:37.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.592+0000 7f3677e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f36780b9280 con 0x7f36780b9f20 2024-05-26T22:53:37.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.593+0000 7f3676000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f36780d27f0 0x7f36780ca470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:37.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.593+0000 7f3676000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f36780d27f0 0x7f36780ca470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51024/0 (socket says 172.21.15.131:51024) 2024-05-26T22:53:37.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.593+0000 7f3676000640 1 -- 172.21.15.131:0/2709205923 learned_addr learned my addr 172.21.15.131:0/2709205923 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:53:37.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.593+0000 7f3677400640 1 --2- 172.21.15.131:0/2709205923 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f367815e340 0x7f36781503a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:37.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.593+0000 7f3676000640 1 -- 172.21.15.131:0/2709205923 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f36780b9f20 msgr2=0x7f3678154590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:37.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.593+0000 7f3676a00640 1 --2- 172.21.15.131:0/2709205923 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f36780b9f20 0x7f3678154590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:37.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.593+0000 7f3676000640 1 --2- 172.21.15.131:0/2709205923 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f36780b9f20 0x7f3678154590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:37.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.593+0000 7f3676000640 1 -- 172.21.15.131:0/2709205923 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f367815e340 msgr2=0x7f36781503a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:37.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.593+0000 7f3676000640 1 --2- 172.21.15.131:0/2709205923 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f367815e340 0x7f36781503a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:37.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.593+0000 7f3676000640 1 -- 172.21.15.131:0/2709205923 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f367815d710 con 0x7f36780d27f0 2024-05-26T22:53:37.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.593+0000 7f3676a00640 1 --2- 172.21.15.131:0/2709205923 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f36780b9f20 0x7f3678154590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:53:37.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.593+0000 7f3677400640 1 --2- 172.21.15.131:0/2709205923 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f367815e340 0x7f36781503a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:53:37.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.594+0000 7f3676000640 1 --2- 172.21.15.131:0/2709205923 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f36780d27f0 0x7f36780ca470 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f3664000c00 tx=0x7f36640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:37.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.594+0000 7f366ea00640 1 -- 172.21.15.131:0/2709205923 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3664041070 con 0x7f36780d27f0 2024-05-26T22:53:37.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.594+0000 7f366ea00640 1 -- 172.21.15.131:0/2709205923 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f366403c870 con 0x7f36780d27f0 2024-05-26T22:53:37.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.594+0000 7f366ea00640 1 -- 172.21.15.131:0/2709205923 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3664037040 con 0x7f36780d27f0 2024-05-26T22:53:37.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.594+0000 7f3677e00640 1 -- 172.21.15.131:0/2709205923 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3678154b70 con 0x7f36780d27f0 2024-05-26T22:53:37.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.594+0000 7f3677e00640 1 -- 172.21.15.131:0/2709205923 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3678155130 con 0x7f36780d27f0 2024-05-26T22:53:37.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.595+0000 7f3677e00640 1 -- 172.21.15.131:0/2709205923 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36780694e0 con 0x7f36780d27f0 2024-05-26T22:53:37.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.596+0000 7f366ea00640 1 -- 172.21.15.131:0/2709205923 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3664044070 con 0x7f36780d27f0 2024-05-26T22:53:37.597 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.597+0000 7f366ea00640 1 --2- 172.21.15.131:0/2709205923 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f365806be30 0x7f365806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:37.597 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.597+0000 7f366ea00640 1 -- 172.21.15.131:0/2709205923 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5767+0+0 (secure 0 0 0) 0x7f36640abb30 con 0x7f36780d27f0 2024-05-26T22:53:37.597 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.597+0000 7f3676a00640 1 --2- 172.21.15.131:0/2709205923 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f365806be30 0x7f365806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:37.597 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.598+0000 7f3676a00640 1 --2- 172.21.15.131:0/2709205923 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f365806be30 0x7f365806e2f0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f366800a810 tx=0x7f3668008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:37.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.601+0000 7f366ea00640 1 -- 172.21.15.131:0/2709205923 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f366407a5b0 con 0x7f36780d27f0 2024-05-26T22:53:37.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.837+0000 7f3677e00640 1 -- 172.21.15.131:0/2709205923 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f3678160ae0 con 0x7f36780d27f0 2024-05-26T22:53:37.838 INFO:tasks.ceph.mon.a.smithi131.stderr:2024-05-26T22:53:37.838+0000 7fcf0d200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-26T22:53:37.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.842+0000 7f366ea00640 1 -- 172.21.15.131:0/2709205923 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v70) ==== 161+0+0 (secure 0 0 0) 0x7f3664079eb0 con 0x7f36780d27f0 2024-05-26T22:53:37.842 INFO:teuthology.orchestra.run.smithi131.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-26T22:53:37.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.845+0000 7f3677e00640 1 -- 172.21.15.131:0/2709205923 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f365806be30 msgr2=0x7f365806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:37.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.845+0000 7f3677e00640 1 --2- 172.21.15.131:0/2709205923 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f365806be30 0x7f365806e2f0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f366800a810 tx=0x7f3668008040 comp rx=0 tx=0).stop 2024-05-26T22:53:37.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.845+0000 7f3677e00640 1 -- 172.21.15.131:0/2709205923 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f36780d27f0 msgr2=0x7f36780ca470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:37.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.845+0000 7f3677e00640 1 --2- 172.21.15.131:0/2709205923 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f36780d27f0 0x7f36780ca470 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f3664000c00 tx=0x7f36640046c0 comp rx=0 tx=0).stop 2024-05-26T22:53:37.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.845+0000 7f3677e00640 1 -- 172.21.15.131:0/2709205923 shutdown_connections 2024-05-26T22:53:37.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.845+0000 7f3677e00640 1 --2- 172.21.15.131:0/2709205923 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f365806be30 0x7f365806e2f0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:37.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.845+0000 7f3677e00640 1 --2- 172.21.15.131:0/2709205923 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f367815e340 0x7f36781503a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:37.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.845+0000 7f3677e00640 1 --2- 172.21.15.131:0/2709205923 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f36780d27f0 0x7f36780ca470 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:37.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.845+0000 7f3677e00640 1 --2- 172.21.15.131:0/2709205923 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f36780b9f20 0x7f3678154590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:37.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.845+0000 7f3677e00640 1 -- 172.21.15.131:0/2709205923 >> 172.21.15.131:0/2709205923 conn(0x7f3678072e60 msgr2=0x7f36781612e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:37.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.846+0000 7f3677e00640 1 -- 172.21.15.131:0/2709205923 shutdown_connections 2024-05-26T22:53:37.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:37.847+0000 7f3677e00640 1 -- 172.21.15.131:0/2709205923 wait complete. 2024-05-26T22:53:37.871 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-26T22:53:37.943 INFO:tasks.ceph.mds.c.smithi131.stdout:starting mds.c at 2024-05-26T22:53:37.948 INFO:tasks.ceph.mds.a.smithi131.stdout:starting mds.a at 2024-05-26T22:53:37.962 INFO:tasks.ceph.mds.d.smithi187.stdout:starting mds.d at 2024-05-26T22:53:38.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:38.044+0000 7f4e75200640 1 Processor -- start 2024-05-26T22:53:38.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:38.044+0000 7f4e75200640 1 -- start start 2024-05-26T22:53:38.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:38.046+0000 7f4e75200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f4e700cc2d0 0x7f4e7006d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:38.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:38.046+0000 7f4e75200640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f4e7006d860 0x7f4e700cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:38.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:38.046+0000 7f4e75200640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e700cc780 con 0x7f4e700cbef0 2024-05-26T22:53:38.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:38.046+0000 7f4e75200640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f4e700ad600 con 0x7f4e700cc2d0 2024-05-26T22:53:38.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:38.046+0000 7f4e75200640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f4e7006dcd0 con 0x7f4e7006d860 2024-05-26T22:53:38.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:38.046+0000 7f4e65400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f4e700cc2d0 0x7f4e7006d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:38.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:38.046+0000 7f4e65400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f4e700cc2d0 0x7f4e7006d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51058/0 (socket says 172.21.15.131:51058) 2024-05-26T22:53:38.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:38.046+0000 7f4e65400640 1 -- 172.21.15.131:0/3235894516 learned_addr learned my addr 172.21.15.131:0/3235894516 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:53:38.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:38.046+0000 7f4e6e800640 1 --2- 172.21.15.131:0/3235894516 >> v2:172.21.15.187:3300/0 conn(0x7f4e7006d860 0x7f4e700cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:38.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:38.046+0000 7f4e65400640 1 -- 172.21.15.131:0/3235894516 >> v2:172.21.15.187:3300/0 conn(0x7f4e7006d860 msgr2=0x7f4e700cea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:38.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:38.046+0000 7f4e65400640 1 --2- 172.21.15.131:0/3235894516 >> v2:172.21.15.187:3300/0 conn(0x7f4e7006d860 0x7f4e700cea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:38.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:38.046+0000 7f4e65400640 1 -- 172.21.15.131:0/3235894516 >> v1:172.21.15.187:6789/0 conn(0x7f4e700cbef0 legacy=0x7f4e700acef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:38.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:38.046+0000 7f4e65400640 1 -- 172.21.15.131:0/3235894516 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e700cdca0 con 0x7f4e700cc2d0 2024-05-26T22:53:38.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:38.047+0000 7f4e65400640 1 --2- 172.21.15.131:0/3235894516 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4e700cc2d0 0x7f4e7006d320 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f4e700cf880 tx=0x7f4e5802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f6a646e0626fbea server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:38.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:38.048+0000 7f4e6d400640 1 -- 172.21.15.131:0/3235894516 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e58037070 con 0x7f4e700cc2d0 2024-05-26T22:53:38.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:38.048+0000 7f4e6d400640 1 -- 172.21.15.131:0/3235894516 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e58032440 con 0x7f4e700cc2d0 2024-05-26T22:53:38.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:38.048+0000 7f4e6d400640 1 -- 172.21.15.131:0/3235894516 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e58032810 con 0x7f4e700cc2d0 2024-05-26T22:53:38.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:38.048+0000 7f4e75200640 1 -- 172.21.15.131:0/3235894516 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4e700cc2d0 msgr2=0x7f4e7006d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:38.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:38.048+0000 7f4e75200640 1 --2- 172.21.15.131:0/3235894516 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4e700cc2d0 0x7f4e7006d320 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f4e700cf880 tx=0x7f4e5802a230 comp rx=0 tx=0).stop 2024-05-26T22:53:38.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:38.048+0000 7f4e75200640 1 -- 172.21.15.131:0/3235894516 shutdown_connections 2024-05-26T22:53:38.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:38.048+0000 7f4e75200640 1 --2- 172.21.15.131:0/3235894516 >> v2:172.21.15.187:3300/0 conn(0x7f4e7006d860 0x7f4e700cea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:38.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:38.048+0000 7f4e75200640 1 --2- 172.21.15.131:0/3235894516 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4e700cc2d0 0x7f4e7006d320 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:38.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:38.048+0000 7f4e75200640 1 -- 172.21.15.131:0/3235894516 >> 172.21.15.131:0/3235894516 conn(0x7f4e7006bc90 msgr2=0x7f4e700b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:38.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:38.048+0000 7f4e75200640 1 -- 172.21.15.131:0/3235894516 shutdown_connections 2024-05-26T22:53:38.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:38.049+0000 7f4e75200640 1 -- 172.21.15.131:0/3235894516 wait complete. 2024-05-26T22:53:38.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:38.050+0000 7f4e75200640 1 Processor -- start 2024-05-26T22:53:38.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:38.050+0000 7f4e75200640 1 -- start start 2024-05-26T22:53:38.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:38.051+0000 7f4e75200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4e7006d860 0x7f4e70148f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:38.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:38.051+0000 7f4e75200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4e700cbef0 0x7f4e700b0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:38.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:38.051+0000 7f4e75200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4e700cc2d0 0x7f4e701507b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:38.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:38.051+0000 7f4e75200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f4e700ce540 con 0x7f4e700cbef0 2024-05-26T22:53:38.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:38.051+0000 7f4e75200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4e700ce260 con 0x7f4e7006d860 2024-05-26T22:53:38.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:38.051+0000 7f4e75200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f4e700cefa0 con 0x7f4e700cc2d0 2024-05-26T22:53:38.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:38.051+0000 7f4e65400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4e700cbef0 0x7f4e700b0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:38.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:38.051+0000 7f4e65400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4e700cbef0 0x7f4e700b0930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51066/0 (socket says 172.21.15.131:51066) 2024-05-26T22:53:38.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:38.051+0000 7f4e65400640 1 -- 172.21.15.131:0/181473403 learned_addr learned my addr 172.21.15.131:0/181473403 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:53:38.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:38.052+0000 7f4e65400640 1 -- 172.21.15.131:0/181473403 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4e700cc2d0 msgr2=0x7f4e701507b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:38.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:38.052+0000 7f4e65400640 1 --2- 172.21.15.131:0/181473403 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4e700cc2d0 0x7f4e701507b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:38.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:38.052+0000 7f4e65400640 1 -- 172.21.15.131:0/181473403 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4e7006d860 msgr2=0x7f4e70148f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:38.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:38.052+0000 7f4e65400640 1 --2- 172.21.15.131:0/181473403 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4e7006d860 0x7f4e70148f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:38.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:38.052+0000 7f4e65400640 1 -- 172.21.15.131:0/181473403 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e70149460 con 0x7f4e700cbef0 2024-05-26T22:53:38.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:38.052+0000 7f4e65400640 1 --2- 172.21.15.131:0/181473403 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4e700cbef0 0x7f4e700b0930 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f4e5802a270 tx=0x7f4e580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:38.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:38.052+0000 7f4e66a00640 1 -- 172.21.15.131:0/181473403 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e58041020 con 0x7f4e700cbef0 2024-05-26T22:53:38.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:38.052+0000 7f4e66a00640 1 -- 172.21.15.131:0/181473403 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e58030990 con 0x7f4e700cbef0 2024-05-26T22:53:38.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:38.052+0000 7f4e75200640 1 -- 172.21.15.131:0/181473403 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e70168ac0 con 0x7f4e700cbef0 2024-05-26T22:53:38.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:38.052+0000 7f4e66a00640 1 -- 172.21.15.131:0/181473403 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e58037040 con 0x7f4e700cbef0 2024-05-26T22:53:38.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:38.053+0000 7f4e75200640 1 -- 172.21.15.131:0/181473403 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e70169080 con 0x7f4e700cbef0 2024-05-26T22:53:38.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:38.054+0000 7f4e75200640 1 -- 172.21.15.131:0/181473403 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e70074e90 con 0x7f4e700cbef0 2024-05-26T22:53:38.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:38.055+0000 7f4e66a00640 1 -- 172.21.15.131:0/181473403 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4e58044020 con 0x7f4e700cbef0 2024-05-26T22:53:38.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:38.056+0000 7f4e66a00640 1 --2- 172.21.15.131:0/181473403 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f4e4c06be30 0x7f4e4c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:38.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:38.056+0000 7f4e66a00640 1 -- 172.21.15.131:0/181473403 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 6025+0+0 (secure 0 0 0) 0x7f4e580aba60 con 0x7f4e700cbef0 2024-05-26T22:53:38.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:38.056+0000 7f4e6de00640 1 --2- 172.21.15.131:0/181473403 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f4e4c06be30 0x7f4e4c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:38.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:38.057+0000 7f4e6de00640 1 --2- 172.21.15.131:0/181473403 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f4e4c06be30 0x7f4e4c06e2f0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f4e70073700 tx=0x7f4e60007990 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:38.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:38.060+0000 7f4e66a00640 1 -- 172.21.15.131:0/181473403 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f4e5804b030 con 0x7f4e700cbef0 2024-05-26T22:53:38.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:38.297+0000 7f4e75200640 1 -- 172.21.15.131:0/181473403 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f4e7006fcd0 con 0x7f4e700cbef0 2024-05-26T22:53:38.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:38.846+0000 7f4e66a00640 1 -- 172.21.15.131:0/181473403 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v71) ==== 104+0+0 (secure 0 0 0) 0x7f4e70149460 con 0x7f4e700cbef0 2024-05-26T22:53:38.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:38.850+0000 7f4e75200640 1 -- 172.21.15.131:0/181473403 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f4e4c06be30 msgr2=0x7f4e4c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:38.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:38.850+0000 7f4e75200640 1 --2- 172.21.15.131:0/181473403 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f4e4c06be30 0x7f4e4c06e2f0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f4e70073700 tx=0x7f4e60007990 comp rx=0 tx=0).stop 2024-05-26T22:53:38.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:38.850+0000 7f4e75200640 1 -- 172.21.15.131:0/181473403 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4e700cbef0 msgr2=0x7f4e700b0930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:38.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:38.850+0000 7f4e75200640 1 --2- 172.21.15.131:0/181473403 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4e700cbef0 0x7f4e700b0930 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f4e5802a270 tx=0x7f4e580046c0 comp rx=0 tx=0).stop 2024-05-26T22:53:38.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:38.850+0000 7f4e75200640 1 -- 172.21.15.131:0/181473403 shutdown_connections 2024-05-26T22:53:38.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:38.850+0000 7f4e75200640 1 --2- 172.21.15.131:0/181473403 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4e700cc2d0 0x7f4e701507b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:38.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:38.850+0000 7f4e75200640 1 --2- 172.21.15.131:0/181473403 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f4e4c06be30 0x7f4e4c06e2f0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:38.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:38.850+0000 7f4e75200640 1 --2- 172.21.15.131:0/181473403 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4e700cbef0 0x7f4e700b0930 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:38.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:38.850+0000 7f4e75200640 1 --2- 172.21.15.131:0/181473403 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4e7006d860 0x7f4e70148f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:38.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:38.850+0000 7f4e75200640 1 -- 172.21.15.131:0/181473403 >> 172.21.15.131:0/181473403 conn(0x7f4e7006bc90 msgr2=0x7f4e700ae3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:38.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:38.850+0000 7f4e75200640 1 -- 172.21.15.131:0/181473403 shutdown_connections 2024-05-26T22:53:38.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:38.850+0000 7f4e75200640 1 -- 172.21.15.131:0/181473403 wait complete. 2024-05-26T22:53:38.875 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-26T22:53:39.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:39.063+0000 7f6d06600640 1 Processor -- start 2024-05-26T22:53:39.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:39.064+0000 7f6d06600640 1 -- start start 2024-05-26T22:53:39.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:39.065+0000 7f6d06600640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6d000cf8f0 0x7f6d00072250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:39.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:39.065+0000 7f6d06600640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f6d000d0360 0x7f6d000d07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:39.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:39.065+0000 7f6d06600640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d000ccb70 con 0x7f6d0006e860 2024-05-26T22:53:39.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:39.065+0000 7f6d06600640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f6d000d0d00 con 0x7f6d000d0360 2024-05-26T22:53:39.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:39.065+0000 7f6d06600640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f6d000cce50 con 0x7f6d000cf8f0 2024-05-26T22:53:39.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:39.066+0000 7f6cfea00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f6d000d0360 0x7f6d000d07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:39.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:39.066+0000 7f6cfea00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f6d000d0360 0x7f6d000d07c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51082/0 (socket says 172.21.15.131:51082) 2024-05-26T22:53:39.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:39.066+0000 7f6cfea00640 1 -- 172.21.15.131:0/1249548358 learned_addr learned my addr 172.21.15.131:0/1249548358 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:53:39.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:39.066+0000 7f6cff400640 1 --2- 172.21.15.131:0/1249548358 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6d000cf8f0 0x7f6d00072250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:39.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:39.066+0000 7f6cfea00640 1 -- 172.21.15.131:0/1249548358 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6d000cf8f0 msgr2=0x7f6d00072250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:39.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:39.066+0000 7f6cfea00640 1 --2- 172.21.15.131:0/1249548358 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6d000cf8f0 0x7f6d00072250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:39.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:39.066+0000 7f6cfea00640 1 -- 172.21.15.131:0/1249548358 >> v1:172.21.15.187:6789/0 conn(0x7f6d0006e860 legacy=0x7f6d0006ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:39.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:39.066+0000 7f6cfea00640 1 -- 172.21.15.131:0/1249548358 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d000cd130 con 0x7f6d000d0360 2024-05-26T22:53:39.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:39.067+0000 7f6cfea00640 1 --2- 172.21.15.131:0/1249548358 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6d000d0360 0x7f6d000d07c0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f6ce8009870 tx=0x7f6ce802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8c7fb0e5f4a5c22 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:39.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:39.068+0000 7f6cfe000640 1 -- 172.21.15.131:0/1249548358 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ce8037070 con 0x7f6d000d0360 2024-05-26T22:53:39.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:39.068+0000 7f6cfe000640 1 -- 172.21.15.131:0/1249548358 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ce8032440 con 0x7f6d000d0360 2024-05-26T22:53:39.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:39.068+0000 7f6cfe000640 1 -- 172.21.15.131:0/1249548358 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ce8032810 con 0x7f6d000d0360 2024-05-26T22:53:39.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:39.068+0000 7f6d06600640 1 -- 172.21.15.131:0/1249548358 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6d000d0360 msgr2=0x7f6d000d07c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:39.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:39.068+0000 7f6d06600640 1 --2- 172.21.15.131:0/1249548358 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6d000d0360 0x7f6d000d07c0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f6ce8009870 tx=0x7f6ce802a210 comp rx=0 tx=0).stop 2024-05-26T22:53:39.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:39.068+0000 7f6d06600640 1 -- 172.21.15.131:0/1249548358 shutdown_connections 2024-05-26T22:53:39.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:39.068+0000 7f6d06600640 1 --2- 172.21.15.131:0/1249548358 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6d000d0360 0x7f6d000d07c0 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:39.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:39.068+0000 7f6d06600640 1 --2- 172.21.15.131:0/1249548358 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6d000cf8f0 0x7f6d00072250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:39.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:39.068+0000 7f6d06600640 1 -- 172.21.15.131:0/1249548358 >> 172.21.15.131:0/1249548358 conn(0x7f6d000b2110 msgr2=0x7f6d000b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:39.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:39.068+0000 7f6d06600640 1 -- 172.21.15.131:0/1249548358 shutdown_connections 2024-05-26T22:53:39.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:39.068+0000 7f6d06600640 1 -- 172.21.15.131:0/1249548358 wait complete. 2024-05-26T22:53:39.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:39.070+0000 7f6d06600640 1 Processor -- start 2024-05-26T22:53:39.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:39.070+0000 7f6d06600640 1 -- start start 2024-05-26T22:53:39.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:39.071+0000 7f6d06600640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6d0006e860 0x7f6d0009dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:39.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:39.071+0000 7f6d06600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6d000cf8f0 0x7f6d001503c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:39.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:39.071+0000 7f6d06600640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6d000d0360 0x7f6d00152de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:39.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:39.071+0000 7f6d06600640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f6d0015ffd0 con 0x7f6d000cf8f0 2024-05-26T22:53:39.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:39.071+0000 7f6d06600640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6d000cfcd0 con 0x7f6d0006e860 2024-05-26T22:53:39.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:39.071+0000 7f6d06600640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f6d00160150 con 0x7f6d000d0360 2024-05-26T22:53:39.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:39.071+0000 7f6cfea00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6d000cf8f0 0x7f6d001503c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:39.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:39.071+0000 7f6cfea00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6d000cf8f0 0x7f6d001503c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51084/0 (socket says 172.21.15.131:51084) 2024-05-26T22:53:39.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:39.071+0000 7f6cfea00640 1 -- 172.21.15.131:0/811853532 learned_addr learned my addr 172.21.15.131:0/811853532 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:53:39.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:39.072+0000 7f6cfea00640 1 -- 172.21.15.131:0/811853532 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6d000d0360 msgr2=0x7f6d00152de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:39.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:39.072+0000 7f6cfea00640 1 --2- 172.21.15.131:0/811853532 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6d000d0360 0x7f6d00152de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:39.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:39.072+0000 7f6cfea00640 1 -- 172.21.15.131:0/811853532 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6d0006e860 msgr2=0x7f6d0009dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:39.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:39.072+0000 7f6cfea00640 1 --2- 172.21.15.131:0/811853532 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6d0006e860 0x7f6d0009dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:39.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:39.072+0000 7f6cfea00640 1 -- 172.21.15.131:0/811853532 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d0009d2f0 con 0x7f6d000cf8f0 2024-05-26T22:53:39.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:39.072+0000 7f6cfea00640 1 --2- 172.21.15.131:0/811853532 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6d000cf8f0 0x7f6d001503c0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f6ce8009870 tx=0x7f6ce80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:39.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:39.072+0000 7f6cf7400640 1 -- 172.21.15.131:0/811853532 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ce8041020 con 0x7f6d000cf8f0 2024-05-26T22:53:39.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:39.072+0000 7f6cf7400640 1 -- 172.21.15.131:0/811853532 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ce8030970 con 0x7f6d000cf8f0 2024-05-26T22:53:39.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:39.072+0000 7f6cf7400640 1 -- 172.21.15.131:0/811853532 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ce8037040 con 0x7f6d000cf8f0 2024-05-26T22:53:39.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:39.072+0000 7f6d06600640 1 -- 172.21.15.131:0/811853532 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d00153720 con 0x7f6d000cf8f0 2024-05-26T22:53:39.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:39.073+0000 7f6d06600640 1 -- 172.21.15.131:0/811853532 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d0009d560 con 0x7f6d000cf8f0 2024-05-26T22:53:39.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:39.074+0000 7f6d06600640 1 -- 172.21.15.131:0/811853532 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d00068440 con 0x7f6d000cf8f0 2024-05-26T22:53:39.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:39.075+0000 7f6cf7400640 1 -- 172.21.15.131:0/811853532 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6ce8044020 con 0x7f6d000cf8f0 2024-05-26T22:53:39.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:39.075+0000 7f6cf7400640 1 --2- 172.21.15.131:0/811853532 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f6cdc06be60 0x7f6cdc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:39.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:39.076+0000 7f6cf7400640 1 -- 172.21.15.131:0/811853532 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 6025+0+0 (secure 0 0 0) 0x7f6ce80ab9d0 con 0x7f6d000cf8f0 2024-05-26T22:53:39.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:39.076+0000 7f6cff400640 1 --2- 172.21.15.131:0/811853532 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f6cdc06be60 0x7f6cdc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:39.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:39.077+0000 7f6cff400640 1 --2- 172.21.15.131:0/811853532 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f6cdc06be60 0x7f6cdc06e320 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f6d000ca290 tx=0x7f6cf0007390 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:39.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:39.081+0000 7f6cf7400640 1 -- 172.21.15.131:0/811853532 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f6ce804b030 con 0x7f6d000cf8f0 2024-05-26T22:53:39.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:39.264+0000 7f6d06600640 1 -- 172.21.15.131:0/811853532 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f6d000ce970 con 0x7f6d000cf8f0 2024-05-26T22:53:39.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:39.849+0000 7f6cf7400640 1 -- 172.21.15.131:0/811853532 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v73) ==== 181+0+0 (secure 0 0 0) 0x7f6ce807a230 con 0x7f6d000cf8f0 2024-05-26T22:53:39.850 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_metadata' removed 2024-05-26T22:53:39.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:39.853+0000 7f6d06600640 1 -- 172.21.15.131:0/811853532 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f6cdc06be60 msgr2=0x7f6cdc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:39.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:39.853+0000 7f6d06600640 1 --2- 172.21.15.131:0/811853532 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f6cdc06be60 0x7f6cdc06e320 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f6d000ca290 tx=0x7f6cf0007390 comp rx=0 tx=0).stop 2024-05-26T22:53:39.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:39.853+0000 7f6d06600640 1 -- 172.21.15.131:0/811853532 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6d000cf8f0 msgr2=0x7f6d001503c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:39.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:39.853+0000 7f6d06600640 1 --2- 172.21.15.131:0/811853532 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6d000cf8f0 0x7f6d001503c0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f6ce8009870 tx=0x7f6ce80046c0 comp rx=0 tx=0).stop 2024-05-26T22:53:39.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:39.853+0000 7f6d06600640 1 -- 172.21.15.131:0/811853532 shutdown_connections 2024-05-26T22:53:39.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:39.853+0000 7f6d06600640 1 --2- 172.21.15.131:0/811853532 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6d000d0360 0x7f6d00152de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:39.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:39.853+0000 7f6d06600640 1 --2- 172.21.15.131:0/811853532 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f6cdc06be60 0x7f6cdc06e320 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:39.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:39.853+0000 7f6d06600640 1 --2- 172.21.15.131:0/811853532 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6d000cf8f0 0x7f6d001503c0 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:39.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:39.853+0000 7f6d06600640 1 --2- 172.21.15.131:0/811853532 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6d0006e860 0x7f6d0009dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:39.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:39.853+0000 7f6d06600640 1 -- 172.21.15.131:0/811853532 >> 172.21.15.131:0/811853532 conn(0x7f6d000b2110 msgr2=0x7f6d0006e0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:39.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:39.854+0000 7f6d06600640 1 -- 172.21.15.131:0/811853532 shutdown_connections 2024-05-26T22:53:39.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:39.854+0000 7f6d06600640 1 -- 172.21.15.131:0/811853532 wait complete. 2024-05-26T22:53:39.878 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-26T22:53:40.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:40.068+0000 7fa867c00640 1 Processor -- start 2024-05-26T22:53:40.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:40.068+0000 7fa867c00640 1 -- start start 2024-05-26T22:53:40.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:40.070+0000 7fa867c00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa86006dd40 0x7fa86015ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:40.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:40.070+0000 7fa867c00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fa8600d0a70 0x7fa8600b4d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:40.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:40.070+0000 7fa867c00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa86015bb40 con 0x7fa8600b48b0 2024-05-26T22:53:40.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:40.070+0000 7fa867c00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fa86015be20 con 0x7fa8600d0a70 2024-05-26T22:53:40.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:40.070+0000 7fa867c00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa86015c100 con 0x7fa86006dd40 2024-05-26T22:53:40.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:40.070+0000 7fa865c00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fa8600d0a70 0x7fa8600b4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:40.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:40.070+0000 7fa857e00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fa8600b48b0 0x7fa8600d0360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:47706/0 (socket says 172.21.15.131:47706) 2024-05-26T22:53:40.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:40.070+0000 7fa857e00640 1 -- 172.21.15.131:0/1863670267 learned_addr learned my addr 172.21.15.131:0/1863670267 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:53:40.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:40.070+0000 7fa865200640 1 --2- 172.21.15.131:0/1863670267 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa86006dd40 0x7fa86015ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:40.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:40.071+0000 7fa865c00640 1 -- 172.21.15.131:0/1863670267 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa86006dd40 msgr2=0x7fa86015ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:40.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:40.071+0000 7fa865c00640 1 --2- 172.21.15.131:0/1863670267 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa86006dd40 0x7fa86015ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:40.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:40.071+0000 7fa865c00640 1 -- 172.21.15.131:0/1863670267 >> v1:172.21.15.131:6789/0 conn(0x7fa8600b48b0 legacy=0x7fa8600d0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:40.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:40.071+0000 7fa865c00640 1 -- 172.21.15.131:0/1863670267 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa86015c3e0 con 0x7fa8600d0a70 2024-05-26T22:53:40.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:40.071+0000 7fa85fe00640 1 -- 172.21.15.131:0/1863670267 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 76451707 0 0) 0x7fa86015bb40 con 0x7fa8600b48b0 2024-05-26T22:53:40.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:40.072+0000 7fa865c00640 1 --2- 172.21.15.131:0/1863670267 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8600d0a70 0x7fa8600b4d10 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fa858009d90 tx=0x7fa85802a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=855e050da9c52f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:40.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:40.072+0000 7fa85fe00640 1 -- 172.21.15.131:0/1863670267 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa858031600 con 0x7fa8600d0a70 2024-05-26T22:53:40.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:40.072+0000 7fa85fe00640 1 -- 172.21.15.131:0/1863670267 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8580317a0 con 0x7fa8600d0a70 2024-05-26T22:53:40.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:40.072+0000 7fa85fe00640 1 -- 172.21.15.131:0/1863670267 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa858031b70 con 0x7fa8600d0a70 2024-05-26T22:53:40.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:40.073+0000 7fa867c00640 1 -- 172.21.15.131:0/1863670267 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8600d0a70 msgr2=0x7fa8600b4d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:40.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:40.073+0000 7fa867c00640 1 --2- 172.21.15.131:0/1863670267 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8600d0a70 0x7fa8600b4d10 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fa858009d90 tx=0x7fa85802a5d0 comp rx=0 tx=0).stop 2024-05-26T22:53:40.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:40.073+0000 7fa867c00640 1 -- 172.21.15.131:0/1863670267 shutdown_connections 2024-05-26T22:53:40.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:40.073+0000 7fa867c00640 1 --2- 172.21.15.131:0/1863670267 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8600d0a70 0x7fa8600b4d10 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:40.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:40.073+0000 7fa867c00640 1 --2- 172.21.15.131:0/1863670267 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa86006dd40 0x7fa86015ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:40.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:40.073+0000 7fa867c00640 1 -- 172.21.15.131:0/1863670267 >> 172.21.15.131:0/1863670267 conn(0x7fa8600b2110 msgr2=0x7fa8600b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:40.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:40.073+0000 7fa867c00640 1 -- 172.21.15.131:0/1863670267 shutdown_connections 2024-05-26T22:53:40.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:40.073+0000 7fa867c00640 1 -- 172.21.15.131:0/1863670267 wait complete. 2024-05-26T22:53:40.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:40.075+0000 7fa867c00640 1 Processor -- start 2024-05-26T22:53:40.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:40.075+0000 7fa867c00640 1 -- start start 2024-05-26T22:53:40.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:40.076+0000 7fa867c00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa86006dd40 0x7fa8600cba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:40.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:40.076+0000 7fa867c00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa8600b48b0 0x7fa8601f37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:40.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:40.076+0000 7fa867c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8600d0a70 0x7fa8600ae5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:40.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:40.076+0000 7fa867c00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa8600b37b0 con 0x7fa8600d0a70 2024-05-26T22:53:40.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:40.076+0000 7fa867c00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa86006e120 con 0x7fa8600b48b0 2024-05-26T22:53:40.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:40.076+0000 7fa867c00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa8600b3930 con 0x7fa86006dd40 2024-05-26T22:53:40.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:40.076+0000 7fa865c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8600d0a70 0x7fa8600ae5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:40.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:40.076+0000 7fa865c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8600d0a70 0x7fa8600ae5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51088/0 (socket says 172.21.15.131:51088) 2024-05-26T22:53:40.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:40.076+0000 7fa865c00640 1 -- 172.21.15.131:0/3006877348 learned_addr learned my addr 172.21.15.131:0/3006877348 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:53:40.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:40.076+0000 7fa857e00640 1 --2- 172.21.15.131:0/3006877348 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa8600b48b0 0x7fa8601f37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:40.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:40.076+0000 7fa865200640 1 --2- 172.21.15.131:0/3006877348 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa86006dd40 0x7fa8600cba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:40.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:40.077+0000 7fa865c00640 1 -- 172.21.15.131:0/3006877348 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa86006dd40 msgr2=0x7fa8600cba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:40.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:40.077+0000 7fa865c00640 1 --2- 172.21.15.131:0/3006877348 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa86006dd40 0x7fa8600cba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:40.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:40.077+0000 7fa865c00640 1 -- 172.21.15.131:0/3006877348 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa8600b48b0 msgr2=0x7fa8601f37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:40.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:40.077+0000 7fa865c00640 1 --2- 172.21.15.131:0/3006877348 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa8600b48b0 0x7fa8601f37c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:40.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:40.077+0000 7fa865c00640 1 -- 172.21.15.131:0/3006877348 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa86009dc30 con 0x7fa8600d0a70 2024-05-26T22:53:40.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:40.077+0000 7fa865c00640 1 --2- 172.21.15.131:0/3006877348 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8600d0a70 0x7fa8600ae5d0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fa858009eb0 tx=0x7fa858004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:40.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:40.077+0000 7fa85d600640 1 -- 172.21.15.131:0/3006877348 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa858040070 con 0x7fa8600d0a70 2024-05-26T22:53:40.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:40.077+0000 7fa85d600640 1 -- 172.21.15.131:0/3006877348 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa858030b60 con 0x7fa8600d0a70 2024-05-26T22:53:40.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:40.077+0000 7fa85d600640 1 -- 172.21.15.131:0/3006877348 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa858009450 con 0x7fa8600d0a70 2024-05-26T22:53:40.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:40.077+0000 7fa867c00640 1 -- 172.21.15.131:0/3006877348 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8600cac80 con 0x7fa8600d0a70 2024-05-26T22:53:40.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:40.078+0000 7fa867c00640 1 -- 172.21.15.131:0/3006877348 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8600cb240 con 0x7fa8600d0a70 2024-05-26T22:53:40.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:40.079+0000 7fa867c00640 1 -- 172.21.15.131:0/3006877348 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa860068440 con 0x7fa8600d0a70 2024-05-26T22:53:40.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:40.080+0000 7fa85d600640 1 -- 172.21.15.131:0/3006877348 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa858039040 con 0x7fa8600d0a70 2024-05-26T22:53:40.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:40.081+0000 7fa85d600640 1 --2- 172.21.15.131:0/3006877348 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa83c06be30 0x7fa83c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:40.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:40.081+0000 7fa85d600640 1 -- 172.21.15.131:0/3006877348 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5587+0+0 (secure 0 0 0) 0x7fa8580ac540 con 0x7fa8600d0a70 2024-05-26T22:53:40.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:40.081+0000 7fa865200640 1 --2- 172.21.15.131:0/3006877348 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa83c06be30 0x7fa83c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:40.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:40.082+0000 7fa865200640 1 --2- 172.21.15.131:0/3006877348 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa83c06be30 0x7fa83c06e2f0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fa850006fd0 tx=0x7fa850008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:40.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:40.086+0000 7fa85d600640 1 -- 172.21.15.131:0/3006877348 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fa85807af50 con 0x7fa8600d0a70 2024-05-26T22:53:40.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:40.296+0000 7fa867c00640 1 -- 172.21.15.131:0/3006877348 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fa8600cfc10 con 0x7fa8600d0a70 2024-05-26T22:53:41.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.299+0000 7fa85d600640 1 -- 172.21.15.131:0/3006877348 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v74) ==== 169+0+0 (secure 0 0 0) 0x7fa85807a850 con 0x7fa8600d0a70 2024-05-26T22:53:41.299 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_data' removed 2024-05-26T22:53:41.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.303+0000 7fa867c00640 1 -- 172.21.15.131:0/3006877348 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa83c06be30 msgr2=0x7fa83c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:41.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.303+0000 7fa867c00640 1 --2- 172.21.15.131:0/3006877348 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa83c06be30 0x7fa83c06e2f0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fa850006fd0 tx=0x7fa850008040 comp rx=0 tx=0).stop 2024-05-26T22:53:41.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.303+0000 7fa867c00640 1 -- 172.21.15.131:0/3006877348 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8600d0a70 msgr2=0x7fa8600ae5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:41.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.303+0000 7fa867c00640 1 --2- 172.21.15.131:0/3006877348 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8600d0a70 0x7fa8600ae5d0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fa858009eb0 tx=0x7fa858004a40 comp rx=0 tx=0).stop 2024-05-26T22:53:41.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.303+0000 7fa867c00640 1 -- 172.21.15.131:0/3006877348 shutdown_connections 2024-05-26T22:53:41.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.303+0000 7fa867c00640 1 --2- 172.21.15.131:0/3006877348 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa83c06be30 0x7fa83c06e2f0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:41.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.303+0000 7fa867c00640 1 --2- 172.21.15.131:0/3006877348 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8600d0a70 0x7fa8600ae5d0 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:41.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.303+0000 7fa867c00640 1 --2- 172.21.15.131:0/3006877348 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa8600b48b0 0x7fa8601f37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:41.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.303+0000 7fa867c00640 1 --2- 172.21.15.131:0/3006877348 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa86006dd40 0x7fa8600cba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:41.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.303+0000 7fa867c00640 1 -- 172.21.15.131:0/3006877348 >> 172.21.15.131:0/3006877348 conn(0x7fa8600b2110 msgr2=0x7fa8600b1140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:41.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.304+0000 7fa867c00640 1 -- 172.21.15.131:0/3006877348 shutdown_connections 2024-05-26T22:53:41.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.304+0000 7fa867c00640 1 -- 172.21.15.131:0/3006877348 wait complete. 2024-05-26T22:53:41.338 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_export_ephemeral_random_max 2024-05-26T22:53:41.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.517+0000 7f5e4b000640 1 Processor -- start 2024-05-26T22:53:41.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.517+0000 7f5e4b000640 1 -- start start 2024-05-26T22:53:41.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.519+0000 7f5e4b000640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f5e440b5450 0x7f5e440b43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:41.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.519+0000 7f5e4b000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5e4406f420 0x7f5e440cfd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:41.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.519+0000 7f5e4b000640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e440b4900 con 0x7f5e440cf590 2024-05-26T22:53:41.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.519+0000 7f5e4b000640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f5e44160d90 con 0x7f5e440b5450 2024-05-26T22:53:41.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.519+0000 7f5e4b000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f5e44160180 con 0x7f5e4406f420 2024-05-26T22:53:41.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.519+0000 7f5e43e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f5e440b5450 0x7f5e440b43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:41.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.519+0000 7f5e43e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f5e440b5450 0x7f5e440b43c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51094/0 (socket says 172.21.15.131:51094) 2024-05-26T22:53:41.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.519+0000 7f5e43e00640 1 -- 172.21.15.131:0/821876350 learned_addr learned my addr 172.21.15.131:0/821876350 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:53:41.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.520+0000 7f5e43e00640 1 -- 172.21.15.131:0/821876350 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5e4406f420 msgr2=0x7f5e440cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:41.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.520+0000 7f5e43e00640 1 --2- 172.21.15.131:0/821876350 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5e4406f420 0x7f5e440cfd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:41.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.520+0000 7f5e43e00640 1 -- 172.21.15.131:0/821876350 >> v1:172.21.15.187:6789/0 conn(0x7f5e440cf590 legacy=0x7f5e4406d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:41.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.520+0000 7f5e43e00640 1 -- 172.21.15.131:0/821876350 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e44160460 con 0x7f5e440b5450 2024-05-26T22:53:41.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.521+0000 7f5e43e00640 1 --2- 172.21.15.131:0/821876350 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5e440b5450 0x7f5e440b43c0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f5e34009870 tx=0x7f5e3402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d4ccc26c4ab57fb server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:41.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.521+0000 7f5e43400640 1 -- 172.21.15.131:0/821876350 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e34037070 con 0x7f5e440b5450 2024-05-26T22:53:41.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.521+0000 7f5e43400640 1 -- 172.21.15.131:0/821876350 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e34032440 con 0x7f5e440b5450 2024-05-26T22:53:41.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.521+0000 7f5e43400640 1 -- 172.21.15.131:0/821876350 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e34032810 con 0x7f5e440b5450 2024-05-26T22:53:41.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.522+0000 7f5e4b000640 1 -- 172.21.15.131:0/821876350 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5e440b5450 msgr2=0x7f5e440b43c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:41.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.522+0000 7f5e4b000640 1 --2- 172.21.15.131:0/821876350 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5e440b5450 0x7f5e440b43c0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f5e34009870 tx=0x7f5e3402a210 comp rx=0 tx=0).stop 2024-05-26T22:53:41.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.522+0000 7f5e4b000640 1 -- 172.21.15.131:0/821876350 shutdown_connections 2024-05-26T22:53:41.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.522+0000 7f5e4b000640 1 --2- 172.21.15.131:0/821876350 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5e4406f420 0x7f5e440cfd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:41.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.522+0000 7f5e4b000640 1 --2- 172.21.15.131:0/821876350 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5e440b5450 0x7f5e440b43c0 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:41.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.522+0000 7f5e4b000640 1 -- 172.21.15.131:0/821876350 >> 172.21.15.131:0/821876350 conn(0x7f5e4406bc90 msgr2=0x7f5e440b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:41.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.522+0000 7f5e4b000640 1 -- 172.21.15.131:0/821876350 shutdown_connections 2024-05-26T22:53:41.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.522+0000 7f5e4b000640 1 -- 172.21.15.131:0/821876350 wait complete. 2024-05-26T22:53:41.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.523+0000 7f5e4b000640 1 Processor -- start 2024-05-26T22:53:41.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.524+0000 7f5e4b000640 1 -- start start 2024-05-26T22:53:41.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.524+0000 7f5e4b000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5e4406f420 0x7f5e44071360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:41.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.524+0000 7f5e4b000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5e440b5450 0x7f5e440718a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:41.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.524+0000 7f5e4b000640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5e440cf590 0x7f5e44144760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:41.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.524+0000 7f5e4b000640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f5e440b5830 con 0x7f5e440b5450 2024-05-26T22:53:41.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.525+0000 7f5e4b000640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5e44160a20 con 0x7f5e440cf590 2024-05-26T22:53:41.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.525+0000 7f5e4b000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f5e4415fde0 con 0x7f5e4406f420 2024-05-26T22:53:41.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.525+0000 7f5e3b400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5e440b5450 0x7f5e440718a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:41.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.525+0000 7f5e3b400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5e440b5450 0x7f5e440718a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51098/0 (socket says 172.21.15.131:51098) 2024-05-26T22:53:41.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.525+0000 7f5e3b400640 1 -- 172.21.15.131:0/1005454608 learned_addr learned my addr 172.21.15.131:0/1005454608 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:53:41.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.525+0000 7f5e43e00640 1 --2- 172.21.15.131:0/1005454608 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5e4406f420 0x7f5e44071360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:41.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.525+0000 7f5e49000640 1 --2- 172.21.15.131:0/1005454608 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5e440cf590 0x7f5e44144760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:41.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.525+0000 7f5e3b400640 1 -- 172.21.15.131:0/1005454608 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5e4406f420 msgr2=0x7f5e44071360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:41.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.525+0000 7f5e3b400640 1 --2- 172.21.15.131:0/1005454608 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5e4406f420 0x7f5e44071360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:41.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.525+0000 7f5e3b400640 1 -- 172.21.15.131:0/1005454608 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5e440cf590 msgr2=0x7f5e44144760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:41.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.525+0000 7f5e3b400640 1 --2- 172.21.15.131:0/1005454608 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5e440cf590 0x7f5e44144760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:41.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.525+0000 7f5e3b400640 1 -- 172.21.15.131:0/1005454608 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e44071ee0 con 0x7f5e440b5450 2024-05-26T22:53:41.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.525+0000 7f5e43e00640 1 --2- 172.21.15.131:0/1005454608 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5e4406f420 0x7f5e44071360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:53:41.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.526+0000 7f5e3b400640 1 --2- 172.21.15.131:0/1005454608 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5e440b5450 0x7f5e440718a0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c00b410 tx=0x7f5e2c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:41.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.526+0000 7f5e40c00640 1 -- 172.21.15.131:0/1005454608 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e2c015020 con 0x7f5e440b5450 2024-05-26T22:53:41.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.526+0000 7f5e40c00640 1 -- 172.21.15.131:0/1005454608 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e2c011850 con 0x7f5e440b5450 2024-05-26T22:53:41.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.526+0000 7f5e40c00640 1 -- 172.21.15.131:0/1005454608 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e2c011c40 con 0x7f5e440b5450 2024-05-26T22:53:41.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.526+0000 7f5e4b000640 1 -- 172.21.15.131:0/1005454608 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e440721c0 con 0x7f5e440b5450 2024-05-26T22:53:41.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.526+0000 7f5e4b000640 1 -- 172.21.15.131:0/1005454608 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e44072780 con 0x7f5e440b5450 2024-05-26T22:53:41.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.527+0000 7f5e4b000640 1 -- 172.21.15.131:0/1005454608 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e44074e90 con 0x7f5e440b5450 2024-05-26T22:53:41.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.528+0000 7f5e40c00640 1 -- 172.21.15.131:0/1005454608 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5e2c018020 con 0x7f5e440b5450 2024-05-26T22:53:41.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.529+0000 7f5e40c00640 1 --2- 172.21.15.131:0/1005454608 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f5e2006be30 0x7f5e2006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:41.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.529+0000 7f5e40c00640 1 -- 172.21.15.131:0/1005454608 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5115+0+0 (secure 0 0 0) 0x7f5e2c08acf0 con 0x7f5e440b5450 2024-05-26T22:53:41.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.529+0000 7f5e43e00640 1 --2- 172.21.15.131:0/1005454608 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f5e2006be30 0x7f5e2006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:41.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.530+0000 7f5e43e00640 1 --2- 172.21.15.131:0/1005454608 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f5e2006be30 0x7f5e2006e2f0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f5e34034000 tx=0x7f5e3402e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:41.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.534+0000 7f5e40c00640 1 -- 172.21.15.131:0/1005454608 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f5e2c01f020 con 0x7f5e440b5450 2024-05-26T22:53:41.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.730+0000 7f5e4b000640 1 -- 172.21.15.131:0/1005454608 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_export_ephemeral_random_max"} v 0) -- 0x7f5e440cee90 con 0x7f5e440b5450 2024-05-26T22:53:41.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.738+0000 7f5e40c00640 1 -- 172.21.15.131:0/1005454608 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_export_ephemeral_random_max"}]=0 v39) ==== 114+0+0 (secure 0 0 0) 0x7f5e2c0172c0 con 0x7f5e440b5450 2024-05-26T22:53:41.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.741+0000 7f5e4b000640 1 -- 172.21.15.131:0/1005454608 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f5e2006be30 msgr2=0x7f5e2006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:41.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.741+0000 7f5e4b000640 1 --2- 172.21.15.131:0/1005454608 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f5e2006be30 0x7f5e2006e2f0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f5e34034000 tx=0x7f5e3402e000 comp rx=0 tx=0).stop 2024-05-26T22:53:41.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.741+0000 7f5e4b000640 1 -- 172.21.15.131:0/1005454608 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5e440b5450 msgr2=0x7f5e440718a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:41.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.741+0000 7f5e4b000640 1 --2- 172.21.15.131:0/1005454608 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5e440b5450 0x7f5e440718a0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c00b410 tx=0x7f5e2c00b8d0 comp rx=0 tx=0).stop 2024-05-26T22:53:41.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.741+0000 7f5e4b000640 1 -- 172.21.15.131:0/1005454608 shutdown_connections 2024-05-26T22:53:41.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.741+0000 7f5e4b000640 1 --2- 172.21.15.131:0/1005454608 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f5e2006be30 0x7f5e2006e2f0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:41.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.741+0000 7f5e4b000640 1 --2- 172.21.15.131:0/1005454608 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5e440cf590 0x7f5e44144760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:41.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.741+0000 7f5e4b000640 1 --2- 172.21.15.131:0/1005454608 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5e440b5450 0x7f5e440718a0 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:41.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.741+0000 7f5e4b000640 1 --2- 172.21.15.131:0/1005454608 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5e4406f420 0x7f5e44071360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:41.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.741+0000 7f5e4b000640 1 -- 172.21.15.131:0/1005454608 >> 172.21.15.131:0/1005454608 conn(0x7f5e4406bc90 msgr2=0x7f5e4415f170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:41.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.741+0000 7f5e4b000640 1 -- 172.21.15.131:0/1005454608 shutdown_connections 2024-05-26T22:53:41.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.741+0000 7f5e4b000640 1 -- 172.21.15.131:0/1005454608 wait complete. 2024-05-26T22:53:41.759 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_export_ephemeral_random 2024-05-26T22:53:41.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.961+0000 7efd86a00640 1 Processor -- start 2024-05-26T22:53:41.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.961+0000 7efd86a00640 1 -- start start 2024-05-26T22:53:41.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.963+0000 7efd86a00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efd8006f420 0x7efd800cfd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:41.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.963+0000 7efd86a00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7efd800cf590 0x7efd800d0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:41.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.963+0000 7efd86a00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd80160180 con 0x7efd800b5450 2024-05-26T22:53:41.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.963+0000 7efd86a00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7efd80160d90 con 0x7efd800cf590 2024-05-26T22:53:41.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.963+0000 7efd86a00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7efd80160460 con 0x7efd8006f420 2024-05-26T22:53:41.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.963+0000 7efd84a00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7efd800cf590 0x7efd800d0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:41.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.963+0000 7efd77e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efd8006f420 0x7efd800cfd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:41.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.964+0000 7efd84a00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7efd800cf590 0x7efd800d0290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.131:44808/0 (socket says 172.21.15.131:44808) 2024-05-26T22:53:41.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.964+0000 7efd84a00640 1 -- 172.21.15.131:0/177945077 learned_addr learned my addr 172.21.15.131:0/177945077 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:53:41.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.964+0000 7efd77e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efd8006f420 0x7efd800cfd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.131:37934/0 (socket says 172.21.15.131:37934) 2024-05-26T22:53:41.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.964+0000 7efd77e00640 1 -- 172.21.15.131:0/177945077 >> v2:172.21.15.187:3300/0 conn(0x7efd800cf590 msgr2=0x7efd800d0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:41.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.964+0000 7efd77e00640 1 --2- 172.21.15.131:0/177945077 >> v2:172.21.15.187:3300/0 conn(0x7efd800cf590 0x7efd800d0290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:41.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.964+0000 7efd77e00640 1 -- 172.21.15.131:0/177945077 >> v1:172.21.15.187:6789/0 conn(0x7efd800b5450 legacy=0x7efd800b43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:41.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.964+0000 7efd77e00640 1 -- 172.21.15.131:0/177945077 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd80160740 con 0x7efd8006f420 2024-05-26T22:53:41.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.965+0000 7efd77e00640 1 --2- 172.21.15.131:0/177945077 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efd8006f420 0x7efd800cfd50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7efd680097a0 tx=0x7efd6802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=106554929791bc51 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:41.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.966+0000 7efd7f400640 1 -- 172.21.15.131:0/177945077 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd68037070 con 0x7efd8006f420 2024-05-26T22:53:41.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.966+0000 7efd7f400640 1 -- 172.21.15.131:0/177945077 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd68032440 con 0x7efd8006f420 2024-05-26T22:53:41.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.966+0000 7efd7f400640 1 -- 172.21.15.131:0/177945077 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd68032810 con 0x7efd8006f420 2024-05-26T22:53:41.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.967+0000 7efd86a00640 1 -- 172.21.15.131:0/177945077 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efd8006f420 msgr2=0x7efd800cfd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:41.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.967+0000 7efd86a00640 1 --2- 172.21.15.131:0/177945077 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efd8006f420 0x7efd800cfd50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7efd680097a0 tx=0x7efd6802a180 comp rx=0 tx=0).stop 2024-05-26T22:53:41.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.967+0000 7efd86a00640 1 -- 172.21.15.131:0/177945077 shutdown_connections 2024-05-26T22:53:41.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.967+0000 7efd86a00640 1 --2- 172.21.15.131:0/177945077 >> v2:172.21.15.187:3300/0 conn(0x7efd800cf590 0x7efd800d0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:41.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.967+0000 7efd86a00640 1 --2- 172.21.15.131:0/177945077 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efd8006f420 0x7efd800cfd50 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:41.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.967+0000 7efd86a00640 1 -- 172.21.15.131:0/177945077 >> 172.21.15.131:0/177945077 conn(0x7efd8006bc90 msgr2=0x7efd800b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:41.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.967+0000 7efd86a00640 1 -- 172.21.15.131:0/177945077 shutdown_connections 2024-05-26T22:53:41.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.967+0000 7efd86a00640 1 -- 172.21.15.131:0/177945077 wait complete. 2024-05-26T22:53:41.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.969+0000 7efd86a00640 1 Processor -- start 2024-05-26T22:53:41.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.969+0000 7efd86a00640 1 -- start start 2024-05-26T22:53:41.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.970+0000 7efd86a00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efd8006f420 0x7efd80152930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:41.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.970+0000 7efd86a00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efd800b5450 0x7efd80072a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:41.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.970+0000 7efd86a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efd800cf590 0x7efd80072f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:41.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.970+0000 7efd86a00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7efd8015fde0 con 0x7efd800cf590 2024-05-26T22:53:41.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.970+0000 7efd86a00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7efd800b5830 con 0x7efd800b5450 2024-05-26T22:53:41.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.970+0000 7efd86a00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7efd8015ff60 con 0x7efd8006f420 2024-05-26T22:53:41.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.970+0000 7efd84a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efd800cf590 0x7efd80072f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:41.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.970+0000 7efd84a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efd800cf590 0x7efd80072f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51104/0 (socket says 172.21.15.131:51104) 2024-05-26T22:53:41.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.970+0000 7efd84a00640 1 -- 172.21.15.131:0/3138715692 learned_addr learned my addr 172.21.15.131:0/3138715692 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:53:41.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.971+0000 7efd7fe00640 1 --2- 172.21.15.131:0/3138715692 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efd8006f420 0x7efd80152930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:41.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.971+0000 7efd77e00640 1 --2- 172.21.15.131:0/3138715692 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efd800b5450 0x7efd80072a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:41.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.971+0000 7efd84a00640 1 -- 172.21.15.131:0/3138715692 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efd8006f420 msgr2=0x7efd80152930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:41.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.971+0000 7efd84a00640 1 --2- 172.21.15.131:0/3138715692 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efd8006f420 0x7efd80152930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:41.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.971+0000 7efd84a00640 1 -- 172.21.15.131:0/3138715692 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efd800b5450 msgr2=0x7efd80072a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:41.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.971+0000 7efd84a00640 1 --2- 172.21.15.131:0/3138715692 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efd800b5450 0x7efd80072a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:41.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.971+0000 7efd84a00640 1 -- 172.21.15.131:0/3138715692 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd8015f430 con 0x7efd800cf590 2024-05-26T22:53:41.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.971+0000 7efd84a00640 1 --2- 172.21.15.131:0/3138715692 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efd800cf590 0x7efd80072f60 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7efd78009d90 tx=0x7efd7800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:41.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.971+0000 7efd7cc00640 1 -- 172.21.15.131:0/3138715692 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd7800cd20 con 0x7efd800cf590 2024-05-26T22:53:41.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.971+0000 7efd7cc00640 1 -- 172.21.15.131:0/3138715692 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd78003990 con 0x7efd800cf590 2024-05-26T22:53:41.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.971+0000 7efd7cc00640 1 -- 172.21.15.131:0/3138715692 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd78003d60 con 0x7efd800cf590 2024-05-26T22:53:41.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.971+0000 7efd86a00640 1 -- 172.21.15.131:0/3138715692 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd801f5240 con 0x7efd800cf590 2024-05-26T22:53:41.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.972+0000 7efd86a00640 1 -- 172.21.15.131:0/3138715692 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd80152e70 con 0x7efd800cf590 2024-05-26T22:53:41.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.973+0000 7efd86a00640 1 -- 172.21.15.131:0/3138715692 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd801f5030 con 0x7efd800cf590 2024-05-26T22:53:41.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.974+0000 7efd7cc00640 1 -- 172.21.15.131:0/3138715692 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7efd78012050 con 0x7efd800cf590 2024-05-26T22:53:41.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.975+0000 7efd7cc00640 1 --2- 172.21.15.131:0/3138715692 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7efd5c06be30 0x7efd5c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:41.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.975+0000 7efd7cc00640 1 -- 172.21.15.131:0/3138715692 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5115+0+0 (secure 0 0 0) 0x7efd7808b140 con 0x7efd800cf590 2024-05-26T22:53:41.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.975+0000 7efd7fe00640 1 --2- 172.21.15.131:0/3138715692 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7efd5c06be30 0x7efd5c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:41.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.976+0000 7efd7fe00640 1 --2- 172.21.15.131:0/3138715692 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7efd5c06be30 0x7efd5c06e2f0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7efd70001640 tx=0x7efd70009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:41.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:41.979+0000 7efd7cc00640 1 -- 172.21.15.131:0/3138715692 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7efd78010040 con 0x7efd800cf590 2024-05-26T22:53:42.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.176+0000 7efd86a00640 1 -- 172.21.15.131:0/3138715692 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_export_ephemeral_random"} v 0) -- 0x7efd800cee90 con 0x7efd800cf590 2024-05-26T22:53:42.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.183+0000 7efd7cc00640 1 -- 172.21.15.131:0/3138715692 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_export_ephemeral_random"}]=0 v40) ==== 110+0+0 (secure 0 0 0) 0x7efd78059d30 con 0x7efd800cf590 2024-05-26T22:53:42.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.186+0000 7efd86a00640 1 -- 172.21.15.131:0/3138715692 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7efd5c06be30 msgr2=0x7efd5c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:42.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.186+0000 7efd86a00640 1 --2- 172.21.15.131:0/3138715692 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7efd5c06be30 0x7efd5c06e2f0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7efd70001640 tx=0x7efd70009040 comp rx=0 tx=0).stop 2024-05-26T22:53:42.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.186+0000 7efd86a00640 1 -- 172.21.15.131:0/3138715692 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efd800cf590 msgr2=0x7efd80072f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:42.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.186+0000 7efd86a00640 1 --2- 172.21.15.131:0/3138715692 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efd800cf590 0x7efd80072f60 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7efd78009d90 tx=0x7efd7800c490 comp rx=0 tx=0).stop 2024-05-26T22:53:42.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.186+0000 7efd86a00640 1 -- 172.21.15.131:0/3138715692 shutdown_connections 2024-05-26T22:53:42.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.186+0000 7efd86a00640 1 --2- 172.21.15.131:0/3138715692 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7efd5c06be30 0x7efd5c06e2f0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:42.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.186+0000 7efd86a00640 1 --2- 172.21.15.131:0/3138715692 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efd800cf590 0x7efd80072f60 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:42.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.186+0000 7efd86a00640 1 --2- 172.21.15.131:0/3138715692 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efd800b5450 0x7efd80072a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:42.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.186+0000 7efd86a00640 1 --2- 172.21.15.131:0/3138715692 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efd8006f420 0x7efd80152930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:42.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.186+0000 7efd86a00640 1 -- 172.21.15.131:0/3138715692 >> 172.21.15.131:0/3138715692 conn(0x7efd8006bc90 msgr2=0x7efd8006e6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:42.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.186+0000 7efd86a00640 1 -- 172.21.15.131:0/3138715692 shutdown_connections 2024-05-26T22:53:42.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.186+0000 7efd86a00640 1 -- 172.21.15.131:0/3138715692 wait complete. 2024-05-26T22:53:42.205 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_export_ephemeral_distributed 2024-05-26T22:53:42.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.405+0000 7f225ca00640 1 Processor -- start 2024-05-26T22:53:42.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.406+0000 7f225ca00640 1 -- start start 2024-05-26T22:53:42.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.407+0000 7f225ca00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f22581554f0 0x7f2258071f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:42.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.407+0000 7f225ca00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f22580724a0 0x7f225806e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:42.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.407+0000 7f225ca00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22581558d0 con 0x7f225815c220 2024-05-26T22:53:42.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.407+0000 7f225ca00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f22580729b0 con 0x7f22581554f0 2024-05-26T22:53:42.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.407+0000 7f225ca00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f225815c750 con 0x7f22580724a0 2024-05-26T22:53:42.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.407+0000 7f2257400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f22581554f0 0x7f2258071f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:42.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.407+0000 7f2257400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f22581554f0 0x7f2258071f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51118/0 (socket says 172.21.15.131:51118) 2024-05-26T22:53:42.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.407+0000 7f2257400640 1 -- 172.21.15.131:0/2298236743 learned_addr learned my addr 172.21.15.131:0/2298236743 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:53:42.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.408+0000 7f224ea00640 1 --2- 172.21.15.131:0/2298236743 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f22580724a0 0x7f225806e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:42.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.408+0000 7f2257400640 1 -- 172.21.15.131:0/2298236743 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f22580724a0 msgr2=0x7f225806e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:42.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.408+0000 7f2257400640 1 --2- 172.21.15.131:0/2298236743 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f22580724a0 0x7f225806e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:42.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.408+0000 7f2257400640 1 -- 172.21.15.131:0/2298236743 >> v1:172.21.15.187:6789/0 conn(0x7f225815c220 legacy=0x7f225815ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:42.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.408+0000 7f2257400640 1 -- 172.21.15.131:0/2298236743 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f225815ca30 con 0x7f22581554f0 2024-05-26T22:53:42.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.408+0000 7f224ea00640 1 --2- 172.21.15.131:0/2298236743 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f22580724a0 0x7f225806e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-26T22:53:42.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.409+0000 7f2257400640 1 --2- 172.21.15.131:0/2298236743 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f22581554f0 0x7f2258071f60 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f2248009870 tx=0x7f224802a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=341b20747173b7ec server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:42.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.410+0000 7f2256a00640 1 -- 172.21.15.131:0/2298236743 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2248037070 con 0x7f22581554f0 2024-05-26T22:53:42.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.410+0000 7f2256a00640 1 -- 172.21.15.131:0/2298236743 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2248032440 con 0x7f22581554f0 2024-05-26T22:53:42.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.410+0000 7f2256a00640 1 -- 172.21.15.131:0/2298236743 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2248032810 con 0x7f22581554f0 2024-05-26T22:53:42.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.410+0000 7f225ca00640 1 -- 172.21.15.131:0/2298236743 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f22581554f0 msgr2=0x7f2258071f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:42.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.410+0000 7f225ca00640 1 --2- 172.21.15.131:0/2298236743 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f22581554f0 0x7f2258071f60 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f2248009870 tx=0x7f224802a200 comp rx=0 tx=0).stop 2024-05-26T22:53:42.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.410+0000 7f225ca00640 1 -- 172.21.15.131:0/2298236743 shutdown_connections 2024-05-26T22:53:42.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.410+0000 7f225ca00640 1 --2- 172.21.15.131:0/2298236743 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f22580724a0 0x7f225806e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:42.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.410+0000 7f225ca00640 1 --2- 172.21.15.131:0/2298236743 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f22581554f0 0x7f2258071f60 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:42.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.410+0000 7f225ca00640 1 -- 172.21.15.131:0/2298236743 >> 172.21.15.131:0/2298236743 conn(0x7f22580cc7f0 msgr2=0x7f22580ccbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:42.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.411+0000 7f225ca00640 1 -- 172.21.15.131:0/2298236743 shutdown_connections 2024-05-26T22:53:42.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.411+0000 7f225ca00640 1 -- 172.21.15.131:0/2298236743 wait complete. 2024-05-26T22:53:42.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.412+0000 7f225ca00640 1 Processor -- start 2024-05-26T22:53:42.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.412+0000 7f225ca00640 1 -- start start 2024-05-26T22:53:42.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.413+0000 7f225ca00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f22580724a0 0x7f22580aec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:42.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.413+0000 7f225ca00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f22581554f0 0x7f22580cff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:42.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.413+0000 7f225ca00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f225815c220 0x7f2258153760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:42.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.414+0000 7f225ca00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f225815e3c0 con 0x7f22581554f0 2024-05-26T22:53:42.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.414+0000 7f225ca00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f225815e240 con 0x7f22580724a0 2024-05-26T22:53:42.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.414+0000 7f225ca00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f225815e540 con 0x7f225815c220 2024-05-26T22:53:42.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.414+0000 7f224ea00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f22581554f0 0x7f22580cff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:42.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.414+0000 7f224ea00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f22581554f0 0x7f22580cff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51122/0 (socket says 172.21.15.131:51122) 2024-05-26T22:53:42.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.414+0000 7f224ea00640 1 -- 172.21.15.131:0/2451023693 learned_addr learned my addr 172.21.15.131:0/2451023693 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:53:42.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.414+0000 7f2257400640 1 --2- 172.21.15.131:0/2451023693 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f22580724a0 0x7f22580aec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:42.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.414+0000 7f2257e00640 1 --2- 172.21.15.131:0/2451023693 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f225815c220 0x7f2258153760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:42.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.414+0000 7f224ea00640 1 -- 172.21.15.131:0/2451023693 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f225815c220 msgr2=0x7f2258153760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:42.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.414+0000 7f224ea00640 1 --2- 172.21.15.131:0/2451023693 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f225815c220 0x7f2258153760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:42.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.414+0000 7f224ea00640 1 -- 172.21.15.131:0/2451023693 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f22580724a0 msgr2=0x7f22580aec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:42.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.414+0000 7f224ea00640 1 --2- 172.21.15.131:0/2451023693 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f22580724a0 0x7f22580aec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:42.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.414+0000 7f224ea00640 1 -- 172.21.15.131:0/2451023693 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22580d0870 con 0x7f22581554f0 2024-05-26T22:53:42.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.414+0000 7f2257400640 1 --2- 172.21.15.131:0/2451023693 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f22580724a0 0x7f22580aec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:53:42.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.414+0000 7f2257e00640 1 --2- 172.21.15.131:0/2451023693 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f225815c220 0x7f2258153760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:53:42.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.415+0000 7f224ea00640 1 --2- 172.21.15.131:0/2451023693 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f22581554f0 0x7f22580cff40 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f2240007ee0 tx=0x7f224000d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:42.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.415+0000 7f224fe00640 1 -- 172.21.15.131:0/2451023693 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f224001a070 con 0x7f22581554f0 2024-05-26T22:53:42.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.415+0000 7f224fe00640 1 -- 172.21.15.131:0/2451023693 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2240003b90 con 0x7f22581554f0 2024-05-26T22:53:42.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.415+0000 7f225ca00640 1 -- 172.21.15.131:0/2451023693 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2258152c80 con 0x7f22581554f0 2024-05-26T22:53:42.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.415+0000 7f224fe00640 1 -- 172.21.15.131:0/2451023693 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2240004030 con 0x7f22581554f0 2024-05-26T22:53:42.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.415+0000 7f225ca00640 1 -- 172.21.15.131:0/2451023693 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2258153240 con 0x7f22581554f0 2024-05-26T22:53:42.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.416+0000 7f225ca00640 1 -- 172.21.15.131:0/2451023693 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22580694e0 con 0x7f22581554f0 2024-05-26T22:53:42.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.417+0000 7f224fe00640 1 -- 172.21.15.131:0/2451023693 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f224001d070 con 0x7f22581554f0 2024-05-26T22:53:42.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.418+0000 7f224fe00640 1 --2- 172.21.15.131:0/2451023693 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f223406be30 0x7f223406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:42.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.418+0000 7f224fe00640 1 -- 172.21.15.131:0/2451023693 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5115+0+0 (secure 0 0 0) 0x7f224008ae40 con 0x7f22581554f0 2024-05-26T22:53:42.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.418+0000 7f2257400640 1 --2- 172.21.15.131:0/2451023693 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f223406be30 0x7f223406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:42.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.419+0000 7f2257400640 1 --2- 172.21.15.131:0/2451023693 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f223406be30 0x7f223406e2f0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f2248034000 tx=0x7f224802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:42.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.423+0000 7f224fe00640 1 -- 172.21.15.131:0/2451023693 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f2240010070 con 0x7f22581554f0 2024-05-26T22:53:42.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.625+0000 7f225ca00640 1 -- 172.21.15.131:0/2451023693 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_export_ephemeral_distributed"} v 0) -- 0x7f22580ce430 con 0x7f22581554f0 2024-05-26T22:53:42.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.632+0000 7f224fe00640 1 -- 172.21.15.131:0/2451023693 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_export_ephemeral_distributed"}]=0 v41) ==== 115+0+0 (secure 0 0 0) 0x7f2240059350 con 0x7f22581554f0 2024-05-26T22:53:42.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.636+0000 7f225ca00640 1 -- 172.21.15.131:0/2451023693 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f223406be30 msgr2=0x7f223406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:42.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.636+0000 7f225ca00640 1 --2- 172.21.15.131:0/2451023693 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f223406be30 0x7f223406e2f0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f2248034000 tx=0x7f224802e000 comp rx=0 tx=0).stop 2024-05-26T22:53:42.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.636+0000 7f225ca00640 1 -- 172.21.15.131:0/2451023693 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f22581554f0 msgr2=0x7f22580cff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:42.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.636+0000 7f225ca00640 1 --2- 172.21.15.131:0/2451023693 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f22581554f0 0x7f22580cff40 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f2240007ee0 tx=0x7f224000d4a0 comp rx=0 tx=0).stop 2024-05-26T22:53:42.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.636+0000 7f225ca00640 1 -- 172.21.15.131:0/2451023693 shutdown_connections 2024-05-26T22:53:42.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.636+0000 7f225ca00640 1 --2- 172.21.15.131:0/2451023693 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f225815c220 0x7f2258153760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:42.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.636+0000 7f225ca00640 1 --2- 172.21.15.131:0/2451023693 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f223406be30 0x7f223406e2f0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:42.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.636+0000 7f225ca00640 1 --2- 172.21.15.131:0/2451023693 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f22581554f0 0x7f22580cff40 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:42.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.636+0000 7f225ca00640 1 --2- 172.21.15.131:0/2451023693 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f22580724a0 0x7f22580aec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:42.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.636+0000 7f225ca00640 1 -- 172.21.15.131:0/2451023693 >> 172.21.15.131:0/2451023693 conn(0x7f22580cc7f0 msgr2=0x7f2258151750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:42.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.636+0000 7f225ca00640 1 -- 172.21.15.131:0/2451023693 shutdown_connections 2024-05-26T22:53:42.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.636+0000 7f225ca00640 1 -- 172.21.15.131:0/2451023693 wait complete. 2024-05-26T22:53:42.653 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-26T22:53:42.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.851+0000 7f1ee7200640 1 Processor -- start 2024-05-26T22:53:42.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.851+0000 7f1ee7200640 1 -- start start 2024-05-26T22:53:42.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.852+0000 7f1ee7200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f1ee01554f0 0x7f1ee0071f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:42.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.852+0000 7f1ee7200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1ee006ebc0 0x7f1ee015d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:42.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.853+0000 7f1ee7200640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ee01558d0 con 0x7f1ee00724a0 2024-05-26T22:53:42.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.853+0000 7f1ee7200640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f1ee00729b0 con 0x7f1ee01554f0 2024-05-26T22:53:42.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.853+0000 7f1ee7200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1ee015c750 con 0x7f1ee006ebc0 2024-05-26T22:53:42.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.853+0000 7f1ee5400640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f1ee00724a0 0x7f1ee006e4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:47720/0 (socket says 172.21.15.131:47720) 2024-05-26T22:53:42.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.853+0000 7f1ee5400640 1 -- 172.21.15.131:0/271904644 learned_addr learned my addr 172.21.15.131:0/271904644 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:53:42.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.853+0000 7f1ee5e00640 1 --2- 172.21.15.131:0/271904644 >> v2:172.21.15.131:3300/0 conn(0x7f1ee01554f0 0x7f1ee0071f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:42.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.853+0000 7f1ee6800640 1 --2- 172.21.15.131:0/271904644 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1ee006ebc0 0x7f1ee015d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:42.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.853+0000 7f1ee5e00640 1 -- 172.21.15.131:0/271904644 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1ee006ebc0 msgr2=0x7f1ee015d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:42.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.854+0000 7f1ee5e00640 1 --2- 172.21.15.131:0/271904644 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1ee006ebc0 0x7f1ee015d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:42.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.854+0000 7f1ee5e00640 1 -- 172.21.15.131:0/271904644 >> v1:172.21.15.131:6789/0 conn(0x7f1ee00724a0 legacy=0x7f1ee006e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:42.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.853+0000 7f1ee4a00640 1 -- 172.21.15.131:0/271904644 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2508269485 0 0) 0x7f1ed0001110 con 0x7f1ee00724a0 2024-05-26T22:53:42.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.854+0000 7f1ee5e00640 1 -- 172.21.15.131:0/271904644 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ee015ca30 con 0x7f1ee01554f0 2024-05-26T22:53:42.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.855+0000 7f1ee5e00640 1 --2- 172.21.15.131:0/271904644 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1ee01554f0 0x7f1ee0071f60 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f1ed40097a0 tx=0x7f1ed402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec4c5ac0608681aa server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:42.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.855+0000 7f1ee4a00640 1 -- 172.21.15.131:0/271904644 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ed4037070 con 0x7f1ee01554f0 2024-05-26T22:53:42.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.855+0000 7f1ee4a00640 1 -- 172.21.15.131:0/271904644 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ed4032440 con 0x7f1ee01554f0 2024-05-26T22:53:42.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.855+0000 7f1ee4a00640 1 -- 172.21.15.131:0/271904644 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ed4032810 con 0x7f1ee01554f0 2024-05-26T22:53:42.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.856+0000 7f1ee7200640 1 -- 172.21.15.131:0/271904644 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1ee01554f0 msgr2=0x7f1ee0071f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:42.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.856+0000 7f1ee7200640 1 --2- 172.21.15.131:0/271904644 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1ee01554f0 0x7f1ee0071f60 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f1ed40097a0 tx=0x7f1ed402a180 comp rx=0 tx=0).stop 2024-05-26T22:53:42.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.856+0000 7f1ee7200640 1 -- 172.21.15.131:0/271904644 shutdown_connections 2024-05-26T22:53:42.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.856+0000 7f1ee7200640 1 --2- 172.21.15.131:0/271904644 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1ee006ebc0 0x7f1ee015d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:42.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.856+0000 7f1ee7200640 1 --2- 172.21.15.131:0/271904644 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1ee01554f0 0x7f1ee0071f60 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:42.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.856+0000 7f1ee7200640 1 -- 172.21.15.131:0/271904644 >> 172.21.15.131:0/271904644 conn(0x7f1ee00cc7f0 msgr2=0x7f1ee00ccbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:42.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.856+0000 7f1ee7200640 1 -- 172.21.15.131:0/271904644 shutdown_connections 2024-05-26T22:53:42.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.856+0000 7f1ee7200640 1 -- 172.21.15.131:0/271904644 wait complete. 2024-05-26T22:53:42.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.858+0000 7f1ee7200640 1 Processor -- start 2024-05-26T22:53:42.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.858+0000 7f1ee7200640 1 -- start start 2024-05-26T22:53:42.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.859+0000 7f1ee7200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1ee006ebc0 0x7f1ee013f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:42.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.859+0000 7f1ee7200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1ee00724a0 0x7f1ee0159ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:42.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.859+0000 7f1ee7200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1ee01554f0 0x7f1ee015a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:42.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.859+0000 7f1ee7200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f1ee015c3a0 con 0x7f1ee006ebc0 2024-05-26T22:53:42.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.859+0000 7f1ee7200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1ee015c220 con 0x7f1ee00724a0 2024-05-26T22:53:42.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.859+0000 7f1ee7200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1ee00ce650 con 0x7f1ee01554f0 2024-05-26T22:53:42.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.859+0000 7f1ee5e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1ee006ebc0 0x7f1ee013f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:42.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.859+0000 7f1ee5e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1ee006ebc0 0x7f1ee013f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51128/0 (socket says 172.21.15.131:51128) 2024-05-26T22:53:42.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.859+0000 7f1ee5e00640 1 -- 172.21.15.131:0/720476286 learned_addr learned my addr 172.21.15.131:0/720476286 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:53:42.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.859+0000 7f1ee5400640 1 --2- 172.21.15.131:0/720476286 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1ee00724a0 0x7f1ee0159ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:42.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.859+0000 7f1ee5e00640 1 -- 172.21.15.131:0/720476286 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1ee01554f0 msgr2=0x7f1ee015a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:42.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.859+0000 7f1ee5e00640 1 --2- 172.21.15.131:0/720476286 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1ee01554f0 0x7f1ee015a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:42.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.859+0000 7f1ee5e00640 1 -- 172.21.15.131:0/720476286 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1ee00724a0 msgr2=0x7f1ee0159ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:42.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.859+0000 7f1ee5e00640 1 --2- 172.21.15.131:0/720476286 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1ee00724a0 0x7f1ee0159ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:42.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.859+0000 7f1ee5e00640 1 -- 172.21.15.131:0/720476286 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ee00ce8a0 con 0x7f1ee006ebc0 2024-05-26T22:53:42.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.860+0000 7f1ee5e00640 1 --2- 172.21.15.131:0/720476286 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1ee006ebc0 0x7f1ee013f6c0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f1ed4035040 tx=0x7f1ed40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:42.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.860+0000 7f1eda000640 1 -- 172.21.15.131:0/720476286 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ed4037040 con 0x7f1ee006ebc0 2024-05-26T22:53:42.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.860+0000 7f1eda000640 1 -- 172.21.15.131:0/720476286 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ed4030770 con 0x7f1ee006ebc0 2024-05-26T22:53:42.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.860+0000 7f1eda000640 1 -- 172.21.15.131:0/720476286 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ed4030b80 con 0x7f1ee006ebc0 2024-05-26T22:53:42.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.860+0000 7f1ee7200640 1 -- 172.21.15.131:0/720476286 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ee00ceb80 con 0x7f1ee006ebc0 2024-05-26T22:53:42.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.860+0000 7f1ee7200640 1 -- 172.21.15.131:0/720476286 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ee00cf140 con 0x7f1ee006ebc0 2024-05-26T22:53:42.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.862+0000 7f1ee7200640 1 -- 172.21.15.131:0/720476286 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ee00694e0 con 0x7f1ee006ebc0 2024-05-26T22:53:42.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.863+0000 7f1eda000640 1 -- 172.21.15.131:0/720476286 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1ed4044020 con 0x7f1ee006ebc0 2024-05-26T22:53:42.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.864+0000 7f1eda000640 1 --2- 172.21.15.131:0/720476286 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f1ec406be30 0x7f1ec406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:42.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.864+0000 7f1eda000640 1 -- 172.21.15.131:0/720476286 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5115+0+0 (secure 0 0 0) 0x7f1ed40ab160 con 0x7f1ee006ebc0 2024-05-26T22:53:42.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.864+0000 7f1ee5400640 1 --2- 172.21.15.131:0/720476286 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f1ec406be30 0x7f1ec406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:42.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.864+0000 7f1ee5400640 1 --2- 172.21.15.131:0/720476286 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f1ec406be30 0x7f1ec406e2f0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f1ee01f7750 tx=0x7f1ed0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:42.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:42.868+0000 7f1eda000640 1 -- 172.21.15.131:0/720476286 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f1ed4032e20 con 0x7f1ee006ebc0 2024-05-26T22:53:43.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:43.065+0000 7f1ee7200640 1 -- 172.21.15.131:0/720476286 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f1ee00ce430 con 0x7f1ee006ebc0 2024-05-26T22:53:43.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:43.072+0000 7f1eda000640 1 -- 172.21.15.131:0/720476286 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v42) ==== 104+0+0 (secure 0 0 0) 0x7f1ed4079d50 con 0x7f1ee006ebc0 2024-05-26T22:53:43.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:43.074+0000 7f1ee7200640 1 -- 172.21.15.131:0/720476286 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f1ec406be30 msgr2=0x7f1ec406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:43.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:43.074+0000 7f1ee7200640 1 --2- 172.21.15.131:0/720476286 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f1ec406be30 0x7f1ec406e2f0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f1ee01f7750 tx=0x7f1ed0009040 comp rx=0 tx=0).stop 2024-05-26T22:53:43.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:43.074+0000 7f1ee7200640 1 -- 172.21.15.131:0/720476286 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1ee006ebc0 msgr2=0x7f1ee013f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:43.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:43.074+0000 7f1ee7200640 1 --2- 172.21.15.131:0/720476286 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1ee006ebc0 0x7f1ee013f6c0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f1ed4035040 tx=0x7f1ed40046c0 comp rx=0 tx=0).stop 2024-05-26T22:53:43.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:43.074+0000 7f1ee7200640 1 -- 172.21.15.131:0/720476286 shutdown_connections 2024-05-26T22:53:43.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:43.074+0000 7f1ee7200640 1 --2- 172.21.15.131:0/720476286 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1ee01554f0 0x7f1ee015a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:43.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:43.074+0000 7f1ee7200640 1 --2- 172.21.15.131:0/720476286 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f1ec406be30 0x7f1ec406e2f0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:43.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:43.074+0000 7f1ee7200640 1 --2- 172.21.15.131:0/720476286 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1ee00724a0 0x7f1ee0159ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:43.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:43.074+0000 7f1ee7200640 1 --2- 172.21.15.131:0/720476286 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1ee006ebc0 0x7f1ee013f6c0 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:43.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:43.074+0000 7f1ee7200640 1 -- 172.21.15.131:0/720476286 >> 172.21.15.131:0/720476286 conn(0x7f1ee00cc7f0 msgr2=0x7f1ee00afaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:43.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:43.075+0000 7f1ee7200640 1 -- 172.21.15.131:0/720476286 shutdown_connections 2024-05-26T22:53:43.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:43.075+0000 7f1ee7200640 1 -- 172.21.15.131:0/720476286 wait complete. 2024-05-26T22:53:43.093 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_exports.TestEphemeralPins.test_ephemeral_pin_dist_override' 2024-05-26T22:53:43.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:43.288+0000 7f449f000640 1 Processor -- start 2024-05-26T22:53:43.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:43.289+0000 7f449f000640 1 -- start start 2024-05-26T22:53:43.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:43.290+0000 7f449f000640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f4498147d20 0x7f4498072400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:43.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:43.290+0000 7f449f000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f449806e700 0x7f44980b2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:43.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:43.290+0000 7f449f000640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4498072b30 con 0x7f44980b34c0 2024-05-26T22:53:43.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:43.290+0000 7f449f000640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f44980736b0 con 0x7f4498147d20 2024-05-26T22:53:43.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:43.290+0000 7f449f000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f4498072e10 con 0x7f449806e700 2024-05-26T22:53:43.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:43.291+0000 7f449e600640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f44980b34c0 0x7f449806b310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:47724/0 (socket says 172.21.15.131:47724) 2024-05-26T22:53:43.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:43.291+0000 7f449e600640 1 -- 172.21.15.131:0/847940622 learned_addr learned my addr 172.21.15.131:0/847940622 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:53:43.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:43.291+0000 7f449d200640 1 --2- 172.21.15.131:0/847940622 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f449806e700 0x7f44980b2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:43.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:43.291+0000 7f449dc00640 1 --2- 172.21.15.131:0/847940622 >> v2:172.21.15.187:3300/0 conn(0x7f4498147d20 0x7f4498072400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:43.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:43.291+0000 7f4497e00640 1 -- 172.21.15.131:0/847940622 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2222872830 0 0) 0x7f4498072b30 con 0x7f44980b34c0 2024-05-26T22:53:43.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:43.291+0000 7f4497e00640 1 -- 172.21.15.131:0/847940622 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4480003520 con 0x7f44980b34c0 2024-05-26T22:53:43.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:43.292+0000 7f4497e00640 1 -- 172.21.15.131:0/847940622 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4245363136 0 0) 0x7f4480003520 con 0x7f44980b34c0 2024-05-26T22:53:43.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:43.292+0000 7f4497e00640 1 -- 172.21.15.131:0/847940622 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f449806e700 msgr2=0x7f44980b2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:43.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:43.292+0000 7f4497e00640 1 --2- 172.21.15.131:0/847940622 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f449806e700 0x7f44980b2f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:43.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:43.292+0000 7f4497e00640 1 -- 172.21.15.131:0/847940622 >> v2:172.21.15.187:3300/0 conn(0x7f4498147d20 msgr2=0x7f4498072400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:43.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:43.292+0000 7f4497e00640 1 --2- 172.21.15.131:0/847940622 >> v2:172.21.15.187:3300/0 conn(0x7f4498147d20 0x7f4498072400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:43.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:43.292+0000 7f4497e00640 1 -- 172.21.15.131:0/847940622 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44980730f0 con 0x7f44980b34c0 2024-05-26T22:53:43.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:43.292+0000 7f4497e00640 1 -- 172.21.15.131:0/847940622 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7f4490001670 con 0x7f44980b34c0 2024-05-26T22:53:43.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:43.292+0000 7f4497e00640 1 -- 172.21.15.131:0/847940622 >> v1:172.21.15.131:6789/0 conn(0x7f44980b34c0 legacy=0x7f449806b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:43.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:43.292+0000 7f4497e00640 1 --2- 172.21.15.131:0/847940622 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f449806e700 0x7f44800039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:43.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:43.292+0000 7f4497e00640 1 --2- 172.21.15.131:0/847940622 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4480003f30 0x7f4480004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:43.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:43.292+0000 7f4497e00640 1 --2- 172.21.15.131:0/847940622 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4480004d40 0x7f44800051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:43.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:43.292+0000 7f4497e00640 1 -- 172.21.15.131:0/847940622 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f44980736b0 con 0x7f4480004d40 2024-05-26T22:53:43.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:43.292+0000 7f4497e00640 1 -- 172.21.15.131:0/847940622 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4498072e10 con 0x7f4480003f30 2024-05-26T22:53:43.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:43.292+0000 7f4497e00640 1 -- 172.21.15.131:0/847940622 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f448000ba10 con 0x7f449806e700 2024-05-26T22:53:43.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:43.292+0000 7f449e600640 1 --2- 172.21.15.131:0/847940622 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4480004d40 0x7f44800051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:43.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:43.292+0000 7f449dc00640 1 --2- 172.21.15.131:0/847940622 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f449806e700 0x7f44800039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:43.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:43.292+0000 7f449e600640 1 -- 172.21.15.131:0/847940622 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f449806e700 msgr2=0x7f44800039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:43.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:43.292+0000 7f449d200640 1 --2- 172.21.15.131:0/847940622 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4480003f30 0x7f4480004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:43.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:43.292+0000 7f449e600640 1 --2- 172.21.15.131:0/847940622 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f449806e700 0x7f44800039b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:43.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:43.292+0000 7f449e600640 1 -- 172.21.15.131:0/847940622 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4480003f30 msgr2=0x7f4480004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:43.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:43.292+0000 7f449e600640 1 --2- 172.21.15.131:0/847940622 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4480003f30 0x7f4480004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:43.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:43.292+0000 7f449e600640 1 -- 172.21.15.131:0/847940622 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4480003520 con 0x7f4480004d40 2024-05-26T22:53:43.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:43.293+0000 7f449e600640 1 --2- 172.21.15.131:0/847940622 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4480004d40 0x7f44800051c0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f449000c2c0 tx=0x7f4480005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=363ea26b3b38b084 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:43.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:43.294+0000 7f4497e00640 1 -- 172.21.15.131:0/847940622 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4490007d50 con 0x7f4480004d40 2024-05-26T22:53:43.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:43.294+0000 7f4497e00640 1 -- 172.21.15.131:0/847940622 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f449002e070 con 0x7f4480004d40 2024-05-26T22:53:43.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:43.294+0000 7f4497e00640 1 -- 172.21.15.131:0/847940622 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4490005d70 con 0x7f4480004d40 2024-05-26T22:53:43.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:43.294+0000 7f449f000640 1 -- 172.21.15.131:0/847940622 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4480004d40 msgr2=0x7f44800051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:43.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:43.294+0000 7f449f000640 1 --2- 172.21.15.131:0/847940622 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4480004d40 0x7f44800051c0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f449000c2c0 tx=0x7f4480005830 comp rx=0 tx=0).stop 2024-05-26T22:53:43.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:43.294+0000 7f449e600640 1 -- 172.21.15.131:0/847940622 reap_dead start 2024-05-26T22:53:43.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:43.294+0000 7f449f000640 1 -- 172.21.15.131:0/847940622 shutdown_connections 2024-05-26T22:53:43.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:43.294+0000 7f449f000640 1 -- 172.21.15.131:0/847940622 >> 172.21.15.131:0/847940622 conn(0x7f4498070100 msgr2=0x7f4498070510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:43.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:43.295+0000 7f449f000640 1 -- 172.21.15.131:0/847940622 shutdown_connections 2024-05-26T22:53:43.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:43.295+0000 7f449f000640 1 -- 172.21.15.131:0/847940622 wait complete. 2024-05-26T22:53:43.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:43.296+0000 7f449f000640 1 Processor -- start 2024-05-26T22:53:43.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:43.296+0000 7f449f000640 1 -- start start 2024-05-26T22:53:43.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:43.297+0000 7f449f000640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4480004d40 0x7f44980b0310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:43.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:43.297+0000 7f449f000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4498144ea0 0x7f44980ad870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:43.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:43.297+0000 7f449f000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f44981f3b10 0x7f44980c8120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:43.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:43.297+0000 7f449f000640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f44980cef50 con 0x7f44981f3b10 2024-05-26T22:53:43.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:43.297+0000 7f449f000640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f44980cedd0 con 0x7f4480004d40 2024-05-26T22:53:43.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:43.297+0000 7f449f000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f44980cf0d0 con 0x7f4498144ea0 2024-05-26T22:53:43.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:43.298+0000 7f449e600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f44981f3b10 0x7f44980c8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:43.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:43.298+0000 7f449e600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f44981f3b10 0x7f44980c8120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51154/0 (socket says 172.21.15.131:51154) 2024-05-26T22:53:43.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:43.298+0000 7f449e600640 1 -- 172.21.15.131:0/2336717718 learned_addr learned my addr 172.21.15.131:0/2336717718 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:53:43.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:43.298+0000 7f449e600640 1 -- 172.21.15.131:0/2336717718 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4498144ea0 msgr2=0x7f44980ad870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:43.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:43.298+0000 7f449e600640 1 --2- 172.21.15.131:0/2336717718 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4498144ea0 0x7f44980ad870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:43.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:43.298+0000 7f449e600640 1 -- 172.21.15.131:0/2336717718 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4480004d40 msgr2=0x7f44980b0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:43.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:43.298+0000 7f449e600640 1 --2- 172.21.15.131:0/2336717718 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4480004d40 0x7f44980b0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:43.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:43.298+0000 7f449e600640 1 -- 172.21.15.131:0/2336717718 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4498145610 con 0x7f44981f3b10 2024-05-26T22:53:43.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:43.298+0000 7f449e600640 1 --2- 172.21.15.131:0/2336717718 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f44981f3b10 0x7f44980c8120 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f4490034040 tx=0x7f44980d1ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:43.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:43.299+0000 7f4495600640 1 -- 172.21.15.131:0/2336717718 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4490007d50 con 0x7f44981f3b10 2024-05-26T22:53:43.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:43.299+0000 7f4495600640 1 -- 172.21.15.131:0/2336717718 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4490043070 con 0x7f44981f3b10 2024-05-26T22:53:43.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:43.299+0000 7f4495600640 1 -- 172.21.15.131:0/2336717718 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4490001c80 con 0x7f44981f3b10 2024-05-26T22:53:43.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:43.299+0000 7f449f000640 1 -- 172.21.15.131:0/2336717718 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4498145330 con 0x7f44981f3b10 2024-05-26T22:53:43.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:43.299+0000 7f449f000640 1 -- 172.21.15.131:0/2336717718 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44980c6360 con 0x7f44981f3b10 2024-05-26T22:53:43.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:43.300+0000 7f449f000640 1 -- 172.21.15.131:0/2336717718 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44980694e0 con 0x7f44981f3b10 2024-05-26T22:53:43.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:43.301+0000 7f4495600640 1 -- 172.21.15.131:0/2336717718 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f449003b040 con 0x7f44981f3b10 2024-05-26T22:53:43.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:43.302+0000 7f4495600640 1 --2- 172.21.15.131:0/2336717718 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f447c06be60 0x7f447c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:43.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:43.302+0000 7f4495600640 1 -- 172.21.15.131:0/2336717718 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5115+0+0 (secure 0 0 0) 0x7f44900aade0 con 0x7f44981f3b10 2024-05-26T22:53:43.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:43.302+0000 7f449dc00640 1 --2- 172.21.15.131:0/2336717718 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f447c06be60 0x7f447c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:43.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:43.303+0000 7f449dc00640 1 --2- 172.21.15.131:0/2336717718 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f447c06be60 0x7f447c06e320 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f448c0099d0 tx=0x7f448c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:43.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:43.306+0000 7f4495600640 1 -- 172.21.15.131:0/2336717718 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f44900799d0 con 0x7f44981f3b10 2024-05-26T22:53:43.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:43.578+0000 7f449f000640 1 -- 172.21.15.131:0/2336717718 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_exports.TestEphemeralPins.test_ephemeral_pin_dist_override"]} v 0) -- 0x7f44980ceaf0 con 0x7f44981f3b10 2024-05-26T22:53:44.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.319+0000 7f4495600640 1 -- 172.21.15.131:0/2336717718 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_exports.TestEphemeralPins.test_ephemeral_pin_dist_override"]}]=0 v277) ==== 155+0+0 (secure 0 0 0) 0x7f44900792d0 con 0x7f44981f3b10 2024-05-26T22:53:44.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.323+0000 7f449f000640 1 -- 172.21.15.131:0/2336717718 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f447c06be60 msgr2=0x7f447c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:44.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.323+0000 7f449f000640 1 --2- 172.21.15.131:0/2336717718 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f447c06be60 0x7f447c06e320 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f448c0099d0 tx=0x7f448c008040 comp rx=0 tx=0).stop 2024-05-26T22:53:44.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.323+0000 7f449f000640 1 -- 172.21.15.131:0/2336717718 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f44981f3b10 msgr2=0x7f44980c8120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:44.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.323+0000 7f449f000640 1 --2- 172.21.15.131:0/2336717718 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f44981f3b10 0x7f44980c8120 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f4490034040 tx=0x7f44980d1ca0 comp rx=0 tx=0).stop 2024-05-26T22:53:44.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.323+0000 7f449f000640 1 -- 172.21.15.131:0/2336717718 shutdown_connections 2024-05-26T22:53:44.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.323+0000 7f449f000640 1 --2- 172.21.15.131:0/2336717718 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4498144ea0 0x7f44980ad870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:44.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.323+0000 7f449f000640 1 --2- 172.21.15.131:0/2336717718 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f447c06be60 0x7f447c06e320 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:44.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.323+0000 7f449f000640 1 --2- 172.21.15.131:0/2336717718 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f44981f3b10 0x7f44980c8120 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:44.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.323+0000 7f449f000640 1 --2- 172.21.15.131:0/2336717718 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4480004d40 0x7f44980b0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:44.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.323+0000 7f449f000640 1 -- 172.21.15.131:0/2336717718 >> 172.21.15.131:0/2336717718 conn(0x7f4498070100 msgr2=0x7f449806bc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:44.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.323+0000 7f449f000640 1 -- 172.21.15.131:0/2336717718 shutdown_connections 2024-05-26T22:53:44.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.323+0000 7f449f000640 1 -- 172.21.15.131:0/2336717718 wait complete. 2024-05-26T22:53:44.345 INFO:tasks.cephfs_test_runner:That an ephemeral distributed pin overrides a normal export pin. ... ok 2024-05-26T22:53:44.345 INFO:tasks.cephfs_test_runner:Starting test: test_ephemeral_pin_dist_override_after (tasks.cephfs.test_exports.TestEphemeralPins) That a conventional export pin overrides the distributed policy _after_ distributed policy is set. 2024-05-26T22:53:44.345 INFO:tasks.cephfs_test_runner:test_ephemeral_pin_dist_override_after (tasks.cephfs.test_exports.TestEphemeralPins) 2024-05-26T22:53:44.346 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-26T22:53:44.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.541+0000 7f2defe00640 1 Processor -- start 2024-05-26T22:53:44.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.542+0000 7f2defe00640 1 -- start start 2024-05-26T22:53:44.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.543+0000 7f2defe00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2df0156750 0x7f2df015fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:44.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.543+0000 7f2defe00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f2df00cc7f0 0x7f2df00ccc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:44.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.543+0000 7f2defe00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2df0160620 con 0x7f2df006e8c0 2024-05-26T22:53:44.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.543+0000 7f2defe00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f2df015d770 con 0x7f2df00cc7f0 2024-05-26T22:53:44.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.543+0000 7f2defe00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f2df00cd1a0 con 0x7f2df0156750 2024-05-26T22:53:44.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.544+0000 7f2de6000640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f2df006e8c0 0x7f2df015d060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:40356/0 (socket says 172.21.15.131:40356) 2024-05-26T22:53:44.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.544+0000 7f2de6000640 1 -- 172.21.15.131:0/2872404049 learned_addr learned my addr 172.21.15.131:0/2872404049 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:53:44.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.544+0000 7f2def400640 1 --2- 172.21.15.131:0/2872404049 >> v2:172.21.15.131:3300/0 conn(0x7f2df00cc7f0 0x7f2df00ccc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:44.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.544+0000 7f2deea00640 1 --2- 172.21.15.131:0/2872404049 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2df0156750 0x7f2df015fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:44.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.544+0000 7f2dee000640 1 -- 172.21.15.131:0/2872404049 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2104050205 0 0) 0x7f2df0160620 con 0x7f2df006e8c0 2024-05-26T22:53:44.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.544+0000 7f2dee000640 1 -- 172.21.15.131:0/2872404049 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2dd4003520 con 0x7f2df006e8c0 2024-05-26T22:53:44.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.544+0000 7f2def400640 1 -- 172.21.15.131:0/2872404049 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2df0156750 msgr2=0x7f2df015fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:44.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.544+0000 7f2def400640 1 --2- 172.21.15.131:0/2872404049 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2df0156750 0x7f2df015fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:44.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.544+0000 7f2def400640 1 -- 172.21.15.131:0/2872404049 >> v1:172.21.15.131:6789/0 conn(0x7f2df006e8c0 legacy=0x7f2df015d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:44.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.544+0000 7f2def400640 1 -- 172.21.15.131:0/2872404049 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2df0160900 con 0x7f2df00cc7f0 2024-05-26T22:53:44.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.546+0000 7f2def400640 1 --2- 172.21.15.131:0/2872404049 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2df00cc7f0 0x7f2df00ccc60 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f2de8009d90 tx=0x7f2de802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edf128e7afa80f9f server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:44.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.546+0000 7f2dee000640 1 -- 172.21.15.131:0/2872404049 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de8007740 con 0x7f2df00cc7f0 2024-05-26T22:53:44.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.546+0000 7f2dee000640 1 -- 172.21.15.131:0/2872404049 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2de80078e0 con 0x7f2df00cc7f0 2024-05-26T22:53:44.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.546+0000 7f2dee000640 1 -- 172.21.15.131:0/2872404049 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de8007cb0 con 0x7f2df00cc7f0 2024-05-26T22:53:44.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.547+0000 7f2defe00640 1 -- 172.21.15.131:0/2872404049 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2df00cc7f0 msgr2=0x7f2df00ccc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:44.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.547+0000 7f2defe00640 1 --2- 172.21.15.131:0/2872404049 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2df00cc7f0 0x7f2df00ccc60 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f2de8009d90 tx=0x7f2de802a660 comp rx=0 tx=0).stop 2024-05-26T22:53:44.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.547+0000 7f2defe00640 1 -- 172.21.15.131:0/2872404049 shutdown_connections 2024-05-26T22:53:44.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.547+0000 7f2defe00640 1 --2- 172.21.15.131:0/2872404049 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2df00cc7f0 0x7f2df00ccc60 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:44.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.547+0000 7f2defe00640 1 --2- 172.21.15.131:0/2872404049 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2df0156750 0x7f2df015fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:44.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.547+0000 7f2defe00640 1 -- 172.21.15.131:0/2872404049 >> 172.21.15.131:0/2872404049 conn(0x7f2df00cff40 msgr2=0x7f2df00d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:44.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.547+0000 7f2defe00640 1 -- 172.21.15.131:0/2872404049 shutdown_connections 2024-05-26T22:53:44.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.547+0000 7f2defe00640 1 -- 172.21.15.131:0/2872404049 wait complete. 2024-05-26T22:53:44.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.548+0000 7f2defe00640 1 Processor -- start 2024-05-26T22:53:44.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.549+0000 7f2defe00640 1 -- start start 2024-05-26T22:53:44.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.549+0000 7f2defe00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2df006e8c0 0x7f2df01445a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:44.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.549+0000 7f2defe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2df00cc7f0 0x7f2df0143d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:44.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.550+0000 7f2defe00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2df0156750 0x7f2df00b0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:44.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.550+0000 7f2defe00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f2df015c860 con 0x7f2df00cc7f0 2024-05-26T22:53:44.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.550+0000 7f2defe00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2df015cbd0 con 0x7f2df006e8c0 2024-05-26T22:53:44.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.550+0000 7f2defe00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f2df0156b30 con 0x7f2df0156750 2024-05-26T22:53:44.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.550+0000 7f2de6000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2df00cc7f0 0x7f2df0143d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:44.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.550+0000 7f2de6000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2df00cc7f0 0x7f2df0143d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60124/0 (socket says 172.21.15.131:60124) 2024-05-26T22:53:44.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.550+0000 7f2de6000640 1 -- 172.21.15.131:0/1547596398 learned_addr learned my addr 172.21.15.131:0/1547596398 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:53:44.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.550+0000 7f2def400640 1 --2- 172.21.15.131:0/1547596398 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2df0156750 0x7f2df00b0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:44.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.550+0000 7f2deea00640 1 --2- 172.21.15.131:0/1547596398 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2df006e8c0 0x7f2df01445a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:44.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.550+0000 7f2de6000640 1 -- 172.21.15.131:0/1547596398 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2df0156750 msgr2=0x7f2df00b0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:44.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.550+0000 7f2de6000640 1 --2- 172.21.15.131:0/1547596398 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2df0156750 0x7f2df00b0ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:44.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.550+0000 7f2de6000640 1 -- 172.21.15.131:0/1547596398 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2df006e8c0 msgr2=0x7f2df01445a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:44.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.550+0000 7f2de6000640 1 --2- 172.21.15.131:0/1547596398 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2df006e8c0 0x7f2df01445a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:44.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.550+0000 7f2de6000640 1 -- 172.21.15.131:0/1547596398 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2df00adc20 con 0x7f2df00cc7f0 2024-05-26T22:53:44.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.551+0000 7f2de6000640 1 --2- 172.21.15.131:0/1547596398 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2df00cc7f0 0x7f2df0143d40 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f2dd800c430 tx=0x7f2dd800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:44.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.551+0000 7f2de7400640 1 -- 172.21.15.131:0/1547596398 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd8017070 con 0x7f2df00cc7f0 2024-05-26T22:53:44.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.551+0000 7f2de7400640 1 -- 172.21.15.131:0/1547596398 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2dd8005ce0 con 0x7f2df00cc7f0 2024-05-26T22:53:44.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.551+0000 7f2de7400640 1 -- 172.21.15.131:0/1547596398 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd80072f0 con 0x7f2df00cc7f0 2024-05-26T22:53:44.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.551+0000 7f2defe00640 1 -- 172.21.15.131:0/1547596398 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2df00adf00 con 0x7f2df00cc7f0 2024-05-26T22:53:44.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.551+0000 7f2defe00640 1 -- 172.21.15.131:0/1547596398 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2df0144280 con 0x7f2df00cc7f0 2024-05-26T22:53:44.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.552+0000 7f2defe00640 1 -- 172.21.15.131:0/1547596398 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2df015b600 con 0x7f2df00cc7f0 2024-05-26T22:53:44.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.554+0000 7f2de7400640 1 -- 172.21.15.131:0/1547596398 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2dd801a070 con 0x7f2df00cc7f0 2024-05-26T22:53:44.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.555+0000 7f2de7400640 1 --2- 172.21.15.131:0/1547596398 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f2dcc06be30 0x7f2dcc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:44.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.555+0000 7f2de7400640 1 -- 172.21.15.131:0/1547596398 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5115+0+0 (secure 0 0 0) 0x7f2dd808a1b0 con 0x7f2df00cc7f0 2024-05-26T22:53:44.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.555+0000 7f2deea00640 1 --2- 172.21.15.131:0/1547596398 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f2dcc06be30 0x7f2dcc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:44.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.555+0000 7f2deea00640 1 --2- 172.21.15.131:0/1547596398 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f2dcc06be30 0x7f2dcc06e2f0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f2de0006fd0 tx=0x7f2de0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:44.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.559+0000 7f2de7400640 1 -- 172.21.15.131:0/1547596398 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f2df015b600 con 0x7f2df00cc7f0 2024-05-26T22:53:44.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.754+0000 7f2defe00640 1 -- 172.21.15.131:0/1547596398 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2df00694e0 con 0x7f2df00cc7f0 2024-05-26T22:53:44.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.756+0000 7f2de7400640 1 -- 172.21.15.131:0/1547596398 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) ==== 74+0+13311 (secure 0 0 0) 0x7f2dd8058d10 con 0x7f2df00cc7f0 2024-05-26T22:53:44.757 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:53:44.757 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":74,"fsid":"46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f","created":"2024-05-26T22:47:57.288905+0000","modified":"2024-05-26T22:53:41.289287+0000","last_up_change":"2024-05-26T22:48:09.602207+0000","last_in_change":"2024-05-26T22:48:00.687341+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-26T22:48:10.463339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"814eaf62-a18d-494b-8e27-fd8fb88adea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6801","nonce":2350940051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6803","nonce":2350940051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6807","nonce":2350940051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6805","nonce":2350940051}]},"public_addr":"172.21.15.131:6801/2350940051","cluster_addr":"172.21.15.131:6803/2350940051","heartbeat_back_addr":"172.21.15.131:6807/2350940051","heartbeat_front_addr":"172.21.15.131:6805/2350940051","state":["exists","up"]},{"osd":1,"uuid":"42e66287-f6c6-4e26-af3c-6dfaa3536d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6825","nonce":1525528247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6827","nonce":1525528247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6831","nonce":1525528247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6829","nonce":1525528247}]},"public_addr":"172.21.15.131:6825/1525528247","cluster_addr":"172.21.15.131:6827/1525528247","heartbeat_back_addr":"172.21.15.131:6831/1525528247","heartbeat_front_addr":"172.21.15.131:6829/1525528247","state":["exists","up"]},{"osd":2,"uuid":"0da34673-04a2-4fd0-b57a-0ff2b679b043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6817","nonce":3627842210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6819","nonce":3627842210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6823","nonce":3627842210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6821","nonce":3627842210}]},"public_addr":"172.21.15.131:6817/3627842210","cluster_addr":"172.21.15.131:6819/3627842210","heartbeat_back_addr":"172.21.15.131:6823/3627842210","heartbeat_front_addr":"172.21.15.131:6821/3627842210","state":["exists","up"]},{"osd":3,"uuid":"cf117a27-d6b2-43d3-ae52-b47988bf8eed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6809","nonce":350790591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6811","nonce":350790591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6815","nonce":350790591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6813","nonce":350790591}]},"public_addr":"172.21.15.131:6809/350790591","cluster_addr":"172.21.15.131:6811/350790591","heartbeat_back_addr":"172.21.15.131:6815/350790591","heartbeat_front_addr":"172.21.15.131:6813/350790591","state":["exists","up"]},{"osd":4,"uuid":"a092ccfc-be89-4a84-88d5-0f781fa715eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6816","nonce":244560995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6819","nonce":244560995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6823","nonce":244560995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6821","nonce":244560995}]},"public_addr":"172.21.15.187:6816/244560995","cluster_addr":"172.21.15.187:6819/244560995","heartbeat_back_addr":"172.21.15.187:6823/244560995","heartbeat_front_addr":"172.21.15.187:6821/244560995","state":["exists","up"]},{"osd":5,"uuid":"7513ba69-dd00-4a35-b190-9e8cc3b2bdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6801","nonce":324619398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6803","nonce":324619398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6812","nonce":324619398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6807","nonce":324619398}]},"public_addr":"172.21.15.187:6801/324619398","cluster_addr":"172.21.15.187:6803/324619398","heartbeat_back_addr":"172.21.15.187:6812/324619398","heartbeat_front_addr":"172.21.15.187:6807/324619398","state":["exists","up"]},{"osd":6,"uuid":"50081810-c093-4d2f-a4d7-b7c695c5400d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6806","nonce":1391347786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6809","nonce":1391347786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6818","nonce":1391347786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6813","nonce":1391347786}]},"public_addr":"172.21.15.187:6806/1391347786","cluster_addr":"172.21.15.187:6809/1391347786","heartbeat_back_addr":"172.21.15.187:6818/1391347786","heartbeat_front_addr":"172.21.15.187:6813/1391347786","state":["exists","up"]},{"osd":7,"uuid":"775dae05-54f5-4ad4-bbce-92863c6ed2ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6825","nonce":1220065622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6827","nonce":1220065622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6831","nonce":1220065622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6829","nonce":1220065622}]},"public_addr":"172.21.15.187:6825/1220065622","cluster_addr":"172.21.15.187:6827/1220065622","heartbeat_back_addr":"172.21.15.187:6831/1220065622","heartbeat_front_addr":"172.21.15.187:6829/1220065622","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:05.983520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.299516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.067323+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.307748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.018742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.258775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.010912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.295355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.187:6834/3680999672":"2024-05-27T22:53:37.839278+0000","172.21.15.131:6836/4223534499":"2024-05-27T22:53:37.839240+0000","172.21.15.131:6834/4223534499":"2024-05-27T22:53:37.839240+0000","172.21.15.131:6837/1418741551":"2024-05-27T22:53:37.839264+0000","172.21.15.187:6832/3680999672":"2024-05-27T22:53:37.839278+0000","172.21.15.131:6835/1418741551":"2024-05-27T22:53:37.839264+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-26T22:53:44.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.758+0000 7f2defe00640 1 -- 172.21.15.131:0/1547596398 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f2dcc06be30 msgr2=0x7f2dcc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:44.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.758+0000 7f2defe00640 1 --2- 172.21.15.131:0/1547596398 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f2dcc06be30 0x7f2dcc06e2f0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f2de0006fd0 tx=0x7f2de0008040 comp rx=0 tx=0).stop 2024-05-26T22:53:44.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.759+0000 7f2defe00640 1 -- 172.21.15.131:0/1547596398 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2df00cc7f0 msgr2=0x7f2df0143d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:44.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.759+0000 7f2defe00640 1 --2- 172.21.15.131:0/1547596398 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2df00cc7f0 0x7f2df0143d40 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f2dd800c430 tx=0x7f2dd800c8f0 comp rx=0 tx=0).stop 2024-05-26T22:53:44.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.759+0000 7f2defe00640 1 -- 172.21.15.131:0/1547596398 shutdown_connections 2024-05-26T22:53:44.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.759+0000 7f2defe00640 1 --2- 172.21.15.131:0/1547596398 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2df0156750 0x7f2df00b0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:44.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.759+0000 7f2defe00640 1 --2- 172.21.15.131:0/1547596398 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f2dcc06be30 0x7f2dcc06e2f0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:44.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.759+0000 7f2defe00640 1 --2- 172.21.15.131:0/1547596398 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2df00cc7f0 0x7f2df0143d40 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:44.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.759+0000 7f2defe00640 1 --2- 172.21.15.131:0/1547596398 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2df006e8c0 0x7f2df01445a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:44.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.759+0000 7f2defe00640 1 -- 172.21.15.131:0/1547596398 >> 172.21.15.131:0/1547596398 conn(0x7f2df00cff40 msgr2=0x7f2df00d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:44.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.759+0000 7f2defe00640 1 -- 172.21.15.131:0/1547596398 shutdown_connections 2024-05-26T22:53:44.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.759+0000 7f2defe00640 1 -- 172.21.15.131:0/1547596398 wait complete. 2024-05-26T22:53:44.780 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-26T22:48:10.463339+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-26T22:53:44.780 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-26T22:53:44.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.982+0000 7f5ffbe00640 1 Processor -- start 2024-05-26T22:53:44.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.983+0000 7f5ffbe00640 1 -- start start 2024-05-26T22:53:44.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.984+0000 7f5ffbe00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f5ff406f420 0x7f5ff40cfd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:44.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.984+0000 7f5ffbe00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f5ff40cf590 0x7f5ff40d0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:44.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.984+0000 7f5ffbe00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ff4160180 con 0x7f5ff40b5450 2024-05-26T22:53:44.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.984+0000 7f5ffbe00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f5ff4160d90 con 0x7f5ff40cf590 2024-05-26T22:53:44.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.984+0000 7f5ffbe00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f5ff4160460 con 0x7f5ff406f420 2024-05-26T22:53:44.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.985+0000 7f5ff9400640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f5ff40b5450 0x7f5ff40b43c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:40370/0 (socket says 172.21.15.131:40370) 2024-05-26T22:53:44.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.985+0000 7f5ff9400640 1 -- 172.21.15.131:0/3946001939 learned_addr learned my addr 172.21.15.131:0/3946001939 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:53:44.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.985+0000 7f5ff9e00640 1 --2- 172.21.15.131:0/3946001939 >> v2:172.21.15.131:3300/0 conn(0x7f5ff40cf590 0x7f5ff40d0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:44.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.985+0000 7f5ff8a00640 1 --2- 172.21.15.131:0/3946001939 >> v2:172.21.15.187:3300/0 conn(0x7f5ff406f420 0x7f5ff40cfd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:44.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.985+0000 7f5ff3e00640 1 -- 172.21.15.131:0/3946001939 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 730756259 0 0) 0x7f5ff4160180 con 0x7f5ff40b5450 2024-05-26T22:53:44.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.985+0000 7f5ff3e00640 1 -- 172.21.15.131:0/3946001939 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5fdc003520 con 0x7f5ff40b5450 2024-05-26T22:53:44.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.985+0000 7f5ff9e00640 1 -- 172.21.15.131:0/3946001939 >> v2:172.21.15.187:3300/0 conn(0x7f5ff406f420 msgr2=0x7f5ff40cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:44.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.985+0000 7f5ff9e00640 1 --2- 172.21.15.131:0/3946001939 >> v2:172.21.15.187:3300/0 conn(0x7f5ff406f420 0x7f5ff40cfd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:44.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.985+0000 7f5ff9e00640 1 -- 172.21.15.131:0/3946001939 >> v1:172.21.15.131:6789/0 conn(0x7f5ff40b5450 legacy=0x7f5ff40b43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:44.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.985+0000 7f5ff9e00640 1 -- 172.21.15.131:0/3946001939 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ff4160740 con 0x7f5ff40cf590 2024-05-26T22:53:44.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.985+0000 7f5ff8a00640 1 --2- 172.21.15.131:0/3946001939 >> v2:172.21.15.187:3300/0 conn(0x7f5ff406f420 0x7f5ff40cfd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:53:44.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.986+0000 7f5ff9e00640 1 --2- 172.21.15.131:0/3946001939 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ff40cf590 0x7f5ff40d0290 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f5ff40b35b0 tx=0x7f5fec02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbc64a797f71258b server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:44.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.987+0000 7f5ff3e00640 1 -- 172.21.15.131:0/3946001939 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fec007740 con 0x7f5ff40cf590 2024-05-26T22:53:44.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.987+0000 7f5ff3e00640 1 -- 172.21.15.131:0/3946001939 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fec0078e0 con 0x7f5ff40cf590 2024-05-26T22:53:44.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.987+0000 7f5ff3e00640 1 -- 172.21.15.131:0/3946001939 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fec007c90 con 0x7f5ff40cf590 2024-05-26T22:53:44.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.987+0000 7f5ffbe00640 1 -- 172.21.15.131:0/3946001939 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ff40cf590 msgr2=0x7f5ff40d0290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:44.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.987+0000 7f5ffbe00640 1 --2- 172.21.15.131:0/3946001939 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ff40cf590 0x7f5ff40d0290 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f5ff40b35b0 tx=0x7f5fec02a680 comp rx=0 tx=0).stop 2024-05-26T22:53:44.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.987+0000 7f5ffbe00640 1 -- 172.21.15.131:0/3946001939 shutdown_connections 2024-05-26T22:53:44.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.987+0000 7f5ffbe00640 1 --2- 172.21.15.131:0/3946001939 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ff40cf590 0x7f5ff40d0290 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:44.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.987+0000 7f5ffbe00640 1 --2- 172.21.15.131:0/3946001939 >> v2:172.21.15.187:3300/0 conn(0x7f5ff406f420 0x7f5ff40cfd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:44.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.987+0000 7f5ffbe00640 1 -- 172.21.15.131:0/3946001939 >> 172.21.15.131:0/3946001939 conn(0x7f5ff406bc90 msgr2=0x7f5ff40b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:44.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.988+0000 7f5ffbe00640 1 -- 172.21.15.131:0/3946001939 shutdown_connections 2024-05-26T22:53:44.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.988+0000 7f5ffbe00640 1 -- 172.21.15.131:0/3946001939 wait complete. 2024-05-26T22:53:44.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.988+0000 7f5ffbe00640 1 Processor -- start 2024-05-26T22:53:44.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.990+0000 7f5ffbe00640 1 -- start start 2024-05-26T22:53:44.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.990+0000 7f5ffbe00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5ff406f420 0x7f5ff40728c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:44.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.990+0000 7f5ffbe00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5ff40b5450 0x7f5ff4145280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:44.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.990+0000 7f5ffbe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ff40cf590 0x7f5ff41f7840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:44.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.990+0000 7f5ffbe00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff415fde0 con 0x7f5ff40cf590 2024-05-26T22:53:44.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.990+0000 7f5ffbe00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff40b5830 con 0x7f5ff406f420 2024-05-26T22:53:44.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.990+0000 7f5ffbe00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f5ff415ff60 con 0x7f5ff40b5450 2024-05-26T22:53:44.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.991+0000 7f5ff9e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ff40cf590 0x7f5ff41f7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:44.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.991+0000 7f5ff9e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ff40cf590 0x7f5ff41f7840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60146/0 (socket says 172.21.15.131:60146) 2024-05-26T22:53:44.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.991+0000 7f5ff9e00640 1 -- 172.21.15.131:0/570782632 learned_addr learned my addr 172.21.15.131:0/570782632 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:53:44.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.991+0000 7f5ff9400640 1 --2- 172.21.15.131:0/570782632 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5ff406f420 0x7f5ff40728c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:44.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.991+0000 7f5ff8a00640 1 --2- 172.21.15.131:0/570782632 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5ff40b5450 0x7f5ff4145280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:44.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.991+0000 7f5ff9e00640 1 -- 172.21.15.131:0/570782632 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5ff40b5450 msgr2=0x7f5ff4145280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:44.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.991+0000 7f5ff9e00640 1 --2- 172.21.15.131:0/570782632 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5ff40b5450 0x7f5ff4145280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:44.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.991+0000 7f5ff9e00640 1 -- 172.21.15.131:0/570782632 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5ff406f420 msgr2=0x7f5ff40728c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:44.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.991+0000 7f5ff9e00640 1 --2- 172.21.15.131:0/570782632 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5ff406f420 0x7f5ff40728c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:44.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.991+0000 7f5ff9e00640 1 -- 172.21.15.131:0/570782632 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ff4151600 con 0x7f5ff40cf590 2024-05-26T22:53:44.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.991+0000 7f5ff8a00640 1 --2- 172.21.15.131:0/570782632 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5ff40b5450 0x7f5ff4145280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:53:44.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.992+0000 7f5ff9e00640 1 --2- 172.21.15.131:0/570782632 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ff40cf590 0x7f5ff41f7840 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f5fec02ac70 tx=0x7f5fec004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:44.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.992+0000 7f5ff1600640 1 -- 172.21.15.131:0/570782632 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fec040070 con 0x7f5ff40cf590 2024-05-26T22:53:44.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.992+0000 7f5ff1600640 1 -- 172.21.15.131:0/570782632 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fec031c60 con 0x7f5ff40cf590 2024-05-26T22:53:44.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.992+0000 7f5ff1600640 1 -- 172.21.15.131:0/570782632 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fec009450 con 0x7f5ff40cf590 2024-05-26T22:53:44.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.992+0000 7f5ffbe00640 1 -- 172.21.15.131:0/570782632 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ff41457c0 con 0x7f5ff40cf590 2024-05-26T22:53:44.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.992+0000 7f5ffbe00640 1 -- 172.21.15.131:0/570782632 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ff4071b00 con 0x7f5ff40cf590 2024-05-26T22:53:44.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.993+0000 7f5ffbe00640 1 -- 172.21.15.131:0/570782632 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ff40695c0 con 0x7f5ff40cf590 2024-05-26T22:53:44.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.994+0000 7f5ff1600640 1 -- 172.21.15.131:0/570782632 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5fec039040 con 0x7f5ff40cf590 2024-05-26T22:53:44.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.995+0000 7f5ff1600640 1 --2- 172.21.15.131:0/570782632 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f5fd806be30 0x7f5fd806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:44.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.995+0000 7f5ff1600640 1 -- 172.21.15.131:0/570782632 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5115+0+0 (secure 0 0 0) 0x7f5fec0ac0d0 con 0x7f5ff40cf590 2024-05-26T22:53:44.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.995+0000 7f5ff9400640 1 --2- 172.21.15.131:0/570782632 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f5fd806be30 0x7f5fd806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:44.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.996+0000 7f5ff9400640 1 --2- 172.21.15.131:0/570782632 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f5fd806be30 0x7f5fd806e2f0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f5fe8005880 tx=0x7f5fe8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:44.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:44.999+0000 7f5ff1600640 1 -- 172.21.15.131:0/570782632 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f5ff4151600 con 0x7f5ff40cf590 2024-05-26T22:53:45.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:45.196+0000 7f5ffbe00640 1 -- 172.21.15.131:0/570782632 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5ff415ea60 con 0x7f5ff40cf590 2024-05-26T22:53:45.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:45.197+0000 7f5ff1600640 1 -- 172.21.15.131:0/570782632 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v74) ==== 93+0+10 (secure 0 0 0) 0x7f5fec038350 con 0x7f5ff40cf590 2024-05-26T22:53:45.197 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2024-05-26T22:53:45.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:45.200+0000 7f5ffbe00640 1 -- 172.21.15.131:0/570782632 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f5fd806be30 msgr2=0x7f5fd806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:45.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:45.200+0000 7f5ffbe00640 1 --2- 172.21.15.131:0/570782632 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f5fd806be30 0x7f5fd806e2f0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f5fe8005880 tx=0x7f5fe8009040 comp rx=0 tx=0).stop 2024-05-26T22:53:45.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:45.200+0000 7f5ffbe00640 1 -- 172.21.15.131:0/570782632 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ff40cf590 msgr2=0x7f5ff41f7840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:45.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:45.200+0000 7f5ffbe00640 1 --2- 172.21.15.131:0/570782632 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ff40cf590 0x7f5ff41f7840 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f5fec02ac70 tx=0x7f5fec004a40 comp rx=0 tx=0).stop 2024-05-26T22:53:45.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:45.200+0000 7f5ffbe00640 1 -- 172.21.15.131:0/570782632 shutdown_connections 2024-05-26T22:53:45.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:45.200+0000 7f5ffbe00640 1 --2- 172.21.15.131:0/570782632 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5ff40b5450 0x7f5ff4145280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:45.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:45.200+0000 7f5ffbe00640 1 --2- 172.21.15.131:0/570782632 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f5fd806be30 0x7f5fd806e2f0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:45.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:45.200+0000 7f5ffbe00640 1 --2- 172.21.15.131:0/570782632 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ff40cf590 0x7f5ff41f7840 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:45.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:45.200+0000 7f5ffbe00640 1 --2- 172.21.15.131:0/570782632 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5ff406f420 0x7f5ff40728c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:45.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:45.200+0000 7f5ffbe00640 1 -- 172.21.15.131:0/570782632 >> 172.21.15.131:0/570782632 conn(0x7f5ff406bc90 msgr2=0x7f5ff40b43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:45.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:45.200+0000 7f5ffbe00640 1 -- 172.21.15.131:0/570782632 shutdown_connections 2024-05-26T22:53:45.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:45.200+0000 7f5ffbe00640 1 -- 172.21.15.131:0/570782632 wait complete. 2024-05-26T22:53:45.219 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_exports.TestEphemeralPins.test_ephemeral_pin_dist_override_after' 2024-05-26T22:53:45.235 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-26T22:53:45.271 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-26T22:53:45.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:45.414+0000 7f63b9e00640 1 Processor -- start 2024-05-26T22:53:45.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:45.415+0000 7f63b9e00640 1 -- start start 2024-05-26T22:53:45.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:45.416+0000 7f63b9e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f63b415c3d0 0x7f63b406ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:45.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:45.416+0000 7f63b9e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f63b406f490 0x7f63b40ade40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:45.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:45.416+0000 7f63b9e00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63b40722a0 con 0x7f63b4155840 2024-05-26T22:53:45.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:45.416+0000 7f63b9e00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f63b4072040 con 0x7f63b406f490 2024-05-26T22:53:45.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:45.416+0000 7f63b9e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f63b4072580 con 0x7f63b415c3d0 2024-05-26T22:53:45.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:45.416+0000 7f63b3400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f63b406f490 0x7f63b40ade40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:45.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:45.416+0000 7f63b2a00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f63b4155840 0x7f63b4071930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:40382/0 (socket says 172.21.15.131:40382) 2024-05-26T22:53:45.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:45.416+0000 7f63b2a00640 1 -- 172.21.15.131:0/2450373768 learned_addr learned my addr 172.21.15.131:0/2450373768 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:53:45.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:45.416+0000 7f63b2000640 1 --2- 172.21.15.131:0/2450373768 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f63b415c3d0 0x7f63b406ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:45.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:45.417+0000 7f63b3400640 1 -- 172.21.15.131:0/2450373768 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f63b415c3d0 msgr2=0x7f63b406ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:45.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:45.417+0000 7f63b3400640 1 --2- 172.21.15.131:0/2450373768 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f63b415c3d0 0x7f63b406ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:45.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:45.417+0000 7f63b3400640 1 -- 172.21.15.131:0/2450373768 >> v1:172.21.15.131:6789/0 conn(0x7f63b4155840 legacy=0x7f63b4071930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:45.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:45.417+0000 7f63b3400640 1 -- 172.21.15.131:0/2450373768 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63b4072860 con 0x7f63b406f490 2024-05-26T22:53:45.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:45.417+0000 7f63b1600640 1 -- 172.21.15.131:0/2450373768 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1921961334 0 0) 0x7f63b40722a0 con 0x7f63b4155840 2024-05-26T22:53:45.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:45.418+0000 7f63b3400640 1 --2- 172.21.15.131:0/2450373768 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63b406f490 0x7f63b40ade40 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f63ac0097a0 tx=0x7f63ac02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df39b7930fa3009a server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:45.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:45.418+0000 7f63b1600640 1 -- 172.21.15.131:0/2450373768 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63ac037070 con 0x7f63b406f490 2024-05-26T22:53:45.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:45.418+0000 7f63b1600640 1 -- 172.21.15.131:0/2450373768 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63ac032440 con 0x7f63b406f490 2024-05-26T22:53:45.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:45.418+0000 7f63b1600640 1 -- 172.21.15.131:0/2450373768 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63ac032810 con 0x7f63b406f490 2024-05-26T22:53:45.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:45.419+0000 7f63b9e00640 1 -- 172.21.15.131:0/2450373768 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63b406f490 msgr2=0x7f63b40ade40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:45.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:45.419+0000 7f63b9e00640 1 --2- 172.21.15.131:0/2450373768 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63b406f490 0x7f63b40ade40 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f63ac0097a0 tx=0x7f63ac02a180 comp rx=0 tx=0).stop 2024-05-26T22:53:45.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:45.419+0000 7f63b9e00640 1 -- 172.21.15.131:0/2450373768 shutdown_connections 2024-05-26T22:53:45.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:45.419+0000 7f63b9e00640 1 --2- 172.21.15.131:0/2450373768 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63b406f490 0x7f63b40ade40 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:45.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:45.419+0000 7f63b9e00640 1 --2- 172.21.15.131:0/2450373768 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f63b415c3d0 0x7f63b406ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:45.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:45.419+0000 7f63b9e00640 1 -- 172.21.15.131:0/2450373768 >> 172.21.15.131:0/2450373768 conn(0x7f63b40b1e90 msgr2=0x7f63b40b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:45.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:45.419+0000 7f63b9e00640 1 -- 172.21.15.131:0/2450373768 shutdown_connections 2024-05-26T22:53:45.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:45.419+0000 7f63b9e00640 1 -- 172.21.15.131:0/2450373768 wait complete. 2024-05-26T22:53:45.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:45.420+0000 7f63b9e00640 1 Processor -- start 2024-05-26T22:53:45.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:45.421+0000 7f63b9e00640 1 -- start start 2024-05-26T22:53:45.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:45.422+0000 7f63b9e00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f63b406f490 0x7f63b40cef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:45.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:45.422+0000 7f63b9e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63b4155840 0x7f63b40b4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:45.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:45.422+0000 7f63b9e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f63b415c3d0 0x7f63b41f41f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:45.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:45.422+0000 7f63b9e00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f63b40ae380 con 0x7f63b4155840 2024-05-26T22:53:45.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:45.422+0000 7f63b9e00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f63b4072e20 con 0x7f63b406f490 2024-05-26T22:53:45.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:45.422+0000 7f63b9e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f63b40ae500 con 0x7f63b415c3d0 2024-05-26T22:53:45.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:45.422+0000 7f63b2000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63b4155840 0x7f63b40b4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:45.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:45.422+0000 7f63b2000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63b4155840 0x7f63b40b4050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60166/0 (socket says 172.21.15.131:60166) 2024-05-26T22:53:45.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:45.422+0000 7f63b2000640 1 -- 172.21.15.131:0/3020019041 learned_addr learned my addr 172.21.15.131:0/3020019041 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:53:45.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:45.422+0000 7f63b3400640 1 --2- 172.21.15.131:0/3020019041 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f63b415c3d0 0x7f63b41f41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:45.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:45.422+0000 7f63b2a00640 1 --2- 172.21.15.131:0/3020019041 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f63b406f490 0x7f63b40cef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:45.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:45.422+0000 7f63b2000640 1 -- 172.21.15.131:0/3020019041 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f63b415c3d0 msgr2=0x7f63b41f41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:45.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:45.422+0000 7f63b2000640 1 --2- 172.21.15.131:0/3020019041 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f63b415c3d0 0x7f63b41f41f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:45.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:45.422+0000 7f63b2000640 1 -- 172.21.15.131:0/3020019041 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f63b406f490 msgr2=0x7f63b40cef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:45.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:45.422+0000 7f63b2000640 1 --2- 172.21.15.131:0/3020019041 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f63b406f490 0x7f63b40cef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:45.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:45.422+0000 7f63b2000640 1 -- 172.21.15.131:0/3020019041 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63b4154750 con 0x7f63b4155840 2024-05-26T22:53:45.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:45.422+0000 7f63b2a00640 1 --2- 172.21.15.131:0/3020019041 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f63b406f490 0x7f63b40cef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:53:45.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:45.423+0000 7f63b2000640 1 --2- 172.21.15.131:0/3020019041 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63b4155840 0x7f63b40b4050 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f639c00d5f0 tx=0x7f639c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:45.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:45.423+0000 7f63ab400640 1 -- 172.21.15.131:0/3020019041 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f639c0125f0 con 0x7f63b4155840 2024-05-26T22:53:45.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:45.423+0000 7f63ab400640 1 -- 172.21.15.131:0/3020019041 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f639c019070 con 0x7f63b4155840 2024-05-26T22:53:45.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:45.423+0000 7f63ab400640 1 -- 172.21.15.131:0/3020019041 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f639c0129d0 con 0x7f63b4155840 2024-05-26T22:53:45.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:45.423+0000 7f63b9e00640 1 -- 172.21.15.131:0/3020019041 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63b41f7770 con 0x7f63b4155840 2024-05-26T22:53:45.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:45.423+0000 7f63b9e00640 1 -- 172.21.15.131:0/3020019041 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63b41f7d30 con 0x7f63b4155840 2024-05-26T22:53:45.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:45.424+0000 7f63b9e00640 1 -- 172.21.15.131:0/3020019041 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63b40694e0 con 0x7f63b4155840 2024-05-26T22:53:45.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:45.425+0000 7f63ab400640 1 -- 172.21.15.131:0/3020019041 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f639c012b70 con 0x7f63b4155840 2024-05-26T22:53:45.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:45.426+0000 7f63ab400640 1 --2- 172.21.15.131:0/3020019041 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f639006be30 0x7f639006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:45.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:45.426+0000 7f63ab400640 1 -- 172.21.15.131:0/3020019041 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5115+0+0 (secure 0 0 0) 0x7f639c08b0c0 con 0x7f63b4155840 2024-05-26T22:53:45.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:45.426+0000 7f63b2a00640 1 --2- 172.21.15.131:0/3020019041 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f639006be30 0x7f639006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:45.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:45.427+0000 7f63b2a00640 1 --2- 172.21.15.131:0/3020019041 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f639006be30 0x7f639006e2f0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f63a40012e0 tx=0x7f63a4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:45.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:45.431+0000 7f63ab400640 1 -- 172.21.15.131:0/3020019041 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f639c059c20 con 0x7f63b4155840 2024-05-26T22:53:45.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:45.684+0000 7f63b9e00640 1 -- 172.21.15.131:0/3020019041 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_exports.TestEphemeralPins.test_ephemeral_pin_dist_override_after"]} v 0) -- 0x7f63b406c590 con 0x7f63b4155840 2024-05-26T22:53:46.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:46.329+0000 7f63ab400640 1 -- 172.21.15.131:0/3020019041 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_exports.TestEphemeralPins.test_ephemeral_pin_dist_override_after"]}]=0 v279) ==== 164+0+0 (secure 0 0 0) 0x7f639c059520 con 0x7f63b4155840 2024-05-26T22:53:46.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:46.333+0000 7f63b9e00640 1 -- 172.21.15.131:0/3020019041 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f639006be30 msgr2=0x7f639006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:46.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:46.333+0000 7f63b9e00640 1 --2- 172.21.15.131:0/3020019041 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f639006be30 0x7f639006e2f0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f63a40012e0 tx=0x7f63a4009040 comp rx=0 tx=0).stop 2024-05-26T22:53:46.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:46.333+0000 7f63b9e00640 1 -- 172.21.15.131:0/3020019041 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63b4155840 msgr2=0x7f63b40b4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:46.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:46.333+0000 7f63b9e00640 1 --2- 172.21.15.131:0/3020019041 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63b4155840 0x7f63b40b4050 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f639c00d5f0 tx=0x7f639c00dab0 comp rx=0 tx=0).stop 2024-05-26T22:53:46.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:46.333+0000 7f63b9e00640 1 -- 172.21.15.131:0/3020019041 shutdown_connections 2024-05-26T22:53:46.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:46.333+0000 7f63b9e00640 1 --2- 172.21.15.131:0/3020019041 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f63b415c3d0 0x7f63b41f41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:46.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:46.333+0000 7f63b9e00640 1 --2- 172.21.15.131:0/3020019041 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f639006be30 0x7f639006e2f0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:46.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:46.333+0000 7f63b9e00640 1 --2- 172.21.15.131:0/3020019041 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63b4155840 0x7f63b40b4050 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:46.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:46.333+0000 7f63b9e00640 1 --2- 172.21.15.131:0/3020019041 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f63b406f490 0x7f63b40cef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:46.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:46.333+0000 7f63b9e00640 1 -- 172.21.15.131:0/3020019041 >> 172.21.15.131:0/3020019041 conn(0x7f63b40b1e90 msgr2=0x7f63b40b0870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:46.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:46.333+0000 7f63b9e00640 1 -- 172.21.15.131:0/3020019041 shutdown_connections 2024-05-26T22:53:46.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:46.333+0000 7f63b9e00640 1 -- 172.21.15.131:0/3020019041 wait complete. 2024-05-26T22:53:46.354 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-26T22:53:46.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:46.601+0000 7fc388c00640 1 Processor -- start 2024-05-26T22:53:46.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:46.601+0000 7fc388c00640 1 -- start start 2024-05-26T22:53:46.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:46.602+0000 7fc388c00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc38415b730 0x7fc38415bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:46.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:46.602+0000 7fc388c00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc38413e820 con 0x7fc3841400c0 2024-05-26T22:53:46.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:46.602+0000 7fc388c00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc38415c4a0 con 0x7fc38406d250 2024-05-26T22:53:46.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:46.602+0000 7fc388c00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc384071260 con 0x7fc38415b730 2024-05-26T22:53:46.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:46.603+0000 7fc383e00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fc3841400c0 0x7fc384140510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:40384/0 (socket says 172.21.15.131:40384) 2024-05-26T22:53:46.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:46.603+0000 7fc383e00640 1 -- 172.21.15.131:0/2525214429 learned_addr learned my addr 172.21.15.131:0/2525214429 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:53:46.604 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:46.603+0000 7fc37be00640 1 --2- 172.21.15.131:0/2525214429 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc38415b730 0x7fc38415bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:46.604 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:46.603+0000 7fc382a00640 1 -- 172.21.15.131:0/2525214429 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3767303300 0 0) 0x7fc38413e820 con 0x7fc3841400c0 2024-05-26T22:53:46.604 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:46.603+0000 7fc382a00640 1 -- 172.21.15.131:0/2525214429 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc368003520 con 0x7fc3841400c0 2024-05-26T22:53:46.604 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:46.603+0000 7fc382a00640 1 -- 172.21.15.131:0/2525214429 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3242612967 0 0) 0x7fc368003520 con 0x7fc3841400c0 2024-05-26T22:53:46.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:46.603+0000 7fc382a00640 1 -- 172.21.15.131:0/2525214429 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc38415b730 msgr2=0x7fc38415bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:46.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:46.603+0000 7fc382a00640 1 --2- 172.21.15.131:0/2525214429 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc38415b730 0x7fc38415bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:46.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:46.603+0000 7fc382a00640 1 -- 172.21.15.131:0/2525214429 >> v1:172.21.15.187:6789/0 conn(0x7fc38406d250 legacy=0x7fc38406d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:46.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:46.604+0000 7fc382a00640 1 -- 172.21.15.131:0/2525214429 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3840714b0 con 0x7fc3841400c0 2024-05-26T22:53:46.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:46.604+0000 7fc382a00640 1 -- 172.21.15.131:0/2525214429 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7fc37c0019f0 con 0x7fc3841400c0 2024-05-26T22:53:46.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:46.604+0000 7fc382a00640 1 -- 172.21.15.131:0/2525214429 >> v1:172.21.15.131:6789/0 conn(0x7fc3841400c0 legacy=0x7fc384140510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:46.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:46.604+0000 7fc382a00640 1 --2- 172.21.15.131:0/2525214429 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc3680039e0 0x7fc368003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:46.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:46.604+0000 7fc382a00640 1 --2- 172.21.15.131:0/2525214429 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc38415b730 0x7fc368004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:46.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:46.604+0000 7fc382a00640 1 --2- 172.21.15.131:0/2525214429 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc368004d40 0x7fc3680051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:46.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:46.604+0000 7fc382a00640 1 -- 172.21.15.131:0/2525214429 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc384071260 con 0x7fc368004d40 2024-05-26T22:53:46.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:46.604+0000 7fc382a00640 1 -- 172.21.15.131:0/2525214429 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc36800dce0 con 0x7fc3680039e0 2024-05-26T22:53:46.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:46.604+0000 7fc382a00640 1 -- 172.21.15.131:0/2525214429 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc36800dee0 con 0x7fc38415b730 2024-05-26T22:53:46.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:46.604+0000 7fc383e00640 1 --2- 172.21.15.131:0/2525214429 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc368004d40 0x7fc3680051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:46.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:46.604+0000 7fc383400640 1 --2- 172.21.15.131:0/2525214429 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc3680039e0 0x7fc368003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:46.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:46.604+0000 7fc37be00640 1 --2- 172.21.15.131:0/2525214429 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc38415b730 0x7fc368004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:46.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:46.604+0000 7fc383e00640 1 -- 172.21.15.131:0/2525214429 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc38415b730 msgr2=0x7fc368004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:46.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:46.604+0000 7fc383e00640 1 --2- 172.21.15.131:0/2525214429 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc38415b730 0x7fc368004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:46.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:46.604+0000 7fc383e00640 1 -- 172.21.15.131:0/2525214429 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc3680039e0 msgr2=0x7fc368003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:46.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:46.604+0000 7fc383e00640 1 --2- 172.21.15.131:0/2525214429 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc3680039e0 0x7fc368003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:46.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:46.604+0000 7fc383e00640 1 -- 172.21.15.131:0/2525214429 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc368003520 con 0x7fc368004d40 2024-05-26T22:53:46.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:46.605+0000 7fc383e00640 1 --2- 172.21.15.131:0/2525214429 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc368004d40 0x7fc3680051c0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fc37c00a740 tx=0x7fc368005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c55621f61e4a0e04 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:46.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:46.606+0000 7fc382a00640 1 -- 172.21.15.131:0/2525214429 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc37c00a390 con 0x7fc368004d40 2024-05-26T22:53:46.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:46.606+0000 7fc382a00640 1 -- 172.21.15.131:0/2525214429 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc37c02f070 con 0x7fc368004d40 2024-05-26T22:53:46.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:46.606+0000 7fc382a00640 1 -- 172.21.15.131:0/2525214429 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc37c02bbc0 con 0x7fc368004d40 2024-05-26T22:53:46.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:46.606+0000 7fc388c00640 1 -- 172.21.15.131:0/2525214429 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc368004d40 msgr2=0x7fc3680051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:46.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:46.606+0000 7fc388c00640 1 --2- 172.21.15.131:0/2525214429 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc368004d40 0x7fc3680051c0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fc37c00a740 tx=0x7fc368005830 comp rx=0 tx=0).stop 2024-05-26T22:53:46.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:46.606+0000 7fc383e00640 1 -- 172.21.15.131:0/2525214429 reap_dead start 2024-05-26T22:53:46.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:46.606+0000 7fc388c00640 1 -- 172.21.15.131:0/2525214429 shutdown_connections 2024-05-26T22:53:46.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:46.606+0000 7fc388c00640 1 -- 172.21.15.131:0/2525214429 >> 172.21.15.131:0/2525214429 conn(0x7fc3840b5660 msgr2=0x7fc3840b5a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:46.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:46.606+0000 7fc388c00640 1 -- 172.21.15.131:0/2525214429 shutdown_connections 2024-05-26T22:53:46.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:46.606+0000 7fc388c00640 1 -- 172.21.15.131:0/2525214429 wait complete. 2024-05-26T22:53:46.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:46.608+0000 7fc388c00640 1 Processor -- start 2024-05-26T22:53:46.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:46.608+0000 7fc388c00640 1 -- start start 2024-05-26T22:53:46.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:46.609+0000 7fc388c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc368004d40 0x7fc38413d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:46.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:46.609+0000 7fc388c00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc384156860 0x7fc3840afc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:46.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:46.609+0000 7fc388c00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc384156ce0 0x7fc3840b01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:46.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:46.609+0000 7fc388c00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc384072fa0 con 0x7fc368004d40 2024-05-26T22:53:46.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:46.609+0000 7fc388c00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc384071a70 con 0x7fc384156860 2024-05-26T22:53:46.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:46.609+0000 7fc388c00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc384073120 con 0x7fc384156ce0 2024-05-26T22:53:46.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:46.609+0000 7fc383400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc368004d40 0x7fc38413d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:46.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:46.609+0000 7fc383400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc368004d40 0x7fc38413d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60182/0 (socket says 172.21.15.131:60182) 2024-05-26T22:53:46.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:46.609+0000 7fc383400640 1 -- 172.21.15.131:0/314406009 learned_addr learned my addr 172.21.15.131:0/314406009 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:53:46.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:46.610+0000 7fc383400640 1 -- 172.21.15.131:0/314406009 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc384156ce0 msgr2=0x7fc3840b01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:46.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:46.610+0000 7fc383400640 1 --2- 172.21.15.131:0/314406009 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc384156ce0 0x7fc3840b01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:46.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:46.610+0000 7fc383400640 1 -- 172.21.15.131:0/314406009 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc384156860 msgr2=0x7fc3840afc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:46.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:46.610+0000 7fc383400640 1 --2- 172.21.15.131:0/314406009 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc384156860 0x7fc3840afc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:46.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:46.610+0000 7fc383400640 1 -- 172.21.15.131:0/314406009 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3840cf5a0 con 0x7fc368004d40 2024-05-26T22:53:46.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:46.610+0000 7fc383400640 1 --2- 172.21.15.131:0/314406009 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc368004d40 0x7fc38413d780 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fc37400e5b0 tx=0x7fc37400ea70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:46.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:46.610+0000 7fc37b400640 1 -- 172.21.15.131:0/314406009 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3740052e0 con 0x7fc368004d40 2024-05-26T22:53:46.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:46.610+0000 7fc37b400640 1 -- 172.21.15.131:0/314406009 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc374018070 con 0x7fc368004d40 2024-05-26T22:53:46.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:46.610+0000 7fc388c00640 1 -- 172.21.15.131:0/314406009 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3840cf880 con 0x7fc368004d40 2024-05-26T22:53:46.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:46.610+0000 7fc37b400640 1 -- 172.21.15.131:0/314406009 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc374003810 con 0x7fc368004d40 2024-05-26T22:53:46.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:46.610+0000 7fc388c00640 1 -- 172.21.15.131:0/314406009 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc384156060 con 0x7fc368004d40 2024-05-26T22:53:46.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:46.612+0000 7fc388c00640 1 -- 172.21.15.131:0/314406009 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3840694e0 con 0x7fc368004d40 2024-05-26T22:53:46.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:46.612+0000 7fc37b400640 1 -- 172.21.15.131:0/314406009 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc3740039b0 con 0x7fc368004d40 2024-05-26T22:53:46.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:46.613+0000 7fc37b400640 1 --2- 172.21.15.131:0/314406009 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc36006bda0 0x7fc36006e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:46.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:46.613+0000 7fc37b400640 1 -- 172.21.15.131:0/314406009 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5115+0+0 (secure 0 0 0) 0x7fc37408a8b0 con 0x7fc368004d40 2024-05-26T22:53:46.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:46.614+0000 7fc37be00640 1 --2- 172.21.15.131:0/314406009 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc36006bda0 0x7fc36006e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:46.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:46.614+0000 7fc37be00640 1 --2- 172.21.15.131:0/314406009 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc36006bda0 0x7fc36006e260 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fc3840b1900 tx=0x7fc36c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:46.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:46.618+0000 7fc37b400640 1 -- 172.21.15.131:0/314406009 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fc374059410 con 0x7fc368004d40 2024-05-26T22:53:46.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:46.815+0000 7fc388c00640 1 -- 172.21.15.131:0/314406009 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fc3840af9b0 con 0x7fc368004d40 2024-05-26T22:53:46.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:46.821+0000 7fc37b400640 1 -- 172.21.15.131:0/314406009 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v43)=0 v43) ==== 122+0+0 (secure 0 0 0) 0x7fc374058d10 con 0x7fc368004d40 2024-05-26T22:53:46.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:46.824+0000 7fc388c00640 1 -- 172.21.15.131:0/314406009 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc36006bda0 msgr2=0x7fc36006e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:46.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:46.824+0000 7fc388c00640 1 --2- 172.21.15.131:0/314406009 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc36006bda0 0x7fc36006e260 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fc3840b1900 tx=0x7fc36c008040 comp rx=0 tx=0).stop 2024-05-26T22:53:46.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:46.824+0000 7fc388c00640 1 -- 172.21.15.131:0/314406009 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc368004d40 msgr2=0x7fc38413d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:46.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:46.824+0000 7fc388c00640 1 --2- 172.21.15.131:0/314406009 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc368004d40 0x7fc38413d780 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fc37400e5b0 tx=0x7fc37400ea70 comp rx=0 tx=0).stop 2024-05-26T22:53:46.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:46.825+0000 7fc388c00640 1 -- 172.21.15.131:0/314406009 shutdown_connections 2024-05-26T22:53:46.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:46.825+0000 7fc388c00640 1 --2- 172.21.15.131:0/314406009 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc384156ce0 0x7fc3840b01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:46.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:46.825+0000 7fc388c00640 1 --2- 172.21.15.131:0/314406009 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc36006bda0 0x7fc36006e260 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:46.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:46.825+0000 7fc388c00640 1 --2- 172.21.15.131:0/314406009 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc384156860 0x7fc3840afc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:46.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:46.825+0000 7fc388c00640 1 --2- 172.21.15.131:0/314406009 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc368004d40 0x7fc38413d780 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:46.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:46.825+0000 7fc388c00640 1 -- 172.21.15.131:0/314406009 >> 172.21.15.131:0/314406009 conn(0x7fc3840b5660 msgr2=0x7fc38415a1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:46.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:46.825+0000 7fc388c00640 1 -- 172.21.15.131:0/314406009 shutdown_connections 2024-05-26T22:53:46.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:46.825+0000 7fc388c00640 1 -- 172.21.15.131:0/314406009 wait complete. 2024-05-26T22:53:46.845 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-05-26T22:53:46.845 DEBUG:teuthology.orchestra.run.smithi131:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-26T22:53:46.960 DEBUG:teuthology.orchestra.run.smithi187:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-26T22:53:47.073 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-05-26T22:53:47.073 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-26T22:53:47.074 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-26T22:53:47.131 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-26T22:53:47.131 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-05-26T22:53:47.131 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-26T22:53:47.187 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-26T22:53:47.187 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-26T22:53:47.187 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-26T22:53:47.246 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-26T22:53:47.247 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-05-26T22:53:47.247 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-26T22:53:47.304 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-26T22:53:47.305 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-26T22:53:47.305 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-26T22:53:47.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:47.502+0000 7f161d600640 1 Processor -- start 2024-05-26T22:53:47.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:47.502+0000 7f161d600640 1 -- start start 2024-05-26T22:53:47.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:47.503+0000 7f161d600640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f16180d23d0 0x7f161815ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:47.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:47.503+0000 7f161d600640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1618162330 con 0x7f1618161de0 2024-05-26T22:53:47.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:47.503+0000 7f161d600640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1618162610 con 0x7f16180b5b90 2024-05-26T22:53:47.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:47.503+0000 7f161d600640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f161815e970 con 0x7f16180d23d0 2024-05-26T22:53:47.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:47.504+0000 7f160fe00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f1618161de0 0x7f16180b9990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:40394/0 (socket says 172.21.15.131:40394) 2024-05-26T22:53:47.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:47.504+0000 7f160fe00640 1 -- 172.21.15.131:0/4018667997 learned_addr learned my addr 172.21.15.131:0/4018667997 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:53:47.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:47.504+0000 7f161cc00640 1 --2- 172.21.15.131:0/4018667997 >> v2:172.21.15.131:3300/0 conn(0x7f16180d23d0 0x7f161815ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:47.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:47.504+0000 7f1617400640 1 -- 172.21.15.131:0/4018667997 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2270130104 0 0) 0x7f1618162330 con 0x7f1618161de0 2024-05-26T22:53:47.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:47.504+0000 7f1617400640 1 -- 172.21.15.131:0/4018667997 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f15fc003520 con 0x7f1618161de0 2024-05-26T22:53:47.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:47.504+0000 7f161cc00640 1 -- 172.21.15.131:0/4018667997 >> v1:172.21.15.187:6789/0 conn(0x7f16180b5b90 legacy=0x7f16180b5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:47.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:47.504+0000 7f161cc00640 1 -- 172.21.15.131:0/4018667997 >> v1:172.21.15.131:6789/0 conn(0x7f1618161de0 legacy=0x7f16180b9990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:47.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:47.504+0000 7f161cc00640 1 -- 172.21.15.131:0/4018667997 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16181628f0 con 0x7f16180d23d0 2024-05-26T22:53:47.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:47.506+0000 7f161cc00640 1 --2- 172.21.15.131:0/4018667997 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f16180d23d0 0x7f161815ee00 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f161815eaf0 tx=0x7f161002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e24f271bebfe288 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:47.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:47.506+0000 7f1617400640 1 -- 172.21.15.131:0/4018667997 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1610007740 con 0x7f16180d23d0 2024-05-26T22:53:47.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:47.506+0000 7f1617400640 1 -- 172.21.15.131:0/4018667997 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16100078e0 con 0x7f16180d23d0 2024-05-26T22:53:47.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:47.506+0000 7f1617400640 1 -- 172.21.15.131:0/4018667997 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1610007cb0 con 0x7f16180d23d0 2024-05-26T22:53:47.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:47.506+0000 7f161d600640 1 -- 172.21.15.131:0/4018667997 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f16180d23d0 msgr2=0x7f161815ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:47.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:47.506+0000 7f161d600640 1 --2- 172.21.15.131:0/4018667997 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f16180d23d0 0x7f161815ee00 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f161815eaf0 tx=0x7f161002a660 comp rx=0 tx=0).stop 2024-05-26T22:53:47.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:47.507+0000 7f161d600640 1 -- 172.21.15.131:0/4018667997 shutdown_connections 2024-05-26T22:53:47.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:47.507+0000 7f161d600640 1 --2- 172.21.15.131:0/4018667997 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f16180d23d0 0x7f161815ee00 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:47.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:47.507+0000 7f161d600640 1 -- 172.21.15.131:0/4018667997 >> 172.21.15.131:0/4018667997 conn(0x7f1618073070 msgr2=0x7f16180d0b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:47.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:47.507+0000 7f161d600640 1 -- 172.21.15.131:0/4018667997 shutdown_connections 2024-05-26T22:53:47.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:47.507+0000 7f161d600640 1 -- 172.21.15.131:0/4018667997 wait complete. 2024-05-26T22:53:47.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:47.508+0000 7f161d600640 1 Processor -- start 2024-05-26T22:53:47.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:47.509+0000 7f161d600640 1 -- start start 2024-05-26T22:53:47.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:47.509+0000 7f161d600640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f16180b5b90 0x7f16180c99b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:47.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:47.509+0000 7f161d600640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f16180d23d0 0x7f16180cd0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:47.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:47.509+0000 7f161d600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1618161de0 0x7f16180b87f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:47.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:47.509+0000 7f161d600640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f161815e5e0 con 0x7f1618161de0 2024-05-26T22:53:47.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:47.509+0000 7f161d600640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f16180d2860 con 0x7f16180d23d0 2024-05-26T22:53:47.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:47.509+0000 7f161d600640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f161815e760 con 0x7f16180b5b90 2024-05-26T22:53:47.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:47.510+0000 7f161cc00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1618161de0 0x7f16180b87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:47.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:47.510+0000 7f161cc00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1618161de0 0x7f16180b87f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60198/0 (socket says 172.21.15.131:60198) 2024-05-26T22:53:47.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:47.510+0000 7f161cc00640 1 -- 172.21.15.131:0/1338335963 learned_addr learned my addr 172.21.15.131:0/1338335963 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:53:47.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:47.510+0000 7f1617e00640 1 --2- 172.21.15.131:0/1338335963 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f16180b5b90 0x7f16180c99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:47.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:47.510+0000 7f160fe00640 1 --2- 172.21.15.131:0/1338335963 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f16180d23d0 0x7f16180cd0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:47.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:47.510+0000 7f161cc00640 1 -- 172.21.15.131:0/1338335963 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f16180b5b90 msgr2=0x7f16180c99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:47.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:47.510+0000 7f161cc00640 1 --2- 172.21.15.131:0/1338335963 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f16180b5b90 0x7f16180c99b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:47.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:47.510+0000 7f161cc00640 1 -- 172.21.15.131:0/1338335963 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f16180d23d0 msgr2=0x7f16180cd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:47.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:47.510+0000 7f161cc00640 1 --2- 172.21.15.131:0/1338335963 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f16180d23d0 0x7f16180cd0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:47.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:47.510+0000 7f161cc00640 1 -- 172.21.15.131:0/1338335963 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1618159350 con 0x7f1618161de0 2024-05-26T22:53:47.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:47.510+0000 7f160fe00640 1 --2- 172.21.15.131:0/1338335963 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f16180d23d0 0x7f16180cd0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:53:47.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:47.510+0000 7f1617e00640 1 --2- 172.21.15.131:0/1338335963 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f16180b5b90 0x7f16180c99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:53:47.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:47.511+0000 7f161cc00640 1 --2- 172.21.15.131:0/1338335963 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1618161de0 0x7f16180b87f0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f161002ac50 tx=0x7f1610004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:47.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:47.511+0000 7f1614c00640 1 -- 172.21.15.131:0/1338335963 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1610040070 con 0x7f1618161de0 2024-05-26T22:53:47.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:47.511+0000 7f1614c00640 1 -- 172.21.15.131:0/1338335963 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1610031bc0 con 0x7f1618161de0 2024-05-26T22:53:47.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:47.511+0000 7f1614c00640 1 -- 172.21.15.131:0/1338335963 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1610009450 con 0x7f1618161de0 2024-05-26T22:53:47.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:47.511+0000 7f161d600640 1 -- 172.21.15.131:0/1338335963 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16180c8630 con 0x7f1618161de0 2024-05-26T22:53:47.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:47.511+0000 7f161d600640 1 -- 172.21.15.131:0/1338335963 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16180c8bf0 con 0x7f1618161de0 2024-05-26T22:53:47.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:47.512+0000 7f161d600640 1 -- 172.21.15.131:0/1338335963 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1618068c50 con 0x7f1618161de0 2024-05-26T22:53:47.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:47.513+0000 7f1614c00640 1 -- 172.21.15.131:0/1338335963 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1610039040 con 0x7f1618161de0 2024-05-26T22:53:47.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:47.514+0000 7f1614c00640 1 --2- 172.21.15.131:0/1338335963 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f15f406be30 0x7f15f406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:47.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:47.514+0000 7f1614c00640 1 -- 172.21.15.131:0/1338335963 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5115+0+0 (secure 0 0 0) 0x7f16100ac520 con 0x7f1618161de0 2024-05-26T22:53:47.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:47.514+0000 7f1617e00640 1 --2- 172.21.15.131:0/1338335963 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f15f406be30 0x7f15f406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:47.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:47.515+0000 7f1617e00640 1 --2- 172.21.15.131:0/1338335963 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f15f406be30 0x7f15f406e2f0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f1608001680 tx=0x7f1608009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:47.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:47.519+0000 7f1614c00640 1 -- 172.21.15.131:0/1338335963 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f1618068c50 con 0x7f1618161de0 2024-05-26T22:53:47.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:47.756+0000 7f161d600640 1 -- 172.21.15.131:0/1338335963 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f16180694e0 con 0x7f1618161de0 2024-05-26T22:53:47.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:47.757+0000 7f1614c00640 1 -- 172.21.15.131:0/1338335963 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) ==== 94+0+3878 (secure 0 0 0) 0x7f161007b080 con 0x7f1618161de0 2024-05-26T22:53:47.757 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:53:47.757 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":71,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5042,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6834/1350327353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1350327353},{"type":"v1","addr":"172.21.15.187:6834","nonce":1350327353}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":5359,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/4204396845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6833","nonce":4204396845},{"type":"v1","addr":"172.21.15.187:6835","nonce":4204396845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59},{"gid":7251,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/212381316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":212381316},{"type":"v1","addr":"172.21.15.131:6835","nonce":212381316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":7254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/3617296636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":3617296636},{"type":"v1","addr":"172.21.15.131:6837","nonce":3617296636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71}],"filesystems":[]} 2024-05-26T22:53:47.757 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 71 2024-05-26T22:53:47.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:47.760+0000 7f161d600640 1 -- 172.21.15.131:0/1338335963 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f15f406be30 msgr2=0x7f15f406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:47.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:47.760+0000 7f161d600640 1 --2- 172.21.15.131:0/1338335963 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f15f406be30 0x7f15f406e2f0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f1608001680 tx=0x7f1608009040 comp rx=0 tx=0).stop 2024-05-26T22:53:47.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:47.760+0000 7f161d600640 1 -- 172.21.15.131:0/1338335963 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1618161de0 msgr2=0x7f16180b87f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:47.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:47.760+0000 7f161d600640 1 --2- 172.21.15.131:0/1338335963 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1618161de0 0x7f16180b87f0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f161002ac50 tx=0x7f1610004a40 comp rx=0 tx=0).stop 2024-05-26T22:53:47.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:47.760+0000 7f161d600640 1 -- 172.21.15.131:0/1338335963 shutdown_connections 2024-05-26T22:53:47.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:47.760+0000 7f161d600640 1 --2- 172.21.15.131:0/1338335963 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f15f406be30 0x7f15f406e2f0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:47.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:47.760+0000 7f161d600640 1 --2- 172.21.15.131:0/1338335963 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1618161de0 0x7f16180b87f0 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:47.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:47.760+0000 7f161d600640 1 --2- 172.21.15.131:0/1338335963 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f16180d23d0 0x7f16180cd0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:47.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:47.760+0000 7f161d600640 1 --2- 172.21.15.131:0/1338335963 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f16180b5b90 0x7f16180c99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:47.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:47.760+0000 7f161d600640 1 -- 172.21.15.131:0/1338335963 >> 172.21.15.131:0/1338335963 conn(0x7f1618073070 msgr2=0x7f16180b4690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:47.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:47.760+0000 7f161d600640 1 -- 172.21.15.131:0/1338335963 shutdown_connections 2024-05-26T22:53:47.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:47.761+0000 7f161d600640 1 -- 172.21.15.131:0/1338335963 wait complete. 2024-05-26T22:53:47.778 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-26T22:53:47.779 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-26T22:53:47.779 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-26T22:53:47.779 INFO:teuthology.orchestra.run:waiting for 300 2024-05-26T22:53:47.779 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-26T22:53:47.779 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-26T22:53:47.780 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-26T22:53:47.780 INFO:teuthology.orchestra.run:waiting for 300 2024-05-26T22:53:47.780 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-26T22:53:47.780 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-26T22:53:47.780 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-26T22:53:47.780 INFO:teuthology.orchestra.run:waiting for 300 2024-05-26T22:53:47.780 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-26T22:53:47.780 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-26T22:53:47.781 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-26T22:53:47.781 INFO:teuthology.orchestra.run:waiting for 300 2024-05-26T22:53:47.796 INFO:tasks.ceph.mds.c:Stopped 2024-05-26T22:53:47.797 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-26T22:53:47.799 INFO:tasks.ceph.mds.a:Stopped 2024-05-26T22:53:47.799 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-26T22:53:47.803 INFO:tasks.ceph.mds.d:Stopped 2024-05-26T22:53:47.803 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-26T22:53:47.804 INFO:tasks.ceph.mds.b:Stopped 2024-05-26T22:53:47.804 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-26T22:53:47.846 INFO:tasks.ceph.mds.c:Started 2024-05-26T22:53:47.847 DEBUG:teuthology.parallel:result is None 2024-05-26T22:53:47.852 INFO:tasks.ceph.mds.d:Started 2024-05-26T22:53:47.852 DEBUG:teuthology.parallel:result is None 2024-05-26T22:53:47.853 INFO:tasks.ceph.mds.a:Started 2024-05-26T22:53:47.853 DEBUG:teuthology.parallel:result is None 2024-05-26T22:53:47.857 INFO:tasks.ceph.mds.b:Started 2024-05-26T22:53:47.858 DEBUG:teuthology.parallel:result is None 2024-05-26T22:53:47.858 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-26T22:53:47.989 INFO:tasks.ceph.mds.a.smithi131.stdout:starting mds.a at 2024-05-26T22:53:47.990 INFO:tasks.ceph.mds.c.smithi131.stdout:starting mds.c at 2024-05-26T22:53:48.005 INFO:tasks.ceph.mds.b.smithi187.stdout:starting mds.b at 2024-05-26T22:53:48.005 INFO:tasks.ceph.mds.d.smithi187.stdout:starting mds.d at 2024-05-26T22:53:48.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.030+0000 7f8170000640 1 Processor -- start 2024-05-26T22:53:48.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.030+0000 7f8170000640 1 -- start start 2024-05-26T22:53:48.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.031+0000 7f8170000640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f8168165e30 0x7f8168072540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:48.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.031+0000 7f8170000640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f81680af230 0x7f81680cfe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:48.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.031+0000 7f8170000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f81680d0390 0x7f8168071360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:48.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.031+0000 7f8170000640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f8168072e90 con 0x7f81680af230 2024-05-26T22:53:48.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.031+0000 7f8170000640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f81680733c0 con 0x7f8168165e30 2024-05-26T22:53:48.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.031+0000 7f8170000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f81680cf540 con 0x7f81680d0390 2024-05-26T22:53:48.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.031+0000 7f816cc00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f81680af230 0x7f81680cfe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:48.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.031+0000 7f816cc00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f81680af230 0x7f81680cfe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60234/0 (socket says 172.21.15.131:60234) 2024-05-26T22:53:48.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.031+0000 7f816cc00640 1 -- 172.21.15.131:0/3054585939 learned_addr learned my addr 172.21.15.131:0/3054585939 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:53:48.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.032+0000 7f816e000640 1 --2- 172.21.15.131:0/3054585939 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f81680d0390 0x7f8168071360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:48.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.032+0000 7f816d600640 1 --2- 172.21.15.131:0/3054585939 >> v2:172.21.15.187:3300/0 conn(0x7f8168165e30 0x7f8168072540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:48.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.032+0000 7f816cc00640 1 -- 172.21.15.131:0/3054585939 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f81680d0390 msgr2=0x7f8168071360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:48.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.032+0000 7f816cc00640 1 --2- 172.21.15.131:0/3054585939 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f81680d0390 0x7f8168071360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:48.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.032+0000 7f816cc00640 1 -- 172.21.15.131:0/3054585939 >> v2:172.21.15.187:3300/0 conn(0x7f8168165e30 msgr2=0x7f8168072540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:48.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.032+0000 7f816cc00640 1 --2- 172.21.15.131:0/3054585939 >> v2:172.21.15.187:3300/0 conn(0x7f8168165e30 0x7f8168072540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:48.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.032+0000 7f816cc00640 1 -- 172.21.15.131:0/3054585939 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81680730e0 con 0x7f81680af230 2024-05-26T22:53:48.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.033+0000 7f816cc00640 1 --2- 172.21.15.131:0/3054585939 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f81680af230 0x7f81680cfe50 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f81680d0c40 tx=0x7f815402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f47e865b6b5775e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:48.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.033+0000 7f8167e00640 1 -- 172.21.15.131:0/3054585939 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8154037070 con 0x7f81680af230 2024-05-26T22:53:48.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.033+0000 7f8167e00640 1 -- 172.21.15.131:0/3054585939 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8154032440 con 0x7f81680af230 2024-05-26T22:53:48.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.033+0000 7f8167e00640 1 -- 172.21.15.131:0/3054585939 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8154032810 con 0x7f81680af230 2024-05-26T22:53:48.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.034+0000 7f8170000640 1 -- 172.21.15.131:0/3054585939 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f81680af230 msgr2=0x7f81680cfe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:48.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.034+0000 7f8170000640 1 --2- 172.21.15.131:0/3054585939 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f81680af230 0x7f81680cfe50 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f81680d0c40 tx=0x7f815402a140 comp rx=0 tx=0).stop 2024-05-26T22:53:48.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.034+0000 7f8170000640 1 -- 172.21.15.131:0/3054585939 shutdown_connections 2024-05-26T22:53:48.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.034+0000 7f8170000640 1 --2- 172.21.15.131:0/3054585939 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f81680d0390 0x7f8168071360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:48.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.034+0000 7f8170000640 1 --2- 172.21.15.131:0/3054585939 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f81680af230 0x7f81680cfe50 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:48.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.034+0000 7f8170000640 1 --2- 172.21.15.131:0/3054585939 >> v2:172.21.15.187:3300/0 conn(0x7f8168165e30 0x7f8168072540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:48.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.034+0000 7f8170000640 1 -- 172.21.15.131:0/3054585939 >> 172.21.15.131:0/3054585939 conn(0x7f816806bc90 msgr2=0x7f81680b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:48.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.034+0000 7f8170000640 1 -- 172.21.15.131:0/3054585939 shutdown_connections 2024-05-26T22:53:48.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.034+0000 7f8170000640 1 -- 172.21.15.131:0/3054585939 wait complete. 2024-05-26T22:53:48.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.036+0000 7f8170000640 1 Processor -- start 2024-05-26T22:53:48.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.036+0000 7f8170000640 1 -- start start 2024-05-26T22:53:48.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.037+0000 7f8170000640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f81680af230 0x7f816815afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:48.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.037+0000 7f8170000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f81680d0390 0x7f816815d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:48.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.037+0000 7f8170000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8168165e30 0x7f81680ce680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:48.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.037+0000 7f8170000640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f81680718a0 con 0x7f81680d0390 2024-05-26T22:53:48.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.037+0000 7f8170000640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f81680cfac0 con 0x7f81680af230 2024-05-26T22:53:48.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.037+0000 7f8170000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f8168071a20 con 0x7f8168165e30 2024-05-26T22:53:48.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.037+0000 7f816cc00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f81680d0390 0x7f816815d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:48.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.037+0000 7f816cc00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f81680d0390 0x7f816815d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60246/0 (socket says 172.21.15.131:60246) 2024-05-26T22:53:48.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.037+0000 7f816cc00640 1 -- 172.21.15.131:0/3410401226 learned_addr learned my addr 172.21.15.131:0/3410401226 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:53:48.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.037+0000 7f816d600640 1 --2- 172.21.15.131:0/3410401226 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f81680af230 0x7f816815afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:48.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.037+0000 7f816e000640 1 --2- 172.21.15.131:0/3410401226 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8168165e30 0x7f81680ce680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:48.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.037+0000 7f816cc00640 1 -- 172.21.15.131:0/3410401226 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8168165e30 msgr2=0x7f81680ce680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:48.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.037+0000 7f816cc00640 1 --2- 172.21.15.131:0/3410401226 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8168165e30 0x7f81680ce680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:48.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.037+0000 7f816cc00640 1 -- 172.21.15.131:0/3410401226 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f81680af230 msgr2=0x7f816815afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:48.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.037+0000 7f816cc00640 1 --2- 172.21.15.131:0/3410401226 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f81680af230 0x7f816815afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:48.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.037+0000 7f816cc00640 1 -- 172.21.15.131:0/3410401226 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f816815b520 con 0x7f81680d0390 2024-05-26T22:53:48.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.037+0000 7f816d600640 1 --2- 172.21.15.131:0/3410401226 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f81680af230 0x7f816815afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-26T22:53:48.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.038+0000 7f816cc00640 1 --2- 172.21.15.131:0/3410401226 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f81680d0390 0x7f816815d2c0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f815402a180 tx=0x7f81540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:48.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.038+0000 7f8165600640 1 -- 172.21.15.131:0/3410401226 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8154041020 con 0x7f81680d0390 2024-05-26T22:53:48.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.038+0000 7f8165600640 1 -- 172.21.15.131:0/3410401226 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f815403d620 con 0x7f81680d0390 2024-05-26T22:53:48.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.038+0000 7f8165600640 1 -- 172.21.15.131:0/3410401226 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8154037040 con 0x7f81680d0390 2024-05-26T22:53:48.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.038+0000 7f8170000640 1 -- 172.21.15.131:0/3410401226 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f816815c500 con 0x7f81680d0390 2024-05-26T22:53:48.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.038+0000 7f8170000640 1 -- 172.21.15.131:0/3410401226 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f816815cac0 con 0x7f81680d0390 2024-05-26T22:53:48.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.039+0000 7f8170000640 1 -- 172.21.15.131:0/3410401226 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8168074e90 con 0x7f81680d0390 2024-05-26T22:53:48.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.040+0000 7f8165600640 1 -- 172.21.15.131:0/3410401226 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8154044020 con 0x7f81680d0390 2024-05-26T22:53:48.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.041+0000 7f8165600640 1 --2- 172.21.15.131:0/3410401226 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f814806be60 0x7f814806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:48.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.041+0000 7f8165600640 1 -- 172.21.15.131:0/3410401226 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5115+0+0 (secure 0 0 0) 0x7f81540ab330 con 0x7f81680d0390 2024-05-26T22:53:48.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.041+0000 7f816d600640 1 --2- 172.21.15.131:0/3410401226 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f814806be60 0x7f814806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:48.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.042+0000 7f816d600640 1 --2- 172.21.15.131:0/3410401226 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f814806be60 0x7f814806e320 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f815c009af0 tx=0x7f815c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:48.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.046+0000 7f8165600640 1 -- 172.21.15.131:0/3410401226 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f815404b030 con 0x7f81680d0390 2024-05-26T22:53:48.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.255+0000 7f8170000640 1 -- 172.21.15.131:0/3410401226 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f816806c320 con 0x7f81680d0390 2024-05-26T22:53:48.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.359+0000 7f8165600640 1 -- 172.21.15.131:0/3410401226 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v76) ==== 97+0+0 (secure 0 0 0) 0x7f8154079ea0 con 0x7f81680d0390 2024-05-26T22:53:48.359 INFO:teuthology.orchestra.run.smithi131.stderr: removed all blocklist entries 2024-05-26T22:53:48.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.363+0000 7f8170000640 1 -- 172.21.15.131:0/3410401226 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f814806be60 msgr2=0x7f814806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:48.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.363+0000 7f8170000640 1 --2- 172.21.15.131:0/3410401226 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f814806be60 0x7f814806e320 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f815c009af0 tx=0x7f815c008040 comp rx=0 tx=0).stop 2024-05-26T22:53:48.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.363+0000 7f8170000640 1 -- 172.21.15.131:0/3410401226 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f81680d0390 msgr2=0x7f816815d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:48.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.363+0000 7f8170000640 1 --2- 172.21.15.131:0/3410401226 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f81680d0390 0x7f816815d2c0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f815402a180 tx=0x7f81540046c0 comp rx=0 tx=0).stop 2024-05-26T22:53:48.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.363+0000 7f8170000640 1 -- 172.21.15.131:0/3410401226 shutdown_connections 2024-05-26T22:53:48.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.363+0000 7f8170000640 1 --2- 172.21.15.131:0/3410401226 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8168165e30 0x7f81680ce680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:48.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.363+0000 7f8170000640 1 --2- 172.21.15.131:0/3410401226 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f814806be60 0x7f814806e320 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:48.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.363+0000 7f8170000640 1 --2- 172.21.15.131:0/3410401226 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f81680d0390 0x7f816815d2c0 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:48.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.363+0000 7f8170000640 1 --2- 172.21.15.131:0/3410401226 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f81680af230 0x7f816815afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:48.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.363+0000 7f8170000640 1 -- 172.21.15.131:0/3410401226 >> 172.21.15.131:0/3410401226 conn(0x7f816806bc90 msgr2=0x7f81680afb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:48.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.364+0000 7f8170000640 1 -- 172.21.15.131:0/3410401226 shutdown_connections 2024-05-26T22:53:48.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.364+0000 7f8170000640 1 -- 172.21.15.131:0/3410401226 wait complete. 2024-05-26T22:53:48.386 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-26T22:53:48.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.572+0000 7f28b5000640 1 Processor -- start 2024-05-26T22:53:48.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.572+0000 7f28b5000640 1 -- start start 2024-05-26T22:53:48.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.574+0000 7f28b5000640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f28b006ebc0 0x7f28b00cedf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:48.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.574+0000 7f28b5000640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f28b0071b70 0x7f28b00b3d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:48.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.574+0000 7f28b5000640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28b00722a0 con 0x7f28b0144340 2024-05-26T22:53:48.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.574+0000 7f28b5000640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f28b0072eb0 con 0x7f28b006ebc0 2024-05-26T22:53:48.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.574+0000 7f28b5000640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f28b0072580 con 0x7f28b0071b70 2024-05-26T22:53:48.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.574+0000 7f28a6a00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f28b006ebc0 0x7f28b00cedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:48.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.574+0000 7f28a6a00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f28b006ebc0 0x7f28b00cedf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60256/0 (socket says 172.21.15.131:60256) 2024-05-26T22:53:48.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.574+0000 7f28a6a00640 1 -- 172.21.15.131:0/3667685897 learned_addr learned my addr 172.21.15.131:0/3667685897 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:53:48.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.574+0000 7f28af400640 1 --1- 172.21.15.131:0/3667685897 >> v1:172.21.15.131:6789/0 conn(0x7f28b0144340 0x7f28b006e4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:40406/0 (socket says 172.21.15.131:40406) 2024-05-26T22:53:48.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.575+0000 7f28afe00640 1 --2- 172.21.15.131:0/3667685897 >> v2:172.21.15.187:3300/0 conn(0x7f28b0071b70 0x7f28b00b3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:48.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.575+0000 7f28a6a00640 1 -- 172.21.15.131:0/3667685897 >> v2:172.21.15.187:3300/0 conn(0x7f28b0071b70 msgr2=0x7f28b00b3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:48.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.575+0000 7f28a6a00640 1 --2- 172.21.15.131:0/3667685897 >> v2:172.21.15.187:3300/0 conn(0x7f28b0071b70 0x7f28b00b3d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:48.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.575+0000 7f28a6a00640 1 -- 172.21.15.131:0/3667685897 >> v1:172.21.15.131:6789/0 conn(0x7f28b0144340 legacy=0x7f28b006e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:48.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.575+0000 7f28a6a00640 1 -- 172.21.15.131:0/3667685897 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28b0072860 con 0x7f28b006ebc0 2024-05-26T22:53:48.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.576+0000 7f28a6a00640 1 --2- 172.21.15.131:0/3667685897 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f28b006ebc0 0x7f28b00cedf0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f28b00afa30 tx=0x7f289802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1505bace023fc0c3 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:48.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.576+0000 7f28aea00640 1 -- 172.21.15.131:0/3667685897 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2898037070 con 0x7f28b006ebc0 2024-05-26T22:53:48.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.576+0000 7f28aea00640 1 -- 172.21.15.131:0/3667685897 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2898032440 con 0x7f28b006ebc0 2024-05-26T22:53:48.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.576+0000 7f28aea00640 1 -- 172.21.15.131:0/3667685897 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2898032810 con 0x7f28b006ebc0 2024-05-26T22:53:48.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.576+0000 7f28b5000640 1 -- 172.21.15.131:0/3667685897 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f28b006ebc0 msgr2=0x7f28b00cedf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:48.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.576+0000 7f28b5000640 1 --2- 172.21.15.131:0/3667685897 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f28b006ebc0 0x7f28b00cedf0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f28b00afa30 tx=0x7f289802a270 comp rx=0 tx=0).stop 2024-05-26T22:53:48.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.577+0000 7f28b5000640 1 -- 172.21.15.131:0/3667685897 shutdown_connections 2024-05-26T22:53:48.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.577+0000 7f28b5000640 1 --2- 172.21.15.131:0/3667685897 >> v2:172.21.15.187:3300/0 conn(0x7f28b0071b70 0x7f28b00b3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:48.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.577+0000 7f28b5000640 1 --2- 172.21.15.131:0/3667685897 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f28b006ebc0 0x7f28b00cedf0 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:48.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.577+0000 7f28b5000640 1 -- 172.21.15.131:0/3667685897 >> 172.21.15.131:0/3667685897 conn(0x7f28b00cff40 msgr2=0x7f28b00d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:48.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.577+0000 7f28b5000640 1 -- 172.21.15.131:0/3667685897 shutdown_connections 2024-05-26T22:53:48.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.577+0000 7f28b5000640 1 -- 172.21.15.131:0/3667685897 wait complete. 2024-05-26T22:53:48.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.578+0000 7f28b5000640 1 Processor -- start 2024-05-26T22:53:48.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.578+0000 7f28b5000640 1 -- start start 2024-05-26T22:53:48.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.579+0000 7f28b5000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f28b006ebc0 0x7f28b01f4230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:48.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.579+0000 7f28b5000640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28b0071b70 0x7f28b00c5d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:48.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.579+0000 7f28b5000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f28b0144340 0x7f28b00cd1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:48.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.579+0000 7f28b5000640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f28b00b4260 con 0x7f28b0144340 2024-05-26T22:53:48.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.579+0000 7f28b5000640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f28b0071f80 con 0x7f28b0071b70 2024-05-26T22:53:48.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.579+0000 7f28b5000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f28b00b43e0 con 0x7f28b006ebc0 2024-05-26T22:53:48.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.580+0000 7f28afe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f28b0144340 0x7f28b00cd1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:48.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.580+0000 7f28afe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f28b0144340 0x7f28b00cd1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60266/0 (socket says 172.21.15.131:60266) 2024-05-26T22:53:48.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.580+0000 7f28afe00640 1 -- 172.21.15.131:0/3486712854 learned_addr learned my addr 172.21.15.131:0/3486712854 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:53:48.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.580+0000 7f28a6a00640 1 --2- 172.21.15.131:0/3486712854 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28b0071b70 0x7f28b00c5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:48.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.580+0000 7f28af400640 1 --2- 172.21.15.131:0/3486712854 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f28b006ebc0 0x7f28b01f4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:48.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.580+0000 7f28afe00640 1 -- 172.21.15.131:0/3486712854 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f28b006ebc0 msgr2=0x7f28b01f4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:48.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.580+0000 7f28afe00640 1 --2- 172.21.15.131:0/3486712854 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f28b006ebc0 0x7f28b01f4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:48.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.580+0000 7f28afe00640 1 -- 172.21.15.131:0/3486712854 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28b0071b70 msgr2=0x7f28b00c5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:48.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.580+0000 7f28afe00640 1 --2- 172.21.15.131:0/3486712854 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28b0071b70 0x7f28b00c5d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:48.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.580+0000 7f28afe00640 1 -- 172.21.15.131:0/3486712854 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28b01f48b0 con 0x7f28b0144340 2024-05-26T22:53:48.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.580+0000 7f28afe00640 1 --2- 172.21.15.131:0/3486712854 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f28b0144340 0x7f28b00cd1f0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f28a800cd70 tx=0x7f28a800a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:48.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.581+0000 7f28a7e00640 1 -- 172.21.15.131:0/3486712854 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28a8003b90 con 0x7f28b0144340 2024-05-26T22:53:48.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.581+0000 7f28a7e00640 1 -- 172.21.15.131:0/3486712854 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28a8003d30 con 0x7f28b0144340 2024-05-26T22:53:48.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.581+0000 7f28b5000640 1 -- 172.21.15.131:0/3486712854 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28b01f4b90 con 0x7f28b0144340 2024-05-26T22:53:48.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.581+0000 7f28a7e00640 1 -- 172.21.15.131:0/3486712854 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28a8004240 con 0x7f28b0144340 2024-05-26T22:53:48.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.581+0000 7f28b5000640 1 -- 172.21.15.131:0/3486712854 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28b01f5150 con 0x7f28b0144340 2024-05-26T22:53:48.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.582+0000 7f28b5000640 1 -- 172.21.15.131:0/3486712854 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28b00b3d00 con 0x7f28b0144340 2024-05-26T22:53:48.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.583+0000 7f28a7e00640 1 -- 172.21.15.131:0/3486712854 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f28a80043e0 con 0x7f28b0144340 2024-05-26T22:53:48.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.584+0000 7f28a7e00640 1 --2- 172.21.15.131:0/3486712854 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f288c06be30 0x7f288c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:48.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.584+0000 7f28a7e00640 1 -- 172.21.15.131:0/3486712854 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 4857+0+0 (secure 0 0 0) 0x7f28a808b700 con 0x7f28b0144340 2024-05-26T22:53:48.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.584+0000 7f28af400640 1 --2- 172.21.15.131:0/3486712854 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f288c06be30 0x7f288c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:48.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.585+0000 7f28af400640 1 --2- 172.21.15.131:0/3486712854 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f288c06be30 0x7f288c06e2f0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f28a0000f60 tx=0x7f28a0007c40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:48.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.588+0000 7f28a7e00640 1 -- 172.21.15.131:0/3486712854 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f28a801d340 con 0x7f28b0144340 2024-05-26T22:53:48.840 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.839+0000 7f28b5000640 1 -- 172.21.15.131:0/3486712854 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f28b00694e0 con 0x7f28b0144340 2024-05-26T22:53:48.840 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.840+0000 7f28a7e00640 1 -- 172.21.15.131:0/3486712854 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) ==== 80+0+5304 (secure 0 0 0) 0x7f28a805a360 con 0x7f28b0144340 2024-05-26T22:53:48.841 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:53:48.841 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-26T22:53:48.841 INFO:teuthology.orchestra.run.smithi131.stdout: "auth_dump": [ 2024-05-26T22:53:48.841 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:53:48.841 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mds.a", 2024-05-26T22:53:48.841 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD1u1NmTQf1DhAAdvEt7wP0IgKhNHOqjVAllA==", 2024-05-26T22:53:48.841 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:53:48.841 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-05-26T22:53:48.841 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2024-05-26T22:53:48.842 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2024-05-26T22:53:48.842 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-05-26T22:53:48.842 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:53:48.842 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:53:48.842 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:53:48.842 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mds.b", 2024-05-26T22:53:48.842 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD1u1NmHNZ8KxAAPgQ/ojbl+EDfPhqrkg7LPQ==", 2024-05-26T22:53:48.842 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:53:48.842 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-05-26T22:53:48.842 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2024-05-26T22:53:48.842 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2024-05-26T22:53:48.842 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-05-26T22:53:48.843 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:53:48.843 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:53:48.843 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:53:48.843 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mds.c", 2024-05-26T22:53:48.843 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD1u1NmFjCHHRAAqix3Ujfsq2tnSU5+d/pSYA==", 2024-05-26T22:53:48.843 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:53:48.843 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-05-26T22:53:48.843 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2024-05-26T22:53:48.843 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2024-05-26T22:53:48.843 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-05-26T22:53:48.843 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:53:48.844 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:53:48.844 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:53:48.844 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mds.d", 2024-05-26T22:53:48.844 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD1u1NmM4LPORAAhtQtr+Vfpf7vdVwjSW6xtA==", 2024-05-26T22:53:48.844 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:53:48.844 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-05-26T22:53:48.844 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2024-05-26T22:53:48.844 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2024-05-26T22:53:48.844 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-05-26T22:53:48.844 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:53:48.844 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:53:48.844 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:53:48.844 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.0", 2024-05-26T22:53:48.845 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD5u1Nm+w1LNhAAsJLQkiKSh3KkMymlrZkcrw==", 2024-05-26T22:53:48.845 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:53:48.845 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-05-26T22:53:48.845 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-05-26T22:53:48.845 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-05-26T22:53:48.845 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:53:48.845 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:53:48.845 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:53:48.845 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.1", 2024-05-26T22:53:48.845 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD9u1Nm4c8RFBAAcn9HQ+izML9rujJ/2/Q/BQ==", 2024-05-26T22:53:48.845 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:53:48.845 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-05-26T22:53:48.845 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-05-26T22:53:48.845 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-05-26T22:53:48.846 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:53:48.846 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:53:48.846 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:53:48.846 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.2", 2024-05-26T22:53:48.846 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAAvFNmR5wmLRAAYEBAXPBZFq67iARZnwF3iw==", 2024-05-26T22:53:48.846 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:53:48.846 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-05-26T22:53:48.846 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-05-26T22:53:48.846 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-05-26T22:53:48.846 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:53:48.846 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:53:48.846 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:53:48.846 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.3", 2024-05-26T22:53:48.847 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAEvFNmJ4G5CxAAnTbP8DinhmuvGNo1VBVzjQ==", 2024-05-26T22:53:48.847 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:53:48.847 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-05-26T22:53:48.847 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-05-26T22:53:48.847 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-05-26T22:53:48.847 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:53:48.847 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:53:48.847 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:53:48.847 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.4", 2024-05-26T22:53:48.847 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAKvFNmcw5rGhAAvnivP6kt3vcN9UPm/svKqw==", 2024-05-26T22:53:48.847 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:53:48.847 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-05-26T22:53:48.847 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-05-26T22:53:48.847 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-05-26T22:53:48.848 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:53:48.848 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:53:48.848 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:53:48.848 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.5", 2024-05-26T22:53:48.848 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQANvFNmTqlzNhAAGfrSvNNHYKGOBzRXo0TEdw==", 2024-05-26T22:53:48.848 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:53:48.848 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-05-26T22:53:48.848 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-05-26T22:53:48.848 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-05-26T22:53:48.848 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:53:48.848 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:53:48.848 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:53:48.848 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.6", 2024-05-26T22:53:48.849 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQARvFNmpzd6FRAAgi2RYlQPvfHNe0iP4bJrwg==", 2024-05-26T22:53:48.849 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:53:48.849 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-05-26T22:53:48.849 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-05-26T22:53:48.849 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-05-26T22:53:48.849 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:53:48.849 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:53:48.849 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:53:48.849 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.7", 2024-05-26T22:53:48.849 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAUvFNmAkC8LhAAsmkKF4eloxdpCi/mrDtzqA==", 2024-05-26T22:53:48.849 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:53:48.849 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-05-26T22:53:48.849 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-05-26T22:53:48.850 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-05-26T22:53:48.850 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:53:48.850 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:53:48.850 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:53:48.850 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.0", 2024-05-26T22:53:48.850 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD2u1NmSGVWChAAH/wq3iaUs6ZELRhLKhmURQ==", 2024-05-26T22:53:48.851 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:53:48.851 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-05-26T22:53:48.851 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow r", 2024-05-26T22:53:48.851 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-05-26T22:53:48.851 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:53:48.851 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:53:48.851 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:53:48.851 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.1", 2024-05-26T22:53:48.851 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD2u1NmnA9CHRAA2XMmZQxysKweHNykiQUEeA==", 2024-05-26T22:53:48.851 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:53:48.851 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-05-26T22:53:48.851 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow r", 2024-05-26T22:53:48.851 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow rw", 2024-05-26T22:53:48.852 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow rwx" 2024-05-26T22:53:48.852 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:53:48.852 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:53:48.852 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:53:48.852 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.2", 2024-05-26T22:53:48.852 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD2u1NmGEI9ExAAvJn1LRwmALkD7zh/AP07vQ==", 2024-05-26T22:53:48.852 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:53:48.852 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-05-26T22:53:48.852 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow r", 2024-05-26T22:53:48.852 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow rw", 2024-05-26T22:53:48.852 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow rwx" 2024-05-26T22:53:48.852 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:53:48.852 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:53:48.852 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:53:48.853 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.3", 2024-05-26T22:53:48.853 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD2u1NmIHOtJhAAJCUm0kGqWu4rWVLsDkW6Eg==", 2024-05-26T22:53:48.853 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:53:48.853 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-05-26T22:53:48.853 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow r", 2024-05-26T22:53:48.853 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow rw", 2024-05-26T22:53:48.853 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow rwx" 2024-05-26T22:53:48.853 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:53:48.853 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:53:48.853 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:53:48.853 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.admin", 2024-05-26T22:53:48.853 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD0u1NmLRGoCBAAhDeVJV14MTAIdUte067jig==", 2024-05-26T22:53:48.853 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:53:48.854 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow *", 2024-05-26T22:53:48.854 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2024-05-26T22:53:48.854 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2024-05-26T22:53:48.854 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-05-26T22:53:48.854 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:53:48.854 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:53:48.854 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:53:48.854 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mgr.x", 2024-05-26T22:53:48.854 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD1u1NmaWhyBBAAlU7Lq6HMjkCPu0kztJIAXQ==", 2024-05-26T22:53:48.854 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:53:48.854 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow *", 2024-05-26T22:53:48.854 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile mgr", 2024-05-26T22:53:48.855 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-05-26T22:53:48.855 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:53:48.855 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:53:48.855 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:53:48.855 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mgr.y", 2024-05-26T22:53:48.855 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD0u1NmX1RhNhAAo5mEOGzUyrdwFlB/XOXKnw==", 2024-05-26T22:53:48.855 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:53:48.855 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow *", 2024-05-26T22:53:48.855 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile mgr", 2024-05-26T22:53:48.855 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-05-26T22:53:48.855 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:53:48.855 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:53:48.855 INFO:teuthology.orchestra.run.smithi131.stdout: ] 2024-05-26T22:53:48.855 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-26T22:53:48.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.842+0000 7f28b5000640 1 -- 172.21.15.131:0/3486712854 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f288c06be30 msgr2=0x7f288c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:48.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.842+0000 7f28b5000640 1 --2- 172.21.15.131:0/3486712854 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f288c06be30 0x7f288c06e2f0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f28a0000f60 tx=0x7f28a0007c40 comp rx=0 tx=0).stop 2024-05-26T22:53:48.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.842+0000 7f28b5000640 1 -- 172.21.15.131:0/3486712854 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f28b0144340 msgr2=0x7f28b00cd1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:48.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.842+0000 7f28b5000640 1 --2- 172.21.15.131:0/3486712854 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f28b0144340 0x7f28b00cd1f0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f28a800cd70 tx=0x7f28a800a660 comp rx=0 tx=0).stop 2024-05-26T22:53:48.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.843+0000 7f28b5000640 1 -- 172.21.15.131:0/3486712854 shutdown_connections 2024-05-26T22:53:48.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.843+0000 7f28b5000640 1 --2- 172.21.15.131:0/3486712854 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f288c06be30 0x7f288c06e2f0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:48.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.843+0000 7f28b5000640 1 --2- 172.21.15.131:0/3486712854 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f28b0144340 0x7f28b00cd1f0 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:48.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.843+0000 7f28b5000640 1 --2- 172.21.15.131:0/3486712854 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28b0071b70 0x7f28b00c5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:48.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.843+0000 7f28b5000640 1 --2- 172.21.15.131:0/3486712854 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f28b006ebc0 0x7f28b01f4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:48.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.843+0000 7f28b5000640 1 -- 172.21.15.131:0/3486712854 >> 172.21.15.131:0/3486712854 conn(0x7f28b00cff40 msgr2=0x7f28b00cec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:48.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.843+0000 7f28b5000640 1 -- 172.21.15.131:0/3486712854 shutdown_connections 2024-05-26T22:53:48.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:48.843+0000 7f28b5000640 1 -- 172.21.15.131:0/3486712854 wait complete. 2024-05-26T22:53:48.862 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-26T22:53:49.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.055+0000 7f6863200640 1 Processor -- start 2024-05-26T22:53:49.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.055+0000 7f6863200640 1 -- start start 2024-05-26T22:53:49.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.057+0000 7f6863200640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f685c071d80 0x7f685c072160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:49.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.057+0000 7f6863200640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f685c0726a0 con 0x7f685c0ad450 2024-05-26T22:53:49.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.057+0000 7f6863200640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f685c0b5190 con 0x7f685c0ad8e0 2024-05-26T22:53:49.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.057+0000 7f6863200640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f685c06dee0 con 0x7f685c071d80 2024-05-26T22:53:49.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.057+0000 7f6853400640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f685c0ad450 0x7f685c06eca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:40408/0 (socket says 172.21.15.131:40408) 2024-05-26T22:53:49.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.057+0000 7f6853400640 1 -- 172.21.15.131:0/1328892687 learned_addr learned my addr 172.21.15.131:0/1328892687 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:53:49.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.057+0000 7f685be00640 1 --2- 172.21.15.131:0/1328892687 >> v2:172.21.15.187:3300/0 conn(0x7f685c071d80 0x7f685c072160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:49.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.058+0000 7f685b400640 1 -- 172.21.15.131:0/1328892687 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2728310918 0 0) 0x7f685c0726a0 con 0x7f685c0ad450 2024-05-26T22:53:49.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.058+0000 7f685b400640 1 -- 172.21.15.131:0/1328892687 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6840003520 con 0x7f685c0ad450 2024-05-26T22:53:49.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.058+0000 7f685b400640 1 -- 172.21.15.131:0/1328892687 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2961271932 0 0) 0x7f6840003520 con 0x7f685c0ad450 2024-05-26T22:53:49.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.058+0000 7f685b400640 1 -- 172.21.15.131:0/1328892687 >> v2:172.21.15.187:3300/0 conn(0x7f685c071d80 msgr2=0x7f685c072160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:49.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.058+0000 7f685b400640 1 --2- 172.21.15.131:0/1328892687 >> v2:172.21.15.187:3300/0 conn(0x7f685c071d80 0x7f685c072160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:49.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.058+0000 7f685b400640 1 -- 172.21.15.131:0/1328892687 >> v1:172.21.15.187:6789/0 conn(0x7f685c0ad8e0 legacy=0x7f685c0b43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:49.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.058+0000 7f685b400640 1 -- 172.21.15.131:0/1328892687 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f685c06e1c0 con 0x7f685c0ad450 2024-05-26T22:53:49.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.058+0000 7f685b400640 1 -- 172.21.15.131:0/1328892687 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7f6844001670 con 0x7f685c0ad450 2024-05-26T22:53:49.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.058+0000 7f685b400640 1 -- 172.21.15.131:0/1328892687 >> v1:172.21.15.131:6789/0 conn(0x7f685c0ad450 legacy=0x7f685c06eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:49.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.058+0000 7f685b400640 1 --2- 172.21.15.131:0/1328892687 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f68400039b0 0x7f6840003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:49.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.058+0000 7f685b400640 1 --2- 172.21.15.131:0/1328892687 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6840004800 0x7f6840004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:49.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.058+0000 7f685b400640 1 --2- 172.21.15.131:0/1328892687 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f68400051c0 0x7f684000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:49.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.058+0000 7f685b400640 1 -- 172.21.15.131:0/1328892687 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f685c06dee0 con 0x7f6840004800 2024-05-26T22:53:49.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.058+0000 7f685b400640 1 -- 172.21.15.131:0/1328892687 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f684000c160 con 0x7f68400051c0 2024-05-26T22:53:49.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.058+0000 7f685b400640 1 -- 172.21.15.131:0/1328892687 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f684000c360 con 0x7f68400039b0 2024-05-26T22:53:49.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.058+0000 7f6853400640 1 --2- 172.21.15.131:0/1328892687 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6840004800 0x7f6840004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:49.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.059+0000 7f6861200640 1 --2- 172.21.15.131:0/1328892687 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f68400051c0 0x7f684000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:49.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.059+0000 7f6853400640 1 -- 172.21.15.131:0/1328892687 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f68400039b0 msgr2=0x7f6840003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:49.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.059+0000 7f6853400640 1 --2- 172.21.15.131:0/1328892687 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f68400039b0 0x7f6840003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:49.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.059+0000 7f6853400640 1 -- 172.21.15.131:0/1328892687 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f68400051c0 msgr2=0x7f684000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:49.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.059+0000 7f6853400640 1 --2- 172.21.15.131:0/1328892687 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f68400051c0 0x7f684000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:49.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.059+0000 7f6853400640 1 -- 172.21.15.131:0/1328892687 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6840003520 con 0x7f6840004800 2024-05-26T22:53:49.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.059+0000 7f6861200640 1 --2- 172.21.15.131:0/1328892687 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f68400051c0 0x7f684000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:53:49.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.059+0000 7f6861200640 1 -- 172.21.15.131:0/1328892687 reap_dead start 2024-05-26T22:53:49.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.060+0000 7f6853400640 1 --2- 172.21.15.131:0/1328892687 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6840004800 0x7f6840004c80 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f684400c240 tx=0x7f6840005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=154b45c92c78a03e server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:49.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.060+0000 7f685b400640 1 -- 172.21.15.131:0/1328892687 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6844007cf0 con 0x7f6840004800 2024-05-26T22:53:49.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.060+0000 7f685b400640 1 -- 172.21.15.131:0/1328892687 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68440058d0 con 0x7f6840004800 2024-05-26T22:53:49.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.060+0000 7f685b400640 1 -- 172.21.15.131:0/1328892687 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6844005d20 con 0x7f6840004800 2024-05-26T22:53:49.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.061+0000 7f6863200640 1 -- 172.21.15.131:0/1328892687 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6840004800 msgr2=0x7f6840004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:49.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.061+0000 7f6863200640 1 --2- 172.21.15.131:0/1328892687 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6840004800 0x7f6840004c80 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f684400c240 tx=0x7f6840005830 comp rx=0 tx=0).stop 2024-05-26T22:53:49.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.061+0000 7f6863200640 1 -- 172.21.15.131:0/1328892687 shutdown_connections 2024-05-26T22:53:49.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.061+0000 7f6863200640 1 --2- 172.21.15.131:0/1328892687 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6840004800 0x7f6840004c80 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:49.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.061+0000 7f6863200640 1 -- 172.21.15.131:0/1328892687 >> 172.21.15.131:0/1328892687 conn(0x7f685c0b1e90 msgr2=0x7f685c0b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:49.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.061+0000 7f6863200640 1 -- 172.21.15.131:0/1328892687 shutdown_connections 2024-05-26T22:53:49.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.061+0000 7f6863200640 1 -- 172.21.15.131:0/1328892687 wait complete. 2024-05-26T22:53:49.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.063+0000 7f6863200640 1 Processor -- start 2024-05-26T22:53:49.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.063+0000 7f6863200640 1 -- start start 2024-05-26T22:53:49.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.064+0000 7f6863200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6840004800 0x7f685c091e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:49.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.064+0000 7f6863200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f685c144d50 0x7f685c1451b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:49.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.064+0000 7f6863200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f685c1456f0 0x7f685c0904c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:49.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.064+0000 7f6863200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f685c15e020 con 0x7f685c1456f0 2024-05-26T22:53:49.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.064+0000 7f6863200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f685c06e780 con 0x7f6840004800 2024-05-26T22:53:49.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.064+0000 7f6863200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f685c15e1a0 con 0x7f685c144d50 2024-05-26T22:53:49.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.064+0000 7f6861200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f685c1456f0 0x7f685c0904c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:49.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.064+0000 7f6861200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f685c1456f0 0x7f685c0904c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60282/0 (socket says 172.21.15.131:60282) 2024-05-26T22:53:49.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.064+0000 7f6861200640 1 -- 172.21.15.131:0/360032338 learned_addr learned my addr 172.21.15.131:0/360032338 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:53:49.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.064+0000 7f685be00640 1 --2- 172.21.15.131:0/360032338 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6840004800 0x7f685c091e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:49.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.064+0000 7f6853400640 1 --2- 172.21.15.131:0/360032338 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f685c144d50 0x7f685c1451b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:49.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.064+0000 7f6861200640 1 -- 172.21.15.131:0/360032338 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f685c144d50 msgr2=0x7f685c1451b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:49.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.064+0000 7f6861200640 1 --2- 172.21.15.131:0/360032338 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f685c144d50 0x7f685c1451b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:49.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.064+0000 7f6861200640 1 -- 172.21.15.131:0/360032338 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6840004800 msgr2=0x7f685c091e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:49.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.064+0000 7f6861200640 1 --2- 172.21.15.131:0/360032338 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6840004800 0x7f685c091e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:49.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.064+0000 7f6861200640 1 -- 172.21.15.131:0/360032338 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f685c145b70 con 0x7f685c1456f0 2024-05-26T22:53:49.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.065+0000 7f6861200640 1 --2- 172.21.15.131:0/360032338 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f685c1456f0 0x7f685c0904c0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f685c073710 tx=0x7f685400d9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:49.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.065+0000 7f6858c00640 1 -- 172.21.15.131:0/360032338 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6854018070 con 0x7f685c1456f0 2024-05-26T22:53:49.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.065+0000 7f6858c00640 1 -- 172.21.15.131:0/360032338 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6854003da0 con 0x7f685c1456f0 2024-05-26T22:53:49.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.065+0000 7f6858c00640 1 -- 172.21.15.131:0/360032338 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68540052b0 con 0x7f685c1456f0 2024-05-26T22:53:49.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.065+0000 7f6863200640 1 -- 172.21.15.131:0/360032338 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f685c0ce670 con 0x7f685c1456f0 2024-05-26T22:53:49.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.065+0000 7f6863200640 1 -- 172.21.15.131:0/360032338 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f685c0cec30 con 0x7f685c1456f0 2024-05-26T22:53:49.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.066+0000 7f6863200640 1 -- 172.21.15.131:0/360032338 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f685c0694e0 con 0x7f685c1456f0 2024-05-26T22:53:49.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.067+0000 7f6858c00640 1 -- 172.21.15.131:0/360032338 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f685401b070 con 0x7f685c1456f0 2024-05-26T22:53:49.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.068+0000 7f6858c00640 1 --2- 172.21.15.131:0/360032338 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f683806be60 0x7f683806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:49.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.068+0000 7f6858c00640 1 -- 172.21.15.131:0/360032338 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 4857+0+0 (secure 0 0 0) 0x7f685408a160 con 0x7f685c1456f0 2024-05-26T22:53:49.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.068+0000 7f685be00640 1 --2- 172.21.15.131:0/360032338 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f683806be60 0x7f683806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:49.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.069+0000 7f685be00640 1 --2- 172.21.15.131:0/360032338 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f683806be60 0x7f683806e320 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f684c0047b0 tx=0x7f684c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:49.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.073+0000 7f6858c00640 1 -- 172.21.15.131:0/360032338 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f6854058dc0 con 0x7f685c1456f0 2024-05-26T22:53:49.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.274+0000 7f6863200640 1 -- 172.21.15.131:0/360032338 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f685c0b12a0 con 0x7f685c1456f0 2024-05-26T22:53:49.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.276+0000 7f6858c00640 1 -- 172.21.15.131:0/360032338 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+12916 (secure 0 0 0) 0x7f68540586c0 con 0x7f685c1456f0 2024-05-26T22:53:49.278 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:53:49.278 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":78,"fsid":"46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f","created":"2024-05-26T22:47:57.288905+0000","modified":"2024-05-26T22:53:48.373076+0000","last_up_change":"2024-05-26T22:48:09.602207+0000","last_in_change":"2024-05-26T22:48:00.687341+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-26T22:48:10.463339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"814eaf62-a18d-494b-8e27-fd8fb88adea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6801","nonce":2350940051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6803","nonce":2350940051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6807","nonce":2350940051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6805","nonce":2350940051}]},"public_addr":"172.21.15.131:6801/2350940051","cluster_addr":"172.21.15.131:6803/2350940051","heartbeat_back_addr":"172.21.15.131:6807/2350940051","heartbeat_front_addr":"172.21.15.131:6805/2350940051","state":["exists","up"]},{"osd":1,"uuid":"42e66287-f6c6-4e26-af3c-6dfaa3536d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6825","nonce":1525528247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6827","nonce":1525528247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6831","nonce":1525528247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6829","nonce":1525528247}]},"public_addr":"172.21.15.131:6825/1525528247","cluster_addr":"172.21.15.131:6827/1525528247","heartbeat_back_addr":"172.21.15.131:6831/1525528247","heartbeat_front_addr":"172.21.15.131:6829/1525528247","state":["exists","up"]},{"osd":2,"uuid":"0da34673-04a2-4fd0-b57a-0ff2b679b043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6817","nonce":3627842210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6819","nonce":3627842210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6823","nonce":3627842210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6821","nonce":3627842210}]},"public_addr":"172.21.15.131:6817/3627842210","cluster_addr":"172.21.15.131:6819/3627842210","heartbeat_back_addr":"172.21.15.131:6823/3627842210","heartbeat_front_addr":"172.21.15.131:6821/3627842210","state":["exists","up"]},{"osd":3,"uuid":"cf117a27-d6b2-43d3-ae52-b47988bf8eed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6809","nonce":350790591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6811","nonce":350790591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6815","nonce":350790591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6813","nonce":350790591}]},"public_addr":"172.21.15.131:6809/350790591","cluster_addr":"172.21.15.131:6811/350790591","heartbeat_back_addr":"172.21.15.131:6815/350790591","heartbeat_front_addr":"172.21.15.131:6813/350790591","state":["exists","up"]},{"osd":4,"uuid":"a092ccfc-be89-4a84-88d5-0f781fa715eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6816","nonce":244560995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6819","nonce":244560995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6823","nonce":244560995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6821","nonce":244560995}]},"public_addr":"172.21.15.187:6816/244560995","cluster_addr":"172.21.15.187:6819/244560995","heartbeat_back_addr":"172.21.15.187:6823/244560995","heartbeat_front_addr":"172.21.15.187:6821/244560995","state":["exists","up"]},{"osd":5,"uuid":"7513ba69-dd00-4a35-b190-9e8cc3b2bdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6801","nonce":324619398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6803","nonce":324619398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6812","nonce":324619398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6807","nonce":324619398}]},"public_addr":"172.21.15.187:6801/324619398","cluster_addr":"172.21.15.187:6803/324619398","heartbeat_back_addr":"172.21.15.187:6812/324619398","heartbeat_front_addr":"172.21.15.187:6807/324619398","state":["exists","up"]},{"osd":6,"uuid":"50081810-c093-4d2f-a4d7-b7c695c5400d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6806","nonce":1391347786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6809","nonce":1391347786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6818","nonce":1391347786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6813","nonce":1391347786}]},"public_addr":"172.21.15.187:6806/1391347786","cluster_addr":"172.21.15.187:6809/1391347786","heartbeat_back_addr":"172.21.15.187:6818/1391347786","heartbeat_front_addr":"172.21.15.187:6813/1391347786","state":["exists","up"]},{"osd":7,"uuid":"775dae05-54f5-4ad4-bbce-92863c6ed2ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6825","nonce":1220065622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6827","nonce":1220065622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6831","nonce":1220065622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6829","nonce":1220065622}]},"public_addr":"172.21.15.187:6825/1220065622","cluster_addr":"172.21.15.187:6827/1220065622","heartbeat_back_addr":"172.21.15.187:6831/1220065622","heartbeat_front_addr":"172.21.15.187:6829/1220065622","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:05.983520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.299516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.067323+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.307748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.018742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.258775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.010912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.295355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-26T22:53:49.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.279+0000 7f6863200640 1 -- 172.21.15.131:0/360032338 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f683806be60 msgr2=0x7f683806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:49.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.279+0000 7f6863200640 1 --2- 172.21.15.131:0/360032338 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f683806be60 0x7f683806e320 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f684c0047b0 tx=0x7f684c008040 comp rx=0 tx=0).stop 2024-05-26T22:53:49.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.279+0000 7f6863200640 1 -- 172.21.15.131:0/360032338 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f685c1456f0 msgr2=0x7f685c0904c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:49.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.279+0000 7f6863200640 1 --2- 172.21.15.131:0/360032338 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f685c1456f0 0x7f685c0904c0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f685c073710 tx=0x7f685400d9d0 comp rx=0 tx=0).stop 2024-05-26T22:53:49.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.279+0000 7f6863200640 1 -- 172.21.15.131:0/360032338 shutdown_connections 2024-05-26T22:53:49.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.279+0000 7f6863200640 1 --2- 172.21.15.131:0/360032338 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f685c144d50 0x7f685c1451b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:49.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.279+0000 7f6863200640 1 --2- 172.21.15.131:0/360032338 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f683806be60 0x7f683806e320 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:49.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.279+0000 7f6863200640 1 --2- 172.21.15.131:0/360032338 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f685c1456f0 0x7f685c0904c0 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:49.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.279+0000 7f6863200640 1 --2- 172.21.15.131:0/360032338 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6840004800 0x7f685c091e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:49.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.279+0000 7f6863200640 1 -- 172.21.15.131:0/360032338 >> 172.21.15.131:0/360032338 conn(0x7f685c0b1e90 msgr2=0x7f685c14df30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:49.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.280+0000 7f6863200640 1 -- 172.21.15.131:0/360032338 shutdown_connections 2024-05-26T22:53:49.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.280+0000 7f6863200640 1 -- 172.21.15.131:0/360032338 wait complete. 2024-05-26T22:53:49.300 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-26T22:48:10.463339+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-26T22:53:49.300 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-26T22:53:49.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.509+0000 7f7f76a00640 1 Processor -- start 2024-05-26T22:53:49.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.510+0000 7f7f76a00640 1 -- start start 2024-05-26T22:53:49.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.511+0000 7f7f76a00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f7f7015e500 0x7f7f70162a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:49.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.511+0000 7f7f76a00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f7f7006d640 0x7f7f7006dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:49.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.511+0000 7f7f76a00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f700b6d60 con 0x7f7f700b6900 2024-05-26T22:53:49.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.511+0000 7f7f76a00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f7f7015e8e0 con 0x7f7f7006d640 2024-05-26T22:53:49.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.511+0000 7f7f76a00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f7f7006dff0 con 0x7f7f7015e500 2024-05-26T22:53:49.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.511+0000 7f7f74a00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f7f7006d640 0x7f7f7006dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:49.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.512+0000 7f7f74a00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f7f7006d640 0x7f7f7006dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60294/0 (socket says 172.21.15.131:60294) 2024-05-26T22:53:49.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.512+0000 7f7f74a00640 1 -- 172.21.15.131:0/3869932912 learned_addr learned my addr 172.21.15.131:0/3869932912 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:53:49.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.512+0000 7f7f6fe00640 1 --2- 172.21.15.131:0/3869932912 >> v2:172.21.15.187:3300/0 conn(0x7f7f7015e500 0x7f7f70162a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:49.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.512+0000 7f7f74a00640 1 -- 172.21.15.131:0/3869932912 >> v2:172.21.15.187:3300/0 conn(0x7f7f7015e500 msgr2=0x7f7f70162a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:49.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.512+0000 7f7f74a00640 1 --2- 172.21.15.131:0/3869932912 >> v2:172.21.15.187:3300/0 conn(0x7f7f7015e500 0x7f7f70162a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:49.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.512+0000 7f7f74a00640 1 -- 172.21.15.131:0/3869932912 >> v1:172.21.15.131:6789/0 conn(0x7f7f700b6900 legacy=0x7f7f700d0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:49.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.512+0000 7f7f74a00640 1 -- 172.21.15.131:0/3869932912 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f700713d0 con 0x7f7f7006d640 2024-05-26T22:53:49.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.513+0000 7f7f74a00640 1 --2- 172.21.15.131:0/3869932912 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7f7006d640 0x7f7f7006dab0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f7f700706f0 tx=0x7f7f6802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=587fa53de675df4c server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:49.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.513+0000 7f7f6ea00640 1 -- 172.21.15.131:0/3869932912 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f68037070 con 0x7f7f7006d640 2024-05-26T22:53:49.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.513+0000 7f7f6ea00640 1 -- 172.21.15.131:0/3869932912 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f68032440 con 0x7f7f7006d640 2024-05-26T22:53:49.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.513+0000 7f7f6ea00640 1 -- 172.21.15.131:0/3869932912 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f68032810 con 0x7f7f7006d640 2024-05-26T22:53:49.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.514+0000 7f7f76a00640 1 -- 172.21.15.131:0/3869932912 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7f7006d640 msgr2=0x7f7f7006dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:49.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.514+0000 7f7f76a00640 1 --2- 172.21.15.131:0/3869932912 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7f7006d640 0x7f7f7006dab0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f7f700706f0 tx=0x7f7f6802a270 comp rx=0 tx=0).stop 2024-05-26T22:53:49.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.514+0000 7f7f76a00640 1 -- 172.21.15.131:0/3869932912 shutdown_connections 2024-05-26T22:53:49.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.514+0000 7f7f76a00640 1 --2- 172.21.15.131:0/3869932912 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7f7006d640 0x7f7f7006dab0 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:49.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.514+0000 7f7f76a00640 1 --2- 172.21.15.131:0/3869932912 >> v2:172.21.15.187:3300/0 conn(0x7f7f7015e500 0x7f7f70162a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:49.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.514+0000 7f7f76a00640 1 -- 172.21.15.131:0/3869932912 >> 172.21.15.131:0/3869932912 conn(0x7f7f7006f6d0 msgr2=0x7f7f7006fae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:49.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.514+0000 7f7f76a00640 1 -- 172.21.15.131:0/3869932912 shutdown_connections 2024-05-26T22:53:49.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.514+0000 7f7f76a00640 1 -- 172.21.15.131:0/3869932912 wait complete. 2024-05-26T22:53:49.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.516+0000 7f7f76a00640 1 Processor -- start 2024-05-26T22:53:49.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.516+0000 7f7f76a00640 1 -- start start 2024-05-26T22:53:49.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.517+0000 7f7f76a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7f7006d640 0x7f7f7015aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:49.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.517+0000 7f7f76a00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7f700b6900 0x7f7f70145880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:49.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.517+0000 7f7f76a00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7f7015e500 0x7f7f701555a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:49.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.517+0000 7f7f76a00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f7f700b52a0 con 0x7f7f7006d640 2024-05-26T22:53:49.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.517+0000 7f7f76a00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7f700b5610 con 0x7f7f700b6900 2024-05-26T22:53:49.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.517+0000 7f7f76a00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f7f700d1580 con 0x7f7f7015e500 2024-05-26T22:53:49.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.517+0000 7f7f6fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7f7006d640 0x7f7f7015aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:49.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.517+0000 7f7f6fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7f7006d640 0x7f7f7015aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60296/0 (socket says 172.21.15.131:60296) 2024-05-26T22:53:49.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.517+0000 7f7f6fe00640 1 -- 172.21.15.131:0/2369672037 learned_addr learned my addr 172.21.15.131:0/2369672037 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:53:49.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.517+0000 7f7f6f400640 1 --2- 172.21.15.131:0/2369672037 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7f700b6900 0x7f7f70145880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:49.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.517+0000 7f7f74a00640 1 --2- 172.21.15.131:0/2369672037 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7f7015e500 0x7f7f701555a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:49.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.517+0000 7f7f6fe00640 1 -- 172.21.15.131:0/2369672037 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7f7015e500 msgr2=0x7f7f701555a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:49.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.517+0000 7f7f6fe00640 1 --2- 172.21.15.131:0/2369672037 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7f7015e500 0x7f7f701555a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:49.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.517+0000 7f7f6fe00640 1 -- 172.21.15.131:0/2369672037 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7f700b6900 msgr2=0x7f7f70145880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:49.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.517+0000 7f7f6fe00640 1 --2- 172.21.15.131:0/2369672037 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7f700b6900 0x7f7f70145880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:49.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.517+0000 7f7f6fe00640 1 -- 172.21.15.131:0/2369672037 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f7015a660 con 0x7f7f7006d640 2024-05-26T22:53:49.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.517+0000 7f7f6f400640 1 --2- 172.21.15.131:0/2369672037 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7f700b6900 0x7f7f70145880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:53:49.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.518+0000 7f7f6fe00640 1 --2- 172.21.15.131:0/2369672037 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7f7006d640 0x7f7f7015aeb0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f7f6000c7f0 tx=0x7f7f6000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:49.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.518+0000 7f7f67e00640 1 -- 172.21.15.131:0/2369672037 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f60015070 con 0x7f7f7006d640 2024-05-26T22:53:49.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.518+0000 7f7f67e00640 1 -- 172.21.15.131:0/2369672037 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f60004220 con 0x7f7f7006d640 2024-05-26T22:53:49.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.518+0000 7f7f67e00640 1 -- 172.21.15.131:0/2369672037 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f60007620 con 0x7f7f7006d640 2024-05-26T22:53:49.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.518+0000 7f7f76a00640 1 -- 172.21.15.131:0/2369672037 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f7015b590 con 0x7f7f7006d640 2024-05-26T22:53:49.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.518+0000 7f7f76a00640 1 -- 172.21.15.131:0/2369672037 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f70145dc0 con 0x7f7f7006d640 2024-05-26T22:53:49.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.519+0000 7f7f76a00640 1 -- 172.21.15.131:0/2369672037 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f70145ff0 con 0x7f7f7006d640 2024-05-26T22:53:49.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.520+0000 7f7f67e00640 1 -- 172.21.15.131:0/2369672037 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7f600079d0 con 0x7f7f7006d640 2024-05-26T22:53:49.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.521+0000 7f7f67e00640 1 --2- 172.21.15.131:0/2369672037 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f7f4c06be30 0x7f7f4c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:49.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.521+0000 7f7f67e00640 1 -- 172.21.15.131:0/2369672037 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 4857+0+0 (secure 0 0 0) 0x7f7f6008ae40 con 0x7f7f7006d640 2024-05-26T22:53:49.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.521+0000 7f7f6f400640 1 --2- 172.21.15.131:0/2369672037 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f7f4c06be30 0x7f7f4c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:49.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.522+0000 7f7f6f400640 1 --2- 172.21.15.131:0/2369672037 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f7f4c06be30 0x7f7f4c06e2f0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f7f70146850 tx=0x7f7f58009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:49.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.525+0000 7f7f67e00640 1 -- 172.21.15.131:0/2369672037 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f7f60059b30 con 0x7f7f7006d640 2024-05-26T22:53:49.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.710+0000 7f7f76a00640 1 -- 172.21.15.131:0/2369672037 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7f700695c0 con 0x7f7f7006d640 2024-05-26T22:53:49.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.711+0000 7f7f67e00640 1 -- 172.21.15.131:0/2369672037 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v78) ==== 93+0+10 (secure 0 0 0) 0x7f7f60059430 con 0x7f7f7006d640 2024-05-26T22:53:49.711 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2024-05-26T22:53:49.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.714+0000 7f7f76a00640 1 -- 172.21.15.131:0/2369672037 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f7f4c06be30 msgr2=0x7f7f4c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:49.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.714+0000 7f7f76a00640 1 --2- 172.21.15.131:0/2369672037 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f7f4c06be30 0x7f7f4c06e2f0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f7f70146850 tx=0x7f7f58009040 comp rx=0 tx=0).stop 2024-05-26T22:53:49.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.714+0000 7f7f76a00640 1 -- 172.21.15.131:0/2369672037 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7f7006d640 msgr2=0x7f7f7015aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:49.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.714+0000 7f7f76a00640 1 --2- 172.21.15.131:0/2369672037 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7f7006d640 0x7f7f7015aeb0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f7f6000c7f0 tx=0x7f7f6000ccb0 comp rx=0 tx=0).stop 2024-05-26T22:53:49.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.714+0000 7f7f76a00640 1 -- 172.21.15.131:0/2369672037 shutdown_connections 2024-05-26T22:53:49.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.714+0000 7f7f76a00640 1 --2- 172.21.15.131:0/2369672037 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7f7015e500 0x7f7f701555a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:49.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.714+0000 7f7f76a00640 1 --2- 172.21.15.131:0/2369672037 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f7f4c06be30 0x7f7f4c06e2f0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:49.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.714+0000 7f7f76a00640 1 --2- 172.21.15.131:0/2369672037 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7f700b6900 0x7f7f70145880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:49.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.714+0000 7f7f76a00640 1 --2- 172.21.15.131:0/2369672037 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7f7006d640 0x7f7f7015aeb0 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:49.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.714+0000 7f7f76a00640 1 -- 172.21.15.131:0/2369672037 >> 172.21.15.131:0/2369672037 conn(0x7f7f7006f6d0 msgr2=0x7f7f701608b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:49.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.714+0000 7f7f76a00640 1 -- 172.21.15.131:0/2369672037 shutdown_connections 2024-05-26T22:53:49.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.714+0000 7f7f76a00640 1 -- 172.21.15.131:0/2369672037 wait complete. 2024-05-26T22:53:49.731 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-26T22:53:49.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.928+0000 7f9c48800640 1 Processor -- start 2024-05-26T22:53:49.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.929+0000 7f9c48800640 1 -- start start 2024-05-26T22:53:49.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.930+0000 7f9c48800640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f9c4006f600 0x7f9c4006d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:49.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.930+0000 7f9c48800640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f9c4006daf0 0x7f9c400ad950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:49.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.930+0000 7f9c48800640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c400ad170 con 0x7f9c40162780 2024-05-26T22:53:49.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.930+0000 7f9c48800640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f9c4006df60 con 0x7f9c4006f600 2024-05-26T22:53:49.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.930+0000 7f9c48800640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f9c400b42b0 con 0x7f9c4006daf0 2024-05-26T22:53:49.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.930+0000 7f9c45400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f9c4006f600 0x7f9c4006d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:49.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.931+0000 7f9c45400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f9c4006f600 0x7f9c4006d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60298/0 (socket says 172.21.15.131:60298) 2024-05-26T22:53:49.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.931+0000 7f9c45400640 1 -- 172.21.15.131:0/4168399541 learned_addr learned my addr 172.21.15.131:0/4168399541 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:53:49.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.931+0000 7f9c46800640 1 --2- 172.21.15.131:0/4168399541 >> v2:172.21.15.187:3300/0 conn(0x7f9c4006daf0 0x7f9c400ad950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:49.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.931+0000 7f9c45400640 1 -- 172.21.15.131:0/4168399541 >> v2:172.21.15.187:3300/0 conn(0x7f9c4006daf0 msgr2=0x7f9c400ad950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:49.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.931+0000 7f9c45400640 1 --2- 172.21.15.131:0/4168399541 >> v2:172.21.15.187:3300/0 conn(0x7f9c4006daf0 0x7f9c400ad950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:49.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.931+0000 7f9c45400640 1 -- 172.21.15.131:0/4168399541 >> v1:172.21.15.131:6789/0 conn(0x7f9c40162780 legacy=0x7f9c400b3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:49.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.931+0000 7f9c45400640 1 -- 172.21.15.131:0/4168399541 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c400b4510 con 0x7f9c4006f600 2024-05-26T22:53:49.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.932+0000 7f9c45400640 1 --2- 172.21.15.131:0/4168399541 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c4006f600 0x7f9c4006d5b0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f9c4006d3b0 tx=0x7f9c3002a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5ae76c1cb3661a1 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:49.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.932+0000 7f9c44a00640 1 -- 172.21.15.131:0/4168399541 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c30037070 con 0x7f9c4006f600 2024-05-26T22:53:49.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.932+0000 7f9c44a00640 1 -- 172.21.15.131:0/4168399541 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c30032440 con 0x7f9c4006f600 2024-05-26T22:53:49.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.932+0000 7f9c44a00640 1 -- 172.21.15.131:0/4168399541 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c30032810 con 0x7f9c4006f600 2024-05-26T22:53:49.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.933+0000 7f9c48800640 1 -- 172.21.15.131:0/4168399541 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c4006f600 msgr2=0x7f9c4006d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:49.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.933+0000 7f9c48800640 1 --2- 172.21.15.131:0/4168399541 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c4006f600 0x7f9c4006d5b0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f9c4006d3b0 tx=0x7f9c3002a2a0 comp rx=0 tx=0).stop 2024-05-26T22:53:49.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.933+0000 7f9c48800640 1 -- 172.21.15.131:0/4168399541 shutdown_connections 2024-05-26T22:53:49.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.933+0000 7f9c48800640 1 --2- 172.21.15.131:0/4168399541 >> v2:172.21.15.187:3300/0 conn(0x7f9c4006daf0 0x7f9c400ad950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:49.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.933+0000 7f9c48800640 1 --2- 172.21.15.131:0/4168399541 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c4006f600 0x7f9c4006d5b0 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:49.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.933+0000 7f9c48800640 1 -- 172.21.15.131:0/4168399541 >> 172.21.15.131:0/4168399541 conn(0x7f9c400b1e90 msgr2=0x7f9c400b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:49.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.933+0000 7f9c48800640 1 -- 172.21.15.131:0/4168399541 shutdown_connections 2024-05-26T22:53:49.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.933+0000 7f9c48800640 1 -- 172.21.15.131:0/4168399541 wait complete. 2024-05-26T22:53:49.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.935+0000 7f9c48800640 1 Processor -- start 2024-05-26T22:53:49.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.935+0000 7f9c48800640 1 -- start start 2024-05-26T22:53:49.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.936+0000 7f9c48800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9c4006daf0 0x7f9c4014c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:49.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.936+0000 7f9c48800640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9c4006f600 0x7f9c400cf120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:49.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.936+0000 7f9c48800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c40162780 0x7f9c400b0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:49.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.936+0000 7f9c48800640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f9c40162b60 con 0x7f9c40162780 2024-05-26T22:53:49.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.936+0000 7f9c48800640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9c400b4ad0 con 0x7f9c4006f600 2024-05-26T22:53:49.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.936+0000 7f9c48800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9c400ade90 con 0x7f9c4006daf0 2024-05-26T22:53:49.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.936+0000 7f9c46800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c40162780 0x7f9c400b0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:49.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.936+0000 7f9c46800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c40162780 0x7f9c400b0ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60306/0 (socket says 172.21.15.131:60306) 2024-05-26T22:53:49.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.936+0000 7f9c46800640 1 -- 172.21.15.131:0/1627052285 learned_addr learned my addr 172.21.15.131:0/1627052285 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:53:49.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.937+0000 7f9c45400640 1 --2- 172.21.15.131:0/1627052285 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9c4006f600 0x7f9c400cf120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:49.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.937+0000 7f9c46800640 1 -- 172.21.15.131:0/1627052285 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9c4006daf0 msgr2=0x7f9c4014c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:49.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.937+0000 7f9c45e00640 1 --2- 172.21.15.131:0/1627052285 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9c4006daf0 0x7f9c4014c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:49.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.937+0000 7f9c46800640 1 --2- 172.21.15.131:0/1627052285 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9c4006daf0 0x7f9c4014c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:49.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.937+0000 7f9c46800640 1 -- 172.21.15.131:0/1627052285 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9c4006f600 msgr2=0x7f9c400cf120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:49.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.937+0000 7f9c46800640 1 --2- 172.21.15.131:0/1627052285 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9c4006f600 0x7f9c400cf120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:49.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.937+0000 7f9c46800640 1 -- 172.21.15.131:0/1627052285 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c4015a720 con 0x7f9c40162780 2024-05-26T22:53:49.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.937+0000 7f9c45e00640 1 --2- 172.21.15.131:0/1627052285 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9c4006daf0 0x7f9c4014c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:53:49.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.937+0000 7f9c46800640 1 --2- 172.21.15.131:0/1627052285 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c40162780 0x7f9c400b0ef0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c00d800 tx=0x7f9c3c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:49.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.937+0000 7f9c36000640 1 -- 172.21.15.131:0/1627052285 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c3c00b6c0 con 0x7f9c40162780 2024-05-26T22:53:49.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.938+0000 7f9c36000640 1 -- 172.21.15.131:0/1627052285 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c3c019070 con 0x7f9c40162780 2024-05-26T22:53:49.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.938+0000 7f9c48800640 1 -- 172.21.15.131:0/1627052285 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c4015aa00 con 0x7f9c40162780 2024-05-26T22:53:49.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.938+0000 7f9c36000640 1 -- 172.21.15.131:0/1627052285 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c3c00bac0 con 0x7f9c40162780 2024-05-26T22:53:49.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.938+0000 7f9c48800640 1 -- 172.21.15.131:0/1627052285 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c4015afc0 con 0x7f9c40162780 2024-05-26T22:53:49.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.939+0000 7f9c48800640 1 -- 172.21.15.131:0/1627052285 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c400b40d0 con 0x7f9c40162780 2024-05-26T22:53:49.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.940+0000 7f9c36000640 1 -- 172.21.15.131:0/1627052285 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9c3c00bc60 con 0x7f9c40162780 2024-05-26T22:53:49.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.941+0000 7f9c36000640 1 --2- 172.21.15.131:0/1627052285 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9c2406be30 0x7f9c2406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:49.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.941+0000 7f9c36000640 1 -- 172.21.15.131:0/1627052285 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 4857+0+0 (secure 0 0 0) 0x7f9c3c08b0d0 con 0x7f9c40162780 2024-05-26T22:53:49.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.941+0000 7f9c45e00640 1 --2- 172.21.15.131:0/1627052285 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9c2406be30 0x7f9c2406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:49.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.941+0000 7f9c45e00640 1 --2- 172.21.15.131:0/1627052285 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9c2406be30 0x7f9c2406e2f0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f9c38001cd0 tx=0x7f9c38009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:49.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:49.945+0000 7f9c36000640 1 -- 172.21.15.131:0/1627052285 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f9c400b40d0 con 0x7f9c40162780 2024-05-26T22:53:50.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:50.141+0000 7f9c48800640 1 -- 172.21.15.131:0/1627052285 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f9c400694e0 con 0x7f9c40162780 2024-05-26T22:53:50.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:50.141+0000 7f9c36000640 1 -- 172.21.15.131:0/1627052285 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v78) ==== 84+0+65 (secure 0 0 0) 0x7f9c3c059ca0 con 0x7f9c40162780 2024-05-26T22:53:50.142 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:53:50.142 INFO:teuthology.orchestra.run.smithi131.stdout:[ 2024-05-26T22:53:50.142 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:53:50.142 INFO:teuthology.orchestra.run.smithi131.stdout: "poolnum": 1, 2024-05-26T22:53:50.142 INFO:teuthology.orchestra.run.smithi131.stdout: "poolname": ".mgr" 2024-05-26T22:53:50.142 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:53:50.142 INFO:teuthology.orchestra.run.smithi131.stdout:] 2024-05-26T22:53:50.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:50.144+0000 7f9c48800640 1 -- 172.21.15.131:0/1627052285 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9c2406be30 msgr2=0x7f9c2406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:50.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:50.144+0000 7f9c48800640 1 --2- 172.21.15.131:0/1627052285 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9c2406be30 0x7f9c2406e2f0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f9c38001cd0 tx=0x7f9c38009040 comp rx=0 tx=0).stop 2024-05-26T22:53:50.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:50.144+0000 7f9c48800640 1 -- 172.21.15.131:0/1627052285 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c40162780 msgr2=0x7f9c400b0ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:50.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:50.144+0000 7f9c48800640 1 --2- 172.21.15.131:0/1627052285 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c40162780 0x7f9c400b0ef0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c00d800 tx=0x7f9c3c00dcc0 comp rx=0 tx=0).stop 2024-05-26T22:53:50.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:50.144+0000 7f9c48800640 1 -- 172.21.15.131:0/1627052285 shutdown_connections 2024-05-26T22:53:50.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:50.144+0000 7f9c48800640 1 --2- 172.21.15.131:0/1627052285 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9c2406be30 0x7f9c2406e2f0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:50.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:50.144+0000 7f9c48800640 1 --2- 172.21.15.131:0/1627052285 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c40162780 0x7f9c400b0ef0 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:50.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:50.144+0000 7f9c48800640 1 --2- 172.21.15.131:0/1627052285 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9c4006f600 0x7f9c400cf120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:50.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:50.144+0000 7f9c48800640 1 --2- 172.21.15.131:0/1627052285 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9c4006daf0 0x7f9c4014c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:50.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:50.144+0000 7f9c48800640 1 -- 172.21.15.131:0/1627052285 >> 172.21.15.131:0/1627052285 conn(0x7f9c400b1e90 msgr2=0x7f9c400b0050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:50.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:50.144+0000 7f9c48800640 1 -- 172.21.15.131:0/1627052285 shutdown_connections 2024-05-26T22:53:50.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:50.144+0000 7f9c48800640 1 -- 172.21.15.131:0/1627052285 wait complete. 2024-05-26T22:53:50.161 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-26T22:53:50.161 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-26T22:53:50.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:50.368+0000 7f2bca400640 1 Processor -- start 2024-05-26T22:53:50.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:50.368+0000 7f2bca400640 1 -- start start 2024-05-26T22:53:50.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:50.369+0000 7f2bca400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f2bc40cf8f0 0x7f2bc4072250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:50.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:50.369+0000 7f2bca400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2bc406ef70 0x7f2bc40d07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:50.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:50.369+0000 7f2bca400640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2bc40ccb70 con 0x7f2bc40d0360 2024-05-26T22:53:50.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:50.369+0000 7f2bca400640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f2bc40d0d00 con 0x7f2bc40cf8f0 2024-05-26T22:53:50.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:50.369+0000 7f2bca400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f2bc40cce50 con 0x7f2bc406ef70 2024-05-26T22:53:50.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:50.370+0000 7f2bc3400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f2bc40cf8f0 0x7f2bc4072250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:50.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:50.370+0000 7f2bc3400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f2bc40cf8f0 0x7f2bc4072250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60312/0 (socket says 172.21.15.131:60312) 2024-05-26T22:53:50.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:50.370+0000 7f2bc2a00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f2bc40d0360 0x7f2bc406e860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:40432/0 (socket says 172.21.15.131:40432) 2024-05-26T22:53:50.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:50.370+0000 7f2bc3400640 1 -- 172.21.15.131:0/3059029105 learned_addr learned my addr 172.21.15.131:0/3059029105 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:53:50.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:50.370+0000 7f2bc3e00640 1 --2- 172.21.15.131:0/3059029105 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2bc406ef70 0x7f2bc40d07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:50.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:50.370+0000 7f2bc3400640 1 -- 172.21.15.131:0/3059029105 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2bc406ef70 msgr2=0x7f2bc40d07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:50.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:50.370+0000 7f2bc3400640 1 --2- 172.21.15.131:0/3059029105 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2bc406ef70 0x7f2bc40d07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:50.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:50.370+0000 7f2bc3400640 1 -- 172.21.15.131:0/3059029105 >> v1:172.21.15.131:6789/0 conn(0x7f2bc40d0360 legacy=0x7f2bc406e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:50.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:50.370+0000 7f2bc3400640 1 -- 172.21.15.131:0/3059029105 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bc40cd130 con 0x7f2bc40cf8f0 2024-05-26T22:53:50.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:50.371+0000 7f2bc3400640 1 --2- 172.21.15.131:0/3059029105 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2bc40cf8f0 0x7f2bc4072250 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f2bb4009870 tx=0x7f2bb402a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd9dce0e50126f81 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:50.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:50.371+0000 7f2bc2000640 1 -- 172.21.15.131:0/3059029105 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb4037070 con 0x7f2bc40cf8f0 2024-05-26T22:53:50.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:50.371+0000 7f2bc2000640 1 -- 172.21.15.131:0/3059029105 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bb4032440 con 0x7f2bc40cf8f0 2024-05-26T22:53:50.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:50.371+0000 7f2bc2000640 1 -- 172.21.15.131:0/3059029105 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb4032810 con 0x7f2bc40cf8f0 2024-05-26T22:53:50.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:50.372+0000 7f2bca400640 1 -- 172.21.15.131:0/3059029105 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2bc40cf8f0 msgr2=0x7f2bc4072250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:50.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:50.372+0000 7f2bca400640 1 --2- 172.21.15.131:0/3059029105 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2bc40cf8f0 0x7f2bc4072250 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f2bb4009870 tx=0x7f2bb402a250 comp rx=0 tx=0).stop 2024-05-26T22:53:50.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:50.372+0000 7f2bca400640 1 -- 172.21.15.131:0/3059029105 shutdown_connections 2024-05-26T22:53:50.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:50.372+0000 7f2bca400640 1 --2- 172.21.15.131:0/3059029105 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2bc406ef70 0x7f2bc40d07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:50.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:50.372+0000 7f2bca400640 1 --2- 172.21.15.131:0/3059029105 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2bc40cf8f0 0x7f2bc4072250 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:50.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:50.372+0000 7f2bca400640 1 -- 172.21.15.131:0/3059029105 >> 172.21.15.131:0/3059029105 conn(0x7f2bc40b2110 msgr2=0x7f2bc40b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:50.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:50.372+0000 7f2bca400640 1 -- 172.21.15.131:0/3059029105 shutdown_connections 2024-05-26T22:53:50.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:50.372+0000 7f2bca400640 1 -- 172.21.15.131:0/3059029105 wait complete. 2024-05-26T22:53:50.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:50.374+0000 7f2bca400640 1 Processor -- start 2024-05-26T22:53:50.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:50.374+0000 7f2bca400640 1 -- start start 2024-05-26T22:53:50.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:50.375+0000 7f2bca400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2bc406ef70 0x7f2bc414fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:50.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:50.375+0000 7f2bca400640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2bc40cf8f0 0x7f2bc40ad9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:50.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:50.375+0000 7f2bca400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2bc40d0360 0x7f2bc40c4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:50.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:50.375+0000 7f2bca400640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f2bc415ffd0 con 0x7f2bc406ef70 2024-05-26T22:53:50.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:50.375+0000 7f2bca400640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2bc40cfcd0 con 0x7f2bc40cf8f0 2024-05-26T22:53:50.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:50.375+0000 7f2bca400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f2bc4160150 con 0x7f2bc40d0360 2024-05-26T22:53:50.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:50.375+0000 7f2bc3400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2bc406ef70 0x7f2bc414fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:50.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:50.375+0000 7f2bc3400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2bc406ef70 0x7f2bc414fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60322/0 (socket says 172.21.15.131:60322) 2024-05-26T22:53:50.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:50.375+0000 7f2bc3400640 1 -- 172.21.15.131:0/2573269255 learned_addr learned my addr 172.21.15.131:0/2573269255 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:53:50.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:50.376+0000 7f2bc3400640 1 -- 172.21.15.131:0/2573269255 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2bc40d0360 msgr2=0x7f2bc40c4f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:50.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:50.376+0000 7f2bc3400640 1 --2- 172.21.15.131:0/2573269255 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2bc40d0360 0x7f2bc40c4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:50.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:50.376+0000 7f2bc3400640 1 -- 172.21.15.131:0/2573269255 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2bc40cf8f0 msgr2=0x7f2bc40ad9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:50.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:50.376+0000 7f2bc2a00640 1 --2- 172.21.15.131:0/2573269255 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2bc40cf8f0 0x7f2bc40ad9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:50.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:50.376+0000 7f2bc3400640 1 --2- 172.21.15.131:0/2573269255 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2bc40cf8f0 0x7f2bc40ad9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:50.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:50.376+0000 7f2bc3400640 1 -- 172.21.15.131:0/2573269255 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bc40ae3a0 con 0x7f2bc406ef70 2024-05-26T22:53:50.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:50.376+0000 7f2bc2a00640 1 --2- 172.21.15.131:0/2573269255 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2bc40cf8f0 0x7f2bc40ad9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:53:50.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:50.376+0000 7f2bc3400640 1 --2- 172.21.15.131:0/2573269255 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2bc406ef70 0x7f2bc414fbe0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f2bb402a290 tx=0x7f2bb40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:50.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:50.377+0000 7f2bbb400640 1 -- 172.21.15.131:0/2573269255 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb4041020 con 0x7f2bc406ef70 2024-05-26T22:53:50.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:50.377+0000 7f2bbb400640 1 -- 172.21.15.131:0/2573269255 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bb40306c0 con 0x7f2bc406ef70 2024-05-26T22:53:50.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:50.377+0000 7f2bbb400640 1 -- 172.21.15.131:0/2573269255 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb4037040 con 0x7f2bc406ef70 2024-05-26T22:53:50.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:50.377+0000 7f2bca400640 1 -- 172.21.15.131:0/2573269255 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bc40ae680 con 0x7f2bc406ef70 2024-05-26T22:53:50.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:50.377+0000 7f2bca400640 1 -- 172.21.15.131:0/2573269255 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bc4158c40 con 0x7f2bc406ef70 2024-05-26T22:53:50.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:50.378+0000 7f2bca400640 1 -- 172.21.15.131:0/2573269255 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bc4068440 con 0x7f2bc406ef70 2024-05-26T22:53:50.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:50.379+0000 7f2bbb400640 1 -- 172.21.15.131:0/2573269255 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2bb4044020 con 0x7f2bc406ef70 2024-05-26T22:53:50.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:50.380+0000 7f2bbb400640 1 --2- 172.21.15.131:0/2573269255 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f2ba406be30 0x7f2ba406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:50.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:50.380+0000 7f2bbb400640 1 -- 172.21.15.131:0/2573269255 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 4857+0+0 (secure 0 0 0) 0x7f2bb40ab370 con 0x7f2bc406ef70 2024-05-26T22:53:50.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:50.380+0000 7f2bc2a00640 1 --2- 172.21.15.131:0/2573269255 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f2ba406be30 0x7f2ba406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:50.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:50.381+0000 7f2bc2a00640 1 --2- 172.21.15.131:0/2573269255 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f2ba406be30 0x7f2ba406e2f0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f2bc40aedb0 tx=0x7f2bb0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:50.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:50.385+0000 7f2bbb400640 1 -- 172.21.15.131:0/2573269255 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f2bb404b030 con 0x7f2bc406ef70 2024-05-26T22:53:50.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:50.571+0000 7f2bca400640 1 -- 172.21.15.131:0/2573269255 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f2bc40ce970 con 0x7f2bc406ef70 2024-05-26T22:53:51.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:51.381+0000 7f2bbb400640 1 -- 172.21.15.131:0/2573269255 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v79) ==== 138+0+0 (secure 0 0 0) 0x7f2bb407a060 con 0x7f2bc406ef70 2024-05-26T22:53:51.381 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_metadata' created 2024-05-26T22:53:51.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:51.385+0000 7f2bca400640 1 -- 172.21.15.131:0/2573269255 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f2ba406be30 msgr2=0x7f2ba406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:51.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:51.385+0000 7f2bca400640 1 --2- 172.21.15.131:0/2573269255 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f2ba406be30 0x7f2ba406e2f0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f2bc40aedb0 tx=0x7f2bb0009040 comp rx=0 tx=0).stop 2024-05-26T22:53:51.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:51.385+0000 7f2bca400640 1 -- 172.21.15.131:0/2573269255 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2bc406ef70 msgr2=0x7f2bc414fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:51.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:51.385+0000 7f2bca400640 1 --2- 172.21.15.131:0/2573269255 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2bc406ef70 0x7f2bc414fbe0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f2bb402a290 tx=0x7f2bb40046c0 comp rx=0 tx=0).stop 2024-05-26T22:53:51.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:51.386+0000 7f2bca400640 1 -- 172.21.15.131:0/2573269255 shutdown_connections 2024-05-26T22:53:51.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:51.386+0000 7f2bca400640 1 --2- 172.21.15.131:0/2573269255 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2bc40d0360 0x7f2bc40c4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:51.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:51.386+0000 7f2bca400640 1 --2- 172.21.15.131:0/2573269255 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f2ba406be30 0x7f2ba406e2f0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:51.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:51.386+0000 7f2bca400640 1 --2- 172.21.15.131:0/2573269255 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2bc40cf8f0 0x7f2bc40ad9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:51.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:51.386+0000 7f2bca400640 1 --2- 172.21.15.131:0/2573269255 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2bc406ef70 0x7f2bc414fbe0 secure :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f2bb402a290 tx=0x7f2bb40046c0 comp rx=0 tx=0).stop 2024-05-26T22:53:51.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:51.386+0000 7f2bca400640 1 -- 172.21.15.131:0/2573269255 >> 172.21.15.131:0/2573269255 conn(0x7f2bc40b2110 msgr2=0x7f2bc406b310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:51.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:51.387+0000 7f2bca400640 1 -- 172.21.15.131:0/2573269255 shutdown_connections 2024-05-26T22:53:51.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:51.387+0000 7f2bca400640 1 -- 172.21.15.131:0/2573269255 wait complete. 2024-05-26T22:53:51.413 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-26T22:53:51.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:51.607+0000 7fee3f200640 1 Processor -- start 2024-05-26T22:53:51.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:51.607+0000 7fee3f200640 1 -- start start 2024-05-26T22:53:51.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:51.609+0000 7fee3f200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fee4015b730 0x7fee4015bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:51.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:51.609+0000 7fee3f200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fee401400c0 0x7fee401404f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:51.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:51.609+0000 7fee3f200640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee4013e820 con 0x7fee4006d250 2024-05-26T22:53:51.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:51.609+0000 7fee3f200640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fee40071260 con 0x7fee4015b730 2024-05-26T22:53:51.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:51.609+0000 7fee3f200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fee4015c4a0 con 0x7fee401400c0 2024-05-26T22:53:51.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:51.609+0000 7fee3de00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fee4006d250 0x7fee4006d630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:40444/0 (socket says 172.21.15.131:40444) 2024-05-26T22:53:51.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:51.609+0000 7fee3de00640 1 -- 172.21.15.131:0/803596675 learned_addr learned my addr 172.21.15.131:0/803596675 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:53:51.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:51.609+0000 7fee3d400640 1 --2- 172.21.15.131:0/803596675 >> v2:172.21.15.131:3300/0 conn(0x7fee4015b730 0x7fee4015bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:51.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:51.610+0000 7fee3e800640 1 --2- 172.21.15.131:0/803596675 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fee401400c0 0x7fee401404f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:51.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:51.610+0000 7fee3ca00640 1 -- 172.21.15.131:0/803596675 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4038850298 0 0) 0x7fee4013e820 con 0x7fee4006d250 2024-05-26T22:53:51.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:51.610+0000 7fee3ca00640 1 -- 172.21.15.131:0/803596675 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fee24003520 con 0x7fee4006d250 2024-05-26T22:53:51.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:51.610+0000 7fee3d400640 1 -- 172.21.15.131:0/803596675 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fee401400c0 msgr2=0x7fee401404f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:51.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:51.610+0000 7fee3d400640 1 --2- 172.21.15.131:0/803596675 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fee401400c0 0x7fee401404f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:51.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:51.610+0000 7fee3d400640 1 -- 172.21.15.131:0/803596675 >> v1:172.21.15.131:6789/0 conn(0x7fee4006d250 legacy=0x7fee4006d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:51.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:51.610+0000 7fee3d400640 1 -- 172.21.15.131:0/803596675 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee400714b0 con 0x7fee4015b730 2024-05-26T22:53:51.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:51.611+0000 7fee3d400640 1 --2- 172.21.15.131:0/803596675 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee4015b730 0x7fee4015bbc0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7fee2c009d90 tx=0x7fee2c02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2bfdfc35afb8426 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:51.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:51.611+0000 7fee3ca00640 1 -- 172.21.15.131:0/803596675 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee2c007740 con 0x7fee4015b730 2024-05-26T22:53:51.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:51.611+0000 7fee3ca00640 1 -- 172.21.15.131:0/803596675 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee2c0078e0 con 0x7fee4015b730 2024-05-26T22:53:51.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:51.611+0000 7fee3ca00640 1 -- 172.21.15.131:0/803596675 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee2c007cb0 con 0x7fee4015b730 2024-05-26T22:53:51.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:51.612+0000 7fee3f200640 1 -- 172.21.15.131:0/803596675 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee4015b730 msgr2=0x7fee4015bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:51.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:51.612+0000 7fee3f200640 1 --2- 172.21.15.131:0/803596675 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee4015b730 0x7fee4015bbc0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7fee2c009d90 tx=0x7fee2c02a660 comp rx=0 tx=0).stop 2024-05-26T22:53:51.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:51.612+0000 7fee3f200640 1 -- 172.21.15.131:0/803596675 shutdown_connections 2024-05-26T22:53:51.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:51.612+0000 7fee3f200640 1 --2- 172.21.15.131:0/803596675 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fee401400c0 0x7fee401404f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:51.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:51.612+0000 7fee3f200640 1 --2- 172.21.15.131:0/803596675 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee4015b730 0x7fee4015bbc0 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:51.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:51.612+0000 7fee3f200640 1 -- 172.21.15.131:0/803596675 >> 172.21.15.131:0/803596675 conn(0x7fee400b5660 msgr2=0x7fee400b5a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:51.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:51.612+0000 7fee3f200640 1 -- 172.21.15.131:0/803596675 shutdown_connections 2024-05-26T22:53:51.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:51.612+0000 7fee3f200640 1 -- 172.21.15.131:0/803596675 wait complete. 2024-05-26T22:53:51.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:51.614+0000 7fee3f200640 1 Processor -- start 2024-05-26T22:53:51.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:51.614+0000 7fee3f200640 1 -- start start 2024-05-26T22:53:51.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:51.615+0000 7fee3f200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fee4006d250 0x7fee4013d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:51.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:51.615+0000 7fee3f200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fee401400c0 0x7fee4015d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:51.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:51.615+0000 7fee3f200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee4015b730 0x7fee400cea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:51.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:51.615+0000 7fee3f200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fee4015c100 con 0x7fee4015b730 2024-05-26T22:53:51.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:51.615+0000 7fee3f200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fee40071a70 con 0x7fee4006d250 2024-05-26T22:53:51.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:51.615+0000 7fee3f200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fee4015c280 con 0x7fee401400c0 2024-05-26T22:53:51.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:51.615+0000 7fee3e800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee4015b730 0x7fee400cea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:51.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:51.615+0000 7fee3e800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee4015b730 0x7fee400cea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60330/0 (socket says 172.21.15.131:60330) 2024-05-26T22:53:51.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:51.615+0000 7fee3e800640 1 -- 172.21.15.131:0/2564423123 learned_addr learned my addr 172.21.15.131:0/2564423123 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:53:51.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:51.615+0000 7fee3d400640 1 --2- 172.21.15.131:0/2564423123 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fee401400c0 0x7fee4015d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:51.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:51.615+0000 7fee3de00640 1 --2- 172.21.15.131:0/2564423123 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fee4006d250 0x7fee4013d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:51.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:51.616+0000 7fee3e800640 1 -- 172.21.15.131:0/2564423123 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fee401400c0 msgr2=0x7fee4015d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:51.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:51.616+0000 7fee3e800640 1 --2- 172.21.15.131:0/2564423123 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fee401400c0 0x7fee4015d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:51.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:51.616+0000 7fee3e800640 1 -- 172.21.15.131:0/2564423123 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fee4006d250 msgr2=0x7fee4013d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:51.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:51.616+0000 7fee3e800640 1 --2- 172.21.15.131:0/2564423123 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fee4006d250 0x7fee4013d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:51.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:51.616+0000 7fee3e800640 1 -- 172.21.15.131:0/2564423123 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee4015d8c0 con 0x7fee4015b730 2024-05-26T22:53:51.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:51.616+0000 7fee3e800640 1 --2- 172.21.15.131:0/2564423123 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee4015b730 0x7fee400cea20 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7fee3800d800 tx=0x7fee3800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:51.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:51.616+0000 7fee36000640 1 -- 172.21.15.131:0/2564423123 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee3800b6c0 con 0x7fee4015b730 2024-05-26T22:53:51.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:51.616+0000 7fee36000640 1 -- 172.21.15.131:0/2564423123 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee38019070 con 0x7fee4015b730 2024-05-26T22:53:51.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:51.616+0000 7fee36000640 1 -- 172.21.15.131:0/2564423123 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee3800bac0 con 0x7fee4015b730 2024-05-26T22:53:51.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:51.616+0000 7fee3f200640 1 -- 172.21.15.131:0/2564423123 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee401f4150 con 0x7fee4015b730 2024-05-26T22:53:51.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:51.616+0000 7fee3f200640 1 -- 172.21.15.131:0/2564423123 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee400b3ef0 con 0x7fee4015b730 2024-05-26T22:53:51.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:51.618+0000 7fee3f200640 1 -- 172.21.15.131:0/2564423123 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee400694e0 con 0x7fee4015b730 2024-05-26T22:53:51.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:51.619+0000 7fee36000640 1 -- 172.21.15.131:0/2564423123 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fee3800bc60 con 0x7fee4015b730 2024-05-26T22:53:51.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:51.620+0000 7fee36000640 1 --2- 172.21.15.131:0/2564423123 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fee2006be30 0x7fee2006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:51.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:51.620+0000 7fee36000640 1 -- 172.21.15.131:0/2564423123 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 5227+0+0 (secure 0 0 0) 0x7fee3808b110 con 0x7fee4015b730 2024-05-26T22:53:51.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:51.620+0000 7fee3de00640 1 --2- 172.21.15.131:0/2564423123 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fee2006be30 0x7fee2006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:51.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:51.621+0000 7fee3de00640 1 --2- 172.21.15.131:0/2564423123 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fee2006be30 0x7fee2006e2f0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fee30000f60 tx=0x7fee30009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:51.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:51.625+0000 7fee36000640 1 -- 172.21.15.131:0/2564423123 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fee401f4150 con 0x7fee4015b730 2024-05-26T22:53:51.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:51.810+0000 7fee3f200640 1 -- 172.21.15.131:0/2564423123 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fee400af9b0 con 0x7fee4015b730 2024-05-26T22:53:52.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:52.387+0000 7fee36000640 1 -- 172.21.15.131:0/2564423123 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v80) ==== 170+0+0 (secure 0 0 0) 0x7fee38059c90 con 0x7fee4015b730 2024-05-26T22:53:52.387 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_data' created 2024-05-26T22:53:52.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:52.391+0000 7fee3f200640 1 -- 172.21.15.131:0/2564423123 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fee2006be30 msgr2=0x7fee2006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:52.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:52.391+0000 7fee3f200640 1 --2- 172.21.15.131:0/2564423123 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fee2006be30 0x7fee2006e2f0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fee30000f60 tx=0x7fee30009040 comp rx=0 tx=0).stop 2024-05-26T22:53:52.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:52.391+0000 7fee3f200640 1 -- 172.21.15.131:0/2564423123 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee4015b730 msgr2=0x7fee400cea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:52.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:52.391+0000 7fee3f200640 1 --2- 172.21.15.131:0/2564423123 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee4015b730 0x7fee400cea20 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7fee3800d800 tx=0x7fee3800dcc0 comp rx=0 tx=0).stop 2024-05-26T22:53:52.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:52.392+0000 7fee3f200640 1 -- 172.21.15.131:0/2564423123 shutdown_connections 2024-05-26T22:53:52.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:52.392+0000 7fee3f200640 1 --2- 172.21.15.131:0/2564423123 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fee401400c0 0x7fee4015d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:52.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:52.392+0000 7fee3f200640 1 --2- 172.21.15.131:0/2564423123 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fee2006be30 0x7fee2006e2f0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:52.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:52.392+0000 7fee3f200640 1 --2- 172.21.15.131:0/2564423123 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee4015b730 0x7fee400cea20 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:52.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:52.392+0000 7fee3f200640 1 --2- 172.21.15.131:0/2564423123 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fee4006d250 0x7fee4013d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:52.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:52.392+0000 7fee3f200640 1 -- 172.21.15.131:0/2564423123 >> 172.21.15.131:0/2564423123 conn(0x7fee400b5660 msgr2=0x7fee40159270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:52.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:52.392+0000 7fee3f200640 1 -- 172.21.15.131:0/2564423123 shutdown_connections 2024-05-26T22:53:52.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:52.393+0000 7fee3f200640 1 -- 172.21.15.131:0/2564423123 wait complete. 2024-05-26T22:53:52.426 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-26T22:53:52.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:52.604+0000 7fa053a00640 1 Processor -- start 2024-05-26T22:53:52.604 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:52.604+0000 7fa053a00640 1 -- start start 2024-05-26T22:53:52.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:52.605+0000 7fa053a00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fa04c0d0560 0x7fa04c072e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:52.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:52.605+0000 7fa053a00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa04c072540 0x7fa04c161ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:52.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:52.605+0000 7fa053a00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa04c0af8e0 con 0x7fa04c06f180 2024-05-26T22:53:52.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:52.606+0000 7fa053a00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fa04c0cfb30 con 0x7fa04c0d0560 2024-05-26T22:53:52.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:52.606+0000 7fa053a00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa04c0cf200 con 0x7fa04c072540 2024-05-26T22:53:52.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:52.606+0000 7fa04be00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fa04c0d0560 0x7fa04c072e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:52.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:52.606+0000 7fa04be00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fa04c0d0560 0x7fa04c072e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60346/0 (socket says 172.21.15.131:60346) 2024-05-26T22:53:52.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:52.606+0000 7fa051000640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fa04c06f180 0x7fa04c0cfe50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:40460/0 (socket says 172.21.15.131:40460) 2024-05-26T22:53:52.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:52.606+0000 7fa04be00640 1 -- 172.21.15.131:0/3798953479 learned_addr learned my addr 172.21.15.131:0/3798953479 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:53:52.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:52.606+0000 7fa051a00640 1 --2- 172.21.15.131:0/3798953479 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa04c072540 0x7fa04c161ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:52.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:52.606+0000 7fa04be00640 1 -- 172.21.15.131:0/3798953479 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa04c072540 msgr2=0x7fa04c161ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:52.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:52.606+0000 7fa04be00640 1 --2- 172.21.15.131:0/3798953479 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa04c072540 0x7fa04c161ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:52.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:52.607+0000 7fa04be00640 1 -- 172.21.15.131:0/3798953479 >> v1:172.21.15.131:6789/0 conn(0x7fa04c06f180 legacy=0x7fa04c0cfe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:52.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:52.607+0000 7fa04be00640 1 -- 172.21.15.131:0/3798953479 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa04c0cf4e0 con 0x7fa04c0d0560 2024-05-26T22:53:52.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:52.607+0000 7fa051a00640 1 --2- 172.21.15.131:0/3798953479 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa04c072540 0x7fa04c161ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:53:52.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:52.607+0000 7fa04b400640 1 -- 172.21.15.131:0/3798953479 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2855695226 0 0) 0x7fa04c0af8e0 con 0x7fa04c06f180 2024-05-26T22:53:52.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:52.608+0000 7fa04be00640 1 --2- 172.21.15.131:0/3798953479 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa04c0d0560 0x7fa04c072e90 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7fa04c0cfcb0 tx=0x7fa03802a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=480780b4a908e228 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:52.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:52.608+0000 7fa04b400640 1 -- 172.21.15.131:0/3798953479 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa038037070 con 0x7fa04c0d0560 2024-05-26T22:53:52.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:52.608+0000 7fa04b400640 1 -- 172.21.15.131:0/3798953479 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa038032440 con 0x7fa04c0d0560 2024-05-26T22:53:52.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:52.608+0000 7fa04b400640 1 -- 172.21.15.131:0/3798953479 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa038032810 con 0x7fa04c0d0560 2024-05-26T22:53:52.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:52.609+0000 7fa053a00640 1 -- 172.21.15.131:0/3798953479 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa04c0d0560 msgr2=0x7fa04c072e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:52.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:52.609+0000 7fa053a00640 1 --2- 172.21.15.131:0/3798953479 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa04c0d0560 0x7fa04c072e90 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7fa04c0cfcb0 tx=0x7fa03802a250 comp rx=0 tx=0).stop 2024-05-26T22:53:52.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:52.609+0000 7fa053a00640 1 -- 172.21.15.131:0/3798953479 shutdown_connections 2024-05-26T22:53:52.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:52.609+0000 7fa053a00640 1 --2- 172.21.15.131:0/3798953479 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa04c072540 0x7fa04c161ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:52.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:52.609+0000 7fa053a00640 1 --2- 172.21.15.131:0/3798953479 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa04c0d0560 0x7fa04c072e90 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:52.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:52.609+0000 7fa053a00640 1 -- 172.21.15.131:0/3798953479 >> 172.21.15.131:0/3798953479 conn(0x7fa04c06bc90 msgr2=0x7fa04c0b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:52.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:52.609+0000 7fa053a00640 1 -- 172.21.15.131:0/3798953479 shutdown_connections 2024-05-26T22:53:52.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:52.609+0000 7fa053a00640 1 -- 172.21.15.131:0/3798953479 wait complete. 2024-05-26T22:53:52.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:52.610+0000 7fa053a00640 1 Processor -- start 2024-05-26T22:53:52.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:52.611+0000 7fa053a00640 1 -- start start 2024-05-26T22:53:52.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:52.611+0000 7fa053a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa04c06f180 0x7fa04c159ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:52.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:52.611+0000 7fa053a00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa04c072540 0x7fa04c0b45c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:52.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:52.612+0000 7fa053a00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa04c0d0560 0x7fa04c0cd9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:52.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:52.612+0000 7fa053a00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa04c06f6e0 con 0x7fa04c06f180 2024-05-26T22:53:52.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:52.612+0000 7fa053a00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa04c06f560 con 0x7fa04c072540 2024-05-26T22:53:52.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:52.612+0000 7fa053a00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa04c0715a0 con 0x7fa04c0d0560 2024-05-26T22:53:52.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:52.612+0000 7fa051000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa04c06f180 0x7fa04c159ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:52.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:52.612+0000 7fa051000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa04c06f180 0x7fa04c159ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60358/0 (socket says 172.21.15.131:60358) 2024-05-26T22:53:52.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:52.612+0000 7fa051000640 1 -- 172.21.15.131:0/2656268546 learned_addr learned my addr 172.21.15.131:0/2656268546 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:53:52.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:52.612+0000 7fa051000640 1 -- 172.21.15.131:0/2656268546 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa04c0d0560 msgr2=0x7fa04c0cd9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:52.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:52.612+0000 7fa051000640 1 --2- 172.21.15.131:0/2656268546 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa04c0d0560 0x7fa04c0cd9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:52.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:52.612+0000 7fa051000640 1 -- 172.21.15.131:0/2656268546 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa04c072540 msgr2=0x7fa04c0b45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:52.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:52.612+0000 7fa04be00640 1 --2- 172.21.15.131:0/2656268546 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa04c072540 0x7fa04c0b45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:52.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:52.612+0000 7fa051000640 1 --2- 172.21.15.131:0/2656268546 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa04c072540 0x7fa04c0b45c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:52.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:52.612+0000 7fa051000640 1 -- 172.21.15.131:0/2656268546 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa04c0b3120 con 0x7fa04c06f180 2024-05-26T22:53:52.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:52.613+0000 7fa051000640 1 --2- 172.21.15.131:0/2656268546 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa04c06f180 0x7fa04c159ae0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fa04000c4d0 tx=0x7fa04000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:52.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:52.613+0000 7fa048c00640 1 -- 172.21.15.131:0/2656268546 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa040018070 con 0x7fa04c06f180 2024-05-26T22:53:52.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:52.613+0000 7fa048c00640 1 -- 172.21.15.131:0/2656268546 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa040007500 con 0x7fa04c06f180 2024-05-26T22:53:52.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:52.613+0000 7fa048c00640 1 -- 172.21.15.131:0/2656268546 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa040004a20 con 0x7fa04c06f180 2024-05-26T22:53:52.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:52.613+0000 7fa053a00640 1 -- 172.21.15.131:0/2656268546 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa04c0b3400 con 0x7fa04c06f180 2024-05-26T22:53:52.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:52.613+0000 7fa053a00640 1 -- 172.21.15.131:0/2656268546 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa04c0b39c0 con 0x7fa04c06f180 2024-05-26T22:53:52.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:52.614+0000 7fa053a00640 1 -- 172.21.15.131:0/2656268546 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa04c074e90 con 0x7fa04c06f180 2024-05-26T22:53:52.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:52.615+0000 7fa048c00640 1 -- 172.21.15.131:0/2656268546 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa040004bc0 con 0x7fa04c06f180 2024-05-26T22:53:52.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:52.616+0000 7fa048c00640 1 --2- 172.21.15.131:0/2656268546 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa02c06be30 0x7fa02c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:52.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:52.616+0000 7fa048c00640 1 -- 172.21.15.131:0/2656268546 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5609+0+0 (secure 0 0 0) 0x7fa04008a180 con 0x7fa04c06f180 2024-05-26T22:53:52.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:52.616+0000 7fa04be00640 1 --2- 172.21.15.131:0/2656268546 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa02c06be30 0x7fa02c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:52.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:52.617+0000 7fa04be00640 1 --2- 172.21.15.131:0/2656268546 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa02c06be30 0x7fa02c06e2f0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fa038034000 tx=0x7fa03802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:52.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:52.621+0000 7fa048c00640 1 -- 172.21.15.131:0/2656268546 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fa040058af0 con 0x7fa04c06f180 2024-05-26T22:53:52.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:52.859+0000 7fa053a00640 1 -- 172.21.15.131:0/2656268546 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fa04c072100 con 0x7fa04c06f180 2024-05-26T22:53:53.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:53.429+0000 7fa048c00640 1 -- 172.21.15.131:0/2656268546 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 12 and data pool 13 v74) ==== 174+0+0 (secure 0 0 0) 0x7fa04c072100 con 0x7fa04c06f180 2024-05-26T22:53:53.430 INFO:teuthology.orchestra.run.smithi131.stderr:new fs with metadata pool 12 and data pool 13 2024-05-26T22:53:53.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:53.437+0000 7fa053a00640 1 -- 172.21.15.131:0/2656268546 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa02c06be30 msgr2=0x7fa02c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:53.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:53.437+0000 7fa053a00640 1 --2- 172.21.15.131:0/2656268546 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa02c06be30 0x7fa02c06e2f0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fa038034000 tx=0x7fa03802e000 comp rx=0 tx=0).stop 2024-05-26T22:53:53.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:53.437+0000 7fa053a00640 1 -- 172.21.15.131:0/2656268546 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa04c06f180 msgr2=0x7fa04c159ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:53.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:53.437+0000 7fa053a00640 1 --2- 172.21.15.131:0/2656268546 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa04c06f180 0x7fa04c159ae0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fa04000c4d0 tx=0x7fa04000c990 comp rx=0 tx=0).stop 2024-05-26T22:53:53.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:53.438+0000 7fa053a00640 1 -- 172.21.15.131:0/2656268546 shutdown_connections 2024-05-26T22:53:53.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:53.438+0000 7fa053a00640 1 --2- 172.21.15.131:0/2656268546 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa04c0d0560 0x7fa04c0cd9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:53.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:53.438+0000 7fa053a00640 1 --2- 172.21.15.131:0/2656268546 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa02c06be30 0x7fa02c06e2f0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:53.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:53.438+0000 7fa053a00640 1 --2- 172.21.15.131:0/2656268546 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa04c072540 0x7fa04c0b45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:53.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:53.438+0000 7fa053a00640 1 --2- 172.21.15.131:0/2656268546 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa04c06f180 0x7fa04c159ae0 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:53.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:53.438+0000 7fa053a00640 1 -- 172.21.15.131:0/2656268546 >> 172.21.15.131:0/2656268546 conn(0x7fa04c06bc90 msgr2=0x7fa04c15fbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:53.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:53.440+0000 7fa053a00640 1 -- 172.21.15.131:0/2656268546 shutdown_connections 2024-05-26T22:53:53.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:53.441+0000 7fa053a00640 1 -- 172.21.15.131:0/2656268546 wait complete. 2024-05-26T22:53:53.474 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-26T22:53:53.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:53.640+0000 7f292f200640 1 Processor -- start 2024-05-26T22:53:53.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:53.640+0000 7f292f200640 1 -- start start 2024-05-26T22:53:53.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:53.641+0000 7f292f200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f292815c3d0 0x7f292806ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:53.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:53.641+0000 7f292f200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f292806f490 0x7f29280ade40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:53.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:53.641+0000 7f292f200640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29280722a0 con 0x7f2928155840 2024-05-26T22:53:53.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:53.641+0000 7f292f200640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f2928072040 con 0x7f292806f490 2024-05-26T22:53:53.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:53.641+0000 7f292f200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f2928072580 con 0x7f292815c3d0 2024-05-26T22:53:53.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:53.642+0000 7f292d200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f292806f490 0x7f29280ade40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:53.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:53.642+0000 7f2927e00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f2928155840 0x7f2928071930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:40476/0 (socket says 172.21.15.131:40476) 2024-05-26T22:53:53.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:53.642+0000 7f2927e00640 1 -- 172.21.15.131:0/153799532 learned_addr learned my addr 172.21.15.131:0/153799532 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:53:53.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:53.642+0000 7f292d200640 1 -- 172.21.15.131:0/153799532 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f292815c3d0 msgr2=0x7f292806ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:53.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:53.642+0000 7f292d200640 1 --2- 172.21.15.131:0/153799532 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f292815c3d0 0x7f292806ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:53.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:53.642+0000 7f292d200640 1 -- 172.21.15.131:0/153799532 >> v1:172.21.15.131:6789/0 conn(0x7f2928155840 legacy=0x7f2928071930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:53.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:53.642+0000 7f292d200640 1 -- 172.21.15.131:0/153799532 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2928072860 con 0x7f292806f490 2024-05-26T22:53:53.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:53.643+0000 7f292d200640 1 --2- 172.21.15.131:0/153799532 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f292806f490 0x7f29280ade40 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f2920009d90 tx=0x7f292002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7a21cf789b2d630 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:53.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:53.643+0000 7f2926a00640 1 -- 172.21.15.131:0/153799532 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2920007740 con 0x7f292806f490 2024-05-26T22:53:53.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:53.644+0000 7f2926a00640 1 -- 172.21.15.131:0/153799532 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29200078e0 con 0x7f292806f490 2024-05-26T22:53:53.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:53.644+0000 7f2926a00640 1 -- 172.21.15.131:0/153799532 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2920007cb0 con 0x7f292806f490 2024-05-26T22:53:53.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:53.644+0000 7f292f200640 1 -- 172.21.15.131:0/153799532 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f292806f490 msgr2=0x7f29280ade40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:53.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:53.644+0000 7f292f200640 1 --2- 172.21.15.131:0/153799532 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f292806f490 0x7f29280ade40 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f2920009d90 tx=0x7f292002a660 comp rx=0 tx=0).stop 2024-05-26T22:53:53.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:53.644+0000 7f292f200640 1 -- 172.21.15.131:0/153799532 shutdown_connections 2024-05-26T22:53:53.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:53.644+0000 7f292f200640 1 --2- 172.21.15.131:0/153799532 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f292806f490 0x7f29280ade40 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:53.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:53.644+0000 7f292f200640 1 --2- 172.21.15.131:0/153799532 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f292815c3d0 0x7f292806ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:53.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:53.644+0000 7f292f200640 1 -- 172.21.15.131:0/153799532 >> 172.21.15.131:0/153799532 conn(0x7f29280b1e90 msgr2=0x7f29280b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:53.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:53.644+0000 7f292f200640 1 -- 172.21.15.131:0/153799532 shutdown_connections 2024-05-26T22:53:53.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:53.644+0000 7f292f200640 1 -- 172.21.15.131:0/153799532 wait complete. 2024-05-26T22:53:53.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:53.646+0000 7f292f200640 1 Processor -- start 2024-05-26T22:53:53.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:53.646+0000 7f292f200640 1 -- start start 2024-05-26T22:53:53.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:53.647+0000 7f292f200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f292806f490 0x7f292815a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:53.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:53.647+0000 7f292f200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2928155840 0x7f292815d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:53.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:53.647+0000 7f292f200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f292815c3d0 0x7f292815da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:53.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:53.647+0000 7f292f200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f29280ae380 con 0x7f2928155840 2024-05-26T22:53:53.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:53.647+0000 7f292f200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2928072e20 con 0x7f292806f490 2024-05-26T22:53:53.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:53.647+0000 7f292f200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f29280ae500 con 0x7f292815c3d0 2024-05-26T22:53:53.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:53.647+0000 7f2927400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2928155840 0x7f292815d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:53.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:53.647+0000 7f2927400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2928155840 0x7f292815d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60362/0 (socket says 172.21.15.131:60362) 2024-05-26T22:53:53.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:53.647+0000 7f2927400640 1 -- 172.21.15.131:0/755550921 learned_addr learned my addr 172.21.15.131:0/755550921 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:53:53.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:53.648+0000 7f2927400640 1 -- 172.21.15.131:0/755550921 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f292815c3d0 msgr2=0x7f292815da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:53.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:53.648+0000 7f2927400640 1 --2- 172.21.15.131:0/755550921 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f292815c3d0 0x7f292815da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:53.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:53.648+0000 7f2927400640 1 -- 172.21.15.131:0/755550921 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f292806f490 msgr2=0x7f292815a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:53.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:53.648+0000 7f2927400640 1 --2- 172.21.15.131:0/755550921 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f292806f490 0x7f292815a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:53.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:53.648+0000 7f2927400640 1 -- 172.21.15.131:0/755550921 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2928150990 con 0x7f2928155840 2024-05-26T22:53:53.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:53.648+0000 7f2927400640 1 --2- 172.21.15.131:0/755550921 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2928155840 0x7f292815d540 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f291400b440 tx=0x7f291400b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:53.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:53.648+0000 7f291fe00640 1 -- 172.21.15.131:0/755550921 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2914015020 con 0x7f2928155840 2024-05-26T22:53:53.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:53.648+0000 7f291fe00640 1 -- 172.21.15.131:0/755550921 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2914011890 con 0x7f2928155840 2024-05-26T22:53:53.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:53.648+0000 7f291fe00640 1 -- 172.21.15.131:0/755550921 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2914011c80 con 0x7f2928155840 2024-05-26T22:53:53.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:53.648+0000 7f292f200640 1 -- 172.21.15.131:0/755550921 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2928150c70 con 0x7f2928155840 2024-05-26T22:53:53.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:53.648+0000 7f292f200640 1 -- 172.21.15.131:0/755550921 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2928151230 con 0x7f2928155840 2024-05-26T22:53:53.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:53.650+0000 7f292f200640 1 -- 172.21.15.131:0/755550921 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f292815aa80 con 0x7f2928155840 2024-05-26T22:53:53.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:53.650+0000 7f291fe00640 1 -- 172.21.15.131:0/755550921 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2914018020 con 0x7f2928155840 2024-05-26T22:53:53.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:53.652+0000 7f291fe00640 1 --2- 172.21.15.131:0/755550921 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f290806be30 0x7f290806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:53.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:53.652+0000 7f291fe00640 1 -- 172.21.15.131:0/755550921 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7f291408afe0 con 0x7f2928155840 2024-05-26T22:53:53.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:53.652+0000 7f2927e00640 1 --2- 172.21.15.131:0/755550921 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f290806be30 0x7f290806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:53.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:53.653+0000 7f2927e00640 1 --2- 172.21.15.131:0/755550921 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f290806be30 0x7f290806e2f0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f2918000f60 tx=0x7f2918007a80 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:53.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:53.657+0000 7f291fe00640 1 -- 172.21.15.131:0/755550921 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f291401f020 con 0x7f2928155840 2024-05-26T22:53:53.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:53.850+0000 7f292f200640 1 -- 172.21.15.131:0/755550921 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f29280694e0 con 0x7f2928155840 2024-05-26T22:53:53.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:53.851+0000 7f291fe00640 1 -- 172.21.15.131:0/755550921 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+16697 (secure 0 0 0) 0x7f291400d040 con 0x7f2928155840 2024-05-26T22:53:53.851 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:53:53.851 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":81,"fsid":"46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f","created":"2024-05-26T22:47:57.288905+0000","modified":"2024-05-26T22:53:52.861674+0000","last_up_change":"2024-05-26T22:48:09.602207+0000","last_in_change":"2024-05-26T22:48:00.687341+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-26T22:48:10.463339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-05-26T22:53:50.625629+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-05-26T22:53:51.864320+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"814eaf62-a18d-494b-8e27-fd8fb88adea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6801","nonce":2350940051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6803","nonce":2350940051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6807","nonce":2350940051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6805","nonce":2350940051}]},"public_addr":"172.21.15.131:6801/2350940051","cluster_addr":"172.21.15.131:6803/2350940051","heartbeat_back_addr":"172.21.15.131:6807/2350940051","heartbeat_front_addr":"172.21.15.131:6805/2350940051","state":["exists","up"]},{"osd":1,"uuid":"42e66287-f6c6-4e26-af3c-6dfaa3536d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6825","nonce":1525528247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6827","nonce":1525528247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6831","nonce":1525528247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6829","nonce":1525528247}]},"public_addr":"172.21.15.131:6825/1525528247","cluster_addr":"172.21.15.131:6827/1525528247","heartbeat_back_addr":"172.21.15.131:6831/1525528247","heartbeat_front_addr":"172.21.15.131:6829/1525528247","state":["exists","up"]},{"osd":2,"uuid":"0da34673-04a2-4fd0-b57a-0ff2b679b043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6817","nonce":3627842210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6819","nonce":3627842210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6823","nonce":3627842210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6821","nonce":3627842210}]},"public_addr":"172.21.15.131:6817/3627842210","cluster_addr":"172.21.15.131:6819/3627842210","heartbeat_back_addr":"172.21.15.131:6823/3627842210","heartbeat_front_addr":"172.21.15.131:6821/3627842210","state":["exists","up"]},{"osd":3,"uuid":"cf117a27-d6b2-43d3-ae52-b47988bf8eed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6809","nonce":350790591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6811","nonce":350790591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6815","nonce":350790591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6813","nonce":350790591}]},"public_addr":"172.21.15.131:6809/350790591","cluster_addr":"172.21.15.131:6811/350790591","heartbeat_back_addr":"172.21.15.131:6815/350790591","heartbeat_front_addr":"172.21.15.131:6813/350790591","state":["exists","up"]},{"osd":4,"uuid":"a092ccfc-be89-4a84-88d5-0f781fa715eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6816","nonce":244560995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6819","nonce":244560995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6823","nonce":244560995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6821","nonce":244560995}]},"public_addr":"172.21.15.187:6816/244560995","cluster_addr":"172.21.15.187:6819/244560995","heartbeat_back_addr":"172.21.15.187:6823/244560995","heartbeat_front_addr":"172.21.15.187:6821/244560995","state":["exists","up"]},{"osd":5,"uuid":"7513ba69-dd00-4a35-b190-9e8cc3b2bdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6801","nonce":324619398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6803","nonce":324619398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6812","nonce":324619398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6807","nonce":324619398}]},"public_addr":"172.21.15.187:6801/324619398","cluster_addr":"172.21.15.187:6803/324619398","heartbeat_back_addr":"172.21.15.187:6812/324619398","heartbeat_front_addr":"172.21.15.187:6807/324619398","state":["exists","up"]},{"osd":6,"uuid":"50081810-c093-4d2f-a4d7-b7c695c5400d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6806","nonce":1391347786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6809","nonce":1391347786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6818","nonce":1391347786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6813","nonce":1391347786}]},"public_addr":"172.21.15.187:6806/1391347786","cluster_addr":"172.21.15.187:6809/1391347786","heartbeat_back_addr":"172.21.15.187:6818/1391347786","heartbeat_front_addr":"172.21.15.187:6813/1391347786","state":["exists","up"]},{"osd":7,"uuid":"775dae05-54f5-4ad4-bbce-92863c6ed2ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6825","nonce":1220065622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6827","nonce":1220065622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6831","nonce":1220065622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6829","nonce":1220065622}]},"public_addr":"172.21.15.187:6825/1220065622","cluster_addr":"172.21.15.187:6827/1220065622","heartbeat_back_addr":"172.21.15.187:6831/1220065622","heartbeat_front_addr":"172.21.15.187:6829/1220065622","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:05.983520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.299516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.067323+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.307748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.018742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.258775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.010912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.295355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-26T22:53:53.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:53.853+0000 7f292f200640 1 -- 172.21.15.131:0/755550921 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f290806be30 msgr2=0x7f290806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:53.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:53.853+0000 7f292f200640 1 --2- 172.21.15.131:0/755550921 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f290806be30 0x7f290806e2f0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f2918000f60 tx=0x7f2918007a80 comp rx=0 tx=0).stop 2024-05-26T22:53:53.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:53.853+0000 7f292f200640 1 -- 172.21.15.131:0/755550921 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2928155840 msgr2=0x7f292815d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:53.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:53.853+0000 7f292f200640 1 --2- 172.21.15.131:0/755550921 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2928155840 0x7f292815d540 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f291400b440 tx=0x7f291400b900 comp rx=0 tx=0).stop 2024-05-26T22:53:53.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:53.854+0000 7f292f200640 1 -- 172.21.15.131:0/755550921 shutdown_connections 2024-05-26T22:53:53.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:53.854+0000 7f292f200640 1 --2- 172.21.15.131:0/755550921 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f292815c3d0 0x7f292815da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:53.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:53.854+0000 7f292f200640 1 --2- 172.21.15.131:0/755550921 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f290806be30 0x7f290806e2f0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:53.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:53.854+0000 7f292f200640 1 --2- 172.21.15.131:0/755550921 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2928155840 0x7f292815d540 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:53.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:53.854+0000 7f292f200640 1 --2- 172.21.15.131:0/755550921 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f292806f490 0x7f292815a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:53.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:53.854+0000 7f292f200640 1 -- 172.21.15.131:0/755550921 >> 172.21.15.131:0/755550921 conn(0x7f29280b1e90 msgr2=0x7f29280cfeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:53.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:53.854+0000 7f292f200640 1 -- 172.21.15.131:0/755550921 shutdown_connections 2024-05-26T22:53:53.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:53.854+0000 7f292f200640 1 -- 172.21.15.131:0/755550921 wait complete. 2024-05-26T22:53:53.871 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-26T22:53:54.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.058+0000 7fec47e00640 1 Processor -- start 2024-05-26T22:53:54.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.058+0000 7fec47e00640 1 -- start start 2024-05-26T22:53:54.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.059+0000 7fec47e00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fec480cc7f0 0x7fec480ccc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:54.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.059+0000 7fec47e00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec48160620 con 0x7fec48156750 2024-05-26T22:53:54.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.059+0000 7fec47e00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec48160900 con 0x7fec4806e8c0 2024-05-26T22:53:54.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.059+0000 7fec47e00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fec4815d770 con 0x7fec480cc7f0 2024-05-26T22:53:54.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.060+0000 7fec46a00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fec48156750 0x7fec4815fd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:40480/0 (socket says 172.21.15.131:40480) 2024-05-26T22:53:54.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.060+0000 7fec46a00640 1 -- 172.21.15.131:0/3846748344 learned_addr learned my addr 172.21.15.131:0/3846748344 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:53:54.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.060+0000 7fec47400640 1 --2- 172.21.15.131:0/3846748344 >> v2:172.21.15.187:3300/0 conn(0x7fec480cc7f0 0x7fec480ccc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:54.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.060+0000 7fec45600640 1 -- 172.21.15.131:0/3846748344 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 258415658 0 0) 0x7fec48160620 con 0x7fec48156750 2024-05-26T22:53:54.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.060+0000 7fec45600640 1 -- 172.21.15.131:0/3846748344 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fec2c003520 con 0x7fec48156750 2024-05-26T22:53:54.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.061+0000 7fec45600640 1 -- 172.21.15.131:0/3846748344 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 145797199 0 0) 0x7fec2c003520 con 0x7fec48156750 2024-05-26T22:53:54.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.061+0000 7fec45600640 1 -- 172.21.15.131:0/3846748344 >> v2:172.21.15.187:3300/0 conn(0x7fec480cc7f0 msgr2=0x7fec480ccc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:54.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.061+0000 7fec45600640 1 --2- 172.21.15.131:0/3846748344 >> v2:172.21.15.187:3300/0 conn(0x7fec480cc7f0 0x7fec480ccc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:54.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.061+0000 7fec45600640 1 -- 172.21.15.131:0/3846748344 >> v1:172.21.15.187:6789/0 conn(0x7fec4806e8c0 legacy=0x7fec4815d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:54.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.061+0000 7fec45600640 1 -- 172.21.15.131:0/3846748344 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec48160be0 con 0x7fec48156750 2024-05-26T22:53:54.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.061+0000 7fec45600640 1 -- 172.21.15.131:0/3846748344 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7fec38001670 con 0x7fec48156750 2024-05-26T22:53:54.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.061+0000 7fec45600640 1 -- 172.21.15.131:0/3846748344 >> v1:172.21.15.131:6789/0 conn(0x7fec48156750 legacy=0x7fec4815fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:54.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.061+0000 7fec45600640 1 --2- 172.21.15.131:0/3846748344 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec2c0039b0 0x7fec2c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:54.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.061+0000 7fec45600640 1 --2- 172.21.15.131:0/3846748344 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fec2c004800 0x7fec2c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:54.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.061+0000 7fec45600640 1 --2- 172.21.15.131:0/3846748344 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fec2c0051c0 0x7fec2c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:54.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.061+0000 7fec45600640 1 -- 172.21.15.131:0/3846748344 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fec4815d770 con 0x7fec2c0039b0 2024-05-26T22:53:54.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.061+0000 7fec45600640 1 -- 172.21.15.131:0/3846748344 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fec2c00c060 con 0x7fec2c0051c0 2024-05-26T22:53:54.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.061+0000 7fec45600640 1 -- 172.21.15.131:0/3846748344 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fec2c00c260 con 0x7fec2c004800 2024-05-26T22:53:54.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.061+0000 7fec46a00640 1 --2- 172.21.15.131:0/3846748344 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec2c0039b0 0x7fec2c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:54.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.061+0000 7fec46000640 1 --2- 172.21.15.131:0/3846748344 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fec2c004800 0x7fec2c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:54.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.061+0000 7fec47400640 1 --2- 172.21.15.131:0/3846748344 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fec2c0051c0 0x7fec2c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:54.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.062+0000 7fec46a00640 1 -- 172.21.15.131:0/3846748344 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fec2c004800 msgr2=0x7fec2c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:54.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.062+0000 7fec46a00640 1 --2- 172.21.15.131:0/3846748344 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fec2c004800 0x7fec2c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:54.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.062+0000 7fec46a00640 1 -- 172.21.15.131:0/3846748344 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fec2c0051c0 msgr2=0x7fec2c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:54.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.062+0000 7fec46a00640 1 --2- 172.21.15.131:0/3846748344 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fec2c0051c0 0x7fec2c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:54.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.062+0000 7fec46a00640 1 -- 172.21.15.131:0/3846748344 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec2c003520 con 0x7fec2c0039b0 2024-05-26T22:53:54.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.062+0000 7fec47400640 1 --2- 172.21.15.131:0/3846748344 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fec2c0051c0 0x7fec2c00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:53:54.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.062+0000 7fec47400640 1 -- 172.21.15.131:0/3846748344 reap_dead start 2024-05-26T22:53:54.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.063+0000 7fec46a00640 1 --2- 172.21.15.131:0/3846748344 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec2c0039b0 0x7fec2c003dd0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7fec3800c240 tx=0x7fec2c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd19b8da730aa188 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:54.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.063+0000 7fec45600640 1 -- 172.21.15.131:0/3846748344 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec38007cf0 con 0x7fec2c0039b0 2024-05-26T22:53:54.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.063+0000 7fec45600640 1 -- 172.21.15.131:0/3846748344 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec38005af0 con 0x7fec2c0039b0 2024-05-26T22:53:54.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.063+0000 7fec45600640 1 -- 172.21.15.131:0/3846748344 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec38007070 con 0x7fec2c0039b0 2024-05-26T22:53:54.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.064+0000 7fec47e00640 1 -- 172.21.15.131:0/3846748344 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec2c0039b0 msgr2=0x7fec2c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:54.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.064+0000 7fec47e00640 1 --2- 172.21.15.131:0/3846748344 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec2c0039b0 0x7fec2c003dd0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7fec3800c240 tx=0x7fec2c005830 comp rx=0 tx=0).stop 2024-05-26T22:53:54.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.064+0000 7fec47e00640 1 -- 172.21.15.131:0/3846748344 shutdown_connections 2024-05-26T22:53:54.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.064+0000 7fec47e00640 1 --2- 172.21.15.131:0/3846748344 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec2c0039b0 0x7fec2c003dd0 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:54.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.064+0000 7fec47e00640 1 -- 172.21.15.131:0/3846748344 >> 172.21.15.131:0/3846748344 conn(0x7fec480cff40 msgr2=0x7fec480d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:54.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.064+0000 7fec47e00640 1 -- 172.21.15.131:0/3846748344 shutdown_connections 2024-05-26T22:53:54.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.064+0000 7fec47e00640 1 -- 172.21.15.131:0/3846748344 wait complete. 2024-05-26T22:53:54.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.066+0000 7fec47e00640 1 Processor -- start 2024-05-26T22:53:54.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.066+0000 7fec47e00640 1 -- start start 2024-05-26T22:53:54.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.067+0000 7fec47e00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fec2c0039b0 0x7fec4814c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:54.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.067+0000 7fec47e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec48072940 0x7fec48072dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:54.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.067+0000 7fec47e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fec481e76d0 0x7fec481e7b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:54.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.067+0000 7fec47e00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fec4815cbd0 con 0x7fec48072940 2024-05-26T22:53:54.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.067+0000 7fec47e00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fec480cd1a0 con 0x7fec2c0039b0 2024-05-26T22:53:54.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.067+0000 7fec47e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fec48156b30 con 0x7fec481e76d0 2024-05-26T22:53:54.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.067+0000 7fec46000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec48072940 0x7fec48072dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:54.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.067+0000 7fec46000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec48072940 0x7fec48072dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60380/0 (socket says 172.21.15.131:60380) 2024-05-26T22:53:54.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.067+0000 7fec46000640 1 -- 172.21.15.131:0/1726719982 learned_addr learned my addr 172.21.15.131:0/1726719982 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:53:54.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.067+0000 7fec47400640 1 --2- 172.21.15.131:0/1726719982 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fec481e76d0 0x7fec481e7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:54.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.067+0000 7fec46a00640 1 --2- 172.21.15.131:0/1726719982 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fec2c0039b0 0x7fec4814c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:54.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.067+0000 7fec46000640 1 -- 172.21.15.131:0/1726719982 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fec481e76d0 msgr2=0x7fec481e7b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:54.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.067+0000 7fec46000640 1 --2- 172.21.15.131:0/1726719982 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fec481e76d0 0x7fec481e7b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:54.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.067+0000 7fec46000640 1 -- 172.21.15.131:0/1726719982 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fec2c0039b0 msgr2=0x7fec4814c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:54.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.067+0000 7fec46000640 1 --2- 172.21.15.131:0/1726719982 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fec2c0039b0 0x7fec4814c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:54.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.067+0000 7fec46000640 1 -- 172.21.15.131:0/1726719982 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec48152af0 con 0x7fec48072940 2024-05-26T22:53:54.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.067+0000 7fec46a00640 1 --2- 172.21.15.131:0/1726719982 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fec2c0039b0 0x7fec4814c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:53:54.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.067+0000 7fec47400640 1 --2- 172.21.15.131:0/1726719982 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fec481e76d0 0x7fec481e7b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:53:54.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.068+0000 7fec46000640 1 --2- 172.21.15.131:0/1726719982 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec48072940 0x7fec48072dc0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7fec3400e4b0 tx=0x7fec3400e970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:54.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.068+0000 7fec3ea00640 1 -- 172.21.15.131:0/1726719982 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec340052e0 con 0x7fec48072940 2024-05-26T22:53:54.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.068+0000 7fec3ea00640 1 -- 172.21.15.131:0/1726719982 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec34018070 con 0x7fec48072940 2024-05-26T22:53:54.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.068+0000 7fec47e00640 1 -- 172.21.15.131:0/1726719982 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec48152dd0 con 0x7fec48072940 2024-05-26T22:53:54.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.068+0000 7fec3ea00640 1 -- 172.21.15.131:0/1726719982 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec34003830 con 0x7fec48072940 2024-05-26T22:53:54.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.068+0000 7fec47e00640 1 -- 172.21.15.131:0/1726719982 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec48153390 con 0x7fec48072940 2024-05-26T22:53:54.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.069+0000 7fec47e00640 1 -- 172.21.15.131:0/1726719982 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec480694e0 con 0x7fec48072940 2024-05-26T22:53:54.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.070+0000 7fec3ea00640 1 -- 172.21.15.131:0/1726719982 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fec340039d0 con 0x7fec48072940 2024-05-26T22:53:54.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.071+0000 7fec3ea00640 1 --2- 172.21.15.131:0/1726719982 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fec2806be60 0x7fec2806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:54.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.071+0000 7fec3ea00640 1 -- 172.21.15.131:0/1726719982 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7fec3408ab60 con 0x7fec48072940 2024-05-26T22:53:54.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.071+0000 7fec46a00640 1 --2- 172.21.15.131:0/1726719982 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fec2806be60 0x7fec2806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:54.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.072+0000 7fec46a00640 1 --2- 172.21.15.131:0/1726719982 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fec2806be60 0x7fec2806e320 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fec38032000 tx=0x7fec38009030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:54.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.076+0000 7fec3ea00640 1 -- 172.21.15.131:0/1726719982 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fec34059470 con 0x7fec48072940 2024-05-26T22:53:54.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.259+0000 7fec47e00640 1 -- 172.21.15.131:0/1726719982 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fec4814c6d0 con 0x7fec48072940 2024-05-26T22:53:54.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.261+0000 7fec3ea00640 1 -- 172.21.15.131:0/1726719982 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+16697 (secure 0 0 0) 0x7fec34058d70 con 0x7fec48072940 2024-05-26T22:53:54.261 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:53:54.261 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":81,"fsid":"46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f","created":"2024-05-26T22:47:57.288905+0000","modified":"2024-05-26T22:53:52.861674+0000","last_up_change":"2024-05-26T22:48:09.602207+0000","last_in_change":"2024-05-26T22:48:00.687341+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-26T22:48:10.463339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-05-26T22:53:50.625629+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-05-26T22:53:51.864320+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"814eaf62-a18d-494b-8e27-fd8fb88adea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6801","nonce":2350940051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6803","nonce":2350940051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6807","nonce":2350940051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6805","nonce":2350940051}]},"public_addr":"172.21.15.131:6801/2350940051","cluster_addr":"172.21.15.131:6803/2350940051","heartbeat_back_addr":"172.21.15.131:6807/2350940051","heartbeat_front_addr":"172.21.15.131:6805/2350940051","state":["exists","up"]},{"osd":1,"uuid":"42e66287-f6c6-4e26-af3c-6dfaa3536d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6825","nonce":1525528247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6827","nonce":1525528247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6831","nonce":1525528247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6829","nonce":1525528247}]},"public_addr":"172.21.15.131:6825/1525528247","cluster_addr":"172.21.15.131:6827/1525528247","heartbeat_back_addr":"172.21.15.131:6831/1525528247","heartbeat_front_addr":"172.21.15.131:6829/1525528247","state":["exists","up"]},{"osd":2,"uuid":"0da34673-04a2-4fd0-b57a-0ff2b679b043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6817","nonce":3627842210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6819","nonce":3627842210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6823","nonce":3627842210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6821","nonce":3627842210}]},"public_addr":"172.21.15.131:6817/3627842210","cluster_addr":"172.21.15.131:6819/3627842210","heartbeat_back_addr":"172.21.15.131:6823/3627842210","heartbeat_front_addr":"172.21.15.131:6821/3627842210","state":["exists","up"]},{"osd":3,"uuid":"cf117a27-d6b2-43d3-ae52-b47988bf8eed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6809","nonce":350790591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6811","nonce":350790591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6815","nonce":350790591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6813","nonce":350790591}]},"public_addr":"172.21.15.131:6809/350790591","cluster_addr":"172.21.15.131:6811/350790591","heartbeat_back_addr":"172.21.15.131:6815/350790591","heartbeat_front_addr":"172.21.15.131:6813/350790591","state":["exists","up"]},{"osd":4,"uuid":"a092ccfc-be89-4a84-88d5-0f781fa715eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6816","nonce":244560995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6819","nonce":244560995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6823","nonce":244560995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6821","nonce":244560995}]},"public_addr":"172.21.15.187:6816/244560995","cluster_addr":"172.21.15.187:6819/244560995","heartbeat_back_addr":"172.21.15.187:6823/244560995","heartbeat_front_addr":"172.21.15.187:6821/244560995","state":["exists","up"]},{"osd":5,"uuid":"7513ba69-dd00-4a35-b190-9e8cc3b2bdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6801","nonce":324619398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6803","nonce":324619398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6812","nonce":324619398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6807","nonce":324619398}]},"public_addr":"172.21.15.187:6801/324619398","cluster_addr":"172.21.15.187:6803/324619398","heartbeat_back_addr":"172.21.15.187:6812/324619398","heartbeat_front_addr":"172.21.15.187:6807/324619398","state":["exists","up"]},{"osd":6,"uuid":"50081810-c093-4d2f-a4d7-b7c695c5400d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6806","nonce":1391347786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6809","nonce":1391347786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6818","nonce":1391347786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6813","nonce":1391347786}]},"public_addr":"172.21.15.187:6806/1391347786","cluster_addr":"172.21.15.187:6809/1391347786","heartbeat_back_addr":"172.21.15.187:6818/1391347786","heartbeat_front_addr":"172.21.15.187:6813/1391347786","state":["exists","up"]},{"osd":7,"uuid":"775dae05-54f5-4ad4-bbce-92863c6ed2ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6825","nonce":1220065622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6827","nonce":1220065622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6831","nonce":1220065622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6829","nonce":1220065622}]},"public_addr":"172.21.15.187:6825/1220065622","cluster_addr":"172.21.15.187:6827/1220065622","heartbeat_back_addr":"172.21.15.187:6831/1220065622","heartbeat_front_addr":"172.21.15.187:6829/1220065622","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:05.983520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.299516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.067323+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.307748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.018742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.258775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.010912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.295355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-26T22:53:54.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.264+0000 7fec47e00640 1 -- 172.21.15.131:0/1726719982 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fec2806be60 msgr2=0x7fec2806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:54.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.264+0000 7fec47e00640 1 --2- 172.21.15.131:0/1726719982 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fec2806be60 0x7fec2806e320 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fec38032000 tx=0x7fec38009030 comp rx=0 tx=0).stop 2024-05-26T22:53:54.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.264+0000 7fec47e00640 1 -- 172.21.15.131:0/1726719982 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec48072940 msgr2=0x7fec48072dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:54.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.264+0000 7fec47e00640 1 --2- 172.21.15.131:0/1726719982 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec48072940 0x7fec48072dc0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7fec3400e4b0 tx=0x7fec3400e970 comp rx=0 tx=0).stop 2024-05-26T22:53:54.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.264+0000 7fec47e00640 1 -- 172.21.15.131:0/1726719982 shutdown_connections 2024-05-26T22:53:54.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.264+0000 7fec47e00640 1 --2- 172.21.15.131:0/1726719982 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fec481e76d0 0x7fec481e7b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:54.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.264+0000 7fec47e00640 1 --2- 172.21.15.131:0/1726719982 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fec2806be60 0x7fec2806e320 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:54.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.264+0000 7fec47e00640 1 --2- 172.21.15.131:0/1726719982 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec48072940 0x7fec48072dc0 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:54.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.264+0000 7fec47e00640 1 --2- 172.21.15.131:0/1726719982 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fec2c0039b0 0x7fec4814c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:54.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.264+0000 7fec47e00640 1 -- 172.21.15.131:0/1726719982 >> 172.21.15.131:0/1726719982 conn(0x7fec480cff40 msgr2=0x7fec480b4b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:54.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.264+0000 7fec47e00640 1 -- 172.21.15.131:0/1726719982 shutdown_connections 2024-05-26T22:53:54.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.264+0000 7fec47e00640 1 -- 172.21.15.131:0/1726719982 wait complete. 2024-05-26T22:53:54.282 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-26T22:53:54.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.487+0000 7fe8ec000640 1 Processor -- start 2024-05-26T22:53:54.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.487+0000 7fe8ec000640 1 -- start start 2024-05-26T22:53:54.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.488+0000 7fe8ec000640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fe8e406dd30 0x7fe8e406e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:54.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.488+0000 7fe8ec000640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8e415c1c0 con 0x7fe8e40725a0 2024-05-26T22:53:54.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.489+0000 7fe8ec000640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8e415c4a0 con 0x7fe8e4155770 2024-05-26T22:53:54.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.489+0000 7fe8ec000640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fe8e4071da0 con 0x7fe8e406dd30 2024-05-26T22:53:54.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.489+0000 7fe8e8c00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fe8e406dd30 0x7fe8e406e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:54.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.489+0000 7fe8e8c00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fe8e406dd30 0x7fe8e406e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56654/0 (socket says 172.21.15.131:56654) 2024-05-26T22:53:54.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.489+0000 7fe8e8c00640 1 -- 172.21.15.131:0/3689355816 learned_addr learned my addr 172.21.15.131:0/3689355816 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:53:54.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.489+0000 7fe8ea000640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fe8e40725a0 0x7fe8e40729a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:50618/0 (socket says 172.21.15.131:50618) 2024-05-26T22:53:54.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.489+0000 7fe8e8c00640 1 -- 172.21.15.131:0/3689355816 >> v1:172.21.15.187:6789/0 conn(0x7fe8e4155770 legacy=0x7fe8e415b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:54.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.489+0000 7fe8e8c00640 1 -- 172.21.15.131:0/3689355816 >> v1:172.21.15.131:6789/0 conn(0x7fe8e40725a0 legacy=0x7fe8e40729a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:54.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.489+0000 7fe8e8c00640 1 -- 172.21.15.131:0/3689355816 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8e4072080 con 0x7fe8e406dd30 2024-05-26T22:53:54.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.489+0000 7fe8e3e00640 1 -- 172.21.15.131:0/3689355816 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2028016947 0 0) 0x7fe8e415c1c0 con 0x7fe8e40725a0 2024-05-26T22:53:54.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.490+0000 7fe8e8c00640 1 --2- 172.21.15.131:0/3689355816 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8e406dd30 0x7fe8e406e1c0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7fe8d000bb10 tx=0x7fe8d002c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14765edafcc1d2f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:54.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.491+0000 7fe8e3e00640 1 -- 172.21.15.131:0/3689355816 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d0035070 con 0x7fe8e406dd30 2024-05-26T22:53:54.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.491+0000 7fe8e3e00640 1 -- 172.21.15.131:0/3689355816 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8d0031610 con 0x7fe8e406dd30 2024-05-26T22:53:54.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.491+0000 7fe8e3e00640 1 -- 172.21.15.131:0/3689355816 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d00319e0 con 0x7fe8e406dd30 2024-05-26T22:53:54.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.491+0000 7fe8ec000640 1 -- 172.21.15.131:0/3689355816 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8e406dd30 msgr2=0x7fe8e406e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:54.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.491+0000 7fe8ec000640 1 --2- 172.21.15.131:0/3689355816 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8e406dd30 0x7fe8e406e1c0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7fe8d000bb10 tx=0x7fe8d002c9f0 comp rx=0 tx=0).stop 2024-05-26T22:53:54.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.491+0000 7fe8ec000640 1 -- 172.21.15.131:0/3689355816 shutdown_connections 2024-05-26T22:53:54.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.491+0000 7fe8ec000640 1 --2- 172.21.15.131:0/3689355816 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8e406dd30 0x7fe8e406e1c0 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:54.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.491+0000 7fe8ec000640 1 -- 172.21.15.131:0/3689355816 >> 172.21.15.131:0/3689355816 conn(0x7fe8e406bc90 msgr2=0x7fe8e40b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:54.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.491+0000 7fe8ec000640 1 -- 172.21.15.131:0/3689355816 shutdown_connections 2024-05-26T22:53:54.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.491+0000 7fe8ec000640 1 -- 172.21.15.131:0/3689355816 wait complete. 2024-05-26T22:53:54.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.493+0000 7fe8ec000640 1 Processor -- start 2024-05-26T22:53:54.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.493+0000 7fe8ec000640 1 -- start start 2024-05-26T22:53:54.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.494+0000 7fe8ec000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe8e406dd30 0x7fe8e41f3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:54.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.494+0000 7fe8ec000640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe8e40725a0 0x7fe8e41e7720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:54.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.494+0000 7fe8ec000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8e4155770 0x7fe8e40b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:54.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.494+0000 7fe8ec000640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e415be10 con 0x7fe8e4155770 2024-05-26T22:53:54.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.494+0000 7fe8ec000640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e415c8c0 con 0x7fe8e40725a0 2024-05-26T22:53:54.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.494+0000 7fe8ec000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe8e409c0f0 con 0x7fe8e406dd30 2024-05-26T22:53:54.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.494+0000 7fe8ea000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8e4155770 0x7fe8e40b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:54.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.494+0000 7fe8ea000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8e4155770 0x7fe8e40b4e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56670/0 (socket says 172.21.15.131:56670) 2024-05-26T22:53:54.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.494+0000 7fe8ea000640 1 -- 172.21.15.131:0/2090652334 learned_addr learned my addr 172.21.15.131:0/2090652334 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:53:54.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.495+0000 7fe8e8c00640 1 --2- 172.21.15.131:0/2090652334 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe8e40725a0 0x7fe8e41e7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:54.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.495+0000 7fe8ea000640 1 -- 172.21.15.131:0/2090652334 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe8e406dd30 msgr2=0x7fe8e41f3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:54.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.495+0000 7fe8ea000640 1 --2- 172.21.15.131:0/2090652334 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe8e406dd30 0x7fe8e41f3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:54.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.495+0000 7fe8ea000640 1 -- 172.21.15.131:0/2090652334 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe8e40725a0 msgr2=0x7fe8e41e7720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:54.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.495+0000 7fe8ea000640 1 --2- 172.21.15.131:0/2090652334 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe8e40725a0 0x7fe8e41e7720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:54.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.495+0000 7fe8ea000640 1 -- 172.21.15.131:0/2090652334 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8e41e7c60 con 0x7fe8e4155770 2024-05-26T22:53:54.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.495+0000 7fe8e8c00640 1 --2- 172.21.15.131:0/2090652334 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe8e40725a0 0x7fe8e41e7720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:53:54.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.495+0000 7fe8ea000640 1 --2- 172.21.15.131:0/2090652334 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8e4155770 0x7fe8e40b4e70 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc00c7a0 tx=0x7fe8dc00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:54.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.495+0000 7fe8e1600640 1 -- 172.21.15.131:0/2090652334 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8dc018070 con 0x7fe8e4155770 2024-05-26T22:53:54.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.495+0000 7fe8e1600640 1 -- 172.21.15.131:0/2090652334 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8dc00a640 con 0x7fe8e4155770 2024-05-26T22:53:54.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.495+0000 7fe8e1600640 1 -- 172.21.15.131:0/2090652334 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8dc00aa70 con 0x7fe8e4155770 2024-05-26T22:53:54.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.495+0000 7fe8ec000640 1 -- 172.21.15.131:0/2090652334 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8e409b800 con 0x7fe8e4155770 2024-05-26T22:53:54.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.496+0000 7fe8ec000640 1 -- 172.21.15.131:0/2090652334 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8e409bd80 con 0x7fe8e4155770 2024-05-26T22:53:54.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.497+0000 7fe8ec000640 1 -- 172.21.15.131:0/2090652334 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8e4074e90 con 0x7fe8e4155770 2024-05-26T22:53:54.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.498+0000 7fe8e1600640 1 -- 172.21.15.131:0/2090652334 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe8dc01b070 con 0x7fe8e4155770 2024-05-26T22:53:54.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.499+0000 7fe8e1600640 1 --2- 172.21.15.131:0/2090652334 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fe8c406be30 0x7fe8c406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:54.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.499+0000 7fe8e1600640 1 -- 172.21.15.131:0/2090652334 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5709+0+0 (secure 0 0 0) 0x7fe8dc08a3a0 con 0x7fe8e4155770 2024-05-26T22:53:54.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.499+0000 7fe8e9600640 1 --2- 172.21.15.131:0/2090652334 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fe8c406be30 0x7fe8c406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:54.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.500+0000 7fe8e9600640 1 --2- 172.21.15.131:0/2090652334 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fe8c406be30 0x7fe8c406e2f0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fe8d8006fd0 tx=0x7fe8d8006bc0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:54.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.503+0000 7fe8e1600640 1 -- 172.21.15.131:0/2090652334 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fe8dc058de0 con 0x7fe8e4155770 2024-05-26T22:53:54.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:54.749+0000 7fe8ec000640 1 -- 172.21.15.131:0/2090652334 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fe8e40695c0 con 0x7fe8e4155770 2024-05-26T22:53:55.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:55.449+0000 7fe8e1600640 1 -- 172.21.15.131:0/2090652334 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v77) ==== 118+0+0 (secure 0 0 0) 0x7fe8dc0586e0 con 0x7fe8e4155770 2024-05-26T22:53:55.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:55.453+0000 7fe8ec000640 1 -- 172.21.15.131:0/2090652334 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fe8c406be30 msgr2=0x7fe8c406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:55.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:55.453+0000 7fe8ec000640 1 --2- 172.21.15.131:0/2090652334 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fe8c406be30 0x7fe8c406e2f0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fe8d8006fd0 tx=0x7fe8d8006bc0 comp rx=0 tx=0).stop 2024-05-26T22:53:55.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:55.453+0000 7fe8ec000640 1 -- 172.21.15.131:0/2090652334 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8e4155770 msgr2=0x7fe8e40b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:55.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:55.453+0000 7fe8ec000640 1 --2- 172.21.15.131:0/2090652334 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8e4155770 0x7fe8e40b4e70 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc00c7a0 tx=0x7fe8dc00cc60 comp rx=0 tx=0).stop 2024-05-26T22:53:55.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:55.453+0000 7fe8ec000640 1 -- 172.21.15.131:0/2090652334 shutdown_connections 2024-05-26T22:53:55.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:55.453+0000 7fe8ec000640 1 --2- 172.21.15.131:0/2090652334 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fe8c406be30 0x7fe8c406e2f0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:55.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:55.453+0000 7fe8ec000640 1 --2- 172.21.15.131:0/2090652334 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8e4155770 0x7fe8e40b4e70 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:55.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:55.453+0000 7fe8ec000640 1 --2- 172.21.15.131:0/2090652334 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe8e40725a0 0x7fe8e41e7720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:55.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:55.453+0000 7fe8ec000640 1 --2- 172.21.15.131:0/2090652334 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe8e406dd30 0x7fe8e41f3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:55.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:55.453+0000 7fe8ec000640 1 -- 172.21.15.131:0/2090652334 >> 172.21.15.131:0/2090652334 conn(0x7fe8e406bc90 msgr2=0x7fe8e406d320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:55.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:55.453+0000 7fe8ec000640 1 -- 172.21.15.131:0/2090652334 shutdown_connections 2024-05-26T22:53:55.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:55.453+0000 7fe8ec000640 1 -- 172.21.15.131:0/2090652334 wait complete. 2024-05-26T22:53:55.475 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-26T22:53:55.476 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-26T22:53:55.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:55.680+0000 7f6db3200640 1 Processor -- start 2024-05-26T22:53:55.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:55.680+0000 7f6db3200640 1 -- start start 2024-05-26T22:53:55.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:55.681+0000 7f6db3200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f6db415b730 0x7f6db415bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:55.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:55.681+0000 7f6db3200640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6db413e820 con 0x7f6db406d250 2024-05-26T22:53:55.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:55.681+0000 7f6db3200640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6db415c4a0 con 0x7f6db41400c0 2024-05-26T22:53:55.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:55.681+0000 7f6db3200640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f6db4071260 con 0x7f6db415b730 2024-05-26T22:53:55.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:55.682+0000 7f6db1e00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f6db406d250 0x7f6db406d630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:50632/0 (socket says 172.21.15.131:50632) 2024-05-26T22:53:55.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:55.682+0000 7f6db1e00640 1 -- 172.21.15.131:0/2216581714 learned_addr learned my addr 172.21.15.131:0/2216581714 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:53:55.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:55.682+0000 7f6db1400640 1 --2- 172.21.15.131:0/2216581714 >> v2:172.21.15.131:3300/0 conn(0x7f6db415b730 0x7f6db415bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:55.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:55.682+0000 7f6db0a00640 1 -- 172.21.15.131:0/2216581714 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 92763266 0 0) 0x7f6db413e820 con 0x7f6db406d250 2024-05-26T22:53:55.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:55.682+0000 7f6db0a00640 1 -- 172.21.15.131:0/2216581714 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6d98003520 con 0x7f6db406d250 2024-05-26T22:53:55.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:55.682+0000 7f6db1400640 1 -- 172.21.15.131:0/2216581714 >> v1:172.21.15.187:6789/0 conn(0x7f6db41400c0 legacy=0x7f6db41404f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:55.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:55.682+0000 7f6db1400640 1 -- 172.21.15.131:0/2216581714 >> v1:172.21.15.131:6789/0 conn(0x7f6db406d250 legacy=0x7f6db406d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:55.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:55.682+0000 7f6db1400640 1 -- 172.21.15.131:0/2216581714 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6db40714b0 con 0x7f6db415b730 2024-05-26T22:53:55.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:55.683+0000 7f6db1400640 1 --2- 172.21.15.131:0/2216581714 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6db415b730 0x7f6db415bbc0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f6db40713e0 tx=0x7f6da002a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fdfd75d01e9de59 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:55.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:55.684+0000 7f6db0a00640 1 -- 172.21.15.131:0/2216581714 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da0037070 con 0x7f6db415b730 2024-05-26T22:53:55.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:55.684+0000 7f6db0a00640 1 -- 172.21.15.131:0/2216581714 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6da0032440 con 0x7f6db415b730 2024-05-26T22:53:55.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:55.684+0000 7f6db0a00640 1 -- 172.21.15.131:0/2216581714 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da0032810 con 0x7f6db415b730 2024-05-26T22:53:55.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:55.684+0000 7f6db3200640 1 -- 172.21.15.131:0/2216581714 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6db415b730 msgr2=0x7f6db415bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:55.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:55.684+0000 7f6db3200640 1 --2- 172.21.15.131:0/2216581714 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6db415b730 0x7f6db415bbc0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f6db40713e0 tx=0x7f6da002a200 comp rx=0 tx=0).stop 2024-05-26T22:53:55.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:55.684+0000 7f6db3200640 1 -- 172.21.15.131:0/2216581714 shutdown_connections 2024-05-26T22:53:55.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:55.684+0000 7f6db3200640 1 --2- 172.21.15.131:0/2216581714 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6db415b730 0x7f6db415bbc0 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:55.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:55.684+0000 7f6db3200640 1 -- 172.21.15.131:0/2216581714 >> 172.21.15.131:0/2216581714 conn(0x7f6db40b5660 msgr2=0x7f6db40b5a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:55.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:55.684+0000 7f6db3200640 1 -- 172.21.15.131:0/2216581714 shutdown_connections 2024-05-26T22:53:55.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:55.684+0000 7f6db3200640 1 -- 172.21.15.131:0/2216581714 wait complete. 2024-05-26T22:53:55.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:55.686+0000 7f6db3200640 1 Processor -- start 2024-05-26T22:53:55.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:55.686+0000 7f6db3200640 1 -- start start 2024-05-26T22:53:55.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:55.687+0000 7f6db3200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6db406d250 0x7f6db4153d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:55.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:55.687+0000 7f6db3200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6db41400c0 0x7f6db40cf950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:55.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:55.687+0000 7f6db3200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6db415b730 0x7f6db413bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:55.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:55.687+0000 7f6db3200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f6db415c100 con 0x7f6db415b730 2024-05-26T22:53:55.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:55.687+0000 7f6db3200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6db4071a70 con 0x7f6db406d250 2024-05-26T22:53:55.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:55.687+0000 7f6db3200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f6db415c280 con 0x7f6db41400c0 2024-05-26T22:53:55.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:55.688+0000 7f6db2800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6db415b730 0x7f6db413bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:55.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:55.688+0000 7f6db2800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6db415b730 0x7f6db413bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56694/0 (socket says 172.21.15.131:56694) 2024-05-26T22:53:55.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:55.688+0000 7f6db2800640 1 -- 172.21.15.131:0/1861718965 learned_addr learned my addr 172.21.15.131:0/1861718965 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:53:55.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:55.688+0000 7f6db1e00640 1 --2- 172.21.15.131:0/1861718965 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6db406d250 0x7f6db4153d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:55.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:55.688+0000 7f6db2800640 1 -- 172.21.15.131:0/1861718965 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6db41400c0 msgr2=0x7f6db40cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:55.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:55.688+0000 7f6db2800640 1 --2- 172.21.15.131:0/1861718965 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6db41400c0 0x7f6db40cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:55.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:55.688+0000 7f6db2800640 1 -- 172.21.15.131:0/1861718965 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6db406d250 msgr2=0x7f6db4153d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:55.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:55.688+0000 7f6db2800640 1 --2- 172.21.15.131:0/1861718965 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6db406d250 0x7f6db4153d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:55.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:55.688+0000 7f6db2800640 1 -- 172.21.15.131:0/1861718965 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6db413cb40 con 0x7f6db415b730 2024-05-26T22:53:55.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:55.688+0000 7f6db2800640 1 --2- 172.21.15.131:0/1861718965 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6db415b730 0x7f6db413bba0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f6dac00c780 tx=0x7f6dac00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:55.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:55.689+0000 7f6daa000640 1 -- 172.21.15.131:0/1861718965 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dac004b60 con 0x7f6db415b730 2024-05-26T22:53:55.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:55.689+0000 7f6daa000640 1 -- 172.21.15.131:0/1861718965 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6dac004d00 con 0x7f6db415b730 2024-05-26T22:53:55.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:55.689+0000 7f6daa000640 1 -- 172.21.15.131:0/1861718965 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dac00aa90 con 0x7f6db415b730 2024-05-26T22:53:55.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:55.689+0000 7f6db3200640 1 -- 172.21.15.131:0/1861718965 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6db413ce20 con 0x7f6db415b730 2024-05-26T22:53:55.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:55.689+0000 7f6db3200640 1 -- 172.21.15.131:0/1861718965 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6db413d3e0 con 0x7f6db415b730 2024-05-26T22:53:55.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:55.690+0000 7f6db3200640 1 -- 172.21.15.131:0/1861718965 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6db40694e0 con 0x7f6db415b730 2024-05-26T22:53:55.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:55.691+0000 7f6daa000640 1 -- 172.21.15.131:0/1861718965 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6dac00ac30 con 0x7f6db415b730 2024-05-26T22:53:55.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:55.692+0000 7f6daa000640 1 --2- 172.21.15.131:0/1861718965 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f6d9406be60 0x7f6d9406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:55.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:55.692+0000 7f6daa000640 1 -- 172.21.15.131:0/1861718965 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5709+0+0 (secure 0 0 0) 0x7f6dac08b130 con 0x7f6db415b730 2024-05-26T22:53:55.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:55.692+0000 7f6db1e00640 1 --2- 172.21.15.131:0/1861718965 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f6d9406be60 0x7f6d9406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:55.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:55.693+0000 7f6db1e00640 1 --2- 172.21.15.131:0/1861718965 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f6d9406be60 0x7f6d9406e320 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f6da4005880 tx=0x7f6da4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:55.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:55.696+0000 7f6daa000640 1 -- 172.21.15.131:0/1861718965 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f6dac020070 con 0x7f6db415b730 2024-05-26T22:53:55.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:55.924+0000 7f6db3200640 1 -- 172.21.15.131:0/1861718965 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f6db40af9b0 con 0x7f6db415b730 2024-05-26T22:53:56.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:56.461+0000 7f6daa000640 1 -- 172.21.15.131:0/1861718965 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v78) ==== 122+0+0 (secure 0 0 0) 0x7f6dac059ad0 con 0x7f6db415b730 2024-05-26T22:53:56.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:56.465+0000 7f6db3200640 1 -- 172.21.15.131:0/1861718965 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f6d9406be60 msgr2=0x7f6d9406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:56.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:56.465+0000 7f6db3200640 1 --2- 172.21.15.131:0/1861718965 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f6d9406be60 0x7f6d9406e320 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f6da4005880 tx=0x7f6da4009040 comp rx=0 tx=0).stop 2024-05-26T22:53:56.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:56.465+0000 7f6db3200640 1 -- 172.21.15.131:0/1861718965 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6db415b730 msgr2=0x7f6db413bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:56.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:56.465+0000 7f6db3200640 1 --2- 172.21.15.131:0/1861718965 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6db415b730 0x7f6db413bba0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f6dac00c780 tx=0x7f6dac00cc40 comp rx=0 tx=0).stop 2024-05-26T22:53:56.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:56.465+0000 7f6db3200640 1 -- 172.21.15.131:0/1861718965 shutdown_connections 2024-05-26T22:53:56.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:56.465+0000 7f6db3200640 1 --2- 172.21.15.131:0/1861718965 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6db41400c0 0x7f6db40cf950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:56.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:56.465+0000 7f6db3200640 1 --2- 172.21.15.131:0/1861718965 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f6d9406be60 0x7f6d9406e320 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:56.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:56.465+0000 7f6db3200640 1 --2- 172.21.15.131:0/1861718965 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6db415b730 0x7f6db413bba0 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:56.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:56.465+0000 7f6db3200640 1 --2- 172.21.15.131:0/1861718965 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6db406d250 0x7f6db4153d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:56.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:56.465+0000 7f6db3200640 1 -- 172.21.15.131:0/1861718965 >> 172.21.15.131:0/1861718965 conn(0x7f6db40b5660 msgr2=0x7f6db40b0440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:56.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:56.466+0000 7f6db3200640 1 -- 172.21.15.131:0/1861718965 shutdown_connections 2024-05-26T22:53:56.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:56.466+0000 7f6db3200640 1 -- 172.21.15.131:0/1861718965 wait complete. 2024-05-26T22:53:56.487 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-26T22:53:56.487 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-26T22:53:56.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:56.686+0000 7f347b400640 1 Processor -- start 2024-05-26T22:53:56.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:56.686+0000 7f347b400640 1 -- start start 2024-05-26T22:53:56.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:56.688+0000 7f347b400640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f347406ee20 0x7f34740b3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:56.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:56.688+0000 7f347b400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f34740ad250 0x7f34740af790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:56.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:56.688+0000 7f347b400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f347406f200 0x7f34741562c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:56.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:56.688+0000 7f347b400640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f347415cb20 con 0x7f34740ad250 2024-05-26T22:53:56.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:56.688+0000 7f347b400640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f347415ce00 con 0x7f347406ee20 2024-05-26T22:53:56.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:56.688+0000 7f347b400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f347415d0e0 con 0x7f347406f200 2024-05-26T22:53:56.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:56.688+0000 7f3473e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f34740ad250 0x7f34740af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:56.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:56.688+0000 7f3473e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f34740ad250 0x7f34740af790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56704/0 (socket says 172.21.15.131:56704) 2024-05-26T22:53:56.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:56.688+0000 7f3473e00640 1 -- 172.21.15.131:0/1571194857 learned_addr learned my addr 172.21.15.131:0/1571194857 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:53:56.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:56.688+0000 7f3478a00640 1 --2- 172.21.15.131:0/1571194857 >> v2:172.21.15.187:3300/0 conn(0x7f347406ee20 0x7f34740b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:56.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:56.688+0000 7f3479400640 1 --2- 172.21.15.131:0/1571194857 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f347406f200 0x7f34741562c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:56.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:56.689+0000 7f3473e00640 1 -- 172.21.15.131:0/1571194857 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f347406f200 msgr2=0x7f34741562c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:56.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:56.689+0000 7f3473e00640 1 --2- 172.21.15.131:0/1571194857 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f347406f200 0x7f34741562c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:56.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:56.689+0000 7f3473e00640 1 -- 172.21.15.131:0/1571194857 >> v2:172.21.15.187:3300/0 conn(0x7f347406ee20 msgr2=0x7f34740b3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:56.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:56.689+0000 7f3473e00640 1 --2- 172.21.15.131:0/1571194857 >> v2:172.21.15.187:3300/0 conn(0x7f347406ee20 0x7f34740b3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:56.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:56.689+0000 7f3473e00640 1 -- 172.21.15.131:0/1571194857 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f347415d3c0 con 0x7f34740ad250 2024-05-26T22:53:56.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:56.690+0000 7f3473e00640 1 --2- 172.21.15.131:0/1571194857 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f34740ad250 0x7f34740af790 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f3474071090 tx=0x7f346002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61d3b5c49d37c4e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:56.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:56.690+0000 7f3473400640 1 -- 172.21.15.131:0/1571194857 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3460037070 con 0x7f34740ad250 2024-05-26T22:53:56.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:56.690+0000 7f3473400640 1 -- 172.21.15.131:0/1571194857 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3460032440 con 0x7f34740ad250 2024-05-26T22:53:56.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:56.690+0000 7f3473400640 1 -- 172.21.15.131:0/1571194857 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3460032810 con 0x7f34740ad250 2024-05-26T22:53:56.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:56.691+0000 7f347b400640 1 -- 172.21.15.131:0/1571194857 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f34740ad250 msgr2=0x7f34740af790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:56.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:56.691+0000 7f347b400640 1 --2- 172.21.15.131:0/1571194857 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f34740ad250 0x7f34740af790 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f3474071090 tx=0x7f346002a1e0 comp rx=0 tx=0).stop 2024-05-26T22:53:56.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:56.691+0000 7f347b400640 1 -- 172.21.15.131:0/1571194857 shutdown_connections 2024-05-26T22:53:56.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:56.691+0000 7f347b400640 1 --2- 172.21.15.131:0/1571194857 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f347406f200 0x7f34741562c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:56.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:56.691+0000 7f347b400640 1 --2- 172.21.15.131:0/1571194857 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f34740ad250 0x7f34740af790 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:56.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:56.691+0000 7f347b400640 1 --2- 172.21.15.131:0/1571194857 >> v2:172.21.15.187:3300/0 conn(0x7f347406ee20 0x7f34740b3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:56.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:56.691+0000 7f347b400640 1 -- 172.21.15.131:0/1571194857 >> 172.21.15.131:0/1571194857 conn(0x7f34740b1e90 msgr2=0x7f34740b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:56.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:56.691+0000 7f347b400640 1 -- 172.21.15.131:0/1571194857 shutdown_connections 2024-05-26T22:53:56.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:56.691+0000 7f347b400640 1 -- 172.21.15.131:0/1571194857 wait complete. 2024-05-26T22:53:56.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:56.693+0000 7f347b400640 1 Processor -- start 2024-05-26T22:53:56.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:56.693+0000 7f347b400640 1 -- start start 2024-05-26T22:53:56.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:56.694+0000 7f347b400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f347406ee20 0x7f34740cdb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:56.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:56.694+0000 7f347b400640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f347406f200 0x7f347409d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:56.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:56.694+0000 7f347b400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f34740ad250 0x7f347409df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:56.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:56.694+0000 7f347b400640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f347415c900 con 0x7f347406ee20 2024-05-26T22:53:56.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:56.694+0000 7f347b400640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f347415c780 con 0x7f347406f200 2024-05-26T22:53:56.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:56.694+0000 7f347b400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3474156800 con 0x7f34740ad250 2024-05-26T22:53:56.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:56.695+0000 7f3478a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f347406ee20 0x7f34740cdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:56.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:56.695+0000 7f3478a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f347406ee20 0x7f34740cdb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56710/0 (socket says 172.21.15.131:56710) 2024-05-26T22:53:56.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:56.695+0000 7f3478a00640 1 -- 172.21.15.131:0/1285507201 learned_addr learned my addr 172.21.15.131:0/1285507201 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:53:56.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:56.695+0000 7f3479400640 1 --2- 172.21.15.131:0/1285507201 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f34740ad250 0x7f347409df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:56.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:56.695+0000 7f3478a00640 1 -- 172.21.15.131:0/1285507201 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f34740ad250 msgr2=0x7f347409df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:56.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:56.695+0000 7f3478a00640 1 --2- 172.21.15.131:0/1285507201 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f34740ad250 0x7f347409df00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:56.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:56.695+0000 7f3473e00640 1 --2- 172.21.15.131:0/1285507201 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f347406f200 0x7f347409d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:56.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:56.695+0000 7f3478a00640 1 -- 172.21.15.131:0/1285507201 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f347406f200 msgr2=0x7f347409d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:56.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:56.695+0000 7f3478a00640 1 --2- 172.21.15.131:0/1285507201 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f347406f200 0x7f347409d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:56.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:56.695+0000 7f3478a00640 1 -- 172.21.15.131:0/1285507201 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3474072040 con 0x7f347406ee20 2024-05-26T22:53:56.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:56.695+0000 7f3478a00640 1 --2- 172.21.15.131:0/1285507201 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f347406ee20 0x7f34740cdb80 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f346400d800 tx=0x7f346400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:56.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:56.696+0000 7f3470c00640 1 -- 172.21.15.131:0/1285507201 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f346400b6c0 con 0x7f347406ee20 2024-05-26T22:53:56.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:56.696+0000 7f3470c00640 1 -- 172.21.15.131:0/1285507201 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3464019070 con 0x7f347406ee20 2024-05-26T22:53:56.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:56.696+0000 7f3470c00640 1 -- 172.21.15.131:0/1285507201 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f346400ba80 con 0x7f347406ee20 2024-05-26T22:53:56.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:56.696+0000 7f347b400640 1 -- 172.21.15.131:0/1285507201 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34740722e0 con 0x7f347406ee20 2024-05-26T22:53:56.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:56.696+0000 7f347b400640 1 -- 172.21.15.131:0/1285507201 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34740728a0 con 0x7f347406ee20 2024-05-26T22:53:56.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:56.697+0000 7f347b400640 1 -- 172.21.15.131:0/1285507201 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34740694e0 con 0x7f347406ee20 2024-05-26T22:53:56.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:56.698+0000 7f3470c00640 1 -- 172.21.15.131:0/1285507201 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f346400bc20 con 0x7f347406ee20 2024-05-26T22:53:56.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:56.699+0000 7f3470c00640 1 --2- 172.21.15.131:0/1285507201 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f345406be30 0x7f345406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:56.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:56.699+0000 7f3470c00640 1 -- 172.21.15.131:0/1285507201 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5709+0+0 (secure 0 0 0) 0x7f346408b2a0 con 0x7f347406ee20 2024-05-26T22:53:56.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:56.699+0000 7f3473e00640 1 --2- 172.21.15.131:0/1285507201 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f345406be30 0x7f345406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:56.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:56.700+0000 7f3473e00640 1 --2- 172.21.15.131:0/1285507201 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f345406be30 0x7f345406e2f0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f3460034000 tx=0x7f346002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:56.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:56.703+0000 7f3470c00640 1 -- 172.21.15.131:0/1285507201 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f3464059ce0 con 0x7f347406ee20 2024-05-26T22:53:56.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:56.932+0000 7f347b400640 1 -- 172.21.15.131:0/1285507201 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3474072c80 con 0x7f347406ee20 2024-05-26T22:53:56.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:56.934+0000 7f3470c00640 1 -- 172.21.15.131:0/1285507201 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) ==== 94+0+5189 (secure 0 0 0) 0x7f34640595e0 con 0x7f347406ee20 2024-05-26T22:53:56.934 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:53:56.934 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":78,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5533,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/993930297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6835","nonce":993930297},{"type":"v1","addr":"172.21.15.131:6837","nonce":993930297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":73},{"gid":5545,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6834/3480101957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3480101957},{"type":"v1","addr":"172.21.15.187:6834","nonce":3480101957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":73},{"gid":5548,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/567389303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6833","nonce":567389303},{"type":"v1","addr":"172.21.15.187:6835","nonce":567389303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-26T22:53:52.862328+0000","modified":"2024-05-26T22:53:55.926183+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7365},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7365":{"gid":7365,"name":"a","rank":0,"incarnation":74,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6836/3411392567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":3411392567},{"type":"v1","addr":"172.21.15.131:6836","nonce":3411392567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7365,"qdb_cluster":[7365]},"id":6}]} 2024-05-26T22:53:56.934 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 78 2024-05-26T22:53:56.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:56.936+0000 7f347b400640 1 -- 172.21.15.131:0/1285507201 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f345406be30 msgr2=0x7f345406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:56.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:56.936+0000 7f347b400640 1 --2- 172.21.15.131:0/1285507201 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f345406be30 0x7f345406e2f0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f3460034000 tx=0x7f346002e000 comp rx=0 tx=0).stop 2024-05-26T22:53:56.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:56.936+0000 7f347b400640 1 -- 172.21.15.131:0/1285507201 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f347406ee20 msgr2=0x7f34740cdb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:56.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:56.936+0000 7f347b400640 1 --2- 172.21.15.131:0/1285507201 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f347406ee20 0x7f34740cdb80 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f346400d800 tx=0x7f346400dcc0 comp rx=0 tx=0).stop 2024-05-26T22:53:56.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:56.936+0000 7f347b400640 1 -- 172.21.15.131:0/1285507201 shutdown_connections 2024-05-26T22:53:56.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:56.936+0000 7f347b400640 1 --2- 172.21.15.131:0/1285507201 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f34740ad250 0x7f347409df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:56.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:56.936+0000 7f347b400640 1 --2- 172.21.15.131:0/1285507201 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f345406be30 0x7f345406e2f0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:56.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:56.936+0000 7f347b400640 1 --2- 172.21.15.131:0/1285507201 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f347406f200 0x7f347409d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:56.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:56.936+0000 7f347b400640 1 --2- 172.21.15.131:0/1285507201 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f347406ee20 0x7f34740cdb80 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:56.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:56.936+0000 7f347b400640 1 -- 172.21.15.131:0/1285507201 >> 172.21.15.131:0/1285507201 conn(0x7f34740b1e90 msgr2=0x7f347415e140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:56.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:56.936+0000 7f347b400640 1 -- 172.21.15.131:0/1285507201 shutdown_connections 2024-05-26T22:53:56.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:56.937+0000 7f347b400640 1 -- 172.21.15.131:0/1285507201 wait complete. 2024-05-26T22:53:56.955 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-26T22:53:57.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.155+0000 7fa8b7c00640 1 Processor -- start 2024-05-26T22:53:57.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.155+0000 7fa8b7c00640 1 -- start start 2024-05-26T22:53:57.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.157+0000 7fa8b7c00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa8b00ad250 0x7fa8b00b3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:57.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.157+0000 7fa8b7c00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8b015cb20 con 0x7fa8b006ee20 2024-05-26T22:53:57.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.157+0000 7fa8b7c00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8b015ce00 con 0x7fa8b01569d0 2024-05-26T22:53:57.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.157+0000 7fa8b7c00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa8b015d0e0 con 0x7fa8b00ad250 2024-05-26T22:53:57.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.157+0000 7fa8b5200640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fa8b006ee20 0x7fa8b01562c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:50638/0 (socket says 172.21.15.131:50638) 2024-05-26T22:53:57.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.157+0000 7fa8b5200640 1 -- 172.21.15.131:0/2616394821 learned_addr learned my addr 172.21.15.131:0/2616394821 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:53:57.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.157+0000 7fa8af400640 1 -- 172.21.15.131:0/2616394821 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1078511532 0 0) 0x7fa8b015cb20 con 0x7fa8b006ee20 2024-05-26T22:53:57.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.157+0000 7fa8af400640 1 -- 172.21.15.131:0/2616394821 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa898003520 con 0x7fa8b006ee20 2024-05-26T22:53:57.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.157+0000 7fa8afe00640 1 --2- 172.21.15.131:0/2616394821 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa8b00ad250 0x7fa8b00b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:57.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.158+0000 7fa8af400640 1 -- 172.21.15.131:0/2616394821 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1265435400 0 0) 0x7fa898003520 con 0x7fa8b006ee20 2024-05-26T22:53:57.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.158+0000 7fa8af400640 1 -- 172.21.15.131:0/2616394821 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa8b00ad250 msgr2=0x7fa8b00b3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:57.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.158+0000 7fa8af400640 1 --2- 172.21.15.131:0/2616394821 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa8b00ad250 0x7fa8b00b3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:57.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.158+0000 7fa8af400640 1 -- 172.21.15.131:0/2616394821 >> v1:172.21.15.187:6789/0 conn(0x7fa8b01569d0 legacy=0x7fa8b0160010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:57.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.158+0000 7fa8af400640 1 -- 172.21.15.131:0/2616394821 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8b015d3c0 con 0x7fa8b006ee20 2024-05-26T22:53:57.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.158+0000 7fa8af400640 1 -- 172.21.15.131:0/2616394821 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7fa8a4001670 con 0x7fa8b006ee20 2024-05-26T22:53:57.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.158+0000 7fa8af400640 1 -- 172.21.15.131:0/2616394821 >> v1:172.21.15.131:6789/0 conn(0x7fa8b006ee20 legacy=0x7fa8b01562c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:57.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.158+0000 7fa8af400640 1 --2- 172.21.15.131:0/2616394821 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8980039e0 0x7fa898003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:57.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.158+0000 7fa8af400640 1 --2- 172.21.15.131:0/2616394821 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa898004800 0x7fa898004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:57.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.158+0000 7fa8af400640 1 --2- 172.21.15.131:0/2616394821 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa8b00ad250 0x7fa898005270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:57.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.158+0000 7fa8af400640 1 -- 172.21.15.131:0/2616394821 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa8b015d0e0 con 0x7fa8980039e0 2024-05-26T22:53:57.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.158+0000 7fa8af400640 1 -- 172.21.15.131:0/2616394821 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa89800bb50 con 0x7fa898004800 2024-05-26T22:53:57.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.158+0000 7fa8af400640 1 -- 172.21.15.131:0/2616394821 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa89800bd50 con 0x7fa8b00ad250 2024-05-26T22:53:57.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.158+0000 7fa8b5200640 1 --2- 172.21.15.131:0/2616394821 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8980039e0 0x7fa898003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:57.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.158+0000 7fa8b5200640 1 -- 172.21.15.131:0/2616394821 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa8b00ad250 msgr2=0x7fa898005270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:57.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.158+0000 7fa8b5200640 1 --2- 172.21.15.131:0/2616394821 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa8b00ad250 0x7fa898005270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:57.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.158+0000 7fa8b5200640 1 -- 172.21.15.131:0/2616394821 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa898004800 msgr2=0x7fa898004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:57.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.158+0000 7fa8b5200640 1 --2- 172.21.15.131:0/2616394821 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa898004800 0x7fa898004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:57.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.158+0000 7fa8b5200640 1 -- 172.21.15.131:0/2616394821 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa898003520 con 0x7fa8980039e0 2024-05-26T22:53:57.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.159+0000 7fa8b5200640 1 --2- 172.21.15.131:0/2616394821 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8980039e0 0x7fa898003de0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7fa8a400c190 tx=0x7fa898005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5076751ce02d0a84 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:57.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.160+0000 7fa8af400640 1 -- 172.21.15.131:0/2616394821 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8a4007c60 con 0x7fa8980039e0 2024-05-26T22:53:57.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.160+0000 7fa8af400640 1 -- 172.21.15.131:0/2616394821 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8a4005ad0 con 0x7fa8980039e0 2024-05-26T22:53:57.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.160+0000 7fa8af400640 1 -- 172.21.15.131:0/2616394821 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8a4007070 con 0x7fa8980039e0 2024-05-26T22:53:57.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.160+0000 7fa8b7c00640 1 -- 172.21.15.131:0/2616394821 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8980039e0 msgr2=0x7fa898003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:57.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.160+0000 7fa8b7c00640 1 --2- 172.21.15.131:0/2616394821 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8980039e0 0x7fa898003de0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7fa8a400c190 tx=0x7fa898005830 comp rx=0 tx=0).stop 2024-05-26T22:53:57.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.160+0000 7fa8b5c00640 1 -- 172.21.15.131:0/2616394821 reap_dead start 2024-05-26T22:53:57.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.160+0000 7fa8b7c00640 1 -- 172.21.15.131:0/2616394821 shutdown_connections 2024-05-26T22:53:57.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.160+0000 7fa8b7c00640 1 -- 172.21.15.131:0/2616394821 >> 172.21.15.131:0/2616394821 conn(0x7fa8b00b1e90 msgr2=0x7fa8b00b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:57.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.161+0000 7fa8b7c00640 1 -- 172.21.15.131:0/2616394821 shutdown_connections 2024-05-26T22:53:57.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.161+0000 7fa8b7c00640 1 -- 172.21.15.131:0/2616394821 wait complete. 2024-05-26T22:53:57.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.162+0000 7fa8b7c00640 1 Processor -- start 2024-05-26T22:53:57.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.162+0000 7fa8b7c00640 1 -- start start 2024-05-26T22:53:57.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.163+0000 7fa8b7c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8980039e0 0x7fa8b0147230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:57.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.163+0000 7fa8b7c00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa8b0146900 0x7fa8b00b1050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:57.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.163+0000 7fa8b7c00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa8b01f6f00 0x7fa8b0154510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:57.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.163+0000 7fa8b7c00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa8b015c900 con 0x7fa8980039e0 2024-05-26T22:53:57.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.163+0000 7fa8b7c00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa8b015c780 con 0x7fa8b0146900 2024-05-26T22:53:57.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.163+0000 7fa8b7c00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa8b006f200 con 0x7fa8b01f6f00 2024-05-26T22:53:57.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.164+0000 7fa8b5200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8980039e0 0x7fa8b0147230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:57.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.164+0000 7fa8b5200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8980039e0 0x7fa8b0147230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56722/0 (socket says 172.21.15.131:56722) 2024-05-26T22:53:57.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.164+0000 7fa8b5200640 1 -- 172.21.15.131:0/954138421 learned_addr learned my addr 172.21.15.131:0/954138421 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:53:57.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.164+0000 7fa8b5200640 1 -- 172.21.15.131:0/954138421 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa8b01f6f00 msgr2=0x7fa8b0154510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:57.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.164+0000 7fa8b5200640 1 --2- 172.21.15.131:0/954138421 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa8b01f6f00 0x7fa8b0154510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:57.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.164+0000 7fa8b5200640 1 -- 172.21.15.131:0/954138421 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa8b0146900 msgr2=0x7fa8b00b1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:57.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.164+0000 7fa8afe00640 1 --2- 172.21.15.131:0/954138421 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa8b0146900 0x7fa8b00b1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:57.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.164+0000 7fa8b5200640 1 --2- 172.21.15.131:0/954138421 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa8b0146900 0x7fa8b00b1050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:57.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.164+0000 7fa8b5200640 1 -- 172.21.15.131:0/954138421 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8b00b1590 con 0x7fa8980039e0 2024-05-26T22:53:57.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.164+0000 7fa8b5200640 1 --2- 172.21.15.131:0/954138421 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8980039e0 0x7fa8b0147230 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fa8a400c2b0 tx=0x7fa8b015cfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:57.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.165+0000 7fa8acc00640 1 -- 172.21.15.131:0/954138421 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8a4043070 con 0x7fa8980039e0 2024-05-26T22:53:57.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.165+0000 7fa8acc00640 1 -- 172.21.15.131:0/954138421 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8a4038040 con 0x7fa8980039e0 2024-05-26T22:53:57.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.165+0000 7fa8b7c00640 1 -- 172.21.15.131:0/954138421 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8b0143de0 con 0x7fa8980039e0 2024-05-26T22:53:57.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.165+0000 7fa8acc00640 1 -- 172.21.15.131:0/954138421 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8a4007070 con 0x7fa8980039e0 2024-05-26T22:53:57.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.165+0000 7fa8b7c00640 1 -- 172.21.15.131:0/954138421 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8b01443a0 con 0x7fa8980039e0 2024-05-26T22:53:57.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.166+0000 7fa8b7c00640 1 -- 172.21.15.131:0/954138421 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8b00694e0 con 0x7fa8980039e0 2024-05-26T22:53:57.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.167+0000 7fa8acc00640 1 -- 172.21.15.131:0/954138421 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa8a4007230 con 0x7fa8980039e0 2024-05-26T22:53:57.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.167+0000 7fa8acc00640 1 --2- 172.21.15.131:0/954138421 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa89006be60 0x7fa89006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:57.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.168+0000 7fa8acc00640 1 -- 172.21.15.131:0/954138421 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5709+0+0 (secure 0 0 0) 0x7fa8a40aa920 con 0x7fa8980039e0 2024-05-26T22:53:57.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.168+0000 7fa8afe00640 1 --2- 172.21.15.131:0/954138421 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa89006be60 0x7fa89006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:57.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.169+0000 7fa8afe00640 1 --2- 172.21.15.131:0/954138421 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa89006be60 0x7fa89006e320 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fa89c006a90 tx=0x7fa89c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:57.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.173+0000 7fa8acc00640 1 -- 172.21.15.131:0/954138421 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fa8a4005aa0 con 0x7fa8980039e0 2024-05-26T22:53:57.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.373+0000 7fa8b7c00640 1 -- 172.21.15.131:0/954138421 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa8b0146ce0 con 0x7fa8980039e0 2024-05-26T22:53:57.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.376+0000 7fa8acc00640 1 -- 172.21.15.131:0/954138421 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+16684 (secure 0 0 0) 0x7fa8a4078c70 con 0x7fa8980039e0 2024-05-26T22:53:57.376 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:53:57.376 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":82,"fsid":"46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f","created":"2024-05-26T22:47:57.288905+0000","modified":"2024-05-26T22:53:54.429609+0000","last_up_change":"2024-05-26T22:48:09.602207+0000","last_in_change":"2024-05-26T22:48:00.687341+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-26T22:48:10.463339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-05-26T22:53:50.625629+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-05-26T22:53:51.864320+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"814eaf62-a18d-494b-8e27-fd8fb88adea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6801","nonce":2350940051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6803","nonce":2350940051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6807","nonce":2350940051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6805","nonce":2350940051}]},"public_addr":"172.21.15.131:6801/2350940051","cluster_addr":"172.21.15.131:6803/2350940051","heartbeat_back_addr":"172.21.15.131:6807/2350940051","heartbeat_front_addr":"172.21.15.131:6805/2350940051","state":["exists","up"]},{"osd":1,"uuid":"42e66287-f6c6-4e26-af3c-6dfaa3536d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6825","nonce":1525528247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6827","nonce":1525528247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6831","nonce":1525528247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6829","nonce":1525528247}]},"public_addr":"172.21.15.131:6825/1525528247","cluster_addr":"172.21.15.131:6827/1525528247","heartbeat_back_addr":"172.21.15.131:6831/1525528247","heartbeat_front_addr":"172.21.15.131:6829/1525528247","state":["exists","up"]},{"osd":2,"uuid":"0da34673-04a2-4fd0-b57a-0ff2b679b043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6817","nonce":3627842210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6819","nonce":3627842210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6823","nonce":3627842210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6821","nonce":3627842210}]},"public_addr":"172.21.15.131:6817/3627842210","cluster_addr":"172.21.15.131:6819/3627842210","heartbeat_back_addr":"172.21.15.131:6823/3627842210","heartbeat_front_addr":"172.21.15.131:6821/3627842210","state":["exists","up"]},{"osd":3,"uuid":"cf117a27-d6b2-43d3-ae52-b47988bf8eed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6809","nonce":350790591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6811","nonce":350790591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6815","nonce":350790591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6813","nonce":350790591}]},"public_addr":"172.21.15.131:6809/350790591","cluster_addr":"172.21.15.131:6811/350790591","heartbeat_back_addr":"172.21.15.131:6815/350790591","heartbeat_front_addr":"172.21.15.131:6813/350790591","state":["exists","up"]},{"osd":4,"uuid":"a092ccfc-be89-4a84-88d5-0f781fa715eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6816","nonce":244560995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6819","nonce":244560995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6823","nonce":244560995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6821","nonce":244560995}]},"public_addr":"172.21.15.187:6816/244560995","cluster_addr":"172.21.15.187:6819/244560995","heartbeat_back_addr":"172.21.15.187:6823/244560995","heartbeat_front_addr":"172.21.15.187:6821/244560995","state":["exists","up"]},{"osd":5,"uuid":"7513ba69-dd00-4a35-b190-9e8cc3b2bdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6801","nonce":324619398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6803","nonce":324619398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6812","nonce":324619398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6807","nonce":324619398}]},"public_addr":"172.21.15.187:6801/324619398","cluster_addr":"172.21.15.187:6803/324619398","heartbeat_back_addr":"172.21.15.187:6812/324619398","heartbeat_front_addr":"172.21.15.187:6807/324619398","state":["exists","up"]},{"osd":6,"uuid":"50081810-c093-4d2f-a4d7-b7c695c5400d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6806","nonce":1391347786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6809","nonce":1391347786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6818","nonce":1391347786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6813","nonce":1391347786}]},"public_addr":"172.21.15.187:6806/1391347786","cluster_addr":"172.21.15.187:6809/1391347786","heartbeat_back_addr":"172.21.15.187:6818/1391347786","heartbeat_front_addr":"172.21.15.187:6813/1391347786","state":["exists","up"]},{"osd":7,"uuid":"775dae05-54f5-4ad4-bbce-92863c6ed2ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6825","nonce":1220065622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6827","nonce":1220065622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6831","nonce":1220065622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6829","nonce":1220065622}]},"public_addr":"172.21.15.187:6825/1220065622","cluster_addr":"172.21.15.187:6827/1220065622","heartbeat_back_addr":"172.21.15.187:6831/1220065622","heartbeat_front_addr":"172.21.15.187:6829/1220065622","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:05.983520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.299516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.067323+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.307748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.018742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.258775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.010912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.295355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-26T22:53:57.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.379+0000 7fa8b7c00640 1 -- 172.21.15.131:0/954138421 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa89006be60 msgr2=0x7fa89006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:57.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.379+0000 7fa8b7c00640 1 --2- 172.21.15.131:0/954138421 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa89006be60 0x7fa89006e320 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fa89c006a90 tx=0x7fa89c008040 comp rx=0 tx=0).stop 2024-05-26T22:53:57.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.379+0000 7fa8b7c00640 1 -- 172.21.15.131:0/954138421 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8980039e0 msgr2=0x7fa8b0147230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:57.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.379+0000 7fa8b7c00640 1 --2- 172.21.15.131:0/954138421 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8980039e0 0x7fa8b0147230 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fa8a400c2b0 tx=0x7fa8b015cfd0 comp rx=0 tx=0).stop 2024-05-26T22:53:57.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.379+0000 7fa8b7c00640 1 -- 172.21.15.131:0/954138421 shutdown_connections 2024-05-26T22:53:57.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.379+0000 7fa8b7c00640 1 --2- 172.21.15.131:0/954138421 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa8b01f6f00 0x7fa8b0154510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:57.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.379+0000 7fa8b7c00640 1 --2- 172.21.15.131:0/954138421 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa89006be60 0x7fa89006e320 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:57.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.379+0000 7fa8b7c00640 1 --2- 172.21.15.131:0/954138421 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa8b0146900 0x7fa8b00b1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:57.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.379+0000 7fa8b7c00640 1 --2- 172.21.15.131:0/954138421 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8980039e0 0x7fa8b0147230 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:57.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.379+0000 7fa8b7c00640 1 -- 172.21.15.131:0/954138421 >> 172.21.15.131:0/954138421 conn(0x7fa8b00b1e90 msgr2=0x7fa8b015e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:57.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.379+0000 7fa8b7c00640 1 -- 172.21.15.131:0/954138421 shutdown_connections 2024-05-26T22:53:57.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.380+0000 7fa8b7c00640 1 -- 172.21.15.131:0/954138421 wait complete. 2024-05-26T22:53:57.398 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-26T22:53:57.399 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-26T22:53:57.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.596+0000 7f022a000640 1 Processor -- start 2024-05-26T22:53:57.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.597+0000 7f022a000640 1 -- start start 2024-05-26T22:53:57.598 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.598+0000 7f022a000640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f0224158c00 0x7f022406d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:57.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.598+0000 7f022a000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f02240b38a0 0x7f022415fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:57.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.598+0000 7f022a000640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02240b3e50 con 0x7f022406daf0 2024-05-26T22:53:57.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.598+0000 7f022a000640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f022406df50 con 0x7f0224158c00 2024-05-26T22:53:57.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.598+0000 7f022a000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f02240b4130 con 0x7f02240b38a0 2024-05-26T22:53:57.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.598+0000 7f0223400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f0224158c00 0x7f022406d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:57.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.598+0000 7f0223400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f0224158c00 0x7f022406d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56738/0 (socket says 172.21.15.131:56738) 2024-05-26T22:53:57.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.598+0000 7f0223400640 1 -- 172.21.15.131:0/1798662089 learned_addr learned my addr 172.21.15.131:0/1798662089 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:53:57.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.598+0000 7f0222a00640 1 --1- 172.21.15.131:0/1798662089 >> v1:172.21.15.131:6789/0 conn(0x7f022406daf0 0x7f02240b3190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:50650/0 (socket says 172.21.15.131:50650) 2024-05-26T22:53:57.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.599+0000 7f0223400640 1 -- 172.21.15.131:0/1798662089 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f02240b38a0 msgr2=0x7f022415fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:57.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.599+0000 7f0223400640 1 --2- 172.21.15.131:0/1798662089 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f02240b38a0 0x7f022415fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:57.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.599+0000 7f0223400640 1 -- 172.21.15.131:0/1798662089 >> v1:172.21.15.131:6789/0 conn(0x7f022406daf0 legacy=0x7f02240b3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:57.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.599+0000 7f0223400640 1 -- 172.21.15.131:0/1798662089 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02240b4410 con 0x7f0224158c00 2024-05-26T22:53:57.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.599+0000 7f0222000640 1 -- 172.21.15.131:0/1798662089 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1600764473 0 0) 0x7f02240b3e50 con 0x7f022406daf0 2024-05-26T22:53:57.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.600+0000 7f0223400640 1 --2- 172.21.15.131:0/1798662089 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0224158c00 0x7f022406d5b0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f0214009870 tx=0x7f021402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35a8d3b5e7240ef3 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:57.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.600+0000 7f0222000640 1 -- 172.21.15.131:0/1798662089 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0214037070 con 0x7f0224158c00 2024-05-26T22:53:57.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.600+0000 7f0222000640 1 -- 172.21.15.131:0/1798662089 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0214032440 con 0x7f0224158c00 2024-05-26T22:53:57.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.600+0000 7f0222000640 1 -- 172.21.15.131:0/1798662089 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0214032810 con 0x7f0224158c00 2024-05-26T22:53:57.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.601+0000 7f022a000640 1 -- 172.21.15.131:0/1798662089 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0224158c00 msgr2=0x7f022406d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:57.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.601+0000 7f022a000640 1 --2- 172.21.15.131:0/1798662089 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0224158c00 0x7f022406d5b0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f0214009870 tx=0x7f021402a1c0 comp rx=0 tx=0).stop 2024-05-26T22:53:57.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.601+0000 7f022a000640 1 -- 172.21.15.131:0/1798662089 shutdown_connections 2024-05-26T22:53:57.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.601+0000 7f022a000640 1 --2- 172.21.15.131:0/1798662089 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f02240b38a0 0x7f022415fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:57.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.601+0000 7f022a000640 1 --2- 172.21.15.131:0/1798662089 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0224158c00 0x7f022406d5b0 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:57.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.601+0000 7f022a000640 1 -- 172.21.15.131:0/1798662089 >> 172.21.15.131:0/1798662089 conn(0x7f02240b1e90 msgr2=0x7f02240b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:57.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.601+0000 7f022a000640 1 -- 172.21.15.131:0/1798662089 shutdown_connections 2024-05-26T22:53:57.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.601+0000 7f022a000640 1 -- 172.21.15.131:0/1798662089 wait complete. 2024-05-26T22:53:57.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.603+0000 7f022a000640 1 Processor -- start 2024-05-26T22:53:57.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.603+0000 7f022a000640 1 -- start start 2024-05-26T22:53:57.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.604+0000 7f022a000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f022406daf0 0x7f022414a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:57.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.604+0000 7f022a000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f02240b38a0 0x7f02241f41b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:57.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.604+0000 7f022a000640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0224158c00 0x7f02241f46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:57.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.604+0000 7f022a000640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f0224160190 con 0x7f022406daf0 2024-05-26T22:53:57.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.604+0000 7f022a000640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f02240b49d0 con 0x7f0224158c00 2024-05-26T22:53:57.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.604+0000 7f022a000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f0224160310 con 0x7f02240b38a0 2024-05-26T22:53:57.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.604+0000 7f0223400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f022406daf0 0x7f022414a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:57.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.604+0000 7f0223400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f022406daf0 0x7f022414a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56750/0 (socket says 172.21.15.131:56750) 2024-05-26T22:53:57.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.604+0000 7f0223400640 1 -- 172.21.15.131:0/1552165336 learned_addr learned my addr 172.21.15.131:0/1552165336 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:53:57.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.604+0000 7f0223400640 1 -- 172.21.15.131:0/1552165336 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f02240b38a0 msgr2=0x7f02241f41b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:57.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.604+0000 7f0223400640 1 --2- 172.21.15.131:0/1552165336 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f02240b38a0 0x7f02241f41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:57.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.604+0000 7f0223400640 1 -- 172.21.15.131:0/1552165336 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0224158c00 msgr2=0x7f02241f46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:57.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.604+0000 7f0223e00640 1 --2- 172.21.15.131:0/1552165336 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0224158c00 0x7f02241f46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:57.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.604+0000 7f0223400640 1 --2- 172.21.15.131:0/1552165336 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0224158c00 0x7f02241f46f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:57.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.604+0000 7f0223400640 1 -- 172.21.15.131:0/1552165336 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f022414aab0 con 0x7f022406daf0 2024-05-26T22:53:57.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.605+0000 7f0223400640 1 --2- 172.21.15.131:0/1552165336 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f022406daf0 0x7f022414a570 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f0214000c00 tx=0x7f02140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:57.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.605+0000 7f021b400640 1 -- 172.21.15.131:0/1552165336 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0214041070 con 0x7f022406daf0 2024-05-26T22:53:57.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.605+0000 7f021b400640 1 -- 172.21.15.131:0/1552165336 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f021403c870 con 0x7f022406daf0 2024-05-26T22:53:57.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.605+0000 7f021b400640 1 -- 172.21.15.131:0/1552165336 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0214037040 con 0x7f022406daf0 2024-05-26T22:53:57.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.605+0000 7f022a000640 1 -- 172.21.15.131:0/1552165336 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0224159e60 con 0x7f022406daf0 2024-05-26T22:53:57.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.605+0000 7f022a000640 1 -- 172.21.15.131:0/1552165336 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f022415a420 con 0x7f022406daf0 2024-05-26T22:53:57.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.607+0000 7f022a000640 1 -- 172.21.15.131:0/1552165336 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02240694e0 con 0x7f022406daf0 2024-05-26T22:53:57.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.607+0000 7f021b400640 1 -- 172.21.15.131:0/1552165336 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0214044070 con 0x7f022406daf0 2024-05-26T22:53:57.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.609+0000 7f021b400640 1 --2- 172.21.15.131:0/1552165336 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f020006be60 0x7f020006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:57.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.609+0000 7f021b400640 1 -- 172.21.15.131:0/1552165336 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5709+0+0 (secure 0 0 0) 0x7f02140aba00 con 0x7f022406daf0 2024-05-26T22:53:57.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.609+0000 7f0222a00640 1 --2- 172.21.15.131:0/1552165336 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f020006be60 0x7f020006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:57.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.610+0000 7f0222a00640 1 --2- 172.21.15.131:0/1552165336 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f020006be60 0x7f020006e320 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f020c001560 tx=0x7f020c007ca0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:57.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.613+0000 7f021b400640 1 -- 172.21.15.131:0/1552165336 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f021407a310 con 0x7f022406daf0 2024-05-26T22:53:57.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.808+0000 7f022a000640 1 -- 172.21.15.131:0/1552165336 --> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f022406ead0 con 0x7f020006be60 2024-05-26T22:53:57.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.828+0000 7f021b400640 1 -- 172.21.15.131:0/1552165336 <== mgr.4104 v2:172.21.15.131:6832/3484832947 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252047 (secure 0 0 0) 0x7f022406ead0 con 0x7f020006be60 2024-05-26T22:53:57.831 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:53:57.833 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-05-26T22:53:57.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.833+0000 7f022a000640 1 -- 172.21.15.131:0/1552165336 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f020006be60 msgr2=0x7f020006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:57.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.833+0000 7f022a000640 1 --2- 172.21.15.131:0/1552165336 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f020006be60 0x7f020006e320 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f020c001560 tx=0x7f020c007ca0 comp rx=0 tx=0).stop 2024-05-26T22:53:57.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.834+0000 7f022a000640 1 -- 172.21.15.131:0/1552165336 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f022406daf0 msgr2=0x7f022414a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:57.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.834+0000 7f022a000640 1 --2- 172.21.15.131:0/1552165336 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f022406daf0 0x7f022414a570 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f0214000c00 tx=0x7f02140046c0 comp rx=0 tx=0).stop 2024-05-26T22:53:57.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.834+0000 7f022a000640 1 -- 172.21.15.131:0/1552165336 shutdown_connections 2024-05-26T22:53:57.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.834+0000 7f022a000640 1 --2- 172.21.15.131:0/1552165336 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f02240b38a0 0x7f02241f41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:57.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.834+0000 7f022a000640 1 --2- 172.21.15.131:0/1552165336 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f020006be60 0x7f020006e320 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:57.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.834+0000 7f022a000640 1 --2- 172.21.15.131:0/1552165336 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0224158c00 0x7f02241f46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:57.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.834+0000 7f022a000640 1 --2- 172.21.15.131:0/1552165336 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f022406daf0 0x7f022414a570 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:57.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.834+0000 7f022a000640 1 -- 172.21.15.131:0/1552165336 >> 172.21.15.131:0/1552165336 conn(0x7f02240b1e90 msgr2=0x7f02240b07d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:57.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.834+0000 7f022a000640 1 -- 172.21.15.131:0/1552165336 shutdown_connections 2024-05-26T22:53:57.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:57.834+0000 7f022a000640 1 -- 172.21.15.131:0/1552165336 wait complete. 2024-05-26T22:53:57.853 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":248,"stamp":"2024-05-26T22:53:56.495283+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":120,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":258508,"kb_used_data":9048,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754716212,"statfs":{"total":773094113280,"available":772829401088,"internally_reserved":0,"allocated":9265152,"data_stored":4635356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.110651"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:54.447805+0000","last_change":"2024-05-26T22:53:53.443732+0000","last_active":"2024-05-26T22:53:54.447805+0000","last_peered":"2024-05-26T22:53:54.447805+0000","last_clean":"2024-05-26T22:53:54.447805+0000","last_became_active":"2024-05-26T22:53:53.443378+0000","last_became_peered":"2024-05-26T22:53:53.443378+0000","last_unstale":"2024-05-26T22:53:54.447805+0000","last_undegraded":"2024-05-26T22:53:54.447805+0000","last_fullsized":"2024-05-26T22:53:54.447805+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:01:50.450776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:54.445455+0000","last_change":"2024-05-26T22:53:53.450331+0000","last_active":"2024-05-26T22:53:54.445455+0000","last_peered":"2024-05-26T22:53:54.445455+0000","last_clean":"2024-05-26T22:53:54.445455+0000","last_became_active":"2024-05-26T22:53:53.450028+0000","last_became_peered":"2024-05-26T22:53:53.450028+0000","last_unstale":"2024-05-26T22:53:54.445455+0000","last_undegraded":"2024-05-26T22:53:54.445455+0000","last_fullsized":"2024-05-26T22:53:54.445455+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:14:20.756909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-26T22:53:53.451778+0000","last_change":"2024-05-26T22:53:53.449387+0000","last_active":"2024-05-26T22:53:53.451778+0000","last_peered":"2024-05-26T22:53:53.451778+0000","last_clean":"2024-05-26T22:53:53.451778+0000","last_became_active":"2024-05-26T22:53:53.449087+0000","last_became_peered":"2024-05-26T22:53:53.449087+0000","last_unstale":"2024-05-26T22:53:53.451778+0000","last_undegraded":"2024-05-26T22:53:53.451778+0000","last_fullsized":"2024-05-26T22:53:53.451778+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T23:49:11.202473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-26T22:53:53.457727+0000","last_change":"2024-05-26T22:53:53.455545+0000","last_active":"2024-05-26T22:53:53.457727+0000","last_peered":"2024-05-26T22:53:53.457727+0000","last_clean":"2024-05-26T22:53:53.457727+0000","last_became_active":"2024-05-26T22:53:53.455244+0000","last_became_peered":"2024-05-26T22:53:53.455244+0000","last_unstale":"2024-05-26T22:53:53.457727+0000","last_undegraded":"2024-05-26T22:53:53.457727+0000","last_fullsized":"2024-05-26T22:53:53.457727+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:27:37.532783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-26T22:53:53.453534+0000","last_change":"2024-05-26T22:53:53.450737+0000","last_active":"2024-05-26T22:53:53.453534+0000","last_peered":"2024-05-26T22:53:53.453534+0000","last_clean":"2024-05-26T22:53:53.453534+0000","last_became_active":"2024-05-26T22:53:53.450450+0000","last_became_peered":"2024-05-26T22:53:53.450450+0000","last_unstale":"2024-05-26T22:53:53.453534+0000","last_undegraded":"2024-05-26T22:53:53.453534+0000","last_fullsized":"2024-05-26T22:53:53.453534+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:27:50.935370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:54.853641+0000","last_change":"2024-05-26T22:53:53.443209+0000","last_active":"2024-05-26T22:53:54.853641+0000","last_peered":"2024-05-26T22:53:54.853641+0000","last_clean":"2024-05-26T22:53:54.853641+0000","last_became_active":"2024-05-26T22:53:53.442925+0000","last_became_peered":"2024-05-26T22:53:53.442925+0000","last_unstale":"2024-05-26T22:53:54.853641+0000","last_undegraded":"2024-05-26T22:53:54.853641+0000","last_fullsized":"2024-05-26T22:53:54.853641+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:34:26.592973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-26T22:53:53.455100+0000","last_change":"2024-05-26T22:53:53.447665+0000","last_active":"2024-05-26T22:53:53.455100+0000","last_peered":"2024-05-26T22:53:53.455100+0000","last_clean":"2024-05-26T22:53:53.455100+0000","last_became_active":"2024-05-26T22:53:53.447351+0000","last_became_peered":"2024-05-26T22:53:53.447351+0000","last_unstale":"2024-05-26T22:53:53.455100+0000","last_undegraded":"2024-05-26T22:53:53.455100+0000","last_fullsized":"2024-05-26T22:53:53.455100+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:26:01.737366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:54.853852+0000","last_change":"2024-05-26T22:53:53.449592+0000","last_active":"2024-05-26T22:53:54.853852+0000","last_peered":"2024-05-26T22:53:54.853852+0000","last_clean":"2024-05-26T22:53:54.853852+0000","last_became_active":"2024-05-26T22:53:53.448033+0000","last_became_peered":"2024-05-26T22:53:53.448033+0000","last_unstale":"2024-05-26T22:53:54.853852+0000","last_undegraded":"2024-05-26T22:53:54.853852+0000","last_fullsized":"2024-05-26T22:53:54.853852+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:43:55.783780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:54.854139+0000","last_change":"2024-05-26T22:53:53.438302+0000","last_active":"2024-05-26T22:53:54.854139+0000","last_peered":"2024-05-26T22:53:54.854139+0000","last_clean":"2024-05-26T22:53:54.854139+0000","last_became_active":"2024-05-26T22:53:53.437951+0000","last_became_peered":"2024-05-26T22:53:53.437951+0000","last_unstale":"2024-05-26T22:53:54.854139+0000","last_undegraded":"2024-05-26T22:53:54.854139+0000","last_fullsized":"2024-05-26T22:53:54.854139+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:19:16.286517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:54.447831+0000","last_change":"2024-05-26T22:53:53.444607+0000","last_active":"2024-05-26T22:53:54.447831+0000","last_peered":"2024-05-26T22:53:54.447831+0000","last_clean":"2024-05-26T22:53:54.447831+0000","last_became_active":"2024-05-26T22:53:53.444354+0000","last_became_peered":"2024-05-26T22:53:53.444354+0000","last_unstale":"2024-05-26T22:53:54.447831+0000","last_undegraded":"2024-05-26T22:53:54.447831+0000","last_fullsized":"2024-05-26T22:53:54.447831+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:03:19.630407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:54.445642+0000","last_change":"2024-05-26T22:53:53.452275+0000","last_active":"2024-05-26T22:53:54.445642+0000","last_peered":"2024-05-26T22:53:54.445642+0000","last_clean":"2024-05-26T22:53:54.445642+0000","last_became_active":"2024-05-26T22:53:53.451820+0000","last_became_peered":"2024-05-26T22:53:53.451820+0000","last_unstale":"2024-05-26T22:53:54.445642+0000","last_undegraded":"2024-05-26T22:53:54.445642+0000","last_fullsized":"2024-05-26T22:53:54.445642+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:33:22.065967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:54.447331+0000","last_change":"2024-05-26T22:53:53.441915+0000","last_active":"2024-05-26T22:53:54.447331+0000","last_peered":"2024-05-26T22:53:54.447331+0000","last_clean":"2024-05-26T22:53:54.447331+0000","last_became_active":"2024-05-26T22:53:53.441573+0000","last_became_peered":"2024-05-26T22:53:53.441573+0000","last_unstale":"2024-05-26T22:53:54.447331+0000","last_undegraded":"2024-05-26T22:53:54.447331+0000","last_fullsized":"2024-05-26T22:53:54.447331+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:04:38.162417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-26T22:53:53.455068+0000","last_change":"2024-05-26T22:53:53.452267+0000","last_active":"2024-05-26T22:53:53.455068+0000","last_peered":"2024-05-26T22:53:53.455068+0000","last_clean":"2024-05-26T22:53:53.455068+0000","last_became_active":"2024-05-26T22:53:53.451971+0000","last_became_peered":"2024-05-26T22:53:53.451971+0000","last_unstale":"2024-05-26T22:53:53.455068+0000","last_undegraded":"2024-05-26T22:53:53.455068+0000","last_fullsized":"2024-05-26T22:53:53.455068+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:40:30.639915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-26T22:53:53.458677+0000","last_change":"2024-05-26T22:53:53.455924+0000","last_active":"2024-05-26T22:53:53.458677+0000","last_peered":"2024-05-26T22:53:53.458677+0000","last_clean":"2024-05-26T22:53:53.458677+0000","last_became_active":"2024-05-26T22:53:53.455735+0000","last_became_peered":"2024-05-26T22:53:53.455735+0000","last_unstale":"2024-05-26T22:53:53.458677+0000","last_undegraded":"2024-05-26T22:53:53.458677+0000","last_fullsized":"2024-05-26T22:53:53.458677+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:16:47.977518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:55.474537+0000","last_change":"2024-05-26T22:53:53.447550+0000","last_active":"2024-05-26T22:53:55.474537+0000","last_peered":"2024-05-26T22:53:55.474537+0000","last_clean":"2024-05-26T22:53:55.474537+0000","last_became_active":"2024-05-26T22:53:53.447253+0000","last_became_peered":"2024-05-26T22:53:53.447253+0000","last_unstale":"2024-05-26T22:53:55.474537+0000","last_undegraded":"2024-05-26T22:53:55.474537+0000","last_fullsized":"2024-05-26T22:53:55.474537+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:21:55.566073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-26T22:53:53.450886+0000","last_change":"2024-05-26T22:53:53.443477+0000","last_active":"2024-05-26T22:53:53.450886+0000","last_peered":"2024-05-26T22:53:53.450886+0000","last_clean":"2024-05-26T22:53:53.450886+0000","last_became_active":"2024-05-26T22:53:53.443202+0000","last_became_peered":"2024-05-26T22:53:53.443202+0000","last_unstale":"2024-05-26T22:53:53.450886+0000","last_undegraded":"2024-05-26T22:53:53.450886+0000","last_fullsized":"2024-05-26T22:53:53.450886+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:13:50.325150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-26T22:53:53.456243+0000","last_change":"2024-05-26T22:53:53.455724+0000","last_active":"2024-05-26T22:53:53.456243+0000","last_peered":"2024-05-26T22:53:53.456243+0000","last_clean":"2024-05-26T22:53:53.456243+0000","last_became_active":"2024-05-26T22:53:53.455434+0000","last_became_peered":"2024-05-26T22:53:53.455434+0000","last_unstale":"2024-05-26T22:53:53.456243+0000","last_undegraded":"2024-05-26T22:53:53.456243+0000","last_fullsized":"2024-05-26T22:53:53.456243+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:37:40.975738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-26T22:53:53.443340+0000","last_change":"2024-05-26T22:53:53.436932+0000","last_active":"2024-05-26T22:53:53.443340+0000","last_peered":"2024-05-26T22:53:53.443340+0000","last_clean":"2024-05-26T22:53:53.443340+0000","last_became_active":"2024-05-26T22:53:53.436334+0000","last_became_peered":"2024-05-26T22:53:53.436334+0000","last_unstale":"2024-05-26T22:53:53.443340+0000","last_undegraded":"2024-05-26T22:53:53.443340+0000","last_fullsized":"2024-05-26T22:53:53.443340+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T22:55:25.504357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:54.443661+0000","last_change":"2024-05-26T22:53:53.442684+0000","last_active":"2024-05-26T22:53:54.443661+0000","last_peered":"2024-05-26T22:53:54.443661+0000","last_clean":"2024-05-26T22:53:54.443661+0000","last_became_active":"2024-05-26T22:53:53.442380+0000","last_became_peered":"2024-05-26T22:53:53.442380+0000","last_unstale":"2024-05-26T22:53:54.443661+0000","last_undegraded":"2024-05-26T22:53:54.443661+0000","last_fullsized":"2024-05-26T22:53:54.443661+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:02:02.966057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-26T22:53:53.445384+0000","last_change":"2024-05-26T22:53:53.437423+0000","last_active":"2024-05-26T22:53:53.445384+0000","last_peered":"2024-05-26T22:53:53.445384+0000","last_clean":"2024-05-26T22:53:53.445384+0000","last_became_active":"2024-05-26T22:53:53.437168+0000","last_became_peered":"2024-05-26T22:53:53.437168+0000","last_unstale":"2024-05-26T22:53:53.445384+0000","last_undegraded":"2024-05-26T22:53:53.445384+0000","last_fullsized":"2024-05-26T22:53:53.445384+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:57:54.261540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-26T22:53:53.444734+0000","last_change":"2024-05-26T22:53:53.439597+0000","last_active":"2024-05-26T22:53:53.444734+0000","last_peered":"2024-05-26T22:53:53.444734+0000","last_clean":"2024-05-26T22:53:53.444734+0000","last_became_active":"2024-05-26T22:53:53.439292+0000","last_became_peered":"2024-05-26T22:53:53.439292+0000","last_unstale":"2024-05-26T22:53:53.444734+0000","last_undegraded":"2024-05-26T22:53:53.444734+0000","last_fullsized":"2024-05-26T22:53:53.444734+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:50:26.618085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:54.445116+0000","last_change":"2024-05-26T22:53:53.459955+0000","last_active":"2024-05-26T22:53:54.445116+0000","last_peered":"2024-05-26T22:53:54.445116+0000","last_clean":"2024-05-26T22:53:54.445116+0000","last_became_active":"2024-05-26T22:53:53.459649+0000","last_became_peered":"2024-05-26T22:53:53.459649+0000","last_unstale":"2024-05-26T22:53:54.445116+0000","last_undegraded":"2024-05-26T22:53:54.445116+0000","last_fullsized":"2024-05-26T22:53:54.445116+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:27:57.433838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:54.446737+0000","last_change":"2024-05-26T22:53:53.443674+0000","last_active":"2024-05-26T22:53:54.446737+0000","last_peered":"2024-05-26T22:53:54.446737+0000","last_clean":"2024-05-26T22:53:54.446737+0000","last_became_active":"2024-05-26T22:53:53.442166+0000","last_became_peered":"2024-05-26T22:53:53.442166+0000","last_unstale":"2024-05-26T22:53:54.446737+0000","last_undegraded":"2024-05-26T22:53:54.446737+0000","last_fullsized":"2024-05-26T22:53:54.446737+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:03:47.595074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-26T22:53:53.443306+0000","last_change":"2024-05-26T22:53:53.436293+0000","last_active":"2024-05-26T22:53:53.443306+0000","last_peered":"2024-05-26T22:53:53.443306+0000","last_clean":"2024-05-26T22:53:53.443306+0000","last_became_active":"2024-05-26T22:53:53.435947+0000","last_became_peered":"2024-05-26T22:53:53.435947+0000","last_unstale":"2024-05-26T22:53:53.443306+0000","last_undegraded":"2024-05-26T22:53:53.443306+0000","last_fullsized":"2024-05-26T22:53:53.443306+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:10:11.640554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:54.445382+0000","last_change":"2024-05-26T22:53:53.453985+0000","last_active":"2024-05-26T22:53:54.445382+0000","last_peered":"2024-05-26T22:53:54.445382+0000","last_clean":"2024-05-26T22:53:54.445382+0000","last_became_active":"2024-05-26T22:53:53.453716+0000","last_became_peered":"2024-05-26T22:53:53.453716+0000","last_unstale":"2024-05-26T22:53:54.445382+0000","last_undegraded":"2024-05-26T22:53:54.445382+0000","last_fullsized":"2024-05-26T22:53:54.445382+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:07:15.226009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-26T22:53:53.458247+0000","last_change":"2024-05-26T22:53:53.456805+0000","last_active":"2024-05-26T22:53:53.458247+0000","last_peered":"2024-05-26T22:53:53.458247+0000","last_clean":"2024-05-26T22:53:53.458247+0000","last_became_active":"2024-05-26T22:53:53.456517+0000","last_became_peered":"2024-05-26T22:53:53.456517+0000","last_unstale":"2024-05-26T22:53:53.458247+0000","last_undegraded":"2024-05-26T22:53:53.458247+0000","last_fullsized":"2024-05-26T22:53:53.458247+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:04:25.600389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-26T22:53:53.455452+0000","last_change":"2024-05-26T22:53:53.453124+0000","last_active":"2024-05-26T22:53:53.455452+0000","last_peered":"2024-05-26T22:53:53.455452+0000","last_clean":"2024-05-26T22:53:53.455452+0000","last_became_active":"2024-05-26T22:53:53.452288+0000","last_became_peered":"2024-05-26T22:53:53.452288+0000","last_unstale":"2024-05-26T22:53:53.455452+0000","last_undegraded":"2024-05-26T22:53:53.455452+0000","last_fullsized":"2024-05-26T22:53:53.455452+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:34:26.592973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:54.447818+0000","last_change":"2024-05-26T22:53:53.445089+0000","last_active":"2024-05-26T22:53:54.447818+0000","last_peered":"2024-05-26T22:53:54.447818+0000","last_clean":"2024-05-26T22:53:54.447818+0000","last_became_active":"2024-05-26T22:53:53.444723+0000","last_became_peered":"2024-05-26T22:53:53.444723+0000","last_unstale":"2024-05-26T22:53:54.447818+0000","last_undegraded":"2024-05-26T22:53:54.447818+0000","last_fullsized":"2024-05-26T22:53:54.447818+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:02:02.966057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:54.445037+0000","last_change":"2024-05-26T22:53:53.450625+0000","last_active":"2024-05-26T22:53:54.445037+0000","last_peered":"2024-05-26T22:53:54.445037+0000","last_clean":"2024-05-26T22:53:54.445037+0000","last_became_active":"2024-05-26T22:53:53.449562+0000","last_became_peered":"2024-05-26T22:53:53.449562+0000","last_unstale":"2024-05-26T22:53:54.445037+0000","last_undegraded":"2024-05-26T22:53:54.445037+0000","last_fullsized":"2024-05-26T22:53:54.445037+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:34:55.983177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:54.445588+0000","last_change":"2024-05-26T22:53:53.447194+0000","last_active":"2024-05-26T22:53:54.445588+0000","last_peered":"2024-05-26T22:53:54.445588+0000","last_clean":"2024-05-26T22:53:54.445588+0000","last_became_active":"2024-05-26T22:53:53.446869+0000","last_became_peered":"2024-05-26T22:53:53.446869+0000","last_unstale":"2024-05-26T22:53:54.445588+0000","last_undegraded":"2024-05-26T22:53:54.445588+0000","last_fullsized":"2024-05-26T22:53:54.445588+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T23:49:11.534276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-26T22:53:53.447419+0000","last_change":"2024-05-26T22:53:53.440971+0000","last_active":"2024-05-26T22:53:53.447419+0000","last_peered":"2024-05-26T22:53:53.447419+0000","last_clean":"2024-05-26T22:53:53.447419+0000","last_became_active":"2024-05-26T22:53:53.439603+0000","last_became_peered":"2024-05-26T22:53:53.439603+0000","last_unstale":"2024-05-26T22:53:53.447419+0000","last_undegraded":"2024-05-26T22:53:53.447419+0000","last_fullsized":"2024-05-26T22:53:53.447419+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T23:49:11.534276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:54.446859+0000","last_change":"2024-05-26T22:53:53.449879+0000","last_active":"2024-05-26T22:53:54.446859+0000","last_peered":"2024-05-26T22:53:54.446859+0000","last_clean":"2024-05-26T22:53:54.446859+0000","last_became_active":"2024-05-26T22:53:53.449592+0000","last_became_peered":"2024-05-26T22:53:53.449592+0000","last_unstale":"2024-05-26T22:53:54.446859+0000","last_undegraded":"2024-05-26T22:53:54.446859+0000","last_fullsized":"2024-05-26T22:53:54.446859+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:52:18.410150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-26T22:53:53.450495+0000","last_change":"2024-05-26T22:53:53.444818+0000","last_active":"2024-05-26T22:53:53.450495+0000","last_peered":"2024-05-26T22:53:53.450495+0000","last_clean":"2024-05-26T22:53:53.450495+0000","last_became_active":"2024-05-26T22:53:53.443818+0000","last_became_peered":"2024-05-26T22:53:53.443818+0000","last_unstale":"2024-05-26T22:53:53.450495+0000","last_undegraded":"2024-05-26T22:53:53.450495+0000","last_fullsized":"2024-05-26T22:53:53.450495+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:14:20.756909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"81'4","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:54.448474+0000","last_change":"2024-05-26T22:53:52.413767+0000","last_active":"2024-05-26T22:53:54.448474+0000","last_peered":"2024-05-26T22:53:54.448474+0000","last_clean":"2024-05-26T22:53:54.448474+0000","last_became_active":"2024-05-26T22:53:52.412731+0000","last_became_peered":"2024-05-26T22:53:52.412731+0000","last_unstale":"2024-05-26T22:53:54.448474+0000","last_undegraded":"2024-05-26T22:53:54.448474+0000","last_fullsized":"2024-05-26T22:53:54.448474+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_clean_scrub_stamp":"2024-05-26T22:53:51.367149+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T23:58:10.687034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:54.448218+0000","last_change":"2024-05-26T22:53:53.442854+0000","last_active":"2024-05-26T22:53:54.448218+0000","last_peered":"2024-05-26T22:53:54.448218+0000","last_clean":"2024-05-26T22:53:54.448218+0000","last_became_active":"2024-05-26T22:53:53.442323+0000","last_became_peered":"2024-05-26T22:53:53.442323+0000","last_unstale":"2024-05-26T22:53:54.448218+0000","last_undegraded":"2024-05-26T22:53:54.448218+0000","last_fullsized":"2024-05-26T22:53:54.448218+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:33:12.920691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"81'1","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-26T22:53:53.467857+0000","last_change":"2024-05-26T22:53:52.430723+0000","last_active":"2024-05-26T22:53:53.467857+0000","last_peered":"2024-05-26T22:53:53.467857+0000","last_clean":"2024-05-26T22:53:53.467857+0000","last_became_active":"2024-05-26T22:53:52.430375+0000","last_became_peered":"2024-05-26T22:53:52.430375+0000","last_unstale":"2024-05-26T22:53:53.467857+0000","last_undegraded":"2024-05-26T22:53:53.467857+0000","last_fullsized":"2024-05-26T22:53:53.467857+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_clean_scrub_stamp":"2024-05-26T22:53:51.367149+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:34:54.968538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:54.447661+0000","last_change":"2024-05-26T22:53:53.450386+0000","last_active":"2024-05-26T22:53:54.447661+0000","last_peered":"2024-05-26T22:53:54.447661+0000","last_clean":"2024-05-26T22:53:54.447661+0000","last_became_active":"2024-05-26T22:53:53.450081+0000","last_became_peered":"2024-05-26T22:53:53.450081+0000","last_unstale":"2024-05-26T22:53:54.447661+0000","last_undegraded":"2024-05-26T22:53:54.447661+0000","last_fullsized":"2024-05-26T22:53:54.447661+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:41:44.342479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:54.443766+0000","last_change":"2024-05-26T22:53:52.413919+0000","last_active":"2024-05-26T22:53:54.443766+0000","last_peered":"2024-05-26T22:53:54.443766+0000","last_clean":"2024-05-26T22:53:54.443766+0000","last_became_active":"2024-05-26T22:53:52.413637+0000","last_became_peered":"2024-05-26T22:53:52.413637+0000","last_unstale":"2024-05-26T22:53:54.443766+0000","last_undegraded":"2024-05-26T22:53:54.443766+0000","last_fullsized":"2024-05-26T22:53:54.443766+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_clean_scrub_stamp":"2024-05-26T22:53:51.367149+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:43:54.769141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-26T22:53:53.449756+0000","last_change":"2024-05-26T22:53:53.447190+0000","last_active":"2024-05-26T22:53:53.449756+0000","last_peered":"2024-05-26T22:53:53.449756+0000","last_clean":"2024-05-26T22:53:53.449756+0000","last_became_active":"2024-05-26T22:53:53.446905+0000","last_became_peered":"2024-05-26T22:53:53.446905+0000","last_unstale":"2024-05-26T22:53:53.449756+0000","last_undegraded":"2024-05-26T22:53:53.449756+0000","last_fullsized":"2024-05-26T22:53:53.449756+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:37:40.975738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:54.443237+0000","last_change":"2024-05-26T22:53:52.413208+0000","last_active":"2024-05-26T22:53:54.443237+0000","last_peered":"2024-05-26T22:53:54.443237+0000","last_clean":"2024-05-26T22:53:54.443237+0000","last_became_active":"2024-05-26T22:53:52.412854+0000","last_became_peered":"2024-05-26T22:53:52.412854+0000","last_unstale":"2024-05-26T22:53:54.443237+0000","last_undegraded":"2024-05-26T22:53:54.443237+0000","last_fullsized":"2024-05-26T22:53:54.443237+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_clean_scrub_stamp":"2024-05-26T22:53:51.367149+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:37:39.961098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:54.445302+0000","last_change":"2024-05-26T22:53:53.444345+0000","last_active":"2024-05-26T22:53:54.445302+0000","last_peered":"2024-05-26T22:53:54.445302+0000","last_clean":"2024-05-26T22:53:54.445302+0000","last_became_active":"2024-05-26T22:53:53.444042+0000","last_became_peered":"2024-05-26T22:53:53.444042+0000","last_unstale":"2024-05-26T22:53:54.445302+0000","last_undegraded":"2024-05-26T22:53:54.445302+0000","last_fullsized":"2024-05-26T22:53:54.445302+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T23:58:11.701674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"81'1","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-26T22:53:53.468784+0000","last_change":"2024-05-26T22:53:52.429470+0000","last_active":"2024-05-26T22:53:53.468784+0000","last_peered":"2024-05-26T22:53:53.468784+0000","last_clean":"2024-05-26T22:53:53.468784+0000","last_became_active":"2024-05-26T22:53:52.428797+0000","last_became_peered":"2024-05-26T22:53:52.428797+0000","last_unstale":"2024-05-26T22:53:53.468784+0000","last_undegraded":"2024-05-26T22:53:53.468784+0000","last_fullsized":"2024-05-26T22:53:53.468784+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_clean_scrub_stamp":"2024-05-26T22:53:51.367149+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:47:54.194267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-26T22:53:53.449521+0000","last_change":"2024-05-26T22:53:53.442679+0000","last_active":"2024-05-26T22:53:53.449521+0000","last_peered":"2024-05-26T22:53:53.449521+0000","last_clean":"2024-05-26T22:53:53.449521+0000","last_became_active":"2024-05-26T22:53:53.442408+0000","last_became_peered":"2024-05-26T22:53:53.442408+0000","last_unstale":"2024-05-26T22:53:53.449521+0000","last_undegraded":"2024-05-26T22:53:53.449521+0000","last_fullsized":"2024-05-26T22:53:53.449521+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T23:58:11.701674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"81'1","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-26T22:53:53.470113+0000","last_change":"2024-05-26T22:53:52.431126+0000","last_active":"2024-05-26T22:53:53.470113+0000","last_peered":"2024-05-26T22:53:53.470113+0000","last_clean":"2024-05-26T22:53:53.470113+0000","last_became_active":"2024-05-26T22:53:52.430077+0000","last_became_peered":"2024-05-26T22:53:52.430077+0000","last_unstale":"2024-05-26T22:53:53.470113+0000","last_undegraded":"2024-05-26T22:53:53.470113+0000","last_fullsized":"2024-05-26T22:53:53.470113+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_clean_scrub_stamp":"2024-05-26T22:53:51.367149+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:38:22.250786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-26T22:53:53.448561+0000","last_change":"2024-05-26T22:53:53.442835+0000","last_active":"2024-05-26T22:53:53.448561+0000","last_peered":"2024-05-26T22:53:53.448561+0000","last_clean":"2024-05-26T22:53:53.448561+0000","last_became_active":"2024-05-26T22:53:53.442558+0000","last_became_peered":"2024-05-26T22:53:53.442558+0000","last_unstale":"2024-05-26T22:53:53.448561+0000","last_undegraded":"2024-05-26T22:53:53.448561+0000","last_fullsized":"2024-05-26T22:53:53.448561+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:27:50.935370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:54.443145+0000","last_change":"2024-05-26T22:53:52.425707+0000","last_active":"2024-05-26T22:53:54.443145+0000","last_peered":"2024-05-26T22:53:54.443145+0000","last_clean":"2024-05-26T22:53:54.443145+0000","last_became_active":"2024-05-26T22:53:52.425396+0000","last_became_peered":"2024-05-26T22:53:52.425396+0000","last_unstale":"2024-05-26T22:53:54.443145+0000","last_undegraded":"2024-05-26T22:53:54.443145+0000","last_fullsized":"2024-05-26T22:53:54.443145+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_clean_scrub_stamp":"2024-05-26T22:53:51.367149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:34:41.929685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:55.474472+0000","last_change":"2024-05-26T22:53:53.447526+0000","last_active":"2024-05-26T22:53:55.474472+0000","last_peered":"2024-05-26T22:53:55.474472+0000","last_clean":"2024-05-26T22:53:55.474472+0000","last_became_active":"2024-05-26T22:53:53.447218+0000","last_became_peered":"2024-05-26T22:53:53.447218+0000","last_unstale":"2024-05-26T22:53:55.474472+0000","last_undegraded":"2024-05-26T22:53:55.474472+0000","last_fullsized":"2024-05-26T22:53:55.474472+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:14:16.522080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:54.444691+0000","last_change":"2024-05-26T22:53:52.428100+0000","last_active":"2024-05-26T22:53:54.444691+0000","last_peered":"2024-05-26T22:53:54.444691+0000","last_clean":"2024-05-26T22:53:54.444691+0000","last_became_active":"2024-05-26T22:53:52.427781+0000","last_became_peered":"2024-05-26T22:53:52.427781+0000","last_unstale":"2024-05-26T22:53:54.444691+0000","last_undegraded":"2024-05-26T22:53:54.444691+0000","last_fullsized":"2024-05-26T22:53:54.444691+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_clean_scrub_stamp":"2024-05-26T22:53:51.367149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T22:56:41.031306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:55.475085+0000","last_change":"2024-05-26T22:53:53.454472+0000","last_active":"2024-05-26T22:53:55.475085+0000","last_peered":"2024-05-26T22:53:55.475085+0000","last_clean":"2024-05-26T22:53:55.475085+0000","last_became_active":"2024-05-26T22:53:53.452707+0000","last_became_peered":"2024-05-26T22:53:53.452707+0000","last_unstale":"2024-05-26T22:53:55.475085+0000","last_undegraded":"2024-05-26T22:53:55.475085+0000","last_fullsized":"2024-05-26T22:53:55.475085+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:30:32.015136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:54.448747+0000","last_change":"2024-05-26T22:53:52.418784+0000","last_active":"2024-05-26T22:53:54.448747+0000","last_peered":"2024-05-26T22:53:54.448747+0000","last_clean":"2024-05-26T22:53:54.448747+0000","last_became_active":"2024-05-26T22:53:52.417510+0000","last_became_peered":"2024-05-26T22:53:52.417510+0000","last_unstale":"2024-05-26T22:53:54.448747+0000","last_undegraded":"2024-05-26T22:53:54.448747+0000","last_fullsized":"2024-05-26T22:53:54.448747+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_clean_scrub_stamp":"2024-05-26T22:53:51.367149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:13:49.310511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:54.854220+0000","last_change":"2024-05-26T22:53:53.442864+0000","last_active":"2024-05-26T22:53:54.854220+0000","last_peered":"2024-05-26T22:53:54.854220+0000","last_clean":"2024-05-26T22:53:54.854220+0000","last_became_active":"2024-05-26T22:53:53.442578+0000","last_became_peered":"2024-05-26T22:53:53.442578+0000","last_unstale":"2024-05-26T22:53:54.854220+0000","last_undegraded":"2024-05-26T22:53:54.854220+0000","last_fullsized":"2024-05-26T22:53:54.854220+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:04:25.600389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:54.447765+0000","last_change":"2024-05-26T22:53:52.428124+0000","last_active":"2024-05-26T22:53:54.447765+0000","last_peered":"2024-05-26T22:53:54.447765+0000","last_clean":"2024-05-26T22:53:54.447765+0000","last_became_active":"2024-05-26T22:53:52.427833+0000","last_became_peered":"2024-05-26T22:53:52.427833+0000","last_unstale":"2024-05-26T22:53:54.447765+0000","last_undegraded":"2024-05-26T22:53:54.447765+0000","last_fullsized":"2024-05-26T22:53:54.447765+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_clean_scrub_stamp":"2024-05-26T22:53:51.367149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:14:19.742269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:54.442936+0000","last_change":"2024-05-26T22:53:53.448600+0000","last_active":"2024-05-26T22:53:54.442936+0000","last_peered":"2024-05-26T22:53:54.442936+0000","last_clean":"2024-05-26T22:53:54.442936+0000","last_became_active":"2024-05-26T22:53:53.448240+0000","last_became_peered":"2024-05-26T22:53:53.448240+0000","last_unstale":"2024-05-26T22:53:54.442936+0000","last_undegraded":"2024-05-26T22:53:54.442936+0000","last_fullsized":"2024-05-26T22:53:54.442936+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:04:38.162417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"81'3","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:54.446769+0000","last_change":"2024-05-26T22:53:52.416203+0000","last_active":"2024-05-26T22:53:54.446769+0000","last_peered":"2024-05-26T22:53:54.446769+0000","last_clean":"2024-05-26T22:53:54.446769+0000","last_became_active":"2024-05-26T22:53:52.415267+0000","last_became_peered":"2024-05-26T22:53:52.415267+0000","last_unstale":"2024-05-26T22:53:54.446769+0000","last_undegraded":"2024-05-26T22:53:54.446769+0000","last_fullsized":"2024-05-26T22:53:54.446769+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_clean_scrub_stamp":"2024-05-26T22:53:51.367149+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:34:54.968538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-26T22:53:53.453019+0000","last_change":"2024-05-26T22:53:53.448609+0000","last_active":"2024-05-26T22:53:53.453019+0000","last_peered":"2024-05-26T22:53:53.453019+0000","last_clean":"2024-05-26T22:53:53.453019+0000","last_became_active":"2024-05-26T22:53:53.448330+0000","last_became_peered":"2024-05-26T22:53:53.448330+0000","last_unstale":"2024-05-26T22:53:53.453019+0000","last_undegraded":"2024-05-26T22:53:53.453019+0000","last_fullsized":"2024-05-26T22:53:53.453019+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T22:55:59.978360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"81'6","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:54.854185+0000","last_change":"2024-05-26T22:53:52.424751+0000","last_active":"2024-05-26T22:53:54.854185+0000","last_peered":"2024-05-26T22:53:54.854185+0000","last_clean":"2024-05-26T22:53:54.854185+0000","last_became_active":"2024-05-26T22:53:52.424440+0000","last_became_peered":"2024-05-26T22:53:52.424440+0000","last_unstale":"2024-05-26T22:53:54.854185+0000","last_undegraded":"2024-05-26T22:53:54.854185+0000","last_fullsized":"2024-05-26T22:53:54.854185+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_clean_scrub_stamp":"2024-05-26T22:53:51.367149+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:27:56.419199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:54.443014+0000","last_change":"2024-05-26T22:53:53.440620+0000","last_active":"2024-05-26T22:53:54.443014+0000","last_peered":"2024-05-26T22:53:54.443014+0000","last_clean":"2024-05-26T22:53:54.443014+0000","last_became_active":"2024-05-26T22:53:53.440330+0000","last_became_peered":"2024-05-26T22:53:53.440330+0000","last_unstale":"2024-05-26T22:53:54.443014+0000","last_undegraded":"2024-05-26T22:53:54.443014+0000","last_fullsized":"2024-05-26T22:53:54.443014+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:41:44.342479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:54.444984+0000","last_change":"2024-05-26T22:53:52.418658+0000","last_active":"2024-05-26T22:53:54.444984+0000","last_peered":"2024-05-26T22:53:54.444984+0000","last_clean":"2024-05-26T22:53:54.444984+0000","last_became_active":"2024-05-26T22:53:52.418338+0000","last_became_peered":"2024-05-26T22:53:52.418338+0000","last_unstale":"2024-05-26T22:53:54.444984+0000","last_undegraded":"2024-05-26T22:53:54.444984+0000","last_fullsized":"2024-05-26T22:53:54.444984+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_clean_scrub_stamp":"2024-05-26T22:53:51.367149+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:23:56.451193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-26T22:53:53.449832+0000","last_change":"2024-05-26T22:53:53.446466+0000","last_active":"2024-05-26T22:53:53.449832+0000","last_peered":"2024-05-26T22:53:53.449832+0000","last_clean":"2024-05-26T22:53:53.449832+0000","last_became_active":"2024-05-26T22:53:53.445639+0000","last_became_peered":"2024-05-26T22:53:53.445639+0000","last_unstale":"2024-05-26T22:53:53.449832+0000","last_undegraded":"2024-05-26T22:53:53.449832+0000","last_fullsized":"2024-05-26T22:53:53.449832+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:34:26.592973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:55.474805+0000","last_change":"2024-05-26T22:53:52.414283+0000","last_active":"2024-05-26T22:53:55.474805+0000","last_peered":"2024-05-26T22:53:55.474805+0000","last_clean":"2024-05-26T22:53:55.474805+0000","last_became_active":"2024-05-26T22:53:52.413937+0000","last_became_peered":"2024-05-26T22:53:52.413937+0000","last_unstale":"2024-05-26T22:53:55.474805+0000","last_undegraded":"2024-05-26T22:53:55.474805+0000","last_fullsized":"2024-05-26T22:53:55.474805+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_clean_scrub_stamp":"2024-05-26T22:53:51.367149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:03:18.615767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:54.444724+0000","last_change":"2024-05-26T22:53:53.447857+0000","last_active":"2024-05-26T22:53:54.444724+0000","last_peered":"2024-05-26T22:53:54.444724+0000","last_clean":"2024-05-26T22:53:54.444724+0000","last_became_active":"2024-05-26T22:53:53.443401+0000","last_became_peered":"2024-05-26T22:53:53.443401+0000","last_unstale":"2024-05-26T22:53:54.444724+0000","last_undegraded":"2024-05-26T22:53:54.444724+0000","last_fullsized":"2024-05-26T22:53:54.444724+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:47:55.208907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:54.853728+0000","last_change":"2024-05-26T22:53:52.428819+0000","last_active":"2024-05-26T22:53:54.853728+0000","last_peered":"2024-05-26T22:53:54.853728+0000","last_clean":"2024-05-26T22:53:54.853728+0000","last_became_active":"2024-05-26T22:53:52.428516+0000","last_became_peered":"2024-05-26T22:53:52.428516+0000","last_unstale":"2024-05-26T22:53:54.853728+0000","last_undegraded":"2024-05-26T22:53:54.853728+0000","last_fullsized":"2024-05-26T22:53:54.853728+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_clean_scrub_stamp":"2024-05-26T22:53:51.367149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T23:49:10.519636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:54.444679+0000","last_change":"2024-05-26T22:53:53.448228+0000","last_active":"2024-05-26T22:53:54.444679+0000","last_peered":"2024-05-26T22:53:54.444679+0000","last_clean":"2024-05-26T22:53:54.444679+0000","last_became_active":"2024-05-26T22:53:53.447919+0000","last_became_peered":"2024-05-26T22:53:53.447919+0000","last_unstale":"2024-05-26T22:53:54.444679+0000","last_undegraded":"2024-05-26T22:53:54.444679+0000","last_fullsized":"2024-05-26T22:53:54.444679+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:38:23.265425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"81'8","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:54.442730+0000","last_change":"2024-05-26T22:53:52.422609+0000","last_active":"2024-05-26T22:53:54.442730+0000","last_peered":"2024-05-26T22:53:54.442730+0000","last_clean":"2024-05-26T22:53:54.442730+0000","last_became_active":"2024-05-26T22:53:52.422109+0000","last_became_peered":"2024-05-26T22:53:52.422109+0000","last_unstale":"2024-05-26T22:53:54.442730+0000","last_undegraded":"2024-05-26T22:53:54.442730+0000","last_fullsized":"2024-05-26T22:53:54.442730+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_clean_scrub_stamp":"2024-05-26T22:53:51.367149+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:34:25.578333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-26T22:53:53.448574+0000","last_change":"2024-05-26T22:53:53.444274+0000","last_active":"2024-05-26T22:53:53.448574+0000","last_peered":"2024-05-26T22:53:53.448574+0000","last_clean":"2024-05-26T22:53:53.448574+0000","last_became_active":"2024-05-26T22:53:53.443963+0000","last_became_peered":"2024-05-26T22:53:53.443963+0000","last_unstale":"2024-05-26T22:53:53.448574+0000","last_undegraded":"2024-05-26T22:53:53.448574+0000","last_fullsized":"2024-05-26T22:53:53.448574+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:34:42.944324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"81'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:54.449001+0000","last_change":"2024-05-26T22:53:52.419826+0000","last_active":"2024-05-26T22:53:54.449001+0000","last_peered":"2024-05-26T22:53:54.449001+0000","last_clean":"2024-05-26T22:53:54.449001+0000","last_became_active":"2024-05-26T22:53:52.419380+0000","last_became_peered":"2024-05-26T22:53:52.419380+0000","last_unstale":"2024-05-26T22:53:54.449001+0000","last_undegraded":"2024-05-26T22:53:54.449001+0000","last_fullsized":"2024-05-26T22:53:54.449001+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_clean_scrub_stamp":"2024-05-26T22:53:51.367149+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:07:14.211370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:54.448478+0000","last_change":"2024-05-26T22:53:53.444381+0000","last_active":"2024-05-26T22:53:54.448478+0000","last_peered":"2024-05-26T22:53:54.448478+0000","last_clean":"2024-05-26T22:53:54.448478+0000","last_became_active":"2024-05-26T22:53:53.444065+0000","last_became_peered":"2024-05-26T22:53:53.444065+0000","last_unstale":"2024-05-26T22:53:54.448478+0000","last_undegraded":"2024-05-26T22:53:54.448478+0000","last_fullsized":"2024-05-26T22:53:54.448478+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:47:50.514023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:54.443656+0000","last_change":"2024-05-26T22:53:52.428127+0000","last_active":"2024-05-26T22:53:54.443656+0000","last_peered":"2024-05-26T22:53:54.443656+0000","last_clean":"2024-05-26T22:53:54.443656+0000","last_became_active":"2024-05-26T22:53:52.427775+0000","last_became_peered":"2024-05-26T22:53:52.427775+0000","last_unstale":"2024-05-26T22:53:54.443656+0000","last_undegraded":"2024-05-26T22:53:54.443656+0000","last_fullsized":"2024-05-26T22:53:54.443656+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_clean_scrub_stamp":"2024-05-26T22:53:51.367149+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:19:15.271878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-26T22:53:53.454915+0000","last_change":"2024-05-26T22:53:53.452374+0000","last_active":"2024-05-26T22:53:53.454915+0000","last_peered":"2024-05-26T22:53:53.454915+0000","last_clean":"2024-05-26T22:53:53.454915+0000","last_became_active":"2024-05-26T22:53:53.451294+0000","last_became_peered":"2024-05-26T22:53:53.451294+0000","last_unstale":"2024-05-26T22:53:53.454915+0000","last_undegraded":"2024-05-26T22:53:53.454915+0000","last_fullsized":"2024-05-26T22:53:53.454915+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:34:42.944324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:54.443403+0000","last_change":"2024-05-26T22:53:52.426053+0000","last_active":"2024-05-26T22:53:54.443403+0000","last_peered":"2024-05-26T22:53:54.443403+0000","last_clean":"2024-05-26T22:53:54.443403+0000","last_became_active":"2024-05-26T22:53:52.425383+0000","last_became_peered":"2024-05-26T22:53:52.425383+0000","last_unstale":"2024-05-26T22:53:54.443403+0000","last_undegraded":"2024-05-26T22:53:54.443403+0000","last_fullsized":"2024-05-26T22:53:54.443403+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_clean_scrub_stamp":"2024-05-26T22:53:51.367149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T23:49:10.187833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-26T22:53:53.457050+0000","last_change":"2024-05-26T22:53:53.452346+0000","last_active":"2024-05-26T22:53:53.457050+0000","last_peered":"2024-05-26T22:53:53.457050+0000","last_clean":"2024-05-26T22:53:53.457050+0000","last_became_active":"2024-05-26T22:53:53.452050+0000","last_became_peered":"2024-05-26T22:53:53.452050+0000","last_unstale":"2024-05-26T22:53:53.457050+0000","last_undegraded":"2024-05-26T22:53:53.457050+0000","last_fullsized":"2024-05-26T22:53:53.457050+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:02:22.099937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:55.474873+0000","last_change":"2024-05-26T22:53:52.414510+0000","last_active":"2024-05-26T22:53:55.474873+0000","last_peered":"2024-05-26T22:53:55.474873+0000","last_clean":"2024-05-26T22:53:55.474873+0000","last_became_active":"2024-05-26T22:53:52.414233+0000","last_became_peered":"2024-05-26T22:53:52.414233+0000","last_unstale":"2024-05-26T22:53:55.474873+0000","last_undegraded":"2024-05-26T22:53:55.474873+0000","last_fullsized":"2024-05-26T22:53:55.474873+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_clean_scrub_stamp":"2024-05-26T22:53:51.367149+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:02:01.951417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:54.853724+0000","last_change":"2024-05-26T22:53:53.442327+0000","last_active":"2024-05-26T22:53:54.853724+0000","last_peered":"2024-05-26T22:53:54.853724+0000","last_clean":"2024-05-26T22:53:54.853724+0000","last_became_active":"2024-05-26T22:53:53.441858+0000","last_became_peered":"2024-05-26T22:53:53.441858+0000","last_unstale":"2024-05-26T22:53:54.853724+0000","last_undegraded":"2024-05-26T22:53:54.853724+0000","last_fullsized":"2024-05-26T22:53:54.853724+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:37:40.975738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-26T22:53:53.418176+0000","last_change":"2024-05-26T22:53:52.419819+0000","last_active":"2024-05-26T22:53:53.418176+0000","last_peered":"2024-05-26T22:53:53.418176+0000","last_clean":"2024-05-26T22:53:53.418176+0000","last_became_active":"2024-05-26T22:53:52.419518+0000","last_became_peered":"2024-05-26T22:53:52.419518+0000","last_unstale":"2024-05-26T22:53:53.418176+0000","last_undegraded":"2024-05-26T22:53:53.418176+0000","last_fullsized":"2024-05-26T22:53:53.418176+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_clean_scrub_stamp":"2024-05-26T22:53:51.367149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:21:14.360322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-26T22:53:53.442612+0000","last_change":"2024-05-26T22:53:53.434377+0000","last_active":"2024-05-26T22:53:53.442612+0000","last_peered":"2024-05-26T22:53:53.442612+0000","last_clean":"2024-05-26T22:53:53.442612+0000","last_became_active":"2024-05-26T22:53:53.434059+0000","last_became_peered":"2024-05-26T22:53:53.434059+0000","last_unstale":"2024-05-26T22:53:53.442612+0000","last_undegraded":"2024-05-26T22:53:53.442612+0000","last_fullsized":"2024-05-26T22:53:53.442612+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T23:56:58.411305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-26T22:53:53.416440+0000","last_change":"2024-05-26T22:53:52.414415+0000","last_active":"2024-05-26T22:53:53.416440+0000","last_peered":"2024-05-26T22:53:53.416440+0000","last_clean":"2024-05-26T22:53:53.416440+0000","last_became_active":"2024-05-26T22:53:52.414141+0000","last_became_peered":"2024-05-26T22:53:52.414141+0000","last_unstale":"2024-05-26T22:53:53.416440+0000","last_undegraded":"2024-05-26T22:53:53.416440+0000","last_fullsized":"2024-05-26T22:53:53.416440+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_clean_scrub_stamp":"2024-05-26T22:53:51.367149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:08:17.591682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:54.853812+0000","last_change":"2024-05-26T22:53:53.437480+0000","last_active":"2024-05-26T22:53:54.853812+0000","last_peered":"2024-05-26T22:53:54.853812+0000","last_clean":"2024-05-26T22:53:54.853812+0000","last_became_active":"2024-05-26T22:53:53.437125+0000","last_became_peered":"2024-05-26T22:53:53.437125+0000","last_unstale":"2024-05-26T22:53:54.853812+0000","last_undegraded":"2024-05-26T22:53:54.853812+0000","last_fullsized":"2024-05-26T22:53:54.853812+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T23:49:11.202473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:55.474227+0000","last_change":"2024-05-26T22:53:52.407508+0000","last_active":"2024-05-26T22:53:55.474227+0000","last_peered":"2024-05-26T22:53:55.474227+0000","last_clean":"2024-05-26T22:53:55.474227+0000","last_became_active":"2024-05-26T22:53:52.407122+0000","last_became_peered":"2024-05-26T22:53:52.407122+0000","last_unstale":"2024-05-26T22:53:55.474227+0000","last_undegraded":"2024-05-26T22:53:55.474227+0000","last_fullsized":"2024-05-26T22:53:55.474227+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_clean_scrub_stamp":"2024-05-26T22:53:51.367149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:01:49.436136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-26T22:53:53.450467+0000","last_change":"2024-05-26T22:53:53.444551+0000","last_active":"2024-05-26T22:53:53.450467+0000","last_peered":"2024-05-26T22:53:53.450467+0000","last_clean":"2024-05-26T22:53:53.450467+0000","last_became_active":"2024-05-26T22:53:53.444103+0000","last_became_peered":"2024-05-26T22:53:53.444103+0000","last_unstale":"2024-05-26T22:53:53.450467+0000","last_undegraded":"2024-05-26T22:53:53.450467+0000","last_fullsized":"2024-05-26T22:53:53.450467+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:07:15.226009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:54.445476+0000","last_change":"2024-05-26T22:53:52.416802+0000","last_active":"2024-05-26T22:53:54.445476+0000","last_peered":"2024-05-26T22:53:54.445476+0000","last_clean":"2024-05-26T22:53:54.445476+0000","last_became_active":"2024-05-26T22:53:52.416484+0000","last_became_peered":"2024-05-26T22:53:52.416484+0000","last_unstale":"2024-05-26T22:53:54.445476+0000","last_undegraded":"2024-05-26T22:53:54.445476+0000","last_fullsized":"2024-05-26T22:53:54.445476+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_clean_scrub_stamp":"2024-05-26T22:53:51.367149+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:03:20.557956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-26T22:53:53.453387+0000","last_change":"2024-05-26T22:53:53.451530+0000","last_active":"2024-05-26T22:53:53.453387+0000","last_peered":"2024-05-26T22:53:53.453387+0000","last_clean":"2024-05-26T22:53:53.453387+0000","last_became_active":"2024-05-26T22:53:53.449803+0000","last_became_peered":"2024-05-26T22:53:53.449803+0000","last_unstale":"2024-05-26T22:53:53.453387+0000","last_undegraded":"2024-05-26T22:53:53.453387+0000","last_fullsized":"2024-05-26T22:53:53.453387+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:43:55.783780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"82'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:54.464243+0000","last_change":"2024-05-26T22:53:52.418176+0000","last_active":"2024-05-26T22:53:54.464243+0000","last_peered":"2024-05-26T22:53:54.464243+0000","last_clean":"2024-05-26T22:53:54.464243+0000","last_became_active":"2024-05-26T22:53:52.417899+0000","last_became_peered":"2024-05-26T22:53:52.417899+0000","last_unstale":"2024-05-26T22:53:54.464243+0000","last_undegraded":"2024-05-26T22:53:54.464243+0000","last_fullsized":"2024-05-26T22:53:54.464243+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_clean_scrub_stamp":"2024-05-26T22:53:51.367149+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:08:19.625298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:54.447951+0000","last_change":"2024-05-26T22:53:53.441197+0000","last_active":"2024-05-26T22:53:54.447951+0000","last_peered":"2024-05-26T22:53:54.447951+0000","last_clean":"2024-05-26T22:53:54.447951+0000","last_became_active":"2024-05-26T22:53:53.436681+0000","last_became_peered":"2024-05-26T22:53:53.436681+0000","last_unstale":"2024-05-26T22:53:54.447951+0000","last_undegraded":"2024-05-26T22:53:54.447951+0000","last_fullsized":"2024-05-26T22:53:54.447951+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:41:10.035551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:54.447278+0000","last_change":"2024-05-26T22:53:52.413388+0000","last_active":"2024-05-26T22:53:54.447278+0000","last_peered":"2024-05-26T22:53:54.447278+0000","last_clean":"2024-05-26T22:53:54.447278+0000","last_became_active":"2024-05-26T22:53:52.413043+0000","last_became_peered":"2024-05-26T22:53:52.413043+0000","last_unstale":"2024-05-26T22:53:54.447278+0000","last_undegraded":"2024-05-26T22:53:54.447278+0000","last_fullsized":"2024-05-26T22:53:54.447278+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_clean_scrub_stamp":"2024-05-26T22:53:51.367149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:27:56.419199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-26T22:53:53.452542+0000","last_change":"2024-05-26T22:53:53.446557+0000","last_active":"2024-05-26T22:53:53.452542+0000","last_peered":"2024-05-26T22:53:53.452542+0000","last_clean":"2024-05-26T22:53:53.452542+0000","last_became_active":"2024-05-26T22:53:53.446276+0000","last_became_peered":"2024-05-26T22:53:53.446276+0000","last_unstale":"2024-05-26T22:53:53.452542+0000","last_undegraded":"2024-05-26T22:53:53.452542+0000","last_fullsized":"2024-05-26T22:53:53.452542+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:33:22.065967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:54.448065+0000","last_change":"2024-05-26T22:53:52.422005+0000","last_active":"2024-05-26T22:53:54.448065+0000","last_peered":"2024-05-26T22:53:54.448065+0000","last_clean":"2024-05-26T22:53:54.448065+0000","last_became_active":"2024-05-26T22:53:52.421720+0000","last_became_peered":"2024-05-26T22:53:52.421720+0000","last_unstale":"2024-05-26T22:53:54.448065+0000","last_undegraded":"2024-05-26T22:53:54.448065+0000","last_fullsized":"2024-05-26T22:53:54.448065+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_clean_scrub_stamp":"2024-05-26T22:53:51.367149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:33:21.051328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:54.853926+0000","last_change":"2024-05-26T22:53:53.441004+0000","last_active":"2024-05-26T22:53:54.853926+0000","last_peered":"2024-05-26T22:53:54.853926+0000","last_clean":"2024-05-26T22:53:54.853926+0000","last_became_active":"2024-05-26T22:53:53.440636+0000","last_became_peered":"2024-05-26T22:53:53.440636+0000","last_unstale":"2024-05-26T22:53:54.853926+0000","last_undegraded":"2024-05-26T22:53:54.853926+0000","last_fullsized":"2024-05-26T22:53:54.853926+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:40:30.639915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:54.443322+0000","last_change":"2024-05-26T22:53:52.422617+0000","last_active":"2024-05-26T22:53:54.443322+0000","last_peered":"2024-05-26T22:53:54.443322+0000","last_clean":"2024-05-26T22:53:54.443322+0000","last_became_active":"2024-05-26T22:53:52.422127+0000","last_became_peered":"2024-05-26T22:53:52.422127+0000","last_unstale":"2024-05-26T22:53:54.443322+0000","last_undegraded":"2024-05-26T22:53:54.443322+0000","last_fullsized":"2024-05-26T22:53:54.443322+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_clean_scrub_stamp":"2024-05-26T22:53:51.367149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:40:29.625276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:54.443529+0000","last_change":"2024-05-26T22:53:53.450328+0000","last_active":"2024-05-26T22:53:54.443529+0000","last_peered":"2024-05-26T22:53:54.443529+0000","last_clean":"2024-05-26T22:53:54.443529+0000","last_became_active":"2024-05-26T22:53:53.447937+0000","last_became_peered":"2024-05-26T22:53:53.447937+0000","last_unstale":"2024-05-26T22:53:54.443529+0000","last_undegraded":"2024-05-26T22:53:54.443529+0000","last_fullsized":"2024-05-26T22:53:54.443529+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:01:50.450776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-26T22:53:53.433248+0000","last_change":"2024-05-26T22:53:52.429754+0000","last_active":"2024-05-26T22:53:53.433248+0000","last_peered":"2024-05-26T22:53:53.433248+0000","last_clean":"2024-05-26T22:53:53.433248+0000","last_became_active":"2024-05-26T22:53:52.429437+0000","last_became_peered":"2024-05-26T22:53:52.429437+0000","last_unstale":"2024-05-26T22:53:53.433248+0000","last_undegraded":"2024-05-26T22:53:53.433248+0000","last_fullsized":"2024-05-26T22:53:53.433248+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_clean_scrub_stamp":"2024-05-26T22:53:51.367149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:27:56.419199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-26T22:53:53.455074+0000","last_change":"2024-05-26T22:53:53.452580+0000","last_active":"2024-05-26T22:53:53.455074+0000","last_peered":"2024-05-26T22:53:53.455074+0000","last_clean":"2024-05-26T22:53:53.455074+0000","last_became_active":"2024-05-26T22:53:53.452004+0000","last_became_peered":"2024-05-26T22:53:53.452004+0000","last_unstale":"2024-05-26T22:53:53.455074+0000","last_undegraded":"2024-05-26T22:53:53.455074+0000","last_fullsized":"2024-05-26T22:53:53.455074+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:19:16.286517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:54.447127+0000","last_change":"2024-05-26T22:53:52.421592+0000","last_active":"2024-05-26T22:53:54.447127+0000","last_peered":"2024-05-26T22:53:54.447127+0000","last_clean":"2024-05-26T22:53:54.447127+0000","last_became_active":"2024-05-26T22:53:52.421270+0000","last_became_peered":"2024-05-26T22:53:52.421270+0000","last_unstale":"2024-05-26T22:53:54.447127+0000","last_undegraded":"2024-05-26T22:53:54.447127+0000","last_fullsized":"2024-05-26T22:53:54.447127+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_clean_scrub_stamp":"2024-05-26T22:53:51.367149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T23:49:10.519636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:54.446859+0000","last_change":"2024-05-26T22:53:53.445240+0000","last_active":"2024-05-26T22:53:54.446859+0000","last_peered":"2024-05-26T22:53:54.446859+0000","last_clean":"2024-05-26T22:53:54.446859+0000","last_became_active":"2024-05-26T22:53:53.444926+0000","last_became_peered":"2024-05-26T22:53:53.444926+0000","last_unstale":"2024-05-26T22:53:54.446859+0000","last_undegraded":"2024-05-26T22:53:54.446859+0000","last_fullsized":"2024-05-26T22:53:54.446859+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:31:00.584149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-26T22:53:53.432229+0000","last_change":"2024-05-26T22:53:52.426301+0000","last_active":"2024-05-26T22:53:53.432229+0000","last_peered":"2024-05-26T22:53:53.432229+0000","last_clean":"2024-05-26T22:53:53.432229+0000","last_became_active":"2024-05-26T22:53:52.425851+0000","last_became_peered":"2024-05-26T22:53:52.425851+0000","last_unstale":"2024-05-26T22:53:53.432229+0000","last_undegraded":"2024-05-26T22:53:53.432229+0000","last_fullsized":"2024-05-26T22:53:53.432229+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_clean_scrub_stamp":"2024-05-26T22:53:51.367149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T23:25:01.321783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:54.445306+0000","last_change":"2024-05-26T22:53:53.446358+0000","last_active":"2024-05-26T22:53:54.445306+0000","last_peered":"2024-05-26T22:53:54.445306+0000","last_clean":"2024-05-26T22:53:54.445306+0000","last_became_active":"2024-05-26T22:53:53.443913+0000","last_became_peered":"2024-05-26T22:53:53.443913+0000","last_unstale":"2024-05-26T22:53:54.445306+0000","last_undegraded":"2024-05-26T22:53:54.445306+0000","last_fullsized":"2024-05-26T22:53:54.445306+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:13:50.325150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"81'1","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-26T22:53:53.470919+0000","last_change":"2024-05-26T22:53:52.426906+0000","last_active":"2024-05-26T22:53:53.470919+0000","last_peered":"2024-05-26T22:53:53.470919+0000","last_clean":"2024-05-26T22:53:53.470919+0000","last_became_active":"2024-05-26T22:53:52.425801+0000","last_became_peered":"2024-05-26T22:53:52.425801+0000","last_unstale":"2024-05-26T22:53:53.470919+0000","last_undegraded":"2024-05-26T22:53:53.470919+0000","last_fullsized":"2024-05-26T22:53:53.470919+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_clean_scrub_stamp":"2024-05-26T22:53:51.367149+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T22:55:58.963721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":284,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-26T22:53:53.431349+0000","last_change":"2024-05-26T22:48:12.068917+0000","last_active":"2024-05-26T22:53:53.431349+0000","last_peered":"2024-05-26T22:53:53.431349+0000","last_clean":"2024-05-26T22:53:53.431349+0000","last_became_active":"2024-05-26T22:48:12.065934+0000","last_became_peered":"2024-05-26T22:48:12.065934+0000","last_unstale":"2024-05-26T22:53:53.431349+0000","last_undegraded":"2024-05-26T22:53:53.431349+0000","last_fullsized":"2024-05-26T22:53:53.431349+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:10.628122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:10.628122+0000","last_clean_scrub_stamp":"2024-05-26T22:48:10.628122+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:14:58.895911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574920,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32736,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339104,"statfs":{"total":96636764160,"available":96603242496,"internally_reserved":0,"allocated":1593344,"data_stored":1018513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574920,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32180,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339660,"statfs":{"total":96636764160,"available":96603811840,"internally_reserved":0,"allocated":1019904,"data_stored":435069,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574921,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32180,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339660,"statfs":{"total":96636764160,"available":96603811840,"internally_reserved":0,"allocated":1024000,"data_stored":436009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574920,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32152,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339688,"statfs":{"total":96636764160,"available":96603840512,"internally_reserved":0,"allocated":995328,"data_stored":428075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574920,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32168,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339672,"statfs":{"total":96636764160,"available":96603824128,"internally_reserved":0,"allocated":1007616,"data_stored":429687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574920,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32156,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339684,"statfs":{"total":96636764160,"available":96603836416,"internally_reserved":0,"allocated":999424,"data_stored":428657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574920,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32180,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339660,"statfs":{"total":96636764160,"available":96603811840,"internally_reserved":0,"allocated":1019904,"data_stored":434569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574920,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32756,"kb_used_data":1568,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339084,"statfs":{"total":96636764160,"available":96603222016,"internally_reserved":0,"allocated":1605632,"data_stored":1024777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-26T22:53:57.857 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-26T22:53:58.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.096+0000 7f71fce00640 1 Processor -- start 2024-05-26T22:53:58.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.096+0000 7f71fce00640 1 -- start start 2024-05-26T22:53:58.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.098+0000 7f71fce00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f71f8144340 0x7f71f80cedf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:58.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.098+0000 7f71fce00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f71f8071b70 0x7f71f806e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:58.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.098+0000 7f71fce00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71f8072000 con 0x7f71f806e9f0 2024-05-26T22:53:58.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.098+0000 7f71fce00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f71f8072eb0 con 0x7f71f8071b70 2024-05-26T22:53:58.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.098+0000 7f71fce00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f71f806ee20 con 0x7f71f8144340 2024-05-26T22:53:58.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.098+0000 7f71f6a00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f71f8071b70 0x7f71f806e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:58.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.098+0000 7f71f6a00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f71f8071b70 0x7f71f806e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56766/0 (socket says 172.21.15.131:56766) 2024-05-26T22:53:58.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.098+0000 7f71f6a00640 1 -- 172.21.15.131:0/1989607094 learned_addr learned my addr 172.21.15.131:0/1989607094 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:53:58.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.098+0000 7f71f7400640 1 --2- 172.21.15.131:0/1989607094 >> v2:172.21.15.187:3300/0 conn(0x7f71f8144340 0x7f71f80cedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:58.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.099+0000 7f71f6a00640 1 -- 172.21.15.131:0/1989607094 >> v2:172.21.15.187:3300/0 conn(0x7f71f8144340 msgr2=0x7f71f80cedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:58.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.099+0000 7f71f6a00640 1 --2- 172.21.15.131:0/1989607094 >> v2:172.21.15.187:3300/0 conn(0x7f71f8144340 0x7f71f80cedf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:58.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.099+0000 7f71f6a00640 1 -- 172.21.15.131:0/1989607094 >> v1:172.21.15.187:6789/0 conn(0x7f71f806e9f0 legacy=0x7f71f815ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:58.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.099+0000 7f71f6a00640 1 -- 172.21.15.131:0/1989607094 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71f80722a0 con 0x7f71f8071b70 2024-05-26T22:53:58.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.100+0000 7f71f6a00640 1 --2- 172.21.15.131:0/1989607094 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71f8071b70 0x7f71f806e4b0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f71f80b2bd0 tx=0x7f71e402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d795ec4d89456487 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:58.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.100+0000 7f71f6000640 1 -- 172.21.15.131:0/1989607094 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e4037070 con 0x7f71f8071b70 2024-05-26T22:53:58.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.100+0000 7f71f6000640 1 -- 172.21.15.131:0/1989607094 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71e4032440 con 0x7f71f8071b70 2024-05-26T22:53:58.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.100+0000 7f71f6000640 1 -- 172.21.15.131:0/1989607094 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e4032810 con 0x7f71f8071b70 2024-05-26T22:53:58.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.100+0000 7f71fce00640 1 -- 172.21.15.131:0/1989607094 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71f8071b70 msgr2=0x7f71f806e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:58.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.100+0000 7f71fce00640 1 --2- 172.21.15.131:0/1989607094 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71f8071b70 0x7f71f806e4b0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f71f80b2bd0 tx=0x7f71e402a230 comp rx=0 tx=0).stop 2024-05-26T22:53:58.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.101+0000 7f71fce00640 1 -- 172.21.15.131:0/1989607094 shutdown_connections 2024-05-26T22:53:58.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.101+0000 7f71fce00640 1 --2- 172.21.15.131:0/1989607094 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71f8071b70 0x7f71f806e4b0 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:58.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.101+0000 7f71fce00640 1 --2- 172.21.15.131:0/1989607094 >> v2:172.21.15.187:3300/0 conn(0x7f71f8144340 0x7f71f80cedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:58.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.101+0000 7f71fce00640 1 -- 172.21.15.131:0/1989607094 >> 172.21.15.131:0/1989607094 conn(0x7f71f80cff40 msgr2=0x7f71f80d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:58.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.101+0000 7f71fce00640 1 -- 172.21.15.131:0/1989607094 shutdown_connections 2024-05-26T22:53:58.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.101+0000 7f71fce00640 1 -- 172.21.15.131:0/1989607094 wait complete. 2024-05-26T22:53:58.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.102+0000 7f71fce00640 1 Processor -- start 2024-05-26T22:53:58.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.102+0000 7f71fce00640 1 -- start start 2024-05-26T22:53:58.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.103+0000 7f71fce00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f71f806e9f0 0x7f71f813e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:58.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.103+0000 7f71fce00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f71f8071b70 0x7f71f81f4190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:58.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.103+0000 7f71fce00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71f8144340 0x7f71f81416e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:58.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.103+0000 7f71fce00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f71f8072b40 con 0x7f71f8144340 2024-05-26T22:53:58.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.103+0000 7f71fce00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f71f8072860 con 0x7f71f8071b70 2024-05-26T22:53:58.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.103+0000 7f71fce00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f71f80b4d30 con 0x7f71f806e9f0 2024-05-26T22:53:58.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.104+0000 7f71f7e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71f8144340 0x7f71f81416e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:58.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.104+0000 7f71f7e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71f8144340 0x7f71f81416e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56774/0 (socket says 172.21.15.131:56774) 2024-05-26T22:53:58.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.104+0000 7f71f7e00640 1 -- 172.21.15.131:0/2019707536 learned_addr learned my addr 172.21.15.131:0/2019707536 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:53:58.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.104+0000 7f71f7400640 1 --2- 172.21.15.131:0/2019707536 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f71f806e9f0 0x7f71f813e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:58.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.104+0000 7f71f7e00640 1 -- 172.21.15.131:0/2019707536 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f71f806e9f0 msgr2=0x7f71f813e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:58.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.104+0000 7f71f6a00640 1 --2- 172.21.15.131:0/2019707536 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f71f8071b70 0x7f71f81f4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:58.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.104+0000 7f71f7e00640 1 --2- 172.21.15.131:0/2019707536 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f71f806e9f0 0x7f71f813e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:58.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.104+0000 7f71f7e00640 1 -- 172.21.15.131:0/2019707536 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f71f8071b70 msgr2=0x7f71f81f4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:58.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.104+0000 7f71f7e00640 1 --2- 172.21.15.131:0/2019707536 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f71f8071b70 0x7f71f81f4190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:58.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.104+0000 7f71f7e00640 1 -- 172.21.15.131:0/2019707536 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71f80ade30 con 0x7f71f8144340 2024-05-26T22:53:58.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.104+0000 7f71f7e00640 1 --2- 172.21.15.131:0/2019707536 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71f8144340 0x7f71f81416e0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f71f000c390 tx=0x7f71f000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:58.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.105+0000 7f71ef400640 1 -- 172.21.15.131:0/2019707536 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f0017070 con 0x7f71f8144340 2024-05-26T22:53:58.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.105+0000 7f71ef400640 1 -- 172.21.15.131:0/2019707536 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71f00035f0 con 0x7f71f8144340 2024-05-26T22:53:58.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.105+0000 7f71ef400640 1 -- 172.21.15.131:0/2019707536 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f00039e0 con 0x7f71f8144340 2024-05-26T22:53:58.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.105+0000 7f71fce00640 1 -- 172.21.15.131:0/2019707536 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71f80ae110 con 0x7f71f8144340 2024-05-26T22:53:58.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.105+0000 7f71fce00640 1 -- 172.21.15.131:0/2019707536 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71f80ae6d0 con 0x7f71f8144340 2024-05-26T22:53:58.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.106+0000 7f71fce00640 1 -- 172.21.15.131:0/2019707536 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71f80694e0 con 0x7f71f8144340 2024-05-26T22:53:58.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.107+0000 7f71ef400640 1 -- 172.21.15.131:0/2019707536 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f71f0003b80 con 0x7f71f8144340 2024-05-26T22:53:58.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.108+0000 7f71ef400640 1 --2- 172.21.15.131:0/2019707536 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f71d806be30 0x7f71d806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:58.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.108+0000 7f71ef400640 1 -- 172.21.15.131:0/2019707536 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5709+0+0 (secure 0 0 0) 0x7f71f008a260 con 0x7f71f8144340 2024-05-26T22:53:58.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.108+0000 7f71f7400640 1 --2- 172.21.15.131:0/2019707536 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f71d806be30 0x7f71d806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:58.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.109+0000 7f71f7400640 1 --2- 172.21.15.131:0/2019707536 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f71d806be30 0x7f71d806e2f0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f71e8008e50 tx=0x7f71e80083a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:58.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.113+0000 7f71ef400640 1 -- 172.21.15.131:0/2019707536 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f71f0058b70 con 0x7f71f8144340 2024-05-26T22:53:58.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.313+0000 7f71fce00640 1 -- 172.21.15.131:0/2019707536 --> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f71f806e5a0 con 0x7f71d806be30 2024-05-26T22:53:58.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.323+0000 7f71ef400640 1 -- 172.21.15.131:0/2019707536 <== mgr.4104 v2:172.21.15.131:6832/3484832947 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252047 (secure 0 0 0) 0x7f71f806e5a0 con 0x7f71d806be30 2024-05-26T22:53:58.326 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:53:58.327 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-05-26T22:53:58.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.328+0000 7f71fce00640 1 -- 172.21.15.131:0/2019707536 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f71d806be30 msgr2=0x7f71d806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:58.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.328+0000 7f71fce00640 1 --2- 172.21.15.131:0/2019707536 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f71d806be30 0x7f71d806e2f0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f71e8008e50 tx=0x7f71e80083a0 comp rx=0 tx=0).stop 2024-05-26T22:53:58.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.328+0000 7f71fce00640 1 -- 172.21.15.131:0/2019707536 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71f8144340 msgr2=0x7f71f81416e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:58.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.328+0000 7f71fce00640 1 --2- 172.21.15.131:0/2019707536 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71f8144340 0x7f71f81416e0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f71f000c390 tx=0x7f71f000c850 comp rx=0 tx=0).stop 2024-05-26T22:53:58.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.328+0000 7f71fce00640 1 -- 172.21.15.131:0/2019707536 shutdown_connections 2024-05-26T22:53:58.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.328+0000 7f71fce00640 1 --2- 172.21.15.131:0/2019707536 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f71d806be30 0x7f71d806e2f0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:58.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.328+0000 7f71fce00640 1 --2- 172.21.15.131:0/2019707536 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71f8144340 0x7f71f81416e0 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:58.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.328+0000 7f71fce00640 1 --2- 172.21.15.131:0/2019707536 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f71f8071b70 0x7f71f81f4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:58.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.328+0000 7f71fce00640 1 --2- 172.21.15.131:0/2019707536 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f71f806e9f0 0x7f71f813e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:58.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.328+0000 7f71fce00640 1 -- 172.21.15.131:0/2019707536 >> 172.21.15.131:0/2019707536 conn(0x7f71f80cff40 msgr2=0x7f71f80b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:58.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.328+0000 7f71fce00640 1 -- 172.21.15.131:0/2019707536 shutdown_connections 2024-05-26T22:53:58.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.328+0000 7f71fce00640 1 -- 172.21.15.131:0/2019707536 wait complete. 2024-05-26T22:53:58.347 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":248,"stamp":"2024-05-26T22:53:56.495283+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":120,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":258508,"kb_used_data":9048,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754716212,"statfs":{"total":773094113280,"available":772829401088,"internally_reserved":0,"allocated":9265152,"data_stored":4635356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.110651"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:54.447805+0000","last_change":"2024-05-26T22:53:53.443732+0000","last_active":"2024-05-26T22:53:54.447805+0000","last_peered":"2024-05-26T22:53:54.447805+0000","last_clean":"2024-05-26T22:53:54.447805+0000","last_became_active":"2024-05-26T22:53:53.443378+0000","last_became_peered":"2024-05-26T22:53:53.443378+0000","last_unstale":"2024-05-26T22:53:54.447805+0000","last_undegraded":"2024-05-26T22:53:54.447805+0000","last_fullsized":"2024-05-26T22:53:54.447805+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:01:50.450776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:54.445455+0000","last_change":"2024-05-26T22:53:53.450331+0000","last_active":"2024-05-26T22:53:54.445455+0000","last_peered":"2024-05-26T22:53:54.445455+0000","last_clean":"2024-05-26T22:53:54.445455+0000","last_became_active":"2024-05-26T22:53:53.450028+0000","last_became_peered":"2024-05-26T22:53:53.450028+0000","last_unstale":"2024-05-26T22:53:54.445455+0000","last_undegraded":"2024-05-26T22:53:54.445455+0000","last_fullsized":"2024-05-26T22:53:54.445455+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:14:20.756909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-26T22:53:53.451778+0000","last_change":"2024-05-26T22:53:53.449387+0000","last_active":"2024-05-26T22:53:53.451778+0000","last_peered":"2024-05-26T22:53:53.451778+0000","last_clean":"2024-05-26T22:53:53.451778+0000","last_became_active":"2024-05-26T22:53:53.449087+0000","last_became_peered":"2024-05-26T22:53:53.449087+0000","last_unstale":"2024-05-26T22:53:53.451778+0000","last_undegraded":"2024-05-26T22:53:53.451778+0000","last_fullsized":"2024-05-26T22:53:53.451778+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T23:49:11.202473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-26T22:53:53.457727+0000","last_change":"2024-05-26T22:53:53.455545+0000","last_active":"2024-05-26T22:53:53.457727+0000","last_peered":"2024-05-26T22:53:53.457727+0000","last_clean":"2024-05-26T22:53:53.457727+0000","last_became_active":"2024-05-26T22:53:53.455244+0000","last_became_peered":"2024-05-26T22:53:53.455244+0000","last_unstale":"2024-05-26T22:53:53.457727+0000","last_undegraded":"2024-05-26T22:53:53.457727+0000","last_fullsized":"2024-05-26T22:53:53.457727+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:27:37.532783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-26T22:53:53.453534+0000","last_change":"2024-05-26T22:53:53.450737+0000","last_active":"2024-05-26T22:53:53.453534+0000","last_peered":"2024-05-26T22:53:53.453534+0000","last_clean":"2024-05-26T22:53:53.453534+0000","last_became_active":"2024-05-26T22:53:53.450450+0000","last_became_peered":"2024-05-26T22:53:53.450450+0000","last_unstale":"2024-05-26T22:53:53.453534+0000","last_undegraded":"2024-05-26T22:53:53.453534+0000","last_fullsized":"2024-05-26T22:53:53.453534+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:27:50.935370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:54.853641+0000","last_change":"2024-05-26T22:53:53.443209+0000","last_active":"2024-05-26T22:53:54.853641+0000","last_peered":"2024-05-26T22:53:54.853641+0000","last_clean":"2024-05-26T22:53:54.853641+0000","last_became_active":"2024-05-26T22:53:53.442925+0000","last_became_peered":"2024-05-26T22:53:53.442925+0000","last_unstale":"2024-05-26T22:53:54.853641+0000","last_undegraded":"2024-05-26T22:53:54.853641+0000","last_fullsized":"2024-05-26T22:53:54.853641+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:34:26.592973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-26T22:53:53.455100+0000","last_change":"2024-05-26T22:53:53.447665+0000","last_active":"2024-05-26T22:53:53.455100+0000","last_peered":"2024-05-26T22:53:53.455100+0000","last_clean":"2024-05-26T22:53:53.455100+0000","last_became_active":"2024-05-26T22:53:53.447351+0000","last_became_peered":"2024-05-26T22:53:53.447351+0000","last_unstale":"2024-05-26T22:53:53.455100+0000","last_undegraded":"2024-05-26T22:53:53.455100+0000","last_fullsized":"2024-05-26T22:53:53.455100+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:26:01.737366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:54.853852+0000","last_change":"2024-05-26T22:53:53.449592+0000","last_active":"2024-05-26T22:53:54.853852+0000","last_peered":"2024-05-26T22:53:54.853852+0000","last_clean":"2024-05-26T22:53:54.853852+0000","last_became_active":"2024-05-26T22:53:53.448033+0000","last_became_peered":"2024-05-26T22:53:53.448033+0000","last_unstale":"2024-05-26T22:53:54.853852+0000","last_undegraded":"2024-05-26T22:53:54.853852+0000","last_fullsized":"2024-05-26T22:53:54.853852+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:43:55.783780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:54.854139+0000","last_change":"2024-05-26T22:53:53.438302+0000","last_active":"2024-05-26T22:53:54.854139+0000","last_peered":"2024-05-26T22:53:54.854139+0000","last_clean":"2024-05-26T22:53:54.854139+0000","last_became_active":"2024-05-26T22:53:53.437951+0000","last_became_peered":"2024-05-26T22:53:53.437951+0000","last_unstale":"2024-05-26T22:53:54.854139+0000","last_undegraded":"2024-05-26T22:53:54.854139+0000","last_fullsized":"2024-05-26T22:53:54.854139+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:19:16.286517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:54.447831+0000","last_change":"2024-05-26T22:53:53.444607+0000","last_active":"2024-05-26T22:53:54.447831+0000","last_peered":"2024-05-26T22:53:54.447831+0000","last_clean":"2024-05-26T22:53:54.447831+0000","last_became_active":"2024-05-26T22:53:53.444354+0000","last_became_peered":"2024-05-26T22:53:53.444354+0000","last_unstale":"2024-05-26T22:53:54.447831+0000","last_undegraded":"2024-05-26T22:53:54.447831+0000","last_fullsized":"2024-05-26T22:53:54.447831+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:03:19.630407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:54.445642+0000","last_change":"2024-05-26T22:53:53.452275+0000","last_active":"2024-05-26T22:53:54.445642+0000","last_peered":"2024-05-26T22:53:54.445642+0000","last_clean":"2024-05-26T22:53:54.445642+0000","last_became_active":"2024-05-26T22:53:53.451820+0000","last_became_peered":"2024-05-26T22:53:53.451820+0000","last_unstale":"2024-05-26T22:53:54.445642+0000","last_undegraded":"2024-05-26T22:53:54.445642+0000","last_fullsized":"2024-05-26T22:53:54.445642+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:33:22.065967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:54.447331+0000","last_change":"2024-05-26T22:53:53.441915+0000","last_active":"2024-05-26T22:53:54.447331+0000","last_peered":"2024-05-26T22:53:54.447331+0000","last_clean":"2024-05-26T22:53:54.447331+0000","last_became_active":"2024-05-26T22:53:53.441573+0000","last_became_peered":"2024-05-26T22:53:53.441573+0000","last_unstale":"2024-05-26T22:53:54.447331+0000","last_undegraded":"2024-05-26T22:53:54.447331+0000","last_fullsized":"2024-05-26T22:53:54.447331+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:04:38.162417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-26T22:53:53.455068+0000","last_change":"2024-05-26T22:53:53.452267+0000","last_active":"2024-05-26T22:53:53.455068+0000","last_peered":"2024-05-26T22:53:53.455068+0000","last_clean":"2024-05-26T22:53:53.455068+0000","last_became_active":"2024-05-26T22:53:53.451971+0000","last_became_peered":"2024-05-26T22:53:53.451971+0000","last_unstale":"2024-05-26T22:53:53.455068+0000","last_undegraded":"2024-05-26T22:53:53.455068+0000","last_fullsized":"2024-05-26T22:53:53.455068+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:40:30.639915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-26T22:53:53.458677+0000","last_change":"2024-05-26T22:53:53.455924+0000","last_active":"2024-05-26T22:53:53.458677+0000","last_peered":"2024-05-26T22:53:53.458677+0000","last_clean":"2024-05-26T22:53:53.458677+0000","last_became_active":"2024-05-26T22:53:53.455735+0000","last_became_peered":"2024-05-26T22:53:53.455735+0000","last_unstale":"2024-05-26T22:53:53.458677+0000","last_undegraded":"2024-05-26T22:53:53.458677+0000","last_fullsized":"2024-05-26T22:53:53.458677+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:16:47.977518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:55.474537+0000","last_change":"2024-05-26T22:53:53.447550+0000","last_active":"2024-05-26T22:53:55.474537+0000","last_peered":"2024-05-26T22:53:55.474537+0000","last_clean":"2024-05-26T22:53:55.474537+0000","last_became_active":"2024-05-26T22:53:53.447253+0000","last_became_peered":"2024-05-26T22:53:53.447253+0000","last_unstale":"2024-05-26T22:53:55.474537+0000","last_undegraded":"2024-05-26T22:53:55.474537+0000","last_fullsized":"2024-05-26T22:53:55.474537+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:21:55.566073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-26T22:53:53.450886+0000","last_change":"2024-05-26T22:53:53.443477+0000","last_active":"2024-05-26T22:53:53.450886+0000","last_peered":"2024-05-26T22:53:53.450886+0000","last_clean":"2024-05-26T22:53:53.450886+0000","last_became_active":"2024-05-26T22:53:53.443202+0000","last_became_peered":"2024-05-26T22:53:53.443202+0000","last_unstale":"2024-05-26T22:53:53.450886+0000","last_undegraded":"2024-05-26T22:53:53.450886+0000","last_fullsized":"2024-05-26T22:53:53.450886+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:13:50.325150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-26T22:53:53.456243+0000","last_change":"2024-05-26T22:53:53.455724+0000","last_active":"2024-05-26T22:53:53.456243+0000","last_peered":"2024-05-26T22:53:53.456243+0000","last_clean":"2024-05-26T22:53:53.456243+0000","last_became_active":"2024-05-26T22:53:53.455434+0000","last_became_peered":"2024-05-26T22:53:53.455434+0000","last_unstale":"2024-05-26T22:53:53.456243+0000","last_undegraded":"2024-05-26T22:53:53.456243+0000","last_fullsized":"2024-05-26T22:53:53.456243+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:37:40.975738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-26T22:53:53.443340+0000","last_change":"2024-05-26T22:53:53.436932+0000","last_active":"2024-05-26T22:53:53.443340+0000","last_peered":"2024-05-26T22:53:53.443340+0000","last_clean":"2024-05-26T22:53:53.443340+0000","last_became_active":"2024-05-26T22:53:53.436334+0000","last_became_peered":"2024-05-26T22:53:53.436334+0000","last_unstale":"2024-05-26T22:53:53.443340+0000","last_undegraded":"2024-05-26T22:53:53.443340+0000","last_fullsized":"2024-05-26T22:53:53.443340+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T22:55:25.504357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:54.443661+0000","last_change":"2024-05-26T22:53:53.442684+0000","last_active":"2024-05-26T22:53:54.443661+0000","last_peered":"2024-05-26T22:53:54.443661+0000","last_clean":"2024-05-26T22:53:54.443661+0000","last_became_active":"2024-05-26T22:53:53.442380+0000","last_became_peered":"2024-05-26T22:53:53.442380+0000","last_unstale":"2024-05-26T22:53:54.443661+0000","last_undegraded":"2024-05-26T22:53:54.443661+0000","last_fullsized":"2024-05-26T22:53:54.443661+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:02:02.966057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-26T22:53:53.445384+0000","last_change":"2024-05-26T22:53:53.437423+0000","last_active":"2024-05-26T22:53:53.445384+0000","last_peered":"2024-05-26T22:53:53.445384+0000","last_clean":"2024-05-26T22:53:53.445384+0000","last_became_active":"2024-05-26T22:53:53.437168+0000","last_became_peered":"2024-05-26T22:53:53.437168+0000","last_unstale":"2024-05-26T22:53:53.445384+0000","last_undegraded":"2024-05-26T22:53:53.445384+0000","last_fullsized":"2024-05-26T22:53:53.445384+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:57:54.261540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-26T22:53:53.444734+0000","last_change":"2024-05-26T22:53:53.439597+0000","last_active":"2024-05-26T22:53:53.444734+0000","last_peered":"2024-05-26T22:53:53.444734+0000","last_clean":"2024-05-26T22:53:53.444734+0000","last_became_active":"2024-05-26T22:53:53.439292+0000","last_became_peered":"2024-05-26T22:53:53.439292+0000","last_unstale":"2024-05-26T22:53:53.444734+0000","last_undegraded":"2024-05-26T22:53:53.444734+0000","last_fullsized":"2024-05-26T22:53:53.444734+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:50:26.618085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:54.445116+0000","last_change":"2024-05-26T22:53:53.459955+0000","last_active":"2024-05-26T22:53:54.445116+0000","last_peered":"2024-05-26T22:53:54.445116+0000","last_clean":"2024-05-26T22:53:54.445116+0000","last_became_active":"2024-05-26T22:53:53.459649+0000","last_became_peered":"2024-05-26T22:53:53.459649+0000","last_unstale":"2024-05-26T22:53:54.445116+0000","last_undegraded":"2024-05-26T22:53:54.445116+0000","last_fullsized":"2024-05-26T22:53:54.445116+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:27:57.433838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:54.446737+0000","last_change":"2024-05-26T22:53:53.443674+0000","last_active":"2024-05-26T22:53:54.446737+0000","last_peered":"2024-05-26T22:53:54.446737+0000","last_clean":"2024-05-26T22:53:54.446737+0000","last_became_active":"2024-05-26T22:53:53.442166+0000","last_became_peered":"2024-05-26T22:53:53.442166+0000","last_unstale":"2024-05-26T22:53:54.446737+0000","last_undegraded":"2024-05-26T22:53:54.446737+0000","last_fullsized":"2024-05-26T22:53:54.446737+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:03:47.595074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-26T22:53:53.443306+0000","last_change":"2024-05-26T22:53:53.436293+0000","last_active":"2024-05-26T22:53:53.443306+0000","last_peered":"2024-05-26T22:53:53.443306+0000","last_clean":"2024-05-26T22:53:53.443306+0000","last_became_active":"2024-05-26T22:53:53.435947+0000","last_became_peered":"2024-05-26T22:53:53.435947+0000","last_unstale":"2024-05-26T22:53:53.443306+0000","last_undegraded":"2024-05-26T22:53:53.443306+0000","last_fullsized":"2024-05-26T22:53:53.443306+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:10:11.640554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:54.445382+0000","last_change":"2024-05-26T22:53:53.453985+0000","last_active":"2024-05-26T22:53:54.445382+0000","last_peered":"2024-05-26T22:53:54.445382+0000","last_clean":"2024-05-26T22:53:54.445382+0000","last_became_active":"2024-05-26T22:53:53.453716+0000","last_became_peered":"2024-05-26T22:53:53.453716+0000","last_unstale":"2024-05-26T22:53:54.445382+0000","last_undegraded":"2024-05-26T22:53:54.445382+0000","last_fullsized":"2024-05-26T22:53:54.445382+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:07:15.226009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-26T22:53:53.458247+0000","last_change":"2024-05-26T22:53:53.456805+0000","last_active":"2024-05-26T22:53:53.458247+0000","last_peered":"2024-05-26T22:53:53.458247+0000","last_clean":"2024-05-26T22:53:53.458247+0000","last_became_active":"2024-05-26T22:53:53.456517+0000","last_became_peered":"2024-05-26T22:53:53.456517+0000","last_unstale":"2024-05-26T22:53:53.458247+0000","last_undegraded":"2024-05-26T22:53:53.458247+0000","last_fullsized":"2024-05-26T22:53:53.458247+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:04:25.600389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-26T22:53:53.455452+0000","last_change":"2024-05-26T22:53:53.453124+0000","last_active":"2024-05-26T22:53:53.455452+0000","last_peered":"2024-05-26T22:53:53.455452+0000","last_clean":"2024-05-26T22:53:53.455452+0000","last_became_active":"2024-05-26T22:53:53.452288+0000","last_became_peered":"2024-05-26T22:53:53.452288+0000","last_unstale":"2024-05-26T22:53:53.455452+0000","last_undegraded":"2024-05-26T22:53:53.455452+0000","last_fullsized":"2024-05-26T22:53:53.455452+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:34:26.592973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:54.447818+0000","last_change":"2024-05-26T22:53:53.445089+0000","last_active":"2024-05-26T22:53:54.447818+0000","last_peered":"2024-05-26T22:53:54.447818+0000","last_clean":"2024-05-26T22:53:54.447818+0000","last_became_active":"2024-05-26T22:53:53.444723+0000","last_became_peered":"2024-05-26T22:53:53.444723+0000","last_unstale":"2024-05-26T22:53:54.447818+0000","last_undegraded":"2024-05-26T22:53:54.447818+0000","last_fullsized":"2024-05-26T22:53:54.447818+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:02:02.966057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:54.445037+0000","last_change":"2024-05-26T22:53:53.450625+0000","last_active":"2024-05-26T22:53:54.445037+0000","last_peered":"2024-05-26T22:53:54.445037+0000","last_clean":"2024-05-26T22:53:54.445037+0000","last_became_active":"2024-05-26T22:53:53.449562+0000","last_became_peered":"2024-05-26T22:53:53.449562+0000","last_unstale":"2024-05-26T22:53:54.445037+0000","last_undegraded":"2024-05-26T22:53:54.445037+0000","last_fullsized":"2024-05-26T22:53:54.445037+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:34:55.983177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:54.445588+0000","last_change":"2024-05-26T22:53:53.447194+0000","last_active":"2024-05-26T22:53:54.445588+0000","last_peered":"2024-05-26T22:53:54.445588+0000","last_clean":"2024-05-26T22:53:54.445588+0000","last_became_active":"2024-05-26T22:53:53.446869+0000","last_became_peered":"2024-05-26T22:53:53.446869+0000","last_unstale":"2024-05-26T22:53:54.445588+0000","last_undegraded":"2024-05-26T22:53:54.445588+0000","last_fullsized":"2024-05-26T22:53:54.445588+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T23:49:11.534276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-26T22:53:53.447419+0000","last_change":"2024-05-26T22:53:53.440971+0000","last_active":"2024-05-26T22:53:53.447419+0000","last_peered":"2024-05-26T22:53:53.447419+0000","last_clean":"2024-05-26T22:53:53.447419+0000","last_became_active":"2024-05-26T22:53:53.439603+0000","last_became_peered":"2024-05-26T22:53:53.439603+0000","last_unstale":"2024-05-26T22:53:53.447419+0000","last_undegraded":"2024-05-26T22:53:53.447419+0000","last_fullsized":"2024-05-26T22:53:53.447419+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T23:49:11.534276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:54.446859+0000","last_change":"2024-05-26T22:53:53.449879+0000","last_active":"2024-05-26T22:53:54.446859+0000","last_peered":"2024-05-26T22:53:54.446859+0000","last_clean":"2024-05-26T22:53:54.446859+0000","last_became_active":"2024-05-26T22:53:53.449592+0000","last_became_peered":"2024-05-26T22:53:53.449592+0000","last_unstale":"2024-05-26T22:53:54.446859+0000","last_undegraded":"2024-05-26T22:53:54.446859+0000","last_fullsized":"2024-05-26T22:53:54.446859+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:52:18.410150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-26T22:53:53.450495+0000","last_change":"2024-05-26T22:53:53.444818+0000","last_active":"2024-05-26T22:53:53.450495+0000","last_peered":"2024-05-26T22:53:53.450495+0000","last_clean":"2024-05-26T22:53:53.450495+0000","last_became_active":"2024-05-26T22:53:53.443818+0000","last_became_peered":"2024-05-26T22:53:53.443818+0000","last_unstale":"2024-05-26T22:53:53.450495+0000","last_undegraded":"2024-05-26T22:53:53.450495+0000","last_fullsized":"2024-05-26T22:53:53.450495+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:14:20.756909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"81'4","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:54.448474+0000","last_change":"2024-05-26T22:53:52.413767+0000","last_active":"2024-05-26T22:53:54.448474+0000","last_peered":"2024-05-26T22:53:54.448474+0000","last_clean":"2024-05-26T22:53:54.448474+0000","last_became_active":"2024-05-26T22:53:52.412731+0000","last_became_peered":"2024-05-26T22:53:52.412731+0000","last_unstale":"2024-05-26T22:53:54.448474+0000","last_undegraded":"2024-05-26T22:53:54.448474+0000","last_fullsized":"2024-05-26T22:53:54.448474+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_clean_scrub_stamp":"2024-05-26T22:53:51.367149+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T23:58:10.687034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:54.448218+0000","last_change":"2024-05-26T22:53:53.442854+0000","last_active":"2024-05-26T22:53:54.448218+0000","last_peered":"2024-05-26T22:53:54.448218+0000","last_clean":"2024-05-26T22:53:54.448218+0000","last_became_active":"2024-05-26T22:53:53.442323+0000","last_became_peered":"2024-05-26T22:53:53.442323+0000","last_unstale":"2024-05-26T22:53:54.448218+0000","last_undegraded":"2024-05-26T22:53:54.448218+0000","last_fullsized":"2024-05-26T22:53:54.448218+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:33:12.920691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"81'1","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-26T22:53:53.467857+0000","last_change":"2024-05-26T22:53:52.430723+0000","last_active":"2024-05-26T22:53:53.467857+0000","last_peered":"2024-05-26T22:53:53.467857+0000","last_clean":"2024-05-26T22:53:53.467857+0000","last_became_active":"2024-05-26T22:53:52.430375+0000","last_became_peered":"2024-05-26T22:53:52.430375+0000","last_unstale":"2024-05-26T22:53:53.467857+0000","last_undegraded":"2024-05-26T22:53:53.467857+0000","last_fullsized":"2024-05-26T22:53:53.467857+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_clean_scrub_stamp":"2024-05-26T22:53:51.367149+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:34:54.968538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:54.447661+0000","last_change":"2024-05-26T22:53:53.450386+0000","last_active":"2024-05-26T22:53:54.447661+0000","last_peered":"2024-05-26T22:53:54.447661+0000","last_clean":"2024-05-26T22:53:54.447661+0000","last_became_active":"2024-05-26T22:53:53.450081+0000","last_became_peered":"2024-05-26T22:53:53.450081+0000","last_unstale":"2024-05-26T22:53:54.447661+0000","last_undegraded":"2024-05-26T22:53:54.447661+0000","last_fullsized":"2024-05-26T22:53:54.447661+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:41:44.342479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:54.443766+0000","last_change":"2024-05-26T22:53:52.413919+0000","last_active":"2024-05-26T22:53:54.443766+0000","last_peered":"2024-05-26T22:53:54.443766+0000","last_clean":"2024-05-26T22:53:54.443766+0000","last_became_active":"2024-05-26T22:53:52.413637+0000","last_became_peered":"2024-05-26T22:53:52.413637+0000","last_unstale":"2024-05-26T22:53:54.443766+0000","last_undegraded":"2024-05-26T22:53:54.443766+0000","last_fullsized":"2024-05-26T22:53:54.443766+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_clean_scrub_stamp":"2024-05-26T22:53:51.367149+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:43:54.769141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-26T22:53:53.449756+0000","last_change":"2024-05-26T22:53:53.447190+0000","last_active":"2024-05-26T22:53:53.449756+0000","last_peered":"2024-05-26T22:53:53.449756+0000","last_clean":"2024-05-26T22:53:53.449756+0000","last_became_active":"2024-05-26T22:53:53.446905+0000","last_became_peered":"2024-05-26T22:53:53.446905+0000","last_unstale":"2024-05-26T22:53:53.449756+0000","last_undegraded":"2024-05-26T22:53:53.449756+0000","last_fullsized":"2024-05-26T22:53:53.449756+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:37:40.975738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:54.443237+0000","last_change":"2024-05-26T22:53:52.413208+0000","last_active":"2024-05-26T22:53:54.443237+0000","last_peered":"2024-05-26T22:53:54.443237+0000","last_clean":"2024-05-26T22:53:54.443237+0000","last_became_active":"2024-05-26T22:53:52.412854+0000","last_became_peered":"2024-05-26T22:53:52.412854+0000","last_unstale":"2024-05-26T22:53:54.443237+0000","last_undegraded":"2024-05-26T22:53:54.443237+0000","last_fullsized":"2024-05-26T22:53:54.443237+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_clean_scrub_stamp":"2024-05-26T22:53:51.367149+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:37:39.961098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:54.445302+0000","last_change":"2024-05-26T22:53:53.444345+0000","last_active":"2024-05-26T22:53:54.445302+0000","last_peered":"2024-05-26T22:53:54.445302+0000","last_clean":"2024-05-26T22:53:54.445302+0000","last_became_active":"2024-05-26T22:53:53.444042+0000","last_became_peered":"2024-05-26T22:53:53.444042+0000","last_unstale":"2024-05-26T22:53:54.445302+0000","last_undegraded":"2024-05-26T22:53:54.445302+0000","last_fullsized":"2024-05-26T22:53:54.445302+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T23:58:11.701674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"81'1","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-26T22:53:53.468784+0000","last_change":"2024-05-26T22:53:52.429470+0000","last_active":"2024-05-26T22:53:53.468784+0000","last_peered":"2024-05-26T22:53:53.468784+0000","last_clean":"2024-05-26T22:53:53.468784+0000","last_became_active":"2024-05-26T22:53:52.428797+0000","last_became_peered":"2024-05-26T22:53:52.428797+0000","last_unstale":"2024-05-26T22:53:53.468784+0000","last_undegraded":"2024-05-26T22:53:53.468784+0000","last_fullsized":"2024-05-26T22:53:53.468784+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_clean_scrub_stamp":"2024-05-26T22:53:51.367149+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:47:54.194267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-26T22:53:53.449521+0000","last_change":"2024-05-26T22:53:53.442679+0000","last_active":"2024-05-26T22:53:53.449521+0000","last_peered":"2024-05-26T22:53:53.449521+0000","last_clean":"2024-05-26T22:53:53.449521+0000","last_became_active":"2024-05-26T22:53:53.442408+0000","last_became_peered":"2024-05-26T22:53:53.442408+0000","last_unstale":"2024-05-26T22:53:53.449521+0000","last_undegraded":"2024-05-26T22:53:53.449521+0000","last_fullsized":"2024-05-26T22:53:53.449521+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T23:58:11.701674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"81'1","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-26T22:53:53.470113+0000","last_change":"2024-05-26T22:53:52.431126+0000","last_active":"2024-05-26T22:53:53.470113+0000","last_peered":"2024-05-26T22:53:53.470113+0000","last_clean":"2024-05-26T22:53:53.470113+0000","last_became_active":"2024-05-26T22:53:52.430077+0000","last_became_peered":"2024-05-26T22:53:52.430077+0000","last_unstale":"2024-05-26T22:53:53.470113+0000","last_undegraded":"2024-05-26T22:53:53.470113+0000","last_fullsized":"2024-05-26T22:53:53.470113+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_clean_scrub_stamp":"2024-05-26T22:53:51.367149+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:38:22.250786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-26T22:53:53.448561+0000","last_change":"2024-05-26T22:53:53.442835+0000","last_active":"2024-05-26T22:53:53.448561+0000","last_peered":"2024-05-26T22:53:53.448561+0000","last_clean":"2024-05-26T22:53:53.448561+0000","last_became_active":"2024-05-26T22:53:53.442558+0000","last_became_peered":"2024-05-26T22:53:53.442558+0000","last_unstale":"2024-05-26T22:53:53.448561+0000","last_undegraded":"2024-05-26T22:53:53.448561+0000","last_fullsized":"2024-05-26T22:53:53.448561+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:27:50.935370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:54.443145+0000","last_change":"2024-05-26T22:53:52.425707+0000","last_active":"2024-05-26T22:53:54.443145+0000","last_peered":"2024-05-26T22:53:54.443145+0000","last_clean":"2024-05-26T22:53:54.443145+0000","last_became_active":"2024-05-26T22:53:52.425396+0000","last_became_peered":"2024-05-26T22:53:52.425396+0000","last_unstale":"2024-05-26T22:53:54.443145+0000","last_undegraded":"2024-05-26T22:53:54.443145+0000","last_fullsized":"2024-05-26T22:53:54.443145+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_clean_scrub_stamp":"2024-05-26T22:53:51.367149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:34:41.929685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:55.474472+0000","last_change":"2024-05-26T22:53:53.447526+0000","last_active":"2024-05-26T22:53:55.474472+0000","last_peered":"2024-05-26T22:53:55.474472+0000","last_clean":"2024-05-26T22:53:55.474472+0000","last_became_active":"2024-05-26T22:53:53.447218+0000","last_became_peered":"2024-05-26T22:53:53.447218+0000","last_unstale":"2024-05-26T22:53:55.474472+0000","last_undegraded":"2024-05-26T22:53:55.474472+0000","last_fullsized":"2024-05-26T22:53:55.474472+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:14:16.522080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:54.444691+0000","last_change":"2024-05-26T22:53:52.428100+0000","last_active":"2024-05-26T22:53:54.444691+0000","last_peered":"2024-05-26T22:53:54.444691+0000","last_clean":"2024-05-26T22:53:54.444691+0000","last_became_active":"2024-05-26T22:53:52.427781+0000","last_became_peered":"2024-05-26T22:53:52.427781+0000","last_unstale":"2024-05-26T22:53:54.444691+0000","last_undegraded":"2024-05-26T22:53:54.444691+0000","last_fullsized":"2024-05-26T22:53:54.444691+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_clean_scrub_stamp":"2024-05-26T22:53:51.367149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T22:56:41.031306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:55.475085+0000","last_change":"2024-05-26T22:53:53.454472+0000","last_active":"2024-05-26T22:53:55.475085+0000","last_peered":"2024-05-26T22:53:55.475085+0000","last_clean":"2024-05-26T22:53:55.475085+0000","last_became_active":"2024-05-26T22:53:53.452707+0000","last_became_peered":"2024-05-26T22:53:53.452707+0000","last_unstale":"2024-05-26T22:53:55.475085+0000","last_undegraded":"2024-05-26T22:53:55.475085+0000","last_fullsized":"2024-05-26T22:53:55.475085+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:30:32.015136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:54.448747+0000","last_change":"2024-05-26T22:53:52.418784+0000","last_active":"2024-05-26T22:53:54.448747+0000","last_peered":"2024-05-26T22:53:54.448747+0000","last_clean":"2024-05-26T22:53:54.448747+0000","last_became_active":"2024-05-26T22:53:52.417510+0000","last_became_peered":"2024-05-26T22:53:52.417510+0000","last_unstale":"2024-05-26T22:53:54.448747+0000","last_undegraded":"2024-05-26T22:53:54.448747+0000","last_fullsized":"2024-05-26T22:53:54.448747+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_clean_scrub_stamp":"2024-05-26T22:53:51.367149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:13:49.310511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:54.854220+0000","last_change":"2024-05-26T22:53:53.442864+0000","last_active":"2024-05-26T22:53:54.854220+0000","last_peered":"2024-05-26T22:53:54.854220+0000","last_clean":"2024-05-26T22:53:54.854220+0000","last_became_active":"2024-05-26T22:53:53.442578+0000","last_became_peered":"2024-05-26T22:53:53.442578+0000","last_unstale":"2024-05-26T22:53:54.854220+0000","last_undegraded":"2024-05-26T22:53:54.854220+0000","last_fullsized":"2024-05-26T22:53:54.854220+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:04:25.600389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:54.447765+0000","last_change":"2024-05-26T22:53:52.428124+0000","last_active":"2024-05-26T22:53:54.447765+0000","last_peered":"2024-05-26T22:53:54.447765+0000","last_clean":"2024-05-26T22:53:54.447765+0000","last_became_active":"2024-05-26T22:53:52.427833+0000","last_became_peered":"2024-05-26T22:53:52.427833+0000","last_unstale":"2024-05-26T22:53:54.447765+0000","last_undegraded":"2024-05-26T22:53:54.447765+0000","last_fullsized":"2024-05-26T22:53:54.447765+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_clean_scrub_stamp":"2024-05-26T22:53:51.367149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:14:19.742269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:54.442936+0000","last_change":"2024-05-26T22:53:53.448600+0000","last_active":"2024-05-26T22:53:54.442936+0000","last_peered":"2024-05-26T22:53:54.442936+0000","last_clean":"2024-05-26T22:53:54.442936+0000","last_became_active":"2024-05-26T22:53:53.448240+0000","last_became_peered":"2024-05-26T22:53:53.448240+0000","last_unstale":"2024-05-26T22:53:54.442936+0000","last_undegraded":"2024-05-26T22:53:54.442936+0000","last_fullsized":"2024-05-26T22:53:54.442936+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:04:38.162417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"81'3","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:54.446769+0000","last_change":"2024-05-26T22:53:52.416203+0000","last_active":"2024-05-26T22:53:54.446769+0000","last_peered":"2024-05-26T22:53:54.446769+0000","last_clean":"2024-05-26T22:53:54.446769+0000","last_became_active":"2024-05-26T22:53:52.415267+0000","last_became_peered":"2024-05-26T22:53:52.415267+0000","last_unstale":"2024-05-26T22:53:54.446769+0000","last_undegraded":"2024-05-26T22:53:54.446769+0000","last_fullsized":"2024-05-26T22:53:54.446769+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_clean_scrub_stamp":"2024-05-26T22:53:51.367149+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:34:54.968538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-26T22:53:53.453019+0000","last_change":"2024-05-26T22:53:53.448609+0000","last_active":"2024-05-26T22:53:53.453019+0000","last_peered":"2024-05-26T22:53:53.453019+0000","last_clean":"2024-05-26T22:53:53.453019+0000","last_became_active":"2024-05-26T22:53:53.448330+0000","last_became_peered":"2024-05-26T22:53:53.448330+0000","last_unstale":"2024-05-26T22:53:53.453019+0000","last_undegraded":"2024-05-26T22:53:53.453019+0000","last_fullsized":"2024-05-26T22:53:53.453019+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T22:55:59.978360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"81'6","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:54.854185+0000","last_change":"2024-05-26T22:53:52.424751+0000","last_active":"2024-05-26T22:53:54.854185+0000","last_peered":"2024-05-26T22:53:54.854185+0000","last_clean":"2024-05-26T22:53:54.854185+0000","last_became_active":"2024-05-26T22:53:52.424440+0000","last_became_peered":"2024-05-26T22:53:52.424440+0000","last_unstale":"2024-05-26T22:53:54.854185+0000","last_undegraded":"2024-05-26T22:53:54.854185+0000","last_fullsized":"2024-05-26T22:53:54.854185+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_clean_scrub_stamp":"2024-05-26T22:53:51.367149+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:27:56.419199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:54.443014+0000","last_change":"2024-05-26T22:53:53.440620+0000","last_active":"2024-05-26T22:53:54.443014+0000","last_peered":"2024-05-26T22:53:54.443014+0000","last_clean":"2024-05-26T22:53:54.443014+0000","last_became_active":"2024-05-26T22:53:53.440330+0000","last_became_peered":"2024-05-26T22:53:53.440330+0000","last_unstale":"2024-05-26T22:53:54.443014+0000","last_undegraded":"2024-05-26T22:53:54.443014+0000","last_fullsized":"2024-05-26T22:53:54.443014+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:41:44.342479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:54.444984+0000","last_change":"2024-05-26T22:53:52.418658+0000","last_active":"2024-05-26T22:53:54.444984+0000","last_peered":"2024-05-26T22:53:54.444984+0000","last_clean":"2024-05-26T22:53:54.444984+0000","last_became_active":"2024-05-26T22:53:52.418338+0000","last_became_peered":"2024-05-26T22:53:52.418338+0000","last_unstale":"2024-05-26T22:53:54.444984+0000","last_undegraded":"2024-05-26T22:53:54.444984+0000","last_fullsized":"2024-05-26T22:53:54.444984+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_clean_scrub_stamp":"2024-05-26T22:53:51.367149+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:23:56.451193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-26T22:53:53.449832+0000","last_change":"2024-05-26T22:53:53.446466+0000","last_active":"2024-05-26T22:53:53.449832+0000","last_peered":"2024-05-26T22:53:53.449832+0000","last_clean":"2024-05-26T22:53:53.449832+0000","last_became_active":"2024-05-26T22:53:53.445639+0000","last_became_peered":"2024-05-26T22:53:53.445639+0000","last_unstale":"2024-05-26T22:53:53.449832+0000","last_undegraded":"2024-05-26T22:53:53.449832+0000","last_fullsized":"2024-05-26T22:53:53.449832+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:34:26.592973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:55.474805+0000","last_change":"2024-05-26T22:53:52.414283+0000","last_active":"2024-05-26T22:53:55.474805+0000","last_peered":"2024-05-26T22:53:55.474805+0000","last_clean":"2024-05-26T22:53:55.474805+0000","last_became_active":"2024-05-26T22:53:52.413937+0000","last_became_peered":"2024-05-26T22:53:52.413937+0000","last_unstale":"2024-05-26T22:53:55.474805+0000","last_undegraded":"2024-05-26T22:53:55.474805+0000","last_fullsized":"2024-05-26T22:53:55.474805+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_clean_scrub_stamp":"2024-05-26T22:53:51.367149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:03:18.615767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:54.444724+0000","last_change":"2024-05-26T22:53:53.447857+0000","last_active":"2024-05-26T22:53:54.444724+0000","last_peered":"2024-05-26T22:53:54.444724+0000","last_clean":"2024-05-26T22:53:54.444724+0000","last_became_active":"2024-05-26T22:53:53.443401+0000","last_became_peered":"2024-05-26T22:53:53.443401+0000","last_unstale":"2024-05-26T22:53:54.444724+0000","last_undegraded":"2024-05-26T22:53:54.444724+0000","last_fullsized":"2024-05-26T22:53:54.444724+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:47:55.208907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:54.853728+0000","last_change":"2024-05-26T22:53:52.428819+0000","last_active":"2024-05-26T22:53:54.853728+0000","last_peered":"2024-05-26T22:53:54.853728+0000","last_clean":"2024-05-26T22:53:54.853728+0000","last_became_active":"2024-05-26T22:53:52.428516+0000","last_became_peered":"2024-05-26T22:53:52.428516+0000","last_unstale":"2024-05-26T22:53:54.853728+0000","last_undegraded":"2024-05-26T22:53:54.853728+0000","last_fullsized":"2024-05-26T22:53:54.853728+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_clean_scrub_stamp":"2024-05-26T22:53:51.367149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T23:49:10.519636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:54.444679+0000","last_change":"2024-05-26T22:53:53.448228+0000","last_active":"2024-05-26T22:53:54.444679+0000","last_peered":"2024-05-26T22:53:54.444679+0000","last_clean":"2024-05-26T22:53:54.444679+0000","last_became_active":"2024-05-26T22:53:53.447919+0000","last_became_peered":"2024-05-26T22:53:53.447919+0000","last_unstale":"2024-05-26T22:53:54.444679+0000","last_undegraded":"2024-05-26T22:53:54.444679+0000","last_fullsized":"2024-05-26T22:53:54.444679+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:38:23.265425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"81'8","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:54.442730+0000","last_change":"2024-05-26T22:53:52.422609+0000","last_active":"2024-05-26T22:53:54.442730+0000","last_peered":"2024-05-26T22:53:54.442730+0000","last_clean":"2024-05-26T22:53:54.442730+0000","last_became_active":"2024-05-26T22:53:52.422109+0000","last_became_peered":"2024-05-26T22:53:52.422109+0000","last_unstale":"2024-05-26T22:53:54.442730+0000","last_undegraded":"2024-05-26T22:53:54.442730+0000","last_fullsized":"2024-05-26T22:53:54.442730+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_clean_scrub_stamp":"2024-05-26T22:53:51.367149+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:34:25.578333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-26T22:53:53.448574+0000","last_change":"2024-05-26T22:53:53.444274+0000","last_active":"2024-05-26T22:53:53.448574+0000","last_peered":"2024-05-26T22:53:53.448574+0000","last_clean":"2024-05-26T22:53:53.448574+0000","last_became_active":"2024-05-26T22:53:53.443963+0000","last_became_peered":"2024-05-26T22:53:53.443963+0000","last_unstale":"2024-05-26T22:53:53.448574+0000","last_undegraded":"2024-05-26T22:53:53.448574+0000","last_fullsized":"2024-05-26T22:53:53.448574+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:34:42.944324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"81'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:54.449001+0000","last_change":"2024-05-26T22:53:52.419826+0000","last_active":"2024-05-26T22:53:54.449001+0000","last_peered":"2024-05-26T22:53:54.449001+0000","last_clean":"2024-05-26T22:53:54.449001+0000","last_became_active":"2024-05-26T22:53:52.419380+0000","last_became_peered":"2024-05-26T22:53:52.419380+0000","last_unstale":"2024-05-26T22:53:54.449001+0000","last_undegraded":"2024-05-26T22:53:54.449001+0000","last_fullsized":"2024-05-26T22:53:54.449001+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_clean_scrub_stamp":"2024-05-26T22:53:51.367149+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:07:14.211370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:54.448478+0000","last_change":"2024-05-26T22:53:53.444381+0000","last_active":"2024-05-26T22:53:54.448478+0000","last_peered":"2024-05-26T22:53:54.448478+0000","last_clean":"2024-05-26T22:53:54.448478+0000","last_became_active":"2024-05-26T22:53:53.444065+0000","last_became_peered":"2024-05-26T22:53:53.444065+0000","last_unstale":"2024-05-26T22:53:54.448478+0000","last_undegraded":"2024-05-26T22:53:54.448478+0000","last_fullsized":"2024-05-26T22:53:54.448478+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:47:50.514023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:54.443656+0000","last_change":"2024-05-26T22:53:52.428127+0000","last_active":"2024-05-26T22:53:54.443656+0000","last_peered":"2024-05-26T22:53:54.443656+0000","last_clean":"2024-05-26T22:53:54.443656+0000","last_became_active":"2024-05-26T22:53:52.427775+0000","last_became_peered":"2024-05-26T22:53:52.427775+0000","last_unstale":"2024-05-26T22:53:54.443656+0000","last_undegraded":"2024-05-26T22:53:54.443656+0000","last_fullsized":"2024-05-26T22:53:54.443656+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_clean_scrub_stamp":"2024-05-26T22:53:51.367149+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:19:15.271878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-26T22:53:53.454915+0000","last_change":"2024-05-26T22:53:53.452374+0000","last_active":"2024-05-26T22:53:53.454915+0000","last_peered":"2024-05-26T22:53:53.454915+0000","last_clean":"2024-05-26T22:53:53.454915+0000","last_became_active":"2024-05-26T22:53:53.451294+0000","last_became_peered":"2024-05-26T22:53:53.451294+0000","last_unstale":"2024-05-26T22:53:53.454915+0000","last_undegraded":"2024-05-26T22:53:53.454915+0000","last_fullsized":"2024-05-26T22:53:53.454915+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:34:42.944324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:54.443403+0000","last_change":"2024-05-26T22:53:52.426053+0000","last_active":"2024-05-26T22:53:54.443403+0000","last_peered":"2024-05-26T22:53:54.443403+0000","last_clean":"2024-05-26T22:53:54.443403+0000","last_became_active":"2024-05-26T22:53:52.425383+0000","last_became_peered":"2024-05-26T22:53:52.425383+0000","last_unstale":"2024-05-26T22:53:54.443403+0000","last_undegraded":"2024-05-26T22:53:54.443403+0000","last_fullsized":"2024-05-26T22:53:54.443403+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_clean_scrub_stamp":"2024-05-26T22:53:51.367149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T23:49:10.187833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-26T22:53:53.457050+0000","last_change":"2024-05-26T22:53:53.452346+0000","last_active":"2024-05-26T22:53:53.457050+0000","last_peered":"2024-05-26T22:53:53.457050+0000","last_clean":"2024-05-26T22:53:53.457050+0000","last_became_active":"2024-05-26T22:53:53.452050+0000","last_became_peered":"2024-05-26T22:53:53.452050+0000","last_unstale":"2024-05-26T22:53:53.457050+0000","last_undegraded":"2024-05-26T22:53:53.457050+0000","last_fullsized":"2024-05-26T22:53:53.457050+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:02:22.099937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:55.474873+0000","last_change":"2024-05-26T22:53:52.414510+0000","last_active":"2024-05-26T22:53:55.474873+0000","last_peered":"2024-05-26T22:53:55.474873+0000","last_clean":"2024-05-26T22:53:55.474873+0000","last_became_active":"2024-05-26T22:53:52.414233+0000","last_became_peered":"2024-05-26T22:53:52.414233+0000","last_unstale":"2024-05-26T22:53:55.474873+0000","last_undegraded":"2024-05-26T22:53:55.474873+0000","last_fullsized":"2024-05-26T22:53:55.474873+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_clean_scrub_stamp":"2024-05-26T22:53:51.367149+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:02:01.951417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:54.853724+0000","last_change":"2024-05-26T22:53:53.442327+0000","last_active":"2024-05-26T22:53:54.853724+0000","last_peered":"2024-05-26T22:53:54.853724+0000","last_clean":"2024-05-26T22:53:54.853724+0000","last_became_active":"2024-05-26T22:53:53.441858+0000","last_became_peered":"2024-05-26T22:53:53.441858+0000","last_unstale":"2024-05-26T22:53:54.853724+0000","last_undegraded":"2024-05-26T22:53:54.853724+0000","last_fullsized":"2024-05-26T22:53:54.853724+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:37:40.975738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-26T22:53:53.418176+0000","last_change":"2024-05-26T22:53:52.419819+0000","last_active":"2024-05-26T22:53:53.418176+0000","last_peered":"2024-05-26T22:53:53.418176+0000","last_clean":"2024-05-26T22:53:53.418176+0000","last_became_active":"2024-05-26T22:53:52.419518+0000","last_became_peered":"2024-05-26T22:53:52.419518+0000","last_unstale":"2024-05-26T22:53:53.418176+0000","last_undegraded":"2024-05-26T22:53:53.418176+0000","last_fullsized":"2024-05-26T22:53:53.418176+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_clean_scrub_stamp":"2024-05-26T22:53:51.367149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:21:14.360322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-26T22:53:53.442612+0000","last_change":"2024-05-26T22:53:53.434377+0000","last_active":"2024-05-26T22:53:53.442612+0000","last_peered":"2024-05-26T22:53:53.442612+0000","last_clean":"2024-05-26T22:53:53.442612+0000","last_became_active":"2024-05-26T22:53:53.434059+0000","last_became_peered":"2024-05-26T22:53:53.434059+0000","last_unstale":"2024-05-26T22:53:53.442612+0000","last_undegraded":"2024-05-26T22:53:53.442612+0000","last_fullsized":"2024-05-26T22:53:53.442612+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T23:56:58.411305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-26T22:53:53.416440+0000","last_change":"2024-05-26T22:53:52.414415+0000","last_active":"2024-05-26T22:53:53.416440+0000","last_peered":"2024-05-26T22:53:53.416440+0000","last_clean":"2024-05-26T22:53:53.416440+0000","last_became_active":"2024-05-26T22:53:52.414141+0000","last_became_peered":"2024-05-26T22:53:52.414141+0000","last_unstale":"2024-05-26T22:53:53.416440+0000","last_undegraded":"2024-05-26T22:53:53.416440+0000","last_fullsized":"2024-05-26T22:53:53.416440+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_clean_scrub_stamp":"2024-05-26T22:53:51.367149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:08:17.591682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:54.853812+0000","last_change":"2024-05-26T22:53:53.437480+0000","last_active":"2024-05-26T22:53:54.853812+0000","last_peered":"2024-05-26T22:53:54.853812+0000","last_clean":"2024-05-26T22:53:54.853812+0000","last_became_active":"2024-05-26T22:53:53.437125+0000","last_became_peered":"2024-05-26T22:53:53.437125+0000","last_unstale":"2024-05-26T22:53:54.853812+0000","last_undegraded":"2024-05-26T22:53:54.853812+0000","last_fullsized":"2024-05-26T22:53:54.853812+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T23:49:11.202473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:55.474227+0000","last_change":"2024-05-26T22:53:52.407508+0000","last_active":"2024-05-26T22:53:55.474227+0000","last_peered":"2024-05-26T22:53:55.474227+0000","last_clean":"2024-05-26T22:53:55.474227+0000","last_became_active":"2024-05-26T22:53:52.407122+0000","last_became_peered":"2024-05-26T22:53:52.407122+0000","last_unstale":"2024-05-26T22:53:55.474227+0000","last_undegraded":"2024-05-26T22:53:55.474227+0000","last_fullsized":"2024-05-26T22:53:55.474227+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_clean_scrub_stamp":"2024-05-26T22:53:51.367149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:01:49.436136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-26T22:53:53.450467+0000","last_change":"2024-05-26T22:53:53.444551+0000","last_active":"2024-05-26T22:53:53.450467+0000","last_peered":"2024-05-26T22:53:53.450467+0000","last_clean":"2024-05-26T22:53:53.450467+0000","last_became_active":"2024-05-26T22:53:53.444103+0000","last_became_peered":"2024-05-26T22:53:53.444103+0000","last_unstale":"2024-05-26T22:53:53.450467+0000","last_undegraded":"2024-05-26T22:53:53.450467+0000","last_fullsized":"2024-05-26T22:53:53.450467+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:07:15.226009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:54.445476+0000","last_change":"2024-05-26T22:53:52.416802+0000","last_active":"2024-05-26T22:53:54.445476+0000","last_peered":"2024-05-26T22:53:54.445476+0000","last_clean":"2024-05-26T22:53:54.445476+0000","last_became_active":"2024-05-26T22:53:52.416484+0000","last_became_peered":"2024-05-26T22:53:52.416484+0000","last_unstale":"2024-05-26T22:53:54.445476+0000","last_undegraded":"2024-05-26T22:53:54.445476+0000","last_fullsized":"2024-05-26T22:53:54.445476+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_clean_scrub_stamp":"2024-05-26T22:53:51.367149+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:03:20.557956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-26T22:53:53.453387+0000","last_change":"2024-05-26T22:53:53.451530+0000","last_active":"2024-05-26T22:53:53.453387+0000","last_peered":"2024-05-26T22:53:53.453387+0000","last_clean":"2024-05-26T22:53:53.453387+0000","last_became_active":"2024-05-26T22:53:53.449803+0000","last_became_peered":"2024-05-26T22:53:53.449803+0000","last_unstale":"2024-05-26T22:53:53.453387+0000","last_undegraded":"2024-05-26T22:53:53.453387+0000","last_fullsized":"2024-05-26T22:53:53.453387+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:43:55.783780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"82'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:54.464243+0000","last_change":"2024-05-26T22:53:52.418176+0000","last_active":"2024-05-26T22:53:54.464243+0000","last_peered":"2024-05-26T22:53:54.464243+0000","last_clean":"2024-05-26T22:53:54.464243+0000","last_became_active":"2024-05-26T22:53:52.417899+0000","last_became_peered":"2024-05-26T22:53:52.417899+0000","last_unstale":"2024-05-26T22:53:54.464243+0000","last_undegraded":"2024-05-26T22:53:54.464243+0000","last_fullsized":"2024-05-26T22:53:54.464243+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_clean_scrub_stamp":"2024-05-26T22:53:51.367149+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:08:19.625298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:54.447951+0000","last_change":"2024-05-26T22:53:53.441197+0000","last_active":"2024-05-26T22:53:54.447951+0000","last_peered":"2024-05-26T22:53:54.447951+0000","last_clean":"2024-05-26T22:53:54.447951+0000","last_became_active":"2024-05-26T22:53:53.436681+0000","last_became_peered":"2024-05-26T22:53:53.436681+0000","last_unstale":"2024-05-26T22:53:54.447951+0000","last_undegraded":"2024-05-26T22:53:54.447951+0000","last_fullsized":"2024-05-26T22:53:54.447951+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:41:10.035551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:54.447278+0000","last_change":"2024-05-26T22:53:52.413388+0000","last_active":"2024-05-26T22:53:54.447278+0000","last_peered":"2024-05-26T22:53:54.447278+0000","last_clean":"2024-05-26T22:53:54.447278+0000","last_became_active":"2024-05-26T22:53:52.413043+0000","last_became_peered":"2024-05-26T22:53:52.413043+0000","last_unstale":"2024-05-26T22:53:54.447278+0000","last_undegraded":"2024-05-26T22:53:54.447278+0000","last_fullsized":"2024-05-26T22:53:54.447278+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_clean_scrub_stamp":"2024-05-26T22:53:51.367149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:27:56.419199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-26T22:53:53.452542+0000","last_change":"2024-05-26T22:53:53.446557+0000","last_active":"2024-05-26T22:53:53.452542+0000","last_peered":"2024-05-26T22:53:53.452542+0000","last_clean":"2024-05-26T22:53:53.452542+0000","last_became_active":"2024-05-26T22:53:53.446276+0000","last_became_peered":"2024-05-26T22:53:53.446276+0000","last_unstale":"2024-05-26T22:53:53.452542+0000","last_undegraded":"2024-05-26T22:53:53.452542+0000","last_fullsized":"2024-05-26T22:53:53.452542+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:33:22.065967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:54.448065+0000","last_change":"2024-05-26T22:53:52.422005+0000","last_active":"2024-05-26T22:53:54.448065+0000","last_peered":"2024-05-26T22:53:54.448065+0000","last_clean":"2024-05-26T22:53:54.448065+0000","last_became_active":"2024-05-26T22:53:52.421720+0000","last_became_peered":"2024-05-26T22:53:52.421720+0000","last_unstale":"2024-05-26T22:53:54.448065+0000","last_undegraded":"2024-05-26T22:53:54.448065+0000","last_fullsized":"2024-05-26T22:53:54.448065+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_clean_scrub_stamp":"2024-05-26T22:53:51.367149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:33:21.051328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:54.853926+0000","last_change":"2024-05-26T22:53:53.441004+0000","last_active":"2024-05-26T22:53:54.853926+0000","last_peered":"2024-05-26T22:53:54.853926+0000","last_clean":"2024-05-26T22:53:54.853926+0000","last_became_active":"2024-05-26T22:53:53.440636+0000","last_became_peered":"2024-05-26T22:53:53.440636+0000","last_unstale":"2024-05-26T22:53:54.853926+0000","last_undegraded":"2024-05-26T22:53:54.853926+0000","last_fullsized":"2024-05-26T22:53:54.853926+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:40:30.639915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:54.443322+0000","last_change":"2024-05-26T22:53:52.422617+0000","last_active":"2024-05-26T22:53:54.443322+0000","last_peered":"2024-05-26T22:53:54.443322+0000","last_clean":"2024-05-26T22:53:54.443322+0000","last_became_active":"2024-05-26T22:53:52.422127+0000","last_became_peered":"2024-05-26T22:53:52.422127+0000","last_unstale":"2024-05-26T22:53:54.443322+0000","last_undegraded":"2024-05-26T22:53:54.443322+0000","last_fullsized":"2024-05-26T22:53:54.443322+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_clean_scrub_stamp":"2024-05-26T22:53:51.367149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:40:29.625276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:54.443529+0000","last_change":"2024-05-26T22:53:53.450328+0000","last_active":"2024-05-26T22:53:54.443529+0000","last_peered":"2024-05-26T22:53:54.443529+0000","last_clean":"2024-05-26T22:53:54.443529+0000","last_became_active":"2024-05-26T22:53:53.447937+0000","last_became_peered":"2024-05-26T22:53:53.447937+0000","last_unstale":"2024-05-26T22:53:54.443529+0000","last_undegraded":"2024-05-26T22:53:54.443529+0000","last_fullsized":"2024-05-26T22:53:54.443529+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:01:50.450776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-26T22:53:53.433248+0000","last_change":"2024-05-26T22:53:52.429754+0000","last_active":"2024-05-26T22:53:53.433248+0000","last_peered":"2024-05-26T22:53:53.433248+0000","last_clean":"2024-05-26T22:53:53.433248+0000","last_became_active":"2024-05-26T22:53:52.429437+0000","last_became_peered":"2024-05-26T22:53:52.429437+0000","last_unstale":"2024-05-26T22:53:53.433248+0000","last_undegraded":"2024-05-26T22:53:53.433248+0000","last_fullsized":"2024-05-26T22:53:53.433248+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_clean_scrub_stamp":"2024-05-26T22:53:51.367149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:27:56.419199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-26T22:53:53.455074+0000","last_change":"2024-05-26T22:53:53.452580+0000","last_active":"2024-05-26T22:53:53.455074+0000","last_peered":"2024-05-26T22:53:53.455074+0000","last_clean":"2024-05-26T22:53:53.455074+0000","last_became_active":"2024-05-26T22:53:53.452004+0000","last_became_peered":"2024-05-26T22:53:53.452004+0000","last_unstale":"2024-05-26T22:53:53.455074+0000","last_undegraded":"2024-05-26T22:53:53.455074+0000","last_fullsized":"2024-05-26T22:53:53.455074+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:19:16.286517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:54.447127+0000","last_change":"2024-05-26T22:53:52.421592+0000","last_active":"2024-05-26T22:53:54.447127+0000","last_peered":"2024-05-26T22:53:54.447127+0000","last_clean":"2024-05-26T22:53:54.447127+0000","last_became_active":"2024-05-26T22:53:52.421270+0000","last_became_peered":"2024-05-26T22:53:52.421270+0000","last_unstale":"2024-05-26T22:53:54.447127+0000","last_undegraded":"2024-05-26T22:53:54.447127+0000","last_fullsized":"2024-05-26T22:53:54.447127+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_clean_scrub_stamp":"2024-05-26T22:53:51.367149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T23:49:10.519636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:54.446859+0000","last_change":"2024-05-26T22:53:53.445240+0000","last_active":"2024-05-26T22:53:54.446859+0000","last_peered":"2024-05-26T22:53:54.446859+0000","last_clean":"2024-05-26T22:53:54.446859+0000","last_became_active":"2024-05-26T22:53:53.444926+0000","last_became_peered":"2024-05-26T22:53:53.444926+0000","last_unstale":"2024-05-26T22:53:54.446859+0000","last_undegraded":"2024-05-26T22:53:54.446859+0000","last_fullsized":"2024-05-26T22:53:54.446859+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:31:00.584149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-26T22:53:53.432229+0000","last_change":"2024-05-26T22:53:52.426301+0000","last_active":"2024-05-26T22:53:53.432229+0000","last_peered":"2024-05-26T22:53:53.432229+0000","last_clean":"2024-05-26T22:53:53.432229+0000","last_became_active":"2024-05-26T22:53:52.425851+0000","last_became_peered":"2024-05-26T22:53:52.425851+0000","last_unstale":"2024-05-26T22:53:53.432229+0000","last_undegraded":"2024-05-26T22:53:53.432229+0000","last_fullsized":"2024-05-26T22:53:53.432229+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_clean_scrub_stamp":"2024-05-26T22:53:51.367149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T23:25:01.321783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-26T22:53:54.445306+0000","last_change":"2024-05-26T22:53:53.446358+0000","last_active":"2024-05-26T22:53:54.445306+0000","last_peered":"2024-05-26T22:53:54.445306+0000","last_clean":"2024-05-26T22:53:54.445306+0000","last_became_active":"2024-05-26T22:53:53.443913+0000","last_became_peered":"2024-05-26T22:53:53.443913+0000","last_unstale":"2024-05-26T22:53:54.445306+0000","last_undegraded":"2024-05-26T22:53:54.445306+0000","last_fullsized":"2024-05-26T22:53:54.445306+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:52.381788+0000","last_clean_scrub_stamp":"2024-05-26T22:53:52.381788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:13:50.325150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"81'1","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-26T22:53:53.470919+0000","last_change":"2024-05-26T22:53:52.426906+0000","last_active":"2024-05-26T22:53:53.470919+0000","last_peered":"2024-05-26T22:53:53.470919+0000","last_clean":"2024-05-26T22:53:53.470919+0000","last_became_active":"2024-05-26T22:53:52.425801+0000","last_became_peered":"2024-05-26T22:53:52.425801+0000","last_unstale":"2024-05-26T22:53:53.470919+0000","last_undegraded":"2024-05-26T22:53:53.470919+0000","last_fullsized":"2024-05-26T22:53:53.470919+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:53:51.367149+0000","last_clean_scrub_stamp":"2024-05-26T22:53:51.367149+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T22:55:58.963721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":284,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-26T22:53:53.431349+0000","last_change":"2024-05-26T22:48:12.068917+0000","last_active":"2024-05-26T22:53:53.431349+0000","last_peered":"2024-05-26T22:53:53.431349+0000","last_clean":"2024-05-26T22:53:53.431349+0000","last_became_active":"2024-05-26T22:48:12.065934+0000","last_became_peered":"2024-05-26T22:48:12.065934+0000","last_unstale":"2024-05-26T22:53:53.431349+0000","last_undegraded":"2024-05-26T22:53:53.431349+0000","last_fullsized":"2024-05-26T22:53:53.431349+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:10.628122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:10.628122+0000","last_clean_scrub_stamp":"2024-05-26T22:48:10.628122+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:14:58.895911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574920,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32736,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339104,"statfs":{"total":96636764160,"available":96603242496,"internally_reserved":0,"allocated":1593344,"data_stored":1018513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574920,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32180,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339660,"statfs":{"total":96636764160,"available":96603811840,"internally_reserved":0,"allocated":1019904,"data_stored":435069,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574921,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32180,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339660,"statfs":{"total":96636764160,"available":96603811840,"internally_reserved":0,"allocated":1024000,"data_stored":436009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574920,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32152,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339688,"statfs":{"total":96636764160,"available":96603840512,"internally_reserved":0,"allocated":995328,"data_stored":428075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574920,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32168,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339672,"statfs":{"total":96636764160,"available":96603824128,"internally_reserved":0,"allocated":1007616,"data_stored":429687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574920,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32156,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339684,"statfs":{"total":96636764160,"available":96603836416,"internally_reserved":0,"allocated":999424,"data_stored":428657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574920,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32180,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339660,"statfs":{"total":96636764160,"available":96603811840,"internally_reserved":0,"allocated":1019904,"data_stored":434569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574920,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32756,"kb_used_data":1568,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339084,"statfs":{"total":96636764160,"available":96603222016,"internally_reserved":0,"allocated":1605632,"data_stored":1024777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-26T22:53:58.351 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-26T22:53:58.351 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-26T22:53:58.357 INFO:tasks.ceph.mds.a.smithi131.stderr:2024-05-26T22:53:58.358+0000 7f8c96c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-26T22:53:58.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.571+0000 7efef8c00640 1 Processor -- start 2024-05-26T22:53:58.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.572+0000 7efef8c00640 1 -- start start 2024-05-26T22:53:58.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.573+0000 7efef8c00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efef40cbef0 0x7efef40cc2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:58.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.573+0000 7efef8c00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7efef40ad370 0x7efef40cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:58.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.573+0000 7efef8c00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efef40cdca0 con 0x7efef40acef0 2024-05-26T22:53:58.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.573+0000 7efef8c00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7efef40cdf80 con 0x7efef40ad370 2024-05-26T22:53:58.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.573+0000 7efef8c00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7efef40ce260 con 0x7efef40cbef0 2024-05-26T22:53:58.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.574+0000 7efef2800640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7efef40ad370 0x7efef40cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:58.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.574+0000 7efef2800640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7efef40ad370 0x7efef40cea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.131:57822/0 (socket says 172.21.15.131:57822) 2024-05-26T22:53:58.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.574+0000 7efef2800640 1 -- 172.21.15.131:0/3654954649 learned_addr learned my addr 172.21.15.131:0/3654954649 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:53:58.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.574+0000 7efef1e00640 1 --2- 172.21.15.131:0/3654954649 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efef40cbef0 0x7efef40cc2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:58.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.575+0000 7efef2800640 1 -- 172.21.15.131:0/3654954649 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efef40cbef0 msgr2=0x7efef40cc2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:58.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.575+0000 7efef2800640 1 --2- 172.21.15.131:0/3654954649 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efef40cbef0 0x7efef40cc2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:58.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.575+0000 7efef2800640 1 -- 172.21.15.131:0/3654954649 >> v1:172.21.15.187:6789/0 conn(0x7efef40acef0 legacy=0x7efef406d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:58.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.575+0000 7efef2800640 1 -- 172.21.15.131:0/3654954649 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efef40ce540 con 0x7efef40ad370 2024-05-26T22:53:58.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.576+0000 7efef2800640 1 --2- 172.21.15.131:0/3654954649 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efef40ad370 0x7efef40cea60 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7efeec00d180 tx=0x7efeec02c960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e96eca612bc09d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:58.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.577+0000 7efef1400640 1 -- 172.21.15.131:0/3654954649 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeec009600 con 0x7efef40ad370 2024-05-26T22:53:58.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.577+0000 7efef1400640 1 -- 172.21.15.131:0/3654954649 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efeec037070 con 0x7efef40ad370 2024-05-26T22:53:58.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.577+0000 7efef1400640 1 -- 172.21.15.131:0/3654954649 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeec0099c0 con 0x7efef40ad370 2024-05-26T22:53:58.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.577+0000 7efef8c00640 1 -- 172.21.15.131:0/3654954649 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efef40ad370 msgr2=0x7efef40cea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:58.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.577+0000 7efef8c00640 1 --2- 172.21.15.131:0/3654954649 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efef40ad370 0x7efef40cea60 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7efeec00d180 tx=0x7efeec02c960 comp rx=0 tx=0).stop 2024-05-26T22:53:58.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.578+0000 7efef8c00640 1 -- 172.21.15.131:0/3654954649 shutdown_connections 2024-05-26T22:53:58.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.578+0000 7efef8c00640 1 --2- 172.21.15.131:0/3654954649 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efef40ad370 0x7efef40cea60 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:58.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.578+0000 7efef8c00640 1 --2- 172.21.15.131:0/3654954649 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efef40cbef0 0x7efef40cc2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:58.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.578+0000 7efef8c00640 1 -- 172.21.15.131:0/3654954649 >> 172.21.15.131:0/3654954649 conn(0x7efef406bc90 msgr2=0x7efef40b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:58.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.578+0000 7efef8c00640 1 -- 172.21.15.131:0/3654954649 shutdown_connections 2024-05-26T22:53:58.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.578+0000 7efef8c00640 1 -- 172.21.15.131:0/3654954649 wait complete. 2024-05-26T22:53:58.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.580+0000 7efef8c00640 1 Processor -- start 2024-05-26T22:53:58.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.580+0000 7efef8c00640 1 -- start start 2024-05-26T22:53:58.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.581+0000 7efef8c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efef40acef0 0x7efef4072fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:58.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.581+0000 7efef8c00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efef40ad370 0x7efef41547f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:58.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.581+0000 7efef8c00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efef40cbef0 0x7efef409c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:58.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.581+0000 7efef8c00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7efef40cefa0 con 0x7efef40acef0 2024-05-26T22:53:58.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.581+0000 7efef8c00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7efef406dc40 con 0x7efef40cbef0 2024-05-26T22:53:58.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.581+0000 7efef8c00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7efef40cf120 con 0x7efef40ad370 2024-05-26T22:53:58.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.581+0000 7efef1e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efef40acef0 0x7efef4072fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:58.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.582+0000 7efef1e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efef40acef0 0x7efef4072fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56782/0 (socket says 172.21.15.131:56782) 2024-05-26T22:53:58.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.582+0000 7efef1e00640 1 -- 172.21.15.131:0/3981909356 learned_addr learned my addr 172.21.15.131:0/3981909356 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:53:58.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.582+0000 7efee9400640 1 --2- 172.21.15.131:0/3981909356 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efef40ad370 0x7efef41547f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:58.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.582+0000 7efef2800640 1 --2- 172.21.15.131:0/3981909356 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efef40cbef0 0x7efef409c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:58.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.582+0000 7efef1e00640 1 -- 172.21.15.131:0/3981909356 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efef40ad370 msgr2=0x7efef41547f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:58.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.582+0000 7efef1e00640 1 --2- 172.21.15.131:0/3981909356 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efef40ad370 0x7efef41547f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:58.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.582+0000 7efef1e00640 1 -- 172.21.15.131:0/3981909356 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efef40cbef0 msgr2=0x7efef409c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:58.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.582+0000 7efef1e00640 1 --2- 172.21.15.131:0/3981909356 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efef40cbef0 0x7efef409c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:58.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.582+0000 7efef1e00640 1 -- 172.21.15.131:0/3981909356 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efef41f3f80 con 0x7efef40acef0 2024-05-26T22:53:58.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.582+0000 7efef1e00640 1 --2- 172.21.15.131:0/3981909356 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efef40acef0 0x7efef4072fa0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7efee400e9b0 tx=0x7efee400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:58.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.583+0000 7efeeaa00640 1 -- 172.21.15.131:0/3981909356 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee4017070 con 0x7efef40acef0 2024-05-26T22:53:58.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.583+0000 7efeeaa00640 1 -- 172.21.15.131:0/3981909356 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efee400c9a0 con 0x7efef40acef0 2024-05-26T22:53:58.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.583+0000 7efef8c00640 1 -- 172.21.15.131:0/3981909356 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efef41f4260 con 0x7efef40acef0 2024-05-26T22:53:58.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.583+0000 7efeeaa00640 1 -- 172.21.15.131:0/3981909356 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee400cd90 con 0x7efef40acef0 2024-05-26T22:53:58.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.583+0000 7efef8c00640 1 -- 172.21.15.131:0/3981909356 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efef41f4820 con 0x7efef40acef0 2024-05-26T22:53:58.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.584+0000 7efef8c00640 1 -- 172.21.15.131:0/3981909356 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efef41f7150 con 0x7efef40acef0 2024-05-26T22:53:58.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.585+0000 7efeeaa00640 1 -- 172.21.15.131:0/3981909356 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7efee401a070 con 0x7efef40acef0 2024-05-26T22:53:58.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.586+0000 7efeeaa00640 1 --2- 172.21.15.131:0/3981909356 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7efed006be60 0x7efed006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:58.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.586+0000 7efeeaa00640 1 -- 172.21.15.131:0/3981909356 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5709+0+0 (secure 0 0 0) 0x7efee408b150 con 0x7efef40acef0 2024-05-26T22:53:58.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.586+0000 7efee9400640 1 --2- 172.21.15.131:0/3981909356 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7efed006be60 0x7efed006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:58.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.587+0000 7efee9400640 1 --2- 172.21.15.131:0/3981909356 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7efed006be60 0x7efed006e320 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7efedc007fd0 tx=0x7efedc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:58.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.590+0000 7efeeaa00640 1 -- 172.21.15.131:0/3981909356 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7efef41f7150 con 0x7efef40acef0 2024-05-26T22:53:58.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.859+0000 7efef8c00640 1 -- 172.21.15.131:0/3981909356 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7efef406c320 con 0x7efef40acef0 2024-05-26T22:53:58.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.865+0000 7efeeaa00640 1 -- 172.21.15.131:0/3981909356 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v17) ==== 190+0+147 (secure 0 0 0) 0x7efee4059b90 con 0x7efef40acef0 2024-05-26T22:53:58.865 INFO:teuthology.orchestra.run.smithi131.stdout:[client.0] 2024-05-26T22:53:58.865 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQD2u1NmSGVWChAAH/wq3iaUs6ZELRhLKhmURQ== 2024-05-26T22:53:58.865 INFO:teuthology.orchestra.run.smithi131.stdout: caps mds = "allow" 2024-05-26T22:53:58.865 INFO:teuthology.orchestra.run.smithi131.stdout: caps mon = "allow r" 2024-05-26T22:53:58.865 INFO:teuthology.orchestra.run.smithi131.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-05-26T22:53:58.865 INFO:teuthology.orchestra.run.smithi131.stderr:updated caps for client.0 2024-05-26T22:53:58.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.868+0000 7efef8c00640 1 -- 172.21.15.131:0/3981909356 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7efed006be60 msgr2=0x7efed006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:58.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.868+0000 7efef8c00640 1 --2- 172.21.15.131:0/3981909356 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7efed006be60 0x7efed006e320 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7efedc007fd0 tx=0x7efedc009040 comp rx=0 tx=0).stop 2024-05-26T22:53:58.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.868+0000 7efef8c00640 1 -- 172.21.15.131:0/3981909356 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efef40acef0 msgr2=0x7efef4072fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:58.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.868+0000 7efef8c00640 1 --2- 172.21.15.131:0/3981909356 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efef40acef0 0x7efef4072fa0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7efee400e9b0 tx=0x7efee400ee70 comp rx=0 tx=0).stop 2024-05-26T22:53:58.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.868+0000 7efef8c00640 1 -- 172.21.15.131:0/3981909356 shutdown_connections 2024-05-26T22:53:58.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.868+0000 7efef8c00640 1 --2- 172.21.15.131:0/3981909356 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efef40ad370 0x7efef41547f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:58.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.868+0000 7efef8c00640 1 --2- 172.21.15.131:0/3981909356 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7efed006be60 0x7efed006e320 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:58.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.868+0000 7efef8c00640 1 --2- 172.21.15.131:0/3981909356 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efef40cbef0 0x7efef409c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:58.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.868+0000 7efef8c00640 1 --2- 172.21.15.131:0/3981909356 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efef40acef0 0x7efef4072fa0 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:58.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.868+0000 7efef8c00640 1 -- 172.21.15.131:0/3981909356 >> 172.21.15.131:0/3981909356 conn(0x7efef406bc90 msgr2=0x7efef40cd2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:58.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.868+0000 7efef8c00640 1 -- 172.21.15.131:0/3981909356 shutdown_connections 2024-05-26T22:53:58.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:58.868+0000 7efef8c00640 1 -- 172.21.15.131:0/3981909356 wait complete. 2024-05-26T22:53:58.887 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-26T22:53:59.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.089+0000 7f33ca200640 1 Processor -- start 2024-05-26T22:53:59.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.089+0000 7f33ca200640 1 -- start start 2024-05-26T22:53:59.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.090+0000 7f33ca200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f33c4155840 0x7f33c4071930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:59.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.090+0000 7f33ca200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f33c415c3d0 0x7f33c406ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:59.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.090+0000 7f33ca200640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33c40722a0 con 0x7f33c406f490 2024-05-26T22:53:59.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.090+0000 7f33ca200640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f33c4072580 con 0x7f33c4155840 2024-05-26T22:53:59.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.090+0000 7f33ca200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f33c4072860 con 0x7f33c415c3d0 2024-05-26T22:53:59.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.090+0000 7f33c3400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f33c4155840 0x7f33c4071930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:59.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.090+0000 7f33c3e00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f33c406f490 0x7f33c415ec80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:50658/0 (socket says 172.21.15.131:50658) 2024-05-26T22:53:59.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.090+0000 7f33c3e00640 1 -- 172.21.15.131:0/1430609393 learned_addr learned my addr 172.21.15.131:0/1430609393 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:53:59.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.090+0000 7f33c3400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f33c4155840 0x7f33c4071930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56786/0 (socket says 172.21.15.131:56786) 2024-05-26T22:53:59.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.091+0000 7f33c3400640 1 -- 172.21.15.131:0/1430609393 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f33c415c3d0 msgr2=0x7f33c406ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:59.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.091+0000 7f33bbe00640 1 --2- 172.21.15.131:0/1430609393 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f33c415c3d0 0x7f33c406ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:59.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.091+0000 7f33c3400640 1 --2- 172.21.15.131:0/1430609393 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f33c415c3d0 0x7f33c406ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:59.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.091+0000 7f33c2a00640 1 -- 172.21.15.131:0/1430609393 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2886043470 0 0) 0x7f33c40722a0 con 0x7f33c406f490 2024-05-26T22:53:59.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.091+0000 7f33c3400640 1 -- 172.21.15.131:0/1430609393 >> v1:172.21.15.131:6789/0 conn(0x7f33c406f490 legacy=0x7f33c415ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:59.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.091+0000 7f33c3400640 1 -- 172.21.15.131:0/1430609393 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33c4072b40 con 0x7f33c4155840 2024-05-26T22:53:59.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.092+0000 7f33c3400640 1 --2- 172.21.15.131:0/1430609393 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f33c4155840 0x7f33c4071930 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f33b40097a0 tx=0x7f33b402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cbf953958bef4db server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:59.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.092+0000 7f33c2a00640 1 -- 172.21.15.131:0/1430609393 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b4037070 con 0x7f33c4155840 2024-05-26T22:53:59.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.092+0000 7f33c2a00640 1 -- 172.21.15.131:0/1430609393 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33b4032440 con 0x7f33c4155840 2024-05-26T22:53:59.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.092+0000 7f33c2a00640 1 -- 172.21.15.131:0/1430609393 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b4032810 con 0x7f33c4155840 2024-05-26T22:53:59.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.093+0000 7f33ca200640 1 -- 172.21.15.131:0/1430609393 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f33c4155840 msgr2=0x7f33c4071930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:59.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.093+0000 7f33ca200640 1 --2- 172.21.15.131:0/1430609393 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f33c4155840 0x7f33c4071930 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f33b40097a0 tx=0x7f33b402a180 comp rx=0 tx=0).stop 2024-05-26T22:53:59.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.093+0000 7f33ca200640 1 -- 172.21.15.131:0/1430609393 shutdown_connections 2024-05-26T22:53:59.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.093+0000 7f33ca200640 1 --2- 172.21.15.131:0/1430609393 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f33c415c3d0 0x7f33c406ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:59.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.093+0000 7f33ca200640 1 --2- 172.21.15.131:0/1430609393 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f33c4155840 0x7f33c4071930 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:59.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.093+0000 7f33ca200640 1 -- 172.21.15.131:0/1430609393 >> 172.21.15.131:0/1430609393 conn(0x7f33c40b1e90 msgr2=0x7f33c40b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:59.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.093+0000 7f33ca200640 1 -- 172.21.15.131:0/1430609393 shutdown_connections 2024-05-26T22:53:59.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.093+0000 7f33ca200640 1 -- 172.21.15.131:0/1430609393 wait complete. 2024-05-26T22:53:59.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.095+0000 7f33ca200640 1 Processor -- start 2024-05-26T22:53:59.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.095+0000 7f33ca200640 1 -- start start 2024-05-26T22:53:59.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.096+0000 7f33ca200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f33c406f490 0x7f33c414d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:59.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.096+0000 7f33ca200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f33c4155840 0x7f33c40ceff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:59.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.096+0000 7f33ca200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f33c415c3d0 0x7f33c40c98d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:59.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.096+0000 7f33ca200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f33c4071ff0 con 0x7f33c406f490 2024-05-26T22:53:59.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.096+0000 7f33ca200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f33c4071e70 con 0x7f33c415c3d0 2024-05-26T22:53:59.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.096+0000 7f33ca200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f33c40aee50 con 0x7f33c4155840 2024-05-26T22:53:59.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.096+0000 7f33c3400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f33c406f490 0x7f33c414d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:59.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.096+0000 7f33c3400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f33c406f490 0x7f33c414d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56796/0 (socket says 172.21.15.131:56796) 2024-05-26T22:53:59.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.096+0000 7f33c3400640 1 -- 172.21.15.131:0/93992485 learned_addr learned my addr 172.21.15.131:0/93992485 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:53:59.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.097+0000 7f33c3400640 1 -- 172.21.15.131:0/93992485 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f33c4155840 msgr2=0x7f33c40ceff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:59.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.097+0000 7f33c3400640 1 --2- 172.21.15.131:0/93992485 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f33c4155840 0x7f33c40ceff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:59.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.097+0000 7f33c3400640 1 -- 172.21.15.131:0/93992485 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f33c415c3d0 msgr2=0x7f33c40c98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:59.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.097+0000 7f33c3400640 1 --2- 172.21.15.131:0/93992485 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f33c415c3d0 0x7f33c40c98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:59.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.097+0000 7f33c3400640 1 -- 172.21.15.131:0/93992485 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33c4159350 con 0x7f33c406f490 2024-05-26T22:53:59.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.097+0000 7f33c3400640 1 --2- 172.21.15.131:0/93992485 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f33c406f490 0x7f33c414d900 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f33b40097a0 tx=0x7f33b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:59.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.097+0000 7f33bb400640 1 -- 172.21.15.131:0/93992485 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b4037050 con 0x7f33c406f490 2024-05-26T22:53:59.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.097+0000 7f33bb400640 1 -- 172.21.15.131:0/93992485 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33b4038040 con 0x7f33c406f490 2024-05-26T22:53:59.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.097+0000 7f33bb400640 1 -- 172.21.15.131:0/93992485 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b4004490 con 0x7f33c406f490 2024-05-26T22:53:59.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.097+0000 7f33ca200640 1 -- 172.21.15.131:0/93992485 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33c40cba20 con 0x7f33c406f490 2024-05-26T22:53:59.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.097+0000 7f33ca200640 1 -- 172.21.15.131:0/93992485 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33c40cbfe0 con 0x7f33c406f490 2024-05-26T22:53:59.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.098+0000 7f33ca200640 1 -- 172.21.15.131:0/93992485 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33c4159520 con 0x7f33c406f490 2024-05-26T22:53:59.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.099+0000 7f33bb400640 1 -- 172.21.15.131:0/93992485 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f33b4030980 con 0x7f33c406f490 2024-05-26T22:53:59.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.101+0000 7f33bb400640 1 --2- 172.21.15.131:0/93992485 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f33a006be30 0x7f33a006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:59.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.101+0000 7f33bb400640 1 -- 172.21.15.131:0/93992485 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5709+0+0 (secure 0 0 0) 0x7f33b40abb60 con 0x7f33c406f490 2024-05-26T22:53:59.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.101+0000 7f33bbe00640 1 --2- 172.21.15.131:0/93992485 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f33a006be30 0x7f33a006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:59.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.102+0000 7f33bbe00640 1 --2- 172.21.15.131:0/93992485 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f33a006be30 0x7f33a006e2f0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f33c41f7c00 tx=0x7f33ac008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:59.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.105+0000 7f33bb400640 1 -- 172.21.15.131:0/93992485 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f33b407a590 con 0x7f33c406f490 2024-05-26T22:53:59.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.345+0000 7f33ca200640 1 -- 172.21.15.131:0/93992485 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f33c40694e0 con 0x7f33c406f490 2024-05-26T22:53:59.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.346+0000 7f33bb400640 1 -- 172.21.15.131:0/93992485 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) ==== 94+0+5189 (secure 0 0 0) 0x7f33b4079e90 con 0x7f33c406f490 2024-05-26T22:53:59.347 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:53:59.347 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":78,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5533,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/993930297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6835","nonce":993930297},{"type":"v1","addr":"172.21.15.131:6837","nonce":993930297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":73},{"gid":5545,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6834/3480101957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3480101957},{"type":"v1","addr":"172.21.15.187:6834","nonce":3480101957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":73},{"gid":5548,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/567389303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6833","nonce":567389303},{"type":"v1","addr":"172.21.15.187:6835","nonce":567389303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-26T22:53:52.862328+0000","modified":"2024-05-26T22:53:55.926183+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7365},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7365":{"gid":7365,"name":"a","rank":0,"incarnation":74,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6836/3411392567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":3411392567},{"type":"v1","addr":"172.21.15.131:6836","nonce":3411392567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7365,"qdb_cluster":[7365]},"id":6}]} 2024-05-26T22:53:59.348 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 78 2024-05-26T22:53:59.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.349+0000 7f33ca200640 1 -- 172.21.15.131:0/93992485 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f33a006be30 msgr2=0x7f33a006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:59.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.349+0000 7f33ca200640 1 --2- 172.21.15.131:0/93992485 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f33a006be30 0x7f33a006e2f0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f33c41f7c00 tx=0x7f33ac008040 comp rx=0 tx=0).stop 2024-05-26T22:53:59.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.349+0000 7f33ca200640 1 -- 172.21.15.131:0/93992485 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f33c406f490 msgr2=0x7f33c414d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:59.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.349+0000 7f33ca200640 1 --2- 172.21.15.131:0/93992485 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f33c406f490 0x7f33c414d900 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f33b40097a0 tx=0x7f33b40046c0 comp rx=0 tx=0).stop 2024-05-26T22:53:59.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.349+0000 7f33ca200640 1 -- 172.21.15.131:0/93992485 shutdown_connections 2024-05-26T22:53:59.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.349+0000 7f33ca200640 1 --2- 172.21.15.131:0/93992485 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f33c4155840 0x7f33c40ceff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:59.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.349+0000 7f33ca200640 1 --2- 172.21.15.131:0/93992485 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f33a006be30 0x7f33a006e2f0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:59.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.349+0000 7f33ca200640 1 --2- 172.21.15.131:0/93992485 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f33c415c3d0 0x7f33c40c98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:59.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.349+0000 7f33ca200640 1 --2- 172.21.15.131:0/93992485 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f33c406f490 0x7f33c414d900 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:59.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.349+0000 7f33ca200640 1 -- 172.21.15.131:0/93992485 >> 172.21.15.131:0/93992485 conn(0x7f33c40b1e90 msgr2=0x7f33c4152150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:59.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.349+0000 7f33ca200640 1 -- 172.21.15.131:0/93992485 shutdown_connections 2024-05-26T22:53:59.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.349+0000 7f33ca200640 1 -- 172.21.15.131:0/93992485 wait complete. 2024-05-26T22:53:59.770 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 78, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-26T22:53:52.862328+0000', 'modified': '2024-05-26T22:53:55.926183+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7365}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7365': {'gid': 7365, 'name': 'a', 'rank': 0, 'incarnation': 74, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.131:6836/3411392567', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6834', 'nonce': 3411392567}, {'type': 'v1', 'addr': '172.21.15.131:6836', 'nonce': 3411392567}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7365, 'qdb_cluster': [7365]} 2024-05-26T22:53:59.770 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-26T22:53:59.770 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-05-26T22:53:59.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.977+0000 7f677ce00640 1 Processor -- start 2024-05-26T22:53:59.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.978+0000 7f677ce00640 1 -- start start 2024-05-26T22:53:59.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.979+0000 7f677ce00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f67780d0460 0x7f677806d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:59.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.979+0000 7f677ce00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6778160180 con 0x7f677806f420 2024-05-26T22:53:59.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.979+0000 7f677ce00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6778160460 con 0x7f67780b5450 2024-05-26T22:53:59.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.979+0000 7f677ce00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f6778160d90 con 0x7f67780d0460 2024-05-26T22:53:59.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.980+0000 7f6776800640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f67780d0460 0x7f677806d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:59.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.980+0000 7f6776800640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f67780d0460 0x7f677806d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56808/0 (socket says 172.21.15.131:56808) 2024-05-26T22:53:59.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.980+0000 7f6776800640 1 -- 172.21.15.131:0/3270636271 learned_addr learned my addr 172.21.15.131:0/3270636271 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:53:59.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.980+0000 7f676fe00640 1 --1- 172.21.15.131:0/3270636271 >> v1:172.21.15.131:6789/0 conn(0x7f677806f420 0x7f67780cfd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:50670/0 (socket says 172.21.15.131:50670) 2024-05-26T22:53:59.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.980+0000 7f6776800640 1 -- 172.21.15.131:0/3270636271 >> v1:172.21.15.187:6789/0 conn(0x7f67780b5450 legacy=0x7f67780b43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:59.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.980+0000 7f6776800640 1 -- 172.21.15.131:0/3270636271 >> v1:172.21.15.131:6789/0 conn(0x7f677806f420 legacy=0x7f67780cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:59.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.980+0000 7f6776800640 1 -- 172.21.15.131:0/3270636271 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6778160740 con 0x7f67780d0460 2024-05-26T22:53:59.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.980+0000 7f6775400640 1 -- 172.21.15.131:0/3270636271 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 360143942 0 0) 0x7f6778160180 con 0x7f677806f420 2024-05-26T22:53:59.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.982+0000 7f6776800640 1 --2- 172.21.15.131:0/3270636271 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f67780d0460 0x7f677806d600 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f6770009e50 tx=0x7f677002a7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d882b42f7819360 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:59.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.982+0000 7f6775400640 1 -- 172.21.15.131:0/3270636271 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6770036070 con 0x7f67780d0460 2024-05-26T22:53:59.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.982+0000 7f6775400640 1 -- 172.21.15.131:0/3270636271 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6770031600 con 0x7f67780d0460 2024-05-26T22:53:59.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.982+0000 7f6775400640 1 -- 172.21.15.131:0/3270636271 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67700319f0 con 0x7f67780d0460 2024-05-26T22:53:59.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.983+0000 7f677ce00640 1 -- 172.21.15.131:0/3270636271 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f67780d0460 msgr2=0x7f677806d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:59.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.983+0000 7f677ce00640 1 --2- 172.21.15.131:0/3270636271 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f67780d0460 0x7f677806d600 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f6770009e50 tx=0x7f677002a7c0 comp rx=0 tx=0).stop 2024-05-26T22:53:59.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.983+0000 7f677ce00640 1 -- 172.21.15.131:0/3270636271 shutdown_connections 2024-05-26T22:53:59.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.983+0000 7f677ce00640 1 --2- 172.21.15.131:0/3270636271 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f67780d0460 0x7f677806d600 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:59.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.983+0000 7f677ce00640 1 -- 172.21.15.131:0/3270636271 >> 172.21.15.131:0/3270636271 conn(0x7f677806bc90 msgr2=0x7f67780b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:59.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.983+0000 7f677ce00640 1 -- 172.21.15.131:0/3270636271 shutdown_connections 2024-05-26T22:53:59.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.983+0000 7f677ce00640 1 -- 172.21.15.131:0/3270636271 wait complete. 2024-05-26T22:53:59.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.985+0000 7f677ce00640 1 Processor -- start 2024-05-26T22:53:59.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.985+0000 7f677ce00640 1 -- start start 2024-05-26T22:53:59.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.986+0000 7f677ce00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f677806f420 0x7f6778142a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:59.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.986+0000 7f677ce00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f67780b5450 0x7f6778157520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:59.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.986+0000 7f677ce00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f67780d0460 0x7f6778151d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:59.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.986+0000 7f677ce00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f677815fde0 con 0x7f67780b5450 2024-05-26T22:53:59.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.986+0000 7f677ce00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f67780b5830 con 0x7f677806f420 2024-05-26T22:53:59.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.986+0000 7f677ce00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f677815ff60 con 0x7f67780d0460 2024-05-26T22:53:59.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.986+0000 7f676fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f67780b5450 0x7f6778157520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:59.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.986+0000 7f676fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f67780b5450 0x7f6778157520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56820/0 (socket says 172.21.15.131:56820) 2024-05-26T22:53:59.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.986+0000 7f676fe00640 1 -- 172.21.15.131:0/616058155 learned_addr learned my addr 172.21.15.131:0/616058155 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:53:59.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.987+0000 7f6775e00640 1 --2- 172.21.15.131:0/616058155 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f677806f420 0x7f6778142a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:59.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.987+0000 7f676fe00640 1 -- 172.21.15.131:0/616058155 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f67780d0460 msgr2=0x7f6778151d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:59.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.987+0000 7f676fe00640 1 --2- 172.21.15.131:0/616058155 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f67780d0460 0x7f6778151d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:59.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.987+0000 7f676fe00640 1 -- 172.21.15.131:0/616058155 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f677806f420 msgr2=0x7f6778142a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:59.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.987+0000 7f676fe00640 1 --2- 172.21.15.131:0/616058155 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f677806f420 0x7f6778142a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:59.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.987+0000 7f676fe00640 1 -- 172.21.15.131:0/616058155 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6778157c20 con 0x7f67780b5450 2024-05-26T22:53:59.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.987+0000 7f676fe00640 1 --2- 172.21.15.131:0/616058155 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f67780b5450 0x7f6778157520 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f676000c3f0 tx=0x7f676000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:59.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.987+0000 7f676e000640 1 -- 172.21.15.131:0/616058155 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67600074e0 con 0x7f67780b5450 2024-05-26T22:53:59.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.987+0000 7f676e000640 1 -- 172.21.15.131:0/616058155 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67600047e0 con 0x7f67780b5450 2024-05-26T22:53:59.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.987+0000 7f677ce00640 1 -- 172.21.15.131:0/616058155 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67780714e0 con 0x7f67780b5450 2024-05-26T22:53:59.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.988+0000 7f676e000640 1 -- 172.21.15.131:0/616058155 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6760004c10 con 0x7f67780b5450 2024-05-26T22:53:59.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.988+0000 7f677ce00640 1 -- 172.21.15.131:0/616058155 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6778142280 con 0x7f67780b5450 2024-05-26T22:53:59.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.991+0000 7f676e000640 1 -- 172.21.15.131:0/616058155 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6760011070 con 0x7f67780b5450 2024-05-26T22:53:59.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.992+0000 7f676e000640 1 --2- 172.21.15.131:0/616058155 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f675406be30 0x7f675406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:59.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.992+0000 7f676e000640 1 -- 172.21.15.131:0/616058155 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5709+0+0 (secure 0 0 0) 0x7f676008ab90 con 0x7f67780b5450 2024-05-26T22:53:59.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.992+0000 7f677fdb0740 1 Processor -- start 2024-05-26T22:53:59.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.992+0000 7f6775e00640 1 --2- 172.21.15.131:0/616058155 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f675406be30 0x7f675406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:59.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.992+0000 7f677fdb0740 1 -- start start 2024-05-26T22:53:59.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.993+0000 7f6775e00640 1 --2- 172.21.15.131:0/616058155 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f675406be30 0x7f675406e2f0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f6768000f60 tx=0x7f6768009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:59.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.993+0000 7f677fdb0740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a24fcd5c70 0x55a24fc88890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:59.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.993+0000 7f677fdb0740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a24fcb8d90 0x55a24fcb8110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:59.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.993+0000 7f677fdb0740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a24fcb8650 0x55a24fc8d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:53:59.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.993+0000 7f677fdb0740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55a24fbf0610 con 0x55a24fcd5c70 2024-05-26T22:53:59.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.993+0000 7f677fdb0740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55a24fb985c0 con 0x55a24fcb8d90 2024-05-26T22:53:59.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.993+0000 7f677fdb0740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x55a24fbec5d0 con 0x55a24fcb8650 2024-05-26T22:53:59.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.993+0000 7f6775e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a24fcd5c70 0x55a24fc88890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:53:59.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.994+0000 7f6775e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a24fcd5c70 0x55a24fc88890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56828/0 (socket says 172.21.15.131:56828) 2024-05-26T22:53:59.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.994+0000 7f6775e00640 1 -- 172.21.15.131:0/1958313403 learned_addr learned my addr 172.21.15.131:0/1958313403 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:53:59.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.994+0000 7f6775e00640 1 -- 172.21.15.131:0/1958313403 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a24fcb8650 msgr2=0x55a24fc8d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:59.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.994+0000 7f6775e00640 1 --2- 172.21.15.131:0/1958313403 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a24fcb8650 0x55a24fc8d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:59.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.994+0000 7f6775e00640 1 -- 172.21.15.131:0/1958313403 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a24fcb8d90 msgr2=0x55a24fcb8110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:53:59.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.994+0000 7f6775e00640 1 --2- 172.21.15.131:0/1958313403 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a24fcb8d90 0x55a24fcb8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:59.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.994+0000 7f6775e00640 1 -- 172.21.15.131:0/1958313403 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a24f9042d0 con 0x55a24fcd5c70 2024-05-26T22:53:59.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.994+0000 7f6775e00640 1 --2- 172.21.15.131:0/1958313403 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a24fcd5c70 0x55a24fc88890 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f67680052f0 tx=0x7f6768010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=199ccf89a587f7d1 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:53:59.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.994+0000 7f6766a00640 1 -- 172.21.15.131:0/1958313403 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6768019030 con 0x55a24fcd5c70 2024-05-26T22:53:59.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.994+0000 7f6766a00640 1 -- 172.21.15.131:0/1958313403 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6768003820 con 0x55a24fcd5c70 2024-05-26T22:53:59.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.994+0000 7f6766a00640 1 -- 172.21.15.131:0/1958313403 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6768003bd0 con 0x55a24fcd5c70 2024-05-26T22:53:59.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.995+0000 7f677fdb0740 1 -- 172.21.15.131:0/1958313403 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a24fcd5c70 msgr2=0x55a24fc88890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:53:59.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.995+0000 7f677fdb0740 1 --2- 172.21.15.131:0/1958313403 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a24fcd5c70 0x55a24fc88890 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f67680052f0 tx=0x7f6768010000 comp rx=0 tx=0).stop 2024-05-26T22:53:59.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.995+0000 7f677fdb0740 1 -- 172.21.15.131:0/1958313403 shutdown_connections 2024-05-26T22:53:59.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.995+0000 7f677fdb0740 1 --2- 172.21.15.131:0/1958313403 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a24fcb8650 0x55a24fc8d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:59.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.995+0000 7f677fdb0740 1 --2- 172.21.15.131:0/1958313403 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a24fcb8d90 0x55a24fcb8110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:59.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.995+0000 7f677fdb0740 1 --2- 172.21.15.131:0/1958313403 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a24fcd5c70 0x55a24fc88890 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:53:59.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.995+0000 7f677fdb0740 1 -- 172.21.15.131:0/1958313403 >> 172.21.15.131:0/1958313403 conn(0x55a24fc8ab00 msgr2=0x55a24fcd3110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:53:59.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.995+0000 7f677fdb0740 1 -- 172.21.15.131:0/1958313403 shutdown_connections 2024-05-26T22:53:59.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.995+0000 7f677fdb0740 1 -- 172.21.15.131:0/1958313403 wait complete. 2024-05-26T22:53:59.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.996+0000 7f677fdb0740 1 -- start start 2024-05-26T22:53:59.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.996+0000 7f677fdb0740 1 Processor -- start 2024-05-26T22:54:00.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.997+0000 7f677fdb0740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a24fcb8d90 0x55a24fc8df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:00.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.997+0000 7f677fdb0740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a24fcd5c70 0x55a24fc8e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:00.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.997+0000 7f677fdb0740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a24fcb9280 0x55a24fcbb390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:00.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.997+0000 7f677fdb0740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55a24fc8ca20 con 0x55a24fcb8d90 2024-05-26T22:54:00.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.997+0000 7f677fdb0740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55a24fbab910 con 0x55a24fcd5c70 2024-05-26T22:54:00.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.997+0000 7f677fdb0740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x55a24fc8be10 con 0x55a24fcb9280 2024-05-26T22:54:00.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.997+0000 7f677fdb0740 10 client.0 _ll_register_callbacks cb 0x55a24fbe7410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-26T22:54:00.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.998+0000 7f6775e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a24fcb8d90 0x55a24fc8df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:00.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.998+0000 7f6775e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a24fcb8d90 0x55a24fc8df00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56840/0 (socket says 172.21.15.131:56840) 2024-05-26T22:54:00.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.998+0000 7f6775e00640 1 -- 172.21.15.131:0/2837805819 learned_addr learned my addr 172.21.15.131:0/2837805819 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:54:00.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.998+0000 7f6776800640 1 --2- 172.21.15.131:0/2837805819 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a24fcb9280 0x55a24fcbb390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:00.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.998+0000 7f676fe00640 1 --2- 172.21.15.131:0/2837805819 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a24fcd5c70 0x55a24fc8e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:00.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.998+0000 7f6775e00640 1 -- 172.21.15.131:0/2837805819 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a24fcb9280 msgr2=0x55a24fcbb390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:00.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.998+0000 7f6775e00640 1 --2- 172.21.15.131:0/2837805819 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a24fcb9280 0x55a24fcbb390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:00.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.998+0000 7f6775e00640 1 -- 172.21.15.131:0/2837805819 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a24fcd5c70 msgr2=0x55a24fc8e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:00.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.998+0000 7f6775e00640 1 --2- 172.21.15.131:0/2837805819 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a24fcd5c70 0x55a24fc8e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:00.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.998+0000 7f6775e00640 1 -- 172.21.15.131:0/2837805819 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a24f9ce6b0 con 0x55a24fcb8d90 2024-05-26T22:54:00.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.998+0000 7f6776800640 1 --2- 172.21.15.131:0/2837805819 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a24fcb9280 0x55a24fcbb390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:54:00.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.998+0000 7f676fe00640 1 --2- 172.21.15.131:0/2837805819 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a24fcd5c70 0x55a24fc8e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:54:00.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.998+0000 7f6775e00640 1 --2- 172.21.15.131:0/2837805819 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a24fcb8d90 0x55a24fc8df00 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f676800d520 tx=0x7f6768000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d418c1b47a4ac5b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:00.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.998+0000 7f675f400640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-26T22:54:00.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.999+0000 7f675f400640 1 -- 172.21.15.131:0/2837805819 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6768007030 con 0x55a24fcb8d90 2024-05-26T22:54:00.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.999+0000 7f675f400640 1 -- 172.21.15.131:0/2837805819 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 5709+0+0 (secure 0 0 0) 0x7f6768028070 con 0x55a24fcb8d90 2024-05-26T22:54:00.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.999+0000 7f677fdb0740 1 -- 172.21.15.131:0/2837805819 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6768003820 con 0x55a24fcb8d90 2024-05-26T22:54:00.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.999+0000 7f675f400640 1 -- 172.21.15.131:0/2837805819 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) ==== 24+0+0 (secure 0 0 0) 0x7f6768035050 con 0x55a24fcb8d90 2024-05-26T22:54:00.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.999+0000 7f677fdb0740 10 client.7530 fetch_fsmap learned FSMap version 78 2024-05-26T22:54:00.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.999+0000 7f677fdb0740 1 -- 172.21.15.131:0/2837805819 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=2+}) -- 0x55a24fcb9f90 con 0x55a24fcb8d90 2024-05-26T22:54:00.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.999+0000 7f675f400640 1 -- 172.21.15.131:0/2837805819 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6768003820 con 0x55a24fcb8d90 2024-05-26T22:54:00.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.999+0000 7f675f400640 1 -- 172.21.15.131:0/2837805819 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 78) ==== 3494+0+0 (secure 0 0 0) 0x7f6768038020 con 0x55a24fcb8d90 2024-05-26T22:54:00.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.999+0000 7f677fdb0740 10 client.7530 fetch_fsmap finished waiting for FSMap version 78 2024-05-26T22:54:00.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.999+0000 7f677fdb0740 10 client.7530 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-26T22:54:00.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:53:59.999+0000 7f677fdb0740 20 client.7530 populate_metadata read hostname 'smithi131' 2024-05-26T22:54:00.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.000+0000 7f677fdb0740 1 --2- 172.21.15.131:0/2837805819 >> [v2:172.21.15.131:6834/3411392567,v1:172.21.15.131:6836/3411392567] conn(0x55a24fce6ef0 0x55a24fcca850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:00.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.000+0000 7f677fdb0740 4 client.7530 mds_command: new command op to 7365 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-26T22:54:00.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.000+0000 7f677fdb0740 1 -- 172.21.15.131:0/2837805819 --> [v2:172.21.15.131:6834/3411392567,v1:172.21.15.131:6836/3411392567] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a24fbfdd40 con 0x55a24fce6ef0 2024-05-26T22:54:00.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.000+0000 7f676fe00640 1 --2- 172.21.15.131:0/2837805819 >> [v2:172.21.15.131:6834/3411392567,v1:172.21.15.131:6836/3411392567] conn(0x55a24fce6ef0 0x55a24fcca850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:00.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.000+0000 7f676fe00640 1 --2- 172.21.15.131:0/2837805819 >> [v2:172.21.15.131:6834/3411392567,v1:172.21.15.131:6836/3411392567] conn(0x55a24fce6ef0 0x55a24fcca850 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e5ef958210d205c server_cookie=1bf196af33f1bb8c in_seq=0 out_seq=0 2024-05-26T22:54:00.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.000+0000 7f675f400640 10 client.7530 ms_handle_connect on v2:172.21.15.131:6834/3411392567 2024-05-26T22:54:00.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.001+0000 7f675f400640 1 -- 172.21.15.131:0/2837805819 <== mds.0 v2:172.21.15.131:6834/3411392567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a24fbfdd40 con 0x55a24fce6ef0 2024-05-26T22:54:00.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.001+0000 7f675f400640 10 client.7530 handle_command_reply: tid=0 2024-05-26T22:54:00.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.002+0000 7f677fdb0740 1 client.7530 shutdown 2024-05-26T22:54:00.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.002+0000 7f677fdb0740 1 -- 172.21.15.131:0/2837805819 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a24fcb8d90 msgr2=0x55a24fc8df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:00.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.002+0000 7f677fdb0740 1 --2- 172.21.15.131:0/2837805819 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a24fcb8d90 0x55a24fc8df00 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f676800d520 tx=0x7f6768000ed0 comp rx=0 tx=0).stop 2024-05-26T22:54:00.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.002+0000 7f677fdb0740 1 -- 172.21.15.131:0/2837805819 shutdown_connections 2024-05-26T22:54:00.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.002+0000 7f677fdb0740 1 --2- 172.21.15.131:0/2837805819 >> [v2:172.21.15.131:6834/3411392567,v1:172.21.15.131:6836/3411392567] conn(0x55a24fce6ef0 0x55a24fcca850 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:00.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.002+0000 7f677fdb0740 1 --2- 172.21.15.131:0/2837805819 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a24fcb9280 0x55a24fcbb390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:00.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.002+0000 7f677fdb0740 1 --2- 172.21.15.131:0/2837805819 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a24fcd5c70 0x55a24fc8e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:00.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.002+0000 7f677fdb0740 1 --2- 172.21.15.131:0/2837805819 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a24fcb8d90 0x55a24fc8df00 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:00.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.002+0000 7f675f400640 0 client.7530 ms_handle_reset on v2:172.21.15.131:6834/3411392567 2024-05-26T22:54:00.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.002+0000 7f677fdb0740 1 -- 172.21.15.131:0/2837805819 >> 172.21.15.131:0/2837805819 conn(0x55a24fcb8650 msgr2=0x55a24fcd3110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:00.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.002+0000 7f677fdb0740 1 -- 172.21.15.131:0/2837805819 shutdown_connections 2024-05-26T22:54:00.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.002+0000 7f677fdb0740 1 -- 172.21.15.131:0/2837805819 wait complete. 2024-05-26T22:54:00.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.002+0000 7f677fdb0740 20 client.7530 trim_cache size 0 max 16384 2024-05-26T22:54:00.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.023+0000 7f677fdb0740 1 Processor -- start 2024-05-26T22:54:00.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.023+0000 7f677fdb0740 1 -- start start 2024-05-26T22:54:00.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.024+0000 7f677fdb0740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a24fcb8d90 0x55a24fccac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:00.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.024+0000 7f677fdb0740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a24fcb9280 0x55a24fcb36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:00.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.024+0000 7f677fdb0740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a24fcd5c70 0x55a24fcb3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:00.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.024+0000 7f677fdb0740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f67781ff990 con 0x55a24fcb9280 2024-05-26T22:54:00.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.024+0000 7f677fdb0740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6778144dd0 con 0x55a24fcd5c70 2024-05-26T22:54:00.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.024+0000 7f677fdb0740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f6778145420 con 0x55a24fcb8d90 2024-05-26T22:54:00.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.024+0000 7f676fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a24fcb9280 0x55a24fcb36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:00.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.024+0000 7f676fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a24fcb9280 0x55a24fcb36a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56850/0 (socket says 172.21.15.131:56850) 2024-05-26T22:54:00.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.024+0000 7f676fe00640 1 -- 172.21.15.131:0/1093712896 learned_addr learned my addr 172.21.15.131:0/1093712896 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:54:00.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.024+0000 7f6775e00640 1 --2- 172.21.15.131:0/1093712896 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a24fcb8d90 0x55a24fccac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:00.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.024+0000 7f676fe00640 1 -- 172.21.15.131:0/1093712896 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a24fcb8d90 msgr2=0x55a24fccac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:00.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.024+0000 7f6776800640 1 --2- 172.21.15.131:0/1093712896 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a24fcd5c70 0x55a24fcb3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:00.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.024+0000 7f676fe00640 1 --2- 172.21.15.131:0/1093712896 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a24fcb8d90 0x55a24fccac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:00.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.025+0000 7f676fe00640 1 -- 172.21.15.131:0/1093712896 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a24fcd5c70 msgr2=0x55a24fcb3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:00.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.025+0000 7f676fe00640 1 --2- 172.21.15.131:0/1093712896 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a24fcd5c70 0x55a24fcb3be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:00.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.025+0000 7f676fe00640 1 -- 172.21.15.131:0/1093712896 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a24fcb43b0 con 0x55a24fcb9280 2024-05-26T22:54:00.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.025+0000 7f676fe00640 1 --2- 172.21.15.131:0/1093712896 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a24fcb9280 0x55a24fcb36a0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f6760007b60 tx=0x7f6760007af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca4a227fe918d8fd server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:00.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.025+0000 7f6766000640 1 -- 172.21.15.131:0/1093712896 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6760075070 con 0x55a24fcb9280 2024-05-26T22:54:00.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.025+0000 7f6766000640 1 -- 172.21.15.131:0/1093712896 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6760061980 con 0x55a24fcb9280 2024-05-26T22:54:00.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.025+0000 7f6766000640 1 -- 172.21.15.131:0/1093712896 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6760057010 con 0x55a24fcb9280 2024-05-26T22:54:00.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.026+0000 7f677fdb0740 1 -- 172.21.15.131:0/1093712896 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a24fcb9280 msgr2=0x55a24fcb36a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:00.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.026+0000 7f677fdb0740 1 --2- 172.21.15.131:0/1093712896 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a24fcb9280 0x55a24fcb36a0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f6760007b60 tx=0x7f6760007af0 comp rx=0 tx=0).stop 2024-05-26T22:54:00.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.026+0000 7f677fdb0740 1 -- 172.21.15.131:0/1093712896 shutdown_connections 2024-05-26T22:54:00.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.026+0000 7f677fdb0740 1 --2- 172.21.15.131:0/1093712896 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a24fcd5c70 0x55a24fcb3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:00.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.026+0000 7f677fdb0740 1 --2- 172.21.15.131:0/1093712896 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a24fcb9280 0x55a24fcb36a0 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:00.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.026+0000 7f677fdb0740 1 --2- 172.21.15.131:0/1093712896 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a24fcb8d90 0x55a24fccac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:00.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.026+0000 7f677fdb0740 1 -- 172.21.15.131:0/1093712896 >> 172.21.15.131:0/1093712896 conn(0x55a24fcb8650 msgr2=0x55a24fc8f310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:00.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.026+0000 7f677fdb0740 1 -- 172.21.15.131:0/1093712896 shutdown_connections 2024-05-26T22:54:00.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.026+0000 7f677fdb0740 1 -- 172.21.15.131:0/1093712896 wait complete. 2024-05-26T22:54:00.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.027+0000 7f677fdb0740 1 -- start start 2024-05-26T22:54:00.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.027+0000 7f677fdb0740 1 Processor -- start 2024-05-26T22:54:00.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.028+0000 7f677fdb0740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a24fcb8d90 0x55a24fcd1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:00.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.028+0000 7f677fdb0740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a24fcb9280 0x55a24fcd2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:00.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.028+0000 7f677fdb0740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a24fcd5c70 0x55a24fcd3500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:00.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.028+0000 7f677fdb0740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55a24fce6d70 con 0x55a24fcb8d90 2024-05-26T22:54:00.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.028+0000 7f677fdb0740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55a24fcba270 con 0x55a24fcb9280 2024-05-26T22:54:00.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.028+0000 7f677fdb0740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x55a24fce7360 con 0x55a24fcd5c70 2024-05-26T22:54:00.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.028+0000 7f677fdb0740 10 client.0 _ll_register_callbacks cb 0x7f6768005160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-26T22:54:00.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.028+0000 7f6775e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a24fcb8d90 0x55a24fcd1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:00.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.028+0000 7f6775e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a24fcb8d90 0x55a24fcd1b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56854/0 (socket says 172.21.15.131:56854) 2024-05-26T22:54:00.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.028+0000 7f6775e00640 1 -- 172.21.15.131:0/1416764162 learned_addr learned my addr 172.21.15.131:0/1416764162 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:54:00.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.028+0000 7f6775e00640 1 -- 172.21.15.131:0/1416764162 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a24fcd5c70 msgr2=0x55a24fcd3500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:00.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.028+0000 7f6775e00640 1 --2- 172.21.15.131:0/1416764162 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a24fcd5c70 0x55a24fcd3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:00.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.028+0000 7f6775e00640 1 -- 172.21.15.131:0/1416764162 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a24fcb9280 msgr2=0x55a24fcd2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:00.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.028+0000 7f6775e00640 1 --2- 172.21.15.131:0/1416764162 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a24fcb9280 0x55a24fcd2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:00.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.028+0000 7f6775e00640 1 -- 172.21.15.131:0/1416764162 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a24fc8e1c0 con 0x55a24fcb8d90 2024-05-26T22:54:00.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.029+0000 7f6775e00640 1 --2- 172.21.15.131:0/1416764162 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a24fcb8d90 0x55a24fcd1b20 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f6768003cd0 tx=0x7f6768013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb1915f50b8018a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:00.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.029+0000 7f675f400640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-26T22:54:00.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.029+0000 7f675f400640 1 -- 172.21.15.131:0/1416764162 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f676803e060 con 0x55a24fcb8d90 2024-05-26T22:54:00.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.029+0000 7f675f400640 1 -- 172.21.15.131:0/1416764162 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 5709+0+0 (secure 0 0 0) 0x7f676801d070 con 0x55a24fcb8d90 2024-05-26T22:54:00.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.029+0000 7f677fdb0740 1 -- 172.21.15.131:0/1416764162 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6760061980 con 0x55a24fcb8d90 2024-05-26T22:54:00.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.029+0000 7f675f400640 1 -- 172.21.15.131:0/1416764162 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) ==== 24+0+0 (secure 0 0 0) 0x7f6768019030 con 0x55a24fcb8d90 2024-05-26T22:54:00.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.029+0000 7f677fdb0740 10 client.7536 fetch_fsmap learned FSMap version 78 2024-05-26T22:54:00.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.029+0000 7f677fdb0740 1 -- 172.21.15.131:0/1416764162 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=2+}) -- 0x55a24fcd1600 con 0x55a24fcb8d90 2024-05-26T22:54:00.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.029+0000 7f675f400640 1 -- 172.21.15.131:0/1416764162 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6760061980 con 0x55a24fcb8d90 2024-05-26T22:54:00.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.030+0000 7f675f400640 1 -- 172.21.15.131:0/1416764162 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 78) ==== 3494+0+0 (secure 0 0 0) 0x7f6768038070 con 0x55a24fcb8d90 2024-05-26T22:54:00.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.030+0000 7f677fdb0740 10 client.7536 fetch_fsmap finished waiting for FSMap version 78 2024-05-26T22:54:00.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.030+0000 7f677fdb0740 10 client.7536 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-26T22:54:00.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.030+0000 7f677fdb0740 20 client.7536 populate_metadata read hostname 'smithi131' 2024-05-26T22:54:00.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.030+0000 7f677fdb0740 1 --2- 172.21.15.131:0/1416764162 >> [v2:172.21.15.131:6834/3411392567,v1:172.21.15.131:6836/3411392567] conn(0x55a24fc8d560 0x55a24fce1810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:00.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.030+0000 7f677fdb0740 4 client.7536 mds_command: new command op to 7365 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-26T22:54:00.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.030+0000 7f677fdb0740 1 -- 172.21.15.131:0/1416764162 --> [v2:172.21.15.131:6834/3411392567,v1:172.21.15.131:6836/3411392567] -- command(tid 0: {"prefix": "status"}) -- 0x55a24fbf8090 con 0x55a24fc8d560 2024-05-26T22:54:00.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.030+0000 7f676fe00640 1 --2- 172.21.15.131:0/1416764162 >> [v2:172.21.15.131:6834/3411392567,v1:172.21.15.131:6836/3411392567] conn(0x55a24fc8d560 0x55a24fce1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:00.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.030+0000 7f676fe00640 1 --2- 172.21.15.131:0/1416764162 >> [v2:172.21.15.131:6834/3411392567,v1:172.21.15.131:6836/3411392567] conn(0x55a24fc8d560 0x55a24fce1810 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2df57fc8695b3be6 server_cookie=df445550816d989 in_seq=0 out_seq=0 2024-05-26T22:54:00.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.030+0000 7f675f400640 10 client.7536 ms_handle_connect on v2:172.21.15.131:6834/3411392567 2024-05-26T22:54:00.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.031+0000 7f675f400640 1 -- 172.21.15.131:0/1416764162 <== mds.0 v2:172.21.15.131:6834/3411392567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x55a24fbf8090 con 0x55a24fc8d560 2024-05-26T22:54:00.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.031+0000 7f675f400640 10 client.7536 handle_command_reply: tid=0 2024-05-26T22:54:00.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.031+0000 7f677fdb0740 1 client.7536 shutdown 2024-05-26T22:54:00.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.031+0000 7f677fdb0740 1 -- 172.21.15.131:0/1416764162 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a24fcb8d90 msgr2=0x55a24fcd1b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:00.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.031+0000 7f677fdb0740 1 --2- 172.21.15.131:0/1416764162 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a24fcb8d90 0x55a24fcd1b20 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f6768003cd0 tx=0x7f6768013000 comp rx=0 tx=0).stop 2024-05-26T22:54:00.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.031+0000 7f677fdb0740 1 -- 172.21.15.131:0/1416764162 shutdown_connections 2024-05-26T22:54:00.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.031+0000 7f677fdb0740 1 --2- 172.21.15.131:0/1416764162 >> [v2:172.21.15.131:6834/3411392567,v1:172.21.15.131:6836/3411392567] conn(0x55a24fc8d560 0x55a24fce1810 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:00.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.031+0000 7f677fdb0740 1 --2- 172.21.15.131:0/1416764162 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a24fcd5c70 0x55a24fcd3500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:00.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.031+0000 7f677fdb0740 1 --2- 172.21.15.131:0/1416764162 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a24fcb9280 0x55a24fcd2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:00.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.031+0000 7f677fdb0740 1 --2- 172.21.15.131:0/1416764162 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a24fcb8d90 0x55a24fcd1b20 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:00.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.031+0000 7f675f400640 0 client.7536 ms_handle_reset on v2:172.21.15.131:6834/3411392567 2024-05-26T22:54:00.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.031+0000 7f677fdb0740 1 -- 172.21.15.131:0/1416764162 >> 172.21.15.131:0/1416764162 conn(0x55a24fcb8650 msgr2=0x55a24fccbb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:00.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.031+0000 7f677fdb0740 1 -- 172.21.15.131:0/1416764162 shutdown_connections 2024-05-26T22:54:00.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.031+0000 7f677fdb0740 1 -- 172.21.15.131:0/1416764162 wait complete. 2024-05-26T22:54:00.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.031+0000 7f677fdb0740 20 client.7536 trim_cache size 0 max 16384 2024-05-26T22:54:00.035 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-26T22:54:00.035 INFO:teuthology.orchestra.run.smithi131.stdout: "cluster_fsid": "46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f", 2024-05-26T22:54:00.035 INFO:teuthology.orchestra.run.smithi131.stdout: "whoami": 0, 2024-05-26T22:54:00.035 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 7365, 2024-05-26T22:54:00.035 INFO:teuthology.orchestra.run.smithi131.stdout: "want_state": "up:active", 2024-05-26T22:54:00.036 INFO:teuthology.orchestra.run.smithi131.stdout: "state": "up:active", 2024-05-26T22:54:00.036 INFO:teuthology.orchestra.run.smithi131.stdout: "fs_name": "cephfs", 2024-05-26T22:54:00.036 INFO:teuthology.orchestra.run.smithi131.stdout: "rank_uptime": 6.5989852950000003, 2024-05-26T22:54:00.036 INFO:teuthology.orchestra.run.smithi131.stdout: "mdsmap_epoch": 78, 2024-05-26T22:54:00.036 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch": 82, 2024-05-26T22:54:00.036 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch_barrier": 82, 2024-05-26T22:54:00.036 INFO:teuthology.orchestra.run.smithi131.stdout: "uptime": 12.037255128 2024-05-26T22:54:00.036 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-26T22:54:00.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.032+0000 7f677ce00640 1 -- 172.21.15.131:0/616058155 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f675406be30 msgr2=0x7f675406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:00.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.032+0000 7f677ce00640 1 --2- 172.21.15.131:0/616058155 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f675406be30 0x7f675406e2f0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f6768000f60 tx=0x7f6768009040 comp rx=0 tx=0).stop 2024-05-26T22:54:00.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.032+0000 7f677ce00640 1 -- 172.21.15.131:0/616058155 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f67780b5450 msgr2=0x7f6778157520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:00.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.032+0000 7f677ce00640 1 --2- 172.21.15.131:0/616058155 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f67780b5450 0x7f6778157520 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f676000c3f0 tx=0x7f676000c8b0 comp rx=0 tx=0).stop 2024-05-26T22:54:00.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.032+0000 7f677ce00640 1 -- 172.21.15.131:0/616058155 shutdown_connections 2024-05-26T22:54:00.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.032+0000 7f677ce00640 1 --2- 172.21.15.131:0/616058155 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f67780d0460 0x7f6778151d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:00.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.032+0000 7f677ce00640 1 --2- 172.21.15.131:0/616058155 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f675406be30 0x7f675406e2f0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:00.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.032+0000 7f677ce00640 1 --2- 172.21.15.131:0/616058155 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f67780b5450 0x7f6778157520 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:00.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.032+0000 7f677ce00640 1 --2- 172.21.15.131:0/616058155 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f677806f420 0x7f6778142a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:00.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.032+0000 7f677ce00640 1 -- 172.21.15.131:0/616058155 >> 172.21.15.131:0/616058155 conn(0x7f677806bc90 msgr2=0x7f677815a4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:00.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.032+0000 7f677ce00640 1 -- 172.21.15.131:0/616058155 shutdown_connections 2024-05-26T22:54:00.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.032+0000 7f677ce00640 1 -- 172.21.15.131:0/616058155 wait complete. 2024-05-26T22:54:00.051 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-26T22:54:00.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.251+0000 7f8cddc00640 1 Processor -- start 2024-05-26T22:54:00.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.251+0000 7f8cddc00640 1 -- start start 2024-05-26T22:54:00.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.252+0000 7f8cddc00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8cd8071500 0x7f8cd80ce2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:00.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.253+0000 7f8cddc00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8cd80ceb40 con 0x7f8cd8150f70 2024-05-26T22:54:00.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.253+0000 7f8cddc00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8cd80cee20 con 0x7f8cd815e320 2024-05-26T22:54:00.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.253+0000 7f8cddc00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f8cd80cfbe0 con 0x7f8cd8071500 2024-05-26T22:54:00.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.253+0000 7f8cdd200640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f8cd8150f70 0x7f8cd8151420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:50678/0 (socket says 172.21.15.131:50678) 2024-05-26T22:54:00.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.253+0000 7f8cdd200640 1 -- 172.21.15.131:0/2591210436 learned_addr learned my addr 172.21.15.131:0/2591210436 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:54:00.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.253+0000 7f8ccf400640 1 --2- 172.21.15.131:0/2591210436 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8cd8071500 0x7f8cd80ce2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:00.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.253+0000 7f8cd7400640 1 -- 172.21.15.131:0/2591210436 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2198026568 0 0) 0x7f8cd80ceb40 con 0x7f8cd8150f70 2024-05-26T22:54:00.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.253+0000 7f8cd7400640 1 -- 172.21.15.131:0/2591210436 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8cbc003520 con 0x7f8cd8150f70 2024-05-26T22:54:00.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.254+0000 7f8cd7400640 1 -- 172.21.15.131:0/2591210436 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3621621853 0 0) 0x7f8cbc003520 con 0x7f8cd8150f70 2024-05-26T22:54:00.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.254+0000 7f8cd7400640 1 -- 172.21.15.131:0/2591210436 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8cd8071500 msgr2=0x7f8cd80ce2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:00.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.254+0000 7f8cd7400640 1 --2- 172.21.15.131:0/2591210436 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8cd8071500 0x7f8cd80ce2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:00.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.254+0000 7f8cd7400640 1 -- 172.21.15.131:0/2591210436 >> v1:172.21.15.187:6789/0 conn(0x7f8cd815e320 legacy=0x7f8cd815e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:00.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.254+0000 7f8cd7400640 1 -- 172.21.15.131:0/2591210436 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cd80cf100 con 0x7f8cd8150f70 2024-05-26T22:54:00.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.254+0000 7f8cd7400640 1 -- 172.21.15.131:0/2591210436 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7f8cd0001670 con 0x7f8cd8150f70 2024-05-26T22:54:00.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.254+0000 7f8cd7400640 1 -- 172.21.15.131:0/2591210436 >> v1:172.21.15.131:6789/0 conn(0x7f8cd8150f70 legacy=0x7f8cd8151420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:00.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.254+0000 7f8cd7400640 1 --2- 172.21.15.131:0/2591210436 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8cd8071500 0x7f8cbc0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:00.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.254+0000 7f8cd7400640 1 --2- 172.21.15.131:0/2591210436 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8cbc003fb0 0x7f8cbc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:00.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.254+0000 7f8cd7400640 1 --2- 172.21.15.131:0/2591210436 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8cbc004d40 0x7f8cbc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:00.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.254+0000 7f8cd7400640 1 -- 172.21.15.131:0/2591210436 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd80cfbe0 con 0x7f8cbc003fb0 2024-05-26T22:54:00.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.254+0000 7f8cd7400640 1 -- 172.21.15.131:0/2591210436 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8cbc00bb00 con 0x7f8cbc004d40 2024-05-26T22:54:00.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.254+0000 7f8cd7400640 1 -- 172.21.15.131:0/2591210436 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f8cbc00df10 con 0x7f8cd8071500 2024-05-26T22:54:00.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.254+0000 7f8ccf400640 1 --2- 172.21.15.131:0/2591210436 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8cbc003fb0 0x7f8cbc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:00.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.255+0000 7f8cd7e00640 1 --2- 172.21.15.131:0/2591210436 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8cd8071500 0x7f8cbc0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:00.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.255+0000 7f8ccf400640 1 -- 172.21.15.131:0/2591210436 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8cd8071500 msgr2=0x7f8cbc0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:00.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.255+0000 7f8ccf400640 1 --2- 172.21.15.131:0/2591210436 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8cd8071500 0x7f8cbc0039e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:00.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.255+0000 7f8ccf400640 1 -- 172.21.15.131:0/2591210436 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8cbc004d40 msgr2=0x7f8cbc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:00.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.255+0000 7f8ccf400640 1 --2- 172.21.15.131:0/2591210436 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8cbc004d40 0x7f8cbc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:00.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.255+0000 7f8ccf400640 1 -- 172.21.15.131:0/2591210436 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cbc003520 con 0x7f8cbc003fb0 2024-05-26T22:54:00.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.256+0000 7f8ccf400640 1 --2- 172.21.15.131:0/2591210436 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8cbc003fb0 0x7f8cbc004800 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f8cc000aa10 tx=0x7f8cbc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ada689ac6ab5411 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:00.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.256+0000 7f8cd7400640 1 -- 172.21.15.131:0/2591210436 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc0036070 con 0x7f8cbc003fb0 2024-05-26T22:54:00.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.256+0000 7f8cd7400640 1 -- 172.21.15.131:0/2591210436 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8cc0031600 con 0x7f8cbc003fb0 2024-05-26T22:54:00.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.256+0000 7f8cd7400640 1 -- 172.21.15.131:0/2591210436 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc000b040 con 0x7f8cbc003fb0 2024-05-26T22:54:00.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.256+0000 7f8cddc00640 1 -- 172.21.15.131:0/2591210436 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8cbc003fb0 msgr2=0x7f8cbc004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:00.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.256+0000 7f8cddc00640 1 --2- 172.21.15.131:0/2591210436 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8cbc003fb0 0x7f8cbc004800 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f8cc000aa10 tx=0x7f8cbc005830 comp rx=0 tx=0).stop 2024-05-26T22:54:00.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.257+0000 7f8cdd200640 1 -- 172.21.15.131:0/2591210436 reap_dead start 2024-05-26T22:54:00.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.257+0000 7f8cddc00640 1 -- 172.21.15.131:0/2591210436 shutdown_connections 2024-05-26T22:54:00.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.257+0000 7f8cddc00640 1 -- 172.21.15.131:0/2591210436 >> 172.21.15.131:0/2591210436 conn(0x7f8cd80cff40 msgr2=0x7f8cd80d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:00.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.257+0000 7f8cddc00640 1 -- 172.21.15.131:0/2591210436 shutdown_connections 2024-05-26T22:54:00.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.257+0000 7f8cddc00640 1 -- 172.21.15.131:0/2591210436 wait complete. 2024-05-26T22:54:00.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.258+0000 7f8cddc00640 1 Processor -- start 2024-05-26T22:54:00.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.259+0000 7f8cddc00640 1 -- start start 2024-05-26T22:54:00.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.260+0000 7f8cddc00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8cbc003fb0 0x7f8cd8142900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:00.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.260+0000 7f8cddc00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8cd8152fb0 0x7f8cd806d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:00.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.260+0000 7f8cddc00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8cd80cdc60 0x7f8cd80ce110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:00.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.260+0000 7f8cddc00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd8151b30 con 0x7f8cd8152fb0 2024-05-26T22:54:00.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.260+0000 7f8cddc00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd80cf6c0 con 0x7f8cd80cdc60 2024-05-26T22:54:00.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.260+0000 7f8cddc00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f8cd8151cb0 con 0x7f8cbc003fb0 2024-05-26T22:54:00.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.260+0000 7f8ccf400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8cd8152fb0 0x7f8cd806d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:00.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.260+0000 7f8ccf400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8cd8152fb0 0x7f8cd806d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56866/0 (socket says 172.21.15.131:56866) 2024-05-26T22:54:00.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.260+0000 7f8ccf400640 1 -- 172.21.15.131:0/74802644 learned_addr learned my addr 172.21.15.131:0/74802644 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:54:00.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.260+0000 7f8cd7e00640 1 --2- 172.21.15.131:0/74802644 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8cbc003fb0 0x7f8cd8142900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:00.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.260+0000 7f8cdd200640 1 --2- 172.21.15.131:0/74802644 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8cd80cdc60 0x7f8cd80ce110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:00.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.260+0000 7f8ccf400640 1 -- 172.21.15.131:0/74802644 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8cbc003fb0 msgr2=0x7f8cd8142900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:00.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.260+0000 7f8ccf400640 1 --2- 172.21.15.131:0/74802644 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8cbc003fb0 0x7f8cd8142900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:00.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.260+0000 7f8ccf400640 1 -- 172.21.15.131:0/74802644 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8cd80cdc60 msgr2=0x7f8cd80ce110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:00.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.260+0000 7f8ccf400640 1 --2- 172.21.15.131:0/74802644 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8cd80cdc60 0x7f8cd80ce110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:00.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.260+0000 7f8ccf400640 1 -- 172.21.15.131:0/74802644 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cd81534b0 con 0x7f8cd8152fb0 2024-05-26T22:54:00.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.260+0000 7f8cd7e00640 1 --2- 172.21.15.131:0/74802644 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8cbc003fb0 0x7f8cd8142900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:54:00.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.261+0000 7f8ccf400640 1 --2- 172.21.15.131:0/74802644 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8cd8152fb0 0x7f8cd806d610 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f8cc000aa10 tx=0x7f8cc0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:00.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.261+0000 7f8cd4c00640 1 -- 172.21.15.131:0/74802644 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc0040020 con 0x7f8cd8152fb0 2024-05-26T22:54:00.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.261+0000 7f8cd4c00640 1 -- 172.21.15.131:0/74802644 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8cc003b670 con 0x7f8cd8152fb0 2024-05-26T22:54:00.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.261+0000 7f8cd4c00640 1 -- 172.21.15.131:0/74802644 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc0036040 con 0x7f8cd8152fb0 2024-05-26T22:54:00.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.261+0000 7f8cddc00640 1 -- 172.21.15.131:0/74802644 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cd8153790 con 0x7f8cd8152fb0 2024-05-26T22:54:00.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.261+0000 7f8cddc00640 1 -- 172.21.15.131:0/74802644 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cd8153d50 con 0x7f8cd8152fb0 2024-05-26T22:54:00.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.262+0000 7f8cddc00640 1 -- 172.21.15.131:0/74802644 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8cd80694e0 con 0x7f8cd8152fb0 2024-05-26T22:54:00.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.263+0000 7f8cd4c00640 1 -- 172.21.15.131:0/74802644 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8cc0043020 con 0x7f8cd8152fb0 2024-05-26T22:54:00.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.264+0000 7f8cd4c00640 1 --2- 172.21.15.131:0/74802644 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f8cb406be30 0x7f8cb406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:00.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.264+0000 7f8cd4c00640 1 -- 172.21.15.131:0/74802644 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5709+0+0 (secure 0 0 0) 0x7f8cc00ab760 con 0x7f8cd8152fb0 2024-05-26T22:54:00.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.264+0000 7f8cd7e00640 1 --2- 172.21.15.131:0/74802644 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f8cb406be30 0x7f8cb406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:00.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.265+0000 7f8cd7e00640 1 --2- 172.21.15.131:0/74802644 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f8cb406be30 0x7f8cb406e2f0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f8cd81f7810 tx=0x7f8cc8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:00.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.269+0000 7f8cd4c00640 1 -- 172.21.15.131:0/74802644 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f8cc004a030 con 0x7f8cd8152fb0 2024-05-26T22:54:00.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.513+0000 7f8cddc00640 1 -- 172.21.15.131:0/74802644 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f8cd80b54a0 con 0x7f8cd8152fb0 2024-05-26T22:54:00.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.514+0000 7f8cd4c00640 1 -- 172.21.15.131:0/74802644 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v78) ==== 53+0+73 (secure 0 0 0) 0x7f8cc007a1a0 con 0x7f8cd8152fb0 2024-05-26T22:54:00.514 INFO:teuthology.orchestra.run.smithi131.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-26T22:54:00.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.517+0000 7f8cddc00640 1 -- 172.21.15.131:0/74802644 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f8cb406be30 msgr2=0x7f8cb406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:00.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.517+0000 7f8cddc00640 1 --2- 172.21.15.131:0/74802644 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f8cb406be30 0x7f8cb406e2f0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f8cd81f7810 tx=0x7f8cc8009040 comp rx=0 tx=0).stop 2024-05-26T22:54:00.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.517+0000 7f8cddc00640 1 -- 172.21.15.131:0/74802644 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8cd8152fb0 msgr2=0x7f8cd806d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:00.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.517+0000 7f8cddc00640 1 --2- 172.21.15.131:0/74802644 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8cd8152fb0 0x7f8cd806d610 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f8cc000aa10 tx=0x7f8cc0004730 comp rx=0 tx=0).stop 2024-05-26T22:54:00.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.517+0000 7f8cddc00640 1 -- 172.21.15.131:0/74802644 shutdown_connections 2024-05-26T22:54:00.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.517+0000 7f8cddc00640 1 --2- 172.21.15.131:0/74802644 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f8cb406be30 0x7f8cb406e2f0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:00.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.517+0000 7f8cddc00640 1 --2- 172.21.15.131:0/74802644 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8cd80cdc60 0x7f8cd80ce110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:00.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.517+0000 7f8cddc00640 1 --2- 172.21.15.131:0/74802644 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8cd8152fb0 0x7f8cd806d610 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:00.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.517+0000 7f8cddc00640 1 --2- 172.21.15.131:0/74802644 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8cbc003fb0 0x7f8cd8142900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:00.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.517+0000 7f8cddc00640 1 -- 172.21.15.131:0/74802644 >> 172.21.15.131:0/74802644 conn(0x7f8cd80cff40 msgr2=0x7f8cd8150740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:00.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.517+0000 7f8cddc00640 1 -- 172.21.15.131:0/74802644 shutdown_connections 2024-05-26T22:54:00.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00.517+0000 7f8cddc00640 1 -- 172.21.15.131:0/74802644 wait complete. 2024-05-26T22:54:00.536 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-26T22:54:00.536 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-26T22:54:00.536 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi131.front.sepia.ceph.com 2024-05-26T22:54:00.536 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-26T22:54:00.536 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-26T22:54:00.537 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-26T22:54:00.537 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-26T22:54:00.537 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-26T22:54:00.537 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-26T22:54:00.537 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:54:00.537 DEBUG:teuthology.orchestra.run.smithi131:> ip addr 2024-05-26T22:54:00.588 INFO:teuthology.orchestra.run.smithi131.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-26T22:54:00.589 INFO:teuthology.orchestra.run.smithi131.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-26T22:54:00.589 INFO:teuthology.orchestra.run.smithi131.stdout: inet 127.0.0.1/8 scope host lo 2024-05-26T22:54:00.589 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-05-26T22:54:00.589 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 ::1/128 scope host 2024-05-26T22:54:00.589 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-05-26T22:54:00.589 INFO:teuthology.orchestra.run.smithi131.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-26T22:54:00.589 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:d8:fb:18 brd ff:ff:ff:ff:ff:ff 2024-05-26T22:54:00.589 INFO:teuthology.orchestra.run.smithi131.stdout: altname enp7s0f0 2024-05-26T22:54:00.589 INFO:teuthology.orchestra.run.smithi131.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-26T22:54:00.589 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:d8:fb:19 brd ff:ff:ff:ff:ff:ff 2024-05-26T22:54:00.589 INFO:teuthology.orchestra.run.smithi131.stdout: altname enp7s0f1 2024-05-26T22:54:00.589 INFO:teuthology.orchestra.run.smithi131.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-26T22:54:00.589 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:88:72:f8 brd ff:ff:ff:ff:ff:ff 2024-05-26T22:54:00.590 INFO:teuthology.orchestra.run.smithi131.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-26T22:54:00.590 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:88:72:f9 brd ff:ff:ff:ff:ff:ff 2024-05-26T22:54:00.590 INFO:teuthology.orchestra.run.smithi131.stdout: inet 172.21.15.131/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-26T22:54:00.590 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft 42257sec preferred_lft 42257sec 2024-05-26T22:54:00.590 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::ec4:7aff:fe88:72f9/64 scope link 2024-05-26T22:54:00.590 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-05-26T22:54:00.590 INFO:teuthology.orchestra.run.smithi131.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-26T22:54:00.590 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 06:f4:05:86:a4:3e brd ff:ff:ff:ff:ff:ff 2024-05-26T22:54:00.590 INFO:teuthology.orchestra.run.smithi131.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-26T22:54:00.590 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-05-26T22:54:00.590 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::8802:8fff:fe41:fc6d/64 scope link 2024-05-26T22:54:00.590 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-05-26T22:54:00.590 INFO:teuthology.orchestra.run.smithi131.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-26T22:54:00.591 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 06:f4:05:86:a4:3e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-26T22:54:00.591 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::4f4:5ff:fe86:a43e/64 scope link 2024-05-26T22:54:00.591 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-05-26T22:54:00.591 INFO:teuthology.orchestra.run.smithi131.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-26T22:54:00.591 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether ee:6f:35:b2:c0:5b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-26T22:54:00.591 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::ec6f:35ff:feb2:c05b/64 scope link 2024-05-26T22:54:00.591 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-05-26T22:54:00.592 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:54:00.592 DEBUG:teuthology.orchestra.run.smithi131:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-26T22:54:00.665 INFO:teuthology.orchestra.run.smithi131.stdout:1 2024-05-26T22:54:00.669 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:54:00.669 DEBUG:teuthology.orchestra.run.smithi131:> route 2024-05-26T22:54:00.719 INFO:teuthology.orchestra.run.smithi131.stdout:Kernel IP routing table 2024-05-26T22:54:00.719 INFO:teuthology.orchestra.run.smithi131.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-26T22:54:00.719 INFO:teuthology.orchestra.run.smithi131.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-26T22:54:00.720 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-26T22:54:00.720 INFO:teuthology.orchestra.run.smithi131.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-26T22:54:00.720 DEBUG:teuthology.orchestra.run.smithi131:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-26T22:54:00.720 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2024-05-26T22:54:00.720 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-26T22:54:00.720 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-26T22:54:00.720 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-26T22:54:00.720 DEBUG:teuthology.orchestra.run.smithi131:> ') 2024-05-26T22:54:00.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-26T22:54:00.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-26T22:54:00.928 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:54:00.928 DEBUG:teuthology.orchestra.run.smithi131:> ip netns list 2024-05-26T22:54:00.978 INFO:teuthology.orchestra.run.smithi131.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-26T22:54:00.978 INFO:teuthology.orchestra.run.smithi131.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-26T22:54:00.979 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-26T22:54:00.979 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:54:00.979 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link set brx.0 up 2024-05-26T22:54:01.059 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-26T22:54:01.059 DEBUG:teuthology.orchestra.run.smithi131:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-26T22:54:01.110 INFO:teuthology.orchestra.run.smithi131.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-26T22:54:01.110 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-26T22:54:01.110 DEBUG:teuthology.orchestra.run.smithi131:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-26T22:54:01.161 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-05-26T22:54:01.161 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:54:01.162 DEBUG:teuthology.orchestra.run.smithi131:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-26T22:54:01.289 INFO:teuthology.orchestra.run.smithi131.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-26T22:54:01.289 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-26T22:54:01.323 INFO:teuthology.orchestra.run.smithi131.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-26T22:54:01.323 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-26T22:54:01.324 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: resolved to: "172.21.15.131:3300,172.21.15.187:3300,172.21.15.187:3301" 2024-05-26T22:54:01.324 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: trying mount with new device syntax: 0@46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f.cephfs=/ 2024-05-26T22:54:01.324 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.131:3300/172.21.15.187:3300/172.21.15.187:3301" will pass to kernel 2024-05-26T22:54:01.326 INFO:teuthology.orchestra.run.smithi131.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-26T22:54:01.326 INFO:teuthology.orchestra.run.smithi131.stdout: You just mounted a file system that supports labels which does not 2024-05-26T22:54:01.326 INFO:teuthology.orchestra.run.smithi131.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-26T22:54:01.326 INFO:teuthology.orchestra.run.smithi131.stdout: applications will generate AVC messages and not be allowed access to 2024-05-26T22:54:01.326 INFO:teuthology.orchestra.run.smithi131.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-26T22:54:01.330 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-26T22:54:01.330 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:54:01.331 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-26T22:54:01.413 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-26T22:54:01.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:01 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-26T22:54:01.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-26T22:54:01.558 INFO:teuthology.orchestra.run.smithi131.stdout:client7545 2024-05-26T22:54:01.558 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-26T22:54:01.558 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-26T22:54:01.610 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_export_ephemeral_random True 2024-05-26T22:54:01.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:01.859+0000 7f189fe00640 1 Processor -- start 2024-05-26T22:54:01.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:01.859+0000 7f189fe00640 1 -- start start 2024-05-26T22:54:01.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:01.861+0000 7f189fe00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f18a015b730 0x7f18a015bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:01.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:01.861+0000 7f189fe00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18a013e820 con 0x7f18a006d250 2024-05-26T22:54:01.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:01.861+0000 7f189fe00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18a015c4a0 con 0x7f18a01400c0 2024-05-26T22:54:01.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:01.861+0000 7f189fe00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f18a0071260 con 0x7f18a015b730 2024-05-26T22:54:01.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:01.861+0000 7f189ea00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f18a006d250 0x7f18a006d630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:50694/0 (socket says 172.21.15.131:50694) 2024-05-26T22:54:01.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:01.861+0000 7f189ea00640 1 -- 172.21.15.131:0/1669894891 learned_addr learned my addr 172.21.15.131:0/1669894891 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:54:01.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:01.861+0000 7f189e000640 1 -- 172.21.15.131:0/1669894891 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3032658757 0 0) 0x7f18a013e820 con 0x7f18a006d250 2024-05-26T22:54:01.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:01.861+0000 7f1896000640 1 --2- 172.21.15.131:0/1669894891 >> v2:172.21.15.187:3300/0 conn(0x7f18a015b730 0x7f18a015bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:01.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:01.861+0000 7f189e000640 1 -- 172.21.15.131:0/1669894891 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1884005730 con 0x7f18a006d250 2024-05-26T22:54:01.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:01.862+0000 7f189e000640 1 -- 172.21.15.131:0/1669894891 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2081784343 0 0) 0x7f1884005730 con 0x7f18a006d250 2024-05-26T22:54:01.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:01.862+0000 7f189e000640 1 -- 172.21.15.131:0/1669894891 >> v2:172.21.15.187:3300/0 conn(0x7f18a015b730 msgr2=0x7f18a015bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:01.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:01.862+0000 7f189e000640 1 --2- 172.21.15.131:0/1669894891 >> v2:172.21.15.187:3300/0 conn(0x7f18a015b730 0x7f18a015bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:01.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:01.862+0000 7f189e000640 1 -- 172.21.15.131:0/1669894891 >> v1:172.21.15.187:6789/0 conn(0x7f18a01400c0 legacy=0x7f18a01404f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:01.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:01.862+0000 7f189e000640 1 -- 172.21.15.131:0/1669894891 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18a00714b0 con 0x7f18a006d250 2024-05-26T22:54:01.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:01.862+0000 7f189e000640 1 -- 172.21.15.131:0/1669894891 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7f1890001670 con 0x7f18a006d250 2024-05-26T22:54:01.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:01.862+0000 7f189e000640 1 -- 172.21.15.131:0/1669894891 >> v1:172.21.15.131:6789/0 conn(0x7f18a006d250 legacy=0x7f18a006d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:01.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:01.862+0000 7f189e000640 1 --2- 172.21.15.131:0/1669894891 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1884005bc0 0x7f1884005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:01.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:01.862+0000 7f189e000640 1 --2- 172.21.15.131:0/1669894891 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1884006a10 0x7f1884006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:01.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:01.862+0000 7f189e000640 1 --2- 172.21.15.131:0/1669894891 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f18840073d0 0x7f188400dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:01.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:01.862+0000 7f189e000640 1 -- 172.21.15.131:0/1669894891 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f18a0071260 con 0x7f1884005bc0 2024-05-26T22:54:01.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:01.862+0000 7f189e000640 1 -- 172.21.15.131:0/1669894891 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f188400e320 con 0x7f18840073d0 2024-05-26T22:54:01.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:01.862+0000 7f189e000640 1 -- 172.21.15.131:0/1669894891 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f188400e520 con 0x7f1884006a10 2024-05-26T22:54:01.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:01.862+0000 7f189ea00640 1 --2- 172.21.15.131:0/1669894891 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1884005bc0 0x7f1884005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:01.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:01.863+0000 7f1896000640 1 --2- 172.21.15.131:0/1669894891 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1884006a10 0x7f1884006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:01.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:01.863+0000 7f189f400640 1 --2- 172.21.15.131:0/1669894891 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f18840073d0 0x7f188400dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:01.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:01.863+0000 7f189ea00640 1 -- 172.21.15.131:0/1669894891 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1884006a10 msgr2=0x7f1884006e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:01.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:01.863+0000 7f189ea00640 1 --2- 172.21.15.131:0/1669894891 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1884006a10 0x7f1884006e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:01.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:01.863+0000 7f189ea00640 1 -- 172.21.15.131:0/1669894891 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f18840073d0 msgr2=0x7f188400dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:01.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:01.863+0000 7f189ea00640 1 --2- 172.21.15.131:0/1669894891 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f18840073d0 0x7f188400dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:01.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:01.863+0000 7f189ea00640 1 -- 172.21.15.131:0/1669894891 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1884005730 con 0x7f1884005bc0 2024-05-26T22:54:01.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:01.863+0000 7f189f400640 1 -- 172.21.15.131:0/1669894891 reap_dead start 2024-05-26T22:54:01.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:01.864+0000 7f189ea00640 1 --2- 172.21.15.131:0/1669894891 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1884005bc0 0x7f1884005fe0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f189000c240 tx=0x7f1884007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ff35379400fbd3c server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:01.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:01.864+0000 7f189e000640 1 -- 172.21.15.131:0/1669894891 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1890007d10 con 0x7f1884005bc0 2024-05-26T22:54:01.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:01.864+0000 7f189e000640 1 -- 172.21.15.131:0/1669894891 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1890005ad0 con 0x7f1884005bc0 2024-05-26T22:54:01.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:01.864+0000 7f189e000640 1 -- 172.21.15.131:0/1669894891 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1890007070 con 0x7f1884005bc0 2024-05-26T22:54:01.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:01.865+0000 7f189fe00640 1 -- 172.21.15.131:0/1669894891 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1884005bc0 msgr2=0x7f1884005fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:01.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:01.865+0000 7f189fe00640 1 --2- 172.21.15.131:0/1669894891 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1884005bc0 0x7f1884005fe0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f189000c240 tx=0x7f1884007a40 comp rx=0 tx=0).stop 2024-05-26T22:54:01.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:01.865+0000 7f189fe00640 1 -- 172.21.15.131:0/1669894891 shutdown_connections 2024-05-26T22:54:01.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:01.865+0000 7f189fe00640 1 --2- 172.21.15.131:0/1669894891 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1884005bc0 0x7f1884005fe0 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:01.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:01.865+0000 7f189fe00640 1 -- 172.21.15.131:0/1669894891 >> 172.21.15.131:0/1669894891 conn(0x7f18a00b5660 msgr2=0x7f18a00b5a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:01.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:01.865+0000 7f189fe00640 1 -- 172.21.15.131:0/1669894891 shutdown_connections 2024-05-26T22:54:01.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:01.865+0000 7f189fe00640 1 -- 172.21.15.131:0/1669894891 wait complete. 2024-05-26T22:54:01.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:01.867+0000 7f189fe00640 1 Processor -- start 2024-05-26T22:54:01.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:01.867+0000 7f189fe00640 1 -- start start 2024-05-26T22:54:01.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:01.868+0000 7f189fe00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1884005bc0 0x7f18a0154980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:01.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:01.868+0000 7f189fe00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f18a00b49d0 0x7f18a0157170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:01.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:01.868+0000 7f189fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18a0157f70 0x7f18a01583f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:01.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:01.868+0000 7f189fe00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f18a006d630 con 0x7f18a0157f70 2024-05-26T22:54:01.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:01.868+0000 7f189fe00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f18a0071a70 con 0x7f18a00b49d0 2024-05-26T22:54:01.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:01.868+0000 7f189fe00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f18a006d7b0 con 0x7f1884005bc0 2024-05-26T22:54:01.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:01.868+0000 7f189f400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18a0157f70 0x7f18a01583f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:01.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:01.868+0000 7f189f400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18a0157f70 0x7f18a01583f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56894/0 (socket says 172.21.15.131:56894) 2024-05-26T22:54:01.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:01.868+0000 7f189f400640 1 -- 172.21.15.131:0/838140470 learned_addr learned my addr 172.21.15.131:0/838140470 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:54:01.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:01.868+0000 7f1896000640 1 --2- 172.21.15.131:0/838140470 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f18a00b49d0 0x7f18a0157170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:01.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:01.868+0000 7f189ea00640 1 --2- 172.21.15.131:0/838140470 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1884005bc0 0x7f18a0154980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:01.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:01.868+0000 7f189f400640 1 -- 172.21.15.131:0/838140470 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1884005bc0 msgr2=0x7f18a0154980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:01.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:01.868+0000 7f189f400640 1 --2- 172.21.15.131:0/838140470 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1884005bc0 0x7f18a0154980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:01.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:01.868+0000 7f189f400640 1 -- 172.21.15.131:0/838140470 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f18a00b49d0 msgr2=0x7f18a0157170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:01.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:01.868+0000 7f189f400640 1 --2- 172.21.15.131:0/838140470 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f18a00b49d0 0x7f18a0157170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:01.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:01.868+0000 7f189f400640 1 -- 172.21.15.131:0/838140470 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18a01560d0 con 0x7f18a0157f70 2024-05-26T22:54:01.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:01.868+0000 7f189ea00640 1 --2- 172.21.15.131:0/838140470 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1884005bc0 0x7f18a0154980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:54:01.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:01.869+0000 7f189f400640 1 --2- 172.21.15.131:0/838140470 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18a0157f70 0x7f18a01583f0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f189800d400 tx=0x7f189800d8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:01.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:01.869+0000 7f1897400640 1 -- 172.21.15.131:0/838140470 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1898019070 con 0x7f18a0157f70 2024-05-26T22:54:01.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:01.869+0000 7f1897400640 1 -- 172.21.15.131:0/838140470 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1898005230 con 0x7f18a0157f70 2024-05-26T22:54:01.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:01.869+0000 7f1897400640 1 -- 172.21.15.131:0/838140470 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1898012600 con 0x7f18a0157f70 2024-05-26T22:54:01.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:01.869+0000 7f189fe00640 1 -- 172.21.15.131:0/838140470 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18a01563b0 con 0x7f18a0157f70 2024-05-26T22:54:01.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:01.869+0000 7f189fe00640 1 -- 172.21.15.131:0/838140470 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18a0156970 con 0x7f18a0157f70 2024-05-26T22:54:01.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:01.870+0000 7f189fe00640 1 -- 172.21.15.131:0/838140470 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18a0074cc0 con 0x7f18a0157f70 2024-05-26T22:54:01.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:01.871+0000 7f1897400640 1 -- 172.21.15.131:0/838140470 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f189801c070 con 0x7f18a0157f70 2024-05-26T22:54:01.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:01.872+0000 7f1897400640 1 --2- 172.21.15.131:0/838140470 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f187c06be30 0x7f187c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:01.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:01.872+0000 7f1897400640 1 -- 172.21.15.131:0/838140470 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5709+0+0 (secure 0 0 0) 0x7f1898089ee0 con 0x7f18a0157f70 2024-05-26T22:54:01.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:01.873+0000 7f189ea00640 1 --2- 172.21.15.131:0/838140470 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f187c06be30 0x7f187c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:01.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:01.873+0000 7f189ea00640 1 --2- 172.21.15.131:0/838140470 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f187c06be30 0x7f187c06e2f0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f1890033000 tx=0x7f1890009030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:01.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:01.877+0000 7f1897400640 1 -- 172.21.15.131:0/838140470 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f18980587f0 con 0x7f18a0157f70 2024-05-26T22:54:02.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.075+0000 7f189fe00640 1 -- 172.21.15.131:0/838140470 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command([{prefix=config set, name=mds_export_ephemeral_random}] v 0) -- 0x7f18a00694e0 con 0x7f18a0157f70 2024-05-26T22:54:02.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.082+0000 7f1897400640 1 -- 172.21.15.131:0/838140470 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_export_ephemeral_random}]=0 v44)=0 v44) ==== 128+0+0 (secure 0 0 0) 0x7f18980580f0 con 0x7f18a0157f70 2024-05-26T22:54:02.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.085+0000 7f189fe00640 1 -- 172.21.15.131:0/838140470 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f187c06be30 msgr2=0x7f187c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:02.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.085+0000 7f189fe00640 1 --2- 172.21.15.131:0/838140470 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f187c06be30 0x7f187c06e2f0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f1890033000 tx=0x7f1890009030 comp rx=0 tx=0).stop 2024-05-26T22:54:02.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.085+0000 7f189fe00640 1 -- 172.21.15.131:0/838140470 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18a0157f70 msgr2=0x7f18a01583f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:02.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.085+0000 7f189fe00640 1 --2- 172.21.15.131:0/838140470 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18a0157f70 0x7f18a01583f0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f189800d400 tx=0x7f189800d8c0 comp rx=0 tx=0).stop 2024-05-26T22:54:02.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.085+0000 7f189fe00640 1 -- 172.21.15.131:0/838140470 shutdown_connections 2024-05-26T22:54:02.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.085+0000 7f189fe00640 1 --2- 172.21.15.131:0/838140470 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f187c06be30 0x7f187c06e2f0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:02.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.085+0000 7f189fe00640 1 --2- 172.21.15.131:0/838140470 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18a0157f70 0x7f18a01583f0 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:02.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.085+0000 7f189fe00640 1 --2- 172.21.15.131:0/838140470 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f18a00b49d0 0x7f18a0157170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:02.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.085+0000 7f189fe00640 1 --2- 172.21.15.131:0/838140470 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1884005bc0 0x7f18a0154980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:02.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.085+0000 7f189fe00640 1 -- 172.21.15.131:0/838140470 >> 172.21.15.131:0/838140470 conn(0x7f18a00b5660 msgr2=0x7f18a013ea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:02.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.085+0000 7f189fe00640 1 -- 172.21.15.131:0/838140470 shutdown_connections 2024-05-26T22:54:02.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.085+0000 7f189fe00640 1 -- 172.21.15.131:0/838140470 wait complete. 2024-05-26T22:54:02.105 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_export_ephemeral_distributed True 2024-05-26T22:54:02.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.308+0000 7fc8bbe00640 1 Processor -- start 2024-05-26T22:54:02.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.308+0000 7fc8bbe00640 1 -- start start 2024-05-26T22:54:02.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.309+0000 7fc8bbe00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fc8bc145a10 0x7fc8bc072180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:02.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.309+0000 7fc8bbe00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fc8bc0712d0 0x7fc8bc06d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:02.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.310+0000 7fc8bbe00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8bc072d60 con 0x7fc8bc06d790 2024-05-26T22:54:02.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.310+0000 7fc8bbe00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fc8bc15c760 con 0x7fc8bc145a10 2024-05-26T22:54:02.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.310+0000 7fc8bbe00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fc8bc06dbc0 con 0x7fc8bc0712d0 2024-05-26T22:54:02.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.310+0000 7fc8baa00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fc8bc145a10 0x7fc8bc072180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:02.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.310+0000 7fc8baa00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fc8bc145a10 0x7fc8bc072180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56906/0 (socket says 172.21.15.131:56906) 2024-05-26T22:54:02.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.310+0000 7fc8baa00640 1 -- 172.21.15.131:0/1598435729 learned_addr learned my addr 172.21.15.131:0/1598435729 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:54:02.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.310+0000 7fc8b2000640 1 --2- 172.21.15.131:0/1598435729 >> v2:172.21.15.187:3300/0 conn(0x7fc8bc0712d0 0x7fc8bc06d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:02.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.310+0000 7fc8baa00640 1 -- 172.21.15.131:0/1598435729 >> v2:172.21.15.187:3300/0 conn(0x7fc8bc0712d0 msgr2=0x7fc8bc06d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:02.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.310+0000 7fc8baa00640 1 --2- 172.21.15.131:0/1598435729 >> v2:172.21.15.187:3300/0 conn(0x7fc8bc0712d0 0x7fc8bc06d250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:02.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.310+0000 7fc8baa00640 1 -- 172.21.15.131:0/1598435729 >> v1:172.21.15.187:6789/0 conn(0x7fc8bc06d790 legacy=0x7fc8bc15fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:02.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.310+0000 7fc8baa00640 1 -- 172.21.15.131:0/1598435729 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8bc072a80 con 0x7fc8bc145a10 2024-05-26T22:54:02.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.311+0000 7fc8baa00640 1 --2- 172.21.15.131:0/1598435729 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc8bc145a10 0x7fc8bc072180 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7fc8bc0af570 tx=0x7fc8ac02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6b5145d19c6d632 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:02.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.312+0000 7fc8ba000640 1 -- 172.21.15.131:0/1598435729 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8ac037070 con 0x7fc8bc145a10 2024-05-26T22:54:02.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.312+0000 7fc8ba000640 1 -- 172.21.15.131:0/1598435729 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8ac032440 con 0x7fc8bc145a10 2024-05-26T22:54:02.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.312+0000 7fc8ba000640 1 -- 172.21.15.131:0/1598435729 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8ac032810 con 0x7fc8bc145a10 2024-05-26T22:54:02.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.312+0000 7fc8bbe00640 1 -- 172.21.15.131:0/1598435729 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc8bc145a10 msgr2=0x7fc8bc072180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:02.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.312+0000 7fc8bbe00640 1 --2- 172.21.15.131:0/1598435729 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc8bc145a10 0x7fc8bc072180 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7fc8bc0af570 tx=0x7fc8ac02a270 comp rx=0 tx=0).stop 2024-05-26T22:54:02.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.312+0000 7fc8bbe00640 1 -- 172.21.15.131:0/1598435729 shutdown_connections 2024-05-26T22:54:02.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.312+0000 7fc8bbe00640 1 --2- 172.21.15.131:0/1598435729 >> v2:172.21.15.187:3300/0 conn(0x7fc8bc0712d0 0x7fc8bc06d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:02.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.312+0000 7fc8bbe00640 1 --2- 172.21.15.131:0/1598435729 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc8bc145a10 0x7fc8bc072180 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:02.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.313+0000 7fc8bbe00640 1 -- 172.21.15.131:0/1598435729 >> 172.21.15.131:0/1598435729 conn(0x7fc8bc0b5660 msgr2=0x7fc8bc0b5a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:02.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.313+0000 7fc8bbe00640 1 -- 172.21.15.131:0/1598435729 shutdown_connections 2024-05-26T22:54:02.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.313+0000 7fc8bbe00640 1 -- 172.21.15.131:0/1598435729 wait complete. 2024-05-26T22:54:02.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.315+0000 7fc8bbe00640 1 Processor -- start 2024-05-26T22:54:02.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.315+0000 7fc8bbe00640 1 -- start start 2024-05-26T22:54:02.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.316+0000 7fc8bbe00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc8bc06d790 0x7fc8bc158d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:02.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.316+0000 7fc8bbe00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc8bc0712d0 0x7fc8bc143eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:02.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.316+0000 7fc8bbe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc8bc145a10 0x7fc8bc1443f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:02.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.316+0000 7fc8bbe00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc8bc0cf070 con 0x7fc8bc145a10 2024-05-26T22:54:02.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.316+0000 7fc8bbe00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc8bc0ced90 con 0x7fc8bc0712d0 2024-05-26T22:54:02.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.316+0000 7fc8bbe00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc8bc145df0 con 0x7fc8bc06d790 2024-05-26T22:54:02.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.316+0000 7fc8bb400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc8bc145a10 0x7fc8bc1443f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:02.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.316+0000 7fc8bb400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc8bc145a10 0x7fc8bc1443f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56916/0 (socket says 172.21.15.131:56916) 2024-05-26T22:54:02.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.316+0000 7fc8bb400640 1 -- 172.21.15.131:0/2568881431 learned_addr learned my addr 172.21.15.131:0/2568881431 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:54:02.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.317+0000 7fc8b2000640 1 --2- 172.21.15.131:0/2568881431 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc8bc0712d0 0x7fc8bc143eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:02.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.317+0000 7fc8baa00640 1 --2- 172.21.15.131:0/2568881431 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc8bc06d790 0x7fc8bc158d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:02.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.317+0000 7fc8bb400640 1 -- 172.21.15.131:0/2568881431 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc8bc06d790 msgr2=0x7fc8bc158d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:02.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.317+0000 7fc8bb400640 1 --2- 172.21.15.131:0/2568881431 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc8bc06d790 0x7fc8bc158d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:02.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.317+0000 7fc8bb400640 1 -- 172.21.15.131:0/2568881431 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc8bc0712d0 msgr2=0x7fc8bc143eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:02.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.317+0000 7fc8bb400640 1 --2- 172.21.15.131:0/2568881431 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc8bc0712d0 0x7fc8bc143eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:02.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.317+0000 7fc8bb400640 1 -- 172.21.15.131:0/2568881431 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8bc13d260 con 0x7fc8bc145a10 2024-05-26T22:54:02.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.317+0000 7fc8baa00640 1 --2- 172.21.15.131:0/2568881431 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc8bc06d790 0x7fc8bc158d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:54:02.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.317+0000 7fc8bb400640 1 --2- 172.21.15.131:0/2568881431 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc8bc145a10 0x7fc8bc1443f0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7fc8b400c7b0 tx=0x7fc8b400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:02.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.317+0000 7fc8b3e00640 1 -- 172.21.15.131:0/2568881431 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b4017070 con 0x7fc8bc145a10 2024-05-26T22:54:02.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.317+0000 7fc8b3e00640 1 -- 172.21.15.131:0/2568881431 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8b4003da0 con 0x7fc8bc145a10 2024-05-26T22:54:02.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.317+0000 7fc8b3e00640 1 -- 172.21.15.131:0/2568881431 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b40052b0 con 0x7fc8bc145a10 2024-05-26T22:54:02.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.317+0000 7fc8bbe00640 1 -- 172.21.15.131:0/2568881431 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8bc13d540 con 0x7fc8bc145a10 2024-05-26T22:54:02.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.318+0000 7fc8bbe00640 1 -- 172.21.15.131:0/2568881431 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8bc13db00 con 0x7fc8bc145a10 2024-05-26T22:54:02.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.319+0000 7fc8bbe00640 1 -- 172.21.15.131:0/2568881431 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8bc141ce0 con 0x7fc8bc145a10 2024-05-26T22:54:02.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.320+0000 7fc8b3e00640 1 -- 172.21.15.131:0/2568881431 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc8b401a070 con 0x7fc8bc145a10 2024-05-26T22:54:02.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.321+0000 7fc8b3e00640 1 --2- 172.21.15.131:0/2568881431 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc89806be30 0x7fc89806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:02.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.321+0000 7fc8b3e00640 1 -- 172.21.15.131:0/2568881431 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5709+0+0 (secure 0 0 0) 0x7fc8b408a530 con 0x7fc8bc145a10 2024-05-26T22:54:02.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.321+0000 7fc8baa00640 1 --2- 172.21.15.131:0/2568881431 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc89806be30 0x7fc89806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:02.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.322+0000 7fc8baa00640 1 --2- 172.21.15.131:0/2568881431 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc89806be30 0x7fc89806e2f0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fc8ac034000 tx=0x7fc8ac02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:02.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.326+0000 7fc8b3e00640 1 -- 172.21.15.131:0/2568881431 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fc8b4058e40 con 0x7fc8bc145a10 2024-05-26T22:54:02.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.526+0000 7fc8bbe00640 1 -- 172.21.15.131:0/2568881431 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command([{prefix=config set, name=mds_export_ephemeral_distributed}] v 0) -- 0x7fc8bc0694e0 con 0x7fc8bc145a10 2024-05-26T22:54:02.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.532+0000 7fc8b3e00640 1 -- 172.21.15.131:0/2568881431 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_export_ephemeral_distributed}]=0 v45)=0 v45) ==== 133+0+0 (secure 0 0 0) 0x7fc8b4058740 con 0x7fc8bc145a10 2024-05-26T22:54:02.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.535+0000 7fc8bbe00640 1 -- 172.21.15.131:0/2568881431 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc89806be30 msgr2=0x7fc89806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:02.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.535+0000 7fc8bbe00640 1 --2- 172.21.15.131:0/2568881431 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc89806be30 0x7fc89806e2f0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fc8ac034000 tx=0x7fc8ac02e000 comp rx=0 tx=0).stop 2024-05-26T22:54:02.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.535+0000 7fc8bbe00640 1 -- 172.21.15.131:0/2568881431 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc8bc145a10 msgr2=0x7fc8bc1443f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:02.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.535+0000 7fc8bbe00640 1 --2- 172.21.15.131:0/2568881431 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc8bc145a10 0x7fc8bc1443f0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7fc8b400c7b0 tx=0x7fc8b400cc70 comp rx=0 tx=0).stop 2024-05-26T22:54:02.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.535+0000 7fc8bbe00640 1 -- 172.21.15.131:0/2568881431 shutdown_connections 2024-05-26T22:54:02.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.535+0000 7fc8bbe00640 1 --2- 172.21.15.131:0/2568881431 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc89806be30 0x7fc89806e2f0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:02.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.535+0000 7fc8bbe00640 1 --2- 172.21.15.131:0/2568881431 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc8bc145a10 0x7fc8bc1443f0 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:02.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.535+0000 7fc8bbe00640 1 --2- 172.21.15.131:0/2568881431 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc8bc0712d0 0x7fc8bc143eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:02.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.535+0000 7fc8bbe00640 1 --2- 172.21.15.131:0/2568881431 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc8bc06d790 0x7fc8bc158d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:02.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.535+0000 7fc8bbe00640 1 -- 172.21.15.131:0/2568881431 >> 172.21.15.131:0/2568881431 conn(0x7fc8bc0b5660 msgr2=0x7fc8bc0b5a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:02.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.535+0000 7fc8bbe00640 1 -- 172.21.15.131:0/2568881431 shutdown_connections 2024-05-26T22:54:02.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.535+0000 7fc8bbe00640 1 -- 172.21.15.131:0/2568881431 wait complete. 2024-05-26T22:54:02.554 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_export_ephemeral_random_max 1.0 2024-05-26T22:54:02.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.757+0000 7fbafbe00640 1 Processor -- start 2024-05-26T22:54:02.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.757+0000 7fbafbe00640 1 -- start start 2024-05-26T22:54:02.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.759+0000 7fbafbe00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fbafc0712d0 0x7fbafc06d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:02.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.759+0000 7fbafbe00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbafc072d60 con 0x7fbafc06d790 2024-05-26T22:54:02.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.759+0000 7fbafbe00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbafc072a80 con 0x7fbafc145a10 2024-05-26T22:54:02.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.759+0000 7fbafbe00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fbafc15c760 con 0x7fbafc0712d0 2024-05-26T22:54:02.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.759+0000 7fbafb400640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fbafc06d790 0x7fbafc0cf590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:50700/0 (socket says 172.21.15.131:50700) 2024-05-26T22:54:02.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.759+0000 7fbafb400640 1 -- 172.21.15.131:0/2932678049 learned_addr learned my addr 172.21.15.131:0/2932678049 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:54:02.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.759+0000 7fbafa000640 1 --2- 172.21.15.131:0/2932678049 >> v2:172.21.15.131:3300/0 conn(0x7fbafc0712d0 0x7fbafc06d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:02.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.760+0000 7fbafa000640 1 -- 172.21.15.131:0/2932678049 >> v1:172.21.15.187:6789/0 conn(0x7fbafc145a10 legacy=0x7fbafc072180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:02.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.760+0000 7fbafa000640 1 -- 172.21.15.131:0/2932678049 >> v1:172.21.15.131:6789/0 conn(0x7fbafc06d790 legacy=0x7fbafc0cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:02.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.760+0000 7fbaf9600640 1 -- 172.21.15.131:0/2932678049 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3992010691 0 0) 0x7fbafc072d60 con 0x7fbafc06d790 2024-05-26T22:54:02.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.760+0000 7fbafa000640 1 -- 172.21.15.131:0/2932678049 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbafc0ceab0 con 0x7fbafc0712d0 2024-05-26T22:54:02.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.761+0000 7fbafa000640 1 --2- 172.21.15.131:0/2932678049 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbafc0712d0 0x7fbafc06d250 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7fbae40097a0 tx=0x7fbae402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7ebfe914d7f88f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:02.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.761+0000 7fbaf9600640 1 -- 172.21.15.131:0/2932678049 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbae4037070 con 0x7fbafc0712d0 2024-05-26T22:54:02.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.761+0000 7fbaf9600640 1 -- 172.21.15.131:0/2932678049 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbae4032440 con 0x7fbafc0712d0 2024-05-26T22:54:02.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.761+0000 7fbaf9600640 1 -- 172.21.15.131:0/2932678049 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbae4032810 con 0x7fbafc0712d0 2024-05-26T22:54:02.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.762+0000 7fbafbe00640 1 -- 172.21.15.131:0/2932678049 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbafc0712d0 msgr2=0x7fbafc06d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:02.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.762+0000 7fbafbe00640 1 --2- 172.21.15.131:0/2932678049 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbafc0712d0 0x7fbafc06d250 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7fbae40097a0 tx=0x7fbae402a230 comp rx=0 tx=0).stop 2024-05-26T22:54:02.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.762+0000 7fbafbe00640 1 -- 172.21.15.131:0/2932678049 shutdown_connections 2024-05-26T22:54:02.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.762+0000 7fbafbe00640 1 --2- 172.21.15.131:0/2932678049 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbafc0712d0 0x7fbafc06d250 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:02.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.762+0000 7fbafbe00640 1 -- 172.21.15.131:0/2932678049 >> 172.21.15.131:0/2932678049 conn(0x7fbafc0b5660 msgr2=0x7fbafc0b5a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:02.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.762+0000 7fbafbe00640 1 -- 172.21.15.131:0/2932678049 shutdown_connections 2024-05-26T22:54:02.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.762+0000 7fbafbe00640 1 -- 172.21.15.131:0/2932678049 wait complete. 2024-05-26T22:54:02.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.764+0000 7fbafbe00640 1 Processor -- start 2024-05-26T22:54:02.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.764+0000 7fbafbe00640 1 -- start start 2024-05-26T22:54:02.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.765+0000 7fbafbe00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbafc06d790 0x7fbafc0b3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:02.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.765+0000 7fbafbe00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbafc0712d0 0x7fbafc1f7260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:02.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.765+0000 7fbafbe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbafc145a10 0x7fbafc0aed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:02.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.765+0000 7fbafbe00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fbafc0cf070 con 0x7fbafc145a10 2024-05-26T22:54:02.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.765+0000 7fbafbe00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fbafc06dbc0 con 0x7fbafc06d790 2024-05-26T22:54:02.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.765+0000 7fbafbe00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fbafc145df0 con 0x7fbafc0712d0 2024-05-26T22:54:02.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.765+0000 7fbafb400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbafc145a10 0x7fbafc0aed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:02.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.765+0000 7fbafb400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbafc145a10 0x7fbafc0aed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56936/0 (socket says 172.21.15.131:56936) 2024-05-26T22:54:02.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.765+0000 7fbafb400640 1 -- 172.21.15.131:0/2236996303 learned_addr learned my addr 172.21.15.131:0/2236996303 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:54:02.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.766+0000 7fbafa000640 1 --2- 172.21.15.131:0/2236996303 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbafc0712d0 0x7fbafc1f7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:02.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.766+0000 7fbafaa00640 1 --2- 172.21.15.131:0/2236996303 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbafc06d790 0x7fbafc0b3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:02.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.766+0000 7fbafb400640 1 -- 172.21.15.131:0/2236996303 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbafc0712d0 msgr2=0x7fbafc1f7260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:02.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.766+0000 7fbafb400640 1 --2- 172.21.15.131:0/2236996303 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbafc0712d0 0x7fbafc1f7260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:02.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.766+0000 7fbafb400640 1 -- 172.21.15.131:0/2236996303 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbafc06d790 msgr2=0x7fbafc0b3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:02.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.766+0000 7fbafb400640 1 --2- 172.21.15.131:0/2236996303 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbafc06d790 0x7fbafc0b3690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:02.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.766+0000 7fbafb400640 1 -- 172.21.15.131:0/2236996303 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbafc1f77a0 con 0x7fbafc145a10 2024-05-26T22:54:02.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.766+0000 7fbafaa00640 1 --2- 172.21.15.131:0/2236996303 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbafc06d790 0x7fbafc0b3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:54:02.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.766+0000 7fbafa000640 1 --2- 172.21.15.131:0/2236996303 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbafc0712d0 0x7fbafc1f7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:54:02.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.766+0000 7fbafb400640 1 --2- 172.21.15.131:0/2236996303 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbafc145a10 0x7fbafc0aed30 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7fbaf400c450 tx=0x7fbaf400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:02.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.766+0000 7fbaf2a00640 1 -- 172.21.15.131:0/2236996303 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf4018070 con 0x7fbafc145a10 2024-05-26T22:54:02.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.766+0000 7fbaf2a00640 1 -- 172.21.15.131:0/2236996303 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbaf4007500 con 0x7fbafc145a10 2024-05-26T22:54:02.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.766+0000 7fbaf2a00640 1 -- 172.21.15.131:0/2236996303 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf4004a20 con 0x7fbafc145a10 2024-05-26T22:54:02.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.766+0000 7fbafbe00640 1 -- 172.21.15.131:0/2236996303 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbafc1f3c20 con 0x7fbafc145a10 2024-05-26T22:54:02.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.766+0000 7fbafbe00640 1 -- 172.21.15.131:0/2236996303 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbafc1f7d20 con 0x7fbafc145a10 2024-05-26T22:54:02.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.767+0000 7fbafbe00640 1 -- 172.21.15.131:0/2236996303 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbafc0694e0 con 0x7fbafc145a10 2024-05-26T22:54:02.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.769+0000 7fbaf2a00640 1 -- 172.21.15.131:0/2236996303 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbaf4004bc0 con 0x7fbafc145a10 2024-05-26T22:54:02.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.770+0000 7fbaf2a00640 1 --2- 172.21.15.131:0/2236996303 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fbad806be30 0x7fbad806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:02.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.770+0000 7fbaf2a00640 1 -- 172.21.15.131:0/2236996303 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5709+0+0 (secure 0 0 0) 0x7fbaf408a070 con 0x7fbafc145a10 2024-05-26T22:54:02.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.770+0000 7fbafaa00640 1 --2- 172.21.15.131:0/2236996303 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fbad806be30 0x7fbad806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:02.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.771+0000 7fbafaa00640 1 --2- 172.21.15.131:0/2236996303 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fbad806be30 0x7fbad806e2f0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fbafc142500 tx=0x7fbaec009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:02.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.775+0000 7fbaf2a00640 1 -- 172.21.15.131:0/2236996303 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fbaf4058a10 con 0x7fbafc145a10 2024-05-26T22:54:02.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.973+0000 7fbafbe00640 1 -- 172.21.15.131:0/2236996303 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command([{prefix=config set, name=mds_export_ephemeral_random_max}] v 0) -- 0x7fbafc06c320 con 0x7fbafc145a10 2024-05-26T22:54:02.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.979+0000 7fbaf2a00640 1 -- 172.21.15.131:0/2236996303 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_export_ephemeral_random_max}]=0 v46)=0 v46) ==== 131+0+0 (secure 0 0 0) 0x7fbaf4058310 con 0x7fbafc145a10 2024-05-26T22:54:02.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.982+0000 7fbafbe00640 1 -- 172.21.15.131:0/2236996303 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fbad806be30 msgr2=0x7fbad806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:02.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.982+0000 7fbafbe00640 1 --2- 172.21.15.131:0/2236996303 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fbad806be30 0x7fbad806e2f0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fbafc142500 tx=0x7fbaec009040 comp rx=0 tx=0).stop 2024-05-26T22:54:02.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.982+0000 7fbafbe00640 1 -- 172.21.15.131:0/2236996303 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbafc145a10 msgr2=0x7fbafc0aed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:02.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.982+0000 7fbafbe00640 1 --2- 172.21.15.131:0/2236996303 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbafc145a10 0x7fbafc0aed30 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7fbaf400c450 tx=0x7fbaf400c910 comp rx=0 tx=0).stop 2024-05-26T22:54:02.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.982+0000 7fbafbe00640 1 -- 172.21.15.131:0/2236996303 shutdown_connections 2024-05-26T22:54:02.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.982+0000 7fbafbe00640 1 --2- 172.21.15.131:0/2236996303 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbafc0712d0 0x7fbafc1f7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:02.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.982+0000 7fbafbe00640 1 --2- 172.21.15.131:0/2236996303 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fbad806be30 0x7fbad806e2f0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:02.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.982+0000 7fbafbe00640 1 --2- 172.21.15.131:0/2236996303 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbafc145a10 0x7fbafc0aed30 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:02.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.982+0000 7fbafbe00640 1 --2- 172.21.15.131:0/2236996303 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbafc06d790 0x7fbafc0b3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:02.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.983+0000 7fbafbe00640 1 -- 172.21.15.131:0/2236996303 >> 172.21.15.131:0/2236996303 conn(0x7fbafc0b5660 msgr2=0x7fbafc15bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:02.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.983+0000 7fbafbe00640 1 -- 172.21.15.131:0/2236996303 shutdown_connections 2024-05-26T22:54:02.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:02.983+0000 7fbafbe00640 1 -- 172.21.15.131:0/2236996303 wait complete. 2024-05-26T22:54:03.003 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-05-26T22:54:03.003 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2024-05-26T22:54:03.003 DEBUG:teuthology.orchestra.run.smithi131:> 2024-05-26T22:54:03.003 DEBUG:teuthology.orchestra.run.smithi131:> # Use up a random number of inode numbers so the ephemeral pinning is not the same every test. 2024-05-26T22:54:03.003 DEBUG:teuthology.orchestra.run.smithi131:> mkdir .inode_number_thrash 2024-05-26T22:54:03.004 DEBUG:teuthology.orchestra.run.smithi131:> count=$((RANDOM % 1024)) 2024-05-26T22:54:03.004 DEBUG:teuthology.orchestra.run.smithi131:> for ((i = 0; i < count; i++)); do touch .inode_number_thrash/$i; done 2024-05-26T22:54:03.004 DEBUG:teuthology.orchestra.run.smithi131:> rm -rf .inode_number_thrash 2024-05-26T22:54:03.004 DEBUG:teuthology.orchestra.run.smithi131:> ') 2024-05-26T22:54:03.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:03 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-26T22:54:07.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-26T22:54:07.709 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2024-05-26T22:54:07.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:07.888+0000 7f64ffa00640 1 Processor -- start 2024-05-26T22:54:07.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:07.889+0000 7f64ffa00640 1 -- start start 2024-05-26T22:54:07.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:07.890+0000 7f64ffa00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f64f8147be0 0x7f64f815fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:07.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:07.890+0000 7f64ffa00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f64f80b5410 0x7f64f80b58a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:07.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:07.890+0000 7f64ffa00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64f80717d0 con 0x7f64f815ff90 2024-05-26T22:54:07.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:07.890+0000 7f64ffa00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f64f806c8b0 con 0x7f64f8147be0 2024-05-26T22:54:07.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:07.890+0000 7f64ffa00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f64f8071ab0 con 0x7f64f80b5410 2024-05-26T22:54:07.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:07.891+0000 7f64fd000640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f64f8147be0 0x7f64f815fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:07.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:07.891+0000 7f64fd000640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f64f8147be0 0x7f64f815fa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46334/0 (socket says 172.21.15.131:46334) 2024-05-26T22:54:07.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:07.891+0000 7f64fd000640 1 -- 172.21.15.131:0/1646880926 learned_addr learned my addr 172.21.15.131:0/1646880926 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:54:07.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:07.891+0000 7f64fda00640 1 --2- 172.21.15.131:0/1646880926 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f64f80b5410 0x7f64f80b58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:07.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:07.891+0000 7f64fd000640 1 -- 172.21.15.131:0/1646880926 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f64f80b5410 msgr2=0x7f64f80b58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:07.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:07.891+0000 7f64fd000640 1 --2- 172.21.15.131:0/1646880926 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f64f80b5410 0x7f64f80b58a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:07.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:07.891+0000 7f64fd000640 1 -- 172.21.15.131:0/1646880926 >> v1:172.21.15.187:6789/0 conn(0x7f64f815ff90 legacy=0x7f64f8070d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:07.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:07.891+0000 7f64fd000640 1 -- 172.21.15.131:0/1646880926 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64f806cb00 con 0x7f64f8147be0 2024-05-26T22:54:07.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:07.892+0000 7f64fd000640 1 --2- 172.21.15.131:0/1646880926 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f64f8147be0 0x7f64f815fa50 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f64e8009870 tx=0x7f64e802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5050b90306035cd9 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:07.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:07.893+0000 7f64f7400640 1 -- 172.21.15.131:0/1646880926 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e8037070 con 0x7f64f8147be0 2024-05-26T22:54:07.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:07.893+0000 7f64f7400640 1 -- 172.21.15.131:0/1646880926 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64e8032440 con 0x7f64f8147be0 2024-05-26T22:54:07.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:07.893+0000 7f64f7400640 1 -- 172.21.15.131:0/1646880926 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e8032810 con 0x7f64f8147be0 2024-05-26T22:54:07.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:07.893+0000 7f64ffa00640 1 -- 172.21.15.131:0/1646880926 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f64f8147be0 msgr2=0x7f64f815fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:07.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:07.893+0000 7f64ffa00640 1 --2- 172.21.15.131:0/1646880926 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f64f8147be0 0x7f64f815fa50 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f64e8009870 tx=0x7f64e802a210 comp rx=0 tx=0).stop 2024-05-26T22:54:07.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:07.894+0000 7f64ffa00640 1 -- 172.21.15.131:0/1646880926 shutdown_connections 2024-05-26T22:54:07.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:07.894+0000 7f64ffa00640 1 --2- 172.21.15.131:0/1646880926 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f64f80b5410 0x7f64f80b58a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:07.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:07.894+0000 7f64ffa00640 1 --2- 172.21.15.131:0/1646880926 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f64f8147be0 0x7f64f815fa50 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:07.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:07.894+0000 7f64ffa00640 1 -- 172.21.15.131:0/1646880926 >> 172.21.15.131:0/1646880926 conn(0x7f64f80b3e00 msgr2=0x7f64f80b41e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:07.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:07.894+0000 7f64ffa00640 1 -- 172.21.15.131:0/1646880926 shutdown_connections 2024-05-26T22:54:07.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:07.894+0000 7f64ffa00640 1 -- 172.21.15.131:0/1646880926 wait complete. 2024-05-26T22:54:07.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:07.895+0000 7f64ffa00640 1 Processor -- start 2024-05-26T22:54:07.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:07.895+0000 7f64ffa00640 1 -- start start 2024-05-26T22:54:07.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:07.896+0000 7f64ffa00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f64f80b5410 0x7f64f8141f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:07.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:07.896+0000 7f64ffa00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f64f8147be0 0x7f64f80cc130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:07.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:07.896+0000 7f64ffa00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f64f815ff90 0x7f64f81f42b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:07.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:07.896+0000 7f64ffa00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f64f8147fc0 con 0x7f64f815ff90 2024-05-26T22:54:07.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:07.896+0000 7f64ffa00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f64f8071490 con 0x7f64f8147be0 2024-05-26T22:54:07.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:07.896+0000 7f64ffa00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f64f81620f0 con 0x7f64f80b5410 2024-05-26T22:54:07.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:07.897+0000 7f64fda00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f64f815ff90 0x7f64f81f42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:07.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:07.897+0000 7f64fda00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f64f815ff90 0x7f64f81f42b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46342/0 (socket says 172.21.15.131:46342) 2024-05-26T22:54:07.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:07.897+0000 7f64fda00640 1 -- 172.21.15.131:0/2471821231 learned_addr learned my addr 172.21.15.131:0/2471821231 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:54:07.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:07.897+0000 7f64fd000640 1 --2- 172.21.15.131:0/2471821231 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f64f80b5410 0x7f64f8141f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:07.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:07.897+0000 7f64f7e00640 1 --2- 172.21.15.131:0/2471821231 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f64f8147be0 0x7f64f80cc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:07.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:07.897+0000 7f64fda00640 1 -- 172.21.15.131:0/2471821231 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f64f80b5410 msgr2=0x7f64f8141f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:07.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:07.897+0000 7f64fda00640 1 --2- 172.21.15.131:0/2471821231 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f64f80b5410 0x7f64f8141f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:07.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:07.897+0000 7f64fda00640 1 -- 172.21.15.131:0/2471821231 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f64f8147be0 msgr2=0x7f64f80cc130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:07.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:07.897+0000 7f64fda00640 1 --2- 172.21.15.131:0/2471821231 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f64f8147be0 0x7f64f80cc130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:07.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:07.897+0000 7f64fda00640 1 -- 172.21.15.131:0/2471821231 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64f81425f0 con 0x7f64f815ff90 2024-05-26T22:54:07.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:07.897+0000 7f64f7e00640 1 --2- 172.21.15.131:0/2471821231 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f64f8147be0 0x7f64f80cc130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:54:07.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:07.898+0000 7f64fda00640 1 --2- 172.21.15.131:0/2471821231 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f64f815ff90 0x7f64f81f42b0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f64f000d800 tx=0x7f64f000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:07.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:07.898+0000 7f64f4c00640 1 -- 172.21.15.131:0/2471821231 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f000b6c0 con 0x7f64f815ff90 2024-05-26T22:54:07.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:07.898+0000 7f64f4c00640 1 -- 172.21.15.131:0/2471821231 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64f0019070 con 0x7f64f815ff90 2024-05-26T22:54:07.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:07.898+0000 7f64f4c00640 1 -- 172.21.15.131:0/2471821231 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f000bac0 con 0x7f64f815ff90 2024-05-26T22:54:07.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:07.898+0000 7f64ffa00640 1 -- 172.21.15.131:0/2471821231 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64f81428d0 con 0x7f64f815ff90 2024-05-26T22:54:07.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:07.898+0000 7f64ffa00640 1 -- 172.21.15.131:0/2471821231 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64f8142e90 con 0x7f64f815ff90 2024-05-26T22:54:07.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:07.899+0000 7f64ffa00640 1 -- 172.21.15.131:0/2471821231 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64f80695c0 con 0x7f64f815ff90 2024-05-26T22:54:07.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:07.900+0000 7f64f4c00640 1 -- 172.21.15.131:0/2471821231 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f64f000bc60 con 0x7f64f815ff90 2024-05-26T22:54:07.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:07.901+0000 7f64f4c00640 1 --2- 172.21.15.131:0/2471821231 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f64d406be30 0x7f64d406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:07.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:07.901+0000 7f64f4c00640 1 -- 172.21.15.131:0/2471821231 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5738+0+0 (secure 0 0 0) 0x7f64f008b420 con 0x7f64f815ff90 2024-05-26T22:54:07.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:07.901+0000 7f64fd000640 1 --2- 172.21.15.131:0/2471821231 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f64d406be30 0x7f64d406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:07.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:07.902+0000 7f64fd000640 1 --2- 172.21.15.131:0/2471821231 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f64d406be30 0x7f64d406e2f0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f64e8034000 tx=0x7f64e802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:07.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:07.906+0000 7f64f4c00640 1 -- 172.21.15.131:0/2471821231 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f64f0059c80 con 0x7f64f815ff90 2024-05-26T22:54:08.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:08.152+0000 7f64ffa00640 1 -- 172.21.15.131:0/2471821231 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7f64f815eff0 con 0x7f64f815ff90 2024-05-26T22:54:09.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:09.135+0000 7f64f4c00640 1 -- 172.21.15.131:0/2471821231 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v79) ==== 105+0+0 (secure 0 0 0) 0x7f64f0059580 con 0x7f64f815ff90 2024-05-26T22:54:09.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:09.139+0000 7f64ffa00640 1 -- 172.21.15.131:0/2471821231 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f64d406be30 msgr2=0x7f64d406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:09.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:09.139+0000 7f64ffa00640 1 --2- 172.21.15.131:0/2471821231 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f64d406be30 0x7f64d406e2f0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f64e8034000 tx=0x7f64e802e000 comp rx=0 tx=0).stop 2024-05-26T22:54:09.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:09.139+0000 7f64ffa00640 1 -- 172.21.15.131:0/2471821231 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f64f815ff90 msgr2=0x7f64f81f42b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:09.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:09.139+0000 7f64ffa00640 1 --2- 172.21.15.131:0/2471821231 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f64f815ff90 0x7f64f81f42b0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f64f000d800 tx=0x7f64f000dcc0 comp rx=0 tx=0).stop 2024-05-26T22:54:09.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:09.139+0000 7f64ffa00640 1 -- 172.21.15.131:0/2471821231 shutdown_connections 2024-05-26T22:54:09.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:09.139+0000 7f64ffa00640 1 --2- 172.21.15.131:0/2471821231 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f64d406be30 0x7f64d406e2f0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:09.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:09.139+0000 7f64ffa00640 1 --2- 172.21.15.131:0/2471821231 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f64f815ff90 0x7f64f81f42b0 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:09.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:09.139+0000 7f64ffa00640 1 --2- 172.21.15.131:0/2471821231 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f64f8147be0 0x7f64f80cc130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:09.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:09.139+0000 7f64ffa00640 1 --2- 172.21.15.131:0/2471821231 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f64f80b5410 0x7f64f8141f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:09.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:09.139+0000 7f64ffa00640 1 -- 172.21.15.131:0/2471821231 >> 172.21.15.131:0/2471821231 conn(0x7f64f80b3e00 msgr2=0x7f64f8070d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:09.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:09.139+0000 7f64ffa00640 1 -- 172.21.15.131:0/2471821231 shutdown_connections 2024-05-26T22:54:09.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:09.139+0000 7f64ffa00640 1 -- 172.21.15.131:0/2471821231 wait complete. 2024-05-26T22:54:09.162 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-26T22:54:09.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:09.346+0000 7f947aa00640 1 Processor -- start 2024-05-26T22:54:09.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:09.346+0000 7f947aa00640 1 -- start start 2024-05-26T22:54:09.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:09.347+0000 7f947aa00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f9474156750 0x7f947415fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:09.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:09.347+0000 7f947aa00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f94740cc7f0 0x7f94740ccc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:09.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:09.347+0000 7f947aa00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9474160620 con 0x7f947406e8c0 2024-05-26T22:54:09.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:09.347+0000 7f947aa00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f947415d770 con 0x7f94740cc7f0 2024-05-26T22:54:09.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:09.347+0000 7f947aa00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f94740cd1a0 con 0x7f9474156750 2024-05-26T22:54:09.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:09.348+0000 7f9478c00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f947406e8c0 0x7f947415d060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:38046/0 (socket says 172.21.15.131:38046) 2024-05-26T22:54:09.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:09.348+0000 7f9478c00640 1 -- 172.21.15.131:0/2210889359 learned_addr learned my addr 172.21.15.131:0/2210889359 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:54:09.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:09.348+0000 7f947a000640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f94740cc7f0 0x7f94740ccc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:09.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:09.348+0000 7f9479600640 1 --2- 172.21.15.131:0/2210889359 >> v2:172.21.15.187:3300/0 conn(0x7f9474156750 0x7f947415fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:09.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:09.348+0000 7f947a000640 1 -- 172.21.15.131:0/2210889359 >> v2:172.21.15.187:3300/0 conn(0x7f9474156750 msgr2=0x7f947415fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:09.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:09.348+0000 7f947a000640 1 --2- 172.21.15.131:0/2210889359 >> v2:172.21.15.187:3300/0 conn(0x7f9474156750 0x7f947415fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:09.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:09.348+0000 7f947a000640 1 -- 172.21.15.131:0/2210889359 >> v1:172.21.15.131:6789/0 conn(0x7f947406e8c0 legacy=0x7f947415d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:09.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:09.348+0000 7f947a000640 1 -- 172.21.15.131:0/2210889359 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9474160900 con 0x7f94740cc7f0 2024-05-26T22:54:09.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:09.348+0000 7f9473e00640 1 -- 172.21.15.131:0/2210889359 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2359236890 0 0) 0x7f9474160620 con 0x7f947406e8c0 2024-05-26T22:54:09.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:09.348+0000 7f9479600640 1 --2- 172.21.15.131:0/2210889359 >> v2:172.21.15.187:3300/0 conn(0x7f9474156750 0x7f947415fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:54:09.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:09.349+0000 7f947a000640 1 --2- 172.21.15.131:0/2210889359 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f94740cc7f0 0x7f94740ccc60 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f94740cdb10 tx=0x7f946c02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1066adef48dc43c server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:09.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:09.350+0000 7f9473e00640 1 -- 172.21.15.131:0/2210889359 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f946c007740 con 0x7f94740cc7f0 2024-05-26T22:54:09.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:09.350+0000 7f9473e00640 1 -- 172.21.15.131:0/2210889359 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f946c0078e0 con 0x7f94740cc7f0 2024-05-26T22:54:09.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:09.350+0000 7f9473e00640 1 -- 172.21.15.131:0/2210889359 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f946c007cb0 con 0x7f94740cc7f0 2024-05-26T22:54:09.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:09.350+0000 7f947aa00640 1 -- 172.21.15.131:0/2210889359 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f94740cc7f0 msgr2=0x7f94740ccc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:09.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:09.350+0000 7f947aa00640 1 --2- 172.21.15.131:0/2210889359 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f94740cc7f0 0x7f94740ccc60 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f94740cdb10 tx=0x7f946c02a680 comp rx=0 tx=0).stop 2024-05-26T22:54:09.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:09.351+0000 7f947aa00640 1 -- 172.21.15.131:0/2210889359 shutdown_connections 2024-05-26T22:54:09.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:09.351+0000 7f947aa00640 1 --2- 172.21.15.131:0/2210889359 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f94740cc7f0 0x7f94740ccc60 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:09.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:09.351+0000 7f947aa00640 1 --2- 172.21.15.131:0/2210889359 >> v2:172.21.15.187:3300/0 conn(0x7f9474156750 0x7f947415fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:09.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:09.351+0000 7f947aa00640 1 -- 172.21.15.131:0/2210889359 >> 172.21.15.131:0/2210889359 conn(0x7f94740cff40 msgr2=0x7f94740d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:09.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:09.351+0000 7f947aa00640 1 -- 172.21.15.131:0/2210889359 shutdown_connections 2024-05-26T22:54:09.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:09.351+0000 7f947aa00640 1 -- 172.21.15.131:0/2210889359 wait complete. 2024-05-26T22:54:09.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:09.352+0000 7f947aa00640 1 Processor -- start 2024-05-26T22:54:09.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:09.352+0000 7f947aa00640 1 -- start start 2024-05-26T22:54:09.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:09.353+0000 7f947aa00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f947406e8c0 0x7f9474072080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:09.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:09.353+0000 7f947aa00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f94740cc7f0 0x7f947415e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:09.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:09.353+0000 7f947aa00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9474156750 0x7f947415e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:09.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:09.353+0000 7f947aa00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f947415c860 con 0x7f947406e8c0 2024-05-26T22:54:09.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:09.353+0000 7f947aa00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f947415cbd0 con 0x7f94740cc7f0 2024-05-26T22:54:09.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:09.353+0000 7f947aa00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9474156b30 con 0x7f9474156750 2024-05-26T22:54:09.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:09.354+0000 7f9479600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f947406e8c0 0x7f9474072080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:09.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:09.354+0000 7f9479600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f947406e8c0 0x7f9474072080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46352/0 (socket says 172.21.15.131:46352) 2024-05-26T22:54:09.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:09.354+0000 7f9479600640 1 -- 172.21.15.131:0/1967954186 learned_addr learned my addr 172.21.15.131:0/1967954186 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:54:09.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:09.354+0000 7f9479600640 1 -- 172.21.15.131:0/1967954186 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9474156750 msgr2=0x7f947415e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:09.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:09.354+0000 7f9479600640 1 --2- 172.21.15.131:0/1967954186 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9474156750 0x7f947415e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:09.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:09.354+0000 7f9479600640 1 -- 172.21.15.131:0/1967954186 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f94740cc7f0 msgr2=0x7f947415e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:09.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:09.354+0000 7f9479600640 1 --2- 172.21.15.131:0/1967954186 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f94740cc7f0 0x7f947415e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:09.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:09.354+0000 7f9479600640 1 -- 172.21.15.131:0/1967954186 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94740b0e10 con 0x7f947406e8c0 2024-05-26T22:54:09.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:09.354+0000 7f9479600640 1 --2- 172.21.15.131:0/1967954186 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f947406e8c0 0x7f9474072080 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f946800c820 tx=0x7f946800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:09.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:09.355+0000 7f9471600640 1 -- 172.21.15.131:0/1967954186 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9468015070 con 0x7f947406e8c0 2024-05-26T22:54:09.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:09.355+0000 7f9471600640 1 -- 172.21.15.131:0/1967954186 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9468007a30 con 0x7f947406e8c0 2024-05-26T22:54:09.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:09.355+0000 7f9471600640 1 -- 172.21.15.131:0/1967954186 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9468007de0 con 0x7f947406e8c0 2024-05-26T22:54:09.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:09.355+0000 7f947aa00640 1 -- 172.21.15.131:0/1967954186 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94740b10f0 con 0x7f947406e8c0 2024-05-26T22:54:09.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:09.355+0000 7f947aa00640 1 -- 172.21.15.131:0/1967954186 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94740b16b0 con 0x7f947406e8c0 2024-05-26T22:54:09.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:09.356+0000 7f947aa00640 1 -- 172.21.15.131:0/1967954186 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94740694e0 con 0x7f947406e8c0 2024-05-26T22:54:09.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:09.357+0000 7f9471600640 1 -- 172.21.15.131:0/1967954186 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9468018070 con 0x7f947406e8c0 2024-05-26T22:54:09.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:09.359+0000 7f9471600640 1 --2- 172.21.15.131:0/1967954186 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f945806be30 0x7f945806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:09.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:09.359+0000 7f9471600640 1 -- 172.21.15.131:0/1967954186 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5738+0+0 (secure 0 0 0) 0x7f946808ae40 con 0x7f947406e8c0 2024-05-26T22:54:09.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:09.360+0000 7f9478c00640 1 --2- 172.21.15.131:0/1967954186 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f945806be30 0x7f945806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:09.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:09.360+0000 7f9478c00640 1 --2- 172.21.15.131:0/1967954186 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f945806be30 0x7f945806e2f0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f9464007fd0 tx=0x7f9464007920 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:09.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:09.365+0000 7f9471600640 1 -- 172.21.15.131:0/1967954186 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f9468059730 con 0x7f947406e8c0 2024-05-26T22:54:09.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:09.613+0000 7f947aa00640 1 -- 172.21.15.131:0/1967954186 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f94740b0af0 con 0x7f947406e8c0 2024-05-26T22:54:09.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:09.615+0000 7f9471600640 1 -- 172.21.15.131:0/1967954186 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) ==== 94+0+5205 (secure 0 0 0) 0x7f9468059030 con 0x7f947406e8c0 2024-05-26T22:54:09.615 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:54:09.615 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":80,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5533,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/993930297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6835","nonce":993930297},{"type":"v1","addr":"172.21.15.131:6837","nonce":993930297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":73},{"gid":5545,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6834/3480101957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3480101957},{"type":"v1","addr":"172.21.15.187:6834","nonce":3480101957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-26T22:53:52.862328+0000","modified":"2024-05-26T22:54:09.141056+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1],"up":{"mds_0":7365,"mds_1":5548},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5548":{"gid":5548,"name":"d","rank":1,"incarnation":80,"state":"up:creating","state_seq":1,"addr":"172.21.15.187:6835/567389303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6833","nonce":567389303},{"type":"v1","addr":"172.21.15.187:6835","nonce":567389303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7365":{"gid":7365,"name":"a","rank":0,"incarnation":74,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6836/3411392567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":3411392567},{"type":"v1","addr":"172.21.15.131:6836","nonce":3411392567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7365,"qdb_cluster":[7365]},"id":6}]} 2024-05-26T22:54:09.615 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 80 2024-05-26T22:54:09.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:09.618+0000 7f947aa00640 1 -- 172.21.15.131:0/1967954186 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f945806be30 msgr2=0x7f945806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:09.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:09.618+0000 7f947aa00640 1 --2- 172.21.15.131:0/1967954186 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f945806be30 0x7f945806e2f0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f9464007fd0 tx=0x7f9464007920 comp rx=0 tx=0).stop 2024-05-26T22:54:09.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:09.618+0000 7f947aa00640 1 -- 172.21.15.131:0/1967954186 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f947406e8c0 msgr2=0x7f9474072080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:09.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:09.618+0000 7f947aa00640 1 --2- 172.21.15.131:0/1967954186 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f947406e8c0 0x7f9474072080 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f946800c820 tx=0x7f946800cce0 comp rx=0 tx=0).stop 2024-05-26T22:54:09.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:09.618+0000 7f947aa00640 1 -- 172.21.15.131:0/1967954186 shutdown_connections 2024-05-26T22:54:09.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:09.618+0000 7f947aa00640 1 --2- 172.21.15.131:0/1967954186 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9474156750 0x7f947415e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:09.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:09.618+0000 7f947aa00640 1 --2- 172.21.15.131:0/1967954186 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f945806be30 0x7f945806e2f0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:09.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:09.618+0000 7f947aa00640 1 --2- 172.21.15.131:0/1967954186 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f94740cc7f0 0x7f947415e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:09.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:09.618+0000 7f947aa00640 1 --2- 172.21.15.131:0/1967954186 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f947406e8c0 0x7f9474072080 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:09.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:09.618+0000 7f947aa00640 1 -- 172.21.15.131:0/1967954186 >> 172.21.15.131:0/1967954186 conn(0x7f94740cff40 msgr2=0x7f94740d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:09.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:09.618+0000 7f947aa00640 1 -- 172.21.15.131:0/1967954186 shutdown_connections 2024-05-26T22:54:09.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:09.618+0000 7f947aa00640 1 -- 172.21.15.131:0/1967954186 wait complete. 2024-05-26T22:54:10.040 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 80, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-26T22:53:52.862328+0000', 'modified': '2024-05-26T22:54:09.141056+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1], 'up': {'mds_0': 7365, 'mds_1': 5548}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5548': {'gid': 5548, 'name': 'd', 'rank': 1, 'incarnation': 80, 'state': 'up:creating', 'state_seq': 1, 'addr': '172.21.15.187:6835/567389303', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.187:6833', 'nonce': 567389303}, {'type': 'v1', 'addr': '172.21.15.187:6835', 'nonce': 567389303}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_7365': {'gid': 7365, 'name': 'a', 'rank': 0, 'incarnation': 74, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.131:6836/3411392567', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6834', 'nonce': 3411392567}, {'type': 'v1', 'addr': '172.21.15.131:6836', 'nonce': 3411392567}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7365, 'qdb_cluster': [7365]} 2024-05-26T22:54:10.040 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_5548:up:creating 2024-05-26T22:54:11.041 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-26T22:54:11.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:11.239+0000 7f7084a00640 1 Processor -- start 2024-05-26T22:54:11.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:11.239+0000 7f7084a00640 1 -- start start 2024-05-26T22:54:11.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:11.240+0000 7f7084a00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f70800b5be0 0x7f70800cef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:11.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:11.240+0000 7f7084a00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f70800d3830 0x7f70800b7160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:11.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:11.240+0000 7f7084a00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f708015e730 con 0x7f70800b5fc0 2024-05-26T22:54:11.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:11.241+0000 7f7084a00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f70800cf4f0 con 0x7f70800d3830 2024-05-26T22:54:11.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:11.241+0000 7f7084a00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f70800d2d50 con 0x7f70800b5be0 2024-05-26T22:54:11.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:11.241+0000 7f7077e00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f70800b5fc0 0x7f70800b9ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:38058/0 (socket says 172.21.15.131:38058) 2024-05-26T22:54:11.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:11.241+0000 7f7077e00640 1 -- 172.21.15.131:0/2861550590 learned_addr learned my addr 172.21.15.131:0/2861550590 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:54:11.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:11.241+0000 7f707f400640 1 --2- 172.21.15.131:0/2861550590 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f70800b5be0 0x7f70800cef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:11.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:11.241+0000 7f707fe00640 1 --2- 172.21.15.131:0/2861550590 >> v2:172.21.15.187:3300/0 conn(0x7f70800d3830 0x7f70800b7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:11.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:11.241+0000 7f707ea00640 1 -- 172.21.15.131:0/2861550590 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1392479642 0 0) 0x7f708015e730 con 0x7f70800b5fc0 2024-05-26T22:54:11.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:11.241+0000 7f707ea00640 1 -- 172.21.15.131:0/2861550590 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7064003520 con 0x7f70800b5fc0 2024-05-26T22:54:11.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:11.242+0000 7f707ea00640 1 -- 172.21.15.131:0/2861550590 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3969995885 0 0) 0x7f7064003520 con 0x7f70800b5fc0 2024-05-26T22:54:11.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:11.242+0000 7f707ea00640 1 -- 172.21.15.131:0/2861550590 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f70800b5be0 msgr2=0x7f70800cef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:11.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:11.242+0000 7f707ea00640 1 --2- 172.21.15.131:0/2861550590 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f70800b5be0 0x7f70800cef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:11.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:11.242+0000 7f707ea00640 1 -- 172.21.15.131:0/2861550590 >> v2:172.21.15.187:3300/0 conn(0x7f70800d3830 msgr2=0x7f70800b7160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:11.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:11.242+0000 7f707ea00640 1 --2- 172.21.15.131:0/2861550590 >> v2:172.21.15.187:3300/0 conn(0x7f70800d3830 0x7f70800b7160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:11.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:11.242+0000 7f707ea00640 1 -- 172.21.15.131:0/2861550590 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70800d3030 con 0x7f70800b5fc0 2024-05-26T22:54:11.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:11.242+0000 7f707ea00640 1 -- 172.21.15.131:0/2861550590 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7f7068001670 con 0x7f70800b5fc0 2024-05-26T22:54:11.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:11.242+0000 7f707fe00640 1 --2- 172.21.15.131:0/2861550590 >> v2:172.21.15.187:3300/0 conn(0x7f70800d3830 0x7f70800b7160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-26T22:54:11.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:11.242+0000 7f707ea00640 1 -- 172.21.15.131:0/2861550590 >> v1:172.21.15.131:6789/0 conn(0x7f70800b5fc0 legacy=0x7f70800b9ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:11.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:11.242+0000 7f707ea00640 1 --2- 172.21.15.131:0/2861550590 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f70640039b0 0x7f7064003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:11.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:11.242+0000 7f707ea00640 1 --2- 172.21.15.131:0/2861550590 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f70800b5be0 0x7f7064004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:11.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:11.242+0000 7f707ea00640 1 --2- 172.21.15.131:0/2861550590 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7064004d40 0x7f70640051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:11.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:11.242+0000 7f707ea00640 1 -- 172.21.15.131:0/2861550590 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f70800cf4f0 con 0x7f70640039b0 2024-05-26T22:54:11.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:11.242+0000 7f707ea00640 1 -- 172.21.15.131:0/2861550590 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f70800d2d50 con 0x7f7064004d40 2024-05-26T22:54:11.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:11.242+0000 7f707ea00640 1 -- 172.21.15.131:0/2861550590 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f706400ba80 con 0x7f70800b5be0 2024-05-26T22:54:11.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:11.242+0000 7f707f400640 1 --2- 172.21.15.131:0/2861550590 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f70640039b0 0x7f7064003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:11.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:11.242+0000 7f7077e00640 1 --2- 172.21.15.131:0/2861550590 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f70800b5be0 0x7f7064004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:11.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:11.242+0000 7f707f400640 1 -- 172.21.15.131:0/2861550590 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f70800b5be0 msgr2=0x7f7064004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:11.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:11.242+0000 7f707f400640 1 --2- 172.21.15.131:0/2861550590 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f70800b5be0 0x7f7064004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:11.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:11.243+0000 7f707f400640 1 -- 172.21.15.131:0/2861550590 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7064004d40 msgr2=0x7f70640051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:11.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:11.243+0000 7f707fe00640 1 --2- 172.21.15.131:0/2861550590 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7064004d40 0x7f70640051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:11.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:11.243+0000 7f707f400640 1 --2- 172.21.15.131:0/2861550590 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7064004d40 0x7f70640051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:11.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:11.243+0000 7f707f400640 1 -- 172.21.15.131:0/2861550590 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7064003520 con 0x7f70640039b0 2024-05-26T22:54:11.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:11.244+0000 7f707f400640 1 --2- 172.21.15.131:0/2861550590 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f70640039b0 0x7f7064003dd0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f7070009ac0 tx=0x7f7064005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c477369ecb6c3f54 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:11.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:11.244+0000 7f707ea00640 1 -- 172.21.15.131:0/2861550590 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7070036070 con 0x7f70640039b0 2024-05-26T22:54:11.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:11.244+0000 7f707ea00640 1 -- 172.21.15.131:0/2861550590 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7070031400 con 0x7f70640039b0 2024-05-26T22:54:11.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:11.244+0000 7f707ea00640 1 -- 172.21.15.131:0/2861550590 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7070009690 con 0x7f70640039b0 2024-05-26T22:54:11.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:11.244+0000 7f7084a00640 1 -- 172.21.15.131:0/2861550590 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f70640039b0 msgr2=0x7f7064003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:11.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:11.244+0000 7f7084a00640 1 --2- 172.21.15.131:0/2861550590 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f70640039b0 0x7f7064003dd0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f7070009ac0 tx=0x7f7064005830 comp rx=0 tx=0).stop 2024-05-26T22:54:11.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:11.244+0000 7f707fe00640 1 -- 172.21.15.131:0/2861550590 reap_dead start 2024-05-26T22:54:11.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:11.244+0000 7f7084a00640 1 -- 172.21.15.131:0/2861550590 shutdown_connections 2024-05-26T22:54:11.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:11.244+0000 7f7084a00640 1 -- 172.21.15.131:0/2861550590 >> 172.21.15.131:0/2861550590 conn(0x7f7080073070 msgr2=0x7f70800d4160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:11.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:11.244+0000 7f7084a00640 1 -- 172.21.15.131:0/2861550590 shutdown_connections 2024-05-26T22:54:11.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:11.244+0000 7f7084a00640 1 -- 172.21.15.131:0/2861550590 wait complete. 2024-05-26T22:54:11.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:11.246+0000 7f7084a00640 1 Processor -- start 2024-05-26T22:54:11.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:11.246+0000 7f7084a00640 1 -- start start 2024-05-26T22:54:11.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:11.247+0000 7f7084a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f70640039b0 0x7f70801461d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:11.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:11.247+0000 7f7084a00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7080153c60 0x7f70801f78c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:11.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:11.247+0000 7f7084a00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7080161420 0x7f70801f7e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:11.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:11.247+0000 7f7084a00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f70800d3cc0 con 0x7f70640039b0 2024-05-26T22:54:11.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:11.247+0000 7f7084a00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f70800b6420 con 0x7f7080161420 2024-05-26T22:54:11.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:11.247+0000 7f7084a00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f70800d3e40 con 0x7f7080153c60 2024-05-26T22:54:11.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:11.247+0000 7f707f400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f70640039b0 0x7f70801461d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:11.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:11.247+0000 7f707f400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f70640039b0 0x7f70801461d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46360/0 (socket says 172.21.15.131:46360) 2024-05-26T22:54:11.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:11.247+0000 7f707f400640 1 -- 172.21.15.131:0/2623775195 learned_addr learned my addr 172.21.15.131:0/2623775195 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:54:11.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:11.248+0000 7f707f400640 1 -- 172.21.15.131:0/2623775195 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7080153c60 msgr2=0x7f70801f78c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:11.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:11.248+0000 7f7077e00640 1 --2- 172.21.15.131:0/2623775195 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7080153c60 0x7f70801f78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:11.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:11.248+0000 7f707f400640 1 --2- 172.21.15.131:0/2623775195 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7080153c60 0x7f70801f78c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:11.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:11.248+0000 7f707f400640 1 -- 172.21.15.131:0/2623775195 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7080161420 msgr2=0x7f70801f7e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:11.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:11.248+0000 7f707f400640 1 --2- 172.21.15.131:0/2623775195 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7080161420 0x7f70801f7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:11.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:11.248+0000 7f707f400640 1 -- 172.21.15.131:0/2623775195 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70801606d0 con 0x7f70640039b0 2024-05-26T22:54:11.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:11.248+0000 7f707f400640 1 --2- 172.21.15.131:0/2623775195 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f70640039b0 0x7f70801461d0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f7070009ac0 tx=0x7f7070004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:11.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:11.248+0000 7f7077400640 1 -- 172.21.15.131:0/2623775195 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7070040020 con 0x7f70640039b0 2024-05-26T22:54:11.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:11.248+0000 7f7077400640 1 -- 172.21.15.131:0/2623775195 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7070003b40 con 0x7f70640039b0 2024-05-26T22:54:11.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:11.248+0000 7f7077400640 1 -- 172.21.15.131:0/2623775195 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7070036040 con 0x7f70640039b0 2024-05-26T22:54:11.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:11.248+0000 7f7084a00640 1 -- 172.21.15.131:0/2623775195 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7080160940 con 0x7f70640039b0 2024-05-26T22:54:11.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:11.249+0000 7f7084a00640 1 -- 172.21.15.131:0/2623775195 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70801540f0 con 0x7f70640039b0 2024-05-26T22:54:11.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:11.250+0000 7f7084a00640 1 -- 172.21.15.131:0/2623775195 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70800694e0 con 0x7f70640039b0 2024-05-26T22:54:11.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:11.251+0000 7f7077400640 1 -- 172.21.15.131:0/2623775195 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7070043020 con 0x7f70640039b0 2024-05-26T22:54:11.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:11.251+0000 7f7077400640 1 --2- 172.21.15.131:0/2623775195 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f705c06be30 0x7f705c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:11.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:11.252+0000 7f7077400640 1 -- 172.21.15.131:0/2623775195 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5738+0+0 (secure 0 0 0) 0x7f70700aaf20 con 0x7f70640039b0 2024-05-26T22:54:11.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:11.252+0000 7f7077e00640 1 --2- 172.21.15.131:0/2623775195 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f705c06be30 0x7f705c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:11.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:11.252+0000 7f7077e00640 1 --2- 172.21.15.131:0/2623775195 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f705c06be30 0x7f705c06e2f0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f7068001d00 tx=0x7f7068001c90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:11.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:11.256+0000 7f7077400640 1 -- 172.21.15.131:0/2623775195 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f707004a070 con 0x7f70640039b0 2024-05-26T22:54:11.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:11.492+0000 7f7084a00640 1 -- 172.21.15.131:0/2623775195 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f70800b38a0 con 0x7f70640039b0 2024-05-26T22:54:11.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:11.493+0000 7f7077400640 1 -- 172.21.15.131:0/2623775195 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) ==== 94+0+5227 (secure 0 0 0) 0x7f70700044e0 con 0x7f70640039b0 2024-05-26T22:54:11.493 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:54:11.493 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":83,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5533,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/993930297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6835","nonce":993930297},{"type":"v1","addr":"172.21.15.131:6837","nonce":993930297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-26T22:53:52.862328+0000","modified":"2024-05-26T22:54:11.157696+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7365,"mds_1":5548,"mds_2":5545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5545":{"gid":5545,"name":"b","rank":2,"incarnation":82,"state":"up:active","state_seq":7,"addr":"172.21.15.187:6834/3480101957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3480101957},{"type":"v1","addr":"172.21.15.187:6834","nonce":3480101957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5548":{"gid":5548,"name":"d","rank":1,"incarnation":80,"state":"up:active","state_seq":7,"addr":"172.21.15.187:6835/567389303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6833","nonce":567389303},{"type":"v1","addr":"172.21.15.187:6835","nonce":567389303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7365":{"gid":7365,"name":"a","rank":0,"incarnation":74,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6836/3411392567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":3411392567},{"type":"v1","addr":"172.21.15.131:6836","nonce":3411392567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7365,"qdb_cluster":[5545,7365,5548]},"id":6}]} 2024-05-26T22:54:11.494 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 83 2024-05-26T22:54:11.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:11.496+0000 7f7084a00640 1 -- 172.21.15.131:0/2623775195 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f705c06be30 msgr2=0x7f705c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:11.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:11.496+0000 7f7084a00640 1 --2- 172.21.15.131:0/2623775195 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f705c06be30 0x7f705c06e2f0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f7068001d00 tx=0x7f7068001c90 comp rx=0 tx=0).stop 2024-05-26T22:54:11.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:11.496+0000 7f7084a00640 1 -- 172.21.15.131:0/2623775195 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f70640039b0 msgr2=0x7f70801461d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:11.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:11.496+0000 7f7084a00640 1 --2- 172.21.15.131:0/2623775195 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f70640039b0 0x7f70801461d0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f7070009ac0 tx=0x7f7070004730 comp rx=0 tx=0).stop 2024-05-26T22:54:11.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:11.496+0000 7f7084a00640 1 -- 172.21.15.131:0/2623775195 shutdown_connections 2024-05-26T22:54:11.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:11.496+0000 7f7084a00640 1 --2- 172.21.15.131:0/2623775195 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7080153c60 0x7f70801f78c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:11.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:11.496+0000 7f7084a00640 1 --2- 172.21.15.131:0/2623775195 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f705c06be30 0x7f705c06e2f0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:11.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:11.496+0000 7f7084a00640 1 --2- 172.21.15.131:0/2623775195 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7080161420 0x7f70801f7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:11.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:11.496+0000 7f7084a00640 1 --2- 172.21.15.131:0/2623775195 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f70640039b0 0x7f70801461d0 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:11.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:11.496+0000 7f7084a00640 1 -- 172.21.15.131:0/2623775195 >> 172.21.15.131:0/2623775195 conn(0x7f7080073070 msgr2=0x7f70800d4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:11.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:11.496+0000 7f7084a00640 1 -- 172.21.15.131:0/2623775195 shutdown_connections 2024-05-26T22:54:11.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:11.496+0000 7f7084a00640 1 -- 172.21.15.131:0/2623775195 wait complete. 2024-05-26T22:54:11.915 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 83, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-26T22:53:52.862328+0000', 'modified': '2024-05-26T22:54:11.157696+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 7365, 'mds_1': 5548, 'mds_2': 5545}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5545': {'gid': 5545, 'name': 'b', 'rank': 2, 'incarnation': 82, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.187:6834/3480101957', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.187:6832', 'nonce': 3480101957}, {'type': 'v1', 'addr': '172.21.15.187:6834', 'nonce': 3480101957}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_5548': {'gid': 5548, 'name': 'd', 'rank': 1, 'incarnation': 80, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.187:6835/567389303', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.187:6833', 'nonce': 567389303}, {'type': 'v1', 'addr': '172.21.15.187:6835', 'nonce': 567389303}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_7365': {'gid': 7365, 'name': 'a', 'rank': 0, 'incarnation': 74, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.131:6836/3411392567', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6834', 'nonce': 3411392567}, {'type': 'v1', 'addr': '172.21.15.131:6836', 'nonce': 3411392567}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7365, 'qdb_cluster': [5545, 7365, 5548]} 2024-05-26T22:54:11.915 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-05-26T22:54:11.915 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-05-26T22:54:12.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.099+0000 7fb6d3a00640 1 Processor -- start 2024-05-26T22:54:12.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.099+0000 7fb6d3a00640 1 -- start start 2024-05-26T22:54:12.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.101+0000 7fb6d3a00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb6cc15e2f0 0x7fb6cc0ae810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:12.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.101+0000 7fb6d3a00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6cc0b4540 con 0x7fb6cc0aed50 2024-05-26T22:54:12.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.101+0000 7fb6d3a00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6cc0b4820 con 0x7fb6cc15e6d0 2024-05-26T22:54:12.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.101+0000 7fb6d3a00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb6cc0b4b00 con 0x7fb6cc15e2f0 2024-05-26T22:54:12.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.101+0000 7fb6d1a00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fb6cc0aed50 0x7fb6cc0b4d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:38072/0 (socket says 172.21.15.131:38072) 2024-05-26T22:54:12.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.102+0000 7fb6d1a00640 1 -- 172.21.15.131:0/3902978982 learned_addr learned my addr 172.21.15.131:0/3902978982 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:54:12.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.102+0000 7fb6d1000640 1 --2- 172.21.15.131:0/3902978982 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb6cc15e2f0 0x7fb6cc0ae810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:12.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.102+0000 7fb6cb400640 1 -- 172.21.15.131:0/3902978982 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4112219283 0 0) 0x7fb6cc0b4540 con 0x7fb6cc0aed50 2024-05-26T22:54:12.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.102+0000 7fb6cb400640 1 -- 172.21.15.131:0/3902978982 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb6b4003520 con 0x7fb6cc0aed50 2024-05-26T22:54:12.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.102+0000 7fb6cb400640 1 -- 172.21.15.131:0/3902978982 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3414975129 0 0) 0x7fb6b4003520 con 0x7fb6cc0aed50 2024-05-26T22:54:12.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.102+0000 7fb6cb400640 1 -- 172.21.15.131:0/3902978982 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb6cc15e2f0 msgr2=0x7fb6cc0ae810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:12.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.102+0000 7fb6cb400640 1 --2- 172.21.15.131:0/3902978982 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb6cc15e2f0 0x7fb6cc0ae810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:12.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.102+0000 7fb6cb400640 1 -- 172.21.15.131:0/3902978982 >> v1:172.21.15.187:6789/0 conn(0x7fb6cc15e6d0 legacy=0x7fb6cc0b3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:12.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.102+0000 7fb6cb400640 1 -- 172.21.15.131:0/3902978982 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6cc06df70 con 0x7fb6cc0aed50 2024-05-26T22:54:12.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.102+0000 7fb6cb400640 1 -- 172.21.15.131:0/3902978982 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7fb6c40019f0 con 0x7fb6cc0aed50 2024-05-26T22:54:12.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.102+0000 7fb6cb400640 1 -- 172.21.15.131:0/3902978982 >> v1:172.21.15.131:6789/0 conn(0x7fb6cc0aed50 legacy=0x7fb6cc0b4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:12.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.102+0000 7fb6cb400640 1 --2- 172.21.15.131:0/3902978982 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb6b40039e0 0x7fb6b4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:12.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.102+0000 7fb6cb400640 1 --2- 172.21.15.131:0/3902978982 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6b4004800 0x7fb6b4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:12.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.102+0000 7fb6cb400640 1 --2- 172.21.15.131:0/3902978982 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb6b40051c0 0x7fb6b400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:12.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.103+0000 7fb6cb400640 1 -- 172.21.15.131:0/3902978982 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fb6cc0b4b00 con 0x7fb6b4004800 2024-05-26T22:54:12.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.103+0000 7fb6cb400640 1 -- 172.21.15.131:0/3902978982 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb6b400c060 con 0x7fb6b40039e0 2024-05-26T22:54:12.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.103+0000 7fb6cb400640 1 -- 172.21.15.131:0/3902978982 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb6b400c260 con 0x7fb6b40051c0 2024-05-26T22:54:12.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.103+0000 7fb6cbe00640 1 --2- 172.21.15.131:0/3902978982 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6b4004800 0x7fb6b4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:12.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.103+0000 7fb6d1000640 1 --2- 172.21.15.131:0/3902978982 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb6b40039e0 0x7fb6b4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:12.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.103+0000 7fb6d1a00640 1 --2- 172.21.15.131:0/3902978982 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb6b40051c0 0x7fb6b400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:12.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.103+0000 7fb6cbe00640 1 -- 172.21.15.131:0/3902978982 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb6b40051c0 msgr2=0x7fb6b400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:12.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.103+0000 7fb6cbe00640 1 --2- 172.21.15.131:0/3902978982 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb6b40051c0 0x7fb6b400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:12.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.103+0000 7fb6cbe00640 1 -- 172.21.15.131:0/3902978982 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb6b40039e0 msgr2=0x7fb6b4003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:12.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.103+0000 7fb6cbe00640 1 --2- 172.21.15.131:0/3902978982 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb6b40039e0 0x7fb6b4003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:12.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.103+0000 7fb6cbe00640 1 -- 172.21.15.131:0/3902978982 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6b4003520 con 0x7fb6b4004800 2024-05-26T22:54:12.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.103+0000 7fb6d1000640 1 --2- 172.21.15.131:0/3902978982 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb6b40039e0 0x7fb6b4003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:54:12.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.103+0000 7fb6d1a00640 1 --2- 172.21.15.131:0/3902978982 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb6b40051c0 0x7fb6b400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:54:12.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.104+0000 7fb6cbe00640 1 --2- 172.21.15.131:0/3902978982 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6b4004800 0x7fb6b4004c80 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7fb6b8001750 tx=0x7fb6b4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2b8e36c85d5ab80 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:12.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.105+0000 7fb6cb400640 1 -- 172.21.15.131:0/3902978982 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6b8001370 con 0x7fb6b4004800 2024-05-26T22:54:12.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.105+0000 7fb6cb400640 1 -- 172.21.15.131:0/3902978982 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6b80052d0 con 0x7fb6b4004800 2024-05-26T22:54:12.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.105+0000 7fb6cb400640 1 -- 172.21.15.131:0/3902978982 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6b802ab40 con 0x7fb6b4004800 2024-05-26T22:54:12.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.105+0000 7fb6d3a00640 1 -- 172.21.15.131:0/3902978982 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6b4004800 msgr2=0x7fb6b4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:12.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.105+0000 7fb6d3a00640 1 --2- 172.21.15.131:0/3902978982 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6b4004800 0x7fb6b4004c80 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7fb6b8001750 tx=0x7fb6b4005830 comp rx=0 tx=0).stop 2024-05-26T22:54:12.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.105+0000 7fb6d1a00640 1 -- 172.21.15.131:0/3902978982 reap_dead start 2024-05-26T22:54:12.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.105+0000 7fb6d3a00640 1 -- 172.21.15.131:0/3902978982 shutdown_connections 2024-05-26T22:54:12.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.105+0000 7fb6d3a00640 1 -- 172.21.15.131:0/3902978982 >> 172.21.15.131:0/3902978982 conn(0x7fb6cc0b27c0 msgr2=0x7fb6cc0b2bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:12.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.105+0000 7fb6d3a00640 1 -- 172.21.15.131:0/3902978982 shutdown_connections 2024-05-26T22:54:12.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.106+0000 7fb6d3a00640 1 -- 172.21.15.131:0/3902978982 wait complete. 2024-05-26T22:54:12.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.108+0000 7fb6d3a00640 1 Processor -- start 2024-05-26T22:54:12.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.108+0000 7fb6d3a00640 1 -- start start 2024-05-26T22:54:12.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.110+0000 7fb6d3a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6b4004800 0x7fb6cc071e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:12.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.110+0000 7fb6d3a00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb6cc15b1c0 0x7fb6cc15b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:12.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.110+0000 7fb6d3a00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb6cc1e7ed0 0x7fb6cc1e8350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:12.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.110+0000 7fb6d3a00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fb6cc0b41d0 con 0x7fb6b4004800 2024-05-26T22:54:12.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.110+0000 7fb6d3a00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb6cc0b5b30 con 0x7fb6cc1e7ed0 2024-05-26T22:54:12.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.110+0000 7fb6d3a00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb6cc0b5470 con 0x7fb6cc15b1c0 2024-05-26T22:54:12.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.110+0000 7fb6d1000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6b4004800 0x7fb6cc071e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:12.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.110+0000 7fb6d1000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6b4004800 0x7fb6cc071e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46364/0 (socket says 172.21.15.131:46364) 2024-05-26T22:54:12.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.110+0000 7fb6d1000640 1 -- 172.21.15.131:0/1689707981 learned_addr learned my addr 172.21.15.131:0/1689707981 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:54:12.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.110+0000 7fb6cbe00640 1 --2- 172.21.15.131:0/1689707981 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb6cc15b1c0 0x7fb6cc15b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:12.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.111+0000 7fb6d1a00640 1 --2- 172.21.15.131:0/1689707981 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb6cc1e7ed0 0x7fb6cc1e8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:12.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.111+0000 7fb6d1000640 1 -- 172.21.15.131:0/1689707981 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb6cc15b1c0 msgr2=0x7fb6cc15b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:12.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.111+0000 7fb6d1000640 1 --2- 172.21.15.131:0/1689707981 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb6cc15b1c0 0x7fb6cc15b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:12.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.111+0000 7fb6d1000640 1 -- 172.21.15.131:0/1689707981 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb6cc1e7ed0 msgr2=0x7fb6cc1e8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:12.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.111+0000 7fb6d1000640 1 --2- 172.21.15.131:0/1689707981 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb6cc1e7ed0 0x7fb6cc1e8350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:12.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.111+0000 7fb6d1000640 1 -- 172.21.15.131:0/1689707981 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6cc1f49d0 con 0x7fb6b4004800 2024-05-26T22:54:12.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.111+0000 7fb6d1a00640 1 --2- 172.21.15.131:0/1689707981 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb6cc1e7ed0 0x7fb6cc1e8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:54:12.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.111+0000 7fb6d1000640 1 --2- 172.21.15.131:0/1689707981 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6b4004800 0x7fb6cc071e10 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7fb6c0010830 tx=0x7fb6c0010cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:12.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.111+0000 7fb6c8c00640 1 -- 172.21.15.131:0/1689707981 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c000f040 con 0x7fb6b4004800 2024-05-26T22:54:12.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.111+0000 7fb6c8c00640 1 -- 172.21.15.131:0/1689707981 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6c000eb60 con 0x7fb6b4004800 2024-05-26T22:54:12.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.111+0000 7fb6c8c00640 1 -- 172.21.15.131:0/1689707981 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c001c400 con 0x7fb6b4004800 2024-05-26T22:54:12.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.111+0000 7fb6d3a00640 1 -- 172.21.15.131:0/1689707981 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6cc1f4cb0 con 0x7fb6b4004800 2024-05-26T22:54:12.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.112+0000 7fb6d3a00640 1 -- 172.21.15.131:0/1689707981 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6cc1f5270 con 0x7fb6b4004800 2024-05-26T22:54:12.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.114+0000 7fb6c8c00640 1 -- 172.21.15.131:0/1689707981 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb6c001c5a0 con 0x7fb6b4004800 2024-05-26T22:54:12.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.115+0000 7fb6c8c00640 1 --2- 172.21.15.131:0/1689707981 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb6ac06be30 0x7fb6ac06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:12.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.115+0000 7fb6c8c00640 1 -- 172.21.15.131:0/1689707981 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5738+0+0 (secure 0 0 0) 0x7fb6c008b5e0 con 0x7fb6b4004800 2024-05-26T22:54:12.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.115+0000 7fb6cbe00640 1 --2- 172.21.15.131:0/1689707981 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb6ac06be30 0x7fb6ac06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:12.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.115+0000 7fb6d67f5740 1 Processor -- start 2024-05-26T22:54:12.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.115+0000 7fb6d67f5740 1 -- start start 2024-05-26T22:54:12.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.116+0000 7fb6cbe00640 1 --2- 172.21.15.131:0/1689707981 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb6ac06be30 0x7fb6ac06e2f0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fb6b8032000 tx=0x7fb6b802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:12.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.116+0000 7fb6d67f5740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a6144cfc70 0x55a614482890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:12.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.116+0000 7fb6d67f5740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a6144b2d90 0x55a6144b2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:12.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.116+0000 7fb6d67f5740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a6144b2650 0x55a614487460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:12.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.116+0000 7fb6d67f5740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55a6143ea610 con 0x55a6144b2650 2024-05-26T22:54:12.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.116+0000 7fb6d67f5740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55a6143925c0 con 0x55a6144b2d90 2024-05-26T22:54:12.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.116+0000 7fb6d67f5740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x55a6143e65d0 con 0x55a6144cfc70 2024-05-26T22:54:12.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.117+0000 7fb6d1a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a6144b2650 0x55a614487460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:12.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.117+0000 7fb6d1a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a6144b2650 0x55a614487460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46376/0 (socket says 172.21.15.131:46376) 2024-05-26T22:54:12.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.117+0000 7fb6d1a00640 1 -- 172.21.15.131:0/1189763438 learned_addr learned my addr 172.21.15.131:0/1189763438 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:54:12.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.117+0000 7fb6d1000640 1 --2- 172.21.15.131:0/1189763438 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a6144cfc70 0x55a614482890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:12.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.117+0000 7fb6d1a00640 1 -- 172.21.15.131:0/1189763438 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a6144cfc70 msgr2=0x55a614482890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:12.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.117+0000 7fb6d1a00640 1 --2- 172.21.15.131:0/1189763438 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a6144cfc70 0x55a614482890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:12.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.117+0000 7fb6d1a00640 1 -- 172.21.15.131:0/1189763438 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a6144b2d90 msgr2=0x55a6144b2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:12.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.117+0000 7fb6cbe00640 1 --2- 172.21.15.131:0/1189763438 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a6144b2d90 0x55a6144b2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:12.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.117+0000 7fb6d1a00640 1 --2- 172.21.15.131:0/1189763438 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a6144b2d90 0x55a6144b2110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:12.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.117+0000 7fb6d1a00640 1 -- 172.21.15.131:0/1189763438 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6140fe2d0 con 0x55a6144b2650 2024-05-26T22:54:12.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.117+0000 7fb6d1a00640 1 --2- 172.21.15.131:0/1189763438 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a6144b2650 0x55a614487460 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7fb6c40021a0 tx=0x7fb6c400e590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=860fd817dc2ad259 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:12.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.118+0000 7fb6bd600640 1 -- 172.21.15.131:0/1189763438 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c4018070 con 0x55a6144b2650 2024-05-26T22:54:12.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.118+0000 7fb6bd600640 1 -- 172.21.15.131:0/1189763438 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6c4004d60 con 0x55a6144b2650 2024-05-26T22:54:12.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.118+0000 7fb6bd600640 1 -- 172.21.15.131:0/1189763438 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c4007270 con 0x55a6144b2650 2024-05-26T22:54:12.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.118+0000 7fb6d67f5740 1 -- 172.21.15.131:0/1189763438 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a6144b2650 msgr2=0x55a614487460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:12.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.118+0000 7fb6d67f5740 1 --2- 172.21.15.131:0/1189763438 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a6144b2650 0x55a614487460 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7fb6c40021a0 tx=0x7fb6c400e590 comp rx=0 tx=0).stop 2024-05-26T22:54:12.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.118+0000 7fb6d67f5740 1 -- 172.21.15.131:0/1189763438 shutdown_connections 2024-05-26T22:54:12.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.118+0000 7fb6d67f5740 1 --2- 172.21.15.131:0/1189763438 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a6144b2650 0x55a614487460 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:12.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.118+0000 7fb6d67f5740 1 --2- 172.21.15.131:0/1189763438 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a6144b2d90 0x55a6144b2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:12.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.118+0000 7fb6d67f5740 1 --2- 172.21.15.131:0/1189763438 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a6144cfc70 0x55a614482890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:12.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.118+0000 7fb6d67f5740 1 -- 172.21.15.131:0/1189763438 >> 172.21.15.131:0/1189763438 conn(0x55a614484b00 msgr2=0x55a6144cd110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:12.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.118+0000 7fb6d67f5740 1 -- 172.21.15.131:0/1189763438 shutdown_connections 2024-05-26T22:54:12.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.118+0000 7fb6d67f5740 1 -- 172.21.15.131:0/1189763438 wait complete. 2024-05-26T22:54:12.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.119+0000 7fb6d67f5740 1 -- start start 2024-05-26T22:54:12.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.119+0000 7fb6d67f5740 1 Processor -- start 2024-05-26T22:54:12.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.120+0000 7fb6d67f5740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a6144b2d90 0x55a6144a2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:12.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.120+0000 7fb6d67f5740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a6144cfc70 0x55a6144e0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:12.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.120+0000 7fb6d67f5740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a6144e0da0 0x55a6144ce060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:12.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.120+0000 7fb6d67f5740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55a614486a20 con 0x55a6144e0da0 2024-05-26T22:54:12.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.120+0000 7fb6d67f5740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55a6143a5910 con 0x55a6144cfc70 2024-05-26T22:54:12.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.120+0000 7fb6d67f5740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x55a614485e10 con 0x55a6144b2d90 2024-05-26T22:54:12.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.120+0000 7fb6d67f5740 10 client.0 _ll_register_callbacks cb 0x55a6143e1410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-26T22:54:12.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.121+0000 7fb6d1a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a6144e0da0 0x55a6144ce060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:12.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.121+0000 7fb6d1a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a6144e0da0 0x55a6144ce060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46388/0 (socket says 172.21.15.131:46388) 2024-05-26T22:54:12.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.121+0000 7fb6d1a00640 1 -- 172.21.15.131:0/3471671857 learned_addr learned my addr 172.21.15.131:0/3471671857 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:54:12.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.121+0000 7fb6d1000640 1 --2- 172.21.15.131:0/3471671857 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a6144b2d90 0x55a6144a2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:12.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.121+0000 7fb6cbe00640 1 --2- 172.21.15.131:0/3471671857 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a6144cfc70 0x55a6144e0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:12.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.121+0000 7fb6d1a00640 1 -- 172.21.15.131:0/3471671857 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a6144b2d90 msgr2=0x55a6144a2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:12.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.121+0000 7fb6d1a00640 1 --2- 172.21.15.131:0/3471671857 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a6144b2d90 0x55a6144a2d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:12.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.121+0000 7fb6d1a00640 1 -- 172.21.15.131:0/3471671857 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a6144cfc70 msgr2=0x55a6144e0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:12.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.121+0000 7fb6d1a00640 1 --2- 172.21.15.131:0/3471671857 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a6144cfc70 0x55a6144e0860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:12.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.121+0000 7fb6d1a00640 1 -- 172.21.15.131:0/3471671857 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6141c86b0 con 0x55a6144e0da0 2024-05-26T22:54:12.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.121+0000 7fb6cbe00640 1 --2- 172.21.15.131:0/3471671857 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a6144cfc70 0x55a6144e0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:54:12.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.121+0000 7fb6d1a00640 1 --2- 172.21.15.131:0/3471671857 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a6144e0da0 0x55a6144ce060 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fb6c4000f80 tx=0x7fb6cc0b49f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7ae5cbdd1e2280 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:12.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.121+0000 7fb6b2000640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-26T22:54:12.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.122+0000 7fb6b2000640 1 -- 172.21.15.131:0/3471671857 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c4018040 con 0x55a6144e0da0 2024-05-26T22:54:12.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.122+0000 7fb6b2000640 1 -- 172.21.15.131:0/3471671857 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 5738+0+0 (secure 0 0 0) 0x7fb6c4007820 con 0x55a6144e0da0 2024-05-26T22:54:12.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.122+0000 7fb6d67f5740 1 -- 172.21.15.131:0/3471671857 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6c4004d60 con 0x55a6144e0da0 2024-05-26T22:54:12.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.122+0000 7fb6b2000640 1 -- 172.21.15.131:0/3471671857 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7fb6c402b050 con 0x55a6144e0da0 2024-05-26T22:54:12.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.122+0000 7fb6d67f5740 10 client.7599 fetch_fsmap learned FSMap version 83 2024-05-26T22:54:12.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.122+0000 7fb6d67f5740 1 -- 172.21.15.131:0/3471671857 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=2+}) -- 0x55a6144b38e0 con 0x55a6144e0da0 2024-05-26T22:54:12.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.122+0000 7fb6b2000640 1 -- 172.21.15.131:0/3471671857 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6c4004d60 con 0x55a6144e0da0 2024-05-26T22:54:12.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.122+0000 7fb6b2000640 1 -- 172.21.15.131:0/3471671857 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 83) ==== 3534+0+0 (secure 0 0 0) 0x7fb6c402e020 con 0x55a6144e0da0 2024-05-26T22:54:12.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.122+0000 7fb6d67f5740 10 client.7599 fetch_fsmap finished waiting for FSMap version 83 2024-05-26T22:54:12.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.122+0000 7fb6d67f5740 10 client.7599 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-26T22:54:12.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.122+0000 7fb6d67f5740 20 client.7599 populate_metadata read hostname 'smithi131' 2024-05-26T22:54:12.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.122+0000 7fb6d67f5740 1 --2- 172.21.15.131:0/3471671857 >> [v2:172.21.15.187:6832/3480101957,v1:172.21.15.187:6834/3480101957] conn(0x55a6144dd6b0 0x55a6144dda90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:12.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.122+0000 7fb6d67f5740 4 client.7599 mds_command: new command op to 5545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-26T22:54:12.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.122+0000 7fb6d67f5740 1 -- 172.21.15.131:0/3471671857 --> [v2:172.21.15.187:6832/3480101957,v1:172.21.15.187:6834/3480101957] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a6143f7d40 con 0x55a6144dd6b0 2024-05-26T22:54:12.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.123+0000 7fb6d1000640 1 --2- 172.21.15.131:0/3471671857 >> [v2:172.21.15.187:6832/3480101957,v1:172.21.15.187:6834/3480101957] conn(0x55a6144dd6b0 0x55a6144dda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:12.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.124+0000 7fb6b2000640 10 client.7599 ms_handle_connect on v2:172.21.15.187:6832/3480101957 2024-05-26T22:54:12.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.124+0000 7fb6d1000640 1 --2- 172.21.15.131:0/3471671857 >> [v2:172.21.15.187:6832/3480101957,v1:172.21.15.187:6834/3480101957] conn(0x55a6144dd6b0 0x55a6144dda90 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8a00ff36e6e54400 server_cookie=db3f808c5f267f7f in_seq=0 out_seq=0 2024-05-26T22:54:12.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.127+0000 7fb6b2000640 1 -- 172.21.15.131:0/3471671857 <== mds.2 v2:172.21.15.187:6832/3480101957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a6143f7d40 con 0x55a6144dd6b0 2024-05-26T22:54:12.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.127+0000 7fb6b2000640 10 client.7599 handle_command_reply: tid=0 2024-05-26T22:54:12.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.127+0000 7fb6d67f5740 1 client.7599 shutdown 2024-05-26T22:54:12.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.127+0000 7fb6d67f5740 1 -- 172.21.15.131:0/3471671857 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a6144e0da0 msgr2=0x55a6144ce060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:12.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.127+0000 7fb6d67f5740 1 --2- 172.21.15.131:0/3471671857 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a6144e0da0 0x55a6144ce060 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fb6c4000f80 tx=0x7fb6cc0b49f0 comp rx=0 tx=0).stop 2024-05-26T22:54:12.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.127+0000 7fb6d67f5740 1 -- 172.21.15.131:0/3471671857 shutdown_connections 2024-05-26T22:54:12.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.127+0000 7fb6d67f5740 1 --2- 172.21.15.131:0/3471671857 >> [v2:172.21.15.187:6832/3480101957,v1:172.21.15.187:6834/3480101957] conn(0x55a6144dd6b0 0x55a6144dda90 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:12.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.127+0000 7fb6d67f5740 1 --2- 172.21.15.131:0/3471671857 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a6144e0da0 0x55a6144ce060 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:12.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.127+0000 7fb6d67f5740 1 --2- 172.21.15.131:0/3471671857 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a6144cfc70 0x55a6144e0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:12.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.127+0000 7fb6d67f5740 1 --2- 172.21.15.131:0/3471671857 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a6144b2d90 0x55a6144a2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:12.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.127+0000 7fb6d67f5740 1 -- 172.21.15.131:0/3471671857 >> 172.21.15.131:0/3471671857 conn(0x55a6144b2650 msgr2=0x55a6144d0530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:12.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.127+0000 7fb6b2000640 0 client.7599 ms_handle_reset on v2:172.21.15.187:6832/3480101957 2024-05-26T22:54:12.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.127+0000 7fb6d67f5740 1 -- 172.21.15.131:0/3471671857 shutdown_connections 2024-05-26T22:54:12.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.127+0000 7fb6d67f5740 1 -- 172.21.15.131:0/3471671857 wait complete. 2024-05-26T22:54:12.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.127+0000 7fb6d67f5740 20 client.7599 trim_cache size 0 max 16384 2024-05-26T22:54:12.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.146+0000 7fb6d67f5740 1 Processor -- start 2024-05-26T22:54:12.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.146+0000 7fb6d67f5740 1 -- start start 2024-05-26T22:54:12.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.147+0000 7fb6d67f5740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a6144b2d90 0x55a6144c6770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:12.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.147+0000 7fb6d67f5740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a6144cfc70 0x55a6144cf2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:12.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.147+0000 7fb6d67f5740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a6144e0da0 0x55a6144b56e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:12.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.147+0000 7fb6d67f5740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fb6cc200560 con 0x55a6144b2d90 2024-05-26T22:54:12.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.147+0000 7fb6d67f5740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb6cc200c30 con 0x55a6144cfc70 2024-05-26T22:54:12.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.147+0000 7fb6d67f5740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb6cc1f8d60 con 0x55a6144e0da0 2024-05-26T22:54:12.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.147+0000 7fb6d1000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a6144b2d90 0x55a6144c6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:12.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.148+0000 7fb6d1000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a6144b2d90 0x55a6144c6770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46390/0 (socket says 172.21.15.131:46390) 2024-05-26T22:54:12.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.148+0000 7fb6d1000640 1 -- 172.21.15.131:0/1559276562 learned_addr learned my addr 172.21.15.131:0/1559276562 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:54:12.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.148+0000 7fb6d1000640 1 -- 172.21.15.131:0/1559276562 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a6144e0da0 msgr2=0x55a6144b56e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:12.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.148+0000 7fb6d1000640 1 --2- 172.21.15.131:0/1559276562 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a6144e0da0 0x55a6144b56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:12.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.148+0000 7fb6d1000640 1 -- 172.21.15.131:0/1559276562 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a6144cfc70 msgr2=0x55a6144cf2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:12.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.148+0000 7fb6d1000640 1 --2- 172.21.15.131:0/1559276562 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a6144cfc70 0x55a6144cf2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:12.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.148+0000 7fb6d1000640 1 -- 172.21.15.131:0/1559276562 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6144c6cb0 con 0x55a6144b2d90 2024-05-26T22:54:12.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.148+0000 7fb6d1000640 1 --2- 172.21.15.131:0/1559276562 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a6144b2d90 0x55a6144c6770 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fb6c0055ce0 tx=0x7fb6c0060040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89fa669104e031fd server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:12.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.148+0000 7fb6bcc00640 1 -- 172.21.15.131:0/1559276562 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c0089070 con 0x55a6144b2d90 2024-05-26T22:54:12.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.148+0000 7fb6bcc00640 1 -- 172.21.15.131:0/1559276562 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6c00623b0 con 0x55a6144b2d90 2024-05-26T22:54:12.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.148+0000 7fb6bcc00640 1 -- 172.21.15.131:0/1559276562 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c0021040 con 0x55a6144b2d90 2024-05-26T22:54:12.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.149+0000 7fb6d67f5740 1 -- 172.21.15.131:0/1559276562 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a6144b2d90 msgr2=0x55a6144c6770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:12.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.149+0000 7fb6d67f5740 1 --2- 172.21.15.131:0/1559276562 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a6144b2d90 0x55a6144c6770 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fb6c0055ce0 tx=0x7fb6c0060040 comp rx=0 tx=0).stop 2024-05-26T22:54:12.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.149+0000 7fb6d67f5740 1 -- 172.21.15.131:0/1559276562 shutdown_connections 2024-05-26T22:54:12.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.149+0000 7fb6d67f5740 1 --2- 172.21.15.131:0/1559276562 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a6144e0da0 0x55a6144b56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:12.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.149+0000 7fb6d67f5740 1 --2- 172.21.15.131:0/1559276562 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a6144cfc70 0x55a6144cf2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:12.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.149+0000 7fb6d67f5740 1 --2- 172.21.15.131:0/1559276562 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a6144b2d90 0x55a6144c6770 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:12.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.149+0000 7fb6d67f5740 1 -- 172.21.15.131:0/1559276562 >> 172.21.15.131:0/1559276562 conn(0x55a6144b2650 msgr2=0x55a6144a21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:12.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.149+0000 7fb6d67f5740 1 -- 172.21.15.131:0/1559276562 shutdown_connections 2024-05-26T22:54:12.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.149+0000 7fb6d67f5740 1 -- 172.21.15.131:0/1559276562 wait complete. 2024-05-26T22:54:12.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.150+0000 7fb6d67f5740 1 -- start start 2024-05-26T22:54:12.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.150+0000 7fb6d67f5740 1 Processor -- start 2024-05-26T22:54:12.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.151+0000 7fb6d67f5740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a6144b2d90 0x55a6144ef9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:12.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.151+0000 7fb6d67f5740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a6144cfc70 0x55a6144dc950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:12.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.151+0000 7fb6d67f5740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a6144e0da0 0x55a6144d0610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:12.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.151+0000 7fb6d67f5740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55a6144b4410 con 0x55a6144b2d90 2024-05-26T22:54:12.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.151+0000 7fb6d67f5740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55a6144b3bc0 con 0x55a6144e0da0 2024-05-26T22:54:12.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.151+0000 7fb6d67f5740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x55a6144c45d0 con 0x55a6144cfc70 2024-05-26T22:54:12.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.151+0000 7fb6d67f5740 10 client.0 _ll_register_callbacks cb 0x7fb6c4016070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-26T22:54:12.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.151+0000 7fb6d1000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a6144b2d90 0x55a6144ef9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:12.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.151+0000 7fb6d1000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a6144b2d90 0x55a6144ef9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46400/0 (socket says 172.21.15.131:46400) 2024-05-26T22:54:12.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.151+0000 7fb6d1000640 1 -- 172.21.15.131:0/1095137337 learned_addr learned my addr 172.21.15.131:0/1095137337 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:54:12.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.151+0000 7fb6d1000640 1 -- 172.21.15.131:0/1095137337 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a6144cfc70 msgr2=0x55a6144dc950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:12.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.151+0000 7fb6d1000640 1 --2- 172.21.15.131:0/1095137337 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a6144cfc70 0x55a6144dc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:12.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.151+0000 7fb6d1000640 1 -- 172.21.15.131:0/1095137337 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a6144e0da0 msgr2=0x55a6144d0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:12.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.151+0000 7fb6d1000640 1 --2- 172.21.15.131:0/1095137337 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a6144e0da0 0x55a6144d0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:12.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.151+0000 7fb6d1000640 1 -- 172.21.15.131:0/1095137337 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6144db8b0 con 0x55a6144b2d90 2024-05-26T22:54:12.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.152+0000 7fb6d1000640 1 --2- 172.21.15.131:0/1095137337 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a6144b2d90 0x55a6144ef9f0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fb6c000e540 tx=0x7fb6c005d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=778ea1810be1752b server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:12.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.152+0000 7fb6b2000640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-26T22:54:12.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.152+0000 7fb6b2000640 1 -- 172.21.15.131:0/1095137337 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c0089070 con 0x55a6144b2d90 2024-05-26T22:54:12.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.152+0000 7fb6b2000640 1 -- 172.21.15.131:0/1095137337 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 5738+0+0 (secure 0 0 0) 0x7fb6c0061f20 con 0x55a6144b2d90 2024-05-26T22:54:12.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.152+0000 7fb6d67f5740 1 -- 172.21.15.131:0/1095137337 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6c00623b0 con 0x55a6144b2d90 2024-05-26T22:54:12.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.152+0000 7fb6b2000640 1 -- 172.21.15.131:0/1095137337 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7fb6c0021040 con 0x55a6144b2d90 2024-05-26T22:54:12.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.152+0000 7fb6d67f5740 10 client.7605 fetch_fsmap learned FSMap version 83 2024-05-26T22:54:12.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.152+0000 7fb6d67f5740 1 -- 172.21.15.131:0/1095137337 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=2+}) -- 0x55a61454c220 con 0x55a6144b2d90 2024-05-26T22:54:12.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.152+0000 7fb6b2000640 1 -- 172.21.15.131:0/1095137337 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6c00623b0 con 0x55a6144b2d90 2024-05-26T22:54:12.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.153+0000 7fb6b2000640 1 -- 172.21.15.131:0/1095137337 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 83) ==== 3534+0+0 (secure 0 0 0) 0x7fb6c008b0c0 con 0x55a6144b2d90 2024-05-26T22:54:12.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.153+0000 7fb6d67f5740 10 client.7605 fetch_fsmap finished waiting for FSMap version 83 2024-05-26T22:54:12.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.153+0000 7fb6d67f5740 10 client.7605 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-26T22:54:12.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.153+0000 7fb6d67f5740 20 client.7605 populate_metadata read hostname 'smithi131' 2024-05-26T22:54:12.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.153+0000 7fb6d67f5740 1 --2- 172.21.15.131:0/1095137337 >> [v2:172.21.15.187:6832/3480101957,v1:172.21.15.187:6834/3480101957] conn(0x55a61454ba10 0x55a6144ae420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:12.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.153+0000 7fb6d67f5740 4 client.7605 mds_command: new command op to 5545 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-26T22:54:12.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.153+0000 7fb6d67f5740 1 -- 172.21.15.131:0/1095137337 --> [v2:172.21.15.187:6832/3480101957,v1:172.21.15.187:6834/3480101957] -- command(tid 0: {"prefix": "status"}) -- 0x55a6143f2090 con 0x55a61454ba10 2024-05-26T22:54:12.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.153+0000 7fb6cbe00640 1 --2- 172.21.15.131:0/1095137337 >> [v2:172.21.15.187:6832/3480101957,v1:172.21.15.187:6834/3480101957] conn(0x55a61454ba10 0x55a6144ae420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:12.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.154+0000 7fb6cbe00640 1 --2- 172.21.15.131:0/1095137337 >> [v2:172.21.15.187:6832/3480101957,v1:172.21.15.187:6834/3480101957] conn(0x55a61454ba10 0x55a6144ae420 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5ce498abd4358b9a server_cookie=c7d00cedb8c4a299 in_seq=0 out_seq=0 2024-05-26T22:54:12.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.154+0000 7fb6b2000640 10 client.7605 ms_handle_connect on v2:172.21.15.187:6832/3480101957 2024-05-26T22:54:12.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.155+0000 7fb6b2000640 1 -- 172.21.15.131:0/1095137337 <== mds.2 v2:172.21.15.187:6832/3480101957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55a6143f2090 con 0x55a61454ba10 2024-05-26T22:54:12.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.155+0000 7fb6b2000640 10 client.7605 handle_command_reply: tid=0 2024-05-26T22:54:12.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.155+0000 7fb6d67f5740 1 client.7605 shutdown 2024-05-26T22:54:12.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.155+0000 7fb6d67f5740 1 -- 172.21.15.131:0/1095137337 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a6144b2d90 msgr2=0x55a6144ef9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:12.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.155+0000 7fb6d67f5740 1 --2- 172.21.15.131:0/1095137337 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a6144b2d90 0x55a6144ef9f0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fb6c000e540 tx=0x7fb6c005d040 comp rx=0 tx=0).stop 2024-05-26T22:54:12.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.155+0000 7fb6d67f5740 1 -- 172.21.15.131:0/1095137337 shutdown_connections 2024-05-26T22:54:12.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.155+0000 7fb6d67f5740 1 --2- 172.21.15.131:0/1095137337 >> [v2:172.21.15.187:6832/3480101957,v1:172.21.15.187:6834/3480101957] conn(0x55a61454ba10 0x55a6144ae420 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:12.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.155+0000 7fb6d67f5740 1 --2- 172.21.15.131:0/1095137337 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55a6144cfc70 0x55a6144dc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:12.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.155+0000 7fb6d67f5740 1 --2- 172.21.15.131:0/1095137337 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a6144e0da0 0x55a6144d0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:12.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.155+0000 7fb6d67f5740 1 --2- 172.21.15.131:0/1095137337 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a6144b2d90 0x55a6144ef9f0 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:12.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.155+0000 7fb6d67f5740 1 -- 172.21.15.131:0/1095137337 >> 172.21.15.131:0/1095137337 conn(0x55a6144b2650 msgr2=0x55a6144cf2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:12.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.155+0000 7fb6b2000640 0 client.7605 ms_handle_reset on v2:172.21.15.187:6832/3480101957 2024-05-26T22:54:12.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.155+0000 7fb6d67f5740 1 -- 172.21.15.131:0/1095137337 shutdown_connections 2024-05-26T22:54:12.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.155+0000 7fb6d67f5740 1 -- 172.21.15.131:0/1095137337 wait complete. 2024-05-26T22:54:12.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.155+0000 7fb6d67f5740 20 client.7605 trim_cache size 0 max 16384 2024-05-26T22:54:12.158 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-26T22:54:12.158 INFO:teuthology.orchestra.run.smithi131.stdout: "cluster_fsid": "46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f", 2024-05-26T22:54:12.158 INFO:teuthology.orchestra.run.smithi131.stdout: "whoami": 2, 2024-05-26T22:54:12.158 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 5545, 2024-05-26T22:54:12.159 INFO:teuthology.orchestra.run.smithi131.stdout: "want_state": "up:active", 2024-05-26T22:54:12.159 INFO:teuthology.orchestra.run.smithi131.stdout: "state": "up:active", 2024-05-26T22:54:12.159 INFO:teuthology.orchestra.run.smithi131.stdout: "fs_name": "cephfs", 2024-05-26T22:54:12.159 INFO:teuthology.orchestra.run.smithi131.stdout: "rank_uptime": 1.9956908870000001, 2024-05-26T22:54:12.159 INFO:teuthology.orchestra.run.smithi131.stdout: "mdsmap_epoch": 83, 2024-05-26T22:54:12.159 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch": 84, 2024-05-26T22:54:12.159 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch_barrier": 84, 2024-05-26T22:54:12.159 INFO:teuthology.orchestra.run.smithi131.stdout: "uptime": 24.146041482000001 2024-05-26T22:54:12.159 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-26T22:54:12.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.156+0000 7fb6d3a00640 1 -- 172.21.15.131:0/1689707981 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb6ac06be30 msgr2=0x7fb6ac06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:12.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.156+0000 7fb6d3a00640 1 --2- 172.21.15.131:0/1689707981 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb6ac06be30 0x7fb6ac06e2f0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fb6b8032000 tx=0x7fb6b802e000 comp rx=0 tx=0).stop 2024-05-26T22:54:12.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.156+0000 7fb6d3a00640 1 -- 172.21.15.131:0/1689707981 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6b4004800 msgr2=0x7fb6cc071e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:12.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.156+0000 7fb6d3a00640 1 --2- 172.21.15.131:0/1689707981 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6b4004800 0x7fb6cc071e10 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7fb6c0010830 tx=0x7fb6c0010cf0 comp rx=0 tx=0).stop 2024-05-26T22:54:12.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.156+0000 7fb6d3a00640 1 -- 172.21.15.131:0/1689707981 shutdown_connections 2024-05-26T22:54:12.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.156+0000 7fb6d3a00640 1 --2- 172.21.15.131:0/1689707981 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb6cc15b1c0 0x7fb6cc15b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:12.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.156+0000 7fb6d3a00640 1 --2- 172.21.15.131:0/1689707981 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb6ac06be30 0x7fb6ac06e2f0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:12.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.156+0000 7fb6d3a00640 1 --2- 172.21.15.131:0/1689707981 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb6cc1e7ed0 0x7fb6cc1e8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:12.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.156+0000 7fb6d3a00640 1 --2- 172.21.15.131:0/1689707981 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6b4004800 0x7fb6cc071e10 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:12.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.156+0000 7fb6d3a00640 1 -- 172.21.15.131:0/1689707981 >> 172.21.15.131:0/1689707981 conn(0x7fb6cc0b27c0 msgr2=0x7fb6cc0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:12.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.156+0000 7fb6d3a00640 1 -- 172.21.15.131:0/1689707981 shutdown_connections 2024-05-26T22:54:12.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.156+0000 7fb6d3a00640 1 -- 172.21.15.131:0/1689707981 wait complete. 2024-05-26T22:54:12.172 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-05-26T22:54:12.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.356+0000 7f8f8d200640 1 Processor -- start 2024-05-26T22:54:12.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.356+0000 7f8f8d200640 1 -- start start 2024-05-26T22:54:12.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.357+0000 7f8f8d200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8f880d0550 0x7f8f880ae250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:12.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.357+0000 7f8f8d200640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f8f880b2f60 0x7f8f880715d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:12.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.357+0000 7f8f8d200640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f880b2440 con 0x7f8f8815e320 2024-05-26T22:54:12.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.357+0000 7f8f8d200640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f8f88161fe0 con 0x7f8f880b2f60 2024-05-26T22:54:12.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.357+0000 7f8f8d200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f8f88162230 con 0x7f8f880d0550 2024-05-26T22:54:12.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.358+0000 7f8f87e00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f8f880b2f60 0x7f8f880715d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:12.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.358+0000 7f8f86a00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8f880d0550 0x7f8f880ae250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:12.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.358+0000 7f8f87e00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f8f880b2f60 0x7f8f880715d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.131:51508/0 (socket says 172.21.15.131:51508) 2024-05-26T22:54:12.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.358+0000 7f8f87e00640 1 -- 172.21.15.131:0/2625938119 learned_addr learned my addr 172.21.15.131:0/2625938119 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:54:12.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.359+0000 7f8f87e00640 1 -- 172.21.15.131:0/2625938119 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8f880d0550 msgr2=0x7f8f880ae250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:12.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.359+0000 7f8f87e00640 1 --2- 172.21.15.131:0/2625938119 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8f880d0550 0x7f8f880ae250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:12.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.359+0000 7f8f87e00640 1 -- 172.21.15.131:0/2625938119 >> v1:172.21.15.187:6789/0 conn(0x7f8f8815e320 legacy=0x7f8f880cfe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:12.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.359+0000 7f8f87e00640 1 -- 172.21.15.131:0/2625938119 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f88162510 con 0x7f8f880b2f60 2024-05-26T22:54:12.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.360+0000 7f8f87e00640 1 --2- 172.21.15.131:0/2625938119 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8f880b2f60 0x7f8f880715d0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f8f80009e50 tx=0x7f8f8002a7a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72bbe456d02a4415 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:12.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.361+0000 7f8f86000640 1 -- 172.21.15.131:0/2625938119 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f80036070 con 0x7f8f880b2f60 2024-05-26T22:54:12.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.361+0000 7f8f86000640 1 -- 172.21.15.131:0/2625938119 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f80031600 con 0x7f8f880b2f60 2024-05-26T22:54:12.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.361+0000 7f8f86000640 1 -- 172.21.15.131:0/2625938119 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f800319d0 con 0x7f8f880b2f60 2024-05-26T22:54:12.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.362+0000 7f8f8d200640 1 -- 172.21.15.131:0/2625938119 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8f880b2f60 msgr2=0x7f8f880715d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:12.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.362+0000 7f8f8d200640 1 --2- 172.21.15.131:0/2625938119 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8f880b2f60 0x7f8f880715d0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f8f80009e50 tx=0x7f8f8002a7a0 comp rx=0 tx=0).stop 2024-05-26T22:54:12.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.362+0000 7f8f8d200640 1 -- 172.21.15.131:0/2625938119 shutdown_connections 2024-05-26T22:54:12.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.362+0000 7f8f8d200640 1 --2- 172.21.15.131:0/2625938119 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8f880b2f60 0x7f8f880715d0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:12.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.362+0000 7f8f8d200640 1 --2- 172.21.15.131:0/2625938119 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8f880d0550 0x7f8f880ae250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:12.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.362+0000 7f8f8d200640 1 -- 172.21.15.131:0/2625938119 >> 172.21.15.131:0/2625938119 conn(0x7f8f880b5660 msgr2=0x7f8f880b5a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:12.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.362+0000 7f8f8d200640 1 -- 172.21.15.131:0/2625938119 shutdown_connections 2024-05-26T22:54:12.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.362+0000 7f8f8d200640 1 -- 172.21.15.131:0/2625938119 wait complete. 2024-05-26T22:54:12.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.364+0000 7f8f8d200640 1 Processor -- start 2024-05-26T22:54:12.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.364+0000 7f8f8d200640 1 -- start start 2024-05-26T22:54:12.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.365+0000 7f8f8d200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8f880b2f60 0x7f8f881f4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:12.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.365+0000 7f8f8d200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8f880d0550 0x7f8f8815ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:12.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.365+0000 7f8f8d200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f8815e320 0x7f8f8815cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:12.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.365+0000 7f8f8d200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8f88162e40 con 0x7f8f8815e320 2024-05-26T22:54:12.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.365+0000 7f8f8d200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8f88162ad0 con 0x7f8f880b2f60 2024-05-26T22:54:12.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.365+0000 7f8f8d200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f8f880b3570 con 0x7f8f880d0550 2024-05-26T22:54:12.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.365+0000 7f8f87e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f8815e320 0x7f8f8815cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:12.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.365+0000 7f8f87e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f8815e320 0x7f8f8815cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46408/0 (socket says 172.21.15.131:46408) 2024-05-26T22:54:12.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.365+0000 7f8f87e00640 1 -- 172.21.15.131:0/1965293890 learned_addr learned my addr 172.21.15.131:0/1965293890 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:54:12.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.366+0000 7f8f87400640 1 --2- 172.21.15.131:0/1965293890 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8f880b2f60 0x7f8f881f4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:12.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.366+0000 7f8f87e00640 1 -- 172.21.15.131:0/1965293890 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8f880d0550 msgr2=0x7f8f8815ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:12.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.366+0000 7f8f86a00640 1 --2- 172.21.15.131:0/1965293890 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8f880d0550 0x7f8f8815ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:12.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.366+0000 7f8f87e00640 1 --2- 172.21.15.131:0/1965293890 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8f880d0550 0x7f8f8815ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:12.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.366+0000 7f8f87e00640 1 -- 172.21.15.131:0/1965293890 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8f880b2f60 msgr2=0x7f8f881f4ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:12.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.366+0000 7f8f87e00640 1 --2- 172.21.15.131:0/1965293890 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8f880b2f60 0x7f8f881f4ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:12.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.366+0000 7f8f87e00640 1 -- 172.21.15.131:0/1965293890 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f8815af80 con 0x7f8f8815e320 2024-05-26T22:54:12.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.366+0000 7f8f87e00640 1 --2- 172.21.15.131:0/1965293890 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f8815e320 0x7f8f8815cf70 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f8f80034040 tx=0x7f8f80004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:12.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.366+0000 7f8f7f400640 1 -- 172.21.15.131:0/1965293890 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f80036050 con 0x7f8f8815e320 2024-05-26T22:54:12.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.366+0000 7f8f7f400640 1 -- 172.21.15.131:0/1965293890 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f80042070 con 0x7f8f8815e320 2024-05-26T22:54:12.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.366+0000 7f8f7f400640 1 -- 172.21.15.131:0/1965293890 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f80009510 con 0x7f8f8815e320 2024-05-26T22:54:12.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.366+0000 7f8f8d200640 1 -- 172.21.15.131:0/1965293890 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f8815b260 con 0x7f8f8815e320 2024-05-26T22:54:12.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.367+0000 7f8f8d200640 1 -- 172.21.15.131:0/1965293890 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f8815b820 con 0x7f8f8815e320 2024-05-26T22:54:12.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.369+0000 7f8f7f400640 1 -- 172.21.15.131:0/1965293890 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8f80007b70 con 0x7f8f8815e320 2024-05-26T22:54:12.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.370+0000 7f8f7f400640 1 --2- 172.21.15.131:0/1965293890 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f8f6806be30 0x7f8f6806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:12.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.370+0000 7f8f7f400640 1 -- 172.21.15.131:0/1965293890 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5738+0+0 (secure 0 0 0) 0x7f8f800abdc0 con 0x7f8f8815e320 2024-05-26T22:54:12.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.371+0000 7f8f87400640 1 --2- 172.21.15.131:0/1965293890 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f8f6806be30 0x7f8f6806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:12.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.371+0000 7f8f91607740 1 Processor -- start 2024-05-26T22:54:12.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.371+0000 7f8f91607740 1 -- start start 2024-05-26T22:54:12.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.371+0000 7f8f87400640 1 --2- 172.21.15.131:0/1965293890 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f8f6806be30 0x7f8f6806e2f0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f8f78007fd0 tx=0x7f8f78009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:12.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.372+0000 7f8f91607740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f714c1e50 0x555f71474a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:12.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.372+0000 7f8f91607740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x555f714a4f70 0x555f714a42f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:12.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.372+0000 7f8f91607740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555f714a4830 0x555f71479640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:12.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.372+0000 7f8f91607740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x555f713dc610 con 0x555f714c1e50 2024-05-26T22:54:12.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.372+0000 7f8f91607740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x555f713845c0 con 0x555f714a4830 2024-05-26T22:54:12.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.372+0000 7f8f91607740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x555f713d85d0 con 0x555f714a4f70 2024-05-26T22:54:12.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.372+0000 7f8f87400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f714c1e50 0x555f71474a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:12.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.372+0000 7f8f87400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f714c1e50 0x555f71474a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46422/0 (socket says 172.21.15.131:46422) 2024-05-26T22:54:12.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.372+0000 7f8f87400640 1 -- 172.21.15.131:0/2285857104 learned_addr learned my addr 172.21.15.131:0/2285857104 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:54:12.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.372+0000 7f8f87400640 1 -- 172.21.15.131:0/2285857104 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x555f714a4f70 msgr2=0x555f714a42f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:12.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.372+0000 7f8f87400640 1 --2- 172.21.15.131:0/2285857104 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x555f714a4f70 0x555f714a42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:12.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.372+0000 7f8f87400640 1 -- 172.21.15.131:0/2285857104 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555f714a4830 msgr2=0x555f71479640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:12.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.372+0000 7f8f87400640 1 --2- 172.21.15.131:0/2285857104 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555f714a4830 0x555f71479640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:12.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.372+0000 7f8f87400640 1 -- 172.21.15.131:0/2285857104 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f710f02d0 con 0x555f714c1e50 2024-05-26T22:54:12.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.373+0000 7f8f87400640 1 --2- 172.21.15.131:0/2285857104 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f714c1e50 0x555f71474a70 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f8f78005310 tx=0x7f8f78011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18563141a41f5db2 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:12.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.373+0000 7f8f73e00640 1 -- 172.21.15.131:0/2285857104 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f78021450 con 0x555f714c1e50 2024-05-26T22:54:12.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.373+0000 7f8f73e00640 1 -- 172.21.15.131:0/2285857104 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f780215f0 con 0x555f714c1e50 2024-05-26T22:54:12.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.373+0000 7f8f73e00640 1 -- 172.21.15.131:0/2285857104 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f780219a0 con 0x555f714c1e50 2024-05-26T22:54:12.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.374+0000 7f8f91607740 1 -- 172.21.15.131:0/2285857104 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f714c1e50 msgr2=0x555f71474a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:12.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.374+0000 7f8f91607740 1 --2- 172.21.15.131:0/2285857104 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f714c1e50 0x555f71474a70 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f8f78005310 tx=0x7f8f78011000 comp rx=0 tx=0).stop 2024-05-26T22:54:12.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.374+0000 7f8f91607740 1 -- 172.21.15.131:0/2285857104 shutdown_connections 2024-05-26T22:54:12.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.374+0000 7f8f91607740 1 --2- 172.21.15.131:0/2285857104 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x555f714a4f70 0x555f714a42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:12.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.374+0000 7f8f91607740 1 --2- 172.21.15.131:0/2285857104 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555f714a4830 0x555f71479640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:12.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.374+0000 7f8f91607740 1 --2- 172.21.15.131:0/2285857104 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f714c1e50 0x555f71474a70 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:12.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.374+0000 7f8f91607740 1 -- 172.21.15.131:0/2285857104 >> 172.21.15.131:0/2285857104 conn(0x555f71476ce0 msgr2=0x555f714bf2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:12.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.374+0000 7f8f91607740 1 -- 172.21.15.131:0/2285857104 shutdown_connections 2024-05-26T22:54:12.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.374+0000 7f8f91607740 1 -- 172.21.15.131:0/2285857104 wait complete. 2024-05-26T22:54:12.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.374+0000 7f8f91607740 1 -- start start 2024-05-26T22:54:12.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.375+0000 7f8f91607740 1 Processor -- start 2024-05-26T22:54:12.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.375+0000 7f8f91607740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x555f714a4f70 0x555f714cf890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:12.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.375+0000 7f8f91607740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555f714c1e50 0x555f714cfdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:12.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.375+0000 7f8f91607740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f714b8530 0x555f714a7790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:12.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.376+0000 7f8f91607740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x555f71478c00 con 0x555f714b8530 2024-05-26T22:54:12.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.376+0000 7f8f91607740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x555f71397910 con 0x555f714c1e50 2024-05-26T22:54:12.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.376+0000 7f8f91607740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x555f71477ff0 con 0x555f714a4f70 2024-05-26T22:54:12.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.376+0000 7f8f91607740 10 client.0 _ll_register_callbacks cb 0x555f713d3410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-26T22:54:12.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.376+0000 7f8f87e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f714b8530 0x555f714a7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:12.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.376+0000 7f8f87e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f714b8530 0x555f714a7790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46428/0 (socket says 172.21.15.131:46428) 2024-05-26T22:54:12.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.376+0000 7f8f87e00640 1 -- 172.21.15.131:0/2155369982 learned_addr learned my addr 172.21.15.131:0/2155369982 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:54:12.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.376+0000 7f8f87e00640 1 -- 172.21.15.131:0/2155369982 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x555f714a4f70 msgr2=0x555f714cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:12.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.376+0000 7f8f87e00640 1 --2- 172.21.15.131:0/2155369982 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x555f714a4f70 0x555f714cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:12.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.376+0000 7f8f87e00640 1 -- 172.21.15.131:0/2155369982 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555f714c1e50 msgr2=0x555f714cfdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:12.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.376+0000 7f8f87e00640 1 --2- 172.21.15.131:0/2155369982 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555f714c1e50 0x555f714cfdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:12.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.376+0000 7f8f87e00640 1 -- 172.21.15.131:0/2155369982 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f711ba6b0 con 0x555f714b8530 2024-05-26T22:54:12.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.376+0000 7f8f87e00640 1 --2- 172.21.15.131:0/2155369982 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f714b8530 0x555f714a7790 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f8f800827b0 tx=0x7f8f80079c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65fdea666a59e5fe server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:12.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.376+0000 7f8f70c00640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-26T22:54:12.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.377+0000 7f8f70c00640 1 -- 172.21.15.131:0/2155369982 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f800a7020 con 0x555f714b8530 2024-05-26T22:54:12.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.377+0000 7f8f70c00640 1 -- 172.21.15.131:0/2155369982 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 5738+0+0 (secure 0 0 0) 0x7f8f80079550 con 0x555f714b8530 2024-05-26T22:54:12.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.377+0000 7f8f91607740 1 -- 172.21.15.131:0/2155369982 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f780215f0 con 0x555f714b8530 2024-05-26T22:54:12.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.377+0000 7f8f70c00640 1 -- 172.21.15.131:0/2155369982 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f8f8007a800 con 0x555f714b8530 2024-05-26T22:54:12.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.377+0000 7f8f91607740 10 client.7614 fetch_fsmap learned FSMap version 83 2024-05-26T22:54:12.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.377+0000 7f8f91607740 1 -- 172.21.15.131:0/2155369982 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=2+}) -- 0x555f714b7a50 con 0x555f714b8530 2024-05-26T22:54:12.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.377+0000 7f8f70c00640 1 -- 172.21.15.131:0/2155369982 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f780215f0 con 0x555f714b8530 2024-05-26T22:54:12.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.377+0000 7f8f70c00640 1 -- 172.21.15.131:0/2155369982 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 83) ==== 3534+0+0 (secure 0 0 0) 0x7f8f80094060 con 0x555f714b8530 2024-05-26T22:54:12.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.377+0000 7f8f91607740 10 client.7614 fetch_fsmap finished waiting for FSMap version 83 2024-05-26T22:54:12.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.377+0000 7f8f91607740 10 client.7614 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-26T22:54:12.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.377+0000 7f8f91607740 20 client.7614 populate_metadata read hostname 'smithi131' 2024-05-26T22:54:12.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.377+0000 7f8f91607740 1 --2- 172.21.15.131:0/2155369982 >> [v2:172.21.15.187:6833/567389303,v1:172.21.15.187:6835/567389303] conn(0x555f714a57c0 0x555f714b6610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:12.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.377+0000 7f8f91607740 4 client.7614 mds_command: new command op to 5548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-26T22:54:12.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.377+0000 7f8f91607740 1 -- 172.21.15.131:0/2155369982 --> [v2:172.21.15.187:6833/567389303,v1:172.21.15.187:6835/567389303] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555f713e9d40 con 0x555f714a57c0 2024-05-26T22:54:12.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.378+0000 7f8f87400640 1 --2- 172.21.15.131:0/2155369982 >> [v2:172.21.15.187:6833/567389303,v1:172.21.15.187:6835/567389303] conn(0x555f714a57c0 0x555f714b6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:12.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.379+0000 7f8f87400640 1 --2- 172.21.15.131:0/2155369982 >> [v2:172.21.15.187:6833/567389303,v1:172.21.15.187:6835/567389303] conn(0x555f714a57c0 0x555f714b6610 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3158067e4ec593b server_cookie=fa18e21386e526cf in_seq=0 out_seq=0 2024-05-26T22:54:12.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.379+0000 7f8f70c00640 10 client.7614 ms_handle_connect on v2:172.21.15.187:6833/567389303 2024-05-26T22:54:12.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.381+0000 7f8f70c00640 1 -- 172.21.15.131:0/2155369982 <== mds.1 v2:172.21.15.187:6833/567389303 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555f713e9d40 con 0x555f714a57c0 2024-05-26T22:54:12.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.381+0000 7f8f70c00640 10 client.7614 handle_command_reply: tid=0 2024-05-26T22:54:12.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.381+0000 7f8f91607740 1 client.7614 shutdown 2024-05-26T22:54:12.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.382+0000 7f8f91607740 1 -- 172.21.15.131:0/2155369982 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f714b8530 msgr2=0x555f714a7790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:12.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.382+0000 7f8f91607740 1 --2- 172.21.15.131:0/2155369982 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f714b8530 0x555f714a7790 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f8f800827b0 tx=0x7f8f80079c40 comp rx=0 tx=0).stop 2024-05-26T22:54:12.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.382+0000 7f8f91607740 1 -- 172.21.15.131:0/2155369982 shutdown_connections 2024-05-26T22:54:12.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.382+0000 7f8f91607740 1 --2- 172.21.15.131:0/2155369982 >> [v2:172.21.15.187:6833/567389303,v1:172.21.15.187:6835/567389303] conn(0x555f714a57c0 0x555f714b6610 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:12.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.382+0000 7f8f91607740 1 --2- 172.21.15.131:0/2155369982 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f714b8530 0x555f714a7790 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:12.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.382+0000 7f8f91607740 1 --2- 172.21.15.131:0/2155369982 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555f714c1e50 0x555f714cfdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:12.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.382+0000 7f8f91607740 1 --2- 172.21.15.131:0/2155369982 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x555f714a4f70 0x555f714cf890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:12.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.382+0000 7f8f91607740 1 -- 172.21.15.131:0/2155369982 >> 172.21.15.131:0/2155369982 conn(0x555f714a4830 msgr2=0x555f714754d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:12.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.382+0000 7f8f70c00640 0 client.7614 ms_handle_reset on v2:172.21.15.187:6833/567389303 2024-05-26T22:54:12.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.382+0000 7f8f91607740 1 -- 172.21.15.131:0/2155369982 shutdown_connections 2024-05-26T22:54:12.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.382+0000 7f8f91607740 1 -- 172.21.15.131:0/2155369982 wait complete. 2024-05-26T22:54:12.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.382+0000 7f8f91607740 20 client.7614 trim_cache size 0 max 16384 2024-05-26T22:54:12.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.401+0000 7f8f91607740 1 Processor -- start 2024-05-26T22:54:12.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.401+0000 7f8f91607740 1 -- start start 2024-05-26T22:54:12.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.402+0000 7f8f91607740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x555f714a4f70 0x555f714a1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:12.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.402+0000 7f8f91607740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555f714b8530 0x555f714cf800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:12.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.402+0000 7f8f91607740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f714c1e50 0x555f714cfd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:12.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.402+0000 7f8f91607740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8f881ffc40 con 0x555f714c1e50 2024-05-26T22:54:12.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.402+0000 7f8f91607740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8f882002e0 con 0x555f714b8530 2024-05-26T22:54:12.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.402+0000 7f8f91607740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f8f881f8290 con 0x555f714a4f70 2024-05-26T22:54:12.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.402+0000 7f8f87e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f714c1e50 0x555f714cfd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:12.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.403+0000 7f8f87e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f714c1e50 0x555f714cfd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46440/0 (socket says 172.21.15.131:46440) 2024-05-26T22:54:12.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.403+0000 7f8f87e00640 1 -- 172.21.15.131:0/2930312595 learned_addr learned my addr 172.21.15.131:0/2930312595 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:54:12.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.403+0000 7f8f87e00640 1 -- 172.21.15.131:0/2930312595 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x555f714a4f70 msgr2=0x555f714a1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:12.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.403+0000 7f8f87e00640 1 --2- 172.21.15.131:0/2930312595 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x555f714a4f70 0x555f714a1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:12.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.403+0000 7f8f87e00640 1 -- 172.21.15.131:0/2930312595 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555f714b8530 msgr2=0x555f714cf800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:12.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.403+0000 7f8f87e00640 1 --2- 172.21.15.131:0/2930312595 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555f714b8530 0x555f714cf800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:12.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.403+0000 7f8f87e00640 1 -- 172.21.15.131:0/2930312595 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f714c0870 con 0x555f714c1e50 2024-05-26T22:54:12.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.403+0000 7f8f87e00640 1 --2- 172.21.15.131:0/2930312595 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f714c1e50 0x555f714cfd40 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f8f800783b0 tx=0x7f8f800a4000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38c78d88871c807f server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:12.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.403+0000 7f8f73400640 1 -- 172.21.15.131:0/2930312595 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f800a7030 con 0x555f714c1e50 2024-05-26T22:54:12.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.403+0000 7f8f73400640 1 -- 172.21.15.131:0/2930312595 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f80037040 con 0x555f714c1e50 2024-05-26T22:54:12.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.403+0000 7f8f73400640 1 -- 172.21.15.131:0/2930312595 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f8007e4b0 con 0x555f714c1e50 2024-05-26T22:54:12.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.404+0000 7f8f91607740 1 -- 172.21.15.131:0/2930312595 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f714c1e50 msgr2=0x555f714cfd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:12.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.404+0000 7f8f91607740 1 --2- 172.21.15.131:0/2930312595 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f714c1e50 0x555f714cfd40 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f8f800783b0 tx=0x7f8f800a4000 comp rx=0 tx=0).stop 2024-05-26T22:54:12.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.404+0000 7f8f91607740 1 -- 172.21.15.131:0/2930312595 shutdown_connections 2024-05-26T22:54:12.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.404+0000 7f8f91607740 1 --2- 172.21.15.131:0/2930312595 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f714c1e50 0x555f714cfd40 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:12.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.404+0000 7f8f91607740 1 --2- 172.21.15.131:0/2930312595 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555f714b8530 0x555f714cf800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:12.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.404+0000 7f8f91607740 1 --2- 172.21.15.131:0/2930312595 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x555f714a4f70 0x555f714a1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:12.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.404+0000 7f8f91607740 1 -- 172.21.15.131:0/2930312595 >> 172.21.15.131:0/2930312595 conn(0x555f714a4830 msgr2=0x555f71470240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:12.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.404+0000 7f8f91607740 1 -- 172.21.15.131:0/2930312595 shutdown_connections 2024-05-26T22:54:12.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.404+0000 7f8f91607740 1 -- 172.21.15.131:0/2930312595 wait complete. 2024-05-26T22:54:12.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.405+0000 7f8f91607740 1 -- start start 2024-05-26T22:54:12.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.405+0000 7f8f91607740 1 Processor -- start 2024-05-26T22:54:12.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.406+0000 7f8f91607740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x555f714a4f70 0x555f715459e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:12.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.406+0000 7f8f91607740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555f714b8530 0x555f714742e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:12.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.406+0000 7f8f91607740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f714c1e50 0x555f71474820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:12.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.406+0000 7f8f91607740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x555f714a5640 con 0x555f714c1e50 2024-05-26T22:54:12.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.406+0000 7f8f91607740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x555f714b7d30 con 0x555f714b8530 2024-05-26T22:54:12.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.406+0000 7f8f91607740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x555f714a5c60 con 0x555f714a4f70 2024-05-26T22:54:12.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.406+0000 7f8f91607740 10 client.0 _ll_register_callbacks cb 0x7f8f80007310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-26T22:54:12.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.406+0000 7f8f87e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f714c1e50 0x555f71474820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:12.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.406+0000 7f8f87e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f714c1e50 0x555f71474820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46444/0 (socket says 172.21.15.131:46444) 2024-05-26T22:54:12.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.406+0000 7f8f87e00640 1 -- 172.21.15.131:0/845964606 learned_addr learned my addr 172.21.15.131:0/845964606 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:54:12.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.406+0000 7f8f87e00640 1 -- 172.21.15.131:0/845964606 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x555f714a4f70 msgr2=0x555f715459e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:12.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.406+0000 7f8f87e00640 1 --2- 172.21.15.131:0/845964606 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x555f714a4f70 0x555f715459e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:12.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.406+0000 7f8f87e00640 1 -- 172.21.15.131:0/845964606 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555f714b8530 msgr2=0x555f714742e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:12.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.406+0000 7f8f87e00640 1 --2- 172.21.15.131:0/845964606 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555f714b8530 0x555f714742e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:12.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.406+0000 7f8f87e00640 1 -- 172.21.15.131:0/845964606 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f71545f20 con 0x555f714c1e50 2024-05-26T22:54:12.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.407+0000 7f8f87e00640 1 --2- 172.21.15.131:0/845964606 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f714c1e50 0x555f71474820 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f8f800073c0 tx=0x7f8f80049070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d8370752a8bb6ed server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:12.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.407+0000 7f8f70c00640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-26T22:54:12.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.407+0000 7f8f70c00640 1 -- 172.21.15.131:0/845964606 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f800763a0 con 0x555f714c1e50 2024-05-26T22:54:12.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.407+0000 7f8f70c00640 1 -- 172.21.15.131:0/845964606 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 5738+0+0 (secure 0 0 0) 0x7f8f80078fc0 con 0x555f714c1e50 2024-05-26T22:54:12.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.407+0000 7f8f91607740 1 -- 172.21.15.131:0/845964606 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f80037040 con 0x555f714c1e50 2024-05-26T22:54:12.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.407+0000 7f8f70c00640 1 -- 172.21.15.131:0/845964606 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f8f80092050 con 0x555f714c1e50 2024-05-26T22:54:12.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.407+0000 7f8f91607740 10 client.7620 fetch_fsmap learned FSMap version 83 2024-05-26T22:54:12.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.407+0000 7f8f91607740 1 -- 172.21.15.131:0/845964606 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=2+}) -- 0x555f71547460 con 0x555f714c1e50 2024-05-26T22:54:12.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.407+0000 7f8f70c00640 1 -- 172.21.15.131:0/845964606 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f80037040 con 0x555f714c1e50 2024-05-26T22:54:12.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.407+0000 7f8f70c00640 1 -- 172.21.15.131:0/845964606 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 83) ==== 3534+0+0 (secure 0 0 0) 0x7f8f800791c0 con 0x555f714c1e50 2024-05-26T22:54:12.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.407+0000 7f8f91607740 10 client.7620 fetch_fsmap finished waiting for FSMap version 83 2024-05-26T22:54:12.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.407+0000 7f8f91607740 10 client.7620 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-26T22:54:12.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.407+0000 7f8f91607740 20 client.7620 populate_metadata read hostname 'smithi131' 2024-05-26T22:54:12.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.407+0000 7f8f91607740 1 --2- 172.21.15.131:0/845964606 >> [v2:172.21.15.187:6833/567389303,v1:172.21.15.187:6835/567389303] conn(0x555f714bf7a0 0x555f715463e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:12.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.407+0000 7f8f91607740 4 client.7620 mds_command: new command op to 5548 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-26T22:54:12.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.407+0000 7f8f91607740 1 -- 172.21.15.131:0/845964606 --> [v2:172.21.15.187:6833/567389303,v1:172.21.15.187:6835/567389303] -- command(tid 0: {"prefix": "status"}) -- 0x555f713e4090 con 0x555f714bf7a0 2024-05-26T22:54:12.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.408+0000 7f8f87400640 1 --2- 172.21.15.131:0/845964606 >> [v2:172.21.15.187:6833/567389303,v1:172.21.15.187:6835/567389303] conn(0x555f714bf7a0 0x555f715463e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:12.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.409+0000 7f8f87400640 1 --2- 172.21.15.131:0/845964606 >> [v2:172.21.15.187:6833/567389303,v1:172.21.15.187:6835/567389303] conn(0x555f714bf7a0 0x555f715463e0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9c2c4558cc4b0277 server_cookie=33d8fe1086bc9ac0 in_seq=0 out_seq=0 2024-05-26T22:54:12.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.409+0000 7f8f70c00640 10 client.7620 ms_handle_connect on v2:172.21.15.187:6833/567389303 2024-05-26T22:54:12.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.409+0000 7f8f70c00640 1 -- 172.21.15.131:0/845964606 <== mds.1 v2:172.21.15.187:6833/567389303 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x555f713e4090 con 0x555f714bf7a0 2024-05-26T22:54:12.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.409+0000 7f8f70c00640 10 client.7620 handle_command_reply: tid=0 2024-05-26T22:54:12.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.409+0000 7f8f91607740 1 client.7620 shutdown 2024-05-26T22:54:12.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.410+0000 7f8f91607740 1 -- 172.21.15.131:0/845964606 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f714c1e50 msgr2=0x555f71474820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:12.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.410+0000 7f8f91607740 1 --2- 172.21.15.131:0/845964606 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f714c1e50 0x555f71474820 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f8f800073c0 tx=0x7f8f80049070 comp rx=0 tx=0).stop 2024-05-26T22:54:12.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.410+0000 7f8f91607740 1 -- 172.21.15.131:0/845964606 shutdown_connections 2024-05-26T22:54:12.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.410+0000 7f8f91607740 1 --2- 172.21.15.131:0/845964606 >> [v2:172.21.15.187:6833/567389303,v1:172.21.15.187:6835/567389303] conn(0x555f714bf7a0 0x555f715463e0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:12.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.410+0000 7f8f91607740 1 --2- 172.21.15.131:0/845964606 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555f714c1e50 0x555f71474820 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:12.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.410+0000 7f8f91607740 1 --2- 172.21.15.131:0/845964606 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555f714b8530 0x555f714742e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:12.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.410+0000 7f8f70c00640 0 client.7620 ms_handle_reset on v2:172.21.15.187:6833/567389303 2024-05-26T22:54:12.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.410+0000 7f8f91607740 1 --2- 172.21.15.131:0/845964606 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x555f714a4f70 0x555f715459e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:12.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.410+0000 7f8f91607740 1 -- 172.21.15.131:0/845964606 >> 172.21.15.131:0/845964606 conn(0x555f714a4830 msgr2=0x555f714a74b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:12.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.410+0000 7f8f91607740 1 -- 172.21.15.131:0/845964606 shutdown_connections 2024-05-26T22:54:12.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.410+0000 7f8f91607740 1 -- 172.21.15.131:0/845964606 wait complete. 2024-05-26T22:54:12.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.410+0000 7f8f91607740 20 client.7620 trim_cache size 0 max 16384 2024-05-26T22:54:12.413 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-26T22:54:12.413 INFO:teuthology.orchestra.run.smithi131.stdout: "cluster_fsid": "46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f", 2024-05-26T22:54:12.413 INFO:teuthology.orchestra.run.smithi131.stdout: "whoami": 1, 2024-05-26T22:54:12.413 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 5548, 2024-05-26T22:54:12.413 INFO:teuthology.orchestra.run.smithi131.stdout: "want_state": "up:active", 2024-05-26T22:54:12.413 INFO:teuthology.orchestra.run.smithi131.stdout: "state": "up:active", 2024-05-26T22:54:12.413 INFO:teuthology.orchestra.run.smithi131.stdout: "fs_name": "cephfs", 2024-05-26T22:54:12.413 INFO:teuthology.orchestra.run.smithi131.stdout: "rank_uptime": 3.2632601860000001, 2024-05-26T22:54:12.414 INFO:teuthology.orchestra.run.smithi131.stdout: "mdsmap_epoch": 83, 2024-05-26T22:54:12.414 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch": 84, 2024-05-26T22:54:12.414 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch_barrier": 84, 2024-05-26T22:54:12.414 INFO:teuthology.orchestra.run.smithi131.stdout: "uptime": 24.399200609000001 2024-05-26T22:54:12.414 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-26T22:54:12.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.411+0000 7f8f8d200640 1 -- 172.21.15.131:0/1965293890 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f8f6806be30 msgr2=0x7f8f6806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:12.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.411+0000 7f8f8d200640 1 --2- 172.21.15.131:0/1965293890 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f8f6806be30 0x7f8f6806e2f0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f8f78007fd0 tx=0x7f8f78009040 comp rx=0 tx=0).stop 2024-05-26T22:54:12.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.411+0000 7f8f8d200640 1 -- 172.21.15.131:0/1965293890 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f8815e320 msgr2=0x7f8f8815cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:12.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.411+0000 7f8f8d200640 1 --2- 172.21.15.131:0/1965293890 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f8815e320 0x7f8f8815cf70 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f8f80034040 tx=0x7f8f80004a40 comp rx=0 tx=0).stop 2024-05-26T22:54:12.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.411+0000 7f8f8d200640 1 -- 172.21.15.131:0/1965293890 shutdown_connections 2024-05-26T22:54:12.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.411+0000 7f8f8d200640 1 --2- 172.21.15.131:0/1965293890 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8f880d0550 0x7f8f8815ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:12.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.411+0000 7f8f8d200640 1 --2- 172.21.15.131:0/1965293890 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f8f6806be30 0x7f8f6806e2f0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:12.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.411+0000 7f8f8d200640 1 --2- 172.21.15.131:0/1965293890 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f8815e320 0x7f8f8815cf70 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:12.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.411+0000 7f8f8d200640 1 --2- 172.21.15.131:0/1965293890 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8f880b2f60 0x7f8f881f4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:12.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.411+0000 7f8f8d200640 1 -- 172.21.15.131:0/1965293890 >> 172.21.15.131:0/1965293890 conn(0x7f8f880b5660 msgr2=0x7f8f880b49e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:12.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.411+0000 7f8f8d200640 1 -- 172.21.15.131:0/1965293890 shutdown_connections 2024-05-26T22:54:12.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.411+0000 7f8f8d200640 1 -- 172.21.15.131:0/1965293890 wait complete. 2024-05-26T22:54:12.426 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-05-26T22:54:12.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.623+0000 7f7beb400640 1 Processor -- start 2024-05-26T22:54:12.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.623+0000 7f7beb400640 1 -- start start 2024-05-26T22:54:12.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.624+0000 7f7beb400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f7be40d0560 0x7f7be4072540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:12.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.624+0000 7f7beb400640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7be40733c0 con 0x7f7be40af230 2024-05-26T22:54:12.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.624+0000 7f7beb400640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7be40cf540 con 0x7f7be4165e30 2024-05-26T22:54:12.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.624+0000 7f7beb400640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f7be4072e90 con 0x7f7be40d0560 2024-05-26T22:54:12.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.625+0000 7f7be9400640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f7be40af230 0x7f7be40cdcd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:38086/0 (socket says 172.21.15.131:38086) 2024-05-26T22:54:12.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.625+0000 7f7bdbe00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f7be40d0560 0x7f7be4072540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:12.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.625+0000 7f7be9400640 1 -- 172.21.15.131:0/3991659707 learned_addr learned my addr 172.21.15.131:0/3991659707 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:54:12.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.625+0000 7f7bdbe00640 1 -- 172.21.15.131:0/3991659707 >> v1:172.21.15.187:6789/0 conn(0x7f7be4165e30 legacy=0x7f7be40cfe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:12.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.625+0000 7f7bdbe00640 1 -- 172.21.15.131:0/3991659707 >> v1:172.21.15.131:6789/0 conn(0x7f7be40af230 legacy=0x7f7be40cdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:12.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.625+0000 7f7be3e00640 1 -- 172.21.15.131:0/3991659707 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1113935940 0 0) 0x7f7be40733c0 con 0x7f7be40af230 2024-05-26T22:54:12.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.625+0000 7f7bdbe00640 1 -- 172.21.15.131:0/3991659707 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7be40730e0 con 0x7f7be40d0560 2024-05-26T22:54:12.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.626+0000 7f7bdbe00640 1 --2- 172.21.15.131:0/3991659707 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7be40d0560 0x7f7be4072540 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f7be40cf710 tx=0x7f7bd002a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26c880b4cfd23cc1 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:12.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.626+0000 7f7be3e00640 1 -- 172.21.15.131:0/3991659707 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bd0037070 con 0x7f7be40d0560 2024-05-26T22:54:12.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.626+0000 7f7be3e00640 1 -- 172.21.15.131:0/3991659707 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bd002ae40 con 0x7f7be40d0560 2024-05-26T22:54:12.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.626+0000 7f7be3e00640 1 -- 172.21.15.131:0/3991659707 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bd0032660 con 0x7f7be40d0560 2024-05-26T22:54:12.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.627+0000 7f7beb400640 1 -- 172.21.15.131:0/3991659707 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7be40d0560 msgr2=0x7f7be4072540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:12.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.627+0000 7f7beb400640 1 --2- 172.21.15.131:0/3991659707 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7be40d0560 0x7f7be4072540 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f7be40cf710 tx=0x7f7bd002a130 comp rx=0 tx=0).stop 2024-05-26T22:54:12.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.627+0000 7f7beb400640 1 -- 172.21.15.131:0/3991659707 shutdown_connections 2024-05-26T22:54:12.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.627+0000 7f7beb400640 1 --2- 172.21.15.131:0/3991659707 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7be40d0560 0x7f7be4072540 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:12.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.627+0000 7f7beb400640 1 -- 172.21.15.131:0/3991659707 >> 172.21.15.131:0/3991659707 conn(0x7f7be406bc90 msgr2=0x7f7be40b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:12.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.627+0000 7f7beb400640 1 -- 172.21.15.131:0/3991659707 shutdown_connections 2024-05-26T22:54:12.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.627+0000 7f7beb400640 1 -- 172.21.15.131:0/3991659707 wait complete. 2024-05-26T22:54:12.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.629+0000 7f7beb400640 1 Processor -- start 2024-05-26T22:54:12.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.629+0000 7f7beb400640 1 -- start start 2024-05-26T22:54:12.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.630+0000 7f7beb400640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7be40af230 0x7f7be4162170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:12.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.630+0000 7f7beb400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7be40d0560 0x7f7be409bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:12.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.630+0000 7f7beb400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7be4165e30 0x7f7be409c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:12.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.630+0000 7f7beb400640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f7be40ce3e0 con 0x7f7be4165e30 2024-05-26T22:54:12.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.630+0000 7f7beb400640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7be40cfac0 con 0x7f7be40af230 2024-05-26T22:54:12.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.630+0000 7f7beb400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f7be40ce560 con 0x7f7be40d0560 2024-05-26T22:54:12.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.630+0000 7f7be9400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7be4165e30 0x7f7be409c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:12.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.630+0000 7f7be9400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7be4165e30 0x7f7be409c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46466/0 (socket says 172.21.15.131:46466) 2024-05-26T22:54:12.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.630+0000 7f7be9400640 1 -- 172.21.15.131:0/3675267251 learned_addr learned my addr 172.21.15.131:0/3675267251 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:54:12.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.631+0000 7f7be8a00640 1 --2- 172.21.15.131:0/3675267251 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7be40af230 0x7f7be4162170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:12.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.631+0000 7f7bdbe00640 1 --2- 172.21.15.131:0/3675267251 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7be40d0560 0x7f7be409bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:12.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.631+0000 7f7be9400640 1 -- 172.21.15.131:0/3675267251 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7be40d0560 msgr2=0x7f7be409bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:12.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.631+0000 7f7be9400640 1 --2- 172.21.15.131:0/3675267251 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7be40d0560 0x7f7be409bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:12.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.631+0000 7f7be9400640 1 -- 172.21.15.131:0/3675267251 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7be40af230 msgr2=0x7f7be4162170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:12.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.631+0000 7f7be9400640 1 --2- 172.21.15.131:0/3675267251 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7be40af230 0x7f7be4162170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:12.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.631+0000 7f7be9400640 1 -- 172.21.15.131:0/3675267251 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7be409ac50 con 0x7f7be4165e30 2024-05-26T22:54:12.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.631+0000 7f7be8a00640 1 --2- 172.21.15.131:0/3675267251 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7be40af230 0x7f7be4162170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:54:12.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.631+0000 7f7be9400640 1 --2- 172.21.15.131:0/3675267251 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7be4165e30 0x7f7be409c920 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f7bdc00c7a0 tx=0x7f7bdc00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:12.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.631+0000 7f7be1600640 1 -- 172.21.15.131:0/3675267251 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bdc018070 con 0x7f7be4165e30 2024-05-26T22:54:12.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.631+0000 7f7be1600640 1 -- 172.21.15.131:0/3675267251 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bdc00a640 con 0x7f7be4165e30 2024-05-26T22:54:12.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.631+0000 7f7be1600640 1 -- 172.21.15.131:0/3675267251 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bdc00aab0 con 0x7f7be4165e30 2024-05-26T22:54:12.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.631+0000 7f7beb400640 1 -- 172.21.15.131:0/3675267251 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7be409af30 con 0x7f7be4165e30 2024-05-26T22:54:12.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.631+0000 7f7beb400640 1 -- 172.21.15.131:0/3675267251 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7be409b4f0 con 0x7f7be4165e30 2024-05-26T22:54:12.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.634+0000 7f7be1600640 1 -- 172.21.15.131:0/3675267251 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7bdc01b070 con 0x7f7be4165e30 2024-05-26T22:54:12.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.634+0000 7f7be1600640 1 --2- 172.21.15.131:0/3675267251 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f7bb806be80 0x7f7bb806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:12.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.635+0000 7f7be1600640 1 -- 172.21.15.131:0/3675267251 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5738+0+0 (secure 0 0 0) 0x7f7bdc08a4b0 con 0x7f7be4165e30 2024-05-26T22:54:12.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.635+0000 7f7be8a00640 1 --2- 172.21.15.131:0/3675267251 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f7bb806be80 0x7f7bb806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:12.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.635+0000 7f7bee1e3740 1 Processor -- start 2024-05-26T22:54:12.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.635+0000 7f7bee1e3740 1 -- start start 2024-05-26T22:54:12.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.635+0000 7f7be8a00640 1 --2- 172.21.15.131:0/3675267251 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f7bb806be80 0x7f7bb806e340 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f7bc800a830 tx=0x7f7bc8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:12.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.636+0000 7f7bee1e3740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e6f7032c70 0x55e6f6fe5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:12.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.636+0000 7f7bee1e3740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e6f7015d90 0x55e6f7015110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:12.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.636+0000 7f7bee1e3740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e6f7015650 0x55e6f6fea460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:12.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.636+0000 7f7bee1e3740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55e6f6f4d610 con 0x55e6f7032c70 2024-05-26T22:54:12.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.636+0000 7f7bee1e3740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55e6f6ef55c0 con 0x55e6f7015650 2024-05-26T22:54:12.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.636+0000 7f7bee1e3740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x55e6f6f495d0 con 0x55e6f7015d90 2024-05-26T22:54:12.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.636+0000 7f7be8a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e6f7032c70 0x55e6f6fe5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:12.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.637+0000 7f7be8a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e6f7032c70 0x55e6f6fe5890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46470/0 (socket says 172.21.15.131:46470) 2024-05-26T22:54:12.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.637+0000 7f7be8a00640 1 -- 172.21.15.131:0/3064067233 learned_addr learned my addr 172.21.15.131:0/3064067233 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:54:12.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.637+0000 7f7be8a00640 1 -- 172.21.15.131:0/3064067233 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e6f7015d90 msgr2=0x55e6f7015110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:12.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.637+0000 7f7be8a00640 1 --2- 172.21.15.131:0/3064067233 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e6f7015d90 0x55e6f7015110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:12.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.637+0000 7f7be8a00640 1 -- 172.21.15.131:0/3064067233 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e6f7015650 msgr2=0x55e6f6fea460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:12.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.637+0000 7f7be8a00640 1 --2- 172.21.15.131:0/3064067233 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e6f7015650 0x55e6f6fea460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:12.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.637+0000 7f7be8a00640 1 -- 172.21.15.131:0/3064067233 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6f6c612d0 con 0x55e6f7032c70 2024-05-26T22:54:12.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.637+0000 7f7be8a00640 1 --2- 172.21.15.131:0/3064067233 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e6f7032c70 0x55e6f6fe5890 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f7bc800cac0 tx=0x7f7bc800f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10831993dbfc735f server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:12.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.637+0000 7f7bd9200640 1 -- 172.21.15.131:0/3064067233 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc800ae60 con 0x55e6f7032c70 2024-05-26T22:54:12.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.637+0000 7f7bd9200640 1 -- 172.21.15.131:0/3064067233 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bc8013dd0 con 0x55e6f7032c70 2024-05-26T22:54:12.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.637+0000 7f7bd9200640 1 -- 172.21.15.131:0/3064067233 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc8004260 con 0x55e6f7032c70 2024-05-26T22:54:12.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.638+0000 7f7bee1e3740 1 -- 172.21.15.131:0/3064067233 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e6f7032c70 msgr2=0x55e6f6fe5890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:12.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.638+0000 7f7bee1e3740 1 --2- 172.21.15.131:0/3064067233 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e6f7032c70 0x55e6f6fe5890 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f7bc800cac0 tx=0x7f7bc800f000 comp rx=0 tx=0).stop 2024-05-26T22:54:12.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.638+0000 7f7bee1e3740 1 -- 172.21.15.131:0/3064067233 shutdown_connections 2024-05-26T22:54:12.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.638+0000 7f7bee1e3740 1 --2- 172.21.15.131:0/3064067233 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e6f7015d90 0x55e6f7015110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:12.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.638+0000 7f7bee1e3740 1 --2- 172.21.15.131:0/3064067233 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e6f7015650 0x55e6f6fea460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:12.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.638+0000 7f7bee1e3740 1 --2- 172.21.15.131:0/3064067233 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e6f7032c70 0x55e6f6fe5890 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:12.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.638+0000 7f7bee1e3740 1 -- 172.21.15.131:0/3064067233 >> 172.21.15.131:0/3064067233 conn(0x55e6f6fe7b00 msgr2=0x55e6f7030110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:12.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.638+0000 7f7bee1e3740 1 -- 172.21.15.131:0/3064067233 shutdown_connections 2024-05-26T22:54:12.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.638+0000 7f7bee1e3740 1 -- 172.21.15.131:0/3064067233 wait complete. 2024-05-26T22:54:12.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.639+0000 7f7bee1e3740 1 -- start start 2024-05-26T22:54:12.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.639+0000 7f7bee1e3740 1 Processor -- start 2024-05-26T22:54:12.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.640+0000 7f7bee1e3740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e6f7015d90 0x55e6f70033e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:12.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.640+0000 7f7bee1e3740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e6f7032c70 0x55e6f7002bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:12.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.640+0000 7f7bee1e3740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e6f7005fb0 0x55e6f70174c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:12.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.640+0000 7f7bee1e3740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55e6f6fe9a20 con 0x55e6f7015d90 2024-05-26T22:54:12.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.640+0000 7f7bee1e3740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55e6f6f08910 con 0x55e6f7005fb0 2024-05-26T22:54:12.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.640+0000 7f7bee1e3740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x55e6f6fe8e10 con 0x55e6f7032c70 2024-05-26T22:54:12.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.640+0000 7f7bee1e3740 10 client.0 _ll_register_callbacks cb 0x55e6f6f44410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-26T22:54:12.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.640+0000 7f7be8a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e6f7015d90 0x55e6f70033e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:12.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.640+0000 7f7be8a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e6f7015d90 0x55e6f70033e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46476/0 (socket says 172.21.15.131:46476) 2024-05-26T22:54:12.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.640+0000 7f7be8a00640 1 -- 172.21.15.131:0/3235292360 learned_addr learned my addr 172.21.15.131:0/3235292360 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:54:12.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.641+0000 7f7be8a00640 1 -- 172.21.15.131:0/3235292360 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e6f7032c70 msgr2=0x55e6f7002bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:12.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.641+0000 7f7be8a00640 1 --2- 172.21.15.131:0/3235292360 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e6f7032c70 0x55e6f7002bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:12.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.641+0000 7f7be8a00640 1 -- 172.21.15.131:0/3235292360 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e6f7005fb0 msgr2=0x55e6f70174c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:12.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.641+0000 7f7be8a00640 1 --2- 172.21.15.131:0/3235292360 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e6f7005fb0 0x55e6f70174c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:12.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.641+0000 7f7be8a00640 1 -- 172.21.15.131:0/3235292360 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6f6d2b6b0 con 0x55e6f7015d90 2024-05-26T22:54:12.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.641+0000 7f7be8a00640 1 --2- 172.21.15.131:0/3235292360 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e6f7015d90 0x55e6f70033e0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f7bc80137f0 tx=0x7f7bc8004680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e05bad7fc46e7a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:12.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.641+0000 7f7bd6000640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-26T22:54:12.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.641+0000 7f7bd6000640 1 -- 172.21.15.131:0/3235292360 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc802e020 con 0x55e6f7015d90 2024-05-26T22:54:12.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.641+0000 7f7bd6000640 1 -- 172.21.15.131:0/3235292360 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 5738+0+0 (secure 0 0 0) 0x7f7bc80035f0 con 0x55e6f7015d90 2024-05-26T22:54:12.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.641+0000 7f7bee1e3740 1 -- 172.21.15.131:0/3235292360 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7bc8013dd0 con 0x55e6f7015d90 2024-05-26T22:54:12.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.642+0000 7f7bd6000640 1 -- 172.21.15.131:0/3235292360 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f7bc8038050 con 0x55e6f7015d90 2024-05-26T22:54:12.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.642+0000 7f7bee1e3740 10 client.7635 fetch_fsmap learned FSMap version 83 2024-05-26T22:54:12.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.642+0000 7f7bee1e3740 1 -- 172.21.15.131:0/3235292360 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=2+}) -- 0x55e6f7016fa0 con 0x55e6f7015d90 2024-05-26T22:54:12.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.642+0000 7f7bd6000640 1 -- 172.21.15.131:0/3235292360 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bc8013dd0 con 0x55e6f7015d90 2024-05-26T22:54:12.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.642+0000 7f7bd6000640 1 -- 172.21.15.131:0/3235292360 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 83) ==== 3534+0+0 (secure 0 0 0) 0x7f7bc803b020 con 0x55e6f7015d90 2024-05-26T22:54:12.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.642+0000 7f7bee1e3740 10 client.7635 fetch_fsmap finished waiting for FSMap version 83 2024-05-26T22:54:12.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.642+0000 7f7bee1e3740 10 client.7635 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-26T22:54:12.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.642+0000 7f7bee1e3740 20 client.7635 populate_metadata read hostname 'smithi131' 2024-05-26T22:54:12.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.642+0000 7f7bee1e3740 1 --2- 172.21.15.131:0/3235292360 >> [v2:172.21.15.131:6834/3411392567,v1:172.21.15.131:6836/3411392567] conn(0x55e6f704c1e0 0x55e6f704e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:12.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.642+0000 7f7bee1e3740 4 client.7635 mds_command: new command op to 7365 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-26T22:54:12.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.642+0000 7f7bee1e3740 1 -- 172.21.15.131:0/3235292360 --> [v2:172.21.15.131:6834/3411392567,v1:172.21.15.131:6836/3411392567] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e6f6f5ad40 con 0x55e6f704c1e0 2024-05-26T22:54:12.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.642+0000 7f7bdbe00640 1 --2- 172.21.15.131:0/3235292360 >> [v2:172.21.15.131:6834/3411392567,v1:172.21.15.131:6836/3411392567] conn(0x55e6f704c1e0 0x55e6f704e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:12.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.643+0000 7f7bdbe00640 1 --2- 172.21.15.131:0/3235292360 >> [v2:172.21.15.131:6834/3411392567,v1:172.21.15.131:6836/3411392567] conn(0x55e6f704c1e0 0x55e6f704e5d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=857b9364d2275d03 server_cookie=34591508b7877c7 in_seq=0 out_seq=0 2024-05-26T22:54:12.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.643+0000 7f7bd6000640 10 client.7635 ms_handle_connect on v2:172.21.15.131:6834/3411392567 2024-05-26T22:54:12.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.644+0000 7f7bd6000640 1 -- 172.21.15.131:0/3235292360 <== mds.0 v2:172.21.15.131:6834/3411392567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e6f6f5ad40 con 0x55e6f704c1e0 2024-05-26T22:54:12.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.644+0000 7f7bd6000640 10 client.7635 handle_command_reply: tid=0 2024-05-26T22:54:12.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.644+0000 7f7bee1e3740 1 client.7635 shutdown 2024-05-26T22:54:12.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.644+0000 7f7bee1e3740 1 -- 172.21.15.131:0/3235292360 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e6f7015d90 msgr2=0x55e6f70033e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:12.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.644+0000 7f7bee1e3740 1 --2- 172.21.15.131:0/3235292360 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e6f7015d90 0x55e6f70033e0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f7bc80137f0 tx=0x7f7bc8004680 comp rx=0 tx=0).stop 2024-05-26T22:54:12.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.644+0000 7f7bee1e3740 1 -- 172.21.15.131:0/3235292360 shutdown_connections 2024-05-26T22:54:12.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.644+0000 7f7bee1e3740 1 --2- 172.21.15.131:0/3235292360 >> [v2:172.21.15.131:6834/3411392567,v1:172.21.15.131:6836/3411392567] conn(0x55e6f704c1e0 0x55e6f704e5d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:12.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.644+0000 7f7bee1e3740 1 --2- 172.21.15.131:0/3235292360 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e6f7032c70 0x55e6f7002bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:12.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.644+0000 7f7bee1e3740 1 --2- 172.21.15.131:0/3235292360 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e6f7005fb0 0x55e6f70174c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:12.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.644+0000 7f7bd6000640 0 client.7635 ms_handle_reset on v2:172.21.15.131:6834/3411392567 2024-05-26T22:54:12.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.644+0000 7f7bee1e3740 1 --2- 172.21.15.131:0/3235292360 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e6f7015d90 0x55e6f70033e0 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:12.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.644+0000 7f7bee1e3740 1 -- 172.21.15.131:0/3235292360 >> 172.21.15.131:0/3235292360 conn(0x55e6f7015650 msgr2=0x55e6f7033530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:12.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.644+0000 7f7bee1e3740 1 -- 172.21.15.131:0/3235292360 shutdown_connections 2024-05-26T22:54:12.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.644+0000 7f7bee1e3740 1 -- 172.21.15.131:0/3235292360 wait complete. 2024-05-26T22:54:12.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.644+0000 7f7bee1e3740 20 client.7635 trim_cache size 0 max 16384 2024-05-26T22:54:12.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.664+0000 7f7bee1e3740 1 Processor -- start 2024-05-26T22:54:12.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.664+0000 7f7bee1e3740 1 -- start start 2024-05-26T22:54:12.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.665+0000 7f7bee1e3740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e6f7005fb0 0x55e6f70033b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:12.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.665+0000 7f7bee1e3740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e6f7015d90 0x55e6f6fe64e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:12.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.665+0000 7f7bee1e3740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e6f7032c70 0x55e6f7017d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:12.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.665+0000 7f7bee1e3740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f7be4203ff0 con 0x55e6f7005fb0 2024-05-26T22:54:12.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.665+0000 7f7bee1e3740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7be42046c0 con 0x55e6f7015d90 2024-05-26T22:54:12.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.665+0000 7f7bee1e3740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f7be40adcb0 con 0x55e6f7032c70 2024-05-26T22:54:12.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.666+0000 7f7be8a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e6f7005fb0 0x55e6f70033b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:12.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.666+0000 7f7be8a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e6f7005fb0 0x55e6f70033b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46480/0 (socket says 172.21.15.131:46480) 2024-05-26T22:54:12.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.666+0000 7f7be8a00640 1 -- 172.21.15.131:0/3830449783 learned_addr learned my addr 172.21.15.131:0/3830449783 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:54:12.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.666+0000 7f7be8a00640 1 -- 172.21.15.131:0/3830449783 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e6f7032c70 msgr2=0x55e6f7017d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:12.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.666+0000 7f7be8a00640 1 --2- 172.21.15.131:0/3830449783 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e6f7032c70 0x55e6f7017d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:12.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.666+0000 7f7be8a00640 1 -- 172.21.15.131:0/3830449783 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e6f7015d90 msgr2=0x55e6f6fe64e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:12.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.666+0000 7f7be8a00640 1 --2- 172.21.15.131:0/3830449783 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e6f7015d90 0x55e6f6fe64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:12.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.666+0000 7f7be8a00640 1 -- 172.21.15.131:0/3830449783 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6f6fe1810 con 0x55e6f7005fb0 2024-05-26T22:54:12.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.666+0000 7f7be8a00640 1 --2- 172.21.15.131:0/3830449783 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e6f7005fb0 0x55e6f70033b0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f7bc8003a00 tx=0x7f7bc8027000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61049828e53260f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:12.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.666+0000 7f7bd8800640 1 -- 172.21.15.131:0/3830449783 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc802e040 con 0x55e6f7005fb0 2024-05-26T22:54:12.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.666+0000 7f7bd8800640 1 -- 172.21.15.131:0/3830449783 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bc8003b60 con 0x55e6f7005fb0 2024-05-26T22:54:12.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.666+0000 7f7bd8800640 1 -- 172.21.15.131:0/3830449783 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc8003f50 con 0x55e6f7005fb0 2024-05-26T22:54:12.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.667+0000 7f7bee1e3740 1 -- 172.21.15.131:0/3830449783 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e6f7005fb0 msgr2=0x55e6f70033b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:12.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.667+0000 7f7bee1e3740 1 --2- 172.21.15.131:0/3830449783 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e6f7005fb0 0x55e6f70033b0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f7bc8003a00 tx=0x7f7bc8027000 comp rx=0 tx=0).stop 2024-05-26T22:54:12.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.667+0000 7f7bee1e3740 1 -- 172.21.15.131:0/3830449783 shutdown_connections 2024-05-26T22:54:12.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.667+0000 7f7bee1e3740 1 --2- 172.21.15.131:0/3830449783 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e6f7032c70 0x55e6f7017d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:12.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.667+0000 7f7bee1e3740 1 --2- 172.21.15.131:0/3830449783 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e6f7015d90 0x55e6f6fe64e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:12.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.667+0000 7f7bee1e3740 1 --2- 172.21.15.131:0/3830449783 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e6f7005fb0 0x55e6f70033b0 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:12.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.667+0000 7f7bee1e3740 1 -- 172.21.15.131:0/3830449783 >> 172.21.15.131:0/3830449783 conn(0x55e6f7015650 msgr2=0x55e6f7033530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:12.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.667+0000 7f7bee1e3740 1 -- 172.21.15.131:0/3830449783 shutdown_connections 2024-05-26T22:54:12.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.667+0000 7f7bee1e3740 1 -- 172.21.15.131:0/3830449783 wait complete. 2024-05-26T22:54:12.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.668+0000 7f7bee1e3740 1 -- start start 2024-05-26T22:54:12.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.668+0000 7f7bee1e3740 1 Processor -- start 2024-05-26T22:54:12.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.669+0000 7f7bee1e3740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e6f7005fb0 0x55e6f70267d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:12.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.669+0000 7f7bee1e3740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e6f7015d90 0x55e6f70940e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:12.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.669+0000 7f7bee1e3740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e6f7032c70 0x55e6f7094a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:12.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.669+0000 7f7bee1e3740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55e6f7027f50 con 0x55e6f7005fb0 2024-05-26T22:54:12.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.669+0000 7f7bee1e3740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55e6f6fe9a20 con 0x55e6f7015d90 2024-05-26T22:54:12.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.669+0000 7f7bee1e3740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x55e6f7028b10 con 0x55e6f7032c70 2024-05-26T22:54:12.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.669+0000 7f7bee1e3740 10 client.0 _ll_register_callbacks cb 0x7f7bc800c960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-26T22:54:12.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.669+0000 7f7be8a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e6f7005fb0 0x55e6f70267d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:12.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.669+0000 7f7be8a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e6f7005fb0 0x55e6f70267d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46488/0 (socket says 172.21.15.131:46488) 2024-05-26T22:54:12.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.669+0000 7f7be8a00640 1 -- 172.21.15.131:0/314333320 learned_addr learned my addr 172.21.15.131:0/314333320 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:54:12.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.670+0000 7f7be9400640 1 --2- 172.21.15.131:0/314333320 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e6f7032c70 0x55e6f7094a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:12.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.670+0000 7f7be8a00640 1 -- 172.21.15.131:0/314333320 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e6f7032c70 msgr2=0x55e6f7094a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:12.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.670+0000 7f7bdbe00640 1 --2- 172.21.15.131:0/314333320 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e6f7015d90 0x55e6f70940e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:12.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.670+0000 7f7be8a00640 1 --2- 172.21.15.131:0/314333320 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e6f7032c70 0x55e6f7094a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:12.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.670+0000 7f7be8a00640 1 -- 172.21.15.131:0/314333320 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e6f7015d90 msgr2=0x55e6f70940e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:12.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.670+0000 7f7be8a00640 1 --2- 172.21.15.131:0/314333320 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e6f7015d90 0x55e6f70940e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:12.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.670+0000 7f7be8a00640 1 -- 172.21.15.131:0/314333320 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6f7003530 con 0x55e6f7005fb0 2024-05-26T22:54:12.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.670+0000 7f7be8a00640 1 --2- 172.21.15.131:0/314333320 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e6f7005fb0 0x55e6f70267d0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f7bc80137f0 tx=0x7f7bc8012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f6bb2e991b08c0f server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:12.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.670+0000 7f7bd6000640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-26T22:54:12.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.670+0000 7f7bd6000640 1 -- 172.21.15.131:0/314333320 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc8042070 con 0x55e6f7005fb0 2024-05-26T22:54:12.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.670+0000 7f7bd6000640 1 -- 172.21.15.131:0/314333320 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 5738+0+0 (secure 0 0 0) 0x7f7bc802a880 con 0x55e6f7005fb0 2024-05-26T22:54:12.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.670+0000 7f7bee1e3740 1 -- 172.21.15.131:0/314333320 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7bc8003b60 con 0x55e6f7005fb0 2024-05-26T22:54:12.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.671+0000 7f7bd6000640 1 -- 172.21.15.131:0/314333320 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f7bc804e050 con 0x55e6f7005fb0 2024-05-26T22:54:12.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.671+0000 7f7bee1e3740 10 client.7641 fetch_fsmap learned FSMap version 83 2024-05-26T22:54:12.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.671+0000 7f7bee1e3740 1 -- 172.21.15.131:0/314333320 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=2+}) -- 0x55e6f7095530 con 0x55e6f7005fb0 2024-05-26T22:54:12.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.671+0000 7f7bd6000640 1 -- 172.21.15.131:0/314333320 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bc8003b60 con 0x55e6f7005fb0 2024-05-26T22:54:12.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.671+0000 7f7bd6000640 1 -- 172.21.15.131:0/314333320 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 83) ==== 3534+0+0 (secure 0 0 0) 0x7f7bc8051020 con 0x55e6f7005fb0 2024-05-26T22:54:12.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.671+0000 7f7bee1e3740 10 client.7641 fetch_fsmap finished waiting for FSMap version 83 2024-05-26T22:54:12.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.671+0000 7f7bee1e3740 10 client.7641 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-26T22:54:12.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.671+0000 7f7bee1e3740 20 client.7641 populate_metadata read hostname 'smithi131' 2024-05-26T22:54:12.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.671+0000 7f7bee1e3740 1 --2- 172.21.15.131:0/314333320 >> [v2:172.21.15.131:6834/3411392567,v1:172.21.15.131:6836/3411392567] conn(0x55e6f6fe4920 0x55e6f6fe4d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:12.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.671+0000 7f7bee1e3740 4 client.7641 mds_command: new command op to 7365 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-26T22:54:12.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.671+0000 7f7bee1e3740 1 -- 172.21.15.131:0/314333320 --> [v2:172.21.15.131:6834/3411392567,v1:172.21.15.131:6836/3411392567] -- command(tid 0: {"prefix": "status"}) -- 0x55e6f6f55090 con 0x55e6f6fe4920 2024-05-26T22:54:12.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.671+0000 7f7bdbe00640 1 --2- 172.21.15.131:0/314333320 >> [v2:172.21.15.131:6834/3411392567,v1:172.21.15.131:6836/3411392567] conn(0x55e6f6fe4920 0x55e6f6fe4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:12.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.672+0000 7f7bdbe00640 1 --2- 172.21.15.131:0/314333320 >> [v2:172.21.15.131:6834/3411392567,v1:172.21.15.131:6836/3411392567] conn(0x55e6f6fe4920 0x55e6f6fe4d00 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5cc4d1ed88893c1 server_cookie=86fa78823013b193 in_seq=0 out_seq=0 2024-05-26T22:54:12.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.672+0000 7f7bd6000640 10 client.7641 ms_handle_connect on v2:172.21.15.131:6834/3411392567 2024-05-26T22:54:12.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.672+0000 7f7bd6000640 1 -- 172.21.15.131:0/314333320 <== mds.0 v2:172.21.15.131:6834/3411392567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x55e6f6f55090 con 0x55e6f6fe4920 2024-05-26T22:54:12.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.672+0000 7f7bd6000640 10 client.7641 handle_command_reply: tid=0 2024-05-26T22:54:12.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.672+0000 7f7bee1e3740 1 client.7641 shutdown 2024-05-26T22:54:12.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.672+0000 7f7bee1e3740 1 -- 172.21.15.131:0/314333320 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e6f7005fb0 msgr2=0x55e6f70267d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:12.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.672+0000 7f7bee1e3740 1 --2- 172.21.15.131:0/314333320 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e6f7005fb0 0x55e6f70267d0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f7bc80137f0 tx=0x7f7bc8012000 comp rx=0 tx=0).stop 2024-05-26T22:54:12.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.672+0000 7f7bee1e3740 1 -- 172.21.15.131:0/314333320 shutdown_connections 2024-05-26T22:54:12.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.672+0000 7f7bee1e3740 1 --2- 172.21.15.131:0/314333320 >> [v2:172.21.15.131:6834/3411392567,v1:172.21.15.131:6836/3411392567] conn(0x55e6f6fe4920 0x55e6f6fe4d00 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:12.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.672+0000 7f7bee1e3740 1 --2- 172.21.15.131:0/314333320 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e6f7032c70 0x55e6f7094a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:12.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.672+0000 7f7bd6000640 0 client.7641 ms_handle_reset on v2:172.21.15.131:6834/3411392567 2024-05-26T22:54:12.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.672+0000 7f7bee1e3740 1 --2- 172.21.15.131:0/314333320 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e6f7015d90 0x55e6f70940e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:12.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.672+0000 7f7bee1e3740 1 --2- 172.21.15.131:0/314333320 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e6f7005fb0 0x55e6f70267d0 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:12.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.672+0000 7f7bee1e3740 1 -- 172.21.15.131:0/314333320 >> 172.21.15.131:0/314333320 conn(0x55e6f7015650 msgr2=0x55e6f7033530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:12.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.672+0000 7f7bee1e3740 1 -- 172.21.15.131:0/314333320 shutdown_connections 2024-05-26T22:54:12.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.672+0000 7f7bee1e3740 1 -- 172.21.15.131:0/314333320 wait complete. 2024-05-26T22:54:12.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.672+0000 7f7bee1e3740 20 client.7641 trim_cache size 0 max 16384 2024-05-26T22:54:12.676 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-26T22:54:12.676 INFO:teuthology.orchestra.run.smithi131.stdout: "cluster_fsid": "46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f", 2024-05-26T22:54:12.676 INFO:teuthology.orchestra.run.smithi131.stdout: "whoami": 0, 2024-05-26T22:54:12.676 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 7365, 2024-05-26T22:54:12.676 INFO:teuthology.orchestra.run.smithi131.stdout: "want_state": "up:active", 2024-05-26T22:54:12.676 INFO:teuthology.orchestra.run.smithi131.stdout: "state": "up:active", 2024-05-26T22:54:12.676 INFO:teuthology.orchestra.run.smithi131.stdout: "fs_name": "cephfs", 2024-05-26T22:54:12.677 INFO:teuthology.orchestra.run.smithi131.stdout: "rank_uptime": 19.240180148, 2024-05-26T22:54:12.677 INFO:teuthology.orchestra.run.smithi131.stdout: "mdsmap_epoch": 83, 2024-05-26T22:54:12.677 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch": 84, 2024-05-26T22:54:12.677 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch_barrier": 84, 2024-05-26T22:54:12.677 INFO:teuthology.orchestra.run.smithi131.stdout: "uptime": 24.678446623999999 2024-05-26T22:54:12.677 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-26T22:54:12.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.673+0000 7f7beb400640 1 -- 172.21.15.131:0/3675267251 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f7bb806be80 msgr2=0x7f7bb806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:12.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.673+0000 7f7beb400640 1 --2- 172.21.15.131:0/3675267251 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f7bb806be80 0x7f7bb806e340 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f7bc800a830 tx=0x7f7bc8008040 comp rx=0 tx=0).stop 2024-05-26T22:54:12.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.673+0000 7f7beb400640 1 -- 172.21.15.131:0/3675267251 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7be4165e30 msgr2=0x7f7be409c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:12.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.673+0000 7f7beb400640 1 --2- 172.21.15.131:0/3675267251 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7be4165e30 0x7f7be409c920 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f7bdc00c7a0 tx=0x7f7bdc00cc60 comp rx=0 tx=0).stop 2024-05-26T22:54:12.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.673+0000 7f7beb400640 1 -- 172.21.15.131:0/3675267251 shutdown_connections 2024-05-26T22:54:12.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.673+0000 7f7beb400640 1 --2- 172.21.15.131:0/3675267251 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7be40d0560 0x7f7be409bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:12.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.673+0000 7f7beb400640 1 --2- 172.21.15.131:0/3675267251 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f7bb806be80 0x7f7bb806e340 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:12.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.673+0000 7f7beb400640 1 --2- 172.21.15.131:0/3675267251 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7be4165e30 0x7f7be409c920 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:12.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.673+0000 7f7beb400640 1 --2- 172.21.15.131:0/3675267251 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7be40af230 0x7f7be4162170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:12.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.673+0000 7f7beb400640 1 -- 172.21.15.131:0/3675267251 >> 172.21.15.131:0/3675267251 conn(0x7f7be406bc90 msgr2=0x7f7be40b1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:12.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.673+0000 7f7beb400640 1 -- 172.21.15.131:0/3675267251 shutdown_connections 2024-05-26T22:54:12.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12.673+0000 7f7beb400640 1 -- 172.21.15.131:0/3675267251 wait complete. 2024-05-26T22:54:12.689 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-05-26T22:54:12.689 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-26T22:54:12.689 DEBUG:teuthology.orchestra.run.smithi131:> mkdir -p tree 2024-05-26T22:54:12.689 DEBUG:teuthology.orchestra.run.smithi131:> 2024-05-26T22:54:12.689 DEBUG:teuthology.orchestra.run.smithi131:> setfattr -n ceph.dir.pin.distributed -v 1 tree 2024-05-26T22:54:12.689 DEBUG:teuthology.orchestra.run.smithi131:> 2024-05-26T22:54:12.689 DEBUG:teuthology.orchestra.run.smithi131:> for ((i = 0; i < 100; i++)); do 2024-05-26T22:54:12.689 DEBUG:teuthology.orchestra.run.smithi131:> mkdir -p "tree/$i" 2024-05-26T22:54:12.689 DEBUG:teuthology.orchestra.run.smithi131:> echo file > "tree/$i/file" 2024-05-26T22:54:12.689 DEBUG:teuthology.orchestra.run.smithi131:> done 2024-05-26T22:54:12.689 DEBUG:teuthology.orchestra.run.smithi131:> ') 2024-05-26T22:54:12.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:12 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-26T22:54:12.788 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree 2024-05-26T22:54:12.795 INFO:teuthology.orchestra.run.smithi131.stderr:+ setfattr -n ceph.dir.pin.distributed -v 1 tree 2024-05-26T22:54:12.833 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i = 0 )) 2024-05-26T22:54:12.833 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:54:12.833 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/0 2024-05-26T22:54:12.838 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:54:12.845 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:54:12.845 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:54:12.845 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/1 2024-05-26T22:54:12.849 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:54:12.851 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:54:12.851 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:54:12.851 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/2 2024-05-26T22:54:12.856 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:54:12.858 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:54:12.858 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:54:12.858 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/3 2024-05-26T22:54:12.863 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:54:12.866 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:54:12.866 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:54:12.866 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/4 2024-05-26T22:54:12.871 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:54:12.873 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:54:12.873 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:54:12.873 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/5 2024-05-26T22:54:12.879 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:54:12.881 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:54:12.881 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:54:12.881 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/6 2024-05-26T22:54:12.886 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:54:12.888 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:54:12.888 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:54:12.888 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/7 2024-05-26T22:54:12.894 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:54:12.896 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:54:12.896 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:54:12.896 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/8 2024-05-26T22:54:12.901 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:54:12.904 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:54:12.904 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:54:12.904 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/9 2024-05-26T22:54:12.909 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:54:12.911 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:54:12.912 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:54:12.912 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/10 2024-05-26T22:54:12.917 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:54:12.919 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:54:12.919 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:54:12.919 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/11 2024-05-26T22:54:12.925 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:54:12.927 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:54:12.927 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:54:12.927 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/12 2024-05-26T22:54:12.933 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:54:12.935 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:54:12.935 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:54:12.935 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/13 2024-05-26T22:54:12.940 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:54:12.942 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:54:12.944 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:54:12.944 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/14 2024-05-26T22:54:12.948 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:54:12.950 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:54:12.950 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:54:12.950 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/15 2024-05-26T22:54:12.955 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:54:12.958 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:54:12.958 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:54:12.958 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/16 2024-05-26T22:54:12.964 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:54:12.966 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:54:12.966 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:54:12.966 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/17 2024-05-26T22:54:12.971 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:54:12.975 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:54:12.975 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:54:12.975 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/18 2024-05-26T22:54:12.982 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:54:12.985 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:54:12.985 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:54:12.985 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/19 2024-05-26T22:54:12.992 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:54:12.995 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:54:12.995 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:54:12.996 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/20 2024-05-26T22:54:13.001 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:54:13.002 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:54:13.002 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:54:13.002 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/21 2024-05-26T22:54:13.009 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:54:13.012 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:54:13.012 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:54:13.012 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/22 2024-05-26T22:54:13.019 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:54:13.022 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:54:13.022 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:54:13.022 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/23 2024-05-26T22:54:13.028 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:54:13.031 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:54:13.031 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:54:13.031 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/24 2024-05-26T22:54:13.036 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:54:13.039 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:54:13.039 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:54:13.039 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/25 2024-05-26T22:54:13.044 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:54:13.047 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:54:13.047 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:54:13.047 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/26 2024-05-26T22:54:13.053 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:54:13.055 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:54:13.055 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:54:13.056 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/27 2024-05-26T22:54:13.061 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:54:13.065 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:54:13.065 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:54:13.065 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/28 2024-05-26T22:54:13.072 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:54:13.077 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:54:13.078 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:54:13.078 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/29 2024-05-26T22:54:13.083 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:54:13.086 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:54:13.086 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:54:13.086 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/30 2024-05-26T22:54:13.092 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:54:13.096 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:54:13.096 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:54:13.096 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/31 2024-05-26T22:54:13.101 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:54:13.105 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:54:13.105 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:54:13.105 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/32 2024-05-26T22:54:13.111 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:54:13.114 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:54:13.114 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:54:13.115 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/33 2024-05-26T22:54:13.120 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:54:13.124 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:54:13.124 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:54:13.124 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/34 2024-05-26T22:54:13.130 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:54:13.133 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:54:13.134 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:54:13.134 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/35 2024-05-26T22:54:13.139 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:54:13.143 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:54:13.143 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:54:13.143 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/36 2024-05-26T22:54:13.149 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:54:13.151 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:54:13.151 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:54:13.151 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/37 2024-05-26T22:54:13.157 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:54:13.159 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:54:13.159 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:54:13.160 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/38 2024-05-26T22:54:13.165 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:54:13.168 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:54:13.168 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:54:13.168 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/39 2024-05-26T22:54:13.175 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:54:13.177 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:54:13.178 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:54:13.178 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/40 2024-05-26T22:54:13.182 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:54:13.184 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:54:13.184 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:54:13.184 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/41 2024-05-26T22:54:13.189 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:54:13.192 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:54:13.192 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:54:13.192 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/42 2024-05-26T22:54:13.197 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:54:13.199 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:54:13.199 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:54:13.199 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/43 2024-05-26T22:54:13.204 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:54:13.207 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:54:13.207 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:54:13.207 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/44 2024-05-26T22:54:13.212 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:54:13.215 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:54:13.215 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:54:13.215 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/45 2024-05-26T22:54:13.220 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:54:13.223 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:54:13.223 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:54:13.223 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/46 2024-05-26T22:54:13.229 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:54:13.230 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:54:13.230 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:54:13.231 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/47 2024-05-26T22:54:13.236 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:54:13.237 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:54:13.237 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:54:13.237 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/48 2024-05-26T22:54:13.242 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:54:13.244 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:54:13.244 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:54:13.244 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/49 2024-05-26T22:54:13.250 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:54:13.252 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:54:13.252 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:54:13.252 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/50 2024-05-26T22:54:13.258 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:54:13.260 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:54:13.260 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:54:13.260 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/51 2024-05-26T22:54:13.266 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:54:13.268 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:54:13.268 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:54:13.269 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/52 2024-05-26T22:54:13.274 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:54:13.276 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:54:13.277 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:54:13.277 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/53 2024-05-26T22:54:13.283 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:54:13.285 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:54:13.285 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:54:13.285 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/54 2024-05-26T22:54:13.291 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:54:13.293 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:54:13.293 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:54:13.294 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/55 2024-05-26T22:54:13.298 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:54:13.300 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:54:13.300 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:54:13.300 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/56 2024-05-26T22:54:13.305 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:54:13.306 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:54:13.306 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:54:13.306 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/57 2024-05-26T22:54:13.311 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:54:13.313 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:54:13.313 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:54:13.313 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/58 2024-05-26T22:54:13.317 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:54:13.318 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:54:13.318 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:54:13.319 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/59 2024-05-26T22:54:13.322 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:54:13.324 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:54:13.324 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:54:13.324 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/60 2024-05-26T22:54:13.328 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:54:13.329 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:54:13.329 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:54:13.329 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/61 2024-05-26T22:54:13.334 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:54:13.335 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:54:13.335 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:54:13.335 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/62 2024-05-26T22:54:13.339 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:54:13.340 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:54:13.340 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:54:13.340 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/63 2024-05-26T22:54:13.344 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:54:13.346 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:54:13.346 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:54:13.346 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/64 2024-05-26T22:54:13.350 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:54:13.351 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:54:13.351 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:54:13.352 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/65 2024-05-26T22:54:13.356 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:54:13.357 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:54:13.357 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:54:13.357 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/66 2024-05-26T22:54:13.359 INFO:tasks.ceph.mds.a.smithi131.stderr:2024-05-26T22:54:13.359+0000 7f8c96c00640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-05-26T22:54:13.359 INFO:tasks.ceph.mds.a.smithi131.stderr:2024-05-26T22:54:13.359+0000 7f8c96c00640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-05-26T22:54:13.361 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:54:13.362 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:54:13.362 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:54:13.362 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/67 2024-05-26T22:54:13.366 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:54:13.367 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:54:13.368 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:54:13.368 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/68 2024-05-26T22:54:13.372 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:54:13.373 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:54:13.373 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:54:13.373 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/69 2024-05-26T22:54:13.378 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:54:13.379 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:54:13.379 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:54:13.379 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/70 2024-05-26T22:54:13.383 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:54:13.384 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:54:13.384 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:54:13.384 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/71 2024-05-26T22:54:13.388 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:54:13.389 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:54:13.390 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:54:13.390 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/72 2024-05-26T22:54:13.393 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:54:13.395 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:54:13.395 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:54:13.395 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/73 2024-05-26T22:54:13.399 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:54:13.400 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:54:13.400 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:54:13.400 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/74 2024-05-26T22:54:13.404 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:54:13.405 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:54:13.405 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:54:13.405 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/75 2024-05-26T22:54:13.409 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:54:13.410 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:54:13.410 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:54:13.411 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/76 2024-05-26T22:54:13.415 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:54:13.416 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:54:13.416 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:54:13.416 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/77 2024-05-26T22:54:13.421 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:54:13.422 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:54:13.422 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:54:13.423 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/78 2024-05-26T22:54:13.427 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:54:13.428 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:54:13.428 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:54:13.428 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/79 2024-05-26T22:54:13.433 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:54:13.434 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:54:13.434 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:54:13.434 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/80 2024-05-26T22:54:13.440 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:54:13.441 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:54:13.441 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:54:13.441 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/81 2024-05-26T22:54:13.445 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:54:13.446 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:54:13.446 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:54:13.446 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/82 2024-05-26T22:54:13.450 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:54:13.452 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:54:13.452 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:54:13.452 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/83 2024-05-26T22:54:13.457 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:54:13.458 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:54:13.458 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:54:13.458 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/84 2024-05-26T22:54:13.462 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:54:13.464 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:54:13.464 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:54:13.464 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/85 2024-05-26T22:54:13.469 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:54:13.470 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:54:13.470 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:54:13.470 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/86 2024-05-26T22:54:13.476 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:54:13.478 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:54:13.478 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:54:13.478 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/87 2024-05-26T22:54:13.487 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:54:13.487 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:54:13.487 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:54:13.487 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/88 2024-05-26T22:54:13.493 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:54:13.495 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:54:13.495 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:54:13.495 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/89 2024-05-26T22:54:13.500 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:54:13.502 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:54:13.502 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:54:13.502 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/90 2024-05-26T22:54:13.507 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:54:13.510 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:54:13.510 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:54:13.510 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/91 2024-05-26T22:54:13.515 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:54:13.517 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:54:13.518 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:54:13.518 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/92 2024-05-26T22:54:13.523 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:54:13.525 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:54:13.525 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:54:13.526 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/93 2024-05-26T22:54:13.531 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:54:13.533 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:54:13.533 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:54:13.533 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/94 2024-05-26T22:54:13.539 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:54:13.541 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:54:13.541 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:54:13.542 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/95 2024-05-26T22:54:13.547 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:54:13.549 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:54:13.549 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:54:13.549 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/96 2024-05-26T22:54:13.554 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:54:13.556 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:54:13.556 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:54:13.556 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/97 2024-05-26T22:54:13.561 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:54:13.564 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:54:13.564 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:54:13.564 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/98 2024-05-26T22:54:13.569 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:54:13.571 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:54:13.571 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:54:13.571 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/99 2024-05-26T22:54:13.576 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:54:13.578 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:54:13.578 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:54:13.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-26T22:54:13.592 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json get subtrees 2024-05-26T22:54:13.821 INFO:teuthology.orchestra.run.smithi187.stdout:[{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds2","dirfrag":"0x102","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"2","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-05-26T22:54:13.822 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "2", "dirfrag": "0x102", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds2", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-05-26T22:54:13.822 DEBUG:tasks.cephfs.cephfs_test_case:[ { "is_auth": true, "auth_first": 2, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "~mds2", "dirfrag": "0x102", "snapid_first": 2, "projected_version": "1", "version": "1", "committing_version": "1", "committed_version": "1", "is_rep": false, "dir_auth": "2", "states": [ "auth", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "dirty": 0, "authpin": 0 }, "nref": 2 } } ] 2024-05-26T22:54:13.822 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json get subtrees 2024-05-26T22:54:14.034 INFO:teuthology.orchestra.run.smithi187.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-05-26T22:54:14.034 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-05-26T22:54:14.035 DEBUG:tasks.cephfs.cephfs_test_case:[ { "is_auth": true, "auth_first": 1, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "~mds1", "dirfrag": "0x101", "snapid_first": 2, "projected_version": "1", "version": "1", "committing_version": "1", "committed_version": "1", "is_rep": false, "dir_auth": "1", "states": [ "auth", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "dirty": 0, "authpin": 0 }, "nref": 2 } } ] 2024-05-26T22:54:14.035 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-05-26T22:54:14.265 INFO:teuthology.orchestra.run.smithi131.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"2930","version":"2770","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":80,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1686","version":"1686","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":1,"authpin":0},"nref":3}}] 2024-05-26T22:54:14.265 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 80, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "2930", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "2770" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 1, "subtree": 1 }, "projected_version": "1686", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "1686" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-05-26T22:54:14.265 DEBUG:tasks.cephfs.cephfs_test_case:[ { "is_auth": true, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "", "dirfrag": "0x1", "snapid_first": 2, "projected_version": "2930", "version": "2770", "committing_version": "1", "committed_version": "1", "is_rep": true, "dir_auth": "0", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 80, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "replicated": 0, "dirty": 1, "authpin": 1 }, "nref": 4 } }, { "is_auth": true, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "~mds0", "dirfrag": "0x100", "snapid_first": 2, "projected_version": "1686", "version": "1686", "committing_version": "1", "committed_version": "1", "is_rep": false, "dir_auth": "0", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "dirty": 1, "authpin": 0 }, "nref": 3 } } ] 2024-05-26T22:54:14.266 INFO:tasks.cephfs.cephfs_test_case:len=0 [] 2024-05-26T22:54:15.389 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-26T22:54:15.397 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-26T22:54:19.267 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json get subtrees 2024-05-26T22:54:19.500 INFO:teuthology.orchestra.run.smithi187.stdout:[{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds2","dirfrag":"0x102","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"2","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-05-26T22:54:19.500 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "2", "dirfrag": "0x102", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds2", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-05-26T22:54:19.501 DEBUG:tasks.cephfs.cephfs_test_case:[ { "is_auth": true, "auth_first": 2, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "~mds2", "dirfrag": "0x102", "snapid_first": 2, "projected_version": "1", "version": "1", "committing_version": "1", "committed_version": "1", "is_rep": false, "dir_auth": "2", "states": [ "auth", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "dirty": 0, "authpin": 0 }, "nref": 2 } } ] 2024-05-26T22:54:19.501 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json get subtrees 2024-05-26T22:54:19.721 INFO:teuthology.orchestra.run.smithi187.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-05-26T22:54:19.721 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-05-26T22:54:19.722 DEBUG:tasks.cephfs.cephfs_test_case:[ { "is_auth": true, "auth_first": 1, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "~mds1", "dirfrag": "0x101", "snapid_first": 2, "projected_version": "1", "version": "1", "committing_version": "1", "committed_version": "1", "is_rep": false, "dir_auth": "1", "states": [ "auth", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "dirty": 0, "authpin": 0 }, "nref": 2 } } ] 2024-05-26T22:54:19.722 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-05-26T22:54:19.953 INFO:teuthology.orchestra.run.smithi131.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"2937","version":"2937","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1686","version":"1686","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":1,"authpin":0},"nref":3}}] 2024-05-26T22:54:19.953 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "2937", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "2937" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 1, "subtree": 1 }, "projected_version": "1686", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "1686" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-05-26T22:54:19.954 DEBUG:tasks.cephfs.cephfs_test_case:[ { "is_auth": true, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "", "dirfrag": "0x1", "snapid_first": 2, "projected_version": "2937", "version": "2937", "committing_version": "1", "committed_version": "1", "is_rep": true, "dir_auth": "0", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "replicated": 0, "dirty": 1, "authpin": 0 }, "nref": 3 } }, { "is_auth": true, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "~mds0", "dirfrag": "0x100", "snapid_first": 2, "projected_version": "1686", "version": "1686", "committing_version": "1", "committed_version": "1", "is_rep": false, "dir_auth": "0", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "dirty": 1, "authpin": 0 }, "nref": 3 } } ] 2024-05-26T22:54:19.954 INFO:tasks.cephfs.cephfs_test_case:len=0 [] 2024-05-26T22:54:24.955 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json get subtrees 2024-05-26T22:54:25.186 INFO:teuthology.orchestra.run.smithi187.stdout:[{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds2","dirfrag":"0x102","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"2","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"2937","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":3},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/tree","dirfrag":"0x1000000041a.111*","snapid_first":2,"projected_version":"416","version":"416","committing_version":"402","committed_version":"402","is_rep":false,"dir_auth":"2","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/tree","dirfrag":"0x1000000041a.100*","snapid_first":2,"projected_version":"426","version":"426","committing_version":"402","committed_version":"402","is_rep":false,"dir_auth":"2","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/tree","dirfrag":"0x1000000041a.010*","snapid_first":2,"projected_version":"426","version":"426","committing_version":"402","committed_version":"402","is_rep":false,"dir_auth":"2","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2024-05-26T22:54:25.187 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "2", "dirfrag": "0x102", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds2", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 3 }, "snapid_first": 2, "states": [], "version": "2937" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "402", "committing_version": "402", "dir_auth": "2", "dirfrag": "0x1000000041a.111*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/tree", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "416", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "416" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 2, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "402", "committing_version": "402", "dir_auth": "2", "dirfrag": "0x1000000041a.100*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/tree", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "426", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "426" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 2, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "402", "committing_version": "402", "dir_auth": "2", "dirfrag": "0x1000000041a.010*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/tree", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "426", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "426" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 2, "is_auth": true, "random_ephemeral_pin": false } ] 2024-05-26T22:54:25.187 DEBUG:tasks.cephfs.cephfs_test_case:[ { "is_auth": true, "auth_first": 2, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "~mds2", "dirfrag": "0x102", "snapid_first": 2, "projected_version": "1", "version": "1", "committing_version": "1", "committed_version": "1", "is_rep": false, "dir_auth": "2", "states": [ "auth", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "dirty": 0, "authpin": 0 }, "nref": 2 } }, { "is_auth": false, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "", "dirfrag": "0x1", "snapid_first": 2, "projected_version": "0", "version": "2937", "committing_version": "0", "committed_version": "0", "is_rep": true, "dir_auth": "0", "states": [], "is_auth": false, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 3 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1 }, "nref": 2 } }, { "is_auth": true, "auth_first": 2, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 2, "dir": { "path": "/tree", "dirfrag": "0x1000000041a.111*", "snapid_first": 2, "projected_version": "416", "version": "416", "committing_version": "402", "committed_version": "402", "is_rep": false, "dir_auth": "2", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": { "0": 1 } }, "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "child": 1, "frozen": 0, "subtree": 1, "importing": 0, "replicated": 1, "dirty": 1 }, "nref": 4 } }, { "is_auth": true, "auth_first": 2, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 2, "dir": { "path": "/tree", "dirfrag": "0x1000000041a.100*", "snapid_first": 2, "projected_version": "426", "version": "426", "committing_version": "402", "committed_version": "402", "is_rep": false, "dir_auth": "2", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": { "0": 1 } }, "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "child": 1, "frozen": 0, "subtree": 1, "importing": 0, "replicated": 1, "dirty": 1 }, "nref": 4 } }, { "is_auth": true, "auth_first": 2, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 2, "dir": { "path": "/tree", "dirfrag": "0x1000000041a.010*", "snapid_first": 2, "projected_version": "426", "version": "426", "committing_version": "402", "committed_version": "402", "is_rep": false, "dir_auth": "2", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": { "0": 1 } }, "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "child": 1, "frozen": 0, "subtree": 1, "importing": 0, "replicated": 1, "dirty": 1 }, "nref": 4 } } ] 2024-05-26T22:54:25.188 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json get subtrees 2024-05-26T22:54:25.402 INFO:teuthology.orchestra.run.smithi187.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"2937","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":2},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree","dirfrag":"0x1000000041a.110*","snapid_first":2,"projected_version":"442","version":"442","committing_version":"402","committed_version":"402","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree","dirfrag":"0x1000000041a.101*","snapid_first":2,"projected_version":"422","version":"422","committing_version":"402","committed_version":"402","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2024-05-26T22:54:25.403 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 2 }, "snapid_first": 2, "states": [], "version": "2937" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "402", "committing_version": "402", "dir_auth": "1", "dirfrag": "0x1000000041a.110*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/tree", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "442", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "442" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "402", "committing_version": "402", "dir_auth": "1", "dirfrag": "0x1000000041a.101*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/tree", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "422", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "422" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-05-26T22:54:25.403 DEBUG:tasks.cephfs.cephfs_test_case:[ { "is_auth": true, "auth_first": 1, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "~mds1", "dirfrag": "0x101", "snapid_first": 2, "projected_version": "1", "version": "1", "committing_version": "1", "committed_version": "1", "is_rep": false, "dir_auth": "1", "states": [ "auth", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "dirty": 0, "authpin": 0 }, "nref": 2 } }, { "is_auth": false, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "", "dirfrag": "0x1", "snapid_first": 2, "projected_version": "0", "version": "2937", "committing_version": "0", "committed_version": "0", "is_rep": true, "dir_auth": "0", "states": [], "is_auth": false, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 2 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1 }, "nref": 2 } }, { "is_auth": true, "auth_first": 1, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 1, "dir": { "path": "/tree", "dirfrag": "0x1000000041a.110*", "snapid_first": 2, "projected_version": "442", "version": "442", "committing_version": "402", "committed_version": "402", "is_rep": false, "dir_auth": "1", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": { "0": 1 } }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "child": 1, "frozen": 0, "subtree": 1, "importing": 0, "replicated": 1, "dirty": 1 }, "nref": 4 } }, { "is_auth": true, "auth_first": 1, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 1, "dir": { "path": "/tree", "dirfrag": "0x1000000041a.101*", "snapid_first": 2, "projected_version": "422", "version": "422", "committing_version": "402", "committed_version": "402", "is_rep": false, "dir_auth": "1", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": { "0": 1 } }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "child": 1, "frozen": 0, "subtree": 1, "importing": 0, "replicated": 1, "dirty": 1 }, "nref": 4 } } ] 2024-05-26T22:54:25.404 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-05-26T22:54:25.625 INFO:teuthology.orchestra.run.smithi131.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"2937","version":"2937","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":2,"2":3}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1686","version":"1686","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/tree","dirfrag":"0x1000000041a.000*","snapid_first":2,"projected_version":"426","version":"426","committing_version":"402","committed_version":"402","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"frozen":0,"subtree":1,"dirty":1,"waiter":0,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/tree","dirfrag":"0x1000000041a.001*","snapid_first":2,"projected_version":"428","version":"428","committing_version":"402","committed_version":"402","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"frozen":0,"subtree":1,"dirty":1,"waiter":0,"authpin":0},"nref":3}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/tree","dirfrag":"0x1000000041a.010*","snapid_first":2,"projected_version":"426","version":"426","committing_version":"402","committed_version":"402","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/tree","dirfrag":"0x1000000041a.011*","snapid_first":2,"projected_version":"430","version":"430","committing_version":"402","committed_version":"402","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"frozen":0,"subtree":1,"dirty":1,"waiter":0,"authpin":0},"nref":3}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/tree","dirfrag":"0x1000000041a.100*","snapid_first":2,"projected_version":"426","version":"426","committing_version":"402","committed_version":"402","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree","dirfrag":"0x1000000041a.101*","snapid_first":2,"projected_version":"422","version":"422","committing_version":"402","committed_version":"402","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree","dirfrag":"0x1000000041a.110*","snapid_first":2,"projected_version":"442","version":"442","committing_version":"402","committed_version":"402","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/tree","dirfrag":"0x1000000041a.111*","snapid_first":2,"projected_version":"416","version":"416","committing_version":"402","committed_version":"402","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}}] 2024-05-26T22:54:25.627 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 2, "2": 3 } }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "2937", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "2937" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 1, "subtree": 1 }, "projected_version": "1686", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "1686" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "402", "committing_version": "402", "dir_auth": "0", "dirfrag": "0x1000000041a.000*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "/tree", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "subtree": 1, "waiter": 0 }, "projected_version": "426", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "426" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "402", "committing_version": "402", "dir_auth": "0", "dirfrag": "0x1000000041a.001*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "/tree", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "subtree": 1, "waiter": 0 }, "projected_version": "428", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "428" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "402", "committing_version": "402", "dir_auth": "2", "dirfrag": "0x1000000041a.010*", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "426", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "426" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 2, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "402", "committing_version": "402", "dir_auth": "0", "dirfrag": "0x1000000041a.011*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "/tree", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "subtree": 1, "waiter": 0 }, "projected_version": "430", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "430" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "402", "committing_version": "402", "dir_auth": "2", "dirfrag": "0x1000000041a.100*", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "426", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "426" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 2, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "402", "committing_version": "402", "dir_auth": "1", "dirfrag": "0x1000000041a.101*", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "422", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "422" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "402", "committing_version": "402", "dir_auth": "1", "dirfrag": "0x1000000041a.110*", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "442", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "442" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "402", "committing_version": "402", "dir_auth": "2", "dirfrag": "0x1000000041a.111*", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "416", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "416" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 2, "is_auth": false, "random_ephemeral_pin": false } ] 2024-05-26T22:54:25.627 DEBUG:tasks.cephfs.cephfs_test_case:[ { "is_auth": true, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "", "dirfrag": "0x1", "snapid_first": 2, "projected_version": "2937", "version": "2937", "committing_version": "1", "committed_version": "1", "is_rep": true, "dir_auth": "0", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": { "1": 2, "2": 3 } }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "replicated": 1, "dirty": 1, "authpin": 0 }, "nref": 4 } }, { "is_auth": true, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "~mds0", "dirfrag": "0x100", "snapid_first": 2, "projected_version": "1686", "version": "1686", "committing_version": "1", "committed_version": "1", "is_rep": false, "dir_auth": "0", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "dirty": 1, "authpin": 0 }, "nref": 3 } }, { "is_auth": true, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 0, "dir": { "path": "/tree", "dirfrag": "0x1000000041a.000*", "snapid_first": 2, "projected_version": "426", "version": "426", "committing_version": "402", "committed_version": "402", "is_rep": false, "dir_auth": "0", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "frozen": 0, "subtree": 1, "dirty": 1, "waiter": 0, "authpin": 0 }, "nref": 3 } }, { "is_auth": true, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 0, "dir": { "path": "/tree", "dirfrag": "0x1000000041a.001*", "snapid_first": 2, "projected_version": "428", "version": "428", "committing_version": "402", "committed_version": "402", "is_rep": false, "dir_auth": "0", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "frozen": 0, "subtree": 1, "dirty": 1, "waiter": 0, "authpin": 0 }, "nref": 3 } }, { "is_auth": false, "auth_first": 2, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 2, "dir": { "path": "/tree", "dirfrag": "0x1000000041a.010*", "snapid_first": 2, "projected_version": "426", "version": "426", "committing_version": "402", "committed_version": "402", "is_rep": false, "dir_auth": "2", "states": [], "is_auth": false, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "request": 0, "child": 0, "frozen": 0, "subtree": 1, "replicated": 0, "dirty": 0, "waiter": 0, "authpin": 0, "tempexporting": 0 }, "nref": 1 } }, { "is_auth": true, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 0, "dir": { "path": "/tree", "dirfrag": "0x1000000041a.011*", "snapid_first": 2, "projected_version": "430", "version": "430", "committing_version": "402", "committed_version": "402", "is_rep": false, "dir_auth": "0", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "frozen": 0, "subtree": 1, "dirty": 1, "waiter": 0, "authpin": 0 }, "nref": 3 } }, { "is_auth": false, "auth_first": 2, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 2, "dir": { "path": "/tree", "dirfrag": "0x1000000041a.100*", "snapid_first": 2, "projected_version": "426", "version": "426", "committing_version": "402", "committed_version": "402", "is_rep": false, "dir_auth": "2", "states": [], "is_auth": false, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "request": 0, "child": 0, "frozen": 0, "subtree": 1, "replicated": 0, "dirty": 0, "waiter": 0, "authpin": 0, "tempexporting": 0 }, "nref": 1 } }, { "is_auth": false, "auth_first": 1, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 1, "dir": { "path": "/tree", "dirfrag": "0x1000000041a.101*", "snapid_first": 2, "projected_version": "422", "version": "422", "committing_version": "402", "committed_version": "402", "is_rep": false, "dir_auth": "1", "states": [], "is_auth": false, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "request": 0, "child": 0, "frozen": 0, "subtree": 1, "replicated": 0, "dirty": 0, "waiter": 0, "authpin": 0, "tempexporting": 0 }, "nref": 1 } }, { "is_auth": false, "auth_first": 1, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 1, "dir": { "path": "/tree", "dirfrag": "0x1000000041a.110*", "snapid_first": 2, "projected_version": "442", "version": "442", "committing_version": "402", "committed_version": "402", "is_rep": false, "dir_auth": "1", "states": [], "is_auth": false, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "request": 0, "child": 0, "frozen": 0, "subtree": 1, "replicated": 0, "dirty": 0, "waiter": 0, "authpin": 0, "tempexporting": 0 }, "nref": 1 } }, { "is_auth": false, "auth_first": 2, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 2, "dir": { "path": "/tree", "dirfrag": "0x1000000041a.111*", "snapid_first": 2, "projected_version": "416", "version": "416", "committing_version": "402", "committed_version": "402", "is_rep": false, "dir_auth": "2", "states": [], "is_auth": false, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "request": 0, "child": 0, "frozen": 0, "subtree": 1, "replicated": 0, "dirty": 0, "waiter": 0, "authpin": 0, "tempexporting": 0 }, "nref": 1 } } ] 2024-05-26T22:54:25.628 INFO:tasks.cephfs.cephfs_test_case:len=8 [ { "is_auth": true, "auth_first": 2, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 2, "dir": { "path": "/tree", "dirfrag": "0x1000000041a.111*", "snapid_first": 2, "projected_version": "416", "version": "416", "committing_version": "402", "committed_version": "402", "is_rep": false, "dir_auth": "2", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": { "0": 1 } }, "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "child": 1, "frozen": 0, "subtree": 1, "importing": 0, "replicated": 1, "dirty": 1 }, "nref": 4 } }, { "is_auth": true, "auth_first": 2, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 2, "dir": { "path": "/tree", "dirfrag": "0x1000000041a.100*", "snapid_first": 2, "projected_version": "426", "version": "426", "committing_version": "402", "committed_version": "402", "is_rep": false, "dir_auth": "2", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": { "0": 1 } }, "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "child": 1, "frozen": 0, "subtree": 1, "importing": 0, "replicated": 1, "dirty": 1 }, "nref": 4 } }, { "is_auth": true, "auth_first": 2, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 2, "dir": { "path": "/tree", "dirfrag": "0x1000000041a.010*", "snapid_first": 2, "projected_version": "426", "version": "426", "committing_version": "402", "committed_version": "402", "is_rep": false, "dir_auth": "2", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": { "0": 1 } }, "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "child": 1, "frozen": 0, "subtree": 1, "importing": 0, "replicated": 1, "dirty": 1 }, "nref": 4 } }, { "is_auth": true, "auth_first": 1, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 1, "dir": { "path": "/tree", "dirfrag": "0x1000000041a.110*", "snapid_first": 2, "projected_version": "442", "version": "442", "committing_version": "402", "committed_version": "402", "is_rep": false, "dir_auth": "1", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": { "0": 1 } }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "child": 1, "frozen": 0, "subtree": 1, "importing": 0, "replicated": 1, "dirty": 1 }, "nref": 4 } }, { "is_auth": true, "auth_first": 1, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 1, "dir": { "path": "/tree", "dirfrag": "0x1000000041a.101*", "snapid_first": 2, "projected_version": "422", "version": "422", "committing_version": "402", "committed_version": "402", "is_rep": false, "dir_auth": "1", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": { "0": 1 } }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "child": 1, "frozen": 0, "subtree": 1, "importing": 0, "replicated": 1, "dirty": 1 }, "nref": 4 } }, { "is_auth": true, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 0, "dir": { "path": "/tree", "dirfrag": "0x1000000041a.000*", "snapid_first": 2, "projected_version": "426", "version": "426", "committing_version": "402", "committed_version": "402", "is_rep": false, "dir_auth": "0", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "frozen": 0, "subtree": 1, "dirty": 1, "waiter": 0, "authpin": 0 }, "nref": 3 } }, { "is_auth": true, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 0, "dir": { "path": "/tree", "dirfrag": "0x1000000041a.001*", "snapid_first": 2, "projected_version": "428", "version": "428", "committing_version": "402", "committed_version": "402", "is_rep": false, "dir_auth": "0", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "frozen": 0, "subtree": 1, "dirty": 1, "waiter": 0, "authpin": 0 }, "nref": 3 } }, { "is_auth": true, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 0, "dir": { "path": "/tree", "dirfrag": "0x1000000041a.011*", "snapid_first": 2, "projected_version": "430", "version": "430", "committing_version": "402", "committed_version": "402", "is_rep": false, "dir_auth": "0", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "frozen": 0, "subtree": 1, "dirty": 1, "waiter": 0, "authpin": 0 }, "nref": 3 } } ] 2024-05-26T22:54:25.628 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:54:25.628 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 1 tree/0) 2024-05-26T22:54:25.682 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:54:25.683 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 1 tree/1) 2024-05-26T22:54:25.738 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:54:25.739 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 1 tree/2) 2024-05-26T22:54:25.792 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:54:25.793 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 1 tree/3) 2024-05-26T22:54:25.848 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:54:25.848 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 1 tree/4) 2024-05-26T22:54:25.903 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:54:25.903 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 1 tree/5) 2024-05-26T22:54:25.957 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:54:25.957 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 1 tree/6) 2024-05-26T22:54:26.009 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:54:26.009 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 1 tree/7) 2024-05-26T22:54:26.065 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:54:26.065 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 1 tree/8) 2024-05-26T22:54:26.117 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:54:26.117 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 1 tree/9) 2024-05-26T22:54:26.168 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json get subtrees 2024-05-26T22:54:26.396 INFO:teuthology.orchestra.run.smithi187.stdout:[{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds2","dirfrag":"0x102","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"2","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"2937","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":3},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/tree","dirfrag":"0x1000000041a.111*","snapid_first":2,"projected_version":"418","version":"416","committing_version":"402","committed_version":"402","is_rep":false,"dir_auth":"2","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1,"authpin":1},"nref":5}},{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/tree","dirfrag":"0x1000000041a.100*","snapid_first":2,"projected_version":"428","version":"426","committing_version":"402","committed_version":"402","is_rep":false,"dir_auth":"2","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1,"authpin":1},"nref":5}},{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/tree","dirfrag":"0x1000000041a.010*","snapid_first":2,"projected_version":"426","version":"426","committing_version":"402","committed_version":"402","is_rep":false,"dir_auth":"2","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2024-05-26T22:54:26.397 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "2", "dirfrag": "0x102", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds2", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 3 }, "snapid_first": 2, "states": [], "version": "2937" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "402", "committing_version": "402", "dir_auth": "2", "dirfrag": "0x1000000041a.111*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 5, "path": "/tree", "pins": { "authpin": 1, "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "418", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "416" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 2, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "402", "committing_version": "402", "dir_auth": "2", "dirfrag": "0x1000000041a.100*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 5, "path": "/tree", "pins": { "authpin": 1, "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "428", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "426" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 2, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "402", "committing_version": "402", "dir_auth": "2", "dirfrag": "0x1000000041a.010*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/tree", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "426", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "426" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 2, "is_auth": true, "random_ephemeral_pin": false } ] 2024-05-26T22:54:26.397 DEBUG:tasks.cephfs.cephfs_test_case:[ { "is_auth": true, "auth_first": 2, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "~mds2", "dirfrag": "0x102", "snapid_first": 2, "projected_version": "1", "version": "1", "committing_version": "1", "committed_version": "1", "is_rep": false, "dir_auth": "2", "states": [ "auth", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "dirty": 0, "authpin": 0 }, "nref": 2 } }, { "is_auth": false, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "", "dirfrag": "0x1", "snapid_first": 2, "projected_version": "0", "version": "2937", "committing_version": "0", "committed_version": "0", "is_rep": true, "dir_auth": "0", "states": [], "is_auth": false, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 3 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1 }, "nref": 2 } }, { "is_auth": true, "auth_first": 2, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 2, "dir": { "path": "/tree", "dirfrag": "0x1000000041a.111*", "snapid_first": 2, "projected_version": "418", "version": "416", "committing_version": "402", "committed_version": "402", "is_rep": false, "dir_auth": "2", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": { "0": 1 } }, "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "auth_pins": 1, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "child": 1, "frozen": 0, "subtree": 1, "importing": 0, "replicated": 1, "dirty": 1, "authpin": 1 }, "nref": 5 } }, { "is_auth": true, "auth_first": 2, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 2, "dir": { "path": "/tree", "dirfrag": "0x1000000041a.100*", "snapid_first": 2, "projected_version": "428", "version": "426", "committing_version": "402", "committed_version": "402", "is_rep": false, "dir_auth": "2", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": { "0": 1 } }, "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "auth_pins": 1, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "child": 1, "frozen": 0, "subtree": 1, "importing": 0, "replicated": 1, "dirty": 1, "authpin": 1 }, "nref": 5 } }, { "is_auth": true, "auth_first": 2, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 2, "dir": { "path": "/tree", "dirfrag": "0x1000000041a.010*", "snapid_first": 2, "projected_version": "426", "version": "426", "committing_version": "402", "committed_version": "402", "is_rep": false, "dir_auth": "2", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": { "0": 1 } }, "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "child": 1, "frozen": 0, "subtree": 1, "importing": 0, "replicated": 1, "dirty": 1 }, "nref": 4 } } ] 2024-05-26T22:54:26.398 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json get subtrees 2024-05-26T22:54:26.620 INFO:teuthology.orchestra.run.smithi187.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"2937","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":2},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree","dirfrag":"0x1000000041a.110*","snapid_first":2,"projected_version":"446","version":"442","committing_version":"402","committed_version":"402","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":2,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1,"authpin":1},"nref":5}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree","dirfrag":"0x1000000041a.101*","snapid_first":2,"projected_version":"422","version":"422","committing_version":"402","committed_version":"402","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2024-05-26T22:54:26.621 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 2 }, "snapid_first": 2, "states": [], "version": "2937" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 2, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "402", "committing_version": "402", "dir_auth": "1", "dirfrag": "0x1000000041a.110*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 5, "path": "/tree", "pins": { "authpin": 1, "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "446", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "442" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "402", "committing_version": "402", "dir_auth": "1", "dirfrag": "0x1000000041a.101*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/tree", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "422", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "422" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-05-26T22:54:26.621 DEBUG:tasks.cephfs.cephfs_test_case:[ { "is_auth": true, "auth_first": 1, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "~mds1", "dirfrag": "0x101", "snapid_first": 2, "projected_version": "1", "version": "1", "committing_version": "1", "committed_version": "1", "is_rep": false, "dir_auth": "1", "states": [ "auth", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "dirty": 0, "authpin": 0 }, "nref": 2 } }, { "is_auth": false, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "", "dirfrag": "0x1", "snapid_first": 2, "projected_version": "0", "version": "2937", "committing_version": "0", "committed_version": "0", "is_rep": true, "dir_auth": "0", "states": [], "is_auth": false, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 2 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1 }, "nref": 2 } }, { "is_auth": true, "auth_first": 1, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 1, "dir": { "path": "/tree", "dirfrag": "0x1000000041a.110*", "snapid_first": 2, "projected_version": "446", "version": "442", "committing_version": "402", "committed_version": "402", "is_rep": false, "dir_auth": "1", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": { "0": 1 } }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "auth_pins": 2, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "child": 1, "frozen": 0, "subtree": 1, "importing": 0, "replicated": 1, "dirty": 1, "authpin": 1 }, "nref": 5 } }, { "is_auth": true, "auth_first": 1, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 1, "dir": { "path": "/tree", "dirfrag": "0x1000000041a.101*", "snapid_first": 2, "projected_version": "422", "version": "422", "committing_version": "402", "committed_version": "402", "is_rep": false, "dir_auth": "1", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": { "0": 1 } }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "child": 1, "frozen": 0, "subtree": 1, "importing": 0, "replicated": 1, "dirty": 1 }, "nref": 4 } } ] 2024-05-26T22:54:26.621 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-05-26T22:54:26.845 INFO:teuthology.orchestra.run.smithi131.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"2949","version":"2937","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":2,"2":3}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":6,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":1},"nref":5}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1686","version":"1686","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/tree","dirfrag":"0x1000000041a.000*","snapid_first":2,"projected_version":"432","version":"426","committing_version":"402","committed_version":"402","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":3,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"frozen":0,"subtree":1,"dirty":1,"waiter":0,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/tree","dirfrag":"0x1000000041a.001*","snapid_first":2,"projected_version":"428","version":"428","committing_version":"402","committed_version":"402","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"frozen":0,"subtree":1,"dirty":1,"waiter":0,"authpin":0},"nref":3}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/tree","dirfrag":"0x1000000041a.010*","snapid_first":2,"projected_version":"426","version":"426","committing_version":"402","committed_version":"402","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/tree","dirfrag":"0x1000000041a.011*","snapid_first":2,"projected_version":"436","version":"430","committing_version":"402","committed_version":"402","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":3,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"frozen":0,"subtree":1,"dirty":1,"waiter":0,"authpin":1},"nref":4}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/tree","dirfrag":"0x1000000041a.100*","snapid_first":2,"projected_version":"426","version":"426","committing_version":"402","committed_version":"402","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree","dirfrag":"0x1000000041a.101*","snapid_first":2,"projected_version":"422","version":"422","committing_version":"402","committed_version":"402","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree","dirfrag":"0x1000000041a.110*","snapid_first":2,"projected_version":"442","version":"442","committing_version":"402","committed_version":"402","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/tree","dirfrag":"0x1000000041a.111*","snapid_first":2,"projected_version":"416","version":"416","committing_version":"402","committed_version":"402","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}}] 2024-05-26T22:54:26.846 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 6, "auth_state": { "replicas": { "1": 2, "2": 3 } }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 5, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "2949", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "2937" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 1, "subtree": 1 }, "projected_version": "1686", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "1686" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 3, "auth_state": { "replicas": {} }, "committed_version": "402", "committing_version": "402", "dir_auth": "0", "dirfrag": "0x1000000041a.000*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/tree", "pins": { "authpin": 1, "child": 1, "dirty": 1, "frozen": 0, "subtree": 1, "waiter": 0 }, "projected_version": "432", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "426" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "402", "committing_version": "402", "dir_auth": "0", "dirfrag": "0x1000000041a.001*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "/tree", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "subtree": 1, "waiter": 0 }, "projected_version": "428", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "428" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "402", "committing_version": "402", "dir_auth": "2", "dirfrag": "0x1000000041a.010*", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "426", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "426" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 2, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 3, "auth_state": { "replicas": {} }, "committed_version": "402", "committing_version": "402", "dir_auth": "0", "dirfrag": "0x1000000041a.011*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/tree", "pins": { "authpin": 1, "child": 1, "dirty": 1, "frozen": 0, "subtree": 1, "waiter": 0 }, "projected_version": "436", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "430" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "402", "committing_version": "402", "dir_auth": "2", "dirfrag": "0x1000000041a.100*", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "426", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "426" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 2, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "402", "committing_version": "402", "dir_auth": "1", "dirfrag": "0x1000000041a.101*", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "422", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "422" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "402", "committing_version": "402", "dir_auth": "1", "dirfrag": "0x1000000041a.110*", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "442", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "442" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "402", "committing_version": "402", "dir_auth": "2", "dirfrag": "0x1000000041a.111*", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "416", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "416" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 2, "is_auth": false, "random_ephemeral_pin": false } ] 2024-05-26T22:54:26.847 DEBUG:tasks.cephfs.cephfs_test_case:[ { "is_auth": true, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "", "dirfrag": "0x1", "snapid_first": 2, "projected_version": "2949", "version": "2937", "committing_version": "1", "committed_version": "1", "is_rep": true, "dir_auth": "0", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": { "1": 2, "2": 3 } }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 6, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "replicated": 1, "dirty": 1, "authpin": 1 }, "nref": 5 } }, { "is_auth": true, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "~mds0", "dirfrag": "0x100", "snapid_first": 2, "projected_version": "1686", "version": "1686", "committing_version": "1", "committed_version": "1", "is_rep": false, "dir_auth": "0", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "dirty": 1, "authpin": 0 }, "nref": 3 } }, { "is_auth": true, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 0, "dir": { "path": "/tree", "dirfrag": "0x1000000041a.000*", "snapid_first": 2, "projected_version": "432", "version": "426", "committing_version": "402", "committed_version": "402", "is_rep": false, "dir_auth": "0", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 3, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "frozen": 0, "subtree": 1, "dirty": 1, "waiter": 0, "authpin": 1 }, "nref": 4 } }, { "is_auth": true, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 0, "dir": { "path": "/tree", "dirfrag": "0x1000000041a.001*", "snapid_first": 2, "projected_version": "428", "version": "428", "committing_version": "402", "committed_version": "402", "is_rep": false, "dir_auth": "0", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "frozen": 0, "subtree": 1, "dirty": 1, "waiter": 0, "authpin": 0 }, "nref": 3 } }, { "is_auth": false, "auth_first": 2, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 2, "dir": { "path": "/tree", "dirfrag": "0x1000000041a.010*", "snapid_first": 2, "projected_version": "426", "version": "426", "committing_version": "402", "committed_version": "402", "is_rep": false, "dir_auth": "2", "states": [], "is_auth": false, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "request": 0, "child": 0, "frozen": 0, "subtree": 1, "replicated": 0, "dirty": 0, "waiter": 0, "authpin": 0, "tempexporting": 0 }, "nref": 1 } }, { "is_auth": true, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 0, "dir": { "path": "/tree", "dirfrag": "0x1000000041a.011*", "snapid_first": 2, "projected_version": "436", "version": "430", "committing_version": "402", "committed_version": "402", "is_rep": false, "dir_auth": "0", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 3, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "frozen": 0, "subtree": 1, "dirty": 1, "waiter": 0, "authpin": 1 }, "nref": 4 } }, { "is_auth": false, "auth_first": 2, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 2, "dir": { "path": "/tree", "dirfrag": "0x1000000041a.100*", "snapid_first": 2, "projected_version": "426", "version": "426", "committing_version": "402", "committed_version": "402", "is_rep": false, "dir_auth": "2", "states": [], "is_auth": false, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "request": 0, "child": 0, "frozen": 0, "subtree": 1, "replicated": 0, "dirty": 0, "waiter": 0, "authpin": 0, "tempexporting": 0 }, "nref": 1 } }, { "is_auth": false, "auth_first": 1, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 1, "dir": { "path": "/tree", "dirfrag": "0x1000000041a.101*", "snapid_first": 2, "projected_version": "422", "version": "422", "committing_version": "402", "committed_version": "402", "is_rep": false, "dir_auth": "1", "states": [], "is_auth": false, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "request": 0, "child": 0, "frozen": 0, "subtree": 1, "replicated": 0, "dirty": 0, "waiter": 0, "authpin": 0, "tempexporting": 0 }, "nref": 1 } }, { "is_auth": false, "auth_first": 1, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 1, "dir": { "path": "/tree", "dirfrag": "0x1000000041a.110*", "snapid_first": 2, "projected_version": "442", "version": "442", "committing_version": "402", "committed_version": "402", "is_rep": false, "dir_auth": "1", "states": [], "is_auth": false, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "request": 0, "child": 0, "frozen": 0, "subtree": 1, "replicated": 0, "dirty": 0, "waiter": 0, "authpin": 0, "tempexporting": 0 }, "nref": 1 } }, { "is_auth": false, "auth_first": 2, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 2, "dir": { "path": "/tree", "dirfrag": "0x1000000041a.111*", "snapid_first": 2, "projected_version": "416", "version": "416", "committing_version": "402", "committed_version": "402", "is_rep": false, "dir_auth": "2", "states": [], "is_auth": false, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "request": 0, "child": 0, "frozen": 0, "subtree": 1, "replicated": 0, "dirty": 0, "waiter": 0, "authpin": 0, "tempexporting": 0 }, "nref": 1 } } ] 2024-05-26T22:54:26.847 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/tree/0', 1), ('/tree/1', 1), ('/tree/2', 1), ('/tree/3', 1), ('/tree/4', 1), ('/tree/5', 1), ('/tree/6', 1), ('/tree/7', 1), ('/tree/8', 1), ('/tree/9', 1)] 2024-05-26T22:54:28.848 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json get subtrees 2024-05-26T22:54:29.076 INFO:teuthology.orchestra.run.smithi187.stdout:[{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds2","dirfrag":"0x102","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"2","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"2937","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":3},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/tree","dirfrag":"0x1000000041a.111*","snapid_first":2,"projected_version":"419","version":"419","committing_version":"402","committed_version":"402","is_rep":false,"dir_auth":"2","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1,"1":1}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/tree","dirfrag":"0x1000000041a.100*","snapid_first":2,"projected_version":"429","version":"429","committing_version":"402","committed_version":"402","is_rep":false,"dir_auth":"2","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1,"1":1}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/tree","dirfrag":"0x1000000041a.010*","snapid_first":2,"projected_version":"427","version":"427","committing_version":"402","committed_version":"402","is_rep":false,"dir_auth":"2","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree/0","dirfrag":"0x1000000041b","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree/6","dirfrag":"0x10000000427","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}}] 2024-05-26T22:54:29.077 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "2", "dirfrag": "0x102", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds2", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 3 }, "snapid_first": 2, "states": [], "version": "2937" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1, "1": 1 } }, "committed_version": "402", "committing_version": "402", "dir_auth": "2", "dirfrag": "0x1000000041a.111*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/tree", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "419", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "419" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 2, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1, "1": 1 } }, "committed_version": "402", "committing_version": "402", "dir_auth": "2", "dirfrag": "0x1000000041a.100*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/tree", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "429", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "429" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 2, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "402", "committing_version": "402", "dir_auth": "2", "dirfrag": "0x1000000041a.010*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/tree", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "427", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "427" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 2, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x1000000041b", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/0", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000427", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/6", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false } ] 2024-05-26T22:54:29.078 DEBUG:tasks.cephfs.cephfs_test_case:[ { "is_auth": true, "auth_first": 2, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "~mds2", "dirfrag": "0x102", "snapid_first": 2, "projected_version": "1", "version": "1", "committing_version": "1", "committed_version": "1", "is_rep": false, "dir_auth": "2", "states": [ "auth", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "dirty": 0, "authpin": 0 }, "nref": 2 } }, { "is_auth": false, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "", "dirfrag": "0x1", "snapid_first": 2, "projected_version": "0", "version": "2937", "committing_version": "0", "committed_version": "0", "is_rep": true, "dir_auth": "0", "states": [], "is_auth": false, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 3 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1 }, "nref": 2 } }, { "is_auth": true, "auth_first": 2, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 2, "dir": { "path": "/tree", "dirfrag": "0x1000000041a.111*", "snapid_first": 2, "projected_version": "419", "version": "419", "committing_version": "402", "committed_version": "402", "is_rep": false, "dir_auth": "2", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": { "0": 1, "1": 1 } }, "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "child": 1, "frozen": 0, "subtree": 1, "importing": 0, "replicated": 1, "dirty": 1, "authpin": 0 }, "nref": 4 } }, { "is_auth": true, "auth_first": 2, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 2, "dir": { "path": "/tree", "dirfrag": "0x1000000041a.100*", "snapid_first": 2, "projected_version": "429", "version": "429", "committing_version": "402", "committed_version": "402", "is_rep": false, "dir_auth": "2", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": { "0": 1, "1": 1 } }, "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "child": 1, "frozen": 0, "subtree": 1, "importing": 0, "replicated": 1, "dirty": 1, "authpin": 0 }, "nref": 4 } }, { "is_auth": true, "auth_first": 2, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 2, "dir": { "path": "/tree", "dirfrag": "0x1000000041a.010*", "snapid_first": 2, "projected_version": "427", "version": "427", "committing_version": "402", "committed_version": "402", "is_rep": false, "dir_auth": "2", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": { "0": 1 } }, "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "child": 1, "frozen": 0, "subtree": 1, "importing": 0, "replicated": 1, "dirty": 1, "authpin": 0 }, "nref": 4 } }, { "is_auth": false, "auth_first": 1, "auth_second": -2, "export_pin": 1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": 1, "dir": { "path": "/tree/0", "dirfrag": "0x1000000041b", "snapid_first": 2, "projected_version": "5", "version": "5", "committing_version": "0", "committed_version": "0", "is_rep": false, "dir_auth": "1", "states": [], "is_auth": false, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "request": 0, "child": 0, "frozen": 0, "subtree": 1, "replicated": 0, "dirty": 0, "waiter": 0, "authpin": 0, "tempexporting": 0 }, "nref": 1 } }, { "is_auth": false, "auth_first": 1, "auth_second": -2, "export_pin": 1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": 1, "dir": { "path": "/tree/6", "dirfrag": "0x10000000427", "snapid_first": 2, "projected_version": "5", "version": "5", "committing_version": "0", "committed_version": "0", "is_rep": false, "dir_auth": "1", "states": [], "is_auth": false, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "request": 0, "child": 0, "frozen": 0, "subtree": 1, "replicated": 0, "dirty": 0, "waiter": 0, "authpin": 0, "tempexporting": 0 }, "nref": 1 } } ] 2024-05-26T22:54:29.078 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json get subtrees 2024-05-26T22:54:29.289 INFO:teuthology.orchestra.run.smithi187.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"2937","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":2},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree","dirfrag":"0x1000000041a.110*","snapid_first":2,"projected_version":"447","version":"447","committing_version":"402","committed_version":"402","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree","dirfrag":"0x1000000041a.101*","snapid_first":2,"projected_version":"423","version":"423","committing_version":"402","committed_version":"402","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree/5","dirfrag":"0x10000000425","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1},"nref":3}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree/7","dirfrag":"0x10000000429","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1},"nref":3}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/tree","dirfrag":"0x1000000041a.100*","snapid_first":2,"projected_version":"0","version":"428","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"dnwaiter":0,"child":1,"subtree":1},"nref":2}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/tree","dirfrag":"0x1000000041a.111*","snapid_first":2,"projected_version":"0","version":"418","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"dnwaiter":0,"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree/0","dirfrag":"0x1000000041b","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"2":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree/6","dirfrag":"0x10000000427","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"2":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2024-05-26T22:54:29.290 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 2 }, "snapid_first": 2, "states": [], "version": "2937" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "402", "committing_version": "402", "dir_auth": "1", "dirfrag": "0x1000000041a.110*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/tree", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "447", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "447" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "402", "committing_version": "402", "dir_auth": "1", "dirfrag": "0x1000000041a.101*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/tree", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "423", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "423" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000425", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "/tree/5", "pins": { "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000429", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "/tree/7", "pins": { "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x1000000041a.100*", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/tree", "pins": { "child": 1, "dnwaiter": 0, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "428" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 2, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x1000000041a.111*", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/tree", "pins": { "child": 1, "dnwaiter": 0, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "418" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 2, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "2": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x1000000041b", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/tree/0", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "2": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000427", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/tree/6", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-05-26T22:54:29.290 DEBUG:tasks.cephfs.cephfs_test_case:[ { "is_auth": true, "auth_first": 1, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "~mds1", "dirfrag": "0x101", "snapid_first": 2, "projected_version": "1", "version": "1", "committing_version": "1", "committed_version": "1", "is_rep": false, "dir_auth": "1", "states": [ "auth", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "dirty": 0, "authpin": 0 }, "nref": 2 } }, { "is_auth": false, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "", "dirfrag": "0x1", "snapid_first": 2, "projected_version": "0", "version": "2937", "committing_version": "0", "committed_version": "0", "is_rep": true, "dir_auth": "0", "states": [], "is_auth": false, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 2 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1 }, "nref": 2 } }, { "is_auth": true, "auth_first": 1, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 1, "dir": { "path": "/tree", "dirfrag": "0x1000000041a.110*", "snapid_first": 2, "projected_version": "447", "version": "447", "committing_version": "402", "committed_version": "402", "is_rep": false, "dir_auth": "1", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": { "0": 1 } }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "child": 1, "frozen": 0, "subtree": 1, "importing": 0, "replicated": 1, "dirty": 1, "authpin": 0 }, "nref": 4 } }, { "is_auth": true, "auth_first": 1, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 1, "dir": { "path": "/tree", "dirfrag": "0x1000000041a.101*", "snapid_first": 2, "projected_version": "423", "version": "423", "committing_version": "402", "committed_version": "402", "is_rep": false, "dir_auth": "1", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": { "0": 1 } }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "child": 1, "frozen": 0, "subtree": 1, "importing": 0, "replicated": 1, "dirty": 1, "authpin": 0 }, "nref": 4 } }, { "is_auth": true, "auth_first": 1, "auth_second": -2, "export_pin": 1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": 1, "dir": { "path": "/tree/5", "dirfrag": "0x10000000425", "snapid_first": 2, "projected_version": "5", "version": "5", "committing_version": "0", "committed_version": "0", "is_rep": false, "dir_auth": "1", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "replicated": 0, "dirty": 1 }, "nref": 3 } }, { "is_auth": true, "auth_first": 1, "auth_second": -2, "export_pin": 1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": 1, "dir": { "path": "/tree/7", "dirfrag": "0x10000000429", "snapid_first": 2, "projected_version": "5", "version": "5", "committing_version": "0", "committed_version": "0", "is_rep": false, "dir_auth": "1", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "replicated": 0, "dirty": 1 }, "nref": 3 } }, { "is_auth": false, "auth_first": 2, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 2, "dir": { "path": "/tree", "dirfrag": "0x1000000041a.100*", "snapid_first": 2, "projected_version": "0", "version": "428", "committing_version": "0", "committed_version": "0", "is_rep": false, "dir_auth": "2", "states": [], "is_auth": false, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "dnwaiter": 0, "child": 1, "subtree": 1 }, "nref": 2 } }, { "is_auth": false, "auth_first": 2, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 2, "dir": { "path": "/tree", "dirfrag": "0x1000000041a.111*", "snapid_first": 2, "projected_version": "0", "version": "418", "committing_version": "0", "committed_version": "0", "is_rep": false, "dir_auth": "2", "states": [], "is_auth": false, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "dnwaiter": 0, "child": 1, "subtree": 1 }, "nref": 2 } }, { "is_auth": true, "auth_first": 1, "auth_second": -2, "export_pin": 1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": 1, "dir": { "path": "/tree/0", "dirfrag": "0x1000000041b", "snapid_first": 2, "projected_version": "5", "version": "5", "committing_version": "0", "committed_version": "0", "is_rep": false, "dir_auth": "1", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": { "2": 1 } }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "child": 1, "frozen": 0, "subtree": 1, "importing": 0, "replicated": 1, "dirty": 1 }, "nref": 4 } }, { "is_auth": true, "auth_first": 1, "auth_second": -2, "export_pin": 1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": 1, "dir": { "path": "/tree/6", "dirfrag": "0x10000000427", "snapid_first": 2, "projected_version": "5", "version": "5", "committing_version": "0", "committed_version": "0", "is_rep": false, "dir_auth": "1", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": { "2": 1 } }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "child": 1, "frozen": 0, "subtree": 1, "importing": 0, "replicated": 1, "dirty": 1 }, "nref": 4 } } ] 2024-05-26T22:54:29.291 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-05-26T22:54:29.509 INFO:teuthology.orchestra.run.smithi131.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"2952","version":"2952","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":2,"2":3}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1686","version":"1686","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/tree","dirfrag":"0x1000000041a.000*","snapid_first":2,"projected_version":"433","version":"433","committing_version":"402","committed_version":"402","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"frozen":0,"subtree":1,"dirty":1,"waiter":0,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/tree","dirfrag":"0x1000000041a.001*","snapid_first":2,"projected_version":"429","version":"429","committing_version":"402","committed_version":"402","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"frozen":0,"subtree":1,"dirty":1,"waiter":0,"authpin":0},"nref":3}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/tree","dirfrag":"0x1000000041a.010*","snapid_first":2,"projected_version":"426","version":"426","committing_version":"402","committed_version":"402","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/tree","dirfrag":"0x1000000041a.011*","snapid_first":2,"projected_version":"437","version":"437","committing_version":"402","committed_version":"402","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"frozen":0,"subtree":1,"dirty":1,"waiter":0,"authpin":0},"nref":3}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/tree","dirfrag":"0x1000000041a.100*","snapid_first":2,"projected_version":"426","version":"426","committing_version":"402","committed_version":"402","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree","dirfrag":"0x1000000041a.101*","snapid_first":2,"projected_version":"422","version":"422","committing_version":"402","committed_version":"402","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree","dirfrag":"0x1000000041a.110*","snapid_first":2,"projected_version":"442","version":"442","committing_version":"402","committed_version":"402","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/tree","dirfrag":"0x1000000041a.111*","snapid_first":2,"projected_version":"416","version":"416","committing_version":"402","committed_version":"402","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}}] 2024-05-26T22:54:29.510 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 2, "2": 3 } }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "2952", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "2952" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 1, "subtree": 1 }, "projected_version": "1686", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "1686" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "402", "committing_version": "402", "dir_auth": "0", "dirfrag": "0x1000000041a.000*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "/tree", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "subtree": 1, "waiter": 0 }, "projected_version": "433", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "433" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "402", "committing_version": "402", "dir_auth": "0", "dirfrag": "0x1000000041a.001*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "/tree", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "subtree": 1, "waiter": 0 }, "projected_version": "429", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "429" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "402", "committing_version": "402", "dir_auth": "2", "dirfrag": "0x1000000041a.010*", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "426", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "426" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 2, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "402", "committing_version": "402", "dir_auth": "0", "dirfrag": "0x1000000041a.011*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "/tree", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "subtree": 1, "waiter": 0 }, "projected_version": "437", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "437" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "402", "committing_version": "402", "dir_auth": "2", "dirfrag": "0x1000000041a.100*", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "426", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "426" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 2, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "402", "committing_version": "402", "dir_auth": "1", "dirfrag": "0x1000000041a.101*", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "422", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "422" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "402", "committing_version": "402", "dir_auth": "1", "dirfrag": "0x1000000041a.110*", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "442", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "442" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "402", "committing_version": "402", "dir_auth": "2", "dirfrag": "0x1000000041a.111*", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "416", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "416" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 2, "is_auth": false, "random_ephemeral_pin": false } ] 2024-05-26T22:54:29.511 DEBUG:tasks.cephfs.cephfs_test_case:[ { "is_auth": true, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "", "dirfrag": "0x1", "snapid_first": 2, "projected_version": "2952", "version": "2952", "committing_version": "1", "committed_version": "1", "is_rep": true, "dir_auth": "0", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": { "1": 2, "2": 3 } }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "replicated": 1, "dirty": 1, "authpin": 0 }, "nref": 4 } }, { "is_auth": true, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "~mds0", "dirfrag": "0x100", "snapid_first": 2, "projected_version": "1686", "version": "1686", "committing_version": "1", "committed_version": "1", "is_rep": false, "dir_auth": "0", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "dirty": 1, "authpin": 0 }, "nref": 3 } }, { "is_auth": true, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 0, "dir": { "path": "/tree", "dirfrag": "0x1000000041a.000*", "snapid_first": 2, "projected_version": "433", "version": "433", "committing_version": "402", "committed_version": "402", "is_rep": false, "dir_auth": "0", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "frozen": 0, "subtree": 1, "dirty": 1, "waiter": 0, "authpin": 0 }, "nref": 3 } }, { "is_auth": true, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 0, "dir": { "path": "/tree", "dirfrag": "0x1000000041a.001*", "snapid_first": 2, "projected_version": "429", "version": "429", "committing_version": "402", "committed_version": "402", "is_rep": false, "dir_auth": "0", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "frozen": 0, "subtree": 1, "dirty": 1, "waiter": 0, "authpin": 0 }, "nref": 3 } }, { "is_auth": false, "auth_first": 2, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 2, "dir": { "path": "/tree", "dirfrag": "0x1000000041a.010*", "snapid_first": 2, "projected_version": "426", "version": "426", "committing_version": "402", "committed_version": "402", "is_rep": false, "dir_auth": "2", "states": [], "is_auth": false, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "request": 0, "child": 0, "frozen": 0, "subtree": 1, "replicated": 0, "dirty": 0, "waiter": 0, "authpin": 0, "tempexporting": 0 }, "nref": 1 } }, { "is_auth": true, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 0, "dir": { "path": "/tree", "dirfrag": "0x1000000041a.011*", "snapid_first": 2, "projected_version": "437", "version": "437", "committing_version": "402", "committed_version": "402", "is_rep": false, "dir_auth": "0", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "frozen": 0, "subtree": 1, "dirty": 1, "waiter": 0, "authpin": 0 }, "nref": 3 } }, { "is_auth": false, "auth_first": 2, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 2, "dir": { "path": "/tree", "dirfrag": "0x1000000041a.100*", "snapid_first": 2, "projected_version": "426", "version": "426", "committing_version": "402", "committed_version": "402", "is_rep": false, "dir_auth": "2", "states": [], "is_auth": false, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "request": 0, "child": 0, "frozen": 0, "subtree": 1, "replicated": 0, "dirty": 0, "waiter": 0, "authpin": 0, "tempexporting": 0 }, "nref": 1 } }, { "is_auth": false, "auth_first": 1, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 1, "dir": { "path": "/tree", "dirfrag": "0x1000000041a.101*", "snapid_first": 2, "projected_version": "422", "version": "422", "committing_version": "402", "committed_version": "402", "is_rep": false, "dir_auth": "1", "states": [], "is_auth": false, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "request": 0, "child": 0, "frozen": 0, "subtree": 1, "replicated": 0, "dirty": 0, "waiter": 0, "authpin": 0, "tempexporting": 0 }, "nref": 1 } }, { "is_auth": false, "auth_first": 1, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 1, "dir": { "path": "/tree", "dirfrag": "0x1000000041a.110*", "snapid_first": 2, "projected_version": "442", "version": "442", "committing_version": "402", "committed_version": "402", "is_rep": false, "dir_auth": "1", "states": [], "is_auth": false, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "request": 0, "child": 0, "frozen": 0, "subtree": 1, "replicated": 0, "dirty": 0, "waiter": 0, "authpin": 0, "tempexporting": 0 }, "nref": 1 } }, { "is_auth": false, "auth_first": 2, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 2, "dir": { "path": "/tree", "dirfrag": "0x1000000041a.111*", "snapid_first": 2, "projected_version": "416", "version": "416", "committing_version": "402", "committed_version": "402", "is_rep": false, "dir_auth": "2", "states": [], "is_auth": false, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "request": 0, "child": 0, "frozen": 0, "subtree": 1, "replicated": 0, "dirty": 0, "waiter": 0, "authpin": 0, "tempexporting": 0 }, "nref": 1 } } ] 2024-05-26T22:54:29.511 INFO:tasks.cephfs.cephfs_test_case:[('/tree/0', 1), ('/tree/5', 1), ('/tree/6', 1), ('/tree/7', 1)] =?= [('/tree/0', 1), ('/tree/1', 1), ('/tree/2', 1), ('/tree/3', 1), ('/tree/4', 1), ('/tree/5', 1), ('/tree/6', 1), ('/tree/7', 1), ('/tree/8', 1), ('/tree/9', 1)] 2024-05-26T22:54:31.512 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json get subtrees 2024-05-26T22:54:31.740 INFO:teuthology.orchestra.run.smithi187.stdout:[{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds2","dirfrag":"0x102","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"2","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"2937","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":3},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/tree","dirfrag":"0x1000000041a.111*","snapid_first":2,"projected_version":"419","version":"419","committing_version":"402","committed_version":"402","is_rep":false,"dir_auth":"2","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1,"1":1}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/tree","dirfrag":"0x1000000041a.100*","snapid_first":2,"projected_version":"429","version":"429","committing_version":"402","committed_version":"402","is_rep":false,"dir_auth":"2","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1,"1":1}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/tree","dirfrag":"0x1000000041a.010*","snapid_first":2,"projected_version":"427","version":"427","committing_version":"402","committed_version":"402","is_rep":false,"dir_auth":"2","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree/0","dirfrag":"0x1000000041b","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree/6","dirfrag":"0x10000000427","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}}] 2024-05-26T22:54:31.741 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "2", "dirfrag": "0x102", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds2", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 3 }, "snapid_first": 2, "states": [], "version": "2937" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1, "1": 1 } }, "committed_version": "402", "committing_version": "402", "dir_auth": "2", "dirfrag": "0x1000000041a.111*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/tree", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "419", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "419" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 2, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1, "1": 1 } }, "committed_version": "402", "committing_version": "402", "dir_auth": "2", "dirfrag": "0x1000000041a.100*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/tree", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "429", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "429" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 2, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "402", "committing_version": "402", "dir_auth": "2", "dirfrag": "0x1000000041a.010*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/tree", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "427", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "427" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 2, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x1000000041b", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/0", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000427", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/6", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false } ] 2024-05-26T22:54:31.741 DEBUG:tasks.cephfs.cephfs_test_case:[ { "is_auth": true, "auth_first": 2, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "~mds2", "dirfrag": "0x102", "snapid_first": 2, "projected_version": "1", "version": "1", "committing_version": "1", "committed_version": "1", "is_rep": false, "dir_auth": "2", "states": [ "auth", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "dirty": 0, "authpin": 0 }, "nref": 2 } }, { "is_auth": false, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "", "dirfrag": "0x1", "snapid_first": 2, "projected_version": "0", "version": "2937", "committing_version": "0", "committed_version": "0", "is_rep": true, "dir_auth": "0", "states": [], "is_auth": false, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 3 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1 }, "nref": 2 } }, { "is_auth": true, "auth_first": 2, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 2, "dir": { "path": "/tree", "dirfrag": "0x1000000041a.111*", "snapid_first": 2, "projected_version": "419", "version": "419", "committing_version": "402", "committed_version": "402", "is_rep": false, "dir_auth": "2", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": { "0": 1, "1": 1 } }, "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "child": 1, "frozen": 0, "subtree": 1, "importing": 0, "replicated": 1, "dirty": 1, "authpin": 0 }, "nref": 4 } }, { "is_auth": true, "auth_first": 2, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 2, "dir": { "path": "/tree", "dirfrag": "0x1000000041a.100*", "snapid_first": 2, "projected_version": "429", "version": "429", "committing_version": "402", "committed_version": "402", "is_rep": false, "dir_auth": "2", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": { "0": 1, "1": 1 } }, "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "child": 1, "frozen": 0, "subtree": 1, "importing": 0, "replicated": 1, "dirty": 1, "authpin": 0 }, "nref": 4 } }, { "is_auth": true, "auth_first": 2, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 2, "dir": { "path": "/tree", "dirfrag": "0x1000000041a.010*", "snapid_first": 2, "projected_version": "427", "version": "427", "committing_version": "402", "committed_version": "402", "is_rep": false, "dir_auth": "2", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": { "0": 1 } }, "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "child": 1, "frozen": 0, "subtree": 1, "importing": 0, "replicated": 1, "dirty": 1, "authpin": 0 }, "nref": 4 } }, { "is_auth": false, "auth_first": 1, "auth_second": -2, "export_pin": 1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": 1, "dir": { "path": "/tree/0", "dirfrag": "0x1000000041b", "snapid_first": 2, "projected_version": "5", "version": "5", "committing_version": "0", "committed_version": "0", "is_rep": false, "dir_auth": "1", "states": [], "is_auth": false, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "request": 0, "child": 0, "frozen": 0, "subtree": 1, "replicated": 0, "dirty": 0, "waiter": 0, "authpin": 0, "tempexporting": 0 }, "nref": 1 } }, { "is_auth": false, "auth_first": 1, "auth_second": -2, "export_pin": 1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": 1, "dir": { "path": "/tree/6", "dirfrag": "0x10000000427", "snapid_first": 2, "projected_version": "5", "version": "5", "committing_version": "0", "committed_version": "0", "is_rep": false, "dir_auth": "1", "states": [], "is_auth": false, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "request": 0, "child": 0, "frozen": 0, "subtree": 1, "replicated": 0, "dirty": 0, "waiter": 0, "authpin": 0, "tempexporting": 0 }, "nref": 1 } } ] 2024-05-26T22:54:31.742 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json get subtrees 2024-05-26T22:54:31.961 INFO:teuthology.orchestra.run.smithi187.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"2937","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":2},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree","dirfrag":"0x1000000041a.110*","snapid_first":2,"projected_version":"447","version":"447","committing_version":"402","committed_version":"402","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree","dirfrag":"0x1000000041a.101*","snapid_first":2,"projected_version":"423","version":"423","committing_version":"402","committed_version":"402","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree/5","dirfrag":"0x10000000425","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1},"nref":3}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree/7","dirfrag":"0x10000000429","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1},"nref":3}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/tree","dirfrag":"0x1000000041a.100*","snapid_first":2,"projected_version":"0","version":"428","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"dnwaiter":0,"child":1,"subtree":1},"nref":2}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/tree","dirfrag":"0x1000000041a.111*","snapid_first":2,"projected_version":"0","version":"418","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"dnwaiter":0,"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree/0","dirfrag":"0x1000000041b","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"2":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree/6","dirfrag":"0x10000000427","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"2":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2024-05-26T22:54:31.962 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 2 }, "snapid_first": 2, "states": [], "version": "2937" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "402", "committing_version": "402", "dir_auth": "1", "dirfrag": "0x1000000041a.110*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/tree", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "447", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "447" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "402", "committing_version": "402", "dir_auth": "1", "dirfrag": "0x1000000041a.101*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/tree", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "423", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "423" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000425", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "/tree/5", "pins": { "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000429", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "/tree/7", "pins": { "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x1000000041a.100*", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/tree", "pins": { "child": 1, "dnwaiter": 0, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "428" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 2, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x1000000041a.111*", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/tree", "pins": { "child": 1, "dnwaiter": 0, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "418" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 2, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "2": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x1000000041b", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/tree/0", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "2": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000427", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/tree/6", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-05-26T22:54:31.962 DEBUG:tasks.cephfs.cephfs_test_case:[ { "is_auth": true, "auth_first": 1, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "~mds1", "dirfrag": "0x101", "snapid_first": 2, "projected_version": "1", "version": "1", "committing_version": "1", "committed_version": "1", "is_rep": false, "dir_auth": "1", "states": [ "auth", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "dirty": 0, "authpin": 0 }, "nref": 2 } }, { "is_auth": false, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "", "dirfrag": "0x1", "snapid_first": 2, "projected_version": "0", "version": "2937", "committing_version": "0", "committed_version": "0", "is_rep": true, "dir_auth": "0", "states": [], "is_auth": false, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 2 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1 }, "nref": 2 } }, { "is_auth": true, "auth_first": 1, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 1, "dir": { "path": "/tree", "dirfrag": "0x1000000041a.110*", "snapid_first": 2, "projected_version": "447", "version": "447", "committing_version": "402", "committed_version": "402", "is_rep": false, "dir_auth": "1", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": { "0": 1 } }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "child": 1, "frozen": 0, "subtree": 1, "importing": 0, "replicated": 1, "dirty": 1, "authpin": 0 }, "nref": 4 } }, { "is_auth": true, "auth_first": 1, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 1, "dir": { "path": "/tree", "dirfrag": "0x1000000041a.101*", "snapid_first": 2, "projected_version": "423", "version": "423", "committing_version": "402", "committed_version": "402", "is_rep": false, "dir_auth": "1", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": { "0": 1 } }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "child": 1, "frozen": 0, "subtree": 1, "importing": 0, "replicated": 1, "dirty": 1, "authpin": 0 }, "nref": 4 } }, { "is_auth": true, "auth_first": 1, "auth_second": -2, "export_pin": 1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": 1, "dir": { "path": "/tree/5", "dirfrag": "0x10000000425", "snapid_first": 2, "projected_version": "5", "version": "5", "committing_version": "0", "committed_version": "0", "is_rep": false, "dir_auth": "1", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "replicated": 0, "dirty": 1 }, "nref": 3 } }, { "is_auth": true, "auth_first": 1, "auth_second": -2, "export_pin": 1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": 1, "dir": { "path": "/tree/7", "dirfrag": "0x10000000429", "snapid_first": 2, "projected_version": "5", "version": "5", "committing_version": "0", "committed_version": "0", "is_rep": false, "dir_auth": "1", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "replicated": 0, "dirty": 1 }, "nref": 3 } }, { "is_auth": false, "auth_first": 2, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 2, "dir": { "path": "/tree", "dirfrag": "0x1000000041a.100*", "snapid_first": 2, "projected_version": "0", "version": "428", "committing_version": "0", "committed_version": "0", "is_rep": false, "dir_auth": "2", "states": [], "is_auth": false, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "dnwaiter": 0, "child": 1, "subtree": 1 }, "nref": 2 } }, { "is_auth": false, "auth_first": 2, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 2, "dir": { "path": "/tree", "dirfrag": "0x1000000041a.111*", "snapid_first": 2, "projected_version": "0", "version": "418", "committing_version": "0", "committed_version": "0", "is_rep": false, "dir_auth": "2", "states": [], "is_auth": false, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "dnwaiter": 0, "child": 1, "subtree": 1 }, "nref": 2 } }, { "is_auth": true, "auth_first": 1, "auth_second": -2, "export_pin": 1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": 1, "dir": { "path": "/tree/0", "dirfrag": "0x1000000041b", "snapid_first": 2, "projected_version": "5", "version": "5", "committing_version": "0", "committed_version": "0", "is_rep": false, "dir_auth": "1", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": { "2": 1 } }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "child": 1, "frozen": 0, "subtree": 1, "importing": 0, "replicated": 1, "dirty": 1 }, "nref": 4 } }, { "is_auth": true, "auth_first": 1, "auth_second": -2, "export_pin": 1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": 1, "dir": { "path": "/tree/6", "dirfrag": "0x10000000427", "snapid_first": 2, "projected_version": "5", "version": "5", "committing_version": "0", "committed_version": "0", "is_rep": false, "dir_auth": "1", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": { "2": 1 } }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "child": 1, "frozen": 0, "subtree": 1, "importing": 0, "replicated": 1, "dirty": 1 }, "nref": 4 } } ] 2024-05-26T22:54:31.963 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-05-26T22:54:32.193 INFO:teuthology.orchestra.run.smithi131.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"2952","version":"2952","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":2,"2":3}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1686","version":"1686","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/tree","dirfrag":"0x1000000041a.000*","snapid_first":2,"projected_version":"433","version":"433","committing_version":"402","committed_version":"402","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"frozen":0,"subtree":1,"dirty":1,"waiter":0,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/tree","dirfrag":"0x1000000041a.001*","snapid_first":2,"projected_version":"429","version":"429","committing_version":"402","committed_version":"402","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"frozen":0,"subtree":1,"dirty":1,"waiter":0,"authpin":0},"nref":3}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/tree","dirfrag":"0x1000000041a.010*","snapid_first":2,"projected_version":"426","version":"426","committing_version":"402","committed_version":"402","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/tree","dirfrag":"0x1000000041a.011*","snapid_first":2,"projected_version":"437","version":"437","committing_version":"402","committed_version":"402","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"frozen":0,"subtree":1,"dirty":1,"waiter":0,"authpin":0},"nref":3}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/tree","dirfrag":"0x1000000041a.100*","snapid_first":2,"projected_version":"426","version":"426","committing_version":"402","committed_version":"402","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree","dirfrag":"0x1000000041a.101*","snapid_first":2,"projected_version":"422","version":"422","committing_version":"402","committed_version":"402","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree","dirfrag":"0x1000000041a.110*","snapid_first":2,"projected_version":"442","version":"442","committing_version":"402","committed_version":"402","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/tree","dirfrag":"0x1000000041a.111*","snapid_first":2,"projected_version":"416","version":"416","committing_version":"402","committed_version":"402","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}}] 2024-05-26T22:54:32.195 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 2, "2": 3 } }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "2952", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "2952" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 1, "subtree": 1 }, "projected_version": "1686", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "1686" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "402", "committing_version": "402", "dir_auth": "0", "dirfrag": "0x1000000041a.000*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "/tree", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "subtree": 1, "waiter": 0 }, "projected_version": "433", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "433" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "402", "committing_version": "402", "dir_auth": "0", "dirfrag": "0x1000000041a.001*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "/tree", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "subtree": 1, "waiter": 0 }, "projected_version": "429", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "429" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "402", "committing_version": "402", "dir_auth": "2", "dirfrag": "0x1000000041a.010*", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "426", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "426" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 2, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "402", "committing_version": "402", "dir_auth": "0", "dirfrag": "0x1000000041a.011*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "/tree", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "subtree": 1, "waiter": 0 }, "projected_version": "437", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "437" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "402", "committing_version": "402", "dir_auth": "2", "dirfrag": "0x1000000041a.100*", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "426", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "426" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 2, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "402", "committing_version": "402", "dir_auth": "1", "dirfrag": "0x1000000041a.101*", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "422", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "422" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "402", "committing_version": "402", "dir_auth": "1", "dirfrag": "0x1000000041a.110*", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "442", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "442" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "402", "committing_version": "402", "dir_auth": "2", "dirfrag": "0x1000000041a.111*", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "416", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "416" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 2, "is_auth": false, "random_ephemeral_pin": false } ] 2024-05-26T22:54:32.195 DEBUG:tasks.cephfs.cephfs_test_case:[ { "is_auth": true, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "", "dirfrag": "0x1", "snapid_first": 2, "projected_version": "2952", "version": "2952", "committing_version": "1", "committed_version": "1", "is_rep": true, "dir_auth": "0", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": { "1": 2, "2": 3 } }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "replicated": 1, "dirty": 1, "authpin": 0 }, "nref": 4 } }, { "is_auth": true, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "~mds0", "dirfrag": "0x100", "snapid_first": 2, "projected_version": "1686", "version": "1686", "committing_version": "1", "committed_version": "1", "is_rep": false, "dir_auth": "0", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "dirty": 1, "authpin": 0 }, "nref": 3 } }, { "is_auth": true, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 0, "dir": { "path": "/tree", "dirfrag": "0x1000000041a.000*", "snapid_first": 2, "projected_version": "433", "version": "433", "committing_version": "402", "committed_version": "402", "is_rep": false, "dir_auth": "0", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "frozen": 0, "subtree": 1, "dirty": 1, "waiter": 0, "authpin": 0 }, "nref": 3 } }, { "is_auth": true, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 0, "dir": { "path": "/tree", "dirfrag": "0x1000000041a.001*", "snapid_first": 2, "projected_version": "429", "version": "429", "committing_version": "402", "committed_version": "402", "is_rep": false, "dir_auth": "0", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "frozen": 0, "subtree": 1, "dirty": 1, "waiter": 0, "authpin": 0 }, "nref": 3 } }, { "is_auth": false, "auth_first": 2, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 2, "dir": { "path": "/tree", "dirfrag": "0x1000000041a.010*", "snapid_first": 2, "projected_version": "426", "version": "426", "committing_version": "402", "committed_version": "402", "is_rep": false, "dir_auth": "2", "states": [], "is_auth": false, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "request": 0, "child": 0, "frozen": 0, "subtree": 1, "replicated": 0, "dirty": 0, "waiter": 0, "authpin": 0, "tempexporting": 0 }, "nref": 1 } }, { "is_auth": true, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 0, "dir": { "path": "/tree", "dirfrag": "0x1000000041a.011*", "snapid_first": 2, "projected_version": "437", "version": "437", "committing_version": "402", "committed_version": "402", "is_rep": false, "dir_auth": "0", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "frozen": 0, "subtree": 1, "dirty": 1, "waiter": 0, "authpin": 0 }, "nref": 3 } }, { "is_auth": false, "auth_first": 2, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 2, "dir": { "path": "/tree", "dirfrag": "0x1000000041a.100*", "snapid_first": 2, "projected_version": "426", "version": "426", "committing_version": "402", "committed_version": "402", "is_rep": false, "dir_auth": "2", "states": [], "is_auth": false, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "request": 0, "child": 0, "frozen": 0, "subtree": 1, "replicated": 0, "dirty": 0, "waiter": 0, "authpin": 0, "tempexporting": 0 }, "nref": 1 } }, { "is_auth": false, "auth_first": 1, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 1, "dir": { "path": "/tree", "dirfrag": "0x1000000041a.101*", "snapid_first": 2, "projected_version": "422", "version": "422", "committing_version": "402", "committed_version": "402", "is_rep": false, "dir_auth": "1", "states": [], "is_auth": false, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "request": 0, "child": 0, "frozen": 0, "subtree": 1, "replicated": 0, "dirty": 0, "waiter": 0, "authpin": 0, "tempexporting": 0 }, "nref": 1 } }, { "is_auth": false, "auth_first": 1, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 1, "dir": { "path": "/tree", "dirfrag": "0x1000000041a.110*", "snapid_first": 2, "projected_version": "442", "version": "442", "committing_version": "402", "committed_version": "402", "is_rep": false, "dir_auth": "1", "states": [], "is_auth": false, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "request": 0, "child": 0, "frozen": 0, "subtree": 1, "replicated": 0, "dirty": 0, "waiter": 0, "authpin": 0, "tempexporting": 0 }, "nref": 1 } }, { "is_auth": false, "auth_first": 2, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 2, "dir": { "path": "/tree", "dirfrag": "0x1000000041a.111*", "snapid_first": 2, "projected_version": "416", "version": "416", "committing_version": "402", "committed_version": "402", "is_rep": false, "dir_auth": "2", "states": [], "is_auth": false, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "request": 0, "child": 0, "frozen": 0, "subtree": 1, "replicated": 0, "dirty": 0, "waiter": 0, "authpin": 0, "tempexporting": 0 }, "nref": 1 } } ] 2024-05-26T22:54:32.195 INFO:tasks.cephfs.cephfs_test_case:[('/tree/0', 1), ('/tree/5', 1), ('/tree/6', 1), ('/tree/7', 1)] =?= [('/tree/0', 1), ('/tree/1', 1), ('/tree/2', 1), ('/tree/3', 1), ('/tree/4', 1), ('/tree/5', 1), ('/tree/6', 1), ('/tree/7', 1), ('/tree/8', 1), ('/tree/9', 1)] 2024-05-26T22:54:34.197 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json get subtrees 2024-05-26T22:54:34.427 INFO:teuthology.orchestra.run.smithi187.stdout:[{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds2","dirfrag":"0x102","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"2","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"2937","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":3},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/tree","dirfrag":"0x1000000041a.111*","snapid_first":2,"projected_version":"419","version":"419","committing_version":"402","committed_version":"402","is_rep":false,"dir_auth":"2","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1,"1":1}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/tree","dirfrag":"0x1000000041a.100*","snapid_first":2,"projected_version":"429","version":"429","committing_version":"402","committed_version":"402","is_rep":false,"dir_auth":"2","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1,"1":1}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/tree","dirfrag":"0x1000000041a.010*","snapid_first":2,"projected_version":"427","version":"427","committing_version":"402","committed_version":"402","is_rep":false,"dir_auth":"2","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree/0","dirfrag":"0x1000000041b","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree/6","dirfrag":"0x10000000427","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}}] 2024-05-26T22:54:34.428 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "2", "dirfrag": "0x102", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds2", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 3 }, "snapid_first": 2, "states": [], "version": "2937" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1, "1": 1 } }, "committed_version": "402", "committing_version": "402", "dir_auth": "2", "dirfrag": "0x1000000041a.111*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/tree", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "419", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "419" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 2, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1, "1": 1 } }, "committed_version": "402", "committing_version": "402", "dir_auth": "2", "dirfrag": "0x1000000041a.100*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/tree", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "429", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "429" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 2, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "402", "committing_version": "402", "dir_auth": "2", "dirfrag": "0x1000000041a.010*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/tree", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "427", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "427" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 2, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x1000000041b", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/0", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000427", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/6", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false } ] 2024-05-26T22:54:34.429 DEBUG:tasks.cephfs.cephfs_test_case:[ { "is_auth": true, "auth_first": 2, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "~mds2", "dirfrag": "0x102", "snapid_first": 2, "projected_version": "1", "version": "1", "committing_version": "1", "committed_version": "1", "is_rep": false, "dir_auth": "2", "states": [ "auth", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "dirty": 0, "authpin": 0 }, "nref": 2 } }, { "is_auth": false, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "", "dirfrag": "0x1", "snapid_first": 2, "projected_version": "0", "version": "2937", "committing_version": "0", "committed_version": "0", "is_rep": true, "dir_auth": "0", "states": [], "is_auth": false, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 3 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1 }, "nref": 2 } }, { "is_auth": true, "auth_first": 2, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 2, "dir": { "path": "/tree", "dirfrag": "0x1000000041a.111*", "snapid_first": 2, "projected_version": "419", "version": "419", "committing_version": "402", "committed_version": "402", "is_rep": false, "dir_auth": "2", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": { "0": 1, "1": 1 } }, "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "child": 1, "frozen": 0, "subtree": 1, "importing": 0, "replicated": 1, "dirty": 1, "authpin": 0 }, "nref": 4 } }, { "is_auth": true, "auth_first": 2, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 2, "dir": { "path": "/tree", "dirfrag": "0x1000000041a.100*", "snapid_first": 2, "projected_version": "429", "version": "429", "committing_version": "402", "committed_version": "402", "is_rep": false, "dir_auth": "2", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": { "0": 1, "1": 1 } }, "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "child": 1, "frozen": 0, "subtree": 1, "importing": 0, "replicated": 1, "dirty": 1, "authpin": 0 }, "nref": 4 } }, { "is_auth": true, "auth_first": 2, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 2, "dir": { "path": "/tree", "dirfrag": "0x1000000041a.010*", "snapid_first": 2, "projected_version": "427", "version": "427", "committing_version": "402", "committed_version": "402", "is_rep": false, "dir_auth": "2", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": { "0": 1 } }, "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "child": 1, "frozen": 0, "subtree": 1, "importing": 0, "replicated": 1, "dirty": 1, "authpin": 0 }, "nref": 4 } }, { "is_auth": false, "auth_first": 1, "auth_second": -2, "export_pin": 1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": 1, "dir": { "path": "/tree/0", "dirfrag": "0x1000000041b", "snapid_first": 2, "projected_version": "5", "version": "5", "committing_version": "0", "committed_version": "0", "is_rep": false, "dir_auth": "1", "states": [], "is_auth": false, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "request": 0, "child": 0, "frozen": 0, "subtree": 1, "replicated": 0, "dirty": 0, "waiter": 0, "authpin": 0, "tempexporting": 0 }, "nref": 1 } }, { "is_auth": false, "auth_first": 1, "auth_second": -2, "export_pin": 1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": 1, "dir": { "path": "/tree/6", "dirfrag": "0x10000000427", "snapid_first": 2, "projected_version": "5", "version": "5", "committing_version": "0", "committed_version": "0", "is_rep": false, "dir_auth": "1", "states": [], "is_auth": false, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "request": 0, "child": 0, "frozen": 0, "subtree": 1, "replicated": 0, "dirty": 0, "waiter": 0, "authpin": 0, "tempexporting": 0 }, "nref": 1 } } ] 2024-05-26T22:54:34.429 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json get subtrees 2024-05-26T22:54:34.648 INFO:teuthology.orchestra.run.smithi187.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"2937","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":2},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree","dirfrag":"0x1000000041a.110*","snapid_first":2,"projected_version":"447","version":"447","committing_version":"402","committed_version":"402","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree","dirfrag":"0x1000000041a.101*","snapid_first":2,"projected_version":"423","version":"423","committing_version":"402","committed_version":"402","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree/5","dirfrag":"0x10000000425","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1},"nref":3}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree/7","dirfrag":"0x10000000429","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1},"nref":3}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/tree","dirfrag":"0x1000000041a.100*","snapid_first":2,"projected_version":"0","version":"428","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"dnwaiter":0,"child":1,"subtree":1},"nref":2}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/tree","dirfrag":"0x1000000041a.111*","snapid_first":2,"projected_version":"0","version":"418","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"dnwaiter":0,"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree/0","dirfrag":"0x1000000041b","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"2":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree/6","dirfrag":"0x10000000427","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"2":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree/1","dirfrag":"0x1000000041d","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree/8","dirfrag":"0x1000000042b","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree/9","dirfrag":"0x1000000042d","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree/4","dirfrag":"0x10000000423","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree/3","dirfrag":"0x10000000421","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree/2","dirfrag":"0x1000000041f","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2024-05-26T22:54:34.649 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 2 }, "snapid_first": 2, "states": [], "version": "2937" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "402", "committing_version": "402", "dir_auth": "1", "dirfrag": "0x1000000041a.110*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/tree", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "447", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "447" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "402", "committing_version": "402", "dir_auth": "1", "dirfrag": "0x1000000041a.101*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/tree", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "423", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "423" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000425", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "/tree/5", "pins": { "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000429", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "/tree/7", "pins": { "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x1000000041a.100*", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/tree", "pins": { "child": 1, "dnwaiter": 0, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "428" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 2, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x1000000041a.111*", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/tree", "pins": { "child": 1, "dnwaiter": 0, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "418" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 2, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "2": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x1000000041b", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/tree/0", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "2": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000427", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/tree/6", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x1000000041d", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/tree/1", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x1000000042b", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/tree/8", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x1000000042d", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/tree/9", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000423", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/tree/4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000421", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/tree/3", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x1000000041f", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/tree/2", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-05-26T22:54:34.650 DEBUG:tasks.cephfs.cephfs_test_case:[ { "is_auth": true, "auth_first": 1, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "~mds1", "dirfrag": "0x101", "snapid_first": 2, "projected_version": "1", "version": "1", "committing_version": "1", "committed_version": "1", "is_rep": false, "dir_auth": "1", "states": [ "auth", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "dirty": 0, "authpin": 0 }, "nref": 2 } }, { "is_auth": false, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "", "dirfrag": "0x1", "snapid_first": 2, "projected_version": "0", "version": "2937", "committing_version": "0", "committed_version": "0", "is_rep": true, "dir_auth": "0", "states": [], "is_auth": false, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 2 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1 }, "nref": 2 } }, { "is_auth": true, "auth_first": 1, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 1, "dir": { "path": "/tree", "dirfrag": "0x1000000041a.110*", "snapid_first": 2, "projected_version": "447", "version": "447", "committing_version": "402", "committed_version": "402", "is_rep": false, "dir_auth": "1", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": { "0": 1 } }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "child": 1, "frozen": 0, "subtree": 1, "importing": 0, "replicated": 1, "dirty": 1, "authpin": 0 }, "nref": 4 } }, { "is_auth": true, "auth_first": 1, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 1, "dir": { "path": "/tree", "dirfrag": "0x1000000041a.101*", "snapid_first": 2, "projected_version": "423", "version": "423", "committing_version": "402", "committed_version": "402", "is_rep": false, "dir_auth": "1", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": { "0": 1 } }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "child": 1, "frozen": 0, "subtree": 1, "importing": 0, "replicated": 1, "dirty": 1, "authpin": 0 }, "nref": 4 } }, { "is_auth": true, "auth_first": 1, "auth_second": -2, "export_pin": 1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": 1, "dir": { "path": "/tree/5", "dirfrag": "0x10000000425", "snapid_first": 2, "projected_version": "5", "version": "5", "committing_version": "0", "committed_version": "0", "is_rep": false, "dir_auth": "1", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "replicated": 0, "dirty": 1 }, "nref": 3 } }, { "is_auth": true, "auth_first": 1, "auth_second": -2, "export_pin": 1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": 1, "dir": { "path": "/tree/7", "dirfrag": "0x10000000429", "snapid_first": 2, "projected_version": "5", "version": "5", "committing_version": "0", "committed_version": "0", "is_rep": false, "dir_auth": "1", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "replicated": 0, "dirty": 1 }, "nref": 3 } }, { "is_auth": false, "auth_first": 2, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 2, "dir": { "path": "/tree", "dirfrag": "0x1000000041a.100*", "snapid_first": 2, "projected_version": "0", "version": "428", "committing_version": "0", "committed_version": "0", "is_rep": false, "dir_auth": "2", "states": [], "is_auth": false, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "dnwaiter": 0, "child": 1, "subtree": 1 }, "nref": 2 } }, { "is_auth": false, "auth_first": 2, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 2, "dir": { "path": "/tree", "dirfrag": "0x1000000041a.111*", "snapid_first": 2, "projected_version": "0", "version": "418", "committing_version": "0", "committed_version": "0", "is_rep": false, "dir_auth": "2", "states": [], "is_auth": false, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "dnwaiter": 0, "child": 1, "subtree": 1 }, "nref": 2 } }, { "is_auth": true, "auth_first": 1, "auth_second": -2, "export_pin": 1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": 1, "dir": { "path": "/tree/0", "dirfrag": "0x1000000041b", "snapid_first": 2, "projected_version": "5", "version": "5", "committing_version": "0", "committed_version": "0", "is_rep": false, "dir_auth": "1", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": { "2": 1 } }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "child": 1, "frozen": 0, "subtree": 1, "importing": 0, "replicated": 1, "dirty": 1 }, "nref": 4 } }, { "is_auth": true, "auth_first": 1, "auth_second": -2, "export_pin": 1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": 1, "dir": { "path": "/tree/6", "dirfrag": "0x10000000427", "snapid_first": 2, "projected_version": "5", "version": "5", "committing_version": "0", "committed_version": "0", "is_rep": false, "dir_auth": "1", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": { "2": 1 } }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "child": 1, "frozen": 0, "subtree": 1, "importing": 0, "replicated": 1, "dirty": 1 }, "nref": 4 } }, { "is_auth": true, "auth_first": 1, "auth_second": -2, "export_pin": 1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": 1, "dir": { "path": "/tree/1", "dirfrag": "0x1000000041d", "snapid_first": 2, "projected_version": "5", "version": "5", "committing_version": "0", "committed_version": "0", "is_rep": false, "dir_auth": "1", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": { "0": 1 } }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "child": 1, "frozen": 0, "subtree": 1, "importing": 0, "replicated": 1, "dirty": 1 }, "nref": 4 } }, { "is_auth": true, "auth_first": 1, "auth_second": -2, "export_pin": 1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": 1, "dir": { "path": "/tree/8", "dirfrag": "0x1000000042b", "snapid_first": 2, "projected_version": "5", "version": "5", "committing_version": "0", "committed_version": "0", "is_rep": false, "dir_auth": "1", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": { "0": 1 } }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "child": 1, "frozen": 0, "subtree": 1, "importing": 0, "replicated": 1, "dirty": 1 }, "nref": 4 } }, { "is_auth": true, "auth_first": 1, "auth_second": -2, "export_pin": 1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": 1, "dir": { "path": "/tree/9", "dirfrag": "0x1000000042d", "snapid_first": 2, "projected_version": "5", "version": "5", "committing_version": "0", "committed_version": "0", "is_rep": false, "dir_auth": "1", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": { "0": 1 } }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "child": 1, "frozen": 0, "subtree": 1, "importing": 0, "replicated": 1, "dirty": 1 }, "nref": 4 } }, { "is_auth": true, "auth_first": 1, "auth_second": -2, "export_pin": 1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": 1, "dir": { "path": "/tree/4", "dirfrag": "0x10000000423", "snapid_first": 2, "projected_version": "5", "version": "5", "committing_version": "0", "committed_version": "0", "is_rep": false, "dir_auth": "1", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": { "0": 1 } }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "child": 1, "frozen": 0, "subtree": 1, "importing": 0, "replicated": 1, "dirty": 1 }, "nref": 4 } }, { "is_auth": true, "auth_first": 1, "auth_second": -2, "export_pin": 1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": 1, "dir": { "path": "/tree/3", "dirfrag": "0x10000000421", "snapid_first": 2, "projected_version": "5", "version": "5", "committing_version": "0", "committed_version": "0", "is_rep": false, "dir_auth": "1", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": { "0": 1 } }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "child": 1, "frozen": 0, "subtree": 1, "importing": 0, "replicated": 1, "dirty": 1 }, "nref": 4 } }, { "is_auth": true, "auth_first": 1, "auth_second": -2, "export_pin": 1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": 1, "dir": { "path": "/tree/2", "dirfrag": "0x1000000041f", "snapid_first": 2, "projected_version": "5", "version": "5", "committing_version": "0", "committed_version": "0", "is_rep": false, "dir_auth": "1", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": { "0": 1 } }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "child": 1, "frozen": 0, "subtree": 1, "importing": 0, "replicated": 1, "dirty": 1 }, "nref": 4 } } ] 2024-05-26T22:54:34.650 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-05-26T22:54:34.882 INFO:teuthology.orchestra.run.smithi131.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"2952","version":"2952","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":2,"2":3}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1686","version":"1686","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/tree","dirfrag":"0x1000000041a.000*","snapid_first":2,"projected_version":"433","version":"433","committing_version":"402","committed_version":"402","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":2}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"frozen":0,"subtree":1,"replicated":1,"dirty":1,"waiter":0,"authpin":0},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/tree","dirfrag":"0x1000000041a.001*","snapid_first":2,"projected_version":"429","version":"429","committing_version":"402","committed_version":"402","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"frozen":0,"subtree":1,"dirty":1,"waiter":0,"authpin":0},"nref":3}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/tree","dirfrag":"0x1000000041a.010*","snapid_first":2,"projected_version":"426","version":"426","committing_version":"402","committed_version":"402","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/tree","dirfrag":"0x1000000041a.011*","snapid_first":2,"projected_version":"437","version":"437","committing_version":"402","committed_version":"402","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":3}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"frozen":0,"subtree":1,"replicated":1,"dirty":1,"waiter":0,"authpin":0},"nref":4}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/tree","dirfrag":"0x1000000041a.100*","snapid_first":2,"projected_version":"426","version":"426","committing_version":"402","committed_version":"402","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree","dirfrag":"0x1000000041a.101*","snapid_first":2,"projected_version":"422","version":"422","committing_version":"402","committed_version":"402","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree","dirfrag":"0x1000000041a.110*","snapid_first":2,"projected_version":"442","version":"442","committing_version":"402","committed_version":"402","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/tree","dirfrag":"0x1000000041a.111*","snapid_first":2,"projected_version":"416","version":"416","committing_version":"402","committed_version":"402","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree/9","dirfrag":"0x1000000042d","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree/8","dirfrag":"0x1000000042b","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree/4","dirfrag":"0x10000000423","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree/3","dirfrag":"0x10000000421","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree/1","dirfrag":"0x1000000041d","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree/2","dirfrag":"0x1000000041f","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}}] 2024-05-26T22:54:34.884 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 2, "2": 3 } }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "2952", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "2952" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 1, "subtree": 1 }, "projected_version": "1686", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "1686" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 2 } }, "committed_version": "402", "committing_version": "402", "dir_auth": "0", "dirfrag": "0x1000000041a.000*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/tree", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "replicated": 1, "subtree": 1, "waiter": 0 }, "projected_version": "433", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "433" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "402", "committing_version": "402", "dir_auth": "0", "dirfrag": "0x1000000041a.001*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "/tree", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "subtree": 1, "waiter": 0 }, "projected_version": "429", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "429" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "402", "committing_version": "402", "dir_auth": "2", "dirfrag": "0x1000000041a.010*", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "426", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "426" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 2, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 3 } }, "committed_version": "402", "committing_version": "402", "dir_auth": "0", "dirfrag": "0x1000000041a.011*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/tree", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "replicated": 1, "subtree": 1, "waiter": 0 }, "projected_version": "437", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "437" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "402", "committing_version": "402", "dir_auth": "2", "dirfrag": "0x1000000041a.100*", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "426", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "426" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 2, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "402", "committing_version": "402", "dir_auth": "1", "dirfrag": "0x1000000041a.101*", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "422", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "422" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "402", "committing_version": "402", "dir_auth": "1", "dirfrag": "0x1000000041a.110*", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "442", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "442" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "402", "committing_version": "402", "dir_auth": "2", "dirfrag": "0x1000000041a.111*", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "416", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "416" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 2, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x1000000042d", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/9", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x1000000042b", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/8", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000423", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/4", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000421", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/3", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x1000000041d", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/1", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x1000000041f", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/2", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false } ] 2024-05-26T22:54:34.885 DEBUG:tasks.cephfs.cephfs_test_case:[ { "is_auth": true, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "", "dirfrag": "0x1", "snapid_first": 2, "projected_version": "2952", "version": "2952", "committing_version": "1", "committed_version": "1", "is_rep": true, "dir_auth": "0", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": { "1": 2, "2": 3 } }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "replicated": 1, "dirty": 1, "authpin": 0 }, "nref": 4 } }, { "is_auth": true, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "~mds0", "dirfrag": "0x100", "snapid_first": 2, "projected_version": "1686", "version": "1686", "committing_version": "1", "committed_version": "1", "is_rep": false, "dir_auth": "0", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "dirty": 1, "authpin": 0 }, "nref": 3 } }, { "is_auth": true, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 0, "dir": { "path": "/tree", "dirfrag": "0x1000000041a.000*", "snapid_first": 2, "projected_version": "433", "version": "433", "committing_version": "402", "committed_version": "402", "is_rep": false, "dir_auth": "0", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": { "1": 2 } }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "frozen": 0, "subtree": 1, "replicated": 1, "dirty": 1, "waiter": 0, "authpin": 0 }, "nref": 4 } }, { "is_auth": true, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 0, "dir": { "path": "/tree", "dirfrag": "0x1000000041a.001*", "snapid_first": 2, "projected_version": "429", "version": "429", "committing_version": "402", "committed_version": "402", "is_rep": false, "dir_auth": "0", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "frozen": 0, "subtree": 1, "dirty": 1, "waiter": 0, "authpin": 0 }, "nref": 3 } }, { "is_auth": false, "auth_first": 2, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 2, "dir": { "path": "/tree", "dirfrag": "0x1000000041a.010*", "snapid_first": 2, "projected_version": "426", "version": "426", "committing_version": "402", "committed_version": "402", "is_rep": false, "dir_auth": "2", "states": [], "is_auth": false, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "request": 0, "child": 0, "frozen": 0, "subtree": 1, "replicated": 0, "dirty": 0, "waiter": 0, "authpin": 0, "tempexporting": 0 }, "nref": 1 } }, { "is_auth": true, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 0, "dir": { "path": "/tree", "dirfrag": "0x1000000041a.011*", "snapid_first": 2, "projected_version": "437", "version": "437", "committing_version": "402", "committed_version": "402", "is_rep": false, "dir_auth": "0", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": { "1": 3 } }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "frozen": 0, "subtree": 1, "replicated": 1, "dirty": 1, "waiter": 0, "authpin": 0 }, "nref": 4 } }, { "is_auth": false, "auth_first": 2, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 2, "dir": { "path": "/tree", "dirfrag": "0x1000000041a.100*", "snapid_first": 2, "projected_version": "426", "version": "426", "committing_version": "402", "committed_version": "402", "is_rep": false, "dir_auth": "2", "states": [], "is_auth": false, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "request": 0, "child": 0, "frozen": 0, "subtree": 1, "replicated": 0, "dirty": 0, "waiter": 0, "authpin": 0, "tempexporting": 0 }, "nref": 1 } }, { "is_auth": false, "auth_first": 1, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 1, "dir": { "path": "/tree", "dirfrag": "0x1000000041a.101*", "snapid_first": 2, "projected_version": "422", "version": "422", "committing_version": "402", "committed_version": "402", "is_rep": false, "dir_auth": "1", "states": [], "is_auth": false, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "request": 0, "child": 0, "frozen": 0, "subtree": 1, "replicated": 0, "dirty": 0, "waiter": 0, "authpin": 0, "tempexporting": 0 }, "nref": 1 } }, { "is_auth": false, "auth_first": 1, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 1, "dir": { "path": "/tree", "dirfrag": "0x1000000041a.110*", "snapid_first": 2, "projected_version": "442", "version": "442", "committing_version": "402", "committed_version": "402", "is_rep": false, "dir_auth": "1", "states": [], "is_auth": false, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "request": 0, "child": 0, "frozen": 0, "subtree": 1, "replicated": 0, "dirty": 0, "waiter": 0, "authpin": 0, "tempexporting": 0 }, "nref": 1 } }, { "is_auth": false, "auth_first": 2, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 2, "dir": { "path": "/tree", "dirfrag": "0x1000000041a.111*", "snapid_first": 2, "projected_version": "416", "version": "416", "committing_version": "402", "committed_version": "402", "is_rep": false, "dir_auth": "2", "states": [], "is_auth": false, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "request": 0, "child": 0, "frozen": 0, "subtree": 1, "replicated": 0, "dirty": 0, "waiter": 0, "authpin": 0, "tempexporting": 0 }, "nref": 1 } }, { "is_auth": false, "auth_first": 1, "auth_second": -2, "export_pin": 1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": 1, "dir": { "path": "/tree/9", "dirfrag": "0x1000000042d", "snapid_first": 2, "projected_version": "5", "version": "5", "committing_version": "0", "committed_version": "0", "is_rep": false, "dir_auth": "1", "states": [], "is_auth": false, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "request": 0, "child": 0, "frozen": 0, "subtree": 1, "replicated": 0, "dirty": 0, "waiter": 0, "authpin": 0, "tempexporting": 0 }, "nref": 1 } }, { "is_auth": false, "auth_first": 1, "auth_second": -2, "export_pin": 1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": 1, "dir": { "path": "/tree/8", "dirfrag": "0x1000000042b", "snapid_first": 2, "projected_version": "5", "version": "5", "committing_version": "0", "committed_version": "0", "is_rep": false, "dir_auth": "1", "states": [], "is_auth": false, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "request": 0, "child": 0, "frozen": 0, "subtree": 1, "replicated": 0, "dirty": 0, "waiter": 0, "authpin": 0, "tempexporting": 0 }, "nref": 1 } }, { "is_auth": false, "auth_first": 1, "auth_second": -2, "export_pin": 1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": 1, "dir": { "path": "/tree/4", "dirfrag": "0x10000000423", "snapid_first": 2, "projected_version": "5", "version": "5", "committing_version": "0", "committed_version": "0", "is_rep": false, "dir_auth": "1", "states": [], "is_auth": false, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "request": 0, "child": 0, "frozen": 0, "subtree": 1, "replicated": 0, "dirty": 0, "waiter": 0, "authpin": 0, "tempexporting": 0 }, "nref": 1 } }, { "is_auth": false, "auth_first": 1, "auth_second": -2, "export_pin": 1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": 1, "dir": { "path": "/tree/3", "dirfrag": "0x10000000421", "snapid_first": 2, "projected_version": "5", "version": "5", "committing_version": "0", "committed_version": "0", "is_rep": false, "dir_auth": "1", "states": [], "is_auth": false, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "request": 0, "child": 0, "frozen": 0, "subtree": 1, "replicated": 0, "dirty": 0, "waiter": 0, "authpin": 0, "tempexporting": 0 }, "nref": 1 } }, { "is_auth": false, "auth_first": 1, "auth_second": -2, "export_pin": 1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": 1, "dir": { "path": "/tree/1", "dirfrag": "0x1000000041d", "snapid_first": 2, "projected_version": "5", "version": "5", "committing_version": "0", "committed_version": "0", "is_rep": false, "dir_auth": "1", "states": [], "is_auth": false, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "request": 0, "child": 0, "frozen": 0, "subtree": 1, "replicated": 0, "dirty": 0, "waiter": 0, "authpin": 0, "tempexporting": 0 }, "nref": 1 } }, { "is_auth": false, "auth_first": 1, "auth_second": -2, "export_pin": 1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": 1, "dir": { "path": "/tree/2", "dirfrag": "0x1000000041f", "snapid_first": 2, "projected_version": "5", "version": "5", "committing_version": "0", "committed_version": "0", "is_rep": false, "dir_auth": "1", "states": [], "is_auth": false, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "request": 0, "child": 0, "frozen": 0, "subtree": 1, "replicated": 0, "dirty": 0, "waiter": 0, "authpin": 0, "tempexporting": 0 }, "nref": 1 } } ] 2024-05-26T22:54:34.885 INFO:tasks.cephfs.cephfs_test_case:[('/tree/0', 1), ('/tree/1', 1), ('/tree/2', 1), ('/tree/3', 1), ('/tree/4', 1), ('/tree/5', 1), ('/tree/6', 1), ('/tree/7', 1), ('/tree/8', 1), ('/tree/9', 1)] =?= [('/tree/0', 1), ('/tree/1', 1), ('/tree/2', 1), ('/tree/3', 1), ('/tree/4', 1), ('/tree/5', 1), ('/tree/6', 1), ('/tree/7', 1), ('/tree/8', 1), ('/tree/9', 1)] 2024-05-26T22:54:34.885 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-05-26T22:54:34.885 DEBUG:teuthology.orchestra.run.smithi131:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-26T22:54:34.989 DEBUG:teuthology.orchestra.run.smithi187:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-26T22:54:35.097 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-05-26T22:54:35.098 INFO:tasks.cephfs.mount:Terminating background process 2024-05-26T22:54:35.098 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-26T22:54:35.098 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-26T22:54:35.165 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-26T22:54:35.165 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-26T22:54:35.229 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-26T22:54:35.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.426+0000 7f13fb000640 1 Processor -- start 2024-05-26T22:54:35.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.426+0000 7f13fb000640 1 -- start start 2024-05-26T22:54:35.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.428+0000 7f13fb000640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f13f4156750 0x7f13f415fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:35.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.428+0000 7f13fb000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f13f40cc7f0 0x7f13f40ccca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:35.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.428+0000 7f13fb000640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13f4160620 con 0x7f13f406e8c0 2024-05-26T22:54:35.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.428+0000 7f13fb000640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f13f415d770 con 0x7f13f4156750 2024-05-26T22:54:35.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.428+0000 7f13fb000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f13f415cbd0 con 0x7f13f40cc7f0 2024-05-26T22:54:35.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.429+0000 7f13f9200640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f13f406e8c0 0x7f13f415d060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:58126/0 (socket says 172.21.15.131:58126) 2024-05-26T22:54:35.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.429+0000 7f13f9200640 1 -- 172.21.15.131:0/2689994567 learned_addr learned my addr 172.21.15.131:0/2689994567 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:54:35.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.429+0000 7f13f9c00640 1 --2- 172.21.15.131:0/2689994567 >> v2:172.21.15.131:3300/0 conn(0x7f13f4156750 0x7f13f415fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:35.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.429+0000 7f13fa600640 1 --2- 172.21.15.131:0/2689994567 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f13f40cc7f0 0x7f13f40ccca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:35.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.429+0000 7f13f9c00640 1 -- 172.21.15.131:0/2689994567 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f13f40cc7f0 msgr2=0x7f13f40ccca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:35.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.429+0000 7f13f3e00640 1 -- 172.21.15.131:0/2689994567 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2562588127 0 0) 0x7f13f4160620 con 0x7f13f406e8c0 2024-05-26T22:54:35.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.429+0000 7f13f9c00640 1 --2- 172.21.15.131:0/2689994567 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f13f40cc7f0 0x7f13f40ccca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:35.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.429+0000 7f13f9c00640 1 -- 172.21.15.131:0/2689994567 >> v1:172.21.15.131:6789/0 conn(0x7f13f406e8c0 legacy=0x7f13f415d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:35.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.429+0000 7f13f9c00640 1 -- 172.21.15.131:0/2689994567 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13f4160900 con 0x7f13f4156750 2024-05-26T22:54:35.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.430+0000 7f13f9c00640 1 --2- 172.21.15.131:0/2689994567 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f13f4156750 0x7f13f415fd40 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f13e80097a0 tx=0x7f13e802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1638d73e82938c03 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:35.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.430+0000 7f13f3e00640 1 -- 172.21.15.131:0/2689994567 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13e8037070 con 0x7f13f4156750 2024-05-26T22:54:35.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.430+0000 7f13f3e00640 1 -- 172.21.15.131:0/2689994567 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13e8032440 con 0x7f13f4156750 2024-05-26T22:54:35.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.430+0000 7f13f3e00640 1 -- 172.21.15.131:0/2689994567 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13e8032810 con 0x7f13f4156750 2024-05-26T22:54:35.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.431+0000 7f13fb000640 1 -- 172.21.15.131:0/2689994567 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f13f4156750 msgr2=0x7f13f415fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:35.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.431+0000 7f13fb000640 1 --2- 172.21.15.131:0/2689994567 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f13f4156750 0x7f13f415fd40 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f13e80097a0 tx=0x7f13e802a180 comp rx=0 tx=0).stop 2024-05-26T22:54:35.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.431+0000 7f13fb000640 1 -- 172.21.15.131:0/2689994567 shutdown_connections 2024-05-26T22:54:35.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.431+0000 7f13fb000640 1 --2- 172.21.15.131:0/2689994567 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f13f40cc7f0 0x7f13f40ccca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:35.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.431+0000 7f13fb000640 1 --2- 172.21.15.131:0/2689994567 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f13f4156750 0x7f13f415fd40 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:35.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.431+0000 7f13fb000640 1 -- 172.21.15.131:0/2689994567 >> 172.21.15.131:0/2689994567 conn(0x7f13f40cff40 msgr2=0x7f13f40d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:35.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.431+0000 7f13fb000640 1 -- 172.21.15.131:0/2689994567 shutdown_connections 2024-05-26T22:54:35.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.431+0000 7f13fb000640 1 -- 172.21.15.131:0/2689994567 wait complete. 2024-05-26T22:54:35.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.433+0000 7f13fb000640 1 Processor -- start 2024-05-26T22:54:35.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.433+0000 7f13fb000640 1 -- start start 2024-05-26T22:54:35.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.434+0000 7f13fb000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f13f406e8c0 0x7f13f41f3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:35.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.434+0000 7f13fb000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f13f40cc7f0 0x7f13f40aee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:35.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.434+0000 7f13fb000640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f13f4156750 0x7f13f40b00b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:35.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.434+0000 7f13fb000640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f13f4156b30 con 0x7f13f406e8c0 2024-05-26T22:54:35.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.434+0000 7f13fb000640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f13f415c860 con 0x7f13f4156750 2024-05-26T22:54:35.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.434+0000 7f13fb000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f13f4160280 con 0x7f13f40cc7f0 2024-05-26T22:54:35.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.434+0000 7f13f9c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f13f406e8c0 0x7f13f41f3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:35.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.434+0000 7f13f9c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f13f406e8c0 0x7f13f41f3f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44136/0 (socket says 172.21.15.131:44136) 2024-05-26T22:54:35.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.434+0000 7f13f9c00640 1 -- 172.21.15.131:0/1108171642 learned_addr learned my addr 172.21.15.131:0/1108171642 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:54:35.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.434+0000 7f13fa600640 1 --2- 172.21.15.131:0/1108171642 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f13f4156750 0x7f13f40b00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:35.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.434+0000 7f13f9c00640 1 -- 172.21.15.131:0/1108171642 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f13f40cc7f0 msgr2=0x7f13f40aee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:35.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.434+0000 7f13f9c00640 1 --2- 172.21.15.131:0/1108171642 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f13f40cc7f0 0x7f13f40aee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:35.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.435+0000 7f13f9c00640 1 -- 172.21.15.131:0/1108171642 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f13f4156750 msgr2=0x7f13f40b00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:35.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.435+0000 7f13f9c00640 1 --2- 172.21.15.131:0/1108171642 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f13f4156750 0x7f13f40b00b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:35.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.435+0000 7f13f9c00640 1 -- 172.21.15.131:0/1108171642 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13f41f4680 con 0x7f13f406e8c0 2024-05-26T22:54:35.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.435+0000 7f13f9c00640 1 --2- 172.21.15.131:0/1108171642 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f13f406e8c0 0x7f13f41f3f70 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f13e80098c0 tx=0x7f13e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:35.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.435+0000 7f13f1600640 1 -- 172.21.15.131:0/1108171642 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13e8037040 con 0x7f13f406e8c0 2024-05-26T22:54:35.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.435+0000 7f13f1600640 1 -- 172.21.15.131:0/1108171642 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13e8030770 con 0x7f13f406e8c0 2024-05-26T22:54:35.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.435+0000 7f13f1600640 1 -- 172.21.15.131:0/1108171642 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13e8030b40 con 0x7f13f406e8c0 2024-05-26T22:54:35.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.435+0000 7f13fb000640 1 -- 172.21.15.131:0/1108171642 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13f41f4960 con 0x7f13f406e8c0 2024-05-26T22:54:35.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.435+0000 7f13fb000640 1 -- 172.21.15.131:0/1108171642 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13f41374f0 con 0x7f13f406e8c0 2024-05-26T22:54:35.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.437+0000 7f13fb000640 1 -- 172.21.15.131:0/1108171642 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13f40694e0 con 0x7f13f406e8c0 2024-05-26T22:54:35.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.438+0000 7f13f1600640 1 -- 172.21.15.131:0/1108171642 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f13e8044020 con 0x7f13f406e8c0 2024-05-26T22:54:35.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.438+0000 7f13f1600640 1 --2- 172.21.15.131:0/1108171642 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f13d406be30 0x7f13d406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:35.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.438+0000 7f13f1600640 1 -- 172.21.15.131:0/1108171642 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5738+0+0 (secure 0 0 0) 0x7f13e80ab410 con 0x7f13f406e8c0 2024-05-26T22:54:35.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.439+0000 7f13f9200640 1 --2- 172.21.15.131:0/1108171642 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f13d406be30 0x7f13d406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:35.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.439+0000 7f13f9200640 1 --2- 172.21.15.131:0/1108171642 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f13d406be30 0x7f13d406e2f0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f13e0007fd0 tx=0x7f13e0007920 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:35.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.445+0000 7f13f1600640 1 -- 172.21.15.131:0/1108171642 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f13e8032e20 con 0x7f13f406e8c0 2024-05-26T22:54:35.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.644+0000 7f13fb000640 1 -- 172.21.15.131:0/1108171642 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f13f41f4e80 con 0x7f13f406e8c0 2024-05-26T22:54:35.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.646+0000 7f13f1600640 1 -- 172.21.15.131:0/1108171642 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+16734 (secure 0 0 0) 0x7f13e8079d00 con 0x7f13f406e8c0 2024-05-26T22:54:35.646 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:54:35.646 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":84,"fsid":"46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f","created":"2024-05-26T22:47:57.288905+0000","modified":"2024-05-26T22:54:07.109294+0000","last_up_change":"2024-05-26T22:48:09.602207+0000","last_in_change":"2024-05-26T22:48:00.687341+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-26T22:48:10.463339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-05-26T22:53:50.625629+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-05-26T22:53:51.864320+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"814eaf62-a18d-494b-8e27-fd8fb88adea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6801","nonce":2350940051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6803","nonce":2350940051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6807","nonce":2350940051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6805","nonce":2350940051}]},"public_addr":"172.21.15.131:6801/2350940051","cluster_addr":"172.21.15.131:6803/2350940051","heartbeat_back_addr":"172.21.15.131:6807/2350940051","heartbeat_front_addr":"172.21.15.131:6805/2350940051","state":["exists","up"]},{"osd":1,"uuid":"42e66287-f6c6-4e26-af3c-6dfaa3536d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6825","nonce":1525528247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6827","nonce":1525528247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6831","nonce":1525528247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6829","nonce":1525528247}]},"public_addr":"172.21.15.131:6825/1525528247","cluster_addr":"172.21.15.131:6827/1525528247","heartbeat_back_addr":"172.21.15.131:6831/1525528247","heartbeat_front_addr":"172.21.15.131:6829/1525528247","state":["exists","up"]},{"osd":2,"uuid":"0da34673-04a2-4fd0-b57a-0ff2b679b043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6817","nonce":3627842210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6819","nonce":3627842210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6823","nonce":3627842210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6821","nonce":3627842210}]},"public_addr":"172.21.15.131:6817/3627842210","cluster_addr":"172.21.15.131:6819/3627842210","heartbeat_back_addr":"172.21.15.131:6823/3627842210","heartbeat_front_addr":"172.21.15.131:6821/3627842210","state":["exists","up"]},{"osd":3,"uuid":"cf117a27-d6b2-43d3-ae52-b47988bf8eed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6809","nonce":350790591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6811","nonce":350790591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6815","nonce":350790591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6813","nonce":350790591}]},"public_addr":"172.21.15.131:6809/350790591","cluster_addr":"172.21.15.131:6811/350790591","heartbeat_back_addr":"172.21.15.131:6815/350790591","heartbeat_front_addr":"172.21.15.131:6813/350790591","state":["exists","up"]},{"osd":4,"uuid":"a092ccfc-be89-4a84-88d5-0f781fa715eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6816","nonce":244560995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6819","nonce":244560995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6823","nonce":244560995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6821","nonce":244560995}]},"public_addr":"172.21.15.187:6816/244560995","cluster_addr":"172.21.15.187:6819/244560995","heartbeat_back_addr":"172.21.15.187:6823/244560995","heartbeat_front_addr":"172.21.15.187:6821/244560995","state":["exists","up"]},{"osd":5,"uuid":"7513ba69-dd00-4a35-b190-9e8cc3b2bdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6801","nonce":324619398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6803","nonce":324619398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6812","nonce":324619398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6807","nonce":324619398}]},"public_addr":"172.21.15.187:6801/324619398","cluster_addr":"172.21.15.187:6803/324619398","heartbeat_back_addr":"172.21.15.187:6812/324619398","heartbeat_front_addr":"172.21.15.187:6807/324619398","state":["exists","up"]},{"osd":6,"uuid":"50081810-c093-4d2f-a4d7-b7c695c5400d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6806","nonce":1391347786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6809","nonce":1391347786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6818","nonce":1391347786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6813","nonce":1391347786}]},"public_addr":"172.21.15.187:6806/1391347786","cluster_addr":"172.21.15.187:6809/1391347786","heartbeat_back_addr":"172.21.15.187:6818/1391347786","heartbeat_front_addr":"172.21.15.187:6813/1391347786","state":["exists","up"]},{"osd":7,"uuid":"775dae05-54f5-4ad4-bbce-92863c6ed2ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6825","nonce":1220065622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6827","nonce":1220065622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6831","nonce":1220065622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6829","nonce":1220065622}]},"public_addr":"172.21.15.187:6825/1220065622","cluster_addr":"172.21.15.187:6827/1220065622","heartbeat_back_addr":"172.21.15.187:6831/1220065622","heartbeat_front_addr":"172.21.15.187:6829/1220065622","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:05.983520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.299516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.067323+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.307748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.018742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.258775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.010912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.295355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.38","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-26T22:54:35.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.649+0000 7f13fb000640 1 -- 172.21.15.131:0/1108171642 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f13d406be30 msgr2=0x7f13d406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:35.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.649+0000 7f13fb000640 1 --2- 172.21.15.131:0/1108171642 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f13d406be30 0x7f13d406e2f0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f13e0007fd0 tx=0x7f13e0007920 comp rx=0 tx=0).stop 2024-05-26T22:54:35.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.649+0000 7f13fb000640 1 -- 172.21.15.131:0/1108171642 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f13f406e8c0 msgr2=0x7f13f41f3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:35.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.649+0000 7f13fb000640 1 --2- 172.21.15.131:0/1108171642 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f13f406e8c0 0x7f13f41f3f70 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f13e80098c0 tx=0x7f13e80046c0 comp rx=0 tx=0).stop 2024-05-26T22:54:35.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.649+0000 7f13fb000640 1 -- 172.21.15.131:0/1108171642 shutdown_connections 2024-05-26T22:54:35.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.649+0000 7f13fb000640 1 --2- 172.21.15.131:0/1108171642 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f13f40cc7f0 0x7f13f40aee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:35.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.649+0000 7f13fb000640 1 --2- 172.21.15.131:0/1108171642 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f13d406be30 0x7f13d406e2f0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:35.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.649+0000 7f13fb000640 1 --2- 172.21.15.131:0/1108171642 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f13f4156750 0x7f13f40b00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:35.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.649+0000 7f13fb000640 1 --2- 172.21.15.131:0/1108171642 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f13f406e8c0 0x7f13f41f3f70 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:35.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.649+0000 7f13fb000640 1 -- 172.21.15.131:0/1108171642 >> 172.21.15.131:0/1108171642 conn(0x7f13f40cff40 msgr2=0x7f13f415ac00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:35.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.649+0000 7f13fb000640 1 -- 172.21.15.131:0/1108171642 shutdown_connections 2024-05-26T22:54:35.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.649+0000 7f13fb000640 1 -- 172.21.15.131:0/1108171642 wait complete. 2024-05-26T22:54:35.668 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-26T22:48:10.463339+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-26T22:53:50.625629+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-05-26T22:53:51.864320+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-26T22:54:35.669 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-26T22:54:35.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.869+0000 7f231e400640 1 Processor -- start 2024-05-26T22:54:35.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.870+0000 7f231e400640 1 -- start start 2024-05-26T22:54:35.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.871+0000 7f231e400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f23180b5450 0x7f23180b43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:35.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.871+0000 7f231e400640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f231806f420 0x7f23180cfd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:35.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.871+0000 7f231e400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23180cf590 0x7f23180d0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:35.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.871+0000 7f231e400640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f2318160d90 con 0x7f23180b5450 2024-05-26T22:54:35.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.871+0000 7f231e400640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f23180b4900 con 0x7f231806f420 2024-05-26T22:54:35.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.871+0000 7f231e400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f2318160180 con 0x7f23180cf590 2024-05-26T22:54:35.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.872+0000 7f2317400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f23180b5450 0x7f23180b43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:35.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.872+0000 7f2317400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f23180b5450 0x7f23180b43c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44144/0 (socket says 172.21.15.131:44144) 2024-05-26T22:54:35.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.872+0000 7f2317400640 1 -- 172.21.15.131:0/3297230749 learned_addr learned my addr 172.21.15.131:0/3297230749 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:54:35.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.872+0000 7f230fe00640 1 --2- 172.21.15.131:0/3297230749 >> v2:172.21.15.187:3300/0 conn(0x7f231806f420 0x7f23180cfd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:35.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.872+0000 7f2317e00640 1 --2- 172.21.15.131:0/3297230749 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23180cf590 0x7f23180d0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:35.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.872+0000 7f2317400640 1 -- 172.21.15.131:0/3297230749 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23180cf590 msgr2=0x7f23180d0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:35.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.872+0000 7f2317400640 1 --2- 172.21.15.131:0/3297230749 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23180cf590 0x7f23180d0290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:35.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.872+0000 7f2317400640 1 -- 172.21.15.131:0/3297230749 >> v2:172.21.15.187:3300/0 conn(0x7f231806f420 msgr2=0x7f23180cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:35.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.872+0000 7f2317400640 1 --2- 172.21.15.131:0/3297230749 >> v2:172.21.15.187:3300/0 conn(0x7f231806f420 0x7f23180cfd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:35.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.872+0000 7f2317400640 1 -- 172.21.15.131:0/3297230749 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2318160460 con 0x7f23180b5450 2024-05-26T22:54:35.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.873+0000 7f2317400640 1 --2- 172.21.15.131:0/3297230749 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f23180b5450 0x7f23180b43c0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f23180b35b0 tx=0x7f230802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1abab3ebc40a234a server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:35.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.874+0000 7f2316a00640 1 -- 172.21.15.131:0/3297230749 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2308037070 con 0x7f23180b5450 2024-05-26T22:54:35.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.874+0000 7f2316a00640 1 -- 172.21.15.131:0/3297230749 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2308032440 con 0x7f23180b5450 2024-05-26T22:54:35.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.874+0000 7f2316a00640 1 -- 172.21.15.131:0/3297230749 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2308032810 con 0x7f23180b5450 2024-05-26T22:54:35.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.874+0000 7f231e400640 1 -- 172.21.15.131:0/3297230749 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f23180b5450 msgr2=0x7f23180b43c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:35.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.874+0000 7f231e400640 1 --2- 172.21.15.131:0/3297230749 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f23180b5450 0x7f23180b43c0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f23180b35b0 tx=0x7f230802a1e0 comp rx=0 tx=0).stop 2024-05-26T22:54:35.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.875+0000 7f231e400640 1 -- 172.21.15.131:0/3297230749 shutdown_connections 2024-05-26T22:54:35.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.875+0000 7f231e400640 1 --2- 172.21.15.131:0/3297230749 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23180cf590 0x7f23180d0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:35.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.875+0000 7f231e400640 1 --2- 172.21.15.131:0/3297230749 >> v2:172.21.15.187:3300/0 conn(0x7f231806f420 0x7f23180cfd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:35.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.875+0000 7f231e400640 1 --2- 172.21.15.131:0/3297230749 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f23180b5450 0x7f23180b43c0 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:35.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.875+0000 7f231e400640 1 -- 172.21.15.131:0/3297230749 >> 172.21.15.131:0/3297230749 conn(0x7f231806bc90 msgr2=0x7f23180b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:35.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.875+0000 7f231e400640 1 -- 172.21.15.131:0/3297230749 shutdown_connections 2024-05-26T22:54:35.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.875+0000 7f231e400640 1 -- 172.21.15.131:0/3297230749 wait complete. 2024-05-26T22:54:35.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.876+0000 7f231e400640 1 Processor -- start 2024-05-26T22:54:35.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.876+0000 7f231e400640 1 -- start start 2024-05-26T22:54:35.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.877+0000 7f231e400640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f231806f420 0x7f231814d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:35.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.877+0000 7f231e400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23180b5450 0x7f231815ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:35.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.877+0000 7f231e400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f23180cf590 0x7f23180cd650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:35.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.877+0000 7f231e400640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f23180b5830 con 0x7f23180cf590 2024-05-26T22:54:35.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.877+0000 7f231e400640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2318160a20 con 0x7f231806f420 2024-05-26T22:54:35.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.877+0000 7f231e400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f231815fde0 con 0x7f23180b5450 2024-05-26T22:54:35.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.878+0000 7f2317e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f23180cf590 0x7f23180cd650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:35.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.878+0000 7f2317e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f23180cf590 0x7f23180cd650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44150/0 (socket says 172.21.15.131:44150) 2024-05-26T22:54:35.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.878+0000 7f2317e00640 1 -- 172.21.15.131:0/1072691830 learned_addr learned my addr 172.21.15.131:0/1072691830 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:54:35.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.878+0000 7f230fe00640 1 --2- 172.21.15.131:0/1072691830 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23180b5450 0x7f231815ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:35.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.878+0000 7f2317400640 1 --2- 172.21.15.131:0/1072691830 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f231806f420 0x7f231814d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:35.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.878+0000 7f2317e00640 1 -- 172.21.15.131:0/1072691830 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23180b5450 msgr2=0x7f231815ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:35.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.878+0000 7f2317e00640 1 --2- 172.21.15.131:0/1072691830 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23180b5450 0x7f231815ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:35.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.878+0000 7f2317e00640 1 -- 172.21.15.131:0/1072691830 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f231806f420 msgr2=0x7f231814d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:35.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.878+0000 7f2317e00640 1 --2- 172.21.15.131:0/1072691830 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f231806f420 0x7f231814d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:35.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.878+0000 7f2317e00640 1 -- 172.21.15.131:0/1072691830 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f231815c750 con 0x7f23180cf590 2024-05-26T22:54:35.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.878+0000 7f230fe00640 1 --2- 172.21.15.131:0/1072691830 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23180b5450 0x7f231815ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:54:35.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.878+0000 7f2317e00640 1 --2- 172.21.15.131:0/1072691830 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f23180cf590 0x7f23180cd650 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f2310007e40 tx=0x7f231000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:35.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.879+0000 7f230f400640 1 -- 172.21.15.131:0/1072691830 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2310014620 con 0x7f23180cf590 2024-05-26T22:54:35.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.879+0000 7f230f400640 1 -- 172.21.15.131:0/1072691830 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2310019070 con 0x7f23180cf590 2024-05-26T22:54:35.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.879+0000 7f230f400640 1 -- 172.21.15.131:0/1072691830 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2310014a20 con 0x7f23180cf590 2024-05-26T22:54:35.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.879+0000 7f231e400640 1 -- 172.21.15.131:0/1072691830 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f231815ca30 con 0x7f23180cf590 2024-05-26T22:54:35.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.879+0000 7f231e400640 1 -- 172.21.15.131:0/1072691830 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f231815cff0 con 0x7f23180cf590 2024-05-26T22:54:35.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.880+0000 7f231e400640 1 -- 172.21.15.131:0/1072691830 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2318074e90 con 0x7f23180cf590 2024-05-26T22:54:35.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.881+0000 7f230f400640 1 -- 172.21.15.131:0/1072691830 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2310014bc0 con 0x7f23180cf590 2024-05-26T22:54:35.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.882+0000 7f230f400640 1 --2- 172.21.15.131:0/1072691830 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f22f406be30 0x7f22f406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:35.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.882+0000 7f230f400640 1 -- 172.21.15.131:0/1072691830 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5738+0+0 (secure 0 0 0) 0x7f231008b7a0 con 0x7f23180cf590 2024-05-26T22:54:35.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.882+0000 7f2317400640 1 --2- 172.21.15.131:0/1072691830 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f22f406be30 0x7f22f406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:35.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.883+0000 7f2317400640 1 --2- 172.21.15.131:0/1072691830 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f22f406be30 0x7f22f406e2f0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f2308034000 tx=0x7f230802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:35.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:35.886+0000 7f230f400640 1 -- 172.21.15.131:0/1072691830 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f231005a090 con 0x7f23180cf590 2024-05-26T22:54:36.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.087+0000 7f231e400640 1 -- 172.21.15.131:0/1072691830 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f23180695c0 con 0x7f23180cf590 2024-05-26T22:54:36.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.087+0000 7f230f400640 1 -- 172.21.15.131:0/1072691830 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) ==== 93+0+10 (secure 0 0 0) 0x7f2310059990 con 0x7f23180cf590 2024-05-26T22:54:36.087 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2024-05-26T22:54:36.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.090+0000 7f231e400640 1 -- 172.21.15.131:0/1072691830 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f22f406be30 msgr2=0x7f22f406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:36.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.090+0000 7f231e400640 1 --2- 172.21.15.131:0/1072691830 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f22f406be30 0x7f22f406e2f0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f2308034000 tx=0x7f230802e000 comp rx=0 tx=0).stop 2024-05-26T22:54:36.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.090+0000 7f231e400640 1 -- 172.21.15.131:0/1072691830 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f23180cf590 msgr2=0x7f23180cd650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:36.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.090+0000 7f231e400640 1 --2- 172.21.15.131:0/1072691830 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f23180cf590 0x7f23180cd650 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f2310007e40 tx=0x7f231000e4b0 comp rx=0 tx=0).stop 2024-05-26T22:54:36.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.090+0000 7f231e400640 1 -- 172.21.15.131:0/1072691830 shutdown_connections 2024-05-26T22:54:36.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.090+0000 7f231e400640 1 --2- 172.21.15.131:0/1072691830 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23180b5450 0x7f231815ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:36.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.090+0000 7f231e400640 1 --2- 172.21.15.131:0/1072691830 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f22f406be30 0x7f22f406e2f0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:36.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.090+0000 7f231e400640 1 --2- 172.21.15.131:0/1072691830 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f23180cf590 0x7f23180cd650 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:36.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.090+0000 7f231e400640 1 --2- 172.21.15.131:0/1072691830 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f231806f420 0x7f231814d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:36.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.090+0000 7f231e400640 1 -- 172.21.15.131:0/1072691830 >> 172.21.15.131:0/1072691830 conn(0x7f231806bc90 msgr2=0x7f23180ccd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:36.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.090+0000 7f231e400640 1 -- 172.21.15.131:0/1072691830 shutdown_connections 2024-05-26T22:54:36.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.090+0000 7f231e400640 1 -- 172.21.15.131:0/1072691830 wait complete. 2024-05-26T22:54:36.108 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-26T22:54:36.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.309+0000 7f47c8600640 1 Processor -- start 2024-05-26T22:54:36.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.309+0000 7f47c8600640 1 -- start start 2024-05-26T22:54:36.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.310+0000 7f47c8600640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f47c006dd30 0x7f47c006e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:36.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.311+0000 7f47c8600640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f47c00725a0 0x7f47c00729a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:36.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.311+0000 7f47c8600640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47c015c1c0 con 0x7f47c0155770 2024-05-26T22:54:36.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.311+0000 7f47c8600640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f47c015c4a0 con 0x7f47c006dd30 2024-05-26T22:54:36.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.311+0000 7f47c8600640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f47c0071da0 con 0x7f47c00725a0 2024-05-26T22:54:36.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.311+0000 7f47c5200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f47c006dd30 0x7f47c006e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:36.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.311+0000 7f47c5200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f47c006dd30 0x7f47c006e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44160/0 (socket says 172.21.15.131:44160) 2024-05-26T22:54:36.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.311+0000 7f47c5200640 1 -- 172.21.15.131:0/2544187550 learned_addr learned my addr 172.21.15.131:0/2544187550 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:54:36.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.311+0000 7f47c6600640 1 --2- 172.21.15.131:0/2544187550 >> v2:172.21.15.187:3300/0 conn(0x7f47c00725a0 0x7f47c00729a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:36.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.311+0000 7f47c5200640 1 -- 172.21.15.131:0/2544187550 >> v2:172.21.15.187:3300/0 conn(0x7f47c00725a0 msgr2=0x7f47c00729a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:36.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.311+0000 7f47c5200640 1 --2- 172.21.15.131:0/2544187550 >> v2:172.21.15.187:3300/0 conn(0x7f47c00725a0 0x7f47c00729a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:36.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.311+0000 7f47c5200640 1 -- 172.21.15.131:0/2544187550 >> v1:172.21.15.187:6789/0 conn(0x7f47c0155770 legacy=0x7f47c015b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:36.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.311+0000 7f47c5200640 1 -- 172.21.15.131:0/2544187550 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47c0072080 con 0x7f47c006dd30 2024-05-26T22:54:36.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.311+0000 7f47c6600640 1 --2- 172.21.15.131:0/2544187550 >> v2:172.21.15.187:3300/0 conn(0x7f47c00725a0 0x7f47c00729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:54:36.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.312+0000 7f47c5200640 1 --2- 172.21.15.131:0/2544187550 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47c006dd30 0x7f47c006e1c0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f47c00afb20 tx=0x7f47ac02a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5ecfb563b20603f server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:36.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.313+0000 7f47bfe00640 1 -- 172.21.15.131:0/2544187550 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47ac037070 con 0x7f47c006dd30 2024-05-26T22:54:36.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.313+0000 7f47bfe00640 1 -- 172.21.15.131:0/2544187550 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47ac032440 con 0x7f47c006dd30 2024-05-26T22:54:36.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.313+0000 7f47bfe00640 1 -- 172.21.15.131:0/2544187550 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47ac032810 con 0x7f47c006dd30 2024-05-26T22:54:36.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.313+0000 7f47c8600640 1 -- 172.21.15.131:0/2544187550 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47c006dd30 msgr2=0x7f47c006e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:36.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.313+0000 7f47c8600640 1 --2- 172.21.15.131:0/2544187550 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47c006dd30 0x7f47c006e1c0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f47c00afb20 tx=0x7f47ac02a2c0 comp rx=0 tx=0).stop 2024-05-26T22:54:36.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.313+0000 7f47c8600640 1 -- 172.21.15.131:0/2544187550 shutdown_connections 2024-05-26T22:54:36.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.313+0000 7f47c8600640 1 --2- 172.21.15.131:0/2544187550 >> v2:172.21.15.187:3300/0 conn(0x7f47c00725a0 0x7f47c00729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:36.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.313+0000 7f47c8600640 1 --2- 172.21.15.131:0/2544187550 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47c006dd30 0x7f47c006e1c0 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:36.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.313+0000 7f47c8600640 1 -- 172.21.15.131:0/2544187550 >> 172.21.15.131:0/2544187550 conn(0x7f47c006bc90 msgr2=0x7f47c00b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:36.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.314+0000 7f47c8600640 1 -- 172.21.15.131:0/2544187550 shutdown_connections 2024-05-26T22:54:36.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.314+0000 7f47c8600640 1 -- 172.21.15.131:0/2544187550 wait complete. 2024-05-26T22:54:36.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.315+0000 7f47c8600640 1 Processor -- start 2024-05-26T22:54:36.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.315+0000 7f47c8600640 1 -- start start 2024-05-26T22:54:36.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.316+0000 7f47c8600640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f47c006dd30 0x7f47c0141880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:36.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.316+0000 7f47c8600640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f47c00725a0 0x7f47c00affc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:36.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.316+0000 7f47c8600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47c0155770 0x7f47c00b4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:36.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.316+0000 7f47c8600640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f47c015be10 con 0x7f47c0155770 2024-05-26T22:54:36.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.316+0000 7f47c8600640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f47c015c8c0 con 0x7f47c00725a0 2024-05-26T22:54:36.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.316+0000 7f47c8600640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f47c0141dc0 con 0x7f47c006dd30 2024-05-26T22:54:36.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.317+0000 7f47c6600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47c0155770 0x7f47c00b4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:36.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.317+0000 7f47c6600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47c0155770 0x7f47c00b4aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44172/0 (socket says 172.21.15.131:44172) 2024-05-26T22:54:36.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.317+0000 7f47c6600640 1 -- 172.21.15.131:0/4050774767 learned_addr learned my addr 172.21.15.131:0/4050774767 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:54:36.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.317+0000 7f47c6600640 1 -- 172.21.15.131:0/4050774767 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f47c006dd30 msgr2=0x7f47c0141880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:36.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.317+0000 7f47c6600640 1 --2- 172.21.15.131:0/4050774767 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f47c006dd30 0x7f47c0141880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:36.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.317+0000 7f47c6600640 1 -- 172.21.15.131:0/4050774767 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f47c00725a0 msgr2=0x7f47c00affc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:36.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.317+0000 7f47c6600640 1 --2- 172.21.15.131:0/4050774767 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f47c00725a0 0x7f47c00affc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:36.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.317+0000 7f47c6600640 1 -- 172.21.15.131:0/4050774767 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47c00b0a60 con 0x7f47c0155770 2024-05-26T22:54:36.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.317+0000 7f47c6600640 1 --2- 172.21.15.131:0/4050774767 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47c0155770 0x7f47c00b4aa0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f47b800cf20 tx=0x7f47b800a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:36.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.317+0000 7f47bd600640 1 -- 172.21.15.131:0/4050774767 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47b801a070 con 0x7f47c0155770 2024-05-26T22:54:36.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.318+0000 7f47bd600640 1 -- 172.21.15.131:0/4050774767 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47b800d840 con 0x7f47c0155770 2024-05-26T22:54:36.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.318+0000 7f47bd600640 1 -- 172.21.15.131:0/4050774767 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47b800dc30 con 0x7f47c0155770 2024-05-26T22:54:36.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.318+0000 7f47c8600640 1 -- 172.21.15.131:0/4050774767 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47c00b0d40 con 0x7f47c0155770 2024-05-26T22:54:36.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.318+0000 7f47c8600640 1 -- 172.21.15.131:0/4050774767 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47c00b1300 con 0x7f47c0155770 2024-05-26T22:54:36.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.319+0000 7f47c8600640 1 -- 172.21.15.131:0/4050774767 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47c0074e90 con 0x7f47c0155770 2024-05-26T22:54:36.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.320+0000 7f47bd600640 1 -- 172.21.15.131:0/4050774767 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f47b801d070 con 0x7f47c0155770 2024-05-26T22:54:36.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.320+0000 7f47bd600640 1 --2- 172.21.15.131:0/4050774767 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f47a006be60 0x7f47a006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:36.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.321+0000 7f47bd600640 1 -- 172.21.15.131:0/4050774767 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5738+0+0 (secure 0 0 0) 0x7f47b808b730 con 0x7f47c0155770 2024-05-26T22:54:36.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.321+0000 7f47c5c00640 1 --2- 172.21.15.131:0/4050774767 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f47a006be60 0x7f47a006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:36.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.321+0000 7f47c5c00640 1 --2- 172.21.15.131:0/4050774767 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f47a006be60 0x7f47a006e320 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f47b4006fd0 tx=0x7f47b4006bc0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:36.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.325+0000 7f47bd600640 1 -- 172.21.15.131:0/4050774767 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f47b805a0b0 con 0x7f47c0155770 2024-05-26T22:54:36.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.522+0000 7f47c8600640 1 -- 172.21.15.131:0/4050774767 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f47c014d700 con 0x7f47c0155770 2024-05-26T22:54:36.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.522+0000 7f47bd600640 1 -- 172.21.15.131:0/4050774767 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v84) ==== 104+0+11 (secure 0 0 0) 0x7f47b80599b0 con 0x7f47c0155770 2024-05-26T22:54:36.522 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-05-26T22:54:36.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.525+0000 7f47c8600640 1 -- 172.21.15.131:0/4050774767 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f47a006be60 msgr2=0x7f47a006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:36.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.525+0000 7f47c8600640 1 --2- 172.21.15.131:0/4050774767 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f47a006be60 0x7f47a006e320 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f47b4006fd0 tx=0x7f47b4006bc0 comp rx=0 tx=0).stop 2024-05-26T22:54:36.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.525+0000 7f47c8600640 1 -- 172.21.15.131:0/4050774767 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47c0155770 msgr2=0x7f47c00b4aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:36.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.525+0000 7f47c8600640 1 --2- 172.21.15.131:0/4050774767 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47c0155770 0x7f47c00b4aa0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f47b800cf20 tx=0x7f47b800a7d0 comp rx=0 tx=0).stop 2024-05-26T22:54:36.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.525+0000 7f47c8600640 1 -- 172.21.15.131:0/4050774767 shutdown_connections 2024-05-26T22:54:36.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.525+0000 7f47c8600640 1 --2- 172.21.15.131:0/4050774767 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f47a006be60 0x7f47a006e320 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:36.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.525+0000 7f47c8600640 1 --2- 172.21.15.131:0/4050774767 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47c0155770 0x7f47c00b4aa0 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:36.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.525+0000 7f47c8600640 1 --2- 172.21.15.131:0/4050774767 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f47c00725a0 0x7f47c00affc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:36.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.525+0000 7f47c8600640 1 --2- 172.21.15.131:0/4050774767 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f47c006dd30 0x7f47c0141880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:36.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.525+0000 7f47c8600640 1 -- 172.21.15.131:0/4050774767 >> 172.21.15.131:0/4050774767 conn(0x7f47c006bc90 msgr2=0x7f47c014dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:36.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.526+0000 7f47c8600640 1 -- 172.21.15.131:0/4050774767 shutdown_connections 2024-05-26T22:54:36.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.526+0000 7f47c8600640 1 -- 172.21.15.131:0/4050774767 wait complete. 2024-05-26T22:54:36.544 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-26T22:54:36.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.751+0000 7f9d87200640 1 Processor -- start 2024-05-26T22:54:36.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.751+0000 7f9d87200640 1 -- start start 2024-05-26T22:54:36.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.752+0000 7f9d87200640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f9d880b9f20 0x7f9d880d22b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:36.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.752+0000 7f9d87200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d880d27f0 0x7f9d880b8d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:36.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.752+0000 7f9d87200640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d880b79f0 con 0x7f9d8815e340 2024-05-26T22:54:36.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.752+0000 7f9d87200640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f9d880b88b0 con 0x7f9d880b9f20 2024-05-26T22:54:36.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.752+0000 7f9d87200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9d8815e720 con 0x7f9d880d27f0 2024-05-26T22:54:36.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.753+0000 7f9d85e00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f9d8815e340 0x7f9d880b72e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:58136/0 (socket says 172.21.15.131:58136) 2024-05-26T22:54:36.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.753+0000 7f9d85e00640 1 -- 172.21.15.131:0/3650940420 learned_addr learned my addr 172.21.15.131:0/3650940420 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:54:36.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.753+0000 7f9d86800640 1 --2- 172.21.15.131:0/3650940420 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d880d27f0 0x7f9d880b8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:36.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.753+0000 7f9d84a00640 1 -- 172.21.15.131:0/3650940420 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2836706385 0 0) 0x7f9d880b79f0 con 0x7f9d8815e340 2024-05-26T22:54:36.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.753+0000 7f9d85400640 1 --2- 172.21.15.131:0/3650940420 >> v2:172.21.15.187:3300/0 conn(0x7f9d880b9f20 0x7f9d880d22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:36.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.753+0000 7f9d84a00640 1 -- 172.21.15.131:0/3650940420 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9d6c003520 con 0x7f9d8815e340 2024-05-26T22:54:36.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.754+0000 7f9d84a00640 1 -- 172.21.15.131:0/3650940420 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1776988030 0 0) 0x7f9d6c003520 con 0x7f9d8815e340 2024-05-26T22:54:36.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.754+0000 7f9d84a00640 1 -- 172.21.15.131:0/3650940420 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d880d27f0 msgr2=0x7f9d880b8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:36.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.754+0000 7f9d84a00640 1 --2- 172.21.15.131:0/3650940420 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d880d27f0 0x7f9d880b8d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:36.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.754+0000 7f9d84a00640 1 -- 172.21.15.131:0/3650940420 >> v2:172.21.15.187:3300/0 conn(0x7f9d880b9f20 msgr2=0x7f9d880d22b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:36.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.754+0000 7f9d84a00640 1 --2- 172.21.15.131:0/3650940420 >> v2:172.21.15.187:3300/0 conn(0x7f9d880b9f20 0x7f9d880d22b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:36.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.754+0000 7f9d84a00640 1 -- 172.21.15.131:0/3650940420 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d880b7ca0 con 0x7f9d8815e340 2024-05-26T22:54:36.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.754+0000 7f9d84a00640 1 -- 172.21.15.131:0/3650940420 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7f9d78001670 con 0x7f9d8815e340 2024-05-26T22:54:36.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.754+0000 7f9d84a00640 1 -- 172.21.15.131:0/3650940420 >> v1:172.21.15.131:6789/0 conn(0x7f9d8815e340 legacy=0x7f9d880b72e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:36.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.754+0000 7f9d84a00640 1 --2- 172.21.15.131:0/3650940420 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d880d27f0 0x7f9d6c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:36.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.754+0000 7f9d84a00640 1 --2- 172.21.15.131:0/3650940420 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d6c003f30 0x7f9d6c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:36.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.754+0000 7f9d84a00640 1 --2- 172.21.15.131:0/3650940420 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d6c004d40 0x7f9d6c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:36.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.754+0000 7f9d84a00640 1 -- 172.21.15.131:0/3650940420 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f9d880b88b0 con 0x7f9d6c003f30 2024-05-26T22:54:36.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.754+0000 7f9d84a00640 1 -- 172.21.15.131:0/3650940420 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9d8815e720 con 0x7f9d6c004d40 2024-05-26T22:54:36.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.754+0000 7f9d84a00640 1 -- 172.21.15.131:0/3650940420 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9d6c00b9f0 con 0x7f9d880d27f0 2024-05-26T22:54:36.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.754+0000 7f9d85400640 1 --2- 172.21.15.131:0/3650940420 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d6c003f30 0x7f9d6c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:36.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.754+0000 7f9d85e00640 1 --2- 172.21.15.131:0/3650940420 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d880d27f0 0x7f9d6c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:36.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.754+0000 7f9d86800640 1 --2- 172.21.15.131:0/3650940420 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d6c004d40 0x7f9d6c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:36.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.755+0000 7f9d85400640 1 -- 172.21.15.131:0/3650940420 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d880d27f0 msgr2=0x7f9d6c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:36.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.755+0000 7f9d85400640 1 --2- 172.21.15.131:0/3650940420 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d880d27f0 0x7f9d6c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:36.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.755+0000 7f9d85400640 1 -- 172.21.15.131:0/3650940420 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d6c004d40 msgr2=0x7f9d6c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:36.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.755+0000 7f9d85400640 1 --2- 172.21.15.131:0/3650940420 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d6c004d40 0x7f9d6c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:36.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.755+0000 7f9d85400640 1 -- 172.21.15.131:0/3650940420 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d6c003520 con 0x7f9d6c003f30 2024-05-26T22:54:36.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.756+0000 7f9d85400640 1 --2- 172.21.15.131:0/3650940420 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d6c003f30 0x7f9d6c004800 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f9d74009ac0 tx=0x7f9d6c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f564afd8751a6bee server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:36.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.756+0000 7f9d84a00640 1 -- 172.21.15.131:0/3650940420 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d74036070 con 0x7f9d6c003f30 2024-05-26T22:54:36.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.756+0000 7f9d84a00640 1 -- 172.21.15.131:0/3650940420 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d74031400 con 0x7f9d6c003f30 2024-05-26T22:54:36.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.756+0000 7f9d84a00640 1 -- 172.21.15.131:0/3650940420 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d74009690 con 0x7f9d6c003f30 2024-05-26T22:54:36.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.756+0000 7f9d87200640 1 -- 172.21.15.131:0/3650940420 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d6c003f30 msgr2=0x7f9d6c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:36.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.756+0000 7f9d87200640 1 --2- 172.21.15.131:0/3650940420 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d6c003f30 0x7f9d6c004800 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f9d74009ac0 tx=0x7f9d6c005830 comp rx=0 tx=0).stop 2024-05-26T22:54:36.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.756+0000 7f9d86800640 1 -- 172.21.15.131:0/3650940420 reap_dead start 2024-05-26T22:54:36.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.757+0000 7f9d87200640 1 -- 172.21.15.131:0/3650940420 shutdown_connections 2024-05-26T22:54:36.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.757+0000 7f9d87200640 1 -- 172.21.15.131:0/3650940420 >> 172.21.15.131:0/3650940420 conn(0x7f9d88072e60 msgr2=0x7f9d88073270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:36.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.757+0000 7f9d87200640 1 -- 172.21.15.131:0/3650940420 shutdown_connections 2024-05-26T22:54:36.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.757+0000 7f9d87200640 1 -- 172.21.15.131:0/3650940420 wait complete. 2024-05-26T22:54:36.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.758+0000 7f9d87200640 1 Processor -- start 2024-05-26T22:54:36.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.759+0000 7f9d87200640 1 -- start start 2024-05-26T22:54:36.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.759+0000 7f9d87200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d6c003f30 0x7f9d88152d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:36.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.759+0000 7f9d87200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d88156de0 0x7f9d88154370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:36.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.759+0000 7f9d87200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d880c91e0 0x7f9d880c9690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:36.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.759+0000 7f9d87200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f9d880b8540 con 0x7f9d6c003f30 2024-05-26T22:54:36.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.759+0000 7f9d87200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9d880b8260 con 0x7f9d88156de0 2024-05-26T22:54:36.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.759+0000 7f9d87200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9d880b8d40 con 0x7f9d880c91e0 2024-05-26T22:54:36.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.760+0000 7f9d85e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d6c003f30 0x7f9d88152d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:36.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.760+0000 7f9d85e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d6c003f30 0x7f9d88152d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44198/0 (socket says 172.21.15.131:44198) 2024-05-26T22:54:36.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.760+0000 7f9d85e00640 1 -- 172.21.15.131:0/696058209 learned_addr learned my addr 172.21.15.131:0/696058209 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:54:36.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.760+0000 7f9d85400640 1 --2- 172.21.15.131:0/696058209 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d88156de0 0x7f9d88154370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:36.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.760+0000 7f9d86800640 1 --2- 172.21.15.131:0/696058209 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d880c91e0 0x7f9d880c9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:36.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.760+0000 7f9d85e00640 1 -- 172.21.15.131:0/696058209 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d880c91e0 msgr2=0x7f9d880c9690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:36.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.760+0000 7f9d85e00640 1 --2- 172.21.15.131:0/696058209 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d880c91e0 0x7f9d880c9690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:36.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.760+0000 7f9d85e00640 1 -- 172.21.15.131:0/696058209 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d88156de0 msgr2=0x7f9d88154370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:36.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.760+0000 7f9d85e00640 1 --2- 172.21.15.131:0/696058209 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d88156de0 0x7f9d88154370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:36.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.760+0000 7f9d85e00640 1 -- 172.21.15.131:0/696058209 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d881575a0 con 0x7f9d6c003f30 2024-05-26T22:54:36.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.760+0000 7f9d85400640 1 --2- 172.21.15.131:0/696058209 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d88156de0 0x7f9d88154370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:54:36.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.760+0000 7f9d86800640 1 --2- 172.21.15.131:0/696058209 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d880c91e0 0x7f9d880c9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:54:36.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.761+0000 7f9d85e00640 1 --2- 172.21.15.131:0/696058209 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d6c003f30 0x7f9d88152d40 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f9d78001200 tx=0x7f9d7800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:36.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.761+0000 7f9d7e000640 1 -- 172.21.15.131:0/696058209 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d7800c8a0 con 0x7f9d6c003f30 2024-05-26T22:54:36.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.761+0000 7f9d7e000640 1 -- 172.21.15.131:0/696058209 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d78018070 con 0x7f9d6c003f30 2024-05-26T22:54:36.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.761+0000 7f9d7e000640 1 -- 172.21.15.131:0/696058209 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d7800cc80 con 0x7f9d6c003f30 2024-05-26T22:54:36.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.761+0000 7f9d87200640 1 -- 172.21.15.131:0/696058209 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d881572c0 con 0x7f9d6c003f30 2024-05-26T22:54:36.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.761+0000 7f9d87200640 1 -- 172.21.15.131:0/696058209 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d880cc780 con 0x7f9d6c003f30 2024-05-26T22:54:36.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.762+0000 7f9d87200640 1 -- 172.21.15.131:0/696058209 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d88153d70 con 0x7f9d6c003f30 2024-05-26T22:54:36.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.763+0000 7f9d7e000640 1 -- 172.21.15.131:0/696058209 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9d7801d460 con 0x7f9d6c003f30 2024-05-26T22:54:36.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.764+0000 7f9d7e000640 1 --2- 172.21.15.131:0/696058209 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9d6806be30 0x7f9d6806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:36.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.764+0000 7f9d7e000640 1 -- 172.21.15.131:0/696058209 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5738+0+0 (secure 0 0 0) 0x7f9d7808aef0 con 0x7f9d6c003f30 2024-05-26T22:54:36.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.764+0000 7f9d85400640 1 --2- 172.21.15.131:0/696058209 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9d6806be30 0x7f9d6806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:36.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.765+0000 7f9d85400640 1 --2- 172.21.15.131:0/696058209 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9d6806be30 0x7f9d6806e2f0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f9d74033000 tx=0x7f9d74008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:36.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.769+0000 7f9d7e000640 1 -- 172.21.15.131:0/696058209 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f9d7800ce20 con 0x7f9d6c003f30 2024-05-26T22:54:36.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.968+0000 7f9d87200640 1 -- 172.21.15.131:0/696058209 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f9d88073640 con 0x7f9d6c003f30 2024-05-26T22:54:36.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.969+0000 7f9d7e000640 1 -- 172.21.15.131:0/696058209 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v84) ==== 100+0+11 (secure 0 0 0) 0x7f9d88073640 con 0x7f9d6c003f30 2024-05-26T22:54:36.969 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 64 2024-05-26T22:54:36.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.972+0000 7f9d87200640 1 -- 172.21.15.131:0/696058209 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9d6806be30 msgr2=0x7f9d6806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:36.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.972+0000 7f9d87200640 1 --2- 172.21.15.131:0/696058209 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9d6806be30 0x7f9d6806e2f0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f9d74033000 tx=0x7f9d74008030 comp rx=0 tx=0).stop 2024-05-26T22:54:36.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.972+0000 7f9d87200640 1 -- 172.21.15.131:0/696058209 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d6c003f30 msgr2=0x7f9d88152d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:36.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.972+0000 7f9d87200640 1 --2- 172.21.15.131:0/696058209 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d6c003f30 0x7f9d88152d40 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f9d78001200 tx=0x7f9d7800ecd0 comp rx=0 tx=0).stop 2024-05-26T22:54:36.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.972+0000 7f9d87200640 1 -- 172.21.15.131:0/696058209 shutdown_connections 2024-05-26T22:54:36.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.972+0000 7f9d87200640 1 --2- 172.21.15.131:0/696058209 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d880c91e0 0x7f9d880c9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:36.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.972+0000 7f9d87200640 1 --2- 172.21.15.131:0/696058209 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9d6806be30 0x7f9d6806e2f0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:36.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.972+0000 7f9d87200640 1 --2- 172.21.15.131:0/696058209 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d88156de0 0x7f9d88154370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:36.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.972+0000 7f9d87200640 1 --2- 172.21.15.131:0/696058209 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d6c003f30 0x7f9d88152d40 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:36.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.972+0000 7f9d87200640 1 -- 172.21.15.131:0/696058209 >> 172.21.15.131:0/696058209 conn(0x7f9d88072e60 msgr2=0x7f9d880d2c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:36.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.972+0000 7f9d87200640 1 -- 172.21.15.131:0/696058209 shutdown_connections 2024-05-26T22:54:36.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:36.972+0000 7f9d87200640 1 -- 172.21.15.131:0/696058209 wait complete. 2024-05-26T22:54:36.991 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-26T22:54:37.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.194+0000 7f5ba5600640 1 Processor -- start 2024-05-26T22:54:37.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.194+0000 7f5ba5600640 1 -- start start 2024-05-26T22:54:37.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.196+0000 7f5ba5600640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f5ba00712d0 0x7f5ba006d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:37.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.196+0000 7f5ba5600640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f5ba006d790 0x7f5ba00cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:37.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.196+0000 7f5ba5600640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ba0072d60 con 0x7f5ba0145a10 2024-05-26T22:54:37.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.196+0000 7f5ba5600640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f5ba015c760 con 0x7f5ba006d790 2024-05-26T22:54:37.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.196+0000 7f5ba5600640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f5ba006dbc0 con 0x7f5ba00712d0 2024-05-26T22:54:37.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.196+0000 7f5ba4c00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f5ba006d790 0x7f5ba00cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:37.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.196+0000 7f5b9fe00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f5ba0145a10 0x7f5ba0072180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:58138/0 (socket says 172.21.15.131:58138) 2024-05-26T22:54:37.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.196+0000 7f5b9fe00640 1 -- 172.21.15.131:0/1176324441 learned_addr learned my addr 172.21.15.131:0/1176324441 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:54:37.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.197+0000 7f5b9f400640 1 --2- 172.21.15.131:0/1176324441 >> v2:172.21.15.187:3300/0 conn(0x7f5ba00712d0 0x7f5ba006d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:37.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.197+0000 7f5ba4c00640 1 -- 172.21.15.131:0/1176324441 >> v2:172.21.15.187:3300/0 conn(0x7f5ba00712d0 msgr2=0x7f5ba006d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:37.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.197+0000 7f5ba4c00640 1 --2- 172.21.15.131:0/1176324441 >> v2:172.21.15.187:3300/0 conn(0x7f5ba00712d0 0x7f5ba006d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:37.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.197+0000 7f5ba4c00640 1 -- 172.21.15.131:0/1176324441 >> v1:172.21.15.131:6789/0 conn(0x7f5ba0145a10 legacy=0x7f5ba0072180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:37.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.197+0000 7f5ba4c00640 1 -- 172.21.15.131:0/1176324441 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ba0072a80 con 0x7f5ba006d790 2024-05-26T22:54:37.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.197+0000 7f5b9f400640 1 --2- 172.21.15.131:0/1176324441 >> v2:172.21.15.187:3300/0 conn(0x7f5ba00712d0 0x7f5ba006d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:54:37.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.198+0000 7f5ba4c00640 1 --2- 172.21.15.131:0/1176324441 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ba006d790 0x7f5ba00cf590 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f5ba00af570 tx=0x7f5b9802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45c5134d03d9496c server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:37.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.198+0000 7f5b9ea00640 1 -- 172.21.15.131:0/1176324441 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b98007740 con 0x7f5ba006d790 2024-05-26T22:54:37.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.198+0000 7f5b9ea00640 1 -- 172.21.15.131:0/1176324441 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b980078e0 con 0x7f5ba006d790 2024-05-26T22:54:37.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.198+0000 7f5b9ea00640 1 -- 172.21.15.131:0/1176324441 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b98007cb0 con 0x7f5ba006d790 2024-05-26T22:54:37.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.199+0000 7f5ba5600640 1 -- 172.21.15.131:0/1176324441 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ba006d790 msgr2=0x7f5ba00cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:37.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.199+0000 7f5ba5600640 1 --2- 172.21.15.131:0/1176324441 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ba006d790 0x7f5ba00cf590 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f5ba00af570 tx=0x7f5b9802a6a0 comp rx=0 tx=0).stop 2024-05-26T22:54:37.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.199+0000 7f5ba5600640 1 -- 172.21.15.131:0/1176324441 shutdown_connections 2024-05-26T22:54:37.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.199+0000 7f5ba5600640 1 --2- 172.21.15.131:0/1176324441 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ba006d790 0x7f5ba00cf590 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:37.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.199+0000 7f5ba5600640 1 --2- 172.21.15.131:0/1176324441 >> v2:172.21.15.187:3300/0 conn(0x7f5ba00712d0 0x7f5ba006d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:37.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.199+0000 7f5ba5600640 1 -- 172.21.15.131:0/1176324441 >> 172.21.15.131:0/1176324441 conn(0x7f5ba00b5660 msgr2=0x7f5ba00b5a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:37.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.199+0000 7f5ba5600640 1 -- 172.21.15.131:0/1176324441 shutdown_connections 2024-05-26T22:54:37.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.199+0000 7f5ba5600640 1 -- 172.21.15.131:0/1176324441 wait complete. 2024-05-26T22:54:37.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.201+0000 7f5ba5600640 1 Processor -- start 2024-05-26T22:54:37.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.201+0000 7f5ba5600640 1 -- start start 2024-05-26T22:54:37.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.202+0000 7f5ba5600640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5ba006d790 0x7f5ba00ad870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:37.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.202+0000 7f5ba5600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ba00712d0 0x7f5ba015a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:37.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.202+0000 7f5ba5600640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5ba0145a10 0x7f5ba015aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:37.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.202+0000 7f5ba5600640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f5ba00cf070 con 0x7f5ba00712d0 2024-05-26T22:54:37.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.202+0000 7f5ba5600640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5ba00ced90 con 0x7f5ba006d790 2024-05-26T22:54:37.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.202+0000 7f5ba5600640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f5ba0145df0 con 0x7f5ba0145a10 2024-05-26T22:54:37.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.202+0000 7f5b9f400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ba00712d0 0x7f5ba015a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:37.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.202+0000 7f5b9f400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ba00712d0 0x7f5ba015a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44220/0 (socket says 172.21.15.131:44220) 2024-05-26T22:54:37.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.202+0000 7f5b9f400640 1 -- 172.21.15.131:0/912311548 learned_addr learned my addr 172.21.15.131:0/912311548 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:54:37.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.203+0000 7f5b9fe00640 1 --2- 172.21.15.131:0/912311548 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5ba006d790 0x7f5ba00ad870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:37.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.203+0000 7f5ba4c00640 1 --2- 172.21.15.131:0/912311548 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5ba0145a10 0x7f5ba015aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:37.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.203+0000 7f5b9f400640 1 -- 172.21.15.131:0/912311548 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5ba0145a10 msgr2=0x7f5ba015aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:37.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.203+0000 7f5b9f400640 1 --2- 172.21.15.131:0/912311548 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5ba0145a10 0x7f5ba015aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:37.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.203+0000 7f5b9f400640 1 -- 172.21.15.131:0/912311548 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5ba006d790 msgr2=0x7f5ba00ad870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:37.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.203+0000 7f5b9f400640 1 --2- 172.21.15.131:0/912311548 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5ba006d790 0x7f5ba00ad870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:37.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.203+0000 7f5b9f400640 1 -- 172.21.15.131:0/912311548 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ba00addb0 con 0x7f5ba00712d0 2024-05-26T22:54:37.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.203+0000 7f5ba4c00640 1 --2- 172.21.15.131:0/912311548 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5ba0145a10 0x7f5ba015aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:54:37.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.203+0000 7f5b9f400640 1 --2- 172.21.15.131:0/912311548 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ba00712d0 0x7f5ba015a510 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f5b8800c7f0 tx=0x7f5b8800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:37.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.203+0000 7f5b97e00640 1 -- 172.21.15.131:0/912311548 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b88015070 con 0x7f5ba00712d0 2024-05-26T22:54:37.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.203+0000 7f5b97e00640 1 -- 172.21.15.131:0/912311548 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b88007a30 con 0x7f5ba00712d0 2024-05-26T22:54:37.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.204+0000 7f5b97e00640 1 -- 172.21.15.131:0/912311548 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b88007e00 con 0x7f5ba00712d0 2024-05-26T22:54:37.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.204+0000 7f5ba5600640 1 -- 172.21.15.131:0/912311548 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ba00ae050 con 0x7f5ba00712d0 2024-05-26T22:54:37.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.204+0000 7f5ba5600640 1 -- 172.21.15.131:0/912311548 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ba00ae610 con 0x7f5ba00712d0 2024-05-26T22:54:37.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.205+0000 7f5ba5600640 1 -- 172.21.15.131:0/912311548 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ba00694e0 con 0x7f5ba00712d0 2024-05-26T22:54:37.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.206+0000 7f5b97e00640 1 -- 172.21.15.131:0/912311548 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5b88018070 con 0x7f5ba00712d0 2024-05-26T22:54:37.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.207+0000 7f5b97e00640 1 --2- 172.21.15.131:0/912311548 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f5b7c06be30 0x7f5b7c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:37.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.207+0000 7f5b97e00640 1 -- 172.21.15.131:0/912311548 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5738+0+0 (secure 0 0 0) 0x7f5b8808ae70 con 0x7f5ba00712d0 2024-05-26T22:54:37.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.207+0000 7f5b9fe00640 1 --2- 172.21.15.131:0/912311548 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f5b7c06be30 0x7f5b7c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:37.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.208+0000 7f5b9fe00640 1 --2- 172.21.15.131:0/912311548 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f5b7c06be30 0x7f5b7c06e2f0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f5ba00cddd0 tx=0x7f5b90009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:37.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.212+0000 7f5b97e00640 1 -- 172.21.15.131:0/912311548 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f5b88059760 con 0x7f5ba00712d0 2024-05-26T22:54:37.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.427+0000 7f5ba5600640 1 -- 172.21.15.131:0/912311548 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f5ba00b0260 con 0x7f5ba00712d0 2024-05-26T22:54:37.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.428+0000 7f5b97e00640 1 -- 172.21.15.131:0/912311548 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v84) ==== 80+0+0 (secure 0 0 0) 0x7f5b88059060 con 0x7f5ba00712d0 2024-05-26T22:54:37.428 INFO:teuthology.orchestra.run.smithi131.stderr:listed 0 entries 2024-05-26T22:54:37.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.431+0000 7f5ba5600640 1 -- 172.21.15.131:0/912311548 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f5b7c06be30 msgr2=0x7f5b7c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:37.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.431+0000 7f5ba5600640 1 --2- 172.21.15.131:0/912311548 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f5b7c06be30 0x7f5b7c06e2f0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f5ba00cddd0 tx=0x7f5b90009040 comp rx=0 tx=0).stop 2024-05-26T22:54:37.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.431+0000 7f5ba5600640 1 -- 172.21.15.131:0/912311548 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ba00712d0 msgr2=0x7f5ba015a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:37.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.431+0000 7f5ba5600640 1 --2- 172.21.15.131:0/912311548 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ba00712d0 0x7f5ba015a510 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f5b8800c7f0 tx=0x7f5b8800ccb0 comp rx=0 tx=0).stop 2024-05-26T22:54:37.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.431+0000 7f5ba5600640 1 -- 172.21.15.131:0/912311548 shutdown_connections 2024-05-26T22:54:37.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.431+0000 7f5ba5600640 1 --2- 172.21.15.131:0/912311548 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5ba0145a10 0x7f5ba015aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:37.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.431+0000 7f5ba5600640 1 --2- 172.21.15.131:0/912311548 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f5b7c06be30 0x7f5b7c06e2f0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:37.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.431+0000 7f5ba5600640 1 --2- 172.21.15.131:0/912311548 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ba00712d0 0x7f5ba015a510 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:37.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.431+0000 7f5ba5600640 1 --2- 172.21.15.131:0/912311548 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5ba006d790 0x7f5ba00ad870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:37.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.431+0000 7f5ba5600640 1 -- 172.21.15.131:0/912311548 >> 172.21.15.131:0/912311548 conn(0x7f5ba00b5660 msgr2=0x7f5ba00b5a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:37.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.431+0000 7f5ba5600640 1 -- 172.21.15.131:0/912311548 shutdown_connections 2024-05-26T22:54:37.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.431+0000 7f5ba5600640 1 -- 172.21.15.131:0/912311548 wait complete. 2024-05-26T22:54:37.449 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-26T22:54:37.449 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:54:37.449 DEBUG:teuthology.orchestra.run.smithi131:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-26T22:54:37.589 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi131.front.sepia.ceph.com 2024-05-26T22:54:37.589 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:54:37.589 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-26T22:54:37.637 INFO:tasks.cephfs.mount:Terminating background process 2024-05-26T22:54:37.637 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-05-26T22:54:37.638 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-26T22:54:37.677 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-26T22:54:37.677 INFO:tasks.cephfs.mount:Terminating background process 2024-05-26T22:54:37.677 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-26T22:54:37.677 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-26T22:54:37.716 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-26T22:54:37.716 INFO:tasks.cephfs.mount:Terminating background process 2024-05-26T22:54:37.717 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-05-26T22:54:37.717 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-26T22:54:37.764 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-26T22:54:37.764 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-26T22:54:37.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.962+0000 7f84e5200640 1 Processor -- start 2024-05-26T22:54:37.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.963+0000 7f84e5200640 1 -- start start 2024-05-26T22:54:37.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.964+0000 7f84e5200640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f84e00ad250 0x7f84e00b3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:37.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.964+0000 7f84e5200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f84e01569d0 0x7f84e00af790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:37.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.964+0000 7f84e5200640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84e015cb20 con 0x7f84e006ee20 2024-05-26T22:54:37.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.964+0000 7f84e5200640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f84e015ce00 con 0x7f84e00ad250 2024-05-26T22:54:37.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.964+0000 7f84e5200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f84e015d0e0 con 0x7f84e01569d0 2024-05-26T22:54:37.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.965+0000 7f84d7e00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f84e00ad250 0x7f84e00b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:37.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.965+0000 7f84de800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f84e01569d0 0x7f84e00af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:37.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.965+0000 7f84de800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f84e01569d0 0x7f84e00af790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.131:55852/0 (socket says 172.21.15.131:55852) 2024-05-26T22:54:37.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.965+0000 7f84de800640 1 -- 172.21.15.131:0/787757893 learned_addr learned my addr 172.21.15.131:0/787757893 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:54:37.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.966+0000 7f84d7e00640 1 -- 172.21.15.131:0/787757893 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f84e01569d0 msgr2=0x7f84e00af790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:37.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.966+0000 7f84d7e00640 1 --2- 172.21.15.131:0/787757893 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f84e01569d0 0x7f84e00af790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:37.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.966+0000 7f84d7e00640 1 -- 172.21.15.131:0/787757893 >> v1:172.21.15.187:6789/0 conn(0x7f84e006ee20 legacy=0x7f84e01562c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:37.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.966+0000 7f84d7e00640 1 -- 172.21.15.131:0/787757893 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84e015d3c0 con 0x7f84e00ad250 2024-05-26T22:54:37.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.966+0000 7f84de800640 1 --2- 172.21.15.131:0/787757893 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f84e01569d0 0x7f84e00af790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-26T22:54:37.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.966+0000 7f84dd400640 1 -- 172.21.15.131:0/787757893 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2325553943 0 0) 0x7f84e015cb20 con 0x7f84e006ee20 2024-05-26T22:54:37.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.967+0000 7f84d7e00640 1 --2- 172.21.15.131:0/787757893 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f84e00ad250 0x7f84e00b3190 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f84c80097a0 tx=0x7f84c802a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3333a570ba975d7c server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:37.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.968+0000 7f84dd400640 1 -- 172.21.15.131:0/787757893 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84c8037070 con 0x7f84e00ad250 2024-05-26T22:54:37.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.968+0000 7f84dd400640 1 -- 172.21.15.131:0/787757893 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84c8032440 con 0x7f84e00ad250 2024-05-26T22:54:37.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.968+0000 7f84dd400640 1 -- 172.21.15.131:0/787757893 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84c8032810 con 0x7f84e00ad250 2024-05-26T22:54:37.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.969+0000 7f84e5200640 1 -- 172.21.15.131:0/787757893 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f84e00ad250 msgr2=0x7f84e00b3190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:37.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.969+0000 7f84e5200640 1 --2- 172.21.15.131:0/787757893 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f84e00ad250 0x7f84e00b3190 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f84c80097a0 tx=0x7f84c802a180 comp rx=0 tx=0).stop 2024-05-26T22:54:37.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.969+0000 7f84e5200640 1 -- 172.21.15.131:0/787757893 shutdown_connections 2024-05-26T22:54:37.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.969+0000 7f84e5200640 1 --2- 172.21.15.131:0/787757893 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f84e01569d0 0x7f84e00af790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:37.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.969+0000 7f84e5200640 1 --2- 172.21.15.131:0/787757893 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f84e00ad250 0x7f84e00b3190 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:37.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.969+0000 7f84e5200640 1 -- 172.21.15.131:0/787757893 >> 172.21.15.131:0/787757893 conn(0x7f84e00b1e90 msgr2=0x7f84e00b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:37.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.969+0000 7f84e5200640 1 -- 172.21.15.131:0/787757893 shutdown_connections 2024-05-26T22:54:37.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.969+0000 7f84e5200640 1 -- 172.21.15.131:0/787757893 wait complete. 2024-05-26T22:54:37.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.971+0000 7f84e5200640 1 Processor -- start 2024-05-26T22:54:37.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.971+0000 7f84e5200640 1 -- start start 2024-05-26T22:54:37.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.972+0000 7f84e5200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84e006ee20 0x7f84e01e4c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:37.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.972+0000 7f84e5200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f84e00ad250 0x7f84e01f8520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:37.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.972+0000 7f84e5200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f84e01569d0 0x7f84e0072850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:37.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.972+0000 7f84e5200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f84e015c900 con 0x7f84e006ee20 2024-05-26T22:54:37.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.972+0000 7f84e5200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f84e015c780 con 0x7f84e01569d0 2024-05-26T22:54:37.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.972+0000 7f84e5200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f84e006f200 con 0x7f84e00ad250 2024-05-26T22:54:37.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.972+0000 7f84dde00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84e006ee20 0x7f84e01e4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:37.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.972+0000 7f84dde00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84e006ee20 0x7f84e01e4c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44232/0 (socket says 172.21.15.131:44232) 2024-05-26T22:54:37.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.972+0000 7f84dde00640 1 -- 172.21.15.131:0/2186811764 learned_addr learned my addr 172.21.15.131:0/2186811764 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:54:37.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.973+0000 7f84de800640 1 --2- 172.21.15.131:0/2186811764 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f84e01569d0 0x7f84e0072850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:37.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.973+0000 7f84d7e00640 1 --2- 172.21.15.131:0/2186811764 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f84e00ad250 0x7f84e01f8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:37.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.973+0000 7f84dde00640 1 -- 172.21.15.131:0/2186811764 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f84e00ad250 msgr2=0x7f84e01f8520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:37.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.973+0000 7f84dde00640 1 --2- 172.21.15.131:0/2186811764 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f84e00ad250 0x7f84e01f8520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:37.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.973+0000 7f84dde00640 1 -- 172.21.15.131:0/2186811764 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f84e01569d0 msgr2=0x7f84e0072850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:37.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.973+0000 7f84dde00640 1 --2- 172.21.15.131:0/2186811764 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f84e01569d0 0x7f84e0072850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:37.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.973+0000 7f84dde00640 1 -- 172.21.15.131:0/2186811764 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84e01f8a60 con 0x7f84e006ee20 2024-05-26T22:54:37.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.973+0000 7f84dde00640 1 --2- 172.21.15.131:0/2186811764 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84e006ee20 0x7f84e01e4c90 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f84d000c460 tx=0x7f84d000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:37.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.973+0000 7f84d6000640 1 -- 172.21.15.131:0/2186811764 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d0005480 con 0x7f84e006ee20 2024-05-26T22:54:37.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.973+0000 7f84d6000640 1 -- 172.21.15.131:0/2186811764 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84d00035f0 con 0x7f84e006ee20 2024-05-26T22:54:37.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.973+0000 7f84e5200640 1 -- 172.21.15.131:0/2186811764 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84e01e5290 con 0x7f84e006ee20 2024-05-26T22:54:37.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.974+0000 7f84d6000640 1 -- 172.21.15.131:0/2186811764 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d0003a00 con 0x7f84e006ee20 2024-05-26T22:54:37.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.974+0000 7f84e5200640 1 -- 172.21.15.131:0/2186811764 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84e01e5850 con 0x7f84e006ee20 2024-05-26T22:54:37.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.975+0000 7f84e5200640 1 -- 172.21.15.131:0/2186811764 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84e00add10 con 0x7f84e006ee20 2024-05-26T22:54:37.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.976+0000 7f84d6000640 1 -- 172.21.15.131:0/2186811764 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f84d0003ba0 con 0x7f84e006ee20 2024-05-26T22:54:37.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.976+0000 7f84d6000640 1 --2- 172.21.15.131:0/2186811764 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f84bc06c000 0x7f84bc06e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:37.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.977+0000 7f84d6000640 1 -- 172.21.15.131:0/2186811764 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5738+0+0 (secure 0 0 0) 0x7f84d008aac0 con 0x7f84e006ee20 2024-05-26T22:54:37.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.977+0000 7f84d7e00640 1 --2- 172.21.15.131:0/2186811764 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f84bc06c000 0x7f84bc06e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:37.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.978+0000 7f84d7e00640 1 --2- 172.21.15.131:0/2186811764 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f84bc06c000 0x7f84bc06e4c0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f84c8034000 tx=0x7f84c802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:37.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:37.981+0000 7f84d6000640 1 -- 172.21.15.131:0/2186811764 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f84e00add10 con 0x7f84e006ee20 2024-05-26T22:54:38.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.214+0000 7f84e5200640 1 -- 172.21.15.131:0/2186811764 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f84e00694e0 con 0x7f84e006ee20 2024-05-26T22:54:38.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.215+0000 7f84d6000640 1 -- 172.21.15.131:0/2186811764 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) ==== 94+0+5231 (secure 0 0 0) 0x7f84d0059440 con 0x7f84e006ee20 2024-05-26T22:54:38.215 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:54:38.215 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":85,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5533,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/993930297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6835","nonce":993930297},{"type":"v1","addr":"172.21.15.131:6837","nonce":993930297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-26T22:53:52.862328+0000","modified":"2024-05-26T22:54:28.015691+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7365,"mds_1":5548,"mds_2":5545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5545":{"gid":5545,"name":"b","rank":2,"incarnation":82,"state":"up:active","state_seq":7,"addr":"172.21.15.187:6834/3480101957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3480101957},{"type":"v1","addr":"172.21.15.187:6834","nonce":3480101957}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5548":{"gid":5548,"name":"d","rank":1,"incarnation":80,"state":"up:active","state_seq":7,"addr":"172.21.15.187:6835/567389303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6833","nonce":567389303},{"type":"v1","addr":"172.21.15.187:6835","nonce":567389303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7365":{"gid":7365,"name":"a","rank":0,"incarnation":74,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6836/3411392567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":3411392567},{"type":"v1","addr":"172.21.15.131:6836","nonce":3411392567}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7365,"qdb_cluster":[5545,7365,5548]},"id":6}]} 2024-05-26T22:54:38.215 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 85 2024-05-26T22:54:38.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.218+0000 7f84e5200640 1 -- 172.21.15.131:0/2186811764 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f84bc06c000 msgr2=0x7f84bc06e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:38.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.218+0000 7f84e5200640 1 --2- 172.21.15.131:0/2186811764 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f84bc06c000 0x7f84bc06e4c0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f84c8034000 tx=0x7f84c802e000 comp rx=0 tx=0).stop 2024-05-26T22:54:38.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.218+0000 7f84e5200640 1 -- 172.21.15.131:0/2186811764 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84e006ee20 msgr2=0x7f84e01e4c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:38.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.218+0000 7f84e5200640 1 --2- 172.21.15.131:0/2186811764 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84e006ee20 0x7f84e01e4c90 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f84d000c460 tx=0x7f84d000c920 comp rx=0 tx=0).stop 2024-05-26T22:54:38.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.218+0000 7f84e5200640 1 -- 172.21.15.131:0/2186811764 shutdown_connections 2024-05-26T22:54:38.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.218+0000 7f84e5200640 1 --2- 172.21.15.131:0/2186811764 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f84e00ad250 0x7f84e01f8520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:38.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.218+0000 7f84e5200640 1 --2- 172.21.15.131:0/2186811764 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f84bc06c000 0x7f84bc06e4c0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:38.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.218+0000 7f84e5200640 1 --2- 172.21.15.131:0/2186811764 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f84e01569d0 0x7f84e0072850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:38.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.218+0000 7f84e5200640 1 --2- 172.21.15.131:0/2186811764 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84e006ee20 0x7f84e01e4c90 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:38.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.218+0000 7f84e5200640 1 -- 172.21.15.131:0/2186811764 >> 172.21.15.131:0/2186811764 conn(0x7f84e00b1e90 msgr2=0x7f84e015d920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:38.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.218+0000 7f84e5200640 1 -- 172.21.15.131:0/2186811764 shutdown_connections 2024-05-26T22:54:38.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.218+0000 7f84e5200640 1 -- 172.21.15.131:0/2186811764 wait complete. 2024-05-26T22:54:38.235 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-26T22:54:38.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.429+0000 7efe88800640 1 Processor -- start 2024-05-26T22:54:38.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.430+0000 7efe88800640 1 -- start start 2024-05-26T22:54:38.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.431+0000 7efe88800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efe800ad250 0x7efe800b3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:38.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.431+0000 7efe88800640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe8015cb20 con 0x7efe801569d0 2024-05-26T22:54:38.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.431+0000 7efe88800640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe8015ce00 con 0x7efe8006ee20 2024-05-26T22:54:38.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.431+0000 7efe88800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7efe8015d0e0 con 0x7efe800ad250 2024-05-26T22:54:38.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.432+0000 7efe86800640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7efe801569d0 0x7efe80160010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:58146/0 (socket says 172.21.15.131:58146) 2024-05-26T22:54:38.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.432+0000 7efe86800640 1 -- 172.21.15.131:0/3757898075 learned_addr learned my addr 172.21.15.131:0/3757898075 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:54:38.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.432+0000 7efe85400640 1 --2- 172.21.15.131:0/3757898075 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efe800ad250 0x7efe800b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:38.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.432+0000 7efe84a00640 1 -- 172.21.15.131:0/3757898075 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3326211138 0 0) 0x7efe8015cb20 con 0x7efe801569d0 2024-05-26T22:54:38.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.432+0000 7efe84a00640 1 -- 172.21.15.131:0/3757898075 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efe68003520 con 0x7efe801569d0 2024-05-26T22:54:38.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.432+0000 7efe84a00640 1 -- 172.21.15.131:0/3757898075 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2929299958 0 0) 0x7efe68003520 con 0x7efe801569d0 2024-05-26T22:54:38.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.432+0000 7efe84a00640 1 -- 172.21.15.131:0/3757898075 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efe800ad250 msgr2=0x7efe800b3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:38.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.432+0000 7efe84a00640 1 --2- 172.21.15.131:0/3757898075 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efe800ad250 0x7efe800b3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:38.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.432+0000 7efe84a00640 1 -- 172.21.15.131:0/3757898075 >> v1:172.21.15.187:6789/0 conn(0x7efe8006ee20 legacy=0x7efe801562c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:38.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.432+0000 7efe84a00640 1 -- 172.21.15.131:0/3757898075 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe8015d3c0 con 0x7efe801569d0 2024-05-26T22:54:38.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.432+0000 7efe84a00640 1 -- 172.21.15.131:0/3757898075 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7efe7c0019f0 con 0x7efe801569d0 2024-05-26T22:54:38.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.433+0000 7efe84a00640 1 -- 172.21.15.131:0/3757898075 >> v1:172.21.15.131:6789/0 conn(0x7efe801569d0 legacy=0x7efe80160010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:38.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.433+0000 7efe84a00640 1 --2- 172.21.15.131:0/3757898075 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efe680039e0 0x7efe68003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:38.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.433+0000 7efe84a00640 1 --2- 172.21.15.131:0/3757898075 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efe800ad250 0x7efe68004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:38.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.433+0000 7efe84a00640 1 --2- 172.21.15.131:0/3757898075 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efe68004d40 0x7efe680051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:38.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.433+0000 7efe84a00640 1 -- 172.21.15.131:0/3757898075 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7efe8015d0e0 con 0x7efe68004d40 2024-05-26T22:54:38.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.433+0000 7efe84a00640 1 -- 172.21.15.131:0/3757898075 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7efe6800baa0 con 0x7efe680039e0 2024-05-26T22:54:38.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.433+0000 7efe84a00640 1 -- 172.21.15.131:0/3757898075 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7efe6800bca0 con 0x7efe800ad250 2024-05-26T22:54:38.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.433+0000 7efe86800640 1 --2- 172.21.15.131:0/3757898075 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efe68004d40 0x7efe680051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:38.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.433+0000 7efe85e00640 1 --2- 172.21.15.131:0/3757898075 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efe680039e0 0x7efe68003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:38.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.433+0000 7efe86800640 1 -- 172.21.15.131:0/3757898075 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efe800ad250 msgr2=0x7efe68004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:38.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.433+0000 7efe86800640 1 --2- 172.21.15.131:0/3757898075 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efe800ad250 0x7efe68004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:38.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.433+0000 7efe86800640 1 -- 172.21.15.131:0/3757898075 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efe680039e0 msgr2=0x7efe68003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:38.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.433+0000 7efe86800640 1 --2- 172.21.15.131:0/3757898075 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efe680039e0 0x7efe68003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:38.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.433+0000 7efe86800640 1 -- 172.21.15.131:0/3757898075 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe68003520 con 0x7efe68004d40 2024-05-26T22:54:38.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.433+0000 7efe85e00640 1 --2- 172.21.15.131:0/3757898075 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efe680039e0 0x7efe68003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:54:38.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.434+0000 7efe86800640 1 --2- 172.21.15.131:0/3757898075 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efe68004d40 0x7efe680051c0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7efe7c00a830 tx=0x7efe68005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4a911f9bd14e6c5 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:38.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.434+0000 7efe84a00640 1 -- 172.21.15.131:0/3757898075 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe7c00a480 con 0x7efe68004d40 2024-05-26T22:54:38.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.434+0000 7efe84a00640 1 -- 172.21.15.131:0/3757898075 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe7c02f070 con 0x7efe68004d40 2024-05-26T22:54:38.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.434+0000 7efe84a00640 1 -- 172.21.15.131:0/3757898075 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe7c02bcb0 con 0x7efe68004d40 2024-05-26T22:54:38.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.435+0000 7efe88800640 1 -- 172.21.15.131:0/3757898075 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efe68004d40 msgr2=0x7efe680051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:38.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.435+0000 7efe88800640 1 --2- 172.21.15.131:0/3757898075 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efe68004d40 0x7efe680051c0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7efe7c00a830 tx=0x7efe68005830 comp rx=0 tx=0).stop 2024-05-26T22:54:38.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.435+0000 7efe86800640 1 -- 172.21.15.131:0/3757898075 reap_dead start 2024-05-26T22:54:38.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.435+0000 7efe88800640 1 -- 172.21.15.131:0/3757898075 shutdown_connections 2024-05-26T22:54:38.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.435+0000 7efe88800640 1 -- 172.21.15.131:0/3757898075 >> 172.21.15.131:0/3757898075 conn(0x7efe800b1e90 msgr2=0x7efe800b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:38.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.435+0000 7efe88800640 1 -- 172.21.15.131:0/3757898075 shutdown_connections 2024-05-26T22:54:38.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.435+0000 7efe88800640 1 -- 172.21.15.131:0/3757898075 wait complete. 2024-05-26T22:54:38.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.437+0000 7efe88800640 1 Processor -- start 2024-05-26T22:54:38.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.437+0000 7efe88800640 1 -- start start 2024-05-26T22:54:38.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.438+0000 7efe88800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efe68004d40 0x7efe801464c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:38.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.438+0000 7efe88800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efe8015b230 0x7efe8015b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:38.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.438+0000 7efe88800640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efe80071690 0x7efe80071b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:38.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.438+0000 7efe88800640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7efe8015c900 con 0x7efe8015b230 2024-05-26T22:54:38.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.438+0000 7efe88800640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7efe8015c780 con 0x7efe80071690 2024-05-26T22:54:38.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.438+0000 7efe88800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7efe8006f200 con 0x7efe68004d40 2024-05-26T22:54:38.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.439+0000 7efe85400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efe8015b230 0x7efe8015b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:38.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.439+0000 7efe85400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efe8015b230 0x7efe8015b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44244/0 (socket says 172.21.15.131:44244) 2024-05-26T22:54:38.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.439+0000 7efe85400640 1 -- 172.21.15.131:0/3156058313 learned_addr learned my addr 172.21.15.131:0/3156058313 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:54:38.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.439+0000 7efe85400640 1 -- 172.21.15.131:0/3156058313 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efe68004d40 msgr2=0x7efe801464c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:38.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.439+0000 7efe85400640 1 --2- 172.21.15.131:0/3156058313 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efe68004d40 0x7efe801464c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:38.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.439+0000 7efe85400640 1 -- 172.21.15.131:0/3156058313 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efe80071690 msgr2=0x7efe80071b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:38.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.439+0000 7efe85400640 1 --2- 172.21.15.131:0/3156058313 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efe80071690 0x7efe80071b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:38.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.439+0000 7efe85400640 1 -- 172.21.15.131:0/3156058313 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe80072430 con 0x7efe8015b230 2024-05-26T22:54:38.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.439+0000 7efe85400640 1 --2- 172.21.15.131:0/3156058313 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efe8015b230 0x7efe8015b670 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7efe6c00e7f0 tx=0x7efe6c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:38.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.439+0000 7efe7a000640 1 -- 172.21.15.131:0/3156058313 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe6c017070 con 0x7efe8015b230 2024-05-26T22:54:38.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.440+0000 7efe7a000640 1 -- 172.21.15.131:0/3156058313 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe6c00c8c0 con 0x7efe8015b230 2024-05-26T22:54:38.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.440+0000 7efe7a000640 1 -- 172.21.15.131:0/3156058313 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe6c00eea0 con 0x7efe8015b230 2024-05-26T22:54:38.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.440+0000 7efe88800640 1 -- 172.21.15.131:0/3156058313 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe80072710 con 0x7efe8015b230 2024-05-26T22:54:38.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.440+0000 7efe88800640 1 -- 172.21.15.131:0/3156058313 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe80072cd0 con 0x7efe8015b230 2024-05-26T22:54:38.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.441+0000 7efe88800640 1 -- 172.21.15.131:0/3156058313 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe800694e0 con 0x7efe8015b230 2024-05-26T22:54:38.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.442+0000 7efe7a000640 1 -- 172.21.15.131:0/3156058313 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7efe6c01a070 con 0x7efe8015b230 2024-05-26T22:54:38.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.443+0000 7efe7a000640 1 --2- 172.21.15.131:0/3156058313 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7efe6006be30 0x7efe6006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:38.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.443+0000 7efe7a000640 1 -- 172.21.15.131:0/3156058313 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5738+0+0 (secure 0 0 0) 0x7efe6c08ac80 con 0x7efe8015b230 2024-05-26T22:54:38.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.443+0000 7efe85e00640 1 --2- 172.21.15.131:0/3156058313 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7efe6006be30 0x7efe6006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:38.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.444+0000 7efe85e00640 1 --2- 172.21.15.131:0/3156058313 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7efe6006be30 0x7efe6006e2f0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7efe740016d0 tx=0x7efe74009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:38.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.447+0000 7efe7a000640 1 -- 172.21.15.131:0/3156058313 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7efe6c059600 con 0x7efe8015b230 2024-05-26T22:54:38.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.648+0000 7efe88800640 1 -- 172.21.15.131:0/3156058313 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efe8006e740 con 0x7efe8015b230 2024-05-26T22:54:38.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.650+0000 7efe7a000640 1 -- 172.21.15.131:0/3156058313 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+16734 (secure 0 0 0) 0x7efe6c058f00 con 0x7efe8015b230 2024-05-26T22:54:38.650 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:54:38.650 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":84,"fsid":"46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f","created":"2024-05-26T22:47:57.288905+0000","modified":"2024-05-26T22:54:07.109294+0000","last_up_change":"2024-05-26T22:48:09.602207+0000","last_in_change":"2024-05-26T22:48:00.687341+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-26T22:48:10.463339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-05-26T22:53:50.625629+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-05-26T22:53:51.864320+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"814eaf62-a18d-494b-8e27-fd8fb88adea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6801","nonce":2350940051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6803","nonce":2350940051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6807","nonce":2350940051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6805","nonce":2350940051}]},"public_addr":"172.21.15.131:6801/2350940051","cluster_addr":"172.21.15.131:6803/2350940051","heartbeat_back_addr":"172.21.15.131:6807/2350940051","heartbeat_front_addr":"172.21.15.131:6805/2350940051","state":["exists","up"]},{"osd":1,"uuid":"42e66287-f6c6-4e26-af3c-6dfaa3536d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6825","nonce":1525528247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6827","nonce":1525528247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6831","nonce":1525528247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6829","nonce":1525528247}]},"public_addr":"172.21.15.131:6825/1525528247","cluster_addr":"172.21.15.131:6827/1525528247","heartbeat_back_addr":"172.21.15.131:6831/1525528247","heartbeat_front_addr":"172.21.15.131:6829/1525528247","state":["exists","up"]},{"osd":2,"uuid":"0da34673-04a2-4fd0-b57a-0ff2b679b043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6817","nonce":3627842210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6819","nonce":3627842210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6823","nonce":3627842210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6821","nonce":3627842210}]},"public_addr":"172.21.15.131:6817/3627842210","cluster_addr":"172.21.15.131:6819/3627842210","heartbeat_back_addr":"172.21.15.131:6823/3627842210","heartbeat_front_addr":"172.21.15.131:6821/3627842210","state":["exists","up"]},{"osd":3,"uuid":"cf117a27-d6b2-43d3-ae52-b47988bf8eed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6809","nonce":350790591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6811","nonce":350790591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6815","nonce":350790591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6813","nonce":350790591}]},"public_addr":"172.21.15.131:6809/350790591","cluster_addr":"172.21.15.131:6811/350790591","heartbeat_back_addr":"172.21.15.131:6815/350790591","heartbeat_front_addr":"172.21.15.131:6813/350790591","state":["exists","up"]},{"osd":4,"uuid":"a092ccfc-be89-4a84-88d5-0f781fa715eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6816","nonce":244560995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6819","nonce":244560995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6823","nonce":244560995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6821","nonce":244560995}]},"public_addr":"172.21.15.187:6816/244560995","cluster_addr":"172.21.15.187:6819/244560995","heartbeat_back_addr":"172.21.15.187:6823/244560995","heartbeat_front_addr":"172.21.15.187:6821/244560995","state":["exists","up"]},{"osd":5,"uuid":"7513ba69-dd00-4a35-b190-9e8cc3b2bdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6801","nonce":324619398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6803","nonce":324619398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6812","nonce":324619398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6807","nonce":324619398}]},"public_addr":"172.21.15.187:6801/324619398","cluster_addr":"172.21.15.187:6803/324619398","heartbeat_back_addr":"172.21.15.187:6812/324619398","heartbeat_front_addr":"172.21.15.187:6807/324619398","state":["exists","up"]},{"osd":6,"uuid":"50081810-c093-4d2f-a4d7-b7c695c5400d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6806","nonce":1391347786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6809","nonce":1391347786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6818","nonce":1391347786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6813","nonce":1391347786}]},"public_addr":"172.21.15.187:6806/1391347786","cluster_addr":"172.21.15.187:6809/1391347786","heartbeat_back_addr":"172.21.15.187:6818/1391347786","heartbeat_front_addr":"172.21.15.187:6813/1391347786","state":["exists","up"]},{"osd":7,"uuid":"775dae05-54f5-4ad4-bbce-92863c6ed2ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6825","nonce":1220065622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6827","nonce":1220065622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6831","nonce":1220065622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6829","nonce":1220065622}]},"public_addr":"172.21.15.187:6825/1220065622","cluster_addr":"172.21.15.187:6827/1220065622","heartbeat_back_addr":"172.21.15.187:6831/1220065622","heartbeat_front_addr":"172.21.15.187:6829/1220065622","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:05.983520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.299516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.067323+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.307748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.018742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.258775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.010912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.295355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.38","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-26T22:54:38.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.653+0000 7efe88800640 1 -- 172.21.15.131:0/3156058313 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7efe6006be30 msgr2=0x7efe6006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:38.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.653+0000 7efe88800640 1 --2- 172.21.15.131:0/3156058313 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7efe6006be30 0x7efe6006e2f0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7efe740016d0 tx=0x7efe74009040 comp rx=0 tx=0).stop 2024-05-26T22:54:38.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.653+0000 7efe88800640 1 -- 172.21.15.131:0/3156058313 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efe8015b230 msgr2=0x7efe8015b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:38.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.653+0000 7efe88800640 1 --2- 172.21.15.131:0/3156058313 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efe8015b230 0x7efe8015b670 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7efe6c00e7f0 tx=0x7efe6c00ecb0 comp rx=0 tx=0).stop 2024-05-26T22:54:38.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.653+0000 7efe88800640 1 -- 172.21.15.131:0/3156058313 shutdown_connections 2024-05-26T22:54:38.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.653+0000 7efe88800640 1 --2- 172.21.15.131:0/3156058313 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7efe6006be30 0x7efe6006e2f0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:38.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.653+0000 7efe88800640 1 --2- 172.21.15.131:0/3156058313 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efe80071690 0x7efe80071b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:38.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.653+0000 7efe88800640 1 --2- 172.21.15.131:0/3156058313 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efe8015b230 0x7efe8015b670 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:38.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.653+0000 7efe88800640 1 --2- 172.21.15.131:0/3156058313 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efe68004d40 0x7efe801464c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:38.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.653+0000 7efe88800640 1 -- 172.21.15.131:0/3156058313 >> 172.21.15.131:0/3156058313 conn(0x7efe800b1e90 msgr2=0x7efe8015e670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:38.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.654+0000 7efe88800640 1 -- 172.21.15.131:0/3156058313 shutdown_connections 2024-05-26T22:54:38.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.654+0000 7efe88800640 1 -- 172.21.15.131:0/3156058313 wait complete. 2024-05-26T22:54:38.673 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-26T22:48:10.463339+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-26T22:53:50.625629+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-05-26T22:53:51.864320+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-26T22:54:38.673 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-26T22:54:38.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.875+0000 7fcd6de00640 1 Processor -- start 2024-05-26T22:54:38.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.875+0000 7fcd6de00640 1 -- start start 2024-05-26T22:54:38.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.877+0000 7fcd6de00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fcd6806dd30 0x7fcd6806e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:38.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.877+0000 7fcd6de00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd6815c1c0 con 0x7fcd680725a0 2024-05-26T22:54:38.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.877+0000 7fcd6de00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd6815c4a0 con 0x7fcd68155770 2024-05-26T22:54:38.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.877+0000 7fcd6de00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fcd68071da0 con 0x7fcd6806dd30 2024-05-26T22:54:38.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.877+0000 7fcd67400640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fcd680725a0 0x7fcd680729a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:58160/0 (socket says 172.21.15.131:58160) 2024-05-26T22:54:38.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.877+0000 7fcd67400640 1 -- 172.21.15.131:0/1838609690 learned_addr learned my addr 172.21.15.131:0/1838609690 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:54:38.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.878+0000 7fcd5e000640 1 --2- 172.21.15.131:0/1838609690 >> v2:172.21.15.187:3300/0 conn(0x7fcd6806dd30 0x7fcd6806e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:38.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.878+0000 7fcd66000640 1 -- 172.21.15.131:0/1838609690 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1066477781 0 0) 0x7fcd6815c1c0 con 0x7fcd680725a0 2024-05-26T22:54:38.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.878+0000 7fcd66000640 1 -- 172.21.15.131:0/1838609690 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcd4c005730 con 0x7fcd680725a0 2024-05-26T22:54:38.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.878+0000 7fcd66000640 1 -- 172.21.15.131:0/1838609690 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1831365014 0 0) 0x7fcd4c005730 con 0x7fcd680725a0 2024-05-26T22:54:38.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.878+0000 7fcd66000640 1 -- 172.21.15.131:0/1838609690 >> v2:172.21.15.187:3300/0 conn(0x7fcd6806dd30 msgr2=0x7fcd6806e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:38.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.878+0000 7fcd66000640 1 --2- 172.21.15.131:0/1838609690 >> v2:172.21.15.187:3300/0 conn(0x7fcd6806dd30 0x7fcd6806e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:38.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.878+0000 7fcd66000640 1 -- 172.21.15.131:0/1838609690 >> v1:172.21.15.187:6789/0 conn(0x7fcd68155770 legacy=0x7fcd6815b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:38.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.878+0000 7fcd66000640 1 -- 172.21.15.131:0/1838609690 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd68072080 con 0x7fcd680725a0 2024-05-26T22:54:38.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.878+0000 7fcd66000640 1 -- 172.21.15.131:0/1838609690 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7fcd600019f0 con 0x7fcd680725a0 2024-05-26T22:54:38.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.878+0000 7fcd5e000640 1 --2- 172.21.15.131:0/1838609690 >> v2:172.21.15.187:3300/0 conn(0x7fcd6806dd30 0x7fcd6806e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-26T22:54:38.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.878+0000 7fcd66000640 1 -- 172.21.15.131:0/1838609690 >> v1:172.21.15.131:6789/0 conn(0x7fcd680725a0 legacy=0x7fcd680729a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:38.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.878+0000 7fcd66000640 1 --2- 172.21.15.131:0/1838609690 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcd4c005bc0 0x7fcd4c005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:38.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.878+0000 7fcd66000640 1 --2- 172.21.15.131:0/1838609690 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcd4c006a10 0x7fcd4c006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:38.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.878+0000 7fcd66000640 1 --2- 172.21.15.131:0/1838609690 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd4c0073d0 0x7fcd4c00dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:38.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.878+0000 7fcd66000640 1 -- 172.21.15.131:0/1838609690 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fcd68071da0 con 0x7fcd4c0073d0 2024-05-26T22:54:38.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.878+0000 7fcd66000640 1 -- 172.21.15.131:0/1838609690 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fcd4c00e300 con 0x7fcd4c005bc0 2024-05-26T22:54:38.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.878+0000 7fcd66000640 1 -- 172.21.15.131:0/1838609690 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fcd4c00e500 con 0x7fcd4c006a10 2024-05-26T22:54:38.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.879+0000 7fcd67400640 1 --2- 172.21.15.131:0/1838609690 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd4c0073d0 0x7fcd4c00dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:38.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.879+0000 7fcd66a00640 1 --2- 172.21.15.131:0/1838609690 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcd4c005bc0 0x7fcd4c005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:38.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.879+0000 7fcd67400640 1 -- 172.21.15.131:0/1838609690 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcd4c006a10 msgr2=0x7fcd4c006e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:38.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.879+0000 7fcd67400640 1 --2- 172.21.15.131:0/1838609690 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcd4c006a10 0x7fcd4c006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:38.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.879+0000 7fcd67400640 1 -- 172.21.15.131:0/1838609690 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcd4c005bc0 msgr2=0x7fcd4c005fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:38.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.879+0000 7fcd67400640 1 --2- 172.21.15.131:0/1838609690 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcd4c005bc0 0x7fcd4c005fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:38.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.879+0000 7fcd67400640 1 -- 172.21.15.131:0/1838609690 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd4c005730 con 0x7fcd4c0073d0 2024-05-26T22:54:38.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.880+0000 7fcd67400640 1 -- 172.21.15.131:0/1838609690 reap_dead start 2024-05-26T22:54:38.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.880+0000 7fcd67400640 1 --2- 172.21.15.131:0/1838609690 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd4c0073d0 0x7fcd4c00dbd0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7fcd6000a840 tx=0x7fcd4c007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7bd770830b0ddaf server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:38.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.880+0000 7fcd66000640 1 -- 172.21.15.131:0/1838609690 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd6000a460 con 0x7fcd4c0073d0 2024-05-26T22:54:38.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.880+0000 7fcd66000640 1 -- 172.21.15.131:0/1838609690 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd6002f070 con 0x7fcd4c0073d0 2024-05-26T22:54:38.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.880+0000 7fcd66000640 1 -- 172.21.15.131:0/1838609690 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd6002bcb0 con 0x7fcd4c0073d0 2024-05-26T22:54:38.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.881+0000 7fcd6de00640 1 -- 172.21.15.131:0/1838609690 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd4c0073d0 msgr2=0x7fcd4c00dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:38.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.881+0000 7fcd6de00640 1 --2- 172.21.15.131:0/1838609690 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd4c0073d0 0x7fcd4c00dbd0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7fcd6000a840 tx=0x7fcd4c007a40 comp rx=0 tx=0).stop 2024-05-26T22:54:38.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.881+0000 7fcd6de00640 1 -- 172.21.15.131:0/1838609690 shutdown_connections 2024-05-26T22:54:38.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.881+0000 7fcd6de00640 1 --2- 172.21.15.131:0/1838609690 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd4c0073d0 0x7fcd4c00dbd0 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:38.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.881+0000 7fcd6de00640 1 -- 172.21.15.131:0/1838609690 >> 172.21.15.131:0/1838609690 conn(0x7fcd6806bc90 msgr2=0x7fcd680b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:38.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.881+0000 7fcd6de00640 1 -- 172.21.15.131:0/1838609690 shutdown_connections 2024-05-26T22:54:38.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.881+0000 7fcd6de00640 1 -- 172.21.15.131:0/1838609690 wait complete. 2024-05-26T22:54:38.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.883+0000 7fcd6de00640 1 Processor -- start 2024-05-26T22:54:38.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.883+0000 7fcd6de00640 1 -- start start 2024-05-26T22:54:38.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.884+0000 7fcd6de00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcd4c0073d0 0x7fcd681549b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:38.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.884+0000 7fcd6de00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcd6809b6c0 0x7fcd681f4540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:38.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.884+0000 7fcd6de00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd681f4a80 0x7fcd681e9430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:38.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.884+0000 7fcd6de00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fcd6809de00 con 0x7fcd681f4a80 2024-05-26T22:54:38.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.884+0000 7fcd6de00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fcd6815c8c0 con 0x7fcd4c0073d0 2024-05-26T22:54:38.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.884+0000 7fcd6de00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fcd6809df80 con 0x7fcd6809b6c0 2024-05-26T22:54:38.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.884+0000 7fcd67400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd681f4a80 0x7fcd681e9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:38.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.884+0000 7fcd67400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd681f4a80 0x7fcd681e9430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44262/0 (socket says 172.21.15.131:44262) 2024-05-26T22:54:38.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.884+0000 7fcd67400640 1 -- 172.21.15.131:0/3489415378 learned_addr learned my addr 172.21.15.131:0/3489415378 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:54:38.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.885+0000 7fcd5e000640 1 --2- 172.21.15.131:0/3489415378 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcd6809b6c0 0x7fcd681f4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:38.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.885+0000 7fcd67400640 1 -- 172.21.15.131:0/3489415378 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcd6809b6c0 msgr2=0x7fcd681f4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:38.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.885+0000 7fcd67400640 1 --2- 172.21.15.131:0/3489415378 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcd6809b6c0 0x7fcd681f4540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:38.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.885+0000 7fcd67400640 1 -- 172.21.15.131:0/3489415378 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcd4c0073d0 msgr2=0x7fcd681549b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:38.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.885+0000 7fcd67400640 1 --2- 172.21.15.131:0/3489415378 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcd4c0073d0 0x7fcd681549b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:38.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.885+0000 7fcd67400640 1 -- 172.21.15.131:0/3489415378 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd6809bc50 con 0x7fcd681f4a80 2024-05-26T22:54:38.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.885+0000 7fcd67400640 1 --2- 172.21.15.131:0/3489415378 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd681f4a80 0x7fcd681e9430 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7fcd60000f80 tx=0x7fcd6815c670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:38.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.885+0000 7fcd5fe00640 1 -- 172.21.15.131:0/3489415378 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd60042070 con 0x7fcd681f4a80 2024-05-26T22:54:38.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.885+0000 7fcd5fe00640 1 -- 172.21.15.131:0/3489415378 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd6003a070 con 0x7fcd681f4a80 2024-05-26T22:54:38.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.885+0000 7fcd6de00640 1 -- 172.21.15.131:0/3489415378 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd681e9ea0 con 0x7fcd681f4a80 2024-05-26T22:54:38.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.885+0000 7fcd5fe00640 1 -- 172.21.15.131:0/3489415378 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd60001fb0 con 0x7fcd681f4a80 2024-05-26T22:54:38.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.885+0000 7fcd6de00640 1 -- 172.21.15.131:0/3489415378 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd6809c1b0 con 0x7fcd681f4a80 2024-05-26T22:54:38.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.886+0000 7fcd6de00640 1 -- 172.21.15.131:0/3489415378 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd680695c0 con 0x7fcd681f4a80 2024-05-26T22:54:38.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.887+0000 7fcd5fe00640 1 -- 172.21.15.131:0/3489415378 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcd60045070 con 0x7fcd681f4a80 2024-05-26T22:54:38.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.889+0000 7fcd5fe00640 1 --2- 172.21.15.131:0/3489415378 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fcd4406be30 0x7fcd4406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:38.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.889+0000 7fcd5fe00640 1 -- 172.21.15.131:0/3489415378 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5738+0+0 (secure 0 0 0) 0x7fcd600aa830 con 0x7fcd681f4a80 2024-05-26T22:54:38.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.889+0000 7fcd66a00640 1 --2- 172.21.15.131:0/3489415378 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fcd4406be30 0x7fcd4406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:38.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.890+0000 7fcd66a00640 1 --2- 172.21.15.131:0/3489415378 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fcd4406be30 0x7fcd4406e2f0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fcd58000f60 tx=0x7fcd58009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:38.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:38.893+0000 7fcd5fe00640 1 -- 172.21.15.131:0/3489415378 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fcd6809bc50 con 0x7fcd681f4a80 2024-05-26T22:54:39.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.087+0000 7fcd6de00640 1 -- 172.21.15.131:0/3489415378 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fcd6815a700 con 0x7fcd681f4a80 2024-05-26T22:54:39.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.088+0000 7fcd5fe00640 1 -- 172.21.15.131:0/3489415378 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) ==== 93+0+10 (secure 0 0 0) 0x7fcd600791b0 con 0x7fcd681f4a80 2024-05-26T22:54:39.088 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2024-05-26T22:54:39.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.092+0000 7fcd6de00640 1 -- 172.21.15.131:0/3489415378 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fcd4406be30 msgr2=0x7fcd4406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:39.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.092+0000 7fcd6de00640 1 --2- 172.21.15.131:0/3489415378 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fcd4406be30 0x7fcd4406e2f0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fcd58000f60 tx=0x7fcd58009040 comp rx=0 tx=0).stop 2024-05-26T22:54:39.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.092+0000 7fcd6de00640 1 -- 172.21.15.131:0/3489415378 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd681f4a80 msgr2=0x7fcd681e9430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:39.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.092+0000 7fcd6de00640 1 --2- 172.21.15.131:0/3489415378 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd681f4a80 0x7fcd681e9430 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7fcd60000f80 tx=0x7fcd6815c670 comp rx=0 tx=0).stop 2024-05-26T22:54:39.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.092+0000 7fcd6de00640 1 -- 172.21.15.131:0/3489415378 shutdown_connections 2024-05-26T22:54:39.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.092+0000 7fcd6de00640 1 --2- 172.21.15.131:0/3489415378 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcd6809b6c0 0x7fcd681f4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:39.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.092+0000 7fcd6de00640 1 --2- 172.21.15.131:0/3489415378 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fcd4406be30 0x7fcd4406e2f0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:39.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.092+0000 7fcd6de00640 1 --2- 172.21.15.131:0/3489415378 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd681f4a80 0x7fcd681e9430 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:39.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.092+0000 7fcd6de00640 1 --2- 172.21.15.131:0/3489415378 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcd4c0073d0 0x7fcd681549b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:39.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.092+0000 7fcd6de00640 1 -- 172.21.15.131:0/3489415378 >> 172.21.15.131:0/3489415378 conn(0x7fcd6806bc90 msgr2=0x7fcd680cf340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:39.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.092+0000 7fcd6de00640 1 -- 172.21.15.131:0/3489415378 shutdown_connections 2024-05-26T22:54:39.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.092+0000 7fcd6de00640 1 -- 172.21.15.131:0/3489415378 wait complete. 2024-05-26T22:54:39.112 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-26T22:54:39.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.315+0000 7fcd82000640 1 Processor -- start 2024-05-26T22:54:39.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.315+0000 7fcd82000640 1 -- start start 2024-05-26T22:54:39.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.317+0000 7fcd82000640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fcd7c1419c0 0x7fcd7c141e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:39.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.317+0000 7fcd82000640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd7c1427b0 con 0x7fcd7c13f210 2024-05-26T22:54:39.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.317+0000 7fcd82000640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd7c13f930 con 0x7fcd7c1403f0 2024-05-26T22:54:39.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.317+0000 7fcd82000640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fcd7c142340 con 0x7fcd7c1419c0 2024-05-26T22:54:39.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.317+0000 7fcd7b400640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fcd7c13f210 0x7fcd7c15fd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:58162/0 (socket says 172.21.15.131:58162) 2024-05-26T22:54:39.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.317+0000 7fcd7b400640 1 -- 172.21.15.131:0/3208834608 learned_addr learned my addr 172.21.15.131:0/3208834608 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:54:39.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.317+0000 7fcd7aa00640 1 --2- 172.21.15.131:0/3208834608 >> v2:172.21.15.131:3300/0 conn(0x7fcd7c1419c0 0x7fcd7c141e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:39.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.317+0000 7fcd7a000640 1 -- 172.21.15.131:0/3208834608 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2850469766 0 0) 0x7fcd7c1427b0 con 0x7fcd7c13f210 2024-05-26T22:54:39.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.318+0000 7fcd7a000640 1 -- 172.21.15.131:0/3208834608 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcd60003520 con 0x7fcd7c13f210 2024-05-26T22:54:39.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.318+0000 7fcd7aa00640 1 -- 172.21.15.131:0/3208834608 >> v1:172.21.15.187:6789/0 conn(0x7fcd7c1403f0 legacy=0x7fcd7c140810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:39.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.318+0000 7fcd7aa00640 1 -- 172.21.15.131:0/3208834608 >> v1:172.21.15.131:6789/0 conn(0x7fcd7c13f210 legacy=0x7fcd7c15fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:39.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.318+0000 7fcd7aa00640 1 -- 172.21.15.131:0/3208834608 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd7c13f5f0 con 0x7fcd7c1419c0 2024-05-26T22:54:39.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.319+0000 7fcd7aa00640 1 --2- 172.21.15.131:0/3208834608 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd7c1419c0 0x7fcd7c141e00 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fcd680097a0 tx=0x7fcd6802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98aa2cdb3627b217 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:39.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.319+0000 7fcd7a000640 1 -- 172.21.15.131:0/3208834608 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd68037070 con 0x7fcd7c1419c0 2024-05-26T22:54:39.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.319+0000 7fcd7a000640 1 -- 172.21.15.131:0/3208834608 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd68032440 con 0x7fcd7c1419c0 2024-05-26T22:54:39.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.319+0000 7fcd7a000640 1 -- 172.21.15.131:0/3208834608 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd68032810 con 0x7fcd7c1419c0 2024-05-26T22:54:39.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.320+0000 7fcd82000640 1 -- 172.21.15.131:0/3208834608 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd7c1419c0 msgr2=0x7fcd7c141e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:39.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.320+0000 7fcd82000640 1 --2- 172.21.15.131:0/3208834608 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd7c1419c0 0x7fcd7c141e00 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fcd680097a0 tx=0x7fcd6802a1e0 comp rx=0 tx=0).stop 2024-05-26T22:54:39.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.320+0000 7fcd82000640 1 -- 172.21.15.131:0/3208834608 shutdown_connections 2024-05-26T22:54:39.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.320+0000 7fcd82000640 1 --2- 172.21.15.131:0/3208834608 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd7c1419c0 0x7fcd7c141e00 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:39.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.320+0000 7fcd82000640 1 -- 172.21.15.131:0/3208834608 >> 172.21.15.131:0/3208834608 conn(0x7fcd7c0b3e00 msgr2=0x7fcd7c0b41e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:39.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.320+0000 7fcd82000640 1 -- 172.21.15.131:0/3208834608 shutdown_connections 2024-05-26T22:54:39.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.320+0000 7fcd82000640 1 -- 172.21.15.131:0/3208834608 wait complete. 2024-05-26T22:54:39.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.322+0000 7fcd82000640 1 Processor -- start 2024-05-26T22:54:39.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.322+0000 7fcd82000640 1 -- start start 2024-05-26T22:54:39.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.323+0000 7fcd82000640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcd7c13f210 0x7fcd7c06de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:39.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.323+0000 7fcd82000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcd7c1403f0 0x7fcd7c1f7450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:39.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.323+0000 7fcd82000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd7c1419c0 0x7fcd7c06c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:39.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.323+0000 7fcd82000640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fcd7c13fbf0 con 0x7fcd7c1419c0 2024-05-26T22:54:39.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.323+0000 7fcd82000640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fcd7c160b80 con 0x7fcd7c13f210 2024-05-26T22:54:39.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.323+0000 7fcd82000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fcd7c13fed0 con 0x7fcd7c1403f0 2024-05-26T22:54:39.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.323+0000 7fcd7be00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd7c1419c0 0x7fcd7c06c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:39.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.323+0000 7fcd7be00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd7c1419c0 0x7fcd7c06c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44282/0 (socket says 172.21.15.131:44282) 2024-05-26T22:54:39.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.323+0000 7fcd7be00640 1 -- 172.21.15.131:0/2901650874 learned_addr learned my addr 172.21.15.131:0/2901650874 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:54:39.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.323+0000 7fcd7b400640 1 --2- 172.21.15.131:0/2901650874 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcd7c13f210 0x7fcd7c06de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:39.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.323+0000 7fcd7be00640 1 -- 172.21.15.131:0/2901650874 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcd7c1403f0 msgr2=0x7fcd7c1f7450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:39.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.323+0000 7fcd7be00640 1 --2- 172.21.15.131:0/2901650874 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcd7c1403f0 0x7fcd7c1f7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:39.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.323+0000 7fcd7be00640 1 -- 172.21.15.131:0/2901650874 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcd7c13f210 msgr2=0x7fcd7c06de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:39.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.323+0000 7fcd7be00640 1 --2- 172.21.15.131:0/2901650874 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcd7c13f210 0x7fcd7c06de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:39.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.323+0000 7fcd7be00640 1 -- 172.21.15.131:0/2901650874 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd7c06cdc0 con 0x7fcd7c1419c0 2024-05-26T22:54:39.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.324+0000 7fcd7be00640 1 --2- 172.21.15.131:0/2901650874 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd7c1419c0 0x7fcd7c06c420 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7fcd7400c780 tx=0x7fcd7400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:39.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.324+0000 7fcd73400640 1 -- 172.21.15.131:0/2901650874 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd74004b60 con 0x7fcd7c1419c0 2024-05-26T22:54:39.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.324+0000 7fcd73400640 1 -- 172.21.15.131:0/2901650874 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd74004d00 con 0x7fcd7c1419c0 2024-05-26T22:54:39.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.324+0000 7fcd82000640 1 -- 172.21.15.131:0/2901650874 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd7c06d0a0 con 0x7fcd7c1419c0 2024-05-26T22:54:39.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.324+0000 7fcd73400640 1 -- 172.21.15.131:0/2901650874 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd7400aa90 con 0x7fcd7c1419c0 2024-05-26T22:54:39.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.324+0000 7fcd82000640 1 -- 172.21.15.131:0/2901650874 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd7c06d660 con 0x7fcd7c1419c0 2024-05-26T22:54:39.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.325+0000 7fcd82000640 1 -- 172.21.15.131:0/2901650874 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd7c0695c0 con 0x7fcd7c1419c0 2024-05-26T22:54:39.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.326+0000 7fcd73400640 1 -- 172.21.15.131:0/2901650874 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcd7400ac30 con 0x7fcd7c1419c0 2024-05-26T22:54:39.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.327+0000 7fcd73400640 1 --2- 172.21.15.131:0/2901650874 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fcd5c06be30 0x7fcd5c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:39.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.327+0000 7fcd73400640 1 -- 172.21.15.131:0/2901650874 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5738+0+0 (secure 0 0 0) 0x7fcd7408b120 con 0x7fcd7c1419c0 2024-05-26T22:54:39.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.327+0000 7fcd7b400640 1 --2- 172.21.15.131:0/2901650874 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fcd5c06be30 0x7fcd5c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:39.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.328+0000 7fcd7b400640 1 --2- 172.21.15.131:0/2901650874 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fcd5c06be30 0x7fcd5c06e2f0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c000f60 tx=0x7fcd6c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:39.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.332+0000 7fcd73400640 1 -- 172.21.15.131:0/2901650874 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fcd74020070 con 0x7fcd7c1419c0 2024-05-26T22:54:39.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.529+0000 7fcd82000640 1 -- 172.21.15.131:0/2901650874 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fcd7c0b1a50 con 0x7fcd7c1419c0 2024-05-26T22:54:39.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.530+0000 7fcd73400640 1 -- 172.21.15.131:0/2901650874 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v84) ==== 104+0+11 (secure 0 0 0) 0x7fcd74059aa0 con 0x7fcd7c1419c0 2024-05-26T22:54:39.530 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-05-26T22:54:39.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.532+0000 7fcd82000640 1 -- 172.21.15.131:0/2901650874 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fcd5c06be30 msgr2=0x7fcd5c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:39.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.532+0000 7fcd82000640 1 --2- 172.21.15.131:0/2901650874 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fcd5c06be30 0x7fcd5c06e2f0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c000f60 tx=0x7fcd6c009040 comp rx=0 tx=0).stop 2024-05-26T22:54:39.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.532+0000 7fcd82000640 1 -- 172.21.15.131:0/2901650874 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd7c1419c0 msgr2=0x7fcd7c06c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:39.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.532+0000 7fcd82000640 1 --2- 172.21.15.131:0/2901650874 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd7c1419c0 0x7fcd7c06c420 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7fcd7400c780 tx=0x7fcd7400cc40 comp rx=0 tx=0).stop 2024-05-26T22:54:39.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.532+0000 7fcd82000640 1 -- 172.21.15.131:0/2901650874 shutdown_connections 2024-05-26T22:54:39.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.532+0000 7fcd82000640 1 --2- 172.21.15.131:0/2901650874 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcd7c1403f0 0x7fcd7c1f7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:39.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.532+0000 7fcd82000640 1 --2- 172.21.15.131:0/2901650874 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fcd5c06be30 0x7fcd5c06e2f0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:39.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.532+0000 7fcd82000640 1 --2- 172.21.15.131:0/2901650874 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd7c1419c0 0x7fcd7c06c420 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:39.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.532+0000 7fcd82000640 1 --2- 172.21.15.131:0/2901650874 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcd7c13f210 0x7fcd7c06de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:39.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.532+0000 7fcd82000640 1 -- 172.21.15.131:0/2901650874 >> 172.21.15.131:0/2901650874 conn(0x7fcd7c0b3e00 msgr2=0x7fcd7c13b9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:39.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.533+0000 7fcd82000640 1 -- 172.21.15.131:0/2901650874 shutdown_connections 2024-05-26T22:54:39.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.533+0000 7fcd82000640 1 -- 172.21.15.131:0/2901650874 wait complete. 2024-05-26T22:54:39.550 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-26T22:54:39.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.742+0000 7f6072a00640 1 Processor -- start 2024-05-26T22:54:39.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.742+0000 7f6072a00640 1 -- start start 2024-05-26T22:54:39.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.744+0000 7f6072a00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f606c0b4050 0x7f606c143c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:39.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.744+0000 7f6072a00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f606c144880 con 0x7f606c1416d0 2024-05-26T22:54:39.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.744+0000 7f6072a00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f606c06b3b0 con 0x7f606c141c10 2024-05-26T22:54:39.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.744+0000 7f6072a00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f606c144b60 con 0x7f606c0b4050 2024-05-26T22:54:39.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.744+0000 7f6071600640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f606c1416d0 0x7f606c0b3940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:58172/0 (socket says 172.21.15.131:58172) 2024-05-26T22:54:39.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.744+0000 7f6071600640 1 -- 172.21.15.131:0/359199857 learned_addr learned my addr 172.21.15.131:0/359199857 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:54:39.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.744+0000 7f6070c00640 1 --2- 172.21.15.131:0/359199857 >> v2:172.21.15.131:3300/0 conn(0x7f606c0b4050 0x7f606c143c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:39.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.744+0000 7f606be00640 1 -- 172.21.15.131:0/359199857 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1136144485 0 0) 0x7f606c144880 con 0x7f606c1416d0 2024-05-26T22:54:39.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.744+0000 7f606be00640 1 -- 172.21.15.131:0/359199857 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6054003520 con 0x7f606c1416d0 2024-05-26T22:54:39.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.744+0000 7f6070c00640 1 -- 172.21.15.131:0/359199857 >> v1:172.21.15.187:6789/0 conn(0x7f606c141c10 legacy=0x7f606c06be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:39.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.744+0000 7f6070c00640 1 -- 172.21.15.131:0/359199857 >> v1:172.21.15.131:6789/0 conn(0x7f606c1416d0 legacy=0x7f606c0b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:39.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.745+0000 7f6070c00640 1 -- 172.21.15.131:0/359199857 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f606c06b690 con 0x7f606c0b4050 2024-05-26T22:54:39.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.746+0000 7f6070c00640 1 --2- 172.21.15.131:0/359199857 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f606c0b4050 0x7f606c143c60 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f60580097a0 tx=0x7f605802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d30018440a90675 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:39.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.746+0000 7f606be00640 1 -- 172.21.15.131:0/359199857 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6058037070 con 0x7f606c0b4050 2024-05-26T22:54:39.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.746+0000 7f606be00640 1 -- 172.21.15.131:0/359199857 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6058032440 con 0x7f606c0b4050 2024-05-26T22:54:39.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.746+0000 7f606be00640 1 -- 172.21.15.131:0/359199857 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60580327f0 con 0x7f606c0b4050 2024-05-26T22:54:39.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.746+0000 7f6072a00640 1 -- 172.21.15.131:0/359199857 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f606c0b4050 msgr2=0x7f606c143c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:39.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.746+0000 7f6072a00640 1 --2- 172.21.15.131:0/359199857 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f606c0b4050 0x7f606c143c60 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f60580097a0 tx=0x7f605802a1e0 comp rx=0 tx=0).stop 2024-05-26T22:54:39.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.746+0000 7f6072a00640 1 -- 172.21.15.131:0/359199857 shutdown_connections 2024-05-26T22:54:39.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.746+0000 7f6072a00640 1 --2- 172.21.15.131:0/359199857 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f606c0b4050 0x7f606c143c60 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:39.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.746+0000 7f6072a00640 1 -- 172.21.15.131:0/359199857 >> 172.21.15.131:0/359199857 conn(0x7f606c0d0660 msgr2=0x7f606c0d0a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:39.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.747+0000 7f6072a00640 1 -- 172.21.15.131:0/359199857 shutdown_connections 2024-05-26T22:54:39.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.747+0000 7f6072a00640 1 -- 172.21.15.131:0/359199857 wait complete. 2024-05-26T22:54:39.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.748+0000 7f6072a00640 1 Processor -- start 2024-05-26T22:54:39.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.748+0000 7f6072a00640 1 -- start start 2024-05-26T22:54:39.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.749+0000 7f6072a00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f606c0b4050 0x7f606c15b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:39.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.749+0000 7f6072a00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f606c1416d0 0x7f606c0b1330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:39.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.749+0000 7f6072a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f606c141c10 0x7f606c0b1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:39.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.749+0000 7f6072a00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f606c144320 con 0x7f606c141c10 2024-05-26T22:54:39.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.749+0000 7f6072a00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f606c1441a0 con 0x7f606c1416d0 2024-05-26T22:54:39.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.749+0000 7f6072a00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f606c1444a0 con 0x7f606c0b4050 2024-05-26T22:54:39.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.749+0000 7f6072000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f606c141c10 0x7f606c0b1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:39.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.749+0000 7f6072000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f606c141c10 0x7f606c0b1870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44304/0 (socket says 172.21.15.131:44304) 2024-05-26T22:54:39.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.749+0000 7f6072000640 1 -- 172.21.15.131:0/4280926524 learned_addr learned my addr 172.21.15.131:0/4280926524 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:54:39.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.750+0000 7f6072000640 1 -- 172.21.15.131:0/4280926524 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f606c0b4050 msgr2=0x7f606c15b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:39.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.750+0000 7f6070c00640 1 --2- 172.21.15.131:0/4280926524 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f606c1416d0 0x7f606c0b1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:39.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.750+0000 7f6071600640 1 --2- 172.21.15.131:0/4280926524 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f606c0b4050 0x7f606c15b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:39.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.750+0000 7f6072000640 1 --2- 172.21.15.131:0/4280926524 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f606c0b4050 0x7f606c15b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:39.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.750+0000 7f6072000640 1 -- 172.21.15.131:0/4280926524 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f606c1416d0 msgr2=0x7f606c0b1330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:39.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.750+0000 7f6072000640 1 --2- 172.21.15.131:0/4280926524 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f606c1416d0 0x7f606c0b1330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:39.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.750+0000 7f6072000640 1 -- 172.21.15.131:0/4280926524 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f606c145a80 con 0x7f606c141c10 2024-05-26T22:54:39.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.750+0000 7f6072000640 1 --2- 172.21.15.131:0/4280926524 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f606c141c10 0x7f606c0b1870 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f606400b9a0 tx=0x7f606400be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:39.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.750+0000 7f6069600640 1 -- 172.21.15.131:0/4280926524 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6064019070 con 0x7f606c141c10 2024-05-26T22:54:39.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.750+0000 7f6069600640 1 -- 172.21.15.131:0/4280926524 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f606400cba0 con 0x7f606c141c10 2024-05-26T22:54:39.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.750+0000 7f6072a00640 1 -- 172.21.15.131:0/4280926524 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f606c145d60 con 0x7f606c141c10 2024-05-26T22:54:39.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.750+0000 7f6069600640 1 -- 172.21.15.131:0/4280926524 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6064014440 con 0x7f606c141c10 2024-05-26T22:54:39.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.751+0000 7f6072a00640 1 -- 172.21.15.131:0/4280926524 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f606c146320 con 0x7f606c141c10 2024-05-26T22:54:39.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.751+0000 7f6072a00640 1 -- 172.21.15.131:0/4280926524 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f606c0694e0 con 0x7f606c141c10 2024-05-26T22:54:39.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.753+0000 7f6069600640 1 -- 172.21.15.131:0/4280926524 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f606401c070 con 0x7f606c141c10 2024-05-26T22:54:39.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.753+0000 7f6069600640 1 --2- 172.21.15.131:0/4280926524 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f604c06be60 0x7f604c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:39.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.754+0000 7f6069600640 1 -- 172.21.15.131:0/4280926524 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5738+0+0 (secure 0 0 0) 0x7f606408ab20 con 0x7f606c141c10 2024-05-26T22:54:39.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.754+0000 7f6071600640 1 --2- 172.21.15.131:0/4280926524 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f604c06be60 0x7f604c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:39.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.754+0000 7f6071600640 1 --2- 172.21.15.131:0/4280926524 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f604c06be60 0x7f604c06e320 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f6060000f60 tx=0x7f6060009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:39.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.758+0000 7f6069600640 1 -- 172.21.15.131:0/4280926524 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f6064059410 con 0x7f606c141c10 2024-05-26T22:54:39.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.956+0000 7f6072a00640 1 -- 172.21.15.131:0/4280926524 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f606c0b47f0 con 0x7f606c141c10 2024-05-26T22:54:39.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.957+0000 7f6069600640 1 -- 172.21.15.131:0/4280926524 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v84) ==== 100+0+11 (secure 0 0 0) 0x7f6064058d10 con 0x7f606c141c10 2024-05-26T22:54:39.957 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 64 2024-05-26T22:54:39.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.960+0000 7f6072a00640 1 -- 172.21.15.131:0/4280926524 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f604c06be60 msgr2=0x7f604c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:39.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.960+0000 7f6072a00640 1 --2- 172.21.15.131:0/4280926524 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f604c06be60 0x7f604c06e320 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f6060000f60 tx=0x7f6060009040 comp rx=0 tx=0).stop 2024-05-26T22:54:39.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.960+0000 7f6072a00640 1 -- 172.21.15.131:0/4280926524 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f606c141c10 msgr2=0x7f606c0b1870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:39.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.960+0000 7f6072a00640 1 --2- 172.21.15.131:0/4280926524 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f606c141c10 0x7f606c0b1870 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f606400b9a0 tx=0x7f606400be60 comp rx=0 tx=0).stop 2024-05-26T22:54:39.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.960+0000 7f6072a00640 1 -- 172.21.15.131:0/4280926524 shutdown_connections 2024-05-26T22:54:39.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.960+0000 7f6072a00640 1 --2- 172.21.15.131:0/4280926524 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f604c06be60 0x7f604c06e320 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:39.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.960+0000 7f6072a00640 1 --2- 172.21.15.131:0/4280926524 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f606c141c10 0x7f606c0b1870 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:39.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.960+0000 7f6072a00640 1 --2- 172.21.15.131:0/4280926524 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f606c1416d0 0x7f606c0b1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:39.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.960+0000 7f6072a00640 1 --2- 172.21.15.131:0/4280926524 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f606c0b4050 0x7f606c15b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:39.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.960+0000 7f6072a00640 1 -- 172.21.15.131:0/4280926524 >> 172.21.15.131:0/4280926524 conn(0x7f606c0d0660 msgr2=0x7f606c15de90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:39.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.961+0000 7f6072a00640 1 -- 172.21.15.131:0/4280926524 shutdown_connections 2024-05-26T22:54:39.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:39.961+0000 7f6072a00640 1 -- 172.21.15.131:0/4280926524 wait complete. 2024-05-26T22:54:39.979 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-26T22:54:40.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.171+0000 7fb1df200640 1 Processor -- start 2024-05-26T22:54:40.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.171+0000 7fb1df200640 1 -- start start 2024-05-26T22:54:40.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.172+0000 7fb1df200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fb1d8156750 0x7fb1d815fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:40.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.173+0000 7fb1df200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb1d806e8c0 0x7fb1d815d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:40.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.173+0000 7fb1df200640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1d8160620 con 0x7fb1d80cc7f0 2024-05-26T22:54:40.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.173+0000 7fb1df200640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fb1d815cbd0 con 0x7fb1d8156750 2024-05-26T22:54:40.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.173+0000 7fb1df200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb1d8160900 con 0x7fb1d806e8c0 2024-05-26T22:54:40.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.173+0000 7fb1dde00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fb1d8156750 0x7fb1d815fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:40.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.173+0000 7fb1dde00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fb1d8156750 0x7fb1d815fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44306/0 (socket says 172.21.15.131:44306) 2024-05-26T22:54:40.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.173+0000 7fb1dde00640 1 -- 172.21.15.131:0/3012275382 learned_addr learned my addr 172.21.15.131:0/3012275382 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:54:40.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.173+0000 7fb1dd400640 1 --2- 172.21.15.131:0/3012275382 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb1d806e8c0 0x7fb1d815d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:40.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.173+0000 7fb1dde00640 1 -- 172.21.15.131:0/3012275382 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb1d806e8c0 msgr2=0x7fb1d815d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:40.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.173+0000 7fb1dde00640 1 --2- 172.21.15.131:0/3012275382 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb1d806e8c0 0x7fb1d815d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:40.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.173+0000 7fb1dde00640 1 -- 172.21.15.131:0/3012275382 >> v1:172.21.15.187:6789/0 conn(0x7fb1d80cc7f0 legacy=0x7fb1d80cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:40.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.173+0000 7fb1dde00640 1 -- 172.21.15.131:0/3012275382 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1d8160be0 con 0x7fb1d8156750 2024-05-26T22:54:40.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.174+0000 7fb1dde00640 1 --2- 172.21.15.131:0/3012275382 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1d8156750 0x7fb1d815fd40 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc009870 tx=0x7fb1cc02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47fbf3f9bcae830e server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:40.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.175+0000 7fb1dca00640 1 -- 172.21.15.131:0/3012275382 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1cc037070 con 0x7fb1d8156750 2024-05-26T22:54:40.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.175+0000 7fb1dca00640 1 -- 172.21.15.131:0/3012275382 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1cc032440 con 0x7fb1d8156750 2024-05-26T22:54:40.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.175+0000 7fb1dca00640 1 -- 172.21.15.131:0/3012275382 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1cc032810 con 0x7fb1d8156750 2024-05-26T22:54:40.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.175+0000 7fb1df200640 1 -- 172.21.15.131:0/3012275382 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1d8156750 msgr2=0x7fb1d815fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:40.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.175+0000 7fb1df200640 1 --2- 172.21.15.131:0/3012275382 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1d8156750 0x7fb1d815fd40 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc009870 tx=0x7fb1cc02a210 comp rx=0 tx=0).stop 2024-05-26T22:54:40.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.175+0000 7fb1df200640 1 -- 172.21.15.131:0/3012275382 shutdown_connections 2024-05-26T22:54:40.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.175+0000 7fb1df200640 1 --2- 172.21.15.131:0/3012275382 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb1d806e8c0 0x7fb1d815d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:40.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.175+0000 7fb1df200640 1 --2- 172.21.15.131:0/3012275382 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1d8156750 0x7fb1d815fd40 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:40.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.175+0000 7fb1df200640 1 -- 172.21.15.131:0/3012275382 >> 172.21.15.131:0/3012275382 conn(0x7fb1d80cff40 msgr2=0x7fb1d80d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:40.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.176+0000 7fb1df200640 1 -- 172.21.15.131:0/3012275382 shutdown_connections 2024-05-26T22:54:40.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.176+0000 7fb1df200640 1 -- 172.21.15.131:0/3012275382 wait complete. 2024-05-26T22:54:40.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.177+0000 7fb1df200640 1 Processor -- start 2024-05-26T22:54:40.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.177+0000 7fb1df200640 1 -- start start 2024-05-26T22:54:40.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.178+0000 7fb1df200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb1d806e8c0 0x7fb1d8147750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:40.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.178+0000 7fb1df200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb1d80cc7f0 0x7fb1d814dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:40.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.178+0000 7fb1df200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1d8156750 0x7fb1d814e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:40.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.178+0000 7fb1df200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fb1d8160280 con 0x7fb1d8156750 2024-05-26T22:54:40.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.178+0000 7fb1df200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb1d8156b30 con 0x7fb1d806e8c0 2024-05-26T22:54:40.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.178+0000 7fb1df200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb1d8160400 con 0x7fb1d80cc7f0 2024-05-26T22:54:40.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.179+0000 7fb1de800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1d8156750 0x7fb1d814e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:40.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.179+0000 7fb1de800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1d8156750 0x7fb1d814e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44312/0 (socket says 172.21.15.131:44312) 2024-05-26T22:54:40.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.179+0000 7fb1de800640 1 -- 172.21.15.131:0/3149481566 learned_addr learned my addr 172.21.15.131:0/3149481566 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:54:40.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.179+0000 7fb1dde00640 1 --2- 172.21.15.131:0/3149481566 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb1d806e8c0 0x7fb1d8147750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:40.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.179+0000 7fb1de800640 1 -- 172.21.15.131:0/3149481566 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb1d80cc7f0 msgr2=0x7fb1d814dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:40.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.179+0000 7fb1de800640 1 --2- 172.21.15.131:0/3149481566 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb1d80cc7f0 0x7fb1d814dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:40.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.179+0000 7fb1de800640 1 -- 172.21.15.131:0/3149481566 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb1d806e8c0 msgr2=0x7fb1d8147750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:40.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.179+0000 7fb1de800640 1 --2- 172.21.15.131:0/3149481566 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb1d806e8c0 0x7fb1d8147750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:40.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.179+0000 7fb1de800640 1 -- 172.21.15.131:0/3149481566 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1d80b1800 con 0x7fb1d8156750 2024-05-26T22:54:40.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.179+0000 7fb1de800640 1 --2- 172.21.15.131:0/3149481566 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1d8156750 0x7fb1d814e000 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7fb1d400c780 tx=0x7fb1d400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:40.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.179+0000 7fb1d2000640 1 -- 172.21.15.131:0/3149481566 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1d4004b60 con 0x7fb1d8156750 2024-05-26T22:54:40.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.180+0000 7fb1d2000640 1 -- 172.21.15.131:0/3149481566 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1d4004d00 con 0x7fb1d8156750 2024-05-26T22:54:40.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.180+0000 7fb1d2000640 1 -- 172.21.15.131:0/3149481566 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1d400aa90 con 0x7fb1d8156750 2024-05-26T22:54:40.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.180+0000 7fb1df200640 1 -- 172.21.15.131:0/3149481566 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1d80b0c80 con 0x7fb1d8156750 2024-05-26T22:54:40.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.180+0000 7fb1df200640 1 -- 172.21.15.131:0/3149481566 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1d80b1240 con 0x7fb1d8156750 2024-05-26T22:54:40.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.181+0000 7fb1df200640 1 -- 172.21.15.131:0/3149481566 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1d80d0320 con 0x7fb1d8156750 2024-05-26T22:54:40.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.182+0000 7fb1d2000640 1 -- 172.21.15.131:0/3149481566 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb1d400ac30 con 0x7fb1d8156750 2024-05-26T22:54:40.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.184+0000 7fb1d2000640 1 --2- 172.21.15.131:0/3149481566 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb1bc06be30 0x7fb1bc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:40.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.184+0000 7fb1d2000640 1 -- 172.21.15.131:0/3149481566 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5738+0+0 (secure 0 0 0) 0x7fb1d408b190 con 0x7fb1d8156750 2024-05-26T22:54:40.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.184+0000 7fb1dde00640 1 --2- 172.21.15.131:0/3149481566 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb1bc06be30 0x7fb1bc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:40.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.184+0000 7fb1dde00640 1 --2- 172.21.15.131:0/3149481566 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb1bc06be30 0x7fb1bc06e2f0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc034000 tx=0x7fb1cc02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:40.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.188+0000 7fb1d2000640 1 -- 172.21.15.131:0/3149481566 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fb1d80d0320 con 0x7fb1d8156750 2024-05-26T22:54:40.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.427+0000 7fb1df200640 1 -- 172.21.15.131:0/3149481566 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb1d80694e0 con 0x7fb1d8156750 2024-05-26T22:54:40.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.428+0000 7fb1d2000640 1 -- 172.21.15.131:0/3149481566 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) ==== 94+0+5231 (secure 0 0 0) 0x7fb1d4020070 con 0x7fb1d8156750 2024-05-26T22:54:40.516 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:54:40.516 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":85,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5533,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/993930297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6835","nonce":993930297},{"type":"v1","addr":"172.21.15.131:6837","nonce":993930297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-26T22:53:52.862328+0000","modified":"2024-05-26T22:54:28.015691+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7365,"mds_1":5548,"mds_2":5545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5545":{"gid":5545,"name":"b","rank":2,"incarnation":82,"state":"up:active","state_seq":7,"addr":"172.21.15.187:6834/3480101957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3480101957},{"type":"v1","addr":"172.21.15.187:6834","nonce":3480101957}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5548":{"gid":5548,"name":"d","rank":1,"incarnation":80,"state":"up:active","state_seq":7,"addr":"172.21.15.187:6835/567389303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6833","nonce":567389303},{"type":"v1","addr":"172.21.15.187:6835","nonce":567389303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7365":{"gid":7365,"name":"a","rank":0,"incarnation":74,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6836/3411392567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":3411392567},{"type":"v1","addr":"172.21.15.131:6836","nonce":3411392567}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7365,"qdb_cluster":[5545,7365,5548]},"id":6}]} 2024-05-26T22:54:40.517 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 85 2024-05-26T22:54:40.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.431+0000 7fb1df200640 1 -- 172.21.15.131:0/3149481566 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb1bc06be30 msgr2=0x7fb1bc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:40.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.431+0000 7fb1df200640 1 --2- 172.21.15.131:0/3149481566 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb1bc06be30 0x7fb1bc06e2f0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc034000 tx=0x7fb1cc02e000 comp rx=0 tx=0).stop 2024-05-26T22:54:40.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.431+0000 7fb1df200640 1 -- 172.21.15.131:0/3149481566 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1d8156750 msgr2=0x7fb1d814e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:40.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.431+0000 7fb1df200640 1 --2- 172.21.15.131:0/3149481566 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1d8156750 0x7fb1d814e000 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7fb1d400c780 tx=0x7fb1d400cc40 comp rx=0 tx=0).stop 2024-05-26T22:54:40.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.431+0000 7fb1df200640 1 -- 172.21.15.131:0/3149481566 shutdown_connections 2024-05-26T22:54:40.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.431+0000 7fb1df200640 1 --2- 172.21.15.131:0/3149481566 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb1d80cc7f0 0x7fb1d814dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:40.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.431+0000 7fb1df200640 1 --2- 172.21.15.131:0/3149481566 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb1bc06be30 0x7fb1bc06e2f0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:40.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.431+0000 7fb1df200640 1 --2- 172.21.15.131:0/3149481566 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1d8156750 0x7fb1d814e000 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:40.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.431+0000 7fb1df200640 1 --2- 172.21.15.131:0/3149481566 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb1d806e8c0 0x7fb1d8147750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:40.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.432+0000 7fb1df200640 1 -- 172.21.15.131:0/3149481566 >> 172.21.15.131:0/3149481566 conn(0x7fb1d80cff40 msgr2=0x7fb1d815a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:40.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.432+0000 7fb1df200640 1 -- 172.21.15.131:0/3149481566 shutdown_connections 2024-05-26T22:54:40.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.432+0000 7fb1df200640 1 -- 172.21.15.131:0/3149481566 wait complete. 2024-05-26T22:54:40.519 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-26T22:54:40.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.724+0000 7fb88de00640 1 Processor -- start 2024-05-26T22:54:40.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.724+0000 7fb88de00640 1 -- start start 2024-05-26T22:54:40.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.725+0000 7fb88de00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb888144340 0x7fb8880cedf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:40.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.725+0000 7fb88de00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb888071fd0 con 0x7fb88806ebc0 2024-05-26T22:54:40.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.725+0000 7fb88de00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8880722a0 con 0x7fb888071b70 2024-05-26T22:54:40.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.725+0000 7fb88de00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb888072eb0 con 0x7fb888144340 2024-05-26T22:54:40.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.726+0000 7fb88d400640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fb88806ebc0 0x7fb8880b3d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:58188/0 (socket says 172.21.15.131:58188) 2024-05-26T22:54:40.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.726+0000 7fb88d400640 1 -- 172.21.15.131:0/536234138 learned_addr learned my addr 172.21.15.131:0/536234138 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:54:40.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.726+0000 7fb88ca00640 1 --2- 172.21.15.131:0/536234138 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb888144340 0x7fb8880cedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:40.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.726+0000 7fb887e00640 1 -- 172.21.15.131:0/536234138 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1583798596 0 0) 0x7fb888071fd0 con 0x7fb88806ebc0 2024-05-26T22:54:40.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.726+0000 7fb887e00640 1 -- 172.21.15.131:0/536234138 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb868003520 con 0x7fb88806ebc0 2024-05-26T22:54:40.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.726+0000 7fb887e00640 1 -- 172.21.15.131:0/536234138 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1899641621 0 0) 0x7fb868003520 con 0x7fb88806ebc0 2024-05-26T22:54:40.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.726+0000 7fb887e00640 1 -- 172.21.15.131:0/536234138 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb888144340 msgr2=0x7fb8880cedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:40.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.726+0000 7fb887e00640 1 --2- 172.21.15.131:0/536234138 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb888144340 0x7fb8880cedf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:40.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.726+0000 7fb887e00640 1 -- 172.21.15.131:0/536234138 >> v1:172.21.15.187:6789/0 conn(0x7fb888071b70 legacy=0x7fb88806e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:40.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.726+0000 7fb887e00640 1 -- 172.21.15.131:0/536234138 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb888072580 con 0x7fb88806ebc0 2024-05-26T22:54:40.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.726+0000 7fb887e00640 1 -- 172.21.15.131:0/536234138 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7fb8800019f0 con 0x7fb88806ebc0 2024-05-26T22:54:40.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.726+0000 7fb887e00640 1 -- 172.21.15.131:0/536234138 >> v1:172.21.15.131:6789/0 conn(0x7fb88806ebc0 legacy=0x7fb8880b3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:40.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.727+0000 7fb887e00640 1 --2- 172.21.15.131:0/536234138 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb888144340 0x7fb8680039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:40.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.727+0000 7fb887e00640 1 --2- 172.21.15.131:0/536234138 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb868003f60 0x7fb868004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:40.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.727+0000 7fb887e00640 1 --2- 172.21.15.131:0/536234138 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb868004d40 0x7fb8680051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:40.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.727+0000 7fb887e00640 1 -- 172.21.15.131:0/536234138 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fb86800dc80 con 0x7fb868003f60 2024-05-26T22:54:40.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.727+0000 7fb887e00640 1 -- 172.21.15.131:0/536234138 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb86800de30 con 0x7fb868004d40 2024-05-26T22:54:40.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.727+0000 7fb887e00640 1 -- 172.21.15.131:0/536234138 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb86800e030 con 0x7fb888144340 2024-05-26T22:54:40.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.727+0000 7fb87fe00640 1 --2- 172.21.15.131:0/536234138 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb868003f60 0x7fb868004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:40.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.727+0000 7fb87fe00640 1 -- 172.21.15.131:0/536234138 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb888144340 msgr2=0x7fb8680039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:40.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.727+0000 7fb88ca00640 1 --2- 172.21.15.131:0/536234138 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb888144340 0x7fb8680039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:40.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.727+0000 7fb88d400640 1 --2- 172.21.15.131:0/536234138 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb868004d40 0x7fb8680051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:40.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.727+0000 7fb87fe00640 1 --2- 172.21.15.131:0/536234138 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb888144340 0x7fb8680039e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:40.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.727+0000 7fb87fe00640 1 -- 172.21.15.131:0/536234138 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb868004d40 msgr2=0x7fb8680051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:40.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.727+0000 7fb87fe00640 1 --2- 172.21.15.131:0/536234138 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb868004d40 0x7fb8680051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:40.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.727+0000 7fb87fe00640 1 -- 172.21.15.131:0/536234138 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb868003520 con 0x7fb868003f60 2024-05-26T22:54:40.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.728+0000 7fb87fe00640 1 --2- 172.21.15.131:0/536234138 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb868003f60 0x7fb868004800 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7fb878001750 tx=0x7fb868005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db0cdbcde63564ba server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:40.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.729+0000 7fb887e00640 1 -- 172.21.15.131:0/536234138 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb878001370 con 0x7fb868003f60 2024-05-26T22:54:40.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.729+0000 7fb887e00640 1 -- 172.21.15.131:0/536234138 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8780052d0 con 0x7fb868003f60 2024-05-26T22:54:40.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.729+0000 7fb887e00640 1 -- 172.21.15.131:0/536234138 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb87802abe0 con 0x7fb868003f60 2024-05-26T22:54:40.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.729+0000 7fb88de00640 1 -- 172.21.15.131:0/536234138 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb868003f60 msgr2=0x7fb868004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:40.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.729+0000 7fb88de00640 1 --2- 172.21.15.131:0/536234138 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb868003f60 0x7fb868004800 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7fb878001750 tx=0x7fb868005830 comp rx=0 tx=0).stop 2024-05-26T22:54:40.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.729+0000 7fb88d400640 1 -- 172.21.15.131:0/536234138 reap_dead start 2024-05-26T22:54:40.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.729+0000 7fb88de00640 1 -- 172.21.15.131:0/536234138 shutdown_connections 2024-05-26T22:54:40.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.729+0000 7fb88de00640 1 -- 172.21.15.131:0/536234138 >> 172.21.15.131:0/536234138 conn(0x7fb8880cff40 msgr2=0x7fb8880d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:40.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.729+0000 7fb88de00640 1 -- 172.21.15.131:0/536234138 shutdown_connections 2024-05-26T22:54:40.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.729+0000 7fb88de00640 1 -- 172.21.15.131:0/536234138 wait complete. 2024-05-26T22:54:40.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.731+0000 7fb88de00640 1 Processor -- start 2024-05-26T22:54:40.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.731+0000 7fb88de00640 1 -- start start 2024-05-26T22:54:40.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.732+0000 7fb88de00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb868003f60 0x7fb8880c87c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:40.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.732+0000 7fb88de00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb8880c71c0 0x7fb8881f4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:40.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.732+0000 7fb88de00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb8880c7640 0x7fb88815d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:40.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.732+0000 7fb88de00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fb8880cedf0 con 0x7fb8880c7640 2024-05-26T22:54:40.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.732+0000 7fb88de00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb888072b40 con 0x7fb8880c71c0 2024-05-26T22:54:40.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.732+0000 7fb88de00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb8880cef70 con 0x7fb868003f60 2024-05-26T22:54:40.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.732+0000 7fb88d400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb8880c7640 0x7fb88815d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:40.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.732+0000 7fb88d400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb8880c7640 0x7fb88815d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44318/0 (socket says 172.21.15.131:44318) 2024-05-26T22:54:40.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.732+0000 7fb88d400640 1 -- 172.21.15.131:0/2275071431 learned_addr learned my addr 172.21.15.131:0/2275071431 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:54:40.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.733+0000 7fb88ca00640 1 --2- 172.21.15.131:0/2275071431 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb868003f60 0x7fb8880c87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:40.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.733+0000 7fb87fe00640 1 --2- 172.21.15.131:0/2275071431 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb8880c71c0 0x7fb8881f4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:40.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.733+0000 7fb88d400640 1 -- 172.21.15.131:0/2275071431 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb868003f60 msgr2=0x7fb8880c87c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:40.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.733+0000 7fb88d400640 1 --2- 172.21.15.131:0/2275071431 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb868003f60 0x7fb8880c87c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:40.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.733+0000 7fb88d400640 1 -- 172.21.15.131:0/2275071431 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb8880c71c0 msgr2=0x7fb8881f4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:40.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.733+0000 7fb88d400640 1 --2- 172.21.15.131:0/2275071431 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb8880c71c0 0x7fb8881f4ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:40.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.733+0000 7fb88d400640 1 -- 172.21.15.131:0/2275071431 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb88815c900 con 0x7fb8880c7640 2024-05-26T22:54:40.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.733+0000 7fb87fe00640 1 --2- 172.21.15.131:0/2275071431 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb8880c71c0 0x7fb8881f4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:54:40.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.733+0000 7fb88d400640 1 --2- 172.21.15.131:0/2275071431 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb8880c7640 0x7fb88815d9a0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7fb880001580 tx=0x7fb88000b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:40.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.733+0000 7fb885600640 1 -- 172.21.15.131:0/2275071431 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb88000bb80 con 0x7fb8880c7640 2024-05-26T22:54:40.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.733+0000 7fb885600640 1 -- 172.21.15.131:0/2275071431 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb88000bd20 con 0x7fb8880c7640 2024-05-26T22:54:40.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.733+0000 7fb88de00640 1 -- 172.21.15.131:0/2275071431 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb88815cbe0 con 0x7fb8880c7640 2024-05-26T22:54:40.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.733+0000 7fb885600640 1 -- 172.21.15.131:0/2275071431 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb880007350 con 0x7fb8880c7640 2024-05-26T22:54:40.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.734+0000 7fb88de00640 1 -- 172.21.15.131:0/2275071431 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb88815d1a0 con 0x7fb8880c7640 2024-05-26T22:54:40.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.735+0000 7fb88de00640 1 -- 172.21.15.131:0/2275071431 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8880d0810 con 0x7fb8880c7640 2024-05-26T22:54:40.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.736+0000 7fb885600640 1 -- 172.21.15.131:0/2275071431 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb88001b060 con 0x7fb8880c7640 2024-05-26T22:54:40.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.737+0000 7fb885600640 1 --2- 172.21.15.131:0/2275071431 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb86406be30 0x7fb86406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:40.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.737+0000 7fb885600640 1 -- 172.21.15.131:0/2275071431 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5738+0+0 (secure 0 0 0) 0x7fb88008af50 con 0x7fb8880c7640 2024-05-26T22:54:40.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.737+0000 7fb88ca00640 1 --2- 172.21.15.131:0/2275071431 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb86406be30 0x7fb86406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:40.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.738+0000 7fb88ca00640 1 --2- 172.21.15.131:0/2275071431 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb86406be30 0x7fb86406e2f0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fb870009910 tx=0x7fb870008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:40.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.741+0000 7fb885600640 1 -- 172.21.15.131:0/2275071431 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fb8880d0810 con 0x7fb8880c7640 2024-05-26T22:54:40.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.929+0000 7fb88de00640 1 -- 172.21.15.131:0/2275071431 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb8880ce720 con 0x7fb8880c7640 2024-05-26T22:54:40.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.931+0000 7fb885600640 1 -- 172.21.15.131:0/2275071431 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+16734 (secure 0 0 0) 0x7fb8800598d0 con 0x7fb8880c7640 2024-05-26T22:54:40.931 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:54:40.931 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":84,"fsid":"46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f","created":"2024-05-26T22:47:57.288905+0000","modified":"2024-05-26T22:54:07.109294+0000","last_up_change":"2024-05-26T22:48:09.602207+0000","last_in_change":"2024-05-26T22:48:00.687341+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-26T22:48:10.463339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-05-26T22:53:50.625629+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-05-26T22:53:51.864320+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"814eaf62-a18d-494b-8e27-fd8fb88adea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6801","nonce":2350940051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6803","nonce":2350940051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6807","nonce":2350940051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6805","nonce":2350940051}]},"public_addr":"172.21.15.131:6801/2350940051","cluster_addr":"172.21.15.131:6803/2350940051","heartbeat_back_addr":"172.21.15.131:6807/2350940051","heartbeat_front_addr":"172.21.15.131:6805/2350940051","state":["exists","up"]},{"osd":1,"uuid":"42e66287-f6c6-4e26-af3c-6dfaa3536d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6825","nonce":1525528247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6827","nonce":1525528247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6831","nonce":1525528247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6829","nonce":1525528247}]},"public_addr":"172.21.15.131:6825/1525528247","cluster_addr":"172.21.15.131:6827/1525528247","heartbeat_back_addr":"172.21.15.131:6831/1525528247","heartbeat_front_addr":"172.21.15.131:6829/1525528247","state":["exists","up"]},{"osd":2,"uuid":"0da34673-04a2-4fd0-b57a-0ff2b679b043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6817","nonce":3627842210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6819","nonce":3627842210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6823","nonce":3627842210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6821","nonce":3627842210}]},"public_addr":"172.21.15.131:6817/3627842210","cluster_addr":"172.21.15.131:6819/3627842210","heartbeat_back_addr":"172.21.15.131:6823/3627842210","heartbeat_front_addr":"172.21.15.131:6821/3627842210","state":["exists","up"]},{"osd":3,"uuid":"cf117a27-d6b2-43d3-ae52-b47988bf8eed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6809","nonce":350790591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6811","nonce":350790591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6815","nonce":350790591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6813","nonce":350790591}]},"public_addr":"172.21.15.131:6809/350790591","cluster_addr":"172.21.15.131:6811/350790591","heartbeat_back_addr":"172.21.15.131:6815/350790591","heartbeat_front_addr":"172.21.15.131:6813/350790591","state":["exists","up"]},{"osd":4,"uuid":"a092ccfc-be89-4a84-88d5-0f781fa715eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6816","nonce":244560995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6819","nonce":244560995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6823","nonce":244560995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6821","nonce":244560995}]},"public_addr":"172.21.15.187:6816/244560995","cluster_addr":"172.21.15.187:6819/244560995","heartbeat_back_addr":"172.21.15.187:6823/244560995","heartbeat_front_addr":"172.21.15.187:6821/244560995","state":["exists","up"]},{"osd":5,"uuid":"7513ba69-dd00-4a35-b190-9e8cc3b2bdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6801","nonce":324619398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6803","nonce":324619398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6812","nonce":324619398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6807","nonce":324619398}]},"public_addr":"172.21.15.187:6801/324619398","cluster_addr":"172.21.15.187:6803/324619398","heartbeat_back_addr":"172.21.15.187:6812/324619398","heartbeat_front_addr":"172.21.15.187:6807/324619398","state":["exists","up"]},{"osd":6,"uuid":"50081810-c093-4d2f-a4d7-b7c695c5400d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6806","nonce":1391347786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6809","nonce":1391347786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6818","nonce":1391347786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6813","nonce":1391347786}]},"public_addr":"172.21.15.187:6806/1391347786","cluster_addr":"172.21.15.187:6809/1391347786","heartbeat_back_addr":"172.21.15.187:6818/1391347786","heartbeat_front_addr":"172.21.15.187:6813/1391347786","state":["exists","up"]},{"osd":7,"uuid":"775dae05-54f5-4ad4-bbce-92863c6ed2ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6825","nonce":1220065622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6827","nonce":1220065622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6831","nonce":1220065622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6829","nonce":1220065622}]},"public_addr":"172.21.15.187:6825/1220065622","cluster_addr":"172.21.15.187:6827/1220065622","heartbeat_back_addr":"172.21.15.187:6831/1220065622","heartbeat_front_addr":"172.21.15.187:6829/1220065622","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:05.983520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.299516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.067323+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.307748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.018742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.258775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.010912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.295355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.38","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-26T22:54:40.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.934+0000 7fb88de00640 1 -- 172.21.15.131:0/2275071431 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb86406be30 msgr2=0x7fb86406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:40.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.934+0000 7fb88de00640 1 --2- 172.21.15.131:0/2275071431 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb86406be30 0x7fb86406e2f0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fb870009910 tx=0x7fb870008040 comp rx=0 tx=0).stop 2024-05-26T22:54:40.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.934+0000 7fb88de00640 1 -- 172.21.15.131:0/2275071431 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb8880c7640 msgr2=0x7fb88815d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:40.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.934+0000 7fb88de00640 1 --2- 172.21.15.131:0/2275071431 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb8880c7640 0x7fb88815d9a0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7fb880001580 tx=0x7fb88000b4a0 comp rx=0 tx=0).stop 2024-05-26T22:54:40.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.934+0000 7fb88de00640 1 -- 172.21.15.131:0/2275071431 shutdown_connections 2024-05-26T22:54:40.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.934+0000 7fb88de00640 1 --2- 172.21.15.131:0/2275071431 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb86406be30 0x7fb86406e2f0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:40.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.934+0000 7fb88de00640 1 --2- 172.21.15.131:0/2275071431 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb8880c7640 0x7fb88815d9a0 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:40.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.934+0000 7fb88de00640 1 --2- 172.21.15.131:0/2275071431 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb8880c71c0 0x7fb8881f4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:40.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.934+0000 7fb88de00640 1 --2- 172.21.15.131:0/2275071431 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb868003f60 0x7fb8880c87c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:40.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.934+0000 7fb88de00640 1 -- 172.21.15.131:0/2275071431 >> 172.21.15.131:0/2275071431 conn(0x7fb8880cff40 msgr2=0x7fb88806e170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:40.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.934+0000 7fb88de00640 1 -- 172.21.15.131:0/2275071431 shutdown_connections 2024-05-26T22:54:40.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:40.934+0000 7fb88de00640 1 -- 172.21.15.131:0/2275071431 wait complete. 2024-05-26T22:54:40.951 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-26T22:54:40.952 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-26T22:54:41.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.147+0000 7f29b3600640 1 Processor -- start 2024-05-26T22:54:41.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.147+0000 7f29b3600640 1 -- start start 2024-05-26T22:54:41.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.149+0000 7f29b3600640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f29ac162780 0x7f29ac0b3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:41.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.149+0000 7f29b3600640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f29ac06f600 0x7f29ac06d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:41.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.149+0000 7f29b3600640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29ac0ad170 con 0x7f29ac06daf0 2024-05-26T22:54:41.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.149+0000 7f29b3600640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f29ac06df50 con 0x7f29ac06f600 2024-05-26T22:54:41.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.149+0000 7f29b3600640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f29ac0b4510 con 0x7f29ac162780 2024-05-26T22:54:41.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.149+0000 7f29b1600640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f29ac06daf0 0x7f29ac166380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:58200/0 (socket says 172.21.15.131:58200) 2024-05-26T22:54:41.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.149+0000 7f29b1600640 1 -- 172.21.15.131:0/640322768 learned_addr learned my addr 172.21.15.131:0/640322768 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:54:41.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.149+0000 7f29b0c00640 1 --2- 172.21.15.131:0/640322768 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f29ac162780 0x7f29ac0b3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:41.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.149+0000 7f29abe00640 1 --2- 172.21.15.131:0/640322768 >> v2:172.21.15.187:3300/0 conn(0x7f29ac06f600 0x7f29ac06d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:41.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.150+0000 7f29ab400640 1 -- 172.21.15.131:0/640322768 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 402403696 0 0) 0x7f29ac0ad170 con 0x7f29ac06daf0 2024-05-26T22:54:41.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.150+0000 7f29ab400640 1 -- 172.21.15.131:0/640322768 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2994003520 con 0x7f29ac06daf0 2024-05-26T22:54:41.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.150+0000 7f29ab400640 1 -- 172.21.15.131:0/640322768 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 246288760 0 0) 0x7f2994003520 con 0x7f29ac06daf0 2024-05-26T22:54:41.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.150+0000 7f29ab400640 1 -- 172.21.15.131:0/640322768 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f29ac162780 msgr2=0x7f29ac0b3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:41.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.150+0000 7f29ab400640 1 --2- 172.21.15.131:0/640322768 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f29ac162780 0x7f29ac0b3ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:41.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.150+0000 7f29ab400640 1 -- 172.21.15.131:0/640322768 >> v2:172.21.15.187:3300/0 conn(0x7f29ac06f600 msgr2=0x7f29ac06d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:41.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.150+0000 7f29ab400640 1 --2- 172.21.15.131:0/640322768 >> v2:172.21.15.187:3300/0 conn(0x7f29ac06f600 0x7f29ac06d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:41.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.150+0000 7f29ab400640 1 -- 172.21.15.131:0/640322768 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29ac0b47f0 con 0x7f29ac06daf0 2024-05-26T22:54:41.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.150+0000 7f29ab400640 1 -- 172.21.15.131:0/640322768 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7f29a40019f0 con 0x7f29ac06daf0 2024-05-26T22:54:41.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.150+0000 7f29ab400640 1 -- 172.21.15.131:0/640322768 >> v1:172.21.15.131:6789/0 conn(0x7f29ac06daf0 legacy=0x7f29ac166380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:41.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.150+0000 7f29ab400640 1 --2- 172.21.15.131:0/640322768 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f29ac162780 0x7f29940039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:41.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.150+0000 7f29ab400640 1 --2- 172.21.15.131:0/640322768 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2994003f80 0x7f2994004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:41.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.150+0000 7f29ab400640 1 --2- 172.21.15.131:0/640322768 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2994004d40 0x7f29940051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:41.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.150+0000 7f29ab400640 1 -- 172.21.15.131:0/640322768 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f29ac0b4510 con 0x7f2994003f80 2024-05-26T22:54:41.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.150+0000 7f29ab400640 1 -- 172.21.15.131:0/640322768 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f299400b9c0 con 0x7f2994004d40 2024-05-26T22:54:41.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.150+0000 7f29ab400640 1 -- 172.21.15.131:0/640322768 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f299400bb40 con 0x7f29ac162780 2024-05-26T22:54:41.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.150+0000 7f29abe00640 1 --2- 172.21.15.131:0/640322768 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2994003f80 0x7f2994004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:41.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.151+0000 7f29abe00640 1 -- 172.21.15.131:0/640322768 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f29ac162780 msgr2=0x7f29940039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:41.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.151+0000 7f29b0c00640 1 --2- 172.21.15.131:0/640322768 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f29ac162780 0x7f29940039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:41.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.151+0000 7f29b1600640 1 --2- 172.21.15.131:0/640322768 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2994004d40 0x7f29940051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:41.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.151+0000 7f29abe00640 1 --2- 172.21.15.131:0/640322768 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f29ac162780 0x7f29940039b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:41.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.151+0000 7f29abe00640 1 -- 172.21.15.131:0/640322768 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2994004d40 msgr2=0x7f29940051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:41.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.151+0000 7f29abe00640 1 --2- 172.21.15.131:0/640322768 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2994004d40 0x7f29940051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:41.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.151+0000 7f29abe00640 1 -- 172.21.15.131:0/640322768 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2994003520 con 0x7f2994003f80 2024-05-26T22:54:41.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.151+0000 7f29b1600640 1 --2- 172.21.15.131:0/640322768 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2994004d40 0x7f29940051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:54:41.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.152+0000 7f29abe00640 1 --2- 172.21.15.131:0/640322768 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2994003f80 0x7f2994004800 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f2998009af0 tx=0x7f2994005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ead73e36071160a2 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:41.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.152+0000 7f29ab400640 1 -- 172.21.15.131:0/640322768 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2998036070 con 0x7f2994003f80 2024-05-26T22:54:41.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.152+0000 7f29ab400640 1 -- 172.21.15.131:0/640322768 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2998031400 con 0x7f2994003f80 2024-05-26T22:54:41.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.152+0000 7f29ab400640 1 -- 172.21.15.131:0/640322768 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2998009690 con 0x7f2994003f80 2024-05-26T22:54:41.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.153+0000 7f29b3600640 1 -- 172.21.15.131:0/640322768 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2994003f80 msgr2=0x7f2994004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:41.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.153+0000 7f29b3600640 1 --2- 172.21.15.131:0/640322768 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2994003f80 0x7f2994004800 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f2998009af0 tx=0x7f2994005830 comp rx=0 tx=0).stop 2024-05-26T22:54:41.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.153+0000 7f29b1600640 1 -- 172.21.15.131:0/640322768 reap_dead start 2024-05-26T22:54:41.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.153+0000 7f29b3600640 1 -- 172.21.15.131:0/640322768 shutdown_connections 2024-05-26T22:54:41.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.153+0000 7f29b3600640 1 -- 172.21.15.131:0/640322768 >> 172.21.15.131:0/640322768 conn(0x7f29ac0b1e90 msgr2=0x7f29ac0b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:41.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.153+0000 7f29b3600640 1 -- 172.21.15.131:0/640322768 shutdown_connections 2024-05-26T22:54:41.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.153+0000 7f29b3600640 1 -- 172.21.15.131:0/640322768 wait complete. 2024-05-26T22:54:41.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.155+0000 7f29b3600640 1 Processor -- start 2024-05-26T22:54:41.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.155+0000 7f29b3600640 1 -- start start 2024-05-26T22:54:41.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.156+0000 7f29b3600640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2994003f80 0x7f29ac14d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:41.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.156+0000 7f29b3600640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f29ac0ae010 0x7f29ac164500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:41.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.156+0000 7f29b3600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f29ac0ae490 0x7f29ac164a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:41.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.156+0000 7f29b3600640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f29ac0ae960 con 0x7f29ac0ae490 2024-05-26T22:54:41.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.156+0000 7f29b3600640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f29ac162b60 con 0x7f29ac0ae010 2024-05-26T22:54:41.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.156+0000 7f29b3600640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f29ac0aeae0 con 0x7f2994003f80 2024-05-26T22:54:41.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.156+0000 7f29b1600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f29ac0ae490 0x7f29ac164a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:41.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.156+0000 7f29b1600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f29ac0ae490 0x7f29ac164a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44342/0 (socket says 172.21.15.131:44342) 2024-05-26T22:54:41.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.156+0000 7f29b1600640 1 -- 172.21.15.131:0/1814582893 learned_addr learned my addr 172.21.15.131:0/1814582893 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:54:41.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.157+0000 7f29b0c00640 1 --2- 172.21.15.131:0/1814582893 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2994003f80 0x7f29ac14d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:41.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.157+0000 7f29b1600640 1 -- 172.21.15.131:0/1814582893 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2994003f80 msgr2=0x7f29ac14d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:41.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.157+0000 7f29b1600640 1 --2- 172.21.15.131:0/1814582893 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2994003f80 0x7f29ac14d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:41.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.157+0000 7f29abe00640 1 --2- 172.21.15.131:0/1814582893 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f29ac0ae010 0x7f29ac164500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:41.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.157+0000 7f29b1600640 1 -- 172.21.15.131:0/1814582893 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f29ac0ae010 msgr2=0x7f29ac164500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:41.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.157+0000 7f29b1600640 1 --2- 172.21.15.131:0/1814582893 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f29ac0ae010 0x7f29ac164500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:41.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.157+0000 7f29b1600640 1 -- 172.21.15.131:0/1814582893 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29ac15bdb0 con 0x7f29ac0ae490 2024-05-26T22:54:41.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.157+0000 7f29b1600640 1 --2- 172.21.15.131:0/1814582893 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f29ac0ae490 0x7f29ac164a40 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f29ac071090 tx=0x7f29a400b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:41.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.157+0000 7f29a8c00640 1 -- 172.21.15.131:0/1814582893 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a4004d60 con 0x7f29ac0ae490 2024-05-26T22:54:41.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.157+0000 7f29a8c00640 1 -- 172.21.15.131:0/1814582893 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29a400bb80 con 0x7f29ac0ae490 2024-05-26T22:54:41.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.157+0000 7f29a8c00640 1 -- 172.21.15.131:0/1814582893 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a4007110 con 0x7f29ac0ae490 2024-05-26T22:54:41.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.157+0000 7f29b3600640 1 -- 172.21.15.131:0/1814582893 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29ac15c090 con 0x7f29ac0ae490 2024-05-26T22:54:41.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.157+0000 7f29b3600640 1 -- 172.21.15.131:0/1814582893 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29ac15c650 con 0x7f29ac0ae490 2024-05-26T22:54:41.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.159+0000 7f29b3600640 1 -- 172.21.15.131:0/1814582893 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29ac0694e0 con 0x7f29ac0ae490 2024-05-26T22:54:41.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.160+0000 7f29a8c00640 1 -- 172.21.15.131:0/1814582893 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f29a401b020 con 0x7f29ac0ae490 2024-05-26T22:54:41.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.161+0000 7f29a8c00640 1 --2- 172.21.15.131:0/1814582893 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f298c06be60 0x7f298c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:41.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.161+0000 7f29a8c00640 1 -- 172.21.15.131:0/1814582893 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5738+0+0 (secure 0 0 0) 0x7f29a408afe0 con 0x7f29ac0ae490 2024-05-26T22:54:41.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.161+0000 7f29b0c00640 1 --2- 172.21.15.131:0/1814582893 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f298c06be60 0x7f298c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:41.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.162+0000 7f29b0c00640 1 --2- 172.21.15.131:0/1814582893 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f298c06be60 0x7f298c06e320 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f29ac165ff0 tx=0x7f29a0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:41.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.165+0000 7f29a8c00640 1 -- 172.21.15.131:0/1814582893 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f29a4059a80 con 0x7f29ac0ae490 2024-05-26T22:54:41.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.400+0000 7f29b3600640 1 -- 172.21.15.131:0/1814582893 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f29ac15fef0 con 0x7f29ac0ae490 2024-05-26T22:54:41.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.402+0000 7f29a8c00640 1 -- 172.21.15.131:0/1814582893 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) ==== 94+0+5231 (secure 0 0 0) 0x7f29a4059380 con 0x7f29ac0ae490 2024-05-26T22:54:41.402 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:54:41.402 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":85,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5533,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/993930297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6835","nonce":993930297},{"type":"v1","addr":"172.21.15.131:6837","nonce":993930297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-26T22:53:52.862328+0000","modified":"2024-05-26T22:54:28.015691+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7365,"mds_1":5548,"mds_2":5545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5545":{"gid":5545,"name":"b","rank":2,"incarnation":82,"state":"up:active","state_seq":7,"addr":"172.21.15.187:6834/3480101957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3480101957},{"type":"v1","addr":"172.21.15.187:6834","nonce":3480101957}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5548":{"gid":5548,"name":"d","rank":1,"incarnation":80,"state":"up:active","state_seq":7,"addr":"172.21.15.187:6835/567389303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6833","nonce":567389303},{"type":"v1","addr":"172.21.15.187:6835","nonce":567389303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7365":{"gid":7365,"name":"a","rank":0,"incarnation":74,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6836/3411392567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":3411392567},{"type":"v1","addr":"172.21.15.131:6836","nonce":3411392567}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7365,"qdb_cluster":[5545,7365,5548]},"id":6}]} 2024-05-26T22:54:41.402 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 85 2024-05-26T22:54:41.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.404+0000 7f29b3600640 1 -- 172.21.15.131:0/1814582893 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f298c06be60 msgr2=0x7f298c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:41.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.404+0000 7f29b3600640 1 --2- 172.21.15.131:0/1814582893 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f298c06be60 0x7f298c06e320 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f29ac165ff0 tx=0x7f29a0008040 comp rx=0 tx=0).stop 2024-05-26T22:54:41.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.404+0000 7f29b3600640 1 -- 172.21.15.131:0/1814582893 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f29ac0ae490 msgr2=0x7f29ac164a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:41.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.404+0000 7f29b3600640 1 --2- 172.21.15.131:0/1814582893 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f29ac0ae490 0x7f29ac164a40 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f29ac071090 tx=0x7f29a400b4a0 comp rx=0 tx=0).stop 2024-05-26T22:54:41.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.405+0000 7f29b3600640 1 -- 172.21.15.131:0/1814582893 shutdown_connections 2024-05-26T22:54:41.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.405+0000 7f29b3600640 1 --2- 172.21.15.131:0/1814582893 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f298c06be60 0x7f298c06e320 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:41.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.405+0000 7f29b3600640 1 --2- 172.21.15.131:0/1814582893 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f29ac0ae490 0x7f29ac164a40 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:41.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.405+0000 7f29b3600640 1 --2- 172.21.15.131:0/1814582893 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f29ac0ae010 0x7f29ac164500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:41.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.405+0000 7f29b3600640 1 --2- 172.21.15.131:0/1814582893 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2994003f80 0x7f29ac14d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:41.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.405+0000 7f29b3600640 1 -- 172.21.15.131:0/1814582893 >> 172.21.15.131:0/1814582893 conn(0x7f29ac0b1e90 msgr2=0x7f29ac071ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:41.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.405+0000 7f29b3600640 1 -- 172.21.15.131:0/1814582893 shutdown_connections 2024-05-26T22:54:41.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.405+0000 7f29b3600640 1 -- 172.21.15.131:0/1814582893 wait complete. 2024-05-26T22:54:41.422 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-26T22:54:41.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.613+0000 7f14aa200640 1 Processor -- start 2024-05-26T22:54:41.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.613+0000 7f14aa200640 1 -- start start 2024-05-26T22:54:41.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.614+0000 7f14aa200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f14a41569d0 0x7f14a40b3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:41.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.615+0000 7f14aa200640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14a415cb20 con 0x7f14a40ad250 2024-05-26T22:54:41.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.615+0000 7f14aa200640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14a415ce00 con 0x7f14a406ee20 2024-05-26T22:54:41.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.615+0000 7f14aa200640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f14a415d0e0 con 0x7f14a41569d0 2024-05-26T22:54:41.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.615+0000 7f14a3e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f14a41569d0 0x7f14a40b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:41.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.615+0000 7f14a3e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f14a41569d0 0x7f14a40b3190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44350/0 (socket says 172.21.15.131:44350) 2024-05-26T22:54:41.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.615+0000 7f14a3e00640 1 -- 172.21.15.131:0/343792694 learned_addr learned my addr 172.21.15.131:0/343792694 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:54:41.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.615+0000 7f14a2a00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f14a40ad250 0x7f14a4160010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:58206/0 (socket says 172.21.15.131:58206) 2024-05-26T22:54:41.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.615+0000 7f14a3e00640 1 -- 172.21.15.131:0/343792694 >> v1:172.21.15.187:6789/0 conn(0x7f14a406ee20 legacy=0x7f14a41562c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:41.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.615+0000 7f14a3e00640 1 -- 172.21.15.131:0/343792694 >> v1:172.21.15.131:6789/0 conn(0x7f14a40ad250 legacy=0x7f14a4160010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:41.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.615+0000 7f14a3e00640 1 -- 172.21.15.131:0/343792694 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14a415d3c0 con 0x7f14a41569d0 2024-05-26T22:54:41.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.616+0000 7f14a3e00640 1 --2- 172.21.15.131:0/343792694 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f14a41569d0 0x7f14a40b3190 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f149c009870 tx=0x7f149c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=105e62c507d4f6b server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:41.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.616+0000 7f14a2000640 1 -- 172.21.15.131:0/343792694 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f149c037070 con 0x7f14a41569d0 2024-05-26T22:54:41.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.616+0000 7f14a2000640 1 -- 172.21.15.131:0/343792694 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f149c032440 con 0x7f14a41569d0 2024-05-26T22:54:41.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.616+0000 7f14a2000640 1 -- 172.21.15.131:0/343792694 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f149c032810 con 0x7f14a41569d0 2024-05-26T22:54:41.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.617+0000 7f14aa200640 1 -- 172.21.15.131:0/343792694 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f14a41569d0 msgr2=0x7f14a40b3190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:41.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.617+0000 7f14aa200640 1 --2- 172.21.15.131:0/343792694 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f14a41569d0 0x7f14a40b3190 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f149c009870 tx=0x7f149c02a210 comp rx=0 tx=0).stop 2024-05-26T22:54:41.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.617+0000 7f14aa200640 1 -- 172.21.15.131:0/343792694 shutdown_connections 2024-05-26T22:54:41.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.617+0000 7f14aa200640 1 --2- 172.21.15.131:0/343792694 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f14a41569d0 0x7f14a40b3190 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:41.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.617+0000 7f14aa200640 1 -- 172.21.15.131:0/343792694 >> 172.21.15.131:0/343792694 conn(0x7f14a40b1e90 msgr2=0x7f14a40b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:41.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.617+0000 7f14aa200640 1 -- 172.21.15.131:0/343792694 shutdown_connections 2024-05-26T22:54:41.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.617+0000 7f14aa200640 1 -- 172.21.15.131:0/343792694 wait complete. 2024-05-26T22:54:41.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.619+0000 7f14aa200640 1 Processor -- start 2024-05-26T22:54:41.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.619+0000 7f14aa200640 1 -- start start 2024-05-26T22:54:41.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.620+0000 7f14aa200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f14a406ee20 0x7f14a414dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:41.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.620+0000 7f14aa200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f14a40ad250 0x7f14a414b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:41.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.620+0000 7f14aa200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f14a41569d0 0x7f14a41f51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:41.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.620+0000 7f14aa200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f14a415c900 con 0x7f14a406ee20 2024-05-26T22:54:41.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.620+0000 7f14aa200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f14a415c780 con 0x7f14a40ad250 2024-05-26T22:54:41.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.620+0000 7f14aa200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f14a406f200 con 0x7f14a41569d0 2024-05-26T22:54:41.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.620+0000 7f14a3400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f14a406ee20 0x7f14a414dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:41.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.620+0000 7f14a3400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f14a406ee20 0x7f14a414dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44366/0 (socket says 172.21.15.131:44366) 2024-05-26T22:54:41.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.620+0000 7f14a3400640 1 -- 172.21.15.131:0/3735964325 learned_addr learned my addr 172.21.15.131:0/3735964325 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:54:41.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.620+0000 7f14a3400640 1 -- 172.21.15.131:0/3735964325 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f14a41569d0 msgr2=0x7f14a41f51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:41.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.620+0000 7f14a3400640 1 --2- 172.21.15.131:0/3735964325 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f14a41569d0 0x7f14a41f51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:41.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.620+0000 7f14a3400640 1 -- 172.21.15.131:0/3735964325 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f14a40ad250 msgr2=0x7f14a414b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:41.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.620+0000 7f14a3400640 1 --2- 172.21.15.131:0/3735964325 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f14a40ad250 0x7f14a414b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:41.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.620+0000 7f14a3400640 1 -- 172.21.15.131:0/3735964325 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14a41516e0 con 0x7f14a406ee20 2024-05-26T22:54:41.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.621+0000 7f14a3400640 1 --2- 172.21.15.131:0/3735964325 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f14a406ee20 0x7f14a414dee0 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f148c00c430 tx=0x7f148c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:41.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.621+0000 7f149b400640 1 -- 172.21.15.131:0/3735964325 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f148c011790 con 0x7f14a406ee20 2024-05-26T22:54:41.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.621+0000 7f149b400640 1 -- 172.21.15.131:0/3735964325 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f148c017070 con 0x7f14a406ee20 2024-05-26T22:54:41.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.621+0000 7f149b400640 1 -- 172.21.15.131:0/3735964325 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f148c011bb0 con 0x7f14a406ee20 2024-05-26T22:54:41.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.621+0000 7f14aa200640 1 -- 172.21.15.131:0/3735964325 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14a41519c0 con 0x7f14a406ee20 2024-05-26T22:54:41.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.621+0000 7f14aa200640 1 -- 172.21.15.131:0/3735964325 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14a413ec60 con 0x7f14a406ee20 2024-05-26T22:54:41.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.622+0000 7f14aa200640 1 -- 172.21.15.131:0/3735964325 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14a40694e0 con 0x7f14a406ee20 2024-05-26T22:54:41.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.623+0000 7f149b400640 1 -- 172.21.15.131:0/3735964325 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f148c01c460 con 0x7f14a406ee20 2024-05-26T22:54:41.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.624+0000 7f149b400640 1 --2- 172.21.15.131:0/3735964325 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f148006be30 0x7f148006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:41.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.624+0000 7f149b400640 1 -- 172.21.15.131:0/3735964325 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5738+0+0 (secure 0 0 0) 0x7f148c08ad50 con 0x7f14a406ee20 2024-05-26T22:54:41.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.624+0000 7f14a2a00640 1 --2- 172.21.15.131:0/3735964325 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f148006be30 0x7f148006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:41.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.625+0000 7f14a2a00640 1 --2- 172.21.15.131:0/3735964325 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f148006be30 0x7f148006e2f0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f1494007fd0 tx=0x7f1494007d80 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:41.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.628+0000 7f149b400640 1 -- 172.21.15.131:0/3735964325 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f148c0596d0 con 0x7f14a406ee20 2024-05-26T22:54:41.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.845+0000 7f14aa200640 1 -- 172.21.15.131:0/3735964325 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f14a4155420 con 0x7f14a406ee20 2024-05-26T22:54:41.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.846+0000 7f149b400640 1 -- 172.21.15.131:0/3735964325 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) ==== 94+0+5231 (secure 0 0 0) 0x7f148c058fd0 con 0x7f14a406ee20 2024-05-26T22:54:41.846 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:54:41.847 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":85,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5533,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/993930297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6835","nonce":993930297},{"type":"v1","addr":"172.21.15.131:6837","nonce":993930297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-26T22:53:52.862328+0000","modified":"2024-05-26T22:54:28.015691+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7365,"mds_1":5548,"mds_2":5545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5545":{"gid":5545,"name":"b","rank":2,"incarnation":82,"state":"up:active","state_seq":7,"addr":"172.21.15.187:6834/3480101957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3480101957},{"type":"v1","addr":"172.21.15.187:6834","nonce":3480101957}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5548":{"gid":5548,"name":"d","rank":1,"incarnation":80,"state":"up:active","state_seq":7,"addr":"172.21.15.187:6835/567389303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6833","nonce":567389303},{"type":"v1","addr":"172.21.15.187:6835","nonce":567389303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7365":{"gid":7365,"name":"a","rank":0,"incarnation":74,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6836/3411392567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":3411392567},{"type":"v1","addr":"172.21.15.131:6836","nonce":3411392567}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7365,"qdb_cluster":[5545,7365,5548]},"id":6}]} 2024-05-26T22:54:41.847 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 85 2024-05-26T22:54:41.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.849+0000 7f14aa200640 1 -- 172.21.15.131:0/3735964325 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f148006be30 msgr2=0x7f148006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:41.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.849+0000 7f14aa200640 1 --2- 172.21.15.131:0/3735964325 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f148006be30 0x7f148006e2f0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f1494007fd0 tx=0x7f1494007d80 comp rx=0 tx=0).stop 2024-05-26T22:54:41.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.849+0000 7f14aa200640 1 -- 172.21.15.131:0/3735964325 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f14a406ee20 msgr2=0x7f14a414dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:41.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.849+0000 7f14aa200640 1 --2- 172.21.15.131:0/3735964325 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f14a406ee20 0x7f14a414dee0 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f148c00c430 tx=0x7f148c00c8f0 comp rx=0 tx=0).stop 2024-05-26T22:54:41.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.849+0000 7f14aa200640 1 -- 172.21.15.131:0/3735964325 shutdown_connections 2024-05-26T22:54:41.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.849+0000 7f14aa200640 1 --2- 172.21.15.131:0/3735964325 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f14a41569d0 0x7f14a41f51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:41.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.849+0000 7f14aa200640 1 --2- 172.21.15.131:0/3735964325 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f148006be30 0x7f148006e2f0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:41.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.849+0000 7f14aa200640 1 --2- 172.21.15.131:0/3735964325 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f14a40ad250 0x7f14a414b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:41.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.849+0000 7f14aa200640 1 --2- 172.21.15.131:0/3735964325 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f14a406ee20 0x7f14a414dee0 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:41.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.849+0000 7f14aa200640 1 -- 172.21.15.131:0/3735964325 >> 172.21.15.131:0/3735964325 conn(0x7f14a40b1e90 msgr2=0x7f14a406d470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:41.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.849+0000 7f14aa200640 1 -- 172.21.15.131:0/3735964325 shutdown_connections 2024-05-26T22:54:41.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:41.849+0000 7f14aa200640 1 -- 172.21.15.131:0/3735964325 wait complete. 2024-05-26T22:54:41.867 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-26T22:54:42.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.061+0000 7fb179a00640 1 Processor -- start 2024-05-26T22:54:42.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.061+0000 7fb179a00640 1 -- start start 2024-05-26T22:54:42.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.063+0000 7fb179a00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fb17406ee20 0x7fb1740b3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:42.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.063+0000 7fb179a00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb17415cb20 con 0x7fb1741569d0 2024-05-26T22:54:42.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.063+0000 7fb179a00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb17415ce00 con 0x7fb1740ad250 2024-05-26T22:54:42.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.063+0000 7fb179a00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fb17415d0e0 con 0x7fb17406ee20 2024-05-26T22:54:42.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.063+0000 7fb173400640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fb1741569d0 0x7fb174160010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:58210/0 (socket says 172.21.15.131:58210) 2024-05-26T22:54:42.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.063+0000 7fb173400640 1 -- 172.21.15.131:0/3255640122 learned_addr learned my addr 172.21.15.131:0/3255640122 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:54:42.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.064+0000 7fb172a00640 1 --2- 172.21.15.131:0/3255640122 >> v2:172.21.15.187:3300/0 conn(0x7fb17406ee20 0x7fb1740b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:42.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.064+0000 7fb172000640 1 -- 172.21.15.131:0/3255640122 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3687778309 0 0) 0x7fb17415cb20 con 0x7fb1741569d0 2024-05-26T22:54:42.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.064+0000 7fb172000640 1 -- 172.21.15.131:0/3255640122 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb158003520 con 0x7fb1741569d0 2024-05-26T22:54:42.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.064+0000 7fb172000640 1 -- 172.21.15.131:0/3255640122 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2478513022 0 0) 0x7fb158003520 con 0x7fb1741569d0 2024-05-26T22:54:42.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.064+0000 7fb172000640 1 -- 172.21.15.131:0/3255640122 >> v2:172.21.15.187:3300/0 conn(0x7fb17406ee20 msgr2=0x7fb1740b3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:42.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.064+0000 7fb172000640 1 --2- 172.21.15.131:0/3255640122 >> v2:172.21.15.187:3300/0 conn(0x7fb17406ee20 0x7fb1740b3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:42.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.064+0000 7fb172000640 1 -- 172.21.15.131:0/3255640122 >> v1:172.21.15.187:6789/0 conn(0x7fb1740ad250 legacy=0x7fb1741562c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:42.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.064+0000 7fb172000640 1 -- 172.21.15.131:0/3255640122 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb17415d3c0 con 0x7fb1741569d0 2024-05-26T22:54:42.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.064+0000 7fb172000640 1 -- 172.21.15.131:0/3255640122 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7fb16c0019f0 con 0x7fb1741569d0 2024-05-26T22:54:42.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.064+0000 7fb172000640 1 -- 172.21.15.131:0/3255640122 >> v1:172.21.15.131:6789/0 conn(0x7fb1741569d0 legacy=0x7fb174160010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:42.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.064+0000 7fb172000640 1 --2- 172.21.15.131:0/3255640122 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb1580039b0 0x7fb158003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:42.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.064+0000 7fb172000640 1 --2- 172.21.15.131:0/3255640122 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb158004800 0x7fb158004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:42.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.064+0000 7fb172000640 1 --2- 172.21.15.131:0/3255640122 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1580051c0 0x7fb15800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:42.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.064+0000 7fb172000640 1 -- 172.21.15.131:0/3255640122 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fb17415d0e0 con 0x7fb1580051c0 2024-05-26T22:54:42.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.064+0000 7fb172000640 1 -- 172.21.15.131:0/3255640122 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb15800c160 con 0x7fb1580039b0 2024-05-26T22:54:42.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.064+0000 7fb172000640 1 -- 172.21.15.131:0/3255640122 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb15800c360 con 0x7fb158004800 2024-05-26T22:54:42.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.065+0000 7fb173400640 1 --2- 172.21.15.131:0/3255640122 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1580051c0 0x7fb15800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:42.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.065+0000 7fb172a00640 1 --2- 172.21.15.131:0/3255640122 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb1580039b0 0x7fb158003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:42.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.065+0000 7fb173400640 1 -- 172.21.15.131:0/3255640122 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb158004800 msgr2=0x7fb158004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:42.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.065+0000 7fb173400640 1 --2- 172.21.15.131:0/3255640122 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb158004800 0x7fb158004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:42.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.065+0000 7fb173400640 1 -- 172.21.15.131:0/3255640122 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb1580039b0 msgr2=0x7fb158003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:42.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.065+0000 7fb173400640 1 --2- 172.21.15.131:0/3255640122 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb1580039b0 0x7fb158003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:42.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.065+0000 7fb173400640 1 -- 172.21.15.131:0/3255640122 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb158003520 con 0x7fb1580051c0 2024-05-26T22:54:42.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.066+0000 7fb173400640 1 -- 172.21.15.131:0/3255640122 reap_dead start 2024-05-26T22:54:42.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.066+0000 7fb173400640 1 --2- 172.21.15.131:0/3255640122 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1580051c0 0x7fb15800b9c0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7fb16c00a830 tx=0x7fb158005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9cb87aafb4517e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:42.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.066+0000 7fb172000640 1 -- 172.21.15.131:0/3255640122 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb16c00a480 con 0x7fb1580051c0 2024-05-26T22:54:42.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.066+0000 7fb172000640 1 -- 172.21.15.131:0/3255640122 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb16c02f070 con 0x7fb1580051c0 2024-05-26T22:54:42.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.066+0000 7fb172000640 1 -- 172.21.15.131:0/3255640122 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb16c02bc70 con 0x7fb1580051c0 2024-05-26T22:54:42.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.067+0000 7fb179a00640 1 -- 172.21.15.131:0/3255640122 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1580051c0 msgr2=0x7fb15800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:42.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.067+0000 7fb179a00640 1 --2- 172.21.15.131:0/3255640122 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1580051c0 0x7fb15800b9c0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7fb16c00a830 tx=0x7fb158005830 comp rx=0 tx=0).stop 2024-05-26T22:54:42.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.067+0000 7fb179a00640 1 -- 172.21.15.131:0/3255640122 shutdown_connections 2024-05-26T22:54:42.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.067+0000 7fb179a00640 1 --2- 172.21.15.131:0/3255640122 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1580051c0 0x7fb15800b9c0 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:42.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.067+0000 7fb179a00640 1 -- 172.21.15.131:0/3255640122 >> 172.21.15.131:0/3255640122 conn(0x7fb1740b1e90 msgr2=0x7fb1740b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:42.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.067+0000 7fb179a00640 1 -- 172.21.15.131:0/3255640122 shutdown_connections 2024-05-26T22:54:42.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.067+0000 7fb179a00640 1 -- 172.21.15.131:0/3255640122 wait complete. 2024-05-26T22:54:42.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.069+0000 7fb179a00640 1 Processor -- start 2024-05-26T22:54:42.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.069+0000 7fb179a00640 1 -- start start 2024-05-26T22:54:42.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.070+0000 7fb179a00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb1580051c0 0x7fb174150cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:42.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.070+0000 7fb179a00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb17415b230 0x7fb17415b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:42.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.070+0000 7fb179a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1741f3d40 0x7fb1741f41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:42.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.070+0000 7fb179a00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fb17415c900 con 0x7fb1741f3d40 2024-05-26T22:54:42.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.070+0000 7fb179a00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb17415c780 con 0x7fb17415b230 2024-05-26T22:54:42.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.070+0000 7fb179a00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb17406f200 con 0x7fb1580051c0 2024-05-26T22:54:42.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.070+0000 7fb173400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1741f3d40 0x7fb1741f41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:42.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.070+0000 7fb173400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1741f3d40 0x7fb1741f41c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44388/0 (socket says 172.21.15.131:44388) 2024-05-26T22:54:42.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.070+0000 7fb173400640 1 -- 172.21.15.131:0/3868166421 learned_addr learned my addr 172.21.15.131:0/3868166421 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:54:42.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.070+0000 7fb172a00640 1 --2- 172.21.15.131:0/3868166421 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb1580051c0 0x7fb174150cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:42.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.070+0000 7fb173400640 1 -- 172.21.15.131:0/3868166421 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb1580051c0 msgr2=0x7fb174150cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:42.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.070+0000 7fb173400640 1 --2- 172.21.15.131:0/3868166421 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb1580051c0 0x7fb174150cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:42.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.070+0000 7fb173400640 1 -- 172.21.15.131:0/3868166421 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb17415b230 msgr2=0x7fb17415b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:42.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.070+0000 7fb16a000640 1 --2- 172.21.15.131:0/3868166421 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb17415b230 0x7fb17415b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:42.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.070+0000 7fb173400640 1 --2- 172.21.15.131:0/3868166421 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb17415b230 0x7fb17415b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:42.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.070+0000 7fb173400640 1 -- 172.21.15.131:0/3868166421 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb174151200 con 0x7fb1741f3d40 2024-05-26T22:54:42.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.071+0000 7fb173400640 1 --2- 172.21.15.131:0/3868166421 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1741f3d40 0x7fb1741f41c0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7fb16c00a830 tx=0x7fb16c02d070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:42.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.071+0000 7fb16b400640 1 -- 172.21.15.131:0/3868166421 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb16c044070 con 0x7fb1741f3d40 2024-05-26T22:54:42.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.071+0000 7fb16b400640 1 -- 172.21.15.131:0/3868166421 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb16c034670 con 0x7fb1741f3d40 2024-05-26T22:54:42.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.071+0000 7fb179a00640 1 -- 172.21.15.131:0/3868166421 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1741f4b60 con 0x7fb1741f3d40 2024-05-26T22:54:42.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.071+0000 7fb16b400640 1 -- 172.21.15.131:0/3868166421 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb16c00a480 con 0x7fb1741f3d40 2024-05-26T22:54:42.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.071+0000 7fb179a00640 1 -- 172.21.15.131:0/3868166421 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1741f5120 con 0x7fb1741f3d40 2024-05-26T22:54:42.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.072+0000 7fb179a00640 1 -- 172.21.15.131:0/3868166421 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1740694e0 con 0x7fb1741f3d40 2024-05-26T22:54:42.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.073+0000 7fb16b400640 1 -- 172.21.15.131:0/3868166421 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb16c047070 con 0x7fb1741f3d40 2024-05-26T22:54:42.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.074+0000 7fb16b400640 1 --2- 172.21.15.131:0/3868166421 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb15006be30 0x7fb15006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:42.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.074+0000 7fb16b400640 1 -- 172.21.15.131:0/3868166421 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5738+0+0 (secure 0 0 0) 0x7fb16c0aa9c0 con 0x7fb1741f3d40 2024-05-26T22:54:42.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.075+0000 7fb172a00640 1 --2- 172.21.15.131:0/3868166421 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb15006be30 0x7fb15006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:42.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.075+0000 7fb172a00640 1 --2- 172.21.15.131:0/3868166421 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb15006be30 0x7fb15006e2f0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fb1640098d0 tx=0x7fb164008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:42.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.079+0000 7fb16b400640 1 -- 172.21.15.131:0/3868166421 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fb16c0ace20 con 0x7fb1741f3d40 2024-05-26T22:54:42.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.263+0000 7fb179a00640 1 -- 172.21.15.131:0/3868166421 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb1740694e0 con 0x7fb1741f3d40 2024-05-26T22:54:42.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.266+0000 7fb16b400640 1 -- 172.21.15.131:0/3868166421 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+16734 (secure 0 0 0) 0x7fb16c079350 con 0x7fb1741f3d40 2024-05-26T22:54:42.266 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:54:42.267 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":84,"fsid":"46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f","created":"2024-05-26T22:47:57.288905+0000","modified":"2024-05-26T22:54:07.109294+0000","last_up_change":"2024-05-26T22:48:09.602207+0000","last_in_change":"2024-05-26T22:48:00.687341+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-26T22:48:10.463339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-05-26T22:53:50.625629+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-05-26T22:53:51.864320+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"814eaf62-a18d-494b-8e27-fd8fb88adea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6801","nonce":2350940051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6803","nonce":2350940051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6807","nonce":2350940051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6805","nonce":2350940051}]},"public_addr":"172.21.15.131:6801/2350940051","cluster_addr":"172.21.15.131:6803/2350940051","heartbeat_back_addr":"172.21.15.131:6807/2350940051","heartbeat_front_addr":"172.21.15.131:6805/2350940051","state":["exists","up"]},{"osd":1,"uuid":"42e66287-f6c6-4e26-af3c-6dfaa3536d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6825","nonce":1525528247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6827","nonce":1525528247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6831","nonce":1525528247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6829","nonce":1525528247}]},"public_addr":"172.21.15.131:6825/1525528247","cluster_addr":"172.21.15.131:6827/1525528247","heartbeat_back_addr":"172.21.15.131:6831/1525528247","heartbeat_front_addr":"172.21.15.131:6829/1525528247","state":["exists","up"]},{"osd":2,"uuid":"0da34673-04a2-4fd0-b57a-0ff2b679b043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6817","nonce":3627842210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6819","nonce":3627842210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6823","nonce":3627842210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6821","nonce":3627842210}]},"public_addr":"172.21.15.131:6817/3627842210","cluster_addr":"172.21.15.131:6819/3627842210","heartbeat_back_addr":"172.21.15.131:6823/3627842210","heartbeat_front_addr":"172.21.15.131:6821/3627842210","state":["exists","up"]},{"osd":3,"uuid":"cf117a27-d6b2-43d3-ae52-b47988bf8eed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6809","nonce":350790591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6811","nonce":350790591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6815","nonce":350790591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6813","nonce":350790591}]},"public_addr":"172.21.15.131:6809/350790591","cluster_addr":"172.21.15.131:6811/350790591","heartbeat_back_addr":"172.21.15.131:6815/350790591","heartbeat_front_addr":"172.21.15.131:6813/350790591","state":["exists","up"]},{"osd":4,"uuid":"a092ccfc-be89-4a84-88d5-0f781fa715eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6816","nonce":244560995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6819","nonce":244560995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6823","nonce":244560995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6821","nonce":244560995}]},"public_addr":"172.21.15.187:6816/244560995","cluster_addr":"172.21.15.187:6819/244560995","heartbeat_back_addr":"172.21.15.187:6823/244560995","heartbeat_front_addr":"172.21.15.187:6821/244560995","state":["exists","up"]},{"osd":5,"uuid":"7513ba69-dd00-4a35-b190-9e8cc3b2bdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6801","nonce":324619398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6803","nonce":324619398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6812","nonce":324619398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6807","nonce":324619398}]},"public_addr":"172.21.15.187:6801/324619398","cluster_addr":"172.21.15.187:6803/324619398","heartbeat_back_addr":"172.21.15.187:6812/324619398","heartbeat_front_addr":"172.21.15.187:6807/324619398","state":["exists","up"]},{"osd":6,"uuid":"50081810-c093-4d2f-a4d7-b7c695c5400d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6806","nonce":1391347786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6809","nonce":1391347786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6818","nonce":1391347786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6813","nonce":1391347786}]},"public_addr":"172.21.15.187:6806/1391347786","cluster_addr":"172.21.15.187:6809/1391347786","heartbeat_back_addr":"172.21.15.187:6818/1391347786","heartbeat_front_addr":"172.21.15.187:6813/1391347786","state":["exists","up"]},{"osd":7,"uuid":"775dae05-54f5-4ad4-bbce-92863c6ed2ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6825","nonce":1220065622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6827","nonce":1220065622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6831","nonce":1220065622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6829","nonce":1220065622}]},"public_addr":"172.21.15.187:6825/1220065622","cluster_addr":"172.21.15.187:6827/1220065622","heartbeat_back_addr":"172.21.15.187:6831/1220065622","heartbeat_front_addr":"172.21.15.187:6829/1220065622","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:05.983520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.299516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.067323+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.307748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.018742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.258775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.010912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.295355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.38","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-26T22:54:42.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.269+0000 7fb179a00640 1 -- 172.21.15.131:0/3868166421 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb15006be30 msgr2=0x7fb15006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:42.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.269+0000 7fb179a00640 1 --2- 172.21.15.131:0/3868166421 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb15006be30 0x7fb15006e2f0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fb1640098d0 tx=0x7fb164008040 comp rx=0 tx=0).stop 2024-05-26T22:54:42.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.269+0000 7fb179a00640 1 -- 172.21.15.131:0/3868166421 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1741f3d40 msgr2=0x7fb1741f41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:42.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.269+0000 7fb179a00640 1 --2- 172.21.15.131:0/3868166421 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1741f3d40 0x7fb1741f41c0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7fb16c00a830 tx=0x7fb16c02d070 comp rx=0 tx=0).stop 2024-05-26T22:54:42.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.269+0000 7fb179a00640 1 -- 172.21.15.131:0/3868166421 shutdown_connections 2024-05-26T22:54:42.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.269+0000 7fb179a00640 1 --2- 172.21.15.131:0/3868166421 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb15006be30 0x7fb15006e2f0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:42.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.269+0000 7fb179a00640 1 --2- 172.21.15.131:0/3868166421 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1741f3d40 0x7fb1741f41c0 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:42.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.269+0000 7fb179a00640 1 --2- 172.21.15.131:0/3868166421 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb17415b230 0x7fb17415b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:42.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.269+0000 7fb179a00640 1 --2- 172.21.15.131:0/3868166421 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb1580051c0 0x7fb174150cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:42.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.269+0000 7fb179a00640 1 -- 172.21.15.131:0/3868166421 >> 172.21.15.131:0/3868166421 conn(0x7fb1740b1e90 msgr2=0x7fb17415e7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:42.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.270+0000 7fb179a00640 1 -- 172.21.15.131:0/3868166421 shutdown_connections 2024-05-26T22:54:42.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.270+0000 7fb179a00640 1 -- 172.21.15.131:0/3868166421 wait complete. 2024-05-26T22:54:42.288 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-26T22:54:42.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.489+0000 7fa92fe00640 1 Processor -- start 2024-05-26T22:54:42.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.490+0000 7fa92fe00640 1 -- start start 2024-05-26T22:54:42.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.491+0000 7fa92fe00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa93006ebb0 0x7fa93015edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:42.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.491+0000 7fa92fe00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fa93015f310 0x7fa9300cfa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:42.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.491+0000 7fa92fe00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa93015fb90 con 0x7fa93013dff0 2024-05-26T22:54:42.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.491+0000 7fa92fe00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fa93015baa0 con 0x7fa93015f310 2024-05-26T22:54:42.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.491+0000 7fa92fe00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa93015bd80 con 0x7fa93006ebb0 2024-05-26T22:54:42.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.492+0000 7fa92f400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fa93015f310 0x7fa9300cfa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:42.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.492+0000 7fa92f400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fa93015f310 0x7fa9300cfa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44402/0 (socket says 172.21.15.131:44402) 2024-05-26T22:54:42.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.492+0000 7fa92f400640 1 -- 172.21.15.131:0/2081420023 learned_addr learned my addr 172.21.15.131:0/2081420023 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:54:42.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.492+0000 7fa926000640 1 --2- 172.21.15.131:0/2081420023 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa93006ebb0 0x7fa93015edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:42.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.492+0000 7fa92f400640 1 -- 172.21.15.131:0/2081420023 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa93006ebb0 msgr2=0x7fa93015edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:42.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.492+0000 7fa92f400640 1 --2- 172.21.15.131:0/2081420023 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa93006ebb0 0x7fa93015edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:42.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.492+0000 7fa92f400640 1 -- 172.21.15.131:0/2081420023 >> v1:172.21.15.187:6789/0 conn(0x7fa93013dff0 legacy=0x7fa93006e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:42.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.492+0000 7fa92f400640 1 -- 172.21.15.131:0/2081420023 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa93015c060 con 0x7fa93015f310 2024-05-26T22:54:42.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.492+0000 7fa926000640 1 --2- 172.21.15.131:0/2081420023 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa93006ebb0 0x7fa93015edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:54:42.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.493+0000 7fa92f400640 1 --2- 172.21.15.131:0/2081420023 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa93015f310 0x7fa9300cfa50 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7fa928009e50 tx=0x7fa92802a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f365d79cf8c9762f server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:42.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.493+0000 7fa92e000640 1 -- 172.21.15.131:0/2081420023 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa928036070 con 0x7fa93015f310 2024-05-26T22:54:42.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.493+0000 7fa92e000640 1 -- 172.21.15.131:0/2081420023 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa928031600 con 0x7fa93015f310 2024-05-26T22:54:42.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.493+0000 7fa92e000640 1 -- 172.21.15.131:0/2081420023 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9280319d0 con 0x7fa93015f310 2024-05-26T22:54:42.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.494+0000 7fa92fe00640 1 -- 172.21.15.131:0/2081420023 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa93015f310 msgr2=0x7fa9300cfa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:42.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.494+0000 7fa92fe00640 1 --2- 172.21.15.131:0/2081420023 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa93015f310 0x7fa9300cfa50 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7fa928009e50 tx=0x7fa92802a6d0 comp rx=0 tx=0).stop 2024-05-26T22:54:42.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.494+0000 7fa92fe00640 1 -- 172.21.15.131:0/2081420023 shutdown_connections 2024-05-26T22:54:42.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.494+0000 7fa92fe00640 1 --2- 172.21.15.131:0/2081420023 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa93015f310 0x7fa9300cfa50 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:42.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.494+0000 7fa92fe00640 1 --2- 172.21.15.131:0/2081420023 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa93006ebb0 0x7fa93015edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:42.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.494+0000 7fa92fe00640 1 -- 172.21.15.131:0/2081420023 >> 172.21.15.131:0/2081420023 conn(0x7fa93006c420 msgr2=0x7fa93006c830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:42.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.494+0000 7fa92fe00640 1 -- 172.21.15.131:0/2081420023 shutdown_connections 2024-05-26T22:54:42.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.494+0000 7fa92fe00640 1 -- 172.21.15.131:0/2081420023 wait complete. 2024-05-26T22:54:42.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.496+0000 7fa92fe00640 1 Processor -- start 2024-05-26T22:54:42.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.496+0000 7fa92fe00640 1 -- start start 2024-05-26T22:54:42.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.497+0000 7fa92fe00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa93006ebb0 0x7fa930139bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:42.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.497+0000 7fa92fe00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa93013dff0 0x7fa9301419d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:42.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.497+0000 7fa92fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa93015f310 0x7fa930141f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:42.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.497+0000 7fa92fe00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa93015c580 con 0x7fa93015f310 2024-05-26T22:54:42.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.497+0000 7fa92fe00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa93015f7c0 con 0x7fa93006ebb0 2024-05-26T22:54:42.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.497+0000 7fa92fe00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa930142450 con 0x7fa93013dff0 2024-05-26T22:54:42.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.497+0000 7fa92f400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa93015f310 0x7fa930141f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:42.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.497+0000 7fa92f400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa93015f310 0x7fa930141f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44406/0 (socket says 172.21.15.131:44406) 2024-05-26T22:54:42.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.497+0000 7fa92f400640 1 -- 172.21.15.131:0/1793776998 learned_addr learned my addr 172.21.15.131:0/1793776998 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:54:42.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.498+0000 7fa92ea00640 1 --2- 172.21.15.131:0/1793776998 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa93006ebb0 0x7fa930139bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:42.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.498+0000 7fa926000640 1 --2- 172.21.15.131:0/1793776998 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa93013dff0 0x7fa9301419d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:42.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.498+0000 7fa92f400640 1 -- 172.21.15.131:0/1793776998 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa93013dff0 msgr2=0x7fa9301419d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:42.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.498+0000 7fa92f400640 1 --2- 172.21.15.131:0/1793776998 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa93013dff0 0x7fa9301419d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:42.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.498+0000 7fa92f400640 1 -- 172.21.15.131:0/1793776998 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa93006ebb0 msgr2=0x7fa930139bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:42.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.498+0000 7fa92f400640 1 --2- 172.21.15.131:0/1793776998 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa93006ebb0 0x7fa930139bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:42.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.498+0000 7fa92f400640 1 -- 172.21.15.131:0/1793776998 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa930137a60 con 0x7fa93015f310 2024-05-26T22:54:42.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.498+0000 7fa926000640 1 --2- 172.21.15.131:0/1793776998 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa93013dff0 0x7fa9301419d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:54:42.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.498+0000 7fa92f400640 1 --2- 172.21.15.131:0/1793776998 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa93015f310 0x7fa930141f10 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7fa928009f70 tx=0x7fa928004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:42.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.499+0000 7fa927400640 1 -- 172.21.15.131:0/1793776998 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa928036050 con 0x7fa93015f310 2024-05-26T22:54:42.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.499+0000 7fa927400640 1 -- 172.21.15.131:0/1793776998 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa928042070 con 0x7fa93015f310 2024-05-26T22:54:42.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.499+0000 7fa92fe00640 1 -- 172.21.15.131:0/1793776998 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa930137310 con 0x7fa93015f310 2024-05-26T22:54:42.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.499+0000 7fa927400640 1 -- 172.21.15.131:0/1793776998 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa928009510 con 0x7fa93015f310 2024-05-26T22:54:42.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.499+0000 7fa92fe00640 1 -- 172.21.15.131:0/1793776998 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa930137d80 con 0x7fa93015f310 2024-05-26T22:54:42.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.500+0000 7fa92fe00640 1 -- 172.21.15.131:0/1793776998 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9300b3c40 con 0x7fa93015f310 2024-05-26T22:54:42.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.501+0000 7fa927400640 1 -- 172.21.15.131:0/1793776998 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa928007bd0 con 0x7fa93015f310 2024-05-26T22:54:42.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.502+0000 7fa927400640 1 --2- 172.21.15.131:0/1793776998 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa90c06be30 0x7fa90c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:42.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.502+0000 7fa927400640 1 -- 172.21.15.131:0/1793776998 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5738+0+0 (secure 0 0 0) 0x7fa9280abe60 con 0x7fa93015f310 2024-05-26T22:54:42.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.502+0000 7fa92ea00640 1 --2- 172.21.15.131:0/1793776998 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa90c06be30 0x7fa90c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:42.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.502+0000 7fa92ea00640 1 --2- 172.21.15.131:0/1793776998 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa90c06be30 0x7fa90c06e2f0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fa93013c1e0 tx=0x7fa920009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:42.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.507+0000 7fa927400640 1 -- 172.21.15.131:0/1793776998 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fa9300b3c40 con 0x7fa93015f310 2024-05-26T22:54:42.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.744+0000 7fa92fe00640 1 -- 172.21.15.131:0/1793776998 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fa9300b43f0 con 0x7fa93015f310 2024-05-26T22:54:42.745 INFO:tasks.ceph.mon.a.smithi131.stderr:2024-05-26T22:54:42.745+0000 7fcf0d200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-26T22:54:42.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.748+0000 7fa927400640 1 -- 172.21.15.131:0/1793776998 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v86) ==== 161+0+0 (secure 0 0 0) 0x7fa9300b43f0 con 0x7fa93015f310 2024-05-26T22:54:42.748 INFO:teuthology.orchestra.run.smithi131.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-26T22:54:42.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.753+0000 7fa92fe00640 1 -- 172.21.15.131:0/1793776998 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa90c06be30 msgr2=0x7fa90c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:42.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.753+0000 7fa92fe00640 1 --2- 172.21.15.131:0/1793776998 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa90c06be30 0x7fa90c06e2f0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fa93013c1e0 tx=0x7fa920009040 comp rx=0 tx=0).stop 2024-05-26T22:54:42.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.753+0000 7fa92fe00640 1 -- 172.21.15.131:0/1793776998 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa93015f310 msgr2=0x7fa930141f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:42.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.753+0000 7fa92fe00640 1 --2- 172.21.15.131:0/1793776998 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa93015f310 0x7fa930141f10 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7fa928009f70 tx=0x7fa928004a40 comp rx=0 tx=0).stop 2024-05-26T22:54:42.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.754+0000 7fa92fe00640 1 -- 172.21.15.131:0/1793776998 shutdown_connections 2024-05-26T22:54:42.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.754+0000 7fa92fe00640 1 --2- 172.21.15.131:0/1793776998 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa93013dff0 0x7fa9301419d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:42.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.754+0000 7fa92fe00640 1 --2- 172.21.15.131:0/1793776998 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa90c06be30 0x7fa90c06e2f0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:42.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.754+0000 7fa92fe00640 1 --2- 172.21.15.131:0/1793776998 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa93015f310 0x7fa930141f10 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:42.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.754+0000 7fa92fe00640 1 --2- 172.21.15.131:0/1793776998 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa93006ebb0 0x7fa930139bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:42.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.754+0000 7fa92fe00640 1 -- 172.21.15.131:0/1793776998 >> 172.21.15.131:0/1793776998 conn(0x7fa93006c420 msgr2=0x7fa9300b6740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:42.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.754+0000 7fa92fe00640 1 -- 172.21.15.131:0/1793776998 shutdown_connections 2024-05-26T22:54:42.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.754+0000 7fa92fe00640 1 -- 172.21.15.131:0/1793776998 wait complete. 2024-05-26T22:54:42.775 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-26T22:54:42.828 INFO:tasks.ceph.mds.a.smithi131.stdout:starting mds.a at 2024-05-26T22:54:42.872 INFO:tasks.ceph.mds.b.smithi187.stdout:starting mds.b at 2024-05-26T22:54:42.873 INFO:tasks.ceph.mds.d.smithi187.stdout:starting mds.d at 2024-05-26T22:54:42.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.956+0000 7fd899600640 1 Processor -- start 2024-05-26T22:54:42.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.956+0000 7fd899600640 1 -- start start 2024-05-26T22:54:42.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.957+0000 7fd899600640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fd894156660 0x7fd8940cfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:42.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.958+0000 7fd899600640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd89415fcc0 con 0x7fd8940d01e0 2024-05-26T22:54:42.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.958+0000 7fd899600640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd89415ffa0 con 0x7fd89415cf00 2024-05-26T22:54:42.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.958+0000 7fd899600640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fd89415d310 con 0x7fd894156660 2024-05-26T22:54:42.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.958+0000 7fd893e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fd894156660 0x7fd8940cfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:42.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.958+0000 7fd88b400640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fd8940d01e0 0x7fd89415c7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:58222/0 (socket says 172.21.15.131:58222) 2024-05-26T22:54:42.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.958+0000 7fd88b400640 1 -- 172.21.15.131:0/1688325048 learned_addr learned my addr 172.21.15.131:0/1688325048 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:54:42.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.958+0000 7fd893e00640 1 -- 172.21.15.131:0/1688325048 >> v1:172.21.15.187:6789/0 conn(0x7fd89415cf00 legacy=0x7fd894144ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:42.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.958+0000 7fd893e00640 1 -- 172.21.15.131:0/1688325048 >> v1:172.21.15.131:6789/0 conn(0x7fd8940d01e0 legacy=0x7fd89415c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:42.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.958+0000 7fd893e00640 1 -- 172.21.15.131:0/1688325048 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd894160280 con 0x7fd894156660 2024-05-26T22:54:42.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.958+0000 7fd893400640 1 -- 172.21.15.131:0/1688325048 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2656925270 0 0) 0x7fd89415fcc0 con 0x7fd8940d01e0 2024-05-26T22:54:42.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.959+0000 7fd893e00640 1 --2- 172.21.15.131:0/1688325048 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd894156660 0x7fd8940cfca0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7fd884009d90 tx=0x7fd88402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2da25d0a468bb4ff server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:42.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.960+0000 7fd893400640 1 -- 172.21.15.131:0/1688325048 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd884007740 con 0x7fd894156660 2024-05-26T22:54:42.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.960+0000 7fd893400640 1 -- 172.21.15.131:0/1688325048 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8840078e0 con 0x7fd894156660 2024-05-26T22:54:42.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.960+0000 7fd893400640 1 -- 172.21.15.131:0/1688325048 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd884007cb0 con 0x7fd894156660 2024-05-26T22:54:42.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.960+0000 7fd899600640 1 -- 172.21.15.131:0/1688325048 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd894156660 msgr2=0x7fd8940cfca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:42.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.960+0000 7fd899600640 1 --2- 172.21.15.131:0/1688325048 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd894156660 0x7fd8940cfca0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7fd884009d90 tx=0x7fd88402a6f0 comp rx=0 tx=0).stop 2024-05-26T22:54:42.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.960+0000 7fd899600640 1 -- 172.21.15.131:0/1688325048 shutdown_connections 2024-05-26T22:54:42.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.960+0000 7fd899600640 1 --2- 172.21.15.131:0/1688325048 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd894156660 0x7fd8940cfca0 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:42.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.960+0000 7fd899600640 1 -- 172.21.15.131:0/1688325048 >> 172.21.15.131:0/1688325048 conn(0x7fd8940cc7f0 msgr2=0x7fd8940ccbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:42.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.961+0000 7fd899600640 1 -- 172.21.15.131:0/1688325048 shutdown_connections 2024-05-26T22:54:42.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.961+0000 7fd899600640 1 -- 172.21.15.131:0/1688325048 wait complete. 2024-05-26T22:54:42.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.962+0000 7fd899600640 1 Processor -- start 2024-05-26T22:54:42.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.962+0000 7fd899600640 1 -- start start 2024-05-26T22:54:42.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.963+0000 7fd899600640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd8940d01e0 0x7fd894147b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:42.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.963+0000 7fd899600640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd894156660 0x7fd894148080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:42.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.963+0000 7fd899600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd89415cf00 0x7fd8941e93b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:42.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.963+0000 7fd899600640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fd894156a40 con 0x7fd89415cf00 2024-05-26T22:54:42.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.963+0000 7fd899600640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd89415d540 con 0x7fd894156660 2024-05-26T22:54:42.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.963+0000 7fd899600640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd894160a80 con 0x7fd8940d01e0 2024-05-26T22:54:42.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.964+0000 7fd898c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd89415cf00 0x7fd8941e93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:42.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.964+0000 7fd898c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd89415cf00 0x7fd8941e93b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44448/0 (socket says 172.21.15.131:44448) 2024-05-26T22:54:42.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.964+0000 7fd898c00640 1 -- 172.21.15.131:0/2760612633 learned_addr learned my addr 172.21.15.131:0/2760612633 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:54:42.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.964+0000 7fd88b400640 1 --2- 172.21.15.131:0/2760612633 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd894156660 0x7fd894148080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:42.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.964+0000 7fd893e00640 1 --2- 172.21.15.131:0/2760612633 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd8940d01e0 0x7fd894147b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:42.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.964+0000 7fd898c00640 1 -- 172.21.15.131:0/2760612633 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd8940d01e0 msgr2=0x7fd894147b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:42.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.964+0000 7fd898c00640 1 --2- 172.21.15.131:0/2760612633 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd8940d01e0 0x7fd894147b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:42.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.964+0000 7fd898c00640 1 -- 172.21.15.131:0/2760612633 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd894156660 msgr2=0x7fd894148080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:42.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.964+0000 7fd898c00640 1 --2- 172.21.15.131:0/2760612633 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd894156660 0x7fd894148080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:42.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.964+0000 7fd898c00640 1 -- 172.21.15.131:0/2760612633 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd894146aa0 con 0x7fd89415cf00 2024-05-26T22:54:42.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.964+0000 7fd898c00640 1 --2- 172.21.15.131:0/2760612633 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd89415cf00 0x7fd8941e93b0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7fd88c00c3c0 tx=0x7fd88c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:42.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.965+0000 7fd890c00640 1 -- 172.21.15.131:0/2760612633 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd88c017070 con 0x7fd89415cf00 2024-05-26T22:54:42.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.965+0000 7fd890c00640 1 -- 172.21.15.131:0/2760612633 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd88c0052d0 con 0x7fd89415cf00 2024-05-26T22:54:42.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.965+0000 7fd899600640 1 -- 172.21.15.131:0/2760612633 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd894146d80 con 0x7fd89415cf00 2024-05-26T22:54:42.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.965+0000 7fd890c00640 1 -- 172.21.15.131:0/2760612633 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd88c003810 con 0x7fd89415cf00 2024-05-26T22:54:42.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.965+0000 7fd899600640 1 -- 172.21.15.131:0/2760612633 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd894147340 con 0x7fd89415cf00 2024-05-26T22:54:42.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.966+0000 7fd899600640 1 -- 172.21.15.131:0/2760612633 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8940694e0 con 0x7fd89415cf00 2024-05-26T22:54:42.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.967+0000 7fd890c00640 1 -- 172.21.15.131:0/2760612633 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd88c01a070 con 0x7fd89415cf00 2024-05-26T22:54:42.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.968+0000 7fd890c00640 1 --2- 172.21.15.131:0/2760612633 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fd87006be30 0x7fd87006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:42.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.968+0000 7fd890c00640 1 -- 172.21.15.131:0/2760612633 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5996+0+0 (secure 0 0 0) 0x7fd88c08a290 con 0x7fd89415cf00 2024-05-26T22:54:42.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.968+0000 7fd893e00640 1 --2- 172.21.15.131:0/2760612633 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fd87006be30 0x7fd87006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:42.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.969+0000 7fd893e00640 1 --2- 172.21.15.131:0/2760612633 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fd87006be30 0x7fd87006e2f0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fd884034000 tx=0x7fd88402f000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:42.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:42.972+0000 7fd890c00640 1 -- 172.21.15.131:0/2760612633 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fd88c058b10 con 0x7fd89415cf00 2024-05-26T22:54:43.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:43.205+0000 7fd899600640 1 -- 172.21.15.131:0/2760612633 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fd89406ede0 con 0x7fd89415cf00 2024-05-26T22:54:43.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:43.752+0000 7fd890c00640 1 -- 172.21.15.131:0/2760612633 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v87) ==== 104+0+0 (secure 0 0 0) 0x7fd88c058410 con 0x7fd89415cf00 2024-05-26T22:54:43.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:43.757+0000 7fd899600640 1 -- 172.21.15.131:0/2760612633 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fd87006be30 msgr2=0x7fd87006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:43.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:43.757+0000 7fd899600640 1 --2- 172.21.15.131:0/2760612633 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fd87006be30 0x7fd87006e2f0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fd884034000 tx=0x7fd88402f000 comp rx=0 tx=0).stop 2024-05-26T22:54:43.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:43.757+0000 7fd899600640 1 -- 172.21.15.131:0/2760612633 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd89415cf00 msgr2=0x7fd8941e93b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:43.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:43.757+0000 7fd899600640 1 --2- 172.21.15.131:0/2760612633 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd89415cf00 0x7fd8941e93b0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7fd88c00c3c0 tx=0x7fd88c00c880 comp rx=0 tx=0).stop 2024-05-26T22:54:43.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:43.757+0000 7fd899600640 1 -- 172.21.15.131:0/2760612633 shutdown_connections 2024-05-26T22:54:43.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:43.757+0000 7fd899600640 1 --2- 172.21.15.131:0/2760612633 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fd87006be30 0x7fd87006e2f0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:43.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:43.757+0000 7fd899600640 1 --2- 172.21.15.131:0/2760612633 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd89415cf00 0x7fd8941e93b0 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:43.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:43.757+0000 7fd899600640 1 --2- 172.21.15.131:0/2760612633 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd894156660 0x7fd894148080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:43.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:43.757+0000 7fd899600640 1 --2- 172.21.15.131:0/2760612633 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd8940d01e0 0x7fd894147b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:43.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:43.757+0000 7fd899600640 1 -- 172.21.15.131:0/2760612633 >> 172.21.15.131:0/2760612633 conn(0x7fd8940cc7f0 msgr2=0x7fd894071b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:43.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:43.757+0000 7fd899600640 1 -- 172.21.15.131:0/2760612633 shutdown_connections 2024-05-26T22:54:43.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:43.757+0000 7fd899600640 1 -- 172.21.15.131:0/2760612633 wait complete. 2024-05-26T22:54:43.779 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-26T22:54:43.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:43.978+0000 7f00f7e00640 1 Processor -- start 2024-05-26T22:54:43.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:43.979+0000 7f00f7e00640 1 -- start start 2024-05-26T22:54:43.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:43.980+0000 7f00f7e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f00f8145880 0x7f00f815cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:43.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:43.980+0000 7f00f7e00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f00f815fba0 0x7f00f806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:43.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:43.980+0000 7f00f7e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f00f8160020 0x7f00f806da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:43.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:43.980+0000 7f00f7e00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f00f815c560 con 0x7f00f8145880 2024-05-26T22:54:43.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:43.980+0000 7f00f7e00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f00f815cb10 con 0x7f00f815fba0 2024-05-26T22:54:43.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:43.980+0000 7f00f7e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f00f815d4e0 con 0x7f00f8160020 2024-05-26T22:54:43.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:43.981+0000 7f00f6a00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f00f8145880 0x7f00f815cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:43.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:43.981+0000 7f00f6a00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f00f8145880 0x7f00f815cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44462/0 (socket says 172.21.15.131:44462) 2024-05-26T22:54:43.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:43.981+0000 7f00f6a00640 1 -- 172.21.15.131:0/78879704 learned_addr learned my addr 172.21.15.131:0/78879704 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:54:43.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:43.981+0000 7f00f6a00640 1 -- 172.21.15.131:0/78879704 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f00f8160020 msgr2=0x7f00f806da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:43.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:43.981+0000 7f00f6a00640 1 --2- 172.21.15.131:0/78879704 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f00f8160020 0x7f00f806da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:43.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:43.981+0000 7f00f6a00640 1 -- 172.21.15.131:0/78879704 >> v2:172.21.15.187:3300/0 conn(0x7f00f815fba0 msgr2=0x7f00f806d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:43.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:43.981+0000 7f00f6a00640 1 --2- 172.21.15.131:0/78879704 >> v2:172.21.15.187:3300/0 conn(0x7f00f815fba0 0x7f00f806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:43.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:43.981+0000 7f00f6a00640 1 -- 172.21.15.131:0/78879704 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00f8145c60 con 0x7f00f8145880 2024-05-26T22:54:43.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:43.982+0000 7f00f6a00640 1 --2- 172.21.15.131:0/78879704 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f00f8145880 0x7f00f815cfa0 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f00f80ce7d0 tx=0x7f00e802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ad9c6a37399d56d server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:43.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:43.982+0000 7f00f6000640 1 -- 172.21.15.131:0/78879704 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00e8037070 con 0x7f00f8145880 2024-05-26T22:54:43.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:43.982+0000 7f00f6000640 1 -- 172.21.15.131:0/78879704 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00e8032440 con 0x7f00f8145880 2024-05-26T22:54:43.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:43.982+0000 7f00f6000640 1 -- 172.21.15.131:0/78879704 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00e8032810 con 0x7f00f8145880 2024-05-26T22:54:43.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:43.983+0000 7f00f7e00640 1 -- 172.21.15.131:0/78879704 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f00f8145880 msgr2=0x7f00f815cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:43.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:43.983+0000 7f00f7e00640 1 --2- 172.21.15.131:0/78879704 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f00f8145880 0x7f00f815cfa0 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f00f80ce7d0 tx=0x7f00e802a1e0 comp rx=0 tx=0).stop 2024-05-26T22:54:43.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:43.983+0000 7f00f7e00640 1 -- 172.21.15.131:0/78879704 shutdown_connections 2024-05-26T22:54:43.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:43.983+0000 7f00f7e00640 1 --2- 172.21.15.131:0/78879704 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f00f8160020 0x7f00f806da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:43.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:43.983+0000 7f00f7e00640 1 --2- 172.21.15.131:0/78879704 >> v2:172.21.15.187:3300/0 conn(0x7f00f815fba0 0x7f00f806d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:43.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:43.983+0000 7f00f7e00640 1 --2- 172.21.15.131:0/78879704 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f00f8145880 0x7f00f815cfa0 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:43.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:43.983+0000 7f00f7e00640 1 -- 172.21.15.131:0/78879704 >> 172.21.15.131:0/78879704 conn(0x7f00f80b5410 msgr2=0x7f00f80b57f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:43.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:43.983+0000 7f00f7e00640 1 -- 172.21.15.131:0/78879704 shutdown_connections 2024-05-26T22:54:43.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:43.983+0000 7f00f7e00640 1 -- 172.21.15.131:0/78879704 wait complete. 2024-05-26T22:54:43.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:43.985+0000 7f00f7e00640 1 Processor -- start 2024-05-26T22:54:43.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:43.985+0000 7f00f7e00640 1 -- start start 2024-05-26T22:54:43.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:43.986+0000 7f00f7e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f00f8145880 0x7f00f815b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:43.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:43.986+0000 7f00f7e00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f00f815fba0 0x7f00f80cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:43.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:43.986+0000 7f00f7e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f00f8160020 0x7f00f80cdd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:43.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:43.986+0000 7f00f7e00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f00f815c7a0 con 0x7f00f8145880 2024-05-26T22:54:43.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:43.986+0000 7f00f7e00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f00f8160890 con 0x7f00f815fba0 2024-05-26T22:54:43.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:43.986+0000 7f00f7e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f00f80b1d10 con 0x7f00f8160020 2024-05-26T22:54:43.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:43.986+0000 7f00f6a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f00f8145880 0x7f00f815b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:43.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:43.986+0000 7f00f6a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f00f8145880 0x7f00f815b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44468/0 (socket says 172.21.15.131:44468) 2024-05-26T22:54:43.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:43.986+0000 7f00f6a00640 1 -- 172.21.15.131:0/1287450025 learned_addr learned my addr 172.21.15.131:0/1287450025 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:54:43.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:43.986+0000 7f00f6a00640 1 -- 172.21.15.131:0/1287450025 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f00f8160020 msgr2=0x7f00f80cdd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:43.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:43.986+0000 7f00f6a00640 1 --2- 172.21.15.131:0/1287450025 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f00f8160020 0x7f00f80cdd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:43.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:43.986+0000 7f00f6a00640 1 -- 172.21.15.131:0/1287450025 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f00f815fba0 msgr2=0x7f00f80cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:43.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:43.986+0000 7f00f6a00640 1 --2- 172.21.15.131:0/1287450025 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f00f815fba0 0x7f00f80cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:43.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:43.986+0000 7f00f6a00640 1 -- 172.21.15.131:0/1287450025 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00f80ca360 con 0x7f00f8145880 2024-05-26T22:54:43.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:43.987+0000 7f00f6a00640 1 --2- 172.21.15.131:0/1287450025 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f00f8145880 0x7f00f815b850 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f00e802a220 tx=0x7f00e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:43.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:43.987+0000 7f00ef400640 1 -- 172.21.15.131:0/1287450025 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00e8041070 con 0x7f00f8145880 2024-05-26T22:54:43.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:43.987+0000 7f00ef400640 1 -- 172.21.15.131:0/1287450025 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00e803c890 con 0x7f00f8145880 2024-05-26T22:54:43.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:43.987+0000 7f00ef400640 1 -- 172.21.15.131:0/1287450025 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00e8037040 con 0x7f00f8145880 2024-05-26T22:54:43.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:43.987+0000 7f00f7e00640 1 -- 172.21.15.131:0/1287450025 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00f80ca640 con 0x7f00f8145880 2024-05-26T22:54:43.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:43.987+0000 7f00f7e00640 1 -- 172.21.15.131:0/1287450025 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00f80cac00 con 0x7f00f8145880 2024-05-26T22:54:43.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:43.988+0000 7f00f7e00640 1 -- 172.21.15.131:0/1287450025 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00f80694e0 con 0x7f00f8145880 2024-05-26T22:54:43.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:43.989+0000 7f00ef400640 1 -- 172.21.15.131:0/1287450025 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f00e8044070 con 0x7f00f8145880 2024-05-26T22:54:43.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:43.990+0000 7f00ef400640 1 --2- 172.21.15.131:0/1287450025 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f00d406be30 0x7f00d406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:43.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:43.991+0000 7f00ef400640 1 -- 172.21.15.131:0/1287450025 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5996+0+0 (secure 0 0 0) 0x7f00e80abc80 con 0x7f00f8145880 2024-05-26T22:54:43.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:43.991+0000 7f00ee000640 1 --2- 172.21.15.131:0/1287450025 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f00d406be30 0x7f00d406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:43.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:43.991+0000 7f00ee000640 1 --2- 172.21.15.131:0/1287450025 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f00d406be30 0x7f00d406e2f0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f00f80b19c0 tx=0x7f00e0006bf0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:43.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:43.995+0000 7f00ef400640 1 -- 172.21.15.131:0/1287450025 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f00e807a470 con 0x7f00f8145880 2024-05-26T22:54:44.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:44.178+0000 7f00f7e00640 1 -- 172.21.15.131:0/1287450025 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f00f80ccfc0 con 0x7f00f8145880 2024-05-26T22:54:44.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:44.756+0000 7f00ef400640 1 -- 172.21.15.131:0/1287450025 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v86) ==== 181+0+0 (secure 0 0 0) 0x7f00e8079d70 con 0x7f00f8145880 2024-05-26T22:54:44.756 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_metadata' removed 2024-05-26T22:54:44.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:44.760+0000 7f00f7e00640 1 -- 172.21.15.131:0/1287450025 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f00d406be30 msgr2=0x7f00d406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:44.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:44.760+0000 7f00f7e00640 1 --2- 172.21.15.131:0/1287450025 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f00d406be30 0x7f00d406e2f0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f00f80b19c0 tx=0x7f00e0006bf0 comp rx=0 tx=0).stop 2024-05-26T22:54:44.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:44.760+0000 7f00f7e00640 1 -- 172.21.15.131:0/1287450025 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f00f8145880 msgr2=0x7f00f815b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:44.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:44.760+0000 7f00f7e00640 1 --2- 172.21.15.131:0/1287450025 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f00f8145880 0x7f00f815b850 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f00e802a220 tx=0x7f00e80046c0 comp rx=0 tx=0).stop 2024-05-26T22:54:44.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:44.760+0000 7f00f7e00640 1 -- 172.21.15.131:0/1287450025 shutdown_connections 2024-05-26T22:54:44.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:44.760+0000 7f00f7e00640 1 --2- 172.21.15.131:0/1287450025 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f00f8160020 0x7f00f80cdd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:44.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:44.760+0000 7f00f7e00640 1 --2- 172.21.15.131:0/1287450025 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f00d406be30 0x7f00d406e2f0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:44.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:44.760+0000 7f00f7e00640 1 --2- 172.21.15.131:0/1287450025 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f00f815fba0 0x7f00f80cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:44.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:44.760+0000 7f00f7e00640 1 --2- 172.21.15.131:0/1287450025 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f00f8145880 0x7f00f815b850 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:44.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:44.760+0000 7f00f7e00640 1 -- 172.21.15.131:0/1287450025 >> 172.21.15.131:0/1287450025 conn(0x7f00f80b5410 msgr2=0x7f00f815eb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:44.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:44.760+0000 7f00f7e00640 1 -- 172.21.15.131:0/1287450025 shutdown_connections 2024-05-26T22:54:44.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:44.760+0000 7f00f7e00640 1 -- 172.21.15.131:0/1287450025 wait complete. 2024-05-26T22:54:44.785 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-26T22:54:44.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:44.971+0000 7f228fe00640 1 Processor -- start 2024-05-26T22:54:44.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:44.971+0000 7f228fe00640 1 -- start start 2024-05-26T22:54:44.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:44.973+0000 7f228fe00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f229015fba0 0x7f229006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:44.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:44.973+0000 7f228fe00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f2290160000 0x7f229006da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:44.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:44.973+0000 7f228fe00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2290145c60 con 0x7f2290145880 2024-05-26T22:54:44.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:44.973+0000 7f228fe00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f229015c560 con 0x7f2290160000 2024-05-26T22:54:44.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:44.973+0000 7f228fe00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f229015cb10 con 0x7f229015fba0 2024-05-26T22:54:44.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:44.973+0000 7f228ea00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f2290145880 0x7f229015cfa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:35246/0 (socket says 172.21.15.131:35246) 2024-05-26T22:54:44.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:44.973+0000 7f228f400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f2290160000 0x7f229006da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:44.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:44.973+0000 7f228ea00640 1 -- 172.21.15.131:0/2755715763 learned_addr learned my addr 172.21.15.131:0/2755715763 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:54:44.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:44.973+0000 7f228e000640 1 --2- 172.21.15.131:0/2755715763 >> v2:172.21.15.187:3300/0 conn(0x7f229015fba0 0x7f229006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:44.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:44.974+0000 7f228f400640 1 -- 172.21.15.131:0/2755715763 >> v2:172.21.15.187:3300/0 conn(0x7f229015fba0 msgr2=0x7f229006d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:44.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:44.974+0000 7f228f400640 1 --2- 172.21.15.131:0/2755715763 >> v2:172.21.15.187:3300/0 conn(0x7f229015fba0 0x7f229006d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:44.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:44.973+0000 7f228d600640 1 -- 172.21.15.131:0/2755715763 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2421571709 0 0) 0x7f2290145c60 con 0x7f2290145880 2024-05-26T22:54:44.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:44.974+0000 7f228f400640 1 -- 172.21.15.131:0/2755715763 >> v1:172.21.15.131:6789/0 conn(0x7f2290145880 legacy=0x7f229015cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:44.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:44.974+0000 7f228f400640 1 -- 172.21.15.131:0/2755715763 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22901605b0 con 0x7f2290160000 2024-05-26T22:54:44.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:44.974+0000 7f228f400640 1 --2- 172.21.15.131:0/2755715763 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2290160000 0x7f229006da20 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f22900ce7d0 tx=0x7f228802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2487e13f978b43a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:44.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:44.975+0000 7f228d600640 1 -- 172.21.15.131:0/2755715763 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2288007740 con 0x7f2290160000 2024-05-26T22:54:44.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:44.975+0000 7f228d600640 1 -- 172.21.15.131:0/2755715763 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22880078e0 con 0x7f2290160000 2024-05-26T22:54:44.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:44.975+0000 7f228d600640 1 -- 172.21.15.131:0/2755715763 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2288007cb0 con 0x7f2290160000 2024-05-26T22:54:44.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:44.975+0000 7f228fe00640 1 -- 172.21.15.131:0/2755715763 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2290160000 msgr2=0x7f229006da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:44.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:44.975+0000 7f228fe00640 1 --2- 172.21.15.131:0/2755715763 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2290160000 0x7f229006da20 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f22900ce7d0 tx=0x7f228802a680 comp rx=0 tx=0).stop 2024-05-26T22:54:44.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:44.975+0000 7f228fe00640 1 -- 172.21.15.131:0/2755715763 shutdown_connections 2024-05-26T22:54:44.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:44.975+0000 7f228fe00640 1 --2- 172.21.15.131:0/2755715763 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2290160000 0x7f229006da20 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:44.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:44.975+0000 7f228fe00640 1 --2- 172.21.15.131:0/2755715763 >> v2:172.21.15.187:3300/0 conn(0x7f229015fba0 0x7f229006d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:44.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:44.975+0000 7f228fe00640 1 -- 172.21.15.131:0/2755715763 >> 172.21.15.131:0/2755715763 conn(0x7f22900b5410 msgr2=0x7f22900b57f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:44.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:44.976+0000 7f228fe00640 1 -- 172.21.15.131:0/2755715763 shutdown_connections 2024-05-26T22:54:44.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:44.976+0000 7f228fe00640 1 -- 172.21.15.131:0/2755715763 wait complete. 2024-05-26T22:54:44.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:44.977+0000 7f228fe00640 1 Processor -- start 2024-05-26T22:54:44.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:44.977+0000 7f228fe00640 1 -- start start 2024-05-26T22:54:44.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:44.978+0000 7f228fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2290145880 0x7f2290144380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:44.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:44.978+0000 7f228fe00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f229015fba0 0x7f22900b0340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:44.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:44.978+0000 7f228fe00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2290160000 0x7f22901f38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:44.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:44.978+0000 7f228fe00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f22900b1d10 con 0x7f2290145880 2024-05-26T22:54:44.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:44.978+0000 7f228fe00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f229015c7a0 con 0x7f229015fba0 2024-05-26T22:54:44.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:44.978+0000 7f228fe00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f22900b1e90 con 0x7f2290160000 2024-05-26T22:54:44.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:44.979+0000 7f228ea00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2290145880 0x7f2290144380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:44.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:44.979+0000 7f228ea00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2290145880 0x7f2290144380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59184/0 (socket says 172.21.15.131:59184) 2024-05-26T22:54:44.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:44.979+0000 7f228ea00640 1 -- 172.21.15.131:0/1386129811 learned_addr learned my addr 172.21.15.131:0/1386129811 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:54:44.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:44.979+0000 7f228e000640 1 --2- 172.21.15.131:0/1386129811 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f229015fba0 0x7f22900b0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:44.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:44.979+0000 7f228f400640 1 --2- 172.21.15.131:0/1386129811 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2290160000 0x7f22901f38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:44.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:44.979+0000 7f228ea00640 1 -- 172.21.15.131:0/1386129811 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2290160000 msgr2=0x7f22901f38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:44.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:44.979+0000 7f228ea00640 1 --2- 172.21.15.131:0/1386129811 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2290160000 0x7f22901f38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:44.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:44.979+0000 7f228ea00640 1 -- 172.21.15.131:0/1386129811 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f229015fba0 msgr2=0x7f22900b0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:44.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:44.979+0000 7f228ea00640 1 --2- 172.21.15.131:0/1386129811 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f229015fba0 0x7f22900b0340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:44.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:44.979+0000 7f228ea00640 1 -- 172.21.15.131:0/1386129811 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22901f4410 con 0x7f2290145880 2024-05-26T22:54:44.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:44.980+0000 7f228ea00640 1 --2- 172.21.15.131:0/1386129811 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2290145880 0x7f2290144380 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f228000c430 tx=0x7f228000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:44.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:44.980+0000 7f2286a00640 1 -- 172.21.15.131:0/1386129811 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22800074e0 con 0x7f2290145880 2024-05-26T22:54:44.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:44.980+0000 7f2286a00640 1 -- 172.21.15.131:0/1386129811 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22800047e0 con 0x7f2290145880 2024-05-26T22:54:44.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:44.980+0000 7f2286a00640 1 -- 172.21.15.131:0/1386129811 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2280004bd0 con 0x7f2290145880 2024-05-26T22:54:44.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:44.980+0000 7f228fe00640 1 -- 172.21.15.131:0/1386129811 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22901f46f0 con 0x7f2290145880 2024-05-26T22:54:44.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:44.980+0000 7f228fe00640 1 -- 172.21.15.131:0/1386129811 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2290143e60 con 0x7f2290145880 2024-05-26T22:54:44.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:44.981+0000 7f228fe00640 1 -- 172.21.15.131:0/1386129811 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22900694e0 con 0x7f2290145880 2024-05-26T22:54:44.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:44.982+0000 7f2286a00640 1 -- 172.21.15.131:0/1386129811 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2280011070 con 0x7f2290145880 2024-05-26T22:54:44.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:44.983+0000 7f2286a00640 1 --2- 172.21.15.131:0/1386129811 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f227006be30 0x7f227006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:44.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:44.983+0000 7f2286a00640 1 -- 172.21.15.131:0/1386129811 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5558+0+0 (secure 0 0 0) 0x7f228008aa50 con 0x7f2290145880 2024-05-26T22:54:44.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:44.983+0000 7f228e000640 1 --2- 172.21.15.131:0/1386129811 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f227006be30 0x7f227006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:44.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:44.984+0000 7f228e000640 1 --2- 172.21.15.131:0/1386129811 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f227006be30 0x7f227006e2f0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f227c006fd0 tx=0x7f227c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:44.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:44.987+0000 7f2286a00640 1 -- 172.21.15.131:0/1386129811 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f2280059480 con 0x7f2290145880 2024-05-26T22:54:45.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:45.187+0000 7f228fe00640 1 -- 172.21.15.131:0/1386129811 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f22900b3a30 con 0x7f2290145880 2024-05-26T22:54:45.536 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-26T22:54:45.544 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-26T22:54:45.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:45.759+0000 7f2286a00640 1 -- 172.21.15.131:0/1386129811 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v87) ==== 169+0+0 (secure 0 0 0) 0x7f2280058d80 con 0x7f2290145880 2024-05-26T22:54:45.760 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_data' removed 2024-05-26T22:54:45.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:45.769+0000 7f228fe00640 1 -- 172.21.15.131:0/1386129811 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f227006be30 msgr2=0x7f227006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:45.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:45.769+0000 7f228fe00640 1 --2- 172.21.15.131:0/1386129811 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f227006be30 0x7f227006e2f0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f227c006fd0 tx=0x7f227c008040 comp rx=0 tx=0).stop 2024-05-26T22:54:45.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:45.769+0000 7f228fe00640 1 -- 172.21.15.131:0/1386129811 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2290145880 msgr2=0x7f2290144380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:45.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:45.769+0000 7f228fe00640 1 --2- 172.21.15.131:0/1386129811 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2290145880 0x7f2290144380 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f228000c430 tx=0x7f228000c8f0 comp rx=0 tx=0).stop 2024-05-26T22:54:45.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:45.769+0000 7f228fe00640 1 -- 172.21.15.131:0/1386129811 shutdown_connections 2024-05-26T22:54:45.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:45.769+0000 7f228fe00640 1 --2- 172.21.15.131:0/1386129811 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2290160000 0x7f22901f38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:45.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:45.769+0000 7f228fe00640 1 --2- 172.21.15.131:0/1386129811 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f227006be30 0x7f227006e2f0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:45.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:45.769+0000 7f228fe00640 1 --2- 172.21.15.131:0/1386129811 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f229015fba0 0x7f22900b0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:45.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:45.769+0000 7f228fe00640 1 --2- 172.21.15.131:0/1386129811 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2290145880 0x7f2290144380 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:45.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:45.769+0000 7f228fe00640 1 -- 172.21.15.131:0/1386129811 >> 172.21.15.131:0/1386129811 conn(0x7f22900b5410 msgr2=0x7f22900b2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:45.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:45.770+0000 7f228fe00640 1 -- 172.21.15.131:0/1386129811 shutdown_connections 2024-05-26T22:54:45.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:45.770+0000 7f228fe00640 1 -- 172.21.15.131:0/1386129811 wait complete. 2024-05-26T22:54:45.796 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_export_ephemeral_random_max 2024-05-26T22:54:45.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:45.985+0000 7f475ae00640 1 Processor -- start 2024-05-26T22:54:45.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:45.985+0000 7f475ae00640 1 -- start start 2024-05-26T22:54:45.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:45.987+0000 7f475ae00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f475406f180 0x7f4754072e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:45.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:45.987+0000 7f475ae00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f47540d0560 0x7f4754161ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:45.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:45.987+0000 7f475ae00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47540af8e0 con 0x7f4754072540 2024-05-26T22:54:45.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:45.987+0000 7f475ae00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f47540cfb30 con 0x7f47540d0560 2024-05-26T22:54:45.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:45.987+0000 7f475ae00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f47540cf200 con 0x7f475406f180 2024-05-26T22:54:45.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:45.987+0000 7f4758e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f47540d0560 0x7f4754161ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:45.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:45.987+0000 7f4753400640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f4754072540 0x7f47540cfe50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:35250/0 (socket says 172.21.15.131:35250) 2024-05-26T22:54:45.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:45.987+0000 7f4753400640 1 -- 172.21.15.131:0/2099425501 learned_addr learned my addr 172.21.15.131:0/2099425501 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:54:45.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:45.987+0000 7f4753e00640 1 --2- 172.21.15.131:0/2099425501 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f475406f180 0x7f4754072e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:45.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:45.987+0000 7f4758e00640 1 -- 172.21.15.131:0/2099425501 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f475406f180 msgr2=0x7f4754072e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:45.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:45.988+0000 7f4758e00640 1 --2- 172.21.15.131:0/2099425501 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f475406f180 0x7f4754072e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:45.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:45.988+0000 7f4758e00640 1 -- 172.21.15.131:0/2099425501 >> v1:172.21.15.131:6789/0 conn(0x7f4754072540 legacy=0x7f47540cfe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:45.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:45.988+0000 7f4758e00640 1 -- 172.21.15.131:0/2099425501 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47540cf4e0 con 0x7f47540d0560 2024-05-26T22:54:45.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:45.988+0000 7f4752a00640 1 -- 172.21.15.131:0/2099425501 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2830080227 0 0) 0x7f47540af8e0 con 0x7f4754072540 2024-05-26T22:54:45.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:45.989+0000 7f4758e00640 1 --2- 172.21.15.131:0/2099425501 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47540d0560 0x7f4754161ff0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f474c009d90 tx=0x7f474c02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b34a6da5e9a26915 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:45.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:45.989+0000 7f4752a00640 1 -- 172.21.15.131:0/2099425501 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f474c007740 con 0x7f47540d0560 2024-05-26T22:54:45.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:45.989+0000 7f4752a00640 1 -- 172.21.15.131:0/2099425501 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f474c0078e0 con 0x7f47540d0560 2024-05-26T22:54:45.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:45.989+0000 7f4752a00640 1 -- 172.21.15.131:0/2099425501 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f474c007cb0 con 0x7f47540d0560 2024-05-26T22:54:45.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:45.989+0000 7f475ae00640 1 -- 172.21.15.131:0/2099425501 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47540d0560 msgr2=0x7f4754161ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:45.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:45.989+0000 7f475ae00640 1 --2- 172.21.15.131:0/2099425501 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47540d0560 0x7f4754161ff0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f474c009d90 tx=0x7f474c02a660 comp rx=0 tx=0).stop 2024-05-26T22:54:45.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:45.990+0000 7f475ae00640 1 -- 172.21.15.131:0/2099425501 shutdown_connections 2024-05-26T22:54:45.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:45.990+0000 7f475ae00640 1 --2- 172.21.15.131:0/2099425501 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47540d0560 0x7f4754161ff0 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:45.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:45.990+0000 7f475ae00640 1 --2- 172.21.15.131:0/2099425501 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f475406f180 0x7f4754072e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:45.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:45.990+0000 7f475ae00640 1 -- 172.21.15.131:0/2099425501 >> 172.21.15.131:0/2099425501 conn(0x7f475406bc90 msgr2=0x7f47540b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:45.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:45.990+0000 7f475ae00640 1 -- 172.21.15.131:0/2099425501 shutdown_connections 2024-05-26T22:54:45.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:45.990+0000 7f475ae00640 1 -- 172.21.15.131:0/2099425501 wait complete. 2024-05-26T22:54:45.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:45.991+0000 7f475ae00640 1 Processor -- start 2024-05-26T22:54:45.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:45.991+0000 7f475ae00640 1 -- start start 2024-05-26T22:54:45.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:45.992+0000 7f475ae00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f475406f180 0x7f47540ceb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:45.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:45.992+0000 7f475ae00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4754072540 0x7f475415bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:45.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:45.992+0000 7f475ae00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f47540d0560 0x7f47540adf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:45.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:45.992+0000 7f475ae00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f475406f6e0 con 0x7f4754072540 2024-05-26T22:54:45.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:45.992+0000 7f475ae00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f475406f560 con 0x7f475406f180 2024-05-26T22:54:45.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:45.992+0000 7f475ae00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f47540715a0 con 0x7f47540d0560 2024-05-26T22:54:45.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:45.993+0000 7f4753400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4754072540 0x7f475415bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:45.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:45.993+0000 7f4753400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4754072540 0x7f475415bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59200/0 (socket says 172.21.15.131:59200) 2024-05-26T22:54:45.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:45.993+0000 7f4753400640 1 -- 172.21.15.131:0/1661066189 learned_addr learned my addr 172.21.15.131:0/1661066189 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:54:45.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:45.993+0000 7f4753e00640 1 --2- 172.21.15.131:0/1661066189 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f475406f180 0x7f47540ceb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:45.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:45.993+0000 7f4758e00640 1 --2- 172.21.15.131:0/1661066189 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f47540d0560 0x7f47540adf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:45.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:45.993+0000 7f4753400640 1 -- 172.21.15.131:0/1661066189 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f47540d0560 msgr2=0x7f47540adf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:45.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:45.993+0000 7f4753400640 1 --2- 172.21.15.131:0/1661066189 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f47540d0560 0x7f47540adf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:45.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:45.993+0000 7f4753400640 1 -- 172.21.15.131:0/1661066189 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f475406f180 msgr2=0x7f47540ceb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:45.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:45.993+0000 7f4753400640 1 --2- 172.21.15.131:0/1661066189 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f475406f180 0x7f47540ceb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:45.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:45.993+0000 7f4753400640 1 -- 172.21.15.131:0/1661066189 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47541f4710 con 0x7f4754072540 2024-05-26T22:54:45.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:45.993+0000 7f4753e00640 1 --2- 172.21.15.131:0/1661066189 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f475406f180 0x7f47540ceb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:54:45.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:45.993+0000 7f4758e00640 1 --2- 172.21.15.131:0/1661066189 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f47540d0560 0x7f47540adf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:54:45.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:45.993+0000 7f4753400640 1 --2- 172.21.15.131:0/1661066189 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4754072540 0x7f475415bc20 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f473c00c4a0 tx=0x7f473c00c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:45.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:45.994+0000 7f474be00640 1 -- 172.21.15.131:0/1661066189 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f473c017070 con 0x7f4754072540 2024-05-26T22:54:45.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:45.994+0000 7f474be00640 1 -- 172.21.15.131:0/1661066189 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f473c005ce0 con 0x7f4754072540 2024-05-26T22:54:45.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:45.994+0000 7f475ae00640 1 -- 172.21.15.131:0/1661066189 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47541f49f0 con 0x7f4754072540 2024-05-26T22:54:45.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:45.994+0000 7f474be00640 1 -- 172.21.15.131:0/1661066189 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f473c0072b0 con 0x7f4754072540 2024-05-26T22:54:45.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:45.994+0000 7f475ae00640 1 -- 172.21.15.131:0/1661066189 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47541f4fb0 con 0x7f4754072540 2024-05-26T22:54:45.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:45.995+0000 7f475ae00640 1 -- 172.21.15.131:0/1661066189 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4754074e90 con 0x7f4754072540 2024-05-26T22:54:45.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:45.996+0000 7f474be00640 1 -- 172.21.15.131:0/1661066189 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f473c01a070 con 0x7f4754072540 2024-05-26T22:54:46.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:45.997+0000 7f474be00640 1 --2- 172.21.15.131:0/1661066189 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f473006be30 0x7f473006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:46.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:45.997+0000 7f474be00640 1 -- 172.21.15.131:0/1661066189 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5115+0+0 (secure 0 0 0) 0x7f473c08a080 con 0x7f4754072540 2024-05-26T22:54:46.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:45.997+0000 7f4753e00640 1 --2- 172.21.15.131:0/1661066189 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f473006be30 0x7f473006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:46.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:45.998+0000 7f4753e00640 1 --2- 172.21.15.131:0/1661066189 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f473006be30 0x7f473006e2f0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f4744006fd0 tx=0x7f4744008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:46.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.001+0000 7f474be00640 1 -- 172.21.15.131:0/1661066189 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f473c058be0 con 0x7f4754072540 2024-05-26T22:54:46.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.199+0000 7f475ae00640 1 -- 172.21.15.131:0/1661066189 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_export_ephemeral_random_max"} v 0) -- 0x7f475415d360 con 0x7f4754072540 2024-05-26T22:54:46.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.205+0000 7f474be00640 1 -- 172.21.15.131:0/1661066189 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_export_ephemeral_random_max"}]=0 v47) ==== 114+0+0 (secure 0 0 0) 0x7f473c0584e0 con 0x7f4754072540 2024-05-26T22:54:46.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.208+0000 7f475ae00640 1 -- 172.21.15.131:0/1661066189 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f473006be30 msgr2=0x7f473006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:46.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.208+0000 7f475ae00640 1 --2- 172.21.15.131:0/1661066189 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f473006be30 0x7f473006e2f0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f4744006fd0 tx=0x7f4744008040 comp rx=0 tx=0).stop 2024-05-26T22:54:46.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.208+0000 7f475ae00640 1 -- 172.21.15.131:0/1661066189 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4754072540 msgr2=0x7f475415bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:46.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.208+0000 7f475ae00640 1 --2- 172.21.15.131:0/1661066189 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4754072540 0x7f475415bc20 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f473c00c4a0 tx=0x7f473c00c960 comp rx=0 tx=0).stop 2024-05-26T22:54:46.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.208+0000 7f475ae00640 1 -- 172.21.15.131:0/1661066189 shutdown_connections 2024-05-26T22:54:46.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.208+0000 7f475ae00640 1 --2- 172.21.15.131:0/1661066189 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f47540d0560 0x7f47540adf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:46.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.208+0000 7f475ae00640 1 --2- 172.21.15.131:0/1661066189 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f473006be30 0x7f473006e2f0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:46.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.208+0000 7f475ae00640 1 --2- 172.21.15.131:0/1661066189 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4754072540 0x7f475415bc20 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:46.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.208+0000 7f475ae00640 1 --2- 172.21.15.131:0/1661066189 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f475406f180 0x7f47540ceb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:46.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.208+0000 7f475ae00640 1 -- 172.21.15.131:0/1661066189 >> 172.21.15.131:0/1661066189 conn(0x7f475406bc90 msgr2=0x7f475415fbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:46.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.208+0000 7f475ae00640 1 -- 172.21.15.131:0/1661066189 shutdown_connections 2024-05-26T22:54:46.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.208+0000 7f475ae00640 1 -- 172.21.15.131:0/1661066189 wait complete. 2024-05-26T22:54:46.228 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_export_ephemeral_random 2024-05-26T22:54:46.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.429+0000 7f804ae00640 1 Processor -- start 2024-05-26T22:54:46.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.430+0000 7f804ae00640 1 -- start start 2024-05-26T22:54:46.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.431+0000 7f804ae00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f80440cee50 0x7f80440cf230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:46.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.431+0000 7f804ae00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f80440b5a90 0x7f80440b5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:46.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.431+0000 7f804ae00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80440cf770 con 0x7f8044072350 2024-05-26T22:54:46.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.431+0000 7f804ae00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f80440b4220 con 0x7f80440cee50 2024-05-26T22:54:46.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.431+0000 7f804ae00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f80440b5060 con 0x7f80440b5a90 2024-05-26T22:54:46.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.431+0000 7f8043e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f80440cee50 0x7f80440cf230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:46.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.431+0000 7f8048e00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f8044072350 0x7f80440727a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:35262/0 (socket says 172.21.15.131:35262) 2024-05-26T22:54:46.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.431+0000 7f8048e00640 1 -- 172.21.15.131:0/2717108675 learned_addr learned my addr 172.21.15.131:0/2717108675 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:54:46.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.432+0000 7f8043400640 1 --2- 172.21.15.131:0/2717108675 >> v2:172.21.15.187:3300/0 conn(0x7f80440b5a90 0x7f80440b5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:46.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.432+0000 7f8043e00640 1 -- 172.21.15.131:0/2717108675 >> v2:172.21.15.187:3300/0 conn(0x7f80440b5a90 msgr2=0x7f80440b5ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:46.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.432+0000 7f8042a00640 1 -- 172.21.15.131:0/2717108675 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1881721917 0 0) 0x7f80440cf770 con 0x7f8044072350 2024-05-26T22:54:46.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.432+0000 7f8043e00640 1 --2- 172.21.15.131:0/2717108675 >> v2:172.21.15.187:3300/0 conn(0x7f80440b5a90 0x7f80440b5ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:46.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.432+0000 7f8043e00640 1 -- 172.21.15.131:0/2717108675 >> v1:172.21.15.131:6789/0 conn(0x7f8044072350 legacy=0x7f80440727a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:46.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.432+0000 7f8043e00640 1 -- 172.21.15.131:0/2717108675 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80440b4450 con 0x7f80440cee50 2024-05-26T22:54:46.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.432+0000 7f8043400640 1 --2- 172.21.15.131:0/2717108675 >> v2:172.21.15.187:3300/0 conn(0x7f80440b5a90 0x7f80440b5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:54:46.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.433+0000 7f8043e00640 1 --2- 172.21.15.131:0/2717108675 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f80440cee50 0x7f80440cf230 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f804406def0 tx=0x7f803402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a44eacda41343abe server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:46.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.433+0000 7f8042a00640 1 -- 172.21.15.131:0/2717108675 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8034007740 con 0x7f80440cee50 2024-05-26T22:54:46.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.433+0000 7f8042a00640 1 -- 172.21.15.131:0/2717108675 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80340078e0 con 0x7f80440cee50 2024-05-26T22:54:46.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.433+0000 7f8042a00640 1 -- 172.21.15.131:0/2717108675 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8034007cb0 con 0x7f80440cee50 2024-05-26T22:54:46.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.434+0000 7f804ae00640 1 -- 172.21.15.131:0/2717108675 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f80440cee50 msgr2=0x7f80440cf230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:46.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.434+0000 7f804ae00640 1 --2- 172.21.15.131:0/2717108675 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f80440cee50 0x7f80440cf230 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f804406def0 tx=0x7f803402a680 comp rx=0 tx=0).stop 2024-05-26T22:54:46.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.434+0000 7f804ae00640 1 -- 172.21.15.131:0/2717108675 shutdown_connections 2024-05-26T22:54:46.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.434+0000 7f804ae00640 1 --2- 172.21.15.131:0/2717108675 >> v2:172.21.15.187:3300/0 conn(0x7f80440b5a90 0x7f80440b5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:46.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.434+0000 7f804ae00640 1 --2- 172.21.15.131:0/2717108675 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f80440cee50 0x7f80440cf230 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:46.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.434+0000 7f804ae00640 1 -- 172.21.15.131:0/2717108675 >> 172.21.15.131:0/2717108675 conn(0x7f80440b2ce0 msgr2=0x7f80440b30f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:46.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.434+0000 7f804ae00640 1 -- 172.21.15.131:0/2717108675 shutdown_connections 2024-05-26T22:54:46.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.434+0000 7f804ae00640 1 -- 172.21.15.131:0/2717108675 wait complete. 2024-05-26T22:54:46.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.436+0000 7f804ae00640 1 Processor -- start 2024-05-26T22:54:46.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.436+0000 7f804ae00640 1 -- start start 2024-05-26T22:54:46.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.437+0000 7f804ae00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8044072350 0x7f804414d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:46.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.437+0000 7f804ae00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f80440b5a90 0x7f80441595b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:46.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.437+0000 7f804ae00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f80440cee50 0x7f80440cb0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:46.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.437+0000 7f804ae00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f80440b4a10 con 0x7f80440cee50 2024-05-26T22:54:46.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.437+0000 7f804ae00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8044070720 con 0x7f8044072350 2024-05-26T22:54:46.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.437+0000 7f804ae00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f80440b4cf0 con 0x7f80440b5a90 2024-05-26T22:54:46.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.437+0000 7f8048e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f80440cee50 0x7f80440cb0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:46.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.437+0000 7f8048e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f80440cee50 0x7f80440cb0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59220/0 (socket says 172.21.15.131:59220) 2024-05-26T22:54:46.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.437+0000 7f8048e00640 1 -- 172.21.15.131:0/3318890576 learned_addr learned my addr 172.21.15.131:0/3318890576 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:54:46.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.438+0000 7f8043e00640 1 --2- 172.21.15.131:0/3318890576 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8044072350 0x7f804414d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:46.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.438+0000 7f8043400640 1 --2- 172.21.15.131:0/3318890576 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f80440b5a90 0x7f80441595b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:46.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.438+0000 7f8048e00640 1 -- 172.21.15.131:0/3318890576 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f80440b5a90 msgr2=0x7f80441595b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:46.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.438+0000 7f8048e00640 1 --2- 172.21.15.131:0/3318890576 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f80440b5a90 0x7f80441595b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:46.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.438+0000 7f8048e00640 1 -- 172.21.15.131:0/3318890576 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8044072350 msgr2=0x7f804414d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:46.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.438+0000 7f8048e00640 1 --2- 172.21.15.131:0/3318890576 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8044072350 0x7f804414d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:46.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.438+0000 7f8048e00640 1 -- 172.21.15.131:0/3318890576 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8044159e90 con 0x7f80440cee50 2024-05-26T22:54:46.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.438+0000 7f8043e00640 1 --2- 172.21.15.131:0/3318890576 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8044072350 0x7f804414d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:54:46.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.438+0000 7f8048e00640 1 --2- 172.21.15.131:0/3318890576 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f80440cee50 0x7f80440cb0c0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f803c00c450 tx=0x7f803c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:46.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.438+0000 7f803be00640 1 -- 172.21.15.131:0/3318890576 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f803c018070 con 0x7f80440cee50 2024-05-26T22:54:46.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.438+0000 7f803be00640 1 -- 172.21.15.131:0/3318890576 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f803c007500 con 0x7f80440cee50 2024-05-26T22:54:46.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.438+0000 7f803be00640 1 -- 172.21.15.131:0/3318890576 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f803c004a20 con 0x7f80440cee50 2024-05-26T22:54:46.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.439+0000 7f804ae00640 1 -- 172.21.15.131:0/3318890576 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f804415a170 con 0x7f80440cee50 2024-05-26T22:54:46.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.439+0000 7f804ae00640 1 -- 172.21.15.131:0/3318890576 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80441f4660 con 0x7f80440cee50 2024-05-26T22:54:46.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.440+0000 7f804ae00640 1 -- 172.21.15.131:0/3318890576 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80440695c0 con 0x7f80440cee50 2024-05-26T22:54:46.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.441+0000 7f803be00640 1 -- 172.21.15.131:0/3318890576 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f803c01b360 con 0x7f80440cee50 2024-05-26T22:54:46.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.443+0000 7f803be00640 1 --2- 172.21.15.131:0/3318890576 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f802006be30 0x7f802006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:46.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.443+0000 7f803be00640 1 -- 172.21.15.131:0/3318890576 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5115+0+0 (secure 0 0 0) 0x7f803c08a1c0 con 0x7f80440cee50 2024-05-26T22:54:46.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.443+0000 7f8043e00640 1 --2- 172.21.15.131:0/3318890576 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f802006be30 0x7f802006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:46.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.444+0000 7f8043e00640 1 --2- 172.21.15.131:0/3318890576 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f802006be30 0x7f802006e2f0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f8034034000 tx=0x7f8034033040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:46.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.446+0000 7f803be00640 1 -- 172.21.15.131:0/3318890576 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f803c058da0 con 0x7f80440cee50 2024-05-26T22:54:46.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.645+0000 7f804ae00640 1 -- 172.21.15.131:0/3318890576 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_export_ephemeral_random"} v 0) -- 0x7f804415e7f0 con 0x7f80440cee50 2024-05-26T22:54:46.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.653+0000 7f803be00640 1 -- 172.21.15.131:0/3318890576 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_export_ephemeral_random"}]=0 v48) ==== 110+0+0 (secure 0 0 0) 0x7f803c0586a0 con 0x7f80440cee50 2024-05-26T22:54:46.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.655+0000 7f804ae00640 1 -- 172.21.15.131:0/3318890576 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f802006be30 msgr2=0x7f802006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:46.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.655+0000 7f804ae00640 1 --2- 172.21.15.131:0/3318890576 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f802006be30 0x7f802006e2f0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f8034034000 tx=0x7f8034033040 comp rx=0 tx=0).stop 2024-05-26T22:54:46.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.655+0000 7f804ae00640 1 -- 172.21.15.131:0/3318890576 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f80440cee50 msgr2=0x7f80440cb0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:46.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.655+0000 7f804ae00640 1 --2- 172.21.15.131:0/3318890576 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f80440cee50 0x7f80440cb0c0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f803c00c450 tx=0x7f803c00c910 comp rx=0 tx=0).stop 2024-05-26T22:54:46.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.656+0000 7f804ae00640 1 -- 172.21.15.131:0/3318890576 shutdown_connections 2024-05-26T22:54:46.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.656+0000 7f804ae00640 1 --2- 172.21.15.131:0/3318890576 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f80440b5a90 0x7f80441595b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:46.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.656+0000 7f804ae00640 1 --2- 172.21.15.131:0/3318890576 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f802006be30 0x7f802006e2f0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:46.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.656+0000 7f804ae00640 1 --2- 172.21.15.131:0/3318890576 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f80440cee50 0x7f80440cb0c0 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:46.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.656+0000 7f804ae00640 1 --2- 172.21.15.131:0/3318890576 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8044072350 0x7f804414d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:46.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.656+0000 7f804ae00640 1 -- 172.21.15.131:0/3318890576 >> 172.21.15.131:0/3318890576 conn(0x7f80440b2ce0 msgr2=0x7f80440727a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:46.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.656+0000 7f804ae00640 1 -- 172.21.15.131:0/3318890576 shutdown_connections 2024-05-26T22:54:46.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.656+0000 7f804ae00640 1 -- 172.21.15.131:0/3318890576 wait complete. 2024-05-26T22:54:46.677 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_export_ephemeral_distributed 2024-05-26T22:54:46.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.869+0000 7f50d6a00640 1 Processor -- start 2024-05-26T22:54:46.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.870+0000 7f50d6a00640 1 -- start start 2024-05-26T22:54:46.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.871+0000 7f50d6a00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f50d00d0560 0x7f50d0072e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:46.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.871+0000 7f50d6a00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f50d0072540 0x7f50d0161ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:46.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.871+0000 7f50d6a00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50d00af8e0 con 0x7f50d006f180 2024-05-26T22:54:46.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.871+0000 7f50d6a00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f50d00cfb30 con 0x7f50d0072540 2024-05-26T22:54:46.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.871+0000 7f50d6a00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f50d00cf200 con 0x7f50d00d0560 2024-05-26T22:54:46.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.872+0000 7f50d4a00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f50d0072540 0x7f50d0161ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:46.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.872+0000 7f50d4a00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f50d0072540 0x7f50d0161ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59228/0 (socket says 172.21.15.131:59228) 2024-05-26T22:54:46.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.872+0000 7f50d4a00640 1 -- 172.21.15.131:0/506525143 learned_addr learned my addr 172.21.15.131:0/506525143 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:54:46.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.872+0000 7f50cf400640 1 --2- 172.21.15.131:0/506525143 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f50d00d0560 0x7f50d0072e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:46.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.872+0000 7f50d4a00640 1 -- 172.21.15.131:0/506525143 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f50d00d0560 msgr2=0x7f50d0072e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:46.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.872+0000 7f50d4a00640 1 --2- 172.21.15.131:0/506525143 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f50d00d0560 0x7f50d0072e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:46.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.872+0000 7f50d4a00640 1 -- 172.21.15.131:0/506525143 >> v1:172.21.15.187:6789/0 conn(0x7f50d006f180 legacy=0x7f50d00cfe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:46.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.872+0000 7f50d4a00640 1 -- 172.21.15.131:0/506525143 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50d00cf4e0 con 0x7f50d0072540 2024-05-26T22:54:46.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.872+0000 7f50cf400640 1 --2- 172.21.15.131:0/506525143 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f50d00d0560 0x7f50d0072e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:54:46.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.873+0000 7f50d4a00640 1 --2- 172.21.15.131:0/506525143 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50d0072540 0x7f50d0161ff0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f50c8009870 tx=0x7f50c802a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23229fe3dfd44999 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:46.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.873+0000 7f50cea00640 1 -- 172.21.15.131:0/506525143 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50c8037070 con 0x7f50d0072540 2024-05-26T22:54:46.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.874+0000 7f50cea00640 1 -- 172.21.15.131:0/506525143 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50c8032440 con 0x7f50d0072540 2024-05-26T22:54:46.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.874+0000 7f50cea00640 1 -- 172.21.15.131:0/506525143 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50c8032810 con 0x7f50d0072540 2024-05-26T22:54:46.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.874+0000 7f50d6a00640 1 -- 172.21.15.131:0/506525143 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50d0072540 msgr2=0x7f50d0161ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:46.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.874+0000 7f50d6a00640 1 --2- 172.21.15.131:0/506525143 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50d0072540 0x7f50d0161ff0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f50c8009870 tx=0x7f50c802a200 comp rx=0 tx=0).stop 2024-05-26T22:54:46.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.874+0000 7f50d6a00640 1 -- 172.21.15.131:0/506525143 shutdown_connections 2024-05-26T22:54:46.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.874+0000 7f50d6a00640 1 --2- 172.21.15.131:0/506525143 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50d0072540 0x7f50d0161ff0 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:46.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.874+0000 7f50d6a00640 1 --2- 172.21.15.131:0/506525143 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f50d00d0560 0x7f50d0072e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:46.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.874+0000 7f50d6a00640 1 -- 172.21.15.131:0/506525143 >> 172.21.15.131:0/506525143 conn(0x7f50d006bc90 msgr2=0x7f50d00b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:46.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.874+0000 7f50d6a00640 1 -- 172.21.15.131:0/506525143 shutdown_connections 2024-05-26T22:54:46.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.874+0000 7f50d6a00640 1 -- 172.21.15.131:0/506525143 wait complete. 2024-05-26T22:54:46.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.876+0000 7f50d6a00640 1 Processor -- start 2024-05-26T22:54:46.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.877+0000 7f50d6a00640 1 -- start start 2024-05-26T22:54:46.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.878+0000 7f50d6a00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f50d006f180 0x7f50d015ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:46.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.878+0000 7f50d6a00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f50d0072540 0x7f50d00adf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:46.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.878+0000 7f50d6a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50d00d0560 0x7f50d00ae480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:46.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.878+0000 7f50d6a00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f50d006f6e0 con 0x7f50d00d0560 2024-05-26T22:54:46.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.878+0000 7f50d6a00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f50d006f560 con 0x7f50d006f180 2024-05-26T22:54:46.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.878+0000 7f50d6a00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f50d00715a0 con 0x7f50d0072540 2024-05-26T22:54:46.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.878+0000 7f50d4a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50d00d0560 0x7f50d00ae480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:46.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.878+0000 7f50d4a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50d00d0560 0x7f50d00ae480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59234/0 (socket says 172.21.15.131:59234) 2024-05-26T22:54:46.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.878+0000 7f50d4a00640 1 -- 172.21.15.131:0/784238772 learned_addr learned my addr 172.21.15.131:0/784238772 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:54:46.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.879+0000 7f50cfe00640 1 --2- 172.21.15.131:0/784238772 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f50d006f180 0x7f50d015ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:46.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.879+0000 7f50cf400640 1 --2- 172.21.15.131:0/784238772 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f50d0072540 0x7f50d00adf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:46.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.879+0000 7f50d4a00640 1 -- 172.21.15.131:0/784238772 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f50d0072540 msgr2=0x7f50d00adf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:46.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.879+0000 7f50d4a00640 1 --2- 172.21.15.131:0/784238772 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f50d0072540 0x7f50d00adf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:46.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.879+0000 7f50d4a00640 1 -- 172.21.15.131:0/784238772 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f50d006f180 msgr2=0x7f50d015ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:46.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.879+0000 7f50d4a00640 1 --2- 172.21.15.131:0/784238772 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f50d006f180 0x7f50d015ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:46.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.879+0000 7f50d4a00640 1 -- 172.21.15.131:0/784238772 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50d00c9330 con 0x7f50d00d0560 2024-05-26T22:54:46.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.879+0000 7f50cfe00640 1 --2- 172.21.15.131:0/784238772 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f50d006f180 0x7f50d015ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:54:46.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.879+0000 7f50d4a00640 1 --2- 172.21.15.131:0/784238772 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50d00d0560 0x7f50d00ae480 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f50c8009870 tx=0x7f50c80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:46.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.879+0000 7f50c7e00640 1 -- 172.21.15.131:0/784238772 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50c8037040 con 0x7f50d00d0560 2024-05-26T22:54:46.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.879+0000 7f50c7e00640 1 -- 172.21.15.131:0/784238772 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50c8042070 con 0x7f50d00d0560 2024-05-26T22:54:46.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.879+0000 7f50d6a00640 1 -- 172.21.15.131:0/784238772 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50d00c95a0 con 0x7f50d00d0560 2024-05-26T22:54:46.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.879+0000 7f50c7e00640 1 -- 172.21.15.131:0/784238772 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50c8030c90 con 0x7f50d00d0560 2024-05-26T22:54:46.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.880+0000 7f50d6a00640 1 -- 172.21.15.131:0/784238772 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50d01f4f70 con 0x7f50d00d0560 2024-05-26T22:54:46.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.881+0000 7f50d6a00640 1 -- 172.21.15.131:0/784238772 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50d0074e90 con 0x7f50d00d0560 2024-05-26T22:54:46.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.882+0000 7f50c7e00640 1 -- 172.21.15.131:0/784238772 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f50c802ad10 con 0x7f50d00d0560 2024-05-26T22:54:46.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.883+0000 7f50c7e00640 1 --2- 172.21.15.131:0/784238772 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f50ac06be80 0x7f50ac06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:46.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.883+0000 7f50c7e00640 1 -- 172.21.15.131:0/784238772 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5115+0+0 (secure 0 0 0) 0x7f50c80ab3e0 con 0x7f50d00d0560 2024-05-26T22:54:46.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.883+0000 7f50cfe00640 1 --2- 172.21.15.131:0/784238772 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f50ac06be80 0x7f50ac06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:46.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.884+0000 7f50cfe00640 1 --2- 172.21.15.131:0/784238772 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f50ac06be80 0x7f50ac06e340 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f50d00b2a80 tx=0x7f50c0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:46.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:46.887+0000 7f50c7e00640 1 -- 172.21.15.131:0/784238772 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f50c807a150 con 0x7f50d00d0560 2024-05-26T22:54:47.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.085+0000 7f50d6a00640 1 -- 172.21.15.131:0/784238772 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_export_ephemeral_distributed"} v 0) -- 0x7f50d0160660 con 0x7f50d00d0560 2024-05-26T22:54:47.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.091+0000 7f50c7e00640 1 -- 172.21.15.131:0/784238772 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_export_ephemeral_distributed"}]=0 v49) ==== 115+0+0 (secure 0 0 0) 0x7f50c8079a50 con 0x7f50d00d0560 2024-05-26T22:54:47.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.093+0000 7f50d6a00640 1 -- 172.21.15.131:0/784238772 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f50ac06be80 msgr2=0x7f50ac06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:47.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.093+0000 7f50d6a00640 1 --2- 172.21.15.131:0/784238772 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f50ac06be80 0x7f50ac06e340 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f50d00b2a80 tx=0x7f50c0009040 comp rx=0 tx=0).stop 2024-05-26T22:54:47.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.094+0000 7f50d6a00640 1 -- 172.21.15.131:0/784238772 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50d00d0560 msgr2=0x7f50d00ae480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:47.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.094+0000 7f50d6a00640 1 --2- 172.21.15.131:0/784238772 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50d00d0560 0x7f50d00ae480 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f50c8009870 tx=0x7f50c80046c0 comp rx=0 tx=0).stop 2024-05-26T22:54:47.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.094+0000 7f50d6a00640 1 -- 172.21.15.131:0/784238772 shutdown_connections 2024-05-26T22:54:47.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.094+0000 7f50d6a00640 1 --2- 172.21.15.131:0/784238772 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f50d0072540 0x7f50d00adf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:47.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.094+0000 7f50d6a00640 1 --2- 172.21.15.131:0/784238772 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f50ac06be80 0x7f50ac06e340 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:47.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.094+0000 7f50d6a00640 1 --2- 172.21.15.131:0/784238772 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50d00d0560 0x7f50d00ae480 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:47.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.094+0000 7f50d6a00640 1 --2- 172.21.15.131:0/784238772 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f50d006f180 0x7f50d015ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:47.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.094+0000 7f50d6a00640 1 -- 172.21.15.131:0/784238772 >> 172.21.15.131:0/784238772 conn(0x7f50d006bc90 msgr2=0x7f50d0160d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:47.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.094+0000 7f50d6a00640 1 -- 172.21.15.131:0/784238772 shutdown_connections 2024-05-26T22:54:47.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.094+0000 7f50d6a00640 1 -- 172.21.15.131:0/784238772 wait complete. 2024-05-26T22:54:47.112 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-26T22:54:47.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.310+0000 7f671f200640 1 Processor -- start 2024-05-26T22:54:47.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.310+0000 7f671f200640 1 -- start start 2024-05-26T22:54:47.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.312+0000 7f671f200640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f67201554f0 0x7f6720071f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:47.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.312+0000 7f671f200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f67200724a0 0x7f672006e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:47.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.312+0000 7f671f200640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67201558d0 con 0x7f672015c220 2024-05-26T22:54:47.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.312+0000 7f671f200640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f67200729b0 con 0x7f67201554f0 2024-05-26T22:54:47.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.312+0000 7f671f200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f672015c750 con 0x7f67200724a0 2024-05-26T22:54:47.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.313+0000 7f671de00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f67201554f0 0x7f6720071f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:47.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.313+0000 7f6717e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f67200724a0 0x7f672006e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:47.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.313+0000 7f671de00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f67201554f0 0x7f6720071f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.131:55496/0 (socket says 172.21.15.131:55496) 2024-05-26T22:54:47.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.313+0000 7f671de00640 1 -- 172.21.15.131:0/348795093 learned_addr learned my addr 172.21.15.131:0/348795093 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:54:47.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.313+0000 7f671de00640 1 -- 172.21.15.131:0/348795093 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f67200724a0 msgr2=0x7f672006e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:47.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.313+0000 7f671de00640 1 --2- 172.21.15.131:0/348795093 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f67200724a0 0x7f672006e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:47.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.313+0000 7f671de00640 1 -- 172.21.15.131:0/348795093 >> v1:172.21.15.187:6789/0 conn(0x7f672015c220 legacy=0x7f672015ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:47.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.313+0000 7f671de00640 1 -- 172.21.15.131:0/348795093 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f672015ca30 con 0x7f67201554f0 2024-05-26T22:54:47.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.314+0000 7f671de00640 1 --2- 172.21.15.131:0/348795093 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f67201554f0 0x7f6720071f60 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f6710009870 tx=0x7f671002a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e81fd06d6102728 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:47.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.315+0000 7f671d400640 1 -- 172.21.15.131:0/348795093 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6710037070 con 0x7f67201554f0 2024-05-26T22:54:47.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.315+0000 7f671d400640 1 -- 172.21.15.131:0/348795093 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6710032440 con 0x7f67201554f0 2024-05-26T22:54:47.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.315+0000 7f671d400640 1 -- 172.21.15.131:0/348795093 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6710032810 con 0x7f67201554f0 2024-05-26T22:54:47.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.315+0000 7f671f200640 1 -- 172.21.15.131:0/348795093 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f67201554f0 msgr2=0x7f6720071f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:47.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.315+0000 7f671f200640 1 --2- 172.21.15.131:0/348795093 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f67201554f0 0x7f6720071f60 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f6710009870 tx=0x7f671002a250 comp rx=0 tx=0).stop 2024-05-26T22:54:47.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.316+0000 7f671f200640 1 -- 172.21.15.131:0/348795093 shutdown_connections 2024-05-26T22:54:47.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.316+0000 7f671f200640 1 --2- 172.21.15.131:0/348795093 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f67200724a0 0x7f672006e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:47.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.316+0000 7f671f200640 1 --2- 172.21.15.131:0/348795093 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f67201554f0 0x7f6720071f60 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:47.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.316+0000 7f671f200640 1 -- 172.21.15.131:0/348795093 >> 172.21.15.131:0/348795093 conn(0x7f67200cc7f0 msgr2=0x7f67200ccbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:47.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.316+0000 7f671f200640 1 -- 172.21.15.131:0/348795093 shutdown_connections 2024-05-26T22:54:47.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.316+0000 7f671f200640 1 -- 172.21.15.131:0/348795093 wait complete. 2024-05-26T22:54:47.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.317+0000 7f671f200640 1 Processor -- start 2024-05-26T22:54:47.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.318+0000 7f671f200640 1 -- start start 2024-05-26T22:54:47.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.318+0000 7f671f200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f67200724a0 0x7f672014a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:47.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.318+0000 7f671f200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f67201554f0 0x7f67200b58c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:47.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.318+0000 7f671f200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f672015c220 0x7f67200b1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:47.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.318+0000 7f671f200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f672015e3c0 con 0x7f67201554f0 2024-05-26T22:54:47.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.318+0000 7f671f200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f672015e240 con 0x7f672015c220 2024-05-26T22:54:47.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.318+0000 7f671f200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f672015e540 con 0x7f67200724a0 2024-05-26T22:54:47.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.319+0000 7f6717e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f67201554f0 0x7f67200b58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:47.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.319+0000 7f6717e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f67201554f0 0x7f67200b58c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59236/0 (socket says 172.21.15.131:59236) 2024-05-26T22:54:47.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.319+0000 7f6717e00640 1 -- 172.21.15.131:0/150878677 learned_addr learned my addr 172.21.15.131:0/150878677 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:54:47.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.319+0000 7f671de00640 1 --2- 172.21.15.131:0/150878677 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f67200724a0 0x7f672014a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:47.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.319+0000 7f671e800640 1 --2- 172.21.15.131:0/150878677 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f672015c220 0x7f67200b1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:47.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.319+0000 7f6717e00640 1 -- 172.21.15.131:0/150878677 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f67200724a0 msgr2=0x7f672014a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:47.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.319+0000 7f6717e00640 1 --2- 172.21.15.131:0/150878677 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f67200724a0 0x7f672014a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:47.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.319+0000 7f6717e00640 1 -- 172.21.15.131:0/150878677 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f672015c220 msgr2=0x7f67200b1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:47.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.319+0000 7f6717e00640 1 --2- 172.21.15.131:0/150878677 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f672015c220 0x7f67200b1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:47.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.319+0000 7f6717e00640 1 -- 172.21.15.131:0/150878677 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6720136ac0 con 0x7f67201554f0 2024-05-26T22:54:47.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.319+0000 7f6717e00640 1 --2- 172.21.15.131:0/150878677 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f67201554f0 0x7f67200b58c0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f670800e840 tx=0x7f670800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:47.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.320+0000 7f6716000640 1 -- 172.21.15.131:0/150878677 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6708017070 con 0x7f67201554f0 2024-05-26T22:54:47.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.320+0000 7f6716000640 1 -- 172.21.15.131:0/150878677 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f670800c940 con 0x7f67201554f0 2024-05-26T22:54:47.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.320+0000 7f6716000640 1 -- 172.21.15.131:0/150878677 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f670800cd10 con 0x7f67201554f0 2024-05-26T22:54:47.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.320+0000 7f671f200640 1 -- 172.21.15.131:0/150878677 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6720136da0 con 0x7f67201554f0 2024-05-26T22:54:47.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.320+0000 7f671f200640 1 -- 172.21.15.131:0/150878677 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6720137360 con 0x7f67201554f0 2024-05-26T22:54:47.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.321+0000 7f671f200640 1 -- 172.21.15.131:0/150878677 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67200694e0 con 0x7f67201554f0 2024-05-26T22:54:47.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.322+0000 7f6716000640 1 -- 172.21.15.131:0/150878677 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f670801a070 con 0x7f67201554f0 2024-05-26T22:54:47.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.323+0000 7f6716000640 1 --2- 172.21.15.131:0/150878677 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f66fc06be60 0x7f66fc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:47.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.323+0000 7f6716000640 1 -- 172.21.15.131:0/150878677 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5115+0+0 (secure 0 0 0) 0x7f670808aeb0 con 0x7f67201554f0 2024-05-26T22:54:47.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.323+0000 7f671de00640 1 --2- 172.21.15.131:0/150878677 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f66fc06be60 0x7f66fc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:47.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.324+0000 7f671de00640 1 --2- 172.21.15.131:0/150878677 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f66fc06be60 0x7f66fc06e320 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f6710034000 tx=0x7f671002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:47.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.328+0000 7f6716000640 1 -- 172.21.15.131:0/150878677 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f6708059a10 con 0x7f67201554f0 2024-05-26T22:54:47.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.528+0000 7f671f200640 1 -- 172.21.15.131:0/150878677 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f672014a9e0 con 0x7f67201554f0 2024-05-26T22:54:47.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.533+0000 7f6716000640 1 -- 172.21.15.131:0/150878677 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v50) ==== 104+0+0 (secure 0 0 0) 0x7f6708059310 con 0x7f67201554f0 2024-05-26T22:54:47.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.536+0000 7f671f200640 1 -- 172.21.15.131:0/150878677 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f66fc06be60 msgr2=0x7f66fc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:47.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.536+0000 7f671f200640 1 --2- 172.21.15.131:0/150878677 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f66fc06be60 0x7f66fc06e320 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f6710034000 tx=0x7f671002e000 comp rx=0 tx=0).stop 2024-05-26T22:54:47.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.536+0000 7f671f200640 1 -- 172.21.15.131:0/150878677 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f67201554f0 msgr2=0x7f67200b58c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:47.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.536+0000 7f671f200640 1 --2- 172.21.15.131:0/150878677 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f67201554f0 0x7f67200b58c0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f670800e840 tx=0x7f670800ed00 comp rx=0 tx=0).stop 2024-05-26T22:54:47.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.536+0000 7f671f200640 1 -- 172.21.15.131:0/150878677 shutdown_connections 2024-05-26T22:54:47.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.536+0000 7f671f200640 1 --2- 172.21.15.131:0/150878677 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f66fc06be60 0x7f66fc06e320 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:47.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.536+0000 7f671f200640 1 --2- 172.21.15.131:0/150878677 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f672015c220 0x7f67200b1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:47.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.536+0000 7f671f200640 1 --2- 172.21.15.131:0/150878677 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f67201554f0 0x7f67200b58c0 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:47.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.536+0000 7f671f200640 1 --2- 172.21.15.131:0/150878677 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f67200724a0 0x7f672014a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:47.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.536+0000 7f671f200640 1 -- 172.21.15.131:0/150878677 >> 172.21.15.131:0/150878677 conn(0x7f67200cc7f0 msgr2=0x7f672006e5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:47.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.536+0000 7f671f200640 1 -- 172.21.15.131:0/150878677 shutdown_connections 2024-05-26T22:54:47.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.536+0000 7f671f200640 1 -- 172.21.15.131:0/150878677 wait complete. 2024-05-26T22:54:47.556 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_exports.TestEphemeralPins.test_ephemeral_pin_dist_override_after' 2024-05-26T22:54:47.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.758+0000 7f0d21e00640 1 Processor -- start 2024-05-26T22:54:47.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.758+0000 7f0d21e00640 1 -- start start 2024-05-26T22:54:47.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.759+0000 7f0d21e00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f0d1c06dcc0 0x7f0d1c0b3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:47.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.759+0000 7f0d21e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0d1c0b36d0 0x7f0d1c15fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:47.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.759+0000 7f0d21e00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d1c0b3e50 con 0x7f0d1c158c00 2024-05-26T22:54:47.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.759+0000 7f0d21e00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f0d1c0b4130 con 0x7f0d1c06dcc0 2024-05-26T22:54:47.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.759+0000 7f0d21e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f0d1c0b4410 con 0x7f0d1c0b36d0 2024-05-26T22:54:47.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.760+0000 7f0d12000640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f0d1c06dcc0 0x7f0d1c0b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:47.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.760+0000 7f0d12000640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f0d1c06dcc0 0x7f0d1c0b3190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.131:55518/0 (socket says 172.21.15.131:55518) 2024-05-26T22:54:47.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.760+0000 7f0d12000640 1 -- 172.21.15.131:0/414514088 learned_addr learned my addr 172.21.15.131:0/414514088 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:54:47.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.760+0000 7f0d1b400640 1 --2- 172.21.15.131:0/414514088 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0d1c0b36d0 0x7f0d1c15fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:47.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.761+0000 7f0d12000640 1 -- 172.21.15.131:0/414514088 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0d1c0b36d0 msgr2=0x7f0d1c15fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:47.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.761+0000 7f0d12000640 1 --2- 172.21.15.131:0/414514088 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0d1c0b36d0 0x7f0d1c15fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:47.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.761+0000 7f0d12000640 1 -- 172.21.15.131:0/414514088 >> v1:172.21.15.187:6789/0 conn(0x7f0d1c158c00 legacy=0x7f0d1c06d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:47.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.761+0000 7f0d12000640 1 -- 172.21.15.131:0/414514088 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d1c0b46f0 con 0x7f0d1c06dcc0 2024-05-26T22:54:47.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.762+0000 7f0d12000640 1 --2- 172.21.15.131:0/414514088 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0d1c06dcc0 0x7f0d1c0b3190 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f0d04009870 tx=0x7f0d0402a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14c58a7e96f91b06 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:47.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.763+0000 7f0d1a000640 1 -- 172.21.15.131:0/414514088 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d04037070 con 0x7f0d1c06dcc0 2024-05-26T22:54:47.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.763+0000 7f0d1a000640 1 -- 172.21.15.131:0/414514088 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d04032440 con 0x7f0d1c06dcc0 2024-05-26T22:54:47.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.763+0000 7f0d1a000640 1 -- 172.21.15.131:0/414514088 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d04032810 con 0x7f0d1c06dcc0 2024-05-26T22:54:47.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.763+0000 7f0d21e00640 1 -- 172.21.15.131:0/414514088 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0d1c06dcc0 msgr2=0x7f0d1c0b3190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:47.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.763+0000 7f0d21e00640 1 --2- 172.21.15.131:0/414514088 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0d1c06dcc0 0x7f0d1c0b3190 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f0d04009870 tx=0x7f0d0402a1c0 comp rx=0 tx=0).stop 2024-05-26T22:54:47.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.763+0000 7f0d21e00640 1 -- 172.21.15.131:0/414514088 shutdown_connections 2024-05-26T22:54:47.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.763+0000 7f0d21e00640 1 --2- 172.21.15.131:0/414514088 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0d1c0b36d0 0x7f0d1c15fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:47.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.763+0000 7f0d21e00640 1 --2- 172.21.15.131:0/414514088 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0d1c06dcc0 0x7f0d1c0b3190 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:47.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.763+0000 7f0d21e00640 1 -- 172.21.15.131:0/414514088 >> 172.21.15.131:0/414514088 conn(0x7f0d1c0b1e90 msgr2=0x7f0d1c0b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:47.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.764+0000 7f0d21e00640 1 -- 172.21.15.131:0/414514088 shutdown_connections 2024-05-26T22:54:47.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.764+0000 7f0d21e00640 1 -- 172.21.15.131:0/414514088 wait complete. 2024-05-26T22:54:47.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.765+0000 7f0d21e00640 1 Processor -- start 2024-05-26T22:54:47.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.765+0000 7f0d21e00640 1 -- start start 2024-05-26T22:54:47.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.766+0000 7f0d21e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0d1c06dcc0 0x7f0d1c159380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:47.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.766+0000 7f0d21e00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0d1c0b36d0 0x7f0d1c099950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:47.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.766+0000 7f0d21e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0d1c158c00 0x7f0d1c15eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:47.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.766+0000 7f0d21e00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f0d1c160190 con 0x7f0d1c06dcc0 2024-05-26T22:54:47.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.766+0000 7f0d21e00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0d1c0b3b60 con 0x7f0d1c0b36d0 2024-05-26T22:54:47.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.766+0000 7f0d21e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f0d1c160310 con 0x7f0d1c158c00 2024-05-26T22:54:47.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.767+0000 7f0d1aa00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0d1c06dcc0 0x7f0d1c159380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:47.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.767+0000 7f0d1aa00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0d1c06dcc0 0x7f0d1c159380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59252/0 (socket says 172.21.15.131:59252) 2024-05-26T22:54:47.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.767+0000 7f0d1aa00640 1 -- 172.21.15.131:0/1467892103 learned_addr learned my addr 172.21.15.131:0/1467892103 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:54:47.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.767+0000 7f0d12000640 1 --2- 172.21.15.131:0/1467892103 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0d1c0b36d0 0x7f0d1c099950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:47.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.767+0000 7f0d1b400640 1 --2- 172.21.15.131:0/1467892103 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0d1c158c00 0x7f0d1c15eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:47.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.767+0000 7f0d1aa00640 1 -- 172.21.15.131:0/1467892103 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0d1c158c00 msgr2=0x7f0d1c15eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:47.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.767+0000 7f0d1aa00640 1 --2- 172.21.15.131:0/1467892103 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0d1c158c00 0x7f0d1c15eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:47.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.767+0000 7f0d1aa00640 1 -- 172.21.15.131:0/1467892103 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0d1c0b36d0 msgr2=0x7f0d1c099950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:47.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.767+0000 7f0d1aa00640 1 --2- 172.21.15.131:0/1467892103 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0d1c0b36d0 0x7f0d1c099950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:47.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.767+0000 7f0d1aa00640 1 -- 172.21.15.131:0/1467892103 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d1c099e90 con 0x7f0d1c06dcc0 2024-05-26T22:54:47.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.767+0000 7f0d1b400640 1 --2- 172.21.15.131:0/1467892103 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0d1c158c00 0x7f0d1c15eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:54:47.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.768+0000 7f0d1aa00640 1 --2- 172.21.15.131:0/1467892103 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0d1c06dcc0 0x7f0d1c159380 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c00c3c0 tx=0x7f0d0c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:47.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.768+0000 7f0d13e00640 1 -- 172.21.15.131:0/1467892103 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c0052d0 con 0x7f0d1c06dcc0 2024-05-26T22:54:47.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.768+0000 7f0d13e00640 1 -- 172.21.15.131:0/1467892103 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d0c005490 con 0x7f0d1c06dcc0 2024-05-26T22:54:47.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.768+0000 7f0d13e00640 1 -- 172.21.15.131:0/1467892103 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c007260 con 0x7f0d1c06dcc0 2024-05-26T22:54:47.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.768+0000 7f0d21e00640 1 -- 172.21.15.131:0/1467892103 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d1c149250 con 0x7f0d1c06dcc0 2024-05-26T22:54:47.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.768+0000 7f0d21e00640 1 -- 172.21.15.131:0/1467892103 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d1c1494f0 con 0x7f0d1c06dcc0 2024-05-26T22:54:47.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.769+0000 7f0d21e00640 1 -- 172.21.15.131:0/1467892103 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d1c0694e0 con 0x7f0d1c06dcc0 2024-05-26T22:54:47.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.770+0000 7f0d13e00640 1 -- 172.21.15.131:0/1467892103 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0d0c007400 con 0x7f0d1c06dcc0 2024-05-26T22:54:47.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.771+0000 7f0d13e00640 1 --2- 172.21.15.131:0/1467892103 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f0cf806be30 0x7f0cf806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:47.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.771+0000 7f0d13e00640 1 -- 172.21.15.131:0/1467892103 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5115+0+0 (secure 0 0 0) 0x7f0d0c08a190 con 0x7f0d1c06dcc0 2024-05-26T22:54:47.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.771+0000 7f0d12000640 1 --2- 172.21.15.131:0/1467892103 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f0cf806be30 0x7f0cf806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:47.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.772+0000 7f0d12000640 1 --2- 172.21.15.131:0/1467892103 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f0cf806be30 0x7f0cf806e2f0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f0d04034000 tx=0x7f0d0402e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:47.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:47.776+0000 7f0d13e00640 1 -- 172.21.15.131:0/1467892103 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f0d1c099e90 con 0x7f0d1c06dcc0 2024-05-26T22:54:48.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:48.049+0000 7f0d21e00640 1 -- 172.21.15.131:0/1467892103 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_exports.TestEphemeralPins.test_ephemeral_pin_dist_override_after"]} v 0) -- 0x7f0d1c06c590 con 0x7f0d1c06dcc0 2024-05-26T22:54:48.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:48.789+0000 7f0d13e00640 1 -- 172.21.15.131:0/1467892103 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_exports.TestEphemeralPins.test_ephemeral_pin_dist_override_after"]}]=0 v328) ==== 161+0+0 (secure 0 0 0) 0x7f0d0c058cf0 con 0x7f0d1c06dcc0 2024-05-26T22:54:48.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:48.792+0000 7f0d21e00640 1 -- 172.21.15.131:0/1467892103 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f0cf806be30 msgr2=0x7f0cf806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:48.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:48.792+0000 7f0d21e00640 1 --2- 172.21.15.131:0/1467892103 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f0cf806be30 0x7f0cf806e2f0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f0d04034000 tx=0x7f0d0402e000 comp rx=0 tx=0).stop 2024-05-26T22:54:48.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:48.792+0000 7f0d21e00640 1 -- 172.21.15.131:0/1467892103 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0d1c06dcc0 msgr2=0x7f0d1c159380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:48.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:48.792+0000 7f0d21e00640 1 --2- 172.21.15.131:0/1467892103 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0d1c06dcc0 0x7f0d1c159380 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c00c3c0 tx=0x7f0d0c00c880 comp rx=0 tx=0).stop 2024-05-26T22:54:48.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:48.793+0000 7f0d21e00640 1 -- 172.21.15.131:0/1467892103 shutdown_connections 2024-05-26T22:54:48.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:48.793+0000 7f0d21e00640 1 --2- 172.21.15.131:0/1467892103 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0d1c158c00 0x7f0d1c15eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:48.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:48.793+0000 7f0d21e00640 1 --2- 172.21.15.131:0/1467892103 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f0cf806be30 0x7f0cf806e2f0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:48.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:48.793+0000 7f0d21e00640 1 --2- 172.21.15.131:0/1467892103 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0d1c0b36d0 0x7f0d1c099950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:48.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:48.793+0000 7f0d21e00640 1 --2- 172.21.15.131:0/1467892103 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0d1c06dcc0 0x7f0d1c159380 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:48.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:48.793+0000 7f0d21e00640 1 -- 172.21.15.131:0/1467892103 >> 172.21.15.131:0/1467892103 conn(0x7f0d1c0b1e90 msgr2=0x7f0d1c0afe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:48.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:48.793+0000 7f0d21e00640 1 -- 172.21.15.131:0/1467892103 shutdown_connections 2024-05-26T22:54:48.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:48.793+0000 7f0d21e00640 1 -- 172.21.15.131:0/1467892103 wait complete. 2024-05-26T22:54:48.813 INFO:tasks.cephfs_test_runner:That a conventional export pin overrides the distributed policy _after_ distributed policy is set. ... ok 2024-05-26T22:54:48.814 INFO:tasks.cephfs_test_runner:Starting test: test_ephemeral_pin_dist_override_before (tasks.cephfs.test_exports.TestEphemeralPins) That a conventional export pin overrides the distributed policy _before_ distributed policy is set. 2024-05-26T22:54:48.814 INFO:tasks.cephfs_test_runner:test_ephemeral_pin_dist_override_before (tasks.cephfs.test_exports.TestEphemeralPins) 2024-05-26T22:54:48.814 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-26T22:54:49.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.014+0000 7ff951400640 1 Processor -- start 2024-05-26T22:54:49.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.014+0000 7ff951400640 1 -- start start 2024-05-26T22:54:49.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.016+0000 7ff951400640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7ff94c155840 0x7ff94c071930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:49.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.016+0000 7ff951400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff94c15c3d0 0x7ff94c06ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:49.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.016+0000 7ff951400640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff94c0722a0 con 0x7ff94c06f490 2024-05-26T22:54:49.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.016+0000 7ff951400640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7ff94c072580 con 0x7ff94c155840 2024-05-26T22:54:49.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.016+0000 7ff951400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff94c072860 con 0x7ff94c15c3d0 2024-05-26T22:54:49.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.016+0000 7ff94b200640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7ff94c06f490 0x7ff94c15ec80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:35264/0 (socket says 172.21.15.131:35264) 2024-05-26T22:54:49.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.016+0000 7ff94b200640 1 -- 172.21.15.131:0/4193011994 learned_addr learned my addr 172.21.15.131:0/4193011994 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:54:49.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.017+0000 7ff94a800640 1 --2- 172.21.15.131:0/4193011994 >> v2:172.21.15.187:3300/0 conn(0x7ff94c155840 0x7ff94c071930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:49.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.017+0000 7ff941e00640 1 --2- 172.21.15.131:0/4193011994 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff94c15c3d0 0x7ff94c06ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:49.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.017+0000 7ff949e00640 1 -- 172.21.15.131:0/4193011994 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 373761869 0 0) 0x7ff94c0722a0 con 0x7ff94c06f490 2024-05-26T22:54:49.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.017+0000 7ff949e00640 1 -- 172.21.15.131:0/4193011994 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff930003520 con 0x7ff94c06f490 2024-05-26T22:54:49.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.017+0000 7ff949e00640 1 -- 172.21.15.131:0/4193011994 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 886751031 0 0) 0x7ff930003520 con 0x7ff94c06f490 2024-05-26T22:54:49.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.017+0000 7ff949e00640 1 -- 172.21.15.131:0/4193011994 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff94c15c3d0 msgr2=0x7ff94c06ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:49.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.017+0000 7ff949e00640 1 --2- 172.21.15.131:0/4193011994 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff94c15c3d0 0x7ff94c06ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:49.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.017+0000 7ff949e00640 1 -- 172.21.15.131:0/4193011994 >> v2:172.21.15.187:3300/0 conn(0x7ff94c155840 msgr2=0x7ff94c071930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:49.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.017+0000 7ff949e00640 1 --2- 172.21.15.131:0/4193011994 >> v2:172.21.15.187:3300/0 conn(0x7ff94c155840 0x7ff94c071930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:49.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.017+0000 7ff949e00640 1 -- 172.21.15.131:0/4193011994 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff94c072b40 con 0x7ff94c06f490 2024-05-26T22:54:49.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.017+0000 7ff949e00640 1 -- 172.21.15.131:0/4193011994 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7ff9440019f0 con 0x7ff94c06f490 2024-05-26T22:54:49.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.017+0000 7ff949e00640 1 -- 172.21.15.131:0/4193011994 >> v1:172.21.15.131:6789/0 conn(0x7ff94c06f490 legacy=0x7ff94c15ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:49.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.017+0000 7ff949e00640 1 --2- 172.21.15.131:0/4193011994 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff9300039b0 0x7ff930003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:49.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.017+0000 7ff949e00640 1 --2- 172.21.15.131:0/4193011994 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff930004800 0x7ff930004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:49.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.017+0000 7ff949e00640 1 --2- 172.21.15.131:0/4193011994 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff94c15c3d0 0x7ff9300051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:49.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.017+0000 7ff949e00640 1 -- 172.21.15.131:0/4193011994 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff94c072580 con 0x7ff9300039b0 2024-05-26T22:54:49.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.017+0000 7ff949e00640 1 -- 172.21.15.131:0/4193011994 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff94c072860 con 0x7ff930004800 2024-05-26T22:54:49.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.017+0000 7ff949e00640 1 -- 172.21.15.131:0/4193011994 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff93000b9c0 con 0x7ff94c15c3d0 2024-05-26T22:54:49.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.018+0000 7ff94a800640 1 --2- 172.21.15.131:0/4193011994 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff9300039b0 0x7ff930003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:49.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.018+0000 7ff941e00640 1 --2- 172.21.15.131:0/4193011994 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff930004800 0x7ff930004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:49.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.018+0000 7ff94b200640 1 --2- 172.21.15.131:0/4193011994 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff94c15c3d0 0x7ff9300051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:49.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.018+0000 7ff94a800640 1 -- 172.21.15.131:0/4193011994 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff94c15c3d0 msgr2=0x7ff9300051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:49.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.018+0000 7ff94a800640 1 --2- 172.21.15.131:0/4193011994 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff94c15c3d0 0x7ff9300051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:49.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.018+0000 7ff94a800640 1 -- 172.21.15.131:0/4193011994 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff930004800 msgr2=0x7ff930004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:49.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.018+0000 7ff94a800640 1 --2- 172.21.15.131:0/4193011994 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff930004800 0x7ff930004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:49.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.018+0000 7ff94a800640 1 -- 172.21.15.131:0/4193011994 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff930003520 con 0x7ff9300039b0 2024-05-26T22:54:49.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.018+0000 7ff941e00640 1 --2- 172.21.15.131:0/4193011994 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff930004800 0x7ff930004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:54:49.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.019+0000 7ff94a800640 1 --2- 172.21.15.131:0/4193011994 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff9300039b0 0x7ff930003dd0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7ff93c009ac0 tx=0x7ff930005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8618857d4f67e239 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:49.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.019+0000 7ff949e00640 1 -- 172.21.15.131:0/4193011994 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff93c036070 con 0x7ff9300039b0 2024-05-26T22:54:49.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.019+0000 7ff949e00640 1 -- 172.21.15.131:0/4193011994 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff93c031400 con 0x7ff9300039b0 2024-05-26T22:54:49.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.019+0000 7ff949e00640 1 -- 172.21.15.131:0/4193011994 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff93c009690 con 0x7ff9300039b0 2024-05-26T22:54:49.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.020+0000 7ff951400640 1 -- 172.21.15.131:0/4193011994 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff9300039b0 msgr2=0x7ff930003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:49.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.020+0000 7ff951400640 1 --2- 172.21.15.131:0/4193011994 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff9300039b0 0x7ff930003dd0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7ff93c009ac0 tx=0x7ff930005830 comp rx=0 tx=0).stop 2024-05-26T22:54:49.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.020+0000 7ff94b200640 1 -- 172.21.15.131:0/4193011994 reap_dead start 2024-05-26T22:54:49.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.020+0000 7ff951400640 1 -- 172.21.15.131:0/4193011994 shutdown_connections 2024-05-26T22:54:49.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.020+0000 7ff951400640 1 -- 172.21.15.131:0/4193011994 >> 172.21.15.131:0/4193011994 conn(0x7ff94c0b1e90 msgr2=0x7ff94c0b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:49.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.020+0000 7ff951400640 1 -- 172.21.15.131:0/4193011994 shutdown_connections 2024-05-26T22:54:49.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.020+0000 7ff951400640 1 -- 172.21.15.131:0/4193011994 wait complete. 2024-05-26T22:54:49.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.022+0000 7ff951400640 1 Processor -- start 2024-05-26T22:54:49.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.022+0000 7ff951400640 1 -- start start 2024-05-26T22:54:49.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.023+0000 7ff951400640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff9300039b0 0x7ff94c155c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:49.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.023+0000 7ff951400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff94c0b1270 0x7ff94c0998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:49.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.023+0000 7ff951400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff94c15d800 0x7ff94c15dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:49.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.023+0000 7ff951400640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff94c071ff0 con 0x7ff94c15d800 2024-05-26T22:54:49.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.023+0000 7ff951400640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff94c071e70 con 0x7ff9300039b0 2024-05-26T22:54:49.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.023+0000 7ff951400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff94c0aee50 con 0x7ff94c0b1270 2024-05-26T22:54:49.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.023+0000 7ff94b200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff94c15d800 0x7ff94c15dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:49.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.023+0000 7ff94b200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff94c15d800 0x7ff94c15dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59270/0 (socket says 172.21.15.131:59270) 2024-05-26T22:54:49.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.023+0000 7ff94b200640 1 -- 172.21.15.131:0/821090770 learned_addr learned my addr 172.21.15.131:0/821090770 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:54:49.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.024+0000 7ff94a800640 1 --2- 172.21.15.131:0/821090770 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff9300039b0 0x7ff94c155c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:49.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.024+0000 7ff941e00640 1 --2- 172.21.15.131:0/821090770 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff94c0b1270 0x7ff94c0998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:49.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.024+0000 7ff94b200640 1 -- 172.21.15.131:0/821090770 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff94c0b1270 msgr2=0x7ff94c0998e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:49.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.024+0000 7ff94b200640 1 --2- 172.21.15.131:0/821090770 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff94c0b1270 0x7ff94c0998e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:49.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.024+0000 7ff94b200640 1 -- 172.21.15.131:0/821090770 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff9300039b0 msgr2=0x7ff94c155c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:49.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.024+0000 7ff94b200640 1 --2- 172.21.15.131:0/821090770 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff9300039b0 0x7ff94c155c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:49.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.024+0000 7ff94b200640 1 -- 172.21.15.131:0/821090770 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff94c0b0680 con 0x7ff94c15d800 2024-05-26T22:54:49.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.024+0000 7ff94b200640 1 --2- 172.21.15.131:0/821090770 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff94c15d800 0x7ff94c15dc80 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7ff94c159280 tx=0x7ff94400b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:49.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.024+0000 7ff943400640 1 -- 172.21.15.131:0/821090770 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff94400de90 con 0x7ff94c15d800 2024-05-26T22:54:49.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.024+0000 7ff943400640 1 -- 172.21.15.131:0/821090770 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff94400bb80 con 0x7ff94c15d800 2024-05-26T22:54:49.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.024+0000 7ff943400640 1 -- 172.21.15.131:0/821090770 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff944007110 con 0x7ff94c15d800 2024-05-26T22:54:49.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.024+0000 7ff951400640 1 -- 172.21.15.131:0/821090770 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff94c0af5e0 con 0x7ff94c15d800 2024-05-26T22:54:49.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.025+0000 7ff951400640 1 -- 172.21.15.131:0/821090770 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff94c0afba0 con 0x7ff94c15d800 2024-05-26T22:54:49.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.026+0000 7ff951400640 1 -- 172.21.15.131:0/821090770 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff94c15e1c0 con 0x7ff94c15d800 2024-05-26T22:54:49.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.027+0000 7ff943400640 1 -- 172.21.15.131:0/821090770 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff94401b020 con 0x7ff94c15d800 2024-05-26T22:54:49.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.027+0000 7ff943400640 1 --2- 172.21.15.131:0/821090770 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ff92806bfd0 0x7ff92806e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:49.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.028+0000 7ff943400640 1 -- 172.21.15.131:0/821090770 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5115+0+0 (secure 0 0 0) 0x7ff94408af10 con 0x7ff94c15d800 2024-05-26T22:54:49.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.028+0000 7ff94a800640 1 --2- 172.21.15.131:0/821090770 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ff92806bfd0 0x7ff92806e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:49.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.028+0000 7ff94a800640 1 --2- 172.21.15.131:0/821090770 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ff92806bfd0 0x7ff92806e490 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7ff93c033000 tx=0x7ff93c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:49.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.032+0000 7ff943400640 1 -- 172.21.15.131:0/821090770 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7ff944059b00 con 0x7ff94c15d800 2024-05-26T22:54:49.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.234+0000 7ff951400640 1 -- 172.21.15.131:0/821090770 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff94c0694e0 con 0x7ff94c15d800 2024-05-26T22:54:49.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.236+0000 7ff943400640 1 -- 172.21.15.131:0/821090770 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+13309 (secure 0 0 0) 0x7ff944059400 con 0x7ff94c15d800 2024-05-26T22:54:49.237 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:54:49.237 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":87,"fsid":"46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f","created":"2024-05-26T22:47:57.288905+0000","modified":"2024-05-26T22:54:45.756565+0000","last_up_change":"2024-05-26T22:48:09.602207+0000","last_in_change":"2024-05-26T22:48:00.687341+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-26T22:48:10.463339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"814eaf62-a18d-494b-8e27-fd8fb88adea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6801","nonce":2350940051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6803","nonce":2350940051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6807","nonce":2350940051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6805","nonce":2350940051}]},"public_addr":"172.21.15.131:6801/2350940051","cluster_addr":"172.21.15.131:6803/2350940051","heartbeat_back_addr":"172.21.15.131:6807/2350940051","heartbeat_front_addr":"172.21.15.131:6805/2350940051","state":["exists","up"]},{"osd":1,"uuid":"42e66287-f6c6-4e26-af3c-6dfaa3536d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6825","nonce":1525528247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6827","nonce":1525528247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6831","nonce":1525528247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6829","nonce":1525528247}]},"public_addr":"172.21.15.131:6825/1525528247","cluster_addr":"172.21.15.131:6827/1525528247","heartbeat_back_addr":"172.21.15.131:6831/1525528247","heartbeat_front_addr":"172.21.15.131:6829/1525528247","state":["exists","up"]},{"osd":2,"uuid":"0da34673-04a2-4fd0-b57a-0ff2b679b043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6817","nonce":3627842210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6819","nonce":3627842210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6823","nonce":3627842210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6821","nonce":3627842210}]},"public_addr":"172.21.15.131:6817/3627842210","cluster_addr":"172.21.15.131:6819/3627842210","heartbeat_back_addr":"172.21.15.131:6823/3627842210","heartbeat_front_addr":"172.21.15.131:6821/3627842210","state":["exists","up"]},{"osd":3,"uuid":"cf117a27-d6b2-43d3-ae52-b47988bf8eed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6809","nonce":350790591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6811","nonce":350790591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6815","nonce":350790591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6813","nonce":350790591}]},"public_addr":"172.21.15.131:6809/350790591","cluster_addr":"172.21.15.131:6811/350790591","heartbeat_back_addr":"172.21.15.131:6815/350790591","heartbeat_front_addr":"172.21.15.131:6813/350790591","state":["exists","up"]},{"osd":4,"uuid":"a092ccfc-be89-4a84-88d5-0f781fa715eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6816","nonce":244560995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6819","nonce":244560995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6823","nonce":244560995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6821","nonce":244560995}]},"public_addr":"172.21.15.187:6816/244560995","cluster_addr":"172.21.15.187:6819/244560995","heartbeat_back_addr":"172.21.15.187:6823/244560995","heartbeat_front_addr":"172.21.15.187:6821/244560995","state":["exists","up"]},{"osd":5,"uuid":"7513ba69-dd00-4a35-b190-9e8cc3b2bdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6801","nonce":324619398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6803","nonce":324619398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6812","nonce":324619398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6807","nonce":324619398}]},"public_addr":"172.21.15.187:6801/324619398","cluster_addr":"172.21.15.187:6803/324619398","heartbeat_back_addr":"172.21.15.187:6812/324619398","heartbeat_front_addr":"172.21.15.187:6807/324619398","state":["exists","up"]},{"osd":6,"uuid":"50081810-c093-4d2f-a4d7-b7c695c5400d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6806","nonce":1391347786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6809","nonce":1391347786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6818","nonce":1391347786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6813","nonce":1391347786}]},"public_addr":"172.21.15.187:6806/1391347786","cluster_addr":"172.21.15.187:6809/1391347786","heartbeat_back_addr":"172.21.15.187:6818/1391347786","heartbeat_front_addr":"172.21.15.187:6813/1391347786","state":["exists","up"]},{"osd":7,"uuid":"775dae05-54f5-4ad4-bbce-92863c6ed2ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6825","nonce":1220065622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6827","nonce":1220065622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6831","nonce":1220065622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6829","nonce":1220065622}]},"public_addr":"172.21.15.187:6825/1220065622","cluster_addr":"172.21.15.187:6827/1220065622","heartbeat_back_addr":"172.21.15.187:6831/1220065622","heartbeat_front_addr":"172.21.15.187:6829/1220065622","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:05.983520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.299516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.067323+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.307748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.018742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.258775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.010912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.295355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.187:6835/567389303":"2024-05-27T22:54:42.746419+0000","172.21.15.187:6834/3480101957":"2024-05-27T22:54:42.746405+0000","172.21.15.187:6832/3480101957":"2024-05-27T22:54:42.746405+0000","172.21.15.187:6833/567389303":"2024-05-27T22:54:42.746419+0000","172.21.15.131:6836/3411392567":"2024-05-27T22:54:42.746423+0000","172.21.15.131:6834/3411392567":"2024-05-27T22:54:42.746423+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-26T22:54:49.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.239+0000 7ff951400640 1 -- 172.21.15.131:0/821090770 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ff92806bfd0 msgr2=0x7ff92806e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:49.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.239+0000 7ff951400640 1 --2- 172.21.15.131:0/821090770 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ff92806bfd0 0x7ff92806e490 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7ff93c033000 tx=0x7ff93c008030 comp rx=0 tx=0).stop 2024-05-26T22:54:49.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.239+0000 7ff951400640 1 -- 172.21.15.131:0/821090770 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff94c15d800 msgr2=0x7ff94c15dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:49.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.239+0000 7ff951400640 1 --2- 172.21.15.131:0/821090770 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff94c15d800 0x7ff94c15dc80 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7ff94c159280 tx=0x7ff94400b4a0 comp rx=0 tx=0).stop 2024-05-26T22:54:49.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.239+0000 7ff951400640 1 -- 172.21.15.131:0/821090770 shutdown_connections 2024-05-26T22:54:49.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.239+0000 7ff951400640 1 --2- 172.21.15.131:0/821090770 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff94c0b1270 0x7ff94c0998e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:49.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.239+0000 7ff951400640 1 --2- 172.21.15.131:0/821090770 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ff92806bfd0 0x7ff92806e490 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:49.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.239+0000 7ff951400640 1 --2- 172.21.15.131:0/821090770 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff94c15d800 0x7ff94c15dc80 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:49.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.239+0000 7ff951400640 1 --2- 172.21.15.131:0/821090770 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff9300039b0 0x7ff94c155c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:49.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.239+0000 7ff951400640 1 -- 172.21.15.131:0/821090770 >> 172.21.15.131:0/821090770 conn(0x7ff94c0b1e90 msgr2=0x7ff94c06e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:49.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.239+0000 7ff951400640 1 -- 172.21.15.131:0/821090770 shutdown_connections 2024-05-26T22:54:49.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.239+0000 7ff951400640 1 -- 172.21.15.131:0/821090770 wait complete. 2024-05-26T22:54:49.257 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-26T22:48:10.463339+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-26T22:54:49.258 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-26T22:54:49.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.456+0000 7f594bc00640 1 Processor -- start 2024-05-26T22:54:49.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.456+0000 7f594bc00640 1 -- start start 2024-05-26T22:54:49.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.458+0000 7f594bc00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f59440d0460 0x7f594406d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:49.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.458+0000 7f594bc00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5944160180 con 0x7f594406f420 2024-05-26T22:54:49.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.458+0000 7f594bc00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5944160460 con 0x7f59440b5450 2024-05-26T22:54:49.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.458+0000 7f594bc00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f5944160d90 con 0x7f59440d0460 2024-05-26T22:54:49.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.458+0000 7f5949c00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f59440d0460 0x7f594406d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:49.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.459+0000 7f5943e00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f594406f420 0x7f59440cfd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:35266/0 (socket says 172.21.15.131:35266) 2024-05-26T22:54:49.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.459+0000 7f5943e00640 1 -- 172.21.15.131:0/3424389743 learned_addr learned my addr 172.21.15.131:0/3424389743 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:54:49.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.459+0000 7f5949c00640 1 -- 172.21.15.131:0/3424389743 >> v1:172.21.15.187:6789/0 conn(0x7f59440b5450 legacy=0x7f59440b43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:49.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.459+0000 7f5949c00640 1 -- 172.21.15.131:0/3424389743 >> v1:172.21.15.131:6789/0 conn(0x7f594406f420 legacy=0x7f59440cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:49.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.459+0000 7f5949c00640 1 -- 172.21.15.131:0/3424389743 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5944160740 con 0x7f59440d0460 2024-05-26T22:54:49.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.460+0000 7f5949c00640 1 --2- 172.21.15.131:0/3424389743 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59440d0460 0x7f594406d600 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f593c009d90 tx=0x7f593c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22ffb3a8f6da110d server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:49.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.460+0000 7f5943400640 1 -- 172.21.15.131:0/3424389743 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f593c007740 con 0x7f59440d0460 2024-05-26T22:54:49.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.460+0000 7f5943400640 1 -- 172.21.15.131:0/3424389743 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f593c0078e0 con 0x7f59440d0460 2024-05-26T22:54:49.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.460+0000 7f5943400640 1 -- 172.21.15.131:0/3424389743 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f593c007cb0 con 0x7f59440d0460 2024-05-26T22:54:49.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.461+0000 7f594bc00640 1 -- 172.21.15.131:0/3424389743 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59440d0460 msgr2=0x7f594406d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:49.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.461+0000 7f594bc00640 1 --2- 172.21.15.131:0/3424389743 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59440d0460 0x7f594406d600 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f593c009d90 tx=0x7f593c02a6a0 comp rx=0 tx=0).stop 2024-05-26T22:54:49.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.461+0000 7f594bc00640 1 -- 172.21.15.131:0/3424389743 shutdown_connections 2024-05-26T22:54:49.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.461+0000 7f594bc00640 1 --2- 172.21.15.131:0/3424389743 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59440d0460 0x7f594406d600 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:49.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.461+0000 7f594bc00640 1 -- 172.21.15.131:0/3424389743 >> 172.21.15.131:0/3424389743 conn(0x7f594406bc90 msgr2=0x7f59440b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:49.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.461+0000 7f594bc00640 1 -- 172.21.15.131:0/3424389743 shutdown_connections 2024-05-26T22:54:49.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.461+0000 7f594bc00640 1 -- 172.21.15.131:0/3424389743 wait complete. 2024-05-26T22:54:49.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.463+0000 7f594bc00640 1 Processor -- start 2024-05-26T22:54:49.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.463+0000 7f594bc00640 1 -- start start 2024-05-26T22:54:49.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.464+0000 7f594bc00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f594406f420 0x7f594414a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:49.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.464+0000 7f594bc00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f59440b5450 0x7f594409b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:49.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.464+0000 7f594bc00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f59440d0460 0x7f594409b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:49.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.464+0000 7f594bc00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f594415fde0 con 0x7f594406f420 2024-05-26T22:54:49.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.464+0000 7f594bc00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f59440b5830 con 0x7f59440d0460 2024-05-26T22:54:49.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.464+0000 7f594bc00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f594415ff60 con 0x7f59440b5450 2024-05-26T22:54:49.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.464+0000 7f5949200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f594406f420 0x7f594414a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:49.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.464+0000 7f5949200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f594406f420 0x7f594414a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59282/0 (socket says 172.21.15.131:59282) 2024-05-26T22:54:49.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.464+0000 7f5949200640 1 -- 172.21.15.131:0/1127061282 learned_addr learned my addr 172.21.15.131:0/1127061282 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:54:49.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.465+0000 7f5949200640 1 -- 172.21.15.131:0/1127061282 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f59440b5450 msgr2=0x7f594409b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:49.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.465+0000 7f5949200640 1 --2- 172.21.15.131:0/1127061282 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f59440b5450 0x7f594409b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:49.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.465+0000 7f5949200640 1 -- 172.21.15.131:0/1127061282 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f59440d0460 msgr2=0x7f594409b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:49.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.465+0000 7f5949200640 1 --2- 172.21.15.131:0/1127061282 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f59440d0460 0x7f594409b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:49.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.465+0000 7f5949200640 1 -- 172.21.15.131:0/1127061282 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5944157dd0 con 0x7f594406f420 2024-05-26T22:54:49.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.465+0000 7f5949200640 1 --2- 172.21.15.131:0/1127061282 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f594406f420 0x7f594414a9f0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f593800c430 tx=0x7f593800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:49.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.465+0000 7f5940c00640 1 -- 172.21.15.131:0/1127061282 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5938011790 con 0x7f594406f420 2024-05-26T22:54:49.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.466+0000 7f5940c00640 1 -- 172.21.15.131:0/1127061282 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5938017070 con 0x7f594406f420 2024-05-26T22:54:49.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.466+0000 7f5940c00640 1 -- 172.21.15.131:0/1127061282 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5938011bb0 con 0x7f594406f420 2024-05-26T22:54:49.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.466+0000 7f594bc00640 1 -- 172.21.15.131:0/1127061282 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f594415ca50 con 0x7f594406f420 2024-05-26T22:54:49.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.466+0000 7f594bc00640 1 -- 172.21.15.131:0/1127061282 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f594409bf80 con 0x7f594406f420 2024-05-26T22:54:49.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.467+0000 7f594bc00640 1 -- 172.21.15.131:0/1127061282 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5944074e90 con 0x7f594406f420 2024-05-26T22:54:49.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.468+0000 7f5940c00640 1 -- 172.21.15.131:0/1127061282 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f593801c460 con 0x7f594406f420 2024-05-26T22:54:49.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.469+0000 7f5940c00640 1 --2- 172.21.15.131:0/1127061282 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f592406be30 0x7f592406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:49.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.469+0000 7f5940c00640 1 -- 172.21.15.131:0/1127061282 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5115+0+0 (secure 0 0 0) 0x7f593808ab70 con 0x7f594406f420 2024-05-26T22:54:49.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.469+0000 7f5943e00640 1 --2- 172.21.15.131:0/1127061282 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f592406be30 0x7f592406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:49.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.470+0000 7f5943e00640 1 --2- 172.21.15.131:0/1127061282 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f592406be30 0x7f592406e2f0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f5930000f60 tx=0x7f5930007a80 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:49.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.474+0000 7f5940c00640 1 -- 172.21.15.131:0/1127061282 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f59380596d0 con 0x7f594406f420 2024-05-26T22:54:49.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.674+0000 7f594bc00640 1 -- 172.21.15.131:0/1127061282 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f59440695c0 con 0x7f594406f420 2024-05-26T22:54:49.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.675+0000 7f5940c00640 1 -- 172.21.15.131:0/1127061282 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) ==== 93+0+10 (secure 0 0 0) 0x7f5938058fd0 con 0x7f594406f420 2024-05-26T22:54:49.675 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2024-05-26T22:54:49.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.677+0000 7f594bc00640 1 -- 172.21.15.131:0/1127061282 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f592406be30 msgr2=0x7f592406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:49.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.677+0000 7f594bc00640 1 --2- 172.21.15.131:0/1127061282 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f592406be30 0x7f592406e2f0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f5930000f60 tx=0x7f5930007a80 comp rx=0 tx=0).stop 2024-05-26T22:54:49.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.677+0000 7f594bc00640 1 -- 172.21.15.131:0/1127061282 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f594406f420 msgr2=0x7f594414a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:49.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.677+0000 7f594bc00640 1 --2- 172.21.15.131:0/1127061282 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f594406f420 0x7f594414a9f0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f593800c430 tx=0x7f593800c8f0 comp rx=0 tx=0).stop 2024-05-26T22:54:49.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.677+0000 7f594bc00640 1 -- 172.21.15.131:0/1127061282 shutdown_connections 2024-05-26T22:54:49.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.677+0000 7f594bc00640 1 --2- 172.21.15.131:0/1127061282 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f59440b5450 0x7f594409b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:49.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.677+0000 7f594bc00640 1 --2- 172.21.15.131:0/1127061282 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f592406be30 0x7f592406e2f0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:49.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.677+0000 7f594bc00640 1 --2- 172.21.15.131:0/1127061282 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f59440d0460 0x7f594409b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:49.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.677+0000 7f594bc00640 1 --2- 172.21.15.131:0/1127061282 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f594406f420 0x7f594414a9f0 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:49.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.677+0000 7f594bc00640 1 -- 172.21.15.131:0/1127061282 >> 172.21.15.131:0/1127061282 conn(0x7f594406bc90 msgr2=0x7f59440af390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:49.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.678+0000 7f594bc00640 1 -- 172.21.15.131:0/1127061282 shutdown_connections 2024-05-26T22:54:49.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.678+0000 7f594bc00640 1 -- 172.21.15.131:0/1127061282 wait complete. 2024-05-26T22:54:49.696 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_exports.TestEphemeralPins.test_ephemeral_pin_dist_override_before' 2024-05-26T22:54:49.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.901+0000 7fe11fe00640 1 Processor -- start 2024-05-26T22:54:49.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.901+0000 7fe11fe00640 1 -- start start 2024-05-26T22:54:49.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.902+0000 7fe11fe00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fe120144800 0x7fe12015fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:49.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.902+0000 7fe11fe00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fe1200ad870 0x7fe1200add00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:49.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.902+0000 7fe11fe00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe12015f530 con 0x7fe12015e9f0 2024-05-26T22:54:49.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.902+0000 7fe11fe00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fe12015ffd0 con 0x7fe120144800 2024-05-26T22:54:49.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.902+0000 7fe11fe00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fe12015f7b0 con 0x7fe1200ad870 2024-05-26T22:54:49.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.903+0000 7fe11ea00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fe120144800 0x7fe12015fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:49.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.903+0000 7fe11ea00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fe120144800 0x7fe12015fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59298/0 (socket says 172.21.15.131:59298) 2024-05-26T22:54:49.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.903+0000 7fe11ea00640 1 -- 172.21.15.131:0/2941804599 learned_addr learned my addr 172.21.15.131:0/2941804599 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:54:49.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.903+0000 7fe117e00640 1 --2- 172.21.15.131:0/2941804599 >> v2:172.21.15.187:3300/0 conn(0x7fe1200ad870 0x7fe1200add00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:49.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.903+0000 7fe11ea00640 1 -- 172.21.15.131:0/2941804599 >> v2:172.21.15.187:3300/0 conn(0x7fe1200ad870 msgr2=0x7fe1200add00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:49.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.903+0000 7fe11ea00640 1 --2- 172.21.15.131:0/2941804599 >> v2:172.21.15.187:3300/0 conn(0x7fe1200ad870 0x7fe1200add00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:49.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.903+0000 7fe11ea00640 1 -- 172.21.15.131:0/2941804599 >> v1:172.21.15.187:6789/0 conn(0x7fe12015e9f0 legacy=0x7fe12015ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:49.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.903+0000 7fe11ea00640 1 -- 172.21.15.131:0/2941804599 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1200cde40 con 0x7fe120144800 2024-05-26T22:54:49.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.904+0000 7fe11ea00640 1 --2- 172.21.15.131:0/2941804599 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe120144800 0x7fe12015fa90 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7fe1200710f0 tx=0x7fe11002a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81704c6c6193ab88 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:49.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.905+0000 7fe11e000640 1 -- 172.21.15.131:0/2941804599 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe110037070 con 0x7fe120144800 2024-05-26T22:54:49.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.905+0000 7fe11e000640 1 -- 172.21.15.131:0/2941804599 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe110032440 con 0x7fe120144800 2024-05-26T22:54:49.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.905+0000 7fe11e000640 1 -- 172.21.15.131:0/2941804599 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe110032810 con 0x7fe120144800 2024-05-26T22:54:49.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.905+0000 7fe11fe00640 1 -- 172.21.15.131:0/2941804599 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe120144800 msgr2=0x7fe12015fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:49.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.905+0000 7fe11fe00640 1 --2- 172.21.15.131:0/2941804599 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe120144800 0x7fe12015fa90 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7fe1200710f0 tx=0x7fe11002a2c0 comp rx=0 tx=0).stop 2024-05-26T22:54:49.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.905+0000 7fe11fe00640 1 -- 172.21.15.131:0/2941804599 shutdown_connections 2024-05-26T22:54:49.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.906+0000 7fe11fe00640 1 --2- 172.21.15.131:0/2941804599 >> v2:172.21.15.187:3300/0 conn(0x7fe1200ad870 0x7fe1200add00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:49.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.906+0000 7fe11fe00640 1 --2- 172.21.15.131:0/2941804599 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe120144800 0x7fe12015fa90 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:49.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.906+0000 7fe11fe00640 1 -- 172.21.15.131:0/2941804599 >> 172.21.15.131:0/2941804599 conn(0x7fe1200cff40 msgr2=0x7fe1200d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:49.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.906+0000 7fe11fe00640 1 -- 172.21.15.131:0/2941804599 shutdown_connections 2024-05-26T22:54:49.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.906+0000 7fe11fe00640 1 -- 172.21.15.131:0/2941804599 wait complete. 2024-05-26T22:54:49.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.907+0000 7fe11fe00640 1 Processor -- start 2024-05-26T22:54:49.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.907+0000 7fe11fe00640 1 -- start start 2024-05-26T22:54:49.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.908+0000 7fe11fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe1200ad870 0x7fe1201f7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:49.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.908+0000 7fe11fe00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe120144800 0x7fe1201f4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:49.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.908+0000 7fe11fe00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe12015e9f0 0x7fe120158810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:49.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.908+0000 7fe11fe00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe120144be0 con 0x7fe1200ad870 2024-05-26T22:54:49.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.908+0000 7fe11fe00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe1200ce400 con 0x7fe120144800 2024-05-26T22:54:49.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.908+0000 7fe11fe00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe1200ce920 con 0x7fe12015e9f0 2024-05-26T22:54:49.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.909+0000 7fe11ea00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe1200ad870 0x7fe1201f7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:49.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.909+0000 7fe11ea00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe1200ad870 0x7fe1201f7880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59302/0 (socket says 172.21.15.131:59302) 2024-05-26T22:54:49.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.909+0000 7fe11ea00640 1 -- 172.21.15.131:0/1707167170 learned_addr learned my addr 172.21.15.131:0/1707167170 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:54:49.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.909+0000 7fe11ea00640 1 -- 172.21.15.131:0/1707167170 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe12015e9f0 msgr2=0x7fe120158810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:49.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.909+0000 7fe11ea00640 1 --2- 172.21.15.131:0/1707167170 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe12015e9f0 0x7fe120158810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:49.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.909+0000 7fe11ea00640 1 -- 172.21.15.131:0/1707167170 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe120144800 msgr2=0x7fe1201f4d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:49.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.909+0000 7fe11ea00640 1 --2- 172.21.15.131:0/1707167170 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe120144800 0x7fe1201f4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:49.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.909+0000 7fe11ea00640 1 -- 172.21.15.131:0/1707167170 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1201f7f40 con 0x7fe1200ad870 2024-05-26T22:54:49.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.909+0000 7fe11ea00640 1 --2- 172.21.15.131:0/1707167170 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe1200ad870 0x7fe1201f7880 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7fe110000c00 tx=0x7fe1100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:49.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.910+0000 7fe117400640 1 -- 172.21.15.131:0/1707167170 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe110041020 con 0x7fe1200ad870 2024-05-26T22:54:49.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.910+0000 7fe117400640 1 -- 172.21.15.131:0/1707167170 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1100306c0 con 0x7fe1200ad870 2024-05-26T22:54:49.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.910+0000 7fe117400640 1 -- 172.21.15.131:0/1707167170 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe110037040 con 0x7fe1200ad870 2024-05-26T22:54:49.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.910+0000 7fe11fe00640 1 -- 172.21.15.131:0/1707167170 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1201f8220 con 0x7fe1200ad870 2024-05-26T22:54:49.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.910+0000 7fe11fe00640 1 -- 172.21.15.131:0/1707167170 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe12013c050 con 0x7fe1200ad870 2024-05-26T22:54:49.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.911+0000 7fe11fe00640 1 -- 172.21.15.131:0/1707167170 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1201f8740 con 0x7fe1200ad870 2024-05-26T22:54:49.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.912+0000 7fe117400640 1 -- 172.21.15.131:0/1707167170 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe110044020 con 0x7fe1200ad870 2024-05-26T22:54:49.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.913+0000 7fe117400640 1 --2- 172.21.15.131:0/1707167170 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fe0fc06be30 0x7fe0fc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:49.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.913+0000 7fe117400640 1 -- 172.21.15.131:0/1707167170 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5115+0+0 (secure 0 0 0) 0x7fe1100ab340 con 0x7fe1200ad870 2024-05-26T22:54:49.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.913+0000 7fe117e00640 1 --2- 172.21.15.131:0/1707167170 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fe0fc06be30 0x7fe0fc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:49.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.914+0000 7fe117e00640 1 --2- 172.21.15.131:0/1707167170 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fe0fc06be30 0x7fe0fc06e2f0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fe1200b5cd0 tx=0x7fe108008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:49.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:49.918+0000 7fe117400640 1 -- 172.21.15.131:0/1707167170 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fe11004b030 con 0x7fe1200ad870 2024-05-26T22:54:50.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:50.187+0000 7fe11fe00640 1 -- 172.21.15.131:0/1707167170 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_exports.TestEphemeralPins.test_ephemeral_pin_dist_override_before"]} v 0) -- 0x7fe1200694e0 con 0x7fe1200ad870 2024-05-26T22:54:50.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:50.802+0000 7fe117400640 1 -- 172.21.15.131:0/1707167170 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_exports.TestEphemeralPins.test_ephemeral_pin_dist_override_before"]}]=0 v330) ==== 165+0+0 (secure 0 0 0) 0x7fe12015f530 con 0x7fe1200ad870 2024-05-26T22:54:50.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:50.805+0000 7fe11fe00640 1 -- 172.21.15.131:0/1707167170 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fe0fc06be30 msgr2=0x7fe0fc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:50.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:50.805+0000 7fe11fe00640 1 --2- 172.21.15.131:0/1707167170 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fe0fc06be30 0x7fe0fc06e2f0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fe1200b5cd0 tx=0x7fe108008040 comp rx=0 tx=0).stop 2024-05-26T22:54:50.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:50.805+0000 7fe11fe00640 1 -- 172.21.15.131:0/1707167170 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe1200ad870 msgr2=0x7fe1201f7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:50.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:50.805+0000 7fe11fe00640 1 --2- 172.21.15.131:0/1707167170 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe1200ad870 0x7fe1201f7880 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7fe110000c00 tx=0x7fe1100046c0 comp rx=0 tx=0).stop 2024-05-26T22:54:50.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:50.806+0000 7fe11fe00640 1 -- 172.21.15.131:0/1707167170 shutdown_connections 2024-05-26T22:54:50.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:50.806+0000 7fe11fe00640 1 --2- 172.21.15.131:0/1707167170 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe12015e9f0 0x7fe120158810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:50.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:50.806+0000 7fe11fe00640 1 --2- 172.21.15.131:0/1707167170 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fe0fc06be30 0x7fe0fc06e2f0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:50.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:50.806+0000 7fe11fe00640 1 --2- 172.21.15.131:0/1707167170 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe120144800 0x7fe1201f4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:50.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:50.806+0000 7fe11fe00640 1 --2- 172.21.15.131:0/1707167170 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe1200ad870 0x7fe1201f7880 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:50.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:50.806+0000 7fe11fe00640 1 -- 172.21.15.131:0/1707167170 >> 172.21.15.131:0/1707167170 conn(0x7fe1200cff40 msgr2=0x7fe1200d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:50.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:50.806+0000 7fe11fe00640 1 -- 172.21.15.131:0/1707167170 shutdown_connections 2024-05-26T22:54:50.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:50.806+0000 7fe11fe00640 1 -- 172.21.15.131:0/1707167170 wait complete. 2024-05-26T22:54:50.826 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-26T22:54:51.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.037+0000 7f9333c00640 1 Processor -- start 2024-05-26T22:54:51.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.037+0000 7f9333c00640 1 -- start start 2024-05-26T22:54:51.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.038+0000 7f9333c00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f932c15c2f0 0x7f932c0afef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:51.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.039+0000 7f9333c00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f932c0ce380 0x7f932c160380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:51.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.039+0000 7f9333c00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f932c06ec10 con 0x7f932c15fef0 2024-05-26T22:54:51.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.039+0000 7f9333c00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f932c0b0430 con 0x7f932c0ce380 2024-05-26T22:54:51.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.039+0000 7f9333c00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f932c06eef0 con 0x7f932c15c2f0 2024-05-26T22:54:51.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.039+0000 7f9331c00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f932c0ce380 0x7f932c160380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:51.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.039+0000 7f9331c00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f932c0ce380 0x7f932c160380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59308/0 (socket says 172.21.15.131:59308) 2024-05-26T22:54:51.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.039+0000 7f9331c00640 1 -- 172.21.15.131:0/2675503596 learned_addr learned my addr 172.21.15.131:0/2675503596 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:54:51.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.039+0000 7f9331200640 1 --2- 172.21.15.131:0/2675503596 >> v2:172.21.15.187:3300/0 conn(0x7f932c15c2f0 0x7f932c0afef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:51.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.040+0000 7f9331c00640 1 -- 172.21.15.131:0/2675503596 >> v2:172.21.15.187:3300/0 conn(0x7f932c15c2f0 msgr2=0x7f932c0afef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:51.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.040+0000 7f9331c00640 1 --2- 172.21.15.131:0/2675503596 >> v2:172.21.15.187:3300/0 conn(0x7f932c15c2f0 0x7f932c0afef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:51.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.040+0000 7f9331c00640 1 -- 172.21.15.131:0/2675503596 >> v1:172.21.15.187:6789/0 conn(0x7f932c15fef0 legacy=0x7f932c06b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:51.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.040+0000 7f9331c00640 1 -- 172.21.15.131:0/2675503596 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f932c06f1d0 con 0x7f932c0ce380 2024-05-26T22:54:51.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.041+0000 7f9331c00640 1 --2- 172.21.15.131:0/2675503596 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f932c0ce380 0x7f932c160380 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f932c0674d0 tx=0x7f932402a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec38efe36b3a86b4 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:51.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.041+0000 7f932b400640 1 -- 172.21.15.131:0/2675503596 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9324036070 con 0x7f932c0ce380 2024-05-26T22:54:51.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.041+0000 7f932b400640 1 -- 172.21.15.131:0/2675503596 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9324031600 con 0x7f932c0ce380 2024-05-26T22:54:51.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.041+0000 7f932b400640 1 -- 172.21.15.131:0/2675503596 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93240319d0 con 0x7f932c0ce380 2024-05-26T22:54:51.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.042+0000 7f9333c00640 1 -- 172.21.15.131:0/2675503596 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f932c0ce380 msgr2=0x7f932c160380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:51.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.042+0000 7f9333c00640 1 --2- 172.21.15.131:0/2675503596 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f932c0ce380 0x7f932c160380 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f932c0674d0 tx=0x7f932402a6c0 comp rx=0 tx=0).stop 2024-05-26T22:54:51.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.042+0000 7f9333c00640 1 -- 172.21.15.131:0/2675503596 shutdown_connections 2024-05-26T22:54:51.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.042+0000 7f9333c00640 1 --2- 172.21.15.131:0/2675503596 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f932c0ce380 0x7f932c160380 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:51.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.042+0000 7f9333c00640 1 --2- 172.21.15.131:0/2675503596 >> v2:172.21.15.187:3300/0 conn(0x7f932c15c2f0 0x7f932c0afef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:51.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.042+0000 7f9333c00640 1 -- 172.21.15.131:0/2675503596 >> 172.21.15.131:0/2675503596 conn(0x7f932c0b27c0 msgr2=0x7f932c0b2bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:51.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.042+0000 7f9333c00640 1 -- 172.21.15.131:0/2675503596 shutdown_connections 2024-05-26T22:54:51.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.042+0000 7f9333c00640 1 -- 172.21.15.131:0/2675503596 wait complete. 2024-05-26T22:54:51.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.043+0000 7f9333c00640 1 Processor -- start 2024-05-26T22:54:51.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.044+0000 7f9333c00640 1 -- start start 2024-05-26T22:54:51.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.044+0000 7f9333c00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f932c0ce380 0x7f932c1463e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:51.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.044+0000 7f9333c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f932c15c2f0 0x7f932c1f49b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:51.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.045+0000 7f9333c00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f932c15fef0 0x7f932c15ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:51.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.045+0000 7f9333c00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f932c15c6d0 con 0x7f932c15c2f0 2024-05-26T22:54:51.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.045+0000 7f9333c00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f932c06e8e0 con 0x7f932c15fef0 2024-05-26T22:54:51.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.045+0000 7f9333c00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f932c160b40 con 0x7f932c0ce380 2024-05-26T22:54:51.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.045+0000 7f932be00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f932c15c2f0 0x7f932c1f49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:51.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.045+0000 7f932be00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f932c15c2f0 0x7f932c1f49b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59322/0 (socket says 172.21.15.131:59322) 2024-05-26T22:54:51.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.045+0000 7f932be00640 1 -- 172.21.15.131:0/238532838 learned_addr learned my addr 172.21.15.131:0/238532838 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:54:51.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.045+0000 7f9331c00640 1 --2- 172.21.15.131:0/238532838 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f932c15fef0 0x7f932c15ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:51.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.045+0000 7f9331200640 1 --2- 172.21.15.131:0/238532838 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f932c0ce380 0x7f932c1463e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:51.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.045+0000 7f932be00640 1 -- 172.21.15.131:0/238532838 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f932c0ce380 msgr2=0x7f932c1463e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:51.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.045+0000 7f932be00640 1 --2- 172.21.15.131:0/238532838 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f932c0ce380 0x7f932c1463e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:51.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.045+0000 7f932be00640 1 -- 172.21.15.131:0/238532838 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f932c15fef0 msgr2=0x7f932c15ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:51.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.045+0000 7f932be00640 1 --2- 172.21.15.131:0/238532838 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f932c15fef0 0x7f932c15ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:51.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.045+0000 7f932be00640 1 -- 172.21.15.131:0/238532838 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f932c159cc0 con 0x7f932c15c2f0 2024-05-26T22:54:51.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.045+0000 7f9331c00640 1 --2- 172.21.15.131:0/238532838 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f932c15fef0 0x7f932c15ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:54:51.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.046+0000 7f932be00640 1 --2- 172.21.15.131:0/238532838 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f932c15c2f0 0x7f932c1f49b0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f931800b420 tx=0x7f931800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:51.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.046+0000 7f9328c00640 1 -- 172.21.15.131:0/238532838 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9318011830 con 0x7f932c15c2f0 2024-05-26T22:54:51.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.046+0000 7f9328c00640 1 -- 172.21.15.131:0/238532838 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9318016070 con 0x7f932c15c2f0 2024-05-26T22:54:51.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.046+0000 7f9328c00640 1 -- 172.21.15.131:0/238532838 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9318011c10 con 0x7f932c15c2f0 2024-05-26T22:54:51.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.046+0000 7f9333c00640 1 -- 172.21.15.131:0/238532838 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f932c159fa0 con 0x7f932c15c2f0 2024-05-26T22:54:51.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.046+0000 7f9333c00640 1 -- 172.21.15.131:0/238532838 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f932c15a560 con 0x7f932c15c2f0 2024-05-26T22:54:51.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.047+0000 7f9333c00640 1 -- 172.21.15.131:0/238532838 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f932c0695c0 con 0x7f932c15c2f0 2024-05-26T22:54:51.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.048+0000 7f9328c00640 1 -- 172.21.15.131:0/238532838 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f931801b460 con 0x7f932c15c2f0 2024-05-26T22:54:51.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.049+0000 7f9328c00640 1 --2- 172.21.15.131:0/238532838 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f930c06be30 0x7f930c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:51.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.049+0000 7f9328c00640 1 -- 172.21.15.131:0/238532838 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5115+0+0 (secure 0 0 0) 0x7f931808a790 con 0x7f932c15c2f0 2024-05-26T22:54:51.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.049+0000 7f9331200640 1 --2- 172.21.15.131:0/238532838 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f930c06be30 0x7f930c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:51.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.050+0000 7f9331200640 1 --2- 172.21.15.131:0/238532838 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f930c06be30 0x7f930c06e2f0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f931c006fd0 tx=0x7f931c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:51.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.053+0000 7f9328c00640 1 -- 172.21.15.131:0/238532838 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f9318020020 con 0x7f932c15c2f0 2024-05-26T22:54:51.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.256+0000 7f9333c00640 1 -- 172.21.15.131:0/238532838 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f932c06c560 con 0x7f932c15c2f0 2024-05-26T22:54:51.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.261+0000 7f9328c00640 1 -- 172.21.15.131:0/238532838 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v51)=0 v51) ==== 122+0+0 (secure 0 0 0) 0x7f93180592f0 con 0x7f932c15c2f0 2024-05-26T22:54:51.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.264+0000 7f9333c00640 1 -- 172.21.15.131:0/238532838 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f930c06be30 msgr2=0x7f930c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:51.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.264+0000 7f9333c00640 1 --2- 172.21.15.131:0/238532838 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f930c06be30 0x7f930c06e2f0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f931c006fd0 tx=0x7f931c008040 comp rx=0 tx=0).stop 2024-05-26T22:54:51.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.264+0000 7f9333c00640 1 -- 172.21.15.131:0/238532838 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f932c15c2f0 msgr2=0x7f932c1f49b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:51.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.264+0000 7f9333c00640 1 --2- 172.21.15.131:0/238532838 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f932c15c2f0 0x7f932c1f49b0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f931800b420 tx=0x7f931800b8e0 comp rx=0 tx=0).stop 2024-05-26T22:54:51.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.264+0000 7f9333c00640 1 -- 172.21.15.131:0/238532838 shutdown_connections 2024-05-26T22:54:51.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.264+0000 7f9333c00640 1 --2- 172.21.15.131:0/238532838 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f930c06be30 0x7f930c06e2f0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:51.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.264+0000 7f9333c00640 1 --2- 172.21.15.131:0/238532838 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f932c15fef0 0x7f932c15ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:51.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.264+0000 7f9333c00640 1 --2- 172.21.15.131:0/238532838 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f932c15c2f0 0x7f932c1f49b0 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:51.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.264+0000 7f9333c00640 1 --2- 172.21.15.131:0/238532838 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f932c0ce380 0x7f932c1463e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:51.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.264+0000 7f9333c00640 1 -- 172.21.15.131:0/238532838 >> 172.21.15.131:0/238532838 conn(0x7f932c0b27c0 msgr2=0x7f932c0ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:51.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.265+0000 7f9333c00640 1 -- 172.21.15.131:0/238532838 shutdown_connections 2024-05-26T22:54:51.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.265+0000 7f9333c00640 1 -- 172.21.15.131:0/238532838 wait complete. 2024-05-26T22:54:51.285 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-05-26T22:54:51.285 DEBUG:teuthology.orchestra.run.smithi131:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-26T22:54:51.387 DEBUG:teuthology.orchestra.run.smithi187:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-26T22:54:51.506 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-05-26T22:54:51.506 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-26T22:54:51.506 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-26T22:54:51.564 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-26T22:54:51.564 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-05-26T22:54:51.564 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-26T22:54:51.621 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-26T22:54:51.621 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-26T22:54:51.622 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-26T22:54:51.680 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-26T22:54:51.680 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-05-26T22:54:51.680 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-26T22:54:51.738 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-26T22:54:51.738 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-26T22:54:51.738 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-26T22:54:51.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.947+0000 7fbb17200640 1 Processor -- start 2024-05-26T22:54:51.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.948+0000 7fbb17200640 1 -- start start 2024-05-26T22:54:51.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.949+0000 7fbb17200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbb1015c3d0 0x7fbb1006ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:51.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.949+0000 7fbb17200640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fbb1006f490 0x7fbb100ade40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:51.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.949+0000 7fbb17200640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb100722a0 con 0x7fbb10155840 2024-05-26T22:54:51.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.949+0000 7fbb17200640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fbb10072040 con 0x7fbb1006f490 2024-05-26T22:54:51.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.949+0000 7fbb17200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fbb10072580 con 0x7fbb1015c3d0 2024-05-26T22:54:51.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.950+0000 7fbb15200640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fbb1006f490 0x7fbb100ade40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:51.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.950+0000 7fbb07400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbb1015c3d0 0x7fbb1006ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:51.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.950+0000 7fbb15200640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fbb1006f490 0x7fbb100ade40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.131:55628/0 (socket says 172.21.15.131:55628) 2024-05-26T22:54:51.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.950+0000 7fbb15200640 1 -- 172.21.15.131:0/3704346271 learned_addr learned my addr 172.21.15.131:0/3704346271 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:54:51.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.951+0000 7fbb15200640 1 -- 172.21.15.131:0/3704346271 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbb1015c3d0 msgr2=0x7fbb1006ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:51.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.951+0000 7fbb15200640 1 --2- 172.21.15.131:0/3704346271 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbb1015c3d0 0x7fbb1006ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:51.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.951+0000 7fbb15200640 1 -- 172.21.15.131:0/3704346271 >> v1:172.21.15.187:6789/0 conn(0x7fbb10155840 legacy=0x7fbb10071930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:51.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.951+0000 7fbb15200640 1 -- 172.21.15.131:0/3704346271 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb10072860 con 0x7fbb1006f490 2024-05-26T22:54:51.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.952+0000 7fbb15200640 1 --2- 172.21.15.131:0/3704346271 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb1006f490 0x7fbb100ade40 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fbb08009870 tx=0x7fbb0802a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9ce553c2b46194e server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:51.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.952+0000 7fbb0f400640 1 -- 172.21.15.131:0/3704346271 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb08037070 con 0x7fbb1006f490 2024-05-26T22:54:51.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.952+0000 7fbb0f400640 1 -- 172.21.15.131:0/3704346271 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb08032440 con 0x7fbb1006f490 2024-05-26T22:54:51.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.952+0000 7fbb0f400640 1 -- 172.21.15.131:0/3704346271 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb08032810 con 0x7fbb1006f490 2024-05-26T22:54:51.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.953+0000 7fbb17200640 1 -- 172.21.15.131:0/3704346271 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb1006f490 msgr2=0x7fbb100ade40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:51.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.953+0000 7fbb17200640 1 --2- 172.21.15.131:0/3704346271 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb1006f490 0x7fbb100ade40 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fbb08009870 tx=0x7fbb0802a270 comp rx=0 tx=0).stop 2024-05-26T22:54:51.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.953+0000 7fbb17200640 1 -- 172.21.15.131:0/3704346271 shutdown_connections 2024-05-26T22:54:51.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.953+0000 7fbb17200640 1 --2- 172.21.15.131:0/3704346271 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb1006f490 0x7fbb100ade40 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:51.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.953+0000 7fbb17200640 1 --2- 172.21.15.131:0/3704346271 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbb1015c3d0 0x7fbb1006ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:51.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.953+0000 7fbb17200640 1 -- 172.21.15.131:0/3704346271 >> 172.21.15.131:0/3704346271 conn(0x7fbb100b1e90 msgr2=0x7fbb100b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:51.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.953+0000 7fbb17200640 1 -- 172.21.15.131:0/3704346271 shutdown_connections 2024-05-26T22:54:51.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.953+0000 7fbb17200640 1 -- 172.21.15.131:0/3704346271 wait complete. 2024-05-26T22:54:51.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.955+0000 7fbb17200640 1 Processor -- start 2024-05-26T22:54:51.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.956+0000 7fbb17200640 1 -- start start 2024-05-26T22:54:51.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.957+0000 7fbb17200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb1006f490 0x7fbb101f1e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:51.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.957+0000 7fbb17200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbb10155840 0x7fbb10149350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:51.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.957+0000 7fbb17200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb1015c3d0 0x7fbb1015a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:51.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.957+0000 7fbb17200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fbb100ae380 con 0x7fbb1006f490 2024-05-26T22:54:51.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.957+0000 7fbb17200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fbb10072e20 con 0x7fbb1015c3d0 2024-05-26T22:54:51.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.957+0000 7fbb17200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fbb100ae500 con 0x7fbb10155840 2024-05-26T22:54:51.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.957+0000 7fbb0fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb1006f490 0x7fbb101f1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:51.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.957+0000 7fbb0fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb1006f490 0x7fbb101f1e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59324/0 (socket says 172.21.15.131:59324) 2024-05-26T22:54:51.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.957+0000 7fbb0fe00640 1 -- 172.21.15.131:0/1237127316 learned_addr learned my addr 172.21.15.131:0/1237127316 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:54:51.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.958+0000 7fbb07400640 1 --2- 172.21.15.131:0/1237127316 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbb10155840 0x7fbb10149350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:51.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.958+0000 7fbb15200640 1 --2- 172.21.15.131:0/1237127316 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb1015c3d0 0x7fbb1015a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:51.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.958+0000 7fbb0fe00640 1 -- 172.21.15.131:0/1237127316 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbb10155840 msgr2=0x7fbb10149350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:51.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.958+0000 7fbb0fe00640 1 --2- 172.21.15.131:0/1237127316 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbb10155840 0x7fbb10149350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:51.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.958+0000 7fbb0fe00640 1 -- 172.21.15.131:0/1237127316 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb1015c3d0 msgr2=0x7fbb1015a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:51.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.958+0000 7fbb0fe00640 1 --2- 172.21.15.131:0/1237127316 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb1015c3d0 0x7fbb1015a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:51.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.958+0000 7fbb0fe00640 1 -- 172.21.15.131:0/1237127316 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb100cdd90 con 0x7fbb1006f490 2024-05-26T22:54:51.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.958+0000 7fbb15200640 1 --2- 172.21.15.131:0/1237127316 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb1015c3d0 0x7fbb1015a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:54:51.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.958+0000 7fbb0fe00640 1 --2- 172.21.15.131:0/1237127316 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb1006f490 0x7fbb101f1e30 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7fbb0000c8a0 tx=0x7fbb0000cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:51.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.958+0000 7fbb0cc00640 1 -- 172.21.15.131:0/1237127316 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb00003b70 con 0x7fbb1006f490 2024-05-26T22:54:51.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.958+0000 7fbb0cc00640 1 -- 172.21.15.131:0/1237127316 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb00003d30 con 0x7fbb1006f490 2024-05-26T22:54:51.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.958+0000 7fbb0cc00640 1 -- 172.21.15.131:0/1237127316 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb0000aa90 con 0x7fbb1006f490 2024-05-26T22:54:51.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.958+0000 7fbb17200640 1 -- 172.21.15.131:0/1237127316 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb100ce070 con 0x7fbb1006f490 2024-05-26T22:54:51.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.959+0000 7fbb17200640 1 -- 172.21.15.131:0/1237127316 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb100ce630 con 0x7fbb1006f490 2024-05-26T22:54:51.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.960+0000 7fbb17200640 1 -- 172.21.15.131:0/1237127316 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb100694e0 con 0x7fbb1006f490 2024-05-26T22:54:51.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.961+0000 7fbb0cc00640 1 -- 172.21.15.131:0/1237127316 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbb0000ac30 con 0x7fbb1006f490 2024-05-26T22:54:51.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.961+0000 7fbb0cc00640 1 --2- 172.21.15.131:0/1237127316 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fbaec06be30 0x7fbaec06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:51.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.962+0000 7fbb0cc00640 1 -- 172.21.15.131:0/1237127316 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5115+0+0 (secure 0 0 0) 0x7fbb0008ab90 con 0x7fbb1006f490 2024-05-26T22:54:51.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.962+0000 7fbb07400640 1 --2- 172.21.15.131:0/1237127316 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fbaec06be30 0x7fbaec06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:51.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.963+0000 7fbb07400640 1 --2- 172.21.15.131:0/1237127316 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fbaec06be30 0x7fbaec06e2f0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fbb100d0ea0 tx=0x7fbaf8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:51.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:51.966+0000 7fbb0cc00640 1 -- 172.21.15.131:0/1237127316 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fbb00059790 con 0x7fbb1006f490 2024-05-26T22:54:52.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:52.203+0000 7fbb17200640 1 -- 172.21.15.131:0/1237127316 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbb100adee0 con 0x7fbb1006f490 2024-05-26T22:54:52.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:52.205+0000 7fbb0cc00640 1 -- 172.21.15.131:0/1237127316 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) ==== 94+0+3878 (secure 0 0 0) 0x7fbb00059090 con 0x7fbb1006f490 2024-05-26T22:54:52.205 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:54:52.205 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":87,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5192,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6834/1741236886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1741236886},{"type":"v1","addr":"172.21.15.187:6834","nonce":1741236886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":87},{"gid":5533,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/993930297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6835","nonce":993930297},{"type":"v1","addr":"172.21.15.131:6837","nonce":993930297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":73},{"gid":7755,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6836/1458732866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1458732866},{"type":"v1","addr":"172.21.15.131:6836","nonce":1458732866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":87},{"gid":7767,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/2594767872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6833","nonce":2594767872},{"type":"v1","addr":"172.21.15.187:6835","nonce":2594767872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":87}],"filesystems":[]} 2024-05-26T22:54:52.205 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 87 2024-05-26T22:54:52.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:52.207+0000 7fbb17200640 1 -- 172.21.15.131:0/1237127316 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fbaec06be30 msgr2=0x7fbaec06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:52.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:52.208+0000 7fbb17200640 1 --2- 172.21.15.131:0/1237127316 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fbaec06be30 0x7fbaec06e2f0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fbb100d0ea0 tx=0x7fbaf8008040 comp rx=0 tx=0).stop 2024-05-26T22:54:52.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:52.208+0000 7fbb17200640 1 -- 172.21.15.131:0/1237127316 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb1006f490 msgr2=0x7fbb101f1e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:52.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:52.208+0000 7fbb17200640 1 --2- 172.21.15.131:0/1237127316 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb1006f490 0x7fbb101f1e30 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7fbb0000c8a0 tx=0x7fbb0000cd60 comp rx=0 tx=0).stop 2024-05-26T22:54:52.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:52.208+0000 7fbb17200640 1 -- 172.21.15.131:0/1237127316 shutdown_connections 2024-05-26T22:54:52.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:52.208+0000 7fbb17200640 1 --2- 172.21.15.131:0/1237127316 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbb10155840 0x7fbb10149350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:52.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:52.208+0000 7fbb17200640 1 --2- 172.21.15.131:0/1237127316 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fbaec06be30 0x7fbaec06e2f0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:52.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:52.208+0000 7fbb17200640 1 --2- 172.21.15.131:0/1237127316 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb1015c3d0 0x7fbb1015a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:52.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:52.208+0000 7fbb17200640 1 --2- 172.21.15.131:0/1237127316 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb1006f490 0x7fbb101f1e30 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:52.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:52.208+0000 7fbb17200640 1 -- 172.21.15.131:0/1237127316 >> 172.21.15.131:0/1237127316 conn(0x7fbb100b1e90 msgr2=0x7fbb1006de20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:52.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:52.208+0000 7fbb17200640 1 -- 172.21.15.131:0/1237127316 shutdown_connections 2024-05-26T22:54:52.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:52.208+0000 7fbb17200640 1 -- 172.21.15.131:0/1237127316 wait complete. 2024-05-26T22:54:52.228 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-26T22:54:52.228 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-26T22:54:52.228 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-26T22:54:52.228 INFO:teuthology.orchestra.run:waiting for 300 2024-05-26T22:54:52.228 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-26T22:54:52.228 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-26T22:54:52.228 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-26T22:54:52.229 INFO:teuthology.orchestra.run:waiting for 300 2024-05-26T22:54:52.229 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-26T22:54:52.229 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-26T22:54:52.229 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-26T22:54:52.229 INFO:teuthology.orchestra.run:waiting for 300 2024-05-26T22:54:52.229 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-26T22:54:52.229 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-26T22:54:52.229 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-26T22:54:52.230 INFO:teuthology.orchestra.run:waiting for 300 2024-05-26T22:54:52.244 INFO:tasks.ceph.mds.a:Stopped 2024-05-26T22:54:52.244 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-26T22:54:52.248 INFO:tasks.ceph.mds.c:Stopped 2024-05-26T22:54:52.248 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-26T22:54:52.252 INFO:tasks.ceph.mds.d:Stopped 2024-05-26T22:54:52.252 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-26T22:54:52.259 INFO:tasks.ceph.mds.b:Stopped 2024-05-26T22:54:52.259 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-26T22:54:52.295 INFO:tasks.ceph.mds.a:Started 2024-05-26T22:54:52.296 DEBUG:teuthology.parallel:result is None 2024-05-26T22:54:52.302 INFO:tasks.ceph.mds.c:Started 2024-05-26T22:54:52.302 DEBUG:teuthology.parallel:result is None 2024-05-26T22:54:52.305 INFO:tasks.ceph.mds.d:Started 2024-05-26T22:54:52.305 DEBUG:teuthology.parallel:result is None 2024-05-26T22:54:52.311 INFO:tasks.ceph.mds.b:Started 2024-05-26T22:54:52.312 DEBUG:teuthology.parallel:result is None 2024-05-26T22:54:52.312 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-26T22:54:52.447 INFO:tasks.ceph.mds.a.smithi131.stdout:starting mds.a at 2024-05-26T22:54:52.448 INFO:tasks.ceph.mds.c.smithi131.stdout:starting mds.c at 2024-05-26T22:54:52.458 INFO:tasks.ceph.mds.d.smithi187.stdout:starting mds.d at 2024-05-26T22:54:52.460 INFO:tasks.ceph.mds.b.smithi187.stdout:starting mds.b at 2024-05-26T22:54:52.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:52.490+0000 7fd62ba00640 1 Processor -- start 2024-05-26T22:54:52.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:52.490+0000 7fd62ba00640 1 -- start start 2024-05-26T22:54:52.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:52.491+0000 7fd62ba00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd6240b5450 0x7fd6240b43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:52.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:52.491+0000 7fd62ba00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fd62406f420 0x7fd6240cfd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:52.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:52.491+0000 7fd62ba00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6240b4900 con 0x7fd6240cf590 2024-05-26T22:54:52.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:52.491+0000 7fd62ba00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fd624160d90 con 0x7fd62406f420 2024-05-26T22:54:52.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:52.491+0000 7fd62ba00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd624160180 con 0x7fd6240b5450 2024-05-26T22:54:52.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:52.492+0000 7fd623e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fd62406f420 0x7fd6240cfd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:52.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:52.492+0000 7fd623e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fd62406f420 0x7fd6240cfd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59362/0 (socket says 172.21.15.131:59362) 2024-05-26T22:54:52.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:52.492+0000 7fd623e00640 1 -- 172.21.15.131:0/2309141293 learned_addr learned my addr 172.21.15.131:0/2309141293 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:54:52.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:52.492+0000 7fd629000640 1 --2- 172.21.15.131:0/2309141293 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd6240b5450 0x7fd6240b43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:52.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:52.492+0000 7fd623e00640 1 -- 172.21.15.131:0/2309141293 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd6240b5450 msgr2=0x7fd6240b43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:52.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:52.492+0000 7fd623e00640 1 --2- 172.21.15.131:0/2309141293 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd6240b5450 0x7fd6240b43c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:52.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:52.492+0000 7fd623e00640 1 -- 172.21.15.131:0/2309141293 >> v1:172.21.15.187:6789/0 conn(0x7fd6240cf590 legacy=0x7fd62406d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:52.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:52.492+0000 7fd623e00640 1 -- 172.21.15.131:0/2309141293 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd624160460 con 0x7fd62406f420 2024-05-26T22:54:52.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:52.493+0000 7fd623e00640 1 --2- 172.21.15.131:0/2309141293 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd62406f420 0x7fd6240cfd50 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7fd610009870 tx=0x7fd61002a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4061840982c24fe2 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:52.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:52.493+0000 7fd623400640 1 -- 172.21.15.131:0/2309141293 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd610037070 con 0x7fd62406f420 2024-05-26T22:54:52.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:52.493+0000 7fd623400640 1 -- 172.21.15.131:0/2309141293 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd610032440 con 0x7fd62406f420 2024-05-26T22:54:52.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:52.493+0000 7fd623400640 1 -- 172.21.15.131:0/2309141293 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd610032810 con 0x7fd62406f420 2024-05-26T22:54:52.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:52.494+0000 7fd62ba00640 1 -- 172.21.15.131:0/2309141293 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd62406f420 msgr2=0x7fd6240cfd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:52.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:52.494+0000 7fd62ba00640 1 --2- 172.21.15.131:0/2309141293 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd62406f420 0x7fd6240cfd50 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7fd610009870 tx=0x7fd61002a250 comp rx=0 tx=0).stop 2024-05-26T22:54:52.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:52.494+0000 7fd62ba00640 1 -- 172.21.15.131:0/2309141293 shutdown_connections 2024-05-26T22:54:52.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:52.494+0000 7fd62ba00640 1 --2- 172.21.15.131:0/2309141293 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd62406f420 0x7fd6240cfd50 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:52.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:52.494+0000 7fd62ba00640 1 --2- 172.21.15.131:0/2309141293 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd6240b5450 0x7fd6240b43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:52.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:52.494+0000 7fd62ba00640 1 -- 172.21.15.131:0/2309141293 >> 172.21.15.131:0/2309141293 conn(0x7fd62406bc90 msgr2=0x7fd6240b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:52.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:52.494+0000 7fd62ba00640 1 -- 172.21.15.131:0/2309141293 shutdown_connections 2024-05-26T22:54:52.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:52.494+0000 7fd62ba00640 1 -- 172.21.15.131:0/2309141293 wait complete. 2024-05-26T22:54:52.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:52.496+0000 7fd62ba00640 1 Processor -- start 2024-05-26T22:54:52.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:52.496+0000 7fd62ba00640 1 -- start start 2024-05-26T22:54:52.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:52.497+0000 7fd62ba00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd62406f420 0x7fd624151eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:52.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:52.497+0000 7fd62ba00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd6240b5450 0x7fd62415ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:52.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:52.497+0000 7fd62ba00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6240cf590 0x7fd6240b1300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:52.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:52.497+0000 7fd62ba00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fd6240b5830 con 0x7fd6240cf590 2024-05-26T22:54:52.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:52.497+0000 7fd62ba00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd624160a20 con 0x7fd6240b5450 2024-05-26T22:54:52.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:52.497+0000 7fd62ba00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd62415fde0 con 0x7fd62406f420 2024-05-26T22:54:52.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:52.497+0000 7fd629a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6240cf590 0x7fd6240b1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:52.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:52.497+0000 7fd629a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6240cf590 0x7fd6240b1300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59378/0 (socket says 172.21.15.131:59378) 2024-05-26T22:54:52.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:52.497+0000 7fd629a00640 1 -- 172.21.15.131:0/3010521626 learned_addr learned my addr 172.21.15.131:0/3010521626 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:54:52.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:52.497+0000 7fd623e00640 1 --2- 172.21.15.131:0/3010521626 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd6240b5450 0x7fd62415ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:52.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:52.498+0000 7fd629000640 1 --2- 172.21.15.131:0/3010521626 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd62406f420 0x7fd624151eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:52.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:52.498+0000 7fd623e00640 1 -- 172.21.15.131:0/3010521626 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd62406f420 msgr2=0x7fd624151eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:52.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:52.498+0000 7fd623e00640 1 --2- 172.21.15.131:0/3010521626 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd62406f420 0x7fd624151eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:52.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:52.498+0000 7fd623e00640 1 -- 172.21.15.131:0/3010521626 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6240cf590 msgr2=0x7fd6240b1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:52.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:52.498+0000 7fd623e00640 1 --2- 172.21.15.131:0/3010521626 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6240cf590 0x7fd6240b1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:52.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:52.498+0000 7fd623e00640 1 -- 172.21.15.131:0/3010521626 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd62415f4d0 con 0x7fd6240b5450 2024-05-26T22:54:52.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:52.498+0000 7fd629a00640 1 --2- 172.21.15.131:0/3010521626 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6240cf590 0x7fd6240b1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-26T22:54:52.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:52.498+0000 7fd629000640 1 --2- 172.21.15.131:0/3010521626 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd62406f420 0x7fd624151eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-26T22:54:52.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:52.498+0000 7fd623e00640 1 --2- 172.21.15.131:0/3010521626 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd6240b5450 0x7fd62415ea10 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fd610031000 tx=0x7fd6100046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:52.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:52.499+0000 7fd620c00640 1 -- 172.21.15.131:0/3010521626 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd610041020 con 0x7fd6240b5450 2024-05-26T22:54:52.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:52.499+0000 7fd620c00640 1 -- 172.21.15.131:0/3010521626 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd610030640 con 0x7fd6240b5450 2024-05-26T22:54:52.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:52.499+0000 7fd620c00640 1 -- 172.21.15.131:0/3010521626 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd610037040 con 0x7fd6240b5450 2024-05-26T22:54:52.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:52.499+0000 7fd62ba00640 1 -- 172.21.15.131:0/3010521626 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd62415d9f0 con 0x7fd6240b5450 2024-05-26T22:54:52.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:52.499+0000 7fd62ba00640 1 -- 172.21.15.131:0/3010521626 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd62415dfb0 con 0x7fd6240b5450 2024-05-26T22:54:52.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:52.500+0000 7fd62ba00640 1 -- 172.21.15.131:0/3010521626 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd624074e90 con 0x7fd6240b5450 2024-05-26T22:54:52.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:52.501+0000 7fd620c00640 1 -- 172.21.15.131:0/3010521626 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd610044020 con 0x7fd6240b5450 2024-05-26T22:54:52.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:52.502+0000 7fd620c00640 1 --2- 172.21.15.131:0/3010521626 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fd60406be30 0x7fd60406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:52.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:52.502+0000 7fd620c00640 1 -- 172.21.15.131:0/3010521626 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5115+0+0 (secure 0 0 0) 0x7fd6100ab280 con 0x7fd6240b5450 2024-05-26T22:54:52.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:52.502+0000 7fd629000640 1 --2- 172.21.15.131:0/3010521626 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fd60406be30 0x7fd60406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:52.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:52.503+0000 7fd629000640 1 --2- 172.21.15.131:0/3010521626 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fd60406be30 0x7fd60406e2f0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fd618009af0 tx=0x7fd618008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:52.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:52.507+0000 7fd620c00640 1 -- 172.21.15.131:0/3010521626 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fd61004b030 con 0x7fd6240b5450 2024-05-26T22:54:52.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:52.710+0000 7fd62ba00640 1 -- 172.21.15.131:0/3010521626 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fd6240cee90 con 0x7fd6240b5450 2024-05-26T22:54:52.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:52.834+0000 7fd620c00640 1 -- 172.21.15.131:0/3010521626 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v89) ==== 97+0+0 (secure 0 0 0) 0x7fd610079f90 con 0x7fd6240b5450 2024-05-26T22:54:52.834 INFO:teuthology.orchestra.run.smithi131.stderr: removed all blocklist entries 2024-05-26T22:54:52.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:52.837+0000 7fd62ba00640 1 -- 172.21.15.131:0/3010521626 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fd60406be30 msgr2=0x7fd60406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:52.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:52.837+0000 7fd62ba00640 1 --2- 172.21.15.131:0/3010521626 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fd60406be30 0x7fd60406e2f0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fd618009af0 tx=0x7fd618008040 comp rx=0 tx=0).stop 2024-05-26T22:54:52.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:52.837+0000 7fd62ba00640 1 -- 172.21.15.131:0/3010521626 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd6240b5450 msgr2=0x7fd62415ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:52.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:52.837+0000 7fd62ba00640 1 --2- 172.21.15.131:0/3010521626 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd6240b5450 0x7fd62415ea10 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fd610031000 tx=0x7fd6100046c0 comp rx=0 tx=0).stop 2024-05-26T22:54:52.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:52.837+0000 7fd62ba00640 1 -- 172.21.15.131:0/3010521626 shutdown_connections 2024-05-26T22:54:52.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:52.837+0000 7fd62ba00640 1 --2- 172.21.15.131:0/3010521626 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fd60406be30 0x7fd60406e2f0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:52.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:52.837+0000 7fd62ba00640 1 --2- 172.21.15.131:0/3010521626 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6240cf590 0x7fd6240b1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:52.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:52.837+0000 7fd62ba00640 1 --2- 172.21.15.131:0/3010521626 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd6240b5450 0x7fd62415ea10 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:52.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:52.837+0000 7fd62ba00640 1 --2- 172.21.15.131:0/3010521626 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd62406f420 0x7fd624151eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:52.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:52.837+0000 7fd62ba00640 1 -- 172.21.15.131:0/3010521626 >> 172.21.15.131:0/3010521626 conn(0x7fd62406bc90 msgr2=0x7fd62415a1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:52.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:52.837+0000 7fd62ba00640 1 -- 172.21.15.131:0/3010521626 shutdown_connections 2024-05-26T22:54:52.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:52.837+0000 7fd62ba00640 1 -- 172.21.15.131:0/3010521626 wait complete. 2024-05-26T22:54:52.857 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-26T22:54:53.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.040+0000 7faf4b200640 1 Processor -- start 2024-05-26T22:54:53.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.040+0000 7faf4b200640 1 -- start start 2024-05-26T22:54:53.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.042+0000 7faf4b200640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7faf440b3a50 0x7faf440b3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:53.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.042+0000 7faf4b200640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf4415e8e0 con 0x7faf441436b0 2024-05-26T22:54:53.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.042+0000 7faf4b200640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf440d2cc0 con 0x7faf44141250 2024-05-26T22:54:53.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.042+0000 7faf4b200640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7faf4415eb80 con 0x7faf440b3a50 2024-05-26T22:54:53.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.042+0000 7faf4a800640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7faf441436b0 0x7faf44143af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:35290/0 (socket says 172.21.15.131:35290) 2024-05-26T22:54:53.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.042+0000 7faf4a800640 1 -- 172.21.15.131:0/1600395629 learned_addr learned my addr 172.21.15.131:0/1600395629 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:54:53.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.042+0000 7faf49400640 1 --2- 172.21.15.131:0/1600395629 >> v2:172.21.15.187:3300/0 conn(0x7faf440b3a50 0x7faf440b3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:53.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.043+0000 7faf48a00640 1 -- 172.21.15.131:0/1600395629 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1218606728 0 0) 0x7faf4415e8e0 con 0x7faf441436b0 2024-05-26T22:54:53.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.043+0000 7faf48a00640 1 -- 172.21.15.131:0/1600395629 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faf2c003520 con 0x7faf441436b0 2024-05-26T22:54:53.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.043+0000 7faf48a00640 1 -- 172.21.15.131:0/1600395629 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3514169419 0 0) 0x7faf2c003520 con 0x7faf441436b0 2024-05-26T22:54:53.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.043+0000 7faf48a00640 1 -- 172.21.15.131:0/1600395629 >> v2:172.21.15.187:3300/0 conn(0x7faf440b3a50 msgr2=0x7faf440b3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:53.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.043+0000 7faf48a00640 1 --2- 172.21.15.131:0/1600395629 >> v2:172.21.15.187:3300/0 conn(0x7faf440b3a50 0x7faf440b3ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:53.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.043+0000 7faf48a00640 1 -- 172.21.15.131:0/1600395629 >> v1:172.21.15.187:6789/0 conn(0x7faf44141250 legacy=0x7faf440b5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:53.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.043+0000 7faf48a00640 1 -- 172.21.15.131:0/1600395629 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf4415ee60 con 0x7faf441436b0 2024-05-26T22:54:53.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.043+0000 7faf48a00640 1 -- 172.21.15.131:0/1600395629 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7faf400019f0 con 0x7faf441436b0 2024-05-26T22:54:53.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.043+0000 7faf48a00640 1 -- 172.21.15.131:0/1600395629 >> v1:172.21.15.131:6789/0 conn(0x7faf441436b0 legacy=0x7faf44143af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:53.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.043+0000 7faf48a00640 1 --2- 172.21.15.131:0/1600395629 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf2c0039b0 0x7faf2c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:53.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.043+0000 7faf48a00640 1 --2- 172.21.15.131:0/1600395629 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faf2c004800 0x7faf2c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:53.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.043+0000 7faf48a00640 1 --2- 172.21.15.131:0/1600395629 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faf2c0051c0 0x7faf2c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:53.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.043+0000 7faf48a00640 1 -- 172.21.15.131:0/1600395629 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7faf4415eb80 con 0x7faf2c0039b0 2024-05-26T22:54:53.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.043+0000 7faf48a00640 1 -- 172.21.15.131:0/1600395629 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7faf2c00c060 con 0x7faf2c0051c0 2024-05-26T22:54:53.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.043+0000 7faf48a00640 1 -- 172.21.15.131:0/1600395629 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7faf2c00c260 con 0x7faf2c004800 2024-05-26T22:54:53.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.043+0000 7faf49e00640 1 --2- 172.21.15.131:0/1600395629 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf2c0039b0 0x7faf2c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:53.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.044+0000 7faf4a800640 1 --2- 172.21.15.131:0/1600395629 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faf2c0051c0 0x7faf2c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:53.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.044+0000 7faf49e00640 1 -- 172.21.15.131:0/1600395629 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faf2c004800 msgr2=0x7faf2c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:53.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.044+0000 7faf49e00640 1 --2- 172.21.15.131:0/1600395629 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faf2c004800 0x7faf2c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:53.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.044+0000 7faf49e00640 1 -- 172.21.15.131:0/1600395629 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faf2c0051c0 msgr2=0x7faf2c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:53.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.044+0000 7faf49e00640 1 --2- 172.21.15.131:0/1600395629 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faf2c0051c0 0x7faf2c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:53.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.044+0000 7faf49e00640 1 -- 172.21.15.131:0/1600395629 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf2c003520 con 0x7faf2c0039b0 2024-05-26T22:54:53.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.044+0000 7faf4a800640 1 --2- 172.21.15.131:0/1600395629 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faf2c0051c0 0x7faf2c00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:54:53.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.044+0000 7faf4a800640 1 -- 172.21.15.131:0/1600395629 reap_dead start 2024-05-26T22:54:53.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.045+0000 7faf49e00640 1 --2- 172.21.15.131:0/1600395629 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf2c0039b0 0x7faf2c003dd0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7faf38001720 tx=0x7faf2c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a543a90a6511b199 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:53.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.045+0000 7faf48a00640 1 -- 172.21.15.131:0/1600395629 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf38001370 con 0x7faf2c0039b0 2024-05-26T22:54:53.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.045+0000 7faf48a00640 1 -- 172.21.15.131:0/1600395629 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf380052d0 con 0x7faf2c0039b0 2024-05-26T22:54:53.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.045+0000 7faf48a00640 1 -- 172.21.15.131:0/1600395629 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf3802ab90 con 0x7faf2c0039b0 2024-05-26T22:54:53.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.045+0000 7faf4b200640 1 -- 172.21.15.131:0/1600395629 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf2c0039b0 msgr2=0x7faf2c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:53.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.045+0000 7faf4b200640 1 --2- 172.21.15.131:0/1600395629 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf2c0039b0 0x7faf2c003dd0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7faf38001720 tx=0x7faf2c005830 comp rx=0 tx=0).stop 2024-05-26T22:54:53.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.046+0000 7faf4b200640 1 -- 172.21.15.131:0/1600395629 shutdown_connections 2024-05-26T22:54:53.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.046+0000 7faf4b200640 1 --2- 172.21.15.131:0/1600395629 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf2c0039b0 0x7faf2c003dd0 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:53.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.046+0000 7faf4b200640 1 -- 172.21.15.131:0/1600395629 >> 172.21.15.131:0/1600395629 conn(0x7faf4406b940 msgr2=0x7faf440b72e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:53.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.046+0000 7faf4b200640 1 -- 172.21.15.131:0/1600395629 shutdown_connections 2024-05-26T22:54:53.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.046+0000 7faf4b200640 1 -- 172.21.15.131:0/1600395629 wait complete. 2024-05-26T22:54:53.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.047+0000 7faf4b200640 1 Processor -- start 2024-05-26T22:54:53.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.047+0000 7faf4b200640 1 -- start start 2024-05-26T22:54:53.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.048+0000 7faf4b200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf2c0039b0 0x7faf4413df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:53.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.048+0000 7faf4b200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faf44145f90 0x7faf44144470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:53.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.048+0000 7faf4b200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faf441449b0 0x7faf44146e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:53.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.048+0000 7faf4b200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7faf441418a0 con 0x7faf2c0039b0 2024-05-26T22:54:53.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.048+0000 7faf4b200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7faf4415f420 con 0x7faf44145f90 2024-05-26T22:54:53.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.048+0000 7faf4b200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7faf44070410 con 0x7faf441449b0 2024-05-26T22:54:53.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.049+0000 7faf49e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf2c0039b0 0x7faf4413df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:53.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.049+0000 7faf49e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf2c0039b0 0x7faf4413df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59400/0 (socket says 172.21.15.131:59400) 2024-05-26T22:54:53.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.049+0000 7faf49e00640 1 -- 172.21.15.131:0/2912923983 learned_addr learned my addr 172.21.15.131:0/2912923983 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:54:53.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.049+0000 7faf49e00640 1 -- 172.21.15.131:0/2912923983 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faf441449b0 msgr2=0x7faf44146e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:53.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.049+0000 7faf49e00640 1 --2- 172.21.15.131:0/2912923983 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faf441449b0 0x7faf44146e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:53.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.049+0000 7faf49e00640 1 -- 172.21.15.131:0/2912923983 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faf44145f90 msgr2=0x7faf44144470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:53.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.049+0000 7faf49e00640 1 --2- 172.21.15.131:0/2912923983 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faf44145f90 0x7faf44144470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:53.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.049+0000 7faf49e00640 1 -- 172.21.15.131:0/2912923983 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf44144ef0 con 0x7faf2c0039b0 2024-05-26T22:54:53.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.049+0000 7faf49e00640 1 --2- 172.21.15.131:0/2912923983 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf2c0039b0 0x7faf4413df70 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7faf3802a540 tx=0x7faf38001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:53.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.050+0000 7faf3e000640 1 -- 172.21.15.131:0/2912923983 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf380076c0 con 0x7faf2c0039b0 2024-05-26T22:54:53.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.050+0000 7faf3e000640 1 -- 172.21.15.131:0/2912923983 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf3803f070 con 0x7faf2c0039b0 2024-05-26T22:54:53.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.050+0000 7faf3e000640 1 -- 172.21.15.131:0/2912923983 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf38001370 con 0x7faf2c0039b0 2024-05-26T22:54:53.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.050+0000 7faf4b200640 1 -- 172.21.15.131:0/2912923983 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf441451d0 con 0x7faf2c0039b0 2024-05-26T22:54:53.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.050+0000 7faf4b200640 1 -- 172.21.15.131:0/2912923983 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf44145790 con 0x7faf2c0039b0 2024-05-26T22:54:53.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.051+0000 7faf4b200640 1 -- 172.21.15.131:0/2912923983 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf440694e0 con 0x7faf2c0039b0 2024-05-26T22:54:53.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.052+0000 7faf3e000640 1 -- 172.21.15.131:0/2912923983 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7faf380052d0 con 0x7faf2c0039b0 2024-05-26T22:54:53.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.053+0000 7faf3e000640 1 --2- 172.21.15.131:0/2912923983 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7faf2806be60 0x7faf2806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:53.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.053+0000 7faf3e000640 1 -- 172.21.15.131:0/2912923983 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 4857+0+0 (secure 0 0 0) 0x7faf3802be00 con 0x7faf2c0039b0 2024-05-26T22:54:53.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.053+0000 7faf49400640 1 --2- 172.21.15.131:0/2912923983 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7faf2806be60 0x7faf2806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:53.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.054+0000 7faf49400640 1 --2- 172.21.15.131:0/2912923983 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7faf2806be60 0x7faf2806e320 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7faf340093d0 tx=0x7faf34009300 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:53.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.057+0000 7faf3e000640 1 -- 172.21.15.131:0/2912923983 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7faf38079580 con 0x7faf2c0039b0 2024-05-26T22:54:53.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.328+0000 7faf4b200640 1 -- 172.21.15.131:0/2912923983 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7faf440700a0 con 0x7faf2c0039b0 2024-05-26T22:54:53.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.329+0000 7faf3e000640 1 -- 172.21.15.131:0/2912923983 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v17) ==== 80+0+5304 (secure 0 0 0) 0x7faf38078e80 con 0x7faf2c0039b0 2024-05-26T22:54:53.328 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:54:53.328 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-26T22:54:53.329 INFO:teuthology.orchestra.run.smithi131.stdout: "auth_dump": [ 2024-05-26T22:54:53.329 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:54:53.329 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mds.a", 2024-05-26T22:54:53.329 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD1u1NmTQf1DhAAdvEt7wP0IgKhNHOqjVAllA==", 2024-05-26T22:54:53.329 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:54:53.329 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-05-26T22:54:53.329 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2024-05-26T22:54:53.329 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2024-05-26T22:54:53.329 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-05-26T22:54:53.329 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:54:53.329 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:54:53.329 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:54:53.329 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mds.b", 2024-05-26T22:54:53.330 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD1u1NmHNZ8KxAAPgQ/ojbl+EDfPhqrkg7LPQ==", 2024-05-26T22:54:53.330 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:54:53.330 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-05-26T22:54:53.330 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2024-05-26T22:54:53.330 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2024-05-26T22:54:53.330 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-05-26T22:54:53.330 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:54:53.330 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:54:53.330 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:54:53.330 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mds.c", 2024-05-26T22:54:53.330 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD1u1NmFjCHHRAAqix3Ujfsq2tnSU5+d/pSYA==", 2024-05-26T22:54:53.330 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:54:53.331 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-05-26T22:54:53.331 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2024-05-26T22:54:53.331 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2024-05-26T22:54:53.331 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-05-26T22:54:53.331 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:54:53.331 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:54:53.331 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:54:53.331 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mds.d", 2024-05-26T22:54:53.331 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD1u1NmM4LPORAAhtQtr+Vfpf7vdVwjSW6xtA==", 2024-05-26T22:54:53.331 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:54:53.331 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-05-26T22:54:53.331 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2024-05-26T22:54:53.331 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2024-05-26T22:54:53.332 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-05-26T22:54:53.332 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:54:53.332 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:54:53.332 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:54:53.332 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.0", 2024-05-26T22:54:53.332 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD5u1Nm+w1LNhAAsJLQkiKSh3KkMymlrZkcrw==", 2024-05-26T22:54:53.332 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:54:53.332 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-05-26T22:54:53.332 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-05-26T22:54:53.332 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-05-26T22:54:53.332 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:54:53.332 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:54:53.332 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:54:53.333 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.1", 2024-05-26T22:54:53.333 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD9u1Nm4c8RFBAAcn9HQ+izML9rujJ/2/Q/BQ==", 2024-05-26T22:54:53.333 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:54:53.333 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-05-26T22:54:53.333 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-05-26T22:54:53.333 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-05-26T22:54:53.333 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:54:53.333 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:54:53.333 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:54:53.333 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.2", 2024-05-26T22:54:53.333 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAAvFNmR5wmLRAAYEBAXPBZFq67iARZnwF3iw==", 2024-05-26T22:54:53.333 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:54:53.333 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-05-26T22:54:53.333 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-05-26T22:54:53.334 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-05-26T22:54:53.334 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:54:53.334 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:54:53.334 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:54:53.334 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.3", 2024-05-26T22:54:53.334 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAEvFNmJ4G5CxAAnTbP8DinhmuvGNo1VBVzjQ==", 2024-05-26T22:54:53.334 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:54:53.334 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-05-26T22:54:53.334 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-05-26T22:54:53.334 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-05-26T22:54:53.334 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:54:53.334 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:54:53.334 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:54:53.335 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.4", 2024-05-26T22:54:53.335 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAKvFNmcw5rGhAAvnivP6kt3vcN9UPm/svKqw==", 2024-05-26T22:54:53.335 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:54:53.335 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-05-26T22:54:53.335 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-05-26T22:54:53.335 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-05-26T22:54:53.335 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:54:53.335 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:54:53.335 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:54:53.335 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.5", 2024-05-26T22:54:53.335 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQANvFNmTqlzNhAAGfrSvNNHYKGOBzRXo0TEdw==", 2024-05-26T22:54:53.335 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:54:53.335 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-05-26T22:54:53.335 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-05-26T22:54:53.336 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-05-26T22:54:53.336 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:54:53.336 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:54:53.336 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:54:53.336 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.6", 2024-05-26T22:54:53.336 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQARvFNmpzd6FRAAgi2RYlQPvfHNe0iP4bJrwg==", 2024-05-26T22:54:53.336 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:54:53.336 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-05-26T22:54:53.336 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-05-26T22:54:53.336 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-05-26T22:54:53.336 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:54:53.336 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:54:53.336 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:54:53.336 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.7", 2024-05-26T22:54:53.337 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAUvFNmAkC8LhAAsmkKF4eloxdpCi/mrDtzqA==", 2024-05-26T22:54:53.337 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:54:53.337 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-05-26T22:54:53.337 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-05-26T22:54:53.337 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-05-26T22:54:53.337 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:54:53.337 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:54:53.337 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:54:53.337 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.0", 2024-05-26T22:54:53.338 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD2u1NmSGVWChAAH/wq3iaUs6ZELRhLKhmURQ==", 2024-05-26T22:54:53.338 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:54:53.338 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-05-26T22:54:53.338 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow r", 2024-05-26T22:54:53.338 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-05-26T22:54:53.338 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:54:53.338 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:54:53.338 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:54:53.338 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.1", 2024-05-26T22:54:53.338 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD2u1NmnA9CHRAA2XMmZQxysKweHNykiQUEeA==", 2024-05-26T22:54:53.338 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:54:53.338 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-05-26T22:54:53.338 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow r", 2024-05-26T22:54:53.339 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow rw", 2024-05-26T22:54:53.339 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow rwx" 2024-05-26T22:54:53.339 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:54:53.339 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:54:53.339 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:54:53.339 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.2", 2024-05-26T22:54:53.339 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD2u1NmGEI9ExAAvJn1LRwmALkD7zh/AP07vQ==", 2024-05-26T22:54:53.339 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:54:53.339 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-05-26T22:54:53.339 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow r", 2024-05-26T22:54:53.339 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow rw", 2024-05-26T22:54:53.339 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow rwx" 2024-05-26T22:54:53.339 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:54:53.340 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:54:53.340 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:54:53.340 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.3", 2024-05-26T22:54:53.340 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD2u1NmIHOtJhAAJCUm0kGqWu4rWVLsDkW6Eg==", 2024-05-26T22:54:53.340 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:54:53.340 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-05-26T22:54:53.340 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow r", 2024-05-26T22:54:53.340 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow rw", 2024-05-26T22:54:53.340 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow rwx" 2024-05-26T22:54:53.340 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:54:53.340 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:54:53.340 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:54:53.340 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.admin", 2024-05-26T22:54:53.340 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD0u1NmLRGoCBAAhDeVJV14MTAIdUte067jig==", 2024-05-26T22:54:53.340 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:54:53.341 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow *", 2024-05-26T22:54:53.341 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2024-05-26T22:54:53.341 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2024-05-26T22:54:53.341 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-05-26T22:54:53.341 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:54:53.341 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:54:53.341 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:54:53.341 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mgr.x", 2024-05-26T22:54:53.341 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD1u1NmaWhyBBAAlU7Lq6HMjkCPu0kztJIAXQ==", 2024-05-26T22:54:53.341 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:54:53.341 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow *", 2024-05-26T22:54:53.341 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile mgr", 2024-05-26T22:54:53.341 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-05-26T22:54:53.341 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:54:53.342 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:54:53.342 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:54:53.342 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mgr.y", 2024-05-26T22:54:53.342 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD0u1NmX1RhNhAAo5mEOGzUyrdwFlB/XOXKnw==", 2024-05-26T22:54:53.342 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:54:53.342 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow *", 2024-05-26T22:54:53.342 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile mgr", 2024-05-26T22:54:53.342 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-05-26T22:54:53.342 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:54:53.342 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:54:53.342 INFO:teuthology.orchestra.run.smithi131.stdout: ] 2024-05-26T22:54:53.342 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-26T22:54:53.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.331+0000 7faf4b200640 1 -- 172.21.15.131:0/2912923983 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7faf2806be60 msgr2=0x7faf2806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:53.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.331+0000 7faf4b200640 1 --2- 172.21.15.131:0/2912923983 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7faf2806be60 0x7faf2806e320 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7faf340093d0 tx=0x7faf34009300 comp rx=0 tx=0).stop 2024-05-26T22:54:53.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.331+0000 7faf4b200640 1 -- 172.21.15.131:0/2912923983 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf2c0039b0 msgr2=0x7faf4413df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:53.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.331+0000 7faf4b200640 1 --2- 172.21.15.131:0/2912923983 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf2c0039b0 0x7faf4413df70 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7faf3802a540 tx=0x7faf38001120 comp rx=0 tx=0).stop 2024-05-26T22:54:53.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.332+0000 7faf4b200640 1 -- 172.21.15.131:0/2912923983 shutdown_connections 2024-05-26T22:54:53.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.332+0000 7faf4b200640 1 --2- 172.21.15.131:0/2912923983 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faf441449b0 0x7faf44146e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:53.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.332+0000 7faf4b200640 1 --2- 172.21.15.131:0/2912923983 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7faf2806be60 0x7faf2806e320 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:53.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.332+0000 7faf4b200640 1 --2- 172.21.15.131:0/2912923983 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faf44145f90 0x7faf44144470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:53.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.332+0000 7faf4b200640 1 --2- 172.21.15.131:0/2912923983 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf2c0039b0 0x7faf4413df70 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:53.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.332+0000 7faf4b200640 1 -- 172.21.15.131:0/2912923983 >> 172.21.15.131:0/2912923983 conn(0x7faf4406b940 msgr2=0x7faf4413fbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:53.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.332+0000 7faf4b200640 1 -- 172.21.15.131:0/2912923983 shutdown_connections 2024-05-26T22:54:53.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.332+0000 7faf4b200640 1 -- 172.21.15.131:0/2912923983 wait complete. 2024-05-26T22:54:53.350 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-26T22:54:53.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.548+0000 7f30f8600640 1 Processor -- start 2024-05-26T22:54:53.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.548+0000 7f30f8600640 1 -- start start 2024-05-26T22:54:53.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.549+0000 7f30f8600640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f30f006dcc0 0x7f30f00b3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:53.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.549+0000 7f30f8600640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f30f00b36d0 0x7f30f015fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:53.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.549+0000 7f30f8600640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30f00b3e50 con 0x7f30f0158c00 2024-05-26T22:54:53.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.549+0000 7f30f8600640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f30f00b4130 con 0x7f30f006dcc0 2024-05-26T22:54:53.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.549+0000 7f30f8600640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f30f00b4410 con 0x7f30f00b36d0 2024-05-26T22:54:53.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.550+0000 7f30f5200640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f30f006dcc0 0x7f30f00b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:53.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.550+0000 7f30f5200640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f30f006dcc0 0x7f30f00b3190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.131:55690/0 (socket says 172.21.15.131:55690) 2024-05-26T22:54:53.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.550+0000 7f30f5200640 1 -- 172.21.15.131:0/1405023812 learned_addr learned my addr 172.21.15.131:0/1405023812 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:54:53.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.550+0000 7f30f6600640 1 --2- 172.21.15.131:0/1405023812 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f30f00b36d0 0x7f30f015fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:53.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.551+0000 7f30f5200640 1 -- 172.21.15.131:0/1405023812 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f30f00b36d0 msgr2=0x7f30f015fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:53.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.551+0000 7f30f5200640 1 --2- 172.21.15.131:0/1405023812 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f30f00b36d0 0x7f30f015fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:53.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.551+0000 7f30f5200640 1 -- 172.21.15.131:0/1405023812 >> v1:172.21.15.187:6789/0 conn(0x7f30f0158c00 legacy=0x7f30f006d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:53.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.551+0000 7f30f5200640 1 -- 172.21.15.131:0/1405023812 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30f00b46f0 con 0x7f30f006dcc0 2024-05-26T22:54:53.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.552+0000 7f30f5200640 1 --2- 172.21.15.131:0/1405023812 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f30f006dcc0 0x7f30f00b3190 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f30e0009870 tx=0x7f30e002a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17082337d2602f77 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:53.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.552+0000 7f30efe00640 1 -- 172.21.15.131:0/1405023812 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e0037070 con 0x7f30f006dcc0 2024-05-26T22:54:53.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.552+0000 7f30efe00640 1 -- 172.21.15.131:0/1405023812 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30e0032440 con 0x7f30f006dcc0 2024-05-26T22:54:53.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.552+0000 7f30efe00640 1 -- 172.21.15.131:0/1405023812 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e0032810 con 0x7f30f006dcc0 2024-05-26T22:54:53.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.553+0000 7f30f8600640 1 -- 172.21.15.131:0/1405023812 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f30f006dcc0 msgr2=0x7f30f00b3190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:53.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.553+0000 7f30f8600640 1 --2- 172.21.15.131:0/1405023812 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f30f006dcc0 0x7f30f00b3190 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f30e0009870 tx=0x7f30e002a1c0 comp rx=0 tx=0).stop 2024-05-26T22:54:53.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.553+0000 7f30f8600640 1 -- 172.21.15.131:0/1405023812 shutdown_connections 2024-05-26T22:54:53.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.553+0000 7f30f8600640 1 --2- 172.21.15.131:0/1405023812 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f30f00b36d0 0x7f30f015fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:53.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.553+0000 7f30f8600640 1 --2- 172.21.15.131:0/1405023812 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f30f006dcc0 0x7f30f00b3190 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:53.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.553+0000 7f30f8600640 1 -- 172.21.15.131:0/1405023812 >> 172.21.15.131:0/1405023812 conn(0x7f30f00b1e90 msgr2=0x7f30f00b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:53.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.553+0000 7f30f8600640 1 -- 172.21.15.131:0/1405023812 shutdown_connections 2024-05-26T22:54:53.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.553+0000 7f30f8600640 1 -- 172.21.15.131:0/1405023812 wait complete. 2024-05-26T22:54:53.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.556+0000 7f30f8600640 1 Processor -- start 2024-05-26T22:54:53.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.556+0000 7f30f8600640 1 -- start start 2024-05-26T22:54:53.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.557+0000 7f30f8600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f30f006dcc0 0x7f30f0072710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:53.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.557+0000 7f30f8600640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f30f00b36d0 0x7f30f015b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:53.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.557+0000 7f30f8600640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f30f0158c00 0x7f30f015b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:53.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.557+0000 7f30f8600640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f30f0160190 con 0x7f30f006dcc0 2024-05-26T22:54:53.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.557+0000 7f30f8600640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f30f00b3b60 con 0x7f30f0158c00 2024-05-26T22:54:53.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.557+0000 7f30f8600640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f30f0160310 con 0x7f30f00b36d0 2024-05-26T22:54:53.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.557+0000 7f30f5c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f30f006dcc0 0x7f30f0072710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:53.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.557+0000 7f30f5c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f30f006dcc0 0x7f30f0072710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59406/0 (socket says 172.21.15.131:59406) 2024-05-26T22:54:53.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.557+0000 7f30f5c00640 1 -- 172.21.15.131:0/4202873333 learned_addr learned my addr 172.21.15.131:0/4202873333 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:54:53.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.558+0000 7f30f5200640 1 --2- 172.21.15.131:0/4202873333 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f30f00b36d0 0x7f30f015b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:53.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.558+0000 7f30f6600640 1 --2- 172.21.15.131:0/4202873333 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f30f0158c00 0x7f30f015b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:53.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.558+0000 7f30f5c00640 1 -- 172.21.15.131:0/4202873333 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f30f00b36d0 msgr2=0x7f30f015b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:53.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.558+0000 7f30f5c00640 1 --2- 172.21.15.131:0/4202873333 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f30f00b36d0 0x7f30f015b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:53.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.558+0000 7f30f5c00640 1 -- 172.21.15.131:0/4202873333 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f30f0158c00 msgr2=0x7f30f015b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:53.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.558+0000 7f30f5c00640 1 --2- 172.21.15.131:0/4202873333 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f30f0158c00 0x7f30f015b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:53.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.558+0000 7f30f5c00640 1 -- 172.21.15.131:0/4202873333 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30f015bd30 con 0x7f30f006dcc0 2024-05-26T22:54:53.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.558+0000 7f30f5c00640 1 --2- 172.21.15.131:0/4202873333 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f30f006dcc0 0x7f30f0072710 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f30e400c3f0 tx=0x7f30e400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:53.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.558+0000 7f30ed600640 1 -- 172.21.15.131:0/4202873333 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e40052d0 con 0x7f30f006dcc0 2024-05-26T22:54:53.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.558+0000 7f30ed600640 1 -- 172.21.15.131:0/4202873333 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30e4005490 con 0x7f30f006dcc0 2024-05-26T22:54:53.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.558+0000 7f30ed600640 1 -- 172.21.15.131:0/4202873333 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e4007260 con 0x7f30f006dcc0 2024-05-26T22:54:53.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.558+0000 7f30f8600640 1 -- 172.21.15.131:0/4202873333 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30f0144dd0 con 0x7f30f006dcc0 2024-05-26T22:54:53.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.559+0000 7f30f8600640 1 -- 172.21.15.131:0/4202873333 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30f0145390 con 0x7f30f006dcc0 2024-05-26T22:54:53.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.560+0000 7f30f8600640 1 -- 172.21.15.131:0/4202873333 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30f00733d0 con 0x7f30f006dcc0 2024-05-26T22:54:53.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.561+0000 7f30ed600640 1 -- 172.21.15.131:0/4202873333 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f30e4007400 con 0x7f30f006dcc0 2024-05-26T22:54:53.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.562+0000 7f30ed600640 1 --2- 172.21.15.131:0/4202873333 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f30d406be30 0x7f30d406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:53.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.562+0000 7f30ed600640 1 -- 172.21.15.131:0/4202873333 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 4857+0+0 (secure 0 0 0) 0x7f30e4089ff0 con 0x7f30f006dcc0 2024-05-26T22:54:53.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.562+0000 7f30f5200640 1 --2- 172.21.15.131:0/4202873333 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f30d406be30 0x7f30d406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:53.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.563+0000 7f30f5200640 1 --2- 172.21.15.131:0/4202873333 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f30d406be30 0x7f30d406e2f0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f30e0034000 tx=0x7f30e002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:53.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.566+0000 7f30ed600640 1 -- 172.21.15.131:0/4202873333 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f30f00733d0 con 0x7f30f006dcc0 2024-05-26T22:54:53.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.760+0000 7f30f8600640 1 -- 172.21.15.131:0/4202873333 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f30f00694e0 con 0x7f30f006dcc0 2024-05-26T22:54:53.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.762+0000 7f30ed600640 1 -- 172.21.15.131:0/4202873333 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+12916 (secure 0 0 0) 0x7f30e4058d00 con 0x7f30f006dcc0 2024-05-26T22:54:53.762 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:54:53.762 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":91,"fsid":"46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f","created":"2024-05-26T22:47:57.288905+0000","modified":"2024-05-26T22:54:52.844533+0000","last_up_change":"2024-05-26T22:48:09.602207+0000","last_in_change":"2024-05-26T22:48:00.687341+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-26T22:48:10.463339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"814eaf62-a18d-494b-8e27-fd8fb88adea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6801","nonce":2350940051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6803","nonce":2350940051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6807","nonce":2350940051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6805","nonce":2350940051}]},"public_addr":"172.21.15.131:6801/2350940051","cluster_addr":"172.21.15.131:6803/2350940051","heartbeat_back_addr":"172.21.15.131:6807/2350940051","heartbeat_front_addr":"172.21.15.131:6805/2350940051","state":["exists","up"]},{"osd":1,"uuid":"42e66287-f6c6-4e26-af3c-6dfaa3536d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6825","nonce":1525528247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6827","nonce":1525528247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6831","nonce":1525528247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6829","nonce":1525528247}]},"public_addr":"172.21.15.131:6825/1525528247","cluster_addr":"172.21.15.131:6827/1525528247","heartbeat_back_addr":"172.21.15.131:6831/1525528247","heartbeat_front_addr":"172.21.15.131:6829/1525528247","state":["exists","up"]},{"osd":2,"uuid":"0da34673-04a2-4fd0-b57a-0ff2b679b043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6817","nonce":3627842210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6819","nonce":3627842210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6823","nonce":3627842210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6821","nonce":3627842210}]},"public_addr":"172.21.15.131:6817/3627842210","cluster_addr":"172.21.15.131:6819/3627842210","heartbeat_back_addr":"172.21.15.131:6823/3627842210","heartbeat_front_addr":"172.21.15.131:6821/3627842210","state":["exists","up"]},{"osd":3,"uuid":"cf117a27-d6b2-43d3-ae52-b47988bf8eed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6809","nonce":350790591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6811","nonce":350790591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6815","nonce":350790591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6813","nonce":350790591}]},"public_addr":"172.21.15.131:6809/350790591","cluster_addr":"172.21.15.131:6811/350790591","heartbeat_back_addr":"172.21.15.131:6815/350790591","heartbeat_front_addr":"172.21.15.131:6813/350790591","state":["exists","up"]},{"osd":4,"uuid":"a092ccfc-be89-4a84-88d5-0f781fa715eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6816","nonce":244560995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6819","nonce":244560995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6823","nonce":244560995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6821","nonce":244560995}]},"public_addr":"172.21.15.187:6816/244560995","cluster_addr":"172.21.15.187:6819/244560995","heartbeat_back_addr":"172.21.15.187:6823/244560995","heartbeat_front_addr":"172.21.15.187:6821/244560995","state":["exists","up"]},{"osd":5,"uuid":"7513ba69-dd00-4a35-b190-9e8cc3b2bdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6801","nonce":324619398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6803","nonce":324619398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6812","nonce":324619398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6807","nonce":324619398}]},"public_addr":"172.21.15.187:6801/324619398","cluster_addr":"172.21.15.187:6803/324619398","heartbeat_back_addr":"172.21.15.187:6812/324619398","heartbeat_front_addr":"172.21.15.187:6807/324619398","state":["exists","up"]},{"osd":6,"uuid":"50081810-c093-4d2f-a4d7-b7c695c5400d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6806","nonce":1391347786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6809","nonce":1391347786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6818","nonce":1391347786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6813","nonce":1391347786}]},"public_addr":"172.21.15.187:6806/1391347786","cluster_addr":"172.21.15.187:6809/1391347786","heartbeat_back_addr":"172.21.15.187:6818/1391347786","heartbeat_front_addr":"172.21.15.187:6813/1391347786","state":["exists","up"]},{"osd":7,"uuid":"775dae05-54f5-4ad4-bbce-92863c6ed2ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6825","nonce":1220065622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6827","nonce":1220065622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6831","nonce":1220065622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6829","nonce":1220065622}]},"public_addr":"172.21.15.187:6825/1220065622","cluster_addr":"172.21.15.187:6827/1220065622","heartbeat_back_addr":"172.21.15.187:6831/1220065622","heartbeat_front_addr":"172.21.15.187:6829/1220065622","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:05.983520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.299516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.067323+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.307748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.018742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.258775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.010912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.295355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-26T22:54:53.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.764+0000 7f30f8600640 1 -- 172.21.15.131:0/4202873333 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f30d406be30 msgr2=0x7f30d406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:53.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.764+0000 7f30f8600640 1 --2- 172.21.15.131:0/4202873333 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f30d406be30 0x7f30d406e2f0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f30e0034000 tx=0x7f30e002e000 comp rx=0 tx=0).stop 2024-05-26T22:54:53.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.764+0000 7f30f8600640 1 -- 172.21.15.131:0/4202873333 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f30f006dcc0 msgr2=0x7f30f0072710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:53.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.764+0000 7f30f8600640 1 --2- 172.21.15.131:0/4202873333 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f30f006dcc0 0x7f30f0072710 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f30e400c3f0 tx=0x7f30e400c8b0 comp rx=0 tx=0).stop 2024-05-26T22:54:53.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.764+0000 7f30f8600640 1 -- 172.21.15.131:0/4202873333 shutdown_connections 2024-05-26T22:54:53.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.764+0000 7f30f8600640 1 --2- 172.21.15.131:0/4202873333 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f30f00b36d0 0x7f30f015b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:53.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.765+0000 7f30f8600640 1 --2- 172.21.15.131:0/4202873333 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f30d406be30 0x7f30d406e2f0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:53.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.765+0000 7f30f8600640 1 --2- 172.21.15.131:0/4202873333 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f30f0158c00 0x7f30f015b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:53.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.765+0000 7f30f8600640 1 --2- 172.21.15.131:0/4202873333 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f30f006dcc0 0x7f30f0072710 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:53.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.765+0000 7f30f8600640 1 -- 172.21.15.131:0/4202873333 >> 172.21.15.131:0/4202873333 conn(0x7f30f00b1e90 msgr2=0x7f30f00af180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:53.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.765+0000 7f30f8600640 1 -- 172.21.15.131:0/4202873333 shutdown_connections 2024-05-26T22:54:53.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.765+0000 7f30f8600640 1 -- 172.21.15.131:0/4202873333 wait complete. 2024-05-26T22:54:53.782 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-26T22:48:10.463339+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-26T22:54:53.783 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-26T22:54:53.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.978+0000 7f828fe00640 1 Processor -- start 2024-05-26T22:54:53.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.978+0000 7f828fe00640 1 -- start start 2024-05-26T22:54:53.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.980+0000 7f828fe00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f82880b5450 0x7f82880b43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:53.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.980+0000 7f828fe00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f828806f420 0x7f82880cfd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:53.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.980+0000 7f828fe00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82880b4900 con 0x7f82880cf590 2024-05-26T22:54:53.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.980+0000 7f828fe00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f8288160d90 con 0x7f828806f420 2024-05-26T22:54:53.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.980+0000 7f828fe00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f8288160180 con 0x7f82880b5450 2024-05-26T22:54:53.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.980+0000 7f828ca00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f828806f420 0x7f82880cfd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:53.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.980+0000 7f828ca00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f828806f420 0x7f82880cfd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59410/0 (socket says 172.21.15.131:59410) 2024-05-26T22:54:53.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.980+0000 7f828ca00640 1 -- 172.21.15.131:0/3384519085 learned_addr learned my addr 172.21.15.131:0/3384519085 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:54:53.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.980+0000 7f828d400640 1 --2- 172.21.15.131:0/3384519085 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f82880b5450 0x7f82880b43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:53.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.981+0000 7f828ca00640 1 -- 172.21.15.131:0/3384519085 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f82880b5450 msgr2=0x7f82880b43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:53.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.981+0000 7f828ca00640 1 --2- 172.21.15.131:0/3384519085 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f82880b5450 0x7f82880b43c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:53.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.981+0000 7f828ca00640 1 -- 172.21.15.131:0/3384519085 >> v1:172.21.15.187:6789/0 conn(0x7f82880cf590 legacy=0x7f828806d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:53.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.981+0000 7f828ca00640 1 -- 172.21.15.131:0/3384519085 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8288160460 con 0x7f828806f420 2024-05-26T22:54:53.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.981+0000 7f828ca00640 1 --2- 172.21.15.131:0/3384519085 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f828806f420 0x7f82880cfd50 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f8274009870 tx=0x7f827402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a396f63589b6fff server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:53.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.982+0000 7f8287e00640 1 -- 172.21.15.131:0/3384519085 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8274037070 con 0x7f828806f420 2024-05-26T22:54:53.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.982+0000 7f8287e00640 1 -- 172.21.15.131:0/3384519085 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8274032440 con 0x7f828806f420 2024-05-26T22:54:53.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.982+0000 7f8287e00640 1 -- 172.21.15.131:0/3384519085 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8274032810 con 0x7f828806f420 2024-05-26T22:54:53.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.982+0000 7f828fe00640 1 -- 172.21.15.131:0/3384519085 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f828806f420 msgr2=0x7f82880cfd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:53.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.982+0000 7f828fe00640 1 --2- 172.21.15.131:0/3384519085 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f828806f420 0x7f82880cfd50 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f8274009870 tx=0x7f827402a210 comp rx=0 tx=0).stop 2024-05-26T22:54:53.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.982+0000 7f828fe00640 1 -- 172.21.15.131:0/3384519085 shutdown_connections 2024-05-26T22:54:53.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.982+0000 7f828fe00640 1 --2- 172.21.15.131:0/3384519085 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f828806f420 0x7f82880cfd50 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:53.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.982+0000 7f828fe00640 1 --2- 172.21.15.131:0/3384519085 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f82880b5450 0x7f82880b43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:53.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.982+0000 7f828fe00640 1 -- 172.21.15.131:0/3384519085 >> 172.21.15.131:0/3384519085 conn(0x7f828806bc90 msgr2=0x7f82880b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:53.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.983+0000 7f828fe00640 1 -- 172.21.15.131:0/3384519085 shutdown_connections 2024-05-26T22:54:53.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.983+0000 7f828fe00640 1 -- 172.21.15.131:0/3384519085 wait complete. 2024-05-26T22:54:53.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.984+0000 7f828fe00640 1 Processor -- start 2024-05-26T22:54:53.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.984+0000 7f828fe00640 1 -- start start 2024-05-26T22:54:53.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.985+0000 7f828fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f828806f420 0x7f828809be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:53.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.985+0000 7f828fe00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f82880b5450 0x7f828815e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:53.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.985+0000 7f828fe00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f82880cf590 0x7f828814a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:53.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.985+0000 7f828fe00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f82880b5830 con 0x7f828806f420 2024-05-26T22:54:53.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.985+0000 7f828fe00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8288160a20 con 0x7f82880cf590 2024-05-26T22:54:53.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.985+0000 7f828fe00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f828815fde0 con 0x7f82880b5450 2024-05-26T22:54:53.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.986+0000 7f828d400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f828806f420 0x7f828809be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:53.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.986+0000 7f828d400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f828806f420 0x7f828809be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59412/0 (socket says 172.21.15.131:59412) 2024-05-26T22:54:53.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.986+0000 7f828d400640 1 -- 172.21.15.131:0/4064551284 learned_addr learned my addr 172.21.15.131:0/4064551284 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:54:53.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.986+0000 7f828ca00640 1 --2- 172.21.15.131:0/4064551284 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f82880b5450 0x7f828815e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:53.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.986+0000 7f828d400640 1 -- 172.21.15.131:0/4064551284 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f82880b5450 msgr2=0x7f828815e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:53.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.986+0000 7f828de00640 1 --2- 172.21.15.131:0/4064551284 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f82880cf590 0x7f828814a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:53.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.986+0000 7f828d400640 1 --2- 172.21.15.131:0/4064551284 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f82880b5450 0x7f828815e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:53.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.986+0000 7f828d400640 1 -- 172.21.15.131:0/4064551284 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f82880cf590 msgr2=0x7f828814a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:53.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.986+0000 7f828d400640 1 --2- 172.21.15.131:0/4064551284 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f82880cf590 0x7f828814a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:53.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.986+0000 7f828d400640 1 -- 172.21.15.131:0/4064551284 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f828809c3a0 con 0x7f828806f420 2024-05-26T22:54:53.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.986+0000 7f828de00640 1 --2- 172.21.15.131:0/4064551284 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f82880cf590 0x7f828814a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:54:53.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.986+0000 7f828d400640 1 --2- 172.21.15.131:0/4064551284 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f828806f420 0x7f828809be60 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f827c00d800 tx=0x7f827c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:53.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.987+0000 7f8285600640 1 -- 172.21.15.131:0/4064551284 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f827c00b6c0 con 0x7f828806f420 2024-05-26T22:54:53.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.987+0000 7f8285600640 1 -- 172.21.15.131:0/4064551284 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f827c019070 con 0x7f828806f420 2024-05-26T22:54:53.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.987+0000 7f8285600640 1 -- 172.21.15.131:0/4064551284 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f827c00ba80 con 0x7f828806f420 2024-05-26T22:54:53.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.987+0000 7f828fe00640 1 -- 172.21.15.131:0/4064551284 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f828809adc0 con 0x7f828806f420 2024-05-26T22:54:53.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.987+0000 7f828fe00640 1 -- 172.21.15.131:0/4064551284 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f828809b380 con 0x7f828806f420 2024-05-26T22:54:53.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.988+0000 7f828fe00640 1 -- 172.21.15.131:0/4064551284 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8288074e90 con 0x7f828806f420 2024-05-26T22:54:53.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.989+0000 7f8285600640 1 -- 172.21.15.131:0/4064551284 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f827c00bc20 con 0x7f828806f420 2024-05-26T22:54:53.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.990+0000 7f8285600640 1 --2- 172.21.15.131:0/4064551284 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f826806be30 0x7f826806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:53.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.990+0000 7f8285600640 1 -- 172.21.15.131:0/4064551284 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 4857+0+0 (secure 0 0 0) 0x7f827c08aff0 con 0x7f828806f420 2024-05-26T22:54:53.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.990+0000 7f828ca00640 1 --2- 172.21.15.131:0/4064551284 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f826806be30 0x7f826806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:53.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.991+0000 7f828ca00640 1 --2- 172.21.15.131:0/4064551284 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f826806be30 0x7f826806e2f0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f8274034000 tx=0x7f827402e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:53.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:53.994+0000 7f8285600640 1 -- 172.21.15.131:0/4064551284 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f827c059ce0 con 0x7f828806f420 2024-05-26T22:54:54.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.190+0000 7f828fe00640 1 -- 172.21.15.131:0/4064551284 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f82880695c0 con 0x7f828806f420 2024-05-26T22:54:54.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.190+0000 7f8285600640 1 -- 172.21.15.131:0/4064551284 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) ==== 93+0+10 (secure 0 0 0) 0x7f827c0595e0 con 0x7f828806f420 2024-05-26T22:54:54.190 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2024-05-26T22:54:54.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.193+0000 7f828fe00640 1 -- 172.21.15.131:0/4064551284 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f826806be30 msgr2=0x7f826806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:54.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.193+0000 7f828fe00640 1 --2- 172.21.15.131:0/4064551284 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f826806be30 0x7f826806e2f0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f8274034000 tx=0x7f827402e000 comp rx=0 tx=0).stop 2024-05-26T22:54:54.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.193+0000 7f828fe00640 1 -- 172.21.15.131:0/4064551284 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f828806f420 msgr2=0x7f828809be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:54.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.193+0000 7f828fe00640 1 --2- 172.21.15.131:0/4064551284 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f828806f420 0x7f828809be60 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f827c00d800 tx=0x7f827c00dcc0 comp rx=0 tx=0).stop 2024-05-26T22:54:54.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.193+0000 7f828fe00640 1 -- 172.21.15.131:0/4064551284 shutdown_connections 2024-05-26T22:54:54.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.193+0000 7f828fe00640 1 --2- 172.21.15.131:0/4064551284 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f82880b5450 0x7f828815e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:54.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.193+0000 7f828fe00640 1 --2- 172.21.15.131:0/4064551284 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f826806be30 0x7f826806e2f0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:54.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.193+0000 7f828fe00640 1 --2- 172.21.15.131:0/4064551284 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f82880cf590 0x7f828814a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:54.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.193+0000 7f828fe00640 1 --2- 172.21.15.131:0/4064551284 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f828806f420 0x7f828809be60 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:54.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.193+0000 7f828fe00640 1 -- 172.21.15.131:0/4064551284 >> 172.21.15.131:0/4064551284 conn(0x7f828806bc90 msgr2=0x7f8288071360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:54.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.193+0000 7f828fe00640 1 -- 172.21.15.131:0/4064551284 shutdown_connections 2024-05-26T22:54:54.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.193+0000 7f828fe00640 1 -- 172.21.15.131:0/4064551284 wait complete. 2024-05-26T22:54:54.211 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-26T22:54:54.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.408+0000 7f3b3d800640 1 Processor -- start 2024-05-26T22:54:54.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.408+0000 7f3b3d800640 1 -- start start 2024-05-26T22:54:54.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.409+0000 7f3b3d800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3b38155840 0x7f3b38071930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:54.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.410+0000 7f3b3d800640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f3b3815c3d0 0x7f3b3806ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:54.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.410+0000 7f3b3d800640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b380722a0 con 0x7f3b3806f490 2024-05-26T22:54:54.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.410+0000 7f3b3d800640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f3b38072580 con 0x7f3b3815c3d0 2024-05-26T22:54:54.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.410+0000 7f3b3d800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3b38072860 con 0x7f3b38155840 2024-05-26T22:54:54.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.410+0000 7f3b36000640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f3b3815c3d0 0x7f3b3806ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:54.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.410+0000 7f3b36000640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f3b3815c3d0 0x7f3b3806ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40918/0 (socket says 172.21.15.131:40918) 2024-05-26T22:54:54.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.410+0000 7f3b36000640 1 -- 172.21.15.131:0/3584015458 learned_addr learned my addr 172.21.15.131:0/3584015458 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:54:54.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.410+0000 7f3b36a00640 1 --2- 172.21.15.131:0/3584015458 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3b38155840 0x7f3b38071930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:54.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.410+0000 7f3b36000640 1 -- 172.21.15.131:0/3584015458 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3b38155840 msgr2=0x7f3b38071930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:54.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.410+0000 7f3b36000640 1 --2- 172.21.15.131:0/3584015458 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3b38155840 0x7f3b38071930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:54.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.410+0000 7f3b36000640 1 -- 172.21.15.131:0/3584015458 >> v1:172.21.15.131:6789/0 conn(0x7f3b3806f490 legacy=0x7f3b3815ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:54.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.411+0000 7f3b36000640 1 -- 172.21.15.131:0/3584015458 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b38072b40 con 0x7f3b3815c3d0 2024-05-26T22:54:54.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.410+0000 7f3b35600640 1 -- 172.21.15.131:0/3584015458 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1262711432 0 0) 0x7f3b380722a0 con 0x7f3b3806f490 2024-05-26T22:54:54.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.412+0000 7f3b36000640 1 --2- 172.21.15.131:0/3584015458 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b3815c3d0 0x7f3b3806ef50 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f3b24009870 tx=0x7f3b2402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2dbe1b99814e6f5 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:54.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.412+0000 7f3b35600640 1 -- 172.21.15.131:0/3584015458 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b24037070 con 0x7f3b3815c3d0 2024-05-26T22:54:54.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.412+0000 7f3b35600640 1 -- 172.21.15.131:0/3584015458 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b24032440 con 0x7f3b3815c3d0 2024-05-26T22:54:54.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.412+0000 7f3b35600640 1 -- 172.21.15.131:0/3584015458 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b24032810 con 0x7f3b3815c3d0 2024-05-26T22:54:54.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.412+0000 7f3b3d800640 1 -- 172.21.15.131:0/3584015458 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b3815c3d0 msgr2=0x7f3b3806ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:54.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.412+0000 7f3b3d800640 1 --2- 172.21.15.131:0/3584015458 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b3815c3d0 0x7f3b3806ef50 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f3b24009870 tx=0x7f3b2402a270 comp rx=0 tx=0).stop 2024-05-26T22:54:54.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.413+0000 7f3b3d800640 1 -- 172.21.15.131:0/3584015458 shutdown_connections 2024-05-26T22:54:54.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.413+0000 7f3b3d800640 1 --2- 172.21.15.131:0/3584015458 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b3815c3d0 0x7f3b3806ef50 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:54.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.413+0000 7f3b3d800640 1 --2- 172.21.15.131:0/3584015458 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3b38155840 0x7f3b38071930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:54.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.413+0000 7f3b3d800640 1 -- 172.21.15.131:0/3584015458 >> 172.21.15.131:0/3584015458 conn(0x7f3b380b1e90 msgr2=0x7f3b380b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:54.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.413+0000 7f3b3d800640 1 -- 172.21.15.131:0/3584015458 shutdown_connections 2024-05-26T22:54:54.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.413+0000 7f3b3d800640 1 -- 172.21.15.131:0/3584015458 wait complete. 2024-05-26T22:54:54.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.414+0000 7f3b3d800640 1 Processor -- start 2024-05-26T22:54:54.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.415+0000 7f3b3d800640 1 -- start start 2024-05-26T22:54:54.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.415+0000 7f3b3d800640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b3806f490 0x7f3b38155c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:54.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.415+0000 7f3b3d800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3b38155840 0x7f3b3814c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:54.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.416+0000 7f3b3d800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b3815c3d0 0x7f3b3815cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:54.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.416+0000 7f3b3d800640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f3b38071ff0 con 0x7f3b3815c3d0 2024-05-26T22:54:54.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.416+0000 7f3b3d800640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3b38071e70 con 0x7f3b3806f490 2024-05-26T22:54:54.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.416+0000 7f3b3d800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3b380aee50 con 0x7f3b38155840 2024-05-26T22:54:54.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.416+0000 7f3b37400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b3815c3d0 0x7f3b3815cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:54.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.416+0000 7f3b37400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b3815c3d0 0x7f3b3815cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40932/0 (socket says 172.21.15.131:40932) 2024-05-26T22:54:54.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.416+0000 7f3b37400640 1 -- 172.21.15.131:0/1724147396 learned_addr learned my addr 172.21.15.131:0/1724147396 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:54:54.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.416+0000 7f3b36a00640 1 --2- 172.21.15.131:0/1724147396 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b3806f490 0x7f3b38155c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:54.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.416+0000 7f3b36000640 1 --2- 172.21.15.131:0/1724147396 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3b38155840 0x7f3b3814c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:54.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.416+0000 7f3b37400640 1 -- 172.21.15.131:0/1724147396 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3b38155840 msgr2=0x7f3b3814c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:54.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.416+0000 7f3b37400640 1 --2- 172.21.15.131:0/1724147396 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3b38155840 0x7f3b3814c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:54.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.416+0000 7f3b37400640 1 -- 172.21.15.131:0/1724147396 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b3806f490 msgr2=0x7f3b38155c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:54.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.416+0000 7f3b37400640 1 --2- 172.21.15.131:0/1724147396 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b3806f490 0x7f3b38155c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:54.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.416+0000 7f3b37400640 1 -- 172.21.15.131:0/1724147396 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b38158500 con 0x7f3b3815c3d0 2024-05-26T22:54:54.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.417+0000 7f3b37400640 1 --2- 172.21.15.131:0/1724147396 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b3815c3d0 0x7f3b3815cd90 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f3b3000c760 tx=0x7f3b3000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:54.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.417+0000 7f3b2ea00640 1 -- 172.21.15.131:0/1724147396 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b30003da0 con 0x7f3b3815c3d0 2024-05-26T22:54:54.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.417+0000 7f3b2ea00640 1 -- 172.21.15.131:0/1724147396 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b30005070 con 0x7f3b3815c3d0 2024-05-26T22:54:54.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.417+0000 7f3b3d800640 1 -- 172.21.15.131:0/1724147396 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b381587e0 con 0x7f3b3815c3d0 2024-05-26T22:54:54.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.417+0000 7f3b2ea00640 1 -- 172.21.15.131:0/1724147396 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b30005460 con 0x7f3b3815c3d0 2024-05-26T22:54:54.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.417+0000 7f3b3d800640 1 -- 172.21.15.131:0/1724147396 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b38158da0 con 0x7f3b3815c3d0 2024-05-26T22:54:54.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.418+0000 7f3b3d800640 1 -- 172.21.15.131:0/1724147396 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b380ccfd0 con 0x7f3b3815c3d0 2024-05-26T22:54:54.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.419+0000 7f3b2ea00640 1 -- 172.21.15.131:0/1724147396 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3b30005600 con 0x7f3b3815c3d0 2024-05-26T22:54:54.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.420+0000 7f3b2ea00640 1 --2- 172.21.15.131:0/1724147396 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f3b1806be30 0x7f3b1806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:54.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.420+0000 7f3b2ea00640 1 -- 172.21.15.131:0/1724147396 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 4857+0+0 (secure 0 0 0) 0x7f3b3008aa60 con 0x7f3b3815c3d0 2024-05-26T22:54:54.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.420+0000 7f3b36a00640 1 --2- 172.21.15.131:0/1724147396 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f3b1806be30 0x7f3b1806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:54.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.421+0000 7f3b36a00640 1 --2- 172.21.15.131:0/1724147396 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f3b1806be30 0x7f3b1806e2f0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f3b28006fd0 tx=0x7f3b28008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:54.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.425+0000 7f3b2ea00640 1 -- 172.21.15.131:0/1724147396 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f3b30059750 con 0x7f3b3815c3d0 2024-05-26T22:54:54.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.625+0000 7f3b3d800640 1 -- 172.21.15.131:0/1724147396 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f3b380b1bb0 con 0x7f3b3815c3d0 2024-05-26T22:54:54.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.626+0000 7f3b2ea00640 1 -- 172.21.15.131:0/1724147396 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v91) ==== 84+0+65 (secure 0 0 0) 0x7f3b30059050 con 0x7f3b3815c3d0 2024-05-26T22:54:54.625 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:54:54.625 INFO:teuthology.orchestra.run.smithi131.stdout:[ 2024-05-26T22:54:54.626 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:54:54.626 INFO:teuthology.orchestra.run.smithi131.stdout: "poolnum": 1, 2024-05-26T22:54:54.626 INFO:teuthology.orchestra.run.smithi131.stdout: "poolname": ".mgr" 2024-05-26T22:54:54.626 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:54:54.626 INFO:teuthology.orchestra.run.smithi131.stdout:] 2024-05-26T22:54:54.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.628+0000 7f3b3d800640 1 -- 172.21.15.131:0/1724147396 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f3b1806be30 msgr2=0x7f3b1806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:54.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.628+0000 7f3b3d800640 1 --2- 172.21.15.131:0/1724147396 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f3b1806be30 0x7f3b1806e2f0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f3b28006fd0 tx=0x7f3b28008040 comp rx=0 tx=0).stop 2024-05-26T22:54:54.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.628+0000 7f3b3d800640 1 -- 172.21.15.131:0/1724147396 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b3815c3d0 msgr2=0x7f3b3815cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:54.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.628+0000 7f3b3d800640 1 --2- 172.21.15.131:0/1724147396 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b3815c3d0 0x7f3b3815cd90 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f3b3000c760 tx=0x7f3b3000cc20 comp rx=0 tx=0).stop 2024-05-26T22:54:54.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.628+0000 7f3b3d800640 1 -- 172.21.15.131:0/1724147396 shutdown_connections 2024-05-26T22:54:54.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.628+0000 7f3b3d800640 1 --2- 172.21.15.131:0/1724147396 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3b38155840 0x7f3b3814c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:54.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.628+0000 7f3b3d800640 1 --2- 172.21.15.131:0/1724147396 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f3b1806be30 0x7f3b1806e2f0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:54.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.628+0000 7f3b3d800640 1 --2- 172.21.15.131:0/1724147396 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b3815c3d0 0x7f3b3815cd90 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:54.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.628+0000 7f3b3d800640 1 --2- 172.21.15.131:0/1724147396 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b3806f490 0x7f3b38155c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:54.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.628+0000 7f3b3d800640 1 -- 172.21.15.131:0/1724147396 >> 172.21.15.131:0/1724147396 conn(0x7f3b380b1e90 msgr2=0x7f3b3806e850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:54.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.628+0000 7f3b3d800640 1 -- 172.21.15.131:0/1724147396 shutdown_connections 2024-05-26T22:54:54.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.629+0000 7f3b3d800640 1 -- 172.21.15.131:0/1724147396 wait complete. 2024-05-26T22:54:54.647 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-26T22:54:54.647 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-26T22:54:54.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.846+0000 7f6d2d200640 1 Processor -- start 2024-05-26T22:54:54.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.846+0000 7f6d2d200640 1 -- start start 2024-05-26T22:54:54.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.848+0000 7f6d2d200640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f6d2813dee0 0x7f6d2806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:54.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.848+0000 7f6d2d200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6d2815bdd0 0x7f6d280ce7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:54.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.848+0000 7f6d2d200640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d2813e2c0 con 0x7f6d2806da20 2024-05-26T22:54:54.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.848+0000 7f6d2d200640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f6d2815c430 con 0x7f6d2813dee0 2024-05-26T22:54:54.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.848+0000 7f6d2d200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f6d2815eb60 con 0x7f6d2815bdd0 2024-05-26T22:54:54.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.849+0000 7f6d27400640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f6d2813dee0 0x7f6d2806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:54.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.849+0000 7f6d27e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6d2815bdd0 0x7f6d280ce7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:54.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.849+0000 7f6d27400640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f6d2813dee0 0x7f6d2806d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.131:56656/0 (socket says 172.21.15.131:56656) 2024-05-26T22:54:54.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.849+0000 7f6d27e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6d2815bdd0 0x7f6d280ce7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.131:40042/0 (socket says 172.21.15.131:40042) 2024-05-26T22:54:54.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.849+0000 7f6d27400640 1 -- 172.21.15.131:0/137809779 learned_addr learned my addr 172.21.15.131:0/137809779 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:54:54.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.850+0000 7f6d27e00640 1 -- 172.21.15.131:0/137809779 >> v2:172.21.15.187:3300/0 conn(0x7f6d2813dee0 msgr2=0x7f6d2806d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:54.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.850+0000 7f6d27e00640 1 --2- 172.21.15.131:0/137809779 >> v2:172.21.15.187:3300/0 conn(0x7f6d2813dee0 0x7f6d2806d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:54.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.850+0000 7f6d27e00640 1 -- 172.21.15.131:0/137809779 >> v1:172.21.15.187:6789/0 conn(0x7f6d2806da20 legacy=0x7f6d2815b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:54.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.850+0000 7f6d27e00640 1 -- 172.21.15.131:0/137809779 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d2815ee40 con 0x7f6d2815bdd0 2024-05-26T22:54:54.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.851+0000 7f6d27e00640 1 --2- 172.21.15.131:0/137809779 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6d2815bdd0 0x7f6d280ce7b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6d20009d90 tx=0x7f6d2002a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61157c3b8eca0787 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:54.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.852+0000 7f6d26a00640 1 -- 172.21.15.131:0/137809779 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d20007740 con 0x7f6d2815bdd0 2024-05-26T22:54:54.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.852+0000 7f6d26a00640 1 -- 172.21.15.131:0/137809779 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d200078e0 con 0x7f6d2815bdd0 2024-05-26T22:54:54.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.852+0000 7f6d26a00640 1 -- 172.21.15.131:0/137809779 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d20007cb0 con 0x7f6d2815bdd0 2024-05-26T22:54:54.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.853+0000 7f6d2d200640 1 -- 172.21.15.131:0/137809779 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6d2815bdd0 msgr2=0x7f6d280ce7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:54.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.853+0000 7f6d2d200640 1 --2- 172.21.15.131:0/137809779 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6d2815bdd0 0x7f6d280ce7b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6d20009d90 tx=0x7f6d2002a660 comp rx=0 tx=0).stop 2024-05-26T22:54:54.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.853+0000 7f6d2d200640 1 -- 172.21.15.131:0/137809779 shutdown_connections 2024-05-26T22:54:54.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.853+0000 7f6d2d200640 1 --2- 172.21.15.131:0/137809779 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6d2815bdd0 0x7f6d280ce7b0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:54.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.853+0000 7f6d2d200640 1 --2- 172.21.15.131:0/137809779 >> v2:172.21.15.187:3300/0 conn(0x7f6d2813dee0 0x7f6d2806d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:54.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.853+0000 7f6d2d200640 1 -- 172.21.15.131:0/137809779 >> 172.21.15.131:0/137809779 conn(0x7f6d280b5410 msgr2=0x7f6d280b57f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:54.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.853+0000 7f6d2d200640 1 -- 172.21.15.131:0/137809779 shutdown_connections 2024-05-26T22:54:54.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.853+0000 7f6d2d200640 1 -- 172.21.15.131:0/137809779 wait complete. 2024-05-26T22:54:54.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.855+0000 7f6d2d200640 1 Processor -- start 2024-05-26T22:54:54.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.856+0000 7f6d2d200640 1 -- start start 2024-05-26T22:54:54.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.857+0000 7f6d2d200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6d2806da20 0x7f6d2813d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:54.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.857+0000 7f6d2d200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6d2813dee0 0x7f6d280b0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:54.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.857+0000 7f6d2d200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6d2815bdd0 0x7f6d28072d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:54.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.857+0000 7f6d2d200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f6d2815f920 con 0x7f6d2813dee0 2024-05-26T22:54:54.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.857+0000 7f6d2d200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6d2815f400 con 0x7f6d2815bdd0 2024-05-26T22:54:54.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.857+0000 7f6d2d200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f6d280afb20 con 0x7f6d2806da20 2024-05-26T22:54:54.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.857+0000 7f6d1fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6d2813dee0 0x7f6d280b0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:54.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.857+0000 7f6d1fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6d2813dee0 0x7f6d280b0730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40944/0 (socket says 172.21.15.131:40944) 2024-05-26T22:54:54.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.857+0000 7f6d1fe00640 1 -- 172.21.15.131:0/4046818719 learned_addr learned my addr 172.21.15.131:0/4046818719 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:54:54.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.858+0000 7f6d27400640 1 --2- 172.21.15.131:0/4046818719 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6d2806da20 0x7f6d2813d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:54.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.858+0000 7f6d27e00640 1 --2- 172.21.15.131:0/4046818719 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6d2815bdd0 0x7f6d28072d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:54.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.858+0000 7f6d1fe00640 1 -- 172.21.15.131:0/4046818719 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6d2806da20 msgr2=0x7f6d2813d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:54.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.858+0000 7f6d1fe00640 1 --2- 172.21.15.131:0/4046818719 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6d2806da20 0x7f6d2813d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:54.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.858+0000 7f6d1fe00640 1 -- 172.21.15.131:0/4046818719 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6d2815bdd0 msgr2=0x7f6d28072d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:54.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.858+0000 7f6d1fe00640 1 --2- 172.21.15.131:0/4046818719 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6d2815bdd0 0x7f6d28072d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:54.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.858+0000 7f6d1fe00640 1 -- 172.21.15.131:0/4046818719 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d280b25e0 con 0x7f6d2813dee0 2024-05-26T22:54:54.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.858+0000 7f6d1fe00640 1 --2- 172.21.15.131:0/4046818719 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6d2813dee0 0x7f6d280b0730 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f6d1000c450 tx=0x7f6d1000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:54.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.858+0000 7f6d1f400640 1 -- 172.21.15.131:0/4046818719 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d100052d0 con 0x7f6d2813dee0 2024-05-26T22:54:54.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.858+0000 7f6d1f400640 1 -- 172.21.15.131:0/4046818719 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d10005470 con 0x7f6d2813dee0 2024-05-26T22:54:54.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.858+0000 7f6d2d200640 1 -- 172.21.15.131:0/4046818719 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d280b2880 con 0x7f6d2813dee0 2024-05-26T22:54:54.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.858+0000 7f6d1f400640 1 -- 172.21.15.131:0/4046818719 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d10012870 con 0x7f6d2813dee0 2024-05-26T22:54:54.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.858+0000 7f6d2d200640 1 -- 172.21.15.131:0/4046818719 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d280b0050 con 0x7f6d2813dee0 2024-05-26T22:54:54.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.859+0000 7f6d2d200640 1 -- 172.21.15.131:0/4046818719 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d280b0290 con 0x7f6d2813dee0 2024-05-26T22:54:54.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.860+0000 7f6d1f400640 1 -- 172.21.15.131:0/4046818719 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6d10012a10 con 0x7f6d2813dee0 2024-05-26T22:54:54.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.861+0000 7f6d1f400640 1 --2- 172.21.15.131:0/4046818719 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f6d0406be30 0x7f6d0406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:54.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.861+0000 7f6d1f400640 1 -- 172.21.15.131:0/4046818719 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 4857+0+0 (secure 0 0 0) 0x7f6d1008a6b0 con 0x7f6d2813dee0 2024-05-26T22:54:54.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.862+0000 7f6d27400640 1 --2- 172.21.15.131:0/4046818719 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f6d0406be30 0x7f6d0406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:54.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.862+0000 7f6d27400640 1 --2- 172.21.15.131:0/4046818719 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f6d0406be30 0x7f6d0406e2f0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f6d18009aa0 tx=0x7f6d18008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:54.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:54.866+0000 7f6d1f400640 1 -- 172.21.15.131:0/4046818719 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f6d280b0290 con 0x7f6d2813dee0 2024-05-26T22:54:55.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:55.065+0000 7f6d2d200640 1 -- 172.21.15.131:0/4046818719 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f6d2815faa0 con 0x7f6d2813dee0 2024-05-26T22:54:55.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:55.856+0000 7f6d1f400640 1 -- 172.21.15.131:0/4046818719 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v92) ==== 138+0+0 (secure 0 0 0) 0x7f6d10059440 con 0x7f6d2813dee0 2024-05-26T22:54:55.856 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_metadata' created 2024-05-26T22:54:55.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:55.860+0000 7f6d2d200640 1 -- 172.21.15.131:0/4046818719 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f6d0406be30 msgr2=0x7f6d0406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:55.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:55.860+0000 7f6d2d200640 1 --2- 172.21.15.131:0/4046818719 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f6d0406be30 0x7f6d0406e2f0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f6d18009aa0 tx=0x7f6d18008040 comp rx=0 tx=0).stop 2024-05-26T22:54:55.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:55.860+0000 7f6d2d200640 1 -- 172.21.15.131:0/4046818719 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6d2813dee0 msgr2=0x7f6d280b0730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:55.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:55.860+0000 7f6d2d200640 1 --2- 172.21.15.131:0/4046818719 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6d2813dee0 0x7f6d280b0730 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f6d1000c450 tx=0x7f6d1000c910 comp rx=0 tx=0).stop 2024-05-26T22:54:55.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:55.860+0000 7f6d2d200640 1 -- 172.21.15.131:0/4046818719 shutdown_connections 2024-05-26T22:54:55.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:55.860+0000 7f6d2d200640 1 --2- 172.21.15.131:0/4046818719 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f6d0406be30 0x7f6d0406e2f0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:55.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:55.860+0000 7f6d2d200640 1 --2- 172.21.15.131:0/4046818719 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6d2815bdd0 0x7f6d28072d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:55.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:55.860+0000 7f6d2d200640 1 --2- 172.21.15.131:0/4046818719 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6d2813dee0 0x7f6d280b0730 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:55.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:55.860+0000 7f6d2d200640 1 --2- 172.21.15.131:0/4046818719 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6d2806da20 0x7f6d2813d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:55.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:55.860+0000 7f6d2d200640 1 -- 172.21.15.131:0/4046818719 >> 172.21.15.131:0/4046818719 conn(0x7f6d280b5410 msgr2=0x7f6d28137390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:55.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:55.861+0000 7f6d2d200640 1 -- 172.21.15.131:0/4046818719 shutdown_connections 2024-05-26T22:54:55.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:55.861+0000 7f6d2d200640 1 -- 172.21.15.131:0/4046818719 wait complete. 2024-05-26T22:54:55.886 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-26T22:54:56.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:56.076+0000 7f3d8a200640 1 Processor -- start 2024-05-26T22:54:56.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:56.076+0000 7f3d8a200640 1 -- start start 2024-05-26T22:54:56.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:56.078+0000 7f3d8a200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f3d8406f420 0x7f3d840cfd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:56.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:56.078+0000 7f3d8a200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3d840cf590 0x7f3d840d0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:56.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:56.078+0000 7f3d8a200640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d84160180 con 0x7f3d840b5450 2024-05-26T22:54:56.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:56.078+0000 7f3d8a200640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f3d84160d90 con 0x7f3d8406f420 2024-05-26T22:54:56.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:56.078+0000 7f3d8a200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3d84160460 con 0x7f3d840cf590 2024-05-26T22:54:56.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:56.078+0000 7f3d82a00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f3d8406f420 0x7f3d840cfd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:56.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:56.078+0000 7f3d82a00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f3d8406f420 0x7f3d840cfd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40960/0 (socket says 172.21.15.131:40960) 2024-05-26T22:54:56.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:56.078+0000 7f3d82a00640 1 -- 172.21.15.131:0/528700577 learned_addr learned my addr 172.21.15.131:0/528700577 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:54:56.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:56.079+0000 7f3d82a00640 1 -- 172.21.15.131:0/528700577 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3d840cf590 msgr2=0x7f3d840d0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:56.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:56.079+0000 7f3d83e00640 1 --2- 172.21.15.131:0/528700577 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3d840cf590 0x7f3d840d0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:56.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:56.079+0000 7f3d82a00640 1 --2- 172.21.15.131:0/528700577 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3d840cf590 0x7f3d840d0290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:56.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:56.079+0000 7f3d82a00640 1 -- 172.21.15.131:0/528700577 >> v1:172.21.15.187:6789/0 conn(0x7f3d840b5450 legacy=0x7f3d840b43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:56.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:56.079+0000 7f3d82a00640 1 -- 172.21.15.131:0/528700577 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d84160740 con 0x7f3d8406f420 2024-05-26T22:54:56.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:56.079+0000 7f3d83e00640 1 --2- 172.21.15.131:0/528700577 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3d840cf590 0x7f3d840d0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:54:56.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:56.080+0000 7f3d82a00640 1 --2- 172.21.15.131:0/528700577 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3d8406f420 0x7f3d840cfd50 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f3d70009870 tx=0x7f3d7002a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=488a66b4ca915eae server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:56.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:56.080+0000 7f3d82000640 1 -- 172.21.15.131:0/528700577 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d70037070 con 0x7f3d8406f420 2024-05-26T22:54:56.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:56.080+0000 7f3d82000640 1 -- 172.21.15.131:0/528700577 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d70032440 con 0x7f3d8406f420 2024-05-26T22:54:56.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:56.080+0000 7f3d82000640 1 -- 172.21.15.131:0/528700577 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d70032810 con 0x7f3d8406f420 2024-05-26T22:54:56.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:56.080+0000 7f3d8a200640 1 -- 172.21.15.131:0/528700577 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3d8406f420 msgr2=0x7f3d840cfd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:56.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:56.080+0000 7f3d8a200640 1 --2- 172.21.15.131:0/528700577 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3d8406f420 0x7f3d840cfd50 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f3d70009870 tx=0x7f3d7002a200 comp rx=0 tx=0).stop 2024-05-26T22:54:56.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:56.081+0000 7f3d8a200640 1 -- 172.21.15.131:0/528700577 shutdown_connections 2024-05-26T22:54:56.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:56.081+0000 7f3d8a200640 1 --2- 172.21.15.131:0/528700577 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3d840cf590 0x7f3d840d0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:56.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:56.081+0000 7f3d8a200640 1 --2- 172.21.15.131:0/528700577 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3d8406f420 0x7f3d840cfd50 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:56.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:56.081+0000 7f3d8a200640 1 -- 172.21.15.131:0/528700577 >> 172.21.15.131:0/528700577 conn(0x7f3d8406bc90 msgr2=0x7f3d840b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:56.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:56.081+0000 7f3d8a200640 1 -- 172.21.15.131:0/528700577 shutdown_connections 2024-05-26T22:54:56.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:56.081+0000 7f3d8a200640 1 -- 172.21.15.131:0/528700577 wait complete. 2024-05-26T22:54:56.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:56.082+0000 7f3d8a200640 1 Processor -- start 2024-05-26T22:54:56.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:56.082+0000 7f3d8a200640 1 -- start start 2024-05-26T22:54:56.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:56.083+0000 7f3d8a200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3d8406f420 0x7f3d8414feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:56.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:56.083+0000 7f3d8a200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3d840b5450 0x7f3d840ae020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:56.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:56.083+0000 7f3d8a200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3d840cf590 0x7f3d840ae560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:56.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:56.083+0000 7f3d8a200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f3d8415fde0 con 0x7f3d840b5450 2024-05-26T22:54:56.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:56.083+0000 7f3d8a200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3d840b5830 con 0x7f3d8406f420 2024-05-26T22:54:56.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:56.083+0000 7f3d8a200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3d8415ff60 con 0x7f3d840cf590 2024-05-26T22:54:56.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:56.084+0000 7f3d82a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3d840b5450 0x7f3d840ae020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:56.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:56.084+0000 7f3d82a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3d840b5450 0x7f3d840ae020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40964/0 (socket says 172.21.15.131:40964) 2024-05-26T22:54:56.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:56.084+0000 7f3d82a00640 1 -- 172.21.15.131:0/2829469329 learned_addr learned my addr 172.21.15.131:0/2829469329 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:54:56.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:56.084+0000 7f3d83400640 1 --2- 172.21.15.131:0/2829469329 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3d8406f420 0x7f3d8414feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:56.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:56.084+0000 7f3d83e00640 1 --2- 172.21.15.131:0/2829469329 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3d840cf590 0x7f3d840ae560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:56.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:56.084+0000 7f3d82a00640 1 -- 172.21.15.131:0/2829469329 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3d840cf590 msgr2=0x7f3d840ae560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:56.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:56.084+0000 7f3d82a00640 1 --2- 172.21.15.131:0/2829469329 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3d840cf590 0x7f3d840ae560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:56.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:56.084+0000 7f3d82a00640 1 -- 172.21.15.131:0/2829469329 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3d8406f420 msgr2=0x7f3d8414feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:56.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:56.084+0000 7f3d82a00640 1 --2- 172.21.15.131:0/2829469329 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3d8406f420 0x7f3d8414feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:56.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:56.084+0000 7f3d82a00640 1 -- 172.21.15.131:0/2829469329 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d84152f90 con 0x7f3d840b5450 2024-05-26T22:54:56.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:56.084+0000 7f3d83400640 1 --2- 172.21.15.131:0/2829469329 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3d8406f420 0x7f3d8414feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:54:56.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:56.084+0000 7f3d82a00640 1 --2- 172.21.15.131:0/2829469329 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3d840b5450 0x7f3d840ae020 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f3d70009870 tx=0x7f3d700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:56.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:56.085+0000 7f3d7b400640 1 -- 172.21.15.131:0/2829469329 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d70041020 con 0x7f3d840b5450 2024-05-26T22:54:56.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:56.085+0000 7f3d7b400640 1 -- 172.21.15.131:0/2829469329 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d70030970 con 0x7f3d840b5450 2024-05-26T22:54:56.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:56.085+0000 7f3d8a200640 1 -- 172.21.15.131:0/2829469329 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d84153270 con 0x7f3d840b5450 2024-05-26T22:54:56.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:56.085+0000 7f3d7b400640 1 -- 172.21.15.131:0/2829469329 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d70037040 con 0x7f3d840b5450 2024-05-26T22:54:56.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:56.085+0000 7f3d8a200640 1 -- 172.21.15.131:0/2829469329 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d841503f0 con 0x7f3d840b5450 2024-05-26T22:54:56.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:56.086+0000 7f3d8a200640 1 -- 172.21.15.131:0/2829469329 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d84074e90 con 0x7f3d840b5450 2024-05-26T22:54:56.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:56.087+0000 7f3d7b400640 1 -- 172.21.15.131:0/2829469329 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3d70044020 con 0x7f3d840b5450 2024-05-26T22:54:56.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:56.088+0000 7f3d7b400640 1 --2- 172.21.15.131:0/2829469329 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f3d6406be60 0x7f3d6406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:56.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:56.088+0000 7f3d7b400640 1 -- 172.21.15.131:0/2829469329 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5227+0+0 (secure 0 0 0) 0x7f3d700ab630 con 0x7f3d840b5450 2024-05-26T22:54:56.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:56.088+0000 7f3d83400640 1 --2- 172.21.15.131:0/2829469329 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f3d6406be60 0x7f3d6406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:56.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:56.089+0000 7f3d83400640 1 --2- 172.21.15.131:0/2829469329 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f3d6406be60 0x7f3d6406e320 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f3d841f7a60 tx=0x7f3d74009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:56.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:56.093+0000 7f3d7b400640 1 -- 172.21.15.131:0/2829469329 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f3d7004b030 con 0x7f3d840b5450 2024-05-26T22:54:56.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:56.291+0000 7f3d8a200640 1 -- 172.21.15.131:0/2829469329 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f3d840af920 con 0x7f3d840b5450 2024-05-26T22:54:56.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:56.863+0000 7f3d7b400640 1 -- 172.21.15.131:0/2829469329 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v93) ==== 170+0+0 (secure 0 0 0) 0x7f3d7007a1b0 con 0x7f3d840b5450 2024-05-26T22:54:56.863 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_data' created 2024-05-26T22:54:56.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:56.867+0000 7f3d8a200640 1 -- 172.21.15.131:0/2829469329 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f3d6406be60 msgr2=0x7f3d6406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:56.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:56.867+0000 7f3d8a200640 1 --2- 172.21.15.131:0/2829469329 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f3d6406be60 0x7f3d6406e320 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f3d841f7a60 tx=0x7f3d74009040 comp rx=0 tx=0).stop 2024-05-26T22:54:56.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:56.867+0000 7f3d8a200640 1 -- 172.21.15.131:0/2829469329 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3d840b5450 msgr2=0x7f3d840ae020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:56.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:56.867+0000 7f3d8a200640 1 --2- 172.21.15.131:0/2829469329 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3d840b5450 0x7f3d840ae020 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f3d70009870 tx=0x7f3d700046c0 comp rx=0 tx=0).stop 2024-05-26T22:54:56.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:56.867+0000 7f3d8a200640 1 -- 172.21.15.131:0/2829469329 shutdown_connections 2024-05-26T22:54:56.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:56.867+0000 7f3d8a200640 1 --2- 172.21.15.131:0/2829469329 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3d840cf590 0x7f3d840ae560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:56.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:56.867+0000 7f3d8a200640 1 --2- 172.21.15.131:0/2829469329 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f3d6406be60 0x7f3d6406e320 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:56.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:56.867+0000 7f3d8a200640 1 --2- 172.21.15.131:0/2829469329 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3d840b5450 0x7f3d840ae020 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:56.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:56.867+0000 7f3d8a200640 1 --2- 172.21.15.131:0/2829469329 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3d8406f420 0x7f3d8414feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:56.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:56.867+0000 7f3d8a200640 1 -- 172.21.15.131:0/2829469329 >> 172.21.15.131:0/2829469329 conn(0x7f3d8406bc90 msgr2=0x7f3d840cc100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:56.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:56.867+0000 7f3d8a200640 1 -- 172.21.15.131:0/2829469329 shutdown_connections 2024-05-26T22:54:56.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:56.867+0000 7f3d8a200640 1 -- 172.21.15.131:0/2829469329 wait complete. 2024-05-26T22:54:56.904 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-26T22:54:57.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:57.083+0000 7f9c33000640 1 Processor -- start 2024-05-26T22:54:57.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:57.083+0000 7f9c33000640 1 -- start start 2024-05-26T22:54:57.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:57.084+0000 7f9c33000640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f9c2c0b5450 0x7f9c2c0b43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:57.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:57.084+0000 7f9c33000640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c2c0b4900 con 0x7f9c2c0d0460 2024-05-26T22:54:57.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:57.085+0000 7f9c33000640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c2c160180 con 0x7f9c2c06f420 2024-05-26T22:54:57.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:57.085+0000 7f9c33000640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f9c2c160d90 con 0x7f9c2c0b5450 2024-05-26T22:54:57.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:57.085+0000 7f9c31000640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f9c2c0d0460 0x7f9c2c06d600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:54504/0 (socket says 172.21.15.131:54504) 2024-05-26T22:54:57.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:57.085+0000 7f9c31000640 1 -- 172.21.15.131:0/1712411085 learned_addr learned my addr 172.21.15.131:0/1712411085 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:54:57.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:57.085+0000 7f9c2be00640 1 --2- 172.21.15.131:0/1712411085 >> v2:172.21.15.187:3300/0 conn(0x7f9c2c0b5450 0x7f9c2c0b43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:57.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:57.085+0000 7f9c2aa00640 1 -- 172.21.15.131:0/1712411085 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2847290006 0 0) 0x7f9c2c0b4900 con 0x7f9c2c0d0460 2024-05-26T22:54:57.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:57.085+0000 7f9c2aa00640 1 -- 172.21.15.131:0/1712411085 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9c10003520 con 0x7f9c2c0d0460 2024-05-26T22:54:57.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:57.086+0000 7f9c2aa00640 1 -- 172.21.15.131:0/1712411085 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2326763609 0 0) 0x7f9c10003520 con 0x7f9c2c0d0460 2024-05-26T22:54:57.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:57.086+0000 7f9c2aa00640 1 -- 172.21.15.131:0/1712411085 >> v2:172.21.15.187:3300/0 conn(0x7f9c2c0b5450 msgr2=0x7f9c2c0b43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:57.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:57.086+0000 7f9c2aa00640 1 --2- 172.21.15.131:0/1712411085 >> v2:172.21.15.187:3300/0 conn(0x7f9c2c0b5450 0x7f9c2c0b43c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:57.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:57.086+0000 7f9c2aa00640 1 -- 172.21.15.131:0/1712411085 >> v1:172.21.15.187:6789/0 conn(0x7f9c2c06f420 legacy=0x7f9c2c0cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:57.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:57.086+0000 7f9c2aa00640 1 -- 172.21.15.131:0/1712411085 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c2c160460 con 0x7f9c2c0d0460 2024-05-26T22:54:57.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:57.086+0000 7f9c2aa00640 1 -- 172.21.15.131:0/1712411085 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7f9c240019f0 con 0x7f9c2c0d0460 2024-05-26T22:54:57.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:57.086+0000 7f9c2aa00640 1 -- 172.21.15.131:0/1712411085 >> v1:172.21.15.131:6789/0 conn(0x7f9c2c0d0460 legacy=0x7f9c2c06d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:57.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:57.086+0000 7f9c2aa00640 1 --2- 172.21.15.131:0/1712411085 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c100039b0 0x7f9c10003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:57.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:57.086+0000 7f9c2aa00640 1 --2- 172.21.15.131:0/1712411085 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9c10004800 0x7f9c10004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:57.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:57.086+0000 7f9c2aa00640 1 --2- 172.21.15.131:0/1712411085 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9c100051c0 0x7f9c1000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:57.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:57.086+0000 7f9c2aa00640 1 -- 172.21.15.131:0/1712411085 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f9c2c160d90 con 0x7f9c100039b0 2024-05-26T22:54:57.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:57.086+0000 7f9c2aa00640 1 -- 172.21.15.131:0/1712411085 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9c1000c060 con 0x7f9c100051c0 2024-05-26T22:54:57.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:57.086+0000 7f9c2aa00640 1 -- 172.21.15.131:0/1712411085 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9c1000c260 con 0x7f9c10004800 2024-05-26T22:54:57.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:57.086+0000 7f9c2be00640 1 --2- 172.21.15.131:0/1712411085 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c100039b0 0x7f9c10003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:57.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:57.087+0000 7f9c2be00640 1 -- 172.21.15.131:0/1712411085 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9c10004800 msgr2=0x7f9c10004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:57.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:57.087+0000 7f9c2be00640 1 --2- 172.21.15.131:0/1712411085 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9c10004800 0x7f9c10004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:57.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:57.087+0000 7f9c2be00640 1 -- 172.21.15.131:0/1712411085 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9c100051c0 msgr2=0x7f9c1000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:57.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:57.087+0000 7f9c2be00640 1 --2- 172.21.15.131:0/1712411085 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9c100051c0 0x7f9c1000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:57.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:57.087+0000 7f9c2be00640 1 -- 172.21.15.131:0/1712411085 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c10003520 con 0x7f9c100039b0 2024-05-26T22:54:57.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:57.087+0000 7f9c31000640 1 -- 172.21.15.131:0/1712411085 reap_dead start 2024-05-26T22:54:57.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:57.088+0000 7f9c2be00640 1 --2- 172.21.15.131:0/1712411085 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c100039b0 0x7f9c10003dd0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c009ac0 tx=0x7f9c10005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeb4fb9c9d1597ba server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:57.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:57.088+0000 7f9c2aa00640 1 -- 172.21.15.131:0/1712411085 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c1c036070 con 0x7f9c100039b0 2024-05-26T22:54:57.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:57.088+0000 7f9c2aa00640 1 -- 172.21.15.131:0/1712411085 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c1c031400 con 0x7f9c100039b0 2024-05-26T22:54:57.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:57.088+0000 7f9c2aa00640 1 -- 172.21.15.131:0/1712411085 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c1c009690 con 0x7f9c100039b0 2024-05-26T22:54:57.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:57.088+0000 7f9c33000640 1 -- 172.21.15.131:0/1712411085 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c100039b0 msgr2=0x7f9c10003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:57.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:57.088+0000 7f9c33000640 1 --2- 172.21.15.131:0/1712411085 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c100039b0 0x7f9c10003dd0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c009ac0 tx=0x7f9c10005830 comp rx=0 tx=0).stop 2024-05-26T22:54:57.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:57.089+0000 7f9c33000640 1 -- 172.21.15.131:0/1712411085 shutdown_connections 2024-05-26T22:54:57.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:57.089+0000 7f9c33000640 1 --2- 172.21.15.131:0/1712411085 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c100039b0 0x7f9c10003dd0 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:57.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:57.089+0000 7f9c33000640 1 -- 172.21.15.131:0/1712411085 >> 172.21.15.131:0/1712411085 conn(0x7f9c2c06bc90 msgr2=0x7f9c2c0b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:57.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:57.089+0000 7f9c33000640 1 -- 172.21.15.131:0/1712411085 shutdown_connections 2024-05-26T22:54:57.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:57.089+0000 7f9c33000640 1 -- 172.21.15.131:0/1712411085 wait complete. 2024-05-26T22:54:57.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:57.090+0000 7f9c33000640 1 Processor -- start 2024-05-26T22:54:57.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:57.091+0000 7f9c33000640 1 -- start start 2024-05-26T22:54:57.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:57.091+0000 7f9c33000640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9c100039b0 0x7f9c2c1515a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:57.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:57.091+0000 7f9c33000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9c2c159620 0x7f9c2c159aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:57.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:57.092+0000 7f9c33000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c2c0c9290 0x7f9c2c0c9710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:57.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:57.092+0000 7f9c33000640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f9c2c0b5830 con 0x7f9c2c0c9290 2024-05-26T22:54:57.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:57.092+0000 7f9c33000640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9c2c160a20 con 0x7f9c100039b0 2024-05-26T22:54:57.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:57.092+0000 7f9c33000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9c2c15fde0 con 0x7f9c2c159620 2024-05-26T22:54:57.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:57.092+0000 7f9c31000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c2c0c9290 0x7f9c2c0c9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:57.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:57.092+0000 7f9c31000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c2c0c9290 0x7f9c2c0c9710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40986/0 (socket says 172.21.15.131:40986) 2024-05-26T22:54:57.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:57.092+0000 7f9c31000640 1 -- 172.21.15.131:0/119653950 learned_addr learned my addr 172.21.15.131:0/119653950 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:54:57.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:57.092+0000 7f9c2be00640 1 --2- 172.21.15.131:0/119653950 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9c100039b0 0x7f9c2c1515a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:57.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:57.092+0000 7f9c2b400640 1 --2- 172.21.15.131:0/119653950 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9c2c159620 0x7f9c2c159aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:57.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:57.092+0000 7f9c31000640 1 -- 172.21.15.131:0/119653950 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9c2c159620 msgr2=0x7f9c2c159aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:57.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:57.092+0000 7f9c31000640 1 --2- 172.21.15.131:0/119653950 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9c2c159620 0x7f9c2c159aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:57.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:57.092+0000 7f9c31000640 1 -- 172.21.15.131:0/119653950 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9c100039b0 msgr2=0x7f9c2c1515a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:57.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:57.092+0000 7f9c31000640 1 --2- 172.21.15.131:0/119653950 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9c100039b0 0x7f9c2c1515a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:57.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:57.092+0000 7f9c31000640 1 -- 172.21.15.131:0/119653950 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c2c0c9c50 con 0x7f9c2c0c9290 2024-05-26T22:54:57.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:57.092+0000 7f9c2b400640 1 --2- 172.21.15.131:0/119653950 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9c2c159620 0x7f9c2c159aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:54:57.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:57.093+0000 7f9c31000640 1 --2- 172.21.15.131:0/119653950 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c2c0c9290 0x7f9c2c0c9710 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c160d00 tx=0x7f9c2400cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:57.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:57.093+0000 7f9c23e00640 1 -- 172.21.15.131:0/119653950 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c2400ce50 con 0x7f9c2c0c9290 2024-05-26T22:54:57.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:57.093+0000 7f9c23e00640 1 -- 172.21.15.131:0/119653950 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c2400a5b0 con 0x7f9c2c0c9290 2024-05-26T22:54:57.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:57.093+0000 7f9c33000640 1 -- 172.21.15.131:0/119653950 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c2c151ae0 con 0x7f9c2c0c9290 2024-05-26T22:54:57.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:57.093+0000 7f9c23e00640 1 -- 172.21.15.131:0/119653950 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c2400a9a0 con 0x7f9c2c0c9290 2024-05-26T22:54:57.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:57.093+0000 7f9c33000640 1 -- 172.21.15.131:0/119653950 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c2c0cb8d0 con 0x7f9c2c0c9290 2024-05-26T22:54:57.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:57.094+0000 7f9c33000640 1 -- 172.21.15.131:0/119653950 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c2c074e90 con 0x7f9c2c0c9290 2024-05-26T22:54:57.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:57.095+0000 7f9c23e00640 1 -- 172.21.15.131:0/119653950 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9c2400ab40 con 0x7f9c2c0c9290 2024-05-26T22:54:57.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:57.096+0000 7f9c23e00640 1 --2- 172.21.15.131:0/119653950 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9c0c06be60 0x7f9c0c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:57.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:57.096+0000 7f9c23e00640 1 -- 172.21.15.131:0/119653950 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5609+0+0 (secure 0 0 0) 0x7f9c2408aaa0 con 0x7f9c2c0c9290 2024-05-26T22:54:57.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:57.096+0000 7f9c2be00640 1 --2- 172.21.15.131:0/119653950 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9c0c06be60 0x7f9c0c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:57.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:57.097+0000 7f9c2be00640 1 --2- 172.21.15.131:0/119653950 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9c0c06be60 0x7f9c0c06e320 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c033000 tx=0x7f9c1c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:57.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:57.101+0000 7f9c23e00640 1 -- 172.21.15.131:0/119653950 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f9c240594a0 con 0x7f9c2c0c9290 2024-05-26T22:54:57.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:57.325+0000 7f9c33000640 1 -- 172.21.15.131:0/119653950 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f9c2c0cc060 con 0x7f9c2c0c9290 2024-05-26T22:54:57.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:57.904+0000 7f9c23e00640 1 -- 172.21.15.131:0/119653950 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 14 and data pool 15 v90) ==== 174+0+0 (secure 0 0 0) 0x7f9c2c0cc060 con 0x7f9c2c0c9290 2024-05-26T22:54:57.910 INFO:teuthology.orchestra.run.smithi131.stderr:new fs with metadata pool 14 and data pool 15 2024-05-26T22:54:57.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:57.916+0000 7f9c33000640 1 -- 172.21.15.131:0/119653950 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9c0c06be60 msgr2=0x7f9c0c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:57.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:57.916+0000 7f9c33000640 1 --2- 172.21.15.131:0/119653950 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9c0c06be60 0x7f9c0c06e320 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c033000 tx=0x7f9c1c008030 comp rx=0 tx=0).stop 2024-05-26T22:54:57.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:57.916+0000 7f9c33000640 1 -- 172.21.15.131:0/119653950 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c2c0c9290 msgr2=0x7f9c2c0c9710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:57.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:57.916+0000 7f9c33000640 1 --2- 172.21.15.131:0/119653950 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c2c0c9290 0x7f9c2c0c9710 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c160d00 tx=0x7f9c2400cc00 comp rx=0 tx=0).stop 2024-05-26T22:54:57.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:57.917+0000 7f9c33000640 1 -- 172.21.15.131:0/119653950 shutdown_connections 2024-05-26T22:54:57.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:57.917+0000 7f9c33000640 1 --2- 172.21.15.131:0/119653950 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9c2c159620 0x7f9c2c159aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:57.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:57.917+0000 7f9c33000640 1 --2- 172.21.15.131:0/119653950 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9c0c06be60 0x7f9c0c06e320 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:57.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:57.917+0000 7f9c33000640 1 --2- 172.21.15.131:0/119653950 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c2c0c9290 0x7f9c2c0c9710 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:57.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:57.917+0000 7f9c33000640 1 --2- 172.21.15.131:0/119653950 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9c100039b0 0x7f9c2c1515a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:57.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:57.917+0000 7f9c33000640 1 -- 172.21.15.131:0/119653950 >> 172.21.15.131:0/119653950 conn(0x7f9c2c06bc90 msgr2=0x7f9c2c15dfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:57.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:57.918+0000 7f9c33000640 1 -- 172.21.15.131:0/119653950 shutdown_connections 2024-05-26T22:54:57.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:57.918+0000 7f9c33000640 1 -- 172.21.15.131:0/119653950 wait complete. 2024-05-26T22:54:57.952 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-26T22:54:58.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.116+0000 7ff846400640 1 Processor -- start 2024-05-26T22:54:58.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.116+0000 7ff846400640 1 -- start start 2024-05-26T22:54:58.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.118+0000 7ff846400640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7ff84015e570 0x7ff8400d4c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:58.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.118+0000 7ff846400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff8400d19b0 0x7ff8400d2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:58.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.118+0000 7ff846400640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8400b37e0 con 0x7ff8400b2f60 2024-05-26T22:54:58.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.118+0000 7ff846400640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7ff8400d51b0 con 0x7ff84015e570 2024-05-26T22:54:58.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.118+0000 7ff846400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff8400d47e0 con 0x7ff8400d19b0 2024-05-26T22:54:58.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.118+0000 7ff83ea00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7ff8400b2f60 0x7ff8400d12a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:54516/0 (socket says 172.21.15.131:54516) 2024-05-26T22:54:58.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.118+0000 7ff83ea00640 1 -- 172.21.15.131:0/1238770148 learned_addr learned my addr 172.21.15.131:0/1238770148 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:54:58.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.118+0000 7ff83e000640 1 -- 172.21.15.131:0/1238770148 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1022784274 0 0) 0x7ff82c001110 con 0x7ff8400b2f60 2024-05-26T22:54:58.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.118+0000 7ff83f400640 1 --2- 172.21.15.131:0/1238770148 >> v2:172.21.15.187:3300/0 conn(0x7ff84015e570 0x7ff8400d4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:58.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.118+0000 7ff83e000640 1 -- 172.21.15.131:0/1238770148 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff824003520 con 0x7ff8400b2f60 2024-05-26T22:54:58.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.118+0000 7ff83fe00640 1 --2- 172.21.15.131:0/1238770148 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff8400d19b0 0x7ff8400d2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:58.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.119+0000 7ff83e000640 1 -- 172.21.15.131:0/1238770148 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3887114646 0 0) 0x7ff824003520 con 0x7ff8400b2f60 2024-05-26T22:54:58.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.119+0000 7ff83e000640 1 -- 172.21.15.131:0/1238770148 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff8400d19b0 msgr2=0x7ff8400d2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:58.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.119+0000 7ff83e000640 1 --2- 172.21.15.131:0/1238770148 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff8400d19b0 0x7ff8400d2590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:58.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.119+0000 7ff83e000640 1 -- 172.21.15.131:0/1238770148 >> v2:172.21.15.187:3300/0 conn(0x7ff84015e570 msgr2=0x7ff8400d4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:58.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.119+0000 7ff83e000640 1 --2- 172.21.15.131:0/1238770148 >> v2:172.21.15.187:3300/0 conn(0x7ff84015e570 0x7ff8400d4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:58.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.119+0000 7ff83e000640 1 -- 172.21.15.131:0/1238770148 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8400b9cf0 con 0x7ff8400b2f60 2024-05-26T22:54:58.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.119+0000 7ff83e000640 1 -- 172.21.15.131:0/1238770148 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7ff82c001840 con 0x7ff8400b2f60 2024-05-26T22:54:58.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.119+0000 7ff83e000640 1 -- 172.21.15.131:0/1238770148 >> v1:172.21.15.131:6789/0 conn(0x7ff8400b2f60 legacy=0x7ff8400d12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:58.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.119+0000 7ff83e000640 1 --2- 172.21.15.131:0/1238770148 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff8240039b0 0x7ff824003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:58.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.119+0000 7ff83e000640 1 --2- 172.21.15.131:0/1238770148 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff8400d19b0 0x7ff824004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:58.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.119+0000 7ff83e000640 1 --2- 172.21.15.131:0/1238770148 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff824004d40 0x7ff8240051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:58.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.119+0000 7ff83e000640 1 -- 172.21.15.131:0/1238770148 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff8400d51b0 con 0x7ff8240039b0 2024-05-26T22:54:58.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.119+0000 7ff83e000640 1 -- 172.21.15.131:0/1238770148 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff8400d47e0 con 0x7ff824004d40 2024-05-26T22:54:58.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.119+0000 7ff83e000640 1 -- 172.21.15.131:0/1238770148 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff82400baf0 con 0x7ff8400d19b0 2024-05-26T22:54:58.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.119+0000 7ff83f400640 1 --2- 172.21.15.131:0/1238770148 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff8240039b0 0x7ff824003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:58.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.120+0000 7ff83f400640 1 -- 172.21.15.131:0/1238770148 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff8400d19b0 msgr2=0x7ff824004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:58.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.120+0000 7ff83f400640 1 --2- 172.21.15.131:0/1238770148 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff8400d19b0 0x7ff824004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:58.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.120+0000 7ff83f400640 1 -- 172.21.15.131:0/1238770148 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff824004d40 msgr2=0x7ff8240051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:58.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.120+0000 7ff83f400640 1 --2- 172.21.15.131:0/1238770148 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff824004d40 0x7ff8240051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:58.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.120+0000 7ff83f400640 1 -- 172.21.15.131:0/1238770148 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff824003520 con 0x7ff8240039b0 2024-05-26T22:54:58.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.121+0000 7ff83f400640 1 --2- 172.21.15.131:0/1238770148 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff8240039b0 0x7ff824003dd0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7ff83000ba30 tx=0x7ff824005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7683aa2b2224f33d server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:58.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.121+0000 7ff83e000640 1 -- 172.21.15.131:0/1238770148 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff830030070 con 0x7ff8240039b0 2024-05-26T22:54:58.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.121+0000 7ff83e000640 1 -- 172.21.15.131:0/1238770148 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff83002ae10 con 0x7ff8240039b0 2024-05-26T22:54:58.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.121+0000 7ff83e000640 1 -- 172.21.15.131:0/1238770148 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff83000b600 con 0x7ff8240039b0 2024-05-26T22:54:58.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.121+0000 7ff846400640 1 -- 172.21.15.131:0/1238770148 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff8240039b0 msgr2=0x7ff824003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:58.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.121+0000 7ff846400640 1 --2- 172.21.15.131:0/1238770148 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff8240039b0 0x7ff824003dd0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7ff83000ba30 tx=0x7ff824005830 comp rx=0 tx=0).stop 2024-05-26T22:54:58.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.121+0000 7ff83fe00640 1 -- 172.21.15.131:0/1238770148 reap_dead start 2024-05-26T22:54:58.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.121+0000 7ff846400640 1 -- 172.21.15.131:0/1238770148 shutdown_connections 2024-05-26T22:54:58.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.121+0000 7ff846400640 1 -- 172.21.15.131:0/1238770148 >> 172.21.15.131:0/1238770148 conn(0x7ff840072e90 msgr2=0x7ff8400732a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:58.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.121+0000 7ff846400640 1 -- 172.21.15.131:0/1238770148 shutdown_connections 2024-05-26T22:54:58.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.121+0000 7ff846400640 1 -- 172.21.15.131:0/1238770148 wait complete. 2024-05-26T22:54:58.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.123+0000 7ff846400640 1 Processor -- start 2024-05-26T22:54:58.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.123+0000 7ff846400640 1 -- start start 2024-05-26T22:54:58.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.124+0000 7ff846400640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff8240039b0 0x7ff840148260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:58.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.124+0000 7ff846400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff8400cbd80 0x7ff8401f4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:58.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.124+0000 7ff846400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff8400cc200 0x7ff8401f4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:58.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.124+0000 7ff846400640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff8400d1ea0 con 0x7ff8400cc200 2024-05-26T22:54:58.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.124+0000 7ff846400640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff8400d4470 con 0x7ff8240039b0 2024-05-26T22:54:58.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.124+0000 7ff846400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff8400d2020 con 0x7ff8400cbd80 2024-05-26T22:54:58.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.124+0000 7ff83fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff8400cc200 0x7ff8401f4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:58.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.124+0000 7ff83fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff8400cc200 0x7ff8401f4bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41002/0 (socket says 172.21.15.131:41002) 2024-05-26T22:54:58.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.124+0000 7ff83fe00640 1 -- 172.21.15.131:0/631184854 learned_addr learned my addr 172.21.15.131:0/631184854 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:54:58.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.125+0000 7ff83fe00640 1 -- 172.21.15.131:0/631184854 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff8400cbd80 msgr2=0x7ff8401f4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:58.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.125+0000 7ff83fe00640 1 --2- 172.21.15.131:0/631184854 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff8400cbd80 0x7ff8401f4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:58.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.125+0000 7ff83fe00640 1 -- 172.21.15.131:0/631184854 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff8240039b0 msgr2=0x7ff840148260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:58.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.125+0000 7ff83fe00640 1 --2- 172.21.15.131:0/631184854 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff8240039b0 0x7ff840148260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:58.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.125+0000 7ff83fe00640 1 -- 172.21.15.131:0/631184854 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff84015ed30 con 0x7ff8400cc200 2024-05-26T22:54:58.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.125+0000 7ff83fe00640 1 --2- 172.21.15.131:0/631184854 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff8400cc200 0x7ff8401f4bd0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7ff84006d360 tx=0x7ff83800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:58.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.125+0000 7ff837400640 1 -- 172.21.15.131:0/631184854 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff83800ed90 con 0x7ff8400cc200 2024-05-26T22:54:58.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.125+0000 7ff837400640 1 -- 172.21.15.131:0/631184854 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff838019070 con 0x7ff8400cc200 2024-05-26T22:54:58.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.125+0000 7ff837400640 1 -- 172.21.15.131:0/631184854 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff838014660 con 0x7ff8400cc200 2024-05-26T22:54:58.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.125+0000 7ff846400640 1 -- 172.21.15.131:0/631184854 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff84015f010 con 0x7ff8400cc200 2024-05-26T22:54:58.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.125+0000 7ff846400640 1 -- 172.21.15.131:0/631184854 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff84015f5d0 con 0x7ff8400cc200 2024-05-26T22:54:58.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.127+0000 7ff846400640 1 -- 172.21.15.131:0/631184854 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8400694e0 con 0x7ff8400cc200 2024-05-26T22:54:58.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.127+0000 7ff837400640 1 -- 172.21.15.131:0/631184854 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff838014800 con 0x7ff8400cc200 2024-05-26T22:54:58.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.129+0000 7ff837400640 1 --2- 172.21.15.131:0/631184854 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ff82006be30 0x7ff82006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:58.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.129+0000 7ff837400640 1 -- 172.21.15.131:0/631184854 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5709+0+0 (secure 0 0 0) 0x7ff83808b3a0 con 0x7ff8400cc200 2024-05-26T22:54:58.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.129+0000 7ff83f400640 1 --2- 172.21.15.131:0/631184854 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ff82006be30 0x7ff82006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:58.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.130+0000 7ff83f400640 1 --2- 172.21.15.131:0/631184854 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ff82006be30 0x7ff82006e2f0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7ff83000ba00 tx=0x7ff83002c000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:58.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.134+0000 7ff837400640 1 -- 172.21.15.131:0/631184854 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7ff838059cb0 con 0x7ff8400cc200 2024-05-26T22:54:58.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.330+0000 7ff846400640 1 -- 172.21.15.131:0/631184854 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff840073790 con 0x7ff8400cc200 2024-05-26T22:54:58.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.333+0000 7ff837400640 1 -- 172.21.15.131:0/631184854 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+16645 (secure 0 0 0) 0x7ff8380595b0 con 0x7ff8400cc200 2024-05-26T22:54:58.333 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:54:58.333 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":94,"fsid":"46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f","created":"2024-05-26T22:47:57.288905+0000","modified":"2024-05-26T22:54:57.327260+0000","last_up_change":"2024-05-26T22:48:09.602207+0000","last_in_change":"2024-05-26T22:48:00.687341+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-26T22:48:10.463339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-05-26T22:54:55.120321+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-05-26T22:54:56.346430+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"814eaf62-a18d-494b-8e27-fd8fb88adea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6801","nonce":2350940051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6803","nonce":2350940051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6807","nonce":2350940051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6805","nonce":2350940051}]},"public_addr":"172.21.15.131:6801/2350940051","cluster_addr":"172.21.15.131:6803/2350940051","heartbeat_back_addr":"172.21.15.131:6807/2350940051","heartbeat_front_addr":"172.21.15.131:6805/2350940051","state":["exists","up"]},{"osd":1,"uuid":"42e66287-f6c6-4e26-af3c-6dfaa3536d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6825","nonce":1525528247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6827","nonce":1525528247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6831","nonce":1525528247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6829","nonce":1525528247}]},"public_addr":"172.21.15.131:6825/1525528247","cluster_addr":"172.21.15.131:6827/1525528247","heartbeat_back_addr":"172.21.15.131:6831/1525528247","heartbeat_front_addr":"172.21.15.131:6829/1525528247","state":["exists","up"]},{"osd":2,"uuid":"0da34673-04a2-4fd0-b57a-0ff2b679b043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6817","nonce":3627842210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6819","nonce":3627842210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6823","nonce":3627842210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6821","nonce":3627842210}]},"public_addr":"172.21.15.131:6817/3627842210","cluster_addr":"172.21.15.131:6819/3627842210","heartbeat_back_addr":"172.21.15.131:6823/3627842210","heartbeat_front_addr":"172.21.15.131:6821/3627842210","state":["exists","up"]},{"osd":3,"uuid":"cf117a27-d6b2-43d3-ae52-b47988bf8eed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6809","nonce":350790591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6811","nonce":350790591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6815","nonce":350790591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6813","nonce":350790591}]},"public_addr":"172.21.15.131:6809/350790591","cluster_addr":"172.21.15.131:6811/350790591","heartbeat_back_addr":"172.21.15.131:6815/350790591","heartbeat_front_addr":"172.21.15.131:6813/350790591","state":["exists","up"]},{"osd":4,"uuid":"a092ccfc-be89-4a84-88d5-0f781fa715eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6816","nonce":244560995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6819","nonce":244560995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6823","nonce":244560995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6821","nonce":244560995}]},"public_addr":"172.21.15.187:6816/244560995","cluster_addr":"172.21.15.187:6819/244560995","heartbeat_back_addr":"172.21.15.187:6823/244560995","heartbeat_front_addr":"172.21.15.187:6821/244560995","state":["exists","up"]},{"osd":5,"uuid":"7513ba69-dd00-4a35-b190-9e8cc3b2bdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6801","nonce":324619398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6803","nonce":324619398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6812","nonce":324619398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6807","nonce":324619398}]},"public_addr":"172.21.15.187:6801/324619398","cluster_addr":"172.21.15.187:6803/324619398","heartbeat_back_addr":"172.21.15.187:6812/324619398","heartbeat_front_addr":"172.21.15.187:6807/324619398","state":["exists","up"]},{"osd":6,"uuid":"50081810-c093-4d2f-a4d7-b7c695c5400d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6806","nonce":1391347786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6809","nonce":1391347786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6818","nonce":1391347786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6813","nonce":1391347786}]},"public_addr":"172.21.15.187:6806/1391347786","cluster_addr":"172.21.15.187:6809/1391347786","heartbeat_back_addr":"172.21.15.187:6818/1391347786","heartbeat_front_addr":"172.21.15.187:6813/1391347786","state":["exists","up"]},{"osd":7,"uuid":"775dae05-54f5-4ad4-bbce-92863c6ed2ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6825","nonce":1220065622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6827","nonce":1220065622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6831","nonce":1220065622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6829","nonce":1220065622}]},"public_addr":"172.21.15.187:6825/1220065622","cluster_addr":"172.21.15.187:6827/1220065622","heartbeat_back_addr":"172.21.15.187:6831/1220065622","heartbeat_front_addr":"172.21.15.187:6829/1220065622","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:05.983520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.299516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.067323+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.307748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.018742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.258775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.010912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.295355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-26T22:54:58.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.336+0000 7ff846400640 1 -- 172.21.15.131:0/631184854 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ff82006be30 msgr2=0x7ff82006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:58.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.336+0000 7ff846400640 1 --2- 172.21.15.131:0/631184854 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ff82006be30 0x7ff82006e2f0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7ff83000ba00 tx=0x7ff83002c000 comp rx=0 tx=0).stop 2024-05-26T22:54:58.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.336+0000 7ff846400640 1 -- 172.21.15.131:0/631184854 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff8400cc200 msgr2=0x7ff8401f4bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:58.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.336+0000 7ff846400640 1 --2- 172.21.15.131:0/631184854 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff8400cc200 0x7ff8401f4bd0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7ff84006d360 tx=0x7ff83800e4b0 comp rx=0 tx=0).stop 2024-05-26T22:54:58.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.336+0000 7ff846400640 1 -- 172.21.15.131:0/631184854 shutdown_connections 2024-05-26T22:54:58.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.336+0000 7ff846400640 1 --2- 172.21.15.131:0/631184854 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff8400cbd80 0x7ff8401f4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:58.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.336+0000 7ff846400640 1 --2- 172.21.15.131:0/631184854 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ff82006be30 0x7ff82006e2f0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:58.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.336+0000 7ff846400640 1 --2- 172.21.15.131:0/631184854 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff8400cc200 0x7ff8401f4bd0 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:58.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.336+0000 7ff846400640 1 --2- 172.21.15.131:0/631184854 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff8240039b0 0x7ff840148260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:58.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.336+0000 7ff846400640 1 -- 172.21.15.131:0/631184854 >> 172.21.15.131:0/631184854 conn(0x7ff840072e90 msgr2=0x7ff8400d2590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:58.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.336+0000 7ff846400640 1 -- 172.21.15.131:0/631184854 shutdown_connections 2024-05-26T22:54:58.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.336+0000 7ff846400640 1 -- 172.21.15.131:0/631184854 wait complete. 2024-05-26T22:54:58.357 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-26T22:54:58.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.543+0000 7f18fa200640 1 Processor -- start 2024-05-26T22:54:58.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.543+0000 7f18fa200640 1 -- start start 2024-05-26T22:54:58.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.545+0000 7f18fa200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f18f4155840 0x7f18f4071930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:58.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.545+0000 7f18fa200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f18f415c3d0 0x7f18f406ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:58.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.545+0000 7f18fa200640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18f40722a0 con 0x7f18f406f490 2024-05-26T22:54:58.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.545+0000 7f18fa200640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f18f4072580 con 0x7f18f4155840 2024-05-26T22:54:58.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.545+0000 7f18fa200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f18f4072860 con 0x7f18f415c3d0 2024-05-26T22:54:58.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.545+0000 7f18f3400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f18f4155840 0x7f18f4071930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:58.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.545+0000 7f18f3e00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f18f406f490 0x7f18f415ec80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:54522/0 (socket says 172.21.15.131:54522) 2024-05-26T22:54:58.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.545+0000 7f18f3e00640 1 -- 172.21.15.131:0/2926445863 learned_addr learned my addr 172.21.15.131:0/2926445863 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:54:58.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.545+0000 7f18f3400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f18f4155840 0x7f18f4071930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41016/0 (socket says 172.21.15.131:41016) 2024-05-26T22:54:58.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.546+0000 7f18f2a00640 1 --2- 172.21.15.131:0/2926445863 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f18f415c3d0 0x7f18f406ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:58.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.546+0000 7f18f3400640 1 -- 172.21.15.131:0/2926445863 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f18f415c3d0 msgr2=0x7f18f406ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:58.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.546+0000 7f18f3400640 1 --2- 172.21.15.131:0/2926445863 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f18f415c3d0 0x7f18f406ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:58.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.546+0000 7f18f3400640 1 -- 172.21.15.131:0/2926445863 >> v1:172.21.15.131:6789/0 conn(0x7f18f406f490 legacy=0x7f18f415ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:58.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.546+0000 7f18f3400640 1 -- 172.21.15.131:0/2926445863 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18f4072b40 con 0x7f18f4155840 2024-05-26T22:54:58.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.546+0000 7f18f2a00640 1 --2- 172.21.15.131:0/2926445863 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f18f415c3d0 0x7f18f406ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:54:58.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.546+0000 7f18f2000640 1 -- 172.21.15.131:0/2926445863 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1121796147 0 0) 0x7f18f40722a0 con 0x7f18f406f490 2024-05-26T22:54:58.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.547+0000 7f18f3400640 1 --2- 172.21.15.131:0/2926445863 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18f4155840 0x7f18f4071930 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f18dc00ba40 tx=0x7f18dc02c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25545e52f6a86695 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:58.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.547+0000 7f18f2000640 1 -- 172.21.15.131:0/2926445863 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18dc032750 con 0x7f18f4155840 2024-05-26T22:54:58.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.547+0000 7f18f2000640 1 -- 172.21.15.131:0/2926445863 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18dc030050 con 0x7f18f4155840 2024-05-26T22:54:58.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.547+0000 7f18f2000640 1 -- 172.21.15.131:0/2926445863 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18dc032b10 con 0x7f18f4155840 2024-05-26T22:54:58.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.548+0000 7f18fa200640 1 -- 172.21.15.131:0/2926445863 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18f4155840 msgr2=0x7f18f4071930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:58.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.548+0000 7f18fa200640 1 --2- 172.21.15.131:0/2926445863 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18f4155840 0x7f18f4071930 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f18dc00ba40 tx=0x7f18dc02c850 comp rx=0 tx=0).stop 2024-05-26T22:54:58.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.548+0000 7f18fa200640 1 -- 172.21.15.131:0/2926445863 shutdown_connections 2024-05-26T22:54:58.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.548+0000 7f18fa200640 1 --2- 172.21.15.131:0/2926445863 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f18f415c3d0 0x7f18f406ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:58.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.548+0000 7f18fa200640 1 --2- 172.21.15.131:0/2926445863 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18f4155840 0x7f18f4071930 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:58.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.548+0000 7f18fa200640 1 -- 172.21.15.131:0/2926445863 >> 172.21.15.131:0/2926445863 conn(0x7f18f40b1e90 msgr2=0x7f18f40b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:58.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.548+0000 7f18fa200640 1 -- 172.21.15.131:0/2926445863 shutdown_connections 2024-05-26T22:54:58.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.548+0000 7f18fa200640 1 -- 172.21.15.131:0/2926445863 wait complete. 2024-05-26T22:54:58.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.550+0000 7f18fa200640 1 Processor -- start 2024-05-26T22:54:58.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.550+0000 7f18fa200640 1 -- start start 2024-05-26T22:54:58.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.551+0000 7f18fa200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f18f406f490 0x7f18f40cb770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:58.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.551+0000 7f18fa200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18f4155840 0x7f18f415a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:58.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.551+0000 7f18fa200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f18f415c3d0 0x7f18f415a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:58.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.551+0000 7f18fa200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f18f4071ff0 con 0x7f18f4155840 2024-05-26T22:54:58.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.551+0000 7f18fa200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f18f4071e70 con 0x7f18f415c3d0 2024-05-26T22:54:58.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.551+0000 7f18fa200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f18f40aee50 con 0x7f18f406f490 2024-05-26T22:54:58.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.551+0000 7f18f2a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18f4155840 0x7f18f415a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:58.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.551+0000 7f18f2a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18f4155840 0x7f18f415a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41024/0 (socket says 172.21.15.131:41024) 2024-05-26T22:54:58.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.551+0000 7f18f2a00640 1 -- 172.21.15.131:0/295087411 learned_addr learned my addr 172.21.15.131:0/295087411 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:54:58.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.551+0000 7f18f2a00640 1 -- 172.21.15.131:0/295087411 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f18f406f490 msgr2=0x7f18f40cb770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:58.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.551+0000 7f18f2a00640 1 --2- 172.21.15.131:0/295087411 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f18f406f490 0x7f18f40cb770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:58.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.551+0000 7f18f2a00640 1 -- 172.21.15.131:0/295087411 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f18f415c3d0 msgr2=0x7f18f415a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:58.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.551+0000 7f18f2a00640 1 --2- 172.21.15.131:0/295087411 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f18f415c3d0 0x7f18f415a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:58.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.551+0000 7f18f2a00640 1 -- 172.21.15.131:0/295087411 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18f4152500 con 0x7f18f4155840 2024-05-26T22:54:58.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.552+0000 7f18f2a00640 1 --2- 172.21.15.131:0/295087411 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18f4155840 0x7f18f415a300 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f18e400ce30 tx=0x7f18e400a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:58.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.552+0000 7f18eb400640 1 -- 172.21.15.131:0/295087411 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18e400f890 con 0x7f18f4155840 2024-05-26T22:54:58.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.552+0000 7f18eb400640 1 -- 172.21.15.131:0/295087411 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18e400fa30 con 0x7f18f4155840 2024-05-26T22:54:58.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.552+0000 7f18eb400640 1 -- 172.21.15.131:0/295087411 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18e400fe00 con 0x7f18f4155840 2024-05-26T22:54:58.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.552+0000 7f18fa200640 1 -- 172.21.15.131:0/295087411 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18f41527a0 con 0x7f18f4155840 2024-05-26T22:54:58.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.552+0000 7f18fa200640 1 -- 172.21.15.131:0/295087411 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18f4152d60 con 0x7f18f4155840 2024-05-26T22:54:58.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.553+0000 7f18fa200640 1 -- 172.21.15.131:0/295087411 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18f40b2ba0 con 0x7f18f4155840 2024-05-26T22:54:58.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.554+0000 7f18eb400640 1 -- 172.21.15.131:0/295087411 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f18e4018400 con 0x7f18f4155840 2024-05-26T22:54:58.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.555+0000 7f18eb400640 1 --2- 172.21.15.131:0/295087411 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f18d006be30 0x7f18d006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:58.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.555+0000 7f18eb400640 1 -- 172.21.15.131:0/295087411 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5709+0+0 (secure 0 0 0) 0x7f18e408bce0 con 0x7f18f4155840 2024-05-26T22:54:58.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.555+0000 7f18f3400640 1 --2- 172.21.15.131:0/295087411 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f18d006be30 0x7f18d006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:58.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.556+0000 7f18f3400640 1 --2- 172.21.15.131:0/295087411 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f18d006be30 0x7f18d006e2f0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f18dc037000 tx=0x7f18dc031000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:58.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.560+0000 7f18eb400640 1 -- 172.21.15.131:0/295087411 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f18e4014030 con 0x7f18f4155840 2024-05-26T22:54:58.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.758+0000 7f18fa200640 1 -- 172.21.15.131:0/295087411 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f18f40694e0 con 0x7f18f4155840 2024-05-26T22:54:58.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.759+0000 7f18eb400640 1 -- 172.21.15.131:0/295087411 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+16645 (secure 0 0 0) 0x7f18e400ae20 con 0x7f18f4155840 2024-05-26T22:54:58.760 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:54:58.760 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":94,"fsid":"46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f","created":"2024-05-26T22:47:57.288905+0000","modified":"2024-05-26T22:54:57.327260+0000","last_up_change":"2024-05-26T22:48:09.602207+0000","last_in_change":"2024-05-26T22:48:00.687341+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-26T22:48:10.463339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-05-26T22:54:55.120321+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-05-26T22:54:56.346430+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"814eaf62-a18d-494b-8e27-fd8fb88adea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6801","nonce":2350940051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6803","nonce":2350940051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6807","nonce":2350940051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6805","nonce":2350940051}]},"public_addr":"172.21.15.131:6801/2350940051","cluster_addr":"172.21.15.131:6803/2350940051","heartbeat_back_addr":"172.21.15.131:6807/2350940051","heartbeat_front_addr":"172.21.15.131:6805/2350940051","state":["exists","up"]},{"osd":1,"uuid":"42e66287-f6c6-4e26-af3c-6dfaa3536d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6825","nonce":1525528247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6827","nonce":1525528247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6831","nonce":1525528247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6829","nonce":1525528247}]},"public_addr":"172.21.15.131:6825/1525528247","cluster_addr":"172.21.15.131:6827/1525528247","heartbeat_back_addr":"172.21.15.131:6831/1525528247","heartbeat_front_addr":"172.21.15.131:6829/1525528247","state":["exists","up"]},{"osd":2,"uuid":"0da34673-04a2-4fd0-b57a-0ff2b679b043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6817","nonce":3627842210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6819","nonce":3627842210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6823","nonce":3627842210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6821","nonce":3627842210}]},"public_addr":"172.21.15.131:6817/3627842210","cluster_addr":"172.21.15.131:6819/3627842210","heartbeat_back_addr":"172.21.15.131:6823/3627842210","heartbeat_front_addr":"172.21.15.131:6821/3627842210","state":["exists","up"]},{"osd":3,"uuid":"cf117a27-d6b2-43d3-ae52-b47988bf8eed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6809","nonce":350790591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6811","nonce":350790591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6815","nonce":350790591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6813","nonce":350790591}]},"public_addr":"172.21.15.131:6809/350790591","cluster_addr":"172.21.15.131:6811/350790591","heartbeat_back_addr":"172.21.15.131:6815/350790591","heartbeat_front_addr":"172.21.15.131:6813/350790591","state":["exists","up"]},{"osd":4,"uuid":"a092ccfc-be89-4a84-88d5-0f781fa715eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6816","nonce":244560995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6819","nonce":244560995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6823","nonce":244560995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6821","nonce":244560995}]},"public_addr":"172.21.15.187:6816/244560995","cluster_addr":"172.21.15.187:6819/244560995","heartbeat_back_addr":"172.21.15.187:6823/244560995","heartbeat_front_addr":"172.21.15.187:6821/244560995","state":["exists","up"]},{"osd":5,"uuid":"7513ba69-dd00-4a35-b190-9e8cc3b2bdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6801","nonce":324619398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6803","nonce":324619398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6812","nonce":324619398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6807","nonce":324619398}]},"public_addr":"172.21.15.187:6801/324619398","cluster_addr":"172.21.15.187:6803/324619398","heartbeat_back_addr":"172.21.15.187:6812/324619398","heartbeat_front_addr":"172.21.15.187:6807/324619398","state":["exists","up"]},{"osd":6,"uuid":"50081810-c093-4d2f-a4d7-b7c695c5400d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6806","nonce":1391347786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6809","nonce":1391347786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6818","nonce":1391347786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6813","nonce":1391347786}]},"public_addr":"172.21.15.187:6806/1391347786","cluster_addr":"172.21.15.187:6809/1391347786","heartbeat_back_addr":"172.21.15.187:6818/1391347786","heartbeat_front_addr":"172.21.15.187:6813/1391347786","state":["exists","up"]},{"osd":7,"uuid":"775dae05-54f5-4ad4-bbce-92863c6ed2ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6825","nonce":1220065622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6827","nonce":1220065622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6831","nonce":1220065622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6829","nonce":1220065622}]},"public_addr":"172.21.15.187:6825/1220065622","cluster_addr":"172.21.15.187:6827/1220065622","heartbeat_back_addr":"172.21.15.187:6831/1220065622","heartbeat_front_addr":"172.21.15.187:6829/1220065622","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:05.983520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.299516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.067323+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.307748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.018742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.258775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.010912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.295355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-26T22:54:58.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.763+0000 7f18fa200640 1 -- 172.21.15.131:0/295087411 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f18d006be30 msgr2=0x7f18d006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:58.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.763+0000 7f18fa200640 1 --2- 172.21.15.131:0/295087411 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f18d006be30 0x7f18d006e2f0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f18dc037000 tx=0x7f18dc031000 comp rx=0 tx=0).stop 2024-05-26T22:54:58.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.763+0000 7f18fa200640 1 -- 172.21.15.131:0/295087411 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18f4155840 msgr2=0x7f18f415a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:58.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.763+0000 7f18fa200640 1 --2- 172.21.15.131:0/295087411 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18f4155840 0x7f18f415a300 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f18e400ce30 tx=0x7f18e400a650 comp rx=0 tx=0).stop 2024-05-26T22:54:58.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.763+0000 7f18fa200640 1 -- 172.21.15.131:0/295087411 shutdown_connections 2024-05-26T22:54:58.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.763+0000 7f18fa200640 1 --2- 172.21.15.131:0/295087411 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f18d006be30 0x7f18d006e2f0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:58.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.763+0000 7f18fa200640 1 --2- 172.21.15.131:0/295087411 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f18f415c3d0 0x7f18f415a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:58.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.763+0000 7f18fa200640 1 --2- 172.21.15.131:0/295087411 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18f4155840 0x7f18f415a300 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:58.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.763+0000 7f18fa200640 1 --2- 172.21.15.131:0/295087411 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f18f406f490 0x7f18f40cb770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:58.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.763+0000 7f18fa200640 1 -- 172.21.15.131:0/295087411 >> 172.21.15.131:0/295087411 conn(0x7f18f40b1e90 msgr2=0x7f18f40b2490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:58.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.763+0000 7f18fa200640 1 -- 172.21.15.131:0/295087411 shutdown_connections 2024-05-26T22:54:58.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.763+0000 7f18fa200640 1 -- 172.21.15.131:0/295087411 wait complete. 2024-05-26T22:54:58.783 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-26T22:54:58.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.990+0000 7f8fbb400640 1 Processor -- start 2024-05-26T22:54:58.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.990+0000 7f8fbb400640 1 -- start start 2024-05-26T22:54:58.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.991+0000 7f8fbb400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8fb40b5450 0x7f8fb40b43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:58.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.992+0000 7f8fbb400640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8fb40b4900 con 0x7f8fb406f420 2024-05-26T22:54:58.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.992+0000 7f8fbb400640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8fb4160180 con 0x7f8fb40d0460 2024-05-26T22:54:58.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.992+0000 7f8fbb400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f8fb4160d90 con 0x7f8fb40b5450 2024-05-26T22:54:58.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.992+0000 7f8fb3e00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f8fb406f420 0x7f8fb40cfd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:54536/0 (socket says 172.21.15.131:54536) 2024-05-26T22:54:58.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.992+0000 7f8fb3e00640 1 -- 172.21.15.131:0/3912400824 learned_addr learned my addr 172.21.15.131:0/3912400824 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:54:58.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.992+0000 7f8fb8a00640 1 --2- 172.21.15.131:0/3912400824 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8fb40b5450 0x7f8fb40b43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:58.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.992+0000 7f8fb3400640 1 -- 172.21.15.131:0/3912400824 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2605710845 0 0) 0x7f8fb40b4900 con 0x7f8fb406f420 2024-05-26T22:54:58.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.992+0000 7f8fb3400640 1 -- 172.21.15.131:0/3912400824 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8f9c003520 con 0x7f8fb406f420 2024-05-26T22:54:58.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.993+0000 7f8fb3400640 1 -- 172.21.15.131:0/3912400824 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1767530764 0 0) 0x7f8f9c003520 con 0x7f8fb406f420 2024-05-26T22:54:58.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.993+0000 7f8fb3400640 1 -- 172.21.15.131:0/3912400824 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8fb40b5450 msgr2=0x7f8fb40b43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:58.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.993+0000 7f8fb3400640 1 --2- 172.21.15.131:0/3912400824 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8fb40b5450 0x7f8fb40b43c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:58.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.993+0000 7f8fb3400640 1 -- 172.21.15.131:0/3912400824 >> v1:172.21.15.187:6789/0 conn(0x7f8fb40d0460 legacy=0x7f8fb406d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:58.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.993+0000 7f8fb3400640 1 -- 172.21.15.131:0/3912400824 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fb4160460 con 0x7f8fb406f420 2024-05-26T22:54:58.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.993+0000 7f8fb3400640 1 -- 172.21.15.131:0/3912400824 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7f8fa0001670 con 0x7f8fb406f420 2024-05-26T22:54:58.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.993+0000 7f8fb3400640 1 -- 172.21.15.131:0/3912400824 >> v1:172.21.15.131:6789/0 conn(0x7f8fb406f420 legacy=0x7f8fb40cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:58.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.993+0000 7f8fb3400640 1 --2- 172.21.15.131:0/3912400824 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8f9c0039e0 0x7f8f9c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:58.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.993+0000 7f8fb3400640 1 --2- 172.21.15.131:0/3912400824 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8fb40b5450 0x7f8f9c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:58.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.993+0000 7f8fb3400640 1 --2- 172.21.15.131:0/3912400824 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f9c004d40 0x7f8f9c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:58.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.993+0000 7f8fb3400640 1 -- 172.21.15.131:0/3912400824 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb4160d90 con 0x7f8f9c004d40 2024-05-26T22:54:58.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.993+0000 7f8fb3400640 1 -- 172.21.15.131:0/3912400824 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8f9c00ba70 con 0x7f8f9c0039e0 2024-05-26T22:54:58.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.993+0000 7f8fb3400640 1 -- 172.21.15.131:0/3912400824 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f8f9c00bc20 con 0x7f8fb40b5450 2024-05-26T22:54:58.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.993+0000 7f8fb9400640 1 --2- 172.21.15.131:0/3912400824 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f9c004d40 0x7f8f9c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:58.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.993+0000 7f8fb3e00640 1 --2- 172.21.15.131:0/3912400824 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8fb40b5450 0x7f8f9c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:58.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.993+0000 7f8fb8a00640 1 --2- 172.21.15.131:0/3912400824 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8f9c0039e0 0x7f8f9c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:58.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.994+0000 7f8fb9400640 1 -- 172.21.15.131:0/3912400824 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8fb40b5450 msgr2=0x7f8f9c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:58.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.994+0000 7f8fb9400640 1 --2- 172.21.15.131:0/3912400824 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8fb40b5450 0x7f8f9c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:58.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.994+0000 7f8fb9400640 1 -- 172.21.15.131:0/3912400824 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8f9c0039e0 msgr2=0x7f8f9c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:54:58.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.994+0000 7f8fb9400640 1 --2- 172.21.15.131:0/3912400824 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8f9c0039e0 0x7f8f9c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:58.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.994+0000 7f8fb9400640 1 -- 172.21.15.131:0/3912400824 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f9c003520 con 0x7f8f9c004d40 2024-05-26T22:54:58.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.995+0000 7f8fb9400640 1 --2- 172.21.15.131:0/3912400824 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f9c004d40 0x7f8f9c0051c0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f8fac0044b0 tx=0x7f8f9c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe214271faa161c3 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:58.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.995+0000 7f8fb3400640 1 -- 172.21.15.131:0/3912400824 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fac037070 con 0x7f8f9c004d40 2024-05-26T22:54:58.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.995+0000 7f8fb3400640 1 -- 172.21.15.131:0/3912400824 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fac02ad80 con 0x7f8f9c004d40 2024-05-26T22:54:58.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.995+0000 7f8fb3400640 1 -- 172.21.15.131:0/3912400824 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fac004080 con 0x7f8f9c004d40 2024-05-26T22:54:58.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.995+0000 7f8fbb400640 1 -- 172.21.15.131:0/3912400824 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f9c004d40 msgr2=0x7f8f9c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:58.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.995+0000 7f8fbb400640 1 --2- 172.21.15.131:0/3912400824 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f9c004d40 0x7f8f9c0051c0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f8fac0044b0 tx=0x7f8f9c005830 comp rx=0 tx=0).stop 2024-05-26T22:54:58.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.995+0000 7f8fb9400640 1 -- 172.21.15.131:0/3912400824 reap_dead start 2024-05-26T22:54:58.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.996+0000 7f8fbb400640 1 -- 172.21.15.131:0/3912400824 shutdown_connections 2024-05-26T22:54:58.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.996+0000 7f8fbb400640 1 -- 172.21.15.131:0/3912400824 >> 172.21.15.131:0/3912400824 conn(0x7f8fb406bc90 msgr2=0x7f8fb40b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:58.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.996+0000 7f8fbb400640 1 -- 172.21.15.131:0/3912400824 shutdown_connections 2024-05-26T22:54:58.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.996+0000 7f8fbb400640 1 -- 172.21.15.131:0/3912400824 wait complete. 2024-05-26T22:54:58.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.997+0000 7f8fbb400640 1 Processor -- start 2024-05-26T22:54:58.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.997+0000 7f8fbb400640 1 -- start start 2024-05-26T22:54:58.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.998+0000 7f8fbb400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8f9c004d40 0x7f8fb409bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:58.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.998+0000 7f8fbb400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8fb415f2f0 0x7f8fb41f2e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:58.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.998+0000 7f8fbb400640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8fb409c060 0x7f8fb41f33d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:58.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.998+0000 7f8fbb400640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb40b5830 con 0x7f8fb415f2f0 2024-05-26T22:54:59.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.998+0000 7f8fbb400640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb4160a20 con 0x7f8fb409c060 2024-05-26T22:54:59.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.998+0000 7f8fbb400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f8fb415fde0 con 0x7f8f9c004d40 2024-05-26T22:54:59.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.998+0000 7f8fb3e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8fb415f2f0 0x7f8fb41f2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:59.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.999+0000 7f8fb3e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8fb415f2f0 0x7f8fb41f2e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41044/0 (socket says 172.21.15.131:41044) 2024-05-26T22:54:59.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.999+0000 7f8fb3e00640 1 -- 172.21.15.131:0/2426207508 learned_addr learned my addr 172.21.15.131:0/2426207508 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:54:59.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.999+0000 7f8fb9400640 1 --2- 172.21.15.131:0/2426207508 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8fb409c060 0x7f8fb41f33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:59.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.999+0000 7f8fb8a00640 1 --2- 172.21.15.131:0/2426207508 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8f9c004d40 0x7f8fb409bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:59.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.999+0000 7f8fb3e00640 1 -- 172.21.15.131:0/2426207508 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8f9c004d40 msgr2=0x7f8fb409bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:59.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.999+0000 7f8fb3e00640 1 --2- 172.21.15.131:0/2426207508 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8f9c004d40 0x7f8fb409bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:59.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.999+0000 7f8fb3e00640 1 -- 172.21.15.131:0/2426207508 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8fb409c060 msgr2=0x7f8fb41f33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:59.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.999+0000 7f8fb3e00640 1 --2- 172.21.15.131:0/2426207508 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8fb409c060 0x7f8fb41f33d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:59.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.999+0000 7f8fb3e00640 1 -- 172.21.15.131:0/2426207508 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fb409aaa0 con 0x7f8fb415f2f0 2024-05-26T22:54:59.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.999+0000 7f8fb3e00640 1 --2- 172.21.15.131:0/2426207508 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8fb415f2f0 0x7f8fb41f2e90 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f8fa000e870 tx=0x7f8fa000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:59.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:58.999+0000 7f8fb0c00640 1 -- 172.21.15.131:0/2426207508 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fa000c8d0 con 0x7f8fb415f2f0 2024-05-26T22:54:59.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:59.000+0000 7f8fb0c00640 1 -- 172.21.15.131:0/2426207508 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fa0018070 con 0x7f8fb415f2f0 2024-05-26T22:54:59.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:59.000+0000 7f8fb0c00640 1 -- 172.21.15.131:0/2426207508 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fa000ccd0 con 0x7f8fb415f2f0 2024-05-26T22:54:59.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:59.000+0000 7f8fbb400640 1 -- 172.21.15.131:0/2426207508 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fb409ad80 con 0x7f8fb415f2f0 2024-05-26T22:54:59.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:59.000+0000 7f8fbb400640 1 -- 172.21.15.131:0/2426207508 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fb409b340 con 0x7f8fb415f2f0 2024-05-26T22:54:59.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:59.001+0000 7f8fbb400640 1 -- 172.21.15.131:0/2426207508 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fb4074e90 con 0x7f8fb415f2f0 2024-05-26T22:54:59.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:59.002+0000 7f8fb0c00640 1 -- 172.21.15.131:0/2426207508 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8fa001d460 con 0x7f8fb415f2f0 2024-05-26T22:54:59.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:59.003+0000 7f8fb0c00640 1 --2- 172.21.15.131:0/2426207508 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f8f9406be60 0x7f8f9406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:54:59.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:59.003+0000 7f8fb0c00640 1 -- 172.21.15.131:0/2426207508 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 5709+0+0 (secure 0 0 0) 0x7f8fa008ae50 con 0x7f8fb415f2f0 2024-05-26T22:54:59.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:59.003+0000 7f8fb8a00640 1 --2- 172.21.15.131:0/2426207508 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f8f9406be60 0x7f8f9406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:54:59.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:59.004+0000 7f8fb8a00640 1 --2- 172.21.15.131:0/2426207508 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f8f9406be60 0x7f8f9406e320 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f8fa8009950 tx=0x7f8fa8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:54:59.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:59.007+0000 7f8fb0c00640 1 -- 172.21.15.131:0/2426207508 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f8fa00597f0 con 0x7f8fb415f2f0 2024-05-26T22:54:59.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:59.236+0000 7f8fbb400640 1 -- 172.21.15.131:0/2426207508 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f8fb40cee90 con 0x7f8fb415f2f0 2024-05-26T22:54:59.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:59.927+0000 7f8fb0c00640 1 -- 172.21.15.131:0/2426207508 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v93) ==== 118+0+0 (secure 0 0 0) 0x7f8fa00590f0 con 0x7f8fb415f2f0 2024-05-26T22:54:59.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:59.930+0000 7f8fbb400640 1 -- 172.21.15.131:0/2426207508 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f8f9406be60 msgr2=0x7f8f9406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:59.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:59.930+0000 7f8fbb400640 1 --2- 172.21.15.131:0/2426207508 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f8f9406be60 0x7f8f9406e320 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f8fa8009950 tx=0x7f8fa8008040 comp rx=0 tx=0).stop 2024-05-26T22:54:59.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:59.930+0000 7f8fbb400640 1 -- 172.21.15.131:0/2426207508 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8fb415f2f0 msgr2=0x7f8fb41f2e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:54:59.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:59.930+0000 7f8fbb400640 1 --2- 172.21.15.131:0/2426207508 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8fb415f2f0 0x7f8fb41f2e90 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f8fa000e870 tx=0x7f8fa000ed30 comp rx=0 tx=0).stop 2024-05-26T22:54:59.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:59.930+0000 7f8fbb400640 1 -- 172.21.15.131:0/2426207508 shutdown_connections 2024-05-26T22:54:59.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:59.930+0000 7f8fbb400640 1 --2- 172.21.15.131:0/2426207508 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f8f9406be60 0x7f8f9406e320 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:59.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:59.930+0000 7f8fbb400640 1 --2- 172.21.15.131:0/2426207508 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8fb409c060 0x7f8fb41f33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:59.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:59.930+0000 7f8fbb400640 1 --2- 172.21.15.131:0/2426207508 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8fb415f2f0 0x7f8fb41f2e90 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:59.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:59.930+0000 7f8fbb400640 1 --2- 172.21.15.131:0/2426207508 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8f9c004d40 0x7f8fb409bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:54:59.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:59.930+0000 7f8fbb400640 1 -- 172.21.15.131:0/2426207508 >> 172.21.15.131:0/2426207508 conn(0x7f8fb406bc90 msgr2=0x7f8fb406e6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:54:59.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:59.931+0000 7f8fbb400640 1 -- 172.21.15.131:0/2426207508 shutdown_connections 2024-05-26T22:54:59.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:54:59.931+0000 7f8fbb400640 1 -- 172.21.15.131:0/2426207508 wait complete. 2024-05-26T22:54:59.950 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-26T22:54:59.950 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-26T22:55:00.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:00.159+0000 7f7e81200640 1 Processor -- start 2024-05-26T22:55:00.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:00.160+0000 7f7e81200640 1 -- start start 2024-05-26T22:55:00.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:00.161+0000 7f7e81200640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f7e7c0cbef0 0x7f7e7c0cc2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:00.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:00.161+0000 7f7e81200640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e7c0cdca0 con 0x7f7e7c0ad3a0 2024-05-26T22:55:00.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:00.161+0000 7f7e81200640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e7c0cdf80 con 0x7f7e7c0acef0 2024-05-26T22:55:00.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:00.161+0000 7f7e81200640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f7e7c0ce260 con 0x7f7e7c0cbef0 2024-05-26T22:55:00.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:00.162+0000 7f7e7a800640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f7e7c0ad3a0 0x7f7e7c0cea60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:54552/0 (socket says 172.21.15.131:54552) 2024-05-26T22:55:00.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:00.162+0000 7f7e7a800640 1 -- 172.21.15.131:0/1651169757 learned_addr learned my addr 172.21.15.131:0/1651169757 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:55:00.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:00.162+0000 7f7e79e00640 1 --2- 172.21.15.131:0/1651169757 >> v2:172.21.15.187:3300/0 conn(0x7f7e7c0cbef0 0x7f7e7c0cc2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:00.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:00.162+0000 7f7e78a00640 1 -- 172.21.15.131:0/1651169757 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2180865172 0 0) 0x7f7e7c0cdca0 con 0x7f7e7c0ad3a0 2024-05-26T22:55:00.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:00.162+0000 7f7e78a00640 1 -- 172.21.15.131:0/1651169757 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7e60003520 con 0x7f7e7c0ad3a0 2024-05-26T22:55:00.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:00.163+0000 7f7e78a00640 1 -- 172.21.15.131:0/1651169757 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 314535021 0 0) 0x7f7e60003520 con 0x7f7e7c0ad3a0 2024-05-26T22:55:00.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:00.163+0000 7f7e78a00640 1 -- 172.21.15.131:0/1651169757 >> v2:172.21.15.187:3300/0 conn(0x7f7e7c0cbef0 msgr2=0x7f7e7c0cc2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:00.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:00.163+0000 7f7e78a00640 1 --2- 172.21.15.131:0/1651169757 >> v2:172.21.15.187:3300/0 conn(0x7f7e7c0cbef0 0x7f7e7c0cc2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:00.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:00.163+0000 7f7e78a00640 1 -- 172.21.15.131:0/1651169757 >> v1:172.21.15.187:6789/0 conn(0x7f7e7c0acef0 legacy=0x7f7e7c06d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:00.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:00.163+0000 7f7e78a00640 1 -- 172.21.15.131:0/1651169757 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e7c0ce540 con 0x7f7e7c0ad3a0 2024-05-26T22:55:00.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:00.163+0000 7f7e78a00640 1 -- 172.21.15.131:0/1651169757 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7f7e74001670 con 0x7f7e7c0ad3a0 2024-05-26T22:55:00.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:00.163+0000 7f7e78a00640 1 -- 172.21.15.131:0/1651169757 >> v1:172.21.15.131:6789/0 conn(0x7f7e7c0ad3a0 legacy=0x7f7e7c0cea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:00.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:00.163+0000 7f7e78a00640 1 --2- 172.21.15.131:0/1651169757 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e60003a00 0x7f7e60003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:00.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:00.163+0000 7f7e78a00640 1 --2- 172.21.15.131:0/1651169757 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7e60004800 0x7f7e60004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:00.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:00.163+0000 7f7e78a00640 1 --2- 172.21.15.131:0/1651169757 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7e600051c0 0x7f7e6000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:00.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:00.163+0000 7f7e78a00640 1 -- 172.21.15.131:0/1651169757 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f7e7c0ce260 con 0x7f7e60003a00 2024-05-26T22:55:00.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:00.163+0000 7f7e78a00640 1 -- 172.21.15.131:0/1651169757 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7e6000c0c0 con 0x7f7e60004800 2024-05-26T22:55:00.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:00.163+0000 7f7e78a00640 1 -- 172.21.15.131:0/1651169757 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f7e6000c2c0 con 0x7f7e600051c0 2024-05-26T22:55:00.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:00.163+0000 7f7e79e00640 1 --2- 172.21.15.131:0/1651169757 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e60003a00 0x7f7e60003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:00.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:00.163+0000 7f7e79e00640 1 -- 172.21.15.131:0/1651169757 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7e600051c0 msgr2=0x7f7e6000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:00.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:00.163+0000 7f7e79e00640 1 --2- 172.21.15.131:0/1651169757 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7e600051c0 0x7f7e6000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:00.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:00.163+0000 7f7e79e00640 1 -- 172.21.15.131:0/1651169757 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7e60004800 msgr2=0x7f7e60004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:00.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:00.163+0000 7f7e79e00640 1 --2- 172.21.15.131:0/1651169757 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7e60004800 0x7f7e60004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:00.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:00.163+0000 7f7e79e00640 1 -- 172.21.15.131:0/1651169757 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e60003520 con 0x7f7e60003a00 2024-05-26T22:55:00.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:00.163+0000 7f7e7a800640 1 -- 172.21.15.131:0/1651169757 reap_dead start 2024-05-26T22:55:00.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:00.165+0000 7f7e79e00640 1 --2- 172.21.15.131:0/1651169757 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e60003a00 0x7f7e60003e20 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f7e6400a7f0 tx=0x7f7e60005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50f8de97c1d97998 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:00.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:00.165+0000 7f7e78a00640 1 -- 172.21.15.131:0/1651169757 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e6403a070 con 0x7f7e60003a00 2024-05-26T22:55:00.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:00.165+0000 7f7e78a00640 1 -- 172.21.15.131:0/1651169757 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e6402d820 con 0x7f7e60003a00 2024-05-26T22:55:00.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:00.165+0000 7f7e78a00640 1 -- 172.21.15.131:0/1651169757 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e6400b040 con 0x7f7e60003a00 2024-05-26T22:55:00.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:00.165+0000 7f7e81200640 1 -- 172.21.15.131:0/1651169757 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e60003a00 msgr2=0x7f7e60003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:00.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:00.165+0000 7f7e81200640 1 --2- 172.21.15.131:0/1651169757 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e60003a00 0x7f7e60003e20 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f7e6400a7f0 tx=0x7f7e60005830 comp rx=0 tx=0).stop 2024-05-26T22:55:00.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:00.165+0000 7f7e81200640 1 -- 172.21.15.131:0/1651169757 shutdown_connections 2024-05-26T22:55:00.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:00.165+0000 7f7e81200640 1 --2- 172.21.15.131:0/1651169757 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e60003a00 0x7f7e60003e20 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:00.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:00.165+0000 7f7e81200640 1 -- 172.21.15.131:0/1651169757 >> 172.21.15.131:0/1651169757 conn(0x7f7e7c06bc90 msgr2=0x7f7e7c0b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:00.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:00.166+0000 7f7e81200640 1 -- 172.21.15.131:0/1651169757 shutdown_connections 2024-05-26T22:55:00.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:00.166+0000 7f7e81200640 1 -- 172.21.15.131:0/1651169757 wait complete. 2024-05-26T22:55:00.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:00.167+0000 7f7e81200640 1 Processor -- start 2024-05-26T22:55:00.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:00.167+0000 7f7e81200640 1 -- start start 2024-05-26T22:55:00.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:00.168+0000 7f7e81200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7e60003a00 0x7f7e7c09af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:00.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:00.168+0000 7f7e81200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e7c1496b0 0x7f7e7c15d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:00.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:00.168+0000 7f7e81200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7e7c15d860 0x7f7e7c1f3270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:00.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:00.168+0000 7f7e81200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f7e7c0cc660 con 0x7f7e7c1496b0 2024-05-26T22:55:00.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:00.168+0000 7f7e81200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7e7c0cc4e0 con 0x7f7e60003a00 2024-05-26T22:55:00.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:00.168+0000 7f7e81200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f7e7c0cc7e0 con 0x7f7e7c15d860 2024-05-26T22:55:00.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:00.169+0000 7f7e79400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e7c1496b0 0x7f7e7c15d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:00.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:00.169+0000 7f7e79400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e7c1496b0 0x7f7e7c15d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41058/0 (socket says 172.21.15.131:41058) 2024-05-26T22:55:00.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:00.169+0000 7f7e79400640 1 -- 172.21.15.131:0/2589955688 learned_addr learned my addr 172.21.15.131:0/2589955688 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:55:00.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:00.169+0000 7f7e79e00640 1 --2- 172.21.15.131:0/2589955688 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7e60003a00 0x7f7e7c09af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:00.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:00.169+0000 7f7e7a800640 1 --2- 172.21.15.131:0/2589955688 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7e7c15d860 0x7f7e7c1f3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:00.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:00.169+0000 7f7e79400640 1 -- 172.21.15.131:0/2589955688 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7e7c15d860 msgr2=0x7f7e7c1f3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:00.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:00.169+0000 7f7e79400640 1 --2- 172.21.15.131:0/2589955688 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7e7c15d860 0x7f7e7c1f3270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:00.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:00.169+0000 7f7e79400640 1 -- 172.21.15.131:0/2589955688 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7e60003a00 msgr2=0x7f7e7c09af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:00.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:00.169+0000 7f7e79400640 1 --2- 172.21.15.131:0/2589955688 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7e60003a00 0x7f7e7c09af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:00.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:00.169+0000 7f7e79400640 1 -- 172.21.15.131:0/2589955688 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e7c1f3f60 con 0x7f7e7c1496b0 2024-05-26T22:55:00.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:00.169+0000 7f7e7a800640 1 --2- 172.21.15.131:0/2589955688 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7e7c15d860 0x7f7e7c1f3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:55:00.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:00.169+0000 7f7e79400640 1 --2- 172.21.15.131:0/2589955688 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e7c1496b0 0x7f7e7c15d320 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c00c7a0 tx=0x7f7e6c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:00.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:00.170+0000 7f7e72000640 1 -- 172.21.15.131:0/2589955688 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e6c003b70 con 0x7f7e7c1496b0 2024-05-26T22:55:00.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:00.170+0000 7f7e72000640 1 -- 172.21.15.131:0/2589955688 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e6c003d30 con 0x7f7e7c1496b0 2024-05-26T22:55:00.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:00.170+0000 7f7e72000640 1 -- 172.21.15.131:0/2589955688 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e6c00aa90 con 0x7f7e7c1496b0 2024-05-26T22:55:00.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:00.170+0000 7f7e81200640 1 -- 172.21.15.131:0/2589955688 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e7c0b2fd0 con 0x7f7e7c1496b0 2024-05-26T22:55:00.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:00.170+0000 7f7e81200640 1 -- 172.21.15.131:0/2589955688 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e7c0b3590 con 0x7f7e7c1496b0 2024-05-26T22:55:00.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:00.171+0000 7f7e81200640 1 -- 172.21.15.131:0/2589955688 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e7c074e90 con 0x7f7e7c1496b0 2024-05-26T22:55:00.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:00.172+0000 7f7e72000640 1 -- 172.21.15.131:0/2589955688 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7e6c00ac30 con 0x7f7e7c1496b0 2024-05-26T22:55:00.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:00.173+0000 7f7e72000640 1 --2- 172.21.15.131:0/2589955688 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f7e5806be60 0x7f7e5806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:00.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:00.173+0000 7f7e72000640 1 -- 172.21.15.131:0/2589955688 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 5709+0+0 (secure 0 0 0) 0x7f7e6c08adc0 con 0x7f7e7c1496b0 2024-05-26T22:55:00.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:00.173+0000 7f7e79e00640 1 --2- 172.21.15.131:0/2589955688 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f7e5806be60 0x7f7e5806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:00.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:00.174+0000 7f7e79e00640 1 --2- 172.21.15.131:0/2589955688 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f7e5806be60 0x7f7e5806e320 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f7e64037000 tx=0x7f7e64031000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:00.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:00.177+0000 7f7e72000640 1 -- 172.21.15.131:0/2589955688 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f7e6c059760 con 0x7f7e7c1496b0 2024-05-26T22:55:00.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:00.413+0000 7f7e81200640 1 -- 172.21.15.131:0/2589955688 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f7e7c0695c0 con 0x7f7e7c1496b0 2024-05-26T22:55:00.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:00.937+0000 7f7e72000640 1 -- 172.21.15.131:0/2589955688 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v94) ==== 122+0+0 (secure 0 0 0) 0x7f7e6c059060 con 0x7f7e7c1496b0 2024-05-26T22:55:00.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:00.941+0000 7f7e81200640 1 -- 172.21.15.131:0/2589955688 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f7e5806be60 msgr2=0x7f7e5806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:00.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:00.941+0000 7f7e81200640 1 --2- 172.21.15.131:0/2589955688 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f7e5806be60 0x7f7e5806e320 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f7e64037000 tx=0x7f7e64031000 comp rx=0 tx=0).stop 2024-05-26T22:55:00.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:00.941+0000 7f7e81200640 1 -- 172.21.15.131:0/2589955688 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e7c1496b0 msgr2=0x7f7e7c15d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:00.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:00.941+0000 7f7e81200640 1 --2- 172.21.15.131:0/2589955688 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e7c1496b0 0x7f7e7c15d320 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c00c7a0 tx=0x7f7e6c00cc60 comp rx=0 tx=0).stop 2024-05-26T22:55:00.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:00.941+0000 7f7e81200640 1 -- 172.21.15.131:0/2589955688 shutdown_connections 2024-05-26T22:55:00.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:00.941+0000 7f7e81200640 1 --2- 172.21.15.131:0/2589955688 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7e7c15d860 0x7f7e7c1f3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:00.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:00.941+0000 7f7e81200640 1 --2- 172.21.15.131:0/2589955688 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f7e5806be60 0x7f7e5806e320 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:00.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:00.941+0000 7f7e81200640 1 --2- 172.21.15.131:0/2589955688 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e7c1496b0 0x7f7e7c15d320 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:00.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:00.941+0000 7f7e81200640 1 --2- 172.21.15.131:0/2589955688 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7e60003a00 0x7f7e7c09af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:00.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:00.941+0000 7f7e81200640 1 -- 172.21.15.131:0/2589955688 >> 172.21.15.131:0/2589955688 conn(0x7f7e7c06bc90 msgr2=0x7f7e7c0cfec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:00.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:00.942+0000 7f7e81200640 1 -- 172.21.15.131:0/2589955688 shutdown_connections 2024-05-26T22:55:00.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:00.942+0000 7f7e81200640 1 -- 172.21.15.131:0/2589955688 wait complete. 2024-05-26T22:55:00.966 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-26T22:55:00.966 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-26T22:55:01.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.152+0000 7fd7bfe00640 1 Processor -- start 2024-05-26T22:55:01.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.152+0000 7fd7bfe00640 1 -- start start 2024-05-26T22:55:01.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.154+0000 7fd7bfe00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fd7c0144800 0x7fd7c015fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:01.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.154+0000 7fd7bfe00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd7c015f100 0x7fd7c00add00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:01.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.154+0000 7fd7bfe00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7c015f7b0 con 0x7fd7c00ad870 2024-05-26T22:55:01.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.154+0000 7fd7bfe00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fd7c015f570 con 0x7fd7c0144800 2024-05-26T22:55:01.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.154+0000 7fd7bfe00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd7c015ffd0 con 0x7fd7c015f100 2024-05-26T22:55:01.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.154+0000 7fd7b7e00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fd7c00ad870 0x7fd7c015e9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:54556/0 (socket says 172.21.15.131:54556) 2024-05-26T22:55:01.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.154+0000 7fd7b7e00640 1 -- 172.21.15.131:0/866471144 learned_addr learned my addr 172.21.15.131:0/866471144 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:55:01.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.154+0000 7fd7bf400640 1 --2- 172.21.15.131:0/866471144 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd7c015f100 0x7fd7c00add00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:01.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.154+0000 7fd7bea00640 1 --2- 172.21.15.131:0/866471144 >> v2:172.21.15.187:3300/0 conn(0x7fd7c0144800 0x7fd7c015fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:01.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.155+0000 7fd7be000640 1 -- 172.21.15.131:0/866471144 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1101614425 0 0) 0x7fd7c015f7b0 con 0x7fd7c00ad870 2024-05-26T22:55:01.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.155+0000 7fd7be000640 1 -- 172.21.15.131:0/866471144 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd7a4003520 con 0x7fd7c00ad870 2024-05-26T22:55:01.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.155+0000 7fd7be000640 1 -- 172.21.15.131:0/866471144 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1486354610 0 0) 0x7fd7a4003520 con 0x7fd7c00ad870 2024-05-26T22:55:01.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.155+0000 7fd7be000640 1 -- 172.21.15.131:0/866471144 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd7c015f100 msgr2=0x7fd7c00add00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:01.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.155+0000 7fd7be000640 1 --2- 172.21.15.131:0/866471144 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd7c015f100 0x7fd7c00add00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:01.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.155+0000 7fd7be000640 1 -- 172.21.15.131:0/866471144 >> v2:172.21.15.187:3300/0 conn(0x7fd7c0144800 msgr2=0x7fd7c015fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:01.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.155+0000 7fd7be000640 1 --2- 172.21.15.131:0/866471144 >> v2:172.21.15.187:3300/0 conn(0x7fd7c0144800 0x7fd7c015fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:01.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.155+0000 7fd7be000640 1 -- 172.21.15.131:0/866471144 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7c00cde40 con 0x7fd7c00ad870 2024-05-26T22:55:01.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.155+0000 7fd7be000640 1 -- 172.21.15.131:0/866471144 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7fd7a8001670 con 0x7fd7c00ad870 2024-05-26T22:55:01.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.155+0000 7fd7be000640 1 -- 172.21.15.131:0/866471144 >> v1:172.21.15.131:6789/0 conn(0x7fd7c00ad870 legacy=0x7fd7c015e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:01.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.155+0000 7fd7be000640 1 --2- 172.21.15.131:0/866471144 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd7a40039b0 0x7fd7a4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:01.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.155+0000 7fd7be000640 1 --2- 172.21.15.131:0/866471144 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd7c015f100 0x7fd7a4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:01.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.155+0000 7fd7be000640 1 --2- 172.21.15.131:0/866471144 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd7a4004d40 0x7fd7a40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:01.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.155+0000 7fd7be000640 1 -- 172.21.15.131:0/866471144 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fd7c015f570 con 0x7fd7a40039b0 2024-05-26T22:55:01.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.155+0000 7fd7be000640 1 -- 172.21.15.131:0/866471144 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd7c015ffd0 con 0x7fd7a4004d40 2024-05-26T22:55:01.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.155+0000 7fd7be000640 1 -- 172.21.15.131:0/866471144 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd7a400b9f0 con 0x7fd7c015f100 2024-05-26T22:55:01.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.156+0000 7fd7bea00640 1 --2- 172.21.15.131:0/866471144 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd7a40039b0 0x7fd7a4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:01.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.156+0000 7fd7b7e00640 1 --2- 172.21.15.131:0/866471144 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd7c015f100 0x7fd7a4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:01.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.156+0000 7fd7bf400640 1 --2- 172.21.15.131:0/866471144 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd7a4004d40 0x7fd7a40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:01.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.156+0000 7fd7bea00640 1 -- 172.21.15.131:0/866471144 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd7c015f100 msgr2=0x7fd7a4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:01.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.156+0000 7fd7bea00640 1 --2- 172.21.15.131:0/866471144 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd7c015f100 0x7fd7a4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:01.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.156+0000 7fd7bea00640 1 -- 172.21.15.131:0/866471144 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd7a4004d40 msgr2=0x7fd7a40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:01.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.156+0000 7fd7bea00640 1 --2- 172.21.15.131:0/866471144 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd7a4004d40 0x7fd7a40051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:01.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.156+0000 7fd7bea00640 1 -- 172.21.15.131:0/866471144 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7a4003520 con 0x7fd7a40039b0 2024-05-26T22:55:01.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.157+0000 7fd7bea00640 1 --2- 172.21.15.131:0/866471144 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd7a40039b0 0x7fd7a4003dd0 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7fd7b0009ac0 tx=0x7fd7a4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5abcec9667aecaad server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:01.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.157+0000 7fd7be000640 1 -- 172.21.15.131:0/866471144 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7b0036070 con 0x7fd7a40039b0 2024-05-26T22:55:01.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.157+0000 7fd7be000640 1 -- 172.21.15.131:0/866471144 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7b0031400 con 0x7fd7a40039b0 2024-05-26T22:55:01.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.157+0000 7fd7be000640 1 -- 172.21.15.131:0/866471144 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7b0009690 con 0x7fd7a40039b0 2024-05-26T22:55:01.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.158+0000 7fd7bfe00640 1 -- 172.21.15.131:0/866471144 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd7a40039b0 msgr2=0x7fd7a4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:01.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.158+0000 7fd7bfe00640 1 --2- 172.21.15.131:0/866471144 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd7a40039b0 0x7fd7a4003dd0 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7fd7b0009ac0 tx=0x7fd7a4005830 comp rx=0 tx=0).stop 2024-05-26T22:55:01.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.158+0000 7fd7bf400640 1 -- 172.21.15.131:0/866471144 reap_dead start 2024-05-26T22:55:01.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.158+0000 7fd7bfe00640 1 -- 172.21.15.131:0/866471144 shutdown_connections 2024-05-26T22:55:01.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.158+0000 7fd7bfe00640 1 -- 172.21.15.131:0/866471144 >> 172.21.15.131:0/866471144 conn(0x7fd7c00cff40 msgr2=0x7fd7c00d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:01.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.158+0000 7fd7bfe00640 1 -- 172.21.15.131:0/866471144 shutdown_connections 2024-05-26T22:55:01.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.158+0000 7fd7bfe00640 1 -- 172.21.15.131:0/866471144 wait complete. 2024-05-26T22:55:01.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.160+0000 7fd7bfe00640 1 Processor -- start 2024-05-26T22:55:01.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.160+0000 7fd7bfe00640 1 -- start start 2024-05-26T22:55:01.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.161+0000 7fd7bfe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd7a40039b0 0x7fd7c00afe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:01.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.161+0000 7fd7bfe00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd7c00b03d0 0x7fd7c015a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:01.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.161+0000 7fd7bfe00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd7c0158880 0x7fd7c015abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:01.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.161+0000 7fd7bfe00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fd7c0144be0 con 0x7fd7a40039b0 2024-05-26T22:55:01.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.161+0000 7fd7bfe00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd7c00ce400 con 0x7fd7c00b03d0 2024-05-26T22:55:01.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.161+0000 7fd7bfe00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd7c00add00 con 0x7fd7c0158880 2024-05-26T22:55:01.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.161+0000 7fd7bea00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd7a40039b0 0x7fd7c00afe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:01.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.161+0000 7fd7bea00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd7a40039b0 0x7fd7c00afe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41076/0 (socket says 172.21.15.131:41076) 2024-05-26T22:55:01.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.161+0000 7fd7bea00640 1 -- 172.21.15.131:0/3480442631 learned_addr learned my addr 172.21.15.131:0/3480442631 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:55:01.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.162+0000 7fd7bf400640 1 --2- 172.21.15.131:0/3480442631 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd7c0158880 0x7fd7c015abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:01.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.162+0000 7fd7b7e00640 1 --2- 172.21.15.131:0/3480442631 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd7c00b03d0 0x7fd7c015a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:01.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.162+0000 7fd7bea00640 1 -- 172.21.15.131:0/3480442631 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd7c0158880 msgr2=0x7fd7c015abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:01.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.162+0000 7fd7bea00640 1 --2- 172.21.15.131:0/3480442631 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd7c0158880 0x7fd7c015abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:01.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.162+0000 7fd7bea00640 1 -- 172.21.15.131:0/3480442631 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd7c00b03d0 msgr2=0x7fd7c015a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:01.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.162+0000 7fd7bea00640 1 --2- 172.21.15.131:0/3480442631 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd7c00b03d0 0x7fd7c015a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:01.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.162+0000 7fd7bea00640 1 -- 172.21.15.131:0/3480442631 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7c00b11f0 con 0x7fd7a40039b0 2024-05-26T22:55:01.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.162+0000 7fd7bf400640 1 --2- 172.21.15.131:0/3480442631 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd7c0158880 0x7fd7c015abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:55:01.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.162+0000 7fd7bea00640 1 --2- 172.21.15.131:0/3480442631 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd7a40039b0 0x7fd7c00afe90 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7fd7b0009bb0 tx=0x7fd7b0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:01.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.162+0000 7fd7b6a00640 1 -- 172.21.15.131:0/3480442631 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7b0040020 con 0x7fd7a40039b0 2024-05-26T22:55:01.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.162+0000 7fd7b6a00640 1 -- 172.21.15.131:0/3480442631 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7b0003900 con 0x7fd7a40039b0 2024-05-26T22:55:01.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.162+0000 7fd7bfe00640 1 -- 172.21.15.131:0/3480442631 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7c00b0950 con 0x7fd7a40039b0 2024-05-26T22:55:01.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.162+0000 7fd7b6a00640 1 -- 172.21.15.131:0/3480442631 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7b0036040 con 0x7fd7a40039b0 2024-05-26T22:55:01.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.163+0000 7fd7bfe00640 1 -- 172.21.15.131:0/3480442631 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7c00b0f10 con 0x7fd7a40039b0 2024-05-26T22:55:01.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.164+0000 7fd7bfe00640 1 -- 172.21.15.131:0/3480442631 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7c01437c0 con 0x7fd7a40039b0 2024-05-26T22:55:01.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.165+0000 7fd7b6a00640 1 -- 172.21.15.131:0/3480442631 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd7b0043020 con 0x7fd7a40039b0 2024-05-26T22:55:01.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.166+0000 7fd7b6a00640 1 --2- 172.21.15.131:0/3480442631 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fd79c06be60 0x7fd79c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:01.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.166+0000 7fd7b6a00640 1 -- 172.21.15.131:0/3480442631 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 5709+0+0 (secure 0 0 0) 0x7fd7b00ab1e0 con 0x7fd7a40039b0 2024-05-26T22:55:01.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.166+0000 7fd7b7e00640 1 --2- 172.21.15.131:0/3480442631 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fd79c06be60 0x7fd79c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:01.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.167+0000 7fd7b7e00640 1 --2- 172.21.15.131:0/3480442631 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fd79c06be60 0x7fd79c06e320 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fd7c00b5c00 tx=0x7fd7a8001430 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:01.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.170+0000 7fd7b6a00640 1 -- 172.21.15.131:0/3480442631 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fd7c01437c0 con 0x7fd7a40039b0 2024-05-26T22:55:01.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.407+0000 7fd7bfe00640 1 -- 172.21.15.131:0/3480442631 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd7c0068650 con 0x7fd7a40039b0 2024-05-26T22:55:01.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.408+0000 7fd7b6a00640 1 -- 172.21.15.131:0/3480442631 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) ==== 94+0+5189 (secure 0 0 0) 0x7fd7c0068650 con 0x7fd7a40039b0 2024-05-26T22:55:01.408 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:55:01.408 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":94,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5231,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/2037149126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6835","nonce":2037149126},{"type":"v1","addr":"172.21.15.131:6837","nonce":2037149126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":89},{"gid":5243,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1314245822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1314245822},{"type":"v1","addr":"172.21.15.187:6833","nonce":1314245822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":89},{"gid":5246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/521878375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":521878375},{"type":"v1","addr":"172.21.15.187:6835","nonce":521878375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":89}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-26T22:54:57.328457+0000","modified":"2024-05-26T22:55:00.415141+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7863},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7863":{"gid":7863,"name":"a","rank":0,"incarnation":90,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6836/789312503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":789312503},{"type":"v1","addr":"172.21.15.131:6836","nonce":789312503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7863,"qdb_cluster":[7863]},"id":7}]} 2024-05-26T22:55:01.408 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 94 2024-05-26T22:55:01.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.410+0000 7fd7bfe00640 1 -- 172.21.15.131:0/3480442631 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fd79c06be60 msgr2=0x7fd79c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:01.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.410+0000 7fd7bfe00640 1 --2- 172.21.15.131:0/3480442631 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fd79c06be60 0x7fd79c06e320 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fd7c00b5c00 tx=0x7fd7a8001430 comp rx=0 tx=0).stop 2024-05-26T22:55:01.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.410+0000 7fd7bfe00640 1 -- 172.21.15.131:0/3480442631 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd7a40039b0 msgr2=0x7fd7c00afe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:01.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.410+0000 7fd7bfe00640 1 --2- 172.21.15.131:0/3480442631 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd7a40039b0 0x7fd7c00afe90 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7fd7b0009bb0 tx=0x7fd7b0004730 comp rx=0 tx=0).stop 2024-05-26T22:55:01.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.411+0000 7fd7bfe00640 1 -- 172.21.15.131:0/3480442631 shutdown_connections 2024-05-26T22:55:01.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.411+0000 7fd7bfe00640 1 --2- 172.21.15.131:0/3480442631 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd7c0158880 0x7fd7c015abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:01.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.411+0000 7fd7bfe00640 1 --2- 172.21.15.131:0/3480442631 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fd79c06be60 0x7fd79c06e320 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:01.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.411+0000 7fd7bfe00640 1 --2- 172.21.15.131:0/3480442631 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd7c00b03d0 0x7fd7c015a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:01.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.411+0000 7fd7bfe00640 1 --2- 172.21.15.131:0/3480442631 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd7a40039b0 0x7fd7c00afe90 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:01.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.411+0000 7fd7bfe00640 1 -- 172.21.15.131:0/3480442631 >> 172.21.15.131:0/3480442631 conn(0x7fd7c00cff40 msgr2=0x7fd7c0071060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:01.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.411+0000 7fd7bfe00640 1 -- 172.21.15.131:0/3480442631 shutdown_connections 2024-05-26T22:55:01.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.411+0000 7fd7bfe00640 1 -- 172.21.15.131:0/3480442631 wait complete. 2024-05-26T22:55:01.429 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-26T22:55:01.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.622+0000 7f7fbb200640 1 Processor -- start 2024-05-26T22:55:01.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.622+0000 7f7fbb200640 1 -- start start 2024-05-26T22:55:01.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.624+0000 7f7fbb200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f7fbc156750 0x7f7fbc15fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:01.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.624+0000 7f7fbb200640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7fbc160620 con 0x7f7fbc06e8c0 2024-05-26T22:55:01.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.624+0000 7f7fbb200640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7fbc160900 con 0x7f7fbc0cc7f0 2024-05-26T22:55:01.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.624+0000 7f7fbb200640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f7fbc15d770 con 0x7f7fbc156750 2024-05-26T22:55:01.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.624+0000 7f7fb9e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f7fbc156750 0x7f7fbc15fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:01.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.624+0000 7f7fb9400640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f7fbc06e8c0 0x7f7fbc15d060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:54562/0 (socket says 172.21.15.131:54562) 2024-05-26T22:55:01.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.624+0000 7f7fb9400640 1 -- 172.21.15.131:0/1869681673 learned_addr learned my addr 172.21.15.131:0/1869681673 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:55:01.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.625+0000 7f7fb9e00640 1 -- 172.21.15.131:0/1869681673 >> v1:172.21.15.187:6789/0 conn(0x7f7fbc0cc7f0 legacy=0x7f7fbc0cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:01.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.625+0000 7f7fb9e00640 1 -- 172.21.15.131:0/1869681673 >> v1:172.21.15.131:6789/0 conn(0x7f7fbc06e8c0 legacy=0x7f7fbc15d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:01.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.625+0000 7f7fb9e00640 1 -- 172.21.15.131:0/1869681673 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fbc160be0 con 0x7f7fbc156750 2024-05-26T22:55:01.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.625+0000 7f7fb8a00640 1 -- 172.21.15.131:0/1869681673 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3873849991 0 0) 0x7f7fbc160620 con 0x7f7fbc06e8c0 2024-05-26T22:55:01.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.626+0000 7f7fb9e00640 1 --2- 172.21.15.131:0/1869681673 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7fbc156750 0x7f7fbc15fd40 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f7fac0097a0 tx=0x7f7fac02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=100dcf90b5354363 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:01.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.626+0000 7f7fb8a00640 1 -- 172.21.15.131:0/1869681673 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fac037070 con 0x7f7fbc156750 2024-05-26T22:55:01.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.626+0000 7f7fb8a00640 1 -- 172.21.15.131:0/1869681673 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fac032440 con 0x7f7fbc156750 2024-05-26T22:55:01.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.626+0000 7f7fb8a00640 1 -- 172.21.15.131:0/1869681673 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fac032810 con 0x7f7fbc156750 2024-05-26T22:55:01.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.627+0000 7f7fbb200640 1 -- 172.21.15.131:0/1869681673 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7fbc156750 msgr2=0x7f7fbc15fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:01.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.627+0000 7f7fbb200640 1 --2- 172.21.15.131:0/1869681673 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7fbc156750 0x7f7fbc15fd40 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f7fac0097a0 tx=0x7f7fac02a1e0 comp rx=0 tx=0).stop 2024-05-26T22:55:01.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.627+0000 7f7fbb200640 1 -- 172.21.15.131:0/1869681673 shutdown_connections 2024-05-26T22:55:01.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.627+0000 7f7fbb200640 1 --2- 172.21.15.131:0/1869681673 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7fbc156750 0x7f7fbc15fd40 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:01.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.627+0000 7f7fbb200640 1 -- 172.21.15.131:0/1869681673 >> 172.21.15.131:0/1869681673 conn(0x7f7fbc0cff40 msgr2=0x7f7fbc0d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:01.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.627+0000 7f7fbb200640 1 -- 172.21.15.131:0/1869681673 shutdown_connections 2024-05-26T22:55:01.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.627+0000 7f7fbb200640 1 -- 172.21.15.131:0/1869681673 wait complete. 2024-05-26T22:55:01.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.629+0000 7f7fbb200640 1 Processor -- start 2024-05-26T22:55:01.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.629+0000 7f7fbb200640 1 -- start start 2024-05-26T22:55:01.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.630+0000 7f7fbb200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7fbc06e8c0 0x7f7fbc0b0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:01.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.630+0000 7f7fbb200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7fbc0cc7f0 0x7f7fbc156cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:01.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.630+0000 7f7fbb200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7fbc156750 0x7f7fbc1f4840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:01.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.630+0000 7f7fbb200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f7fbc156b30 con 0x7f7fbc06e8c0 2024-05-26T22:55:01.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.630+0000 7f7fbb200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7fbc15cbd0 con 0x7f7fbc156750 2024-05-26T22:55:01.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.630+0000 7f7fbb200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f7fbc160280 con 0x7f7fbc0cc7f0 2024-05-26T22:55:01.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.630+0000 7f7fb9e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7fbc06e8c0 0x7f7fbc0b0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:01.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.630+0000 7f7fb9e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7fbc06e8c0 0x7f7fbc0b0420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41082/0 (socket says 172.21.15.131:41082) 2024-05-26T22:55:01.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.630+0000 7f7fb9e00640 1 -- 172.21.15.131:0/3522037470 learned_addr learned my addr 172.21.15.131:0/3522037470 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:55:01.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.631+0000 7f7fba800640 1 --2- 172.21.15.131:0/3522037470 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7fbc156750 0x7f7fbc1f4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:01.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.631+0000 7f7fb9e00640 1 -- 172.21.15.131:0/3522037470 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7fbc0cc7f0 msgr2=0x7f7fbc156cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:01.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.631+0000 7f7fb9e00640 1 --2- 172.21.15.131:0/3522037470 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7fbc0cc7f0 0x7f7fbc156cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:01.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.631+0000 7f7fb9e00640 1 -- 172.21.15.131:0/3522037470 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7fbc156750 msgr2=0x7f7fbc1f4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:01.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.631+0000 7f7fb9e00640 1 --2- 172.21.15.131:0/3522037470 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7fbc156750 0x7f7fbc1f4840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:01.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.631+0000 7f7fb9e00640 1 -- 172.21.15.131:0/3522037470 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fbc0afbb0 con 0x7f7fbc06e8c0 2024-05-26T22:55:01.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.631+0000 7f7fb9e00640 1 --2- 172.21.15.131:0/3522037470 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7fbc06e8c0 0x7f7fbc0b0420 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f7fac0098c0 tx=0x7f7fac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:01.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.631+0000 7f7fb2000640 1 -- 172.21.15.131:0/3522037470 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fac041070 con 0x7f7fbc06e8c0 2024-05-26T22:55:01.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.632+0000 7f7fb2000640 1 -- 172.21.15.131:0/3522037470 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fac03c870 con 0x7f7fbc06e8c0 2024-05-26T22:55:01.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.632+0000 7f7fb2000640 1 -- 172.21.15.131:0/3522037470 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fac037040 con 0x7f7fbc06e8c0 2024-05-26T22:55:01.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.632+0000 7f7fbb200640 1 -- 172.21.15.131:0/3522037470 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fbc0afe90 con 0x7f7fbc06e8c0 2024-05-26T22:55:01.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.632+0000 7f7fbb200640 1 -- 172.21.15.131:0/3522037470 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fbc0b2170 con 0x7f7fbc06e8c0 2024-05-26T22:55:01.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.633+0000 7f7fbb200640 1 -- 172.21.15.131:0/3522037470 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fbc0694e0 con 0x7f7fbc06e8c0 2024-05-26T22:55:01.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.634+0000 7f7fb2000640 1 -- 172.21.15.131:0/3522037470 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7fac044070 con 0x7f7fbc06e8c0 2024-05-26T22:55:01.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.634+0000 7f7fb2000640 1 --2- 172.21.15.131:0/3522037470 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f7f9c06be30 0x7f7f9c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:01.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.635+0000 7f7fb2000640 1 -- 172.21.15.131:0/3522037470 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 5709+0+0 (secure 0 0 0) 0x7f7fac0abac0 con 0x7f7fbc06e8c0 2024-05-26T22:55:01.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.635+0000 7f7fb9400640 1 --2- 172.21.15.131:0/3522037470 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f7f9c06be30 0x7f7f9c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:01.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.635+0000 7f7fb9400640 1 --2- 172.21.15.131:0/3522037470 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f7f9c06be30 0x7f7f9c06e2f0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f7fa8007fd0 tx=0x7f7fa8007920 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:01.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.639+0000 7f7fb2000640 1 -- 172.21.15.131:0/3522037470 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f7fac07a580 con 0x7f7fbc06e8c0 2024-05-26T22:55:01.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.838+0000 7f7fbb200640 1 -- 172.21.15.131:0/3522037470 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7fbc0b5e80 con 0x7f7fbc06e8c0 2024-05-26T22:55:01.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.840+0000 7f7fb2000640 1 -- 172.21.15.131:0/3522037470 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) ==== 74+0+16632 (secure 0 0 0) 0x7f7fac079e80 con 0x7f7fbc06e8c0 2024-05-26T22:55:01.840 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:55:01.840 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":95,"fsid":"46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f","created":"2024-05-26T22:47:57.288905+0000","modified":"2024-05-26T22:54:58.901569+0000","last_up_change":"2024-05-26T22:48:09.602207+0000","last_in_change":"2024-05-26T22:48:00.687341+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-26T22:48:10.463339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-05-26T22:54:55.120321+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-05-26T22:54:56.346430+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"814eaf62-a18d-494b-8e27-fd8fb88adea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6801","nonce":2350940051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6803","nonce":2350940051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6807","nonce":2350940051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6805","nonce":2350940051}]},"public_addr":"172.21.15.131:6801/2350940051","cluster_addr":"172.21.15.131:6803/2350940051","heartbeat_back_addr":"172.21.15.131:6807/2350940051","heartbeat_front_addr":"172.21.15.131:6805/2350940051","state":["exists","up"]},{"osd":1,"uuid":"42e66287-f6c6-4e26-af3c-6dfaa3536d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6825","nonce":1525528247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6827","nonce":1525528247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6831","nonce":1525528247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6829","nonce":1525528247}]},"public_addr":"172.21.15.131:6825/1525528247","cluster_addr":"172.21.15.131:6827/1525528247","heartbeat_back_addr":"172.21.15.131:6831/1525528247","heartbeat_front_addr":"172.21.15.131:6829/1525528247","state":["exists","up"]},{"osd":2,"uuid":"0da34673-04a2-4fd0-b57a-0ff2b679b043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6817","nonce":3627842210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6819","nonce":3627842210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6823","nonce":3627842210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6821","nonce":3627842210}]},"public_addr":"172.21.15.131:6817/3627842210","cluster_addr":"172.21.15.131:6819/3627842210","heartbeat_back_addr":"172.21.15.131:6823/3627842210","heartbeat_front_addr":"172.21.15.131:6821/3627842210","state":["exists","up"]},{"osd":3,"uuid":"cf117a27-d6b2-43d3-ae52-b47988bf8eed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6809","nonce":350790591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6811","nonce":350790591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6815","nonce":350790591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6813","nonce":350790591}]},"public_addr":"172.21.15.131:6809/350790591","cluster_addr":"172.21.15.131:6811/350790591","heartbeat_back_addr":"172.21.15.131:6815/350790591","heartbeat_front_addr":"172.21.15.131:6813/350790591","state":["exists","up"]},{"osd":4,"uuid":"a092ccfc-be89-4a84-88d5-0f781fa715eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6816","nonce":244560995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6819","nonce":244560995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6823","nonce":244560995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6821","nonce":244560995}]},"public_addr":"172.21.15.187:6816/244560995","cluster_addr":"172.21.15.187:6819/244560995","heartbeat_back_addr":"172.21.15.187:6823/244560995","heartbeat_front_addr":"172.21.15.187:6821/244560995","state":["exists","up"]},{"osd":5,"uuid":"7513ba69-dd00-4a35-b190-9e8cc3b2bdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6801","nonce":324619398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6803","nonce":324619398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6812","nonce":324619398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6807","nonce":324619398}]},"public_addr":"172.21.15.187:6801/324619398","cluster_addr":"172.21.15.187:6803/324619398","heartbeat_back_addr":"172.21.15.187:6812/324619398","heartbeat_front_addr":"172.21.15.187:6807/324619398","state":["exists","up"]},{"osd":6,"uuid":"50081810-c093-4d2f-a4d7-b7c695c5400d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6806","nonce":1391347786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6809","nonce":1391347786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6818","nonce":1391347786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6813","nonce":1391347786}]},"public_addr":"172.21.15.187:6806/1391347786","cluster_addr":"172.21.15.187:6809/1391347786","heartbeat_back_addr":"172.21.15.187:6818/1391347786","heartbeat_front_addr":"172.21.15.187:6813/1391347786","state":["exists","up"]},{"osd":7,"uuid":"775dae05-54f5-4ad4-bbce-92863c6ed2ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6825","nonce":1220065622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6827","nonce":1220065622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6831","nonce":1220065622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6829","nonce":1220065622}]},"public_addr":"172.21.15.187:6825/1220065622","cluster_addr":"172.21.15.187:6827/1220065622","heartbeat_back_addr":"172.21.15.187:6831/1220065622","heartbeat_front_addr":"172.21.15.187:6829/1220065622","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:05.983520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.299516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.067323+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.307748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.018742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.258775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.010912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.295355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-26T22:55:01.843 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.842+0000 7f7fbb200640 1 -- 172.21.15.131:0/3522037470 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f7f9c06be30 msgr2=0x7f7f9c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:01.843 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.842+0000 7f7fbb200640 1 --2- 172.21.15.131:0/3522037470 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f7f9c06be30 0x7f7f9c06e2f0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f7fa8007fd0 tx=0x7f7fa8007920 comp rx=0 tx=0).stop 2024-05-26T22:55:01.843 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.843+0000 7f7fbb200640 1 -- 172.21.15.131:0/3522037470 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7fbc06e8c0 msgr2=0x7f7fbc0b0420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:01.843 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.843+0000 7f7fbb200640 1 --2- 172.21.15.131:0/3522037470 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7fbc06e8c0 0x7f7fbc0b0420 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f7fac0098c0 tx=0x7f7fac0046c0 comp rx=0 tx=0).stop 2024-05-26T22:55:01.843 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.843+0000 7f7fbb200640 1 -- 172.21.15.131:0/3522037470 shutdown_connections 2024-05-26T22:55:01.843 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.843+0000 7f7fbb200640 1 --2- 172.21.15.131:0/3522037470 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7fbc0cc7f0 0x7f7fbc156cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:01.843 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.843+0000 7f7fbb200640 1 --2- 172.21.15.131:0/3522037470 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f7f9c06be30 0x7f7f9c06e2f0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:01.843 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.843+0000 7f7fbb200640 1 --2- 172.21.15.131:0/3522037470 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7fbc156750 0x7f7fbc1f4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:01.843 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.843+0000 7f7fbb200640 1 --2- 172.21.15.131:0/3522037470 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7fbc06e8c0 0x7f7fbc0b0420 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:01.843 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.843+0000 7f7fbb200640 1 -- 172.21.15.131:0/3522037470 >> 172.21.15.131:0/3522037470 conn(0x7f7fbc0cff40 msgr2=0x7f7fbc15b140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:01.843 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.843+0000 7f7fbb200640 1 -- 172.21.15.131:0/3522037470 shutdown_connections 2024-05-26T22:55:01.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:01.843+0000 7f7fbb200640 1 -- 172.21.15.131:0/3522037470 wait complete. 2024-05-26T22:55:01.861 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-26T22:55:01.861 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-26T22:55:02.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.060+0000 7fc255e00640 1 Processor -- start 2024-05-26T22:55:02.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.060+0000 7fc255e00640 1 -- start start 2024-05-26T22:55:02.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.061+0000 7fc255e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fc250155840 0x7fc250071930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:02.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.061+0000 7fc255e00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fc25015c3d0 0x7fc25006ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:02.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.061+0000 7fc255e00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2500722a0 con 0x7fc25006f490 2024-05-26T22:55:02.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.061+0000 7fc255e00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fc250072580 con 0x7fc250155840 2024-05-26T22:55:02.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.061+0000 7fc255e00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fc250072860 con 0x7fc25015c3d0 2024-05-26T22:55:02.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.062+0000 7fc24ea00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fc250155840 0x7fc250071930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:02.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.062+0000 7fc24ea00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fc250155840 0x7fc250071930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41084/0 (socket says 172.21.15.131:41084) 2024-05-26T22:55:02.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.062+0000 7fc24ea00640 1 -- 172.21.15.131:0/650292905 learned_addr learned my addr 172.21.15.131:0/650292905 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:55:02.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.062+0000 7fc24e000640 1 --2- 172.21.15.131:0/650292905 >> v2:172.21.15.187:3300/0 conn(0x7fc25015c3d0 0x7fc25006ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:02.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.062+0000 7fc24ea00640 1 -- 172.21.15.131:0/650292905 >> v2:172.21.15.187:3300/0 conn(0x7fc25015c3d0 msgr2=0x7fc25006ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:02.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.062+0000 7fc24ea00640 1 --2- 172.21.15.131:0/650292905 >> v2:172.21.15.187:3300/0 conn(0x7fc25015c3d0 0x7fc25006ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:02.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.062+0000 7fc24ea00640 1 -- 172.21.15.131:0/650292905 >> v1:172.21.15.187:6789/0 conn(0x7fc25006f490 legacy=0x7fc25015ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:02.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.062+0000 7fc24ea00640 1 -- 172.21.15.131:0/650292905 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc250072b40 con 0x7fc250155840 2024-05-26T22:55:02.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.062+0000 7fc24e000640 1 --2- 172.21.15.131:0/650292905 >> v2:172.21.15.187:3300/0 conn(0x7fc25015c3d0 0x7fc25006ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:55:02.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.063+0000 7fc24ea00640 1 --2- 172.21.15.131:0/650292905 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc250155840 0x7fc250071930 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7fc250159280 tx=0x7fc24002c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ed23ad1fcdfde90 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:02.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.064+0000 7fc24d600640 1 -- 172.21.15.131:0/650292905 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2400325a0 con 0x7fc250155840 2024-05-26T22:55:02.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.064+0000 7fc24d600640 1 -- 172.21.15.131:0/650292905 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc240038070 con 0x7fc250155840 2024-05-26T22:55:02.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.064+0000 7fc24d600640 1 -- 172.21.15.131:0/650292905 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc240032960 con 0x7fc250155840 2024-05-26T22:55:02.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.064+0000 7fc255e00640 1 -- 172.21.15.131:0/650292905 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc250155840 msgr2=0x7fc250071930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:02.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.064+0000 7fc255e00640 1 --2- 172.21.15.131:0/650292905 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc250155840 0x7fc250071930 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7fc250159280 tx=0x7fc24002c8e0 comp rx=0 tx=0).stop 2024-05-26T22:55:02.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.064+0000 7fc255e00640 1 -- 172.21.15.131:0/650292905 shutdown_connections 2024-05-26T22:55:02.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.064+0000 7fc255e00640 1 --2- 172.21.15.131:0/650292905 >> v2:172.21.15.187:3300/0 conn(0x7fc25015c3d0 0x7fc25006ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:02.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.064+0000 7fc255e00640 1 --2- 172.21.15.131:0/650292905 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc250155840 0x7fc250071930 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:02.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.064+0000 7fc255e00640 1 -- 172.21.15.131:0/650292905 >> 172.21.15.131:0/650292905 conn(0x7fc2500b1e90 msgr2=0x7fc2500b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:02.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.065+0000 7fc255e00640 1 -- 172.21.15.131:0/650292905 shutdown_connections 2024-05-26T22:55:02.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.065+0000 7fc255e00640 1 -- 172.21.15.131:0/650292905 wait complete. 2024-05-26T22:55:02.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.066+0000 7fc255e00640 1 Processor -- start 2024-05-26T22:55:02.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.066+0000 7fc255e00640 1 -- start start 2024-05-26T22:55:02.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.067+0000 7fc255e00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc25006f490 0x7fc25014b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:02.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.067+0000 7fc255e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc250155840 0x7fc2501f3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:02.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.067+0000 7fc255e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc25015c3d0 0x7fc25015a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:02.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.067+0000 7fc255e00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc250071ff0 con 0x7fc250155840 2024-05-26T22:55:02.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.067+0000 7fc255e00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc250071e70 con 0x7fc25006f490 2024-05-26T22:55:02.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.067+0000 7fc255e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc2500aee50 con 0x7fc25015c3d0 2024-05-26T22:55:02.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.067+0000 7fc24e000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc250155840 0x7fc2501f3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:02.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.068+0000 7fc24e000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc250155840 0x7fc2501f3fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41090/0 (socket says 172.21.15.131:41090) 2024-05-26T22:55:02.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.068+0000 7fc24e000640 1 -- 172.21.15.131:0/436414284 learned_addr learned my addr 172.21.15.131:0/436414284 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:55:02.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.068+0000 7fc24ea00640 1 --2- 172.21.15.131:0/436414284 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc25006f490 0x7fc25014b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:02.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.068+0000 7fc24f400640 1 --2- 172.21.15.131:0/436414284 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc25015c3d0 0x7fc25015a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:02.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.068+0000 7fc24e000640 1 -- 172.21.15.131:0/436414284 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc25015c3d0 msgr2=0x7fc25015a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:02.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.068+0000 7fc24e000640 1 --2- 172.21.15.131:0/436414284 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc25015c3d0 0x7fc25015a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:02.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.068+0000 7fc24e000640 1 -- 172.21.15.131:0/436414284 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc25006f490 msgr2=0x7fc25014b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:02.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.068+0000 7fc24e000640 1 --2- 172.21.15.131:0/436414284 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc25006f490 0x7fc25014b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:02.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.068+0000 7fc24e000640 1 -- 172.21.15.131:0/436414284 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2501f4a90 con 0x7fc250155840 2024-05-26T22:55:02.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.068+0000 7fc24e000640 1 --2- 172.21.15.131:0/436414284 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc250155840 0x7fc2501f3fd0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7fc23c00c7f0 tx=0x7fc23c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:02.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.069+0000 7fc247400640 1 -- 172.21.15.131:0/436414284 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc23c015070 con 0x7fc250155840 2024-05-26T22:55:02.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.069+0000 7fc247400640 1 -- 172.21.15.131:0/436414284 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc23c007820 con 0x7fc250155840 2024-05-26T22:55:02.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.069+0000 7fc247400640 1 -- 172.21.15.131:0/436414284 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc23c007bd0 con 0x7fc250155840 2024-05-26T22:55:02.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.069+0000 7fc255e00640 1 -- 172.21.15.131:0/436414284 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2501f4d70 con 0x7fc250155840 2024-05-26T22:55:02.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.069+0000 7fc255e00640 1 -- 172.21.15.131:0/436414284 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2501f5330 con 0x7fc250155840 2024-05-26T22:55:02.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.070+0000 7fc255e00640 1 -- 172.21.15.131:0/436414284 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2500694e0 con 0x7fc250155840 2024-05-26T22:55:02.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.071+0000 7fc247400640 1 -- 172.21.15.131:0/436414284 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc23c018070 con 0x7fc250155840 2024-05-26T22:55:02.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.072+0000 7fc247400640 1 --2- 172.21.15.131:0/436414284 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc23006be30 0x7fc23006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:02.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.072+0000 7fc247400640 1 -- 172.21.15.131:0/436414284 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 5709+0+0 (secure 0 0 0) 0x7fc23c08b070 con 0x7fc250155840 2024-05-26T22:55:02.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.072+0000 7fc24ea00640 1 --2- 172.21.15.131:0/436414284 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc23006be30 0x7fc23006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:02.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.073+0000 7fc24ea00640 1 --2- 172.21.15.131:0/436414284 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc23006be30 0x7fc23006e2f0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fc240037000 tx=0x7fc240032000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:02.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.076+0000 7fc247400640 1 -- 172.21.15.131:0/436414284 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fc23c059980 con 0x7fc250155840 2024-05-26T22:55:02.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.273+0000 7fc255e00640 1 -- 172.21.15.131:0/436414284 --> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc25015e770 con 0x7fc23006be30 2024-05-26T22:55:02.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.292+0000 7fc247400640 1 -- 172.21.15.131:0/436414284 <== mgr.4104 v2:172.21.15.131:6832/3484832947 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252084 (secure 0 0 0) 0x7fc25015e770 con 0x7fc23006be30 2024-05-26T22:55:02.296 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:55:02.298 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-05-26T22:55:02.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.299+0000 7fc255e00640 1 -- 172.21.15.131:0/436414284 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc23006be30 msgr2=0x7fc23006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:02.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.299+0000 7fc255e00640 1 --2- 172.21.15.131:0/436414284 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc23006be30 0x7fc23006e2f0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fc240037000 tx=0x7fc240032000 comp rx=0 tx=0).stop 2024-05-26T22:55:02.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.299+0000 7fc255e00640 1 -- 172.21.15.131:0/436414284 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc250155840 msgr2=0x7fc2501f3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:02.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.299+0000 7fc255e00640 1 --2- 172.21.15.131:0/436414284 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc250155840 0x7fc2501f3fd0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7fc23c00c7f0 tx=0x7fc23c00ccb0 comp rx=0 tx=0).stop 2024-05-26T22:55:02.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.299+0000 7fc255e00640 1 -- 172.21.15.131:0/436414284 shutdown_connections 2024-05-26T22:55:02.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.299+0000 7fc255e00640 1 --2- 172.21.15.131:0/436414284 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc25015c3d0 0x7fc25015a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:02.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.299+0000 7fc255e00640 1 --2- 172.21.15.131:0/436414284 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc23006be30 0x7fc23006e2f0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:02.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.299+0000 7fc255e00640 1 --2- 172.21.15.131:0/436414284 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc250155840 0x7fc2501f3fd0 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:02.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.299+0000 7fc255e00640 1 --2- 172.21.15.131:0/436414284 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc25006f490 0x7fc25014b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:02.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.299+0000 7fc255e00640 1 -- 172.21.15.131:0/436414284 >> 172.21.15.131:0/436414284 conn(0x7fc2500b1e90 msgr2=0x7fc2500b0d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:02.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.299+0000 7fc255e00640 1 -- 172.21.15.131:0/436414284 shutdown_connections 2024-05-26T22:55:02.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.300+0000 7fc255e00640 1 -- 172.21.15.131:0/436414284 wait complete. 2024-05-26T22:55:02.318 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":293,"stamp":"2024-05-26T22:55:00.516825+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":139,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":259888,"kb_used_data":10408,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754714832,"statfs":{"total":773094113280,"available":772827987968,"internally_reserved":0,"allocated":10657792,"data_stored":5270060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.657078"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.468848+0000","last_change":"2024-05-26T22:54:57.918186+0000","last_active":"2024-05-26T22:54:59.468848+0000","last_peered":"2024-05-26T22:54:59.468848+0000","last_clean":"2024-05-26T22:54:59.468848+0000","last_became_active":"2024-05-26T22:54:57.914078+0000","last_became_peered":"2024-05-26T22:54:57.914078+0000","last_unstale":"2024-05-26T22:54:59.468848+0000","last_undegraded":"2024-05-26T22:54:59.468848+0000","last_fullsized":"2024-05-26T22:54:59.468848+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:17:44.544345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.469251+0000","last_change":"2024-05-26T22:54:57.919704+0000","last_active":"2024-05-26T22:54:59.469251+0000","last_peered":"2024-05-26T22:54:59.469251+0000","last_clean":"2024-05-26T22:54:59.469251+0000","last_became_active":"2024-05-26T22:54:57.918561+0000","last_became_peered":"2024-05-26T22:54:57.918561+0000","last_unstale":"2024-05-26T22:54:59.469251+0000","last_undegraded":"2024-05-26T22:54:59.469251+0000","last_fullsized":"2024-05-26T22:54:59.469251+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:23:02.876957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.467359+0000","last_change":"2024-05-26T22:54:57.916253+0000","last_active":"2024-05-26T22:54:59.467359+0000","last_peered":"2024-05-26T22:54:59.467359+0000","last_clean":"2024-05-26T22:54:59.467359+0000","last_became_active":"2024-05-26T22:54:57.915974+0000","last_became_peered":"2024-05-26T22:54:57.915974+0000","last_unstale":"2024-05-26T22:54:59.467359+0000","last_undegraded":"2024-05-26T22:54:59.467359+0000","last_fullsized":"2024-05-26T22:54:59.467359+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:03:18.231409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:58.918267+0000","last_change":"2024-05-26T22:54:57.936364+0000","last_active":"2024-05-26T22:54:58.918267+0000","last_peered":"2024-05-26T22:54:58.918267+0000","last_clean":"2024-05-26T22:54:58.918267+0000","last_became_active":"2024-05-26T22:54:57.936056+0000","last_became_peered":"2024-05-26T22:54:57.936056+0000","last_unstale":"2024-05-26T22:54:58.918267+0000","last_undegraded":"2024-05-26T22:54:58.918267+0000","last_fullsized":"2024-05-26T22:54:58.918267+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:34:05.444741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:58.922617+0000","last_change":"2024-05-26T22:54:57.920124+0000","last_active":"2024-05-26T22:54:58.922617+0000","last_peered":"2024-05-26T22:54:58.922617+0000","last_clean":"2024-05-26T22:54:58.922617+0000","last_became_active":"2024-05-26T22:54:57.918447+0000","last_became_peered":"2024-05-26T22:54:57.918447+0000","last_unstale":"2024-05-26T22:54:58.922617+0000","last_undegraded":"2024-05-26T22:54:58.922617+0000","last_fullsized":"2024-05-26T22:54:58.922617+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:17:20.674336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.468249+0000","last_change":"2024-05-26T22:54:57.925304+0000","last_active":"2024-05-26T22:54:59.468249+0000","last_peered":"2024-05-26T22:54:59.468249+0000","last_clean":"2024-05-26T22:54:59.468249+0000","last_became_active":"2024-05-26T22:54:57.924992+0000","last_became_peered":"2024-05-26T22:54:57.924992+0000","last_unstale":"2024-05-26T22:54:59.468249+0000","last_undegraded":"2024-05-26T22:54:59.468249+0000","last_fullsized":"2024-05-26T22:54:59.468249+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:08:38.719400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.468182+0000","last_change":"2024-05-26T22:54:57.924242+0000","last_active":"2024-05-26T22:54:59.468182+0000","last_peered":"2024-05-26T22:54:59.468182+0000","last_clean":"2024-05-26T22:54:59.468182+0000","last_became_active":"2024-05-26T22:54:57.923117+0000","last_became_peered":"2024-05-26T22:54:57.923117+0000","last_unstale":"2024-05-26T22:54:59.468182+0000","last_undegraded":"2024-05-26T22:54:59.468182+0000","last_fullsized":"2024-05-26T22:54:59.468182+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:18:01.132850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.468125+0000","last_change":"2024-05-26T22:54:57.913255+0000","last_active":"2024-05-26T22:54:59.468125+0000","last_peered":"2024-05-26T22:54:59.468125+0000","last_clean":"2024-05-26T22:54:59.468125+0000","last_became_active":"2024-05-26T22:54:57.912879+0000","last_became_peered":"2024-05-26T22:54:57.912879+0000","last_unstale":"2024-05-26T22:54:59.468125+0000","last_undegraded":"2024-05-26T22:54:59.468125+0000","last_fullsized":"2024-05-26T22:54:59.468125+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:13:37.825933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.149451+0000","last_change":"2024-05-26T22:54:57.933121+0000","last_active":"2024-05-26T22:54:59.149451+0000","last_peered":"2024-05-26T22:54:59.149451+0000","last_clean":"2024-05-26T22:54:59.149451+0000","last_became_active":"2024-05-26T22:54:57.932442+0000","last_became_peered":"2024-05-26T22:54:57.932442+0000","last_unstale":"2024-05-26T22:54:59.149451+0000","last_undegraded":"2024-05-26T22:54:59.149451+0000","last_fullsized":"2024-05-26T22:54:59.149451+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:08:38.719400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:58.923347+0000","last_change":"2024-05-26T22:54:57.908115+0000","last_active":"2024-05-26T22:54:58.923347+0000","last_peered":"2024-05-26T22:54:58.923347+0000","last_clean":"2024-05-26T22:54:58.923347+0000","last_became_active":"2024-05-26T22:54:57.907450+0000","last_became_peered":"2024-05-26T22:54:57.907450+0000","last_unstale":"2024-05-26T22:54:58.923347+0000","last_undegraded":"2024-05-26T22:54:58.923347+0000","last_fullsized":"2024-05-26T22:54:58.923347+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:02:32.509633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.468753+0000","last_change":"2024-05-26T22:54:57.926696+0000","last_active":"2024-05-26T22:54:59.468753+0000","last_peered":"2024-05-26T22:54:59.468753+0000","last_clean":"2024-05-26T22:54:59.468753+0000","last_became_active":"2024-05-26T22:54:57.925819+0000","last_became_peered":"2024-05-26T22:54:57.925819+0000","last_unstale":"2024-05-26T22:54:59.468753+0000","last_undegraded":"2024-05-26T22:54:59.468753+0000","last_fullsized":"2024-05-26T22:54:59.468753+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:45:18.352727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.148734+0000","last_change":"2024-05-26T22:54:57.931493+0000","last_active":"2024-05-26T22:54:59.148734+0000","last_peered":"2024-05-26T22:54:59.148734+0000","last_clean":"2024-05-26T22:54:59.148734+0000","last_became_active":"2024-05-26T22:54:57.931028+0000","last_became_peered":"2024-05-26T22:54:57.931028+0000","last_unstale":"2024-05-26T22:54:59.148734+0000","last_undegraded":"2024-05-26T22:54:59.148734+0000","last_fullsized":"2024-05-26T22:54:59.148734+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:27:18.238559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:58.917954+0000","last_change":"2024-05-26T22:54:57.938554+0000","last_active":"2024-05-26T22:54:58.917954+0000","last_peered":"2024-05-26T22:54:58.917954+0000","last_clean":"2024-05-26T22:54:58.917954+0000","last_became_active":"2024-05-26T22:54:57.938196+0000","last_became_peered":"2024-05-26T22:54:57.938196+0000","last_unstale":"2024-05-26T22:54:58.917954+0000","last_undegraded":"2024-05-26T22:54:58.917954+0000","last_fullsized":"2024-05-26T22:54:58.917954+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:17:20.674336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:58.917641+0000","last_change":"2024-05-26T22:54:57.922229+0000","last_active":"2024-05-26T22:54:58.917641+0000","last_peered":"2024-05-26T22:54:58.917641+0000","last_clean":"2024-05-26T22:54:58.917641+0000","last_became_active":"2024-05-26T22:54:57.921924+0000","last_became_peered":"2024-05-26T22:54:57.921924+0000","last_unstale":"2024-05-26T22:54:58.917641+0000","last_undegraded":"2024-05-26T22:54:58.917641+0000","last_fullsized":"2024-05-26T22:54:58.917641+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:41:35.115195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:58.917329+0000","last_change":"2024-05-26T22:54:57.918087+0000","last_active":"2024-05-26T22:54:58.917329+0000","last_peered":"2024-05-26T22:54:58.917329+0000","last_clean":"2024-05-26T22:54:58.917329+0000","last_became_active":"2024-05-26T22:54:57.917127+0000","last_became_peered":"2024-05-26T22:54:57.917127+0000","last_unstale":"2024-05-26T22:54:58.917329+0000","last_undegraded":"2024-05-26T22:54:58.917329+0000","last_fullsized":"2024-05-26T22:54:58.917329+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:52:37.214179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.468026+0000","last_change":"2024-05-26T22:54:57.924322+0000","last_active":"2024-05-26T22:54:59.468026+0000","last_peered":"2024-05-26T22:54:59.468026+0000","last_clean":"2024-05-26T22:54:59.468026+0000","last_became_active":"2024-05-26T22:54:57.923770+0000","last_became_peered":"2024-05-26T22:54:57.923770+0000","last_unstale":"2024-05-26T22:54:59.468026+0000","last_undegraded":"2024-05-26T22:54:59.468026+0000","last_fullsized":"2024-05-26T22:54:59.468026+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T23:32:13.073868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.469588+0000","last_change":"2024-05-26T22:54:57.924552+0000","last_active":"2024-05-26T22:54:59.469588+0000","last_peered":"2024-05-26T22:54:59.469588+0000","last_clean":"2024-05-26T22:54:59.469588+0000","last_became_active":"2024-05-26T22:54:57.924261+0000","last_became_peered":"2024-05-26T22:54:57.924261+0000","last_unstale":"2024-05-26T22:54:59.469588+0000","last_undegraded":"2024-05-26T22:54:59.469588+0000","last_fullsized":"2024-05-26T22:54:59.469588+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:14:19.122236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:58.918517+0000","last_change":"2024-05-26T22:54:57.936956+0000","last_active":"2024-05-26T22:54:58.918517+0000","last_peered":"2024-05-26T22:54:58.918517+0000","last_clean":"2024-05-26T22:54:58.918517+0000","last_became_active":"2024-05-26T22:54:57.936691+0000","last_became_peered":"2024-05-26T22:54:57.936691+0000","last_unstale":"2024-05-26T22:54:58.918517+0000","last_undegraded":"2024-05-26T22:54:58.918517+0000","last_fullsized":"2024-05-26T22:54:58.918517+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:35:20.830276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:58.918961+0000","last_change":"2024-05-26T22:54:57.935715+0000","last_active":"2024-05-26T22:54:58.918961+0000","last_peered":"2024-05-26T22:54:58.918961+0000","last_clean":"2024-05-26T22:54:58.918961+0000","last_became_active":"2024-05-26T22:54:57.933970+0000","last_became_peered":"2024-05-26T22:54:57.933970+0000","last_unstale":"2024-05-26T22:54:58.918961+0000","last_undegraded":"2024-05-26T22:54:58.918961+0000","last_fullsized":"2024-05-26T22:54:58.918961+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:04:24.105687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.149000+0000","last_change":"2024-05-26T22:54:57.931818+0000","last_active":"2024-05-26T22:54:59.149000+0000","last_peered":"2024-05-26T22:54:59.149000+0000","last_clean":"2024-05-26T22:54:59.149000+0000","last_became_active":"2024-05-26T22:54:57.931575+0000","last_became_peered":"2024-05-26T22:54:57.931575+0000","last_unstale":"2024-05-26T22:54:59.149000+0000","last_undegraded":"2024-05-26T22:54:59.149000+0000","last_fullsized":"2024-05-26T22:54:59.149000+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:18:01.132850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:58.917779+0000","last_change":"2024-05-26T22:54:57.924324+0000","last_active":"2024-05-26T22:54:58.917779+0000","last_peered":"2024-05-26T22:54:58.917779+0000","last_clean":"2024-05-26T22:54:58.917779+0000","last_became_active":"2024-05-26T22:54:57.923195+0000","last_became_peered":"2024-05-26T22:54:57.923195+0000","last_unstale":"2024-05-26T22:54:58.917779+0000","last_undegraded":"2024-05-26T22:54:58.917779+0000","last_fullsized":"2024-05-26T22:54:58.917779+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:20:20.761797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.467192+0000","last_change":"2024-05-26T22:54:57.919100+0000","last_active":"2024-05-26T22:54:59.467192+0000","last_peered":"2024-05-26T22:54:59.467192+0000","last_clean":"2024-05-26T22:54:59.467192+0000","last_became_active":"2024-05-26T22:54:57.918795+0000","last_became_peered":"2024-05-26T22:54:57.918795+0000","last_unstale":"2024-05-26T22:54:59.467192+0000","last_undegraded":"2024-05-26T22:54:59.467192+0000","last_fullsized":"2024-05-26T22:54:59.467192+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T23:00:15.189681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.466940+0000","last_change":"2024-05-26T22:54:57.917401+0000","last_active":"2024-05-26T22:54:59.466940+0000","last_peered":"2024-05-26T22:54:59.466940+0000","last_clean":"2024-05-26T22:54:59.466940+0000","last_became_active":"2024-05-26T22:54:57.917104+0000","last_became_peered":"2024-05-26T22:54:57.917104+0000","last_unstale":"2024-05-26T22:54:59.466940+0000","last_undegraded":"2024-05-26T22:54:59.466940+0000","last_fullsized":"2024-05-26T22:54:59.466940+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:09:57.630908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.468270+0000","last_change":"2024-05-26T22:54:57.931944+0000","last_active":"2024-05-26T22:54:59.468270+0000","last_peered":"2024-05-26T22:54:59.468270+0000","last_clean":"2024-05-26T22:54:59.468270+0000","last_became_active":"2024-05-26T22:54:57.931255+0000","last_became_peered":"2024-05-26T22:54:57.931255+0000","last_unstale":"2024-05-26T22:54:59.468270+0000","last_undegraded":"2024-05-26T22:54:59.468270+0000","last_fullsized":"2024-05-26T22:54:59.468270+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:52:37.214179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:58.923510+0000","last_change":"2024-05-26T22:54:57.912470+0000","last_active":"2024-05-26T22:54:58.923510+0000","last_peered":"2024-05-26T22:54:58.923510+0000","last_clean":"2024-05-26T22:54:58.923510+0000","last_became_active":"2024-05-26T22:54:57.911461+0000","last_became_peered":"2024-05-26T22:54:57.911461+0000","last_unstale":"2024-05-26T22:54:58.923510+0000","last_undegraded":"2024-05-26T22:54:58.923510+0000","last_fullsized":"2024-05-26T22:54:58.923510+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:01:36.256567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.469629+0000","last_change":"2024-05-26T22:54:57.908004+0000","last_active":"2024-05-26T22:54:59.469629+0000","last_peered":"2024-05-26T22:54:59.469629+0000","last_clean":"2024-05-26T22:54:59.469629+0000","last_became_active":"2024-05-26T22:54:57.905624+0000","last_became_peered":"2024-05-26T22:54:57.905624+0000","last_unstale":"2024-05-26T22:54:59.469629+0000","last_undegraded":"2024-05-26T22:54:59.469629+0000","last_fullsized":"2024-05-26T22:54:59.469629+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:42:41.925687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.468874+0000","last_change":"2024-05-26T22:54:57.919261+0000","last_active":"2024-05-26T22:54:59.468874+0000","last_peered":"2024-05-26T22:54:59.468874+0000","last_clean":"2024-05-26T22:54:59.468874+0000","last_became_active":"2024-05-26T22:54:57.918886+0000","last_became_peered":"2024-05-26T22:54:57.918886+0000","last_unstale":"2024-05-26T22:54:59.468874+0000","last_undegraded":"2024-05-26T22:54:59.468874+0000","last_fullsized":"2024-05-26T22:54:59.468874+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:17:20.674336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:58.922807+0000","last_change":"2024-05-26T22:54:57.926694+0000","last_active":"2024-05-26T22:54:58.922807+0000","last_peered":"2024-05-26T22:54:58.922807+0000","last_clean":"2024-05-26T22:54:58.922807+0000","last_became_active":"2024-05-26T22:54:57.925650+0000","last_became_peered":"2024-05-26T22:54:57.925650+0000","last_unstale":"2024-05-26T22:54:58.922807+0000","last_undegraded":"2024-05-26T22:54:58.922807+0000","last_fullsized":"2024-05-26T22:54:58.922807+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:34:05.444741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.467824+0000","last_change":"2024-05-26T22:54:57.904422+0000","last_active":"2024-05-26T22:54:59.467824+0000","last_peered":"2024-05-26T22:54:59.467824+0000","last_clean":"2024-05-26T22:54:59.467824+0000","last_became_active":"2024-05-26T22:54:57.903593+0000","last_became_peered":"2024-05-26T22:54:57.903593+0000","last_unstale":"2024-05-26T22:54:59.467824+0000","last_undegraded":"2024-05-26T22:54:59.467824+0000","last_fullsized":"2024-05-26T22:54:59.467824+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:24:41.881300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:58.918824+0000","last_change":"2024-05-26T22:54:57.933567+0000","last_active":"2024-05-26T22:54:58.918824+0000","last_peered":"2024-05-26T22:54:58.918824+0000","last_clean":"2024-05-26T22:54:58.918824+0000","last_became_active":"2024-05-26T22:54:57.932909+0000","last_became_peered":"2024-05-26T22:54:57.932909+0000","last_unstale":"2024-05-26T22:54:58.918824+0000","last_undegraded":"2024-05-26T22:54:58.918824+0000","last_fullsized":"2024-05-26T22:54:58.918824+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:03:07.441337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:58.917362+0000","last_change":"2024-05-26T22:54:57.920028+0000","last_active":"2024-05-26T22:54:58.917362+0000","last_peered":"2024-05-26T22:54:58.917362+0000","last_clean":"2024-05-26T22:54:58.917362+0000","last_became_active":"2024-05-26T22:54:57.919450+0000","last_became_peered":"2024-05-26T22:54:57.919450+0000","last_unstale":"2024-05-26T22:54:58.917362+0000","last_undegraded":"2024-05-26T22:54:58.917362+0000","last_fullsized":"2024-05-26T22:54:58.917362+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:46:04.317115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.467521+0000","last_change":"2024-05-26T22:54:57.906455+0000","last_active":"2024-05-26T22:54:59.467521+0000","last_peered":"2024-05-26T22:54:59.467521+0000","last_clean":"2024-05-26T22:54:59.467521+0000","last_became_active":"2024-05-26T22:54:57.906162+0000","last_became_peered":"2024-05-26T22:54:57.906162+0000","last_unstale":"2024-05-26T22:54:59.467521+0000","last_undegraded":"2024-05-26T22:54:59.467521+0000","last_fullsized":"2024-05-26T22:54:59.467521+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:27:41.142373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:58.918595+0000","last_change":"2024-05-26T22:54:57.933428+0000","last_active":"2024-05-26T22:54:58.918595+0000","last_peered":"2024-05-26T22:54:58.918595+0000","last_clean":"2024-05-26T22:54:58.918595+0000","last_became_active":"2024-05-26T22:54:57.932926+0000","last_became_peered":"2024-05-26T22:54:57.932926+0000","last_unstale":"2024-05-26T22:54:58.918595+0000","last_undegraded":"2024-05-26T22:54:58.918595+0000","last_fullsized":"2024-05-26T22:54:58.918595+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:05:42.637697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"94'4","reported_seq":18,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.469462+0000","last_change":"2024-05-26T22:54:56.893757+0000","last_active":"2024-05-26T22:54:59.469462+0000","last_peered":"2024-05-26T22:54:59.469462+0000","last_clean":"2024-05-26T22:54:59.469462+0000","last_became_active":"2024-05-26T22:54:56.893411+0000","last_became_peered":"2024-05-26T22:54:56.893411+0000","last_unstale":"2024-05-26T22:54:59.469462+0000","last_undegraded":"2024-05-26T22:54:59.469462+0000","last_fullsized":"2024-05-26T22:54:59.469462+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_clean_scrub_stamp":"2024-05-26T22:54:55.841532+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:08:37.703863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:58.919346+0000","last_change":"2024-05-26T22:54:57.928717+0000","last_active":"2024-05-26T22:54:58.919346+0000","last_peered":"2024-05-26T22:54:58.919346+0000","last_clean":"2024-05-26T22:54:58.919346+0000","last_became_active":"2024-05-26T22:54:57.928417+0000","last_became_peered":"2024-05-26T22:54:57.928417+0000","last_unstale":"2024-05-26T22:54:58.919346+0000","last_undegraded":"2024-05-26T22:54:58.919346+0000","last_fullsized":"2024-05-26T22:54:58.919346+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:02:32.509633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"94'1","reported_seq":15,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.468449+0000","last_change":"2024-05-26T22:54:56.902755+0000","last_active":"2024-05-26T22:54:59.468449+0000","last_peered":"2024-05-26T22:54:59.468449+0000","last_clean":"2024-05-26T22:54:59.468449+0000","last_became_active":"2024-05-26T22:54:56.902439+0000","last_became_peered":"2024-05-26T22:54:56.902439+0000","last_unstale":"2024-05-26T22:54:59.468449+0000","last_undegraded":"2024-05-26T22:54:59.468449+0000","last_fullsized":"2024-05-26T22:54:59.468449+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_clean_scrub_stamp":"2024-05-26T22:54:55.841532+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T23:23:59.941266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.467842+0000","last_change":"2024-05-26T22:54:57.922650+0000","last_active":"2024-05-26T22:54:59.467842+0000","last_peered":"2024-05-26T22:54:59.467842+0000","last_clean":"2024-05-26T22:54:59.467842+0000","last_became_active":"2024-05-26T22:54:57.922379+0000","last_became_peered":"2024-05-26T22:54:57.922379+0000","last_unstale":"2024-05-26T22:54:59.467842+0000","last_undegraded":"2024-05-26T22:54:59.467842+0000","last_fullsized":"2024-05-26T22:54:59.467842+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:57:55.839877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"94'2","reported_seq":16,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.468907+0000","last_change":"2024-05-26T22:54:56.887767+0000","last_active":"2024-05-26T22:54:59.468907+0000","last_peered":"2024-05-26T22:54:59.468907+0000","last_clean":"2024-05-26T22:54:59.468907+0000","last_became_active":"2024-05-26T22:54:56.886622+0000","last_became_peered":"2024-05-26T22:54:56.886622+0000","last_unstale":"2024-05-26T22:54:59.468907+0000","last_undegraded":"2024-05-26T22:54:59.468907+0000","last_fullsized":"2024-05-26T22:54:59.468907+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_clean_scrub_stamp":"2024-05-26T22:54:55.841532+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:27:17.223023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.149568+0000","last_change":"2024-05-26T22:54:57.930000+0000","last_active":"2024-05-26T22:54:59.149568+0000","last_peered":"2024-05-26T22:54:59.149568+0000","last_clean":"2024-05-26T22:54:59.149568+0000","last_became_active":"2024-05-26T22:54:57.928087+0000","last_became_peered":"2024-05-26T22:54:57.928087+0000","last_unstale":"2024-05-26T22:54:59.149568+0000","last_undegraded":"2024-05-26T22:54:59.149568+0000","last_fullsized":"2024-05-26T22:54:59.149568+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:52:37.214179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"94'2","reported_seq":16,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.468739+0000","last_change":"2024-05-26T22:54:56.899380+0000","last_active":"2024-05-26T22:54:59.468739+0000","last_peered":"2024-05-26T22:54:59.468739+0000","last_clean":"2024-05-26T22:54:59.468739+0000","last_became_active":"2024-05-26T22:54:56.898426+0000","last_became_peered":"2024-05-26T22:54:56.898426+0000","last_unstale":"2024-05-26T22:54:59.468739+0000","last_undegraded":"2024-05-26T22:54:59.468739+0000","last_fullsized":"2024-05-26T22:54:59.468739+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_clean_scrub_stamp":"2024-05-26T22:54:55.841532+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:24:01.985079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.466784+0000","last_change":"2024-05-26T22:54:57.919904+0000","last_active":"2024-05-26T22:54:59.466784+0000","last_peered":"2024-05-26T22:54:59.466784+0000","last_clean":"2024-05-26T22:54:59.466784+0000","last_became_active":"2024-05-26T22:54:57.919622+0000","last_became_peered":"2024-05-26T22:54:57.919622+0000","last_unstale":"2024-05-26T22:54:59.466784+0000","last_undegraded":"2024-05-26T22:54:59.466784+0000","last_fullsized":"2024-05-26T22:54:59.466784+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:17:44.544345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"94'1","reported_seq":15,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.467304+0000","last_change":"2024-05-26T22:54:56.894238+0000","last_active":"2024-05-26T22:54:59.467304+0000","last_peered":"2024-05-26T22:54:59.467304+0000","last_clean":"2024-05-26T22:54:59.467304+0000","last_became_active":"2024-05-26T22:54:56.893894+0000","last_became_peered":"2024-05-26T22:54:56.893894+0000","last_unstale":"2024-05-26T22:54:59.467304+0000","last_undegraded":"2024-05-26T22:54:59.467304+0000","last_fullsized":"2024-05-26T22:54:59.467304+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_clean_scrub_stamp":"2024-05-26T22:54:55.841532+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:22:59.025816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.468695+0000","last_change":"2024-05-26T22:54:57.924957+0000","last_active":"2024-05-26T22:54:59.468695+0000","last_peered":"2024-05-26T22:54:59.468695+0000","last_clean":"2024-05-26T22:54:59.468695+0000","last_became_active":"2024-05-26T22:54:57.924707+0000","last_became_peered":"2024-05-26T22:54:57.924707+0000","last_unstale":"2024-05-26T22:54:59.468695+0000","last_undegraded":"2024-05-26T22:54:59.468695+0000","last_fullsized":"2024-05-26T22:54:59.468695+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:46:04.317115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"94'1","reported_seq":15,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.467483+0000","last_change":"2024-05-26T22:54:56.885319+0000","last_active":"2024-05-26T22:54:59.467483+0000","last_peered":"2024-05-26T22:54:59.467483+0000","last_clean":"2024-05-26T22:54:59.467483+0000","last_became_active":"2024-05-26T22:54:56.884985+0000","last_became_peered":"2024-05-26T22:54:56.884985+0000","last_unstale":"2024-05-26T22:54:59.467483+0000","last_undegraded":"2024-05-26T22:54:59.467483+0000","last_fullsized":"2024-05-26T22:54:59.467483+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_clean_scrub_stamp":"2024-05-26T22:54:55.841532+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:39:59.569280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:58.917316+0000","last_change":"2024-05-26T22:54:57.931093+0000","last_active":"2024-05-26T22:54:58.917316+0000","last_peered":"2024-05-26T22:54:58.917316+0000","last_clean":"2024-05-26T22:54:58.917316+0000","last_became_active":"2024-05-26T22:54:57.930567+0000","last_became_peered":"2024-05-26T22:54:57.930567+0000","last_unstale":"2024-05-26T22:54:58.917316+0000","last_undegraded":"2024-05-26T22:54:58.917316+0000","last_fullsized":"2024-05-26T22:54:58.917316+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:08:38.719400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:58.922791+0000","last_change":"2024-05-26T22:54:56.884617+0000","last_active":"2024-05-26T22:54:58.922791+0000","last_peered":"2024-05-26T22:54:58.922791+0000","last_clean":"2024-05-26T22:54:58.922791+0000","last_became_active":"2024-05-26T22:54:56.880110+0000","last_became_peered":"2024-05-26T22:54:56.880110+0000","last_unstale":"2024-05-26T22:54:58.922791+0000","last_undegraded":"2024-05-26T22:54:58.922791+0000","last_fullsized":"2024-05-26T22:54:58.922791+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_clean_scrub_stamp":"2024-05-26T22:54:55.841532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:26:42.775770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.148640+0000","last_change":"2024-05-26T22:54:57.922822+0000","last_active":"2024-05-26T22:54:59.148640+0000","last_peered":"2024-05-26T22:54:59.148640+0000","last_clean":"2024-05-26T22:54:59.148640+0000","last_became_active":"2024-05-26T22:54:57.922543+0000","last_became_peered":"2024-05-26T22:54:57.922543+0000","last_unstale":"2024-05-26T22:54:59.148640+0000","last_undegraded":"2024-05-26T22:54:59.148640+0000","last_fullsized":"2024-05-26T22:54:59.148640+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:52:52.534985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.467889+0000","last_change":"2024-05-26T22:54:56.889205+0000","last_active":"2024-05-26T22:54:59.467889+0000","last_peered":"2024-05-26T22:54:59.467889+0000","last_clean":"2024-05-26T22:54:59.467889+0000","last_became_active":"2024-05-26T22:54:56.888805+0000","last_became_peered":"2024-05-26T22:54:56.888805+0000","last_unstale":"2024-05-26T22:54:59.467889+0000","last_undegraded":"2024-05-26T22:54:59.467889+0000","last_fullsized":"2024-05-26T22:54:59.467889+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_clean_scrub_stamp":"2024-05-26T22:54:55.841532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:36:16.322392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.469620+0000","last_change":"2024-05-26T22:54:57.915217+0000","last_active":"2024-05-26T22:54:59.469620+0000","last_peered":"2024-05-26T22:54:59.469620+0000","last_clean":"2024-05-26T22:54:59.469620+0000","last_became_active":"2024-05-26T22:54:57.914894+0000","last_became_peered":"2024-05-26T22:54:57.914894+0000","last_unstale":"2024-05-26T22:54:59.469620+0000","last_undegraded":"2024-05-26T22:54:59.469620+0000","last_fullsized":"2024-05-26T22:54:59.469620+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:56:23.782834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.470096+0000","last_change":"2024-05-26T22:54:56.896440+0000","last_active":"2024-05-26T22:54:59.470096+0000","last_peered":"2024-05-26T22:54:59.470096+0000","last_clean":"2024-05-26T22:54:59.470096+0000","last_became_active":"2024-05-26T22:54:56.896177+0000","last_became_peered":"2024-05-26T22:54:56.896177+0000","last_unstale":"2024-05-26T22:54:59.470096+0000","last_undegraded":"2024-05-26T22:54:59.470096+0000","last_fullsized":"2024-05-26T22:54:59.470096+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_clean_scrub_stamp":"2024-05-26T22:54:55.841532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:31:35.474879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.149761+0000","last_change":"2024-05-26T22:54:57.921014+0000","last_active":"2024-05-26T22:54:59.149761+0000","last_peered":"2024-05-26T22:54:59.149761+0000","last_clean":"2024-05-26T22:54:59.149761+0000","last_became_active":"2024-05-26T22:54:57.920738+0000","last_became_peered":"2024-05-26T22:54:57.920738+0000","last_unstale":"2024-05-26T22:54:59.149761+0000","last_undegraded":"2024-05-26T22:54:59.149761+0000","last_fullsized":"2024-05-26T22:54:59.149761+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:46:04.317115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:58.918615+0000","last_change":"2024-05-26T22:54:56.890912+0000","last_active":"2024-05-26T22:54:58.918615+0000","last_peered":"2024-05-26T22:54:58.918615+0000","last_clean":"2024-05-26T22:54:58.918615+0000","last_became_active":"2024-05-26T22:54:56.890618+0000","last_became_peered":"2024-05-26T22:54:56.890618+0000","last_unstale":"2024-05-26T22:54:58.918615+0000","last_undegraded":"2024-05-26T22:54:58.918615+0000","last_fullsized":"2024-05-26T22:54:58.918615+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_clean_scrub_stamp":"2024-05-26T22:54:55.841532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:26:42.775770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:58.923511+0000","last_change":"2024-05-26T22:54:57.903878+0000","last_active":"2024-05-26T22:54:58.923511+0000","last_peered":"2024-05-26T22:54:58.923511+0000","last_clean":"2024-05-26T22:54:58.923511+0000","last_became_active":"2024-05-26T22:54:57.901707+0000","last_became_peered":"2024-05-26T22:54:57.901707+0000","last_unstale":"2024-05-26T22:54:58.923511+0000","last_undegraded":"2024-05-26T22:54:58.923511+0000","last_fullsized":"2024-05-26T22:54:58.923511+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:27:30.904396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"94'3","reported_seq":17,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.469427+0000","last_change":"2024-05-26T22:54:56.904426+0000","last_active":"2024-05-26T22:54:59.469427+0000","last_peered":"2024-05-26T22:54:59.469427+0000","last_clean":"2024-05-26T22:54:59.469427+0000","last_became_active":"2024-05-26T22:54:56.904089+0000","last_became_peered":"2024-05-26T22:54:56.904089+0000","last_unstale":"2024-05-26T22:54:59.469427+0000","last_undegraded":"2024-05-26T22:54:59.469427+0000","last_fullsized":"2024-05-26T22:54:59.469427+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_clean_scrub_stamp":"2024-05-26T22:54:55.841532+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:44:33.082467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.469178+0000","last_change":"2024-05-26T22:54:57.923341+0000","last_active":"2024-05-26T22:54:59.469178+0000","last_peered":"2024-05-26T22:54:59.469178+0000","last_clean":"2024-05-26T22:54:59.469178+0000","last_became_active":"2024-05-26T22:54:57.923044+0000","last_became_peered":"2024-05-26T22:54:57.923044+0000","last_unstale":"2024-05-26T22:54:59.469178+0000","last_undegraded":"2024-05-26T22:54:59.469178+0000","last_fullsized":"2024-05-26T22:54:59.469178+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:34:05.444741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"94'6","reported_seq":20,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.149441+0000","last_change":"2024-05-26T22:54:56.901384+0000","last_active":"2024-05-26T22:54:59.149441+0000","last_peered":"2024-05-26T22:54:59.149441+0000","last_clean":"2024-05-26T22:54:59.149441+0000","last_became_active":"2024-05-26T22:54:56.900899+0000","last_became_peered":"2024-05-26T22:54:56.900899+0000","last_unstale":"2024-05-26T22:54:59.149441+0000","last_undegraded":"2024-05-26T22:54:59.149441+0000","last_fullsized":"2024-05-26T22:54:59.149441+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_clean_scrub_stamp":"2024-05-26T22:54:55.841532+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:58:36.036557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:58.917796+0000","last_change":"2024-05-26T22:54:57.920690+0000","last_active":"2024-05-26T22:54:58.917796+0000","last_peered":"2024-05-26T22:54:58.917796+0000","last_clean":"2024-05-26T22:54:58.917796+0000","last_became_active":"2024-05-26T22:54:57.920434+0000","last_became_peered":"2024-05-26T22:54:57.920434+0000","last_unstale":"2024-05-26T22:54:58.917796+0000","last_undegraded":"2024-05-26T22:54:58.917796+0000","last_fullsized":"2024-05-26T22:54:58.917796+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:45:18.352727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"94'2","reported_seq":16,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:58.918184+0000","last_change":"2024-05-26T22:54:56.898635+0000","last_active":"2024-05-26T22:54:58.918184+0000","last_peered":"2024-05-26T22:54:58.918184+0000","last_clean":"2024-05-26T22:54:58.918184+0000","last_became_active":"2024-05-26T22:54:56.898383+0000","last_became_peered":"2024-05-26T22:54:56.898383+0000","last_unstale":"2024-05-26T22:54:58.918184+0000","last_undegraded":"2024-05-26T22:54:58.918184+0000","last_fullsized":"2024-05-26T22:54:58.918184+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_clean_scrub_stamp":"2024-05-26T22:54:55.841532+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T23:50:14.994019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.469107+0000","last_change":"2024-05-26T22:54:57.926490+0000","last_active":"2024-05-26T22:54:59.469107+0000","last_peered":"2024-05-26T22:54:59.469107+0000","last_clean":"2024-05-26T22:54:59.469107+0000","last_became_active":"2024-05-26T22:54:57.921270+0000","last_became_peered":"2024-05-26T22:54:57.921270+0000","last_unstale":"2024-05-26T22:54:59.469107+0000","last_undegraded":"2024-05-26T22:54:59.469107+0000","last_fullsized":"2024-05-26T22:54:59.469107+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:44:34.098004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:58.918061+0000","last_change":"2024-05-26T22:54:56.889666+0000","last_active":"2024-05-26T22:54:58.918061+0000","last_peered":"2024-05-26T22:54:58.918061+0000","last_clean":"2024-05-26T22:54:58.918061+0000","last_became_active":"2024-05-26T22:54:56.889288+0000","last_became_peered":"2024-05-26T22:54:56.889288+0000","last_unstale":"2024-05-26T22:54:58.918061+0000","last_undegraded":"2024-05-26T22:54:58.918061+0000","last_fullsized":"2024-05-26T22:54:58.918061+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_clean_scrub_stamp":"2024-05-26T22:54:55.841532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:29:00.893581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.148643+0000","last_change":"2024-05-26T22:54:57.916031+0000","last_active":"2024-05-26T22:54:59.148643+0000","last_peered":"2024-05-26T22:54:59.148643+0000","last_clean":"2024-05-26T22:54:59.148643+0000","last_became_active":"2024-05-26T22:54:57.915672+0000","last_became_peered":"2024-05-26T22:54:57.915672+0000","last_unstale":"2024-05-26T22:54:59.148643+0000","last_undegraded":"2024-05-26T22:54:59.148643+0000","last_fullsized":"2024-05-26T22:54:59.148643+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:24:03.000615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.149441+0000","last_change":"2024-05-26T22:54:56.902700+0000","last_active":"2024-05-26T22:54:59.149441+0000","last_peered":"2024-05-26T22:54:59.149441+0000","last_clean":"2024-05-26T22:54:59.149441+0000","last_became_active":"2024-05-26T22:54:56.902290+0000","last_became_peered":"2024-05-26T22:54:56.902290+0000","last_unstale":"2024-05-26T22:54:59.149441+0000","last_undegraded":"2024-05-26T22:54:59.149441+0000","last_fullsized":"2024-05-26T22:54:59.149441+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_clean_scrub_stamp":"2024-05-26T22:54:55.841532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:17:43.528808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:58.918346+0000","last_change":"2024-05-26T22:54:57.935060+0000","last_active":"2024-05-26T22:54:58.918346+0000","last_peered":"2024-05-26T22:54:58.918346+0000","last_clean":"2024-05-26T22:54:58.918346+0000","last_became_active":"2024-05-26T22:54:57.931110+0000","last_became_peered":"2024-05-26T22:54:57.931110+0000","last_unstale":"2024-05-26T22:54:58.918346+0000","last_undegraded":"2024-05-26T22:54:58.918346+0000","last_fullsized":"2024-05-26T22:54:58.918346+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:53:22.885429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"94'8","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.468164+0000","last_change":"2024-05-26T22:54:56.897834+0000","last_active":"2024-05-26T22:54:59.468164+0000","last_peered":"2024-05-26T22:54:59.468164+0000","last_clean":"2024-05-26T22:54:59.468164+0000","last_became_active":"2024-05-26T22:54:56.897593+0000","last_became_peered":"2024-05-26T22:54:56.897593+0000","last_unstale":"2024-05-26T22:54:59.468164+0000","last_undegraded":"2024-05-26T22:54:59.468164+0000","last_fullsized":"2024-05-26T22:54:59.468164+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_clean_scrub_stamp":"2024-05-26T22:54:55.841532+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:15:19.981823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:58.923749+0000","last_change":"2024-05-26T22:54:57.921641+0000","last_active":"2024-05-26T22:54:58.923749+0000","last_peered":"2024-05-26T22:54:58.923749+0000","last_clean":"2024-05-26T22:54:58.923749+0000","last_became_active":"2024-05-26T22:54:57.921376+0000","last_became_peered":"2024-05-26T22:54:57.921376+0000","last_unstale":"2024-05-26T22:54:58.923749+0000","last_undegraded":"2024-05-26T22:54:58.923749+0000","last_fullsized":"2024-05-26T22:54:58.923749+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:21:42.898512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"94'2","reported_seq":16,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.149344+0000","last_change":"2024-05-26T22:54:56.903218+0000","last_active":"2024-05-26T22:54:59.149344+0000","last_peered":"2024-05-26T22:54:59.149344+0000","last_clean":"2024-05-26T22:54:59.149344+0000","last_became_active":"2024-05-26T22:54:56.902855+0000","last_became_peered":"2024-05-26T22:54:56.902855+0000","last_unstale":"2024-05-26T22:54:59.149344+0000","last_undegraded":"2024-05-26T22:54:59.149344+0000","last_fullsized":"2024-05-26T22:54:59.149344+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_clean_scrub_stamp":"2024-05-26T22:54:55.841532+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:44:33.082467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.466947+0000","last_change":"2024-05-26T22:54:57.917046+0000","last_active":"2024-05-26T22:54:59.466947+0000","last_peered":"2024-05-26T22:54:59.466947+0000","last_clean":"2024-05-26T22:54:59.466947+0000","last_became_active":"2024-05-26T22:54:57.916762+0000","last_became_peered":"2024-05-26T22:54:57.916762+0000","last_unstale":"2024-05-26T22:54:59.466947+0000","last_undegraded":"2024-05-26T22:54:59.466947+0000","last_fullsized":"2024-05-26T22:54:59.466947+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:44:34.098004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"94'1","reported_seq":15,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:58.923335+0000","last_change":"2024-05-26T22:54:56.896866+0000","last_active":"2024-05-26T22:54:58.923335+0000","last_peered":"2024-05-26T22:54:58.923335+0000","last_clean":"2024-05-26T22:54:58.923335+0000","last_became_active":"2024-05-26T22:54:56.896575+0000","last_became_peered":"2024-05-26T22:54:56.896575+0000","last_unstale":"2024-05-26T22:54:58.923335+0000","last_undegraded":"2024-05-26T22:54:58.923335+0000","last_fullsized":"2024-05-26T22:54:58.923335+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_clean_scrub_stamp":"2024-05-26T22:54:55.841532+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:44:40.935181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.469278+0000","last_change":"2024-05-26T22:54:57.928029+0000","last_active":"2024-05-26T22:54:59.469278+0000","last_peered":"2024-05-26T22:54:59.469278+0000","last_clean":"2024-05-26T22:54:59.469278+0000","last_became_active":"2024-05-26T22:54:57.926967+0000","last_became_peered":"2024-05-26T22:54:57.926967+0000","last_unstale":"2024-05-26T22:54:59.469278+0000","last_undegraded":"2024-05-26T22:54:59.469278+0000","last_fullsized":"2024-05-26T22:54:59.469278+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:35:20.830276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.468931+0000","last_change":"2024-05-26T22:54:56.898440+0000","last_active":"2024-05-26T22:54:59.468931+0000","last_peered":"2024-05-26T22:54:59.468931+0000","last_clean":"2024-05-26T22:54:59.468931+0000","last_became_active":"2024-05-26T22:54:56.898035+0000","last_became_peered":"2024-05-26T22:54:56.898035+0000","last_unstale":"2024-05-26T22:54:59.468931+0000","last_undegraded":"2024-05-26T22:54:59.468931+0000","last_fullsized":"2024-05-26T22:54:59.468931+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_clean_scrub_stamp":"2024-05-26T22:54:55.841532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T23:32:12.058331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:58.917802+0000","last_change":"2024-05-26T22:54:57.925549+0000","last_active":"2024-05-26T22:54:58.917802+0000","last_peered":"2024-05-26T22:54:58.917802+0000","last_clean":"2024-05-26T22:54:58.917802+0000","last_became_active":"2024-05-26T22:54:57.925243+0000","last_became_peered":"2024-05-26T22:54:57.925243+0000","last_unstale":"2024-05-26T22:54:58.917802+0000","last_undegraded":"2024-05-26T22:54:58.917802+0000","last_fullsized":"2024-05-26T22:54:58.917802+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:05:30.075668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"94'2","reported_seq":16,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.468132+0000","last_change":"2024-05-26T22:54:56.884317+0000","last_active":"2024-05-26T22:54:59.468132+0000","last_peered":"2024-05-26T22:54:59.468132+0000","last_clean":"2024-05-26T22:54:59.468132+0000","last_became_active":"2024-05-26T22:54:56.884047+0000","last_became_peered":"2024-05-26T22:54:56.884047+0000","last_unstale":"2024-05-26T22:54:59.468132+0000","last_undegraded":"2024-05-26T22:54:59.468132+0000","last_fullsized":"2024-05-26T22:54:59.468132+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_clean_scrub_stamp":"2024-05-26T22:54:55.841532+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:14:34.736481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:58.917927+0000","last_change":"2024-05-26T22:54:57.924558+0000","last_active":"2024-05-26T22:54:58.917927+0000","last_peered":"2024-05-26T22:54:58.917927+0000","last_clean":"2024-05-26T22:54:58.917927+0000","last_became_active":"2024-05-26T22:54:57.924058+0000","last_became_peered":"2024-05-26T22:54:57.924058+0000","last_unstale":"2024-05-26T22:54:58.917927+0000","last_undegraded":"2024-05-26T22:54:58.917927+0000","last_fullsized":"2024-05-26T22:54:58.917927+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T23:05:17.072534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.466597+0000","last_change":"2024-05-26T22:54:56.894081+0000","last_active":"2024-05-26T22:54:59.466597+0000","last_peered":"2024-05-26T22:54:59.466597+0000","last_clean":"2024-05-26T22:54:59.466597+0000","last_became_active":"2024-05-26T22:54:56.893780+0000","last_became_peered":"2024-05-26T22:54:56.893780+0000","last_unstale":"2024-05-26T22:54:59.466597+0000","last_undegraded":"2024-05-26T22:54:59.466597+0000","last_fullsized":"2024-05-26T22:54:59.466597+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_clean_scrub_stamp":"2024-05-26T22:54:55.841532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:03:17.215873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:58.918490+0000","last_change":"2024-05-26T22:54:57.926240+0000","last_active":"2024-05-26T22:54:58.918490+0000","last_peered":"2024-05-26T22:54:58.918490+0000","last_clean":"2024-05-26T22:54:58.918490+0000","last_became_active":"2024-05-26T22:54:57.925962+0000","last_became_peered":"2024-05-26T22:54:57.925962+0000","last_unstale":"2024-05-26T22:54:58.918490+0000","last_undegraded":"2024-05-26T22:54:58.918490+0000","last_fullsized":"2024-05-26T22:54:58.918490+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:32:05.059429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.466601+0000","last_change":"2024-05-26T22:54:56.892638+0000","last_active":"2024-05-26T22:54:59.466601+0000","last_peered":"2024-05-26T22:54:59.466601+0000","last_clean":"2024-05-26T22:54:59.466601+0000","last_became_active":"2024-05-26T22:54:56.891218+0000","last_became_peered":"2024-05-26T22:54:56.891218+0000","last_unstale":"2024-05-26T22:54:59.466601+0000","last_undegraded":"2024-05-26T22:54:59.466601+0000","last_fullsized":"2024-05-26T22:54:59.466601+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_clean_scrub_stamp":"2024-05-26T22:54:55.841532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:14:06.230411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:58.922481+0000","last_change":"2024-05-26T22:54:57.908838+0000","last_active":"2024-05-26T22:54:58.922481+0000","last_peered":"2024-05-26T22:54:58.922481+0000","last_clean":"2024-05-26T22:54:58.922481+0000","last_became_active":"2024-05-26T22:54:57.907882+0000","last_became_peered":"2024-05-26T22:54:57.907882+0000","last_unstale":"2024-05-26T22:54:58.922481+0000","last_undegraded":"2024-05-26T22:54:58.922481+0000","last_fullsized":"2024-05-26T22:54:58.922481+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:45:18.352727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.468405+0000","last_change":"2024-05-26T22:54:56.885815+0000","last_active":"2024-05-26T22:54:59.468405+0000","last_peered":"2024-05-26T22:54:59.468405+0000","last_clean":"2024-05-26T22:54:59.468405+0000","last_became_active":"2024-05-26T22:54:56.884912+0000","last_became_peered":"2024-05-26T22:54:56.884912+0000","last_unstale":"2024-05-26T22:54:59.468405+0000","last_undegraded":"2024-05-26T22:54:59.468405+0000","last_fullsized":"2024-05-26T22:54:59.468405+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_clean_scrub_stamp":"2024-05-26T22:54:55.841532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:44:40.935181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.469871+0000","last_change":"2024-05-26T22:54:57.918824+0000","last_active":"2024-05-26T22:54:59.469871+0000","last_peered":"2024-05-26T22:54:59.469871+0000","last_clean":"2024-05-26T22:54:59.469871+0000","last_became_active":"2024-05-26T22:54:57.918509+0000","last_became_peered":"2024-05-26T22:54:57.918509+0000","last_unstale":"2024-05-26T22:54:59.469871+0000","last_undegraded":"2024-05-26T22:54:59.469871+0000","last_fullsized":"2024-05-26T22:54:59.469871+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:14:16.578637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"94'2","reported_seq":16,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.466972+0000","last_change":"2024-05-26T22:54:56.900340+0000","last_active":"2024-05-26T22:54:59.466972+0000","last_peered":"2024-05-26T22:54:59.466972+0000","last_clean":"2024-05-26T22:54:59.466972+0000","last_became_active":"2024-05-26T22:54:56.899759+0000","last_became_peered":"2024-05-26T22:54:56.899759+0000","last_unstale":"2024-05-26T22:54:59.466972+0000","last_undegraded":"2024-05-26T22:54:59.466972+0000","last_fullsized":"2024-05-26T22:54:59.466972+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_clean_scrub_stamp":"2024-05-26T22:54:55.841532+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:27:40.126836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.469635+0000","last_change":"2024-05-26T22:54:57.920980+0000","last_active":"2024-05-26T22:54:59.469635+0000","last_peered":"2024-05-26T22:54:59.469635+0000","last_clean":"2024-05-26T22:54:59.469635+0000","last_became_active":"2024-05-26T22:54:57.920631+0000","last_became_peered":"2024-05-26T22:54:57.920631+0000","last_unstale":"2024-05-26T22:54:59.469635+0000","last_undegraded":"2024-05-26T22:54:59.469635+0000","last_fullsized":"2024-05-26T22:54:59.469635+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T23:24:00.956802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"95'1","reported_seq":15,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.917742+0000","last_change":"2024-05-26T22:54:56.887664+0000","last_active":"2024-05-26T22:54:59.917742+0000","last_peered":"2024-05-26T22:54:59.917742+0000","last_clean":"2024-05-26T22:54:59.917742+0000","last_became_active":"2024-05-26T22:54:56.887399+0000","last_became_peered":"2024-05-26T22:54:56.887399+0000","last_unstale":"2024-05-26T22:54:59.917742+0000","last_undegraded":"2024-05-26T22:54:59.917742+0000","last_fullsized":"2024-05-26T22:54:59.917742+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_clean_scrub_stamp":"2024-05-26T22:54:55.841532+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:52:36.198643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.468279+0000","last_change":"2024-05-26T22:54:57.924268+0000","last_active":"2024-05-26T22:54:59.468279+0000","last_peered":"2024-05-26T22:54:59.468279+0000","last_clean":"2024-05-26T22:54:59.468279+0000","last_became_active":"2024-05-26T22:54:57.920466+0000","last_became_peered":"2024-05-26T22:54:57.920466+0000","last_unstale":"2024-05-26T22:54:59.468279+0000","last_undegraded":"2024-05-26T22:54:59.468279+0000","last_fullsized":"2024-05-26T22:54:59.468279+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:58:37.052094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.469259+0000","last_change":"2024-05-26T22:54:56.898732+0000","last_active":"2024-05-26T22:54:59.469259+0000","last_peered":"2024-05-26T22:54:59.469259+0000","last_clean":"2024-05-26T22:54:59.469259+0000","last_became_active":"2024-05-26T22:54:56.898091+0000","last_became_peered":"2024-05-26T22:54:56.898091+0000","last_unstale":"2024-05-26T22:54:59.469259+0000","last_undegraded":"2024-05-26T22:54:59.469259+0000","last_fullsized":"2024-05-26T22:54:59.469259+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_clean_scrub_stamp":"2024-05-26T22:54:55.841532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:18:00.117314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:58.923137+0000","last_change":"2024-05-26T22:54:57.927226+0000","last_active":"2024-05-26T22:54:58.923137+0000","last_peered":"2024-05-26T22:54:58.923137+0000","last_clean":"2024-05-26T22:54:58.923137+0000","last_became_active":"2024-05-26T22:54:57.925136+0000","last_became_peered":"2024-05-26T22:54:57.925136+0000","last_unstale":"2024-05-26T22:54:58.923137+0000","last_undegraded":"2024-05-26T22:54:58.923137+0000","last_fullsized":"2024-05-26T22:54:58.923137+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:35:20.830276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:58.923135+0000","last_change":"2024-05-26T22:54:56.901260+0000","last_active":"2024-05-26T22:54:58.923135+0000","last_peered":"2024-05-26T22:54:58.923135+0000","last_clean":"2024-05-26T22:54:58.923135+0000","last_became_active":"2024-05-26T22:54:56.900507+0000","last_became_peered":"2024-05-26T22:54:56.900507+0000","last_unstale":"2024-05-26T22:54:58.923135+0000","last_undegraded":"2024-05-26T22:54:58.923135+0000","last_fullsized":"2024-05-26T22:54:58.923135+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_clean_scrub_stamp":"2024-05-26T22:54:55.841532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:32:01.500293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:58.917948+0000","last_change":"2024-05-26T22:54:57.929952+0000","last_active":"2024-05-26T22:54:58.917948+0000","last_peered":"2024-05-26T22:54:58.917948+0000","last_clean":"2024-05-26T22:54:58.917948+0000","last_became_active":"2024-05-26T22:54:57.928945+0000","last_became_peered":"2024-05-26T22:54:57.928945+0000","last_unstale":"2024-05-26T22:54:58.917948+0000","last_undegraded":"2024-05-26T22:54:58.917948+0000","last_fullsized":"2024-05-26T22:54:58.917948+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:04:52.070354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.468397+0000","last_change":"2024-05-26T22:54:56.890912+0000","last_active":"2024-05-26T22:54:59.468397+0000","last_peered":"2024-05-26T22:54:59.468397+0000","last_clean":"2024-05-26T22:54:59.468397+0000","last_became_active":"2024-05-26T22:54:56.890586+0000","last_became_peered":"2024-05-26T22:54:56.890586+0000","last_unstale":"2024-05-26T22:54:59.468397+0000","last_undegraded":"2024-05-26T22:54:59.468397+0000","last_fullsized":"2024-05-26T22:54:59.468397+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_clean_scrub_stamp":"2024-05-26T22:54:55.841532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:52:51.519449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.148906+0000","last_change":"2024-05-26T22:54:57.919463+0000","last_active":"2024-05-26T22:54:59.148906+0000","last_peered":"2024-05-26T22:54:59.148906+0000","last_clean":"2024-05-26T22:54:59.148906+0000","last_became_active":"2024-05-26T22:54:57.918851+0000","last_became_peered":"2024-05-26T22:54:57.918851+0000","last_unstale":"2024-05-26T22:54:59.148906+0000","last_undegraded":"2024-05-26T22:54:59.148906+0000","last_fullsized":"2024-05-26T22:54:59.148906+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T23:32:13.073868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:58.922011+0000","last_change":"2024-05-26T22:54:56.889430+0000","last_active":"2024-05-26T22:54:58.922011+0000","last_peered":"2024-05-26T22:54:58.922011+0000","last_clean":"2024-05-26T22:54:58.922011+0000","last_became_active":"2024-05-26T22:54:56.887174+0000","last_became_peered":"2024-05-26T22:54:56.887174+0000","last_unstale":"2024-05-26T22:54:58.922011+0000","last_undegraded":"2024-05-26T22:54:58.922011+0000","last_fullsized":"2024-05-26T22:54:58.922011+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_clean_scrub_stamp":"2024-05-26T22:54:55.841532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:33:55.154572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:58.918635+0000","last_change":"2024-05-26T22:54:57.914623+0000","last_active":"2024-05-26T22:54:58.918635+0000","last_peered":"2024-05-26T22:54:58.918635+0000","last_clean":"2024-05-26T22:54:58.918635+0000","last_became_active":"2024-05-26T22:54:57.914272+0000","last_became_peered":"2024-05-26T22:54:57.914272+0000","last_unstale":"2024-05-26T22:54:58.918635+0000","last_undegraded":"2024-05-26T22:54:58.918635+0000","last_fullsized":"2024-05-26T22:54:58.918635+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:42:48.817759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:58.917468+0000","last_change":"2024-05-26T22:54:56.895568+0000","last_active":"2024-05-26T22:54:58.917468+0000","last_peered":"2024-05-26T22:54:58.917468+0000","last_clean":"2024-05-26T22:54:58.917468+0000","last_became_active":"2024-05-26T22:54:56.894157+0000","last_became_peered":"2024-05-26T22:54:56.894157+0000","last_unstale":"2024-05-26T22:54:58.917468+0000","last_undegraded":"2024-05-26T22:54:58.917468+0000","last_fullsized":"2024-05-26T22:54:58.917468+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_clean_scrub_stamp":"2024-05-26T22:54:55.841532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:33:55.154572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:58.922474+0000","last_change":"2024-05-26T22:54:57.923510+0000","last_active":"2024-05-26T22:54:58.922474+0000","last_peered":"2024-05-26T22:54:58.922474+0000","last_clean":"2024-05-26T22:54:58.922474+0000","last_became_active":"2024-05-26T22:54:57.923173+0000","last_became_peered":"2024-05-26T22:54:57.923173+0000","last_unstale":"2024-05-26T22:54:58.922474+0000","last_undegraded":"2024-05-26T22:54:58.922474+0000","last_fullsized":"2024-05-26T22:54:58.922474+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:46:04.317115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:58.918728+0000","last_change":"2024-05-26T22:54:56.894405+0000","last_active":"2024-05-26T22:54:58.918728+0000","last_peered":"2024-05-26T22:54:58.918728+0000","last_clean":"2024-05-26T22:54:58.918728+0000","last_became_active":"2024-05-26T22:54:56.894161+0000","last_became_peered":"2024-05-26T22:54:56.894161+0000","last_unstale":"2024-05-26T22:54:58.918728+0000","last_undegraded":"2024-05-26T22:54:58.918728+0000","last_fullsized":"2024-05-26T22:54:58.918728+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_clean_scrub_stamp":"2024-05-26T22:54:55.841532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:35:59.442921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:58.918743+0000","last_change":"2024-05-26T22:54:57.930127+0000","last_active":"2024-05-26T22:54:58.918743+0000","last_peered":"2024-05-26T22:54:58.918743+0000","last_clean":"2024-05-26T22:54:58.918743+0000","last_became_active":"2024-05-26T22:54:57.929268+0000","last_became_peered":"2024-05-26T22:54:57.929268+0000","last_unstale":"2024-05-26T22:54:58.918743+0000","last_undegraded":"2024-05-26T22:54:58.918743+0000","last_fullsized":"2024-05-26T22:54:58.918743+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:02:54.926056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"94'1","reported_seq":15,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.469425+0000","last_change":"2024-05-26T22:54:56.905163+0000","last_active":"2024-05-26T22:54:59.469425+0000","last_peered":"2024-05-26T22:54:59.469425+0000","last_clean":"2024-05-26T22:54:59.469425+0000","last_became_active":"2024-05-26T22:54:56.904858+0000","last_became_peered":"2024-05-26T22:54:56.904858+0000","last_unstale":"2024-05-26T22:54:59.469425+0000","last_undegraded":"2024-05-26T22:54:59.469425+0000","last_fullsized":"2024-05-26T22:54:59.469425+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_clean_scrub_stamp":"2024-05-26T22:54:55.841532+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:28:16.747428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":309,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.468417+0000","last_change":"2024-05-26T22:48:12.068917+0000","last_active":"2024-05-26T22:54:59.468417+0000","last_peered":"2024-05-26T22:54:59.468417+0000","last_clean":"2024-05-26T22:54:59.468417+0000","last_became_active":"2024-05-26T22:48:12.065934+0000","last_became_peered":"2024-05-26T22:48:12.065934+0000","last_unstale":"2024-05-26T22:54:59.468417+0000","last_undegraded":"2024-05-26T22:54:59.468417+0000","last_fullsized":"2024-05-26T22:54:59.468417+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:10.628122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:10.628122+0000","last_clean_scrub_stamp":"2024-05-26T22:48:10.628122+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:14:58.895911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574933,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32916,"kb_used_data":1736,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338924,"statfs":{"total":96636764160,"available":96603058176,"internally_reserved":0,"allocated":1777664,"data_stored":1101043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574933,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32344,"kb_used_data":1156,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339496,"statfs":{"total":96636764160,"available":96603643904,"internally_reserved":0,"allocated":1183744,"data_stored":511215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574934,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32340,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339500,"statfs":{"total":96636764160,"available":96603648000,"internally_reserved":0,"allocated":1179648,"data_stored":510633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574933,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32356,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339484,"statfs":{"total":96636764160,"available":96603631616,"internally_reserved":0,"allocated":1196032,"data_stored":512299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574933,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32344,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339496,"statfs":{"total":96636764160,"available":96603643904,"internally_reserved":0,"allocated":1191936,"data_stored":512197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574933,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32340,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339500,"statfs":{"total":96636764160,"available":96603648000,"internally_reserved":0,"allocated":1179648,"data_stored":510597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574933,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32336,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339504,"statfs":{"total":96636764160,"available":96603652096,"internally_reserved":0,"allocated":1175552,"data_stored":510563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574933,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32912,"kb_used_data":1732,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338928,"statfs":{"total":96636764160,"available":96603062272,"internally_reserved":0,"allocated":1773568,"data_stored":1101513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-26T22:55:02.323 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-26T22:55:02.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.563+0000 7f8f1de00640 1 Processor -- start 2024-05-26T22:55:02.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.563+0000 7f8f1de00640 1 -- start start 2024-05-26T22:55:02.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.564+0000 7f8f1de00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f8f18156750 0x7f8f1815fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:02.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.565+0000 7f8f1de00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f18160620 con 0x7f8f1806e8c0 2024-05-26T22:55:02.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.565+0000 7f8f1de00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f18160900 con 0x7f8f180cc7f0 2024-05-26T22:55:02.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.565+0000 7f8f1de00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f8f1815d770 con 0x7f8f18156750 2024-05-26T22:55:02.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.565+0000 7f8f1ca00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f8f18156750 0x7f8f1815fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:02.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.565+0000 7f8f0fe00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f8f1806e8c0 0x7f8f1815d060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:54572/0 (socket says 172.21.15.131:54572) 2024-05-26T22:55:02.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.565+0000 7f8f0fe00640 1 -- 172.21.15.131:0/1738568736 learned_addr learned my addr 172.21.15.131:0/1738568736 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:55:02.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.565+0000 7f8f1ca00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f8f18156750 0x7f8f1815fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41094/0 (socket says 172.21.15.131:41094) 2024-05-26T22:55:02.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.565+0000 7f8f1ca00640 1 -- 172.21.15.131:0/1738568736 >> v1:172.21.15.187:6789/0 conn(0x7f8f180cc7f0 legacy=0x7f8f180cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:02.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.565+0000 7f8f1ca00640 1 -- 172.21.15.131:0/1738568736 >> v1:172.21.15.131:6789/0 conn(0x7f8f1806e8c0 legacy=0x7f8f1815d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:02.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.565+0000 7f8f1ca00640 1 -- 172.21.15.131:0/1738568736 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f18160be0 con 0x7f8f18156750 2024-05-26T22:55:02.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.566+0000 7f8f1ca00640 1 --2- 172.21.15.131:0/1738568736 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f18156750 0x7f8f1815fd40 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f8f080097a0 tx=0x7f8f0802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adee2024214a7601 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:02.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.567+0000 7f8f17e00640 1 -- 172.21.15.131:0/1738568736 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f08037070 con 0x7f8f18156750 2024-05-26T22:55:02.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.567+0000 7f8f17e00640 1 -- 172.21.15.131:0/1738568736 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f08032440 con 0x7f8f18156750 2024-05-26T22:55:02.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.567+0000 7f8f17e00640 1 -- 172.21.15.131:0/1738568736 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f08032810 con 0x7f8f18156750 2024-05-26T22:55:02.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.567+0000 7f8f1de00640 1 -- 172.21.15.131:0/1738568736 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f18156750 msgr2=0x7f8f1815fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:02.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.567+0000 7f8f1de00640 1 --2- 172.21.15.131:0/1738568736 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f18156750 0x7f8f1815fd40 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f8f080097a0 tx=0x7f8f0802a1e0 comp rx=0 tx=0).stop 2024-05-26T22:55:02.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.567+0000 7f8f1de00640 1 -- 172.21.15.131:0/1738568736 shutdown_connections 2024-05-26T22:55:02.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.567+0000 7f8f1de00640 1 --2- 172.21.15.131:0/1738568736 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f18156750 0x7f8f1815fd40 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:02.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.567+0000 7f8f1de00640 1 -- 172.21.15.131:0/1738568736 >> 172.21.15.131:0/1738568736 conn(0x7f8f180cff40 msgr2=0x7f8f180d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:02.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.567+0000 7f8f1de00640 1 -- 172.21.15.131:0/1738568736 shutdown_connections 2024-05-26T22:55:02.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.567+0000 7f8f1de00640 1 -- 172.21.15.131:0/1738568736 wait complete. 2024-05-26T22:55:02.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.569+0000 7f8f1de00640 1 Processor -- start 2024-05-26T22:55:02.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.569+0000 7f8f1de00640 1 -- start start 2024-05-26T22:55:02.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.570+0000 7f8f1de00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8f1806e8c0 0x7f8f18159ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:02.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.570+0000 7f8f1de00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f180cc7f0 0x7f8f18156cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:02.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.570+0000 7f8f1de00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8f18156750 0x7f8f181571f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:02.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.570+0000 7f8f1de00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8f18156b30 con 0x7f8f180cc7f0 2024-05-26T22:55:02.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.570+0000 7f8f1de00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8f1815cbd0 con 0x7f8f18156750 2024-05-26T22:55:02.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.570+0000 7f8f1de00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f8f18160280 con 0x7f8f1806e8c0 2024-05-26T22:55:02.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.571+0000 7f8f0fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f180cc7f0 0x7f8f18156cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:02.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.571+0000 7f8f0fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f180cc7f0 0x7f8f18156cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41104/0 (socket says 172.21.15.131:41104) 2024-05-26T22:55:02.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.571+0000 7f8f0fe00640 1 -- 172.21.15.131:0/2363568776 learned_addr learned my addr 172.21.15.131:0/2363568776 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:55:02.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.571+0000 7f8f1ca00640 1 --2- 172.21.15.131:0/2363568776 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8f1806e8c0 0x7f8f18159ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:02.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.571+0000 7f8f1d400640 1 --2- 172.21.15.131:0/2363568776 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8f18156750 0x7f8f181571f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:02.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.571+0000 7f8f0fe00640 1 -- 172.21.15.131:0/2363568776 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8f1806e8c0 msgr2=0x7f8f18159ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:02.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.571+0000 7f8f0fe00640 1 --2- 172.21.15.131:0/2363568776 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8f1806e8c0 0x7f8f18159ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:02.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.571+0000 7f8f0fe00640 1 -- 172.21.15.131:0/2363568776 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8f18156750 msgr2=0x7f8f181571f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:02.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.571+0000 7f8f0fe00640 1 --2- 172.21.15.131:0/2363568776 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8f18156750 0x7f8f181571f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:02.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.571+0000 7f8f0fe00640 1 -- 172.21.15.131:0/2363568776 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f18158f80 con 0x7f8f180cc7f0 2024-05-26T22:55:02.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.571+0000 7f8f1ca00640 1 --2- 172.21.15.131:0/2363568776 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8f1806e8c0 0x7f8f18159ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:55:02.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.571+0000 7f8f0fe00640 1 --2- 172.21.15.131:0/2363568776 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f180cc7f0 0x7f8f18156cb0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f8f0000c430 tx=0x7f8f0000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:02.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.571+0000 7f8f15600640 1 -- 172.21.15.131:0/2363568776 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f00017070 con 0x7f8f180cc7f0 2024-05-26T22:55:02.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.572+0000 7f8f15600640 1 -- 172.21.15.131:0/2363568776 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f00005ce0 con 0x7f8f180cc7f0 2024-05-26T22:55:02.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.572+0000 7f8f15600640 1 -- 172.21.15.131:0/2363568776 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f000072f0 con 0x7f8f180cc7f0 2024-05-26T22:55:02.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.572+0000 7f8f1de00640 1 -- 172.21.15.131:0/2363568776 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f181384c0 con 0x7f8f180cc7f0 2024-05-26T22:55:02.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.572+0000 7f8f1de00640 1 -- 172.21.15.131:0/2363568776 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f181594e0 con 0x7f8f180cc7f0 2024-05-26T22:55:02.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.573+0000 7f8f1de00640 1 -- 172.21.15.131:0/2363568776 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f180694e0 con 0x7f8f180cc7f0 2024-05-26T22:55:02.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.574+0000 7f8f15600640 1 -- 172.21.15.131:0/2363568776 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8f0001a070 con 0x7f8f180cc7f0 2024-05-26T22:55:02.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.575+0000 7f8f15600640 1 --2- 172.21.15.131:0/2363568776 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f8ef406be30 0x7f8ef406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:02.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.575+0000 7f8f15600640 1 -- 172.21.15.131:0/2363568776 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 5709+0+0 (secure 0 0 0) 0x7f8f0008a270 con 0x7f8f180cc7f0 2024-05-26T22:55:02.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.575+0000 7f8f1ca00640 1 --2- 172.21.15.131:0/2363568776 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f8ef406be30 0x7f8ef406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:02.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.576+0000 7f8f1ca00640 1 --2- 172.21.15.131:0/2363568776 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f8ef406be30 0x7f8ef406e2f0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f8f08034000 tx=0x7f8f0802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:02.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.580+0000 7f8f15600640 1 -- 172.21.15.131:0/2363568776 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f8f18158f80 con 0x7f8f180cc7f0 2024-05-26T22:55:02.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.773+0000 7f8f1de00640 1 -- 172.21.15.131:0/2363568776 --> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8f18145010 con 0x7f8ef406be30 2024-05-26T22:55:02.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.782+0000 7f8f15600640 1 -- 172.21.15.131:0/2363568776 <== mgr.4104 v2:172.21.15.131:6832/3484832947 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252084 (secure 0 0 0) 0x7f8f18145010 con 0x7f8ef406be30 2024-05-26T22:55:02.785 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:55:02.786 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-05-26T22:55:02.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.788+0000 7f8f1de00640 1 -- 172.21.15.131:0/2363568776 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f8ef406be30 msgr2=0x7f8ef406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:02.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.788+0000 7f8f1de00640 1 --2- 172.21.15.131:0/2363568776 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f8ef406be30 0x7f8ef406e2f0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f8f08034000 tx=0x7f8f0802e000 comp rx=0 tx=0).stop 2024-05-26T22:55:02.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.788+0000 7f8f1de00640 1 -- 172.21.15.131:0/2363568776 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f180cc7f0 msgr2=0x7f8f18156cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:02.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.788+0000 7f8f1de00640 1 --2- 172.21.15.131:0/2363568776 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f180cc7f0 0x7f8f18156cb0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f8f0000c430 tx=0x7f8f0000c8f0 comp rx=0 tx=0).stop 2024-05-26T22:55:02.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.788+0000 7f8f1de00640 1 -- 172.21.15.131:0/2363568776 shutdown_connections 2024-05-26T22:55:02.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.788+0000 7f8f1de00640 1 --2- 172.21.15.131:0/2363568776 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f8ef406be30 0x7f8ef406e2f0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:02.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.788+0000 7f8f1de00640 1 --2- 172.21.15.131:0/2363568776 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8f18156750 0x7f8f181571f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:02.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.788+0000 7f8f1de00640 1 --2- 172.21.15.131:0/2363568776 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8f180cc7f0 0x7f8f18156cb0 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:02.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.788+0000 7f8f1de00640 1 --2- 172.21.15.131:0/2363568776 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8f1806e8c0 0x7f8f18159ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:02.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.788+0000 7f8f1de00640 1 -- 172.21.15.131:0/2363568776 >> 172.21.15.131:0/2363568776 conn(0x7f8f180cff40 msgr2=0x7f8f18144280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:02.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.788+0000 7f8f1de00640 1 -- 172.21.15.131:0/2363568776 shutdown_connections 2024-05-26T22:55:02.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:02.788+0000 7f8f1de00640 1 -- 172.21.15.131:0/2363568776 wait complete. 2024-05-26T22:55:02.807 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":294,"stamp":"2024-05-26T22:55:02.517651+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":139,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":259888,"kb_used_data":10408,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754714832,"statfs":{"total":773094113280,"available":772827987968,"internally_reserved":0,"allocated":10657792,"data_stored":5270060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002021"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.468848+0000","last_change":"2024-05-26T22:54:57.918186+0000","last_active":"2024-05-26T22:54:59.468848+0000","last_peered":"2024-05-26T22:54:59.468848+0000","last_clean":"2024-05-26T22:54:59.468848+0000","last_became_active":"2024-05-26T22:54:57.914078+0000","last_became_peered":"2024-05-26T22:54:57.914078+0000","last_unstale":"2024-05-26T22:54:59.468848+0000","last_undegraded":"2024-05-26T22:54:59.468848+0000","last_fullsized":"2024-05-26T22:54:59.468848+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:17:44.544345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.469251+0000","last_change":"2024-05-26T22:54:57.919704+0000","last_active":"2024-05-26T22:54:59.469251+0000","last_peered":"2024-05-26T22:54:59.469251+0000","last_clean":"2024-05-26T22:54:59.469251+0000","last_became_active":"2024-05-26T22:54:57.918561+0000","last_became_peered":"2024-05-26T22:54:57.918561+0000","last_unstale":"2024-05-26T22:54:59.469251+0000","last_undegraded":"2024-05-26T22:54:59.469251+0000","last_fullsized":"2024-05-26T22:54:59.469251+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:23:02.876957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.467359+0000","last_change":"2024-05-26T22:54:57.916253+0000","last_active":"2024-05-26T22:54:59.467359+0000","last_peered":"2024-05-26T22:54:59.467359+0000","last_clean":"2024-05-26T22:54:59.467359+0000","last_became_active":"2024-05-26T22:54:57.915974+0000","last_became_peered":"2024-05-26T22:54:57.915974+0000","last_unstale":"2024-05-26T22:54:59.467359+0000","last_undegraded":"2024-05-26T22:54:59.467359+0000","last_fullsized":"2024-05-26T22:54:59.467359+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:03:18.231409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:58.918267+0000","last_change":"2024-05-26T22:54:57.936364+0000","last_active":"2024-05-26T22:54:58.918267+0000","last_peered":"2024-05-26T22:54:58.918267+0000","last_clean":"2024-05-26T22:54:58.918267+0000","last_became_active":"2024-05-26T22:54:57.936056+0000","last_became_peered":"2024-05-26T22:54:57.936056+0000","last_unstale":"2024-05-26T22:54:58.918267+0000","last_undegraded":"2024-05-26T22:54:58.918267+0000","last_fullsized":"2024-05-26T22:54:58.918267+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:34:05.444741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:58.922617+0000","last_change":"2024-05-26T22:54:57.920124+0000","last_active":"2024-05-26T22:54:58.922617+0000","last_peered":"2024-05-26T22:54:58.922617+0000","last_clean":"2024-05-26T22:54:58.922617+0000","last_became_active":"2024-05-26T22:54:57.918447+0000","last_became_peered":"2024-05-26T22:54:57.918447+0000","last_unstale":"2024-05-26T22:54:58.922617+0000","last_undegraded":"2024-05-26T22:54:58.922617+0000","last_fullsized":"2024-05-26T22:54:58.922617+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:17:20.674336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.468249+0000","last_change":"2024-05-26T22:54:57.925304+0000","last_active":"2024-05-26T22:54:59.468249+0000","last_peered":"2024-05-26T22:54:59.468249+0000","last_clean":"2024-05-26T22:54:59.468249+0000","last_became_active":"2024-05-26T22:54:57.924992+0000","last_became_peered":"2024-05-26T22:54:57.924992+0000","last_unstale":"2024-05-26T22:54:59.468249+0000","last_undegraded":"2024-05-26T22:54:59.468249+0000","last_fullsized":"2024-05-26T22:54:59.468249+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:08:38.719400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.468182+0000","last_change":"2024-05-26T22:54:57.924242+0000","last_active":"2024-05-26T22:54:59.468182+0000","last_peered":"2024-05-26T22:54:59.468182+0000","last_clean":"2024-05-26T22:54:59.468182+0000","last_became_active":"2024-05-26T22:54:57.923117+0000","last_became_peered":"2024-05-26T22:54:57.923117+0000","last_unstale":"2024-05-26T22:54:59.468182+0000","last_undegraded":"2024-05-26T22:54:59.468182+0000","last_fullsized":"2024-05-26T22:54:59.468182+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:18:01.132850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.468125+0000","last_change":"2024-05-26T22:54:57.913255+0000","last_active":"2024-05-26T22:54:59.468125+0000","last_peered":"2024-05-26T22:54:59.468125+0000","last_clean":"2024-05-26T22:54:59.468125+0000","last_became_active":"2024-05-26T22:54:57.912879+0000","last_became_peered":"2024-05-26T22:54:57.912879+0000","last_unstale":"2024-05-26T22:54:59.468125+0000","last_undegraded":"2024-05-26T22:54:59.468125+0000","last_fullsized":"2024-05-26T22:54:59.468125+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:13:37.825933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.149451+0000","last_change":"2024-05-26T22:54:57.933121+0000","last_active":"2024-05-26T22:54:59.149451+0000","last_peered":"2024-05-26T22:54:59.149451+0000","last_clean":"2024-05-26T22:54:59.149451+0000","last_became_active":"2024-05-26T22:54:57.932442+0000","last_became_peered":"2024-05-26T22:54:57.932442+0000","last_unstale":"2024-05-26T22:54:59.149451+0000","last_undegraded":"2024-05-26T22:54:59.149451+0000","last_fullsized":"2024-05-26T22:54:59.149451+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:08:38.719400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:58.923347+0000","last_change":"2024-05-26T22:54:57.908115+0000","last_active":"2024-05-26T22:54:58.923347+0000","last_peered":"2024-05-26T22:54:58.923347+0000","last_clean":"2024-05-26T22:54:58.923347+0000","last_became_active":"2024-05-26T22:54:57.907450+0000","last_became_peered":"2024-05-26T22:54:57.907450+0000","last_unstale":"2024-05-26T22:54:58.923347+0000","last_undegraded":"2024-05-26T22:54:58.923347+0000","last_fullsized":"2024-05-26T22:54:58.923347+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:02:32.509633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.468753+0000","last_change":"2024-05-26T22:54:57.926696+0000","last_active":"2024-05-26T22:54:59.468753+0000","last_peered":"2024-05-26T22:54:59.468753+0000","last_clean":"2024-05-26T22:54:59.468753+0000","last_became_active":"2024-05-26T22:54:57.925819+0000","last_became_peered":"2024-05-26T22:54:57.925819+0000","last_unstale":"2024-05-26T22:54:59.468753+0000","last_undegraded":"2024-05-26T22:54:59.468753+0000","last_fullsized":"2024-05-26T22:54:59.468753+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:45:18.352727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.148734+0000","last_change":"2024-05-26T22:54:57.931493+0000","last_active":"2024-05-26T22:54:59.148734+0000","last_peered":"2024-05-26T22:54:59.148734+0000","last_clean":"2024-05-26T22:54:59.148734+0000","last_became_active":"2024-05-26T22:54:57.931028+0000","last_became_peered":"2024-05-26T22:54:57.931028+0000","last_unstale":"2024-05-26T22:54:59.148734+0000","last_undegraded":"2024-05-26T22:54:59.148734+0000","last_fullsized":"2024-05-26T22:54:59.148734+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:27:18.238559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:58.917954+0000","last_change":"2024-05-26T22:54:57.938554+0000","last_active":"2024-05-26T22:54:58.917954+0000","last_peered":"2024-05-26T22:54:58.917954+0000","last_clean":"2024-05-26T22:54:58.917954+0000","last_became_active":"2024-05-26T22:54:57.938196+0000","last_became_peered":"2024-05-26T22:54:57.938196+0000","last_unstale":"2024-05-26T22:54:58.917954+0000","last_undegraded":"2024-05-26T22:54:58.917954+0000","last_fullsized":"2024-05-26T22:54:58.917954+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:17:20.674336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:58.917641+0000","last_change":"2024-05-26T22:54:57.922229+0000","last_active":"2024-05-26T22:54:58.917641+0000","last_peered":"2024-05-26T22:54:58.917641+0000","last_clean":"2024-05-26T22:54:58.917641+0000","last_became_active":"2024-05-26T22:54:57.921924+0000","last_became_peered":"2024-05-26T22:54:57.921924+0000","last_unstale":"2024-05-26T22:54:58.917641+0000","last_undegraded":"2024-05-26T22:54:58.917641+0000","last_fullsized":"2024-05-26T22:54:58.917641+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:41:35.115195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:58.917329+0000","last_change":"2024-05-26T22:54:57.918087+0000","last_active":"2024-05-26T22:54:58.917329+0000","last_peered":"2024-05-26T22:54:58.917329+0000","last_clean":"2024-05-26T22:54:58.917329+0000","last_became_active":"2024-05-26T22:54:57.917127+0000","last_became_peered":"2024-05-26T22:54:57.917127+0000","last_unstale":"2024-05-26T22:54:58.917329+0000","last_undegraded":"2024-05-26T22:54:58.917329+0000","last_fullsized":"2024-05-26T22:54:58.917329+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:52:37.214179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.468026+0000","last_change":"2024-05-26T22:54:57.924322+0000","last_active":"2024-05-26T22:54:59.468026+0000","last_peered":"2024-05-26T22:54:59.468026+0000","last_clean":"2024-05-26T22:54:59.468026+0000","last_became_active":"2024-05-26T22:54:57.923770+0000","last_became_peered":"2024-05-26T22:54:57.923770+0000","last_unstale":"2024-05-26T22:54:59.468026+0000","last_undegraded":"2024-05-26T22:54:59.468026+0000","last_fullsized":"2024-05-26T22:54:59.468026+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T23:32:13.073868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.469588+0000","last_change":"2024-05-26T22:54:57.924552+0000","last_active":"2024-05-26T22:54:59.469588+0000","last_peered":"2024-05-26T22:54:59.469588+0000","last_clean":"2024-05-26T22:54:59.469588+0000","last_became_active":"2024-05-26T22:54:57.924261+0000","last_became_peered":"2024-05-26T22:54:57.924261+0000","last_unstale":"2024-05-26T22:54:59.469588+0000","last_undegraded":"2024-05-26T22:54:59.469588+0000","last_fullsized":"2024-05-26T22:54:59.469588+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:14:19.122236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:58.918517+0000","last_change":"2024-05-26T22:54:57.936956+0000","last_active":"2024-05-26T22:54:58.918517+0000","last_peered":"2024-05-26T22:54:58.918517+0000","last_clean":"2024-05-26T22:54:58.918517+0000","last_became_active":"2024-05-26T22:54:57.936691+0000","last_became_peered":"2024-05-26T22:54:57.936691+0000","last_unstale":"2024-05-26T22:54:58.918517+0000","last_undegraded":"2024-05-26T22:54:58.918517+0000","last_fullsized":"2024-05-26T22:54:58.918517+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:35:20.830276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:58.918961+0000","last_change":"2024-05-26T22:54:57.935715+0000","last_active":"2024-05-26T22:54:58.918961+0000","last_peered":"2024-05-26T22:54:58.918961+0000","last_clean":"2024-05-26T22:54:58.918961+0000","last_became_active":"2024-05-26T22:54:57.933970+0000","last_became_peered":"2024-05-26T22:54:57.933970+0000","last_unstale":"2024-05-26T22:54:58.918961+0000","last_undegraded":"2024-05-26T22:54:58.918961+0000","last_fullsized":"2024-05-26T22:54:58.918961+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:04:24.105687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.149000+0000","last_change":"2024-05-26T22:54:57.931818+0000","last_active":"2024-05-26T22:54:59.149000+0000","last_peered":"2024-05-26T22:54:59.149000+0000","last_clean":"2024-05-26T22:54:59.149000+0000","last_became_active":"2024-05-26T22:54:57.931575+0000","last_became_peered":"2024-05-26T22:54:57.931575+0000","last_unstale":"2024-05-26T22:54:59.149000+0000","last_undegraded":"2024-05-26T22:54:59.149000+0000","last_fullsized":"2024-05-26T22:54:59.149000+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:18:01.132850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:58.917779+0000","last_change":"2024-05-26T22:54:57.924324+0000","last_active":"2024-05-26T22:54:58.917779+0000","last_peered":"2024-05-26T22:54:58.917779+0000","last_clean":"2024-05-26T22:54:58.917779+0000","last_became_active":"2024-05-26T22:54:57.923195+0000","last_became_peered":"2024-05-26T22:54:57.923195+0000","last_unstale":"2024-05-26T22:54:58.917779+0000","last_undegraded":"2024-05-26T22:54:58.917779+0000","last_fullsized":"2024-05-26T22:54:58.917779+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:20:20.761797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.467192+0000","last_change":"2024-05-26T22:54:57.919100+0000","last_active":"2024-05-26T22:54:59.467192+0000","last_peered":"2024-05-26T22:54:59.467192+0000","last_clean":"2024-05-26T22:54:59.467192+0000","last_became_active":"2024-05-26T22:54:57.918795+0000","last_became_peered":"2024-05-26T22:54:57.918795+0000","last_unstale":"2024-05-26T22:54:59.467192+0000","last_undegraded":"2024-05-26T22:54:59.467192+0000","last_fullsized":"2024-05-26T22:54:59.467192+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T23:00:15.189681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.466940+0000","last_change":"2024-05-26T22:54:57.917401+0000","last_active":"2024-05-26T22:54:59.466940+0000","last_peered":"2024-05-26T22:54:59.466940+0000","last_clean":"2024-05-26T22:54:59.466940+0000","last_became_active":"2024-05-26T22:54:57.917104+0000","last_became_peered":"2024-05-26T22:54:57.917104+0000","last_unstale":"2024-05-26T22:54:59.466940+0000","last_undegraded":"2024-05-26T22:54:59.466940+0000","last_fullsized":"2024-05-26T22:54:59.466940+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:09:57.630908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.468270+0000","last_change":"2024-05-26T22:54:57.931944+0000","last_active":"2024-05-26T22:54:59.468270+0000","last_peered":"2024-05-26T22:54:59.468270+0000","last_clean":"2024-05-26T22:54:59.468270+0000","last_became_active":"2024-05-26T22:54:57.931255+0000","last_became_peered":"2024-05-26T22:54:57.931255+0000","last_unstale":"2024-05-26T22:54:59.468270+0000","last_undegraded":"2024-05-26T22:54:59.468270+0000","last_fullsized":"2024-05-26T22:54:59.468270+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:52:37.214179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:58.923510+0000","last_change":"2024-05-26T22:54:57.912470+0000","last_active":"2024-05-26T22:54:58.923510+0000","last_peered":"2024-05-26T22:54:58.923510+0000","last_clean":"2024-05-26T22:54:58.923510+0000","last_became_active":"2024-05-26T22:54:57.911461+0000","last_became_peered":"2024-05-26T22:54:57.911461+0000","last_unstale":"2024-05-26T22:54:58.923510+0000","last_undegraded":"2024-05-26T22:54:58.923510+0000","last_fullsized":"2024-05-26T22:54:58.923510+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:01:36.256567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.469629+0000","last_change":"2024-05-26T22:54:57.908004+0000","last_active":"2024-05-26T22:54:59.469629+0000","last_peered":"2024-05-26T22:54:59.469629+0000","last_clean":"2024-05-26T22:54:59.469629+0000","last_became_active":"2024-05-26T22:54:57.905624+0000","last_became_peered":"2024-05-26T22:54:57.905624+0000","last_unstale":"2024-05-26T22:54:59.469629+0000","last_undegraded":"2024-05-26T22:54:59.469629+0000","last_fullsized":"2024-05-26T22:54:59.469629+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:42:41.925687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.468874+0000","last_change":"2024-05-26T22:54:57.919261+0000","last_active":"2024-05-26T22:54:59.468874+0000","last_peered":"2024-05-26T22:54:59.468874+0000","last_clean":"2024-05-26T22:54:59.468874+0000","last_became_active":"2024-05-26T22:54:57.918886+0000","last_became_peered":"2024-05-26T22:54:57.918886+0000","last_unstale":"2024-05-26T22:54:59.468874+0000","last_undegraded":"2024-05-26T22:54:59.468874+0000","last_fullsized":"2024-05-26T22:54:59.468874+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:17:20.674336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:58.922807+0000","last_change":"2024-05-26T22:54:57.926694+0000","last_active":"2024-05-26T22:54:58.922807+0000","last_peered":"2024-05-26T22:54:58.922807+0000","last_clean":"2024-05-26T22:54:58.922807+0000","last_became_active":"2024-05-26T22:54:57.925650+0000","last_became_peered":"2024-05-26T22:54:57.925650+0000","last_unstale":"2024-05-26T22:54:58.922807+0000","last_undegraded":"2024-05-26T22:54:58.922807+0000","last_fullsized":"2024-05-26T22:54:58.922807+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:34:05.444741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.467824+0000","last_change":"2024-05-26T22:54:57.904422+0000","last_active":"2024-05-26T22:54:59.467824+0000","last_peered":"2024-05-26T22:54:59.467824+0000","last_clean":"2024-05-26T22:54:59.467824+0000","last_became_active":"2024-05-26T22:54:57.903593+0000","last_became_peered":"2024-05-26T22:54:57.903593+0000","last_unstale":"2024-05-26T22:54:59.467824+0000","last_undegraded":"2024-05-26T22:54:59.467824+0000","last_fullsized":"2024-05-26T22:54:59.467824+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:24:41.881300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:58.918824+0000","last_change":"2024-05-26T22:54:57.933567+0000","last_active":"2024-05-26T22:54:58.918824+0000","last_peered":"2024-05-26T22:54:58.918824+0000","last_clean":"2024-05-26T22:54:58.918824+0000","last_became_active":"2024-05-26T22:54:57.932909+0000","last_became_peered":"2024-05-26T22:54:57.932909+0000","last_unstale":"2024-05-26T22:54:58.918824+0000","last_undegraded":"2024-05-26T22:54:58.918824+0000","last_fullsized":"2024-05-26T22:54:58.918824+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:03:07.441337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:58.917362+0000","last_change":"2024-05-26T22:54:57.920028+0000","last_active":"2024-05-26T22:54:58.917362+0000","last_peered":"2024-05-26T22:54:58.917362+0000","last_clean":"2024-05-26T22:54:58.917362+0000","last_became_active":"2024-05-26T22:54:57.919450+0000","last_became_peered":"2024-05-26T22:54:57.919450+0000","last_unstale":"2024-05-26T22:54:58.917362+0000","last_undegraded":"2024-05-26T22:54:58.917362+0000","last_fullsized":"2024-05-26T22:54:58.917362+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:46:04.317115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.467521+0000","last_change":"2024-05-26T22:54:57.906455+0000","last_active":"2024-05-26T22:54:59.467521+0000","last_peered":"2024-05-26T22:54:59.467521+0000","last_clean":"2024-05-26T22:54:59.467521+0000","last_became_active":"2024-05-26T22:54:57.906162+0000","last_became_peered":"2024-05-26T22:54:57.906162+0000","last_unstale":"2024-05-26T22:54:59.467521+0000","last_undegraded":"2024-05-26T22:54:59.467521+0000","last_fullsized":"2024-05-26T22:54:59.467521+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:27:41.142373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:58.918595+0000","last_change":"2024-05-26T22:54:57.933428+0000","last_active":"2024-05-26T22:54:58.918595+0000","last_peered":"2024-05-26T22:54:58.918595+0000","last_clean":"2024-05-26T22:54:58.918595+0000","last_became_active":"2024-05-26T22:54:57.932926+0000","last_became_peered":"2024-05-26T22:54:57.932926+0000","last_unstale":"2024-05-26T22:54:58.918595+0000","last_undegraded":"2024-05-26T22:54:58.918595+0000","last_fullsized":"2024-05-26T22:54:58.918595+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:05:42.637697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"94'4","reported_seq":18,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.469462+0000","last_change":"2024-05-26T22:54:56.893757+0000","last_active":"2024-05-26T22:54:59.469462+0000","last_peered":"2024-05-26T22:54:59.469462+0000","last_clean":"2024-05-26T22:54:59.469462+0000","last_became_active":"2024-05-26T22:54:56.893411+0000","last_became_peered":"2024-05-26T22:54:56.893411+0000","last_unstale":"2024-05-26T22:54:59.469462+0000","last_undegraded":"2024-05-26T22:54:59.469462+0000","last_fullsized":"2024-05-26T22:54:59.469462+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_clean_scrub_stamp":"2024-05-26T22:54:55.841532+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:08:37.703863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:58.919346+0000","last_change":"2024-05-26T22:54:57.928717+0000","last_active":"2024-05-26T22:54:58.919346+0000","last_peered":"2024-05-26T22:54:58.919346+0000","last_clean":"2024-05-26T22:54:58.919346+0000","last_became_active":"2024-05-26T22:54:57.928417+0000","last_became_peered":"2024-05-26T22:54:57.928417+0000","last_unstale":"2024-05-26T22:54:58.919346+0000","last_undegraded":"2024-05-26T22:54:58.919346+0000","last_fullsized":"2024-05-26T22:54:58.919346+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:02:32.509633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"94'1","reported_seq":15,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.468449+0000","last_change":"2024-05-26T22:54:56.902755+0000","last_active":"2024-05-26T22:54:59.468449+0000","last_peered":"2024-05-26T22:54:59.468449+0000","last_clean":"2024-05-26T22:54:59.468449+0000","last_became_active":"2024-05-26T22:54:56.902439+0000","last_became_peered":"2024-05-26T22:54:56.902439+0000","last_unstale":"2024-05-26T22:54:59.468449+0000","last_undegraded":"2024-05-26T22:54:59.468449+0000","last_fullsized":"2024-05-26T22:54:59.468449+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_clean_scrub_stamp":"2024-05-26T22:54:55.841532+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T23:23:59.941266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.467842+0000","last_change":"2024-05-26T22:54:57.922650+0000","last_active":"2024-05-26T22:54:59.467842+0000","last_peered":"2024-05-26T22:54:59.467842+0000","last_clean":"2024-05-26T22:54:59.467842+0000","last_became_active":"2024-05-26T22:54:57.922379+0000","last_became_peered":"2024-05-26T22:54:57.922379+0000","last_unstale":"2024-05-26T22:54:59.467842+0000","last_undegraded":"2024-05-26T22:54:59.467842+0000","last_fullsized":"2024-05-26T22:54:59.467842+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:57:55.839877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"94'2","reported_seq":16,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.468907+0000","last_change":"2024-05-26T22:54:56.887767+0000","last_active":"2024-05-26T22:54:59.468907+0000","last_peered":"2024-05-26T22:54:59.468907+0000","last_clean":"2024-05-26T22:54:59.468907+0000","last_became_active":"2024-05-26T22:54:56.886622+0000","last_became_peered":"2024-05-26T22:54:56.886622+0000","last_unstale":"2024-05-26T22:54:59.468907+0000","last_undegraded":"2024-05-26T22:54:59.468907+0000","last_fullsized":"2024-05-26T22:54:59.468907+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_clean_scrub_stamp":"2024-05-26T22:54:55.841532+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:27:17.223023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.149568+0000","last_change":"2024-05-26T22:54:57.930000+0000","last_active":"2024-05-26T22:54:59.149568+0000","last_peered":"2024-05-26T22:54:59.149568+0000","last_clean":"2024-05-26T22:54:59.149568+0000","last_became_active":"2024-05-26T22:54:57.928087+0000","last_became_peered":"2024-05-26T22:54:57.928087+0000","last_unstale":"2024-05-26T22:54:59.149568+0000","last_undegraded":"2024-05-26T22:54:59.149568+0000","last_fullsized":"2024-05-26T22:54:59.149568+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:52:37.214179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"94'2","reported_seq":16,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.468739+0000","last_change":"2024-05-26T22:54:56.899380+0000","last_active":"2024-05-26T22:54:59.468739+0000","last_peered":"2024-05-26T22:54:59.468739+0000","last_clean":"2024-05-26T22:54:59.468739+0000","last_became_active":"2024-05-26T22:54:56.898426+0000","last_became_peered":"2024-05-26T22:54:56.898426+0000","last_unstale":"2024-05-26T22:54:59.468739+0000","last_undegraded":"2024-05-26T22:54:59.468739+0000","last_fullsized":"2024-05-26T22:54:59.468739+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_clean_scrub_stamp":"2024-05-26T22:54:55.841532+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:24:01.985079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.466784+0000","last_change":"2024-05-26T22:54:57.919904+0000","last_active":"2024-05-26T22:54:59.466784+0000","last_peered":"2024-05-26T22:54:59.466784+0000","last_clean":"2024-05-26T22:54:59.466784+0000","last_became_active":"2024-05-26T22:54:57.919622+0000","last_became_peered":"2024-05-26T22:54:57.919622+0000","last_unstale":"2024-05-26T22:54:59.466784+0000","last_undegraded":"2024-05-26T22:54:59.466784+0000","last_fullsized":"2024-05-26T22:54:59.466784+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:17:44.544345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"94'1","reported_seq":15,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.467304+0000","last_change":"2024-05-26T22:54:56.894238+0000","last_active":"2024-05-26T22:54:59.467304+0000","last_peered":"2024-05-26T22:54:59.467304+0000","last_clean":"2024-05-26T22:54:59.467304+0000","last_became_active":"2024-05-26T22:54:56.893894+0000","last_became_peered":"2024-05-26T22:54:56.893894+0000","last_unstale":"2024-05-26T22:54:59.467304+0000","last_undegraded":"2024-05-26T22:54:59.467304+0000","last_fullsized":"2024-05-26T22:54:59.467304+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_clean_scrub_stamp":"2024-05-26T22:54:55.841532+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:22:59.025816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.468695+0000","last_change":"2024-05-26T22:54:57.924957+0000","last_active":"2024-05-26T22:54:59.468695+0000","last_peered":"2024-05-26T22:54:59.468695+0000","last_clean":"2024-05-26T22:54:59.468695+0000","last_became_active":"2024-05-26T22:54:57.924707+0000","last_became_peered":"2024-05-26T22:54:57.924707+0000","last_unstale":"2024-05-26T22:54:59.468695+0000","last_undegraded":"2024-05-26T22:54:59.468695+0000","last_fullsized":"2024-05-26T22:54:59.468695+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:46:04.317115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"94'1","reported_seq":15,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.467483+0000","last_change":"2024-05-26T22:54:56.885319+0000","last_active":"2024-05-26T22:54:59.467483+0000","last_peered":"2024-05-26T22:54:59.467483+0000","last_clean":"2024-05-26T22:54:59.467483+0000","last_became_active":"2024-05-26T22:54:56.884985+0000","last_became_peered":"2024-05-26T22:54:56.884985+0000","last_unstale":"2024-05-26T22:54:59.467483+0000","last_undegraded":"2024-05-26T22:54:59.467483+0000","last_fullsized":"2024-05-26T22:54:59.467483+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_clean_scrub_stamp":"2024-05-26T22:54:55.841532+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:39:59.569280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:58.917316+0000","last_change":"2024-05-26T22:54:57.931093+0000","last_active":"2024-05-26T22:54:58.917316+0000","last_peered":"2024-05-26T22:54:58.917316+0000","last_clean":"2024-05-26T22:54:58.917316+0000","last_became_active":"2024-05-26T22:54:57.930567+0000","last_became_peered":"2024-05-26T22:54:57.930567+0000","last_unstale":"2024-05-26T22:54:58.917316+0000","last_undegraded":"2024-05-26T22:54:58.917316+0000","last_fullsized":"2024-05-26T22:54:58.917316+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:08:38.719400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:58.922791+0000","last_change":"2024-05-26T22:54:56.884617+0000","last_active":"2024-05-26T22:54:58.922791+0000","last_peered":"2024-05-26T22:54:58.922791+0000","last_clean":"2024-05-26T22:54:58.922791+0000","last_became_active":"2024-05-26T22:54:56.880110+0000","last_became_peered":"2024-05-26T22:54:56.880110+0000","last_unstale":"2024-05-26T22:54:58.922791+0000","last_undegraded":"2024-05-26T22:54:58.922791+0000","last_fullsized":"2024-05-26T22:54:58.922791+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_clean_scrub_stamp":"2024-05-26T22:54:55.841532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:26:42.775770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.148640+0000","last_change":"2024-05-26T22:54:57.922822+0000","last_active":"2024-05-26T22:54:59.148640+0000","last_peered":"2024-05-26T22:54:59.148640+0000","last_clean":"2024-05-26T22:54:59.148640+0000","last_became_active":"2024-05-26T22:54:57.922543+0000","last_became_peered":"2024-05-26T22:54:57.922543+0000","last_unstale":"2024-05-26T22:54:59.148640+0000","last_undegraded":"2024-05-26T22:54:59.148640+0000","last_fullsized":"2024-05-26T22:54:59.148640+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:52:52.534985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.467889+0000","last_change":"2024-05-26T22:54:56.889205+0000","last_active":"2024-05-26T22:54:59.467889+0000","last_peered":"2024-05-26T22:54:59.467889+0000","last_clean":"2024-05-26T22:54:59.467889+0000","last_became_active":"2024-05-26T22:54:56.888805+0000","last_became_peered":"2024-05-26T22:54:56.888805+0000","last_unstale":"2024-05-26T22:54:59.467889+0000","last_undegraded":"2024-05-26T22:54:59.467889+0000","last_fullsized":"2024-05-26T22:54:59.467889+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_clean_scrub_stamp":"2024-05-26T22:54:55.841532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:36:16.322392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.469620+0000","last_change":"2024-05-26T22:54:57.915217+0000","last_active":"2024-05-26T22:54:59.469620+0000","last_peered":"2024-05-26T22:54:59.469620+0000","last_clean":"2024-05-26T22:54:59.469620+0000","last_became_active":"2024-05-26T22:54:57.914894+0000","last_became_peered":"2024-05-26T22:54:57.914894+0000","last_unstale":"2024-05-26T22:54:59.469620+0000","last_undegraded":"2024-05-26T22:54:59.469620+0000","last_fullsized":"2024-05-26T22:54:59.469620+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:56:23.782834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.470096+0000","last_change":"2024-05-26T22:54:56.896440+0000","last_active":"2024-05-26T22:54:59.470096+0000","last_peered":"2024-05-26T22:54:59.470096+0000","last_clean":"2024-05-26T22:54:59.470096+0000","last_became_active":"2024-05-26T22:54:56.896177+0000","last_became_peered":"2024-05-26T22:54:56.896177+0000","last_unstale":"2024-05-26T22:54:59.470096+0000","last_undegraded":"2024-05-26T22:54:59.470096+0000","last_fullsized":"2024-05-26T22:54:59.470096+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_clean_scrub_stamp":"2024-05-26T22:54:55.841532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:31:35.474879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.149761+0000","last_change":"2024-05-26T22:54:57.921014+0000","last_active":"2024-05-26T22:54:59.149761+0000","last_peered":"2024-05-26T22:54:59.149761+0000","last_clean":"2024-05-26T22:54:59.149761+0000","last_became_active":"2024-05-26T22:54:57.920738+0000","last_became_peered":"2024-05-26T22:54:57.920738+0000","last_unstale":"2024-05-26T22:54:59.149761+0000","last_undegraded":"2024-05-26T22:54:59.149761+0000","last_fullsized":"2024-05-26T22:54:59.149761+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:46:04.317115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:58.918615+0000","last_change":"2024-05-26T22:54:56.890912+0000","last_active":"2024-05-26T22:54:58.918615+0000","last_peered":"2024-05-26T22:54:58.918615+0000","last_clean":"2024-05-26T22:54:58.918615+0000","last_became_active":"2024-05-26T22:54:56.890618+0000","last_became_peered":"2024-05-26T22:54:56.890618+0000","last_unstale":"2024-05-26T22:54:58.918615+0000","last_undegraded":"2024-05-26T22:54:58.918615+0000","last_fullsized":"2024-05-26T22:54:58.918615+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_clean_scrub_stamp":"2024-05-26T22:54:55.841532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:26:42.775770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:58.923511+0000","last_change":"2024-05-26T22:54:57.903878+0000","last_active":"2024-05-26T22:54:58.923511+0000","last_peered":"2024-05-26T22:54:58.923511+0000","last_clean":"2024-05-26T22:54:58.923511+0000","last_became_active":"2024-05-26T22:54:57.901707+0000","last_became_peered":"2024-05-26T22:54:57.901707+0000","last_unstale":"2024-05-26T22:54:58.923511+0000","last_undegraded":"2024-05-26T22:54:58.923511+0000","last_fullsized":"2024-05-26T22:54:58.923511+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:27:30.904396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"94'3","reported_seq":17,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.469427+0000","last_change":"2024-05-26T22:54:56.904426+0000","last_active":"2024-05-26T22:54:59.469427+0000","last_peered":"2024-05-26T22:54:59.469427+0000","last_clean":"2024-05-26T22:54:59.469427+0000","last_became_active":"2024-05-26T22:54:56.904089+0000","last_became_peered":"2024-05-26T22:54:56.904089+0000","last_unstale":"2024-05-26T22:54:59.469427+0000","last_undegraded":"2024-05-26T22:54:59.469427+0000","last_fullsized":"2024-05-26T22:54:59.469427+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_clean_scrub_stamp":"2024-05-26T22:54:55.841532+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:44:33.082467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.469178+0000","last_change":"2024-05-26T22:54:57.923341+0000","last_active":"2024-05-26T22:54:59.469178+0000","last_peered":"2024-05-26T22:54:59.469178+0000","last_clean":"2024-05-26T22:54:59.469178+0000","last_became_active":"2024-05-26T22:54:57.923044+0000","last_became_peered":"2024-05-26T22:54:57.923044+0000","last_unstale":"2024-05-26T22:54:59.469178+0000","last_undegraded":"2024-05-26T22:54:59.469178+0000","last_fullsized":"2024-05-26T22:54:59.469178+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:34:05.444741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"94'6","reported_seq":20,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.149441+0000","last_change":"2024-05-26T22:54:56.901384+0000","last_active":"2024-05-26T22:54:59.149441+0000","last_peered":"2024-05-26T22:54:59.149441+0000","last_clean":"2024-05-26T22:54:59.149441+0000","last_became_active":"2024-05-26T22:54:56.900899+0000","last_became_peered":"2024-05-26T22:54:56.900899+0000","last_unstale":"2024-05-26T22:54:59.149441+0000","last_undegraded":"2024-05-26T22:54:59.149441+0000","last_fullsized":"2024-05-26T22:54:59.149441+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_clean_scrub_stamp":"2024-05-26T22:54:55.841532+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:58:36.036557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:58.917796+0000","last_change":"2024-05-26T22:54:57.920690+0000","last_active":"2024-05-26T22:54:58.917796+0000","last_peered":"2024-05-26T22:54:58.917796+0000","last_clean":"2024-05-26T22:54:58.917796+0000","last_became_active":"2024-05-26T22:54:57.920434+0000","last_became_peered":"2024-05-26T22:54:57.920434+0000","last_unstale":"2024-05-26T22:54:58.917796+0000","last_undegraded":"2024-05-26T22:54:58.917796+0000","last_fullsized":"2024-05-26T22:54:58.917796+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:45:18.352727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"94'2","reported_seq":16,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:58.918184+0000","last_change":"2024-05-26T22:54:56.898635+0000","last_active":"2024-05-26T22:54:58.918184+0000","last_peered":"2024-05-26T22:54:58.918184+0000","last_clean":"2024-05-26T22:54:58.918184+0000","last_became_active":"2024-05-26T22:54:56.898383+0000","last_became_peered":"2024-05-26T22:54:56.898383+0000","last_unstale":"2024-05-26T22:54:58.918184+0000","last_undegraded":"2024-05-26T22:54:58.918184+0000","last_fullsized":"2024-05-26T22:54:58.918184+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_clean_scrub_stamp":"2024-05-26T22:54:55.841532+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T23:50:14.994019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.469107+0000","last_change":"2024-05-26T22:54:57.926490+0000","last_active":"2024-05-26T22:54:59.469107+0000","last_peered":"2024-05-26T22:54:59.469107+0000","last_clean":"2024-05-26T22:54:59.469107+0000","last_became_active":"2024-05-26T22:54:57.921270+0000","last_became_peered":"2024-05-26T22:54:57.921270+0000","last_unstale":"2024-05-26T22:54:59.469107+0000","last_undegraded":"2024-05-26T22:54:59.469107+0000","last_fullsized":"2024-05-26T22:54:59.469107+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:44:34.098004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:58.918061+0000","last_change":"2024-05-26T22:54:56.889666+0000","last_active":"2024-05-26T22:54:58.918061+0000","last_peered":"2024-05-26T22:54:58.918061+0000","last_clean":"2024-05-26T22:54:58.918061+0000","last_became_active":"2024-05-26T22:54:56.889288+0000","last_became_peered":"2024-05-26T22:54:56.889288+0000","last_unstale":"2024-05-26T22:54:58.918061+0000","last_undegraded":"2024-05-26T22:54:58.918061+0000","last_fullsized":"2024-05-26T22:54:58.918061+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_clean_scrub_stamp":"2024-05-26T22:54:55.841532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:29:00.893581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.148643+0000","last_change":"2024-05-26T22:54:57.916031+0000","last_active":"2024-05-26T22:54:59.148643+0000","last_peered":"2024-05-26T22:54:59.148643+0000","last_clean":"2024-05-26T22:54:59.148643+0000","last_became_active":"2024-05-26T22:54:57.915672+0000","last_became_peered":"2024-05-26T22:54:57.915672+0000","last_unstale":"2024-05-26T22:54:59.148643+0000","last_undegraded":"2024-05-26T22:54:59.148643+0000","last_fullsized":"2024-05-26T22:54:59.148643+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:24:03.000615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.149441+0000","last_change":"2024-05-26T22:54:56.902700+0000","last_active":"2024-05-26T22:54:59.149441+0000","last_peered":"2024-05-26T22:54:59.149441+0000","last_clean":"2024-05-26T22:54:59.149441+0000","last_became_active":"2024-05-26T22:54:56.902290+0000","last_became_peered":"2024-05-26T22:54:56.902290+0000","last_unstale":"2024-05-26T22:54:59.149441+0000","last_undegraded":"2024-05-26T22:54:59.149441+0000","last_fullsized":"2024-05-26T22:54:59.149441+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_clean_scrub_stamp":"2024-05-26T22:54:55.841532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:17:43.528808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:58.918346+0000","last_change":"2024-05-26T22:54:57.935060+0000","last_active":"2024-05-26T22:54:58.918346+0000","last_peered":"2024-05-26T22:54:58.918346+0000","last_clean":"2024-05-26T22:54:58.918346+0000","last_became_active":"2024-05-26T22:54:57.931110+0000","last_became_peered":"2024-05-26T22:54:57.931110+0000","last_unstale":"2024-05-26T22:54:58.918346+0000","last_undegraded":"2024-05-26T22:54:58.918346+0000","last_fullsized":"2024-05-26T22:54:58.918346+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:53:22.885429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"94'8","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.468164+0000","last_change":"2024-05-26T22:54:56.897834+0000","last_active":"2024-05-26T22:54:59.468164+0000","last_peered":"2024-05-26T22:54:59.468164+0000","last_clean":"2024-05-26T22:54:59.468164+0000","last_became_active":"2024-05-26T22:54:56.897593+0000","last_became_peered":"2024-05-26T22:54:56.897593+0000","last_unstale":"2024-05-26T22:54:59.468164+0000","last_undegraded":"2024-05-26T22:54:59.468164+0000","last_fullsized":"2024-05-26T22:54:59.468164+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_clean_scrub_stamp":"2024-05-26T22:54:55.841532+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:15:19.981823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:58.923749+0000","last_change":"2024-05-26T22:54:57.921641+0000","last_active":"2024-05-26T22:54:58.923749+0000","last_peered":"2024-05-26T22:54:58.923749+0000","last_clean":"2024-05-26T22:54:58.923749+0000","last_became_active":"2024-05-26T22:54:57.921376+0000","last_became_peered":"2024-05-26T22:54:57.921376+0000","last_unstale":"2024-05-26T22:54:58.923749+0000","last_undegraded":"2024-05-26T22:54:58.923749+0000","last_fullsized":"2024-05-26T22:54:58.923749+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:21:42.898512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"94'2","reported_seq":16,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.149344+0000","last_change":"2024-05-26T22:54:56.903218+0000","last_active":"2024-05-26T22:54:59.149344+0000","last_peered":"2024-05-26T22:54:59.149344+0000","last_clean":"2024-05-26T22:54:59.149344+0000","last_became_active":"2024-05-26T22:54:56.902855+0000","last_became_peered":"2024-05-26T22:54:56.902855+0000","last_unstale":"2024-05-26T22:54:59.149344+0000","last_undegraded":"2024-05-26T22:54:59.149344+0000","last_fullsized":"2024-05-26T22:54:59.149344+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_clean_scrub_stamp":"2024-05-26T22:54:55.841532+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:44:33.082467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.466947+0000","last_change":"2024-05-26T22:54:57.917046+0000","last_active":"2024-05-26T22:54:59.466947+0000","last_peered":"2024-05-26T22:54:59.466947+0000","last_clean":"2024-05-26T22:54:59.466947+0000","last_became_active":"2024-05-26T22:54:57.916762+0000","last_became_peered":"2024-05-26T22:54:57.916762+0000","last_unstale":"2024-05-26T22:54:59.466947+0000","last_undegraded":"2024-05-26T22:54:59.466947+0000","last_fullsized":"2024-05-26T22:54:59.466947+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:44:34.098004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"94'1","reported_seq":15,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:58.923335+0000","last_change":"2024-05-26T22:54:56.896866+0000","last_active":"2024-05-26T22:54:58.923335+0000","last_peered":"2024-05-26T22:54:58.923335+0000","last_clean":"2024-05-26T22:54:58.923335+0000","last_became_active":"2024-05-26T22:54:56.896575+0000","last_became_peered":"2024-05-26T22:54:56.896575+0000","last_unstale":"2024-05-26T22:54:58.923335+0000","last_undegraded":"2024-05-26T22:54:58.923335+0000","last_fullsized":"2024-05-26T22:54:58.923335+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_clean_scrub_stamp":"2024-05-26T22:54:55.841532+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:44:40.935181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.469278+0000","last_change":"2024-05-26T22:54:57.928029+0000","last_active":"2024-05-26T22:54:59.469278+0000","last_peered":"2024-05-26T22:54:59.469278+0000","last_clean":"2024-05-26T22:54:59.469278+0000","last_became_active":"2024-05-26T22:54:57.926967+0000","last_became_peered":"2024-05-26T22:54:57.926967+0000","last_unstale":"2024-05-26T22:54:59.469278+0000","last_undegraded":"2024-05-26T22:54:59.469278+0000","last_fullsized":"2024-05-26T22:54:59.469278+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:35:20.830276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.468931+0000","last_change":"2024-05-26T22:54:56.898440+0000","last_active":"2024-05-26T22:54:59.468931+0000","last_peered":"2024-05-26T22:54:59.468931+0000","last_clean":"2024-05-26T22:54:59.468931+0000","last_became_active":"2024-05-26T22:54:56.898035+0000","last_became_peered":"2024-05-26T22:54:56.898035+0000","last_unstale":"2024-05-26T22:54:59.468931+0000","last_undegraded":"2024-05-26T22:54:59.468931+0000","last_fullsized":"2024-05-26T22:54:59.468931+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_clean_scrub_stamp":"2024-05-26T22:54:55.841532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T23:32:12.058331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:58.917802+0000","last_change":"2024-05-26T22:54:57.925549+0000","last_active":"2024-05-26T22:54:58.917802+0000","last_peered":"2024-05-26T22:54:58.917802+0000","last_clean":"2024-05-26T22:54:58.917802+0000","last_became_active":"2024-05-26T22:54:57.925243+0000","last_became_peered":"2024-05-26T22:54:57.925243+0000","last_unstale":"2024-05-26T22:54:58.917802+0000","last_undegraded":"2024-05-26T22:54:58.917802+0000","last_fullsized":"2024-05-26T22:54:58.917802+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:05:30.075668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"94'2","reported_seq":16,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.468132+0000","last_change":"2024-05-26T22:54:56.884317+0000","last_active":"2024-05-26T22:54:59.468132+0000","last_peered":"2024-05-26T22:54:59.468132+0000","last_clean":"2024-05-26T22:54:59.468132+0000","last_became_active":"2024-05-26T22:54:56.884047+0000","last_became_peered":"2024-05-26T22:54:56.884047+0000","last_unstale":"2024-05-26T22:54:59.468132+0000","last_undegraded":"2024-05-26T22:54:59.468132+0000","last_fullsized":"2024-05-26T22:54:59.468132+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_clean_scrub_stamp":"2024-05-26T22:54:55.841532+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:14:34.736481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:58.917927+0000","last_change":"2024-05-26T22:54:57.924558+0000","last_active":"2024-05-26T22:54:58.917927+0000","last_peered":"2024-05-26T22:54:58.917927+0000","last_clean":"2024-05-26T22:54:58.917927+0000","last_became_active":"2024-05-26T22:54:57.924058+0000","last_became_peered":"2024-05-26T22:54:57.924058+0000","last_unstale":"2024-05-26T22:54:58.917927+0000","last_undegraded":"2024-05-26T22:54:58.917927+0000","last_fullsized":"2024-05-26T22:54:58.917927+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T23:05:17.072534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.466597+0000","last_change":"2024-05-26T22:54:56.894081+0000","last_active":"2024-05-26T22:54:59.466597+0000","last_peered":"2024-05-26T22:54:59.466597+0000","last_clean":"2024-05-26T22:54:59.466597+0000","last_became_active":"2024-05-26T22:54:56.893780+0000","last_became_peered":"2024-05-26T22:54:56.893780+0000","last_unstale":"2024-05-26T22:54:59.466597+0000","last_undegraded":"2024-05-26T22:54:59.466597+0000","last_fullsized":"2024-05-26T22:54:59.466597+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_clean_scrub_stamp":"2024-05-26T22:54:55.841532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:03:17.215873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:58.918490+0000","last_change":"2024-05-26T22:54:57.926240+0000","last_active":"2024-05-26T22:54:58.918490+0000","last_peered":"2024-05-26T22:54:58.918490+0000","last_clean":"2024-05-26T22:54:58.918490+0000","last_became_active":"2024-05-26T22:54:57.925962+0000","last_became_peered":"2024-05-26T22:54:57.925962+0000","last_unstale":"2024-05-26T22:54:58.918490+0000","last_undegraded":"2024-05-26T22:54:58.918490+0000","last_fullsized":"2024-05-26T22:54:58.918490+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:32:05.059429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.466601+0000","last_change":"2024-05-26T22:54:56.892638+0000","last_active":"2024-05-26T22:54:59.466601+0000","last_peered":"2024-05-26T22:54:59.466601+0000","last_clean":"2024-05-26T22:54:59.466601+0000","last_became_active":"2024-05-26T22:54:56.891218+0000","last_became_peered":"2024-05-26T22:54:56.891218+0000","last_unstale":"2024-05-26T22:54:59.466601+0000","last_undegraded":"2024-05-26T22:54:59.466601+0000","last_fullsized":"2024-05-26T22:54:59.466601+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_clean_scrub_stamp":"2024-05-26T22:54:55.841532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:14:06.230411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:58.922481+0000","last_change":"2024-05-26T22:54:57.908838+0000","last_active":"2024-05-26T22:54:58.922481+0000","last_peered":"2024-05-26T22:54:58.922481+0000","last_clean":"2024-05-26T22:54:58.922481+0000","last_became_active":"2024-05-26T22:54:57.907882+0000","last_became_peered":"2024-05-26T22:54:57.907882+0000","last_unstale":"2024-05-26T22:54:58.922481+0000","last_undegraded":"2024-05-26T22:54:58.922481+0000","last_fullsized":"2024-05-26T22:54:58.922481+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:45:18.352727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.468405+0000","last_change":"2024-05-26T22:54:56.885815+0000","last_active":"2024-05-26T22:54:59.468405+0000","last_peered":"2024-05-26T22:54:59.468405+0000","last_clean":"2024-05-26T22:54:59.468405+0000","last_became_active":"2024-05-26T22:54:56.884912+0000","last_became_peered":"2024-05-26T22:54:56.884912+0000","last_unstale":"2024-05-26T22:54:59.468405+0000","last_undegraded":"2024-05-26T22:54:59.468405+0000","last_fullsized":"2024-05-26T22:54:59.468405+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_clean_scrub_stamp":"2024-05-26T22:54:55.841532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:44:40.935181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.469871+0000","last_change":"2024-05-26T22:54:57.918824+0000","last_active":"2024-05-26T22:54:59.469871+0000","last_peered":"2024-05-26T22:54:59.469871+0000","last_clean":"2024-05-26T22:54:59.469871+0000","last_became_active":"2024-05-26T22:54:57.918509+0000","last_became_peered":"2024-05-26T22:54:57.918509+0000","last_unstale":"2024-05-26T22:54:59.469871+0000","last_undegraded":"2024-05-26T22:54:59.469871+0000","last_fullsized":"2024-05-26T22:54:59.469871+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:14:16.578637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"94'2","reported_seq":16,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.466972+0000","last_change":"2024-05-26T22:54:56.900340+0000","last_active":"2024-05-26T22:54:59.466972+0000","last_peered":"2024-05-26T22:54:59.466972+0000","last_clean":"2024-05-26T22:54:59.466972+0000","last_became_active":"2024-05-26T22:54:56.899759+0000","last_became_peered":"2024-05-26T22:54:56.899759+0000","last_unstale":"2024-05-26T22:54:59.466972+0000","last_undegraded":"2024-05-26T22:54:59.466972+0000","last_fullsized":"2024-05-26T22:54:59.466972+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_clean_scrub_stamp":"2024-05-26T22:54:55.841532+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:27:40.126836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.469635+0000","last_change":"2024-05-26T22:54:57.920980+0000","last_active":"2024-05-26T22:54:59.469635+0000","last_peered":"2024-05-26T22:54:59.469635+0000","last_clean":"2024-05-26T22:54:59.469635+0000","last_became_active":"2024-05-26T22:54:57.920631+0000","last_became_peered":"2024-05-26T22:54:57.920631+0000","last_unstale":"2024-05-26T22:54:59.469635+0000","last_undegraded":"2024-05-26T22:54:59.469635+0000","last_fullsized":"2024-05-26T22:54:59.469635+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T23:24:00.956802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"95'1","reported_seq":15,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.917742+0000","last_change":"2024-05-26T22:54:56.887664+0000","last_active":"2024-05-26T22:54:59.917742+0000","last_peered":"2024-05-26T22:54:59.917742+0000","last_clean":"2024-05-26T22:54:59.917742+0000","last_became_active":"2024-05-26T22:54:56.887399+0000","last_became_peered":"2024-05-26T22:54:56.887399+0000","last_unstale":"2024-05-26T22:54:59.917742+0000","last_undegraded":"2024-05-26T22:54:59.917742+0000","last_fullsized":"2024-05-26T22:54:59.917742+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_clean_scrub_stamp":"2024-05-26T22:54:55.841532+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:52:36.198643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.468279+0000","last_change":"2024-05-26T22:54:57.924268+0000","last_active":"2024-05-26T22:54:59.468279+0000","last_peered":"2024-05-26T22:54:59.468279+0000","last_clean":"2024-05-26T22:54:59.468279+0000","last_became_active":"2024-05-26T22:54:57.920466+0000","last_became_peered":"2024-05-26T22:54:57.920466+0000","last_unstale":"2024-05-26T22:54:59.468279+0000","last_undegraded":"2024-05-26T22:54:59.468279+0000","last_fullsized":"2024-05-26T22:54:59.468279+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:58:37.052094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.469259+0000","last_change":"2024-05-26T22:54:56.898732+0000","last_active":"2024-05-26T22:54:59.469259+0000","last_peered":"2024-05-26T22:54:59.469259+0000","last_clean":"2024-05-26T22:54:59.469259+0000","last_became_active":"2024-05-26T22:54:56.898091+0000","last_became_peered":"2024-05-26T22:54:56.898091+0000","last_unstale":"2024-05-26T22:54:59.469259+0000","last_undegraded":"2024-05-26T22:54:59.469259+0000","last_fullsized":"2024-05-26T22:54:59.469259+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_clean_scrub_stamp":"2024-05-26T22:54:55.841532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:18:00.117314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:58.923137+0000","last_change":"2024-05-26T22:54:57.927226+0000","last_active":"2024-05-26T22:54:58.923137+0000","last_peered":"2024-05-26T22:54:58.923137+0000","last_clean":"2024-05-26T22:54:58.923137+0000","last_became_active":"2024-05-26T22:54:57.925136+0000","last_became_peered":"2024-05-26T22:54:57.925136+0000","last_unstale":"2024-05-26T22:54:58.923137+0000","last_undegraded":"2024-05-26T22:54:58.923137+0000","last_fullsized":"2024-05-26T22:54:58.923137+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:35:20.830276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:58.923135+0000","last_change":"2024-05-26T22:54:56.901260+0000","last_active":"2024-05-26T22:54:58.923135+0000","last_peered":"2024-05-26T22:54:58.923135+0000","last_clean":"2024-05-26T22:54:58.923135+0000","last_became_active":"2024-05-26T22:54:56.900507+0000","last_became_peered":"2024-05-26T22:54:56.900507+0000","last_unstale":"2024-05-26T22:54:58.923135+0000","last_undegraded":"2024-05-26T22:54:58.923135+0000","last_fullsized":"2024-05-26T22:54:58.923135+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_clean_scrub_stamp":"2024-05-26T22:54:55.841532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:32:01.500293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:58.917948+0000","last_change":"2024-05-26T22:54:57.929952+0000","last_active":"2024-05-26T22:54:58.917948+0000","last_peered":"2024-05-26T22:54:58.917948+0000","last_clean":"2024-05-26T22:54:58.917948+0000","last_became_active":"2024-05-26T22:54:57.928945+0000","last_became_peered":"2024-05-26T22:54:57.928945+0000","last_unstale":"2024-05-26T22:54:58.917948+0000","last_undegraded":"2024-05-26T22:54:58.917948+0000","last_fullsized":"2024-05-26T22:54:58.917948+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:04:52.070354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.468397+0000","last_change":"2024-05-26T22:54:56.890912+0000","last_active":"2024-05-26T22:54:59.468397+0000","last_peered":"2024-05-26T22:54:59.468397+0000","last_clean":"2024-05-26T22:54:59.468397+0000","last_became_active":"2024-05-26T22:54:56.890586+0000","last_became_peered":"2024-05-26T22:54:56.890586+0000","last_unstale":"2024-05-26T22:54:59.468397+0000","last_undegraded":"2024-05-26T22:54:59.468397+0000","last_fullsized":"2024-05-26T22:54:59.468397+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_clean_scrub_stamp":"2024-05-26T22:54:55.841532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:52:51.519449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.148906+0000","last_change":"2024-05-26T22:54:57.919463+0000","last_active":"2024-05-26T22:54:59.148906+0000","last_peered":"2024-05-26T22:54:59.148906+0000","last_clean":"2024-05-26T22:54:59.148906+0000","last_became_active":"2024-05-26T22:54:57.918851+0000","last_became_peered":"2024-05-26T22:54:57.918851+0000","last_unstale":"2024-05-26T22:54:59.148906+0000","last_undegraded":"2024-05-26T22:54:59.148906+0000","last_fullsized":"2024-05-26T22:54:59.148906+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T23:32:13.073868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:58.922011+0000","last_change":"2024-05-26T22:54:56.889430+0000","last_active":"2024-05-26T22:54:58.922011+0000","last_peered":"2024-05-26T22:54:58.922011+0000","last_clean":"2024-05-26T22:54:58.922011+0000","last_became_active":"2024-05-26T22:54:56.887174+0000","last_became_peered":"2024-05-26T22:54:56.887174+0000","last_unstale":"2024-05-26T22:54:58.922011+0000","last_undegraded":"2024-05-26T22:54:58.922011+0000","last_fullsized":"2024-05-26T22:54:58.922011+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_clean_scrub_stamp":"2024-05-26T22:54:55.841532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:33:55.154572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:58.918635+0000","last_change":"2024-05-26T22:54:57.914623+0000","last_active":"2024-05-26T22:54:58.918635+0000","last_peered":"2024-05-26T22:54:58.918635+0000","last_clean":"2024-05-26T22:54:58.918635+0000","last_became_active":"2024-05-26T22:54:57.914272+0000","last_became_peered":"2024-05-26T22:54:57.914272+0000","last_unstale":"2024-05-26T22:54:58.918635+0000","last_undegraded":"2024-05-26T22:54:58.918635+0000","last_fullsized":"2024-05-26T22:54:58.918635+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:42:48.817759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:58.917468+0000","last_change":"2024-05-26T22:54:56.895568+0000","last_active":"2024-05-26T22:54:58.917468+0000","last_peered":"2024-05-26T22:54:58.917468+0000","last_clean":"2024-05-26T22:54:58.917468+0000","last_became_active":"2024-05-26T22:54:56.894157+0000","last_became_peered":"2024-05-26T22:54:56.894157+0000","last_unstale":"2024-05-26T22:54:58.917468+0000","last_undegraded":"2024-05-26T22:54:58.917468+0000","last_fullsized":"2024-05-26T22:54:58.917468+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_clean_scrub_stamp":"2024-05-26T22:54:55.841532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:33:55.154572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:58.922474+0000","last_change":"2024-05-26T22:54:57.923510+0000","last_active":"2024-05-26T22:54:58.922474+0000","last_peered":"2024-05-26T22:54:58.922474+0000","last_clean":"2024-05-26T22:54:58.922474+0000","last_became_active":"2024-05-26T22:54:57.923173+0000","last_became_peered":"2024-05-26T22:54:57.923173+0000","last_unstale":"2024-05-26T22:54:58.922474+0000","last_undegraded":"2024-05-26T22:54:58.922474+0000","last_fullsized":"2024-05-26T22:54:58.922474+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:46:04.317115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:58.918728+0000","last_change":"2024-05-26T22:54:56.894405+0000","last_active":"2024-05-26T22:54:58.918728+0000","last_peered":"2024-05-26T22:54:58.918728+0000","last_clean":"2024-05-26T22:54:58.918728+0000","last_became_active":"2024-05-26T22:54:56.894161+0000","last_became_peered":"2024-05-26T22:54:56.894161+0000","last_unstale":"2024-05-26T22:54:58.918728+0000","last_undegraded":"2024-05-26T22:54:58.918728+0000","last_fullsized":"2024-05-26T22:54:58.918728+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_clean_scrub_stamp":"2024-05-26T22:54:55.841532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:35:59.442921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:58.918743+0000","last_change":"2024-05-26T22:54:57.930127+0000","last_active":"2024-05-26T22:54:58.918743+0000","last_peered":"2024-05-26T22:54:58.918743+0000","last_clean":"2024-05-26T22:54:58.918743+0000","last_became_active":"2024-05-26T22:54:57.929268+0000","last_became_peered":"2024-05-26T22:54:57.929268+0000","last_unstale":"2024-05-26T22:54:58.918743+0000","last_undegraded":"2024-05-26T22:54:58.918743+0000","last_fullsized":"2024-05-26T22:54:58.918743+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:56.857068+0000","last_clean_scrub_stamp":"2024-05-26T22:54:56.857068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:02:54.926056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"94'1","reported_seq":15,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.469425+0000","last_change":"2024-05-26T22:54:56.905163+0000","last_active":"2024-05-26T22:54:59.469425+0000","last_peered":"2024-05-26T22:54:59.469425+0000","last_clean":"2024-05-26T22:54:59.469425+0000","last_became_active":"2024-05-26T22:54:56.904858+0000","last_became_peered":"2024-05-26T22:54:56.904858+0000","last_unstale":"2024-05-26T22:54:59.469425+0000","last_undegraded":"2024-05-26T22:54:59.469425+0000","last_fullsized":"2024-05-26T22:54:59.469425+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:54:55.841532+0000","last_clean_scrub_stamp":"2024-05-26T22:54:55.841532+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:28:16.747428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":309,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-05-26T22:54:59.468417+0000","last_change":"2024-05-26T22:48:12.068917+0000","last_active":"2024-05-26T22:54:59.468417+0000","last_peered":"2024-05-26T22:54:59.468417+0000","last_clean":"2024-05-26T22:54:59.468417+0000","last_became_active":"2024-05-26T22:48:12.065934+0000","last_became_peered":"2024-05-26T22:48:12.065934+0000","last_unstale":"2024-05-26T22:54:59.468417+0000","last_undegraded":"2024-05-26T22:54:59.468417+0000","last_fullsized":"2024-05-26T22:54:59.468417+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:10.628122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:10.628122+0000","last_clean_scrub_stamp":"2024-05-26T22:48:10.628122+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:14:58.895911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574933,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32916,"kb_used_data":1736,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338924,"statfs":{"total":96636764160,"available":96603058176,"internally_reserved":0,"allocated":1777664,"data_stored":1101043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574933,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32344,"kb_used_data":1156,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339496,"statfs":{"total":96636764160,"available":96603643904,"internally_reserved":0,"allocated":1183744,"data_stored":511215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574934,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32340,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339500,"statfs":{"total":96636764160,"available":96603648000,"internally_reserved":0,"allocated":1179648,"data_stored":510633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574933,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32356,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339484,"statfs":{"total":96636764160,"available":96603631616,"internally_reserved":0,"allocated":1196032,"data_stored":512299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574933,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32344,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339496,"statfs":{"total":96636764160,"available":96603643904,"internally_reserved":0,"allocated":1191936,"data_stored":512197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574933,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32340,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339500,"statfs":{"total":96636764160,"available":96603648000,"internally_reserved":0,"allocated":1179648,"data_stored":510597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574933,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32336,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339504,"statfs":{"total":96636764160,"available":96603652096,"internally_reserved":0,"allocated":1175552,"data_stored":510563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574933,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32912,"kb_used_data":1732,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338928,"statfs":{"total":96636764160,"available":96603062272,"internally_reserved":0,"allocated":1773568,"data_stored":1101513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-26T22:55:02.811 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-26T22:55:02.812 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-26T22:55:02.832 INFO:tasks.ceph.mds.a.smithi131.stderr:2024-05-26T22:55:02.833+0000 7f2ce4000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-26T22:55:03.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.050+0000 7f311ac00640 1 Processor -- start 2024-05-26T22:55:03.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.050+0000 7f311ac00640 1 -- start start 2024-05-26T22:55:03.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.052+0000 7f311ac00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f31140b9f00 0x7f31140b5c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:03.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.052+0000 7f311ac00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f31140d3c40 0x7f31140b6190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:03.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.052+0000 7f311ac00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31140d2e80 con 0x7f311415e3e0 2024-05-26T22:55:03.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.052+0000 7f311ac00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f31140cf910 con 0x7f31140b9f00 2024-05-26T22:55:03.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.052+0000 7f311ac00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f31140d2c30 con 0x7f31140d3c40 2024-05-26T22:55:03.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.052+0000 7f3119800640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f311415e3e0 0x7f31140cf010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:54574/0 (socket says 172.21.15.131:54574) 2024-05-26T22:55:03.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.052+0000 7f3119800640 1 -- 172.21.15.131:0/2847777292 learned_addr learned my addr 172.21.15.131:0/2847777292 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:55:03.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.053+0000 7f311a200640 1 --2- 172.21.15.131:0/2847777292 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f31140d3c40 0x7f31140b6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:03.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.053+0000 7f3118e00640 1 --2- 172.21.15.131:0/2847777292 >> v2:172.21.15.187:3300/0 conn(0x7f31140b9f00 0x7f31140b5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:03.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.053+0000 7f3113e00640 1 -- 172.21.15.131:0/2847777292 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 746568472 0 0) 0x7f31140d2e80 con 0x7f311415e3e0 2024-05-26T22:55:03.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.053+0000 7f3113e00640 1 -- 172.21.15.131:0/2847777292 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f30fc003520 con 0x7f311415e3e0 2024-05-26T22:55:03.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.053+0000 7f3113e00640 1 -- 172.21.15.131:0/2847777292 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2280175476 0 0) 0x7f30fc003520 con 0x7f311415e3e0 2024-05-26T22:55:03.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.053+0000 7f3113e00640 1 -- 172.21.15.131:0/2847777292 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f31140d3c40 msgr2=0x7f31140b6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:03.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.053+0000 7f3113e00640 1 --2- 172.21.15.131:0/2847777292 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f31140d3c40 0x7f31140b6190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:03.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.053+0000 7f3113e00640 1 -- 172.21.15.131:0/2847777292 >> v2:172.21.15.187:3300/0 conn(0x7f31140b9f00 msgr2=0x7f31140b5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:03.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.053+0000 7f3113e00640 1 --2- 172.21.15.131:0/2847777292 >> v2:172.21.15.187:3300/0 conn(0x7f31140b9f00 0x7f31140b5c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:03.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.053+0000 7f3113e00640 1 -- 172.21.15.131:0/2847777292 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31140d3160 con 0x7f311415e3e0 2024-05-26T22:55:03.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.053+0000 7f3113e00640 1 -- 172.21.15.131:0/2847777292 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7f3108001690 con 0x7f311415e3e0 2024-05-26T22:55:03.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.053+0000 7f3113e00640 1 -- 172.21.15.131:0/2847777292 >> v1:172.21.15.131:6789/0 conn(0x7f311415e3e0 legacy=0x7f31140cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:03.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.053+0000 7f3113e00640 1 --2- 172.21.15.131:0/2847777292 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f30fc0039b0 0x7f30fc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:03.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.053+0000 7f3113e00640 1 --2- 172.21.15.131:0/2847777292 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f31140d3c40 0x7f30fc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:03.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.053+0000 7f3113e00640 1 --2- 172.21.15.131:0/2847777292 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f30fc004d40 0x7f30fc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:03.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.053+0000 7f3113e00640 1 -- 172.21.15.131:0/2847777292 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f31140d2c30 con 0x7f30fc004d40 2024-05-26T22:55:03.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.053+0000 7f3113e00640 1 -- 172.21.15.131:0/2847777292 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f30fc00b9c0 con 0x7f30fc0039b0 2024-05-26T22:55:03.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.053+0000 7f3113e00640 1 -- 172.21.15.131:0/2847777292 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f30fc00bb40 con 0x7f31140d3c40 2024-05-26T22:55:03.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.054+0000 7f311a200640 1 --2- 172.21.15.131:0/2847777292 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f30fc004d40 0x7f30fc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:03.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.054+0000 7f3118e00640 1 --2- 172.21.15.131:0/2847777292 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f31140d3c40 0x7f30fc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:03.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.054+0000 7f3119800640 1 --2- 172.21.15.131:0/2847777292 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f30fc0039b0 0x7f30fc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:03.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.054+0000 7f311a200640 1 -- 172.21.15.131:0/2847777292 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f31140d3c40 msgr2=0x7f30fc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:03.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.054+0000 7f311a200640 1 --2- 172.21.15.131:0/2847777292 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f31140d3c40 0x7f30fc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:03.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.054+0000 7f311a200640 1 -- 172.21.15.131:0/2847777292 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f30fc0039b0 msgr2=0x7f30fc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:03.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.054+0000 7f311a200640 1 --2- 172.21.15.131:0/2847777292 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f30fc0039b0 0x7f30fc003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:03.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.054+0000 7f311a200640 1 -- 172.21.15.131:0/2847777292 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30fc003520 con 0x7f30fc004d40 2024-05-26T22:55:03.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.055+0000 7f311a200640 1 --2- 172.21.15.131:0/2847777292 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f30fc004d40 0x7f30fc0051c0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f310c00e170 tx=0x7f30fc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=840d37b8ef7ab9 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:03.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.056+0000 7f3113e00640 1 -- 172.21.15.131:0/2847777292 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f310c034020 con 0x7f30fc004d40 2024-05-26T22:55:03.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.056+0000 7f3113e00640 1 -- 172.21.15.131:0/2847777292 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f310c0305f0 con 0x7f30fc004d40 2024-05-26T22:55:03.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.056+0000 7f3113e00640 1 -- 172.21.15.131:0/2847777292 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f310c009c10 con 0x7f30fc004d40 2024-05-26T22:55:03.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.056+0000 7f311ac00640 1 -- 172.21.15.131:0/2847777292 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f30fc004d40 msgr2=0x7f30fc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:03.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.056+0000 7f311ac00640 1 --2- 172.21.15.131:0/2847777292 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f30fc004d40 0x7f30fc0051c0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f310c00e170 tx=0x7f30fc005830 comp rx=0 tx=0).stop 2024-05-26T22:55:03.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.056+0000 7f311a200640 1 -- 172.21.15.131:0/2847777292 reap_dead start 2024-05-26T22:55:03.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.056+0000 7f311ac00640 1 -- 172.21.15.131:0/2847777292 shutdown_connections 2024-05-26T22:55:03.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.056+0000 7f311ac00640 1 -- 172.21.15.131:0/2847777292 >> 172.21.15.131:0/2847777292 conn(0x7f3114073070 msgr2=0x7f31140d0c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:03.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.057+0000 7f311ac00640 1 -- 172.21.15.131:0/2847777292 shutdown_connections 2024-05-26T22:55:03.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.057+0000 7f311ac00640 1 -- 172.21.15.131:0/2847777292 wait complete. 2024-05-26T22:55:03.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.058+0000 7f311ac00640 1 Processor -- start 2024-05-26T22:55:03.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.058+0000 7f311ac00640 1 -- start start 2024-05-26T22:55:03.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.059+0000 7f311ac00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f30fc004d40 0x7f31140ccf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:03.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.059+0000 7f311ac00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f311414d1b0 0x7f31140b3b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:03.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.059+0000 7f311ac00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f31140b4050 0x7f311414f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:03.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.059+0000 7f311ac00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f311415e7c0 con 0x7f311414d1b0 2024-05-26T22:55:03.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.059+0000 7f311ac00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f31140d3720 con 0x7f31140b4050 2024-05-26T22:55:03.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.059+0000 7f311ac00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f31140b6190 con 0x7f30fc004d40 2024-05-26T22:55:03.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.060+0000 7f3118e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f311414d1b0 0x7f31140b3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:03.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.060+0000 7f3118e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f311414d1b0 0x7f31140b3b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41126/0 (socket says 172.21.15.131:41126) 2024-05-26T22:55:03.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.060+0000 7f3118e00640 1 -- 172.21.15.131:0/2612449602 learned_addr learned my addr 172.21.15.131:0/2612449602 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:55:03.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.060+0000 7f3119800640 1 --2- 172.21.15.131:0/2612449602 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f30fc004d40 0x7f31140ccf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:03.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.060+0000 7f311a200640 1 --2- 172.21.15.131:0/2612449602 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f31140b4050 0x7f311414f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:03.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.060+0000 7f3118e00640 1 -- 172.21.15.131:0/2612449602 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f30fc004d40 msgr2=0x7f31140ccf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:03.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.060+0000 7f3118e00640 1 --2- 172.21.15.131:0/2612449602 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f30fc004d40 0x7f31140ccf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:03.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.060+0000 7f3118e00640 1 -- 172.21.15.131:0/2612449602 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f31140b4050 msgr2=0x7f311414f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:03.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.060+0000 7f3118e00640 1 --2- 172.21.15.131:0/2612449602 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f31140b4050 0x7f311414f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:03.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.060+0000 7f3118e00640 1 -- 172.21.15.131:0/2612449602 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3114150570 con 0x7f311414d1b0 2024-05-26T22:55:03.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.060+0000 7f311a200640 1 --2- 172.21.15.131:0/2612449602 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f31140b4050 0x7f311414f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:55:03.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.060+0000 7f3118e00640 1 --2- 172.21.15.131:0/2612449602 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f311414d1b0 0x7f31140b3b10 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f31000116d0 tx=0x7f3100011b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:03.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.061+0000 7f3111600640 1 -- 172.21.15.131:0/2612449602 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f310000f9c0 con 0x7f311414d1b0 2024-05-26T22:55:03.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.061+0000 7f3111600640 1 -- 172.21.15.131:0/2612449602 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f310000fb60 con 0x7f311414d1b0 2024-05-26T22:55:03.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.061+0000 7f311ac00640 1 -- 172.21.15.131:0/2612449602 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31140cc490 con 0x7f311414d1b0 2024-05-26T22:55:03.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.061+0000 7f3111600640 1 -- 172.21.15.131:0/2612449602 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3100015460 con 0x7f311414d1b0 2024-05-26T22:55:03.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.061+0000 7f311ac00640 1 -- 172.21.15.131:0/2612449602 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31140cca50 con 0x7f311414d1b0 2024-05-26T22:55:03.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.062+0000 7f311ac00640 1 -- 172.21.15.131:0/2612449602 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31140694e0 con 0x7f311414d1b0 2024-05-26T22:55:03.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.063+0000 7f3111600640 1 -- 172.21.15.131:0/2612449602 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3100015600 con 0x7f311414d1b0 2024-05-26T22:55:03.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.064+0000 7f3111600640 1 --2- 172.21.15.131:0/2612449602 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f30f406be30 0x7f30f406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:03.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.064+0000 7f3111600640 1 -- 172.21.15.131:0/2612449602 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 5709+0+0 (secure 0 0 0) 0x7f310008b8e0 con 0x7f311414d1b0 2024-05-26T22:55:03.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.064+0000 7f3119800640 1 --2- 172.21.15.131:0/2612449602 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f30f406be30 0x7f30f406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:03.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.065+0000 7f3119800640 1 --2- 172.21.15.131:0/2612449602 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f30f406be30 0x7f30f406e2f0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f3108000c00 tx=0x7f3108009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:03.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.068+0000 7f3111600640 1 -- 172.21.15.131:0/2612449602 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f3100019030 con 0x7f311414d1b0 2024-05-26T22:55:03.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.342+0000 7f311ac00640 1 -- 172.21.15.131:0/2612449602 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f31140b4430 con 0x7f311414d1b0 2024-05-26T22:55:03.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.346+0000 7f3111600640 1 -- 172.21.15.131:0/2612449602 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v18) ==== 190+0+147 (secure 0 0 0) 0x7f310005a280 con 0x7f311414d1b0 2024-05-26T22:55:03.346 INFO:teuthology.orchestra.run.smithi131.stdout:[client.0] 2024-05-26T22:55:03.346 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQD2u1NmSGVWChAAH/wq3iaUs6ZELRhLKhmURQ== 2024-05-26T22:55:03.346 INFO:teuthology.orchestra.run.smithi131.stdout: caps mds = "allow" 2024-05-26T22:55:03.346 INFO:teuthology.orchestra.run.smithi131.stdout: caps mon = "allow r" 2024-05-26T22:55:03.346 INFO:teuthology.orchestra.run.smithi131.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-05-26T22:55:03.346 INFO:teuthology.orchestra.run.smithi131.stderr:updated caps for client.0 2024-05-26T22:55:03.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.349+0000 7f311ac00640 1 -- 172.21.15.131:0/2612449602 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f30f406be30 msgr2=0x7f30f406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:03.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.349+0000 7f311ac00640 1 --2- 172.21.15.131:0/2612449602 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f30f406be30 0x7f30f406e2f0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f3108000c00 tx=0x7f3108009040 comp rx=0 tx=0).stop 2024-05-26T22:55:03.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.349+0000 7f311ac00640 1 -- 172.21.15.131:0/2612449602 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f311414d1b0 msgr2=0x7f31140b3b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:03.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.349+0000 7f311ac00640 1 --2- 172.21.15.131:0/2612449602 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f311414d1b0 0x7f31140b3b10 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f31000116d0 tx=0x7f3100011b90 comp rx=0 tx=0).stop 2024-05-26T22:55:03.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.349+0000 7f311ac00640 1 -- 172.21.15.131:0/2612449602 shutdown_connections 2024-05-26T22:55:03.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.349+0000 7f311ac00640 1 --2- 172.21.15.131:0/2612449602 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f30f406be30 0x7f30f406e2f0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:03.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.349+0000 7f311ac00640 1 --2- 172.21.15.131:0/2612449602 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f31140b4050 0x7f311414f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:03.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.349+0000 7f311ac00640 1 --2- 172.21.15.131:0/2612449602 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f311414d1b0 0x7f31140b3b10 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:03.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.349+0000 7f311ac00640 1 --2- 172.21.15.131:0/2612449602 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f30fc004d40 0x7f31140ccf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:03.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.349+0000 7f311ac00640 1 -- 172.21.15.131:0/2612449602 >> 172.21.15.131:0/2612449602 conn(0x7f3114073070 msgr2=0x7f31140c75c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:03.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.349+0000 7f311ac00640 1 -- 172.21.15.131:0/2612449602 shutdown_connections 2024-05-26T22:55:03.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.349+0000 7f311ac00640 1 -- 172.21.15.131:0/2612449602 wait complete. 2024-05-26T22:55:03.368 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-26T22:55:03.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.565+0000 7f843d200640 1 Processor -- start 2024-05-26T22:55:03.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.565+0000 7f843d200640 1 -- start start 2024-05-26T22:55:03.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.566+0000 7f843d200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f843806e8c0 0x7f843815d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:03.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.566+0000 7f843d200640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f84380cc7f0 0x7f84380ccc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:03.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.566+0000 7f843d200640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8438160620 con 0x7f8438156750 2024-05-26T22:55:03.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.566+0000 7f843d200640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f84380cd1a0 con 0x7f84380cc7f0 2024-05-26T22:55:03.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.566+0000 7f843d200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f843815cbd0 con 0x7f843806e8c0 2024-05-26T22:55:03.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.567+0000 7f8437400640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f8438156750 0x7f843815fd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:54588/0 (socket says 172.21.15.131:54588) 2024-05-26T22:55:03.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.567+0000 7f8437400640 1 -- 172.21.15.131:0/1733512838 learned_addr learned my addr 172.21.15.131:0/1733512838 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:55:03.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.567+0000 7f8437e00640 1 --2- 172.21.15.131:0/1733512838 >> v2:172.21.15.187:3300/0 conn(0x7f84380cc7f0 0x7f84380ccc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:03.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.567+0000 7f8436000640 1 -- 172.21.15.131:0/1733512838 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2950373226 0 0) 0x7f8438160620 con 0x7f8438156750 2024-05-26T22:55:03.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.567+0000 7f8436000640 1 -- 172.21.15.131:0/1733512838 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f841c003520 con 0x7f8438156750 2024-05-26T22:55:03.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.567+0000 7f8436a00640 1 --2- 172.21.15.131:0/1733512838 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f843806e8c0 0x7f843815d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:03.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.568+0000 7f8436000640 1 -- 172.21.15.131:0/1733512838 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3909522063 0 0) 0x7f841c003520 con 0x7f8438156750 2024-05-26T22:55:03.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.568+0000 7f8436000640 1 -- 172.21.15.131:0/1733512838 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f843806e8c0 msgr2=0x7f843815d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:03.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.568+0000 7f8436000640 1 --2- 172.21.15.131:0/1733512838 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f843806e8c0 0x7f843815d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:03.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.568+0000 7f8436000640 1 -- 172.21.15.131:0/1733512838 >> v2:172.21.15.187:3300/0 conn(0x7f84380cc7f0 msgr2=0x7f84380ccc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:03.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.568+0000 7f8436000640 1 --2- 172.21.15.131:0/1733512838 >> v2:172.21.15.187:3300/0 conn(0x7f84380cc7f0 0x7f84380ccc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:03.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.568+0000 7f8436000640 1 -- 172.21.15.131:0/1733512838 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8438160900 con 0x7f8438156750 2024-05-26T22:55:03.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.568+0000 7f8436000640 1 -- 172.21.15.131:0/1733512838 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7f8428001670 con 0x7f8438156750 2024-05-26T22:55:03.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.568+0000 7f8436000640 1 -- 172.21.15.131:0/1733512838 >> v1:172.21.15.131:6789/0 conn(0x7f8438156750 legacy=0x7f843815fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:03.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.568+0000 7f8436000640 1 --2- 172.21.15.131:0/1733512838 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f841c0039b0 0x7f841c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:03.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.568+0000 7f8436000640 1 --2- 172.21.15.131:0/1733512838 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f843806e8c0 0x7f841c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:03.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.568+0000 7f8436000640 1 --2- 172.21.15.131:0/1733512838 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f841c004d40 0x7f841c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:03.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.568+0000 7f8436000640 1 -- 172.21.15.131:0/1733512838 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f84380cd1a0 con 0x7f841c004d40 2024-05-26T22:55:03.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.568+0000 7f8436000640 1 -- 172.21.15.131:0/1733512838 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f843815cbd0 con 0x7f841c0039b0 2024-05-26T22:55:03.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.568+0000 7f8436000640 1 -- 172.21.15.131:0/1733512838 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f841c00b9c0 con 0x7f843806e8c0 2024-05-26T22:55:03.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.568+0000 7f8437e00640 1 --2- 172.21.15.131:0/1733512838 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f841c004d40 0x7f841c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:03.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.568+0000 7f8437400640 1 --2- 172.21.15.131:0/1733512838 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f841c0039b0 0x7f841c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:03.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.568+0000 7f8436a00640 1 --2- 172.21.15.131:0/1733512838 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f843806e8c0 0x7f841c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:03.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.569+0000 7f8437e00640 1 -- 172.21.15.131:0/1733512838 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f843806e8c0 msgr2=0x7f841c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:03.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.569+0000 7f8437e00640 1 --2- 172.21.15.131:0/1733512838 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f843806e8c0 0x7f841c004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:03.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.569+0000 7f8437e00640 1 -- 172.21.15.131:0/1733512838 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f841c0039b0 msgr2=0x7f841c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:03.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.569+0000 7f8437e00640 1 --2- 172.21.15.131:0/1733512838 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f841c0039b0 0x7f841c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:03.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.569+0000 7f8437e00640 1 -- 172.21.15.131:0/1733512838 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f841c003520 con 0x7f841c004d40 2024-05-26T22:55:03.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.570+0000 7f8437e00640 1 --2- 172.21.15.131:0/1733512838 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f841c004d40 0x7f841c0051c0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f843000e170 tx=0x7f841c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dec6605968a91b5f server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:03.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.570+0000 7f8436000640 1 -- 172.21.15.131:0/1733512838 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8430034020 con 0x7f841c004d40 2024-05-26T22:55:03.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.570+0000 7f8436000640 1 -- 172.21.15.131:0/1733512838 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8430030430 con 0x7f841c004d40 2024-05-26T22:55:03.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.570+0000 7f8436000640 1 -- 172.21.15.131:0/1733512838 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8430009c10 con 0x7f841c004d40 2024-05-26T22:55:03.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.570+0000 7f843d200640 1 -- 172.21.15.131:0/1733512838 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f841c004d40 msgr2=0x7f841c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:03.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.570+0000 7f843d200640 1 --2- 172.21.15.131:0/1733512838 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f841c004d40 0x7f841c0051c0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f843000e170 tx=0x7f841c005830 comp rx=0 tx=0).stop 2024-05-26T22:55:03.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.570+0000 7f8437e00640 1 -- 172.21.15.131:0/1733512838 reap_dead start 2024-05-26T22:55:03.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.571+0000 7f843d200640 1 -- 172.21.15.131:0/1733512838 shutdown_connections 2024-05-26T22:55:03.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.571+0000 7f843d200640 1 -- 172.21.15.131:0/1733512838 >> 172.21.15.131:0/1733512838 conn(0x7f84380cff40 msgr2=0x7f84380d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:03.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.571+0000 7f843d200640 1 -- 172.21.15.131:0/1733512838 shutdown_connections 2024-05-26T22:55:03.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.571+0000 7f843d200640 1 -- 172.21.15.131:0/1733512838 wait complete. 2024-05-26T22:55:03.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.572+0000 7f843d200640 1 Processor -- start 2024-05-26T22:55:03.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.572+0000 7f843d200640 1 -- start start 2024-05-26T22:55:03.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.573+0000 7f843d200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f841c004d40 0x7f84380b2480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:03.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.573+0000 7f843d200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f843815d3f0 0x7f8438139900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:03.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.573+0000 7f843d200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f843815ea00 0x7f8438158890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:03.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.573+0000 7f843d200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8438156b30 con 0x7f841c004d40 2024-05-26T22:55:03.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.573+0000 7f843d200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f843815c860 con 0x7f843815ea00 2024-05-26T22:55:03.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.573+0000 7f843d200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f843815d060 con 0x7f843815d3f0 2024-05-26T22:55:03.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.574+0000 7f8437400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f841c004d40 0x7f84380b2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:03.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.574+0000 7f8437400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f841c004d40 0x7f84380b2480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41146/0 (socket says 172.21.15.131:41146) 2024-05-26T22:55:03.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.574+0000 7f8437400640 1 -- 172.21.15.131:0/4193143312 learned_addr learned my addr 172.21.15.131:0/4193143312 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:55:03.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.574+0000 7f8436a00640 1 --2- 172.21.15.131:0/4193143312 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f843815d3f0 0x7f8438139900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:03.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.574+0000 7f8437e00640 1 --2- 172.21.15.131:0/4193143312 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f843815ea00 0x7f8438158890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:03.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.574+0000 7f8437400640 1 -- 172.21.15.131:0/4193143312 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f843815d3f0 msgr2=0x7f8438139900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:03.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.574+0000 7f8437400640 1 --2- 172.21.15.131:0/4193143312 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f843815d3f0 0x7f8438139900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:03.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.574+0000 7f8437400640 1 -- 172.21.15.131:0/4193143312 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f843815ea00 msgr2=0x7f8438158890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:03.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.574+0000 7f8437400640 1 --2- 172.21.15.131:0/4193143312 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f843815ea00 0x7f8438158890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:03.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.574+0000 7f8437400640 1 -- 172.21.15.131:0/4193143312 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f843813a450 con 0x7f841c004d40 2024-05-26T22:55:03.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.574+0000 7f8437400640 1 --2- 172.21.15.131:0/4193143312 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f841c004d40 0x7f84380b2480 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f842800e870 tx=0x7f842800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:03.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.575+0000 7f842f400640 1 -- 172.21.15.131:0/4193143312 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f842800c7f0 con 0x7f841c004d40 2024-05-26T22:55:03.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.575+0000 7f842f400640 1 -- 172.21.15.131:0/4193143312 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8428018070 con 0x7f841c004d40 2024-05-26T22:55:03.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.575+0000 7f842f400640 1 -- 172.21.15.131:0/4193143312 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f842800cbd0 con 0x7f841c004d40 2024-05-26T22:55:03.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.575+0000 7f843d200640 1 -- 172.21.15.131:0/4193143312 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f843815d960 con 0x7f841c004d40 2024-05-26T22:55:03.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.575+0000 7f843d200640 1 -- 172.21.15.131:0/4193143312 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f843815df20 con 0x7f841c004d40 2024-05-26T22:55:03.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.576+0000 7f843d200640 1 -- 172.21.15.131:0/4193143312 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84380694e0 con 0x7f841c004d40 2024-05-26T22:55:03.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.577+0000 7f842f400640 1 -- 172.21.15.131:0/4193143312 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f842801d460 con 0x7f841c004d40 2024-05-26T22:55:03.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.578+0000 7f842f400640 1 --2- 172.21.15.131:0/4193143312 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f841806be30 0x7f841806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:03.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.578+0000 7f842f400640 1 -- 172.21.15.131:0/4193143312 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 5709+0+0 (secure 0 0 0) 0x7f842808ad90 con 0x7f841c004d40 2024-05-26T22:55:03.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.578+0000 7f8436a00640 1 --2- 172.21.15.131:0/4193143312 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f841806be30 0x7f841806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:03.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.579+0000 7f8436a00640 1 --2- 172.21.15.131:0/4193143312 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f841806be30 0x7f841806e2f0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f84381415c0 tx=0x7f8424008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:03.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.582+0000 7f842f400640 1 -- 172.21.15.131:0/4193143312 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f843813a450 con 0x7f841c004d40 2024-05-26T22:55:03.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.819+0000 7f843d200640 1 -- 172.21.15.131:0/4193143312 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f84380b5ed0 con 0x7f841c004d40 2024-05-26T22:55:03.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.820+0000 7f842f400640 1 -- 172.21.15.131:0/4193143312 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) ==== 94+0+5189 (secure 0 0 0) 0x7f8428059760 con 0x7f841c004d40 2024-05-26T22:55:03.820 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:55:03.820 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":94,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5231,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/2037149126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6835","nonce":2037149126},{"type":"v1","addr":"172.21.15.131:6837","nonce":2037149126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":89},{"gid":5243,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1314245822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1314245822},{"type":"v1","addr":"172.21.15.187:6833","nonce":1314245822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":89},{"gid":5246,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/521878375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":521878375},{"type":"v1","addr":"172.21.15.187:6835","nonce":521878375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":89}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-26T22:54:57.328457+0000","modified":"2024-05-26T22:55:00.415141+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7863},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7863":{"gid":7863,"name":"a","rank":0,"incarnation":90,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6836/789312503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":789312503},{"type":"v1","addr":"172.21.15.131:6836","nonce":789312503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7863,"qdb_cluster":[7863]},"id":7}]} 2024-05-26T22:55:03.820 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 94 2024-05-26T22:55:03.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.822+0000 7f843d200640 1 -- 172.21.15.131:0/4193143312 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f841806be30 msgr2=0x7f841806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:03.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.822+0000 7f843d200640 1 --2- 172.21.15.131:0/4193143312 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f841806be30 0x7f841806e2f0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f84381415c0 tx=0x7f8424008040 comp rx=0 tx=0).stop 2024-05-26T22:55:03.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.822+0000 7f843d200640 1 -- 172.21.15.131:0/4193143312 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f841c004d40 msgr2=0x7f84380b2480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:03.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.822+0000 7f843d200640 1 --2- 172.21.15.131:0/4193143312 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f841c004d40 0x7f84380b2480 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f842800e870 tx=0x7f842800ed30 comp rx=0 tx=0).stop 2024-05-26T22:55:03.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.822+0000 7f843d200640 1 -- 172.21.15.131:0/4193143312 shutdown_connections 2024-05-26T22:55:03.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.822+0000 7f843d200640 1 --2- 172.21.15.131:0/4193143312 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f843815d3f0 0x7f8438139900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:03.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.823+0000 7f843d200640 1 --2- 172.21.15.131:0/4193143312 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f841806be30 0x7f841806e2f0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:03.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.823+0000 7f843d200640 1 --2- 172.21.15.131:0/4193143312 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f843815ea00 0x7f8438158890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:03.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.823+0000 7f843d200640 1 --2- 172.21.15.131:0/4193143312 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f841c004d40 0x7f84380b2480 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:03.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.823+0000 7f843d200640 1 -- 172.21.15.131:0/4193143312 >> 172.21.15.131:0/4193143312 conn(0x7f84380cff40 msgr2=0x7f84380cf370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:03.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.823+0000 7f843d200640 1 -- 172.21.15.131:0/4193143312 shutdown_connections 2024-05-26T22:55:03.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:03.823+0000 7f843d200640 1 -- 172.21.15.131:0/4193143312 wait complete. 2024-05-26T22:55:04.245 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 94, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-26T22:54:57.328457+0000', 'modified': '2024-05-26T22:55:00.415141+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7863}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7863': {'gid': 7863, 'name': 'a', 'rank': 0, 'incarnation': 90, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.131:6836/789312503', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6834', 'nonce': 789312503}, {'type': 'v1', 'addr': '172.21.15.131:6836', 'nonce': 789312503}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7863, 'qdb_cluster': [7863]} 2024-05-26T22:55:04.245 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-26T22:55:04.246 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-05-26T22:55:04.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.458+0000 7f9015400640 1 Processor -- start 2024-05-26T22:55:04.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.458+0000 7f9015400640 1 -- start start 2024-05-26T22:55:04.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.459+0000 7f9015400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f90100ceae0 0x7f901015c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:04.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.459+0000 7f9015400640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f9010071230 0x7f90100b6560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:04.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.459+0000 7f9015400640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90100716c0 con 0x7f9010145c00 2024-05-26T22:55:04.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.459+0000 7f9015400640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f90100d2040 con 0x7f9010071230 2024-05-26T22:55:04.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.459+0000 7f9015400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f90100d2290 con 0x7f90100ceae0 2024-05-26T22:55:04.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.460+0000 7f900de00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f9010145c00 0x7f90100ceec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:60878/0 (socket says 172.21.15.131:60878) 2024-05-26T22:55:04.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.460+0000 7f900de00640 1 -- 172.21.15.131:0/2084290465 learned_addr learned my addr 172.21.15.131:0/2084290465 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:55:04.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.460+0000 7f900e800640 1 --2- 172.21.15.131:0/2084290465 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f90100ceae0 0x7f901015c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:04.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.460+0000 7f900f200640 1 --2- 172.21.15.131:0/2084290465 >> v2:172.21.15.187:3300/0 conn(0x7f9010071230 0x7f90100b6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:04.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.460+0000 7f900d400640 1 -- 172.21.15.131:0/2084290465 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3941308362 0 0) 0x7f90100716c0 con 0x7f9010145c00 2024-05-26T22:55:04.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.460+0000 7f900d400640 1 -- 172.21.15.131:0/2084290465 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8ff4003520 con 0x7f9010145c00 2024-05-26T22:55:04.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.461+0000 7f900d400640 1 -- 172.21.15.131:0/2084290465 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2355313727 0 0) 0x7f8ff4003520 con 0x7f9010145c00 2024-05-26T22:55:04.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.461+0000 7f900d400640 1 -- 172.21.15.131:0/2084290465 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f90100ceae0 msgr2=0x7f901015c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:04.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.461+0000 7f900d400640 1 --2- 172.21.15.131:0/2084290465 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f90100ceae0 0x7f901015c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:04.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.461+0000 7f900d400640 1 -- 172.21.15.131:0/2084290465 >> v2:172.21.15.187:3300/0 conn(0x7f9010071230 msgr2=0x7f90100b6560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:04.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.461+0000 7f900d400640 1 --2- 172.21.15.131:0/2084290465 >> v2:172.21.15.187:3300/0 conn(0x7f9010071230 0x7f90100b6560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:04.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.461+0000 7f900d400640 1 -- 172.21.15.131:0/2084290465 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90100d2570 con 0x7f9010145c00 2024-05-26T22:55:04.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.461+0000 7f900d400640 1 -- 172.21.15.131:0/2084290465 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7f90000019f0 con 0x7f9010145c00 2024-05-26T22:55:04.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.461+0000 7f900d400640 1 -- 172.21.15.131:0/2084290465 >> v1:172.21.15.131:6789/0 conn(0x7f9010145c00 legacy=0x7f90100ceec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:04.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.461+0000 7f900d400640 1 --2- 172.21.15.131:0/2084290465 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ff40039b0 0x7f8ff4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:04.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.461+0000 7f900d400640 1 --2- 172.21.15.131:0/2084290465 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f90100ceae0 0x7f8ff4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:04.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.461+0000 7f900d400640 1 --2- 172.21.15.131:0/2084290465 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ff4004d40 0x7f8ff40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:04.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.461+0000 7f900d400640 1 -- 172.21.15.131:0/2084290465 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f90100d2040 con 0x7f8ff4004d40 2024-05-26T22:55:04.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.461+0000 7f900d400640 1 -- 172.21.15.131:0/2084290465 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f90100d2290 con 0x7f8ff40039b0 2024-05-26T22:55:04.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.461+0000 7f900d400640 1 -- 172.21.15.131:0/2084290465 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f8ff400b9c0 con 0x7f90100ceae0 2024-05-26T22:55:04.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.461+0000 7f900f200640 1 --2- 172.21.15.131:0/2084290465 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ff4004d40 0x7f8ff40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:04.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.461+0000 7f900de00640 1 --2- 172.21.15.131:0/2084290465 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f90100ceae0 0x7f8ff4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:04.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.462+0000 7f900e800640 1 --2- 172.21.15.131:0/2084290465 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ff40039b0 0x7f8ff4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:04.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.462+0000 7f900f200640 1 -- 172.21.15.131:0/2084290465 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f90100ceae0 msgr2=0x7f8ff4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:04.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.462+0000 7f900f200640 1 --2- 172.21.15.131:0/2084290465 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f90100ceae0 0x7f8ff4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:04.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.462+0000 7f900f200640 1 -- 172.21.15.131:0/2084290465 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ff40039b0 msgr2=0x7f8ff4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:04.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.462+0000 7f900f200640 1 --2- 172.21.15.131:0/2084290465 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ff40039b0 0x7f8ff4003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:04.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.462+0000 7f900f200640 1 -- 172.21.15.131:0/2084290465 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ff4003520 con 0x7f8ff4004d40 2024-05-26T22:55:04.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.462+0000 7f900de00640 1 --2- 172.21.15.131:0/2084290465 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f90100ceae0 0x7f8ff4004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:55:04.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.463+0000 7f900f200640 1 --2- 172.21.15.131:0/2084290465 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ff4004d40 0x7f8ff40051c0 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f9008009ac0 tx=0x7f8ff4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67ce5c181ac899a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:04.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.463+0000 7f900d400640 1 -- 172.21.15.131:0/2084290465 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9008036070 con 0x7f8ff4004d40 2024-05-26T22:55:04.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.463+0000 7f900d400640 1 -- 172.21.15.131:0/2084290465 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9008031400 con 0x7f8ff4004d40 2024-05-26T22:55:04.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.463+0000 7f900d400640 1 -- 172.21.15.131:0/2084290465 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9008009690 con 0x7f8ff4004d40 2024-05-26T22:55:04.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.463+0000 7f9015400640 1 -- 172.21.15.131:0/2084290465 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ff4004d40 msgr2=0x7f8ff40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:04.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.463+0000 7f9015400640 1 --2- 172.21.15.131:0/2084290465 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ff4004d40 0x7f8ff40051c0 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f9008009ac0 tx=0x7f8ff4005830 comp rx=0 tx=0).stop 2024-05-26T22:55:04.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.464+0000 7f900f200640 1 -- 172.21.15.131:0/2084290465 reap_dead start 2024-05-26T22:55:04.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.464+0000 7f9015400640 1 -- 172.21.15.131:0/2084290465 shutdown_connections 2024-05-26T22:55:04.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.464+0000 7f9015400640 1 -- 172.21.15.131:0/2084290465 >> 172.21.15.131:0/2084290465 conn(0x7f901006f6d0 msgr2=0x7f901006fae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:04.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.464+0000 7f9015400640 1 -- 172.21.15.131:0/2084290465 shutdown_connections 2024-05-26T22:55:04.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.464+0000 7f9015400640 1 -- 172.21.15.131:0/2084290465 wait complete. 2024-05-26T22:55:04.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.466+0000 7f9015400640 1 Processor -- start 2024-05-26T22:55:04.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.466+0000 7f9015400640 1 -- start start 2024-05-26T22:55:04.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.467+0000 7f9015400640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ff4004d40 0x7f9010140ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:04.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.467+0000 7f9015400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f901014eca0 0x7f9010143440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:04.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.467+0000 7f9015400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9010143980 0x7f90101f79b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:04.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.467+0000 7f9015400640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f90100d2e10 con 0x7f9010143980 2024-05-26T22:55:04.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.467+0000 7f9015400640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f90100d2b30 con 0x7f8ff4004d40 2024-05-26T22:55:04.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.467+0000 7f9015400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9010145850 con 0x7f901014eca0 2024-05-26T22:55:04.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.467+0000 7f900f200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9010143980 0x7f90101f79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:04.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.467+0000 7f900f200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9010143980 0x7f90101f79b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51028/0 (socket says 172.21.15.131:51028) 2024-05-26T22:55:04.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.467+0000 7f900f200640 1 -- 172.21.15.131:0/1714352109 learned_addr learned my addr 172.21.15.131:0/1714352109 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:55:04.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.467+0000 7f900e800640 1 --2- 172.21.15.131:0/1714352109 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ff4004d40 0x7f9010140ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:04.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.467+0000 7f900de00640 1 --2- 172.21.15.131:0/1714352109 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f901014eca0 0x7f9010143440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:04.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.467+0000 7f900f200640 1 -- 172.21.15.131:0/1714352109 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f901014eca0 msgr2=0x7f9010143440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:04.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.467+0000 7f900f200640 1 --2- 172.21.15.131:0/1714352109 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f901014eca0 0x7f9010143440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:04.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.467+0000 7f900f200640 1 -- 172.21.15.131:0/1714352109 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ff4004d40 msgr2=0x7f9010140ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:04.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.467+0000 7f900f200640 1 --2- 172.21.15.131:0/1714352109 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ff4004d40 0x7f9010140ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:04.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.467+0000 7f900f200640 1 -- 172.21.15.131:0/1714352109 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90101e49e0 con 0x7f9010143980 2024-05-26T22:55:04.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.467+0000 7f900de00640 1 --2- 172.21.15.131:0/1714352109 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f901014eca0 0x7f9010143440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:55:04.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.468+0000 7f900f200640 1 --2- 172.21.15.131:0/1714352109 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9010143980 0x7f90101f79b0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f9008009ac0 tx=0x7f9008004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:04.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.468+0000 7f9006a00640 1 -- 172.21.15.131:0/1714352109 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f900803f020 con 0x7f9010143980 2024-05-26T22:55:04.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.468+0000 7f9006a00640 1 -- 172.21.15.131:0/1714352109 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90080038a0 con 0x7f9010143980 2024-05-26T22:55:04.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.468+0000 7f9006a00640 1 -- 172.21.15.131:0/1714352109 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9008036040 con 0x7f9010143980 2024-05-26T22:55:04.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.468+0000 7f9015400640 1 -- 172.21.15.131:0/1714352109 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f901014c2e0 con 0x7f9010143980 2024-05-26T22:55:04.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.468+0000 7f9015400640 1 -- 172.21.15.131:0/1714352109 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9010152580 con 0x7f9010143980 2024-05-26T22:55:04.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.471+0000 7f9006a00640 1 -- 172.21.15.131:0/1714352109 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9008042020 con 0x7f9010143980 2024-05-26T22:55:04.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.471+0000 7f9006a00640 1 --2- 172.21.15.131:0/1714352109 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f8fec06be30 0x7f8fec06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:04.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.472+0000 7f9006a00640 1 -- 172.21.15.131:0/1714352109 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 5709+0+0 (secure 0 0 0) 0x7f90080aa4f0 con 0x7f9010143980 2024-05-26T22:55:04.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.472+0000 7f900e800640 1 --2- 172.21.15.131:0/1714352109 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f8fec06be30 0x7f8fec06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:04.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.472+0000 7f9018229740 1 Processor -- start 2024-05-26T22:55:04.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.472+0000 7f9018229740 1 -- start start 2024-05-26T22:55:04.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.472+0000 7f900e800640 1 --2- 172.21.15.131:0/1714352109 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f8fec06be30 0x7f8fec06e2f0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f8ff8009910 tx=0x7f8ff8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:04.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.473+0000 7f9018229740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e61689fc70 0x55e616852890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:04.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.473+0000 7f9018229740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e616882d90 0x55e616882110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:04.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.473+0000 7f9018229740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e616882650 0x55e616857460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:04.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.473+0000 7f9018229740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55e6167ba610 con 0x55e616882650 2024-05-26T22:55:04.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.473+0000 7f9018229740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55e6167625c0 con 0x55e61689fc70 2024-05-26T22:55:04.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.473+0000 7f9018229740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x55e6167b65d0 con 0x55e616882d90 2024-05-26T22:55:04.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.473+0000 7f900f200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e616882650 0x55e616857460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:04.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.473+0000 7f900f200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e616882650 0x55e616857460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51038/0 (socket says 172.21.15.131:51038) 2024-05-26T22:55:04.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.473+0000 7f900f200640 1 -- 172.21.15.131:0/2891942622 learned_addr learned my addr 172.21.15.131:0/2891942622 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:55:04.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.474+0000 7f900f200640 1 -- 172.21.15.131:0/2891942622 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e616882d90 msgr2=0x55e616882110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:04.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.474+0000 7f900f200640 1 --2- 172.21.15.131:0/2891942622 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e616882d90 0x55e616882110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:04.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.474+0000 7f900f200640 1 -- 172.21.15.131:0/2891942622 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e61689fc70 msgr2=0x55e616852890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:04.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.474+0000 7f900f200640 1 --2- 172.21.15.131:0/2891942622 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e61689fc70 0x55e616852890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:04.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.474+0000 7f900f200640 1 -- 172.21.15.131:0/2891942622 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6164ce2d0 con 0x55e616882650 2024-05-26T22:55:04.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.474+0000 7f900f200640 1 --2- 172.21.15.131:0/2891942622 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e616882650 0x55e616857460 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f90080a8950 tx=0x7f9008078fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68965665a9ac9d5d server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:04.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.474+0000 7f8fff000640 1 -- 172.21.15.131:0/2891942622 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9008081370 con 0x55e616882650 2024-05-26T22:55:04.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.474+0000 7f8fff000640 1 -- 172.21.15.131:0/2891942622 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9008076a90 con 0x55e616882650 2024-05-26T22:55:04.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.474+0000 7f8fff000640 1 -- 172.21.15.131:0/2891942622 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9008074e00 con 0x55e616882650 2024-05-26T22:55:04.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.475+0000 7f9018229740 1 -- 172.21.15.131:0/2891942622 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e616882650 msgr2=0x55e616857460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:04.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.475+0000 7f9018229740 1 --2- 172.21.15.131:0/2891942622 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e616882650 0x55e616857460 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f90080a8950 tx=0x7f9008078fd0 comp rx=0 tx=0).stop 2024-05-26T22:55:04.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.475+0000 7f9018229740 1 -- 172.21.15.131:0/2891942622 shutdown_connections 2024-05-26T22:55:04.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.475+0000 7f9018229740 1 --2- 172.21.15.131:0/2891942622 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e616882d90 0x55e616882110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:04.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.475+0000 7f9018229740 1 --2- 172.21.15.131:0/2891942622 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e616882650 0x55e616857460 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:04.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.475+0000 7f9018229740 1 --2- 172.21.15.131:0/2891942622 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e61689fc70 0x55e616852890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:04.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.475+0000 7f9018229740 1 -- 172.21.15.131:0/2891942622 >> 172.21.15.131:0/2891942622 conn(0x55e616854b00 msgr2=0x55e61689d110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:04.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.475+0000 7f9018229740 1 -- 172.21.15.131:0/2891942622 shutdown_connections 2024-05-26T22:55:04.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.475+0000 7f9018229740 1 -- 172.21.15.131:0/2891942622 wait complete. 2024-05-26T22:55:04.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.476+0000 7f9018229740 1 -- start start 2024-05-26T22:55:04.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.476+0000 7f9018229740 1 Processor -- start 2024-05-26T22:55:04.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.477+0000 7f9018229740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e616882d90 0x55e61687d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:04.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.477+0000 7f9018229740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e61689fc70 0x55e616885b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:04.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.477+0000 7f9018229740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e61687d9e0 0x55e6168acd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:04.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.477+0000 7f9018229740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55e6167b65d0 con 0x55e61687d9e0 2024-05-26T22:55:04.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.477+0000 7f9018229740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55e616775910 con 0x55e616882d90 2024-05-26T22:55:04.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.477+0000 7f9018229740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x55e6167ba610 con 0x55e61689fc70 2024-05-26T22:55:04.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.477+0000 7f9018229740 10 client.0 _ll_register_callbacks cb 0x55e6167b1410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-26T22:55:04.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.477+0000 7f900f200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e61687d9e0 0x55e6168acd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:04.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.477+0000 7f900f200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e61687d9e0 0x55e6168acd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51040/0 (socket says 172.21.15.131:51040) 2024-05-26T22:55:04.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.477+0000 7f900f200640 1 -- 172.21.15.131:0/2908123750 learned_addr learned my addr 172.21.15.131:0/2908123750 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:55:04.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.478+0000 7f900e800640 1 --2- 172.21.15.131:0/2908123750 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e616882d90 0x55e61687d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:04.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.478+0000 7f900f200640 1 -- 172.21.15.131:0/2908123750 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e61689fc70 msgr2=0x55e616885b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:04.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.478+0000 7f900f200640 1 --2- 172.21.15.131:0/2908123750 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e61689fc70 0x55e616885b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:04.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.478+0000 7f900f200640 1 -- 172.21.15.131:0/2908123750 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e616882d90 msgr2=0x55e61687d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:04.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.478+0000 7f900f200640 1 --2- 172.21.15.131:0/2908123750 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e616882d90 0x55e61687d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:04.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.478+0000 7f900f200640 1 -- 172.21.15.131:0/2908123750 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6165986b0 con 0x55e61687d9e0 2024-05-26T22:55:04.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.478+0000 7f900f200640 1 --2- 172.21.15.131:0/2908123750 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e61687d9e0 0x55e6168acd60 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f9008046040 tx=0x7f9008009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=103b8a5ea904ad1e server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:04.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.478+0000 7f8ff3400640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-26T22:55:04.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.478+0000 7f8ff3400640 1 -- 172.21.15.131:0/2908123750 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90080aa0c0 con 0x55e61687d9e0 2024-05-26T22:55:04.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.478+0000 7f8ff3400640 1 -- 172.21.15.131:0/2908123750 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 5709+0+0 (secure 0 0 0) 0x7f90080aa260 con 0x55e61687d9e0 2024-05-26T22:55:04.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.478+0000 7f9018229740 1 -- 172.21.15.131:0/2908123750 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9008076a90 con 0x55e61687d9e0 2024-05-26T22:55:04.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.478+0000 7f8ff3400640 1 -- 172.21.15.131:0/2908123750 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7f9008080ee0 con 0x55e61687d9e0 2024-05-26T22:55:04.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.479+0000 7f9018229740 10 client.7998 fetch_fsmap learned FSMap version 94 2024-05-26T22:55:04.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.479+0000 7f9018229740 1 -- 172.21.15.131:0/2908123750 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=94,monmap=2+}) -- 0x55e616885380 con 0x55e61687d9e0 2024-05-26T22:55:04.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.479+0000 7f8ff3400640 1 -- 172.21.15.131:0/2908123750 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9008076a90 con 0x55e61687d9e0 2024-05-26T22:55:04.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.479+0000 7f8ff3400640 1 -- 172.21.15.131:0/2908123750 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 94) ==== 3494+0+0 (secure 0 0 0) 0x7f9008097b30 con 0x55e61687d9e0 2024-05-26T22:55:04.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.479+0000 7f9018229740 10 client.7998 fetch_fsmap finished waiting for FSMap version 94 2024-05-26T22:55:04.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.479+0000 7f9018229740 10 client.7998 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-26T22:55:04.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.479+0000 7f9018229740 20 client.7998 populate_metadata read hostname 'smithi131' 2024-05-26T22:55:04.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.479+0000 7f9018229740 1 --2- 172.21.15.131:0/2908123750 >> [v2:172.21.15.131:6834/789312503,v1:172.21.15.131:6836/789312503] conn(0x55e6168b86f0 0x55e6168baae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:04.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.479+0000 7f9018229740 4 client.7998 mds_command: new command op to 7863 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-26T22:55:04.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.479+0000 7f9018229740 1 -- 172.21.15.131:0/2908123750 --> [v2:172.21.15.131:6834/789312503,v1:172.21.15.131:6836/789312503] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e6167c7d40 con 0x55e6168b86f0 2024-05-26T22:55:04.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.479+0000 7f900e800640 1 --2- 172.21.15.131:0/2908123750 >> [v2:172.21.15.131:6834/789312503,v1:172.21.15.131:6836/789312503] conn(0x55e6168b86f0 0x55e6168baae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:04.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.480+0000 7f900e800640 1 --2- 172.21.15.131:0/2908123750 >> [v2:172.21.15.131:6834/789312503,v1:172.21.15.131:6836/789312503] conn(0x55e6168b86f0 0x55e6168baae0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28a54976545f4b19 server_cookie=f59aae62a5f69e09 in_seq=0 out_seq=0 2024-05-26T22:55:04.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.480+0000 7f8ff3400640 10 client.7998 ms_handle_connect on v2:172.21.15.131:6834/789312503 2024-05-26T22:55:04.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.481+0000 7f8ff3400640 1 -- 172.21.15.131:0/2908123750 <== mds.0 v2:172.21.15.131:6834/789312503 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e6167c7d40 con 0x55e6168b86f0 2024-05-26T22:55:04.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.481+0000 7f8ff3400640 10 client.7998 handle_command_reply: tid=0 2024-05-26T22:55:04.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.481+0000 7f9018229740 1 client.7998 shutdown 2024-05-26T22:55:04.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.481+0000 7f9018229740 1 -- 172.21.15.131:0/2908123750 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e61687d9e0 msgr2=0x55e6168acd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:04.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.481+0000 7f9018229740 1 --2- 172.21.15.131:0/2908123750 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e61687d9e0 0x55e6168acd60 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f9008046040 tx=0x7f9008009190 comp rx=0 tx=0).stop 2024-05-26T22:55:04.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.481+0000 7f9018229740 1 -- 172.21.15.131:0/2908123750 shutdown_connections 2024-05-26T22:55:04.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.481+0000 7f9018229740 1 --2- 172.21.15.131:0/2908123750 >> [v2:172.21.15.131:6834/789312503,v1:172.21.15.131:6836/789312503] conn(0x55e6168b86f0 0x55e6168baae0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:04.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.481+0000 7f9018229740 1 --2- 172.21.15.131:0/2908123750 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e61689fc70 0x55e616885b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:04.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.481+0000 7f9018229740 1 --2- 172.21.15.131:0/2908123750 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e61687d9e0 0x55e6168acd60 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:04.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.481+0000 7f9018229740 1 --2- 172.21.15.131:0/2908123750 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e616882d90 0x55e61687d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:04.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.481+0000 7f8ff3400640 0 client.7998 ms_handle_reset on v2:172.21.15.131:6834/789312503 2024-05-26T22:55:04.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.481+0000 7f9018229740 1 -- 172.21.15.131:0/2908123750 >> 172.21.15.131:0/2908123750 conn(0x55e616882650 msgr2=0x55e6168560f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:04.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.481+0000 7f9018229740 1 -- 172.21.15.131:0/2908123750 shutdown_connections 2024-05-26T22:55:04.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.481+0000 7f9018229740 1 -- 172.21.15.131:0/2908123750 wait complete. 2024-05-26T22:55:04.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.482+0000 7f9018229740 20 client.7998 trim_cache size 0 max 16384 2024-05-26T22:55:04.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.502+0000 7f9018229740 1 Processor -- start 2024-05-26T22:55:04.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.502+0000 7f9018229740 1 -- start start 2024-05-26T22:55:04.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.503+0000 7f9018229740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e61687d9e0 0x55e616852b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:04.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.503+0000 7f9018229740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e616882d90 0x55e61689ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:04.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.503+0000 7f9018229740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e61689fc70 0x55e616911e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:04.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.503+0000 7f9018229740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f9010200120 con 0x55e61689fc70 2024-05-26T22:55:04.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.503+0000 7f9018229740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f901014da00 con 0x55e616882d90 2024-05-26T22:55:04.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.503+0000 7f9018229740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f901014e050 con 0x55e61687d9e0 2024-05-26T22:55:04.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.504+0000 7f900f200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e61689fc70 0x55e616911e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:04.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.504+0000 7f900f200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e61689fc70 0x55e616911e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51054/0 (socket says 172.21.15.131:51054) 2024-05-26T22:55:04.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.504+0000 7f900f200640 1 -- 172.21.15.131:0/2173453394 learned_addr learned my addr 172.21.15.131:0/2173453394 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:55:04.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.504+0000 7f900de00640 1 --2- 172.21.15.131:0/2173453394 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e616882d90 0x55e61689ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:04.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.504+0000 7f900e800640 1 --2- 172.21.15.131:0/2173453394 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e61687d9e0 0x55e616852b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:04.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.504+0000 7f900f200640 1 -- 172.21.15.131:0/2173453394 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e61687d9e0 msgr2=0x55e616852b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:04.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.504+0000 7f900f200640 1 --2- 172.21.15.131:0/2173453394 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e61687d9e0 0x55e616852b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:04.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.504+0000 7f900f200640 1 -- 172.21.15.131:0/2173453394 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e616882d90 msgr2=0x55e61689ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:04.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.504+0000 7f900f200640 1 --2- 172.21.15.131:0/2173453394 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e616882d90 0x55e61689ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:04.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.504+0000 7f900f200640 1 -- 172.21.15.131:0/2173453394 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e616853080 con 0x55e61689fc70 2024-05-26T22:55:04.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.504+0000 7f900de00640 1 --2- 172.21.15.131:0/2173453394 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e616882d90 0x55e61689ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-26T22:55:04.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.504+0000 7f900f200640 1 --2- 172.21.15.131:0/2173453394 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e61689fc70 0x55e616911e60 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f9008000c00 tx=0x7f900803d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4820e503aeb44e18 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:04.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.505+0000 7f8ffe600640 1 -- 172.21.15.131:0/2173453394 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9008097b30 con 0x55e61689fc70 2024-05-26T22:55:04.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.505+0000 7f8ffe600640 1 -- 172.21.15.131:0/2173453394 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9008003c90 con 0x55e61689fc70 2024-05-26T22:55:04.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.505+0000 7f8ffe600640 1 -- 172.21.15.131:0/2173453394 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90080aa2c0 con 0x55e61689fc70 2024-05-26T22:55:04.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.505+0000 7f9018229740 1 -- 172.21.15.131:0/2173453394 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e61689fc70 msgr2=0x55e616911e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:04.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.505+0000 7f9018229740 1 --2- 172.21.15.131:0/2173453394 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e61689fc70 0x55e616911e60 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f9008000c00 tx=0x7f900803d040 comp rx=0 tx=0).stop 2024-05-26T22:55:04.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.505+0000 7f9018229740 1 -- 172.21.15.131:0/2173453394 shutdown_connections 2024-05-26T22:55:04.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.505+0000 7f9018229740 1 --2- 172.21.15.131:0/2173453394 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e61689fc70 0x55e616911e60 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:04.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.505+0000 7f9018229740 1 --2- 172.21.15.131:0/2173453394 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e616882d90 0x55e61689ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:04.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.505+0000 7f9018229740 1 --2- 172.21.15.131:0/2173453394 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e61687d9e0 0x55e616852b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:04.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.505+0000 7f9018229740 1 -- 172.21.15.131:0/2173453394 >> 172.21.15.131:0/2173453394 conn(0x55e616882650 msgr2=0x55e61684e040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:04.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.505+0000 7f9018229740 1 -- 172.21.15.131:0/2173453394 shutdown_connections 2024-05-26T22:55:04.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.505+0000 7f9018229740 1 -- 172.21.15.131:0/2173453394 wait complete. 2024-05-26T22:55:04.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.506+0000 7f9018229740 1 -- start start 2024-05-26T22:55:04.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.506+0000 7f9018229740 1 Processor -- start 2024-05-26T22:55:04.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.507+0000 7f9018229740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e61687d9e0 0x55e6169110e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:04.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.507+0000 7f9018229740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e616882d90 0x55e616851920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:04.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.507+0000 7f9018229740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e61689fc70 0x55e616851e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:04.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.507+0000 7f9018229740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55e6167b65d0 con 0x55e616882d90 2024-05-26T22:55:04.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.507+0000 7f9018229740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55e6167625c0 con 0x55e61689fc70 2024-05-26T22:55:04.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.507+0000 7f9018229740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x55e6168845f0 con 0x55e61687d9e0 2024-05-26T22:55:04.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.507+0000 7f9018229740 10 client.0 _ll_register_callbacks cb 0x7f9008078180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-26T22:55:04.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.508+0000 7f900de00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e616882d90 0x55e616851920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:04.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.508+0000 7f900de00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e616882d90 0x55e616851920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51064/0 (socket says 172.21.15.131:51064) 2024-05-26T22:55:04.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.508+0000 7f900de00640 1 -- 172.21.15.131:0/1155448972 learned_addr learned my addr 172.21.15.131:0/1155448972 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:55:04.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.508+0000 7f900de00640 1 -- 172.21.15.131:0/1155448972 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e61687d9e0 msgr2=0x55e6169110e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:04.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.508+0000 7f900de00640 1 --2- 172.21.15.131:0/1155448972 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e61687d9e0 0x55e6169110e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:04.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.508+0000 7f900de00640 1 -- 172.21.15.131:0/1155448972 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e61689fc70 msgr2=0x55e616851e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:04.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.508+0000 7f900de00640 1 --2- 172.21.15.131:0/1155448972 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e61689fc70 0x55e616851e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:04.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.508+0000 7f900de00640 1 -- 172.21.15.131:0/1155448972 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e616856ce0 con 0x55e616882d90 2024-05-26T22:55:04.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.508+0000 7f900de00640 1 --2- 172.21.15.131:0/1155448972 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e616882d90 0x55e616851920 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x55e61689f710 tx=0x7f90000078d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8876d351c07e2194 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:04.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.508+0000 7f8ff3400640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-26T22:55:04.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.509+0000 7f8ff3400640 1 -- 172.21.15.131:0/1155448972 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9000018020 con 0x55e616882d90 2024-05-26T22:55:04.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.509+0000 7f8ff3400640 1 -- 172.21.15.131:0/1155448972 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 5709+0+0 (secure 0 0 0) 0x7f9000010ba0 con 0x55e616882d90 2024-05-26T22:55:04.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.509+0000 7f9018229740 1 -- 172.21.15.131:0/1155448972 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9008003c90 con 0x55e616882d90 2024-05-26T22:55:04.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.509+0000 7f8ff3400640 1 -- 172.21.15.131:0/1155448972 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7f9000022050 con 0x55e616882d90 2024-05-26T22:55:04.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.509+0000 7f9018229740 10 client.8004 fetch_fsmap learned FSMap version 94 2024-05-26T22:55:04.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.509+0000 7f9018229740 1 -- 172.21.15.131:0/1155448972 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=94,monmap=2+}) -- 0x55e61690eff0 con 0x55e616882d90 2024-05-26T22:55:04.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.509+0000 7f8ff3400640 1 -- 172.21.15.131:0/1155448972 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9008003c90 con 0x55e616882d90 2024-05-26T22:55:04.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.509+0000 7f8ff3400640 1 -- 172.21.15.131:0/1155448972 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 94) ==== 3494+0+0 (secure 0 0 0) 0x7f9000025020 con 0x55e616882d90 2024-05-26T22:55:04.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.509+0000 7f9018229740 10 client.8004 fetch_fsmap finished waiting for FSMap version 94 2024-05-26T22:55:04.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.509+0000 7f9018229740 10 client.8004 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-26T22:55:04.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.509+0000 7f9018229740 20 client.8004 populate_metadata read hostname 'smithi131' 2024-05-26T22:55:04.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.509+0000 7f9018229740 1 --2- 172.21.15.131:0/1155448972 >> [v2:172.21.15.131:6834/789312503,v1:172.21.15.131:6836/789312503] conn(0x55e6168ab810 0x55e6168abbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:04.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.509+0000 7f9018229740 4 client.8004 mds_command: new command op to 7863 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-26T22:55:04.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.509+0000 7f9018229740 1 -- 172.21.15.131:0/1155448972 --> [v2:172.21.15.131:6834/789312503,v1:172.21.15.131:6836/789312503] -- command(tid 0: {"prefix": "status"}) -- 0x55e6167c2090 con 0x55e6168ab810 2024-05-26T22:55:04.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.510+0000 7f900e800640 1 --2- 172.21.15.131:0/1155448972 >> [v2:172.21.15.131:6834/789312503,v1:172.21.15.131:6836/789312503] conn(0x55e6168ab810 0x55e6168abbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:04.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.510+0000 7f900e800640 1 --2- 172.21.15.131:0/1155448972 >> [v2:172.21.15.131:6834/789312503,v1:172.21.15.131:6836/789312503] conn(0x55e6168ab810 0x55e6168abbf0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=443e842837b3f829 server_cookie=c33827beaa95514d in_seq=0 out_seq=0 2024-05-26T22:55:04.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.510+0000 7f8ff3400640 10 client.8004 ms_handle_connect on v2:172.21.15.131:6834/789312503 2024-05-26T22:55:04.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.510+0000 7f8ff3400640 1 -- 172.21.15.131:0/1155448972 <== mds.0 v2:172.21.15.131:6834/789312503 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x55e6167c2090 con 0x55e6168ab810 2024-05-26T22:55:04.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.510+0000 7f8ff3400640 10 client.8004 handle_command_reply: tid=0 2024-05-26T22:55:04.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.510+0000 7f9018229740 1 client.8004 shutdown 2024-05-26T22:55:04.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.511+0000 7f9018229740 1 -- 172.21.15.131:0/1155448972 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e616882d90 msgr2=0x55e616851920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:04.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.511+0000 7f9018229740 1 --2- 172.21.15.131:0/1155448972 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e616882d90 0x55e616851920 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x55e61689f710 tx=0x7f90000078d0 comp rx=0 tx=0).stop 2024-05-26T22:55:04.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.511+0000 7f9018229740 1 -- 172.21.15.131:0/1155448972 shutdown_connections 2024-05-26T22:55:04.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.511+0000 7f9018229740 1 --2- 172.21.15.131:0/1155448972 >> [v2:172.21.15.131:6834/789312503,v1:172.21.15.131:6836/789312503] conn(0x55e6168ab810 0x55e6168abbf0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:04.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.511+0000 7f9018229740 1 --2- 172.21.15.131:0/1155448972 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e61689fc70 0x55e616851e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:04.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.511+0000 7f9018229740 1 --2- 172.21.15.131:0/1155448972 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e616882d90 0x55e616851920 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:04.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.511+0000 7f9018229740 1 --2- 172.21.15.131:0/1155448972 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e61687d9e0 0x55e6169110e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:04.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.511+0000 7f8ff3400640 0 client.8004 ms_handle_reset on v2:172.21.15.131:6834/789312503 2024-05-26T22:55:04.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.511+0000 7f9018229740 1 -- 172.21.15.131:0/1155448972 >> 172.21.15.131:0/1155448972 conn(0x55e616882650 msgr2=0x55e61689ed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:04.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.511+0000 7f9018229740 1 -- 172.21.15.131:0/1155448972 shutdown_connections 2024-05-26T22:55:04.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.511+0000 7f9018229740 1 -- 172.21.15.131:0/1155448972 wait complete. 2024-05-26T22:55:04.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.511+0000 7f9018229740 20 client.8004 trim_cache size 0 max 16384 2024-05-26T22:55:04.518 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-26T22:55:04.518 INFO:teuthology.orchestra.run.smithi131.stdout: "cluster_fsid": "46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f", 2024-05-26T22:55:04.518 INFO:teuthology.orchestra.run.smithi131.stdout: "whoami": 0, 2024-05-26T22:55:04.518 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 7863, 2024-05-26T22:55:04.518 INFO:teuthology.orchestra.run.smithi131.stdout: "want_state": "up:active", 2024-05-26T22:55:04.519 INFO:teuthology.orchestra.run.smithi131.stdout: "state": "up:active", 2024-05-26T22:55:04.519 INFO:teuthology.orchestra.run.smithi131.stdout: "fs_name": "cephfs", 2024-05-26T22:55:04.519 INFO:teuthology.orchestra.run.smithi131.stdout: "rank_uptime": 6.6048068989999997, 2024-05-26T22:55:04.519 INFO:teuthology.orchestra.run.smithi131.stdout: "mdsmap_epoch": 94, 2024-05-26T22:55:04.519 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch": 95, 2024-05-26T22:55:04.519 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch_barrier": 95, 2024-05-26T22:55:04.519 INFO:teuthology.orchestra.run.smithi131.stdout: "uptime": 12.059575097 2024-05-26T22:55:04.519 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-26T22:55:04.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.512+0000 7f9015400640 1 -- 172.21.15.131:0/1714352109 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f8fec06be30 msgr2=0x7f8fec06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:04.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.512+0000 7f9015400640 1 --2- 172.21.15.131:0/1714352109 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f8fec06be30 0x7f8fec06e2f0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f8ff8009910 tx=0x7f8ff8008040 comp rx=0 tx=0).stop 2024-05-26T22:55:04.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.512+0000 7f9015400640 1 -- 172.21.15.131:0/1714352109 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9010143980 msgr2=0x7f90101f79b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:04.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.512+0000 7f9015400640 1 --2- 172.21.15.131:0/1714352109 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9010143980 0x7f90101f79b0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f9008009ac0 tx=0x7f9008004730 comp rx=0 tx=0).stop 2024-05-26T22:55:04.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.512+0000 7f9015400640 1 -- 172.21.15.131:0/1714352109 shutdown_connections 2024-05-26T22:55:04.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.512+0000 7f9015400640 1 --2- 172.21.15.131:0/1714352109 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f901014eca0 0x7f9010143440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:04.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.512+0000 7f9015400640 1 --2- 172.21.15.131:0/1714352109 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f8fec06be30 0x7f8fec06e2f0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:04.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.512+0000 7f9015400640 1 --2- 172.21.15.131:0/1714352109 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9010143980 0x7f90101f79b0 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:04.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.512+0000 7f9015400640 1 --2- 172.21.15.131:0/1714352109 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ff4004d40 0x7f9010140ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:04.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.512+0000 7f9015400640 1 -- 172.21.15.131:0/1714352109 >> 172.21.15.131:0/1714352109 conn(0x7f901006f6d0 msgr2=0x7f901015e2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:04.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.512+0000 7f9015400640 1 -- 172.21.15.131:0/1714352109 shutdown_connections 2024-05-26T22:55:04.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.512+0000 7f9015400640 1 -- 172.21.15.131:0/1714352109 wait complete. 2024-05-26T22:55:04.529 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-26T22:55:04.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.730+0000 7f398fc00640 1 Processor -- start 2024-05-26T22:55:04.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.731+0000 7f398fc00640 1 -- start start 2024-05-26T22:55:04.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.732+0000 7f398fc00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f3988158c00 0x7f398806d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:04.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.732+0000 7f398fc00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f398806daf0 0x7f39880b3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:04.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.732+0000 7f398fc00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f39880b36d0 0x7f3988161fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:04.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.732+0000 7f398fc00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f398806df50 con 0x7f3988158c00 2024-05-26T22:55:04.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.732+0000 7f398fc00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f39880b3e50 con 0x7f398806daf0 2024-05-26T22:55:04.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.732+0000 7f398fc00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f39880b4130 con 0x7f39880b36d0 2024-05-26T22:55:04.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.733+0000 7f398d200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f3988158c00 0x7f398806d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:04.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.733+0000 7f398d200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f3988158c00 0x7f398806d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51072/0 (socket says 172.21.15.131:51072) 2024-05-26T22:55:04.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.733+0000 7f398d200640 1 -- 172.21.15.131:0/1524551250 learned_addr learned my addr 172.21.15.131:0/1524551250 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:55:04.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.733+0000 7f3987e00640 1 --2- 172.21.15.131:0/1524551250 >> v2:172.21.15.187:3300/0 conn(0x7f398806daf0 0x7f39880b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:04.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.733+0000 7f398d200640 1 -- 172.21.15.131:0/1524551250 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f39880b36d0 msgr2=0x7f3988161fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:04.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.733+0000 7f398d200640 1 --2- 172.21.15.131:0/1524551250 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f39880b36d0 0x7f3988161fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:04.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.733+0000 7f398d200640 1 -- 172.21.15.131:0/1524551250 >> v2:172.21.15.187:3300/0 conn(0x7f398806daf0 msgr2=0x7f39880b3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:04.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.733+0000 7f398d200640 1 --2- 172.21.15.131:0/1524551250 >> v2:172.21.15.187:3300/0 conn(0x7f398806daf0 0x7f39880b3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:04.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.733+0000 7f398d200640 1 -- 172.21.15.131:0/1524551250 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39880b4410 con 0x7f3988158c00 2024-05-26T22:55:04.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.733+0000 7f3987e00640 1 --2- 172.21.15.131:0/1524551250 >> v2:172.21.15.187:3300/0 conn(0x7f398806daf0 0x7f39880b3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:55:04.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.734+0000 7f398d200640 1 --2- 172.21.15.131:0/1524551250 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3988158c00 0x7f398806d5b0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f3988071090 tx=0x7f397802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc34ea62fb3f37dc server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:04.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.735+0000 7f3987400640 1 -- 172.21.15.131:0/1524551250 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3978037070 con 0x7f3988158c00 2024-05-26T22:55:04.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.735+0000 7f3987400640 1 -- 172.21.15.131:0/1524551250 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3978032440 con 0x7f3988158c00 2024-05-26T22:55:04.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.735+0000 7f3987400640 1 -- 172.21.15.131:0/1524551250 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3978032810 con 0x7f3988158c00 2024-05-26T22:55:04.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.735+0000 7f398fc00640 1 -- 172.21.15.131:0/1524551250 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3988158c00 msgr2=0x7f398806d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:04.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.735+0000 7f398fc00640 1 --2- 172.21.15.131:0/1524551250 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3988158c00 0x7f398806d5b0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f3988071090 tx=0x7f397802a140 comp rx=0 tx=0).stop 2024-05-26T22:55:04.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.736+0000 7f398fc00640 1 -- 172.21.15.131:0/1524551250 shutdown_connections 2024-05-26T22:55:04.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.736+0000 7f398fc00640 1 --2- 172.21.15.131:0/1524551250 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f39880b36d0 0x7f3988161fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:04.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.736+0000 7f398fc00640 1 --2- 172.21.15.131:0/1524551250 >> v2:172.21.15.187:3300/0 conn(0x7f398806daf0 0x7f39880b3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:04.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.736+0000 7f398fc00640 1 --2- 172.21.15.131:0/1524551250 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3988158c00 0x7f398806d5b0 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:04.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.736+0000 7f398fc00640 1 -- 172.21.15.131:0/1524551250 >> 172.21.15.131:0/1524551250 conn(0x7f39880b1e90 msgr2=0x7f39880b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:04.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.736+0000 7f398fc00640 1 -- 172.21.15.131:0/1524551250 shutdown_connections 2024-05-26T22:55:04.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.736+0000 7f398fc00640 1 -- 172.21.15.131:0/1524551250 wait complete. 2024-05-26T22:55:04.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.737+0000 7f398fc00640 1 Processor -- start 2024-05-26T22:55:04.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.738+0000 7f398fc00640 1 -- start start 2024-05-26T22:55:04.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.738+0000 7f398fc00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f398806daf0 0x7f39880999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:04.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.739+0000 7f398fc00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f39880b36d0 0x7f39880aeda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:04.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.739+0000 7f398fc00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3988158c00 0x7f39880af2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:04.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.739+0000 7f398fc00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f39880b3b60 con 0x7f3988158c00 2024-05-26T22:55:04.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.739+0000 7f398fc00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f39880b49d0 con 0x7f398806daf0 2024-05-26T22:55:04.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.739+0000 7f398fc00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f39881624e0 con 0x7f39880b36d0 2024-05-26T22:55:04.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.739+0000 7f398dc00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3988158c00 0x7f39880af2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:04.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.739+0000 7f398dc00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3988158c00 0x7f39880af2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51074/0 (socket says 172.21.15.131:51074) 2024-05-26T22:55:04.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.739+0000 7f398dc00640 1 -- 172.21.15.131:0/2512241051 learned_addr learned my addr 172.21.15.131:0/2512241051 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:55:04.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.739+0000 7f398d200640 1 --2- 172.21.15.131:0/2512241051 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f398806daf0 0x7f39880999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:04.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.739+0000 7f3987e00640 1 --2- 172.21.15.131:0/2512241051 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f39880b36d0 0x7f39880aeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:04.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.739+0000 7f398dc00640 1 -- 172.21.15.131:0/2512241051 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f39880b36d0 msgr2=0x7f39880aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:04.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.739+0000 7f398dc00640 1 --2- 172.21.15.131:0/2512241051 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f39880b36d0 0x7f39880aeda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:04.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.739+0000 7f398dc00640 1 -- 172.21.15.131:0/2512241051 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f398806daf0 msgr2=0x7f39880999c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:04.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.739+0000 7f398dc00640 1 --2- 172.21.15.131:0/2512241051 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f398806daf0 0x7f39880999c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:04.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.739+0000 7f398dc00640 1 -- 172.21.15.131:0/2512241051 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39881483b0 con 0x7f3988158c00 2024-05-26T22:55:04.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.740+0000 7f398dc00640 1 --2- 172.21.15.131:0/2512241051 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3988158c00 0x7f39880af2e0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f398000b9a0 tx=0x7f398000be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:04.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.740+0000 7f3984c00640 1 -- 172.21.15.131:0/2512241051 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3980019070 con 0x7f3988158c00 2024-05-26T22:55:04.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.740+0000 7f3984c00640 1 -- 172.21.15.131:0/2512241051 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f398000cbc0 con 0x7f3988158c00 2024-05-26T22:55:04.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.740+0000 7f3984c00640 1 -- 172.21.15.131:0/2512241051 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3980014440 con 0x7f3988158c00 2024-05-26T22:55:04.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.740+0000 7f398fc00640 1 -- 172.21.15.131:0/2512241051 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3988148690 con 0x7f3988158c00 2024-05-26T22:55:04.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.740+0000 7f398fc00640 1 -- 172.21.15.131:0/2512241051 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3988148c50 con 0x7f3988158c00 2024-05-26T22:55:04.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.741+0000 7f398fc00640 1 -- 172.21.15.131:0/2512241051 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39880694e0 con 0x7f3988158c00 2024-05-26T22:55:04.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.742+0000 7f3984c00640 1 -- 172.21.15.131:0/2512241051 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f398001c070 con 0x7f3988158c00 2024-05-26T22:55:04.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.743+0000 7f3984c00640 1 --2- 172.21.15.131:0/2512241051 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f396406be30 0x7f396406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:04.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.743+0000 7f3984c00640 1 -- 172.21.15.131:0/2512241051 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 5709+0+0 (secure 0 0 0) 0x7f398008ac60 con 0x7f3988158c00 2024-05-26T22:55:04.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.744+0000 7f398d200640 1 --2- 172.21.15.131:0/2512241051 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f396406be30 0x7f396406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:04.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.744+0000 7f398d200640 1 --2- 172.21.15.131:0/2512241051 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f396406be30 0x7f396406e2f0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f3978034000 tx=0x7f397802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:04.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.748+0000 7f3984c00640 1 -- 172.21.15.131:0/2512241051 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f3980059580 con 0x7f3988158c00 2024-05-26T22:55:04.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.991+0000 7f398fc00640 1 -- 172.21.15.131:0/2512241051 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f3988160c90 con 0x7f3988158c00 2024-05-26T22:55:04.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.991+0000 7f3984c00640 1 -- 172.21.15.131:0/2512241051 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v94) ==== 53+0+73 (secure 0 0 0) 0x7f3980058e80 con 0x7f3988158c00 2024-05-26T22:55:04.991 INFO:teuthology.orchestra.run.smithi131.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-26T22:55:04.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.994+0000 7f398fc00640 1 -- 172.21.15.131:0/2512241051 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f396406be30 msgr2=0x7f396406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:04.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.994+0000 7f398fc00640 1 --2- 172.21.15.131:0/2512241051 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f396406be30 0x7f396406e2f0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f3978034000 tx=0x7f397802e000 comp rx=0 tx=0).stop 2024-05-26T22:55:04.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.994+0000 7f398fc00640 1 -- 172.21.15.131:0/2512241051 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3988158c00 msgr2=0x7f39880af2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:04.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.994+0000 7f398fc00640 1 --2- 172.21.15.131:0/2512241051 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3988158c00 0x7f39880af2e0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f398000b9a0 tx=0x7f398000be60 comp rx=0 tx=0).stop 2024-05-26T22:55:04.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.994+0000 7f398fc00640 1 -- 172.21.15.131:0/2512241051 shutdown_connections 2024-05-26T22:55:04.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.994+0000 7f398fc00640 1 --2- 172.21.15.131:0/2512241051 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f39880b36d0 0x7f39880aeda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:04.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.994+0000 7f398fc00640 1 --2- 172.21.15.131:0/2512241051 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f396406be30 0x7f396406e2f0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:04.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.994+0000 7f398fc00640 1 --2- 172.21.15.131:0/2512241051 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3988158c00 0x7f39880af2e0 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:04.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.994+0000 7f398fc00640 1 --2- 172.21.15.131:0/2512241051 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f398806daf0 0x7f39880999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:04.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.994+0000 7f398fc00640 1 -- 172.21.15.131:0/2512241051 >> 172.21.15.131:0/2512241051 conn(0x7f39880b1e90 msgr2=0x7f39880b2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:04.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.994+0000 7f398fc00640 1 -- 172.21.15.131:0/2512241051 shutdown_connections 2024-05-26T22:55:04.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:04.994+0000 7f398fc00640 1 -- 172.21.15.131:0/2512241051 wait complete. 2024-05-26T22:55:05.012 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-26T22:55:05.012 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-26T22:55:05.012 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi131.front.sepia.ceph.com 2024-05-26T22:55:05.012 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-26T22:55:05.013 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-26T22:55:05.013 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-26T22:55:05.013 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-26T22:55:05.013 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-26T22:55:05.013 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-26T22:55:05.013 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:55:05.013 DEBUG:teuthology.orchestra.run.smithi131:> ip addr 2024-05-26T22:55:05.063 INFO:teuthology.orchestra.run.smithi131.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-26T22:55:05.063 INFO:teuthology.orchestra.run.smithi131.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-26T22:55:05.063 INFO:teuthology.orchestra.run.smithi131.stdout: inet 127.0.0.1/8 scope host lo 2024-05-26T22:55:05.063 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-05-26T22:55:05.063 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 ::1/128 scope host 2024-05-26T22:55:05.063 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-05-26T22:55:05.063 INFO:teuthology.orchestra.run.smithi131.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-26T22:55:05.063 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:d8:fb:18 brd ff:ff:ff:ff:ff:ff 2024-05-26T22:55:05.063 INFO:teuthology.orchestra.run.smithi131.stdout: altname enp7s0f0 2024-05-26T22:55:05.063 INFO:teuthology.orchestra.run.smithi131.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-26T22:55:05.063 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:d8:fb:19 brd ff:ff:ff:ff:ff:ff 2024-05-26T22:55:05.064 INFO:teuthology.orchestra.run.smithi131.stdout: altname enp7s0f1 2024-05-26T22:55:05.064 INFO:teuthology.orchestra.run.smithi131.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-26T22:55:05.064 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:88:72:f8 brd ff:ff:ff:ff:ff:ff 2024-05-26T22:55:05.064 INFO:teuthology.orchestra.run.smithi131.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-26T22:55:05.064 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:88:72:f9 brd ff:ff:ff:ff:ff:ff 2024-05-26T22:55:05.064 INFO:teuthology.orchestra.run.smithi131.stdout: inet 172.21.15.131/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-26T22:55:05.064 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft 42192sec preferred_lft 42192sec 2024-05-26T22:55:05.064 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::ec4:7aff:fe88:72f9/64 scope link 2024-05-26T22:55:05.064 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-05-26T22:55:05.064 INFO:teuthology.orchestra.run.smithi131.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-26T22:55:05.064 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 06:f4:05:86:a4:3e brd ff:ff:ff:ff:ff:ff 2024-05-26T22:55:05.064 INFO:teuthology.orchestra.run.smithi131.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-26T22:55:05.064 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-05-26T22:55:05.065 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::8802:8fff:fe41:fc6d/64 scope link 2024-05-26T22:55:05.065 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-05-26T22:55:05.065 INFO:teuthology.orchestra.run.smithi131.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-26T22:55:05.065 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 06:f4:05:86:a4:3e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-26T22:55:05.065 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::4f4:5ff:fe86:a43e/64 scope link 2024-05-26T22:55:05.065 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-05-26T22:55:05.065 INFO:teuthology.orchestra.run.smithi131.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-26T22:55:05.065 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether ee:6f:35:b2:c0:5b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-26T22:55:05.065 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::ec6f:35ff:feb2:c05b/64 scope link 2024-05-26T22:55:05.065 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-05-26T22:55:05.066 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:55:05.066 DEBUG:teuthology.orchestra.run.smithi131:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-26T22:55:05.143 INFO:teuthology.orchestra.run.smithi131.stdout:1 2024-05-26T22:55:05.147 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:55:05.147 DEBUG:teuthology.orchestra.run.smithi131:> route 2024-05-26T22:55:05.197 INFO:teuthology.orchestra.run.smithi131.stdout:Kernel IP routing table 2024-05-26T22:55:05.198 INFO:teuthology.orchestra.run.smithi131.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-26T22:55:05.198 INFO:teuthology.orchestra.run.smithi131.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-26T22:55:05.198 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-26T22:55:05.198 INFO:teuthology.orchestra.run.smithi131.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-26T22:55:05.198 DEBUG:teuthology.orchestra.run.smithi131:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-26T22:55:05.198 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2024-05-26T22:55:05.198 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-26T22:55:05.198 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-26T22:55:05.199 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-26T22:55:05.199 DEBUG:teuthology.orchestra.run.smithi131:> ') 2024-05-26T22:55:05.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-26T22:55:05.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-26T22:55:05.405 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:55:05.405 DEBUG:teuthology.orchestra.run.smithi131:> ip netns list 2024-05-26T22:55:05.453 INFO:teuthology.orchestra.run.smithi131.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-26T22:55:05.454 INFO:teuthology.orchestra.run.smithi131.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-26T22:55:05.454 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-26T22:55:05.454 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:55:05.454 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link set brx.0 up 2024-05-26T22:55:05.532 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-26T22:55:05.533 DEBUG:teuthology.orchestra.run.smithi131:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-26T22:55:05.584 INFO:teuthology.orchestra.run.smithi131.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-26T22:55:05.584 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-26T22:55:05.584 DEBUG:teuthology.orchestra.run.smithi131:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-26T22:55:05.636 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-05-26T22:55:05.636 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:55:05.636 DEBUG:teuthology.orchestra.run.smithi131:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-26T22:55:05.755 INFO:teuthology.orchestra.run.smithi131.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-26T22:55:05.755 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-26T22:55:05.794 INFO:teuthology.orchestra.run.smithi131.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-26T22:55:05.794 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-26T22:55:05.794 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: resolved to: "172.21.15.131:3300,172.21.15.187:3300,172.21.15.187:3301" 2024-05-26T22:55:05.794 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: trying mount with new device syntax: 0@46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f.cephfs=/ 2024-05-26T22:55:05.794 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.131:3300/172.21.15.187:3300/172.21.15.187:3301" will pass to kernel 2024-05-26T22:55:05.796 INFO:teuthology.orchestra.run.smithi131.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-26T22:55:05.796 INFO:teuthology.orchestra.run.smithi131.stdout: You just mounted a file system that supports labels which does not 2024-05-26T22:55:05.797 INFO:teuthology.orchestra.run.smithi131.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-26T22:55:05.797 INFO:teuthology.orchestra.run.smithi131.stdout: applications will generate AVC messages and not be allowed access to 2024-05-26T22:55:05.797 INFO:teuthology.orchestra.run.smithi131.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-26T22:55:05.801 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-26T22:55:05.801 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:55:05.801 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-26T22:55:05.871 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-26T22:55:05.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:05 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-26T22:55:05.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-26T22:55:06.006 INFO:teuthology.orchestra.run.smithi131.stdout:client5303 2024-05-26T22:55:06.006 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-26T22:55:06.006 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-26T22:55:06.057 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_export_ephemeral_random True 2024-05-26T22:55:06.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.274+0000 7f1a39200640 1 Processor -- start 2024-05-26T22:55:06.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.274+0000 7f1a39200640 1 -- start start 2024-05-26T22:55:06.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.276+0000 7f1a39200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f1a340712d0 0x7f1a3406d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:06.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.276+0000 7f1a39200640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f1a3406d790 0x7f1a340cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:06.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.276+0000 7f1a39200640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a34072d60 con 0x7f1a34145a10 2024-05-26T22:55:06.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.276+0000 7f1a39200640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f1a3415c760 con 0x7f1a340712d0 2024-05-26T22:55:06.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.276+0000 7f1a39200640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f1a3406dbc0 con 0x7f1a3406d790 2024-05-26T22:55:06.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.276+0000 7f1a32a00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f1a340712d0 0x7f1a3406d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:06.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.277+0000 7f1a32a00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f1a340712d0 0x7f1a3406d250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51082/0 (socket says 172.21.15.131:51082) 2024-05-26T22:55:06.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.277+0000 7f1a32a00640 1 -- 172.21.15.131:0/2572394590 learned_addr learned my addr 172.21.15.131:0/2572394590 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:55:06.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.277+0000 7f1a33e00640 1 --2- 172.21.15.131:0/2572394590 >> v2:172.21.15.187:3300/0 conn(0x7f1a3406d790 0x7f1a340cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:06.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.277+0000 7f1a32a00640 1 -- 172.21.15.131:0/2572394590 >> v2:172.21.15.187:3300/0 conn(0x7f1a3406d790 msgr2=0x7f1a340cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:06.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.277+0000 7f1a32a00640 1 --2- 172.21.15.131:0/2572394590 >> v2:172.21.15.187:3300/0 conn(0x7f1a3406d790 0x7f1a340cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:06.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.277+0000 7f1a32a00640 1 -- 172.21.15.131:0/2572394590 >> v1:172.21.15.187:6789/0 conn(0x7f1a34145a10 legacy=0x7f1a34072180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:06.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.277+0000 7f1a32a00640 1 -- 172.21.15.131:0/2572394590 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a34072a80 con 0x7f1a340712d0 2024-05-26T22:55:06.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.277+0000 7f1a33e00640 1 --2- 172.21.15.131:0/2572394590 >> v2:172.21.15.187:3300/0 conn(0x7f1a3406d790 0x7f1a340cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:55:06.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.278+0000 7f1a32a00640 1 --2- 172.21.15.131:0/2572394590 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a340712d0 0x7f1a3406d250 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f1a3415fc80 tx=0x7f1a1c02a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcdc696e61addaf3 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:06.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.278+0000 7f1a32000640 1 -- 172.21.15.131:0/2572394590 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a1c037070 con 0x7f1a340712d0 2024-05-26T22:55:06.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.279+0000 7f1a32000640 1 -- 172.21.15.131:0/2572394590 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a1c032440 con 0x7f1a340712d0 2024-05-26T22:55:06.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.279+0000 7f1a32000640 1 -- 172.21.15.131:0/2572394590 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a1c032810 con 0x7f1a340712d0 2024-05-26T22:55:06.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.279+0000 7f1a39200640 1 -- 172.21.15.131:0/2572394590 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a340712d0 msgr2=0x7f1a3406d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:06.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.279+0000 7f1a39200640 1 --2- 172.21.15.131:0/2572394590 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a340712d0 0x7f1a3406d250 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f1a3415fc80 tx=0x7f1a1c02a250 comp rx=0 tx=0).stop 2024-05-26T22:55:06.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.279+0000 7f1a39200640 1 -- 172.21.15.131:0/2572394590 shutdown_connections 2024-05-26T22:55:06.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.279+0000 7f1a39200640 1 --2- 172.21.15.131:0/2572394590 >> v2:172.21.15.187:3300/0 conn(0x7f1a3406d790 0x7f1a340cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:06.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.279+0000 7f1a39200640 1 --2- 172.21.15.131:0/2572394590 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a340712d0 0x7f1a3406d250 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:06.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.279+0000 7f1a39200640 1 -- 172.21.15.131:0/2572394590 >> 172.21.15.131:0/2572394590 conn(0x7f1a340b5660 msgr2=0x7f1a340b5a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:06.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.279+0000 7f1a39200640 1 -- 172.21.15.131:0/2572394590 shutdown_connections 2024-05-26T22:55:06.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.279+0000 7f1a39200640 1 -- 172.21.15.131:0/2572394590 wait complete. 2024-05-26T22:55:06.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.281+0000 7f1a39200640 1 Processor -- start 2024-05-26T22:55:06.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.281+0000 7f1a39200640 1 -- start start 2024-05-26T22:55:06.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.282+0000 7f1a39200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1a3406d790 0x7f1a340ae5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:06.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.282+0000 7f1a39200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1a340712d0 0x7f1a3413bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:06.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.282+0000 7f1a39200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a34145a10 0x7f1a34144050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:06.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.282+0000 7f1a39200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f1a340cf070 con 0x7f1a34145a10 2024-05-26T22:55:06.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.282+0000 7f1a39200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1a340ced90 con 0x7f1a340712d0 2024-05-26T22:55:06.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.282+0000 7f1a39200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1a34145df0 con 0x7f1a3406d790 2024-05-26T22:55:06.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.283+0000 7f1a33e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a34145a10 0x7f1a34144050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:06.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.283+0000 7f1a33e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a34145a10 0x7f1a34144050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51086/0 (socket says 172.21.15.131:51086) 2024-05-26T22:55:06.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.283+0000 7f1a33e00640 1 -- 172.21.15.131:0/3983803054 learned_addr learned my addr 172.21.15.131:0/3983803054 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:55:06.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.283+0000 7f1a32a00640 1 --2- 172.21.15.131:0/3983803054 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1a340712d0 0x7f1a3413bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:06.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.283+0000 7f1a33400640 1 --2- 172.21.15.131:0/3983803054 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1a3406d790 0x7f1a340ae5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:06.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.283+0000 7f1a33e00640 1 -- 172.21.15.131:0/3983803054 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1a3406d790 msgr2=0x7f1a340ae5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:06.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.283+0000 7f1a33e00640 1 --2- 172.21.15.131:0/3983803054 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1a3406d790 0x7f1a340ae5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:06.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.283+0000 7f1a33e00640 1 -- 172.21.15.131:0/3983803054 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1a340712d0 msgr2=0x7f1a3413bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:06.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.283+0000 7f1a33e00640 1 --2- 172.21.15.131:0/3983803054 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1a340712d0 0x7f1a3413bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:06.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.283+0000 7f1a33e00640 1 -- 172.21.15.131:0/3983803054 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a3413f190 con 0x7f1a34145a10 2024-05-26T22:55:06.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.284+0000 7f1a33e00640 1 --2- 172.21.15.131:0/3983803054 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a34145a10 0x7f1a34144050 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c00c820 tx=0x7f1a2c00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:06.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.284+0000 7f1a2b400640 1 -- 172.21.15.131:0/3983803054 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a2c015070 con 0x7f1a34145a10 2024-05-26T22:55:06.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.284+0000 7f1a2b400640 1 -- 172.21.15.131:0/3983803054 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a2c007a30 con 0x7f1a34145a10 2024-05-26T22:55:06.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.284+0000 7f1a2b400640 1 -- 172.21.15.131:0/3983803054 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a2c007e20 con 0x7f1a34145a10 2024-05-26T22:55:06.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.284+0000 7f1a39200640 1 -- 172.21.15.131:0/3983803054 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a340adac0 con 0x7f1a34145a10 2024-05-26T22:55:06.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.284+0000 7f1a39200640 1 -- 172.21.15.131:0/3983803054 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a340ae080 con 0x7f1a34145a10 2024-05-26T22:55:06.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.285+0000 7f1a39200640 1 -- 172.21.15.131:0/3983803054 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a340694e0 con 0x7f1a34145a10 2024-05-26T22:55:06.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.286+0000 7f1a2b400640 1 -- 172.21.15.131:0/3983803054 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1a2c018070 con 0x7f1a34145a10 2024-05-26T22:55:06.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.287+0000 7f1a2b400640 1 --2- 172.21.15.131:0/3983803054 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f1a1006be30 0x7f1a1006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:06.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.287+0000 7f1a2b400640 1 -- 172.21.15.131:0/3983803054 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 5709+0+0 (secure 0 0 0) 0x7f1a2c08ae20 con 0x7f1a34145a10 2024-05-26T22:55:06.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.287+0000 7f1a33400640 1 --2- 172.21.15.131:0/3983803054 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f1a1006be30 0x7f1a1006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:06.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.288+0000 7f1a33400640 1 --2- 172.21.15.131:0/3983803054 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f1a1006be30 0x7f1a1006e2f0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f1a24007dc0 tx=0x7f1a240072f0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:06.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.291+0000 7f1a2b400640 1 -- 172.21.15.131:0/3983803054 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f1a2c0597c0 con 0x7f1a34145a10 2024-05-26T22:55:06.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.496+0000 7f1a39200640 1 -- 172.21.15.131:0/3983803054 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command([{prefix=config set, name=mds_export_ephemeral_random}] v 0) -- 0x7f1a340b5ea0 con 0x7f1a34145a10 2024-05-26T22:55:06.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.502+0000 7f1a2b400640 1 -- 172.21.15.131:0/3983803054 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_export_ephemeral_random}]=0 v52)=0 v52) ==== 128+0+0 (secure 0 0 0) 0x7f1a2c0590c0 con 0x7f1a34145a10 2024-05-26T22:55:06.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.505+0000 7f1a39200640 1 -- 172.21.15.131:0/3983803054 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f1a1006be30 msgr2=0x7f1a1006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:06.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.505+0000 7f1a39200640 1 --2- 172.21.15.131:0/3983803054 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f1a1006be30 0x7f1a1006e2f0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f1a24007dc0 tx=0x7f1a240072f0 comp rx=0 tx=0).stop 2024-05-26T22:55:06.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.505+0000 7f1a39200640 1 -- 172.21.15.131:0/3983803054 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a34145a10 msgr2=0x7f1a34144050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:06.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.505+0000 7f1a39200640 1 --2- 172.21.15.131:0/3983803054 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a34145a10 0x7f1a34144050 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c00c820 tx=0x7f1a2c00cce0 comp rx=0 tx=0).stop 2024-05-26T22:55:06.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.505+0000 7f1a39200640 1 -- 172.21.15.131:0/3983803054 shutdown_connections 2024-05-26T22:55:06.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.505+0000 7f1a39200640 1 --2- 172.21.15.131:0/3983803054 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f1a1006be30 0x7f1a1006e2f0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:06.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.505+0000 7f1a39200640 1 --2- 172.21.15.131:0/3983803054 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a34145a10 0x7f1a34144050 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:06.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.505+0000 7f1a39200640 1 --2- 172.21.15.131:0/3983803054 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1a340712d0 0x7f1a3413bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:06.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.505+0000 7f1a39200640 1 --2- 172.21.15.131:0/3983803054 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1a3406d790 0x7f1a340ae5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:06.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.505+0000 7f1a39200640 1 -- 172.21.15.131:0/3983803054 >> 172.21.15.131:0/3983803054 conn(0x7f1a340b5660 msgr2=0x7f1a340d0480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:06.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.506+0000 7f1a39200640 1 -- 172.21.15.131:0/3983803054 shutdown_connections 2024-05-26T22:55:06.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.506+0000 7f1a39200640 1 -- 172.21.15.131:0/3983803054 wait complete. 2024-05-26T22:55:06.526 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_export_ephemeral_distributed True 2024-05-26T22:55:06.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.741+0000 7f0f3ee00640 1 Processor -- start 2024-05-26T22:55:06.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.742+0000 7f0f3ee00640 1 -- start start 2024-05-26T22:55:06.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.743+0000 7f0f3ee00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f0f3806f180 0x7f0f38072e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:06.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.743+0000 7f0f3ee00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f0f38072540 0x7f0f38161ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:06.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.743+0000 7f0f3ee00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f380af8e0 con 0x7f0f380cfe50 2024-05-26T22:55:06.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.743+0000 7f0f3ee00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f0f380cfb30 con 0x7f0f3806f180 2024-05-26T22:55:06.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.743+0000 7f0f3ee00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f0f380cf200 con 0x7f0f38072540 2024-05-26T22:55:06.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.744+0000 7f0f37e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f0f3806f180 0x7f0f38072e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:06.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.744+0000 7f0f37e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f0f3806f180 0x7f0f38072e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51102/0 (socket says 172.21.15.131:51102) 2024-05-26T22:55:06.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.744+0000 7f0f37e00640 1 -- 172.21.15.131:0/2874082384 learned_addr learned my addr 172.21.15.131:0/2874082384 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:55:06.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.744+0000 7f0f37400640 1 --2- 172.21.15.131:0/2874082384 >> v2:172.21.15.187:3300/0 conn(0x7f0f38072540 0x7f0f38161ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:06.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.744+0000 7f0f37e00640 1 -- 172.21.15.131:0/2874082384 >> v2:172.21.15.187:3300/0 conn(0x7f0f38072540 msgr2=0x7f0f38161ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:06.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.744+0000 7f0f37e00640 1 --2- 172.21.15.131:0/2874082384 >> v2:172.21.15.187:3300/0 conn(0x7f0f38072540 0x7f0f38161ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:06.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.744+0000 7f0f37e00640 1 -- 172.21.15.131:0/2874082384 >> v1:172.21.15.187:6789/0 conn(0x7f0f380cfe50 legacy=0x7f0f380715a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:06.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.744+0000 7f0f37e00640 1 -- 172.21.15.131:0/2874082384 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f380cf4e0 con 0x7f0f3806f180 2024-05-26T22:55:06.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.744+0000 7f0f37400640 1 --2- 172.21.15.131:0/2874082384 >> v2:172.21.15.187:3300/0 conn(0x7f0f38072540 0x7f0f38161ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:55:06.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.746+0000 7f0f37e00640 1 --2- 172.21.15.131:0/2874082384 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0f3806f180 0x7f0f38072e90 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f0f380d0c40 tx=0x7f0f2802a6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a911599cbfe1b759 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:06.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.746+0000 7f0f36a00640 1 -- 172.21.15.131:0/2874082384 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f28036070 con 0x7f0f3806f180 2024-05-26T22:55:06.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.746+0000 7f0f36a00640 1 -- 172.21.15.131:0/2874082384 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f28031600 con 0x7f0f3806f180 2024-05-26T22:55:06.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.746+0000 7f0f36a00640 1 -- 172.21.15.131:0/2874082384 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f280319d0 con 0x7f0f3806f180 2024-05-26T22:55:06.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.747+0000 7f0f3ee00640 1 -- 172.21.15.131:0/2874082384 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0f3806f180 msgr2=0x7f0f38072e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:06.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.747+0000 7f0f3ee00640 1 --2- 172.21.15.131:0/2874082384 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0f3806f180 0x7f0f38072e90 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f0f380d0c40 tx=0x7f0f2802a6b0 comp rx=0 tx=0).stop 2024-05-26T22:55:06.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.747+0000 7f0f3ee00640 1 -- 172.21.15.131:0/2874082384 shutdown_connections 2024-05-26T22:55:06.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.747+0000 7f0f3ee00640 1 --2- 172.21.15.131:0/2874082384 >> v2:172.21.15.187:3300/0 conn(0x7f0f38072540 0x7f0f38161ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:06.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.747+0000 7f0f3ee00640 1 --2- 172.21.15.131:0/2874082384 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0f3806f180 0x7f0f38072e90 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:06.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.747+0000 7f0f3ee00640 1 -- 172.21.15.131:0/2874082384 >> 172.21.15.131:0/2874082384 conn(0x7f0f3806bc90 msgr2=0x7f0f380b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:06.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.747+0000 7f0f3ee00640 1 -- 172.21.15.131:0/2874082384 shutdown_connections 2024-05-26T22:55:06.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.747+0000 7f0f3ee00640 1 -- 172.21.15.131:0/2874082384 wait complete. 2024-05-26T22:55:06.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.749+0000 7f0f3ee00640 1 Processor -- start 2024-05-26T22:55:06.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.749+0000 7f0f3ee00640 1 -- start start 2024-05-26T22:55:06.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.750+0000 7f0f3ee00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0f3806f180 0x7f0f3814c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:06.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.750+0000 7f0f3ee00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0f38072540 0x7f0f381492f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:06.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.750+0000 7f0f3ee00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0f380cfe50 0x7f0f381604e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:06.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.750+0000 7f0f3ee00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f0f3806f6e0 con 0x7f0f3806f180 2024-05-26T22:55:06.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.750+0000 7f0f3ee00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0f3806f560 con 0x7f0f380cfe50 2024-05-26T22:55:06.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.750+0000 7f0f3ee00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f0f38071cb0 con 0x7f0f38072540 2024-05-26T22:55:06.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.750+0000 7f0f37e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0f3806f180 0x7f0f3814c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:06.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.750+0000 7f0f37e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0f3806f180 0x7f0f3814c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51114/0 (socket says 172.21.15.131:51114) 2024-05-26T22:55:06.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.750+0000 7f0f37e00640 1 -- 172.21.15.131:0/1096447707 learned_addr learned my addr 172.21.15.131:0/1096447707 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:55:06.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.750+0000 7f0f37e00640 1 -- 172.21.15.131:0/1096447707 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0f38072540 msgr2=0x7f0f381492f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:06.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.750+0000 7f0f37e00640 1 --2- 172.21.15.131:0/1096447707 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0f38072540 0x7f0f381492f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:06.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.750+0000 7f0f37e00640 1 -- 172.21.15.131:0/1096447707 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0f380cfe50 msgr2=0x7f0f381604e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:06.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.750+0000 7f0f37e00640 1 --2- 172.21.15.131:0/1096447707 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0f380cfe50 0x7f0f381604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:06.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.750+0000 7f0f37e00640 1 -- 172.21.15.131:0/1096447707 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f38160df0 con 0x7f0f3806f180 2024-05-26T22:55:06.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.751+0000 7f0f37e00640 1 --2- 172.21.15.131:0/1096447707 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0f3806f180 0x7f0f3814c5c0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f0f28034040 tx=0x7f0f28004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:06.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.751+0000 7f0f2fe00640 1 -- 172.21.15.131:0/1096447707 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f28036050 con 0x7f0f3806f180 2024-05-26T22:55:06.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.751+0000 7f0f2fe00640 1 -- 172.21.15.131:0/1096447707 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f28042070 con 0x7f0f3806f180 2024-05-26T22:55:06.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.751+0000 7f0f2fe00640 1 -- 172.21.15.131:0/1096447707 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f28009510 con 0x7f0f3806f180 2024-05-26T22:55:06.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.751+0000 7f0f3ee00640 1 -- 172.21.15.131:0/1096447707 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f381610d0 con 0x7f0f3806f180 2024-05-26T22:55:06.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.751+0000 7f0f3ee00640 1 -- 172.21.15.131:0/1096447707 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f38161690 con 0x7f0f3806f180 2024-05-26T22:55:06.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.752+0000 7f0f3ee00640 1 -- 172.21.15.131:0/1096447707 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f380695c0 con 0x7f0f3806f180 2024-05-26T22:55:06.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.753+0000 7f0f2fe00640 1 -- 172.21.15.131:0/1096447707 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0f28007c50 con 0x7f0f3806f180 2024-05-26T22:55:06.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.754+0000 7f0f2fe00640 1 --2- 172.21.15.131:0/1096447707 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f0f1406be30 0x7f0f1406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:06.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.754+0000 7f0f2fe00640 1 -- 172.21.15.131:0/1096447707 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 5709+0+0 (secure 0 0 0) 0x7f0f280abef0 con 0x7f0f3806f180 2024-05-26T22:55:06.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.754+0000 7f0f37400640 1 --2- 172.21.15.131:0/1096447707 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f0f1406be30 0x7f0f1406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:06.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.755+0000 7f0f37400640 1 --2- 172.21.15.131:0/1096447707 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f0f1406be30 0x7f0f1406e2f0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f0f20006fd0 tx=0x7f0f20008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:06.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.759+0000 7f0f2fe00640 1 -- 172.21.15.131:0/1096447707 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f0f2807a800 con 0x7f0f3806f180 2024-05-26T22:55:06.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.957+0000 7f0f3ee00640 1 -- 172.21.15.131:0/1096447707 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command([{prefix=config set, name=mds_export_ephemeral_distributed}] v 0) -- 0x7f0f38071990 con 0x7f0f3806f180 2024-05-26T22:55:06.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.965+0000 7f0f2fe00640 1 -- 172.21.15.131:0/1096447707 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_export_ephemeral_distributed}]=0 v53)=0 v53) ==== 133+0+0 (secure 0 0 0) 0x7f0f2807a100 con 0x7f0f3806f180 2024-05-26T22:55:06.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.967+0000 7f0f3ee00640 1 -- 172.21.15.131:0/1096447707 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f0f1406be30 msgr2=0x7f0f1406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:06.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.967+0000 7f0f3ee00640 1 --2- 172.21.15.131:0/1096447707 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f0f1406be30 0x7f0f1406e2f0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f0f20006fd0 tx=0x7f0f20008040 comp rx=0 tx=0).stop 2024-05-26T22:55:06.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.968+0000 7f0f3ee00640 1 -- 172.21.15.131:0/1096447707 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0f3806f180 msgr2=0x7f0f3814c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:06.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.968+0000 7f0f3ee00640 1 --2- 172.21.15.131:0/1096447707 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0f3806f180 0x7f0f3814c5c0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f0f28034040 tx=0x7f0f28004a40 comp rx=0 tx=0).stop 2024-05-26T22:55:06.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.968+0000 7f0f3ee00640 1 -- 172.21.15.131:0/1096447707 shutdown_connections 2024-05-26T22:55:06.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.968+0000 7f0f3ee00640 1 --2- 172.21.15.131:0/1096447707 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0f38072540 0x7f0f381492f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:06.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.968+0000 7f0f3ee00640 1 --2- 172.21.15.131:0/1096447707 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f0f1406be30 0x7f0f1406e2f0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:06.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.968+0000 7f0f3ee00640 1 --2- 172.21.15.131:0/1096447707 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0f380cfe50 0x7f0f381604e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:06.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.968+0000 7f0f3ee00640 1 --2- 172.21.15.131:0/1096447707 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0f3806f180 0x7f0f3814c5c0 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:06.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.968+0000 7f0f3ee00640 1 -- 172.21.15.131:0/1096447707 >> 172.21.15.131:0/1096447707 conn(0x7f0f3806bc90 msgr2=0x7f0f380ad430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:06.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.968+0000 7f0f3ee00640 1 -- 172.21.15.131:0/1096447707 shutdown_connections 2024-05-26T22:55:06.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:06.968+0000 7f0f3ee00640 1 -- 172.21.15.131:0/1096447707 wait complete. 2024-05-26T22:55:06.987 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_export_ephemeral_random_max 1.0 2024-05-26T22:55:07.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:07.190+0000 7f776e000640 1 Processor -- start 2024-05-26T22:55:07.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:07.190+0000 7f776e000640 1 -- start start 2024-05-26T22:55:07.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:07.192+0000 7f776e000640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f7768155770 0x7f776815b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:07.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:07.192+0000 7f776e000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f776815bc40 0x7f776806dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:07.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:07.192+0000 7f776e000640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77680735b0 con 0x7f776806e270 2024-05-26T22:55:07.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:07.192+0000 7f776e000640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f776806e6a0 con 0x7f7768155770 2024-05-26T22:55:07.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:07.192+0000 7f776e000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f776815c1c0 con 0x7f776815bc40 2024-05-26T22:55:07.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:07.192+0000 7f7767400640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f7768155770 0x7f776815b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:07.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:07.192+0000 7f7766a00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f776815bc40 0x7f776806dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:07.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:07.192+0000 7f7767400640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f7768155770 0x7f776815b700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.131:60596/0 (socket says 172.21.15.131:60596) 2024-05-26T22:55:07.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:07.192+0000 7f7767400640 1 -- 172.21.15.131:0/292837217 learned_addr learned my addr 172.21.15.131:0/292837217 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:55:07.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:07.193+0000 7f7766a00640 1 -- 172.21.15.131:0/292837217 >> v2:172.21.15.187:3300/0 conn(0x7f7768155770 msgr2=0x7f776815b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:07.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:07.193+0000 7f7766a00640 1 --2- 172.21.15.131:0/292837217 >> v2:172.21.15.187:3300/0 conn(0x7f7768155770 0x7f776815b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:07.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:07.193+0000 7f7766a00640 1 -- 172.21.15.131:0/292837217 >> v1:172.21.15.187:6789/0 conn(0x7f776806e270 legacy=0x7f77680ceba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:07.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:07.193+0000 7f7766a00640 1 -- 172.21.15.131:0/292837217 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f776815c4a0 con 0x7f776815bc40 2024-05-26T22:55:07.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:07.194+0000 7f7766a00640 1 --2- 172.21.15.131:0/292837217 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f776815bc40 0x7f776806dd30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f77540097a0 tx=0x7f775402a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91879b5775b30e9d server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:07.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:07.195+0000 7f7766000640 1 -- 172.21.15.131:0/292837217 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7754037070 con 0x7f776815bc40 2024-05-26T22:55:07.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:07.195+0000 7f7766000640 1 -- 172.21.15.131:0/292837217 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7754032440 con 0x7f776815bc40 2024-05-26T22:55:07.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:07.195+0000 7f7766000640 1 -- 172.21.15.131:0/292837217 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7754032810 con 0x7f776815bc40 2024-05-26T22:55:07.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:07.196+0000 7f776e000640 1 -- 172.21.15.131:0/292837217 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f776815bc40 msgr2=0x7f776806dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:07.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:07.196+0000 7f776e000640 1 --2- 172.21.15.131:0/292837217 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f776815bc40 0x7f776806dd30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f77540097a0 tx=0x7f775402a210 comp rx=0 tx=0).stop 2024-05-26T22:55:07.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:07.196+0000 7f776e000640 1 -- 172.21.15.131:0/292837217 shutdown_connections 2024-05-26T22:55:07.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:07.196+0000 7f776e000640 1 --2- 172.21.15.131:0/292837217 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f776815bc40 0x7f776806dd30 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:07.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:07.196+0000 7f776e000640 1 --2- 172.21.15.131:0/292837217 >> v2:172.21.15.187:3300/0 conn(0x7f7768155770 0x7f776815b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:07.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:07.196+0000 7f776e000640 1 -- 172.21.15.131:0/292837217 >> 172.21.15.131:0/292837217 conn(0x7f776806bc90 msgr2=0x7f77680b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:07.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:07.196+0000 7f776e000640 1 -- 172.21.15.131:0/292837217 shutdown_connections 2024-05-26T22:55:07.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:07.196+0000 7f776e000640 1 -- 172.21.15.131:0/292837217 wait complete. 2024-05-26T22:55:07.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:07.197+0000 7f776e000640 1 Processor -- start 2024-05-26T22:55:07.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:07.198+0000 7f776e000640 1 -- start start 2024-05-26T22:55:07.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:07.199+0000 7f776e000640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f776806e270 0x7f77681e41b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:07.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:07.199+0000 7f776e000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7768155770 0x7f77681f2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:07.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:07.199+0000 7f776e000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f776815bc40 0x7f7768154990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:07.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:07.199+0000 7f776e000640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f7768155b50 con 0x7f776815bc40 2024-05-26T22:55:07.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:07.199+0000 7f776e000640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7768072080 con 0x7f776806e270 2024-05-26T22:55:07.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:07.199+0000 7f776e000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f776815c8c0 con 0x7f7768155770 2024-05-26T22:55:07.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:07.199+0000 7f7767e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f776815bc40 0x7f7768154990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:07.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:07.199+0000 7f7767e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f776815bc40 0x7f7768154990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51120/0 (socket says 172.21.15.131:51120) 2024-05-26T22:55:07.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:07.199+0000 7f7767e00640 1 -- 172.21.15.131:0/313379381 learned_addr learned my addr 172.21.15.131:0/313379381 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:55:07.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:07.199+0000 7f7767400640 1 --2- 172.21.15.131:0/313379381 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f776806e270 0x7f77681e41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:07.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:07.199+0000 7f7766a00640 1 --2- 172.21.15.131:0/313379381 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7768155770 0x7f77681f2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:07.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:07.200+0000 7f7767e00640 1 -- 172.21.15.131:0/313379381 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7768155770 msgr2=0x7f77681f2b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:07.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:07.200+0000 7f7767e00640 1 --2- 172.21.15.131:0/313379381 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7768155770 0x7f77681f2b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:07.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:07.200+0000 7f7767e00640 1 -- 172.21.15.131:0/313379381 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f776806e270 msgr2=0x7f77681e41b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:07.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:07.200+0000 7f7767e00640 1 --2- 172.21.15.131:0/313379381 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f776806e270 0x7f77681e41b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:07.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:07.200+0000 7f7767e00640 1 -- 172.21.15.131:0/313379381 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77681e48b0 con 0x7f776815bc40 2024-05-26T22:55:07.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:07.200+0000 7f7767400640 1 --2- 172.21.15.131:0/313379381 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f776806e270 0x7f77681e41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-26T22:55:07.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:07.200+0000 7f7767e00640 1 --2- 172.21.15.131:0/313379381 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f776815bc40 0x7f7768154990 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f776000c760 tx=0x7f776000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:07.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:07.200+0000 7f775f400640 1 -- 172.21.15.131:0/313379381 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7760003b90 con 0x7f776815bc40 2024-05-26T22:55:07.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:07.200+0000 7f775f400640 1 -- 172.21.15.131:0/313379381 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7760003d30 con 0x7f776815bc40 2024-05-26T22:55:07.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:07.200+0000 7f775f400640 1 -- 172.21.15.131:0/313379381 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f776000aa90 con 0x7f776815bc40 2024-05-26T22:55:07.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:07.200+0000 7f776e000640 1 -- 172.21.15.131:0/313379381 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77681e4b90 con 0x7f776815bc40 2024-05-26T22:55:07.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:07.200+0000 7f776e000640 1 -- 172.21.15.131:0/313379381 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f776809bb80 con 0x7f776815bc40 2024-05-26T22:55:07.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:07.201+0000 7f776e000640 1 -- 172.21.15.131:0/313379381 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7768074e90 con 0x7f776815bc40 2024-05-26T22:55:07.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:07.203+0000 7f775f400640 1 -- 172.21.15.131:0/313379381 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f776000ac30 con 0x7f776815bc40 2024-05-26T22:55:07.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:07.203+0000 7f775f400640 1 --2- 172.21.15.131:0/313379381 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f774806be30 0x7f774806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:07.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:07.204+0000 7f775f400640 1 -- 172.21.15.131:0/313379381 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 5709+0+0 (secure 0 0 0) 0x7f776008ad60 con 0x7f776815bc40 2024-05-26T22:55:07.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:07.204+0000 7f7767400640 1 --2- 172.21.15.131:0/313379381 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f774806be30 0x7f774806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:07.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:07.204+0000 7f7767400640 1 --2- 172.21.15.131:0/313379381 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f774806be30 0x7f774806e2f0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f7758009c20 tx=0x7f7758008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:07.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:07.208+0000 7f775f400640 1 -- 172.21.15.131:0/313379381 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f7760059700 con 0x7f776815bc40 2024-05-26T22:55:07.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:07.411+0000 7f776e000640 1 -- 172.21.15.131:0/313379381 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command([{prefix=config set, name=mds_export_ephemeral_random_max}] v 0) -- 0x7f77680695c0 con 0x7f776815bc40 2024-05-26T22:55:07.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:07.418+0000 7f775f400640 1 -- 172.21.15.131:0/313379381 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_export_ephemeral_random_max}]=0 v54)=0 v54) ==== 131+0+0 (secure 0 0 0) 0x7f7760059000 con 0x7f776815bc40 2024-05-26T22:55:07.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:07.421+0000 7f776e000640 1 -- 172.21.15.131:0/313379381 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f774806be30 msgr2=0x7f774806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:07.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:07.421+0000 7f776e000640 1 --2- 172.21.15.131:0/313379381 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f774806be30 0x7f774806e2f0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f7758009c20 tx=0x7f7758008040 comp rx=0 tx=0).stop 2024-05-26T22:55:07.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:07.422+0000 7f776e000640 1 -- 172.21.15.131:0/313379381 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f776815bc40 msgr2=0x7f7768154990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:07.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:07.422+0000 7f776e000640 1 --2- 172.21.15.131:0/313379381 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f776815bc40 0x7f7768154990 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f776000c760 tx=0x7f776000cc20 comp rx=0 tx=0).stop 2024-05-26T22:55:07.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:07.422+0000 7f776e000640 1 -- 172.21.15.131:0/313379381 shutdown_connections 2024-05-26T22:55:07.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:07.422+0000 7f776e000640 1 --2- 172.21.15.131:0/313379381 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7768155770 0x7f77681f2b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:07.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:07.422+0000 7f776e000640 1 --2- 172.21.15.131:0/313379381 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f774806be30 0x7f774806e2f0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:07.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:07.422+0000 7f776e000640 1 --2- 172.21.15.131:0/313379381 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f776815bc40 0x7f7768154990 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:07.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:07.422+0000 7f776e000640 1 --2- 172.21.15.131:0/313379381 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f776806e270 0x7f77681e41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:07.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:07.422+0000 7f776e000640 1 -- 172.21.15.131:0/313379381 >> 172.21.15.131:0/313379381 conn(0x7f776806bc90 msgr2=0x7f776806d320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:07.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:07.422+0000 7f776e000640 1 -- 172.21.15.131:0/313379381 shutdown_connections 2024-05-26T22:55:07.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:07.422+0000 7f776e000640 1 -- 172.21.15.131:0/313379381 wait complete. 2024-05-26T22:55:07.441 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-05-26T22:55:07.441 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2024-05-26T22:55:07.441 DEBUG:teuthology.orchestra.run.smithi131:> 2024-05-26T22:55:07.441 DEBUG:teuthology.orchestra.run.smithi131:> # Use up a random number of inode numbers so the ephemeral pinning is not the same every test. 2024-05-26T22:55:07.441 DEBUG:teuthology.orchestra.run.smithi131:> mkdir .inode_number_thrash 2024-05-26T22:55:07.441 DEBUG:teuthology.orchestra.run.smithi131:> count=$((RANDOM % 1024)) 2024-05-26T22:55:07.441 DEBUG:teuthology.orchestra.run.smithi131:> for ((i = 0; i < count; i++)); do touch .inode_number_thrash/$i; done 2024-05-26T22:55:07.441 DEBUG:teuthology.orchestra.run.smithi131:> rm -rf .inode_number_thrash 2024-05-26T22:55:07.441 DEBUG:teuthology.orchestra.run.smithi131:> ') 2024-05-26T22:55:07.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:07 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-26T22:55:11.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-26T22:55:11.447 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2024-05-26T22:55:11.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:11.645+0000 7f71f3200640 1 Processor -- start 2024-05-26T22:55:11.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:11.645+0000 7f71f3200640 1 -- start start 2024-05-26T22:55:11.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:11.647+0000 7f71f3200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f71ec0b5450 0x7f71ec0b43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:11.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:11.647+0000 7f71f3200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f71ec06f420 0x7f71ec0cfd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:11.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:11.647+0000 7f71f3200640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f71ec0cf590 0x7f71ec0d0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:11.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:11.647+0000 7f71f3200640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f71ec160d90 con 0x7f71ec0b5450 2024-05-26T22:55:11.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:11.647+0000 7f71f3200640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f71ec0b4900 con 0x7f71ec0cf590 2024-05-26T22:55:11.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:11.647+0000 7f71f3200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f71ec160180 con 0x7f71ec06f420 2024-05-26T22:55:11.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:11.647+0000 7f71ebe00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f71ec0b5450 0x7f71ec0b43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:11.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:11.647+0000 7f71ebe00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f71ec0b5450 0x7f71ec0b43c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51126/0 (socket says 172.21.15.131:51126) 2024-05-26T22:55:11.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:11.647+0000 7f71ebe00640 1 -- 172.21.15.131:0/4230250363 learned_addr learned my addr 172.21.15.131:0/4230250363 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:55:11.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:11.648+0000 7f71f1200640 1 --2- 172.21.15.131:0/4230250363 >> v2:172.21.15.187:3300/0 conn(0x7f71ec0cf590 0x7f71ec0d0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:11.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:11.648+0000 7f71eb400640 1 --2- 172.21.15.131:0/4230250363 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f71ec06f420 0x7f71ec0cfd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:11.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:11.648+0000 7f71ebe00640 1 -- 172.21.15.131:0/4230250363 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f71ec06f420 msgr2=0x7f71ec0cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:11.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:11.648+0000 7f71ebe00640 1 --2- 172.21.15.131:0/4230250363 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f71ec06f420 0x7f71ec0cfd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:11.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:11.648+0000 7f71ebe00640 1 -- 172.21.15.131:0/4230250363 >> v2:172.21.15.187:3300/0 conn(0x7f71ec0cf590 msgr2=0x7f71ec0d0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:11.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:11.648+0000 7f71ebe00640 1 --2- 172.21.15.131:0/4230250363 >> v2:172.21.15.187:3300/0 conn(0x7f71ec0cf590 0x7f71ec0d0290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:11.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:11.648+0000 7f71ebe00640 1 -- 172.21.15.131:0/4230250363 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71ec160460 con 0x7f71ec0b5450 2024-05-26T22:55:11.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:11.648+0000 7f71f1200640 1 --2- 172.21.15.131:0/4230250363 >> v2:172.21.15.187:3300/0 conn(0x7f71ec0cf590 0x7f71ec0d0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:55:11.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:11.648+0000 7f71eb400640 1 --2- 172.21.15.131:0/4230250363 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f71ec06f420 0x7f71ec0cfd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:55:11.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:11.649+0000 7f71ebe00640 1 --2- 172.21.15.131:0/4230250363 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71ec0b5450 0x7f71ec0b43c0 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f71ec15fd40 tx=0x7f71e402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8d964a7ad5bdb5d server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:11.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:11.650+0000 7f71eaa00640 1 -- 172.21.15.131:0/4230250363 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e4037070 con 0x7f71ec0b5450 2024-05-26T22:55:11.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:11.650+0000 7f71eaa00640 1 -- 172.21.15.131:0/4230250363 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71e4032440 con 0x7f71ec0b5450 2024-05-26T22:55:11.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:11.650+0000 7f71eaa00640 1 -- 172.21.15.131:0/4230250363 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e4032810 con 0x7f71ec0b5450 2024-05-26T22:55:11.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:11.650+0000 7f71f3200640 1 -- 172.21.15.131:0/4230250363 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71ec0b5450 msgr2=0x7f71ec0b43c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:11.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:11.650+0000 7f71f3200640 1 --2- 172.21.15.131:0/4230250363 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71ec0b5450 0x7f71ec0b43c0 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f71ec15fd40 tx=0x7f71e402a1e0 comp rx=0 tx=0).stop 2024-05-26T22:55:11.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:11.650+0000 7f71f3200640 1 -- 172.21.15.131:0/4230250363 shutdown_connections 2024-05-26T22:55:11.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:11.650+0000 7f71f3200640 1 --2- 172.21.15.131:0/4230250363 >> v2:172.21.15.187:3300/0 conn(0x7f71ec0cf590 0x7f71ec0d0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:11.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:11.650+0000 7f71f3200640 1 --2- 172.21.15.131:0/4230250363 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f71ec06f420 0x7f71ec0cfd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:11.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:11.650+0000 7f71f3200640 1 --2- 172.21.15.131:0/4230250363 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71ec0b5450 0x7f71ec0b43c0 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:11.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:11.650+0000 7f71f3200640 1 -- 172.21.15.131:0/4230250363 >> 172.21.15.131:0/4230250363 conn(0x7f71ec06bc90 msgr2=0x7f71ec0b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:11.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:11.650+0000 7f71f3200640 1 -- 172.21.15.131:0/4230250363 shutdown_connections 2024-05-26T22:55:11.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:11.650+0000 7f71f3200640 1 -- 172.21.15.131:0/4230250363 wait complete. 2024-05-26T22:55:11.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:11.652+0000 7f71f3200640 1 Processor -- start 2024-05-26T22:55:11.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:11.652+0000 7f71f3200640 1 -- start start 2024-05-26T22:55:11.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:11.653+0000 7f71f3200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71ec06f420 0x7f71ec152a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:11.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:11.653+0000 7f71f3200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f71ec0b5450 0x7f71ec1576f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:11.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:11.653+0000 7f71f3200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f71ec0cf590 0x7f71ec155b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:11.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:11.653+0000 7f71f3200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f71ec0b5830 con 0x7f71ec06f420 2024-05-26T22:55:11.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:11.653+0000 7f71f3200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f71ec160a20 con 0x7f71ec0cf590 2024-05-26T22:55:11.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:11.653+0000 7f71f3200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f71ec15fde0 con 0x7f71ec0b5450 2024-05-26T22:55:11.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:11.653+0000 7f71ebe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71ec06f420 0x7f71ec152a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:11.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:11.653+0000 7f71ebe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71ec06f420 0x7f71ec152a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51134/0 (socket says 172.21.15.131:51134) 2024-05-26T22:55:11.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:11.653+0000 7f71ebe00640 1 -- 172.21.15.131:0/2504377034 learned_addr learned my addr 172.21.15.131:0/2504377034 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:55:11.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:11.654+0000 7f71ebe00640 1 -- 172.21.15.131:0/2504377034 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f71ec0b5450 msgr2=0x7f71ec1576f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:11.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:11.654+0000 7f71ebe00640 1 --2- 172.21.15.131:0/2504377034 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f71ec0b5450 0x7f71ec1576f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:11.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:11.654+0000 7f71f1200640 1 --2- 172.21.15.131:0/2504377034 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f71ec0cf590 0x7f71ec155b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:11.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:11.654+0000 7f71ebe00640 1 -- 172.21.15.131:0/2504377034 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f71ec0cf590 msgr2=0x7f71ec155b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:11.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:11.654+0000 7f71ebe00640 1 --2- 172.21.15.131:0/2504377034 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f71ec0cf590 0x7f71ec155b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:11.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:11.654+0000 7f71ebe00640 1 -- 172.21.15.131:0/2504377034 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71ec157c30 con 0x7f71ec06f420 2024-05-26T22:55:11.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:11.654+0000 7f71f1200640 1 --2- 172.21.15.131:0/2504377034 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f71ec0cf590 0x7f71ec155b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:55:11.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:11.654+0000 7f71ebe00640 1 --2- 172.21.15.131:0/2504377034 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71ec06f420 0x7f71ec152a90 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f71e402a220 tx=0x7f71e40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:11.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:11.655+0000 7f71e3e00640 1 -- 172.21.15.131:0/2504377034 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e4037040 con 0x7f71ec06f420 2024-05-26T22:55:11.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:11.655+0000 7f71e3e00640 1 -- 172.21.15.131:0/2504377034 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71e4042070 con 0x7f71ec06f420 2024-05-26T22:55:11.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:11.655+0000 7f71e3e00640 1 -- 172.21.15.131:0/2504377034 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e4030bb0 con 0x7f71ec06f420 2024-05-26T22:55:11.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:11.655+0000 7f71f3200640 1 -- 172.21.15.131:0/2504377034 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71ec09ced0 con 0x7f71ec06f420 2024-05-26T22:55:11.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:11.655+0000 7f71f3200640 1 -- 172.21.15.131:0/2504377034 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71ec153b00 con 0x7f71ec06f420 2024-05-26T22:55:11.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:11.656+0000 7f71f3200640 1 -- 172.21.15.131:0/2504377034 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71ec074e90 con 0x7f71ec06f420 2024-05-26T22:55:11.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:11.657+0000 7f71e3e00640 1 -- 172.21.15.131:0/2504377034 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f71e4047460 con 0x7f71ec06f420 2024-05-26T22:55:11.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:11.658+0000 7f71e3e00640 1 --2- 172.21.15.131:0/2504377034 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f71c806be60 0x7f71c806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:11.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:11.658+0000 7f71e3e00640 1 -- 172.21.15.131:0/2504377034 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 5709+0+0 (secure 0 0 0) 0x7f71e40abb20 con 0x7f71ec06f420 2024-05-26T22:55:11.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:11.658+0000 7f71eb400640 1 --2- 172.21.15.131:0/2504377034 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f71c806be60 0x7f71c806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:11.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:11.659+0000 7f71eb400640 1 --2- 172.21.15.131:0/2504377034 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f71c806be60 0x7f71c806e320 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f71d4007130 tx=0x7f71d4007300 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:11.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:11.663+0000 7f71e3e00640 1 -- 172.21.15.131:0/2504377034 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f71e4045360 con 0x7f71ec06f420 2024-05-26T22:55:11.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:11.900+0000 7f71f3200640 1 -- 172.21.15.131:0/2504377034 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7f71ec0695c0 con 0x7f71ec06f420 2024-05-26T22:55:12.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:12.158+0000 7f71e3e00640 1 -- 172.21.15.131:0/2504377034 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v95) ==== 105+0+0 (secure 0 0 0) 0x7f71e407a430 con 0x7f71ec06f420 2024-05-26T22:55:12.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:12.161+0000 7f71f3200640 1 -- 172.21.15.131:0/2504377034 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f71c806be60 msgr2=0x7f71c806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:12.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:12.161+0000 7f71f3200640 1 --2- 172.21.15.131:0/2504377034 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f71c806be60 0x7f71c806e320 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f71d4007130 tx=0x7f71d4007300 comp rx=0 tx=0).stop 2024-05-26T22:55:12.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:12.161+0000 7f71f3200640 1 -- 172.21.15.131:0/2504377034 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71ec06f420 msgr2=0x7f71ec152a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:12.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:12.161+0000 7f71f3200640 1 --2- 172.21.15.131:0/2504377034 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71ec06f420 0x7f71ec152a90 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f71e402a220 tx=0x7f71e40046c0 comp rx=0 tx=0).stop 2024-05-26T22:55:12.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:12.161+0000 7f71f3200640 1 -- 172.21.15.131:0/2504377034 shutdown_connections 2024-05-26T22:55:12.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:12.161+0000 7f71f3200640 1 --2- 172.21.15.131:0/2504377034 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f71ec0b5450 0x7f71ec1576f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:12.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:12.161+0000 7f71f3200640 1 --2- 172.21.15.131:0/2504377034 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f71c806be60 0x7f71c806e320 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:12.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:12.161+0000 7f71f3200640 1 --2- 172.21.15.131:0/2504377034 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f71ec0cf590 0x7f71ec155b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:12.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:12.161+0000 7f71f3200640 1 --2- 172.21.15.131:0/2504377034 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71ec06f420 0x7f71ec152a90 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:12.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:12.161+0000 7f71f3200640 1 -- 172.21.15.131:0/2504377034 >> 172.21.15.131:0/2504377034 conn(0x7f71ec06bc90 msgr2=0x7f71ec0cd870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:12.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:12.161+0000 7f71f3200640 1 -- 172.21.15.131:0/2504377034 shutdown_connections 2024-05-26T22:55:12.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:12.161+0000 7f71f3200640 1 -- 172.21.15.131:0/2504377034 wait complete. 2024-05-26T22:55:12.183 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-26T22:55:12.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:12.352+0000 7fc50f200640 1 Processor -- start 2024-05-26T22:55:12.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:12.352+0000 7fc50f200640 1 -- start start 2024-05-26T22:55:12.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:12.354+0000 7fc50f200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fc508156750 0x7fc50815fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:12.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:12.354+0000 7fc50f200640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fc5080cc7f0 0x7fc5080ccc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:12.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:12.354+0000 7fc50f200640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc508160620 con 0x7fc50806e8c0 2024-05-26T22:55:12.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:12.354+0000 7fc50f200640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fc50815d770 con 0x7fc508156750 2024-05-26T22:55:12.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:12.354+0000 7fc50f200640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fc5080cd1a0 con 0x7fc5080cc7f0 2024-05-26T22:55:12.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:12.354+0000 7fc50de00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fc508156750 0x7fc50815fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:12.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:12.354+0000 7fc50de00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fc508156750 0x7fc50815fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51146/0 (socket says 172.21.15.131:51146) 2024-05-26T22:55:12.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:12.354+0000 7fc50de00640 1 -- 172.21.15.131:0/4009413004 learned_addr learned my addr 172.21.15.131:0/4009413004 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:55:12.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:12.355+0000 7fc50de00640 1 -- 172.21.15.131:0/4009413004 >> v2:172.21.15.187:3300/0 conn(0x7fc5080cc7f0 msgr2=0x7fc5080ccc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:12.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:12.355+0000 7fc50e800640 1 --2- 172.21.15.131:0/4009413004 >> v2:172.21.15.187:3300/0 conn(0x7fc5080cc7f0 0x7fc5080ccc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:12.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:12.355+0000 7fc50de00640 1 --2- 172.21.15.131:0/4009413004 >> v2:172.21.15.187:3300/0 conn(0x7fc5080cc7f0 0x7fc5080ccc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:12.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:12.355+0000 7fc50de00640 1 -- 172.21.15.131:0/4009413004 >> v1:172.21.15.187:6789/0 conn(0x7fc50806e8c0 legacy=0x7fc50815d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:12.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:12.355+0000 7fc50de00640 1 -- 172.21.15.131:0/4009413004 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc508160900 con 0x7fc508156750 2024-05-26T22:55:12.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:12.355+0000 7fc50e800640 1 --2- 172.21.15.131:0/4009413004 >> v2:172.21.15.187:3300/0 conn(0x7fc5080cc7f0 0x7fc5080ccc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:55:12.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:12.356+0000 7fc50de00640 1 --2- 172.21.15.131:0/4009413004 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc508156750 0x7fc50815fd40 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7fc50815a320 tx=0x7fc4f402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16684db059e81f28 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:12.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:12.356+0000 7fc50ca00640 1 -- 172.21.15.131:0/4009413004 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f4037070 con 0x7fc508156750 2024-05-26T22:55:12.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:12.356+0000 7fc50ca00640 1 -- 172.21.15.131:0/4009413004 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4f4032440 con 0x7fc508156750 2024-05-26T22:55:12.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:12.356+0000 7fc50ca00640 1 -- 172.21.15.131:0/4009413004 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f4032810 con 0x7fc508156750 2024-05-26T22:55:12.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:12.357+0000 7fc50f200640 1 -- 172.21.15.131:0/4009413004 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc508156750 msgr2=0x7fc50815fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:12.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:12.357+0000 7fc50f200640 1 --2- 172.21.15.131:0/4009413004 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc508156750 0x7fc50815fd40 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7fc50815a320 tx=0x7fc4f402a230 comp rx=0 tx=0).stop 2024-05-26T22:55:12.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:12.357+0000 7fc50f200640 1 -- 172.21.15.131:0/4009413004 shutdown_connections 2024-05-26T22:55:12.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:12.357+0000 7fc50f200640 1 --2- 172.21.15.131:0/4009413004 >> v2:172.21.15.187:3300/0 conn(0x7fc5080cc7f0 0x7fc5080ccc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:12.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:12.357+0000 7fc50f200640 1 --2- 172.21.15.131:0/4009413004 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc508156750 0x7fc50815fd40 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:12.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:12.357+0000 7fc50f200640 1 -- 172.21.15.131:0/4009413004 >> 172.21.15.131:0/4009413004 conn(0x7fc5080cff40 msgr2=0x7fc5080d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:12.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:12.357+0000 7fc50f200640 1 -- 172.21.15.131:0/4009413004 shutdown_connections 2024-05-26T22:55:12.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:12.357+0000 7fc50f200640 1 -- 172.21.15.131:0/4009413004 wait complete. 2024-05-26T22:55:12.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:12.359+0000 7fc50f200640 1 Processor -- start 2024-05-26T22:55:12.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:12.359+0000 7fc50f200640 1 -- start start 2024-05-26T22:55:12.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:12.360+0000 7fc50f200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc50806e8c0 0x7fc5080b1a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:12.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:12.360+0000 7fc50f200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc5080cc7f0 0x7fc5081557e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:12.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:12.360+0000 7fc50f200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc508156750 0x7fc508144bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:12.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:12.360+0000 7fc50f200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc50815c860 con 0x7fc50806e8c0 2024-05-26T22:55:12.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:12.360+0000 7fc50f200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc50815cbd0 con 0x7fc508156750 2024-05-26T22:55:12.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:12.360+0000 7fc50f200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc508156b30 con 0x7fc5080cc7f0 2024-05-26T22:55:12.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:12.360+0000 7fc50de00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc50806e8c0 0x7fc5080b1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:12.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:12.360+0000 7fc50de00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc50806e8c0 0x7fc5080b1a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51156/0 (socket says 172.21.15.131:51156) 2024-05-26T22:55:12.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:12.360+0000 7fc50de00640 1 -- 172.21.15.131:0/3458932355 learned_addr learned my addr 172.21.15.131:0/3458932355 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:55:12.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:12.361+0000 7fc50de00640 1 -- 172.21.15.131:0/3458932355 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc5080cc7f0 msgr2=0x7fc5081557e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:12.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:12.361+0000 7fc50de00640 1 --2- 172.21.15.131:0/3458932355 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc5080cc7f0 0x7fc5081557e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:12.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:12.361+0000 7fc50de00640 1 -- 172.21.15.131:0/3458932355 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc508156750 msgr2=0x7fc508144bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:12.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:12.361+0000 7fc50de00640 1 --2- 172.21.15.131:0/3458932355 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc508156750 0x7fc508144bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:12.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:12.361+0000 7fc50de00640 1 -- 172.21.15.131:0/3458932355 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5081f7bb0 con 0x7fc50806e8c0 2024-05-26T22:55:12.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:12.361+0000 7fc50de00640 1 --2- 172.21.15.131:0/3458932355 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc50806e8c0 0x7fc5080b1a30 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7fc4f402a270 tx=0x7fc4f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:12.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:12.361+0000 7fc502000640 1 -- 172.21.15.131:0/3458932355 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f4037040 con 0x7fc50806e8c0 2024-05-26T22:55:12.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:12.361+0000 7fc502000640 1 -- 172.21.15.131:0/3458932355 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4f4042070 con 0x7fc50806e8c0 2024-05-26T22:55:12.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:12.361+0000 7fc502000640 1 -- 172.21.15.131:0/3458932355 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f4030bd0 con 0x7fc50806e8c0 2024-05-26T22:55:12.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:12.361+0000 7fc50f200640 1 -- 172.21.15.131:0/3458932355 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5081f7e50 con 0x7fc50806e8c0 2024-05-26T22:55:12.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:12.362+0000 7fc50f200640 1 -- 172.21.15.131:0/3458932355 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5081f8410 con 0x7fc50806e8c0 2024-05-26T22:55:12.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:12.363+0000 7fc50f200640 1 -- 172.21.15.131:0/3458932355 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5080694e0 con 0x7fc50806e8c0 2024-05-26T22:55:12.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:12.364+0000 7fc502000640 1 -- 172.21.15.131:0/3458932355 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc4f4047460 con 0x7fc50806e8c0 2024-05-26T22:55:12.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:12.365+0000 7fc502000640 1 --2- 172.21.15.131:0/3458932355 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc4e806be60 0x7fc4e806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:12.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:12.365+0000 7fc502000640 1 -- 172.21.15.131:0/3458932355 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 5709+0+0 (secure 0 0 0) 0x7fc4f40abb20 con 0x7fc50806e8c0 2024-05-26T22:55:12.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:12.365+0000 7fc50d400640 1 --2- 172.21.15.131:0/3458932355 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc4e806be60 0x7fc4e806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:12.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:12.365+0000 7fc50d400640 1 --2- 172.21.15.131:0/3458932355 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc4e806be60 0x7fc4e806e320 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fc5080b3270 tx=0x7fc4fc007990 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:12.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:12.369+0000 7fc502000640 1 -- 172.21.15.131:0/3458932355 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fc4f4045360 con 0x7fc50806e8c0 2024-05-26T22:55:12.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:12.606+0000 7fc50f200640 1 -- 172.21.15.131:0/3458932355 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc5080d07f0 con 0x7fc50806e8c0 2024-05-26T22:55:12.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:12.607+0000 7fc502000640 1 -- 172.21.15.131:0/3458932355 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 95 v95) ==== 94+0+5205 (secure 0 0 0) 0x7fc4f407a4c0 con 0x7fc50806e8c0 2024-05-26T22:55:12.607 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:55:12.607 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":95,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5231,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/2037149126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6835","nonce":2037149126},{"type":"v1","addr":"172.21.15.131:6837","nonce":2037149126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":89},{"gid":5243,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1314245822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1314245822},{"type":"v1","addr":"172.21.15.187:6833","nonce":1314245822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":89}],"filesystems":[{"mdsmap":{"epoch":95,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-26T22:54:57.328457+0000","modified":"2024-05-26T22:55:12.157376+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1],"up":{"mds_0":7863,"mds_1":5246},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5246":{"gid":5246,"name":"b","rank":1,"incarnation":95,"state":"up:creating","state_seq":1,"addr":"172.21.15.187:6835/521878375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":521878375},{"type":"v1","addr":"172.21.15.187:6835","nonce":521878375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7863":{"gid":7863,"name":"a","rank":0,"incarnation":90,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6836/789312503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":789312503},{"type":"v1","addr":"172.21.15.131:6836","nonce":789312503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7863,"qdb_cluster":[7863]},"id":7}]} 2024-05-26T22:55:12.607 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 95 2024-05-26T22:55:12.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:12.609+0000 7fc50f200640 1 -- 172.21.15.131:0/3458932355 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc4e806be60 msgr2=0x7fc4e806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:12.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:12.609+0000 7fc50f200640 1 --2- 172.21.15.131:0/3458932355 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc4e806be60 0x7fc4e806e320 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fc5080b3270 tx=0x7fc4fc007990 comp rx=0 tx=0).stop 2024-05-26T22:55:12.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:12.609+0000 7fc50f200640 1 -- 172.21.15.131:0/3458932355 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc50806e8c0 msgr2=0x7fc5080b1a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:12.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:12.609+0000 7fc50f200640 1 --2- 172.21.15.131:0/3458932355 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc50806e8c0 0x7fc5080b1a30 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7fc4f402a270 tx=0x7fc4f40046c0 comp rx=0 tx=0).stop 2024-05-26T22:55:12.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:12.609+0000 7fc50f200640 1 -- 172.21.15.131:0/3458932355 shutdown_connections 2024-05-26T22:55:12.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:12.609+0000 7fc50f200640 1 --2- 172.21.15.131:0/3458932355 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc5080cc7f0 0x7fc5081557e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:12.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:12.609+0000 7fc50f200640 1 --2- 172.21.15.131:0/3458932355 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc4e806be60 0x7fc4e806e320 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:12.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:12.609+0000 7fc50f200640 1 --2- 172.21.15.131:0/3458932355 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc508156750 0x7fc508144bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:12.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:12.609+0000 7fc50f200640 1 --2- 172.21.15.131:0/3458932355 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc50806e8c0 0x7fc5080b1a30 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:12.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:12.609+0000 7fc50f200640 1 -- 172.21.15.131:0/3458932355 >> 172.21.15.131:0/3458932355 conn(0x7fc5080cff40 msgr2=0x7fc5080cdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:12.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:12.609+0000 7fc50f200640 1 -- 172.21.15.131:0/3458932355 shutdown_connections 2024-05-26T22:55:12.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:12.610+0000 7fc50f200640 1 -- 172.21.15.131:0/3458932355 wait complete. 2024-05-26T22:55:13.029 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 95, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-26T22:54:57.328457+0000', 'modified': '2024-05-26T22:55:12.157376+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1], 'up': {'mds_0': 7863, 'mds_1': 5246}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5246': {'gid': 5246, 'name': 'b', 'rank': 1, 'incarnation': 95, 'state': 'up:creating', 'state_seq': 1, 'addr': '172.21.15.187:6835/521878375', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.187:6834', 'nonce': 521878375}, {'type': 'v1', 'addr': '172.21.15.187:6835', 'nonce': 521878375}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_7863': {'gid': 7863, 'name': 'a', 'rank': 0, 'incarnation': 90, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.131:6836/789312503', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6834', 'nonce': 789312503}, {'type': 'v1', 'addr': '172.21.15.131:6836', 'nonce': 789312503}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7863, 'qdb_cluster': [7863]} 2024-05-26T22:55:13.029 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_5246:up:creating 2024-05-26T22:55:14.029 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-26T22:55:14.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:14.221+0000 7fef92e00640 1 Processor -- start 2024-05-26T22:55:14.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:14.221+0000 7fef92e00640 1 -- start start 2024-05-26T22:55:14.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:14.222+0000 7fef92e00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fef8c0d0380 0x7fef8c070de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:14.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:14.222+0000 7fef92e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fef8c0ceec0 0x7fef8c071320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:14.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:14.222+0000 7fef92e00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef8c161e60 con 0x7fef8c0ceae0 2024-05-26T22:55:14.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:14.222+0000 7fef92e00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fef8c162140 con 0x7fef8c0d0380 2024-05-26T22:55:14.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:14.222+0000 7fef92e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fef8c162420 con 0x7fef8c0ceec0 2024-05-26T22:55:14.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:14.223+0000 7fef91a00640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7fef8c0ceae0 0x7fef8c1469f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.131:52568/0 (socket says 172.21.15.131:52568) 2024-05-26T22:55:14.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:14.223+0000 7fef91a00640 1 -- 172.21.15.131:0/3279085940 learned_addr learned my addr 172.21.15.131:0/3279085940 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:55:14.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:14.223+0000 7fef92400640 1 --2- 172.21.15.131:0/3279085940 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fef8c0ceec0 0x7fef8c071320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:14.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:14.223+0000 7fef91000640 1 --2- 172.21.15.131:0/3279085940 >> v2:172.21.15.187:3300/0 conn(0x7fef8c0d0380 0x7fef8c070de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:14.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:14.224+0000 7fef92400640 1 -- 172.21.15.131:0/3279085940 >> v2:172.21.15.187:3300/0 conn(0x7fef8c0d0380 msgr2=0x7fef8c070de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:14.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:14.224+0000 7fef92400640 1 --2- 172.21.15.131:0/3279085940 >> v2:172.21.15.187:3300/0 conn(0x7fef8c0d0380 0x7fef8c070de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:14.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:14.224+0000 7fef92400640 1 -- 172.21.15.131:0/3279085940 >> v1:172.21.15.187:6789/0 conn(0x7fef8c0ceae0 legacy=0x7fef8c1469f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:14.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:14.224+0000 7fef92400640 1 -- 172.21.15.131:0/3279085940 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef8c162700 con 0x7fef8c0ceec0 2024-05-26T22:55:14.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:14.225+0000 7fef92400640 1 --2- 172.21.15.131:0/3279085940 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fef8c0ceec0 0x7fef8c071320 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fef84009d90 tx=0x7fef8402a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1072a465eb440a02 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:14.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:14.225+0000 7fef8be00640 1 -- 172.21.15.131:0/3279085940 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef84007740 con 0x7fef8c0ceec0 2024-05-26T22:55:14.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:14.226+0000 7fef8be00640 1 -- 172.21.15.131:0/3279085940 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef840078e0 con 0x7fef8c0ceec0 2024-05-26T22:55:14.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:14.226+0000 7fef8be00640 1 -- 172.21.15.131:0/3279085940 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef84007cb0 con 0x7fef8c0ceec0 2024-05-26T22:55:14.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:14.226+0000 7fef92e00640 1 -- 172.21.15.131:0/3279085940 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fef8c0ceec0 msgr2=0x7fef8c071320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:14.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:14.226+0000 7fef92e00640 1 --2- 172.21.15.131:0/3279085940 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fef8c0ceec0 0x7fef8c071320 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fef84009d90 tx=0x7fef8402a660 comp rx=0 tx=0).stop 2024-05-26T22:55:14.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:14.226+0000 7fef92e00640 1 -- 172.21.15.131:0/3279085940 shutdown_connections 2024-05-26T22:55:14.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:14.226+0000 7fef92e00640 1 --2- 172.21.15.131:0/3279085940 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fef8c0ceec0 0x7fef8c071320 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:14.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:14.226+0000 7fef92e00640 1 --2- 172.21.15.131:0/3279085940 >> v2:172.21.15.187:3300/0 conn(0x7fef8c0d0380 0x7fef8c070de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:14.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:14.226+0000 7fef92e00640 1 -- 172.21.15.131:0/3279085940 >> 172.21.15.131:0/3279085940 conn(0x7fef8c0b3ba0 msgr2=0x7fef8c0b3f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:14.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:14.226+0000 7fef92e00640 1 -- 172.21.15.131:0/3279085940 shutdown_connections 2024-05-26T22:55:14.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:14.226+0000 7fef92e00640 1 -- 172.21.15.131:0/3279085940 wait complete. 2024-05-26T22:55:14.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:14.228+0000 7fef92e00640 1 Processor -- start 2024-05-26T22:55:14.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:14.228+0000 7fef92e00640 1 -- start start 2024-05-26T22:55:14.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:14.229+0000 7fef92e00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fef8c0ceae0 0x7fef8c0d11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:14.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:14.229+0000 7fef92e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fef8c0ceec0 0x7fef8c1e8560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:14.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:14.229+0000 7fef92e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fef8c0d0380 0x7fef8c15d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:14.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:14.229+0000 7fef92e00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fef8c147100 con 0x7fef8c0d0380 2024-05-26T22:55:14.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:14.229+0000 7fef92e00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fef8c162cc0 con 0x7fef8c0ceae0 2024-05-26T22:55:14.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:14.229+0000 7fef92e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fef8c147280 con 0x7fef8c0ceec0 2024-05-26T22:55:14.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:14.230+0000 7fef92400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fef8c0d0380 0x7fef8c15d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:14.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:14.230+0000 7fef92400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fef8c0d0380 0x7fef8c15d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51162/0 (socket says 172.21.15.131:51162) 2024-05-26T22:55:14.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:14.230+0000 7fef92400640 1 -- 172.21.15.131:0/1953493158 learned_addr learned my addr 172.21.15.131:0/1953493158 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:55:14.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:14.230+0000 7fef91a00640 1 --2- 172.21.15.131:0/1953493158 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fef8c0ceae0 0x7fef8c0d11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:14.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:14.230+0000 7fef91000640 1 --2- 172.21.15.131:0/1953493158 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fef8c0ceec0 0x7fef8c1e8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:14.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:14.230+0000 7fef92400640 1 -- 172.21.15.131:0/1953493158 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fef8c0ceec0 msgr2=0x7fef8c1e8560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:14.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:14.230+0000 7fef92400640 1 --2- 172.21.15.131:0/1953493158 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fef8c0ceec0 0x7fef8c1e8560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:14.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:14.230+0000 7fef92400640 1 -- 172.21.15.131:0/1953493158 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fef8c0ceae0 msgr2=0x7fef8c0d11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:14.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:14.230+0000 7fef92400640 1 --2- 172.21.15.131:0/1953493158 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fef8c0ceae0 0x7fef8c0d11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:14.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:14.230+0000 7fef92400640 1 -- 172.21.15.131:0/1953493158 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef8c1e8f40 con 0x7fef8c0d0380 2024-05-26T22:55:14.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:14.230+0000 7fef92400640 1 --2- 172.21.15.131:0/1953493158 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fef8c0d0380 0x7fef8c15d530 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7fef84009eb0 tx=0x7fef84004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:14.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:14.231+0000 7fef89600640 1 -- 172.21.15.131:0/1953493158 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef84040070 con 0x7fef8c0d0380 2024-05-26T22:55:14.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:14.231+0000 7fef89600640 1 -- 172.21.15.131:0/1953493158 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef84031c60 con 0x7fef8c0d0380 2024-05-26T22:55:14.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:14.231+0000 7fef89600640 1 -- 172.21.15.131:0/1953493158 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef84009450 con 0x7fef8c0d0380 2024-05-26T22:55:14.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:14.231+0000 7fef92e00640 1 -- 172.21.15.131:0/1953493158 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef8c1e9220 con 0x7fef8c0d0380 2024-05-26T22:55:14.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:14.231+0000 7fef92e00640 1 -- 172.21.15.131:0/1953493158 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef8c1e97e0 con 0x7fef8c0d0380 2024-05-26T22:55:14.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:14.232+0000 7fef92e00640 1 -- 172.21.15.131:0/1953493158 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef8c0694e0 con 0x7fef8c0d0380 2024-05-26T22:55:14.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:14.234+0000 7fef89600640 1 -- 172.21.15.131:0/1953493158 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fef84039040 con 0x7fef8c0d0380 2024-05-26T22:55:14.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:14.235+0000 7fef89600640 1 --2- 172.21.15.131:0/1953493158 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fef6c06be30 0x7fef6c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:14.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:14.235+0000 7fef89600640 1 -- 172.21.15.131:0/1953493158 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 5709+0+0 (secure 0 0 0) 0x7fef840ab9d0 con 0x7fef8c0d0380 2024-05-26T22:55:14.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:14.235+0000 7fef91a00640 1 --2- 172.21.15.131:0/1953493158 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fef6c06be30 0x7fef6c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:14.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:14.235+0000 7fef91a00640 1 --2- 172.21.15.131:0/1953493158 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fef6c06be30 0x7fef6c06e2f0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fef80005880 tx=0x7fef80009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:14.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:14.240+0000 7fef89600640 1 -- 172.21.15.131:0/1953493158 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fef8404b020 con 0x7fef8c0d0380 2024-05-26T22:55:14.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:14.473+0000 7fef92e00640 1 -- 172.21.15.131:0/1953493158 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fef8c06fef0 con 0x7fef8c0d0380 2024-05-26T22:55:14.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:14.474+0000 7fef89600640 1 -- 172.21.15.131:0/1953493158 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 98 v98) ==== 94+0+5227 (secure 0 0 0) 0x7fef8407a250 con 0x7fef8c0d0380 2024-05-26T22:55:14.474 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:55:14.474 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":98,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5231,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/2037149126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6835","nonce":2037149126},{"type":"v1","addr":"172.21.15.131:6837","nonce":2037149126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":89}],"filesystems":[{"mdsmap":{"epoch":98,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-26T22:54:57.328457+0000","modified":"2024-05-26T22:55:14.171774+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7863,"mds_1":5246,"mds_2":5243},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5243":{"gid":5243,"name":"d","rank":2,"incarnation":97,"state":"up:active","state_seq":7,"addr":"172.21.15.187:6833/1314245822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1314245822},{"type":"v1","addr":"172.21.15.187:6833","nonce":1314245822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5246":{"gid":5246,"name":"b","rank":1,"incarnation":95,"state":"up:active","state_seq":6,"addr":"172.21.15.187:6835/521878375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":521878375},{"type":"v1","addr":"172.21.15.187:6835","nonce":521878375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7863":{"gid":7863,"name":"a","rank":0,"incarnation":90,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6836/789312503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":789312503},{"type":"v1","addr":"172.21.15.131:6836","nonce":789312503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7863,"qdb_cluster":[5243,5246,7863]},"id":7}]} 2024-05-26T22:55:14.475 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 98 2024-05-26T22:55:14.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:14.477+0000 7fef92e00640 1 -- 172.21.15.131:0/1953493158 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fef6c06be30 msgr2=0x7fef6c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:14.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:14.477+0000 7fef92e00640 1 --2- 172.21.15.131:0/1953493158 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fef6c06be30 0x7fef6c06e2f0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fef80005880 tx=0x7fef80009040 comp rx=0 tx=0).stop 2024-05-26T22:55:14.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:14.477+0000 7fef92e00640 1 -- 172.21.15.131:0/1953493158 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fef8c0d0380 msgr2=0x7fef8c15d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:14.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:14.477+0000 7fef92e00640 1 --2- 172.21.15.131:0/1953493158 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fef8c0d0380 0x7fef8c15d530 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7fef84009eb0 tx=0x7fef84004a40 comp rx=0 tx=0).stop 2024-05-26T22:55:14.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:14.477+0000 7fef92e00640 1 -- 172.21.15.131:0/1953493158 shutdown_connections 2024-05-26T22:55:14.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:14.477+0000 7fef92e00640 1 --2- 172.21.15.131:0/1953493158 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fef8c0ceec0 0x7fef8c1e8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:14.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:14.477+0000 7fef92e00640 1 --2- 172.21.15.131:0/1953493158 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fef6c06be30 0x7fef6c06e2f0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:14.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:14.477+0000 7fef92e00640 1 --2- 172.21.15.131:0/1953493158 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fef8c0d0380 0x7fef8c15d530 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:14.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:14.477+0000 7fef92e00640 1 --2- 172.21.15.131:0/1953493158 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fef8c0ceae0 0x7fef8c0d11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:14.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:14.477+0000 7fef92e00640 1 -- 172.21.15.131:0/1953493158 >> 172.21.15.131:0/1953493158 conn(0x7fef8c0b3ba0 msgr2=0x7fef8c06c650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:14.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:14.477+0000 7fef92e00640 1 -- 172.21.15.131:0/1953493158 shutdown_connections 2024-05-26T22:55:14.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:14.477+0000 7fef92e00640 1 -- 172.21.15.131:0/1953493158 wait complete. 2024-05-26T22:55:14.897 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 98, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-26T22:54:57.328457+0000', 'modified': '2024-05-26T22:55:14.171774+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 7863, 'mds_1': 5246, 'mds_2': 5243}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5243': {'gid': 5243, 'name': 'd', 'rank': 2, 'incarnation': 97, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.187:6833/1314245822', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.187:6832', 'nonce': 1314245822}, {'type': 'v1', 'addr': '172.21.15.187:6833', 'nonce': 1314245822}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_5246': {'gid': 5246, 'name': 'b', 'rank': 1, 'incarnation': 95, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.187:6835/521878375', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.187:6834', 'nonce': 521878375}, {'type': 'v1', 'addr': '172.21.15.187:6835', 'nonce': 521878375}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_7863': {'gid': 7863, 'name': 'a', 'rank': 0, 'incarnation': 90, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.131:6836/789312503', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6834', 'nonce': 789312503}, {'type': 'v1', 'addr': '172.21.15.131:6836', 'nonce': 789312503}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7863, 'qdb_cluster': [5243, 5246, 7863]} 2024-05-26T22:55:14.898 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-05-26T22:55:14.898 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-05-26T22:55:15.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.091+0000 7f6739e00640 1 Processor -- start 2024-05-26T22:55:15.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.091+0000 7f6739e00640 1 -- start start 2024-05-26T22:55:15.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.093+0000 7f6739e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f673406e5e0 0x7f67340b47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:15.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.093+0000 7f6739e00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f673415e210 con 0x7f673415de30 2024-05-26T22:55:15.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.093+0000 7f6739e00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67340b5e20 con 0x7f67340b5880 2024-05-26T22:55:15.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.093+0000 7f6739e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f67340b3ac0 con 0x7f673406e5e0 2024-05-26T22:55:15.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.093+0000 7f6732a00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f673415de30 0x7f673406ded0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:58430/0 (socket says 172.21.15.131:58430) 2024-05-26T22:55:15.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.093+0000 7f6732a00640 1 -- 172.21.15.131:0/3423162263 learned_addr learned my addr 172.21.15.131:0/3423162263 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:55:15.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.094+0000 7f6732000640 1 -- 172.21.15.131:0/3423162263 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1130884646 0 0) 0x7f673415e210 con 0x7f673415de30 2024-05-26T22:55:15.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.094+0000 7f6732000640 1 -- 172.21.15.131:0/3423162263 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6718005730 con 0x7f673415de30 2024-05-26T22:55:15.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.094+0000 7f672a000640 1 --2- 172.21.15.131:0/3423162263 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f673406e5e0 0x7f67340b47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:15.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.094+0000 7f6732000640 1 -- 172.21.15.131:0/3423162263 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1731141339 0 0) 0x7f6718005730 con 0x7f673415de30 2024-05-26T22:55:15.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.094+0000 7f6732000640 1 -- 172.21.15.131:0/3423162263 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f673406e5e0 msgr2=0x7f67340b47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:15.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.094+0000 7f6732000640 1 --2- 172.21.15.131:0/3423162263 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f673406e5e0 0x7f67340b47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:15.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.094+0000 7f6732000640 1 -- 172.21.15.131:0/3423162263 >> v1:172.21.15.187:6789/0 conn(0x7f67340b5880 legacy=0x7f67340ced60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:15.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.094+0000 7f6732000640 1 -- 172.21.15.131:0/3423162263 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67340b3d10 con 0x7f673415de30 2024-05-26T22:55:15.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.094+0000 7f6732000640 1 -- 172.21.15.131:0/3423162263 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7f6724001670 con 0x7f673415de30 2024-05-26T22:55:15.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.094+0000 7f6732000640 1 -- 172.21.15.131:0/3423162263 >> v1:172.21.15.131:6789/0 conn(0x7f673415de30 legacy=0x7f673406ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:15.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.094+0000 7f6732000640 1 --2- 172.21.15.131:0/3423162263 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f673406e5e0 0x7f6718005bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:15.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.094+0000 7f6732000640 1 --2- 172.21.15.131:0/3423162263 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6718006220 0x7f6718006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:15.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.094+0000 7f6732000640 1 --2- 172.21.15.131:0/3423162263 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6718006f50 0x7f67180073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:15.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.094+0000 7f6732000640 1 -- 172.21.15.131:0/3423162263 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f67340b3ac0 con 0x7f6718006220 2024-05-26T22:55:15.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.094+0000 7f6732000640 1 -- 172.21.15.131:0/3423162263 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f671800de10 con 0x7f6718006f50 2024-05-26T22:55:15.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.094+0000 7f6732000640 1 -- 172.21.15.131:0/3423162263 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f671800e010 con 0x7f673406e5e0 2024-05-26T22:55:15.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.095+0000 7f672a000640 1 --2- 172.21.15.131:0/3423162263 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6718006220 0x7f6718006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:15.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.095+0000 7f6732a00640 1 --2- 172.21.15.131:0/3423162263 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f673406e5e0 0x7f6718005bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:15.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.095+0000 7f672a000640 1 -- 172.21.15.131:0/3423162263 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f673406e5e0 msgr2=0x7f6718005bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:15.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.095+0000 7f672a000640 1 --2- 172.21.15.131:0/3423162263 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f673406e5e0 0x7f6718005bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:15.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.095+0000 7f672a000640 1 -- 172.21.15.131:0/3423162263 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6718006f50 msgr2=0x7f67180073d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:15.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.095+0000 7f6733400640 1 --2- 172.21.15.131:0/3423162263 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6718006f50 0x7f67180073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:15.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.095+0000 7f672a000640 1 --2- 172.21.15.131:0/3423162263 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6718006f50 0x7f67180073d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:15.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.095+0000 7f672a000640 1 -- 172.21.15.131:0/3423162263 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6718005730 con 0x7f6718006220 2024-05-26T22:55:15.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.096+0000 7f672a000640 1 --2- 172.21.15.131:0/3423162263 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6718006220 0x7f6718006a10 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f671c00ba30 tx=0x7f6718007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28e11c0e026df347 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:15.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.096+0000 7f6732000640 1 -- 172.21.15.131:0/3423162263 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f671c030070 con 0x7f6718006220 2024-05-26T22:55:15.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.096+0000 7f6732000640 1 -- 172.21.15.131:0/3423162263 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f671c02ade0 con 0x7f6718006220 2024-05-26T22:55:15.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.096+0000 7f6732000640 1 -- 172.21.15.131:0/3423162263 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f671c00b600 con 0x7f6718006220 2024-05-26T22:55:15.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.097+0000 7f6739e00640 1 -- 172.21.15.131:0/3423162263 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6718006220 msgr2=0x7f6718006a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:15.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.097+0000 7f6739e00640 1 --2- 172.21.15.131:0/3423162263 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6718006220 0x7f6718006a10 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f671c00ba30 tx=0x7f6718007a40 comp rx=0 tx=0).stop 2024-05-26T22:55:15.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.097+0000 7f6733400640 1 -- 172.21.15.131:0/3423162263 reap_dead start 2024-05-26T22:55:15.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.097+0000 7f6739e00640 1 -- 172.21.15.131:0/3423162263 shutdown_connections 2024-05-26T22:55:15.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.097+0000 7f6739e00640 1 -- 172.21.15.131:0/3423162263 >> 172.21.15.131:0/3423162263 conn(0x7f67340b27c0 msgr2=0x7f67340b2bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:15.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.097+0000 7f6739e00640 1 -- 172.21.15.131:0/3423162263 shutdown_connections 2024-05-26T22:55:15.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.097+0000 7f6739e00640 1 -- 172.21.15.131:0/3423162263 wait complete. 2024-05-26T22:55:15.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.099+0000 7f6739e00640 1 Processor -- start 2024-05-26T22:55:15.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.099+0000 7f6739e00640 1 -- start start 2024-05-26T22:55:15.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.100+0000 7f6739e00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6718006220 0x7f67341f4230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:15.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.100+0000 7f6739e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f673414c280 0x7f673414d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:15.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.100+0000 7f6739e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f67340b1b90 0x7f67340b2040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:15.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.100+0000 7f6739e00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f67340b47f0 con 0x7f67340b1b90 2024-05-26T22:55:15.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.100+0000 7f6739e00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f67340b42d0 con 0x7f6718006220 2024-05-26T22:55:15.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.100+0000 7f6739e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f67340b4970 con 0x7f673414c280 2024-05-26T22:55:15.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.100+0000 7f6733400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f67340b1b90 0x7f67340b2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:15.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.100+0000 7f6733400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f67340b1b90 0x7f67340b2040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59704/0 (socket says 172.21.15.131:59704) 2024-05-26T22:55:15.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.100+0000 7f6733400640 1 -- 172.21.15.131:0/3420362080 learned_addr learned my addr 172.21.15.131:0/3420362080 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:55:15.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.100+0000 7f672a000640 1 --2- 172.21.15.131:0/3420362080 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f673414c280 0x7f673414d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:15.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.100+0000 7f6732a00640 1 --2- 172.21.15.131:0/3420362080 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6718006220 0x7f67341f4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:15.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.100+0000 7f6733400640 1 -- 172.21.15.131:0/3420362080 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f673414c280 msgr2=0x7f673414d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:15.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.100+0000 7f6733400640 1 --2- 172.21.15.131:0/3420362080 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f673414c280 0x7f673414d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:15.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.101+0000 7f6733400640 1 -- 172.21.15.131:0/3420362080 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6718006220 msgr2=0x7f67341f4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:15.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.101+0000 7f6733400640 1 --2- 172.21.15.131:0/3420362080 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6718006220 0x7f67341f4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:15.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.101+0000 7f6733400640 1 -- 172.21.15.131:0/3420362080 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f673414cb60 con 0x7f67340b1b90 2024-05-26T22:55:15.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.101+0000 7f6733400640 1 --2- 172.21.15.131:0/3420362080 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f67340b1b90 0x7f67340b2040 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f67340b3650 tx=0x7f672c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:15.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.101+0000 7f672be00640 1 -- 172.21.15.131:0/3420362080 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f672c018070 con 0x7f67340b1b90 2024-05-26T22:55:15.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.101+0000 7f672be00640 1 -- 172.21.15.131:0/3420362080 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f672c003b70 con 0x7f67340b1b90 2024-05-26T22:55:15.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.101+0000 7f672be00640 1 -- 172.21.15.131:0/3420362080 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f672c005070 con 0x7f67340b1b90 2024-05-26T22:55:15.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.101+0000 7f6739e00640 1 -- 172.21.15.131:0/3420362080 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f673414ce40 con 0x7f67340b1b90 2024-05-26T22:55:15.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.101+0000 7f6739e00640 1 -- 172.21.15.131:0/3420362080 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67341f3a30 con 0x7f67340b1b90 2024-05-26T22:55:15.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.104+0000 7f672be00640 1 -- 172.21.15.131:0/3420362080 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f672c01b070 con 0x7f67340b1b90 2024-05-26T22:55:15.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.105+0000 7f673cc2c740 1 Processor -- start 2024-05-26T22:55:15.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.105+0000 7f673cc2c740 1 -- start start 2024-05-26T22:55:15.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.105+0000 7f672be00640 1 --2- 172.21.15.131:0/3420362080 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f671006be60 0x7f671006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:15.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.105+0000 7f672be00640 1 -- 172.21.15.131:0/3420362080 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 5709+0+0 (secure 0 0 0) 0x7f672c08a4f0 con 0x7f67340b1b90 2024-05-26T22:55:15.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.106+0000 7f672a000640 1 --2- 172.21.15.131:0/3420362080 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f671006be60 0x7f671006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:15.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.106+0000 7f673cc2c740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c4ee747c70 0x55c4ee6fa890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:15.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.106+0000 7f673cc2c740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4ee72ad90 0x55c4ee72a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:15.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.106+0000 7f673cc2c740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55c4ee72a650 0x55c4ee6ff460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:15.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.106+0000 7f673cc2c740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55c4ee662610 con 0x55c4ee72ad90 2024-05-26T22:55:15.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.106+0000 7f673cc2c740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55c4ee60a5c0 con 0x55c4ee747c70 2024-05-26T22:55:15.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.106+0000 7f673cc2c740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x55c4ee65e5d0 con 0x55c4ee72a650 2024-05-26T22:55:15.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.106+0000 7f672a000640 1 --2- 172.21.15.131:0/3420362080 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f671006be60 0x7f671006e320 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f671c00ba00 tx=0x7f671c030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:15.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.106+0000 7f672a000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4ee72ad90 0x55c4ee72a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:15.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.106+0000 7f672a000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4ee72ad90 0x55c4ee72a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59714/0 (socket says 172.21.15.131:59714) 2024-05-26T22:55:15.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.106+0000 7f672a000640 1 -- 172.21.15.131:0/478735232 learned_addr learned my addr 172.21.15.131:0/478735232 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:55:15.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.107+0000 7f6733400640 1 --2- 172.21.15.131:0/478735232 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55c4ee72a650 0x55c4ee6ff460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:15.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.107+0000 7f672a000640 1 -- 172.21.15.131:0/478735232 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55c4ee72a650 msgr2=0x55c4ee6ff460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:15.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.107+0000 7f6732a00640 1 --2- 172.21.15.131:0/478735232 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c4ee747c70 0x55c4ee6fa890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:15.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.107+0000 7f672a000640 1 --2- 172.21.15.131:0/478735232 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55c4ee72a650 0x55c4ee6ff460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:15.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.107+0000 7f672a000640 1 -- 172.21.15.131:0/478735232 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c4ee747c70 msgr2=0x55c4ee6fa890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:15.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.107+0000 7f672a000640 1 --2- 172.21.15.131:0/478735232 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c4ee747c70 0x55c4ee6fa890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:15.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.107+0000 7f672a000640 1 -- 172.21.15.131:0/478735232 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4ee3762d0 con 0x55c4ee72ad90 2024-05-26T22:55:15.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.107+0000 7f672a000640 1 --2- 172.21.15.131:0/478735232 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4ee72ad90 0x55c4ee72a110 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f671c02a840 tx=0x7f671c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b71020422b2e23b1 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:15.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.107+0000 7f6723400640 1 -- 172.21.15.131:0/478735232 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f671c03d410 con 0x55c4ee72ad90 2024-05-26T22:55:15.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.107+0000 7f6723400640 1 -- 172.21.15.131:0/478735232 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f671c03d5b0 con 0x55c4ee72ad90 2024-05-26T22:55:15.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.107+0000 7f6723400640 1 -- 172.21.15.131:0/478735232 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f671c03d960 con 0x55c4ee72ad90 2024-05-26T22:55:15.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.108+0000 7f673cc2c740 1 -- 172.21.15.131:0/478735232 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4ee72ad90 msgr2=0x55c4ee72a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:15.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.108+0000 7f673cc2c740 1 --2- 172.21.15.131:0/478735232 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4ee72ad90 0x55c4ee72a110 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f671c02a840 tx=0x7f671c008040 comp rx=0 tx=0).stop 2024-05-26T22:55:15.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.108+0000 7f673cc2c740 1 -- 172.21.15.131:0/478735232 shutdown_connections 2024-05-26T22:55:15.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.108+0000 7f673cc2c740 1 --2- 172.21.15.131:0/478735232 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55c4ee72a650 0x55c4ee6ff460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:15.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.108+0000 7f673cc2c740 1 --2- 172.21.15.131:0/478735232 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4ee72ad90 0x55c4ee72a110 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:15.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.108+0000 7f673cc2c740 1 --2- 172.21.15.131:0/478735232 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c4ee747c70 0x55c4ee6fa890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:15.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.108+0000 7f673cc2c740 1 -- 172.21.15.131:0/478735232 >> 172.21.15.131:0/478735232 conn(0x55c4ee6fcb00 msgr2=0x55c4ee745110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:15.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.108+0000 7f673cc2c740 1 -- 172.21.15.131:0/478735232 shutdown_connections 2024-05-26T22:55:15.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.108+0000 7f673cc2c740 1 -- 172.21.15.131:0/478735232 wait complete. 2024-05-26T22:55:15.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.109+0000 7f673cc2c740 1 -- start start 2024-05-26T22:55:15.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.109+0000 7f673cc2c740 1 Processor -- start 2024-05-26T22:55:15.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.110+0000 7f673cc2c740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55c4ee72ad90 0x55c4ee72ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:15.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.110+0000 7f673cc2c740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c4ee747c70 0x55c4ee7183e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:15.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.110+0000 7f673cc2c740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4ee718920 0x55c4ee743bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:15.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.110+0000 7f673cc2c740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55c4ee6fea20 con 0x55c4ee718920 2024-05-26T22:55:15.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.110+0000 7f673cc2c740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55c4ee61d910 con 0x55c4ee747c70 2024-05-26T22:55:15.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.110+0000 7f673cc2c740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x55c4ee6fde10 con 0x55c4ee72ad90 2024-05-26T22:55:15.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.110+0000 7f673cc2c740 10 client.0 _ll_register_callbacks cb 0x55c4ee659410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-26T22:55:15.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.110+0000 7f6733400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4ee718920 0x55c4ee743bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:15.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.110+0000 7f6733400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4ee718920 0x55c4ee743bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59716/0 (socket says 172.21.15.131:59716) 2024-05-26T22:55:15.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.110+0000 7f6733400640 1 -- 172.21.15.131:0/3051630282 learned_addr learned my addr 172.21.15.131:0/3051630282 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:55:15.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.111+0000 7f6733400640 1 -- 172.21.15.131:0/3051630282 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55c4ee72ad90 msgr2=0x55c4ee72ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:15.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.111+0000 7f6733400640 1 --2- 172.21.15.131:0/3051630282 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55c4ee72ad90 0x55c4ee72ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:15.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.111+0000 7f6733400640 1 -- 172.21.15.131:0/3051630282 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c4ee747c70 msgr2=0x55c4ee7183e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:15.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.111+0000 7f6733400640 1 --2- 172.21.15.131:0/3051630282 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c4ee747c70 0x55c4ee7183e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:15.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.111+0000 7f6733400640 1 -- 172.21.15.131:0/3051630282 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4ee4406b0 con 0x55c4ee718920 2024-05-26T22:55:15.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.111+0000 7f6733400640 1 --2- 172.21.15.131:0/3051630282 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4ee718920 0x55c4ee743bb0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f672c0889b0 tx=0x7f672c0583d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff581fb2e7b59acc server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:15.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.111+0000 7f6717e00640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-26T22:55:15.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.111+0000 7f6717e00640 1 -- 172.21.15.131:0/3051630282 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f672c08a2c0 con 0x55c4ee718920 2024-05-26T22:55:15.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.111+0000 7f673cc2c740 1 -- 172.21.15.131:0/3051630282 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f671c03d5b0 con 0x55c4ee718920 2024-05-26T22:55:15.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.111+0000 7f6717e00640 1 -- 172.21.15.131:0/3051630282 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 5709+0+0 (secure 0 0 0) 0x7f672c057be0 con 0x55c4ee718920 2024-05-26T22:55:15.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.111+0000 7f6717e00640 1 -- 172.21.15.131:0/3051630282 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=98) ==== 24+0+0 (secure 0 0 0) 0x7f672c05cb40 con 0x55c4ee718920 2024-05-26T22:55:15.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.111+0000 7f673cc2c740 10 client.8052 fetch_fsmap learned FSMap version 98 2024-05-26T22:55:15.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.111+0000 7f673cc2c740 1 -- 172.21.15.131:0/3051630282 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=98,monmap=2+}) -- 0x55c4ee72c4e0 con 0x55c4ee718920 2024-05-26T22:55:15.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.112+0000 7f6717e00640 1 -- 172.21.15.131:0/3051630282 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f671c03d5b0 con 0x55c4ee718920 2024-05-26T22:55:15.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.112+0000 7f6717e00640 1 -- 172.21.15.131:0/3051630282 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 98) ==== 3534+0+0 (secure 0 0 0) 0x7f672c0544e0 con 0x55c4ee718920 2024-05-26T22:55:15.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.112+0000 7f673cc2c740 10 client.8052 fetch_fsmap finished waiting for FSMap version 98 2024-05-26T22:55:15.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.112+0000 7f673cc2c740 10 client.8052 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-26T22:55:15.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.112+0000 7f673cc2c740 20 client.8052 populate_metadata read hostname 'smithi131' 2024-05-26T22:55:15.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.112+0000 7f673cc2c740 1 --2- 172.21.15.131:0/3051630282 >> [v2:172.21.15.187:6832/1314245822,v1:172.21.15.187:6833/1314245822] conn(0x55c4ee745960 0x55c4ee763830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:15.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.112+0000 7f673cc2c740 4 client.8052 mds_command: new command op to 5243 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-26T22:55:15.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.112+0000 7f673cc2c740 1 -- 172.21.15.131:0/3051630282 --> [v2:172.21.15.187:6832/1314245822,v1:172.21.15.187:6833/1314245822] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c4ee66fd40 con 0x55c4ee745960 2024-05-26T22:55:15.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.112+0000 7f6732a00640 1 --2- 172.21.15.131:0/3051630282 >> [v2:172.21.15.187:6832/1314245822,v1:172.21.15.187:6833/1314245822] conn(0x55c4ee745960 0x55c4ee763830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:15.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.113+0000 7f6732a00640 1 --2- 172.21.15.131:0/3051630282 >> [v2:172.21.15.187:6832/1314245822,v1:172.21.15.187:6833/1314245822] conn(0x55c4ee745960 0x55c4ee763830 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=90285136cd38fe50 server_cookie=eaddcb55632d4a44 in_seq=0 out_seq=0 2024-05-26T22:55:15.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.113+0000 7f6717e00640 10 client.8052 ms_handle_connect on v2:172.21.15.187:6832/1314245822 2024-05-26T22:55:15.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.115+0000 7f6717e00640 1 -- 172.21.15.131:0/3051630282 <== mds.2 v2:172.21.15.187:6832/1314245822 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c4ee66fd40 con 0x55c4ee745960 2024-05-26T22:55:15.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.115+0000 7f6717e00640 10 client.8052 handle_command_reply: tid=0 2024-05-26T22:55:15.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.115+0000 7f673cc2c740 1 client.8052 shutdown 2024-05-26T22:55:15.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.115+0000 7f673cc2c740 1 -- 172.21.15.131:0/3051630282 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4ee718920 msgr2=0x55c4ee743bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:15.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.115+0000 7f673cc2c740 1 --2- 172.21.15.131:0/3051630282 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4ee718920 0x55c4ee743bb0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f672c0889b0 tx=0x7f672c0583d0 comp rx=0 tx=0).stop 2024-05-26T22:55:15.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.115+0000 7f673cc2c740 1 -- 172.21.15.131:0/3051630282 shutdown_connections 2024-05-26T22:55:15.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.115+0000 7f673cc2c740 1 --2- 172.21.15.131:0/3051630282 >> [v2:172.21.15.187:6832/1314245822,v1:172.21.15.187:6833/1314245822] conn(0x55c4ee745960 0x55c4ee763830 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:15.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.115+0000 7f673cc2c740 1 --2- 172.21.15.131:0/3051630282 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4ee718920 0x55c4ee743bb0 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:15.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.115+0000 7f673cc2c740 1 --2- 172.21.15.131:0/3051630282 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c4ee747c70 0x55c4ee7183e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:15.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.115+0000 7f673cc2c740 1 --2- 172.21.15.131:0/3051630282 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55c4ee72ad90 0x55c4ee72ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:15.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.115+0000 7f6717e00640 0 client.8052 ms_handle_reset on v2:172.21.15.187:6832/1314245822 2024-05-26T22:55:15.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.115+0000 7f673cc2c740 1 -- 172.21.15.131:0/3051630282 >> 172.21.15.131:0/3051630282 conn(0x55c4ee72a650 msgr2=0x55c4ee6fb2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:15.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.116+0000 7f673cc2c740 1 -- 172.21.15.131:0/3051630282 shutdown_connections 2024-05-26T22:55:15.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.116+0000 7f673cc2c740 1 -- 172.21.15.131:0/3051630282 wait complete. 2024-05-26T22:55:15.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.116+0000 7f673cc2c740 20 client.8052 trim_cache size 0 max 16384 2024-05-26T22:55:15.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.136+0000 7f673cc2c740 1 Processor -- start 2024-05-26T22:55:15.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.136+0000 7f673cc2c740 1 -- start start 2024-05-26T22:55:15.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.137+0000 7f673cc2c740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c4ee72a650 0x55c4ee7435b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:15.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.137+0000 7f673cc2c740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55c4ee72ad90 0x55c4ee743af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:15.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.137+0000 7f673cc2c740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4ee747c70 0x55c4ee753750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:15.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.137+0000 7f673cc2c740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f67341fc1a0 con 0x55c4ee747c70 2024-05-26T22:55:15.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.137+0000 7f673cc2c740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f67341fc870 con 0x55c4ee72a650 2024-05-26T22:55:15.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.137+0000 7f673cc2c740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f6734159800 con 0x55c4ee72ad90 2024-05-26T22:55:15.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.137+0000 7f6733400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4ee747c70 0x55c4ee753750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:15.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.137+0000 7f6733400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4ee747c70 0x55c4ee753750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59728/0 (socket says 172.21.15.131:59728) 2024-05-26T22:55:15.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.137+0000 7f6733400640 1 -- 172.21.15.131:0/1921303980 learned_addr learned my addr 172.21.15.131:0/1921303980 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:55:15.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.137+0000 7f672a000640 1 --2- 172.21.15.131:0/1921303980 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55c4ee72ad90 0x55c4ee743af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:15.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.137+0000 7f6733400640 1 -- 172.21.15.131:0/1921303980 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55c4ee72ad90 msgr2=0x55c4ee743af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:15.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.137+0000 7f6733400640 1 --2- 172.21.15.131:0/1921303980 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55c4ee72ad90 0x55c4ee743af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:15.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.137+0000 7f6733400640 1 -- 172.21.15.131:0/1921303980 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c4ee72a650 msgr2=0x55c4ee7435b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:15.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.137+0000 7f6732a00640 1 --2- 172.21.15.131:0/1921303980 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c4ee72a650 0x55c4ee7435b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:15.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.137+0000 7f6733400640 1 --2- 172.21.15.131:0/1921303980 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c4ee72a650 0x55c4ee7435b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:15.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.137+0000 7f6733400640 1 -- 172.21.15.131:0/1921303980 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4ee718260 con 0x55c4ee747c70 2024-05-26T22:55:15.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.138+0000 7f6733400640 1 --2- 172.21.15.131:0/1921303980 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4ee747c70 0x55c4ee753750 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f672c055110 tx=0x7f672c08a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c17d8a5f2c24283 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:15.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.138+0000 7f6722a00640 1 -- 172.21.15.131:0/1921303980 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f672c08a0c0 con 0x55c4ee747c70 2024-05-26T22:55:15.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.138+0000 7f6722a00640 1 -- 172.21.15.131:0/1921303980 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f672c08a260 con 0x55c4ee747c70 2024-05-26T22:55:15.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.138+0000 7f6722a00640 1 -- 172.21.15.131:0/1921303980 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f672c057e00 con 0x55c4ee747c70 2024-05-26T22:55:15.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.138+0000 7f673cc2c740 1 -- 172.21.15.131:0/1921303980 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4ee747c70 msgr2=0x55c4ee753750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:15.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.138+0000 7f673cc2c740 1 --2- 172.21.15.131:0/1921303980 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4ee747c70 0x55c4ee753750 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f672c055110 tx=0x7f672c08a050 comp rx=0 tx=0).stop 2024-05-26T22:55:15.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.138+0000 7f673cc2c740 1 -- 172.21.15.131:0/1921303980 shutdown_connections 2024-05-26T22:55:15.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.138+0000 7f673cc2c740 1 --2- 172.21.15.131:0/1921303980 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55c4ee72ad90 0x55c4ee743af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:15.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.139+0000 7f673cc2c740 1 --2- 172.21.15.131:0/1921303980 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4ee747c70 0x55c4ee753750 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:15.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.139+0000 7f673cc2c740 1 --2- 172.21.15.131:0/1921303980 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c4ee72a650 0x55c4ee7435b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:15.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.139+0000 7f673cc2c740 1 -- 172.21.15.131:0/1921303980 >> 172.21.15.131:0/1921303980 conn(0x55c4ee718540 msgr2=0x55c4ee6fa780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:15.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.139+0000 7f673cc2c740 1 -- 172.21.15.131:0/1921303980 shutdown_connections 2024-05-26T22:55:15.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.139+0000 7f673cc2c740 1 -- 172.21.15.131:0/1921303980 wait complete. 2024-05-26T22:55:15.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.140+0000 7f673cc2c740 1 -- start start 2024-05-26T22:55:15.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.140+0000 7f673cc2c740 1 Processor -- start 2024-05-26T22:55:15.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.141+0000 7f673cc2c740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4ee72a650 0x55c4ee7255e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:15.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.141+0000 7f673cc2c740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55c4ee72ad90 0x55c4ee726870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:15.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.141+0000 7f673cc2c740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c4ee747c70 0x55c4ee7489a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:15.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.141+0000 7f673cc2c740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55c4ee725fb0 con 0x55c4ee72a650 2024-05-26T22:55:15.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.141+0000 7f673cc2c740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55c4ee72cfa0 con 0x55c4ee747c70 2024-05-26T22:55:15.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.141+0000 7f673cc2c740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x55c4ee744260 con 0x55c4ee72ad90 2024-05-26T22:55:15.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.141+0000 7f673cc2c740 10 client.0 _ll_register_callbacks cb 0x7f672c0016a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-26T22:55:15.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.142+0000 7f6732a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4ee72a650 0x55c4ee7255e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:15.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.142+0000 7f6732a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4ee72a650 0x55c4ee7255e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59742/0 (socket says 172.21.15.131:59742) 2024-05-26T22:55:15.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.142+0000 7f6732a00640 1 -- 172.21.15.131:0/2858921430 learned_addr learned my addr 172.21.15.131:0/2858921430 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:55:15.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.142+0000 7f6732a00640 1 -- 172.21.15.131:0/2858921430 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55c4ee72ad90 msgr2=0x55c4ee726870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:15.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.142+0000 7f6732a00640 1 --2- 172.21.15.131:0/2858921430 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55c4ee72ad90 0x55c4ee726870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:15.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.142+0000 7f6732a00640 1 -- 172.21.15.131:0/2858921430 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c4ee747c70 msgr2=0x55c4ee7489a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:15.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.142+0000 7f6733400640 1 --2- 172.21.15.131:0/2858921430 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c4ee747c70 0x55c4ee7489a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:15.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.142+0000 7f6732a00640 1 --2- 172.21.15.131:0/2858921430 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c4ee747c70 0x55c4ee7489a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:15.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.142+0000 7f6732a00640 1 -- 172.21.15.131:0/2858921430 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4ee7cc4e0 con 0x55c4ee72a650 2024-05-26T22:55:15.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.142+0000 7f6732a00640 1 --2- 172.21.15.131:0/2858921430 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4ee72a650 0x55c4ee7255e0 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f67240017e0 tx=0x7f6724018810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f1d8c9c5d7f7ec7 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:15.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.142+0000 7f6717e00640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-26T22:55:15.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.143+0000 7f6717e00640 1 -- 172.21.15.131:0/2858921430 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6724018ac0 con 0x55c4ee72a650 2024-05-26T22:55:15.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.143+0000 7f6717e00640 1 -- 172.21.15.131:0/2858921430 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 5709+0+0 (secure 0 0 0) 0x7f6724022af0 con 0x55c4ee72a650 2024-05-26T22:55:15.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.143+0000 7f673cc2c740 1 -- 172.21.15.131:0/2858921430 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f672c08a260 con 0x55c4ee72a650 2024-05-26T22:55:15.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.143+0000 7f6717e00640 1 -- 172.21.15.131:0/2858921430 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=98) ==== 24+0+0 (secure 0 0 0) 0x7f6724022cf0 con 0x55c4ee72a650 2024-05-26T22:55:15.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.143+0000 7f673cc2c740 10 client.8058 fetch_fsmap learned FSMap version 98 2024-05-26T22:55:15.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.143+0000 7f673cc2c740 1 -- 172.21.15.131:0/2858921430 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=98,monmap=2+}) -- 0x55c4ee743860 con 0x55c4ee72a650 2024-05-26T22:55:15.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.143+0000 7f6717e00640 1 -- 172.21.15.131:0/2858921430 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f672c08a260 con 0x55c4ee72a650 2024-05-26T22:55:15.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.143+0000 7f6717e00640 1 -- 172.21.15.131:0/2858921430 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 98) ==== 3534+0+0 (secure 0 0 0) 0x7f6724029050 con 0x55c4ee72a650 2024-05-26T22:55:15.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.143+0000 7f673cc2c740 10 client.8058 fetch_fsmap finished waiting for FSMap version 98 2024-05-26T22:55:15.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.143+0000 7f673cc2c740 10 client.8058 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-26T22:55:15.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.143+0000 7f673cc2c740 20 client.8058 populate_metadata read hostname 'smithi131' 2024-05-26T22:55:15.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.143+0000 7f673cc2c740 1 --2- 172.21.15.131:0/2858921430 >> [v2:172.21.15.187:6832/1314245822,v1:172.21.15.187:6833/1314245822] conn(0x55c4ee7be550 0x55c4ee716110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:15.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.143+0000 7f673cc2c740 4 client.8058 mds_command: new command op to 5243 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-26T22:55:15.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.143+0000 7f673cc2c740 1 -- 172.21.15.131:0/2858921430 --> [v2:172.21.15.187:6832/1314245822,v1:172.21.15.187:6833/1314245822] -- command(tid 0: {"prefix": "status"}) -- 0x55c4ee66a090 con 0x55c4ee7be550 2024-05-26T22:55:15.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.144+0000 7f672a000640 1 --2- 172.21.15.131:0/2858921430 >> [v2:172.21.15.187:6832/1314245822,v1:172.21.15.187:6833/1314245822] conn(0x55c4ee7be550 0x55c4ee716110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:15.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.145+0000 7f672a000640 1 --2- 172.21.15.131:0/2858921430 >> [v2:172.21.15.187:6832/1314245822,v1:172.21.15.187:6833/1314245822] conn(0x55c4ee7be550 0x55c4ee716110 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c648f50c267f5559 server_cookie=ca26e2e3555b1e76 in_seq=0 out_seq=0 2024-05-26T22:55:15.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.145+0000 7f6717e00640 10 client.8058 ms_handle_connect on v2:172.21.15.187:6832/1314245822 2024-05-26T22:55:15.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.146+0000 7f6717e00640 1 -- 172.21.15.131:0/2858921430 <== mds.2 v2:172.21.15.187:6832/1314245822 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x55c4ee66a090 con 0x55c4ee7be550 2024-05-26T22:55:15.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.146+0000 7f6717e00640 10 client.8058 handle_command_reply: tid=0 2024-05-26T22:55:15.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.146+0000 7f673cc2c740 1 client.8058 shutdown 2024-05-26T22:55:15.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.146+0000 7f673cc2c740 1 -- 172.21.15.131:0/2858921430 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4ee72a650 msgr2=0x55c4ee7255e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:15.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.146+0000 7f673cc2c740 1 --2- 172.21.15.131:0/2858921430 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4ee72a650 0x55c4ee7255e0 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f67240017e0 tx=0x7f6724018810 comp rx=0 tx=0).stop 2024-05-26T22:55:15.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.146+0000 7f673cc2c740 1 -- 172.21.15.131:0/2858921430 shutdown_connections 2024-05-26T22:55:15.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.146+0000 7f673cc2c740 1 --2- 172.21.15.131:0/2858921430 >> [v2:172.21.15.187:6832/1314245822,v1:172.21.15.187:6833/1314245822] conn(0x55c4ee7be550 0x55c4ee716110 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:15.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.146+0000 7f673cc2c740 1 --2- 172.21.15.131:0/2858921430 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55c4ee72ad90 0x55c4ee726870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:15.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.146+0000 7f673cc2c740 1 --2- 172.21.15.131:0/2858921430 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c4ee747c70 0x55c4ee7489a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:15.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.146+0000 7f6717e00640 0 client.8058 ms_handle_reset on v2:172.21.15.187:6832/1314245822 2024-05-26T22:55:15.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.146+0000 7f673cc2c740 1 --2- 172.21.15.131:0/2858921430 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4ee72a650 0x55c4ee7255e0 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:15.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.146+0000 7f673cc2c740 1 -- 172.21.15.131:0/2858921430 >> 172.21.15.131:0/2858921430 conn(0x55c4ee718540 msgr2=0x55c4ee6fb2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:15.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.146+0000 7f673cc2c740 1 -- 172.21.15.131:0/2858921430 shutdown_connections 2024-05-26T22:55:15.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.146+0000 7f673cc2c740 1 -- 172.21.15.131:0/2858921430 wait complete. 2024-05-26T22:55:15.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.146+0000 7f673cc2c740 20 client.8058 trim_cache size 0 max 16384 2024-05-26T22:55:15.148 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-26T22:55:15.148 INFO:teuthology.orchestra.run.smithi131.stdout: "cluster_fsid": "46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f", 2024-05-26T22:55:15.148 INFO:teuthology.orchestra.run.smithi131.stdout: "whoami": 2, 2024-05-26T22:55:15.148 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 5243, 2024-05-26T22:55:15.149 INFO:teuthology.orchestra.run.smithi131.stdout: "want_state": "up:active", 2024-05-26T22:55:15.149 INFO:teuthology.orchestra.run.smithi131.stdout: "state": "up:active", 2024-05-26T22:55:15.149 INFO:teuthology.orchestra.run.smithi131.stdout: "fs_name": "cephfs", 2024-05-26T22:55:15.149 INFO:teuthology.orchestra.run.smithi131.stdout: "rank_uptime": 1.9729797549999999, 2024-05-26T22:55:15.149 INFO:teuthology.orchestra.run.smithi131.stdout: "mdsmap_epoch": 98, 2024-05-26T22:55:15.149 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch": 95, 2024-05-26T22:55:15.149 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch_barrier": 95, 2024-05-26T22:55:15.149 INFO:teuthology.orchestra.run.smithi131.stdout: "uptime": 22.683177809 2024-05-26T22:55:15.149 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-26T22:55:15.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.147+0000 7f6739e00640 1 -- 172.21.15.131:0/3420362080 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f671006be60 msgr2=0x7f671006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:15.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.147+0000 7f6739e00640 1 --2- 172.21.15.131:0/3420362080 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f671006be60 0x7f671006e320 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f671c00ba00 tx=0x7f671c030000 comp rx=0 tx=0).stop 2024-05-26T22:55:15.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.147+0000 7f6739e00640 1 -- 172.21.15.131:0/3420362080 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f67340b1b90 msgr2=0x7f67340b2040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:15.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.147+0000 7f6739e00640 1 --2- 172.21.15.131:0/3420362080 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f67340b1b90 0x7f67340b2040 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f67340b3650 tx=0x7f672c00dc00 comp rx=0 tx=0).stop 2024-05-26T22:55:15.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.147+0000 7f6739e00640 1 -- 172.21.15.131:0/3420362080 shutdown_connections 2024-05-26T22:55:15.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.147+0000 7f6739e00640 1 --2- 172.21.15.131:0/3420362080 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f673414c280 0x7f673414d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:15.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.147+0000 7f6739e00640 1 --2- 172.21.15.131:0/3420362080 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f671006be60 0x7f671006e320 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:15.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.147+0000 7f6739e00640 1 --2- 172.21.15.131:0/3420362080 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f67340b1b90 0x7f67340b2040 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:15.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.147+0000 7f6739e00640 1 --2- 172.21.15.131:0/3420362080 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6718006220 0x7f67341f4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:15.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.147+0000 7f6739e00640 1 -- 172.21.15.131:0/3420362080 >> 172.21.15.131:0/3420362080 conn(0x7f67340b27c0 msgr2=0x7f67340cfa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:15.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.147+0000 7f6739e00640 1 -- 172.21.15.131:0/3420362080 shutdown_connections 2024-05-26T22:55:15.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.147+0000 7f6739e00640 1 -- 172.21.15.131:0/3420362080 wait complete. 2024-05-26T22:55:15.162 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-05-26T22:55:15.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.349+0000 7f3af6e00640 1 Processor -- start 2024-05-26T22:55:15.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.349+0000 7f3af6e00640 1 -- start start 2024-05-26T22:55:15.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.350+0000 7f3af6e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3af00b5450 0x7f3af00b43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:15.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.350+0000 7f3af6e00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f3af00d0460 0x7f3af006d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:15.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.350+0000 7f3af6e00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3af00b4900 con 0x7f3af006f420 2024-05-26T22:55:15.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.350+0000 7f3af6e00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f3af0160d90 con 0x7f3af00d0460 2024-05-26T22:55:15.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.350+0000 7f3af6e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3af0160180 con 0x7f3af00b5450 2024-05-26T22:55:15.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.351+0000 7f3ae7400640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f3af006f420 0x7f3af00cfd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:58442/0 (socket says 172.21.15.131:58442) 2024-05-26T22:55:15.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.351+0000 7f3ae7400640 1 -- 172.21.15.131:0/2610389826 learned_addr learned my addr 172.21.15.131:0/2610389826 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:55:15.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.351+0000 7f3af4e00640 1 --2- 172.21.15.131:0/2610389826 >> v2:172.21.15.187:3300/0 conn(0x7f3af00d0460 0x7f3af006d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:15.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.351+0000 7f3aef400640 1 -- 172.21.15.131:0/2610389826 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2535181405 0 0) 0x7f3af00b4900 con 0x7f3af006f420 2024-05-26T22:55:15.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.351+0000 7f3aefe00640 1 --2- 172.21.15.131:0/2610389826 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3af00b5450 0x7f3af00b43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:15.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.351+0000 7f3aef400640 1 -- 172.21.15.131:0/2610389826 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3ad4003520 con 0x7f3af006f420 2024-05-26T22:55:15.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.352+0000 7f3aef400640 1 -- 172.21.15.131:0/2610389826 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 194176504 0 0) 0x7f3ad4003520 con 0x7f3af006f420 2024-05-26T22:55:15.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.352+0000 7f3aef400640 1 -- 172.21.15.131:0/2610389826 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3af00b5450 msgr2=0x7f3af00b43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:15.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.352+0000 7f3aef400640 1 --2- 172.21.15.131:0/2610389826 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3af00b5450 0x7f3af00b43c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:15.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.352+0000 7f3aef400640 1 -- 172.21.15.131:0/2610389826 >> v2:172.21.15.187:3300/0 conn(0x7f3af00d0460 msgr2=0x7f3af006d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:15.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.352+0000 7f3aef400640 1 --2- 172.21.15.131:0/2610389826 >> v2:172.21.15.187:3300/0 conn(0x7f3af00d0460 0x7f3af006d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:15.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.352+0000 7f3aef400640 1 -- 172.21.15.131:0/2610389826 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3af0160460 con 0x7f3af006f420 2024-05-26T22:55:15.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.352+0000 7f3aef400640 1 -- 172.21.15.131:0/2610389826 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7f3ad8001670 con 0x7f3af006f420 2024-05-26T22:55:15.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.352+0000 7f3aef400640 1 -- 172.21.15.131:0/2610389826 >> v1:172.21.15.131:6789/0 conn(0x7f3af006f420 legacy=0x7f3af00cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:15.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.352+0000 7f3aef400640 1 --2- 172.21.15.131:0/2610389826 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3ad40039b0 0x7f3ad4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:15.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.352+0000 7f3aef400640 1 --2- 172.21.15.131:0/2610389826 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ad4004800 0x7f3ad4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:15.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.352+0000 7f3aef400640 1 --2- 172.21.15.131:0/2610389826 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3ad40051c0 0x7f3ad400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:15.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.352+0000 7f3aef400640 1 -- 172.21.15.131:0/2610389826 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f3af0160d90 con 0x7f3ad4004800 2024-05-26T22:55:15.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.352+0000 7f3aef400640 1 -- 172.21.15.131:0/2610389826 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3af0160180 con 0x7f3ad40051c0 2024-05-26T22:55:15.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.352+0000 7f3aef400640 1 -- 172.21.15.131:0/2610389826 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3ad400c060 con 0x7f3ad40039b0 2024-05-26T22:55:15.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.352+0000 7f3ae7400640 1 --2- 172.21.15.131:0/2610389826 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ad4004800 0x7f3ad4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:15.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.352+0000 7f3aefe00640 1 --2- 172.21.15.131:0/2610389826 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3ad40039b0 0x7f3ad4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:15.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.353+0000 7f3af4e00640 1 --2- 172.21.15.131:0/2610389826 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3ad40051c0 0x7f3ad400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:15.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.353+0000 7f3ae7400640 1 -- 172.21.15.131:0/2610389826 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3ad40039b0 msgr2=0x7f3ad4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:15.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.353+0000 7f3ae7400640 1 --2- 172.21.15.131:0/2610389826 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3ad40039b0 0x7f3ad4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:15.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.353+0000 7f3ae7400640 1 -- 172.21.15.131:0/2610389826 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3ad40051c0 msgr2=0x7f3ad400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:15.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.353+0000 7f3ae7400640 1 --2- 172.21.15.131:0/2610389826 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3ad40051c0 0x7f3ad400b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:15.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.353+0000 7f3ae7400640 1 -- 172.21.15.131:0/2610389826 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ad4003520 con 0x7f3ad4004800 2024-05-26T22:55:15.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.354+0000 7f3ae7400640 1 --2- 172.21.15.131:0/2610389826 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ad4004800 0x7f3ad4004c80 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f3ad800c190 tx=0x7f3ad4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e6d30594b4e794b server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:15.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.354+0000 7f3aef400640 1 -- 172.21.15.131:0/2610389826 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ad8007c60 con 0x7f3ad4004800 2024-05-26T22:55:15.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.354+0000 7f3aef400640 1 -- 172.21.15.131:0/2610389826 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ad8005af0 con 0x7f3ad4004800 2024-05-26T22:55:15.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.354+0000 7f3aef400640 1 -- 172.21.15.131:0/2610389826 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ad8007070 con 0x7f3ad4004800 2024-05-26T22:55:15.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.355+0000 7f3af6e00640 1 -- 172.21.15.131:0/2610389826 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ad4004800 msgr2=0x7f3ad4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:15.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.355+0000 7f3af6e00640 1 --2- 172.21.15.131:0/2610389826 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ad4004800 0x7f3ad4004c80 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f3ad800c190 tx=0x7f3ad4005830 comp rx=0 tx=0).stop 2024-05-26T22:55:15.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.355+0000 7f3af4e00640 1 -- 172.21.15.131:0/2610389826 reap_dead start 2024-05-26T22:55:15.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.355+0000 7f3af6e00640 1 -- 172.21.15.131:0/2610389826 shutdown_connections 2024-05-26T22:55:15.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.355+0000 7f3af6e00640 1 -- 172.21.15.131:0/2610389826 >> 172.21.15.131:0/2610389826 conn(0x7f3af006bc90 msgr2=0x7f3af00b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:15.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.355+0000 7f3af6e00640 1 -- 172.21.15.131:0/2610389826 shutdown_connections 2024-05-26T22:55:15.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.355+0000 7f3af6e00640 1 -- 172.21.15.131:0/2610389826 wait complete. 2024-05-26T22:55:15.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.357+0000 7f3af6e00640 1 Processor -- start 2024-05-26T22:55:15.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.357+0000 7f3af6e00640 1 -- start start 2024-05-26T22:55:15.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.358+0000 7f3af6e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ad4004800 0x7f3af01f2180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:15.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.358+0000 7f3af6e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3af009bb80 0x7f3af009c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:15.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.358+0000 7f3af6e00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3af00afcf0 0x7f3af00b0170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:15.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.358+0000 7f3af6e00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f3af00b5830 con 0x7f3ad4004800 2024-05-26T22:55:15.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.358+0000 7f3af6e00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3af0160a20 con 0x7f3af00afcf0 2024-05-26T22:55:15.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.358+0000 7f3af6e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3af015fde0 con 0x7f3af009bb80 2024-05-26T22:55:15.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.358+0000 7f3aefe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ad4004800 0x7f3af01f2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:15.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.359+0000 7f3aefe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ad4004800 0x7f3af01f2180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59754/0 (socket says 172.21.15.131:59754) 2024-05-26T22:55:15.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.359+0000 7f3aefe00640 1 -- 172.21.15.131:0/2365063752 learned_addr learned my addr 172.21.15.131:0/2365063752 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:55:15.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.359+0000 7f3ae7400640 1 --2- 172.21.15.131:0/2365063752 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3af009bb80 0x7f3af009c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:15.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.359+0000 7f3af4e00640 1 --2- 172.21.15.131:0/2365063752 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3af00afcf0 0x7f3af00b0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:15.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.359+0000 7f3aefe00640 1 -- 172.21.15.131:0/2365063752 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3af009bb80 msgr2=0x7f3af009c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:15.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.359+0000 7f3aefe00640 1 --2- 172.21.15.131:0/2365063752 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3af009bb80 0x7f3af009c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:15.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.359+0000 7f3aefe00640 1 -- 172.21.15.131:0/2365063752 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3af00afcf0 msgr2=0x7f3af00b0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:15.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.359+0000 7f3aefe00640 1 --2- 172.21.15.131:0/2365063752 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3af00afcf0 0x7f3af00b0170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:15.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.359+0000 7f3aefe00640 1 -- 172.21.15.131:0/2365063752 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3af00b06b0 con 0x7f3ad4004800 2024-05-26T22:55:15.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.359+0000 7f3aefe00640 1 --2- 172.21.15.131:0/2365063752 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ad4004800 0x7f3af01f2180 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f3ae000b6f0 tx=0x7f3ae000bbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:15.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.359+0000 7f3aecc00640 1 -- 172.21.15.131:0/2365063752 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae0018070 con 0x7f3ad4004800 2024-05-26T22:55:15.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.360+0000 7f3aecc00640 1 -- 172.21.15.131:0/2365063752 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ae0010830 con 0x7f3ad4004800 2024-05-26T22:55:15.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.360+0000 7f3af6e00640 1 -- 172.21.15.131:0/2365063752 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3af009c640 con 0x7f3ad4004800 2024-05-26T22:55:15.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.360+0000 7f3aecc00640 1 -- 172.21.15.131:0/2365063752 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae0010c20 con 0x7f3ad4004800 2024-05-26T22:55:15.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.360+0000 7f3af6e00640 1 -- 172.21.15.131:0/2365063752 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3af009cc00 con 0x7f3ad4004800 2024-05-26T22:55:15.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.362+0000 7f3aecc00640 1 -- 172.21.15.131:0/2365063752 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3ae001b070 con 0x7f3ad4004800 2024-05-26T22:55:15.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.363+0000 7f3aecc00640 1 --2- 172.21.15.131:0/2365063752 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f3acc06be60 0x7f3acc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:15.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.363+0000 7f3aecc00640 1 -- 172.21.15.131:0/2365063752 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 5709+0+0 (secure 0 0 0) 0x7f3ae008b140 con 0x7f3ad4004800 2024-05-26T22:55:15.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.363+0000 7f3ae7400640 1 --2- 172.21.15.131:0/2365063752 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f3acc06be60 0x7f3acc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:15.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.364+0000 7f3af9c6b740 1 Processor -- start 2024-05-26T22:55:15.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.364+0000 7f3ae7400640 1 --2- 172.21.15.131:0/2365063752 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f3acc06be60 0x7f3acc06e320 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f3ad8032000 tx=0x7f3ad8007000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:15.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.364+0000 7f3af9c6b740 1 -- start start 2024-05-26T22:55:15.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.364+0000 7f3af9c6b740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c429baf080 0x55c429b61ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:15.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.365+0000 7f3af9c6b740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c429b921a0 0x55c429b91520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:15.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.365+0000 7f3af9c6b740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55c429b91a60 0x55c429b66870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:15.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.365+0000 7f3af9c6b740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55c429ac5610 con 0x55c429baf080 2024-05-26T22:55:15.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.365+0000 7f3af9c6b740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55c429a6d5c0 con 0x55c429b921a0 2024-05-26T22:55:15.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.365+0000 7f3af9c6b740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x55c429ac15d0 con 0x55c429b91a60 2024-05-26T22:55:15.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.365+0000 7f3aefe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c429baf080 0x55c429b61ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:15.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.365+0000 7f3aefe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c429baf080 0x55c429b61ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59764/0 (socket says 172.21.15.131:59764) 2024-05-26T22:55:15.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.365+0000 7f3aefe00640 1 -- 172.21.15.131:0/2315686983 learned_addr learned my addr 172.21.15.131:0/2315686983 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:55:15.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.365+0000 7f3aefe00640 1 -- 172.21.15.131:0/2315686983 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55c429b91a60 msgr2=0x55c429b66870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:15.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.365+0000 7f3aefe00640 1 --2- 172.21.15.131:0/2315686983 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55c429b91a60 0x55c429b66870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:15.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.365+0000 7f3aefe00640 1 -- 172.21.15.131:0/2315686983 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c429b921a0 msgr2=0x55c429b91520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:15.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.365+0000 7f3aefe00640 1 --2- 172.21.15.131:0/2315686983 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c429b921a0 0x55c429b91520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:15.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.365+0000 7f3aefe00640 1 -- 172.21.15.131:0/2315686983 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4297d92d0 con 0x55c429baf080 2024-05-26T22:55:15.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.365+0000 7f3aefe00640 1 --2- 172.21.15.131:0/2315686983 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c429baf080 0x55c429b61ca0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f3ae00557b0 tx=0x7f3ae0058fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3ab861a7537bcc5 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:15.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.366+0000 7f3ae4c00640 1 -- 172.21.15.131:0/2315686983 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae0022040 con 0x55c429baf080 2024-05-26T22:55:15.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.366+0000 7f3ae4c00640 1 -- 172.21.15.131:0/2315686983 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ae0061f30 con 0x55c429baf080 2024-05-26T22:55:15.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.366+0000 7f3ae4c00640 1 -- 172.21.15.131:0/2315686983 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae0057650 con 0x55c429baf080 2024-05-26T22:55:15.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.366+0000 7f3af9c6b740 1 -- 172.21.15.131:0/2315686983 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c429baf080 msgr2=0x55c429b61ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:15.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.366+0000 7f3af9c6b740 1 --2- 172.21.15.131:0/2315686983 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c429baf080 0x55c429b61ca0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f3ae00557b0 tx=0x7f3ae0058fc0 comp rx=0 tx=0).stop 2024-05-26T22:55:15.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.366+0000 7f3af9c6b740 1 -- 172.21.15.131:0/2315686983 shutdown_connections 2024-05-26T22:55:15.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.366+0000 7f3af9c6b740 1 --2- 172.21.15.131:0/2315686983 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55c429b91a60 0x55c429b66870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:15.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.366+0000 7f3af9c6b740 1 --2- 172.21.15.131:0/2315686983 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c429b921a0 0x55c429b91520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:15.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.366+0000 7f3af9c6b740 1 --2- 172.21.15.131:0/2315686983 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c429baf080 0x55c429b61ca0 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:15.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.366+0000 7f3af9c6b740 1 -- 172.21.15.131:0/2315686983 >> 172.21.15.131:0/2315686983 conn(0x55c429b63f10 msgr2=0x55c429bac520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:15.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.366+0000 7f3af9c6b740 1 -- 172.21.15.131:0/2315686983 shutdown_connections 2024-05-26T22:55:15.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.366+0000 7f3af9c6b740 1 -- 172.21.15.131:0/2315686983 wait complete. 2024-05-26T22:55:15.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.367+0000 7f3af9c6b740 1 -- start start 2024-05-26T22:55:15.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.367+0000 7f3af9c6b740 1 Processor -- start 2024-05-26T22:55:15.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.368+0000 7f3af9c6b740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c429b921a0 0x55c429b82010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:15.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.368+0000 7f3af9c6b740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c429baf080 0x55c429bae220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:15.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.368+0000 7f3af9c6b740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55c429ba3a60 0x55c429ba3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:15.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.368+0000 7f3af9c6b740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55c429b65e30 con 0x55c429b921a0 2024-05-26T22:55:15.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.368+0000 7f3af9c6b740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55c429a80910 con 0x55c429baf080 2024-05-26T22:55:15.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.368+0000 7f3af9c6b740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x55c429b65220 con 0x55c429ba3a60 2024-05-26T22:55:15.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.368+0000 7f3af9c6b740 10 client.0 _ll_register_callbacks cb 0x55c429abc410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-26T22:55:15.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.369+0000 7f3aefe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c429b921a0 0x55c429b82010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:15.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.369+0000 7f3aefe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c429b921a0 0x55c429b82010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59768/0 (socket says 172.21.15.131:59768) 2024-05-26T22:55:15.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.369+0000 7f3aefe00640 1 -- 172.21.15.131:0/2870125232 learned_addr learned my addr 172.21.15.131:0/2870125232 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:55:15.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.369+0000 7f3aefe00640 1 -- 172.21.15.131:0/2870125232 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55c429ba3a60 msgr2=0x55c429ba3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:15.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.369+0000 7f3aefe00640 1 --2- 172.21.15.131:0/2870125232 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55c429ba3a60 0x55c429ba3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:15.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.369+0000 7f3aefe00640 1 -- 172.21.15.131:0/2870125232 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c429baf080 msgr2=0x55c429bae220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:15.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.369+0000 7f3aefe00640 1 --2- 172.21.15.131:0/2870125232 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c429baf080 0x55c429bae220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:15.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.369+0000 7f3aefe00640 1 -- 172.21.15.131:0/2870125232 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4298a36b0 con 0x55c429b921a0 2024-05-26T22:55:15.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.369+0000 7f3aefe00640 1 --2- 172.21.15.131:0/2870125232 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c429b921a0 0x55c429b82010 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f3ae0059050 tx=0x7f3ae001e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=734d76c4f82ce276 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:15.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.369+0000 7f3add600640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-26T22:55:15.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.369+0000 7f3add600640 1 -- 172.21.15.131:0/2870125232 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae0022040 con 0x55c429b921a0 2024-05-26T22:55:15.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.369+0000 7f3add600640 1 -- 172.21.15.131:0/2870125232 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 5709+0+0 (secure 0 0 0) 0x7f3ae0089270 con 0x55c429b921a0 2024-05-26T22:55:15.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.369+0000 7f3af9c6b740 1 -- 172.21.15.131:0/2870125232 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ae0061f30 con 0x55c429b921a0 2024-05-26T22:55:15.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.370+0000 7f3add600640 1 -- 172.21.15.131:0/2870125232 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=98) ==== 24+0+0 (secure 0 0 0) 0x7f3ae0078280 con 0x55c429b921a0 2024-05-26T22:55:15.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.370+0000 7f3af9c6b740 10 client.8070 fetch_fsmap learned FSMap version 98 2024-05-26T22:55:15.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.370+0000 7f3af9c6b740 1 -- 172.21.15.131:0/2870125232 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=98,monmap=2+}) -- 0x55c429b947e0 con 0x55c429b921a0 2024-05-26T22:55:15.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.370+0000 7f3add600640 1 -- 172.21.15.131:0/2870125232 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ae0061f30 con 0x55c429b921a0 2024-05-26T22:55:15.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.370+0000 7f3add600640 1 -- 172.21.15.131:0/2870125232 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 98) ==== 3534+0+0 (secure 0 0 0) 0x7f3ae0071180 con 0x55c429b921a0 2024-05-26T22:55:15.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.370+0000 7f3af9c6b740 10 client.8070 fetch_fsmap finished waiting for FSMap version 98 2024-05-26T22:55:15.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.370+0000 7f3af9c6b740 10 client.8070 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-26T22:55:15.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.370+0000 7f3af9c6b740 20 client.8070 populate_metadata read hostname 'smithi131' 2024-05-26T22:55:15.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.370+0000 7f3af9c6b740 1 --2- 172.21.15.131:0/2870125232 >> [v2:172.21.15.187:6834/521878375,v1:172.21.15.187:6835/521878375] conn(0x55c429bbc650 0x55c429bbca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:15.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.370+0000 7f3af9c6b740 4 client.8070 mds_command: new command op to 5246 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-26T22:55:15.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.370+0000 7f3af9c6b740 1 -- 172.21.15.131:0/2870125232 --> [v2:172.21.15.187:6834/521878375,v1:172.21.15.187:6835/521878375] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c429ad2d40 con 0x55c429bbc650 2024-05-26T22:55:15.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.371+0000 7f3ae7400640 1 --2- 172.21.15.131:0/2870125232 >> [v2:172.21.15.187:6834/521878375,v1:172.21.15.187:6835/521878375] conn(0x55c429bbc650 0x55c429bbca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:15.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.372+0000 7f3ae7400640 1 --2- 172.21.15.131:0/2870125232 >> [v2:172.21.15.187:6834/521878375,v1:172.21.15.187:6835/521878375] conn(0x55c429bbc650 0x55c429bbca30 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1ad0cdc619a86c0a server_cookie=f89e33e8d24f0d82 in_seq=0 out_seq=0 2024-05-26T22:55:15.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.372+0000 7f3add600640 10 client.8070 ms_handle_connect on v2:172.21.15.187:6834/521878375 2024-05-26T22:55:15.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.374+0000 7f3add600640 1 -- 172.21.15.131:0/2870125232 <== mds.1 v2:172.21.15.187:6834/521878375 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c429ad2d40 con 0x55c429bbc650 2024-05-26T22:55:15.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.374+0000 7f3add600640 10 client.8070 handle_command_reply: tid=0 2024-05-26T22:55:15.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.374+0000 7f3af9c6b740 1 client.8070 shutdown 2024-05-26T22:55:15.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.374+0000 7f3af9c6b740 1 -- 172.21.15.131:0/2870125232 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c429b921a0 msgr2=0x55c429b82010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:15.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.374+0000 7f3af9c6b740 1 --2- 172.21.15.131:0/2870125232 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c429b921a0 0x55c429b82010 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f3ae0059050 tx=0x7f3ae001e000 comp rx=0 tx=0).stop 2024-05-26T22:55:15.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.374+0000 7f3af9c6b740 1 -- 172.21.15.131:0/2870125232 shutdown_connections 2024-05-26T22:55:15.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.374+0000 7f3af9c6b740 1 --2- 172.21.15.131:0/2870125232 >> [v2:172.21.15.187:6834/521878375,v1:172.21.15.187:6835/521878375] conn(0x55c429bbc650 0x55c429bbca30 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:15.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.374+0000 7f3af9c6b740 1 --2- 172.21.15.131:0/2870125232 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55c429ba3a60 0x55c429ba3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:15.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.374+0000 7f3af9c6b740 1 --2- 172.21.15.131:0/2870125232 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c429baf080 0x55c429bae220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:15.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.374+0000 7f3af9c6b740 1 --2- 172.21.15.131:0/2870125232 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c429b921a0 0x55c429b82010 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:15.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.374+0000 7f3add600640 0 client.8070 ms_handle_reset on v2:172.21.15.187:6834/521878375 2024-05-26T22:55:15.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.374+0000 7f3af9c6b740 1 -- 172.21.15.131:0/2870125232 >> 172.21.15.131:0/2870125232 conn(0x55c429b91a60 msgr2=0x55c429b623d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:15.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.375+0000 7f3af9c6b740 1 -- 172.21.15.131:0/2870125232 shutdown_connections 2024-05-26T22:55:15.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.375+0000 7f3af9c6b740 1 -- 172.21.15.131:0/2870125232 wait complete. 2024-05-26T22:55:15.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.375+0000 7f3af9c6b740 20 client.8070 trim_cache size 0 max 16384 2024-05-26T22:55:15.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.395+0000 7f3af9c6b740 1 Processor -- start 2024-05-26T22:55:15.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.395+0000 7f3af9c6b740 1 -- start start 2024-05-26T22:55:15.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.396+0000 7f3af9c6b740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55c429b921a0 0x55c429bac510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:15.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.396+0000 7f3af9c6b740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c429ba3a60 0x55c429b61bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:15.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.396+0000 7f3af9c6b740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c429baf080 0x55c429b62130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:15.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.396+0000 7f3af9c6b740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f3af0071540 con 0x55c429baf080 2024-05-26T22:55:15.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.396+0000 7f3af9c6b740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3af0071c10 con 0x55c429ba3a60 2024-05-26T22:55:15.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.396+0000 7f3af9c6b740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3af0147900 con 0x55c429b921a0 2024-05-26T22:55:15.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.396+0000 7f3af4e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c429baf080 0x55c429b62130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:15.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.396+0000 7f3af4e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c429baf080 0x55c429b62130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59772/0 (socket says 172.21.15.131:59772) 2024-05-26T22:55:15.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.396+0000 7f3af4e00640 1 -- 172.21.15.131:0/856069721 learned_addr learned my addr 172.21.15.131:0/856069721 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:55:15.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.396+0000 7f3ae7400640 1 --2- 172.21.15.131:0/856069721 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c429ba3a60 0x55c429b61bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:15.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.397+0000 7f3aefe00640 1 --2- 172.21.15.131:0/856069721 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55c429b921a0 0x55c429bac510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:15.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.397+0000 7f3af4e00640 1 -- 172.21.15.131:0/856069721 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55c429b921a0 msgr2=0x55c429bac510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:15.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.397+0000 7f3af4e00640 1 --2- 172.21.15.131:0/856069721 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55c429b921a0 0x55c429bac510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:15.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.397+0000 7f3af4e00640 1 -- 172.21.15.131:0/856069721 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c429ba3a60 msgr2=0x55c429b61bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:15.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.397+0000 7f3af4e00640 1 --2- 172.21.15.131:0/856069721 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c429ba3a60 0x55c429b61bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:15.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.397+0000 7f3af4e00640 1 -- 172.21.15.131:0/856069721 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c429b59b90 con 0x55c429baf080 2024-05-26T22:55:15.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.397+0000 7f3af4e00640 1 --2- 172.21.15.131:0/856069721 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c429baf080 0x55c429b62130 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f3af00b3010 tx=0x7f3ae8004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb8581fc861e9607 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:15.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.397+0000 7f3adfe00640 1 -- 172.21.15.131:0/856069721 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae8004860 con 0x55c429baf080 2024-05-26T22:55:15.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.397+0000 7f3adfe00640 1 -- 172.21.15.131:0/856069721 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ae8019070 con 0x55c429baf080 2024-05-26T22:55:15.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.397+0000 7f3adfe00640 1 -- 172.21.15.131:0/856069721 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae8014650 con 0x55c429baf080 2024-05-26T22:55:15.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.398+0000 7f3af9c6b740 1 -- 172.21.15.131:0/856069721 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c429baf080 msgr2=0x55c429b62130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:15.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.398+0000 7f3af9c6b740 1 --2- 172.21.15.131:0/856069721 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c429baf080 0x55c429b62130 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f3af00b3010 tx=0x7f3ae8004610 comp rx=0 tx=0).stop 2024-05-26T22:55:15.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.398+0000 7f3af9c6b740 1 -- 172.21.15.131:0/856069721 shutdown_connections 2024-05-26T22:55:15.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.398+0000 7f3af9c6b740 1 --2- 172.21.15.131:0/856069721 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c429baf080 0x55c429b62130 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:15.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.398+0000 7f3af9c6b740 1 --2- 172.21.15.131:0/856069721 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c429ba3a60 0x55c429b61bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:15.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.398+0000 7f3af9c6b740 1 --2- 172.21.15.131:0/856069721 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55c429b921a0 0x55c429bac510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:15.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.398+0000 7f3af9c6b740 1 -- 172.21.15.131:0/856069721 >> 172.21.15.131:0/856069721 conn(0x55c429b91a60 msgr2=0x55c429b81e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:15.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.398+0000 7f3af9c6b740 1 -- 172.21.15.131:0/856069721 shutdown_connections 2024-05-26T22:55:15.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.398+0000 7f3af9c6b740 1 -- 172.21.15.131:0/856069721 wait complete. 2024-05-26T22:55:15.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.399+0000 7f3af9c6b740 1 -- start start 2024-05-26T22:55:15.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.399+0000 7f3af9c6b740 1 Processor -- start 2024-05-26T22:55:15.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.400+0000 7f3af9c6b740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55c429b921a0 0x55c429c29770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:15.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.400+0000 7f3af9c6b740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c429ba3a60 0x55c429bc6050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:15.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.400+0000 7f3af9c6b740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c429baf080 0x55c429b92690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:15.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.400+0000 7f3af9c6b740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55c429b65e30 con 0x55c429baf080 2024-05-26T22:55:15.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.400+0000 7f3af9c6b740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55c429b65220 con 0x55c429ba3a60 2024-05-26T22:55:15.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.400+0000 7f3af9c6b740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x55c429a80910 con 0x55c429b921a0 2024-05-26T22:55:15.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.400+0000 7f3af9c6b740 10 client.0 _ll_register_callbacks cb 0x7f3ae0058cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-26T22:55:15.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.400+0000 7f3af4e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c429baf080 0x55c429b92690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:15.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.400+0000 7f3af4e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c429baf080 0x55c429b92690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59782/0 (socket says 172.21.15.131:59782) 2024-05-26T22:55:15.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.400+0000 7f3af4e00640 1 -- 172.21.15.131:0/3868829190 learned_addr learned my addr 172.21.15.131:0/3868829190 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:55:15.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.401+0000 7f3af4e00640 1 -- 172.21.15.131:0/3868829190 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55c429b921a0 msgr2=0x55c429c29770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:15.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.401+0000 7f3af4e00640 1 --2- 172.21.15.131:0/3868829190 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55c429b921a0 0x55c429c29770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:15.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.401+0000 7f3af4e00640 1 -- 172.21.15.131:0/3868829190 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c429ba3a60 msgr2=0x55c429bc6050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:15.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.401+0000 7f3af4e00640 1 --2- 172.21.15.131:0/3868829190 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c429ba3a60 0x55c429bc6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:15.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.401+0000 7f3af4e00640 1 -- 172.21.15.131:0/3868829190 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c429bc54f0 con 0x55c429baf080 2024-05-26T22:55:15.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.401+0000 7f3af4e00640 1 --2- 172.21.15.131:0/3868829190 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c429baf080 0x55c429b92690 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f3af0073660 tx=0x7f3ae8007040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74143c9768dcc99b server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:15.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.401+0000 7f3add600640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-26T22:55:15.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.401+0000 7f3add600640 1 -- 172.21.15.131:0/3868829190 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae8003de0 con 0x55c429baf080 2024-05-26T22:55:15.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.401+0000 7f3add600640 1 -- 172.21.15.131:0/3868829190 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 5709+0+0 (secure 0 0 0) 0x7f3ae8014b70 con 0x55c429baf080 2024-05-26T22:55:15.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.401+0000 7f3af9c6b740 1 -- 172.21.15.131:0/3868829190 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ae8019070 con 0x55c429baf080 2024-05-26T22:55:15.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.401+0000 7f3add600640 1 -- 172.21.15.131:0/3868829190 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=98) ==== 24+0+0 (secure 0 0 0) 0x7f3ae8030050 con 0x55c429baf080 2024-05-26T22:55:15.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.402+0000 7f3af9c6b740 10 client.8076 fetch_fsmap learned FSMap version 98 2024-05-26T22:55:15.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.402+0000 7f3af9c6b740 1 -- 172.21.15.131:0/3868829190 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=98,monmap=2+}) -- 0x55c429bc5790 con 0x55c429baf080 2024-05-26T22:55:15.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.402+0000 7f3add600640 1 -- 172.21.15.131:0/3868829190 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ae8019070 con 0x55c429baf080 2024-05-26T22:55:15.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.402+0000 7f3add600640 1 -- 172.21.15.131:0/3868829190 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 98) ==== 3534+0+0 (secure 0 0 0) 0x7f3ae8033020 con 0x55c429baf080 2024-05-26T22:55:15.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.402+0000 7f3af9c6b740 10 client.8076 fetch_fsmap finished waiting for FSMap version 98 2024-05-26T22:55:15.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.402+0000 7f3af9c6b740 10 client.8076 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-26T22:55:15.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.402+0000 7f3af9c6b740 20 client.8076 populate_metadata read hostname 'smithi131' 2024-05-26T22:55:15.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.402+0000 7f3af9c6b740 1 --2- 172.21.15.131:0/3868829190 >> [v2:172.21.15.187:6834/521878375,v1:172.21.15.187:6835/521878375] conn(0x55c429b61bf0 0x55c429b61fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:15.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.402+0000 7f3af9c6b740 4 client.8076 mds_command: new command op to 5246 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-26T22:55:15.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.402+0000 7f3af9c6b740 1 -- 172.21.15.131:0/3868829190 --> [v2:172.21.15.187:6834/521878375,v1:172.21.15.187:6835/521878375] -- command(tid 0: {"prefix": "status"}) -- 0x55c429acd090 con 0x55c429b61bf0 2024-05-26T22:55:15.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.403+0000 7f3aefe00640 1 --2- 172.21.15.131:0/3868829190 >> [v2:172.21.15.187:6834/521878375,v1:172.21.15.187:6835/521878375] conn(0x55c429b61bf0 0x55c429b61fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:15.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.404+0000 7f3aefe00640 1 --2- 172.21.15.131:0/3868829190 >> [v2:172.21.15.187:6834/521878375,v1:172.21.15.187:6835/521878375] conn(0x55c429b61bf0 0x55c429b61fd0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d833c69e7d9df393 server_cookie=54572c047cba63e9 in_seq=0 out_seq=0 2024-05-26T22:55:15.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.404+0000 7f3add600640 10 client.8076 ms_handle_connect on v2:172.21.15.187:6834/521878375 2024-05-26T22:55:15.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.404+0000 7f3add600640 1 -- 172.21.15.131:0/3868829190 <== mds.1 v2:172.21.15.187:6834/521878375 1 ==== command_reply(tid 0: 0 ) ==== 8+0+318 (crc 0 0 0) 0x55c429acd090 con 0x55c429b61bf0 2024-05-26T22:55:15.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.404+0000 7f3add600640 10 client.8076 handle_command_reply: tid=0 2024-05-26T22:55:15.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.404+0000 7f3af9c6b740 1 client.8076 shutdown 2024-05-26T22:55:15.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.404+0000 7f3af9c6b740 1 -- 172.21.15.131:0/3868829190 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c429baf080 msgr2=0x55c429b92690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:15.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.404+0000 7f3af9c6b740 1 --2- 172.21.15.131:0/3868829190 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c429baf080 0x55c429b92690 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f3af0073660 tx=0x7f3ae8007040 comp rx=0 tx=0).stop 2024-05-26T22:55:15.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.404+0000 7f3af9c6b740 1 -- 172.21.15.131:0/3868829190 shutdown_connections 2024-05-26T22:55:15.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.404+0000 7f3af9c6b740 1 --2- 172.21.15.131:0/3868829190 >> [v2:172.21.15.187:6834/521878375,v1:172.21.15.187:6835/521878375] conn(0x55c429b61bf0 0x55c429b61fd0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:15.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.404+0000 7f3af9c6b740 1 --2- 172.21.15.131:0/3868829190 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c429baf080 0x55c429b92690 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:15.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.404+0000 7f3af9c6b740 1 --2- 172.21.15.131:0/3868829190 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c429ba3a60 0x55c429bc6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:15.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.404+0000 7f3af9c6b740 1 --2- 172.21.15.131:0/3868829190 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55c429b921a0 0x55c429c29770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:15.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.404+0000 7f3add600640 0 client.8076 ms_handle_reset on v2:172.21.15.187:6834/521878375 2024-05-26T22:55:15.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.404+0000 7f3af9c6b740 1 -- 172.21.15.131:0/3868829190 >> 172.21.15.131:0/3868829190 conn(0x55c429b91a60 msgr2=0x55c429bcfed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:15.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.405+0000 7f3af9c6b740 1 -- 172.21.15.131:0/3868829190 shutdown_connections 2024-05-26T22:55:15.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.405+0000 7f3af9c6b740 1 -- 172.21.15.131:0/3868829190 wait complete. 2024-05-26T22:55:15.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.405+0000 7f3af9c6b740 20 client.8076 trim_cache size 0 max 16384 2024-05-26T22:55:15.407 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-26T22:55:15.407 INFO:teuthology.orchestra.run.smithi131.stdout: "cluster_fsid": "46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f", 2024-05-26T22:55:15.407 INFO:teuthology.orchestra.run.smithi131.stdout: "whoami": 1, 2024-05-26T22:55:15.407 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 5246, 2024-05-26T22:55:15.407 INFO:teuthology.orchestra.run.smithi131.stdout: "want_state": "up:active", 2024-05-26T22:55:15.407 INFO:teuthology.orchestra.run.smithi131.stdout: "state": "up:active", 2024-05-26T22:55:15.408 INFO:teuthology.orchestra.run.smithi131.stdout: "fs_name": "cephfs", 2024-05-26T22:55:15.408 INFO:teuthology.orchestra.run.smithi131.stdout: "rank_uptime": 3.243941285, 2024-05-26T22:55:15.408 INFO:teuthology.orchestra.run.smithi131.stdout: "mdsmap_epoch": 98, 2024-05-26T22:55:15.408 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch": 95, 2024-05-26T22:55:15.408 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch_barrier": 95, 2024-05-26T22:55:15.408 INFO:teuthology.orchestra.run.smithi131.stdout: "uptime": 22.938679042 2024-05-26T22:55:15.408 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-26T22:55:15.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.405+0000 7f3af6e00640 1 -- 172.21.15.131:0/2365063752 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f3acc06be60 msgr2=0x7f3acc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:15.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.405+0000 7f3af6e00640 1 --2- 172.21.15.131:0/2365063752 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f3acc06be60 0x7f3acc06e320 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f3ad8032000 tx=0x7f3ad8007000 comp rx=0 tx=0).stop 2024-05-26T22:55:15.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.405+0000 7f3af6e00640 1 -- 172.21.15.131:0/2365063752 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ad4004800 msgr2=0x7f3af01f2180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:15.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.405+0000 7f3af6e00640 1 --2- 172.21.15.131:0/2365063752 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ad4004800 0x7f3af01f2180 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f3ae000b6f0 tx=0x7f3ae000bbb0 comp rx=0 tx=0).stop 2024-05-26T22:55:15.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.405+0000 7f3af6e00640 1 -- 172.21.15.131:0/2365063752 shutdown_connections 2024-05-26T22:55:15.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.405+0000 7f3af6e00640 1 --2- 172.21.15.131:0/2365063752 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3af009bb80 0x7f3af009c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:15.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.405+0000 7f3af6e00640 1 --2- 172.21.15.131:0/2365063752 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f3acc06be60 0x7f3acc06e320 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:15.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.405+0000 7f3af6e00640 1 --2- 172.21.15.131:0/2365063752 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3af00afcf0 0x7f3af00b0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:15.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.405+0000 7f3af6e00640 1 --2- 172.21.15.131:0/2365063752 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ad4004800 0x7f3af01f2180 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:15.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.405+0000 7f3af6e00640 1 -- 172.21.15.131:0/2365063752 >> 172.21.15.131:0/2365063752 conn(0x7f3af006bc90 msgr2=0x7f3af015da80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:15.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.406+0000 7f3af6e00640 1 -- 172.21.15.131:0/2365063752 shutdown_connections 2024-05-26T22:55:15.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.406+0000 7f3af6e00640 1 -- 172.21.15.131:0/2365063752 wait complete. 2024-05-26T22:55:15.422 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-05-26T22:55:15.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.611+0000 7f59fb200640 1 Processor -- start 2024-05-26T22:55:15.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.612+0000 7f59fb200640 1 -- start start 2024-05-26T22:55:15.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.613+0000 7f59fb200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f59fc0b9f20 0x7f59fc0d22b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:15.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.613+0000 7f59fb200640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59fc0b79f0 con 0x7f59fc0d27f0 2024-05-26T22:55:15.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.613+0000 7f59fb200640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59fc15e720 con 0x7f59fc15e340 2024-05-26T22:55:15.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.613+0000 7f59fb200640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f59fc0b88b0 con 0x7f59fc0b9f20 2024-05-26T22:55:15.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.614+0000 7f59f9400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f59fc0b9f20 0x7f59fc0d22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:15.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.614+0000 7f59fa800640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f59fc0d27f0 0x7f59fc0b8d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:58450/0 (socket says 172.21.15.131:58450) 2024-05-26T22:55:15.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.614+0000 7f59fa800640 1 -- 172.21.15.131:0/1878914384 learned_addr learned my addr 172.21.15.131:0/1878914384 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:55:15.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.614+0000 7f59f9400640 1 -- 172.21.15.131:0/1878914384 >> v1:172.21.15.187:6789/0 conn(0x7f59fc15e340 legacy=0x7f59fc0b72e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:15.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.614+0000 7f59f9400640 1 -- 172.21.15.131:0/1878914384 >> v1:172.21.15.131:6789/0 conn(0x7f59fc0d27f0 legacy=0x7f59fc0b8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:15.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.614+0000 7f59f9400640 1 -- 172.21.15.131:0/1878914384 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59fc0b7ca0 con 0x7f59fc0b9f20 2024-05-26T22:55:15.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.614+0000 7f59f8a00640 1 -- 172.21.15.131:0/1878914384 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2998358959 0 0) 0x7f59fc0b79f0 con 0x7f59fc0d27f0 2024-05-26T22:55:15.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.615+0000 7f59f9400640 1 --2- 172.21.15.131:0/1878914384 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59fc0b9f20 0x7f59fc0d22b0 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f59e4009d90 tx=0x7f59e402a780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4336b1466ab3775 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:15.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.616+0000 7f59f8a00640 1 -- 172.21.15.131:0/1878914384 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59e4037070 con 0x7f59fc0b9f20 2024-05-26T22:55:15.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.616+0000 7f59f8a00640 1 -- 172.21.15.131:0/1878914384 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59e4007740 con 0x7f59fc0b9f20 2024-05-26T22:55:15.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.616+0000 7f59f8a00640 1 -- 172.21.15.131:0/1878914384 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59e4007b10 con 0x7f59fc0b9f20 2024-05-26T22:55:15.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.616+0000 7f59fb200640 1 -- 172.21.15.131:0/1878914384 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59fc0b9f20 msgr2=0x7f59fc0d22b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:15.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.616+0000 7f59fb200640 1 --2- 172.21.15.131:0/1878914384 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59fc0b9f20 0x7f59fc0d22b0 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f59e4009d90 tx=0x7f59e402a780 comp rx=0 tx=0).stop 2024-05-26T22:55:15.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.616+0000 7f59fb200640 1 -- 172.21.15.131:0/1878914384 shutdown_connections 2024-05-26T22:55:15.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.616+0000 7f59fb200640 1 --2- 172.21.15.131:0/1878914384 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59fc0b9f20 0x7f59fc0d22b0 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:15.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.616+0000 7f59fb200640 1 -- 172.21.15.131:0/1878914384 >> 172.21.15.131:0/1878914384 conn(0x7f59fc072e60 msgr2=0x7f59fc073270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:15.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.616+0000 7f59fb200640 1 -- 172.21.15.131:0/1878914384 shutdown_connections 2024-05-26T22:55:15.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.616+0000 7f59fb200640 1 -- 172.21.15.131:0/1878914384 wait complete. 2024-05-26T22:55:15.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.618+0000 7f59fb200640 1 Processor -- start 2024-05-26T22:55:15.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.618+0000 7f59fb200640 1 -- start start 2024-05-26T22:55:15.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.619+0000 7f59fb200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f59fc0b9f20 0x7f59fc1f6d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:15.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.619+0000 7f59fb200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59fc0d27f0 0x7f59fc0cf110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:15.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.619+0000 7f59fb200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f59fc15e340 0x7f59fc0cf650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:15.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.619+0000 7f59fb200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f59fc0b8540 con 0x7f59fc0d27f0 2024-05-26T22:55:15.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.619+0000 7f59fb200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f59fc0b8260 con 0x7f59fc15e340 2024-05-26T22:55:15.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.619+0000 7f59fb200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f59fc0b9450 con 0x7f59fc0b9f20 2024-05-26T22:55:15.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.619+0000 7f59f9400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59fc0d27f0 0x7f59fc0cf110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:15.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.619+0000 7f59f9400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59fc0d27f0 0x7f59fc0cf110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59806/0 (socket says 172.21.15.131:59806) 2024-05-26T22:55:15.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.619+0000 7f59f9400640 1 -- 172.21.15.131:0/2502081632 learned_addr learned my addr 172.21.15.131:0/2502081632 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:55:15.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.620+0000 7f59f9400640 1 -- 172.21.15.131:0/2502081632 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f59fc0b9f20 msgr2=0x7f59fc1f6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:15.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.620+0000 7f59f9400640 1 --2- 172.21.15.131:0/2502081632 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f59fc0b9f20 0x7f59fc1f6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:15.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.620+0000 7f59f9400640 1 -- 172.21.15.131:0/2502081632 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f59fc15e340 msgr2=0x7f59fc0cf650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:15.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.620+0000 7f59f9400640 1 --2- 172.21.15.131:0/2502081632 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f59fc15e340 0x7f59fc0cf650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:15.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.620+0000 7f59f9400640 1 -- 172.21.15.131:0/2502081632 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59fc15bd50 con 0x7f59fc0d27f0 2024-05-26T22:55:15.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.620+0000 7f59f9400640 1 --2- 172.21.15.131:0/2502081632 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59fc0d27f0 0x7f59fc0cf110 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f59e4035040 tx=0x7f59e4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:15.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.620+0000 7f59f2000640 1 -- 172.21.15.131:0/2502081632 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59e4042020 con 0x7f59fc0d27f0 2024-05-26T22:55:15.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.620+0000 7f59f2000640 1 -- 172.21.15.131:0/2502081632 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59e4032a70 con 0x7f59fc0d27f0 2024-05-26T22:55:15.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.620+0000 7f59f2000640 1 -- 172.21.15.131:0/2502081632 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59e4037040 con 0x7f59fc0d27f0 2024-05-26T22:55:15.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.620+0000 7f59fb200640 1 -- 172.21.15.131:0/2502081632 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59fc0cdda0 con 0x7f59fc0d27f0 2024-05-26T22:55:15.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.621+0000 7f59fb200640 1 -- 172.21.15.131:0/2502081632 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59fc0ce360 con 0x7f59fc0d27f0 2024-05-26T22:55:15.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.623+0000 7f59f2000640 1 -- 172.21.15.131:0/2502081632 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f59e4005d10 con 0x7f59fc0d27f0 2024-05-26T22:55:15.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.624+0000 7f59f2000640 1 --2- 172.21.15.131:0/2502081632 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f59d806be30 0x7f59d806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:15.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.625+0000 7f59f2000640 1 -- 172.21.15.131:0/2502081632 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 5709+0+0 (secure 0 0 0) 0x7f59e40ac0c0 con 0x7f59fc0d27f0 2024-05-26T22:55:15.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.625+0000 7f59f9e00640 1 --2- 172.21.15.131:0/2502081632 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f59d806be30 0x7f59d806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:15.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.625+0000 7f5a04083740 1 Processor -- start 2024-05-26T22:55:15.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.625+0000 7f5a04083740 1 -- start start 2024-05-26T22:55:15.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.625+0000 7f59f9e00640 1 --2- 172.21.15.131:0/2502081632 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f59d806be30 0x7f59d806e2f0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f59ec000f60 tx=0x7f59ec0079b0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:15.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.626+0000 7f5a04083740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55db76394e50 0x55db76347a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:15.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.626+0000 7f5a04083740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55db76377f70 0x55db763772f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:15.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.626+0000 7f5a04083740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55db76377830 0x55db7634c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:15.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.626+0000 7f5a04083740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55db762af610 con 0x55db76394e50 2024-05-26T22:55:15.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.626+0000 7f5a04083740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55db762575c0 con 0x55db76377f70 2024-05-26T22:55:15.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.626+0000 7f5a04083740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x55db762ab5d0 con 0x55db76377830 2024-05-26T22:55:15.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.626+0000 7f59f9e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55db76394e50 0x55db76347a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:15.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.626+0000 7f59f9e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55db76394e50 0x55db76347a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59816/0 (socket says 172.21.15.131:59816) 2024-05-26T22:55:15.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.626+0000 7f59f9e00640 1 -- 172.21.15.131:0/3558469502 learned_addr learned my addr 172.21.15.131:0/3558469502 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:55:15.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.627+0000 7f59fa800640 1 --2- 172.21.15.131:0/3558469502 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55db76377830 0x55db7634c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:15.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.627+0000 7f59f9e00640 1 -- 172.21.15.131:0/3558469502 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55db76377830 msgr2=0x55db7634c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:15.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.627+0000 7f59f9e00640 1 --2- 172.21.15.131:0/3558469502 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55db76377830 0x55db7634c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:15.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.627+0000 7f59f9e00640 1 -- 172.21.15.131:0/3558469502 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55db76377f70 msgr2=0x55db763772f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:15.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.627+0000 7f59f9400640 1 --2- 172.21.15.131:0/3558469502 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55db76377f70 0x55db763772f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:15.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.627+0000 7f59f9e00640 1 --2- 172.21.15.131:0/3558469502 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55db76377f70 0x55db763772f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:15.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.627+0000 7f59f9e00640 1 -- 172.21.15.131:0/3558469502 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db75fc32d0 con 0x55db76394e50 2024-05-26T22:55:15.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.627+0000 7f59fa800640 1 --2- 172.21.15.131:0/3558469502 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55db76377830 0x55db7634c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:55:15.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.627+0000 7f59f9e00640 1 --2- 172.21.15.131:0/3558469502 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55db76394e50 0x55db76347a70 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f59ec0168b0 tx=0x7f59ec00f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6516c01c5ce38c07 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:15.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.627+0000 7f59eaa00640 1 -- 172.21.15.131:0/3558469502 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59ec0162e0 con 0x55db76394e50 2024-05-26T22:55:15.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.627+0000 7f59eaa00640 1 -- 172.21.15.131:0/3558469502 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59ec005110 con 0x55db76394e50 2024-05-26T22:55:15.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.627+0000 7f59eaa00640 1 -- 172.21.15.131:0/3558469502 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59ec0138b0 con 0x55db76394e50 2024-05-26T22:55:15.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.628+0000 7f5a04083740 1 -- 172.21.15.131:0/3558469502 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55db76394e50 msgr2=0x55db76347a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:15.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.628+0000 7f5a04083740 1 --2- 172.21.15.131:0/3558469502 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55db76394e50 0x55db76347a70 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f59ec0168b0 tx=0x7f59ec00f000 comp rx=0 tx=0).stop 2024-05-26T22:55:15.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.628+0000 7f5a04083740 1 -- 172.21.15.131:0/3558469502 shutdown_connections 2024-05-26T22:55:15.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.628+0000 7f5a04083740 1 --2- 172.21.15.131:0/3558469502 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55db76377830 0x55db7634c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:15.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.628+0000 7f5a04083740 1 --2- 172.21.15.131:0/3558469502 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55db76377f70 0x55db763772f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:15.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.628+0000 7f5a04083740 1 --2- 172.21.15.131:0/3558469502 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55db76394e50 0x55db76347a70 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:15.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.628+0000 7f5a04083740 1 -- 172.21.15.131:0/3558469502 >> 172.21.15.131:0/3558469502 conn(0x55db76349ce0 msgr2=0x55db763922f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:15.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.628+0000 7f5a04083740 1 -- 172.21.15.131:0/3558469502 shutdown_connections 2024-05-26T22:55:15.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.628+0000 7f5a04083740 1 -- 172.21.15.131:0/3558469502 wait complete. 2024-05-26T22:55:15.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.629+0000 7f5a04083740 1 -- start start 2024-05-26T22:55:15.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.629+0000 7f5a04083740 1 Processor -- start 2024-05-26T22:55:15.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.630+0000 7f5a04083740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55db76377f70 0x55db7637a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:15.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.630+0000 7f5a04083740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55db76394e50 0x55db76393420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:15.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.630+0000 7f5a04083740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55db76364fb0 0x55db76393960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:15.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.630+0000 7f5a04083740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55db7634bc00 con 0x55db76377f70 2024-05-26T22:55:15.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.630+0000 7f5a04083740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55db7626a910 con 0x55db76394e50 2024-05-26T22:55:15.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.630+0000 7f5a04083740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x55db7634aff0 con 0x55db76364fb0 2024-05-26T22:55:15.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.630+0000 7f5a04083740 10 client.0 _ll_register_callbacks cb 0x55db762a6410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-26T22:55:15.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.630+0000 7f59f9e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55db76377f70 0x55db7637a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:15.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.630+0000 7f59f9e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55db76377f70 0x55db7637a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59820/0 (socket says 172.21.15.131:59820) 2024-05-26T22:55:15.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.630+0000 7f59f9e00640 1 -- 172.21.15.131:0/2246863839 learned_addr learned my addr 172.21.15.131:0/2246863839 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:55:15.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.631+0000 7f59f9400640 1 --2- 172.21.15.131:0/2246863839 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55db76394e50 0x55db76393420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:15.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.631+0000 7f59f9e00640 1 -- 172.21.15.131:0/2246863839 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55db76364fb0 msgr2=0x55db76393960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:15.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.631+0000 7f59f9e00640 1 --2- 172.21.15.131:0/2246863839 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55db76364fb0 0x55db76393960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:15.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.631+0000 7f59f9e00640 1 -- 172.21.15.131:0/2246863839 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55db76394e50 msgr2=0x55db76393420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:15.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.631+0000 7f59f9e00640 1 --2- 172.21.15.131:0/2246863839 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55db76394e50 0x55db76393420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:15.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.631+0000 7f59f9e00640 1 -- 172.21.15.131:0/2246863839 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db7608d6b0 con 0x55db76377f70 2024-05-26T22:55:15.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.631+0000 7f59f9e00640 1 --2- 172.21.15.131:0/2246863839 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55db76377f70 0x55db7637a1b0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f59ec0165f0 tx=0x7f59ec0010f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c63974ae3354201d server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:15.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.631+0000 7f59e3400640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-26T22:55:15.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.631+0000 7f59e3400640 1 -- 172.21.15.131:0/2246863839 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59ec013b80 con 0x55db76377f70 2024-05-26T22:55:15.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.631+0000 7f59e3400640 1 -- 172.21.15.131:0/2246863839 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 5709+0+0 (secure 0 0 0) 0x7f59ec013d20 con 0x55db76377f70 2024-05-26T22:55:15.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.631+0000 7f5a04083740 1 -- 172.21.15.131:0/2246863839 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59ec005110 con 0x55db76377f70 2024-05-26T22:55:15.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.632+0000 7f59e3400640 1 -- 172.21.15.131:0/2246863839 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=98) ==== 24+0+0 (secure 0 0 0) 0x7f59ec031050 con 0x55db76377f70 2024-05-26T22:55:15.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.632+0000 7f5a04083740 10 client.8091 fetch_fsmap learned FSMap version 98 2024-05-26T22:55:15.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.632+0000 7f5a04083740 1 -- 172.21.15.131:0/2246863839 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=98,monmap=2+}) -- 0x55db763899c0 con 0x55db76377f70 2024-05-26T22:55:15.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.632+0000 7f59e3400640 1 -- 172.21.15.131:0/2246863839 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59ec005110 con 0x55db76377f70 2024-05-26T22:55:15.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.632+0000 7f59e3400640 1 -- 172.21.15.131:0/2246863839 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 98) ==== 3534+0+0 (secure 0 0 0) 0x7f59ec034020 con 0x55db76377f70 2024-05-26T22:55:15.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.632+0000 7f5a04083740 10 client.8091 fetch_fsmap finished waiting for FSMap version 98 2024-05-26T22:55:15.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.632+0000 7f5a04083740 10 client.8091 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-26T22:55:15.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.632+0000 7f5a04083740 20 client.8091 populate_metadata read hostname 'smithi131' 2024-05-26T22:55:15.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.632+0000 7f5a04083740 1 --2- 172.21.15.131:0/2246863839 >> [v2:172.21.15.131:6834/789312503,v1:172.21.15.131:6836/789312503] conn(0x55db763741e0 0x55db763b0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:15.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.632+0000 7f5a04083740 4 client.8091 mds_command: new command op to 7863 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-26T22:55:15.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.632+0000 7f5a04083740 1 -- 172.21.15.131:0/2246863839 --> [v2:172.21.15.131:6834/789312503,v1:172.21.15.131:6836/789312503] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55db762bcd40 con 0x55db763741e0 2024-05-26T22:55:15.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.632+0000 7f59f9400640 1 --2- 172.21.15.131:0/2246863839 >> [v2:172.21.15.131:6834/789312503,v1:172.21.15.131:6836/789312503] conn(0x55db763741e0 0x55db763b0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:15.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.633+0000 7f59f9400640 1 --2- 172.21.15.131:0/2246863839 >> [v2:172.21.15.131:6834/789312503,v1:172.21.15.131:6836/789312503] conn(0x55db763741e0 0x55db763b0c00 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9666df661e3a7b5 server_cookie=189a512b1424d251 in_seq=0 out_seq=0 2024-05-26T22:55:15.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.633+0000 7f59e3400640 10 client.8091 ms_handle_connect on v2:172.21.15.131:6834/789312503 2024-05-26T22:55:15.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.633+0000 7f59e3400640 1 -- 172.21.15.131:0/2246863839 <== mds.0 v2:172.21.15.131:6834/789312503 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55db762bcd40 con 0x55db763741e0 2024-05-26T22:55:15.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.633+0000 7f59e3400640 10 client.8091 handle_command_reply: tid=0 2024-05-26T22:55:15.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.634+0000 7f5a04083740 1 client.8091 shutdown 2024-05-26T22:55:15.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.634+0000 7f5a04083740 1 -- 172.21.15.131:0/2246863839 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55db76377f70 msgr2=0x55db7637a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:15.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.634+0000 7f5a04083740 1 --2- 172.21.15.131:0/2246863839 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55db76377f70 0x55db7637a1b0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f59ec0165f0 tx=0x7f59ec0010f0 comp rx=0 tx=0).stop 2024-05-26T22:55:15.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.634+0000 7f5a04083740 1 -- 172.21.15.131:0/2246863839 shutdown_connections 2024-05-26T22:55:15.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.634+0000 7f5a04083740 1 --2- 172.21.15.131:0/2246863839 >> [v2:172.21.15.131:6834/789312503,v1:172.21.15.131:6836/789312503] conn(0x55db763741e0 0x55db763b0c00 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:15.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.634+0000 7f5a04083740 1 --2- 172.21.15.131:0/2246863839 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55db76364fb0 0x55db76393960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:15.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.634+0000 7f5a04083740 1 --2- 172.21.15.131:0/2246863839 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55db76394e50 0x55db76393420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:15.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.634+0000 7f5a04083740 1 --2- 172.21.15.131:0/2246863839 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55db76377f70 0x55db7637a1b0 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:15.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.634+0000 7f5a04083740 1 -- 172.21.15.131:0/2246863839 >> 172.21.15.131:0/2246863839 conn(0x55db76377830 msgr2=0x55db763484d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:15.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.634+0000 7f59e3400640 0 client.8091 ms_handle_reset on v2:172.21.15.131:6834/789312503 2024-05-26T22:55:15.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.634+0000 7f5a04083740 1 -- 172.21.15.131:0/2246863839 shutdown_connections 2024-05-26T22:55:15.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.634+0000 7f5a04083740 1 -- 172.21.15.131:0/2246863839 wait complete. 2024-05-26T22:55:15.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.634+0000 7f5a04083740 20 client.8091 trim_cache size 0 max 16384 2024-05-26T22:55:15.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.654+0000 7f5a04083740 1 Processor -- start 2024-05-26T22:55:15.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.654+0000 7f5a04083740 1 -- start start 2024-05-26T22:55:15.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.655+0000 7f5a04083740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55db76364fb0 0x55db763920f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:15.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.655+0000 7f5a04083740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55db76377f70 0x55db76392630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:15.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.655+0000 7f5a04083740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55db76394e50 0x55db763aba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:15.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.655+0000 7f5a04083740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f59fc1ffa60 con 0x55db76377f70 2024-05-26T22:55:15.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.655+0000 7f5a04083740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f59fc200130 con 0x55db76394e50 2024-05-26T22:55:15.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.655+0000 7f5a04083740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f59fc0cc1a0 con 0x55db76364fb0 2024-05-26T22:55:15.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.656+0000 7f59f9400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55db76377f70 0x55db76392630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:15.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.656+0000 7f59f9400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55db76377f70 0x55db76392630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59836/0 (socket says 172.21.15.131:59836) 2024-05-26T22:55:15.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.656+0000 7f59f9400640 1 -- 172.21.15.131:0/3044036068 learned_addr learned my addr 172.21.15.131:0/3044036068 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:55:15.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.656+0000 7f59f9e00640 1 --2- 172.21.15.131:0/3044036068 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55db76364fb0 0x55db763920f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:15.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.656+0000 7f59fa800640 1 --2- 172.21.15.131:0/3044036068 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55db76394e50 0x55db763aba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:15.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.656+0000 7f59f9400640 1 -- 172.21.15.131:0/3044036068 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55db76364fb0 msgr2=0x55db763920f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:15.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.656+0000 7f59f9400640 1 --2- 172.21.15.131:0/3044036068 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55db76364fb0 0x55db763920f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:15.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.656+0000 7f59f9400640 1 -- 172.21.15.131:0/3044036068 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55db76394e50 msgr2=0x55db763aba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:15.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.656+0000 7f59f9400640 1 --2- 172.21.15.131:0/3044036068 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55db76394e50 0x55db763aba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:15.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.656+0000 7f59f9400640 1 -- 172.21.15.131:0/3044036068 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db76348240 con 0x55db76377f70 2024-05-26T22:55:15.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.656+0000 7f59fa800640 1 --2- 172.21.15.131:0/3044036068 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55db76394e50 0x55db763aba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:55:15.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.656+0000 7f59f9400640 1 --2- 172.21.15.131:0/3044036068 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55db76377f70 0x55db76392630 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f59e407a060 tx=0x7f59e4040040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0a3b713d605f0c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:15.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.657+0000 7f59ea000640 1 -- 172.21.15.131:0/3044036068 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59e407e710 con 0x55db76377f70 2024-05-26T22:55:15.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.657+0000 7f59ea000640 1 -- 172.21.15.131:0/3044036068 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59e4082f40 con 0x55db76377f70 2024-05-26T22:55:15.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.657+0000 7f59ea000640 1 -- 172.21.15.131:0/3044036068 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59e4078660 con 0x55db76377f70 2024-05-26T22:55:15.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.657+0000 7f5a04083740 1 -- 172.21.15.131:0/3044036068 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55db76377f70 msgr2=0x55db76392630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:15.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.657+0000 7f5a04083740 1 --2- 172.21.15.131:0/3044036068 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55db76377f70 0x55db76392630 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f59e407a060 tx=0x7f59e4040040 comp rx=0 tx=0).stop 2024-05-26T22:55:15.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.657+0000 7f5a04083740 1 -- 172.21.15.131:0/3044036068 shutdown_connections 2024-05-26T22:55:15.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.657+0000 7f5a04083740 1 --2- 172.21.15.131:0/3044036068 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55db76394e50 0x55db763aba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:15.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.657+0000 7f5a04083740 1 --2- 172.21.15.131:0/3044036068 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55db76377f70 0x55db76392630 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:15.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.657+0000 7f5a04083740 1 --2- 172.21.15.131:0/3044036068 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55db76364fb0 0x55db763920f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:15.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.657+0000 7f5a04083740 1 -- 172.21.15.131:0/3044036068 >> 172.21.15.131:0/3044036068 conn(0x55db76377830 msgr2=0x55db763725e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:15.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.657+0000 7f5a04083740 1 -- 172.21.15.131:0/3044036068 shutdown_connections 2024-05-26T22:55:15.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.657+0000 7f5a04083740 1 -- 172.21.15.131:0/3044036068 wait complete. 2024-05-26T22:55:15.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.658+0000 7f5a04083740 1 -- start start 2024-05-26T22:55:15.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.658+0000 7f5a04083740 1 Processor -- start 2024-05-26T22:55:15.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.659+0000 7f5a04083740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55db76364fb0 0x55db763474a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:15.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.659+0000 7f5a04083740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55db76377f70 0x55db76395710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:15.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.659+0000 7f5a04083740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55db76394e50 0x55db763932a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:15.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.659+0000 7f5a04083740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55db76366190 con 0x55db76364fb0 2024-05-26T22:55:15.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.659+0000 7f5a04083740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55db7634bc00 con 0x55db76377f70 2024-05-26T22:55:15.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.659+0000 7f5a04083740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x55db76365b20 con 0x55db76394e50 2024-05-26T22:55:15.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.659+0000 7f5a04083740 10 client.0 _ll_register_callbacks cb 0x7f59ec016720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-26T22:55:15.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.659+0000 7f59f9e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55db76364fb0 0x55db763474a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:15.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.659+0000 7f59f9e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55db76364fb0 0x55db763474a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:59842/0 (socket says 172.21.15.131:59842) 2024-05-26T22:55:15.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.659+0000 7f59f9e00640 1 -- 172.21.15.131:0/1958549733 learned_addr learned my addr 172.21.15.131:0/1958549733 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:55:15.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.660+0000 7f59f9e00640 1 -- 172.21.15.131:0/1958549733 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55db76394e50 msgr2=0x55db763932a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:15.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.660+0000 7f59fa800640 1 --2- 172.21.15.131:0/1958549733 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55db76394e50 0x55db763932a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:15.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.660+0000 7f59f9400640 1 --2- 172.21.15.131:0/1958549733 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55db76377f70 0x55db76395710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:15.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.660+0000 7f59f9e00640 1 --2- 172.21.15.131:0/1958549733 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55db76394e50 0x55db763932a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:15.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.660+0000 7f59f9e00640 1 -- 172.21.15.131:0/1958549733 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55db76377f70 msgr2=0x55db76395710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:15.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.660+0000 7f59f9e00640 1 --2- 172.21.15.131:0/1958549733 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55db76377f70 0x55db76395710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:15.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.660+0000 7f59f9e00640 1 -- 172.21.15.131:0/1958549733 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db76396360 con 0x55db76364fb0 2024-05-26T22:55:15.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.660+0000 7f59f9e00640 1 --2- 172.21.15.131:0/1958549733 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55db76364fb0 0x55db763474a0 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f59ec001eb0 tx=0x7f59ec012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=becc6141bcdbb3fb server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:15.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.660+0000 7f59e3400640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-26T22:55:15.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.660+0000 7f59e3400640 1 -- 172.21.15.131:0/1958549733 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59ec0050d0 con 0x55db76364fb0 2024-05-26T22:55:15.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.660+0000 7f59e3400640 1 -- 172.21.15.131:0/1958549733 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 5709+0+0 (secure 0 0 0) 0x7f59ec01c040 con 0x55db76364fb0 2024-05-26T22:55:15.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.660+0000 7f5a04083740 1 -- 172.21.15.131:0/1958549733 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59e4082f40 con 0x55db76364fb0 2024-05-26T22:55:15.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.661+0000 7f59e3400640 1 -- 172.21.15.131:0/1958549733 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=98) ==== 24+0+0 (secure 0 0 0) 0x7f59ec013d90 con 0x55db76364fb0 2024-05-26T22:55:15.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.661+0000 7f5a04083740 10 client.8097 fetch_fsmap learned FSMap version 98 2024-05-26T22:55:15.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.661+0000 7f5a04083740 1 -- 172.21.15.131:0/1958549733 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=98,monmap=2+}) -- 0x55db764109f0 con 0x55db76364fb0 2024-05-26T22:55:15.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.661+0000 7f59e3400640 1 -- 172.21.15.131:0/1958549733 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59e4082f40 con 0x55db76364fb0 2024-05-26T22:55:15.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.661+0000 7f59e3400640 1 -- 172.21.15.131:0/1958549733 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 98) ==== 3534+0+0 (secure 0 0 0) 0x7f59ec043020 con 0x55db76364fb0 2024-05-26T22:55:15.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.661+0000 7f5a04083740 10 client.8097 fetch_fsmap finished waiting for FSMap version 98 2024-05-26T22:55:15.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.661+0000 7f5a04083740 10 client.8097 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-26T22:55:15.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.661+0000 7f5a04083740 20 client.8097 populate_metadata read hostname 'smithi131' 2024-05-26T22:55:15.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.661+0000 7f5a04083740 1 --2- 172.21.15.131:0/1958549733 >> [v2:172.21.15.131:6834/789312503,v1:172.21.15.131:6836/789312503] conn(0x55db76410590 0x55db76387c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:15.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.661+0000 7f5a04083740 4 client.8097 mds_command: new command op to 7863 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-26T22:55:15.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.661+0000 7f5a04083740 1 -- 172.21.15.131:0/1958549733 --> [v2:172.21.15.131:6834/789312503,v1:172.21.15.131:6836/789312503] -- command(tid 0: {"prefix": "status"}) -- 0x55db76349130 con 0x55db76410590 2024-05-26T22:55:15.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.661+0000 7f59f9400640 1 --2- 172.21.15.131:0/1958549733 >> [v2:172.21.15.131:6834/789312503,v1:172.21.15.131:6836/789312503] conn(0x55db76410590 0x55db76387c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:15.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.662+0000 7f59f9400640 1 --2- 172.21.15.131:0/1958549733 >> [v2:172.21.15.131:6834/789312503,v1:172.21.15.131:6836/789312503] conn(0x55db76410590 0x55db76387c80 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2d9eeee9846d745 server_cookie=72d25a0b54e3a7d9 in_seq=0 out_seq=0 2024-05-26T22:55:15.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.662+0000 7f59e3400640 10 client.8097 ms_handle_connect on v2:172.21.15.131:6834/789312503 2024-05-26T22:55:15.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.662+0000 7f59e3400640 1 -- 172.21.15.131:0/1958549733 <== mds.0 v2:172.21.15.131:6834/789312503 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55db76349130 con 0x55db76410590 2024-05-26T22:55:15.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.662+0000 7f59e3400640 10 client.8097 handle_command_reply: tid=0 2024-05-26T22:55:15.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.662+0000 7f5a04083740 1 client.8097 shutdown 2024-05-26T22:55:15.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.662+0000 7f5a04083740 1 -- 172.21.15.131:0/1958549733 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55db76364fb0 msgr2=0x55db763474a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:15.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.662+0000 7f5a04083740 1 --2- 172.21.15.131:0/1958549733 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55db76364fb0 0x55db763474a0 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f59ec001eb0 tx=0x7f59ec012000 comp rx=0 tx=0).stop 2024-05-26T22:55:15.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.662+0000 7f5a04083740 1 -- 172.21.15.131:0/1958549733 shutdown_connections 2024-05-26T22:55:15.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.662+0000 7f5a04083740 1 --2- 172.21.15.131:0/1958549733 >> [v2:172.21.15.131:6834/789312503,v1:172.21.15.131:6836/789312503] conn(0x55db76410590 0x55db76387c80 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:15.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.662+0000 7f5a04083740 1 --2- 172.21.15.131:0/1958549733 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55db76394e50 0x55db763932a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:15.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.662+0000 7f5a04083740 1 --2- 172.21.15.131:0/1958549733 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55db76377f70 0x55db76395710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:15.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.663+0000 7f59e3400640 0 client.8097 ms_handle_reset on v2:172.21.15.131:6834/789312503 2024-05-26T22:55:15.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.663+0000 7f5a04083740 1 --2- 172.21.15.131:0/1958549733 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55db76364fb0 0x55db763474a0 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:15.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.663+0000 7f5a04083740 1 -- 172.21.15.131:0/1958549733 >> 172.21.15.131:0/1958549733 conn(0x55db76377830 msgr2=0x55db7637ad70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:15.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.663+0000 7f5a04083740 1 -- 172.21.15.131:0/1958549733 shutdown_connections 2024-05-26T22:55:15.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.663+0000 7f5a04083740 1 -- 172.21.15.131:0/1958549733 wait complete. 2024-05-26T22:55:15.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.663+0000 7f5a04083740 20 client.8097 trim_cache size 0 max 16384 2024-05-26T22:55:15.667 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-26T22:55:15.667 INFO:teuthology.orchestra.run.smithi131.stdout: "cluster_fsid": "46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f", 2024-05-26T22:55:15.667 INFO:teuthology.orchestra.run.smithi131.stdout: "whoami": 0, 2024-05-26T22:55:15.667 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 7863, 2024-05-26T22:55:15.667 INFO:teuthology.orchestra.run.smithi131.stdout: "want_state": "up:active", 2024-05-26T22:55:15.667 INFO:teuthology.orchestra.run.smithi131.stdout: "state": "up:active", 2024-05-26T22:55:15.667 INFO:teuthology.orchestra.run.smithi131.stdout: "fs_name": "cephfs", 2024-05-26T22:55:15.667 INFO:teuthology.orchestra.run.smithi131.stdout: "rank_uptime": 17.756697971000001, 2024-05-26T22:55:15.667 INFO:teuthology.orchestra.run.smithi131.stdout: "mdsmap_epoch": 98, 2024-05-26T22:55:15.667 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch": 95, 2024-05-26T22:55:15.667 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch_barrier": 95, 2024-05-26T22:55:15.667 INFO:teuthology.orchestra.run.smithi131.stdout: "uptime": 23.211465013000002 2024-05-26T22:55:15.668 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-26T22:55:15.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.663+0000 7f59fb200640 1 -- 172.21.15.131:0/2502081632 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f59d806be30 msgr2=0x7f59d806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:15.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.663+0000 7f59fb200640 1 --2- 172.21.15.131:0/2502081632 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f59d806be30 0x7f59d806e2f0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f59ec000f60 tx=0x7f59ec0079b0 comp rx=0 tx=0).stop 2024-05-26T22:55:15.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.663+0000 7f59fb200640 1 -- 172.21.15.131:0/2502081632 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59fc0d27f0 msgr2=0x7f59fc0cf110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:15.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.663+0000 7f59fb200640 1 --2- 172.21.15.131:0/2502081632 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59fc0d27f0 0x7f59fc0cf110 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f59e4035040 tx=0x7f59e4004a40 comp rx=0 tx=0).stop 2024-05-26T22:55:15.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.664+0000 7f59fb200640 1 -- 172.21.15.131:0/2502081632 shutdown_connections 2024-05-26T22:55:15.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.664+0000 7f59fb200640 1 --2- 172.21.15.131:0/2502081632 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f59d806be30 0x7f59d806e2f0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:15.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.664+0000 7f59fb200640 1 --2- 172.21.15.131:0/2502081632 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f59fc15e340 0x7f59fc0cf650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:15.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.664+0000 7f59fb200640 1 --2- 172.21.15.131:0/2502081632 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59fc0d27f0 0x7f59fc0cf110 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:15.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.664+0000 7f59fb200640 1 --2- 172.21.15.131:0/2502081632 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f59fc0b9f20 0x7f59fc1f6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:15.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.664+0000 7f59fb200640 1 -- 172.21.15.131:0/2502081632 >> 172.21.15.131:0/2502081632 conn(0x7f59fc072e60 msgr2=0x7f59fc0c9db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:15.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.664+0000 7f59fb200640 1 -- 172.21.15.131:0/2502081632 shutdown_connections 2024-05-26T22:55:15.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15.664+0000 7f59fb200640 1 -- 172.21.15.131:0/2502081632 wait complete. 2024-05-26T22:55:15.680 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-05-26T22:55:15.680 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-26T22:55:15.680 DEBUG:teuthology.orchestra.run.smithi131:> mkdir -p tree 2024-05-26T22:55:15.680 DEBUG:teuthology.orchestra.run.smithi131:> 2024-05-26T22:55:15.680 DEBUG:teuthology.orchestra.run.smithi131:> 2024-05-26T22:55:15.680 DEBUG:teuthology.orchestra.run.smithi131:> 2024-05-26T22:55:15.680 DEBUG:teuthology.orchestra.run.smithi131:> for ((i = 0; i < 10; i++)); do 2024-05-26T22:55:15.681 DEBUG:teuthology.orchestra.run.smithi131:> mkdir -p "tree/$i" 2024-05-26T22:55:15.681 DEBUG:teuthology.orchestra.run.smithi131:> echo file > "tree/$i/file" 2024-05-26T22:55:15.681 DEBUG:teuthology.orchestra.run.smithi131:> done 2024-05-26T22:55:15.681 DEBUG:teuthology.orchestra.run.smithi131:> ') 2024-05-26T22:55:15.698 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-26T22:55:15.731 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-26T22:55:15.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-26T22:55:15.784 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree 2024-05-26T22:55:15.788 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i = 0 )) 2024-05-26T22:55:15.788 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 10 )) 2024-05-26T22:55:15.788 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/0 2024-05-26T22:55:15.792 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:55:15.798 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:55:15.798 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 10 )) 2024-05-26T22:55:15.798 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/1 2024-05-26T22:55:15.801 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:55:15.802 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:55:15.802 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 10 )) 2024-05-26T22:55:15.802 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/2 2024-05-26T22:55:15.805 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:55:15.807 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:55:15.807 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 10 )) 2024-05-26T22:55:15.807 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/3 2024-05-26T22:55:15.810 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:55:15.811 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:55:15.812 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 10 )) 2024-05-26T22:55:15.812 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/4 2024-05-26T22:55:15.816 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:55:15.817 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:55:15.817 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 10 )) 2024-05-26T22:55:15.817 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/5 2024-05-26T22:55:15.822 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:55:15.823 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:55:15.824 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 10 )) 2024-05-26T22:55:15.824 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/6 2024-05-26T22:55:15.827 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:55:15.829 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:55:15.829 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 10 )) 2024-05-26T22:55:15.829 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/7 2024-05-26T22:55:15.833 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:55:15.835 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:55:15.835 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 10 )) 2024-05-26T22:55:15.835 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/8 2024-05-26T22:55:15.839 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:55:15.841 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:55:15.841 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 10 )) 2024-05-26T22:55:15.841 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/9 2024-05-26T22:55:15.845 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:55:15.846 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:55:15.846 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 10 )) 2024-05-26T22:55:15.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-26T22:55:15.856 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:55:15.857 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 1 tree/0) 2024-05-26T22:55:15.957 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:55:15.957 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 1 tree/1) 2024-05-26T22:55:16.001 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:55:16.002 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 1 tree/2) 2024-05-26T22:55:16.098 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:55:16.098 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 1 tree/3) 2024-05-26T22:55:16.152 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:55:16.152 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 1 tree/4) 2024-05-26T22:55:16.208 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:55:16.208 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 1 tree/5) 2024-05-26T22:55:16.259 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:55:16.259 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 1 tree/6) 2024-05-26T22:55:16.311 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:55:16.311 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 1 tree/7) 2024-05-26T22:55:16.366 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:55:16.366 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 1 tree/8) 2024-05-26T22:55:16.418 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:55:16.418 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 1 tree/9) 2024-05-26T22:55:16.474 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:55:16.474 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin.distributed -v 1 tree) 2024-05-26T22:55:17.834 INFO:tasks.ceph.mds.a.smithi131.stderr:2024-05-26T22:55:17.835+0000 7f2ce4000640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-05-26T22:55:31.529 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json get subtrees 2024-05-26T22:55:31.758 INFO:teuthology.orchestra.run.smithi187.stdout:[{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds2","dirfrag":"0x102","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"2","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"2514","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":2},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/tree","dirfrag":"0x100000003ff.111*","snapid_first":2,"projected_version":"82","version":"82","committing_version":"82","committed_version":"82","is_rep":false,"dir_auth":"2","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1,"1":1}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1},"nref":3}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree/0","dirfrag":"0x10000000400","snapid_first":2,"projected_version":"0","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"subtree":1,"importbound":0,"sticky":0},"nref":1}},{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/tree","dirfrag":"0x100000003ff.100*","snapid_first":2,"projected_version":"82","version":"82","committing_version":"82","committed_version":"82","is_rep":false,"dir_auth":"2","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1,"1":1}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1},"nref":3}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree/6","dirfrag":"0x1000000040c","snapid_first":2,"projected_version":"0","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"subtree":1,"importbound":0,"sticky":0},"nref":1}}] 2024-05-26T22:55:31.759 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "2", "dirfrag": "0x102", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds2", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 2 }, "snapid_first": 2, "states": [], "version": "2514" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1, "1": 1 } }, "committed_version": "82", "committing_version": "82", "dir_auth": "2", "dirfrag": "0x100000003ff.111*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "/tree", "pins": { "child": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "82", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "82" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 2, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000400", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/0", "pins": { "importbound": 0, "sticky": 0, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1, "1": 1 } }, "committed_version": "82", "committing_version": "82", "dir_auth": "2", "dirfrag": "0x100000003ff.100*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "/tree", "pins": { "child": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "82", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "82" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 2, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x1000000040c", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/6", "pins": { "importbound": 0, "sticky": 0, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false } ] 2024-05-26T22:55:31.759 DEBUG:tasks.cephfs.cephfs_test_case:[ { "is_auth": true, "auth_first": 2, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "~mds2", "dirfrag": "0x102", "snapid_first": 2, "projected_version": "1", "version": "1", "committing_version": "1", "committed_version": "1", "is_rep": false, "dir_auth": "2", "states": [ "auth", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "dirty": 0, "authpin": 0 }, "nref": 2 } }, { "is_auth": false, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "", "dirfrag": "0x1", "snapid_first": 2, "projected_version": "0", "version": "2514", "committing_version": "0", "committed_version": "0", "is_rep": true, "dir_auth": "0", "states": [], "is_auth": false, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 2 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1 }, "nref": 2 } }, { "is_auth": true, "auth_first": 2, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 2, "dir": { "path": "/tree", "dirfrag": "0x100000003ff.111*", "snapid_first": 2, "projected_version": "82", "version": "82", "committing_version": "82", "committed_version": "82", "is_rep": false, "dir_auth": "2", "states": [ "auth", "complete" ], "is_auth": true, "auth_state": { "replicas": { "0": 1, "1": 1 } }, "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "child": 1, "frozen": 0, "subtree": 1, "importing": 0, "replicated": 1 }, "nref": 3 } }, { "is_auth": false, "auth_first": 1, "auth_second": -2, "export_pin": 1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": 1, "dir": { "path": "/tree/0", "dirfrag": "0x10000000400", "snapid_first": 2, "projected_version": "0", "version": "5", "committing_version": "0", "committed_version": "0", "is_rep": false, "dir_auth": "1", "states": [], "is_auth": false, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "subtree": 1, "importbound": 0, "sticky": 0 }, "nref": 1 } }, { "is_auth": true, "auth_first": 2, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 2, "dir": { "path": "/tree", "dirfrag": "0x100000003ff.100*", "snapid_first": 2, "projected_version": "82", "version": "82", "committing_version": "82", "committed_version": "82", "is_rep": false, "dir_auth": "2", "states": [ "auth", "complete" ], "is_auth": true, "auth_state": { "replicas": { "0": 1, "1": 1 } }, "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "child": 1, "frozen": 0, "subtree": 1, "importing": 0, "replicated": 1 }, "nref": 3 } }, { "is_auth": false, "auth_first": 1, "auth_second": -2, "export_pin": 1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": 1, "dir": { "path": "/tree/6", "dirfrag": "0x1000000040c", "snapid_first": 2, "projected_version": "0", "version": "5", "committing_version": "0", "committed_version": "0", "is_rep": false, "dir_auth": "1", "states": [], "is_auth": false, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "subtree": 1, "importbound": 0, "sticky": 0 }, "nref": 1 } } ] 2024-05-26T22:55:31.760 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json get subtrees 2024-05-26T22:55:31.973 INFO:teuthology.orchestra.run.smithi187.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"2514","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree/3","dirfrag":"0x10000000406","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree/0","dirfrag":"0x10000000400","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"2":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree/9","dirfrag":"0x10000000412","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree/8","dirfrag":"0x10000000410","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/tree","dirfrag":"0x100000003ff.111*","snapid_first":2,"projected_version":"0","version":"82","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree/4","dirfrag":"0x10000000408","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"importbound":0,"sticky":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree","dirfrag":"0x100000003ff.011*","snapid_first":2,"projected_version":"82","version":"82","committing_version":"82","committed_version":"82","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"dnwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1},"nref":3}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/tree","dirfrag":"0x100000003ff.100*","snapid_first":2,"projected_version":"0","version":"82","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree/5","dirfrag":"0x1000000040a","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree/6","dirfrag":"0x1000000040c","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"2":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree/7","dirfrag":"0x1000000040e","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree/2","dirfrag":"0x10000000404","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"importbound":0,"sticky":0,"replicated":0,"dirty":1},"nref":3}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree/1","dirfrag":"0x10000000402","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"importbound":0,"sticky":0,"replicated":0,"dirty":1},"nref":3}}] 2024-05-26T22:55:31.974 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "2514" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000406", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/tree/3", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "2": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000400", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/tree/0", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000412", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/tree/9", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000410", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/tree/8", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x100000003ff.111*", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/tree", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "82" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 2, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000408", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/tree/4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importbound": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "sticky": 0, "subtree": 1 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "82", "committing_version": "82", "dir_auth": "1", "dirfrag": "0x100000003ff.011*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "/tree", "pins": { "child": 1, "dnwaiter": 0, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "82", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "82" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x100000003ff.100*", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/tree", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "82" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 2, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x1000000040a", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/tree/5", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "2": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x1000000040c", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/tree/6", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x1000000040e", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/tree/7", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000404", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "/tree/2", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importbound": 0, "importing": 0, "ptrwaiter": 0, "replicated": 0, "sticky": 0, "subtree": 1 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000402", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "/tree/1", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importbound": 0, "importing": 0, "ptrwaiter": 0, "replicated": 0, "sticky": 0, "subtree": 1 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-05-26T22:55:31.975 DEBUG:tasks.cephfs.cephfs_test_case:[ { "is_auth": true, "auth_first": 1, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "~mds1", "dirfrag": "0x101", "snapid_first": 2, "projected_version": "1", "version": "1", "committing_version": "1", "committed_version": "1", "is_rep": false, "dir_auth": "1", "states": [ "auth", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "dirty": 0, "authpin": 0 }, "nref": 2 } }, { "is_auth": false, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "", "dirfrag": "0x1", "snapid_first": 2, "projected_version": "0", "version": "2514", "committing_version": "0", "committed_version": "0", "is_rep": true, "dir_auth": "0", "states": [], "is_auth": false, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1 }, "nref": 2 } }, { "is_auth": true, "auth_first": 1, "auth_second": -2, "export_pin": 1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": 1, "dir": { "path": "/tree/3", "dirfrag": "0x10000000406", "snapid_first": 2, "projected_version": "5", "version": "5", "committing_version": "0", "committed_version": "0", "is_rep": false, "dir_auth": "1", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": { "0": 1 } }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "child": 1, "frozen": 0, "subtree": 1, "importing": 0, "replicated": 1, "dirty": 1 }, "nref": 4 } }, { "is_auth": true, "auth_first": 1, "auth_second": -2, "export_pin": 1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": 1, "dir": { "path": "/tree/0", "dirfrag": "0x10000000400", "snapid_first": 2, "projected_version": "5", "version": "5", "committing_version": "0", "committed_version": "0", "is_rep": false, "dir_auth": "1", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": { "2": 1 } }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "child": 1, "frozen": 0, "subtree": 1, "importing": 0, "replicated": 1, "dirty": 1 }, "nref": 4 } }, { "is_auth": true, "auth_first": 1, "auth_second": -2, "export_pin": 1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": 1, "dir": { "path": "/tree/9", "dirfrag": "0x10000000412", "snapid_first": 2, "projected_version": "5", "version": "5", "committing_version": "0", "committed_version": "0", "is_rep": false, "dir_auth": "1", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": { "0": 1 } }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "child": 1, "frozen": 0, "subtree": 1, "importing": 0, "replicated": 1, "dirty": 1 }, "nref": 4 } }, { "is_auth": true, "auth_first": 1, "auth_second": -2, "export_pin": 1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": 1, "dir": { "path": "/tree/8", "dirfrag": "0x10000000410", "snapid_first": 2, "projected_version": "5", "version": "5", "committing_version": "0", "committed_version": "0", "is_rep": false, "dir_auth": "1", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": { "0": 1 } }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "child": 1, "frozen": 0, "subtree": 1, "importing": 0, "replicated": 1, "dirty": 1 }, "nref": 4 } }, { "is_auth": false, "auth_first": 2, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 2, "dir": { "path": "/tree", "dirfrag": "0x100000003ff.111*", "snapid_first": 2, "projected_version": "0", "version": "82", "committing_version": "0", "committed_version": "0", "is_rep": false, "dir_auth": "2", "states": [], "is_auth": false, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1 }, "nref": 2 } }, { "is_auth": true, "auth_first": 1, "auth_second": -2, "export_pin": 1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": 1, "dir": { "path": "/tree/4", "dirfrag": "0x10000000408", "snapid_first": 2, "projected_version": "5", "version": "5", "committing_version": "0", "committed_version": "0", "is_rep": false, "dir_auth": "1", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": { "0": 1 } }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "child": 1, "frozen": 0, "subtree": 1, "importing": 0, "importbound": 0, "sticky": 0, "replicated": 1, "dirty": 1 }, "nref": 4 } }, { "is_auth": true, "auth_first": 1, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 1, "dir": { "path": "/tree", "dirfrag": "0x100000003ff.011*", "snapid_first": 2, "projected_version": "82", "version": "82", "committing_version": "82", "committed_version": "82", "is_rep": false, "dir_auth": "1", "states": [ "auth", "complete" ], "is_auth": true, "auth_state": { "replicas": { "0": 1 } }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "dnwaiter": 0, "child": 1, "frozen": 0, "subtree": 1, "importing": 0, "replicated": 1 }, "nref": 3 } }, { "is_auth": false, "auth_first": 2, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 2, "dir": { "path": "/tree", "dirfrag": "0x100000003ff.100*", "snapid_first": 2, "projected_version": "0", "version": "82", "committing_version": "0", "committed_version": "0", "is_rep": false, "dir_auth": "2", "states": [], "is_auth": false, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1 }, "nref": 2 } }, { "is_auth": true, "auth_first": 1, "auth_second": -2, "export_pin": 1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": 1, "dir": { "path": "/tree/5", "dirfrag": "0x1000000040a", "snapid_first": 2, "projected_version": "5", "version": "5", "committing_version": "0", "committed_version": "0", "is_rep": false, "dir_auth": "1", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": { "0": 1 } }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "child": 1, "frozen": 0, "subtree": 1, "importing": 0, "replicated": 1, "dirty": 1 }, "nref": 4 } }, { "is_auth": true, "auth_first": 1, "auth_second": -2, "export_pin": 1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": 1, "dir": { "path": "/tree/6", "dirfrag": "0x1000000040c", "snapid_first": 2, "projected_version": "5", "version": "5", "committing_version": "0", "committed_version": "0", "is_rep": false, "dir_auth": "1", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": { "2": 1 } }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "child": 1, "frozen": 0, "subtree": 1, "importing": 0, "replicated": 1, "dirty": 1 }, "nref": 4 } }, { "is_auth": true, "auth_first": 1, "auth_second": -2, "export_pin": 1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": 1, "dir": { "path": "/tree/7", "dirfrag": "0x1000000040e", "snapid_first": 2, "projected_version": "5", "version": "5", "committing_version": "0", "committed_version": "0", "is_rep": false, "dir_auth": "1", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": { "0": 1 } }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "child": 1, "frozen": 0, "subtree": 1, "importing": 0, "replicated": 1, "dirty": 1 }, "nref": 4 } }, { "is_auth": true, "auth_first": 1, "auth_second": -2, "export_pin": 1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": 1, "dir": { "path": "/tree/2", "dirfrag": "0x10000000404", "snapid_first": 2, "projected_version": "5", "version": "5", "committing_version": "0", "committed_version": "0", "is_rep": false, "dir_auth": "1", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "child": 1, "frozen": 0, "subtree": 1, "importing": 0, "importbound": 0, "sticky": 0, "replicated": 0, "dirty": 1 }, "nref": 3 } }, { "is_auth": true, "auth_first": 1, "auth_second": -2, "export_pin": 1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": 1, "dir": { "path": "/tree/1", "dirfrag": "0x10000000402", "snapid_first": 2, "projected_version": "5", "version": "5", "committing_version": "0", "committed_version": "0", "is_rep": false, "dir_auth": "1", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "child": 1, "frozen": 0, "subtree": 1, "importing": 0, "importbound": 0, "sticky": 0, "replicated": 0, "dirty": 1 }, "nref": 3 } } ] 2024-05-26T22:55:31.975 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-05-26T22:55:32.206 INFO:teuthology.orchestra.run.smithi131.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"2514","version":"2514","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1,"2":2}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1734","version":"1734","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/tree","dirfrag":"0x100000003ff.000*","snapid_first":2,"projected_version":"82","version":"82","committing_version":"82","committed_version":"82","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"frozen":0,"subtree":1,"replicated":1,"dirty":0,"authpin":0},"nref":3}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree","dirfrag":"0x100000003ff.011*","snapid_first":2,"projected_version":"82","version":"82","committing_version":"82","committed_version":"82","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":1,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":2}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/tree","dirfrag":"0x100000003ff.100*","snapid_first":2,"projected_version":"82","version":"82","committing_version":"82","committed_version":"82","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":1,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":2}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/tree","dirfrag":"0x100000003ff.110*","snapid_first":2,"projected_version":"82","version":"82","committing_version":"82","committed_version":"82","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{"1":2}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"frozen":0,"subtree":1,"replicated":1,"dirty":0,"authpin":0},"nref":3}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/tree","dirfrag":"0x100000003ff.111*","snapid_first":2,"projected_version":"82","version":"82","committing_version":"82","committed_version":"82","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":1,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":2}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree/9","dirfrag":"0x10000000412","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree/8","dirfrag":"0x10000000410","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree/7","dirfrag":"0x1000000040e","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree/5","dirfrag":"0x1000000040a","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree/3","dirfrag":"0x10000000406","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}}] 2024-05-26T22:55:32.207 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1, "2": 2 } }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "2514", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "2514" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 1, "subtree": 1 }, "projected_version": "1734", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "1734" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "82", "committing_version": "82", "dir_auth": "0", "dirfrag": "0x100000003ff.000*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "/tree", "pins": { "authpin": 0, "child": 1, "dirty": 0, "frozen": 0, "replicated": 1, "subtree": 1 }, "projected_version": "82", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "82" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "82", "committing_version": "82", "dir_auth": "1", "dirfrag": "0x100000003ff.011*", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/tree", "pins": { "authpin": 0, "child": 1, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "82", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "82" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "82", "committing_version": "82", "dir_auth": "2", "dirfrag": "0x100000003ff.100*", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/tree", "pins": { "authpin": 0, "child": 1, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "82", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "82" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 2, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 2 } }, "committed_version": "82", "committing_version": "82", "dir_auth": "0", "dirfrag": "0x100000003ff.110*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "/tree", "pins": { "authpin": 0, "child": 1, "dirty": 0, "frozen": 0, "replicated": 1, "subtree": 1 }, "projected_version": "82", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "82" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "82", "committing_version": "82", "dir_auth": "2", "dirfrag": "0x100000003ff.111*", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/tree", "pins": { "authpin": 0, "child": 1, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "82", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "82" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 2, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000412", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/9", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000410", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/8", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x1000000040e", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/7", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x1000000040a", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/5", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000406", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree/3", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false } ] 2024-05-26T22:55:32.208 DEBUG:tasks.cephfs.cephfs_test_case:[ { "is_auth": true, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "", "dirfrag": "0x1", "snapid_first": 2, "projected_version": "2514", "version": "2514", "committing_version": "1", "committed_version": "1", "is_rep": true, "dir_auth": "0", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": { "1": 1, "2": 2 } }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "replicated": 1, "dirty": 1, "authpin": 0 }, "nref": 4 } }, { "is_auth": true, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "~mds0", "dirfrag": "0x100", "snapid_first": 2, "projected_version": "1734", "version": "1734", "committing_version": "1", "committed_version": "1", "is_rep": false, "dir_auth": "0", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "dirty": 1, "authpin": 0 }, "nref": 3 } }, { "is_auth": true, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 0, "dir": { "path": "/tree", "dirfrag": "0x100000003ff.000*", "snapid_first": 2, "projected_version": "82", "version": "82", "committing_version": "82", "committed_version": "82", "is_rep": false, "dir_auth": "0", "states": [ "auth", "complete" ], "is_auth": true, "auth_state": { "replicas": { "1": 1 } }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "frozen": 0, "subtree": 1, "replicated": 1, "dirty": 0, "authpin": 0 }, "nref": 3 } }, { "is_auth": false, "auth_first": 1, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 1, "dir": { "path": "/tree", "dirfrag": "0x100000003ff.011*", "snapid_first": 2, "projected_version": "82", "version": "82", "committing_version": "82", "committed_version": "82", "is_rep": false, "dir_auth": "1", "states": [], "is_auth": false, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "request": 0, "child": 1, "frozen": 0, "subtree": 1, "replicated": 0, "dirty": 0, "waiter": 0, "authpin": 0, "tempexporting": 0 }, "nref": 2 } }, { "is_auth": false, "auth_first": 2, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 2, "dir": { "path": "/tree", "dirfrag": "0x100000003ff.100*", "snapid_first": 2, "projected_version": "82", "version": "82", "committing_version": "82", "committed_version": "82", "is_rep": false, "dir_auth": "2", "states": [], "is_auth": false, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "request": 0, "child": 1, "frozen": 0, "subtree": 1, "replicated": 0, "dirty": 0, "waiter": 0, "authpin": 0, "tempexporting": 0 }, "nref": 2 } }, { "is_auth": true, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 0, "dir": { "path": "/tree", "dirfrag": "0x100000003ff.110*", "snapid_first": 2, "projected_version": "82", "version": "82", "committing_version": "82", "committed_version": "82", "is_rep": false, "dir_auth": "0", "states": [ "auth", "complete" ], "is_auth": true, "auth_state": { "replicas": { "1": 2 } }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "frozen": 0, "subtree": 1, "replicated": 1, "dirty": 0, "authpin": 0 }, "nref": 3 } }, { "is_auth": false, "auth_first": 2, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 2, "dir": { "path": "/tree", "dirfrag": "0x100000003ff.111*", "snapid_first": 2, "projected_version": "82", "version": "82", "committing_version": "82", "committed_version": "82", "is_rep": false, "dir_auth": "2", "states": [], "is_auth": false, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "request": 0, "child": 1, "frozen": 0, "subtree": 1, "replicated": 0, "dirty": 0, "waiter": 0, "authpin": 0, "tempexporting": 0 }, "nref": 2 } }, { "is_auth": false, "auth_first": 1, "auth_second": -2, "export_pin": 1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": 1, "dir": { "path": "/tree/9", "dirfrag": "0x10000000412", "snapid_first": 2, "projected_version": "5", "version": "5", "committing_version": "0", "committed_version": "0", "is_rep": false, "dir_auth": "1", "states": [], "is_auth": false, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "request": 0, "child": 0, "frozen": 0, "subtree": 1, "replicated": 0, "dirty": 0, "waiter": 0, "authpin": 0, "tempexporting": 0 }, "nref": 1 } }, { "is_auth": false, "auth_first": 1, "auth_second": -2, "export_pin": 1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": 1, "dir": { "path": "/tree/8", "dirfrag": "0x10000000410", "snapid_first": 2, "projected_version": "5", "version": "5", "committing_version": "0", "committed_version": "0", "is_rep": false, "dir_auth": "1", "states": [], "is_auth": false, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "request": 0, "child": 0, "frozen": 0, "subtree": 1, "replicated": 0, "dirty": 0, "waiter": 0, "authpin": 0, "tempexporting": 0 }, "nref": 1 } }, { "is_auth": false, "auth_first": 1, "auth_second": -2, "export_pin": 1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": 1, "dir": { "path": "/tree/7", "dirfrag": "0x1000000040e", "snapid_first": 2, "projected_version": "5", "version": "5", "committing_version": "0", "committed_version": "0", "is_rep": false, "dir_auth": "1", "states": [], "is_auth": false, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "request": 0, "child": 0, "frozen": 0, "subtree": 1, "replicated": 0, "dirty": 0, "waiter": 0, "authpin": 0, "tempexporting": 0 }, "nref": 1 } }, { "is_auth": false, "auth_first": 1, "auth_second": -2, "export_pin": 1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": 1, "dir": { "path": "/tree/5", "dirfrag": "0x1000000040a", "snapid_first": 2, "projected_version": "5", "version": "5", "committing_version": "0", "committed_version": "0", "is_rep": false, "dir_auth": "1", "states": [], "is_auth": false, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "request": 0, "child": 0, "frozen": 0, "subtree": 1, "replicated": 0, "dirty": 0, "waiter": 0, "authpin": 0, "tempexporting": 0 }, "nref": 1 } }, { "is_auth": false, "auth_first": 1, "auth_second": -2, "export_pin": 1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": 1, "dir": { "path": "/tree/3", "dirfrag": "0x10000000406", "snapid_first": 2, "projected_version": "5", "version": "5", "committing_version": "0", "committed_version": "0", "is_rep": false, "dir_auth": "1", "states": [], "is_auth": false, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "request": 0, "child": 0, "frozen": 0, "subtree": 1, "replicated": 0, "dirty": 0, "waiter": 0, "authpin": 0, "tempexporting": 0 }, "nref": 1 } } ] 2024-05-26T22:55:32.208 INFO:tasks.cephfs.cephfs_test_case:[('/tree/0', 1), ('/tree/1', 1), ('/tree/2', 1), ('/tree/3', 1), ('/tree/4', 1), ('/tree/5', 1), ('/tree/6', 1), ('/tree/7', 1), ('/tree/8', 1), ('/tree/9', 1)] =?= [('/tree/0', 1), ('/tree/1', 1), ('/tree/2', 1), ('/tree/3', 1), ('/tree/4', 1), ('/tree/5', 1), ('/tree/6', 1), ('/tree/7', 1), ('/tree/8', 1), ('/tree/9', 1)] 2024-05-26T22:55:32.209 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-05-26T22:55:32.209 DEBUG:teuthology.orchestra.run.smithi131:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-26T22:55:32.317 DEBUG:teuthology.orchestra.run.smithi187:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-26T22:55:32.434 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-05-26T22:55:32.435 INFO:tasks.cephfs.mount:Terminating background process 2024-05-26T22:55:32.435 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-26T22:55:32.435 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-26T22:55:32.492 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-26T22:55:32.492 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-26T22:55:32.548 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-26T22:55:32.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:32.752+0000 7f4ddbc00640 1 Processor -- start 2024-05-26T22:55:32.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:32.753+0000 7f4ddbc00640 1 -- start start 2024-05-26T22:55:32.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:32.754+0000 7f4ddbc00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f4dd40ad250 0x7f4dd40b3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:32.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:32.754+0000 7f4ddbc00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4dd415cb20 con 0x7f4dd41569d0 2024-05-26T22:55:32.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:32.754+0000 7f4ddbc00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4dd415ce00 con 0x7f4dd406ee20 2024-05-26T22:55:32.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:32.754+0000 7f4ddbc00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f4dd415d0e0 con 0x7f4dd40ad250 2024-05-26T22:55:32.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:32.754+0000 7f4dcbe00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f4dd40ad250 0x7f4dd40b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:32.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:32.754+0000 7f4dd9c00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f4dd41569d0 0x7f4dd4160010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:54012/0 (socket says 172.21.15.131:54012) 2024-05-26T22:55:32.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:32.755+0000 7f4dd9c00640 1 -- 172.21.15.131:0/1779111897 learned_addr learned my addr 172.21.15.131:0/1779111897 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:55:32.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:32.754+0000 7f4dcbe00640 1 --2- 172.21.15.131:0/1779111897 >> v2:172.21.15.131:3300/0 conn(0x7f4dd40ad250 0x7f4dd40b3190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52234/0 (socket says 172.21.15.131:52234) 2024-05-26T22:55:32.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:32.755+0000 7f4dcbe00640 1 -- 172.21.15.131:0/1779111897 >> v1:172.21.15.187:6789/0 conn(0x7f4dd406ee20 legacy=0x7f4dd41562c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:32.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:32.755+0000 7f4dcbe00640 1 -- 172.21.15.131:0/1779111897 >> v1:172.21.15.131:6789/0 conn(0x7f4dd41569d0 legacy=0x7f4dd4160010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:32.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:32.755+0000 7f4dcbe00640 1 -- 172.21.15.131:0/1779111897 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dd415d3c0 con 0x7f4dd40ad250 2024-05-26T22:55:32.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:32.756+0000 7f4dcbe00640 1 --2- 172.21.15.131:0/1779111897 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4dd40ad250 0x7f4dd40b3190 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc0097a0 tx=0x7f4dbc02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcfcd24700b2f329 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:32.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:32.756+0000 7f4dd3e00640 1 -- 172.21.15.131:0/1779111897 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dbc037070 con 0x7f4dd40ad250 2024-05-26T22:55:32.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:32.756+0000 7f4dd3e00640 1 -- 172.21.15.131:0/1779111897 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4dbc02ae50 con 0x7f4dd40ad250 2024-05-26T22:55:32.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:32.756+0000 7f4dd3e00640 1 -- 172.21.15.131:0/1779111897 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dbc032660 con 0x7f4dd40ad250 2024-05-26T22:55:32.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:32.757+0000 7f4ddbc00640 1 -- 172.21.15.131:0/1779111897 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4dd40ad250 msgr2=0x7f4dd40b3190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:32.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:32.757+0000 7f4ddbc00640 1 --2- 172.21.15.131:0/1779111897 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4dd40ad250 0x7f4dd40b3190 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc0097a0 tx=0x7f4dbc02a140 comp rx=0 tx=0).stop 2024-05-26T22:55:32.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:32.757+0000 7f4ddbc00640 1 -- 172.21.15.131:0/1779111897 shutdown_connections 2024-05-26T22:55:32.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:32.757+0000 7f4ddbc00640 1 --2- 172.21.15.131:0/1779111897 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4dd40ad250 0x7f4dd40b3190 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:32.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:32.757+0000 7f4ddbc00640 1 -- 172.21.15.131:0/1779111897 >> 172.21.15.131:0/1779111897 conn(0x7f4dd40b1e90 msgr2=0x7f4dd40b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:32.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:32.757+0000 7f4ddbc00640 1 -- 172.21.15.131:0/1779111897 shutdown_connections 2024-05-26T22:55:32.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:32.757+0000 7f4ddbc00640 1 -- 172.21.15.131:0/1779111897 wait complete. 2024-05-26T22:55:32.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:32.759+0000 7f4ddbc00640 1 Processor -- start 2024-05-26T22:55:32.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:32.759+0000 7f4ddbc00640 1 -- start start 2024-05-26T22:55:32.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:32.760+0000 7f4ddbc00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4dd406ee20 0x7f4dd40998c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:32.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:32.760+0000 7f4ddbc00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4dd40ad250 0x7f4dd414fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:32.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:32.760+0000 7f4ddbc00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4dd41569d0 0x7f4dd409c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:32.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:32.760+0000 7f4ddbc00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd415c900 con 0x7f4dd40ad250 2024-05-26T22:55:32.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:32.760+0000 7f4ddbc00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd415c780 con 0x7f4dd41569d0 2024-05-26T22:55:32.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:32.760+0000 7f4ddbc00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f4dd406f200 con 0x7f4dd406ee20 2024-05-26T22:55:32.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:32.760+0000 7f4dcbe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4dd40ad250 0x7f4dd414fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:32.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:32.760+0000 7f4dcbe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4dd40ad250 0x7f4dd414fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52236/0 (socket says 172.21.15.131:52236) 2024-05-26T22:55:32.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:32.760+0000 7f4dcbe00640 1 -- 172.21.15.131:0/1283440491 learned_addr learned my addr 172.21.15.131:0/1283440491 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:55:32.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:32.760+0000 7f4dcbe00640 1 -- 172.21.15.131:0/1283440491 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4dd406ee20 msgr2=0x7f4dd40998c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:32.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:32.760+0000 7f4dcbe00640 1 --2- 172.21.15.131:0/1283440491 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4dd406ee20 0x7f4dd40998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:32.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:32.761+0000 7f4dcbe00640 1 -- 172.21.15.131:0/1283440491 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4dd41569d0 msgr2=0x7f4dd409c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:32.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:32.761+0000 7f4dcbe00640 1 --2- 172.21.15.131:0/1283440491 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4dd41569d0 0x7f4dd409c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:32.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:32.761+0000 7f4dcbe00640 1 -- 172.21.15.131:0/1283440491 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dd41506e0 con 0x7f4dd40ad250 2024-05-26T22:55:32.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:32.761+0000 7f4dcbe00640 1 --2- 172.21.15.131:0/1283440491 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4dd40ad250 0x7f4dd414fd00 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc000c00 tx=0x7f4dbc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:32.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:32.761+0000 7f4dd1600640 1 -- 172.21.15.131:0/1283440491 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dbc042070 con 0x7f4dd40ad250 2024-05-26T22:55:32.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:32.761+0000 7f4dd1600640 1 -- 172.21.15.131:0/1283440491 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4dbc03e670 con 0x7f4dd40ad250 2024-05-26T22:55:32.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:32.761+0000 7f4dd1600640 1 -- 172.21.15.131:0/1283440491 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dbc037040 con 0x7f4dd40ad250 2024-05-26T22:55:32.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:32.761+0000 7f4ddbc00640 1 -- 172.21.15.131:0/1283440491 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4dd41509c0 con 0x7f4dd40ad250 2024-05-26T22:55:32.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:32.761+0000 7f4ddbc00640 1 -- 172.21.15.131:0/1283440491 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4dd4150f80 con 0x7f4dd40ad250 2024-05-26T22:55:32.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:32.763+0000 7f4ddbc00640 1 -- 172.21.15.131:0/1283440491 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4dd40694e0 con 0x7f4dd40ad250 2024-05-26T22:55:32.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:32.763+0000 7f4dd1600640 1 -- 172.21.15.131:0/1283440491 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4dbc045070 con 0x7f4dd40ad250 2024-05-26T22:55:32.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:32.764+0000 7f4dd1600640 1 --2- 172.21.15.131:0/1283440491 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f4db006be30 0x7f4db006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:32.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:32.764+0000 7f4dd1600640 1 -- 172.21.15.131:0/1283440491 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 5709+0+0 (secure 0 0 0) 0x7f4dbc0ab910 con 0x7f4dd40ad250 2024-05-26T22:55:32.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:32.765+0000 7f4dd9200640 1 --2- 172.21.15.131:0/1283440491 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f4db006be30 0x7f4db006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:32.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:32.765+0000 7f4dd9200640 1 --2- 172.21.15.131:0/1283440491 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f4db006be30 0x7f4db006e2f0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f4dc4003cc0 tx=0x7f4dc40079b0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:32.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:32.769+0000 7f4dd1600640 1 -- 172.21.15.131:0/1283440491 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f4dbc07a340 con 0x7f4dd40ad250 2024-05-26T22:55:32.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:32.970+0000 7f4ddbc00640 1 -- 172.21.15.131:0/1283440491 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4dd406c590 con 0x7f4dd40ad250 2024-05-26T22:55:32.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:32.973+0000 7f4dd1600640 1 -- 172.21.15.131:0/1283440491 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) ==== 74+0+16632 (secure 0 0 0) 0x7f4dbc079c40 con 0x7f4dd40ad250 2024-05-26T22:55:32.973 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:55:32.973 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":95,"fsid":"46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f","created":"2024-05-26T22:47:57.288905+0000","modified":"2024-05-26T22:54:58.901569+0000","last_up_change":"2024-05-26T22:48:09.602207+0000","last_in_change":"2024-05-26T22:48:00.687341+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-26T22:48:10.463339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-05-26T22:54:55.120321+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-05-26T22:54:56.346430+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"814eaf62-a18d-494b-8e27-fd8fb88adea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6801","nonce":2350940051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6803","nonce":2350940051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6807","nonce":2350940051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6805","nonce":2350940051}]},"public_addr":"172.21.15.131:6801/2350940051","cluster_addr":"172.21.15.131:6803/2350940051","heartbeat_back_addr":"172.21.15.131:6807/2350940051","heartbeat_front_addr":"172.21.15.131:6805/2350940051","state":["exists","up"]},{"osd":1,"uuid":"42e66287-f6c6-4e26-af3c-6dfaa3536d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6825","nonce":1525528247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6827","nonce":1525528247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6831","nonce":1525528247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6829","nonce":1525528247}]},"public_addr":"172.21.15.131:6825/1525528247","cluster_addr":"172.21.15.131:6827/1525528247","heartbeat_back_addr":"172.21.15.131:6831/1525528247","heartbeat_front_addr":"172.21.15.131:6829/1525528247","state":["exists","up"]},{"osd":2,"uuid":"0da34673-04a2-4fd0-b57a-0ff2b679b043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6817","nonce":3627842210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6819","nonce":3627842210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6823","nonce":3627842210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6821","nonce":3627842210}]},"public_addr":"172.21.15.131:6817/3627842210","cluster_addr":"172.21.15.131:6819/3627842210","heartbeat_back_addr":"172.21.15.131:6823/3627842210","heartbeat_front_addr":"172.21.15.131:6821/3627842210","state":["exists","up"]},{"osd":3,"uuid":"cf117a27-d6b2-43d3-ae52-b47988bf8eed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6809","nonce":350790591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6811","nonce":350790591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6815","nonce":350790591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6813","nonce":350790591}]},"public_addr":"172.21.15.131:6809/350790591","cluster_addr":"172.21.15.131:6811/350790591","heartbeat_back_addr":"172.21.15.131:6815/350790591","heartbeat_front_addr":"172.21.15.131:6813/350790591","state":["exists","up"]},{"osd":4,"uuid":"a092ccfc-be89-4a84-88d5-0f781fa715eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6816","nonce":244560995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6819","nonce":244560995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6823","nonce":244560995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6821","nonce":244560995}]},"public_addr":"172.21.15.187:6816/244560995","cluster_addr":"172.21.15.187:6819/244560995","heartbeat_back_addr":"172.21.15.187:6823/244560995","heartbeat_front_addr":"172.21.15.187:6821/244560995","state":["exists","up"]},{"osd":5,"uuid":"7513ba69-dd00-4a35-b190-9e8cc3b2bdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6801","nonce":324619398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6803","nonce":324619398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6812","nonce":324619398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6807","nonce":324619398}]},"public_addr":"172.21.15.187:6801/324619398","cluster_addr":"172.21.15.187:6803/324619398","heartbeat_back_addr":"172.21.15.187:6812/324619398","heartbeat_front_addr":"172.21.15.187:6807/324619398","state":["exists","up"]},{"osd":6,"uuid":"50081810-c093-4d2f-a4d7-b7c695c5400d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6806","nonce":1391347786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6809","nonce":1391347786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6818","nonce":1391347786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6813","nonce":1391347786}]},"public_addr":"172.21.15.187:6806/1391347786","cluster_addr":"172.21.15.187:6809/1391347786","heartbeat_back_addr":"172.21.15.187:6818/1391347786","heartbeat_front_addr":"172.21.15.187:6813/1391347786","state":["exists","up"]},{"osd":7,"uuid":"775dae05-54f5-4ad4-bbce-92863c6ed2ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6825","nonce":1220065622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6827","nonce":1220065622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6831","nonce":1220065622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6829","nonce":1220065622}]},"public_addr":"172.21.15.187:6825/1220065622","cluster_addr":"172.21.15.187:6827/1220065622","heartbeat_back_addr":"172.21.15.187:6831/1220065622","heartbeat_front_addr":"172.21.15.187:6829/1220065622","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:05.983520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.299516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.067323+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.307748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.018742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.258775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.010912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.295355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-26T22:55:32.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:32.975+0000 7f4ddbc00640 1 -- 172.21.15.131:0/1283440491 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f4db006be30 msgr2=0x7f4db006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:32.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:32.975+0000 7f4ddbc00640 1 --2- 172.21.15.131:0/1283440491 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f4db006be30 0x7f4db006e2f0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f4dc4003cc0 tx=0x7f4dc40079b0 comp rx=0 tx=0).stop 2024-05-26T22:55:32.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:32.975+0000 7f4ddbc00640 1 -- 172.21.15.131:0/1283440491 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4dd40ad250 msgr2=0x7f4dd414fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:32.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:32.975+0000 7f4ddbc00640 1 --2- 172.21.15.131:0/1283440491 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4dd40ad250 0x7f4dd414fd00 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc000c00 tx=0x7f4dbc0046c0 comp rx=0 tx=0).stop 2024-05-26T22:55:32.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:32.975+0000 7f4ddbc00640 1 -- 172.21.15.131:0/1283440491 shutdown_connections 2024-05-26T22:55:32.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:32.975+0000 7f4ddbc00640 1 --2- 172.21.15.131:0/1283440491 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f4db006be30 0x7f4db006e2f0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:32.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:32.975+0000 7f4ddbc00640 1 --2- 172.21.15.131:0/1283440491 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4dd41569d0 0x7f4dd409c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:32.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:32.975+0000 7f4ddbc00640 1 --2- 172.21.15.131:0/1283440491 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4dd40ad250 0x7f4dd414fd00 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:32.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:32.975+0000 7f4ddbc00640 1 --2- 172.21.15.131:0/1283440491 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4dd406ee20 0x7f4dd40998c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:32.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:32.975+0000 7f4ddbc00640 1 -- 172.21.15.131:0/1283440491 >> 172.21.15.131:0/1283440491 conn(0x7f4dd40b1e90 msgr2=0x7f4dd406d470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:32.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:32.976+0000 7f4ddbc00640 1 -- 172.21.15.131:0/1283440491 shutdown_connections 2024-05-26T22:55:32.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:32.976+0000 7f4ddbc00640 1 -- 172.21.15.131:0/1283440491 wait complete. 2024-05-26T22:55:32.995 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-26T22:48:10.463339+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-26T22:54:55.120321+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '94', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2024-05-26T22:54:56.346430+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '95', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-26T22:55:32.995 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-26T22:55:33.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.205+0000 7f7ac0600640 1 Processor -- start 2024-05-26T22:55:33.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.205+0000 7f7ac0600640 1 -- start start 2024-05-26T22:55:33.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.207+0000 7f7ac0600640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7ab806f420 0x7f7ab80cfd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:33.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.207+0000 7f7ac0600640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f7ab80cf590 0x7f7ab80d0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:33.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.207+0000 7f7ac0600640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ab8160180 con 0x7f7ab80b5450 2024-05-26T22:55:33.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.207+0000 7f7ac0600640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f7ab8160d90 con 0x7f7ab80cf590 2024-05-26T22:55:33.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.207+0000 7f7ac0600640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f7ab8160460 con 0x7f7ab806f420 2024-05-26T22:55:33.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.208+0000 7f7abe600640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f7ab80cf590 0x7f7ab80d0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:33.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.208+0000 7f7abd200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7ab806f420 0x7f7ab80cfd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:33.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.208+0000 7f7abe600640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f7ab80cf590 0x7f7ab80d0290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.131:52936/0 (socket says 172.21.15.131:52936) 2024-05-26T22:55:33.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.208+0000 7f7abe600640 1 -- 172.21.15.131:0/2817204147 learned_addr learned my addr 172.21.15.131:0/2817204147 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:55:33.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.209+0000 7f7abd200640 1 -- 172.21.15.131:0/2817204147 >> v2:172.21.15.187:3300/0 conn(0x7f7ab80cf590 msgr2=0x7f7ab80d0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:33.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.209+0000 7f7abd200640 1 --2- 172.21.15.131:0/2817204147 >> v2:172.21.15.187:3300/0 conn(0x7f7ab80cf590 0x7f7ab80d0290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:33.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.209+0000 7f7abd200640 1 -- 172.21.15.131:0/2817204147 >> v1:172.21.15.187:6789/0 conn(0x7f7ab80b5450 legacy=0x7f7ab80b43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:33.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.209+0000 7f7abd200640 1 -- 172.21.15.131:0/2817204147 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ab8160740 con 0x7f7ab806f420 2024-05-26T22:55:33.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.210+0000 7f7abd200640 1 --2- 172.21.15.131:0/2817204147 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7ab806f420 0x7f7ab80cfd50 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7aa40097a0 tx=0x7f7aa402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56cbef32725b5bba server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:33.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.211+0000 7f7ab7e00640 1 -- 172.21.15.131:0/2817204147 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa4037070 con 0x7f7ab806f420 2024-05-26T22:55:33.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.211+0000 7f7ab7e00640 1 -- 172.21.15.131:0/2817204147 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7aa4032440 con 0x7f7ab806f420 2024-05-26T22:55:33.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.211+0000 7f7ab7e00640 1 -- 172.21.15.131:0/2817204147 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa4032810 con 0x7f7ab806f420 2024-05-26T22:55:33.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.212+0000 7f7ac0600640 1 -- 172.21.15.131:0/2817204147 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7ab806f420 msgr2=0x7f7ab80cfd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:33.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.212+0000 7f7ac0600640 1 --2- 172.21.15.131:0/2817204147 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7ab806f420 0x7f7ab80cfd50 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7aa40097a0 tx=0x7f7aa402a180 comp rx=0 tx=0).stop 2024-05-26T22:55:33.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.212+0000 7f7ac0600640 1 -- 172.21.15.131:0/2817204147 shutdown_connections 2024-05-26T22:55:33.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.212+0000 7f7ac0600640 1 --2- 172.21.15.131:0/2817204147 >> v2:172.21.15.187:3300/0 conn(0x7f7ab80cf590 0x7f7ab80d0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:33.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.212+0000 7f7ac0600640 1 --2- 172.21.15.131:0/2817204147 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7ab806f420 0x7f7ab80cfd50 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:33.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.212+0000 7f7ac0600640 1 -- 172.21.15.131:0/2817204147 >> 172.21.15.131:0/2817204147 conn(0x7f7ab806bc90 msgr2=0x7f7ab80b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:33.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.212+0000 7f7ac0600640 1 -- 172.21.15.131:0/2817204147 shutdown_connections 2024-05-26T22:55:33.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.212+0000 7f7ac0600640 1 -- 172.21.15.131:0/2817204147 wait complete. 2024-05-26T22:55:33.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.214+0000 7f7ac0600640 1 Processor -- start 2024-05-26T22:55:33.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.214+0000 7f7ac0600640 1 -- start start 2024-05-26T22:55:33.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.215+0000 7f7ac0600640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ab806f420 0x7f7ab8145040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:33.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.215+0000 7f7ac0600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ab80b5450 0x7f7ab809d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:33.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.215+0000 7f7ac0600640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7ab80cf590 0x7f7ab809d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:33.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.215+0000 7f7ac0600640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab815fde0 con 0x7f7ab80b5450 2024-05-26T22:55:33.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.215+0000 7f7ac0600640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab80b5830 con 0x7f7ab806f420 2024-05-26T22:55:33.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.215+0000 7f7ac0600640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f7ab815ff60 con 0x7f7ab80cf590 2024-05-26T22:55:33.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.215+0000 7f7abd200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ab80b5450 0x7f7ab809d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:33.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.215+0000 7f7abd200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ab80b5450 0x7f7ab809d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52246/0 (socket says 172.21.15.131:52246) 2024-05-26T22:55:33.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.215+0000 7f7abd200640 1 -- 172.21.15.131:0/1919595903 learned_addr learned my addr 172.21.15.131:0/1919595903 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:55:33.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.215+0000 7f7abe600640 1 --2- 172.21.15.131:0/1919595903 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7ab80cf590 0x7f7ab809d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:33.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.215+0000 7f7abdc00640 1 --2- 172.21.15.131:0/1919595903 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ab806f420 0x7f7ab8145040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:33.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.215+0000 7f7abd200640 1 -- 172.21.15.131:0/1919595903 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7ab80cf590 msgr2=0x7f7ab809d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:33.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.215+0000 7f7abd200640 1 --2- 172.21.15.131:0/1919595903 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7ab80cf590 0x7f7ab809d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:33.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.215+0000 7f7abd200640 1 -- 172.21.15.131:0/1919595903 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ab806f420 msgr2=0x7f7ab8145040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:33.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.215+0000 7f7abd200640 1 --2- 172.21.15.131:0/1919595903 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ab806f420 0x7f7ab8145040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:33.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.216+0000 7f7abd200640 1 -- 172.21.15.131:0/1919595903 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ab81520e0 con 0x7f7ab80b5450 2024-05-26T22:55:33.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.216+0000 7f7abdc00640 1 --2- 172.21.15.131:0/1919595903 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ab806f420 0x7f7ab8145040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:55:33.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.216+0000 7f7abd200640 1 --2- 172.21.15.131:0/1919595903 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ab80b5450 0x7f7ab809d0b0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f7aa4035040 tx=0x7f7aa40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:33.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.216+0000 7f7ab5600640 1 -- 172.21.15.131:0/1919595903 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa4037040 con 0x7f7ab80b5450 2024-05-26T22:55:33.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.216+0000 7f7ab5600640 1 -- 172.21.15.131:0/1919595903 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7aa4030500 con 0x7f7ab80b5450 2024-05-26T22:55:33.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.216+0000 7f7ab5600640 1 -- 172.21.15.131:0/1919595903 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa4030910 con 0x7f7ab80b5450 2024-05-26T22:55:33.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.216+0000 7f7ac0600640 1 -- 172.21.15.131:0/1919595903 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ab81523c0 con 0x7f7ab80b5450 2024-05-26T22:55:33.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.216+0000 7f7ac0600640 1 -- 172.21.15.131:0/1919595903 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ab8152980 con 0x7f7ab80b5450 2024-05-26T22:55:33.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.217+0000 7f7ac0600640 1 -- 172.21.15.131:0/1919595903 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ab8074e90 con 0x7f7ab80b5450 2024-05-26T22:55:33.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.218+0000 7f7ab5600640 1 -- 172.21.15.131:0/1919595903 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7aa4044020 con 0x7f7ab80b5450 2024-05-26T22:55:33.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.219+0000 7f7ab5600640 1 --2- 172.21.15.131:0/1919595903 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f7a9806be30 0x7f7a9806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:33.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.219+0000 7f7ab5600640 1 -- 172.21.15.131:0/1919595903 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 5709+0+0 (secure 0 0 0) 0x7f7aa40ab180 con 0x7f7ab80b5450 2024-05-26T22:55:33.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.220+0000 7f7abdc00640 1 --2- 172.21.15.131:0/1919595903 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f7a9806be30 0x7f7a9806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:33.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.220+0000 7f7abdc00640 1 --2- 172.21.15.131:0/1919595903 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f7a9806be30 0x7f7a9806e2f0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f7aa8001680 tx=0x7f7aa8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:33.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.226+0000 7f7ab5600640 1 -- 172.21.15.131:0/1919595903 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f7aa4079b20 con 0x7f7ab80b5450 2024-05-26T22:55:33.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.429+0000 7f7ac0600640 1 -- 172.21.15.131:0/1919595903 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7ab80695c0 con 0x7f7ab80b5450 2024-05-26T22:55:33.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.430+0000 7f7ab5600640 1 -- 172.21.15.131:0/1919595903 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) ==== 93+0+10 (secure 0 0 0) 0x7f7aa4079420 con 0x7f7ab80b5450 2024-05-26T22:55:33.430 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2024-05-26T22:55:33.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.432+0000 7f7ac0600640 1 -- 172.21.15.131:0/1919595903 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f7a9806be30 msgr2=0x7f7a9806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:33.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.432+0000 7f7ac0600640 1 --2- 172.21.15.131:0/1919595903 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f7a9806be30 0x7f7a9806e2f0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f7aa8001680 tx=0x7f7aa8009040 comp rx=0 tx=0).stop 2024-05-26T22:55:33.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.432+0000 7f7ac0600640 1 -- 172.21.15.131:0/1919595903 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ab80b5450 msgr2=0x7f7ab809d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:33.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.432+0000 7f7ac0600640 1 --2- 172.21.15.131:0/1919595903 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ab80b5450 0x7f7ab809d0b0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f7aa4035040 tx=0x7f7aa40046c0 comp rx=0 tx=0).stop 2024-05-26T22:55:33.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.433+0000 7f7ac0600640 1 -- 172.21.15.131:0/1919595903 shutdown_connections 2024-05-26T22:55:33.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.433+0000 7f7ac0600640 1 --2- 172.21.15.131:0/1919595903 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7ab80cf590 0x7f7ab809d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:33.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.433+0000 7f7ac0600640 1 --2- 172.21.15.131:0/1919595903 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f7a9806be30 0x7f7a9806e2f0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:33.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.433+0000 7f7ac0600640 1 --2- 172.21.15.131:0/1919595903 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ab80b5450 0x7f7ab809d0b0 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:33.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.433+0000 7f7ac0600640 1 --2- 172.21.15.131:0/1919595903 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ab806f420 0x7f7ab8145040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:33.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.433+0000 7f7ac0600640 1 -- 172.21.15.131:0/1919595903 >> 172.21.15.131:0/1919595903 conn(0x7f7ab806bc90 msgr2=0x7f7ab80cda00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:33.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.433+0000 7f7ac0600640 1 -- 172.21.15.131:0/1919595903 shutdown_connections 2024-05-26T22:55:33.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.433+0000 7f7ac0600640 1 -- 172.21.15.131:0/1919595903 wait complete. 2024-05-26T22:55:33.452 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-26T22:55:33.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.657+0000 7f1671c00640 1 Processor -- start 2024-05-26T22:55:33.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.657+0000 7f1671c00640 1 -- start start 2024-05-26T22:55:33.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.659+0000 7f1671c00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f166c0b5450 0x7f166c0b43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:33.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.659+0000 7f1671c00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f166c0d0460 0x7f166c06d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:33.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.659+0000 7f1671c00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f166c0b4900 con 0x7f166c06f420 2024-05-26T22:55:33.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.659+0000 7f1671c00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f166c160d90 con 0x7f166c0b5450 2024-05-26T22:55:33.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.659+0000 7f1671c00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f166c160180 con 0x7f166c0d0460 2024-05-26T22:55:33.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.659+0000 7f166aa00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f166c0b5450 0x7f166c0b43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:33.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.659+0000 7f166aa00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f166c0b5450 0x7f166c0b43c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52252/0 (socket says 172.21.15.131:52252) 2024-05-26T22:55:33.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.660+0000 7f166aa00640 1 -- 172.21.15.131:0/2554046599 learned_addr learned my addr 172.21.15.131:0/2554046599 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:55:33.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.660+0000 7f166b400640 1 --2- 172.21.15.131:0/2554046599 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f166c0d0460 0x7f166c06d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:33.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.660+0000 7f166aa00640 1 -- 172.21.15.131:0/2554046599 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f166c0d0460 msgr2=0x7f166c06d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:33.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.660+0000 7f166aa00640 1 --2- 172.21.15.131:0/2554046599 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f166c0d0460 0x7f166c06d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:33.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.660+0000 7f166aa00640 1 -- 172.21.15.131:0/2554046599 >> v1:172.21.15.187:6789/0 conn(0x7f166c06f420 legacy=0x7f166c0cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:33.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.660+0000 7f166aa00640 1 -- 172.21.15.131:0/2554046599 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f166c160460 con 0x7f166c0b5450 2024-05-26T22:55:33.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.661+0000 7f166aa00640 1 --2- 172.21.15.131:0/2554046599 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f166c0b5450 0x7f166c0b43c0 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f165c009870 tx=0x7f165c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b0d87dbbe6f36ff server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:33.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.661+0000 7f1669600640 1 -- 172.21.15.131:0/2554046599 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f165c037070 con 0x7f166c0b5450 2024-05-26T22:55:33.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.661+0000 7f1669600640 1 -- 172.21.15.131:0/2554046599 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f165c032440 con 0x7f166c0b5450 2024-05-26T22:55:33.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.661+0000 7f1669600640 1 -- 172.21.15.131:0/2554046599 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f165c032810 con 0x7f166c0b5450 2024-05-26T22:55:33.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.662+0000 7f1671c00640 1 -- 172.21.15.131:0/2554046599 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f166c0b5450 msgr2=0x7f166c0b43c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:33.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.662+0000 7f1671c00640 1 --2- 172.21.15.131:0/2554046599 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f166c0b5450 0x7f166c0b43c0 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f165c009870 tx=0x7f165c02a210 comp rx=0 tx=0).stop 2024-05-26T22:55:33.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.662+0000 7f1671c00640 1 -- 172.21.15.131:0/2554046599 shutdown_connections 2024-05-26T22:55:33.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.662+0000 7f1671c00640 1 --2- 172.21.15.131:0/2554046599 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f166c0d0460 0x7f166c06d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:33.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.662+0000 7f1671c00640 1 --2- 172.21.15.131:0/2554046599 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f166c0b5450 0x7f166c0b43c0 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:33.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.662+0000 7f1671c00640 1 -- 172.21.15.131:0/2554046599 >> 172.21.15.131:0/2554046599 conn(0x7f166c06bc90 msgr2=0x7f166c0b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:33.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.662+0000 7f1671c00640 1 -- 172.21.15.131:0/2554046599 shutdown_connections 2024-05-26T22:55:33.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.662+0000 7f1671c00640 1 -- 172.21.15.131:0/2554046599 wait complete. 2024-05-26T22:55:33.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.664+0000 7f1671c00640 1 Processor -- start 2024-05-26T22:55:33.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.664+0000 7f1671c00640 1 -- start start 2024-05-26T22:55:33.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.665+0000 7f1671c00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f166c06f420 0x7f166c09d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:33.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.665+0000 7f1671c00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f166c0b5450 0x7f166c09abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:33.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.665+0000 7f1671c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f166c0d0460 0x7f166c1f4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:33.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.665+0000 7f1671c00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f166c0b5830 con 0x7f166c0d0460 2024-05-26T22:55:33.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.665+0000 7f1671c00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f166c160a20 con 0x7f166c0b5450 2024-05-26T22:55:33.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.665+0000 7f1671c00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f166c15fde0 con 0x7f166c06f420 2024-05-26T22:55:33.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.666+0000 7f166b400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f166c0d0460 0x7f166c1f4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:33.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.666+0000 7f166b400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f166c0d0460 0x7f166c1f4900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52254/0 (socket says 172.21.15.131:52254) 2024-05-26T22:55:33.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.666+0000 7f166b400640 1 -- 172.21.15.131:0/3924125184 learned_addr learned my addr 172.21.15.131:0/3924125184 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:55:33.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.666+0000 7f166a000640 1 --2- 172.21.15.131:0/3924125184 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f166c0b5450 0x7f166c09abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:33.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.666+0000 7f166aa00640 1 --2- 172.21.15.131:0/3924125184 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f166c06f420 0x7f166c09d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:33.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.666+0000 7f166b400640 1 -- 172.21.15.131:0/3924125184 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f166c06f420 msgr2=0x7f166c09d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:33.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.666+0000 7f166b400640 1 --2- 172.21.15.131:0/3924125184 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f166c06f420 0x7f166c09d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:33.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.666+0000 7f166b400640 1 -- 172.21.15.131:0/3924125184 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f166c0b5450 msgr2=0x7f166c09abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:33.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.666+0000 7f166b400640 1 --2- 172.21.15.131:0/3924125184 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f166c0b5450 0x7f166c09abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:33.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.666+0000 7f166b400640 1 -- 172.21.15.131:0/3924125184 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f166c1f4140 con 0x7f166c0d0460 2024-05-26T22:55:33.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.666+0000 7f166aa00640 1 --2- 172.21.15.131:0/3924125184 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f166c06f420 0x7f166c09d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:55:33.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.666+0000 7f166b400640 1 --2- 172.21.15.131:0/3924125184 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f166c0d0460 0x7f166c1f4900 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f166400cda0 tx=0x7f166400a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:33.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.667+0000 7f1662a00640 1 -- 172.21.15.131:0/3924125184 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1664003b90 con 0x7f166c0d0460 2024-05-26T22:55:33.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.667+0000 7f1662a00640 1 -- 172.21.15.131:0/3924125184 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1664003d30 con 0x7f166c0d0460 2024-05-26T22:55:33.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.667+0000 7f1662a00640 1 -- 172.21.15.131:0/3924125184 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1664004240 con 0x7f166c0d0460 2024-05-26T22:55:33.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.667+0000 7f1671c00640 1 -- 172.21.15.131:0/3924125184 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f166c09d9d0 con 0x7f166c0d0460 2024-05-26T22:55:33.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.667+0000 7f1671c00640 1 -- 172.21.15.131:0/3924125184 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f166c09b120 con 0x7f166c0d0460 2024-05-26T22:55:33.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.668+0000 7f1671c00640 1 -- 172.21.15.131:0/3924125184 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f166c0695c0 con 0x7f166c0d0460 2024-05-26T22:55:33.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.669+0000 7f1662a00640 1 -- 172.21.15.131:0/3924125184 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f16640043e0 con 0x7f166c0d0460 2024-05-26T22:55:33.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.670+0000 7f1662a00640 1 --2- 172.21.15.131:0/3924125184 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f164c06be30 0x7f164c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:33.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.670+0000 7f1662a00640 1 -- 172.21.15.131:0/3924125184 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 5709+0+0 (secure 0 0 0) 0x7f166408ba20 con 0x7f166c0d0460 2024-05-26T22:55:33.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.670+0000 7f166aa00640 1 --2- 172.21.15.131:0/3924125184 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f164c06be30 0x7f164c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:33.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.671+0000 7f166aa00640 1 --2- 172.21.15.131:0/3924125184 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f164c06be30 0x7f164c06e2f0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f165c034000 tx=0x7f165c02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:33.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.674+0000 7f1662a00640 1 -- 172.21.15.131:0/3924125184 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f166401d340 con 0x7f166c0d0460 2024-05-26T22:55:33.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.871+0000 7f1671c00640 1 -- 172.21.15.131:0/3924125184 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f166c0b1260 con 0x7f166c0d0460 2024-05-26T22:55:33.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.872+0000 7f1662a00640 1 -- 172.21.15.131:0/3924125184 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v95) ==== 104+0+11 (secure 0 0 0) 0x7f166405a330 con 0x7f166c0d0460 2024-05-26T22:55:33.872 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-05-26T22:55:33.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.875+0000 7f1671c00640 1 -- 172.21.15.131:0/3924125184 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f164c06be30 msgr2=0x7f164c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:33.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.875+0000 7f1671c00640 1 --2- 172.21.15.131:0/3924125184 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f164c06be30 0x7f164c06e2f0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f165c034000 tx=0x7f165c02e000 comp rx=0 tx=0).stop 2024-05-26T22:55:33.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.875+0000 7f1671c00640 1 -- 172.21.15.131:0/3924125184 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f166c0d0460 msgr2=0x7f166c1f4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:33.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.875+0000 7f1671c00640 1 --2- 172.21.15.131:0/3924125184 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f166c0d0460 0x7f166c1f4900 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f166400cda0 tx=0x7f166400a660 comp rx=0 tx=0).stop 2024-05-26T22:55:33.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.875+0000 7f1671c00640 1 -- 172.21.15.131:0/3924125184 shutdown_connections 2024-05-26T22:55:33.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.875+0000 7f1671c00640 1 --2- 172.21.15.131:0/3924125184 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f164c06be30 0x7f164c06e2f0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:33.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.875+0000 7f1671c00640 1 --2- 172.21.15.131:0/3924125184 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f166c0d0460 0x7f166c1f4900 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:33.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.875+0000 7f1671c00640 1 --2- 172.21.15.131:0/3924125184 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f166c0b5450 0x7f166c09abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:33.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.875+0000 7f1671c00640 1 --2- 172.21.15.131:0/3924125184 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f166c06f420 0x7f166c09d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:33.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.875+0000 7f1671c00640 1 -- 172.21.15.131:0/3924125184 >> 172.21.15.131:0/3924125184 conn(0x7f166c06bc90 msgr2=0x7f166c0cb920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:33.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.875+0000 7f1671c00640 1 -- 172.21.15.131:0/3924125184 shutdown_connections 2024-05-26T22:55:33.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:33.875+0000 7f1671c00640 1 -- 172.21.15.131:0/3924125184 wait complete. 2024-05-26T22:55:33.894 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-26T22:55:34.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.101+0000 7f49b5800640 1 Processor -- start 2024-05-26T22:55:34.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.101+0000 7f49b5800640 1 -- start start 2024-05-26T22:55:34.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.103+0000 7f49b5800640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f49b015dc40 0x7f49b006d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:34.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.103+0000 7f49b5800640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f49b0071c10 0x7f49b006d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:34.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.103+0000 7f49b5800640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49b01552d0 con 0x7f49b015bfa0 2024-05-26T22:55:34.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.103+0000 7f49b5800640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f49b00b3f30 con 0x7f49b0071c10 2024-05-26T22:55:34.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.103+0000 7f49b5800640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f49b0072230 con 0x7f49b015dc40 2024-05-26T22:55:34.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.103+0000 7f49b4e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f49b0071c10 0x7f49b006d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:34.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.103+0000 7f49b4e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f49b0071c10 0x7f49b006d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52260/0 (socket says 172.21.15.131:52260) 2024-05-26T22:55:34.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.103+0000 7f49a7400640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f49b015bfa0 0x7f49b0071500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:54018/0 (socket says 172.21.15.131:54018) 2024-05-26T22:55:34.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.103+0000 7f49b4e00640 1 -- 172.21.15.131:0/2418938435 learned_addr learned my addr 172.21.15.131:0/2418938435 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:55:34.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.104+0000 7f49b4e00640 1 -- 172.21.15.131:0/2418938435 >> v2:172.21.15.187:3300/0 conn(0x7f49b015dc40 msgr2=0x7f49b006d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:34.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.104+0000 7f49afe00640 1 --2- 172.21.15.131:0/2418938435 >> v2:172.21.15.187:3300/0 conn(0x7f49b015dc40 0x7f49b006d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:34.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.104+0000 7f49b4e00640 1 --2- 172.21.15.131:0/2418938435 >> v2:172.21.15.187:3300/0 conn(0x7f49b015dc40 0x7f49b006d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:34.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.104+0000 7f49b4e00640 1 -- 172.21.15.131:0/2418938435 >> v1:172.21.15.131:6789/0 conn(0x7f49b015bfa0 legacy=0x7f49b0071500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:34.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.104+0000 7f49b4e00640 1 -- 172.21.15.131:0/2418938435 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49b0072510 con 0x7f49b0071c10 2024-05-26T22:55:34.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.104+0000 7f49af400640 1 -- 172.21.15.131:0/2418938435 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 131324078 0 0) 0x7f49b01552d0 con 0x7f49b015bfa0 2024-05-26T22:55:34.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.104+0000 7f49afe00640 1 --2- 172.21.15.131:0/2418938435 >> v2:172.21.15.187:3300/0 conn(0x7f49b015dc40 0x7f49b006d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:55:34.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.105+0000 7f49b4e00640 1 --2- 172.21.15.131:0/2418938435 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49b0071c10 0x7f49b006d790 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f49b006f3c0 tx=0x7f49a802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61367fb7fd1dbf26 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:34.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.105+0000 7f49af400640 1 -- 172.21.15.131:0/2418938435 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49a8036070 con 0x7f49b0071c10 2024-05-26T22:55:34.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.105+0000 7f49af400640 1 -- 172.21.15.131:0/2418938435 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49a8031600 con 0x7f49b0071c10 2024-05-26T22:55:34.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.105+0000 7f49af400640 1 -- 172.21.15.131:0/2418938435 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49a80319d0 con 0x7f49b0071c10 2024-05-26T22:55:34.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.106+0000 7f49b5800640 1 -- 172.21.15.131:0/2418938435 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49b0071c10 msgr2=0x7f49b006d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:34.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.106+0000 7f49b5800640 1 --2- 172.21.15.131:0/2418938435 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49b0071c10 0x7f49b006d790 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f49b006f3c0 tx=0x7f49a802a730 comp rx=0 tx=0).stop 2024-05-26T22:55:34.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.106+0000 7f49b5800640 1 -- 172.21.15.131:0/2418938435 shutdown_connections 2024-05-26T22:55:34.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.106+0000 7f49b5800640 1 --2- 172.21.15.131:0/2418938435 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49b0071c10 0x7f49b006d790 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:34.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.106+0000 7f49b5800640 1 --2- 172.21.15.131:0/2418938435 >> v2:172.21.15.187:3300/0 conn(0x7f49b015dc40 0x7f49b006d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:34.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.106+0000 7f49b5800640 1 -- 172.21.15.131:0/2418938435 >> 172.21.15.131:0/2418938435 conn(0x7f49b00b5660 msgr2=0x7f49b00b5a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:34.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.106+0000 7f49b5800640 1 -- 172.21.15.131:0/2418938435 shutdown_connections 2024-05-26T22:55:34.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.106+0000 7f49b5800640 1 -- 172.21.15.131:0/2418938435 wait complete. 2024-05-26T22:55:34.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.108+0000 7f49b5800640 1 Processor -- start 2024-05-26T22:55:34.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.108+0000 7f49b5800640 1 -- start start 2024-05-26T22:55:34.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.109+0000 7f49b5800640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f49b0071c10 0x7f49b0143980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:34.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.109+0000 7f49b5800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49b015bfa0 0x7f49b00ccec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:34.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.109+0000 7f49b5800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f49b015dc40 0x7f49b00cd400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:34.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.109+0000 7f49b5800640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f49b015e1a0 con 0x7f49b015bfa0 2024-05-26T22:55:34.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.109+0000 7f49b5800640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f49b015e020 con 0x7f49b0071c10 2024-05-26T22:55:34.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.109+0000 7f49b5800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f49b00b2080 con 0x7f49b015dc40 2024-05-26T22:55:34.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.109+0000 7f49a7400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49b015bfa0 0x7f49b00ccec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:34.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.109+0000 7f49a7400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49b015bfa0 0x7f49b00ccec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52276/0 (socket says 172.21.15.131:52276) 2024-05-26T22:55:34.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.109+0000 7f49a7400640 1 -- 172.21.15.131:0/31122433 learned_addr learned my addr 172.21.15.131:0/31122433 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:55:34.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.109+0000 7f49afe00640 1 --2- 172.21.15.131:0/31122433 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f49b0071c10 0x7f49b0143980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:34.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.109+0000 7f49b4e00640 1 --2- 172.21.15.131:0/31122433 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f49b015dc40 0x7f49b00cd400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:34.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.109+0000 7f49a7400640 1 -- 172.21.15.131:0/31122433 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f49b015dc40 msgr2=0x7f49b00cd400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:34.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.109+0000 7f49a7400640 1 --2- 172.21.15.131:0/31122433 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f49b015dc40 0x7f49b00cd400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:34.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.109+0000 7f49a7400640 1 -- 172.21.15.131:0/31122433 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f49b0071c10 msgr2=0x7f49b0143980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:34.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.109+0000 7f49a7400640 1 --2- 172.21.15.131:0/31122433 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f49b0071c10 0x7f49b0143980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:34.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.109+0000 7f49a7400640 1 -- 172.21.15.131:0/31122433 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49b0143ec0 con 0x7f49b015bfa0 2024-05-26T22:55:34.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.110+0000 7f49a7400640 1 --2- 172.21.15.131:0/31122433 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49b015bfa0 0x7f49b00ccec0 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f499800c4d0 tx=0x7f499800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:34.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.110+0000 7f49acc00640 1 -- 172.21.15.131:0/31122433 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4998018070 con 0x7f49b015bfa0 2024-05-26T22:55:34.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.110+0000 7f49acc00640 1 -- 172.21.15.131:0/31122433 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4998007500 con 0x7f49b015bfa0 2024-05-26T22:55:34.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.110+0000 7f49b5800640 1 -- 172.21.15.131:0/31122433 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49b00cda30 con 0x7f49b015bfa0 2024-05-26T22:55:34.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.110+0000 7f49acc00640 1 -- 172.21.15.131:0/31122433 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4998004a20 con 0x7f49b015bfa0 2024-05-26T22:55:34.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.110+0000 7f49b5800640 1 -- 172.21.15.131:0/31122433 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49b00cdff0 con 0x7f49b015bfa0 2024-05-26T22:55:34.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.111+0000 7f49b5800640 1 -- 172.21.15.131:0/31122433 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49b00694e0 con 0x7f49b015bfa0 2024-05-26T22:55:34.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.112+0000 7f49acc00640 1 -- 172.21.15.131:0/31122433 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f499801b070 con 0x7f49b015bfa0 2024-05-26T22:55:34.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.113+0000 7f49acc00640 1 --2- 172.21.15.131:0/31122433 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f498c06be30 0x7f498c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:34.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.113+0000 7f49acc00640 1 -- 172.21.15.131:0/31122433 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 5709+0+0 (secure 0 0 0) 0x7f499808a250 con 0x7f49b015bfa0 2024-05-26T22:55:34.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.113+0000 7f49afe00640 1 --2- 172.21.15.131:0/31122433 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f498c06be30 0x7f498c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:34.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.114+0000 7f49afe00640 1 --2- 172.21.15.131:0/31122433 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f498c06be30 0x7f498c06e2f0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f49b00d03c0 tx=0x7f49a0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:34.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.117+0000 7f49acc00640 1 -- 172.21.15.131:0/31122433 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f4998058b60 con 0x7f49b015bfa0 2024-05-26T22:55:34.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.316+0000 7f49b5800640 1 -- 172.21.15.131:0/31122433 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f49b00d0620 con 0x7f49b015bfa0 2024-05-26T22:55:34.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.317+0000 7f49acc00640 1 -- 172.21.15.131:0/31122433 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v95) ==== 100+0+11 (secure 0 0 0) 0x7f4998058460 con 0x7f49b015bfa0 2024-05-26T22:55:34.317 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 64 2024-05-26T22:55:34.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.320+0000 7f49b5800640 1 -- 172.21.15.131:0/31122433 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f498c06be30 msgr2=0x7f498c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:34.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.320+0000 7f49b5800640 1 --2- 172.21.15.131:0/31122433 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f498c06be30 0x7f498c06e2f0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f49b00d03c0 tx=0x7f49a0008040 comp rx=0 tx=0).stop 2024-05-26T22:55:34.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.320+0000 7f49b5800640 1 -- 172.21.15.131:0/31122433 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49b015bfa0 msgr2=0x7f49b00ccec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:34.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.320+0000 7f49b5800640 1 --2- 172.21.15.131:0/31122433 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49b015bfa0 0x7f49b00ccec0 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f499800c4d0 tx=0x7f499800c990 comp rx=0 tx=0).stop 2024-05-26T22:55:34.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.320+0000 7f49b5800640 1 -- 172.21.15.131:0/31122433 shutdown_connections 2024-05-26T22:55:34.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.320+0000 7f49b5800640 1 --2- 172.21.15.131:0/31122433 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f49b015dc40 0x7f49b00cd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:34.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.320+0000 7f49b5800640 1 --2- 172.21.15.131:0/31122433 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f498c06be30 0x7f498c06e2f0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:34.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.320+0000 7f49b5800640 1 --2- 172.21.15.131:0/31122433 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49b015bfa0 0x7f49b00ccec0 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:34.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.320+0000 7f49b5800640 1 --2- 172.21.15.131:0/31122433 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f49b0071c10 0x7f49b0143980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:34.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.320+0000 7f49b5800640 1 -- 172.21.15.131:0/31122433 >> 172.21.15.131:0/31122433 conn(0x7f49b00b5660 msgr2=0x7f49b0153a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:34.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.320+0000 7f49b5800640 1 -- 172.21.15.131:0/31122433 shutdown_connections 2024-05-26T22:55:34.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.320+0000 7f49b5800640 1 -- 172.21.15.131:0/31122433 wait complete. 2024-05-26T22:55:34.339 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-26T22:55:34.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.539+0000 7ff4d0e00640 1 Processor -- start 2024-05-26T22:55:34.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.540+0000 7ff4d0e00640 1 -- start start 2024-05-26T22:55:34.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.541+0000 7ff4d0e00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7ff4cc06f420 0x7ff4cc0cfd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:34.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.541+0000 7ff4d0e00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4cc160180 con 0x7ff4cc0cf590 2024-05-26T22:55:34.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.541+0000 7ff4d0e00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4cc160460 con 0x7ff4cc0b5450 2024-05-26T22:55:34.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.541+0000 7ff4d0e00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7ff4cc160d90 con 0x7ff4cc06f420 2024-05-26T22:55:34.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.542+0000 7ff4ca800640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7ff4cc0cf590 0x7ff4cc06d600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:42050/0 (socket says 172.21.15.131:42050) 2024-05-26T22:55:34.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.542+0000 7ff4ca800640 1 -- 172.21.15.131:0/4252532973 learned_addr learned my addr 172.21.15.131:0/4252532973 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:55:34.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.542+0000 7ff4c9400640 1 --2- 172.21.15.131:0/4252532973 >> v2:172.21.15.187:3300/0 conn(0x7ff4cc06f420 0x7ff4cc0cfd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:34.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.542+0000 7ff4c8a00640 1 -- 172.21.15.131:0/4252532973 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 539632813 0 0) 0x7ff4cc160180 con 0x7ff4cc0cf590 2024-05-26T22:55:34.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.543+0000 7ff4c8a00640 1 -- 172.21.15.131:0/4252532973 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff4b0003520 con 0x7ff4cc0cf590 2024-05-26T22:55:34.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.543+0000 7ff4c8a00640 1 -- 172.21.15.131:0/4252532973 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 969835637 0 0) 0x7ff4b0003520 con 0x7ff4cc0cf590 2024-05-26T22:55:34.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.543+0000 7ff4c8a00640 1 -- 172.21.15.131:0/4252532973 >> v2:172.21.15.187:3300/0 conn(0x7ff4cc06f420 msgr2=0x7ff4cc0cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:34.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.543+0000 7ff4c8a00640 1 --2- 172.21.15.131:0/4252532973 >> v2:172.21.15.187:3300/0 conn(0x7ff4cc06f420 0x7ff4cc0cfd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:34.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.543+0000 7ff4c8a00640 1 -- 172.21.15.131:0/4252532973 >> v1:172.21.15.187:6789/0 conn(0x7ff4cc0b5450 legacy=0x7ff4cc0b43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:34.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.543+0000 7ff4c8a00640 1 -- 172.21.15.131:0/4252532973 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4cc160740 con 0x7ff4cc0cf590 2024-05-26T22:55:34.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.543+0000 7ff4c8a00640 1 -- 172.21.15.131:0/4252532973 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7ff4c40019f0 con 0x7ff4cc0cf590 2024-05-26T22:55:34.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.543+0000 7ff4c8a00640 1 -- 172.21.15.131:0/4252532973 >> v1:172.21.15.131:6789/0 conn(0x7ff4cc0cf590 legacy=0x7ff4cc06d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:34.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.543+0000 7ff4c8a00640 1 --2- 172.21.15.131:0/4252532973 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff4b00039b0 0x7ff4b0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:34.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.543+0000 7ff4c8a00640 1 --2- 172.21.15.131:0/4252532973 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff4b0004800 0x7ff4b0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:34.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.543+0000 7ff4c8a00640 1 --2- 172.21.15.131:0/4252532973 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff4b00051c0 0x7ff4b000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:34.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.543+0000 7ff4c8a00640 1 -- 172.21.15.131:0/4252532973 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff4cc160d90 con 0x7ff4b00051c0 2024-05-26T22:55:34.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.543+0000 7ff4c8a00640 1 -- 172.21.15.131:0/4252532973 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b000c060 con 0x7ff4b00039b0 2024-05-26T22:55:34.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.543+0000 7ff4c8a00640 1 -- 172.21.15.131:0/4252532973 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff4b000c260 con 0x7ff4b0004800 2024-05-26T22:55:34.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.543+0000 7ff4ca800640 1 --2- 172.21.15.131:0/4252532973 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff4b00051c0 0x7ff4b000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:34.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.544+0000 7ff4c9e00640 1 --2- 172.21.15.131:0/4252532973 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff4b00039b0 0x7ff4b0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:34.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.544+0000 7ff4c9400640 1 --2- 172.21.15.131:0/4252532973 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff4b0004800 0x7ff4b0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:34.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.544+0000 7ff4ca800640 1 -- 172.21.15.131:0/4252532973 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff4b0004800 msgr2=0x7ff4b0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:34.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.544+0000 7ff4ca800640 1 --2- 172.21.15.131:0/4252532973 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff4b0004800 0x7ff4b0004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:34.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.544+0000 7ff4ca800640 1 -- 172.21.15.131:0/4252532973 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff4b00039b0 msgr2=0x7ff4b0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:34.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.544+0000 7ff4ca800640 1 --2- 172.21.15.131:0/4252532973 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff4b00039b0 0x7ff4b0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:34.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.544+0000 7ff4ca800640 1 -- 172.21.15.131:0/4252532973 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4b0003520 con 0x7ff4b00051c0 2024-05-26T22:55:34.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.544+0000 7ff4c9e00640 1 --2- 172.21.15.131:0/4252532973 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff4b00039b0 0x7ff4b0003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:55:34.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.545+0000 7ff4ca800640 1 -- 172.21.15.131:0/4252532973 reap_dead start 2024-05-26T22:55:34.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.545+0000 7ff4ca800640 1 --2- 172.21.15.131:0/4252532973 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff4b00051c0 0x7ff4b000b9c0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7ff4c400a840 tx=0x7ff4b0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4b066a03f85368d server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:34.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.545+0000 7ff4c8a00640 1 -- 172.21.15.131:0/4252532973 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4c400a460 con 0x7ff4b00051c0 2024-05-26T22:55:34.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.545+0000 7ff4c8a00640 1 -- 172.21.15.131:0/4252532973 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4c402f070 con 0x7ff4b00051c0 2024-05-26T22:55:34.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.545+0000 7ff4c8a00640 1 -- 172.21.15.131:0/4252532973 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4c402bcb0 con 0x7ff4b00051c0 2024-05-26T22:55:34.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.546+0000 7ff4d0e00640 1 -- 172.21.15.131:0/4252532973 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff4b00051c0 msgr2=0x7ff4b000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:34.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.546+0000 7ff4d0e00640 1 --2- 172.21.15.131:0/4252532973 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff4b00051c0 0x7ff4b000b9c0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7ff4c400a840 tx=0x7ff4b0005830 comp rx=0 tx=0).stop 2024-05-26T22:55:34.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.546+0000 7ff4d0e00640 1 -- 172.21.15.131:0/4252532973 shutdown_connections 2024-05-26T22:55:34.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.546+0000 7ff4d0e00640 1 --2- 172.21.15.131:0/4252532973 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff4b00051c0 0x7ff4b000b9c0 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:34.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.546+0000 7ff4d0e00640 1 -- 172.21.15.131:0/4252532973 >> 172.21.15.131:0/4252532973 conn(0x7ff4cc06bc90 msgr2=0x7ff4cc0b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:34.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.546+0000 7ff4d0e00640 1 -- 172.21.15.131:0/4252532973 shutdown_connections 2024-05-26T22:55:34.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.546+0000 7ff4d0e00640 1 -- 172.21.15.131:0/4252532973 wait complete. 2024-05-26T22:55:34.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.548+0000 7ff4d0e00640 1 Processor -- start 2024-05-26T22:55:34.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.548+0000 7ff4d0e00640 1 -- start start 2024-05-26T22:55:34.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.549+0000 7ff4d0e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff4b00051c0 0x7ff4cc071360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:34.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.549+0000 7ff4d0e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff4cc1509d0 0x7ff4cc0ce2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:34.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.549+0000 7ff4d0e00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff4cc1500f0 0x7ff4cc153c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:34.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.549+0000 7ff4d0e00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff4cc15fde0 con 0x7ff4cc1509d0 2024-05-26T22:55:34.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.549+0000 7ff4d0e00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff4cc0b5830 con 0x7ff4cc1500f0 2024-05-26T22:55:34.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.549+0000 7ff4d0e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff4cc15ff60 con 0x7ff4b00051c0 2024-05-26T22:55:34.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.549+0000 7ff4c9400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff4cc1509d0 0x7ff4cc0ce2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:34.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.549+0000 7ff4c9400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff4cc1509d0 0x7ff4cc0ce2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:36290/0 (socket says 172.21.15.131:36290) 2024-05-26T22:55:34.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.549+0000 7ff4c9400640 1 -- 172.21.15.131:0/1878162803 learned_addr learned my addr 172.21.15.131:0/1878162803 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:55:34.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.549+0000 7ff4ca800640 1 --2- 172.21.15.131:0/1878162803 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff4cc1500f0 0x7ff4cc153c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:34.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.549+0000 7ff4c9e00640 1 --2- 172.21.15.131:0/1878162803 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff4b00051c0 0x7ff4cc071360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:34.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.549+0000 7ff4c9400640 1 -- 172.21.15.131:0/1878162803 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff4b00051c0 msgr2=0x7ff4cc071360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:34.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.549+0000 7ff4c9400640 1 --2- 172.21.15.131:0/1878162803 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff4b00051c0 0x7ff4cc071360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:34.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.549+0000 7ff4c9400640 1 -- 172.21.15.131:0/1878162803 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff4cc1500f0 msgr2=0x7ff4cc153c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:34.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.549+0000 7ff4c9400640 1 --2- 172.21.15.131:0/1878162803 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff4cc1500f0 0x7ff4cc153c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:34.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.549+0000 7ff4c9400640 1 -- 172.21.15.131:0/1878162803 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4cc0c93c0 con 0x7ff4cc1509d0 2024-05-26T22:55:34.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.549+0000 7ff4ca800640 1 --2- 172.21.15.131:0/1878162803 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff4cc1500f0 0x7ff4cc153c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:55:34.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.550+0000 7ff4c9400640 1 --2- 172.21.15.131:0/1878162803 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff4cc1509d0 0x7ff4cc0ce2c0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7ff4b4010860 tx=0x7ff4b4010d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:34.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.550+0000 7ff4c2000640 1 -- 172.21.15.131:0/1878162803 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b400f040 con 0x7ff4cc1509d0 2024-05-26T22:55:34.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.550+0000 7ff4c2000640 1 -- 172.21.15.131:0/1878162803 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4b400eb00 con 0x7ff4cc1509d0 2024-05-26T22:55:34.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.550+0000 7ff4d0e00640 1 -- 172.21.15.131:0/1878162803 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4cc0c96a0 con 0x7ff4cc1509d0 2024-05-26T22:55:34.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.550+0000 7ff4c2000640 1 -- 172.21.15.131:0/1878162803 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b401c400 con 0x7ff4cc1509d0 2024-05-26T22:55:34.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.550+0000 7ff4d0e00640 1 -- 172.21.15.131:0/1878162803 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4cc152510 con 0x7ff4cc1509d0 2024-05-26T22:55:34.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.551+0000 7ff4d0e00640 1 -- 172.21.15.131:0/1878162803 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4cc0695c0 con 0x7ff4cc1509d0 2024-05-26T22:55:34.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.552+0000 7ff4c2000640 1 -- 172.21.15.131:0/1878162803 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff4b401c5a0 con 0x7ff4cc1509d0 2024-05-26T22:55:34.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.553+0000 7ff4c2000640 1 --2- 172.21.15.131:0/1878162803 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ff4a806be30 0x7ff4a806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:34.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.553+0000 7ff4c2000640 1 -- 172.21.15.131:0/1878162803 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 5709+0+0 (secure 0 0 0) 0x7ff4b408b3b0 con 0x7ff4cc1509d0 2024-05-26T22:55:34.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.553+0000 7ff4c9e00640 1 --2- 172.21.15.131:0/1878162803 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ff4a806be30 0x7ff4a806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:34.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.554+0000 7ff4c9e00640 1 --2- 172.21.15.131:0/1878162803 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ff4a806be30 0x7ff4a806e2f0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc0b4890 tx=0x7ff4bc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:34.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.558+0000 7ff4c2000640 1 -- 172.21.15.131:0/1878162803 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7ff4b4018030 con 0x7ff4cc1509d0 2024-05-26T22:55:34.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.769+0000 7ff4d0e00640 1 -- 172.21.15.131:0/1878162803 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ff4cc0cee90 con 0x7ff4cc1509d0 2024-05-26T22:55:34.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.770+0000 7ff4c2000640 1 -- 172.21.15.131:0/1878162803 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v95) ==== 80+0+0 (secure 0 0 0) 0x7ff4b4059df0 con 0x7ff4cc1509d0 2024-05-26T22:55:34.770 INFO:teuthology.orchestra.run.smithi131.stderr:listed 0 entries 2024-05-26T22:55:34.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.773+0000 7ff4d0e00640 1 -- 172.21.15.131:0/1878162803 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ff4a806be30 msgr2=0x7ff4a806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:34.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.773+0000 7ff4d0e00640 1 --2- 172.21.15.131:0/1878162803 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ff4a806be30 0x7ff4a806e2f0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc0b4890 tx=0x7ff4bc009040 comp rx=0 tx=0).stop 2024-05-26T22:55:34.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.773+0000 7ff4d0e00640 1 -- 172.21.15.131:0/1878162803 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff4cc1509d0 msgr2=0x7ff4cc0ce2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:34.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.773+0000 7ff4d0e00640 1 --2- 172.21.15.131:0/1878162803 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff4cc1509d0 0x7ff4cc0ce2c0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7ff4b4010860 tx=0x7ff4b4010d20 comp rx=0 tx=0).stop 2024-05-26T22:55:34.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.773+0000 7ff4d0e00640 1 -- 172.21.15.131:0/1878162803 shutdown_connections 2024-05-26T22:55:34.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.773+0000 7ff4d0e00640 1 --2- 172.21.15.131:0/1878162803 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ff4a806be30 0x7ff4a806e2f0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:34.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.773+0000 7ff4d0e00640 1 --2- 172.21.15.131:0/1878162803 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff4cc1500f0 0x7ff4cc153c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:34.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.773+0000 7ff4d0e00640 1 --2- 172.21.15.131:0/1878162803 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff4cc1509d0 0x7ff4cc0ce2c0 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:34.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.773+0000 7ff4d0e00640 1 --2- 172.21.15.131:0/1878162803 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff4b00051c0 0x7ff4cc071360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:34.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.773+0000 7ff4d0e00640 1 -- 172.21.15.131:0/1878162803 >> 172.21.15.131:0/1878162803 conn(0x7ff4cc06bc90 msgr2=0x7ff4cc06d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:34.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.773+0000 7ff4d0e00640 1 -- 172.21.15.131:0/1878162803 shutdown_connections 2024-05-26T22:55:34.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:34.773+0000 7ff4d0e00640 1 -- 172.21.15.131:0/1878162803 wait complete. 2024-05-26T22:55:34.793 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-26T22:55:34.793 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:55:34.793 DEBUG:teuthology.orchestra.run.smithi131:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-26T22:55:34.928 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi131.front.sepia.ceph.com 2024-05-26T22:55:34.929 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:55:34.929 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-26T22:55:34.973 INFO:tasks.cephfs.mount:Terminating background process 2024-05-26T22:55:34.973 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-05-26T22:55:34.973 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-26T22:55:35.014 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-26T22:55:35.014 INFO:tasks.cephfs.mount:Terminating background process 2024-05-26T22:55:35.014 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-26T22:55:35.014 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-26T22:55:35.054 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-26T22:55:35.054 INFO:tasks.cephfs.mount:Terminating background process 2024-05-26T22:55:35.055 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-05-26T22:55:35.055 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-26T22:55:35.101 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-26T22:55:35.102 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-26T22:55:35.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.299+0000 7f613ba00640 1 Processor -- start 2024-05-26T22:55:35.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.299+0000 7f613ba00640 1 -- start start 2024-05-26T22:55:35.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.301+0000 7f613ba00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f613406ee20 0x7f61340b3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:35.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.301+0000 7f613ba00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f61341569d0 0x7f61340af790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:35.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.301+0000 7f613ba00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f613415cb20 con 0x7f61340ad250 2024-05-26T22:55:35.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.301+0000 7f613ba00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f613415ce00 con 0x7f613406ee20 2024-05-26T22:55:35.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.301+0000 7f613ba00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f613415d0e0 con 0x7f61341569d0 2024-05-26T22:55:35.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.301+0000 7f6133e00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f61340ad250 0x7f61341562c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:42060/0 (socket says 172.21.15.131:42060) 2024-05-26T22:55:35.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.301+0000 7f6133e00640 1 -- 172.21.15.131:0/3041819952 learned_addr learned my addr 172.21.15.131:0/3041819952 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:55:35.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.302+0000 7f6139000640 1 --2- 172.21.15.131:0/3041819952 >> v2:172.21.15.187:3300/0 conn(0x7f613406ee20 0x7f61340b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:35.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.302+0000 7f6139a00640 1 --2- 172.21.15.131:0/3041819952 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f61341569d0 0x7f61340af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:35.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.302+0000 7f6133400640 1 -- 172.21.15.131:0/3041819952 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3429998865 0 0) 0x7f613415cb20 con 0x7f61340ad250 2024-05-26T22:55:35.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.302+0000 7f6133400640 1 -- 172.21.15.131:0/3041819952 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6118003520 con 0x7f61340ad250 2024-05-26T22:55:35.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.302+0000 7f6133400640 1 -- 172.21.15.131:0/3041819952 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3697693301 0 0) 0x7f6118003520 con 0x7f61340ad250 2024-05-26T22:55:35.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.302+0000 7f6133400640 1 -- 172.21.15.131:0/3041819952 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f61341569d0 msgr2=0x7f61340af790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:35.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.302+0000 7f6133400640 1 --2- 172.21.15.131:0/3041819952 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f61341569d0 0x7f61340af790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:35.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.302+0000 7f6133400640 1 -- 172.21.15.131:0/3041819952 >> v2:172.21.15.187:3300/0 conn(0x7f613406ee20 msgr2=0x7f61340b3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:35.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.302+0000 7f6133400640 1 --2- 172.21.15.131:0/3041819952 >> v2:172.21.15.187:3300/0 conn(0x7f613406ee20 0x7f61340b3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:35.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.302+0000 7f6133400640 1 -- 172.21.15.131:0/3041819952 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f613415d3c0 con 0x7f61340ad250 2024-05-26T22:55:35.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.302+0000 7f6133400640 1 -- 172.21.15.131:0/3041819952 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7f611c001670 con 0x7f61340ad250 2024-05-26T22:55:35.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.302+0000 7f6139000640 1 --2- 172.21.15.131:0/3041819952 >> v2:172.21.15.187:3300/0 conn(0x7f613406ee20 0x7f61340b3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-26T22:55:35.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.302+0000 7f6133400640 1 -- 172.21.15.131:0/3041819952 >> v1:172.21.15.131:6789/0 conn(0x7f61340ad250 legacy=0x7f61341562c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:35.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.302+0000 7f6133400640 1 --2- 172.21.15.131:0/3041819952 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f61180039b0 0x7f6118003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:35.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.302+0000 7f6133400640 1 --2- 172.21.15.131:0/3041819952 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6118004800 0x7f6118004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:35.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.302+0000 7f6133400640 1 --2- 172.21.15.131:0/3041819952 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f61341569d0 0x7f61180051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:35.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.302+0000 7f6133400640 1 -- 172.21.15.131:0/3041819952 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f613415ce00 con 0x7f6118004800 2024-05-26T22:55:35.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.302+0000 7f6133400640 1 -- 172.21.15.131:0/3041819952 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f613415d0e0 con 0x7f61180039b0 2024-05-26T22:55:35.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.302+0000 7f6133400640 1 -- 172.21.15.131:0/3041819952 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f611800b9c0 con 0x7f61341569d0 2024-05-26T22:55:35.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.303+0000 7f6133e00640 1 --2- 172.21.15.131:0/3041819952 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6118004800 0x7f6118004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:35.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.303+0000 7f6139a00640 1 --2- 172.21.15.131:0/3041819952 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f61341569d0 0x7f61180051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:35.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.303+0000 7f6139000640 1 --2- 172.21.15.131:0/3041819952 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f61180039b0 0x7f6118003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:35.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.303+0000 7f6133e00640 1 -- 172.21.15.131:0/3041819952 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f61341569d0 msgr2=0x7f61180051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:35.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.303+0000 7f6133e00640 1 --2- 172.21.15.131:0/3041819952 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f61341569d0 0x7f61180051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:35.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.303+0000 7f6133e00640 1 -- 172.21.15.131:0/3041819952 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f61180039b0 msgr2=0x7f6118003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:35.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.303+0000 7f6133e00640 1 --2- 172.21.15.131:0/3041819952 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f61180039b0 0x7f6118003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:35.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.303+0000 7f6133e00640 1 -- 172.21.15.131:0/3041819952 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6118003520 con 0x7f6118004800 2024-05-26T22:55:35.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.303+0000 7f6139000640 1 --2- 172.21.15.131:0/3041819952 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f61180039b0 0x7f6118003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:55:35.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.304+0000 7f6133e00640 1 --2- 172.21.15.131:0/3041819952 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6118004800 0x7f6118004c80 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f611c007fd0 tx=0x7f6118005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5e16d2dae0b852d server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:35.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.304+0000 7f6133400640 1 -- 172.21.15.131:0/3041819952 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f611c007d50 con 0x7f6118004800 2024-05-26T22:55:35.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.304+0000 7f6133400640 1 -- 172.21.15.131:0/3041819952 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f611c005ad0 con 0x7f6118004800 2024-05-26T22:55:35.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.304+0000 7f6133400640 1 -- 172.21.15.131:0/3041819952 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f611c007070 con 0x7f6118004800 2024-05-26T22:55:35.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.305+0000 7f613ba00640 1 -- 172.21.15.131:0/3041819952 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6118004800 msgr2=0x7f6118004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:35.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.305+0000 7f613ba00640 1 --2- 172.21.15.131:0/3041819952 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6118004800 0x7f6118004c80 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f611c007fd0 tx=0x7f6118005830 comp rx=0 tx=0).stop 2024-05-26T22:55:35.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.305+0000 7f6139a00640 1 -- 172.21.15.131:0/3041819952 reap_dead start 2024-05-26T22:55:35.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.305+0000 7f613ba00640 1 -- 172.21.15.131:0/3041819952 shutdown_connections 2024-05-26T22:55:35.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.305+0000 7f613ba00640 1 -- 172.21.15.131:0/3041819952 >> 172.21.15.131:0/3041819952 conn(0x7f61340b1e90 msgr2=0x7f61340b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:35.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.305+0000 7f613ba00640 1 -- 172.21.15.131:0/3041819952 shutdown_connections 2024-05-26T22:55:35.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.305+0000 7f613ba00640 1 -- 172.21.15.131:0/3041819952 wait complete. 2024-05-26T22:55:35.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.307+0000 7f613ba00640 1 Processor -- start 2024-05-26T22:55:35.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.307+0000 7f613ba00640 1 -- start start 2024-05-26T22:55:35.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.308+0000 7f613ba00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6118004800 0x7f61341f4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:35.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.308+0000 7f613ba00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f613406e060 0x7f613414d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:35.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.308+0000 7f613ba00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f613415eae0 0x7f613415ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:35.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.308+0000 7f613ba00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f613415c900 con 0x7f613406e060 2024-05-26T22:55:35.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.308+0000 7f613ba00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f613415c780 con 0x7f6118004800 2024-05-26T22:55:35.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.308+0000 7f613ba00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f613406f200 con 0x7f613415eae0 2024-05-26T22:55:35.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.308+0000 7f6133e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f613406e060 0x7f613414d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:35.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.308+0000 7f6133e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f613406e060 0x7f613414d190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:36298/0 (socket says 172.21.15.131:36298) 2024-05-26T22:55:35.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.308+0000 7f6133e00640 1 -- 172.21.15.131:0/1719179526 learned_addr learned my addr 172.21.15.131:0/1719179526 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:55:35.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.309+0000 7f6139000640 1 --2- 172.21.15.131:0/1719179526 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6118004800 0x7f61341f4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:35.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.309+0000 7f6139a00640 1 --2- 172.21.15.131:0/1719179526 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f613415eae0 0x7f613415ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:35.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.309+0000 7f6133e00640 1 -- 172.21.15.131:0/1719179526 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f613415eae0 msgr2=0x7f613415ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:35.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.309+0000 7f6133e00640 1 --2- 172.21.15.131:0/1719179526 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f613415eae0 0x7f613415ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:35.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.309+0000 7f6133e00640 1 -- 172.21.15.131:0/1719179526 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6118004800 msgr2=0x7f61341f4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:35.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.309+0000 7f6133e00640 1 --2- 172.21.15.131:0/1719179526 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6118004800 0x7f61341f4290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:35.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.309+0000 7f6133e00640 1 -- 172.21.15.131:0/1719179526 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61341f47d0 con 0x7f613406e060 2024-05-26T22:55:35.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.309+0000 7f6133e00640 1 --2- 172.21.15.131:0/1719179526 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f613406e060 0x7f613414d190 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f611c034040 tx=0x7f613415d590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:35.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.309+0000 7f6130c00640 1 -- 172.21.15.131:0/1719179526 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f611c042020 con 0x7f613406e060 2024-05-26T22:55:35.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.309+0000 7f6130c00640 1 -- 172.21.15.131:0/1719179526 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f611c004bf0 con 0x7f613406e060 2024-05-26T22:55:35.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.309+0000 7f6130c00640 1 -- 172.21.15.131:0/1719179526 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f611c001c80 con 0x7f613406e060 2024-05-26T22:55:35.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.309+0000 7f613ba00640 1 -- 172.21.15.131:0/1719179526 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61341f7ea0 con 0x7f613406e060 2024-05-26T22:55:35.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.310+0000 7f613ba00640 1 -- 172.21.15.131:0/1719179526 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61341f8420 con 0x7f613406e060 2024-05-26T22:55:35.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.311+0000 7f613ba00640 1 -- 172.21.15.131:0/1719179526 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61340694e0 con 0x7f613406e060 2024-05-26T22:55:35.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.311+0000 7f6130c00640 1 -- 172.21.15.131:0/1719179526 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f611c045020 con 0x7f613406e060 2024-05-26T22:55:35.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.312+0000 7f6130c00640 1 --2- 172.21.15.131:0/1719179526 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f611006be60 0x7f611006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:35.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.312+0000 7f6130c00640 1 -- 172.21.15.131:0/1719179526 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 5709+0+0 (secure 0 0 0) 0x7f611c0ab2d0 con 0x7f613406e060 2024-05-26T22:55:35.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.313+0000 7f6139000640 1 --2- 172.21.15.131:0/1719179526 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f611006be60 0x7f611006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:35.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.313+0000 7f6139000640 1 --2- 172.21.15.131:0/1719179526 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f611006be60 0x7f611006e320 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f6124006fd0 tx=0x7f6124008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:35.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.317+0000 7f6130c00640 1 -- 172.21.15.131:0/1719179526 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f611c079b50 con 0x7f613406e060 2024-05-26T22:55:35.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.563+0000 7f613ba00640 1 -- 172.21.15.131:0/1719179526 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6134155cf0 con 0x7f613406e060 2024-05-26T22:55:35.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.564+0000 7f6130c00640 1 -- 172.21.15.131:0/1719179526 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) ==== 95+0+5232 (secure 0 0 0) 0x7f611c079450 con 0x7f613406e060 2024-05-26T22:55:35.564 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:55:35.564 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":100,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5231,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/2037149126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6835","nonce":2037149126},{"type":"v1","addr":"172.21.15.131:6837","nonce":2037149126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":89}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-26T22:54:57.328457+0000","modified":"2024-05-26T22:55:27.458302+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7863,"mds_1":5246,"mds_2":5243},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5243":{"gid":5243,"name":"d","rank":2,"incarnation":97,"state":"up:active","state_seq":7,"addr":"172.21.15.187:6833/1314245822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1314245822},{"type":"v1","addr":"172.21.15.187:6833","nonce":1314245822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5246":{"gid":5246,"name":"b","rank":1,"incarnation":95,"state":"up:active","state_seq":6,"addr":"172.21.15.187:6835/521878375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":521878375},{"type":"v1","addr":"172.21.15.187:6835","nonce":521878375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7863":{"gid":7863,"name":"a","rank":0,"incarnation":90,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6836/789312503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":789312503},{"type":"v1","addr":"172.21.15.131:6836","nonce":789312503}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7863,"qdb_cluster":[5243,5246,7863]},"id":7}]} 2024-05-26T22:55:35.564 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 100 2024-05-26T22:55:35.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.567+0000 7f613ba00640 1 -- 172.21.15.131:0/1719179526 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f611006be60 msgr2=0x7f611006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:35.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.567+0000 7f613ba00640 1 --2- 172.21.15.131:0/1719179526 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f611006be60 0x7f611006e320 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f6124006fd0 tx=0x7f6124008040 comp rx=0 tx=0).stop 2024-05-26T22:55:35.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.567+0000 7f613ba00640 1 -- 172.21.15.131:0/1719179526 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f613406e060 msgr2=0x7f613414d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:35.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.567+0000 7f613ba00640 1 --2- 172.21.15.131:0/1719179526 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f613406e060 0x7f613414d190 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f611c034040 tx=0x7f613415d590 comp rx=0 tx=0).stop 2024-05-26T22:55:35.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.567+0000 7f613ba00640 1 -- 172.21.15.131:0/1719179526 shutdown_connections 2024-05-26T22:55:35.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.567+0000 7f613ba00640 1 --2- 172.21.15.131:0/1719179526 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f613415eae0 0x7f613415ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:35.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.567+0000 7f613ba00640 1 --2- 172.21.15.131:0/1719179526 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f611006be60 0x7f611006e320 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:35.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.567+0000 7f613ba00640 1 --2- 172.21.15.131:0/1719179526 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f613406e060 0x7f613414d190 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:35.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.567+0000 7f613ba00640 1 --2- 172.21.15.131:0/1719179526 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6118004800 0x7f61341f4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:35.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.567+0000 7f613ba00640 1 -- 172.21.15.131:0/1719179526 >> 172.21.15.131:0/1719179526 conn(0x7f61340b1e90 msgr2=0x7f613415b1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:35.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.567+0000 7f613ba00640 1 -- 172.21.15.131:0/1719179526 shutdown_connections 2024-05-26T22:55:35.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.567+0000 7f613ba00640 1 -- 172.21.15.131:0/1719179526 wait complete. 2024-05-26T22:55:35.586 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-26T22:55:35.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.787+0000 7fba19e00640 1 Processor -- start 2024-05-26T22:55:35.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.787+0000 7fba19e00640 1 -- start start 2024-05-26T22:55:35.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.788+0000 7fba19e00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fba1415e570 0x7fba140d4c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:35.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.788+0000 7fba19e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fba140b2f60 0x7fba140d12a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:35.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.788+0000 7fba19e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fba140d1ea0 0x7fba140d17e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:35.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.788+0000 7fba19e00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fba140d51b0 con 0x7fba140b2f60 2024-05-26T22:55:35.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.788+0000 7fba19e00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fba140d47e0 con 0x7fba1415e570 2024-05-26T22:55:35.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.788+0000 7fba19e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fba140b37e0 con 0x7fba140d1ea0 2024-05-26T22:55:35.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.789+0000 7fba12000640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fba140b2f60 0x7fba140d12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:35.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.789+0000 7fba12000640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fba140b2f60 0x7fba140d12a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:36314/0 (socket says 172.21.15.131:36314) 2024-05-26T22:55:35.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.789+0000 7fba12000640 1 -- 172.21.15.131:0/925765270 learned_addr learned my addr 172.21.15.131:0/925765270 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:55:35.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.789+0000 7fba13400640 1 --2- 172.21.15.131:0/925765270 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fba140d1ea0 0x7fba140d17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:35.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.789+0000 7fba12a00640 1 --2- 172.21.15.131:0/925765270 >> v2:172.21.15.187:3300/0 conn(0x7fba1415e570 0x7fba140d4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:35.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.789+0000 7fba12000640 1 -- 172.21.15.131:0/925765270 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fba140d1ea0 msgr2=0x7fba140d17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:35.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.789+0000 7fba12000640 1 --2- 172.21.15.131:0/925765270 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fba140d1ea0 0x7fba140d17e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:35.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.789+0000 7fba12000640 1 -- 172.21.15.131:0/925765270 >> v2:172.21.15.187:3300/0 conn(0x7fba1415e570 msgr2=0x7fba140d4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:35.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.789+0000 7fba12000640 1 --2- 172.21.15.131:0/925765270 >> v2:172.21.15.187:3300/0 conn(0x7fba1415e570 0x7fba140d4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:35.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.789+0000 7fba12000640 1 -- 172.21.15.131:0/925765270 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba140b9cf0 con 0x7fba140b2f60 2024-05-26T22:55:35.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.789+0000 7fba12a00640 1 --2- 172.21.15.131:0/925765270 >> v2:172.21.15.187:3300/0 conn(0x7fba1415e570 0x7fba140d4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:55:35.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.790+0000 7fba12000640 1 --2- 172.21.15.131:0/925765270 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba140b2f60 0x7fba140d12a0 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7fba140b6a90 tx=0x7fb9fc02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3173f2c2a0f9234 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:35.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.791+0000 7fba11600640 1 -- 172.21.15.131:0/925765270 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9fc037070 con 0x7fba140b2f60 2024-05-26T22:55:35.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.791+0000 7fba11600640 1 -- 172.21.15.131:0/925765270 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9fc032440 con 0x7fba140b2f60 2024-05-26T22:55:35.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.791+0000 7fba11600640 1 -- 172.21.15.131:0/925765270 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9fc032810 con 0x7fba140b2f60 2024-05-26T22:55:35.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.791+0000 7fba19e00640 1 -- 172.21.15.131:0/925765270 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba140b2f60 msgr2=0x7fba140d12a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:35.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.791+0000 7fba19e00640 1 --2- 172.21.15.131:0/925765270 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba140b2f60 0x7fba140d12a0 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7fba140b6a90 tx=0x7fb9fc02a140 comp rx=0 tx=0).stop 2024-05-26T22:55:35.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.791+0000 7fba19e00640 1 -- 172.21.15.131:0/925765270 shutdown_connections 2024-05-26T22:55:35.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.791+0000 7fba19e00640 1 --2- 172.21.15.131:0/925765270 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fba140d1ea0 0x7fba140d17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:35.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.791+0000 7fba19e00640 1 --2- 172.21.15.131:0/925765270 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba140b2f60 0x7fba140d12a0 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:35.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.791+0000 7fba19e00640 1 --2- 172.21.15.131:0/925765270 >> v2:172.21.15.187:3300/0 conn(0x7fba1415e570 0x7fba140d4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:35.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.791+0000 7fba19e00640 1 -- 172.21.15.131:0/925765270 >> 172.21.15.131:0/925765270 conn(0x7fba14072e90 msgr2=0x7fba140732a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:35.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.792+0000 7fba19e00640 1 -- 172.21.15.131:0/925765270 shutdown_connections 2024-05-26T22:55:35.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.792+0000 7fba19e00640 1 -- 172.21.15.131:0/925765270 wait complete. 2024-05-26T22:55:35.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.793+0000 7fba19e00640 1 Processor -- start 2024-05-26T22:55:35.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.793+0000 7fba19e00640 1 -- start start 2024-05-26T22:55:35.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.794+0000 7fba19e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba140b2f60 0x7fba14152a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:35.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.794+0000 7fba19e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fba140d1ea0 0x7fba140ca570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:35.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.794+0000 7fba19e00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fba1415e570 0x7fba140cec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:35.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.794+0000 7fba19e00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fba140d2590 con 0x7fba140b2f60 2024-05-26T22:55:35.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.794+0000 7fba19e00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fba140d4470 con 0x7fba1415e570 2024-05-26T22:55:35.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.794+0000 7fba19e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fba140d2710 con 0x7fba140d1ea0 2024-05-26T22:55:35.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.795+0000 7fba12a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba140b2f60 0x7fba14152a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:35.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.795+0000 7fba12a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba140b2f60 0x7fba14152a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:36322/0 (socket says 172.21.15.131:36322) 2024-05-26T22:55:35.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.795+0000 7fba12a00640 1 -- 172.21.15.131:0/1418757145 learned_addr learned my addr 172.21.15.131:0/1418757145 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:55:35.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.795+0000 7fba12000640 1 --2- 172.21.15.131:0/1418757145 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fba140d1ea0 0x7fba140ca570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:35.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.795+0000 7fba13400640 1 --2- 172.21.15.131:0/1418757145 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fba1415e570 0x7fba140cec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:35.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.795+0000 7fba12a00640 1 -- 172.21.15.131:0/1418757145 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fba140d1ea0 msgr2=0x7fba140ca570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:35.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.795+0000 7fba12a00640 1 --2- 172.21.15.131:0/1418757145 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fba140d1ea0 0x7fba140ca570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:35.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.795+0000 7fba12a00640 1 -- 172.21.15.131:0/1418757145 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fba1415e570 msgr2=0x7fba140cec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:35.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.795+0000 7fba12a00640 1 --2- 172.21.15.131:0/1418757145 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fba1415e570 0x7fba140cec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:35.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.795+0000 7fba12a00640 1 -- 172.21.15.131:0/1418757145 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba140cc960 con 0x7fba140b2f60 2024-05-26T22:55:35.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.795+0000 7fba12000640 1 --2- 172.21.15.131:0/1418757145 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fba140d1ea0 0x7fba140ca570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:55:35.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.795+0000 7fba12a00640 1 --2- 172.21.15.131:0/1418757145 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba140b2f60 0x7fba14152a80 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7fba0400c7f0 tx=0x7fba0400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:35.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.796+0000 7fba0b400640 1 -- 172.21.15.131:0/1418757145 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba04015070 con 0x7fba140b2f60 2024-05-26T22:55:35.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.796+0000 7fba0b400640 1 -- 172.21.15.131:0/1418757145 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba04007820 con 0x7fba140b2f60 2024-05-26T22:55:35.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.796+0000 7fba19e00640 1 -- 172.21.15.131:0/1418757145 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba140ccc40 con 0x7fba140b2f60 2024-05-26T22:55:35.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.796+0000 7fba0b400640 1 -- 172.21.15.131:0/1418757145 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba04007bd0 con 0x7fba140b2f60 2024-05-26T22:55:35.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.796+0000 7fba19e00640 1 -- 172.21.15.131:0/1418757145 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba140cd200 con 0x7fba140b2f60 2024-05-26T22:55:35.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.797+0000 7fba19e00640 1 -- 172.21.15.131:0/1418757145 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba140694e0 con 0x7fba140b2f60 2024-05-26T22:55:35.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.798+0000 7fba0b400640 1 -- 172.21.15.131:0/1418757145 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fba04018070 con 0x7fba140b2f60 2024-05-26T22:55:35.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.799+0000 7fba0b400640 1 --2- 172.21.15.131:0/1418757145 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb9f006be30 0x7fb9f006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:35.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.799+0000 7fba0b400640 1 -- 172.21.15.131:0/1418757145 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 5709+0+0 (secure 0 0 0) 0x7fba0408af70 con 0x7fba140b2f60 2024-05-26T22:55:35.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.799+0000 7fba12000640 1 --2- 172.21.15.131:0/1418757145 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb9f006be30 0x7fb9f006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:35.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.800+0000 7fba12000640 1 --2- 172.21.15.131:0/1418757145 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb9f006be30 0x7fb9f006e2f0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc034000 tx=0x7fb9fc02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:35.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:35.803+0000 7fba0b400640 1 -- 172.21.15.131:0/1418757145 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fba04059910 con 0x7fba140b2f60 2024-05-26T22:55:36.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.002+0000 7fba19e00640 1 -- 172.21.15.131:0/1418757145 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fba14155520 con 0x7fba140b2f60 2024-05-26T22:55:36.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.005+0000 7fba0b400640 1 -- 172.21.15.131:0/1418757145 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) ==== 74+0+16632 (secure 0 0 0) 0x7fba04059210 con 0x7fba140b2f60 2024-05-26T22:55:36.005 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:55:36.005 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":95,"fsid":"46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f","created":"2024-05-26T22:47:57.288905+0000","modified":"2024-05-26T22:54:58.901569+0000","last_up_change":"2024-05-26T22:48:09.602207+0000","last_in_change":"2024-05-26T22:48:00.687341+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-26T22:48:10.463339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-05-26T22:54:55.120321+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-05-26T22:54:56.346430+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"814eaf62-a18d-494b-8e27-fd8fb88adea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6801","nonce":2350940051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6803","nonce":2350940051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6807","nonce":2350940051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6805","nonce":2350940051}]},"public_addr":"172.21.15.131:6801/2350940051","cluster_addr":"172.21.15.131:6803/2350940051","heartbeat_back_addr":"172.21.15.131:6807/2350940051","heartbeat_front_addr":"172.21.15.131:6805/2350940051","state":["exists","up"]},{"osd":1,"uuid":"42e66287-f6c6-4e26-af3c-6dfaa3536d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6825","nonce":1525528247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6827","nonce":1525528247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6831","nonce":1525528247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6829","nonce":1525528247}]},"public_addr":"172.21.15.131:6825/1525528247","cluster_addr":"172.21.15.131:6827/1525528247","heartbeat_back_addr":"172.21.15.131:6831/1525528247","heartbeat_front_addr":"172.21.15.131:6829/1525528247","state":["exists","up"]},{"osd":2,"uuid":"0da34673-04a2-4fd0-b57a-0ff2b679b043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6817","nonce":3627842210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6819","nonce":3627842210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6823","nonce":3627842210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6821","nonce":3627842210}]},"public_addr":"172.21.15.131:6817/3627842210","cluster_addr":"172.21.15.131:6819/3627842210","heartbeat_back_addr":"172.21.15.131:6823/3627842210","heartbeat_front_addr":"172.21.15.131:6821/3627842210","state":["exists","up"]},{"osd":3,"uuid":"cf117a27-d6b2-43d3-ae52-b47988bf8eed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6809","nonce":350790591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6811","nonce":350790591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6815","nonce":350790591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6813","nonce":350790591}]},"public_addr":"172.21.15.131:6809/350790591","cluster_addr":"172.21.15.131:6811/350790591","heartbeat_back_addr":"172.21.15.131:6815/350790591","heartbeat_front_addr":"172.21.15.131:6813/350790591","state":["exists","up"]},{"osd":4,"uuid":"a092ccfc-be89-4a84-88d5-0f781fa715eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6816","nonce":244560995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6819","nonce":244560995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6823","nonce":244560995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6821","nonce":244560995}]},"public_addr":"172.21.15.187:6816/244560995","cluster_addr":"172.21.15.187:6819/244560995","heartbeat_back_addr":"172.21.15.187:6823/244560995","heartbeat_front_addr":"172.21.15.187:6821/244560995","state":["exists","up"]},{"osd":5,"uuid":"7513ba69-dd00-4a35-b190-9e8cc3b2bdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6801","nonce":324619398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6803","nonce":324619398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6812","nonce":324619398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6807","nonce":324619398}]},"public_addr":"172.21.15.187:6801/324619398","cluster_addr":"172.21.15.187:6803/324619398","heartbeat_back_addr":"172.21.15.187:6812/324619398","heartbeat_front_addr":"172.21.15.187:6807/324619398","state":["exists","up"]},{"osd":6,"uuid":"50081810-c093-4d2f-a4d7-b7c695c5400d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6806","nonce":1391347786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6809","nonce":1391347786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6818","nonce":1391347786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6813","nonce":1391347786}]},"public_addr":"172.21.15.187:6806/1391347786","cluster_addr":"172.21.15.187:6809/1391347786","heartbeat_back_addr":"172.21.15.187:6818/1391347786","heartbeat_front_addr":"172.21.15.187:6813/1391347786","state":["exists","up"]},{"osd":7,"uuid":"775dae05-54f5-4ad4-bbce-92863c6ed2ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6825","nonce":1220065622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6827","nonce":1220065622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6831","nonce":1220065622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6829","nonce":1220065622}]},"public_addr":"172.21.15.187:6825/1220065622","cluster_addr":"172.21.15.187:6827/1220065622","heartbeat_back_addr":"172.21.15.187:6831/1220065622","heartbeat_front_addr":"172.21.15.187:6829/1220065622","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:05.983520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.299516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.067323+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.307748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.018742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.258775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.010912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.295355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-26T22:55:36.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.007+0000 7fba19e00640 1 -- 172.21.15.131:0/1418757145 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb9f006be30 msgr2=0x7fb9f006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:36.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.007+0000 7fba19e00640 1 --2- 172.21.15.131:0/1418757145 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb9f006be30 0x7fb9f006e2f0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc034000 tx=0x7fb9fc02e000 comp rx=0 tx=0).stop 2024-05-26T22:55:36.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.007+0000 7fba19e00640 1 -- 172.21.15.131:0/1418757145 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba140b2f60 msgr2=0x7fba14152a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:36.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.007+0000 7fba19e00640 1 --2- 172.21.15.131:0/1418757145 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba140b2f60 0x7fba14152a80 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7fba0400c7f0 tx=0x7fba0400ccb0 comp rx=0 tx=0).stop 2024-05-26T22:55:36.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.008+0000 7fba19e00640 1 -- 172.21.15.131:0/1418757145 shutdown_connections 2024-05-26T22:55:36.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.008+0000 7fba19e00640 1 --2- 172.21.15.131:0/1418757145 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fba140d1ea0 0x7fba140ca570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:36.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.008+0000 7fba19e00640 1 --2- 172.21.15.131:0/1418757145 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb9f006be30 0x7fb9f006e2f0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:36.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.008+0000 7fba19e00640 1 --2- 172.21.15.131:0/1418757145 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fba1415e570 0x7fba140cec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:36.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.008+0000 7fba19e00640 1 --2- 172.21.15.131:0/1418757145 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba140b2f60 0x7fba14152a80 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:36.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.008+0000 7fba19e00640 1 -- 172.21.15.131:0/1418757145 >> 172.21.15.131:0/1418757145 conn(0x7fba14072e90 msgr2=0x7fba140b73f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:36.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.008+0000 7fba19e00640 1 -- 172.21.15.131:0/1418757145 shutdown_connections 2024-05-26T22:55:36.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.008+0000 7fba19e00640 1 -- 172.21.15.131:0/1418757145 wait complete. 2024-05-26T22:55:36.025 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-26T22:48:10.463339+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-26T22:54:55.120321+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '94', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2024-05-26T22:54:56.346430+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '95', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-26T22:55:36.026 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-26T22:55:36.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.223+0000 7f660fe00640 1 Processor -- start 2024-05-26T22:55:36.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.223+0000 7f660fe00640 1 -- start start 2024-05-26T22:55:36.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.225+0000 7f660fe00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f6610071260 0x7f6610071640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:36.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.225+0000 7f660fe00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66101646a0 con 0x7f66101638e0 2024-05-26T22:55:36.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.225+0000 7f660fe00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6610071b80 con 0x7f6610162dd0 2024-05-26T22:55:36.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.225+0000 7f660fe00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f6610166c10 con 0x7f6610071260 2024-05-26T22:55:36.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.225+0000 7f660f400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f6610071260 0x7f6610071640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:36.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.225+0000 7f660ea00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f66101638e0 0x7f66101663c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:42062/0 (socket says 172.21.15.131:42062) 2024-05-26T22:55:36.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.225+0000 7f660f400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f6610071260 0x7f6610071640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:36326/0 (socket says 172.21.15.131:36326) 2024-05-26T22:55:36.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.225+0000 7f660ea00640 1 -- 172.21.15.131:0/2112094192 learned_addr learned my addr 172.21.15.131:0/2112094192 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:55:36.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.226+0000 7f660f400640 1 -- 172.21.15.131:0/2112094192 >> v1:172.21.15.187:6789/0 conn(0x7f6610162dd0 legacy=0x7f6610163f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:36.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.226+0000 7f660f400640 1 -- 172.21.15.131:0/2112094192 >> v1:172.21.15.131:6789/0 conn(0x7f66101638e0 legacy=0x7f66101663c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:36.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.226+0000 7f660d600640 1 -- 172.21.15.131:0/2112094192 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1749032505 0 0) 0x7f66101646a0 con 0x7f66101638e0 2024-05-26T22:55:36.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.226+0000 7f660f400640 1 -- 172.21.15.131:0/2112094192 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6610166ef0 con 0x7f6610071260 2024-05-26T22:55:36.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.227+0000 7f660f400640 1 --2- 172.21.15.131:0/2112094192 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6610071260 0x7f6610071640 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f66080097a0 tx=0x7f660802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f140fe90ff119284 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:36.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.227+0000 7f660d600640 1 -- 172.21.15.131:0/2112094192 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6608037070 con 0x7f6610071260 2024-05-26T22:55:36.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.227+0000 7f660d600640 1 -- 172.21.15.131:0/2112094192 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f660802ae50 con 0x7f6610071260 2024-05-26T22:55:36.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.227+0000 7f660d600640 1 -- 172.21.15.131:0/2112094192 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6608032660 con 0x7f6610071260 2024-05-26T22:55:36.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.228+0000 7f660fe00640 1 -- 172.21.15.131:0/2112094192 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6610071260 msgr2=0x7f6610071640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:36.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.228+0000 7f660fe00640 1 --2- 172.21.15.131:0/2112094192 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6610071260 0x7f6610071640 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f66080097a0 tx=0x7f660802a140 comp rx=0 tx=0).stop 2024-05-26T22:55:36.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.228+0000 7f660fe00640 1 -- 172.21.15.131:0/2112094192 shutdown_connections 2024-05-26T22:55:36.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.228+0000 7f660fe00640 1 --2- 172.21.15.131:0/2112094192 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6610071260 0x7f6610071640 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:36.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.228+0000 7f660fe00640 1 -- 172.21.15.131:0/2112094192 >> 172.21.15.131:0/2112094192 conn(0x7f66100b5660 msgr2=0x7f66100b5a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:36.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.228+0000 7f660fe00640 1 -- 172.21.15.131:0/2112094192 shutdown_connections 2024-05-26T22:55:36.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.228+0000 7f660fe00640 1 -- 172.21.15.131:0/2112094192 wait complete. 2024-05-26T22:55:36.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.230+0000 7f660fe00640 1 Processor -- start 2024-05-26T22:55:36.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.230+0000 7f660fe00640 1 -- start start 2024-05-26T22:55:36.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.231+0000 7f660fe00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6610071260 0x7f66100aeaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:36.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.231+0000 7f660fe00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6610162dd0 0x7f66101f7930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:36.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.231+0000 7f660fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f66101638e0 0x7f66100acf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:36.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.231+0000 7f660fe00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f66100b3850 con 0x7f66101638e0 2024-05-26T22:55:36.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.231+0000 7f660fe00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f66100b3570 con 0x7f6610162dd0 2024-05-26T22:55:36.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.231+0000 7f660fe00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f6610163cc0 con 0x7f6610071260 2024-05-26T22:55:36.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.231+0000 7f660f400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f66101638e0 0x7f66100acf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:36.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.231+0000 7f660f400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f66101638e0 0x7f66100acf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:36342/0 (socket says 172.21.15.131:36342) 2024-05-26T22:55:36.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.231+0000 7f660f400640 1 -- 172.21.15.131:0/2497912434 learned_addr learned my addr 172.21.15.131:0/2497912434 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:55:36.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.231+0000 7f660ea00640 1 --2- 172.21.15.131:0/2497912434 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6610071260 0x7f66100aeaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:36.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.231+0000 7f660f400640 1 -- 172.21.15.131:0/2497912434 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6610071260 msgr2=0x7f66100aeaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:36.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.231+0000 7f660e000640 1 --2- 172.21.15.131:0/2497912434 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6610162dd0 0x7f66101f7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:36.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.231+0000 7f660f400640 1 --2- 172.21.15.131:0/2497912434 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6610071260 0x7f66100aeaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:36.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.231+0000 7f660f400640 1 -- 172.21.15.131:0/2497912434 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6610162dd0 msgr2=0x7f66101f7930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:36.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.231+0000 7f660f400640 1 --2- 172.21.15.131:0/2497912434 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6610162dd0 0x7f66101f7930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:36.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.231+0000 7f660f400640 1 -- 172.21.15.131:0/2497912434 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f661014a6b0 con 0x7f66101638e0 2024-05-26T22:55:36.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.231+0000 7f660ea00640 1 --2- 172.21.15.131:0/2497912434 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6610071260 0x7f66100aeaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:55:36.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.231+0000 7f660e000640 1 --2- 172.21.15.131:0/2497912434 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6610162dd0 0x7f66101f7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:55:36.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.232+0000 7f660f400640 1 --2- 172.21.15.131:0/2497912434 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f66101638e0 0x7f66100acf00 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f66080097a0 tx=0x7f66080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:36.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.232+0000 7f6606a00640 1 -- 172.21.15.131:0/2497912434 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6608042070 con 0x7f66101638e0 2024-05-26T22:55:36.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.232+0000 7f6606a00640 1 -- 172.21.15.131:0/2497912434 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6608032e10 con 0x7f66101638e0 2024-05-26T22:55:36.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.232+0000 7f6606a00640 1 -- 172.21.15.131:0/2497912434 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6608037040 con 0x7f66101638e0 2024-05-26T22:55:36.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.232+0000 7f660fe00640 1 -- 172.21.15.131:0/2497912434 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66100ada50 con 0x7f66101638e0 2024-05-26T22:55:36.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.232+0000 7f660fe00640 1 -- 172.21.15.131:0/2497912434 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66100ae010 con 0x7f66101638e0 2024-05-26T22:55:36.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.233+0000 7f660fe00640 1 -- 172.21.15.131:0/2497912434 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66100694e0 con 0x7f66101638e0 2024-05-26T22:55:36.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.234+0000 7f6606a00640 1 -- 172.21.15.131:0/2497912434 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6608045070 con 0x7f66101638e0 2024-05-26T22:55:36.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.235+0000 7f6606a00640 1 --2- 172.21.15.131:0/2497912434 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f65ec06be30 0x7f65ec06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:36.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.235+0000 7f6606a00640 1 -- 172.21.15.131:0/2497912434 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 5709+0+0 (secure 0 0 0) 0x7f66080ab670 con 0x7f66101638e0 2024-05-26T22:55:36.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.235+0000 7f660ea00640 1 --2- 172.21.15.131:0/2497912434 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f65ec06be30 0x7f65ec06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:36.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.236+0000 7f660ea00640 1 --2- 172.21.15.131:0/2497912434 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f65ec06be30 0x7f65ec06e2f0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f66000012e0 tx=0x7f6600009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:36.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.239+0000 7f6606a00640 1 -- 172.21.15.131:0/2497912434 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f660807a130 con 0x7f66101638e0 2024-05-26T22:55:36.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.439+0000 7f660fe00640 1 -- 172.21.15.131:0/2497912434 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6610072bc0 con 0x7f66101638e0 2024-05-26T22:55:36.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.440+0000 7f6606a00640 1 -- 172.21.15.131:0/2497912434 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) ==== 93+0+10 (secure 0 0 0) 0x7f6608079a30 con 0x7f66101638e0 2024-05-26T22:55:36.440 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2024-05-26T22:55:36.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.443+0000 7f660fe00640 1 -- 172.21.15.131:0/2497912434 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f65ec06be30 msgr2=0x7f65ec06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:36.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.443+0000 7f660fe00640 1 --2- 172.21.15.131:0/2497912434 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f65ec06be30 0x7f65ec06e2f0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f66000012e0 tx=0x7f6600009040 comp rx=0 tx=0).stop 2024-05-26T22:55:36.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.443+0000 7f660fe00640 1 -- 172.21.15.131:0/2497912434 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f66101638e0 msgr2=0x7f66100acf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:36.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.443+0000 7f660fe00640 1 --2- 172.21.15.131:0/2497912434 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f66101638e0 0x7f66100acf00 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f66080097a0 tx=0x7f66080046c0 comp rx=0 tx=0).stop 2024-05-26T22:55:36.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.443+0000 7f660fe00640 1 -- 172.21.15.131:0/2497912434 shutdown_connections 2024-05-26T22:55:36.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.443+0000 7f660fe00640 1 --2- 172.21.15.131:0/2497912434 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f65ec06be30 0x7f65ec06e2f0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:36.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.443+0000 7f660fe00640 1 --2- 172.21.15.131:0/2497912434 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f66101638e0 0x7f66100acf00 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:36.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.443+0000 7f660fe00640 1 --2- 172.21.15.131:0/2497912434 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6610162dd0 0x7f66101f7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:36.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.443+0000 7f660fe00640 1 --2- 172.21.15.131:0/2497912434 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6610071260 0x7f66100aeaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:36.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.443+0000 7f660fe00640 1 -- 172.21.15.131:0/2497912434 >> 172.21.15.131:0/2497912434 conn(0x7f66100b5660 msgr2=0x7f66100b3d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:36.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.443+0000 7f660fe00640 1 -- 172.21.15.131:0/2497912434 shutdown_connections 2024-05-26T22:55:36.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.443+0000 7f660fe00640 1 -- 172.21.15.131:0/2497912434 wait complete. 2024-05-26T22:55:36.463 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-26T22:55:36.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.672+0000 7fdf03e00640 1 Processor -- start 2024-05-26T22:55:36.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.672+0000 7fdf03e00640 1 -- start start 2024-05-26T22:55:36.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.674+0000 7fdf03e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdefc06f420 0x7fdefc0cfd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:36.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.674+0000 7fdf03e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fdefc0cf590 0x7fdefc0d0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:36.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.674+0000 7fdf03e00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdefc160180 con 0x7fdefc0b5450 2024-05-26T22:55:36.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.674+0000 7fdf03e00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fdefc160d90 con 0x7fdefc0cf590 2024-05-26T22:55:36.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.674+0000 7fdf03e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fdefc160460 con 0x7fdefc06f420 2024-05-26T22:55:36.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.674+0000 7fdf01e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fdefc0cf590 0x7fdefc0d0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:36.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.674+0000 7fdf01e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fdefc0cf590 0x7fdefc0d0290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:36358/0 (socket says 172.21.15.131:36358) 2024-05-26T22:55:36.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.674+0000 7fdf01e00640 1 -- 172.21.15.131:0/2144052365 learned_addr learned my addr 172.21.15.131:0/2144052365 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:55:36.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.674+0000 7fdf00a00640 1 --2- 172.21.15.131:0/2144052365 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdefc06f420 0x7fdefc0cfd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:36.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.675+0000 7fdf01e00640 1 -- 172.21.15.131:0/2144052365 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdefc06f420 msgr2=0x7fdefc0cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:36.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.675+0000 7fdf01e00640 1 --2- 172.21.15.131:0/2144052365 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdefc06f420 0x7fdefc0cfd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:36.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.675+0000 7fdf01e00640 1 -- 172.21.15.131:0/2144052365 >> v1:172.21.15.187:6789/0 conn(0x7fdefc0b5450 legacy=0x7fdefc0b43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:36.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.675+0000 7fdf01e00640 1 -- 172.21.15.131:0/2144052365 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdefc160740 con 0x7fdefc0cf590 2024-05-26T22:55:36.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.676+0000 7fdf01e00640 1 --2- 172.21.15.131:0/2144052365 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdefc0cf590 0x7fdefc0d0290 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7fdef4009e50 tx=0x7fdef402a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b57bf0c0e211da3 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:36.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.676+0000 7fdefbe00640 1 -- 172.21.15.131:0/2144052365 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdef4036070 con 0x7fdefc0cf590 2024-05-26T22:55:36.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.676+0000 7fdefbe00640 1 -- 172.21.15.131:0/2144052365 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdef4031600 con 0x7fdefc0cf590 2024-05-26T22:55:36.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.676+0000 7fdefbe00640 1 -- 172.21.15.131:0/2144052365 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdef40319d0 con 0x7fdefc0cf590 2024-05-26T22:55:36.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.677+0000 7fdf03e00640 1 -- 172.21.15.131:0/2144052365 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdefc0cf590 msgr2=0x7fdefc0d0290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:36.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.677+0000 7fdf03e00640 1 --2- 172.21.15.131:0/2144052365 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdefc0cf590 0x7fdefc0d0290 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7fdef4009e50 tx=0x7fdef402a6c0 comp rx=0 tx=0).stop 2024-05-26T22:55:36.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.677+0000 7fdf03e00640 1 -- 172.21.15.131:0/2144052365 shutdown_connections 2024-05-26T22:55:36.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.677+0000 7fdf03e00640 1 --2- 172.21.15.131:0/2144052365 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdefc0cf590 0x7fdefc0d0290 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:36.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.677+0000 7fdf03e00640 1 --2- 172.21.15.131:0/2144052365 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdefc06f420 0x7fdefc0cfd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:36.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.677+0000 7fdf03e00640 1 -- 172.21.15.131:0/2144052365 >> 172.21.15.131:0/2144052365 conn(0x7fdefc06bc90 msgr2=0x7fdefc0b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:36.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.677+0000 7fdf03e00640 1 -- 172.21.15.131:0/2144052365 shutdown_connections 2024-05-26T22:55:36.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.677+0000 7fdf03e00640 1 -- 172.21.15.131:0/2144052365 wait complete. 2024-05-26T22:55:36.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.679+0000 7fdf03e00640 1 Processor -- start 2024-05-26T22:55:36.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.679+0000 7fdf03e00640 1 -- start start 2024-05-26T22:55:36.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.680+0000 7fdf03e00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdefc06f420 0x7fdefc1f44e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:36.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.680+0000 7fdf03e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdefc0b5450 0x7fdefc06e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:36.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.680+0000 7fdf03e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdefc0cf590 0x7fdefc14c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:36.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.680+0000 7fdf03e00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fdefc15fde0 con 0x7fdefc0cf590 2024-05-26T22:55:36.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.680+0000 7fdf03e00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fdefc0b5830 con 0x7fdefc06f420 2024-05-26T22:55:36.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.680+0000 7fdf03e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fdefc15ff60 con 0x7fdefc0b5450 2024-05-26T22:55:36.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.680+0000 7fdf01e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdefc0cf590 0x7fdefc14c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:36.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.680+0000 7fdf01e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdefc0cf590 0x7fdefc14c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:36374/0 (socket says 172.21.15.131:36374) 2024-05-26T22:55:36.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.680+0000 7fdf01e00640 1 -- 172.21.15.131:0/3179077825 learned_addr learned my addr 172.21.15.131:0/3179077825 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:55:36.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.680+0000 7fdf01400640 1 --2- 172.21.15.131:0/3179077825 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdefc06f420 0x7fdefc1f44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:36.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.680+0000 7fdf00a00640 1 --2- 172.21.15.131:0/3179077825 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdefc0b5450 0x7fdefc06e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:36.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.680+0000 7fdf01e00640 1 -- 172.21.15.131:0/3179077825 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdefc0b5450 msgr2=0x7fdefc06e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:36.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.681+0000 7fdf01e00640 1 --2- 172.21.15.131:0/3179077825 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdefc0b5450 0x7fdefc06e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:36.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.681+0000 7fdf01e00640 1 -- 172.21.15.131:0/3179077825 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdefc06f420 msgr2=0x7fdefc1f44e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:36.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.681+0000 7fdf01e00640 1 --2- 172.21.15.131:0/3179077825 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdefc06f420 0x7fdefc1f44e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:36.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.681+0000 7fdf01e00640 1 -- 172.21.15.131:0/3179077825 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdefc1f4a20 con 0x7fdefc0cf590 2024-05-26T22:55:36.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.681+0000 7fdf00a00640 1 --2- 172.21.15.131:0/3179077825 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdefc0b5450 0x7fdefc06e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:55:36.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.681+0000 7fdf01e00640 1 --2- 172.21.15.131:0/3179077825 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdefc0cf590 0x7fdefc14c600 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7fdef4009e50 tx=0x7fdef4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:36.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.681+0000 7fdef9600640 1 -- 172.21.15.131:0/3179077825 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdef4041020 con 0x7fdefc0cf590 2024-05-26T22:55:36.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.681+0000 7fdef9600640 1 -- 172.21.15.131:0/3179077825 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdef4007780 con 0x7fdefc0cf590 2024-05-26T22:55:36.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.681+0000 7fdf03e00640 1 -- 172.21.15.131:0/3179077825 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdefc09c9a0 con 0x7fdefc0cf590 2024-05-26T22:55:36.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.681+0000 7fdef9600640 1 -- 172.21.15.131:0/3179077825 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdef4036040 con 0x7fdefc0cf590 2024-05-26T22:55:36.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.682+0000 7fdf03e00640 1 -- 172.21.15.131:0/3179077825 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdefc1e61e0 con 0x7fdefc0cf590 2024-05-26T22:55:36.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.683+0000 7fdf03e00640 1 -- 172.21.15.131:0/3179077825 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdefc074e90 con 0x7fdefc0cf590 2024-05-26T22:55:36.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.684+0000 7fdef9600640 1 -- 172.21.15.131:0/3179077825 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdef4044020 con 0x7fdefc0cf590 2024-05-26T22:55:36.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.685+0000 7fdef9600640 1 --2- 172.21.15.131:0/3179077825 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fdedc06be80 0x7fdedc06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:36.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.685+0000 7fdef9600640 1 -- 172.21.15.131:0/3179077825 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 5709+0+0 (secure 0 0 0) 0x7fdef40aba90 con 0x7fdefc0cf590 2024-05-26T22:55:36.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.685+0000 7fdf01400640 1 --2- 172.21.15.131:0/3179077825 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fdedc06be80 0x7fdedc06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:36.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.686+0000 7fdf01400640 1 --2- 172.21.15.131:0/3179077825 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fdedc06be80 0x7fdedc06e340 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fdef0004ea0 tx=0x7fdef0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:36.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.689+0000 7fdef9600640 1 -- 172.21.15.131:0/3179077825 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fdef404b030 con 0x7fdefc0cf590 2024-05-26T22:55:36.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.890+0000 7fdf03e00640 1 -- 172.21.15.131:0/3179077825 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fdefc0cee90 con 0x7fdefc0cf590 2024-05-26T22:55:36.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.891+0000 7fdef9600640 1 -- 172.21.15.131:0/3179077825 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v95) ==== 104+0+11 (secure 0 0 0) 0x7fdef40ab070 con 0x7fdefc0cf590 2024-05-26T22:55:36.891 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-05-26T22:55:36.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.894+0000 7fdf03e00640 1 -- 172.21.15.131:0/3179077825 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fdedc06be80 msgr2=0x7fdedc06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:36.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.894+0000 7fdf03e00640 1 --2- 172.21.15.131:0/3179077825 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fdedc06be80 0x7fdedc06e340 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fdef0004ea0 tx=0x7fdef0009040 comp rx=0 tx=0).stop 2024-05-26T22:55:36.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.894+0000 7fdf03e00640 1 -- 172.21.15.131:0/3179077825 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdefc0cf590 msgr2=0x7fdefc14c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:36.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.894+0000 7fdf03e00640 1 --2- 172.21.15.131:0/3179077825 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdefc0cf590 0x7fdefc14c600 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7fdef4009e50 tx=0x7fdef4004a40 comp rx=0 tx=0).stop 2024-05-26T22:55:36.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.894+0000 7fdf03e00640 1 -- 172.21.15.131:0/3179077825 shutdown_connections 2024-05-26T22:55:36.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.894+0000 7fdf03e00640 1 --2- 172.21.15.131:0/3179077825 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdefc0b5450 0x7fdefc06e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:36.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.894+0000 7fdf03e00640 1 --2- 172.21.15.131:0/3179077825 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fdedc06be80 0x7fdedc06e340 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:36.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.894+0000 7fdf03e00640 1 --2- 172.21.15.131:0/3179077825 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdefc0cf590 0x7fdefc14c600 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:36.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.894+0000 7fdf03e00640 1 --2- 172.21.15.131:0/3179077825 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdefc06f420 0x7fdefc1f44e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:36.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.894+0000 7fdf03e00640 1 -- 172.21.15.131:0/3179077825 >> 172.21.15.131:0/3179077825 conn(0x7fdefc06bc90 msgr2=0x7fdefc15e990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:36.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.894+0000 7fdf03e00640 1 -- 172.21.15.131:0/3179077825 shutdown_connections 2024-05-26T22:55:36.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:36.894+0000 7fdf03e00640 1 -- 172.21.15.131:0/3179077825 wait complete. 2024-05-26T22:55:36.914 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-26T22:55:37.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.116+0000 7f6fa9400640 1 Processor -- start 2024-05-26T22:55:37.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.116+0000 7f6fa9400640 1 -- start start 2024-05-26T22:55:37.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.117+0000 7f6fa9400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6fa4145a10 0x7f6fa4072180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:37.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.117+0000 7f6fa9400640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f6fa40712d0 0x7f6fa406d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:37.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.118+0000 7f6fa9400640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6fa4072d60 con 0x7f6fa406d790 2024-05-26T22:55:37.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.118+0000 7f6fa9400640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f6fa415c760 con 0x7f6fa40712d0 2024-05-26T22:55:37.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.118+0000 7f6fa9400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f6fa4072a80 con 0x7f6fa4145a10 2024-05-26T22:55:37.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.118+0000 7f6fa8a00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f6fa406d790 0x7f6fa415fd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:42072/0 (socket says 172.21.15.131:42072) 2024-05-26T22:55:37.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.118+0000 7f6fa8a00640 1 -- 172.21.15.131:0/3111524343 learned_addr learned my addr 172.21.15.131:0/3111524343 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:55:37.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.118+0000 7f6fa3e00640 1 --2- 172.21.15.131:0/3111524343 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6fa4145a10 0x7f6fa4072180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:37.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.118+0000 7f6f9b400640 1 --2- 172.21.15.131:0/3111524343 >> v2:172.21.15.187:3300/0 conn(0x7f6fa40712d0 0x7f6fa406d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:37.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.118+0000 7f6fa3400640 1 -- 172.21.15.131:0/3111524343 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 538588000 0 0) 0x7f6fa4072d60 con 0x7f6fa406d790 2024-05-26T22:55:37.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.118+0000 7f6fa3400640 1 -- 172.21.15.131:0/3111524343 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6f88003520 con 0x7f6fa406d790 2024-05-26T22:55:37.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.119+0000 7f6fa3400640 1 -- 172.21.15.131:0/3111524343 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1585909205 0 0) 0x7f6f88003520 con 0x7f6fa406d790 2024-05-26T22:55:37.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.119+0000 7f6fa3400640 1 -- 172.21.15.131:0/3111524343 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6fa4145a10 msgr2=0x7f6fa4072180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:37.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.119+0000 7f6fa3400640 1 --2- 172.21.15.131:0/3111524343 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6fa4145a10 0x7f6fa4072180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:37.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.119+0000 7f6fa3400640 1 -- 172.21.15.131:0/3111524343 >> v2:172.21.15.187:3300/0 conn(0x7f6fa40712d0 msgr2=0x7f6fa406d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:37.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.119+0000 7f6fa3400640 1 --2- 172.21.15.131:0/3111524343 >> v2:172.21.15.187:3300/0 conn(0x7f6fa40712d0 0x7f6fa406d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:37.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.119+0000 7f6fa3400640 1 -- 172.21.15.131:0/3111524343 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fa40ceab0 con 0x7f6fa406d790 2024-05-26T22:55:37.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.119+0000 7f6fa3400640 1 -- 172.21.15.131:0/3111524343 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7f6f9c001670 con 0x7f6fa406d790 2024-05-26T22:55:37.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.119+0000 7f6fa3e00640 1 --2- 172.21.15.131:0/3111524343 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6fa4145a10 0x7f6fa4072180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-26T22:55:37.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.119+0000 7f6fa3400640 1 -- 172.21.15.131:0/3111524343 >> v1:172.21.15.131:6789/0 conn(0x7f6fa406d790 legacy=0x7f6fa415fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:37.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.119+0000 7f6fa3400640 1 --2- 172.21.15.131:0/3111524343 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6fa4145a10 0x7f6f880039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:37.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.119+0000 7f6fa3400640 1 --2- 172.21.15.131:0/3111524343 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6f88003ef0 0x7f6f88004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:37.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.119+0000 7f6fa3400640 1 --2- 172.21.15.131:0/3111524343 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6f88004d40 0x7f6f880051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:37.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.119+0000 7f6fa3400640 1 -- 172.21.15.131:0/3111524343 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa415c760 con 0x7f6f88004d40 2024-05-26T22:55:37.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.119+0000 7f6fa3400640 1 -- 172.21.15.131:0/3111524343 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa4072a80 con 0x7f6f88003ef0 2024-05-26T22:55:37.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.119+0000 7f6fa3400640 1 -- 172.21.15.131:0/3111524343 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f6f8800b9f0 con 0x7f6fa4145a10 2024-05-26T22:55:37.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.119+0000 7f6fa8a00640 1 --2- 172.21.15.131:0/3111524343 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6f88004d40 0x7f6f880051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:37.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.120+0000 7f6fa3e00640 1 --2- 172.21.15.131:0/3111524343 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6fa4145a10 0x7f6f880039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:37.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.120+0000 7f6f9b400640 1 --2- 172.21.15.131:0/3111524343 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6f88003ef0 0x7f6f88004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:37.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.120+0000 7f6fa8a00640 1 -- 172.21.15.131:0/3111524343 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6fa4145a10 msgr2=0x7f6f880039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:37.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.120+0000 7f6fa8a00640 1 --2- 172.21.15.131:0/3111524343 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6fa4145a10 0x7f6f880039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:37.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.120+0000 7f6fa8a00640 1 -- 172.21.15.131:0/3111524343 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6f88003ef0 msgr2=0x7f6f88004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:37.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.120+0000 7f6fa8a00640 1 --2- 172.21.15.131:0/3111524343 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6f88003ef0 0x7f6f88004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:37.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.120+0000 7f6fa8a00640 1 -- 172.21.15.131:0/3111524343 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f88003520 con 0x7f6f88004d40 2024-05-26T22:55:37.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.120+0000 7f6f9b400640 1 --2- 172.21.15.131:0/3111524343 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6f88003ef0 0x7f6f88004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:55:37.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.121+0000 7f6fa8a00640 1 --2- 172.21.15.131:0/3111524343 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6f88004d40 0x7f6f880051c0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c00c210 tx=0x7f6f88005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a638d8a7a34f0e9c server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:37.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.121+0000 7f6fa3400640 1 -- 172.21.15.131:0/3111524343 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f9c007d50 con 0x7f6f88004d40 2024-05-26T22:55:37.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.121+0000 7f6fa3400640 1 -- 172.21.15.131:0/3111524343 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f9c005af0 con 0x7f6f88004d40 2024-05-26T22:55:37.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.121+0000 7f6fa3400640 1 -- 172.21.15.131:0/3111524343 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f9c007070 con 0x7f6f88004d40 2024-05-26T22:55:37.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.122+0000 7f6fa9400640 1 -- 172.21.15.131:0/3111524343 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6f88004d40 msgr2=0x7f6f880051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:37.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.122+0000 7f6fa9400640 1 --2- 172.21.15.131:0/3111524343 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6f88004d40 0x7f6f880051c0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c00c210 tx=0x7f6f88005830 comp rx=0 tx=0).stop 2024-05-26T22:55:37.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.122+0000 7f6fa8a00640 1 -- 172.21.15.131:0/3111524343 reap_dead start 2024-05-26T22:55:37.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.122+0000 7f6fa9400640 1 -- 172.21.15.131:0/3111524343 shutdown_connections 2024-05-26T22:55:37.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.122+0000 7f6fa9400640 1 -- 172.21.15.131:0/3111524343 >> 172.21.15.131:0/3111524343 conn(0x7f6fa40b5660 msgr2=0x7f6fa40b5a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:37.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.122+0000 7f6fa9400640 1 -- 172.21.15.131:0/3111524343 shutdown_connections 2024-05-26T22:55:37.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.122+0000 7f6fa9400640 1 -- 172.21.15.131:0/3111524343 wait complete. 2024-05-26T22:55:37.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.124+0000 7f6fa9400640 1 Processor -- start 2024-05-26T22:55:37.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.124+0000 7f6fa9400640 1 -- start start 2024-05-26T22:55:37.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.125+0000 7f6fa9400640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6f88004d40 0x7f6fa415f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:37.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.125+0000 7f6fa9400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6fa40aa740 0x7f6fa40aabc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:37.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.125+0000 7f6fa9400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6fa40ac5e0 0x7f6fa40aca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:37.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.125+0000 7f6fa9400640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa4145df0 con 0x7f6fa40aa740 2024-05-26T22:55:37.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.125+0000 7f6fa9400640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa40cf070 con 0x7f6f88004d40 2024-05-26T22:55:37.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.125+0000 7f6fa9400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f6fa40d05a0 con 0x7f6fa40ac5e0 2024-05-26T22:55:37.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.125+0000 7f6f9b400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6fa40aa740 0x7f6fa40aabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:37.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.125+0000 7f6f9b400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6fa40aa740 0x7f6fa40aabc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:36386/0 (socket says 172.21.15.131:36386) 2024-05-26T22:55:37.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.125+0000 7f6f9b400640 1 -- 172.21.15.131:0/3003353679 learned_addr learned my addr 172.21.15.131:0/3003353679 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:55:37.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.125+0000 7f6fa3e00640 1 --2- 172.21.15.131:0/3003353679 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6f88004d40 0x7f6fa415f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:37.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.126+0000 7f6fa8a00640 1 --2- 172.21.15.131:0/3003353679 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6fa40ac5e0 0x7f6fa40aca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:37.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.126+0000 7f6f9b400640 1 -- 172.21.15.131:0/3003353679 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6fa40ac5e0 msgr2=0x7f6fa40aca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:37.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.126+0000 7f6f9b400640 1 --2- 172.21.15.131:0/3003353679 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6fa40ac5e0 0x7f6fa40aca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:37.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.126+0000 7f6f9b400640 1 -- 172.21.15.131:0/3003353679 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6f88004d40 msgr2=0x7f6fa415f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:37.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.126+0000 7f6f9b400640 1 --2- 172.21.15.131:0/3003353679 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6f88004d40 0x7f6fa415f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:37.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.126+0000 7f6f9b400640 1 -- 172.21.15.131:0/3003353679 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fa415be00 con 0x7f6fa40aa740 2024-05-26T22:55:37.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.126+0000 7f6fa3e00640 1 --2- 172.21.15.131:0/3003353679 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6f88004d40 0x7f6fa415f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:55:37.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.126+0000 7f6f9b400640 1 --2- 172.21.15.131:0/3003353679 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6fa40aa740 0x7f6fa40aabc0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c0109d0 tx=0x7f6f8c010e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:37.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.126+0000 7f6fa0c00640 1 -- 172.21.15.131:0/3003353679 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f8c00f040 con 0x7f6fa40aa740 2024-05-26T22:55:37.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.126+0000 7f6fa0c00640 1 -- 172.21.15.131:0/3003353679 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f8c00ec20 con 0x7f6fa40aa740 2024-05-26T22:55:37.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.126+0000 7f6fa9400640 1 -- 172.21.15.131:0/3003353679 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fa415e590 con 0x7f6fa40aa740 2024-05-26T22:55:37.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.126+0000 7f6fa0c00640 1 -- 172.21.15.131:0/3003353679 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f8c01c400 con 0x7f6fa40aa740 2024-05-26T22:55:37.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.126+0000 7f6fa9400640 1 -- 172.21.15.131:0/3003353679 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fa415eb50 con 0x7f6fa40aa740 2024-05-26T22:55:37.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.127+0000 7f6fa9400640 1 -- 172.21.15.131:0/3003353679 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fa40694e0 con 0x7f6fa40aa740 2024-05-26T22:55:37.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.128+0000 7f6fa0c00640 1 -- 172.21.15.131:0/3003353679 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6f8c01c5a0 con 0x7f6fa40aa740 2024-05-26T22:55:37.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.129+0000 7f6fa0c00640 1 --2- 172.21.15.131:0/3003353679 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f6f8006be60 0x7f6f8006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:37.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.129+0000 7f6fa0c00640 1 -- 172.21.15.131:0/3003353679 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 5709+0+0 (secure 0 0 0) 0x7f6f8c08b690 con 0x7f6fa40aa740 2024-05-26T22:55:37.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.130+0000 7f6fa3e00640 1 --2- 172.21.15.131:0/3003353679 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f6f8006be60 0x7f6f8006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:37.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.130+0000 7f6fa3e00640 1 --2- 172.21.15.131:0/3003353679 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f6f8006be60 0x7f6f8006e320 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f6f940098e0 tx=0x7f6f9400b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:37.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.134+0000 7f6fa0c00640 1 -- 172.21.15.131:0/3003353679 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f6f8c018030 con 0x7f6fa40aa740 2024-05-26T22:55:37.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.336+0000 7f6fa9400640 1 -- 172.21.15.131:0/3003353679 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f6fa4144a20 con 0x7f6fa40aa740 2024-05-26T22:55:37.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.337+0000 7f6fa0c00640 1 -- 172.21.15.131:0/3003353679 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v95) ==== 100+0+11 (secure 0 0 0) 0x7f6f8c059fa0 con 0x7f6fa40aa740 2024-05-26T22:55:37.336 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 64 2024-05-26T22:55:37.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.340+0000 7f6fa9400640 1 -- 172.21.15.131:0/3003353679 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f6f8006be60 msgr2=0x7f6f8006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:37.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.340+0000 7f6fa9400640 1 --2- 172.21.15.131:0/3003353679 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f6f8006be60 0x7f6f8006e320 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f6f940098e0 tx=0x7f6f9400b040 comp rx=0 tx=0).stop 2024-05-26T22:55:37.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.340+0000 7f6fa9400640 1 -- 172.21.15.131:0/3003353679 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6fa40aa740 msgr2=0x7f6fa40aabc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:37.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.340+0000 7f6fa9400640 1 --2- 172.21.15.131:0/3003353679 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6fa40aa740 0x7f6fa40aabc0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c0109d0 tx=0x7f6f8c010e90 comp rx=0 tx=0).stop 2024-05-26T22:55:37.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.340+0000 7f6fa9400640 1 -- 172.21.15.131:0/3003353679 shutdown_connections 2024-05-26T22:55:37.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.340+0000 7f6fa9400640 1 --2- 172.21.15.131:0/3003353679 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6fa40ac5e0 0x7f6fa40aca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:37.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.340+0000 7f6fa9400640 1 --2- 172.21.15.131:0/3003353679 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f6f8006be60 0x7f6f8006e320 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:37.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.340+0000 7f6fa9400640 1 --2- 172.21.15.131:0/3003353679 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6fa40aa740 0x7f6fa40aabc0 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:37.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.340+0000 7f6fa9400640 1 --2- 172.21.15.131:0/3003353679 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6f88004d40 0x7f6fa415f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:37.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.340+0000 7f6fa9400640 1 -- 172.21.15.131:0/3003353679 >> 172.21.15.131:0/3003353679 conn(0x7f6fa40b5660 msgr2=0x7f6fa40cd160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:37.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.340+0000 7f6fa9400640 1 -- 172.21.15.131:0/3003353679 shutdown_connections 2024-05-26T22:55:37.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.340+0000 7f6fa9400640 1 -- 172.21.15.131:0/3003353679 wait complete. 2024-05-26T22:55:37.359 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-26T22:55:37.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.564+0000 7f888ae00640 1 Processor -- start 2024-05-26T22:55:37.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.564+0000 7f888ae00640 1 -- start start 2024-05-26T22:55:37.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.565+0000 7f888ae00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f8884158c00 0x7f888406d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:37.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.566+0000 7f888ae00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f88840b38a0 0x7f888415fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:37.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.566+0000 7f888ae00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88840b3e50 con 0x7f888406daf0 2024-05-26T22:55:37.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.566+0000 7f888ae00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f888406df50 con 0x7f8884158c00 2024-05-26T22:55:37.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.566+0000 7f888ae00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f88840b4130 con 0x7f88840b38a0 2024-05-26T22:55:37.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.566+0000 7f8883e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f8884158c00 0x7f888406d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:37.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.566+0000 7f8883e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f8884158c00 0x7f888406d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:36402/0 (socket says 172.21.15.131:36402) 2024-05-26T22:55:37.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.566+0000 7f8883e00640 1 -- 172.21.15.131:0/644474278 learned_addr learned my addr 172.21.15.131:0/644474278 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:55:37.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.566+0000 7f8888e00640 1 --2- 172.21.15.131:0/644474278 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f88840b38a0 0x7f888415fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:37.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.567+0000 7f8883e00640 1 -- 172.21.15.131:0/644474278 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f88840b38a0 msgr2=0x7f888415fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:37.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.567+0000 7f8883e00640 1 --2- 172.21.15.131:0/644474278 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f88840b38a0 0x7f888415fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:37.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.567+0000 7f8883e00640 1 -- 172.21.15.131:0/644474278 >> v1:172.21.15.187:6789/0 conn(0x7f888406daf0 legacy=0x7f88840b3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:37.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.567+0000 7f8883e00640 1 -- 172.21.15.131:0/644474278 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88840b4410 con 0x7f8884158c00 2024-05-26T22:55:37.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.568+0000 7f8883e00640 1 --2- 172.21.15.131:0/644474278 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8884158c00 0x7f888406d5b0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f8874009870 tx=0x7f887402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=272016133a92c5ec server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:37.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.568+0000 7f8882a00640 1 -- 172.21.15.131:0/644474278 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8874037070 con 0x7f8884158c00 2024-05-26T22:55:37.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.568+0000 7f8882a00640 1 -- 172.21.15.131:0/644474278 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8874032440 con 0x7f8884158c00 2024-05-26T22:55:37.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.568+0000 7f8882a00640 1 -- 172.21.15.131:0/644474278 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8874032810 con 0x7f8884158c00 2024-05-26T22:55:37.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.569+0000 7f888ae00640 1 -- 172.21.15.131:0/644474278 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8884158c00 msgr2=0x7f888406d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:37.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.569+0000 7f888ae00640 1 --2- 172.21.15.131:0/644474278 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8884158c00 0x7f888406d5b0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f8874009870 tx=0x7f887402a180 comp rx=0 tx=0).stop 2024-05-26T22:55:37.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.569+0000 7f888ae00640 1 -- 172.21.15.131:0/644474278 shutdown_connections 2024-05-26T22:55:37.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.569+0000 7f888ae00640 1 --2- 172.21.15.131:0/644474278 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f88840b38a0 0x7f888415fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:37.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.569+0000 7f888ae00640 1 --2- 172.21.15.131:0/644474278 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8884158c00 0x7f888406d5b0 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:37.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.569+0000 7f888ae00640 1 -- 172.21.15.131:0/644474278 >> 172.21.15.131:0/644474278 conn(0x7f88840b1e90 msgr2=0x7f88840b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:37.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.569+0000 7f888ae00640 1 -- 172.21.15.131:0/644474278 shutdown_connections 2024-05-26T22:55:37.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.569+0000 7f888ae00640 1 -- 172.21.15.131:0/644474278 wait complete. 2024-05-26T22:55:37.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.571+0000 7f888ae00640 1 Processor -- start 2024-05-26T22:55:37.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.571+0000 7f888ae00640 1 -- start start 2024-05-26T22:55:37.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.573+0000 7f888ae00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f888406daf0 0x7f888415ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:37.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.573+0000 7f888ae00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f88840b38a0 0x7f88841f7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:37.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.573+0000 7f888ae00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8884158c00 0x7f88841f8010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:37.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.573+0000 7f888ae00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8884160190 con 0x7f888406daf0 2024-05-26T22:55:37.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.573+0000 7f888ae00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f88840b49d0 con 0x7f8884158c00 2024-05-26T22:55:37.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.573+0000 7f888ae00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f8884160310 con 0x7f88840b38a0 2024-05-26T22:55:37.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.573+0000 7f8883e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f888406daf0 0x7f888415ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:37.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.573+0000 7f8883e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f888406daf0 0x7f888415ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:36408/0 (socket says 172.21.15.131:36408) 2024-05-26T22:55:37.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.573+0000 7f8883e00640 1 -- 172.21.15.131:0/349917154 learned_addr learned my addr 172.21.15.131:0/349917154 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:55:37.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.573+0000 7f8883e00640 1 -- 172.21.15.131:0/349917154 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f88840b38a0 msgr2=0x7f88841f7ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:37.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.573+0000 7f8883400640 1 --2- 172.21.15.131:0/349917154 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f88840b38a0 0x7f88841f7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:37.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.573+0000 7f8888e00640 1 --2- 172.21.15.131:0/349917154 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8884158c00 0x7f88841f8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:37.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.573+0000 7f8883e00640 1 --2- 172.21.15.131:0/349917154 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f88840b38a0 0x7f88841f7ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:37.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.573+0000 7f8883e00640 1 -- 172.21.15.131:0/349917154 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8884158c00 msgr2=0x7f88841f8010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:37.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.573+0000 7f8883e00640 1 --2- 172.21.15.131:0/349917154 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8884158c00 0x7f88841f8010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:37.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.573+0000 7f8883e00640 1 -- 172.21.15.131:0/349917154 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88840ae4a0 con 0x7f888406daf0 2024-05-26T22:55:37.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.573+0000 7f8883400640 1 --2- 172.21.15.131:0/349917154 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f88840b38a0 0x7f88841f7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:55:37.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.574+0000 7f8883e00640 1 --2- 172.21.15.131:0/349917154 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f888406daf0 0x7f888415ad20 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f8874009870 tx=0x7f88740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:37.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.574+0000 7f887be00640 1 -- 172.21.15.131:0/349917154 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8874041020 con 0x7f888406daf0 2024-05-26T22:55:37.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.574+0000 7f887be00640 1 -- 172.21.15.131:0/349917154 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88740306c0 con 0x7f888406daf0 2024-05-26T22:55:37.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.574+0000 7f887be00640 1 -- 172.21.15.131:0/349917154 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8874037040 con 0x7f888406daf0 2024-05-26T22:55:37.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.574+0000 7f888ae00640 1 -- 172.21.15.131:0/349917154 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88840ae1c0 con 0x7f888406daf0 2024-05-26T22:55:37.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.574+0000 7f888ae00640 1 -- 172.21.15.131:0/349917154 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88840af460 con 0x7f888406daf0 2024-05-26T22:55:37.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.576+0000 7f888ae00640 1 -- 172.21.15.131:0/349917154 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88840694e0 con 0x7f888406daf0 2024-05-26T22:55:37.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.576+0000 7f887be00640 1 -- 172.21.15.131:0/349917154 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8874044020 con 0x7f888406daf0 2024-05-26T22:55:37.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.577+0000 7f887be00640 1 --2- 172.21.15.131:0/349917154 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f886406be30 0x7f886406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:37.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.577+0000 7f887be00640 1 -- 172.21.15.131:0/349917154 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 5709+0+0 (secure 0 0 0) 0x7f88740ab780 con 0x7f888406daf0 2024-05-26T22:55:37.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.578+0000 7f8883400640 1 --2- 172.21.15.131:0/349917154 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f886406be30 0x7f886406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:37.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.578+0000 7f8883400640 1 --2- 172.21.15.131:0/349917154 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f886406be30 0x7f886406e2f0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f888415df80 tx=0x7f8870009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:37.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.582+0000 7f887be00640 1 -- 172.21.15.131:0/349917154 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f887404b030 con 0x7f888406daf0 2024-05-26T22:55:37.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.823+0000 7f888ae00640 1 -- 172.21.15.131:0/349917154 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8884162b70 con 0x7f888406daf0 2024-05-26T22:55:37.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.824+0000 7f887be00640 1 -- 172.21.15.131:0/349917154 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) ==== 95+0+5232 (secure 0 0 0) 0x7f887407a090 con 0x7f888406daf0 2024-05-26T22:55:37.824 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:55:37.824 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":100,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5231,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/2037149126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6835","nonce":2037149126},{"type":"v1","addr":"172.21.15.131:6837","nonce":2037149126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":89}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-26T22:54:57.328457+0000","modified":"2024-05-26T22:55:27.458302+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7863,"mds_1":5246,"mds_2":5243},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5243":{"gid":5243,"name":"d","rank":2,"incarnation":97,"state":"up:active","state_seq":7,"addr":"172.21.15.187:6833/1314245822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1314245822},{"type":"v1","addr":"172.21.15.187:6833","nonce":1314245822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5246":{"gid":5246,"name":"b","rank":1,"incarnation":95,"state":"up:active","state_seq":6,"addr":"172.21.15.187:6835/521878375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":521878375},{"type":"v1","addr":"172.21.15.187:6835","nonce":521878375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7863":{"gid":7863,"name":"a","rank":0,"incarnation":90,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6836/789312503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":789312503},{"type":"v1","addr":"172.21.15.131:6836","nonce":789312503}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7863,"qdb_cluster":[5243,5246,7863]},"id":7}]} 2024-05-26T22:55:37.825 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 100 2024-05-26T22:55:37.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.827+0000 7f888ae00640 1 -- 172.21.15.131:0/349917154 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f886406be30 msgr2=0x7f886406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:37.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.827+0000 7f888ae00640 1 --2- 172.21.15.131:0/349917154 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f886406be30 0x7f886406e2f0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f888415df80 tx=0x7f8870009040 comp rx=0 tx=0).stop 2024-05-26T22:55:37.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.827+0000 7f888ae00640 1 -- 172.21.15.131:0/349917154 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f888406daf0 msgr2=0x7f888415ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:37.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.827+0000 7f888ae00640 1 --2- 172.21.15.131:0/349917154 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f888406daf0 0x7f888415ad20 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f8874009870 tx=0x7f88740046c0 comp rx=0 tx=0).stop 2024-05-26T22:55:37.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.827+0000 7f888ae00640 1 -- 172.21.15.131:0/349917154 shutdown_connections 2024-05-26T22:55:37.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.827+0000 7f888ae00640 1 --2- 172.21.15.131:0/349917154 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f88840b38a0 0x7f88841f7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:37.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.827+0000 7f888ae00640 1 --2- 172.21.15.131:0/349917154 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f886406be30 0x7f886406e2f0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:37.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.827+0000 7f888ae00640 1 --2- 172.21.15.131:0/349917154 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8884158c00 0x7f88841f8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:37.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.827+0000 7f888ae00640 1 --2- 172.21.15.131:0/349917154 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f888406daf0 0x7f888415ad20 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:37.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.827+0000 7f888ae00640 1 -- 172.21.15.131:0/349917154 >> 172.21.15.131:0/349917154 conn(0x7f88840b1e90 msgr2=0x7f88840b0730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:37.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.827+0000 7f888ae00640 1 -- 172.21.15.131:0/349917154 shutdown_connections 2024-05-26T22:55:37.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:37.827+0000 7f888ae00640 1 -- 172.21.15.131:0/349917154 wait complete. 2024-05-26T22:55:37.846 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-26T22:55:38.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.047+0000 7fafbac00640 1 Processor -- start 2024-05-26T22:55:38.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.047+0000 7fafbac00640 1 -- start start 2024-05-26T22:55:38.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.048+0000 7fafbac00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fafb4155840 0x7fafb4071930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:38.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.048+0000 7fafbac00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fafb415c3d0 0x7fafb406ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:38.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.048+0000 7fafbac00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fafb40722a0 con 0x7fafb406f490 2024-05-26T22:55:38.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.048+0000 7fafbac00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fafb4072580 con 0x7fafb4155840 2024-05-26T22:55:38.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.048+0000 7fafbac00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fafb4072860 con 0x7fafb415c3d0 2024-05-26T22:55:38.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.049+0000 7fafb8c00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fafb406f490 0x7fafb415ec80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:42080/0 (socket says 172.21.15.131:42080) 2024-05-26T22:55:38.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.049+0000 7fafb8c00640 1 -- 172.21.15.131:0/1414744653 learned_addr learned my addr 172.21.15.131:0/1414744653 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:55:38.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.049+0000 7fafb3e00640 1 --2- 172.21.15.131:0/1414744653 >> v2:172.21.15.187:3300/0 conn(0x7fafb4155840 0x7fafb4071930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:38.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.049+0000 7fafb3400640 1 --2- 172.21.15.131:0/1414744653 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fafb415c3d0 0x7fafb406ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:38.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.049+0000 7fafb2a00640 1 -- 172.21.15.131:0/1414744653 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3522658331 0 0) 0x7fafb40722a0 con 0x7fafb406f490 2024-05-26T22:55:38.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.049+0000 7fafb2a00640 1 -- 172.21.15.131:0/1414744653 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faf98003520 con 0x7fafb406f490 2024-05-26T22:55:38.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.049+0000 7fafb2a00640 1 -- 172.21.15.131:0/1414744653 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3541668079 0 0) 0x7faf98003520 con 0x7fafb406f490 2024-05-26T22:55:38.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.049+0000 7fafb2a00640 1 -- 172.21.15.131:0/1414744653 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fafb415c3d0 msgr2=0x7fafb406ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:38.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.049+0000 7fafb2a00640 1 --2- 172.21.15.131:0/1414744653 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fafb415c3d0 0x7fafb406ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:38.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.049+0000 7fafb2a00640 1 -- 172.21.15.131:0/1414744653 >> v2:172.21.15.187:3300/0 conn(0x7fafb4155840 msgr2=0x7fafb4071930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:38.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.049+0000 7fafb2a00640 1 --2- 172.21.15.131:0/1414744653 >> v2:172.21.15.187:3300/0 conn(0x7fafb4155840 0x7fafb4071930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:38.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.050+0000 7fafb2a00640 1 -- 172.21.15.131:0/1414744653 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafb4072b40 con 0x7fafb406f490 2024-05-26T22:55:38.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.050+0000 7fafb2a00640 1 -- 172.21.15.131:0/1414744653 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7fafac0019f0 con 0x7fafb406f490 2024-05-26T22:55:38.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.050+0000 7fafb2a00640 1 -- 172.21.15.131:0/1414744653 >> v1:172.21.15.131:6789/0 conn(0x7fafb406f490 legacy=0x7fafb415ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:38.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.050+0000 7fafb2a00640 1 --2- 172.21.15.131:0/1414744653 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf980039b0 0x7faf98003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:38.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.050+0000 7fafb2a00640 1 --2- 172.21.15.131:0/1414744653 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fafb415c3d0 0x7faf98004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:38.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.050+0000 7fafb2a00640 1 --2- 172.21.15.131:0/1414744653 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faf98004d40 0x7faf980051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:38.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.050+0000 7fafb2a00640 1 -- 172.21.15.131:0/1414744653 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fafb4072580 con 0x7faf980039b0 2024-05-26T22:55:38.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.050+0000 7fafb2a00640 1 -- 172.21.15.131:0/1414744653 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fafb4072860 con 0x7faf98004d40 2024-05-26T22:55:38.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.050+0000 7fafb2a00640 1 -- 172.21.15.131:0/1414744653 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7faf9800b9c0 con 0x7fafb415c3d0 2024-05-26T22:55:38.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.050+0000 7fafb3e00640 1 --2- 172.21.15.131:0/1414744653 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf980039b0 0x7faf98003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:38.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.050+0000 7fafb3e00640 1 -- 172.21.15.131:0/1414744653 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fafb415c3d0 msgr2=0x7faf98004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:38.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.050+0000 7fafb3e00640 1 --2- 172.21.15.131:0/1414744653 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fafb415c3d0 0x7faf98004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:38.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.050+0000 7fafb3e00640 1 -- 172.21.15.131:0/1414744653 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faf98004d40 msgr2=0x7faf980051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:38.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.050+0000 7fafb3e00640 1 --2- 172.21.15.131:0/1414744653 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faf98004d40 0x7faf980051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:38.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.050+0000 7fafb3e00640 1 -- 172.21.15.131:0/1414744653 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf98003520 con 0x7faf980039b0 2024-05-26T22:55:38.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.051+0000 7fafb3e00640 1 --2- 172.21.15.131:0/1414744653 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf980039b0 0x7faf98003dd0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7fafa4009ac0 tx=0x7faf98005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1ccfcb14a92a361 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:38.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.052+0000 7fafb2a00640 1 -- 172.21.15.131:0/1414744653 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa403a070 con 0x7faf980039b0 2024-05-26T22:55:38.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.052+0000 7fafb2a00640 1 -- 172.21.15.131:0/1414744653 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafa402d720 con 0x7faf980039b0 2024-05-26T22:55:38.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.052+0000 7fafb2a00640 1 -- 172.21.15.131:0/1414744653 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa4009690 con 0x7faf980039b0 2024-05-26T22:55:38.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.052+0000 7fafbac00640 1 -- 172.21.15.131:0/1414744653 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf980039b0 msgr2=0x7faf98003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:38.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.052+0000 7fafbac00640 1 --2- 172.21.15.131:0/1414744653 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf980039b0 0x7faf98003dd0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7fafa4009ac0 tx=0x7faf98005830 comp rx=0 tx=0).stop 2024-05-26T22:55:38.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.052+0000 7fafb8c00640 1 -- 172.21.15.131:0/1414744653 reap_dead start 2024-05-26T22:55:38.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.052+0000 7fafbac00640 1 -- 172.21.15.131:0/1414744653 shutdown_connections 2024-05-26T22:55:38.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.052+0000 7fafbac00640 1 -- 172.21.15.131:0/1414744653 >> 172.21.15.131:0/1414744653 conn(0x7fafb40b1e90 msgr2=0x7fafb40b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:38.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.052+0000 7fafbac00640 1 -- 172.21.15.131:0/1414744653 shutdown_connections 2024-05-26T22:55:38.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.052+0000 7fafbac00640 1 -- 172.21.15.131:0/1414744653 wait complete. 2024-05-26T22:55:38.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.054+0000 7fafbac00640 1 Processor -- start 2024-05-26T22:55:38.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.054+0000 7fafbac00640 1 -- start start 2024-05-26T22:55:38.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.055+0000 7fafbac00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faf980039b0 0x7fafb41592b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:38.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.055+0000 7fafbac00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fafb4155c20 0x7fafb4156080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:38.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.055+0000 7fafbac00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fafb41580d0 0x7fafb4158550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:38.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.055+0000 7fafbac00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fafb4071ff0 con 0x7fafb41580d0 2024-05-26T22:55:38.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.055+0000 7fafbac00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fafb4071e70 con 0x7fafb4155c20 2024-05-26T22:55:38.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.055+0000 7fafbac00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fafb40aee50 con 0x7faf980039b0 2024-05-26T22:55:38.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.055+0000 7fafb8c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fafb41580d0 0x7fafb4158550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:38.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.056+0000 7fafb8c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fafb41580d0 0x7fafb4158550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:36424/0 (socket says 172.21.15.131:36424) 2024-05-26T22:55:38.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.056+0000 7fafb8c00640 1 -- 172.21.15.131:0/2286056963 learned_addr learned my addr 172.21.15.131:0/2286056963 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:55:38.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.056+0000 7fafb8c00640 1 -- 172.21.15.131:0/2286056963 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faf980039b0 msgr2=0x7fafb41592b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:38.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.056+0000 7fafb8c00640 1 --2- 172.21.15.131:0/2286056963 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faf980039b0 0x7fafb41592b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:38.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.056+0000 7fafb8c00640 1 -- 172.21.15.131:0/2286056963 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fafb4155c20 msgr2=0x7fafb4156080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:38.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.056+0000 7fafb8c00640 1 --2- 172.21.15.131:0/2286056963 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fafb4155c20 0x7fafb4156080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:38.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.056+0000 7fafb8c00640 1 -- 172.21.15.131:0/2286056963 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafb4159ae0 con 0x7fafb41580d0 2024-05-26T22:55:38.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.056+0000 7fafb8c00640 1 --2- 172.21.15.131:0/2286056963 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fafb41580d0 0x7fafb4158550 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7fafac0012e0 tx=0x7fafac00cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:38.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.056+0000 7fafabe00640 1 -- 172.21.15.131:0/2286056963 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafac00a7b0 con 0x7fafb41580d0 2024-05-26T22:55:38.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.057+0000 7fafabe00640 1 -- 172.21.15.131:0/2286056963 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafac00a950 con 0x7fafb41580d0 2024-05-26T22:55:38.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.057+0000 7fafabe00640 1 -- 172.21.15.131:0/2286056963 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafac00ad40 con 0x7fafb41580d0 2024-05-26T22:55:38.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.057+0000 7fafbac00640 1 -- 172.21.15.131:0/2286056963 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafb4159dc0 con 0x7fafb41580d0 2024-05-26T22:55:38.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.057+0000 7fafbac00640 1 -- 172.21.15.131:0/2286056963 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafb415a380 con 0x7fafb41580d0 2024-05-26T22:55:38.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.058+0000 7fafbac00640 1 -- 172.21.15.131:0/2286056963 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafb40694e0 con 0x7fafb41580d0 2024-05-26T22:55:38.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.059+0000 7fafabe00640 1 -- 172.21.15.131:0/2286056963 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fafac007070 con 0x7fafb41580d0 2024-05-26T22:55:38.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.060+0000 7fafabe00640 1 --2- 172.21.15.131:0/2286056963 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7faf9406be60 0x7faf9406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:38.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.060+0000 7fafabe00640 1 -- 172.21.15.131:0/2286056963 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 5709+0+0 (secure 0 0 0) 0x7fafac08a620 con 0x7fafb41580d0 2024-05-26T22:55:38.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.060+0000 7fafb3e00640 1 --2- 172.21.15.131:0/2286056963 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7faf9406be60 0x7faf9406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:38.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.061+0000 7fafb3e00640 1 --2- 172.21.15.131:0/2286056963 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7faf9406be60 0x7faf9406e320 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fafa4037000 tx=0x7fafa402f000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:38.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.064+0000 7fafabe00640 1 -- 172.21.15.131:0/2286056963 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fafac01a030 con 0x7fafb41580d0 2024-05-26T22:55:38.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.264+0000 7fafbac00640 1 -- 172.21.15.131:0/2286056963 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fafb40b42e0 con 0x7fafb41580d0 2024-05-26T22:55:38.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.265+0000 7fafabe00640 1 -- 172.21.15.131:0/2286056963 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) ==== 74+0+16632 (secure 0 0 0) 0x7fafac01b2b0 con 0x7fafb41580d0 2024-05-26T22:55:38.266 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:55:38.266 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":95,"fsid":"46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f","created":"2024-05-26T22:47:57.288905+0000","modified":"2024-05-26T22:54:58.901569+0000","last_up_change":"2024-05-26T22:48:09.602207+0000","last_in_change":"2024-05-26T22:48:00.687341+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-26T22:48:10.463339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-05-26T22:54:55.120321+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-05-26T22:54:56.346430+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"814eaf62-a18d-494b-8e27-fd8fb88adea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6801","nonce":2350940051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6803","nonce":2350940051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6807","nonce":2350940051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6805","nonce":2350940051}]},"public_addr":"172.21.15.131:6801/2350940051","cluster_addr":"172.21.15.131:6803/2350940051","heartbeat_back_addr":"172.21.15.131:6807/2350940051","heartbeat_front_addr":"172.21.15.131:6805/2350940051","state":["exists","up"]},{"osd":1,"uuid":"42e66287-f6c6-4e26-af3c-6dfaa3536d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6825","nonce":1525528247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6827","nonce":1525528247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6831","nonce":1525528247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6829","nonce":1525528247}]},"public_addr":"172.21.15.131:6825/1525528247","cluster_addr":"172.21.15.131:6827/1525528247","heartbeat_back_addr":"172.21.15.131:6831/1525528247","heartbeat_front_addr":"172.21.15.131:6829/1525528247","state":["exists","up"]},{"osd":2,"uuid":"0da34673-04a2-4fd0-b57a-0ff2b679b043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6817","nonce":3627842210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6819","nonce":3627842210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6823","nonce":3627842210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6821","nonce":3627842210}]},"public_addr":"172.21.15.131:6817/3627842210","cluster_addr":"172.21.15.131:6819/3627842210","heartbeat_back_addr":"172.21.15.131:6823/3627842210","heartbeat_front_addr":"172.21.15.131:6821/3627842210","state":["exists","up"]},{"osd":3,"uuid":"cf117a27-d6b2-43d3-ae52-b47988bf8eed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6809","nonce":350790591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6811","nonce":350790591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6815","nonce":350790591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6813","nonce":350790591}]},"public_addr":"172.21.15.131:6809/350790591","cluster_addr":"172.21.15.131:6811/350790591","heartbeat_back_addr":"172.21.15.131:6815/350790591","heartbeat_front_addr":"172.21.15.131:6813/350790591","state":["exists","up"]},{"osd":4,"uuid":"a092ccfc-be89-4a84-88d5-0f781fa715eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6816","nonce":244560995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6819","nonce":244560995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6823","nonce":244560995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6821","nonce":244560995}]},"public_addr":"172.21.15.187:6816/244560995","cluster_addr":"172.21.15.187:6819/244560995","heartbeat_back_addr":"172.21.15.187:6823/244560995","heartbeat_front_addr":"172.21.15.187:6821/244560995","state":["exists","up"]},{"osd":5,"uuid":"7513ba69-dd00-4a35-b190-9e8cc3b2bdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6801","nonce":324619398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6803","nonce":324619398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6812","nonce":324619398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6807","nonce":324619398}]},"public_addr":"172.21.15.187:6801/324619398","cluster_addr":"172.21.15.187:6803/324619398","heartbeat_back_addr":"172.21.15.187:6812/324619398","heartbeat_front_addr":"172.21.15.187:6807/324619398","state":["exists","up"]},{"osd":6,"uuid":"50081810-c093-4d2f-a4d7-b7c695c5400d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6806","nonce":1391347786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6809","nonce":1391347786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6818","nonce":1391347786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6813","nonce":1391347786}]},"public_addr":"172.21.15.187:6806/1391347786","cluster_addr":"172.21.15.187:6809/1391347786","heartbeat_back_addr":"172.21.15.187:6818/1391347786","heartbeat_front_addr":"172.21.15.187:6813/1391347786","state":["exists","up"]},{"osd":7,"uuid":"775dae05-54f5-4ad4-bbce-92863c6ed2ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6825","nonce":1220065622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6827","nonce":1220065622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6831","nonce":1220065622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6829","nonce":1220065622}]},"public_addr":"172.21.15.187:6825/1220065622","cluster_addr":"172.21.15.187:6827/1220065622","heartbeat_back_addr":"172.21.15.187:6831/1220065622","heartbeat_front_addr":"172.21.15.187:6829/1220065622","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:05.983520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.299516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.067323+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.307748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.018742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.258775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.010912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.295355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-26T22:55:38.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.268+0000 7fafbac00640 1 -- 172.21.15.131:0/2286056963 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7faf9406be60 msgr2=0x7faf9406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:38.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.268+0000 7fafbac00640 1 --2- 172.21.15.131:0/2286056963 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7faf9406be60 0x7faf9406e320 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fafa4037000 tx=0x7fafa402f000 comp rx=0 tx=0).stop 2024-05-26T22:55:38.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.268+0000 7fafbac00640 1 -- 172.21.15.131:0/2286056963 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fafb41580d0 msgr2=0x7fafb4158550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:38.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.268+0000 7fafbac00640 1 --2- 172.21.15.131:0/2286056963 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fafb41580d0 0x7fafb4158550 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7fafac0012e0 tx=0x7fafac00cf00 comp rx=0 tx=0).stop 2024-05-26T22:55:38.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.269+0000 7fafbac00640 1 -- 172.21.15.131:0/2286056963 shutdown_connections 2024-05-26T22:55:38.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.269+0000 7fafbac00640 1 --2- 172.21.15.131:0/2286056963 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7faf9406be60 0x7faf9406e320 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:38.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.269+0000 7fafbac00640 1 --2- 172.21.15.131:0/2286056963 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fafb41580d0 0x7fafb4158550 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:38.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.269+0000 7fafbac00640 1 --2- 172.21.15.131:0/2286056963 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fafb4155c20 0x7fafb4156080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:38.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.269+0000 7fafbac00640 1 --2- 172.21.15.131:0/2286056963 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7faf980039b0 0x7fafb41592b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:38.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.269+0000 7fafbac00640 1 -- 172.21.15.131:0/2286056963 >> 172.21.15.131:0/2286056963 conn(0x7fafb40b1e90 msgr2=0x7fafb4154900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:38.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.269+0000 7fafbac00640 1 -- 172.21.15.131:0/2286056963 shutdown_connections 2024-05-26T22:55:38.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.269+0000 7fafbac00640 1 -- 172.21.15.131:0/2286056963 wait complete. 2024-05-26T22:55:38.287 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-26T22:55:38.288 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-26T22:55:38.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.483+0000 7fd1ca200640 1 Processor -- start 2024-05-26T22:55:38.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.484+0000 7fd1ca200640 1 -- start start 2024-05-26T22:55:38.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.485+0000 7fd1ca200640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fd1c4144340 0x7fd1c40cedf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:38.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.485+0000 7fd1ca200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fd1c4071b70 0x7fd1c406e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:38.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.485+0000 7fd1ca200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd1c406e9f0 0x7fd1c415ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:38.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.485+0000 7fd1ca200640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fd1c406ee60 con 0x7fd1c4071b70 2024-05-26T22:55:38.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.485+0000 7fd1ca200640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fd1c4072eb0 con 0x7fd1c4144340 2024-05-26T22:55:38.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.485+0000 7fd1ca200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd1c4072000 con 0x7fd1c406e9f0 2024-05-26T22:55:38.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.485+0000 7fd1bbe00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fd1c4071b70 0x7fd1c406e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:38.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.485+0000 7fd1bbe00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fd1c4071b70 0x7fd1c406e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:36438/0 (socket says 172.21.15.131:36438) 2024-05-26T22:55:38.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.485+0000 7fd1bbe00640 1 -- 172.21.15.131:0/913469176 learned_addr learned my addr 172.21.15.131:0/913469176 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:55:38.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.486+0000 7fd1c8e00640 1 --2- 172.21.15.131:0/913469176 >> v2:172.21.15.187:3300/0 conn(0x7fd1c4144340 0x7fd1c40cedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:38.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.486+0000 7fd1bbe00640 1 -- 172.21.15.131:0/913469176 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd1c406e9f0 msgr2=0x7fd1c415ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:38.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.486+0000 7fd1bbe00640 1 --2- 172.21.15.131:0/913469176 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd1c406e9f0 0x7fd1c415ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:38.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.486+0000 7fd1bbe00640 1 -- 172.21.15.131:0/913469176 >> v2:172.21.15.187:3300/0 conn(0x7fd1c4144340 msgr2=0x7fd1c40cedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:38.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.486+0000 7fd1bbe00640 1 --2- 172.21.15.131:0/913469176 >> v2:172.21.15.187:3300/0 conn(0x7fd1c4144340 0x7fd1c40cedf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:38.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.486+0000 7fd1bbe00640 1 -- 172.21.15.131:0/913469176 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1c40722a0 con 0x7fd1c4071b70 2024-05-26T22:55:38.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.486+0000 7fd1c8e00640 1 --2- 172.21.15.131:0/913469176 >> v2:172.21.15.187:3300/0 conn(0x7fd1c4144340 0x7fd1c40cedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:55:38.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.487+0000 7fd1bbe00640 1 --2- 172.21.15.131:0/913469176 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd1c4071b70 0x7fd1c406e4b0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7fd1c40b2bd0 tx=0x7fd1ac02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e98f7a366f06f8f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:38.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.487+0000 7fd1c3e00640 1 -- 172.21.15.131:0/913469176 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1ac037070 con 0x7fd1c4071b70 2024-05-26T22:55:38.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.487+0000 7fd1c3e00640 1 -- 172.21.15.131:0/913469176 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1ac032440 con 0x7fd1c4071b70 2024-05-26T22:55:38.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.487+0000 7fd1c3e00640 1 -- 172.21.15.131:0/913469176 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1ac032810 con 0x7fd1c4071b70 2024-05-26T22:55:38.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.488+0000 7fd1ca200640 1 -- 172.21.15.131:0/913469176 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd1c4071b70 msgr2=0x7fd1c406e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:38.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.488+0000 7fd1ca200640 1 --2- 172.21.15.131:0/913469176 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd1c4071b70 0x7fd1c406e4b0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7fd1c40b2bd0 tx=0x7fd1ac02a1e0 comp rx=0 tx=0).stop 2024-05-26T22:55:38.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.488+0000 7fd1ca200640 1 -- 172.21.15.131:0/913469176 shutdown_connections 2024-05-26T22:55:38.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.488+0000 7fd1ca200640 1 --2- 172.21.15.131:0/913469176 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd1c406e9f0 0x7fd1c415ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:38.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.488+0000 7fd1ca200640 1 --2- 172.21.15.131:0/913469176 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd1c4071b70 0x7fd1c406e4b0 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:38.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.488+0000 7fd1ca200640 1 --2- 172.21.15.131:0/913469176 >> v2:172.21.15.187:3300/0 conn(0x7fd1c4144340 0x7fd1c40cedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:38.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.488+0000 7fd1ca200640 1 -- 172.21.15.131:0/913469176 >> 172.21.15.131:0/913469176 conn(0x7fd1c40cff40 msgr2=0x7fd1c40d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:38.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.488+0000 7fd1ca200640 1 -- 172.21.15.131:0/913469176 shutdown_connections 2024-05-26T22:55:38.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.488+0000 7fd1ca200640 1 -- 172.21.15.131:0/913469176 wait complete. 2024-05-26T22:55:38.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.489+0000 7fd1ca200640 1 Processor -- start 2024-05-26T22:55:38.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.490+0000 7fd1ca200640 1 -- start start 2024-05-26T22:55:38.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.490+0000 7fd1ca200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd1c406e9f0 0x7fd1c413cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:38.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.491+0000 7fd1ca200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd1c4071b70 0x7fd1c41e4a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:38.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.491+0000 7fd1ca200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd1c4144340 0x7fd1c41e4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:38.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.491+0000 7fd1ca200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c4072b40 con 0x7fd1c4071b70 2024-05-26T22:55:38.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.491+0000 7fd1ca200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c4072860 con 0x7fd1c4144340 2024-05-26T22:55:38.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.491+0000 7fd1ca200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd1c415f170 con 0x7fd1c406e9f0 2024-05-26T22:55:38.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.491+0000 7fd1bbe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd1c4071b70 0x7fd1c41e4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:38.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.491+0000 7fd1bbe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd1c4071b70 0x7fd1c41e4a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:36446/0 (socket says 172.21.15.131:36446) 2024-05-26T22:55:38.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.491+0000 7fd1bbe00640 1 -- 172.21.15.131:0/1149979370 learned_addr learned my addr 172.21.15.131:0/1149979370 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:55:38.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.491+0000 7fd1bbe00640 1 -- 172.21.15.131:0/1149979370 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd1c406e9f0 msgr2=0x7fd1c413cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:38.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.491+0000 7fd1bbe00640 1 --2- 172.21.15.131:0/1149979370 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd1c406e9f0 0x7fd1c413cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:38.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.491+0000 7fd1bbe00640 1 -- 172.21.15.131:0/1149979370 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd1c4144340 msgr2=0x7fd1c41e4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:38.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.491+0000 7fd1bbe00640 1 --2- 172.21.15.131:0/1149979370 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd1c4144340 0x7fd1c41e4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:38.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.491+0000 7fd1bbe00640 1 -- 172.21.15.131:0/1149979370 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1c413d070 con 0x7fd1c4071b70 2024-05-26T22:55:38.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.491+0000 7fd1bbe00640 1 --2- 172.21.15.131:0/1149979370 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd1c4071b70 0x7fd1c41e4a70 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac02a220 tx=0x7fd1ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:38.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.492+0000 7fd1c1600640 1 -- 172.21.15.131:0/1149979370 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1ac037040 con 0x7fd1c4071b70 2024-05-26T22:55:38.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.492+0000 7fd1c1600640 1 -- 172.21.15.131:0/1149979370 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1ac042070 con 0x7fd1c4071b70 2024-05-26T22:55:38.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.492+0000 7fd1c1600640 1 -- 172.21.15.131:0/1149979370 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1ac030bb0 con 0x7fd1c4071b70 2024-05-26T22:55:38.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.492+0000 7fd1ca200640 1 -- 172.21.15.131:0/1149979370 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1c413d320 con 0x7fd1c4071b70 2024-05-26T22:55:38.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.492+0000 7fd1ca200640 1 -- 172.21.15.131:0/1149979370 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1c413d8e0 con 0x7fd1c4071b70 2024-05-26T22:55:38.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.493+0000 7fd1ca200640 1 -- 172.21.15.131:0/1149979370 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1c40694e0 con 0x7fd1c4071b70 2024-05-26T22:55:38.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.494+0000 7fd1c1600640 1 -- 172.21.15.131:0/1149979370 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd1ac047460 con 0x7fd1c4071b70 2024-05-26T22:55:38.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.495+0000 7fd1c1600640 1 --2- 172.21.15.131:0/1149979370 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fd1a006be30 0x7fd1a006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:38.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.495+0000 7fd1c1600640 1 -- 172.21.15.131:0/1149979370 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 5709+0+0 (secure 0 0 0) 0x7fd1ac0aba40 con 0x7fd1c4071b70 2024-05-26T22:55:38.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.495+0000 7fd1c8e00640 1 --2- 172.21.15.131:0/1149979370 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fd1a006be30 0x7fd1a006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:38.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.496+0000 7fd1c8e00640 1 --2- 172.21.15.131:0/1149979370 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fd1a006be30 0x7fd1a006e2f0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fd1b4006fd0 tx=0x7fd1b4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:38.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.500+0000 7fd1c1600640 1 -- 172.21.15.131:0/1149979370 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fd1ac045360 con 0x7fd1c4071b70 2024-05-26T22:55:38.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.719+0000 7fd1ca200640 1 -- 172.21.15.131:0/1149979370 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd1c40d07c0 con 0x7fd1c4071b70 2024-05-26T22:55:38.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.721+0000 7fd1c1600640 1 -- 172.21.15.131:0/1149979370 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) ==== 95+0+5232 (secure 0 0 0) 0x7fd1ac0abe20 con 0x7fd1c4071b70 2024-05-26T22:55:38.721 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:55:38.721 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":100,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5231,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/2037149126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6835","nonce":2037149126},{"type":"v1","addr":"172.21.15.131:6837","nonce":2037149126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":89}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-26T22:54:57.328457+0000","modified":"2024-05-26T22:55:27.458302+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7863,"mds_1":5246,"mds_2":5243},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5243":{"gid":5243,"name":"d","rank":2,"incarnation":97,"state":"up:active","state_seq":7,"addr":"172.21.15.187:6833/1314245822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1314245822},{"type":"v1","addr":"172.21.15.187:6833","nonce":1314245822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5246":{"gid":5246,"name":"b","rank":1,"incarnation":95,"state":"up:active","state_seq":6,"addr":"172.21.15.187:6835/521878375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":521878375},{"type":"v1","addr":"172.21.15.187:6835","nonce":521878375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7863":{"gid":7863,"name":"a","rank":0,"incarnation":90,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6836/789312503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":789312503},{"type":"v1","addr":"172.21.15.131:6836","nonce":789312503}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7863,"qdb_cluster":[5243,5246,7863]},"id":7}]} 2024-05-26T22:55:38.721 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 100 2024-05-26T22:55:38.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.724+0000 7fd1ca200640 1 -- 172.21.15.131:0/1149979370 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fd1a006be30 msgr2=0x7fd1a006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:38.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.724+0000 7fd1ca200640 1 --2- 172.21.15.131:0/1149979370 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fd1a006be30 0x7fd1a006e2f0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fd1b4006fd0 tx=0x7fd1b4008040 comp rx=0 tx=0).stop 2024-05-26T22:55:38.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.724+0000 7fd1ca200640 1 -- 172.21.15.131:0/1149979370 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd1c4071b70 msgr2=0x7fd1c41e4a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:38.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.724+0000 7fd1ca200640 1 --2- 172.21.15.131:0/1149979370 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd1c4071b70 0x7fd1c41e4a70 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac02a220 tx=0x7fd1ac0046c0 comp rx=0 tx=0).stop 2024-05-26T22:55:38.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.724+0000 7fd1ca200640 1 -- 172.21.15.131:0/1149979370 shutdown_connections 2024-05-26T22:55:38.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.724+0000 7fd1ca200640 1 --2- 172.21.15.131:0/1149979370 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fd1a006be30 0x7fd1a006e2f0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:38.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.724+0000 7fd1ca200640 1 --2- 172.21.15.131:0/1149979370 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd1c4144340 0x7fd1c41e4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:38.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.724+0000 7fd1ca200640 1 --2- 172.21.15.131:0/1149979370 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd1c4071b70 0x7fd1c41e4a70 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:38.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.724+0000 7fd1ca200640 1 --2- 172.21.15.131:0/1149979370 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd1c406e9f0 0x7fd1c413cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:38.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.724+0000 7fd1ca200640 1 -- 172.21.15.131:0/1149979370 >> 172.21.15.131:0/1149979370 conn(0x7fd1c40cff40 msgr2=0x7fd1c40b1af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:38.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.724+0000 7fd1ca200640 1 -- 172.21.15.131:0/1149979370 shutdown_connections 2024-05-26T22:55:38.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.724+0000 7fd1ca200640 1 -- 172.21.15.131:0/1149979370 wait complete. 2024-05-26T22:55:38.742 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-26T22:55:38.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.933+0000 7f3bbfe00640 1 Processor -- start 2024-05-26T22:55:38.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.933+0000 7f3bbfe00640 1 -- start start 2024-05-26T22:55:38.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.934+0000 7f3bbfe00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3bc006ebc0 0x7f3bc00cedf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:38.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.934+0000 7f3bbfe00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f3bc0071b70 0x7f3bc00b3d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:38.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.934+0000 7f3bbfe00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3bc00722a0 con 0x7f3bc0144340 2024-05-26T22:55:38.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.934+0000 7f3bbfe00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f3bc0072eb0 con 0x7f3bc0071b70 2024-05-26T22:55:38.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.934+0000 7f3bbfe00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3bc0072580 con 0x7f3bc006ebc0 2024-05-26T22:55:38.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.935+0000 7f3bb6000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3bc006ebc0 0x7f3bc00cedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:38.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.935+0000 7f3bbf400640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f3bc0071b70 0x7f3bc00b3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:38.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.935+0000 7f3bb6000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3bc006ebc0 0x7f3bc00cedf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.131:37128/0 (socket says 172.21.15.131:37128) 2024-05-26T22:55:38.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.935+0000 7f3bbf400640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f3bc0071b70 0x7f3bc00b3d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.131:54340/0 (socket says 172.21.15.131:54340) 2024-05-26T22:55:38.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.935+0000 7f3bb6000640 1 -- 172.21.15.131:0/3187379064 learned_addr learned my addr 172.21.15.131:0/3187379064 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:55:38.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.936+0000 7f3bbf400640 1 -- 172.21.15.131:0/3187379064 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3bc006ebc0 msgr2=0x7f3bc00cedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:38.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.936+0000 7f3bbf400640 1 --2- 172.21.15.131:0/3187379064 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3bc006ebc0 0x7f3bc00cedf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:38.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.936+0000 7f3bbf400640 1 -- 172.21.15.131:0/3187379064 >> v1:172.21.15.187:6789/0 conn(0x7f3bc0144340 legacy=0x7f3bc006e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:38.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.936+0000 7f3bbf400640 1 -- 172.21.15.131:0/3187379064 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bc0072860 con 0x7f3bc0071b70 2024-05-26T22:55:38.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.937+0000 7f3bbf400640 1 --2- 172.21.15.131:0/3187379064 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3bc0071b70 0x7f3bc00b3d20 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f3bb80097a0 tx=0x7f3bb802a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db1781d3dc3e7481 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:38.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.938+0000 7f3bbe000640 1 -- 172.21.15.131:0/3187379064 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bb8037070 con 0x7f3bc0071b70 2024-05-26T22:55:38.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.938+0000 7f3bbe000640 1 -- 172.21.15.131:0/3187379064 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3bb8032440 con 0x7f3bc0071b70 2024-05-26T22:55:38.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.938+0000 7f3bbe000640 1 -- 172.21.15.131:0/3187379064 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bb8032810 con 0x7f3bc0071b70 2024-05-26T22:55:38.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.938+0000 7f3bbfe00640 1 -- 172.21.15.131:0/3187379064 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3bc0071b70 msgr2=0x7f3bc00b3d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:38.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.938+0000 7f3bbfe00640 1 --2- 172.21.15.131:0/3187379064 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3bc0071b70 0x7f3bc00b3d20 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f3bb80097a0 tx=0x7f3bb802a180 comp rx=0 tx=0).stop 2024-05-26T22:55:38.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.939+0000 7f3bbfe00640 1 -- 172.21.15.131:0/3187379064 shutdown_connections 2024-05-26T22:55:38.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.939+0000 7f3bbfe00640 1 --2- 172.21.15.131:0/3187379064 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3bc0071b70 0x7f3bc00b3d20 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:38.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.939+0000 7f3bbfe00640 1 --2- 172.21.15.131:0/3187379064 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3bc006ebc0 0x7f3bc00cedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:38.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.939+0000 7f3bbfe00640 1 -- 172.21.15.131:0/3187379064 >> 172.21.15.131:0/3187379064 conn(0x7f3bc00cff40 msgr2=0x7f3bc00d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:38.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.939+0000 7f3bbfe00640 1 -- 172.21.15.131:0/3187379064 shutdown_connections 2024-05-26T22:55:38.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.939+0000 7f3bbfe00640 1 -- 172.21.15.131:0/3187379064 wait complete. 2024-05-26T22:55:38.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.940+0000 7f3bbfe00640 1 Processor -- start 2024-05-26T22:55:38.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.940+0000 7f3bbfe00640 1 -- start start 2024-05-26T22:55:38.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.941+0000 7f3bbfe00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3bc006ebc0 0x7f3bc015a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:38.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.941+0000 7f3bbfe00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3bc0071b70 0x7f3bc01f7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:38.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.941+0000 7f3bbfe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3bc0144340 0x7f3bc00b0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:38.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.941+0000 7f3bbfe00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f3bc00b4260 con 0x7f3bc0144340 2024-05-26T22:55:38.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.941+0000 7f3bbfe00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3bc0071fc0 con 0x7f3bc0071b70 2024-05-26T22:55:38.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.941+0000 7f3bbfe00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3bc00b43e0 con 0x7f3bc006ebc0 2024-05-26T22:55:38.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.941+0000 7f3bbf400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3bc0144340 0x7f3bc00b0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:38.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.942+0000 7f3bbf400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3bc0144340 0x7f3bc00b0c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:36462/0 (socket says 172.21.15.131:36462) 2024-05-26T22:55:38.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.942+0000 7f3bbf400640 1 -- 172.21.15.131:0/678738605 learned_addr learned my addr 172.21.15.131:0/678738605 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:55:38.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.942+0000 7f3bbf400640 1 -- 172.21.15.131:0/678738605 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3bc006ebc0 msgr2=0x7f3bc015a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:38.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.942+0000 7f3bbf400640 1 --2- 172.21.15.131:0/678738605 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3bc006ebc0 0x7f3bc015a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:38.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.942+0000 7f3bbf400640 1 -- 172.21.15.131:0/678738605 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3bc0071b70 msgr2=0x7f3bc01f7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:38.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.942+0000 7f3bb6000640 1 --2- 172.21.15.131:0/678738605 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3bc0071b70 0x7f3bc01f7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:38.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.942+0000 7f3bbf400640 1 --2- 172.21.15.131:0/678738605 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3bc0071b70 0x7f3bc01f7be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:38.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.942+0000 7f3bbf400640 1 -- 172.21.15.131:0/678738605 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bc013b720 con 0x7f3bc0144340 2024-05-26T22:55:38.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.942+0000 7f3bbf400640 1 --2- 172.21.15.131:0/678738605 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3bc0144340 0x7f3bc00b0c00 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f3bb8035040 tx=0x7f3bb80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:38.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.942+0000 7f3bb7400640 1 -- 172.21.15.131:0/678738605 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bb8037040 con 0x7f3bc0144340 2024-05-26T22:55:38.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.943+0000 7f3bb7400640 1 -- 172.21.15.131:0/678738605 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3bb8042070 con 0x7f3bc0144340 2024-05-26T22:55:38.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.943+0000 7f3bb7400640 1 -- 172.21.15.131:0/678738605 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bb80308a0 con 0x7f3bc0144340 2024-05-26T22:55:38.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.943+0000 7f3bbfe00640 1 -- 172.21.15.131:0/678738605 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bc01f86a0 con 0x7f3bc0144340 2024-05-26T22:55:38.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.943+0000 7f3bbfe00640 1 -- 172.21.15.131:0/678738605 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bc013b9e0 con 0x7f3bc0144340 2024-05-26T22:55:38.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.944+0000 7f3bbfe00640 1 -- 172.21.15.131:0/678738605 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bc00d0850 con 0x7f3bc0144340 2024-05-26T22:55:38.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.945+0000 7f3bb7400640 1 -- 172.21.15.131:0/678738605 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3bb8030a40 con 0x7f3bc0144340 2024-05-26T22:55:38.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.945+0000 7f3bb7400640 1 --2- 172.21.15.131:0/678738605 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f3b9c06be60 0x7f3b9c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:38.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.946+0000 7f3bb7400640 1 -- 172.21.15.131:0/678738605 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 5709+0+0 (secure 0 0 0) 0x7f3bb80ab800 con 0x7f3bc0144340 2024-05-26T22:55:38.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.946+0000 7f3bbea00640 1 --2- 172.21.15.131:0/678738605 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f3b9c06be60 0x7f3b9c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:38.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.946+0000 7f3bbea00640 1 --2- 172.21.15.131:0/678738605 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f3b9c06be60 0x7f3b9c06e320 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f3bb00012e0 tx=0x7f3bb0007d50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:38.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:38.950+0000 7f3bb7400640 1 -- 172.21.15.131:0/678738605 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f3bc00d0850 con 0x7f3bc0144340 2024-05-26T22:55:39.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.185+0000 7f3bbfe00640 1 -- 172.21.15.131:0/678738605 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3bc00694e0 con 0x7f3bc0144340 2024-05-26T22:55:39.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.187+0000 7f3bb7400640 1 -- 172.21.15.131:0/678738605 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) ==== 95+0+5232 (secure 0 0 0) 0x7f3bb807a110 con 0x7f3bc0144340 2024-05-26T22:55:39.187 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:55:39.188 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":100,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5231,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/2037149126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6835","nonce":2037149126},{"type":"v1","addr":"172.21.15.131:6837","nonce":2037149126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":89}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-26T22:54:57.328457+0000","modified":"2024-05-26T22:55:27.458302+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7863,"mds_1":5246,"mds_2":5243},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5243":{"gid":5243,"name":"d","rank":2,"incarnation":97,"state":"up:active","state_seq":7,"addr":"172.21.15.187:6833/1314245822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1314245822},{"type":"v1","addr":"172.21.15.187:6833","nonce":1314245822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5246":{"gid":5246,"name":"b","rank":1,"incarnation":95,"state":"up:active","state_seq":6,"addr":"172.21.15.187:6835/521878375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":521878375},{"type":"v1","addr":"172.21.15.187:6835","nonce":521878375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7863":{"gid":7863,"name":"a","rank":0,"incarnation":90,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6836/789312503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":789312503},{"type":"v1","addr":"172.21.15.131:6836","nonce":789312503}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7863,"qdb_cluster":[5243,5246,7863]},"id":7}]} 2024-05-26T22:55:39.188 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 100 2024-05-26T22:55:39.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.189+0000 7f3bbfe00640 1 -- 172.21.15.131:0/678738605 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f3b9c06be60 msgr2=0x7f3b9c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:39.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.189+0000 7f3bbfe00640 1 --2- 172.21.15.131:0/678738605 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f3b9c06be60 0x7f3b9c06e320 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f3bb00012e0 tx=0x7f3bb0007d50 comp rx=0 tx=0).stop 2024-05-26T22:55:39.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.189+0000 7f3bbfe00640 1 -- 172.21.15.131:0/678738605 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3bc0144340 msgr2=0x7f3bc00b0c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:39.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.189+0000 7f3bbfe00640 1 --2- 172.21.15.131:0/678738605 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3bc0144340 0x7f3bc00b0c00 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f3bb8035040 tx=0x7f3bb80046c0 comp rx=0 tx=0).stop 2024-05-26T22:55:39.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.189+0000 7f3bbfe00640 1 -- 172.21.15.131:0/678738605 shutdown_connections 2024-05-26T22:55:39.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.189+0000 7f3bbfe00640 1 --2- 172.21.15.131:0/678738605 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f3b9c06be60 0x7f3b9c06e320 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:39.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.189+0000 7f3bbfe00640 1 --2- 172.21.15.131:0/678738605 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3bc0144340 0x7f3bc00b0c00 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:39.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.189+0000 7f3bbfe00640 1 --2- 172.21.15.131:0/678738605 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3bc0071b70 0x7f3bc01f7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:39.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.189+0000 7f3bbfe00640 1 --2- 172.21.15.131:0/678738605 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3bc006ebc0 0x7f3bc015a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:39.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.189+0000 7f3bbfe00640 1 -- 172.21.15.131:0/678738605 >> 172.21.15.131:0/678738605 conn(0x7f3bc00cff40 msgr2=0x7f3bc015df40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:39.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.190+0000 7f3bbfe00640 1 -- 172.21.15.131:0/678738605 shutdown_connections 2024-05-26T22:55:39.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.190+0000 7f3bbfe00640 1 -- 172.21.15.131:0/678738605 wait complete. 2024-05-26T22:55:39.207 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-26T22:55:39.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.407+0000 7fea38200640 1 Processor -- start 2024-05-26T22:55:39.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.407+0000 7fea38200640 1 -- start start 2024-05-26T22:55:39.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.408+0000 7fea38200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fea3006ee20 0x7fea300b3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:39.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.408+0000 7fea38200640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fea301569d0 0x7fea300af790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:39.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.408+0000 7fea38200640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea3015cb20 con 0x7fea300ad250 2024-05-26T22:55:39.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.408+0000 7fea38200640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fea3015ce00 con 0x7fea301569d0 2024-05-26T22:55:39.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.408+0000 7fea38200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fea3015d0e0 con 0x7fea3006ee20 2024-05-26T22:55:39.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.409+0000 7fea34e00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fea300ad250 0x7fea301562c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:42082/0 (socket says 172.21.15.131:42082) 2024-05-26T22:55:39.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.409+0000 7fea34e00640 1 -- 172.21.15.131:0/397225327 learned_addr learned my addr 172.21.15.131:0/397225327 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:55:39.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.409+0000 7fea36200640 1 --2- 172.21.15.131:0/397225327 >> v2:172.21.15.187:3300/0 conn(0x7fea301569d0 0x7fea300af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:39.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.409+0000 7fea35800640 1 --2- 172.21.15.131:0/397225327 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fea3006ee20 0x7fea300b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:39.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.409+0000 7fea2fe00640 1 -- 172.21.15.131:0/397225327 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 777643267 0 0) 0x7fea3015cb20 con 0x7fea300ad250 2024-05-26T22:55:39.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.409+0000 7fea2fe00640 1 -- 172.21.15.131:0/397225327 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fea18003520 con 0x7fea300ad250 2024-05-26T22:55:39.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.409+0000 7fea2fe00640 1 -- 172.21.15.131:0/397225327 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1865970305 0 0) 0x7fea18003520 con 0x7fea300ad250 2024-05-26T22:55:39.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.409+0000 7fea2fe00640 1 -- 172.21.15.131:0/397225327 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fea3006ee20 msgr2=0x7fea300b3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:39.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.409+0000 7fea2fe00640 1 --2- 172.21.15.131:0/397225327 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fea3006ee20 0x7fea300b3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:39.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.409+0000 7fea2fe00640 1 -- 172.21.15.131:0/397225327 >> v2:172.21.15.187:3300/0 conn(0x7fea301569d0 msgr2=0x7fea300af790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:39.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.409+0000 7fea2fe00640 1 --2- 172.21.15.131:0/397225327 >> v2:172.21.15.187:3300/0 conn(0x7fea301569d0 0x7fea300af790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:39.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.410+0000 7fea2fe00640 1 -- 172.21.15.131:0/397225327 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea3015d3c0 con 0x7fea300ad250 2024-05-26T22:55:39.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.410+0000 7fea2fe00640 1 -- 172.21.15.131:0/397225327 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7fea1c001670 con 0x7fea300ad250 2024-05-26T22:55:39.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.410+0000 7fea2fe00640 1 -- 172.21.15.131:0/397225327 >> v1:172.21.15.131:6789/0 conn(0x7fea300ad250 legacy=0x7fea301562c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:39.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.410+0000 7fea2fe00640 1 --2- 172.21.15.131:0/397225327 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fea3006ee20 0x7fea180039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:39.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.410+0000 7fea2fe00640 1 --2- 172.21.15.131:0/397225327 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea18003f80 0x7fea18004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:39.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.410+0000 7fea2fe00640 1 --2- 172.21.15.131:0/397225327 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fea18004d40 0x7fea180051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:39.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.410+0000 7fea2fe00640 1 -- 172.21.15.131:0/397225327 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fea3015ce00 con 0x7fea18003f80 2024-05-26T22:55:39.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.410+0000 7fea2fe00640 1 -- 172.21.15.131:0/397225327 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fea3015d0e0 con 0x7fea18004d40 2024-05-26T22:55:39.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.410+0000 7fea2fe00640 1 -- 172.21.15.131:0/397225327 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fea1800b9c0 con 0x7fea3006ee20 2024-05-26T22:55:39.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.410+0000 7fea34e00640 1 --2- 172.21.15.131:0/397225327 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea18003f80 0x7fea18004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:39.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.410+0000 7fea34e00640 1 -- 172.21.15.131:0/397225327 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fea3006ee20 msgr2=0x7fea180039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:39.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.410+0000 7fea34e00640 1 --2- 172.21.15.131:0/397225327 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fea3006ee20 0x7fea180039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:39.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.410+0000 7fea34e00640 1 -- 172.21.15.131:0/397225327 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fea18004d40 msgr2=0x7fea180051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:39.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.410+0000 7fea36200640 1 --2- 172.21.15.131:0/397225327 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fea18004d40 0x7fea180051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:39.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.410+0000 7fea34e00640 1 --2- 172.21.15.131:0/397225327 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fea18004d40 0x7fea180051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:39.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.410+0000 7fea34e00640 1 -- 172.21.15.131:0/397225327 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea18003520 con 0x7fea18003f80 2024-05-26T22:55:39.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.411+0000 7fea34e00640 1 --2- 172.21.15.131:0/397225327 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea18003f80 0x7fea18004800 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7fea1c00c140 tx=0x7fea18005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35e0fb0e3381b842 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:39.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.412+0000 7fea2fe00640 1 -- 172.21.15.131:0/397225327 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea1c007c40 con 0x7fea18003f80 2024-05-26T22:55:39.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.412+0000 7fea2fe00640 1 -- 172.21.15.131:0/397225327 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea1c005af0 con 0x7fea18003f80 2024-05-26T22:55:39.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.412+0000 7fea2fe00640 1 -- 172.21.15.131:0/397225327 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea1c007070 con 0x7fea18003f80 2024-05-26T22:55:39.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.412+0000 7fea38200640 1 -- 172.21.15.131:0/397225327 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea18003f80 msgr2=0x7fea18004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:39.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.412+0000 7fea38200640 1 --2- 172.21.15.131:0/397225327 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea18003f80 0x7fea18004800 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7fea1c00c140 tx=0x7fea18005830 comp rx=0 tx=0).stop 2024-05-26T22:55:39.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.412+0000 7fea36200640 1 -- 172.21.15.131:0/397225327 reap_dead start 2024-05-26T22:55:39.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.412+0000 7fea38200640 1 -- 172.21.15.131:0/397225327 shutdown_connections 2024-05-26T22:55:39.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.412+0000 7fea38200640 1 -- 172.21.15.131:0/397225327 >> 172.21.15.131:0/397225327 conn(0x7fea300b1e90 msgr2=0x7fea300b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:39.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.412+0000 7fea38200640 1 -- 172.21.15.131:0/397225327 shutdown_connections 2024-05-26T22:55:39.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.412+0000 7fea38200640 1 -- 172.21.15.131:0/397225327 wait complete. 2024-05-26T22:55:39.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.414+0000 7fea38200640 1 Processor -- start 2024-05-26T22:55:39.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.414+0000 7fea38200640 1 -- start start 2024-05-26T22:55:39.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.415+0000 7fea38200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fea18003f80 0x7fea301514a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:39.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.415+0000 7fea38200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea30153490 0x7fea301f3b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:39.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.415+0000 7fea38200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fea300b4350 0x7fea300b47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:39.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.415+0000 7fea38200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fea3015c900 con 0x7fea30153490 2024-05-26T22:55:39.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.415+0000 7fea38200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fea3015c780 con 0x7fea300b4350 2024-05-26T22:55:39.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.415+0000 7fea38200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fea3006f200 con 0x7fea18003f80 2024-05-26T22:55:39.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.415+0000 7fea34e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea30153490 0x7fea301f3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:39.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.415+0000 7fea34e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea30153490 0x7fea301f3b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:36476/0 (socket says 172.21.15.131:36476) 2024-05-26T22:55:39.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.415+0000 7fea34e00640 1 -- 172.21.15.131:0/3561022881 learned_addr learned my addr 172.21.15.131:0/3561022881 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:55:39.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.416+0000 7fea35800640 1 --2- 172.21.15.131:0/3561022881 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fea18003f80 0x7fea301514a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:39.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.416+0000 7fea36200640 1 --2- 172.21.15.131:0/3561022881 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fea300b4350 0x7fea300b47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:39.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.416+0000 7fea34e00640 1 -- 172.21.15.131:0/3561022881 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fea18003f80 msgr2=0x7fea301514a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:39.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.416+0000 7fea34e00640 1 --2- 172.21.15.131:0/3561022881 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fea18003f80 0x7fea301514a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:39.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.416+0000 7fea34e00640 1 -- 172.21.15.131:0/3561022881 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fea300b4350 msgr2=0x7fea300b47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:39.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.416+0000 7fea34e00640 1 --2- 172.21.15.131:0/3561022881 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fea300b4350 0x7fea300b47d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:39.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.416+0000 7fea34e00640 1 -- 172.21.15.131:0/3561022881 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea30151cf0 con 0x7fea30153490 2024-05-26T22:55:39.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.416+0000 7fea34e00640 1 --2- 172.21.15.131:0/3561022881 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea30153490 0x7fea301f3b20 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7fea1c02ad80 tx=0x7fea3015d590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:39.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.417+0000 7fea2d600640 1 -- 172.21.15.131:0/3561022881 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea1c043070 con 0x7fea30153490 2024-05-26T22:55:39.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.417+0000 7fea2d600640 1 -- 172.21.15.131:0/3561022881 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea1c038040 con 0x7fea30153490 2024-05-26T22:55:39.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.417+0000 7fea2d600640 1 -- 172.21.15.131:0/3561022881 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea1c007070 con 0x7fea30153490 2024-05-26T22:55:39.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.417+0000 7fea38200640 1 -- 172.21.15.131:0/3561022881 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea30151fd0 con 0x7fea30153490 2024-05-26T22:55:39.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.417+0000 7fea38200640 1 -- 172.21.15.131:0/3561022881 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea30152590 con 0x7fea30153490 2024-05-26T22:55:39.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.418+0000 7fea38200640 1 -- 172.21.15.131:0/3561022881 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea300694e0 con 0x7fea30153490 2024-05-26T22:55:39.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.419+0000 7fea2d600640 1 -- 172.21.15.131:0/3561022881 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fea1c007210 con 0x7fea30153490 2024-05-26T22:55:39.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.420+0000 7fea2d600640 1 --2- 172.21.15.131:0/3561022881 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fea1006be60 0x7fea1006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:39.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.420+0000 7fea2d600640 1 -- 172.21.15.131:0/3561022881 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 5709+0+0 (secure 0 0 0) 0x7fea1c0aab10 con 0x7fea30153490 2024-05-26T22:55:39.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.420+0000 7fea35800640 1 --2- 172.21.15.131:0/3561022881 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fea1006be60 0x7fea1006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:39.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.421+0000 7fea35800640 1 --2- 172.21.15.131:0/3561022881 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fea1006be60 0x7fea1006e320 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fea24009890 tx=0x7fea24008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:39.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.424+0000 7fea2d600640 1 -- 172.21.15.131:0/3561022881 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fea1c079390 con 0x7fea30153490 2024-05-26T22:55:39.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.624+0000 7fea38200640 1 -- 172.21.15.131:0/3561022881 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fea3015c0e0 con 0x7fea30153490 2024-05-26T22:55:39.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.627+0000 7fea2d600640 1 -- 172.21.15.131:0/3561022881 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) ==== 74+0+16632 (secure 0 0 0) 0x7fea1c078c90 con 0x7fea30153490 2024-05-26T22:55:39.627 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:55:39.627 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":95,"fsid":"46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f","created":"2024-05-26T22:47:57.288905+0000","modified":"2024-05-26T22:54:58.901569+0000","last_up_change":"2024-05-26T22:48:09.602207+0000","last_in_change":"2024-05-26T22:48:00.687341+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-26T22:48:10.463339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-05-26T22:54:55.120321+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-05-26T22:54:56.346430+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"814eaf62-a18d-494b-8e27-fd8fb88adea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6801","nonce":2350940051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6803","nonce":2350940051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6807","nonce":2350940051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6805","nonce":2350940051}]},"public_addr":"172.21.15.131:6801/2350940051","cluster_addr":"172.21.15.131:6803/2350940051","heartbeat_back_addr":"172.21.15.131:6807/2350940051","heartbeat_front_addr":"172.21.15.131:6805/2350940051","state":["exists","up"]},{"osd":1,"uuid":"42e66287-f6c6-4e26-af3c-6dfaa3536d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6825","nonce":1525528247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6827","nonce":1525528247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6831","nonce":1525528247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6829","nonce":1525528247}]},"public_addr":"172.21.15.131:6825/1525528247","cluster_addr":"172.21.15.131:6827/1525528247","heartbeat_back_addr":"172.21.15.131:6831/1525528247","heartbeat_front_addr":"172.21.15.131:6829/1525528247","state":["exists","up"]},{"osd":2,"uuid":"0da34673-04a2-4fd0-b57a-0ff2b679b043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6817","nonce":3627842210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6819","nonce":3627842210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6823","nonce":3627842210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6821","nonce":3627842210}]},"public_addr":"172.21.15.131:6817/3627842210","cluster_addr":"172.21.15.131:6819/3627842210","heartbeat_back_addr":"172.21.15.131:6823/3627842210","heartbeat_front_addr":"172.21.15.131:6821/3627842210","state":["exists","up"]},{"osd":3,"uuid":"cf117a27-d6b2-43d3-ae52-b47988bf8eed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6809","nonce":350790591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6811","nonce":350790591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6815","nonce":350790591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6813","nonce":350790591}]},"public_addr":"172.21.15.131:6809/350790591","cluster_addr":"172.21.15.131:6811/350790591","heartbeat_back_addr":"172.21.15.131:6815/350790591","heartbeat_front_addr":"172.21.15.131:6813/350790591","state":["exists","up"]},{"osd":4,"uuid":"a092ccfc-be89-4a84-88d5-0f781fa715eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6816","nonce":244560995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6819","nonce":244560995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6823","nonce":244560995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6821","nonce":244560995}]},"public_addr":"172.21.15.187:6816/244560995","cluster_addr":"172.21.15.187:6819/244560995","heartbeat_back_addr":"172.21.15.187:6823/244560995","heartbeat_front_addr":"172.21.15.187:6821/244560995","state":["exists","up"]},{"osd":5,"uuid":"7513ba69-dd00-4a35-b190-9e8cc3b2bdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6801","nonce":324619398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6803","nonce":324619398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6812","nonce":324619398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6807","nonce":324619398}]},"public_addr":"172.21.15.187:6801/324619398","cluster_addr":"172.21.15.187:6803/324619398","heartbeat_back_addr":"172.21.15.187:6812/324619398","heartbeat_front_addr":"172.21.15.187:6807/324619398","state":["exists","up"]},{"osd":6,"uuid":"50081810-c093-4d2f-a4d7-b7c695c5400d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6806","nonce":1391347786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6809","nonce":1391347786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6818","nonce":1391347786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6813","nonce":1391347786}]},"public_addr":"172.21.15.187:6806/1391347786","cluster_addr":"172.21.15.187:6809/1391347786","heartbeat_back_addr":"172.21.15.187:6818/1391347786","heartbeat_front_addr":"172.21.15.187:6813/1391347786","state":["exists","up"]},{"osd":7,"uuid":"775dae05-54f5-4ad4-bbce-92863c6ed2ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6825","nonce":1220065622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6827","nonce":1220065622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6831","nonce":1220065622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6829","nonce":1220065622}]},"public_addr":"172.21.15.187:6825/1220065622","cluster_addr":"172.21.15.187:6827/1220065622","heartbeat_back_addr":"172.21.15.187:6831/1220065622","heartbeat_front_addr":"172.21.15.187:6829/1220065622","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:05.983520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.299516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.067323+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.307748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.018742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.258775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.010912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.295355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-26T22:55:39.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.629+0000 7fea38200640 1 -- 172.21.15.131:0/3561022881 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fea1006be60 msgr2=0x7fea1006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:39.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.629+0000 7fea38200640 1 --2- 172.21.15.131:0/3561022881 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fea1006be60 0x7fea1006e320 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fea24009890 tx=0x7fea24008040 comp rx=0 tx=0).stop 2024-05-26T22:55:39.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.629+0000 7fea38200640 1 -- 172.21.15.131:0/3561022881 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea30153490 msgr2=0x7fea301f3b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:39.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.629+0000 7fea38200640 1 --2- 172.21.15.131:0/3561022881 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea30153490 0x7fea301f3b20 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7fea1c02ad80 tx=0x7fea3015d590 comp rx=0 tx=0).stop 2024-05-26T22:55:39.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.629+0000 7fea38200640 1 -- 172.21.15.131:0/3561022881 shutdown_connections 2024-05-26T22:55:39.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.629+0000 7fea38200640 1 --2- 172.21.15.131:0/3561022881 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fea1006be60 0x7fea1006e320 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:39.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.629+0000 7fea38200640 1 --2- 172.21.15.131:0/3561022881 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fea300b4350 0x7fea300b47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:39.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.629+0000 7fea38200640 1 --2- 172.21.15.131:0/3561022881 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea30153490 0x7fea301f3b20 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:39.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.629+0000 7fea38200640 1 --2- 172.21.15.131:0/3561022881 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fea18003f80 0x7fea301514a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:39.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.629+0000 7fea38200640 1 -- 172.21.15.131:0/3561022881 >> 172.21.15.131:0/3561022881 conn(0x7fea300b1e90 msgr2=0x7fea300b1520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:39.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.630+0000 7fea38200640 1 -- 172.21.15.131:0/3561022881 shutdown_connections 2024-05-26T22:55:39.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.630+0000 7fea38200640 1 -- 172.21.15.131:0/3561022881 wait complete. 2024-05-26T22:55:39.647 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-26T22:55:39.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.845+0000 7f43be400640 1 Processor -- start 2024-05-26T22:55:39.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.845+0000 7f43be400640 1 -- start start 2024-05-26T22:55:39.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.847+0000 7f43be400640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f43b81410d0 0x7f43b8165dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:39.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.847+0000 7f43be400640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43b813e820 con 0x7f43b806d250 2024-05-26T22:55:39.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.847+0000 7f43be400640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43b815c4a0 con 0x7f43b815b730 2024-05-26T22:55:39.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.847+0000 7f43be400640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f43b8071260 con 0x7f43b81410d0 2024-05-26T22:55:39.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.847+0000 7f43bd000640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f43b806d250 0x7f43b806d630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:42096/0 (socket says 172.21.15.131:42096) 2024-05-26T22:55:39.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.847+0000 7f43bd000640 1 -- 172.21.15.131:0/2371680255 learned_addr learned my addr 172.21.15.131:0/2371680255 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:55:39.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.848+0000 7f43b7e00640 1 -- 172.21.15.131:0/2371680255 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2164201032 0 0) 0x7f43b813e820 con 0x7f43b806d250 2024-05-26T22:55:39.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.848+0000 7f43b7e00640 1 -- 172.21.15.131:0/2371680255 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f439c003520 con 0x7f43b806d250 2024-05-26T22:55:39.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.848+0000 7f43bda00640 1 --2- 172.21.15.131:0/2371680255 >> v2:172.21.15.187:3300/0 conn(0x7f43b81410d0 0x7f43b8165dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:39.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.848+0000 7f43b7e00640 1 -- 172.21.15.131:0/2371680255 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1256421768 0 0) 0x7f439c003520 con 0x7f43b806d250 2024-05-26T22:55:39.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.848+0000 7f43b7e00640 1 -- 172.21.15.131:0/2371680255 >> v2:172.21.15.187:3300/0 conn(0x7f43b81410d0 msgr2=0x7f43b8165dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:39.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.848+0000 7f43b7e00640 1 --2- 172.21.15.131:0/2371680255 >> v2:172.21.15.187:3300/0 conn(0x7f43b81410d0 0x7f43b8165dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:39.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.848+0000 7f43b7e00640 1 -- 172.21.15.131:0/2371680255 >> v1:172.21.15.187:6789/0 conn(0x7f43b815b730 legacy=0x7f43b815bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:39.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.848+0000 7f43b7e00640 1 -- 172.21.15.131:0/2371680255 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43b80714b0 con 0x7f43b806d250 2024-05-26T22:55:39.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.848+0000 7f43b7e00640 1 -- 172.21.15.131:0/2371680255 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7f43a8001670 con 0x7f43b806d250 2024-05-26T22:55:39.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.848+0000 7f43b7e00640 1 -- 172.21.15.131:0/2371680255 >> v1:172.21.15.131:6789/0 conn(0x7f43b806d250 legacy=0x7f43b806d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:39.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.848+0000 7f43b7e00640 1 --2- 172.21.15.131:0/2371680255 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f439c003a30 0x7f439c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:39.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.848+0000 7f43b7e00640 1 --2- 172.21.15.131:0/2371680255 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f439c004800 0x7f439c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:39.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.848+0000 7f43b7e00640 1 --2- 172.21.15.131:0/2371680255 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f439c0052e0 0x7f439c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:39.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.848+0000 7f43b7e00640 1 -- 172.21.15.131:0/2371680255 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f43b8071260 con 0x7f439c004800 2024-05-26T22:55:39.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.848+0000 7f43b7e00640 1 -- 172.21.15.131:0/2371680255 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f439c00c1b0 con 0x7f439c003a30 2024-05-26T22:55:39.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.849+0000 7f43b7e00640 1 -- 172.21.15.131:0/2371680255 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f439c00e5c0 con 0x7f439c0052e0 2024-05-26T22:55:39.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.849+0000 7f43bda00640 1 --2- 172.21.15.131:0/2371680255 >> v2:172.21.15.187:3300/0 conn(0x7f43b81410d0 0x7f43b8165dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-26T22:55:39.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.849+0000 7f43afe00640 1 --2- 172.21.15.131:0/2371680255 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f439c004800 0x7f439c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:39.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.849+0000 7f43bd000640 1 --2- 172.21.15.131:0/2371680255 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f439c003a30 0x7f439c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:39.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.849+0000 7f43afe00640 1 -- 172.21.15.131:0/2371680255 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f439c0052e0 msgr2=0x7f439c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:39.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.849+0000 7f43afe00640 1 --2- 172.21.15.131:0/2371680255 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f439c0052e0 0x7f439c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:39.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.849+0000 7f43afe00640 1 -- 172.21.15.131:0/2371680255 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f439c003a30 msgr2=0x7f439c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:39.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.849+0000 7f43afe00640 1 --2- 172.21.15.131:0/2371680255 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f439c003a30 0x7f439c003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:39.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.849+0000 7f43afe00640 1 -- 172.21.15.131:0/2371680255 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f439c003520 con 0x7f439c004800 2024-05-26T22:55:39.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.849+0000 7f43bd000640 1 --2- 172.21.15.131:0/2371680255 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f439c003a30 0x7f439c003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:55:39.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.849+0000 7f43bda00640 1 -- 172.21.15.131:0/2371680255 reap_dead start 2024-05-26T22:55:39.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.850+0000 7f43afe00640 1 --2- 172.21.15.131:0/2371680255 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f439c004800 0x7f439c004c80 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f43a0001720 tx=0x7f439c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=526c541142569cb4 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:39.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.851+0000 7f43b7e00640 1 -- 172.21.15.131:0/2371680255 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a0001370 con 0x7f439c004800 2024-05-26T22:55:39.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.851+0000 7f43b7e00640 1 -- 172.21.15.131:0/2371680255 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43a00052d0 con 0x7f439c004800 2024-05-26T22:55:39.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.851+0000 7f43b7e00640 1 -- 172.21.15.131:0/2371680255 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a002ab70 con 0x7f439c004800 2024-05-26T22:55:39.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.851+0000 7f43be400640 1 -- 172.21.15.131:0/2371680255 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f439c004800 msgr2=0x7f439c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:39.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.851+0000 7f43be400640 1 --2- 172.21.15.131:0/2371680255 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f439c004800 0x7f439c004c80 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f43a0001720 tx=0x7f439c005830 comp rx=0 tx=0).stop 2024-05-26T22:55:39.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.851+0000 7f43be400640 1 -- 172.21.15.131:0/2371680255 shutdown_connections 2024-05-26T22:55:39.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.851+0000 7f43be400640 1 --2- 172.21.15.131:0/2371680255 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f439c004800 0x7f439c004c80 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:39.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.851+0000 7f43be400640 1 -- 172.21.15.131:0/2371680255 >> 172.21.15.131:0/2371680255 conn(0x7f43b80b5660 msgr2=0x7f43b80b5a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:39.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.851+0000 7f43be400640 1 -- 172.21.15.131:0/2371680255 shutdown_connections 2024-05-26T22:55:39.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.851+0000 7f43be400640 1 -- 172.21.15.131:0/2371680255 wait complete. 2024-05-26T22:55:39.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.853+0000 7f43be400640 1 Processor -- start 2024-05-26T22:55:39.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.853+0000 7f43be400640 1 -- start start 2024-05-26T22:55:39.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.854+0000 7f43be400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f439c004800 0x7f43b8154910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:39.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.854+0000 7f43be400640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f43b815d840 0x7f43b815dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:39.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.854+0000 7f43be400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f43b8157050 0x7f43b81574d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:39.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.854+0000 7f43be400640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f43b8141500 con 0x7f439c004800 2024-05-26T22:55:39.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.854+0000 7f43be400640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f43b8071a70 con 0x7f43b815d840 2024-05-26T22:55:39.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.854+0000 7f43be400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f43b8141680 con 0x7f43b8157050 2024-05-26T22:55:39.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.854+0000 7f43bd000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f439c004800 0x7f43b8154910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:39.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.854+0000 7f43bd000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f439c004800 0x7f43b8154910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:36494/0 (socket says 172.21.15.131:36494) 2024-05-26T22:55:39.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.854+0000 7f43bd000640 1 -- 172.21.15.131:0/1956819357 learned_addr learned my addr 172.21.15.131:0/1956819357 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:55:39.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.855+0000 7f43bd000640 1 -- 172.21.15.131:0/1956819357 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f43b8157050 msgr2=0x7f43b81574d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:39.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.855+0000 7f43bd000640 1 --2- 172.21.15.131:0/1956819357 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f43b8157050 0x7f43b81574d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:39.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.855+0000 7f43bd000640 1 -- 172.21.15.131:0/1956819357 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f43b815d840 msgr2=0x7f43b815dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:39.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.855+0000 7f43bd000640 1 --2- 172.21.15.131:0/1956819357 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f43b815d840 0x7f43b815dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:39.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.855+0000 7f43bd000640 1 -- 172.21.15.131:0/1956819357 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43b812d540 con 0x7f439c004800 2024-05-26T22:55:39.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.855+0000 7f43bd000640 1 --2- 172.21.15.131:0/1956819357 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f439c004800 0x7f43b8154910 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f43a800d7a0 tx=0x7f43a800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:39.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.855+0000 7f43b5600640 1 -- 172.21.15.131:0/1956819357 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a8019070 con 0x7f439c004800 2024-05-26T22:55:39.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.855+0000 7f43b5600640 1 -- 172.21.15.131:0/1956819357 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43a80074e0 con 0x7f439c004800 2024-05-26T22:55:39.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.855+0000 7f43be400640 1 -- 172.21.15.131:0/1956819357 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43b8157d40 con 0x7f439c004800 2024-05-26T22:55:39.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.855+0000 7f43b5600640 1 -- 172.21.15.131:0/1956819357 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a80049e0 con 0x7f439c004800 2024-05-26T22:55:39.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.856+0000 7f43be400640 1 -- 172.21.15.131:0/1956819357 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43b8158300 con 0x7f439c004800 2024-05-26T22:55:39.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.857+0000 7f43be400640 1 -- 172.21.15.131:0/1956819357 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43b80694e0 con 0x7f439c004800 2024-05-26T22:55:39.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.858+0000 7f43b5600640 1 -- 172.21.15.131:0/1956819357 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f43a801c070 con 0x7f439c004800 2024-05-26T22:55:39.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.858+0000 7f43b5600640 1 --2- 172.21.15.131:0/1956819357 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f439406be30 0x7f439406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:39.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.859+0000 7f43b5600640 1 -- 172.21.15.131:0/1956819357 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 5709+0+0 (secure 0 0 0) 0x7f43a808a940 con 0x7f439c004800 2024-05-26T22:55:39.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.859+0000 7f43afe00640 1 --2- 172.21.15.131:0/1956819357 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f439406be30 0x7f439406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:39.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.859+0000 7f43afe00640 1 --2- 172.21.15.131:0/1956819357 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f439406be30 0x7f439406e2f0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f43a0032000 tx=0x7f43a002c000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:39.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:39.863+0000 7f43b5600640 1 -- 172.21.15.131:0/1956819357 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f43a8059250 con 0x7f439c004800 2024-05-26T22:55:40.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:40.104+0000 7f43be400640 1 -- 172.21.15.131:0/1956819357 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f43b80b4d20 con 0x7f439c004800 2024-05-26T22:55:40.104 INFO:tasks.ceph.mon.a.smithi131.stderr:2024-05-26T22:55:40.105+0000 7fcf0d200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-26T22:55:40.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:40.109+0000 7f43b5600640 1 -- 172.21.15.131:0/1956819357 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v101) ==== 161+0+0 (secure 0 0 0) 0x7f43a8058b50 con 0x7f439c004800 2024-05-26T22:55:40.108 INFO:teuthology.orchestra.run.smithi131.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-26T22:55:40.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:40.112+0000 7f43be400640 1 -- 172.21.15.131:0/1956819357 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f439406be30 msgr2=0x7f439406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:40.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:40.112+0000 7f43be400640 1 --2- 172.21.15.131:0/1956819357 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f439406be30 0x7f439406e2f0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f43a0032000 tx=0x7f43a002c000 comp rx=0 tx=0).stop 2024-05-26T22:55:40.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:40.112+0000 7f43be400640 1 -- 172.21.15.131:0/1956819357 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f439c004800 msgr2=0x7f43b8154910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:40.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:40.112+0000 7f43be400640 1 --2- 172.21.15.131:0/1956819357 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f439c004800 0x7f43b8154910 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f43a800d7a0 tx=0x7f43a800dc60 comp rx=0 tx=0).stop 2024-05-26T22:55:40.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:40.112+0000 7f43be400640 1 -- 172.21.15.131:0/1956819357 shutdown_connections 2024-05-26T22:55:40.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:40.112+0000 7f43be400640 1 --2- 172.21.15.131:0/1956819357 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f43b8157050 0x7f43b81574d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:40.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:40.112+0000 7f43be400640 1 --2- 172.21.15.131:0/1956819357 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f439406be30 0x7f439406e2f0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:40.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:40.112+0000 7f43be400640 1 --2- 172.21.15.131:0/1956819357 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f43b815d840 0x7f43b815dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:40.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:40.112+0000 7f43be400640 1 --2- 172.21.15.131:0/1956819357 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f439c004800 0x7f43b8154910 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:40.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:40.112+0000 7f43be400640 1 -- 172.21.15.131:0/1956819357 >> 172.21.15.131:0/1956819357 conn(0x7f43b80b5660 msgr2=0x7f43b8159c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:40.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:40.112+0000 7f43be400640 1 -- 172.21.15.131:0/1956819357 shutdown_connections 2024-05-26T22:55:40.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:40.112+0000 7f43be400640 1 -- 172.21.15.131:0/1956819357 wait complete. 2024-05-26T22:55:40.133 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-26T22:55:40.150 INFO:tasks.ceph.mds.a.smithi131.stderr: -8395> 2024-05-26T22:55:17.835+0000 7f2ce4000640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-05-26T22:55:40.192 INFO:tasks.ceph.mds.d.smithi187.stdout:starting mds.d at 2024-05-26T22:55:40.206 INFO:tasks.ceph.mds.b.smithi187.stdout:starting mds.b at 2024-05-26T22:55:40.230 INFO:tasks.ceph.mds.a.smithi131.stdout:starting mds.a at 2024-05-26T22:55:40.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:40.322+0000 7f48ebe00640 1 Processor -- start 2024-05-26T22:55:40.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:40.322+0000 7f48ebe00640 1 -- start start 2024-05-26T22:55:40.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:40.323+0000 7f48ebe00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f48ec15e5b0 0x7f48ec0ced40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:40.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:40.323+0000 7f48ebe00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48ec0b3430 0x7f48ec0cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:40.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:40.323+0000 7f48ebe00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48ec0d2780 con 0x7f48ec0cf7c0 2024-05-26T22:55:40.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:40.323+0000 7f48ebe00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f48ec0d2a60 con 0x7f48ec15e5b0 2024-05-26T22:55:40.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:40.323+0000 7f48ebe00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f48ec0d2d40 con 0x7f48ec0b3430 2024-05-26T22:55:40.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:40.323+0000 7f48eaa00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f48ec15e5b0 0x7f48ec0ced40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:40.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:40.323+0000 7f48eaa00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f48ec15e5b0 0x7f48ec0ced40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:36498/0 (socket says 172.21.15.131:36498) 2024-05-26T22:55:40.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:40.323+0000 7f48eaa00640 1 -- 172.21.15.131:0/2004251821 learned_addr learned my addr 172.21.15.131:0/2004251821 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:55:40.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:40.324+0000 7f48ea000640 1 --2- 172.21.15.131:0/2004251821 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48ec0b3430 0x7f48ec0cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:40.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:40.324+0000 7f48eaa00640 1 -- 172.21.15.131:0/2004251821 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48ec0b3430 msgr2=0x7f48ec0cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:40.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:40.324+0000 7f48eaa00640 1 --2- 172.21.15.131:0/2004251821 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48ec0b3430 0x7f48ec0cf280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:40.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:40.324+0000 7f48eaa00640 1 -- 172.21.15.131:0/2004251821 >> v1:172.21.15.131:6789/0 conn(0x7f48ec0cf7c0 legacy=0x7f48ec0d09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:40.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:40.324+0000 7f48eaa00640 1 -- 172.21.15.131:0/2004251821 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48ec0d3020 con 0x7f48ec15e5b0 2024-05-26T22:55:40.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:40.325+0000 7f48eaa00640 1 --2- 172.21.15.131:0/2004251821 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48ec15e5b0 0x7f48ec0ced40 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f48dc009870 tx=0x7f48dc02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=699b75d914421f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:40.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:40.325+0000 7f48e9600640 1 -- 172.21.15.131:0/2004251821 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48dc037070 con 0x7f48ec15e5b0 2024-05-26T22:55:40.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:40.325+0000 7f48e9600640 1 -- 172.21.15.131:0/2004251821 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48dc032440 con 0x7f48ec15e5b0 2024-05-26T22:55:40.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:40.325+0000 7f48e9600640 1 -- 172.21.15.131:0/2004251821 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48dc032810 con 0x7f48ec15e5b0 2024-05-26T22:55:40.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:40.325+0000 7f48ebe00640 1 -- 172.21.15.131:0/2004251821 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48ec15e5b0 msgr2=0x7f48ec0ced40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:40.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:40.325+0000 7f48ebe00640 1 --2- 172.21.15.131:0/2004251821 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48ec15e5b0 0x7f48ec0ced40 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f48dc009870 tx=0x7f48dc02a1c0 comp rx=0 tx=0).stop 2024-05-26T22:55:40.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:40.326+0000 7f48ebe00640 1 -- 172.21.15.131:0/2004251821 shutdown_connections 2024-05-26T22:55:40.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:40.326+0000 7f48ebe00640 1 --2- 172.21.15.131:0/2004251821 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48ec0b3430 0x7f48ec0cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:40.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:40.326+0000 7f48ebe00640 1 --2- 172.21.15.131:0/2004251821 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48ec15e5b0 0x7f48ec0ced40 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:40.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:40.326+0000 7f48ebe00640 1 -- 172.21.15.131:0/2004251821 >> 172.21.15.131:0/2004251821 conn(0x7f48ec09d4d0 msgr2=0x7f48ec0d41a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:40.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:40.326+0000 7f48ebe00640 1 -- 172.21.15.131:0/2004251821 shutdown_connections 2024-05-26T22:55:40.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:40.326+0000 7f48ebe00640 1 -- 172.21.15.131:0/2004251821 wait complete. 2024-05-26T22:55:40.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:40.327+0000 7f48ebe00640 1 Processor -- start 2024-05-26T22:55:40.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:40.328+0000 7f48ebe00640 1 -- start start 2024-05-26T22:55:40.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:40.328+0000 7f48ebe00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f48ec0b3430 0x7f48ec1f3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:40.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:40.328+0000 7f48ebe00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48ec0cf7c0 0x7f48ec1f78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:40.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:40.328+0000 7f48ebe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48ec0b6dc0 0x7f48ec1f7e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:40.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:40.328+0000 7f48ebe00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f48ec0b5030 con 0x7f48ec0b6dc0 2024-05-26T22:55:40.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:40.328+0000 7f48ebe00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f48ec0cfc10 con 0x7f48ec0b3430 2024-05-26T22:55:40.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:40.328+0000 7f48ebe00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f48ec0b51b0 con 0x7f48ec0cf7c0 2024-05-26T22:55:40.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:40.329+0000 7f48eb400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48ec0b6dc0 0x7f48ec1f7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:40.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:40.329+0000 7f48eb400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48ec0b6dc0 0x7f48ec1f7e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:36512/0 (socket says 172.21.15.131:36512) 2024-05-26T22:55:40.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:40.329+0000 7f48eb400640 1 -- 172.21.15.131:0/912266340 learned_addr learned my addr 172.21.15.131:0/912266340 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:55:40.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:40.329+0000 7f48ea000640 1 --2- 172.21.15.131:0/912266340 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48ec0cf7c0 0x7f48ec1f78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:40.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:40.329+0000 7f48eb400640 1 -- 172.21.15.131:0/912266340 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48ec0cf7c0 msgr2=0x7f48ec1f78d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:40.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:40.329+0000 7f48eb400640 1 --2- 172.21.15.131:0/912266340 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48ec0cf7c0 0x7f48ec1f78d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:40.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:40.329+0000 7f48eaa00640 1 --2- 172.21.15.131:0/912266340 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f48ec0b3430 0x7f48ec1f3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:40.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:40.329+0000 7f48eb400640 1 -- 172.21.15.131:0/912266340 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f48ec0b3430 msgr2=0x7f48ec1f3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:40.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:40.329+0000 7f48eb400640 1 --2- 172.21.15.131:0/912266340 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f48ec0b3430 0x7f48ec1f3c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:40.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:40.329+0000 7f48eb400640 1 -- 172.21.15.131:0/912266340 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48ec1f41c0 con 0x7f48ec0b6dc0 2024-05-26T22:55:40.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:40.330+0000 7f48eb400640 1 --2- 172.21.15.131:0/912266340 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48ec0b6dc0 0x7f48ec1f7e10 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f48e400c7a0 tx=0x7f48e400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:40.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:40.330+0000 7f48e2a00640 1 -- 172.21.15.131:0/912266340 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48e4003b70 con 0x7f48ec0b6dc0 2024-05-26T22:55:40.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:40.330+0000 7f48e2a00640 1 -- 172.21.15.131:0/912266340 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48e4003d30 con 0x7f48ec0b6dc0 2024-05-26T22:55:40.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:40.330+0000 7f48e2a00640 1 -- 172.21.15.131:0/912266340 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48e400aa90 con 0x7f48ec0b6dc0 2024-05-26T22:55:40.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:40.330+0000 7f48ebe00640 1 -- 172.21.15.131:0/912266340 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48ec0b7330 con 0x7f48ec0b6dc0 2024-05-26T22:55:40.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:40.330+0000 7f48ebe00640 1 -- 172.21.15.131:0/912266340 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48ec0ccfb0 con 0x7f48ec0b6dc0 2024-05-26T22:55:40.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:40.331+0000 7f48ebe00640 1 -- 172.21.15.131:0/912266340 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48ec0694e0 con 0x7f48ec0b6dc0 2024-05-26T22:55:40.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:40.332+0000 7f48e2a00640 1 -- 172.21.15.131:0/912266340 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f48e400ac30 con 0x7f48ec0b6dc0 2024-05-26T22:55:40.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:40.333+0000 7f48e2a00640 1 --2- 172.21.15.131:0/912266340 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f48cc06be30 0x7f48cc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:40.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:40.333+0000 7f48e2a00640 1 -- 172.21.15.131:0/912266340 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5967+0+0 (secure 0 0 0) 0x7f48e408af40 con 0x7f48ec0b6dc0 2024-05-26T22:55:40.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:40.333+0000 7f48eaa00640 1 --2- 172.21.15.131:0/912266340 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f48cc06be30 0x7f48cc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:40.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:40.334+0000 7f48eaa00640 1 --2- 172.21.15.131:0/912266340 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f48cc06be30 0x7f48cc06e2f0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f48dc034000 tx=0x7f48dc02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:40.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:40.338+0000 7f48e2a00640 1 -- 172.21.15.131:0/912266340 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f48e4059750 con 0x7f48ec0b6dc0 2024-05-26T22:55:40.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:40.575+0000 7f48ebe00640 1 -- 172.21.15.131:0/912266340 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f48ec0d0d30 con 0x7f48ec0b6dc0 2024-05-26T22:55:41.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:41.112+0000 7f48e2a00640 1 -- 172.21.15.131:0/912266340 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v102) ==== 104+0+0 (secure 0 0 0) 0x7f48e4059050 con 0x7f48ec0b6dc0 2024-05-26T22:55:41.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:41.116+0000 7f48ebe00640 1 -- 172.21.15.131:0/912266340 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f48cc06be30 msgr2=0x7f48cc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:41.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:41.116+0000 7f48ebe00640 1 --2- 172.21.15.131:0/912266340 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f48cc06be30 0x7f48cc06e2f0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f48dc034000 tx=0x7f48dc02e000 comp rx=0 tx=0).stop 2024-05-26T22:55:41.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:41.116+0000 7f48ebe00640 1 -- 172.21.15.131:0/912266340 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48ec0b6dc0 msgr2=0x7f48ec1f7e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:41.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:41.116+0000 7f48ebe00640 1 --2- 172.21.15.131:0/912266340 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48ec0b6dc0 0x7f48ec1f7e10 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f48e400c7a0 tx=0x7f48e400cc60 comp rx=0 tx=0).stop 2024-05-26T22:55:41.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:41.117+0000 7f48ebe00640 1 -- 172.21.15.131:0/912266340 shutdown_connections 2024-05-26T22:55:41.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:41.117+0000 7f48ebe00640 1 --2- 172.21.15.131:0/912266340 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48ec0cf7c0 0x7f48ec1f78d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:41.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:41.117+0000 7f48ebe00640 1 --2- 172.21.15.131:0/912266340 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f48cc06be30 0x7f48cc06e2f0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:41.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:41.117+0000 7f48ebe00640 1 --2- 172.21.15.131:0/912266340 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48ec0b6dc0 0x7f48ec1f7e10 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:41.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:41.117+0000 7f48ebe00640 1 --2- 172.21.15.131:0/912266340 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f48ec0b3430 0x7f48ec1f3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:41.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:41.117+0000 7f48ebe00640 1 -- 172.21.15.131:0/912266340 >> 172.21.15.131:0/912266340 conn(0x7f48ec09d4d0 msgr2=0x7f48ec0d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:41.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:41.117+0000 7f48ebe00640 1 -- 172.21.15.131:0/912266340 shutdown_connections 2024-05-26T22:55:41.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:41.117+0000 7f48ebe00640 1 -- 172.21.15.131:0/912266340 wait complete. 2024-05-26T22:55:41.141 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-26T22:55:41.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:41.392+0000 7f6bb6c00640 1 Processor -- start 2024-05-26T22:55:41.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:41.392+0000 7f6bb6c00640 1 -- start start 2024-05-26T22:55:41.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:41.394+0000 7f6bb6c00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f6bb00d0a70 0x7f6bb0072250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:41.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:41.394+0000 7f6bb6c00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f6bb006e860 0x7f6bb006ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:41.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:41.394+0000 7f6bb6c00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6bb006f210 con 0x7f6bb00cf8f0 2024-05-26T22:55:41.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:41.394+0000 7f6bb6c00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f6bb00ccb70 con 0x7f6bb006e860 2024-05-26T22:55:41.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:41.394+0000 7f6bb6c00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f6bb00cce50 con 0x7f6bb00d0a70 2024-05-26T22:55:41.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:41.394+0000 7f6bb4c00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f6bb006e860 0x7f6bb006ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:41.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:41.394+0000 7f6bafe00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f6bb00cf8f0 0x7f6bb00d0360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:42108/0 (socket says 172.21.15.131:42108) 2024-05-26T22:55:41.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:41.394+0000 7f6bafe00640 1 -- 172.21.15.131:0/4192613570 learned_addr learned my addr 172.21.15.131:0/4192613570 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:55:41.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:41.395+0000 7f6bb4c00640 1 -- 172.21.15.131:0/4192613570 >> v2:172.21.15.187:3300/0 conn(0x7f6bb00d0a70 msgr2=0x7f6bb0072250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:41.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:41.395+0000 7f6bb4c00640 1 --2- 172.21.15.131:0/4192613570 >> v2:172.21.15.187:3300/0 conn(0x7f6bb00d0a70 0x7f6bb0072250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:41.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:41.395+0000 7f6bb4c00640 1 -- 172.21.15.131:0/4192613570 >> v1:172.21.15.131:6789/0 conn(0x7f6bb00cf8f0 legacy=0x7f6bb00d0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:41.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:41.395+0000 7f6bb4c00640 1 -- 172.21.15.131:0/4192613570 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bb00cd130 con 0x7f6bb006e860 2024-05-26T22:55:41.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:41.395+0000 7f6baf400640 1 -- 172.21.15.131:0/4192613570 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 875249564 0 0) 0x7f6bb006f210 con 0x7f6bb00cf8f0 2024-05-26T22:55:41.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:41.396+0000 7f6bb4c00640 1 --2- 172.21.15.131:0/4192613570 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6bb006e860 0x7f6bb006ecd0 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f6bb006c3d0 tx=0x7f6ba802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3033dfd87a8b32d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:41.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:41.396+0000 7f6baf400640 1 -- 172.21.15.131:0/4192613570 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba8007740 con 0x7f6bb006e860 2024-05-26T22:55:41.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:41.396+0000 7f6baf400640 1 -- 172.21.15.131:0/4192613570 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ba80078e0 con 0x7f6bb006e860 2024-05-26T22:55:41.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:41.396+0000 7f6baf400640 1 -- 172.21.15.131:0/4192613570 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba8007cb0 con 0x7f6bb006e860 2024-05-26T22:55:41.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:41.397+0000 7f6bb6c00640 1 -- 172.21.15.131:0/4192613570 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6bb006e860 msgr2=0x7f6bb006ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:41.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:41.397+0000 7f6bb6c00640 1 --2- 172.21.15.131:0/4192613570 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6bb006e860 0x7f6bb006ecd0 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f6bb006c3d0 tx=0x7f6ba802a680 comp rx=0 tx=0).stop 2024-05-26T22:55:41.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:41.397+0000 7f6bb6c00640 1 -- 172.21.15.131:0/4192613570 shutdown_connections 2024-05-26T22:55:41.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:41.397+0000 7f6bb6c00640 1 --2- 172.21.15.131:0/4192613570 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6bb006e860 0x7f6bb006ecd0 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:41.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:41.397+0000 7f6bb6c00640 1 --2- 172.21.15.131:0/4192613570 >> v2:172.21.15.187:3300/0 conn(0x7f6bb00d0a70 0x7f6bb0072250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:41.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:41.397+0000 7f6bb6c00640 1 -- 172.21.15.131:0/4192613570 >> 172.21.15.131:0/4192613570 conn(0x7f6bb00b2110 msgr2=0x7f6bb00b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:41.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:41.397+0000 7f6bb6c00640 1 -- 172.21.15.131:0/4192613570 shutdown_connections 2024-05-26T22:55:41.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:41.397+0000 7f6bb6c00640 1 -- 172.21.15.131:0/4192613570 wait complete. 2024-05-26T22:55:41.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:41.399+0000 7f6bb6c00640 1 Processor -- start 2024-05-26T22:55:41.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:41.399+0000 7f6bb6c00640 1 -- start start 2024-05-26T22:55:41.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:41.400+0000 7f6bb6c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6bb006e860 0x7f6bb00b1240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:41.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:41.400+0000 7f6bb6c00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6bb00cf8f0 0x7f6bb00cb250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:41.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:41.400+0000 7f6bb6c00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6bb00d0a70 0x7f6bb0152c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:41.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:41.400+0000 7f6bb6c00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb015ffd0 con 0x7f6bb006e860 2024-05-26T22:55:41.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:41.400+0000 7f6bb6c00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb00cfcd0 con 0x7f6bb00cf8f0 2024-05-26T22:55:41.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:41.400+0000 7f6bb6c00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f6bb0160150 con 0x7f6bb00d0a70 2024-05-26T22:55:41.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:41.400+0000 7f6bafe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6bb006e860 0x7f6bb00b1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:41.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:41.400+0000 7f6bafe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6bb006e860 0x7f6bb00b1240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:36534/0 (socket says 172.21.15.131:36534) 2024-05-26T22:55:41.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:41.400+0000 7f6bafe00640 1 -- 172.21.15.131:0/2650346572 learned_addr learned my addr 172.21.15.131:0/2650346572 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:55:41.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:41.401+0000 7f6bafe00640 1 -- 172.21.15.131:0/2650346572 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6bb00d0a70 msgr2=0x7f6bb0152c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:41.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:41.401+0000 7f6bafe00640 1 --2- 172.21.15.131:0/2650346572 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6bb00d0a70 0x7f6bb0152c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:41.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:41.401+0000 7f6bafe00640 1 -- 172.21.15.131:0/2650346572 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6bb00cf8f0 msgr2=0x7f6bb00cb250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:41.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:41.401+0000 7f6ba7400640 1 --2- 172.21.15.131:0/2650346572 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6bb00cf8f0 0x7f6bb00cb250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:41.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:41.401+0000 7f6bafe00640 1 --2- 172.21.15.131:0/2650346572 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6bb00cf8f0 0x7f6bb00cb250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:41.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:41.401+0000 7f6bafe00640 1 -- 172.21.15.131:0/2650346572 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bb00cbbc0 con 0x7f6bb006e860 2024-05-26T22:55:41.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:41.401+0000 7f6ba7400640 1 --2- 172.21.15.131:0/2650346572 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6bb00cf8f0 0x7f6bb00cb250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:55:41.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:41.401+0000 7f6bafe00640 1 --2- 172.21.15.131:0/2650346572 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6bb006e860 0x7f6bb00b1240 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f6ba000c4d0 tx=0x7f6ba000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:41.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:41.401+0000 7f6bacc00640 1 -- 172.21.15.131:0/2650346572 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba0018070 con 0x7f6bb006e860 2024-05-26T22:55:41.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:41.401+0000 7f6bacc00640 1 -- 172.21.15.131:0/2650346572 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ba0007500 con 0x7f6bb006e860 2024-05-26T22:55:41.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:41.401+0000 7f6bb6c00640 1 -- 172.21.15.131:0/2650346572 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bb009bca0 con 0x7f6bb006e860 2024-05-26T22:55:41.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:41.401+0000 7f6bacc00640 1 -- 172.21.15.131:0/2650346572 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba0004a20 con 0x7f6bb006e860 2024-05-26T22:55:41.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:41.402+0000 7f6bb6c00640 1 -- 172.21.15.131:0/2650346572 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bb009c260 con 0x7f6bb006e860 2024-05-26T22:55:41.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:41.403+0000 7f6bb6c00640 1 -- 172.21.15.131:0/2650346572 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bb0068440 con 0x7f6bb006e860 2024-05-26T22:55:41.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:41.404+0000 7f6bacc00640 1 -- 172.21.15.131:0/2650346572 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6ba001b070 con 0x7f6bb006e860 2024-05-26T22:55:41.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:41.405+0000 7f6bacc00640 1 --2- 172.21.15.131:0/2650346572 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f6b8c06be30 0x7f6b8c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:41.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:41.405+0000 7f6bacc00640 1 -- 172.21.15.131:0/2650346572 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5967+0+0 (secure 0 0 0) 0x7f6ba008a380 con 0x7f6bb006e860 2024-05-26T22:55:41.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:41.405+0000 7f6ba7400640 1 --2- 172.21.15.131:0/2650346572 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f6b8c06be30 0x7f6b8c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:41.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:41.405+0000 7f6ba7400640 1 --2- 172.21.15.131:0/2650346572 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f6b8c06be30 0x7f6b8c06e2f0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f6b98006fd0 tx=0x7f6b98008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:41.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:41.409+0000 7f6bacc00640 1 -- 172.21.15.131:0/2650346572 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f6ba0058b90 con 0x7f6bb006e860 2024-05-26T22:55:41.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:41.606+0000 7f6bb6c00640 1 -- 172.21.15.131:0/2650346572 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f6bb00ae5d0 con 0x7f6bb006e860 2024-05-26T22:55:42.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:42.338+0000 7f6bacc00640 1 -- 172.21.15.131:0/2650346572 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v97) ==== 181+0+0 (secure 0 0 0) 0x7f6ba0058490 con 0x7f6bb006e860 2024-05-26T22:55:42.339 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_metadata' removed 2024-05-26T22:55:42.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:42.344+0000 7f6bb6c00640 1 -- 172.21.15.131:0/2650346572 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f6b8c06be30 msgr2=0x7f6b8c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:42.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:42.344+0000 7f6bb6c00640 1 --2- 172.21.15.131:0/2650346572 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f6b8c06be30 0x7f6b8c06e2f0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f6b98006fd0 tx=0x7f6b98008040 comp rx=0 tx=0).stop 2024-05-26T22:55:42.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:42.344+0000 7f6bb6c00640 1 -- 172.21.15.131:0/2650346572 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6bb006e860 msgr2=0x7f6bb00b1240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:42.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:42.344+0000 7f6bb6c00640 1 --2- 172.21.15.131:0/2650346572 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6bb006e860 0x7f6bb00b1240 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f6ba000c4d0 tx=0x7f6ba000c990 comp rx=0 tx=0).stop 2024-05-26T22:55:42.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:42.346+0000 7f6bb6c00640 1 -- 172.21.15.131:0/2650346572 shutdown_connections 2024-05-26T22:55:42.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:42.346+0000 7f6bb6c00640 1 --2- 172.21.15.131:0/2650346572 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6bb00d0a70 0x7f6bb0152c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:42.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:42.346+0000 7f6bb6c00640 1 --2- 172.21.15.131:0/2650346572 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f6b8c06be30 0x7f6b8c06e2f0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:42.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:42.346+0000 7f6bb6c00640 1 --2- 172.21.15.131:0/2650346572 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6bb00cf8f0 0x7f6bb00cb250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:42.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:42.346+0000 7f6bb6c00640 1 --2- 172.21.15.131:0/2650346572 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6bb006e860 0x7f6bb00b1240 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:42.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:42.346+0000 7f6bb6c00640 1 -- 172.21.15.131:0/2650346572 >> 172.21.15.131:0/2650346572 conn(0x7f6bb00b2110 msgr2=0x7f6bb00ae9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:42.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:42.347+0000 7f6bb6c00640 1 -- 172.21.15.131:0/2650346572 shutdown_connections 2024-05-26T22:55:42.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:42.347+0000 7f6bb6c00640 1 -- 172.21.15.131:0/2650346572 wait complete. 2024-05-26T22:55:42.374 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-26T22:55:42.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:42.610+0000 7fcf67c00640 1 Processor -- start 2024-05-26T22:55:42.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:42.610+0000 7fcf67c00640 1 -- start start 2024-05-26T22:55:42.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:42.612+0000 7fcf67c00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fcf600d0a70 0x7fcf60072250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:42.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:42.612+0000 7fcf67c00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fcf6006e860 0x7fcf6006ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:42.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:42.612+0000 7fcf67c00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf6006f210 con 0x7fcf600cf8f0 2024-05-26T22:55:42.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:42.612+0000 7fcf67c00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fcf600ccb70 con 0x7fcf6006e860 2024-05-26T22:55:42.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:42.612+0000 7fcf67c00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fcf600cce50 con 0x7fcf600d0a70 2024-05-26T22:55:42.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:42.612+0000 7fcf65200640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fcf600cf8f0 0x7fcf600d0360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:42112/0 (socket says 172.21.15.131:42112) 2024-05-26T22:55:42.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:42.612+0000 7fcf65200640 1 -- 172.21.15.131:0/943411943 learned_addr learned my addr 172.21.15.131:0/943411943 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:55:42.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:42.612+0000 7fcf65c00640 1 --2- 172.21.15.131:0/943411943 >> v2:172.21.15.131:3300/0 conn(0x7fcf6006e860 0x7fcf6006ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:42.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:42.613+0000 7fcf65c00640 1 -- 172.21.15.131:0/943411943 >> v2:172.21.15.187:3300/0 conn(0x7fcf600d0a70 msgr2=0x7fcf60072250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:42.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:42.613+0000 7fcf65c00640 1 --2- 172.21.15.131:0/943411943 >> v2:172.21.15.187:3300/0 conn(0x7fcf600d0a70 0x7fcf60072250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:42.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:42.613+0000 7fcf65c00640 1 -- 172.21.15.131:0/943411943 >> v1:172.21.15.131:6789/0 conn(0x7fcf600cf8f0 legacy=0x7fcf600d0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:42.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:42.613+0000 7fcf65c00640 1 -- 172.21.15.131:0/943411943 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf600cd130 con 0x7fcf6006e860 2024-05-26T22:55:42.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:42.613+0000 7fcf5f400640 1 -- 172.21.15.131:0/943411943 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1806545665 0 0) 0x7fcf6006f210 con 0x7fcf600cf8f0 2024-05-26T22:55:42.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:42.614+0000 7fcf65c00640 1 --2- 172.21.15.131:0/943411943 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf6006e860 0x7fcf6006ecd0 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7fcf6006c3d0 tx=0x7fcf5802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=394cbd4455098160 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:42.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:42.614+0000 7fcf5f400640 1 -- 172.21.15.131:0/943411943 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf58007740 con 0x7fcf6006e860 2024-05-26T22:55:42.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:42.614+0000 7fcf5f400640 1 -- 172.21.15.131:0/943411943 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf580078e0 con 0x7fcf6006e860 2024-05-26T22:55:42.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:42.614+0000 7fcf5f400640 1 -- 172.21.15.131:0/943411943 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf58007cb0 con 0x7fcf6006e860 2024-05-26T22:55:42.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:42.614+0000 7fcf67c00640 1 -- 172.21.15.131:0/943411943 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf6006e860 msgr2=0x7fcf6006ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:42.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:42.614+0000 7fcf67c00640 1 --2- 172.21.15.131:0/943411943 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf6006e860 0x7fcf6006ecd0 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7fcf6006c3d0 tx=0x7fcf5802a680 comp rx=0 tx=0).stop 2024-05-26T22:55:42.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:42.615+0000 7fcf67c00640 1 -- 172.21.15.131:0/943411943 shutdown_connections 2024-05-26T22:55:42.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:42.615+0000 7fcf67c00640 1 --2- 172.21.15.131:0/943411943 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf6006e860 0x7fcf6006ecd0 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:42.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:42.615+0000 7fcf67c00640 1 --2- 172.21.15.131:0/943411943 >> v2:172.21.15.187:3300/0 conn(0x7fcf600d0a70 0x7fcf60072250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:42.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:42.615+0000 7fcf67c00640 1 -- 172.21.15.131:0/943411943 >> 172.21.15.131:0/943411943 conn(0x7fcf600b2110 msgr2=0x7fcf600b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:42.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:42.615+0000 7fcf67c00640 1 -- 172.21.15.131:0/943411943 shutdown_connections 2024-05-26T22:55:42.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:42.615+0000 7fcf67c00640 1 -- 172.21.15.131:0/943411943 wait complete. 2024-05-26T22:55:42.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:42.616+0000 7fcf67c00640 1 Processor -- start 2024-05-26T22:55:42.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:42.616+0000 7fcf67c00640 1 -- start start 2024-05-26T22:55:42.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:42.617+0000 7fcf67c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf6006e860 0x7fcf6015ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:42.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:42.617+0000 7fcf67c00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcf600cf8f0 0x7fcf6015ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:42.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:42.617+0000 7fcf67c00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcf600d0a70 0x7fcf601561d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:42.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:42.617+0000 7fcf67c00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fcf6015ffd0 con 0x7fcf6006e860 2024-05-26T22:55:42.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:42.617+0000 7fcf67c00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fcf600cfcd0 con 0x7fcf600d0a70 2024-05-26T22:55:42.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:42.617+0000 7fcf67c00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fcf60160150 con 0x7fcf600cf8f0 2024-05-26T22:55:42.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:42.617+0000 7fcf65200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf6006e860 0x7fcf6015ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:42.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:42.618+0000 7fcf65200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf6006e860 0x7fcf6015ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:36554/0 (socket says 172.21.15.131:36554) 2024-05-26T22:55:42.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:42.618+0000 7fcf65200640 1 -- 172.21.15.131:0/3477904292 learned_addr learned my addr 172.21.15.131:0/3477904292 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:55:42.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:42.618+0000 7fcf65200640 1 -- 172.21.15.131:0/3477904292 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcf600cf8f0 msgr2=0x7fcf6015ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:42.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:42.618+0000 7fcf5fe00640 1 --2- 172.21.15.131:0/3477904292 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcf600cf8f0 0x7fcf6015ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:42.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:42.618+0000 7fcf65c00640 1 --2- 172.21.15.131:0/3477904292 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcf600d0a70 0x7fcf601561d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:42.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:42.618+0000 7fcf65200640 1 --2- 172.21.15.131:0/3477904292 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcf600cf8f0 0x7fcf6015ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:42.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:42.618+0000 7fcf65200640 1 -- 172.21.15.131:0/3477904292 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcf600d0a70 msgr2=0x7fcf601561d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:42.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:42.618+0000 7fcf65200640 1 --2- 172.21.15.131:0/3477904292 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcf600d0a70 0x7fcf601561d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:42.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:42.618+0000 7fcf65200640 1 -- 172.21.15.131:0/3477904292 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf6015f620 con 0x7fcf6006e860 2024-05-26T22:55:42.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:42.618+0000 7fcf65200640 1 --2- 172.21.15.131:0/3477904292 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf6006e860 0x7fcf6015ad80 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7fcf5000c4d0 tx=0x7fcf5000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:42.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:42.619+0000 7fcf5cc00640 1 -- 172.21.15.131:0/3477904292 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf50018070 con 0x7fcf6006e860 2024-05-26T22:55:42.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:42.619+0000 7fcf5cc00640 1 -- 172.21.15.131:0/3477904292 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf50007500 con 0x7fcf6006e860 2024-05-26T22:55:42.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:42.619+0000 7fcf5cc00640 1 -- 172.21.15.131:0/3477904292 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf50004a20 con 0x7fcf6006e860 2024-05-26T22:55:42.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:42.619+0000 7fcf67c00640 1 -- 172.21.15.131:0/3477904292 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf6015a2a0 con 0x7fcf6006e860 2024-05-26T22:55:42.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:42.619+0000 7fcf67c00640 1 -- 172.21.15.131:0/3477904292 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf6015a860 con 0x7fcf6006e860 2024-05-26T22:55:42.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:42.620+0000 7fcf67c00640 1 -- 172.21.15.131:0/3477904292 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf60068440 con 0x7fcf6006e860 2024-05-26T22:55:42.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:42.621+0000 7fcf5cc00640 1 -- 172.21.15.131:0/3477904292 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcf50004bc0 con 0x7fcf6006e860 2024-05-26T22:55:42.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:42.622+0000 7fcf5cc00640 1 --2- 172.21.15.131:0/3477904292 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fcf3c06be30 0x7fcf3c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:42.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:42.622+0000 7fcf5cc00640 1 -- 172.21.15.131:0/3477904292 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5529+0+0 (secure 0 0 0) 0x7fcf5008a120 con 0x7fcf6006e860 2024-05-26T22:55:42.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:42.622+0000 7fcf5fe00640 1 --2- 172.21.15.131:0/3477904292 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fcf3c06be30 0x7fcf3c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:42.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:42.623+0000 7fcf5fe00640 1 --2- 172.21.15.131:0/3477904292 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fcf3c06be30 0x7fcf3c06e2f0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fcf6015e230 tx=0x7fcf48008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:42.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:42.627+0000 7fcf5cc00640 1 -- 172.21.15.131:0/3477904292 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fcf50058ae0 con 0x7fcf6006e860 2024-05-26T22:55:42.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:42.823+0000 7fcf67c00640 1 -- 172.21.15.131:0/3477904292 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fcf600d0610 con 0x7fcf6006e860 2024-05-26T22:55:43.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:43.342+0000 7fcf5cc00640 1 -- 172.21.15.131:0/3477904292 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v98) ==== 169+0+0 (secure 0 0 0) 0x7fcf500583e0 con 0x7fcf6006e860 2024-05-26T22:55:43.342 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_data' removed 2024-05-26T22:55:43.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:43.348+0000 7fcf67c00640 1 -- 172.21.15.131:0/3477904292 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fcf3c06be30 msgr2=0x7fcf3c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:43.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:43.348+0000 7fcf67c00640 1 --2- 172.21.15.131:0/3477904292 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fcf3c06be30 0x7fcf3c06e2f0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fcf6015e230 tx=0x7fcf48008040 comp rx=0 tx=0).stop 2024-05-26T22:55:43.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:43.348+0000 7fcf67c00640 1 -- 172.21.15.131:0/3477904292 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf6006e860 msgr2=0x7fcf6015ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:43.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:43.348+0000 7fcf67c00640 1 --2- 172.21.15.131:0/3477904292 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf6006e860 0x7fcf6015ad80 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7fcf5000c4d0 tx=0x7fcf5000c990 comp rx=0 tx=0).stop 2024-05-26T22:55:43.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:43.350+0000 7fcf67c00640 1 -- 172.21.15.131:0/3477904292 shutdown_connections 2024-05-26T22:55:43.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:43.350+0000 7fcf67c00640 1 --2- 172.21.15.131:0/3477904292 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcf600cf8f0 0x7fcf6015ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:43.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:43.350+0000 7fcf67c00640 1 --2- 172.21.15.131:0/3477904292 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fcf3c06be30 0x7fcf3c06e2f0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:43.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:43.350+0000 7fcf67c00640 1 --2- 172.21.15.131:0/3477904292 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcf600d0a70 0x7fcf601561d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:43.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:43.350+0000 7fcf67c00640 1 --2- 172.21.15.131:0/3477904292 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf6006e860 0x7fcf6015ad80 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:43.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:43.350+0000 7fcf67c00640 1 -- 172.21.15.131:0/3477904292 >> 172.21.15.131:0/3477904292 conn(0x7fcf600b2110 msgr2=0x7fcf600acfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:43.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:43.352+0000 7fcf67c00640 1 -- 172.21.15.131:0/3477904292 shutdown_connections 2024-05-26T22:55:43.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:43.353+0000 7fcf67c00640 1 -- 172.21.15.131:0/3477904292 wait complete. 2024-05-26T22:55:43.380 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_export_ephemeral_random_max 2024-05-26T22:55:43.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:43.564+0000 7fe63fe00640 1 Processor -- start 2024-05-26T22:55:43.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:43.564+0000 7fe63fe00640 1 -- start start 2024-05-26T22:55:43.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:43.565+0000 7fe63fe00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fe64015c220 0x7fe64006e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:43.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:43.566+0000 7fe63fe00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe64006ee60 con 0x7fe64006e9f0 2024-05-26T22:55:43.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:43.566+0000 7fe63fe00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6401558d0 con 0x7fe6401554f0 2024-05-26T22:55:43.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:43.566+0000 7fe63fe00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fe6400729b0 con 0x7fe64015c220 2024-05-26T22:55:43.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:43.566+0000 7fe63f400640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fe64006e9f0 0x7fe64015d230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:42114/0 (socket says 172.21.15.131:42114) 2024-05-26T22:55:43.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:43.566+0000 7fe63f400640 1 -- 172.21.15.131:0/3208836830 learned_addr learned my addr 172.21.15.131:0/3208836830 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:55:43.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:43.566+0000 7fe636000640 1 --2- 172.21.15.131:0/3208836830 >> v2:172.21.15.187:3300/0 conn(0x7fe64015c220 0x7fe64006e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:43.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:43.567+0000 7fe63e000640 1 -- 172.21.15.131:0/3208836830 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4052820219 0 0) 0x7fe64006ee60 con 0x7fe64006e9f0 2024-05-26T22:55:43.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:43.567+0000 7fe63e000640 1 -- 172.21.15.131:0/3208836830 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe624003520 con 0x7fe64006e9f0 2024-05-26T22:55:43.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:43.567+0000 7fe636000640 1 -- 172.21.15.131:0/3208836830 >> v1:172.21.15.187:6789/0 conn(0x7fe6401554f0 legacy=0x7fe640071f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:43.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:43.567+0000 7fe636000640 1 -- 172.21.15.131:0/3208836830 >> v1:172.21.15.131:6789/0 conn(0x7fe64006e9f0 legacy=0x7fe64015d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:43.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:43.567+0000 7fe636000640 1 -- 172.21.15.131:0/3208836830 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe64015c750 con 0x7fe64015c220 2024-05-26T22:55:43.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:43.568+0000 7fe636000640 1 --2- 172.21.15.131:0/3208836830 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe64015c220 0x7fe64006e4b0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fe6280097d0 tx=0x7fe62802a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f280c4d07df360a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:43.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:43.569+0000 7fe63e000640 1 -- 172.21.15.131:0/3208836830 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe628037070 con 0x7fe64015c220 2024-05-26T22:55:43.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:43.569+0000 7fe63e000640 1 -- 172.21.15.131:0/3208836830 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe628032440 con 0x7fe64015c220 2024-05-26T22:55:43.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:43.569+0000 7fe63e000640 1 -- 172.21.15.131:0/3208836830 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe628032810 con 0x7fe64015c220 2024-05-26T22:55:43.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:43.569+0000 7fe63fe00640 1 -- 172.21.15.131:0/3208836830 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe64015c220 msgr2=0x7fe64006e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:43.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:43.569+0000 7fe63fe00640 1 --2- 172.21.15.131:0/3208836830 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe64015c220 0x7fe64006e4b0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fe6280097d0 tx=0x7fe62802a210 comp rx=0 tx=0).stop 2024-05-26T22:55:43.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:43.569+0000 7fe63fe00640 1 -- 172.21.15.131:0/3208836830 shutdown_connections 2024-05-26T22:55:43.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:43.569+0000 7fe63fe00640 1 --2- 172.21.15.131:0/3208836830 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe64015c220 0x7fe64006e4b0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:43.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:43.569+0000 7fe63fe00640 1 -- 172.21.15.131:0/3208836830 >> 172.21.15.131:0/3208836830 conn(0x7fe6400cc7f0 msgr2=0x7fe6400ccbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:43.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:43.569+0000 7fe63fe00640 1 -- 172.21.15.131:0/3208836830 shutdown_connections 2024-05-26T22:55:43.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:43.570+0000 7fe63fe00640 1 -- 172.21.15.131:0/3208836830 wait complete. 2024-05-26T22:55:43.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:43.571+0000 7fe63fe00640 1 Processor -- start 2024-05-26T22:55:43.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:43.571+0000 7fe63fe00640 1 -- start start 2024-05-26T22:55:43.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:43.572+0000 7fe63fe00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe64006e9f0 0x7fe6401f1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:43.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:43.572+0000 7fe63fe00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe6401554f0 0x7fe640146600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:43.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:43.572+0000 7fe63fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe64015c220 0x7fe640157ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:43.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:43.572+0000 7fe63fe00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe640072670 con 0x7fe64015c220 2024-05-26T22:55:43.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:43.572+0000 7fe63fe00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe64015cd10 con 0x7fe6401554f0 2024-05-26T22:55:43.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:43.572+0000 7fe63fe00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe64015d940 con 0x7fe64006e9f0 2024-05-26T22:55:43.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:43.573+0000 7fe63f400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe64015c220 0x7fe640157ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:43.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:43.573+0000 7fe63f400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe64015c220 0x7fe640157ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:36570/0 (socket says 172.21.15.131:36570) 2024-05-26T22:55:43.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:43.573+0000 7fe63f400640 1 -- 172.21.15.131:0/1056090632 learned_addr learned my addr 172.21.15.131:0/1056090632 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:55:43.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:43.573+0000 7fe636000640 1 --2- 172.21.15.131:0/1056090632 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe6401554f0 0x7fe640146600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:43.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:43.573+0000 7fe63f400640 1 -- 172.21.15.131:0/1056090632 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe64006e9f0 msgr2=0x7fe6401f1dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:43.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:43.573+0000 7fe63ea00640 1 --2- 172.21.15.131:0/1056090632 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe64006e9f0 0x7fe6401f1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:43.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:43.573+0000 7fe63f400640 1 --2- 172.21.15.131:0/1056090632 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe64006e9f0 0x7fe6401f1dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:43.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:43.573+0000 7fe63f400640 1 -- 172.21.15.131:0/1056090632 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe6401554f0 msgr2=0x7fe640146600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:43.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:43.573+0000 7fe63f400640 1 --2- 172.21.15.131:0/1056090632 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe6401554f0 0x7fe640146600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:43.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:43.573+0000 7fe63f400640 1 -- 172.21.15.131:0/1056090632 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe640153be0 con 0x7fe64015c220 2024-05-26T22:55:43.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:43.573+0000 7fe636000640 1 --2- 172.21.15.131:0/1056090632 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe6401554f0 0x7fe640146600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:55:43.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:43.574+0000 7fe63f400640 1 --2- 172.21.15.131:0/1056090632 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe64015c220 0x7fe640157ba0 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7fe63800cb40 tx=0x7fe63800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:43.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:43.574+0000 7fe637400640 1 -- 172.21.15.131:0/1056090632 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe638017070 con 0x7fe64015c220 2024-05-26T22:55:43.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:43.574+0000 7fe637400640 1 -- 172.21.15.131:0/1056090632 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe63800aca0 con 0x7fe64015c220 2024-05-26T22:55:43.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:43.574+0000 7fe637400640 1 -- 172.21.15.131:0/1056090632 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe638012620 con 0x7fe64015c220 2024-05-26T22:55:43.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:43.574+0000 7fe63fe00640 1 -- 172.21.15.131:0/1056090632 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe64015a9c0 con 0x7fe64015c220 2024-05-26T22:55:43.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:43.574+0000 7fe63fe00640 1 -- 172.21.15.131:0/1056090632 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6401e5d60 con 0x7fe64015c220 2024-05-26T22:55:43.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:43.575+0000 7fe63fe00640 1 -- 172.21.15.131:0/1056090632 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6400694e0 con 0x7fe64015c220 2024-05-26T22:55:43.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:43.576+0000 7fe637400640 1 -- 172.21.15.131:0/1056090632 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe63801a070 con 0x7fe64015c220 2024-05-26T22:55:43.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:43.577+0000 7fe637400640 1 --2- 172.21.15.131:0/1056090632 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fe61c06be30 0x7fe61c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:43.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:43.577+0000 7fe637400640 1 -- 172.21.15.131:0/1056090632 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5115+0+0 (secure 0 0 0) 0x7fe63808abb0 con 0x7fe64015c220 2024-05-26T22:55:43.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:43.577+0000 7fe63ea00640 1 --2- 172.21.15.131:0/1056090632 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fe61c06be30 0x7fe61c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:43.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:43.578+0000 7fe63ea00640 1 --2- 172.21.15.131:0/1056090632 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fe61c06be30 0x7fe61c06e2f0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fe630003d90 tx=0x7fe630009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:43.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:43.582+0000 7fe637400640 1 -- 172.21.15.131:0/1056090632 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fe638059680 con 0x7fe64015c220 2024-05-26T22:55:43.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:43.791+0000 7fe63fe00640 1 -- 172.21.15.131:0/1056090632 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_export_ephemeral_random_max"} v 0) -- 0x7fe6400ce430 con 0x7fe64015c220 2024-05-26T22:55:43.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:43.796+0000 7fe637400640 1 -- 172.21.15.131:0/1056090632 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_export_ephemeral_random_max"}]=0 v55) ==== 114+0+0 (secure 0 0 0) 0x7fe638058f80 con 0x7fe64015c220 2024-05-26T22:55:43.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:43.799+0000 7fe63fe00640 1 -- 172.21.15.131:0/1056090632 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fe61c06be30 msgr2=0x7fe61c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:43.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:43.799+0000 7fe63fe00640 1 --2- 172.21.15.131:0/1056090632 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fe61c06be30 0x7fe61c06e2f0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fe630003d90 tx=0x7fe630009040 comp rx=0 tx=0).stop 2024-05-26T22:55:43.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:43.799+0000 7fe63fe00640 1 -- 172.21.15.131:0/1056090632 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe64015c220 msgr2=0x7fe640157ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:43.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:43.799+0000 7fe63fe00640 1 --2- 172.21.15.131:0/1056090632 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe64015c220 0x7fe640157ba0 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7fe63800cb40 tx=0x7fe63800a060 comp rx=0 tx=0).stop 2024-05-26T22:55:43.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:43.799+0000 7fe63fe00640 1 -- 172.21.15.131:0/1056090632 shutdown_connections 2024-05-26T22:55:43.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:43.799+0000 7fe63fe00640 1 --2- 172.21.15.131:0/1056090632 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fe61c06be30 0x7fe61c06e2f0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:43.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:43.799+0000 7fe63fe00640 1 --2- 172.21.15.131:0/1056090632 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe64015c220 0x7fe640157ba0 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:43.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:43.799+0000 7fe63fe00640 1 --2- 172.21.15.131:0/1056090632 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe6401554f0 0x7fe640146600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:43.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:43.799+0000 7fe63fe00640 1 --2- 172.21.15.131:0/1056090632 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe64006e9f0 0x7fe6401f1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:43.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:43.799+0000 7fe63fe00640 1 -- 172.21.15.131:0/1056090632 >> 172.21.15.131:0/1056090632 conn(0x7fe6400cc7f0 msgr2=0x7fe64015d230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:43.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:43.799+0000 7fe63fe00640 1 -- 172.21.15.131:0/1056090632 shutdown_connections 2024-05-26T22:55:43.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:43.799+0000 7fe63fe00640 1 -- 172.21.15.131:0/1056090632 wait complete. 2024-05-26T22:55:43.818 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_export_ephemeral_random 2024-05-26T22:55:44.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.013+0000 7ff6f2e00640 1 Processor -- start 2024-05-26T22:55:44.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.014+0000 7ff6f2e00640 1 -- start start 2024-05-26T22:55:44.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.015+0000 7ff6f2e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff6ec0cf4c0 0x7ff6ec143f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:44.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.015+0000 7ff6f2e00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6ec15f140 con 0x7ff6ec1556e0 2024-05-26T22:55:44.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.015+0000 7ff6f2e00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6ec15f420 con 0x7ff6ec15ec80 2024-05-26T22:55:44.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.015+0000 7ff6f2e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff6ec06eb00 con 0x7ff6ec0cf4c0 2024-05-26T22:55:44.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.015+0000 7ff6f1a00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7ff6ec1556e0 0x7ff6ec15c0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:42120/0 (socket says 172.21.15.131:42120) 2024-05-26T22:55:44.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.015+0000 7ff6f1a00640 1 -- 172.21.15.131:0/3889166186 learned_addr learned my addr 172.21.15.131:0/3889166186 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:55:44.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.016+0000 7ff6ebe00640 1 -- 172.21.15.131:0/3889166186 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2426047238 0 0) 0x7ff6ec15f140 con 0x7ff6ec1556e0 2024-05-26T22:55:44.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.016+0000 7ff6ebe00640 1 -- 172.21.15.131:0/3889166186 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff6d4003520 con 0x7ff6ec1556e0 2024-05-26T22:55:44.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.016+0000 7ff6f2400640 1 --2- 172.21.15.131:0/3889166186 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff6ec0cf4c0 0x7ff6ec143f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:44.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.016+0000 7ff6ebe00640 1 -- 172.21.15.131:0/3889166186 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3834494476 0 0) 0x7ff6d4003520 con 0x7ff6ec1556e0 2024-05-26T22:55:44.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.016+0000 7ff6ebe00640 1 -- 172.21.15.131:0/3889166186 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff6ec0cf4c0 msgr2=0x7ff6ec143f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:44.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.016+0000 7ff6ebe00640 1 --2- 172.21.15.131:0/3889166186 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff6ec0cf4c0 0x7ff6ec143f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:44.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.016+0000 7ff6ebe00640 1 -- 172.21.15.131:0/3889166186 >> v1:172.21.15.187:6789/0 conn(0x7ff6ec15ec80 legacy=0x7ff6ec0cedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:44.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.016+0000 7ff6ebe00640 1 -- 172.21.15.131:0/3889166186 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6ec15f700 con 0x7ff6ec1556e0 2024-05-26T22:55:44.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.016+0000 7ff6ebe00640 1 -- 172.21.15.131:0/3889166186 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7ff6e0001690 con 0x7ff6ec1556e0 2024-05-26T22:55:44.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.016+0000 7ff6ebe00640 1 -- 172.21.15.131:0/3889166186 >> v1:172.21.15.131:6789/0 conn(0x7ff6ec1556e0 legacy=0x7ff6ec15c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:44.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.016+0000 7ff6ebe00640 1 --2- 172.21.15.131:0/3889166186 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff6d40039e0 0x7ff6d4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:44.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.017+0000 7ff6ebe00640 1 --2- 172.21.15.131:0/3889166186 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6d4004800 0x7ff6d4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:44.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.017+0000 7ff6ebe00640 1 --2- 172.21.15.131:0/3889166186 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff6ec0cf4c0 0x7ff6d4005250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:44.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.017+0000 7ff6ebe00640 1 -- 172.21.15.131:0/3889166186 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff6d400bb80 con 0x7ff6d40039e0 2024-05-26T22:55:44.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.017+0000 7ff6ebe00640 1 -- 172.21.15.131:0/3889166186 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff6d400bd30 con 0x7ff6d4004800 2024-05-26T22:55:44.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.017+0000 7ff6ebe00640 1 -- 172.21.15.131:0/3889166186 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff6d400bf30 con 0x7ff6ec0cf4c0 2024-05-26T22:55:44.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.017+0000 7ff6f1a00640 1 --2- 172.21.15.131:0/3889166186 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff6d40039e0 0x7ff6d4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:44.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.017+0000 7ff6f1000640 1 --2- 172.21.15.131:0/3889166186 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6d4004800 0x7ff6d4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:44.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.017+0000 7ff6f2400640 1 --2- 172.21.15.131:0/3889166186 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff6ec0cf4c0 0x7ff6d4005250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:44.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.017+0000 7ff6f1a00640 1 -- 172.21.15.131:0/3889166186 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff6ec0cf4c0 msgr2=0x7ff6d4005250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:44.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.017+0000 7ff6f1a00640 1 --2- 172.21.15.131:0/3889166186 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff6ec0cf4c0 0x7ff6d4005250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:44.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.017+0000 7ff6f1a00640 1 -- 172.21.15.131:0/3889166186 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6d4004800 msgr2=0x7ff6d4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:44.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.017+0000 7ff6f1a00640 1 --2- 172.21.15.131:0/3889166186 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6d4004800 0x7ff6d4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:44.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.017+0000 7ff6f1a00640 1 -- 172.21.15.131:0/3889166186 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6d4003520 con 0x7ff6d40039e0 2024-05-26T22:55:44.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.017+0000 7ff6f1000640 1 --2- 172.21.15.131:0/3889166186 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6d4004800 0x7ff6d4004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:55:44.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.018+0000 7ff6f1a00640 1 --2- 172.21.15.131:0/3889166186 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff6d40039e0 0x7ff6d4003e00 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec06e700 tx=0x7ff6d4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24944468f1de2a2c server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:44.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.019+0000 7ff6ebe00640 1 -- 172.21.15.131:0/3889166186 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6e0007c40 con 0x7ff6d40039e0 2024-05-26T22:55:44.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.019+0000 7ff6ebe00640 1 -- 172.21.15.131:0/3889166186 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6e0005ad0 con 0x7ff6d40039e0 2024-05-26T22:55:44.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.019+0000 7ff6ebe00640 1 -- 172.21.15.131:0/3889166186 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6e0007070 con 0x7ff6d40039e0 2024-05-26T22:55:44.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.019+0000 7ff6f2e00640 1 -- 172.21.15.131:0/3889166186 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff6d40039e0 msgr2=0x7ff6d4003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:44.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.019+0000 7ff6f2e00640 1 --2- 172.21.15.131:0/3889166186 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff6d40039e0 0x7ff6d4003e00 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec06e700 tx=0x7ff6d4005830 comp rx=0 tx=0).stop 2024-05-26T22:55:44.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.019+0000 7ff6f2400640 1 -- 172.21.15.131:0/3889166186 reap_dead start 2024-05-26T22:55:44.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.019+0000 7ff6f2e00640 1 -- 172.21.15.131:0/3889166186 shutdown_connections 2024-05-26T22:55:44.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.019+0000 7ff6f2e00640 1 -- 172.21.15.131:0/3889166186 >> 172.21.15.131:0/3889166186 conn(0x7ff6ec0cc860 msgr2=0x7ff6ec0ccc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:44.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.019+0000 7ff6f2e00640 1 -- 172.21.15.131:0/3889166186 shutdown_connections 2024-05-26T22:55:44.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.020+0000 7ff6f2e00640 1 -- 172.21.15.131:0/3889166186 wait complete. 2024-05-26T22:55:44.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.021+0000 7ff6f2e00640 1 Processor -- start 2024-05-26T22:55:44.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.021+0000 7ff6f2e00640 1 -- start start 2024-05-26T22:55:44.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.022+0000 7ff6f2e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff6d40039e0 0x7ff6ec14c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:44.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.022+0000 7ff6f2e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff6ec15a7f0 0x7ff6ec15ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:44.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.022+0000 7ff6f2e00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6ec158640 0x7ff6ec158ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:44.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.022+0000 7ff6f2e00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff6ec072510 con 0x7ff6ec15a7f0 2024-05-26T22:55:44.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.022+0000 7ff6f2e00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff6ec155ac0 con 0x7ff6ec158640 2024-05-26T22:55:44.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.022+0000 7ff6f2e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff6ec072690 con 0x7ff6d40039e0 2024-05-26T22:55:44.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.023+0000 7ff6f1000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff6ec15a7f0 0x7ff6ec15ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:44.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.023+0000 7ff6f1000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff6ec15a7f0 0x7ff6ec15ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:36586/0 (socket says 172.21.15.131:36586) 2024-05-26T22:55:44.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.023+0000 7ff6f1000640 1 -- 172.21.15.131:0/4193763640 learned_addr learned my addr 172.21.15.131:0/4193763640 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:55:44.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.023+0000 7ff6f2400640 1 --2- 172.21.15.131:0/4193763640 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6ec158640 0x7ff6ec158ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:44.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.023+0000 7ff6f1a00640 1 --2- 172.21.15.131:0/4193763640 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff6d40039e0 0x7ff6ec14c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:44.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.023+0000 7ff6f1000640 1 -- 172.21.15.131:0/4193763640 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff6d40039e0 msgr2=0x7ff6ec14c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:44.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.023+0000 7ff6f1000640 1 --2- 172.21.15.131:0/4193763640 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff6d40039e0 0x7ff6ec14c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:44.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.023+0000 7ff6f1000640 1 -- 172.21.15.131:0/4193763640 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6ec158640 msgr2=0x7ff6ec158ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:44.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.023+0000 7ff6f1000640 1 --2- 172.21.15.131:0/4193763640 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6ec158640 0x7ff6ec158ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:44.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.023+0000 7ff6f1000640 1 -- 172.21.15.131:0/4193763640 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6ec14ce40 con 0x7ff6ec15a7f0 2024-05-26T22:55:44.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.023+0000 7ff6f2400640 1 --2- 172.21.15.131:0/4193763640 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6ec158640 0x7ff6ec158ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:55:44.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.023+0000 7ff6f1000640 1 --2- 172.21.15.131:0/4193763640 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff6ec15a7f0 0x7ff6ec15ac70 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7ff6d800d570 tx=0x7ff6d800da30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:44.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.024+0000 7ff6e9600640 1 -- 172.21.15.131:0/4193763640 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6d8005480 con 0x7ff6ec15a7f0 2024-05-26T22:55:44.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.024+0000 7ff6e9600640 1 -- 172.21.15.131:0/4193763640 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6d8017070 con 0x7ff6ec15a7f0 2024-05-26T22:55:44.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.024+0000 7ff6e9600640 1 -- 172.21.15.131:0/4193763640 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6d8003830 con 0x7ff6ec15a7f0 2024-05-26T22:55:44.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.024+0000 7ff6f2e00640 1 -- 172.21.15.131:0/4193763640 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6ec0ae660 con 0x7ff6ec15a7f0 2024-05-26T22:55:44.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.024+0000 7ff6f2e00640 1 -- 172.21.15.131:0/4193763640 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6ec0aec20 con 0x7ff6ec15a7f0 2024-05-26T22:55:44.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.025+0000 7ff6f2e00640 1 -- 172.21.15.131:0/4193763640 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6ec0694e0 con 0x7ff6ec15a7f0 2024-05-26T22:55:44.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.026+0000 7ff6e9600640 1 -- 172.21.15.131:0/4193763640 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff6d80039d0 con 0x7ff6ec15a7f0 2024-05-26T22:55:44.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.027+0000 7ff6e9600640 1 --2- 172.21.15.131:0/4193763640 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ff6cc06be60 0x7ff6cc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:44.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.027+0000 7ff6e9600640 1 -- 172.21.15.131:0/4193763640 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5115+0+0 (secure 0 0 0) 0x7ff6d808aa20 con 0x7ff6ec15a7f0 2024-05-26T22:55:44.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.027+0000 7ff6f1a00640 1 --2- 172.21.15.131:0/4193763640 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ff6cc06be60 0x7ff6cc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:44.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.028+0000 7ff6f1a00640 1 --2- 172.21.15.131:0/4193763640 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ff6cc06be60 0x7ff6cc06e320 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7ff6e0032000 tx=0x7ff6e0009030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:44.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.032+0000 7ff6e9600640 1 -- 172.21.15.131:0/4193763640 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7ff6d8059620 con 0x7ff6ec15a7f0 2024-05-26T22:55:44.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.232+0000 7ff6f2e00640 1 -- 172.21.15.131:0/4193763640 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_export_ephemeral_random"} v 0) -- 0x7ff6ec15ca00 con 0x7ff6ec15a7f0 2024-05-26T22:55:44.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.237+0000 7ff6e9600640 1 -- 172.21.15.131:0/4193763640 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_export_ephemeral_random"}]=0 v56) ==== 110+0+0 (secure 0 0 0) 0x7ff6d8058f20 con 0x7ff6ec15a7f0 2024-05-26T22:55:44.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.240+0000 7ff6f2e00640 1 -- 172.21.15.131:0/4193763640 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ff6cc06be60 msgr2=0x7ff6cc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:44.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.240+0000 7ff6f2e00640 1 --2- 172.21.15.131:0/4193763640 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ff6cc06be60 0x7ff6cc06e320 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7ff6e0032000 tx=0x7ff6e0009030 comp rx=0 tx=0).stop 2024-05-26T22:55:44.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.240+0000 7ff6f2e00640 1 -- 172.21.15.131:0/4193763640 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff6ec15a7f0 msgr2=0x7ff6ec15ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:44.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.240+0000 7ff6f2e00640 1 --2- 172.21.15.131:0/4193763640 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff6ec15a7f0 0x7ff6ec15ac70 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7ff6d800d570 tx=0x7ff6d800da30 comp rx=0 tx=0).stop 2024-05-26T22:55:44.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.240+0000 7ff6f2e00640 1 -- 172.21.15.131:0/4193763640 shutdown_connections 2024-05-26T22:55:44.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.240+0000 7ff6f2e00640 1 --2- 172.21.15.131:0/4193763640 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ff6cc06be60 0x7ff6cc06e320 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:44.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.240+0000 7ff6f2e00640 1 --2- 172.21.15.131:0/4193763640 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6ec158640 0x7ff6ec158ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:44.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.240+0000 7ff6f2e00640 1 --2- 172.21.15.131:0/4193763640 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff6ec15a7f0 0x7ff6ec15ac70 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:44.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.240+0000 7ff6f2e00640 1 --2- 172.21.15.131:0/4193763640 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff6d40039e0 0x7ff6ec14c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:44.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.240+0000 7ff6f2e00640 1 -- 172.21.15.131:0/4193763640 >> 172.21.15.131:0/4193763640 conn(0x7ff6ec0cc860 msgr2=0x7ff6ec06ed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:44.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.240+0000 7ff6f2e00640 1 -- 172.21.15.131:0/4193763640 shutdown_connections 2024-05-26T22:55:44.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.240+0000 7ff6f2e00640 1 -- 172.21.15.131:0/4193763640 wait complete. 2024-05-26T22:55:44.259 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_export_ephemeral_distributed 2024-05-26T22:55:44.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.462+0000 7f51c5800640 1 Processor -- start 2024-05-26T22:55:44.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.462+0000 7f51c5800640 1 -- start start 2024-05-26T22:55:44.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.464+0000 7f51c5800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f51c0156660 0x7f51c00cfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:44.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.464+0000 7f51c5800640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f51c00d01e0 0x7f51c015c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:44.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.464+0000 7f51c5800640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51c015fcc0 con 0x7f51c015cd30 2024-05-26T22:55:44.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.464+0000 7f51c5800640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f51c015ffa0 con 0x7f51c00d01e0 2024-05-26T22:55:44.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.464+0000 7f51c5800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f51c0160280 con 0x7f51c0156660 2024-05-26T22:55:44.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.465+0000 7f51bf400640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f51c00d01e0 0x7f51c015c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:44.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.465+0000 7f51bfe00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f51c0156660 0x7f51c00cfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:44.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.465+0000 7f51bf400640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f51c00d01e0 0x7f51c015c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.131:52420/0 (socket says 172.21.15.131:52420) 2024-05-26T22:55:44.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.465+0000 7f51bf400640 1 -- 172.21.15.131:0/1884725523 learned_addr learned my addr 172.21.15.131:0/1884725523 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:55:44.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.466+0000 7f51bf400640 1 -- 172.21.15.131:0/1884725523 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f51c0156660 msgr2=0x7f51c00cfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:44.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.466+0000 7f51bf400640 1 --2- 172.21.15.131:0/1884725523 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f51c0156660 0x7f51c00cfca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:44.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.466+0000 7f51bf400640 1 -- 172.21.15.131:0/1884725523 >> v1:172.21.15.187:6789/0 conn(0x7f51c015cd30 legacy=0x7f51c0144ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:44.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.466+0000 7f51bf400640 1 -- 172.21.15.131:0/1884725523 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51c0160560 con 0x7f51c00d01e0 2024-05-26T22:55:44.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.466+0000 7f51bea00640 1 -- 172.21.15.131:0/1884725523 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 450472925 0 0) 0x7f51c015fcc0 con 0x7f51c015cd30 2024-05-26T22:55:44.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.467+0000 7f51bf400640 1 --2- 172.21.15.131:0/1884725523 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f51c00d01e0 0x7f51c015c7f0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f51a8009e50 tx=0x7f51a802a710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5fd360145d45ac51 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:44.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.468+0000 7f51bea00640 1 -- 172.21.15.131:0/1884725523 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a8036070 con 0x7f51c00d01e0 2024-05-26T22:55:44.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.468+0000 7f51bea00640 1 -- 172.21.15.131:0/1884725523 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51a8031600 con 0x7f51c00d01e0 2024-05-26T22:55:44.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.468+0000 7f51bea00640 1 -- 172.21.15.131:0/1884725523 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a80319d0 con 0x7f51c00d01e0 2024-05-26T22:55:44.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.468+0000 7f51c5800640 1 -- 172.21.15.131:0/1884725523 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f51c00d01e0 msgr2=0x7f51c015c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:44.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.468+0000 7f51c5800640 1 --2- 172.21.15.131:0/1884725523 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f51c00d01e0 0x7f51c015c7f0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f51a8009e50 tx=0x7f51a802a710 comp rx=0 tx=0).stop 2024-05-26T22:55:44.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.468+0000 7f51c5800640 1 -- 172.21.15.131:0/1884725523 shutdown_connections 2024-05-26T22:55:44.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.468+0000 7f51c5800640 1 --2- 172.21.15.131:0/1884725523 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f51c00d01e0 0x7f51c015c7f0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:44.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.468+0000 7f51c5800640 1 --2- 172.21.15.131:0/1884725523 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f51c0156660 0x7f51c00cfca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:44.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.468+0000 7f51c5800640 1 -- 172.21.15.131:0/1884725523 >> 172.21.15.131:0/1884725523 conn(0x7f51c00cc7f0 msgr2=0x7f51c00ccbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:44.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.469+0000 7f51c5800640 1 -- 172.21.15.131:0/1884725523 shutdown_connections 2024-05-26T22:55:44.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.469+0000 7f51c5800640 1 -- 172.21.15.131:0/1884725523 wait complete. 2024-05-26T22:55:44.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.470+0000 7f51c5800640 1 Processor -- start 2024-05-26T22:55:44.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.470+0000 7f51c5800640 1 -- start start 2024-05-26T22:55:44.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.471+0000 7f51c5800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f51c00d01e0 0x7f51c01f4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:44.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.471+0000 7f51c5800640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f51c0156660 0x7f51c0146400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:44.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.471+0000 7f51c5800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f51c015cd30 0x7f51c0151ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:44.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.471+0000 7f51c5800640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f51c0160a80 con 0x7f51c015cd30 2024-05-26T22:55:44.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.471+0000 7f51c5800640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f51c0156a40 con 0x7f51c0156660 2024-05-26T22:55:44.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.471+0000 7f51c5800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f51c015d1a0 con 0x7f51c00d01e0 2024-05-26T22:55:44.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.472+0000 7f51c4e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f51c015cd30 0x7f51c0151ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:44.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.472+0000 7f51c4e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f51c015cd30 0x7f51c0151ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38302/0 (socket says 172.21.15.131:38302) 2024-05-26T22:55:44.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.472+0000 7f51c4e00640 1 -- 172.21.15.131:0/3745123370 learned_addr learned my addr 172.21.15.131:0/3745123370 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:55:44.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.472+0000 7f51bfe00640 1 --2- 172.21.15.131:0/3745123370 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f51c00d01e0 0x7f51c01f4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:44.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.472+0000 7f51bf400640 1 --2- 172.21.15.131:0/3745123370 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f51c0156660 0x7f51c0146400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:44.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.472+0000 7f51c4e00640 1 -- 172.21.15.131:0/3745123370 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f51c00d01e0 msgr2=0x7f51c01f4f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:44.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.472+0000 7f51c4e00640 1 --2- 172.21.15.131:0/3745123370 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f51c00d01e0 0x7f51c01f4f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:44.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.472+0000 7f51c4e00640 1 -- 172.21.15.131:0/3745123370 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f51c0156660 msgr2=0x7f51c0146400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:44.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.472+0000 7f51c4e00640 1 --2- 172.21.15.131:0/3745123370 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f51c0156660 0x7f51c0146400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:44.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.472+0000 7f51c4e00640 1 -- 172.21.15.131:0/3745123370 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51c0152420 con 0x7f51c015cd30 2024-05-26T22:55:44.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.472+0000 7f51c4e00640 1 --2- 172.21.15.131:0/3745123370 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f51c015cd30 0x7f51c0151ee0 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f51b000c430 tx=0x7f51b000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:44.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.473+0000 7f51b7e00640 1 -- 172.21.15.131:0/3745123370 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51b0017070 con 0x7f51c015cd30 2024-05-26T22:55:44.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.473+0000 7f51b7e00640 1 -- 172.21.15.131:0/3745123370 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51b00035f0 con 0x7f51c015cd30 2024-05-26T22:55:44.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.473+0000 7f51b7e00640 1 -- 172.21.15.131:0/3745123370 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51b00039e0 con 0x7f51c015cd30 2024-05-26T22:55:44.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.473+0000 7f51c5800640 1 -- 172.21.15.131:0/3745123370 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51c014f540 con 0x7f51c015cd30 2024-05-26T22:55:44.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.473+0000 7f51c5800640 1 -- 172.21.15.131:0/3745123370 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51c01f4780 con 0x7f51c015cd30 2024-05-26T22:55:44.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.474+0000 7f51c5800640 1 -- 172.21.15.131:0/3745123370 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51c00b3c50 con 0x7f51c015cd30 2024-05-26T22:55:44.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.475+0000 7f51b7e00640 1 -- 172.21.15.131:0/3745123370 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f51b001a070 con 0x7f51c015cd30 2024-05-26T22:55:44.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.476+0000 7f51b7e00640 1 --2- 172.21.15.131:0/3745123370 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f519c06be30 0x7f519c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:44.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.476+0000 7f51b7e00640 1 -- 172.21.15.131:0/3745123370 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5115+0+0 (secure 0 0 0) 0x7f51b008a0b0 con 0x7f51c015cd30 2024-05-26T22:55:44.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.476+0000 7f51bfe00640 1 --2- 172.21.15.131:0/3745123370 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f519c06be30 0x7f519c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:44.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.477+0000 7f51bfe00640 1 --2- 172.21.15.131:0/3745123370 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f519c06be30 0x7f519c06e2f0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f51b8009ad0 tx=0x7f51b8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:44.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.480+0000 7f51b7e00640 1 -- 172.21.15.131:0/3745123370 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f51c00b3c50 con 0x7f51c015cd30 2024-05-26T22:55:44.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.680+0000 7f51c5800640 1 -- 172.21.15.131:0/3745123370 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_export_ephemeral_distributed"} v 0) -- 0x7f51c00694e0 con 0x7f51c015cd30 2024-05-26T22:55:44.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.685+0000 7f51b7e00640 1 -- 172.21.15.131:0/3745123370 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_export_ephemeral_distributed"}]=0 v57) ==== 115+0+0 (secure 0 0 0) 0x7f51b0058c20 con 0x7f51c015cd30 2024-05-26T22:55:44.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.688+0000 7f51c5800640 1 -- 172.21.15.131:0/3745123370 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f519c06be30 msgr2=0x7f519c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:44.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.688+0000 7f51c5800640 1 --2- 172.21.15.131:0/3745123370 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f519c06be30 0x7f519c06e2f0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f51b8009ad0 tx=0x7f51b8008040 comp rx=0 tx=0).stop 2024-05-26T22:55:44.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.688+0000 7f51c5800640 1 -- 172.21.15.131:0/3745123370 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f51c015cd30 msgr2=0x7f51c0151ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:44.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.688+0000 7f51c5800640 1 --2- 172.21.15.131:0/3745123370 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f51c015cd30 0x7f51c0151ee0 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f51b000c430 tx=0x7f51b000c8f0 comp rx=0 tx=0).stop 2024-05-26T22:55:44.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.688+0000 7f51c5800640 1 -- 172.21.15.131:0/3745123370 shutdown_connections 2024-05-26T22:55:44.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.688+0000 7f51c5800640 1 --2- 172.21.15.131:0/3745123370 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f519c06be30 0x7f519c06e2f0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:44.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.688+0000 7f51c5800640 1 --2- 172.21.15.131:0/3745123370 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f51c015cd30 0x7f51c0151ee0 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:44.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.688+0000 7f51c5800640 1 --2- 172.21.15.131:0/3745123370 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f51c0156660 0x7f51c0146400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:44.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.688+0000 7f51c5800640 1 --2- 172.21.15.131:0/3745123370 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f51c00d01e0 0x7f51c01f4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:44.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.688+0000 7f51c5800640 1 -- 172.21.15.131:0/3745123370 >> 172.21.15.131:0/3745123370 conn(0x7f51c00cc7f0 msgr2=0x7f51c00b4ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:44.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.689+0000 7f51c5800640 1 -- 172.21.15.131:0/3745123370 shutdown_connections 2024-05-26T22:55:44.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.689+0000 7f51c5800640 1 -- 172.21.15.131:0/3745123370 wait complete. 2024-05-26T22:55:44.708 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-26T22:55:44.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.907+0000 7f740d800640 1 Processor -- start 2024-05-26T22:55:44.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.907+0000 7f740d800640 1 -- start start 2024-05-26T22:55:44.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.909+0000 7f740d800640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f740815e690 0x7f74080b9a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:44.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.909+0000 7f740d800640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f74080b7010 0x7f74080b74a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:44.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.909+0000 7f740d800640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74080d0510 con 0x7f740815e2b0 2024-05-26T22:55:44.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.909+0000 7f740d800640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f74080d1120 con 0x7f740815e690 2024-05-26T22:55:44.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.909+0000 7f740d800640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f74080d15b0 con 0x7f74080b7010 2024-05-26T22:55:44.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.909+0000 7f73ff400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f740815e690 0x7f74080b9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:44.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.909+0000 7f73ff400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f740815e690 0x7f74080b9a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38310/0 (socket says 172.21.15.131:38310) 2024-05-26T22:55:44.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.909+0000 7f73ff400640 1 -- 172.21.15.131:0/626712136 learned_addr learned my addr 172.21.15.131:0/626712136 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:55:44.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.910+0000 7f73ff400640 1 -- 172.21.15.131:0/626712136 >> v2:172.21.15.187:3300/0 conn(0x7f74080b7010 msgr2=0x7f74080b74a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:44.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.910+0000 7f73ff400640 1 --2- 172.21.15.131:0/626712136 >> v2:172.21.15.187:3300/0 conn(0x7f74080b7010 0x7f74080b74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:44.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.910+0000 7f73ff400640 1 -- 172.21.15.131:0/626712136 >> v1:172.21.15.187:6789/0 conn(0x7f740815e2b0 legacy=0x7f74080b9330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:44.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.910+0000 7f73ff400640 1 -- 172.21.15.131:0/626712136 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74080d07f0 con 0x7f740815e690 2024-05-26T22:55:44.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.911+0000 7f73ff400640 1 --2- 172.21.15.131:0/626712136 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f740815e690 0x7f74080b9a40 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f74081630d0 tx=0x7f73f002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=317d31195e53c62 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:44.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.911+0000 7f7407400640 1 -- 172.21.15.131:0/626712136 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f0037070 con 0x7f740815e690 2024-05-26T22:55:44.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.911+0000 7f7407400640 1 -- 172.21.15.131:0/626712136 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73f0032440 con 0x7f740815e690 2024-05-26T22:55:44.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.911+0000 7f7407400640 1 -- 172.21.15.131:0/626712136 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f0032810 con 0x7f740815e690 2024-05-26T22:55:44.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.912+0000 7f740d800640 1 -- 172.21.15.131:0/626712136 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f740815e690 msgr2=0x7f74080b9a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:44.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.912+0000 7f740d800640 1 --2- 172.21.15.131:0/626712136 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f740815e690 0x7f74080b9a40 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f74081630d0 tx=0x7f73f002a230 comp rx=0 tx=0).stop 2024-05-26T22:55:44.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.912+0000 7f740d800640 1 -- 172.21.15.131:0/626712136 shutdown_connections 2024-05-26T22:55:44.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.912+0000 7f740d800640 1 --2- 172.21.15.131:0/626712136 >> v2:172.21.15.187:3300/0 conn(0x7f74080b7010 0x7f74080b74a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:44.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.912+0000 7f740d800640 1 --2- 172.21.15.131:0/626712136 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f740815e690 0x7f74080b9a40 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:44.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.912+0000 7f740d800640 1 -- 172.21.15.131:0/626712136 >> 172.21.15.131:0/626712136 conn(0x7f740809d500 msgr2=0x7f74080d41c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:44.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.912+0000 7f740d800640 1 -- 172.21.15.131:0/626712136 shutdown_connections 2024-05-26T22:55:44.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.912+0000 7f740d800640 1 -- 172.21.15.131:0/626712136 wait complete. 2024-05-26T22:55:44.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.914+0000 7f740d800640 1 Processor -- start 2024-05-26T22:55:44.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.914+0000 7f740d800640 1 -- start start 2024-05-26T22:55:44.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.915+0000 7f740d800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f74080b7010 0x7f74081f4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:44.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.915+0000 7f740d800640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f740815e2b0 0x7f7408153320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:44.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.915+0000 7f740d800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f740815e690 0x7f740815b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:44.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.915+0000 7f740d800640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f74080b9f80 con 0x7f74080b7010 2024-05-26T22:55:44.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.915+0000 7f740d800640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f74080d0db0 con 0x7f740815e2b0 2024-05-26T22:55:44.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.915+0000 7f740d800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f74080ba100 con 0x7f740815e690 2024-05-26T22:55:44.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.915+0000 7f7407e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f74080b7010 0x7f74081f4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:44.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.915+0000 7f7407e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f74080b7010 0x7f74081f4fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38312/0 (socket says 172.21.15.131:38312) 2024-05-26T22:55:44.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.915+0000 7f7407e00640 1 -- 172.21.15.131:0/4106387213 learned_addr learned my addr 172.21.15.131:0/4106387213 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:55:44.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.916+0000 7f7407e00640 1 -- 172.21.15.131:0/4106387213 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f740815e690 msgr2=0x7f740815b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:44.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.916+0000 7f7407e00640 1 --2- 172.21.15.131:0/4106387213 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f740815e690 0x7f740815b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:44.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.916+0000 7f7407e00640 1 -- 172.21.15.131:0/4106387213 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f740815e2b0 msgr2=0x7f7408153320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:44.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.916+0000 7f7407e00640 1 --2- 172.21.15.131:0/4106387213 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f740815e2b0 0x7f7408153320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:44.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.916+0000 7f7407e00640 1 -- 172.21.15.131:0/4106387213 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7408153860 con 0x7f74080b7010 2024-05-26T22:55:44.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.916+0000 7f7407e00640 1 --2- 172.21.15.131:0/4106387213 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f74080b7010 0x7f74081f4fb0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f73f800c430 tx=0x7f73f800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:44.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.916+0000 7f7404c00640 1 -- 172.21.15.131:0/4106387213 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f8011790 con 0x7f74080b7010 2024-05-26T22:55:44.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.916+0000 7f7404c00640 1 -- 172.21.15.131:0/4106387213 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73f8017070 con 0x7f74080b7010 2024-05-26T22:55:44.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.916+0000 7f7404c00640 1 -- 172.21.15.131:0/4106387213 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f8011bb0 con 0x7f74080b7010 2024-05-26T22:55:44.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.916+0000 7f740d800640 1 -- 172.21.15.131:0/4106387213 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74080cc580 con 0x7f74080b7010 2024-05-26T22:55:44.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.916+0000 7f740d800640 1 -- 172.21.15.131:0/4106387213 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74080cb7c0 con 0x7f74080b7010 2024-05-26T22:55:44.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.917+0000 7f740d800640 1 -- 172.21.15.131:0/4106387213 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74080cb270 con 0x7f74080b7010 2024-05-26T22:55:44.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.918+0000 7f7404c00640 1 -- 172.21.15.131:0/4106387213 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f73f801c460 con 0x7f74080b7010 2024-05-26T22:55:44.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.919+0000 7f7404c00640 1 --2- 172.21.15.131:0/4106387213 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f73e406be60 0x7f73e406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:44.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.920+0000 7f7404c00640 1 -- 172.21.15.131:0/4106387213 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5115+0+0 (secure 0 0 0) 0x7f73f808aa50 con 0x7f74080b7010 2024-05-26T22:55:44.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.920+0000 7f73ff400640 1 --2- 172.21.15.131:0/4106387213 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f73e406be60 0x7f73e406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:44.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.920+0000 7f73ff400640 1 --2- 172.21.15.131:0/4106387213 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f73e406be60 0x7f73e406e320 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f73f0034000 tx=0x7f73f002c000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:44.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:44.924+0000 7f7404c00640 1 -- 172.21.15.131:0/4106387213 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f73f80596e0 con 0x7f74080b7010 2024-05-26T22:55:45.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:45.122+0000 7f740d800640 1 -- 172.21.15.131:0/4106387213 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f74080694e0 con 0x7f74080b7010 2024-05-26T22:55:45.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:45.129+0000 7f7404c00640 1 -- 172.21.15.131:0/4106387213 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v58) ==== 104+0+0 (secure 0 0 0) 0x7f73f8058fe0 con 0x7f74080b7010 2024-05-26T22:55:45.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:45.132+0000 7f740d800640 1 -- 172.21.15.131:0/4106387213 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f73e406be60 msgr2=0x7f73e406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:45.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:45.132+0000 7f740d800640 1 --2- 172.21.15.131:0/4106387213 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f73e406be60 0x7f73e406e320 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f73f0034000 tx=0x7f73f002c000 comp rx=0 tx=0).stop 2024-05-26T22:55:45.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:45.132+0000 7f740d800640 1 -- 172.21.15.131:0/4106387213 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f74080b7010 msgr2=0x7f74081f4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:45.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:45.132+0000 7f740d800640 1 --2- 172.21.15.131:0/4106387213 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f74080b7010 0x7f74081f4fb0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f73f800c430 tx=0x7f73f800c8f0 comp rx=0 tx=0).stop 2024-05-26T22:55:45.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:45.132+0000 7f740d800640 1 -- 172.21.15.131:0/4106387213 shutdown_connections 2024-05-26T22:55:45.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:45.132+0000 7f740d800640 1 --2- 172.21.15.131:0/4106387213 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f740815e690 0x7f740815b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:45.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:45.132+0000 7f740d800640 1 --2- 172.21.15.131:0/4106387213 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f73e406be60 0x7f73e406e320 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:45.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:45.132+0000 7f740d800640 1 --2- 172.21.15.131:0/4106387213 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f740815e2b0 0x7f7408153320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:45.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:45.132+0000 7f740d800640 1 --2- 172.21.15.131:0/4106387213 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f74080b7010 0x7f74081f4fb0 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:45.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:45.132+0000 7f740d800640 1 -- 172.21.15.131:0/4106387213 >> 172.21.15.131:0/4106387213 conn(0x7f740809d500 msgr2=0x7f7408161d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:45.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:45.132+0000 7f740d800640 1 -- 172.21.15.131:0/4106387213 shutdown_connections 2024-05-26T22:55:45.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:45.132+0000 7f740d800640 1 -- 172.21.15.131:0/4106387213 wait complete. 2024-05-26T22:55:45.150 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_exports.TestEphemeralPins.test_ephemeral_pin_dist_override_before' 2024-05-26T22:55:45.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:45.351+0000 7f49cf800640 1 Processor -- start 2024-05-26T22:55:45.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:45.351+0000 7f49cf800640 1 -- start start 2024-05-26T22:55:45.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:45.352+0000 7f49cf800640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f49c815c3d0 0x7f49c806ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:45.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:45.352+0000 7f49cf800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f49c806f490 0x7f49c80ade40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:45.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:45.352+0000 7f49cf800640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49c80722a0 con 0x7f49c8155840 2024-05-26T22:55:45.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:45.352+0000 7f49cf800640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f49c8072040 con 0x7f49c815c3d0 2024-05-26T22:55:45.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:45.352+0000 7f49cf800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f49c8072580 con 0x7f49c806f490 2024-05-26T22:55:45.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:45.353+0000 7f49cd800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f49c806f490 0x7f49c80ade40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:45.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:45.353+0000 7f49bfe00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f49c815c3d0 0x7f49c806ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:45.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:45.353+0000 7f49bfe00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f49c815c3d0 0x7f49c806ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.131:52448/0 (socket says 172.21.15.131:52448) 2024-05-26T22:55:45.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:45.353+0000 7f49cce00640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f49c8155840 0x7f49c8071930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.131:37644/0 (socket says 172.21.15.131:37644) 2024-05-26T22:55:45.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:45.353+0000 7f49cd800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f49c806f490 0x7f49c80ade40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.131:57594/0 (socket says 172.21.15.131:57594) 2024-05-26T22:55:45.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:45.353+0000 7f49cce00640 1 -- 172.21.15.131:0/2950263114 learned_addr learned my addr 172.21.15.131:0/2950263114 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:55:45.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:45.354+0000 7f49bfe00640 1 -- 172.21.15.131:0/2950263114 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f49c806f490 msgr2=0x7f49c80ade40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:45.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:45.354+0000 7f49bfe00640 1 --2- 172.21.15.131:0/2950263114 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f49c806f490 0x7f49c80ade40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:45.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:45.354+0000 7f49bfe00640 1 -- 172.21.15.131:0/2950263114 >> v1:172.21.15.187:6789/0 conn(0x7f49c8155840 legacy=0x7f49c8071930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:45.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:45.354+0000 7f49bfe00640 1 -- 172.21.15.131:0/2950263114 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49c8072860 con 0x7f49c815c3d0 2024-05-26T22:55:45.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:45.355+0000 7f49bfe00640 1 --2- 172.21.15.131:0/2950263114 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f49c815c3d0 0x7f49c806ef50 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f49b00097d0 tx=0x7f49b002a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2532c0b9b94b4b93 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:45.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:45.355+0000 7f49c7e00640 1 -- 172.21.15.131:0/2950263114 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49b0037070 con 0x7f49c815c3d0 2024-05-26T22:55:45.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:45.356+0000 7f49c7e00640 1 -- 172.21.15.131:0/2950263114 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49b0032440 con 0x7f49c815c3d0 2024-05-26T22:55:45.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:45.356+0000 7f49c7e00640 1 -- 172.21.15.131:0/2950263114 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49b0032810 con 0x7f49c815c3d0 2024-05-26T22:55:45.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:45.356+0000 7f49cf800640 1 -- 172.21.15.131:0/2950263114 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f49c815c3d0 msgr2=0x7f49c806ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:45.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:45.356+0000 7f49cf800640 1 --2- 172.21.15.131:0/2950263114 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f49c815c3d0 0x7f49c806ef50 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f49b00097d0 tx=0x7f49b002a1c0 comp rx=0 tx=0).stop 2024-05-26T22:55:45.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:45.356+0000 7f49cf800640 1 -- 172.21.15.131:0/2950263114 shutdown_connections 2024-05-26T22:55:45.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:45.356+0000 7f49cf800640 1 --2- 172.21.15.131:0/2950263114 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f49c806f490 0x7f49c80ade40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:45.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:45.356+0000 7f49cf800640 1 --2- 172.21.15.131:0/2950263114 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f49c815c3d0 0x7f49c806ef50 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:45.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:45.356+0000 7f49cf800640 1 -- 172.21.15.131:0/2950263114 >> 172.21.15.131:0/2950263114 conn(0x7f49c80b1e90 msgr2=0x7f49c80b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:45.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:45.356+0000 7f49cf800640 1 -- 172.21.15.131:0/2950263114 shutdown_connections 2024-05-26T22:55:45.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:45.356+0000 7f49cf800640 1 -- 172.21.15.131:0/2950263114 wait complete. 2024-05-26T22:55:45.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:45.358+0000 7f49cf800640 1 Processor -- start 2024-05-26T22:55:45.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:45.358+0000 7f49cf800640 1 -- start start 2024-05-26T22:55:45.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:45.359+0000 7f49cf800640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f49c806f490 0x7f49c80cd670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:45.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:45.359+0000 7f49cf800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49c8155840 0x7f49c814bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:45.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:45.359+0000 7f49cf800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f49c815c3d0 0x7f49c815d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:45.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:45.359+0000 7f49cf800640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f49c80ae380 con 0x7f49c8155840 2024-05-26T22:55:45.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:45.359+0000 7f49cf800640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f49c8072e20 con 0x7f49c806f490 2024-05-26T22:55:45.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:45.359+0000 7f49cf800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f49c80ae500 con 0x7f49c815c3d0 2024-05-26T22:55:45.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:45.359+0000 7f49bfe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49c8155840 0x7f49c814bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:45.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:45.360+0000 7f49bfe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49c8155840 0x7f49c814bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38324/0 (socket says 172.21.15.131:38324) 2024-05-26T22:55:45.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:45.360+0000 7f49bfe00640 1 -- 172.21.15.131:0/3703135606 learned_addr learned my addr 172.21.15.131:0/3703135606 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:55:45.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:45.360+0000 7f49cd800640 1 --2- 172.21.15.131:0/3703135606 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f49c815c3d0 0x7f49c815d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:45.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:45.360+0000 7f49cce00640 1 --2- 172.21.15.131:0/3703135606 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f49c806f490 0x7f49c80cd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:45.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:45.360+0000 7f49bfe00640 1 -- 172.21.15.131:0/3703135606 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f49c815c3d0 msgr2=0x7f49c815d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:45.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:45.360+0000 7f49bfe00640 1 --2- 172.21.15.131:0/3703135606 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f49c815c3d0 0x7f49c815d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:45.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:45.360+0000 7f49bfe00640 1 -- 172.21.15.131:0/3703135606 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f49c806f490 msgr2=0x7f49c80cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:45.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:45.360+0000 7f49bfe00640 1 --2- 172.21.15.131:0/3703135606 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f49c806f490 0x7f49c80cd670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:45.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:45.360+0000 7f49bfe00640 1 -- 172.21.15.131:0/3703135606 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49c80ccbd0 con 0x7f49c8155840 2024-05-26T22:55:45.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:45.360+0000 7f49cd800640 1 --2- 172.21.15.131:0/3703135606 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f49c815c3d0 0x7f49c815d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:55:45.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:45.360+0000 7f49cce00640 1 --2- 172.21.15.131:0/3703135606 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f49c806f490 0x7f49c80cd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:55:45.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:45.360+0000 7f49bfe00640 1 --2- 172.21.15.131:0/3703135606 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49c8155840 0x7f49c814bc90 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f49b00097d0 tx=0x7f49b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:45.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:45.361+0000 7f49c5600640 1 -- 172.21.15.131:0/3703135606 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49b0037040 con 0x7f49c8155840 2024-05-26T22:55:45.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:45.361+0000 7f49c5600640 1 -- 172.21.15.131:0/3703135606 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49b00304c0 con 0x7f49c8155840 2024-05-26T22:55:45.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:45.361+0000 7f49c5600640 1 -- 172.21.15.131:0/3703135606 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49b0030890 con 0x7f49c8155840 2024-05-26T22:55:45.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:45.361+0000 7f49cf800640 1 -- 172.21.15.131:0/3703135606 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49c8149eb0 con 0x7f49c8155840 2024-05-26T22:55:45.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:45.361+0000 7f49cf800640 1 -- 172.21.15.131:0/3703135606 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49c814a190 con 0x7f49c8155840 2024-05-26T22:55:45.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:45.362+0000 7f49cf800640 1 -- 172.21.15.131:0/3703135606 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49c80694e0 con 0x7f49c8155840 2024-05-26T22:55:45.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:45.363+0000 7f49c5600640 1 -- 172.21.15.131:0/3703135606 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f49b0044020 con 0x7f49c8155840 2024-05-26T22:55:45.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:45.364+0000 7f49c5600640 1 --2- 172.21.15.131:0/3703135606 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f49a406be30 0x7f49a406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:45.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:45.365+0000 7f49c5600640 1 -- 172.21.15.131:0/3703135606 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5115+0+0 (secure 0 0 0) 0x7f49b00aac40 con 0x7f49c8155840 2024-05-26T22:55:45.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:45.365+0000 7f49cce00640 1 --2- 172.21.15.131:0/3703135606 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f49a406be30 0x7f49a406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:45.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:45.365+0000 7f49cce00640 1 --2- 172.21.15.131:0/3703135606 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f49a406be30 0x7f49a406e2f0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f49b8000f60 tx=0x7f49b8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:45.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:45.369+0000 7f49c5600640 1 -- 172.21.15.131:0/3703135606 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f49b004b020 con 0x7f49c8155840 2024-05-26T22:55:45.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:45.638+0000 7f49cf800640 1 -- 172.21.15.131:0/3703135606 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_exports.TestEphemeralPins.test_ephemeral_pin_dist_override_before"]} v 0) -- 0x7f49c80b1030 con 0x7f49c8155840 2024-05-26T22:55:45.854 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-26T22:55:45.863 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-26T22:55:46.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:46.373+0000 7f49c5600640 1 -- 172.21.15.131:0/3703135606 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_exports.TestEphemeralPins.test_ephemeral_pin_dist_override_before"]}]=0 v374) ==== 162+0+0 (secure 0 0 0) 0x7f49b00798d0 con 0x7f49c8155840 2024-05-26T22:55:46.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:46.377+0000 7f49cf800640 1 -- 172.21.15.131:0/3703135606 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f49a406be30 msgr2=0x7f49a406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:46.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:46.377+0000 7f49cf800640 1 --2- 172.21.15.131:0/3703135606 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f49a406be30 0x7f49a406e2f0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f49b8000f60 tx=0x7f49b8009040 comp rx=0 tx=0).stop 2024-05-26T22:55:46.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:46.377+0000 7f49cf800640 1 -- 172.21.15.131:0/3703135606 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49c8155840 msgr2=0x7f49c814bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:46.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:46.377+0000 7f49cf800640 1 --2- 172.21.15.131:0/3703135606 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49c8155840 0x7f49c814bc90 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f49b00097d0 tx=0x7f49b00046c0 comp rx=0 tx=0).stop 2024-05-26T22:55:46.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:46.377+0000 7f49cf800640 1 -- 172.21.15.131:0/3703135606 shutdown_connections 2024-05-26T22:55:46.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:46.377+0000 7f49cf800640 1 --2- 172.21.15.131:0/3703135606 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f49c815c3d0 0x7f49c815d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:46.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:46.377+0000 7f49cf800640 1 --2- 172.21.15.131:0/3703135606 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f49a406be30 0x7f49a406e2f0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:46.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:46.377+0000 7f49cf800640 1 --2- 172.21.15.131:0/3703135606 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49c8155840 0x7f49c814bc90 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:46.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:46.377+0000 7f49cf800640 1 --2- 172.21.15.131:0/3703135606 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f49c806f490 0x7f49c80cd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:46.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:46.377+0000 7f49cf800640 1 -- 172.21.15.131:0/3703135606 >> 172.21.15.131:0/3703135606 conn(0x7f49c80b1e90 msgr2=0x7f49c80b1590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:46.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:46.377+0000 7f49cf800640 1 -- 172.21.15.131:0/3703135606 shutdown_connections 2024-05-26T22:55:46.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:46.377+0000 7f49cf800640 1 -- 172.21.15.131:0/3703135606 wait complete. 2024-05-26T22:55:46.398 INFO:tasks.cephfs_test_runner:That a conventional export pin overrides the distributed policy _before_ distributed policy is set. ... ok 2024-05-26T22:55:46.398 INFO:tasks.cephfs_test_runner:Starting test: test_ephemeral_pin_dist_override_pin (tasks.cephfs.test_exports.TestEphemeralPins) That an export pin overrides an ephemerally pinned directory. 2024-05-26T22:55:46.398 INFO:tasks.cephfs_test_runner:test_ephemeral_pin_dist_override_pin (tasks.cephfs.test_exports.TestEphemeralPins) 2024-05-26T22:55:46.399 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-26T22:55:46.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:46.602+0000 7fb361200640 1 Processor -- start 2024-05-26T22:55:46.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:46.602+0000 7fb361200640 1 -- start start 2024-05-26T22:55:46.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:46.604+0000 7fb361200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fb35c06ee20 0x7fb35c0b3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:46.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:46.604+0000 7fb361200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb35c1569d0 0x7fb35c0af790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:46.604 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:46.604+0000 7fb361200640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb35c15cb20 con 0x7fb35c0ad250 2024-05-26T22:55:46.604 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:46.604+0000 7fb361200640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fb35c15ce00 con 0x7fb35c06ee20 2024-05-26T22:55:46.604 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:46.604+0000 7fb361200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb35c15d0e0 con 0x7fb35c1569d0 2024-05-26T22:55:46.604 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:46.604+0000 7fb359e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fb35c06ee20 0x7fb35c0b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:46.604 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:46.604+0000 7fb359e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fb35c06ee20 0x7fb35c0b3190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38332/0 (socket says 172.21.15.131:38332) 2024-05-26T22:55:46.604 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:46.604+0000 7fb359e00640 1 -- 172.21.15.131:0/3395678774 learned_addr learned my addr 172.21.15.131:0/3395678774 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:55:46.604 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:46.604+0000 7fb35a800640 1 --2- 172.21.15.131:0/3395678774 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb35c1569d0 0x7fb35c0af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:46.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:46.604+0000 7fb359e00640 1 -- 172.21.15.131:0/3395678774 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb35c1569d0 msgr2=0x7fb35c0af790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:46.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:46.604+0000 7fb359e00640 1 --2- 172.21.15.131:0/3395678774 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb35c1569d0 0x7fb35c0af790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:46.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:46.604+0000 7fb359e00640 1 -- 172.21.15.131:0/3395678774 >> v1:172.21.15.187:6789/0 conn(0x7fb35c0ad250 legacy=0x7fb35c1562c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:46.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:46.604+0000 7fb359e00640 1 -- 172.21.15.131:0/3395678774 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb35c15d3c0 con 0x7fb35c06ee20 2024-05-26T22:55:46.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:46.605+0000 7fb359e00640 1 --2- 172.21.15.131:0/3395678774 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb35c06ee20 0x7fb35c0b3190 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7fb34c009870 tx=0x7fb34c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9421d253a069ea20 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:46.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:46.606+0000 7fb358a00640 1 -- 172.21.15.131:0/3395678774 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb34c037070 con 0x7fb35c06ee20 2024-05-26T22:55:46.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:46.606+0000 7fb358a00640 1 -- 172.21.15.131:0/3395678774 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb34c032440 con 0x7fb35c06ee20 2024-05-26T22:55:46.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:46.606+0000 7fb358a00640 1 -- 172.21.15.131:0/3395678774 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb34c032810 con 0x7fb35c06ee20 2024-05-26T22:55:46.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:46.606+0000 7fb361200640 1 -- 172.21.15.131:0/3395678774 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb35c06ee20 msgr2=0x7fb35c0b3190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:46.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:46.606+0000 7fb361200640 1 --2- 172.21.15.131:0/3395678774 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb35c06ee20 0x7fb35c0b3190 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7fb34c009870 tx=0x7fb34c02a210 comp rx=0 tx=0).stop 2024-05-26T22:55:46.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:46.606+0000 7fb361200640 1 -- 172.21.15.131:0/3395678774 shutdown_connections 2024-05-26T22:55:46.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:46.606+0000 7fb361200640 1 --2- 172.21.15.131:0/3395678774 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb35c1569d0 0x7fb35c0af790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:46.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:46.606+0000 7fb361200640 1 --2- 172.21.15.131:0/3395678774 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb35c06ee20 0x7fb35c0b3190 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:46.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:46.606+0000 7fb361200640 1 -- 172.21.15.131:0/3395678774 >> 172.21.15.131:0/3395678774 conn(0x7fb35c0b1e90 msgr2=0x7fb35c0b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:46.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:46.607+0000 7fb361200640 1 -- 172.21.15.131:0/3395678774 shutdown_connections 2024-05-26T22:55:46.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:46.607+0000 7fb361200640 1 -- 172.21.15.131:0/3395678774 wait complete. 2024-05-26T22:55:46.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:46.608+0000 7fb361200640 1 Processor -- start 2024-05-26T22:55:46.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:46.608+0000 7fb361200640 1 -- start start 2024-05-26T22:55:46.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:46.609+0000 7fb361200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb35c06ee20 0x7fb35c1f4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:46.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:46.609+0000 7fb361200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb35c0ad250 0x7fb35c151db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:46.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:46.609+0000 7fb361200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb35c1569d0 0x7fb35c0729d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:46.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:46.609+0000 7fb361200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fb35c15c900 con 0x7fb35c06ee20 2024-05-26T22:55:46.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:46.609+0000 7fb361200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb35c15c780 con 0x7fb35c0ad250 2024-05-26T22:55:46.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:46.609+0000 7fb361200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb35c06f200 con 0x7fb35c1569d0 2024-05-26T22:55:46.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:46.609+0000 7fb359e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb35c06ee20 0x7fb35c1f4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:46.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:46.610+0000 7fb359e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb35c06ee20 0x7fb35c1f4060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38342/0 (socket says 172.21.15.131:38342) 2024-05-26T22:55:46.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:46.610+0000 7fb359e00640 1 -- 172.21.15.131:0/3790363588 learned_addr learned my addr 172.21.15.131:0/3790363588 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:55:46.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:46.610+0000 7fb359e00640 1 -- 172.21.15.131:0/3790363588 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb35c1569d0 msgr2=0x7fb35c0729d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:46.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:46.610+0000 7fb359e00640 1 --2- 172.21.15.131:0/3790363588 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb35c1569d0 0x7fb35c0729d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:46.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:46.610+0000 7fb359e00640 1 -- 172.21.15.131:0/3790363588 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb35c0ad250 msgr2=0x7fb35c151db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:46.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:46.610+0000 7fb359e00640 1 --2- 172.21.15.131:0/3790363588 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb35c0ad250 0x7fb35c151db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:46.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:46.610+0000 7fb359e00640 1 -- 172.21.15.131:0/3790363588 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb35c152890 con 0x7fb35c06ee20 2024-05-26T22:55:46.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:46.610+0000 7fb359e00640 1 --2- 172.21.15.131:0/3790363588 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb35c06ee20 0x7fb35c1f4060 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7fb34c009870 tx=0x7fb34c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:46.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:46.611+0000 7fb352000640 1 -- 172.21.15.131:0/3790363588 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb34c041020 con 0x7fb35c06ee20 2024-05-26T22:55:46.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:46.611+0000 7fb352000640 1 -- 172.21.15.131:0/3790363588 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb34c030970 con 0x7fb35c06ee20 2024-05-26T22:55:46.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:46.611+0000 7fb361200640 1 -- 172.21.15.131:0/3790363588 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb35c152b70 con 0x7fb35c06ee20 2024-05-26T22:55:46.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:46.611+0000 7fb352000640 1 -- 172.21.15.131:0/3790363588 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb34c037040 con 0x7fb35c06ee20 2024-05-26T22:55:46.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:46.611+0000 7fb361200640 1 -- 172.21.15.131:0/3790363588 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb35c153130 con 0x7fb35c06ee20 2024-05-26T22:55:46.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:46.612+0000 7fb361200640 1 -- 172.21.15.131:0/3790363588 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb35c0694e0 con 0x7fb35c06ee20 2024-05-26T22:55:46.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:46.613+0000 7fb352000640 1 -- 172.21.15.131:0/3790363588 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb34c044020 con 0x7fb35c06ee20 2024-05-26T22:55:46.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:46.614+0000 7fb352000640 1 --2- 172.21.15.131:0/3790363588 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb33c06be30 0x7fb33c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:46.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:46.614+0000 7fb352000640 1 -- 172.21.15.131:0/3790363588 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5115+0+0 (secure 0 0 0) 0x7fb34c0ab680 con 0x7fb35c06ee20 2024-05-26T22:55:46.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:46.614+0000 7fb359400640 1 --2- 172.21.15.131:0/3790363588 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb33c06be30 0x7fb33c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:46.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:46.615+0000 7fb359400640 1 --2- 172.21.15.131:0/3790363588 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb33c06be30 0x7fb33c06e2f0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fb35c1559a0 tx=0x7fb3480079b0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:46.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:46.618+0000 7fb352000640 1 -- 172.21.15.131:0/3790363588 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fb34c04b030 con 0x7fb35c06ee20 2024-05-26T22:55:46.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:46.821+0000 7fb361200640 1 -- 172.21.15.131:0/3790363588 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb35c06e280 con 0x7fb35c06ee20 2024-05-26T22:55:46.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:46.823+0000 7fb352000640 1 -- 172.21.15.131:0/3790363588 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) ==== 74+0+13307 (secure 0 0 0) 0x7fb34c07a1e0 con 0x7fb35c06ee20 2024-05-26T22:55:46.824 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:55:46.824 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":98,"fsid":"46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f","created":"2024-05-26T22:47:57.288905+0000","modified":"2024-05-26T22:55:43.339506+0000","last_up_change":"2024-05-26T22:48:09.602207+0000","last_in_change":"2024-05-26T22:48:00.687341+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-26T22:48:10.463339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"814eaf62-a18d-494b-8e27-fd8fb88adea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6801","nonce":2350940051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6803","nonce":2350940051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6807","nonce":2350940051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6805","nonce":2350940051}]},"public_addr":"172.21.15.131:6801/2350940051","cluster_addr":"172.21.15.131:6803/2350940051","heartbeat_back_addr":"172.21.15.131:6807/2350940051","heartbeat_front_addr":"172.21.15.131:6805/2350940051","state":["exists","up"]},{"osd":1,"uuid":"42e66287-f6c6-4e26-af3c-6dfaa3536d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6825","nonce":1525528247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6827","nonce":1525528247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6831","nonce":1525528247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6829","nonce":1525528247}]},"public_addr":"172.21.15.131:6825/1525528247","cluster_addr":"172.21.15.131:6827/1525528247","heartbeat_back_addr":"172.21.15.131:6831/1525528247","heartbeat_front_addr":"172.21.15.131:6829/1525528247","state":["exists","up"]},{"osd":2,"uuid":"0da34673-04a2-4fd0-b57a-0ff2b679b043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6817","nonce":3627842210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6819","nonce":3627842210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6823","nonce":3627842210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6821","nonce":3627842210}]},"public_addr":"172.21.15.131:6817/3627842210","cluster_addr":"172.21.15.131:6819/3627842210","heartbeat_back_addr":"172.21.15.131:6823/3627842210","heartbeat_front_addr":"172.21.15.131:6821/3627842210","state":["exists","up"]},{"osd":3,"uuid":"cf117a27-d6b2-43d3-ae52-b47988bf8eed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6809","nonce":350790591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6811","nonce":350790591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6815","nonce":350790591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6813","nonce":350790591}]},"public_addr":"172.21.15.131:6809/350790591","cluster_addr":"172.21.15.131:6811/350790591","heartbeat_back_addr":"172.21.15.131:6815/350790591","heartbeat_front_addr":"172.21.15.131:6813/350790591","state":["exists","up"]},{"osd":4,"uuid":"a092ccfc-be89-4a84-88d5-0f781fa715eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6816","nonce":244560995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6819","nonce":244560995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6823","nonce":244560995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6821","nonce":244560995}]},"public_addr":"172.21.15.187:6816/244560995","cluster_addr":"172.21.15.187:6819/244560995","heartbeat_back_addr":"172.21.15.187:6823/244560995","heartbeat_front_addr":"172.21.15.187:6821/244560995","state":["exists","up"]},{"osd":5,"uuid":"7513ba69-dd00-4a35-b190-9e8cc3b2bdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6801","nonce":324619398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6803","nonce":324619398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6812","nonce":324619398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6807","nonce":324619398}]},"public_addr":"172.21.15.187:6801/324619398","cluster_addr":"172.21.15.187:6803/324619398","heartbeat_back_addr":"172.21.15.187:6812/324619398","heartbeat_front_addr":"172.21.15.187:6807/324619398","state":["exists","up"]},{"osd":6,"uuid":"50081810-c093-4d2f-a4d7-b7c695c5400d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6806","nonce":1391347786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6809","nonce":1391347786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6818","nonce":1391347786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6813","nonce":1391347786}]},"public_addr":"172.21.15.187:6806/1391347786","cluster_addr":"172.21.15.187:6809/1391347786","heartbeat_back_addr":"172.21.15.187:6818/1391347786","heartbeat_front_addr":"172.21.15.187:6813/1391347786","state":["exists","up"]},{"osd":7,"uuid":"775dae05-54f5-4ad4-bbce-92863c6ed2ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6825","nonce":1220065622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6827","nonce":1220065622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6831","nonce":1220065622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6829","nonce":1220065622}]},"public_addr":"172.21.15.187:6825/1220065622","cluster_addr":"172.21.15.187:6827/1220065622","heartbeat_back_addr":"172.21.15.187:6831/1220065622","heartbeat_front_addr":"172.21.15.187:6829/1220065622","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:05.983520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.299516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.067323+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.307748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.018742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.258775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.010912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.295355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:6836/789312503":"2024-05-27T22:55:40.105920+0000","172.21.15.187:6833/1314245822":"2024-05-27T22:55:40.105883+0000","172.21.15.187:6832/1314245822":"2024-05-27T22:55:40.105883+0000","172.21.15.131:6834/789312503":"2024-05-27T22:55:40.105920+0000","172.21.15.187:6835/521878375":"2024-05-27T22:55:40.105908+0000","172.21.15.187:6834/521878375":"2024-05-27T22:55:40.105908+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-26T22:55:46.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:46.826+0000 7fb361200640 1 -- 172.21.15.131:0/3790363588 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb33c06be30 msgr2=0x7fb33c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:46.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:46.826+0000 7fb361200640 1 --2- 172.21.15.131:0/3790363588 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb33c06be30 0x7fb33c06e2f0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fb35c1559a0 tx=0x7fb3480079b0 comp rx=0 tx=0).stop 2024-05-26T22:55:46.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:46.826+0000 7fb361200640 1 -- 172.21.15.131:0/3790363588 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb35c06ee20 msgr2=0x7fb35c1f4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:46.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:46.826+0000 7fb361200640 1 --2- 172.21.15.131:0/3790363588 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb35c06ee20 0x7fb35c1f4060 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7fb34c009870 tx=0x7fb34c0046c0 comp rx=0 tx=0).stop 2024-05-26T22:55:46.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:46.827+0000 7fb361200640 1 -- 172.21.15.131:0/3790363588 shutdown_connections 2024-05-26T22:55:46.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:46.827+0000 7fb361200640 1 --2- 172.21.15.131:0/3790363588 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb35c1569d0 0x7fb35c0729d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:46.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:46.827+0000 7fb361200640 1 --2- 172.21.15.131:0/3790363588 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb33c06be30 0x7fb33c06e2f0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:46.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:46.827+0000 7fb361200640 1 --2- 172.21.15.131:0/3790363588 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb35c0ad250 0x7fb35c151db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:46.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:46.827+0000 7fb361200640 1 --2- 172.21.15.131:0/3790363588 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb35c06ee20 0x7fb35c1f4060 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:46.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:46.827+0000 7fb361200640 1 -- 172.21.15.131:0/3790363588 >> 172.21.15.131:0/3790363588 conn(0x7fb35c0b1e90 msgr2=0x7fb35c0b3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:46.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:46.827+0000 7fb361200640 1 -- 172.21.15.131:0/3790363588 shutdown_connections 2024-05-26T22:55:46.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:46.827+0000 7fb361200640 1 -- 172.21.15.131:0/3790363588 wait complete. 2024-05-26T22:55:46.845 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-26T22:48:10.463339+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-26T22:55:46.846 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-26T22:55:47.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.044+0000 7f752a000640 1 Processor -- start 2024-05-26T22:55:47.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.044+0000 7f752a000640 1 -- start start 2024-05-26T22:55:47.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.046+0000 7f752a000640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f75240712d0 0x7f752406d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:47.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.046+0000 7f752a000640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7524072d60 con 0x7f7524145a10 2024-05-26T22:55:47.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.046+0000 7f752a000640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7524072a80 con 0x7f752406d790 2024-05-26T22:55:47.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.046+0000 7f752a000640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f752415c760 con 0x7f75240712d0 2024-05-26T22:55:47.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.046+0000 7f7523e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f75240712d0 0x7f752406d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:47.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.046+0000 7f7528c00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f7524145a10 0x7f7524072180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:47790/0 (socket says 172.21.15.131:47790) 2024-05-26T22:55:47.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.046+0000 7f7528c00640 1 -- 172.21.15.131:0/2667397517 learned_addr learned my addr 172.21.15.131:0/2667397517 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:55:47.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.047+0000 7f7523e00640 1 -- 172.21.15.131:0/2667397517 >> v1:172.21.15.187:6789/0 conn(0x7f752406d790 legacy=0x7f75240cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:47.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.047+0000 7f7523e00640 1 -- 172.21.15.131:0/2667397517 >> v1:172.21.15.131:6789/0 conn(0x7f7524145a10 legacy=0x7f7524072180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:47.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.047+0000 7f7523e00640 1 -- 172.21.15.131:0/2667397517 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75240ceab0 con 0x7f75240712d0 2024-05-26T22:55:47.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.047+0000 7f7523400640 1 -- 172.21.15.131:0/2667397517 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1917940036 0 0) 0x7f7524072d60 con 0x7f7524145a10 2024-05-26T22:55:47.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.048+0000 7f7523e00640 1 --2- 172.21.15.131:0/2667397517 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75240712d0 0x7f752406d250 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f75100097a0 tx=0x7f751002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=329d85943ce451a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:47.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.048+0000 7f7523400640 1 -- 172.21.15.131:0/2667397517 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7510037070 con 0x7f75240712d0 2024-05-26T22:55:47.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.048+0000 7f7523400640 1 -- 172.21.15.131:0/2667397517 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7510032440 con 0x7f75240712d0 2024-05-26T22:55:47.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.048+0000 7f7523400640 1 -- 172.21.15.131:0/2667397517 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7510032810 con 0x7f75240712d0 2024-05-26T22:55:47.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.049+0000 7f752a000640 1 -- 172.21.15.131:0/2667397517 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75240712d0 msgr2=0x7f752406d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:47.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.049+0000 7f752a000640 1 --2- 172.21.15.131:0/2667397517 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75240712d0 0x7f752406d250 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f75100097a0 tx=0x7f751002a1e0 comp rx=0 tx=0).stop 2024-05-26T22:55:47.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.049+0000 7f752a000640 1 -- 172.21.15.131:0/2667397517 shutdown_connections 2024-05-26T22:55:47.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.049+0000 7f752a000640 1 --2- 172.21.15.131:0/2667397517 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75240712d0 0x7f752406d250 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:47.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.049+0000 7f752a000640 1 -- 172.21.15.131:0/2667397517 >> 172.21.15.131:0/2667397517 conn(0x7f75240b5660 msgr2=0x7f75240b5a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:47.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.049+0000 7f752a000640 1 -- 172.21.15.131:0/2667397517 shutdown_connections 2024-05-26T22:55:47.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.049+0000 7f752a000640 1 -- 172.21.15.131:0/2667397517 wait complete. 2024-05-26T22:55:47.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.051+0000 7f752a000640 1 Processor -- start 2024-05-26T22:55:47.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.051+0000 7f752a000640 1 -- start start 2024-05-26T22:55:47.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.052+0000 7f752a000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f752406d790 0x7f752415ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:47.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.052+0000 7f752a000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75240712d0 0x7f75241f1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:47.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.052+0000 7f752a000640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7524145a10 0x7f75241f15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:47.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.052+0000 7f752a000640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f75240cf070 con 0x7f75240712d0 2024-05-26T22:55:47.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.052+0000 7f752a000640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f752406dbc0 con 0x7f7524145a10 2024-05-26T22:55:47.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.052+0000 7f752a000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f7524145df0 con 0x7f752406d790 2024-05-26T22:55:47.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.052+0000 7f7523e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75240712d0 0x7f75241f1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:47.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.052+0000 7f7523e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75240712d0 0x7f75241f1080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38358/0 (socket says 172.21.15.131:38358) 2024-05-26T22:55:47.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.052+0000 7f7523e00640 1 -- 172.21.15.131:0/3465755081 learned_addr learned my addr 172.21.15.131:0/3465755081 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:55:47.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.052+0000 7f7523e00640 1 -- 172.21.15.131:0/3465755081 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f752406d790 msgr2=0x7f752415ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:47.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.052+0000 7f7528c00640 1 --2- 172.21.15.131:0/3465755081 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f752406d790 0x7f752415ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:47.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.052+0000 7f7529600640 1 --2- 172.21.15.131:0/3465755081 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7524145a10 0x7f75241f15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:47.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.052+0000 7f7523e00640 1 --2- 172.21.15.131:0/3465755081 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f752406d790 0x7f752415ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:47.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.052+0000 7f7523e00640 1 -- 172.21.15.131:0/3465755081 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7524145a10 msgr2=0x7f75241f15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:47.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.052+0000 7f7523e00640 1 --2- 172.21.15.131:0/3465755081 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7524145a10 0x7f75241f15c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:47.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.052+0000 7f7523e00640 1 -- 172.21.15.131:0/3465755081 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f752413d480 con 0x7f75240712d0 2024-05-26T22:55:47.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.053+0000 7f7523e00640 1 --2- 172.21.15.131:0/3465755081 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75240712d0 0x7f75241f1080 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f75100097a0 tx=0x7f75100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:47.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.053+0000 7f7520c00640 1 -- 172.21.15.131:0/3465755081 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7510041070 con 0x7f75240712d0 2024-05-26T22:55:47.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.053+0000 7f7520c00640 1 -- 172.21.15.131:0/3465755081 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f751003c870 con 0x7f75240712d0 2024-05-26T22:55:47.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.053+0000 7f752a000640 1 -- 172.21.15.131:0/3465755081 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75241f5410 con 0x7f75240712d0 2024-05-26T22:55:47.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.053+0000 7f7520c00640 1 -- 172.21.15.131:0/3465755081 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7510037040 con 0x7f75240712d0 2024-05-26T22:55:47.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.053+0000 7f752a000640 1 -- 172.21.15.131:0/3465755081 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75241f59d0 con 0x7f75240712d0 2024-05-26T22:55:47.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.054+0000 7f752a000640 1 -- 172.21.15.131:0/3465755081 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75240694e0 con 0x7f75240712d0 2024-05-26T22:55:47.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.055+0000 7f7520c00640 1 -- 172.21.15.131:0/3465755081 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7510044070 con 0x7f75240712d0 2024-05-26T22:55:47.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.056+0000 7f7520c00640 1 --2- 172.21.15.131:0/3465755081 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f750406be30 0x7f750406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:47.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.056+0000 7f7520c00640 1 -- 172.21.15.131:0/3465755081 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5115+0+0 (secure 0 0 0) 0x7f75100abae0 con 0x7f75240712d0 2024-05-26T22:55:47.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.057+0000 7f7528c00640 1 --2- 172.21.15.131:0/3465755081 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f750406be30 0x7f750406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:47.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.057+0000 7f7528c00640 1 --2- 172.21.15.131:0/3465755081 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f750406be30 0x7f750406e2f0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f751c0012e0 tx=0x7f751c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:47.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.061+0000 7f7520c00640 1 -- 172.21.15.131:0/3465755081 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f751007a7f0 con 0x7f75240712d0 2024-05-26T22:55:47.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.260+0000 7f752a000640 1 -- 172.21.15.131:0/3465755081 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f75240b5d20 con 0x7f75240712d0 2024-05-26T22:55:47.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.260+0000 7f7520c00640 1 -- 172.21.15.131:0/3465755081 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v98) ==== 93+0+10 (secure 0 0 0) 0x7f751007a0f0 con 0x7f75240712d0 2024-05-26T22:55:47.263 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2024-05-26T22:55:47.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.263+0000 7f752a000640 1 -- 172.21.15.131:0/3465755081 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f750406be30 msgr2=0x7f750406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:47.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.263+0000 7f752a000640 1 --2- 172.21.15.131:0/3465755081 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f750406be30 0x7f750406e2f0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f751c0012e0 tx=0x7f751c009040 comp rx=0 tx=0).stop 2024-05-26T22:55:47.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.263+0000 7f752a000640 1 -- 172.21.15.131:0/3465755081 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75240712d0 msgr2=0x7f75241f1080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:47.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.263+0000 7f752a000640 1 --2- 172.21.15.131:0/3465755081 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75240712d0 0x7f75241f1080 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f75100097a0 tx=0x7f75100046c0 comp rx=0 tx=0).stop 2024-05-26T22:55:47.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.263+0000 7f752a000640 1 -- 172.21.15.131:0/3465755081 shutdown_connections 2024-05-26T22:55:47.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.263+0000 7f752a000640 1 --2- 172.21.15.131:0/3465755081 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f750406be30 0x7f750406e2f0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:47.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.263+0000 7f752a000640 1 --2- 172.21.15.131:0/3465755081 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7524145a10 0x7f75241f15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:47.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.263+0000 7f752a000640 1 --2- 172.21.15.131:0/3465755081 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75240712d0 0x7f75241f1080 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:47.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.263+0000 7f752a000640 1 --2- 172.21.15.131:0/3465755081 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f752406d790 0x7f752415ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:47.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.263+0000 7f752a000640 1 -- 172.21.15.131:0/3465755081 >> 172.21.15.131:0/3465755081 conn(0x7f75240b5660 msgr2=0x7f75240af8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:47.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.264+0000 7f752a000640 1 -- 172.21.15.131:0/3465755081 shutdown_connections 2024-05-26T22:55:47.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.264+0000 7f752a000640 1 -- 172.21.15.131:0/3465755081 wait complete. 2024-05-26T22:55:47.283 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_exports.TestEphemeralPins.test_ephemeral_pin_dist_override_pin' 2024-05-26T22:55:47.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.484+0000 7f2465200640 1 Processor -- start 2024-05-26T22:55:47.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.484+0000 7f2465200640 1 -- start start 2024-05-26T22:55:47.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.486+0000 7f2465200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f24600ad250 0x7f24600b3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:47.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.486+0000 7f2465200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f24601569d0 0x7f24600af790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:47.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.486+0000 7f2465200640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f246015cb20 con 0x7f246006ee20 2024-05-26T22:55:47.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.486+0000 7f2465200640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f246015ce00 con 0x7f24601569d0 2024-05-26T22:55:47.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.486+0000 7f2465200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f246015d0e0 con 0x7f24600ad250 2024-05-26T22:55:47.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.486+0000 7f245e800640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f24601569d0 0x7f24600af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:47.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.486+0000 7f245de00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f246006ee20 0x7f24601562c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:47796/0 (socket says 172.21.15.131:47796) 2024-05-26T22:55:47.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.486+0000 7f245e800640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f24601569d0 0x7f24600af790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38368/0 (socket says 172.21.15.131:38368) 2024-05-26T22:55:47.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.486+0000 7f245de00640 1 -- 172.21.15.131:0/865600414 learned_addr learned my addr 172.21.15.131:0/865600414 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:55:47.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.487+0000 7f2457e00640 1 --2- 172.21.15.131:0/865600414 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f24600ad250 0x7f24600b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:47.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.487+0000 7f245e800640 1 -- 172.21.15.131:0/865600414 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f24600ad250 msgr2=0x7f24600b3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:47.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.487+0000 7f245e800640 1 --2- 172.21.15.131:0/865600414 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f24600ad250 0x7f24600b3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:47.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.487+0000 7f245e800640 1 -- 172.21.15.131:0/865600414 >> v1:172.21.15.131:6789/0 conn(0x7f246006ee20 legacy=0x7f24601562c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:47.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.487+0000 7f245d400640 1 -- 172.21.15.131:0/865600414 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2838076606 0 0) 0x7f246015cb20 con 0x7f246006ee20 2024-05-26T22:55:47.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.487+0000 7f245e800640 1 -- 172.21.15.131:0/865600414 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f246015d3c0 con 0x7f24601569d0 2024-05-26T22:55:47.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.488+0000 7f245e800640 1 --2- 172.21.15.131:0/865600414 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f24601569d0 0x7f24600af790 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f2458009dc0 tx=0x7f245802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2400fad468a3a582 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:47.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.488+0000 7f245d400640 1 -- 172.21.15.131:0/865600414 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f245802ae60 con 0x7f24601569d0 2024-05-26T22:55:47.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.488+0000 7f245d400640 1 -- 172.21.15.131:0/865600414 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2458007740 con 0x7f24601569d0 2024-05-26T22:55:47.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.488+0000 7f245d400640 1 -- 172.21.15.131:0/865600414 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2458007b10 con 0x7f24601569d0 2024-05-26T22:55:47.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.489+0000 7f2465200640 1 -- 172.21.15.131:0/865600414 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f24601569d0 msgr2=0x7f24600af790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:47.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.489+0000 7f2465200640 1 --2- 172.21.15.131:0/865600414 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f24601569d0 0x7f24600af790 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f2458009dc0 tx=0x7f245802a680 comp rx=0 tx=0).stop 2024-05-26T22:55:47.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.489+0000 7f2465200640 1 -- 172.21.15.131:0/865600414 shutdown_connections 2024-05-26T22:55:47.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.489+0000 7f2465200640 1 --2- 172.21.15.131:0/865600414 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f24601569d0 0x7f24600af790 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:47.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.489+0000 7f2465200640 1 --2- 172.21.15.131:0/865600414 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f24600ad250 0x7f24600b3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:47.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.489+0000 7f2465200640 1 -- 172.21.15.131:0/865600414 >> 172.21.15.131:0/865600414 conn(0x7f24600b1e90 msgr2=0x7f24600b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:47.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.489+0000 7f2465200640 1 -- 172.21.15.131:0/865600414 shutdown_connections 2024-05-26T22:55:47.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.489+0000 7f2465200640 1 -- 172.21.15.131:0/865600414 wait complete. 2024-05-26T22:55:47.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.491+0000 7f2465200640 1 Processor -- start 2024-05-26T22:55:47.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.491+0000 7f2465200640 1 -- start start 2024-05-26T22:55:47.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.491+0000 7f2465200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f246006ee20 0x7f246014c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:47.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.492+0000 7f2465200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f24600ad250 0x7f2460071590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:47.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.492+0000 7f2465200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f24601569d0 0x7f246014ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:47.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.492+0000 7f2465200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f246015c900 con 0x7f24600ad250 2024-05-26T22:55:47.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.492+0000 7f2465200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f246015c780 con 0x7f246006ee20 2024-05-26T22:55:47.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.492+0000 7f2465200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f246006f200 con 0x7f24601569d0 2024-05-26T22:55:47.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.492+0000 7f2457e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f24600ad250 0x7f2460071590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:47.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.492+0000 7f2457e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f24600ad250 0x7f2460071590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38374/0 (socket says 172.21.15.131:38374) 2024-05-26T22:55:47.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.492+0000 7f2457e00640 1 -- 172.21.15.131:0/957365786 learned_addr learned my addr 172.21.15.131:0/957365786 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:55:47.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.492+0000 7f245de00640 1 --2- 172.21.15.131:0/957365786 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f246006ee20 0x7f246014c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:47.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.492+0000 7f2457e00640 1 -- 172.21.15.131:0/957365786 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f24601569d0 msgr2=0x7f246014ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:47.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.492+0000 7f2457e00640 1 --2- 172.21.15.131:0/957365786 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f24601569d0 0x7f246014ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:47.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.492+0000 7f2457e00640 1 -- 172.21.15.131:0/957365786 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f246006ee20 msgr2=0x7f246014c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:47.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.492+0000 7f2457e00640 1 --2- 172.21.15.131:0/957365786 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f246006ee20 0x7f246014c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:47.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.492+0000 7f2457e00640 1 -- 172.21.15.131:0/957365786 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24600b0790 con 0x7f24600ad250 2024-05-26T22:55:47.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.493+0000 7f2457e00640 1 --2- 172.21.15.131:0/957365786 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f24600ad250 0x7f2460071590 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f244800d830 tx=0x7f244800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:47.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.493+0000 7f2456000640 1 -- 172.21.15.131:0/957365786 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f244800b6c0 con 0x7f24600ad250 2024-05-26T22:55:47.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.493+0000 7f2456000640 1 -- 172.21.15.131:0/957365786 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2448019070 con 0x7f24600ad250 2024-05-26T22:55:47.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.493+0000 7f2456000640 1 -- 172.21.15.131:0/957365786 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f244800ba80 con 0x7f24600ad250 2024-05-26T22:55:47.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.493+0000 7f2465200640 1 -- 172.21.15.131:0/957365786 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24600b0a70 con 0x7f24600ad250 2024-05-26T22:55:47.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.493+0000 7f2465200640 1 -- 172.21.15.131:0/957365786 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f246014bfa0 con 0x7f24600ad250 2024-05-26T22:55:47.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.494+0000 7f2465200640 1 -- 172.21.15.131:0/957365786 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24600694e0 con 0x7f24600ad250 2024-05-26T22:55:47.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.495+0000 7f2456000640 1 -- 172.21.15.131:0/957365786 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f244801e460 con 0x7f24600ad250 2024-05-26T22:55:47.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.496+0000 7f2456000640 1 --2- 172.21.15.131:0/957365786 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f243c06be30 0x7f243c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:47.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.496+0000 7f2456000640 1 -- 172.21.15.131:0/957365786 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5115+0+0 (secure 0 0 0) 0x7f244808b2f0 con 0x7f24600ad250 2024-05-26T22:55:47.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.496+0000 7f245de00640 1 --2- 172.21.15.131:0/957365786 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f243c06be30 0x7f243c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:47.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.497+0000 7f245de00640 1 --2- 172.21.15.131:0/957365786 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f243c06be30 0x7f243c06e2f0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f2450007fd0 tx=0x7f2450009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:47.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.501+0000 7f2456000640 1 -- 172.21.15.131:0/957365786 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f2448059ef0 con 0x7f24600ad250 2024-05-26T22:55:47.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:47.770+0000 7f2465200640 1 -- 172.21.15.131:0/957365786 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_exports.TestEphemeralPins.test_ephemeral_pin_dist_override_pin"]} v 0) -- 0x7f246006c590 con 0x7f24600ad250 2024-05-26T22:55:48.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:48.387+0000 7f2456000640 1 -- 172.21.15.131:0/957365786 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_exports.TestEphemeralPins.test_ephemeral_pin_dist_override_pin"]}]=0 v376) ==== 162+0+0 (secure 0 0 0) 0x7f24480597f0 con 0x7f24600ad250 2024-05-26T22:55:48.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:48.393+0000 7f2465200640 1 -- 172.21.15.131:0/957365786 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f243c06be30 msgr2=0x7f243c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:48.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:48.393+0000 7f2465200640 1 --2- 172.21.15.131:0/957365786 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f243c06be30 0x7f243c06e2f0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f2450007fd0 tx=0x7f2450009040 comp rx=0 tx=0).stop 2024-05-26T22:55:48.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:48.394+0000 7f2465200640 1 -- 172.21.15.131:0/957365786 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f24600ad250 msgr2=0x7f2460071590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:48.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:48.394+0000 7f2465200640 1 --2- 172.21.15.131:0/957365786 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f24600ad250 0x7f2460071590 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f244800d830 tx=0x7f244800dcf0 comp rx=0 tx=0).stop 2024-05-26T22:55:48.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:48.394+0000 7f2465200640 1 -- 172.21.15.131:0/957365786 shutdown_connections 2024-05-26T22:55:48.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:48.394+0000 7f2465200640 1 --2- 172.21.15.131:0/957365786 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f24601569d0 0x7f246014ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:48.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:48.394+0000 7f2465200640 1 --2- 172.21.15.131:0/957365786 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f243c06be30 0x7f243c06e2f0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:48.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:48.394+0000 7f2465200640 1 --2- 172.21.15.131:0/957365786 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f24600ad250 0x7f2460071590 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:48.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:48.394+0000 7f2465200640 1 --2- 172.21.15.131:0/957365786 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f246006ee20 0x7f246014c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:48.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:48.394+0000 7f2465200640 1 -- 172.21.15.131:0/957365786 >> 172.21.15.131:0/957365786 conn(0x7f24600b1e90 msgr2=0x7f24600b3750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:48.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:48.394+0000 7f2465200640 1 -- 172.21.15.131:0/957365786 shutdown_connections 2024-05-26T22:55:48.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:48.394+0000 7f2465200640 1 -- 172.21.15.131:0/957365786 wait complete. 2024-05-26T22:55:48.418 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-26T22:55:48.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:48.627+0000 7f31fb200640 1 Processor -- start 2024-05-26T22:55:48.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:48.627+0000 7f31fb200640 1 -- start start 2024-05-26T22:55:48.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:48.629+0000 7f31fb200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f31fc0712d0 0x7f31fc06d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:48.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:48.629+0000 7f31fb200640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f31fc06d790 0x7f31fc0cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:48.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:48.629+0000 7f31fb200640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31fc072d60 con 0x7f31fc145a10 2024-05-26T22:55:48.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:48.629+0000 7f31fb200640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f31fc15c760 con 0x7f31fc06d790 2024-05-26T22:55:48.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:48.629+0000 7f31fb200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f31fc072a80 con 0x7f31fc0712d0 2024-05-26T22:55:48.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:48.630+0000 7f31f9400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f31fc0712d0 0x7f31fc06d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:48.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:48.630+0000 7f31fa800640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f31fc06d790 0x7f31fc0cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:48.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:48.630+0000 7f31fa800640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f31fc06d790 0x7f31fc0cf590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.131:52488/0 (socket says 172.21.15.131:52488) 2024-05-26T22:55:48.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:48.630+0000 7f31fa800640 1 -- 172.21.15.131:0/3079958865 learned_addr learned my addr 172.21.15.131:0/3079958865 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:55:48.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:48.630+0000 7f31f9400640 1 -- 172.21.15.131:0/3079958865 >> v2:172.21.15.187:3300/0 conn(0x7f31fc06d790 msgr2=0x7f31fc0cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:48.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:48.630+0000 7f31f9400640 1 --2- 172.21.15.131:0/3079958865 >> v2:172.21.15.187:3300/0 conn(0x7f31fc06d790 0x7f31fc0cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:48.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:48.630+0000 7f31f9400640 1 -- 172.21.15.131:0/3079958865 >> v1:172.21.15.187:6789/0 conn(0x7f31fc145a10 legacy=0x7f31fc072180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:48.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:48.630+0000 7f31f9400640 1 -- 172.21.15.131:0/3079958865 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31fc0ceab0 con 0x7f31fc0712d0 2024-05-26T22:55:48.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:48.632+0000 7f31f9400640 1 --2- 172.21.15.131:0/3079958865 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f31fc0712d0 0x7f31fc06d250 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f31e80097a0 tx=0x7f31e802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44b3d1e9e0667710 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:48.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:48.632+0000 7f31f8a00640 1 -- 172.21.15.131:0/3079958865 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31e8037070 con 0x7f31fc0712d0 2024-05-26T22:55:48.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:48.633+0000 7f31f8a00640 1 -- 172.21.15.131:0/3079958865 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31e8032440 con 0x7f31fc0712d0 2024-05-26T22:55:48.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:48.633+0000 7f31f8a00640 1 -- 172.21.15.131:0/3079958865 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31e8032810 con 0x7f31fc0712d0 2024-05-26T22:55:48.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:48.633+0000 7f31fb200640 1 -- 172.21.15.131:0/3079958865 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f31fc0712d0 msgr2=0x7f31fc06d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:48.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:48.633+0000 7f31fb200640 1 --2- 172.21.15.131:0/3079958865 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f31fc0712d0 0x7f31fc06d250 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f31e80097a0 tx=0x7f31e802a180 comp rx=0 tx=0).stop 2024-05-26T22:55:48.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:48.633+0000 7f31fb200640 1 -- 172.21.15.131:0/3079958865 shutdown_connections 2024-05-26T22:55:48.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:48.633+0000 7f31fb200640 1 --2- 172.21.15.131:0/3079958865 >> v2:172.21.15.187:3300/0 conn(0x7f31fc06d790 0x7f31fc0cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:48.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:48.633+0000 7f31fb200640 1 --2- 172.21.15.131:0/3079958865 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f31fc0712d0 0x7f31fc06d250 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:48.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:48.633+0000 7f31fb200640 1 -- 172.21.15.131:0/3079958865 >> 172.21.15.131:0/3079958865 conn(0x7f31fc0b5660 msgr2=0x7f31fc0b5a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:48.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:48.633+0000 7f31fb200640 1 -- 172.21.15.131:0/3079958865 shutdown_connections 2024-05-26T22:55:48.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:48.634+0000 7f31fb200640 1 -- 172.21.15.131:0/3079958865 wait complete. 2024-05-26T22:55:48.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:48.636+0000 7f31fb200640 1 Processor -- start 2024-05-26T22:55:48.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:48.636+0000 7f31fb200640 1 -- start start 2024-05-26T22:55:48.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:48.637+0000 7f31fb200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f31fc06d790 0x7f31fc15c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:48.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:48.637+0000 7f31fb200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f31fc0712d0 0x7f31fc13e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:48.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:48.637+0000 7f31fb200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f31fc145a10 0x7f31fc13ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:48.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:48.637+0000 7f31fb200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f31fc145df0 con 0x7f31fc06d790 2024-05-26T22:55:48.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:48.637+0000 7f31fb200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f31fc0cf070 con 0x7f31fc145a10 2024-05-26T22:55:48.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:48.637+0000 7f31fb200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f31fc0cfad0 con 0x7f31fc0712d0 2024-05-26T22:55:48.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:48.638+0000 7f31f9e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f31fc06d790 0x7f31fc15c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:48.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:48.638+0000 7f31f9e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f31fc06d790 0x7f31fc15c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38386/0 (socket says 172.21.15.131:38386) 2024-05-26T22:55:48.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:48.638+0000 7f31f9e00640 1 -- 172.21.15.131:0/3249899874 learned_addr learned my addr 172.21.15.131:0/3249899874 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:55:48.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:48.638+0000 7f31f9400640 1 --2- 172.21.15.131:0/3249899874 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f31fc0712d0 0x7f31fc13e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:48.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:48.638+0000 7f31fa800640 1 --2- 172.21.15.131:0/3249899874 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f31fc145a10 0x7f31fc13ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:48.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:48.638+0000 7f31f9e00640 1 -- 172.21.15.131:0/3249899874 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f31fc0712d0 msgr2=0x7f31fc13e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:48.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:48.638+0000 7f31f9e00640 1 --2- 172.21.15.131:0/3249899874 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f31fc0712d0 0x7f31fc13e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:48.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:48.638+0000 7f31f9e00640 1 -- 172.21.15.131:0/3249899874 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f31fc145a10 msgr2=0x7f31fc13ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:48.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:48.638+0000 7f31f9e00640 1 --2- 172.21.15.131:0/3249899874 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f31fc145a10 0x7f31fc13ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:48.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:48.638+0000 7f31f9e00640 1 -- 172.21.15.131:0/3249899874 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31fc15d050 con 0x7f31fc06d790 2024-05-26T22:55:48.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:48.638+0000 7f31f9400640 1 --2- 172.21.15.131:0/3249899874 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f31fc0712d0 0x7f31fc13e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:55:48.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:48.638+0000 7f31fa800640 1 --2- 172.21.15.131:0/3249899874 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f31fc145a10 0x7f31fc13ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:55:48.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:48.638+0000 7f31f9e00640 1 --2- 172.21.15.131:0/3249899874 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f31fc06d790 0x7f31fc15c8e0 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f31ec00c3c0 tx=0x7f31ec00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:48.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:48.639+0000 7f31f2000640 1 -- 172.21.15.131:0/3249899874 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31ec0052d0 con 0x7f31fc06d790 2024-05-26T22:55:48.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:48.639+0000 7f31f2000640 1 -- 172.21.15.131:0/3249899874 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31ec005490 con 0x7f31fc06d790 2024-05-26T22:55:48.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:48.639+0000 7f31fb200640 1 -- 172.21.15.131:0/3249899874 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31fc15d330 con 0x7f31fc06d790 2024-05-26T22:55:48.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:48.639+0000 7f31f2000640 1 -- 172.21.15.131:0/3249899874 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31ec007260 con 0x7f31fc06d790 2024-05-26T22:55:48.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:48.639+0000 7f31fb200640 1 -- 172.21.15.131:0/3249899874 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31fc15d8f0 con 0x7f31fc06d790 2024-05-26T22:55:48.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:48.640+0000 7f31fb200640 1 -- 172.21.15.131:0/3249899874 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31fc0694e0 con 0x7f31fc06d790 2024-05-26T22:55:48.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:48.641+0000 7f31f2000640 1 -- 172.21.15.131:0/3249899874 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f31ec007400 con 0x7f31fc06d790 2024-05-26T22:55:48.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:48.642+0000 7f31f2000640 1 --2- 172.21.15.131:0/3249899874 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f31dc06be30 0x7f31dc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:48.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:48.642+0000 7f31f2000640 1 -- 172.21.15.131:0/3249899874 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5115+0+0 (secure 0 0 0) 0x7f31ec08a0f0 con 0x7f31fc06d790 2024-05-26T22:55:48.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:48.642+0000 7f31f9400640 1 --2- 172.21.15.131:0/3249899874 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f31dc06be30 0x7f31dc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:48.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:48.643+0000 7f31f9400640 1 --2- 172.21.15.131:0/3249899874 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f31dc06be30 0x7f31dc06e2f0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f31e8034000 tx=0x7f31e802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:48.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:48.647+0000 7f31f2000640 1 -- 172.21.15.131:0/3249899874 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f31ec058cf0 con 0x7f31fc06d790 2024-05-26T22:55:48.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:48.847+0000 7f31fb200640 1 -- 172.21.15.131:0/3249899874 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f31fc0cd900 con 0x7f31fc06d790 2024-05-26T22:55:48.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:48.854+0000 7f31f2000640 1 -- 172.21.15.131:0/3249899874 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v59)=0 v59) ==== 122+0+0 (secure 0 0 0) 0x7f31ec0585f0 con 0x7f31fc06d790 2024-05-26T22:55:48.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:48.857+0000 7f31fb200640 1 -- 172.21.15.131:0/3249899874 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f31dc06be30 msgr2=0x7f31dc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:48.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:48.857+0000 7f31fb200640 1 --2- 172.21.15.131:0/3249899874 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f31dc06be30 0x7f31dc06e2f0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f31e8034000 tx=0x7f31e802e000 comp rx=0 tx=0).stop 2024-05-26T22:55:48.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:48.857+0000 7f31fb200640 1 -- 172.21.15.131:0/3249899874 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f31fc06d790 msgr2=0x7f31fc15c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:48.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:48.857+0000 7f31fb200640 1 --2- 172.21.15.131:0/3249899874 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f31fc06d790 0x7f31fc15c8e0 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f31ec00c3c0 tx=0x7f31ec00c880 comp rx=0 tx=0).stop 2024-05-26T22:55:48.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:48.857+0000 7f31fb200640 1 -- 172.21.15.131:0/3249899874 shutdown_connections 2024-05-26T22:55:48.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:48.857+0000 7f31fb200640 1 --2- 172.21.15.131:0/3249899874 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f31fc0712d0 0x7f31fc13e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:48.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:48.857+0000 7f31fb200640 1 --2- 172.21.15.131:0/3249899874 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f31dc06be30 0x7f31dc06e2f0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:48.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:48.857+0000 7f31fb200640 1 --2- 172.21.15.131:0/3249899874 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f31fc145a10 0x7f31fc13ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:48.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:48.857+0000 7f31fb200640 1 --2- 172.21.15.131:0/3249899874 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f31fc06d790 0x7f31fc15c8e0 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:48.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:48.857+0000 7f31fb200640 1 -- 172.21.15.131:0/3249899874 >> 172.21.15.131:0/3249899874 conn(0x7f31fc0b5660 msgr2=0x7f31fc0b0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:48.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:48.858+0000 7f31fb200640 1 -- 172.21.15.131:0/3249899874 shutdown_connections 2024-05-26T22:55:48.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:48.858+0000 7f31fb200640 1 -- 172.21.15.131:0/3249899874 wait complete. 2024-05-26T22:55:48.876 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-05-26T22:55:48.876 DEBUG:teuthology.orchestra.run.smithi131:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-26T22:55:48.989 DEBUG:teuthology.orchestra.run.smithi187:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-26T22:55:49.107 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-05-26T22:55:49.107 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-26T22:55:49.107 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-26T22:55:49.167 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-26T22:55:49.167 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-05-26T22:55:49.167 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-26T22:55:49.223 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-26T22:55:49.223 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-26T22:55:49.224 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-26T22:55:49.280 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-26T22:55:49.280 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-05-26T22:55:49.281 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-26T22:55:49.337 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-26T22:55:49.337 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-26T22:55:49.337 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-26T22:55:49.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:49.550+0000 7f21d0600640 1 Processor -- start 2024-05-26T22:55:49.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:49.551+0000 7f21d0600640 1 -- start start 2024-05-26T22:55:49.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:49.552+0000 7f21d0600640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f21c80b4090 0x7f21c815f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:49.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:49.552+0000 7f21d0600640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f21c815fe60 0x7f21c81602f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:49.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:49.552+0000 7f21d0600640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21c80b28e0 con 0x7f21c806b310 2024-05-26T22:55:49.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:49.552+0000 7f21d0600640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f21c80b6400 con 0x7f21c80b4090 2024-05-26T22:55:49.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:49.552+0000 7f21d0600640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f21c80b5e40 con 0x7f21c815fe60 2024-05-26T22:55:49.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:49.553+0000 7f21cd200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f21c80b4090 0x7f21c815f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:49.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:49.553+0000 7f21cd200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f21c80b4090 0x7f21c815f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38390/0 (socket says 172.21.15.131:38390) 2024-05-26T22:55:49.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:49.553+0000 7f21cd200640 1 -- 172.21.15.131:0/261049094 learned_addr learned my addr 172.21.15.131:0/261049094 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:55:49.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:49.553+0000 7f21ce600640 1 --2- 172.21.15.131:0/261049094 >> v2:172.21.15.187:3300/0 conn(0x7f21c815fe60 0x7f21c81602f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:49.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:49.553+0000 7f21cd200640 1 -- 172.21.15.131:0/261049094 >> v2:172.21.15.187:3300/0 conn(0x7f21c815fe60 msgr2=0x7f21c81602f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:49.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:49.553+0000 7f21cd200640 1 --2- 172.21.15.131:0/261049094 >> v2:172.21.15.187:3300/0 conn(0x7f21c815fe60 0x7f21c81602f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:49.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:49.553+0000 7f21cd200640 1 -- 172.21.15.131:0/261049094 >> v1:172.21.15.187:6789/0 conn(0x7f21c806b310 legacy=0x7f21c806b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:49.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:49.553+0000 7f21cd200640 1 -- 172.21.15.131:0/261049094 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21c80b6120 con 0x7f21c80b4090 2024-05-26T22:55:49.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:49.554+0000 7f21cd200640 1 --2- 172.21.15.131:0/261049094 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f21c80b4090 0x7f21c815f920 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f21c80b2180 tx=0x7f21b402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af903bd80e5395b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:49.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:49.555+0000 7f21c7e00640 1 -- 172.21.15.131:0/261049094 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21b4037070 con 0x7f21c80b4090 2024-05-26T22:55:49.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:49.555+0000 7f21c7e00640 1 -- 172.21.15.131:0/261049094 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21b4032440 con 0x7f21c80b4090 2024-05-26T22:55:49.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:49.555+0000 7f21c7e00640 1 -- 172.21.15.131:0/261049094 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21b4032810 con 0x7f21c80b4090 2024-05-26T22:55:49.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:49.555+0000 7f21d0600640 1 -- 172.21.15.131:0/261049094 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f21c80b4090 msgr2=0x7f21c815f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:49.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:49.555+0000 7f21d0600640 1 --2- 172.21.15.131:0/261049094 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f21c80b4090 0x7f21c815f920 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f21c80b2180 tx=0x7f21b402a230 comp rx=0 tx=0).stop 2024-05-26T22:55:49.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:49.555+0000 7f21d0600640 1 -- 172.21.15.131:0/261049094 shutdown_connections 2024-05-26T22:55:49.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:49.555+0000 7f21d0600640 1 --2- 172.21.15.131:0/261049094 >> v2:172.21.15.187:3300/0 conn(0x7f21c815fe60 0x7f21c81602f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:49.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:49.555+0000 7f21d0600640 1 --2- 172.21.15.131:0/261049094 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f21c80b4090 0x7f21c815f920 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:49.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:49.555+0000 7f21d0600640 1 -- 172.21.15.131:0/261049094 >> 172.21.15.131:0/261049094 conn(0x7f21c80b65b0 msgr2=0x7f21c80b69c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:49.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:49.556+0000 7f21d0600640 1 -- 172.21.15.131:0/261049094 shutdown_connections 2024-05-26T22:55:49.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:49.556+0000 7f21d0600640 1 -- 172.21.15.131:0/261049094 wait complete. 2024-05-26T22:55:49.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:49.557+0000 7f21d0600640 1 Processor -- start 2024-05-26T22:55:49.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:49.557+0000 7f21d0600640 1 -- start start 2024-05-26T22:55:49.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:49.558+0000 7f21d0600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f21c806b310 0x7f21c809ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:49.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:49.558+0000 7f21d0600640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f21c80b4090 0x7f21c81543c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:49.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:49.558+0000 7f21d0600640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f21c815fe60 0x7f21c80afa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:49.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:49.558+0000 7f21d0600640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f21c81480d0 con 0x7f21c806b310 2024-05-26T22:55:49.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:49.558+0000 7f21d0600640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f21c8147f50 con 0x7f21c815fe60 2024-05-26T22:55:49.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:49.558+0000 7f21d0600640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f21c80b2500 con 0x7f21c80b4090 2024-05-26T22:55:49.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:49.559+0000 7f21cdc00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f21c806b310 0x7f21c809ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:49.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:49.559+0000 7f21cdc00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f21c806b310 0x7f21c809ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38398/0 (socket says 172.21.15.131:38398) 2024-05-26T22:55:49.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:49.559+0000 7f21cdc00640 1 -- 172.21.15.131:0/53657455 learned_addr learned my addr 172.21.15.131:0/53657455 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:55:49.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:49.559+0000 7f21ce600640 1 --2- 172.21.15.131:0/53657455 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f21c815fe60 0x7f21c80afa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:49.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:49.559+0000 7f21cd200640 1 --2- 172.21.15.131:0/53657455 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f21c80b4090 0x7f21c81543c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:49.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:49.559+0000 7f21cdc00640 1 -- 172.21.15.131:0/53657455 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f21c80b4090 msgr2=0x7f21c81543c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:49.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:49.559+0000 7f21cdc00640 1 --2- 172.21.15.131:0/53657455 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f21c80b4090 0x7f21c81543c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:49.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:49.559+0000 7f21cdc00640 1 -- 172.21.15.131:0/53657455 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f21c815fe60 msgr2=0x7f21c80afa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:49.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:49.559+0000 7f21cdc00640 1 --2- 172.21.15.131:0/53657455 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f21c815fe60 0x7f21c80afa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:49.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:49.559+0000 7f21cdc00640 1 -- 172.21.15.131:0/53657455 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21c814ec60 con 0x7f21c806b310 2024-05-26T22:55:49.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:49.559+0000 7f21ce600640 1 --2- 172.21.15.131:0/53657455 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f21c815fe60 0x7f21c80afa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:55:49.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:49.559+0000 7f21cdc00640 1 --2- 172.21.15.131:0/53657455 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f21c806b310 0x7f21c809ca00 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f21bc00b420 tx=0x7f21bc00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:49.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:49.560+0000 7f21c5600640 1 -- 172.21.15.131:0/53657455 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21bc011830 con 0x7f21c806b310 2024-05-26T22:55:49.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:49.560+0000 7f21c5600640 1 -- 172.21.15.131:0/53657455 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21bc016070 con 0x7f21c806b310 2024-05-26T22:55:49.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:49.560+0000 7f21c5600640 1 -- 172.21.15.131:0/53657455 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21bc011c10 con 0x7f21c806b310 2024-05-26T22:55:49.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:49.560+0000 7f21d0600640 1 -- 172.21.15.131:0/53657455 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21c814ef40 con 0x7f21c806b310 2024-05-26T22:55:49.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:49.560+0000 7f21d0600640 1 -- 172.21.15.131:0/53657455 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21c814f500 con 0x7f21c806b310 2024-05-26T22:55:49.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:49.561+0000 7f21d0600640 1 -- 172.21.15.131:0/53657455 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21c80694e0 con 0x7f21c806b310 2024-05-26T22:55:49.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:49.562+0000 7f21c5600640 1 -- 172.21.15.131:0/53657455 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f21bc01b460 con 0x7f21c806b310 2024-05-26T22:55:49.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:49.563+0000 7f21c5600640 1 --2- 172.21.15.131:0/53657455 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f21a806be60 0x7f21a806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:49.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:49.563+0000 7f21c5600640 1 -- 172.21.15.131:0/53657455 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5115+0+0 (secure 0 0 0) 0x7f21bc08a780 con 0x7f21c806b310 2024-05-26T22:55:49.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:49.563+0000 7f21cd200640 1 --2- 172.21.15.131:0/53657455 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f21a806be60 0x7f21a806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:49.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:49.564+0000 7f21cd200640 1 --2- 172.21.15.131:0/53657455 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f21a806be60 0x7f21a806e320 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f21b4034000 tx=0x7f21b402e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:49.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:49.567+0000 7f21c5600640 1 -- 172.21.15.131:0/53657455 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f21bc020020 con 0x7f21c806b310 2024-05-26T22:55:49.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:49.805+0000 7f21d0600640 1 -- 172.21.15.131:0/53657455 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f21c806d430 con 0x7f21c806b310 2024-05-26T22:55:49.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:49.806+0000 7f21c5600640 1 -- 172.21.15.131:0/53657455 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 102 v102) ==== 95+0+3885 (secure 0 0 0) 0x7f21bc0592e0 con 0x7f21c806b310 2024-05-26T22:55:49.806 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:55:49.806 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":102,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5231,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/2037149126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6835","nonce":2037149126},{"type":"v1","addr":"172.21.15.131:6837","nonce":2037149126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":89},{"gid":5366,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/3119990961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3119990961},{"type":"v1","addr":"172.21.15.187:6833","nonce":3119990961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":102},{"gid":5908,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/4169502371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":4169502371},{"type":"v1","addr":"172.21.15.187:6835","nonce":4169502371}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":102},{"gid":8211,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6836/1568985004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1568985004},{"type":"v1","addr":"172.21.15.131:6836","nonce":1568985004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":102}],"filesystems":[]} 2024-05-26T22:55:49.807 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 102 2024-05-26T22:55:49.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:49.809+0000 7f21d0600640 1 -- 172.21.15.131:0/53657455 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f21a806be60 msgr2=0x7f21a806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:49.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:49.809+0000 7f21d0600640 1 --2- 172.21.15.131:0/53657455 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f21a806be60 0x7f21a806e320 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f21b4034000 tx=0x7f21b402e000 comp rx=0 tx=0).stop 2024-05-26T22:55:49.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:49.809+0000 7f21d0600640 1 -- 172.21.15.131:0/53657455 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f21c806b310 msgr2=0x7f21c809ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:49.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:49.809+0000 7f21d0600640 1 --2- 172.21.15.131:0/53657455 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f21c806b310 0x7f21c809ca00 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f21bc00b420 tx=0x7f21bc00b8e0 comp rx=0 tx=0).stop 2024-05-26T22:55:49.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:49.809+0000 7f21d0600640 1 -- 172.21.15.131:0/53657455 shutdown_connections 2024-05-26T22:55:49.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:49.809+0000 7f21d0600640 1 --2- 172.21.15.131:0/53657455 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f21c80b4090 0x7f21c81543c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:49.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:49.809+0000 7f21d0600640 1 --2- 172.21.15.131:0/53657455 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f21a806be60 0x7f21a806e320 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:49.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:49.809+0000 7f21d0600640 1 --2- 172.21.15.131:0/53657455 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f21c815fe60 0x7f21c80afa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:49.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:49.809+0000 7f21d0600640 1 --2- 172.21.15.131:0/53657455 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f21c806b310 0x7f21c809ca00 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:49.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:49.809+0000 7f21d0600640 1 -- 172.21.15.131:0/53657455 >> 172.21.15.131:0/53657455 conn(0x7f21c80b65b0 msgr2=0x7f21c815eb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:49.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:49.809+0000 7f21d0600640 1 -- 172.21.15.131:0/53657455 shutdown_connections 2024-05-26T22:55:49.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:49.810+0000 7f21d0600640 1 -- 172.21.15.131:0/53657455 wait complete. 2024-05-26T22:55:49.828 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-26T22:55:49.829 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-26T22:55:49.829 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-26T22:55:49.829 INFO:teuthology.orchestra.run:waiting for 300 2024-05-26T22:55:49.829 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-26T22:55:49.829 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-26T22:55:49.829 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-26T22:55:49.829 INFO:teuthology.orchestra.run:waiting for 300 2024-05-26T22:55:49.830 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-26T22:55:49.830 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-26T22:55:49.830 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-26T22:55:49.830 INFO:teuthology.orchestra.run:waiting for 300 2024-05-26T22:55:49.830 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-26T22:55:49.830 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-26T22:55:49.831 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-26T22:55:49.831 INFO:teuthology.orchestra.run:waiting for 300 2024-05-26T22:55:49.852 INFO:tasks.ceph.mds.c:Stopped 2024-05-26T22:55:49.853 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-26T22:55:49.853 INFO:tasks.ceph.mds.a:Stopped 2024-05-26T22:55:49.854 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-26T22:55:49.873 INFO:tasks.ceph.mds.b:Stopped 2024-05-26T22:55:49.873 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-26T22:55:49.880 INFO:tasks.ceph.mds.b:Started 2024-05-26T22:55:49.881 DEBUG:teuthology.parallel:result is None 2024-05-26T22:55:49.898 INFO:tasks.ceph.mds.c:Started 2024-05-26T22:55:49.899 DEBUG:teuthology.parallel:result is None 2024-05-26T22:55:49.904 INFO:tasks.ceph.mds.a:Started 2024-05-26T22:55:49.905 DEBUG:teuthology.parallel:result is None 2024-05-26T22:55:50.045 INFO:tasks.ceph.mds.a.smithi131.stdout:starting mds.a at 2024-05-26T22:55:50.046 INFO:tasks.ceph.mds.c.smithi131.stdout:starting mds.c at 2024-05-26T22:55:50.051 INFO:tasks.ceph.mds.b.smithi187.stdout:starting mds.b at 2024-05-26T22:55:52.022 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2024-05-26T22:55:55.873 INFO:tasks.ceph.mds.d:Stopped 2024-05-26T22:55:55.873 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-26T22:55:55.881 INFO:tasks.ceph.mds.d:Started 2024-05-26T22:55:55.882 DEBUG:teuthology.parallel:result is None 2024-05-26T22:55:55.882 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-26T22:55:56.050 INFO:tasks.ceph.mds.d.smithi187.stdout:starting mds.d at 2024-05-26T22:55:56.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:56.096+0000 7f3201a00640 1 Processor -- start 2024-05-26T22:55:56.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:56.096+0000 7f3201a00640 1 -- start start 2024-05-26T22:55:56.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:56.098+0000 7f3201a00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f31fc06f420 0x7f31fc0cfd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:56.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:56.098+0000 7f3201a00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f31fc0cf590 0x7f31fc0d0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:56.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:56.098+0000 7f3201a00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31fc160180 con 0x7f31fc0b5450 2024-05-26T22:55:56.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:56.098+0000 7f3201a00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f31fc160d90 con 0x7f31fc0cf590 2024-05-26T22:55:56.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:56.098+0000 7f3201a00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f31fc160460 con 0x7f31fc06f420 2024-05-26T22:55:56.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:56.098+0000 7f31fb400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f31fc0cf590 0x7f31fc0d0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:56.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:56.098+0000 7f31fb400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f31fc0cf590 0x7f31fc0d0290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:36684/0 (socket says 172.21.15.131:36684) 2024-05-26T22:55:56.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:56.099+0000 7f31fb400640 1 -- 172.21.15.131:0/2550518912 learned_addr learned my addr 172.21.15.131:0/2550518912 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:55:56.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:56.099+0000 7f31f3e00640 1 --2- 172.21.15.131:0/2550518912 >> v2:172.21.15.187:3300/0 conn(0x7f31fc06f420 0x7f31fc0cfd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:56.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:56.099+0000 7f31fb400640 1 -- 172.21.15.131:0/2550518912 >> v2:172.21.15.187:3300/0 conn(0x7f31fc06f420 msgr2=0x7f31fc0cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:56.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:56.099+0000 7f31fb400640 1 --2- 172.21.15.131:0/2550518912 >> v2:172.21.15.187:3300/0 conn(0x7f31fc06f420 0x7f31fc0cfd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:56.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:56.099+0000 7f31fb400640 1 -- 172.21.15.131:0/2550518912 >> v1:172.21.15.187:6789/0 conn(0x7f31fc0b5450 legacy=0x7f31fc0b43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:56.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:56.099+0000 7f31fb400640 1 -- 172.21.15.131:0/2550518912 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31fc160740 con 0x7f31fc0cf590 2024-05-26T22:55:56.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:56.099+0000 7f31f3e00640 1 --2- 172.21.15.131:0/2550518912 >> v2:172.21.15.187:3300/0 conn(0x7f31fc06f420 0x7f31fc0cfd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-26T22:55:56.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:56.100+0000 7f31fb400640 1 --2- 172.21.15.131:0/2550518912 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f31fc0cf590 0x7f31fc0d0290 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f31fc0b35b0 tx=0x7f31f402a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae4631ffcbe333f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:56.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:56.100+0000 7f31fa000640 1 -- 172.21.15.131:0/2550518912 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31f4036070 con 0x7f31fc0cf590 2024-05-26T22:55:56.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:56.100+0000 7f31fa000640 1 -- 172.21.15.131:0/2550518912 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31f4031600 con 0x7f31fc0cf590 2024-05-26T22:55:56.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:56.100+0000 7f31fa000640 1 -- 172.21.15.131:0/2550518912 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31f40319d0 con 0x7f31fc0cf590 2024-05-26T22:55:56.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:56.101+0000 7f3201a00640 1 -- 172.21.15.131:0/2550518912 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f31fc0cf590 msgr2=0x7f31fc0d0290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:56.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:56.101+0000 7f3201a00640 1 --2- 172.21.15.131:0/2550518912 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f31fc0cf590 0x7f31fc0d0290 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f31fc0b35b0 tx=0x7f31f402a730 comp rx=0 tx=0).stop 2024-05-26T22:55:56.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:56.101+0000 7f3201a00640 1 -- 172.21.15.131:0/2550518912 shutdown_connections 2024-05-26T22:55:56.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:56.101+0000 7f3201a00640 1 --2- 172.21.15.131:0/2550518912 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f31fc0cf590 0x7f31fc0d0290 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:56.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:56.101+0000 7f3201a00640 1 --2- 172.21.15.131:0/2550518912 >> v2:172.21.15.187:3300/0 conn(0x7f31fc06f420 0x7f31fc0cfd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:56.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:56.101+0000 7f3201a00640 1 -- 172.21.15.131:0/2550518912 >> 172.21.15.131:0/2550518912 conn(0x7f31fc06bc90 msgr2=0x7f31fc0b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:56.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:56.101+0000 7f3201a00640 1 -- 172.21.15.131:0/2550518912 shutdown_connections 2024-05-26T22:55:56.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:56.101+0000 7f3201a00640 1 -- 172.21.15.131:0/2550518912 wait complete. 2024-05-26T22:55:56.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:56.103+0000 7f3201a00640 1 Processor -- start 2024-05-26T22:55:56.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:56.103+0000 7f3201a00640 1 -- start start 2024-05-26T22:55:56.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:56.104+0000 7f3201a00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f31fc06f420 0x7f31fc09ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:56.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:56.104+0000 7f3201a00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f31fc0b5450 0x7f31fc09be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:56.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:56.104+0000 7f3201a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f31fc0cf590 0x7f31fc1507b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:56.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:56.104+0000 7f3201a00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f31fc15fde0 con 0x7f31fc0cf590 2024-05-26T22:55:56.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:56.104+0000 7f3201a00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f31fc0b5830 con 0x7f31fc06f420 2024-05-26T22:55:56.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:56.104+0000 7f3201a00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f31fc15ff60 con 0x7f31fc0b5450 2024-05-26T22:55:56.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:56.104+0000 7f31fb400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f31fc0cf590 0x7f31fc1507b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:56.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:56.104+0000 7f31fb400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f31fc0cf590 0x7f31fc1507b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:36690/0 (socket says 172.21.15.131:36690) 2024-05-26T22:55:56.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:56.104+0000 7f31fb400640 1 -- 172.21.15.131:0/4197529650 learned_addr learned my addr 172.21.15.131:0/4197529650 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:55:56.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:56.104+0000 7f31faa00640 1 --2- 172.21.15.131:0/4197529650 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f31fc06f420 0x7f31fc09ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:56.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:56.105+0000 7f31f3e00640 1 --2- 172.21.15.131:0/4197529650 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f31fc0b5450 0x7f31fc09be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:56.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:56.105+0000 7f31fb400640 1 -- 172.21.15.131:0/4197529650 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f31fc0b5450 msgr2=0x7f31fc09be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:56.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:56.105+0000 7f31fb400640 1 --2- 172.21.15.131:0/4197529650 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f31fc0b5450 0x7f31fc09be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:56.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:56.105+0000 7f31fb400640 1 -- 172.21.15.131:0/4197529650 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f31fc06f420 msgr2=0x7f31fc09ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:56.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:56.105+0000 7f31fb400640 1 --2- 172.21.15.131:0/4197529650 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f31fc06f420 0x7f31fc09ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:56.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:56.105+0000 7f31fb400640 1 -- 172.21.15.131:0/4197529650 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31fc15c9f0 con 0x7f31fc0cf590 2024-05-26T22:55:56.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:56.105+0000 7f31fb400640 1 --2- 172.21.15.131:0/4197529650 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f31fc0cf590 0x7f31fc1507b0 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f31f4009f40 tx=0x7f31f4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:56.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:56.105+0000 7f31f2a00640 1 -- 172.21.15.131:0/4197529650 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31f4041020 con 0x7f31fc0cf590 2024-05-26T22:55:56.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:56.105+0000 7f31f2a00640 1 -- 172.21.15.131:0/4197529650 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31f40077a0 con 0x7f31fc0cf590 2024-05-26T22:55:56.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:56.105+0000 7f3201a00640 1 -- 172.21.15.131:0/4197529650 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31fc15ccd0 con 0x7f31fc0cf590 2024-05-26T22:55:56.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:56.105+0000 7f31f2a00640 1 -- 172.21.15.131:0/4197529650 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31f4036040 con 0x7f31fc0cf590 2024-05-26T22:55:56.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:56.106+0000 7f3201a00640 1 -- 172.21.15.131:0/4197529650 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31fc15d290 con 0x7f31fc0cf590 2024-05-26T22:55:56.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:56.107+0000 7f3201a00640 1 -- 172.21.15.131:0/4197529650 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31fc074e90 con 0x7f31fc0cf590 2024-05-26T22:55:56.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:56.108+0000 7f31f2a00640 1 -- 172.21.15.131:0/4197529650 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f31f4044020 con 0x7f31fc0cf590 2024-05-26T22:55:56.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:56.108+0000 7f31f2a00640 1 --2- 172.21.15.131:0/4197529650 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f31d806be30 0x7f31d806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:56.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:56.109+0000 7f31f2a00640 1 -- 172.21.15.131:0/4197529650 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5115+0+0 (secure 0 0 0) 0x7f31f40abba0 con 0x7f31fc0cf590 2024-05-26T22:55:56.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:56.109+0000 7f31faa00640 1 --2- 172.21.15.131:0/4197529650 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f31d806be30 0x7f31d806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:56.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:56.109+0000 7f31faa00640 1 --2- 172.21.15.131:0/4197529650 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f31d806be30 0x7f31d806e2f0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f31ec007fd0 tx=0x7f31ec009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:56.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:56.114+0000 7f31f2a00640 1 -- 172.21.15.131:0/4197529650 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f31f404b020 con 0x7f31fc0cf590 2024-05-26T22:55:56.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:56.333+0000 7f3201a00640 1 -- 172.21.15.131:0/4197529650 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f31fc0cee90 con 0x7f31fc0cf590 2024-05-26T22:55:57.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:57.454+0000 7f31f2a00640 1 -- 172.21.15.131:0/4197529650 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v103) ==== 97+0+0 (secure 0 0 0) 0x7f31f407a970 con 0x7f31fc0cf590 2024-05-26T22:55:57.454 INFO:teuthology.orchestra.run.smithi131.stderr: removed all blocklist entries 2024-05-26T22:55:57.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:57.458+0000 7f3201a00640 1 -- 172.21.15.131:0/4197529650 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f31d806be30 msgr2=0x7f31d806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:57.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:57.458+0000 7f3201a00640 1 --2- 172.21.15.131:0/4197529650 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f31d806be30 0x7f31d806e2f0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f31ec007fd0 tx=0x7f31ec009040 comp rx=0 tx=0).stop 2024-05-26T22:55:57.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:57.458+0000 7f3201a00640 1 -- 172.21.15.131:0/4197529650 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f31fc0cf590 msgr2=0x7f31fc1507b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:57.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:57.458+0000 7f3201a00640 1 --2- 172.21.15.131:0/4197529650 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f31fc0cf590 0x7f31fc1507b0 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f31f4009f40 tx=0x7f31f4004a40 comp rx=0 tx=0).stop 2024-05-26T22:55:57.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:57.459+0000 7f3201a00640 1 -- 172.21.15.131:0/4197529650 shutdown_connections 2024-05-26T22:55:57.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:57.459+0000 7f3201a00640 1 --2- 172.21.15.131:0/4197529650 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f31fc0b5450 0x7f31fc09be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:57.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:57.459+0000 7f3201a00640 1 --2- 172.21.15.131:0/4197529650 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f31d806be30 0x7f31d806e2f0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:57.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:57.459+0000 7f3201a00640 1 --2- 172.21.15.131:0/4197529650 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f31fc0cf590 0x7f31fc1507b0 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:57.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:57.459+0000 7f3201a00640 1 --2- 172.21.15.131:0/4197529650 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f31fc06f420 0x7f31fc09ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:57.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:57.459+0000 7f3201a00640 1 -- 172.21.15.131:0/4197529650 >> 172.21.15.131:0/4197529650 conn(0x7f31fc06bc90 msgr2=0x7f31fc072910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:57.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:57.459+0000 7f3201a00640 1 -- 172.21.15.131:0/4197529650 shutdown_connections 2024-05-26T22:55:57.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:57.459+0000 7f3201a00640 1 -- 172.21.15.131:0/4197529650 wait complete. 2024-05-26T22:55:57.481 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-26T22:55:57.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:57.675+0000 7f7d07200640 1 Processor -- start 2024-05-26T22:55:57.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:57.675+0000 7f7d07200640 1 -- start start 2024-05-26T22:55:57.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:57.676+0000 7f7d07200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7d0806e8c0 0x7f7d0815d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:57.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:57.676+0000 7f7d07200640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d08160620 con 0x7f7d080cc7f0 2024-05-26T22:55:57.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:57.676+0000 7f7d07200640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d08160900 con 0x7f7d08156750 2024-05-26T22:55:57.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:57.676+0000 7f7d07200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f7d0815cbd0 con 0x7f7d0806e8c0 2024-05-26T22:55:57.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:57.677+0000 7f7d06800640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f7d080cc7f0 0x7f7d080cdc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:60420/0 (socket says 172.21.15.131:60420) 2024-05-26T22:55:57.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:57.677+0000 7f7d06800640 1 -- 172.21.15.131:0/4019357146 learned_addr learned my addr 172.21.15.131:0/4019357146 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:55:57.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:57.677+0000 7f7d05400640 1 --2- 172.21.15.131:0/4019357146 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7d0806e8c0 0x7f7d0815d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:57.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:57.677+0000 7f7d04a00640 1 -- 172.21.15.131:0/4019357146 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3098885558 0 0) 0x7f7d08160620 con 0x7f7d080cc7f0 2024-05-26T22:55:57.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:57.677+0000 7f7d04a00640 1 -- 172.21.15.131:0/4019357146 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7cec003520 con 0x7f7d080cc7f0 2024-05-26T22:55:57.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:57.677+0000 7f7d04a00640 1 -- 172.21.15.131:0/4019357146 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4070748724 0 0) 0x7f7cec003520 con 0x7f7d080cc7f0 2024-05-26T22:55:57.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:57.677+0000 7f7d04a00640 1 -- 172.21.15.131:0/4019357146 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7d0806e8c0 msgr2=0x7f7d0815d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:57.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:57.677+0000 7f7d04a00640 1 --2- 172.21.15.131:0/4019357146 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7d0806e8c0 0x7f7d0815d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:57.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:57.677+0000 7f7d04a00640 1 -- 172.21.15.131:0/4019357146 >> v1:172.21.15.187:6789/0 conn(0x7f7d08156750 legacy=0x7f7d0815fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:57.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:57.678+0000 7f7d04a00640 1 -- 172.21.15.131:0/4019357146 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d08160be0 con 0x7f7d080cc7f0 2024-05-26T22:55:57.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:57.678+0000 7f7d04a00640 1 -- 172.21.15.131:0/4019357146 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7f7d000019f0 con 0x7f7d080cc7f0 2024-05-26T22:55:57.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:57.678+0000 7f7d04a00640 1 -- 172.21.15.131:0/4019357146 >> v1:172.21.15.131:6789/0 conn(0x7f7d080cc7f0 legacy=0x7f7d080cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:57.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:57.678+0000 7f7d04a00640 1 --2- 172.21.15.131:0/4019357146 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7cec0039e0 0x7f7cec003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:57.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:57.678+0000 7f7d04a00640 1 --2- 172.21.15.131:0/4019357146 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7cec004800 0x7f7cec004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:57.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:57.678+0000 7f7d04a00640 1 --2- 172.21.15.131:0/4019357146 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7d0806e8c0 0x7f7cec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:57.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:57.678+0000 7f7d04a00640 1 -- 172.21.15.131:0/4019357146 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f7d0815cbd0 con 0x7f7cec0039e0 2024-05-26T22:55:57.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:57.678+0000 7f7d04a00640 1 -- 172.21.15.131:0/4019357146 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7cec00ba70 con 0x7f7cec004800 2024-05-26T22:55:57.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:57.678+0000 7f7d04a00640 1 -- 172.21.15.131:0/4019357146 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f7cec00bc70 con 0x7f7d0806e8c0 2024-05-26T22:55:57.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:57.678+0000 7f7d05e00640 1 --2- 172.21.15.131:0/4019357146 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7cec0039e0 0x7f7cec003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:57.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:57.678+0000 7f7d05e00640 1 -- 172.21.15.131:0/4019357146 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7d0806e8c0 msgr2=0x7f7cec0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:57.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:57.678+0000 7f7d05e00640 1 --2- 172.21.15.131:0/4019357146 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7d0806e8c0 0x7f7cec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:57.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:57.678+0000 7f7d05e00640 1 -- 172.21.15.131:0/4019357146 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7cec004800 msgr2=0x7f7cec004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:57.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:57.678+0000 7f7d05e00640 1 --2- 172.21.15.131:0/4019357146 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7cec004800 0x7f7cec004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:57.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:57.678+0000 7f7d05e00640 1 -- 172.21.15.131:0/4019357146 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cec003520 con 0x7f7cec0039e0 2024-05-26T22:55:57.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:57.679+0000 7f7d05e00640 1 --2- 172.21.15.131:0/4019357146 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7cec0039e0 0x7f7cec003e00 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f7cf8001720 tx=0x7f7cec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bf4c40bb513e255 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:57.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:57.680+0000 7f7d04a00640 1 -- 172.21.15.131:0/4019357146 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cf8001370 con 0x7f7cec0039e0 2024-05-26T22:55:57.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:57.680+0000 7f7d04a00640 1 -- 172.21.15.131:0/4019357146 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cf80052d0 con 0x7f7cec0039e0 2024-05-26T22:55:57.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:57.680+0000 7f7d04a00640 1 -- 172.21.15.131:0/4019357146 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cf802ab90 con 0x7f7cec0039e0 2024-05-26T22:55:57.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:57.680+0000 7f7d07200640 1 -- 172.21.15.131:0/4019357146 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7cec0039e0 msgr2=0x7f7cec003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:57.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:57.680+0000 7f7d07200640 1 --2- 172.21.15.131:0/4019357146 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7cec0039e0 0x7f7cec003e00 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f7cf8001720 tx=0x7f7cec005830 comp rx=0 tx=0).stop 2024-05-26T22:55:57.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:57.680+0000 7f7d06800640 1 -- 172.21.15.131:0/4019357146 reap_dead start 2024-05-26T22:55:57.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:57.680+0000 7f7d07200640 1 -- 172.21.15.131:0/4019357146 shutdown_connections 2024-05-26T22:55:57.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:57.680+0000 7f7d07200640 1 -- 172.21.15.131:0/4019357146 >> 172.21.15.131:0/4019357146 conn(0x7f7d080cff40 msgr2=0x7f7d080d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:57.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:57.680+0000 7f7d07200640 1 -- 172.21.15.131:0/4019357146 shutdown_connections 2024-05-26T22:55:57.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:57.680+0000 7f7d07200640 1 -- 172.21.15.131:0/4019357146 wait complete. 2024-05-26T22:55:57.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:57.682+0000 7f7d07200640 1 Processor -- start 2024-05-26T22:55:57.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:57.682+0000 7f7d07200640 1 -- start start 2024-05-26T22:55:57.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:57.683+0000 7f7d07200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7cec0039e0 0x7f7d081556a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:57.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:57.683+0000 7f7d07200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7d080b1b20 0x7f7d0815e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:57.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:57.683+0000 7f7d07200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d0815eac0 0x7f7d080afa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:57.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:57.683+0000 7f7d07200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f7d080ce370 con 0x7f7d0815eac0 2024-05-26T22:55:57.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:57.683+0000 7f7d07200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7d08156b30 con 0x7f7cec0039e0 2024-05-26T22:55:57.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:57.683+0000 7f7d07200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f7d080ce4f0 con 0x7f7d080b1b20 2024-05-26T22:55:57.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:57.683+0000 7f7d06800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d0815eac0 0x7f7d080afa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:57.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:57.683+0000 7f7d06800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d0815eac0 0x7f7d080afa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:36704/0 (socket says 172.21.15.131:36704) 2024-05-26T22:55:57.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:57.683+0000 7f7d06800640 1 -- 172.21.15.131:0/66576187 learned_addr learned my addr 172.21.15.131:0/66576187 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:55:57.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:57.684+0000 7f7d06800640 1 -- 172.21.15.131:0/66576187 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7d080b1b20 msgr2=0x7f7d0815e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:57.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:57.684+0000 7f7d05e00640 1 --2- 172.21.15.131:0/66576187 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7cec0039e0 0x7f7d081556a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:57.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:57.684+0000 7f7d05400640 1 --2- 172.21.15.131:0/66576187 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7d080b1b20 0x7f7d0815e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:57.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:57.684+0000 7f7d06800640 1 --2- 172.21.15.131:0/66576187 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7d080b1b20 0x7f7d0815e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:57.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:57.684+0000 7f7d06800640 1 -- 172.21.15.131:0/66576187 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7cec0039e0 msgr2=0x7f7d081556a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:57.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:57.684+0000 7f7d06800640 1 --2- 172.21.15.131:0/66576187 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7cec0039e0 0x7f7d081556a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:57.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:57.684+0000 7f7d06800640 1 -- 172.21.15.131:0/66576187 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d08157100 con 0x7f7d0815eac0 2024-05-26T22:55:57.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:57.684+0000 7f7d06800640 1 --2- 172.21.15.131:0/66576187 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d0815eac0 0x7f7d080afa80 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f7d0806f330 tx=0x7f7d0000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:57.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:57.684+0000 7f7cfe000640 1 -- 172.21.15.131:0/66576187 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d0000a5b0 con 0x7f7d0815eac0 2024-05-26T22:55:57.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:57.685+0000 7f7cfe000640 1 -- 172.21.15.131:0/66576187 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d0000a750 con 0x7f7d0815eac0 2024-05-26T22:55:57.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:57.685+0000 7f7cfe000640 1 -- 172.21.15.131:0/66576187 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d0000ab40 con 0x7f7d0815eac0 2024-05-26T22:55:57.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:57.685+0000 7f7d07200640 1 -- 172.21.15.131:0/66576187 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d081573e0 con 0x7f7d0815eac0 2024-05-26T22:55:57.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:57.685+0000 7f7d07200640 1 -- 172.21.15.131:0/66576187 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d081579a0 con 0x7f7d0815eac0 2024-05-26T22:55:57.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:57.686+0000 7f7d07200640 1 -- 172.21.15.131:0/66576187 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d080b4130 con 0x7f7d0815eac0 2024-05-26T22:55:57.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:57.687+0000 7f7cfe000640 1 -- 172.21.15.131:0/66576187 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7d0000ace0 con 0x7f7d0815eac0 2024-05-26T22:55:57.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:57.688+0000 7f7cfe000640 1 --2- 172.21.15.131:0/66576187 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f7ce806be30 0x7f7ce806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:57.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:57.688+0000 7f7cfe000640 1 -- 172.21.15.131:0/66576187 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 4857+0+0 (secure 0 0 0) 0x7f7d0008a380 con 0x7f7d0815eac0 2024-05-26T22:55:57.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:57.688+0000 7f7d05e00640 1 --2- 172.21.15.131:0/66576187 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f7ce806be30 0x7f7ce806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:57.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:57.689+0000 7f7d05e00640 1 --2- 172.21.15.131:0/66576187 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f7ce806be30 0x7f7ce806e2f0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f7cf8032000 tx=0x7f7cf802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:57.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:57.693+0000 7f7cfe000640 1 -- 172.21.15.131:0/66576187 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f7d080b4130 con 0x7f7d0815eac0 2024-05-26T22:55:57.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:57.962+0000 7f7d07200640 1 -- 172.21.15.131:0/66576187 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f7d080694e0 con 0x7f7d0815eac0 2024-05-26T22:55:57.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:57.963+0000 7f7cfe000640 1 -- 172.21.15.131:0/66576187 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v18) ==== 80+0+5304 (secure 0 0 0) 0x7f7d0005de20 con 0x7f7d0815eac0 2024-05-26T22:55:57.963 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:55:57.963 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-26T22:55:57.963 INFO:teuthology.orchestra.run.smithi131.stdout: "auth_dump": [ 2024-05-26T22:55:57.963 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:55:57.963 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mds.a", 2024-05-26T22:55:57.963 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD1u1NmTQf1DhAAdvEt7wP0IgKhNHOqjVAllA==", 2024-05-26T22:55:57.964 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:55:57.964 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-05-26T22:55:57.964 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2024-05-26T22:55:57.964 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2024-05-26T22:55:57.964 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-05-26T22:55:57.964 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:55:57.964 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:55:57.964 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:55:57.964 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mds.b", 2024-05-26T22:55:57.964 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD1u1NmHNZ8KxAAPgQ/ojbl+EDfPhqrkg7LPQ==", 2024-05-26T22:55:57.964 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:55:57.964 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-05-26T22:55:57.965 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2024-05-26T22:55:57.965 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2024-05-26T22:55:57.965 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-05-26T22:55:57.965 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:55:57.965 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:55:57.965 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:55:57.965 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mds.c", 2024-05-26T22:55:57.965 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD1u1NmFjCHHRAAqix3Ujfsq2tnSU5+d/pSYA==", 2024-05-26T22:55:57.965 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:55:57.965 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-05-26T22:55:57.965 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2024-05-26T22:55:57.965 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2024-05-26T22:55:57.966 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-05-26T22:55:57.966 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:55:57.966 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:55:57.966 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:55:57.966 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mds.d", 2024-05-26T22:55:57.966 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD1u1NmM4LPORAAhtQtr+Vfpf7vdVwjSW6xtA==", 2024-05-26T22:55:57.966 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:55:57.966 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-05-26T22:55:57.966 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2024-05-26T22:55:57.966 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2024-05-26T22:55:57.966 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-05-26T22:55:57.966 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:55:57.967 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:55:57.967 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:55:57.967 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.0", 2024-05-26T22:55:57.967 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD5u1Nm+w1LNhAAsJLQkiKSh3KkMymlrZkcrw==", 2024-05-26T22:55:57.967 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:55:57.967 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-05-26T22:55:57.967 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-05-26T22:55:57.967 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-05-26T22:55:57.967 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:55:57.967 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:55:57.967 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:55:57.967 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.1", 2024-05-26T22:55:57.967 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD9u1Nm4c8RFBAAcn9HQ+izML9rujJ/2/Q/BQ==", 2024-05-26T22:55:57.967 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:55:57.968 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-05-26T22:55:57.968 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-05-26T22:55:57.968 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-05-26T22:55:57.968 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:55:57.968 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:55:57.968 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:55:57.968 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.2", 2024-05-26T22:55:57.968 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAAvFNmR5wmLRAAYEBAXPBZFq67iARZnwF3iw==", 2024-05-26T22:55:57.968 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:55:57.968 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-05-26T22:55:57.968 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-05-26T22:55:57.968 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-05-26T22:55:57.968 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:55:57.968 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:55:57.969 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:55:57.969 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.3", 2024-05-26T22:55:57.969 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAEvFNmJ4G5CxAAnTbP8DinhmuvGNo1VBVzjQ==", 2024-05-26T22:55:57.969 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:55:57.969 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-05-26T22:55:57.969 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-05-26T22:55:57.969 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-05-26T22:55:57.969 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:55:57.969 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:55:57.969 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:55:57.969 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.4", 2024-05-26T22:55:57.969 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAKvFNmcw5rGhAAvnivP6kt3vcN9UPm/svKqw==", 2024-05-26T22:55:57.969 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:55:57.970 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-05-26T22:55:57.970 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-05-26T22:55:57.970 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-05-26T22:55:57.970 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:55:57.970 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:55:57.970 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:55:57.970 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.5", 2024-05-26T22:55:57.970 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQANvFNmTqlzNhAAGfrSvNNHYKGOBzRXo0TEdw==", 2024-05-26T22:55:57.970 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:55:57.970 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-05-26T22:55:57.970 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-05-26T22:55:57.970 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-05-26T22:55:57.970 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:55:57.971 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:55:57.971 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:55:57.971 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.6", 2024-05-26T22:55:57.971 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQARvFNmpzd6FRAAgi2RYlQPvfHNe0iP4bJrwg==", 2024-05-26T22:55:57.971 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:55:57.971 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-05-26T22:55:57.971 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-05-26T22:55:57.971 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-05-26T22:55:57.971 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:55:57.971 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:55:57.971 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:55:57.971 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.7", 2024-05-26T22:55:57.971 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAUvFNmAkC8LhAAsmkKF4eloxdpCi/mrDtzqA==", 2024-05-26T22:55:57.972 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:55:57.972 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-05-26T22:55:57.972 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-05-26T22:55:57.972 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-05-26T22:55:57.972 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:55:57.972 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:55:57.972 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:55:57.972 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.0", 2024-05-26T22:55:57.972 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD2u1NmSGVWChAAH/wq3iaUs6ZELRhLKhmURQ==", 2024-05-26T22:55:57.972 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:55:57.972 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-05-26T22:55:57.972 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow r", 2024-05-26T22:55:57.973 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-05-26T22:55:57.973 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:55:57.973 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:55:57.973 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:55:57.973 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.1", 2024-05-26T22:55:57.973 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD2u1NmnA9CHRAA2XMmZQxysKweHNykiQUEeA==", 2024-05-26T22:55:57.973 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:55:57.973 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-05-26T22:55:57.973 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow r", 2024-05-26T22:55:57.973 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow rw", 2024-05-26T22:55:57.973 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow rwx" 2024-05-26T22:55:57.973 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:55:57.974 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:55:57.974 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:55:57.974 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.2", 2024-05-26T22:55:57.974 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD2u1NmGEI9ExAAvJn1LRwmALkD7zh/AP07vQ==", 2024-05-26T22:55:57.974 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:55:57.974 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-05-26T22:55:57.974 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow r", 2024-05-26T22:55:57.974 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow rw", 2024-05-26T22:55:57.974 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow rwx" 2024-05-26T22:55:57.974 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:55:57.974 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:55:57.974 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:55:57.975 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.3", 2024-05-26T22:55:57.975 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD2u1NmIHOtJhAAJCUm0kGqWu4rWVLsDkW6Eg==", 2024-05-26T22:55:57.975 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:55:57.975 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-05-26T22:55:57.975 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow r", 2024-05-26T22:55:57.975 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow rw", 2024-05-26T22:55:57.975 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow rwx" 2024-05-26T22:55:57.975 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:55:57.975 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:55:57.975 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:55:57.975 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.admin", 2024-05-26T22:55:57.976 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD0u1NmLRGoCBAAhDeVJV14MTAIdUte067jig==", 2024-05-26T22:55:57.976 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:55:57.976 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow *", 2024-05-26T22:55:57.976 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2024-05-26T22:55:57.976 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2024-05-26T22:55:57.976 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-05-26T22:55:57.976 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:55:57.976 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:55:57.976 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:55:57.977 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mgr.x", 2024-05-26T22:55:57.977 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD1u1NmaWhyBBAAlU7Lq6HMjkCPu0kztJIAXQ==", 2024-05-26T22:55:57.977 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:55:57.977 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow *", 2024-05-26T22:55:57.977 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile mgr", 2024-05-26T22:55:57.977 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-05-26T22:55:57.977 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:55:57.977 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-05-26T22:55:57.977 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:55:57.977 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mgr.y", 2024-05-26T22:55:57.977 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD0u1NmX1RhNhAAo5mEOGzUyrdwFlB/XOXKnw==", 2024-05-26T22:55:57.977 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-05-26T22:55:57.977 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow *", 2024-05-26T22:55:57.978 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile mgr", 2024-05-26T22:55:57.978 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-05-26T22:55:57.978 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:55:57.978 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:55:57.978 INFO:teuthology.orchestra.run.smithi131.stdout: ] 2024-05-26T22:55:57.978 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-26T22:55:57.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:57.966+0000 7f7d07200640 1 -- 172.21.15.131:0/66576187 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f7ce806be30 msgr2=0x7f7ce806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:57.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:57.966+0000 7f7d07200640 1 --2- 172.21.15.131:0/66576187 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f7ce806be30 0x7f7ce806e2f0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f7cf8032000 tx=0x7f7cf802e000 comp rx=0 tx=0).stop 2024-05-26T22:55:57.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:57.966+0000 7f7d07200640 1 -- 172.21.15.131:0/66576187 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d0815eac0 msgr2=0x7f7d080afa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:57.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:57.966+0000 7f7d07200640 1 --2- 172.21.15.131:0/66576187 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d0815eac0 0x7f7d080afa80 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f7d0806f330 tx=0x7f7d0000ccb0 comp rx=0 tx=0).stop 2024-05-26T22:55:57.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:57.966+0000 7f7d07200640 1 -- 172.21.15.131:0/66576187 shutdown_connections 2024-05-26T22:55:57.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:57.966+0000 7f7d07200640 1 --2- 172.21.15.131:0/66576187 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7d080b1b20 0x7f7d0815e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:57.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:57.966+0000 7f7d07200640 1 --2- 172.21.15.131:0/66576187 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f7ce806be30 0x7f7ce806e2f0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:57.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:57.966+0000 7f7d07200640 1 --2- 172.21.15.131:0/66576187 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d0815eac0 0x7f7d080afa80 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:57.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:57.966+0000 7f7d07200640 1 --2- 172.21.15.131:0/66576187 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7cec0039e0 0x7f7d081556a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:57.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:57.966+0000 7f7d07200640 1 -- 172.21.15.131:0/66576187 >> 172.21.15.131:0/66576187 conn(0x7f7d080cff40 msgr2=0x7f7d080cf370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:57.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:57.967+0000 7f7d07200640 1 -- 172.21.15.131:0/66576187 shutdown_connections 2024-05-26T22:55:57.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:57.967+0000 7f7d07200640 1 -- 172.21.15.131:0/66576187 wait complete. 2024-05-26T22:55:57.986 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-26T22:55:58.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.184+0000 7f1fb5000640 1 Processor -- start 2024-05-26T22:55:58.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.184+0000 7f1fb5000640 1 -- start start 2024-05-26T22:55:58.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.186+0000 7f1fb5000640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f1fb0144800 0x7f1fb015fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:58.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.186+0000 7f1fb5000640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1fb015f530 con 0x7f1fb00ad870 2024-05-26T22:55:58.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.186+0000 7f1fb5000640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1fb015f7b0 con 0x7f1fb015f100 2024-05-26T22:55:58.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.186+0000 7f1fb5000640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f1fb015ffd0 con 0x7f1fb0144800 2024-05-26T22:55:58.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.186+0000 7f1fa6a00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f1fb00ad870 0x7f1fb015e9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:60432/0 (socket says 172.21.15.131:60432) 2024-05-26T22:55:58.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.186+0000 7f1fa6a00640 1 -- 172.21.15.131:0/1000469961 learned_addr learned my addr 172.21.15.131:0/1000469961 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:55:58.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.186+0000 7f1faf400640 1 --2- 172.21.15.131:0/1000469961 >> v2:172.21.15.131:3300/0 conn(0x7f1fb0144800 0x7f1fb015fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:58.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.187+0000 7f1faf400640 1 -- 172.21.15.131:0/1000469961 >> v1:172.21.15.187:6789/0 conn(0x7f1fb015f100 legacy=0x7f1fb00ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:58.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.187+0000 7f1faf400640 1 -- 172.21.15.131:0/1000469961 >> v1:172.21.15.131:6789/0 conn(0x7f1fb00ad870 legacy=0x7f1fb015e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:58.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.187+0000 7f1faf400640 1 -- 172.21.15.131:0/1000469961 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fb00cde40 con 0x7f1fb0144800 2024-05-26T22:55:58.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.188+0000 7f1faf400640 1 --2- 172.21.15.131:0/1000469961 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1fb0144800 0x7f1fb015fa90 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f1fa00097a0 tx=0x7f1fa002a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=436762c0e36c58e1 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:58.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.188+0000 7f1faea00640 1 -- 172.21.15.131:0/1000469961 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa0037070 con 0x7f1fb0144800 2024-05-26T22:55:58.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.188+0000 7f1faea00640 1 -- 172.21.15.131:0/1000469961 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fa0032640 con 0x7f1fb0144800 2024-05-26T22:55:58.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.188+0000 7f1faea00640 1 -- 172.21.15.131:0/1000469961 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa0032a10 con 0x7f1fb0144800 2024-05-26T22:55:58.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.189+0000 7f1fb5000640 1 -- 172.21.15.131:0/1000469961 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1fb0144800 msgr2=0x7f1fb015fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:58.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.189+0000 7f1fb5000640 1 --2- 172.21.15.131:0/1000469961 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1fb0144800 0x7f1fb015fa90 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f1fa00097a0 tx=0x7f1fa002a2c0 comp rx=0 tx=0).stop 2024-05-26T22:55:58.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.189+0000 7f1fb5000640 1 -- 172.21.15.131:0/1000469961 shutdown_connections 2024-05-26T22:55:58.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.189+0000 7f1fb5000640 1 --2- 172.21.15.131:0/1000469961 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1fb0144800 0x7f1fb015fa90 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:58.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.189+0000 7f1fb5000640 1 -- 172.21.15.131:0/1000469961 >> 172.21.15.131:0/1000469961 conn(0x7f1fb00cff40 msgr2=0x7f1fb00d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:58.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.189+0000 7f1fb5000640 1 -- 172.21.15.131:0/1000469961 shutdown_connections 2024-05-26T22:55:58.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.189+0000 7f1fb5000640 1 -- 172.21.15.131:0/1000469961 wait complete. 2024-05-26T22:55:58.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.190+0000 7f1fb5000640 1 Processor -- start 2024-05-26T22:55:58.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.191+0000 7f1fb5000640 1 -- start start 2024-05-26T22:55:58.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.191+0000 7f1fb5000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1fb00ad870 0x7f1fb01f4490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:58.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.192+0000 7f1fb5000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1fb0144800 0x7f1fb01f8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:58.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.192+0000 7f1fb5000640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1fb015f100 0x7f1fb00726f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:58.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.192+0000 7f1fb5000640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb0144be0 con 0x7f1fb0144800 2024-05-26T22:55:58.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.192+0000 7f1fb5000640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb00ce400 con 0x7f1fb015f100 2024-05-26T22:55:58.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.192+0000 7f1fb5000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1fb00add00 con 0x7f1fb00ad870 2024-05-26T22:55:58.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.192+0000 7f1fa6a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1fb0144800 0x7f1fb01f8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:58.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.192+0000 7f1fa6a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1fb0144800 0x7f1fb01f8ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:36724/0 (socket says 172.21.15.131:36724) 2024-05-26T22:55:58.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.192+0000 7f1fa6a00640 1 -- 172.21.15.131:0/1490837690 learned_addr learned my addr 172.21.15.131:0/1490837690 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:55:58.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.192+0000 7f1fafe00640 1 --2- 172.21.15.131:0/1490837690 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1fb015f100 0x7f1fb00726f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:58.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.192+0000 7f1fa6a00640 1 -- 172.21.15.131:0/1490837690 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1fb00ad870 msgr2=0x7f1fb01f4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:58.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.192+0000 7f1faf400640 1 --2- 172.21.15.131:0/1490837690 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1fb00ad870 0x7f1fb01f4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:58.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.192+0000 7f1fa6a00640 1 --2- 172.21.15.131:0/1490837690 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1fb00ad870 0x7f1fb01f4490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:58.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.192+0000 7f1fa6a00640 1 -- 172.21.15.131:0/1490837690 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1fb015f100 msgr2=0x7f1fb00726f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:58.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.192+0000 7f1fa6a00640 1 --2- 172.21.15.131:0/1490837690 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1fb015f100 0x7f1fb00726f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:58.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.192+0000 7f1fa6a00640 1 -- 172.21.15.131:0/1490837690 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fb01f8ff0 con 0x7f1fb0144800 2024-05-26T22:55:58.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.192+0000 7f1fafe00640 1 --2- 172.21.15.131:0/1490837690 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1fb015f100 0x7f1fb00726f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:55:58.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.193+0000 7f1fa6a00640 1 --2- 172.21.15.131:0/1490837690 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1fb0144800 0x7f1fb01f8ab0 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f1f9800c4a0 tx=0x7f1f9800c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:58.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.193+0000 7f1fa7e00640 1 -- 172.21.15.131:0/1490837690 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f98017070 con 0x7f1fb0144800 2024-05-26T22:55:58.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.193+0000 7f1fa7e00640 1 -- 172.21.15.131:0/1490837690 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f98005ce0 con 0x7f1fb0144800 2024-05-26T22:55:58.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.193+0000 7f1fb5000640 1 -- 172.21.15.131:0/1490837690 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fb00afee0 con 0x7f1fb0144800 2024-05-26T22:55:58.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.193+0000 7f1fa7e00640 1 -- 172.21.15.131:0/1490837690 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f98007270 con 0x7f1fb0144800 2024-05-26T22:55:58.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.193+0000 7f1fb5000640 1 -- 172.21.15.131:0/1490837690 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fb00b04a0 con 0x7f1fb0144800 2024-05-26T22:55:58.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.194+0000 7f1fb5000640 1 -- 172.21.15.131:0/1490837690 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fb00694e0 con 0x7f1fb0144800 2024-05-26T22:55:58.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.195+0000 7f1fa7e00640 1 -- 172.21.15.131:0/1490837690 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1f9801a070 con 0x7f1fb0144800 2024-05-26T22:55:58.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.196+0000 7f1fa7e00640 1 --2- 172.21.15.131:0/1490837690 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f1f8c06be30 0x7f1f8c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:58.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.196+0000 7f1fa7e00640 1 -- 172.21.15.131:0/1490837690 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 4857+0+0 (secure 0 0 0) 0x7f1f98089d40 con 0x7f1fb0144800 2024-05-26T22:55:58.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.196+0000 7f1faf400640 1 --2- 172.21.15.131:0/1490837690 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f1f8c06be30 0x7f1f8c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:58.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.197+0000 7f1faf400640 1 --2- 172.21.15.131:0/1490837690 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f1f8c06be30 0x7f1f8c06e2f0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f1fa0034000 tx=0x7f1fa002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:58.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.201+0000 7f1fa7e00640 1 -- 172.21.15.131:0/1490837690 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f1fb01f8ff0 con 0x7f1fb0144800 2024-05-26T22:55:58.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.394+0000 7f1fb5000640 1 -- 172.21.15.131:0/1490837690 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1fb006eb90 con 0x7f1fb0144800 2024-05-26T22:55:58.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.396+0000 7f1fa7e00640 1 -- 172.21.15.131:0/1490837690 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) ==== 74+0+12917 (secure 0 0 0) 0x7f1fb015f530 con 0x7f1fb0144800 2024-05-26T22:55:58.396 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:55:58.396 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":103,"fsid":"46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f","created":"2024-05-26T22:47:57.288905+0000","modified":"2024-05-26T22:55:57.452068+0000","last_up_change":"2024-05-26T22:48:09.602207+0000","last_in_change":"2024-05-26T22:48:00.687341+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-26T22:48:10.463339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"814eaf62-a18d-494b-8e27-fd8fb88adea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6801","nonce":2350940051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6803","nonce":2350940051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6807","nonce":2350940051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6805","nonce":2350940051}]},"public_addr":"172.21.15.131:6801/2350940051","cluster_addr":"172.21.15.131:6803/2350940051","heartbeat_back_addr":"172.21.15.131:6807/2350940051","heartbeat_front_addr":"172.21.15.131:6805/2350940051","state":["exists","up"]},{"osd":1,"uuid":"42e66287-f6c6-4e26-af3c-6dfaa3536d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6825","nonce":1525528247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6827","nonce":1525528247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6831","nonce":1525528247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6829","nonce":1525528247}]},"public_addr":"172.21.15.131:6825/1525528247","cluster_addr":"172.21.15.131:6827/1525528247","heartbeat_back_addr":"172.21.15.131:6831/1525528247","heartbeat_front_addr":"172.21.15.131:6829/1525528247","state":["exists","up"]},{"osd":2,"uuid":"0da34673-04a2-4fd0-b57a-0ff2b679b043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6817","nonce":3627842210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6819","nonce":3627842210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6823","nonce":3627842210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6821","nonce":3627842210}]},"public_addr":"172.21.15.131:6817/3627842210","cluster_addr":"172.21.15.131:6819/3627842210","heartbeat_back_addr":"172.21.15.131:6823/3627842210","heartbeat_front_addr":"172.21.15.131:6821/3627842210","state":["exists","up"]},{"osd":3,"uuid":"cf117a27-d6b2-43d3-ae52-b47988bf8eed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6809","nonce":350790591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6811","nonce":350790591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6815","nonce":350790591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6813","nonce":350790591}]},"public_addr":"172.21.15.131:6809/350790591","cluster_addr":"172.21.15.131:6811/350790591","heartbeat_back_addr":"172.21.15.131:6815/350790591","heartbeat_front_addr":"172.21.15.131:6813/350790591","state":["exists","up"]},{"osd":4,"uuid":"a092ccfc-be89-4a84-88d5-0f781fa715eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6816","nonce":244560995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6819","nonce":244560995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6823","nonce":244560995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6821","nonce":244560995}]},"public_addr":"172.21.15.187:6816/244560995","cluster_addr":"172.21.15.187:6819/244560995","heartbeat_back_addr":"172.21.15.187:6823/244560995","heartbeat_front_addr":"172.21.15.187:6821/244560995","state":["exists","up"]},{"osd":5,"uuid":"7513ba69-dd00-4a35-b190-9e8cc3b2bdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6801","nonce":324619398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6803","nonce":324619398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6812","nonce":324619398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6807","nonce":324619398}]},"public_addr":"172.21.15.187:6801/324619398","cluster_addr":"172.21.15.187:6803/324619398","heartbeat_back_addr":"172.21.15.187:6812/324619398","heartbeat_front_addr":"172.21.15.187:6807/324619398","state":["exists","up"]},{"osd":6,"uuid":"50081810-c093-4d2f-a4d7-b7c695c5400d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6806","nonce":1391347786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6809","nonce":1391347786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6818","nonce":1391347786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6813","nonce":1391347786}]},"public_addr":"172.21.15.187:6806/1391347786","cluster_addr":"172.21.15.187:6809/1391347786","heartbeat_back_addr":"172.21.15.187:6818/1391347786","heartbeat_front_addr":"172.21.15.187:6813/1391347786","state":["exists","up"]},{"osd":7,"uuid":"775dae05-54f5-4ad4-bbce-92863c6ed2ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6825","nonce":1220065622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6827","nonce":1220065622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6831","nonce":1220065622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6829","nonce":1220065622}]},"public_addr":"172.21.15.187:6825/1220065622","cluster_addr":"172.21.15.187:6827/1220065622","heartbeat_back_addr":"172.21.15.187:6831/1220065622","heartbeat_front_addr":"172.21.15.187:6829/1220065622","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:05.983520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.299516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.067323+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.307748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.018742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.258775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.010912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.295355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-26T22:55:58.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.398+0000 7f1fb5000640 1 -- 172.21.15.131:0/1490837690 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f1f8c06be30 msgr2=0x7f1f8c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:58.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.398+0000 7f1fb5000640 1 --2- 172.21.15.131:0/1490837690 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f1f8c06be30 0x7f1f8c06e2f0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f1fa0034000 tx=0x7f1fa002e000 comp rx=0 tx=0).stop 2024-05-26T22:55:58.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.398+0000 7f1fb5000640 1 -- 172.21.15.131:0/1490837690 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1fb0144800 msgr2=0x7f1fb01f8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:58.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.398+0000 7f1fb5000640 1 --2- 172.21.15.131:0/1490837690 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1fb0144800 0x7f1fb01f8ab0 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f1f9800c4a0 tx=0x7f1f9800c960 comp rx=0 tx=0).stop 2024-05-26T22:55:58.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.398+0000 7f1fb5000640 1 -- 172.21.15.131:0/1490837690 shutdown_connections 2024-05-26T22:55:58.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.398+0000 7f1fb5000640 1 --2- 172.21.15.131:0/1490837690 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f1f8c06be30 0x7f1f8c06e2f0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:58.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.398+0000 7f1fb5000640 1 --2- 172.21.15.131:0/1490837690 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1fb015f100 0x7f1fb00726f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:58.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.398+0000 7f1fb5000640 1 --2- 172.21.15.131:0/1490837690 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1fb0144800 0x7f1fb01f8ab0 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:58.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.398+0000 7f1fb5000640 1 --2- 172.21.15.131:0/1490837690 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1fb00ad870 0x7f1fb01f4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:58.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.398+0000 7f1fb5000640 1 -- 172.21.15.131:0/1490837690 >> 172.21.15.131:0/1490837690 conn(0x7f1fb00cff40 msgr2=0x7f1fb0143130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:58.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.399+0000 7f1fb5000640 1 -- 172.21.15.131:0/1490837690 shutdown_connections 2024-05-26T22:55:58.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.399+0000 7f1fb5000640 1 -- 172.21.15.131:0/1490837690 wait complete. 2024-05-26T22:55:58.415 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-26T22:48:10.463339+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-26T22:55:58.415 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-26T22:55:58.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.617+0000 7fba5c800640 1 Processor -- start 2024-05-26T22:55:58.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.617+0000 7fba5c800640 1 -- start start 2024-05-26T22:55:58.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.619+0000 7fba5c800640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fba5415e4e0 0x7fba540b9c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:58.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.619+0000 7fba5c800640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba540d28f0 con 0x7fba540cf720 2024-05-26T22:55:58.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.619+0000 7fba5c800640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba540d2bd0 con 0x7fba540d2330 2024-05-26T22:55:58.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.619+0000 7fba5c800640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fba540d2eb0 con 0x7fba5415e4e0 2024-05-26T22:55:58.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.619+0000 7fba5a800640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fba540cf720 0x7fba540b7400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:60436/0 (socket says 172.21.15.131:60436) 2024-05-26T22:55:58.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.619+0000 7fba5a800640 1 -- 172.21.15.131:0/2923033009 learned_addr learned my addr 172.21.15.131:0/2923033009 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:55:58.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.620+0000 7fba58a00640 1 -- 172.21.15.131:0/2923033009 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2596540938 0 0) 0x7fba540d28f0 con 0x7fba540cf720 2024-05-26T22:55:58.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.620+0000 7fba58a00640 1 -- 172.21.15.131:0/2923033009 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fba40003520 con 0x7fba540cf720 2024-05-26T22:55:58.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.620+0000 7fba59e00640 1 --2- 172.21.15.131:0/2923033009 >> v2:172.21.15.187:3300/0 conn(0x7fba5415e4e0 0x7fba540b9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:58.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.620+0000 7fba58a00640 1 -- 172.21.15.131:0/2923033009 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1449937227 0 0) 0x7fba40003520 con 0x7fba540cf720 2024-05-26T22:55:58.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.620+0000 7fba58a00640 1 -- 172.21.15.131:0/2923033009 >> v2:172.21.15.187:3300/0 conn(0x7fba5415e4e0 msgr2=0x7fba540b9c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:58.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.620+0000 7fba58a00640 1 --2- 172.21.15.131:0/2923033009 >> v2:172.21.15.187:3300/0 conn(0x7fba5415e4e0 0x7fba540b9c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:58.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.620+0000 7fba58a00640 1 -- 172.21.15.131:0/2923033009 >> v1:172.21.15.187:6789/0 conn(0x7fba540d2330 legacy=0x7fba540cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:58.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.620+0000 7fba58a00640 1 -- 172.21.15.131:0/2923033009 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba540d3190 con 0x7fba540cf720 2024-05-26T22:55:58.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.620+0000 7fba58a00640 1 -- 172.21.15.131:0/2923033009 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7fba50001670 con 0x7fba540cf720 2024-05-26T22:55:58.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.620+0000 7fba58a00640 1 -- 172.21.15.131:0/2923033009 >> v1:172.21.15.131:6789/0 conn(0x7fba540cf720 legacy=0x7fba540b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:58.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.620+0000 7fba58a00640 1 --2- 172.21.15.131:0/2923033009 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fba40003a00 0x7fba40003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:58.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.620+0000 7fba58a00640 1 --2- 172.21.15.131:0/2923033009 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fba40004800 0x7fba40004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:58.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.620+0000 7fba58a00640 1 --2- 172.21.15.131:0/2923033009 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba400052e0 0x7fba4000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:58.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.620+0000 7fba58a00640 1 -- 172.21.15.131:0/2923033009 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fba540d2eb0 con 0x7fba400052e0 2024-05-26T22:55:58.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.620+0000 7fba58a00640 1 -- 172.21.15.131:0/2923033009 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fba4000c200 con 0x7fba40004800 2024-05-26T22:55:58.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.620+0000 7fba58a00640 1 -- 172.21.15.131:0/2923033009 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fba4000c400 con 0x7fba40003a00 2024-05-26T22:55:58.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.620+0000 7fba5a800640 1 --2- 172.21.15.131:0/2923033009 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba400052e0 0x7fba4000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:58.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.621+0000 7fba5a800640 1 -- 172.21.15.131:0/2923033009 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fba40003a00 msgr2=0x7fba40003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:58.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.621+0000 7fba5a800640 1 --2- 172.21.15.131:0/2923033009 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fba40003a00 0x7fba40003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:58.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.621+0000 7fba5a800640 1 -- 172.21.15.131:0/2923033009 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fba40004800 msgr2=0x7fba40004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:58.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.621+0000 7fba5a800640 1 --2- 172.21.15.131:0/2923033009 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fba40004800 0x7fba40004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:58.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.621+0000 7fba5a800640 1 -- 172.21.15.131:0/2923033009 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba40003520 con 0x7fba400052e0 2024-05-26T22:55:58.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.622+0000 7fba5a800640 1 -- 172.21.15.131:0/2923033009 reap_dead start 2024-05-26T22:55:58.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.622+0000 7fba5a800640 1 --2- 172.21.15.131:0/2923033009 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba400052e0 0x7fba4000b9c0 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7fba50007fd0 tx=0x7fba40005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a853f21b9d4bdd0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:58.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.622+0000 7fba58a00640 1 -- 172.21.15.131:0/2923033009 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba50007c20 con 0x7fba400052e0 2024-05-26T22:55:58.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.622+0000 7fba58a00640 1 -- 172.21.15.131:0/2923033009 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba50005af0 con 0x7fba400052e0 2024-05-26T22:55:58.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.622+0000 7fba58a00640 1 -- 172.21.15.131:0/2923033009 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba50007070 con 0x7fba400052e0 2024-05-26T22:55:58.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.623+0000 7fba5c800640 1 -- 172.21.15.131:0/2923033009 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba400052e0 msgr2=0x7fba4000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:58.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.623+0000 7fba5c800640 1 --2- 172.21.15.131:0/2923033009 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba400052e0 0x7fba4000b9c0 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7fba50007fd0 tx=0x7fba40005830 comp rx=0 tx=0).stop 2024-05-26T22:55:58.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.623+0000 7fba5c800640 1 -- 172.21.15.131:0/2923033009 shutdown_connections 2024-05-26T22:55:58.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.623+0000 7fba5c800640 1 --2- 172.21.15.131:0/2923033009 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba400052e0 0x7fba4000b9c0 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:58.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.623+0000 7fba5c800640 1 -- 172.21.15.131:0/2923033009 >> 172.21.15.131:0/2923033009 conn(0x7fba54072e90 msgr2=0x7fba540732a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:58.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.623+0000 7fba5c800640 1 -- 172.21.15.131:0/2923033009 shutdown_connections 2024-05-26T22:55:58.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.623+0000 7fba5c800640 1 -- 172.21.15.131:0/2923033009 wait complete. 2024-05-26T22:55:58.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.624+0000 7fba5c800640 1 Processor -- start 2024-05-26T22:55:58.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.625+0000 7fba5c800640 1 -- start start 2024-05-26T22:55:58.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.625+0000 7fba5c800640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fba400052e0 0x7fba541517b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:58.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.625+0000 7fba5c800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba541536e0 0x7fba5415a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:58.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.626+0000 7fba5c800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fba5415aaa0 0x7fba541f8350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:58.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.626+0000 7fba5c800640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fba540d3990 con 0x7fba541536e0 2024-05-26T22:55:58.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.626+0000 7fba5c800640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fba5415e8c0 con 0x7fba400052e0 2024-05-26T22:55:58.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.626+0000 7fba5c800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fba540cf010 con 0x7fba5415aaa0 2024-05-26T22:55:58.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.626+0000 7fba59400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba541536e0 0x7fba5415a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:58.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.626+0000 7fba59400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba541536e0 0x7fba5415a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:36738/0 (socket says 172.21.15.131:36738) 2024-05-26T22:55:58.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.626+0000 7fba59400640 1 -- 172.21.15.131:0/612776932 learned_addr learned my addr 172.21.15.131:0/612776932 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:55:58.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.626+0000 7fba59400640 1 -- 172.21.15.131:0/612776932 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fba5415aaa0 msgr2=0x7fba541f8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:58.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.626+0000 7fba59400640 1 --2- 172.21.15.131:0/612776932 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fba5415aaa0 0x7fba541f8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:58.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.626+0000 7fba59400640 1 -- 172.21.15.131:0/612776932 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fba400052e0 msgr2=0x7fba541517b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:58.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.626+0000 7fba59400640 1 --2- 172.21.15.131:0/612776932 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fba400052e0 0x7fba541517b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:58.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.626+0000 7fba59400640 1 -- 172.21.15.131:0/612776932 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba54160960 con 0x7fba541536e0 2024-05-26T22:55:58.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.626+0000 7fba59400640 1 --2- 172.21.15.131:0/612776932 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba541536e0 0x7fba5415a560 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7fba4400c530 tx=0x7fba4400c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:58.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.627+0000 7fba4a000640 1 -- 172.21.15.131:0/612776932 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba440052d0 con 0x7fba541536e0 2024-05-26T22:55:58.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.627+0000 7fba4a000640 1 -- 172.21.15.131:0/612776932 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba44005470 con 0x7fba541536e0 2024-05-26T22:55:58.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.627+0000 7fba4a000640 1 -- 172.21.15.131:0/612776932 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba44012850 con 0x7fba541536e0 2024-05-26T22:55:58.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.627+0000 7fba5c800640 1 -- 172.21.15.131:0/612776932 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba54160c40 con 0x7fba541536e0 2024-05-26T22:55:58.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.627+0000 7fba5c800640 1 -- 172.21.15.131:0/612776932 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba54151cf0 con 0x7fba541536e0 2024-05-26T22:55:58.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.628+0000 7fba5c800640 1 -- 172.21.15.131:0/612776932 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba540695c0 con 0x7fba541536e0 2024-05-26T22:55:58.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.629+0000 7fba4a000640 1 -- 172.21.15.131:0/612776932 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fba440129f0 con 0x7fba541536e0 2024-05-26T22:55:58.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.630+0000 7fba4a000640 1 --2- 172.21.15.131:0/612776932 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fba3806be30 0x7fba3806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:58.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.630+0000 7fba4a000640 1 -- 172.21.15.131:0/612776932 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 4857+0+0 (secure 0 0 0) 0x7fba4408a8a0 con 0x7fba541536e0 2024-05-26T22:55:58.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.630+0000 7fba59e00640 1 --2- 172.21.15.131:0/612776932 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fba3806be30 0x7fba3806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:58.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.630+0000 7fba59e00640 1 --2- 172.21.15.131:0/612776932 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fba3806be30 0x7fba3806e2f0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fba4c009e40 tx=0x7fba4c0094a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:58.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.634+0000 7fba4a000640 1 -- 172.21.15.131:0/612776932 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fba44059590 con 0x7fba541536e0 2024-05-26T22:55:58.736 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2024-05-26T22:55:58.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.831+0000 7fba5c800640 1 -- 172.21.15.131:0/612776932 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fba540b4e20 con 0x7fba541536e0 2024-05-26T22:55:58.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.832+0000 7fba4a000640 1 -- 172.21.15.131:0/612776932 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v103) ==== 93+0+10 (secure 0 0 0) 0x7fba44058e90 con 0x7fba541536e0 2024-05-26T22:55:58.832 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2024-05-26T22:55:58.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.835+0000 7fba5c800640 1 -- 172.21.15.131:0/612776932 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fba3806be30 msgr2=0x7fba3806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:58.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.835+0000 7fba5c800640 1 --2- 172.21.15.131:0/612776932 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fba3806be30 0x7fba3806e2f0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fba4c009e40 tx=0x7fba4c0094a0 comp rx=0 tx=0).stop 2024-05-26T22:55:58.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.835+0000 7fba5c800640 1 -- 172.21.15.131:0/612776932 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba541536e0 msgr2=0x7fba5415a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:58.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.835+0000 7fba5c800640 1 --2- 172.21.15.131:0/612776932 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba541536e0 0x7fba5415a560 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7fba4400c530 tx=0x7fba4400c9f0 comp rx=0 tx=0).stop 2024-05-26T22:55:58.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.835+0000 7fba5c800640 1 -- 172.21.15.131:0/612776932 shutdown_connections 2024-05-26T22:55:58.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.835+0000 7fba5c800640 1 --2- 172.21.15.131:0/612776932 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fba5415aaa0 0x7fba541f8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:58.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.835+0000 7fba5c800640 1 --2- 172.21.15.131:0/612776932 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fba3806be30 0x7fba3806e2f0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:58.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.835+0000 7fba5c800640 1 --2- 172.21.15.131:0/612776932 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba541536e0 0x7fba5415a560 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:58.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.835+0000 7fba5c800640 1 --2- 172.21.15.131:0/612776932 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fba400052e0 0x7fba541517b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:58.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.835+0000 7fba5c800640 1 -- 172.21.15.131:0/612776932 >> 172.21.15.131:0/612776932 conn(0x7fba54072e90 msgr2=0x7fba540b8390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:58.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.835+0000 7fba5c800640 1 -- 172.21.15.131:0/612776932 shutdown_connections 2024-05-26T22:55:58.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:58.835+0000 7fba5c800640 1 -- 172.21.15.131:0/612776932 wait complete. 2024-05-26T22:55:58.854 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-26T22:55:59.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.054+0000 7f23e9800640 1 Processor -- start 2024-05-26T22:55:59.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.054+0000 7f23e9800640 1 -- start start 2024-05-26T22:55:59.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.055+0000 7f23e9800640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f23e4144340 0x7f23e40cedf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:59.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.055+0000 7f23e9800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23e406ebc0 0x7f23e40b3d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:59.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.055+0000 7f23e9800640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23e4072000 con 0x7f23e4071b70 2024-05-26T22:55:59.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.055+0000 7f23e9800640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f23e4072eb0 con 0x7f23e4144340 2024-05-26T22:55:59.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.055+0000 7f23e9800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f23e40722a0 con 0x7f23e406ebc0 2024-05-26T22:55:59.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.056+0000 7f23db400640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f23e4071b70 0x7f23e406e4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.131:53746/0 (socket says 172.21.15.131:53746) 2024-05-26T22:55:59.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.056+0000 7f23db400640 1 -- 172.21.15.131:0/394047726 learned_addr learned my addr 172.21.15.131:0/394047726 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:55:59.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.056+0000 7f23e8e00640 1 --2- 172.21.15.131:0/394047726 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23e406ebc0 0x7f23e40b3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:59.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.056+0000 7f23e3e00640 1 --2- 172.21.15.131:0/394047726 >> v2:172.21.15.187:3300/0 conn(0x7f23e4144340 0x7f23e40cedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:59.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.057+0000 7f23e3400640 1 -- 172.21.15.131:0/394047726 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1874090051 0 0) 0x7f23e4072000 con 0x7f23e4071b70 2024-05-26T22:55:59.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.057+0000 7f23e3400640 1 -- 172.21.15.131:0/394047726 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f23c8003520 con 0x7f23e4071b70 2024-05-26T22:55:59.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.057+0000 7f23e8e00640 1 -- 172.21.15.131:0/394047726 >> v2:172.21.15.187:3300/0 conn(0x7f23e4144340 msgr2=0x7f23e40cedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:59.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.057+0000 7f23e8e00640 1 --2- 172.21.15.131:0/394047726 >> v2:172.21.15.187:3300/0 conn(0x7f23e4144340 0x7f23e40cedf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:59.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.058+0000 7f23e8e00640 1 -- 172.21.15.131:0/394047726 >> v1:172.21.15.187:6789/0 conn(0x7f23e4071b70 legacy=0x7f23e406e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:59.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.058+0000 7f23e8e00640 1 -- 172.21.15.131:0/394047726 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23e4072580 con 0x7f23e406ebc0 2024-05-26T22:55:59.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.058+0000 7f23e3e00640 1 --2- 172.21.15.131:0/394047726 >> v2:172.21.15.187:3300/0 conn(0x7f23e4144340 0x7f23e40cedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-26T22:55:59.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.059+0000 7f23e8e00640 1 --2- 172.21.15.131:0/394047726 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23e406ebc0 0x7f23e40b3d20 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f23dc0097a0 tx=0x7f23dc02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6ce044dc3c3f140 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:59.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.059+0000 7f23e3400640 1 -- 172.21.15.131:0/394047726 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23dc037070 con 0x7f23e406ebc0 2024-05-26T22:55:59.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.059+0000 7f23e3400640 1 -- 172.21.15.131:0/394047726 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23dc032440 con 0x7f23e406ebc0 2024-05-26T22:55:59.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.059+0000 7f23e3400640 1 -- 172.21.15.131:0/394047726 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23dc0327f0 con 0x7f23e406ebc0 2024-05-26T22:55:59.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.060+0000 7f23e9800640 1 -- 172.21.15.131:0/394047726 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23e406ebc0 msgr2=0x7f23e40b3d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:59.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.060+0000 7f23e9800640 1 --2- 172.21.15.131:0/394047726 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23e406ebc0 0x7f23e40b3d20 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f23dc0097a0 tx=0x7f23dc02a180 comp rx=0 tx=0).stop 2024-05-26T22:55:59.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.060+0000 7f23e9800640 1 -- 172.21.15.131:0/394047726 shutdown_connections 2024-05-26T22:55:59.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.060+0000 7f23e9800640 1 --2- 172.21.15.131:0/394047726 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23e406ebc0 0x7f23e40b3d20 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:59.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.060+0000 7f23e9800640 1 --2- 172.21.15.131:0/394047726 >> v2:172.21.15.187:3300/0 conn(0x7f23e4144340 0x7f23e40cedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:59.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.060+0000 7f23e9800640 1 -- 172.21.15.131:0/394047726 >> 172.21.15.131:0/394047726 conn(0x7f23e40cff40 msgr2=0x7f23e40d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:59.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.060+0000 7f23e9800640 1 -- 172.21.15.131:0/394047726 shutdown_connections 2024-05-26T22:55:59.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.060+0000 7f23e9800640 1 -- 172.21.15.131:0/394047726 wait complete. 2024-05-26T22:55:59.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.062+0000 7f23e9800640 1 Processor -- start 2024-05-26T22:55:59.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.062+0000 7f23e9800640 1 -- start start 2024-05-26T22:55:59.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.063+0000 7f23e9800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f23e406ebc0 0x7f23e413a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:59.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.063+0000 7f23e9800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23e4071b70 0x7f23e4151730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:59.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.063+0000 7f23e9800640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f23e4144340 0x7f23e413cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:59.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.063+0000 7f23e9800640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f23e40b4260 con 0x7f23e406ebc0 2024-05-26T22:55:59.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.063+0000 7f23e9800640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f23e4072b40 con 0x7f23e4144340 2024-05-26T22:55:59.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.063+0000 7f23e9800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f23e40b43e0 con 0x7f23e4071b70 2024-05-26T22:55:59.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.063+0000 7f23e3e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f23e406ebc0 0x7f23e413a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:59.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.063+0000 7f23e3e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f23e406ebc0 0x7f23e413a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:36746/0 (socket says 172.21.15.131:36746) 2024-05-26T22:55:59.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.063+0000 7f23e3e00640 1 -- 172.21.15.131:0/3024509425 learned_addr learned my addr 172.21.15.131:0/3024509425 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:55:59.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.063+0000 7f23e3e00640 1 -- 172.21.15.131:0/3024509425 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23e4071b70 msgr2=0x7f23e4151730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:59.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.063+0000 7f23e8e00640 1 --2- 172.21.15.131:0/3024509425 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f23e4144340 0x7f23e413cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:59.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.063+0000 7f23db400640 1 --2- 172.21.15.131:0/3024509425 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23e4071b70 0x7f23e4151730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:59.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.063+0000 7f23e3e00640 1 --2- 172.21.15.131:0/3024509425 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23e4071b70 0x7f23e4151730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:59.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.064+0000 7f23e3e00640 1 -- 172.21.15.131:0/3024509425 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f23e4144340 msgr2=0x7f23e413cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:59.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.064+0000 7f23e3e00640 1 --2- 172.21.15.131:0/3024509425 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f23e4144340 0x7f23e413cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:59.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.064+0000 7f23e3e00640 1 -- 172.21.15.131:0/3024509425 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23e4140270 con 0x7f23e406ebc0 2024-05-26T22:55:59.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.064+0000 7f23e8e00640 1 --2- 172.21.15.131:0/3024509425 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f23e4144340 0x7f23e413cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:55:59.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.064+0000 7f23e3e00640 1 --2- 172.21.15.131:0/3024509425 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f23e406ebc0 0x7f23e413a330 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f23d400efc0 tx=0x7f23d400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:59.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.064+0000 7f23e0c00640 1 -- 172.21.15.131:0/3024509425 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23d4018020 con 0x7f23e406ebc0 2024-05-26T22:55:59.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.064+0000 7f23e0c00640 1 -- 172.21.15.131:0/3024509425 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23d4014680 con 0x7f23e406ebc0 2024-05-26T22:55:59.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.064+0000 7f23e0c00640 1 -- 172.21.15.131:0/3024509425 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23d4014a50 con 0x7f23e406ebc0 2024-05-26T22:55:59.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.064+0000 7f23e9800640 1 -- 172.21.15.131:0/3024509425 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23e413aaa0 con 0x7f23e406ebc0 2024-05-26T22:55:59.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.064+0000 7f23e9800640 1 -- 172.21.15.131:0/3024509425 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23e413b060 con 0x7f23e406ebc0 2024-05-26T22:55:59.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.065+0000 7f23e9800640 1 -- 172.21.15.131:0/3024509425 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23e40694e0 con 0x7f23e406ebc0 2024-05-26T22:55:59.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.066+0000 7f23e0c00640 1 -- 172.21.15.131:0/3024509425 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f23d4005d10 con 0x7f23e406ebc0 2024-05-26T22:55:59.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.067+0000 7f23e0c00640 1 --2- 172.21.15.131:0/3024509425 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f23c006be60 0x7f23c006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:59.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.067+0000 7f23e0c00640 1 -- 172.21.15.131:0/3024509425 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 4857+0+0 (secure 0 0 0) 0x7f23d408b8e0 con 0x7f23e406ebc0 2024-05-26T22:55:59.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.067+0000 7f23db400640 1 --2- 172.21.15.131:0/3024509425 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f23c006be60 0x7f23c006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:59.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.068+0000 7f23db400640 1 --2- 172.21.15.131:0/3024509425 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f23c006be60 0x7f23c006e320 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f23e41f56f0 tx=0x7f23cc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:59.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.072+0000 7f23e0c00640 1 -- 172.21.15.131:0/3024509425 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f23d4022030 con 0x7f23e406ebc0 2024-05-26T22:55:59.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.271+0000 7f23e9800640 1 -- 172.21.15.131:0/3024509425 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f23e40ce720 con 0x7f23e406ebc0 2024-05-26T22:55:59.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.272+0000 7f23e0c00640 1 -- 172.21.15.131:0/3024509425 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v103) ==== 84+0+65 (secure 0 0 0) 0x7f23d4010040 con 0x7f23e406ebc0 2024-05-26T22:55:59.272 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:55:59.272 INFO:teuthology.orchestra.run.smithi131.stdout:[ 2024-05-26T22:55:59.272 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-05-26T22:55:59.272 INFO:teuthology.orchestra.run.smithi131.stdout: "poolnum": 1, 2024-05-26T22:55:59.272 INFO:teuthology.orchestra.run.smithi131.stdout: "poolname": ".mgr" 2024-05-26T22:55:59.272 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-05-26T22:55:59.272 INFO:teuthology.orchestra.run.smithi131.stdout:] 2024-05-26T22:55:59.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.274+0000 7f23e9800640 1 -- 172.21.15.131:0/3024509425 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f23c006be60 msgr2=0x7f23c006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:59.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.274+0000 7f23e9800640 1 --2- 172.21.15.131:0/3024509425 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f23c006be60 0x7f23c006e320 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f23e41f56f0 tx=0x7f23cc009040 comp rx=0 tx=0).stop 2024-05-26T22:55:59.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.274+0000 7f23e9800640 1 -- 172.21.15.131:0/3024509425 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f23e406ebc0 msgr2=0x7f23e413a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:59.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.274+0000 7f23e9800640 1 --2- 172.21.15.131:0/3024509425 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f23e406ebc0 0x7f23e413a330 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f23d400efc0 tx=0x7f23d400c490 comp rx=0 tx=0).stop 2024-05-26T22:55:59.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.274+0000 7f23e9800640 1 -- 172.21.15.131:0/3024509425 shutdown_connections 2024-05-26T22:55:59.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.274+0000 7f23e9800640 1 --2- 172.21.15.131:0/3024509425 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23e4071b70 0x7f23e4151730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:59.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.274+0000 7f23e9800640 1 --2- 172.21.15.131:0/3024509425 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f23c006be60 0x7f23c006e320 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:59.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.274+0000 7f23e9800640 1 --2- 172.21.15.131:0/3024509425 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f23e4144340 0x7f23e413cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:59.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.274+0000 7f23e9800640 1 --2- 172.21.15.131:0/3024509425 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f23e406ebc0 0x7f23e413a330 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:59.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.275+0000 7f23e9800640 1 -- 172.21.15.131:0/3024509425 >> 172.21.15.131:0/3024509425 conn(0x7f23e40cff40 msgr2=0x7f23e406d3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:59.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.275+0000 7f23e9800640 1 -- 172.21.15.131:0/3024509425 shutdown_connections 2024-05-26T22:55:59.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.275+0000 7f23e9800640 1 -- 172.21.15.131:0/3024509425 wait complete. 2024-05-26T22:55:59.293 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-26T22:55:59.293 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-26T22:55:59.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.503+0000 7f00ed200640 1 Processor -- start 2024-05-26T22:55:59.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.503+0000 7f00ed200640 1 -- start start 2024-05-26T22:55:59.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.504+0000 7f00ed200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f00e815e560 0x7f00e80d0c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:59.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.504+0000 7f00ed200640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f00e80d1180 0x7f00e80cf960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:59.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.504+0000 7f00ed200640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00e80d2030 con 0x7f00e80cfea0 2024-05-26T22:55:59.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.504+0000 7f00ed200640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f00e80d15e0 con 0x7f00e80d1180 2024-05-26T22:55:59.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.504+0000 7f00ed200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f00e80d2c40 con 0x7f00e815e560 2024-05-26T22:55:59.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.505+0000 7f00e5e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f00e815e560 0x7f00e80d0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:59.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.505+0000 7f00e5400640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f00e80d1180 0x7f00e80cf960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:59.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.505+0000 7f00e6800640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f00e80cfea0 0x7f00e8162150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.131:53760/0 (socket says 172.21.15.131:53760) 2024-05-26T22:55:59.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.505+0000 7f00e6800640 1 -- 172.21.15.131:0/255984752 learned_addr learned my addr 172.21.15.131:0/255984752 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:55:59.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.506+0000 7f00e5e00640 1 -- 172.21.15.131:0/255984752 >> v2:172.21.15.187:3300/0 conn(0x7f00e80d1180 msgr2=0x7f00e80cf960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:55:59.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.506+0000 7f00e5e00640 1 --2- 172.21.15.131:0/255984752 >> v2:172.21.15.187:3300/0 conn(0x7f00e80d1180 0x7f00e80cf960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:59.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.506+0000 7f00e5e00640 1 -- 172.21.15.131:0/255984752 >> v1:172.21.15.187:6789/0 conn(0x7f00e80cfea0 legacy=0x7f00e8162150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:59.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.506+0000 7f00e5e00640 1 -- 172.21.15.131:0/255984752 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00e80d2310 con 0x7f00e815e560 2024-05-26T22:55:59.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.507+0000 7f00e5e00640 1 --2- 172.21.15.131:0/255984752 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f00e815e560 0x7f00e80d0c40 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f00d80097a0 tx=0x7f00d802a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3dde581ea99998 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:59.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.508+0000 7f00e4a00640 1 -- 172.21.15.131:0/255984752 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d8037070 con 0x7f00e815e560 2024-05-26T22:55:59.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.508+0000 7f00e4a00640 1 -- 172.21.15.131:0/255984752 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00d802ae50 con 0x7f00e815e560 2024-05-26T22:55:59.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.508+0000 7f00e4a00640 1 -- 172.21.15.131:0/255984752 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d8032660 con 0x7f00e815e560 2024-05-26T22:55:59.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.508+0000 7f00ed200640 1 -- 172.21.15.131:0/255984752 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f00e815e560 msgr2=0x7f00e80d0c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:59.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.508+0000 7f00ed200640 1 --2- 172.21.15.131:0/255984752 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f00e815e560 0x7f00e80d0c40 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f00d80097a0 tx=0x7f00d802a100 comp rx=0 tx=0).stop 2024-05-26T22:55:59.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.508+0000 7f00ed200640 1 -- 172.21.15.131:0/255984752 shutdown_connections 2024-05-26T22:55:59.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.508+0000 7f00ed200640 1 --2- 172.21.15.131:0/255984752 >> v2:172.21.15.187:3300/0 conn(0x7f00e80d1180 0x7f00e80cf960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:59.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.508+0000 7f00ed200640 1 --2- 172.21.15.131:0/255984752 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f00e815e560 0x7f00e80d0c40 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:59.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.508+0000 7f00ed200640 1 -- 172.21.15.131:0/255984752 >> 172.21.15.131:0/255984752 conn(0x7f00e809d640 msgr2=0x7f00e80d42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:55:59.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.509+0000 7f00ed200640 1 -- 172.21.15.131:0/255984752 shutdown_connections 2024-05-26T22:55:59.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.509+0000 7f00ed200640 1 -- 172.21.15.131:0/255984752 wait complete. 2024-05-26T22:55:59.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.510+0000 7f00ed200640 1 Processor -- start 2024-05-26T22:55:59.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.510+0000 7f00ed200640 1 -- start start 2024-05-26T22:55:59.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.511+0000 7f00ed200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f00e80cfea0 0x7f00e815e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:59.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.511+0000 7f00ed200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f00e80d1180 0x7f00e8160430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:59.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.511+0000 7f00ed200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f00e815e560 0x7f00e8160970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:59.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.511+0000 7f00ed200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f00e80d28d0 con 0x7f00e80cfea0 2024-05-26T22:55:59.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.511+0000 7f00ed200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f00e80d0330 con 0x7f00e80d1180 2024-05-26T22:55:59.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.511+0000 7f00ed200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f00e80d30d0 con 0x7f00e815e560 2024-05-26T22:55:59.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.512+0000 7f00e5e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f00e80cfea0 0x7f00e815e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:59.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.512+0000 7f00e5e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f00e80cfea0 0x7f00e815e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:36752/0 (socket says 172.21.15.131:36752) 2024-05-26T22:55:59.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.512+0000 7f00e5e00640 1 -- 172.21.15.131:0/1503919599 learned_addr learned my addr 172.21.15.131:0/1503919599 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:55:59.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.512+0000 7f00e5400640 1 --2- 172.21.15.131:0/1503919599 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f00e80d1180 0x7f00e8160430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:59.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.512+0000 7f00e5e00640 1 -- 172.21.15.131:0/1503919599 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f00e815e560 msgr2=0x7f00e8160970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:59.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.512+0000 7f00e5e00640 1 --2- 172.21.15.131:0/1503919599 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f00e815e560 0x7f00e8160970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:59.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.512+0000 7f00e5e00640 1 -- 172.21.15.131:0/1503919599 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f00e80d1180 msgr2=0x7f00e8160430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:55:59.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.512+0000 7f00e5e00640 1 --2- 172.21.15.131:0/1503919599 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f00e80d1180 0x7f00e8160430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:55:59.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.512+0000 7f00e5e00640 1 -- 172.21.15.131:0/1503919599 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00e80cc2e0 con 0x7f00e80cfea0 2024-05-26T22:55:59.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.512+0000 7f00e5e00640 1 --2- 172.21.15.131:0/1503919599 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f00e80cfea0 0x7f00e815e940 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f00d8000c00 tx=0x7f00d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:59.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.513+0000 7f00de000640 1 -- 172.21.15.131:0/1503919599 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d8041070 con 0x7f00e80cfea0 2024-05-26T22:55:59.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.513+0000 7f00de000640 1 -- 172.21.15.131:0/1503919599 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00d803c460 con 0x7f00e80cfea0 2024-05-26T22:55:59.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.513+0000 7f00de000640 1 -- 172.21.15.131:0/1503919599 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d8037040 con 0x7f00e80cfea0 2024-05-26T22:55:59.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.513+0000 7f00ed200640 1 -- 172.21.15.131:0/1503919599 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00e80cc580 con 0x7f00e80cfea0 2024-05-26T22:55:59.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.513+0000 7f00ed200640 1 -- 172.21.15.131:0/1503919599 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00e80cce20 con 0x7f00e80cfea0 2024-05-26T22:55:59.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.514+0000 7f00ed200640 1 -- 172.21.15.131:0/1503919599 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00e8068440 con 0x7f00e80cfea0 2024-05-26T22:55:59.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.515+0000 7f00de000640 1 -- 172.21.15.131:0/1503919599 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f00d8044070 con 0x7f00e80cfea0 2024-05-26T22:55:59.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.516+0000 7f00de000640 1 --2- 172.21.15.131:0/1503919599 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f00c406be30 0x7f00c406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:55:59.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.517+0000 7f00de000640 1 -- 172.21.15.131:0/1503919599 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 4857+0+0 (secure 0 0 0) 0x7f00d80aaff0 con 0x7f00e80cfea0 2024-05-26T22:55:59.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.517+0000 7f00e5400640 1 --2- 172.21.15.131:0/1503919599 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f00c406be30 0x7f00c406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:55:59.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.517+0000 7f00e5400640 1 --2- 172.21.15.131:0/1503919599 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f00c406be30 0x7f00c406e2f0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f00e80ca530 tx=0x7f00d0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:55:59.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.521+0000 7f00de000640 1 -- 172.21.15.131:0/1503919599 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f00d8079ce0 con 0x7f00e80cfea0 2024-05-26T22:55:59.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:55:59.720+0000 7f00ed200640 1 -- 172.21.15.131:0/1503919599 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f00e80b33a0 con 0x7f00e80cfea0 2024-05-26T22:56:00.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:00.496+0000 7f00de000640 1 -- 172.21.15.131:0/1503919599 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v104) ==== 138+0+0 (secure 0 0 0) 0x7f00d80795e0 con 0x7f00e80cfea0 2024-05-26T22:56:00.496 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_metadata' created 2024-05-26T22:56:00.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:00.500+0000 7f00ed200640 1 -- 172.21.15.131:0/1503919599 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f00c406be30 msgr2=0x7f00c406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:00.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:00.500+0000 7f00ed200640 1 --2- 172.21.15.131:0/1503919599 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f00c406be30 0x7f00c406e2f0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f00e80ca530 tx=0x7f00d0008040 comp rx=0 tx=0).stop 2024-05-26T22:56:00.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:00.500+0000 7f00ed200640 1 -- 172.21.15.131:0/1503919599 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f00e80cfea0 msgr2=0x7f00e815e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:00.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:00.500+0000 7f00ed200640 1 --2- 172.21.15.131:0/1503919599 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f00e80cfea0 0x7f00e815e940 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f00d8000c00 tx=0x7f00d80046c0 comp rx=0 tx=0).stop 2024-05-26T22:56:00.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:00.501+0000 7f00ed200640 1 -- 172.21.15.131:0/1503919599 shutdown_connections 2024-05-26T22:56:00.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:00.501+0000 7f00ed200640 1 --2- 172.21.15.131:0/1503919599 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f00e815e560 0x7f00e8160970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:00.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:00.501+0000 7f00ed200640 1 --2- 172.21.15.131:0/1503919599 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f00c406be30 0x7f00c406e2f0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:00.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:00.501+0000 7f00ed200640 1 --2- 172.21.15.131:0/1503919599 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f00e80d1180 0x7f00e8160430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:00.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:00.501+0000 7f00ed200640 1 --2- 172.21.15.131:0/1503919599 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f00e80cfea0 0x7f00e815e940 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:00.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:00.501+0000 7f00ed200640 1 -- 172.21.15.131:0/1503919599 >> 172.21.15.131:0/1503919599 conn(0x7f00e809d640 msgr2=0x7f00e806c550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:56:00.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:00.501+0000 7f00ed200640 1 -- 172.21.15.131:0/1503919599 shutdown_connections 2024-05-26T22:56:00.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:00.501+0000 7f00ed200640 1 -- 172.21.15.131:0/1503919599 wait complete. 2024-05-26T22:56:00.529 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-26T22:56:00.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:00.716+0000 7eff09e00640 1 Processor -- start 2024-05-26T22:56:00.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:00.716+0000 7eff09e00640 1 -- start start 2024-05-26T22:56:00.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:00.718+0000 7eff09e00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7eff041410d0 0x7eff04165dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:00.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:00.718+0000 7eff09e00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff0413e820 con 0x7eff0406d250 2024-05-26T22:56:00.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:00.718+0000 7eff09e00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff0415c4a0 con 0x7eff0415b730 2024-05-26T22:56:00.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:00.718+0000 7eff09e00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7eff04071260 con 0x7eff041410d0 2024-05-26T22:56:00.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:00.718+0000 7eff08a00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7eff0406d250 0x7eff0406d630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:60448/0 (socket says 172.21.15.131:60448) 2024-05-26T22:56:00.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:00.718+0000 7eff08a00640 1 -- 172.21.15.131:0/3434521217 learned_addr learned my addr 172.21.15.131:0/3434521217 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:56:00.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:00.718+0000 7eff03400640 1 -- 172.21.15.131:0/3434521217 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1956234281 0 0) 0x7eff0413e820 con 0x7eff0406d250 2024-05-26T22:56:00.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:00.718+0000 7eff03400640 1 -- 172.21.15.131:0/3434521217 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efeec005730 con 0x7eff0406d250 2024-05-26T22:56:00.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:00.719+0000 7eff09400640 1 --2- 172.21.15.131:0/3434521217 >> v2:172.21.15.187:3300/0 conn(0x7eff041410d0 0x7eff04165dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:00.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:00.719+0000 7eff03400640 1 -- 172.21.15.131:0/3434521217 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 901825798 0 0) 0x7efeec005730 con 0x7eff0406d250 2024-05-26T22:56:00.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:00.719+0000 7eff03400640 1 -- 172.21.15.131:0/3434521217 >> v2:172.21.15.187:3300/0 conn(0x7eff041410d0 msgr2=0x7eff04165dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:00.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:00.719+0000 7eff03400640 1 --2- 172.21.15.131:0/3434521217 >> v2:172.21.15.187:3300/0 conn(0x7eff041410d0 0x7eff04165dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:00.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:00.719+0000 7eff03400640 1 -- 172.21.15.131:0/3434521217 >> v1:172.21.15.187:6789/0 conn(0x7eff0415b730 legacy=0x7eff0415bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:00.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:00.719+0000 7eff03400640 1 -- 172.21.15.131:0/3434521217 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff040714b0 con 0x7eff0406d250 2024-05-26T22:56:00.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:00.719+0000 7eff03400640 1 -- 172.21.15.131:0/3434521217 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7efef8001670 con 0x7eff0406d250 2024-05-26T22:56:00.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:00.719+0000 7eff03400640 1 -- 172.21.15.131:0/3434521217 >> v1:172.21.15.131:6789/0 conn(0x7eff0406d250 legacy=0x7eff0406d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:00.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:00.719+0000 7eff03400640 1 --2- 172.21.15.131:0/3434521217 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efeec005c40 0x7efeec006080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:00.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:00.719+0000 7eff03400640 1 --2- 172.21.15.131:0/3434521217 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efeec006a10 0x7efeec006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:00.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:00.719+0000 7eff03400640 1 --2- 172.21.15.131:0/3434521217 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efeec0074f0 0x7efeec00dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:00.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:00.720+0000 7eff03400640 1 -- 172.21.15.131:0/3434521217 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7eff04071260 con 0x7efeec005c40 2024-05-26T22:56:00.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:00.720+0000 7eff03400640 1 -- 172.21.15.131:0/3434521217 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7efeec00e3c0 con 0x7efeec006a10 2024-05-26T22:56:00.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:00.720+0000 7eff03400640 1 -- 172.21.15.131:0/3434521217 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7efeec00e5c0 con 0x7efeec0074f0 2024-05-26T22:56:00.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:00.720+0000 7eff08a00640 1 --2- 172.21.15.131:0/3434521217 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efeec005c40 0x7efeec006080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:00.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:00.720+0000 7eff03e00640 1 --2- 172.21.15.131:0/3434521217 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efeec006a10 0x7efeec006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:00.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:00.720+0000 7eff09400640 1 --2- 172.21.15.131:0/3434521217 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efeec0074f0 0x7efeec00dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:00.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:00.721+0000 7eff08a00640 1 -- 172.21.15.131:0/3434521217 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efeec0074f0 msgr2=0x7efeec00dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:00.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:00.721+0000 7eff08a00640 1 --2- 172.21.15.131:0/3434521217 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efeec0074f0 0x7efeec00dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:00.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:00.721+0000 7eff08a00640 1 -- 172.21.15.131:0/3434521217 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efeec006a10 msgr2=0x7efeec006e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:00.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:00.721+0000 7eff08a00640 1 --2- 172.21.15.131:0/3434521217 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efeec006a10 0x7efeec006e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:00.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:00.721+0000 7eff08a00640 1 -- 172.21.15.131:0/3434521217 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efeec005730 con 0x7efeec005c40 2024-05-26T22:56:00.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:00.721+0000 7eff09400640 1 -- 172.21.15.131:0/3434521217 reap_dead start 2024-05-26T22:56:00.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:00.722+0000 7eff08a00640 1 --2- 172.21.15.131:0/3434521217 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efeec005c40 0x7efeec006080 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7efef8007fd0 tx=0x7efeec007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74fe9dc59c576bd7 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:00.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:00.722+0000 7eff03400640 1 -- 172.21.15.131:0/3434521217 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efef8007c20 con 0x7efeec005c40 2024-05-26T22:56:00.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:00.722+0000 7eff03400640 1 -- 172.21.15.131:0/3434521217 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efef8005af0 con 0x7efeec005c40 2024-05-26T22:56:00.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:00.722+0000 7eff03400640 1 -- 172.21.15.131:0/3434521217 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efef8007070 con 0x7efeec005c40 2024-05-26T22:56:00.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:00.723+0000 7eff09e00640 1 -- 172.21.15.131:0/3434521217 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efeec005c40 msgr2=0x7efeec006080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:00.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:00.723+0000 7eff09e00640 1 --2- 172.21.15.131:0/3434521217 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efeec005c40 0x7efeec006080 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7efef8007fd0 tx=0x7efeec007a40 comp rx=0 tx=0).stop 2024-05-26T22:56:00.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:00.723+0000 7eff09e00640 1 -- 172.21.15.131:0/3434521217 shutdown_connections 2024-05-26T22:56:00.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:00.723+0000 7eff09e00640 1 --2- 172.21.15.131:0/3434521217 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efeec005c40 0x7efeec006080 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:00.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:00.723+0000 7eff09e00640 1 -- 172.21.15.131:0/3434521217 >> 172.21.15.131:0/3434521217 conn(0x7eff040b5660 msgr2=0x7eff040b5a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:56:00.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:00.723+0000 7eff09e00640 1 -- 172.21.15.131:0/3434521217 shutdown_connections 2024-05-26T22:56:00.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:00.723+0000 7eff09e00640 1 -- 172.21.15.131:0/3434521217 wait complete. 2024-05-26T22:56:00.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:00.725+0000 7eff09e00640 1 Processor -- start 2024-05-26T22:56:00.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:00.725+0000 7eff09e00640 1 -- start start 2024-05-26T22:56:00.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:00.726+0000 7eff09e00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efeec005c40 0x7eff0413b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:00.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:00.726+0000 7eff09e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7eff0415e3b0 0x7eff0415cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:00.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:00.726+0000 7eff09e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7eff040d08c0 0x7eff040cd860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:00.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:00.726+0000 7eff09e00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7eff04141500 con 0x7eff040d08c0 2024-05-26T22:56:00.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:00.726+0000 7eff09e00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7eff04071a70 con 0x7efeec005c40 2024-05-26T22:56:00.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:00.726+0000 7eff09e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7eff04141680 con 0x7eff0415e3b0 2024-05-26T22:56:00.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:00.727+0000 7eff09400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7eff040d08c0 0x7eff040cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:00.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:00.727+0000 7eff09400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7eff040d08c0 0x7eff040cd860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:36770/0 (socket says 172.21.15.131:36770) 2024-05-26T22:56:00.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:00.727+0000 7eff09400640 1 -- 172.21.15.131:0/2530783969 learned_addr learned my addr 172.21.15.131:0/2530783969 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:56:00.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:00.727+0000 7eff08a00640 1 --2- 172.21.15.131:0/2530783969 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efeec005c40 0x7eff0413b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:00.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:00.727+0000 7eff09400640 1 -- 172.21.15.131:0/2530783969 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7eff0415e3b0 msgr2=0x7eff0415cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:00.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:00.727+0000 7eff03e00640 1 --2- 172.21.15.131:0/2530783969 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7eff0415e3b0 0x7eff0415cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:00.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:00.727+0000 7eff09400640 1 --2- 172.21.15.131:0/2530783969 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7eff0415e3b0 0x7eff0415cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:00.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:00.727+0000 7eff09400640 1 -- 172.21.15.131:0/2530783969 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efeec005c40 msgr2=0x7eff0413b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:00.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:00.727+0000 7eff09400640 1 --2- 172.21.15.131:0/2530783969 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efeec005c40 0x7eff0413b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:00.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:00.727+0000 7eff09400640 1 -- 172.21.15.131:0/2530783969 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff040d0d70 con 0x7eff040d08c0 2024-05-26T22:56:00.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:00.727+0000 7eff09400640 1 --2- 172.21.15.131:0/2530783969 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7eff040d08c0 0x7eff040cd860 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7eff040ccd80 tx=0x7efefc007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:00.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:00.728+0000 7eff00c00640 1 -- 172.21.15.131:0/2530783969 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efefc014410 con 0x7eff040d08c0 2024-05-26T22:56:00.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:00.728+0000 7eff00c00640 1 -- 172.21.15.131:0/2530783969 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efefc019070 con 0x7eff040d08c0 2024-05-26T22:56:00.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:00.728+0000 7eff09e00640 1 -- 172.21.15.131:0/2530783969 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff0415d310 con 0x7eff040d08c0 2024-05-26T22:56:00.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:00.728+0000 7eff00c00640 1 -- 172.21.15.131:0/2530783969 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efefc014810 con 0x7eff040d08c0 2024-05-26T22:56:00.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:00.728+0000 7eff09e00640 1 -- 172.21.15.131:0/2530783969 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff0415d8d0 con 0x7eff040d08c0 2024-05-26T22:56:00.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:00.729+0000 7eff09e00640 1 -- 172.21.15.131:0/2530783969 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff040694e0 con 0x7eff040d08c0 2024-05-26T22:56:00.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:00.730+0000 7eff00c00640 1 -- 172.21.15.131:0/2530783969 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7efefc0149b0 con 0x7eff040d08c0 2024-05-26T22:56:00.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:00.731+0000 7eff00c00640 1 --2- 172.21.15.131:0/2530783969 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7efee406be30 0x7efee406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:00.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:00.732+0000 7eff00c00640 1 -- 172.21.15.131:0/2530783969 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5227+0+0 (secure 0 0 0) 0x7efefc08b2b0 con 0x7eff040d08c0 2024-05-26T22:56:00.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:00.732+0000 7eff08a00640 1 --2- 172.21.15.131:0/2530783969 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7efee406be30 0x7efee406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:00.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:00.732+0000 7eff08a00640 1 --2- 172.21.15.131:0/2530783969 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7efee406be30 0x7efee406e2f0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7efef8032000 tx=0x7efef8009030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:00.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:00.737+0000 7eff00c00640 1 -- 172.21.15.131:0/2530783969 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7eff040d0d70 con 0x7eff040d08c0 2024-05-26T22:56:00.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:00.937+0000 7eff09e00640 1 -- 172.21.15.131:0/2530783969 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7eff040af9b0 con 0x7eff040d08c0 2024-05-26T22:56:01.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:01.512+0000 7eff00c00640 1 -- 172.21.15.131:0/2530783969 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v105) ==== 170+0+0 (secure 0 0 0) 0x7efefc059da0 con 0x7eff040d08c0 2024-05-26T22:56:01.513 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_data' created 2024-05-26T22:56:01.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:01.516+0000 7eff09e00640 1 -- 172.21.15.131:0/2530783969 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7efee406be30 msgr2=0x7efee406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:01.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:01.516+0000 7eff09e00640 1 --2- 172.21.15.131:0/2530783969 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7efee406be30 0x7efee406e2f0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7efef8032000 tx=0x7efef8009030 comp rx=0 tx=0).stop 2024-05-26T22:56:01.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:01.516+0000 7eff09e00640 1 -- 172.21.15.131:0/2530783969 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7eff040d08c0 msgr2=0x7eff040cd860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:01.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:01.517+0000 7eff09e00640 1 --2- 172.21.15.131:0/2530783969 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7eff040d08c0 0x7eff040cd860 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7eff040ccd80 tx=0x7efefc007700 comp rx=0 tx=0).stop 2024-05-26T22:56:01.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:01.517+0000 7eff09e00640 1 -- 172.21.15.131:0/2530783969 shutdown_connections 2024-05-26T22:56:01.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:01.517+0000 7eff09e00640 1 --2- 172.21.15.131:0/2530783969 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7eff0415e3b0 0x7eff0415cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:01.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:01.517+0000 7eff09e00640 1 --2- 172.21.15.131:0/2530783969 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7efee406be30 0x7efee406e2f0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:01.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:01.517+0000 7eff09e00640 1 --2- 172.21.15.131:0/2530783969 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7eff040d08c0 0x7eff040cd860 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:01.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:01.517+0000 7eff09e00640 1 --2- 172.21.15.131:0/2530783969 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efeec005c40 0x7eff0413b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:01.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:01.517+0000 7eff09e00640 1 -- 172.21.15.131:0/2530783969 >> 172.21.15.131:0/2530783969 conn(0x7eff040b5660 msgr2=0x7eff041405e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:56:01.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:01.518+0000 7eff09e00640 1 -- 172.21.15.131:0/2530783969 shutdown_connections 2024-05-26T22:56:01.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:01.518+0000 7eff09e00640 1 -- 172.21.15.131:0/2530783969 wait complete. 2024-05-26T22:56:01.556 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-26T22:56:01.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:01.733+0000 7fe0f0800640 1 Processor -- start 2024-05-26T22:56:01.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:01.733+0000 7fe0f0800640 1 -- start start 2024-05-26T22:56:01.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:01.735+0000 7fe0f0800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe0e806dd30 0x7fe0e806e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:01.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:01.735+0000 7fe0f0800640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0e815c1c0 con 0x7fe0e8155770 2024-05-26T22:56:01.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:01.735+0000 7fe0f0800640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0e815c4a0 con 0x7fe0e80725a0 2024-05-26T22:56:01.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:01.735+0000 7fe0f0800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe0e806e6d0 con 0x7fe0e806dd30 2024-05-26T22:56:01.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:01.735+0000 7fe0ede00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fe0e8155770 0x7fe0e815b700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:60454/0 (socket says 172.21.15.131:60454) 2024-05-26T22:56:01.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:01.735+0000 7fe0ede00640 1 -- 172.21.15.131:0/2617223092 learned_addr learned my addr 172.21.15.131:0/2617223092 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:56:01.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:01.736+0000 7fe0ed400640 1 --2- 172.21.15.131:0/2617223092 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe0e806dd30 0x7fe0e806e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:01.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:01.736+0000 7fe0eca00640 1 -- 172.21.15.131:0/2617223092 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4237515798 0 0) 0x7fe0e815c1c0 con 0x7fe0e8155770 2024-05-26T22:56:01.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:01.736+0000 7fe0eca00640 1 -- 172.21.15.131:0/2617223092 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe0d0003520 con 0x7fe0e8155770 2024-05-26T22:56:01.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:01.736+0000 7fe0eca00640 1 -- 172.21.15.131:0/2617223092 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1564721772 0 0) 0x7fe0d0003520 con 0x7fe0e8155770 2024-05-26T22:56:01.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:01.736+0000 7fe0eca00640 1 -- 172.21.15.131:0/2617223092 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe0e806dd30 msgr2=0x7fe0e806e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:01.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:01.736+0000 7fe0eca00640 1 --2- 172.21.15.131:0/2617223092 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe0e806dd30 0x7fe0e806e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:01.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:01.736+0000 7fe0eca00640 1 -- 172.21.15.131:0/2617223092 >> v1:172.21.15.187:6789/0 conn(0x7fe0e80725a0 legacy=0x7fe0e80729d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:01.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:01.736+0000 7fe0eca00640 1 -- 172.21.15.131:0/2617223092 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0e8071da0 con 0x7fe0e8155770 2024-05-26T22:56:01.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:01.736+0000 7fe0eca00640 1 -- 172.21.15.131:0/2617223092 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7fe0dc001670 con 0x7fe0e8155770 2024-05-26T22:56:01.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:01.736+0000 7fe0eca00640 1 -- 172.21.15.131:0/2617223092 >> v1:172.21.15.131:6789/0 conn(0x7fe0e8155770 legacy=0x7fe0e815b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:01.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:01.736+0000 7fe0eca00640 1 --2- 172.21.15.131:0/2617223092 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0d00039e0 0x7fe0d0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:01.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:01.736+0000 7fe0eca00640 1 --2- 172.21.15.131:0/2617223092 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe0e806dd30 0x7fe0d0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:01.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:01.736+0000 7fe0eca00640 1 --2- 172.21.15.131:0/2617223092 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe0d0004d40 0x7fe0d00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:01.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:01.736+0000 7fe0eca00640 1 -- 172.21.15.131:0/2617223092 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe0d000bb00 con 0x7fe0d0004d40 2024-05-26T22:56:01.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:01.736+0000 7fe0eca00640 1 -- 172.21.15.131:0/2617223092 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe0d000bcb0 con 0x7fe0d00039e0 2024-05-26T22:56:01.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:01.736+0000 7fe0eca00640 1 -- 172.21.15.131:0/2617223092 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe0d000e0c0 con 0x7fe0e806dd30 2024-05-26T22:56:01.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:01.737+0000 7fe0ee800640 1 --2- 172.21.15.131:0/2617223092 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe0d0004d40 0x7fe0d00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:01.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:01.737+0000 7fe0ed400640 1 --2- 172.21.15.131:0/2617223092 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe0e806dd30 0x7fe0d0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:01.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:01.737+0000 7fe0ede00640 1 --2- 172.21.15.131:0/2617223092 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0d00039e0 0x7fe0d0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:01.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:01.737+0000 7fe0ee800640 1 -- 172.21.15.131:0/2617223092 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe0e806dd30 msgr2=0x7fe0d0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:01.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:01.737+0000 7fe0ee800640 1 --2- 172.21.15.131:0/2617223092 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe0e806dd30 0x7fe0d0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:01.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:01.737+0000 7fe0ee800640 1 -- 172.21.15.131:0/2617223092 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0d00039e0 msgr2=0x7fe0d0003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:01.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:01.737+0000 7fe0ee800640 1 --2- 172.21.15.131:0/2617223092 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0d00039e0 0x7fe0d0003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:01.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:01.737+0000 7fe0ee800640 1 -- 172.21.15.131:0/2617223092 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0d0003520 con 0x7fe0d0004d40 2024-05-26T22:56:01.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:01.737+0000 7fe0ed400640 1 --2- 172.21.15.131:0/2617223092 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe0e806dd30 0x7fe0d0004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:56:01.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:01.737+0000 7fe0ede00640 1 --2- 172.21.15.131:0/2617223092 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0d00039e0 0x7fe0d0003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:56:01.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:01.738+0000 7fe0ee800640 1 --2- 172.21.15.131:0/2617223092 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe0d0004d40 0x7fe0d00051c0 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7fe0e4001e20 tx=0x7fe0d0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87e2069ef0bb9de3 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:01.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:01.738+0000 7fe0eca00640 1 -- 172.21.15.131:0/2617223092 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0e4001a70 con 0x7fe0d0004d40 2024-05-26T22:56:01.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:01.738+0000 7fe0eca00640 1 -- 172.21.15.131:0/2617223092 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0e402ae40 con 0x7fe0d0004d40 2024-05-26T22:56:01.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:01.738+0000 7fe0eca00640 1 -- 172.21.15.131:0/2617223092 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0e4003bb0 con 0x7fe0d0004d40 2024-05-26T22:56:01.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:01.739+0000 7fe0f0800640 1 -- 172.21.15.131:0/2617223092 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe0d0004d40 msgr2=0x7fe0d00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:01.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:01.739+0000 7fe0f0800640 1 --2- 172.21.15.131:0/2617223092 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe0d0004d40 0x7fe0d00051c0 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7fe0e4001e20 tx=0x7fe0d0005830 comp rx=0 tx=0).stop 2024-05-26T22:56:01.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:01.739+0000 7fe0ee800640 1 -- 172.21.15.131:0/2617223092 reap_dead start 2024-05-26T22:56:01.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:01.739+0000 7fe0f0800640 1 -- 172.21.15.131:0/2617223092 shutdown_connections 2024-05-26T22:56:01.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:01.739+0000 7fe0f0800640 1 -- 172.21.15.131:0/2617223092 >> 172.21.15.131:0/2617223092 conn(0x7fe0e806bc90 msgr2=0x7fe0e80b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:56:01.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:01.739+0000 7fe0f0800640 1 -- 172.21.15.131:0/2617223092 shutdown_connections 2024-05-26T22:56:01.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:01.739+0000 7fe0f0800640 1 -- 172.21.15.131:0/2617223092 wait complete. 2024-05-26T22:56:01.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:01.741+0000 7fe0f0800640 1 Processor -- start 2024-05-26T22:56:01.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:01.741+0000 7fe0f0800640 1 -- start start 2024-05-26T22:56:01.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:01.742+0000 7fe0f0800640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0e80cffb0 0x7fe0e81570c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:01.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:01.742+0000 7fe0f0800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe0e80d0390 0x7fe0e8157600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:01.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:01.742+0000 7fe0f0800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe0e809c430 0x7fe0e809c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:01.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:01.742+0000 7fe0f0800640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe0e815c8c0 con 0x7fe0e809c430 2024-05-26T22:56:01.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:01.742+0000 7fe0f0800640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe0e8155b50 con 0x7fe0e80cffb0 2024-05-26T22:56:01.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:01.742+0000 7fe0f0800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe0e815be10 con 0x7fe0e80d0390 2024-05-26T22:56:01.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:01.742+0000 7fe0ee800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe0e809c430 0x7fe0e809c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:01.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:01.742+0000 7fe0ee800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe0e809c430 0x7fe0e809c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:36796/0 (socket says 172.21.15.131:36796) 2024-05-26T22:56:01.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:01.742+0000 7fe0ee800640 1 -- 172.21.15.131:0/2476712556 learned_addr learned my addr 172.21.15.131:0/2476712556 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:56:01.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:01.743+0000 7fe0ed400640 1 --2- 172.21.15.131:0/2476712556 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe0e80d0390 0x7fe0e8157600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:01.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:01.743+0000 7fe0ee800640 1 -- 172.21.15.131:0/2476712556 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe0e80d0390 msgr2=0x7fe0e8157600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:01.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:01.743+0000 7fe0ee800640 1 --2- 172.21.15.131:0/2476712556 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe0e80d0390 0x7fe0e8157600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:01.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:01.743+0000 7fe0ee800640 1 -- 172.21.15.131:0/2476712556 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0e80cffb0 msgr2=0x7fe0e81570c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:01.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:01.743+0000 7fe0ee800640 1 --2- 172.21.15.131:0/2476712556 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0e80cffb0 0x7fe0e81570c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:01.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:01.743+0000 7fe0ee800640 1 -- 172.21.15.131:0/2476712556 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0e809aaa0 con 0x7fe0e809c430 2024-05-26T22:56:01.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:01.743+0000 7fe0ee800640 1 --2- 172.21.15.131:0/2476712556 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe0e809c430 0x7fe0e809c8b0 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7fe0e402a8f0 tx=0x7fe0e40014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:01.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:01.743+0000 7fe0e2000640 1 -- 172.21.15.131:0/2476712556 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0e4037040 con 0x7fe0e809c430 2024-05-26T22:56:01.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:01.743+0000 7fe0e2000640 1 -- 172.21.15.131:0/2476712556 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0e4006b50 con 0x7fe0e809c430 2024-05-26T22:56:01.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:01.743+0000 7fe0f0800640 1 -- 172.21.15.131:0/2476712556 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0e8157b40 con 0x7fe0e809c430 2024-05-26T22:56:01.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:01.743+0000 7fe0e2000640 1 -- 172.21.15.131:0/2476712556 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0e4001a70 con 0x7fe0e809c430 2024-05-26T22:56:01.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:01.744+0000 7fe0f0800640 1 -- 172.21.15.131:0/2476712556 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0e809b020 con 0x7fe0e809c430 2024-05-26T22:56:01.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:01.745+0000 7fe0f0800640 1 -- 172.21.15.131:0/2476712556 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0e8074e90 con 0x7fe0e809c430 2024-05-26T22:56:01.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:01.746+0000 7fe0e2000640 1 -- 172.21.15.131:0/2476712556 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe0e4005400 con 0x7fe0e809c430 2024-05-26T22:56:01.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:01.747+0000 7fe0e2000640 1 --2- 172.21.15.131:0/2476712556 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fe0cc06be30 0x7fe0cc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:01.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:01.747+0000 7fe0e2000640 1 -- 172.21.15.131:0/2476712556 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5609+0+0 (secure 0 0 0) 0x7fe0e40aaca0 con 0x7fe0e809c430 2024-05-26T22:56:01.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:01.747+0000 7fe0ede00640 1 --2- 172.21.15.131:0/2476712556 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fe0cc06be30 0x7fe0cc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:01.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:01.748+0000 7fe0ede00640 1 --2- 172.21.15.131:0/2476712556 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fe0cc06be30 0x7fe0cc06e2f0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc0017e0 tx=0x7fe0dc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:01.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:01.751+0000 7fe0e2000640 1 -- 172.21.15.131:0/2476712556 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fe0e40797b0 con 0x7fe0e809c430 2024-05-26T22:56:01.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:01.977+0000 7fe0f0800640 1 -- 172.21.15.131:0/2476712556 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fe0e80695c0 con 0x7fe0e809c430 2024-05-26T22:56:02.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:02.516+0000 7fe0e2000640 1 -- 172.21.15.131:0/2476712556 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 16 and data pool 17 v106) ==== 174+0+0 (secure 0 0 0) 0x7fe0e40790b0 con 0x7fe0e809c430 2024-05-26T22:56:02.516 INFO:teuthology.orchestra.run.smithi131.stderr:new fs with metadata pool 16 and data pool 17 2024-05-26T22:56:02.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:02.520+0000 7fe0f0800640 1 -- 172.21.15.131:0/2476712556 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fe0cc06be30 msgr2=0x7fe0cc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:02.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:02.520+0000 7fe0f0800640 1 --2- 172.21.15.131:0/2476712556 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fe0cc06be30 0x7fe0cc06e2f0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc0017e0 tx=0x7fe0dc009040 comp rx=0 tx=0).stop 2024-05-26T22:56:02.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:02.520+0000 7fe0f0800640 1 -- 172.21.15.131:0/2476712556 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe0e809c430 msgr2=0x7fe0e809c8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:02.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:02.520+0000 7fe0f0800640 1 --2- 172.21.15.131:0/2476712556 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe0e809c430 0x7fe0e809c8b0 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7fe0e402a8f0 tx=0x7fe0e40014a0 comp rx=0 tx=0).stop 2024-05-26T22:56:02.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:02.521+0000 7fe0f0800640 1 -- 172.21.15.131:0/2476712556 shutdown_connections 2024-05-26T22:56:02.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:02.521+0000 7fe0f0800640 1 --2- 172.21.15.131:0/2476712556 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe0e80d0390 0x7fe0e8157600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:02.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:02.521+0000 7fe0f0800640 1 --2- 172.21.15.131:0/2476712556 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fe0cc06be30 0x7fe0cc06e2f0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:02.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:02.521+0000 7fe0f0800640 1 --2- 172.21.15.131:0/2476712556 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe0e809c430 0x7fe0e809c8b0 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:02.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:02.521+0000 7fe0f0800640 1 --2- 172.21.15.131:0/2476712556 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0e80cffb0 0x7fe0e81570c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:02.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:02.521+0000 7fe0f0800640 1 -- 172.21.15.131:0/2476712556 >> 172.21.15.131:0/2476712556 conn(0x7fe0e806bc90 msgr2=0x7fe0e80b5280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:56:02.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:02.521+0000 7fe0f0800640 1 -- 172.21.15.131:0/2476712556 shutdown_connections 2024-05-26T22:56:02.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:02.521+0000 7fe0f0800640 1 -- 172.21.15.131:0/2476712556 wait complete. 2024-05-26T22:56:02.547 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-26T22:56:02.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:02.719+0000 7efeeb200640 1 Processor -- start 2024-05-26T22:56:02.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:02.719+0000 7efeeb200640 1 -- start start 2024-05-26T22:56:02.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:02.721+0000 7efeeb200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7efee4158c00 0x7efee406d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:02.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:02.721+0000 7efeeb200640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efee40b3e50 con 0x7efee40b38a0 2024-05-26T22:56:02.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:02.721+0000 7efeeb200640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efee40b4130 con 0x7efee406daf0 2024-05-26T22:56:02.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:02.721+0000 7efeeb200640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7efee406df50 con 0x7efee4158c00 2024-05-26T22:56:02.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:02.721+0000 7efee3e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7efee4158c00 0x7efee406d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:02.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:02.721+0000 7efee9200640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7efee40b38a0 0x7efee415fc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:60466/0 (socket says 172.21.15.131:60466) 2024-05-26T22:56:02.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:02.721+0000 7efee9200640 1 -- 172.21.15.131:0/1590733327 learned_addr learned my addr 172.21.15.131:0/1590733327 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:56:02.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:02.722+0000 7efee3e00640 1 -- 172.21.15.131:0/1590733327 >> v1:172.21.15.187:6789/0 conn(0x7efee406daf0 legacy=0x7efee40b3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:02.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:02.722+0000 7efee3e00640 1 -- 172.21.15.131:0/1590733327 >> v1:172.21.15.131:6789/0 conn(0x7efee40b38a0 legacy=0x7efee415fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:02.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:02.722+0000 7efee2a00640 1 -- 172.21.15.131:0/1590733327 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3892804465 0 0) 0x7efee40b3e50 con 0x7efee40b38a0 2024-05-26T22:56:02.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:02.722+0000 7efee3e00640 1 -- 172.21.15.131:0/1590733327 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efee40b4410 con 0x7efee4158c00 2024-05-26T22:56:02.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:02.723+0000 7efee3e00640 1 --2- 172.21.15.131:0/1590733327 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efee4158c00 0x7efee406d5b0 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7efed40097a0 tx=0x7efed402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1a4124b8ecd53da server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:02.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:02.723+0000 7efee2a00640 1 -- 172.21.15.131:0/1590733327 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed4037070 con 0x7efee4158c00 2024-05-26T22:56:02.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:02.723+0000 7efee2a00640 1 -- 172.21.15.131:0/1590733327 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efed402ae50 con 0x7efee4158c00 2024-05-26T22:56:02.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:02.723+0000 7efee2a00640 1 -- 172.21.15.131:0/1590733327 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed4032660 con 0x7efee4158c00 2024-05-26T22:56:02.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:02.724+0000 7efeeb200640 1 -- 172.21.15.131:0/1590733327 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efee4158c00 msgr2=0x7efee406d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:02.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:02.724+0000 7efeeb200640 1 --2- 172.21.15.131:0/1590733327 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efee4158c00 0x7efee406d5b0 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7efed40097a0 tx=0x7efed402a140 comp rx=0 tx=0).stop 2024-05-26T22:56:02.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:02.724+0000 7efeeb200640 1 -- 172.21.15.131:0/1590733327 shutdown_connections 2024-05-26T22:56:02.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:02.724+0000 7efeeb200640 1 --2- 172.21.15.131:0/1590733327 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efee4158c00 0x7efee406d5b0 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:02.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:02.724+0000 7efeeb200640 1 -- 172.21.15.131:0/1590733327 >> 172.21.15.131:0/1590733327 conn(0x7efee40b1e90 msgr2=0x7efee40b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:56:02.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:02.724+0000 7efeeb200640 1 -- 172.21.15.131:0/1590733327 shutdown_connections 2024-05-26T22:56:02.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:02.724+0000 7efeeb200640 1 -- 172.21.15.131:0/1590733327 wait complete. 2024-05-26T22:56:02.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:02.726+0000 7efeeb200640 1 Processor -- start 2024-05-26T22:56:02.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:02.726+0000 7efeeb200640 1 -- start start 2024-05-26T22:56:02.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:02.727+0000 7efeeb200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efee406daf0 0x7efee4147520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:02.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:02.727+0000 7efeeb200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efee40b38a0 0x7efee4099930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:02.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:02.727+0000 7efeeb200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efee4158c00 0x7efee40ae1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:02.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:02.727+0000 7efeeb200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7efee4160360 con 0x7efee406daf0 2024-05-26T22:56:02.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:02.727+0000 7efeeb200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7efee40b49d0 con 0x7efee4158c00 2024-05-26T22:56:02.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:02.727+0000 7efeeb200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7efee41604e0 con 0x7efee40b38a0 2024-05-26T22:56:02.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:02.727+0000 7efee3e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efee406daf0 0x7efee4147520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:02.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:02.727+0000 7efee3e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efee406daf0 0x7efee4147520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:36822/0 (socket says 172.21.15.131:36822) 2024-05-26T22:56:02.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:02.727+0000 7efee3e00640 1 -- 172.21.15.131:0/3260221475 learned_addr learned my addr 172.21.15.131:0/3260221475 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:56:02.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:02.728+0000 7efee9200640 1 --2- 172.21.15.131:0/3260221475 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efee4158c00 0x7efee40ae1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:02.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:02.728+0000 7efee3400640 1 --2- 172.21.15.131:0/3260221475 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efee40b38a0 0x7efee4099930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:02.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:02.728+0000 7efee3e00640 1 -- 172.21.15.131:0/3260221475 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efee40b38a0 msgr2=0x7efee4099930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:02.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:02.728+0000 7efee3e00640 1 --2- 172.21.15.131:0/3260221475 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efee40b38a0 0x7efee4099930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:02.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:02.728+0000 7efee3e00640 1 -- 172.21.15.131:0/3260221475 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efee4158c00 msgr2=0x7efee40ae1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:02.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:02.728+0000 7efee3e00640 1 --2- 172.21.15.131:0/3260221475 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efee4158c00 0x7efee40ae1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:02.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:02.728+0000 7efee3e00640 1 -- 172.21.15.131:0/3260221475 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efee4099e70 con 0x7efee406daf0 2024-05-26T22:56:02.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:02.728+0000 7efee3400640 1 --2- 172.21.15.131:0/3260221475 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efee40b38a0 0x7efee4099930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:56:02.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:02.728+0000 7efee9200640 1 --2- 172.21.15.131:0/3260221475 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efee4158c00 0x7efee40ae1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:56:02.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:02.728+0000 7efee3e00640 1 --2- 172.21.15.131:0/3260221475 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efee406daf0 0x7efee4147520 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7efed4000c00 tx=0x7efed40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:02.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:02.728+0000 7efedbe00640 1 -- 172.21.15.131:0/3260221475 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed4037040 con 0x7efee406daf0 2024-05-26T22:56:02.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:02.728+0000 7efedbe00640 1 -- 172.21.15.131:0/3260221475 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efed4004230 con 0x7efee406daf0 2024-05-26T22:56:02.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:02.728+0000 7efedbe00640 1 -- 172.21.15.131:0/3260221475 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed4003610 con 0x7efee406daf0 2024-05-26T22:56:02.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:02.729+0000 7efeeb200640 1 -- 172.21.15.131:0/3260221475 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efee4147b20 con 0x7efee406daf0 2024-05-26T22:56:02.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:02.729+0000 7efeeb200640 1 -- 172.21.15.131:0/3260221475 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efee415a640 con 0x7efee406daf0 2024-05-26T22:56:02.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:02.730+0000 7efeeb200640 1 -- 172.21.15.131:0/3260221475 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efee40694e0 con 0x7efee406daf0 2024-05-26T22:56:02.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:02.731+0000 7efedbe00640 1 -- 172.21.15.131:0/3260221475 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7efed40037b0 con 0x7efee406daf0 2024-05-26T22:56:02.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:02.732+0000 7efedbe00640 1 --2- 172.21.15.131:0/3260221475 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7efec006be30 0x7efec006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:02.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:02.732+0000 7efedbe00640 1 -- 172.21.15.131:0/3260221475 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 5709+0+0 (secure 0 0 0) 0x7efed40abda0 con 0x7efee406daf0 2024-05-26T22:56:02.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:02.732+0000 7efee3400640 1 --2- 172.21.15.131:0/3260221475 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7efec006be30 0x7efec006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:02.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:02.733+0000 7efee3400640 1 --2- 172.21.15.131:0/3260221475 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7efec006be30 0x7efec006e2f0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7efee40af370 tx=0x7efecc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:02.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:02.736+0000 7efedbe00640 1 -- 172.21.15.131:0/3260221475 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7efed407a860 con 0x7efee406daf0 2024-05-26T22:56:02.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:02.936+0000 7efeeb200640 1 -- 172.21.15.131:0/3260221475 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efee40694e0 con 0x7efee406daf0 2024-05-26T22:56:02.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:02.939+0000 7efedbe00640 1 -- 172.21.15.131:0/3260221475 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) ==== 74+0+16664 (secure 0 0 0) 0x7efed407a160 con 0x7efee406daf0 2024-05-26T22:56:02.941 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:56:02.941 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":106,"fsid":"46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f","created":"2024-05-26T22:47:57.288905+0000","modified":"2024-05-26T22:56:01.979406+0000","last_up_change":"2024-05-26T22:48:09.602207+0000","last_in_change":"2024-05-26T22:48:00.687341+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-26T22:48:10.463339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-05-26T22:55:59.775225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-05-26T22:56:00.988905+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"814eaf62-a18d-494b-8e27-fd8fb88adea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6801","nonce":2350940051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6803","nonce":2350940051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6807","nonce":2350940051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6805","nonce":2350940051}]},"public_addr":"172.21.15.131:6801/2350940051","cluster_addr":"172.21.15.131:6803/2350940051","heartbeat_back_addr":"172.21.15.131:6807/2350940051","heartbeat_front_addr":"172.21.15.131:6805/2350940051","state":["exists","up"]},{"osd":1,"uuid":"42e66287-f6c6-4e26-af3c-6dfaa3536d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6825","nonce":1525528247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6827","nonce":1525528247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6831","nonce":1525528247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6829","nonce":1525528247}]},"public_addr":"172.21.15.131:6825/1525528247","cluster_addr":"172.21.15.131:6827/1525528247","heartbeat_back_addr":"172.21.15.131:6831/1525528247","heartbeat_front_addr":"172.21.15.131:6829/1525528247","state":["exists","up"]},{"osd":2,"uuid":"0da34673-04a2-4fd0-b57a-0ff2b679b043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6817","nonce":3627842210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6819","nonce":3627842210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6823","nonce":3627842210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6821","nonce":3627842210}]},"public_addr":"172.21.15.131:6817/3627842210","cluster_addr":"172.21.15.131:6819/3627842210","heartbeat_back_addr":"172.21.15.131:6823/3627842210","heartbeat_front_addr":"172.21.15.131:6821/3627842210","state":["exists","up"]},{"osd":3,"uuid":"cf117a27-d6b2-43d3-ae52-b47988bf8eed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6809","nonce":350790591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6811","nonce":350790591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6815","nonce":350790591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6813","nonce":350790591}]},"public_addr":"172.21.15.131:6809/350790591","cluster_addr":"172.21.15.131:6811/350790591","heartbeat_back_addr":"172.21.15.131:6815/350790591","heartbeat_front_addr":"172.21.15.131:6813/350790591","state":["exists","up"]},{"osd":4,"uuid":"a092ccfc-be89-4a84-88d5-0f781fa715eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6816","nonce":244560995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6819","nonce":244560995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6823","nonce":244560995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6821","nonce":244560995}]},"public_addr":"172.21.15.187:6816/244560995","cluster_addr":"172.21.15.187:6819/244560995","heartbeat_back_addr":"172.21.15.187:6823/244560995","heartbeat_front_addr":"172.21.15.187:6821/244560995","state":["exists","up"]},{"osd":5,"uuid":"7513ba69-dd00-4a35-b190-9e8cc3b2bdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6801","nonce":324619398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6803","nonce":324619398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6812","nonce":324619398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6807","nonce":324619398}]},"public_addr":"172.21.15.187:6801/324619398","cluster_addr":"172.21.15.187:6803/324619398","heartbeat_back_addr":"172.21.15.187:6812/324619398","heartbeat_front_addr":"172.21.15.187:6807/324619398","state":["exists","up"]},{"osd":6,"uuid":"50081810-c093-4d2f-a4d7-b7c695c5400d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6806","nonce":1391347786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6809","nonce":1391347786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6818","nonce":1391347786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6813","nonce":1391347786}]},"public_addr":"172.21.15.187:6806/1391347786","cluster_addr":"172.21.15.187:6809/1391347786","heartbeat_back_addr":"172.21.15.187:6818/1391347786","heartbeat_front_addr":"172.21.15.187:6813/1391347786","state":["exists","up"]},{"osd":7,"uuid":"775dae05-54f5-4ad4-bbce-92863c6ed2ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6825","nonce":1220065622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6827","nonce":1220065622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6831","nonce":1220065622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6829","nonce":1220065622}]},"public_addr":"172.21.15.187:6825/1220065622","cluster_addr":"172.21.15.187:6827/1220065622","heartbeat_back_addr":"172.21.15.187:6831/1220065622","heartbeat_front_addr":"172.21.15.187:6829/1220065622","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:05.983520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.299516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.067323+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.307748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.018742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.258775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.010912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.295355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-26T22:56:02.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:02.942+0000 7efeeb200640 1 -- 172.21.15.131:0/3260221475 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7efec006be30 msgr2=0x7efec006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:02.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:02.942+0000 7efeeb200640 1 --2- 172.21.15.131:0/3260221475 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7efec006be30 0x7efec006e2f0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7efee40af370 tx=0x7efecc009040 comp rx=0 tx=0).stop 2024-05-26T22:56:02.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:02.942+0000 7efeeb200640 1 -- 172.21.15.131:0/3260221475 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efee406daf0 msgr2=0x7efee4147520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:02.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:02.942+0000 7efeeb200640 1 --2- 172.21.15.131:0/3260221475 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efee406daf0 0x7efee4147520 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7efed4000c00 tx=0x7efed40046c0 comp rx=0 tx=0).stop 2024-05-26T22:56:02.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:02.942+0000 7efeeb200640 1 -- 172.21.15.131:0/3260221475 shutdown_connections 2024-05-26T22:56:02.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:02.942+0000 7efeeb200640 1 --2- 172.21.15.131:0/3260221475 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efee40b38a0 0x7efee4099930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:02.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:02.942+0000 7efeeb200640 1 --2- 172.21.15.131:0/3260221475 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7efec006be30 0x7efec006e2f0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:02.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:02.942+0000 7efeeb200640 1 --2- 172.21.15.131:0/3260221475 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efee4158c00 0x7efee40ae1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:02.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:02.942+0000 7efeeb200640 1 --2- 172.21.15.131:0/3260221475 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efee406daf0 0x7efee4147520 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:02.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:02.942+0000 7efeeb200640 1 -- 172.21.15.131:0/3260221475 >> 172.21.15.131:0/3260221475 conn(0x7efee40b1e90 msgr2=0x7efee40b2400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:56:02.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:02.942+0000 7efeeb200640 1 -- 172.21.15.131:0/3260221475 shutdown_connections 2024-05-26T22:56:02.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:02.942+0000 7efeeb200640 1 -- 172.21.15.131:0/3260221475 wait complete. 2024-05-26T22:56:02.960 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-26T22:56:03.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.167+0000 7ff34ea00640 1 Processor -- start 2024-05-26T22:56:03.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.168+0000 7ff34ea00640 1 -- start start 2024-05-26T22:56:03.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.169+0000 7ff34ea00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7ff348071d80 0x7ff348072160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:03.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.169+0000 7ff34ea00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7ff3480ad450 0x7ff34806eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:03.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.169+0000 7ff34ea00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff3480ad8e0 0x7ff34806f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:03.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.169+0000 7ff34ea00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7ff3480726a0 con 0x7ff348071d80 2024-05-26T22:56:03.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.169+0000 7ff34ea00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7ff3480b5190 con 0x7ff3480ad450 2024-05-26T22:56:03.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.169+0000 7ff34ea00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff34806dee0 con 0x7ff3480ad8e0 2024-05-26T22:56:03.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.169+0000 7ff347e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7ff348071d80 0x7ff348072160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:03.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.170+0000 7ff347e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7ff348071d80 0x7ff348072160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:36824/0 (socket says 172.21.15.131:36824) 2024-05-26T22:56:03.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.170+0000 7ff347e00640 1 -- 172.21.15.131:0/2028399274 learned_addr learned my addr 172.21.15.131:0/2028399274 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:56:03.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.170+0000 7ff33f400640 1 --2- 172.21.15.131:0/2028399274 >> v2:172.21.15.187:3300/0 conn(0x7ff3480ad450 0x7ff34806eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:03.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.170+0000 7ff34ca00640 1 --2- 172.21.15.131:0/2028399274 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff3480ad8e0 0x7ff34806f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:03.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.170+0000 7ff347e00640 1 -- 172.21.15.131:0/2028399274 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff3480ad8e0 msgr2=0x7ff34806f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:03.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.170+0000 7ff347e00640 1 --2- 172.21.15.131:0/2028399274 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff3480ad8e0 0x7ff34806f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:03.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.170+0000 7ff347e00640 1 -- 172.21.15.131:0/2028399274 >> v2:172.21.15.187:3300/0 conn(0x7ff3480ad450 msgr2=0x7ff34806eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:03.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.170+0000 7ff347e00640 1 --2- 172.21.15.131:0/2028399274 >> v2:172.21.15.187:3300/0 conn(0x7ff3480ad450 0x7ff34806eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:03.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.170+0000 7ff347e00640 1 -- 172.21.15.131:0/2028399274 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff34806e1c0 con 0x7ff348071d80 2024-05-26T22:56:03.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.170+0000 7ff34ca00640 1 --2- 172.21.15.131:0/2028399274 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff3480ad8e0 0x7ff34806f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:56:03.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.171+0000 7ff347e00640 1 --2- 172.21.15.131:0/2028399274 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff348071d80 0x7ff348072160 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7ff34806c1b0 tx=0x7ff33802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b3e684b4f6925ec server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:03.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.172+0000 7ff347400640 1 -- 172.21.15.131:0/2028399274 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff338037070 con 0x7ff348071d80 2024-05-26T22:56:03.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.172+0000 7ff347400640 1 -- 172.21.15.131:0/2028399274 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff338032440 con 0x7ff348071d80 2024-05-26T22:56:03.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.172+0000 7ff347400640 1 -- 172.21.15.131:0/2028399274 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff338032810 con 0x7ff348071d80 2024-05-26T22:56:03.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.172+0000 7ff34ea00640 1 -- 172.21.15.131:0/2028399274 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff348071d80 msgr2=0x7ff348072160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:03.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.172+0000 7ff34ea00640 1 --2- 172.21.15.131:0/2028399274 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff348071d80 0x7ff348072160 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7ff34806c1b0 tx=0x7ff33802a140 comp rx=0 tx=0).stop 2024-05-26T22:56:03.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.172+0000 7ff34ea00640 1 -- 172.21.15.131:0/2028399274 shutdown_connections 2024-05-26T22:56:03.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.172+0000 7ff34ea00640 1 --2- 172.21.15.131:0/2028399274 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff3480ad8e0 0x7ff34806f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:03.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.172+0000 7ff34ea00640 1 --2- 172.21.15.131:0/2028399274 >> v2:172.21.15.187:3300/0 conn(0x7ff3480ad450 0x7ff34806eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:03.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.172+0000 7ff34ea00640 1 --2- 172.21.15.131:0/2028399274 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff348071d80 0x7ff348072160 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:03.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.172+0000 7ff34ea00640 1 -- 172.21.15.131:0/2028399274 >> 172.21.15.131:0/2028399274 conn(0x7ff3480b1e90 msgr2=0x7ff3480b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:56:03.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.173+0000 7ff34ea00640 1 -- 172.21.15.131:0/2028399274 shutdown_connections 2024-05-26T22:56:03.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.173+0000 7ff34ea00640 1 -- 172.21.15.131:0/2028399274 wait complete. 2024-05-26T22:56:03.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.174+0000 7ff34ea00640 1 Processor -- start 2024-05-26T22:56:03.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.175+0000 7ff34ea00640 1 -- start start 2024-05-26T22:56:03.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.176+0000 7ff34ea00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff348071d80 0x7ff348147860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:03.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.176+0000 7ff34ea00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff3480ad450 0x7ff348152570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:03.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.176+0000 7ff34ea00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff3481480b0 0x7ff34815a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:03.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.176+0000 7ff34ea00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff34806f720 con 0x7ff348071d80 2024-05-26T22:56:03.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.176+0000 7ff34ea00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff34806e780 con 0x7ff3481480b0 2024-05-26T22:56:03.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.176+0000 7ff34ea00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff34815e020 con 0x7ff3480ad450 2024-05-26T22:56:03.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.176+0000 7ff347e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff348071d80 0x7ff348147860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:03.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.176+0000 7ff347e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff348071d80 0x7ff348147860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:36840/0 (socket says 172.21.15.131:36840) 2024-05-26T22:56:03.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.176+0000 7ff347e00640 1 -- 172.21.15.131:0/842816410 learned_addr learned my addr 172.21.15.131:0/842816410 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:56:03.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.176+0000 7ff347e00640 1 -- 172.21.15.131:0/842816410 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff3480ad450 msgr2=0x7ff348152570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:03.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.176+0000 7ff33f400640 1 --2- 172.21.15.131:0/842816410 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff3480ad450 0x7ff348152570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:03.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.176+0000 7ff34ca00640 1 --2- 172.21.15.131:0/842816410 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff3481480b0 0x7ff34815a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:03.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.176+0000 7ff347e00640 1 --2- 172.21.15.131:0/842816410 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff3480ad450 0x7ff348152570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:03.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.176+0000 7ff347e00640 1 -- 172.21.15.131:0/842816410 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff3481480b0 msgr2=0x7ff34815a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:03.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.176+0000 7ff347e00640 1 --2- 172.21.15.131:0/842816410 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff3481480b0 0x7ff34815a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:03.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.176+0000 7ff347e00640 1 -- 172.21.15.131:0/842816410 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff348153fe0 con 0x7ff348071d80 2024-05-26T22:56:03.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.176+0000 7ff33f400640 1 --2- 172.21.15.131:0/842816410 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff3480ad450 0x7ff348152570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:56:03.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.177+0000 7ff34ca00640 1 --2- 172.21.15.131:0/842816410 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff3481480b0 0x7ff34815a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:56:03.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.177+0000 7ff347e00640 1 --2- 172.21.15.131:0/842816410 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff348071d80 0x7ff348147860 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7ff33802a180 tx=0x7ff3380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:03.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.179+0000 7ff344c00640 1 -- 172.21.15.131:0/842816410 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff338041020 con 0x7ff348071d80 2024-05-26T22:56:03.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.179+0000 7ff344c00640 1 -- 172.21.15.131:0/842816410 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3380306c0 con 0x7ff348071d80 2024-05-26T22:56:03.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.179+0000 7ff344c00640 1 -- 172.21.15.131:0/842816410 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff338037040 con 0x7ff348071d80 2024-05-26T22:56:03.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.179+0000 7ff34ea00640 1 -- 172.21.15.131:0/842816410 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff348154280 con 0x7ff348071d80 2024-05-26T22:56:03.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.180+0000 7ff34ea00640 1 -- 172.21.15.131:0/842816410 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff348147da0 con 0x7ff348071d80 2024-05-26T22:56:03.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.181+0000 7ff34ea00640 1 -- 172.21.15.131:0/842816410 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3480cd990 con 0x7ff348071d80 2024-05-26T22:56:03.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.182+0000 7ff344c00640 1 -- 172.21.15.131:0/842816410 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff338044020 con 0x7ff348071d80 2024-05-26T22:56:03.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.183+0000 7ff344c00640 1 --2- 172.21.15.131:0/842816410 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ff32406be30 0x7ff32406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:03.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.183+0000 7ff344c00640 1 -- 172.21.15.131:0/842816410 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5709+0+0 (secure 0 0 0) 0x7ff3380ab610 con 0x7ff348071d80 2024-05-26T22:56:03.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.183+0000 7ff33f400640 1 --2- 172.21.15.131:0/842816410 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ff32406be30 0x7ff32406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:03.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.184+0000 7ff33f400640 1 --2- 172.21.15.131:0/842816410 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ff32406be30 0x7ff32406e2f0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7ff348148a50 tx=0x7ff330008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:03.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.188+0000 7ff344c00640 1 -- 172.21.15.131:0/842816410 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7ff3480cd990 con 0x7ff348071d80 2024-05-26T22:56:03.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.389+0000 7ff34ea00640 1 -- 172.21.15.131:0/842816410 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff3480694e0 con 0x7ff348071d80 2024-05-26T22:56:03.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.392+0000 7ff344c00640 1 -- 172.21.15.131:0/842816410 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v107) ==== 74+0+16651 (secure 0 0 0) 0x7ff33804b030 con 0x7ff348071d80 2024-05-26T22:56:03.392 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:56:03.392 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":107,"fsid":"46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f","created":"2024-05-26T22:47:57.288905+0000","modified":"2024-05-26T22:56:03.175526+0000","last_up_change":"2024-05-26T22:48:09.602207+0000","last_in_change":"2024-05-26T22:48:00.687341+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-26T22:48:10.463339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-05-26T22:55:59.775225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-05-26T22:56:00.988905+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"814eaf62-a18d-494b-8e27-fd8fb88adea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6801","nonce":2350940051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6803","nonce":2350940051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6807","nonce":2350940051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6805","nonce":2350940051}]},"public_addr":"172.21.15.131:6801/2350940051","cluster_addr":"172.21.15.131:6803/2350940051","heartbeat_back_addr":"172.21.15.131:6807/2350940051","heartbeat_front_addr":"172.21.15.131:6805/2350940051","state":["exists","up"]},{"osd":1,"uuid":"42e66287-f6c6-4e26-af3c-6dfaa3536d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6825","nonce":1525528247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6827","nonce":1525528247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6831","nonce":1525528247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6829","nonce":1525528247}]},"public_addr":"172.21.15.131:6825/1525528247","cluster_addr":"172.21.15.131:6827/1525528247","heartbeat_back_addr":"172.21.15.131:6831/1525528247","heartbeat_front_addr":"172.21.15.131:6829/1525528247","state":["exists","up"]},{"osd":2,"uuid":"0da34673-04a2-4fd0-b57a-0ff2b679b043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6817","nonce":3627842210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6819","nonce":3627842210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6823","nonce":3627842210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6821","nonce":3627842210}]},"public_addr":"172.21.15.131:6817/3627842210","cluster_addr":"172.21.15.131:6819/3627842210","heartbeat_back_addr":"172.21.15.131:6823/3627842210","heartbeat_front_addr":"172.21.15.131:6821/3627842210","state":["exists","up"]},{"osd":3,"uuid":"cf117a27-d6b2-43d3-ae52-b47988bf8eed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6809","nonce":350790591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6811","nonce":350790591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6815","nonce":350790591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6813","nonce":350790591}]},"public_addr":"172.21.15.131:6809/350790591","cluster_addr":"172.21.15.131:6811/350790591","heartbeat_back_addr":"172.21.15.131:6815/350790591","heartbeat_front_addr":"172.21.15.131:6813/350790591","state":["exists","up"]},{"osd":4,"uuid":"a092ccfc-be89-4a84-88d5-0f781fa715eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6816","nonce":244560995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6819","nonce":244560995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6823","nonce":244560995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6821","nonce":244560995}]},"public_addr":"172.21.15.187:6816/244560995","cluster_addr":"172.21.15.187:6819/244560995","heartbeat_back_addr":"172.21.15.187:6823/244560995","heartbeat_front_addr":"172.21.15.187:6821/244560995","state":["exists","up"]},{"osd":5,"uuid":"7513ba69-dd00-4a35-b190-9e8cc3b2bdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6801","nonce":324619398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6803","nonce":324619398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6812","nonce":324619398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6807","nonce":324619398}]},"public_addr":"172.21.15.187:6801/324619398","cluster_addr":"172.21.15.187:6803/324619398","heartbeat_back_addr":"172.21.15.187:6812/324619398","heartbeat_front_addr":"172.21.15.187:6807/324619398","state":["exists","up"]},{"osd":6,"uuid":"50081810-c093-4d2f-a4d7-b7c695c5400d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6806","nonce":1391347786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6809","nonce":1391347786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6818","nonce":1391347786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6813","nonce":1391347786}]},"public_addr":"172.21.15.187:6806/1391347786","cluster_addr":"172.21.15.187:6809/1391347786","heartbeat_back_addr":"172.21.15.187:6818/1391347786","heartbeat_front_addr":"172.21.15.187:6813/1391347786","state":["exists","up"]},{"osd":7,"uuid":"775dae05-54f5-4ad4-bbce-92863c6ed2ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6825","nonce":1220065622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6827","nonce":1220065622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6831","nonce":1220065622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6829","nonce":1220065622}]},"public_addr":"172.21.15.187:6825/1220065622","cluster_addr":"172.21.15.187:6827/1220065622","heartbeat_back_addr":"172.21.15.187:6831/1220065622","heartbeat_front_addr":"172.21.15.187:6829/1220065622","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:05.983520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.299516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.067323+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.307748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.018742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.258775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.010912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.295355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-26T22:56:03.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.395+0000 7ff34ea00640 1 -- 172.21.15.131:0/842816410 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ff32406be30 msgr2=0x7ff32406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:03.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.395+0000 7ff34ea00640 1 --2- 172.21.15.131:0/842816410 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ff32406be30 0x7ff32406e2f0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7ff348148a50 tx=0x7ff330008040 comp rx=0 tx=0).stop 2024-05-26T22:56:03.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.395+0000 7ff34ea00640 1 -- 172.21.15.131:0/842816410 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff348071d80 msgr2=0x7ff348147860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:03.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.395+0000 7ff34ea00640 1 --2- 172.21.15.131:0/842816410 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff348071d80 0x7ff348147860 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7ff33802a180 tx=0x7ff3380046c0 comp rx=0 tx=0).stop 2024-05-26T22:56:03.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.395+0000 7ff34ea00640 1 -- 172.21.15.131:0/842816410 shutdown_connections 2024-05-26T22:56:03.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.395+0000 7ff34ea00640 1 --2- 172.21.15.131:0/842816410 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff3480ad450 0x7ff348152570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:03.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.395+0000 7ff34ea00640 1 --2- 172.21.15.131:0/842816410 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ff32406be30 0x7ff32406e2f0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:03.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.395+0000 7ff34ea00640 1 --2- 172.21.15.131:0/842816410 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff3481480b0 0x7ff34815a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:03.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.395+0000 7ff34ea00640 1 --2- 172.21.15.131:0/842816410 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff348071d80 0x7ff348147860 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:03.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.395+0000 7ff34ea00640 1 -- 172.21.15.131:0/842816410 >> 172.21.15.131:0/842816410 conn(0x7ff3480b1e90 msgr2=0x7ff3480b13d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:56:03.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.395+0000 7ff34ea00640 1 -- 172.21.15.131:0/842816410 shutdown_connections 2024-05-26T22:56:03.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.395+0000 7ff34ea00640 1 -- 172.21.15.131:0/842816410 wait complete. 2024-05-26T22:56:03.415 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-26T22:56:03.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.627+0000 7fa8eee00640 1 Processor -- start 2024-05-26T22:56:03.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.627+0000 7fa8eee00640 1 -- start start 2024-05-26T22:56:03.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.628+0000 7fa8eee00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fa8e806d250 0x7fa8e806d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:03.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.629+0000 7fa8eee00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fa8e815b730 0x7fa8e815bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:03.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.629+0000 7fa8eee00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8e806db70 con 0x7fa8e8071f90 2024-05-26T22:56:03.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.629+0000 7fa8eee00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fa8e8071260 con 0x7fa8e815b730 2024-05-26T22:56:03.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.629+0000 7fa8eee00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fa8e813e820 con 0x7fa8e806d250 2024-05-26T22:56:03.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.629+0000 7fa8ed000640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fa8e815b730 0x7fa8e815bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:03.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.629+0000 7fa8ee400640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fa8e8071f90 0x7fa8e80723c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:60480/0 (socket says 172.21.15.131:60480) 2024-05-26T22:56:03.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.629+0000 7fa8ee400640 1 -- 172.21.15.131:0/1046014035 learned_addr learned my addr 172.21.15.131:0/1046014035 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:56:03.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.629+0000 7fa8eda00640 1 --2- 172.21.15.131:0/1046014035 >> v2:172.21.15.187:3300/0 conn(0x7fa8e806d250 0x7fa8e806d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:03.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.629+0000 7fa8ed000640 1 -- 172.21.15.131:0/1046014035 >> v2:172.21.15.187:3300/0 conn(0x7fa8e806d250 msgr2=0x7fa8e806d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:03.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.629+0000 7fa8ed000640 1 --2- 172.21.15.131:0/1046014035 >> v2:172.21.15.187:3300/0 conn(0x7fa8e806d250 0x7fa8e806d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:03.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.629+0000 7fa8ed000640 1 -- 172.21.15.131:0/1046014035 >> v1:172.21.15.131:6789/0 conn(0x7fa8e8071f90 legacy=0x7fa8e80723c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:03.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.630+0000 7fa8ed000640 1 -- 172.21.15.131:0/1046014035 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8e815c4a0 con 0x7fa8e815b730 2024-05-26T22:56:03.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.629+0000 7fa8e7e00640 1 -- 172.21.15.131:0/1046014035 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3985481251 0 0) 0x7fa8e806db70 con 0x7fa8e8071f90 2024-05-26T22:56:03.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.631+0000 7fa8ed000640 1 --2- 172.21.15.131:0/1046014035 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8e815b730 0x7fa8e815bbc0 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7fa8e806f3c0 tx=0x7fa8d402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5926b70194d80cfa server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:03.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.631+0000 7fa8e7e00640 1 -- 172.21.15.131:0/1046014035 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d4037070 con 0x7fa8e815b730 2024-05-26T22:56:03.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.631+0000 7fa8e7e00640 1 -- 172.21.15.131:0/1046014035 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8d4032440 con 0x7fa8e815b730 2024-05-26T22:56:03.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.631+0000 7fa8e7e00640 1 -- 172.21.15.131:0/1046014035 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d4032810 con 0x7fa8e815b730 2024-05-26T22:56:03.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.631+0000 7fa8eee00640 1 -- 172.21.15.131:0/1046014035 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8e815b730 msgr2=0x7fa8e815bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:03.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.631+0000 7fa8eee00640 1 --2- 172.21.15.131:0/1046014035 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8e815b730 0x7fa8e815bbc0 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7fa8e806f3c0 tx=0x7fa8d402a1a0 comp rx=0 tx=0).stop 2024-05-26T22:56:03.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.631+0000 7fa8eee00640 1 -- 172.21.15.131:0/1046014035 shutdown_connections 2024-05-26T22:56:03.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.632+0000 7fa8eee00640 1 --2- 172.21.15.131:0/1046014035 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8e815b730 0x7fa8e815bbc0 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:03.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.632+0000 7fa8eee00640 1 --2- 172.21.15.131:0/1046014035 >> v2:172.21.15.187:3300/0 conn(0x7fa8e806d250 0x7fa8e806d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:03.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.632+0000 7fa8eee00640 1 -- 172.21.15.131:0/1046014035 >> 172.21.15.131:0/1046014035 conn(0x7fa8e80b5660 msgr2=0x7fa8e80b5a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:56:03.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.632+0000 7fa8eee00640 1 -- 172.21.15.131:0/1046014035 shutdown_connections 2024-05-26T22:56:03.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.632+0000 7fa8eee00640 1 -- 172.21.15.131:0/1046014035 wait complete. 2024-05-26T22:56:03.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.633+0000 7fa8eee00640 1 Processor -- start 2024-05-26T22:56:03.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.633+0000 7fa8eee00640 1 -- start start 2024-05-26T22:56:03.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.634+0000 7fa8eee00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8e806d250 0x7fa8e813b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:03.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.634+0000 7fa8eee00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa8e8071f90 0x7fa8e80b3260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:03.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.634+0000 7fa8eee00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa8e815b730 0x7fa8e80b4a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:03.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.634+0000 7fa8eee00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa8e8071a70 con 0x7fa8e806d250 2024-05-26T22:56:03.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.634+0000 7fa8eee00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa8e8071790 con 0x7fa8e8071f90 2024-05-26T22:56:03.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.634+0000 7fa8eee00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa8e815c100 con 0x7fa8e815b730 2024-05-26T22:56:03.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.635+0000 7fa8eda00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8e806d250 0x7fa8e813b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:03.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.635+0000 7fa8eda00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8e806d250 0x7fa8e813b920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:36852/0 (socket says 172.21.15.131:36852) 2024-05-26T22:56:03.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.635+0000 7fa8eda00640 1 -- 172.21.15.131:0/865585879 learned_addr learned my addr 172.21.15.131:0/865585879 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:56:03.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.635+0000 7fa8ed000640 1 --2- 172.21.15.131:0/865585879 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa8e8071f90 0x7fa8e80b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:03.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.635+0000 7fa8ee400640 1 --2- 172.21.15.131:0/865585879 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa8e815b730 0x7fa8e80b4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:03.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.635+0000 7fa8eda00640 1 -- 172.21.15.131:0/865585879 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa8e815b730 msgr2=0x7fa8e80b4a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:03.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.635+0000 7fa8eda00640 1 --2- 172.21.15.131:0/865585879 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa8e815b730 0x7fa8e80b4a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:03.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.635+0000 7fa8eda00640 1 -- 172.21.15.131:0/865585879 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa8e8071f90 msgr2=0x7fa8e80b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:03.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.635+0000 7fa8eda00640 1 --2- 172.21.15.131:0/865585879 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa8e8071f90 0x7fa8e80b3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:03.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.635+0000 7fa8eda00640 1 -- 172.21.15.131:0/865585879 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8e80b39d0 con 0x7fa8e806d250 2024-05-26T22:56:03.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.635+0000 7fa8eda00640 1 --2- 172.21.15.131:0/865585879 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8e806d250 0x7fa8e813b920 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7fa8dc00d800 tx=0x7fa8dc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:03.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.636+0000 7fa8e5600640 1 -- 172.21.15.131:0/865585879 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8dc00b6c0 con 0x7fa8e806d250 2024-05-26T22:56:03.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.636+0000 7fa8e5600640 1 -- 172.21.15.131:0/865585879 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8dc019070 con 0x7fa8e806d250 2024-05-26T22:56:03.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.636+0000 7fa8e5600640 1 -- 172.21.15.131:0/865585879 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8dc00ba80 con 0x7fa8e806d250 2024-05-26T22:56:03.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.636+0000 7fa8eee00640 1 -- 172.21.15.131:0/865585879 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8e80b3cb0 con 0x7fa8e806d250 2024-05-26T22:56:03.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.636+0000 7fa8eee00640 1 -- 172.21.15.131:0/865585879 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8e80b4270 con 0x7fa8e806d250 2024-05-26T22:56:03.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.637+0000 7fa8eee00640 1 -- 172.21.15.131:0/865585879 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8e80694e0 con 0x7fa8e806d250 2024-05-26T22:56:03.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.638+0000 7fa8e5600640 1 -- 172.21.15.131:0/865585879 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa8dc00bc20 con 0x7fa8e806d250 2024-05-26T22:56:03.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.639+0000 7fa8e5600640 1 --2- 172.21.15.131:0/865585879 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa8c806be30 0x7fa8c806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:03.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.639+0000 7fa8e5600640 1 -- 172.21.15.131:0/865585879 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5709+0+0 (secure 0 0 0) 0x7fa8dc08b3d0 con 0x7fa8e806d250 2024-05-26T22:56:03.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.639+0000 7fa8ed000640 1 --2- 172.21.15.131:0/865585879 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa8c806be30 0x7fa8c806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:03.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.640+0000 7fa8ed000640 1 --2- 172.21.15.131:0/865585879 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa8c806be30 0x7fa8c806e2f0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fa8d4034000 tx=0x7fa8d402e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:03.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.644+0000 7fa8e5600640 1 -- 172.21.15.131:0/865585879 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fa8dc059ce0 con 0x7fa8e806d250 2024-05-26T22:56:03.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:03.878+0000 7fa8eee00640 1 -- 172.21.15.131:0/865585879 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fa8e80cc470 con 0x7fa8e806d250 2024-05-26T22:56:04.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:04.552+0000 7fa8e5600640 1 -- 172.21.15.131:0/865585879 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v109) ==== 118+0+0 (secure 0 0 0) 0x7fa8dc0595e0 con 0x7fa8e806d250 2024-05-26T22:56:04.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:04.557+0000 7fa8eee00640 1 -- 172.21.15.131:0/865585879 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa8c806be30 msgr2=0x7fa8c806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:04.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:04.557+0000 7fa8eee00640 1 --2- 172.21.15.131:0/865585879 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa8c806be30 0x7fa8c806e2f0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fa8d4034000 tx=0x7fa8d402e000 comp rx=0 tx=0).stop 2024-05-26T22:56:04.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:04.557+0000 7fa8eee00640 1 -- 172.21.15.131:0/865585879 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8e806d250 msgr2=0x7fa8e813b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:04.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:04.557+0000 7fa8eee00640 1 --2- 172.21.15.131:0/865585879 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8e806d250 0x7fa8e813b920 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7fa8dc00d800 tx=0x7fa8dc00dcc0 comp rx=0 tx=0).stop 2024-05-26T22:56:04.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:04.557+0000 7fa8eee00640 1 -- 172.21.15.131:0/865585879 shutdown_connections 2024-05-26T22:56:04.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:04.557+0000 7fa8eee00640 1 --2- 172.21.15.131:0/865585879 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa8e815b730 0x7fa8e80b4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:04.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:04.557+0000 7fa8eee00640 1 --2- 172.21.15.131:0/865585879 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa8c806be30 0x7fa8c806e2f0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:04.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:04.557+0000 7fa8eee00640 1 --2- 172.21.15.131:0/865585879 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa8e8071f90 0x7fa8e80b3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:04.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:04.557+0000 7fa8eee00640 1 --2- 172.21.15.131:0/865585879 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8e806d250 0x7fa8e813b920 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:04.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:04.557+0000 7fa8eee00640 1 -- 172.21.15.131:0/865585879 >> 172.21.15.131:0/865585879 conn(0x7fa8e80b5660 msgr2=0x7fa8e8072c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:56:04.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:04.558+0000 7fa8eee00640 1 -- 172.21.15.131:0/865585879 shutdown_connections 2024-05-26T22:56:04.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:04.558+0000 7fa8eee00640 1 -- 172.21.15.131:0/865585879 wait complete. 2024-05-26T22:56:04.577 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-26T22:56:04.577 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-26T22:56:04.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:04.781+0000 7f1213c00640 1 Processor -- start 2024-05-26T22:56:04.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:04.782+0000 7f1213c00640 1 -- start start 2024-05-26T22:56:04.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:04.783+0000 7f1213c00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f120c0b5450 0x7f120c0b43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:04.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:04.783+0000 7f1213c00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f120c06f420 0x7f120c0cfd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:04.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:04.783+0000 7f1213c00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f120c0cf590 0x7f120c0d0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:04.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:04.783+0000 7f1213c00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f120c160d90 con 0x7f120c0cf590 2024-05-26T22:56:04.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:04.783+0000 7f1213c00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f120c0b4900 con 0x7f120c0b5450 2024-05-26T22:56:04.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:04.783+0000 7f1213c00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f120c160180 con 0x7f120c06f420 2024-05-26T22:56:04.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:04.784+0000 7f1211c00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f120c0cf590 0x7f120c0d0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:04.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:04.784+0000 7f1211c00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f120c0cf590 0x7f120c0d0290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54394/0 (socket says 172.21.15.131:54394) 2024-05-26T22:56:04.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:04.784+0000 7f1211c00640 1 -- 172.21.15.131:0/1370583701 learned_addr learned my addr 172.21.15.131:0/1370583701 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:56:04.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:04.784+0000 7f120be00640 1 --2- 172.21.15.131:0/1370583701 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f120c06f420 0x7f120c0cfd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:04.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:04.784+0000 7f1211200640 1 --2- 172.21.15.131:0/1370583701 >> v2:172.21.15.187:3300/0 conn(0x7f120c0b5450 0x7f120c0b43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:04.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:04.784+0000 7f1211c00640 1 -- 172.21.15.131:0/1370583701 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f120c06f420 msgr2=0x7f120c0cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:04.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:04.784+0000 7f1211c00640 1 --2- 172.21.15.131:0/1370583701 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f120c06f420 0x7f120c0cfd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:04.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:04.784+0000 7f1211c00640 1 -- 172.21.15.131:0/1370583701 >> v2:172.21.15.187:3300/0 conn(0x7f120c0b5450 msgr2=0x7f120c0b43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:04.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:04.784+0000 7f1211c00640 1 --2- 172.21.15.131:0/1370583701 >> v2:172.21.15.187:3300/0 conn(0x7f120c0b5450 0x7f120c0b43c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:04.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:04.784+0000 7f1211c00640 1 -- 172.21.15.131:0/1370583701 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f120c160460 con 0x7f120c0cf590 2024-05-26T22:56:04.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:04.786+0000 7f1211c00640 1 --2- 172.21.15.131:0/1370583701 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f120c0cf590 0x7f120c0d0290 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f120c0d0c40 tx=0x7f120402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6744e26819c65c3c server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:04.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:04.786+0000 7f120b400640 1 -- 172.21.15.131:0/1370583701 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1204036070 con 0x7f120c0cf590 2024-05-26T22:56:04.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:04.786+0000 7f120b400640 1 -- 172.21.15.131:0/1370583701 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1204031600 con 0x7f120c0cf590 2024-05-26T22:56:04.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:04.786+0000 7f120b400640 1 -- 172.21.15.131:0/1370583701 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12040319d0 con 0x7f120c0cf590 2024-05-26T22:56:04.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:04.786+0000 7f1213c00640 1 -- 172.21.15.131:0/1370583701 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f120c0cf590 msgr2=0x7f120c0d0290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:04.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:04.786+0000 7f1213c00640 1 --2- 172.21.15.131:0/1370583701 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f120c0cf590 0x7f120c0d0290 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f120c0d0c40 tx=0x7f120402a6a0 comp rx=0 tx=0).stop 2024-05-26T22:56:04.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:04.787+0000 7f1213c00640 1 -- 172.21.15.131:0/1370583701 shutdown_connections 2024-05-26T22:56:04.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:04.787+0000 7f1213c00640 1 --2- 172.21.15.131:0/1370583701 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f120c0cf590 0x7f120c0d0290 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:04.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:04.787+0000 7f1213c00640 1 --2- 172.21.15.131:0/1370583701 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f120c06f420 0x7f120c0cfd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:04.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:04.787+0000 7f1213c00640 1 --2- 172.21.15.131:0/1370583701 >> v2:172.21.15.187:3300/0 conn(0x7f120c0b5450 0x7f120c0b43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:04.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:04.787+0000 7f1213c00640 1 -- 172.21.15.131:0/1370583701 >> 172.21.15.131:0/1370583701 conn(0x7f120c06bc90 msgr2=0x7f120c0b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:56:04.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:04.787+0000 7f1213c00640 1 -- 172.21.15.131:0/1370583701 shutdown_connections 2024-05-26T22:56:04.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:04.787+0000 7f1213c00640 1 -- 172.21.15.131:0/1370583701 wait complete. 2024-05-26T22:56:04.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:04.788+0000 7f1213c00640 1 Processor -- start 2024-05-26T22:56:04.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:04.789+0000 7f1213c00640 1 -- start start 2024-05-26T22:56:04.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:04.790+0000 7f1213c00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f120c06f420 0x7f120c0ce130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:04.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:04.790+0000 7f1213c00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f120c0b5450 0x7f120c0b09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:04.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:04.790+0000 7f1213c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f120c0cf590 0x7f120c0acef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:04.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:04.790+0000 7f1213c00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f120c0b5830 con 0x7f120c0cf590 2024-05-26T22:56:04.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:04.790+0000 7f1213c00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f120c160a20 con 0x7f120c06f420 2024-05-26T22:56:04.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:04.790+0000 7f1213c00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f120c15fde0 con 0x7f120c0b5450 2024-05-26T22:56:04.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:04.790+0000 7f1211c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f120c0cf590 0x7f120c0acef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:04.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:04.790+0000 7f1211c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f120c0cf590 0x7f120c0acef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54406/0 (socket says 172.21.15.131:54406) 2024-05-26T22:56:04.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:04.790+0000 7f1211c00640 1 -- 172.21.15.131:0/1503515224 learned_addr learned my addr 172.21.15.131:0/1503515224 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:56:04.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:04.790+0000 7f120be00640 1 --2- 172.21.15.131:0/1503515224 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f120c0b5450 0x7f120c0b09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:04.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:04.790+0000 7f1211200640 1 --2- 172.21.15.131:0/1503515224 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f120c06f420 0x7f120c0ce130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:04.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:04.790+0000 7f1211c00640 1 -- 172.21.15.131:0/1503515224 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f120c0b5450 msgr2=0x7f120c0b09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:04.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:04.790+0000 7f1211c00640 1 --2- 172.21.15.131:0/1503515224 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f120c0b5450 0x7f120c0b09a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:04.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:04.790+0000 7f1211c00640 1 -- 172.21.15.131:0/1503515224 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f120c06f420 msgr2=0x7f120c0ce130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:04.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:04.790+0000 7f1211c00640 1 --2- 172.21.15.131:0/1503515224 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f120c06f420 0x7f120c0ce130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:04.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:04.790+0000 7f1211c00640 1 -- 172.21.15.131:0/1503515224 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f120c0ad740 con 0x7f120c0cf590 2024-05-26T22:56:04.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:04.791+0000 7f1211c00640 1 --2- 172.21.15.131:0/1503515224 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f120c0cf590 0x7f120c0acef0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f120402ac90 tx=0x7f1204004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:04.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:04.791+0000 7f1208c00640 1 -- 172.21.15.131:0/1503515224 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1204041020 con 0x7f120c0cf590 2024-05-26T22:56:04.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:04.791+0000 7f1208c00640 1 -- 172.21.15.131:0/1503515224 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1204007780 con 0x7f120c0cf590 2024-05-26T22:56:04.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:04.791+0000 7f1208c00640 1 -- 172.21.15.131:0/1503515224 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1204036040 con 0x7f120c0cf590 2024-05-26T22:56:04.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:04.791+0000 7f1213c00640 1 -- 172.21.15.131:0/1503515224 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f120c0ada20 con 0x7f120c0cf590 2024-05-26T22:56:04.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:04.791+0000 7f1213c00640 1 -- 172.21.15.131:0/1503515224 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f120c0b01a0 con 0x7f120c0cf590 2024-05-26T22:56:04.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:04.792+0000 7f1213c00640 1 -- 172.21.15.131:0/1503515224 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f120c074e90 con 0x7f120c0cf590 2024-05-26T22:56:04.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:04.793+0000 7f1208c00640 1 -- 172.21.15.131:0/1503515224 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1204044020 con 0x7f120c0cf590 2024-05-26T22:56:04.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:04.794+0000 7f1208c00640 1 --2- 172.21.15.131:0/1503515224 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f11ec06be30 0x7f11ec06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:04.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:04.794+0000 7f1208c00640 1 -- 172.21.15.131:0/1503515224 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5709+0+0 (secure 0 0 0) 0x7f12040abbf0 con 0x7f120c0cf590 2024-05-26T22:56:04.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:04.794+0000 7f1211200640 1 --2- 172.21.15.131:0/1503515224 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f11ec06be30 0x7f11ec06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:04.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:04.795+0000 7f1211200640 1 --2- 172.21.15.131:0/1503515224 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f11ec06be30 0x7f11ec06e2f0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f11fc006fd0 tx=0x7f11fc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:04.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:04.799+0000 7f1208c00640 1 -- 172.21.15.131:0/1503515224 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f120404b030 con 0x7f120c0cf590 2024-05-26T22:56:05.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:05.021+0000 7f1213c00640 1 -- 172.21.15.131:0/1503515224 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f120c0695c0 con 0x7f120c0cf590 2024-05-26T22:56:05.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:05.556+0000 7f1208c00640 1 -- 172.21.15.131:0/1503515224 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v110) ==== 122+0+0 (secure 0 0 0) 0x7f120407a500 con 0x7f120c0cf590 2024-05-26T22:56:05.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:05.559+0000 7f1213c00640 1 -- 172.21.15.131:0/1503515224 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f11ec06be30 msgr2=0x7f11ec06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:05.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:05.559+0000 7f1213c00640 1 --2- 172.21.15.131:0/1503515224 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f11ec06be30 0x7f11ec06e2f0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f11fc006fd0 tx=0x7f11fc008040 comp rx=0 tx=0).stop 2024-05-26T22:56:05.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:05.559+0000 7f1213c00640 1 -- 172.21.15.131:0/1503515224 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f120c0cf590 msgr2=0x7f120c0acef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:05.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:05.559+0000 7f1213c00640 1 --2- 172.21.15.131:0/1503515224 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f120c0cf590 0x7f120c0acef0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f120402ac90 tx=0x7f1204004a40 comp rx=0 tx=0).stop 2024-05-26T22:56:05.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:05.560+0000 7f1213c00640 1 -- 172.21.15.131:0/1503515224 shutdown_connections 2024-05-26T22:56:05.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:05.560+0000 7f1213c00640 1 --2- 172.21.15.131:0/1503515224 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f120c0b5450 0x7f120c0b09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:05.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:05.560+0000 7f1213c00640 1 --2- 172.21.15.131:0/1503515224 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f11ec06be30 0x7f11ec06e2f0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:05.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:05.560+0000 7f1213c00640 1 --2- 172.21.15.131:0/1503515224 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f120c0cf590 0x7f120c0acef0 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:05.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:05.560+0000 7f1213c00640 1 --2- 172.21.15.131:0/1503515224 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f120c06f420 0x7f120c0ce130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:05.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:05.560+0000 7f1213c00640 1 -- 172.21.15.131:0/1503515224 >> 172.21.15.131:0/1503515224 conn(0x7f120c06bc90 msgr2=0x7f120c15db40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:56:05.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:05.560+0000 7f1213c00640 1 -- 172.21.15.131:0/1503515224 shutdown_connections 2024-05-26T22:56:05.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:05.560+0000 7f1213c00640 1 -- 172.21.15.131:0/1503515224 wait complete. 2024-05-26T22:56:05.580 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-26T22:56:05.581 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-26T22:56:05.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:05.772+0000 7fd690c00640 1 Processor -- start 2024-05-26T22:56:05.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:05.772+0000 7fd690c00640 1 -- start start 2024-05-26T22:56:05.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:05.773+0000 7fd690c00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fd68c156750 0x7fd68c15fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:05.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:05.773+0000 7fd690c00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fd68c06e8c0 0x7fd68c15d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:05.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:05.773+0000 7fd690c00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd68c0cc7f0 0x7fd68c0ccca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:05.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:05.773+0000 7fd690c00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fd68c15cbd0 con 0x7fd68c06e8c0 2024-05-26T22:56:05.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:05.773+0000 7fd690c00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fd68c160620 con 0x7fd68c156750 2024-05-26T22:56:05.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:05.773+0000 7fd690c00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd68c160900 con 0x7fd68c0cc7f0 2024-05-26T22:56:05.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:05.774+0000 7fd683e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fd68c06e8c0 0x7fd68c15d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:05.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:05.774+0000 7fd683e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fd68c06e8c0 0x7fd68c15d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54416/0 (socket says 172.21.15.131:54416) 2024-05-26T22:56:05.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:05.774+0000 7fd683e00640 1 -- 172.21.15.131:0/3009641758 learned_addr learned my addr 172.21.15.131:0/3009641758 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:56:05.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:05.774+0000 7fd68b400640 1 --2- 172.21.15.131:0/3009641758 >> v2:172.21.15.187:3300/0 conn(0x7fd68c156750 0x7fd68c15fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:05.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:05.774+0000 7fd68be00640 1 --2- 172.21.15.131:0/3009641758 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd68c0cc7f0 0x7fd68c0ccca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:05.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:05.774+0000 7fd683e00640 1 -- 172.21.15.131:0/3009641758 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd68c0cc7f0 msgr2=0x7fd68c0ccca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:05.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:05.774+0000 7fd683e00640 1 --2- 172.21.15.131:0/3009641758 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd68c0cc7f0 0x7fd68c0ccca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:05.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:05.774+0000 7fd683e00640 1 -- 172.21.15.131:0/3009641758 >> v2:172.21.15.187:3300/0 conn(0x7fd68c156750 msgr2=0x7fd68c15fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:05.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:05.774+0000 7fd683e00640 1 --2- 172.21.15.131:0/3009641758 >> v2:172.21.15.187:3300/0 conn(0x7fd68c156750 0x7fd68c15fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:05.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:05.774+0000 7fd683e00640 1 -- 172.21.15.131:0/3009641758 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd68c160be0 con 0x7fd68c06e8c0 2024-05-26T22:56:05.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:05.775+0000 7fd683e00640 1 --2- 172.21.15.131:0/3009641758 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd68c06e8c0 0x7fd68c15d060 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7fd68c0cdb10 tx=0x7fd67402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b8d6ac4d4851074 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:05.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:05.776+0000 7fd68aa00640 1 -- 172.21.15.131:0/3009641758 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd674037070 con 0x7fd68c06e8c0 2024-05-26T22:56:05.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:05.776+0000 7fd68aa00640 1 -- 172.21.15.131:0/3009641758 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd674032440 con 0x7fd68c06e8c0 2024-05-26T22:56:05.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:05.776+0000 7fd68aa00640 1 -- 172.21.15.131:0/3009641758 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd674032810 con 0x7fd68c06e8c0 2024-05-26T22:56:05.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:05.776+0000 7fd690c00640 1 -- 172.21.15.131:0/3009641758 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd68c06e8c0 msgr2=0x7fd68c15d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:05.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:05.776+0000 7fd690c00640 1 --2- 172.21.15.131:0/3009641758 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd68c06e8c0 0x7fd68c15d060 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7fd68c0cdb10 tx=0x7fd67402a1e0 comp rx=0 tx=0).stop 2024-05-26T22:56:05.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:05.777+0000 7fd690c00640 1 -- 172.21.15.131:0/3009641758 shutdown_connections 2024-05-26T22:56:05.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:05.777+0000 7fd690c00640 1 --2- 172.21.15.131:0/3009641758 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd68c0cc7f0 0x7fd68c0ccca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:05.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:05.777+0000 7fd690c00640 1 --2- 172.21.15.131:0/3009641758 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd68c06e8c0 0x7fd68c15d060 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:05.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:05.777+0000 7fd690c00640 1 --2- 172.21.15.131:0/3009641758 >> v2:172.21.15.187:3300/0 conn(0x7fd68c156750 0x7fd68c15fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:05.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:05.777+0000 7fd690c00640 1 -- 172.21.15.131:0/3009641758 >> 172.21.15.131:0/3009641758 conn(0x7fd68c0cff40 msgr2=0x7fd68c0d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:56:05.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:05.777+0000 7fd690c00640 1 -- 172.21.15.131:0/3009641758 shutdown_connections 2024-05-26T22:56:05.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:05.777+0000 7fd690c00640 1 -- 172.21.15.131:0/3009641758 wait complete. 2024-05-26T22:56:05.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:05.778+0000 7fd690c00640 1 Processor -- start 2024-05-26T22:56:05.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:05.778+0000 7fd690c00640 1 -- start start 2024-05-26T22:56:05.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:05.779+0000 7fd690c00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd68c06e8c0 0x7fd68c143450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:05.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:05.779+0000 7fd690c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd68c0cc7f0 0x7fd68c1f84c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:05.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:05.779+0000 7fd690c00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd68c156750 0x7fd68c0713a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:05.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:05.779+0000 7fd690c00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fd68c160280 con 0x7fd68c0cc7f0 2024-05-26T22:56:05.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:05.779+0000 7fd690c00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd68c156b30 con 0x7fd68c06e8c0 2024-05-26T22:56:05.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:05.779+0000 7fd690c00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd68c160400 con 0x7fd68c156750 2024-05-26T22:56:05.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:05.780+0000 7fd683e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd68c0cc7f0 0x7fd68c1f84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:05.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:05.780+0000 7fd683e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd68c0cc7f0 0x7fd68c1f84c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54424/0 (socket says 172.21.15.131:54424) 2024-05-26T22:56:05.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:05.780+0000 7fd683e00640 1 -- 172.21.15.131:0/2529633433 learned_addr learned my addr 172.21.15.131:0/2529633433 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:56:05.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:05.780+0000 7fd683e00640 1 -- 172.21.15.131:0/2529633433 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd68c156750 msgr2=0x7fd68c0713a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:05.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:05.780+0000 7fd683e00640 1 --2- 172.21.15.131:0/2529633433 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd68c156750 0x7fd68c0713a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:05.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:05.780+0000 7fd683e00640 1 -- 172.21.15.131:0/2529633433 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd68c06e8c0 msgr2=0x7fd68c143450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:05.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:05.780+0000 7fd683e00640 1 --2- 172.21.15.131:0/2529633433 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd68c06e8c0 0x7fd68c143450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:05.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:05.780+0000 7fd683e00640 1 -- 172.21.15.131:0/2529633433 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd68c159730 con 0x7fd68c0cc7f0 2024-05-26T22:56:05.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:05.780+0000 7fd683e00640 1 --2- 172.21.15.131:0/2529633433 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd68c0cc7f0 0x7fd68c1f84c0 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7fd674000c00 tx=0x7fd6740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:05.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:05.781+0000 7fd683400640 1 -- 172.21.15.131:0/2529633433 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd674041020 con 0x7fd68c0cc7f0 2024-05-26T22:56:05.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:05.781+0000 7fd683400640 1 -- 172.21.15.131:0/2529633433 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6740306c0 con 0x7fd68c0cc7f0 2024-05-26T22:56:05.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:05.781+0000 7fd683400640 1 -- 172.21.15.131:0/2529633433 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd674037040 con 0x7fd68c0cc7f0 2024-05-26T22:56:05.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:05.781+0000 7fd690c00640 1 -- 172.21.15.131:0/2529633433 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd68c159a10 con 0x7fd68c0cc7f0 2024-05-26T22:56:05.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:05.781+0000 7fd690c00640 1 -- 172.21.15.131:0/2529633433 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd68c1592c0 con 0x7fd68c0cc7f0 2024-05-26T22:56:05.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:05.782+0000 7fd690c00640 1 -- 172.21.15.131:0/2529633433 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd68c1f7e00 con 0x7fd68c0cc7f0 2024-05-26T22:56:05.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:05.783+0000 7fd683400640 1 -- 172.21.15.131:0/2529633433 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd674044020 con 0x7fd68c0cc7f0 2024-05-26T22:56:05.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:05.784+0000 7fd683400640 1 --2- 172.21.15.131:0/2529633433 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fd66806be30 0x7fd66806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:05.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:05.784+0000 7fd683400640 1 -- 172.21.15.131:0/2529633433 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5709+0+0 (secure 0 0 0) 0x7fd6740ab6b0 con 0x7fd68c0cc7f0 2024-05-26T22:56:05.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:05.784+0000 7fd68b400640 1 --2- 172.21.15.131:0/2529633433 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fd66806be30 0x7fd66806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:05.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:05.785+0000 7fd68b400640 1 --2- 172.21.15.131:0/2529633433 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fd66806be30 0x7fd66806e2f0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fd68c156100 tx=0x7fd67c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:05.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:05.789+0000 7fd683400640 1 -- 172.21.15.131:0/2529633433 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fd68c1f7e00 con 0x7fd68c0cc7f0 2024-05-26T22:56:06.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.028+0000 7fd690c00640 1 -- 172.21.15.131:0/2529633433 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd68c0b6130 con 0x7fd68c0cc7f0 2024-05-26T22:56:06.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.030+0000 7fd683400640 1 -- 172.21.15.131:0/2529633433 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 110 v110) ==== 95+0+5201 (secure 0 0 0) 0x7fd68c0b6130 con 0x7fd68c0cc7f0 2024-05-26T22:56:06.030 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:56:06.030 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":110,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5959,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/2949813640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":2949813640},{"type":"v1","addr":"172.21.15.187:6833","nonce":2949813640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104},{"gid":5968,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/2594604065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2594604065},{"type":"v1","addr":"172.21.15.187:6835","nonce":2594604065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":105},{"gid":8307,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6836/2710427205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":2710427205},{"type":"v1","addr":"172.21.15.131:6836","nonce":2710427205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":103}],"filesystems":[{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-26T22:56:01.980221+0000","modified":"2024-05-26T22:56:05.022938+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8310},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8310":{"gid":8310,"name":"c","rank":0,"incarnation":106,"state":"up:active","state_seq":5,"addr":"172.21.15.131:6837/2725520331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6835","nonce":2725520331},{"type":"v1","addr":"172.21.15.131:6837","nonce":2725520331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8310,"qdb_cluster":[8310]},"id":8}]} 2024-05-26T22:56:06.030 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 110 2024-05-26T22:56:06.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.033+0000 7fd690c00640 1 -- 172.21.15.131:0/2529633433 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fd66806be30 msgr2=0x7fd66806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:06.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.033+0000 7fd690c00640 1 --2- 172.21.15.131:0/2529633433 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fd66806be30 0x7fd66806e2f0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fd68c156100 tx=0x7fd67c008040 comp rx=0 tx=0).stop 2024-05-26T22:56:06.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.033+0000 7fd690c00640 1 -- 172.21.15.131:0/2529633433 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd68c0cc7f0 msgr2=0x7fd68c1f84c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:06.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.033+0000 7fd690c00640 1 --2- 172.21.15.131:0/2529633433 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd68c0cc7f0 0x7fd68c1f84c0 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7fd674000c00 tx=0x7fd6740046c0 comp rx=0 tx=0).stop 2024-05-26T22:56:06.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.033+0000 7fd690c00640 1 -- 172.21.15.131:0/2529633433 shutdown_connections 2024-05-26T22:56:06.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.033+0000 7fd690c00640 1 --2- 172.21.15.131:0/2529633433 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd68c156750 0x7fd68c0713a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:06.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.033+0000 7fd690c00640 1 --2- 172.21.15.131:0/2529633433 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fd66806be30 0x7fd66806e2f0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:06.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.033+0000 7fd690c00640 1 --2- 172.21.15.131:0/2529633433 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd68c0cc7f0 0x7fd68c1f84c0 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:06.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.033+0000 7fd690c00640 1 --2- 172.21.15.131:0/2529633433 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd68c06e8c0 0x7fd68c143450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:06.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.033+0000 7fd690c00640 1 -- 172.21.15.131:0/2529633433 >> 172.21.15.131:0/2529633433 conn(0x7fd68c0cff40 msgr2=0x7fd68c15f500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:56:06.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.033+0000 7fd690c00640 1 -- 172.21.15.131:0/2529633433 shutdown_connections 2024-05-26T22:56:06.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.033+0000 7fd690c00640 1 -- 172.21.15.131:0/2529633433 wait complete. 2024-05-26T22:56:06.053 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-26T22:56:06.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.250+0000 7fafcda00640 1 Processor -- start 2024-05-26T22:56:06.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.250+0000 7fafcda00640 1 -- start start 2024-05-26T22:56:06.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.252+0000 7fafcda00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fafc815e6f0 0x7fafc80cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:06.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.252+0000 7fafcda00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fafc815ead0 0x7fafc815ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:06.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.252+0000 7fafcda00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fafc80cee40 0x7fafc80cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:06.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.252+0000 7fafcda00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fafc80cd830 con 0x7fafc815e6f0 2024-05-26T22:56:06.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.252+0000 7fafcda00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fafc80b4660 con 0x7fafc815ead0 2024-05-26T22:56:06.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.252+0000 7fafcda00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fafc80b4940 con 0x7fafc80cee40 2024-05-26T22:56:06.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.252+0000 7fafc6a00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fafc815e6f0 0x7fafc80cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:06.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.252+0000 7fafc6a00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fafc815e6f0 0x7fafc80cd2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54430/0 (socket says 172.21.15.131:54430) 2024-05-26T22:56:06.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.252+0000 7fafc6a00640 1 -- 172.21.15.131:0/4083978745 learned_addr learned my addr 172.21.15.131:0/4083978745 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:56:06.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.253+0000 7fafc6000640 1 --2- 172.21.15.131:0/4083978745 >> v2:172.21.15.187:3300/0 conn(0x7fafc815ead0 0x7fafc815ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:06.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.253+0000 7fafc7400640 1 --2- 172.21.15.131:0/4083978745 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fafc80cee40 0x7fafc80cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:06.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.253+0000 7fafc6a00640 1 -- 172.21.15.131:0/4083978745 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fafc80cee40 msgr2=0x7fafc80cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:06.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.253+0000 7fafc6a00640 1 --2- 172.21.15.131:0/4083978745 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fafc80cee40 0x7fafc80cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:06.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.253+0000 7fafc6a00640 1 -- 172.21.15.131:0/4083978745 >> v2:172.21.15.187:3300/0 conn(0x7fafc815ead0 msgr2=0x7fafc815ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:06.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.253+0000 7fafc6a00640 1 --2- 172.21.15.131:0/4083978745 >> v2:172.21.15.187:3300/0 conn(0x7fafc815ead0 0x7fafc815ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:06.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.253+0000 7fafc6a00640 1 -- 172.21.15.131:0/4083978745 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafc80b4c20 con 0x7fafc815e6f0 2024-05-26T22:56:06.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.253+0000 7fafc7400640 1 --2- 172.21.15.131:0/4083978745 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fafc80cee40 0x7fafc80cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:56:06.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.254+0000 7fafc6a00640 1 --2- 172.21.15.131:0/4083978745 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fafc815e6f0 0x7fafc80cd2f0 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7fafc806cda0 tx=0x7fafb802c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89b7d6a8e200ac4 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:06.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.255+0000 7fafc5600640 1 -- 172.21.15.131:0/4083978745 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb8035400 con 0x7fafc815e6f0 2024-05-26T22:56:06.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.255+0000 7fafc5600640 1 -- 172.21.15.131:0/4083978745 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafb80355a0 con 0x7fafc815e6f0 2024-05-26T22:56:06.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.255+0000 7fafc5600640 1 -- 172.21.15.131:0/4083978745 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb8035970 con 0x7fafc815e6f0 2024-05-26T22:56:06.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.255+0000 7fafcda00640 1 -- 172.21.15.131:0/4083978745 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fafc815e6f0 msgr2=0x7fafc80cd2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:06.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.255+0000 7fafcda00640 1 --2- 172.21.15.131:0/4083978745 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fafc815e6f0 0x7fafc80cd2f0 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7fafc806cda0 tx=0x7fafb802c8a0 comp rx=0 tx=0).stop 2024-05-26T22:56:06.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.256+0000 7fafcda00640 1 -- 172.21.15.131:0/4083978745 shutdown_connections 2024-05-26T22:56:06.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.256+0000 7fafcda00640 1 --2- 172.21.15.131:0/4083978745 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fafc80cee40 0x7fafc80cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:06.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.256+0000 7fafcda00640 1 --2- 172.21.15.131:0/4083978745 >> v2:172.21.15.187:3300/0 conn(0x7fafc815ead0 0x7fafc815ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:06.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.256+0000 7fafcda00640 1 --2- 172.21.15.131:0/4083978745 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fafc815e6f0 0x7fafc80cd2f0 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:06.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.256+0000 7fafcda00640 1 -- 172.21.15.131:0/4083978745 >> 172.21.15.131:0/4083978745 conn(0x7fafc8072ad0 msgr2=0x7fafc806b310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:56:06.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.256+0000 7fafcda00640 1 -- 172.21.15.131:0/4083978745 shutdown_connections 2024-05-26T22:56:06.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.256+0000 7fafcda00640 1 -- 172.21.15.131:0/4083978745 wait complete. 2024-05-26T22:56:06.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.258+0000 7fafcda00640 1 Processor -- start 2024-05-26T22:56:06.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.258+0000 7fafcda00640 1 -- start start 2024-05-26T22:56:06.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.259+0000 7fafcda00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fafc80cee40 0x7fafc80aa4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:06.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.259+0000 7fafcda00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fafc815e6f0 0x7fafc80cc090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:06.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.259+0000 7fafcda00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fafc815ead0 0x7fafc8143370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:06.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.259+0000 7fafcda00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fafc80b4350 con 0x7fafc815ead0 2024-05-26T22:56:06.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.259+0000 7fafcda00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fafc815f4a0 con 0x7fafc80cee40 2024-05-26T22:56:06.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.259+0000 7fafcda00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fafc81446c0 con 0x7fafc815e6f0 2024-05-26T22:56:06.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.259+0000 7fafc7400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fafc815ead0 0x7fafc8143370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:06.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.259+0000 7fafc7400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fafc815ead0 0x7fafc8143370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54444/0 (socket says 172.21.15.131:54444) 2024-05-26T22:56:06.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.259+0000 7fafc7400640 1 -- 172.21.15.131:0/781055459 learned_addr learned my addr 172.21.15.131:0/781055459 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:56:06.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.260+0000 7fafc6a00640 1 --2- 172.21.15.131:0/781055459 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fafc80cee40 0x7fafc80aa4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:06.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.260+0000 7fafc6000640 1 --2- 172.21.15.131:0/781055459 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fafc815e6f0 0x7fafc80cc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:06.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.260+0000 7fafc7400640 1 -- 172.21.15.131:0/781055459 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fafc815e6f0 msgr2=0x7fafc80cc090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:06.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.260+0000 7fafc7400640 1 --2- 172.21.15.131:0/781055459 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fafc815e6f0 0x7fafc80cc090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:06.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.260+0000 7fafc7400640 1 -- 172.21.15.131:0/781055459 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fafc80cee40 msgr2=0x7fafc80aa4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:06.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.260+0000 7fafc7400640 1 --2- 172.21.15.131:0/781055459 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fafc80cee40 0x7fafc80aa4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:06.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.260+0000 7fafc7400640 1 -- 172.21.15.131:0/781055459 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafc80cc5d0 con 0x7fafc815ead0 2024-05-26T22:56:06.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.260+0000 7fafc7400640 1 --2- 172.21.15.131:0/781055459 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fafc815ead0 0x7fafc8143370 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7fafc000c7f0 tx=0x7fafc000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:06.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.260+0000 7fafbea00640 1 -- 172.21.15.131:0/781055459 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc0015070 con 0x7fafc815ead0 2024-05-26T22:56:06.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.260+0000 7fafbea00640 1 -- 172.21.15.131:0/781055459 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafc0007820 con 0x7fafc815ead0 2024-05-26T22:56:06.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.260+0000 7fafbea00640 1 -- 172.21.15.131:0/781055459 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc0007c10 con 0x7fafc815ead0 2024-05-26T22:56:06.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.260+0000 7fafcda00640 1 -- 172.21.15.131:0/781055459 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafc80cc870 con 0x7fafc815ead0 2024-05-26T22:56:06.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.261+0000 7fafcda00640 1 -- 172.21.15.131:0/781055459 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafc80cce30 con 0x7fafc815ead0 2024-05-26T22:56:06.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.262+0000 7fafcda00640 1 -- 172.21.15.131:0/781055459 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafc80694e0 con 0x7fafc815ead0 2024-05-26T22:56:06.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.263+0000 7fafbea00640 1 -- 172.21.15.131:0/781055459 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fafc0018070 con 0x7fafc815ead0 2024-05-26T22:56:06.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.264+0000 7fafbea00640 1 --2- 172.21.15.131:0/781055459 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fafa806be30 0x7fafa806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:06.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.264+0000 7fafbea00640 1 -- 172.21.15.131:0/781055459 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5709+0+0 (secure 0 0 0) 0x7fafc008afa0 con 0x7fafc815ead0 2024-05-26T22:56:06.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.264+0000 7fafc6a00640 1 --2- 172.21.15.131:0/781055459 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fafa806be30 0x7fafa806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:06.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.265+0000 7fafc6a00640 1 --2- 172.21.15.131:0/781055459 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fafa806be30 0x7fafa806e2f0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fafb8038000 tx=0x7fafb8031000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:06.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.270+0000 7fafbea00640 1 -- 172.21.15.131:0/781055459 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fafc0059a00 con 0x7fafc815ead0 2024-05-26T22:56:06.442 INFO:tasks.ceph.mds.c.smithi131.stderr:2024-05-26T22:56:06.442+0000 7f3ec4400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-26T22:56:06.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.470+0000 7fafcda00640 1 -- 172.21.15.131:0/781055459 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fafc806cae0 con 0x7fafc815ead0 2024-05-26T22:56:06.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.471+0000 7fafbea00640 1 -- 172.21.15.131:0/781055459 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v107) ==== 74+0+16651 (secure 0 0 0) 0x7fafc0082e20 con 0x7fafc815ead0 2024-05-26T22:56:06.472 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:56:06.472 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":107,"fsid":"46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f","created":"2024-05-26T22:47:57.288905+0000","modified":"2024-05-26T22:56:03.175526+0000","last_up_change":"2024-05-26T22:48:09.602207+0000","last_in_change":"2024-05-26T22:48:00.687341+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-26T22:48:10.463339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-05-26T22:55:59.775225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-05-26T22:56:00.988905+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"814eaf62-a18d-494b-8e27-fd8fb88adea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6801","nonce":2350940051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6803","nonce":2350940051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6807","nonce":2350940051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6805","nonce":2350940051}]},"public_addr":"172.21.15.131:6801/2350940051","cluster_addr":"172.21.15.131:6803/2350940051","heartbeat_back_addr":"172.21.15.131:6807/2350940051","heartbeat_front_addr":"172.21.15.131:6805/2350940051","state":["exists","up"]},{"osd":1,"uuid":"42e66287-f6c6-4e26-af3c-6dfaa3536d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6825","nonce":1525528247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6827","nonce":1525528247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6831","nonce":1525528247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6829","nonce":1525528247}]},"public_addr":"172.21.15.131:6825/1525528247","cluster_addr":"172.21.15.131:6827/1525528247","heartbeat_back_addr":"172.21.15.131:6831/1525528247","heartbeat_front_addr":"172.21.15.131:6829/1525528247","state":["exists","up"]},{"osd":2,"uuid":"0da34673-04a2-4fd0-b57a-0ff2b679b043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6817","nonce":3627842210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6819","nonce":3627842210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6823","nonce":3627842210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6821","nonce":3627842210}]},"public_addr":"172.21.15.131:6817/3627842210","cluster_addr":"172.21.15.131:6819/3627842210","heartbeat_back_addr":"172.21.15.131:6823/3627842210","heartbeat_front_addr":"172.21.15.131:6821/3627842210","state":["exists","up"]},{"osd":3,"uuid":"cf117a27-d6b2-43d3-ae52-b47988bf8eed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6809","nonce":350790591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6811","nonce":350790591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6815","nonce":350790591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6813","nonce":350790591}]},"public_addr":"172.21.15.131:6809/350790591","cluster_addr":"172.21.15.131:6811/350790591","heartbeat_back_addr":"172.21.15.131:6815/350790591","heartbeat_front_addr":"172.21.15.131:6813/350790591","state":["exists","up"]},{"osd":4,"uuid":"a092ccfc-be89-4a84-88d5-0f781fa715eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6816","nonce":244560995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6819","nonce":244560995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6823","nonce":244560995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6821","nonce":244560995}]},"public_addr":"172.21.15.187:6816/244560995","cluster_addr":"172.21.15.187:6819/244560995","heartbeat_back_addr":"172.21.15.187:6823/244560995","heartbeat_front_addr":"172.21.15.187:6821/244560995","state":["exists","up"]},{"osd":5,"uuid":"7513ba69-dd00-4a35-b190-9e8cc3b2bdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6801","nonce":324619398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6803","nonce":324619398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6812","nonce":324619398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6807","nonce":324619398}]},"public_addr":"172.21.15.187:6801/324619398","cluster_addr":"172.21.15.187:6803/324619398","heartbeat_back_addr":"172.21.15.187:6812/324619398","heartbeat_front_addr":"172.21.15.187:6807/324619398","state":["exists","up"]},{"osd":6,"uuid":"50081810-c093-4d2f-a4d7-b7c695c5400d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6806","nonce":1391347786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6809","nonce":1391347786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6818","nonce":1391347786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6813","nonce":1391347786}]},"public_addr":"172.21.15.187:6806/1391347786","cluster_addr":"172.21.15.187:6809/1391347786","heartbeat_back_addr":"172.21.15.187:6818/1391347786","heartbeat_front_addr":"172.21.15.187:6813/1391347786","state":["exists","up"]},{"osd":7,"uuid":"775dae05-54f5-4ad4-bbce-92863c6ed2ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6825","nonce":1220065622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6827","nonce":1220065622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6831","nonce":1220065622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6829","nonce":1220065622}]},"public_addr":"172.21.15.187:6825/1220065622","cluster_addr":"172.21.15.187:6827/1220065622","heartbeat_back_addr":"172.21.15.187:6831/1220065622","heartbeat_front_addr":"172.21.15.187:6829/1220065622","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:05.983520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.299516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.067323+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.307748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.018742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.258775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.010912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.295355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-26T22:56:06.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.475+0000 7fafcda00640 1 -- 172.21.15.131:0/781055459 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fafa806be30 msgr2=0x7fafa806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:06.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.475+0000 7fafcda00640 1 --2- 172.21.15.131:0/781055459 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fafa806be30 0x7fafa806e2f0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fafb8038000 tx=0x7fafb8031000 comp rx=0 tx=0).stop 2024-05-26T22:56:06.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.475+0000 7fafcda00640 1 -- 172.21.15.131:0/781055459 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fafc815ead0 msgr2=0x7fafc8143370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:06.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.475+0000 7fafcda00640 1 --2- 172.21.15.131:0/781055459 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fafc815ead0 0x7fafc8143370 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7fafc000c7f0 tx=0x7fafc000ccb0 comp rx=0 tx=0).stop 2024-05-26T22:56:06.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.475+0000 7fafcda00640 1 -- 172.21.15.131:0/781055459 shutdown_connections 2024-05-26T22:56:06.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.475+0000 7fafcda00640 1 --2- 172.21.15.131:0/781055459 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fafc815e6f0 0x7fafc80cc090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:06.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.475+0000 7fafcda00640 1 --2- 172.21.15.131:0/781055459 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fafa806be30 0x7fafa806e2f0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:06.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.475+0000 7fafcda00640 1 --2- 172.21.15.131:0/781055459 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fafc815ead0 0x7fafc8143370 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:06.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.475+0000 7fafcda00640 1 --2- 172.21.15.131:0/781055459 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fafc80cee40 0x7fafc80aa4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:06.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.475+0000 7fafcda00640 1 -- 172.21.15.131:0/781055459 >> 172.21.15.131:0/781055459 conn(0x7fafc8072ad0 msgr2=0x7fafc8141760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:56:06.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.475+0000 7fafcda00640 1 -- 172.21.15.131:0/781055459 shutdown_connections 2024-05-26T22:56:06.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.475+0000 7fafcda00640 1 -- 172.21.15.131:0/781055459 wait complete. 2024-05-26T22:56:06.494 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-26T22:56:06.495 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-26T22:56:06.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.701+0000 7ff42e000640 1 Processor -- start 2024-05-26T22:56:06.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.701+0000 7ff42e000640 1 -- start start 2024-05-26T22:56:06.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.701+0000 7ff42e000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff42806ebc0 0x7ff4280cedf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:06.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.701+0000 7ff42e000640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7ff428071b70 0x7ff4280b3d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:06.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.701+0000 7ff42e000640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4280722a0 con 0x7ff428144340 2024-05-26T22:56:06.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.701+0000 7ff42e000640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7ff428072eb0 con 0x7ff428071b70 2024-05-26T22:56:06.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.701+0000 7ff42e000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff428072580 con 0x7ff42806ebc0 2024-05-26T22:56:06.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.702+0000 7ff42cc00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7ff428144340 0x7ff42806e4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:45780/0 (socket says 172.21.15.131:45780) 2024-05-26T22:56:06.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.702+0000 7ff42cc00640 1 -- 172.21.15.131:0/1860388874 learned_addr learned my addr 172.21.15.131:0/1860388874 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:56:06.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.702+0000 7ff427e00640 1 -- 172.21.15.131:0/1860388874 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2168765023 0 0) 0x7ff4280722a0 con 0x7ff428144340 2024-05-26T22:56:06.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.702+0000 7ff427e00640 1 -- 172.21.15.131:0/1860388874 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff40c003520 con 0x7ff428144340 2024-05-26T22:56:06.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.702+0000 7ff41fe00640 1 --2- 172.21.15.131:0/1860388874 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff42806ebc0 0x7ff4280cedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:06.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.702+0000 7ff42d600640 1 --2- 172.21.15.131:0/1860388874 >> v2:172.21.15.187:3300/0 conn(0x7ff428071b70 0x7ff4280b3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:06.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.702+0000 7ff427e00640 1 -- 172.21.15.131:0/1860388874 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3481727556 0 0) 0x7ff40c003520 con 0x7ff428144340 2024-05-26T22:56:06.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.703+0000 7ff427e00640 1 -- 172.21.15.131:0/1860388874 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff42806ebc0 msgr2=0x7ff4280cedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:06.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.703+0000 7ff427e00640 1 --2- 172.21.15.131:0/1860388874 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff42806ebc0 0x7ff4280cedf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:06.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.703+0000 7ff427e00640 1 -- 172.21.15.131:0/1860388874 >> v2:172.21.15.187:3300/0 conn(0x7ff428071b70 msgr2=0x7ff4280b3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:06.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.703+0000 7ff427e00640 1 --2- 172.21.15.131:0/1860388874 >> v2:172.21.15.187:3300/0 conn(0x7ff428071b70 0x7ff4280b3d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:06.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.703+0000 7ff427e00640 1 -- 172.21.15.131:0/1860388874 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff428072860 con 0x7ff428144340 2024-05-26T22:56:06.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.703+0000 7ff427e00640 1 -- 172.21.15.131:0/1860388874 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7ff418001670 con 0x7ff428144340 2024-05-26T22:56:06.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.703+0000 7ff427e00640 1 -- 172.21.15.131:0/1860388874 >> v1:172.21.15.131:6789/0 conn(0x7ff428144340 legacy=0x7ff42806e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:06.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.703+0000 7ff427e00640 1 --2- 172.21.15.131:0/1860388874 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff40c0039b0 0x7ff40c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:06.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.703+0000 7ff427e00640 1 --2- 172.21.15.131:0/1860388874 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff42806ebc0 0x7ff40c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:06.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.703+0000 7ff427e00640 1 --2- 172.21.15.131:0/1860388874 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff40c004d40 0x7ff40c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:06.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.703+0000 7ff427e00640 1 -- 172.21.15.131:0/1860388874 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff428072580 con 0x7ff40c004d40 2024-05-26T22:56:06.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.703+0000 7ff427e00640 1 -- 172.21.15.131:0/1860388874 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff40c00ba70 con 0x7ff40c0039b0 2024-05-26T22:56:06.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.703+0000 7ff427e00640 1 -- 172.21.15.131:0/1860388874 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff40c00bc70 con 0x7ff42806ebc0 2024-05-26T22:56:06.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.703+0000 7ff42d600640 1 --2- 172.21.15.131:0/1860388874 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff40c004d40 0x7ff40c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:06.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.703+0000 7ff41fe00640 1 --2- 172.21.15.131:0/1860388874 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff42806ebc0 0x7ff40c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:06.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.703+0000 7ff42cc00640 1 --2- 172.21.15.131:0/1860388874 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff40c0039b0 0x7ff40c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:06.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.703+0000 7ff42d600640 1 -- 172.21.15.131:0/1860388874 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff42806ebc0 msgr2=0x7ff40c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:06.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.703+0000 7ff42d600640 1 --2- 172.21.15.131:0/1860388874 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff42806ebc0 0x7ff40c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:06.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.703+0000 7ff42d600640 1 -- 172.21.15.131:0/1860388874 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff40c0039b0 msgr2=0x7ff40c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:06.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.703+0000 7ff42d600640 1 --2- 172.21.15.131:0/1860388874 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff40c0039b0 0x7ff40c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:06.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.703+0000 7ff42d600640 1 -- 172.21.15.131:0/1860388874 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff40c003520 con 0x7ff40c004d40 2024-05-26T22:56:06.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.704+0000 7ff42cc00640 1 --2- 172.21.15.131:0/1860388874 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff40c0039b0 0x7ff40c003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:56:06.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.705+0000 7ff42d600640 1 --2- 172.21.15.131:0/1860388874 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff40c004d40 0x7ff40c0051c0 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7ff420007360 tx=0x7ff40c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5fcc456138c1e6f server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:06.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.705+0000 7ff427e00640 1 -- 172.21.15.131:0/1860388874 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff420009be0 con 0x7ff40c004d40 2024-05-26T22:56:06.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.705+0000 7ff427e00640 1 -- 172.21.15.131:0/1860388874 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff42002e620 con 0x7ff40c004d40 2024-05-26T22:56:06.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.705+0000 7ff427e00640 1 -- 172.21.15.131:0/1860388874 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff42002e9f0 con 0x7ff40c004d40 2024-05-26T22:56:06.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.705+0000 7ff42e000640 1 -- 172.21.15.131:0/1860388874 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff40c004d40 msgr2=0x7ff40c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:06.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.705+0000 7ff42e000640 1 --2- 172.21.15.131:0/1860388874 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff40c004d40 0x7ff40c0051c0 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7ff420007360 tx=0x7ff40c005830 comp rx=0 tx=0).stop 2024-05-26T22:56:06.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.705+0000 7ff42d600640 1 -- 172.21.15.131:0/1860388874 reap_dead start 2024-05-26T22:56:06.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.705+0000 7ff42e000640 1 -- 172.21.15.131:0/1860388874 shutdown_connections 2024-05-26T22:56:06.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.705+0000 7ff42e000640 1 -- 172.21.15.131:0/1860388874 >> 172.21.15.131:0/1860388874 conn(0x7ff4280cff40 msgr2=0x7ff4280d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:56:06.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.706+0000 7ff42e000640 1 -- 172.21.15.131:0/1860388874 shutdown_connections 2024-05-26T22:56:06.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.706+0000 7ff42e000640 1 -- 172.21.15.131:0/1860388874 wait complete. 2024-05-26T22:56:06.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.707+0000 7ff42e000640 1 Processor -- start 2024-05-26T22:56:06.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.707+0000 7ff42e000640 1 -- start start 2024-05-26T22:56:06.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.708+0000 7ff42e000640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff40c004d40 0x7ff4280b12f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:06.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.708+0000 7ff42e000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff42815a640 0x7ff4280aa630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:06.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.708+0000 7ff42e000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff428141c60 0x7ff42813d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:06.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.708+0000 7ff42e000640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff4280cedf0 con 0x7ff42815a640 2024-05-26T22:56:06.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.708+0000 7ff42e000640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff428071fc0 con 0x7ff40c004d40 2024-05-26T22:56:06.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.708+0000 7ff42e000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff4280cef70 con 0x7ff428141c60 2024-05-26T22:56:06.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.710+0000 7ff41fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff42815a640 0x7ff4280aa630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:06.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.710+0000 7ff41fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff42815a640 0x7ff4280aa630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54452/0 (socket says 172.21.15.131:54452) 2024-05-26T22:56:06.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.710+0000 7ff41fe00640 1 -- 172.21.15.131:0/3750316426 learned_addr learned my addr 172.21.15.131:0/3750316426 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:56:06.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.710+0000 7ff42cc00640 1 --2- 172.21.15.131:0/3750316426 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff40c004d40 0x7ff4280b12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:06.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.710+0000 7ff42d600640 1 --2- 172.21.15.131:0/3750316426 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff428141c60 0x7ff42813d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:06.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.710+0000 7ff41fe00640 1 -- 172.21.15.131:0/3750316426 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff428141c60 msgr2=0x7ff42813d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:06.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.710+0000 7ff41fe00640 1 --2- 172.21.15.131:0/3750316426 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff428141c60 0x7ff42813d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:06.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.710+0000 7ff41fe00640 1 -- 172.21.15.131:0/3750316426 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff40c004d40 msgr2=0x7ff4280b12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:06.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.710+0000 7ff41fe00640 1 --2- 172.21.15.131:0/3750316426 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff40c004d40 0x7ff4280b12f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:06.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.710+0000 7ff41fe00640 1 -- 172.21.15.131:0/3750316426 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4280b1830 con 0x7ff42815a640 2024-05-26T22:56:06.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.710+0000 7ff42cc00640 1 --2- 172.21.15.131:0/3750316426 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff40c004d40 0x7ff4280b12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:56:06.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.710+0000 7ff42d600640 1 --2- 172.21.15.131:0/3750316426 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff428141c60 0x7ff42813d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:56:06.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.710+0000 7ff41fe00640 1 --2- 172.21.15.131:0/3750316426 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff42815a640 0x7ff4280aa630 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7ff41000f580 tx=0x7ff41000fa40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:06.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.710+0000 7ff425600640 1 -- 172.21.15.131:0/3750316426 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff410019020 con 0x7ff42815a640 2024-05-26T22:56:06.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.710+0000 7ff425600640 1 -- 172.21.15.131:0/3750316426 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff410014870 con 0x7ff42815a640 2024-05-26T22:56:06.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.710+0000 7ff425600640 1 -- 172.21.15.131:0/3750316426 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff410014c60 con 0x7ff42815a640 2024-05-26T22:56:06.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.710+0000 7ff42e000640 1 -- 172.21.15.131:0/3750316426 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4280a9590 con 0x7ff42815a640 2024-05-26T22:56:06.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.710+0000 7ff42e000640 1 -- 172.21.15.131:0/3750316426 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4280a9b50 con 0x7ff42815a640 2024-05-26T22:56:06.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.711+0000 7ff42e000640 1 -- 172.21.15.131:0/3750316426 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4280694e0 con 0x7ff42815a640 2024-05-26T22:56:06.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.712+0000 7ff425600640 1 -- 172.21.15.131:0/3750316426 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff41001c020 con 0x7ff42815a640 2024-05-26T22:56:06.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.713+0000 7ff425600640 1 --2- 172.21.15.131:0/3750316426 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ff40406be30 0x7ff40406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:06.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.714+0000 7ff425600640 1 -- 172.21.15.131:0/3750316426 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5709+0+0 (secure 0 0 0) 0x7ff41008b6e0 con 0x7ff42815a640 2024-05-26T22:56:06.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.714+0000 7ff42cc00640 1 --2- 172.21.15.131:0/3750316426 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ff40406be30 0x7ff40406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:06.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.714+0000 7ff42cc00640 1 --2- 172.21.15.131:0/3750316426 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ff40406be30 0x7ff40406e2f0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7ff4180017e0 tx=0x7ff418009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:06.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.718+0000 7ff425600640 1 -- 172.21.15.131:0/3750316426 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7ff410023020 con 0x7ff42815a640 2024-05-26T22:56:06.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.913+0000 7ff42e000640 1 -- 172.21.15.131:0/3750316426 --> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff4280ce720 con 0x7ff40406be30 2024-05-26T22:56:06.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.933+0000 7ff425600640 1 -- 172.21.15.131:0/3750316426 <== mgr.4104 v2:172.21.15.131:6832/3484832947 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252483 (secure 0 0 0) 0x7ff4280ce720 con 0x7ff40406be30 2024-05-26T22:56:06.938 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:56:06.940 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-05-26T22:56:06.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.939+0000 7ff42e000640 1 -- 172.21.15.131:0/3750316426 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ff40406be30 msgr2=0x7ff40406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:06.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.939+0000 7ff42e000640 1 --2- 172.21.15.131:0/3750316426 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ff40406be30 0x7ff40406e2f0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7ff4180017e0 tx=0x7ff418009040 comp rx=0 tx=0).stop 2024-05-26T22:56:06.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.939+0000 7ff42e000640 1 -- 172.21.15.131:0/3750316426 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff42815a640 msgr2=0x7ff4280aa630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:06.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.939+0000 7ff42e000640 1 --2- 172.21.15.131:0/3750316426 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff42815a640 0x7ff4280aa630 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7ff41000f580 tx=0x7ff41000fa40 comp rx=0 tx=0).stop 2024-05-26T22:56:06.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.939+0000 7ff42e000640 1 -- 172.21.15.131:0/3750316426 shutdown_connections 2024-05-26T22:56:06.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.939+0000 7ff42e000640 1 --2- 172.21.15.131:0/3750316426 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff428141c60 0x7ff42813d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:06.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.939+0000 7ff42e000640 1 --2- 172.21.15.131:0/3750316426 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ff40406be30 0x7ff40406e2f0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:06.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.939+0000 7ff42e000640 1 --2- 172.21.15.131:0/3750316426 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff42815a640 0x7ff4280aa630 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:06.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.939+0000 7ff42e000640 1 --2- 172.21.15.131:0/3750316426 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff40c004d40 0x7ff4280b12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:06.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.939+0000 7ff42e000640 1 -- 172.21.15.131:0/3750316426 >> 172.21.15.131:0/3750316426 conn(0x7ff4280cff40 msgr2=0x7ff42806d3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:56:06.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.939+0000 7ff42e000640 1 -- 172.21.15.131:0/3750316426 shutdown_connections 2024-05-26T22:56:06.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:06.939+0000 7ff42e000640 1 -- 172.21.15.131:0/3750316426 wait complete. 2024-05-26T22:56:06.960 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":338,"stamp":"2024-05-26T22:56:06.539824+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":273552,"kb_used_data":11624,"kb_used_omap":12,"kb_used_meta":261811,"kb_avail":754701168,"statfs":{"total":773094113280,"available":772813996032,"internally_reserved":0,"allocated":11902976,"data_stored":5830308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":268095025},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.004022"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:04.858763+0000","last_change":"2024-05-26T22:56:02.206696+0000","last_active":"2024-05-26T22:56:04.858763+0000","last_peered":"2024-05-26T22:56:04.858763+0000","last_clean":"2024-05-26T22:56:04.858763+0000","last_became_active":"2024-05-26T22:56:02.206382+0000","last_became_peered":"2024-05-26T22:56:02.206382+0000","last_unstale":"2024-05-26T22:56:04.858763+0000","last_undegraded":"2024-05-26T22:56:04.858763+0000","last_fullsized":"2024-05-26T22:56:04.858763+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:37:23.034583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:03.190349+0000","last_change":"2024-05-26T22:56:02.206286+0000","last_active":"2024-05-26T22:56:03.190349+0000","last_peered":"2024-05-26T22:56:03.190349+0000","last_clean":"2024-05-26T22:56:03.190349+0000","last_became_active":"2024-05-26T22:56:02.205988+0000","last_became_peered":"2024-05-26T22:56:02.205988+0000","last_unstale":"2024-05-26T22:56:03.190349+0000","last_undegraded":"2024-05-26T22:56:03.190349+0000","last_fullsized":"2024-05-26T22:56:03.190349+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:01:44.644419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:04.619111+0000","last_change":"2024-05-26T22:56:02.220188+0000","last_active":"2024-05-26T22:56:04.619111+0000","last_peered":"2024-05-26T22:56:04.619111+0000","last_clean":"2024-05-26T22:56:04.619111+0000","last_became_active":"2024-05-26T22:56:02.219879+0000","last_became_peered":"2024-05-26T22:56:02.219879+0000","last_unstale":"2024-05-26T22:56:04.619111+0000","last_undegraded":"2024-05-26T22:56:04.619111+0000","last_fullsized":"2024-05-26T22:56:04.619111+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T23:00:17.718633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:04.857924+0000","last_change":"2024-05-26T22:56:02.212605+0000","last_active":"2024-05-26T22:56:04.857924+0000","last_peered":"2024-05-26T22:56:04.857924+0000","last_clean":"2024-05-26T22:56:04.857924+0000","last_became_active":"2024-05-26T22:56:02.210277+0000","last_became_peered":"2024-05-26T22:56:02.210277+0000","last_unstale":"2024-05-26T22:56:04.857924+0000","last_undegraded":"2024-05-26T22:56:04.857924+0000","last_fullsized":"2024-05-26T22:56:04.857924+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:26:04.745401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:03.180622+0000","last_change":"2024-05-26T22:56:02.211088+0000","last_active":"2024-05-26T22:56:03.180622+0000","last_peered":"2024-05-26T22:56:03.180622+0000","last_clean":"2024-05-26T22:56:03.180622+0000","last_became_active":"2024-05-26T22:56:02.210534+0000","last_became_peered":"2024-05-26T22:56:02.210534+0000","last_unstale":"2024-05-26T22:56:03.180622+0000","last_undegraded":"2024-05-26T22:56:03.180622+0000","last_fullsized":"2024-05-26T22:56:03.180622+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:09:43.360447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:04.713885+0000","last_change":"2024-05-26T22:56:02.210694+0000","last_active":"2024-05-26T22:56:04.713885+0000","last_peered":"2024-05-26T22:56:04.713885+0000","last_clean":"2024-05-26T22:56:04.713885+0000","last_became_active":"2024-05-26T22:56:02.210406+0000","last_became_peered":"2024-05-26T22:56:02.210406+0000","last_unstale":"2024-05-26T22:56:04.713885+0000","last_undegraded":"2024-05-26T22:56:04.713885+0000","last_fullsized":"2024-05-26T22:56:04.713885+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T23:18:00.396595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:03.792540+0000","last_change":"2024-05-26T22:56:02.203361+0000","last_active":"2024-05-26T22:56:03.792540+0000","last_peered":"2024-05-26T22:56:03.792540+0000","last_clean":"2024-05-26T22:56:03.792540+0000","last_became_active":"2024-05-26T22:56:02.200587+0000","last_became_peered":"2024-05-26T22:56:02.200587+0000","last_unstale":"2024-05-26T22:56:03.792540+0000","last_undegraded":"2024-05-26T22:56:03.792540+0000","last_fullsized":"2024-05-26T22:56:03.792540+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T23:18:00.396595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:04.618463+0000","last_change":"2024-05-26T22:56:02.213343+0000","last_active":"2024-05-26T22:56:04.618463+0000","last_peered":"2024-05-26T22:56:04.618463+0000","last_clean":"2024-05-26T22:56:04.618463+0000","last_became_active":"2024-05-26T22:56:02.212641+0000","last_became_peered":"2024-05-26T22:56:02.212641+0000","last_unstale":"2024-05-26T22:56:04.618463+0000","last_undegraded":"2024-05-26T22:56:04.618463+0000","last_fullsized":"2024-05-26T22:56:04.618463+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:26:53.366151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:04.859062+0000","last_change":"2024-05-26T22:56:02.218504+0000","last_active":"2024-05-26T22:56:04.859062+0000","last_peered":"2024-05-26T22:56:04.859062+0000","last_clean":"2024-05-26T22:56:04.859062+0000","last_became_active":"2024-05-26T22:56:02.217386+0000","last_became_peered":"2024-05-26T22:56:02.217386+0000","last_unstale":"2024-05-26T22:56:04.859062+0000","last_undegraded":"2024-05-26T22:56:04.859062+0000","last_fullsized":"2024-05-26T22:56:04.859062+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:26:29.808147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:04.858473+0000","last_change":"2024-05-26T22:56:02.219576+0000","last_active":"2024-05-26T22:56:04.858473+0000","last_peered":"2024-05-26T22:56:04.858473+0000","last_clean":"2024-05-26T22:56:04.858473+0000","last_became_active":"2024-05-26T22:56:02.219240+0000","last_became_peered":"2024-05-26T22:56:02.219240+0000","last_unstale":"2024-05-26T22:56:04.858473+0000","last_undegraded":"2024-05-26T22:56:04.858473+0000","last_fullsized":"2024-05-26T22:56:04.858473+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:51:15.280396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:03.188069+0000","last_change":"2024-05-26T22:56:02.207870+0000","last_active":"2024-05-26T22:56:03.188069+0000","last_peered":"2024-05-26T22:56:03.188069+0000","last_clean":"2024-05-26T22:56:03.188069+0000","last_became_active":"2024-05-26T22:56:02.207450+0000","last_became_peered":"2024-05-26T22:56:02.207450+0000","last_unstale":"2024-05-26T22:56:03.188069+0000","last_undegraded":"2024-05-26T22:56:03.188069+0000","last_fullsized":"2024-05-26T22:56:03.188069+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:42:08.736487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:04.858110+0000","last_change":"2024-05-26T22:56:02.217924+0000","last_active":"2024-05-26T22:56:04.858110+0000","last_peered":"2024-05-26T22:56:04.858110+0000","last_clean":"2024-05-26T22:56:04.858110+0000","last_became_active":"2024-05-26T22:56:02.217534+0000","last_became_peered":"2024-05-26T22:56:02.217534+0000","last_unstale":"2024-05-26T22:56:04.858110+0000","last_undegraded":"2024-05-26T22:56:04.858110+0000","last_fullsized":"2024-05-26T22:56:04.858110+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:48:21.809824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:04.619185+0000","last_change":"2024-05-26T22:56:02.221256+0000","last_active":"2024-05-26T22:56:04.619185+0000","last_peered":"2024-05-26T22:56:04.619185+0000","last_clean":"2024-05-26T22:56:04.619185+0000","last_became_active":"2024-05-26T22:56:02.220940+0000","last_became_peered":"2024-05-26T22:56:02.220940+0000","last_unstale":"2024-05-26T22:56:04.619185+0000","last_undegraded":"2024-05-26T22:56:04.619185+0000","last_fullsized":"2024-05-26T22:56:04.619185+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:31:16.025886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:04.618524+0000","last_change":"2024-05-26T22:56:02.216297+0000","last_active":"2024-05-26T22:56:04.618524+0000","last_peered":"2024-05-26T22:56:04.618524+0000","last_clean":"2024-05-26T22:56:04.618524+0000","last_became_active":"2024-05-26T22:56:02.215965+0000","last_became_peered":"2024-05-26T22:56:02.215965+0000","last_unstale":"2024-05-26T22:56:04.618524+0000","last_undegraded":"2024-05-26T22:56:04.618524+0000","last_fullsized":"2024-05-26T22:56:04.618524+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:42:08.736487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:04.713769+0000","last_change":"2024-05-26T22:56:02.208433+0000","last_active":"2024-05-26T22:56:04.713769+0000","last_peered":"2024-05-26T22:56:04.713769+0000","last_clean":"2024-05-26T22:56:04.713769+0000","last_became_active":"2024-05-26T22:56:02.207879+0000","last_became_peered":"2024-05-26T22:56:02.207879+0000","last_unstale":"2024-05-26T22:56:04.713769+0000","last_undegraded":"2024-05-26T22:56:04.713769+0000","last_fullsized":"2024-05-26T22:56:04.713769+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:10:26.651321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:04.857726+0000","last_change":"2024-05-26T22:56:02.214802+0000","last_active":"2024-05-26T22:56:04.857726+0000","last_peered":"2024-05-26T22:56:04.857726+0000","last_clean":"2024-05-26T22:56:04.857726+0000","last_became_active":"2024-05-26T22:56:02.214480+0000","last_became_peered":"2024-05-26T22:56:02.214480+0000","last_unstale":"2024-05-26T22:56:04.857726+0000","last_undegraded":"2024-05-26T22:56:04.857726+0000","last_fullsized":"2024-05-26T22:56:04.857726+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:29:54.582494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:03.188644+0000","last_change":"2024-05-26T22:56:02.205936+0000","last_active":"2024-05-26T22:56:03.188644+0000","last_peered":"2024-05-26T22:56:03.188644+0000","last_clean":"2024-05-26T22:56:03.188644+0000","last_became_active":"2024-05-26T22:56:02.204220+0000","last_became_peered":"2024-05-26T22:56:02.204220+0000","last_unstale":"2024-05-26T22:56:03.188644+0000","last_undegraded":"2024-05-26T22:56:03.188644+0000","last_fullsized":"2024-05-26T22:56:03.188644+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:11:53.760719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:03.189211+0000","last_change":"2024-05-26T22:56:02.209865+0000","last_active":"2024-05-26T22:56:03.189211+0000","last_peered":"2024-05-26T22:56:03.189211+0000","last_clean":"2024-05-26T22:56:03.189211+0000","last_became_active":"2024-05-26T22:56:02.209586+0000","last_became_peered":"2024-05-26T22:56:02.209586+0000","last_unstale":"2024-05-26T22:56:03.189211+0000","last_undegraded":"2024-05-26T22:56:03.189211+0000","last_fullsized":"2024-05-26T22:56:03.189211+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:42:08.736487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:04.619319+0000","last_change":"2024-05-26T22:56:02.220013+0000","last_active":"2024-05-26T22:56:04.619319+0000","last_peered":"2024-05-26T22:56:04.619319+0000","last_clean":"2024-05-26T22:56:04.619319+0000","last_became_active":"2024-05-26T22:56:02.219629+0000","last_became_peered":"2024-05-26T22:56:02.219629+0000","last_unstale":"2024-05-26T22:56:04.619319+0000","last_undegraded":"2024-05-26T22:56:04.619319+0000","last_fullsized":"2024-05-26T22:56:04.619319+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:39:26.930899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:03.189843+0000","last_change":"2024-05-26T22:56:02.211102+0000","last_active":"2024-05-26T22:56:03.189843+0000","last_peered":"2024-05-26T22:56:03.189843+0000","last_clean":"2024-05-26T22:56:03.189843+0000","last_became_active":"2024-05-26T22:56:02.210226+0000","last_became_peered":"2024-05-26T22:56:02.210226+0000","last_unstale":"2024-05-26T22:56:03.189843+0000","last_undegraded":"2024-05-26T22:56:03.189843+0000","last_fullsized":"2024-05-26T22:56:03.189843+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:39:26.930899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:03.189193+0000","last_change":"2024-05-26T22:56:02.207390+0000","last_active":"2024-05-26T22:56:03.189193+0000","last_peered":"2024-05-26T22:56:03.189193+0000","last_clean":"2024-05-26T22:56:03.189193+0000","last_became_active":"2024-05-26T22:56:02.207090+0000","last_became_peered":"2024-05-26T22:56:02.207090+0000","last_unstale":"2024-05-26T22:56:03.189193+0000","last_undegraded":"2024-05-26T22:56:03.189193+0000","last_fullsized":"2024-05-26T22:56:03.189193+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:31:16.025886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:03.500824+0000","last_change":"2024-05-26T22:56:02.216774+0000","last_active":"2024-05-26T22:56:03.500824+0000","last_peered":"2024-05-26T22:56:03.500824+0000","last_clean":"2024-05-26T22:56:03.500824+0000","last_became_active":"2024-05-26T22:56:02.214545+0000","last_became_peered":"2024-05-26T22:56:02.214545+0000","last_unstale":"2024-05-26T22:56:03.500824+0000","last_undegraded":"2024-05-26T22:56:03.500824+0000","last_fullsized":"2024-05-26T22:56:03.500824+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:01:44.644419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:03.187020+0000","last_change":"2024-05-26T22:56:02.209906+0000","last_active":"2024-05-26T22:56:03.187020+0000","last_peered":"2024-05-26T22:56:03.187020+0000","last_clean":"2024-05-26T22:56:03.187020+0000","last_became_active":"2024-05-26T22:56:02.209633+0000","last_became_peered":"2024-05-26T22:56:02.209633+0000","last_unstale":"2024-05-26T22:56:03.187020+0000","last_undegraded":"2024-05-26T22:56:03.187020+0000","last_fullsized":"2024-05-26T22:56:03.187020+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:18:00.119575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:03.500325+0000","last_change":"2024-05-26T22:56:02.217896+0000","last_active":"2024-05-26T22:56:03.500325+0000","last_peered":"2024-05-26T22:56:03.500325+0000","last_clean":"2024-05-26T22:56:03.500325+0000","last_became_active":"2024-05-26T22:56:02.217589+0000","last_became_peered":"2024-05-26T22:56:02.217589+0000","last_unstale":"2024-05-26T22:56:03.500325+0000","last_undegraded":"2024-05-26T22:56:03.500325+0000","last_fullsized":"2024-05-26T22:56:03.500325+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:51:41.862299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:03.189052+0000","last_change":"2024-05-26T22:56:02.204631+0000","last_active":"2024-05-26T22:56:03.189052+0000","last_peered":"2024-05-26T22:56:03.189052+0000","last_clean":"2024-05-26T22:56:03.189052+0000","last_became_active":"2024-05-26T22:56:02.203750+0000","last_became_peered":"2024-05-26T22:56:02.203750+0000","last_unstale":"2024-05-26T22:56:03.189052+0000","last_undegraded":"2024-05-26T22:56:03.189052+0000","last_fullsized":"2024-05-26T22:56:03.189052+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T23:00:17.718633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:04.858124+0000","last_change":"2024-05-26T22:56:02.216396+0000","last_active":"2024-05-26T22:56:04.858124+0000","last_peered":"2024-05-26T22:56:04.858124+0000","last_clean":"2024-05-26T22:56:04.858124+0000","last_became_active":"2024-05-26T22:56:02.216136+0000","last_became_peered":"2024-05-26T22:56:02.216136+0000","last_unstale":"2024-05-26T22:56:04.858124+0000","last_undegraded":"2024-05-26T22:56:04.858124+0000","last_fullsized":"2024-05-26T22:56:04.858124+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:56:56.613417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:04.714541+0000","last_change":"2024-05-26T22:56:02.205608+0000","last_active":"2024-05-26T22:56:04.714541+0000","last_peered":"2024-05-26T22:56:04.714541+0000","last_clean":"2024-05-26T22:56:04.714541+0000","last_became_active":"2024-05-26T22:56:02.205301+0000","last_became_peered":"2024-05-26T22:56:02.205301+0000","last_unstale":"2024-05-26T22:56:04.714541+0000","last_undegraded":"2024-05-26T22:56:04.714541+0000","last_fullsized":"2024-05-26T22:56:04.714541+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:09:51.987645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:03.501248+0000","last_change":"2024-05-26T22:56:02.218062+0000","last_active":"2024-05-26T22:56:03.501248+0000","last_peered":"2024-05-26T22:56:03.501248+0000","last_clean":"2024-05-26T22:56:03.501248+0000","last_became_active":"2024-05-26T22:56:02.216743+0000","last_became_peered":"2024-05-26T22:56:02.216743+0000","last_unstale":"2024-05-26T22:56:03.501248+0000","last_undegraded":"2024-05-26T22:56:03.501248+0000","last_fullsized":"2024-05-26T22:56:03.501248+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:06:11.312538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:03.188128+0000","last_change":"2024-05-26T22:56:02.216764+0000","last_active":"2024-05-26T22:56:03.188128+0000","last_peered":"2024-05-26T22:56:03.188128+0000","last_clean":"2024-05-26T22:56:03.188128+0000","last_became_active":"2024-05-26T22:56:02.216392+0000","last_became_peered":"2024-05-26T22:56:02.216392+0000","last_unstale":"2024-05-26T22:56:03.188128+0000","last_undegraded":"2024-05-26T22:56:03.188128+0000","last_fullsized":"2024-05-26T22:56:03.188128+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:33:00.893414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:03.792928+0000","last_change":"2024-05-26T22:56:02.213781+0000","last_active":"2024-05-26T22:56:03.792928+0000","last_peered":"2024-05-26T22:56:03.792928+0000","last_clean":"2024-05-26T22:56:03.792928+0000","last_became_active":"2024-05-26T22:56:02.213494+0000","last_became_peered":"2024-05-26T22:56:02.213494+0000","last_unstale":"2024-05-26T22:56:03.792928+0000","last_undegraded":"2024-05-26T22:56:03.792928+0000","last_fullsized":"2024-05-26T22:56:03.792928+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:09:51.987645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:04.858331+0000","last_change":"2024-05-26T22:56:02.215800+0000","last_active":"2024-05-26T22:56:04.858331+0000","last_peered":"2024-05-26T22:56:04.858331+0000","last_clean":"2024-05-26T22:56:04.858331+0000","last_became_active":"2024-05-26T22:56:02.215492+0000","last_became_peered":"2024-05-26T22:56:02.215492+0000","last_unstale":"2024-05-26T22:56:04.858331+0000","last_undegraded":"2024-05-26T22:56:04.858331+0000","last_fullsized":"2024-05-26T22:56:04.858331+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:25:21.205123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:04.713754+0000","last_change":"2024-05-26T22:56:02.204465+0000","last_active":"2024-05-26T22:56:04.713754+0000","last_peered":"2024-05-26T22:56:04.713754+0000","last_clean":"2024-05-26T22:56:04.713754+0000","last_became_active":"2024-05-26T22:56:02.202855+0000","last_became_peered":"2024-05-26T22:56:02.202855+0000","last_unstale":"2024-05-26T22:56:04.713754+0000","last_undegraded":"2024-05-26T22:56:04.713754+0000","last_fullsized":"2024-05-26T22:56:04.713754+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:37:30.606174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:04.714812+0000","last_change":"2024-05-26T22:56:02.201346+0000","last_active":"2024-05-26T22:56:04.714812+0000","last_peered":"2024-05-26T22:56:04.714812+0000","last_clean":"2024-05-26T22:56:04.714812+0000","last_became_active":"2024-05-26T22:56:02.200664+0000","last_became_peered":"2024-05-26T22:56:02.200664+0000","last_unstale":"2024-05-26T22:56:04.714812+0000","last_undegraded":"2024-05-26T22:56:04.714812+0000","last_fullsized":"2024-05-26T22:56:04.714812+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:23:33.849977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"106'4","reported_seq":18,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:03.793067+0000","last_change":"2024-05-26T22:56:01.547833+0000","last_active":"2024-05-26T22:56:03.793067+0000","last_peered":"2024-05-26T22:56:03.793067+0000","last_clean":"2024-05-26T22:56:03.793067+0000","last_became_active":"2024-05-26T22:56:01.546355+0000","last_became_peered":"2024-05-26T22:56:01.546355+0000","last_unstale":"2024-05-26T22:56:03.793067+0000","last_undegraded":"2024-05-26T22:56:03.793067+0000","last_fullsized":"2024-05-26T22:56:03.793067+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_clean_scrub_stamp":"2024-05-26T22:56:00.479878+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:57:27.405643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:03.190291+0000","last_change":"2024-05-26T22:56:02.209334+0000","last_active":"2024-05-26T22:56:03.190291+0000","last_peered":"2024-05-26T22:56:03.190291+0000","last_clean":"2024-05-26T22:56:03.190291+0000","last_became_active":"2024-05-26T22:56:02.208968+0000","last_became_peered":"2024-05-26T22:56:02.208968+0000","last_unstale":"2024-05-26T22:56:03.190291+0000","last_undegraded":"2024-05-26T22:56:03.190291+0000","last_fullsized":"2024-05-26T22:56:03.190291+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:32:11.698764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"106'1","reported_seq":15,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:04.619240+0000","last_change":"2024-05-26T22:56:01.546843+0000","last_active":"2024-05-26T22:56:04.619240+0000","last_peered":"2024-05-26T22:56:04.619240+0000","last_clean":"2024-05-26T22:56:04.619240+0000","last_became_active":"2024-05-26T22:56:01.546473+0000","last_became_peered":"2024-05-26T22:56:01.546473+0000","last_unstale":"2024-05-26T22:56:04.619240+0000","last_undegraded":"2024-05-26T22:56:04.619240+0000","last_fullsized":"2024-05-26T22:56:04.619240+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_clean_scrub_stamp":"2024-05-26T22:56:00.479878+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T23:17:59.378357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:04.714796+0000","last_change":"2024-05-26T22:56:02.205289+0000","last_active":"2024-05-26T22:56:04.714796+0000","last_peered":"2024-05-26T22:56:04.714796+0000","last_clean":"2024-05-26T22:56:04.714796+0000","last_became_active":"2024-05-26T22:56:02.204940+0000","last_became_peered":"2024-05-26T22:56:02.204940+0000","last_unstale":"2024-05-26T22:56:04.714796+0000","last_undegraded":"2024-05-26T22:56:04.714796+0000","last_fullsized":"2024-05-26T22:56:04.714796+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:41:04.672377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"106'2","reported_seq":16,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:03.180709+0000","last_change":"2024-05-26T22:56:01.547301+0000","last_active":"2024-05-26T22:56:03.180709+0000","last_peered":"2024-05-26T22:56:03.180709+0000","last_clean":"2024-05-26T22:56:03.180709+0000","last_became_active":"2024-05-26T22:56:01.545934+0000","last_became_peered":"2024-05-26T22:56:01.545934+0000","last_unstale":"2024-05-26T22:56:03.180709+0000","last_undegraded":"2024-05-26T22:56:03.180709+0000","last_fullsized":"2024-05-26T22:56:03.180709+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_clean_scrub_stamp":"2024-05-26T22:56:00.479878+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:25:06.623425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:04.619509+0000","last_change":"2024-05-26T22:56:02.219500+0000","last_active":"2024-05-26T22:56:04.619509+0000","last_peered":"2024-05-26T22:56:04.619509+0000","last_clean":"2024-05-26T22:56:04.619509+0000","last_became_active":"2024-05-26T22:56:02.219204+0000","last_became_peered":"2024-05-26T22:56:02.219204+0000","last_unstale":"2024-05-26T22:56:04.619509+0000","last_undegraded":"2024-05-26T22:56:04.619509+0000","last_fullsized":"2024-05-26T22:56:04.619509+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:01:44.644419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"106'2","reported_seq":16,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:03.180283+0000","last_change":"2024-05-26T22:56:01.546594+0000","last_active":"2024-05-26T22:56:03.180283+0000","last_peered":"2024-05-26T22:56:03.180283+0000","last_clean":"2024-05-26T22:56:03.180283+0000","last_became_active":"2024-05-26T22:56:01.546275+0000","last_became_peered":"2024-05-26T22:56:01.546275+0000","last_unstale":"2024-05-26T22:56:03.180283+0000","last_undegraded":"2024-05-26T22:56:03.180283+0000","last_fullsized":"2024-05-26T22:56:03.180283+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_clean_scrub_stamp":"2024-05-26T22:56:00.479878+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:15:39.374826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:04.857809+0000","last_change":"2024-05-26T22:56:02.219950+0000","last_active":"2024-05-26T22:56:04.857809+0000","last_peered":"2024-05-26T22:56:04.857809+0000","last_clean":"2024-05-26T22:56:04.857809+0000","last_became_active":"2024-05-26T22:56:02.219719+0000","last_became_peered":"2024-05-26T22:56:02.219719+0000","last_unstale":"2024-05-26T22:56:04.857809+0000","last_undegraded":"2024-05-26T22:56:04.857809+0000","last_fullsized":"2024-05-26T22:56:04.857809+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:26:23.188344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"106'1","reported_seq":15,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:03.792371+0000","last_change":"2024-05-26T22:56:01.546848+0000","last_active":"2024-05-26T22:56:03.792371+0000","last_peered":"2024-05-26T22:56:03.792371+0000","last_clean":"2024-05-26T22:56:03.792371+0000","last_became_active":"2024-05-26T22:56:01.546563+0000","last_became_peered":"2024-05-26T22:56:01.546563+0000","last_unstale":"2024-05-26T22:56:03.792371+0000","last_undegraded":"2024-05-26T22:56:03.792371+0000","last_fullsized":"2024-05-26T22:56:03.792371+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_clean_scrub_stamp":"2024-05-26T22:56:00.479878+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:24:06.499767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:03.188597+0000","last_change":"2024-05-26T22:56:02.211063+0000","last_active":"2024-05-26T22:56:03.188597+0000","last_peered":"2024-05-26T22:56:03.188597+0000","last_clean":"2024-05-26T22:56:03.188597+0000","last_became_active":"2024-05-26T22:56:02.210720+0000","last_became_peered":"2024-05-26T22:56:02.210720+0000","last_unstale":"2024-05-26T22:56:03.188597+0000","last_undegraded":"2024-05-26T22:56:03.188597+0000","last_fullsized":"2024-05-26T22:56:03.188597+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:32:11.698764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"106'1","reported_seq":15,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:03.500939+0000","last_change":"2024-05-26T22:56:01.541444+0000","last_active":"2024-05-26T22:56:03.500939+0000","last_peered":"2024-05-26T22:56:03.500939+0000","last_clean":"2024-05-26T22:56:03.500939+0000","last_became_active":"2024-05-26T22:56:01.541115+0000","last_became_peered":"2024-05-26T22:56:01.541115+0000","last_unstale":"2024-05-26T22:56:03.500939+0000","last_undegraded":"2024-05-26T22:56:03.500939+0000","last_fullsized":"2024-05-26T22:56:03.500939+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_clean_scrub_stamp":"2024-05-26T22:56:00.479878+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:32:11.114165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:04.858507+0000","last_change":"2024-05-26T22:56:02.207933+0000","last_active":"2024-05-26T22:56:04.858507+0000","last_peered":"2024-05-26T22:56:04.858507+0000","last_clean":"2024-05-26T22:56:04.858507+0000","last_became_active":"2024-05-26T22:56:02.207438+0000","last_became_peered":"2024-05-26T22:56:02.207438+0000","last_unstale":"2024-05-26T22:56:04.858507+0000","last_undegraded":"2024-05-26T22:56:04.858507+0000","last_fullsized":"2024-05-26T22:56:04.858507+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:21:01.569286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:03.187301+0000","last_change":"2024-05-26T22:56:01.542399+0000","last_active":"2024-05-26T22:56:03.187301+0000","last_peered":"2024-05-26T22:56:03.187301+0000","last_clean":"2024-05-26T22:56:03.187301+0000","last_became_active":"2024-05-26T22:56:01.541752+0000","last_became_peered":"2024-05-26T22:56:01.541752+0000","last_unstale":"2024-05-26T22:56:03.187301+0000","last_undegraded":"2024-05-26T22:56:03.187301+0000","last_fullsized":"2024-05-26T22:56:03.187301+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_clean_scrub_stamp":"2024-05-26T22:56:00.479878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:37:29.587936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:03.792354+0000","last_change":"2024-05-26T22:56:02.204777+0000","last_active":"2024-05-26T22:56:03.792354+0000","last_peered":"2024-05-26T22:56:03.792354+0000","last_clean":"2024-05-26T22:56:03.792354+0000","last_became_active":"2024-05-26T22:56:02.204419+0000","last_became_peered":"2024-05-26T22:56:02.204419+0000","last_unstale":"2024-05-26T22:56:03.792354+0000","last_undegraded":"2024-05-26T22:56:03.792354+0000","last_fullsized":"2024-05-26T22:56:03.792354+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:10:26.651321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:04.858610+0000","last_change":"2024-05-26T22:56:01.545325+0000","last_active":"2024-05-26T22:56:04.858610+0000","last_peered":"2024-05-26T22:56:04.858610+0000","last_clean":"2024-05-26T22:56:04.858610+0000","last_became_active":"2024-05-26T22:56:01.545047+0000","last_became_peered":"2024-05-26T22:56:01.545047+0000","last_unstale":"2024-05-26T22:56:04.858610+0000","last_undegraded":"2024-05-26T22:56:04.858610+0000","last_fullsized":"2024-05-26T22:56:04.858610+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_clean_scrub_stamp":"2024-05-26T22:56:00.479878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:51:40.844061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:03.793186+0000","last_change":"2024-05-26T22:56:02.206158+0000","last_active":"2024-05-26T22:56:03.793186+0000","last_peered":"2024-05-26T22:56:03.793186+0000","last_clean":"2024-05-26T22:56:03.793186+0000","last_became_active":"2024-05-26T22:56:02.205813+0000","last_became_peered":"2024-05-26T22:56:02.205813+0000","last_unstale":"2024-05-26T22:56:03.793186+0000","last_undegraded":"2024-05-26T22:56:03.793186+0000","last_fullsized":"2024-05-26T22:56:03.793186+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:41:04.672377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:03.180351+0000","last_change":"2024-05-26T22:56:01.540411+0000","last_active":"2024-05-26T22:56:03.180351+0000","last_peered":"2024-05-26T22:56:03.180351+0000","last_clean":"2024-05-26T22:56:03.180351+0000","last_became_active":"2024-05-26T22:56:01.540152+0000","last_became_peered":"2024-05-26T22:56:01.540152+0000","last_unstale":"2024-05-26T22:56:03.180351+0000","last_undegraded":"2024-05-26T22:56:03.180351+0000","last_fullsized":"2024-05-26T22:56:03.180351+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_clean_scrub_stamp":"2024-05-26T22:56:00.479878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:41:04.207626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:03.190488+0000","last_change":"2024-05-26T22:56:02.204825+0000","last_active":"2024-05-26T22:56:03.190488+0000","last_peered":"2024-05-26T22:56:03.190488+0000","last_clean":"2024-05-26T22:56:03.190488+0000","last_became_active":"2024-05-26T22:56:02.204460+0000","last_became_peered":"2024-05-26T22:56:02.204460+0000","last_unstale":"2024-05-26T22:56:03.190488+0000","last_undegraded":"2024-05-26T22:56:03.190488+0000","last_fullsized":"2024-05-26T22:56:03.190488+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:11:53.760719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:03.501579+0000","last_change":"2024-05-26T22:56:01.544461+0000","last_active":"2024-05-26T22:56:03.501579+0000","last_peered":"2024-05-26T22:56:03.501579+0000","last_clean":"2024-05-26T22:56:03.501579+0000","last_became_active":"2024-05-26T22:56:01.544192+0000","last_became_peered":"2024-05-26T22:56:01.544192+0000","last_unstale":"2024-05-26T22:56:03.501579+0000","last_undegraded":"2024-05-26T22:56:03.501579+0000","last_fullsized":"2024-05-26T22:56:03.501579+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_clean_scrub_stamp":"2024-05-26T22:56:00.479878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:41:03.654139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:03.501556+0000","last_change":"2024-05-26T22:56:02.213676+0000","last_active":"2024-05-26T22:56:03.501556+0000","last_peered":"2024-05-26T22:56:03.501556+0000","last_clean":"2024-05-26T22:56:03.501556+0000","last_became_active":"2024-05-26T22:56:02.213348+0000","last_became_peered":"2024-05-26T22:56:02.213348+0000","last_unstale":"2024-05-26T22:56:03.501556+0000","last_undegraded":"2024-05-26T22:56:03.501556+0000","last_fullsized":"2024-05-26T22:56:03.501556+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:34:58.504442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"106'3","reported_seq":17,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:04.858483+0000","last_change":"2024-05-26T22:56:01.524769+0000","last_active":"2024-05-26T22:56:04.858483+0000","last_peered":"2024-05-26T22:56:04.858483+0000","last_clean":"2024-05-26T22:56:04.858483+0000","last_became_active":"2024-05-26T22:56:01.524362+0000","last_became_peered":"2024-05-26T22:56:01.524362+0000","last_unstale":"2024-05-26T22:56:04.858483+0000","last_undegraded":"2024-05-26T22:56:04.858483+0000","last_fullsized":"2024-05-26T22:56:04.858483+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_clean_scrub_stamp":"2024-05-26T22:56:00.479878+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:31:15.007648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:03.792986+0000","last_change":"2024-05-26T22:56:02.212450+0000","last_active":"2024-05-26T22:56:03.792986+0000","last_peered":"2024-05-26T22:56:03.792986+0000","last_clean":"2024-05-26T22:56:03.792986+0000","last_became_active":"2024-05-26T22:56:02.212117+0000","last_became_peered":"2024-05-26T22:56:02.212117+0000","last_unstale":"2024-05-26T22:56:03.792986+0000","last_undegraded":"2024-05-26T22:56:03.792986+0000","last_fullsized":"2024-05-26T22:56:03.792986+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:47:42.868120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"106'6","reported_seq":20,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:04.858746+0000","last_change":"2024-05-26T22:56:01.539032+0000","last_active":"2024-05-26T22:56:04.858746+0000","last_peered":"2024-05-26T22:56:04.858746+0000","last_clean":"2024-05-26T22:56:04.858746+0000","last_became_active":"2024-05-26T22:56:01.538397+0000","last_became_peered":"2024-05-26T22:56:01.538397+0000","last_unstale":"2024-05-26T22:56:04.858746+0000","last_undegraded":"2024-05-26T22:56:04.858746+0000","last_fullsized":"2024-05-26T22:56:04.858746+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_clean_scrub_stamp":"2024-05-26T22:56:00.479878+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:01:43.626181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:03.500782+0000","last_change":"2024-05-26T22:56:02.208793+0000","last_active":"2024-05-26T22:56:03.500782+0000","last_peered":"2024-05-26T22:56:03.500782+0000","last_clean":"2024-05-26T22:56:03.500782+0000","last_became_active":"2024-05-26T22:56:02.208502+0000","last_became_peered":"2024-05-26T22:56:02.208502+0000","last_unstale":"2024-05-26T22:56:03.500782+0000","last_undegraded":"2024-05-26T22:56:03.500782+0000","last_fullsized":"2024-05-26T22:56:03.500782+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:39:26.930899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"106'2","reported_seq":16,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:04.714131+0000","last_change":"2024-05-26T22:56:01.547593+0000","last_active":"2024-05-26T22:56:04.714131+0000","last_peered":"2024-05-26T22:56:04.714131+0000","last_clean":"2024-05-26T22:56:04.714131+0000","last_became_active":"2024-05-26T22:56:01.547233+0000","last_became_peered":"2024-05-26T22:56:01.547233+0000","last_unstale":"2024-05-26T22:56:04.714131+0000","last_undegraded":"2024-05-26T22:56:04.714131+0000","last_fullsized":"2024-05-26T22:56:04.714131+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_clean_scrub_stamp":"2024-05-26T22:56:00.479878+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:24:06.499767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:03.500985+0000","last_change":"2024-05-26T22:56:02.217045+0000","last_active":"2024-05-26T22:56:03.500985+0000","last_peered":"2024-05-26T22:56:03.500985+0000","last_clean":"2024-05-26T22:56:03.500985+0000","last_became_active":"2024-05-26T22:56:02.215543+0000","last_became_peered":"2024-05-26T22:56:02.215543+0000","last_unstale":"2024-05-26T22:56:03.500985+0000","last_undegraded":"2024-05-26T22:56:03.500985+0000","last_fullsized":"2024-05-26T22:56:03.500985+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:50:35.747435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:03.187728+0000","last_change":"2024-05-26T22:56:01.531697+0000","last_active":"2024-05-26T22:56:03.187728+0000","last_peered":"2024-05-26T22:56:03.187728+0000","last_clean":"2024-05-26T22:56:03.187728+0000","last_became_active":"2024-05-26T22:56:01.529624+0000","last_became_peered":"2024-05-26T22:56:01.529624+0000","last_unstale":"2024-05-26T22:56:03.187728+0000","last_undegraded":"2024-05-26T22:56:03.187728+0000","last_fullsized":"2024-05-26T22:56:03.187728+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_clean_scrub_stamp":"2024-05-26T22:56:00.479878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:45:22.206973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:03.500721+0000","last_change":"2024-05-26T22:56:02.217069+0000","last_active":"2024-05-26T22:56:03.500721+0000","last_peered":"2024-05-26T22:56:03.500721+0000","last_clean":"2024-05-26T22:56:03.500721+0000","last_became_active":"2024-05-26T22:56:02.216761+0000","last_became_peered":"2024-05-26T22:56:02.216761+0000","last_unstale":"2024-05-26T22:56:03.500721+0000","last_undegraded":"2024-05-26T22:56:03.500721+0000","last_fullsized":"2024-05-26T22:56:03.500721+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T23:00:17.718633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"106'8","reported_seq":22,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:03.180171+0000","last_change":"2024-05-26T22:56:01.540246+0000","last_active":"2024-05-26T22:56:03.180171+0000","last_peered":"2024-05-26T22:56:03.180171+0000","last_clean":"2024-05-26T22:56:03.180171+0000","last_became_active":"2024-05-26T22:56:01.539929+0000","last_became_peered":"2024-05-26T22:56:01.539929+0000","last_unstale":"2024-05-26T22:56:03.180171+0000","last_undegraded":"2024-05-26T22:56:03.180171+0000","last_fullsized":"2024-05-26T22:56:03.180171+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_clean_scrub_stamp":"2024-05-26T22:56:00.479878+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:33:06.138639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:03.180575+0000","last_change":"2024-05-26T22:56:02.213335+0000","last_active":"2024-05-26T22:56:03.180575+0000","last_peered":"2024-05-26T22:56:03.180575+0000","last_clean":"2024-05-26T22:56:03.180575+0000","last_became_active":"2024-05-26T22:56:02.212699+0000","last_became_peered":"2024-05-26T22:56:02.212699+0000","last_unstale":"2024-05-26T22:56:03.180575+0000","last_undegraded":"2024-05-26T22:56:03.180575+0000","last_fullsized":"2024-05-26T22:56:03.180575+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:19:05.773898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"106'1","reported_seq":15,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:04.859542+0000","last_change":"2024-05-26T22:56:01.544226+0000","last_active":"2024-05-26T22:56:04.859542+0000","last_peered":"2024-05-26T22:56:04.859542+0000","last_clean":"2024-05-26T22:56:04.859542+0000","last_became_active":"2024-05-26T22:56:01.543914+0000","last_became_peered":"2024-05-26T22:56:01.543914+0000","last_unstale":"2024-05-26T22:56:04.859542+0000","last_undegraded":"2024-05-26T22:56:04.859542+0000","last_fullsized":"2024-05-26T22:56:04.859542+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_clean_scrub_stamp":"2024-05-26T22:56:00.479878+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:50:34.729197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:04.714767+0000","last_change":"2024-05-26T22:56:02.204502+0000","last_active":"2024-05-26T22:56:04.714767+0000","last_peered":"2024-05-26T22:56:04.714767+0000","last_clean":"2024-05-26T22:56:04.714767+0000","last_became_active":"2024-05-26T22:56:02.203835+0000","last_became_peered":"2024-05-26T22:56:02.203835+0000","last_unstale":"2024-05-26T22:56:04.714767+0000","last_undegraded":"2024-05-26T22:56:04.714767+0000","last_fullsized":"2024-05-26T22:56:04.714767+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:47:42.868120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"106'2","reported_seq":16,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:03.180618+0000","last_change":"2024-05-26T22:56:01.549401+0000","last_active":"2024-05-26T22:56:03.180618+0000","last_peered":"2024-05-26T22:56:03.180618+0000","last_clean":"2024-05-26T22:56:03.180618+0000","last_became_active":"2024-05-26T22:56:01.548769+0000","last_became_peered":"2024-05-26T22:56:01.548769+0000","last_unstale":"2024-05-26T22:56:03.180618+0000","last_undegraded":"2024-05-26T22:56:03.180618+0000","last_fullsized":"2024-05-26T22:56:03.180618+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_clean_scrub_stamp":"2024-05-26T22:56:00.479878+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:53:56.157795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:04.618757+0000","last_change":"2024-05-26T22:56:02.221755+0000","last_active":"2024-05-26T22:56:04.618757+0000","last_peered":"2024-05-26T22:56:04.618757+0000","last_clean":"2024-05-26T22:56:04.618757+0000","last_became_active":"2024-05-26T22:56:02.221283+0000","last_became_peered":"2024-05-26T22:56:02.221283+0000","last_unstale":"2024-05-26T22:56:04.618757+0000","last_undegraded":"2024-05-26T22:56:04.618757+0000","last_fullsized":"2024-05-26T22:56:04.618757+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:11:53.760719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"106'1","reported_seq":15,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:03.189216+0000","last_change":"2024-05-26T22:56:01.539836+0000","last_active":"2024-05-26T22:56:03.189216+0000","last_peered":"2024-05-26T22:56:03.189216+0000","last_clean":"2024-05-26T22:56:03.189216+0000","last_became_active":"2024-05-26T22:56:01.539241+0000","last_became_peered":"2024-05-26T22:56:01.539241+0000","last_unstale":"2024-05-26T22:56:03.189216+0000","last_undegraded":"2024-05-26T22:56:03.189216+0000","last_fullsized":"2024-05-26T22:56:03.189216+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_clean_scrub_stamp":"2024-05-26T22:56:00.479878+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:32:11.114165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:04.619652+0000","last_change":"2024-05-26T22:56:02.216727+0000","last_active":"2024-05-26T22:56:04.619652+0000","last_peered":"2024-05-26T22:56:04.619652+0000","last_clean":"2024-05-26T22:56:04.619652+0000","last_became_active":"2024-05-26T22:56:02.216384+0000","last_became_peered":"2024-05-26T22:56:02.216384+0000","last_unstale":"2024-05-26T22:56:04.619652+0000","last_undegraded":"2024-05-26T22:56:04.619652+0000","last_fullsized":"2024-05-26T22:56:04.619652+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:50:35.747435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:03.792444+0000","last_change":"2024-05-26T22:56:01.549211+0000","last_active":"2024-05-26T22:56:03.792444+0000","last_peered":"2024-05-26T22:56:03.792444+0000","last_clean":"2024-05-26T22:56:03.792444+0000","last_became_active":"2024-05-26T22:56:01.548445+0000","last_became_peered":"2024-05-26T22:56:01.548445+0000","last_unstale":"2024-05-26T22:56:03.792444+0000","last_undegraded":"2024-05-26T22:56:03.792444+0000","last_fullsized":"2024-05-26T22:56:03.792444+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_clean_scrub_stamp":"2024-05-26T22:56:00.479878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:43:45.548497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:04.618527+0000","last_change":"2024-05-26T22:56:02.220132+0000","last_active":"2024-05-26T22:56:04.618527+0000","last_peered":"2024-05-26T22:56:04.618527+0000","last_clean":"2024-05-26T22:56:04.618527+0000","last_became_active":"2024-05-26T22:56:02.219844+0000","last_became_peered":"2024-05-26T22:56:02.219844+0000","last_unstale":"2024-05-26T22:56:04.618527+0000","last_undegraded":"2024-05-26T22:56:04.618527+0000","last_fullsized":"2024-05-26T22:56:04.618527+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:32:11.698764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"106'2","reported_seq":16,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:04.714236+0000","last_change":"2024-05-26T22:56:01.541924+0000","last_active":"2024-05-26T22:56:04.714236+0000","last_peered":"2024-05-26T22:56:04.714236+0000","last_clean":"2024-05-26T22:56:04.714236+0000","last_became_active":"2024-05-26T22:56:01.541666+0000","last_became_peered":"2024-05-26T22:56:01.541666+0000","last_unstale":"2024-05-26T22:56:04.714236+0000","last_undegraded":"2024-05-26T22:56:04.714236+0000","last_fullsized":"2024-05-26T22:56:04.714236+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_clean_scrub_stamp":"2024-05-26T22:56:00.479878+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:15:22.745045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:04.858419+0000","last_change":"2024-05-26T22:56:02.217119+0000","last_active":"2024-05-26T22:56:04.858419+0000","last_peered":"2024-05-26T22:56:04.858419+0000","last_clean":"2024-05-26T22:56:04.858419+0000","last_became_active":"2024-05-26T22:56:02.215372+0000","last_became_peered":"2024-05-26T22:56:02.215372+0000","last_unstale":"2024-05-26T22:56:04.858419+0000","last_undegraded":"2024-05-26T22:56:04.858419+0000","last_fullsized":"2024-05-26T22:56:04.858419+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:33:06.814045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:03.180154+0000","last_change":"2024-05-26T22:56:01.547356+0000","last_active":"2024-05-26T22:56:03.180154+0000","last_peered":"2024-05-26T22:56:03.180154+0000","last_clean":"2024-05-26T22:56:03.180154+0000","last_became_active":"2024-05-26T22:56:01.546071+0000","last_became_peered":"2024-05-26T22:56:01.546071+0000","last_unstale":"2024-05-26T22:56:03.180154+0000","last_undegraded":"2024-05-26T22:56:03.180154+0000","last_fullsized":"2024-05-26T22:56:03.180154+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_clean_scrub_stamp":"2024-05-26T22:56:00.479878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:45:45.573526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:04.618529+0000","last_change":"2024-05-26T22:56:02.222258+0000","last_active":"2024-05-26T22:56:04.618529+0000","last_peered":"2024-05-26T22:56:04.618529+0000","last_clean":"2024-05-26T22:56:04.618529+0000","last_became_active":"2024-05-26T22:56:02.220689+0000","last_became_peered":"2024-05-26T22:56:02.220689+0000","last_unstale":"2024-05-26T22:56:04.618529+0000","last_undegraded":"2024-05-26T22:56:04.618529+0000","last_fullsized":"2024-05-26T22:56:04.618529+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:33:00.893414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:04.713758+0000","last_change":"2024-05-26T22:56:01.540055+0000","last_active":"2024-05-26T22:56:04.713758+0000","last_peered":"2024-05-26T22:56:04.713758+0000","last_clean":"2024-05-26T22:56:04.713758+0000","last_became_active":"2024-05-26T22:56:01.539794+0000","last_became_peered":"2024-05-26T22:56:01.539794+0000","last_unstale":"2024-05-26T22:56:04.713758+0000","last_undegraded":"2024-05-26T22:56:04.713758+0000","last_fullsized":"2024-05-26T22:56:04.713758+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_clean_scrub_stamp":"2024-05-26T22:56:00.479878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:14:41.448743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:03.500739+0000","last_change":"2024-05-26T22:56:02.214861+0000","last_active":"2024-05-26T22:56:03.500739+0000","last_peered":"2024-05-26T22:56:03.500739+0000","last_clean":"2024-05-26T22:56:03.500739+0000","last_became_active":"2024-05-26T22:56:02.214046+0000","last_became_peered":"2024-05-26T22:56:02.214046+0000","last_unstale":"2024-05-26T22:56:03.500739+0000","last_undegraded":"2024-05-26T22:56:03.500739+0000","last_fullsized":"2024-05-26T22:56:03.500739+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:31:16.025886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:04.858791+0000","last_change":"2024-05-26T22:56:01.528791+0000","last_active":"2024-05-26T22:56:04.858791+0000","last_peered":"2024-05-26T22:56:04.858791+0000","last_clean":"2024-05-26T22:56:04.858791+0000","last_became_active":"2024-05-26T22:56:01.528468+0000","last_became_peered":"2024-05-26T22:56:01.528468+0000","last_unstale":"2024-05-26T22:56:04.858791+0000","last_undegraded":"2024-05-26T22:56:04.858791+0000","last_fullsized":"2024-05-26T22:56:04.858791+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_clean_scrub_stamp":"2024-05-26T22:56:00.479878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:11:52.742481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:03.189612+0000","last_change":"2024-05-26T22:56:02.214278+0000","last_active":"2024-05-26T22:56:03.189612+0000","last_peered":"2024-05-26T22:56:03.189612+0000","last_clean":"2024-05-26T22:56:03.189612+0000","last_became_active":"2024-05-26T22:56:02.213982+0000","last_became_peered":"2024-05-26T22:56:02.213982+0000","last_unstale":"2024-05-26T22:56:03.189612+0000","last_undegraded":"2024-05-26T22:56:03.189612+0000","last_fullsized":"2024-05-26T22:56:03.189612+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:33:00.893414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"106'2","reported_seq":16,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:03.501391+0000","last_change":"2024-05-26T22:56:01.537496+0000","last_active":"2024-05-26T22:56:03.501391+0000","last_peered":"2024-05-26T22:56:03.501391+0000","last_clean":"2024-05-26T22:56:03.501391+0000","last_became_active":"2024-05-26T22:56:01.537171+0000","last_became_peered":"2024-05-26T22:56:01.537171+0000","last_unstale":"2024-05-26T22:56:03.501391+0000","last_undegraded":"2024-05-26T22:56:03.501391+0000","last_fullsized":"2024-05-26T22:56:03.501391+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_clean_scrub_stamp":"2024-05-26T22:56:00.479878+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:23:32.831739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:04.859322+0000","last_change":"2024-05-26T22:56:02.212367+0000","last_active":"2024-05-26T22:56:04.859322+0000","last_peered":"2024-05-26T22:56:04.859322+0000","last_clean":"2024-05-26T22:56:04.859322+0000","last_became_active":"2024-05-26T22:56:02.211946+0000","last_became_peered":"2024-05-26T22:56:02.211946+0000","last_unstale":"2024-05-26T22:56:04.859322+0000","last_undegraded":"2024-05-26T22:56:04.859322+0000","last_fullsized":"2024-05-26T22:56:04.859322+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:04:27.002049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"107'1","reported_seq":15,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:03.189685+0000","last_change":"2024-05-26T22:56:01.545880+0000","last_active":"2024-05-26T22:56:03.189685+0000","last_peered":"2024-05-26T22:56:03.189685+0000","last_clean":"2024-05-26T22:56:03.189685+0000","last_became_active":"2024-05-26T22:56:01.544569+0000","last_became_peered":"2024-05-26T22:56:01.544569+0000","last_unstale":"2024-05-26T22:56:03.189685+0000","last_undegraded":"2024-05-26T22:56:03.189685+0000","last_fullsized":"2024-05-26T22:56:03.189685+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_clean_scrub_stamp":"2024-05-26T22:56:00.479878+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:13:54.327398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:04.858298+0000","last_change":"2024-05-26T22:56:02.214933+0000","last_active":"2024-05-26T22:56:04.858298+0000","last_peered":"2024-05-26T22:56:04.858298+0000","last_clean":"2024-05-26T22:56:04.858298+0000","last_became_active":"2024-05-26T22:56:02.214621+0000","last_became_peered":"2024-05-26T22:56:02.214621+0000","last_unstale":"2024-05-26T22:56:04.858298+0000","last_undegraded":"2024-05-26T22:56:04.858298+0000","last_fullsized":"2024-05-26T22:56:04.858298+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:35:23.017773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:04.858695+0000","last_change":"2024-05-26T22:56:01.532602+0000","last_active":"2024-05-26T22:56:04.858695+0000","last_peered":"2024-05-26T22:56:04.858695+0000","last_clean":"2024-05-26T22:56:04.858695+0000","last_became_active":"2024-05-26T22:56:01.532262+0000","last_became_peered":"2024-05-26T22:56:01.532262+0000","last_unstale":"2024-05-26T22:56:04.858695+0000","last_undegraded":"2024-05-26T22:56:04.858695+0000","last_fullsized":"2024-05-26T22:56:04.858695+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_clean_scrub_stamp":"2024-05-26T22:56:00.479878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:39:25.912660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:03.188880+0000","last_change":"2024-05-26T22:56:02.205119+0000","last_active":"2024-05-26T22:56:03.188880+0000","last_peered":"2024-05-26T22:56:03.188880+0000","last_clean":"2024-05-26T22:56:03.188880+0000","last_became_active":"2024-05-26T22:56:02.204653+0000","last_became_peered":"2024-05-26T22:56:02.204653+0000","last_unstale":"2024-05-26T22:56:03.188880+0000","last_undegraded":"2024-05-26T22:56:03.188880+0000","last_fullsized":"2024-05-26T22:56:03.188880+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:26:53.366151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:03.501670+0000","last_change":"2024-05-26T22:56:01.542219+0000","last_active":"2024-05-26T22:56:03.501670+0000","last_peered":"2024-05-26T22:56:03.501670+0000","last_clean":"2024-05-26T22:56:03.501670+0000","last_became_active":"2024-05-26T22:56:01.540883+0000","last_became_peered":"2024-05-26T22:56:01.540883+0000","last_unstale":"2024-05-26T22:56:03.501670+0000","last_undegraded":"2024-05-26T22:56:03.501670+0000","last_fullsized":"2024-05-26T22:56:03.501670+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_clean_scrub_stamp":"2024-05-26T22:56:00.479878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:45:22.206973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:03.187074+0000","last_change":"2024-05-26T22:56:02.210259+0000","last_active":"2024-05-26T22:56:03.187074+0000","last_peered":"2024-05-26T22:56:03.187074+0000","last_clean":"2024-05-26T22:56:03.187074+0000","last_became_active":"2024-05-26T22:56:02.209981+0000","last_became_peered":"2024-05-26T22:56:02.209981+0000","last_unstale":"2024-05-26T22:56:03.187074+0000","last_undegraded":"2024-05-26T22:56:03.187074+0000","last_fullsized":"2024-05-26T22:56:03.187074+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:26:53.366151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:03.792461+0000","last_change":"2024-05-26T22:56:01.547965+0000","last_active":"2024-05-26T22:56:03.792461+0000","last_peered":"2024-05-26T22:56:03.792461+0000","last_clean":"2024-05-26T22:56:03.792461+0000","last_became_active":"2024-05-26T22:56:01.547653+0000","last_became_peered":"2024-05-26T22:56:01.547653+0000","last_unstale":"2024-05-26T22:56:03.792461+0000","last_undegraded":"2024-05-26T22:56:03.792461+0000","last_fullsized":"2024-05-26T22:56:03.792461+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_clean_scrub_stamp":"2024-05-26T22:56:00.479878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:15:22.745045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:03.188888+0000","last_change":"2024-05-26T22:56:02.207758+0000","last_active":"2024-05-26T22:56:03.188888+0000","last_peered":"2024-05-26T22:56:03.188888+0000","last_clean":"2024-05-26T22:56:03.188888+0000","last_became_active":"2024-05-26T22:56:02.202470+0000","last_became_peered":"2024-05-26T22:56:02.202470+0000","last_unstale":"2024-05-26T22:56:03.188888+0000","last_undegraded":"2024-05-26T22:56:03.188888+0000","last_fullsized":"2024-05-26T22:56:03.188888+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:51:41.862299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:03.187523+0000","last_change":"2024-05-26T22:56:01.544943+0000","last_active":"2024-05-26T22:56:03.187523+0000","last_peered":"2024-05-26T22:56:03.187523+0000","last_clean":"2024-05-26T22:56:03.187523+0000","last_became_active":"2024-05-26T22:56:01.544654+0000","last_became_peered":"2024-05-26T22:56:01.544654+0000","last_unstale":"2024-05-26T22:56:03.187523+0000","last_undegraded":"2024-05-26T22:56:03.187523+0000","last_fullsized":"2024-05-26T22:56:03.187523+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_clean_scrub_stamp":"2024-05-26T22:56:00.479878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:10:25.633083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:04.618817+0000","last_change":"2024-05-26T22:56:02.222135+0000","last_active":"2024-05-26T22:56:04.618817+0000","last_peered":"2024-05-26T22:56:04.618817+0000","last_clean":"2024-05-26T22:56:04.618817+0000","last_became_active":"2024-05-26T22:56:02.221328+0000","last_became_peered":"2024-05-26T22:56:02.221328+0000","last_unstale":"2024-05-26T22:56:04.618817+0000","last_undegraded":"2024-05-26T22:56:04.618817+0000","last_fullsized":"2024-05-26T22:56:04.618817+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:51:41.862299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:03.180164+0000","last_change":"2024-05-26T22:56:01.542729+0000","last_active":"2024-05-26T22:56:03.180164+0000","last_peered":"2024-05-26T22:56:03.180164+0000","last_clean":"2024-05-26T22:56:03.180164+0000","last_became_active":"2024-05-26T22:56:01.542220+0000","last_became_peered":"2024-05-26T22:56:01.542220+0000","last_unstale":"2024-05-26T22:56:03.180164+0000","last_undegraded":"2024-05-26T22:56:03.180164+0000","last_fullsized":"2024-05-26T22:56:03.180164+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_clean_scrub_stamp":"2024-05-26T22:56:00.479878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:27:47.414115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:04.858103+0000","last_change":"2024-05-26T22:56:02.207968+0000","last_active":"2024-05-26T22:56:04.858103+0000","last_peered":"2024-05-26T22:56:04.858103+0000","last_clean":"2024-05-26T22:56:04.858103+0000","last_became_active":"2024-05-26T22:56:02.207662+0000","last_became_peered":"2024-05-26T22:56:02.207662+0000","last_unstale":"2024-05-26T22:56:04.858103+0000","last_undegraded":"2024-05-26T22:56:04.858103+0000","last_fullsized":"2024-05-26T22:56:04.858103+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:49:15.779421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:04.858717+0000","last_change":"2024-05-26T22:56:01.528583+0000","last_active":"2024-05-26T22:56:04.858717+0000","last_peered":"2024-05-26T22:56:04.858717+0000","last_clean":"2024-05-26T22:56:04.858717+0000","last_became_active":"2024-05-26T22:56:01.528313+0000","last_became_peered":"2024-05-26T22:56:01.528313+0000","last_unstale":"2024-05-26T22:56:04.858717+0000","last_undegraded":"2024-05-26T22:56:04.858717+0000","last_fullsized":"2024-05-26T22:56:04.858717+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_clean_scrub_stamp":"2024-05-26T22:56:00.479878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T23:00:16.700395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:03.500260+0000","last_change":"2024-05-26T22:56:02.216541+0000","last_active":"2024-05-26T22:56:03.500260+0000","last_peered":"2024-05-26T22:56:03.500260+0000","last_clean":"2024-05-26T22:56:03.500260+0000","last_became_active":"2024-05-26T22:56:02.216205+0000","last_became_peered":"2024-05-26T22:56:02.216205+0000","last_unstale":"2024-05-26T22:56:03.500260+0000","last_undegraded":"2024-05-26T22:56:03.500260+0000","last_fullsized":"2024-05-26T22:56:03.500260+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:11:53.760719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:03.792626+0000","last_change":"2024-05-26T22:56:01.547906+0000","last_active":"2024-05-26T22:56:03.792626+0000","last_peered":"2024-05-26T22:56:03.792626+0000","last_clean":"2024-05-26T22:56:03.792626+0000","last_became_active":"2024-05-26T22:56:01.547614+0000","last_became_peered":"2024-05-26T22:56:01.547614+0000","last_unstale":"2024-05-26T22:56:03.792626+0000","last_undegraded":"2024-05-26T22:56:03.792626+0000","last_fullsized":"2024-05-26T22:56:03.792626+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_clean_scrub_stamp":"2024-05-26T22:56:00.479878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:15:20.201446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":331,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:04.618984+0000","last_change":"2024-05-26T22:48:12.068917+0000","last_active":"2024-05-26T22:56:04.618984+0000","last_peered":"2024-05-26T22:56:04.618984+0000","last_clean":"2024-05-26T22:56:04.618984+0000","last_became_active":"2024-05-26T22:48:12.065934+0000","last_became_peered":"2024-05-26T22:48:12.065934+0000","last_unstale":"2024-05-26T22:56:04.618984+0000","last_undegraded":"2024-05-26T22:56:04.618984+0000","last_fullsized":"2024-05-26T22:56:04.618984+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:10.628122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:10.628122+0000","last_clean_scrub_stamp":"2024-05-26T22:48:10.628122+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:14:58.895911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574946,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33072,"kb_used_data":1888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338768,"statfs":{"total":96636764160,"available":96602898432,"internally_reserved":0,"allocated":1933312,"data_stored":1171578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574946,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32484,"kb_used_data":1304,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339356,"statfs":{"total":96636764160,"available":96603500544,"internally_reserved":0,"allocated":1335296,"data_stored":580664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574947,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32488,"kb_used_data":1308,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339352,"statfs":{"total":96636764160,"available":96603496448,"internally_reserved":0,"allocated":1339392,"data_stored":581198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574946,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32480,"kb_used_data":1300,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339360,"statfs":{"total":96636764160,"available":96603504640,"internally_reserved":0,"allocated":1331200,"data_stored":580594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574946,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36660,"kb_used_data":1316,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335180,"statfs":{"total":96636764160,"available":96599224320,"internally_reserved":0,"allocated":1347584,"data_stored":582276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574946,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32480,"kb_used_data":1300,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339360,"statfs":{"total":96636764160,"available":96603504640,"internally_reserved":0,"allocated":1331200,"data_stored":580594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574946,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36648,"kb_used_data":1308,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335192,"statfs":{"total":96636764160,"available":96599236608,"internally_reserved":0,"allocated":1339392,"data_stored":580718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574946,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37240,"kb_used_data":1900,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334600,"statfs":{"total":96636764160,"available":96598630400,"internally_reserved":0,"allocated":1945600,"data_stored":1172686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-26T22:56:06.965 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-26T22:56:07.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.202+0000 7fab7f200640 1 Processor -- start 2024-05-26T22:56:07.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.202+0000 7fab7f200640 1 -- start start 2024-05-26T22:56:07.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.203+0000 7fab7f200640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fab8006ebc0 0x7fab800cedf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:07.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.203+0000 7fab7f200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fab80071b70 0x7fab800b3d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:07.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.203+0000 7fab7f200640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab800722a0 con 0x7fab80144340 2024-05-26T22:56:07.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.203+0000 7fab7f200640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fab80072eb0 con 0x7fab8006ebc0 2024-05-26T22:56:07.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.203+0000 7fab7f200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fab80072580 con 0x7fab80071b70 2024-05-26T22:56:07.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.204+0000 7fab7e800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fab80071b70 0x7fab800b3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:07.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.204+0000 7fab7de00640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7fab80144340 0x7fab8006e4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.131:39672/0 (socket says 172.21.15.131:39672) 2024-05-26T22:56:07.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.204+0000 7fab7de00640 1 -- 172.21.15.131:0/1207998773 learned_addr learned my addr 172.21.15.131:0/1207998773 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:56:07.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.205+0000 7fab75400640 1 --2- 172.21.15.131:0/1207998773 >> v2:172.21.15.187:3300/0 conn(0x7fab8006ebc0 0x7fab800cedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:07.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.205+0000 7fab7e800640 1 -- 172.21.15.131:0/1207998773 >> v2:172.21.15.187:3300/0 conn(0x7fab8006ebc0 msgr2=0x7fab800cedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:07.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.205+0000 7fab7e800640 1 --2- 172.21.15.131:0/1207998773 >> v2:172.21.15.187:3300/0 conn(0x7fab8006ebc0 0x7fab800cedf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:07.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.205+0000 7fab7e800640 1 -- 172.21.15.131:0/1207998773 >> v1:172.21.15.187:6789/0 conn(0x7fab80144340 legacy=0x7fab8006e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:07.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.205+0000 7fab7e800640 1 -- 172.21.15.131:0/1207998773 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab80072860 con 0x7fab80071b70 2024-05-26T22:56:07.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.206+0000 7fab7e800640 1 --2- 172.21.15.131:0/1207998773 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fab80071b70 0x7fab800b3d20 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fab780097a0 tx=0x7fab7802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b14a20c7ae365336 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:07.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.207+0000 7fab7d400640 1 -- 172.21.15.131:0/1207998773 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab78037070 con 0x7fab80071b70 2024-05-26T22:56:07.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.207+0000 7fab7d400640 1 -- 172.21.15.131:0/1207998773 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab78032440 con 0x7fab80071b70 2024-05-26T22:56:07.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.207+0000 7fab7d400640 1 -- 172.21.15.131:0/1207998773 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab78032810 con 0x7fab80071b70 2024-05-26T22:56:07.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.208+0000 7fab7f200640 1 -- 172.21.15.131:0/1207998773 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fab80071b70 msgr2=0x7fab800b3d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:07.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.208+0000 7fab7f200640 1 --2- 172.21.15.131:0/1207998773 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fab80071b70 0x7fab800b3d20 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fab780097a0 tx=0x7fab7802a180 comp rx=0 tx=0).stop 2024-05-26T22:56:07.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.208+0000 7fab7f200640 1 -- 172.21.15.131:0/1207998773 shutdown_connections 2024-05-26T22:56:07.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.208+0000 7fab7f200640 1 --2- 172.21.15.131:0/1207998773 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fab80071b70 0x7fab800b3d20 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:07.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.208+0000 7fab7f200640 1 --2- 172.21.15.131:0/1207998773 >> v2:172.21.15.187:3300/0 conn(0x7fab8006ebc0 0x7fab800cedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:07.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.208+0000 7fab7f200640 1 -- 172.21.15.131:0/1207998773 >> 172.21.15.131:0/1207998773 conn(0x7fab800cff40 msgr2=0x7fab800d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:56:07.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.208+0000 7fab7f200640 1 -- 172.21.15.131:0/1207998773 shutdown_connections 2024-05-26T22:56:07.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.208+0000 7fab7f200640 1 -- 172.21.15.131:0/1207998773 wait complete. 2024-05-26T22:56:07.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.210+0000 7fab7f200640 1 Processor -- start 2024-05-26T22:56:07.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.210+0000 7fab7f200640 1 -- start start 2024-05-26T22:56:07.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.211+0000 7fab7f200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab8006ebc0 0x7fab8015a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:07.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.211+0000 7fab7f200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fab80071b70 0x7fab801f45e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:07.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.211+0000 7fab7f200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fab80144340 0x7fab800b0e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:07.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.211+0000 7fab7f200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fab800b4260 con 0x7fab8006ebc0 2024-05-26T22:56:07.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.211+0000 7fab7f200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fab80071fa0 con 0x7fab80071b70 2024-05-26T22:56:07.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.211+0000 7fab7f200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fab800b43e0 con 0x7fab80144340 2024-05-26T22:56:07.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.212+0000 7fab7de00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab8006ebc0 0x7fab8015a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:07.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.212+0000 7fab7de00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab8006ebc0 0x7fab8015a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54454/0 (socket says 172.21.15.131:54454) 2024-05-26T22:56:07.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.212+0000 7fab7de00640 1 -- 172.21.15.131:0/1633865131 learned_addr learned my addr 172.21.15.131:0/1633865131 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:56:07.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.212+0000 7fab7de00640 1 -- 172.21.15.131:0/1633865131 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fab80144340 msgr2=0x7fab800b0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:07.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.212+0000 7fab7de00640 1 --2- 172.21.15.131:0/1633865131 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fab80144340 0x7fab800b0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:07.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.212+0000 7fab7de00640 1 -- 172.21.15.131:0/1633865131 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fab80071b70 msgr2=0x7fab801f45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:07.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.212+0000 7fab7de00640 1 --2- 172.21.15.131:0/1633865131 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fab80071b70 0x7fab801f45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:07.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.212+0000 7fab7de00640 1 -- 172.21.15.131:0/1633865131 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab801f4e70 con 0x7fab8006ebc0 2024-05-26T22:56:07.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.212+0000 7fab7de00640 1 --2- 172.21.15.131:0/1633865131 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab8006ebc0 0x7fab8015a8f0 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7fab7000ca70 tx=0x7fab7000cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:07.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.213+0000 7fab76a00640 1 -- 172.21.15.131:0/1633865131 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab70017070 con 0x7fab8006ebc0 2024-05-26T22:56:07.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.213+0000 7fab76a00640 1 -- 172.21.15.131:0/1633865131 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab7000abd0 con 0x7fab8006ebc0 2024-05-26T22:56:07.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.213+0000 7fab76a00640 1 -- 172.21.15.131:0/1633865131 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab70012400 con 0x7fab8006ebc0 2024-05-26T22:56:07.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.213+0000 7fab7f200640 1 -- 172.21.15.131:0/1633865131 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab801f5150 con 0x7fab8006ebc0 2024-05-26T22:56:07.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.213+0000 7fab7f200640 1 -- 172.21.15.131:0/1633865131 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab8015b470 con 0x7fab8006ebc0 2024-05-26T22:56:07.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.214+0000 7fab7f200640 1 -- 172.21.15.131:0/1633865131 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab800694e0 con 0x7fab8006ebc0 2024-05-26T22:56:07.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.215+0000 7fab76a00640 1 -- 172.21.15.131:0/1633865131 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fab7001a070 con 0x7fab8006ebc0 2024-05-26T22:56:07.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.216+0000 7fab76a00640 1 --2- 172.21.15.131:0/1633865131 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fab5c06be30 0x7fab5c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:07.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.217+0000 7fab76a00640 1 -- 172.21.15.131:0/1633865131 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5709+0+0 (secure 0 0 0) 0x7fab7008ab70 con 0x7fab8006ebc0 2024-05-26T22:56:07.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.217+0000 7fab75400640 1 --2- 172.21.15.131:0/1633865131 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fab5c06be30 0x7fab5c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:07.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.218+0000 7fab75400640 1 --2- 172.21.15.131:0/1633865131 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fab5c06be30 0x7fab5c06e2f0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fab8015d980 tx=0x7fab68008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:07.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.222+0000 7fab76a00640 1 -- 172.21.15.131:0/1633865131 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fab70059510 con 0x7fab8006ebc0 2024-05-26T22:56:07.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.421+0000 7fab7f200640 1 -- 172.21.15.131:0/1633865131 --> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fab800ce720 con 0x7fab5c06be30 2024-05-26T22:56:07.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.429+0000 7fab76a00640 1 -- 172.21.15.131:0/1633865131 <== mgr.4104 v2:172.21.15.131:6832/3484832947 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252483 (secure 0 0 0) 0x7fab800ce720 con 0x7fab5c06be30 2024-05-26T22:56:07.431 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:56:07.433 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-05-26T22:56:07.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.433+0000 7fab7f200640 1 -- 172.21.15.131:0/1633865131 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fab5c06be30 msgr2=0x7fab5c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:07.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.433+0000 7fab7f200640 1 --2- 172.21.15.131:0/1633865131 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fab5c06be30 0x7fab5c06e2f0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fab8015d980 tx=0x7fab68008040 comp rx=0 tx=0).stop 2024-05-26T22:56:07.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.433+0000 7fab7f200640 1 -- 172.21.15.131:0/1633865131 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab8006ebc0 msgr2=0x7fab8015a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:07.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.433+0000 7fab7f200640 1 --2- 172.21.15.131:0/1633865131 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab8006ebc0 0x7fab8015a8f0 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7fab7000ca70 tx=0x7fab7000cf30 comp rx=0 tx=0).stop 2024-05-26T22:56:07.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.433+0000 7fab7f200640 1 -- 172.21.15.131:0/1633865131 shutdown_connections 2024-05-26T22:56:07.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.433+0000 7fab7f200640 1 --2- 172.21.15.131:0/1633865131 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fab80144340 0x7fab800b0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:07.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.433+0000 7fab7f200640 1 --2- 172.21.15.131:0/1633865131 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fab5c06be30 0x7fab5c06e2f0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:07.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.433+0000 7fab7f200640 1 --2- 172.21.15.131:0/1633865131 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fab80071b70 0x7fab801f45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:07.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.433+0000 7fab7f200640 1 --2- 172.21.15.131:0/1633865131 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab8006ebc0 0x7fab8015a8f0 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:07.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.433+0000 7fab7f200640 1 -- 172.21.15.131:0/1633865131 >> 172.21.15.131:0/1633865131 conn(0x7fab800cff40 msgr2=0x7fab8015ded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:56:07.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.433+0000 7fab7f200640 1 -- 172.21.15.131:0/1633865131 shutdown_connections 2024-05-26T22:56:07.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.434+0000 7fab7f200640 1 -- 172.21.15.131:0/1633865131 wait complete. 2024-05-26T22:56:07.452 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":338,"stamp":"2024-05-26T22:56:06.539824+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":273552,"kb_used_data":11624,"kb_used_omap":12,"kb_used_meta":261811,"kb_avail":754701168,"statfs":{"total":773094113280,"available":772813996032,"internally_reserved":0,"allocated":11902976,"data_stored":5830308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":268095025},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.004022"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:04.858763+0000","last_change":"2024-05-26T22:56:02.206696+0000","last_active":"2024-05-26T22:56:04.858763+0000","last_peered":"2024-05-26T22:56:04.858763+0000","last_clean":"2024-05-26T22:56:04.858763+0000","last_became_active":"2024-05-26T22:56:02.206382+0000","last_became_peered":"2024-05-26T22:56:02.206382+0000","last_unstale":"2024-05-26T22:56:04.858763+0000","last_undegraded":"2024-05-26T22:56:04.858763+0000","last_fullsized":"2024-05-26T22:56:04.858763+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:37:23.034583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:03.190349+0000","last_change":"2024-05-26T22:56:02.206286+0000","last_active":"2024-05-26T22:56:03.190349+0000","last_peered":"2024-05-26T22:56:03.190349+0000","last_clean":"2024-05-26T22:56:03.190349+0000","last_became_active":"2024-05-26T22:56:02.205988+0000","last_became_peered":"2024-05-26T22:56:02.205988+0000","last_unstale":"2024-05-26T22:56:03.190349+0000","last_undegraded":"2024-05-26T22:56:03.190349+0000","last_fullsized":"2024-05-26T22:56:03.190349+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:01:44.644419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:04.619111+0000","last_change":"2024-05-26T22:56:02.220188+0000","last_active":"2024-05-26T22:56:04.619111+0000","last_peered":"2024-05-26T22:56:04.619111+0000","last_clean":"2024-05-26T22:56:04.619111+0000","last_became_active":"2024-05-26T22:56:02.219879+0000","last_became_peered":"2024-05-26T22:56:02.219879+0000","last_unstale":"2024-05-26T22:56:04.619111+0000","last_undegraded":"2024-05-26T22:56:04.619111+0000","last_fullsized":"2024-05-26T22:56:04.619111+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T23:00:17.718633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:04.857924+0000","last_change":"2024-05-26T22:56:02.212605+0000","last_active":"2024-05-26T22:56:04.857924+0000","last_peered":"2024-05-26T22:56:04.857924+0000","last_clean":"2024-05-26T22:56:04.857924+0000","last_became_active":"2024-05-26T22:56:02.210277+0000","last_became_peered":"2024-05-26T22:56:02.210277+0000","last_unstale":"2024-05-26T22:56:04.857924+0000","last_undegraded":"2024-05-26T22:56:04.857924+0000","last_fullsized":"2024-05-26T22:56:04.857924+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:26:04.745401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:03.180622+0000","last_change":"2024-05-26T22:56:02.211088+0000","last_active":"2024-05-26T22:56:03.180622+0000","last_peered":"2024-05-26T22:56:03.180622+0000","last_clean":"2024-05-26T22:56:03.180622+0000","last_became_active":"2024-05-26T22:56:02.210534+0000","last_became_peered":"2024-05-26T22:56:02.210534+0000","last_unstale":"2024-05-26T22:56:03.180622+0000","last_undegraded":"2024-05-26T22:56:03.180622+0000","last_fullsized":"2024-05-26T22:56:03.180622+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:09:43.360447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:04.713885+0000","last_change":"2024-05-26T22:56:02.210694+0000","last_active":"2024-05-26T22:56:04.713885+0000","last_peered":"2024-05-26T22:56:04.713885+0000","last_clean":"2024-05-26T22:56:04.713885+0000","last_became_active":"2024-05-26T22:56:02.210406+0000","last_became_peered":"2024-05-26T22:56:02.210406+0000","last_unstale":"2024-05-26T22:56:04.713885+0000","last_undegraded":"2024-05-26T22:56:04.713885+0000","last_fullsized":"2024-05-26T22:56:04.713885+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T23:18:00.396595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:03.792540+0000","last_change":"2024-05-26T22:56:02.203361+0000","last_active":"2024-05-26T22:56:03.792540+0000","last_peered":"2024-05-26T22:56:03.792540+0000","last_clean":"2024-05-26T22:56:03.792540+0000","last_became_active":"2024-05-26T22:56:02.200587+0000","last_became_peered":"2024-05-26T22:56:02.200587+0000","last_unstale":"2024-05-26T22:56:03.792540+0000","last_undegraded":"2024-05-26T22:56:03.792540+0000","last_fullsized":"2024-05-26T22:56:03.792540+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T23:18:00.396595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:04.618463+0000","last_change":"2024-05-26T22:56:02.213343+0000","last_active":"2024-05-26T22:56:04.618463+0000","last_peered":"2024-05-26T22:56:04.618463+0000","last_clean":"2024-05-26T22:56:04.618463+0000","last_became_active":"2024-05-26T22:56:02.212641+0000","last_became_peered":"2024-05-26T22:56:02.212641+0000","last_unstale":"2024-05-26T22:56:04.618463+0000","last_undegraded":"2024-05-26T22:56:04.618463+0000","last_fullsized":"2024-05-26T22:56:04.618463+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:26:53.366151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:04.859062+0000","last_change":"2024-05-26T22:56:02.218504+0000","last_active":"2024-05-26T22:56:04.859062+0000","last_peered":"2024-05-26T22:56:04.859062+0000","last_clean":"2024-05-26T22:56:04.859062+0000","last_became_active":"2024-05-26T22:56:02.217386+0000","last_became_peered":"2024-05-26T22:56:02.217386+0000","last_unstale":"2024-05-26T22:56:04.859062+0000","last_undegraded":"2024-05-26T22:56:04.859062+0000","last_fullsized":"2024-05-26T22:56:04.859062+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:26:29.808147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:04.858473+0000","last_change":"2024-05-26T22:56:02.219576+0000","last_active":"2024-05-26T22:56:04.858473+0000","last_peered":"2024-05-26T22:56:04.858473+0000","last_clean":"2024-05-26T22:56:04.858473+0000","last_became_active":"2024-05-26T22:56:02.219240+0000","last_became_peered":"2024-05-26T22:56:02.219240+0000","last_unstale":"2024-05-26T22:56:04.858473+0000","last_undegraded":"2024-05-26T22:56:04.858473+0000","last_fullsized":"2024-05-26T22:56:04.858473+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:51:15.280396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:03.188069+0000","last_change":"2024-05-26T22:56:02.207870+0000","last_active":"2024-05-26T22:56:03.188069+0000","last_peered":"2024-05-26T22:56:03.188069+0000","last_clean":"2024-05-26T22:56:03.188069+0000","last_became_active":"2024-05-26T22:56:02.207450+0000","last_became_peered":"2024-05-26T22:56:02.207450+0000","last_unstale":"2024-05-26T22:56:03.188069+0000","last_undegraded":"2024-05-26T22:56:03.188069+0000","last_fullsized":"2024-05-26T22:56:03.188069+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:42:08.736487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:04.858110+0000","last_change":"2024-05-26T22:56:02.217924+0000","last_active":"2024-05-26T22:56:04.858110+0000","last_peered":"2024-05-26T22:56:04.858110+0000","last_clean":"2024-05-26T22:56:04.858110+0000","last_became_active":"2024-05-26T22:56:02.217534+0000","last_became_peered":"2024-05-26T22:56:02.217534+0000","last_unstale":"2024-05-26T22:56:04.858110+0000","last_undegraded":"2024-05-26T22:56:04.858110+0000","last_fullsized":"2024-05-26T22:56:04.858110+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:48:21.809824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:04.619185+0000","last_change":"2024-05-26T22:56:02.221256+0000","last_active":"2024-05-26T22:56:04.619185+0000","last_peered":"2024-05-26T22:56:04.619185+0000","last_clean":"2024-05-26T22:56:04.619185+0000","last_became_active":"2024-05-26T22:56:02.220940+0000","last_became_peered":"2024-05-26T22:56:02.220940+0000","last_unstale":"2024-05-26T22:56:04.619185+0000","last_undegraded":"2024-05-26T22:56:04.619185+0000","last_fullsized":"2024-05-26T22:56:04.619185+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:31:16.025886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:04.618524+0000","last_change":"2024-05-26T22:56:02.216297+0000","last_active":"2024-05-26T22:56:04.618524+0000","last_peered":"2024-05-26T22:56:04.618524+0000","last_clean":"2024-05-26T22:56:04.618524+0000","last_became_active":"2024-05-26T22:56:02.215965+0000","last_became_peered":"2024-05-26T22:56:02.215965+0000","last_unstale":"2024-05-26T22:56:04.618524+0000","last_undegraded":"2024-05-26T22:56:04.618524+0000","last_fullsized":"2024-05-26T22:56:04.618524+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:42:08.736487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:04.713769+0000","last_change":"2024-05-26T22:56:02.208433+0000","last_active":"2024-05-26T22:56:04.713769+0000","last_peered":"2024-05-26T22:56:04.713769+0000","last_clean":"2024-05-26T22:56:04.713769+0000","last_became_active":"2024-05-26T22:56:02.207879+0000","last_became_peered":"2024-05-26T22:56:02.207879+0000","last_unstale":"2024-05-26T22:56:04.713769+0000","last_undegraded":"2024-05-26T22:56:04.713769+0000","last_fullsized":"2024-05-26T22:56:04.713769+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:10:26.651321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:04.857726+0000","last_change":"2024-05-26T22:56:02.214802+0000","last_active":"2024-05-26T22:56:04.857726+0000","last_peered":"2024-05-26T22:56:04.857726+0000","last_clean":"2024-05-26T22:56:04.857726+0000","last_became_active":"2024-05-26T22:56:02.214480+0000","last_became_peered":"2024-05-26T22:56:02.214480+0000","last_unstale":"2024-05-26T22:56:04.857726+0000","last_undegraded":"2024-05-26T22:56:04.857726+0000","last_fullsized":"2024-05-26T22:56:04.857726+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:29:54.582494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:03.188644+0000","last_change":"2024-05-26T22:56:02.205936+0000","last_active":"2024-05-26T22:56:03.188644+0000","last_peered":"2024-05-26T22:56:03.188644+0000","last_clean":"2024-05-26T22:56:03.188644+0000","last_became_active":"2024-05-26T22:56:02.204220+0000","last_became_peered":"2024-05-26T22:56:02.204220+0000","last_unstale":"2024-05-26T22:56:03.188644+0000","last_undegraded":"2024-05-26T22:56:03.188644+0000","last_fullsized":"2024-05-26T22:56:03.188644+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:11:53.760719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:03.189211+0000","last_change":"2024-05-26T22:56:02.209865+0000","last_active":"2024-05-26T22:56:03.189211+0000","last_peered":"2024-05-26T22:56:03.189211+0000","last_clean":"2024-05-26T22:56:03.189211+0000","last_became_active":"2024-05-26T22:56:02.209586+0000","last_became_peered":"2024-05-26T22:56:02.209586+0000","last_unstale":"2024-05-26T22:56:03.189211+0000","last_undegraded":"2024-05-26T22:56:03.189211+0000","last_fullsized":"2024-05-26T22:56:03.189211+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:42:08.736487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:04.619319+0000","last_change":"2024-05-26T22:56:02.220013+0000","last_active":"2024-05-26T22:56:04.619319+0000","last_peered":"2024-05-26T22:56:04.619319+0000","last_clean":"2024-05-26T22:56:04.619319+0000","last_became_active":"2024-05-26T22:56:02.219629+0000","last_became_peered":"2024-05-26T22:56:02.219629+0000","last_unstale":"2024-05-26T22:56:04.619319+0000","last_undegraded":"2024-05-26T22:56:04.619319+0000","last_fullsized":"2024-05-26T22:56:04.619319+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:39:26.930899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:03.189843+0000","last_change":"2024-05-26T22:56:02.211102+0000","last_active":"2024-05-26T22:56:03.189843+0000","last_peered":"2024-05-26T22:56:03.189843+0000","last_clean":"2024-05-26T22:56:03.189843+0000","last_became_active":"2024-05-26T22:56:02.210226+0000","last_became_peered":"2024-05-26T22:56:02.210226+0000","last_unstale":"2024-05-26T22:56:03.189843+0000","last_undegraded":"2024-05-26T22:56:03.189843+0000","last_fullsized":"2024-05-26T22:56:03.189843+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:39:26.930899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:03.189193+0000","last_change":"2024-05-26T22:56:02.207390+0000","last_active":"2024-05-26T22:56:03.189193+0000","last_peered":"2024-05-26T22:56:03.189193+0000","last_clean":"2024-05-26T22:56:03.189193+0000","last_became_active":"2024-05-26T22:56:02.207090+0000","last_became_peered":"2024-05-26T22:56:02.207090+0000","last_unstale":"2024-05-26T22:56:03.189193+0000","last_undegraded":"2024-05-26T22:56:03.189193+0000","last_fullsized":"2024-05-26T22:56:03.189193+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:31:16.025886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:03.500824+0000","last_change":"2024-05-26T22:56:02.216774+0000","last_active":"2024-05-26T22:56:03.500824+0000","last_peered":"2024-05-26T22:56:03.500824+0000","last_clean":"2024-05-26T22:56:03.500824+0000","last_became_active":"2024-05-26T22:56:02.214545+0000","last_became_peered":"2024-05-26T22:56:02.214545+0000","last_unstale":"2024-05-26T22:56:03.500824+0000","last_undegraded":"2024-05-26T22:56:03.500824+0000","last_fullsized":"2024-05-26T22:56:03.500824+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:01:44.644419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:03.187020+0000","last_change":"2024-05-26T22:56:02.209906+0000","last_active":"2024-05-26T22:56:03.187020+0000","last_peered":"2024-05-26T22:56:03.187020+0000","last_clean":"2024-05-26T22:56:03.187020+0000","last_became_active":"2024-05-26T22:56:02.209633+0000","last_became_peered":"2024-05-26T22:56:02.209633+0000","last_unstale":"2024-05-26T22:56:03.187020+0000","last_undegraded":"2024-05-26T22:56:03.187020+0000","last_fullsized":"2024-05-26T22:56:03.187020+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:18:00.119575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:03.500325+0000","last_change":"2024-05-26T22:56:02.217896+0000","last_active":"2024-05-26T22:56:03.500325+0000","last_peered":"2024-05-26T22:56:03.500325+0000","last_clean":"2024-05-26T22:56:03.500325+0000","last_became_active":"2024-05-26T22:56:02.217589+0000","last_became_peered":"2024-05-26T22:56:02.217589+0000","last_unstale":"2024-05-26T22:56:03.500325+0000","last_undegraded":"2024-05-26T22:56:03.500325+0000","last_fullsized":"2024-05-26T22:56:03.500325+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:51:41.862299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:03.189052+0000","last_change":"2024-05-26T22:56:02.204631+0000","last_active":"2024-05-26T22:56:03.189052+0000","last_peered":"2024-05-26T22:56:03.189052+0000","last_clean":"2024-05-26T22:56:03.189052+0000","last_became_active":"2024-05-26T22:56:02.203750+0000","last_became_peered":"2024-05-26T22:56:02.203750+0000","last_unstale":"2024-05-26T22:56:03.189052+0000","last_undegraded":"2024-05-26T22:56:03.189052+0000","last_fullsized":"2024-05-26T22:56:03.189052+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T23:00:17.718633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:04.858124+0000","last_change":"2024-05-26T22:56:02.216396+0000","last_active":"2024-05-26T22:56:04.858124+0000","last_peered":"2024-05-26T22:56:04.858124+0000","last_clean":"2024-05-26T22:56:04.858124+0000","last_became_active":"2024-05-26T22:56:02.216136+0000","last_became_peered":"2024-05-26T22:56:02.216136+0000","last_unstale":"2024-05-26T22:56:04.858124+0000","last_undegraded":"2024-05-26T22:56:04.858124+0000","last_fullsized":"2024-05-26T22:56:04.858124+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:56:56.613417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:04.714541+0000","last_change":"2024-05-26T22:56:02.205608+0000","last_active":"2024-05-26T22:56:04.714541+0000","last_peered":"2024-05-26T22:56:04.714541+0000","last_clean":"2024-05-26T22:56:04.714541+0000","last_became_active":"2024-05-26T22:56:02.205301+0000","last_became_peered":"2024-05-26T22:56:02.205301+0000","last_unstale":"2024-05-26T22:56:04.714541+0000","last_undegraded":"2024-05-26T22:56:04.714541+0000","last_fullsized":"2024-05-26T22:56:04.714541+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:09:51.987645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:03.501248+0000","last_change":"2024-05-26T22:56:02.218062+0000","last_active":"2024-05-26T22:56:03.501248+0000","last_peered":"2024-05-26T22:56:03.501248+0000","last_clean":"2024-05-26T22:56:03.501248+0000","last_became_active":"2024-05-26T22:56:02.216743+0000","last_became_peered":"2024-05-26T22:56:02.216743+0000","last_unstale":"2024-05-26T22:56:03.501248+0000","last_undegraded":"2024-05-26T22:56:03.501248+0000","last_fullsized":"2024-05-26T22:56:03.501248+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:06:11.312538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:03.188128+0000","last_change":"2024-05-26T22:56:02.216764+0000","last_active":"2024-05-26T22:56:03.188128+0000","last_peered":"2024-05-26T22:56:03.188128+0000","last_clean":"2024-05-26T22:56:03.188128+0000","last_became_active":"2024-05-26T22:56:02.216392+0000","last_became_peered":"2024-05-26T22:56:02.216392+0000","last_unstale":"2024-05-26T22:56:03.188128+0000","last_undegraded":"2024-05-26T22:56:03.188128+0000","last_fullsized":"2024-05-26T22:56:03.188128+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:33:00.893414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:03.792928+0000","last_change":"2024-05-26T22:56:02.213781+0000","last_active":"2024-05-26T22:56:03.792928+0000","last_peered":"2024-05-26T22:56:03.792928+0000","last_clean":"2024-05-26T22:56:03.792928+0000","last_became_active":"2024-05-26T22:56:02.213494+0000","last_became_peered":"2024-05-26T22:56:02.213494+0000","last_unstale":"2024-05-26T22:56:03.792928+0000","last_undegraded":"2024-05-26T22:56:03.792928+0000","last_fullsized":"2024-05-26T22:56:03.792928+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:09:51.987645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:04.858331+0000","last_change":"2024-05-26T22:56:02.215800+0000","last_active":"2024-05-26T22:56:04.858331+0000","last_peered":"2024-05-26T22:56:04.858331+0000","last_clean":"2024-05-26T22:56:04.858331+0000","last_became_active":"2024-05-26T22:56:02.215492+0000","last_became_peered":"2024-05-26T22:56:02.215492+0000","last_unstale":"2024-05-26T22:56:04.858331+0000","last_undegraded":"2024-05-26T22:56:04.858331+0000","last_fullsized":"2024-05-26T22:56:04.858331+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:25:21.205123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:04.713754+0000","last_change":"2024-05-26T22:56:02.204465+0000","last_active":"2024-05-26T22:56:04.713754+0000","last_peered":"2024-05-26T22:56:04.713754+0000","last_clean":"2024-05-26T22:56:04.713754+0000","last_became_active":"2024-05-26T22:56:02.202855+0000","last_became_peered":"2024-05-26T22:56:02.202855+0000","last_unstale":"2024-05-26T22:56:04.713754+0000","last_undegraded":"2024-05-26T22:56:04.713754+0000","last_fullsized":"2024-05-26T22:56:04.713754+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:37:30.606174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:04.714812+0000","last_change":"2024-05-26T22:56:02.201346+0000","last_active":"2024-05-26T22:56:04.714812+0000","last_peered":"2024-05-26T22:56:04.714812+0000","last_clean":"2024-05-26T22:56:04.714812+0000","last_became_active":"2024-05-26T22:56:02.200664+0000","last_became_peered":"2024-05-26T22:56:02.200664+0000","last_unstale":"2024-05-26T22:56:04.714812+0000","last_undegraded":"2024-05-26T22:56:04.714812+0000","last_fullsized":"2024-05-26T22:56:04.714812+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:23:33.849977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"106'4","reported_seq":18,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:03.793067+0000","last_change":"2024-05-26T22:56:01.547833+0000","last_active":"2024-05-26T22:56:03.793067+0000","last_peered":"2024-05-26T22:56:03.793067+0000","last_clean":"2024-05-26T22:56:03.793067+0000","last_became_active":"2024-05-26T22:56:01.546355+0000","last_became_peered":"2024-05-26T22:56:01.546355+0000","last_unstale":"2024-05-26T22:56:03.793067+0000","last_undegraded":"2024-05-26T22:56:03.793067+0000","last_fullsized":"2024-05-26T22:56:03.793067+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_clean_scrub_stamp":"2024-05-26T22:56:00.479878+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:57:27.405643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:03.190291+0000","last_change":"2024-05-26T22:56:02.209334+0000","last_active":"2024-05-26T22:56:03.190291+0000","last_peered":"2024-05-26T22:56:03.190291+0000","last_clean":"2024-05-26T22:56:03.190291+0000","last_became_active":"2024-05-26T22:56:02.208968+0000","last_became_peered":"2024-05-26T22:56:02.208968+0000","last_unstale":"2024-05-26T22:56:03.190291+0000","last_undegraded":"2024-05-26T22:56:03.190291+0000","last_fullsized":"2024-05-26T22:56:03.190291+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:32:11.698764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"106'1","reported_seq":15,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:04.619240+0000","last_change":"2024-05-26T22:56:01.546843+0000","last_active":"2024-05-26T22:56:04.619240+0000","last_peered":"2024-05-26T22:56:04.619240+0000","last_clean":"2024-05-26T22:56:04.619240+0000","last_became_active":"2024-05-26T22:56:01.546473+0000","last_became_peered":"2024-05-26T22:56:01.546473+0000","last_unstale":"2024-05-26T22:56:04.619240+0000","last_undegraded":"2024-05-26T22:56:04.619240+0000","last_fullsized":"2024-05-26T22:56:04.619240+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_clean_scrub_stamp":"2024-05-26T22:56:00.479878+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T23:17:59.378357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:04.714796+0000","last_change":"2024-05-26T22:56:02.205289+0000","last_active":"2024-05-26T22:56:04.714796+0000","last_peered":"2024-05-26T22:56:04.714796+0000","last_clean":"2024-05-26T22:56:04.714796+0000","last_became_active":"2024-05-26T22:56:02.204940+0000","last_became_peered":"2024-05-26T22:56:02.204940+0000","last_unstale":"2024-05-26T22:56:04.714796+0000","last_undegraded":"2024-05-26T22:56:04.714796+0000","last_fullsized":"2024-05-26T22:56:04.714796+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:41:04.672377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"106'2","reported_seq":16,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:03.180709+0000","last_change":"2024-05-26T22:56:01.547301+0000","last_active":"2024-05-26T22:56:03.180709+0000","last_peered":"2024-05-26T22:56:03.180709+0000","last_clean":"2024-05-26T22:56:03.180709+0000","last_became_active":"2024-05-26T22:56:01.545934+0000","last_became_peered":"2024-05-26T22:56:01.545934+0000","last_unstale":"2024-05-26T22:56:03.180709+0000","last_undegraded":"2024-05-26T22:56:03.180709+0000","last_fullsized":"2024-05-26T22:56:03.180709+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_clean_scrub_stamp":"2024-05-26T22:56:00.479878+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:25:06.623425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:04.619509+0000","last_change":"2024-05-26T22:56:02.219500+0000","last_active":"2024-05-26T22:56:04.619509+0000","last_peered":"2024-05-26T22:56:04.619509+0000","last_clean":"2024-05-26T22:56:04.619509+0000","last_became_active":"2024-05-26T22:56:02.219204+0000","last_became_peered":"2024-05-26T22:56:02.219204+0000","last_unstale":"2024-05-26T22:56:04.619509+0000","last_undegraded":"2024-05-26T22:56:04.619509+0000","last_fullsized":"2024-05-26T22:56:04.619509+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:01:44.644419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"106'2","reported_seq":16,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:03.180283+0000","last_change":"2024-05-26T22:56:01.546594+0000","last_active":"2024-05-26T22:56:03.180283+0000","last_peered":"2024-05-26T22:56:03.180283+0000","last_clean":"2024-05-26T22:56:03.180283+0000","last_became_active":"2024-05-26T22:56:01.546275+0000","last_became_peered":"2024-05-26T22:56:01.546275+0000","last_unstale":"2024-05-26T22:56:03.180283+0000","last_undegraded":"2024-05-26T22:56:03.180283+0000","last_fullsized":"2024-05-26T22:56:03.180283+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_clean_scrub_stamp":"2024-05-26T22:56:00.479878+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:15:39.374826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:04.857809+0000","last_change":"2024-05-26T22:56:02.219950+0000","last_active":"2024-05-26T22:56:04.857809+0000","last_peered":"2024-05-26T22:56:04.857809+0000","last_clean":"2024-05-26T22:56:04.857809+0000","last_became_active":"2024-05-26T22:56:02.219719+0000","last_became_peered":"2024-05-26T22:56:02.219719+0000","last_unstale":"2024-05-26T22:56:04.857809+0000","last_undegraded":"2024-05-26T22:56:04.857809+0000","last_fullsized":"2024-05-26T22:56:04.857809+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:26:23.188344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"106'1","reported_seq":15,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:03.792371+0000","last_change":"2024-05-26T22:56:01.546848+0000","last_active":"2024-05-26T22:56:03.792371+0000","last_peered":"2024-05-26T22:56:03.792371+0000","last_clean":"2024-05-26T22:56:03.792371+0000","last_became_active":"2024-05-26T22:56:01.546563+0000","last_became_peered":"2024-05-26T22:56:01.546563+0000","last_unstale":"2024-05-26T22:56:03.792371+0000","last_undegraded":"2024-05-26T22:56:03.792371+0000","last_fullsized":"2024-05-26T22:56:03.792371+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_clean_scrub_stamp":"2024-05-26T22:56:00.479878+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:24:06.499767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:03.188597+0000","last_change":"2024-05-26T22:56:02.211063+0000","last_active":"2024-05-26T22:56:03.188597+0000","last_peered":"2024-05-26T22:56:03.188597+0000","last_clean":"2024-05-26T22:56:03.188597+0000","last_became_active":"2024-05-26T22:56:02.210720+0000","last_became_peered":"2024-05-26T22:56:02.210720+0000","last_unstale":"2024-05-26T22:56:03.188597+0000","last_undegraded":"2024-05-26T22:56:03.188597+0000","last_fullsized":"2024-05-26T22:56:03.188597+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:32:11.698764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"106'1","reported_seq":15,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:03.500939+0000","last_change":"2024-05-26T22:56:01.541444+0000","last_active":"2024-05-26T22:56:03.500939+0000","last_peered":"2024-05-26T22:56:03.500939+0000","last_clean":"2024-05-26T22:56:03.500939+0000","last_became_active":"2024-05-26T22:56:01.541115+0000","last_became_peered":"2024-05-26T22:56:01.541115+0000","last_unstale":"2024-05-26T22:56:03.500939+0000","last_undegraded":"2024-05-26T22:56:03.500939+0000","last_fullsized":"2024-05-26T22:56:03.500939+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_clean_scrub_stamp":"2024-05-26T22:56:00.479878+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:32:11.114165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:04.858507+0000","last_change":"2024-05-26T22:56:02.207933+0000","last_active":"2024-05-26T22:56:04.858507+0000","last_peered":"2024-05-26T22:56:04.858507+0000","last_clean":"2024-05-26T22:56:04.858507+0000","last_became_active":"2024-05-26T22:56:02.207438+0000","last_became_peered":"2024-05-26T22:56:02.207438+0000","last_unstale":"2024-05-26T22:56:04.858507+0000","last_undegraded":"2024-05-26T22:56:04.858507+0000","last_fullsized":"2024-05-26T22:56:04.858507+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:21:01.569286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:03.187301+0000","last_change":"2024-05-26T22:56:01.542399+0000","last_active":"2024-05-26T22:56:03.187301+0000","last_peered":"2024-05-26T22:56:03.187301+0000","last_clean":"2024-05-26T22:56:03.187301+0000","last_became_active":"2024-05-26T22:56:01.541752+0000","last_became_peered":"2024-05-26T22:56:01.541752+0000","last_unstale":"2024-05-26T22:56:03.187301+0000","last_undegraded":"2024-05-26T22:56:03.187301+0000","last_fullsized":"2024-05-26T22:56:03.187301+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_clean_scrub_stamp":"2024-05-26T22:56:00.479878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:37:29.587936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:03.792354+0000","last_change":"2024-05-26T22:56:02.204777+0000","last_active":"2024-05-26T22:56:03.792354+0000","last_peered":"2024-05-26T22:56:03.792354+0000","last_clean":"2024-05-26T22:56:03.792354+0000","last_became_active":"2024-05-26T22:56:02.204419+0000","last_became_peered":"2024-05-26T22:56:02.204419+0000","last_unstale":"2024-05-26T22:56:03.792354+0000","last_undegraded":"2024-05-26T22:56:03.792354+0000","last_fullsized":"2024-05-26T22:56:03.792354+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:10:26.651321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:04.858610+0000","last_change":"2024-05-26T22:56:01.545325+0000","last_active":"2024-05-26T22:56:04.858610+0000","last_peered":"2024-05-26T22:56:04.858610+0000","last_clean":"2024-05-26T22:56:04.858610+0000","last_became_active":"2024-05-26T22:56:01.545047+0000","last_became_peered":"2024-05-26T22:56:01.545047+0000","last_unstale":"2024-05-26T22:56:04.858610+0000","last_undegraded":"2024-05-26T22:56:04.858610+0000","last_fullsized":"2024-05-26T22:56:04.858610+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_clean_scrub_stamp":"2024-05-26T22:56:00.479878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:51:40.844061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:03.793186+0000","last_change":"2024-05-26T22:56:02.206158+0000","last_active":"2024-05-26T22:56:03.793186+0000","last_peered":"2024-05-26T22:56:03.793186+0000","last_clean":"2024-05-26T22:56:03.793186+0000","last_became_active":"2024-05-26T22:56:02.205813+0000","last_became_peered":"2024-05-26T22:56:02.205813+0000","last_unstale":"2024-05-26T22:56:03.793186+0000","last_undegraded":"2024-05-26T22:56:03.793186+0000","last_fullsized":"2024-05-26T22:56:03.793186+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:41:04.672377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:03.180351+0000","last_change":"2024-05-26T22:56:01.540411+0000","last_active":"2024-05-26T22:56:03.180351+0000","last_peered":"2024-05-26T22:56:03.180351+0000","last_clean":"2024-05-26T22:56:03.180351+0000","last_became_active":"2024-05-26T22:56:01.540152+0000","last_became_peered":"2024-05-26T22:56:01.540152+0000","last_unstale":"2024-05-26T22:56:03.180351+0000","last_undegraded":"2024-05-26T22:56:03.180351+0000","last_fullsized":"2024-05-26T22:56:03.180351+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_clean_scrub_stamp":"2024-05-26T22:56:00.479878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:41:04.207626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:03.190488+0000","last_change":"2024-05-26T22:56:02.204825+0000","last_active":"2024-05-26T22:56:03.190488+0000","last_peered":"2024-05-26T22:56:03.190488+0000","last_clean":"2024-05-26T22:56:03.190488+0000","last_became_active":"2024-05-26T22:56:02.204460+0000","last_became_peered":"2024-05-26T22:56:02.204460+0000","last_unstale":"2024-05-26T22:56:03.190488+0000","last_undegraded":"2024-05-26T22:56:03.190488+0000","last_fullsized":"2024-05-26T22:56:03.190488+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:11:53.760719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:03.501579+0000","last_change":"2024-05-26T22:56:01.544461+0000","last_active":"2024-05-26T22:56:03.501579+0000","last_peered":"2024-05-26T22:56:03.501579+0000","last_clean":"2024-05-26T22:56:03.501579+0000","last_became_active":"2024-05-26T22:56:01.544192+0000","last_became_peered":"2024-05-26T22:56:01.544192+0000","last_unstale":"2024-05-26T22:56:03.501579+0000","last_undegraded":"2024-05-26T22:56:03.501579+0000","last_fullsized":"2024-05-26T22:56:03.501579+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_clean_scrub_stamp":"2024-05-26T22:56:00.479878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:41:03.654139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:03.501556+0000","last_change":"2024-05-26T22:56:02.213676+0000","last_active":"2024-05-26T22:56:03.501556+0000","last_peered":"2024-05-26T22:56:03.501556+0000","last_clean":"2024-05-26T22:56:03.501556+0000","last_became_active":"2024-05-26T22:56:02.213348+0000","last_became_peered":"2024-05-26T22:56:02.213348+0000","last_unstale":"2024-05-26T22:56:03.501556+0000","last_undegraded":"2024-05-26T22:56:03.501556+0000","last_fullsized":"2024-05-26T22:56:03.501556+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:34:58.504442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"106'3","reported_seq":17,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:04.858483+0000","last_change":"2024-05-26T22:56:01.524769+0000","last_active":"2024-05-26T22:56:04.858483+0000","last_peered":"2024-05-26T22:56:04.858483+0000","last_clean":"2024-05-26T22:56:04.858483+0000","last_became_active":"2024-05-26T22:56:01.524362+0000","last_became_peered":"2024-05-26T22:56:01.524362+0000","last_unstale":"2024-05-26T22:56:04.858483+0000","last_undegraded":"2024-05-26T22:56:04.858483+0000","last_fullsized":"2024-05-26T22:56:04.858483+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_clean_scrub_stamp":"2024-05-26T22:56:00.479878+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:31:15.007648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:03.792986+0000","last_change":"2024-05-26T22:56:02.212450+0000","last_active":"2024-05-26T22:56:03.792986+0000","last_peered":"2024-05-26T22:56:03.792986+0000","last_clean":"2024-05-26T22:56:03.792986+0000","last_became_active":"2024-05-26T22:56:02.212117+0000","last_became_peered":"2024-05-26T22:56:02.212117+0000","last_unstale":"2024-05-26T22:56:03.792986+0000","last_undegraded":"2024-05-26T22:56:03.792986+0000","last_fullsized":"2024-05-26T22:56:03.792986+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:47:42.868120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"106'6","reported_seq":20,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:04.858746+0000","last_change":"2024-05-26T22:56:01.539032+0000","last_active":"2024-05-26T22:56:04.858746+0000","last_peered":"2024-05-26T22:56:04.858746+0000","last_clean":"2024-05-26T22:56:04.858746+0000","last_became_active":"2024-05-26T22:56:01.538397+0000","last_became_peered":"2024-05-26T22:56:01.538397+0000","last_unstale":"2024-05-26T22:56:04.858746+0000","last_undegraded":"2024-05-26T22:56:04.858746+0000","last_fullsized":"2024-05-26T22:56:04.858746+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_clean_scrub_stamp":"2024-05-26T22:56:00.479878+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:01:43.626181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:03.500782+0000","last_change":"2024-05-26T22:56:02.208793+0000","last_active":"2024-05-26T22:56:03.500782+0000","last_peered":"2024-05-26T22:56:03.500782+0000","last_clean":"2024-05-26T22:56:03.500782+0000","last_became_active":"2024-05-26T22:56:02.208502+0000","last_became_peered":"2024-05-26T22:56:02.208502+0000","last_unstale":"2024-05-26T22:56:03.500782+0000","last_undegraded":"2024-05-26T22:56:03.500782+0000","last_fullsized":"2024-05-26T22:56:03.500782+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:39:26.930899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"106'2","reported_seq":16,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:04.714131+0000","last_change":"2024-05-26T22:56:01.547593+0000","last_active":"2024-05-26T22:56:04.714131+0000","last_peered":"2024-05-26T22:56:04.714131+0000","last_clean":"2024-05-26T22:56:04.714131+0000","last_became_active":"2024-05-26T22:56:01.547233+0000","last_became_peered":"2024-05-26T22:56:01.547233+0000","last_unstale":"2024-05-26T22:56:04.714131+0000","last_undegraded":"2024-05-26T22:56:04.714131+0000","last_fullsized":"2024-05-26T22:56:04.714131+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_clean_scrub_stamp":"2024-05-26T22:56:00.479878+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T02:24:06.499767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:03.500985+0000","last_change":"2024-05-26T22:56:02.217045+0000","last_active":"2024-05-26T22:56:03.500985+0000","last_peered":"2024-05-26T22:56:03.500985+0000","last_clean":"2024-05-26T22:56:03.500985+0000","last_became_active":"2024-05-26T22:56:02.215543+0000","last_became_peered":"2024-05-26T22:56:02.215543+0000","last_unstale":"2024-05-26T22:56:03.500985+0000","last_undegraded":"2024-05-26T22:56:03.500985+0000","last_fullsized":"2024-05-26T22:56:03.500985+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:50:35.747435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:03.187728+0000","last_change":"2024-05-26T22:56:01.531697+0000","last_active":"2024-05-26T22:56:03.187728+0000","last_peered":"2024-05-26T22:56:03.187728+0000","last_clean":"2024-05-26T22:56:03.187728+0000","last_became_active":"2024-05-26T22:56:01.529624+0000","last_became_peered":"2024-05-26T22:56:01.529624+0000","last_unstale":"2024-05-26T22:56:03.187728+0000","last_undegraded":"2024-05-26T22:56:03.187728+0000","last_fullsized":"2024-05-26T22:56:03.187728+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_clean_scrub_stamp":"2024-05-26T22:56:00.479878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:45:22.206973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:03.500721+0000","last_change":"2024-05-26T22:56:02.217069+0000","last_active":"2024-05-26T22:56:03.500721+0000","last_peered":"2024-05-26T22:56:03.500721+0000","last_clean":"2024-05-26T22:56:03.500721+0000","last_became_active":"2024-05-26T22:56:02.216761+0000","last_became_peered":"2024-05-26T22:56:02.216761+0000","last_unstale":"2024-05-26T22:56:03.500721+0000","last_undegraded":"2024-05-26T22:56:03.500721+0000","last_fullsized":"2024-05-26T22:56:03.500721+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T23:00:17.718633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"106'8","reported_seq":22,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:03.180171+0000","last_change":"2024-05-26T22:56:01.540246+0000","last_active":"2024-05-26T22:56:03.180171+0000","last_peered":"2024-05-26T22:56:03.180171+0000","last_clean":"2024-05-26T22:56:03.180171+0000","last_became_active":"2024-05-26T22:56:01.539929+0000","last_became_peered":"2024-05-26T22:56:01.539929+0000","last_unstale":"2024-05-26T22:56:03.180171+0000","last_undegraded":"2024-05-26T22:56:03.180171+0000","last_fullsized":"2024-05-26T22:56:03.180171+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_clean_scrub_stamp":"2024-05-26T22:56:00.479878+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:33:06.138639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:03.180575+0000","last_change":"2024-05-26T22:56:02.213335+0000","last_active":"2024-05-26T22:56:03.180575+0000","last_peered":"2024-05-26T22:56:03.180575+0000","last_clean":"2024-05-26T22:56:03.180575+0000","last_became_active":"2024-05-26T22:56:02.212699+0000","last_became_peered":"2024-05-26T22:56:02.212699+0000","last_unstale":"2024-05-26T22:56:03.180575+0000","last_undegraded":"2024-05-26T22:56:03.180575+0000","last_fullsized":"2024-05-26T22:56:03.180575+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:19:05.773898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"106'1","reported_seq":15,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:04.859542+0000","last_change":"2024-05-26T22:56:01.544226+0000","last_active":"2024-05-26T22:56:04.859542+0000","last_peered":"2024-05-26T22:56:04.859542+0000","last_clean":"2024-05-26T22:56:04.859542+0000","last_became_active":"2024-05-26T22:56:01.543914+0000","last_became_peered":"2024-05-26T22:56:01.543914+0000","last_unstale":"2024-05-26T22:56:04.859542+0000","last_undegraded":"2024-05-26T22:56:04.859542+0000","last_fullsized":"2024-05-26T22:56:04.859542+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_clean_scrub_stamp":"2024-05-26T22:56:00.479878+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:50:34.729197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:04.714767+0000","last_change":"2024-05-26T22:56:02.204502+0000","last_active":"2024-05-26T22:56:04.714767+0000","last_peered":"2024-05-26T22:56:04.714767+0000","last_clean":"2024-05-26T22:56:04.714767+0000","last_became_active":"2024-05-26T22:56:02.203835+0000","last_became_peered":"2024-05-26T22:56:02.203835+0000","last_unstale":"2024-05-26T22:56:04.714767+0000","last_undegraded":"2024-05-26T22:56:04.714767+0000","last_fullsized":"2024-05-26T22:56:04.714767+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:47:42.868120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"106'2","reported_seq":16,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:03.180618+0000","last_change":"2024-05-26T22:56:01.549401+0000","last_active":"2024-05-26T22:56:03.180618+0000","last_peered":"2024-05-26T22:56:03.180618+0000","last_clean":"2024-05-26T22:56:03.180618+0000","last_became_active":"2024-05-26T22:56:01.548769+0000","last_became_peered":"2024-05-26T22:56:01.548769+0000","last_unstale":"2024-05-26T22:56:03.180618+0000","last_undegraded":"2024-05-26T22:56:03.180618+0000","last_fullsized":"2024-05-26T22:56:03.180618+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_clean_scrub_stamp":"2024-05-26T22:56:00.479878+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:53:56.157795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:04.618757+0000","last_change":"2024-05-26T22:56:02.221755+0000","last_active":"2024-05-26T22:56:04.618757+0000","last_peered":"2024-05-26T22:56:04.618757+0000","last_clean":"2024-05-26T22:56:04.618757+0000","last_became_active":"2024-05-26T22:56:02.221283+0000","last_became_peered":"2024-05-26T22:56:02.221283+0000","last_unstale":"2024-05-26T22:56:04.618757+0000","last_undegraded":"2024-05-26T22:56:04.618757+0000","last_fullsized":"2024-05-26T22:56:04.618757+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:11:53.760719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"106'1","reported_seq":15,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:03.189216+0000","last_change":"2024-05-26T22:56:01.539836+0000","last_active":"2024-05-26T22:56:03.189216+0000","last_peered":"2024-05-26T22:56:03.189216+0000","last_clean":"2024-05-26T22:56:03.189216+0000","last_became_active":"2024-05-26T22:56:01.539241+0000","last_became_peered":"2024-05-26T22:56:01.539241+0000","last_unstale":"2024-05-26T22:56:03.189216+0000","last_undegraded":"2024-05-26T22:56:03.189216+0000","last_fullsized":"2024-05-26T22:56:03.189216+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_clean_scrub_stamp":"2024-05-26T22:56:00.479878+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:32:11.114165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:04.619652+0000","last_change":"2024-05-26T22:56:02.216727+0000","last_active":"2024-05-26T22:56:04.619652+0000","last_peered":"2024-05-26T22:56:04.619652+0000","last_clean":"2024-05-26T22:56:04.619652+0000","last_became_active":"2024-05-26T22:56:02.216384+0000","last_became_peered":"2024-05-26T22:56:02.216384+0000","last_unstale":"2024-05-26T22:56:04.619652+0000","last_undegraded":"2024-05-26T22:56:04.619652+0000","last_fullsized":"2024-05-26T22:56:04.619652+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:50:35.747435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:03.792444+0000","last_change":"2024-05-26T22:56:01.549211+0000","last_active":"2024-05-26T22:56:03.792444+0000","last_peered":"2024-05-26T22:56:03.792444+0000","last_clean":"2024-05-26T22:56:03.792444+0000","last_became_active":"2024-05-26T22:56:01.548445+0000","last_became_peered":"2024-05-26T22:56:01.548445+0000","last_unstale":"2024-05-26T22:56:03.792444+0000","last_undegraded":"2024-05-26T22:56:03.792444+0000","last_fullsized":"2024-05-26T22:56:03.792444+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_clean_scrub_stamp":"2024-05-26T22:56:00.479878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:43:45.548497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:04.618527+0000","last_change":"2024-05-26T22:56:02.220132+0000","last_active":"2024-05-26T22:56:04.618527+0000","last_peered":"2024-05-26T22:56:04.618527+0000","last_clean":"2024-05-26T22:56:04.618527+0000","last_became_active":"2024-05-26T22:56:02.219844+0000","last_became_peered":"2024-05-26T22:56:02.219844+0000","last_unstale":"2024-05-26T22:56:04.618527+0000","last_undegraded":"2024-05-26T22:56:04.618527+0000","last_fullsized":"2024-05-26T22:56:04.618527+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:32:11.698764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"106'2","reported_seq":16,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:04.714236+0000","last_change":"2024-05-26T22:56:01.541924+0000","last_active":"2024-05-26T22:56:04.714236+0000","last_peered":"2024-05-26T22:56:04.714236+0000","last_clean":"2024-05-26T22:56:04.714236+0000","last_became_active":"2024-05-26T22:56:01.541666+0000","last_became_peered":"2024-05-26T22:56:01.541666+0000","last_unstale":"2024-05-26T22:56:04.714236+0000","last_undegraded":"2024-05-26T22:56:04.714236+0000","last_fullsized":"2024-05-26T22:56:04.714236+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_clean_scrub_stamp":"2024-05-26T22:56:00.479878+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:15:22.745045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:04.858419+0000","last_change":"2024-05-26T22:56:02.217119+0000","last_active":"2024-05-26T22:56:04.858419+0000","last_peered":"2024-05-26T22:56:04.858419+0000","last_clean":"2024-05-26T22:56:04.858419+0000","last_became_active":"2024-05-26T22:56:02.215372+0000","last_became_peered":"2024-05-26T22:56:02.215372+0000","last_unstale":"2024-05-26T22:56:04.858419+0000","last_undegraded":"2024-05-26T22:56:04.858419+0000","last_fullsized":"2024-05-26T22:56:04.858419+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:33:06.814045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:03.180154+0000","last_change":"2024-05-26T22:56:01.547356+0000","last_active":"2024-05-26T22:56:03.180154+0000","last_peered":"2024-05-26T22:56:03.180154+0000","last_clean":"2024-05-26T22:56:03.180154+0000","last_became_active":"2024-05-26T22:56:01.546071+0000","last_became_peered":"2024-05-26T22:56:01.546071+0000","last_unstale":"2024-05-26T22:56:03.180154+0000","last_undegraded":"2024-05-26T22:56:03.180154+0000","last_fullsized":"2024-05-26T22:56:03.180154+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_clean_scrub_stamp":"2024-05-26T22:56:00.479878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:45:45.573526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:04.618529+0000","last_change":"2024-05-26T22:56:02.222258+0000","last_active":"2024-05-26T22:56:04.618529+0000","last_peered":"2024-05-26T22:56:04.618529+0000","last_clean":"2024-05-26T22:56:04.618529+0000","last_became_active":"2024-05-26T22:56:02.220689+0000","last_became_peered":"2024-05-26T22:56:02.220689+0000","last_unstale":"2024-05-26T22:56:04.618529+0000","last_undegraded":"2024-05-26T22:56:04.618529+0000","last_fullsized":"2024-05-26T22:56:04.618529+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:33:00.893414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:04.713758+0000","last_change":"2024-05-26T22:56:01.540055+0000","last_active":"2024-05-26T22:56:04.713758+0000","last_peered":"2024-05-26T22:56:04.713758+0000","last_clean":"2024-05-26T22:56:04.713758+0000","last_became_active":"2024-05-26T22:56:01.539794+0000","last_became_peered":"2024-05-26T22:56:01.539794+0000","last_unstale":"2024-05-26T22:56:04.713758+0000","last_undegraded":"2024-05-26T22:56:04.713758+0000","last_fullsized":"2024-05-26T22:56:04.713758+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_clean_scrub_stamp":"2024-05-26T22:56:00.479878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:14:41.448743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:03.500739+0000","last_change":"2024-05-26T22:56:02.214861+0000","last_active":"2024-05-26T22:56:03.500739+0000","last_peered":"2024-05-26T22:56:03.500739+0000","last_clean":"2024-05-26T22:56:03.500739+0000","last_became_active":"2024-05-26T22:56:02.214046+0000","last_became_peered":"2024-05-26T22:56:02.214046+0000","last_unstale":"2024-05-26T22:56:03.500739+0000","last_undegraded":"2024-05-26T22:56:03.500739+0000","last_fullsized":"2024-05-26T22:56:03.500739+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:31:16.025886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:04.858791+0000","last_change":"2024-05-26T22:56:01.528791+0000","last_active":"2024-05-26T22:56:04.858791+0000","last_peered":"2024-05-26T22:56:04.858791+0000","last_clean":"2024-05-26T22:56:04.858791+0000","last_became_active":"2024-05-26T22:56:01.528468+0000","last_became_peered":"2024-05-26T22:56:01.528468+0000","last_unstale":"2024-05-26T22:56:04.858791+0000","last_undegraded":"2024-05-26T22:56:04.858791+0000","last_fullsized":"2024-05-26T22:56:04.858791+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_clean_scrub_stamp":"2024-05-26T22:56:00.479878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:11:52.742481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:03.189612+0000","last_change":"2024-05-26T22:56:02.214278+0000","last_active":"2024-05-26T22:56:03.189612+0000","last_peered":"2024-05-26T22:56:03.189612+0000","last_clean":"2024-05-26T22:56:03.189612+0000","last_became_active":"2024-05-26T22:56:02.213982+0000","last_became_peered":"2024-05-26T22:56:02.213982+0000","last_unstale":"2024-05-26T22:56:03.189612+0000","last_undegraded":"2024-05-26T22:56:03.189612+0000","last_fullsized":"2024-05-26T22:56:03.189612+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T03:33:00.893414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"106'2","reported_seq":16,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:03.501391+0000","last_change":"2024-05-26T22:56:01.537496+0000","last_active":"2024-05-26T22:56:03.501391+0000","last_peered":"2024-05-26T22:56:03.501391+0000","last_clean":"2024-05-26T22:56:03.501391+0000","last_became_active":"2024-05-26T22:56:01.537171+0000","last_became_peered":"2024-05-26T22:56:01.537171+0000","last_unstale":"2024-05-26T22:56:03.501391+0000","last_undegraded":"2024-05-26T22:56:03.501391+0000","last_fullsized":"2024-05-26T22:56:03.501391+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_clean_scrub_stamp":"2024-05-26T22:56:00.479878+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T05:23:32.831739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:04.859322+0000","last_change":"2024-05-26T22:56:02.212367+0000","last_active":"2024-05-26T22:56:04.859322+0000","last_peered":"2024-05-26T22:56:04.859322+0000","last_clean":"2024-05-26T22:56:04.859322+0000","last_became_active":"2024-05-26T22:56:02.211946+0000","last_became_peered":"2024-05-26T22:56:02.211946+0000","last_unstale":"2024-05-26T22:56:04.859322+0000","last_undegraded":"2024-05-26T22:56:04.859322+0000","last_fullsized":"2024-05-26T22:56:04.859322+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:04:27.002049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"107'1","reported_seq":15,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:03.189685+0000","last_change":"2024-05-26T22:56:01.545880+0000","last_active":"2024-05-26T22:56:03.189685+0000","last_peered":"2024-05-26T22:56:03.189685+0000","last_clean":"2024-05-26T22:56:03.189685+0000","last_became_active":"2024-05-26T22:56:01.544569+0000","last_became_peered":"2024-05-26T22:56:01.544569+0000","last_unstale":"2024-05-26T22:56:03.189685+0000","last_undegraded":"2024-05-26T22:56:03.189685+0000","last_fullsized":"2024-05-26T22:56:03.189685+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_clean_scrub_stamp":"2024-05-26T22:56:00.479878+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:13:54.327398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:04.858298+0000","last_change":"2024-05-26T22:56:02.214933+0000","last_active":"2024-05-26T22:56:04.858298+0000","last_peered":"2024-05-26T22:56:04.858298+0000","last_clean":"2024-05-26T22:56:04.858298+0000","last_became_active":"2024-05-26T22:56:02.214621+0000","last_became_peered":"2024-05-26T22:56:02.214621+0000","last_unstale":"2024-05-26T22:56:04.858298+0000","last_undegraded":"2024-05-26T22:56:04.858298+0000","last_fullsized":"2024-05-26T22:56:04.858298+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:35:23.017773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:04.858695+0000","last_change":"2024-05-26T22:56:01.532602+0000","last_active":"2024-05-26T22:56:04.858695+0000","last_peered":"2024-05-26T22:56:04.858695+0000","last_clean":"2024-05-26T22:56:04.858695+0000","last_became_active":"2024-05-26T22:56:01.532262+0000","last_became_peered":"2024-05-26T22:56:01.532262+0000","last_unstale":"2024-05-26T22:56:04.858695+0000","last_undegraded":"2024-05-26T22:56:04.858695+0000","last_fullsized":"2024-05-26T22:56:04.858695+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_clean_scrub_stamp":"2024-05-26T22:56:00.479878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:39:25.912660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:03.188880+0000","last_change":"2024-05-26T22:56:02.205119+0000","last_active":"2024-05-26T22:56:03.188880+0000","last_peered":"2024-05-26T22:56:03.188880+0000","last_clean":"2024-05-26T22:56:03.188880+0000","last_became_active":"2024-05-26T22:56:02.204653+0000","last_became_peered":"2024-05-26T22:56:02.204653+0000","last_unstale":"2024-05-26T22:56:03.188880+0000","last_undegraded":"2024-05-26T22:56:03.188880+0000","last_fullsized":"2024-05-26T22:56:03.188880+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:26:53.366151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:03.501670+0000","last_change":"2024-05-26T22:56:01.542219+0000","last_active":"2024-05-26T22:56:03.501670+0000","last_peered":"2024-05-26T22:56:03.501670+0000","last_clean":"2024-05-26T22:56:03.501670+0000","last_became_active":"2024-05-26T22:56:01.540883+0000","last_became_peered":"2024-05-26T22:56:01.540883+0000","last_unstale":"2024-05-26T22:56:03.501670+0000","last_undegraded":"2024-05-26T22:56:03.501670+0000","last_fullsized":"2024-05-26T22:56:03.501670+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_clean_scrub_stamp":"2024-05-26T22:56:00.479878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:45:22.206973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:03.187074+0000","last_change":"2024-05-26T22:56:02.210259+0000","last_active":"2024-05-26T22:56:03.187074+0000","last_peered":"2024-05-26T22:56:03.187074+0000","last_clean":"2024-05-26T22:56:03.187074+0000","last_became_active":"2024-05-26T22:56:02.209981+0000","last_became_peered":"2024-05-26T22:56:02.209981+0000","last_unstale":"2024-05-26T22:56:03.187074+0000","last_undegraded":"2024-05-26T22:56:03.187074+0000","last_fullsized":"2024-05-26T22:56:03.187074+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:26:53.366151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:03.792461+0000","last_change":"2024-05-26T22:56:01.547965+0000","last_active":"2024-05-26T22:56:03.792461+0000","last_peered":"2024-05-26T22:56:03.792461+0000","last_clean":"2024-05-26T22:56:03.792461+0000","last_became_active":"2024-05-26T22:56:01.547653+0000","last_became_peered":"2024-05-26T22:56:01.547653+0000","last_unstale":"2024-05-26T22:56:03.792461+0000","last_undegraded":"2024-05-26T22:56:03.792461+0000","last_fullsized":"2024-05-26T22:56:03.792461+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_clean_scrub_stamp":"2024-05-26T22:56:00.479878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:15:22.745045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:03.188888+0000","last_change":"2024-05-26T22:56:02.207758+0000","last_active":"2024-05-26T22:56:03.188888+0000","last_peered":"2024-05-26T22:56:03.188888+0000","last_clean":"2024-05-26T22:56:03.188888+0000","last_became_active":"2024-05-26T22:56:02.202470+0000","last_became_peered":"2024-05-26T22:56:02.202470+0000","last_unstale":"2024-05-26T22:56:03.188888+0000","last_undegraded":"2024-05-26T22:56:03.188888+0000","last_fullsized":"2024-05-26T22:56:03.188888+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:51:41.862299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:03.187523+0000","last_change":"2024-05-26T22:56:01.544943+0000","last_active":"2024-05-26T22:56:03.187523+0000","last_peered":"2024-05-26T22:56:03.187523+0000","last_clean":"2024-05-26T22:56:03.187523+0000","last_became_active":"2024-05-26T22:56:01.544654+0000","last_became_peered":"2024-05-26T22:56:01.544654+0000","last_unstale":"2024-05-26T22:56:03.187523+0000","last_undegraded":"2024-05-26T22:56:03.187523+0000","last_fullsized":"2024-05-26T22:56:03.187523+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_clean_scrub_stamp":"2024-05-26T22:56:00.479878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:10:25.633083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:04.618817+0000","last_change":"2024-05-26T22:56:02.222135+0000","last_active":"2024-05-26T22:56:04.618817+0000","last_peered":"2024-05-26T22:56:04.618817+0000","last_clean":"2024-05-26T22:56:04.618817+0000","last_became_active":"2024-05-26T22:56:02.221328+0000","last_became_peered":"2024-05-26T22:56:02.221328+0000","last_unstale":"2024-05-26T22:56:04.618817+0000","last_undegraded":"2024-05-26T22:56:04.618817+0000","last_fullsized":"2024-05-26T22:56:04.618817+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:51:41.862299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:03.180164+0000","last_change":"2024-05-26T22:56:01.542729+0000","last_active":"2024-05-26T22:56:03.180164+0000","last_peered":"2024-05-26T22:56:03.180164+0000","last_clean":"2024-05-26T22:56:03.180164+0000","last_became_active":"2024-05-26T22:56:01.542220+0000","last_became_peered":"2024-05-26T22:56:01.542220+0000","last_unstale":"2024-05-26T22:56:03.180164+0000","last_undegraded":"2024-05-26T22:56:03.180164+0000","last_fullsized":"2024-05-26T22:56:03.180164+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_clean_scrub_stamp":"2024-05-26T22:56:00.479878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T04:27:47.414115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:04.858103+0000","last_change":"2024-05-26T22:56:02.207968+0000","last_active":"2024-05-26T22:56:04.858103+0000","last_peered":"2024-05-26T22:56:04.858103+0000","last_clean":"2024-05-26T22:56:04.858103+0000","last_became_active":"2024-05-26T22:56:02.207662+0000","last_became_peered":"2024-05-26T22:56:02.207662+0000","last_unstale":"2024-05-26T22:56:04.858103+0000","last_undegraded":"2024-05-26T22:56:04.858103+0000","last_fullsized":"2024-05-26T22:56:04.858103+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T06:49:15.779421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:04.858717+0000","last_change":"2024-05-26T22:56:01.528583+0000","last_active":"2024-05-26T22:56:04.858717+0000","last_peered":"2024-05-26T22:56:04.858717+0000","last_clean":"2024-05-26T22:56:04.858717+0000","last_became_active":"2024-05-26T22:56:01.528313+0000","last_became_peered":"2024-05-26T22:56:01.528313+0000","last_unstale":"2024-05-26T22:56:04.858717+0000","last_undegraded":"2024-05-26T22:56:04.858717+0000","last_fullsized":"2024-05-26T22:56:04.858717+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_clean_scrub_stamp":"2024-05-26T22:56:00.479878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-27T23:00:16.700395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:03.500260+0000","last_change":"2024-05-26T22:56:02.216541+0000","last_active":"2024-05-26T22:56:03.500260+0000","last_peered":"2024-05-26T22:56:03.500260+0000","last_clean":"2024-05-26T22:56:03.500260+0000","last_became_active":"2024-05-26T22:56:02.216205+0000","last_became_peered":"2024-05-26T22:56:02.216205+0000","last_unstale":"2024-05-26T22:56:03.500260+0000","last_undegraded":"2024-05-26T22:56:03.500260+0000","last_fullsized":"2024-05-26T22:56:03.500260+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:01.498116+0000","last_clean_scrub_stamp":"2024-05-26T22:56:01.498116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:11:53.760719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:03.792626+0000","last_change":"2024-05-26T22:56:01.547906+0000","last_active":"2024-05-26T22:56:03.792626+0000","last_peered":"2024-05-26T22:56:03.792626+0000","last_clean":"2024-05-26T22:56:03.792626+0000","last_became_active":"2024-05-26T22:56:01.547614+0000","last_became_peered":"2024-05-26T22:56:01.547614+0000","last_unstale":"2024-05-26T22:56:03.792626+0000","last_undegraded":"2024-05-26T22:56:03.792626+0000","last_fullsized":"2024-05-26T22:56:03.792626+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:56:00.479878+0000","last_clean_scrub_stamp":"2024-05-26T22:56:00.479878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T01:15:20.201446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":331,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-05-26T22:56:04.618984+0000","last_change":"2024-05-26T22:48:12.068917+0000","last_active":"2024-05-26T22:56:04.618984+0000","last_peered":"2024-05-26T22:56:04.618984+0000","last_clean":"2024-05-26T22:56:04.618984+0000","last_became_active":"2024-05-26T22:48:12.065934+0000","last_became_peered":"2024-05-26T22:48:12.065934+0000","last_unstale":"2024-05-26T22:56:04.618984+0000","last_undegraded":"2024-05-26T22:56:04.618984+0000","last_fullsized":"2024-05-26T22:56:04.618984+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:10.628122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:10.628122+0000","last_clean_scrub_stamp":"2024-05-26T22:48:10.628122+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:14:58.895911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574946,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33072,"kb_used_data":1888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338768,"statfs":{"total":96636764160,"available":96602898432,"internally_reserved":0,"allocated":1933312,"data_stored":1171578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574946,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32484,"kb_used_data":1304,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339356,"statfs":{"total":96636764160,"available":96603500544,"internally_reserved":0,"allocated":1335296,"data_stored":580664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574947,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32488,"kb_used_data":1308,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339352,"statfs":{"total":96636764160,"available":96603496448,"internally_reserved":0,"allocated":1339392,"data_stored":581198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574946,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32480,"kb_used_data":1300,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339360,"statfs":{"total":96636764160,"available":96603504640,"internally_reserved":0,"allocated":1331200,"data_stored":580594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574946,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36660,"kb_used_data":1316,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335180,"statfs":{"total":96636764160,"available":96599224320,"internally_reserved":0,"allocated":1347584,"data_stored":582276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574946,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32480,"kb_used_data":1300,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339360,"statfs":{"total":96636764160,"available":96603504640,"internally_reserved":0,"allocated":1331200,"data_stored":580594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574946,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36648,"kb_used_data":1308,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335192,"statfs":{"total":96636764160,"available":96599236608,"internally_reserved":0,"allocated":1339392,"data_stored":580718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574946,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37240,"kb_used_data":1900,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334600,"statfs":{"total":96636764160,"available":96598630400,"internally_reserved":0,"allocated":1945600,"data_stored":1172686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-26T22:56:07.456 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-26T22:56:07.456 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-26T22:56:07.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.702+0000 7f04c8200640 1 Processor -- start 2024-05-26T22:56:07.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.702+0000 7f04c8200640 1 -- start start 2024-05-26T22:56:07.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.704+0000 7f04c8200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04c0155770 0x7f04c015b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:07.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.704+0000 7f04c8200640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f04c006e440 0x7f04c00725a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:07.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.704+0000 7f04c8200640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04c015c1c0 con 0x7f04c015bc40 2024-05-26T22:56:07.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.704+0000 7f04c8200640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f04c015c4a0 con 0x7f04c006e440 2024-05-26T22:56:07.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.704+0000 7f04c8200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f04c0071da0 con 0x7f04c0155770 2024-05-26T22:56:07.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.704+0000 7f04c4e00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f04c015bc40 0x7f04c006dd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:45796/0 (socket says 172.21.15.131:45796) 2024-05-26T22:56:07.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.704+0000 7f04c4e00640 1 -- 172.21.15.131:0/914997406 learned_addr learned my addr 172.21.15.131:0/914997406 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:56:07.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.705+0000 7f04c5800640 1 --2- 172.21.15.131:0/914997406 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04c0155770 0x7f04c015b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:07.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.705+0000 7f04c6200640 1 --2- 172.21.15.131:0/914997406 >> v2:172.21.15.187:3300/0 conn(0x7f04c006e440 0x7f04c00725a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:07.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.705+0000 7f04bfe00640 1 -- 172.21.15.131:0/914997406 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1867010559 0 0) 0x7f04c015c1c0 con 0x7f04c015bc40 2024-05-26T22:56:07.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.705+0000 7f04bfe00640 1 -- 172.21.15.131:0/914997406 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f04a8003520 con 0x7f04c015bc40 2024-05-26T22:56:07.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.705+0000 7f04bfe00640 1 -- 172.21.15.131:0/914997406 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1864222301 0 0) 0x7f04a8003520 con 0x7f04c015bc40 2024-05-26T22:56:07.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.705+0000 7f04bfe00640 1 -- 172.21.15.131:0/914997406 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04c0155770 msgr2=0x7f04c015b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:07.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.705+0000 7f04bfe00640 1 --2- 172.21.15.131:0/914997406 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04c0155770 0x7f04c015b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:07.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.705+0000 7f04bfe00640 1 -- 172.21.15.131:0/914997406 >> v2:172.21.15.187:3300/0 conn(0x7f04c006e440 msgr2=0x7f04c00725a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:07.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.705+0000 7f04bfe00640 1 --2- 172.21.15.131:0/914997406 >> v2:172.21.15.187:3300/0 conn(0x7f04c006e440 0x7f04c00725a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:07.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.705+0000 7f04bfe00640 1 -- 172.21.15.131:0/914997406 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04c0072080 con 0x7f04c015bc40 2024-05-26T22:56:07.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.705+0000 7f04bfe00640 1 -- 172.21.15.131:0/914997406 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7f04ac001670 con 0x7f04c015bc40 2024-05-26T22:56:07.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.705+0000 7f04bfe00640 1 -- 172.21.15.131:0/914997406 >> v1:172.21.15.131:6789/0 conn(0x7f04c015bc40 legacy=0x7f04c006dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:07.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.705+0000 7f04bfe00640 1 --2- 172.21.15.131:0/914997406 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04c0155770 0x7f04a80039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:07.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.705+0000 7f04bfe00640 1 --2- 172.21.15.131:0/914997406 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04a8003f30 0x7f04a8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:07.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.705+0000 7f04bfe00640 1 --2- 172.21.15.131:0/914997406 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04a8004d40 0x7f04a80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:07.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.705+0000 7f04bfe00640 1 -- 172.21.15.131:0/914997406 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f04c015c4a0 con 0x7f04a8003f30 2024-05-26T22:56:07.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.705+0000 7f04bfe00640 1 -- 172.21.15.131:0/914997406 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f04c0071da0 con 0x7f04a8004d40 2024-05-26T22:56:07.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.705+0000 7f04bfe00640 1 -- 172.21.15.131:0/914997406 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f04a800dcb0 con 0x7f04c0155770 2024-05-26T22:56:07.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.705+0000 7f04c6200640 1 --2- 172.21.15.131:0/914997406 >> v2:172.21.15.187:3300/0 conn(0x7f04c006e440 0x7f04c00725a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-26T22:56:07.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.706+0000 7f04c4e00640 1 --2- 172.21.15.131:0/914997406 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04a8003f30 0x7f04a8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:07.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.706+0000 7f04c5800640 1 --2- 172.21.15.131:0/914997406 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04c0155770 0x7f04a80039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:07.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.706+0000 7f04c4e00640 1 -- 172.21.15.131:0/914997406 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04c0155770 msgr2=0x7f04a80039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:07.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.706+0000 7f04c6200640 1 --2- 172.21.15.131:0/914997406 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04a8004d40 0x7f04a80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:07.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.706+0000 7f04c4e00640 1 --2- 172.21.15.131:0/914997406 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04c0155770 0x7f04a80039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:07.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.706+0000 7f04c4e00640 1 -- 172.21.15.131:0/914997406 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04a8004d40 msgr2=0x7f04a80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:07.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.706+0000 7f04c4e00640 1 --2- 172.21.15.131:0/914997406 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04a8004d40 0x7f04a80051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:07.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.706+0000 7f04c4e00640 1 -- 172.21.15.131:0/914997406 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04a8003520 con 0x7f04a8003f30 2024-05-26T22:56:07.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.707+0000 7f04c4e00640 1 --2- 172.21.15.131:0/914997406 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04a8003f30 0x7f04a8004800 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f04ac00c240 tx=0x7f04a8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3f5e3e0b8b37c5c server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:07.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.707+0000 7f04bfe00640 1 -- 172.21.15.131:0/914997406 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04ac02e070 con 0x7f04a8003f30 2024-05-26T22:56:07.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.707+0000 7f04bfe00640 1 -- 172.21.15.131:0/914997406 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04ac005af0 con 0x7f04a8003f30 2024-05-26T22:56:07.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.708+0000 7f04bfe00640 1 -- 172.21.15.131:0/914997406 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04ac007d30 con 0x7f04a8003f30 2024-05-26T22:56:07.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.708+0000 7f04c8200640 1 -- 172.21.15.131:0/914997406 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04a8003f30 msgr2=0x7f04a8004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:07.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.708+0000 7f04c8200640 1 --2- 172.21.15.131:0/914997406 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04a8003f30 0x7f04a8004800 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f04ac00c240 tx=0x7f04a8005830 comp rx=0 tx=0).stop 2024-05-26T22:56:07.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.708+0000 7f04c6200640 1 -- 172.21.15.131:0/914997406 reap_dead start 2024-05-26T22:56:07.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.708+0000 7f04c8200640 1 -- 172.21.15.131:0/914997406 shutdown_connections 2024-05-26T22:56:07.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.708+0000 7f04c8200640 1 -- 172.21.15.131:0/914997406 >> 172.21.15.131:0/914997406 conn(0x7f04c006bc90 msgr2=0x7f04c00b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:56:07.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.708+0000 7f04c8200640 1 -- 172.21.15.131:0/914997406 shutdown_connections 2024-05-26T22:56:07.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.708+0000 7f04c8200640 1 -- 172.21.15.131:0/914997406 wait complete. 2024-05-26T22:56:07.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.710+0000 7f04c8200640 1 Processor -- start 2024-05-26T22:56:07.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.710+0000 7f04c8200640 1 -- start start 2024-05-26T22:56:07.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.711+0000 7f04c8200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04a8003f30 0x7f04c0151750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:07.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.711+0000 7f04c8200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04c014fd10 0x7f04c014b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:07.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.711+0000 7f04c8200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04c00b4ef0 0x7f04c00b53a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:07.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.711+0000 7f04c8200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f04c009de00 con 0x7f04c014fd10 2024-05-26T22:56:07.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.711+0000 7f04c8200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f04c015c8c0 con 0x7f04a8003f30 2024-05-26T22:56:07.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.711+0000 7f04c8200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f04c009df80 con 0x7f04c00b4ef0 2024-05-26T22:56:07.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.711+0000 7f04c4e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04c014fd10 0x7f04c014b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:07.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.711+0000 7f04c4e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04c014fd10 0x7f04c014b5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54478/0 (socket says 172.21.15.131:54478) 2024-05-26T22:56:07.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.711+0000 7f04c4e00640 1 -- 172.21.15.131:0/27153455 learned_addr learned my addr 172.21.15.131:0/27153455 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:56:07.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.712+0000 7f04c6200640 1 --2- 172.21.15.131:0/27153455 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04c00b4ef0 0x7f04c00b53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:07.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.712+0000 7f04c5800640 1 --2- 172.21.15.131:0/27153455 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04a8003f30 0x7f04c0151750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:07.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.712+0000 7f04c4e00640 1 -- 172.21.15.131:0/27153455 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04c00b4ef0 msgr2=0x7f04c00b53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:07.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.712+0000 7f04c4e00640 1 --2- 172.21.15.131:0/27153455 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04c00b4ef0 0x7f04c00b53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:07.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.712+0000 7f04c4e00640 1 -- 172.21.15.131:0/27153455 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04a8003f30 msgr2=0x7f04c0151750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:07.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.712+0000 7f04c4e00640 1 --2- 172.21.15.131:0/27153455 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04a8003f30 0x7f04c0151750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:07.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.712+0000 7f04c4e00640 1 -- 172.21.15.131:0/27153455 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04c0151ee0 con 0x7f04c014fd10 2024-05-26T22:56:07.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.712+0000 7f04c5800640 1 --2- 172.21.15.131:0/27153455 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04a8003f30 0x7f04c0151750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:56:07.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.712+0000 7f04c4e00640 1 --2- 172.21.15.131:0/27153455 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04c014fd10 0x7f04c014b5e0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f04ac00c360 tx=0x7f04c0072250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:07.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.713+0000 7f04bd600640 1 -- 172.21.15.131:0/27153455 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04ac042070 con 0x7f04c014fd10 2024-05-26T22:56:07.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.713+0000 7f04bd600640 1 -- 172.21.15.131:0/27153455 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04ac007480 con 0x7f04c014fd10 2024-05-26T22:56:07.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.713+0000 7f04bd600640 1 -- 172.21.15.131:0/27153455 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04ac001c70 con 0x7f04c014fd10 2024-05-26T22:56:07.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.713+0000 7f04c8200640 1 -- 172.21.15.131:0/27153455 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04c01521c0 con 0x7f04c014fd10 2024-05-26T22:56:07.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.713+0000 7f04c8200640 1 -- 172.21.15.131:0/27153455 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04c0152780 con 0x7f04c014fd10 2024-05-26T22:56:07.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.714+0000 7f04c8200640 1 -- 172.21.15.131:0/27153455 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04c0074e90 con 0x7f04c014fd10 2024-05-26T22:56:07.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.715+0000 7f04bd600640 1 -- 172.21.15.131:0/27153455 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f04ac045070 con 0x7f04c014fd10 2024-05-26T22:56:07.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.717+0000 7f04bd600640 1 --2- 172.21.15.131:0/27153455 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f04a006be30 0x7f04a006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:07.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.717+0000 7f04bd600640 1 -- 172.21.15.131:0/27153455 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5709+0+0 (secure 0 0 0) 0x7f04ac0ab900 con 0x7f04c014fd10 2024-05-26T22:56:07.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.717+0000 7f04c5800640 1 --2- 172.21.15.131:0/27153455 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f04a006be30 0x7f04a006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:07.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.718+0000 7f04c5800640 1 --2- 172.21.15.131:0/27153455 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f04a006be30 0x7f04a006e2f0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f04c0154190 tx=0x7f04b4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:07.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:07.722+0000 7f04bd600640 1 -- 172.21.15.131:0/27153455 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f04ac005ac0 con 0x7f04c014fd10 2024-05-26T22:56:08.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:08.003+0000 7f04c8200640 1 -- 172.21.15.131:0/27153455 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f04c00695c0 con 0x7f04c014fd10 2024-05-26T22:56:08.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:08.008+0000 7f04bd600640 1 -- 172.21.15.131:0/27153455 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v20) ==== 190+0+147 (secure 0 0 0) 0x7f04ac0ab070 con 0x7f04c014fd10 2024-05-26T22:56:08.008 INFO:teuthology.orchestra.run.smithi131.stdout:[client.0] 2024-05-26T22:56:08.008 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQD2u1NmSGVWChAAH/wq3iaUs6ZELRhLKhmURQ== 2024-05-26T22:56:08.008 INFO:teuthology.orchestra.run.smithi131.stdout: caps mds = "allow" 2024-05-26T22:56:08.008 INFO:teuthology.orchestra.run.smithi131.stdout: caps mon = "allow r" 2024-05-26T22:56:08.008 INFO:teuthology.orchestra.run.smithi131.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-05-26T22:56:08.009 INFO:teuthology.orchestra.run.smithi131.stderr:updated caps for client.0 2024-05-26T22:56:08.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:08.011+0000 7f04c8200640 1 -- 172.21.15.131:0/27153455 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f04a006be30 msgr2=0x7f04a006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:08.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:08.011+0000 7f04c8200640 1 --2- 172.21.15.131:0/27153455 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f04a006be30 0x7f04a006e2f0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f04c0154190 tx=0x7f04b4008040 comp rx=0 tx=0).stop 2024-05-26T22:56:08.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:08.011+0000 7f04c8200640 1 -- 172.21.15.131:0/27153455 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04c014fd10 msgr2=0x7f04c014b5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:08.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:08.011+0000 7f04c8200640 1 --2- 172.21.15.131:0/27153455 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04c014fd10 0x7f04c014b5e0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f04ac00c360 tx=0x7f04c0072250 comp rx=0 tx=0).stop 2024-05-26T22:56:08.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:08.012+0000 7f04c8200640 1 -- 172.21.15.131:0/27153455 shutdown_connections 2024-05-26T22:56:08.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:08.012+0000 7f04c8200640 1 --2- 172.21.15.131:0/27153455 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04c00b4ef0 0x7f04c00b53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:08.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:08.012+0000 7f04c8200640 1 --2- 172.21.15.131:0/27153455 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f04a006be30 0x7f04a006e2f0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:08.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:08.012+0000 7f04c8200640 1 --2- 172.21.15.131:0/27153455 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04c014fd10 0x7f04c014b5e0 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:08.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:08.012+0000 7f04c8200640 1 --2- 172.21.15.131:0/27153455 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04a8003f30 0x7f04c0151750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:08.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:08.012+0000 7f04c8200640 1 -- 172.21.15.131:0/27153455 >> 172.21.15.131:0/27153455 conn(0x7f04c006bc90 msgr2=0x7f04c00ae960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:56:08.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:08.012+0000 7f04c8200640 1 -- 172.21.15.131:0/27153455 shutdown_connections 2024-05-26T22:56:08.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:08.012+0000 7f04c8200640 1 -- 172.21.15.131:0/27153455 wait complete. 2024-05-26T22:56:08.030 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-26T22:56:08.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:08.226+0000 7f1b27400640 1 Processor -- start 2024-05-26T22:56:08.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:08.226+0000 7f1b27400640 1 -- start start 2024-05-26T22:56:08.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:08.227+0000 7f1b27400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1b2006ee20 0x7f1b200b3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:08.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:08.227+0000 7f1b27400640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f1b200ad250 0x7f1b200af790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:08.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:08.228+0000 7f1b27400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f1b2006f200 0x7f1b201562c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:08.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:08.228+0000 7f1b27400640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f1b2015cb20 con 0x7f1b2006f200 2024-05-26T22:56:08.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:08.228+0000 7f1b27400640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f1b2015ce00 con 0x7f1b200ad250 2024-05-26T22:56:08.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:08.228+0000 7f1b27400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1b2015d0e0 con 0x7f1b2006ee20 2024-05-26T22:56:08.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:08.228+0000 7f1b25400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f1b2006f200 0x7f1b201562c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:08.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:08.228+0000 7f1b25400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f1b2006f200 0x7f1b201562c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54486/0 (socket says 172.21.15.131:54486) 2024-05-26T22:56:08.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:08.228+0000 7f1b25400640 1 -- 172.21.15.131:0/750836754 learned_addr learned my addr 172.21.15.131:0/750836754 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:56:08.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:08.228+0000 7f1b24a00640 1 --2- 172.21.15.131:0/750836754 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1b2006ee20 0x7f1b200b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:08.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:08.228+0000 7f1b1fe00640 1 --2- 172.21.15.131:0/750836754 >> v2:172.21.15.187:3300/0 conn(0x7f1b200ad250 0x7f1b200af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:08.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:08.228+0000 7f1b25400640 1 -- 172.21.15.131:0/750836754 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1b2006ee20 msgr2=0x7f1b200b3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:08.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:08.228+0000 7f1b25400640 1 --2- 172.21.15.131:0/750836754 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1b2006ee20 0x7f1b200b3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:08.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:08.228+0000 7f1b25400640 1 -- 172.21.15.131:0/750836754 >> v2:172.21.15.187:3300/0 conn(0x7f1b200ad250 msgr2=0x7f1b200af790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:08.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:08.228+0000 7f1b25400640 1 --2- 172.21.15.131:0/750836754 >> v2:172.21.15.187:3300/0 conn(0x7f1b200ad250 0x7f1b200af790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:08.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:08.228+0000 7f1b25400640 1 -- 172.21.15.131:0/750836754 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b2015d3c0 con 0x7f1b2006f200 2024-05-26T22:56:08.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:08.229+0000 7f1b25400640 1 --2- 172.21.15.131:0/750836754 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1b2006f200 0x7f1b201562c0 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f1b2006c250 tx=0x7f1b1802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17e397b55e6c151f server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:08.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:08.230+0000 7f1b1f400640 1 -- 172.21.15.131:0/750836754 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b18036070 con 0x7f1b2006f200 2024-05-26T22:56:08.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:08.230+0000 7f1b1f400640 1 -- 172.21.15.131:0/750836754 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b18031600 con 0x7f1b2006f200 2024-05-26T22:56:08.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:08.230+0000 7f1b1f400640 1 -- 172.21.15.131:0/750836754 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b180319d0 con 0x7f1b2006f200 2024-05-26T22:56:08.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:08.230+0000 7f1b27400640 1 -- 172.21.15.131:0/750836754 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1b2006f200 msgr2=0x7f1b201562c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:08.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:08.230+0000 7f1b27400640 1 --2- 172.21.15.131:0/750836754 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1b2006f200 0x7f1b201562c0 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f1b2006c250 tx=0x7f1b1802a6a0 comp rx=0 tx=0).stop 2024-05-26T22:56:08.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:08.230+0000 7f1b27400640 1 -- 172.21.15.131:0/750836754 shutdown_connections 2024-05-26T22:56:08.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:08.230+0000 7f1b27400640 1 --2- 172.21.15.131:0/750836754 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1b2006f200 0x7f1b201562c0 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:08.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:08.230+0000 7f1b27400640 1 --2- 172.21.15.131:0/750836754 >> v2:172.21.15.187:3300/0 conn(0x7f1b200ad250 0x7f1b200af790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:08.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:08.230+0000 7f1b27400640 1 --2- 172.21.15.131:0/750836754 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1b2006ee20 0x7f1b200b3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:08.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:08.230+0000 7f1b27400640 1 -- 172.21.15.131:0/750836754 >> 172.21.15.131:0/750836754 conn(0x7f1b200b1e90 msgr2=0x7f1b200b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:56:08.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:08.230+0000 7f1b27400640 1 -- 172.21.15.131:0/750836754 shutdown_connections 2024-05-26T22:56:08.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:08.231+0000 7f1b27400640 1 -- 172.21.15.131:0/750836754 wait complete. 2024-05-26T22:56:08.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:08.232+0000 7f1b27400640 1 Processor -- start 2024-05-26T22:56:08.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:08.232+0000 7f1b27400640 1 -- start start 2024-05-26T22:56:08.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:08.233+0000 7f1b27400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1b2006ee20 0x7f1b201498b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:08.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:08.233+0000 7f1b27400640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1b2006f200 0x7f1b2009ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:08.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:08.233+0000 7f1b27400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1b200ad250 0x7f1b200b1330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:08.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:08.233+0000 7f1b27400640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f1b2015c900 con 0x7f1b2006ee20 2024-05-26T22:56:08.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:08.233+0000 7f1b27400640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1b2015c780 con 0x7f1b2006f200 2024-05-26T22:56:08.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:08.233+0000 7f1b27400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1b20156800 con 0x7f1b200ad250 2024-05-26T22:56:08.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:08.233+0000 7f1b24a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1b2006ee20 0x7f1b201498b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:08.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:08.233+0000 7f1b24a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1b2006ee20 0x7f1b201498b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54494/0 (socket says 172.21.15.131:54494) 2024-05-26T22:56:08.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:08.233+0000 7f1b24a00640 1 -- 172.21.15.131:0/4055331687 learned_addr learned my addr 172.21.15.131:0/4055331687 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:56:08.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:08.234+0000 7f1b24a00640 1 -- 172.21.15.131:0/4055331687 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1b200ad250 msgr2=0x7f1b200b1330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:08.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:08.234+0000 7f1b24a00640 1 --2- 172.21.15.131:0/4055331687 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1b200ad250 0x7f1b200b1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:08.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:08.234+0000 7f1b24a00640 1 -- 172.21.15.131:0/4055331687 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1b2006f200 msgr2=0x7f1b2009ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:08.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:08.234+0000 7f1b24a00640 1 --2- 172.21.15.131:0/4055331687 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1b2006f200 0x7f1b2009ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:08.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:08.234+0000 7f1b24a00640 1 -- 172.21.15.131:0/4055331687 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b2009e130 con 0x7f1b2006ee20 2024-05-26T22:56:08.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:08.234+0000 7f1b24a00640 1 --2- 172.21.15.131:0/4055331687 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1b2006ee20 0x7f1b201498b0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f1b1400d800 tx=0x7f1b1400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:08.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:08.234+0000 7f1b1cc00640 1 -- 172.21.15.131:0/4055331687 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b1400b6c0 con 0x7f1b2006ee20 2024-05-26T22:56:08.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:08.234+0000 7f1b1cc00640 1 -- 172.21.15.131:0/4055331687 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b14019070 con 0x7f1b2006ee20 2024-05-26T22:56:08.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:08.234+0000 7f1b27400640 1 -- 172.21.15.131:0/4055331687 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b2009d5b0 con 0x7f1b2006ee20 2024-05-26T22:56:08.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:08.234+0000 7f1b1cc00640 1 -- 172.21.15.131:0/4055331687 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b1400ba80 con 0x7f1b2006ee20 2024-05-26T22:56:08.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:08.235+0000 7f1b27400640 1 -- 172.21.15.131:0/4055331687 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b2009db70 con 0x7f1b2006ee20 2024-05-26T22:56:08.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:08.236+0000 7f1b27400640 1 -- 172.21.15.131:0/4055331687 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b200694e0 con 0x7f1b2006ee20 2024-05-26T22:56:08.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:08.237+0000 7f1b1cc00640 1 -- 172.21.15.131:0/4055331687 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1b1400bc20 con 0x7f1b2006ee20 2024-05-26T22:56:08.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:08.237+0000 7f1b1cc00640 1 --2- 172.21.15.131:0/4055331687 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f1b0006be60 0x7f1b0006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:08.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:08.238+0000 7f1b1cc00640 1 -- 172.21.15.131:0/4055331687 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5709+0+0 (secure 0 0 0) 0x7f1b1408b2e0 con 0x7f1b2006ee20 2024-05-26T22:56:08.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:08.238+0000 7f1b1fe00640 1 --2- 172.21.15.131:0/4055331687 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f1b0006be60 0x7f1b0006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:08.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:08.238+0000 7f1b1fe00640 1 --2- 172.21.15.131:0/4055331687 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f1b0006be60 0x7f1b0006e320 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f1b200b39c0 tx=0x7f1b0c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:08.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:08.242+0000 7f1b1cc00640 1 -- 172.21.15.131:0/4055331687 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f1b1401c360 con 0x7f1b2006ee20 2024-05-26T22:56:08.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:08.459+0000 7f1b27400640 1 -- 172.21.15.131:0/4055331687 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1b200b2520 con 0x7f1b2006ee20 2024-05-26T22:56:08.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:08.461+0000 7f1b1cc00640 1 -- 172.21.15.131:0/4055331687 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 110 v110) ==== 95+0+5201 (secure 0 0 0) 0x7f1b14059c80 con 0x7f1b2006ee20 2024-05-26T22:56:08.460 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:56:08.460 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":110,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5959,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/2949813640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":2949813640},{"type":"v1","addr":"172.21.15.187:6833","nonce":2949813640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104},{"gid":5968,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/2594604065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2594604065},{"type":"v1","addr":"172.21.15.187:6835","nonce":2594604065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":105},{"gid":8307,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6836/2710427205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":2710427205},{"type":"v1","addr":"172.21.15.131:6836","nonce":2710427205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":103}],"filesystems":[{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-26T22:56:01.980221+0000","modified":"2024-05-26T22:56:05.022938+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8310},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8310":{"gid":8310,"name":"c","rank":0,"incarnation":106,"state":"up:active","state_seq":5,"addr":"172.21.15.131:6837/2725520331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6835","nonce":2725520331},{"type":"v1","addr":"172.21.15.131:6837","nonce":2725520331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8310,"qdb_cluster":[8310]},"id":8}]} 2024-05-26T22:56:08.461 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 110 2024-05-26T22:56:08.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:08.463+0000 7f1b27400640 1 -- 172.21.15.131:0/4055331687 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f1b0006be60 msgr2=0x7f1b0006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:08.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:08.463+0000 7f1b27400640 1 --2- 172.21.15.131:0/4055331687 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f1b0006be60 0x7f1b0006e320 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f1b200b39c0 tx=0x7f1b0c008040 comp rx=0 tx=0).stop 2024-05-26T22:56:08.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:08.463+0000 7f1b27400640 1 -- 172.21.15.131:0/4055331687 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1b2006ee20 msgr2=0x7f1b201498b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:08.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:08.463+0000 7f1b27400640 1 --2- 172.21.15.131:0/4055331687 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1b2006ee20 0x7f1b201498b0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f1b1400d800 tx=0x7f1b1400dcc0 comp rx=0 tx=0).stop 2024-05-26T22:56:08.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:08.463+0000 7f1b27400640 1 -- 172.21.15.131:0/4055331687 shutdown_connections 2024-05-26T22:56:08.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:08.463+0000 7f1b27400640 1 --2- 172.21.15.131:0/4055331687 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1b200ad250 0x7f1b200b1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:08.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:08.463+0000 7f1b27400640 1 --2- 172.21.15.131:0/4055331687 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f1b0006be60 0x7f1b0006e320 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:08.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:08.463+0000 7f1b27400640 1 --2- 172.21.15.131:0/4055331687 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1b2006f200 0x7f1b2009ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:08.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:08.463+0000 7f1b27400640 1 --2- 172.21.15.131:0/4055331687 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1b2006ee20 0x7f1b201498b0 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:08.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:08.463+0000 7f1b27400640 1 -- 172.21.15.131:0/4055331687 >> 172.21.15.131:0/4055331687 conn(0x7f1b200b1e90 msgr2=0x7f1b2015e7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:56:08.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:08.464+0000 7f1b27400640 1 -- 172.21.15.131:0/4055331687 shutdown_connections 2024-05-26T22:56:08.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:08.464+0000 7f1b27400640 1 -- 172.21.15.131:0/4055331687 wait complete. 2024-05-26T22:56:08.884 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 110, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-26T22:56:01.980221+0000', 'modified': '2024-05-26T22:56:05.022938+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8310}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8310': {'gid': 8310, 'name': 'c', 'rank': 0, 'incarnation': 106, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.131:6837/2725520331', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6835', 'nonce': 2725520331}, {'type': 'v1', 'addr': '172.21.15.131:6837', 'nonce': 2725520331}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8310, 'qdb_cluster': [8310]} 2024-05-26T22:56:08.885 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-26T22:56:08.885 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-05-26T22:56:09.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.093+0000 7f385be00640 1 Processor -- start 2024-05-26T22:56:09.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.093+0000 7f385be00640 1 -- start start 2024-05-26T22:56:09.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.095+0000 7f385be00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f385c15b730 0x7f385c15bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:09.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.095+0000 7f385be00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f385c1400c0 0x7f385c1404f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:09.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.095+0000 7f385be00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f385c13e820 con 0x7f385c06d250 2024-05-26T22:56:09.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.095+0000 7f385be00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f385c071260 con 0x7f385c15b730 2024-05-26T22:56:09.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.095+0000 7f385be00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f385c15c4a0 con 0x7f385c1400c0 2024-05-26T22:56:09.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.095+0000 7f385aa00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f385c06d250 0x7f385c06d630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:45812/0 (socket says 172.21.15.131:45812) 2024-05-26T22:56:09.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.095+0000 7f385aa00640 1 -- 172.21.15.131:0/3639725284 learned_addr learned my addr 172.21.15.131:0/3639725284 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:56:09.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.096+0000 7f385b400640 1 --2- 172.21.15.131:0/3639725284 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f385c1400c0 0x7f385c1404f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:09.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.096+0000 7f385a000640 1 -- 172.21.15.131:0/3639725284 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 121269647 0 0) 0x7f385c13e820 con 0x7f385c06d250 2024-05-26T22:56:09.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.096+0000 7f385a000640 1 -- 172.21.15.131:0/3639725284 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3840003520 con 0x7f385c06d250 2024-05-26T22:56:09.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.096+0000 7f3853e00640 1 --2- 172.21.15.131:0/3639725284 >> v2:172.21.15.187:3300/0 conn(0x7f385c15b730 0x7f385c15bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:09.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.096+0000 7f385a000640 1 -- 172.21.15.131:0/3639725284 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 818041091 0 0) 0x7f3840003520 con 0x7f385c06d250 2024-05-26T22:56:09.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.096+0000 7f385a000640 1 -- 172.21.15.131:0/3639725284 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f385c1400c0 msgr2=0x7f385c1404f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:09.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.096+0000 7f385a000640 1 --2- 172.21.15.131:0/3639725284 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f385c1400c0 0x7f385c1404f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:09.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.096+0000 7f385a000640 1 -- 172.21.15.131:0/3639725284 >> v2:172.21.15.187:3300/0 conn(0x7f385c15b730 msgr2=0x7f385c15bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:09.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.096+0000 7f385a000640 1 --2- 172.21.15.131:0/3639725284 >> v2:172.21.15.187:3300/0 conn(0x7f385c15b730 0x7f385c15bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:09.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.096+0000 7f385a000640 1 -- 172.21.15.131:0/3639725284 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f385c0714b0 con 0x7f385c06d250 2024-05-26T22:56:09.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.096+0000 7f385a000640 1 -- 172.21.15.131:0/3639725284 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7f384c001670 con 0x7f385c06d250 2024-05-26T22:56:09.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.096+0000 7f385a000640 1 -- 172.21.15.131:0/3639725284 >> v1:172.21.15.131:6789/0 conn(0x7f385c06d250 legacy=0x7f385c06d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:09.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.096+0000 7f385a000640 1 --2- 172.21.15.131:0/3639725284 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f385c1400c0 0x7f38400039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:09.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.096+0000 7f385a000640 1 --2- 172.21.15.131:0/3639725284 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3840003f80 0x7f3840004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:09.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.096+0000 7f385a000640 1 --2- 172.21.15.131:0/3639725284 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3840004d40 0x7f38400051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:09.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.096+0000 7f385a000640 1 -- 172.21.15.131:0/3639725284 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f385c071260 con 0x7f3840003f80 2024-05-26T22:56:09.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.096+0000 7f385a000640 1 -- 172.21.15.131:0/3639725284 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f385c15c4a0 con 0x7f3840004d40 2024-05-26T22:56:09.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.096+0000 7f385a000640 1 -- 172.21.15.131:0/3639725284 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f384000dc80 con 0x7f385c1400c0 2024-05-26T22:56:09.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.097+0000 7f3853e00640 1 --2- 172.21.15.131:0/3639725284 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3840003f80 0x7f3840004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:09.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.097+0000 7f385aa00640 1 --2- 172.21.15.131:0/3639725284 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f385c1400c0 0x7f38400039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:09.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.097+0000 7f3853e00640 1 -- 172.21.15.131:0/3639725284 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f385c1400c0 msgr2=0x7f38400039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:09.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.097+0000 7f3853e00640 1 --2- 172.21.15.131:0/3639725284 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f385c1400c0 0x7f38400039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:09.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.097+0000 7f3853e00640 1 -- 172.21.15.131:0/3639725284 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3840004d40 msgr2=0x7f38400051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:09.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.097+0000 7f3853e00640 1 --2- 172.21.15.131:0/3639725284 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3840004d40 0x7f38400051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:09.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.097+0000 7f3853e00640 1 -- 172.21.15.131:0/3639725284 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3840003520 con 0x7f3840003f80 2024-05-26T22:56:09.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.097+0000 7f385aa00640 1 --2- 172.21.15.131:0/3639725284 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f385c1400c0 0x7f38400039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:56:09.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.098+0000 7f3853e00640 1 --2- 172.21.15.131:0/3639725284 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3840003f80 0x7f3840004800 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f384400ba30 tx=0x7f3840005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bc14e56b0f7eabe server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:09.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.098+0000 7f385a000640 1 -- 172.21.15.131:0/3639725284 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3844030070 con 0x7f3840003f80 2024-05-26T22:56:09.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.098+0000 7f385a000640 1 -- 172.21.15.131:0/3639725284 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f384402ae00 con 0x7f3840003f80 2024-05-26T22:56:09.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.098+0000 7f385a000640 1 -- 172.21.15.131:0/3639725284 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f384400b600 con 0x7f3840003f80 2024-05-26T22:56:09.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.099+0000 7f385be00640 1 -- 172.21.15.131:0/3639725284 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3840003f80 msgr2=0x7f3840004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:09.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.099+0000 7f385be00640 1 --2- 172.21.15.131:0/3639725284 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3840003f80 0x7f3840004800 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f384400ba30 tx=0x7f3840005830 comp rx=0 tx=0).stop 2024-05-26T22:56:09.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.099+0000 7f385b400640 1 -- 172.21.15.131:0/3639725284 reap_dead start 2024-05-26T22:56:09.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.099+0000 7f385be00640 1 -- 172.21.15.131:0/3639725284 shutdown_connections 2024-05-26T22:56:09.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.099+0000 7f385be00640 1 -- 172.21.15.131:0/3639725284 >> 172.21.15.131:0/3639725284 conn(0x7f385c0b5660 msgr2=0x7f385c0b5a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:56:09.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.099+0000 7f385be00640 1 -- 172.21.15.131:0/3639725284 shutdown_connections 2024-05-26T22:56:09.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.099+0000 7f385be00640 1 -- 172.21.15.131:0/3639725284 wait complete. 2024-05-26T22:56:09.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.101+0000 7f385be00640 1 Processor -- start 2024-05-26T22:56:09.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.101+0000 7f385be00640 1 -- start start 2024-05-26T22:56:09.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.102+0000 7f385be00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3840003f80 0x7f385c13b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:09.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.102+0000 7f385be00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f385c138e20 0x7f385c1f3500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:09.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.102+0000 7f385be00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f385c151df0 0x7f385c15db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:09.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.102+0000 7f385be00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f385c15c100 con 0x7f385c151df0 2024-05-26T22:56:09.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.102+0000 7f385be00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f385c071a70 con 0x7f385c138e20 2024-05-26T22:56:09.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.102+0000 7f385be00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f385c15c280 con 0x7f3840003f80 2024-05-26T22:56:09.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.102+0000 7f385b400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f385c151df0 0x7f385c15db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:09.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.102+0000 7f385b400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f385c151df0 0x7f385c15db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54510/0 (socket says 172.21.15.131:54510) 2024-05-26T22:56:09.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.102+0000 7f385b400640 1 -- 172.21.15.131:0/2086553630 learned_addr learned my addr 172.21.15.131:0/2086553630 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:56:09.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.102+0000 7f385aa00640 1 --2- 172.21.15.131:0/2086553630 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3840003f80 0x7f385c13b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:09.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.102+0000 7f385b400640 1 -- 172.21.15.131:0/2086553630 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3840003f80 msgr2=0x7f385c13b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:09.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.102+0000 7f385b400640 1 --2- 172.21.15.131:0/2086553630 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3840003f80 0x7f385c13b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:09.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.102+0000 7f385b400640 1 -- 172.21.15.131:0/2086553630 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f385c138e20 msgr2=0x7f385c1f3500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:09.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.102+0000 7f385b400640 1 --2- 172.21.15.131:0/2086553630 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f385c138e20 0x7f385c1f3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:09.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.102+0000 7f385b400640 1 -- 172.21.15.131:0/2086553630 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f385c13bdc0 con 0x7f385c151df0 2024-05-26T22:56:09.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.102+0000 7f385aa00640 1 --2- 172.21.15.131:0/2086553630 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3840003f80 0x7f385c13b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:56:09.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.103+0000 7f385b400640 1 --2- 172.21.15.131:0/2086553630 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f385c151df0 0x7f385c15db60 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f385c159220 tx=0x7f385400b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:09.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.103+0000 7f3852a00640 1 -- 172.21.15.131:0/2086553630 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3854018020 con 0x7f385c151df0 2024-05-26T22:56:09.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.103+0000 7f3852a00640 1 -- 172.21.15.131:0/2086553630 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f385400bae0 con 0x7f385c151df0 2024-05-26T22:56:09.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.103+0000 7f385be00640 1 -- 172.21.15.131:0/2086553630 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f385c13c0a0 con 0x7f385c151df0 2024-05-26T22:56:09.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.103+0000 7f3852a00640 1 -- 172.21.15.131:0/2086553630 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3854014400 con 0x7f385c151df0 2024-05-26T22:56:09.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.103+0000 7f385be00640 1 -- 172.21.15.131:0/2086553630 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f385c13c660 con 0x7f385c151df0 2024-05-26T22:56:09.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.105+0000 7f3852a00640 1 -- 172.21.15.131:0/2086553630 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3854005d10 con 0x7f385c151df0 2024-05-26T22:56:09.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.106+0000 7f38645b1740 1 Processor -- start 2024-05-26T22:56:09.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.107+0000 7f38645b1740 1 -- start start 2024-05-26T22:56:09.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.107+0000 7f38645b1740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5651eaa50e50 0x5651eaa03a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:09.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.107+0000 7f38645b1740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5651eaa33f70 0x5651eaa332f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:09.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.107+0000 7f38645b1740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5651eaa33830 0x5651eaa08640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:09.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.107+0000 7f38645b1740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5651ea96b610 con 0x5651eaa50e50 2024-05-26T22:56:09.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.107+0000 7f38645b1740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5651ea9135c0 con 0x5651eaa33f70 2024-05-26T22:56:09.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.107+0000 7f38645b1740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x5651ea9675d0 con 0x5651eaa33830 2024-05-26T22:56:09.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.108+0000 7f3853e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5651eaa50e50 0x5651eaa03a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:09.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.108+0000 7f3853e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5651eaa50e50 0x5651eaa03a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54522/0 (socket says 172.21.15.131:54522) 2024-05-26T22:56:09.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.108+0000 7f3853e00640 1 -- 172.21.15.131:0/2597418081 learned_addr learned my addr 172.21.15.131:0/2597418081 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:56:09.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.108+0000 7f3853e00640 1 -- 172.21.15.131:0/2597418081 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5651eaa33830 msgr2=0x5651eaa08640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:09.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.108+0000 7f3853e00640 1 --2- 172.21.15.131:0/2597418081 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5651eaa33830 0x5651eaa08640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:09.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.108+0000 7f385aa00640 1 --2- 172.21.15.131:0/2597418081 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5651eaa33f70 0x5651eaa332f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:09.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.108+0000 7f3853e00640 1 -- 172.21.15.131:0/2597418081 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5651eaa33f70 msgr2=0x5651eaa332f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:09.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.108+0000 7f3853e00640 1 --2- 172.21.15.131:0/2597418081 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5651eaa33f70 0x5651eaa332f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:09.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.108+0000 7f3853e00640 1 -- 172.21.15.131:0/2597418081 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651ea67f2d0 con 0x5651eaa50e50 2024-05-26T22:56:09.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.108+0000 7f3852a00640 1 --2- 172.21.15.131:0/2086553630 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f383806be30 0x7f383806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:09.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.109+0000 7f3853e00640 1 --2- 172.21.15.131:0/2086553630 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f383806be30 0x7f383806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:09.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.109+0000 7f3852a00640 1 -- 172.21.15.131:0/2086553630 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5709+0+0 (secure 0 0 0) 0x7f385408b450 con 0x7f385c151df0 2024-05-26T22:56:09.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.109+0000 7f3853e00640 1 --2- 172.21.15.131:0/2597418081 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5651eaa50e50 0x5651eaa03a70 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f384402a8f0 tx=0x7f38440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee1a11542a69ffc server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:09.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.109+0000 7f384b000640 1 -- 172.21.15.131:0/2597418081 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f384403d880 con 0x5651eaa50e50 2024-05-26T22:56:09.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.109+0000 7f384b000640 1 -- 172.21.15.131:0/2597418081 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f384403daa0 con 0x5651eaa50e50 2024-05-26T22:56:09.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.109+0000 7f384b000640 1 -- 172.21.15.131:0/2597418081 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3844034400 con 0x5651eaa50e50 2024-05-26T22:56:09.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.110+0000 7f38645b1740 1 -- 172.21.15.131:0/2597418081 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5651eaa50e50 msgr2=0x5651eaa03a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:09.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.110+0000 7f38645b1740 1 --2- 172.21.15.131:0/2597418081 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5651eaa50e50 0x5651eaa03a70 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f384402a8f0 tx=0x7f38440046c0 comp rx=0 tx=0).stop 2024-05-26T22:56:09.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.110+0000 7f3853e00640 1 --2- 172.21.15.131:0/2086553630 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f383806be30 0x7f383806e2f0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f384400b850 tx=0x7f3844043000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:09.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.110+0000 7f38645b1740 1 -- 172.21.15.131:0/2597418081 shutdown_connections 2024-05-26T22:56:09.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.110+0000 7f38645b1740 1 --2- 172.21.15.131:0/2597418081 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5651eaa33830 0x5651eaa08640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:09.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.110+0000 7f38645b1740 1 --2- 172.21.15.131:0/2597418081 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5651eaa33f70 0x5651eaa332f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:09.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.110+0000 7f38645b1740 1 --2- 172.21.15.131:0/2597418081 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5651eaa50e50 0x5651eaa03a70 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:09.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.110+0000 7f38645b1740 1 -- 172.21.15.131:0/2597418081 >> 172.21.15.131:0/2597418081 conn(0x5651eaa05ce0 msgr2=0x5651eaa4e2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:56:09.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.110+0000 7f38645b1740 1 -- 172.21.15.131:0/2597418081 shutdown_connections 2024-05-26T22:56:09.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.110+0000 7f38645b1740 1 -- 172.21.15.131:0/2597418081 wait complete. 2024-05-26T22:56:09.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.111+0000 7f38645b1740 1 -- start start 2024-05-26T22:56:09.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.111+0000 7f38645b1740 1 Processor -- start 2024-05-26T22:56:09.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.112+0000 7f38645b1740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5651eaa33f70 0x5651eaa5e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:09.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.112+0000 7f38645b1740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5651eaa50e50 0x5651eaa2ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:09.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.112+0000 7f38645b1740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5651eaa363f0 0x5651eaa47170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:09.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.112+0000 7f38645b1740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5651eaa07c00 con 0x5651eaa363f0 2024-05-26T22:56:09.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.112+0000 7f38645b1740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5651ea926910 con 0x5651eaa33f70 2024-05-26T22:56:09.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.112+0000 7f38645b1740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x5651eaa06ff0 con 0x5651eaa50e50 2024-05-26T22:56:09.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.112+0000 7f38645b1740 10 client.0 _ll_register_callbacks cb 0x5651ea962410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-26T22:56:09.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.112+0000 7f385b400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5651eaa363f0 0x5651eaa47170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:09.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.112+0000 7f385b400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5651eaa363f0 0x5651eaa47170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54538/0 (socket says 172.21.15.131:54538) 2024-05-26T22:56:09.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.112+0000 7f385b400640 1 -- 172.21.15.131:0/3131029598 learned_addr learned my addr 172.21.15.131:0/3131029598 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:56:09.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.112+0000 7f385b400640 1 -- 172.21.15.131:0/3131029598 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5651eaa50e50 msgr2=0x5651eaa2ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:09.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.112+0000 7f385b400640 1 --2- 172.21.15.131:0/3131029598 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5651eaa50e50 0x5651eaa2ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:09.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.112+0000 7f385b400640 1 -- 172.21.15.131:0/3131029598 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5651eaa33f70 msgr2=0x5651eaa5e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:09.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.112+0000 7f385b400640 1 --2- 172.21.15.131:0/3131029598 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5651eaa33f70 0x5651eaa5e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:09.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.112+0000 7f385b400640 1 -- 172.21.15.131:0/3131029598 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651ea7496b0 con 0x5651eaa363f0 2024-05-26T22:56:09.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.113+0000 7f385b400640 1 --2- 172.21.15.131:0/3131029598 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5651eaa363f0 0x5651eaa47170 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f3854061da0 tx=0x7f3854059eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b98f4d6ef80efa9d server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:09.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.113+0000 7f383f400640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-26T22:56:09.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.113+0000 7f383f400640 1 -- 172.21.15.131:0/3131029598 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3854071070 con 0x5651eaa363f0 2024-05-26T22:56:09.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.113+0000 7f383f400640 1 -- 172.21.15.131:0/3131029598 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 5709+0+0 (secure 0 0 0) 0x7f3854059950 con 0x5651eaa363f0 2024-05-26T22:56:09.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.113+0000 7f38645b1740 1 -- 172.21.15.131:0/3131029598 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f384403daa0 con 0x5651eaa363f0 2024-05-26T22:56:09.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.113+0000 7f383f400640 1 -- 172.21.15.131:0/3131029598 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=110) ==== 24+0+0 (secure 0 0 0) 0x7f38540622d0 con 0x5651eaa363f0 2024-05-26T22:56:09.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.113+0000 7f38645b1740 10 client.8433 fetch_fsmap learned FSMap version 110 2024-05-26T22:56:09.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.113+0000 7f38645b1740 1 -- 172.21.15.131:0/3131029598 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=110,monmap=2+}) -- 0x5651eaa36eb0 con 0x5651eaa363f0 2024-05-26T22:56:09.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.113+0000 7f383f400640 1 -- 172.21.15.131:0/3131029598 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f384403daa0 con 0x5651eaa363f0 2024-05-26T22:56:09.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.113+0000 7f383f400640 1 -- 172.21.15.131:0/3131029598 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 110) ==== 3494+0+0 (secure 0 0 0) 0x7f3854061e40 con 0x5651eaa363f0 2024-05-26T22:56:09.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.114+0000 7f38645b1740 10 client.8433 fetch_fsmap finished waiting for FSMap version 110 2024-05-26T22:56:09.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.114+0000 7f38645b1740 10 client.8433 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-26T22:56:09.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.114+0000 7f38645b1740 20 client.8433 populate_metadata read hostname 'smithi131' 2024-05-26T22:56:09.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.114+0000 7f38645b1740 1 --2- 172.21.15.131:0/3131029598 >> [v2:172.21.15.131:6835/2725520331,v1:172.21.15.131:6837/2725520331] conn(0x5651eaa45d20 0x5651eaa34d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:09.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.114+0000 7f38645b1740 4 client.8433 mds_command: new command op to 8310 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-26T22:56:09.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.114+0000 7f38645b1740 1 -- 172.21.15.131:0/3131029598 --> [v2:172.21.15.131:6835/2725520331,v1:172.21.15.131:6837/2725520331] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5651ea978d40 con 0x5651eaa45d20 2024-05-26T22:56:09.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.114+0000 7f385aa00640 1 --2- 172.21.15.131:0/3131029598 >> [v2:172.21.15.131:6835/2725520331,v1:172.21.15.131:6837/2725520331] conn(0x5651eaa45d20 0x5651eaa34d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:09.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.114+0000 7f385aa00640 1 --2- 172.21.15.131:0/3131029598 >> [v2:172.21.15.131:6835/2725520331,v1:172.21.15.131:6837/2725520331] conn(0x5651eaa45d20 0x5651eaa34d90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71631ad0fd606ff7 server_cookie=ea5073f4e69ba64e in_seq=0 out_seq=0 2024-05-26T22:56:09.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.114+0000 7f383f400640 10 client.8433 ms_handle_connect on v2:172.21.15.131:6835/2725520331 2024-05-26T22:56:09.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.115+0000 7f383f400640 1 -- 172.21.15.131:0/3131029598 <== mds.0 v2:172.21.15.131:6835/2725520331 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5651ea978d40 con 0x5651eaa45d20 2024-05-26T22:56:09.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.115+0000 7f383f400640 10 client.8433 handle_command_reply: tid=0 2024-05-26T22:56:09.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.115+0000 7f38645b1740 1 client.8433 shutdown 2024-05-26T22:56:09.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.116+0000 7f38645b1740 1 -- 172.21.15.131:0/3131029598 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5651eaa363f0 msgr2=0x5651eaa47170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:09.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.116+0000 7f38645b1740 1 --2- 172.21.15.131:0/3131029598 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5651eaa363f0 0x5651eaa47170 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f3854061da0 tx=0x7f3854059eb0 comp rx=0 tx=0).stop 2024-05-26T22:56:09.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.116+0000 7f38645b1740 1 -- 172.21.15.131:0/3131029598 shutdown_connections 2024-05-26T22:56:09.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.116+0000 7f38645b1740 1 --2- 172.21.15.131:0/3131029598 >> [v2:172.21.15.131:6835/2725520331,v1:172.21.15.131:6837/2725520331] conn(0x5651eaa45d20 0x5651eaa34d90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:09.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.116+0000 7f38645b1740 1 --2- 172.21.15.131:0/3131029598 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5651eaa50e50 0x5651eaa2ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:09.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.116+0000 7f38645b1740 1 --2- 172.21.15.131:0/3131029598 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5651eaa363f0 0x5651eaa47170 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:09.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.116+0000 7f38645b1740 1 --2- 172.21.15.131:0/3131029598 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5651eaa33f70 0x5651eaa5e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:09.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.116+0000 7f383f400640 0 client.8433 ms_handle_reset on v2:172.21.15.131:6835/2725520331 2024-05-26T22:56:09.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.116+0000 7f38645b1740 1 -- 172.21.15.131:0/3131029598 >> 172.21.15.131:0/3131029598 conn(0x5651eaa33830 msgr2=0x5651eaa044d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:56:09.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.116+0000 7f38645b1740 1 -- 172.21.15.131:0/3131029598 shutdown_connections 2024-05-26T22:56:09.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.116+0000 7f38645b1740 1 -- 172.21.15.131:0/3131029598 wait complete. 2024-05-26T22:56:09.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.116+0000 7f38645b1740 20 client.8433 trim_cache size 0 max 16384 2024-05-26T22:56:09.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.136+0000 7f38645b1740 1 Processor -- start 2024-05-26T22:56:09.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.136+0000 7f38645b1740 1 -- start start 2024-05-26T22:56:09.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.137+0000 7f38645b1740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5651eaa33f70 0x5651eaa03b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:09.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.137+0000 7f38645b1740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5651eaa363f0 0x5651eaa51f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:09.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.137+0000 7f38645b1740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5651eaa50e50 0x5651eaa4cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:09.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.137+0000 7f38645b1740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f385c1ffd20 con 0x5651eaa50e50 2024-05-26T22:56:09.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.137+0000 7f38645b1740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f385c0ae4f0 con 0x5651eaa33f70 2024-05-26T22:56:09.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.137+0000 7f38645b1740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f385c1ff6b0 con 0x5651eaa363f0 2024-05-26T22:56:09.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.137+0000 7f385b400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5651eaa50e50 0x5651eaa4cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:09.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.137+0000 7f385b400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5651eaa50e50 0x5651eaa4cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54550/0 (socket says 172.21.15.131:54550) 2024-05-26T22:56:09.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.137+0000 7f385b400640 1 -- 172.21.15.131:0/282848290 learned_addr learned my addr 172.21.15.131:0/282848290 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:56:09.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.137+0000 7f385aa00640 1 --2- 172.21.15.131:0/282848290 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5651eaa33f70 0x5651eaa03b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:09.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.138+0000 7f385b400640 1 -- 172.21.15.131:0/282848290 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5651eaa363f0 msgr2=0x5651eaa51f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:09.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.138+0000 7f385b400640 1 --2- 172.21.15.131:0/282848290 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5651eaa363f0 0x5651eaa51f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:09.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.138+0000 7f385b400640 1 -- 172.21.15.131:0/282848290 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5651eaa33f70 msgr2=0x5651eaa03b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:09.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.138+0000 7f385b400640 1 --2- 172.21.15.131:0/282848290 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5651eaa33f70 0x5651eaa03b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:09.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.138+0000 7f385b400640 1 -- 172.21.15.131:0/282848290 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651eaa4e8b0 con 0x5651eaa50e50 2024-05-26T22:56:09.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.138+0000 7f385b400640 1 --2- 172.21.15.131:0/282848290 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5651eaa50e50 0x5651eaa4cd60 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f3854055b50 tx=0x7f385401f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd3bb8a462c2f00f server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:09.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.138+0000 7f384a600640 1 -- 172.21.15.131:0/282848290 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3854071040 con 0x5651eaa50e50 2024-05-26T22:56:09.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.138+0000 7f384a600640 1 -- 172.21.15.131:0/282848290 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3854059ae0 con 0x5651eaa50e50 2024-05-26T22:56:09.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.138+0000 7f384a600640 1 -- 172.21.15.131:0/282848290 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f385406db80 con 0x5651eaa50e50 2024-05-26T22:56:09.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.139+0000 7f38645b1740 1 -- 172.21.15.131:0/282848290 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5651eaa50e50 msgr2=0x5651eaa4cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:09.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.139+0000 7f38645b1740 1 --2- 172.21.15.131:0/282848290 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5651eaa50e50 0x5651eaa4cd60 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f3854055b50 tx=0x7f385401f000 comp rx=0 tx=0).stop 2024-05-26T22:56:09.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.139+0000 7f38645b1740 1 -- 172.21.15.131:0/282848290 shutdown_connections 2024-05-26T22:56:09.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.139+0000 7f38645b1740 1 --2- 172.21.15.131:0/282848290 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5651eaa363f0 0x5651eaa51f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:09.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.139+0000 7f38645b1740 1 --2- 172.21.15.131:0/282848290 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5651eaa50e50 0x5651eaa4cd60 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:09.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.139+0000 7f38645b1740 1 --2- 172.21.15.131:0/282848290 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5651eaa33f70 0x5651eaa03b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:09.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.139+0000 7f38645b1740 1 -- 172.21.15.131:0/282848290 >> 172.21.15.131:0/282848290 conn(0x5651eaa33830 msgr2=0x5651eaa4e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:56:09.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.139+0000 7f38645b1740 1 -- 172.21.15.131:0/282848290 shutdown_connections 2024-05-26T22:56:09.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.139+0000 7f38645b1740 1 -- 172.21.15.131:0/282848290 wait complete. 2024-05-26T22:56:09.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.140+0000 7f38645b1740 1 -- start start 2024-05-26T22:56:09.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.140+0000 7f38645b1740 1 Processor -- start 2024-05-26T22:56:09.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.141+0000 7f38645b1740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5651eaa33f70 0x5651eaad51f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:09.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.141+0000 7f38645b1740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5651eaa363f0 0x5651eaa30150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:09.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.141+0000 7f38645b1740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5651eaa50e50 0x5651eaa30690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:09.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.141+0000 7f38645b1740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5651eaa37230 con 0x5651eaa50e50 2024-05-26T22:56:09.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.141+0000 7f38645b1740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5651eaa453c0 con 0x5651eaa33f70 2024-05-26T22:56:09.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.141+0000 7f38645b1740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x5651eaa46180 con 0x5651eaa363f0 2024-05-26T22:56:09.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.141+0000 7f38645b1740 10 client.0 _ll_register_callbacks cb 0x7f3854060050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-26T22:56:09.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.141+0000 7f385b400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5651eaa50e50 0x5651eaa30690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:09.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.141+0000 7f385b400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5651eaa50e50 0x5651eaa30690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54556/0 (socket says 172.21.15.131:54556) 2024-05-26T22:56:09.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.141+0000 7f385b400640 1 -- 172.21.15.131:0/872350598 learned_addr learned my addr 172.21.15.131:0/872350598 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:56:09.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.141+0000 7f385aa00640 1 --2- 172.21.15.131:0/872350598 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5651eaa33f70 0x5651eaad51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:09.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.141+0000 7f385b400640 1 -- 172.21.15.131:0/872350598 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5651eaa363f0 msgr2=0x5651eaa30150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:09.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.141+0000 7f3853e00640 1 --2- 172.21.15.131:0/872350598 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5651eaa363f0 0x5651eaa30150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:09.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.141+0000 7f385b400640 1 --2- 172.21.15.131:0/872350598 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5651eaa363f0 0x5651eaa30150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:09.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.141+0000 7f385b400640 1 -- 172.21.15.131:0/872350598 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5651eaa33f70 msgr2=0x5651eaad51f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:09.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.141+0000 7f385b400640 1 --2- 172.21.15.131:0/872350598 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5651eaa33f70 0x5651eaad51f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:09.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.141+0000 7f385b400640 1 -- 172.21.15.131:0/872350598 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651eaa718c0 con 0x5651eaa50e50 2024-05-26T22:56:09.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.142+0000 7f385b400640 1 --2- 172.21.15.131:0/872350598 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5651eaa50e50 0x5651eaa30690 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f3854059680 tx=0x7f385401f070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6b20a96cae5d1c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:09.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.142+0000 7f383f400640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-26T22:56:09.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.142+0000 7f383f400640 1 -- 172.21.15.131:0/872350598 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3854071040 con 0x5651eaa50e50 2024-05-26T22:56:09.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.142+0000 7f383f400640 1 -- 172.21.15.131:0/872350598 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 5709+0+0 (secure 0 0 0) 0x7f385406db80 con 0x5651eaa50e50 2024-05-26T22:56:09.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.142+0000 7f38645b1740 1 -- 172.21.15.131:0/872350598 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3854059ae0 con 0x5651eaa50e50 2024-05-26T22:56:09.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.142+0000 7f383f400640 1 -- 172.21.15.131:0/872350598 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=110) ==== 24+0+0 (secure 0 0 0) 0x7f385406dd80 con 0x5651eaa50e50 2024-05-26T22:56:09.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.142+0000 7f38645b1740 10 client.8439 fetch_fsmap learned FSMap version 110 2024-05-26T22:56:09.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.142+0000 7f38645b1740 1 -- 172.21.15.131:0/872350598 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=110,monmap=2+}) -- 0x5651eaa718c0 con 0x5651eaa50e50 2024-05-26T22:56:09.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.142+0000 7f383f400640 1 -- 172.21.15.131:0/872350598 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3854059ae0 con 0x5651eaa50e50 2024-05-26T22:56:09.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.142+0000 7f383f400640 1 -- 172.21.15.131:0/872350598 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 110) ==== 3494+0+0 (secure 0 0 0) 0x7f3854058b40 con 0x5651eaa50e50 2024-05-26T22:56:09.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.142+0000 7f38645b1740 10 client.8439 fetch_fsmap finished waiting for FSMap version 110 2024-05-26T22:56:09.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.142+0000 7f38645b1740 10 client.8439 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-26T22:56:09.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.142+0000 7f38645b1740 20 client.8439 populate_metadata read hostname 'smithi131' 2024-05-26T22:56:09.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.142+0000 7f38645b1740 1 --2- 172.21.15.131:0/872350598 >> [v2:172.21.15.131:6835/2725520331,v1:172.21.15.131:6837/2725520331] conn(0x5651eaa44400 0x5651eaa447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:09.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.142+0000 7f38645b1740 4 client.8439 mds_command: new command op to 8310 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-26T22:56:09.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.142+0000 7f38645b1740 1 -- 172.21.15.131:0/872350598 --> [v2:172.21.15.131:6835/2725520331,v1:172.21.15.131:6837/2725520331] -- command(tid 0: {"prefix": "status"}) -- 0x5651ea973090 con 0x5651eaa44400 2024-05-26T22:56:09.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.143+0000 7f385aa00640 1 --2- 172.21.15.131:0/872350598 >> [v2:172.21.15.131:6835/2725520331,v1:172.21.15.131:6837/2725520331] conn(0x5651eaa44400 0x5651eaa447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:09.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.143+0000 7f385aa00640 1 --2- 172.21.15.131:0/872350598 >> [v2:172.21.15.131:6835/2725520331,v1:172.21.15.131:6837/2725520331] conn(0x5651eaa44400 0x5651eaa447e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85e560301488b2c3 server_cookie=353cafc1717b8c57 in_seq=0 out_seq=0 2024-05-26T22:56:09.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.143+0000 7f383f400640 10 client.8439 ms_handle_connect on v2:172.21.15.131:6835/2725520331 2024-05-26T22:56:09.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.143+0000 7f383f400640 1 -- 172.21.15.131:0/872350598 <== mds.0 v2:172.21.15.131:6835/2725520331 1 ==== command_reply(tid 0: 0 ) ==== 8+0+334 (crc 0 0 0) 0x5651ea973090 con 0x5651eaa44400 2024-05-26T22:56:09.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.143+0000 7f383f400640 10 client.8439 handle_command_reply: tid=0 2024-05-26T22:56:09.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.143+0000 7f38645b1740 1 client.8439 shutdown 2024-05-26T22:56:09.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.143+0000 7f38645b1740 1 -- 172.21.15.131:0/872350598 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5651eaa50e50 msgr2=0x5651eaa30690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:09.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.143+0000 7f38645b1740 1 --2- 172.21.15.131:0/872350598 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5651eaa50e50 0x5651eaa30690 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f3854059680 tx=0x7f385401f070 comp rx=0 tx=0).stop 2024-05-26T22:56:09.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.144+0000 7f38645b1740 1 -- 172.21.15.131:0/872350598 shutdown_connections 2024-05-26T22:56:09.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.144+0000 7f38645b1740 1 --2- 172.21.15.131:0/872350598 >> [v2:172.21.15.131:6835/2725520331,v1:172.21.15.131:6837/2725520331] conn(0x5651eaa44400 0x5651eaa447e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:09.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.144+0000 7f38645b1740 1 --2- 172.21.15.131:0/872350598 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x5651eaa363f0 0x5651eaa30150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:09.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.144+0000 7f38645b1740 1 --2- 172.21.15.131:0/872350598 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5651eaa50e50 0x5651eaa30690 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:09.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.144+0000 7f383f400640 0 client.8439 ms_handle_reset on v2:172.21.15.131:6835/2725520331 2024-05-26T22:56:09.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.144+0000 7f38645b1740 1 --2- 172.21.15.131:0/872350598 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5651eaa33f70 0x5651eaad51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:09.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.144+0000 7f38645b1740 1 -- 172.21.15.131:0/872350598 >> 172.21.15.131:0/872350598 conn(0x5651eaa33830 msgr2=0x5651eaa51d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:56:09.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.144+0000 7f38645b1740 1 -- 172.21.15.131:0/872350598 shutdown_connections 2024-05-26T22:56:09.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.144+0000 7f38645b1740 1 -- 172.21.15.131:0/872350598 wait complete. 2024-05-26T22:56:09.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.144+0000 7f38645b1740 20 client.8439 trim_cache size 0 max 16384 2024-05-26T22:56:09.149 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-26T22:56:09.149 INFO:teuthology.orchestra.run.smithi131.stdout: "cluster_fsid": "46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f", 2024-05-26T22:56:09.149 INFO:teuthology.orchestra.run.smithi131.stdout: "whoami": 0, 2024-05-26T22:56:09.149 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 8310, 2024-05-26T22:56:09.149 INFO:teuthology.orchestra.run.smithi131.stdout: "want_state": "up:active", 2024-05-26T22:56:09.150 INFO:teuthology.orchestra.run.smithi131.stdout: "state": "up:active", 2024-05-26T22:56:09.150 INFO:teuthology.orchestra.run.smithi131.stdout: "fs_name": "cephfs", 2024-05-26T22:56:09.150 INFO:teuthology.orchestra.run.smithi131.stdout: "rank_uptime": 6.6263421889999998, 2024-05-26T22:56:09.150 INFO:teuthology.orchestra.run.smithi131.stdout: "mdsmap_epoch": 110, 2024-05-26T22:56:09.150 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch": 107, 2024-05-26T22:56:09.150 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch_barrier": 107, 2024-05-26T22:56:09.150 INFO:teuthology.orchestra.run.smithi131.stdout: "uptime": 19.092075850000001 2024-05-26T22:56:09.150 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-26T22:56:09.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.144+0000 7f385be00640 1 -- 172.21.15.131:0/2086553630 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f383806be30 msgr2=0x7f383806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:09.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.144+0000 7f385be00640 1 --2- 172.21.15.131:0/2086553630 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f383806be30 0x7f383806e2f0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f384400b850 tx=0x7f3844043000 comp rx=0 tx=0).stop 2024-05-26T22:56:09.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.144+0000 7f385be00640 1 -- 172.21.15.131:0/2086553630 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f385c151df0 msgr2=0x7f385c15db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:09.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.144+0000 7f385be00640 1 --2- 172.21.15.131:0/2086553630 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f385c151df0 0x7f385c15db60 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f385c159220 tx=0x7f385400b480 comp rx=0 tx=0).stop 2024-05-26T22:56:09.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.145+0000 7f385be00640 1 -- 172.21.15.131:0/2086553630 shutdown_connections 2024-05-26T22:56:09.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.145+0000 7f385be00640 1 --2- 172.21.15.131:0/2086553630 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f383806be30 0x7f383806e2f0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:09.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.145+0000 7f385be00640 1 --2- 172.21.15.131:0/2086553630 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f385c151df0 0x7f385c15db60 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:09.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.145+0000 7f385be00640 1 --2- 172.21.15.131:0/2086553630 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f385c138e20 0x7f385c1f3500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:09.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.145+0000 7f385be00640 1 --2- 172.21.15.131:0/2086553630 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3840003f80 0x7f385c13b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:09.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.145+0000 7f385be00640 1 -- 172.21.15.131:0/2086553630 >> 172.21.15.131:0/2086553630 conn(0x7f385c0b5660 msgr2=0x7f385c159250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:56:09.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.145+0000 7f385be00640 1 -- 172.21.15.131:0/2086553630 shutdown_connections 2024-05-26T22:56:09.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.145+0000 7f385be00640 1 -- 172.21.15.131:0/2086553630 wait complete. 2024-05-26T22:56:09.160 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-26T22:56:09.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.347+0000 7f03ecc00640 1 Processor -- start 2024-05-26T22:56:09.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.347+0000 7f03ecc00640 1 -- start start 2024-05-26T22:56:09.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.348+0000 7f03ecc00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f03e8158c00 0x7f03e806d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:09.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.348+0000 7f03ecc00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f03e80b38a0 0x7f03e815fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:09.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.348+0000 7f03ecc00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03e80b3e50 con 0x7f03e806daf0 2024-05-26T22:56:09.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.348+0000 7f03ecc00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f03e806df50 con 0x7f03e8158c00 2024-05-26T22:56:09.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.348+0000 7f03ecc00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f03e80b4130 con 0x7f03e80b38a0 2024-05-26T22:56:09.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.349+0000 7f03e5e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f03e8158c00 0x7f03e806d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:09.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.349+0000 7f03e5400640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f03e806daf0 0x7f03e80b3190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:45824/0 (socket says 172.21.15.131:45824) 2024-05-26T22:56:09.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.349+0000 7f03e5400640 1 -- 172.21.15.131:0/1069926767 learned_addr learned my addr 172.21.15.131:0/1069926767 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:56:09.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.349+0000 7f03e6800640 1 --2- 172.21.15.131:0/1069926767 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f03e80b38a0 0x7f03e815fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:09.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.349+0000 7f03e5e00640 1 -- 172.21.15.131:0/1069926767 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f03e80b38a0 msgr2=0x7f03e815fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:09.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.349+0000 7f03e5e00640 1 --2- 172.21.15.131:0/1069926767 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f03e80b38a0 0x7f03e815fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:09.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.349+0000 7f03e5e00640 1 -- 172.21.15.131:0/1069926767 >> v1:172.21.15.131:6789/0 conn(0x7f03e806daf0 legacy=0x7f03e80b3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:09.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.349+0000 7f03e5e00640 1 -- 172.21.15.131:0/1069926767 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03e80b4410 con 0x7f03e8158c00 2024-05-26T22:56:09.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.349+0000 7f03e4a00640 1 -- 172.21.15.131:0/1069926767 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3036684196 0 0) 0x7f03e80b3e50 con 0x7f03e806daf0 2024-05-26T22:56:09.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.350+0000 7f03e5e00640 1 --2- 172.21.15.131:0/1069926767 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03e8158c00 0x7f03e806d5b0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f03d80097a0 tx=0x7f03d802a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0d5ea9f14e8f822 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:09.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.351+0000 7f03e4a00640 1 -- 172.21.15.131:0/1069926767 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d8037070 con 0x7f03e8158c00 2024-05-26T22:56:09.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.351+0000 7f03e4a00640 1 -- 172.21.15.131:0/1069926767 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03d802ae50 con 0x7f03e8158c00 2024-05-26T22:56:09.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.351+0000 7f03e4a00640 1 -- 172.21.15.131:0/1069926767 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d8032660 con 0x7f03e8158c00 2024-05-26T22:56:09.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.351+0000 7f03ecc00640 1 -- 172.21.15.131:0/1069926767 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03e8158c00 msgr2=0x7f03e806d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:09.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.351+0000 7f03ecc00640 1 --2- 172.21.15.131:0/1069926767 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03e8158c00 0x7f03e806d5b0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f03d80097a0 tx=0x7f03d802a100 comp rx=0 tx=0).stop 2024-05-26T22:56:09.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.351+0000 7f03ecc00640 1 -- 172.21.15.131:0/1069926767 shutdown_connections 2024-05-26T22:56:09.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.351+0000 7f03ecc00640 1 --2- 172.21.15.131:0/1069926767 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f03e80b38a0 0x7f03e815fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:09.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.351+0000 7f03ecc00640 1 --2- 172.21.15.131:0/1069926767 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03e8158c00 0x7f03e806d5b0 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:09.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.351+0000 7f03ecc00640 1 -- 172.21.15.131:0/1069926767 >> 172.21.15.131:0/1069926767 conn(0x7f03e80b1e90 msgr2=0x7f03e80b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:56:09.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.351+0000 7f03ecc00640 1 -- 172.21.15.131:0/1069926767 shutdown_connections 2024-05-26T22:56:09.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.351+0000 7f03ecc00640 1 -- 172.21.15.131:0/1069926767 wait complete. 2024-05-26T22:56:09.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.353+0000 7f03ecc00640 1 Processor -- start 2024-05-26T22:56:09.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.353+0000 7f03ecc00640 1 -- start start 2024-05-26T22:56:09.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.354+0000 7f03ecc00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f03e806daf0 0x7f03e8154910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:09.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.354+0000 7f03ecc00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f03e80b38a0 0x7f03e80adb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:09.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.354+0000 7f03ecc00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03e8158c00 0x7f03e815a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:09.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.354+0000 7f03ecc00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f03e8160190 con 0x7f03e8158c00 2024-05-26T22:56:09.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.354+0000 7f03ecc00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f03e80b49d0 con 0x7f03e80b38a0 2024-05-26T22:56:09.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.354+0000 7f03ecc00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f03e8160310 con 0x7f03e806daf0 2024-05-26T22:56:09.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.354+0000 7f03e6800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03e8158c00 0x7f03e815a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:09.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.354+0000 7f03e6800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03e8158c00 0x7f03e815a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54576/0 (socket says 172.21.15.131:54576) 2024-05-26T22:56:09.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.354+0000 7f03e6800640 1 -- 172.21.15.131:0/509617802 learned_addr learned my addr 172.21.15.131:0/509617802 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:56:09.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.355+0000 7f03e5e00640 1 --2- 172.21.15.131:0/509617802 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f03e806daf0 0x7f03e8154910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:09.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.355+0000 7f03e6800640 1 -- 172.21.15.131:0/509617802 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f03e806daf0 msgr2=0x7f03e8154910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:09.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.355+0000 7f03e6800640 1 --2- 172.21.15.131:0/509617802 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f03e806daf0 0x7f03e8154910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:09.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.355+0000 7f03e6800640 1 -- 172.21.15.131:0/509617802 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f03e80b38a0 msgr2=0x7f03e80adb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:09.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.355+0000 7f03e6800640 1 --2- 172.21.15.131:0/509617802 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f03e80b38a0 0x7f03e80adb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:09.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.355+0000 7f03e6800640 1 -- 172.21.15.131:0/509617802 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03e8154e50 con 0x7f03e8158c00 2024-05-26T22:56:09.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.355+0000 7f03e6800640 1 --2- 172.21.15.131:0/509617802 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03e8158c00 0x7f03e815a4d0 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f03e0007e40 tx=0x7f03e000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:09.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.355+0000 7f03de000640 1 -- 172.21.15.131:0/509617802 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03e0014620 con 0x7f03e8158c00 2024-05-26T22:56:09.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.355+0000 7f03de000640 1 -- 172.21.15.131:0/509617802 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03e0019070 con 0x7f03e8158c00 2024-05-26T22:56:09.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.355+0000 7f03de000640 1 -- 172.21.15.131:0/509617802 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03e0014a20 con 0x7f03e8158c00 2024-05-26T22:56:09.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.355+0000 7f03ecc00640 1 -- 172.21.15.131:0/509617802 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03e80ae460 con 0x7f03e8158c00 2024-05-26T22:56:09.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.355+0000 7f03ecc00640 1 -- 172.21.15.131:0/509617802 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03e80aea20 con 0x7f03e8158c00 2024-05-26T22:56:09.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.356+0000 7f03ecc00640 1 -- 172.21.15.131:0/509617802 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03e80694e0 con 0x7f03e8158c00 2024-05-26T22:56:09.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.358+0000 7f03de000640 1 -- 172.21.15.131:0/509617802 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f03e0014bc0 con 0x7f03e8158c00 2024-05-26T22:56:09.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.358+0000 7f03de000640 1 --2- 172.21.15.131:0/509617802 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f03c406be60 0x7f03c406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:09.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.359+0000 7f03de000640 1 -- 172.21.15.131:0/509617802 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5709+0+0 (secure 0 0 0) 0x7f03e008b7c0 con 0x7f03e8158c00 2024-05-26T22:56:09.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.359+0000 7f03e5e00640 1 --2- 172.21.15.131:0/509617802 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f03c406be60 0x7f03c406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:09.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.359+0000 7f03e5e00640 1 --2- 172.21.15.131:0/509617802 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f03c406be60 0x7f03c406e320 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f03d8034000 tx=0x7f03d802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:09.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.363+0000 7f03de000640 1 -- 172.21.15.131:0/509617802 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f03e8154e50 con 0x7f03e8158c00 2024-05-26T22:56:09.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.590+0000 7f03ecc00640 1 -- 172.21.15.131:0/509617802 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f03e806ed20 con 0x7f03e8158c00 2024-05-26T22:56:09.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.591+0000 7f03de000640 1 -- 172.21.15.131:0/509617802 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v110) ==== 53+0+73 (secure 0 0 0) 0x7f03e005a0d0 con 0x7f03e8158c00 2024-05-26T22:56:09.590 INFO:teuthology.orchestra.run.smithi131.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-26T22:56:09.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.593+0000 7f03ecc00640 1 -- 172.21.15.131:0/509617802 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f03c406be60 msgr2=0x7f03c406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:09.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.593+0000 7f03ecc00640 1 --2- 172.21.15.131:0/509617802 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f03c406be60 0x7f03c406e320 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f03d8034000 tx=0x7f03d802e000 comp rx=0 tx=0).stop 2024-05-26T22:56:09.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.593+0000 7f03ecc00640 1 -- 172.21.15.131:0/509617802 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03e8158c00 msgr2=0x7f03e815a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:09.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.593+0000 7f03ecc00640 1 --2- 172.21.15.131:0/509617802 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03e8158c00 0x7f03e815a4d0 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f03e0007e40 tx=0x7f03e000e4b0 comp rx=0 tx=0).stop 2024-05-26T22:56:09.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.593+0000 7f03ecc00640 1 -- 172.21.15.131:0/509617802 shutdown_connections 2024-05-26T22:56:09.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.593+0000 7f03ecc00640 1 --2- 172.21.15.131:0/509617802 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f03c406be60 0x7f03c406e320 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:09.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.593+0000 7f03ecc00640 1 --2- 172.21.15.131:0/509617802 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03e8158c00 0x7f03e815a4d0 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:09.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.593+0000 7f03ecc00640 1 --2- 172.21.15.131:0/509617802 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f03e80b38a0 0x7f03e80adb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:09.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.593+0000 7f03ecc00640 1 --2- 172.21.15.131:0/509617802 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f03e806daf0 0x7f03e8154910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:09.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.593+0000 7f03ecc00640 1 -- 172.21.15.131:0/509617802 >> 172.21.15.131:0/509617802 conn(0x7f03e80b1e90 msgr2=0x7f03e80b0880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:56:09.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.593+0000 7f03ecc00640 1 -- 172.21.15.131:0/509617802 shutdown_connections 2024-05-26T22:56:09.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09.594+0000 7f03ecc00640 1 -- 172.21.15.131:0/509617802 wait complete. 2024-05-26T22:56:09.611 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-26T22:56:09.611 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-26T22:56:09.611 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi131.front.sepia.ceph.com 2024-05-26T22:56:09.611 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-26T22:56:09.611 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-26T22:56:09.611 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-26T22:56:09.612 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-26T22:56:09.612 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-26T22:56:09.612 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-26T22:56:09.612 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:56:09.612 DEBUG:teuthology.orchestra.run.smithi131:> ip addr 2024-05-26T22:56:09.660 INFO:teuthology.orchestra.run.smithi131.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-26T22:56:09.660 INFO:teuthology.orchestra.run.smithi131.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-26T22:56:09.660 INFO:teuthology.orchestra.run.smithi131.stdout: inet 127.0.0.1/8 scope host lo 2024-05-26T22:56:09.660 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-05-26T22:56:09.660 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 ::1/128 scope host 2024-05-26T22:56:09.660 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-05-26T22:56:09.660 INFO:teuthology.orchestra.run.smithi131.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-26T22:56:09.660 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:d8:fb:18 brd ff:ff:ff:ff:ff:ff 2024-05-26T22:56:09.660 INFO:teuthology.orchestra.run.smithi131.stdout: altname enp7s0f0 2024-05-26T22:56:09.661 INFO:teuthology.orchestra.run.smithi131.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-26T22:56:09.661 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:d8:fb:19 brd ff:ff:ff:ff:ff:ff 2024-05-26T22:56:09.661 INFO:teuthology.orchestra.run.smithi131.stdout: altname enp7s0f1 2024-05-26T22:56:09.661 INFO:teuthology.orchestra.run.smithi131.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-26T22:56:09.661 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:88:72:f8 brd ff:ff:ff:ff:ff:ff 2024-05-26T22:56:09.661 INFO:teuthology.orchestra.run.smithi131.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-26T22:56:09.661 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:88:72:f9 brd ff:ff:ff:ff:ff:ff 2024-05-26T22:56:09.661 INFO:teuthology.orchestra.run.smithi131.stdout: inet 172.21.15.131/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-26T22:56:09.661 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft 42127sec preferred_lft 42127sec 2024-05-26T22:56:09.661 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::ec4:7aff:fe88:72f9/64 scope link 2024-05-26T22:56:09.661 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-05-26T22:56:09.661 INFO:teuthology.orchestra.run.smithi131.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-26T22:56:09.661 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 06:f4:05:86:a4:3e brd ff:ff:ff:ff:ff:ff 2024-05-26T22:56:09.662 INFO:teuthology.orchestra.run.smithi131.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-26T22:56:09.662 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-05-26T22:56:09.662 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::8802:8fff:fe41:fc6d/64 scope link 2024-05-26T22:56:09.662 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-05-26T22:56:09.662 INFO:teuthology.orchestra.run.smithi131.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-26T22:56:09.662 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 06:f4:05:86:a4:3e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-26T22:56:09.662 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::4f4:5ff:fe86:a43e/64 scope link 2024-05-26T22:56:09.662 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-05-26T22:56:09.662 INFO:teuthology.orchestra.run.smithi131.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-26T22:56:09.662 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether ee:6f:35:b2:c0:5b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-26T22:56:09.662 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::ec6f:35ff:feb2:c05b/64 scope link 2024-05-26T22:56:09.662 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-05-26T22:56:09.663 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:56:09.663 DEBUG:teuthology.orchestra.run.smithi131:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-26T22:56:09.737 INFO:teuthology.orchestra.run.smithi131.stdout:1 2024-05-26T22:56:09.741 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:56:09.741 DEBUG:teuthology.orchestra.run.smithi131:> route 2024-05-26T22:56:09.787 INFO:teuthology.orchestra.run.smithi131.stdout:Kernel IP routing table 2024-05-26T22:56:09.787 INFO:teuthology.orchestra.run.smithi131.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-26T22:56:09.787 INFO:teuthology.orchestra.run.smithi131.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-26T22:56:09.787 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-26T22:56:09.787 INFO:teuthology.orchestra.run.smithi131.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-26T22:56:09.788 DEBUG:teuthology.orchestra.run.smithi131:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-26T22:56:09.789 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2024-05-26T22:56:09.789 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-26T22:56:09.789 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-26T22:56:09.789 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-26T22:56:09.789 DEBUG:teuthology.orchestra.run.smithi131:> ') 2024-05-26T22:56:09.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-26T22:56:09.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-26T22:56:09.991 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:56:09.991 DEBUG:teuthology.orchestra.run.smithi131:> ip netns list 2024-05-26T22:56:10.041 INFO:teuthology.orchestra.run.smithi131.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-26T22:56:10.041 INFO:teuthology.orchestra.run.smithi131.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-26T22:56:10.042 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-26T22:56:10.042 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:56:10.043 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link set brx.0 up 2024-05-26T22:56:10.116 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-26T22:56:10.116 DEBUG:teuthology.orchestra.run.smithi131:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-26T22:56:10.162 INFO:teuthology.orchestra.run.smithi131.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-26T22:56:10.162 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-26T22:56:10.162 DEBUG:teuthology.orchestra.run.smithi131:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-26T22:56:10.262 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-05-26T22:56:10.262 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:56:10.262 DEBUG:teuthology.orchestra.run.smithi131:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-26T22:56:10.384 INFO:teuthology.orchestra.run.smithi131.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-26T22:56:10.384 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-26T22:56:10.416 INFO:teuthology.orchestra.run.smithi131.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-26T22:56:10.416 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-26T22:56:10.416 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: resolved to: "172.21.15.131:3300,172.21.15.187:3300,172.21.15.187:3301" 2024-05-26T22:56:10.416 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: trying mount with new device syntax: 0@46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f.cephfs=/ 2024-05-26T22:56:10.416 INFO:teuthology.orchestra.run.smithi131.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.131:3300/172.21.15.187:3300/172.21.15.187:3301" will pass to kernel 2024-05-26T22:56:10.418 INFO:teuthology.orchestra.run.smithi131.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-26T22:56:10.418 INFO:teuthology.orchestra.run.smithi131.stdout: You just mounted a file system that supports labels which does not 2024-05-26T22:56:10.418 INFO:teuthology.orchestra.run.smithi131.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-26T22:56:10.419 INFO:teuthology.orchestra.run.smithi131.stdout: applications will generate AVC messages and not be allowed access to 2024-05-26T22:56:10.419 INFO:teuthology.orchestra.run.smithi131.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-26T22:56:10.422 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-26T22:56:10.422 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:56:10.422 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-26T22:56:10.491 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-26T22:56:10.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:10 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-26T22:56:10.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-26T22:56:10.614 INFO:teuthology.orchestra.run.smithi131.stdout:client5465 2024-05-26T22:56:10.615 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-26T22:56:10.615 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-26T22:56:10.663 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_export_ephemeral_random True 2024-05-26T22:56:10.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:10.915+0000 7fbc61c00640 1 Processor -- start 2024-05-26T22:56:10.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:10.915+0000 7fbc61c00640 1 -- start start 2024-05-26T22:56:10.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:10.917+0000 7fbc61c00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fbc5c0716d0 0x7fbc5c0b56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:10.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:10.917+0000 7fbc61c00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbc5c0b5be0 0x7fbc5c0b6070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:10.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:10.917+0000 7fbc61c00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc5c071d30 con 0x7fbc5c159f40 2024-05-26T22:56:10.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:10.917+0000 7fbc61c00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fbc5c0d0cd0 con 0x7fbc5c0716d0 2024-05-26T22:56:10.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:10.917+0000 7fbc61c00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fbc5c0d0fb0 con 0x7fbc5c0b5be0 2024-05-26T22:56:10.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:10.918+0000 7fbc5b400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbc5c0b5be0 0x7fbc5c0b6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:10.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:10.918+0000 7fbc5b400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbc5c0b5be0 0x7fbc5c0b6070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.131:59652/0 (socket says 172.21.15.131:59652) 2024-05-26T22:56:10.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:10.918+0000 7fbc5b400640 1 -- 172.21.15.131:0/3238011411 learned_addr learned my addr 172.21.15.131:0/3238011411 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:56:10.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:10.918+0000 7fbc5a000640 1 --2- 172.21.15.131:0/3238011411 >> v2:172.21.15.187:3300/0 conn(0x7fbc5c0716d0 0x7fbc5c0b56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:10.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:10.918+0000 7fbc5aa00640 1 --1- 172.21.15.131:0/3238011411 >> v1:172.21.15.187:6789/0 conn(0x7fbc5c159f40 0x7fbc5c070fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.131:39686/0 (socket says 172.21.15.131:39686) 2024-05-26T22:56:10.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:10.919+0000 7fbc5b400640 1 -- 172.21.15.131:0/3238011411 >> v2:172.21.15.187:3300/0 conn(0x7fbc5c0716d0 msgr2=0x7fbc5c0b56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:10.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:10.919+0000 7fbc5b400640 1 --2- 172.21.15.131:0/3238011411 >> v2:172.21.15.187:3300/0 conn(0x7fbc5c0716d0 0x7fbc5c0b56a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:10.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:10.919+0000 7fbc5b400640 1 -- 172.21.15.131:0/3238011411 >> v1:172.21.15.187:6789/0 conn(0x7fbc5c159f40 legacy=0x7fbc5c070fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:10.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:10.919+0000 7fbc5b400640 1 -- 172.21.15.131:0/3238011411 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc5c0d1290 con 0x7fbc5c0b5be0 2024-05-26T22:56:10.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:10.920+0000 7fbc5b400640 1 --2- 172.21.15.131:0/3238011411 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbc5c0b5be0 0x7fbc5c0b6070 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fbc54009e50 tx=0x7fbc5402a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4911a42b261cc6fa server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:10.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:10.920+0000 7fbc59600640 1 -- 172.21.15.131:0/3238011411 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc54036070 con 0x7fbc5c0b5be0 2024-05-26T22:56:10.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:10.920+0000 7fbc59600640 1 -- 172.21.15.131:0/3238011411 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc54031600 con 0x7fbc5c0b5be0 2024-05-26T22:56:10.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:10.920+0000 7fbc59600640 1 -- 172.21.15.131:0/3238011411 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc540319d0 con 0x7fbc5c0b5be0 2024-05-26T22:56:10.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:10.921+0000 7fbc61c00640 1 -- 172.21.15.131:0/3238011411 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbc5c0b5be0 msgr2=0x7fbc5c0b6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:10.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:10.921+0000 7fbc61c00640 1 --2- 172.21.15.131:0/3238011411 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbc5c0b5be0 0x7fbc5c0b6070 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fbc54009e50 tx=0x7fbc5402a710 comp rx=0 tx=0).stop 2024-05-26T22:56:10.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:10.921+0000 7fbc61c00640 1 -- 172.21.15.131:0/3238011411 shutdown_connections 2024-05-26T22:56:10.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:10.921+0000 7fbc61c00640 1 --2- 172.21.15.131:0/3238011411 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbc5c0b5be0 0x7fbc5c0b6070 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:10.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:10.921+0000 7fbc61c00640 1 --2- 172.21.15.131:0/3238011411 >> v2:172.21.15.187:3300/0 conn(0x7fbc5c0716d0 0x7fbc5c0b56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:10.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:10.921+0000 7fbc61c00640 1 -- 172.21.15.131:0/3238011411 >> 172.21.15.131:0/3238011411 conn(0x7fbc5c06f670 msgr2=0x7fbc5c06fa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:56:10.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:10.921+0000 7fbc61c00640 1 -- 172.21.15.131:0/3238011411 shutdown_connections 2024-05-26T22:56:10.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:10.921+0000 7fbc61c00640 1 -- 172.21.15.131:0/3238011411 wait complete. 2024-05-26T22:56:10.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:10.923+0000 7fbc61c00640 1 Processor -- start 2024-05-26T22:56:10.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:10.923+0000 7fbc61c00640 1 -- start start 2024-05-26T22:56:10.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:10.924+0000 7fbc61c00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbc5c0716d0 0x7fbc5c14cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:10.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:10.924+0000 7fbc61c00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbc5c0b5be0 0x7fbc5c0afc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:10.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:10.924+0000 7fbc61c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc5c159f40 0x7fbc5c0b01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:10.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:10.924+0000 7fbc61c00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fbc5c0d1c10 con 0x7fbc5c159f40 2024-05-26T22:56:10.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:10.924+0000 7fbc61c00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fbc5c0d1a90 con 0x7fbc5c0716d0 2024-05-26T22:56:10.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:10.924+0000 7fbc61c00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fbc5c0d1d90 con 0x7fbc5c0b5be0 2024-05-26T22:56:10.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:10.924+0000 7fbc5b400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc5c159f40 0x7fbc5c0b01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:10.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:10.924+0000 7fbc5b400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc5c159f40 0x7fbc5c0b01a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54586/0 (socket says 172.21.15.131:54586) 2024-05-26T22:56:10.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:10.924+0000 7fbc5b400640 1 -- 172.21.15.131:0/4227619029 learned_addr learned my addr 172.21.15.131:0/4227619029 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:56:10.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:10.925+0000 7fbc5aa00640 1 --2- 172.21.15.131:0/4227619029 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbc5c0716d0 0x7fbc5c14cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:10.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:10.925+0000 7fbc5a000640 1 --2- 172.21.15.131:0/4227619029 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbc5c0b5be0 0x7fbc5c0afc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:10.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:10.925+0000 7fbc5b400640 1 -- 172.21.15.131:0/4227619029 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbc5c0b5be0 msgr2=0x7fbc5c0afc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:10.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:10.925+0000 7fbc5b400640 1 --2- 172.21.15.131:0/4227619029 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbc5c0b5be0 0x7fbc5c0afc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:10.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:10.925+0000 7fbc5b400640 1 -- 172.21.15.131:0/4227619029 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbc5c0716d0 msgr2=0x7fbc5c14cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:10.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:10.925+0000 7fbc5b400640 1 --2- 172.21.15.131:0/4227619029 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbc5c0716d0 0x7fbc5c14cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:10.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:10.925+0000 7fbc5b400640 1 -- 172.21.15.131:0/4227619029 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc5c0b36a0 con 0x7fbc5c159f40 2024-05-26T22:56:10.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:10.925+0000 7fbc5b400640 1 --2- 172.21.15.131:0/4227619029 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc5c159f40 0x7fbc5c0b01a0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7fbc54034040 tx=0x7fbc54004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:10.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:10.926+0000 7fbc52a00640 1 -- 172.21.15.131:0/4227619029 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc5403f020 con 0x7fbc5c159f40 2024-05-26T22:56:10.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:10.926+0000 7fbc52a00640 1 -- 172.21.15.131:0/4227619029 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc54037040 con 0x7fbc5c159f40 2024-05-26T22:56:10.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:10.926+0000 7fbc52a00640 1 -- 172.21.15.131:0/4227619029 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc54036070 con 0x7fbc5c159f40 2024-05-26T22:56:10.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:10.926+0000 7fbc61c00640 1 -- 172.21.15.131:0/4227619029 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc5c0b2900 con 0x7fbc5c159f40 2024-05-26T22:56:10.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:10.926+0000 7fbc61c00640 1 -- 172.21.15.131:0/4227619029 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc5c0b2ea0 con 0x7fbc5c159f40 2024-05-26T22:56:10.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:10.927+0000 7fbc61c00640 1 -- 172.21.15.131:0/4227619029 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc5c0695c0 con 0x7fbc5c159f40 2024-05-26T22:56:10.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:10.928+0000 7fbc52a00640 1 -- 172.21.15.131:0/4227619029 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbc54042020 con 0x7fbc5c159f40 2024-05-26T22:56:10.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:10.929+0000 7fbc52a00640 1 --2- 172.21.15.131:0/4227619029 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fbc3c06be30 0x7fbc3c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:10.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:10.929+0000 7fbc52a00640 1 -- 172.21.15.131:0/4227619029 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5709+0+0 (secure 0 0 0) 0x7fbc540ab2d0 con 0x7fbc5c159f40 2024-05-26T22:56:10.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:10.930+0000 7fbc5aa00640 1 --2- 172.21.15.131:0/4227619029 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fbc3c06be30 0x7fbc3c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:10.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:10.930+0000 7fbc5aa00640 1 --2- 172.21.15.131:0/4227619029 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fbc3c06be30 0x7fbc3c06e2f0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c000f60 tx=0x7fbc4c007ce0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:10.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:10.934+0000 7fbc52a00640 1 -- 172.21.15.131:0/4227619029 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fbc54049030 con 0x7fbc5c159f40 2024-05-26T22:56:11.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.133+0000 7fbc61c00640 1 -- 172.21.15.131:0/4227619029 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command([{prefix=config set, name=mds_export_ephemeral_random}] v 0) -- 0x7fbc5c0d2ee0 con 0x7fbc5c159f40 2024-05-26T22:56:11.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.139+0000 7fbc52a00640 1 -- 172.21.15.131:0/4227619029 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_export_ephemeral_random}]=0 v60)=0 v60) ==== 128+0+0 (secure 0 0 0) 0x7fbc54079590 con 0x7fbc5c159f40 2024-05-26T22:56:11.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.142+0000 7fbc61c00640 1 -- 172.21.15.131:0/4227619029 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fbc3c06be30 msgr2=0x7fbc3c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:11.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.142+0000 7fbc61c00640 1 --2- 172.21.15.131:0/4227619029 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fbc3c06be30 0x7fbc3c06e2f0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c000f60 tx=0x7fbc4c007ce0 comp rx=0 tx=0).stop 2024-05-26T22:56:11.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.142+0000 7fbc61c00640 1 -- 172.21.15.131:0/4227619029 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc5c159f40 msgr2=0x7fbc5c0b01a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:11.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.142+0000 7fbc61c00640 1 --2- 172.21.15.131:0/4227619029 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc5c159f40 0x7fbc5c0b01a0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7fbc54034040 tx=0x7fbc54004a40 comp rx=0 tx=0).stop 2024-05-26T22:56:11.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.142+0000 7fbc61c00640 1 -- 172.21.15.131:0/4227619029 shutdown_connections 2024-05-26T22:56:11.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.142+0000 7fbc61c00640 1 --2- 172.21.15.131:0/4227619029 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbc5c0b5be0 0x7fbc5c0afc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:11.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.142+0000 7fbc61c00640 1 --2- 172.21.15.131:0/4227619029 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fbc3c06be30 0x7fbc3c06e2f0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:11.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.142+0000 7fbc61c00640 1 --2- 172.21.15.131:0/4227619029 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc5c159f40 0x7fbc5c0b01a0 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:11.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.142+0000 7fbc61c00640 1 --2- 172.21.15.131:0/4227619029 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbc5c0716d0 0x7fbc5c14cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:11.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.142+0000 7fbc61c00640 1 -- 172.21.15.131:0/4227619029 >> 172.21.15.131:0/4227619029 conn(0x7fbc5c06f670 msgr2=0x7fbc5c15b890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:56:11.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.142+0000 7fbc61c00640 1 -- 172.21.15.131:0/4227619029 shutdown_connections 2024-05-26T22:56:11.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.142+0000 7fbc61c00640 1 -- 172.21.15.131:0/4227619029 wait complete. 2024-05-26T22:56:11.160 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_export_ephemeral_distributed True 2024-05-26T22:56:11.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.361+0000 7fd98a000640 1 Processor -- start 2024-05-26T22:56:11.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.362+0000 7fd98a000640 1 -- start start 2024-05-26T22:56:11.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.363+0000 7fd98a000640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fd9840b5450 0x7fd9840b43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:11.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.363+0000 7fd98a000640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fd98406f420 0x7fd9840cfd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:11.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.363+0000 7fd98a000640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9840b4900 con 0x7fd9840cf590 2024-05-26T22:56:11.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.363+0000 7fd98a000640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fd984160d90 con 0x7fd9840b5450 2024-05-26T22:56:11.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.363+0000 7fd98a000640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fd984160180 con 0x7fd98406f420 2024-05-26T22:56:11.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.364+0000 7fd983400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fd9840b5450 0x7fd9840b43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:11.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.364+0000 7fd983400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fd9840b5450 0x7fd9840b43c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54594/0 (socket says 172.21.15.131:54594) 2024-05-26T22:56:11.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.364+0000 7fd983400640 1 -- 172.21.15.131:0/3513174117 learned_addr learned my addr 172.21.15.131:0/3513174117 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:56:11.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.364+0000 7fd982a00640 1 --2- 172.21.15.131:0/3513174117 >> v2:172.21.15.187:3300/0 conn(0x7fd98406f420 0x7fd9840cfd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:11.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.364+0000 7fd983400640 1 -- 172.21.15.131:0/3513174117 >> v2:172.21.15.187:3300/0 conn(0x7fd98406f420 msgr2=0x7fd9840cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:11.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.364+0000 7fd983400640 1 --2- 172.21.15.131:0/3513174117 >> v2:172.21.15.187:3300/0 conn(0x7fd98406f420 0x7fd9840cfd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:11.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.364+0000 7fd983400640 1 -- 172.21.15.131:0/3513174117 >> v1:172.21.15.187:6789/0 conn(0x7fd9840cf590 legacy=0x7fd98406d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:11.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.364+0000 7fd983400640 1 -- 172.21.15.131:0/3513174117 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd984160460 con 0x7fd9840b5450 2024-05-26T22:56:11.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.366+0000 7fd983400640 1 --2- 172.21.15.131:0/3513174117 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd9840b5450 0x7fd9840b43c0 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7fd9840b35b0 tx=0x7fd97402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d94c402ed4ffaed server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:11.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.366+0000 7fd982000640 1 -- 172.21.15.131:0/3513174117 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd974037070 con 0x7fd9840b5450 2024-05-26T22:56:11.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.366+0000 7fd982000640 1 -- 172.21.15.131:0/3513174117 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd974032440 con 0x7fd9840b5450 2024-05-26T22:56:11.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.366+0000 7fd982000640 1 -- 172.21.15.131:0/3513174117 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd974032810 con 0x7fd9840b5450 2024-05-26T22:56:11.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.367+0000 7fd98a000640 1 -- 172.21.15.131:0/3513174117 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd9840b5450 msgr2=0x7fd9840b43c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:11.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.367+0000 7fd98a000640 1 --2- 172.21.15.131:0/3513174117 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd9840b5450 0x7fd9840b43c0 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7fd9840b35b0 tx=0x7fd97402a230 comp rx=0 tx=0).stop 2024-05-26T22:56:11.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.367+0000 7fd98a000640 1 -- 172.21.15.131:0/3513174117 shutdown_connections 2024-05-26T22:56:11.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.367+0000 7fd98a000640 1 --2- 172.21.15.131:0/3513174117 >> v2:172.21.15.187:3300/0 conn(0x7fd98406f420 0x7fd9840cfd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:11.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.367+0000 7fd98a000640 1 --2- 172.21.15.131:0/3513174117 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd9840b5450 0x7fd9840b43c0 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:11.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.367+0000 7fd98a000640 1 -- 172.21.15.131:0/3513174117 >> 172.21.15.131:0/3513174117 conn(0x7fd98406bc90 msgr2=0x7fd9840b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:56:11.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.367+0000 7fd98a000640 1 -- 172.21.15.131:0/3513174117 shutdown_connections 2024-05-26T22:56:11.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.367+0000 7fd98a000640 1 -- 172.21.15.131:0/3513174117 wait complete. 2024-05-26T22:56:11.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.369+0000 7fd98a000640 1 Processor -- start 2024-05-26T22:56:11.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.369+0000 7fd98a000640 1 -- start start 2024-05-26T22:56:11.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.371+0000 7fd98a000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd98406f420 0x7fd98415e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:11.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.371+0000 7fd98a000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd9840b5450 0x7fd98415e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:11.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.371+0000 7fd98a000640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd9840cf590 0x7fd984071cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:11.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.371+0000 7fd98a000640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fd9840b5830 con 0x7fd9840b5450 2024-05-26T22:56:11.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.371+0000 7fd98a000640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd984160a20 con 0x7fd9840cf590 2024-05-26T22:56:11.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.371+0000 7fd98a000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd98415fde0 con 0x7fd98406f420 2024-05-26T22:56:11.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.371+0000 7fd982a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd9840b5450 0x7fd98415e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:11.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.371+0000 7fd982a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd9840b5450 0x7fd98415e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54602/0 (socket says 172.21.15.131:54602) 2024-05-26T22:56:11.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.371+0000 7fd982a00640 1 -- 172.21.15.131:0/3346195677 learned_addr learned my addr 172.21.15.131:0/3346195677 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:56:11.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.371+0000 7fd983400640 1 --2- 172.21.15.131:0/3346195677 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd98406f420 0x7fd98415e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:11.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.371+0000 7fd983e00640 1 --2- 172.21.15.131:0/3346195677 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd9840cf590 0x7fd984071cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:11.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.371+0000 7fd982a00640 1 -- 172.21.15.131:0/3346195677 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd98406f420 msgr2=0x7fd98415e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:11.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.371+0000 7fd982a00640 1 --2- 172.21.15.131:0/3346195677 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd98406f420 0x7fd98415e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:11.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.372+0000 7fd982a00640 1 -- 172.21.15.131:0/3346195677 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd9840cf590 msgr2=0x7fd984071cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:11.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.372+0000 7fd982a00640 1 --2- 172.21.15.131:0/3346195677 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd9840cf590 0x7fd984071cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:11.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.372+0000 7fd982a00640 1 -- 172.21.15.131:0/3346195677 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd98415ef70 con 0x7fd9840b5450 2024-05-26T22:56:11.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.372+0000 7fd983e00640 1 --2- 172.21.15.131:0/3346195677 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd9840cf590 0x7fd984071cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:56:11.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.372+0000 7fd982a00640 1 --2- 172.21.15.131:0/3346195677 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd9840b5450 0x7fd98415e710 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7fd97000d5f0 tx=0x7fd97000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:11.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.372+0000 7fd97b400640 1 -- 172.21.15.131:0/3346195677 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9700125f0 con 0x7fd9840b5450 2024-05-26T22:56:11.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.372+0000 7fd97b400640 1 -- 172.21.15.131:0/3346195677 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd970019070 con 0x7fd9840b5450 2024-05-26T22:56:11.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.372+0000 7fd97b400640 1 -- 172.21.15.131:0/3346195677 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9700129d0 con 0x7fd9840b5450 2024-05-26T22:56:11.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.372+0000 7fd98a000640 1 -- 172.21.15.131:0/3346195677 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd98409ac50 con 0x7fd9840b5450 2024-05-26T22:56:11.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.372+0000 7fd98a000640 1 -- 172.21.15.131:0/3346195677 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd98415ec50 con 0x7fd9840b5450 2024-05-26T22:56:11.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.373+0000 7fd98a000640 1 -- 172.21.15.131:0/3346195677 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd984074e90 con 0x7fd9840b5450 2024-05-26T22:56:11.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.374+0000 7fd97b400640 1 -- 172.21.15.131:0/3346195677 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd970012b70 con 0x7fd9840b5450 2024-05-26T22:56:11.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.375+0000 7fd97b400640 1 --2- 172.21.15.131:0/3346195677 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fd96406be60 0x7fd96406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:11.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.375+0000 7fd97b400640 1 -- 172.21.15.131:0/3346195677 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5709+0+0 (secure 0 0 0) 0x7fd97008b2e0 con 0x7fd9840b5450 2024-05-26T22:56:11.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.376+0000 7fd983400640 1 --2- 172.21.15.131:0/3346195677 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fd96406be60 0x7fd96406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:11.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.376+0000 7fd983400640 1 --2- 172.21.15.131:0/3346195677 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fd96406be60 0x7fd96406e320 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fd974034000 tx=0x7fd97402e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:11.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.380+0000 7fd97b400640 1 -- 172.21.15.131:0/3346195677 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fd970059bf0 con 0x7fd9840b5450 2024-05-26T22:56:11.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.582+0000 7fd98a000640 1 -- 172.21.15.131:0/3346195677 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command([{prefix=config set, name=mds_export_ephemeral_distributed}] v 0) -- 0x7fd9840695c0 con 0x7fd9840b5450 2024-05-26T22:56:11.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.587+0000 7fd97b400640 1 -- 172.21.15.131:0/3346195677 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_export_ephemeral_distributed}]=0 v61)=0 v61) ==== 133+0+0 (secure 0 0 0) 0x7fd97008b0c0 con 0x7fd9840b5450 2024-05-26T22:56:11.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.589+0000 7fd98a000640 1 -- 172.21.15.131:0/3346195677 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fd96406be60 msgr2=0x7fd96406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:11.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.590+0000 7fd98a000640 1 --2- 172.21.15.131:0/3346195677 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fd96406be60 0x7fd96406e320 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fd974034000 tx=0x7fd97402e000 comp rx=0 tx=0).stop 2024-05-26T22:56:11.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.590+0000 7fd98a000640 1 -- 172.21.15.131:0/3346195677 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd9840b5450 msgr2=0x7fd98415e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:11.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.590+0000 7fd98a000640 1 --2- 172.21.15.131:0/3346195677 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd9840b5450 0x7fd98415e710 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7fd97000d5f0 tx=0x7fd97000dab0 comp rx=0 tx=0).stop 2024-05-26T22:56:11.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.590+0000 7fd98a000640 1 -- 172.21.15.131:0/3346195677 shutdown_connections 2024-05-26T22:56:11.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.590+0000 7fd98a000640 1 --2- 172.21.15.131:0/3346195677 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fd96406be60 0x7fd96406e320 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:11.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.590+0000 7fd98a000640 1 --2- 172.21.15.131:0/3346195677 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd9840cf590 0x7fd984071cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:11.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.590+0000 7fd98a000640 1 --2- 172.21.15.131:0/3346195677 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd9840b5450 0x7fd98415e710 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:11.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.590+0000 7fd98a000640 1 --2- 172.21.15.131:0/3346195677 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd98406f420 0x7fd98415e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:11.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.590+0000 7fd98a000640 1 -- 172.21.15.131:0/3346195677 >> 172.21.15.131:0/3346195677 conn(0x7fd98406bc90 msgr2=0x7fd98415a430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:56:11.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.590+0000 7fd98a000640 1 -- 172.21.15.131:0/3346195677 shutdown_connections 2024-05-26T22:56:11.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.590+0000 7fd98a000640 1 -- 172.21.15.131:0/3346195677 wait complete. 2024-05-26T22:56:11.609 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_export_ephemeral_random_max 1.0 2024-05-26T22:56:11.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.812+0000 7f481e600640 1 Processor -- start 2024-05-26T22:56:11.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.813+0000 7f481e600640 1 -- start start 2024-05-26T22:56:11.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.814+0000 7f481e600640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48180cbef0 0x7f48180cc2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:11.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.814+0000 7f481e600640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f48180ad370 0x7f48180cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:11.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.814+0000 7f481e600640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48180cdca0 con 0x7f48180acef0 2024-05-26T22:56:11.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.815+0000 7f481e600640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f48180cdf80 con 0x7f48180ad370 2024-05-26T22:56:11.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.815+0000 7f481e600640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f48180ce260 con 0x7f48180cbef0 2024-05-26T22:56:11.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.815+0000 7f4816a00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f48180acef0 0x7f481806d320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:45832/0 (socket says 172.21.15.131:45832) 2024-05-26T22:56:11.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.815+0000 7f4817e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f48180ad370 0x7f48180cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:11.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.815+0000 7f4816a00640 1 -- 172.21.15.131:0/2035189824 learned_addr learned my addr 172.21.15.131:0/2035189824 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:56:11.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.815+0000 7f4817e00640 1 -- 172.21.15.131:0/2035189824 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48180cbef0 msgr2=0x7f48180cc2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:11.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.815+0000 7f4817400640 1 --2- 172.21.15.131:0/2035189824 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48180cbef0 0x7f48180cc2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:11.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.815+0000 7f4817e00640 1 --2- 172.21.15.131:0/2035189824 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48180cbef0 0x7f48180cc2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:11.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.815+0000 7f4817e00640 1 -- 172.21.15.131:0/2035189824 >> v1:172.21.15.131:6789/0 conn(0x7f48180acef0 legacy=0x7f481806d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:11.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.815+0000 7f4817e00640 1 -- 172.21.15.131:0/2035189824 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48180ce540 con 0x7f48180ad370 2024-05-26T22:56:11.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.815+0000 7f4816000640 1 -- 172.21.15.131:0/2035189824 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3343776210 0 0) 0x7f48180cdca0 con 0x7f48180acef0 2024-05-26T22:56:11.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.815+0000 7f4817400640 1 --2- 172.21.15.131:0/2035189824 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48180cbef0 0x7f48180cc2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:56:11.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.816+0000 7f4817e00640 1 --2- 172.21.15.131:0/2035189824 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48180ad370 0x7f48180cea60 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f4810009dc0 tx=0x7f481002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8678368e62b01a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:11.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.817+0000 7f4816000640 1 -- 172.21.15.131:0/2035189824 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f481002ae60 con 0x7f48180ad370 2024-05-26T22:56:11.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.817+0000 7f4816000640 1 -- 172.21.15.131:0/2035189824 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4810007740 con 0x7f48180ad370 2024-05-26T22:56:11.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.817+0000 7f4816000640 1 -- 172.21.15.131:0/2035189824 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4810007b10 con 0x7f48180ad370 2024-05-26T22:56:11.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.817+0000 7f481e600640 1 -- 172.21.15.131:0/2035189824 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48180ad370 msgr2=0x7f48180cea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:11.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.817+0000 7f481e600640 1 --2- 172.21.15.131:0/2035189824 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48180ad370 0x7f48180cea60 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f4810009dc0 tx=0x7f481002a680 comp rx=0 tx=0).stop 2024-05-26T22:56:11.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.818+0000 7f481e600640 1 -- 172.21.15.131:0/2035189824 shutdown_connections 2024-05-26T22:56:11.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.818+0000 7f481e600640 1 --2- 172.21.15.131:0/2035189824 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48180ad370 0x7f48180cea60 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:11.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.818+0000 7f481e600640 1 --2- 172.21.15.131:0/2035189824 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48180cbef0 0x7f48180cc2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:11.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.818+0000 7f481e600640 1 -- 172.21.15.131:0/2035189824 >> 172.21.15.131:0/2035189824 conn(0x7f481806bc90 msgr2=0x7f48180b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:56:11.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.818+0000 7f481e600640 1 -- 172.21.15.131:0/2035189824 shutdown_connections 2024-05-26T22:56:11.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.818+0000 7f481e600640 1 -- 172.21.15.131:0/2035189824 wait complete. 2024-05-26T22:56:11.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.819+0000 7f481e600640 1 Processor -- start 2024-05-26T22:56:11.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.820+0000 7f481e600640 1 -- start start 2024-05-26T22:56:11.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.820+0000 7f481e600640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f48180acef0 0x7f481814e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:11.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.820+0000 7f481e600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48180ad370 0x7f4818159730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:11.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.820+0000 7f481e600640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48180cbef0 0x7f4818159c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:11.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.820+0000 7f481e600640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f48180cefa0 con 0x7f48180ad370 2024-05-26T22:56:11.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.820+0000 7f481e600640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f481806dc40 con 0x7f48180acef0 2024-05-26T22:56:11.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.820+0000 7f481e600640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f48180cf120 con 0x7f48180cbef0 2024-05-26T22:56:11.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.821+0000 7f4816a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48180ad370 0x7f4818159730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:11.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.821+0000 7f4816a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48180ad370 0x7f4818159730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54620/0 (socket says 172.21.15.131:54620) 2024-05-26T22:56:11.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.821+0000 7f4816a00640 1 -- 172.21.15.131:0/2915194318 learned_addr learned my addr 172.21.15.131:0/2915194318 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:56:11.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.821+0000 7f4817e00640 1 --2- 172.21.15.131:0/2915194318 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48180cbef0 0x7f4818159c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:11.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.821+0000 7f4817400640 1 --2- 172.21.15.131:0/2915194318 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f48180acef0 0x7f481814e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:11.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.821+0000 7f4816a00640 1 -- 172.21.15.131:0/2915194318 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48180cbef0 msgr2=0x7f4818159c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:11.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.821+0000 7f4816a00640 1 --2- 172.21.15.131:0/2915194318 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48180cbef0 0x7f4818159c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:11.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.821+0000 7f4816a00640 1 -- 172.21.15.131:0/2915194318 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f48180acef0 msgr2=0x7f481814e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:11.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.821+0000 7f4816a00640 1 --2- 172.21.15.131:0/2915194318 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f48180acef0 0x7f481814e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:11.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.821+0000 7f4816a00640 1 -- 172.21.15.131:0/2915194318 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f481809b650 con 0x7f48180ad370 2024-05-26T22:56:11.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.821+0000 7f4816a00640 1 --2- 172.21.15.131:0/2915194318 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48180ad370 0x7f4818159730 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f480400c430 tx=0x7f480400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:11.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.822+0000 7f480f400640 1 -- 172.21.15.131:0/2915194318 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48040074e0 con 0x7f48180ad370 2024-05-26T22:56:11.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.822+0000 7f480f400640 1 -- 172.21.15.131:0/2915194318 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48040047e0 con 0x7f48180ad370 2024-05-26T22:56:11.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.822+0000 7f481e600640 1 -- 172.21.15.131:0/2915194318 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f481809b930 con 0x7f48180ad370 2024-05-26T22:56:11.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.822+0000 7f480f400640 1 -- 172.21.15.131:0/2915194318 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4804004bd0 con 0x7f48180ad370 2024-05-26T22:56:11.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.822+0000 7f481e600640 1 -- 172.21.15.131:0/2915194318 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f481809bef0 con 0x7f48180ad370 2024-05-26T22:56:11.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.823+0000 7f481e600640 1 -- 172.21.15.131:0/2915194318 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48180695c0 con 0x7f48180ad370 2024-05-26T22:56:11.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.824+0000 7f480f400640 1 -- 172.21.15.131:0/2915194318 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4804011070 con 0x7f48180ad370 2024-05-26T22:56:11.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.825+0000 7f480f400640 1 --2- 172.21.15.131:0/2915194318 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f47f806be30 0x7f47f806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:11.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.825+0000 7f480f400640 1 -- 172.21.15.131:0/2915194318 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5709+0+0 (secure 0 0 0) 0x7f480408aba0 con 0x7f48180ad370 2024-05-26T22:56:11.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.825+0000 7f4817400640 1 --2- 172.21.15.131:0/2915194318 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f47f806be30 0x7f47f806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:11.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.826+0000 7f4817400640 1 --2- 172.21.15.131:0/2915194318 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f47f806be30 0x7f47f806e2f0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f4808008850 tx=0x7f4808008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:11.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:11.830+0000 7f480f400640 1 -- 172.21.15.131:0/2915194318 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f48040594b0 con 0x7f48180ad370 2024-05-26T22:56:12.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:12.027+0000 7f481e600640 1 -- 172.21.15.131:0/2915194318 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command([{prefix=config set, name=mds_export_ephemeral_random_max}] v 0) -- 0x7f48180cd060 con 0x7f48180ad370 2024-05-26T22:56:12.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:12.032+0000 7f480f400640 1 -- 172.21.15.131:0/2915194318 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_export_ephemeral_random_max}]=0 v62)=0 v62) ==== 131+0+0 (secure 0 0 0) 0x7f4804058db0 con 0x7f48180ad370 2024-05-26T22:56:12.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:12.036+0000 7f481e600640 1 -- 172.21.15.131:0/2915194318 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f47f806be30 msgr2=0x7f47f806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:12.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:12.036+0000 7f481e600640 1 --2- 172.21.15.131:0/2915194318 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f47f806be30 0x7f47f806e2f0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f4808008850 tx=0x7f4808008040 comp rx=0 tx=0).stop 2024-05-26T22:56:12.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:12.036+0000 7f481e600640 1 -- 172.21.15.131:0/2915194318 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48180ad370 msgr2=0x7f4818159730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:12.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:12.036+0000 7f481e600640 1 --2- 172.21.15.131:0/2915194318 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48180ad370 0x7f4818159730 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f480400c430 tx=0x7f480400c8f0 comp rx=0 tx=0).stop 2024-05-26T22:56:12.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:12.036+0000 7f481e600640 1 -- 172.21.15.131:0/2915194318 shutdown_connections 2024-05-26T22:56:12.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:12.036+0000 7f481e600640 1 --2- 172.21.15.131:0/2915194318 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f48180cbef0 0x7f4818159c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:12.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:12.036+0000 7f481e600640 1 --2- 172.21.15.131:0/2915194318 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f47f806be30 0x7f47f806e2f0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:12.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:12.036+0000 7f481e600640 1 --2- 172.21.15.131:0/2915194318 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48180ad370 0x7f4818159730 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:12.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:12.036+0000 7f481e600640 1 --2- 172.21.15.131:0/2915194318 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f48180acef0 0x7f481814e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:12.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:12.036+0000 7f481e600640 1 -- 172.21.15.131:0/2915194318 >> 172.21.15.131:0/2915194318 conn(0x7f481806bc90 msgr2=0x7f48181549a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:56:12.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:12.036+0000 7f481e600640 1 -- 172.21.15.131:0/2915194318 shutdown_connections 2024-05-26T22:56:12.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:12.036+0000 7f481e600640 1 -- 172.21.15.131:0/2915194318 wait complete. 2024-05-26T22:56:12.055 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-05-26T22:56:12.056 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2024-05-26T22:56:12.056 DEBUG:teuthology.orchestra.run.smithi131:> 2024-05-26T22:56:12.056 DEBUG:teuthology.orchestra.run.smithi131:> # Use up a random number of inode numbers so the ephemeral pinning is not the same every test. 2024-05-26T22:56:12.056 DEBUG:teuthology.orchestra.run.smithi131:> mkdir .inode_number_thrash 2024-05-26T22:56:12.056 DEBUG:teuthology.orchestra.run.smithi131:> count=$((RANDOM % 1024)) 2024-05-26T22:56:12.056 DEBUG:teuthology.orchestra.run.smithi131:> for ((i = 0; i < count; i++)); do touch .inode_number_thrash/$i; done 2024-05-26T22:56:12.056 DEBUG:teuthology.orchestra.run.smithi131:> rm -rf .inode_number_thrash 2024-05-26T22:56:12.056 DEBUG:teuthology.orchestra.run.smithi131:> ') 2024-05-26T22:56:12.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:12 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-26T22:56:15.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-26T22:56:15.191 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2024-05-26T22:56:15.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:15.378+0000 7f73a8200640 1 Processor -- start 2024-05-26T22:56:15.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:15.378+0000 7f73a8200640 1 -- start start 2024-05-26T22:56:15.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:15.380+0000 7f73a8200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f73a006f180 0x7f73a0072e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:15.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:15.380+0000 7f73a8200640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f73a00d0560 0x7f73a0161ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:15.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:15.380+0000 7f73a8200640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73a00af8e0 con 0x7f73a0072540 2024-05-26T22:56:15.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:15.380+0000 7f73a8200640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f73a00cfb30 con 0x7f73a00d0560 2024-05-26T22:56:15.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:15.380+0000 7f73a8200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f73a00cf200 con 0x7f73a006f180 2024-05-26T22:56:15.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:15.381+0000 7f73a5800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f73a006f180 0x7f73a0072e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:15.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:15.381+0000 7f73a5800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f73a006f180 0x7f73a0072e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.131:39834/0 (socket says 172.21.15.131:39834) 2024-05-26T22:56:15.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:15.381+0000 7f73a5800640 1 -- 172.21.15.131:0/924591373 learned_addr learned my addr 172.21.15.131:0/924591373 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:56:15.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:15.381+0000 7f73a6200640 1 --2- 172.21.15.131:0/924591373 >> v2:172.21.15.187:3300/0 conn(0x7f73a00d0560 0x7f73a0161ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:15.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:15.382+0000 7f739fe00640 1 -- 172.21.15.131:0/924591373 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2313013323 0 0) 0x7f73a00af8e0 con 0x7f73a0072540 2024-05-26T22:56:15.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:15.382+0000 7f73a5800640 1 -- 172.21.15.131:0/924591373 >> v2:172.21.15.187:3300/0 conn(0x7f73a00d0560 msgr2=0x7f73a0161ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:15.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:15.382+0000 7f73a5800640 1 --2- 172.21.15.131:0/924591373 >> v2:172.21.15.187:3300/0 conn(0x7f73a00d0560 0x7f73a0161ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:15.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:15.382+0000 7f73a5800640 1 -- 172.21.15.131:0/924591373 >> v1:172.21.15.187:6789/0 conn(0x7f73a0072540 legacy=0x7f73a00cfe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:15.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:15.382+0000 7f73a5800640 1 -- 172.21.15.131:0/924591373 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73a00cf4e0 con 0x7f73a006f180 2024-05-26T22:56:15.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:15.383+0000 7f73a5800640 1 --2- 172.21.15.131:0/924591373 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f73a006f180 0x7f73a0072e90 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7394009870 tx=0x7f739402a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a0f92f000b2ac5a server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:15.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:15.383+0000 7f739fe00640 1 -- 172.21.15.131:0/924591373 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7394037070 con 0x7f73a006f180 2024-05-26T22:56:15.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:15.383+0000 7f739fe00640 1 -- 172.21.15.131:0/924591373 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7394032440 con 0x7f73a006f180 2024-05-26T22:56:15.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:15.383+0000 7f739fe00640 1 -- 172.21.15.131:0/924591373 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7394032810 con 0x7f73a006f180 2024-05-26T22:56:15.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:15.384+0000 7f73a8200640 1 -- 172.21.15.131:0/924591373 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f73a006f180 msgr2=0x7f73a0072e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:15.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:15.384+0000 7f73a8200640 1 --2- 172.21.15.131:0/924591373 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f73a006f180 0x7f73a0072e90 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7394009870 tx=0x7f739402a250 comp rx=0 tx=0).stop 2024-05-26T22:56:15.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:15.384+0000 7f73a8200640 1 -- 172.21.15.131:0/924591373 shutdown_connections 2024-05-26T22:56:15.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:15.384+0000 7f73a8200640 1 --2- 172.21.15.131:0/924591373 >> v2:172.21.15.187:3300/0 conn(0x7f73a00d0560 0x7f73a0161ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:15.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:15.384+0000 7f73a8200640 1 --2- 172.21.15.131:0/924591373 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f73a006f180 0x7f73a0072e90 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:15.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:15.384+0000 7f73a8200640 1 -- 172.21.15.131:0/924591373 >> 172.21.15.131:0/924591373 conn(0x7f73a006bc90 msgr2=0x7f73a00b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:56:15.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:15.384+0000 7f73a8200640 1 -- 172.21.15.131:0/924591373 shutdown_connections 2024-05-26T22:56:15.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:15.384+0000 7f73a8200640 1 -- 172.21.15.131:0/924591373 wait complete. 2024-05-26T22:56:15.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:15.386+0000 7f73a8200640 1 Processor -- start 2024-05-26T22:56:15.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:15.387+0000 7f73a8200640 1 -- start start 2024-05-26T22:56:15.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:15.388+0000 7f73a8200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f73a006f180 0x7f73a014dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:15.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:15.388+0000 7f73a8200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f73a0072540 0x7f73a015d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:15.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:15.388+0000 7f73a8200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f73a00d0560 0x7f73a015dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:15.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:15.388+0000 7f73a8200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f73a006f6e0 con 0x7f73a006f180 2024-05-26T22:56:15.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:15.388+0000 7f73a8200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f73a006f560 con 0x7f73a0072540 2024-05-26T22:56:15.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:15.388+0000 7f73a8200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f73a00715a0 con 0x7f73a00d0560 2024-05-26T22:56:15.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:15.388+0000 7f73a5800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f73a006f180 0x7f73a014dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:15.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:15.388+0000 7f73a5800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f73a006f180 0x7f73a014dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55902/0 (socket says 172.21.15.131:55902) 2024-05-26T22:56:15.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:15.388+0000 7f73a5800640 1 -- 172.21.15.131:0/3256415291 learned_addr learned my addr 172.21.15.131:0/3256415291 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:56:15.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:15.389+0000 7f73a4e00640 1 --2- 172.21.15.131:0/3256415291 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f73a0072540 0x7f73a015d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:15.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:15.389+0000 7f73a6200640 1 --2- 172.21.15.131:0/3256415291 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f73a00d0560 0x7f73a015dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:15.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:15.389+0000 7f73a5800640 1 -- 172.21.15.131:0/3256415291 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f73a00d0560 msgr2=0x7f73a015dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:15.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:15.389+0000 7f73a5800640 1 --2- 172.21.15.131:0/3256415291 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f73a00d0560 0x7f73a015dce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:15.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:15.389+0000 7f73a5800640 1 -- 172.21.15.131:0/3256415291 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f73a0072540 msgr2=0x7f73a015d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:15.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:15.389+0000 7f73a5800640 1 --2- 172.21.15.131:0/3256415291 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f73a0072540 0x7f73a015d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:15.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:15.389+0000 7f73a5800640 1 -- 172.21.15.131:0/3256415291 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73a00c95c0 con 0x7f73a006f180 2024-05-26T22:56:15.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:15.389+0000 7f73a4e00640 1 --2- 172.21.15.131:0/3256415291 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f73a0072540 0x7f73a015d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:56:15.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:15.389+0000 7f73a5800640 1 --2- 172.21.15.131:0/3256415291 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f73a006f180 0x7f73a014dad0 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f7394000c00 tx=0x7f73940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:15.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:15.390+0000 7f739d600640 1 -- 172.21.15.131:0/3256415291 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7394037040 con 0x7f73a006f180 2024-05-26T22:56:15.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:15.390+0000 7f739d600640 1 -- 172.21.15.131:0/3256415291 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7394042070 con 0x7f73a006f180 2024-05-26T22:56:15.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:15.390+0000 7f73a8200640 1 -- 172.21.15.131:0/3256415291 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73a01f8140 con 0x7f73a006f180 2024-05-26T22:56:15.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:15.390+0000 7f739d600640 1 -- 172.21.15.131:0/3256415291 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7394030880 con 0x7f73a006f180 2024-05-26T22:56:15.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:15.390+0000 7f73a8200640 1 -- 172.21.15.131:0/3256415291 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73a01f43c0 con 0x7f73a006f180 2024-05-26T22:56:15.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:15.391+0000 7f73a8200640 1 -- 172.21.15.131:0/3256415291 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73a0074e90 con 0x7f73a006f180 2024-05-26T22:56:15.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:15.392+0000 7f739d600640 1 -- 172.21.15.131:0/3256415291 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7394030a20 con 0x7f73a006f180 2024-05-26T22:56:15.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:15.393+0000 7f739d600640 1 --2- 172.21.15.131:0/3256415291 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f738006be60 0x7f738006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:15.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:15.393+0000 7f739d600640 1 -- 172.21.15.131:0/3256415291 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5709+0+0 (secure 0 0 0) 0x7f73940ab770 con 0x7f73a006f180 2024-05-26T22:56:15.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:15.393+0000 7f73a4e00640 1 --2- 172.21.15.131:0/3256415291 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f738006be60 0x7f738006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:15.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:15.394+0000 7f73a4e00640 1 --2- 172.21.15.131:0/3256415291 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f738006be60 0x7f738006e320 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f738c007fd0 tx=0x7f738c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:15.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:15.398+0000 7f739d600640 1 -- 172.21.15.131:0/3256415291 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f73a00c95c0 con 0x7f73a006f180 2024-05-26T22:56:15.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:15.702+0000 7f73a8200640 1 -- 172.21.15.131:0/3256415291 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7f73a00695c0 con 0x7f73a006f180 2024-05-26T22:56:15.983 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-26T22:56:15.992 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-26T22:56:16.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:16.152+0000 7f739d600640 1 -- 172.21.15.131:0/3256415291 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v111) ==== 105+0+0 (secure 0 0 0) 0x7f739407a080 con 0x7f73a006f180 2024-05-26T22:56:16.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:16.156+0000 7f73a8200640 1 -- 172.21.15.131:0/3256415291 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f738006be60 msgr2=0x7f738006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:16.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:16.156+0000 7f73a8200640 1 --2- 172.21.15.131:0/3256415291 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f738006be60 0x7f738006e320 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f738c007fd0 tx=0x7f738c009040 comp rx=0 tx=0).stop 2024-05-26T22:56:16.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:16.156+0000 7f73a8200640 1 -- 172.21.15.131:0/3256415291 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f73a006f180 msgr2=0x7f73a014dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:16.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:16.156+0000 7f73a8200640 1 --2- 172.21.15.131:0/3256415291 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f73a006f180 0x7f73a014dad0 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f7394000c00 tx=0x7f73940046c0 comp rx=0 tx=0).stop 2024-05-26T22:56:16.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:16.156+0000 7f73a8200640 1 -- 172.21.15.131:0/3256415291 shutdown_connections 2024-05-26T22:56:16.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:16.156+0000 7f73a8200640 1 --2- 172.21.15.131:0/3256415291 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f73a00d0560 0x7f73a015dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:16.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:16.156+0000 7f73a8200640 1 --2- 172.21.15.131:0/3256415291 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f738006be60 0x7f738006e320 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:16.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:16.156+0000 7f73a8200640 1 --2- 172.21.15.131:0/3256415291 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f73a0072540 0x7f73a015d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:16.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:16.156+0000 7f73a8200640 1 --2- 172.21.15.131:0/3256415291 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f73a006f180 0x7f73a014dad0 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:16.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:16.156+0000 7f73a8200640 1 -- 172.21.15.131:0/3256415291 >> 172.21.15.131:0/3256415291 conn(0x7f73a006bc90 msgr2=0x7f73a00b0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:56:16.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:16.156+0000 7f73a8200640 1 -- 172.21.15.131:0/3256415291 shutdown_connections 2024-05-26T22:56:16.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:16.156+0000 7f73a8200640 1 -- 172.21.15.131:0/3256415291 wait complete. 2024-05-26T22:56:16.182 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-26T22:56:16.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:16.357+0000 7f037fe00640 1 Processor -- start 2024-05-26T22:56:16.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:16.357+0000 7f037fe00640 1 -- start start 2024-05-26T22:56:16.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:16.358+0000 7f037fe00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f038006e8c0 0x7f038015d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:16.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:16.358+0000 7f037fe00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0380160620 con 0x7f0380156750 2024-05-26T22:56:16.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:16.358+0000 7f037fe00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0380160900 con 0x7f03800cc7f0 2024-05-26T22:56:16.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:16.358+0000 7f037fe00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f038015cbd0 con 0x7f038006e8c0 2024-05-26T22:56:16.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:16.359+0000 7f037ea00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f0380156750 0x7f038015fd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:50078/0 (socket says 172.21.15.131:50078) 2024-05-26T22:56:16.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:16.359+0000 7f037ea00640 1 -- 172.21.15.131:0/2279478291 learned_addr learned my addr 172.21.15.131:0/2279478291 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:56:16.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:16.359+0000 7f037d600640 1 -- 172.21.15.131:0/2279478291 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4141113395 0 0) 0x7f0380160620 con 0x7f0380156750 2024-05-26T22:56:16.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:16.359+0000 7f037d600640 1 -- 172.21.15.131:0/2279478291 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0364003520 con 0x7f0380156750 2024-05-26T22:56:16.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:16.359+0000 7f037e000640 1 --2- 172.21.15.131:0/2279478291 >> v2:172.21.15.187:3300/0 conn(0x7f038006e8c0 0x7f038015d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:16.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:16.360+0000 7f037d600640 1 -- 172.21.15.131:0/2279478291 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3443745053 0 0) 0x7f0364003520 con 0x7f0380156750 2024-05-26T22:56:16.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:16.360+0000 7f037d600640 1 -- 172.21.15.131:0/2279478291 >> v2:172.21.15.187:3300/0 conn(0x7f038006e8c0 msgr2=0x7f038015d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:16.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:16.360+0000 7f037d600640 1 --2- 172.21.15.131:0/2279478291 >> v2:172.21.15.187:3300/0 conn(0x7f038006e8c0 0x7f038015d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:16.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:16.360+0000 7f037d600640 1 -- 172.21.15.131:0/2279478291 >> v1:172.21.15.187:6789/0 conn(0x7f03800cc7f0 legacy=0x7f03800cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:16.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:16.360+0000 7f037d600640 1 -- 172.21.15.131:0/2279478291 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0380160be0 con 0x7f0380156750 2024-05-26T22:56:16.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:16.360+0000 7f037d600640 1 -- 172.21.15.131:0/2279478291 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7f0370001670 con 0x7f0380156750 2024-05-26T22:56:16.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:16.360+0000 7f037d600640 1 -- 172.21.15.131:0/2279478291 >> v1:172.21.15.131:6789/0 conn(0x7f0380156750 legacy=0x7f038015fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:16.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:16.360+0000 7f037d600640 1 --2- 172.21.15.131:0/2279478291 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03640039b0 0x7f0364003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:16.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:16.360+0000 7f037d600640 1 --2- 172.21.15.131:0/2279478291 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0364004800 0x7f0364004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:16.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:16.360+0000 7f037d600640 1 --2- 172.21.15.131:0/2279478291 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0364005210 0x7f036400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:16.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:16.360+0000 7f037d600640 1 -- 172.21.15.131:0/2279478291 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f038015cbd0 con 0x7f03640039b0 2024-05-26T22:56:16.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:16.360+0000 7f037d600640 1 -- 172.21.15.131:0/2279478291 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f036400c140 con 0x7f0364005210 2024-05-26T22:56:16.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:16.360+0000 7f037d600640 1 -- 172.21.15.131:0/2279478291 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f036400c340 con 0x7f0364004800 2024-05-26T22:56:16.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:16.360+0000 7f037ea00640 1 --2- 172.21.15.131:0/2279478291 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03640039b0 0x7f0364003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:16.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:16.360+0000 7f037e000640 1 --2- 172.21.15.131:0/2279478291 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0364004800 0x7f0364004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:16.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:16.360+0000 7f037ea00640 1 -- 172.21.15.131:0/2279478291 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0364004800 msgr2=0x7f0364004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:16.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:16.360+0000 7f037ea00640 1 --2- 172.21.15.131:0/2279478291 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0364004800 0x7f0364004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:16.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:16.360+0000 7f037f400640 1 --2- 172.21.15.131:0/2279478291 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0364005210 0x7f036400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:16.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:16.360+0000 7f037ea00640 1 -- 172.21.15.131:0/2279478291 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0364005210 msgr2=0x7f036400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:16.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:16.361+0000 7f037ea00640 1 --2- 172.21.15.131:0/2279478291 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0364005210 0x7f036400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:16.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:16.361+0000 7f037ea00640 1 -- 172.21.15.131:0/2279478291 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0364003520 con 0x7f03640039b0 2024-05-26T22:56:16.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:16.361+0000 7f037f400640 1 --2- 172.21.15.131:0/2279478291 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0364005210 0x7f036400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:56:16.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:16.361+0000 7f037f400640 1 -- 172.21.15.131:0/2279478291 reap_dead start 2024-05-26T22:56:16.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:16.362+0000 7f037ea00640 1 --2- 172.21.15.131:0/2279478291 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03640039b0 0x7f0364003df0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f037000c240 tx=0x7f0364005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b67fa8e51456c995 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:16.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:16.362+0000 7f037d600640 1 -- 172.21.15.131:0/2279478291 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0370007cf0 con 0x7f03640039b0 2024-05-26T22:56:16.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:16.362+0000 7f037d600640 1 -- 172.21.15.131:0/2279478291 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03700058d0 con 0x7f03640039b0 2024-05-26T22:56:16.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:16.362+0000 7f037d600640 1 -- 172.21.15.131:0/2279478291 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0370005d20 con 0x7f03640039b0 2024-05-26T22:56:16.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:16.362+0000 7f037fe00640 1 -- 172.21.15.131:0/2279478291 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03640039b0 msgr2=0x7f0364003df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:16.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:16.362+0000 7f037fe00640 1 --2- 172.21.15.131:0/2279478291 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03640039b0 0x7f0364003df0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f037000c240 tx=0x7f0364005830 comp rx=0 tx=0).stop 2024-05-26T22:56:16.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:16.362+0000 7f037fe00640 1 -- 172.21.15.131:0/2279478291 shutdown_connections 2024-05-26T22:56:16.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:16.362+0000 7f037fe00640 1 --2- 172.21.15.131:0/2279478291 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03640039b0 0x7f0364003df0 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:16.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:16.363+0000 7f037fe00640 1 -- 172.21.15.131:0/2279478291 >> 172.21.15.131:0/2279478291 conn(0x7f03800cff40 msgr2=0x7f03800d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:56:16.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:16.363+0000 7f037fe00640 1 -- 172.21.15.131:0/2279478291 shutdown_connections 2024-05-26T22:56:16.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:16.363+0000 7f037fe00640 1 -- 172.21.15.131:0/2279478291 wait complete. 2024-05-26T22:56:16.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:16.364+0000 7f037fe00640 1 Processor -- start 2024-05-26T22:56:16.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:16.364+0000 7f037fe00640 1 -- start start 2024-05-26T22:56:16.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:16.365+0000 7f037fe00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f03640039b0 0x7f03800aea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:16.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:16.365+0000 7f037fe00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f03800aef80 0x7f03800aa320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:16.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:16.365+0000 7f037fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03800aa860 0x7f03800aace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:16.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:16.365+0000 7f037fe00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f03800ccc60 con 0x7f03800aa860 2024-05-26T22:56:16.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:16.365+0000 7f037fe00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0380156b30 con 0x7f03640039b0 2024-05-26T22:56:16.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:16.365+0000 7f037fe00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f03800ccde0 con 0x7f03800aef80 2024-05-26T22:56:16.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:16.366+0000 7f037f400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03800aa860 0x7f03800aace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:16.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:16.366+0000 7f037f400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03800aa860 0x7f03800aace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55912/0 (socket says 172.21.15.131:55912) 2024-05-26T22:56:16.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:16.366+0000 7f037f400640 1 -- 172.21.15.131:0/2664646564 learned_addr learned my addr 172.21.15.131:0/2664646564 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:56:16.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:16.366+0000 7f037ea00640 1 --2- 172.21.15.131:0/2664646564 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f03640039b0 0x7f03800aea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:16.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:16.366+0000 7f037e000640 1 --2- 172.21.15.131:0/2664646564 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f03800aef80 0x7f03800aa320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:16.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:16.366+0000 7f037f400640 1 -- 172.21.15.131:0/2664646564 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f03800aef80 msgr2=0x7f03800aa320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:16.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:16.366+0000 7f037f400640 1 --2- 172.21.15.131:0/2664646564 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f03800aef80 0x7f03800aa320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:16.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:16.366+0000 7f037f400640 1 -- 172.21.15.131:0/2664646564 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f03640039b0 msgr2=0x7f03800aea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:16.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:16.366+0000 7f037f400640 1 --2- 172.21.15.131:0/2664646564 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f03640039b0 0x7f03800aea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:16.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:16.366+0000 7f037f400640 1 -- 172.21.15.131:0/2664646564 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03800ac7e0 con 0x7f03800aa860 2024-05-26T22:56:16.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:16.367+0000 7f037f400640 1 --2- 172.21.15.131:0/2664646564 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03800aa860 0x7f03800aace0 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f03800707f0 tx=0x7f037800d9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:16.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:16.367+0000 7f0377400640 1 -- 172.21.15.131:0/2664646564 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0378018070 con 0x7f03800aa860 2024-05-26T22:56:16.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:16.367+0000 7f0377400640 1 -- 172.21.15.131:0/2664646564 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0378003da0 con 0x7f03800aa860 2024-05-26T22:56:16.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:16.367+0000 7f037fe00640 1 -- 172.21.15.131:0/2664646564 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03800aca80 con 0x7f03800aa860 2024-05-26T22:56:16.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:16.367+0000 7f0377400640 1 -- 172.21.15.131:0/2664646564 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03780052b0 con 0x7f03800aa860 2024-05-26T22:56:16.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:16.367+0000 7f037fe00640 1 -- 172.21.15.131:0/2664646564 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03800ad040 con 0x7f03800aa860 2024-05-26T22:56:16.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:16.368+0000 7f037fe00640 1 -- 172.21.15.131:0/2664646564 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03800694e0 con 0x7f03800aa860 2024-05-26T22:56:16.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:16.369+0000 7f0377400640 1 -- 172.21.15.131:0/2664646564 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f037801b070 con 0x7f03800aa860 2024-05-26T22:56:16.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:16.370+0000 7f0377400640 1 --2- 172.21.15.131:0/2664646564 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f036006be60 0x7f036006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:16.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:16.370+0000 7f0377400640 1 -- 172.21.15.131:0/2664646564 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5709+0+0 (secure 0 0 0) 0x7f037808a530 con 0x7f03800aa860 2024-05-26T22:56:16.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:16.370+0000 7f037ea00640 1 --2- 172.21.15.131:0/2664646564 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f036006be60 0x7f036006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:16.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:16.371+0000 7f037ea00640 1 --2- 172.21.15.131:0/2664646564 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f036006be60 0x7f036006e320 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f0370033000 tx=0x7f0370009030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:16.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:16.375+0000 7f0377400640 1 -- 172.21.15.131:0/2664646564 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f0378058e40 con 0x7f03800aa860 2024-05-26T22:56:16.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:16.616+0000 7f037fe00640 1 -- 172.21.15.131:0/2664646564 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0380144930 con 0x7f03800aa860 2024-05-26T22:56:16.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:16.617+0000 7f0377400640 1 -- 172.21.15.131:0/2664646564 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) ==== 95+0+5217 (secure 0 0 0) 0x7f0378058740 con 0x7f03800aa860 2024-05-26T22:56:16.617 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:56:16.617 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":112,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5959,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/2949813640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":2949813640},{"type":"v1","addr":"172.21.15.187:6833","nonce":2949813640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104},{"gid":5968,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/2594604065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2594604065},{"type":"v1","addr":"172.21.15.187:6835","nonce":2594604065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":105}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-26T22:56:01.980221+0000","modified":"2024-05-26T22:56:16.157078+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1],"up":{"mds_0":8310,"mds_1":8307},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8307":{"gid":8307,"name":"a","rank":1,"incarnation":112,"state":"up:creating","state_seq":1,"addr":"172.21.15.131:6836/2710427205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":2710427205},{"type":"v1","addr":"172.21.15.131:6836","nonce":2710427205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8310":{"gid":8310,"name":"c","rank":0,"incarnation":106,"state":"up:active","state_seq":5,"addr":"172.21.15.131:6837/2725520331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6835","nonce":2725520331},{"type":"v1","addr":"172.21.15.131:6837","nonce":2725520331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8310,"qdb_cluster":[8310]},"id":8}]} 2024-05-26T22:56:16.618 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 112 2024-05-26T22:56:16.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:16.620+0000 7f037fe00640 1 -- 172.21.15.131:0/2664646564 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f036006be60 msgr2=0x7f036006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:16.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:16.620+0000 7f037fe00640 1 --2- 172.21.15.131:0/2664646564 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f036006be60 0x7f036006e320 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f0370033000 tx=0x7f0370009030 comp rx=0 tx=0).stop 2024-05-26T22:56:16.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:16.620+0000 7f037fe00640 1 -- 172.21.15.131:0/2664646564 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03800aa860 msgr2=0x7f03800aace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:16.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:16.620+0000 7f037fe00640 1 --2- 172.21.15.131:0/2664646564 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03800aa860 0x7f03800aace0 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f03800707f0 tx=0x7f037800d9d0 comp rx=0 tx=0).stop 2024-05-26T22:56:16.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:16.620+0000 7f037fe00640 1 -- 172.21.15.131:0/2664646564 shutdown_connections 2024-05-26T22:56:16.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:16.620+0000 7f037fe00640 1 --2- 172.21.15.131:0/2664646564 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f03800aef80 0x7f03800aa320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:16.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:16.620+0000 7f037fe00640 1 --2- 172.21.15.131:0/2664646564 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f036006be60 0x7f036006e320 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:16.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:16.620+0000 7f037fe00640 1 --2- 172.21.15.131:0/2664646564 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03800aa860 0x7f03800aace0 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:16.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:16.620+0000 7f037fe00640 1 --2- 172.21.15.131:0/2664646564 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f03640039b0 0x7f03800aea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:16.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:16.620+0000 7f037fe00640 1 -- 172.21.15.131:0/2664646564 >> 172.21.15.131:0/2664646564 conn(0x7f03800cff40 msgr2=0x7f038015b7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:56:16.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:16.620+0000 7f037fe00640 1 -- 172.21.15.131:0/2664646564 shutdown_connections 2024-05-26T22:56:16.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:16.620+0000 7f037fe00640 1 -- 172.21.15.131:0/2664646564 wait complete. 2024-05-26T22:56:17.041 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 112, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-26T22:56:01.980221+0000', 'modified': '2024-05-26T22:56:16.157078+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1], 'up': {'mds_0': 8310, 'mds_1': 8307}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8307': {'gid': 8307, 'name': 'a', 'rank': 1, 'incarnation': 112, 'state': 'up:creating', 'state_seq': 1, 'addr': '172.21.15.131:6836/2710427205', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6834', 'nonce': 2710427205}, {'type': 'v1', 'addr': '172.21.15.131:6836', 'nonce': 2710427205}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_8310': {'gid': 8310, 'name': 'c', 'rank': 0, 'incarnation': 106, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.131:6837/2725520331', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6835', 'nonce': 2725520331}, {'type': 'v1', 'addr': '172.21.15.131:6837', 'nonce': 2725520331}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8310, 'qdb_cluster': [8310]} 2024-05-26T22:56:17.042 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8307:up:creating 2024-05-26T22:56:18.043 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-26T22:56:18.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:18.235+0000 7f19f7e00640 1 Processor -- start 2024-05-26T22:56:18.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:18.235+0000 7f19f7e00640 1 -- start start 2024-05-26T22:56:18.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:18.236+0000 7f19f7e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f19f006ee20 0x7f19f00b3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:18.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:18.237+0000 7f19f7e00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19f015cb20 con 0x7f19f00ad250 2024-05-26T22:56:18.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:18.237+0000 7f19f7e00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19f015ce00 con 0x7f19f01569d0 2024-05-26T22:56:18.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:18.237+0000 7f19f7e00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f19f015d0e0 con 0x7f19f006ee20 2024-05-26T22:56:18.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:18.237+0000 7f19f4a00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f19f00ad250 0x7f19f01562c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:50086/0 (socket says 172.21.15.131:50086) 2024-05-26T22:56:18.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:18.237+0000 7f19f4a00640 1 -- 172.21.15.131:0/3495900607 learned_addr learned my addr 172.21.15.131:0/3495900607 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:56:18.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:18.237+0000 7f19f5400640 1 --2- 172.21.15.131:0/3495900607 >> v2:172.21.15.131:3300/0 conn(0x7f19f006ee20 0x7f19f00b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:18.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:18.237+0000 7f19f5400640 1 -- 172.21.15.131:0/3495900607 >> v1:172.21.15.187:6789/0 conn(0x7f19f01569d0 legacy=0x7f19f0160010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:18.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:18.237+0000 7f19f5400640 1 -- 172.21.15.131:0/3495900607 >> v1:172.21.15.131:6789/0 conn(0x7f19f00ad250 legacy=0x7f19f01562c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:18.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:18.237+0000 7f19f5400640 1 -- 172.21.15.131:0/3495900607 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19f015d3c0 con 0x7f19f006ee20 2024-05-26T22:56:18.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:18.237+0000 7f19efe00640 1 -- 172.21.15.131:0/3495900607 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 733200927 0 0) 0x7f19f015cb20 con 0x7f19f00ad250 2024-05-26T22:56:18.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:18.238+0000 7f19f5400640 1 --2- 172.21.15.131:0/3495900607 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f19f006ee20 0x7f19f00b3190 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f19e40097a0 tx=0x7f19e402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=356f515368cbf6bf server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:18.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:18.239+0000 7f19efe00640 1 -- 172.21.15.131:0/3495900607 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e4037070 con 0x7f19f006ee20 2024-05-26T22:56:18.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:18.239+0000 7f19efe00640 1 -- 172.21.15.131:0/3495900607 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19e402ae50 con 0x7f19f006ee20 2024-05-26T22:56:18.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:18.239+0000 7f19efe00640 1 -- 172.21.15.131:0/3495900607 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e4032660 con 0x7f19f006ee20 2024-05-26T22:56:18.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:18.239+0000 7f19f7e00640 1 -- 172.21.15.131:0/3495900607 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f19f006ee20 msgr2=0x7f19f00b3190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:18.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:18.239+0000 7f19f7e00640 1 --2- 172.21.15.131:0/3495900607 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f19f006ee20 0x7f19f00b3190 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f19e40097a0 tx=0x7f19e402a140 comp rx=0 tx=0).stop 2024-05-26T22:56:18.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:18.239+0000 7f19f7e00640 1 -- 172.21.15.131:0/3495900607 shutdown_connections 2024-05-26T22:56:18.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:18.239+0000 7f19f7e00640 1 --2- 172.21.15.131:0/3495900607 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f19f006ee20 0x7f19f00b3190 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:18.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:18.239+0000 7f19f7e00640 1 -- 172.21.15.131:0/3495900607 >> 172.21.15.131:0/3495900607 conn(0x7f19f00b1e90 msgr2=0x7f19f00b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:56:18.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:18.240+0000 7f19f7e00640 1 -- 172.21.15.131:0/3495900607 shutdown_connections 2024-05-26T22:56:18.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:18.240+0000 7f19f7e00640 1 -- 172.21.15.131:0/3495900607 wait complete. 2024-05-26T22:56:18.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:18.241+0000 7f19f7e00640 1 Processor -- start 2024-05-26T22:56:18.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:18.241+0000 7f19f7e00640 1 -- start start 2024-05-26T22:56:18.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:18.242+0000 7f19f7e00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f19f006ee20 0x7f19f01f4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:18.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:18.242+0000 7f19f7e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f19f00ad250 0x7f19f014f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:18.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:18.242+0000 7f19f7e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f19f01569d0 0x7f19f015b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:18.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:18.242+0000 7f19f7e00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f19f015c900 con 0x7f19f01569d0 2024-05-26T22:56:18.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:18.242+0000 7f19f7e00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f19f015c780 con 0x7f19f006ee20 2024-05-26T22:56:18.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:18.242+0000 7f19f7e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f19f006f200 con 0x7f19f00ad250 2024-05-26T22:56:18.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:18.243+0000 7f19f5e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f19f01569d0 0x7f19f015b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:18.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:18.243+0000 7f19f5e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f19f01569d0 0x7f19f015b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55928/0 (socket says 172.21.15.131:55928) 2024-05-26T22:56:18.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:18.243+0000 7f19f5e00640 1 -- 172.21.15.131:0/2577354398 learned_addr learned my addr 172.21.15.131:0/2577354398 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:56:18.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:18.243+0000 7f19f4a00640 1 --2- 172.21.15.131:0/2577354398 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f19f00ad250 0x7f19f014f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:18.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:18.243+0000 7f19f5400640 1 --2- 172.21.15.131:0/2577354398 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f19f006ee20 0x7f19f01f4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:18.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:18.243+0000 7f19f5e00640 1 -- 172.21.15.131:0/2577354398 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f19f00ad250 msgr2=0x7f19f014f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:18.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:18.243+0000 7f19f5e00640 1 --2- 172.21.15.131:0/2577354398 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f19f00ad250 0x7f19f014f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:18.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:18.243+0000 7f19f5e00640 1 -- 172.21.15.131:0/2577354398 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f19f006ee20 msgr2=0x7f19f01f4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:18.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:18.243+0000 7f19f5e00640 1 --2- 172.21.15.131:0/2577354398 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f19f006ee20 0x7f19f01f4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:18.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:18.243+0000 7f19f5e00640 1 -- 172.21.15.131:0/2577354398 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19f014fbb0 con 0x7f19f01569d0 2024-05-26T22:56:18.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:18.243+0000 7f19f5400640 1 --2- 172.21.15.131:0/2577354398 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f19f006ee20 0x7f19f01f4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:56:18.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:18.243+0000 7f19f5e00640 1 --2- 172.21.15.131:0/2577354398 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f19f01569d0 0x7f19f015b250 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f19e800b9e0 tx=0x7f19e800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:18.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:18.244+0000 7f19ed600640 1 -- 172.21.15.131:0/2577354398 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e8003b70 con 0x7f19f01569d0 2024-05-26T22:56:18.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:18.244+0000 7f19ed600640 1 -- 172.21.15.131:0/2577354398 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19e8003d10 con 0x7f19f01569d0 2024-05-26T22:56:18.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:18.244+0000 7f19ed600640 1 -- 172.21.15.131:0/2577354398 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e800cb60 con 0x7f19f01569d0 2024-05-26T22:56:18.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:18.244+0000 7f19f7e00640 1 -- 172.21.15.131:0/2577354398 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19f014fe90 con 0x7f19f01569d0 2024-05-26T22:56:18.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:18.244+0000 7f19f7e00640 1 -- 172.21.15.131:0/2577354398 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19f0150450 con 0x7f19f01569d0 2024-05-26T22:56:18.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:18.245+0000 7f19f7e00640 1 -- 172.21.15.131:0/2577354398 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19f00694e0 con 0x7f19f01569d0 2024-05-26T22:56:18.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:18.246+0000 7f19ed600640 1 -- 172.21.15.131:0/2577354398 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f19e800cd00 con 0x7f19f01569d0 2024-05-26T22:56:18.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:18.247+0000 7f19ed600640 1 --2- 172.21.15.131:0/2577354398 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f19d006be30 0x7f19d006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:18.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:18.247+0000 7f19ed600640 1 -- 172.21.15.131:0/2577354398 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5709+0+0 (secure 0 0 0) 0x7f19e808ac50 con 0x7f19f01569d0 2024-05-26T22:56:18.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:18.247+0000 7f19f5400640 1 --2- 172.21.15.131:0/2577354398 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f19d006be30 0x7f19d006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:18.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:18.248+0000 7f19f5400640 1 --2- 172.21.15.131:0/2577354398 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f19d006be30 0x7f19d006e2f0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f19e4034000 tx=0x7f19e402e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:18.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:18.256+0000 7f19ed600640 1 -- 172.21.15.131:0/2577354398 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f19e80595f0 con 0x7f19f01569d0 2024-05-26T22:56:18.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:18.497+0000 7f19f7e00640 1 -- 172.21.15.131:0/2577354398 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f19f00b2660 con 0x7f19f01569d0 2024-05-26T22:56:18.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:18.497+0000 7f19ed600640 1 -- 172.21.15.131:0/2577354398 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 115 v115) ==== 95+0+5239 (secure 0 0 0) 0x7f19e8058ef0 con 0x7f19f01569d0 2024-05-26T22:56:18.497 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:56:18.497 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":115,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5959,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/2949813640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":2949813640},{"type":"v1","addr":"172.21.15.187:6833","nonce":2949813640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104}],"filesystems":[{"mdsmap":{"epoch":115,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-26T22:56:01.980221+0000","modified":"2024-05-26T22:56:18.184120+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8310,"mds_1":8307,"mds_2":5968},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5968":{"gid":5968,"name":"d","rank":2,"incarnation":114,"state":"up:active","state_seq":7,"addr":"172.21.15.187:6835/2594604065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2594604065},{"type":"v1","addr":"172.21.15.187:6835","nonce":2594604065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8307":{"gid":8307,"name":"a","rank":1,"incarnation":112,"state":"up:active","state_seq":8,"addr":"172.21.15.131:6836/2710427205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":2710427205},{"type":"v1","addr":"172.21.15.131:6836","nonce":2710427205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8310":{"gid":8310,"name":"c","rank":0,"incarnation":106,"state":"up:active","state_seq":5,"addr":"172.21.15.131:6837/2725520331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6835","nonce":2725520331},{"type":"v1","addr":"172.21.15.131:6837","nonce":2725520331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8310,"qdb_cluster":[5968,8307,8310]},"id":8}]} 2024-05-26T22:56:18.498 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 115 2024-05-26T22:56:18.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:18.500+0000 7f19f7e00640 1 -- 172.21.15.131:0/2577354398 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f19d006be30 msgr2=0x7f19d006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:18.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:18.500+0000 7f19f7e00640 1 --2- 172.21.15.131:0/2577354398 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f19d006be30 0x7f19d006e2f0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f19e4034000 tx=0x7f19e402e000 comp rx=0 tx=0).stop 2024-05-26T22:56:18.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:18.500+0000 7f19f7e00640 1 -- 172.21.15.131:0/2577354398 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f19f01569d0 msgr2=0x7f19f015b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:18.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:18.500+0000 7f19f7e00640 1 --2- 172.21.15.131:0/2577354398 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f19f01569d0 0x7f19f015b250 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f19e800b9e0 tx=0x7f19e800bea0 comp rx=0 tx=0).stop 2024-05-26T22:56:18.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:18.500+0000 7f19f7e00640 1 -- 172.21.15.131:0/2577354398 shutdown_connections 2024-05-26T22:56:18.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:18.500+0000 7f19f7e00640 1 --2- 172.21.15.131:0/2577354398 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f19f00ad250 0x7f19f014f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:18.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:18.500+0000 7f19f7e00640 1 --2- 172.21.15.131:0/2577354398 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f19d006be30 0x7f19d006e2f0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:18.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:18.501+0000 7f19f7e00640 1 --2- 172.21.15.131:0/2577354398 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f19f01569d0 0x7f19f015b250 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:18.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:18.501+0000 7f19f7e00640 1 --2- 172.21.15.131:0/2577354398 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f19f006ee20 0x7f19f01f4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:18.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:18.501+0000 7f19f7e00640 1 -- 172.21.15.131:0/2577354398 >> 172.21.15.131:0/2577354398 conn(0x7f19f00b1e90 msgr2=0x7f19f006e460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:56:18.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:18.501+0000 7f19f7e00640 1 -- 172.21.15.131:0/2577354398 shutdown_connections 2024-05-26T22:56:18.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:18.501+0000 7f19f7e00640 1 -- 172.21.15.131:0/2577354398 wait complete. 2024-05-26T22:56:18.929 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 115, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-26T22:56:01.980221+0000', 'modified': '2024-05-26T22:56:18.184120+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 8310, 'mds_1': 8307, 'mds_2': 5968}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5968': {'gid': 5968, 'name': 'd', 'rank': 2, 'incarnation': 114, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.187:6835/2594604065', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.187:6834', 'nonce': 2594604065}, {'type': 'v1', 'addr': '172.21.15.187:6835', 'nonce': 2594604065}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_8307': {'gid': 8307, 'name': 'a', 'rank': 1, 'incarnation': 112, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.131:6836/2710427205', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6834', 'nonce': 2710427205}, {'type': 'v1', 'addr': '172.21.15.131:6836', 'nonce': 2710427205}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_8310': {'gid': 8310, 'name': 'c', 'rank': 0, 'incarnation': 106, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.131:6837/2725520331', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6835', 'nonce': 2725520331}, {'type': 'v1', 'addr': '172.21.15.131:6837', 'nonce': 2725520331}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8310, 'qdb_cluster': [5968, 8307, 8310]} 2024-05-26T22:56:18.929 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-05-26T22:56:18.930 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-05-26T22:56:19.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.108+0000 7f2d0fe00640 1 Processor -- start 2024-05-26T22:56:19.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.108+0000 7f2d0fe00640 1 -- start start 2024-05-26T22:56:19.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.110+0000 7f2d0fe00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f2d100d1190 0x7f2d100d25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:19.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.110+0000 7f2d0fe00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f2d100b7330 0x7f2d100b77c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:19.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.110+0000 7f2d0fe00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d100ba140 con 0x7f2d100b9ce0 2024-05-26T22:56:19.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.110+0000 7f2d0fe00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f2d100d2cd0 con 0x7f2d100b7330 2024-05-26T22:56:19.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.110+0000 7f2d0fe00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f2d100d2fb0 con 0x7f2d100d1190 2024-05-26T22:56:19.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.110+0000 7f2d0f400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f2d100b7330 0x7f2d100b77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:19.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.110+0000 7f2d0f400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f2d100b7330 0x7f2d100b77c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55940/0 (socket says 172.21.15.131:55940) 2024-05-26T22:56:19.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.110+0000 7f2d0f400640 1 -- 172.21.15.131:0/362684854 learned_addr learned my addr 172.21.15.131:0/362684854 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:56:19.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.111+0000 7f2d0ea00640 1 --2- 172.21.15.131:0/362684854 >> v2:172.21.15.187:3300/0 conn(0x7f2d100d1190 0x7f2d100d25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:19.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.111+0000 7f2d0f400640 1 -- 172.21.15.131:0/362684854 >> v2:172.21.15.187:3300/0 conn(0x7f2d100d1190 msgr2=0x7f2d100d25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:19.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.111+0000 7f2d0f400640 1 --2- 172.21.15.131:0/362684854 >> v2:172.21.15.187:3300/0 conn(0x7f2d100d1190 0x7f2d100d25d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:19.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.111+0000 7f2d0f400640 1 -- 172.21.15.131:0/362684854 >> v1:172.21.15.187:6789/0 conn(0x7f2d100b9ce0 legacy=0x7f2d100d1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:19.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.111+0000 7f2d0f400640 1 -- 172.21.15.131:0/362684854 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d100d3290 con 0x7f2d100b7330 2024-05-26T22:56:19.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.112+0000 7f2d0f400640 1 --2- 172.21.15.131:0/362684854 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2d100b7330 0x7f2d100b77c0 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f2d1006bf00 tx=0x7f2d0802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcd3a97e64ed88f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:19.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.112+0000 7f2d0e000640 1 -- 172.21.15.131:0/362684854 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d08036070 con 0x7f2d100b7330 2024-05-26T22:56:19.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.113+0000 7f2d0e000640 1 -- 172.21.15.131:0/362684854 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d08031600 con 0x7f2d100b7330 2024-05-26T22:56:19.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.113+0000 7f2d0e000640 1 -- 172.21.15.131:0/362684854 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d080319d0 con 0x7f2d100b7330 2024-05-26T22:56:19.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.113+0000 7f2d0fe00640 1 -- 172.21.15.131:0/362684854 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2d100b7330 msgr2=0x7f2d100b77c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:19.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.113+0000 7f2d0fe00640 1 --2- 172.21.15.131:0/362684854 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2d100b7330 0x7f2d100b77c0 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f2d1006bf00 tx=0x7f2d0802a6f0 comp rx=0 tx=0).stop 2024-05-26T22:56:19.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.113+0000 7f2d0fe00640 1 -- 172.21.15.131:0/362684854 shutdown_connections 2024-05-26T22:56:19.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.113+0000 7f2d0fe00640 1 --2- 172.21.15.131:0/362684854 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2d100b7330 0x7f2d100b77c0 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:19.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.113+0000 7f2d0fe00640 1 --2- 172.21.15.131:0/362684854 >> v2:172.21.15.187:3300/0 conn(0x7f2d100d1190 0x7f2d100d25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:19.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.113+0000 7f2d0fe00640 1 -- 172.21.15.131:0/362684854 >> 172.21.15.131:0/362684854 conn(0x7f2d10072e60 msgr2=0x7f2d10073270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:56:19.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.113+0000 7f2d0fe00640 1 -- 172.21.15.131:0/362684854 shutdown_connections 2024-05-26T22:56:19.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.113+0000 7f2d0fe00640 1 -- 172.21.15.131:0/362684854 wait complete. 2024-05-26T22:56:19.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.115+0000 7f2d0fe00640 1 Processor -- start 2024-05-26T22:56:19.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.115+0000 7f2d0fe00640 1 -- start start 2024-05-26T22:56:19.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.116+0000 7f2d0fe00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2d100b7330 0x7f2d10152710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:19.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.116+0000 7f2d0fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2d100b9ce0 0x7f2d1015ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:19.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.116+0000 7f2d0fe00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2d100d1190 0x7f2d1015efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:19.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.116+0000 7f2d0fe00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f2d100d3d70 con 0x7f2d100b9ce0 2024-05-26T22:56:19.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.116+0000 7f2d0fe00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2d100d3850 con 0x7f2d100b7330 2024-05-26T22:56:19.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.116+0000 7f2d0fe00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f2d100cbea0 con 0x7f2d100d1190 2024-05-26T22:56:19.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.117+0000 7f2d07e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2d100b9ce0 0x7f2d1015ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:19.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.117+0000 7f2d07e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2d100b9ce0 0x7f2d1015ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55952/0 (socket says 172.21.15.131:55952) 2024-05-26T22:56:19.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.117+0000 7f2d07e00640 1 -- 172.21.15.131:0/394926496 learned_addr learned my addr 172.21.15.131:0/394926496 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:56:19.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.117+0000 7f2d0ea00640 1 --2- 172.21.15.131:0/394926496 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2d100b7330 0x7f2d10152710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:19.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.117+0000 7f2d07e00640 1 -- 172.21.15.131:0/394926496 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2d100d1190 msgr2=0x7f2d1015efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:19.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.117+0000 7f2d07e00640 1 --2- 172.21.15.131:0/394926496 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2d100d1190 0x7f2d1015efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:19.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.117+0000 7f2d07e00640 1 -- 172.21.15.131:0/394926496 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2d100b7330 msgr2=0x7f2d10152710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:19.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.117+0000 7f2d07e00640 1 --2- 172.21.15.131:0/394926496 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2d100b7330 0x7f2d10152710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:19.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.117+0000 7f2d07e00640 1 -- 172.21.15.131:0/394926496 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d10152c50 con 0x7f2d100b9ce0 2024-05-26T22:56:19.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.117+0000 7f2d07e00640 1 --2- 172.21.15.131:0/394926496 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2d100b9ce0 0x7f2d1015ea90 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f2cf800c460 tx=0x7f2cf800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:19.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.118+0000 7f2d06a00640 1 -- 172.21.15.131:0/394926496 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cf8016020 con 0x7f2d100b9ce0 2024-05-26T22:56:19.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.118+0000 7f2d06a00640 1 -- 172.21.15.131:0/394926496 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2cf8012870 con 0x7f2d100b9ce0 2024-05-26T22:56:19.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.118+0000 7f2d0fe00640 1 -- 172.21.15.131:0/394926496 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d100cb570 con 0x7f2d100b9ce0 2024-05-26T22:56:19.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.118+0000 7f2d06a00640 1 -- 172.21.15.131:0/394926496 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cf8012c60 con 0x7f2d100b9ce0 2024-05-26T22:56:19.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.118+0000 7f2d0fe00640 1 -- 172.21.15.131:0/394926496 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d100cbb30 con 0x7f2d100b9ce0 2024-05-26T22:56:19.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.120+0000 7f2d06a00640 1 -- 172.21.15.131:0/394926496 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2cf8019020 con 0x7f2d100b9ce0 2024-05-26T22:56:19.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.121+0000 7f2d06a00640 1 --2- 172.21.15.131:0/394926496 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f2cec06be30 0x7f2cec06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:19.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.121+0000 7f2d06a00640 1 -- 172.21.15.131:0/394926496 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5709+0+0 (secure 0 0 0) 0x7f2cf808ac30 con 0x7f2d100b9ce0 2024-05-26T22:56:19.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.121+0000 7f2d0ea00640 1 --2- 172.21.15.131:0/394926496 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f2cec06be30 0x7f2cec06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:19.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.122+0000 7f2d0ea00640 1 --2- 172.21.15.131:0/394926496 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f2cec06be30 0x7f2cec06e2f0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f2d0000b7c0 tx=0x7f2d00008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:19.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.123+0000 7f2d17ffd740 1 Processor -- start 2024-05-26T22:56:19.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.123+0000 7f2d17ffd740 1 -- start start 2024-05-26T22:56:19.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.124+0000 7f2d17ffd740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fe7c4cae50 0x55fe7c47da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:19.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.124+0000 7f2d17ffd740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55fe7c4adf70 0x55fe7c4ad2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:19.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.124+0000 7f2d17ffd740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe7c4ad830 0x55fe7c482640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:19.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.124+0000 7f2d17ffd740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55fe7c3e5610 con 0x55fe7c4ad830 2024-05-26T22:56:19.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.125+0000 7f2d17ffd740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55fe7c38d5c0 con 0x55fe7c4cae50 2024-05-26T22:56:19.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.125+0000 7f2d17ffd740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x55fe7c3e15d0 con 0x55fe7c4adf70 2024-05-26T22:56:19.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.125+0000 7f2d0f400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe7c4ad830 0x55fe7c482640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:19.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.125+0000 7f2d0f400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe7c4ad830 0x55fe7c482640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55966/0 (socket says 172.21.15.131:55966) 2024-05-26T22:56:19.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.125+0000 7f2d0f400640 1 -- 172.21.15.131:0/2873679176 learned_addr learned my addr 172.21.15.131:0/2873679176 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:56:19.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.125+0000 7f2d0ea00640 1 --2- 172.21.15.131:0/2873679176 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fe7c4cae50 0x55fe7c47da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:19.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.125+0000 7f2d07e00640 1 --2- 172.21.15.131:0/2873679176 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55fe7c4adf70 0x55fe7c4ad2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:19.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.125+0000 7f2d0f400640 1 -- 172.21.15.131:0/2873679176 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55fe7c4adf70 msgr2=0x55fe7c4ad2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:19.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.125+0000 7f2d0f400640 1 --2- 172.21.15.131:0/2873679176 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55fe7c4adf70 0x55fe7c4ad2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:19.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.125+0000 7f2d0f400640 1 -- 172.21.15.131:0/2873679176 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fe7c4cae50 msgr2=0x55fe7c47da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:19.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.125+0000 7f2d0f400640 1 --2- 172.21.15.131:0/2873679176 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fe7c4cae50 0x55fe7c47da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:19.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.125+0000 7f2d0f400640 1 -- 172.21.15.131:0/2873679176 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe7c0f92d0 con 0x55fe7c4ad830 2024-05-26T22:56:19.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.125+0000 7f2d0ea00640 1 --2- 172.21.15.131:0/2873679176 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fe7c4cae50 0x55fe7c47da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:56:19.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.126+0000 7f2d0f400640 1 --2- 172.21.15.131:0/2873679176 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe7c4ad830 0x55fe7c482640 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f2d0802ace0 tx=0x7f2d0802c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec1cf8ad13a42d56 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:19.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.126+0000 7f2cff000640 1 -- 172.21.15.131:0/2873679176 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d08036070 con 0x55fe7c4ad830 2024-05-26T22:56:19.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.126+0000 7f2cff000640 1 -- 172.21.15.131:0/2873679176 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d08040070 con 0x55fe7c4ad830 2024-05-26T22:56:19.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.126+0000 7f2cff000640 1 -- 172.21.15.131:0/2873679176 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d0803b860 con 0x55fe7c4ad830 2024-05-26T22:56:19.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.126+0000 7f2d17ffd740 1 -- 172.21.15.131:0/2873679176 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe7c4ad830 msgr2=0x55fe7c482640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:19.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.126+0000 7f2d17ffd740 1 --2- 172.21.15.131:0/2873679176 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe7c4ad830 0x55fe7c482640 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f2d0802ace0 tx=0x7f2d0802c000 comp rx=0 tx=0).stop 2024-05-26T22:56:19.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.127+0000 7f2d17ffd740 1 -- 172.21.15.131:0/2873679176 shutdown_connections 2024-05-26T22:56:19.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.127+0000 7f2d17ffd740 1 --2- 172.21.15.131:0/2873679176 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55fe7c4adf70 0x55fe7c4ad2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:19.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.127+0000 7f2d17ffd740 1 --2- 172.21.15.131:0/2873679176 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe7c4ad830 0x55fe7c482640 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:19.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.127+0000 7f2d17ffd740 1 --2- 172.21.15.131:0/2873679176 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fe7c4cae50 0x55fe7c47da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:19.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.127+0000 7f2d17ffd740 1 -- 172.21.15.131:0/2873679176 >> 172.21.15.131:0/2873679176 conn(0x55fe7c47fce0 msgr2=0x55fe7c4c82f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:56:19.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.127+0000 7f2d17ffd740 1 -- 172.21.15.131:0/2873679176 shutdown_connections 2024-05-26T22:56:19.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.127+0000 7f2d17ffd740 1 -- 172.21.15.131:0/2873679176 wait complete. 2024-05-26T22:56:19.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.127+0000 7f2d17ffd740 1 -- start start 2024-05-26T22:56:19.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.128+0000 7f2d17ffd740 1 Processor -- start 2024-05-26T22:56:19.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.129+0000 7f2d17ffd740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fe7c4adf70 0x55fe7c4a8c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:19.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.129+0000 7f2d17ffd740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55fe7c4cae50 0x55fe7c4dbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:19.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.129+0000 7f2d17ffd740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe7c4b0190 0x55fe7c4c9df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:19.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.129+0000 7f2d17ffd740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55fe7c481c00 con 0x55fe7c4b0190 2024-05-26T22:56:19.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.129+0000 7f2d17ffd740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55fe7c3a0910 con 0x55fe7c4adf70 2024-05-26T22:56:19.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.129+0000 7f2d17ffd740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x55fe7c480ff0 con 0x55fe7c4cae50 2024-05-26T22:56:19.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.129+0000 7f2d17ffd740 10 client.0 _ll_register_callbacks cb 0x55fe7c3dc410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-26T22:56:19.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.129+0000 7f2d0f400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe7c4b0190 0x55fe7c4c9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:19.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.129+0000 7f2d0f400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe7c4b0190 0x55fe7c4c9df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55978/0 (socket says 172.21.15.131:55978) 2024-05-26T22:56:19.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.129+0000 7f2d0f400640 1 -- 172.21.15.131:0/1238995946 learned_addr learned my addr 172.21.15.131:0/1238995946 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:56:19.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.129+0000 7f2d07e00640 1 --2- 172.21.15.131:0/1238995946 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55fe7c4cae50 0x55fe7c4dbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:19.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.129+0000 7f2d0ea00640 1 --2- 172.21.15.131:0/1238995946 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fe7c4adf70 0x55fe7c4a8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:19.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.129+0000 7f2d0f400640 1 -- 172.21.15.131:0/1238995946 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55fe7c4cae50 msgr2=0x55fe7c4dbcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:19.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.129+0000 7f2d0f400640 1 --2- 172.21.15.131:0/1238995946 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55fe7c4cae50 0x55fe7c4dbcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:19.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.129+0000 7f2d0f400640 1 -- 172.21.15.131:0/1238995946 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fe7c4adf70 msgr2=0x55fe7c4a8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:19.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.129+0000 7f2d0f400640 1 --2- 172.21.15.131:0/1238995946 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fe7c4adf70 0x55fe7c4a8c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:19.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.129+0000 7f2d0f400640 1 -- 172.21.15.131:0/1238995946 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe7c1c36b0 con 0x55fe7c4b0190 2024-05-26T22:56:19.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.130+0000 7f2d0f400640 1 --2- 172.21.15.131:0/1238995946 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe7c4b0190 0x55fe7c4c9df0 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f2d08009f10 tx=0x7f2d08004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b04c47c3fec8ccaf server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:19.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.130+0000 7f2cf3400640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-26T22:56:19.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.130+0000 7f2cf3400640 1 -- 172.21.15.131:0/1238995946 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d08004620 con 0x55fe7c4b0190 2024-05-26T22:56:19.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.130+0000 7f2cf3400640 1 -- 172.21.15.131:0/1238995946 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 5709+0+0 (secure 0 0 0) 0x7f2d080047c0 con 0x55fe7c4b0190 2024-05-26T22:56:19.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.130+0000 7f2d17ffd740 1 -- 172.21.15.131:0/1238995946 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d08040070 con 0x55fe7c4b0190 2024-05-26T22:56:19.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.130+0000 7f2cf3400640 1 -- 172.21.15.131:0/1238995946 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=115) ==== 24+0+0 (secure 0 0 0) 0x7f2d08009af0 con 0x55fe7c4b0190 2024-05-26T22:56:19.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.130+0000 7f2d17ffd740 10 client.8496 fetch_fsmap learned FSMap version 115 2024-05-26T22:56:19.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.130+0000 7f2d17ffd740 1 -- 172.21.15.131:0/1238995946 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=115,monmap=2+}) -- 0x55fe7c4dc9e0 con 0x55fe7c4b0190 2024-05-26T22:56:19.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.131+0000 7f2cf3400640 1 -- 172.21.15.131:0/1238995946 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d08040070 con 0x55fe7c4b0190 2024-05-26T22:56:19.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.131+0000 7f2cf3400640 1 -- 172.21.15.131:0/1238995946 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 115) ==== 3534+0+0 (secure 0 0 0) 0x7f2d08059080 con 0x55fe7c4b0190 2024-05-26T22:56:19.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.131+0000 7f2d17ffd740 10 client.8496 fetch_fsmap finished waiting for FSMap version 115 2024-05-26T22:56:19.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.131+0000 7f2d17ffd740 10 client.8496 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-26T22:56:19.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.131+0000 7f2d17ffd740 20 client.8496 populate_metadata read hostname 'smithi131' 2024-05-26T22:56:19.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.131+0000 7f2d17ffd740 1 --2- 172.21.15.131:0/1238995946 >> [v2:172.21.15.187:6834/2594604065,v1:172.21.15.187:6835/2594604065] conn(0x55fe7c4ab900 0x55fe7c4abce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:19.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.131+0000 7f2d17ffd740 4 client.8496 mds_command: new command op to 5968 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-26T22:56:19.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.131+0000 7f2d17ffd740 1 -- 172.21.15.131:0/1238995946 --> [v2:172.21.15.187:6834/2594604065,v1:172.21.15.187:6835/2594604065] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fe7c3f2d40 con 0x55fe7c4ab900 2024-05-26T22:56:19.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.131+0000 7f2d0ea00640 1 --2- 172.21.15.131:0/1238995946 >> [v2:172.21.15.187:6834/2594604065,v1:172.21.15.187:6835/2594604065] conn(0x55fe7c4ab900 0x55fe7c4abce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:19.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.132+0000 7f2d0ea00640 1 --2- 172.21.15.131:0/1238995946 >> [v2:172.21.15.187:6834/2594604065,v1:172.21.15.187:6835/2594604065] conn(0x55fe7c4ab900 0x55fe7c4abce0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f11e7f184ddf16b server_cookie=5da6a17656cbb66a in_seq=0 out_seq=0 2024-05-26T22:56:19.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.132+0000 7f2cf3400640 10 client.8496 ms_handle_connect on v2:172.21.15.187:6834/2594604065 2024-05-26T22:56:19.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.135+0000 7f2cf3400640 1 -- 172.21.15.131:0/1238995946 <== mds.2 v2:172.21.15.187:6834/2594604065 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55fe7c3f2d40 con 0x55fe7c4ab900 2024-05-26T22:56:19.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.135+0000 7f2cf3400640 10 client.8496 handle_command_reply: tid=0 2024-05-26T22:56:19.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.135+0000 7f2d17ffd740 1 client.8496 shutdown 2024-05-26T22:56:19.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.135+0000 7f2d17ffd740 1 -- 172.21.15.131:0/1238995946 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe7c4b0190 msgr2=0x55fe7c4c9df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:19.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.135+0000 7f2d17ffd740 1 --2- 172.21.15.131:0/1238995946 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe7c4b0190 0x55fe7c4c9df0 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f2d08009f10 tx=0x7f2d08004ab0 comp rx=0 tx=0).stop 2024-05-26T22:56:19.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.135+0000 7f2d17ffd740 1 -- 172.21.15.131:0/1238995946 shutdown_connections 2024-05-26T22:56:19.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.135+0000 7f2d17ffd740 1 --2- 172.21.15.131:0/1238995946 >> [v2:172.21.15.187:6834/2594604065,v1:172.21.15.187:6835/2594604065] conn(0x55fe7c4ab900 0x55fe7c4abce0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:19.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.135+0000 7f2d17ffd740 1 --2- 172.21.15.131:0/1238995946 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55fe7c4cae50 0x55fe7c4dbcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:19.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.135+0000 7f2d17ffd740 1 --2- 172.21.15.131:0/1238995946 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe7c4b0190 0x55fe7c4c9df0 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:19.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.135+0000 7f2d17ffd740 1 --2- 172.21.15.131:0/1238995946 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fe7c4adf70 0x55fe7c4a8c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:19.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.135+0000 7f2d17ffd740 1 -- 172.21.15.131:0/1238995946 >> 172.21.15.131:0/1238995946 conn(0x55fe7c4ad830 msgr2=0x55fe7c47e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:56:19.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.135+0000 7f2cf3400640 0 client.8496 ms_handle_reset on v2:172.21.15.187:6834/2594604065 2024-05-26T22:56:19.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.135+0000 7f2d17ffd740 1 -- 172.21.15.131:0/1238995946 shutdown_connections 2024-05-26T22:56:19.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.135+0000 7f2d17ffd740 1 -- 172.21.15.131:0/1238995946 wait complete. 2024-05-26T22:56:19.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.135+0000 7f2d17ffd740 20 client.8496 trim_cache size 0 max 16384 2024-05-26T22:56:19.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.155+0000 7f2d17ffd740 1 Processor -- start 2024-05-26T22:56:19.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.155+0000 7f2d17ffd740 1 -- start start 2024-05-26T22:56:19.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.156+0000 7f2d17ffd740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fe7c4adf70 0x55fe7c4b10d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:19.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.156+0000 7f2d17ffd740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55fe7c4b0190 0x55fe7c4d8c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:19.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.156+0000 7f2d17ffd740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe7c4cae50 0x55fe7c4d9150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:19.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.156+0000 7f2d17ffd740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f2d101ffcb0 con 0x55fe7c4cae50 2024-05-26T22:56:19.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.156+0000 7f2d17ffd740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2d10200360 con 0x55fe7c4adf70 2024-05-26T22:56:19.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.156+0000 7f2d17ffd740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f2d101f79d0 con 0x55fe7c4b0190 2024-05-26T22:56:19.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.157+0000 7f2d0f400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe7c4cae50 0x55fe7c4d9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:19.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.157+0000 7f2d0f400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe7c4cae50 0x55fe7c4d9150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55984/0 (socket says 172.21.15.131:55984) 2024-05-26T22:56:19.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.157+0000 7f2d0f400640 1 -- 172.21.15.131:0/695330807 learned_addr learned my addr 172.21.15.131:0/695330807 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:56:19.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.157+0000 7f2d0f400640 1 -- 172.21.15.131:0/695330807 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55fe7c4b0190 msgr2=0x55fe7c4d8c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:19.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.157+0000 7f2d0f400640 1 --2- 172.21.15.131:0/695330807 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55fe7c4b0190 0x55fe7c4d8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:19.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.157+0000 7f2d0f400640 1 -- 172.21.15.131:0/695330807 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fe7c4adf70 msgr2=0x55fe7c4b10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:19.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.157+0000 7f2d0f400640 1 --2- 172.21.15.131:0/695330807 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fe7c4adf70 0x55fe7c4b10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:19.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.157+0000 7f2d0f400640 1 -- 172.21.15.131:0/695330807 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe7c4ad550 con 0x55fe7c4cae50 2024-05-26T22:56:19.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.157+0000 7f2d0f400640 1 --2- 172.21.15.131:0/695330807 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe7c4cae50 0x55fe7c4d9150 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f2d08000f80 tx=0x7f2d08047070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=260284dbc6b7643c server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:19.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.157+0000 7f2cfe600640 1 -- 172.21.15.131:0/695330807 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d0805c070 con 0x55fe7c4cae50 2024-05-26T22:56:19.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.157+0000 7f2cfe600640 1 -- 172.21.15.131:0/695330807 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d08040070 con 0x55fe7c4cae50 2024-05-26T22:56:19.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.158+0000 7f2cfe600640 1 -- 172.21.15.131:0/695330807 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d08004730 con 0x55fe7c4cae50 2024-05-26T22:56:19.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.158+0000 7f2d17ffd740 1 -- 172.21.15.131:0/695330807 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe7c4cae50 msgr2=0x55fe7c4d9150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:19.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.158+0000 7f2d17ffd740 1 --2- 172.21.15.131:0/695330807 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe7c4cae50 0x55fe7c4d9150 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f2d08000f80 tx=0x7f2d08047070 comp rx=0 tx=0).stop 2024-05-26T22:56:19.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.158+0000 7f2d17ffd740 1 -- 172.21.15.131:0/695330807 shutdown_connections 2024-05-26T22:56:19.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.158+0000 7f2d17ffd740 1 --2- 172.21.15.131:0/695330807 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55fe7c4b0190 0x55fe7c4d8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:19.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.158+0000 7f2d17ffd740 1 --2- 172.21.15.131:0/695330807 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe7c4cae50 0x55fe7c4d9150 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:19.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.158+0000 7f2d17ffd740 1 --2- 172.21.15.131:0/695330807 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fe7c4adf70 0x55fe7c4b10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:19.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.158+0000 7f2d17ffd740 1 -- 172.21.15.131:0/695330807 >> 172.21.15.131:0/695330807 conn(0x55fe7c4ad830 msgr2=0x55fe7c4db800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:56:19.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.158+0000 7f2d17ffd740 1 -- 172.21.15.131:0/695330807 shutdown_connections 2024-05-26T22:56:19.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.158+0000 7f2d17ffd740 1 -- 172.21.15.131:0/695330807 wait complete. 2024-05-26T22:56:19.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.159+0000 7f2d17ffd740 1 -- start start 2024-05-26T22:56:19.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.159+0000 7f2d17ffd740 1 Processor -- start 2024-05-26T22:56:19.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.160+0000 7f2d17ffd740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe7c4adf70 0x55fe7c5493c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:19.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.160+0000 7f2d17ffd740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fe7c4b0190 0x55fe7c4c92f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:19.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.160+0000 7f2d17ffd740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55fe7c4cae50 0x55fe7c4c9830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:19.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.160+0000 7f2d17ffd740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55fe7c4b06e0 con 0x55fe7c4adf70 2024-05-26T22:56:19.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.160+0000 7f2d17ffd740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55fe7c481c00 con 0x55fe7c4b0190 2024-05-26T22:56:19.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.160+0000 7f2d17ffd740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x55fe7c4af460 con 0x55fe7c4cae50 2024-05-26T22:56:19.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.160+0000 7f2d17ffd740 10 client.0 _ll_register_callbacks cb 0x7f2d08009d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-26T22:56:19.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.160+0000 7f2d0ea00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe7c4adf70 0x55fe7c5493c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:19.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.160+0000 7f2d0ea00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe7c4adf70 0x55fe7c5493c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55988/0 (socket says 172.21.15.131:55988) 2024-05-26T22:56:19.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.160+0000 7f2d0ea00640 1 -- 172.21.15.131:0/1931262633 learned_addr learned my addr 172.21.15.131:0/1931262633 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:56:19.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.160+0000 7f2d0f400640 1 --2- 172.21.15.131:0/1931262633 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55fe7c4cae50 0x55fe7c4c9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:19.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.161+0000 7f2d0ea00640 1 -- 172.21.15.131:0/1931262633 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55fe7c4cae50 msgr2=0x55fe7c4c9830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:19.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.161+0000 7f2d0ea00640 1 --2- 172.21.15.131:0/1931262633 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55fe7c4cae50 0x55fe7c4c9830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:19.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.161+0000 7f2d0ea00640 1 -- 172.21.15.131:0/1931262633 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fe7c4b0190 msgr2=0x55fe7c4c92f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:19.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.161+0000 7f2d0ea00640 1 --2- 172.21.15.131:0/1931262633 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fe7c4b0190 0x55fe7c4c92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:19.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.161+0000 7f2d0ea00640 1 -- 172.21.15.131:0/1931262633 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe7c4c5380 con 0x55fe7c4adf70 2024-05-26T22:56:19.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.161+0000 7f2d0ea00640 1 --2- 172.21.15.131:0/1931262633 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe7c4adf70 0x55fe7c5493c0 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f2d0000dcd0 tx=0x7f2d000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f07a025e727ed39f server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:19.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.161+0000 7f2cf3400640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-26T22:56:19.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.161+0000 7f2cf3400640 1 -- 172.21.15.131:0/1931262633 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d00004220 con 0x55fe7c4adf70 2024-05-26T22:56:19.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.161+0000 7f2cf3400640 1 -- 172.21.15.131:0/1931262633 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 5709+0+0 (secure 0 0 0) 0x7f2d000043c0 con 0x55fe7c4adf70 2024-05-26T22:56:19.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.161+0000 7f2d17ffd740 1 -- 172.21.15.131:0/1931262633 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d08040070 con 0x55fe7c4adf70 2024-05-26T22:56:19.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.161+0000 7f2cf3400640 1 -- 172.21.15.131:0/1931262633 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=115) ==== 24+0+0 (secure 0 0 0) 0x7f2d00032050 con 0x55fe7c4adf70 2024-05-26T22:56:19.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.161+0000 7f2d17ffd740 10 client.8502 fetch_fsmap learned FSMap version 115 2024-05-26T22:56:19.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.161+0000 7f2d17ffd740 1 -- 172.21.15.131:0/1931262633 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=115,monmap=2+}) -- 0x55fe7c4a7fc0 con 0x55fe7c4adf70 2024-05-26T22:56:19.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.161+0000 7f2cf3400640 1 -- 172.21.15.131:0/1931262633 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d08040070 con 0x55fe7c4adf70 2024-05-26T22:56:19.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.162+0000 7f2cf3400640 1 -- 172.21.15.131:0/1931262633 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 115) ==== 3534+0+0 (secure 0 0 0) 0x7f2d00035020 con 0x55fe7c4adf70 2024-05-26T22:56:19.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.162+0000 7f2d17ffd740 10 client.8502 fetch_fsmap finished waiting for FSMap version 115 2024-05-26T22:56:19.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.162+0000 7f2d17ffd740 10 client.8502 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-26T22:56:19.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.162+0000 7f2d17ffd740 20 client.8502 populate_metadata read hostname 'smithi131' 2024-05-26T22:56:19.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.162+0000 7f2d17ffd740 1 --2- 172.21.15.131:0/1931262633 >> [v2:172.21.15.187:6834/2594604065,v1:172.21.15.187:6835/2594604065] conn(0x55fe7c4c7ec0 0x55fe7c4c82a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:19.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.162+0000 7f2d17ffd740 4 client.8502 mds_command: new command op to 5968 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-26T22:56:19.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.162+0000 7f2d17ffd740 1 -- 172.21.15.131:0/1931262633 --> [v2:172.21.15.187:6834/2594604065,v1:172.21.15.187:6835/2594604065] -- command(tid 0: {"prefix": "status"}) -- 0x55fe7c3ed090 con 0x55fe7c4c7ec0 2024-05-26T22:56:19.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.162+0000 7f2d07e00640 1 --2- 172.21.15.131:0/1931262633 >> [v2:172.21.15.187:6834/2594604065,v1:172.21.15.187:6835/2594604065] conn(0x55fe7c4c7ec0 0x55fe7c4c82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:19.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.163+0000 7f2d07e00640 1 --2- 172.21.15.131:0/1931262633 >> [v2:172.21.15.187:6834/2594604065,v1:172.21.15.187:6835/2594604065] conn(0x55fe7c4c7ec0 0x55fe7c4c82a0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ce4ca7675bd5be85 server_cookie=480088841604529c in_seq=0 out_seq=0 2024-05-26T22:56:19.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.163+0000 7f2cf3400640 10 client.8502 ms_handle_connect on v2:172.21.15.187:6834/2594604065 2024-05-26T22:56:19.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.164+0000 7f2cf3400640 1 -- 172.21.15.131:0/1931262633 <== mds.2 v2:172.21.15.187:6834/2594604065 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x55fe7c3ed090 con 0x55fe7c4c7ec0 2024-05-26T22:56:19.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.164+0000 7f2cf3400640 10 client.8502 handle_command_reply: tid=0 2024-05-26T22:56:19.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.164+0000 7f2d17ffd740 1 client.8502 shutdown 2024-05-26T22:56:19.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.164+0000 7f2d17ffd740 1 -- 172.21.15.131:0/1931262633 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe7c4adf70 msgr2=0x55fe7c5493c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:19.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.164+0000 7f2d17ffd740 1 --2- 172.21.15.131:0/1931262633 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe7c4adf70 0x55fe7c5493c0 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f2d0000dcd0 tx=0x7f2d000046c0 comp rx=0 tx=0).stop 2024-05-26T22:56:19.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.164+0000 7f2d17ffd740 1 -- 172.21.15.131:0/1931262633 shutdown_connections 2024-05-26T22:56:19.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.164+0000 7f2d17ffd740 1 --2- 172.21.15.131:0/1931262633 >> [v2:172.21.15.187:6834/2594604065,v1:172.21.15.187:6835/2594604065] conn(0x55fe7c4c7ec0 0x55fe7c4c82a0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:19.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.164+0000 7f2d17ffd740 1 --2- 172.21.15.131:0/1931262633 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55fe7c4cae50 0x55fe7c4c9830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:19.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.164+0000 7f2d17ffd740 1 --2- 172.21.15.131:0/1931262633 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fe7c4b0190 0x55fe7c4c92f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:19.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.164+0000 7f2cf3400640 0 client.8502 ms_handle_reset on v2:172.21.15.187:6834/2594604065 2024-05-26T22:56:19.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.164+0000 7f2d17ffd740 1 --2- 172.21.15.131:0/1931262633 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe7c4adf70 0x55fe7c5493c0 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:19.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.164+0000 7f2d17ffd740 1 -- 172.21.15.131:0/1931262633 >> 172.21.15.131:0/1931262633 conn(0x55fe7c4ad830 msgr2=0x55fe7c47e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:56:19.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.164+0000 7f2d17ffd740 1 -- 172.21.15.131:0/1931262633 shutdown_connections 2024-05-26T22:56:19.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.164+0000 7f2d17ffd740 1 -- 172.21.15.131:0/1931262633 wait complete. 2024-05-26T22:56:19.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.164+0000 7f2d17ffd740 20 client.8502 trim_cache size 0 max 16384 2024-05-26T22:56:19.168 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-26T22:56:19.168 INFO:teuthology.orchestra.run.smithi131.stdout: "cluster_fsid": "46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f", 2024-05-26T22:56:19.168 INFO:teuthology.orchestra.run.smithi131.stdout: "whoami": 2, 2024-05-26T22:56:19.168 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 5968, 2024-05-26T22:56:19.168 INFO:teuthology.orchestra.run.smithi131.stdout: "want_state": "up:active", 2024-05-26T22:56:19.168 INFO:teuthology.orchestra.run.smithi131.stdout: "state": "up:active", 2024-05-26T22:56:19.168 INFO:teuthology.orchestra.run.smithi131.stdout: "fs_name": "cephfs", 2024-05-26T22:56:19.168 INFO:teuthology.orchestra.run.smithi131.stdout: "rank_uptime": 1.97746376, 2024-05-26T22:56:19.168 INFO:teuthology.orchestra.run.smithi131.stdout: "mdsmap_epoch": 115, 2024-05-26T22:56:19.168 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch": 107, 2024-05-26T22:56:19.168 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch_barrier": 107, 2024-05-26T22:56:19.168 INFO:teuthology.orchestra.run.smithi131.stdout: "uptime": 23.109517912000001 2024-05-26T22:56:19.169 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-26T22:56:19.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.165+0000 7f2d0fe00640 1 -- 172.21.15.131:0/394926496 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f2cec06be30 msgr2=0x7f2cec06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:19.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.165+0000 7f2d0fe00640 1 --2- 172.21.15.131:0/394926496 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f2cec06be30 0x7f2cec06e2f0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f2d0000b7c0 tx=0x7f2d00008040 comp rx=0 tx=0).stop 2024-05-26T22:56:19.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.165+0000 7f2d0fe00640 1 -- 172.21.15.131:0/394926496 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2d100b9ce0 msgr2=0x7f2d1015ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:19.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.165+0000 7f2d0fe00640 1 --2- 172.21.15.131:0/394926496 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2d100b9ce0 0x7f2d1015ea90 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f2cf800c460 tx=0x7f2cf800c920 comp rx=0 tx=0).stop 2024-05-26T22:56:19.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.165+0000 7f2d0fe00640 1 -- 172.21.15.131:0/394926496 shutdown_connections 2024-05-26T22:56:19.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.165+0000 7f2d0fe00640 1 --2- 172.21.15.131:0/394926496 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2d100d1190 0x7f2d1015efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:19.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.165+0000 7f2d0fe00640 1 --2- 172.21.15.131:0/394926496 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f2cec06be30 0x7f2cec06e2f0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:19.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.165+0000 7f2d0fe00640 1 --2- 172.21.15.131:0/394926496 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2d100b9ce0 0x7f2d1015ea90 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:19.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.165+0000 7f2d0fe00640 1 --2- 172.21.15.131:0/394926496 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2d100b7330 0x7f2d10152710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:19.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.165+0000 7f2d0fe00640 1 -- 172.21.15.131:0/394926496 >> 172.21.15.131:0/394926496 conn(0x7f2d10072e60 msgr2=0x7f2d100b4770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:56:19.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.165+0000 7f2d0fe00640 1 -- 172.21.15.131:0/394926496 shutdown_connections 2024-05-26T22:56:19.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.165+0000 7f2d0fe00640 1 -- 172.21.15.131:0/394926496 wait complete. 2024-05-26T22:56:19.180 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-05-26T22:56:19.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.372+0000 7f47f8c00640 1 Processor -- start 2024-05-26T22:56:19.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.372+0000 7f47f8c00640 1 -- start start 2024-05-26T22:56:19.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.373+0000 7f47f8c00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f47f415b730 0x7f47f415bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:19.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.373+0000 7f47f8c00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f47f41400c0 0x7f47f41404f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:19.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.373+0000 7f47f8c00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47f413e820 con 0x7f47f406d250 2024-05-26T22:56:19.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.373+0000 7f47f8c00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f47f4071260 con 0x7f47f41400c0 2024-05-26T22:56:19.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.373+0000 7f47f8c00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f47f415c4a0 con 0x7f47f415b730 2024-05-26T22:56:19.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.374+0000 7f47f3e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f47f41400c0 0x7f47f41404f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:19.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.374+0000 7f47f3e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f47f41400c0 0x7f47f41404f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55990/0 (socket says 172.21.15.131:55990) 2024-05-26T22:56:19.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.374+0000 7f47f3e00640 1 -- 172.21.15.131:0/1803998746 learned_addr learned my addr 172.21.15.131:0/1803998746 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:56:19.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.374+0000 7f47f2a00640 1 --2- 172.21.15.131:0/1803998746 >> v2:172.21.15.187:3300/0 conn(0x7f47f415b730 0x7f47f415bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:19.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.374+0000 7f47f3e00640 1 -- 172.21.15.131:0/1803998746 >> v2:172.21.15.187:3300/0 conn(0x7f47f415b730 msgr2=0x7f47f415bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:19.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.374+0000 7f47f3e00640 1 --2- 172.21.15.131:0/1803998746 >> v2:172.21.15.187:3300/0 conn(0x7f47f415b730 0x7f47f415bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:19.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.374+0000 7f47f3e00640 1 -- 172.21.15.131:0/1803998746 >> v1:172.21.15.187:6789/0 conn(0x7f47f406d250 legacy=0x7f47f406d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:19.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.374+0000 7f47f3e00640 1 -- 172.21.15.131:0/1803998746 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47f40714b0 con 0x7f47f41400c0 2024-05-26T22:56:19.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.374+0000 7f47f2a00640 1 --2- 172.21.15.131:0/1803998746 >> v2:172.21.15.187:3300/0 conn(0x7f47f415b730 0x7f47f415bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:56:19.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.375+0000 7f47f3e00640 1 --2- 172.21.15.131:0/1803998746 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47f41400c0 0x7f47f41404f0 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f47ec009e50 tx=0x7f47ec02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae989f7511b9b71a server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:19.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.376+0000 7f47f2000640 1 -- 172.21.15.131:0/1803998746 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47ec036070 con 0x7f47f41400c0 2024-05-26T22:56:19.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.376+0000 7f47f2000640 1 -- 172.21.15.131:0/1803998746 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47ec031600 con 0x7f47f41400c0 2024-05-26T22:56:19.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.376+0000 7f47f2000640 1 -- 172.21.15.131:0/1803998746 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47ec0319d0 con 0x7f47f41400c0 2024-05-26T22:56:19.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.376+0000 7f47f8c00640 1 -- 172.21.15.131:0/1803998746 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47f41400c0 msgr2=0x7f47f41404f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:19.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.376+0000 7f47f8c00640 1 --2- 172.21.15.131:0/1803998746 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47f41400c0 0x7f47f41404f0 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f47ec009e50 tx=0x7f47ec02a710 comp rx=0 tx=0).stop 2024-05-26T22:56:19.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.376+0000 7f47f8c00640 1 -- 172.21.15.131:0/1803998746 shutdown_connections 2024-05-26T22:56:19.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.376+0000 7f47f8c00640 1 --2- 172.21.15.131:0/1803998746 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47f41400c0 0x7f47f41404f0 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:19.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.376+0000 7f47f8c00640 1 --2- 172.21.15.131:0/1803998746 >> v2:172.21.15.187:3300/0 conn(0x7f47f415b730 0x7f47f415bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:19.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.376+0000 7f47f8c00640 1 -- 172.21.15.131:0/1803998746 >> 172.21.15.131:0/1803998746 conn(0x7f47f40b5660 msgr2=0x7f47f40b5a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:56:19.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.377+0000 7f47f8c00640 1 -- 172.21.15.131:0/1803998746 shutdown_connections 2024-05-26T22:56:19.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.377+0000 7f47f8c00640 1 -- 172.21.15.131:0/1803998746 wait complete. 2024-05-26T22:56:19.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.378+0000 7f47f8c00640 1 Processor -- start 2024-05-26T22:56:19.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.378+0000 7f47f8c00640 1 -- start start 2024-05-26T22:56:19.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.379+0000 7f47f8c00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f47f406d250 0x7f47f41f5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:19.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.379+0000 7f47f8c00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f47f41400c0 0x7f47f41f70f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:19.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.379+0000 7f47f8c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47f415b730 0x7f47f41f78c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:19.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.379+0000 7f47f8c00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f47f415c100 con 0x7f47f415b730 2024-05-26T22:56:19.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.379+0000 7f47f8c00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f47f4071a70 con 0x7f47f406d250 2024-05-26T22:56:19.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.379+0000 7f47f8c00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f47f415c280 con 0x7f47f41400c0 2024-05-26T22:56:19.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.380+0000 7f47f3e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47f415b730 0x7f47f41f78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:19.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.380+0000 7f47f3e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47f415b730 0x7f47f41f78c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55992/0 (socket says 172.21.15.131:55992) 2024-05-26T22:56:19.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.380+0000 7f47f3e00640 1 -- 172.21.15.131:0/808927195 learned_addr learned my addr 172.21.15.131:0/808927195 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:56:19.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.380+0000 7f47f2a00640 1 --2- 172.21.15.131:0/808927195 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f47f41400c0 0x7f47f41f70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:19.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.380+0000 7f47f3e00640 1 -- 172.21.15.131:0/808927195 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f47f41400c0 msgr2=0x7f47f41f70f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:19.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.380+0000 7f47f3e00640 1 --2- 172.21.15.131:0/808927195 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f47f41400c0 0x7f47f41f70f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:19.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.380+0000 7f47f3e00640 1 -- 172.21.15.131:0/808927195 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f47f406d250 msgr2=0x7f47f41f5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:19.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.380+0000 7f47f3e00640 1 --2- 172.21.15.131:0/808927195 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f47f406d250 0x7f47f41f5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:19.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.380+0000 7f47f3e00640 1 -- 172.21.15.131:0/808927195 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47f41578f0 con 0x7f47f415b730 2024-05-26T22:56:19.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.380+0000 7f47f3e00640 1 --2- 172.21.15.131:0/808927195 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47f415b730 0x7f47f41f78c0 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f47ec009e50 tx=0x7f47ec004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:19.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.381+0000 7f47eb400640 1 -- 172.21.15.131:0/808927195 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47ec036050 con 0x7f47f415b730 2024-05-26T22:56:19.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.381+0000 7f47eb400640 1 -- 172.21.15.131:0/808927195 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47ec042070 con 0x7f47f415b730 2024-05-26T22:56:19.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.381+0000 7f47eb400640 1 -- 172.21.15.131:0/808927195 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47ec009510 con 0x7f47f415b730 2024-05-26T22:56:19.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.381+0000 7f47f8c00640 1 -- 172.21.15.131:0/808927195 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47f41f7630 con 0x7f47f415b730 2024-05-26T22:56:19.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.381+0000 7f47f8c00640 1 -- 172.21.15.131:0/808927195 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47f41f85f0 con 0x7f47f415b730 2024-05-26T22:56:19.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.383+0000 7f47eb400640 1 -- 172.21.15.131:0/808927195 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f47ec007bb0 con 0x7f47f415b730 2024-05-26T22:56:19.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.384+0000 7f47eb400640 1 --2- 172.21.15.131:0/808927195 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f47d406be30 0x7f47d406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:19.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.384+0000 7f47eb400640 1 -- 172.21.15.131:0/808927195 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5709+0+0 (secure 0 0 0) 0x7f47ec0abeb0 con 0x7f47f415b730 2024-05-26T22:56:19.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.384+0000 7f47f3400640 1 --2- 172.21.15.131:0/808927195 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f47d406be30 0x7f47d406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:19.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.385+0000 7f47fcff5740 1 Processor -- start 2024-05-26T22:56:19.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.385+0000 7f47fcff5740 1 -- start start 2024-05-26T22:56:19.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.385+0000 7f47f3400640 1 --2- 172.21.15.131:0/808927195 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f47d406be30 0x7f47d406e2f0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f47f40b60d0 tx=0x7f47e40079b0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:19.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.386+0000 7f47fcff5740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x555a332cde50 0x555a33280a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:19.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.386+0000 7f47fcff5740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555a332b0f70 0x555a332b02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:19.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.386+0000 7f47fcff5740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555a332b0830 0x555a33285640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:19.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.386+0000 7f47fcff5740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x555a331e8610 con 0x555a332b0830 2024-05-26T22:56:19.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.386+0000 7f47fcff5740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x555a331905c0 con 0x555a332b0f70 2024-05-26T22:56:19.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.386+0000 7f47fcff5740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x555a331e45d0 con 0x555a332cde50 2024-05-26T22:56:19.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.386+0000 7f47f3e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555a332b0830 0x555a33285640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:19.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.386+0000 7f47f3e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555a332b0830 0x555a33285640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55998/0 (socket says 172.21.15.131:55998) 2024-05-26T22:56:19.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.386+0000 7f47f3e00640 1 -- 172.21.15.131:0/2373779017 learned_addr learned my addr 172.21.15.131:0/2373779017 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:56:19.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.386+0000 7f47f3400640 1 --2- 172.21.15.131:0/2373779017 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x555a332cde50 0x555a33280a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:19.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.386+0000 7f47f3e00640 1 -- 172.21.15.131:0/2373779017 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x555a332cde50 msgr2=0x555a33280a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:19.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.386+0000 7f47f3e00640 1 --2- 172.21.15.131:0/2373779017 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x555a332cde50 0x555a33280a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:19.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.386+0000 7f47f3e00640 1 -- 172.21.15.131:0/2373779017 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555a332b0f70 msgr2=0x555a332b02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:19.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.386+0000 7f47f2a00640 1 --2- 172.21.15.131:0/2373779017 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555a332b0f70 0x555a332b02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:19.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.386+0000 7f47f3e00640 1 --2- 172.21.15.131:0/2373779017 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555a332b0f70 0x555a332b02f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:19.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.386+0000 7f47f3e00640 1 -- 172.21.15.131:0/2373779017 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a32efc2d0 con 0x555a332b0830 2024-05-26T22:56:19.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.387+0000 7f47f3e00640 1 --2- 172.21.15.131:0/2373779017 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555a332b0830 0x555a33285640 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f47ec034040 tx=0x7f47ec079c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3ae3502157e1e13 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:19.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.387+0000 7f47df400640 1 -- 172.21.15.131:0/2373779017 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47ec092020 con 0x555a332b0830 2024-05-26T22:56:19.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.387+0000 7f47df400640 1 -- 172.21.15.131:0/2373779017 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47ec037040 con 0x555a332b0830 2024-05-26T22:56:19.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.387+0000 7f47df400640 1 -- 172.21.15.131:0/2373779017 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47ec031600 con 0x555a332b0830 2024-05-26T22:56:19.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.387+0000 7f47fcff5740 1 -- 172.21.15.131:0/2373779017 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555a332b0830 msgr2=0x555a33285640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:19.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.387+0000 7f47fcff5740 1 --2- 172.21.15.131:0/2373779017 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555a332b0830 0x555a33285640 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f47ec034040 tx=0x7f47ec079c00 comp rx=0 tx=0).stop 2024-05-26T22:56:19.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.388+0000 7f47fcff5740 1 -- 172.21.15.131:0/2373779017 shutdown_connections 2024-05-26T22:56:19.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.388+0000 7f47fcff5740 1 --2- 172.21.15.131:0/2373779017 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555a332b0830 0x555a33285640 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:19.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.388+0000 7f47fcff5740 1 --2- 172.21.15.131:0/2373779017 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555a332b0f70 0x555a332b02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:19.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.388+0000 7f47fcff5740 1 --2- 172.21.15.131:0/2373779017 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x555a332cde50 0x555a33280a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:19.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.388+0000 7f47fcff5740 1 -- 172.21.15.131:0/2373779017 >> 172.21.15.131:0/2373779017 conn(0x555a33282ce0 msgr2=0x555a332cb2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:56:19.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.388+0000 7f47fcff5740 1 -- 172.21.15.131:0/2373779017 shutdown_connections 2024-05-26T22:56:19.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.388+0000 7f47fcff5740 1 -- 172.21.15.131:0/2373779017 wait complete. 2024-05-26T22:56:19.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.388+0000 7f47fcff5740 1 -- start start 2024-05-26T22:56:19.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.389+0000 7f47fcff5740 1 Processor -- start 2024-05-26T22:56:19.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.389+0000 7f47fcff5740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x555a332b0f70 0x555a332a0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:19.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.389+0000 7f47fcff5740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555a332cde50 0x555a332b26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:19.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.390+0000 7f47fcff5740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555a332b2c30 0x555a3327f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:19.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.390+0000 7f47fcff5740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x555a331e8610 con 0x555a332b2c30 2024-05-26T22:56:19.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.390+0000 7f47fcff5740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x555a331a3910 con 0x555a332cde50 2024-05-26T22:56:19.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.390+0000 7f47fcff5740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x555a33284c00 con 0x555a332b0f70 2024-05-26T22:56:19.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.390+0000 7f47fcff5740 10 client.0 _ll_register_callbacks cb 0x555a331df410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-26T22:56:19.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.390+0000 7f47f3e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555a332b2c30 0x555a3327f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:19.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.390+0000 7f47f3e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555a332b2c30 0x555a3327f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56000/0 (socket says 172.21.15.131:56000) 2024-05-26T22:56:19.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.390+0000 7f47f3e00640 1 -- 172.21.15.131:0/1440654444 learned_addr learned my addr 172.21.15.131:0/1440654444 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:56:19.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.390+0000 7f47f2a00640 1 --2- 172.21.15.131:0/1440654444 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555a332cde50 0x555a332b26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:19.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.390+0000 7f47f3400640 1 --2- 172.21.15.131:0/1440654444 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x555a332b0f70 0x555a332a0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:19.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.390+0000 7f47f3e00640 1 -- 172.21.15.131:0/1440654444 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x555a332b0f70 msgr2=0x555a332a0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:19.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.390+0000 7f47f3e00640 1 --2- 172.21.15.131:0/1440654444 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x555a332b0f70 0x555a332a0dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:19.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.390+0000 7f47f3e00640 1 -- 172.21.15.131:0/1440654444 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555a332cde50 msgr2=0x555a332b26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:19.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.390+0000 7f47f3e00640 1 --2- 172.21.15.131:0/1440654444 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555a332cde50 0x555a332b26f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:19.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.390+0000 7f47f3e00640 1 -- 172.21.15.131:0/1440654444 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a32fc66b0 con 0x555a332b2c30 2024-05-26T22:56:19.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.390+0000 7f47f2a00640 1 --2- 172.21.15.131:0/1440654444 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555a332cde50 0x555a332b26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:56:19.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.390+0000 7f47f3e00640 1 --2- 172.21.15.131:0/1440654444 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555a332b2c30 0x555a3327f920 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f47ec007fd0 tx=0x7f47ec009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86bc438d48ff0381 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:19.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.391+0000 7f47d3e00640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-26T22:56:19.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.391+0000 7f47d3e00640 1 -- 172.21.15.131:0/1440654444 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47ec092040 con 0x555a332b2c30 2024-05-26T22:56:19.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.391+0000 7f47d3e00640 1 -- 172.21.15.131:0/1440654444 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 5709+0+0 (secure 0 0 0) 0x7f47ec079510 con 0x555a332b2c30 2024-05-26T22:56:19.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.391+0000 7f47fcff5740 1 -- 172.21.15.131:0/1440654444 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47ec037040 con 0x555a332b2c30 2024-05-26T22:56:19.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.391+0000 7f47d3e00640 1 -- 172.21.15.131:0/1440654444 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=115) ==== 24+0+0 (secure 0 0 0) 0x7f47ec099180 con 0x555a332b2c30 2024-05-26T22:56:19.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.391+0000 7f47fcff5740 10 client.8514 fetch_fsmap learned FSMap version 115 2024-05-26T22:56:19.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.391+0000 7f47fcff5740 1 -- 172.21.15.131:0/1440654444 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=115,monmap=2+}) -- 0x555a332b39c0 con 0x555a332b2c30 2024-05-26T22:56:19.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.391+0000 7f47d3e00640 1 -- 172.21.15.131:0/1440654444 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47ec037040 con 0x555a332b2c30 2024-05-26T22:56:19.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.391+0000 7f47d3e00640 1 -- 172.21.15.131:0/1440654444 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 115) ==== 3534+0+0 (secure 0 0 0) 0x7f47ec075e10 con 0x555a332b2c30 2024-05-26T22:56:19.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.391+0000 7f47fcff5740 10 client.8514 fetch_fsmap finished waiting for FSMap version 115 2024-05-26T22:56:19.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.391+0000 7f47fcff5740 10 client.8514 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-26T22:56:19.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.391+0000 7f47fcff5740 20 client.8514 populate_metadata read hostname 'smithi131' 2024-05-26T22:56:19.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.391+0000 7f47fcff5740 1 --2- 172.21.15.131:0/1440654444 >> [v2:172.21.15.131:6834/2710427205,v1:172.21.15.131:6836/2710427205] conn(0x555a332db590 0x555a332db970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:19.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.391+0000 7f47fcff5740 4 client.8514 mds_command: new command op to 8307 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-26T22:56:19.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.391+0000 7f47fcff5740 1 -- 172.21.15.131:0/1440654444 --> [v2:172.21.15.131:6834/2710427205,v1:172.21.15.131:6836/2710427205] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555a331f5d40 con 0x555a332db590 2024-05-26T22:56:19.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.392+0000 7f47f3400640 1 --2- 172.21.15.131:0/1440654444 >> [v2:172.21.15.131:6834/2710427205,v1:172.21.15.131:6836/2710427205] conn(0x555a332db590 0x555a332db970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:19.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.392+0000 7f47f3400640 1 --2- 172.21.15.131:0/1440654444 >> [v2:172.21.15.131:6834/2710427205,v1:172.21.15.131:6836/2710427205] conn(0x555a332db590 0x555a332db970 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c38d8eb8f6f7458e server_cookie=9d3fc5df96b33f98 in_seq=0 out_seq=0 2024-05-26T22:56:19.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.392+0000 7f47d3e00640 10 client.8514 ms_handle_connect on v2:172.21.15.131:6834/2710427205 2024-05-26T22:56:19.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.393+0000 7f47d3e00640 1 -- 172.21.15.131:0/1440654444 <== mds.1 v2:172.21.15.131:6834/2710427205 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555a331f5d40 con 0x555a332db590 2024-05-26T22:56:19.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.393+0000 7f47d3e00640 10 client.8514 handle_command_reply: tid=0 2024-05-26T22:56:19.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.393+0000 7f47fcff5740 1 client.8514 shutdown 2024-05-26T22:56:19.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.393+0000 7f47fcff5740 1 -- 172.21.15.131:0/1440654444 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555a332b2c30 msgr2=0x555a3327f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:19.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.393+0000 7f47fcff5740 1 --2- 172.21.15.131:0/1440654444 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555a332b2c30 0x555a3327f920 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f47ec007fd0 tx=0x7f47ec009810 comp rx=0 tx=0).stop 2024-05-26T22:56:19.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.394+0000 7f47fcff5740 1 -- 172.21.15.131:0/1440654444 shutdown_connections 2024-05-26T22:56:19.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.394+0000 7f47fcff5740 1 --2- 172.21.15.131:0/1440654444 >> [v2:172.21.15.131:6834/2710427205,v1:172.21.15.131:6836/2710427205] conn(0x555a332db590 0x555a332db970 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:19.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.394+0000 7f47fcff5740 1 --2- 172.21.15.131:0/1440654444 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555a332b2c30 0x555a3327f920 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:19.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.394+0000 7f47fcff5740 1 --2- 172.21.15.131:0/1440654444 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555a332cde50 0x555a332b26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:19.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.394+0000 7f47fcff5740 1 --2- 172.21.15.131:0/1440654444 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x555a332b0f70 0x555a332a0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:19.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.394+0000 7f47d3e00640 0 client.8514 ms_handle_reset on v2:172.21.15.131:6834/2710427205 2024-05-26T22:56:19.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.394+0000 7f47fcff5740 1 -- 172.21.15.131:0/1440654444 >> 172.21.15.131:0/1440654444 conn(0x555a332b0830 msgr2=0x555a332842d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:56:19.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.394+0000 7f47fcff5740 1 -- 172.21.15.131:0/1440654444 shutdown_connections 2024-05-26T22:56:19.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.394+0000 7f47fcff5740 1 -- 172.21.15.131:0/1440654444 wait complete. 2024-05-26T22:56:19.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.394+0000 7f47fcff5740 20 client.8514 trim_cache size 0 max 16384 2024-05-26T22:56:19.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.414+0000 7f47fcff5740 1 Processor -- start 2024-05-26T22:56:19.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.414+0000 7f47fcff5740 1 -- start start 2024-05-26T22:56:19.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.415+0000 7f47fcff5740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555a332b0f70 0x555a33286470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:19.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.415+0000 7f47fcff5740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555a332b2c30 0x555a33280680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:19.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.415+0000 7f47fcff5740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x555a332cde50 0x555a33284cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:19.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.415+0000 7f47fcff5740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f47f41ffc50 con 0x555a332b0f70 2024-05-26T22:56:19.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.415+0000 7f47fcff5740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f47f4200120 con 0x555a332b2c30 2024-05-26T22:56:19.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.415+0000 7f47fcff5740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f47f41f4310 con 0x555a332cde50 2024-05-26T22:56:19.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.415+0000 7f47f3400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555a332b0f70 0x555a33286470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:19.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.415+0000 7f47f3400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555a332b0f70 0x555a33286470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56008/0 (socket says 172.21.15.131:56008) 2024-05-26T22:56:19.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.415+0000 7f47f3400640 1 -- 172.21.15.131:0/1186054890 learned_addr learned my addr 172.21.15.131:0/1186054890 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:56:19.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.416+0000 7f47f3400640 1 -- 172.21.15.131:0/1186054890 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x555a332cde50 msgr2=0x555a33284cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:19.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.416+0000 7f47f3400640 1 --2- 172.21.15.131:0/1186054890 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x555a332cde50 0x555a33284cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:19.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.416+0000 7f47f3400640 1 -- 172.21.15.131:0/1186054890 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555a332b2c30 msgr2=0x555a33280680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:19.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.416+0000 7f47f3400640 1 --2- 172.21.15.131:0/1186054890 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555a332b2c30 0x555a33280680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:19.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.416+0000 7f47f3400640 1 -- 172.21.15.131:0/1186054890 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a3321dbb0 con 0x555a332b0f70 2024-05-26T22:56:19.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.416+0000 7f47f3400640 1 --2- 172.21.15.131:0/1186054890 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555a332b0f70 0x555a33286470 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f47e40050e0 tx=0x7f47e400f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92f54f160266d62 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:19.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.416+0000 7f47dea00640 1 -- 172.21.15.131:0/1186054890 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e4029070 con 0x555a332b0f70 2024-05-26T22:56:19.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.416+0000 7f47dea00640 1 -- 172.21.15.131:0/1186054890 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47e4023640 con 0x555a332b0f70 2024-05-26T22:56:19.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.416+0000 7f47dea00640 1 -- 172.21.15.131:0/1186054890 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e4023a30 con 0x555a332b0f70 2024-05-26T22:56:19.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.417+0000 7f47fcff5740 1 -- 172.21.15.131:0/1186054890 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555a332b0f70 msgr2=0x555a33286470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:19.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.417+0000 7f47fcff5740 1 --2- 172.21.15.131:0/1186054890 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555a332b0f70 0x555a33286470 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f47e40050e0 tx=0x7f47e400f000 comp rx=0 tx=0).stop 2024-05-26T22:56:19.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.417+0000 7f47fcff5740 1 -- 172.21.15.131:0/1186054890 shutdown_connections 2024-05-26T22:56:19.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.417+0000 7f47fcff5740 1 --2- 172.21.15.131:0/1186054890 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x555a332cde50 0x555a33284cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:19.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.417+0000 7f47fcff5740 1 --2- 172.21.15.131:0/1186054890 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555a332b2c30 0x555a33280680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:19.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.417+0000 7f47fcff5740 1 --2- 172.21.15.131:0/1186054890 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555a332b0f70 0x555a33286470 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:19.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.417+0000 7f47fcff5740 1 -- 172.21.15.131:0/1186054890 >> 172.21.15.131:0/1186054890 conn(0x555a332b0830 msgr2=0x555a332b23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:56:19.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.417+0000 7f47fcff5740 1 -- 172.21.15.131:0/1186054890 shutdown_connections 2024-05-26T22:56:19.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.417+0000 7f47fcff5740 1 -- 172.21.15.131:0/1186054890 wait complete. 2024-05-26T22:56:19.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.418+0000 7f47fcff5740 1 -- start start 2024-05-26T22:56:19.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.418+0000 7f47fcff5740 1 Processor -- start 2024-05-26T22:56:19.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.419+0000 7f47fcff5740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x555a332b0f70 0x555a33284c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:19.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.419+0000 7f47fcff5740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555a332b2c30 0x555a332851a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:19.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.419+0000 7f47fcff5740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555a332cde50 0x555a332d9930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:19.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.419+0000 7f47fcff5740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x555a331e8610 con 0x555a332cde50 2024-05-26T22:56:19.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.419+0000 7f47fcff5740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x555a331a3910 con 0x555a332b2c30 2024-05-26T22:56:19.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.419+0000 7f47fcff5740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x555a332b1c10 con 0x555a332b0f70 2024-05-26T22:56:19.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.419+0000 7f47fcff5740 10 client.0 _ll_register_callbacks cb 0x7f47ec0763f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-26T22:56:19.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.419+0000 7f47f3e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555a332cde50 0x555a332d9930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:19.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.419+0000 7f47f3e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555a332cde50 0x555a332d9930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56022/0 (socket says 172.21.15.131:56022) 2024-05-26T22:56:19.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.419+0000 7f47f3e00640 1 -- 172.21.15.131:0/1596421040 learned_addr learned my addr 172.21.15.131:0/1596421040 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:56:19.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.420+0000 7f47f3e00640 1 -- 172.21.15.131:0/1596421040 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x555a332b0f70 msgr2=0x555a33284c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:19.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.420+0000 7f47f3e00640 1 --2- 172.21.15.131:0/1596421040 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x555a332b0f70 0x555a33284c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:19.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.420+0000 7f47f3e00640 1 -- 172.21.15.131:0/1596421040 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555a332b2c30 msgr2=0x555a332851a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:19.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.420+0000 7f47f3e00640 1 --2- 172.21.15.131:0/1596421040 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555a332b2c30 0x555a332851a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:19.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.420+0000 7f47f3e00640 1 -- 172.21.15.131:0/1596421040 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a332e4200 con 0x555a332cde50 2024-05-26T22:56:19.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.420+0000 7f47f3e00640 1 --2- 172.21.15.131:0/1596421040 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555a332cde50 0x555a332d9930 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f47ec078370 tx=0x7f47ec0aa210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55813bc758f3bc79 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:19.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.420+0000 7f47d3e00640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-26T22:56:19.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.420+0000 7f47d3e00640 1 -- 172.21.15.131:0/1596421040 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47ec092040 con 0x555a332cde50 2024-05-26T22:56:19.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.420+0000 7f47d3e00640 1 -- 172.21.15.131:0/1596421040 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 5709+0+0 (secure 0 0 0) 0x7f47ec079510 con 0x555a332cde50 2024-05-26T22:56:19.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.420+0000 7f47fcff5740 1 -- 172.21.15.131:0/1596421040 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47e4023640 con 0x555a332cde50 2024-05-26T22:56:19.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.420+0000 7f47d3e00640 1 -- 172.21.15.131:0/1596421040 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=115) ==== 24+0+0 (secure 0 0 0) 0x7f47ec08e050 con 0x555a332cde50 2024-05-26T22:56:19.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.420+0000 7f47fcff5740 10 client.8520 fetch_fsmap learned FSMap version 115 2024-05-26T22:56:19.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.420+0000 7f47fcff5740 1 -- 172.21.15.131:0/1596421040 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=115,monmap=2+}) -- 0x555a332e4200 con 0x555a332cde50 2024-05-26T22:56:19.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.421+0000 7f47d3e00640 1 -- 172.21.15.131:0/1596421040 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47e4023640 con 0x555a332cde50 2024-05-26T22:56:19.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.421+0000 7f47d3e00640 1 -- 172.21.15.131:0/1596421040 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 115) ==== 3534+0+0 (secure 0 0 0) 0x7f47ec078f80 con 0x555a332cde50 2024-05-26T22:56:19.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.421+0000 7f47fcff5740 10 client.8520 fetch_fsmap finished waiting for FSMap version 115 2024-05-26T22:56:19.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.421+0000 7f47fcff5740 10 client.8520 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-26T22:56:19.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.421+0000 7f47fcff5740 20 client.8520 populate_metadata read hostname 'smithi131' 2024-05-26T22:56:19.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.421+0000 7f47fcff5740 1 --2- 172.21.15.131:0/1596421040 >> [v2:172.21.15.131:6834/2710427205,v1:172.21.15.131:6836/2710427205] conn(0x555a332da3c0 0x555a332c82f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:19.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.421+0000 7f47fcff5740 4 client.8520 mds_command: new command op to 8307 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-26T22:56:19.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.421+0000 7f47fcff5740 1 -- 172.21.15.131:0/1596421040 --> [v2:172.21.15.131:6834/2710427205,v1:172.21.15.131:6836/2710427205] -- command(tid 0: {"prefix": "status"}) -- 0x555a331f0090 con 0x555a332da3c0 2024-05-26T22:56:19.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.421+0000 7f47f3400640 1 --2- 172.21.15.131:0/1596421040 >> [v2:172.21.15.131:6834/2710427205,v1:172.21.15.131:6836/2710427205] conn(0x555a332da3c0 0x555a332c82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:19.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.421+0000 7f47f3400640 1 --2- 172.21.15.131:0/1596421040 >> [v2:172.21.15.131:6834/2710427205,v1:172.21.15.131:6836/2710427205] conn(0x555a332da3c0 0x555a332c82f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=162046e9699e5458 server_cookie=132a930253891b89 in_seq=0 out_seq=0 2024-05-26T22:56:19.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.421+0000 7f47d3e00640 10 client.8520 ms_handle_connect on v2:172.21.15.131:6834/2710427205 2024-05-26T22:56:19.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.422+0000 7f47d3e00640 1 -- 172.21.15.131:0/1596421040 <== mds.1 v2:172.21.15.131:6834/2710427205 1 ==== command_reply(tid 0: 0 ) ==== 8+0+334 (crc 0 0 0) 0x555a331f0090 con 0x555a332da3c0 2024-05-26T22:56:19.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.422+0000 7f47d3e00640 10 client.8520 handle_command_reply: tid=0 2024-05-26T22:56:19.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.422+0000 7f47fcff5740 1 client.8520 shutdown 2024-05-26T22:56:19.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.422+0000 7f47fcff5740 1 -- 172.21.15.131:0/1596421040 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555a332cde50 msgr2=0x555a332d9930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:19.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.422+0000 7f47fcff5740 1 --2- 172.21.15.131:0/1596421040 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555a332cde50 0x555a332d9930 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f47ec078370 tx=0x7f47ec0aa210 comp rx=0 tx=0).stop 2024-05-26T22:56:19.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.422+0000 7f47fcff5740 1 -- 172.21.15.131:0/1596421040 shutdown_connections 2024-05-26T22:56:19.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.422+0000 7f47fcff5740 1 --2- 172.21.15.131:0/1596421040 >> [v2:172.21.15.131:6834/2710427205,v1:172.21.15.131:6836/2710427205] conn(0x555a332da3c0 0x555a332c82f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:19.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.422+0000 7f47fcff5740 1 --2- 172.21.15.131:0/1596421040 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555a332cde50 0x555a332d9930 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:19.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.422+0000 7f47fcff5740 1 --2- 172.21.15.131:0/1596421040 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555a332b2c30 0x555a332851a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:19.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.422+0000 7f47fcff5740 1 --2- 172.21.15.131:0/1596421040 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x555a332b0f70 0x555a33284c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:19.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.422+0000 7f47d3e00640 0 client.8520 ms_handle_reset on v2:172.21.15.131:6834/2710427205 2024-05-26T22:56:19.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.422+0000 7f47fcff5740 1 -- 172.21.15.131:0/1596421040 >> 172.21.15.131:0/1596421040 conn(0x555a332b0830 msgr2=0x555a332859a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:56:19.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.422+0000 7f47fcff5740 1 -- 172.21.15.131:0/1596421040 shutdown_connections 2024-05-26T22:56:19.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.422+0000 7f47fcff5740 1 -- 172.21.15.131:0/1596421040 wait complete. 2024-05-26T22:56:19.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.422+0000 7f47fcff5740 20 client.8520 trim_cache size 0 max 16384 2024-05-26T22:56:19.425 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-26T22:56:19.426 INFO:teuthology.orchestra.run.smithi131.stdout: "cluster_fsid": "46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f", 2024-05-26T22:56:19.426 INFO:teuthology.orchestra.run.smithi131.stdout: "whoami": 1, 2024-05-26T22:56:19.426 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 8307, 2024-05-26T22:56:19.426 INFO:teuthology.orchestra.run.smithi131.stdout: "want_state": "up:active", 2024-05-26T22:56:19.426 INFO:teuthology.orchestra.run.smithi131.stdout: "state": "up:active", 2024-05-26T22:56:19.426 INFO:teuthology.orchestra.run.smithi131.stdout: "fs_name": "cephfs", 2024-05-26T22:56:19.426 INFO:teuthology.orchestra.run.smithi131.stdout: "rank_uptime": 3.2621879900000001, 2024-05-26T22:56:19.426 INFO:teuthology.orchestra.run.smithi131.stdout: "mdsmap_epoch": 115, 2024-05-26T22:56:19.426 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch": 107, 2024-05-26T22:56:19.426 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch_barrier": 107, 2024-05-26T22:56:19.426 INFO:teuthology.orchestra.run.smithi131.stdout: "uptime": 29.371440440000001 2024-05-26T22:56:19.426 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-26T22:56:19.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.423+0000 7f47f8c00640 1 -- 172.21.15.131:0/808927195 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f47d406be30 msgr2=0x7f47d406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:19.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.423+0000 7f47f8c00640 1 --2- 172.21.15.131:0/808927195 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f47d406be30 0x7f47d406e2f0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f47f40b60d0 tx=0x7f47e40079b0 comp rx=0 tx=0).stop 2024-05-26T22:56:19.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.423+0000 7f47f8c00640 1 -- 172.21.15.131:0/808927195 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47f415b730 msgr2=0x7f47f41f78c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:19.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.423+0000 7f47f8c00640 1 --2- 172.21.15.131:0/808927195 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47f415b730 0x7f47f41f78c0 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f47ec009e50 tx=0x7f47ec004a40 comp rx=0 tx=0).stop 2024-05-26T22:56:19.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.423+0000 7f47f8c00640 1 -- 172.21.15.131:0/808927195 shutdown_connections 2024-05-26T22:56:19.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.423+0000 7f47f8c00640 1 --2- 172.21.15.131:0/808927195 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f47f41400c0 0x7f47f41f70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:19.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.423+0000 7f47f8c00640 1 --2- 172.21.15.131:0/808927195 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f47d406be30 0x7f47d406e2f0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:19.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.423+0000 7f47f8c00640 1 --2- 172.21.15.131:0/808927195 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47f415b730 0x7f47f41f78c0 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:19.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.423+0000 7f47f8c00640 1 --2- 172.21.15.131:0/808927195 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f47f406d250 0x7f47f41f5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:19.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.423+0000 7f47f8c00640 1 -- 172.21.15.131:0/808927195 >> 172.21.15.131:0/808927195 conn(0x7f47f40b5660 msgr2=0x7f47f40b0e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:56:19.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.423+0000 7f47f8c00640 1 -- 172.21.15.131:0/808927195 shutdown_connections 2024-05-26T22:56:19.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.423+0000 7f47f8c00640 1 -- 172.21.15.131:0/808927195 wait complete. 2024-05-26T22:56:19.440 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-05-26T22:56:19.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.637+0000 7f2cda600640 1 Processor -- start 2024-05-26T22:56:19.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.638+0000 7f2cda600640 1 -- start start 2024-05-26T22:56:19.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.639+0000 7f2cda600640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2cd415e320 0x7f2cd40ae250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:19.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.639+0000 7f2cda600640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f2cd40d0550 0x7f2cd40715d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:19.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.639+0000 7f2cda600640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2cd40b2440 con 0x7f2cd40b2f60 2024-05-26T22:56:19.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.639+0000 7f2cda600640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f2cd4161fe0 con 0x7f2cd40d0550 2024-05-26T22:56:19.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.639+0000 7f2cda600640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f2cd4162230 con 0x7f2cd415e320 2024-05-26T22:56:19.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.640+0000 7f2cd3e00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f2cd40b2f60 0x7f2cd40cfe40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:50102/0 (socket says 172.21.15.131:50102) 2024-05-26T22:56:19.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.640+0000 7f2cd3e00640 1 -- 172.21.15.131:0/101302009 learned_addr learned my addr 172.21.15.131:0/101302009 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:56:19.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.640+0000 7f2cd9c00640 1 --2- 172.21.15.131:0/101302009 >> v2:172.21.15.131:3300/0 conn(0x7f2cd40d0550 0x7f2cd40715d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:19.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.640+0000 7f2cd9200640 1 --2- 172.21.15.131:0/101302009 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2cd415e320 0x7f2cd40ae250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:19.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.640+0000 7f2cd3400640 1 -- 172.21.15.131:0/101302009 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1802539917 0 0) 0x7f2cd40b2440 con 0x7f2cd40b2f60 2024-05-26T22:56:19.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.640+0000 7f2cd3400640 1 -- 172.21.15.131:0/101302009 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2cb8003520 con 0x7f2cd40b2f60 2024-05-26T22:56:19.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.640+0000 7f2cd9c00640 1 -- 172.21.15.131:0/101302009 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2cd415e320 msgr2=0x7f2cd40ae250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:19.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.640+0000 7f2cd9c00640 1 --2- 172.21.15.131:0/101302009 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2cd415e320 0x7f2cd40ae250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:19.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.640+0000 7f2cd9c00640 1 -- 172.21.15.131:0/101302009 >> v1:172.21.15.131:6789/0 conn(0x7f2cd40b2f60 legacy=0x7f2cd40cfe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:19.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.640+0000 7f2cd9c00640 1 -- 172.21.15.131:0/101302009 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cd4162510 con 0x7f2cd40d0550 2024-05-26T22:56:19.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.640+0000 7f2cd9200640 1 --2- 172.21.15.131:0/101302009 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2cd415e320 0x7f2cd40ae250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-26T22:56:19.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.641+0000 7f2cd9c00640 1 --2- 172.21.15.131:0/101302009 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2cd40d0550 0x7f2cd40715d0 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc009d90 tx=0x7f2ccc02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1ddbd4aa1f93ecb server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:19.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.642+0000 7f2cd3400640 1 -- 172.21.15.131:0/101302009 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ccc007740 con 0x7f2cd40d0550 2024-05-26T22:56:19.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.642+0000 7f2cd3400640 1 -- 172.21.15.131:0/101302009 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ccc0078e0 con 0x7f2cd40d0550 2024-05-26T22:56:19.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.642+0000 7f2cd3400640 1 -- 172.21.15.131:0/101302009 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ccc007cb0 con 0x7f2cd40d0550 2024-05-26T22:56:19.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.642+0000 7f2cda600640 1 -- 172.21.15.131:0/101302009 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2cd40d0550 msgr2=0x7f2cd40715d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:19.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.642+0000 7f2cda600640 1 --2- 172.21.15.131:0/101302009 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2cd40d0550 0x7f2cd40715d0 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc009d90 tx=0x7f2ccc02a6f0 comp rx=0 tx=0).stop 2024-05-26T22:56:19.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.642+0000 7f2cda600640 1 -- 172.21.15.131:0/101302009 shutdown_connections 2024-05-26T22:56:19.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.642+0000 7f2cda600640 1 --2- 172.21.15.131:0/101302009 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2cd40d0550 0x7f2cd40715d0 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:19.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.642+0000 7f2cda600640 1 --2- 172.21.15.131:0/101302009 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2cd415e320 0x7f2cd40ae250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:19.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.642+0000 7f2cda600640 1 -- 172.21.15.131:0/101302009 >> 172.21.15.131:0/101302009 conn(0x7f2cd40b5660 msgr2=0x7f2cd40b5a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:56:19.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.642+0000 7f2cda600640 1 -- 172.21.15.131:0/101302009 shutdown_connections 2024-05-26T22:56:19.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.642+0000 7f2cda600640 1 -- 172.21.15.131:0/101302009 wait complete. 2024-05-26T22:56:19.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.644+0000 7f2cda600640 1 Processor -- start 2024-05-26T22:56:19.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.644+0000 7f2cda600640 1 -- start start 2024-05-26T22:56:19.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.645+0000 7f2cda600640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2cd40b2f60 0x7f2cd41474e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:19.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.645+0000 7f2cda600640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2cd40d0550 0x7f2cd4144d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:19.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.645+0000 7f2cda600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2cd415e320 0x7f2cd414b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:19.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.645+0000 7f2cda600640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f2cd4162e40 con 0x7f2cd415e320 2024-05-26T22:56:19.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.645+0000 7f2cda600640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2cd4162ad0 con 0x7f2cd40b2f60 2024-05-26T22:56:19.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.645+0000 7f2cda600640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f2cd40b3570 con 0x7f2cd40d0550 2024-05-26T22:56:19.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.646+0000 7f2cd9c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2cd415e320 0x7f2cd414b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:19.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.646+0000 7f2cd9c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2cd415e320 0x7f2cd414b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56046/0 (socket says 172.21.15.131:56046) 2024-05-26T22:56:19.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.646+0000 7f2cd9c00640 1 -- 172.21.15.131:0/1370606780 learned_addr learned my addr 172.21.15.131:0/1370606780 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:56:19.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.646+0000 7f2cd3e00640 1 --2- 172.21.15.131:0/1370606780 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2cd40d0550 0x7f2cd4144d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:19.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.646+0000 7f2cd9200640 1 --2- 172.21.15.131:0/1370606780 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2cd40b2f60 0x7f2cd41474e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:19.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.646+0000 7f2cd9c00640 1 -- 172.21.15.131:0/1370606780 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2cd40d0550 msgr2=0x7f2cd4144d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:19.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.646+0000 7f2cd9c00640 1 --2- 172.21.15.131:0/1370606780 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2cd40d0550 0x7f2cd4144d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:19.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.646+0000 7f2cd9c00640 1 -- 172.21.15.131:0/1370606780 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2cd40b2f60 msgr2=0x7f2cd41474e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:19.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.646+0000 7f2cd9c00640 1 --2- 172.21.15.131:0/1370606780 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2cd40b2f60 0x7f2cd41474e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:19.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.646+0000 7f2cd9c00640 1 -- 172.21.15.131:0/1370606780 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cd4147ae0 con 0x7f2cd415e320 2024-05-26T22:56:19.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.646+0000 7f2cd3e00640 1 --2- 172.21.15.131:0/1370606780 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2cd40d0550 0x7f2cd4144d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-26T22:56:19.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.647+0000 7f2cd9c00640 1 --2- 172.21.15.131:0/1370606780 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2cd415e320 0x7f2cd414b530 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc02ace0 tx=0x7f2ccc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:19.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.647+0000 7f2cd0c00640 1 -- 172.21.15.131:0/1370606780 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ccc040070 con 0x7f2cd415e320 2024-05-26T22:56:19.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.647+0000 7f2cd0c00640 1 -- 172.21.15.131:0/1370606780 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ccc031bc0 con 0x7f2cd415e320 2024-05-26T22:56:19.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.647+0000 7f2cd0c00640 1 -- 172.21.15.131:0/1370606780 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ccc009450 con 0x7f2cd415e320 2024-05-26T22:56:19.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.647+0000 7f2cda600640 1 -- 172.21.15.131:0/1370606780 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cd4147dc0 con 0x7f2cd415e320 2024-05-26T22:56:19.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.647+0000 7f2cda600640 1 -- 172.21.15.131:0/1370606780 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cd41f7670 con 0x7f2cd415e320 2024-05-26T22:56:19.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.649+0000 7f2cd0c00640 1 -- 172.21.15.131:0/1370606780 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2ccc039040 con 0x7f2cd415e320 2024-05-26T22:56:19.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.650+0000 7f2cd0c00640 1 --2- 172.21.15.131:0/1370606780 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f2cb006be30 0x7f2cb006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:19.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.650+0000 7f2cd0c00640 1 -- 172.21.15.131:0/1370606780 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5709+0+0 (secure 0 0 0) 0x7f2ccc0ac080 con 0x7f2cd415e320 2024-05-26T22:56:19.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.651+0000 7f2cd9200640 1 --2- 172.21.15.131:0/1370606780 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f2cb006be30 0x7f2cb006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:19.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.651+0000 7f2cdea07740 1 Processor -- start 2024-05-26T22:56:19.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.651+0000 7f2cd9200640 1 --2- 172.21.15.131:0/1370606780 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f2cb006be30 0x7f2cb006e2f0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f2cc4006fd0 tx=0x7f2cc4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:19.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.651+0000 7f2cdea07740 1 -- start start 2024-05-26T22:56:19.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.652+0000 7f2cdea07740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55c25bf3ce50 0x55c25beefa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:19.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.652+0000 7f2cdea07740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c25bf1ff70 0x55c25bf1f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:19.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.652+0000 7f2cdea07740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c25bf1f830 0x55c25bef4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:19.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.652+0000 7f2cdea07740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55c25be57610 con 0x55c25bf1f830 2024-05-26T22:56:19.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.652+0000 7f2cdea07740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55c25bdff5c0 con 0x55c25bf1ff70 2024-05-26T22:56:19.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.652+0000 7f2cdea07740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x55c25be535d0 con 0x55c25bf3ce50 2024-05-26T22:56:19.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.652+0000 7f2cd9c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c25bf1f830 0x55c25bef4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:19.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.652+0000 7f2cd9c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c25bf1f830 0x55c25bef4640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56058/0 (socket says 172.21.15.131:56058) 2024-05-26T22:56:19.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.652+0000 7f2cd9c00640 1 -- 172.21.15.131:0/605458162 learned_addr learned my addr 172.21.15.131:0/605458162 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:56:19.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.653+0000 7f2cd9200640 1 --2- 172.21.15.131:0/605458162 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55c25bf3ce50 0x55c25beefa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:19.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.653+0000 7f2cd9c00640 1 -- 172.21.15.131:0/605458162 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55c25bf3ce50 msgr2=0x55c25beefa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:19.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.653+0000 7f2cd9c00640 1 --2- 172.21.15.131:0/605458162 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55c25bf3ce50 0x55c25beefa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:19.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.653+0000 7f2cd9c00640 1 -- 172.21.15.131:0/605458162 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c25bf1ff70 msgr2=0x55c25bf1f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:19.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.653+0000 7f2cd9c00640 1 --2- 172.21.15.131:0/605458162 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c25bf1ff70 0x55c25bf1f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:19.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.653+0000 7f2cd9c00640 1 -- 172.21.15.131:0/605458162 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c25bb6b2d0 con 0x55c25bf1f830 2024-05-26T22:56:19.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.653+0000 7f2cd9c00640 1 --2- 172.21.15.131:0/605458162 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c25bf1f830 0x55c25bef4640 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc0aa4f0 tx=0x7f2ccc009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98d057eec93b2061 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:19.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.653+0000 7f2cc9600640 1 -- 172.21.15.131:0/605458162 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ccc092030 con 0x55c25bf1f830 2024-05-26T22:56:19.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.653+0000 7f2cc9600640 1 -- 172.21.15.131:0/605458162 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ccc07e6e0 con 0x55c25bf1f830 2024-05-26T22:56:19.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.653+0000 7f2cc9600640 1 -- 172.21.15.131:0/605458162 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ccc082f10 con 0x55c25bf1f830 2024-05-26T22:56:19.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.654+0000 7f2cdea07740 1 -- 172.21.15.131:0/605458162 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c25bf1f830 msgr2=0x55c25bef4640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:19.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.654+0000 7f2cdea07740 1 --2- 172.21.15.131:0/605458162 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c25bf1f830 0x55c25bef4640 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc0aa4f0 tx=0x7f2ccc009750 comp rx=0 tx=0).stop 2024-05-26T22:56:19.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.654+0000 7f2cdea07740 1 -- 172.21.15.131:0/605458162 shutdown_connections 2024-05-26T22:56:19.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.654+0000 7f2cdea07740 1 --2- 172.21.15.131:0/605458162 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c25bf1f830 0x55c25bef4640 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:19.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.654+0000 7f2cdea07740 1 --2- 172.21.15.131:0/605458162 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c25bf1ff70 0x55c25bf1f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:19.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.654+0000 7f2cdea07740 1 --2- 172.21.15.131:0/605458162 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55c25bf3ce50 0x55c25beefa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:19.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.654+0000 7f2cdea07740 1 -- 172.21.15.131:0/605458162 >> 172.21.15.131:0/605458162 conn(0x55c25bef1ce0 msgr2=0x55c25bf3a2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:56:19.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.654+0000 7f2cdea07740 1 -- 172.21.15.131:0/605458162 shutdown_connections 2024-05-26T22:56:19.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.654+0000 7f2cdea07740 1 -- 172.21.15.131:0/605458162 wait complete. 2024-05-26T22:56:19.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.655+0000 7f2cdea07740 1 -- start start 2024-05-26T22:56:19.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.655+0000 7f2cdea07740 1 Processor -- start 2024-05-26T22:56:19.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.656+0000 7f2cdea07740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55c25bf1ff70 0x55c25bf388b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:19.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.656+0000 7f2cdea07740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c25bf3ce50 0x55c25bef4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:19.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.656+0000 7f2cdea07740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c25bf0ff50 0x55c25bef4540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:19.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.656+0000 7f2cdea07740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55c25be57610 con 0x55c25bf3ce50 2024-05-26T22:56:19.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.656+0000 7f2cdea07740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55c25be12910 con 0x55c25bf0ff50 2024-05-26T22:56:19.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.656+0000 7f2cdea07740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x55c25bef3c00 con 0x55c25bf1ff70 2024-05-26T22:56:19.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.656+0000 7f2cdea07740 10 client.0 _ll_register_callbacks cb 0x55c25be4e410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-26T22:56:19.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.656+0000 7f2cd3e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c25bf3ce50 0x55c25bef4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:19.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.656+0000 7f2cd3e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c25bf3ce50 0x55c25bef4000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56064/0 (socket says 172.21.15.131:56064) 2024-05-26T22:56:19.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.656+0000 7f2cd3e00640 1 -- 172.21.15.131:0/2281512358 learned_addr learned my addr 172.21.15.131:0/2281512358 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:56:19.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.657+0000 7f2cd9200640 1 --2- 172.21.15.131:0/2281512358 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55c25bf1ff70 0x55c25bf388b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:19.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.657+0000 7f2cd3e00640 1 -- 172.21.15.131:0/2281512358 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55c25bf1ff70 msgr2=0x55c25bf388b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:19.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.657+0000 7f2cd3e00640 1 --2- 172.21.15.131:0/2281512358 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55c25bf1ff70 0x55c25bf388b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:19.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.657+0000 7f2cd3e00640 1 -- 172.21.15.131:0/2281512358 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c25bf0ff50 msgr2=0x55c25bef4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:19.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.657+0000 7f2cd3e00640 1 --2- 172.21.15.131:0/2281512358 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c25bf0ff50 0x55c25bef4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:19.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.657+0000 7f2cd3e00640 1 -- 172.21.15.131:0/2281512358 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c25bc356b0 con 0x55c25bf3ce50 2024-05-26T22:56:19.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.657+0000 7f2cd3e00640 1 --2- 172.21.15.131:0/2281512358 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c25bf3ce50 0x55c25bef4000 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc00e190 tx=0x7f2cbc00e650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2bc0e176594ddd2 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:19.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.657+0000 7f2cc2000640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-26T22:56:19.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.657+0000 7f2cc2000640 1 -- 172.21.15.131:0/2281512358 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cbc019060 con 0x55c25bf3ce50 2024-05-26T22:56:19.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.657+0000 7f2cc2000640 1 -- 172.21.15.131:0/2281512358 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 5709+0+0 (secure 0 0 0) 0x7f2cbc0058b0 con 0x55c25bf3ce50 2024-05-26T22:56:19.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.657+0000 7f2cdea07740 1 -- 172.21.15.131:0/2281512358 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ccc07e6e0 con 0x55c25bf3ce50 2024-05-26T22:56:19.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.658+0000 7f2cc2000640 1 -- 172.21.15.131:0/2281512358 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=115) ==== 24+0+0 (secure 0 0 0) 0x7f2cbc01f050 con 0x55c25bf3ce50 2024-05-26T22:56:19.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.658+0000 7f2cdea07740 10 client.8535 fetch_fsmap learned FSMap version 115 2024-05-26T22:56:19.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.658+0000 7f2cdea07740 1 -- 172.21.15.131:0/2281512358 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=115,monmap=2+}) -- 0x55c25bf39300 con 0x55c25bf3ce50 2024-05-26T22:56:19.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.658+0000 7f2cc2000640 1 -- 172.21.15.131:0/2281512358 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ccc07e6e0 con 0x55c25bf3ce50 2024-05-26T22:56:19.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.658+0000 7f2cc2000640 1 -- 172.21.15.131:0/2281512358 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 115) ==== 3534+0+0 (secure 0 0 0) 0x7f2cbc022020 con 0x55c25bf3ce50 2024-05-26T22:56:19.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.658+0000 7f2cdea07740 10 client.8535 fetch_fsmap finished waiting for FSMap version 115 2024-05-26T22:56:19.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.658+0000 7f2cdea07740 10 client.8535 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-26T22:56:19.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.658+0000 7f2cdea07740 20 client.8535 populate_metadata read hostname 'smithi131' 2024-05-26T22:56:19.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.658+0000 7f2cdea07740 1 --2- 172.21.15.131:0/2281512358 >> [v2:172.21.15.131:6835/2725520331,v1:172.21.15.131:6837/2725520331] conn(0x55c25bf21c30 0x55c25bf58a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:19.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.658+0000 7f2cdea07740 4 client.8535 mds_command: new command op to 8310 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-26T22:56:19.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.658+0000 7f2cdea07740 1 -- 172.21.15.131:0/2281512358 --> [v2:172.21.15.131:6835/2725520331,v1:172.21.15.131:6837/2725520331] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c25be64d40 con 0x55c25bf21c30 2024-05-26T22:56:19.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.658+0000 7f2cd9200640 1 --2- 172.21.15.131:0/2281512358 >> [v2:172.21.15.131:6835/2725520331,v1:172.21.15.131:6837/2725520331] conn(0x55c25bf21c30 0x55c25bf58a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:19.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.659+0000 7f2cd9200640 1 --2- 172.21.15.131:0/2281512358 >> [v2:172.21.15.131:6835/2725520331,v1:172.21.15.131:6837/2725520331] conn(0x55c25bf21c30 0x55c25bf58a40 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6728f11a4537b193 server_cookie=4850ed0350ae5ca4 in_seq=0 out_seq=0 2024-05-26T22:56:19.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.659+0000 7f2cc2000640 10 client.8535 ms_handle_connect on v2:172.21.15.131:6835/2725520331 2024-05-26T22:56:19.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.659+0000 7f2cc2000640 1 -- 172.21.15.131:0/2281512358 <== mds.0 v2:172.21.15.131:6835/2725520331 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c25be64d40 con 0x55c25bf21c30 2024-05-26T22:56:19.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.659+0000 7f2cc2000640 10 client.8535 handle_command_reply: tid=0 2024-05-26T22:56:19.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.660+0000 7f2cdea07740 1 client.8535 shutdown 2024-05-26T22:56:19.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.660+0000 7f2cdea07740 1 -- 172.21.15.131:0/2281512358 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c25bf3ce50 msgr2=0x55c25bef4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:19.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.660+0000 7f2cdea07740 1 --2- 172.21.15.131:0/2281512358 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c25bf3ce50 0x55c25bef4000 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc00e190 tx=0x7f2cbc00e650 comp rx=0 tx=0).stop 2024-05-26T22:56:19.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.660+0000 7f2cdea07740 1 -- 172.21.15.131:0/2281512358 shutdown_connections 2024-05-26T22:56:19.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.660+0000 7f2cdea07740 1 --2- 172.21.15.131:0/2281512358 >> [v2:172.21.15.131:6835/2725520331,v1:172.21.15.131:6837/2725520331] conn(0x55c25bf21c30 0x55c25bf58a40 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:19.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.660+0000 7f2cdea07740 1 --2- 172.21.15.131:0/2281512358 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c25bf0ff50 0x55c25bef4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:19.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.660+0000 7f2cdea07740 1 --2- 172.21.15.131:0/2281512358 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c25bf3ce50 0x55c25bef4000 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:19.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.660+0000 7f2cdea07740 1 --2- 172.21.15.131:0/2281512358 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55c25bf1ff70 0x55c25bf388b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:19.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.660+0000 7f2cc2000640 0 client.8535 ms_handle_reset on v2:172.21.15.131:6835/2725520331 2024-05-26T22:56:19.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.660+0000 7f2cdea07740 1 -- 172.21.15.131:0/2281512358 >> 172.21.15.131:0/2281512358 conn(0x55c25bf1f830 msgr2=0x55c25beef1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:56:19.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.660+0000 7f2cdea07740 1 -- 172.21.15.131:0/2281512358 shutdown_connections 2024-05-26T22:56:19.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.660+0000 7f2cdea07740 1 -- 172.21.15.131:0/2281512358 wait complete. 2024-05-26T22:56:19.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.660+0000 7f2cdea07740 20 client.8535 trim_cache size 0 max 16384 2024-05-26T22:56:19.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.680+0000 7f2cdea07740 1 Processor -- start 2024-05-26T22:56:19.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.680+0000 7f2cdea07740 1 -- start start 2024-05-26T22:56:19.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.681+0000 7f2cdea07740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c25bf0ff50 0x55c25bf37080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:19.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.681+0000 7f2cdea07740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55c25bf1ff70 0x55c25beef930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:19.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.681+0000 7f2cdea07740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c25bf3ce50 0x55c25beefe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:19.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.681+0000 7f2cdea07740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f2cd42005a0 con 0x55c25bf0ff50 2024-05-26T22:56:19.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.681+0000 7f2cdea07740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2cd41ff960 con 0x55c25bf3ce50 2024-05-26T22:56:19.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.681+0000 7f2cdea07740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f2cd41ffed0 con 0x55c25bf1ff70 2024-05-26T22:56:19.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.681+0000 7f2cd9200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c25bf0ff50 0x55c25bf37080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:19.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.681+0000 7f2cd9200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c25bf0ff50 0x55c25bf37080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56078/0 (socket says 172.21.15.131:56078) 2024-05-26T22:56:19.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.681+0000 7f2cd9200640 1 -- 172.21.15.131:0/148826499 learned_addr learned my addr 172.21.15.131:0/148826499 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:56:19.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.682+0000 7f2cd9200640 1 -- 172.21.15.131:0/148826499 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55c25bf1ff70 msgr2=0x55c25beef930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:19.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.682+0000 7f2cd9200640 1 --2- 172.21.15.131:0/148826499 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55c25bf1ff70 0x55c25beef930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:19.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.682+0000 7f2cd9200640 1 -- 172.21.15.131:0/148826499 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c25bf3ce50 msgr2=0x55c25beefe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:19.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.682+0000 7f2cd9200640 1 --2- 172.21.15.131:0/148826499 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c25bf3ce50 0x55c25beefe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:19.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.682+0000 7f2cd9200640 1 -- 172.21.15.131:0/148826499 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c25bf0f980 con 0x55c25bf0ff50 2024-05-26T22:56:19.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.682+0000 7f2cd9200640 1 --2- 172.21.15.131:0/148826499 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c25bf0ff50 0x55c25bf37080 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x55c25bf37050 tx=0x7f2cc4015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50901e3e0923d128 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:19.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.682+0000 7f2cc8c00640 1 -- 172.21.15.131:0/148826499 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc4014cb0 con 0x55c25bf0ff50 2024-05-26T22:56:19.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.682+0000 7f2cc8c00640 1 -- 172.21.15.131:0/148826499 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2cc4014e50 con 0x55c25bf0ff50 2024-05-26T22:56:19.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.682+0000 7f2cc8c00640 1 -- 172.21.15.131:0/148826499 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc40273e0 con 0x55c25bf0ff50 2024-05-26T22:56:19.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.683+0000 7f2cdea07740 1 -- 172.21.15.131:0/148826499 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c25bf0ff50 msgr2=0x55c25bf37080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:19.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.683+0000 7f2cdea07740 1 --2- 172.21.15.131:0/148826499 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c25bf0ff50 0x55c25bf37080 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x55c25bf37050 tx=0x7f2cc4015000 comp rx=0 tx=0).stop 2024-05-26T22:56:19.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.683+0000 7f2cdea07740 1 -- 172.21.15.131:0/148826499 shutdown_connections 2024-05-26T22:56:19.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.683+0000 7f2cdea07740 1 --2- 172.21.15.131:0/148826499 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55c25bf1ff70 0x55c25beef930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:19.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.683+0000 7f2cdea07740 1 --2- 172.21.15.131:0/148826499 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c25bf3ce50 0x55c25beefe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:19.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.683+0000 7f2cdea07740 1 --2- 172.21.15.131:0/148826499 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c25bf0ff50 0x55c25bf37080 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:19.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.683+0000 7f2cdea07740 1 -- 172.21.15.131:0/148826499 >> 172.21.15.131:0/148826499 conn(0x55c25bf1f830 msgr2=0x55c25beeb040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:56:19.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.683+0000 7f2cdea07740 1 -- 172.21.15.131:0/148826499 shutdown_connections 2024-05-26T22:56:19.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.683+0000 7f2cdea07740 1 -- 172.21.15.131:0/148826499 wait complete. 2024-05-26T22:56:19.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.684+0000 7f2cdea07740 1 -- start start 2024-05-26T22:56:19.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.684+0000 7f2cdea07740 1 Processor -- start 2024-05-26T22:56:19.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.685+0000 7f2cdea07740 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55c25bf0ff50 0x55c25beefc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:19.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.685+0000 7f2cdea07740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c25bf1ff70 0x55c25bfb7b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:19.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.685+0000 7f2cdea07740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c25bf3ce50 0x55c25bf1b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:19.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.685+0000 7f2cdea07740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55c25bf20930 con 0x55c25bf3ce50 2024-05-26T22:56:19.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.685+0000 7f2cdea07740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2cd415f880 con 0x55c25bf1ff70 2024-05-26T22:56:19.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.685+0000 7f2cdea07740 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x55c25bf215b0 con 0x55c25bf0ff50 2024-05-26T22:56:19.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.685+0000 7f2cdea07740 10 client.0 _ll_register_callbacks cb 0x7f2cbc007d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-26T22:56:19.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.685+0000 7f2cd9c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c25bf3ce50 0x55c25bf1b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:19.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.685+0000 7f2cd9c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c25bf3ce50 0x55c25bf1b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56088/0 (socket says 172.21.15.131:56088) 2024-05-26T22:56:19.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.685+0000 7f2cd9c00640 1 -- 172.21.15.131:0/2088317198 learned_addr learned my addr 172.21.15.131:0/2088317198 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:56:19.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.686+0000 7f2cd9200640 1 --2- 172.21.15.131:0/2088317198 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55c25bf0ff50 0x55c25beefc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:19.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.686+0000 7f2cd9c00640 1 -- 172.21.15.131:0/2088317198 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55c25bf0ff50 msgr2=0x55c25beefc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:19.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.686+0000 7f2cd3e00640 1 --2- 172.21.15.131:0/2088317198 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c25bf1ff70 0x55c25bfb7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:19.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.686+0000 7f2cd9c00640 1 --2- 172.21.15.131:0/2088317198 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55c25bf0ff50 0x55c25beefc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:19.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.686+0000 7f2cd9c00640 1 -- 172.21.15.131:0/2088317198 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c25bf1ff70 msgr2=0x55c25bfb7b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:19.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.686+0000 7f2cd9c00640 1 --2- 172.21.15.131:0/2088317198 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c25bf1ff70 0x55c25bfb7b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:19.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.686+0000 7f2cd9c00640 1 -- 172.21.15.131:0/2088317198 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c25bf385b0 con 0x55c25bf3ce50 2024-05-26T22:56:19.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.686+0000 7f2cd9c00640 1 --2- 172.21.15.131:0/2088317198 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c25bf3ce50 0x55c25bf1b6f0 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc02a730 tx=0x7f2ccc02ad70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44cdf8a4c9c75a39 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:19.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.686+0000 7f2cc2000640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-05-26T22:56:19.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.686+0000 7f2cc2000640 1 -- 172.21.15.131:0/2088317198 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ccc082f10 con 0x55c25bf3ce50 2024-05-26T22:56:19.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.686+0000 7f2cc2000640 1 -- 172.21.15.131:0/2088317198 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 5709+0+0 (secure 0 0 0) 0x7f2ccc093c40 con 0x55c25bf3ce50 2024-05-26T22:56:19.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.686+0000 7f2cdea07740 1 -- 172.21.15.131:0/2088317198 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2cc4014e50 con 0x55c25bf3ce50 2024-05-26T22:56:19.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.686+0000 7f2cc2000640 1 -- 172.21.15.131:0/2088317198 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=115) ==== 24+0+0 (secure 0 0 0) 0x7f2ccc07e6e0 con 0x55c25bf3ce50 2024-05-26T22:56:19.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.686+0000 7f2cdea07740 10 client.8541 fetch_fsmap learned FSMap version 115 2024-05-26T22:56:19.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.686+0000 7f2cdea07740 1 -- 172.21.15.131:0/2088317198 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=115,monmap=2+}) -- 0x55c25bf1bf20 con 0x55c25bf3ce50 2024-05-26T22:56:19.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.687+0000 7f2cc2000640 1 -- 172.21.15.131:0/2088317198 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2cc4014e50 con 0x55c25bf3ce50 2024-05-26T22:56:19.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.687+0000 7f2cc2000640 1 -- 172.21.15.131:0/2088317198 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 115) ==== 3534+0+0 (secure 0 0 0) 0x7f2ccc082a80 con 0x55c25bf3ce50 2024-05-26T22:56:19.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.687+0000 7f2cdea07740 10 client.8541 fetch_fsmap finished waiting for FSMap version 115 2024-05-26T22:56:19.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.687+0000 7f2cdea07740 10 client.8541 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-26T22:56:19.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.687+0000 7f2cdea07740 20 client.8541 populate_metadata read hostname 'smithi131' 2024-05-26T22:56:19.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.687+0000 7f2cdea07740 1 --2- 172.21.15.131:0/2088317198 >> [v2:172.21.15.131:6835/2725520331,v1:172.21.15.131:6837/2725520331] conn(0x55c25bef4720 0x55c25bfbb730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:19.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.687+0000 7f2cdea07740 4 client.8541 mds_command: new command op to 8310 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-26T22:56:19.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.687+0000 7f2cdea07740 1 -- 172.21.15.131:0/2088317198 --> [v2:172.21.15.131:6835/2725520331,v1:172.21.15.131:6837/2725520331] -- command(tid 0: {"prefix": "status"}) -- 0x55c25bf39d30 con 0x55c25bef4720 2024-05-26T22:56:19.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.687+0000 7f2cd9200640 1 --2- 172.21.15.131:0/2088317198 >> [v2:172.21.15.131:6835/2725520331,v1:172.21.15.131:6837/2725520331] conn(0x55c25bef4720 0x55c25bfbb730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:19.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.688+0000 7f2cd9200640 1 --2- 172.21.15.131:0/2088317198 >> [v2:172.21.15.131:6835/2725520331,v1:172.21.15.131:6837/2725520331] conn(0x55c25bef4720 0x55c25bfbb730 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b1c55c216151840 server_cookie=b10fbc55bbef35ac in_seq=0 out_seq=0 2024-05-26T22:56:19.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.688+0000 7f2cc2000640 10 client.8541 ms_handle_connect on v2:172.21.15.131:6835/2725520331 2024-05-26T22:56:19.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.688+0000 7f2cc2000640 1 -- 172.21.15.131:0/2088317198 <== mds.0 v2:172.21.15.131:6835/2725520331 1 ==== command_reply(tid 0: 0 ) ==== 8+0+328 (crc 0 0 0) 0x55c25bf39d30 con 0x55c25bef4720 2024-05-26T22:56:19.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.688+0000 7f2cc2000640 10 client.8541 handle_command_reply: tid=0 2024-05-26T22:56:19.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.688+0000 7f2cdea07740 1 client.8541 shutdown 2024-05-26T22:56:19.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.688+0000 7f2cdea07740 1 -- 172.21.15.131:0/2088317198 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c25bf3ce50 msgr2=0x55c25bf1b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:19.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.688+0000 7f2cdea07740 1 --2- 172.21.15.131:0/2088317198 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c25bf3ce50 0x55c25bf1b6f0 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc02a730 tx=0x7f2ccc02ad70 comp rx=0 tx=0).stop 2024-05-26T22:56:19.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.688+0000 7f2cdea07740 1 -- 172.21.15.131:0/2088317198 shutdown_connections 2024-05-26T22:56:19.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.688+0000 7f2cdea07740 1 --2- 172.21.15.131:0/2088317198 >> [v2:172.21.15.131:6835/2725520331,v1:172.21.15.131:6837/2725520331] conn(0x55c25bef4720 0x55c25bfbb730 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:19.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.688+0000 7f2cdea07740 1 --2- 172.21.15.131:0/2088317198 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c25bf3ce50 0x55c25bf1b6f0 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:19.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.688+0000 7f2cdea07740 1 --2- 172.21.15.131:0/2088317198 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c25bf1ff70 0x55c25bfb7b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:19.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.688+0000 7f2cdea07740 1 --2- 172.21.15.131:0/2088317198 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55c25bf0ff50 0x55c25beefc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:19.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.688+0000 7f2cc2000640 0 client.8541 ms_handle_reset on v2:172.21.15.131:6835/2725520331 2024-05-26T22:56:19.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.688+0000 7f2cdea07740 1 -- 172.21.15.131:0/2088317198 >> 172.21.15.131:0/2088317198 conn(0x55c25bf1f830 msgr2=0x55c25bef0600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:56:19.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.688+0000 7f2cdea07740 1 -- 172.21.15.131:0/2088317198 shutdown_connections 2024-05-26T22:56:19.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.688+0000 7f2cdea07740 1 -- 172.21.15.131:0/2088317198 wait complete. 2024-05-26T22:56:19.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.688+0000 7f2cdea07740 20 client.8541 trim_cache size 0 max 16384 2024-05-26T22:56:19.693 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-26T22:56:19.693 INFO:teuthology.orchestra.run.smithi131.stdout: "cluster_fsid": "46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f", 2024-05-26T22:56:19.693 INFO:teuthology.orchestra.run.smithi131.stdout: "whoami": 0, 2024-05-26T22:56:19.693 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 8310, 2024-05-26T22:56:19.694 INFO:teuthology.orchestra.run.smithi131.stdout: "want_state": "up:active", 2024-05-26T22:56:19.694 INFO:teuthology.orchestra.run.smithi131.stdout: "state": "up:active", 2024-05-26T22:56:19.694 INFO:teuthology.orchestra.run.smithi131.stdout: "fs_name": "cephfs", 2024-05-26T22:56:19.694 INFO:teuthology.orchestra.run.smithi131.stdout: "rank_uptime": 17.170887376, 2024-05-26T22:56:19.694 INFO:teuthology.orchestra.run.smithi131.stdout: "mdsmap_epoch": 115, 2024-05-26T22:56:19.694 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch": 107, 2024-05-26T22:56:19.694 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch_barrier": 107, 2024-05-26T22:56:19.694 INFO:teuthology.orchestra.run.smithi131.stdout: "uptime": 29.636621823999999 2024-05-26T22:56:19.694 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-26T22:56:19.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.689+0000 7f2cda600640 1 -- 172.21.15.131:0/1370606780 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f2cb006be30 msgr2=0x7f2cb006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:19.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.689+0000 7f2cda600640 1 --2- 172.21.15.131:0/1370606780 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f2cb006be30 0x7f2cb006e2f0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f2cc4006fd0 tx=0x7f2cc4008040 comp rx=0 tx=0).stop 2024-05-26T22:56:19.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.689+0000 7f2cda600640 1 -- 172.21.15.131:0/1370606780 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2cd415e320 msgr2=0x7f2cd414b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:19.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.689+0000 7f2cda600640 1 --2- 172.21.15.131:0/1370606780 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2cd415e320 0x7f2cd414b530 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc02ace0 tx=0x7f2ccc004a40 comp rx=0 tx=0).stop 2024-05-26T22:56:19.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.689+0000 7f2cda600640 1 -- 172.21.15.131:0/1370606780 shutdown_connections 2024-05-26T22:56:19.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.689+0000 7f2cda600640 1 --2- 172.21.15.131:0/1370606780 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2cd40d0550 0x7f2cd4144d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:19.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.689+0000 7f2cda600640 1 --2- 172.21.15.131:0/1370606780 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f2cb006be30 0x7f2cb006e2f0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:19.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.689+0000 7f2cda600640 1 --2- 172.21.15.131:0/1370606780 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2cd415e320 0x7f2cd414b530 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:19.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.689+0000 7f2cda600640 1 --2- 172.21.15.131:0/1370606780 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2cd40b2f60 0x7f2cd41474e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:19.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.689+0000 7f2cda600640 1 -- 172.21.15.131:0/1370606780 >> 172.21.15.131:0/1370606780 conn(0x7f2cd40b5660 msgr2=0x7f2cd40b4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:56:19.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.689+0000 7f2cda600640 1 -- 172.21.15.131:0/1370606780 shutdown_connections 2024-05-26T22:56:19.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19.689+0000 7f2cda600640 1 -- 172.21.15.131:0/1370606780 wait complete. 2024-05-26T22:56:19.706 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c ' 2024-05-26T22:56:19.706 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-26T22:56:19.706 DEBUG:teuthology.orchestra.run.smithi131:> mkdir -p tree 2024-05-26T22:56:19.706 DEBUG:teuthology.orchestra.run.smithi131:> 2024-05-26T22:56:19.706 DEBUG:teuthology.orchestra.run.smithi131:> setfattr -n ceph.dir.pin.distributed -v 1 tree 2024-05-26T22:56:19.706 DEBUG:teuthology.orchestra.run.smithi131:> 2024-05-26T22:56:19.706 DEBUG:teuthology.orchestra.run.smithi131:> for ((i = 0; i < 100; i++)); do 2024-05-26T22:56:19.706 DEBUG:teuthology.orchestra.run.smithi131:> mkdir -p "tree/$i" 2024-05-26T22:56:19.706 DEBUG:teuthology.orchestra.run.smithi131:> echo file > "tree/$i/file" 2024-05-26T22:56:19.706 DEBUG:teuthology.orchestra.run.smithi131:> done 2024-05-26T22:56:19.706 DEBUG:teuthology.orchestra.run.smithi131:> ') 2024-05-26T22:56:19.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:19 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-26T22:56:19.808 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree 2024-05-26T22:56:19.813 INFO:teuthology.orchestra.run.smithi131.stderr:+ setfattr -n ceph.dir.pin.distributed -v 1 tree 2024-05-26T22:56:19.820 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i = 0 )) 2024-05-26T22:56:19.820 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:56:19.820 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/0 2024-05-26T22:56:19.823 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:56:19.830 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:56:19.830 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:56:19.830 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/1 2024-05-26T22:56:19.833 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:56:19.835 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:56:19.835 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:56:19.835 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/2 2024-05-26T22:56:19.838 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:56:19.839 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:56:19.839 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:56:19.839 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/3 2024-05-26T22:56:19.843 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:56:19.844 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:56:19.844 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:56:19.844 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/4 2024-05-26T22:56:19.847 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:56:19.848 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:56:19.848 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:56:19.848 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/5 2024-05-26T22:56:19.851 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:56:19.852 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:56:19.852 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:56:19.852 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/6 2024-05-26T22:56:19.856 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:56:19.856 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:56:19.857 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:56:19.857 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/7 2024-05-26T22:56:19.860 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:56:19.861 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:56:19.862 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:56:19.862 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/8 2024-05-26T22:56:19.867 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:56:19.870 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:56:19.870 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:56:19.870 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/9 2024-05-26T22:56:19.875 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:56:19.878 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:56:19.879 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:56:19.879 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/10 2024-05-26T22:56:19.886 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:56:19.887 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:56:19.887 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:56:19.887 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/11 2024-05-26T22:56:19.893 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:56:19.896 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:56:19.896 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:56:19.896 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/12 2024-05-26T22:56:19.902 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:56:19.905 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:56:19.905 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:56:19.905 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/13 2024-05-26T22:56:19.910 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:56:19.913 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:56:19.913 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:56:19.913 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/14 2024-05-26T22:56:19.919 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:56:19.922 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:56:19.922 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:56:19.922 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/15 2024-05-26T22:56:19.927 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:56:19.931 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:56:19.931 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:56:19.931 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/16 2024-05-26T22:56:19.937 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:56:19.941 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:56:19.941 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:56:19.941 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/17 2024-05-26T22:56:19.947 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:56:19.950 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:56:19.951 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:56:19.951 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/18 2024-05-26T22:56:19.957 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:56:19.960 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:56:19.960 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:56:19.960 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/19 2024-05-26T22:56:19.966 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:56:19.970 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:56:19.970 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:56:19.970 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/20 2024-05-26T22:56:19.976 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:56:19.980 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:56:19.980 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:56:19.980 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/21 2024-05-26T22:56:19.988 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:56:19.992 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:56:19.992 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:56:19.992 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/22 2024-05-26T22:56:19.999 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:56:20.002 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:56:20.002 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:56:20.002 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/23 2024-05-26T22:56:20.009 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:56:20.012 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:56:20.012 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:56:20.012 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/24 2024-05-26T22:56:20.020 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:56:20.023 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:56:20.023 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:56:20.023 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/25 2024-05-26T22:56:20.030 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:56:20.034 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:56:20.034 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:56:20.034 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/26 2024-05-26T22:56:20.041 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:56:20.045 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:56:20.045 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:56:20.045 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/27 2024-05-26T22:56:20.052 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:56:20.055 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:56:20.055 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:56:20.055 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/28 2024-05-26T22:56:20.065 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:56:20.069 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:56:20.069 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:56:20.069 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/29 2024-05-26T22:56:20.077 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:56:20.080 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:56:20.080 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:56:20.080 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/30 2024-05-26T22:56:20.086 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:56:20.090 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:56:20.090 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:56:20.090 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/31 2024-05-26T22:56:20.093 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:56:20.095 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:56:20.095 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:56:20.095 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/32 2024-05-26T22:56:20.102 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:56:20.103 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:56:20.103 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:56:20.103 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/33 2024-05-26T22:56:20.110 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:56:20.113 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:56:20.113 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:56:20.113 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/34 2024-05-26T22:56:20.120 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:56:20.123 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:56:20.123 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:56:20.123 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/35 2024-05-26T22:56:20.129 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:56:20.132 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:56:20.132 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:56:20.133 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/36 2024-05-26T22:56:20.139 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:56:20.143 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:56:20.143 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:56:20.143 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/37 2024-05-26T22:56:20.150 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:56:20.153 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:56:20.154 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:56:20.154 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/38 2024-05-26T22:56:20.160 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:56:20.164 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:56:20.164 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:56:20.164 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/39 2024-05-26T22:56:20.170 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:56:20.174 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:56:20.174 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:56:20.174 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/40 2024-05-26T22:56:20.181 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:56:20.184 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:56:20.184 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:56:20.185 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/41 2024-05-26T22:56:20.191 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:56:20.195 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:56:20.195 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:56:20.195 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/42 2024-05-26T22:56:20.202 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:56:20.206 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:56:20.206 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:56:20.206 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/43 2024-05-26T22:56:20.211 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:56:20.213 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:56:20.213 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:56:20.213 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/44 2024-05-26T22:56:20.219 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:56:20.221 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:56:20.221 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:56:20.221 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/45 2024-05-26T22:56:20.227 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:56:20.228 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:56:20.228 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:56:20.228 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/46 2024-05-26T22:56:20.234 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:56:20.235 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:56:20.235 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:56:20.235 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/47 2024-05-26T22:56:20.242 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:56:20.244 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:56:20.245 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:56:20.245 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/48 2024-05-26T22:56:20.250 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:56:20.253 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:56:20.253 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:56:20.253 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/49 2024-05-26T22:56:20.259 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:56:20.262 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:56:20.262 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:56:20.262 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/50 2024-05-26T22:56:20.269 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:56:20.271 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:56:20.271 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:56:20.271 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/51 2024-05-26T22:56:20.279 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:56:20.281 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:56:20.282 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:56:20.282 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/52 2024-05-26T22:56:20.288 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:56:20.290 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:56:20.290 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:56:20.290 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/53 2024-05-26T22:56:20.296 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:56:20.298 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:56:20.298 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:56:20.298 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/54 2024-05-26T22:56:20.304 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:56:20.306 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:56:20.307 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:56:20.307 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/55 2024-05-26T22:56:20.311 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:56:20.313 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:56:20.314 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:56:20.314 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/56 2024-05-26T22:56:20.318 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:56:20.320 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:56:20.320 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:56:20.320 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/57 2024-05-26T22:56:20.325 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:56:20.327 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:56:20.327 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:56:20.328 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/58 2024-05-26T22:56:20.332 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:56:20.334 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:56:20.334 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:56:20.335 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/59 2024-05-26T22:56:20.340 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:56:20.341 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:56:20.342 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:56:20.342 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/60 2024-05-26T22:56:20.348 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:56:20.349 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:56:20.349 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:56:20.350 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/61 2024-05-26T22:56:20.355 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:56:20.356 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:56:20.357 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:56:20.357 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/62 2024-05-26T22:56:20.361 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:56:20.364 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:56:20.364 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:56:20.364 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/63 2024-05-26T22:56:20.370 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:56:20.373 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:56:20.373 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:56:20.373 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/64 2024-05-26T22:56:20.380 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:56:20.383 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:56:20.383 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:56:20.384 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/65 2024-05-26T22:56:20.389 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:56:20.393 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:56:20.393 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:56:20.393 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/66 2024-05-26T22:56:20.399 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:56:20.402 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:56:20.403 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:56:20.403 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/67 2024-05-26T22:56:20.408 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:56:20.411 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:56:20.412 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:56:20.412 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/68 2024-05-26T22:56:20.418 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:56:20.421 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:56:20.421 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:56:20.421 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/69 2024-05-26T22:56:20.429 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:56:20.431 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:56:20.431 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:56:20.431 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/70 2024-05-26T22:56:20.437 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:56:20.440 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:56:20.440 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:56:20.440 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/71 2024-05-26T22:56:20.446 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:56:20.449 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:56:20.449 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:56:20.449 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/72 2024-05-26T22:56:20.455 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:56:20.457 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:56:20.457 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:56:20.457 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/73 2024-05-26T22:56:20.463 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:56:20.466 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:56:20.466 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:56:20.466 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/74 2024-05-26T22:56:20.472 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:56:20.475 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:56:20.475 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:56:20.475 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/75 2024-05-26T22:56:20.481 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:56:20.483 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:56:20.483 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:56:20.484 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/76 2024-05-26T22:56:20.489 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:56:20.492 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:56:20.492 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:56:20.492 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/77 2024-05-26T22:56:20.498 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:56:20.502 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:56:20.503 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:56:20.503 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/78 2024-05-26T22:56:20.507 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:56:20.510 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:56:20.510 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:56:20.510 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/79 2024-05-26T22:56:20.515 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:56:20.518 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:56:20.518 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:56:20.518 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/80 2024-05-26T22:56:20.525 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:56:20.528 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:56:20.529 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:56:20.529 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/81 2024-05-26T22:56:20.535 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:56:20.538 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:56:20.538 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:56:20.538 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/82 2024-05-26T22:56:20.544 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:56:20.546 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:56:20.547 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:56:20.547 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/83 2024-05-26T22:56:20.553 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:56:20.556 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:56:20.556 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:56:20.556 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/84 2024-05-26T22:56:20.562 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:56:20.564 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:56:20.565 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:56:20.565 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/85 2024-05-26T22:56:20.571 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:56:20.573 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:56:20.574 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:56:20.574 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/86 2024-05-26T22:56:20.579 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:56:20.581 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:56:20.581 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:56:20.581 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/87 2024-05-26T22:56:20.587 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:56:20.590 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:56:20.590 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:56:20.590 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/88 2024-05-26T22:56:20.596 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:56:20.601 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:56:20.601 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:56:20.602 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/89 2024-05-26T22:56:20.610 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:56:20.612 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:56:20.612 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:56:20.612 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/90 2024-05-26T22:56:20.619 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:56:20.622 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:56:20.622 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:56:20.622 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/91 2024-05-26T22:56:20.628 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:56:20.630 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:56:20.630 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:56:20.630 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/92 2024-05-26T22:56:20.636 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:56:20.638 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:56:20.638 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:56:20.638 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/93 2024-05-26T22:56:20.644 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:56:20.650 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:56:20.650 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:56:20.650 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/94 2024-05-26T22:56:20.652 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:56:20.655 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:56:20.655 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:56:20.655 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/95 2024-05-26T22:56:20.660 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:56:20.663 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:56:20.663 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:56:20.663 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/96 2024-05-26T22:56:20.669 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:56:20.671 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:56:20.671 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:56:20.671 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/97 2024-05-26T22:56:20.677 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:56:20.679 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:56:20.679 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:56:20.679 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/98 2024-05-26T22:56:20.685 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:56:20.688 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:56:20.688 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:56:20.688 INFO:teuthology.orchestra.run.smithi131.stderr:+ mkdir -p tree/99 2024-05-26T22:56:20.693 INFO:teuthology.orchestra.run.smithi131.stderr:+ echo file 2024-05-26T22:56:20.696 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i++ )) 2024-05-26T22:56:20.696 INFO:teuthology.orchestra.run.smithi131.stderr:+ (( i < 100 )) 2024-05-26T22:56:20.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-26T22:56:20.708 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json get subtrees 2024-05-26T22:56:20.937 INFO:teuthology.orchestra.run.smithi187.stdout:[{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds2","dirfrag":"0x102","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"2","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-05-26T22:56:20.938 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "2", "dirfrag": "0x102", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds2", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-05-26T22:56:20.938 DEBUG:tasks.cephfs.cephfs_test_case:[ { "is_auth": true, "auth_first": 2, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "~mds2", "dirfrag": "0x102", "snapid_first": 2, "projected_version": "1", "version": "1", "committing_version": "1", "committed_version": "1", "is_rep": false, "dir_auth": "2", "states": [ "auth", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "dirty": 0, "authpin": 0 }, "nref": 2 } } ] 2024-05-26T22:56:20.938 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-05-26T22:56:21.165 INFO:teuthology.orchestra.run.smithi131.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-05-26T22:56:21.166 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-05-26T22:56:21.166 DEBUG:tasks.cephfs.cephfs_test_case:[ { "is_auth": true, "auth_first": 1, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "~mds1", "dirfrag": "0x101", "snapid_first": 2, "projected_version": "1", "version": "1", "committing_version": "1", "committed_version": "1", "is_rep": false, "dir_auth": "1", "states": [ "auth", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "dirty": 0, "authpin": 0 }, "nref": 2 } } ] 2024-05-26T22:56:21.166 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-05-26T22:56:21.390 INFO:teuthology.orchestra.run.smithi131.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"2296","version":"2152","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":72,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1264","version":"1264","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":1,"authpin":0},"nref":3}}] 2024-05-26T22:56:21.390 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 72, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "2296", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "2152" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 1, "subtree": 1 }, "projected_version": "1264", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "1264" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-05-26T22:56:21.390 DEBUG:tasks.cephfs.cephfs_test_case:[ { "is_auth": true, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "", "dirfrag": "0x1", "snapid_first": 2, "projected_version": "2296", "version": "2152", "committing_version": "1", "committed_version": "1", "is_rep": true, "dir_auth": "0", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 72, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "replicated": 0, "dirty": 1, "authpin": 1 }, "nref": 4 } }, { "is_auth": true, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "~mds0", "dirfrag": "0x100", "snapid_first": 2, "projected_version": "1264", "version": "1264", "committing_version": "1", "committed_version": "1", "is_rep": false, "dir_auth": "0", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "dirty": 1, "authpin": 0 }, "nref": 3 } } ] 2024-05-26T22:56:21.391 INFO:tasks.cephfs.cephfs_test_case:len=0 [] 2024-05-26T22:56:26.392 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json get subtrees 2024-05-26T22:56:26.620 INFO:teuthology.orchestra.run.smithi187.stdout:[{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds2","dirfrag":"0x102","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"2","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-05-26T22:56:26.620 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "2", "dirfrag": "0x102", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds2", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-05-26T22:56:26.620 DEBUG:tasks.cephfs.cephfs_test_case:[ { "is_auth": true, "auth_first": 2, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "~mds2", "dirfrag": "0x102", "snapid_first": 2, "projected_version": "1", "version": "1", "committing_version": "1", "committed_version": "1", "is_rep": false, "dir_auth": "2", "states": [ "auth", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "dirty": 0, "authpin": 0 }, "nref": 2 } } ] 2024-05-26T22:56:26.621 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-05-26T22:56:26.842 INFO:teuthology.orchestra.run.smithi131.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-05-26T22:56:26.843 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-05-26T22:56:26.843 DEBUG:tasks.cephfs.cephfs_test_case:[ { "is_auth": true, "auth_first": 1, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "~mds1", "dirfrag": "0x101", "snapid_first": 2, "projected_version": "1", "version": "1", "committing_version": "1", "committed_version": "1", "is_rep": false, "dir_auth": "1", "states": [ "auth", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "dirty": 0, "authpin": 0 }, "nref": 2 } } ] 2024-05-26T22:56:26.843 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-05-26T22:56:27.065 INFO:teuthology.orchestra.run.smithi131.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"2302","version":"2302","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1264","version":"1264","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":1,"authpin":0},"nref":3}}] 2024-05-26T22:56:27.065 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "2302", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "2302" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 1, "subtree": 1 }, "projected_version": "1264", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "1264" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-05-26T22:56:27.065 DEBUG:tasks.cephfs.cephfs_test_case:[ { "is_auth": true, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "", "dirfrag": "0x1", "snapid_first": 2, "projected_version": "2302", "version": "2302", "committing_version": "1", "committed_version": "1", "is_rep": true, "dir_auth": "0", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "replicated": 0, "dirty": 1, "authpin": 0 }, "nref": 3 } }, { "is_auth": true, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "~mds0", "dirfrag": "0x100", "snapid_first": 2, "projected_version": "1264", "version": "1264", "committing_version": "1", "committed_version": "1", "is_rep": false, "dir_auth": "0", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "dirty": 1, "authpin": 0 }, "nref": 3 } } ] 2024-05-26T22:56:27.066 INFO:tasks.cephfs.cephfs_test_case:len=0 [] 2024-05-26T22:56:32.067 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json get subtrees 2024-05-26T22:56:32.287 INFO:teuthology.orchestra.run.smithi187.stdout:[{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/tree","dirfrag":"0x10000000397.111*","snapid_first":2,"projected_version":"406","version":"406","committing_version":"402","committed_version":"402","is_rep":false,"dir_auth":"2","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds2","dirfrag":"0x102","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"2","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"2303","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":3},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/tree","dirfrag":"0x10000000397.100*","snapid_first":2,"projected_version":"406","version":"406","committing_version":"402","committed_version":"402","is_rep":false,"dir_auth":"2","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/tree","dirfrag":"0x10000000397.001*","snapid_first":2,"projected_version":"408","version":"408","committing_version":"402","committed_version":"402","is_rep":false,"dir_auth":"2","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2024-05-26T22:56:32.288 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "402", "committing_version": "402", "dir_auth": "2", "dirfrag": "0x10000000397.111*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/tree", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "406", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "406" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 2, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "2", "dirfrag": "0x102", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds2", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 3 }, "snapid_first": 2, "states": [], "version": "2303" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "402", "committing_version": "402", "dir_auth": "2", "dirfrag": "0x10000000397.100*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/tree", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "406", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "406" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 2, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "402", "committing_version": "402", "dir_auth": "2", "dirfrag": "0x10000000397.001*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/tree", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "408", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "408" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 2, "is_auth": true, "random_ephemeral_pin": false } ] 2024-05-26T22:56:32.289 DEBUG:tasks.cephfs.cephfs_test_case:[ { "is_auth": true, "auth_first": 2, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 2, "dir": { "path": "/tree", "dirfrag": "0x10000000397.111*", "snapid_first": 2, "projected_version": "406", "version": "406", "committing_version": "402", "committed_version": "402", "is_rep": false, "dir_auth": "2", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": { "0": 1 } }, "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "child": 1, "frozen": 0, "subtree": 1, "importing": 0, "replicated": 1, "dirty": 1 }, "nref": 4 } }, { "is_auth": true, "auth_first": 2, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "~mds2", "dirfrag": "0x102", "snapid_first": 2, "projected_version": "1", "version": "1", "committing_version": "1", "committed_version": "1", "is_rep": false, "dir_auth": "2", "states": [ "auth", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "dirty": 0, "authpin": 0 }, "nref": 2 } }, { "is_auth": false, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "", "dirfrag": "0x1", "snapid_first": 2, "projected_version": "0", "version": "2303", "committing_version": "0", "committed_version": "0", "is_rep": true, "dir_auth": "0", "states": [], "is_auth": false, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 3 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1 }, "nref": 2 } }, { "is_auth": true, "auth_first": 2, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 2, "dir": { "path": "/tree", "dirfrag": "0x10000000397.100*", "snapid_first": 2, "projected_version": "406", "version": "406", "committing_version": "402", "committed_version": "402", "is_rep": false, "dir_auth": "2", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": { "0": 1 } }, "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "child": 1, "frozen": 0, "subtree": 1, "importing": 0, "replicated": 1, "dirty": 1 }, "nref": 4 } }, { "is_auth": true, "auth_first": 2, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 2, "dir": { "path": "/tree", "dirfrag": "0x10000000397.001*", "snapid_first": 2, "projected_version": "408", "version": "408", "committing_version": "402", "committed_version": "402", "is_rep": false, "dir_auth": "2", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": { "0": 1 } }, "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "child": 1, "frozen": 0, "subtree": 1, "importing": 0, "replicated": 1, "dirty": 1 }, "nref": 4 } } ] 2024-05-26T22:56:32.289 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-05-26T22:56:32.518 INFO:teuthology.orchestra.run.smithi131.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree","dirfrag":"0x10000000397.101*","snapid_first":2,"projected_version":"402","version":"402","committing_version":"402","committed_version":"402","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1},"nref":3}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"2303","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":3},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree","dirfrag":"0x10000000397.011*","snapid_first":2,"projected_version":"406","version":"406","committing_version":"402","committed_version":"402","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree","dirfrag":"0x10000000397.000*","snapid_first":2,"projected_version":"404","version":"404","committing_version":"402","committed_version":"402","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-05-26T22:56:32.519 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "402", "committing_version": "402", "dir_auth": "1", "dirfrag": "0x10000000397.101*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "/tree", "pins": { "child": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "402", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "402" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 3 }, "snapid_first": 2, "states": [], "version": "2303" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "402", "committing_version": "402", "dir_auth": "1", "dirfrag": "0x10000000397.011*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/tree", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "406", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "406" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "402", "committing_version": "402", "dir_auth": "1", "dirfrag": "0x10000000397.000*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/tree", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "404", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "404" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-05-26T22:56:32.519 DEBUG:tasks.cephfs.cephfs_test_case:[ { "is_auth": true, "auth_first": 1, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 1, "dir": { "path": "/tree", "dirfrag": "0x10000000397.101*", "snapid_first": 2, "projected_version": "402", "version": "402", "committing_version": "402", "committed_version": "402", "is_rep": false, "dir_auth": "1", "states": [ "auth", "complete" ], "is_auth": true, "auth_state": { "replicas": { "0": 1 } }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "child": 1, "frozen": 0, "subtree": 1, "importing": 0, "replicated": 1 }, "nref": 3 } }, { "is_auth": false, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "", "dirfrag": "0x1", "snapid_first": 2, "projected_version": "0", "version": "2303", "committing_version": "0", "committed_version": "0", "is_rep": true, "dir_auth": "0", "states": [], "is_auth": false, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 3 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1 }, "nref": 2 } }, { "is_auth": true, "auth_first": 1, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 1, "dir": { "path": "/tree", "dirfrag": "0x10000000397.011*", "snapid_first": 2, "projected_version": "406", "version": "406", "committing_version": "402", "committed_version": "402", "is_rep": false, "dir_auth": "1", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": { "0": 1 } }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "child": 1, "frozen": 0, "subtree": 1, "importing": 0, "replicated": 1, "dirty": 1 }, "nref": 4 } }, { "is_auth": true, "auth_first": 1, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 1, "dir": { "path": "/tree", "dirfrag": "0x10000000397.000*", "snapid_first": 2, "projected_version": "404", "version": "404", "committing_version": "402", "committed_version": "402", "is_rep": false, "dir_auth": "1", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": { "0": 1 } }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "child": 1, "frozen": 0, "subtree": 1, "importing": 0, "replicated": 1, "dirty": 1 }, "nref": 4 } }, { "is_auth": true, "auth_first": 1, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "~mds1", "dirfrag": "0x101", "snapid_first": 2, "projected_version": "1", "version": "1", "committing_version": "1", "committed_version": "1", "is_rep": false, "dir_auth": "1", "states": [ "auth", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "dirty": 0, "authpin": 0 }, "nref": 2 } } ] 2024-05-26T22:56:32.520 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-05-26T22:56:32.738 INFO:teuthology.orchestra.run.smithi131.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"2303","version":"2303","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":3,"2":3}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1264","version":"1264","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":1,"authpin":0},"nref":3}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/tree","dirfrag":"0x10000000397.111*","snapid_first":2,"projected_version":"406","version":"406","committing_version":"402","committed_version":"402","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/tree","dirfrag":"0x10000000397.110*","snapid_first":2,"projected_version":"406","version":"406","committing_version":"402","committed_version":"402","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"frozen":0,"subtree":1,"dirty":1,"waiter":0,"authpin":0},"nref":3}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree","dirfrag":"0x10000000397.101*","snapid_first":2,"projected_version":"402","version":"402","committing_version":"402","committed_version":"402","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/tree","dirfrag":"0x10000000397.100*","snapid_first":2,"projected_version":"406","version":"406","committing_version":"402","committed_version":"402","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree","dirfrag":"0x10000000397.011*","snapid_first":2,"projected_version":"406","version":"406","committing_version":"402","committed_version":"402","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/tree","dirfrag":"0x10000000397.010*","snapid_first":2,"projected_version":"406","version":"406","committing_version":"402","committed_version":"402","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"frozen":0,"subtree":1,"dirty":1,"waiter":0,"authpin":0},"nref":3}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/tree","dirfrag":"0x10000000397.001*","snapid_first":2,"projected_version":"408","version":"408","committing_version":"402","committed_version":"402","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":true,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/tree","dirfrag":"0x10000000397.000*","snapid_first":2,"projected_version":"404","version":"404","committing_version":"402","committed_version":"402","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}}] 2024-05-26T22:56:32.739 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 3, "2": 3 } }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "2303", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "2303" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 1, "subtree": 1 }, "projected_version": "1264", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "1264" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "402", "committing_version": "402", "dir_auth": "2", "dirfrag": "0x10000000397.111*", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "406", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "406" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 2, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "402", "committing_version": "402", "dir_auth": "0", "dirfrag": "0x10000000397.110*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "/tree", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "subtree": 1, "waiter": 0 }, "projected_version": "406", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "406" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "402", "committing_version": "402", "dir_auth": "1", "dirfrag": "0x10000000397.101*", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "402", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "402" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "402", "committing_version": "402", "dir_auth": "2", "dirfrag": "0x10000000397.100*", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "406", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "406" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 2, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "402", "committing_version": "402", "dir_auth": "1", "dirfrag": "0x10000000397.011*", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "406", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "406" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "402", "committing_version": "402", "dir_auth": "0", "dirfrag": "0x10000000397.010*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "/tree", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "subtree": 1, "waiter": 0 }, "projected_version": "406", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "406" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "402", "committing_version": "402", "dir_auth": "2", "dirfrag": "0x10000000397.001*", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "408", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "408" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 2, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "402", "committing_version": "402", "dir_auth": "1", "dirfrag": "0x10000000397.000*", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "404", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "404" }, "distributed_ephemeral_pin": true, "export_pin": -1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false } ] 2024-05-26T22:56:32.740 DEBUG:tasks.cephfs.cephfs_test_case:[ { "is_auth": true, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "", "dirfrag": "0x1", "snapid_first": 2, "projected_version": "2303", "version": "2303", "committing_version": "1", "committed_version": "1", "is_rep": true, "dir_auth": "0", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": { "1": 3, "2": 3 } }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "replicated": 1, "dirty": 1, "authpin": 0 }, "nref": 4 } }, { "is_auth": true, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": false, "random_ephemeral_pin": false, "export_pin_target": -1, "dir": { "path": "~mds0", "dirfrag": "0x100", "snapid_first": 2, "projected_version": "1264", "version": "1264", "committing_version": "1", "committed_version": "1", "is_rep": false, "dir_auth": "0", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "subtree": 1, "dirty": 1, "authpin": 0 }, "nref": 3 } }, { "is_auth": false, "auth_first": 2, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 2, "dir": { "path": "/tree", "dirfrag": "0x10000000397.111*", "snapid_first": 2, "projected_version": "406", "version": "406", "committing_version": "402", "committed_version": "402", "is_rep": false, "dir_auth": "2", "states": [], "is_auth": false, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "request": 0, "child": 0, "frozen": 0, "subtree": 1, "replicated": 0, "dirty": 0, "waiter": 0, "authpin": 0, "tempexporting": 0 }, "nref": 1 } }, { "is_auth": true, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 0, "dir": { "path": "/tree", "dirfrag": "0x10000000397.110*", "snapid_first": 2, "projected_version": "406", "version": "406", "committing_version": "402", "committed_version": "402", "is_rep": false, "dir_auth": "0", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "frozen": 0, "subtree": 1, "dirty": 1, "waiter": 0, "authpin": 0 }, "nref": 3 } }, { "is_auth": false, "auth_first": 1, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 1, "dir": { "path": "/tree", "dirfrag": "0x10000000397.101*", "snapid_first": 2, "projected_version": "402", "version": "402", "committing_version": "402", "committed_version": "402", "is_rep": false, "dir_auth": "1", "states": [], "is_auth": false, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "request": 0, "child": 0, "frozen": 0, "subtree": 1, "replicated": 0, "dirty": 0, "waiter": 0, "authpin": 0, "tempexporting": 0 }, "nref": 1 } }, { "is_auth": false, "auth_first": 2, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 2, "dir": { "path": "/tree", "dirfrag": "0x10000000397.100*", "snapid_first": 2, "projected_version": "406", "version": "406", "committing_version": "402", "committed_version": "402", "is_rep": false, "dir_auth": "2", "states": [], "is_auth": false, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "request": 0, "child": 0, "frozen": 0, "subtree": 1, "replicated": 0, "dirty": 0, "waiter": 0, "authpin": 0, "tempexporting": 0 }, "nref": 1 } }, { "is_auth": false, "auth_first": 1, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 1, "dir": { "path": "/tree", "dirfrag": "0x10000000397.011*", "snapid_first": 2, "projected_version": "406", "version": "406", "committing_version": "402", "committed_version": "402", "is_rep": false, "dir_auth": "1", "states": [], "is_auth": false, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "request": 0, "child": 0, "frozen": 0, "subtree": 1, "replicated": 0, "dirty": 0, "waiter": 0, "authpin": 0, "tempexporting": 0 }, "nref": 1 } }, { "is_auth": true, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 0, "dir": { "path": "/tree", "dirfrag": "0x10000000397.010*", "snapid_first": 2, "projected_version": "406", "version": "406", "committing_version": "402", "committed_version": "402", "is_rep": false, "dir_auth": "0", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "frozen": 0, "subtree": 1, "dirty": 1, "waiter": 0, "authpin": 0 }, "nref": 3 } }, { "is_auth": false, "auth_first": 2, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 2, "dir": { "path": "/tree", "dirfrag": "0x10000000397.001*", "snapid_first": 2, "projected_version": "408", "version": "408", "committing_version": "402", "committed_version": "402", "is_rep": false, "dir_auth": "2", "states": [], "is_auth": false, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "request": 0, "child": 0, "frozen": 0, "subtree": 1, "replicated": 0, "dirty": 0, "waiter": 0, "authpin": 0, "tempexporting": 0 }, "nref": 1 } }, { "is_auth": false, "auth_first": 1, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 1, "dir": { "path": "/tree", "dirfrag": "0x10000000397.000*", "snapid_first": 2, "projected_version": "404", "version": "404", "committing_version": "402", "committed_version": "402", "is_rep": false, "dir_auth": "1", "states": [], "is_auth": false, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "request": 0, "child": 0, "frozen": 0, "subtree": 1, "replicated": 0, "dirty": 0, "waiter": 0, "authpin": 0, "tempexporting": 0 }, "nref": 1 } } ] 2024-05-26T22:56:32.741 INFO:tasks.cephfs.cephfs_test_case:len=8 [ { "is_auth": true, "auth_first": 2, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 2, "dir": { "path": "/tree", "dirfrag": "0x10000000397.111*", "snapid_first": 2, "projected_version": "406", "version": "406", "committing_version": "402", "committed_version": "402", "is_rep": false, "dir_auth": "2", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": { "0": 1 } }, "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "child": 1, "frozen": 0, "subtree": 1, "importing": 0, "replicated": 1, "dirty": 1 }, "nref": 4 } }, { "is_auth": true, "auth_first": 2, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 2, "dir": { "path": "/tree", "dirfrag": "0x10000000397.100*", "snapid_first": 2, "projected_version": "406", "version": "406", "committing_version": "402", "committed_version": "402", "is_rep": false, "dir_auth": "2", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": { "0": 1 } }, "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "child": 1, "frozen": 0, "subtree": 1, "importing": 0, "replicated": 1, "dirty": 1 }, "nref": 4 } }, { "is_auth": true, "auth_first": 2, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 2, "dir": { "path": "/tree", "dirfrag": "0x10000000397.001*", "snapid_first": 2, "projected_version": "408", "version": "408", "committing_version": "402", "committed_version": "402", "is_rep": false, "dir_auth": "2", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": { "0": 1 } }, "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "child": 1, "frozen": 0, "subtree": 1, "importing": 0, "replicated": 1, "dirty": 1 }, "nref": 4 } }, { "is_auth": true, "auth_first": 1, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 1, "dir": { "path": "/tree", "dirfrag": "0x10000000397.101*", "snapid_first": 2, "projected_version": "402", "version": "402", "committing_version": "402", "committed_version": "402", "is_rep": false, "dir_auth": "1", "states": [ "auth", "complete" ], "is_auth": true, "auth_state": { "replicas": { "0": 1 } }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "child": 1, "frozen": 0, "subtree": 1, "importing": 0, "replicated": 1 }, "nref": 3 } }, { "is_auth": true, "auth_first": 1, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 1, "dir": { "path": "/tree", "dirfrag": "0x10000000397.011*", "snapid_first": 2, "projected_version": "406", "version": "406", "committing_version": "402", "committed_version": "402", "is_rep": false, "dir_auth": "1", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": { "0": 1 } }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "child": 1, "frozen": 0, "subtree": 1, "importing": 0, "replicated": 1, "dirty": 1 }, "nref": 4 } }, { "is_auth": true, "auth_first": 1, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 1, "dir": { "path": "/tree", "dirfrag": "0x10000000397.000*", "snapid_first": 2, "projected_version": "404", "version": "404", "committing_version": "402", "committed_version": "402", "is_rep": false, "dir_auth": "1", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": { "0": 1 } }, "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "ptrwaiter": 0, "child": 1, "frozen": 0, "subtree": 1, "importing": 0, "replicated": 1, "dirty": 1 }, "nref": 4 } }, { "is_auth": true, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 0, "dir": { "path": "/tree", "dirfrag": "0x10000000397.110*", "snapid_first": 2, "projected_version": "406", "version": "406", "committing_version": "402", "committed_version": "402", "is_rep": false, "dir_auth": "0", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "frozen": 0, "subtree": 1, "dirty": 1, "waiter": 0, "authpin": 0 }, "nref": 3 } }, { "is_auth": true, "auth_first": 0, "auth_second": -2, "export_pin": -1, "distributed_ephemeral_pin": true, "random_ephemeral_pin": false, "export_pin_target": 0, "dir": { "path": "/tree", "dirfrag": "0x10000000397.010*", "snapid_first": 2, "projected_version": "406", "version": "406", "committing_version": "402", "committed_version": "402", "is_rep": false, "dir_auth": "0", "states": [ "auth", "dirty", "complete" ], "is_auth": true, "auth_state": { "replicas": {} }, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "auth_pins": 0, "is_frozen": false, "is_freezing": false, "pins": { "child": 1, "frozen": 0, "subtree": 1, "dirty": 1, "waiter": 0, "authpin": 0 }, "nref": 3 } } ] 2024-05-26T22:56:32.741 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:56:32.741 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 0 tree) 2024-05-26T22:56:46.112 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-26T22:56:46.121 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-26T22:56:47.796 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-05-26T22:56:48.023 INFO:teuthology.orchestra.run.smithi131.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"2368","version":"2368","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":3,"2":3}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1264","version":"1264","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":0,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/tree","dirfrag":"0x10000000397.111*","snapid_first":2,"projected_version":"419","version":"419","committing_version":"402","committed_version":"402","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"2":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1,"waiter":0,"authpin":0,"tempexporting":0},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":0,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/tree","dirfrag":"0x10000000397.110*","snapid_first":2,"projected_version":"445","version":"445","committing_version":"402","committed_version":"402","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"frozen":0,"subtree":1,"dirty":1,"waiter":0,"authpin":0},"nref":3}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":0,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/tree","dirfrag":"0x10000000397.101*","snapid_first":2,"projected_version":"402","version":"402","committing_version":"402","committed_version":"402","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":0,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/tree","dirfrag":"0x10000000397.100*","snapid_first":2,"projected_version":"429","version":"429","committing_version":"402","committed_version":"402","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"2":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1,"waiter":0,"authpin":0,"tempexporting":0},"nref":4}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":0,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/tree","dirfrag":"0x10000000397.011*","snapid_first":2,"projected_version":"406","version":"406","committing_version":"402","committed_version":"402","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":0,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/tree","dirfrag":"0x10000000397.010*","snapid_first":2,"projected_version":"429","version":"429","committing_version":"402","committed_version":"402","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"frozen":0,"subtree":1,"dirty":1,"waiter":0,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":0,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/tree","dirfrag":"0x10000000397.001*","snapid_first":2,"projected_version":"431","version":"431","committing_version":"402","committed_version":"402","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"2":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1,"waiter":0,"authpin":0,"tempexporting":0},"nref":4}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":0,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/tree","dirfrag":"0x10000000397.000*","snapid_first":2,"projected_version":"404","version":"404","committing_version":"402","committed_version":"402","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}}] 2024-05-26T22:56:48.024 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 3, "2": 3 } }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "2368", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "2368" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 1, "subtree": 1 }, "projected_version": "1264", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "1264" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "2": 1 } }, "committed_version": "402", "committing_version": "402", "dir_auth": "0", "dirfrag": "0x10000000397.111*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/tree", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "419", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "419" }, "distributed_ephemeral_pin": false, "export_pin": 0, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "402", "committing_version": "402", "dir_auth": "0", "dirfrag": "0x10000000397.110*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "/tree", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "subtree": 1, "waiter": 0 }, "projected_version": "445", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "445" }, "distributed_ephemeral_pin": false, "export_pin": 0, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "402", "committing_version": "402", "dir_auth": "1", "dirfrag": "0x10000000397.101*", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "402", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "402" }, "distributed_ephemeral_pin": false, "export_pin": 0, "export_pin_target": 0, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "2": 1 } }, "committed_version": "402", "committing_version": "402", "dir_auth": "0", "dirfrag": "0x10000000397.100*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/tree", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "429", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "429" }, "distributed_ephemeral_pin": false, "export_pin": 0, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "402", "committing_version": "402", "dir_auth": "1", "dirfrag": "0x10000000397.011*", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "406", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "406" }, "distributed_ephemeral_pin": false, "export_pin": 0, "export_pin_target": 0, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "402", "committing_version": "402", "dir_auth": "0", "dirfrag": "0x10000000397.010*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "/tree", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "subtree": 1, "waiter": 0 }, "projected_version": "429", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "429" }, "distributed_ephemeral_pin": false, "export_pin": 0, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "2": 1 } }, "committed_version": "402", "committing_version": "402", "dir_auth": "0", "dirfrag": "0x10000000397.001*", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/tree", "pins": { "authpin": 0, "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "431", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "431" }, "distributed_ephemeral_pin": false, "export_pin": 0, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "402", "committing_version": "402", "dir_auth": "1", "dirfrag": "0x10000000397.000*", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/tree", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "404", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "404" }, "distributed_ephemeral_pin": false, "export_pin": 0, "export_pin_target": 0, "is_auth": false, "random_ephemeral_pin": false } ] 2024-05-26T22:56:48.025 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-05-26T22:56:48.025 DEBUG:teuthology.orchestra.run.smithi131:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-26T22:56:48.132 DEBUG:teuthology.orchestra.run.smithi187:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-26T22:56:48.247 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-05-26T22:56:48.247 INFO:tasks.cephfs.mount:Terminating background process 2024-05-26T22:56:48.247 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-26T22:56:48.247 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-26T22:56:48.305 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-26T22:56:48.305 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-26T22:56:48.359 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-26T22:56:48.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:48.555+0000 7f2a17400640 1 Processor -- start 2024-05-26T22:56:48.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:48.555+0000 7f2a17400640 1 -- start start 2024-05-26T22:56:48.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:48.557+0000 7f2a17400640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f2a10158c00 0x7f2a1006d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:48.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:48.557+0000 7f2a17400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f2a1006daf0 0x7f2a100b3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:48.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:48.557+0000 7f2a17400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2a100b36d0 0x7f2a10161fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:48.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:48.557+0000 7f2a17400640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f2a1006df50 con 0x7f2a1006daf0 2024-05-26T22:56:48.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:48.557+0000 7f2a17400640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f2a100b3e50 con 0x7f2a10158c00 2024-05-26T22:56:48.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:48.557+0000 7f2a17400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f2a100b4130 con 0x7f2a100b36d0 2024-05-26T22:56:48.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:48.557+0000 7f2a0fe00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f2a1006daf0 0x7f2a100b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:48.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:48.557+0000 7f2a0fe00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f2a1006daf0 0x7f2a100b3190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41800/0 (socket says 172.21.15.131:41800) 2024-05-26T22:56:48.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:48.557+0000 7f2a0fe00640 1 -- 172.21.15.131:0/2600088916 learned_addr learned my addr 172.21.15.131:0/2600088916 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:56:48.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:48.558+0000 7f2a15400640 1 --2- 172.21.15.131:0/2600088916 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2a100b36d0 0x7f2a10161fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:48.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:48.558+0000 7f2a0fe00640 1 -- 172.21.15.131:0/2600088916 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2a100b36d0 msgr2=0x7f2a10161fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:48.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:48.558+0000 7f2a0fe00640 1 --2- 172.21.15.131:0/2600088916 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2a100b36d0 0x7f2a10161fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:48.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:48.558+0000 7f2a0fe00640 1 -- 172.21.15.131:0/2600088916 >> v2:172.21.15.187:3300/0 conn(0x7f2a10158c00 msgr2=0x7f2a1006d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:48.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:48.558+0000 7f2a0fe00640 1 --2- 172.21.15.131:0/2600088916 >> v2:172.21.15.187:3300/0 conn(0x7f2a10158c00 0x7f2a1006d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:48.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:48.558+0000 7f2a0fe00640 1 -- 172.21.15.131:0/2600088916 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a100b4410 con 0x7f2a1006daf0 2024-05-26T22:56:48.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:48.558+0000 7f2a15400640 1 --2- 172.21.15.131:0/2600088916 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2a100b36d0 0x7f2a10161fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:56:48.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:48.559+0000 7f2a0fe00640 1 --2- 172.21.15.131:0/2600088916 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a1006daf0 0x7f2a100b3190 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f2a1006c1f0 tx=0x7f29fc02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d86fa88ab38947ae server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:48.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:48.559+0000 7f2a0f400640 1 -- 172.21.15.131:0/2600088916 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29fc037070 con 0x7f2a1006daf0 2024-05-26T22:56:48.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:48.559+0000 7f2a0f400640 1 -- 172.21.15.131:0/2600088916 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29fc032440 con 0x7f2a1006daf0 2024-05-26T22:56:48.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:48.559+0000 7f2a0f400640 1 -- 172.21.15.131:0/2600088916 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29fc032810 con 0x7f2a1006daf0 2024-05-26T22:56:48.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:48.560+0000 7f2a17400640 1 -- 172.21.15.131:0/2600088916 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a1006daf0 msgr2=0x7f2a100b3190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:48.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:48.560+0000 7f2a17400640 1 --2- 172.21.15.131:0/2600088916 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a1006daf0 0x7f2a100b3190 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f2a1006c1f0 tx=0x7f29fc02a140 comp rx=0 tx=0).stop 2024-05-26T22:56:48.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:48.560+0000 7f2a17400640 1 -- 172.21.15.131:0/2600088916 shutdown_connections 2024-05-26T22:56:48.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:48.560+0000 7f2a17400640 1 --2- 172.21.15.131:0/2600088916 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2a100b36d0 0x7f2a10161fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:48.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:48.560+0000 7f2a17400640 1 --2- 172.21.15.131:0/2600088916 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a1006daf0 0x7f2a100b3190 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:48.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:48.560+0000 7f2a17400640 1 --2- 172.21.15.131:0/2600088916 >> v2:172.21.15.187:3300/0 conn(0x7f2a10158c00 0x7f2a1006d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:48.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:48.560+0000 7f2a17400640 1 -- 172.21.15.131:0/2600088916 >> 172.21.15.131:0/2600088916 conn(0x7f2a100b1e90 msgr2=0x7f2a100b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:56:48.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:48.560+0000 7f2a17400640 1 -- 172.21.15.131:0/2600088916 shutdown_connections 2024-05-26T22:56:48.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:48.560+0000 7f2a17400640 1 -- 172.21.15.131:0/2600088916 wait complete. 2024-05-26T22:56:48.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:48.562+0000 7f2a17400640 1 Processor -- start 2024-05-26T22:56:48.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:48.562+0000 7f2a17400640 1 -- start start 2024-05-26T22:56:48.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:48.563+0000 7f2a17400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2a1006daf0 0x7f2a101f7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:48.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:48.563+0000 7f2a17400640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2a100b36d0 0x7f2a100ae890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:48.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:48.563+0000 7f2a17400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a10158c00 0x7f2a101598d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:48.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:48.563+0000 7f2a17400640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f2a100b3b60 con 0x7f2a10158c00 2024-05-26T22:56:48.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:48.563+0000 7f2a17400640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2a100b49d0 con 0x7f2a100b36d0 2024-05-26T22:56:48.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:48.563+0000 7f2a17400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f2a101624e0 con 0x7f2a1006daf0 2024-05-26T22:56:48.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:48.563+0000 7f2a15400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a10158c00 0x7f2a101598d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:48.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:48.563+0000 7f2a15400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a10158c00 0x7f2a101598d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41802/0 (socket says 172.21.15.131:41802) 2024-05-26T22:56:48.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:48.563+0000 7f2a15400640 1 -- 172.21.15.131:0/1762391365 learned_addr learned my addr 172.21.15.131:0/1762391365 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:56:48.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:48.563+0000 7f2a0fe00640 1 --2- 172.21.15.131:0/1762391365 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2a100b36d0 0x7f2a100ae890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:48.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:48.563+0000 7f2a14a00640 1 --2- 172.21.15.131:0/1762391365 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2a1006daf0 0x7f2a101f7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:48.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:48.563+0000 7f2a15400640 1 -- 172.21.15.131:0/1762391365 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2a1006daf0 msgr2=0x7f2a101f7120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:48.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:48.563+0000 7f2a15400640 1 --2- 172.21.15.131:0/1762391365 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2a1006daf0 0x7f2a101f7120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:48.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:48.563+0000 7f2a15400640 1 -- 172.21.15.131:0/1762391365 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2a100b36d0 msgr2=0x7f2a100ae890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:48.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:48.563+0000 7f2a15400640 1 --2- 172.21.15.131:0/1762391365 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2a100b36d0 0x7f2a100ae890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:48.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:48.563+0000 7f2a15400640 1 -- 172.21.15.131:0/1762391365 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a101553e0 con 0x7f2a10158c00 2024-05-26T22:56:48.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:48.564+0000 7f2a15400640 1 --2- 172.21.15.131:0/1762391365 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a10158c00 0x7f2a101598d0 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f2a0800cf20 tx=0x7f2a0800a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:48.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:48.564+0000 7f2a0cc00640 1 -- 172.21.15.131:0/1762391365 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a0801a070 con 0x7f2a10158c00 2024-05-26T22:56:48.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:48.564+0000 7f2a0cc00640 1 -- 172.21.15.131:0/1762391365 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a0800d840 con 0x7f2a10158c00 2024-05-26T22:56:48.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:48.564+0000 7f2a0cc00640 1 -- 172.21.15.131:0/1762391365 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a0800dc30 con 0x7f2a10158c00 2024-05-26T22:56:48.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:48.564+0000 7f2a17400640 1 -- 172.21.15.131:0/1762391365 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a1015bd30 con 0x7f2a10158c00 2024-05-26T22:56:48.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:48.564+0000 7f2a17400640 1 -- 172.21.15.131:0/1762391365 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a10155b80 con 0x7f2a10158c00 2024-05-26T22:56:48.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:48.565+0000 7f2a17400640 1 -- 172.21.15.131:0/1762391365 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a100694e0 con 0x7f2a10158c00 2024-05-26T22:56:48.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:48.566+0000 7f2a0cc00640 1 -- 172.21.15.131:0/1762391365 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2a0801d070 con 0x7f2a10158c00 2024-05-26T22:56:48.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:48.567+0000 7f2a0cc00640 1 --2- 172.21.15.131:0/1762391365 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f29f006be60 0x7f29f006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:48.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:48.567+0000 7f2a0cc00640 1 -- 172.21.15.131:0/1762391365 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5709+0+0 (secure 0 0 0) 0x7f2a0808b6e0 con 0x7f2a10158c00 2024-05-26T22:56:48.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:48.567+0000 7f2a14a00640 1 --2- 172.21.15.131:0/1762391365 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f29f006be60 0x7f29f006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:48.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:48.568+0000 7f2a14a00640 1 --2- 172.21.15.131:0/1762391365 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f29f006be60 0x7f29f006e320 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f2a0400b810 tx=0x7f2a04008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:48.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:48.572+0000 7f2a0cc00640 1 -- 172.21.15.131:0/1762391365 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f2a0805a080 con 0x7f2a10158c00 2024-05-26T22:56:48.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:48.772+0000 7f2a17400640 1 -- 172.21.15.131:0/1762391365 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2a100d0510 con 0x7f2a10158c00 2024-05-26T22:56:48.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:48.774+0000 7f2a0cc00640 1 -- 172.21.15.131:0/1762391365 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v107) ==== 74+0+16651 (secure 0 0 0) 0x7f2a08059980 con 0x7f2a10158c00 2024-05-26T22:56:48.774 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:56:48.774 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":107,"fsid":"46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f","created":"2024-05-26T22:47:57.288905+0000","modified":"2024-05-26T22:56:03.175526+0000","last_up_change":"2024-05-26T22:48:09.602207+0000","last_in_change":"2024-05-26T22:48:00.687341+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-26T22:48:10.463339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-05-26T22:55:59.775225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-05-26T22:56:00.988905+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"814eaf62-a18d-494b-8e27-fd8fb88adea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6801","nonce":2350940051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6803","nonce":2350940051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6807","nonce":2350940051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6805","nonce":2350940051}]},"public_addr":"172.21.15.131:6801/2350940051","cluster_addr":"172.21.15.131:6803/2350940051","heartbeat_back_addr":"172.21.15.131:6807/2350940051","heartbeat_front_addr":"172.21.15.131:6805/2350940051","state":["exists","up"]},{"osd":1,"uuid":"42e66287-f6c6-4e26-af3c-6dfaa3536d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6825","nonce":1525528247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6827","nonce":1525528247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6831","nonce":1525528247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6829","nonce":1525528247}]},"public_addr":"172.21.15.131:6825/1525528247","cluster_addr":"172.21.15.131:6827/1525528247","heartbeat_back_addr":"172.21.15.131:6831/1525528247","heartbeat_front_addr":"172.21.15.131:6829/1525528247","state":["exists","up"]},{"osd":2,"uuid":"0da34673-04a2-4fd0-b57a-0ff2b679b043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6817","nonce":3627842210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6819","nonce":3627842210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6823","nonce":3627842210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6821","nonce":3627842210}]},"public_addr":"172.21.15.131:6817/3627842210","cluster_addr":"172.21.15.131:6819/3627842210","heartbeat_back_addr":"172.21.15.131:6823/3627842210","heartbeat_front_addr":"172.21.15.131:6821/3627842210","state":["exists","up"]},{"osd":3,"uuid":"cf117a27-d6b2-43d3-ae52-b47988bf8eed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6809","nonce":350790591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6811","nonce":350790591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6815","nonce":350790591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6813","nonce":350790591}]},"public_addr":"172.21.15.131:6809/350790591","cluster_addr":"172.21.15.131:6811/350790591","heartbeat_back_addr":"172.21.15.131:6815/350790591","heartbeat_front_addr":"172.21.15.131:6813/350790591","state":["exists","up"]},{"osd":4,"uuid":"a092ccfc-be89-4a84-88d5-0f781fa715eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6816","nonce":244560995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6819","nonce":244560995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6823","nonce":244560995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6821","nonce":244560995}]},"public_addr":"172.21.15.187:6816/244560995","cluster_addr":"172.21.15.187:6819/244560995","heartbeat_back_addr":"172.21.15.187:6823/244560995","heartbeat_front_addr":"172.21.15.187:6821/244560995","state":["exists","up"]},{"osd":5,"uuid":"7513ba69-dd00-4a35-b190-9e8cc3b2bdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6801","nonce":324619398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6803","nonce":324619398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6812","nonce":324619398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6807","nonce":324619398}]},"public_addr":"172.21.15.187:6801/324619398","cluster_addr":"172.21.15.187:6803/324619398","heartbeat_back_addr":"172.21.15.187:6812/324619398","heartbeat_front_addr":"172.21.15.187:6807/324619398","state":["exists","up"]},{"osd":6,"uuid":"50081810-c093-4d2f-a4d7-b7c695c5400d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6806","nonce":1391347786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6809","nonce":1391347786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6818","nonce":1391347786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6813","nonce":1391347786}]},"public_addr":"172.21.15.187:6806/1391347786","cluster_addr":"172.21.15.187:6809/1391347786","heartbeat_back_addr":"172.21.15.187:6818/1391347786","heartbeat_front_addr":"172.21.15.187:6813/1391347786","state":["exists","up"]},{"osd":7,"uuid":"775dae05-54f5-4ad4-bbce-92863c6ed2ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6825","nonce":1220065622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6827","nonce":1220065622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6831","nonce":1220065622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6829","nonce":1220065622}]},"public_addr":"172.21.15.187:6825/1220065622","cluster_addr":"172.21.15.187:6827/1220065622","heartbeat_back_addr":"172.21.15.187:6831/1220065622","heartbeat_front_addr":"172.21.15.187:6829/1220065622","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:05.983520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.299516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.067323+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.307748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.018742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.258775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.010912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.295355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-26T22:56:48.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:48.777+0000 7f2a17400640 1 -- 172.21.15.131:0/1762391365 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f29f006be60 msgr2=0x7f29f006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:48.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:48.777+0000 7f2a17400640 1 --2- 172.21.15.131:0/1762391365 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f29f006be60 0x7f29f006e320 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f2a0400b810 tx=0x7f2a04008040 comp rx=0 tx=0).stop 2024-05-26T22:56:48.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:48.777+0000 7f2a17400640 1 -- 172.21.15.131:0/1762391365 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a10158c00 msgr2=0x7f2a101598d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:48.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:48.777+0000 7f2a17400640 1 --2- 172.21.15.131:0/1762391365 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a10158c00 0x7f2a101598d0 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f2a0800cf20 tx=0x7f2a0800a7d0 comp rx=0 tx=0).stop 2024-05-26T22:56:48.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:48.777+0000 7f2a17400640 1 -- 172.21.15.131:0/1762391365 shutdown_connections 2024-05-26T22:56:48.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:48.777+0000 7f2a17400640 1 --2- 172.21.15.131:0/1762391365 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f29f006be60 0x7f29f006e320 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:48.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:48.777+0000 7f2a17400640 1 --2- 172.21.15.131:0/1762391365 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a10158c00 0x7f2a101598d0 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:48.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:48.777+0000 7f2a17400640 1 --2- 172.21.15.131:0/1762391365 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2a100b36d0 0x7f2a100ae890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:48.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:48.777+0000 7f2a17400640 1 --2- 172.21.15.131:0/1762391365 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2a1006daf0 0x7f2a101f7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:48.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:48.777+0000 7f2a17400640 1 -- 172.21.15.131:0/1762391365 >> 172.21.15.131:0/1762391365 conn(0x7f2a100b1e90 msgr2=0x7f2a100b2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:56:48.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:48.778+0000 7f2a17400640 1 -- 172.21.15.131:0/1762391365 shutdown_connections 2024-05-26T22:56:48.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:48.778+0000 7f2a17400640 1 -- 172.21.15.131:0/1762391365 wait complete. 2024-05-26T22:56:48.795 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-26T22:48:10.463339+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-26T22:55:59.775225+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '106', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-05-26T22:56:00.988905+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '107', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-26T22:56:48.795 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-26T22:56:48.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:48.998+0000 7ff50be00640 1 Processor -- start 2024-05-26T22:56:48.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:48.999+0000 7ff50be00640 1 -- start start 2024-05-26T22:56:48.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.000+0000 7ff50be00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7ff50c1618f0 0x7ff50c0d4f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:49.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.000+0000 7ff50be00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7ff50c06cf10 0x7ff50c0cf090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:49.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.000+0000 7ff50be00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff50c06e250 con 0x7ff50c06c3d0 2024-05-26T22:56:49.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.000+0000 7ff50be00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7ff50c06d6d0 con 0x7ff50c1618f0 2024-05-26T22:56:49.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.000+0000 7ff50be00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7ff50c06d9b0 con 0x7ff50c06cf10 2024-05-26T22:56:49.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.000+0000 7ff50aa00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7ff50c1618f0 0x7ff50c0d4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:49.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.000+0000 7ff502000640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7ff50c06c3d0 0x7ff50c06c800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:48786/0 (socket says 172.21.15.131:48786) 2024-05-26T22:56:49.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.000+0000 7ff502000640 1 -- 172.21.15.131:0/117573574 learned_addr learned my addr 172.21.15.131:0/117573574 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:56:49.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.001+0000 7ff50b400640 1 --2- 172.21.15.131:0/117573574 >> v2:172.21.15.187:3300/0 conn(0x7ff50c06cf10 0x7ff50c0cf090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:49.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.001+0000 7ff50aa00640 1 -- 172.21.15.131:0/117573574 >> v2:172.21.15.187:3300/0 conn(0x7ff50c06cf10 msgr2=0x7ff50c0cf090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:49.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.001+0000 7ff50aa00640 1 --2- 172.21.15.131:0/117573574 >> v2:172.21.15.187:3300/0 conn(0x7ff50c06cf10 0x7ff50c0cf090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:49.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.001+0000 7ff50aa00640 1 -- 172.21.15.131:0/117573574 >> v1:172.21.15.131:6789/0 conn(0x7ff50c06c3d0 legacy=0x7ff50c06c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:49.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.001+0000 7ff50aa00640 1 -- 172.21.15.131:0/117573574 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff50c06dc90 con 0x7ff50c1618f0 2024-05-26T22:56:49.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.001+0000 7ff50a000640 1 -- 172.21.15.131:0/117573574 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 352321229 0 0) 0x7ff50c06e250 con 0x7ff50c06c3d0 2024-05-26T22:56:49.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.001+0000 7ff50b400640 1 --2- 172.21.15.131:0/117573574 >> v2:172.21.15.187:3300/0 conn(0x7ff50c06cf10 0x7ff50c0cf090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:56:49.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.002+0000 7ff50aa00640 1 --2- 172.21.15.131:0/117573574 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff50c1618f0 0x7ff50c0d4f50 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7ff50c08ccb0 tx=0x7ff4fc02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d01b0e4280218cad server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:49.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.002+0000 7ff50a000640 1 -- 172.21.15.131:0/117573574 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4fc007740 con 0x7ff50c1618f0 2024-05-26T22:56:49.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.002+0000 7ff50a000640 1 -- 172.21.15.131:0/117573574 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4fc0078e0 con 0x7ff50c1618f0 2024-05-26T22:56:49.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.002+0000 7ff50a000640 1 -- 172.21.15.131:0/117573574 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4fc007cb0 con 0x7ff50c1618f0 2024-05-26T22:56:49.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.003+0000 7ff50be00640 1 -- 172.21.15.131:0/117573574 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff50c1618f0 msgr2=0x7ff50c0d4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:49.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.003+0000 7ff50be00640 1 --2- 172.21.15.131:0/117573574 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff50c1618f0 0x7ff50c0d4f50 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7ff50c08ccb0 tx=0x7ff4fc02a710 comp rx=0 tx=0).stop 2024-05-26T22:56:49.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.003+0000 7ff50be00640 1 -- 172.21.15.131:0/117573574 shutdown_connections 2024-05-26T22:56:49.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.003+0000 7ff50be00640 1 --2- 172.21.15.131:0/117573574 >> v2:172.21.15.187:3300/0 conn(0x7ff50c06cf10 0x7ff50c0cf090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:49.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.003+0000 7ff50be00640 1 --2- 172.21.15.131:0/117573574 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff50c1618f0 0x7ff50c0d4f50 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:49.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.003+0000 7ff50be00640 1 -- 172.21.15.131:0/117573574 >> 172.21.15.131:0/117573574 conn(0x7ff50c09d720 msgr2=0x7ff50c0d43a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:56:49.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.003+0000 7ff50be00640 1 -- 172.21.15.131:0/117573574 shutdown_connections 2024-05-26T22:56:49.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.003+0000 7ff50be00640 1 -- 172.21.15.131:0/117573574 wait complete. 2024-05-26T22:56:49.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.005+0000 7ff50be00640 1 Processor -- start 2024-05-26T22:56:49.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.005+0000 7ff50be00640 1 -- start start 2024-05-26T22:56:49.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.006+0000 7ff50be00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff50c06c3d0 0x7ff50c0d1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:49.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.006+0000 7ff50be00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff50c06cf10 0x7ff50c0c9b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:49.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.006+0000 7ff50be00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff50c1618f0 0x7ff50c15ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:49.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.006+0000 7ff50be00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff50c161e50 con 0x7ff50c1618f0 2024-05-26T22:56:49.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.006+0000 7ff50be00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff50c161cd0 con 0x7ff50c06cf10 2024-05-26T22:56:49.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.006+0000 7ff50be00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff50c0cf5d0 con 0x7ff50c06c3d0 2024-05-26T22:56:49.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.006+0000 7ff50b400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff50c1618f0 0x7ff50c15ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:49.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.006+0000 7ff50b400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff50c1618f0 0x7ff50c15ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41830/0 (socket says 172.21.15.131:41830) 2024-05-26T22:56:49.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.006+0000 7ff50b400640 1 -- 172.21.15.131:0/1178579969 learned_addr learned my addr 172.21.15.131:0/1178579969 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:56:49.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.006+0000 7ff50aa00640 1 --2- 172.21.15.131:0/1178579969 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff50c06c3d0 0x7ff50c0d1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:49.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.006+0000 7ff502000640 1 --2- 172.21.15.131:0/1178579969 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff50c06cf10 0x7ff50c0c9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:49.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.006+0000 7ff50b400640 1 -- 172.21.15.131:0/1178579969 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff50c06c3d0 msgr2=0x7ff50c0d1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:49.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.007+0000 7ff50b400640 1 --2- 172.21.15.131:0/1178579969 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff50c06c3d0 0x7ff50c0d1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:49.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.007+0000 7ff50b400640 1 -- 172.21.15.131:0/1178579969 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff50c06cf10 msgr2=0x7ff50c0c9b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:49.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.007+0000 7ff50b400640 1 --2- 172.21.15.131:0/1178579969 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff50c06cf10 0x7ff50c0c9b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:49.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.007+0000 7ff50b400640 1 -- 172.21.15.131:0/1178579969 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff50c0ca580 con 0x7ff50c1618f0 2024-05-26T22:56:49.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.007+0000 7ff50aa00640 1 --2- 172.21.15.131:0/1178579969 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff50c06c3d0 0x7ff50c0d1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:56:49.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.007+0000 7ff50b400640 1 --2- 172.21.15.131:0/1178579969 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff50c1618f0 0x7ff50c15ba60 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7ff50400c7f0 tx=0x7ff50400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:49.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.007+0000 7ff503400640 1 -- 172.21.15.131:0/1178579969 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff504015070 con 0x7ff50c1618f0 2024-05-26T22:56:49.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.007+0000 7ff503400640 1 -- 172.21.15.131:0/1178579969 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff504007820 con 0x7ff50c1618f0 2024-05-26T22:56:49.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.007+0000 7ff503400640 1 -- 172.21.15.131:0/1178579969 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff504007c50 con 0x7ff50c1618f0 2024-05-26T22:56:49.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.007+0000 7ff50be00640 1 -- 172.21.15.131:0/1178579969 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff50c0c8120 con 0x7ff50c1618f0 2024-05-26T22:56:49.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.007+0000 7ff50be00640 1 -- 172.21.15.131:0/1178579969 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff50c0c86e0 con 0x7ff50c1618f0 2024-05-26T22:56:49.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.008+0000 7ff50be00640 1 -- 172.21.15.131:0/1178579969 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff50c0c88e0 con 0x7ff50c1618f0 2024-05-26T22:56:49.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.009+0000 7ff503400640 1 -- 172.21.15.131:0/1178579969 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff504018070 con 0x7ff50c1618f0 2024-05-26T22:56:49.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.010+0000 7ff503400640 1 --2- 172.21.15.131:0/1178579969 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ff4e806be60 0x7ff4e806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:49.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.010+0000 7ff503400640 1 -- 172.21.15.131:0/1178579969 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5709+0+0 (secure 0 0 0) 0x7ff50408aff0 con 0x7ff50c1618f0 2024-05-26T22:56:49.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.011+0000 7ff50aa00640 1 --2- 172.21.15.131:0/1178579969 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ff4e806be60 0x7ff4e806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:49.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.011+0000 7ff50aa00640 1 --2- 172.21.15.131:0/1178579969 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ff4e806be60 0x7ff4e806e320 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc034000 tx=0x7ff4fc033040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:49.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.015+0000 7ff503400640 1 -- 172.21.15.131:0/1178579969 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7ff50c0c88e0 con 0x7ff50c1618f0 2024-05-26T22:56:49.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.212+0000 7ff50be00640 1 -- 172.21.15.131:0/1178579969 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff50c0694e0 con 0x7ff50c1618f0 2024-05-26T22:56:49.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.214+0000 7ff503400640 1 -- 172.21.15.131:0/1178579969 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v107) ==== 93+0+10 (secure 0 0 0) 0x7ff504059960 con 0x7ff50c1618f0 2024-05-26T22:56:49.213 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2024-05-26T22:56:49.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.216+0000 7ff50be00640 1 -- 172.21.15.131:0/1178579969 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ff4e806be60 msgr2=0x7ff4e806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:49.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.216+0000 7ff50be00640 1 --2- 172.21.15.131:0/1178579969 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ff4e806be60 0x7ff4e806e320 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc034000 tx=0x7ff4fc033040 comp rx=0 tx=0).stop 2024-05-26T22:56:49.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.216+0000 7ff50be00640 1 -- 172.21.15.131:0/1178579969 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff50c1618f0 msgr2=0x7ff50c15ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:49.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.216+0000 7ff50be00640 1 --2- 172.21.15.131:0/1178579969 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff50c1618f0 0x7ff50c15ba60 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7ff50400c7f0 tx=0x7ff50400ccb0 comp rx=0 tx=0).stop 2024-05-26T22:56:49.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.216+0000 7ff50be00640 1 -- 172.21.15.131:0/1178579969 shutdown_connections 2024-05-26T22:56:49.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.216+0000 7ff50be00640 1 --2- 172.21.15.131:0/1178579969 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ff4e806be60 0x7ff4e806e320 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:49.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.216+0000 7ff50be00640 1 --2- 172.21.15.131:0/1178579969 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff50c1618f0 0x7ff50c15ba60 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:49.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.216+0000 7ff50be00640 1 --2- 172.21.15.131:0/1178579969 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff50c06cf10 0x7ff50c0c9b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:49.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.216+0000 7ff50be00640 1 --2- 172.21.15.131:0/1178579969 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff50c06c3d0 0x7ff50c0d1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:49.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.216+0000 7ff50be00640 1 -- 172.21.15.131:0/1178579969 >> 172.21.15.131:0/1178579969 conn(0x7ff50c09d720 msgr2=0x7ff50c0b0f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:56:49.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.217+0000 7ff50be00640 1 -- 172.21.15.131:0/1178579969 shutdown_connections 2024-05-26T22:56:49.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.217+0000 7ff50be00640 1 -- 172.21.15.131:0/1178579969 wait complete. 2024-05-26T22:56:49.234 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-26T22:56:49.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.439+0000 7f4843e00640 1 Processor -- start 2024-05-26T22:56:49.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.440+0000 7f4843e00640 1 -- start start 2024-05-26T22:56:49.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.441+0000 7f4843e00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f483c0b5450 0x7f483c0b43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:49.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.441+0000 7f4843e00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f483c0b4900 con 0x7f483c06f420 2024-05-26T22:56:49.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.441+0000 7f4843e00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f483c160180 con 0x7f483c0d0460 2024-05-26T22:56:49.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.441+0000 7f4843e00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f483c160d90 con 0x7f483c0b5450 2024-05-26T22:56:49.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.442+0000 7f4840a00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f483c06f420 0x7f483c0cfd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:48790/0 (socket says 172.21.15.131:48790) 2024-05-26T22:56:49.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.442+0000 7f4840a00640 1 -- 172.21.15.131:0/4026187574 learned_addr learned my addr 172.21.15.131:0/4026187574 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:56:49.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.442+0000 7f483be00640 1 -- 172.21.15.131:0/4026187574 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3059754285 0 0) 0x7f483c0b4900 con 0x7f483c06f420 2024-05-26T22:56:49.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.442+0000 7f4841400640 1 --2- 172.21.15.131:0/4026187574 >> v2:172.21.15.187:3300/0 conn(0x7f483c0b5450 0x7f483c0b43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:49.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.442+0000 7f483be00640 1 -- 172.21.15.131:0/4026187574 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4820003520 con 0x7f483c06f420 2024-05-26T22:56:49.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.442+0000 7f483be00640 1 -- 172.21.15.131:0/4026187574 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4044188040 0 0) 0x7f4820003520 con 0x7f483c06f420 2024-05-26T22:56:49.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.442+0000 7f483be00640 1 -- 172.21.15.131:0/4026187574 >> v2:172.21.15.187:3300/0 conn(0x7f483c0b5450 msgr2=0x7f483c0b43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:49.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.442+0000 7f483be00640 1 --2- 172.21.15.131:0/4026187574 >> v2:172.21.15.187:3300/0 conn(0x7f483c0b5450 0x7f483c0b43c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:49.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.442+0000 7f483be00640 1 -- 172.21.15.131:0/4026187574 >> v1:172.21.15.187:6789/0 conn(0x7f483c0d0460 legacy=0x7f483c06d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:49.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.442+0000 7f483be00640 1 -- 172.21.15.131:0/4026187574 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f483c160460 con 0x7f483c06f420 2024-05-26T22:56:49.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.443+0000 7f483be00640 1 -- 172.21.15.131:0/4026187574 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7f4828001670 con 0x7f483c06f420 2024-05-26T22:56:49.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.443+0000 7f483be00640 1 -- 172.21.15.131:0/4026187574 >> v1:172.21.15.131:6789/0 conn(0x7f483c06f420 legacy=0x7f483c0cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:49.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.443+0000 7f483be00640 1 --2- 172.21.15.131:0/4026187574 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4820003a00 0x7f4820003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:49.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.443+0000 7f483be00640 1 --2- 172.21.15.131:0/4026187574 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4820004800 0x7f4820004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:49.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.443+0000 7f483be00640 1 --2- 172.21.15.131:0/4026187574 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4820005210 0x7f482000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:49.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.443+0000 7f483be00640 1 -- 172.21.15.131:0/4026187574 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f483c160d90 con 0x7f4820005210 2024-05-26T22:56:49.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.443+0000 7f483be00640 1 -- 172.21.15.131:0/4026187574 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f482000c160 con 0x7f4820003a00 2024-05-26T22:56:49.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.443+0000 7f483be00640 1 -- 172.21.15.131:0/4026187574 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f482000c360 con 0x7f4820004800 2024-05-26T22:56:49.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.443+0000 7f4841e00640 1 --2- 172.21.15.131:0/4026187574 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4820005210 0x7f482000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:49.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.443+0000 7f4840a00640 1 --2- 172.21.15.131:0/4026187574 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4820004800 0x7f4820004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:49.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.443+0000 7f4841400640 1 --2- 172.21.15.131:0/4026187574 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4820003a00 0x7f4820003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:49.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.443+0000 7f4841e00640 1 -- 172.21.15.131:0/4026187574 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4820004800 msgr2=0x7f4820004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:49.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.443+0000 7f4841e00640 1 --2- 172.21.15.131:0/4026187574 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4820004800 0x7f4820004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:49.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.443+0000 7f4841e00640 1 -- 172.21.15.131:0/4026187574 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4820003a00 msgr2=0x7f4820003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:49.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.443+0000 7f4841e00640 1 --2- 172.21.15.131:0/4026187574 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4820003a00 0x7f4820003e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:49.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.443+0000 7f4841e00640 1 -- 172.21.15.131:0/4026187574 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4820003520 con 0x7f4820005210 2024-05-26T22:56:49.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.443+0000 7f4840a00640 1 --2- 172.21.15.131:0/4026187574 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4820004800 0x7f4820004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:56:49.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.444+0000 7f4841e00640 1 -- 172.21.15.131:0/4026187574 reap_dead start 2024-05-26T22:56:49.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.444+0000 7f4841e00640 1 --2- 172.21.15.131:0/4026187574 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4820005210 0x7f482000b9c0 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f4834001e50 tx=0x7f4820005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b45ef2a9b9a1e959 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:49.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.445+0000 7f483be00640 1 -- 172.21.15.131:0/4026187574 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4834001a70 con 0x7f4820005210 2024-05-26T22:56:49.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.445+0000 7f483be00640 1 -- 172.21.15.131:0/4026187574 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4834003970 con 0x7f4820005210 2024-05-26T22:56:49.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.445+0000 7f483be00640 1 -- 172.21.15.131:0/4026187574 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4834003d60 con 0x7f4820005210 2024-05-26T22:56:49.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.445+0000 7f4843e00640 1 -- 172.21.15.131:0/4026187574 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4820005210 msgr2=0x7f482000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:49.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.445+0000 7f4843e00640 1 --2- 172.21.15.131:0/4026187574 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4820005210 0x7f482000b9c0 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f4834001e50 tx=0x7f4820005830 comp rx=0 tx=0).stop 2024-05-26T22:56:49.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.445+0000 7f4843e00640 1 -- 172.21.15.131:0/4026187574 shutdown_connections 2024-05-26T22:56:49.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.445+0000 7f4843e00640 1 --2- 172.21.15.131:0/4026187574 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4820005210 0x7f482000b9c0 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:49.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.445+0000 7f4843e00640 1 -- 172.21.15.131:0/4026187574 >> 172.21.15.131:0/4026187574 conn(0x7f483c06bc90 msgr2=0x7f483c0b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:56:49.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.446+0000 7f4843e00640 1 -- 172.21.15.131:0/4026187574 shutdown_connections 2024-05-26T22:56:49.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.446+0000 7f4843e00640 1 -- 172.21.15.131:0/4026187574 wait complete. 2024-05-26T22:56:49.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.447+0000 7f4843e00640 1 Processor -- start 2024-05-26T22:56:49.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.447+0000 7f4843e00640 1 -- start start 2024-05-26T22:56:49.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.448+0000 7f4843e00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4820005210 0x7f483c157a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:49.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.448+0000 7f4843e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f483c154ad0 0x7f483c153fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:49.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.448+0000 7f4843e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f483c154520 0x7f483c1447c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:49.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.448+0000 7f4843e00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f483c0b5830 con 0x7f483c154520 2024-05-26T22:56:49.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.448+0000 7f4843e00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f483c160a20 con 0x7f4820005210 2024-05-26T22:56:49.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.448+0000 7f4843e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f483c15fde0 con 0x7f483c154ad0 2024-05-26T22:56:49.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.449+0000 7f4841e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f483c154520 0x7f483c1447c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:49.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.449+0000 7f4841e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f483c154520 0x7f483c1447c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41838/0 (socket says 172.21.15.131:41838) 2024-05-26T22:56:49.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.449+0000 7f4841e00640 1 -- 172.21.15.131:0/1628330855 learned_addr learned my addr 172.21.15.131:0/1628330855 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:56:49.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.449+0000 7f4841400640 1 --2- 172.21.15.131:0/1628330855 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4820005210 0x7f483c157a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:49.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.449+0000 7f4840a00640 1 --2- 172.21.15.131:0/1628330855 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f483c154ad0 0x7f483c153fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:49.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.449+0000 7f4841e00640 1 -- 172.21.15.131:0/1628330855 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f483c154ad0 msgr2=0x7f483c153fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:49.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.449+0000 7f4841e00640 1 --2- 172.21.15.131:0/1628330855 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f483c154ad0 0x7f483c153fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:49.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.449+0000 7f4841e00640 1 -- 172.21.15.131:0/1628330855 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4820005210 msgr2=0x7f483c157a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:49.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.449+0000 7f4841e00640 1 --2- 172.21.15.131:0/1628330855 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4820005210 0x7f483c157a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:49.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.449+0000 7f4841e00640 1 -- 172.21.15.131:0/1628330855 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f483c072c60 con 0x7f483c154520 2024-05-26T22:56:49.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.449+0000 7f4841400640 1 --2- 172.21.15.131:0/1628330855 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4820005210 0x7f483c157a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:56:49.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.450+0000 7f4841e00640 1 --2- 172.21.15.131:0/1628330855 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f483c154520 0x7f483c1447c0 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f4834001f70 tx=0x7f48340014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:49.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.450+0000 7f4839600640 1 -- 172.21.15.131:0/1628330855 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4834001a70 con 0x7f483c154520 2024-05-26T22:56:49.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.450+0000 7f4839600640 1 -- 172.21.15.131:0/1628330855 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4834001250 con 0x7f483c154520 2024-05-26T22:56:49.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.450+0000 7f4839600640 1 -- 172.21.15.131:0/1628330855 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4834006d70 con 0x7f483c154520 2024-05-26T22:56:49.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.450+0000 7f4843e00640 1 -- 172.21.15.131:0/1628330855 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f483c071ea0 con 0x7f483c154520 2024-05-26T22:56:49.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.450+0000 7f4843e00640 1 -- 172.21.15.131:0/1628330855 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f483c072460 con 0x7f483c154520 2024-05-26T22:56:49.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.451+0000 7f4843e00640 1 -- 172.21.15.131:0/1628330855 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f483c0695c0 con 0x7f483c154520 2024-05-26T22:56:49.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.452+0000 7f4839600640 1 -- 172.21.15.131:0/1628330855 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4834007030 con 0x7f483c154520 2024-05-26T22:56:49.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.453+0000 7f4839600640 1 --2- 172.21.15.131:0/1628330855 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f481c06be30 0x7f481c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:49.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.453+0000 7f4839600640 1 -- 172.21.15.131:0/1628330855 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5709+0+0 (secure 0 0 0) 0x7f48340aa640 con 0x7f483c154520 2024-05-26T22:56:49.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.453+0000 7f4841400640 1 --2- 172.21.15.131:0/1628330855 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f481c06be30 0x7f481c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:49.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.454+0000 7f4841400640 1 --2- 172.21.15.131:0/1628330855 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f481c06be30 0x7f481c06e2f0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f482c006fd0 tx=0x7f482c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:49.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.458+0000 7f4839600640 1 -- 172.21.15.131:0/1628330855 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f4834078fe0 con 0x7f483c154520 2024-05-26T22:56:49.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.657+0000 7f4843e00640 1 -- 172.21.15.131:0/1628330855 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f483c0b1e40 con 0x7f483c154520 2024-05-26T22:56:49.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.658+0000 7f4839600640 1 -- 172.21.15.131:0/1628330855 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v107) ==== 104+0+11 (secure 0 0 0) 0x7f483407de20 con 0x7f483c154520 2024-05-26T22:56:49.658 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-05-26T22:56:49.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.661+0000 7f4843e00640 1 -- 172.21.15.131:0/1628330855 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f481c06be30 msgr2=0x7f481c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:49.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.661+0000 7f4843e00640 1 --2- 172.21.15.131:0/1628330855 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f481c06be30 0x7f481c06e2f0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f482c006fd0 tx=0x7f482c008040 comp rx=0 tx=0).stop 2024-05-26T22:56:49.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.661+0000 7f4843e00640 1 -- 172.21.15.131:0/1628330855 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f483c154520 msgr2=0x7f483c1447c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:49.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.661+0000 7f4843e00640 1 --2- 172.21.15.131:0/1628330855 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f483c154520 0x7f483c1447c0 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f4834001f70 tx=0x7f48340014a0 comp rx=0 tx=0).stop 2024-05-26T22:56:49.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.661+0000 7f4843e00640 1 -- 172.21.15.131:0/1628330855 shutdown_connections 2024-05-26T22:56:49.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.661+0000 7f4843e00640 1 --2- 172.21.15.131:0/1628330855 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f483c154ad0 0x7f483c153fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:49.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.661+0000 7f4843e00640 1 --2- 172.21.15.131:0/1628330855 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f481c06be30 0x7f481c06e2f0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:49.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.661+0000 7f4843e00640 1 --2- 172.21.15.131:0/1628330855 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f483c154520 0x7f483c1447c0 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:49.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.661+0000 7f4843e00640 1 --2- 172.21.15.131:0/1628330855 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4820005210 0x7f483c157a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:49.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.661+0000 7f4843e00640 1 -- 172.21.15.131:0/1628330855 >> 172.21.15.131:0/1628330855 conn(0x7f483c06bc90 msgr2=0x7f483c15dae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:56:49.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.661+0000 7f4843e00640 1 -- 172.21.15.131:0/1628330855 shutdown_connections 2024-05-26T22:56:49.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.661+0000 7f4843e00640 1 -- 172.21.15.131:0/1628330855 wait complete. 2024-05-26T22:56:49.680 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-26T22:56:49.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.886+0000 7ff4e7e00640 1 Processor -- start 2024-05-26T22:56:49.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.886+0000 7ff4e7e00640 1 -- start start 2024-05-26T22:56:49.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.887+0000 7ff4e7e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7ff4e8145a10 0x7ff4e8072180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:49.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.888+0000 7ff4e7e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff4e80712d0 0x7ff4e806d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:49.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.888+0000 7ff4e7e00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7ff4e806d790 0x7ff4e815fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:49.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.888+0000 7ff4e7e00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7ff4e815c760 con 0x7ff4e8145a10 2024-05-26T22:56:49.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.888+0000 7ff4e7e00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7ff4e8072d60 con 0x7ff4e806d790 2024-05-26T22:56:49.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.888+0000 7ff4e7e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff4e8072a80 con 0x7ff4e80712d0 2024-05-26T22:56:49.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.888+0000 7ff4e6a00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7ff4e8145a10 0x7ff4e8072180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:49.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.888+0000 7ff4e6a00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7ff4e8145a10 0x7ff4e8072180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41848/0 (socket says 172.21.15.131:41848) 2024-05-26T22:56:49.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.888+0000 7ff4e6a00640 1 -- 172.21.15.131:0/508999459 learned_addr learned my addr 172.21.15.131:0/508999459 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:56:49.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.888+0000 7ff4e6000640 1 --2- 172.21.15.131:0/508999459 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff4e80712d0 0x7ff4e806d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:49.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.888+0000 7ff4e7400640 1 --2- 172.21.15.131:0/508999459 >> v2:172.21.15.187:3300/0 conn(0x7ff4e806d790 0x7ff4e815fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:49.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.888+0000 7ff4e6a00640 1 -- 172.21.15.131:0/508999459 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff4e80712d0 msgr2=0x7ff4e806d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:49.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.888+0000 7ff4e6a00640 1 --2- 172.21.15.131:0/508999459 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff4e80712d0 0x7ff4e806d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:49.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.889+0000 7ff4e6a00640 1 -- 172.21.15.131:0/508999459 >> v2:172.21.15.187:3300/0 conn(0x7ff4e806d790 msgr2=0x7ff4e815fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:49.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.889+0000 7ff4e6a00640 1 --2- 172.21.15.131:0/508999459 >> v2:172.21.15.187:3300/0 conn(0x7ff4e806d790 0x7ff4e815fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:49.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.889+0000 7ff4e6a00640 1 -- 172.21.15.131:0/508999459 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4e80ceab0 con 0x7ff4e8145a10 2024-05-26T22:56:49.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.889+0000 7ff4e7400640 1 --2- 172.21.15.131:0/508999459 >> v2:172.21.15.187:3300/0 conn(0x7ff4e806d790 0x7ff4e815fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:56:49.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.890+0000 7ff4e6a00640 1 --2- 172.21.15.131:0/508999459 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff4e8145a10 0x7ff4e8072180 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7ff4e80b25f0 tx=0x7ff4d802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcfe42e97dc0472b server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:49.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.890+0000 7ff4e5600640 1 -- 172.21.15.131:0/508999459 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d8037070 con 0x7ff4e8145a10 2024-05-26T22:56:49.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.890+0000 7ff4e5600640 1 -- 172.21.15.131:0/508999459 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4d8032440 con 0x7ff4e8145a10 2024-05-26T22:56:49.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.890+0000 7ff4e5600640 1 -- 172.21.15.131:0/508999459 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d8032810 con 0x7ff4e8145a10 2024-05-26T22:56:49.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.891+0000 7ff4e7e00640 1 -- 172.21.15.131:0/508999459 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff4e8145a10 msgr2=0x7ff4e8072180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:49.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.891+0000 7ff4e7e00640 1 --2- 172.21.15.131:0/508999459 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff4e8145a10 0x7ff4e8072180 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7ff4e80b25f0 tx=0x7ff4d802a1e0 comp rx=0 tx=0).stop 2024-05-26T22:56:49.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.891+0000 7ff4e7e00640 1 -- 172.21.15.131:0/508999459 shutdown_connections 2024-05-26T22:56:49.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.891+0000 7ff4e7e00640 1 --2- 172.21.15.131:0/508999459 >> v2:172.21.15.187:3300/0 conn(0x7ff4e806d790 0x7ff4e815fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:49.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.891+0000 7ff4e7e00640 1 --2- 172.21.15.131:0/508999459 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff4e80712d0 0x7ff4e806d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:49.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.891+0000 7ff4e7e00640 1 --2- 172.21.15.131:0/508999459 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff4e8145a10 0x7ff4e8072180 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:49.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.891+0000 7ff4e7e00640 1 -- 172.21.15.131:0/508999459 >> 172.21.15.131:0/508999459 conn(0x7ff4e80b5660 msgr2=0x7ff4e80b5a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:56:49.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.891+0000 7ff4e7e00640 1 -- 172.21.15.131:0/508999459 shutdown_connections 2024-05-26T22:56:49.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.891+0000 7ff4e7e00640 1 -- 172.21.15.131:0/508999459 wait complete. 2024-05-26T22:56:49.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.893+0000 7ff4e7e00640 1 Processor -- start 2024-05-26T22:56:49.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.893+0000 7ff4e7e00640 1 -- start start 2024-05-26T22:56:49.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.894+0000 7ff4e7e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff4e806d790 0x7ff4e8158760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:49.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.894+0000 7ff4e7e00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff4e80712d0 0x7ff4e813bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:49.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.894+0000 7ff4e7e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff4e8145a10 0x7ff4e8140910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:49.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.894+0000 7ff4e7e00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e8145df0 con 0x7ff4e806d790 2024-05-26T22:56:49.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.894+0000 7ff4e7e00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e80cf070 con 0x7ff4e80712d0 2024-05-26T22:56:49.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.894+0000 7ff4e7e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff4e80726c0 con 0x7ff4e8145a10 2024-05-26T22:56:49.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.894+0000 7ff4e6a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff4e806d790 0x7ff4e8158760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:49.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.894+0000 7ff4e6a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff4e806d790 0x7ff4e8158760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41856/0 (socket says 172.21.15.131:41856) 2024-05-26T22:56:49.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.894+0000 7ff4e6a00640 1 -- 172.21.15.131:0/986097192 learned_addr learned my addr 172.21.15.131:0/986097192 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:56:49.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.895+0000 7ff4e7400640 1 --2- 172.21.15.131:0/986097192 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff4e8145a10 0x7ff4e8140910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:49.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.895+0000 7ff4e6a00640 1 -- 172.21.15.131:0/986097192 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff4e8145a10 msgr2=0x7ff4e8140910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:49.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.895+0000 7ff4e6000640 1 --2- 172.21.15.131:0/986097192 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff4e80712d0 0x7ff4e813bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:49.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.895+0000 7ff4e6a00640 1 --2- 172.21.15.131:0/986097192 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff4e8145a10 0x7ff4e8140910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:49.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.895+0000 7ff4e6a00640 1 -- 172.21.15.131:0/986097192 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff4e80712d0 msgr2=0x7ff4e813bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:49.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.895+0000 7ff4e6a00640 1 --2- 172.21.15.131:0/986097192 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff4e80712d0 0x7ff4e813bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:49.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.895+0000 7ff4e6a00640 1 -- 172.21.15.131:0/986097192 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4e80b3060 con 0x7ff4e806d790 2024-05-26T22:56:49.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.895+0000 7ff4e7400640 1 --2- 172.21.15.131:0/986097192 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff4e8145a10 0x7ff4e8140910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:56:49.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.895+0000 7ff4e6a00640 1 --2- 172.21.15.131:0/986097192 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff4e806d790 0x7ff4e8158760 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7ff4d8000c00 tx=0x7ff4d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:49.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.895+0000 7ff4dea00640 1 -- 172.21.15.131:0/986097192 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d8041070 con 0x7ff4e806d790 2024-05-26T22:56:49.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.895+0000 7ff4dea00640 1 -- 172.21.15.131:0/986097192 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4d803c870 con 0x7ff4e806d790 2024-05-26T22:56:49.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.895+0000 7ff4dea00640 1 -- 172.21.15.131:0/986097192 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d8037040 con 0x7ff4e806d790 2024-05-26T22:56:49.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.896+0000 7ff4e7e00640 1 -- 172.21.15.131:0/986097192 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4e80b4940 con 0x7ff4e806d790 2024-05-26T22:56:49.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.896+0000 7ff4e7e00640 1 -- 172.21.15.131:0/986097192 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4e80b3600 con 0x7ff4e806d790 2024-05-26T22:56:49.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.897+0000 7ff4e7e00640 1 -- 172.21.15.131:0/986097192 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4e80694e0 con 0x7ff4e806d790 2024-05-26T22:56:49.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.898+0000 7ff4dea00640 1 -- 172.21.15.131:0/986097192 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff4d8044070 con 0x7ff4e806d790 2024-05-26T22:56:49.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.898+0000 7ff4dea00640 1 --2- 172.21.15.131:0/986097192 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ff4c806be30 0x7ff4c806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:49.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.899+0000 7ff4dea00640 1 -- 172.21.15.131:0/986097192 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5709+0+0 (secure 0 0 0) 0x7ff4d80abbb0 con 0x7ff4e806d790 2024-05-26T22:56:49.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.899+0000 7ff4e6000640 1 --2- 172.21.15.131:0/986097192 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ff4c806be30 0x7ff4c806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:49.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.899+0000 7ff4e6000640 1 --2- 172.21.15.131:0/986097192 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ff4c806be30 0x7ff4c806e2f0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7ff4e815b6c0 tx=0x7ff4d4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:49.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:49.903+0000 7ff4dea00640 1 -- 172.21.15.131:0/986097192 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7ff4d807a5e0 con 0x7ff4e806d790 2024-05-26T22:56:50.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:50.114+0000 7ff4e7e00640 1 -- 172.21.15.131:0/986097192 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff4e80af310 con 0x7ff4e806d790 2024-05-26T22:56:50.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:50.114+0000 7ff4dea00640 1 -- 172.21.15.131:0/986097192 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v107) ==== 100+0+11 (secure 0 0 0) 0x7ff4d8079ee0 con 0x7ff4e806d790 2024-05-26T22:56:50.114 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 64 2024-05-26T22:56:50.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:50.117+0000 7ff4e7e00640 1 -- 172.21.15.131:0/986097192 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ff4c806be30 msgr2=0x7ff4c806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:50.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:50.117+0000 7ff4e7e00640 1 --2- 172.21.15.131:0/986097192 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ff4c806be30 0x7ff4c806e2f0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7ff4e815b6c0 tx=0x7ff4d4008040 comp rx=0 tx=0).stop 2024-05-26T22:56:50.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:50.117+0000 7ff4e7e00640 1 -- 172.21.15.131:0/986097192 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff4e806d790 msgr2=0x7ff4e8158760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:50.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:50.117+0000 7ff4e7e00640 1 --2- 172.21.15.131:0/986097192 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff4e806d790 0x7ff4e8158760 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7ff4d8000c00 tx=0x7ff4d80046c0 comp rx=0 tx=0).stop 2024-05-26T22:56:50.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:50.117+0000 7ff4e7e00640 1 -- 172.21.15.131:0/986097192 shutdown_connections 2024-05-26T22:56:50.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:50.117+0000 7ff4e7e00640 1 --2- 172.21.15.131:0/986097192 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff4e8145a10 0x7ff4e8140910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:50.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:50.117+0000 7ff4e7e00640 1 --2- 172.21.15.131:0/986097192 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ff4c806be30 0x7ff4c806e2f0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:50.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:50.117+0000 7ff4e7e00640 1 --2- 172.21.15.131:0/986097192 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff4e80712d0 0x7ff4e813bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:50.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:50.117+0000 7ff4e7e00640 1 --2- 172.21.15.131:0/986097192 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff4e806d790 0x7ff4e8158760 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:50.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:50.117+0000 7ff4e7e00640 1 -- 172.21.15.131:0/986097192 >> 172.21.15.131:0/986097192 conn(0x7ff4e80b5660 msgr2=0x7ff4e80b1020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:56:50.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:50.117+0000 7ff4e7e00640 1 -- 172.21.15.131:0/986097192 shutdown_connections 2024-05-26T22:56:50.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:50.117+0000 7ff4e7e00640 1 -- 172.21.15.131:0/986097192 wait complete. 2024-05-26T22:56:50.134 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-26T22:56:50.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:50.324+0000 7f93e8c00640 1 Processor -- start 2024-05-26T22:56:50.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:50.324+0000 7f93e8c00640 1 -- start start 2024-05-26T22:56:50.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:50.326+0000 7f93e8c00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f93e406dd30 0x7f93e406e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:50.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:50.326+0000 7f93e8c00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f93e40725a0 0x7f93e40729a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:50.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:50.326+0000 7f93e8c00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93e415c1c0 con 0x7f93e4155770 2024-05-26T22:56:50.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:50.326+0000 7f93e8c00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f93e415c4a0 con 0x7f93e406dd30 2024-05-26T22:56:50.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:50.326+0000 7f93e8c00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f93e4071da0 con 0x7f93e40725a0 2024-05-26T22:56:50.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:50.326+0000 7f93e1e00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f93e4155770 0x7f93e415b700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:48794/0 (socket says 172.21.15.131:48794) 2024-05-26T22:56:50.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:50.326+0000 7f93e1e00640 1 -- 172.21.15.131:0/3923859616 learned_addr learned my addr 172.21.15.131:0/3923859616 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:56:50.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:50.327+0000 7f93e1400640 1 -- 172.21.15.131:0/3923859616 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 64754858 0 0) 0x7f93e415c1c0 con 0x7f93e4155770 2024-05-26T22:56:50.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:50.327+0000 7f93e2800640 1 --2- 172.21.15.131:0/3923859616 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f93e40725a0 0x7f93e40729a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:50.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:50.327+0000 7f93e1400640 1 -- 172.21.15.131:0/3923859616 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f93c8005730 con 0x7f93e4155770 2024-05-26T22:56:50.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:50.327+0000 7f93dbe00640 1 --2- 172.21.15.131:0/3923859616 >> v2:172.21.15.187:3300/0 conn(0x7f93e406dd30 0x7f93e406e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:50.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:50.327+0000 7f93e1400640 1 -- 172.21.15.131:0/3923859616 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3574242843 0 0) 0x7f93c8005730 con 0x7f93e4155770 2024-05-26T22:56:50.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:50.327+0000 7f93e1400640 1 -- 172.21.15.131:0/3923859616 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f93e40725a0 msgr2=0x7f93e40729a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:50.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:50.327+0000 7f93e1400640 1 --2- 172.21.15.131:0/3923859616 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f93e40725a0 0x7f93e40729a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:50.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:50.327+0000 7f93e1400640 1 -- 172.21.15.131:0/3923859616 >> v2:172.21.15.187:3300/0 conn(0x7f93e406dd30 msgr2=0x7f93e406e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:50.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:50.327+0000 7f93e1400640 1 --2- 172.21.15.131:0/3923859616 >> v2:172.21.15.187:3300/0 conn(0x7f93e406dd30 0x7f93e406e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:50.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:50.327+0000 7f93e1400640 1 -- 172.21.15.131:0/3923859616 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93e4072080 con 0x7f93e4155770 2024-05-26T22:56:50.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:50.327+0000 7f93e1400640 1 -- 172.21.15.131:0/3923859616 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7f93d4001670 con 0x7f93e4155770 2024-05-26T22:56:50.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:50.327+0000 7f93e1400640 1 -- 172.21.15.131:0/3923859616 >> v1:172.21.15.131:6789/0 conn(0x7f93e4155770 legacy=0x7f93e415b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:50.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:50.327+0000 7f93e1400640 1 --2- 172.21.15.131:0/3923859616 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f93e40725a0 0x7f93c8005bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:50.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:50.327+0000 7f93e1400640 1 --2- 172.21.15.131:0/3923859616 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f93c80061f0 0x7f93c8006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:50.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:50.327+0000 7f93e1400640 1 --2- 172.21.15.131:0/3923859616 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f93c8006f50 0x7f93c80073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:50.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:50.327+0000 7f93e1400640 1 -- 172.21.15.131:0/3923859616 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f93e415c4a0 con 0x7f93c8006f50 2024-05-26T22:56:50.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:50.327+0000 7f93e1400640 1 -- 172.21.15.131:0/3923859616 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f93e4071da0 con 0x7f93c80061f0 2024-05-26T22:56:50.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:50.327+0000 7f93e1400640 1 -- 172.21.15.131:0/3923859616 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f93c800dd60 con 0x7f93e40725a0 2024-05-26T22:56:50.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:50.328+0000 7f93e2800640 1 --2- 172.21.15.131:0/3923859616 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f93c8006f50 0x7f93c80073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:50.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:50.328+0000 7f93e1e00640 1 --2- 172.21.15.131:0/3923859616 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f93e40725a0 0x7f93c8005bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:50.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:50.328+0000 7f93dbe00640 1 --2- 172.21.15.131:0/3923859616 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f93c80061f0 0x7f93c8006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:50.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:50.328+0000 7f93e2800640 1 -- 172.21.15.131:0/3923859616 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f93e40725a0 msgr2=0x7f93c8005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:50.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:50.328+0000 7f93e2800640 1 --2- 172.21.15.131:0/3923859616 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f93e40725a0 0x7f93c8005bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:50.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:50.328+0000 7f93e2800640 1 -- 172.21.15.131:0/3923859616 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f93c80061f0 msgr2=0x7f93c8006a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:50.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:50.328+0000 7f93e2800640 1 --2- 172.21.15.131:0/3923859616 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f93c80061f0 0x7f93c8006a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:50.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:50.328+0000 7f93e2800640 1 -- 172.21.15.131:0/3923859616 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93c8005730 con 0x7f93c8006f50 2024-05-26T22:56:50.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:50.328+0000 7f93dbe00640 1 --2- 172.21.15.131:0/3923859616 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f93c80061f0 0x7f93c8006a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:56:50.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:50.329+0000 7f93e2800640 1 --2- 172.21.15.131:0/3923859616 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f93c8006f50 0x7f93c80073d0 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f93dc007360 tx=0x7f93c8007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=356d1198c1891b54 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:50.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:50.329+0000 7f93e1400640 1 -- 172.21.15.131:0/3923859616 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93dc009be0 con 0x7f93c8006f50 2024-05-26T22:56:50.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:50.329+0000 7f93e1400640 1 -- 172.21.15.131:0/3923859616 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93dc036600 con 0x7f93c8006f50 2024-05-26T22:56:50.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:50.329+0000 7f93e1400640 1 -- 172.21.15.131:0/3923859616 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93dc0369d0 con 0x7f93c8006f50 2024-05-26T22:56:50.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:50.330+0000 7f93e8c00640 1 -- 172.21.15.131:0/3923859616 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f93c8006f50 msgr2=0x7f93c80073d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:50.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:50.330+0000 7f93e8c00640 1 --2- 172.21.15.131:0/3923859616 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f93c8006f50 0x7f93c80073d0 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f93dc007360 tx=0x7f93c8007a40 comp rx=0 tx=0).stop 2024-05-26T22:56:50.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:50.330+0000 7f93e2800640 1 -- 172.21.15.131:0/3923859616 reap_dead start 2024-05-26T22:56:50.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:50.330+0000 7f93e8c00640 1 -- 172.21.15.131:0/3923859616 shutdown_connections 2024-05-26T22:56:50.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:50.330+0000 7f93e8c00640 1 -- 172.21.15.131:0/3923859616 >> 172.21.15.131:0/3923859616 conn(0x7f93e406bc90 msgr2=0x7f93e40b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:56:50.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:50.330+0000 7f93e8c00640 1 -- 172.21.15.131:0/3923859616 shutdown_connections 2024-05-26T22:56:50.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:50.330+0000 7f93e8c00640 1 -- 172.21.15.131:0/3923859616 wait complete. 2024-05-26T22:56:50.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:50.332+0000 7f93e8c00640 1 Processor -- start 2024-05-26T22:56:50.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:50.332+0000 7f93e8c00640 1 -- start start 2024-05-26T22:56:50.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:50.333+0000 7f93e8c00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f93c8006f50 0x7f93e4151550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:50.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:50.333+0000 7f93e8c00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f93e414bb20 0x7f93e41f80a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:50.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:50.333+0000 7f93e8c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f93e40ae070 0x7f93e40ae4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:50.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:50.333+0000 7f93e8c00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f93e415be10 con 0x7f93e40ae070 2024-05-26T22:56:50.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:50.333+0000 7f93e8c00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f93e415c8c0 con 0x7f93c8006f50 2024-05-26T22:56:50.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:50.333+0000 7f93e8c00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f93e409b8b0 con 0x7f93e414bb20 2024-05-26T22:56:50.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:50.333+0000 7f93e2800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f93e40ae070 0x7f93e40ae4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:50.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:50.333+0000 7f93e2800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f93e40ae070 0x7f93e40ae4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41870/0 (socket says 172.21.15.131:41870) 2024-05-26T22:56:50.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:50.333+0000 7f93e2800640 1 -- 172.21.15.131:0/2424387473 learned_addr learned my addr 172.21.15.131:0/2424387473 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:56:50.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:50.334+0000 7f93e1e00640 1 --2- 172.21.15.131:0/2424387473 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f93c8006f50 0x7f93e4151550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:50.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:50.334+0000 7f93e2800640 1 -- 172.21.15.131:0/2424387473 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f93e414bb20 msgr2=0x7f93e41f80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:50.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:50.334+0000 7f93dbe00640 1 --2- 172.21.15.131:0/2424387473 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f93e414bb20 0x7f93e41f80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:50.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:50.334+0000 7f93e2800640 1 --2- 172.21.15.131:0/2424387473 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f93e414bb20 0x7f93e41f80a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:50.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:50.334+0000 7f93e2800640 1 -- 172.21.15.131:0/2424387473 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f93c8006f50 msgr2=0x7f93e4151550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:50.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:50.334+0000 7f93e2800640 1 --2- 172.21.15.131:0/2424387473 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f93c8006f50 0x7f93e4151550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:50.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:50.334+0000 7f93e2800640 1 -- 172.21.15.131:0/2424387473 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93e41f85e0 con 0x7f93e40ae070 2024-05-26T22:56:50.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:50.334+0000 7f93e1e00640 1 --2- 172.21.15.131:0/2424387473 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f93c8006f50 0x7f93e4151550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:56:50.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:50.334+0000 7f93e2800640 1 --2- 172.21.15.131:0/2424387473 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f93e40ae070 0x7f93e40ae4f0 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f93dc007480 tx=0x7f93dc004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:50.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:50.334+0000 7f93da000640 1 -- 172.21.15.131:0/2424387473 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93dc045070 con 0x7f93e40ae070 2024-05-26T22:56:50.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:50.334+0000 7f93da000640 1 -- 172.21.15.131:0/2424387473 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93dc003990 con 0x7f93e40ae070 2024-05-26T22:56:50.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:50.334+0000 7f93da000640 1 -- 172.21.15.131:0/2424387473 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93dc009be0 con 0x7f93e40ae070 2024-05-26T22:56:50.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:50.334+0000 7f93e8c00640 1 -- 172.21.15.131:0/2424387473 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93e409aa00 con 0x7f93e40ae070 2024-05-26T22:56:50.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:50.335+0000 7f93e8c00640 1 -- 172.21.15.131:0/2424387473 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93e409af80 con 0x7f93e40ae070 2024-05-26T22:56:50.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:50.336+0000 7f93e8c00640 1 -- 172.21.15.131:0/2424387473 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93e40aea30 con 0x7f93e40ae070 2024-05-26T22:56:50.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:50.337+0000 7f93da000640 1 -- 172.21.15.131:0/2424387473 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f93dc048070 con 0x7f93e40ae070 2024-05-26T22:56:50.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:50.337+0000 7f93da000640 1 --2- 172.21.15.131:0/2424387473 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f93c006be30 0x7f93c006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:50.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:50.338+0000 7f93da000640 1 -- 172.21.15.131:0/2424387473 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5709+0+0 (secure 0 0 0) 0x7f93dc0ab2d0 con 0x7f93e40ae070 2024-05-26T22:56:50.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:50.338+0000 7f93e1e00640 1 --2- 172.21.15.131:0/2424387473 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f93c006be30 0x7f93c006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:50.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:50.338+0000 7f93e1e00640 1 --2- 172.21.15.131:0/2424387473 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f93c006be30 0x7f93c006e2f0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f93d40017e0 tx=0x7f93d4000ed0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:50.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:50.342+0000 7f93da000640 1 -- 172.21.15.131:0/2424387473 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f93e41f85e0 con 0x7f93e40ae070 2024-05-26T22:56:50.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:50.541+0000 7f93e8c00640 1 -- 172.21.15.131:0/2424387473 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f93e4074e90 con 0x7f93e40ae070 2024-05-26T22:56:50.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:50.541+0000 7f93da000640 1 -- 172.21.15.131:0/2424387473 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v107) ==== 80+0+0 (secure 0 0 0) 0x7f93dc03d070 con 0x7f93e40ae070 2024-05-26T22:56:50.541 INFO:teuthology.orchestra.run.smithi131.stderr:listed 0 entries 2024-05-26T22:56:50.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:50.544+0000 7f93e8c00640 1 -- 172.21.15.131:0/2424387473 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f93c006be30 msgr2=0x7f93c006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:50.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:50.544+0000 7f93e8c00640 1 --2- 172.21.15.131:0/2424387473 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f93c006be30 0x7f93c006e2f0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f93d40017e0 tx=0x7f93d4000ed0 comp rx=0 tx=0).stop 2024-05-26T22:56:50.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:50.544+0000 7f93e8c00640 1 -- 172.21.15.131:0/2424387473 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f93e40ae070 msgr2=0x7f93e40ae4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:50.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:50.544+0000 7f93e8c00640 1 --2- 172.21.15.131:0/2424387473 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f93e40ae070 0x7f93e40ae4f0 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f93dc007480 tx=0x7f93dc004ab0 comp rx=0 tx=0).stop 2024-05-26T22:56:50.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:50.544+0000 7f93e8c00640 1 -- 172.21.15.131:0/2424387473 shutdown_connections 2024-05-26T22:56:50.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:50.544+0000 7f93e8c00640 1 --2- 172.21.15.131:0/2424387473 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f93e414bb20 0x7f93e41f80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:50.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:50.544+0000 7f93e8c00640 1 --2- 172.21.15.131:0/2424387473 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f93c006be30 0x7f93c006e2f0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:50.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:50.544+0000 7f93e8c00640 1 --2- 172.21.15.131:0/2424387473 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f93e40ae070 0x7f93e40ae4f0 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:50.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:50.544+0000 7f93e8c00640 1 --2- 172.21.15.131:0/2424387473 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f93c8006f50 0x7f93e4151550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:50.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:50.544+0000 7f93e8c00640 1 -- 172.21.15.131:0/2424387473 >> 172.21.15.131:0/2424387473 conn(0x7f93e406bc90 msgr2=0x7f93e40b2810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:56:50.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:50.544+0000 7f93e8c00640 1 -- 172.21.15.131:0/2424387473 shutdown_connections 2024-05-26T22:56:50.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:50.544+0000 7f93e8c00640 1 -- 172.21.15.131:0/2424387473 wait complete. 2024-05-26T22:56:50.562 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-26T22:56:50.562 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:56:50.563 DEBUG:teuthology.orchestra.run.smithi131:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-26T22:56:50.701 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi131.front.sepia.ceph.com 2024-05-26T22:56:50.701 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:56:50.701 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-26T22:56:50.738 INFO:tasks.cephfs.mount:Terminating background process 2024-05-26T22:56:50.738 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-05-26T22:56:50.738 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-26T22:56:50.779 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-26T22:56:50.780 INFO:tasks.cephfs.mount:Terminating background process 2024-05-26T22:56:50.780 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-26T22:56:50.780 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-26T22:56:50.823 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-26T22:56:50.823 INFO:tasks.cephfs.mount:Terminating background process 2024-05-26T22:56:50.823 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-05-26T22:56:50.823 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-26T22:56:50.875 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-26T22:56:50.875 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-26T22:56:51.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.069+0000 7fe207e00640 1 Processor -- start 2024-05-26T22:56:51.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.069+0000 7fe207e00640 1 -- start start 2024-05-26T22:56:51.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.070+0000 7fe207e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe208144340 0x7fe2080cedf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:51.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.070+0000 7fe207e00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fe208071b70 0x7fe20806e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:51.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.071+0000 7fe207e00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe208071fd0 con 0x7fe20806e9f0 2024-05-26T22:56:51.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.071+0000 7fe207e00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fe20806ee60 con 0x7fe208071b70 2024-05-26T22:56:51.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.071+0000 7fe207e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe208072eb0 con 0x7fe208144340 2024-05-26T22:56:51.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.071+0000 7fe207400640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fe20806e9f0 0x7fe20815ec30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:48804/0 (socket says 172.21.15.131:48804) 2024-05-26T22:56:51.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.071+0000 7fe207400640 1 -- 172.21.15.131:0/1639710035 learned_addr learned my addr 172.21.15.131:0/1639710035 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:56:51.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.071+0000 7fe206a00640 1 --2- 172.21.15.131:0/1639710035 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe208144340 0x7fe2080cedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:51.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.071+0000 7fe1fe000640 1 --2- 172.21.15.131:0/1639710035 >> v2:172.21.15.187:3300/0 conn(0x7fe208071b70 0x7fe20806e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:51.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.071+0000 7fe206000640 1 -- 172.21.15.131:0/1639710035 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2982197581 0 0) 0x7fe208071fd0 con 0x7fe20806e9f0 2024-05-26T22:56:51.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.071+0000 7fe206000640 1 -- 172.21.15.131:0/1639710035 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe1ec003520 con 0x7fe20806e9f0 2024-05-26T22:56:51.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.072+0000 7fe206000640 1 -- 172.21.15.131:0/1639710035 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 235222583 0 0) 0x7fe1ec003520 con 0x7fe20806e9f0 2024-05-26T22:56:51.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.072+0000 7fe206000640 1 -- 172.21.15.131:0/1639710035 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe208144340 msgr2=0x7fe2080cedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:51.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.072+0000 7fe206000640 1 --2- 172.21.15.131:0/1639710035 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe208144340 0x7fe2080cedf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:51.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.072+0000 7fe206000640 1 -- 172.21.15.131:0/1639710035 >> v2:172.21.15.187:3300/0 conn(0x7fe208071b70 msgr2=0x7fe20806e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:51.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.072+0000 7fe206000640 1 --2- 172.21.15.131:0/1639710035 >> v2:172.21.15.187:3300/0 conn(0x7fe208071b70 0x7fe20806e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:51.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.072+0000 7fe206000640 1 -- 172.21.15.131:0/1639710035 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2080722a0 con 0x7fe20806e9f0 2024-05-26T22:56:51.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.072+0000 7fe206000640 1 -- 172.21.15.131:0/1639710035 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7fe2000019f0 con 0x7fe20806e9f0 2024-05-26T22:56:51.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.072+0000 7fe1fe000640 1 --2- 172.21.15.131:0/1639710035 >> v2:172.21.15.187:3300/0 conn(0x7fe208071b70 0x7fe20806e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-26T22:56:51.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.072+0000 7fe206000640 1 -- 172.21.15.131:0/1639710035 >> v1:172.21.15.131:6789/0 conn(0x7fe20806e9f0 legacy=0x7fe20815ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:51.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.072+0000 7fe206000640 1 --2- 172.21.15.131:0/1639710035 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe1ec0039b0 0x7fe1ec003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:51.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.072+0000 7fe206000640 1 --2- 172.21.15.131:0/1639710035 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe208144340 0x7fe1ec004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:51.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.072+0000 7fe206000640 1 --2- 172.21.15.131:0/1639710035 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe1ec004d40 0x7fe1ec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:51.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.072+0000 7fe206000640 1 -- 172.21.15.131:0/1639710035 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe20806ee60 con 0x7fe1ec0039b0 2024-05-26T22:56:51.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.072+0000 7fe206000640 1 -- 172.21.15.131:0/1639710035 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe1ec00b9c0 con 0x7fe1ec004d40 2024-05-26T22:56:51.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.072+0000 7fe206000640 1 -- 172.21.15.131:0/1639710035 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe1ec00bbc0 con 0x7fe208144340 2024-05-26T22:56:51.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.072+0000 7fe206a00640 1 --2- 172.21.15.131:0/1639710035 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe1ec0039b0 0x7fe1ec003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:51.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.073+0000 7fe206a00640 1 -- 172.21.15.131:0/1639710035 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe208144340 msgr2=0x7fe1ec004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:51.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.073+0000 7fe206a00640 1 --2- 172.21.15.131:0/1639710035 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe208144340 0x7fe1ec004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:51.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.073+0000 7fe206a00640 1 -- 172.21.15.131:0/1639710035 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe1ec004d40 msgr2=0x7fe1ec0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:51.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.073+0000 7fe206a00640 1 --2- 172.21.15.131:0/1639710035 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe1ec004d40 0x7fe1ec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:51.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.073+0000 7fe206a00640 1 -- 172.21.15.131:0/1639710035 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1ec003520 con 0x7fe1ec0039b0 2024-05-26T22:56:51.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.074+0000 7fe206a00640 1 --2- 172.21.15.131:0/1639710035 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe1ec0039b0 0x7fe1ec003dd0 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7fe1f8009ac0 tx=0x7fe1ec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1afc02d0c27fb95 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:51.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.074+0000 7fe206000640 1 -- 172.21.15.131:0/1639710035 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1f8036070 con 0x7fe1ec0039b0 2024-05-26T22:56:51.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.074+0000 7fe206000640 1 -- 172.21.15.131:0/1639710035 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1f8031400 con 0x7fe1ec0039b0 2024-05-26T22:56:51.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.074+0000 7fe206000640 1 -- 172.21.15.131:0/1639710035 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1f8009690 con 0x7fe1ec0039b0 2024-05-26T22:56:51.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.074+0000 7fe207e00640 1 -- 172.21.15.131:0/1639710035 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe1ec0039b0 msgr2=0x7fe1ec003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:51.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.074+0000 7fe207e00640 1 --2- 172.21.15.131:0/1639710035 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe1ec0039b0 0x7fe1ec003dd0 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7fe1f8009ac0 tx=0x7fe1ec005830 comp rx=0 tx=0).stop 2024-05-26T22:56:51.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.075+0000 7fe207400640 1 -- 172.21.15.131:0/1639710035 reap_dead start 2024-05-26T22:56:51.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.075+0000 7fe207e00640 1 -- 172.21.15.131:0/1639710035 shutdown_connections 2024-05-26T22:56:51.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.075+0000 7fe207e00640 1 -- 172.21.15.131:0/1639710035 >> 172.21.15.131:0/1639710035 conn(0x7fe2080cff40 msgr2=0x7fe2080d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:56:51.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.075+0000 7fe207e00640 1 -- 172.21.15.131:0/1639710035 shutdown_connections 2024-05-26T22:56:51.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.075+0000 7fe207e00640 1 -- 172.21.15.131:0/1639710035 wait complete. 2024-05-26T22:56:51.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.076+0000 7fe207e00640 1 Processor -- start 2024-05-26T22:56:51.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.077+0000 7fe207e00640 1 -- start start 2024-05-26T22:56:51.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.077+0000 7fe207e00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe1ec0039b0 0x7fe20813f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:51.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.077+0000 7fe207e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe20815a8c0 0x7fe208159470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:51.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.077+0000 7fe207e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe208143c90 0x7fe20815b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:51.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.077+0000 7fe207e00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe208072b40 con 0x7fe208143c90 2024-05-26T22:56:51.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.077+0000 7fe207e00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe208072860 con 0x7fe1ec0039b0 2024-05-26T22:56:51.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.077+0000 7fe207e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe2080b4d30 con 0x7fe20815a8c0 2024-05-26T22:56:51.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.078+0000 7fe207400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe208143c90 0x7fe20815b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:51.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.078+0000 7fe207400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe208143c90 0x7fe20815b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41888/0 (socket says 172.21.15.131:41888) 2024-05-26T22:56:51.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.078+0000 7fe207400640 1 -- 172.21.15.131:0/3225856126 learned_addr learned my addr 172.21.15.131:0/3225856126 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:56:51.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.078+0000 7fe207400640 1 -- 172.21.15.131:0/3225856126 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe20815a8c0 msgr2=0x7fe208159470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:51.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.078+0000 7fe207400640 1 --2- 172.21.15.131:0/3225856126 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe20815a8c0 0x7fe208159470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:51.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.078+0000 7fe206a00640 1 --2- 172.21.15.131:0/3225856126 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe1ec0039b0 0x7fe20813f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:51.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.078+0000 7fe207400640 1 -- 172.21.15.131:0/3225856126 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe1ec0039b0 msgr2=0x7fe20813f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:51.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.078+0000 7fe207400640 1 --2- 172.21.15.131:0/3225856126 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe1ec0039b0 0x7fe20813f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:51.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.078+0000 7fe207400640 1 -- 172.21.15.131:0/3225856126 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe20813e750 con 0x7fe208143c90 2024-05-26T22:56:51.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.078+0000 7fe206a00640 1 --2- 172.21.15.131:0/3225856126 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe1ec0039b0 0x7fe20813f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:56:51.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.078+0000 7fe207400640 1 --2- 172.21.15.131:0/3225856126 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe208143c90 0x7fe20815b6a0 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7fe200001560 tx=0x7fe20000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:51.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.079+0000 7fe1ff400640 1 -- 172.21.15.131:0/3225856126 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe200007110 con 0x7fe208143c90 2024-05-26T22:56:51.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.079+0000 7fe1ff400640 1 -- 172.21.15.131:0/3225856126 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2000072b0 con 0x7fe208143c90 2024-05-26T22:56:51.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.079+0000 7fe1ff400640 1 -- 172.21.15.131:0/3225856126 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe200007680 con 0x7fe208143c90 2024-05-26T22:56:51.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.079+0000 7fe207e00640 1 -- 172.21.15.131:0/3225856126 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe20813ea30 con 0x7fe208143c90 2024-05-26T22:56:51.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.079+0000 7fe207e00640 1 -- 172.21.15.131:0/3225856126 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe20813fa50 con 0x7fe208143c90 2024-05-26T22:56:51.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.080+0000 7fe207e00640 1 -- 172.21.15.131:0/3225856126 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2080694e0 con 0x7fe208143c90 2024-05-26T22:56:51.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.081+0000 7fe1ff400640 1 -- 172.21.15.131:0/3225856126 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe200011070 con 0x7fe208143c90 2024-05-26T22:56:51.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.082+0000 7fe1ff400640 1 --2- 172.21.15.131:0/3225856126 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fe1e406be30 0x7fe1e406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:51.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.082+0000 7fe1ff400640 1 -- 172.21.15.131:0/3225856126 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5709+0+0 (secure 0 0 0) 0x7fe20008a470 con 0x7fe208143c90 2024-05-26T22:56:51.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.082+0000 7fe206a00640 1 --2- 172.21.15.131:0/3225856126 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fe1e406be30 0x7fe1e406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:51.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.083+0000 7fe206a00640 1 --2- 172.21.15.131:0/3225856126 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fe1e406be30 0x7fe1e406e2f0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fe1f8033000 tx=0x7fe1f8008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:51.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.087+0000 7fe1ff400640 1 -- 172.21.15.131:0/3225856126 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fe200058e10 con 0x7fe208143c90 2024-05-26T22:56:51.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.310+0000 7fe207e00640 1 -- 172.21.15.131:0/3225856126 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe208074830 con 0x7fe208143c90 2024-05-26T22:56:51.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.311+0000 7fe1ff400640 1 -- 172.21.15.131:0/3225856126 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 118 v118) ==== 95+0+5244 (secure 0 0 0) 0x7fe200058710 con 0x7fe208143c90 2024-05-26T22:56:51.311 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:56:51.311 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":118,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5959,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/2949813640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":2949813640},{"type":"v1","addr":"172.21.15.187:6833","nonce":2949813640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104}],"filesystems":[{"mdsmap":{"epoch":118,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-26T22:56:01.980221+0000","modified":"2024-05-26T22:56:40.084341+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8310,"mds_1":8307,"mds_2":5968},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5968":{"gid":5968,"name":"d","rank":2,"incarnation":114,"state":"up:active","state_seq":7,"addr":"172.21.15.187:6835/2594604065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2594604065},{"type":"v1","addr":"172.21.15.187:6835","nonce":2594604065}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8307":{"gid":8307,"name":"a","rank":1,"incarnation":112,"state":"up:active","state_seq":8,"addr":"172.21.15.131:6836/2710427205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":2710427205},{"type":"v1","addr":"172.21.15.131:6836","nonce":2710427205}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8310":{"gid":8310,"name":"c","rank":0,"incarnation":106,"state":"up:active","state_seq":5,"addr":"172.21.15.131:6837/2725520331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6835","nonce":2725520331},{"type":"v1","addr":"172.21.15.131:6837","nonce":2725520331}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8310,"qdb_cluster":[8310,8307,5968]},"id":8}]} 2024-05-26T22:56:51.311 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 118 2024-05-26T22:56:51.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.314+0000 7fe207e00640 1 -- 172.21.15.131:0/3225856126 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fe1e406be30 msgr2=0x7fe1e406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:51.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.314+0000 7fe207e00640 1 --2- 172.21.15.131:0/3225856126 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fe1e406be30 0x7fe1e406e2f0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fe1f8033000 tx=0x7fe1f8008030 comp rx=0 tx=0).stop 2024-05-26T22:56:51.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.314+0000 7fe207e00640 1 -- 172.21.15.131:0/3225856126 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe208143c90 msgr2=0x7fe20815b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:51.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.314+0000 7fe207e00640 1 --2- 172.21.15.131:0/3225856126 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe208143c90 0x7fe20815b6a0 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7fe200001560 tx=0x7fe20000cc60 comp rx=0 tx=0).stop 2024-05-26T22:56:51.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.314+0000 7fe207e00640 1 -- 172.21.15.131:0/3225856126 shutdown_connections 2024-05-26T22:56:51.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.314+0000 7fe207e00640 1 --2- 172.21.15.131:0/3225856126 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe20815a8c0 0x7fe208159470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:51.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.314+0000 7fe207e00640 1 --2- 172.21.15.131:0/3225856126 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fe1e406be30 0x7fe1e406e2f0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:51.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.314+0000 7fe207e00640 1 --2- 172.21.15.131:0/3225856126 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe208143c90 0x7fe20815b6a0 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:51.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.314+0000 7fe207e00640 1 --2- 172.21.15.131:0/3225856126 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe1ec0039b0 0x7fe20813f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:51.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.314+0000 7fe207e00640 1 -- 172.21.15.131:0/3225856126 >> 172.21.15.131:0/3225856126 conn(0x7fe2080cff40 msgr2=0x7fe2080cedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:56:51.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.314+0000 7fe207e00640 1 -- 172.21.15.131:0/3225856126 shutdown_connections 2024-05-26T22:56:51.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.314+0000 7fe207e00640 1 -- 172.21.15.131:0/3225856126 wait complete. 2024-05-26T22:56:51.332 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-26T22:56:51.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.521+0000 7f8bfb000640 1 Processor -- start 2024-05-26T22:56:51.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.521+0000 7f8bfb000640 1 -- start start 2024-05-26T22:56:51.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.523+0000 7f8bfb000640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f8bf4156750 0x7f8bf415fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:51.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.523+0000 7f8bfb000640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8bf4160620 con 0x7f8bf406e8c0 2024-05-26T22:56:51.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.523+0000 7f8bfb000640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8bf4160900 con 0x7f8bf40cc7f0 2024-05-26T22:56:51.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.523+0000 7f8bfb000640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f8bf415d770 con 0x7f8bf4156750 2024-05-26T22:56:51.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.523+0000 7f8bf9200640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f8bf406e8c0 0x7f8bf415d060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:48818/0 (socket says 172.21.15.131:48818) 2024-05-26T22:56:51.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.523+0000 7f8bf9200640 1 -- 172.21.15.131:0/2998684805 learned_addr learned my addr 172.21.15.131:0/2998684805 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:56:51.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.524+0000 7f8bf3e00640 1 -- 172.21.15.131:0/2998684805 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 113776235 0 0) 0x7f8bf4160620 con 0x7f8bf406e8c0 2024-05-26T22:56:51.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.524+0000 7f8bf3e00640 1 -- 172.21.15.131:0/2998684805 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8bdc003520 con 0x7f8bf406e8c0 2024-05-26T22:56:51.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.524+0000 7f8bf9c00640 1 --2- 172.21.15.131:0/2998684805 >> v2:172.21.15.187:3300/0 conn(0x7f8bf4156750 0x7f8bf415fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:51.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.524+0000 7f8bf3e00640 1 -- 172.21.15.131:0/2998684805 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1307988129 0 0) 0x7f8bdc003520 con 0x7f8bf406e8c0 2024-05-26T22:56:51.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.524+0000 7f8bf3e00640 1 -- 172.21.15.131:0/2998684805 >> v2:172.21.15.187:3300/0 conn(0x7f8bf4156750 msgr2=0x7f8bf415fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:51.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.524+0000 7f8bf3e00640 1 --2- 172.21.15.131:0/2998684805 >> v2:172.21.15.187:3300/0 conn(0x7f8bf4156750 0x7f8bf415fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:51.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.524+0000 7f8bf3e00640 1 -- 172.21.15.131:0/2998684805 >> v1:172.21.15.187:6789/0 conn(0x7f8bf40cc7f0 legacy=0x7f8bf40cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:51.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.524+0000 7f8bf3e00640 1 -- 172.21.15.131:0/2998684805 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bf4160be0 con 0x7f8bf406e8c0 2024-05-26T22:56:51.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.524+0000 7f8bf3e00640 1 -- 172.21.15.131:0/2998684805 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7f8be0001670 con 0x7f8bf406e8c0 2024-05-26T22:56:51.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.524+0000 7f8bf9c00640 1 --2- 172.21.15.131:0/2998684805 >> v2:172.21.15.187:3300/0 conn(0x7f8bf4156750 0x7f8bf415fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-26T22:56:51.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.524+0000 7f8bf3e00640 1 -- 172.21.15.131:0/2998684805 >> v1:172.21.15.131:6789/0 conn(0x7f8bf406e8c0 legacy=0x7f8bf415d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:51.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.524+0000 7f8bf3e00640 1 --2- 172.21.15.131:0/2998684805 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8bdc0039b0 0x7f8bdc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:51.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.524+0000 7f8bf3e00640 1 --2- 172.21.15.131:0/2998684805 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8bdc004800 0x7f8bdc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:51.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.524+0000 7f8bf3e00640 1 --2- 172.21.15.131:0/2998684805 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8bdc0051c0 0x7f8bdc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:51.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.524+0000 7f8bf3e00640 1 -- 172.21.15.131:0/2998684805 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf415d770 con 0x7f8bdc004800 2024-05-26T22:56:51.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.524+0000 7f8bf3e00640 1 -- 172.21.15.131:0/2998684805 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8bdc00c110 con 0x7f8bdc0039b0 2024-05-26T22:56:51.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.524+0000 7f8bf3e00640 1 -- 172.21.15.131:0/2998684805 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f8bdc00c310 con 0x7f8bdc0051c0 2024-05-26T22:56:51.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.525+0000 7f8bf9200640 1 --2- 172.21.15.131:0/2998684805 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8bdc004800 0x7f8bdc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:51.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.525+0000 7f8bf9c00640 1 --2- 172.21.15.131:0/2998684805 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8bdc0039b0 0x7f8bdc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:51.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.525+0000 7f8bfa600640 1 --2- 172.21.15.131:0/2998684805 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8bdc0051c0 0x7f8bdc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:51.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.525+0000 7f8bf9200640 1 -- 172.21.15.131:0/2998684805 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8bdc0051c0 msgr2=0x7f8bdc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:51.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.525+0000 7f8bf9200640 1 --2- 172.21.15.131:0/2998684805 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8bdc0051c0 0x7f8bdc00b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:51.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.525+0000 7f8bf9200640 1 -- 172.21.15.131:0/2998684805 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8bdc0039b0 msgr2=0x7f8bdc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:51.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.525+0000 7f8bf9200640 1 --2- 172.21.15.131:0/2998684805 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8bdc0039b0 0x7f8bdc003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:51.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.525+0000 7f8bf9200640 1 -- 172.21.15.131:0/2998684805 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bdc003520 con 0x7f8bdc004800 2024-05-26T22:56:51.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.525+0000 7f8bfa600640 1 -- 172.21.15.131:0/2998684805 reap_dead start 2024-05-26T22:56:51.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.526+0000 7f8bf9200640 1 --2- 172.21.15.131:0/2998684805 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8bdc004800 0x7f8bdc004c80 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f8be000c240 tx=0x7f8bdc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eab7d751ec56f69a server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:51.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.526+0000 7f8bf3e00640 1 -- 172.21.15.131:0/2998684805 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8be0007cf0 con 0x7f8bdc004800 2024-05-26T22:56:51.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.526+0000 7f8bf3e00640 1 -- 172.21.15.131:0/2998684805 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8be00058d0 con 0x7f8bdc004800 2024-05-26T22:56:51.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.526+0000 7f8bf3e00640 1 -- 172.21.15.131:0/2998684805 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8be0005d20 con 0x7f8bdc004800 2024-05-26T22:56:51.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.527+0000 7f8bfb000640 1 -- 172.21.15.131:0/2998684805 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8bdc004800 msgr2=0x7f8bdc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:51.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.527+0000 7f8bfb000640 1 --2- 172.21.15.131:0/2998684805 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8bdc004800 0x7f8bdc004c80 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f8be000c240 tx=0x7f8bdc005830 comp rx=0 tx=0).stop 2024-05-26T22:56:51.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.527+0000 7f8bfb000640 1 -- 172.21.15.131:0/2998684805 shutdown_connections 2024-05-26T22:56:51.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.527+0000 7f8bfb000640 1 --2- 172.21.15.131:0/2998684805 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8bdc004800 0x7f8bdc004c80 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:51.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.527+0000 7f8bfb000640 1 -- 172.21.15.131:0/2998684805 >> 172.21.15.131:0/2998684805 conn(0x7f8bf40cff40 msgr2=0x7f8bf40d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:56:51.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.527+0000 7f8bfb000640 1 -- 172.21.15.131:0/2998684805 shutdown_connections 2024-05-26T22:56:51.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.527+0000 7f8bfb000640 1 -- 172.21.15.131:0/2998684805 wait complete. 2024-05-26T22:56:51.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.529+0000 7f8bfb000640 1 Processor -- start 2024-05-26T22:56:51.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.529+0000 7f8bfb000640 1 -- start start 2024-05-26T22:56:51.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.530+0000 7f8bfb000640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8bdc004800 0x7f8bf41f4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:51.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.530+0000 7f8bfb000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8bf415b0c0 0x7f8bf41f7d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:51.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.530+0000 7f8bfb000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8bf4143330 0x7f8bf415ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:51.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.530+0000 7f8bfb000640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf4156b30 con 0x7f8bf4143330 2024-05-26T22:56:51.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.530+0000 7f8bfb000640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf415cbd0 con 0x7f8bdc004800 2024-05-26T22:56:51.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.530+0000 7f8bfb000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f8bf40ccc60 con 0x7f8bf415b0c0 2024-05-26T22:56:51.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.530+0000 7f8bfa600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8bf4143330 0x7f8bf415ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:51.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.530+0000 7f8bfa600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8bf4143330 0x7f8bf415ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41914/0 (socket says 172.21.15.131:41914) 2024-05-26T22:56:51.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.530+0000 7f8bfa600640 1 -- 172.21.15.131:0/1572311821 learned_addr learned my addr 172.21.15.131:0/1572311821 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:56:51.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.531+0000 7f8bf9c00640 1 --2- 172.21.15.131:0/1572311821 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8bdc004800 0x7f8bf41f4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:51.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.531+0000 7f8bfa600640 1 -- 172.21.15.131:0/1572311821 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8bf415b0c0 msgr2=0x7f8bf41f7d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:51.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.531+0000 7f8bfa600640 1 --2- 172.21.15.131:0/1572311821 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8bf415b0c0 0x7f8bf41f7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:51.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.531+0000 7f8bfa600640 1 -- 172.21.15.131:0/1572311821 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8bdc004800 msgr2=0x7f8bf41f4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:51.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.531+0000 7f8bfa600640 1 --2- 172.21.15.131:0/1572311821 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8bdc004800 0x7f8bf41f4790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:51.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.531+0000 7f8bfa600640 1 -- 172.21.15.131:0/1572311821 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bf40b1d70 con 0x7f8bf4143330 2024-05-26T22:56:51.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.531+0000 7f8bfa600640 1 --2- 172.21.15.131:0/1572311821 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8bf4143330 0x7f8bf415ea70 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f8bf40707f0 tx=0x7f8bec00d9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:51.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.531+0000 7f8bf1600640 1 -- 172.21.15.131:0/1572311821 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bec003d80 con 0x7f8bf4143330 2024-05-26T22:56:51.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.531+0000 7f8bf1600640 1 -- 172.21.15.131:0/1572311821 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bec017070 con 0x7f8bf4143330 2024-05-26T22:56:51.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.531+0000 7f8bfb000640 1 -- 172.21.15.131:0/1572311821 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bf41f8470 con 0x7f8bf4143330 2024-05-26T22:56:51.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.532+0000 7f8bf1600640 1 -- 172.21.15.131:0/1572311821 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bec0052b0 con 0x7f8bf4143330 2024-05-26T22:56:51.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.532+0000 7f8bfb000640 1 -- 172.21.15.131:0/1572311821 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bf41f8a30 con 0x7f8bf4143330 2024-05-26T22:56:51.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.533+0000 7f8bfb000640 1 -- 172.21.15.131:0/1572311821 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bf40694e0 con 0x7f8bf4143330 2024-05-26T22:56:51.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.534+0000 7f8bf1600640 1 -- 172.21.15.131:0/1572311821 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8bec005450 con 0x7f8bf4143330 2024-05-26T22:56:51.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.535+0000 7f8bf1600640 1 --2- 172.21.15.131:0/1572311821 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f8bd406be60 0x7f8bd406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:51.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.535+0000 7f8bf1600640 1 -- 172.21.15.131:0/1572311821 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5709+0+0 (secure 0 0 0) 0x7f8bec08abd0 con 0x7f8bf4143330 2024-05-26T22:56:51.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.535+0000 7f8bf9c00640 1 --2- 172.21.15.131:0/1572311821 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f8bd406be60 0x7f8bd406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:51.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.535+0000 7f8bf9c00640 1 --2- 172.21.15.131:0/1572311821 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f8bd406be60 0x7f8bd406e320 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f8bf41e8700 tx=0x7f8be8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:51.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.539+0000 7f8bf1600640 1 -- 172.21.15.131:0/1572311821 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f8bec058e90 con 0x7f8bf4143330 2024-05-26T22:56:51.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.743+0000 7f8bfb000640 1 -- 172.21.15.131:0/1572311821 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8bf415be80 con 0x7f8bf4143330 2024-05-26T22:56:51.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.746+0000 7f8bf1600640 1 -- 172.21.15.131:0/1572311821 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v107) ==== 74+0+16651 (secure 0 0 0) 0x7f8bec05d190 con 0x7f8bf4143330 2024-05-26T22:56:51.746 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:56:51.746 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":107,"fsid":"46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f","created":"2024-05-26T22:47:57.288905+0000","modified":"2024-05-26T22:56:03.175526+0000","last_up_change":"2024-05-26T22:48:09.602207+0000","last_in_change":"2024-05-26T22:48:00.687341+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-26T22:48:10.463339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-05-26T22:55:59.775225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-05-26T22:56:00.988905+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"814eaf62-a18d-494b-8e27-fd8fb88adea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6801","nonce":2350940051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6803","nonce":2350940051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6807","nonce":2350940051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6805","nonce":2350940051}]},"public_addr":"172.21.15.131:6801/2350940051","cluster_addr":"172.21.15.131:6803/2350940051","heartbeat_back_addr":"172.21.15.131:6807/2350940051","heartbeat_front_addr":"172.21.15.131:6805/2350940051","state":["exists","up"]},{"osd":1,"uuid":"42e66287-f6c6-4e26-af3c-6dfaa3536d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6825","nonce":1525528247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6827","nonce":1525528247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6831","nonce":1525528247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6829","nonce":1525528247}]},"public_addr":"172.21.15.131:6825/1525528247","cluster_addr":"172.21.15.131:6827/1525528247","heartbeat_back_addr":"172.21.15.131:6831/1525528247","heartbeat_front_addr":"172.21.15.131:6829/1525528247","state":["exists","up"]},{"osd":2,"uuid":"0da34673-04a2-4fd0-b57a-0ff2b679b043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6817","nonce":3627842210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6819","nonce":3627842210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6823","nonce":3627842210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6821","nonce":3627842210}]},"public_addr":"172.21.15.131:6817/3627842210","cluster_addr":"172.21.15.131:6819/3627842210","heartbeat_back_addr":"172.21.15.131:6823/3627842210","heartbeat_front_addr":"172.21.15.131:6821/3627842210","state":["exists","up"]},{"osd":3,"uuid":"cf117a27-d6b2-43d3-ae52-b47988bf8eed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6809","nonce":350790591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6811","nonce":350790591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6815","nonce":350790591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6813","nonce":350790591}]},"public_addr":"172.21.15.131:6809/350790591","cluster_addr":"172.21.15.131:6811/350790591","heartbeat_back_addr":"172.21.15.131:6815/350790591","heartbeat_front_addr":"172.21.15.131:6813/350790591","state":["exists","up"]},{"osd":4,"uuid":"a092ccfc-be89-4a84-88d5-0f781fa715eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6816","nonce":244560995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6819","nonce":244560995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6823","nonce":244560995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6821","nonce":244560995}]},"public_addr":"172.21.15.187:6816/244560995","cluster_addr":"172.21.15.187:6819/244560995","heartbeat_back_addr":"172.21.15.187:6823/244560995","heartbeat_front_addr":"172.21.15.187:6821/244560995","state":["exists","up"]},{"osd":5,"uuid":"7513ba69-dd00-4a35-b190-9e8cc3b2bdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6801","nonce":324619398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6803","nonce":324619398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6812","nonce":324619398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6807","nonce":324619398}]},"public_addr":"172.21.15.187:6801/324619398","cluster_addr":"172.21.15.187:6803/324619398","heartbeat_back_addr":"172.21.15.187:6812/324619398","heartbeat_front_addr":"172.21.15.187:6807/324619398","state":["exists","up"]},{"osd":6,"uuid":"50081810-c093-4d2f-a4d7-b7c695c5400d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6806","nonce":1391347786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6809","nonce":1391347786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6818","nonce":1391347786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6813","nonce":1391347786}]},"public_addr":"172.21.15.187:6806/1391347786","cluster_addr":"172.21.15.187:6809/1391347786","heartbeat_back_addr":"172.21.15.187:6818/1391347786","heartbeat_front_addr":"172.21.15.187:6813/1391347786","state":["exists","up"]},{"osd":7,"uuid":"775dae05-54f5-4ad4-bbce-92863c6ed2ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6825","nonce":1220065622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6827","nonce":1220065622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6831","nonce":1220065622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6829","nonce":1220065622}]},"public_addr":"172.21.15.187:6825/1220065622","cluster_addr":"172.21.15.187:6827/1220065622","heartbeat_back_addr":"172.21.15.187:6831/1220065622","heartbeat_front_addr":"172.21.15.187:6829/1220065622","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:05.983520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.299516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.067323+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.307748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.018742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.258775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.010912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.295355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-26T22:56:51.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.749+0000 7f8bfb000640 1 -- 172.21.15.131:0/1572311821 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f8bd406be60 msgr2=0x7f8bd406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:51.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.749+0000 7f8bfb000640 1 --2- 172.21.15.131:0/1572311821 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f8bd406be60 0x7f8bd406e320 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f8bf41e8700 tx=0x7f8be8008040 comp rx=0 tx=0).stop 2024-05-26T22:56:51.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.749+0000 7f8bfb000640 1 -- 172.21.15.131:0/1572311821 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8bf4143330 msgr2=0x7f8bf415ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:51.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.749+0000 7f8bfb000640 1 --2- 172.21.15.131:0/1572311821 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8bf4143330 0x7f8bf415ea70 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f8bf40707f0 tx=0x7f8bec00d9d0 comp rx=0 tx=0).stop 2024-05-26T22:56:51.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.749+0000 7f8bfb000640 1 -- 172.21.15.131:0/1572311821 shutdown_connections 2024-05-26T22:56:51.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.749+0000 7f8bfb000640 1 --2- 172.21.15.131:0/1572311821 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8bf415b0c0 0x7f8bf41f7d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:51.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.749+0000 7f8bfb000640 1 --2- 172.21.15.131:0/1572311821 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f8bd406be60 0x7f8bd406e320 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:51.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.749+0000 7f8bfb000640 1 --2- 172.21.15.131:0/1572311821 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8bf4143330 0x7f8bf415ea70 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:51.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.749+0000 7f8bfb000640 1 --2- 172.21.15.131:0/1572311821 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8bdc004800 0x7f8bf41f4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:51.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.749+0000 7f8bfb000640 1 -- 172.21.15.131:0/1572311821 >> 172.21.15.131:0/1572311821 conn(0x7f8bf40cff40 msgr2=0x7f8bf415f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:56:51.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.749+0000 7f8bfb000640 1 -- 172.21.15.131:0/1572311821 shutdown_connections 2024-05-26T22:56:51.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.749+0000 7f8bfb000640 1 -- 172.21.15.131:0/1572311821 wait complete. 2024-05-26T22:56:51.768 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-26T22:48:10.463339+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-26T22:55:59.775225+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '106', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-05-26T22:56:00.988905+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '107', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-26T22:56:51.769 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-26T22:56:51.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.975+0000 7feb15000640 1 Processor -- start 2024-05-26T22:56:51.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.975+0000 7feb15000640 1 -- start start 2024-05-26T22:56:51.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.977+0000 7feb15000640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7feb1006d250 0x7feb1006d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:51.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.977+0000 7feb15000640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7feb1015b730 0x7feb1015bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:51.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.977+0000 7feb15000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb10071f90 0x7feb100723c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:51.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.977+0000 7feb15000640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7feb10071260 con 0x7feb1006d250 2024-05-26T22:56:51.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.977+0000 7feb15000640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7feb1006db70 con 0x7feb1015b730 2024-05-26T22:56:51.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.977+0000 7feb15000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7feb1013e820 con 0x7feb10071f90 2024-05-26T22:56:51.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.977+0000 7feb0f400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7feb1006d250 0x7feb1006d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:51.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.977+0000 7feb0f400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7feb1006d250 0x7feb1006d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41926/0 (socket says 172.21.15.131:41926) 2024-05-26T22:56:51.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.977+0000 7feb0f400640 1 -- 172.21.15.131:0/2068088477 learned_addr learned my addr 172.21.15.131:0/2068088477 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:56:51.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.977+0000 7feb06a00640 1 --2- 172.21.15.131:0/2068088477 >> v2:172.21.15.187:3300/0 conn(0x7feb1015b730 0x7feb1015bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:51.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.978+0000 7feb0fe00640 1 --2- 172.21.15.131:0/2068088477 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb10071f90 0x7feb100723c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:51.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.978+0000 7feb0f400640 1 -- 172.21.15.131:0/2068088477 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb10071f90 msgr2=0x7feb100723c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:51.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.978+0000 7feb0f400640 1 --2- 172.21.15.131:0/2068088477 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb10071f90 0x7feb100723c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:51.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.978+0000 7feb0f400640 1 -- 172.21.15.131:0/2068088477 >> v2:172.21.15.187:3300/0 conn(0x7feb1015b730 msgr2=0x7feb1015bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:51.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.978+0000 7feb0f400640 1 --2- 172.21.15.131:0/2068088477 >> v2:172.21.15.187:3300/0 conn(0x7feb1015b730 0x7feb1015bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:51.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.978+0000 7feb0f400640 1 -- 172.21.15.131:0/2068088477 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb1015c4a0 con 0x7feb1006d250 2024-05-26T22:56:51.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.979+0000 7feb0f400640 1 --2- 172.21.15.131:0/2068088477 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb1006d250 0x7feb1006d630 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7feb1015b650 tx=0x7feb0002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e935a14ddb17c25 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:51.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.979+0000 7feb0ea00640 1 -- 172.21.15.131:0/2068088477 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb00037070 con 0x7feb1006d250 2024-05-26T22:56:51.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.979+0000 7feb0ea00640 1 -- 172.21.15.131:0/2068088477 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb00032440 con 0x7feb1006d250 2024-05-26T22:56:51.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.979+0000 7feb0ea00640 1 -- 172.21.15.131:0/2068088477 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb00032810 con 0x7feb1006d250 2024-05-26T22:56:51.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.980+0000 7feb15000640 1 -- 172.21.15.131:0/2068088477 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb1006d250 msgr2=0x7feb1006d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:51.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.980+0000 7feb15000640 1 --2- 172.21.15.131:0/2068088477 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb1006d250 0x7feb1006d630 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7feb1015b650 tx=0x7feb0002a1e0 comp rx=0 tx=0).stop 2024-05-26T22:56:51.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.980+0000 7feb15000640 1 -- 172.21.15.131:0/2068088477 shutdown_connections 2024-05-26T22:56:51.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.980+0000 7feb15000640 1 --2- 172.21.15.131:0/2068088477 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb10071f90 0x7feb100723c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:51.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.980+0000 7feb15000640 1 --2- 172.21.15.131:0/2068088477 >> v2:172.21.15.187:3300/0 conn(0x7feb1015b730 0x7feb1015bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:51.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.980+0000 7feb15000640 1 --2- 172.21.15.131:0/2068088477 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb1006d250 0x7feb1006d630 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:51.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.980+0000 7feb15000640 1 -- 172.21.15.131:0/2068088477 >> 172.21.15.131:0/2068088477 conn(0x7feb100b5660 msgr2=0x7feb100b5a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:56:51.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.980+0000 7feb15000640 1 -- 172.21.15.131:0/2068088477 shutdown_connections 2024-05-26T22:56:51.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.980+0000 7feb15000640 1 -- 172.21.15.131:0/2068088477 wait complete. 2024-05-26T22:56:51.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.982+0000 7feb15000640 1 Processor -- start 2024-05-26T22:56:51.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.982+0000 7feb15000640 1 -- start start 2024-05-26T22:56:51.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.983+0000 7feb15000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb1006d250 0x7feb1013d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:51.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.983+0000 7feb15000640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feb10071f90 0x7feb1015e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:51.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.983+0000 7feb15000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb1015b730 0x7feb10152270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:51.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.983+0000 7feb15000640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7feb10071a70 con 0x7feb1006d250 2024-05-26T22:56:51.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.983+0000 7feb15000640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7feb10071790 con 0x7feb10071f90 2024-05-26T22:56:51.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.983+0000 7feb15000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7feb1015c100 con 0x7feb1015b730 2024-05-26T22:56:51.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.983+0000 7feb0f400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb1006d250 0x7feb1013d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:51.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.983+0000 7feb0f400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb1006d250 0x7feb1013d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41936/0 (socket says 172.21.15.131:41936) 2024-05-26T22:56:51.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.983+0000 7feb0f400640 1 -- 172.21.15.131:0/2765009952 learned_addr learned my addr 172.21.15.131:0/2765009952 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:56:51.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.984+0000 7feb0f400640 1 -- 172.21.15.131:0/2765009952 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb1015b730 msgr2=0x7feb10152270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:51.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.984+0000 7feb0f400640 1 --2- 172.21.15.131:0/2765009952 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb1015b730 0x7feb10152270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:51.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.984+0000 7feb06a00640 1 --2- 172.21.15.131:0/2765009952 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feb10071f90 0x7feb1015e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:51.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.984+0000 7feb0f400640 1 -- 172.21.15.131:0/2765009952 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feb10071f90 msgr2=0x7feb1015e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:51.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.984+0000 7feb0f400640 1 --2- 172.21.15.131:0/2765009952 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feb10071f90 0x7feb1015e0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:51.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.984+0000 7feb0f400640 1 -- 172.21.15.131:0/2765009952 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb100cdcf0 con 0x7feb1006d250 2024-05-26T22:56:51.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.984+0000 7feb0f400640 1 --2- 172.21.15.131:0/2765009952 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb1006d250 0x7feb1013d2b0 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7feb0002a220 tx=0x7feb000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:51.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.984+0000 7feb07e00640 1 -- 172.21.15.131:0/2765009952 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb00041020 con 0x7feb1006d250 2024-05-26T22:56:51.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.985+0000 7feb07e00640 1 -- 172.21.15.131:0/2765009952 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb000306c0 con 0x7feb1006d250 2024-05-26T22:56:51.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.985+0000 7feb07e00640 1 -- 172.21.15.131:0/2765009952 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb00037040 con 0x7feb1006d250 2024-05-26T22:56:51.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.985+0000 7feb15000640 1 -- 172.21.15.131:0/2765009952 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb100cdfd0 con 0x7feb1006d250 2024-05-26T22:56:51.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.985+0000 7feb15000640 1 -- 172.21.15.131:0/2765009952 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb101f3e20 con 0x7feb1006d250 2024-05-26T22:56:51.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.986+0000 7feb15000640 1 -- 172.21.15.131:0/2765009952 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb100694e0 con 0x7feb1006d250 2024-05-26T22:56:51.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.987+0000 7feb07e00640 1 -- 172.21.15.131:0/2765009952 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7feb00044020 con 0x7feb1006d250 2024-05-26T22:56:51.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.988+0000 7feb07e00640 1 --2- 172.21.15.131:0/2765009952 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7feaec06be60 0x7feaec06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:51.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.988+0000 7feb07e00640 1 -- 172.21.15.131:0/2765009952 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5709+0+0 (secure 0 0 0) 0x7feb000ab5f0 con 0x7feb1006d250 2024-05-26T22:56:51.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.988+0000 7feb06a00640 1 --2- 172.21.15.131:0/2765009952 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7feaec06be60 0x7feaec06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:51.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.989+0000 7feb06a00640 1 --2- 172.21.15.131:0/2765009952 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7feaec06be60 0x7feaec06e320 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7feaf8006fd0 tx=0x7feaf8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:51.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:51.993+0000 7feb07e00640 1 -- 172.21.15.131:0/2765009952 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7feb0004b030 con 0x7feb1006d250 2024-05-26T22:56:52.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.192+0000 7feb15000640 1 -- 172.21.15.131:0/2765009952 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7feb100d0a70 con 0x7feb1006d250 2024-05-26T22:56:52.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.193+0000 7feb07e00640 1 -- 172.21.15.131:0/2765009952 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v107) ==== 93+0+10 (secure 0 0 0) 0x7feb00079f90 con 0x7feb1006d250 2024-05-26T22:56:52.193 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2024-05-26T22:56:52.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.196+0000 7feb15000640 1 -- 172.21.15.131:0/2765009952 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7feaec06be60 msgr2=0x7feaec06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:52.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.196+0000 7feb15000640 1 --2- 172.21.15.131:0/2765009952 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7feaec06be60 0x7feaec06e320 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7feaf8006fd0 tx=0x7feaf8008040 comp rx=0 tx=0).stop 2024-05-26T22:56:52.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.196+0000 7feb15000640 1 -- 172.21.15.131:0/2765009952 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb1006d250 msgr2=0x7feb1013d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:52.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.196+0000 7feb15000640 1 --2- 172.21.15.131:0/2765009952 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb1006d250 0x7feb1013d2b0 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7feb0002a220 tx=0x7feb000046c0 comp rx=0 tx=0).stop 2024-05-26T22:56:52.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.196+0000 7feb15000640 1 -- 172.21.15.131:0/2765009952 shutdown_connections 2024-05-26T22:56:52.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.196+0000 7feb15000640 1 --2- 172.21.15.131:0/2765009952 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feb1015b730 0x7feb10152270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:52.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.196+0000 7feb15000640 1 --2- 172.21.15.131:0/2765009952 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7feaec06be60 0x7feaec06e320 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:52.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.196+0000 7feb15000640 1 --2- 172.21.15.131:0/2765009952 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feb10071f90 0x7feb1015e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:52.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.196+0000 7feb15000640 1 --2- 172.21.15.131:0/2765009952 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb1006d250 0x7feb1013d2b0 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:52.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.196+0000 7feb15000640 1 -- 172.21.15.131:0/2765009952 >> 172.21.15.131:0/2765009952 conn(0x7feb100b5660 msgr2=0x7feb10154cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:56:52.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.196+0000 7feb15000640 1 -- 172.21.15.131:0/2765009952 shutdown_connections 2024-05-26T22:56:52.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.196+0000 7feb15000640 1 -- 172.21.15.131:0/2765009952 wait complete. 2024-05-26T22:56:52.215 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-26T22:56:52.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.418+0000 7f9a2ae00640 1 Processor -- start 2024-05-26T22:56:52.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.419+0000 7f9a2ae00640 1 -- start start 2024-05-26T22:56:52.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.421+0000 7f9a2ae00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f9a2406f420 0x7f9a240cfd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:52.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.421+0000 7f9a2ae00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a24160180 con 0x7f9a240b5450 2024-05-26T22:56:52.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.421+0000 7f9a2ae00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a24160460 con 0x7f9a240cf590 2024-05-26T22:56:52.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.421+0000 7f9a2ae00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f9a24160d90 con 0x7f9a2406f420 2024-05-26T22:56:52.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.421+0000 7f9a23400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f9a2406f420 0x7f9a240cfd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:52.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.421+0000 7f9a23e00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f9a240b5450 0x7f9a240b43c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:48820/0 (socket says 172.21.15.131:48820) 2024-05-26T22:56:52.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.421+0000 7f9a23e00640 1 -- 172.21.15.131:0/951876551 learned_addr learned my addr 172.21.15.131:0/951876551 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:56:52.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.422+0000 7f9a22a00640 1 -- 172.21.15.131:0/951876551 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1551920701 0 0) 0x7f9a24160180 con 0x7f9a240b5450 2024-05-26T22:56:52.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.422+0000 7f9a23400640 1 -- 172.21.15.131:0/951876551 >> v1:172.21.15.187:6789/0 conn(0x7f9a240cf590 legacy=0x7f9a2406d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:52.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.422+0000 7f9a23400640 1 -- 172.21.15.131:0/951876551 >> v1:172.21.15.131:6789/0 conn(0x7f9a240b5450 legacy=0x7f9a240b43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:52.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.422+0000 7f9a23400640 1 -- 172.21.15.131:0/951876551 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a24160740 con 0x7f9a2406f420 2024-05-26T22:56:52.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.423+0000 7f9a23400640 1 --2- 172.21.15.131:0/951876551 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9a2406f420 0x7f9a240cfd50 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f9a100097a0 tx=0x7f9a1002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53805d95d3002b9c server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:52.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.423+0000 7f9a22a00640 1 -- 172.21.15.131:0/951876551 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a10037070 con 0x7f9a2406f420 2024-05-26T22:56:52.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.423+0000 7f9a22a00640 1 -- 172.21.15.131:0/951876551 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a10032440 con 0x7f9a2406f420 2024-05-26T22:56:52.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.423+0000 7f9a22a00640 1 -- 172.21.15.131:0/951876551 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a10032810 con 0x7f9a2406f420 2024-05-26T22:56:52.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.424+0000 7f9a2ae00640 1 -- 172.21.15.131:0/951876551 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9a2406f420 msgr2=0x7f9a240cfd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:52.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.424+0000 7f9a2ae00640 1 --2- 172.21.15.131:0/951876551 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9a2406f420 0x7f9a240cfd50 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f9a100097a0 tx=0x7f9a1002a1e0 comp rx=0 tx=0).stop 2024-05-26T22:56:52.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.424+0000 7f9a2ae00640 1 -- 172.21.15.131:0/951876551 shutdown_connections 2024-05-26T22:56:52.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.424+0000 7f9a2ae00640 1 --2- 172.21.15.131:0/951876551 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9a2406f420 0x7f9a240cfd50 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:52.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.424+0000 7f9a2ae00640 1 -- 172.21.15.131:0/951876551 >> 172.21.15.131:0/951876551 conn(0x7f9a2406bc90 msgr2=0x7f9a240b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:56:52.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.424+0000 7f9a2ae00640 1 -- 172.21.15.131:0/951876551 shutdown_connections 2024-05-26T22:56:52.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.424+0000 7f9a2ae00640 1 -- 172.21.15.131:0/951876551 wait complete. 2024-05-26T22:56:52.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.426+0000 7f9a2ae00640 1 Processor -- start 2024-05-26T22:56:52.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.426+0000 7f9a2ae00640 1 -- start start 2024-05-26T22:56:52.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.427+0000 7f9a2ae00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9a2406f420 0x7f9a2409bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:52.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.427+0000 7f9a2ae00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9a240b5450 0x7f9a24153340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:52.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.427+0000 7f9a2ae00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9a240cf590 0x7f9a24157220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:52.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.427+0000 7f9a2ae00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f9a2415fde0 con 0x7f9a240b5450 2024-05-26T22:56:52.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.427+0000 7f9a2ae00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9a240b5830 con 0x7f9a2406f420 2024-05-26T22:56:52.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.427+0000 7f9a2ae00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9a2415ff60 con 0x7f9a240cf590 2024-05-26T22:56:52.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.427+0000 7f9a23400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9a240b5450 0x7f9a24153340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:52.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.427+0000 7f9a23400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9a240b5450 0x7f9a24153340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41950/0 (socket says 172.21.15.131:41950) 2024-05-26T22:56:52.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.427+0000 7f9a23400640 1 -- 172.21.15.131:0/2093694516 learned_addr learned my addr 172.21.15.131:0/2093694516 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:56:52.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.427+0000 7f9a23400640 1 -- 172.21.15.131:0/2093694516 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9a240cf590 msgr2=0x7f9a24157220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:52.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.427+0000 7f9a23400640 1 --2- 172.21.15.131:0/2093694516 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9a240cf590 0x7f9a24157220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:52.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.427+0000 7f9a23400640 1 -- 172.21.15.131:0/2093694516 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9a2406f420 msgr2=0x7f9a2409bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:52.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.427+0000 7f9a23e00640 1 --2- 172.21.15.131:0/2093694516 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9a2406f420 0x7f9a2409bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:52.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.427+0000 7f9a23400640 1 --2- 172.21.15.131:0/2093694516 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9a2406f420 0x7f9a2409bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:52.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.428+0000 7f9a23400640 1 -- 172.21.15.131:0/2093694516 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a24154980 con 0x7f9a240b5450 2024-05-26T22:56:52.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.428+0000 7f9a23e00640 1 --2- 172.21.15.131:0/2093694516 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9a2406f420 0x7f9a2409bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:56:52.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.428+0000 7f9a23400640 1 --2- 172.21.15.131:0/2093694516 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9a240b5450 0x7f9a24153340 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f9a10000c00 tx=0x7f9a100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:52.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.428+0000 7f9a1be00640 1 -- 172.21.15.131:0/2093694516 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a10037040 con 0x7f9a240b5450 2024-05-26T22:56:52.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.428+0000 7f9a1be00640 1 -- 172.21.15.131:0/2093694516 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a10042070 con 0x7f9a240b5450 2024-05-26T22:56:52.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.428+0000 7f9a2ae00640 1 -- 172.21.15.131:0/2093694516 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a2409c010 con 0x7f9a240b5450 2024-05-26T22:56:52.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.428+0000 7f9a1be00640 1 -- 172.21.15.131:0/2093694516 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a10030a70 con 0x7f9a240b5450 2024-05-26T22:56:52.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.428+0000 7f9a2ae00640 1 -- 172.21.15.131:0/2093694516 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a24154f00 con 0x7f9a240b5450 2024-05-26T22:56:52.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.430+0000 7f9a2ae00640 1 -- 172.21.15.131:0/2093694516 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a24074e90 con 0x7f9a240b5450 2024-05-26T22:56:52.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.430+0000 7f9a1be00640 1 -- 172.21.15.131:0/2093694516 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9a10030c10 con 0x7f9a240b5450 2024-05-26T22:56:52.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.431+0000 7f9a1be00640 1 --2- 172.21.15.131:0/2093694516 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9a0406be30 0x7f9a0406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:52.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.431+0000 7f9a1be00640 1 -- 172.21.15.131:0/2093694516 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5709+0+0 (secure 0 0 0) 0x7f9a100ab9e0 con 0x7f9a240b5450 2024-05-26T22:56:52.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.432+0000 7f9a23e00640 1 --2- 172.21.15.131:0/2093694516 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9a0406be30 0x7f9a0406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:52.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.433+0000 7f9a23e00640 1 --2- 172.21.15.131:0/2093694516 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9a0406be30 0x7f9a0406e2f0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f9a14000f60 tx=0x7f9a14009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:52.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.437+0000 7f9a1be00640 1 -- 172.21.15.131:0/2093694516 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f9a1007a2f0 con 0x7f9a240b5450 2024-05-26T22:56:52.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.636+0000 7f9a2ae00640 1 -- 172.21.15.131:0/2093694516 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f9a2415f0d0 con 0x7f9a240b5450 2024-05-26T22:56:52.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.637+0000 7f9a1be00640 1 -- 172.21.15.131:0/2093694516 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v107) ==== 104+0+11 (secure 0 0 0) 0x7f9a10079bf0 con 0x7f9a240b5450 2024-05-26T22:56:52.637 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-05-26T22:56:52.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.639+0000 7f9a2ae00640 1 -- 172.21.15.131:0/2093694516 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9a0406be30 msgr2=0x7f9a0406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:52.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.639+0000 7f9a2ae00640 1 --2- 172.21.15.131:0/2093694516 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9a0406be30 0x7f9a0406e2f0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f9a14000f60 tx=0x7f9a14009040 comp rx=0 tx=0).stop 2024-05-26T22:56:52.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.639+0000 7f9a2ae00640 1 -- 172.21.15.131:0/2093694516 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9a240b5450 msgr2=0x7f9a24153340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:52.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.639+0000 7f9a2ae00640 1 --2- 172.21.15.131:0/2093694516 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9a240b5450 0x7f9a24153340 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f9a10000c00 tx=0x7f9a100046c0 comp rx=0 tx=0).stop 2024-05-26T22:56:52.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.639+0000 7f9a2ae00640 1 -- 172.21.15.131:0/2093694516 shutdown_connections 2024-05-26T22:56:52.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.639+0000 7f9a2ae00640 1 --2- 172.21.15.131:0/2093694516 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9a240cf590 0x7f9a24157220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:52.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.639+0000 7f9a2ae00640 1 --2- 172.21.15.131:0/2093694516 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9a0406be30 0x7f9a0406e2f0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:52.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.639+0000 7f9a2ae00640 1 --2- 172.21.15.131:0/2093694516 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9a240b5450 0x7f9a24153340 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:52.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.639+0000 7f9a2ae00640 1 --2- 172.21.15.131:0/2093694516 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9a2406f420 0x7f9a2409bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:52.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.640+0000 7f9a2ae00640 1 -- 172.21.15.131:0/2093694516 >> 172.21.15.131:0/2093694516 conn(0x7f9a2406bc90 msgr2=0x7f9a2415e8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:56:52.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.640+0000 7f9a2ae00640 1 -- 172.21.15.131:0/2093694516 shutdown_connections 2024-05-26T22:56:52.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.640+0000 7f9a2ae00640 1 -- 172.21.15.131:0/2093694516 wait complete. 2024-05-26T22:56:52.657 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-26T22:56:52.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.862+0000 7fb372200640 1 Processor -- start 2024-05-26T22:56:52.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.862+0000 7fb372200640 1 -- start start 2024-05-26T22:56:52.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.864+0000 7fb372200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fb36c06f420 0x7fb36c0cfd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:52.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.864+0000 7fb372200640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fb36c0cf590 0x7fb36c0d0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:52.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.864+0000 7fb372200640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb36c160180 con 0x7fb36c0b5450 2024-05-26T22:56:52.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.864+0000 7fb372200640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fb36c160d90 con 0x7fb36c06f420 2024-05-26T22:56:52.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.864+0000 7fb372200640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fb36c160460 con 0x7fb36c0cf590 2024-05-26T22:56:52.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.864+0000 7fb36b400640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fb36c0b5450 0x7fb36c0b43c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:48826/0 (socket says 172.21.15.131:48826) 2024-05-26T22:56:52.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.864+0000 7fb36aa00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fb36c06f420 0x7fb36c0cfd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:52.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.864+0000 7fb36b400640 1 -- 172.21.15.131:0/4281670935 learned_addr learned my addr 172.21.15.131:0/4281670935 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:56:52.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.865+0000 7fb36be00640 1 --2- 172.21.15.131:0/4281670935 >> v2:172.21.15.187:3300/0 conn(0x7fb36c0cf590 0x7fb36c0d0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:52.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.865+0000 7fb36aa00640 1 -- 172.21.15.131:0/4281670935 >> v2:172.21.15.187:3300/0 conn(0x7fb36c0cf590 msgr2=0x7fb36c0d0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:52.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.865+0000 7fb36aa00640 1 --2- 172.21.15.131:0/4281670935 >> v2:172.21.15.187:3300/0 conn(0x7fb36c0cf590 0x7fb36c0d0290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:52.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.865+0000 7fb36aa00640 1 -- 172.21.15.131:0/4281670935 >> v1:172.21.15.131:6789/0 conn(0x7fb36c0b5450 legacy=0x7fb36c0b43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:52.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.865+0000 7fb36aa00640 1 -- 172.21.15.131:0/4281670935 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb36c160740 con 0x7fb36c06f420 2024-05-26T22:56:52.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.865+0000 7fb36be00640 1 --2- 172.21.15.131:0/4281670935 >> v2:172.21.15.187:3300/0 conn(0x7fb36c0cf590 0x7fb36c0d0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:56:52.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.865+0000 7fb36a000640 1 -- 172.21.15.131:0/4281670935 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1063339168 0 0) 0x7fb36c160180 con 0x7fb36c0b5450 2024-05-26T22:56:52.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.866+0000 7fb36aa00640 1 --2- 172.21.15.131:0/4281670935 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb36c06f420 0x7fb36c0cfd50 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7fb36c0b4ad0 tx=0x7fb35402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a38a248776a29a9c server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:52.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.866+0000 7fb36a000640 1 -- 172.21.15.131:0/4281670935 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb354037070 con 0x7fb36c06f420 2024-05-26T22:56:52.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.866+0000 7fb36a000640 1 -- 172.21.15.131:0/4281670935 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb354032440 con 0x7fb36c06f420 2024-05-26T22:56:52.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.866+0000 7fb36a000640 1 -- 172.21.15.131:0/4281670935 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb354032810 con 0x7fb36c06f420 2024-05-26T22:56:52.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.867+0000 7fb372200640 1 -- 172.21.15.131:0/4281670935 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb36c06f420 msgr2=0x7fb36c0cfd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:52.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.867+0000 7fb372200640 1 --2- 172.21.15.131:0/4281670935 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb36c06f420 0x7fb36c0cfd50 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7fb36c0b4ad0 tx=0x7fb35402a1a0 comp rx=0 tx=0).stop 2024-05-26T22:56:52.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.867+0000 7fb372200640 1 -- 172.21.15.131:0/4281670935 shutdown_connections 2024-05-26T22:56:52.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.867+0000 7fb372200640 1 --2- 172.21.15.131:0/4281670935 >> v2:172.21.15.187:3300/0 conn(0x7fb36c0cf590 0x7fb36c0d0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:52.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.867+0000 7fb372200640 1 --2- 172.21.15.131:0/4281670935 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb36c06f420 0x7fb36c0cfd50 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:52.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.867+0000 7fb372200640 1 -- 172.21.15.131:0/4281670935 >> 172.21.15.131:0/4281670935 conn(0x7fb36c06bc90 msgr2=0x7fb36c0b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:56:52.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.867+0000 7fb372200640 1 -- 172.21.15.131:0/4281670935 shutdown_connections 2024-05-26T22:56:52.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.867+0000 7fb372200640 1 -- 172.21.15.131:0/4281670935 wait complete. 2024-05-26T22:56:52.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.869+0000 7fb372200640 1 Processor -- start 2024-05-26T22:56:52.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.869+0000 7fb372200640 1 -- start start 2024-05-26T22:56:52.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.870+0000 7fb372200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb36c06f420 0x7fb36c151af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:52.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.870+0000 7fb372200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb36c0b5450 0x7fb36c09cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:52.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.870+0000 7fb372200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb36c0cf590 0x7fb36c072960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:52.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.870+0000 7fb372200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fb36c15fde0 con 0x7fb36c06f420 2024-05-26T22:56:52.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.870+0000 7fb372200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb36c0b5830 con 0x7fb36c0cf590 2024-05-26T22:56:52.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.870+0000 7fb372200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb36c15ff60 con 0x7fb36c0b5450 2024-05-26T22:56:52.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.870+0000 7fb36b400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb36c06f420 0x7fb36c151af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:52.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.870+0000 7fb36b400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb36c06f420 0x7fb36c151af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41972/0 (socket says 172.21.15.131:41972) 2024-05-26T22:56:52.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.870+0000 7fb36b400640 1 -- 172.21.15.131:0/2805730989 learned_addr learned my addr 172.21.15.131:0/2805730989 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:56:52.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.870+0000 7fb36b400640 1 -- 172.21.15.131:0/2805730989 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb36c0b5450 msgr2=0x7fb36c09cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:52.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.870+0000 7fb36b400640 1 --2- 172.21.15.131:0/2805730989 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb36c0b5450 0x7fb36c09cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:52.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.870+0000 7fb36b400640 1 -- 172.21.15.131:0/2805730989 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb36c0cf590 msgr2=0x7fb36c072960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:52.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.870+0000 7fb36b400640 1 --2- 172.21.15.131:0/2805730989 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb36c0cf590 0x7fb36c072960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:52.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.870+0000 7fb36b400640 1 -- 172.21.15.131:0/2805730989 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb36c1f4f10 con 0x7fb36c06f420 2024-05-26T22:56:52.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.871+0000 7fb36b400640 1 --2- 172.21.15.131:0/2805730989 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb36c06f420 0x7fb36c151af0 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7fb35c00c4d0 tx=0x7fb35c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:52.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.871+0000 7fb363400640 1 -- 172.21.15.131:0/2805730989 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb35c018070 con 0x7fb36c06f420 2024-05-26T22:56:52.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.871+0000 7fb363400640 1 -- 172.21.15.131:0/2805730989 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb35c007500 con 0x7fb36c06f420 2024-05-26T22:56:52.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.871+0000 7fb372200640 1 -- 172.21.15.131:0/2805730989 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb36c1f18f0 con 0x7fb36c06f420 2024-05-26T22:56:52.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.871+0000 7fb363400640 1 -- 172.21.15.131:0/2805730989 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb35c004a20 con 0x7fb36c06f420 2024-05-26T22:56:52.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.871+0000 7fb372200640 1 -- 172.21.15.131:0/2805730989 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb36c1f5490 con 0x7fb36c06f420 2024-05-26T22:56:52.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.872+0000 7fb372200640 1 -- 172.21.15.131:0/2805730989 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb36c074e90 con 0x7fb36c06f420 2024-05-26T22:56:52.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.874+0000 7fb363400640 1 -- 172.21.15.131:0/2805730989 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb35c01b070 con 0x7fb36c06f420 2024-05-26T22:56:52.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.874+0000 7fb363400640 1 --2- 172.21.15.131:0/2805730989 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb34806be30 0x7fb34806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:52.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.875+0000 7fb363400640 1 -- 172.21.15.131:0/2805730989 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5709+0+0 (secure 0 0 0) 0x7fb35c08a280 con 0x7fb36c06f420 2024-05-26T22:56:52.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.875+0000 7fb36aa00640 1 --2- 172.21.15.131:0/2805730989 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb34806be30 0x7fb34806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:52.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.875+0000 7fb36aa00640 1 --2- 172.21.15.131:0/2805730989 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb34806be30 0x7fb34806e2f0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fb354034000 tx=0x7fb35402c000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:52.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:52.879+0000 7fb363400640 1 -- 172.21.15.131:0/2805730989 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fb35c058b90 con 0x7fb36c06f420 2024-05-26T22:56:53.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.073+0000 7fb372200640 1 -- 172.21.15.131:0/2805730989 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb36c0cee90 con 0x7fb36c06f420 2024-05-26T22:56:53.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.074+0000 7fb363400640 1 -- 172.21.15.131:0/2805730989 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v107) ==== 100+0+11 (secure 0 0 0) 0x7fb35c058490 con 0x7fb36c06f420 2024-05-26T22:56:53.073 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 64 2024-05-26T22:56:53.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.076+0000 7fb372200640 1 -- 172.21.15.131:0/2805730989 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb34806be30 msgr2=0x7fb34806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:53.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.076+0000 7fb372200640 1 --2- 172.21.15.131:0/2805730989 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb34806be30 0x7fb34806e2f0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fb354034000 tx=0x7fb35402c000 comp rx=0 tx=0).stop 2024-05-26T22:56:53.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.076+0000 7fb372200640 1 -- 172.21.15.131:0/2805730989 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb36c06f420 msgr2=0x7fb36c151af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:53.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.076+0000 7fb372200640 1 --2- 172.21.15.131:0/2805730989 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb36c06f420 0x7fb36c151af0 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7fb35c00c4d0 tx=0x7fb35c00c990 comp rx=0 tx=0).stop 2024-05-26T22:56:53.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.077+0000 7fb372200640 1 -- 172.21.15.131:0/2805730989 shutdown_connections 2024-05-26T22:56:53.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.077+0000 7fb372200640 1 --2- 172.21.15.131:0/2805730989 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb36c0b5450 0x7fb36c09cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:53.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.077+0000 7fb372200640 1 --2- 172.21.15.131:0/2805730989 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb34806be30 0x7fb34806e2f0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:53.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.077+0000 7fb372200640 1 --2- 172.21.15.131:0/2805730989 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb36c0cf590 0x7fb36c072960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:53.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.077+0000 7fb372200640 1 --2- 172.21.15.131:0/2805730989 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb36c06f420 0x7fb36c151af0 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:53.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.077+0000 7fb372200640 1 -- 172.21.15.131:0/2805730989 >> 172.21.15.131:0/2805730989 conn(0x7fb36c06bc90 msgr2=0x7fb36c15ba80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:56:53.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.077+0000 7fb372200640 1 -- 172.21.15.131:0/2805730989 shutdown_connections 2024-05-26T22:56:53.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.077+0000 7fb372200640 1 -- 172.21.15.131:0/2805730989 wait complete. 2024-05-26T22:56:53.094 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-26T22:56:53.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.287+0000 7fcafec00640 1 Processor -- start 2024-05-26T22:56:53.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.287+0000 7fcafec00640 1 -- start start 2024-05-26T22:56:53.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.288+0000 7fcafec00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fcaf815e320 0x7fcaf815e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:53.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.288+0000 7fcafec00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcaf8150f70 0x7fcaf81513c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:53.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.288+0000 7fcafec00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcaf815ec40 con 0x7fcaf8071500 2024-05-26T22:56:53.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.288+0000 7fcafec00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fcaf80cfbe0 con 0x7fcaf815e320 2024-05-26T22:56:53.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.288+0000 7fcafec00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fcaf80ceb40 con 0x7fcaf8150f70 2024-05-26T22:56:53.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.289+0000 7fcafe200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcaf8150f70 0x7fcaf81513c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:53.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.289+0000 7fcafe200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcaf8150f70 0x7fcaf81513c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.131:52332/0 (socket says 172.21.15.131:52332) 2024-05-26T22:56:53.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.289+0000 7fcafe200640 1 -- 172.21.15.131:0/763466963 learned_addr learned my addr 172.21.15.131:0/763466963 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:56:53.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.289+0000 7fcafce00640 1 --1- 172.21.15.131:0/763466963 >> v1:172.21.15.187:6789/0 conn(0x7fcaf8071500 0x7fcaf80ce2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.131:35426/0 (socket says 172.21.15.131:35426) 2024-05-26T22:56:53.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.289+0000 7fcafd800640 1 --2- 172.21.15.131:0/763466963 >> v2:172.21.15.187:3300/0 conn(0x7fcaf815e320 0x7fcaf815e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:53.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.290+0000 7fcafd800640 1 -- 172.21.15.131:0/763466963 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcaf8150f70 msgr2=0x7fcaf81513c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:53.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.290+0000 7fcafd800640 1 --2- 172.21.15.131:0/763466963 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcaf8150f70 0x7fcaf81513c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:53.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.290+0000 7fcafd800640 1 -- 172.21.15.131:0/763466963 >> v1:172.21.15.187:6789/0 conn(0x7fcaf8071500 legacy=0x7fcaf80ce2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:53.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.290+0000 7fcafd800640 1 -- 172.21.15.131:0/763466963 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaf80cee20 con 0x7fcaf815e320 2024-05-26T22:56:53.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.290+0000 7fcafe200640 1 --2- 172.21.15.131:0/763466963 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcaf8150f70 0x7fcaf81513c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-26T22:56:53.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.292+0000 7fcafd800640 1 --2- 172.21.15.131:0/763466963 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcaf815e320 0x7fcaf815e700 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fcaec0097a0 tx=0x7fcaec02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a849ba6bdfb7e108 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:53.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.293+0000 7fcaf7e00640 1 -- 172.21.15.131:0/763466963 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaec037070 con 0x7fcaf815e320 2024-05-26T22:56:53.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.293+0000 7fcaf7e00640 1 -- 172.21.15.131:0/763466963 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcaec032440 con 0x7fcaf815e320 2024-05-26T22:56:53.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.293+0000 7fcaf7e00640 1 -- 172.21.15.131:0/763466963 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaec032810 con 0x7fcaf815e320 2024-05-26T22:56:53.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.293+0000 7fcafec00640 1 -- 172.21.15.131:0/763466963 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcaf815e320 msgr2=0x7fcaf815e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:53.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.293+0000 7fcafec00640 1 --2- 172.21.15.131:0/763466963 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcaf815e320 0x7fcaf815e700 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fcaec0097a0 tx=0x7fcaec02a180 comp rx=0 tx=0).stop 2024-05-26T22:56:53.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.293+0000 7fcafec00640 1 -- 172.21.15.131:0/763466963 shutdown_connections 2024-05-26T22:56:53.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.293+0000 7fcafec00640 1 --2- 172.21.15.131:0/763466963 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcaf8150f70 0x7fcaf81513c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:53.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.293+0000 7fcafec00640 1 --2- 172.21.15.131:0/763466963 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcaf815e320 0x7fcaf815e700 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:53.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.293+0000 7fcafec00640 1 -- 172.21.15.131:0/763466963 >> 172.21.15.131:0/763466963 conn(0x7fcaf80cff40 msgr2=0x7fcaf80d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:56:53.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.294+0000 7fcafec00640 1 -- 172.21.15.131:0/763466963 shutdown_connections 2024-05-26T22:56:53.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.294+0000 7fcafec00640 1 -- 172.21.15.131:0/763466963 wait complete. 2024-05-26T22:56:53.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.295+0000 7fcafec00640 1 Processor -- start 2024-05-26T22:56:53.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.295+0000 7fcafec00640 1 -- start start 2024-05-26T22:56:53.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.296+0000 7fcafec00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcaf8071500 0x7fcaf814ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:53.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.296+0000 7fcafec00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcaf8150f70 0x7fcaf81e4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:53.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.296+0000 7fcafec00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcaf815e320 0x7fcaf80b22b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:53.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.296+0000 7fcafec00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf80cf6c0 con 0x7fcaf8071500 2024-05-26T22:56:53.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.296+0000 7fcafec00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf80cf3e0 con 0x7fcaf8150f70 2024-05-26T22:56:53.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.296+0000 7fcafec00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fcaf8072580 con 0x7fcaf815e320 2024-05-26T22:56:53.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.297+0000 7fcafd800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcaf8071500 0x7fcaf814ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:53.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.297+0000 7fcafd800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcaf8071500 0x7fcaf814ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41982/0 (socket says 172.21.15.131:41982) 2024-05-26T22:56:53.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.297+0000 7fcafd800640 1 -- 172.21.15.131:0/2113991505 learned_addr learned my addr 172.21.15.131:0/2113991505 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:56:53.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.297+0000 7fcafd800640 1 -- 172.21.15.131:0/2113991505 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcaf815e320 msgr2=0x7fcaf80b22b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:53.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.297+0000 7fcafd800640 1 --2- 172.21.15.131:0/2113991505 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcaf815e320 0x7fcaf80b22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:53.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.297+0000 7fcafd800640 1 -- 172.21.15.131:0/2113991505 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcaf8150f70 msgr2=0x7fcaf81e4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:53.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.297+0000 7fcafd800640 1 --2- 172.21.15.131:0/2113991505 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcaf8150f70 0x7fcaf81e4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:53.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.297+0000 7fcafd800640 1 -- 172.21.15.131:0/2113991505 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaf81e5480 con 0x7fcaf8071500 2024-05-26T22:56:53.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.297+0000 7fcafd800640 1 --2- 172.21.15.131:0/2113991505 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcaf8071500 0x7fcaf814ec50 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7fcaec000c00 tx=0x7fcaec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:53.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.298+0000 7fcaf5600640 1 -- 172.21.15.131:0/2113991505 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaec037040 con 0x7fcaf8071500 2024-05-26T22:56:53.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.298+0000 7fcaf5600640 1 -- 172.21.15.131:0/2113991505 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcaec042070 con 0x7fcaf8071500 2024-05-26T22:56:53.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.298+0000 7fcaf5600640 1 -- 172.21.15.131:0/2113991505 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaec030940 con 0x7fcaf8071500 2024-05-26T22:56:53.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.298+0000 7fcafec00640 1 -- 172.21.15.131:0/2113991505 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcaf81e5760 con 0x7fcaf8071500 2024-05-26T22:56:53.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.298+0000 7fcafec00640 1 -- 172.21.15.131:0/2113991505 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcaf81514b0 con 0x7fcaf8071500 2024-05-26T22:56:53.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.299+0000 7fcafec00640 1 -- 172.21.15.131:0/2113991505 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcaf80694e0 con 0x7fcaf8071500 2024-05-26T22:56:53.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.300+0000 7fcaf5600640 1 -- 172.21.15.131:0/2113991505 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcaec030ae0 con 0x7fcaf8071500 2024-05-26T22:56:53.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.301+0000 7fcaf5600640 1 --2- 172.21.15.131:0/2113991505 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fcad806be30 0x7fcad806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:53.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.301+0000 7fcaf5600640 1 -- 172.21.15.131:0/2113991505 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5709+0+0 (secure 0 0 0) 0x7fcaec0ab710 con 0x7fcaf8071500 2024-05-26T22:56:53.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.301+0000 7fcafce00640 1 --2- 172.21.15.131:0/2113991505 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fcad806be30 0x7fcad806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:53.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.302+0000 7fcafce00640 1 --2- 172.21.15.131:0/2113991505 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fcad806be30 0x7fcad806e2f0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fcaf80cc2d0 tx=0x7fcae40079c0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:53.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.305+0000 7fcaf5600640 1 -- 172.21.15.131:0/2113991505 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fcaec07a0b0 con 0x7fcaf8071500 2024-05-26T22:56:53.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.531+0000 7fcafec00640 1 -- 172.21.15.131:0/2113991505 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcaf81549a0 con 0x7fcaf8071500 2024-05-26T22:56:53.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.532+0000 7fcaf5600640 1 -- 172.21.15.131:0/2113991505 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 118 v118) ==== 95+0+5244 (secure 0 0 0) 0x7fcaec0799b0 con 0x7fcaf8071500 2024-05-26T22:56:53.532 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:56:53.532 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":118,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5959,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/2949813640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":2949813640},{"type":"v1","addr":"172.21.15.187:6833","nonce":2949813640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104}],"filesystems":[{"mdsmap":{"epoch":118,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-26T22:56:01.980221+0000","modified":"2024-05-26T22:56:40.084341+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8310,"mds_1":8307,"mds_2":5968},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5968":{"gid":5968,"name":"d","rank":2,"incarnation":114,"state":"up:active","state_seq":7,"addr":"172.21.15.187:6835/2594604065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2594604065},{"type":"v1","addr":"172.21.15.187:6835","nonce":2594604065}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8307":{"gid":8307,"name":"a","rank":1,"incarnation":112,"state":"up:active","state_seq":8,"addr":"172.21.15.131:6836/2710427205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":2710427205},{"type":"v1","addr":"172.21.15.131:6836","nonce":2710427205}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8310":{"gid":8310,"name":"c","rank":0,"incarnation":106,"state":"up:active","state_seq":5,"addr":"172.21.15.131:6837/2725520331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6835","nonce":2725520331},{"type":"v1","addr":"172.21.15.131:6837","nonce":2725520331}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8310,"qdb_cluster":[8310,8307,5968]},"id":8}]} 2024-05-26T22:56:53.532 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 118 2024-05-26T22:56:53.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.534+0000 7fcafec00640 1 -- 172.21.15.131:0/2113991505 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fcad806be30 msgr2=0x7fcad806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:53.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.534+0000 7fcafec00640 1 --2- 172.21.15.131:0/2113991505 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fcad806be30 0x7fcad806e2f0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fcaf80cc2d0 tx=0x7fcae40079c0 comp rx=0 tx=0).stop 2024-05-26T22:56:53.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.534+0000 7fcafec00640 1 -- 172.21.15.131:0/2113991505 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcaf8071500 msgr2=0x7fcaf814ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:53.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.534+0000 7fcafec00640 1 --2- 172.21.15.131:0/2113991505 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcaf8071500 0x7fcaf814ec50 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7fcaec000c00 tx=0x7fcaec0046c0 comp rx=0 tx=0).stop 2024-05-26T22:56:53.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.535+0000 7fcafec00640 1 -- 172.21.15.131:0/2113991505 shutdown_connections 2024-05-26T22:56:53.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.535+0000 7fcafec00640 1 --2- 172.21.15.131:0/2113991505 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcaf815e320 0x7fcaf80b22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:53.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.535+0000 7fcafec00640 1 --2- 172.21.15.131:0/2113991505 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fcad806be30 0x7fcad806e2f0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:53.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.535+0000 7fcafec00640 1 --2- 172.21.15.131:0/2113991505 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcaf8150f70 0x7fcaf81e4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:53.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.535+0000 7fcafec00640 1 --2- 172.21.15.131:0/2113991505 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcaf8071500 0x7fcaf814ec50 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:53.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.535+0000 7fcafec00640 1 -- 172.21.15.131:0/2113991505 >> 172.21.15.131:0/2113991505 conn(0x7fcaf80cff40 msgr2=0x7fcaf80cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:56:53.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.535+0000 7fcafec00640 1 -- 172.21.15.131:0/2113991505 shutdown_connections 2024-05-26T22:56:53.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.535+0000 7fcafec00640 1 -- 172.21.15.131:0/2113991505 wait complete. 2024-05-26T22:56:53.552 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-26T22:56:53.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.745+0000 7f0f7e000640 1 Processor -- start 2024-05-26T22:56:53.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.745+0000 7f0f7e000640 1 -- start start 2024-05-26T22:56:53.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.746+0000 7f0f7e000640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f0f780ad250 0x7f0f780b3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:53.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.746+0000 7f0f7e000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0f781569d0 0x7f0f780af790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:53.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.746+0000 7f0f7e000640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f7815cb20 con 0x7f0f7806ee20 2024-05-26T22:56:53.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.746+0000 7f0f7e000640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f0f7815ce00 con 0x7f0f780ad250 2024-05-26T22:56:53.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.746+0000 7f0f7e000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f0f7815d0e0 con 0x7f0f781569d0 2024-05-26T22:56:53.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.747+0000 7f0f77e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0f781569d0 0x7f0f780af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:53.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.747+0000 7f0f77400640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f0f7806ee20 0x7f0f781562c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.131:35442/0 (socket says 172.21.15.131:35442) 2024-05-26T22:56:53.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.747+0000 7f0f77400640 1 -- 172.21.15.131:0/3479190335 learned_addr learned my addr 172.21.15.131:0/3479190335 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:56:53.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.747+0000 7f0f76a00640 1 --2- 172.21.15.131:0/3479190335 >> v2:172.21.15.187:3300/0 conn(0x7f0f780ad250 0x7f0f780b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:53.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.747+0000 7f0f76a00640 1 -- 172.21.15.131:0/3479190335 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0f781569d0 msgr2=0x7f0f780af790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:53.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.747+0000 7f0f76a00640 1 --2- 172.21.15.131:0/3479190335 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0f781569d0 0x7f0f780af790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:53.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.747+0000 7f0f76a00640 1 -- 172.21.15.131:0/3479190335 >> v1:172.21.15.187:6789/0 conn(0x7f0f7806ee20 legacy=0x7f0f781562c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:53.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.747+0000 7f0f76a00640 1 -- 172.21.15.131:0/3479190335 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f7815d3c0 con 0x7f0f780ad250 2024-05-26T22:56:53.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.748+0000 7f0f77e00640 1 --2- 172.21.15.131:0/3479190335 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0f781569d0 0x7f0f780af790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-26T22:56:53.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.749+0000 7f0f76a00640 1 --2- 172.21.15.131:0/3479190335 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0f780ad250 0x7f0f780b3190 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f0f640097a0 tx=0x7f0f6402a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9126540c1218cfe6 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:53.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.749+0000 7f0f76000640 1 -- 172.21.15.131:0/3479190335 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f64037070 con 0x7f0f780ad250 2024-05-26T22:56:53.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.749+0000 7f0f76000640 1 -- 172.21.15.131:0/3479190335 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f64032440 con 0x7f0f780ad250 2024-05-26T22:56:53.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.749+0000 7f0f76000640 1 -- 172.21.15.131:0/3479190335 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f64032810 con 0x7f0f780ad250 2024-05-26T22:56:53.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.750+0000 7f0f7e000640 1 -- 172.21.15.131:0/3479190335 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0f780ad250 msgr2=0x7f0f780b3190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:53.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.750+0000 7f0f7e000640 1 --2- 172.21.15.131:0/3479190335 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0f780ad250 0x7f0f780b3190 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f0f640097a0 tx=0x7f0f6402a180 comp rx=0 tx=0).stop 2024-05-26T22:56:53.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.750+0000 7f0f7e000640 1 -- 172.21.15.131:0/3479190335 shutdown_connections 2024-05-26T22:56:53.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.750+0000 7f0f7e000640 1 --2- 172.21.15.131:0/3479190335 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0f781569d0 0x7f0f780af790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:53.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.750+0000 7f0f7e000640 1 --2- 172.21.15.131:0/3479190335 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0f780ad250 0x7f0f780b3190 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:53.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.750+0000 7f0f7e000640 1 -- 172.21.15.131:0/3479190335 >> 172.21.15.131:0/3479190335 conn(0x7f0f780b1e90 msgr2=0x7f0f780b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:56:53.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.750+0000 7f0f7e000640 1 -- 172.21.15.131:0/3479190335 shutdown_connections 2024-05-26T22:56:53.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.750+0000 7f0f7e000640 1 -- 172.21.15.131:0/3479190335 wait complete. 2024-05-26T22:56:53.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.752+0000 7f0f7e000640 1 Processor -- start 2024-05-26T22:56:53.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.752+0000 7f0f7e000640 1 -- start start 2024-05-26T22:56:53.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.753+0000 7f0f7e000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0f7806ee20 0x7f0f7809c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:53.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.753+0000 7f0f7e000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0f780ad250 0x7f0f7809b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:53.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.753+0000 7f0f7e000640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0f781569d0 0x7f0f78154540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:53.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.753+0000 7f0f7e000640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f0f7815c900 con 0x7f0f7806ee20 2024-05-26T22:56:53.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.753+0000 7f0f7e000640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0f7815c780 con 0x7f0f781569d0 2024-05-26T22:56:53.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.753+0000 7f0f7e000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f0f7806f200 con 0x7f0f780ad250 2024-05-26T22:56:53.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.753+0000 7f0f77400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0f7806ee20 0x7f0f7809c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:53.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.754+0000 7f0f77400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0f7806ee20 0x7f0f7809c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41994/0 (socket says 172.21.15.131:41994) 2024-05-26T22:56:53.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.754+0000 7f0f77400640 1 -- 172.21.15.131:0/71080216 learned_addr learned my addr 172.21.15.131:0/71080216 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:56:53.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.754+0000 7f0f76a00640 1 --2- 172.21.15.131:0/71080216 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0f780ad250 0x7f0f7809b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:53.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.754+0000 7f0f77400640 1 -- 172.21.15.131:0/71080216 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0f780ad250 msgr2=0x7f0f7809b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:53.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.754+0000 7f0f77e00640 1 --2- 172.21.15.131:0/71080216 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0f781569d0 0x7f0f78154540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:53.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.754+0000 7f0f77400640 1 --2- 172.21.15.131:0/71080216 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0f780ad250 0x7f0f7809b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:53.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.754+0000 7f0f77400640 1 -- 172.21.15.131:0/71080216 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0f781569d0 msgr2=0x7f0f78154540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:53.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.754+0000 7f0f77400640 1 --2- 172.21.15.131:0/71080216 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0f781569d0 0x7f0f78154540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:53.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.754+0000 7f0f77400640 1 -- 172.21.15.131:0/71080216 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f7809a420 con 0x7f0f7806ee20 2024-05-26T22:56:53.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.754+0000 7f0f76a00640 1 --2- 172.21.15.131:0/71080216 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0f780ad250 0x7f0f7809b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:56:53.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.754+0000 7f0f77400640 1 --2- 172.21.15.131:0/71080216 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0f7806ee20 0x7f0f7809c6d0 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f0f6800c400 tx=0x7f0f6800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:53.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.755+0000 7f0f6f400640 1 -- 172.21.15.131:0/71080216 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f68017070 con 0x7f0f7806ee20 2024-05-26T22:56:53.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.755+0000 7f0f6f400640 1 -- 172.21.15.131:0/71080216 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f68005ce0 con 0x7f0f7806ee20 2024-05-26T22:56:53.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.755+0000 7f0f6f400640 1 -- 172.21.15.131:0/71080216 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f680072f0 con 0x7f0f7806ee20 2024-05-26T22:56:53.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.755+0000 7f0f7e000640 1 -- 172.21.15.131:0/71080216 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f78072e60 con 0x7f0f7806ee20 2024-05-26T22:56:53.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.755+0000 7f0f7e000640 1 -- 172.21.15.131:0/71080216 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f7809a9a0 con 0x7f0f7806ee20 2024-05-26T22:56:53.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.756+0000 7f0f7e000640 1 -- 172.21.15.131:0/71080216 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f780694e0 con 0x7f0f7806ee20 2024-05-26T22:56:53.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.757+0000 7f0f6f400640 1 -- 172.21.15.131:0/71080216 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0f6801a070 con 0x7f0f7806ee20 2024-05-26T22:56:53.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.758+0000 7f0f6f400640 1 --2- 172.21.15.131:0/71080216 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f0f5806be60 0x7f0f5806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:53.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.758+0000 7f0f6f400640 1 -- 172.21.15.131:0/71080216 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5709+0+0 (secure 0 0 0) 0x7f0f6808a340 con 0x7f0f7806ee20 2024-05-26T22:56:53.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.758+0000 7f0f76a00640 1 --2- 172.21.15.131:0/71080216 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f0f5806be60 0x7f0f5806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:53.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.759+0000 7f0f76a00640 1 --2- 172.21.15.131:0/71080216 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f0f5806be60 0x7f0f5806e320 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f0f64034000 tx=0x7f0f6402e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:53.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.765+0000 7f0f6f400640 1 -- 172.21.15.131:0/71080216 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f0f68058c50 con 0x7f0f7806ee20 2024-05-26T22:56:53.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.950+0000 7f0f7e000640 1 -- 172.21.15.131:0/71080216 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0f7809aff0 con 0x7f0f7806ee20 2024-05-26T22:56:53.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.953+0000 7f0f6f400640 1 -- 172.21.15.131:0/71080216 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v107) ==== 74+0+16651 (secure 0 0 0) 0x7f0f68058550 con 0x7f0f7806ee20 2024-05-26T22:56:53.953 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:56:53.953 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":107,"fsid":"46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f","created":"2024-05-26T22:47:57.288905+0000","modified":"2024-05-26T22:56:03.175526+0000","last_up_change":"2024-05-26T22:48:09.602207+0000","last_in_change":"2024-05-26T22:48:00.687341+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-26T22:48:10.463339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-05-26T22:55:59.775225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-05-26T22:56:00.988905+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"814eaf62-a18d-494b-8e27-fd8fb88adea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6801","nonce":2350940051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6803","nonce":2350940051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6807","nonce":2350940051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6805","nonce":2350940051}]},"public_addr":"172.21.15.131:6801/2350940051","cluster_addr":"172.21.15.131:6803/2350940051","heartbeat_back_addr":"172.21.15.131:6807/2350940051","heartbeat_front_addr":"172.21.15.131:6805/2350940051","state":["exists","up"]},{"osd":1,"uuid":"42e66287-f6c6-4e26-af3c-6dfaa3536d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6825","nonce":1525528247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6827","nonce":1525528247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6831","nonce":1525528247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6829","nonce":1525528247}]},"public_addr":"172.21.15.131:6825/1525528247","cluster_addr":"172.21.15.131:6827/1525528247","heartbeat_back_addr":"172.21.15.131:6831/1525528247","heartbeat_front_addr":"172.21.15.131:6829/1525528247","state":["exists","up"]},{"osd":2,"uuid":"0da34673-04a2-4fd0-b57a-0ff2b679b043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6817","nonce":3627842210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6819","nonce":3627842210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6823","nonce":3627842210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6821","nonce":3627842210}]},"public_addr":"172.21.15.131:6817/3627842210","cluster_addr":"172.21.15.131:6819/3627842210","heartbeat_back_addr":"172.21.15.131:6823/3627842210","heartbeat_front_addr":"172.21.15.131:6821/3627842210","state":["exists","up"]},{"osd":3,"uuid":"cf117a27-d6b2-43d3-ae52-b47988bf8eed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6809","nonce":350790591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6811","nonce":350790591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6815","nonce":350790591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6813","nonce":350790591}]},"public_addr":"172.21.15.131:6809/350790591","cluster_addr":"172.21.15.131:6811/350790591","heartbeat_back_addr":"172.21.15.131:6815/350790591","heartbeat_front_addr":"172.21.15.131:6813/350790591","state":["exists","up"]},{"osd":4,"uuid":"a092ccfc-be89-4a84-88d5-0f781fa715eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6816","nonce":244560995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6819","nonce":244560995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6823","nonce":244560995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6821","nonce":244560995}]},"public_addr":"172.21.15.187:6816/244560995","cluster_addr":"172.21.15.187:6819/244560995","heartbeat_back_addr":"172.21.15.187:6823/244560995","heartbeat_front_addr":"172.21.15.187:6821/244560995","state":["exists","up"]},{"osd":5,"uuid":"7513ba69-dd00-4a35-b190-9e8cc3b2bdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6801","nonce":324619398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6803","nonce":324619398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6812","nonce":324619398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6807","nonce":324619398}]},"public_addr":"172.21.15.187:6801/324619398","cluster_addr":"172.21.15.187:6803/324619398","heartbeat_back_addr":"172.21.15.187:6812/324619398","heartbeat_front_addr":"172.21.15.187:6807/324619398","state":["exists","up"]},{"osd":6,"uuid":"50081810-c093-4d2f-a4d7-b7c695c5400d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6806","nonce":1391347786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6809","nonce":1391347786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6818","nonce":1391347786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6813","nonce":1391347786}]},"public_addr":"172.21.15.187:6806/1391347786","cluster_addr":"172.21.15.187:6809/1391347786","heartbeat_back_addr":"172.21.15.187:6818/1391347786","heartbeat_front_addr":"172.21.15.187:6813/1391347786","state":["exists","up"]},{"osd":7,"uuid":"775dae05-54f5-4ad4-bbce-92863c6ed2ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6825","nonce":1220065622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6827","nonce":1220065622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6831","nonce":1220065622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6829","nonce":1220065622}]},"public_addr":"172.21.15.187:6825/1220065622","cluster_addr":"172.21.15.187:6827/1220065622","heartbeat_back_addr":"172.21.15.187:6831/1220065622","heartbeat_front_addr":"172.21.15.187:6829/1220065622","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:05.983520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.299516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.067323+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.307748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.018742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.258775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.010912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.295355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-26T22:56:53.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.955+0000 7f0f7e000640 1 -- 172.21.15.131:0/71080216 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f0f5806be60 msgr2=0x7f0f5806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:53.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.955+0000 7f0f7e000640 1 --2- 172.21.15.131:0/71080216 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f0f5806be60 0x7f0f5806e320 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f0f64034000 tx=0x7f0f6402e000 comp rx=0 tx=0).stop 2024-05-26T22:56:53.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.955+0000 7f0f7e000640 1 -- 172.21.15.131:0/71080216 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0f7806ee20 msgr2=0x7f0f7809c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:53.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.955+0000 7f0f7e000640 1 --2- 172.21.15.131:0/71080216 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0f7806ee20 0x7f0f7809c6d0 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f0f6800c400 tx=0x7f0f6800c8c0 comp rx=0 tx=0).stop 2024-05-26T22:56:53.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.955+0000 7f0f7e000640 1 -- 172.21.15.131:0/71080216 shutdown_connections 2024-05-26T22:56:53.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.955+0000 7f0f7e000640 1 --2- 172.21.15.131:0/71080216 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0f780ad250 0x7f0f7809b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:53.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.955+0000 7f0f7e000640 1 --2- 172.21.15.131:0/71080216 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f0f5806be60 0x7f0f5806e320 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:53.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.955+0000 7f0f7e000640 1 --2- 172.21.15.131:0/71080216 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0f781569d0 0x7f0f78154540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:53.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.955+0000 7f0f7e000640 1 --2- 172.21.15.131:0/71080216 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0f7806ee20 0x7f0f7809c6d0 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:53.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.955+0000 7f0f7e000640 1 -- 172.21.15.131:0/71080216 >> 172.21.15.131:0/71080216 conn(0x7f0f780b1e90 msgr2=0x7f0f780b44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:56:53.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.956+0000 7f0f7e000640 1 -- 172.21.15.131:0/71080216 shutdown_connections 2024-05-26T22:56:53.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:53.956+0000 7f0f7e000640 1 -- 172.21.15.131:0/71080216 wait complete. 2024-05-26T22:56:53.972 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-26T22:56:53.972 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-26T22:56:54.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.166+0000 7fa4f4e00640 1 Processor -- start 2024-05-26T22:56:54.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.166+0000 7fa4f4e00640 1 -- start start 2024-05-26T22:56:54.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.168+0000 7fa4f4e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fa4f0156750 0x7fa4f015fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:54.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.168+0000 7fa4f4e00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fa4f00cc7f0 0x7fa4f00ccc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:54.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.168+0000 7fa4f4e00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4f0160620 con 0x7fa4f006e8c0 2024-05-26T22:56:54.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.168+0000 7fa4f4e00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fa4f015d770 con 0x7fa4f0156750 2024-05-26T22:56:54.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.168+0000 7fa4f4e00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fa4f00cd1a0 con 0x7fa4f00cc7f0 2024-05-26T22:56:54.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.168+0000 7fa4ef400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fa4f0156750 0x7fa4f015fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:54.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.168+0000 7fa4eea00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fa4f006e8c0 0x7fa4f015d060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:48832/0 (socket says 172.21.15.131:48832) 2024-05-26T22:56:54.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.168+0000 7fa4eea00640 1 -- 172.21.15.131:0/2910106915 learned_addr learned my addr 172.21.15.131:0/2910106915 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:56:54.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.168+0000 7fa4efe00640 1 --2- 172.21.15.131:0/2910106915 >> v2:172.21.15.187:3300/0 conn(0x7fa4f00cc7f0 0x7fa4f00ccc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:54.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.168+0000 7fa4ef400640 1 -- 172.21.15.131:0/2910106915 >> v2:172.21.15.187:3300/0 conn(0x7fa4f00cc7f0 msgr2=0x7fa4f00ccc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:54.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.168+0000 7fa4ef400640 1 --2- 172.21.15.131:0/2910106915 >> v2:172.21.15.187:3300/0 conn(0x7fa4f00cc7f0 0x7fa4f00ccc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:54.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.168+0000 7fa4ef400640 1 -- 172.21.15.131:0/2910106915 >> v1:172.21.15.131:6789/0 conn(0x7fa4f006e8c0 legacy=0x7fa4f015d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:54.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.168+0000 7fa4ef400640 1 -- 172.21.15.131:0/2910106915 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4f0160900 con 0x7fa4f0156750 2024-05-26T22:56:54.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.169+0000 7fa4ef400640 1 --2- 172.21.15.131:0/2910106915 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4f0156750 0x7fa4f015fd40 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7fa4f015a320 tx=0x7fa4e002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6bb698e1932e543 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:54.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.169+0000 7fa4ee000640 1 -- 172.21.15.131:0/2910106915 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e0037070 con 0x7fa4f0156750 2024-05-26T22:56:54.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.169+0000 7fa4ee000640 1 -- 172.21.15.131:0/2910106915 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4e0032440 con 0x7fa4f0156750 2024-05-26T22:56:54.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.169+0000 7fa4ee000640 1 -- 172.21.15.131:0/2910106915 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e0032810 con 0x7fa4f0156750 2024-05-26T22:56:54.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.170+0000 7fa4f4e00640 1 -- 172.21.15.131:0/2910106915 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4f0156750 msgr2=0x7fa4f015fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:54.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.170+0000 7fa4f4e00640 1 --2- 172.21.15.131:0/2910106915 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4f0156750 0x7fa4f015fd40 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7fa4f015a320 tx=0x7fa4e002a1a0 comp rx=0 tx=0).stop 2024-05-26T22:56:54.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.170+0000 7fa4f4e00640 1 -- 172.21.15.131:0/2910106915 shutdown_connections 2024-05-26T22:56:54.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.170+0000 7fa4f4e00640 1 --2- 172.21.15.131:0/2910106915 >> v2:172.21.15.187:3300/0 conn(0x7fa4f00cc7f0 0x7fa4f00ccc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:54.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.170+0000 7fa4f4e00640 1 --2- 172.21.15.131:0/2910106915 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4f0156750 0x7fa4f015fd40 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:54.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.170+0000 7fa4f4e00640 1 -- 172.21.15.131:0/2910106915 >> 172.21.15.131:0/2910106915 conn(0x7fa4f00cff40 msgr2=0x7fa4f00d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:56:54.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.170+0000 7fa4f4e00640 1 -- 172.21.15.131:0/2910106915 shutdown_connections 2024-05-26T22:56:54.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.170+0000 7fa4f4e00640 1 -- 172.21.15.131:0/2910106915 wait complete. 2024-05-26T22:56:54.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.171+0000 7fa4f4e00640 1 Processor -- start 2024-05-26T22:56:54.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.172+0000 7fa4f4e00640 1 -- start start 2024-05-26T22:56:54.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.172+0000 7fa4f4e00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa4f006e8c0 0x7fa4f013a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:54.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.172+0000 7fa4f4e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa4f00cc7f0 0x7fa4f015b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:54.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.172+0000 7fa4f4e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4f0156750 0x7fa4f013f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:54.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.172+0000 7fa4f4e00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f015c860 con 0x7fa4f0156750 2024-05-26T22:56:54.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.172+0000 7fa4f4e00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f015cbd0 con 0x7fa4f006e8c0 2024-05-26T22:56:54.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.172+0000 7fa4f4e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa4f0156b30 con 0x7fa4f00cc7f0 2024-05-26T22:56:54.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.173+0000 7fa4efe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4f0156750 0x7fa4f013f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:54.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.173+0000 7fa4efe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4f0156750 0x7fa4f013f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42010/0 (socket says 172.21.15.131:42010) 2024-05-26T22:56:54.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.173+0000 7fa4efe00640 1 -- 172.21.15.131:0/1710447847 learned_addr learned my addr 172.21.15.131:0/1710447847 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:56:54.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.173+0000 7fa4efe00640 1 -- 172.21.15.131:0/1710447847 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa4f00cc7f0 msgr2=0x7fa4f015b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:54.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.173+0000 7fa4efe00640 1 --2- 172.21.15.131:0/1710447847 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa4f00cc7f0 0x7fa4f015b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:54.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.173+0000 7fa4efe00640 1 -- 172.21.15.131:0/1710447847 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa4f006e8c0 msgr2=0x7fa4f013a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:54.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.173+0000 7fa4ef400640 1 --2- 172.21.15.131:0/1710447847 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa4f006e8c0 0x7fa4f013a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:54.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.173+0000 7fa4efe00640 1 --2- 172.21.15.131:0/1710447847 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa4f006e8c0 0x7fa4f013a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:54.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.173+0000 7fa4efe00640 1 -- 172.21.15.131:0/1710447847 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4f0141be0 con 0x7fa4f0156750 2024-05-26T22:56:54.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.173+0000 7fa4efe00640 1 --2- 172.21.15.131:0/1710447847 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4f0156750 0x7fa4f013f040 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7fa4e8007e70 tx=0x7fa4e800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:54.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.174+0000 7fa4e7400640 1 -- 172.21.15.131:0/1710447847 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e8014620 con 0x7fa4f0156750 2024-05-26T22:56:54.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.174+0000 7fa4e7400640 1 -- 172.21.15.131:0/1710447847 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4e8019070 con 0x7fa4f0156750 2024-05-26T22:56:54.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.174+0000 7fa4e7400640 1 -- 172.21.15.131:0/1710447847 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e8014a20 con 0x7fa4f0156750 2024-05-26T22:56:54.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.174+0000 7fa4f4e00640 1 -- 172.21.15.131:0/1710447847 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4f013aa70 con 0x7fa4f0156750 2024-05-26T22:56:54.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.174+0000 7fa4f4e00640 1 -- 172.21.15.131:0/1710447847 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4f0141620 con 0x7fa4f0156750 2024-05-26T22:56:54.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.175+0000 7fa4f4e00640 1 -- 172.21.15.131:0/1710447847 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4f00694e0 con 0x7fa4f0156750 2024-05-26T22:56:54.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.176+0000 7fa4e7400640 1 -- 172.21.15.131:0/1710447847 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa4e8014bc0 con 0x7fa4f0156750 2024-05-26T22:56:54.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.177+0000 7fa4e7400640 1 --2- 172.21.15.131:0/1710447847 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa4d006be30 0x7fa4d006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:54.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.177+0000 7fa4e7400640 1 -- 172.21.15.131:0/1710447847 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5709+0+0 (secure 0 0 0) 0x7fa4e808b820 con 0x7fa4f0156750 2024-05-26T22:56:54.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.177+0000 7fa4ef400640 1 --2- 172.21.15.131:0/1710447847 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa4d006be30 0x7fa4d006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:54.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.178+0000 7fa4ef400640 1 --2- 172.21.15.131:0/1710447847 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa4d006be30 0x7fa4d006e2f0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fa4e0034000 tx=0x7fa4e002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:54.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.181+0000 7fa4e7400640 1 -- 172.21.15.131:0/1710447847 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fa4e805a130 con 0x7fa4f0156750 2024-05-26T22:56:54.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.400+0000 7fa4f4e00640 1 -- 172.21.15.131:0/1710447847 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa4f00ce070 con 0x7fa4f0156750 2024-05-26T22:56:54.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.402+0000 7fa4e7400640 1 -- 172.21.15.131:0/1710447847 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 118 v118) ==== 95+0+5244 (secure 0 0 0) 0x7fa4e8059a30 con 0x7fa4f0156750 2024-05-26T22:56:54.402 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:56:54.402 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":118,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5959,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/2949813640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":2949813640},{"type":"v1","addr":"172.21.15.187:6833","nonce":2949813640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104}],"filesystems":[{"mdsmap":{"epoch":118,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-26T22:56:01.980221+0000","modified":"2024-05-26T22:56:40.084341+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8310,"mds_1":8307,"mds_2":5968},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5968":{"gid":5968,"name":"d","rank":2,"incarnation":114,"state":"up:active","state_seq":7,"addr":"172.21.15.187:6835/2594604065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2594604065},{"type":"v1","addr":"172.21.15.187:6835","nonce":2594604065}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8307":{"gid":8307,"name":"a","rank":1,"incarnation":112,"state":"up:active","state_seq":8,"addr":"172.21.15.131:6836/2710427205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":2710427205},{"type":"v1","addr":"172.21.15.131:6836","nonce":2710427205}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8310":{"gid":8310,"name":"c","rank":0,"incarnation":106,"state":"up:active","state_seq":5,"addr":"172.21.15.131:6837/2725520331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6835","nonce":2725520331},{"type":"v1","addr":"172.21.15.131:6837","nonce":2725520331}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8310,"qdb_cluster":[8310,8307,5968]},"id":8}]} 2024-05-26T22:56:54.403 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 118 2024-05-26T22:56:54.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.405+0000 7fa4f4e00640 1 -- 172.21.15.131:0/1710447847 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa4d006be30 msgr2=0x7fa4d006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:54.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.405+0000 7fa4f4e00640 1 --2- 172.21.15.131:0/1710447847 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa4d006be30 0x7fa4d006e2f0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fa4e0034000 tx=0x7fa4e002e000 comp rx=0 tx=0).stop 2024-05-26T22:56:54.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.405+0000 7fa4f4e00640 1 -- 172.21.15.131:0/1710447847 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4f0156750 msgr2=0x7fa4f013f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:54.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.405+0000 7fa4f4e00640 1 --2- 172.21.15.131:0/1710447847 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4f0156750 0x7fa4f013f040 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7fa4e8007e70 tx=0x7fa4e800e4b0 comp rx=0 tx=0).stop 2024-05-26T22:56:54.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.405+0000 7fa4f4e00640 1 -- 172.21.15.131:0/1710447847 shutdown_connections 2024-05-26T22:56:54.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.405+0000 7fa4f4e00640 1 --2- 172.21.15.131:0/1710447847 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa4f00cc7f0 0x7fa4f015b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:54.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.405+0000 7fa4f4e00640 1 --2- 172.21.15.131:0/1710447847 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa4d006be30 0x7fa4d006e2f0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:54.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.405+0000 7fa4f4e00640 1 --2- 172.21.15.131:0/1710447847 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4f0156750 0x7fa4f013f040 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:54.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.405+0000 7fa4f4e00640 1 --2- 172.21.15.131:0/1710447847 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa4f006e8c0 0x7fa4f013a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:54.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.405+0000 7fa4f4e00640 1 -- 172.21.15.131:0/1710447847 >> 172.21.15.131:0/1710447847 conn(0x7fa4f00cff40 msgr2=0x7fa4f00ce590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:56:54.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.405+0000 7fa4f4e00640 1 -- 172.21.15.131:0/1710447847 shutdown_connections 2024-05-26T22:56:54.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.405+0000 7fa4f4e00640 1 -- 172.21.15.131:0/1710447847 wait complete. 2024-05-26T22:56:54.422 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-26T22:56:54.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.611+0000 7f3ad8400640 1 Processor -- start 2024-05-26T22:56:54.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.612+0000 7f3ad8400640 1 -- start start 2024-05-26T22:56:54.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.613+0000 7f3ad8400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f3ad0158c00 0x7f3ad006d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:54.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.613+0000 7f3ad8400640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ad00b3e50 con 0x7f3ad00b38a0 2024-05-26T22:56:54.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.613+0000 7f3ad8400640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ad00b4130 con 0x7f3ad006daf0 2024-05-26T22:56:54.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.613+0000 7f3ad8400640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f3ad006df50 con 0x7f3ad0158c00 2024-05-26T22:56:54.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.614+0000 7f3ad5000640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f3ad006daf0 0x7f3ad00b3190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.131:47372/0 (socket says 172.21.15.131:47372) 2024-05-26T22:56:54.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.614+0000 7f3ad6400640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f3ad00b38a0 0x7f3ad015fc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:33708/0 (socket says 172.21.15.131:33708) 2024-05-26T22:56:54.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.614+0000 7f3ad5000640 1 -- 172.21.15.131:0/2359688697 learned_addr learned my addr 172.21.15.131:0/2359688697 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:56:54.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.614+0000 7f3ad5a00640 1 --2- 172.21.15.131:0/2359688697 >> v2:172.21.15.131:3300/0 conn(0x7f3ad0158c00 0x7f3ad006d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:54.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.614+0000 7f3acfe00640 1 -- 172.21.15.131:0/2359688697 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3979446831 0 0) 0x7f3ad00b3e50 con 0x7f3ad00b38a0 2024-05-26T22:56:54.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.614+0000 7f3acfe00640 1 -- 172.21.15.131:0/2359688697 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3ab8003520 con 0x7f3ad00b38a0 2024-05-26T22:56:54.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.615+0000 7f3ad5a00640 1 -- 172.21.15.131:0/2359688697 >> v1:172.21.15.187:6789/0 conn(0x7f3ad006daf0 legacy=0x7f3ad00b3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:54.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.615+0000 7f3ad5a00640 1 -- 172.21.15.131:0/2359688697 >> v1:172.21.15.131:6789/0 conn(0x7f3ad00b38a0 legacy=0x7f3ad015fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:54.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.615+0000 7f3ad5a00640 1 -- 172.21.15.131:0/2359688697 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ad00b4410 con 0x7f3ad0158c00 2024-05-26T22:56:54.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.616+0000 7f3ad5a00640 1 --2- 172.21.15.131:0/2359688697 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ad0158c00 0x7f3ad006d5b0 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f3ac40097a0 tx=0x7f3ac402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=addd6c3c1066bce0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:54.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.616+0000 7f3acfe00640 1 -- 172.21.15.131:0/2359688697 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ac4037070 con 0x7f3ad0158c00 2024-05-26T22:56:54.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.616+0000 7f3acfe00640 1 -- 172.21.15.131:0/2359688697 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ac4032440 con 0x7f3ad0158c00 2024-05-26T22:56:54.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.616+0000 7f3acfe00640 1 -- 172.21.15.131:0/2359688697 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ac4032810 con 0x7f3ad0158c00 2024-05-26T22:56:54.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.617+0000 7f3ad8400640 1 -- 172.21.15.131:0/2359688697 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ad0158c00 msgr2=0x7f3ad006d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:54.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.617+0000 7f3ad8400640 1 --2- 172.21.15.131:0/2359688697 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ad0158c00 0x7f3ad006d5b0 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f3ac40097a0 tx=0x7f3ac402a1a0 comp rx=0 tx=0).stop 2024-05-26T22:56:54.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.617+0000 7f3ad8400640 1 -- 172.21.15.131:0/2359688697 shutdown_connections 2024-05-26T22:56:54.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.617+0000 7f3ad8400640 1 --2- 172.21.15.131:0/2359688697 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ad0158c00 0x7f3ad006d5b0 unknown :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:54.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.617+0000 7f3ad8400640 1 -- 172.21.15.131:0/2359688697 >> 172.21.15.131:0/2359688697 conn(0x7f3ad00b1e90 msgr2=0x7f3ad00b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:56:54.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.617+0000 7f3ad8400640 1 -- 172.21.15.131:0/2359688697 shutdown_connections 2024-05-26T22:56:54.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.617+0000 7f3ad8400640 1 -- 172.21.15.131:0/2359688697 wait complete. 2024-05-26T22:56:54.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.618+0000 7f3ad8400640 1 Processor -- start 2024-05-26T22:56:54.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.619+0000 7f3ad8400640 1 -- start start 2024-05-26T22:56:54.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.619+0000 7f3ad8400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3ad006daf0 0x7f3ad0144ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:54.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.620+0000 7f3ad8400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ad00b38a0 0x7f3ad01451e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:54.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.620+0000 7f3ad8400640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3ad0158c00 0x7f3ad00ad170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:54.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.620+0000 7f3ad8400640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad0160360 con 0x7f3ad00b38a0 2024-05-26T22:56:54.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.620+0000 7f3ad8400640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad00b49d0 con 0x7f3ad0158c00 2024-05-26T22:56:54.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.620+0000 7f3ad8400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3ad01604e0 con 0x7f3ad006daf0 2024-05-26T22:56:54.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.620+0000 7f3ad5000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ad00b38a0 0x7f3ad01451e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:54.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.620+0000 7f3ad5000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ad00b38a0 0x7f3ad01451e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40930/0 (socket says 172.21.15.131:40930) 2024-05-26T22:56:54.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.620+0000 7f3ad5000640 1 -- 172.21.15.131:0/1380898994 learned_addr learned my addr 172.21.15.131:0/1380898994 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:56:54.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.620+0000 7f3ad5000640 1 -- 172.21.15.131:0/1380898994 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3ad006daf0 msgr2=0x7f3ad0144ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:54.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.620+0000 7f3ad5000640 1 --2- 172.21.15.131:0/1380898994 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3ad006daf0 0x7f3ad0144ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:54.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.620+0000 7f3ad5000640 1 -- 172.21.15.131:0/1380898994 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3ad0158c00 msgr2=0x7f3ad00ad170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:54.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.620+0000 7f3ad5000640 1 --2- 172.21.15.131:0/1380898994 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3ad0158c00 0x7f3ad00ad170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:54.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.620+0000 7f3ad5000640 1 -- 172.21.15.131:0/1380898994 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ad0099ea0 con 0x7f3ad00b38a0 2024-05-26T22:56:54.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.621+0000 7f3ad5000640 1 --2- 172.21.15.131:0/1380898994 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ad00b38a0 0x7f3ad01451e0 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f3abc00c4d0 tx=0x7f3abc00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:54.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.621+0000 7f3acd600640 1 -- 172.21.15.131:0/1380898994 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3abc017070 con 0x7f3ad00b38a0 2024-05-26T22:56:54.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.621+0000 7f3acd600640 1 -- 172.21.15.131:0/1380898994 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3abc005ce0 con 0x7f3ad00b38a0 2024-05-26T22:56:54.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.621+0000 7f3ad8400640 1 -- 172.21.15.131:0/1380898994 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ad01577e0 con 0x7f3ad00b38a0 2024-05-26T22:56:54.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.621+0000 7f3acd600640 1 -- 172.21.15.131:0/1380898994 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3abc007270 con 0x7f3ad00b38a0 2024-05-26T22:56:54.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.621+0000 7f3ad8400640 1 -- 172.21.15.131:0/1380898994 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ad0145be0 con 0x7f3ad00b38a0 2024-05-26T22:56:54.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.622+0000 7f3ad8400640 1 -- 172.21.15.131:0/1380898994 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ad0145720 con 0x7f3ad00b38a0 2024-05-26T22:56:54.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.623+0000 7f3acd600640 1 -- 172.21.15.131:0/1380898994 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3abc01a070 con 0x7f3ad00b38a0 2024-05-26T22:56:54.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.624+0000 7f3acd600640 1 --2- 172.21.15.131:0/1380898994 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f3ab006be30 0x7f3ab006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:54.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.624+0000 7f3acd600640 1 -- 172.21.15.131:0/1380898994 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5709+0+0 (secure 0 0 0) 0x7f3abc08a100 con 0x7f3ad00b38a0 2024-05-26T22:56:54.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.624+0000 7f3ad5a00640 1 --2- 172.21.15.131:0/1380898994 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f3ab006be30 0x7f3ab006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:54.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.625+0000 7f3ad5a00640 1 --2- 172.21.15.131:0/1380898994 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f3ab006be30 0x7f3ab006e2f0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f3ac4034000 tx=0x7f3ac402c000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:54.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.628+0000 7f3acd600640 1 -- 172.21.15.131:0/1380898994 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f3ad0145720 con 0x7f3ad00b38a0 2024-05-26T22:56:54.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.876+0000 7f3ad8400640 1 -- 172.21.15.131:0/1380898994 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3ad00694e0 con 0x7f3ad00b38a0 2024-05-26T22:56:54.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.877+0000 7f3acd600640 1 -- 172.21.15.131:0/1380898994 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 118 v118) ==== 95+0+5244 (secure 0 0 0) 0x7f3abc058450 con 0x7f3ad00b38a0 2024-05-26T22:56:54.877 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:56:54.877 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":118,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5959,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/2949813640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":2949813640},{"type":"v1","addr":"172.21.15.187:6833","nonce":2949813640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104}],"filesystems":[{"mdsmap":{"epoch":118,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-26T22:56:01.980221+0000","modified":"2024-05-26T22:56:40.084341+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8310,"mds_1":8307,"mds_2":5968},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5968":{"gid":5968,"name":"d","rank":2,"incarnation":114,"state":"up:active","state_seq":7,"addr":"172.21.15.187:6835/2594604065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2594604065},{"type":"v1","addr":"172.21.15.187:6835","nonce":2594604065}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8307":{"gid":8307,"name":"a","rank":1,"incarnation":112,"state":"up:active","state_seq":8,"addr":"172.21.15.131:6836/2710427205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":2710427205},{"type":"v1","addr":"172.21.15.131:6836","nonce":2710427205}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8310":{"gid":8310,"name":"c","rank":0,"incarnation":106,"state":"up:active","state_seq":5,"addr":"172.21.15.131:6837/2725520331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6835","nonce":2725520331},{"type":"v1","addr":"172.21.15.131:6837","nonce":2725520331}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8310,"qdb_cluster":[8310,8307,5968]},"id":8}]} 2024-05-26T22:56:54.877 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 118 2024-05-26T22:56:54.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.880+0000 7f3ad8400640 1 -- 172.21.15.131:0/1380898994 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f3ab006be30 msgr2=0x7f3ab006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:54.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.880+0000 7f3ad8400640 1 --2- 172.21.15.131:0/1380898994 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f3ab006be30 0x7f3ab006e2f0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f3ac4034000 tx=0x7f3ac402c000 comp rx=0 tx=0).stop 2024-05-26T22:56:54.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.880+0000 7f3ad8400640 1 -- 172.21.15.131:0/1380898994 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ad00b38a0 msgr2=0x7f3ad01451e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:54.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.880+0000 7f3ad8400640 1 --2- 172.21.15.131:0/1380898994 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ad00b38a0 0x7f3ad01451e0 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f3abc00c4d0 tx=0x7f3abc00c990 comp rx=0 tx=0).stop 2024-05-26T22:56:54.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.880+0000 7f3ad8400640 1 -- 172.21.15.131:0/1380898994 shutdown_connections 2024-05-26T22:56:54.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.880+0000 7f3ad8400640 1 --2- 172.21.15.131:0/1380898994 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f3ab006be30 0x7f3ab006e2f0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:54.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.880+0000 7f3ad8400640 1 --2- 172.21.15.131:0/1380898994 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3ad0158c00 0x7f3ad00ad170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:54.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.880+0000 7f3ad8400640 1 --2- 172.21.15.131:0/1380898994 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ad00b38a0 0x7f3ad01451e0 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:54.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.880+0000 7f3ad8400640 1 --2- 172.21.15.131:0/1380898994 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3ad006daf0 0x7f3ad0144ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:54.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.880+0000 7f3ad8400640 1 -- 172.21.15.131:0/1380898994 >> 172.21.15.131:0/1380898994 conn(0x7f3ad00b1e90 msgr2=0x7f3ad00b0700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:56:54.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.880+0000 7f3ad8400640 1 -- 172.21.15.131:0/1380898994 shutdown_connections 2024-05-26T22:56:54.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:54.880+0000 7f3ad8400640 1 -- 172.21.15.131:0/1380898994 wait complete. 2024-05-26T22:56:54.899 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-26T22:56:55.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.093+0000 7f9224800640 1 Processor -- start 2024-05-26T22:56:55.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.093+0000 7f9224800640 1 -- start start 2024-05-26T22:56:55.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.094+0000 7f9224800640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f921c0ad8e0 0x7f921c06f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:55.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.094+0000 7f9224800640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f921c0b5190 con 0x7f921c071d80 2024-05-26T22:56:55.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.094+0000 7f9224800640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f921c06dee0 con 0x7f921c0ad450 2024-05-26T22:56:55.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.094+0000 7f9224800640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f921c06e1c0 con 0x7f921c0ad8e0 2024-05-26T22:56:55.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.095+0000 7f9221e00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f921c071d80 0x7f921c072160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:33714/0 (socket says 172.21.15.131:33714) 2024-05-26T22:56:55.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.095+0000 7f9221e00640 1 -- 172.21.15.131:0/3161202479 learned_addr learned my addr 172.21.15.131:0/3161202479 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:56:55.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.095+0000 7f9222800640 1 --2- 172.21.15.131:0/3161202479 >> v2:172.21.15.131:3300/0 conn(0x7f921c0ad8e0 0x7f921c06f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:55.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.095+0000 7f9220a00640 1 -- 172.21.15.131:0/3161202479 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 762146728 0 0) 0x7f921c0b5190 con 0x7f921c071d80 2024-05-26T22:56:55.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.095+0000 7f9220a00640 1 -- 172.21.15.131:0/3161202479 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9204003520 con 0x7f921c071d80 2024-05-26T22:56:55.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.095+0000 7f9222800640 1 -- 172.21.15.131:0/3161202479 >> v1:172.21.15.187:6789/0 conn(0x7f921c0ad450 legacy=0x7f921c06eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:55.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.095+0000 7f9222800640 1 -- 172.21.15.131:0/3161202479 >> v1:172.21.15.131:6789/0 conn(0x7f921c071d80 legacy=0x7f921c072160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:55.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.095+0000 7f9222800640 1 -- 172.21.15.131:0/3161202479 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f921c06e4a0 con 0x7f921c0ad8e0 2024-05-26T22:56:55.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.097+0000 7f9222800640 1 --2- 172.21.15.131:0/3161202479 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f921c0ad8e0 0x7f921c06f3b0 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f9218009d90 tx=0x7f921802a580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24980d56e2c7c756 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:55.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.097+0000 7f9220a00640 1 -- 172.21.15.131:0/3161202479 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9218007740 con 0x7f921c0ad8e0 2024-05-26T22:56:55.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.097+0000 7f9220a00640 1 -- 172.21.15.131:0/3161202479 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92180078e0 con 0x7f921c0ad8e0 2024-05-26T22:56:55.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.097+0000 7f9220a00640 1 -- 172.21.15.131:0/3161202479 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9218007cb0 con 0x7f921c0ad8e0 2024-05-26T22:56:55.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.098+0000 7f9224800640 1 -- 172.21.15.131:0/3161202479 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f921c0ad8e0 msgr2=0x7f921c06f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:55.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.098+0000 7f9224800640 1 --2- 172.21.15.131:0/3161202479 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f921c0ad8e0 0x7f921c06f3b0 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f9218009d90 tx=0x7f921802a580 comp rx=0 tx=0).stop 2024-05-26T22:56:55.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.098+0000 7f9224800640 1 -- 172.21.15.131:0/3161202479 shutdown_connections 2024-05-26T22:56:55.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.098+0000 7f9224800640 1 --2- 172.21.15.131:0/3161202479 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f921c0ad8e0 0x7f921c06f3b0 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:55.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.098+0000 7f9224800640 1 -- 172.21.15.131:0/3161202479 >> 172.21.15.131:0/3161202479 conn(0x7f921c0b1e90 msgr2=0x7f921c0b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:56:55.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.098+0000 7f9224800640 1 -- 172.21.15.131:0/3161202479 shutdown_connections 2024-05-26T22:56:55.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.098+0000 7f9224800640 1 -- 172.21.15.131:0/3161202479 wait complete. 2024-05-26T22:56:55.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.100+0000 7f9224800640 1 Processor -- start 2024-05-26T22:56:55.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.100+0000 7f9224800640 1 -- start start 2024-05-26T22:56:55.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.101+0000 7f9224800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f921c071d80 0x7f921c144430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:55.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.101+0000 7f9224800640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f921c0ad450 0x7f921c15c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:55.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.101+0000 7f9224800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f921c0ccb70 0x7f921c15cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:55.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.101+0000 7f9224800640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f921c15e1a0 con 0x7f921c0ccb70 2024-05-26T22:56:55.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.101+0000 7f9224800640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f921c15e020 con 0x7f921c0ad450 2024-05-26T22:56:55.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.101+0000 7f9224800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f921c06d5b0 con 0x7f921c071d80 2024-05-26T22:56:55.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.101+0000 7f9222800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f921c0ccb70 0x7f921c15cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:55.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.101+0000 7f9222800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f921c0ccb70 0x7f921c15cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40948/0 (socket says 172.21.15.131:40948) 2024-05-26T22:56:55.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.101+0000 7f9222800640 1 -- 172.21.15.131:0/1291215125 learned_addr learned my addr 172.21.15.131:0/1291215125 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:56:55.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.101+0000 7f9221400640 1 --2- 172.21.15.131:0/1291215125 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f921c0ad450 0x7f921c15c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:55.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.101+0000 7f9221e00640 1 --2- 172.21.15.131:0/1291215125 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f921c071d80 0x7f921c144430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:55.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.101+0000 7f9222800640 1 -- 172.21.15.131:0/1291215125 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f921c071d80 msgr2=0x7f921c144430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:55.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.101+0000 7f9222800640 1 --2- 172.21.15.131:0/1291215125 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f921c071d80 0x7f921c144430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:55.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.101+0000 7f9222800640 1 -- 172.21.15.131:0/1291215125 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f921c0ad450 msgr2=0x7f921c15c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:55.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.101+0000 7f9222800640 1 --2- 172.21.15.131:0/1291215125 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f921c0ad450 0x7f921c15c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:55.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.101+0000 7f9222800640 1 -- 172.21.15.131:0/1291215125 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f921c0cf6f0 con 0x7f921c0ccb70 2024-05-26T22:56:55.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.101+0000 7f9221400640 1 --2- 172.21.15.131:0/1291215125 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f921c0ad450 0x7f921c15c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:56:55.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.102+0000 7f9222800640 1 --2- 172.21.15.131:0/1291215125 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f921c0ccb70 0x7f921c15cba0 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f9218009eb0 tx=0x7f9218004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:55.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.102+0000 7f9216000640 1 -- 172.21.15.131:0/1291215125 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f921803f070 con 0x7f921c0ccb70 2024-05-26T22:56:55.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.102+0000 7f9216000640 1 -- 172.21.15.131:0/1291215125 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9218031bc0 con 0x7f921c0ccb70 2024-05-26T22:56:55.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.102+0000 7f9216000640 1 -- 172.21.15.131:0/1291215125 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9218009450 con 0x7f921c0ccb70 2024-05-26T22:56:55.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.102+0000 7f9224800640 1 -- 172.21.15.131:0/1291215125 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f921c0cf9d0 con 0x7f921c0ccb70 2024-05-26T22:56:55.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.102+0000 7f9224800640 1 -- 172.21.15.131:0/1291215125 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f921c154e90 con 0x7f921c0ccb70 2024-05-26T22:56:55.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.103+0000 7f9224800640 1 -- 172.21.15.131:0/1291215125 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f921c0694e0 con 0x7f921c0ccb70 2024-05-26T22:56:55.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.105+0000 7f9216000640 1 -- 172.21.15.131:0/1291215125 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f921803b410 con 0x7f921c0ccb70 2024-05-26T22:56:55.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.105+0000 7f9216000640 1 --2- 172.21.15.131:0/1291215125 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f920006be60 0x7f920006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:55.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.106+0000 7f9216000640 1 -- 172.21.15.131:0/1291215125 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5709+0+0 (secure 0 0 0) 0x7f92180abf50 con 0x7f921c0ccb70 2024-05-26T22:56:55.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.106+0000 7f9221e00640 1 --2- 172.21.15.131:0/1291215125 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f920006be60 0x7f920006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:55.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.106+0000 7f9221e00640 1 --2- 172.21.15.131:0/1291215125 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f920006be60 0x7f920006e320 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f9210000f60 tx=0x7f9210009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:55.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.110+0000 7f9216000640 1 -- 172.21.15.131:0/1291215125 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f921807a860 con 0x7f921c0ccb70 2024-05-26T22:56:55.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.310+0000 7f9224800640 1 -- 172.21.15.131:0/1291215125 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f921c0b27a0 con 0x7f921c0ccb70 2024-05-26T22:56:55.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.312+0000 7f9216000640 1 -- 172.21.15.131:0/1291215125 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v107) ==== 74+0+16651 (secure 0 0 0) 0x7f921807a160 con 0x7f921c0ccb70 2024-05-26T22:56:55.312 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:56:55.312 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":107,"fsid":"46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f","created":"2024-05-26T22:47:57.288905+0000","modified":"2024-05-26T22:56:03.175526+0000","last_up_change":"2024-05-26T22:48:09.602207+0000","last_in_change":"2024-05-26T22:48:00.687341+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-26T22:48:10.463339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-05-26T22:55:59.775225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-05-26T22:56:00.988905+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"814eaf62-a18d-494b-8e27-fd8fb88adea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6801","nonce":2350940051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6803","nonce":2350940051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6807","nonce":2350940051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6805","nonce":2350940051}]},"public_addr":"172.21.15.131:6801/2350940051","cluster_addr":"172.21.15.131:6803/2350940051","heartbeat_back_addr":"172.21.15.131:6807/2350940051","heartbeat_front_addr":"172.21.15.131:6805/2350940051","state":["exists","up"]},{"osd":1,"uuid":"42e66287-f6c6-4e26-af3c-6dfaa3536d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6825","nonce":1525528247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6827","nonce":1525528247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6831","nonce":1525528247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6829","nonce":1525528247}]},"public_addr":"172.21.15.131:6825/1525528247","cluster_addr":"172.21.15.131:6827/1525528247","heartbeat_back_addr":"172.21.15.131:6831/1525528247","heartbeat_front_addr":"172.21.15.131:6829/1525528247","state":["exists","up"]},{"osd":2,"uuid":"0da34673-04a2-4fd0-b57a-0ff2b679b043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6817","nonce":3627842210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6819","nonce":3627842210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6823","nonce":3627842210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6821","nonce":3627842210}]},"public_addr":"172.21.15.131:6817/3627842210","cluster_addr":"172.21.15.131:6819/3627842210","heartbeat_back_addr":"172.21.15.131:6823/3627842210","heartbeat_front_addr":"172.21.15.131:6821/3627842210","state":["exists","up"]},{"osd":3,"uuid":"cf117a27-d6b2-43d3-ae52-b47988bf8eed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6809","nonce":350790591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6811","nonce":350790591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6815","nonce":350790591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6813","nonce":350790591}]},"public_addr":"172.21.15.131:6809/350790591","cluster_addr":"172.21.15.131:6811/350790591","heartbeat_back_addr":"172.21.15.131:6815/350790591","heartbeat_front_addr":"172.21.15.131:6813/350790591","state":["exists","up"]},{"osd":4,"uuid":"a092ccfc-be89-4a84-88d5-0f781fa715eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6816","nonce":244560995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6819","nonce":244560995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6823","nonce":244560995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6821","nonce":244560995}]},"public_addr":"172.21.15.187:6816/244560995","cluster_addr":"172.21.15.187:6819/244560995","heartbeat_back_addr":"172.21.15.187:6823/244560995","heartbeat_front_addr":"172.21.15.187:6821/244560995","state":["exists","up"]},{"osd":5,"uuid":"7513ba69-dd00-4a35-b190-9e8cc3b2bdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6801","nonce":324619398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6803","nonce":324619398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6812","nonce":324619398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6807","nonce":324619398}]},"public_addr":"172.21.15.187:6801/324619398","cluster_addr":"172.21.15.187:6803/324619398","heartbeat_back_addr":"172.21.15.187:6812/324619398","heartbeat_front_addr":"172.21.15.187:6807/324619398","state":["exists","up"]},{"osd":6,"uuid":"50081810-c093-4d2f-a4d7-b7c695c5400d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6806","nonce":1391347786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6809","nonce":1391347786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6818","nonce":1391347786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6813","nonce":1391347786}]},"public_addr":"172.21.15.187:6806/1391347786","cluster_addr":"172.21.15.187:6809/1391347786","heartbeat_back_addr":"172.21.15.187:6818/1391347786","heartbeat_front_addr":"172.21.15.187:6813/1391347786","state":["exists","up"]},{"osd":7,"uuid":"775dae05-54f5-4ad4-bbce-92863c6ed2ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6825","nonce":1220065622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6827","nonce":1220065622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6831","nonce":1220065622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6829","nonce":1220065622}]},"public_addr":"172.21.15.187:6825/1220065622","cluster_addr":"172.21.15.187:6827/1220065622","heartbeat_back_addr":"172.21.15.187:6831/1220065622","heartbeat_front_addr":"172.21.15.187:6829/1220065622","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:05.983520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.299516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.067323+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.307748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.018742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.258775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.010912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.295355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-26T22:56:55.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.314+0000 7f9224800640 1 -- 172.21.15.131:0/1291215125 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f920006be60 msgr2=0x7f920006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:55.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.314+0000 7f9224800640 1 --2- 172.21.15.131:0/1291215125 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f920006be60 0x7f920006e320 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f9210000f60 tx=0x7f9210009040 comp rx=0 tx=0).stop 2024-05-26T22:56:55.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.314+0000 7f9224800640 1 -- 172.21.15.131:0/1291215125 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f921c0ccb70 msgr2=0x7f921c15cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:55.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.314+0000 7f9224800640 1 --2- 172.21.15.131:0/1291215125 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f921c0ccb70 0x7f921c15cba0 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f9218009eb0 tx=0x7f9218004a40 comp rx=0 tx=0).stop 2024-05-26T22:56:55.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.315+0000 7f9224800640 1 -- 172.21.15.131:0/1291215125 shutdown_connections 2024-05-26T22:56:55.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.315+0000 7f9224800640 1 --2- 172.21.15.131:0/1291215125 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f920006be60 0x7f920006e320 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:55.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.315+0000 7f9224800640 1 --2- 172.21.15.131:0/1291215125 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f921c0ccb70 0x7f921c15cba0 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:55.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.315+0000 7f9224800640 1 --2- 172.21.15.131:0/1291215125 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f921c0ad450 0x7f921c15c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:55.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.315+0000 7f9224800640 1 --2- 172.21.15.131:0/1291215125 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f921c071d80 0x7f921c144430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:55.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.315+0000 7f9224800640 1 -- 172.21.15.131:0/1291215125 >> 172.21.15.131:0/1291215125 conn(0x7f921c0b1e90 msgr2=0x7f921c0b0670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:56:55.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.315+0000 7f9224800640 1 -- 172.21.15.131:0/1291215125 shutdown_connections 2024-05-26T22:56:55.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.315+0000 7f9224800640 1 -- 172.21.15.131:0/1291215125 wait complete. 2024-05-26T22:56:55.333 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-26T22:56:55.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.523+0000 7efef5e00640 1 Processor -- start 2024-05-26T22:56:55.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.524+0000 7efef5e00640 1 -- start start 2024-05-26T22:56:55.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.525+0000 7efef5e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efef0155770 0x7efef015b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:55.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.525+0000 7efef5e00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efef015c1c0 con 0x7efef015bc40 2024-05-26T22:56:55.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.525+0000 7efef5e00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efef015c4a0 con 0x7efef006e440 2024-05-26T22:56:55.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.525+0000 7efef5e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7efef0071da0 con 0x7efef0155770 2024-05-26T22:56:55.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.526+0000 7efeee000640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7efef015bc40 0x7efef006dd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:33724/0 (socket says 172.21.15.131:33724) 2024-05-26T22:56:55.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.526+0000 7efeee000640 1 -- 172.21.15.131:0/1617092005 learned_addr learned my addr 172.21.15.131:0/1617092005 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:56:55.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.526+0000 7efeed600640 1 -- 172.21.15.131:0/1617092005 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1344763893 0 0) 0x7efef015c1c0 con 0x7efef015bc40 2024-05-26T22:56:55.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.526+0000 7efeeea00640 1 --2- 172.21.15.131:0/1617092005 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efef0155770 0x7efef015b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:55.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.526+0000 7efeed600640 1 -- 172.21.15.131:0/1617092005 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efed4003520 con 0x7efef015bc40 2024-05-26T22:56:55.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.526+0000 7efeed600640 1 -- 172.21.15.131:0/1617092005 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3475647892 0 0) 0x7efed4003520 con 0x7efef015bc40 2024-05-26T22:56:55.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.526+0000 7efeed600640 1 -- 172.21.15.131:0/1617092005 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efef0155770 msgr2=0x7efef015b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:55.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.526+0000 7efeed600640 1 --2- 172.21.15.131:0/1617092005 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efef0155770 0x7efef015b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:55.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.526+0000 7efeed600640 1 -- 172.21.15.131:0/1617092005 >> v1:172.21.15.187:6789/0 conn(0x7efef006e440 legacy=0x7efef00725a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:55.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.526+0000 7efeed600640 1 -- 172.21.15.131:0/1617092005 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efef0072080 con 0x7efef015bc40 2024-05-26T22:56:55.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.526+0000 7efeed600640 1 -- 172.21.15.131:0/1617092005 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7efed8001670 con 0x7efef015bc40 2024-05-26T22:56:55.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.526+0000 7efeed600640 1 -- 172.21.15.131:0/1617092005 >> v1:172.21.15.131:6789/0 conn(0x7efef015bc40 legacy=0x7efef006dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:55.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.527+0000 7efeed600640 1 --2- 172.21.15.131:0/1617092005 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efef0155770 0x7efed40039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:55.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.527+0000 7efeed600640 1 --2- 172.21.15.131:0/1617092005 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efed4004010 0x7efed4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:55.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.527+0000 7efeed600640 1 --2- 172.21.15.131:0/1617092005 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efed4004d40 0x7efed40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:55.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.527+0000 7efeed600640 1 -- 172.21.15.131:0/1617092005 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7efef0071da0 con 0x7efed4004010 2024-05-26T22:56:55.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.527+0000 7efeed600640 1 -- 172.21.15.131:0/1617092005 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7efed400de20 con 0x7efed4004d40 2024-05-26T22:56:55.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.527+0000 7efeed600640 1 -- 172.21.15.131:0/1617092005 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7efed400e020 con 0x7efef0155770 2024-05-26T22:56:55.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.527+0000 7efeee000640 1 --2- 172.21.15.131:0/1617092005 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efed4004010 0x7efed4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:55.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.527+0000 7efeeea00640 1 --2- 172.21.15.131:0/1617092005 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efef0155770 0x7efed40039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:55.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.527+0000 7efeef400640 1 --2- 172.21.15.131:0/1617092005 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efed4004d40 0x7efed40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:55.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.527+0000 7efeee000640 1 -- 172.21.15.131:0/1617092005 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efef0155770 msgr2=0x7efed40039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:55.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.527+0000 7efeee000640 1 --2- 172.21.15.131:0/1617092005 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efef0155770 0x7efed40039e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:55.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.527+0000 7efeee000640 1 -- 172.21.15.131:0/1617092005 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efed4004d40 msgr2=0x7efed40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:55.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.527+0000 7efeee000640 1 --2- 172.21.15.131:0/1617092005 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efed4004d40 0x7efed40051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:55.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.527+0000 7efeee000640 1 -- 172.21.15.131:0/1617092005 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efed4003520 con 0x7efed4004010 2024-05-26T22:56:55.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.528+0000 7efeee000640 1 --2- 172.21.15.131:0/1617092005 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efed4004010 0x7efed4004800 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7efed800c240 tx=0x7efed4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72ed1c758e1c1898 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:55.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.529+0000 7efeed600640 1 -- 172.21.15.131:0/1617092005 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed8007cf0 con 0x7efed4004010 2024-05-26T22:56:55.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.529+0000 7efeed600640 1 -- 172.21.15.131:0/1617092005 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efed80058d0 con 0x7efed4004010 2024-05-26T22:56:55.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.529+0000 7efeed600640 1 -- 172.21.15.131:0/1617092005 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed8005d20 con 0x7efed4004010 2024-05-26T22:56:55.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.529+0000 7efef5e00640 1 -- 172.21.15.131:0/1617092005 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efed4004010 msgr2=0x7efed4004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:55.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.529+0000 7efef5e00640 1 --2- 172.21.15.131:0/1617092005 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efed4004010 0x7efed4004800 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7efed800c240 tx=0x7efed4005830 comp rx=0 tx=0).stop 2024-05-26T22:56:55.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.529+0000 7efeef400640 1 -- 172.21.15.131:0/1617092005 reap_dead start 2024-05-26T22:56:55.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.529+0000 7efef5e00640 1 -- 172.21.15.131:0/1617092005 shutdown_connections 2024-05-26T22:56:55.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.529+0000 7efef5e00640 1 -- 172.21.15.131:0/1617092005 >> 172.21.15.131:0/1617092005 conn(0x7efef006bc90 msgr2=0x7efef00b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:56:55.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.529+0000 7efef5e00640 1 -- 172.21.15.131:0/1617092005 shutdown_connections 2024-05-26T22:56:55.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.529+0000 7efef5e00640 1 -- 172.21.15.131:0/1617092005 wait complete. 2024-05-26T22:56:55.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.531+0000 7efef5e00640 1 Processor -- start 2024-05-26T22:56:55.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.531+0000 7efef5e00640 1 -- start start 2024-05-26T22:56:55.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.532+0000 7efef5e00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efed4004010 0x7efef01f3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:55.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.532+0000 7efef5e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efef014c770 0x7efef00724a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:55.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.532+0000 7efef5e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efef014cbf0 0x7efef00729e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:55.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.532+0000 7efef5e00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7efef009de00 con 0x7efef014cbf0 2024-05-26T22:56:55.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.532+0000 7efef5e00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7efef015c8c0 con 0x7efed4004010 2024-05-26T22:56:55.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.532+0000 7efef5e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7efef009df80 con 0x7efef014c770 2024-05-26T22:56:55.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.532+0000 7efeef400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efef014cbf0 0x7efef00729e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:55.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.533+0000 7efeef400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efef014cbf0 0x7efef00729e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40968/0 (socket says 172.21.15.131:40968) 2024-05-26T22:56:55.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.533+0000 7efeef400640 1 -- 172.21.15.131:0/408497166 learned_addr learned my addr 172.21.15.131:0/408497166 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:56:55.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.533+0000 7efeef400640 1 -- 172.21.15.131:0/408497166 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efef014c770 msgr2=0x7efef00724a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:55.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.533+0000 7efeef400640 1 --2- 172.21.15.131:0/408497166 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efef014c770 0x7efef00724a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:55.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.533+0000 7efeef400640 1 -- 172.21.15.131:0/408497166 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efed4004010 msgr2=0x7efef01f3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:55.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.533+0000 7efeef400640 1 --2- 172.21.15.131:0/408497166 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efed4004010 0x7efef01f3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:55.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.533+0000 7efeef400640 1 -- 172.21.15.131:0/408497166 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efef0151600 con 0x7efef014cbf0 2024-05-26T22:56:55.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.533+0000 7efeef400640 1 --2- 172.21.15.131:0/408497166 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efef014cbf0 0x7efef00729e0 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7efef00cfc40 tx=0x7efee800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:55.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.533+0000 7efee7400640 1 -- 172.21.15.131:0/408497166 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee8018070 con 0x7efef014cbf0 2024-05-26T22:56:55.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.534+0000 7efee7400640 1 -- 172.21.15.131:0/408497166 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efee8003b70 con 0x7efef014cbf0 2024-05-26T22:56:55.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.534+0000 7efee7400640 1 -- 172.21.15.131:0/408497166 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee8005070 con 0x7efef014cbf0 2024-05-26T22:56:55.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.534+0000 7efef5e00640 1 -- 172.21.15.131:0/408497166 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efef01518e0 con 0x7efef014cbf0 2024-05-26T22:56:55.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.534+0000 7efef5e00640 1 -- 172.21.15.131:0/408497166 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efef0151ea0 con 0x7efef014cbf0 2024-05-26T22:56:55.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.535+0000 7efef5e00640 1 -- 172.21.15.131:0/408497166 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efef0074e90 con 0x7efef014cbf0 2024-05-26T22:56:55.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.536+0000 7efee7400640 1 -- 172.21.15.131:0/408497166 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7efee801b070 con 0x7efef014cbf0 2024-05-26T22:56:55.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.537+0000 7efee7400640 1 --2- 172.21.15.131:0/408497166 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7efecc06be30 0x7efecc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:55.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.537+0000 7efee7400640 1 -- 172.21.15.131:0/408497166 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5709+0+0 (secure 0 0 0) 0x7efee808a770 con 0x7efef014cbf0 2024-05-26T22:56:55.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.537+0000 7efeeea00640 1 --2- 172.21.15.131:0/408497166 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7efecc06be30 0x7efecc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:55.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.537+0000 7efeeea00640 1 --2- 172.21.15.131:0/408497166 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7efecc06be30 0x7efecc06e2f0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7efee0006a90 tx=0x7efee0006a20 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:55.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.541+0000 7efee7400640 1 -- 172.21.15.131:0/408497166 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7efee8059110 con 0x7efef014cbf0 2024-05-26T22:56:55.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.781+0000 7efef5e00640 1 -- 172.21.15.131:0/408497166 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7efef006d570 con 0x7efef014cbf0 2024-05-26T22:56:55.782 INFO:tasks.ceph.mon.a.smithi131.stderr:2024-05-26T22:56:55.782+0000 7fcf0d200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-26T22:56:55.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.786+0000 7efee7400640 1 -- 172.21.15.131:0/408497166 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v119) ==== 161+0+0 (secure 0 0 0) 0x7efee8058a10 con 0x7efef014cbf0 2024-05-26T22:56:55.786 INFO:teuthology.orchestra.run.smithi131.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-26T22:56:55.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.789+0000 7efef5e00640 1 -- 172.21.15.131:0/408497166 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7efecc06be30 msgr2=0x7efecc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:55.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.789+0000 7efef5e00640 1 --2- 172.21.15.131:0/408497166 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7efecc06be30 0x7efecc06e2f0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7efee0006a90 tx=0x7efee0006a20 comp rx=0 tx=0).stop 2024-05-26T22:56:55.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.789+0000 7efef5e00640 1 -- 172.21.15.131:0/408497166 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efef014cbf0 msgr2=0x7efef00729e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:55.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.789+0000 7efef5e00640 1 --2- 172.21.15.131:0/408497166 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efef014cbf0 0x7efef00729e0 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7efef00cfc40 tx=0x7efee800dc00 comp rx=0 tx=0).stop 2024-05-26T22:56:55.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.790+0000 7efef5e00640 1 -- 172.21.15.131:0/408497166 shutdown_connections 2024-05-26T22:56:55.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.790+0000 7efef5e00640 1 --2- 172.21.15.131:0/408497166 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efef014c770 0x7efef00724a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:55.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.790+0000 7efef5e00640 1 --2- 172.21.15.131:0/408497166 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7efecc06be30 0x7efecc06e2f0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:55.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.790+0000 7efef5e00640 1 --2- 172.21.15.131:0/408497166 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efef014cbf0 0x7efef00729e0 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:55.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.790+0000 7efef5e00640 1 --2- 172.21.15.131:0/408497166 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efed4004010 0x7efef01f3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:55.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.790+0000 7efef5e00640 1 -- 172.21.15.131:0/408497166 >> 172.21.15.131:0/408497166 conn(0x7efef006bc90 msgr2=0x7efef015af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:56:55.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.790+0000 7efef5e00640 1 -- 172.21.15.131:0/408497166 shutdown_connections 2024-05-26T22:56:55.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:55.790+0000 7efef5e00640 1 -- 172.21.15.131:0/408497166 wait complete. 2024-05-26T22:56:55.810 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-26T22:56:55.890 INFO:tasks.ceph.mds.c.smithi131.stdout:starting mds.c at 2024-05-26T22:56:55.891 INFO:tasks.ceph.mds.a.smithi131.stdout:starting mds.a at 2024-05-26T22:56:55.920 INFO:tasks.ceph.mds.d.smithi187.stdout:starting mds.d at 2024-05-26T22:56:56.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:56.002+0000 7fdddec00640 1 Processor -- start 2024-05-26T22:56:56.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:56.002+0000 7fdddec00640 1 -- start start 2024-05-26T22:56:56.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:56.004+0000 7fdddec00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fddd806f180 0x7fddd8072e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:56.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:56.004+0000 7fdddec00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fddd8072540 0x7fddd8161ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:56.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:56.004+0000 7fdddec00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fddd80cfe50 0x7fddd80d02e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:56.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:56.004+0000 7fdddec00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fddd80cfb30 con 0x7fddd8072540 2024-05-26T22:56:56.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:56.004+0000 7fdddec00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fddd80af8e0 con 0x7fddd806f180 2024-05-26T22:56:56.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:56.004+0000 7fdddec00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fddd80cf200 con 0x7fddd80cfe50 2024-05-26T22:56:56.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:56.004+0000 7fddd7400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fddd8072540 0x7fddd8161ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:56.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:56.004+0000 7fddd7400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fddd8072540 0x7fddd8161ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41012/0 (socket says 172.21.15.131:41012) 2024-05-26T22:56:56.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:56.004+0000 7fddd7400640 1 -- 172.21.15.131:0/358687168 learned_addr learned my addr 172.21.15.131:0/358687168 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:56:56.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:56.004+0000 7fdddcc00640 1 --2- 172.21.15.131:0/358687168 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fddd80cfe50 0x7fddd80d02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:56.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:56.004+0000 7fddd7e00640 1 --2- 172.21.15.131:0/358687168 >> v2:172.21.15.187:3300/0 conn(0x7fddd806f180 0x7fddd8072e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:56.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:56.005+0000 7fddd7400640 1 -- 172.21.15.131:0/358687168 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fddd80cfe50 msgr2=0x7fddd80d02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:56.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:56.005+0000 7fddd7400640 1 --2- 172.21.15.131:0/358687168 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fddd80cfe50 0x7fddd80d02e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:56.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:56.005+0000 7fddd7400640 1 -- 172.21.15.131:0/358687168 >> v2:172.21.15.187:3300/0 conn(0x7fddd806f180 msgr2=0x7fddd8072e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:56.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:56.005+0000 7fddd7400640 1 --2- 172.21.15.131:0/358687168 >> v2:172.21.15.187:3300/0 conn(0x7fddd806f180 0x7fddd8072e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:56.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:56.005+0000 7fddd7400640 1 -- 172.21.15.131:0/358687168 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddd80cf4e0 con 0x7fddd8072540 2024-05-26T22:56:56.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:56.006+0000 7fddd7400640 1 --2- 172.21.15.131:0/358687168 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fddd8072540 0x7fddd8161ff0 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7fddd80d0c40 tx=0x7fddc002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b459bab50c7d5d81 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:56.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:56.006+0000 7fddd6a00640 1 -- 172.21.15.131:0/358687168 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddc0037070 con 0x7fddd8072540 2024-05-26T22:56:56.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:56.006+0000 7fddd6a00640 1 -- 172.21.15.131:0/358687168 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddc0032440 con 0x7fddd8072540 2024-05-26T22:56:56.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:56.006+0000 7fddd6a00640 1 -- 172.21.15.131:0/358687168 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddc0032810 con 0x7fddd8072540 2024-05-26T22:56:56.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:56.007+0000 7fdddec00640 1 -- 172.21.15.131:0/358687168 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fddd8072540 msgr2=0x7fddd8161ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:56.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:56.007+0000 7fdddec00640 1 --2- 172.21.15.131:0/358687168 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fddd8072540 0x7fddd8161ff0 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7fddd80d0c40 tx=0x7fddc002a1e0 comp rx=0 tx=0).stop 2024-05-26T22:56:56.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:56.007+0000 7fdddec00640 1 -- 172.21.15.131:0/358687168 shutdown_connections 2024-05-26T22:56:56.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:56.007+0000 7fdddec00640 1 --2- 172.21.15.131:0/358687168 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fddd80cfe50 0x7fddd80d02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:56.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:56.007+0000 7fdddec00640 1 --2- 172.21.15.131:0/358687168 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fddd8072540 0x7fddd8161ff0 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:56.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:56.007+0000 7fdddec00640 1 --2- 172.21.15.131:0/358687168 >> v2:172.21.15.187:3300/0 conn(0x7fddd806f180 0x7fddd8072e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:56.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:56.007+0000 7fdddec00640 1 -- 172.21.15.131:0/358687168 >> 172.21.15.131:0/358687168 conn(0x7fddd806bc90 msgr2=0x7fddd80b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:56:56.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:56.007+0000 7fdddec00640 1 -- 172.21.15.131:0/358687168 shutdown_connections 2024-05-26T22:56:56.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:56.007+0000 7fdddec00640 1 -- 172.21.15.131:0/358687168 wait complete. 2024-05-26T22:56:56.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:56.009+0000 7fdddec00640 1 Processor -- start 2024-05-26T22:56:56.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:56.009+0000 7fdddec00640 1 -- start start 2024-05-26T22:56:56.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:56.010+0000 7fdddec00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fddd806f180 0x7fddd8151b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:56.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:56.010+0000 7fdddec00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fddd8072540 0x7fddd80cd350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:56.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:56.010+0000 7fdddec00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fddd80cfe50 0x7fddd809c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:56.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:56.010+0000 7fdddec00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fddd806f6e0 con 0x7fddd80cfe50 2024-05-26T22:56:56.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:56.010+0000 7fdddec00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fddd806f560 con 0x7fddd806f180 2024-05-26T22:56:56.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:56.010+0000 7fdddec00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fddd80715a0 con 0x7fddd8072540 2024-05-26T22:56:56.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:56.010+0000 7fdddcc00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fddd80cfe50 0x7fddd809c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:56.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:56.010+0000 7fdddcc00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fddd80cfe50 0x7fddd809c0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41026/0 (socket says 172.21.15.131:41026) 2024-05-26T22:56:56.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:56.010+0000 7fdddcc00640 1 -- 172.21.15.131:0/737480500 learned_addr learned my addr 172.21.15.131:0/737480500 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:56:56.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:56.010+0000 7fddd7400640 1 --2- 172.21.15.131:0/737480500 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fddd8072540 0x7fddd80cd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:56.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:56.010+0000 7fddd7e00640 1 --2- 172.21.15.131:0/737480500 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fddd806f180 0x7fddd8151b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:56.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:56.010+0000 7fdddcc00640 1 -- 172.21.15.131:0/737480500 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fddd8072540 msgr2=0x7fddd80cd350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:56.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:56.010+0000 7fdddcc00640 1 --2- 172.21.15.131:0/737480500 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fddd8072540 0x7fddd80cd350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:56.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:56.010+0000 7fdddcc00640 1 -- 172.21.15.131:0/737480500 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fddd806f180 msgr2=0x7fddd8151b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:56.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:56.010+0000 7fdddcc00640 1 --2- 172.21.15.131:0/737480500 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fddd806f180 0x7fddd8151b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:56.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:56.010+0000 7fdddcc00640 1 -- 172.21.15.131:0/737480500 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddd809c620 con 0x7fddd80cfe50 2024-05-26T22:56:56.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:56.010+0000 7fddd7e00640 1 --2- 172.21.15.131:0/737480500 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fddd806f180 0x7fddd8151b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:56:56.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:56.011+0000 7fdddcc00640 1 --2- 172.21.15.131:0/737480500 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fddd80cfe50 0x7fddd809c0e0 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7fddc800d600 tx=0x7fddc800dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:56.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:56.011+0000 7fddcfe00640 1 -- 172.21.15.131:0/737480500 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddc80125f0 con 0x7fddd80cfe50 2024-05-26T22:56:56.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:56.011+0000 7fddcfe00640 1 -- 172.21.15.131:0/737480500 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddc8019070 con 0x7fddd80cfe50 2024-05-26T22:56:56.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:56.011+0000 7fdddec00640 1 -- 172.21.15.131:0/737480500 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddd80cd9e0 con 0x7fddd80cfe50 2024-05-26T22:56:56.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:56.011+0000 7fddcfe00640 1 -- 172.21.15.131:0/737480500 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddc80129f0 con 0x7fddd80cfe50 2024-05-26T22:56:56.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:56.011+0000 7fdddec00640 1 -- 172.21.15.131:0/737480500 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddd80cdfa0 con 0x7fddd80cfe50 2024-05-26T22:56:56.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:56.012+0000 7fdddec00640 1 -- 172.21.15.131:0/737480500 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddd8074e90 con 0x7fddd80cfe50 2024-05-26T22:56:56.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:56.013+0000 7fddcfe00640 1 -- 172.21.15.131:0/737480500 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fddc8012b90 con 0x7fddd80cfe50 2024-05-26T22:56:56.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:56.015+0000 7fddcfe00640 1 --2- 172.21.15.131:0/737480500 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fddb406be30 0x7fddb406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:56.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:56.015+0000 7fddcfe00640 1 -- 172.21.15.131:0/737480500 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 5967+0+0 (secure 0 0 0) 0x7fddc808b3f0 con 0x7fddd80cfe50 2024-05-26T22:56:56.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:56.015+0000 7fddd7e00640 1 --2- 172.21.15.131:0/737480500 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fddb406be30 0x7fddb406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:56.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:56.016+0000 7fddd7e00640 1 --2- 172.21.15.131:0/737480500 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fddb406be30 0x7fddb406e2f0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fddd0009f70 tx=0x7fddd000b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:56.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:56.019+0000 7fddcfe00640 1 -- 172.21.15.131:0/737480500 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fddc8059c20 con 0x7fddd80cfe50 2024-05-26T22:56:56.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:56.255+0000 7fdddec00640 1 -- 172.21.15.131:0/737480500 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fddd80695c0 con 0x7fddd80cfe50 2024-05-26T22:56:56.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:56.790+0000 7fddcfe00640 1 -- 172.21.15.131:0/737480500 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v120) ==== 104+0+0 (secure 0 0 0) 0x7fddc8059520 con 0x7fddd80cfe50 2024-05-26T22:56:56.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:56.794+0000 7fdddec00640 1 -- 172.21.15.131:0/737480500 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fddb406be30 msgr2=0x7fddb406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:56.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:56.794+0000 7fdddec00640 1 --2- 172.21.15.131:0/737480500 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fddb406be30 0x7fddb406e2f0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fddd0009f70 tx=0x7fddd000b040 comp rx=0 tx=0).stop 2024-05-26T22:56:56.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:56.794+0000 7fdddec00640 1 -- 172.21.15.131:0/737480500 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fddd80cfe50 msgr2=0x7fddd809c0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:56.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:56.794+0000 7fdddec00640 1 --2- 172.21.15.131:0/737480500 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fddd80cfe50 0x7fddd809c0e0 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7fddc800d600 tx=0x7fddc800dac0 comp rx=0 tx=0).stop 2024-05-26T22:56:56.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:56.794+0000 7fdddec00640 1 -- 172.21.15.131:0/737480500 shutdown_connections 2024-05-26T22:56:56.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:56.794+0000 7fdddec00640 1 --2- 172.21.15.131:0/737480500 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fddd8072540 0x7fddd80cd350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:56.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:56.794+0000 7fdddec00640 1 --2- 172.21.15.131:0/737480500 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fddb406be30 0x7fddb406e2f0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:56.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:56.794+0000 7fdddec00640 1 --2- 172.21.15.131:0/737480500 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fddd80cfe50 0x7fddd809c0e0 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:56.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:56.794+0000 7fdddec00640 1 --2- 172.21.15.131:0/737480500 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fddd806f180 0x7fddd8151b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:56.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:56.794+0000 7fdddec00640 1 -- 172.21.15.131:0/737480500 >> 172.21.15.131:0/737480500 conn(0x7fddd806bc90 msgr2=0x7fddd815de20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:56:56.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:56.794+0000 7fdddec00640 1 -- 172.21.15.131:0/737480500 shutdown_connections 2024-05-26T22:56:56.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:56.794+0000 7fdddec00640 1 -- 172.21.15.131:0/737480500 wait complete. 2024-05-26T22:56:56.815 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-26T22:56:57.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:57.011+0000 7f46db000640 1 Processor -- start 2024-05-26T22:56:57.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:57.011+0000 7f46db000640 1 -- start start 2024-05-26T22:56:57.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:57.013+0000 7f46db000640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f46d4145880 0x7f46d415cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:57.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:57.013+0000 7f46db000640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f46d415fba0 0x7f46d406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:57.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:57.013+0000 7f46db000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f46d4160020 0x7f46d406da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:57.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:57.013+0000 7f46db000640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f46d415c560 con 0x7f46d4145880 2024-05-26T22:56:57.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:57.013+0000 7f46db000640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f46d415cb10 con 0x7f46d415fba0 2024-05-26T22:56:57.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:57.013+0000 7f46db000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f46d415d4e0 con 0x7f46d4160020 2024-05-26T22:56:57.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:57.014+0000 7f46d9c00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f46d4145880 0x7f46d415cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:57.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:57.014+0000 7f46d9c00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f46d4145880 0x7f46d415cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41028/0 (socket says 172.21.15.131:41028) 2024-05-26T22:56:57.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:57.014+0000 7f46d9c00640 1 -- 172.21.15.131:0/1860720599 learned_addr learned my addr 172.21.15.131:0/1860720599 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:56:57.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:57.014+0000 7f46d9200640 1 --2- 172.21.15.131:0/1860720599 >> v2:172.21.15.187:3300/0 conn(0x7f46d415fba0 0x7f46d406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:57.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:57.014+0000 7f46da600640 1 --2- 172.21.15.131:0/1860720599 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f46d4160020 0x7f46d406da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:57.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:57.014+0000 7f46d9c00640 1 -- 172.21.15.131:0/1860720599 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f46d4160020 msgr2=0x7f46d406da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:57.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:57.014+0000 7f46d9c00640 1 --2- 172.21.15.131:0/1860720599 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f46d4160020 0x7f46d406da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:57.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:57.014+0000 7f46d9c00640 1 -- 172.21.15.131:0/1860720599 >> v2:172.21.15.187:3300/0 conn(0x7f46d415fba0 msgr2=0x7f46d406d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:57.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:57.014+0000 7f46d9c00640 1 --2- 172.21.15.131:0/1860720599 >> v2:172.21.15.187:3300/0 conn(0x7f46d415fba0 0x7f46d406d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:57.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:57.014+0000 7f46d9c00640 1 -- 172.21.15.131:0/1860720599 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46d4145c60 con 0x7f46d4145880 2024-05-26T22:56:57.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:57.014+0000 7f46d9200640 1 --2- 172.21.15.131:0/1860720599 >> v2:172.21.15.187:3300/0 conn(0x7f46d415fba0 0x7f46d406d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-26T22:56:57.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:57.016+0000 7f46d9c00640 1 --2- 172.21.15.131:0/1860720599 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f46d4145880 0x7f46d415cfa0 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f46d40ce7d0 tx=0x7f46c802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0fd60e271d4fdc6 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:57.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:57.016+0000 7f46d3e00640 1 -- 172.21.15.131:0/1860720599 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c8037070 con 0x7f46d4145880 2024-05-26T22:56:57.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:57.016+0000 7f46d3e00640 1 -- 172.21.15.131:0/1860720599 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46c8032440 con 0x7f46d4145880 2024-05-26T22:56:57.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:57.016+0000 7f46d3e00640 1 -- 172.21.15.131:0/1860720599 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c8032810 con 0x7f46d4145880 2024-05-26T22:56:57.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:57.017+0000 7f46db000640 1 -- 172.21.15.131:0/1860720599 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f46d4145880 msgr2=0x7f46d415cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:57.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:57.017+0000 7f46db000640 1 --2- 172.21.15.131:0/1860720599 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f46d4145880 0x7f46d415cfa0 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f46d40ce7d0 tx=0x7f46c802a1e0 comp rx=0 tx=0).stop 2024-05-26T22:56:57.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:57.017+0000 7f46db000640 1 -- 172.21.15.131:0/1860720599 shutdown_connections 2024-05-26T22:56:57.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:57.017+0000 7f46db000640 1 --2- 172.21.15.131:0/1860720599 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f46d4160020 0x7f46d406da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:57.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:57.017+0000 7f46db000640 1 --2- 172.21.15.131:0/1860720599 >> v2:172.21.15.187:3300/0 conn(0x7f46d415fba0 0x7f46d406d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:57.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:57.017+0000 7f46db000640 1 --2- 172.21.15.131:0/1860720599 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f46d4145880 0x7f46d415cfa0 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:57.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:57.017+0000 7f46db000640 1 -- 172.21.15.131:0/1860720599 >> 172.21.15.131:0/1860720599 conn(0x7f46d40b5410 msgr2=0x7f46d40b57f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:56:57.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:57.017+0000 7f46db000640 1 -- 172.21.15.131:0/1860720599 shutdown_connections 2024-05-26T22:56:57.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:57.017+0000 7f46db000640 1 -- 172.21.15.131:0/1860720599 wait complete. 2024-05-26T22:56:57.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:57.019+0000 7f46db000640 1 Processor -- start 2024-05-26T22:56:57.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:57.019+0000 7f46db000640 1 -- start start 2024-05-26T22:56:57.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:57.020+0000 7f46db000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f46d4145880 0x7f46d4072c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:57.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:57.020+0000 7f46db000640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f46d415fba0 0x7f46d41e80f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:57.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:57.020+0000 7f46db000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f46d4160020 0x7f46d4140000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:57.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:57.020+0000 7f46db000640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f46d415c7a0 con 0x7f46d4145880 2024-05-26T22:56:57.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:57.020+0000 7f46db000640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f46d4160890 con 0x7f46d415fba0 2024-05-26T22:56:57.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:57.020+0000 7f46db000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f46d40b1d10 con 0x7f46d4160020 2024-05-26T22:56:57.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:57.020+0000 7f46d9c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f46d4145880 0x7f46d4072c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:57.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:57.020+0000 7f46d9c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f46d4145880 0x7f46d4072c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41042/0 (socket says 172.21.15.131:41042) 2024-05-26T22:56:57.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:57.020+0000 7f46d9c00640 1 -- 172.21.15.131:0/357077147 learned_addr learned my addr 172.21.15.131:0/357077147 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:56:57.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:57.021+0000 7f46d9c00640 1 -- 172.21.15.131:0/357077147 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f46d4160020 msgr2=0x7f46d4140000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:57.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:57.021+0000 7f46d9c00640 1 --2- 172.21.15.131:0/357077147 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f46d4160020 0x7f46d4140000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:57.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:57.021+0000 7f46d9c00640 1 -- 172.21.15.131:0/357077147 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f46d415fba0 msgr2=0x7f46d41e80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:57.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:57.021+0000 7f46d9c00640 1 --2- 172.21.15.131:0/357077147 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f46d415fba0 0x7f46d41e80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:57.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:57.021+0000 7f46d9c00640 1 -- 172.21.15.131:0/357077147 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46d41430f0 con 0x7f46d4145880 2024-05-26T22:56:57.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:57.021+0000 7f46d9c00640 1 --2- 172.21.15.131:0/357077147 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f46d4145880 0x7f46d4072c50 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f46c802a220 tx=0x7f46c80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:57.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:57.021+0000 7f46d1600640 1 -- 172.21.15.131:0/357077147 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c8041020 con 0x7f46d4145880 2024-05-26T22:56:57.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:57.021+0000 7f46d1600640 1 -- 172.21.15.131:0/357077147 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46c80306c0 con 0x7f46d4145880 2024-05-26T22:56:57.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:57.021+0000 7f46d1600640 1 -- 172.21.15.131:0/357077147 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c8037040 con 0x7f46d4145880 2024-05-26T22:56:57.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:57.021+0000 7f46db000640 1 -- 172.21.15.131:0/357077147 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46d41433d0 con 0x7f46d4145880 2024-05-26T22:56:57.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:57.022+0000 7f46db000640 1 -- 172.21.15.131:0/357077147 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46d4143990 con 0x7f46d4145880 2024-05-26T22:56:57.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:57.023+0000 7f46db000640 1 -- 172.21.15.131:0/357077147 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46d40694e0 con 0x7f46d4145880 2024-05-26T22:56:57.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:57.023+0000 7f46d1600640 1 -- 172.21.15.131:0/357077147 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f46c8044020 con 0x7f46d4145880 2024-05-26T22:56:57.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:57.024+0000 7f46d1600640 1 --2- 172.21.15.131:0/357077147 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f46b406be30 0x7f46b406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:57.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:57.025+0000 7f46d1600640 1 -- 172.21.15.131:0/357077147 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 5967+0+0 (secure 0 0 0) 0x7f46c80ab7c0 con 0x7f46d4145880 2024-05-26T22:56:57.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:57.025+0000 7f46d9200640 1 --2- 172.21.15.131:0/357077147 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f46b406be30 0x7f46b406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:57.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:57.025+0000 7f46d9200640 1 --2- 172.21.15.131:0/357077147 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f46b406be30 0x7f46b406e2f0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f46c0006fd0 tx=0x7f46c0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:57.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:57.029+0000 7f46d1600640 1 -- 172.21.15.131:0/357077147 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f46c804b030 con 0x7f46d4145880 2024-05-26T22:56:57.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:57.226+0000 7f46db000640 1 -- 172.21.15.131:0/357077147 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f46d40ccfa0 con 0x7f46d4145880 2024-05-26T22:56:57.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:57.793+0000 7f46d1600640 1 -- 172.21.15.131:0/357077147 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v109) ==== 181+0+0 (secure 0 0 0) 0x7f46c807a060 con 0x7f46d4145880 2024-05-26T22:56:57.794 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_metadata' removed 2024-05-26T22:56:57.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:57.798+0000 7f46db000640 1 -- 172.21.15.131:0/357077147 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f46b406be30 msgr2=0x7f46b406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:57.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:57.798+0000 7f46db000640 1 --2- 172.21.15.131:0/357077147 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f46b406be30 0x7f46b406e2f0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f46c0006fd0 tx=0x7f46c0008040 comp rx=0 tx=0).stop 2024-05-26T22:56:57.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:57.798+0000 7f46db000640 1 -- 172.21.15.131:0/357077147 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f46d4145880 msgr2=0x7f46d4072c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:57.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:57.798+0000 7f46db000640 1 --2- 172.21.15.131:0/357077147 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f46d4145880 0x7f46d4072c50 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f46c802a220 tx=0x7f46c80046c0 comp rx=0 tx=0).stop 2024-05-26T22:56:57.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:57.798+0000 7f46db000640 1 -- 172.21.15.131:0/357077147 shutdown_connections 2024-05-26T22:56:57.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:57.798+0000 7f46db000640 1 --2- 172.21.15.131:0/357077147 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f46d4160020 0x7f46d4140000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:57.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:57.798+0000 7f46db000640 1 --2- 172.21.15.131:0/357077147 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f46b406be30 0x7f46b406e2f0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:57.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:57.798+0000 7f46db000640 1 --2- 172.21.15.131:0/357077147 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f46d415fba0 0x7f46d41e80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:57.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:57.798+0000 7f46db000640 1 --2- 172.21.15.131:0/357077147 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f46d4145880 0x7f46d4072c50 unknown :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:57.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:57.798+0000 7f46db000640 1 -- 172.21.15.131:0/357077147 >> 172.21.15.131:0/357077147 conn(0x7f46d40b5410 msgr2=0x7f46d40b2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:56:57.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:57.800+0000 7f46db000640 1 -- 172.21.15.131:0/357077147 shutdown_connections 2024-05-26T22:56:57.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:57.801+0000 7f46db000640 1 -- 172.21.15.131:0/357077147 wait complete. 2024-05-26T22:56:57.828 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-26T22:56:58.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:58.022+0000 7f1797200640 1 Processor -- start 2024-05-26T22:56:58.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:58.022+0000 7f1797200640 1 -- start start 2024-05-26T22:56:58.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:58.024+0000 7f1797200640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f17900cf8f0 0x7f1790072250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:58.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:58.024+0000 7f1797200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f179006ef70 0x7f17900d07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:58.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:58.024+0000 7f1797200640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17900ccb70 con 0x7f17900d0360 2024-05-26T22:56:58.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:58.024+0000 7f1797200640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f17900d0d00 con 0x7f17900cf8f0 2024-05-26T22:56:58.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:58.024+0000 7f1797200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f17900cce50 con 0x7f179006ef70 2024-05-26T22:56:58.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:58.024+0000 7f1787400640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f17900d0360 0x7f179006e860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:33744/0 (socket says 172.21.15.131:33744) 2024-05-26T22:56:58.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:58.024+0000 7f1787400640 1 -- 172.21.15.131:0/2269464483 learned_addr learned my addr 172.21.15.131:0/2269464483 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:56:58.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:58.025+0000 7f1795200640 1 --2- 172.21.15.131:0/2269464483 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f179006ef70 0x7f17900d07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:58.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:58.025+0000 7f178fe00640 1 --2- 172.21.15.131:0/2269464483 >> v2:172.21.15.187:3300/0 conn(0x7f17900cf8f0 0x7f1790072250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:58.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:58.025+0000 7f178f400640 1 -- 172.21.15.131:0/2269464483 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 561264563 0 0) 0x7f17900ccb70 con 0x7f17900d0360 2024-05-26T22:56:58.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:58.025+0000 7f178f400640 1 -- 172.21.15.131:0/2269464483 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1774003520 con 0x7f17900d0360 2024-05-26T22:56:58.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:58.025+0000 7f178f400640 1 -- 172.21.15.131:0/2269464483 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4051753006 0 0) 0x7f1774003520 con 0x7f17900d0360 2024-05-26T22:56:58.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:58.025+0000 7f178f400640 1 -- 172.21.15.131:0/2269464483 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f179006ef70 msgr2=0x7f17900d07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:58.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:58.025+0000 7f178f400640 1 --2- 172.21.15.131:0/2269464483 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f179006ef70 0x7f17900d07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:58.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:58.025+0000 7f178f400640 1 -- 172.21.15.131:0/2269464483 >> v2:172.21.15.187:3300/0 conn(0x7f17900cf8f0 msgr2=0x7f1790072250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:58.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:58.025+0000 7f178f400640 1 --2- 172.21.15.131:0/2269464483 >> v2:172.21.15.187:3300/0 conn(0x7f17900cf8f0 0x7f1790072250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:58.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:58.025+0000 7f178f400640 1 -- 172.21.15.131:0/2269464483 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17900cd130 con 0x7f17900d0360 2024-05-26T22:56:58.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:58.025+0000 7f178f400640 1 -- 172.21.15.131:0/2269464483 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7f17780019f0 con 0x7f17900d0360 2024-05-26T22:56:58.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:58.025+0000 7f178f400640 1 -- 172.21.15.131:0/2269464483 >> v1:172.21.15.131:6789/0 conn(0x7f17900d0360 legacy=0x7f179006e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:58.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:58.025+0000 7f178f400640 1 --2- 172.21.15.131:0/2269464483 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f17740039b0 0x7f1774003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:58.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:58.025+0000 7f178f400640 1 --2- 172.21.15.131:0/2269464483 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f179006ef70 0x7f1774004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:58.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:58.025+0000 7f178f400640 1 --2- 172.21.15.131:0/2269464483 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1774004d40 0x7f17740051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:58.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:58.025+0000 7f178f400640 1 -- 172.21.15.131:0/2269464483 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f17900cce50 con 0x7f1774004d40 2024-05-26T22:56:58.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:58.025+0000 7f178f400640 1 -- 172.21.15.131:0/2269464483 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f177400b9f0 con 0x7f17740039b0 2024-05-26T22:56:58.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:58.025+0000 7f178f400640 1 -- 172.21.15.131:0/2269464483 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f177400bbf0 con 0x7f179006ef70 2024-05-26T22:56:58.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:58.026+0000 7f1795200640 1 --2- 172.21.15.131:0/2269464483 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1774004d40 0x7f17740051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:58.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:58.026+0000 7f178fe00640 1 --2- 172.21.15.131:0/2269464483 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f17740039b0 0x7f1774003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:58.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:58.026+0000 7f1787400640 1 --2- 172.21.15.131:0/2269464483 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f179006ef70 0x7f1774004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:58.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:58.026+0000 7f1795200640 1 -- 172.21.15.131:0/2269464483 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f179006ef70 msgr2=0x7f1774004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:58.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:58.026+0000 7f1795200640 1 --2- 172.21.15.131:0/2269464483 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f179006ef70 0x7f1774004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:58.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:58.026+0000 7f1795200640 1 -- 172.21.15.131:0/2269464483 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f17740039b0 msgr2=0x7f1774003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:58.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:58.026+0000 7f1795200640 1 --2- 172.21.15.131:0/2269464483 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f17740039b0 0x7f1774003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:58.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:58.026+0000 7f1795200640 1 -- 172.21.15.131:0/2269464483 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1774003520 con 0x7f1774004d40 2024-05-26T22:56:58.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:58.027+0000 7f1795200640 1 --2- 172.21.15.131:0/2269464483 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1774004d40 0x7f17740051c0 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f1788009a90 tx=0x7f1774005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=587be49bf9cd4a74 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:58.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:58.028+0000 7f178f400640 1 -- 172.21.15.131:0/2269464483 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1788036070 con 0x7f1774004d40 2024-05-26T22:56:58.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:58.028+0000 7f178f400640 1 -- 172.21.15.131:0/2269464483 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1788031400 con 0x7f1774004d40 2024-05-26T22:56:58.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:58.028+0000 7f178f400640 1 -- 172.21.15.131:0/2269464483 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1788009690 con 0x7f1774004d40 2024-05-26T22:56:58.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:58.028+0000 7f1797200640 1 -- 172.21.15.131:0/2269464483 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1774004d40 msgr2=0x7f17740051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:58.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:58.028+0000 7f1797200640 1 --2- 172.21.15.131:0/2269464483 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1774004d40 0x7f17740051c0 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f1788009a90 tx=0x7f1774005830 comp rx=0 tx=0).stop 2024-05-26T22:56:58.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:58.028+0000 7f1795200640 1 -- 172.21.15.131:0/2269464483 reap_dead start 2024-05-26T22:56:58.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:58.028+0000 7f1797200640 1 -- 172.21.15.131:0/2269464483 shutdown_connections 2024-05-26T22:56:58.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:58.028+0000 7f1797200640 1 -- 172.21.15.131:0/2269464483 >> 172.21.15.131:0/2269464483 conn(0x7f17900b2110 msgr2=0x7f17900b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:56:58.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:58.028+0000 7f1797200640 1 -- 172.21.15.131:0/2269464483 shutdown_connections 2024-05-26T22:56:58.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:58.028+0000 7f1797200640 1 -- 172.21.15.131:0/2269464483 wait complete. 2024-05-26T22:56:58.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:58.030+0000 7f1797200640 1 Processor -- start 2024-05-26T22:56:58.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:58.030+0000 7f1797200640 1 -- start start 2024-05-26T22:56:58.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:58.031+0000 7f1797200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1774004d40 0x7f179015c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:58.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:58.031+0000 7f1797200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f179015f1e0 0x7f179015d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:58.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:58.031+0000 7f1797200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f17900cb610 0x7f17901505a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:58.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:58.031+0000 7f1797200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f17900d07c0 con 0x7f179015f1e0 2024-05-26T22:56:58.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:58.031+0000 7f1797200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f17900cfcd0 con 0x7f17900cb610 2024-05-26T22:56:58.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:58.031+0000 7f1797200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f17900d0940 con 0x7f1774004d40 2024-05-26T22:56:58.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:58.031+0000 7f1787400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f179015f1e0 0x7f179015d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:58.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:58.032+0000 7f1787400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f179015f1e0 0x7f179015d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41052/0 (socket says 172.21.15.131:41052) 2024-05-26T22:56:58.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:58.032+0000 7f1787400640 1 -- 172.21.15.131:0/2955343090 learned_addr learned my addr 172.21.15.131:0/2955343090 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:56:58.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:58.032+0000 7f178fe00640 1 --2- 172.21.15.131:0/2955343090 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1774004d40 0x7f179015c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:58.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:58.032+0000 7f1795200640 1 --2- 172.21.15.131:0/2955343090 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f17900cb610 0x7f17901505a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:58.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:58.032+0000 7f1787400640 1 -- 172.21.15.131:0/2955343090 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1774004d40 msgr2=0x7f179015c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:58.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:58.032+0000 7f1787400640 1 --2- 172.21.15.131:0/2955343090 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1774004d40 0x7f179015c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:58.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:58.032+0000 7f1787400640 1 -- 172.21.15.131:0/2955343090 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f17900cb610 msgr2=0x7f17901505a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:58.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:58.032+0000 7f1787400640 1 --2- 172.21.15.131:0/2955343090 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f17900cb610 0x7f17901505a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:58.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:58.032+0000 7f1787400640 1 -- 172.21.15.131:0/2955343090 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17900ca8c0 con 0x7f179015f1e0 2024-05-26T22:56:58.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:58.032+0000 7f1787400640 1 --2- 172.21.15.131:0/2955343090 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f179015f1e0 0x7f179015d4a0 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f1778007b00 tx=0x7f177800e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:58.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:58.033+0000 7f178cc00640 1 -- 172.21.15.131:0/2955343090 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f177800ee90 con 0x7f179015f1e0 2024-05-26T22:56:58.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:58.033+0000 7f178cc00640 1 -- 172.21.15.131:0/2955343090 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1778018070 con 0x7f179015f1e0 2024-05-26T22:56:58.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:58.033+0000 7f178cc00640 1 -- 172.21.15.131:0/2955343090 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1778013640 con 0x7f179015f1e0 2024-05-26T22:56:58.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:58.033+0000 7f1797200640 1 -- 172.21.15.131:0/2955343090 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17900cab30 con 0x7f179015f1e0 2024-05-26T22:56:58.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:58.033+0000 7f1797200640 1 -- 172.21.15.131:0/2955343090 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17900cb0f0 con 0x7f179015f1e0 2024-05-26T22:56:58.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:58.034+0000 7f1797200640 1 -- 172.21.15.131:0/2955343090 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1790068440 con 0x7f179015f1e0 2024-05-26T22:56:58.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:58.035+0000 7f178cc00640 1 -- 172.21.15.131:0/2955343090 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f17780137e0 con 0x7f179015f1e0 2024-05-26T22:56:58.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:58.036+0000 7f178cc00640 1 --2- 172.21.15.131:0/2955343090 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f176c06bfd0 0x7f176c06e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:58.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:58.036+0000 7f178cc00640 1 -- 172.21.15.131:0/2955343090 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5529+0+0 (secure 0 0 0) 0x7f177808b530 con 0x7f179015f1e0 2024-05-26T22:56:58.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:58.036+0000 7f178fe00640 1 --2- 172.21.15.131:0/2955343090 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f176c06bfd0 0x7f176c06e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:58.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:58.037+0000 7f178fe00640 1 --2- 172.21.15.131:0/2955343090 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f176c06bfd0 0x7f176c06e490 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f179015a830 tx=0x7f1780008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:58.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:58.040+0000 7f178cc00640 1 -- 172.21.15.131:0/2955343090 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f1778059f80 con 0x7f179015f1e0 2024-05-26T22:56:58.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:58.241+0000 7f1797200640 1 -- 172.21.15.131:0/2955343090 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f17900ce970 con 0x7f179015f1e0 2024-05-26T22:56:58.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:58.797+0000 7f178cc00640 1 -- 172.21.15.131:0/2955343090 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v110) ==== 169+0+0 (secure 0 0 0) 0x7f1778059880 con 0x7f179015f1e0 2024-05-26T22:56:58.797 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_data' removed 2024-05-26T22:56:58.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:58.800+0000 7f1797200640 1 -- 172.21.15.131:0/2955343090 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f176c06bfd0 msgr2=0x7f176c06e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:58.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:58.800+0000 7f1797200640 1 --2- 172.21.15.131:0/2955343090 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f176c06bfd0 0x7f176c06e490 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f179015a830 tx=0x7f1780008040 comp rx=0 tx=0).stop 2024-05-26T22:56:58.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:58.801+0000 7f1797200640 1 -- 172.21.15.131:0/2955343090 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f179015f1e0 msgr2=0x7f179015d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:58.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:58.801+0000 7f1797200640 1 --2- 172.21.15.131:0/2955343090 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f179015f1e0 0x7f179015d4a0 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f1778007b00 tx=0x7f177800e530 comp rx=0 tx=0).stop 2024-05-26T22:56:58.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:58.801+0000 7f1797200640 1 -- 172.21.15.131:0/2955343090 shutdown_connections 2024-05-26T22:56:58.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:58.801+0000 7f1797200640 1 --2- 172.21.15.131:0/2955343090 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f176c06bfd0 0x7f176c06e490 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:58.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:58.801+0000 7f1797200640 1 --2- 172.21.15.131:0/2955343090 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f17900cb610 0x7f17901505a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:58.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:58.801+0000 7f1797200640 1 --2- 172.21.15.131:0/2955343090 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f179015f1e0 0x7f179015d4a0 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:58.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:58.801+0000 7f1797200640 1 --2- 172.21.15.131:0/2955343090 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1774004d40 0x7f179015c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:58.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:58.801+0000 7f1797200640 1 -- 172.21.15.131:0/2955343090 >> 172.21.15.131:0/2955343090 conn(0x7f17900b2110 msgr2=0x7f17900ae490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:56:58.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:58.802+0000 7f1797200640 1 -- 172.21.15.131:0/2955343090 shutdown_connections 2024-05-26T22:56:58.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:58.802+0000 7f1797200640 1 -- 172.21.15.131:0/2955343090 wait complete. 2024-05-26T22:56:58.837 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_export_ephemeral_random_max 2024-05-26T22:56:59.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.013+0000 7f1e6cc00640 1 Processor -- start 2024-05-26T22:56:59.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.013+0000 7f1e6cc00640 1 -- start start 2024-05-26T22:56:59.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.015+0000 7f1e6cc00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1e680d0560 0x7f1e68072e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:59.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.015+0000 7f1e6cc00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e680af8e0 con 0x7f1e6806f180 2024-05-26T22:56:59.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.015+0000 7f1e6cc00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e680cf200 con 0x7f1e68072540 2024-05-26T22:56:59.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.015+0000 7f1e6cc00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1e680cfb30 con 0x7f1e680d0560 2024-05-26T22:56:59.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.015+0000 7f1e65e00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f1e6806f180 0x7f1e680cfe50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:33750/0 (socket says 172.21.15.131:33750) 2024-05-26T22:56:59.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.015+0000 7f1e65e00640 1 -- 172.21.15.131:0/443224316 learned_addr learned my addr 172.21.15.131:0/443224316 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:56:59.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.016+0000 7f1e65400640 1 --2- 172.21.15.131:0/443224316 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1e680d0560 0x7f1e68072e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:59.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.017+0000 7f1e65400640 1 -- 172.21.15.131:0/443224316 >> v1:172.21.15.187:6789/0 conn(0x7f1e68072540 legacy=0x7f1e680715a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:59.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.017+0000 7f1e65400640 1 -- 172.21.15.131:0/443224316 >> v1:172.21.15.131:6789/0 conn(0x7f1e6806f180 legacy=0x7f1e680cfe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:59.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.017+0000 7f1e65400640 1 -- 172.21.15.131:0/443224316 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e680cf4e0 con 0x7f1e680d0560 2024-05-26T22:56:59.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.021+0000 7f1e65400640 1 --2- 172.21.15.131:0/443224316 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1e680d0560 0x7f1e68072e90 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1e680cfcb0 tx=0x7f1e5002a0f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4a013b9206024f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:59.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.023+0000 7f1e64a00640 1 -- 172.21.15.131:0/443224316 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e50037070 con 0x7f1e680d0560 2024-05-26T22:56:59.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.023+0000 7f1e64a00640 1 -- 172.21.15.131:0/443224316 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e5002ad80 con 0x7f1e680d0560 2024-05-26T22:56:59.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.023+0000 7f1e64a00640 1 -- 172.21.15.131:0/443224316 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e50032660 con 0x7f1e680d0560 2024-05-26T22:56:59.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.024+0000 7f1e6cc00640 1 -- 172.21.15.131:0/443224316 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1e680d0560 msgr2=0x7f1e68072e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:59.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.024+0000 7f1e6cc00640 1 --2- 172.21.15.131:0/443224316 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1e680d0560 0x7f1e68072e90 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1e680cfcb0 tx=0x7f1e5002a0f0 comp rx=0 tx=0).stop 2024-05-26T22:56:59.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.024+0000 7f1e6cc00640 1 -- 172.21.15.131:0/443224316 shutdown_connections 2024-05-26T22:56:59.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.024+0000 7f1e6cc00640 1 --2- 172.21.15.131:0/443224316 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1e680d0560 0x7f1e68072e90 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:59.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.024+0000 7f1e6cc00640 1 -- 172.21.15.131:0/443224316 >> 172.21.15.131:0/443224316 conn(0x7f1e6806bc90 msgr2=0x7f1e680b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:56:59.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.024+0000 7f1e6cc00640 1 -- 172.21.15.131:0/443224316 shutdown_connections 2024-05-26T22:56:59.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.025+0000 7f1e6cc00640 1 -- 172.21.15.131:0/443224316 wait complete. 2024-05-26T22:56:59.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.026+0000 7f1e6cc00640 1 Processor -- start 2024-05-26T22:56:59.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.027+0000 7f1e6cc00640 1 -- start start 2024-05-26T22:56:59.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.027+0000 7f1e6cc00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1e6806f180 0x7f1e6815ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:59.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.027+0000 7f1e6cc00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1e68072540 0x7f1e6815c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:59.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.028+0000 7f1e6cc00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1e680d0560 0x7f1e6815ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:59.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.028+0000 7f1e6cc00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f1e6806f6e0 con 0x7f1e680d0560 2024-05-26T22:56:59.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.028+0000 7f1e6cc00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1e6806f560 con 0x7f1e68072540 2024-05-26T22:56:59.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.028+0000 7f1e6cc00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1e68161ff0 con 0x7f1e6806f180 2024-05-26T22:56:59.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.028+0000 7f1e66800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1e680d0560 0x7f1e6815ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:59.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.028+0000 7f1e66800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1e680d0560 0x7f1e6815ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41056/0 (socket says 172.21.15.131:41056) 2024-05-26T22:56:59.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.028+0000 7f1e66800640 1 -- 172.21.15.131:0/3258545977 learned_addr learned my addr 172.21.15.131:0/3258545977 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:56:59.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.028+0000 7f1e66800640 1 -- 172.21.15.131:0/3258545977 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1e6806f180 msgr2=0x7f1e6815ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:59.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.028+0000 7f1e66800640 1 --2- 172.21.15.131:0/3258545977 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1e6806f180 0x7f1e6815ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:59.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.028+0000 7f1e66800640 1 -- 172.21.15.131:0/3258545977 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1e68072540 msgr2=0x7f1e6815c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:59.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.028+0000 7f1e66800640 1 --2- 172.21.15.131:0/3258545977 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1e68072540 0x7f1e6815c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:59.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.028+0000 7f1e66800640 1 -- 172.21.15.131:0/3258545977 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e680ad1a0 con 0x7f1e680d0560 2024-05-26T22:56:59.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.029+0000 7f1e66800640 1 --2- 172.21.15.131:0/3258545977 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1e680d0560 0x7f1e6815ce20 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f1e60001eb0 tx=0x7f1e6000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:59.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.029+0000 7f1e5e000640 1 -- 172.21.15.131:0/3258545977 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e60017070 con 0x7f1e680d0560 2024-05-26T22:56:59.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.029+0000 7f1e5e000640 1 -- 172.21.15.131:0/3258545977 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e600052d0 con 0x7f1e680d0560 2024-05-26T22:56:59.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.029+0000 7f1e5e000640 1 -- 172.21.15.131:0/3258545977 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e60003810 con 0x7f1e680d0560 2024-05-26T22:56:59.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.029+0000 7f1e6cc00640 1 -- 172.21.15.131:0/3258545977 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e680ad480 con 0x7f1e680d0560 2024-05-26T22:56:59.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.029+0000 7f1e6cc00640 1 -- 172.21.15.131:0/3258545977 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e680ca110 con 0x7f1e680d0560 2024-05-26T22:56:59.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.031+0000 7f1e6cc00640 1 -- 172.21.15.131:0/3258545977 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e680695c0 con 0x7f1e680d0560 2024-05-26T22:56:59.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.031+0000 7f1e5e000640 1 -- 172.21.15.131:0/3258545977 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1e6001a070 con 0x7f1e680d0560 2024-05-26T22:56:59.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.032+0000 7f1e5e000640 1 --2- 172.21.15.131:0/3258545977 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f1e4406be60 0x7f1e4406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:59.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.032+0000 7f1e5e000640 1 -- 172.21.15.131:0/3258545977 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5115+0+0 (secure 0 0 0) 0x7f1e6008a060 con 0x7f1e680d0560 2024-05-26T22:56:59.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.033+0000 7f1e65e00640 1 --2- 172.21.15.131:0/3258545977 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f1e4406be60 0x7f1e4406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:59.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.033+0000 7f1e65e00640 1 --2- 172.21.15.131:0/3258545977 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f1e4406be60 0x7f1e4406e320 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f1e58001ea0 tx=0x7f1e58007d20 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:59.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.038+0000 7f1e5e000640 1 -- 172.21.15.131:0/3258545977 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f1e60058be0 con 0x7f1e680d0560 2024-05-26T22:56:59.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.236+0000 7f1e6cc00640 1 -- 172.21.15.131:0/3258545977 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_export_ephemeral_random_max"} v 0) -- 0x7f1e6815e290 con 0x7f1e680d0560 2024-05-26T22:56:59.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.241+0000 7f1e5e000640 1 -- 172.21.15.131:0/3258545977 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_export_ephemeral_random_max"}]=0 v63) ==== 114+0+0 (secure 0 0 0) 0x7f1e600584e0 con 0x7f1e680d0560 2024-05-26T22:56:59.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.245+0000 7f1e6cc00640 1 -- 172.21.15.131:0/3258545977 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f1e4406be60 msgr2=0x7f1e4406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:59.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.245+0000 7f1e6cc00640 1 --2- 172.21.15.131:0/3258545977 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f1e4406be60 0x7f1e4406e320 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f1e58001ea0 tx=0x7f1e58007d20 comp rx=0 tx=0).stop 2024-05-26T22:56:59.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.245+0000 7f1e6cc00640 1 -- 172.21.15.131:0/3258545977 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1e680d0560 msgr2=0x7f1e6815ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:59.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.245+0000 7f1e6cc00640 1 --2- 172.21.15.131:0/3258545977 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1e680d0560 0x7f1e6815ce20 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f1e60001eb0 tx=0x7f1e6000c890 comp rx=0 tx=0).stop 2024-05-26T22:56:59.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.245+0000 7f1e6cc00640 1 -- 172.21.15.131:0/3258545977 shutdown_connections 2024-05-26T22:56:59.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.245+0000 7f1e6cc00640 1 --2- 172.21.15.131:0/3258545977 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f1e4406be60 0x7f1e4406e320 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:59.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.245+0000 7f1e6cc00640 1 --2- 172.21.15.131:0/3258545977 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1e680d0560 0x7f1e6815ce20 unknown :-1 s=CLOSED pgs=1636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:59.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.245+0000 7f1e6cc00640 1 --2- 172.21.15.131:0/3258545977 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1e68072540 0x7f1e6815c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:59.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.245+0000 7f1e6cc00640 1 --2- 172.21.15.131:0/3258545977 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1e6806f180 0x7f1e6815ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:59.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.245+0000 7f1e6cc00640 1 -- 172.21.15.131:0/3258545977 >> 172.21.15.131:0/3258545977 conn(0x7f1e6806bc90 msgr2=0x7f1e680715a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:56:59.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.245+0000 7f1e6cc00640 1 -- 172.21.15.131:0/3258545977 shutdown_connections 2024-05-26T22:56:59.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.246+0000 7f1e6cc00640 1 -- 172.21.15.131:0/3258545977 wait complete. 2024-05-26T22:56:59.264 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_export_ephemeral_random 2024-05-26T22:56:59.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.468+0000 7fd57d800640 1 Processor -- start 2024-05-26T22:56:59.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.468+0000 7fd57d800640 1 -- start start 2024-05-26T22:56:59.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.469+0000 7fd57d800640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fd57806f420 0x7fd5780cfd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:59.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.469+0000 7fd57d800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd5780cf590 0x7fd5780d0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:59.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.469+0000 7fd57d800640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd578160180 con 0x7fd5780b5450 2024-05-26T22:56:59.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.470+0000 7fd57d800640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fd578160d90 con 0x7fd57806f420 2024-05-26T22:56:59.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.470+0000 7fd57d800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd578160460 con 0x7fd5780cf590 2024-05-26T22:56:59.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.470+0000 7fd576000640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fd57806f420 0x7fd5780cfd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:59.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.470+0000 7fd576a00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fd5780b5450 0x7fd5780b43c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:33766/0 (socket says 172.21.15.131:33766) 2024-05-26T22:56:59.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.470+0000 7fd576a00640 1 -- 172.21.15.131:0/3211705050 learned_addr learned my addr 172.21.15.131:0/3211705050 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:56:59.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.470+0000 7fd577400640 1 --2- 172.21.15.131:0/3211705050 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd5780cf590 0x7fd5780d0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:59.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.470+0000 7fd576000640 1 -- 172.21.15.131:0/3211705050 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd5780cf590 msgr2=0x7fd5780d0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:59.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.470+0000 7fd576000640 1 --2- 172.21.15.131:0/3211705050 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd5780cf590 0x7fd5780d0290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:59.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.470+0000 7fd576000640 1 -- 172.21.15.131:0/3211705050 >> v1:172.21.15.131:6789/0 conn(0x7fd5780b5450 legacy=0x7fd5780b43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:59.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.470+0000 7fd576000640 1 -- 172.21.15.131:0/3211705050 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd578160740 con 0x7fd57806f420 2024-05-26T22:56:59.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.470+0000 7fd577400640 1 --2- 172.21.15.131:0/3211705050 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd5780cf590 0x7fd5780d0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:56:59.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.472+0000 7fd576000640 1 --2- 172.21.15.131:0/3211705050 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd57806f420 0x7fd5780cfd50 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7fd5640097a0 tx=0x7fd56402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f82ae353aebd82fa server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:59.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.472+0000 7fd575600640 1 -- 172.21.15.131:0/3211705050 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd564037070 con 0x7fd57806f420 2024-05-26T22:56:59.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.472+0000 7fd575600640 1 -- 172.21.15.131:0/3211705050 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd564032440 con 0x7fd57806f420 2024-05-26T22:56:59.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.472+0000 7fd575600640 1 -- 172.21.15.131:0/3211705050 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd564032810 con 0x7fd57806f420 2024-05-26T22:56:59.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.473+0000 7fd57d800640 1 -- 172.21.15.131:0/3211705050 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd57806f420 msgr2=0x7fd5780cfd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:59.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.473+0000 7fd57d800640 1 --2- 172.21.15.131:0/3211705050 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd57806f420 0x7fd5780cfd50 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7fd5640097a0 tx=0x7fd56402a180 comp rx=0 tx=0).stop 2024-05-26T22:56:59.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.473+0000 7fd57d800640 1 -- 172.21.15.131:0/3211705050 shutdown_connections 2024-05-26T22:56:59.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.473+0000 7fd57d800640 1 --2- 172.21.15.131:0/3211705050 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd5780cf590 0x7fd5780d0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:59.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.473+0000 7fd57d800640 1 --2- 172.21.15.131:0/3211705050 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd57806f420 0x7fd5780cfd50 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:59.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.473+0000 7fd57d800640 1 -- 172.21.15.131:0/3211705050 >> 172.21.15.131:0/3211705050 conn(0x7fd57806bc90 msgr2=0x7fd5780b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:56:59.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.473+0000 7fd57d800640 1 -- 172.21.15.131:0/3211705050 shutdown_connections 2024-05-26T22:56:59.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.473+0000 7fd57d800640 1 -- 172.21.15.131:0/3211705050 wait complete. 2024-05-26T22:56:59.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.474+0000 7fd57d800640 1 Processor -- start 2024-05-26T22:56:59.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.475+0000 7fd57d800640 1 -- start start 2024-05-26T22:56:59.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.476+0000 7fd57d800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd57806f420 0x7fd5781f2890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:59.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.476+0000 7fd57d800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5780b5450 0x7fd57809b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:59.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.476+0000 7fd57d800640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd5780cf590 0x7fd57815eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:59.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.476+0000 7fd57d800640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fd57815fde0 con 0x7fd5780b5450 2024-05-26T22:56:59.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.476+0000 7fd57d800640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd5780b5830 con 0x7fd5780cf590 2024-05-26T22:56:59.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.476+0000 7fd57d800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd57815ff60 con 0x7fd57806f420 2024-05-26T22:56:59.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.476+0000 7fd576000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5780b5450 0x7fd57809b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:59.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.476+0000 7fd576000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5780b5450 0x7fd57809b5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41072/0 (socket says 172.21.15.131:41072) 2024-05-26T22:56:59.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.476+0000 7fd576000640 1 -- 172.21.15.131:0/23579692 learned_addr learned my addr 172.21.15.131:0/23579692 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:56:59.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.476+0000 7fd576000640 1 -- 172.21.15.131:0/23579692 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd57806f420 msgr2=0x7fd5781f2890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:59.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.476+0000 7fd576a00640 1 --2- 172.21.15.131:0/23579692 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd57806f420 0x7fd5781f2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:59.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.476+0000 7fd576000640 1 --2- 172.21.15.131:0/23579692 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd57806f420 0x7fd5781f2890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:59.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.476+0000 7fd576000640 1 -- 172.21.15.131:0/23579692 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd5780cf590 msgr2=0x7fd57815eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:59.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.476+0000 7fd576000640 1 --2- 172.21.15.131:0/23579692 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd5780cf590 0x7fd57815eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:59.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.476+0000 7fd576000640 1 -- 172.21.15.131:0/23579692 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd57809bec0 con 0x7fd5780b5450 2024-05-26T22:56:59.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.476+0000 7fd576a00640 1 --2- 172.21.15.131:0/23579692 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd57806f420 0x7fd5781f2890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:56:59.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.477+0000 7fd576000640 1 --2- 172.21.15.131:0/23579692 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5780b5450 0x7fd57809b5e0 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7fd564035040 tx=0x7fd5640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:59.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.477+0000 7fd56ea00640 1 -- 172.21.15.131:0/23579692 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd564037040 con 0x7fd5780b5450 2024-05-26T22:56:59.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.477+0000 7fd56ea00640 1 -- 172.21.15.131:0/23579692 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd564030770 con 0x7fd5780b5450 2024-05-26T22:56:59.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.477+0000 7fd56ea00640 1 -- 172.21.15.131:0/23579692 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd564030b80 con 0x7fd5780b5450 2024-05-26T22:56:59.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.477+0000 7fd57d800640 1 -- 172.21.15.131:0/23579692 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd57815c980 con 0x7fd5780b5450 2024-05-26T22:56:59.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.477+0000 7fd57d800640 1 -- 172.21.15.131:0/23579692 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd57815cf40 con 0x7fd5780b5450 2024-05-26T22:56:59.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.478+0000 7fd57d800640 1 -- 172.21.15.131:0/23579692 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd578074e90 con 0x7fd5780b5450 2024-05-26T22:56:59.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.479+0000 7fd56ea00640 1 -- 172.21.15.131:0/23579692 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd564044020 con 0x7fd5780b5450 2024-05-26T22:56:59.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.480+0000 7fd56ea00640 1 --2- 172.21.15.131:0/23579692 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fd55806be30 0x7fd55806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:59.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.480+0000 7fd56ea00640 1 -- 172.21.15.131:0/23579692 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5115+0+0 (secure 0 0 0) 0x7fd5640ab320 con 0x7fd5780b5450 2024-05-26T22:56:59.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.480+0000 7fd576a00640 1 --2- 172.21.15.131:0/23579692 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fd55806be30 0x7fd55806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:59.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.481+0000 7fd576a00640 1 --2- 172.21.15.131:0/23579692 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fd55806be30 0x7fd55806e2f0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fd5781ef850 tx=0x7fd568009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:59.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.484+0000 7fd56ea00640 1 -- 172.21.15.131:0/23579692 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fd564032e20 con 0x7fd5780b5450 2024-05-26T22:56:59.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.683+0000 7fd57d800640 1 -- 172.21.15.131:0/23579692 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_export_ephemeral_random"} v 0) -- 0x7fd5780cee90 con 0x7fd5780b5450 2024-05-26T22:56:59.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.689+0000 7fd56ea00640 1 -- 172.21.15.131:0/23579692 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_export_ephemeral_random"}]=0 v64) ==== 110+0+0 (secure 0 0 0) 0x7fd5640a9e20 con 0x7fd5780b5450 2024-05-26T22:56:59.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.692+0000 7fd57d800640 1 -- 172.21.15.131:0/23579692 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fd55806be30 msgr2=0x7fd55806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:59.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.692+0000 7fd57d800640 1 --2- 172.21.15.131:0/23579692 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fd55806be30 0x7fd55806e2f0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fd5781ef850 tx=0x7fd568009040 comp rx=0 tx=0).stop 2024-05-26T22:56:59.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.692+0000 7fd57d800640 1 -- 172.21.15.131:0/23579692 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5780b5450 msgr2=0x7fd57809b5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:59.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.692+0000 7fd57d800640 1 --2- 172.21.15.131:0/23579692 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5780b5450 0x7fd57809b5e0 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7fd564035040 tx=0x7fd5640046c0 comp rx=0 tx=0).stop 2024-05-26T22:56:59.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.693+0000 7fd57d800640 1 -- 172.21.15.131:0/23579692 shutdown_connections 2024-05-26T22:56:59.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.693+0000 7fd57d800640 1 --2- 172.21.15.131:0/23579692 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fd55806be30 0x7fd55806e2f0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:59.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.693+0000 7fd57d800640 1 --2- 172.21.15.131:0/23579692 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd5780cf590 0x7fd57815eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:59.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.693+0000 7fd57d800640 1 --2- 172.21.15.131:0/23579692 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5780b5450 0x7fd57809b5e0 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:59.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.693+0000 7fd57d800640 1 --2- 172.21.15.131:0/23579692 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd57806f420 0x7fd5781f2890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:59.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.693+0000 7fd57d800640 1 -- 172.21.15.131:0/23579692 >> 172.21.15.131:0/23579692 conn(0x7fd57806bc90 msgr2=0x7fd5780af300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:56:59.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.693+0000 7fd57d800640 1 -- 172.21.15.131:0/23579692 shutdown_connections 2024-05-26T22:56:59.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.693+0000 7fd57d800640 1 -- 172.21.15.131:0/23579692 wait complete. 2024-05-26T22:56:59.712 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_export_ephemeral_distributed 2024-05-26T22:56:59.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.907+0000 7f40a9c00640 1 Processor -- start 2024-05-26T22:56:59.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.908+0000 7f40a9c00640 1 -- start start 2024-05-26T22:56:59.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.910+0000 7f40a9c00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f40a4140e80 0x7f40a406d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:59.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.910+0000 7f40a9c00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f40a40d01f0 0x7f40a415eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:59.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.910+0000 7f40a9c00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40a4162d00 con 0x7f40a4141260 2024-05-26T22:56:59.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.910+0000 7f40a9c00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f40a40cc600 con 0x7f40a4140e80 2024-05-26T22:56:59.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.910+0000 7f40a9c00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f40a41620f0 con 0x7f40a40d01f0 2024-05-26T22:56:59.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.910+0000 7f40a9200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f40a40d01f0 0x7f40a415eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:59.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.910+0000 7f409be00640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f40a4141260 0x7f40a40cfae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.131:47420/0 (socket says 172.21.15.131:47420) 2024-05-26T22:56:59.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.910+0000 7f409be00640 1 -- 172.21.15.131:0/1869086473 learned_addr learned my addr 172.21.15.131:0/1869086473 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:56:59.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.911+0000 7f40a3e00640 1 --2- 172.21.15.131:0/1869086473 >> v2:172.21.15.187:3300/0 conn(0x7f40a4140e80 0x7f40a406d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:59.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.911+0000 7f40a9200640 1 -- 172.21.15.131:0/1869086473 >> v2:172.21.15.187:3300/0 conn(0x7f40a4140e80 msgr2=0x7f40a406d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:56:59.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.911+0000 7f40a9200640 1 --2- 172.21.15.131:0/1869086473 >> v2:172.21.15.187:3300/0 conn(0x7f40a4140e80 0x7f40a406d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:59.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.911+0000 7f40a9200640 1 -- 172.21.15.131:0/1869086473 >> v1:172.21.15.187:6789/0 conn(0x7f40a4141260 legacy=0x7f40a40cfae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:59.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.911+0000 7f40a9200640 1 -- 172.21.15.131:0/1869086473 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40a41623d0 con 0x7f40a40d01f0 2024-05-26T22:56:59.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.912+0000 7f40a9200640 1 --2- 172.21.15.131:0/1869086473 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f40a40d01f0 0x7f40a415eac0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f409c009d90 tx=0x7f409c02a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e175e174fe75c439 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:59.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.913+0000 7f40a3400640 1 -- 172.21.15.131:0/1869086473 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f409c007740 con 0x7f40a40d01f0 2024-05-26T22:56:59.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.913+0000 7f40a3400640 1 -- 172.21.15.131:0/1869086473 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f409c0078e0 con 0x7f40a40d01f0 2024-05-26T22:56:59.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.913+0000 7f40a3400640 1 -- 172.21.15.131:0/1869086473 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f409c007cb0 con 0x7f40a40d01f0 2024-05-26T22:56:59.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.914+0000 7f40a9c00640 1 -- 172.21.15.131:0/1869086473 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f40a40d01f0 msgr2=0x7f40a415eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:59.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.914+0000 7f40a9c00640 1 --2- 172.21.15.131:0/1869086473 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f40a40d01f0 0x7f40a415eac0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f409c009d90 tx=0x7f409c02a660 comp rx=0 tx=0).stop 2024-05-26T22:56:59.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.914+0000 7f40a9c00640 1 -- 172.21.15.131:0/1869086473 shutdown_connections 2024-05-26T22:56:59.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.914+0000 7f40a9c00640 1 --2- 172.21.15.131:0/1869086473 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f40a40d01f0 0x7f40a415eac0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:59.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.914+0000 7f40a9c00640 1 --2- 172.21.15.131:0/1869086473 >> v2:172.21.15.187:3300/0 conn(0x7f40a4140e80 0x7f40a406d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:59.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.914+0000 7f40a9c00640 1 -- 172.21.15.131:0/1869086473 >> 172.21.15.131:0/1869086473 conn(0x7f40a40cc7f0 msgr2=0x7f40a40ccbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:56:59.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.914+0000 7f40a9c00640 1 -- 172.21.15.131:0/1869086473 shutdown_connections 2024-05-26T22:56:59.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.914+0000 7f40a9c00640 1 -- 172.21.15.131:0/1869086473 wait complete. 2024-05-26T22:56:59.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.916+0000 7f40a9c00640 1 Processor -- start 2024-05-26T22:56:59.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.916+0000 7f40a9c00640 1 -- start start 2024-05-26T22:56:59.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.917+0000 7f40a9c00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f40a40d01f0 0x7f40a40b2480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:59.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.917+0000 7f40a9c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f40a4140e80 0x7f40a41f54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:59.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.917+0000 7f40a9c00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f40a4141260 0x7f40a4156e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:59.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.917+0000 7f40a9c00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f40a4161da0 con 0x7f40a4140e80 2024-05-26T22:56:59.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.917+0000 7f40a9c00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f40a4162990 con 0x7f40a40d01f0 2024-05-26T22:56:59.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.917+0000 7f40a9c00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f40a40cdac0 con 0x7f40a4141260 2024-05-26T22:56:59.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.917+0000 7f409be00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f40a4140e80 0x7f40a41f54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:59.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.917+0000 7f409be00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f40a4140e80 0x7f40a41f54e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41076/0 (socket says 172.21.15.131:41076) 2024-05-26T22:56:59.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.917+0000 7f409be00640 1 -- 172.21.15.131:0/660272714 learned_addr learned my addr 172.21.15.131:0/660272714 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:56:59.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.917+0000 7f40a9200640 1 --2- 172.21.15.131:0/660272714 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f40a4141260 0x7f40a4156e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:59.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.917+0000 7f40a3e00640 1 --2- 172.21.15.131:0/660272714 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f40a40d01f0 0x7f40a40b2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:59.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.917+0000 7f409be00640 1 -- 172.21.15.131:0/660272714 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f40a4141260 msgr2=0x7f40a4156e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:59.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.918+0000 7f409be00640 1 --2- 172.21.15.131:0/660272714 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f40a4141260 0x7f40a4156e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:59.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.918+0000 7f409be00640 1 -- 172.21.15.131:0/660272714 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f40a40d01f0 msgr2=0x7f40a40b2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:56:59.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.918+0000 7f409be00640 1 --2- 172.21.15.131:0/660272714 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f40a40d01f0 0x7f40a40b2480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:56:59.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.918+0000 7f409be00640 1 -- 172.21.15.131:0/660272714 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40a41f5a20 con 0x7f40a4140e80 2024-05-26T22:56:59.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.918+0000 7f40a9200640 1 --2- 172.21.15.131:0/660272714 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f40a4141260 0x7f40a4156e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:56:59.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.918+0000 7f40a3e00640 1 --2- 172.21.15.131:0/660272714 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f40a40d01f0 0x7f40a40b2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:56:59.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.918+0000 7f409be00640 1 --2- 172.21.15.131:0/660272714 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f40a4140e80 0x7f40a41f54e0 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f408c00c440 tx=0x7f408c001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:59.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.918+0000 7f40a0c00640 1 -- 172.21.15.131:0/660272714 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f408c017070 con 0x7f40a4140e80 2024-05-26T22:56:59.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.918+0000 7f40a0c00640 1 -- 172.21.15.131:0/660272714 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f408c005d00 con 0x7f40a4140e80 2024-05-26T22:56:59.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.918+0000 7f40a9c00640 1 -- 172.21.15.131:0/660272714 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40a40afed0 con 0x7f40a4140e80 2024-05-26T22:56:59.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.918+0000 7f40a0c00640 1 -- 172.21.15.131:0/660272714 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f408c007310 con 0x7f40a4140e80 2024-05-26T22:56:59.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.918+0000 7f40a9c00640 1 -- 172.21.15.131:0/660272714 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40a40b0490 con 0x7f40a4140e80 2024-05-26T22:56:59.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.919+0000 7f40a9c00640 1 -- 172.21.15.131:0/660272714 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40a40694e0 con 0x7f40a4140e80 2024-05-26T22:56:59.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.920+0000 7f40a0c00640 1 -- 172.21.15.131:0/660272714 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f408c01a070 con 0x7f40a4140e80 2024-05-26T22:56:59.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.921+0000 7f40a0c00640 1 --2- 172.21.15.131:0/660272714 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f408006be30 0x7f408006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:56:59.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.921+0000 7f40a0c00640 1 -- 172.21.15.131:0/660272714 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5115+0+0 (secure 0 0 0) 0x7f408c089ff0 con 0x7f40a4140e80 2024-05-26T22:56:59.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.921+0000 7f40a3e00640 1 --2- 172.21.15.131:0/660272714 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f408006be30 0x7f408006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:56:59.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.922+0000 7f40a3e00640 1 --2- 172.21.15.131:0/660272714 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f408006be30 0x7f408006e2f0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f4094006fd0 tx=0x7f4094008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:56:59.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:56:59.925+0000 7f40a0c00640 1 -- 172.21.15.131:0/660272714 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f408c058be0 con 0x7f40a4140e80 2024-05-26T22:57:00.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.127+0000 7f40a9c00640 1 -- 172.21.15.131:0/660272714 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_export_ephemeral_distributed"} v 0) -- 0x7f40a4073500 con 0x7f40a4140e80 2024-05-26T22:57:00.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.134+0000 7f40a0c00640 1 -- 172.21.15.131:0/660272714 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_export_ephemeral_distributed"}]=0 v65) ==== 115+0+0 (secure 0 0 0) 0x7f408c0584e0 con 0x7f40a4140e80 2024-05-26T22:57:00.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.137+0000 7f40a9c00640 1 -- 172.21.15.131:0/660272714 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f408006be30 msgr2=0x7f408006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:00.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.137+0000 7f40a9c00640 1 --2- 172.21.15.131:0/660272714 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f408006be30 0x7f408006e2f0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f4094006fd0 tx=0x7f4094008040 comp rx=0 tx=0).stop 2024-05-26T22:57:00.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.137+0000 7f40a9c00640 1 -- 172.21.15.131:0/660272714 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f40a4140e80 msgr2=0x7f40a41f54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:00.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.137+0000 7f40a9c00640 1 --2- 172.21.15.131:0/660272714 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f40a4140e80 0x7f40a41f54e0 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f408c00c440 tx=0x7f408c001e70 comp rx=0 tx=0).stop 2024-05-26T22:57:00.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.137+0000 7f40a9c00640 1 -- 172.21.15.131:0/660272714 shutdown_connections 2024-05-26T22:57:00.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.137+0000 7f40a9c00640 1 --2- 172.21.15.131:0/660272714 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f40a4141260 0x7f40a4156e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:00.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.137+0000 7f40a9c00640 1 --2- 172.21.15.131:0/660272714 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f408006be30 0x7f408006e2f0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:00.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.137+0000 7f40a9c00640 1 --2- 172.21.15.131:0/660272714 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f40a4140e80 0x7f40a41f54e0 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:00.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.137+0000 7f40a9c00640 1 --2- 172.21.15.131:0/660272714 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f40a40d01f0 0x7f40a40b2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:00.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.137+0000 7f40a9c00640 1 -- 172.21.15.131:0/660272714 >> 172.21.15.131:0/660272714 conn(0x7f40a40cc7f0 msgr2=0x7f40a40adfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:57:00.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.138+0000 7f40a9c00640 1 -- 172.21.15.131:0/660272714 shutdown_connections 2024-05-26T22:57:00.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.138+0000 7f40a9c00640 1 -- 172.21.15.131:0/660272714 wait complete. 2024-05-26T22:57:00.157 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-26T22:57:00.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.355+0000 7f9d7d000640 1 Processor -- start 2024-05-26T22:57:00.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.355+0000 7f9d7d000640 1 -- start start 2024-05-26T22:57:00.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.357+0000 7f9d7d000640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f9d781554f0 0x7f9d78071f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:00.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.357+0000 7f9d7d000640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f9d7806ebc0 0x7f9d7815d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:00.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.357+0000 7f9d7d000640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d781558d0 con 0x7f9d780724a0 2024-05-26T22:57:00.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.357+0000 7f9d7d000640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f9d780729b0 con 0x7f9d7806ebc0 2024-05-26T22:57:00.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.357+0000 7f9d7d000640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f9d7815c750 con 0x7f9d781554f0 2024-05-26T22:57:00.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.357+0000 7f9d6fe00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f9d780724a0 0x7f9d7806e4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:33774/0 (socket says 172.21.15.131:33774) 2024-05-26T22:57:00.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.357+0000 7f9d6fe00640 1 -- 172.21.15.131:0/1993586017 learned_addr learned my addr 172.21.15.131:0/1993586017 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:57:00.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.357+0000 7f9d77e00640 1 --2- 172.21.15.131:0/1993586017 >> v2:172.21.15.131:3300/0 conn(0x7f9d7806ebc0 0x7f9d7815d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:00.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.357+0000 7f9d77400640 1 --2- 172.21.15.131:0/1993586017 >> v2:172.21.15.187:3300/0 conn(0x7f9d781554f0 0x7f9d78071f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:00.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.357+0000 7f9d76a00640 1 -- 172.21.15.131:0/1993586017 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1940652173 0 0) 0x7f9d60001110 con 0x7f9d780724a0 2024-05-26T22:57:00.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.358+0000 7f9d76a00640 1 -- 172.21.15.131:0/1993586017 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9d5c005730 con 0x7f9d780724a0 2024-05-26T22:57:00.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.358+0000 7f9d77e00640 1 -- 172.21.15.131:0/1993586017 >> v2:172.21.15.187:3300/0 conn(0x7f9d781554f0 msgr2=0x7f9d78071f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:57:00.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.358+0000 7f9d77e00640 1 --2- 172.21.15.131:0/1993586017 >> v2:172.21.15.187:3300/0 conn(0x7f9d781554f0 0x7f9d78071f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:00.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.358+0000 7f9d77e00640 1 -- 172.21.15.131:0/1993586017 >> v1:172.21.15.131:6789/0 conn(0x7f9d780724a0 legacy=0x7f9d7806e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:00.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.358+0000 7f9d77e00640 1 -- 172.21.15.131:0/1993586017 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d7815ca30 con 0x7f9d7806ebc0 2024-05-26T22:57:00.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.359+0000 7f9d77e00640 1 --2- 172.21.15.131:0/1993586017 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d7806ebc0 0x7f9d7815d230 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f9d780cda90 tx=0x7f9d7002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c46990fdcbb8fbbc server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:57:00.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.359+0000 7f9d76a00640 1 -- 172.21.15.131:0/1993586017 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d70007740 con 0x7f9d7806ebc0 2024-05-26T22:57:00.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.359+0000 7f9d76a00640 1 -- 172.21.15.131:0/1993586017 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d700078e0 con 0x7f9d7806ebc0 2024-05-26T22:57:00.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.359+0000 7f9d76a00640 1 -- 172.21.15.131:0/1993586017 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d70007c90 con 0x7f9d7806ebc0 2024-05-26T22:57:00.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.360+0000 7f9d7d000640 1 -- 172.21.15.131:0/1993586017 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d7806ebc0 msgr2=0x7f9d7815d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:00.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.360+0000 7f9d7d000640 1 --2- 172.21.15.131:0/1993586017 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d7806ebc0 0x7f9d7815d230 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f9d780cda90 tx=0x7f9d7002a680 comp rx=0 tx=0).stop 2024-05-26T22:57:00.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.360+0000 7f9d7d000640 1 -- 172.21.15.131:0/1993586017 shutdown_connections 2024-05-26T22:57:00.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.360+0000 7f9d7d000640 1 --2- 172.21.15.131:0/1993586017 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d7806ebc0 0x7f9d7815d230 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:00.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.360+0000 7f9d7d000640 1 --2- 172.21.15.131:0/1993586017 >> v2:172.21.15.187:3300/0 conn(0x7f9d781554f0 0x7f9d78071f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:00.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.360+0000 7f9d7d000640 1 -- 172.21.15.131:0/1993586017 >> 172.21.15.131:0/1993586017 conn(0x7f9d780cc7f0 msgr2=0x7f9d780ccbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:57:00.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.360+0000 7f9d7d000640 1 -- 172.21.15.131:0/1993586017 shutdown_connections 2024-05-26T22:57:00.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.360+0000 7f9d7d000640 1 -- 172.21.15.131:0/1993586017 wait complete. 2024-05-26T22:57:00.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.361+0000 7f9d7d000640 1 Processor -- start 2024-05-26T22:57:00.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.362+0000 7f9d7d000640 1 -- start start 2024-05-26T22:57:00.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.363+0000 7f9d7d000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d7806ebc0 0x7f9d781f72e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:00.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.363+0000 7f9d7d000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d780724a0 0x7f9d78151620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:00.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.363+0000 7f9d7d000640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d781554f0 0x7f9d78151b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:00.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.363+0000 7f9d7d000640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f9d7815c3a0 con 0x7f9d780724a0 2024-05-26T22:57:00.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.363+0000 7f9d7d000640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9d7815c220 con 0x7f9d781554f0 2024-05-26T22:57:00.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.363+0000 7f9d7d000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9d781f7820 con 0x7f9d7806ebc0 2024-05-26T22:57:00.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.363+0000 7f9d6fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d780724a0 0x7f9d78151620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:00.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.363+0000 7f9d6fe00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d780724a0 0x7f9d78151620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41084/0 (socket says 172.21.15.131:41084) 2024-05-26T22:57:00.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.363+0000 7f9d6fe00640 1 -- 172.21.15.131:0/2061971318 learned_addr learned my addr 172.21.15.131:0/2061971318 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:57:00.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.363+0000 7f9d77400640 1 --2- 172.21.15.131:0/2061971318 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d7806ebc0 0x7f9d781f72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:00.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.363+0000 7f9d77e00640 1 --2- 172.21.15.131:0/2061971318 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d781554f0 0x7f9d78151b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:00.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.363+0000 7f9d6fe00640 1 -- 172.21.15.131:0/2061971318 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d7806ebc0 msgr2=0x7f9d781f72e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:00.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.363+0000 7f9d6fe00640 1 --2- 172.21.15.131:0/2061971318 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d7806ebc0 0x7f9d781f72e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:00.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.363+0000 7f9d6fe00640 1 -- 172.21.15.131:0/2061971318 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d781554f0 msgr2=0x7f9d78151b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:00.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.363+0000 7f9d6fe00640 1 --2- 172.21.15.131:0/2061971318 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d781554f0 0x7f9d78151b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:00.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.363+0000 7f9d6fe00640 1 -- 172.21.15.131:0/2061971318 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d780b4da0 con 0x7f9d780724a0 2024-05-26T22:57:00.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.363+0000 7f9d77400640 1 --2- 172.21.15.131:0/2061971318 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d7806ebc0 0x7f9d781f72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:57:00.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.364+0000 7f9d6fe00640 1 --2- 172.21.15.131:0/2061971318 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d780724a0 0x7f9d78151620 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f9d6000c3a0 tx=0x7f9d6000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:57:00.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.364+0000 7f9d6f400640 1 -- 172.21.15.131:0/2061971318 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d60017070 con 0x7f9d780724a0 2024-05-26T22:57:00.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.364+0000 7f9d7d000640 1 -- 172.21.15.131:0/2061971318 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d780b5080 con 0x7f9d780724a0 2024-05-26T22:57:00.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.364+0000 7f9d6f400640 1 -- 172.21.15.131:0/2061971318 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d60005ce0 con 0x7f9d780724a0 2024-05-26T22:57:00.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.364+0000 7f9d6f400640 1 -- 172.21.15.131:0/2061971318 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d60007310 con 0x7f9d780724a0 2024-05-26T22:57:00.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.364+0000 7f9d7d000640 1 -- 172.21.15.131:0/2061971318 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d780b5640 con 0x7f9d780724a0 2024-05-26T22:57:00.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.365+0000 7f9d7d000640 1 -- 172.21.15.131:0/2061971318 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d780694e0 con 0x7f9d780724a0 2024-05-26T22:57:00.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.366+0000 7f9d6f400640 1 -- 172.21.15.131:0/2061971318 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9d6001a070 con 0x7f9d780724a0 2024-05-26T22:57:00.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.368+0000 7f9d6f400640 1 --2- 172.21.15.131:0/2061971318 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9d5406be60 0x7f9d5406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:00.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.368+0000 7f9d6f400640 1 -- 172.21.15.131:0/2061971318 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5115+0+0 (secure 0 0 0) 0x7f9d6008a080 con 0x7f9d780724a0 2024-05-26T22:57:00.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.368+0000 7f9d77400640 1 --2- 172.21.15.131:0/2061971318 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9d5406be60 0x7f9d5406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:00.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.369+0000 7f9d77400640 1 --2- 172.21.15.131:0/2061971318 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9d5406be60 0x7f9d5406e320 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f9d68006fd0 tx=0x7f9d68008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:57:00.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.372+0000 7f9d6f400640 1 -- 172.21.15.131:0/2061971318 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f9d781558d0 con 0x7f9d780724a0 2024-05-26T22:57:00.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.557+0000 7f9d7d000640 1 -- 172.21.15.131:0/2061971318 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f9d780ce430 con 0x7f9d780724a0 2024-05-26T22:57:00.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.563+0000 7f9d6f400640 1 -- 172.21.15.131:0/2061971318 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v66) ==== 104+0+0 (secure 0 0 0) 0x7f9d60058bf0 con 0x7f9d780724a0 2024-05-26T22:57:00.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.566+0000 7f9d7d000640 1 -- 172.21.15.131:0/2061971318 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9d5406be60 msgr2=0x7f9d5406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:00.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.566+0000 7f9d7d000640 1 --2- 172.21.15.131:0/2061971318 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9d5406be60 0x7f9d5406e320 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f9d68006fd0 tx=0x7f9d68008040 comp rx=0 tx=0).stop 2024-05-26T22:57:00.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.566+0000 7f9d7d000640 1 -- 172.21.15.131:0/2061971318 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d780724a0 msgr2=0x7f9d78151620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:00.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.566+0000 7f9d7d000640 1 --2- 172.21.15.131:0/2061971318 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d780724a0 0x7f9d78151620 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f9d6000c3a0 tx=0x7f9d6000c860 comp rx=0 tx=0).stop 2024-05-26T22:57:00.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.566+0000 7f9d7d000640 1 -- 172.21.15.131:0/2061971318 shutdown_connections 2024-05-26T22:57:00.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.566+0000 7f9d7d000640 1 --2- 172.21.15.131:0/2061971318 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f9d5406be60 0x7f9d5406e320 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:00.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.566+0000 7f9d7d000640 1 --2- 172.21.15.131:0/2061971318 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d781554f0 0x7f9d78151b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:00.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.566+0000 7f9d7d000640 1 --2- 172.21.15.131:0/2061971318 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9d780724a0 0x7f9d78151620 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:00.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.566+0000 7f9d7d000640 1 --2- 172.21.15.131:0/2061971318 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9d7806ebc0 0x7f9d781f72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:00.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.566+0000 7f9d7d000640 1 -- 172.21.15.131:0/2061971318 >> 172.21.15.131:0/2061971318 conn(0x7f9d780cc7f0 msgr2=0x7f9d780b09e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:57:00.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.567+0000 7f9d7d000640 1 -- 172.21.15.131:0/2061971318 shutdown_connections 2024-05-26T22:57:00.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.567+0000 7f9d7d000640 1 -- 172.21.15.131:0/2061971318 wait complete. 2024-05-26T22:57:00.584 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_exports.TestEphemeralPins.test_ephemeral_pin_dist_override_pin' 2024-05-26T22:57:00.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.784+0000 7fc7e8600640 1 Processor -- start 2024-05-26T22:57:00.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.784+0000 7fc7e8600640 1 -- start start 2024-05-26T22:57:00.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.786+0000 7fc7e8600640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fc7e01569d0 0x7fc7e00b3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:00.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.786+0000 7fc7e8600640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7e015cb20 con 0x7fc7e006ee20 2024-05-26T22:57:00.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.786+0000 7fc7e8600640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7e015ce00 con 0x7fc7e00ad250 2024-05-26T22:57:00.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.786+0000 7fc7e8600640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fc7e015d0e0 con 0x7fc7e01569d0 2024-05-26T22:57:00.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.786+0000 7fc7e5c00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fc7e006ee20 0x7fc7e01562c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:33776/0 (socket says 172.21.15.131:33776) 2024-05-26T22:57:00.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.786+0000 7fc7e5c00640 1 -- 172.21.15.131:0/1414158508 learned_addr learned my addr 172.21.15.131:0/1414158508 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:57:00.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.787+0000 7fc7e6600640 1 --2- 172.21.15.131:0/1414158508 >> v2:172.21.15.187:3300/0 conn(0x7fc7e01569d0 0x7fc7e00b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:00.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.787+0000 7fc7dfe00640 1 -- 172.21.15.131:0/1414158508 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 649245919 0 0) 0x7fc7e015cb20 con 0x7fc7e006ee20 2024-05-26T22:57:00.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.787+0000 7fc7dfe00640 1 -- 172.21.15.131:0/1414158508 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc7c8003520 con 0x7fc7e006ee20 2024-05-26T22:57:00.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.787+0000 7fc7dfe00640 1 -- 172.21.15.131:0/1414158508 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2587729891 0 0) 0x7fc7c8003520 con 0x7fc7e006ee20 2024-05-26T22:57:00.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.787+0000 7fc7dfe00640 1 -- 172.21.15.131:0/1414158508 >> v2:172.21.15.187:3300/0 conn(0x7fc7e01569d0 msgr2=0x7fc7e00b3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:57:00.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.787+0000 7fc7dfe00640 1 --2- 172.21.15.131:0/1414158508 >> v2:172.21.15.187:3300/0 conn(0x7fc7e01569d0 0x7fc7e00b3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:00.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.787+0000 7fc7dfe00640 1 -- 172.21.15.131:0/1414158508 >> v1:172.21.15.187:6789/0 conn(0x7fc7e00ad250 legacy=0x7fc7e0160010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:00.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.787+0000 7fc7dfe00640 1 -- 172.21.15.131:0/1414158508 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7e015d3c0 con 0x7fc7e006ee20 2024-05-26T22:57:00.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.787+0000 7fc7dfe00640 1 -- 172.21.15.131:0/1414158508 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7fc7d4001670 con 0x7fc7e006ee20 2024-05-26T22:57:00.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.787+0000 7fc7dfe00640 1 -- 172.21.15.131:0/1414158508 >> v1:172.21.15.131:6789/0 conn(0x7fc7e006ee20 legacy=0x7fc7e01562c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:00.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.787+0000 7fc7dfe00640 1 --2- 172.21.15.131:0/1414158508 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7c80039b0 0x7fc7c8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:00.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.787+0000 7fc7dfe00640 1 --2- 172.21.15.131:0/1414158508 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc7c8004800 0x7fc7c8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:00.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.787+0000 7fc7dfe00640 1 --2- 172.21.15.131:0/1414158508 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc7c80051c0 0x7fc7c800fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:00.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.787+0000 7fc7dfe00640 1 -- 172.21.15.131:0/1414158508 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e015d0e0 con 0x7fc7c80039b0 2024-05-26T22:57:00.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.787+0000 7fc7dfe00640 1 -- 172.21.15.131:0/1414158508 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc7c8010570 con 0x7fc7c80051c0 2024-05-26T22:57:00.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.787+0000 7fc7dfe00640 1 -- 172.21.15.131:0/1414158508 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc7c8010770 con 0x7fc7c8004800 2024-05-26T22:57:00.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.788+0000 7fc7e5c00640 1 --2- 172.21.15.131:0/1414158508 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7c80039b0 0x7fc7c8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:00.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.788+0000 7fc7e5200640 1 --2- 172.21.15.131:0/1414158508 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc7c8004800 0x7fc7c8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:00.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.788+0000 7fc7e6600640 1 --2- 172.21.15.131:0/1414158508 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc7c80051c0 0x7fc7c800fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:00.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.788+0000 7fc7e5c00640 1 -- 172.21.15.131:0/1414158508 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc7c8004800 msgr2=0x7fc7c8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:57:00.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.788+0000 7fc7e5c00640 1 --2- 172.21.15.131:0/1414158508 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc7c8004800 0x7fc7c8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:00.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.788+0000 7fc7e5c00640 1 -- 172.21.15.131:0/1414158508 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc7c80051c0 msgr2=0x7fc7c800fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:57:00.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.788+0000 7fc7e5c00640 1 --2- 172.21.15.131:0/1414158508 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc7c80051c0 0x7fc7c800fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:00.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.788+0000 7fc7e5c00640 1 -- 172.21.15.131:0/1414158508 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7c8003520 con 0x7fc7c80039b0 2024-05-26T22:57:00.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.788+0000 7fc7e6600640 1 -- 172.21.15.131:0/1414158508 reap_dead start 2024-05-26T22:57:00.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.790+0000 7fc7e5c00640 1 --2- 172.21.15.131:0/1414158508 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7c80039b0 0x7fc7c8003dd0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7fc7d400c240 tx=0x7fc7c8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c23105b6b81b61ef server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:57:00.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.790+0000 7fc7dfe00640 1 -- 172.21.15.131:0/1414158508 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d4007d10 con 0x7fc7c80039b0 2024-05-26T22:57:00.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.790+0000 7fc7dfe00640 1 -- 172.21.15.131:0/1414158508 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7d40058d0 con 0x7fc7c80039b0 2024-05-26T22:57:00.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.790+0000 7fc7dfe00640 1 -- 172.21.15.131:0/1414158508 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d4005d40 con 0x7fc7c80039b0 2024-05-26T22:57:00.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.791+0000 7fc7e8600640 1 -- 172.21.15.131:0/1414158508 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7c80039b0 msgr2=0x7fc7c8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:00.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.791+0000 7fc7e8600640 1 --2- 172.21.15.131:0/1414158508 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7c80039b0 0x7fc7c8003dd0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7fc7d400c240 tx=0x7fc7c8005830 comp rx=0 tx=0).stop 2024-05-26T22:57:00.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.791+0000 7fc7e8600640 1 -- 172.21.15.131:0/1414158508 shutdown_connections 2024-05-26T22:57:00.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.791+0000 7fc7e8600640 1 --2- 172.21.15.131:0/1414158508 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7c80039b0 0x7fc7c8003dd0 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:00.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.791+0000 7fc7e8600640 1 -- 172.21.15.131:0/1414158508 >> 172.21.15.131:0/1414158508 conn(0x7fc7e00b1e90 msgr2=0x7fc7e00b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:57:00.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.791+0000 7fc7e8600640 1 -- 172.21.15.131:0/1414158508 shutdown_connections 2024-05-26T22:57:00.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.791+0000 7fc7e8600640 1 -- 172.21.15.131:0/1414158508 wait complete. 2024-05-26T22:57:00.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.793+0000 7fc7e8600640 1 Processor -- start 2024-05-26T22:57:00.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.793+0000 7fc7e8600640 1 -- start start 2024-05-26T22:57:00.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.794+0000 7fc7e8600640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc7c80039b0 0x7fc7e0099790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:00.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.794+0000 7fc7e8600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7e009dbf0 0x7fc7e00afba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:00.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.794+0000 7fc7e8600640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc7e00b00e0 0x7fc7e00b0590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:00.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.794+0000 7fc7e8600640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e015c900 con 0x7fc7e009dbf0 2024-05-26T22:57:00.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.794+0000 7fc7e8600640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e015c780 con 0x7fc7c80039b0 2024-05-26T22:57:00.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.794+0000 7fc7e8600640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc7e006f200 con 0x7fc7e00b00e0 2024-05-26T22:57:00.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.794+0000 7fc7e5200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7e009dbf0 0x7fc7e00afba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:00.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.794+0000 7fc7e5200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7e009dbf0 0x7fc7e00afba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41096/0 (socket says 172.21.15.131:41096) 2024-05-26T22:57:00.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.794+0000 7fc7e5200640 1 -- 172.21.15.131:0/2949507767 learned_addr learned my addr 172.21.15.131:0/2949507767 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:57:00.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.795+0000 7fc7e5200640 1 -- 172.21.15.131:0/2949507767 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc7e00b00e0 msgr2=0x7fc7e00b0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:00.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.795+0000 7fc7e5200640 1 --2- 172.21.15.131:0/2949507767 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc7e00b00e0 0x7fc7e00b0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:00.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.795+0000 7fc7e5200640 1 -- 172.21.15.131:0/2949507767 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc7c80039b0 msgr2=0x7fc7e0099790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:00.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.795+0000 7fc7e5200640 1 --2- 172.21.15.131:0/2949507767 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc7c80039b0 0x7fc7e0099790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:00.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.795+0000 7fc7e5200640 1 -- 172.21.15.131:0/2949507767 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7e009e0a0 con 0x7fc7e009dbf0 2024-05-26T22:57:00.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.795+0000 7fc7e5200640 1 --2- 172.21.15.131:0/2949507767 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7e009dbf0 0x7fc7e00afba0 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7fc7cc00f540 tx=0x7fc7cc00fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:57:00.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.795+0000 7fc7dd600640 1 -- 172.21.15.131:0/2949507767 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7cc0052e0 con 0x7fc7e009dbf0 2024-05-26T22:57:00.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.795+0000 7fc7dd600640 1 -- 172.21.15.131:0/2949507767 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7cc018070 con 0x7fc7e009dbf0 2024-05-26T22:57:00.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.795+0000 7fc7dd600640 1 -- 172.21.15.131:0/2949507767 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7cc003830 con 0x7fc7e009dbf0 2024-05-26T22:57:00.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.796+0000 7fc7e8600640 1 -- 172.21.15.131:0/2949507767 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7e01410e0 con 0x7fc7e009dbf0 2024-05-26T22:57:00.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.796+0000 7fc7e8600640 1 -- 172.21.15.131:0/2949507767 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7e01541d0 con 0x7fc7e009dbf0 2024-05-26T22:57:00.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.797+0000 7fc7e8600640 1 -- 172.21.15.131:0/2949507767 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7e00694e0 con 0x7fc7e009dbf0 2024-05-26T22:57:00.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.798+0000 7fc7dd600640 1 -- 172.21.15.131:0/2949507767 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc7cc0039d0 con 0x7fc7e009dbf0 2024-05-26T22:57:00.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.799+0000 7fc7dd600640 1 --2- 172.21.15.131:0/2949507767 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc7c006be60 0x7fc7c006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:00.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.799+0000 7fc7dd600640 1 -- 172.21.15.131:0/2949507767 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5115+0+0 (secure 0 0 0) 0x7fc7cc08aa10 con 0x7fc7e009dbf0 2024-05-26T22:57:00.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.799+0000 7fc7e5c00640 1 --2- 172.21.15.131:0/2949507767 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc7c006be60 0x7fc7c006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:00.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.800+0000 7fc7e5c00640 1 --2- 172.21.15.131:0/2949507767 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc7c006be60 0x7fc7c006e320 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fc7d4033000 tx=0x7fc7d4009030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:57:00.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:00.803+0000 7fc7dd600640 1 -- 172.21.15.131:0/2949507767 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fc7cc0594e0 con 0x7fc7e009dbf0 2024-05-26T22:57:01.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:01.079+0000 7fc7e8600640 1 -- 172.21.15.131:0/2949507767 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_exports.TestEphemeralPins.test_ephemeral_pin_dist_override_pin"]} v 0) -- 0x7fc7e0073550 con 0x7fc7e009dbf0 2024-05-26T22:57:01.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:01.809+0000 7fc7dd600640 1 -- 172.21.15.131:0/2949507767 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_exports.TestEphemeralPins.test_ephemeral_pin_dist_override_pin"]}]=0 v433) ==== 159+0+0 (secure 0 0 0) 0x7fc7cc058de0 con 0x7fc7e009dbf0 2024-05-26T22:57:01.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:01.812+0000 7fc7e8600640 1 -- 172.21.15.131:0/2949507767 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc7c006be60 msgr2=0x7fc7c006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:01.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:01.812+0000 7fc7e8600640 1 --2- 172.21.15.131:0/2949507767 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc7c006be60 0x7fc7c006e320 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fc7d4033000 tx=0x7fc7d4009030 comp rx=0 tx=0).stop 2024-05-26T22:57:01.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:01.812+0000 7fc7e8600640 1 -- 172.21.15.131:0/2949507767 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7e009dbf0 msgr2=0x7fc7e00afba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:01.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:01.812+0000 7fc7e8600640 1 --2- 172.21.15.131:0/2949507767 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7e009dbf0 0x7fc7e00afba0 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7fc7cc00f540 tx=0x7fc7cc00fa00 comp rx=0 tx=0).stop 2024-05-26T22:57:01.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:01.812+0000 7fc7e8600640 1 -- 172.21.15.131:0/2949507767 shutdown_connections 2024-05-26T22:57:01.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:01.812+0000 7fc7e8600640 1 --2- 172.21.15.131:0/2949507767 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc7e00b00e0 0x7fc7e00b0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:01.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:01.812+0000 7fc7e8600640 1 --2- 172.21.15.131:0/2949507767 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fc7c006be60 0x7fc7c006e320 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:01.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:01.812+0000 7fc7e8600640 1 --2- 172.21.15.131:0/2949507767 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7e009dbf0 0x7fc7e00afba0 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:01.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:01.812+0000 7fc7e8600640 1 --2- 172.21.15.131:0/2949507767 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc7c80039b0 0x7fc7e0099790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:01.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:01.812+0000 7fc7e8600640 1 -- 172.21.15.131:0/2949507767 >> 172.21.15.131:0/2949507767 conn(0x7fc7e00b1e90 msgr2=0x7fc7e015d8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:57:01.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:01.813+0000 7fc7e8600640 1 -- 172.21.15.131:0/2949507767 shutdown_connections 2024-05-26T22:57:01.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:01.813+0000 7fc7e8600640 1 -- 172.21.15.131:0/2949507767 wait complete. 2024-05-26T22:57:01.835 INFO:tasks.cephfs_test_runner:That an export pin overrides an ephemerally pinned directory. ... FAIL 2024-05-26T22:57:01.835 INFO:tasks.cephfs_test_runner: 2024-05-26T22:57:01.835 INFO:tasks.cephfs_test_runner:====================================================================== 2024-05-26T22:57:01.835 INFO:tasks.cephfs_test_runner:FAIL: test_ephemeral_pin_dist_override_pin (tasks.cephfs.test_exports.TestEphemeralPins) 2024-05-26T22:57:01.835 INFO:tasks.cephfs_test_runner:That an export pin overrides an ephemerally pinned directory. 2024-05-26T22:57:01.836 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-05-26T22:57:01.836 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-05-26T22:57:01.836 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph_7d7cb32d1d130135a98121a564d709025c2154b6/qa/tasks/cephfs/test_exports.py", line 330, in test_ephemeral_pin_dist_override_pin 2024-05-26T22:57:01.836 INFO:tasks.cephfs_test_runner: self.assertEqual(s['auth_first'], 0) 2024-05-26T22:57:01.836 INFO:tasks.cephfs_test_runner:AssertionError: 1 != 0 2024-05-26T22:57:01.836 INFO:tasks.cephfs_test_runner: 2024-05-26T22:57:01.836 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-05-26T22:57:01.836 INFO:tasks.cephfs_test_runner:Ran 7 tests in 480.801s 2024-05-26T22:57:01.836 INFO:tasks.cephfs_test_runner: 2024-05-26T22:57:01.836 INFO:tasks.cephfs_test_runner:FAILED (failures=1) 2024-05-26T22:57:01.836 INFO:tasks.cephfs_test_runner: 2024-05-26T22:57:01.836 INFO:tasks.cephfs_test_runner:====================================================================== 2024-05-26T22:57:01.837 INFO:tasks.cephfs_test_runner:FAIL: test_ephemeral_pin_dist_override_pin (tasks.cephfs.test_exports.TestEphemeralPins) 2024-05-26T22:57:01.837 INFO:tasks.cephfs_test_runner:That an export pin overrides an ephemerally pinned directory. 2024-05-26T22:57:01.837 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-05-26T22:57:01.837 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-05-26T22:57:01.837 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph_7d7cb32d1d130135a98121a564d709025c2154b6/qa/tasks/cephfs/test_exports.py", line 330, in test_ephemeral_pin_dist_override_pin 2024-05-26T22:57:01.837 INFO:tasks.cephfs_test_runner: self.assertEqual(s['auth_first'], 0) 2024-05-26T22:57:01.837 INFO:tasks.cephfs_test_runner:AssertionError: 1 != 0 2024-05-26T22:57:01.837 INFO:tasks.cephfs_test_runner: 2024-05-26T22:57:01.837 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.10/contextlib.py", line 135, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph_7d7cb32d1d130135a98121a564d709025c2154b6/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_ephemeral_pin_dist_override_pin (tasks.cephfs.test_exports.TestEphemeralPins) 2024-05-26T22:57:02.065 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=d548d81f515d49148ccecd462d3a7664 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.10/contextlib.py", line 135, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph_7d7cb32d1d130135a98121a564d709025c2154b6/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_ephemeral_pin_dist_override_pin (tasks.cephfs.test_exports.TestEphemeralPins) 2024-05-26T22:57:02.066 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-05-26T22:57:02.077 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-05-26T22:57:02.152 INFO:tasks.kclient:Unmounting kernel clients... 2024-05-26T22:57:02.152 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-26T22:57:02.152 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-26T22:57:02.213 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-26T22:57:02.213 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-05-26T22:57:02.213 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-26T22:57:02.265 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-26T22:57:02.265 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-05-26T22:57:02.265 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-26T22:57:02.322 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-26T22:57:02.322 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-05-26T22:57:02.322 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-26T22:57:02.376 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-26T22:57:02.377 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:57:02.377 DEBUG:teuthology.orchestra.run.smithi131:> ip netns list 2024-05-26T22:57:02.435 INFO:teuthology.orchestra.run.smithi131.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-26T22:57:02.435 INFO:teuthology.orchestra.run.smithi131.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-26T22:57:02.435 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:57:02.436 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-26T22:57:02.523 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:57:02.523 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-26T22:57:02.599 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:57:02.599 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link delete ceph-brx 2024-05-26T22:57:02.721 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:57:02.721 DEBUG:teuthology.orchestra.run.smithi131:> ip netns list 2024-05-26T22:57:02.766 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:57:02.766 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link delete ceph-brx 2024-05-26T22:57:02.898 INFO:teuthology.orchestra.run.smithi131.stderr:Cannot find device "ceph-brx" 2024-05-26T22:57:02.899 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-26T22:57:02.900 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:57:02.900 DEBUG:teuthology.orchestra.run.smithi187:> ip netns list 2024-05-26T22:57:02.957 INFO:teuthology.orchestra.run.smithi187.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-05-26T22:57:02.957 INFO:teuthology.orchestra.run.smithi187.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-05-26T22:57:02.958 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:57:02.958 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-26T22:57:03.045 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:57:03.045 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-26T22:57:03.119 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:57:03.119 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip link delete ceph-brx 2024-05-26T22:57:03.236 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:57:03.236 DEBUG:teuthology.orchestra.run.smithi187:> ip netns list 2024-05-26T22:57:03.284 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-26T22:57:03.285 DEBUG:teuthology.orchestra.run.smithi187:> sudo ip link delete ceph-brx 2024-05-26T22:57:03.405 INFO:teuthology.orchestra.run.smithi187.stderr:Cannot find device "ceph-brx" 2024-05-26T22:57:03.409 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-26T22:57:03.409 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-05-26T22:57:03.419 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-26T22:57:03.419 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-26T22:57:03.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:03.629+0000 7fed4b000640 1 Processor -- start 2024-05-26T22:57:03.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:03.629+0000 7fed4b000640 1 -- start start 2024-05-26T22:57:03.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:03.631+0000 7fed4b000640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fed4406ebc0 0x7fed440cedf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:03.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:03.631+0000 7fed4b000640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fed44071b70 0x7fed440b3d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:03.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:03.631+0000 7fed4b000640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed440722a0 con 0x7fed44144340 2024-05-26T22:57:03.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:03.631+0000 7fed4b000640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fed44072eb0 con 0x7fed4406ebc0 2024-05-26T22:57:03.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:03.631+0000 7fed4b000640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fed44072580 con 0x7fed44071b70 2024-05-26T22:57:03.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:03.631+0000 7fed49200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fed4406ebc0 0x7fed440cedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:03.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:03.631+0000 7fed49200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fed4406ebc0 0x7fed440cedf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41110/0 (socket says 172.21.15.131:41110) 2024-05-26T22:57:03.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:03.631+0000 7fed49200640 1 -- 172.21.15.131:0/3704195153 learned_addr learned my addr 172.21.15.131:0/3704195153 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:57:03.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:03.631+0000 7fed49200640 1 -- 172.21.15.131:0/3704195153 >> v2:172.21.15.187:3300/0 conn(0x7fed44071b70 msgr2=0x7fed440b3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:57:03.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:03.631+0000 7fed49200640 1 --2- 172.21.15.131:0/3704195153 >> v2:172.21.15.187:3300/0 conn(0x7fed44071b70 0x7fed440b3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:03.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:03.631+0000 7fed49200640 1 -- 172.21.15.131:0/3704195153 >> v1:172.21.15.187:6789/0 conn(0x7fed44144340 legacy=0x7fed4406e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:57:03.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:03.632+0000 7fed49200640 1 -- 172.21.15.131:0/3704195153 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed44072860 con 0x7fed4406ebc0 2024-05-26T22:57:03.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:03.633+0000 7fed49200640 1 --2- 172.21.15.131:0/3704195153 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed4406ebc0 0x7fed440cedf0 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7fed440cfc40 tx=0x7fed3002a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=593bab79d4006165 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:57:03.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:03.633+0000 7fed43e00640 1 -- 172.21.15.131:0/3704195153 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed30037070 con 0x7fed4406ebc0 2024-05-26T22:57:03.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:03.633+0000 7fed43e00640 1 -- 172.21.15.131:0/3704195153 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed30032440 con 0x7fed4406ebc0 2024-05-26T22:57:03.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:03.633+0000 7fed43e00640 1 -- 172.21.15.131:0/3704195153 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed30032810 con 0x7fed4406ebc0 2024-05-26T22:57:03.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:03.633+0000 7fed4b000640 1 -- 172.21.15.131:0/3704195153 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed4406ebc0 msgr2=0x7fed440cedf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:03.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:03.633+0000 7fed4b000640 1 --2- 172.21.15.131:0/3704195153 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed4406ebc0 0x7fed440cedf0 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7fed440cfc40 tx=0x7fed3002a250 comp rx=0 tx=0).stop 2024-05-26T22:57:03.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:03.634+0000 7fed4b000640 1 -- 172.21.15.131:0/3704195153 shutdown_connections 2024-05-26T22:57:03.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:03.634+0000 7fed4b000640 1 --2- 172.21.15.131:0/3704195153 >> v2:172.21.15.187:3300/0 conn(0x7fed44071b70 0x7fed440b3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:03.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:03.634+0000 7fed4b000640 1 --2- 172.21.15.131:0/3704195153 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed4406ebc0 0x7fed440cedf0 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:03.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:03.634+0000 7fed4b000640 1 -- 172.21.15.131:0/3704195153 >> 172.21.15.131:0/3704195153 conn(0x7fed440cff40 msgr2=0x7fed440d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:57:03.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:03.634+0000 7fed4b000640 1 -- 172.21.15.131:0/3704195153 shutdown_connections 2024-05-26T22:57:03.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:03.634+0000 7fed4b000640 1 -- 172.21.15.131:0/3704195153 wait complete. 2024-05-26T22:57:03.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:03.635+0000 7fed4b000640 1 Processor -- start 2024-05-26T22:57:03.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:03.636+0000 7fed4b000640 1 -- start start 2024-05-26T22:57:03.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:03.636+0000 7fed4b000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed4406ebc0 0x7fed44141430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:03.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:03.636+0000 7fed4b000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fed44071b70 0x7fed4415e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:03.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:03.636+0000 7fed4b000640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fed44144340 0x7fed4413e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:03.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:03.636+0000 7fed4b000640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fed440b4260 con 0x7fed4406ebc0 2024-05-26T22:57:03.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:03.636+0000 7fed4b000640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fed44071f80 con 0x7fed44144340 2024-05-26T22:57:03.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:03.636+0000 7fed4b000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fed440b43e0 con 0x7fed44071b70 2024-05-26T22:57:03.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:03.637+0000 7fed49c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed4406ebc0 0x7fed44141430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:03.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:03.637+0000 7fed49c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed4406ebc0 0x7fed44141430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41120/0 (socket says 172.21.15.131:41120) 2024-05-26T22:57:03.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:03.637+0000 7fed49c00640 1 -- 172.21.15.131:0/2122962690 learned_addr learned my addr 172.21.15.131:0/2122962690 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:57:03.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:03.637+0000 7fed49c00640 1 -- 172.21.15.131:0/2122962690 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fed44071b70 msgr2=0x7fed4415e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:03.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:03.637+0000 7fed49c00640 1 --2- 172.21.15.131:0/2122962690 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fed44071b70 0x7fed4415e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:03.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:03.637+0000 7fed49c00640 1 -- 172.21.15.131:0/2122962690 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fed44144340 msgr2=0x7fed4413e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:03.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:03.637+0000 7fed49c00640 1 --2- 172.21.15.131:0/2122962690 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fed44144340 0x7fed4413e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:03.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:03.637+0000 7fed49c00640 1 -- 172.21.15.131:0/2122962690 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed441428e0 con 0x7fed4406ebc0 2024-05-26T22:57:03.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:03.637+0000 7fed49c00640 1 --2- 172.21.15.131:0/2122962690 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed4406ebc0 0x7fed44141430 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7fed3800b410 tx=0x7fed3800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:57:03.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:03.638+0000 7fed41600640 1 -- 172.21.15.131:0/2122962690 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed38015020 con 0x7fed4406ebc0 2024-05-26T22:57:03.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:03.638+0000 7fed41600640 1 -- 172.21.15.131:0/2122962690 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed38011830 con 0x7fed4406ebc0 2024-05-26T22:57:03.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:03.638+0000 7fed4b000640 1 -- 172.21.15.131:0/2122962690 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed44141cd0 con 0x7fed4406ebc0 2024-05-26T22:57:03.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:03.638+0000 7fed41600640 1 -- 172.21.15.131:0/2122962690 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed38011c20 con 0x7fed4406ebc0 2024-05-26T22:57:03.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:03.638+0000 7fed4b000640 1 -- 172.21.15.131:0/2122962690 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed44142290 con 0x7fed4406ebc0 2024-05-26T22:57:03.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:03.639+0000 7fed4b000640 1 -- 172.21.15.131:0/2122962690 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed440694e0 con 0x7fed4406ebc0 2024-05-26T22:57:03.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:03.640+0000 7fed41600640 1 -- 172.21.15.131:0/2122962690 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fed38018020 con 0x7fed4406ebc0 2024-05-26T22:57:03.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:03.641+0000 7fed41600640 1 --2- 172.21.15.131:0/2122962690 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fed2406be60 0x7fed2406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:03.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:03.641+0000 7fed41600640 1 -- 172.21.15.131:0/2122962690 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5115+0+0 (secure 0 0 0) 0x7fed3808a660 con 0x7fed4406ebc0 2024-05-26T22:57:03.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:03.641+0000 7fed49200640 1 --2- 172.21.15.131:0/2122962690 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fed2406be60 0x7fed2406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:03.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:03.642+0000 7fed49200640 1 --2- 172.21.15.131:0/2122962690 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fed2406be60 0x7fed2406e320 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fed30034000 tx=0x7fed3002c000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:57:03.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:03.645+0000 7fed41600640 1 -- 172.21.15.131:0/2122962690 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fed3801f070 con 0x7fed4406ebc0 2024-05-26T22:57:03.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:03.839+0000 7fed4b000640 1 -- 172.21.15.131:0/2122962690 --> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fed44168db0 con 0x7fed2406be60 2024-05-26T22:57:03.840 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:03.840+0000 7fed41600640 1 -- 172.21.15.131:0/2122962690 <== mgr.4104 v2:172.21.15.131:6832/3484832947 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13514 (secure 0 0 0) 0x7fed44168db0 con 0x7fed2406be60 2024-05-26T22:57:03.841 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:57:03.841 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-05-26T22:57:03.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:03.843+0000 7fed4b000640 1 -- 172.21.15.131:0/2122962690 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fed2406be60 msgr2=0x7fed2406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:03.843 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:03.843+0000 7fed4b000640 1 --2- 172.21.15.131:0/2122962690 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fed2406be60 0x7fed2406e320 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fed30034000 tx=0x7fed3002c000 comp rx=0 tx=0).stop 2024-05-26T22:57:03.843 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:03.843+0000 7fed4b000640 1 -- 172.21.15.131:0/2122962690 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed4406ebc0 msgr2=0x7fed44141430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:03.843 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:03.843+0000 7fed4b000640 1 --2- 172.21.15.131:0/2122962690 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed4406ebc0 0x7fed44141430 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7fed3800b410 tx=0x7fed3800b8d0 comp rx=0 tx=0).stop 2024-05-26T22:57:03.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:03.843+0000 7fed4b000640 1 -- 172.21.15.131:0/2122962690 shutdown_connections 2024-05-26T22:57:03.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:03.843+0000 7fed4b000640 1 --2- 172.21.15.131:0/2122962690 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fed44071b70 0x7fed4415e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:03.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:03.843+0000 7fed4b000640 1 --2- 172.21.15.131:0/2122962690 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fed2406be60 0x7fed2406e320 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:03.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:03.843+0000 7fed4b000640 1 --2- 172.21.15.131:0/2122962690 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fed44144340 0x7fed4413e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:03.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:03.843+0000 7fed4b000640 1 --2- 172.21.15.131:0/2122962690 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed4406ebc0 0x7fed44141430 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:03.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:03.843+0000 7fed4b000640 1 -- 172.21.15.131:0/2122962690 >> 172.21.15.131:0/2122962690 conn(0x7fed440cff40 msgr2=0x7fed440cd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:57:03.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:03.843+0000 7fed4b000640 1 -- 172.21.15.131:0/2122962690 shutdown_connections 2024-05-26T22:57:03.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:03.844+0000 7fed4b000640 1 -- 172.21.15.131:0/2122962690 wait complete. 2024-05-26T22:57:03.862 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":369,"stamp":"2024-05-26T22:57:02.560023+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":99,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":290496,"kb_used_data":11912,"kb_used_omap":12,"kb_used_meta":278451,"kb_avail":754684224,"statfs":{"total":773094113280,"available":772796645376,"internally_reserved":0,"allocated":12197888,"data_stored":5972184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":285134385},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-4613491,"num_objects":-171,"num_object_clones":0,"num_object_copies":-342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-171,"num_whiteouts":0,"num_read":-4,"num_read_kb":-44,"num_write":-435,"num_write_kb":-4914,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-48,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.769343"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":338,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-05-26T22:56:59.023584+0000","last_change":"2024-05-26T22:48:12.068917+0000","last_active":"2024-05-26T22:56:59.023584+0000","last_peered":"2024-05-26T22:56:59.023584+0000","last_clean":"2024-05-26T22:56:59.023584+0000","last_became_active":"2024-05-26T22:48:12.065934+0000","last_became_peered":"2024-05-26T22:48:12.065934+0000","last_unstale":"2024-05-26T22:56:59.023584+0000","last_undegraded":"2024-05-26T22:56:59.023584+0000","last_fullsized":"2024-05-26T22:56:59.023584+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:10.628122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:10.628122+0000","last_clean_scrub_stamp":"2024-05-26T22:48:10.628122+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:14:58.895911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574957,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37268,"kb_used_data":1924,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334572,"statfs":{"total":96636764160,"available":96598601728,"internally_reserved":0,"allocated":1970176,"data_stored":1189299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574957,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36688,"kb_used_data":1344,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335152,"statfs":{"total":96636764160,"available":96599195648,"internally_reserved":0,"allocated":1376256,"data_stored":598931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574958,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32528,"kb_used_data":1344,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339312,"statfs":{"total":96636764160,"available":96603455488,"internally_reserved":0,"allocated":1376256,"data_stored":598931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574957,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36684,"kb_used_data":1344,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335156,"statfs":{"total":96636764160,"available":96599199744,"internally_reserved":0,"allocated":1376256,"data_stored":598931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574957,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36684,"kb_used_data":1344,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335156,"statfs":{"total":96636764160,"available":96599199744,"internally_reserved":0,"allocated":1376256,"data_stored":598931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574957,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36684,"kb_used_data":1344,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335156,"statfs":{"total":96636764160,"available":96599199744,"internally_reserved":0,"allocated":1376256,"data_stored":598931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574957,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36692,"kb_used_data":1344,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335148,"statfs":{"total":96636764160,"available":96599191552,"internally_reserved":0,"allocated":1376256,"data_stored":598931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574957,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37268,"kb_used_data":1924,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334572,"statfs":{"total":96636764160,"available":96598601728,"internally_reserved":0,"allocated":1970176,"data_stored":1189299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-26T22:57:03.863 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-26T22:57:04.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.055+0000 7f613c400640 1 Processor -- start 2024-05-26T22:57:04.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.055+0000 7f613c400640 1 -- start start 2024-05-26T22:57:04.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.057+0000 7f613c400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f61340ad250 0x7f61340b3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:04.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.057+0000 7f613c400640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f61341569d0 0x7f61340af790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:04.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.057+0000 7f613c400640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f613415cb20 con 0x7f613406ee20 2024-05-26T22:57:04.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.057+0000 7f613c400640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f613415ce00 con 0x7f61341569d0 2024-05-26T22:57:04.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.057+0000 7f613c400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f613415d0e0 con 0x7f61340ad250 2024-05-26T22:57:04.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.057+0000 7f6139a00640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f613406ee20 0x7f61341562c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.131:47456/0 (socket says 172.21.15.131:47456) 2024-05-26T22:57:04.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.057+0000 7f6139a00640 1 -- 172.21.15.131:0/451657109 learned_addr learned my addr 172.21.15.131:0/451657109 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:57:04.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.058+0000 7f6139000640 1 --2- 172.21.15.131:0/451657109 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f61340ad250 0x7f61340b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:04.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.058+0000 7f613a400640 1 --2- 172.21.15.131:0/451657109 >> v2:172.21.15.187:3300/0 conn(0x7f61341569d0 0x7f61340af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:04.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.058+0000 7f6139000640 1 -- 172.21.15.131:0/451657109 >> v2:172.21.15.187:3300/0 conn(0x7f61341569d0 msgr2=0x7f61340af790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:57:04.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.058+0000 7f6139000640 1 --2- 172.21.15.131:0/451657109 >> v2:172.21.15.187:3300/0 conn(0x7f61341569d0 0x7f61340af790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:04.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.058+0000 7f6139000640 1 -- 172.21.15.131:0/451657109 >> v1:172.21.15.187:6789/0 conn(0x7f613406ee20 legacy=0x7f61341562c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:04.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.058+0000 7f6139000640 1 -- 172.21.15.131:0/451657109 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f613415d3c0 con 0x7f61340ad250 2024-05-26T22:57:04.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.060+0000 7f6139000640 1 --2- 172.21.15.131:0/451657109 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f61340ad250 0x7f61340b3190 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f61200097d0 tx=0x7f612002a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd322f2b1ea85875 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:57:04.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.060+0000 7f6133e00640 1 -- 172.21.15.131:0/451657109 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6120037070 con 0x7f61340ad250 2024-05-26T22:57:04.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.061+0000 7f6133e00640 1 -- 172.21.15.131:0/451657109 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6120032440 con 0x7f61340ad250 2024-05-26T22:57:04.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.061+0000 7f6133e00640 1 -- 172.21.15.131:0/451657109 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6120032810 con 0x7f61340ad250 2024-05-26T22:57:04.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.061+0000 7f613c400640 1 -- 172.21.15.131:0/451657109 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f61340ad250 msgr2=0x7f61340b3190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:04.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.061+0000 7f613c400640 1 --2- 172.21.15.131:0/451657109 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f61340ad250 0x7f61340b3190 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f61200097d0 tx=0x7f612002a1c0 comp rx=0 tx=0).stop 2024-05-26T22:57:04.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.061+0000 7f613c400640 1 -- 172.21.15.131:0/451657109 shutdown_connections 2024-05-26T22:57:04.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.061+0000 7f613c400640 1 --2- 172.21.15.131:0/451657109 >> v2:172.21.15.187:3300/0 conn(0x7f61341569d0 0x7f61340af790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:04.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.061+0000 7f613c400640 1 --2- 172.21.15.131:0/451657109 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f61340ad250 0x7f61340b3190 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:04.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.061+0000 7f613c400640 1 -- 172.21.15.131:0/451657109 >> 172.21.15.131:0/451657109 conn(0x7f61340b1e90 msgr2=0x7f61340b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:57:04.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.061+0000 7f613c400640 1 -- 172.21.15.131:0/451657109 shutdown_connections 2024-05-26T22:57:04.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.061+0000 7f613c400640 1 -- 172.21.15.131:0/451657109 wait complete. 2024-05-26T22:57:04.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.064+0000 7f613c400640 1 Processor -- start 2024-05-26T22:57:04.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.064+0000 7f613c400640 1 -- start start 2024-05-26T22:57:04.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.065+0000 7f613c400640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f613406ee20 0x7f6134154160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:04.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.065+0000 7f613c400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61340ad250 0x7f6134151f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:04.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.065+0000 7f613c400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f61341569d0 0x7f61341e70a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:04.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.065+0000 7f613c400640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f613415c900 con 0x7f61340ad250 2024-05-26T22:57:04.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.065+0000 7f613c400640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f613415c780 con 0x7f613406ee20 2024-05-26T22:57:04.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.065+0000 7f613c400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f613406f200 con 0x7f61341569d0 2024-05-26T22:57:04.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.066+0000 7f6139000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61340ad250 0x7f6134151f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:04.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.066+0000 7f6139000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61340ad250 0x7f6134151f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41122/0 (socket says 172.21.15.131:41122) 2024-05-26T22:57:04.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.066+0000 7f6139000640 1 -- 172.21.15.131:0/2805482768 learned_addr learned my addr 172.21.15.131:0/2805482768 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:57:04.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.066+0000 7f6139a00640 1 --2- 172.21.15.131:0/2805482768 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f613406ee20 0x7f6134154160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:04.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.066+0000 7f613a400640 1 --2- 172.21.15.131:0/2805482768 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f61341569d0 0x7f61341e70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:04.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.066+0000 7f6139000640 1 -- 172.21.15.131:0/2805482768 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f61341569d0 msgr2=0x7f61341e70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:04.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.066+0000 7f6139000640 1 --2- 172.21.15.131:0/2805482768 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f61341569d0 0x7f61341e70a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:04.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.066+0000 7f6139000640 1 -- 172.21.15.131:0/2805482768 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f613406ee20 msgr2=0x7f6134154160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:04.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.066+0000 7f6139000640 1 --2- 172.21.15.131:0/2805482768 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f613406ee20 0x7f6134154160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:04.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.066+0000 7f6139000640 1 -- 172.21.15.131:0/2805482768 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6134152920 con 0x7f61340ad250 2024-05-26T22:57:04.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.066+0000 7f6139a00640 1 --2- 172.21.15.131:0/2805482768 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f613406ee20 0x7f6134154160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:57:04.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.067+0000 7f6139000640 1 --2- 172.21.15.131:0/2805482768 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61340ad250 0x7f6134151f40 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f61200098f0 tx=0x7f61200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:57:04.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.067+0000 7f6131600640 1 -- 172.21.15.131:0/2805482768 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6120037040 con 0x7f61340ad250 2024-05-26T22:57:04.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.067+0000 7f6131600640 1 -- 172.21.15.131:0/2805482768 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6120042070 con 0x7f61340ad250 2024-05-26T22:57:04.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.067+0000 7f6131600640 1 -- 172.21.15.131:0/2805482768 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f612003b930 con 0x7f61340ad250 2024-05-26T22:57:04.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.067+0000 7f613c400640 1 -- 172.21.15.131:0/2805482768 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6134152c00 con 0x7f61340ad250 2024-05-26T22:57:04.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.067+0000 7f613c400640 1 -- 172.21.15.131:0/2805482768 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61341531c0 con 0x7f61340ad250 2024-05-26T22:57:04.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.068+0000 7f613c400640 1 -- 172.21.15.131:0/2805482768 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61340694e0 con 0x7f61340ad250 2024-05-26T22:57:04.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.069+0000 7f6131600640 1 -- 172.21.15.131:0/2805482768 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f612003bad0 con 0x7f61340ad250 2024-05-26T22:57:04.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.070+0000 7f6131600640 1 --2- 172.21.15.131:0/2805482768 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f611406be30 0x7f611406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:04.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.070+0000 7f6131600640 1 -- 172.21.15.131:0/2805482768 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5115+0+0 (secure 0 0 0) 0x7f61200ab4b0 con 0x7f61340ad250 2024-05-26T22:57:04.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.071+0000 7f6139a00640 1 --2- 172.21.15.131:0/2805482768 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f611406be30 0x7f611406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:04.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.071+0000 7f6139a00640 1 --2- 172.21.15.131:0/2805482768 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f611406be30 0x7f611406e2f0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f6128005880 tx=0x7f6128009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:57:04.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.075+0000 7f6131600640 1 -- 172.21.15.131:0/2805482768 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f612007a0a0 con 0x7f61340ad250 2024-05-26T22:57:04.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.268+0000 7f613c400640 1 -- 172.21.15.131:0/2805482768 --> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f61340b38f0 con 0x7f611406be30 2024-05-26T22:57:04.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.270+0000 7f6131600640 1 -- 172.21.15.131:0/2805482768 <== mgr.4104 v2:172.21.15.131:6832/3484832947 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13514 (secure 0 0 0) 0x7f61340b38f0 con 0x7f611406be30 2024-05-26T22:57:04.270 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:57:04.270 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-05-26T22:57:04.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.272+0000 7f613c400640 1 -- 172.21.15.131:0/2805482768 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f611406be30 msgr2=0x7f611406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:04.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.272+0000 7f613c400640 1 --2- 172.21.15.131:0/2805482768 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f611406be30 0x7f611406e2f0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f6128005880 tx=0x7f6128009040 comp rx=0 tx=0).stop 2024-05-26T22:57:04.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.272+0000 7f613c400640 1 -- 172.21.15.131:0/2805482768 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61340ad250 msgr2=0x7f6134151f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:04.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.272+0000 7f613c400640 1 --2- 172.21.15.131:0/2805482768 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61340ad250 0x7f6134151f40 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f61200098f0 tx=0x7f61200046c0 comp rx=0 tx=0).stop 2024-05-26T22:57:04.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.272+0000 7f613c400640 1 -- 172.21.15.131:0/2805482768 shutdown_connections 2024-05-26T22:57:04.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.272+0000 7f613c400640 1 --2- 172.21.15.131:0/2805482768 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f61341569d0 0x7f61341e70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:04.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.272+0000 7f613c400640 1 --2- 172.21.15.131:0/2805482768 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f611406be30 0x7f611406e2f0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:04.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.272+0000 7f613c400640 1 --2- 172.21.15.131:0/2805482768 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61340ad250 0x7f6134151f40 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:04.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.272+0000 7f613c400640 1 --2- 172.21.15.131:0/2805482768 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f613406ee20 0x7f6134154160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:04.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.272+0000 7f613c400640 1 -- 172.21.15.131:0/2805482768 >> 172.21.15.131:0/2805482768 conn(0x7f61340b1e90 msgr2=0x7f613406e2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:57:04.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.273+0000 7f613c400640 1 -- 172.21.15.131:0/2805482768 shutdown_connections 2024-05-26T22:57:04.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.273+0000 7f613c400640 1 -- 172.21.15.131:0/2805482768 wait complete. 2024-05-26T22:57:04.292 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":369,"stamp":"2024-05-26T22:57:02.560023+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":99,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":290496,"kb_used_data":11912,"kb_used_omap":12,"kb_used_meta":278451,"kb_avail":754684224,"statfs":{"total":773094113280,"available":772796645376,"internally_reserved":0,"allocated":12197888,"data_stored":5972184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":285134385},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-4613491,"num_objects":-171,"num_object_clones":0,"num_object_copies":-342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-171,"num_whiteouts":0,"num_read":-4,"num_read_kb":-44,"num_write":-435,"num_write_kb":-4914,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-48,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.769343"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":338,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-05-26T22:56:59.023584+0000","last_change":"2024-05-26T22:48:12.068917+0000","last_active":"2024-05-26T22:56:59.023584+0000","last_peered":"2024-05-26T22:56:59.023584+0000","last_clean":"2024-05-26T22:56:59.023584+0000","last_became_active":"2024-05-26T22:48:12.065934+0000","last_became_peered":"2024-05-26T22:48:12.065934+0000","last_unstale":"2024-05-26T22:56:59.023584+0000","last_undegraded":"2024-05-26T22:56:59.023584+0000","last_fullsized":"2024-05-26T22:56:59.023584+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:10.628122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:10.628122+0000","last_clean_scrub_stamp":"2024-05-26T22:48:10.628122+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:14:58.895911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574957,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37268,"kb_used_data":1924,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334572,"statfs":{"total":96636764160,"available":96598601728,"internally_reserved":0,"allocated":1970176,"data_stored":1189299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574957,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36688,"kb_used_data":1344,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335152,"statfs":{"total":96636764160,"available":96599195648,"internally_reserved":0,"allocated":1376256,"data_stored":598931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574958,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32528,"kb_used_data":1344,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339312,"statfs":{"total":96636764160,"available":96603455488,"internally_reserved":0,"allocated":1376256,"data_stored":598931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574957,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36684,"kb_used_data":1344,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335156,"statfs":{"total":96636764160,"available":96599199744,"internally_reserved":0,"allocated":1376256,"data_stored":598931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574957,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36684,"kb_used_data":1344,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335156,"statfs":{"total":96636764160,"available":96599199744,"internally_reserved":0,"allocated":1376256,"data_stored":598931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574957,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36684,"kb_used_data":1344,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335156,"statfs":{"total":96636764160,"available":96599199744,"internally_reserved":0,"allocated":1376256,"data_stored":598931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574957,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36692,"kb_used_data":1344,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335148,"statfs":{"total":96636764160,"available":96599191552,"internally_reserved":0,"allocated":1376256,"data_stored":598931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574957,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37268,"kb_used_data":1924,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334572,"statfs":{"total":96636764160,"available":96598601728,"internally_reserved":0,"allocated":1970176,"data_stored":1189299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-26T22:57:04.292 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-26T22:57:04.293 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-26T22:57:04.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.485+0000 7f1bdf000640 1 Processor -- start 2024-05-26T22:57:04.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.485+0000 7f1bdf000640 1 -- start start 2024-05-26T22:57:04.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.487+0000 7f1bdf000640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f1bd806ee20 0x7f1bd80b3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:04.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.487+0000 7f1bdf000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1bd80ad250 0x7f1bd80af790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:04.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.487+0000 7f1bdf000640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f1bd806f200 0x7f1bd81562c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:04.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.487+0000 7f1bdf000640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f1bd815cb20 con 0x7f1bd806ee20 2024-05-26T22:57:04.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.487+0000 7f1bdf000640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f1bd815ce00 con 0x7f1bd806f200 2024-05-26T22:57:04.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.487+0000 7f1bdf000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1bd815d0e0 con 0x7f1bd80ad250 2024-05-26T22:57:04.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.487+0000 7f1bd7e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f1bd806ee20 0x7f1bd80b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:04.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.487+0000 7f1bd7e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f1bd806ee20 0x7f1bd80b3190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57136/0 (socket says 172.21.15.131:57136) 2024-05-26T22:57:04.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.487+0000 7f1bd7e00640 1 -- 172.21.15.131:0/2190569716 learned_addr learned my addr 172.21.15.131:0/2190569716 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:57:04.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.488+0000 7f1bdd000640 1 --2- 172.21.15.131:0/2190569716 >> v2:172.21.15.187:3300/0 conn(0x7f1bd806f200 0x7f1bd81562c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:04.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.488+0000 7f1bd7400640 1 --2- 172.21.15.131:0/2190569716 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1bd80ad250 0x7f1bd80af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:04.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.488+0000 7f1bd7e00640 1 -- 172.21.15.131:0/2190569716 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1bd80ad250 msgr2=0x7f1bd80af790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:57:04.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.488+0000 7f1bd7e00640 1 --2- 172.21.15.131:0/2190569716 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1bd80ad250 0x7f1bd80af790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:04.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.488+0000 7f1bd7e00640 1 -- 172.21.15.131:0/2190569716 >> v2:172.21.15.187:3300/0 conn(0x7f1bd806f200 msgr2=0x7f1bd81562c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:57:04.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.488+0000 7f1bd7e00640 1 --2- 172.21.15.131:0/2190569716 >> v2:172.21.15.187:3300/0 conn(0x7f1bd806f200 0x7f1bd81562c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:04.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.488+0000 7f1bd7e00640 1 -- 172.21.15.131:0/2190569716 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bd815d3c0 con 0x7f1bd806ee20 2024-05-26T22:57:04.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.488+0000 7f1bd7400640 1 --2- 172.21.15.131:0/2190569716 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1bd80ad250 0x7f1bd80af790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:57:04.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.488+0000 7f1bdd000640 1 --2- 172.21.15.131:0/2190569716 >> v2:172.21.15.187:3300/0 conn(0x7f1bd806f200 0x7f1bd81562c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:57:04.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.489+0000 7f1bd7e00640 1 --2- 172.21.15.131:0/2190569716 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1bd806ee20 0x7f1bd80b3190 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f1bd80adf50 tx=0x7f1bc802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d02a5a9a589ea62d server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:57:04.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.489+0000 7f1bd6a00640 1 -- 172.21.15.131:0/2190569716 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc8037070 con 0x7f1bd806ee20 2024-05-26T22:57:04.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.489+0000 7f1bd6a00640 1 -- 172.21.15.131:0/2190569716 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bc8032440 con 0x7f1bd806ee20 2024-05-26T22:57:04.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.489+0000 7f1bd6a00640 1 -- 172.21.15.131:0/2190569716 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc8032810 con 0x7f1bd806ee20 2024-05-26T22:57:04.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.490+0000 7f1bdf000640 1 -- 172.21.15.131:0/2190569716 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1bd806ee20 msgr2=0x7f1bd80b3190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:04.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.490+0000 7f1bdf000640 1 --2- 172.21.15.131:0/2190569716 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1bd806ee20 0x7f1bd80b3190 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f1bd80adf50 tx=0x7f1bc802a1e0 comp rx=0 tx=0).stop 2024-05-26T22:57:04.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.490+0000 7f1bdf000640 1 -- 172.21.15.131:0/2190569716 shutdown_connections 2024-05-26T22:57:04.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.490+0000 7f1bdf000640 1 --2- 172.21.15.131:0/2190569716 >> v2:172.21.15.187:3300/0 conn(0x7f1bd806f200 0x7f1bd81562c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:04.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.490+0000 7f1bdf000640 1 --2- 172.21.15.131:0/2190569716 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1bd80ad250 0x7f1bd80af790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:04.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.490+0000 7f1bdf000640 1 --2- 172.21.15.131:0/2190569716 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1bd806ee20 0x7f1bd80b3190 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:04.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.490+0000 7f1bdf000640 1 -- 172.21.15.131:0/2190569716 >> 172.21.15.131:0/2190569716 conn(0x7f1bd80b1e90 msgr2=0x7f1bd80b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:57:04.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.490+0000 7f1bdf000640 1 -- 172.21.15.131:0/2190569716 shutdown_connections 2024-05-26T22:57:04.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.490+0000 7f1bdf000640 1 -- 172.21.15.131:0/2190569716 wait complete. 2024-05-26T22:57:04.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.492+0000 7f1bdf000640 1 Processor -- start 2024-05-26T22:57:04.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.492+0000 7f1bdf000640 1 -- start start 2024-05-26T22:57:04.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.493+0000 7f1bdf000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1bd806ee20 0x7f1bd81f7470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:04.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.493+0000 7f1bdf000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1bd806f200 0x7f1bd815b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:04.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.493+0000 7f1bdf000640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1bd80ad250 0x7f1bd815b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:04.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.493+0000 7f1bdf000640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f1bd815c900 con 0x7f1bd806f200 2024-05-26T22:57:04.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.493+0000 7f1bdf000640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1bd815c780 con 0x7f1bd80ad250 2024-05-26T22:57:04.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.493+0000 7f1bdf000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1bd8156800 con 0x7f1bd806ee20 2024-05-26T22:57:04.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.493+0000 7f1bd7400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1bd806f200 0x7f1bd815b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:04.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.493+0000 7f1bd7400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1bd806f200 0x7f1bd815b020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57146/0 (socket says 172.21.15.131:57146) 2024-05-26T22:57:04.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.493+0000 7f1bd7400640 1 -- 172.21.15.131:0/1106449075 learned_addr learned my addr 172.21.15.131:0/1106449075 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:57:04.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.494+0000 7f1bd7e00640 1 --2- 172.21.15.131:0/1106449075 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1bd806ee20 0x7f1bd81f7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:04.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.494+0000 7f1bd7400640 1 -- 172.21.15.131:0/1106449075 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1bd806ee20 msgr2=0x7f1bd81f7470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:04.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.494+0000 7f1bd7400640 1 --2- 172.21.15.131:0/1106449075 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1bd806ee20 0x7f1bd81f7470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:04.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.494+0000 7f1bd7400640 1 -- 172.21.15.131:0/1106449075 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1bd80ad250 msgr2=0x7f1bd815b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:04.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.494+0000 7f1bd7400640 1 --2- 172.21.15.131:0/1106449075 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1bd80ad250 0x7f1bd815b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:04.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.494+0000 7f1bd7400640 1 -- 172.21.15.131:0/1106449075 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bd814fd30 con 0x7f1bd806f200 2024-05-26T22:57:04.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.494+0000 7f1bd7400640 1 --2- 172.21.15.131:0/1106449075 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1bd806f200 0x7f1bd815b020 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f1bc400c820 tx=0x7f1bc400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:57:04.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.494+0000 7f1bcfe00640 1 -- 172.21.15.131:0/1106449075 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc4015070 con 0x7f1bd806f200 2024-05-26T22:57:04.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.494+0000 7f1bcfe00640 1 -- 172.21.15.131:0/1106449075 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bc4007a30 con 0x7f1bd806f200 2024-05-26T22:57:04.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.494+0000 7f1bdf000640 1 -- 172.21.15.131:0/1106449075 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bd8150010 con 0x7f1bd806f200 2024-05-26T22:57:04.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.494+0000 7f1bcfe00640 1 -- 172.21.15.131:0/1106449075 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc4007de0 con 0x7f1bd806f200 2024-05-26T22:57:04.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.494+0000 7f1bdf000640 1 -- 172.21.15.131:0/1106449075 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bd8151900 con 0x7f1bd806f200 2024-05-26T22:57:04.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.496+0000 7f1bdf000640 1 -- 172.21.15.131:0/1106449075 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bd80694e0 con 0x7f1bd806f200 2024-05-26T22:57:04.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.497+0000 7f1bcfe00640 1 -- 172.21.15.131:0/1106449075 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1bc4018070 con 0x7f1bd806f200 2024-05-26T22:57:04.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.498+0000 7f1bcfe00640 1 --2- 172.21.15.131:0/1106449075 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f1bb806be30 0x7f1bb806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:04.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.498+0000 7f1bcfe00640 1 -- 172.21.15.131:0/1106449075 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5115+0+0 (secure 0 0 0) 0x7f1bc408ab50 con 0x7f1bd806f200 2024-05-26T22:57:04.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.498+0000 7f1bd7e00640 1 --2- 172.21.15.131:0/1106449075 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f1bb806be30 0x7f1bb806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:04.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.499+0000 7f1bd7e00640 1 --2- 172.21.15.131:0/1106449075 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f1bb806be30 0x7f1bb806e2f0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f1bc8034000 tx=0x7f1bc802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:57:04.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.503+0000 7f1bcfe00640 1 -- 172.21.15.131:0/1106449075 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f1bc40596d0 con 0x7f1bd806f200 2024-05-26T22:57:04.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.702+0000 7f1bdf000640 1 -- 172.21.15.131:0/1106449075 --> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1bd80aef10 con 0x7f1bb806be30 2024-05-26T22:57:04.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.703+0000 7f1bcfe00640 1 -- 172.21.15.131:0/1106449075 <== mgr.4104 v2:172.21.15.131:6832/3484832947 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13514 (secure 0 0 0) 0x7f1bd80aef10 con 0x7f1bb806be30 2024-05-26T22:57:04.703 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:57:04.704 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-05-26T22:57:04.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.706+0000 7f1bdf000640 1 -- 172.21.15.131:0/1106449075 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f1bb806be30 msgr2=0x7f1bb806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:04.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.706+0000 7f1bdf000640 1 --2- 172.21.15.131:0/1106449075 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f1bb806be30 0x7f1bb806e2f0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f1bc8034000 tx=0x7f1bc802e000 comp rx=0 tx=0).stop 2024-05-26T22:57:04.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.706+0000 7f1bdf000640 1 -- 172.21.15.131:0/1106449075 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1bd806f200 msgr2=0x7f1bd815b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:04.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.706+0000 7f1bdf000640 1 --2- 172.21.15.131:0/1106449075 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1bd806f200 0x7f1bd815b020 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f1bc400c820 tx=0x7f1bc400cce0 comp rx=0 tx=0).stop 2024-05-26T22:57:04.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.706+0000 7f1bdf000640 1 -- 172.21.15.131:0/1106449075 shutdown_connections 2024-05-26T22:57:04.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.706+0000 7f1bdf000640 1 --2- 172.21.15.131:0/1106449075 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f1bb806be30 0x7f1bb806e2f0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:04.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.706+0000 7f1bdf000640 1 --2- 172.21.15.131:0/1106449075 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1bd80ad250 0x7f1bd815b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:04.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.706+0000 7f1bdf000640 1 --2- 172.21.15.131:0/1106449075 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1bd806f200 0x7f1bd815b020 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:04.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.706+0000 7f1bdf000640 1 --2- 172.21.15.131:0/1106449075 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1bd806ee20 0x7f1bd81f7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:04.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.706+0000 7f1bdf000640 1 -- 172.21.15.131:0/1106449075 >> 172.21.15.131:0/1106449075 conn(0x7f1bd80b1e90 msgr2=0x7f1bd80b2f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:57:04.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.707+0000 7f1bdf000640 1 -- 172.21.15.131:0/1106449075 shutdown_connections 2024-05-26T22:57:04.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.707+0000 7f1bdf000640 1 -- 172.21.15.131:0/1106449075 wait complete. 2024-05-26T22:57:04.724 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":370,"stamp":"2024-05-26T22:57:04.560437+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":99,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":290496,"kb_used_data":11912,"kb_used_omap":12,"kb_used_meta":278451,"kb_avail":754684224,"statfs":{"total":773094113280,"available":772796645376,"internally_reserved":0,"allocated":12197888,"data_stored":5972184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":285134385},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-4613491,"num_objects":-171,"num_object_clones":0,"num_object_copies":-342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-171,"num_whiteouts":0,"num_read":-4,"num_read_kb":-44,"num_write":-435,"num_write_kb":-4914,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-48,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002039"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":338,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-05-26T22:56:59.023584+0000","last_change":"2024-05-26T22:48:12.068917+0000","last_active":"2024-05-26T22:56:59.023584+0000","last_peered":"2024-05-26T22:56:59.023584+0000","last_clean":"2024-05-26T22:56:59.023584+0000","last_became_active":"2024-05-26T22:48:12.065934+0000","last_became_peered":"2024-05-26T22:48:12.065934+0000","last_unstale":"2024-05-26T22:56:59.023584+0000","last_undegraded":"2024-05-26T22:56:59.023584+0000","last_fullsized":"2024-05-26T22:56:59.023584+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:10.628122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:10.628122+0000","last_clean_scrub_stamp":"2024-05-26T22:48:10.628122+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:14:58.895911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574957,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37268,"kb_used_data":1924,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334572,"statfs":{"total":96636764160,"available":96598601728,"internally_reserved":0,"allocated":1970176,"data_stored":1189299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574957,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36688,"kb_used_data":1344,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335152,"statfs":{"total":96636764160,"available":96599195648,"internally_reserved":0,"allocated":1376256,"data_stored":598931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574958,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32528,"kb_used_data":1344,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339312,"statfs":{"total":96636764160,"available":96603455488,"internally_reserved":0,"allocated":1376256,"data_stored":598931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574957,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36684,"kb_used_data":1344,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335156,"statfs":{"total":96636764160,"available":96599199744,"internally_reserved":0,"allocated":1376256,"data_stored":598931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574957,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36684,"kb_used_data":1344,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335156,"statfs":{"total":96636764160,"available":96599199744,"internally_reserved":0,"allocated":1376256,"data_stored":598931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574957,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36684,"kb_used_data":1344,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335156,"statfs":{"total":96636764160,"available":96599199744,"internally_reserved":0,"allocated":1376256,"data_stored":598931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574957,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36692,"kb_used_data":1344,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335148,"statfs":{"total":96636764160,"available":96599191552,"internally_reserved":0,"allocated":1376256,"data_stored":598931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574957,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37268,"kb_used_data":1924,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334572,"statfs":{"total":96636764160,"available":96598601728,"internally_reserved":0,"allocated":1970176,"data_stored":1189299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-26T22:57:04.725 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-26T22:57:04.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.904+0000 7f65d5000640 1 Processor -- start 2024-05-26T22:57:04.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.905+0000 7f65d5000640 1 -- start start 2024-05-26T22:57:04.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.906+0000 7f65d5000640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f65d015e580 0x7f65d006c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:04.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.906+0000 7f65d5000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f65d0070dd0 0x7f65d00cbe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:04.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.906+0000 7f65d5000640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65d00714f0 con 0x7f65d006d260 2024-05-26T22:57:04.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.906+0000 7f65d5000640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f65d00cb990 con 0x7f65d015e580 2024-05-26T22:57:04.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.906+0000 7f65d5000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f65d00717d0 con 0x7f65d0070dd0 2024-05-26T22:57:04.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.906+0000 7f65ce800640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f65d006d260 0x7f65d00d0930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:37598/0 (socket says 172.21.15.131:37598) 2024-05-26T22:57:04.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.906+0000 7f65ce800640 1 -- 172.21.15.131:0/3036730203 learned_addr learned my addr 172.21.15.131:0/3036730203 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:57:04.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.907+0000 7f65cd400640 1 -- 172.21.15.131:0/3036730203 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 523563066 0 0) 0x7f65d00714f0 con 0x7f65d006d260 2024-05-26T22:57:04.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.907+0000 7f65cde00640 1 --2- 172.21.15.131:0/3036730203 >> v2:172.21.15.187:3300/0 conn(0x7f65d015e580 0x7f65d006c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:04.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.907+0000 7f65c5400640 1 --2- 172.21.15.131:0/3036730203 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f65d0070dd0 0x7f65d00cbe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:04.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.907+0000 7f65cd400640 1 -- 172.21.15.131:0/3036730203 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f65b4003520 con 0x7f65d006d260 2024-05-26T22:57:04.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.907+0000 7f65cd400640 1 -- 172.21.15.131:0/3036730203 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3345455858 0 0) 0x7f65b4003520 con 0x7f65d006d260 2024-05-26T22:57:04.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.907+0000 7f65cd400640 1 -- 172.21.15.131:0/3036730203 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f65d0070dd0 msgr2=0x7f65d00cbe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:57:04.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.907+0000 7f65cd400640 1 --2- 172.21.15.131:0/3036730203 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f65d0070dd0 0x7f65d00cbe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:04.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.907+0000 7f65cd400640 1 -- 172.21.15.131:0/3036730203 >> v2:172.21.15.187:3300/0 conn(0x7f65d015e580 msgr2=0x7f65d006c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:57:04.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.907+0000 7f65cd400640 1 --2- 172.21.15.131:0/3036730203 >> v2:172.21.15.187:3300/0 conn(0x7f65d015e580 0x7f65d006c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:04.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.907+0000 7f65cd400640 1 -- 172.21.15.131:0/3036730203 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65d0071ab0 con 0x7f65d006d260 2024-05-26T22:57:04.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.907+0000 7f65cd400640 1 -- 172.21.15.131:0/3036730203 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7f65c80019f0 con 0x7f65d006d260 2024-05-26T22:57:04.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.907+0000 7f65cd400640 1 -- 172.21.15.131:0/3036730203 >> v1:172.21.15.131:6789/0 conn(0x7f65d006d260 legacy=0x7f65d00d0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:04.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.908+0000 7f65cd400640 1 --2- 172.21.15.131:0/3036730203 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f65d0070dd0 0x7f65b40039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:04.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.908+0000 7f65cd400640 1 --2- 172.21.15.131:0/3036730203 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f65b4003f80 0x7f65b4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:04.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.908+0000 7f65cd400640 1 --2- 172.21.15.131:0/3036730203 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f65b4004d40 0x7f65b40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:04.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.908+0000 7f65cd400640 1 -- 172.21.15.131:0/3036730203 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f65d00cb990 con 0x7f65b4003f80 2024-05-26T22:57:04.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.908+0000 7f65cd400640 1 -- 172.21.15.131:0/3036730203 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f65d00717d0 con 0x7f65b4004d40 2024-05-26T22:57:04.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.908+0000 7f65cd400640 1 -- 172.21.15.131:0/3036730203 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f65b400ba70 con 0x7f65d0070dd0 2024-05-26T22:57:04.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.908+0000 7f65cde00640 1 --2- 172.21.15.131:0/3036730203 >> v2:172.21.15.187:3300/0 conn(0x7f65d015e580 0x7f65d006c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-26T22:57:04.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.908+0000 7f65c5400640 1 --2- 172.21.15.131:0/3036730203 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f65b4003f80 0x7f65b4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:04.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.908+0000 7f65ce800640 1 --2- 172.21.15.131:0/3036730203 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f65b4004d40 0x7f65b40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:04.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.908+0000 7f65cde00640 1 --2- 172.21.15.131:0/3036730203 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f65d0070dd0 0x7f65b40039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:04.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.908+0000 7f65c5400640 1 -- 172.21.15.131:0/3036730203 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f65d0070dd0 msgr2=0x7f65b40039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:57:04.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.908+0000 7f65c5400640 1 --2- 172.21.15.131:0/3036730203 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f65d0070dd0 0x7f65b40039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:04.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.908+0000 7f65c5400640 1 -- 172.21.15.131:0/3036730203 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f65b4004d40 msgr2=0x7f65b40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:57:04.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.908+0000 7f65c5400640 1 --2- 172.21.15.131:0/3036730203 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f65b4004d40 0x7f65b40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:04.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.908+0000 7f65c5400640 1 -- 172.21.15.131:0/3036730203 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65b4003520 con 0x7f65b4003f80 2024-05-26T22:57:04.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.909+0000 7f65cde00640 1 --2- 172.21.15.131:0/3036730203 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f65d0070dd0 0x7f65b40039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:57:04.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.909+0000 7f65c5400640 1 --2- 172.21.15.131:0/3036730203 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f65b4003f80 0x7f65b4004800 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f65b8009a90 tx=0x7f65b4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7b30905f7621b7a server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:57:04.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.910+0000 7f65cd400640 1 -- 172.21.15.131:0/3036730203 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b8036070 con 0x7f65b4003f80 2024-05-26T22:57:04.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.910+0000 7f65cd400640 1 -- 172.21.15.131:0/3036730203 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65b8031400 con 0x7f65b4003f80 2024-05-26T22:57:04.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.910+0000 7f65cd400640 1 -- 172.21.15.131:0/3036730203 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b8009690 con 0x7f65b4003f80 2024-05-26T22:57:04.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.910+0000 7f65d5000640 1 -- 172.21.15.131:0/3036730203 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f65b4003f80 msgr2=0x7f65b4004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:04.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.910+0000 7f65d5000640 1 --2- 172.21.15.131:0/3036730203 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f65b4003f80 0x7f65b4004800 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f65b8009a90 tx=0x7f65b4005830 comp rx=0 tx=0).stop 2024-05-26T22:57:04.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.910+0000 7f65ce800640 1 -- 172.21.15.131:0/3036730203 reap_dead start 2024-05-26T22:57:04.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.910+0000 7f65d5000640 1 -- 172.21.15.131:0/3036730203 shutdown_connections 2024-05-26T22:57:04.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.910+0000 7f65d5000640 1 -- 172.21.15.131:0/3036730203 >> 172.21.15.131:0/3036730203 conn(0x7f65d00b3e00 msgr2=0x7f65d00b4210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:57:04.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.911+0000 7f65d5000640 1 -- 172.21.15.131:0/3036730203 shutdown_connections 2024-05-26T22:57:04.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.911+0000 7f65d5000640 1 -- 172.21.15.131:0/3036730203 wait complete. 2024-05-26T22:57:04.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.912+0000 7f65d5000640 1 Processor -- start 2024-05-26T22:57:04.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.912+0000 7f65d5000640 1 -- start start 2024-05-26T22:57:04.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.913+0000 7f65d5000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f65b4003f80 0x7f65d0152230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:04.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.913+0000 7f65d5000640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f65d00b7c20 0x7f65d014fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:04.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.913+0000 7f65d5000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f65d01f7b10 0x7f65d01f7f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:04.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.913+0000 7f65d5000640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f65d006d6c0 con 0x7f65d01f7b10 2024-05-26T22:57:04.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.913+0000 7f65d5000640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f65d015e960 con 0x7f65d00b7c20 2024-05-26T22:57:04.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.913+0000 7f65d5000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f65d006d840 con 0x7f65b4003f80 2024-05-26T22:57:04.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.914+0000 7f65ce800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f65d01f7b10 0x7f65d01f7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:04.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.914+0000 7f65ce800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f65d01f7b10 0x7f65d01f7f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57166/0 (socket says 172.21.15.131:57166) 2024-05-26T22:57:04.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.914+0000 7f65ce800640 1 -- 172.21.15.131:0/2411890062 learned_addr learned my addr 172.21.15.131:0/2411890062 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:57:04.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.914+0000 7f65c5400640 1 --2- 172.21.15.131:0/2411890062 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f65d00b7c20 0x7f65d014fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:04.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.914+0000 7f65ce800640 1 -- 172.21.15.131:0/2411890062 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f65b4003f80 msgr2=0x7f65d0152230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:04.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.914+0000 7f65cde00640 1 --2- 172.21.15.131:0/2411890062 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f65b4003f80 0x7f65d0152230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:04.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.914+0000 7f65ce800640 1 --2- 172.21.15.131:0/2411890062 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f65b4003f80 0x7f65d0152230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:04.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.914+0000 7f65ce800640 1 -- 172.21.15.131:0/2411890062 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f65d00b7c20 msgr2=0x7f65d014fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:04.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.914+0000 7f65ce800640 1 --2- 172.21.15.131:0/2411890062 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f65d00b7c20 0x7f65d014fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:04.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.914+0000 7f65ce800640 1 -- 172.21.15.131:0/2411890062 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65d015f360 con 0x7f65d01f7b10 2024-05-26T22:57:04.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.914+0000 7f65c5400640 1 --2- 172.21.15.131:0/2411890062 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f65d00b7c20 0x7f65d014fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:57:04.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.914+0000 7f65cde00640 1 --2- 172.21.15.131:0/2411890062 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f65b4003f80 0x7f65d0152230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:57:04.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.914+0000 7f65ce800640 1 --2- 172.21.15.131:0/2411890062 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f65d01f7b10 0x7f65d01f7f90 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f65d00693f0 tx=0x7f65c8007b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:57:04.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.915+0000 7f65c6a00640 1 -- 172.21.15.131:0/2411890062 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c800e960 con 0x7f65d01f7b10 2024-05-26T22:57:04.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.915+0000 7f65c6a00640 1 -- 172.21.15.131:0/2411890062 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65c8018070 con 0x7f65d01f7b10 2024-05-26T22:57:04.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.915+0000 7f65c6a00640 1 -- 172.21.15.131:0/2411890062 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c800ed40 con 0x7f65d01f7b10 2024-05-26T22:57:04.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.915+0000 7f65d5000640 1 -- 172.21.15.131:0/2411890062 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65d00b80a0 con 0x7f65d01f7b10 2024-05-26T22:57:04.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.915+0000 7f65d5000640 1 -- 172.21.15.131:0/2411890062 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65d0152b80 con 0x7f65d01f7b10 2024-05-26T22:57:04.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.916+0000 7f65d5000640 1 -- 172.21.15.131:0/2411890062 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65d00694e0 con 0x7f65d01f7b10 2024-05-26T22:57:04.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.917+0000 7f65c6a00640 1 -- 172.21.15.131:0/2411890062 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f65c801d460 con 0x7f65d01f7b10 2024-05-26T22:57:04.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.918+0000 7f65c6a00640 1 --2- 172.21.15.131:0/2411890062 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f65ac06be30 0x7f65ac06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:04.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.918+0000 7f65c6a00640 1 -- 172.21.15.131:0/2411890062 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5115+0+0 (secure 0 0 0) 0x7f65c808add0 con 0x7f65d01f7b10 2024-05-26T22:57:04.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.918+0000 7f65cde00640 1 --2- 172.21.15.131:0/2411890062 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f65ac06be30 0x7f65ac06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:04.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.919+0000 7f65cde00640 1 --2- 172.21.15.131:0/2411890062 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f65ac06be30 0x7f65ac06e2f0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f65c0006e80 tx=0x7f65c0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:57:04.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:04.926+0000 7f65c6a00640 1 -- 172.21.15.131:0/2411890062 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f65c80599c0 con 0x7f65d01f7b10 2024-05-26T22:57:05.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:05.124+0000 7f65d5000640 1 -- 172.21.15.131:0/2411890062 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f65d00cf6e0 con 0x7f65d01f7b10 2024-05-26T22:57:05.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:05.126+0000 7f65c6a00640 1 -- 172.21.15.131:0/2411890062 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) ==== 74+0+13320 (secure 0 0 0) 0x7f65c80592c0 con 0x7f65d01f7b10 2024-05-26T22:57:05.126 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:57:05.126 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":110,"fsid":"46fd52e0-cb8f-4ad3-b7c6-3a7b0e73525f","created":"2024-05-26T22:47:57.288905+0000","modified":"2024-05-26T22:56:58.794268+0000","last_up_change":"2024-05-26T22:48:09.602207+0000","last_in_change":"2024-05-26T22:48:00.687341+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-26T22:48:10.463339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"814eaf62-a18d-494b-8e27-fd8fb88adea6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6801","nonce":2350940051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6803","nonce":2350940051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6807","nonce":2350940051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":2350940051},{"type":"v1","addr":"172.21.15.131:6805","nonce":2350940051}]},"public_addr":"172.21.15.131:6801/2350940051","cluster_addr":"172.21.15.131:6803/2350940051","heartbeat_back_addr":"172.21.15.131:6807/2350940051","heartbeat_front_addr":"172.21.15.131:6805/2350940051","state":["exists","up"]},{"osd":1,"uuid":"42e66287-f6c6-4e26-af3c-6dfaa3536d27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6825","nonce":1525528247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6827","nonce":1525528247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6831","nonce":1525528247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1525528247},{"type":"v1","addr":"172.21.15.131:6829","nonce":1525528247}]},"public_addr":"172.21.15.131:6825/1525528247","cluster_addr":"172.21.15.131:6827/1525528247","heartbeat_back_addr":"172.21.15.131:6831/1525528247","heartbeat_front_addr":"172.21.15.131:6829/1525528247","state":["exists","up"]},{"osd":2,"uuid":"0da34673-04a2-4fd0-b57a-0ff2b679b043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6817","nonce":3627842210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6819","nonce":3627842210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6823","nonce":3627842210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":3627842210},{"type":"v1","addr":"172.21.15.131:6821","nonce":3627842210}]},"public_addr":"172.21.15.131:6817/3627842210","cluster_addr":"172.21.15.131:6819/3627842210","heartbeat_back_addr":"172.21.15.131:6823/3627842210","heartbeat_front_addr":"172.21.15.131:6821/3627842210","state":["exists","up"]},{"osd":3,"uuid":"cf117a27-d6b2-43d3-ae52-b47988bf8eed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6809","nonce":350790591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6811","nonce":350790591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6815","nonce":350790591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":350790591},{"type":"v1","addr":"172.21.15.131:6813","nonce":350790591}]},"public_addr":"172.21.15.131:6809/350790591","cluster_addr":"172.21.15.131:6811/350790591","heartbeat_back_addr":"172.21.15.131:6815/350790591","heartbeat_front_addr":"172.21.15.131:6813/350790591","state":["exists","up"]},{"osd":4,"uuid":"a092ccfc-be89-4a84-88d5-0f781fa715eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6816","nonce":244560995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6817","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6819","nonce":244560995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6823","nonce":244560995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":244560995},{"type":"v1","addr":"172.21.15.187:6821","nonce":244560995}]},"public_addr":"172.21.15.187:6816/244560995","cluster_addr":"172.21.15.187:6819/244560995","heartbeat_back_addr":"172.21.15.187:6823/244560995","heartbeat_front_addr":"172.21.15.187:6821/244560995","state":["exists","up"]},{"osd":5,"uuid":"7513ba69-dd00-4a35-b190-9e8cc3b2bdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6801","nonce":324619398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6803","nonce":324619398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6812","nonce":324619398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":324619398},{"type":"v1","addr":"172.21.15.187:6807","nonce":324619398}]},"public_addr":"172.21.15.187:6801/324619398","cluster_addr":"172.21.15.187:6803/324619398","heartbeat_back_addr":"172.21.15.187:6812/324619398","heartbeat_front_addr":"172.21.15.187:6807/324619398","state":["exists","up"]},{"osd":6,"uuid":"50081810-c093-4d2f-a4d7-b7c695c5400d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6806","nonce":1391347786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6809","nonce":1391347786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6818","nonce":1391347786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":1391347786},{"type":"v1","addr":"172.21.15.187:6813","nonce":1391347786}]},"public_addr":"172.21.15.187:6806/1391347786","cluster_addr":"172.21.15.187:6809/1391347786","heartbeat_back_addr":"172.21.15.187:6818/1391347786","heartbeat_front_addr":"172.21.15.187:6813/1391347786","state":["exists","up"]},{"osd":7,"uuid":"775dae05-54f5-4ad4-bbce-92863c6ed2ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6825","nonce":1220065622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6827","nonce":1220065622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6831","nonce":1220065622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":1220065622},{"type":"v1","addr":"172.21.15.187:6829","nonce":1220065622}]},"public_addr":"172.21.15.187:6825/1220065622","cluster_addr":"172.21.15.187:6827/1220065622","heartbeat_back_addr":"172.21.15.187:6831/1220065622","heartbeat_front_addr":"172.21.15.187:6829/1220065622","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:05.983520+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.299516+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.067323+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.307748+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.018742+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.258775+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.010912+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-26T22:48:06.295355+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:6837/2725520331":"2024-05-27T22:56:55.783271+0000","172.21.15.131:6835/2725520331":"2024-05-27T22:56:55.783271+0000","172.21.15.187:6835/2594604065":"2024-05-27T22:56:55.783235+0000","172.21.15.131:6836/2710427205":"2024-05-27T22:56:55.783260+0000","172.21.15.131:6834/2710427205":"2024-05-27T22:56:55.783260+0000","172.21.15.187:6834/2594604065":"2024-05-27T22:56:55.783235+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-26T22:57:05.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:05.128+0000 7f65d5000640 1 -- 172.21.15.131:0/2411890062 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f65ac06be30 msgr2=0x7f65ac06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:05.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:05.128+0000 7f65d5000640 1 --2- 172.21.15.131:0/2411890062 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f65ac06be30 0x7f65ac06e2f0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f65c0006e80 tx=0x7f65c0008040 comp rx=0 tx=0).stop 2024-05-26T22:57:05.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:05.128+0000 7f65d5000640 1 -- 172.21.15.131:0/2411890062 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f65d01f7b10 msgr2=0x7f65d01f7f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:05.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:05.128+0000 7f65d5000640 1 --2- 172.21.15.131:0/2411890062 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f65d01f7b10 0x7f65d01f7f90 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f65d00693f0 tx=0x7f65c8007b10 comp rx=0 tx=0).stop 2024-05-26T22:57:05.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:05.129+0000 7f65d5000640 1 -- 172.21.15.131:0/2411890062 shutdown_connections 2024-05-26T22:57:05.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:05.129+0000 7f65d5000640 1 --2- 172.21.15.131:0/2411890062 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f65ac06be30 0x7f65ac06e2f0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:05.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:05.129+0000 7f65d5000640 1 --2- 172.21.15.131:0/2411890062 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f65d01f7b10 0x7f65d01f7f90 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:05.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:05.129+0000 7f65d5000640 1 --2- 172.21.15.131:0/2411890062 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f65d00b7c20 0x7f65d014fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:05.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:05.129+0000 7f65d5000640 1 --2- 172.21.15.131:0/2411890062 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f65b4003f80 0x7f65d0152230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:05.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:05.129+0000 7f65d5000640 1 -- 172.21.15.131:0/2411890062 >> 172.21.15.131:0/2411890062 conn(0x7f65d00b3e00 msgr2=0x7f65d00cbd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:57:05.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:05.129+0000 7f65d5000640 1 -- 172.21.15.131:0/2411890062 shutdown_connections 2024-05-26T22:57:05.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:05.129+0000 7f65d5000640 1 -- 172.21.15.131:0/2411890062 wait complete. 2024-05-26T22:57:06.147 INFO:tasks.ceph:Scrubbing osd.0 2024-05-26T22:57:06.148 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-05-26T22:57:06.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.364+0000 7fae97e00640 1 Processor -- start 2024-05-26T22:57:06.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.364+0000 7fae97e00640 1 -- start start 2024-05-26T22:57:06.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.366+0000 7fae97e00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fae9813dee0 0x7fae9806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:06.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.366+0000 7fae97e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fae9806da20 0x7fae9815b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:06.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.366+0000 7fae97e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fae9815bc00 0x7fae980b1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:06.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.366+0000 7fae97e00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fae9813e2c0 con 0x7fae9815bc00 2024-05-26T22:57:06.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.366+0000 7fae97e00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fae9815c430 con 0x7fae9813dee0 2024-05-26T22:57:06.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.366+0000 7fae97e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fae9815eb60 con 0x7fae9806da20 2024-05-26T22:57:06.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.366+0000 7fae97400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fae9815bc00 0x7fae980b1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:06.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.367+0000 7fae97400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fae9815bc00 0x7fae980b1b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57182/0 (socket says 172.21.15.131:57182) 2024-05-26T22:57:06.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.367+0000 7fae97400640 1 -- 172.21.15.131:0/2302516775 learned_addr learned my addr 172.21.15.131:0/2302516775 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:57:06.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.367+0000 7fae96a00640 1 --2- 172.21.15.131:0/2302516775 >> v2:172.21.15.187:3300/0 conn(0x7fae9813dee0 0x7fae9806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:06.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.367+0000 7fae97400640 1 -- 172.21.15.131:0/2302516775 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fae9806da20 msgr2=0x7fae9815b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:57:06.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.367+0000 7fae96000640 1 --2- 172.21.15.131:0/2302516775 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fae9806da20 0x7fae9815b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:06.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.367+0000 7fae97400640 1 --2- 172.21.15.131:0/2302516775 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fae9806da20 0x7fae9815b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:06.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.367+0000 7fae97400640 1 -- 172.21.15.131:0/2302516775 >> v2:172.21.15.187:3300/0 conn(0x7fae9813dee0 msgr2=0x7fae9806d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:57:06.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.367+0000 7fae97400640 1 --2- 172.21.15.131:0/2302516775 >> v2:172.21.15.187:3300/0 conn(0x7fae9813dee0 0x7fae9806d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:06.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.367+0000 7fae97400640 1 -- 172.21.15.131:0/2302516775 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae9815ee40 con 0x7fae9815bc00 2024-05-26T22:57:06.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.367+0000 7fae96a00640 1 --2- 172.21.15.131:0/2302516775 >> v2:172.21.15.187:3300/0 conn(0x7fae9813dee0 0x7fae9806d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:57:06.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.368+0000 7fae97400640 1 --2- 172.21.15.131:0/2302516775 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fae9815bc00 0x7fae980b1b80 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7fae9806e340 tx=0x7fae9002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c8362b1f97f103 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:57:06.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.368+0000 7fae95600640 1 -- 172.21.15.131:0/2302516775 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae90037070 con 0x7fae9815bc00 2024-05-26T22:57:06.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.368+0000 7fae95600640 1 -- 172.21.15.131:0/2302516775 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae90032440 con 0x7fae9815bc00 2024-05-26T22:57:06.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.368+0000 7fae95600640 1 -- 172.21.15.131:0/2302516775 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae90032810 con 0x7fae9815bc00 2024-05-26T22:57:06.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.369+0000 7fae97e00640 1 -- 172.21.15.131:0/2302516775 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fae9815bc00 msgr2=0x7fae980b1b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:06.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.369+0000 7fae97e00640 1 --2- 172.21.15.131:0/2302516775 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fae9815bc00 0x7fae980b1b80 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7fae9806e340 tx=0x7fae9002a1e0 comp rx=0 tx=0).stop 2024-05-26T22:57:06.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.369+0000 7fae97e00640 1 -- 172.21.15.131:0/2302516775 shutdown_connections 2024-05-26T22:57:06.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.369+0000 7fae97e00640 1 --2- 172.21.15.131:0/2302516775 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fae9815bc00 0x7fae980b1b80 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:06.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.369+0000 7fae97e00640 1 --2- 172.21.15.131:0/2302516775 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fae9806da20 0x7fae9815b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:06.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.369+0000 7fae97e00640 1 --2- 172.21.15.131:0/2302516775 >> v2:172.21.15.187:3300/0 conn(0x7fae9813dee0 0x7fae9806d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:06.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.369+0000 7fae97e00640 1 -- 172.21.15.131:0/2302516775 >> 172.21.15.131:0/2302516775 conn(0x7fae980b5410 msgr2=0x7fae980b57f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:57:06.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.369+0000 7fae97e00640 1 -- 172.21.15.131:0/2302516775 shutdown_connections 2024-05-26T22:57:06.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.369+0000 7fae97e00640 1 -- 172.21.15.131:0/2302516775 wait complete. 2024-05-26T22:57:06.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.371+0000 7fae97e00640 1 Processor -- start 2024-05-26T22:57:06.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.371+0000 7fae97e00640 1 -- start start 2024-05-26T22:57:06.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.372+0000 7fae97e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fae9806da20 0x7fae98151c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:06.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.372+0000 7fae97e00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fae9813dee0 0x7fae980afb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:06.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.372+0000 7fae97e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fae9815bc00 0x7fae98153b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:06.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.372+0000 7fae97e00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fae9815c090 con 0x7fae9815bc00 2024-05-26T22:57:06.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.372+0000 7fae97e00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fae9815f400 con 0x7fae9813dee0 2024-05-26T22:57:06.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.372+0000 7fae97e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fae9815c210 con 0x7fae9806da20 2024-05-26T22:57:06.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.372+0000 7fae97400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fae9815bc00 0x7fae98153b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:06.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.372+0000 7fae97400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fae9815bc00 0x7fae98153b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57190/0 (socket says 172.21.15.131:57190) 2024-05-26T22:57:06.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.372+0000 7fae97400640 1 -- 172.21.15.131:0/2361613324 learned_addr learned my addr 172.21.15.131:0/2361613324 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:57:06.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.373+0000 7fae97400640 1 -- 172.21.15.131:0/2361613324 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fae9806da20 msgr2=0x7fae98151c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:06.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.373+0000 7fae97400640 1 --2- 172.21.15.131:0/2361613324 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fae9806da20 0x7fae98151c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:06.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.373+0000 7fae97400640 1 -- 172.21.15.131:0/2361613324 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fae9813dee0 msgr2=0x7fae980afb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:06.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.373+0000 7fae96000640 1 --2- 172.21.15.131:0/2361613324 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fae9813dee0 0x7fae980afb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:06.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.373+0000 7fae97400640 1 --2- 172.21.15.131:0/2361613324 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fae9813dee0 0x7fae980afb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:06.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.373+0000 7fae97400640 1 -- 172.21.15.131:0/2361613324 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae980b05b0 con 0x7fae9815bc00 2024-05-26T22:57:06.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.373+0000 7fae97400640 1 --2- 172.21.15.131:0/2361613324 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fae9815bc00 0x7fae98153b00 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7fae900090b0 tx=0x7fae900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:57:06.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.373+0000 7fae8ea00640 1 -- 172.21.15.131:0/2361613324 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae90037040 con 0x7fae9815bc00 2024-05-26T22:57:06.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.373+0000 7fae8ea00640 1 -- 172.21.15.131:0/2361613324 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae90042070 con 0x7fae9815bc00 2024-05-26T22:57:06.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.373+0000 7fae8ea00640 1 -- 172.21.15.131:0/2361613324 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae90030bd0 con 0x7fae9815bc00 2024-05-26T22:57:06.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.373+0000 7fae97e00640 1 -- 172.21.15.131:0/2361613324 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae980b0890 con 0x7fae9815bc00 2024-05-26T22:57:06.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.373+0000 7fae97e00640 1 -- 172.21.15.131:0/2361613324 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae980b0e50 con 0x7fae9815bc00 2024-05-26T22:57:06.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.374+0000 7fae97e00640 1 -- 172.21.15.131:0/2361613324 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fae980694e0 con 0x7fae9815bc00 2024-05-26T22:57:06.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.375+0000 7fae8ea00640 1 -- 172.21.15.131:0/2361613324 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fae90047460 con 0x7fae9815bc00 2024-05-26T22:57:06.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.376+0000 7fae8ea00640 1 --2- 172.21.15.131:0/2361613324 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fae7806be30 0x7fae7806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:06.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.376+0000 7fae8ea00640 1 -- 172.21.15.131:0/2361613324 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5115+0+0 (secure 0 0 0) 0x7fae900ab8a0 con 0x7fae9815bc00 2024-05-26T22:57:06.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.377+0000 7fae96a00640 1 --2- 172.21.15.131:0/2361613324 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fae7806be30 0x7fae7806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:06.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.377+0000 7fae8ea00640 1 --2- 172.21.15.131:0/2361613324 >> [v2:172.21.15.131:6800/2350940051,v1:172.21.15.131:6801/2350940051] conn(0x7fae78071e30 0x7fae78074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:06.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.377+0000 7fae8ea00640 1 -- 172.21.15.131:0/2361613324 --> [v2:172.21.15.131:6800/2350940051,v1:172.21.15.131:6801/2350940051] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fae9002be50 con 0x7fae78071e30 2024-05-26T22:57:06.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.377+0000 7fae8ea00640 1 -- 172.21.15.131:0/2361613324 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_get_version_reply(handle=1 version=110) ==== 24+0+0 (secure 0 0 0) 0x7fae900b02c0 con 0x7fae9815bc00 2024-05-26T22:57:06.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.377+0000 7fae96000640 1 --2- 172.21.15.131:0/2361613324 >> [v2:172.21.15.131:6800/2350940051,v1:172.21.15.131:6801/2350940051] conn(0x7fae78071e30 0x7fae78074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:06.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.377+0000 7fae96a00640 1 --2- 172.21.15.131:0/2361613324 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fae7806be30 0x7fae7806e2f0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fae88007e70 tx=0x7fae880073a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:57:06.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.377+0000 7fae96000640 1 --2- 172.21.15.131:0/2361613324 >> [v2:172.21.15.131:6800/2350940051,v1:172.21.15.131:6801/2350940051] conn(0x7fae78071e30 0x7fae78074250 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:57:06.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.378+0000 7fae8ea00640 1 -- 172.21.15.131:0/2361613324 <== osd.0 v2:172.21.15.131:6800/2350940051 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fae9002be50 con 0x7fae78071e30 2024-05-26T22:57:06.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.392+0000 7fae97e00640 1 -- 172.21.15.131:0/2361613324 --> [v2:172.21.15.131:6800/2350940051,v1:172.21.15.131:6801/2350940051] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fae980b4e60 con 0x7fae78071e30 2024-05-26T22:57:06.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.392+0000 7fae8ea00640 1 -- 172.21.15.131:0/2361613324 <== osd.0 v2:172.21.15.131:6800/2350940051 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fae980b4e60 con 0x7fae78071e30 2024-05-26T22:57:06.392 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-26T22:57:06.392 INFO:teuthology.orchestra.run.smithi131.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-26T22:57:06.392 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-26T22:57:06.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.393+0000 7fae97e00640 1 -- 172.21.15.131:0/2361613324 >> [v2:172.21.15.131:6800/2350940051,v1:172.21.15.131:6801/2350940051] conn(0x7fae78071e30 msgr2=0x7fae78074250 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:06.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.393+0000 7fae97e00640 1 --2- 172.21.15.131:0/2361613324 >> [v2:172.21.15.131:6800/2350940051,v1:172.21.15.131:6801/2350940051] conn(0x7fae78071e30 0x7fae78074250 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:06.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.393+0000 7fae97e00640 1 -- 172.21.15.131:0/2361613324 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fae7806be30 msgr2=0x7fae7806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:06.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.393+0000 7fae97e00640 1 --2- 172.21.15.131:0/2361613324 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fae7806be30 0x7fae7806e2f0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fae88007e70 tx=0x7fae880073a0 comp rx=0 tx=0).stop 2024-05-26T22:57:06.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.393+0000 7fae97e00640 1 -- 172.21.15.131:0/2361613324 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fae9815bc00 msgr2=0x7fae98153b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:06.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.393+0000 7fae97e00640 1 --2- 172.21.15.131:0/2361613324 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fae9815bc00 0x7fae98153b00 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7fae900090b0 tx=0x7fae900046c0 comp rx=0 tx=0).stop 2024-05-26T22:57:06.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.393+0000 7fae97400640 1 -- 172.21.15.131:0/2361613324 reap_dead start 2024-05-26T22:57:06.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.393+0000 7fae97e00640 1 -- 172.21.15.131:0/2361613324 shutdown_connections 2024-05-26T22:57:06.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.393+0000 7fae97e00640 1 -- 172.21.15.131:0/2361613324 >> 172.21.15.131:0/2361613324 conn(0x7fae980b5410 msgr2=0x7fae980b57f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:57:06.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.393+0000 7fae97e00640 1 -- 172.21.15.131:0/2361613324 shutdown_connections 2024-05-26T22:57:06.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.393+0000 7fae97e00640 1 -- 172.21.15.131:0/2361613324 wait complete. 2024-05-26T22:57:06.410 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-05-26T22:57:06.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.600+0000 7f33bea00640 1 Processor -- start 2024-05-26T22:57:06.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.600+0000 7f33bea00640 1 -- start start 2024-05-26T22:57:06.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.602+0000 7f33bea00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f33b80b2f60 0x7f33b80ae250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:06.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.602+0000 7f33bea00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33b80b2440 con 0x7f33b815e320 2024-05-26T22:57:06.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.602+0000 7f33bea00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33b8162230 con 0x7f33b80d0550 2024-05-26T22:57:06.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.602+0000 7f33bea00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f33b8161fe0 con 0x7f33b80b2f60 2024-05-26T22:57:06.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.602+0000 7f33bd600640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f33b815e320 0x7f33b80cfe40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:37614/0 (socket says 172.21.15.131:37614) 2024-05-26T22:57:06.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.602+0000 7f33bd600640 1 -- 172.21.15.131:0/1966227873 learned_addr learned my addr 172.21.15.131:0/1966227873 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:57:06.604 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.602+0000 7f33be000640 1 --2- 172.21.15.131:0/1966227873 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f33b80b2f60 0x7f33b80ae250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:06.604 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.603+0000 7f33b7e00640 1 -- 172.21.15.131:0/1966227873 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 496185061 0 0) 0x7f33b80b2440 con 0x7f33b815e320 2024-05-26T22:57:06.604 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.603+0000 7f33b7e00640 1 -- 172.21.15.131:0/1966227873 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f33a0003520 con 0x7f33b815e320 2024-05-26T22:57:06.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.603+0000 7f33b7e00640 1 -- 172.21.15.131:0/1966227873 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1350166617 0 0) 0x7f33a0003520 con 0x7f33b815e320 2024-05-26T22:57:06.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.603+0000 7f33b7e00640 1 -- 172.21.15.131:0/1966227873 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f33b80b2f60 msgr2=0x7f33b80ae250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:57:06.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.603+0000 7f33b7e00640 1 --2- 172.21.15.131:0/1966227873 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f33b80b2f60 0x7f33b80ae250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:06.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.603+0000 7f33b7e00640 1 -- 172.21.15.131:0/1966227873 >> v1:172.21.15.187:6789/0 conn(0x7f33b80d0550 legacy=0x7f33b80715d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:06.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.603+0000 7f33b7e00640 1 -- 172.21.15.131:0/1966227873 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33b8162510 con 0x7f33b815e320 2024-05-26T22:57:06.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.603+0000 7f33b7e00640 1 -- 172.21.15.131:0/1966227873 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7f33ac001670 con 0x7f33b815e320 2024-05-26T22:57:06.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.603+0000 7f33be000640 1 --2- 172.21.15.131:0/1966227873 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f33b80b2f60 0x7f33b80ae250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-26T22:57:06.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.603+0000 7f33b7e00640 1 -- 172.21.15.131:0/1966227873 >> v1:172.21.15.131:6789/0 conn(0x7f33b815e320 legacy=0x7f33b80cfe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:06.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.603+0000 7f33b7e00640 1 --2- 172.21.15.131:0/1966227873 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f33a00039e0 0x7f33a0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:06.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.603+0000 7f33b7e00640 1 --2- 172.21.15.131:0/1966227873 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f33a0004800 0x7f33a0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:06.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.603+0000 7f33b7e00640 1 --2- 172.21.15.131:0/1966227873 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f33b80b2f60 0x7f33a00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:06.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.603+0000 7f33b7e00640 1 -- 172.21.15.131:0/1966227873 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f33b8161fe0 con 0x7f33a00039e0 2024-05-26T22:57:06.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.603+0000 7f33b7e00640 1 -- 172.21.15.131:0/1966227873 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f33a000bb80 con 0x7f33a0004800 2024-05-26T22:57:06.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.603+0000 7f33b7e00640 1 -- 172.21.15.131:0/1966227873 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f33a000bd80 con 0x7f33b80b2f60 2024-05-26T22:57:06.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.603+0000 7f33bd600640 1 --2- 172.21.15.131:0/1966227873 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f33a00039e0 0x7f33a0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:06.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.604+0000 7f33bd600640 1 -- 172.21.15.131:0/1966227873 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f33b80b2f60 msgr2=0x7f33a00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:57:06.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.604+0000 7f33bd600640 1 --2- 172.21.15.131:0/1966227873 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f33b80b2f60 0x7f33a00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:06.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.604+0000 7f33bd600640 1 -- 172.21.15.131:0/1966227873 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f33a0004800 msgr2=0x7f33a0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:57:06.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.604+0000 7f33bd600640 1 --2- 172.21.15.131:0/1966227873 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f33a0004800 0x7f33a0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:06.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.604+0000 7f33bd600640 1 -- 172.21.15.131:0/1966227873 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33a0003520 con 0x7f33a00039e0 2024-05-26T22:57:06.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.605+0000 7f33bd600640 1 --2- 172.21.15.131:0/1966227873 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f33a00039e0 0x7f33a0003e00 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f33ac00c210 tx=0x7f33a0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27479ae57ef6e26e server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:57:06.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.605+0000 7f33b7e00640 1 -- 172.21.15.131:0/1966227873 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33ac007d30 con 0x7f33a00039e0 2024-05-26T22:57:06.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.605+0000 7f33b7e00640 1 -- 172.21.15.131:0/1966227873 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33ac005ad0 con 0x7f33a00039e0 2024-05-26T22:57:06.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.605+0000 7f33b7e00640 1 -- 172.21.15.131:0/1966227873 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33ac007070 con 0x7f33a00039e0 2024-05-26T22:57:06.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.606+0000 7f33bea00640 1 -- 172.21.15.131:0/1966227873 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f33a00039e0 msgr2=0x7f33a0003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:06.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.606+0000 7f33bea00640 1 --2- 172.21.15.131:0/1966227873 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f33a00039e0 0x7f33a0003e00 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f33ac00c210 tx=0x7f33a0005830 comp rx=0 tx=0).stop 2024-05-26T22:57:06.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.606+0000 7f33be000640 1 -- 172.21.15.131:0/1966227873 reap_dead start 2024-05-26T22:57:06.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.606+0000 7f33bea00640 1 -- 172.21.15.131:0/1966227873 shutdown_connections 2024-05-26T22:57:06.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.606+0000 7f33bea00640 1 -- 172.21.15.131:0/1966227873 >> 172.21.15.131:0/1966227873 conn(0x7f33b80b5660 msgr2=0x7f33b80b5a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:57:06.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.606+0000 7f33bea00640 1 -- 172.21.15.131:0/1966227873 shutdown_connections 2024-05-26T22:57:06.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.606+0000 7f33bea00640 1 -- 172.21.15.131:0/1966227873 wait complete. 2024-05-26T22:57:06.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.607+0000 7f33bea00640 1 Processor -- start 2024-05-26T22:57:06.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.608+0000 7f33bea00640 1 -- start start 2024-05-26T22:57:06.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.608+0000 7f33bea00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f33a00039e0 0x7f33b8155410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:06.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.608+0000 7f33bea00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f33b814e9d0 0x7f33b81f7780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:06.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.608+0000 7f33bea00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f33b81f7cc0 0x7f33b81f8170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:06.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.608+0000 7f33bea00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f33b8162e40 con 0x7f33a00039e0 2024-05-26T22:57:06.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.608+0000 7f33bea00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f33b8162ad0 con 0x7f33b814e9d0 2024-05-26T22:57:06.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.608+0000 7f33bea00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f33b80b3570 con 0x7f33b81f7cc0 2024-05-26T22:57:06.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.609+0000 7f33bd600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f33a00039e0 0x7f33b8155410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:06.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.609+0000 7f33bd600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f33a00039e0 0x7f33b8155410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57216/0 (socket says 172.21.15.131:57216) 2024-05-26T22:57:06.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.609+0000 7f33bd600640 1 -- 172.21.15.131:0/1951086324 learned_addr learned my addr 172.21.15.131:0/1951086324 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:57:06.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.609+0000 7f33bd600640 1 -- 172.21.15.131:0/1951086324 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f33b81f7cc0 msgr2=0x7f33b81f8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:06.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.609+0000 7f33bd600640 1 --2- 172.21.15.131:0/1951086324 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f33b81f7cc0 0x7f33b81f8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:06.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.609+0000 7f33bd600640 1 -- 172.21.15.131:0/1951086324 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f33b814e9d0 msgr2=0x7f33b81f7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:06.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.609+0000 7f33bd600640 1 --2- 172.21.15.131:0/1951086324 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f33b814e9d0 0x7f33b81f7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:06.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.609+0000 7f33bd600640 1 -- 172.21.15.131:0/1951086324 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33b815f6c0 con 0x7f33a00039e0 2024-05-26T22:57:06.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.609+0000 7f33bd600640 1 --2- 172.21.15.131:0/1951086324 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f33a00039e0 0x7f33b8155410 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f33ac001970 tx=0x7f33ac001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:57:06.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.610+0000 7f33b5600640 1 -- 172.21.15.131:0/1951086324 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33ac042020 con 0x7f33a00039e0 2024-05-26T22:57:06.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.610+0000 7f33b5600640 1 -- 172.21.15.131:0/1951086324 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33ac004c10 con 0x7f33a00039e0 2024-05-26T22:57:06.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.610+0000 7f33b5600640 1 -- 172.21.15.131:0/1951086324 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33ac007d10 con 0x7f33a00039e0 2024-05-26T22:57:06.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.610+0000 7f33bea00640 1 -- 172.21.15.131:0/1951086324 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33b815f9a0 con 0x7f33a00039e0 2024-05-26T22:57:06.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.610+0000 7f33bea00640 1 -- 172.21.15.131:0/1951086324 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33b8160480 con 0x7f33a00039e0 2024-05-26T22:57:06.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.611+0000 7f33bea00640 1 -- 172.21.15.131:0/1951086324 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33b80694e0 con 0x7f33a00039e0 2024-05-26T22:57:06.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.612+0000 7f33b5600640 1 -- 172.21.15.131:0/1951086324 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f33ac045020 con 0x7f33a00039e0 2024-05-26T22:57:06.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.612+0000 7f33b5600640 1 --2- 172.21.15.131:0/1951086324 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f339806be30 0x7f339806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:06.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.613+0000 7f33b5600640 1 -- 172.21.15.131:0/1951086324 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5115+0+0 (secure 0 0 0) 0x7f33ac0ab150 con 0x7f33a00039e0 2024-05-26T22:57:06.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.613+0000 7f33bcc00640 1 --2- 172.21.15.131:0/1951086324 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f339806be30 0x7f339806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:06.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.613+0000 7f33bcc00640 1 --2- 172.21.15.131:0/1951086324 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f339806be30 0x7f339806e2f0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f33b8150440 tx=0x7f33a4007990 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:57:06.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.617+0000 7f33b5600640 1 -- 172.21.15.131:0/1951086324 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f33ac079d40 con 0x7f33a00039e0 2024-05-26T22:57:06.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.816+0000 7f33bea00640 1 -- 172.21.15.131:0/1951086324 --> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7f33b80cfc20 con 0x7f339806be30 2024-05-26T22:57:06.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.817+0000 7f33b5600640 1 -- 172.21.15.131:0/1951086324 <== mgr.4104 v2:172.21.15.131:6832/3484832947 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f33b80cfc20 con 0x7f339806be30 2024-05-26T22:57:06.817 INFO:teuthology.orchestra.run.smithi131.stderr:instructed osd(s) 0 to deep-scrub 2024-05-26T22:57:06.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.820+0000 7f33bea00640 1 -- 172.21.15.131:0/1951086324 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f339806be30 msgr2=0x7f339806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:06.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.820+0000 7f33bea00640 1 --2- 172.21.15.131:0/1951086324 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f339806be30 0x7f339806e2f0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f33b8150440 tx=0x7f33a4007990 comp rx=0 tx=0).stop 2024-05-26T22:57:06.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.820+0000 7f33bea00640 1 -- 172.21.15.131:0/1951086324 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f33a00039e0 msgr2=0x7f33b8155410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:06.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.820+0000 7f33bea00640 1 --2- 172.21.15.131:0/1951086324 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f33a00039e0 0x7f33b8155410 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f33ac001970 tx=0x7f33ac001a40 comp rx=0 tx=0).stop 2024-05-26T22:57:06.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.820+0000 7f33bea00640 1 -- 172.21.15.131:0/1951086324 shutdown_connections 2024-05-26T22:57:06.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.820+0000 7f33bea00640 1 --2- 172.21.15.131:0/1951086324 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f33b81f7cc0 0x7f33b81f8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:06.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.820+0000 7f33bea00640 1 --2- 172.21.15.131:0/1951086324 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f339806be30 0x7f339806e2f0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:06.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.820+0000 7f33bea00640 1 --2- 172.21.15.131:0/1951086324 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f33b814e9d0 0x7f33b81f7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:06.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.820+0000 7f33bea00640 1 --2- 172.21.15.131:0/1951086324 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f33a00039e0 0x7f33b8155410 unknown :-1 s=CLOSED pgs=1659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:06.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.820+0000 7f33bea00640 1 -- 172.21.15.131:0/1951086324 >> 172.21.15.131:0/1951086324 conn(0x7f33b80b5660 msgr2=0x7f33b80b0750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:57:06.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.820+0000 7f33bea00640 1 -- 172.21.15.131:0/1951086324 shutdown_connections 2024-05-26T22:57:06.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:06.821+0000 7f33bea00640 1 -- 172.21.15.131:0/1951086324 wait complete. 2024-05-26T22:57:06.840 INFO:tasks.ceph:Scrubbing osd.1 2024-05-26T22:57:06.840 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-05-26T22:57:07.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.042+0000 7f6e15600640 1 Processor -- start 2024-05-26T22:57:07.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.042+0000 7f6e15600640 1 -- start start 2024-05-26T22:57:07.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.044+0000 7f6e15600640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f6e10145880 0x7f6e1015cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:07.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.044+0000 7f6e15600640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f6e1015fba0 0x7f6e1006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:07.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.044+0000 7f6e15600640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e1015d4e0 con 0x7f6e10160020 2024-05-26T22:57:07.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.044+0000 7f6e15600640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f6e1015c560 con 0x7f6e10145880 2024-05-26T22:57:07.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.044+0000 7f6e15600640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f6e1015cb10 con 0x7f6e1015fba0 2024-05-26T22:57:07.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.045+0000 7f6e14c00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f6e10160020 0x7f6e100b37f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:37620/0 (socket says 172.21.15.131:37620) 2024-05-26T22:57:07.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.045+0000 7f6e14c00640 1 -- 172.21.15.131:0/3969278891 learned_addr learned my addr 172.21.15.131:0/3969278891 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:57:07.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.045+0000 7f6e0fe00640 1 --2- 172.21.15.131:0/3969278891 >> v2:172.21.15.131:3300/0 conn(0x7f6e10145880 0x7f6e1015cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:07.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.045+0000 7f6e0f400640 1 --2- 172.21.15.131:0/3969278891 >> v2:172.21.15.187:3300/0 conn(0x7f6e1015fba0 0x7f6e1006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:07.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.045+0000 7f6e0fe00640 1 -- 172.21.15.131:0/3969278891 >> v2:172.21.15.187:3300/0 conn(0x7f6e1015fba0 msgr2=0x7f6e1006d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:57:07.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.045+0000 7f6e0fe00640 1 --2- 172.21.15.131:0/3969278891 >> v2:172.21.15.187:3300/0 conn(0x7f6e1015fba0 0x7f6e1006d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:07.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.045+0000 7f6e0fe00640 1 -- 172.21.15.131:0/3969278891 >> v1:172.21.15.131:6789/0 conn(0x7f6e10160020 legacy=0x7f6e100b37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:07.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.045+0000 7f6e0fe00640 1 -- 172.21.15.131:0/3969278891 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e10145c60 con 0x7f6e10145880 2024-05-26T22:57:07.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.045+0000 7f6e0ea00640 1 -- 172.21.15.131:0/3969278891 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3663913676 0 0) 0x7f6e1015d4e0 con 0x7f6e10160020 2024-05-26T22:57:07.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.046+0000 7f6e0fe00640 1 --2- 172.21.15.131:0/3969278891 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e10145880 0x7f6e1015cfa0 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f6e100ce7d0 tx=0x7f6e0002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8617b37ae3e1082 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:57:07.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.047+0000 7f6e0ea00640 1 -- 172.21.15.131:0/3969278891 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e00037070 con 0x7f6e10145880 2024-05-26T22:57:07.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.047+0000 7f6e0ea00640 1 -- 172.21.15.131:0/3969278891 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e00032440 con 0x7f6e10145880 2024-05-26T22:57:07.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.047+0000 7f6e0ea00640 1 -- 172.21.15.131:0/3969278891 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e00032810 con 0x7f6e10145880 2024-05-26T22:57:07.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.047+0000 7f6e15600640 1 -- 172.21.15.131:0/3969278891 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e10145880 msgr2=0x7f6e1015cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:07.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.047+0000 7f6e15600640 1 --2- 172.21.15.131:0/3969278891 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e10145880 0x7f6e1015cfa0 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f6e100ce7d0 tx=0x7f6e0002a1a0 comp rx=0 tx=0).stop 2024-05-26T22:57:07.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.047+0000 7f6e15600640 1 -- 172.21.15.131:0/3969278891 shutdown_connections 2024-05-26T22:57:07.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.047+0000 7f6e15600640 1 --2- 172.21.15.131:0/3969278891 >> v2:172.21.15.187:3300/0 conn(0x7f6e1015fba0 0x7f6e1006d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:07.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.047+0000 7f6e15600640 1 --2- 172.21.15.131:0/3969278891 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e10145880 0x7f6e1015cfa0 unknown :-1 s=CLOSED pgs=1661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:07.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.047+0000 7f6e15600640 1 -- 172.21.15.131:0/3969278891 >> 172.21.15.131:0/3969278891 conn(0x7f6e100b5410 msgr2=0x7f6e100b57f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:57:07.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.047+0000 7f6e15600640 1 -- 172.21.15.131:0/3969278891 shutdown_connections 2024-05-26T22:57:07.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.047+0000 7f6e15600640 1 -- 172.21.15.131:0/3969278891 wait complete. 2024-05-26T22:57:07.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.049+0000 7f6e15600640 1 Processor -- start 2024-05-26T22:57:07.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.049+0000 7f6e15600640 1 -- start start 2024-05-26T22:57:07.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.050+0000 7f6e15600640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6e10145880 0x7f6e10138b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:07.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.050+0000 7f6e15600640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6e1015fba0 0x7f6e10137be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:07.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.050+0000 7f6e15600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e10160020 0x7f6e101516a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:07.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.050+0000 7f6e15600640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f6e1015c7a0 con 0x7f6e10160020 2024-05-26T22:57:07.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.050+0000 7f6e15600640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6e10160890 con 0x7f6e1015fba0 2024-05-26T22:57:07.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.050+0000 7f6e15600640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f6e100b1d10 con 0x7f6e10145880 2024-05-26T22:57:07.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.050+0000 7f6e14c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e10160020 0x7f6e101516a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:07.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.051+0000 7f6e14c00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e10160020 0x7f6e101516a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57228/0 (socket says 172.21.15.131:57228) 2024-05-26T22:57:07.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.051+0000 7f6e14c00640 1 -- 172.21.15.131:0/2204954436 learned_addr learned my addr 172.21.15.131:0/2204954436 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:57:07.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.051+0000 7f6e0f400640 1 --2- 172.21.15.131:0/2204954436 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6e1015fba0 0x7f6e10137be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:07.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.051+0000 7f6e0fe00640 1 --2- 172.21.15.131:0/2204954436 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6e10145880 0x7f6e10138b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:07.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.051+0000 7f6e14c00640 1 -- 172.21.15.131:0/2204954436 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6e10145880 msgr2=0x7f6e10138b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:07.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.051+0000 7f6e14c00640 1 --2- 172.21.15.131:0/2204954436 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6e10145880 0x7f6e10138b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:07.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.051+0000 7f6e14c00640 1 -- 172.21.15.131:0/2204954436 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6e1015fba0 msgr2=0x7f6e10137be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:07.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.051+0000 7f6e14c00640 1 --2- 172.21.15.131:0/2204954436 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6e1015fba0 0x7f6e10137be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:07.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.051+0000 7f6e14c00640 1 -- 172.21.15.131:0/2204954436 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e101f1e10 con 0x7f6e10160020 2024-05-26T22:57:07.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.051+0000 7f6e14c00640 1 --2- 172.21.15.131:0/2204954436 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e10160020 0x7f6e101516a0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f6e0800c7a0 tx=0x7f6e0800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:57:07.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.052+0000 7f6e07e00640 1 -- 172.21.15.131:0/2204954436 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e08018070 con 0x7f6e10160020 2024-05-26T22:57:07.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.052+0000 7f6e07e00640 1 -- 172.21.15.131:0/2204954436 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e0800a640 con 0x7f6e10160020 2024-05-26T22:57:07.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.052+0000 7f6e07e00640 1 -- 172.21.15.131:0/2204954436 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e0800aab0 con 0x7f6e10160020 2024-05-26T22:57:07.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.052+0000 7f6e15600640 1 -- 172.21.15.131:0/2204954436 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e10151be0 con 0x7f6e10160020 2024-05-26T22:57:07.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.052+0000 7f6e15600640 1 -- 172.21.15.131:0/2204954436 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e10139550 con 0x7f6e10160020 2024-05-26T22:57:07.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.053+0000 7f6e15600640 1 -- 172.21.15.131:0/2204954436 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6e100694e0 con 0x7f6e10160020 2024-05-26T22:57:07.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.054+0000 7f6e07e00640 1 -- 172.21.15.131:0/2204954436 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6e0801b070 con 0x7f6e10160020 2024-05-26T22:57:07.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.055+0000 7f6e07e00640 1 --2- 172.21.15.131:0/2204954436 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f6df006be30 0x7f6df006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:07.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.055+0000 7f6e07e00640 1 -- 172.21.15.131:0/2204954436 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5115+0+0 (secure 0 0 0) 0x7f6e0808a2f0 con 0x7f6e10160020 2024-05-26T22:57:07.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.055+0000 7f6e0fe00640 1 --2- 172.21.15.131:0/2204954436 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f6df006be30 0x7f6df006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:07.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.055+0000 7f6e07e00640 1 --2- 172.21.15.131:0/2204954436 >> [v2:172.21.15.131:6824/1525528247,v1:172.21.15.131:6825/1525528247] conn(0x7f6df0071e30 0x7f6df0074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:07.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.055+0000 7f6e07e00640 1 -- 172.21.15.131:0/2204954436 --> [v2:172.21.15.131:6824/1525528247,v1:172.21.15.131:6825/1525528247] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6df0074900 con 0x7f6df0071e30 2024-05-26T22:57:07.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.055+0000 7f6e07e00640 1 -- 172.21.15.131:0/2204954436 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_get_version_reply(handle=1 version=110) ==== 24+0+0 (secure 0 0 0) 0x7f6e08091050 con 0x7f6e10160020 2024-05-26T22:57:07.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.056+0000 7f6e0f400640 1 --2- 172.21.15.131:0/2204954436 >> [v2:172.21.15.131:6824/1525528247,v1:172.21.15.131:6825/1525528247] conn(0x7f6df0071e30 0x7f6df0074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:07.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.056+0000 7f6e0fe00640 1 --2- 172.21.15.131:0/2204954436 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f6df006be30 0x7f6df006e2f0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f6e00034000 tx=0x7f6e0002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:57:07.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.056+0000 7f6e0f400640 1 --2- 172.21.15.131:0/2204954436 >> [v2:172.21.15.131:6824/1525528247,v1:172.21.15.131:6825/1525528247] conn(0x7f6df0071e30 0x7f6df0074250 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:57:07.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.057+0000 7f6e07e00640 1 -- 172.21.15.131:0/2204954436 <== osd.1 v2:172.21.15.131:6824/1525528247 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6df0074900 con 0x7f6df0071e30 2024-05-26T22:57:07.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.071+0000 7f6e15600640 1 -- 172.21.15.131:0/2204954436 --> [v2:172.21.15.131:6824/1525528247,v1:172.21.15.131:6825/1525528247] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f6e101f2230 con 0x7f6df0071e30 2024-05-26T22:57:07.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.072+0000 7f6e07e00640 1 -- 172.21.15.131:0/2204954436 <== osd.1 v2:172.21.15.131:6824/1525528247 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f6e101f2230 con 0x7f6df0071e30 2024-05-26T22:57:07.072 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-26T22:57:07.072 INFO:teuthology.orchestra.run.smithi131.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-26T22:57:07.072 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-26T22:57:07.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.072+0000 7f6e15600640 1 -- 172.21.15.131:0/2204954436 >> [v2:172.21.15.131:6824/1525528247,v1:172.21.15.131:6825/1525528247] conn(0x7f6df0071e30 msgr2=0x7f6df0074250 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:07.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.072+0000 7f6e15600640 1 --2- 172.21.15.131:0/2204954436 >> [v2:172.21.15.131:6824/1525528247,v1:172.21.15.131:6825/1525528247] conn(0x7f6df0071e30 0x7f6df0074250 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:07.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.073+0000 7f6e15600640 1 -- 172.21.15.131:0/2204954436 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f6df006be30 msgr2=0x7f6df006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:07.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.073+0000 7f6e15600640 1 --2- 172.21.15.131:0/2204954436 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f6df006be30 0x7f6df006e2f0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f6e00034000 tx=0x7f6e0002e000 comp rx=0 tx=0).stop 2024-05-26T22:57:07.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.073+0000 7f6e15600640 1 -- 172.21.15.131:0/2204954436 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e10160020 msgr2=0x7f6e101516a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:07.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.073+0000 7f6e15600640 1 --2- 172.21.15.131:0/2204954436 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e10160020 0x7f6e101516a0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f6e0800c7a0 tx=0x7f6e0800cc60 comp rx=0 tx=0).stop 2024-05-26T22:57:07.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.073+0000 7f6e14c00640 1 -- 172.21.15.131:0/2204954436 reap_dead start 2024-05-26T22:57:07.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.073+0000 7f6e15600640 1 -- 172.21.15.131:0/2204954436 shutdown_connections 2024-05-26T22:57:07.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.073+0000 7f6e15600640 1 -- 172.21.15.131:0/2204954436 >> 172.21.15.131:0/2204954436 conn(0x7f6e100b5410 msgr2=0x7f6e100ce180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:57:07.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.073+0000 7f6e15600640 1 -- 172.21.15.131:0/2204954436 shutdown_connections 2024-05-26T22:57:07.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.073+0000 7f6e15600640 1 -- 172.21.15.131:0/2204954436 wait complete. 2024-05-26T22:57:07.091 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-05-26T22:57:07.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.297+0000 7ff20a400640 1 Processor -- start 2024-05-26T22:57:07.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.298+0000 7ff20a400640 1 -- start start 2024-05-26T22:57:07.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.299+0000 7ff20a400640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7ff20415e340 0x7ff2040b72e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:07.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.299+0000 7ff20a400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7ff2040b7820 0x7ff2040d22b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:07.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.299+0000 7ff20a400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff2040d27f0 0x7ff2040b8d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:07.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.299+0000 7ff20a400640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7ff2040b88b0 con 0x7ff2040b7820 2024-05-26T22:57:07.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.299+0000 7ff20a400640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7ff20415e720 con 0x7ff20415e340 2024-05-26T22:57:07.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.299+0000 7ff20a400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff2040b7ca0 con 0x7ff2040d27f0 2024-05-26T22:57:07.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.299+0000 7ff203e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7ff2040b7820 0x7ff2040d22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:07.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.299+0000 7ff203e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7ff2040b7820 0x7ff2040d22b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57238/0 (socket says 172.21.15.131:57238) 2024-05-26T22:57:07.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.299+0000 7ff203e00640 1 -- 172.21.15.131:0/3131551571 learned_addr learned my addr 172.21.15.131:0/3131551571 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:57:07.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.300+0000 7ff209000640 1 --2- 172.21.15.131:0/3131551571 >> v2:172.21.15.187:3300/0 conn(0x7ff20415e340 0x7ff2040b72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:07.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.300+0000 7ff209a00640 1 --2- 172.21.15.131:0/3131551571 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff2040d27f0 0x7ff2040b8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:07.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.300+0000 7ff203e00640 1 -- 172.21.15.131:0/3131551571 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff2040d27f0 msgr2=0x7ff2040b8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:57:07.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.300+0000 7ff203e00640 1 --2- 172.21.15.131:0/3131551571 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff2040d27f0 0x7ff2040b8d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:07.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.300+0000 7ff203e00640 1 -- 172.21.15.131:0/3131551571 >> v2:172.21.15.187:3300/0 conn(0x7ff20415e340 msgr2=0x7ff2040b72e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:57:07.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.300+0000 7ff203e00640 1 --2- 172.21.15.131:0/3131551571 >> v2:172.21.15.187:3300/0 conn(0x7ff20415e340 0x7ff2040b72e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:07.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.300+0000 7ff203e00640 1 -- 172.21.15.131:0/3131551571 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2040b7f80 con 0x7ff2040b7820 2024-05-26T22:57:07.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.300+0000 7ff209a00640 1 --2- 172.21.15.131:0/3131551571 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff2040d27f0 0x7ff2040b8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:57:07.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.300+0000 7ff209000640 1 --2- 172.21.15.131:0/3131551571 >> v2:172.21.15.187:3300/0 conn(0x7ff20415e340 0x7ff2040b72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:57:07.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.301+0000 7ff203e00640 1 --2- 172.21.15.131:0/3131551571 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff2040b7820 0x7ff2040d22b0 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7ff2040b7eb0 tx=0x7ff1ec02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87ea5a73b60996d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:57:07.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.302+0000 7ff203400640 1 -- 172.21.15.131:0/3131551571 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1ec037070 con 0x7ff2040b7820 2024-05-26T22:57:07.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.302+0000 7ff203400640 1 -- 172.21.15.131:0/3131551571 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1ec032440 con 0x7ff2040b7820 2024-05-26T22:57:07.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.302+0000 7ff203400640 1 -- 172.21.15.131:0/3131551571 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1ec032810 con 0x7ff2040b7820 2024-05-26T22:57:07.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.302+0000 7ff20a400640 1 -- 172.21.15.131:0/3131551571 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff2040b7820 msgr2=0x7ff2040d22b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:07.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.302+0000 7ff20a400640 1 --2- 172.21.15.131:0/3131551571 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff2040b7820 0x7ff2040d22b0 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7ff2040b7eb0 tx=0x7ff1ec02a270 comp rx=0 tx=0).stop 2024-05-26T22:57:07.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.302+0000 7ff20a400640 1 -- 172.21.15.131:0/3131551571 shutdown_connections 2024-05-26T22:57:07.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.302+0000 7ff20a400640 1 --2- 172.21.15.131:0/3131551571 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff2040d27f0 0x7ff2040b8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:07.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.302+0000 7ff20a400640 1 --2- 172.21.15.131:0/3131551571 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff2040b7820 0x7ff2040d22b0 unknown :-1 s=CLOSED pgs=1663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:07.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.302+0000 7ff20a400640 1 --2- 172.21.15.131:0/3131551571 >> v2:172.21.15.187:3300/0 conn(0x7ff20415e340 0x7ff2040b72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:07.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.302+0000 7ff20a400640 1 -- 172.21.15.131:0/3131551571 >> 172.21.15.131:0/3131551571 conn(0x7ff204072e60 msgr2=0x7ff204073270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:57:07.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.302+0000 7ff20a400640 1 -- 172.21.15.131:0/3131551571 shutdown_connections 2024-05-26T22:57:07.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.303+0000 7ff20a400640 1 -- 172.21.15.131:0/3131551571 wait complete. 2024-05-26T22:57:07.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.304+0000 7ff20a400640 1 Processor -- start 2024-05-26T22:57:07.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.304+0000 7ff20a400640 1 -- start start 2024-05-26T22:57:07.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.305+0000 7ff20a400640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff2040b7820 0x7ff204153e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:07.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.305+0000 7ff20a400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff2040d27f0 0x7ff20414fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:07.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.305+0000 7ff20a400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff20415e340 0x7ff2041efb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:07.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.305+0000 7ff20a400640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff2040b9f20 con 0x7ff20415e340 2024-05-26T22:57:07.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.305+0000 7ff20a400640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff2040b8540 con 0x7ff2040b7820 2024-05-26T22:57:07.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.305+0000 7ff20a400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff2040ba0a0 con 0x7ff2040d27f0 2024-05-26T22:57:07.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.305+0000 7ff209a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff20415e340 0x7ff2041efb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:07.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.306+0000 7ff209a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff20415e340 0x7ff2041efb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57242/0 (socket says 172.21.15.131:57242) 2024-05-26T22:57:07.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.306+0000 7ff209a00640 1 -- 172.21.15.131:0/471940132 learned_addr learned my addr 172.21.15.131:0/471940132 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:57:07.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.306+0000 7ff203e00640 1 --2- 172.21.15.131:0/471940132 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff2040d27f0 0x7ff20414fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:07.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.306+0000 7ff209a00640 1 -- 172.21.15.131:0/471940132 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff2040d27f0 msgr2=0x7ff20414fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:07.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.306+0000 7ff209000640 1 --2- 172.21.15.131:0/471940132 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff2040b7820 0x7ff204153e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:07.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.306+0000 7ff209a00640 1 --2- 172.21.15.131:0/471940132 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff2040d27f0 0x7ff20414fa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:07.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.306+0000 7ff209a00640 1 -- 172.21.15.131:0/471940132 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff2040b7820 msgr2=0x7ff204153e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:07.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.306+0000 7ff209a00640 1 --2- 172.21.15.131:0/471940132 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff2040b7820 0x7ff204153e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:07.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.306+0000 7ff209a00640 1 -- 172.21.15.131:0/471940132 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff204154380 con 0x7ff20415e340 2024-05-26T22:57:07.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.306+0000 7ff209000640 1 --2- 172.21.15.131:0/471940132 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff2040b7820 0x7ff204153e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:57:07.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.306+0000 7ff209a00640 1 --2- 172.21.15.131:0/471940132 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff20415e340 0x7ff2041efb20 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc00ce50 tx=0x7ff1fc00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:57:07.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.307+0000 7ff200c00640 1 -- 172.21.15.131:0/471940132 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1fc003b90 con 0x7ff20415e340 2024-05-26T22:57:07.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.307+0000 7ff200c00640 1 -- 172.21.15.131:0/471940132 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1fc003d50 con 0x7ff20415e340 2024-05-26T22:57:07.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.307+0000 7ff20a400640 1 -- 172.21.15.131:0/471940132 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff204155f60 con 0x7ff20415e340 2024-05-26T22:57:07.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.307+0000 7ff200c00640 1 -- 172.21.15.131:0/471940132 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1fc004260 con 0x7ff20415e340 2024-05-26T22:57:07.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.307+0000 7ff20a400640 1 -- 172.21.15.131:0/471940132 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2041564e0 con 0x7ff20415e340 2024-05-26T22:57:07.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.308+0000 7ff20a400640 1 -- 172.21.15.131:0/471940132 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2040694e0 con 0x7ff20415e340 2024-05-26T22:57:07.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.309+0000 7ff200c00640 1 -- 172.21.15.131:0/471940132 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff1fc004400 con 0x7ff20415e340 2024-05-26T22:57:07.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.310+0000 7ff200c00640 1 --2- 172.21.15.131:0/471940132 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ff1e006be30 0x7ff1e006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:07.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.310+0000 7ff200c00640 1 -- 172.21.15.131:0/471940132 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5115+0+0 (secure 0 0 0) 0x7ff1fc08b650 con 0x7ff20415e340 2024-05-26T22:57:07.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.310+0000 7ff209000640 1 --2- 172.21.15.131:0/471940132 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ff1e006be30 0x7ff1e006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:07.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.311+0000 7ff209000640 1 --2- 172.21.15.131:0/471940132 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ff1e006be30 0x7ff1e006e2f0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7ff1f4006fd0 tx=0x7ff1f4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:57:07.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.315+0000 7ff200c00640 1 -- 172.21.15.131:0/471940132 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7ff1fc05a1b0 con 0x7ff20415e340 2024-05-26T22:57:07.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.515+0000 7ff20a400640 1 -- 172.21.15.131:0/471940132 --> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7ff204160d90 con 0x7ff1e006be30 2024-05-26T22:57:07.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.516+0000 7ff200c00640 1 -- 172.21.15.131:0/471940132 <== mgr.4104 v2:172.21.15.131:6832/3484832947 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7ff204160d90 con 0x7ff1e006be30 2024-05-26T22:57:07.515 INFO:teuthology.orchestra.run.smithi131.stderr:instructed osd(s) 1 to deep-scrub 2024-05-26T22:57:07.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.518+0000 7ff20a400640 1 -- 172.21.15.131:0/471940132 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ff1e006be30 msgr2=0x7ff1e006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:07.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.518+0000 7ff20a400640 1 --2- 172.21.15.131:0/471940132 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ff1e006be30 0x7ff1e006e2f0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7ff1f4006fd0 tx=0x7ff1f4008040 comp rx=0 tx=0).stop 2024-05-26T22:57:07.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.518+0000 7ff20a400640 1 -- 172.21.15.131:0/471940132 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff20415e340 msgr2=0x7ff2041efb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:07.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.518+0000 7ff20a400640 1 --2- 172.21.15.131:0/471940132 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff20415e340 0x7ff2041efb20 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc00ce50 tx=0x7ff1fc00a710 comp rx=0 tx=0).stop 2024-05-26T22:57:07.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.519+0000 7ff20a400640 1 -- 172.21.15.131:0/471940132 shutdown_connections 2024-05-26T22:57:07.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.519+0000 7ff20a400640 1 --2- 172.21.15.131:0/471940132 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff2040d27f0 0x7ff20414fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:07.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.519+0000 7ff20a400640 1 --2- 172.21.15.131:0/471940132 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ff1e006be30 0x7ff1e006e2f0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:07.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.519+0000 7ff20a400640 1 --2- 172.21.15.131:0/471940132 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff20415e340 0x7ff2041efb20 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:07.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.519+0000 7ff20a400640 1 --2- 172.21.15.131:0/471940132 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff2040b7820 0x7ff204153e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:07.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.519+0000 7ff20a400640 1 -- 172.21.15.131:0/471940132 >> 172.21.15.131:0/471940132 conn(0x7ff204072e60 msgr2=0x7ff2041612b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:57:07.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.519+0000 7ff20a400640 1 -- 172.21.15.131:0/471940132 shutdown_connections 2024-05-26T22:57:07.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.519+0000 7ff20a400640 1 -- 172.21.15.131:0/471940132 wait complete. 2024-05-26T22:57:07.537 INFO:tasks.ceph:Scrubbing osd.2 2024-05-26T22:57:07.537 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-05-26T22:57:07.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.743+0000 7fd201000640 1 Processor -- start 2024-05-26T22:57:07.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.743+0000 7fd201000640 1 -- start start 2024-05-26T22:57:07.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.745+0000 7fd201000640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fd1fc0d3dc0 0x7fd1fc0b9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:07.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.745+0000 7fd201000640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1fc0d2d20 con 0x7fd1fc15e4b0 2024-05-26T22:57:07.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.745+0000 7fd201000640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1fc0d3000 con 0x7fd1fc0d1370 2024-05-26T22:57:07.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.745+0000 7fd201000640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fd1fc0d32e0 con 0x7fd1fc0d3dc0 2024-05-26T22:57:07.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.745+0000 7fd1fb400640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fd1fc15e4b0 0x7fd1fc0d0c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:37628/0 (socket says 172.21.15.131:37628) 2024-05-26T22:57:07.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.745+0000 7fd1fb400640 1 -- 172.21.15.131:0/3263964934 learned_addr learned my addr 172.21.15.131:0/3263964934 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:57:07.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.745+0000 7fd1fbe00640 1 --2- 172.21.15.131:0/3263964934 >> v2:172.21.15.131:3300/0 conn(0x7fd1fc0d3dc0 0x7fd1fc0b9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:07.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.746+0000 7fd1fa000640 1 -- 172.21.15.131:0/3263964934 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 806340749 0 0) 0x7fd1fc0d2d20 con 0x7fd1fc15e4b0 2024-05-26T22:57:07.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.746+0000 7fd1fa000640 1 -- 172.21.15.131:0/3263964934 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd1e0003520 con 0x7fd1fc15e4b0 2024-05-26T22:57:07.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.746+0000 7fd1fbe00640 1 -- 172.21.15.131:0/3263964934 >> v1:172.21.15.187:6789/0 conn(0x7fd1fc0d1370 legacy=0x7fd1fc0b7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:07.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.746+0000 7fd1fbe00640 1 -- 172.21.15.131:0/3263964934 >> v1:172.21.15.131:6789/0 conn(0x7fd1fc15e4b0 legacy=0x7fd1fc0d0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:07.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.746+0000 7fd1fbe00640 1 -- 172.21.15.131:0/3263964934 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1fc0d35c0 con 0x7fd1fc0d3dc0 2024-05-26T22:57:07.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.747+0000 7fd1fbe00640 1 --2- 172.21.15.131:0/3263964934 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd1fc0d3dc0 0x7fd1fc0b9d90 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7fd1f4009d90 tx=0x7fd1f402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca4a8c939f7f7d20 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:57:07.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.747+0000 7fd1fa000640 1 -- 172.21.15.131:0/3263964934 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1f4007740 con 0x7fd1fc0d3dc0 2024-05-26T22:57:07.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.747+0000 7fd1fa000640 1 -- 172.21.15.131:0/3263964934 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1f40078e0 con 0x7fd1fc0d3dc0 2024-05-26T22:57:07.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.747+0000 7fd1fa000640 1 -- 172.21.15.131:0/3263964934 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1f4007cb0 con 0x7fd1fc0d3dc0 2024-05-26T22:57:07.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.748+0000 7fd201000640 1 -- 172.21.15.131:0/3263964934 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd1fc0d3dc0 msgr2=0x7fd1fc0b9d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:07.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.748+0000 7fd201000640 1 --2- 172.21.15.131:0/3263964934 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd1fc0d3dc0 0x7fd1fc0b9d90 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7fd1f4009d90 tx=0x7fd1f402a6f0 comp rx=0 tx=0).stop 2024-05-26T22:57:07.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.748+0000 7fd201000640 1 -- 172.21.15.131:0/3263964934 shutdown_connections 2024-05-26T22:57:07.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.748+0000 7fd201000640 1 --2- 172.21.15.131:0/3263964934 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd1fc0d3dc0 0x7fd1fc0b9d90 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:07.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.748+0000 7fd201000640 1 -- 172.21.15.131:0/3263964934 >> 172.21.15.131:0/3263964934 conn(0x7fd1fc073070 msgr2=0x7fd1fc06c2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:57:07.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.748+0000 7fd201000640 1 -- 172.21.15.131:0/3263964934 shutdown_connections 2024-05-26T22:57:07.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.748+0000 7fd201000640 1 -- 172.21.15.131:0/3263964934 wait complete. 2024-05-26T22:57:07.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.750+0000 7fd201000640 1 Processor -- start 2024-05-26T22:57:07.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.750+0000 7fd201000640 1 -- start start 2024-05-26T22:57:07.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.751+0000 7fd201000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd1fc0d1370 0x7fd1fc146300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:07.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.751+0000 7fd201000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd1fc0d3dc0 0x7fd1fc15c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:07.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.751+0000 7fd201000640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd1fc15e4b0 0x7fd1fc1f6b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:07.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.751+0000 7fd201000640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fd1fc0d29c0 con 0x7fd1fc0d1370 2024-05-26T22:57:07.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.751+0000 7fd201000640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd1fc0d2840 con 0x7fd1fc15e4b0 2024-05-26T22:57:07.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.751+0000 7fd201000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd1fc0b7a90 con 0x7fd1fc0d3dc0 2024-05-26T22:57:07.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.751+0000 7fd1fb400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd1fc0d1370 0x7fd1fc146300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:07.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.751+0000 7fd1fb400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd1fc0d1370 0x7fd1fc146300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57266/0 (socket says 172.21.15.131:57266) 2024-05-26T22:57:07.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.751+0000 7fd1fb400640 1 -- 172.21.15.131:0/2837994993 learned_addr learned my addr 172.21.15.131:0/2837994993 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:57:07.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.752+0000 7fd1fb400640 1 -- 172.21.15.131:0/2837994993 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd1fc0d3dc0 msgr2=0x7fd1fc15c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:07.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.752+0000 7fd1faa00640 1 --2- 172.21.15.131:0/2837994993 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd1fc0d3dc0 0x7fd1fc15c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:07.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.752+0000 7fd1fbe00640 1 --2- 172.21.15.131:0/2837994993 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd1fc15e4b0 0x7fd1fc1f6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:07.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.752+0000 7fd1fb400640 1 --2- 172.21.15.131:0/2837994993 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd1fc0d3dc0 0x7fd1fc15c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:07.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.752+0000 7fd1fb400640 1 -- 172.21.15.131:0/2837994993 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd1fc15e4b0 msgr2=0x7fd1fc1f6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:07.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.752+0000 7fd1fb400640 1 --2- 172.21.15.131:0/2837994993 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd1fc15e4b0 0x7fd1fc1f6b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:07.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.752+0000 7fd1fb400640 1 -- 172.21.15.131:0/2837994993 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1fc146950 con 0x7fd1fc0d1370 2024-05-26T22:57:07.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.752+0000 7fd1faa00640 1 --2- 172.21.15.131:0/2837994993 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd1fc0d3dc0 0x7fd1fc15c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:57:07.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.752+0000 7fd1fb400640 1 --2- 172.21.15.131:0/2837994993 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd1fc0d1370 0x7fd1fc146300 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec00c440 tx=0x7fd1ec00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:57:07.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.752+0000 7fd1f3400640 1 -- 172.21.15.131:0/2837994993 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1ec017070 con 0x7fd1fc0d1370 2024-05-26T22:57:07.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.752+0000 7fd1f3400640 1 -- 172.21.15.131:0/2837994993 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1ec005ce0 con 0x7fd1fc0d1370 2024-05-26T22:57:07.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.752+0000 7fd1f3400640 1 -- 172.21.15.131:0/2837994993 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1ec0072b0 con 0x7fd1fc0d1370 2024-05-26T22:57:07.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.752+0000 7fd201000640 1 -- 172.21.15.131:0/2837994993 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1fc1618a0 con 0x7fd1fc0d1370 2024-05-26T22:57:07.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.752+0000 7fd201000640 1 -- 172.21.15.131:0/2837994993 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1fc15eb70 con 0x7fd1fc0d1370 2024-05-26T22:57:07.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.753+0000 7fd201000640 1 -- 172.21.15.131:0/2837994993 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd1fc0694e0 con 0x7fd1fc0d1370 2024-05-26T22:57:07.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.754+0000 7fd1f3400640 1 -- 172.21.15.131:0/2837994993 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd1ec01a070 con 0x7fd1fc0d1370 2024-05-26T22:57:07.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.755+0000 7fd1f3400640 1 --2- 172.21.15.131:0/2837994993 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fd1d806be60 0x7fd1d806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:07.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.755+0000 7fd1f3400640 1 -- 172.21.15.131:0/2837994993 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5115+0+0 (secure 0 0 0) 0x7fd1ec089f50 con 0x7fd1fc0d1370 2024-05-26T22:57:07.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.756+0000 7fd1f3400640 1 --2- 172.21.15.131:0/2837994993 >> [v2:172.21.15.131:6816/3627842210,v1:172.21.15.131:6817/3627842210] conn(0x7fd1d8071e60 0x7fd1d8074280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:07.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.756+0000 7fd1f3400640 1 -- 172.21.15.131:0/2837994993 --> [v2:172.21.15.131:6816/3627842210,v1:172.21.15.131:6817/3627842210] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd1ec014ec0 con 0x7fd1d8071e60 2024-05-26T22:57:07.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.756+0000 7fd1f3400640 1 -- 172.21.15.131:0/2837994993 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_get_version_reply(handle=1 version=110) ==== 24+0+0 (secure 0 0 0) 0x7fd1ec08c5f0 con 0x7fd1fc0d1370 2024-05-26T22:57:07.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.756+0000 7fd1faa00640 1 --2- 172.21.15.131:0/2837994993 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fd1d806be60 0x7fd1d806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:07.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.756+0000 7fd1fbe00640 1 --2- 172.21.15.131:0/2837994993 >> [v2:172.21.15.131:6816/3627842210,v1:172.21.15.131:6817/3627842210] conn(0x7fd1d8071e60 0x7fd1d8074280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:07.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.756+0000 7fd1faa00640 1 --2- 172.21.15.131:0/2837994993 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fd1d806be60 0x7fd1d806e320 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc0b3c60 tx=0x7fd1e4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:57:07.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.757+0000 7fd1fbe00640 1 --2- 172.21.15.131:0/2837994993 >> [v2:172.21.15.131:6816/3627842210,v1:172.21.15.131:6817/3627842210] conn(0x7fd1d8071e60 0x7fd1d8074280 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:57:07.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.758+0000 7fd1f3400640 1 -- 172.21.15.131:0/2837994993 <== osd.2 v2:172.21.15.131:6816/3627842210 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd1ec014ec0 con 0x7fd1d8071e60 2024-05-26T22:57:07.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.772+0000 7fd201000640 1 -- 172.21.15.131:0/2837994993 --> [v2:172.21.15.131:6816/3627842210,v1:172.21.15.131:6817/3627842210] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fd1fc0d20c0 con 0x7fd1d8071e60 2024-05-26T22:57:07.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.772+0000 7fd1f3400640 1 -- 172.21.15.131:0/2837994993 <== osd.2 v2:172.21.15.131:6816/3627842210 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fd1fc0d20c0 con 0x7fd1d8071e60 2024-05-26T22:57:07.772 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-26T22:57:07.772 INFO:teuthology.orchestra.run.smithi131.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-26T22:57:07.773 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-26T22:57:07.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.773+0000 7fd201000640 1 -- 172.21.15.131:0/2837994993 >> [v2:172.21.15.131:6816/3627842210,v1:172.21.15.131:6817/3627842210] conn(0x7fd1d8071e60 msgr2=0x7fd1d8074280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:07.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.773+0000 7fd201000640 1 --2- 172.21.15.131:0/2837994993 >> [v2:172.21.15.131:6816/3627842210,v1:172.21.15.131:6817/3627842210] conn(0x7fd1d8071e60 0x7fd1d8074280 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:07.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.773+0000 7fd201000640 1 -- 172.21.15.131:0/2837994993 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fd1d806be60 msgr2=0x7fd1d806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:07.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.773+0000 7fd201000640 1 --2- 172.21.15.131:0/2837994993 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fd1d806be60 0x7fd1d806e320 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc0b3c60 tx=0x7fd1e4009040 comp rx=0 tx=0).stop 2024-05-26T22:57:07.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.773+0000 7fd201000640 1 -- 172.21.15.131:0/2837994993 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd1fc0d1370 msgr2=0x7fd1fc146300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:07.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.773+0000 7fd201000640 1 --2- 172.21.15.131:0/2837994993 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd1fc0d1370 0x7fd1fc146300 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec00c440 tx=0x7fd1ec00c900 comp rx=0 tx=0).stop 2024-05-26T22:57:07.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.773+0000 7fd1fbe00640 1 -- 172.21.15.131:0/2837994993 reap_dead start 2024-05-26T22:57:07.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.773+0000 7fd201000640 1 -- 172.21.15.131:0/2837994993 shutdown_connections 2024-05-26T22:57:07.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.773+0000 7fd201000640 1 -- 172.21.15.131:0/2837994993 >> 172.21.15.131:0/2837994993 conn(0x7fd1fc073070 msgr2=0x7fd1fc0c8020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:57:07.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.774+0000 7fd201000640 1 -- 172.21.15.131:0/2837994993 shutdown_connections 2024-05-26T22:57:07.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.774+0000 7fd201000640 1 -- 172.21.15.131:0/2837994993 wait complete. 2024-05-26T22:57:07.792 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-05-26T22:57:07.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.992+0000 7f036da00640 1 Processor -- start 2024-05-26T22:57:07.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.992+0000 7f036da00640 1 -- start start 2024-05-26T22:57:07.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.994+0000 7f036da00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f036806f420 0x7f03680cfd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:07.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.994+0000 7f036da00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f03680cf590 0x7f03680d0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:07.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.994+0000 7f036da00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0368160180 con 0x7f03680b5450 2024-05-26T22:57:07.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.994+0000 7f036da00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f0368160d90 con 0x7f03680cf590 2024-05-26T22:57:07.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.994+0000 7f036da00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f0368160460 con 0x7f036806f420 2024-05-26T22:57:07.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.994+0000 7f0367400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f03680cf590 0x7f03680d0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:07.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.994+0000 7f0367400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f03680cf590 0x7f03680d0290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57268/0 (socket says 172.21.15.131:57268) 2024-05-26T22:57:07.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.994+0000 7f0367400640 1 -- 172.21.15.131:0/3517662299 learned_addr learned my addr 172.21.15.131:0/3517662299 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:57:07.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.995+0000 7f0367400640 1 -- 172.21.15.131:0/3517662299 >> v2:172.21.15.187:3300/0 conn(0x7f036806f420 msgr2=0x7f03680cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:57:07.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.995+0000 7f0367400640 1 --2- 172.21.15.131:0/3517662299 >> v2:172.21.15.187:3300/0 conn(0x7f036806f420 0x7f03680cfd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:07.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.995+0000 7f0367400640 1 -- 172.21.15.131:0/3517662299 >> v1:172.21.15.187:6789/0 conn(0x7f03680b5450 legacy=0x7f03680b43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:57:07.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.995+0000 7f0367400640 1 -- 172.21.15.131:0/3517662299 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0368160740 con 0x7f03680cf590 2024-05-26T22:57:07.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.996+0000 7f0367400640 1 --2- 172.21.15.131:0/3517662299 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03680cf590 0x7f03680d0290 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f03680b35b0 tx=0x7f036002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2647ac333dd68a4e server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:57:07.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.996+0000 7f0365600640 1 -- 172.21.15.131:0/3517662299 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0360036070 con 0x7f03680cf590 2024-05-26T22:57:07.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.996+0000 7f0365600640 1 -- 172.21.15.131:0/3517662299 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0360031600 con 0x7f03680cf590 2024-05-26T22:57:07.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.996+0000 7f0365600640 1 -- 172.21.15.131:0/3517662299 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03600319d0 con 0x7f03680cf590 2024-05-26T22:57:07.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.997+0000 7f036da00640 1 -- 172.21.15.131:0/3517662299 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03680cf590 msgr2=0x7f03680d0290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:07.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.997+0000 7f036da00640 1 --2- 172.21.15.131:0/3517662299 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03680cf590 0x7f03680d0290 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f03680b35b0 tx=0x7f036002a6f0 comp rx=0 tx=0).stop 2024-05-26T22:57:07.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.997+0000 7f036da00640 1 -- 172.21.15.131:0/3517662299 shutdown_connections 2024-05-26T22:57:07.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.997+0000 7f036da00640 1 --2- 172.21.15.131:0/3517662299 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03680cf590 0x7f03680d0290 unknown :-1 s=CLOSED pgs=1668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:07.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.997+0000 7f036da00640 1 --2- 172.21.15.131:0/3517662299 >> v2:172.21.15.187:3300/0 conn(0x7f036806f420 0x7f03680cfd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:07.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.997+0000 7f036da00640 1 -- 172.21.15.131:0/3517662299 >> 172.21.15.131:0/3517662299 conn(0x7f036806bc90 msgr2=0x7f03680b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:57:07.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.997+0000 7f036da00640 1 -- 172.21.15.131:0/3517662299 shutdown_connections 2024-05-26T22:57:07.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.997+0000 7f036da00640 1 -- 172.21.15.131:0/3517662299 wait complete. 2024-05-26T22:57:07.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.999+0000 7f036da00640 1 Processor -- start 2024-05-26T22:57:07.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:07.999+0000 7f036da00640 1 -- start start 2024-05-26T22:57:08.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.000+0000 7f036da00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f036806f420 0x7f03681499e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:08.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.000+0000 7f036da00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f03680b5450 0x7f0368158a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:08.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.000+0000 7f036da00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03680cf590 0x7f036809b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:08.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.000+0000 7f036da00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f036815fde0 con 0x7f03680cf590 2024-05-26T22:57:08.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.000+0000 7f036da00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f03680b5830 con 0x7f036806f420 2024-05-26T22:57:08.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.000+0000 7f036da00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f036815ff60 con 0x7f03680b5450 2024-05-26T22:57:08.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.000+0000 7f0367400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03680cf590 0x7f036809b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:08.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.000+0000 7f0367400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03680cf590 0x7f036809b7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57278/0 (socket says 172.21.15.131:57278) 2024-05-26T22:57:08.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.000+0000 7f0367400640 1 -- 172.21.15.131:0/1593330749 learned_addr learned my addr 172.21.15.131:0/1593330749 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:57:08.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.000+0000 7f0367400640 1 -- 172.21.15.131:0/1593330749 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f03680b5450 msgr2=0x7f0368158a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:08.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.000+0000 7f0367400640 1 --2- 172.21.15.131:0/1593330749 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f03680b5450 0x7f0368158a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:08.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.000+0000 7f0367400640 1 -- 172.21.15.131:0/1593330749 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f036806f420 msgr2=0x7f03681499e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:08.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.000+0000 7f0367400640 1 --2- 172.21.15.131:0/1593330749 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f036806f420 0x7f03681499e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:08.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.000+0000 7f0367400640 1 -- 172.21.15.131:0/1593330749 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0368159270 con 0x7f03680cf590 2024-05-26T22:57:08.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.001+0000 7f0367400640 1 --2- 172.21.15.131:0/1593330749 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03680cf590 0x7f036809b7d0 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f0360034040 tx=0x7f0360004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:57:08.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.001+0000 7f035ea00640 1 -- 172.21.15.131:0/1593330749 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0360041020 con 0x7f03680cf590 2024-05-26T22:57:08.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.001+0000 7f035ea00640 1 -- 172.21.15.131:0/1593330749 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0360007780 con 0x7f03680cf590 2024-05-26T22:57:08.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.001+0000 7f035ea00640 1 -- 172.21.15.131:0/1593330749 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0360036040 con 0x7f03680cf590 2024-05-26T22:57:08.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.001+0000 7f036da00640 1 -- 172.21.15.131:0/1593330749 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0368159550 con 0x7f03680cf590 2024-05-26T22:57:08.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.001+0000 7f036da00640 1 -- 172.21.15.131:0/1593330749 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f036815e010 con 0x7f03680cf590 2024-05-26T22:57:08.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.003+0000 7f036da00640 1 -- 172.21.15.131:0/1593330749 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0368074e90 con 0x7f03680cf590 2024-05-26T22:57:08.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.004+0000 7f035ea00640 1 -- 172.21.15.131:0/1593330749 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0360044020 con 0x7f03680cf590 2024-05-26T22:57:08.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.004+0000 7f035ea00640 1 --2- 172.21.15.131:0/1593330749 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f034806be30 0x7f034806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:08.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.005+0000 7f035ea00640 1 -- 172.21.15.131:0/1593330749 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5115+0+0 (secure 0 0 0) 0x7f03600ab9e0 con 0x7f03680cf590 2024-05-26T22:57:08.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.005+0000 7f0366a00640 1 --2- 172.21.15.131:0/1593330749 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f034806be30 0x7f034806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:08.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.005+0000 7f0366a00640 1 --2- 172.21.15.131:0/1593330749 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f034806be30 0x7f034806e2f0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f03680b1b70 tx=0x7f0358007990 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:57:08.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.009+0000 7f035ea00640 1 -- 172.21.15.131:0/1593330749 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f036004b030 con 0x7f03680cf590 2024-05-26T22:57:08.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.207+0000 7f036da00640 1 -- 172.21.15.131:0/1593330749 --> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7f03680cee90 con 0x7f034806be30 2024-05-26T22:57:08.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.208+0000 7f035ea00640 1 -- 172.21.15.131:0/1593330749 <== mgr.4104 v2:172.21.15.131:6832/3484832947 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f03680cee90 con 0x7f034806be30 2024-05-26T22:57:08.208 INFO:teuthology.orchestra.run.smithi131.stderr:instructed osd(s) 2 to deep-scrub 2024-05-26T22:57:08.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.211+0000 7f036da00640 1 -- 172.21.15.131:0/1593330749 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f034806be30 msgr2=0x7f034806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:08.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.211+0000 7f036da00640 1 --2- 172.21.15.131:0/1593330749 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f034806be30 0x7f034806e2f0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f03680b1b70 tx=0x7f0358007990 comp rx=0 tx=0).stop 2024-05-26T22:57:08.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.211+0000 7f036da00640 1 -- 172.21.15.131:0/1593330749 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03680cf590 msgr2=0x7f036809b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:08.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.211+0000 7f036da00640 1 --2- 172.21.15.131:0/1593330749 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03680cf590 0x7f036809b7d0 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f0360034040 tx=0x7f0360004a40 comp rx=0 tx=0).stop 2024-05-26T22:57:08.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.211+0000 7f036da00640 1 -- 172.21.15.131:0/1593330749 shutdown_connections 2024-05-26T22:57:08.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.211+0000 7f036da00640 1 --2- 172.21.15.131:0/1593330749 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f03680b5450 0x7f0368158a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:08.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.211+0000 7f036da00640 1 --2- 172.21.15.131:0/1593330749 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f034806be30 0x7f034806e2f0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:08.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.211+0000 7f036da00640 1 --2- 172.21.15.131:0/1593330749 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03680cf590 0x7f036809b7d0 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:08.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.211+0000 7f036da00640 1 --2- 172.21.15.131:0/1593330749 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f036806f420 0x7f03681499e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:08.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.211+0000 7f036da00640 1 -- 172.21.15.131:0/1593330749 >> 172.21.15.131:0/1593330749 conn(0x7f036806bc90 msgr2=0x7f03680cd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:57:08.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.211+0000 7f036da00640 1 -- 172.21.15.131:0/1593330749 shutdown_connections 2024-05-26T22:57:08.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.211+0000 7f036da00640 1 -- 172.21.15.131:0/1593330749 wait complete. 2024-05-26T22:57:08.230 INFO:tasks.ceph:Scrubbing osd.3 2024-05-26T22:57:08.231 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-05-26T22:57:08.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.437+0000 7f4a93a00640 1 Processor -- start 2024-05-26T22:57:08.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.437+0000 7f4a93a00640 1 -- start start 2024-05-26T22:57:08.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.439+0000 7f4a93a00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f4a8c15ef20 0x7f4a8c0cc9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:08.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.439+0000 7f4a93a00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a8c0af200 con 0x7f4a8c15aa70 2024-05-26T22:57:08.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.439+0000 7f4a93a00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a8c0af4a0 con 0x7f4a8c0ccf10 2024-05-26T22:57:08.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.439+0000 7f4a93a00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f4a8c0afa60 con 0x7f4a8c15ef20 2024-05-26T22:57:08.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.439+0000 7f4a91000640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f4a8c15aa70 0x7f4a8c0b4db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:37644/0 (socket says 172.21.15.131:37644) 2024-05-26T22:57:08.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.439+0000 7f4a91000640 1 -- 172.21.15.131:0/2641552333 learned_addr learned my addr 172.21.15.131:0/2641552333 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:57:08.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.439+0000 7f4a8b400640 1 -- 172.21.15.131:0/2641552333 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2762034590 0 0) 0x7f4a8c0af200 con 0x7f4a8c15aa70 2024-05-26T22:57:08.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.439+0000 7f4a8be00640 1 --2- 172.21.15.131:0/2641552333 >> v2:172.21.15.187:3300/0 conn(0x7f4a8c15ef20 0x7f4a8c0cc9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:08.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.439+0000 7f4a8b400640 1 -- 172.21.15.131:0/2641552333 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4a74003520 con 0x7f4a8c15aa70 2024-05-26T22:57:08.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.440+0000 7f4a8b400640 1 -- 172.21.15.131:0/2641552333 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2432014494 0 0) 0x7f4a74003520 con 0x7f4a8c15aa70 2024-05-26T22:57:08.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.440+0000 7f4a8b400640 1 -- 172.21.15.131:0/2641552333 >> v2:172.21.15.187:3300/0 conn(0x7f4a8c15ef20 msgr2=0x7f4a8c0cc9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:57:08.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.440+0000 7f4a8b400640 1 --2- 172.21.15.131:0/2641552333 >> v2:172.21.15.187:3300/0 conn(0x7f4a8c15ef20 0x7f4a8c0cc9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:08.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.440+0000 7f4a8b400640 1 -- 172.21.15.131:0/2641552333 >> v1:172.21.15.187:6789/0 conn(0x7f4a8c0ccf10 legacy=0x7f4a8c0b2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:08.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.440+0000 7f4a8b400640 1 -- 172.21.15.131:0/2641552333 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a8c0af780 con 0x7f4a8c15aa70 2024-05-26T22:57:08.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.440+0000 7f4a8b400640 1 -- 172.21.15.131:0/2641552333 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7f4a80001670 con 0x7f4a8c15aa70 2024-05-26T22:57:08.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.440+0000 7f4a8b400640 1 -- 172.21.15.131:0/2641552333 >> v1:172.21.15.131:6789/0 conn(0x7f4a8c15aa70 legacy=0x7f4a8c0b4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:08.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.440+0000 7f4a8b400640 1 --2- 172.21.15.131:0/2641552333 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a740039b0 0x7f4a74003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:08.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.440+0000 7f4a8b400640 1 --2- 172.21.15.131:0/2641552333 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4a74004800 0x7f4a74004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:08.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.440+0000 7f4a8b400640 1 --2- 172.21.15.131:0/2641552333 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4a740051c0 0x7f4a7400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:08.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.440+0000 7f4a8b400640 1 -- 172.21.15.131:0/2641552333 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f4a8c0afa60 con 0x7f4a740039b0 2024-05-26T22:57:08.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.440+0000 7f4a8b400640 1 -- 172.21.15.131:0/2641552333 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4a7400c160 con 0x7f4a740051c0 2024-05-26T22:57:08.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.440+0000 7f4a8b400640 1 -- 172.21.15.131:0/2641552333 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f4a7400c360 con 0x7f4a74004800 2024-05-26T22:57:08.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.440+0000 7f4a91000640 1 --2- 172.21.15.131:0/2641552333 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a740039b0 0x7f4a74003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:08.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.441+0000 7f4a91000640 1 -- 172.21.15.131:0/2641552333 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4a74004800 msgr2=0x7f4a74004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:57:08.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.441+0000 7f4a91000640 1 --2- 172.21.15.131:0/2641552333 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4a74004800 0x7f4a74004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:08.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.441+0000 7f4a91000640 1 -- 172.21.15.131:0/2641552333 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4a740051c0 msgr2=0x7f4a7400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:57:08.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.441+0000 7f4a91000640 1 --2- 172.21.15.131:0/2641552333 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4a740051c0 0x7f4a7400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:08.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.441+0000 7f4a91000640 1 -- 172.21.15.131:0/2641552333 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a74003520 con 0x7f4a740039b0 2024-05-26T22:57:08.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.441+0000 7f4a91a00640 1 -- 172.21.15.131:0/2641552333 reap_dead start 2024-05-26T22:57:08.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.442+0000 7f4a91000640 1 --2- 172.21.15.131:0/2641552333 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a740039b0 0x7f4a74003dd0 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f4a80007cb0 tx=0x7f4a74005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de4fd0429aa7f0e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:57:08.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.442+0000 7f4a8b400640 1 -- 172.21.15.131:0/2641552333 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a80007cf0 con 0x7f4a740039b0 2024-05-26T22:57:08.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.442+0000 7f4a8b400640 1 -- 172.21.15.131:0/2641552333 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a80033070 con 0x7f4a740039b0 2024-05-26T22:57:08.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.442+0000 7f4a8b400640 1 -- 172.21.15.131:0/2641552333 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a80005d70 con 0x7f4a740039b0 2024-05-26T22:57:08.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.443+0000 7f4a93a00640 1 -- 172.21.15.131:0/2641552333 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a740039b0 msgr2=0x7f4a74003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:08.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.443+0000 7f4a93a00640 1 --2- 172.21.15.131:0/2641552333 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a740039b0 0x7f4a74003dd0 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f4a80007cb0 tx=0x7f4a74005830 comp rx=0 tx=0).stop 2024-05-26T22:57:08.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.443+0000 7f4a93a00640 1 -- 172.21.15.131:0/2641552333 shutdown_connections 2024-05-26T22:57:08.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.443+0000 7f4a93a00640 1 --2- 172.21.15.131:0/2641552333 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a740039b0 0x7f4a74003dd0 unknown :-1 s=CLOSED pgs=1671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:08.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.443+0000 7f4a93a00640 1 -- 172.21.15.131:0/2641552333 >> 172.21.15.131:0/2641552333 conn(0x7f4a8c0b2110 msgr2=0x7f4a8c0b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:57:08.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.443+0000 7f4a93a00640 1 -- 172.21.15.131:0/2641552333 shutdown_connections 2024-05-26T22:57:08.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.443+0000 7f4a93a00640 1 -- 172.21.15.131:0/2641552333 wait complete. 2024-05-26T22:57:08.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.445+0000 7f4a93a00640 1 Processor -- start 2024-05-26T22:57:08.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.445+0000 7f4a93a00640 1 -- start start 2024-05-26T22:57:08.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.446+0000 7f4a93a00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4a740039b0 0x7f4a8c1f41e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:08.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.446+0000 7f4a93a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a8c145c60 0x7f4a8c15d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:08.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.446+0000 7f4a93a00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4a8c15d5b0 0x7f4a8c0afc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:08.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.446+0000 7f4a93a00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f4a8c15ae50 con 0x7f4a8c145c60 2024-05-26T22:57:08.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.446+0000 7f4a93a00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4a8c15ea00 con 0x7f4a8c15d5b0 2024-05-26T22:57:08.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.446+0000 7f4a93a00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f4a8c15f380 con 0x7f4a740039b0 2024-05-26T22:57:08.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.446+0000 7f4a8be00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a8c145c60 0x7f4a8c15d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:08.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.446+0000 7f4a8be00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a8c145c60 0x7f4a8c15d070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57306/0 (socket says 172.21.15.131:57306) 2024-05-26T22:57:08.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.446+0000 7f4a8be00640 1 -- 172.21.15.131:0/772768048 learned_addr learned my addr 172.21.15.131:0/772768048 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:57:08.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.447+0000 7f4a91000640 1 --2- 172.21.15.131:0/772768048 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4a740039b0 0x7f4a8c1f41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:08.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.447+0000 7f4a91a00640 1 --2- 172.21.15.131:0/772768048 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4a8c15d5b0 0x7f4a8c0afc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:08.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.447+0000 7f4a8be00640 1 -- 172.21.15.131:0/772768048 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4a740039b0 msgr2=0x7f4a8c1f41e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:08.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.447+0000 7f4a8be00640 1 --2- 172.21.15.131:0/772768048 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4a740039b0 0x7f4a8c1f41e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:08.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.447+0000 7f4a8be00640 1 -- 172.21.15.131:0/772768048 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4a8c15d5b0 msgr2=0x7f4a8c0afc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:08.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.447+0000 7f4a8be00640 1 --2- 172.21.15.131:0/772768048 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4a8c15d5b0 0x7f4a8c0afc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:08.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.447+0000 7f4a8be00640 1 -- 172.21.15.131:0/772768048 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a8c15df10 con 0x7f4a8c145c60 2024-05-26T22:57:08.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.447+0000 7f4a8be00640 1 --2- 172.21.15.131:0/772768048 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a8c145c60 0x7f4a8c15d070 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f4a7800e7d0 tx=0x7f4a7800ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:57:08.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.447+0000 7f4a88c00640 1 -- 172.21.15.131:0/772768048 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a78017070 con 0x7f4a8c145c60 2024-05-26T22:57:08.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.447+0000 7f4a88c00640 1 -- 172.21.15.131:0/772768048 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a7800c730 con 0x7f4a8c145c60 2024-05-26T22:57:08.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.447+0000 7f4a93a00640 1 -- 172.21.15.131:0/772768048 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a8c1f3740 con 0x7f4a8c145c60 2024-05-26T22:57:08.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.447+0000 7f4a88c00640 1 -- 172.21.15.131:0/772768048 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a7800cb00 con 0x7f4a8c145c60 2024-05-26T22:57:08.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.447+0000 7f4a93a00640 1 -- 172.21.15.131:0/772768048 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a8c1f39e0 con 0x7f4a8c145c60 2024-05-26T22:57:08.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.448+0000 7f4a93a00640 1 -- 172.21.15.131:0/772768048 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4a8c068440 con 0x7f4a8c145c60 2024-05-26T22:57:08.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.449+0000 7f4a88c00640 1 -- 172.21.15.131:0/772768048 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4a7801a070 con 0x7f4a8c145c60 2024-05-26T22:57:08.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.450+0000 7f4a88c00640 1 --2- 172.21.15.131:0/772768048 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f4a6c06be30 0x7f4a6c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:08.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.450+0000 7f4a88c00640 1 -- 172.21.15.131:0/772768048 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5115+0+0 (secure 0 0 0) 0x7f4a7808ac10 con 0x7f4a8c145c60 2024-05-26T22:57:08.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.451+0000 7f4a91000640 1 --2- 172.21.15.131:0/772768048 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f4a6c06be30 0x7f4a6c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:08.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.451+0000 7f4a88c00640 1 --2- 172.21.15.131:0/772768048 >> [v2:172.21.15.131:6808/350790591,v1:172.21.15.131:6809/350790591] conn(0x7f4a6c071e30 0x7f4a6c074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:08.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.451+0000 7f4a88c00640 1 -- 172.21.15.131:0/772768048 --> [v2:172.21.15.131:6808/350790591,v1:172.21.15.131:6809/350790591] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4a6c074900 con 0x7f4a6c071e30 2024-05-26T22:57:08.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.451+0000 7f4a88c00640 1 -- 172.21.15.131:0/772768048 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_get_version_reply(handle=1 version=110) ==== 24+0+0 (secure 0 0 0) 0x7f4a7808f2a0 con 0x7f4a8c145c60 2024-05-26T22:57:08.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.451+0000 7f4a91a00640 1 --2- 172.21.15.131:0/772768048 >> [v2:172.21.15.131:6808/350790591,v1:172.21.15.131:6809/350790591] conn(0x7f4a6c071e30 0x7f4a6c074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:08.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.451+0000 7f4a91000640 1 --2- 172.21.15.131:0/772768048 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f4a6c06be30 0x7f4a6c06e2f0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f4a80037000 tx=0x7f4a8003c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:57:08.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.452+0000 7f4a91a00640 1 --2- 172.21.15.131:0/772768048 >> [v2:172.21.15.131:6808/350790591,v1:172.21.15.131:6809/350790591] conn(0x7f4a6c071e30 0x7f4a6c074250 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:57:08.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.452+0000 7f4a88c00640 1 -- 172.21.15.131:0/772768048 <== osd.3 v2:172.21.15.131:6808/350790591 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4a6c074900 con 0x7f4a6c071e30 2024-05-26T22:57:08.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.467+0000 7f4a93a00640 1 -- 172.21.15.131:0/772768048 --> [v2:172.21.15.131:6808/350790591,v1:172.21.15.131:6809/350790591] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f4a8c0cf3a0 con 0x7f4a6c071e30 2024-05-26T22:57:08.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.467+0000 7f4a88c00640 1 -- 172.21.15.131:0/772768048 <== osd.3 v2:172.21.15.131:6808/350790591 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f4a8c0cf3a0 con 0x7f4a6c071e30 2024-05-26T22:57:08.467 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-26T22:57:08.467 INFO:teuthology.orchestra.run.smithi131.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-26T22:57:08.467 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-26T22:57:08.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.468+0000 7f4a93a00640 1 -- 172.21.15.131:0/772768048 >> [v2:172.21.15.131:6808/350790591,v1:172.21.15.131:6809/350790591] conn(0x7f4a6c071e30 msgr2=0x7f4a6c074250 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:08.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.468+0000 7f4a93a00640 1 --2- 172.21.15.131:0/772768048 >> [v2:172.21.15.131:6808/350790591,v1:172.21.15.131:6809/350790591] conn(0x7f4a6c071e30 0x7f4a6c074250 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:08.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.468+0000 7f4a93a00640 1 -- 172.21.15.131:0/772768048 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f4a6c06be30 msgr2=0x7f4a6c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:08.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.468+0000 7f4a93a00640 1 --2- 172.21.15.131:0/772768048 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f4a6c06be30 0x7f4a6c06e2f0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f4a80037000 tx=0x7f4a8003c040 comp rx=0 tx=0).stop 2024-05-26T22:57:08.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.468+0000 7f4a93a00640 1 -- 172.21.15.131:0/772768048 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a8c145c60 msgr2=0x7f4a8c15d070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:08.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.468+0000 7f4a93a00640 1 --2- 172.21.15.131:0/772768048 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a8c145c60 0x7f4a8c15d070 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f4a7800e7d0 tx=0x7f4a7800ebd0 comp rx=0 tx=0).stop 2024-05-26T22:57:08.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.468+0000 7f4a91a00640 1 -- 172.21.15.131:0/772768048 reap_dead start 2024-05-26T22:57:08.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.468+0000 7f4a93a00640 1 -- 172.21.15.131:0/772768048 shutdown_connections 2024-05-26T22:57:08.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.468+0000 7f4a93a00640 1 -- 172.21.15.131:0/772768048 >> 172.21.15.131:0/772768048 conn(0x7f4a8c0b2110 msgr2=0x7f4a8c0b2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:57:08.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.468+0000 7f4a93a00640 1 -- 172.21.15.131:0/772768048 shutdown_connections 2024-05-26T22:57:08.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.468+0000 7f4a93a00640 1 -- 172.21.15.131:0/772768048 wait complete. 2024-05-26T22:57:08.487 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-05-26T22:57:08.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.683+0000 7fd547a00640 1 Processor -- start 2024-05-26T22:57:08.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.684+0000 7fd547a00640 1 -- start start 2024-05-26T22:57:08.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.685+0000 7fd547a00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fd54006f180 0x7fd540072e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:08.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.685+0000 7fd547a00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fd540072540 0x7fd540161ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:08.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.685+0000 7fd547a00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd5400cfe50 0x7fd5400d02e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:08.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.685+0000 7fd547a00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fd5400cfb30 con 0x7fd54006f180 2024-05-26T22:57:08.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.685+0000 7fd547a00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fd5400af8e0 con 0x7fd540072540 2024-05-26T22:57:08.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.685+0000 7fd547a00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd5400cf200 con 0x7fd5400cfe50 2024-05-26T22:57:08.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.685+0000 7fd545000640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fd54006f180 0x7fd540072e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:08.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.686+0000 7fd545000640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fd54006f180 0x7fd540072e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57308/0 (socket says 172.21.15.131:57308) 2024-05-26T22:57:08.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.686+0000 7fd545000640 1 -- 172.21.15.131:0/67822367 learned_addr learned my addr 172.21.15.131:0/67822367 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:57:08.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.686+0000 7fd545000640 1 -- 172.21.15.131:0/67822367 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd5400cfe50 msgr2=0x7fd5400d02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:57:08.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.686+0000 7fd545000640 1 --2- 172.21.15.131:0/67822367 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd5400cfe50 0x7fd5400d02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:08.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.686+0000 7fd545000640 1 -- 172.21.15.131:0/67822367 >> v2:172.21.15.187:3300/0 conn(0x7fd540072540 msgr2=0x7fd540161ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:57:08.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.686+0000 7fd545000640 1 --2- 172.21.15.131:0/67822367 >> v2:172.21.15.187:3300/0 conn(0x7fd540072540 0x7fd540161ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:08.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.686+0000 7fd545000640 1 -- 172.21.15.131:0/67822367 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5400cf4e0 con 0x7fd54006f180 2024-05-26T22:57:08.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.687+0000 7fd545000640 1 --2- 172.21.15.131:0/67822367 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd54006f180 0x7fd540072e90 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7fd54006dc80 tx=0x7fd53402a690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ae1418489a5219e server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:57:08.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.687+0000 7fd53f400640 1 -- 172.21.15.131:0/67822367 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd534036070 con 0x7fd54006f180 2024-05-26T22:57:08.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.687+0000 7fd53f400640 1 -- 172.21.15.131:0/67822367 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd534031600 con 0x7fd54006f180 2024-05-26T22:57:08.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.687+0000 7fd53f400640 1 -- 172.21.15.131:0/67822367 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5340319d0 con 0x7fd54006f180 2024-05-26T22:57:08.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.688+0000 7fd547a00640 1 -- 172.21.15.131:0/67822367 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd54006f180 msgr2=0x7fd540072e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:08.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.688+0000 7fd547a00640 1 --2- 172.21.15.131:0/67822367 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd54006f180 0x7fd540072e90 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7fd54006dc80 tx=0x7fd53402a690 comp rx=0 tx=0).stop 2024-05-26T22:57:08.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.688+0000 7fd547a00640 1 -- 172.21.15.131:0/67822367 shutdown_connections 2024-05-26T22:57:08.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.688+0000 7fd547a00640 1 --2- 172.21.15.131:0/67822367 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd5400cfe50 0x7fd5400d02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:08.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.688+0000 7fd547a00640 1 --2- 172.21.15.131:0/67822367 >> v2:172.21.15.187:3300/0 conn(0x7fd540072540 0x7fd540161ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:08.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.688+0000 7fd547a00640 1 --2- 172.21.15.131:0/67822367 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd54006f180 0x7fd540072e90 unknown :-1 s=CLOSED pgs=1673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:08.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.688+0000 7fd547a00640 1 -- 172.21.15.131:0/67822367 >> 172.21.15.131:0/67822367 conn(0x7fd54006bc90 msgr2=0x7fd5400b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:57:08.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.688+0000 7fd547a00640 1 -- 172.21.15.131:0/67822367 shutdown_connections 2024-05-26T22:57:08.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.688+0000 7fd547a00640 1 -- 172.21.15.131:0/67822367 wait complete. 2024-05-26T22:57:08.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.690+0000 7fd547a00640 1 Processor -- start 2024-05-26T22:57:08.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.690+0000 7fd547a00640 1 -- start start 2024-05-26T22:57:08.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.691+0000 7fd547a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd54006f180 0x7fd540151830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:08.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.691+0000 7fd547a00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd540072540 0x7fd54009c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:08.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.691+0000 7fd547a00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd5400cfe50 0x7fd54009aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:08.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.691+0000 7fd547a00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fd54006f6e0 con 0x7fd54006f180 2024-05-26T22:57:08.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.691+0000 7fd547a00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd54006f560 con 0x7fd5400cfe50 2024-05-26T22:57:08.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.691+0000 7fd547a00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd5400715a0 con 0x7fd540072540 2024-05-26T22:57:08.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.692+0000 7fd545000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd54006f180 0x7fd540151830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:08.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.692+0000 7fd545000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd54006f180 0x7fd540151830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57312/0 (socket says 172.21.15.131:57312) 2024-05-26T22:57:08.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.692+0000 7fd545000640 1 -- 172.21.15.131:0/1435843847 learned_addr learned my addr 172.21.15.131:0/1435843847 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:57:08.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.692+0000 7fd545a00640 1 --2- 172.21.15.131:0/1435843847 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd5400cfe50 0x7fd54009aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:08.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.692+0000 7fd53fe00640 1 --2- 172.21.15.131:0/1435843847 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd540072540 0x7fd54009c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:08.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.692+0000 7fd545000640 1 -- 172.21.15.131:0/1435843847 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd540072540 msgr2=0x7fd54009c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:08.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.692+0000 7fd545000640 1 --2- 172.21.15.131:0/1435843847 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd540072540 0x7fd54009c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:08.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.692+0000 7fd545000640 1 -- 172.21.15.131:0/1435843847 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd5400cfe50 msgr2=0x7fd54009aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:08.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.692+0000 7fd545000640 1 --2- 172.21.15.131:0/1435843847 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd5400cfe50 0x7fd54009aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:08.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.692+0000 7fd545000640 1 -- 172.21.15.131:0/1435843847 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd54009b3c0 con 0x7fd54006f180 2024-05-26T22:57:08.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.693+0000 7fd545000640 1 --2- 172.21.15.131:0/1435843847 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd54006f180 0x7fd540151830 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7fd534034040 tx=0x7fd534004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:57:08.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.693+0000 7fd53cc00640 1 -- 172.21.15.131:0/1435843847 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd534036050 con 0x7fd54006f180 2024-05-26T22:57:08.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.693+0000 7fd53cc00640 1 -- 172.21.15.131:0/1435843847 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd534042070 con 0x7fd54006f180 2024-05-26T22:57:08.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.693+0000 7fd53cc00640 1 -- 172.21.15.131:0/1435843847 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd534009510 con 0x7fd54006f180 2024-05-26T22:57:08.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.693+0000 7fd547a00640 1 -- 172.21.15.131:0/1435843847 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd54009b6a0 con 0x7fd54006f180 2024-05-26T22:57:08.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.693+0000 7fd547a00640 1 -- 172.21.15.131:0/1435843847 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd54009bc60 con 0x7fd54006f180 2024-05-26T22:57:08.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.694+0000 7fd547a00640 1 -- 172.21.15.131:0/1435843847 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd540074e90 con 0x7fd54006f180 2024-05-26T22:57:08.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.695+0000 7fd53cc00640 1 -- 172.21.15.131:0/1435843847 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd534007c50 con 0x7fd54006f180 2024-05-26T22:57:08.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.697+0000 7fd53cc00640 1 --2- 172.21.15.131:0/1435843847 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fd52006be30 0x7fd52006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:08.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.697+0000 7fd53cc00640 1 -- 172.21.15.131:0/1435843847 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5115+0+0 (secure 0 0 0) 0x7fd5340abcf0 con 0x7fd54006f180 2024-05-26T22:57:08.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.697+0000 7fd53fe00640 1 --2- 172.21.15.131:0/1435843847 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fd52006be30 0x7fd52006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:08.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.698+0000 7fd53fe00640 1 --2- 172.21.15.131:0/1435843847 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fd52006be30 0x7fd52006e2f0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fd5400b4ff0 tx=0x7fd52c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:57:08.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.701+0000 7fd53cc00640 1 -- 172.21.15.131:0/1435843847 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fd53407a850 con 0x7fd54006f180 2024-05-26T22:57:08.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.905+0000 7fd547a00640 1 -- 172.21.15.131:0/1435843847 --> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7fd5400695c0 con 0x7fd52006be30 2024-05-26T22:57:08.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.906+0000 7fd53cc00640 1 -- 172.21.15.131:0/1435843847 <== mgr.4104 v2:172.21.15.131:6832/3484832947 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fd5400695c0 con 0x7fd52006be30 2024-05-26T22:57:08.906 INFO:teuthology.orchestra.run.smithi131.stderr:instructed osd(s) 3 to deep-scrub 2024-05-26T22:57:08.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.909+0000 7fd547a00640 1 -- 172.21.15.131:0/1435843847 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fd52006be30 msgr2=0x7fd52006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:08.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.909+0000 7fd547a00640 1 --2- 172.21.15.131:0/1435843847 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fd52006be30 0x7fd52006e2f0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fd5400b4ff0 tx=0x7fd52c008040 comp rx=0 tx=0).stop 2024-05-26T22:57:08.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.909+0000 7fd547a00640 1 -- 172.21.15.131:0/1435843847 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd54006f180 msgr2=0x7fd540151830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:08.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.909+0000 7fd547a00640 1 --2- 172.21.15.131:0/1435843847 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd54006f180 0x7fd540151830 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7fd534034040 tx=0x7fd534004a40 comp rx=0 tx=0).stop 2024-05-26T22:57:08.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.909+0000 7fd547a00640 1 -- 172.21.15.131:0/1435843847 shutdown_connections 2024-05-26T22:57:08.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.909+0000 7fd547a00640 1 --2- 172.21.15.131:0/1435843847 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd540072540 0x7fd54009c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:08.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.909+0000 7fd547a00640 1 --2- 172.21.15.131:0/1435843847 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fd52006be30 0x7fd52006e2f0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:08.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.909+0000 7fd547a00640 1 --2- 172.21.15.131:0/1435843847 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd5400cfe50 0x7fd54009aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:08.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.909+0000 7fd547a00640 1 --2- 172.21.15.131:0/1435843847 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd54006f180 0x7fd540151830 unknown :-1 s=CLOSED pgs=1674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:08.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.909+0000 7fd547a00640 1 -- 172.21.15.131:0/1435843847 >> 172.21.15.131:0/1435843847 conn(0x7fd54006bc90 msgr2=0x7fd54015ce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:57:08.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.909+0000 7fd547a00640 1 -- 172.21.15.131:0/1435843847 shutdown_connections 2024-05-26T22:57:08.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:08.909+0000 7fd547a00640 1 -- 172.21.15.131:0/1435843847 wait complete. 2024-05-26T22:57:08.927 INFO:tasks.ceph:Scrubbing osd.4 2024-05-26T22:57:08.927 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-05-26T22:57:09.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.129+0000 7ff8d6200640 1 Processor -- start 2024-05-26T22:57:09.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.129+0000 7ff8d6200640 1 -- start start 2024-05-26T22:57:09.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.131+0000 7ff8d6200640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7ff8d015fba0 0x7ff8d006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:09.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.131+0000 7ff8d6200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff8d0160000 0x7ff8d006da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:09.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.131+0000 7ff8d6200640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8d0145c60 con 0x7ff8d0145880 2024-05-26T22:57:09.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.131+0000 7ff8d6200640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7ff8d015c560 con 0x7ff8d015fba0 2024-05-26T22:57:09.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.131+0000 7ff8d6200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff8d015cb10 con 0x7ff8d0160000 2024-05-26T22:57:09.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.132+0000 7ff8d5800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff8d0160000 0x7ff8d006da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:09.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.132+0000 7ff8d4e00640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7ff8d0145880 0x7ff8d015cfa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.131:38522/0 (socket says 172.21.15.131:38522) 2024-05-26T22:57:09.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.132+0000 7ff8d4e00640 1 -- 172.21.15.131:0/1764360102 learned_addr learned my addr 172.21.15.131:0/1764360102 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:57:09.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.132+0000 7ff8c7e00640 1 --2- 172.21.15.131:0/1764360102 >> v2:172.21.15.187:3300/0 conn(0x7ff8d015fba0 0x7ff8d006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:09.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.133+0000 7ff8d5800640 1 -- 172.21.15.131:0/1764360102 >> v2:172.21.15.187:3300/0 conn(0x7ff8d015fba0 msgr2=0x7ff8d006d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:57:09.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.133+0000 7ff8d5800640 1 --2- 172.21.15.131:0/1764360102 >> v2:172.21.15.187:3300/0 conn(0x7ff8d015fba0 0x7ff8d006d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:09.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.133+0000 7ff8d5800640 1 -- 172.21.15.131:0/1764360102 >> v1:172.21.15.187:6789/0 conn(0x7ff8d0145880 legacy=0x7ff8d015cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:09.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.133+0000 7ff8d5800640 1 -- 172.21.15.131:0/1764360102 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8d01605b0 con 0x7ff8d0160000 2024-05-26T22:57:09.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.134+0000 7ff8d5800640 1 --2- 172.21.15.131:0/1764360102 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff8d0160000 0x7ff8d006da20 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff8c8009d90 tx=0x7ff8c802a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d01848ce944a1d22 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:57:09.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.134+0000 7ff8cfe00640 1 -- 172.21.15.131:0/1764360102 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8c8007740 con 0x7ff8d0160000 2024-05-26T22:57:09.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.134+0000 7ff8cfe00640 1 -- 172.21.15.131:0/1764360102 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8c80078e0 con 0x7ff8d0160000 2024-05-26T22:57:09.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.134+0000 7ff8cfe00640 1 -- 172.21.15.131:0/1764360102 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8c8007cb0 con 0x7ff8d0160000 2024-05-26T22:57:09.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.135+0000 7ff8d6200640 1 -- 172.21.15.131:0/1764360102 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff8d0160000 msgr2=0x7ff8d006da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:09.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.135+0000 7ff8d6200640 1 --2- 172.21.15.131:0/1764360102 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff8d0160000 0x7ff8d006da20 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff8c8009d90 tx=0x7ff8c802a660 comp rx=0 tx=0).stop 2024-05-26T22:57:09.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.135+0000 7ff8d6200640 1 -- 172.21.15.131:0/1764360102 shutdown_connections 2024-05-26T22:57:09.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.135+0000 7ff8d6200640 1 --2- 172.21.15.131:0/1764360102 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff8d0160000 0x7ff8d006da20 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:09.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.135+0000 7ff8d6200640 1 --2- 172.21.15.131:0/1764360102 >> v2:172.21.15.187:3300/0 conn(0x7ff8d015fba0 0x7ff8d006d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:09.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.135+0000 7ff8d6200640 1 -- 172.21.15.131:0/1764360102 >> 172.21.15.131:0/1764360102 conn(0x7ff8d00b5410 msgr2=0x7ff8d00b57f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:57:09.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.135+0000 7ff8d6200640 1 -- 172.21.15.131:0/1764360102 shutdown_connections 2024-05-26T22:57:09.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.135+0000 7ff8d6200640 1 -- 172.21.15.131:0/1764360102 wait complete. 2024-05-26T22:57:09.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.137+0000 7ff8d6200640 1 Processor -- start 2024-05-26T22:57:09.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.137+0000 7ff8d6200640 1 -- start start 2024-05-26T22:57:09.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.138+0000 7ff8d6200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff8d0145880 0x7ff8d013f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:09.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.138+0000 7ff8d6200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff8d015fba0 0x7ff8d015e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:09.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.138+0000 7ff8d6200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff8d0160000 0x7ff8d01f71f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:09.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.138+0000 7ff8d6200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff8d00b1d10 con 0x7ff8d0160000 2024-05-26T22:57:09.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.138+0000 7ff8d6200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff8d015c7a0 con 0x7ff8d015fba0 2024-05-26T22:57:09.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.138+0000 7ff8d6200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff8d00b1e90 con 0x7ff8d0145880 2024-05-26T22:57:09.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.138+0000 7ff8d5800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff8d0160000 0x7ff8d01f71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:09.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.138+0000 7ff8d5800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff8d0160000 0x7ff8d01f71f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57324/0 (socket says 172.21.15.131:57324) 2024-05-26T22:57:09.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.138+0000 7ff8d5800640 1 -- 172.21.15.131:0/1634873246 learned_addr learned my addr 172.21.15.131:0/1634873246 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:57:09.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.139+0000 7ff8d4e00640 1 --2- 172.21.15.131:0/1634873246 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff8d0145880 0x7ff8d013f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:09.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.139+0000 7ff8c7e00640 1 --2- 172.21.15.131:0/1634873246 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff8d015fba0 0x7ff8d015e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:09.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.139+0000 7ff8d5800640 1 -- 172.21.15.131:0/1634873246 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff8d0145880 msgr2=0x7ff8d013f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:09.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.139+0000 7ff8d5800640 1 --2- 172.21.15.131:0/1634873246 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff8d0145880 0x7ff8d013f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:09.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.139+0000 7ff8d5800640 1 -- 172.21.15.131:0/1634873246 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff8d015fba0 msgr2=0x7ff8d015e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:09.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.139+0000 7ff8d5800640 1 --2- 172.21.15.131:0/1634873246 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff8d015fba0 0x7ff8d015e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:09.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.139+0000 7ff8d5800640 1 -- 172.21.15.131:0/1634873246 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8d01450f0 con 0x7ff8d0160000 2024-05-26T22:57:09.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.139+0000 7ff8d5800640 1 --2- 172.21.15.131:0/1634873246 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff8d0160000 0x7ff8d01f71f0 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7ff8c8009eb0 tx=0x7ff8c8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:57:09.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.139+0000 7ff8cd600640 1 -- 172.21.15.131:0/1634873246 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8c8040070 con 0x7ff8d0160000 2024-05-26T22:57:09.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.140+0000 7ff8cd600640 1 -- 172.21.15.131:0/1634873246 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8c8031bc0 con 0x7ff8d0160000 2024-05-26T22:57:09.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.140+0000 7ff8cd600640 1 -- 172.21.15.131:0/1634873246 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8c8009450 con 0x7ff8d0160000 2024-05-26T22:57:09.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.140+0000 7ff8d6200640 1 -- 172.21.15.131:0/1634873246 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8d015e9b0 con 0x7ff8d0160000 2024-05-26T22:57:09.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.140+0000 7ff8d6200640 1 -- 172.21.15.131:0/1634873246 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8d015ef70 con 0x7ff8d0160000 2024-05-26T22:57:09.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.141+0000 7ff8d6200640 1 -- 172.21.15.131:0/1634873246 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff8d015f1b0 con 0x7ff8d0160000 2024-05-26T22:57:09.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.142+0000 7ff8cd600640 1 -- 172.21.15.131:0/1634873246 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff8c8039040 con 0x7ff8d0160000 2024-05-26T22:57:09.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.143+0000 7ff8cd600640 1 --2- 172.21.15.131:0/1634873246 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ff8ac06be30 0x7ff8ac06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:09.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.143+0000 7ff8cd600640 1 -- 172.21.15.131:0/1634873246 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5115+0+0 (secure 0 0 0) 0x7ff8c80ac530 con 0x7ff8d0160000 2024-05-26T22:57:09.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.143+0000 7ff8d4e00640 1 --2- 172.21.15.131:0/1634873246 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ff8ac06be30 0x7ff8ac06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:09.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.143+0000 7ff8cd600640 1 --2- 172.21.15.131:0/1634873246 >> [v2:172.21.15.187:6814/244560995,v1:172.21.15.187:6816/244560995] conn(0x7ff8ac071e30 0x7ff8ac074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:09.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.143+0000 7ff8cd600640 1 -- 172.21.15.131:0/1634873246 --> [v2:172.21.15.187:6814/244560995,v1:172.21.15.187:6816/244560995] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff8c804cec0 con 0x7ff8ac071e30 2024-05-26T22:57:09.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.143+0000 7ff8cd600640 1 -- 172.21.15.131:0/1634873246 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_get_version_reply(handle=1 version=110) ==== 24+0+0 (secure 0 0 0) 0x7ff8c80b3050 con 0x7ff8d0160000 2024-05-26T22:57:09.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.144+0000 7ff8d4e00640 1 --2- 172.21.15.131:0/1634873246 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ff8ac06be30 0x7ff8ac06e2f0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7ff8c0000f60 tx=0x7ff8c0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:57:09.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.144+0000 7ff8c7e00640 1 --2- 172.21.15.131:0/1634873246 >> [v2:172.21.15.187:6814/244560995,v1:172.21.15.187:6816/244560995] conn(0x7ff8ac071e30 0x7ff8ac074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:09.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.145+0000 7ff8c7e00640 1 --2- 172.21.15.131:0/1634873246 >> [v2:172.21.15.187:6814/244560995,v1:172.21.15.187:6816/244560995] conn(0x7ff8ac071e30 0x7ff8ac074250 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:57:09.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.147+0000 7ff8cd600640 1 -- 172.21.15.131:0/1634873246 <== osd.4 v2:172.21.15.187:6814/244560995 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff8c804cec0 con 0x7ff8ac071e30 2024-05-26T22:57:09.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.161+0000 7ff8d6200640 1 -- 172.21.15.131:0/1634873246 --> [v2:172.21.15.187:6814/244560995,v1:172.21.15.187:6816/244560995] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7ff8d00694e0 con 0x7ff8ac071e30 2024-05-26T22:57:09.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.162+0000 7ff8cd600640 1 -- 172.21.15.131:0/1634873246 <== osd.4 v2:172.21.15.187:6814/244560995 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7ff8d00694e0 con 0x7ff8ac071e30 2024-05-26T22:57:09.162 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-26T22:57:09.162 INFO:teuthology.orchestra.run.smithi131.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-26T22:57:09.162 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-26T22:57:09.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.162+0000 7ff8d6200640 1 -- 172.21.15.131:0/1634873246 >> [v2:172.21.15.187:6814/244560995,v1:172.21.15.187:6816/244560995] conn(0x7ff8ac071e30 msgr2=0x7ff8ac074250 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:09.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.162+0000 7ff8d6200640 1 --2- 172.21.15.131:0/1634873246 >> [v2:172.21.15.187:6814/244560995,v1:172.21.15.187:6816/244560995] conn(0x7ff8ac071e30 0x7ff8ac074250 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:09.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.162+0000 7ff8d6200640 1 -- 172.21.15.131:0/1634873246 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ff8ac06be30 msgr2=0x7ff8ac06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:09.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.162+0000 7ff8d6200640 1 --2- 172.21.15.131:0/1634873246 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7ff8ac06be30 0x7ff8ac06e2f0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7ff8c0000f60 tx=0x7ff8c0009040 comp rx=0 tx=0).stop 2024-05-26T22:57:09.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.163+0000 7ff8d6200640 1 -- 172.21.15.131:0/1634873246 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff8d0160000 msgr2=0x7ff8d01f71f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:09.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.163+0000 7ff8d6200640 1 --2- 172.21.15.131:0/1634873246 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff8d0160000 0x7ff8d01f71f0 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7ff8c8009eb0 tx=0x7ff8c8004a40 comp rx=0 tx=0).stop 2024-05-26T22:57:09.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.163+0000 7ff8d5800640 1 -- 172.21.15.131:0/1634873246 reap_dead start 2024-05-26T22:57:09.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.163+0000 7ff8d6200640 1 -- 172.21.15.131:0/1634873246 shutdown_connections 2024-05-26T22:57:09.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.163+0000 7ff8d6200640 1 -- 172.21.15.131:0/1634873246 >> 172.21.15.131:0/1634873246 conn(0x7ff8d00b5410 msgr2=0x7ff8d00b0c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:57:09.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.163+0000 7ff8d6200640 1 -- 172.21.15.131:0/1634873246 shutdown_connections 2024-05-26T22:57:09.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.163+0000 7ff8d6200640 1 -- 172.21.15.131:0/1634873246 wait complete. 2024-05-26T22:57:09.181 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-05-26T22:57:09.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.379+0000 7f565dc00640 1 Processor -- start 2024-05-26T22:57:09.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.379+0000 7f565dc00640 1 -- start start 2024-05-26T22:57:09.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.380+0000 7f565dc00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f56580b5450 0x7f56580b43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:09.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.380+0000 7f565dc00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56580b4900 con 0x7f565806f420 2024-05-26T22:57:09.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.380+0000 7f565dc00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5658160180 con 0x7f56580d0460 2024-05-26T22:57:09.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.380+0000 7f565dc00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f5658160d90 con 0x7f56580b5450 2024-05-26T22:57:09.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.381+0000 7f564e000640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f565806f420 0x7f56580cfd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:37660/0 (socket says 172.21.15.131:37660) 2024-05-26T22:57:09.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.381+0000 7f564e000640 1 -- 172.21.15.131:0/3654491910 learned_addr learned my addr 172.21.15.131:0/3654491910 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:57:09.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.381+0000 7f5656a00640 1 --2- 172.21.15.131:0/3654491910 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f56580b5450 0x7f56580b43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:09.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.382+0000 7f5656000640 1 -- 172.21.15.131:0/3654491910 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3919803114 0 0) 0x7f56580b4900 con 0x7f565806f420 2024-05-26T22:57:09.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.382+0000 7f5656000640 1 -- 172.21.15.131:0/3654491910 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f563c003520 con 0x7f565806f420 2024-05-26T22:57:09.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.382+0000 7f5656a00640 1 -- 172.21.15.131:0/3654491910 >> v1:172.21.15.187:6789/0 conn(0x7f56580d0460 legacy=0x7f565806d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:09.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.382+0000 7f5656a00640 1 -- 172.21.15.131:0/3654491910 >> v1:172.21.15.131:6789/0 conn(0x7f565806f420 legacy=0x7f56580cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:09.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.382+0000 7f5656a00640 1 -- 172.21.15.131:0/3654491910 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5658160460 con 0x7f56580b5450 2024-05-26T22:57:09.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.383+0000 7f5656a00640 1 --2- 172.21.15.131:0/3654491910 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f56580b5450 0x7f56580b43c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f56480097d0 tx=0x7f564802a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73493a00cbdb05ca server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:57:09.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.384+0000 7f5656000640 1 -- 172.21.15.131:0/3654491910 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5648037070 con 0x7f56580b5450 2024-05-26T22:57:09.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.384+0000 7f5656000640 1 -- 172.21.15.131:0/3654491910 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5648032440 con 0x7f56580b5450 2024-05-26T22:57:09.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.384+0000 7f5656000640 1 -- 172.21.15.131:0/3654491910 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5648032810 con 0x7f56580b5450 2024-05-26T22:57:09.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.384+0000 7f565dc00640 1 -- 172.21.15.131:0/3654491910 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f56580b5450 msgr2=0x7f56580b43c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:09.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.384+0000 7f565dc00640 1 --2- 172.21.15.131:0/3654491910 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f56580b5450 0x7f56580b43c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f56480097d0 tx=0x7f564802a210 comp rx=0 tx=0).stop 2024-05-26T22:57:09.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.385+0000 7f565dc00640 1 -- 172.21.15.131:0/3654491910 shutdown_connections 2024-05-26T22:57:09.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.385+0000 7f565dc00640 1 --2- 172.21.15.131:0/3654491910 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f56580b5450 0x7f56580b43c0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:09.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.385+0000 7f565dc00640 1 -- 172.21.15.131:0/3654491910 >> 172.21.15.131:0/3654491910 conn(0x7f565806bc90 msgr2=0x7f56580b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:57:09.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.385+0000 7f565dc00640 1 -- 172.21.15.131:0/3654491910 shutdown_connections 2024-05-26T22:57:09.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.385+0000 7f565dc00640 1 -- 172.21.15.131:0/3654491910 wait complete. 2024-05-26T22:57:09.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.387+0000 7f565dc00640 1 Processor -- start 2024-05-26T22:57:09.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.387+0000 7f565dc00640 1 -- start start 2024-05-26T22:57:09.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.388+0000 7f565dc00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f565806f420 0x7f56580cb420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:09.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.388+0000 7f565dc00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f56580b5450 0x7f56580ca5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:09.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.389+0000 7f565dc00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f56580d0460 0x7f56581f3db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:09.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.389+0000 7f565dc00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f56580b5830 con 0x7f565806f420 2024-05-26T22:57:09.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.389+0000 7f565dc00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5658160a20 con 0x7f56580d0460 2024-05-26T22:57:09.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.389+0000 7f565dc00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f565815fde0 con 0x7f56580b5450 2024-05-26T22:57:09.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.389+0000 7f5656a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f565806f420 0x7f56580cb420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:09.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.389+0000 7f5656a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f565806f420 0x7f56580cb420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57338/0 (socket says 172.21.15.131:57338) 2024-05-26T22:57:09.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.389+0000 7f5656a00640 1 -- 172.21.15.131:0/937079522 learned_addr learned my addr 172.21.15.131:0/937079522 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:57:09.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.389+0000 7f564e000640 1 --2- 172.21.15.131:0/937079522 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f56580b5450 0x7f56580ca5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:09.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.389+0000 7f5656a00640 1 -- 172.21.15.131:0/937079522 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f56580b5450 msgr2=0x7f56580ca5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:09.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.389+0000 7f5656a00640 1 --2- 172.21.15.131:0/937079522 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f56580b5450 0x7f56580ca5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:09.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.389+0000 7f5656a00640 1 -- 172.21.15.131:0/937079522 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f56580d0460 msgr2=0x7f56581f3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:09.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.389+0000 7f5656a00640 1 --2- 172.21.15.131:0/937079522 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f56580d0460 0x7f56581f3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:09.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.389+0000 7f5656a00640 1 -- 172.21.15.131:0/937079522 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56580caae0 con 0x7f565806f420 2024-05-26T22:57:09.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.390+0000 7f5656a00640 1 --2- 172.21.15.131:0/937079522 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f565806f420 0x7f56580cb420 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f5648035040 tx=0x7f56480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:57:09.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.390+0000 7f564f400640 1 -- 172.21.15.131:0/937079522 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5648037040 con 0x7f565806f420 2024-05-26T22:57:09.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.390+0000 7f564f400640 1 -- 172.21.15.131:0/937079522 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5648030500 con 0x7f565806f420 2024-05-26T22:57:09.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.390+0000 7f564f400640 1 -- 172.21.15.131:0/937079522 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5648030910 con 0x7f565806f420 2024-05-26T22:57:09.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.390+0000 7f565dc00640 1 -- 172.21.15.131:0/937079522 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56581f45f0 con 0x7f565806f420 2024-05-26T22:57:09.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.390+0000 7f565dc00640 1 -- 172.21.15.131:0/937079522 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56581f4bb0 con 0x7f565806f420 2024-05-26T22:57:09.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.391+0000 7f565dc00640 1 -- 172.21.15.131:0/937079522 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5658074e90 con 0x7f565806f420 2024-05-26T22:57:09.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.392+0000 7f564f400640 1 -- 172.21.15.131:0/937079522 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5648044020 con 0x7f565806f420 2024-05-26T22:57:09.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.393+0000 7f564f400640 1 --2- 172.21.15.131:0/937079522 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f563406be30 0x7f563406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:09.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.393+0000 7f564f400640 1 -- 172.21.15.131:0/937079522 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5115+0+0 (secure 0 0 0) 0x7f56480aae60 con 0x7f565806f420 2024-05-26T22:57:09.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.393+0000 7f564e000640 1 --2- 172.21.15.131:0/937079522 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f563406be30 0x7f563406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:09.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.394+0000 7f564e000640 1 --2- 172.21.15.131:0/937079522 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f563406be30 0x7f563406e2f0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f56580aeda0 tx=0x7f5640009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:57:09.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.399+0000 7f564f400640 1 -- 172.21.15.131:0/937079522 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f5648079af0 con 0x7f565806f420 2024-05-26T22:57:09.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.600+0000 7f565dc00640 1 -- 172.21.15.131:0/937079522 --> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7f56581f4df0 con 0x7f563406be30 2024-05-26T22:57:09.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.601+0000 7f564f400640 1 -- 172.21.15.131:0/937079522 <== mgr.4104 v2:172.21.15.131:6832/3484832947 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f564000ec40 con 0x7f563406be30 2024-05-26T22:57:09.601 INFO:teuthology.orchestra.run.smithi131.stderr:instructed osd(s) 4 to deep-scrub 2024-05-26T22:57:09.604 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.603+0000 7f565dc00640 1 -- 172.21.15.131:0/937079522 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f563406be30 msgr2=0x7f563406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:09.604 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.603+0000 7f565dc00640 1 --2- 172.21.15.131:0/937079522 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f563406be30 0x7f563406e2f0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f56580aeda0 tx=0x7f5640009040 comp rx=0 tx=0).stop 2024-05-26T22:57:09.604 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.603+0000 7f565dc00640 1 -- 172.21.15.131:0/937079522 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f565806f420 msgr2=0x7f56580cb420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:09.604 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.603+0000 7f565dc00640 1 --2- 172.21.15.131:0/937079522 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f565806f420 0x7f56580cb420 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f5648035040 tx=0x7f56480046c0 comp rx=0 tx=0).stop 2024-05-26T22:57:09.604 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.604+0000 7f565dc00640 1 -- 172.21.15.131:0/937079522 shutdown_connections 2024-05-26T22:57:09.604 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.604+0000 7f565dc00640 1 --2- 172.21.15.131:0/937079522 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f56580b5450 0x7f56580ca5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:09.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.604+0000 7f565dc00640 1 --2- 172.21.15.131:0/937079522 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f563406be30 0x7f563406e2f0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:09.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.604+0000 7f565dc00640 1 --2- 172.21.15.131:0/937079522 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f56580d0460 0x7f56581f3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:09.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.604+0000 7f565dc00640 1 --2- 172.21.15.131:0/937079522 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f565806f420 0x7f56580cb420 unknown :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:09.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.604+0000 7f565dc00640 1 -- 172.21.15.131:0/937079522 >> 172.21.15.131:0/937079522 conn(0x7f565806bc90 msgr2=0x7f56580af120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:57:09.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.604+0000 7f565dc00640 1 -- 172.21.15.131:0/937079522 shutdown_connections 2024-05-26T22:57:09.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.604+0000 7f565dc00640 1 -- 172.21.15.131:0/937079522 wait complete. 2024-05-26T22:57:09.621 INFO:tasks.ceph:Scrubbing osd.5 2024-05-26T22:57:09.621 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-05-26T22:57:09.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.825+0000 7fda8ae00640 1 Processor -- start 2024-05-26T22:57:09.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.825+0000 7fda8ae00640 1 -- start start 2024-05-26T22:57:09.826 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.827+0000 7fda8ae00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fda840d0a70 0x7fda8406e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:09.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.827+0000 7fda8ae00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda8415ef10 con 0x7fda840b4af0 2024-05-26T22:57:09.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.827+0000 7fda8ae00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda8415f1f0 con 0x7fda840b4ed0 2024-05-26T22:57:09.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.827+0000 7fda8ae00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fda8406ea20 con 0x7fda840d0a70 2024-05-26T22:57:09.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.827+0000 7fda83e00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fda840b4af0 0x7fda840cf5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:37666/0 (socket says 172.21.15.131:37666) 2024-05-26T22:57:09.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.827+0000 7fda83e00640 1 -- 172.21.15.131:0/4073077264 learned_addr learned my addr 172.21.15.131:0/4073077264 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:57:09.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.827+0000 7fda82a00640 1 -- 172.21.15.131:0/4073077264 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3765246698 0 0) 0x7fda8415ef10 con 0x7fda840b4af0 2024-05-26T22:57:09.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.828+0000 7fda82a00640 1 -- 172.21.15.131:0/4073077264 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fda68003520 con 0x7fda840b4af0 2024-05-26T22:57:09.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.828+0000 7fda88e00640 1 --2- 172.21.15.131:0/4073077264 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fda840d0a70 0x7fda8406e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:09.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.828+0000 7fda82a00640 1 -- 172.21.15.131:0/4073077264 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2166828368 0 0) 0x7fda68003520 con 0x7fda840b4af0 2024-05-26T22:57:09.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.828+0000 7fda82a00640 1 -- 172.21.15.131:0/4073077264 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fda840d0a70 msgr2=0x7fda8406e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:57:09.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.828+0000 7fda82a00640 1 --2- 172.21.15.131:0/4073077264 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fda840d0a70 0x7fda8406e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:09.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.828+0000 7fda82a00640 1 -- 172.21.15.131:0/4073077264 >> v1:172.21.15.187:6789/0 conn(0x7fda840b4ed0 legacy=0x7fda840d0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:09.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.828+0000 7fda82a00640 1 -- 172.21.15.131:0/4073077264 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda8415f4d0 con 0x7fda840b4af0 2024-05-26T22:57:09.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.828+0000 7fda82a00640 1 -- 172.21.15.131:0/4073077264 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7fda74001670 con 0x7fda840b4af0 2024-05-26T22:57:09.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.828+0000 7fda82a00640 1 -- 172.21.15.131:0/4073077264 >> v1:172.21.15.131:6789/0 conn(0x7fda840b4af0 legacy=0x7fda840cf5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:09.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.828+0000 7fda82a00640 1 --2- 172.21.15.131:0/4073077264 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda680039e0 0x7fda68003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:09.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.828+0000 7fda82a00640 1 --2- 172.21.15.131:0/4073077264 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fda68004800 0x7fda68004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:09.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.828+0000 7fda82a00640 1 --2- 172.21.15.131:0/4073077264 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fda840d0a70 0x7fda68005250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:09.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.828+0000 7fda82a00640 1 -- 172.21.15.131:0/4073077264 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fda6800bb50 con 0x7fda680039e0 2024-05-26T22:57:09.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.828+0000 7fda82a00640 1 -- 172.21.15.131:0/4073077264 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fda6800bd00 con 0x7fda68004800 2024-05-26T22:57:09.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.828+0000 7fda82a00640 1 -- 172.21.15.131:0/4073077264 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fda6800bf00 con 0x7fda840d0a70 2024-05-26T22:57:09.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.828+0000 7fda83e00640 1 --2- 172.21.15.131:0/4073077264 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda680039e0 0x7fda68003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:09.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.829+0000 7fda88e00640 1 --2- 172.21.15.131:0/4073077264 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fda840d0a70 0x7fda68005250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:09.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.829+0000 7fda83400640 1 --2- 172.21.15.131:0/4073077264 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fda68004800 0x7fda68004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:09.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.829+0000 7fda83e00640 1 -- 172.21.15.131:0/4073077264 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fda840d0a70 msgr2=0x7fda68005250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:57:09.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.829+0000 7fda83e00640 1 --2- 172.21.15.131:0/4073077264 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fda840d0a70 0x7fda68005250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:09.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.829+0000 7fda83e00640 1 -- 172.21.15.131:0/4073077264 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fda68004800 msgr2=0x7fda68004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:57:09.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.829+0000 7fda83e00640 1 --2- 172.21.15.131:0/4073077264 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fda68004800 0x7fda68004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:09.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.829+0000 7fda83e00640 1 -- 172.21.15.131:0/4073077264 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda68003520 con 0x7fda680039e0 2024-05-26T22:57:09.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.829+0000 7fda88e00640 1 --2- 172.21.15.131:0/4073077264 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fda840d0a70 0x7fda68005250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:57:09.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.830+0000 7fda83e00640 1 --2- 172.21.15.131:0/4073077264 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda680039e0 0x7fda68003e00 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7fda7400c110 tx=0x7fda68005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff6b918760943fcd server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:57:09.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.830+0000 7fda82a00640 1 -- 172.21.15.131:0/4073077264 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda74007c40 con 0x7fda680039e0 2024-05-26T22:57:09.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.830+0000 7fda82a00640 1 -- 172.21.15.131:0/4073077264 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda74005af0 con 0x7fda680039e0 2024-05-26T22:57:09.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.830+0000 7fda82a00640 1 -- 172.21.15.131:0/4073077264 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda74007070 con 0x7fda680039e0 2024-05-26T22:57:09.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.831+0000 7fda8ae00640 1 -- 172.21.15.131:0/4073077264 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda680039e0 msgr2=0x7fda68003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:09.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.831+0000 7fda8ae00640 1 --2- 172.21.15.131:0/4073077264 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda680039e0 0x7fda68003e00 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7fda7400c110 tx=0x7fda68005830 comp rx=0 tx=0).stop 2024-05-26T22:57:09.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.831+0000 7fda88e00640 1 -- 172.21.15.131:0/4073077264 reap_dead start 2024-05-26T22:57:09.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.831+0000 7fda8ae00640 1 -- 172.21.15.131:0/4073077264 shutdown_connections 2024-05-26T22:57:09.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.831+0000 7fda8ae00640 1 -- 172.21.15.131:0/4073077264 >> 172.21.15.131:0/4073077264 conn(0x7fda840b2110 msgr2=0x7fda840b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:57:09.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.831+0000 7fda8ae00640 1 -- 172.21.15.131:0/4073077264 shutdown_connections 2024-05-26T22:57:09.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.831+0000 7fda8ae00640 1 -- 172.21.15.131:0/4073077264 wait complete. 2024-05-26T22:57:09.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.833+0000 7fda8ae00640 1 Processor -- start 2024-05-26T22:57:09.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.833+0000 7fda8ae00640 1 -- start start 2024-05-26T22:57:09.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.834+0000 7fda8ae00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fda680039e0 0x7fda840c9370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:09.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.834+0000 7fda8ae00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fda8414ba10 0x7fda841f77c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:09.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.834+0000 7fda8ae00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda841f7d00 0x7fda84159630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:09.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.834+0000 7fda8ae00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fda8415eac0 con 0x7fda841f7d00 2024-05-26T22:57:09.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.834+0000 7fda8ae00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fda840cfcb0 con 0x7fda8414ba10 2024-05-26T22:57:09.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.834+0000 7fda8ae00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fda8415ec40 con 0x7fda680039e0 2024-05-26T22:57:09.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.834+0000 7fda88e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda841f7d00 0x7fda84159630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:09.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.834+0000 7fda88e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda841f7d00 0x7fda84159630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57356/0 (socket says 172.21.15.131:57356) 2024-05-26T22:57:09.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.834+0000 7fda88e00640 1 -- 172.21.15.131:0/2267537825 learned_addr learned my addr 172.21.15.131:0/2267537825 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:57:09.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.834+0000 7fda83400640 1 --2- 172.21.15.131:0/2267537825 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fda8414ba10 0x7fda841f77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:09.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.834+0000 7fda83e00640 1 --2- 172.21.15.131:0/2267537825 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fda680039e0 0x7fda840c9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:09.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.835+0000 7fda88e00640 1 -- 172.21.15.131:0/2267537825 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fda680039e0 msgr2=0x7fda840c9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:09.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.835+0000 7fda88e00640 1 --2- 172.21.15.131:0/2267537825 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fda680039e0 0x7fda840c9370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:09.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.835+0000 7fda88e00640 1 -- 172.21.15.131:0/2267537825 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fda8414ba10 msgr2=0x7fda841f77c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:09.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.835+0000 7fda88e00640 1 --2- 172.21.15.131:0/2267537825 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fda8414ba10 0x7fda841f77c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:09.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.835+0000 7fda88e00640 1 -- 172.21.15.131:0/2267537825 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda8409d8d0 con 0x7fda841f7d00 2024-05-26T22:57:09.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.835+0000 7fda83e00640 1 --2- 172.21.15.131:0/2267537825 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fda680039e0 0x7fda840c9370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:57:09.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.835+0000 7fda88e00640 1 --2- 172.21.15.131:0/2267537825 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda841f7d00 0x7fda84159630 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7fda7c0098a0 tx=0x7fda7c009d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:57:09.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.835+0000 7fda7be00640 1 -- 172.21.15.131:0/2267537825 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda7c019070 con 0x7fda841f7d00 2024-05-26T22:57:09.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.835+0000 7fda7be00640 1 -- 172.21.15.131:0/2267537825 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda7c007ce0 con 0x7fda841f7d00 2024-05-26T22:57:09.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.835+0000 7fda7be00640 1 -- 172.21.15.131:0/2267537825 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda7c014640 con 0x7fda841f7d00 2024-05-26T22:57:09.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.835+0000 7fda8ae00640 1 -- 172.21.15.131:0/2267537825 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda840cb020 con 0x7fda841f7d00 2024-05-26T22:57:09.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.835+0000 7fda8ae00640 1 -- 172.21.15.131:0/2267537825 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda840cb5e0 con 0x7fda841f7d00 2024-05-26T22:57:09.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.836+0000 7fda8ae00640 1 -- 172.21.15.131:0/2267537825 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fda84068440 con 0x7fda841f7d00 2024-05-26T22:57:09.840 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.838+0000 7fda7be00640 1 -- 172.21.15.131:0/2267537825 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fda7c01c070 con 0x7fda841f7d00 2024-05-26T22:57:09.840 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.838+0000 7fda7be00640 1 --2- 172.21.15.131:0/2267537825 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fda6406be30 0x7fda6406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:09.840 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.839+0000 7fda7be00640 1 -- 172.21.15.131:0/2267537825 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5115+0+0 (secure 0 0 0) 0x7fda7c08a670 con 0x7fda841f7d00 2024-05-26T22:57:09.840 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.839+0000 7fda83e00640 1 --2- 172.21.15.131:0/2267537825 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fda6406be30 0x7fda6406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:09.840 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.839+0000 7fda7be00640 1 --2- 172.21.15.131:0/2267537825 >> [v2:172.21.15.187:6800/324619398,v1:172.21.15.187:6801/324619398] conn(0x7fda64071e30 0x7fda64074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:09.840 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.839+0000 7fda7be00640 1 -- 172.21.15.131:0/2267537825 --> [v2:172.21.15.187:6800/324619398,v1:172.21.15.187:6801/324619398] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fda7c006bf0 con 0x7fda64071e30 2024-05-26T22:57:09.840 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.839+0000 7fda7be00640 1 -- 172.21.15.131:0/2267537825 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_get_version_reply(handle=1 version=110) ==== 24+0+0 (secure 0 0 0) 0x7fda7c091050 con 0x7fda841f7d00 2024-05-26T22:57:09.841 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.839+0000 7fda83400640 1 --2- 172.21.15.131:0/2267537825 >> [v2:172.21.15.187:6800/324619398,v1:172.21.15.187:6801/324619398] conn(0x7fda64071e30 0x7fda64074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:09.841 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.840+0000 7fda83e00640 1 --2- 172.21.15.131:0/2267537825 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fda6406be30 0x7fda6406e2f0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fda74032000 tx=0x7fda74009030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:57:09.841 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.840+0000 7fda83400640 1 --2- 172.21.15.131:0/2267537825 >> [v2:172.21.15.187:6800/324619398,v1:172.21.15.187:6801/324619398] conn(0x7fda64071e30 0x7fda64074250 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:57:09.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.842+0000 7fda7be00640 1 -- 172.21.15.131:0/2267537825 <== osd.5 v2:172.21.15.187:6800/324619398 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fda7c006bf0 con 0x7fda64071e30 2024-05-26T22:57:09.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.857+0000 7fda8ae00640 1 -- 172.21.15.131:0/2267537825 --> [v2:172.21.15.187:6800/324619398,v1:172.21.15.187:6801/324619398] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fda840ceb00 con 0x7fda64071e30 2024-05-26T22:57:09.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.858+0000 7fda7be00640 1 -- 172.21.15.131:0/2267537825 <== osd.5 v2:172.21.15.187:6800/324619398 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fda840ceb00 con 0x7fda64071e30 2024-05-26T22:57:09.859 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-26T22:57:09.859 INFO:teuthology.orchestra.run.smithi131.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-26T22:57:09.859 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-26T22:57:09.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.859+0000 7fda8ae00640 1 -- 172.21.15.131:0/2267537825 >> [v2:172.21.15.187:6800/324619398,v1:172.21.15.187:6801/324619398] conn(0x7fda64071e30 msgr2=0x7fda64074250 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:09.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.859+0000 7fda8ae00640 1 --2- 172.21.15.131:0/2267537825 >> [v2:172.21.15.187:6800/324619398,v1:172.21.15.187:6801/324619398] conn(0x7fda64071e30 0x7fda64074250 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:09.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.859+0000 7fda8ae00640 1 -- 172.21.15.131:0/2267537825 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fda6406be30 msgr2=0x7fda6406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:09.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.859+0000 7fda8ae00640 1 --2- 172.21.15.131:0/2267537825 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fda6406be30 0x7fda6406e2f0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fda74032000 tx=0x7fda74009030 comp rx=0 tx=0).stop 2024-05-26T22:57:09.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.859+0000 7fda8ae00640 1 -- 172.21.15.131:0/2267537825 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda841f7d00 msgr2=0x7fda84159630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:09.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.859+0000 7fda8ae00640 1 --2- 172.21.15.131:0/2267537825 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda841f7d00 0x7fda84159630 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7fda7c0098a0 tx=0x7fda7c009d60 comp rx=0 tx=0).stop 2024-05-26T22:57:09.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.859+0000 7fda88e00640 1 -- 172.21.15.131:0/2267537825 reap_dead start 2024-05-26T22:57:09.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.860+0000 7fda8ae00640 1 -- 172.21.15.131:0/2267537825 shutdown_connections 2024-05-26T22:57:09.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.860+0000 7fda8ae00640 1 -- 172.21.15.131:0/2267537825 >> 172.21.15.131:0/2267537825 conn(0x7fda840b2110 msgr2=0x7fda8415c650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:57:09.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.860+0000 7fda8ae00640 1 -- 172.21.15.131:0/2267537825 shutdown_connections 2024-05-26T22:57:09.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:09.860+0000 7fda8ae00640 1 -- 172.21.15.131:0/2267537825 wait complete. 2024-05-26T22:57:09.877 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-05-26T22:57:10.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.070+0000 7f706d600640 1 Processor -- start 2024-05-26T22:57:10.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.071+0000 7f706d600640 1 -- start start 2024-05-26T22:57:10.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.072+0000 7f706d600640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f706806dd30 0x7f706806e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:10.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.072+0000 7f706d600640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f706815c1c0 con 0x7f70680725a0 2024-05-26T22:57:10.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.072+0000 7f706d600640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f706815c4a0 con 0x7f7068155770 2024-05-26T22:57:10.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.072+0000 7f706d600640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f7068071da0 con 0x7f706806dd30 2024-05-26T22:57:10.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.072+0000 7f7067400640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f70680725a0 0x7f70680729a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:37680/0 (socket says 172.21.15.131:37680) 2024-05-26T22:57:10.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.072+0000 7f7067400640 1 -- 172.21.15.131:0/940850478 learned_addr learned my addr 172.21.15.131:0/940850478 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:57:10.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.073+0000 7f7065600640 1 -- 172.21.15.131:0/940850478 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 164654336 0 0) 0x7f706815c1c0 con 0x7f70680725a0 2024-05-26T22:57:10.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.073+0000 7f7065600640 1 -- 172.21.15.131:0/940850478 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f704c005730 con 0x7f70680725a0 2024-05-26T22:57:10.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.073+0000 7f7066000640 1 --2- 172.21.15.131:0/940850478 >> v2:172.21.15.187:3300/0 conn(0x7f706806dd30 0x7f706806e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:10.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.073+0000 7f7065600640 1 -- 172.21.15.131:0/940850478 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1481058642 0 0) 0x7f704c005730 con 0x7f70680725a0 2024-05-26T22:57:10.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.073+0000 7f7065600640 1 -- 172.21.15.131:0/940850478 >> v2:172.21.15.187:3300/0 conn(0x7f706806dd30 msgr2=0x7f706806e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:57:10.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.073+0000 7f7065600640 1 --2- 172.21.15.131:0/940850478 >> v2:172.21.15.187:3300/0 conn(0x7f706806dd30 0x7f706806e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:10.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.073+0000 7f7065600640 1 -- 172.21.15.131:0/940850478 >> v1:172.21.15.187:6789/0 conn(0x7f7068155770 legacy=0x7f706815b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:10.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.073+0000 7f7065600640 1 -- 172.21.15.131:0/940850478 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7068072080 con 0x7f70680725a0 2024-05-26T22:57:10.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.073+0000 7f7065600640 1 -- 172.21.15.131:0/940850478 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2895061575 0 0) 0x7f70600019f0 con 0x7f70680725a0 2024-05-26T22:57:10.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.073+0000 7f7065600640 1 -- 172.21.15.131:0/940850478 >> v1:172.21.15.131:6789/0 conn(0x7f70680725a0 legacy=0x7f70680729a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:10.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.073+0000 7f7065600640 1 --2- 172.21.15.131:0/940850478 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f704c005bc0 0x7f704c005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:10.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.073+0000 7f7065600640 1 --2- 172.21.15.131:0/940850478 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f704c006a10 0x7f704c006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:10.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.073+0000 7f7065600640 1 --2- 172.21.15.131:0/940850478 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f704c0073d0 0x7f704c00dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:10.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.073+0000 7f7065600640 1 -- 172.21.15.131:0/940850478 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f7068071da0 con 0x7f704c0073d0 2024-05-26T22:57:10.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.073+0000 7f7065600640 1 -- 172.21.15.131:0/940850478 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f704c00e300 con 0x7f704c005bc0 2024-05-26T22:57:10.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.073+0000 7f7065600640 1 -- 172.21.15.131:0/940850478 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f704c00e500 con 0x7f704c006a10 2024-05-26T22:57:10.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.074+0000 7f7067400640 1 --2- 172.21.15.131:0/940850478 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f704c0073d0 0x7f704c00dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:10.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.074+0000 7f7066a00640 1 --2- 172.21.15.131:0/940850478 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f704c005bc0 0x7f704c005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:10.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.074+0000 7f7066000640 1 --2- 172.21.15.131:0/940850478 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f704c006a10 0x7f704c006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:10.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.074+0000 7f7067400640 1 -- 172.21.15.131:0/940850478 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f704c006a10 msgr2=0x7f704c006e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:57:10.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.074+0000 7f7067400640 1 --2- 172.21.15.131:0/940850478 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f704c006a10 0x7f704c006e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:10.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.074+0000 7f7067400640 1 -- 172.21.15.131:0/940850478 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f704c005bc0 msgr2=0x7f704c005fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:57:10.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.074+0000 7f7067400640 1 --2- 172.21.15.131:0/940850478 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f704c005bc0 0x7f704c005fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:10.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.074+0000 7f7067400640 1 -- 172.21.15.131:0/940850478 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f704c005730 con 0x7f704c0073d0 2024-05-26T22:57:10.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.075+0000 7f7067400640 1 -- 172.21.15.131:0/940850478 reap_dead start 2024-05-26T22:57:10.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.075+0000 7f7067400640 1 --2- 172.21.15.131:0/940850478 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f704c0073d0 0x7f704c00dbd0 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f706000a840 tx=0x7f704c007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68ceff6d388c5042 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:57:10.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.076+0000 7f7065600640 1 -- 172.21.15.131:0/940850478 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f706000a460 con 0x7f704c0073d0 2024-05-26T22:57:10.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.076+0000 7f7065600640 1 -- 172.21.15.131:0/940850478 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f706002f070 con 0x7f704c0073d0 2024-05-26T22:57:10.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.076+0000 7f7065600640 1 -- 172.21.15.131:0/940850478 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f706002bcb0 con 0x7f704c0073d0 2024-05-26T22:57:10.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.076+0000 7f706d600640 1 -- 172.21.15.131:0/940850478 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f704c0073d0 msgr2=0x7f704c00dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:10.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.076+0000 7f706d600640 1 --2- 172.21.15.131:0/940850478 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f704c0073d0 0x7f704c00dbd0 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f706000a840 tx=0x7f704c007a40 comp rx=0 tx=0).stop 2024-05-26T22:57:10.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.076+0000 7f706d600640 1 -- 172.21.15.131:0/940850478 shutdown_connections 2024-05-26T22:57:10.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.076+0000 7f706d600640 1 --2- 172.21.15.131:0/940850478 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f704c0073d0 0x7f704c00dbd0 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:10.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.076+0000 7f706d600640 1 -- 172.21.15.131:0/940850478 >> 172.21.15.131:0/940850478 conn(0x7f706806bc90 msgr2=0x7f70680b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:57:10.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.076+0000 7f706d600640 1 -- 172.21.15.131:0/940850478 shutdown_connections 2024-05-26T22:57:10.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.076+0000 7f706d600640 1 -- 172.21.15.131:0/940850478 wait complete. 2024-05-26T22:57:10.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.078+0000 7f706d600640 1 Processor -- start 2024-05-26T22:57:10.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.078+0000 7f706d600640 1 -- start start 2024-05-26T22:57:10.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.079+0000 7f706d600640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f704c0073d0 0x7f70680c4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:10.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.079+0000 7f706d600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f706814f9e0 0x7f706814d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:10.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.079+0000 7f706d600640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f706809cf90 0x7f706815a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:10.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.079+0000 7f706d600640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f706809de00 con 0x7f706814f9e0 2024-05-26T22:57:10.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.079+0000 7f706d600640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f706815c8c0 con 0x7f704c0073d0 2024-05-26T22:57:10.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.079+0000 7f706d600640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f706809df80 con 0x7f706809cf90 2024-05-26T22:57:10.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.079+0000 7f7066000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f706814f9e0 0x7f706814d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:10.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.080+0000 7f7066000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f706814f9e0 0x7f706814d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57370/0 (socket says 172.21.15.131:57370) 2024-05-26T22:57:10.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.080+0000 7f7066000640 1 -- 172.21.15.131:0/120269415 learned_addr learned my addr 172.21.15.131:0/120269415 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:57:10.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.080+0000 7f7067400640 1 --2- 172.21.15.131:0/120269415 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f706809cf90 0x7f706815a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:10.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.080+0000 7f7066a00640 1 --2- 172.21.15.131:0/120269415 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f704c0073d0 0x7f70680c4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:10.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.080+0000 7f7066000640 1 -- 172.21.15.131:0/120269415 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f706809cf90 msgr2=0x7f706815a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:10.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.080+0000 7f7066000640 1 --2- 172.21.15.131:0/120269415 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f706809cf90 0x7f706815a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:10.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.080+0000 7f7066000640 1 -- 172.21.15.131:0/120269415 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f704c0073d0 msgr2=0x7f70680c4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:10.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.080+0000 7f7066000640 1 --2- 172.21.15.131:0/120269415 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f704c0073d0 0x7f70680c4e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:10.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.080+0000 7f7066000640 1 -- 172.21.15.131:0/120269415 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70680c5510 con 0x7f706814f9e0 2024-05-26T22:57:10.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.080+0000 7f7067400640 1 --2- 172.21.15.131:0/120269415 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f706809cf90 0x7f706815a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:57:10.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.080+0000 7f7066000640 1 --2- 172.21.15.131:0/120269415 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f706814f9e0 0x7f706814d480 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f705000b7f0 tx=0x7f705000bcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:57:10.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.081+0000 7f705ea00640 1 -- 172.21.15.131:0/120269415 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7050018070 con 0x7f706814f9e0 2024-05-26T22:57:10.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.081+0000 7f705ea00640 1 -- 172.21.15.131:0/120269415 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7050010830 con 0x7f706814f9e0 2024-05-26T22:57:10.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.081+0000 7f705ea00640 1 -- 172.21.15.131:0/120269415 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7050010c00 con 0x7f706814f9e0 2024-05-26T22:57:10.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.081+0000 7f706d600640 1 -- 172.21.15.131:0/120269415 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70680c57f0 con 0x7f706814f9e0 2024-05-26T22:57:10.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.081+0000 7f706d600640 1 -- 172.21.15.131:0/120269415 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70680c5db0 con 0x7f706814f9e0 2024-05-26T22:57:10.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.082+0000 7f706d600640 1 -- 172.21.15.131:0/120269415 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7068072280 con 0x7f706814f9e0 2024-05-26T22:57:10.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.083+0000 7f705ea00640 1 -- 172.21.15.131:0/120269415 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f705001b070 con 0x7f706814f9e0 2024-05-26T22:57:10.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.084+0000 7f705ea00640 1 --2- 172.21.15.131:0/120269415 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f704406be60 0x7f704406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:10.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.084+0000 7f705ea00640 1 -- 172.21.15.131:0/120269415 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5115+0+0 (secure 0 0 0) 0x7f705008b580 con 0x7f706814f9e0 2024-05-26T22:57:10.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.084+0000 7f7066a00640 1 --2- 172.21.15.131:0/120269415 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f704406be60 0x7f704406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:10.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.085+0000 7f7066a00640 1 --2- 172.21.15.131:0/120269415 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f704406be60 0x7f704406e320 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f7058005510 tx=0x7f7058009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:57:10.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.088+0000 7f705ea00640 1 -- 172.21.15.131:0/120269415 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f7068072280 con 0x7f706814f9e0 2024-05-26T22:57:10.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.285+0000 7f706d600640 1 -- 172.21.15.131:0/120269415 --> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7f70680b1fc0 con 0x7f704406be60 2024-05-26T22:57:10.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.286+0000 7f705ea00640 1 -- 172.21.15.131:0/120269415 <== mgr.4104 v2:172.21.15.131:6832/3484832947 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f70680b1fc0 con 0x7f704406be60 2024-05-26T22:57:10.287 INFO:teuthology.orchestra.run.smithi131.stderr:instructed osd(s) 5 to deep-scrub 2024-05-26T22:57:10.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.289+0000 7f706d600640 1 -- 172.21.15.131:0/120269415 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f704406be60 msgr2=0x7f704406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:10.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.289+0000 7f706d600640 1 --2- 172.21.15.131:0/120269415 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f704406be60 0x7f704406e320 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f7058005510 tx=0x7f7058009040 comp rx=0 tx=0).stop 2024-05-26T22:57:10.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.289+0000 7f706d600640 1 -- 172.21.15.131:0/120269415 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f706814f9e0 msgr2=0x7f706814d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:10.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.289+0000 7f706d600640 1 --2- 172.21.15.131:0/120269415 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f706814f9e0 0x7f706814d480 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f705000b7f0 tx=0x7f705000bcb0 comp rx=0 tx=0).stop 2024-05-26T22:57:10.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.289+0000 7f706d600640 1 -- 172.21.15.131:0/120269415 shutdown_connections 2024-05-26T22:57:10.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.289+0000 7f706d600640 1 --2- 172.21.15.131:0/120269415 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f706809cf90 0x7f706815a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:10.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.289+0000 7f706d600640 1 --2- 172.21.15.131:0/120269415 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f704406be60 0x7f704406e320 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:10.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.289+0000 7f706d600640 1 --2- 172.21.15.131:0/120269415 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f706814f9e0 0x7f706814d480 unknown :-1 s=CLOSED pgs=1683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:10.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.289+0000 7f706d600640 1 --2- 172.21.15.131:0/120269415 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f704c0073d0 0x7f70680c4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:10.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.289+0000 7f706d600640 1 -- 172.21.15.131:0/120269415 >> 172.21.15.131:0/120269415 conn(0x7f706806bc90 msgr2=0x7f70680cfd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:57:10.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.289+0000 7f706d600640 1 -- 172.21.15.131:0/120269415 shutdown_connections 2024-05-26T22:57:10.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.290+0000 7f706d600640 1 -- 172.21.15.131:0/120269415 wait complete. 2024-05-26T22:57:10.308 INFO:tasks.ceph:Scrubbing osd.6 2024-05-26T22:57:10.309 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-05-26T22:57:10.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.518+0000 7f7e15000640 1 Processor -- start 2024-05-26T22:57:10.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.519+0000 7f7e15000640 1 -- start start 2024-05-26T22:57:10.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.520+0000 7f7e15000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7e100b4af0 0x7f7e1006e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:10.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.520+0000 7f7e15000640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f7e100b4ed0 0x7f7e100cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:10.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.520+0000 7f7e15000640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f7e100d0360 0x7f7e100d07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:10.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.520+0000 7f7e15000640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f7e1006ea20 con 0x7f7e100b4ed0 2024-05-26T22:57:10.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.521+0000 7f7e15000640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f7e1015ef10 con 0x7f7e100d0360 2024-05-26T22:57:10.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.521+0000 7f7e15000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f7e1015f1f0 con 0x7f7e100b4af0 2024-05-26T22:57:10.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.521+0000 7f7e07e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f7e100b4ed0 0x7f7e100cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:10.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.521+0000 7f7e07e00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f7e100b4ed0 0x7f7e100cf5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57374/0 (socket says 172.21.15.131:57374) 2024-05-26T22:57:10.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.521+0000 7f7e07e00640 1 -- 172.21.15.131:0/472160768 learned_addr learned my addr 172.21.15.131:0/472160768 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:57:10.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.521+0000 7f7e0e800640 1 --2- 172.21.15.131:0/472160768 >> v2:172.21.15.187:3300/0 conn(0x7f7e100d0360 0x7f7e100d07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:10.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.521+0000 7f7e0de00640 1 --2- 172.21.15.131:0/472160768 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7e100b4af0 0x7f7e1006e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:10.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.521+0000 7f7e07e00640 1 -- 172.21.15.131:0/472160768 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7e100b4af0 msgr2=0x7f7e1006e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:57:10.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.521+0000 7f7e07e00640 1 --2- 172.21.15.131:0/472160768 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7e100b4af0 0x7f7e1006e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:10.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.521+0000 7f7e07e00640 1 -- 172.21.15.131:0/472160768 >> v2:172.21.15.187:3300/0 conn(0x7f7e100d0360 msgr2=0x7f7e100d07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:57:10.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.521+0000 7f7e07e00640 1 --2- 172.21.15.131:0/472160768 >> v2:172.21.15.187:3300/0 conn(0x7f7e100d0360 0x7f7e100d07f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:10.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.521+0000 7f7e07e00640 1 -- 172.21.15.131:0/472160768 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e1015f4d0 con 0x7f7e100b4ed0 2024-05-26T22:57:10.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.523+0000 7f7e07e00640 1 --2- 172.21.15.131:0/472160768 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e100b4ed0 0x7f7e100cf5a0 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f7e1015c600 tx=0x7f7df802a1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36b92fd99504dccd server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:57:10.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.523+0000 7f7e0d400640 1 -- 172.21.15.131:0/472160768 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7df8037070 con 0x7f7e100b4ed0 2024-05-26T22:57:10.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.523+0000 7f7e0d400640 1 -- 172.21.15.131:0/472160768 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7df8032440 con 0x7f7e100b4ed0 2024-05-26T22:57:10.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.523+0000 7f7e0d400640 1 -- 172.21.15.131:0/472160768 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7df8032810 con 0x7f7e100b4ed0 2024-05-26T22:57:10.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.524+0000 7f7e15000640 1 -- 172.21.15.131:0/472160768 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e100b4ed0 msgr2=0x7f7e100cf5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:10.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.524+0000 7f7e15000640 1 --2- 172.21.15.131:0/472160768 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e100b4ed0 0x7f7e100cf5a0 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f7e1015c600 tx=0x7f7df802a1d0 comp rx=0 tx=0).stop 2024-05-26T22:57:10.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.524+0000 7f7e15000640 1 -- 172.21.15.131:0/472160768 shutdown_connections 2024-05-26T22:57:10.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.524+0000 7f7e15000640 1 --2- 172.21.15.131:0/472160768 >> v2:172.21.15.187:3300/0 conn(0x7f7e100d0360 0x7f7e100d07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:10.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.524+0000 7f7e15000640 1 --2- 172.21.15.131:0/472160768 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e100b4ed0 0x7f7e100cf5a0 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:10.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.524+0000 7f7e15000640 1 --2- 172.21.15.131:0/472160768 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7e100b4af0 0x7f7e1006e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:10.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.524+0000 7f7e15000640 1 -- 172.21.15.131:0/472160768 >> 172.21.15.131:0/472160768 conn(0x7f7e100b2110 msgr2=0x7f7e100b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:57:10.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.524+0000 7f7e15000640 1 -- 172.21.15.131:0/472160768 shutdown_connections 2024-05-26T22:57:10.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.524+0000 7f7e15000640 1 -- 172.21.15.131:0/472160768 wait complete. 2024-05-26T22:57:10.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.526+0000 7f7e15000640 1 Processor -- start 2024-05-26T22:57:10.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.526+0000 7f7e15000640 1 -- start start 2024-05-26T22:57:10.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.527+0000 7f7e15000640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7e100b4af0 0x7f7e10145560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:10.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.527+0000 7f7e15000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7e100b4ed0 0x7f7e10147390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:10.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.527+0000 7f7e15000640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e100d0360 0x7f7e1009aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:10.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.527+0000 7f7e15000640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f7e100cfc60 con 0x7f7e100d0360 2024-05-26T22:57:10.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.527+0000 7f7e15000640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7e100cfae0 con 0x7f7e100b4af0 2024-05-26T22:57:10.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.527+0000 7f7e15000640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f7e1015eac0 con 0x7f7e100b4ed0 2024-05-26T22:57:10.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.527+0000 7f7e0e800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e100d0360 0x7f7e1009aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:10.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.527+0000 7f7e0e800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e100d0360 0x7f7e1009aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57384/0 (socket says 172.21.15.131:57384) 2024-05-26T22:57:10.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.527+0000 7f7e0e800640 1 -- 172.21.15.131:0/3027146960 learned_addr learned my addr 172.21.15.131:0/3027146960 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:57:10.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.527+0000 7f7e0de00640 1 --2- 172.21.15.131:0/3027146960 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7e100b4af0 0x7f7e10145560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:10.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.527+0000 7f7e07e00640 1 --2- 172.21.15.131:0/3027146960 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7e100b4ed0 0x7f7e10147390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:10.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.527+0000 7f7e0e800640 1 -- 172.21.15.131:0/3027146960 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7e100b4ed0 msgr2=0x7f7e10147390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:10.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.527+0000 7f7e0e800640 1 --2- 172.21.15.131:0/3027146960 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7e100b4ed0 0x7f7e10147390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:10.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.527+0000 7f7e0e800640 1 -- 172.21.15.131:0/3027146960 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7e100b4af0 msgr2=0x7f7e10145560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:10.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.527+0000 7f7e0e800640 1 --2- 172.21.15.131:0/3027146960 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7e100b4af0 0x7f7e10145560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:10.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.527+0000 7f7e0e800640 1 -- 172.21.15.131:0/3027146960 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e10145aa0 con 0x7f7e100d0360 2024-05-26T22:57:10.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.528+0000 7f7e0e800640 1 --2- 172.21.15.131:0/3027146960 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e100d0360 0x7f7e1009aa00 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f7e08007e70 tx=0x7f7e0800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:57:10.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.528+0000 7f7e06000640 1 -- 172.21.15.131:0/3027146960 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e08014620 con 0x7f7e100d0360 2024-05-26T22:57:10.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.528+0000 7f7e06000640 1 -- 172.21.15.131:0/3027146960 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e08019070 con 0x7f7e100d0360 2024-05-26T22:57:10.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.528+0000 7f7e15000640 1 -- 172.21.15.131:0/3027146960 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e10158150 con 0x7f7e100d0360 2024-05-26T22:57:10.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.528+0000 7f7e06000640 1 -- 172.21.15.131:0/3027146960 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e08014a20 con 0x7f7e100d0360 2024-05-26T22:57:10.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.528+0000 7f7e15000640 1 -- 172.21.15.131:0/3027146960 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e101465d0 con 0x7f7e100d0360 2024-05-26T22:57:10.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.529+0000 7f7e15000640 1 -- 172.21.15.131:0/3027146960 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7e10068440 con 0x7f7e100d0360 2024-05-26T22:57:10.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.530+0000 7f7e06000640 1 -- 172.21.15.131:0/3027146960 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7e08014bc0 con 0x7f7e100d0360 2024-05-26T22:57:10.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.531+0000 7f7e06000640 1 --2- 172.21.15.131:0/3027146960 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f7dec06be30 0x7f7dec06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:10.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.531+0000 7f7e06000640 1 -- 172.21.15.131:0/3027146960 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5115+0+0 (secure 0 0 0) 0x7f7e0808b4c0 con 0x7f7e100d0360 2024-05-26T22:57:10.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.531+0000 7f7e0de00640 1 --2- 172.21.15.131:0/3027146960 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f7dec06be30 0x7f7dec06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:10.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.531+0000 7f7e06000640 1 --2- 172.21.15.131:0/3027146960 >> [v2:172.21.15.187:6804/1391347786,v1:172.21.15.187:6806/1391347786] conn(0x7f7dec071e30 0x7f7dec074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:10.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.531+0000 7f7e06000640 1 -- 172.21.15.131:0/3027146960 --> [v2:172.21.15.187:6804/1391347786,v1:172.21.15.187:6806/1391347786] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7e0800df30 con 0x7f7dec071e30 2024-05-26T22:57:10.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.531+0000 7f7e06000640 1 -- 172.21.15.131:0/3027146960 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_get_version_reply(handle=1 version=110) ==== 24+0+0 (secure 0 0 0) 0x7f7e08092050 con 0x7f7e100d0360 2024-05-26T22:57:10.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.532+0000 7f7e07e00640 1 --2- 172.21.15.131:0/3027146960 >> [v2:172.21.15.187:6804/1391347786,v1:172.21.15.187:6806/1391347786] conn(0x7f7dec071e30 0x7f7dec074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:10.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.532+0000 7f7e0de00640 1 --2- 172.21.15.131:0/3027146960 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f7dec06be30 0x7f7dec06e2f0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f7e00006fd0 tx=0x7f7e00008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:57:10.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.533+0000 7f7e07e00640 1 --2- 172.21.15.131:0/3027146960 >> [v2:172.21.15.187:6804/1391347786,v1:172.21.15.187:6806/1391347786] conn(0x7f7dec071e30 0x7f7dec074250 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:57:10.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.536+0000 7f7e06000640 1 -- 172.21.15.131:0/3027146960 <== osd.6 v2:172.21.15.187:6804/1391347786 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7e0800df30 con 0x7f7dec071e30 2024-05-26T22:57:10.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.550+0000 7f7e15000640 1 -- 172.21.15.131:0/3027146960 --> [v2:172.21.15.187:6804/1391347786,v1:172.21.15.187:6806/1391347786] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f7e10147b20 con 0x7f7dec071e30 2024-05-26T22:57:10.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.551+0000 7f7e06000640 1 -- 172.21.15.131:0/3027146960 <== osd.6 v2:172.21.15.187:6804/1391347786 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f7e10147b20 con 0x7f7dec071e30 2024-05-26T22:57:10.551 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-26T22:57:10.551 INFO:teuthology.orchestra.run.smithi131.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-26T22:57:10.551 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-26T22:57:10.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.551+0000 7f7e15000640 1 -- 172.21.15.131:0/3027146960 >> [v2:172.21.15.187:6804/1391347786,v1:172.21.15.187:6806/1391347786] conn(0x7f7dec071e30 msgr2=0x7f7dec074250 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:10.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.551+0000 7f7e15000640 1 --2- 172.21.15.131:0/3027146960 >> [v2:172.21.15.187:6804/1391347786,v1:172.21.15.187:6806/1391347786] conn(0x7f7dec071e30 0x7f7dec074250 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:10.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.552+0000 7f7e15000640 1 -- 172.21.15.131:0/3027146960 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f7dec06be30 msgr2=0x7f7dec06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:10.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.552+0000 7f7e15000640 1 --2- 172.21.15.131:0/3027146960 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f7dec06be30 0x7f7dec06e2f0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f7e00006fd0 tx=0x7f7e00008040 comp rx=0 tx=0).stop 2024-05-26T22:57:10.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.552+0000 7f7e15000640 1 -- 172.21.15.131:0/3027146960 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e100d0360 msgr2=0x7f7e1009aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:10.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.552+0000 7f7e15000640 1 --2- 172.21.15.131:0/3027146960 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e100d0360 0x7f7e1009aa00 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f7e08007e70 tx=0x7f7e0800e4b0 comp rx=0 tx=0).stop 2024-05-26T22:57:10.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.552+0000 7f7e0e800640 1 -- 172.21.15.131:0/3027146960 reap_dead start 2024-05-26T22:57:10.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.552+0000 7f7e15000640 1 -- 172.21.15.131:0/3027146960 shutdown_connections 2024-05-26T22:57:10.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.552+0000 7f7e15000640 1 -- 172.21.15.131:0/3027146960 >> 172.21.15.131:0/3027146960 conn(0x7f7e100b2110 msgr2=0x7f7e100acfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:57:10.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.552+0000 7f7e15000640 1 -- 172.21.15.131:0/3027146960 shutdown_connections 2024-05-26T22:57:10.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.552+0000 7f7e15000640 1 -- 172.21.15.131:0/3027146960 wait complete. 2024-05-26T22:57:10.571 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-05-26T22:57:10.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.769+0000 7fd6f3200640 1 Processor -- start 2024-05-26T22:57:10.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.769+0000 7fd6f3200640 1 -- start start 2024-05-26T22:57:10.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.771+0000 7fd6f3200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fd6ec15b730 0x7fd6ec15bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:10.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.771+0000 7fd6f3200640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fd6ec1400c0 0x7fd6ec1404f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:10.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.771+0000 7fd6f3200640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6ec13e820 con 0x7fd6ec06d250 2024-05-26T22:57:10.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.771+0000 7fd6f3200640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fd6ec071260 con 0x7fd6ec15b730 2024-05-26T22:57:10.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.771+0000 7fd6f3200640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fd6ec15c4a0 con 0x7fd6ec1400c0 2024-05-26T22:57:10.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.771+0000 7fd6f1400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fd6ec15b730 0x7fd6ec15bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:10.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.771+0000 7fd6f1400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fd6ec15b730 0x7fd6ec15bbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57398/0 (socket says 172.21.15.131:57398) 2024-05-26T22:57:10.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.771+0000 7fd6f1400640 1 -- 172.21.15.131:0/1014138539 learned_addr learned my addr 172.21.15.131:0/1014138539 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:57:10.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.772+0000 7fd6f1400640 1 -- 172.21.15.131:0/1014138539 >> v2:172.21.15.187:3300/0 conn(0x7fd6ec1400c0 msgr2=0x7fd6ec1404f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:57:10.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.772+0000 7fd6f2800640 1 --2- 172.21.15.131:0/1014138539 >> v2:172.21.15.187:3300/0 conn(0x7fd6ec1400c0 0x7fd6ec1404f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:10.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.772+0000 7fd6f1400640 1 --2- 172.21.15.131:0/1014138539 >> v2:172.21.15.187:3300/0 conn(0x7fd6ec1400c0 0x7fd6ec1404f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:10.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.772+0000 7fd6f1400640 1 -- 172.21.15.131:0/1014138539 >> v1:172.21.15.187:6789/0 conn(0x7fd6ec06d250 legacy=0x7fd6ec06d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:57:10.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.772+0000 7fd6f1400640 1 -- 172.21.15.131:0/1014138539 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6ec0714b0 con 0x7fd6ec15b730 2024-05-26T22:57:10.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.773+0000 7fd6f1400640 1 --2- 172.21.15.131:0/1014138539 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6ec15b730 0x7fd6ec15bbc0 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7fd6dc009870 tx=0x7fd6dc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0f410fbeb415414 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:57:10.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.773+0000 7fd6f0a00640 1 -- 172.21.15.131:0/1014138539 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6dc037070 con 0x7fd6ec15b730 2024-05-26T22:57:10.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.774+0000 7fd6f0a00640 1 -- 172.21.15.131:0/1014138539 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6dc032440 con 0x7fd6ec15b730 2024-05-26T22:57:10.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.774+0000 7fd6f0a00640 1 -- 172.21.15.131:0/1014138539 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6dc032810 con 0x7fd6ec15b730 2024-05-26T22:57:10.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.774+0000 7fd6f3200640 1 -- 172.21.15.131:0/1014138539 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6ec15b730 msgr2=0x7fd6ec15bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:10.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.774+0000 7fd6f3200640 1 --2- 172.21.15.131:0/1014138539 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6ec15b730 0x7fd6ec15bbc0 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7fd6dc009870 tx=0x7fd6dc02a270 comp rx=0 tx=0).stop 2024-05-26T22:57:10.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.774+0000 7fd6f3200640 1 -- 172.21.15.131:0/1014138539 shutdown_connections 2024-05-26T22:57:10.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.774+0000 7fd6f3200640 1 --2- 172.21.15.131:0/1014138539 >> v2:172.21.15.187:3300/0 conn(0x7fd6ec1400c0 0x7fd6ec1404f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:10.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.774+0000 7fd6f3200640 1 --2- 172.21.15.131:0/1014138539 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6ec15b730 0x7fd6ec15bbc0 unknown :-1 s=CLOSED pgs=1686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:10.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.774+0000 7fd6f3200640 1 -- 172.21.15.131:0/1014138539 >> 172.21.15.131:0/1014138539 conn(0x7fd6ec0b5660 msgr2=0x7fd6ec0b5a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:57:10.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.775+0000 7fd6f3200640 1 -- 172.21.15.131:0/1014138539 shutdown_connections 2024-05-26T22:57:10.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.775+0000 7fd6f3200640 1 -- 172.21.15.131:0/1014138539 wait complete. 2024-05-26T22:57:10.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.776+0000 7fd6f3200640 1 Processor -- start 2024-05-26T22:57:10.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.776+0000 7fd6f3200640 1 -- start start 2024-05-26T22:57:10.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.777+0000 7fd6f3200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd6ec06d250 0x7fd6ec13d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:10.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.777+0000 7fd6f3200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6ec1400c0 0x7fd6ec153f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:10.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.777+0000 7fd6f3200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd6ec15b730 0x7fd6ec0cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:10.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.777+0000 7fd6f3200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fd6ec15c100 con 0x7fd6ec1400c0 2024-05-26T22:57:10.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.777+0000 7fd6f3200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd6ec071a70 con 0x7fd6ec15b730 2024-05-26T22:57:10.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.777+0000 7fd6f3200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd6ec15c280 con 0x7fd6ec06d250 2024-05-26T22:57:10.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.778+0000 7fd6f1400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6ec1400c0 0x7fd6ec153f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:10.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.778+0000 7fd6f1400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6ec1400c0 0x7fd6ec153f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57404/0 (socket says 172.21.15.131:57404) 2024-05-26T22:57:10.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.778+0000 7fd6f1400640 1 -- 172.21.15.131:0/4151130476 learned_addr learned my addr 172.21.15.131:0/4151130476 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:57:10.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.778+0000 7fd6f1400640 1 -- 172.21.15.131:0/4151130476 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd6ec06d250 msgr2=0x7fd6ec13d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:10.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.778+0000 7fd6f1e00640 1 --2- 172.21.15.131:0/4151130476 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd6ec06d250 0x7fd6ec13d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:10.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.778+0000 7fd6f2800640 1 --2- 172.21.15.131:0/4151130476 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd6ec15b730 0x7fd6ec0cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:10.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.778+0000 7fd6f1400640 1 --2- 172.21.15.131:0/4151130476 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd6ec06d250 0x7fd6ec13d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:10.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.778+0000 7fd6f1400640 1 -- 172.21.15.131:0/4151130476 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd6ec15b730 msgr2=0x7fd6ec0cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:10.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.778+0000 7fd6f1400640 1 --2- 172.21.15.131:0/4151130476 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd6ec15b730 0x7fd6ec0cfb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:10.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.778+0000 7fd6f1400640 1 -- 172.21.15.131:0/4151130476 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6ec0af170 con 0x7fd6ec1400c0 2024-05-26T22:57:10.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.778+0000 7fd6f1e00640 1 --2- 172.21.15.131:0/4151130476 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd6ec06d250 0x7fd6ec13d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:57:10.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.779+0000 7fd6f1400640 1 --2- 172.21.15.131:0/4151130476 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6ec1400c0 0x7fd6ec153f10 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7fd6dc000c00 tx=0x7fd6dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:57:10.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.779+0000 7fd6e6000640 1 -- 172.21.15.131:0/4151130476 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6dc037040 con 0x7fd6ec1400c0 2024-05-26T22:57:10.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.779+0000 7fd6e6000640 1 -- 172.21.15.131:0/4151130476 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6dc004230 con 0x7fd6ec1400c0 2024-05-26T22:57:10.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.779+0000 7fd6e6000640 1 -- 172.21.15.131:0/4151130476 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6dc030620 con 0x7fd6ec1400c0 2024-05-26T22:57:10.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.779+0000 7fd6f3200640 1 -- 172.21.15.131:0/4151130476 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6ec0ae3b0 con 0x7fd6ec1400c0 2024-05-26T22:57:10.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.779+0000 7fd6f3200640 1 -- 172.21.15.131:0/4151130476 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6ec0ae970 con 0x7fd6ec1400c0 2024-05-26T22:57:10.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.780+0000 7fd6f3200640 1 -- 172.21.15.131:0/4151130476 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6ec0694e0 con 0x7fd6ec1400c0 2024-05-26T22:57:10.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.781+0000 7fd6e6000640 1 -- 172.21.15.131:0/4151130476 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd6dc0307c0 con 0x7fd6ec1400c0 2024-05-26T22:57:10.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.782+0000 7fd6e6000640 1 --2- 172.21.15.131:0/4151130476 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fd6d006be30 0x7fd6d006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:10.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.782+0000 7fd6e6000640 1 -- 172.21.15.131:0/4151130476 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5115+0+0 (secure 0 0 0) 0x7fd6dc0abaa0 con 0x7fd6ec1400c0 2024-05-26T22:57:10.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.782+0000 7fd6f1e00640 1 --2- 172.21.15.131:0/4151130476 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fd6d006be30 0x7fd6d006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:10.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.783+0000 7fd6f1e00640 1 --2- 172.21.15.131:0/4151130476 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fd6d006be30 0x7fd6d006e2f0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec073040 tx=0x7fd6e0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:57:10.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.786+0000 7fd6e6000640 1 -- 172.21.15.131:0/4151130476 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fd6dc07a690 con 0x7fd6ec1400c0 2024-05-26T22:57:10.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.986+0000 7fd6f3200640 1 -- 172.21.15.131:0/4151130476 --> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7fd6ec0cf030 con 0x7fd6d006be30 2024-05-26T22:57:10.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.987+0000 7fd6e6000640 1 -- 172.21.15.131:0/4151130476 <== mgr.4104 v2:172.21.15.131:6832/3484832947 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fd6e0005050 con 0x7fd6d006be30 2024-05-26T22:57:10.987 INFO:teuthology.orchestra.run.smithi131.stderr:instructed osd(s) 6 to deep-scrub 2024-05-26T22:57:10.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.990+0000 7fd6f3200640 1 -- 172.21.15.131:0/4151130476 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fd6d006be30 msgr2=0x7fd6d006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:10.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.990+0000 7fd6f3200640 1 --2- 172.21.15.131:0/4151130476 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fd6d006be30 0x7fd6d006e2f0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec073040 tx=0x7fd6e0009040 comp rx=0 tx=0).stop 2024-05-26T22:57:10.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.990+0000 7fd6f3200640 1 -- 172.21.15.131:0/4151130476 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6ec1400c0 msgr2=0x7fd6ec153f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:10.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.990+0000 7fd6f3200640 1 --2- 172.21.15.131:0/4151130476 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6ec1400c0 0x7fd6ec153f10 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7fd6dc000c00 tx=0x7fd6dc0046c0 comp rx=0 tx=0).stop 2024-05-26T22:57:10.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.990+0000 7fd6f3200640 1 -- 172.21.15.131:0/4151130476 shutdown_connections 2024-05-26T22:57:10.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.990+0000 7fd6f3200640 1 --2- 172.21.15.131:0/4151130476 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fd6d006be30 0x7fd6d006e2f0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:10.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.990+0000 7fd6f3200640 1 --2- 172.21.15.131:0/4151130476 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd6ec15b730 0x7fd6ec0cfb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:10.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.990+0000 7fd6f3200640 1 --2- 172.21.15.131:0/4151130476 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6ec1400c0 0x7fd6ec153f10 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:10.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.990+0000 7fd6f3200640 1 --2- 172.21.15.131:0/4151130476 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd6ec06d250 0x7fd6ec13d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:10.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.990+0000 7fd6f3200640 1 -- 172.21.15.131:0/4151130476 >> 172.21.15.131:0/4151130476 conn(0x7fd6ec0b5660 msgr2=0x7fd6ec0b0440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:57:10.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.990+0000 7fd6f3200640 1 -- 172.21.15.131:0/4151130476 shutdown_connections 2024-05-26T22:57:10.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:10.990+0000 7fd6f3200640 1 -- 172.21.15.131:0/4151130476 wait complete. 2024-05-26T22:57:11.011 INFO:tasks.ceph:Scrubbing osd.7 2024-05-26T22:57:11.011 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-05-26T22:57:11.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.209+0000 7f4730a00640 1 Processor -- start 2024-05-26T22:57:11.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.209+0000 7f4730a00640 1 -- start start 2024-05-26T22:57:11.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.211+0000 7f4730a00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f472c1478d0 0x7f472c161dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:11.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.211+0000 7f4730a00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f472c0ae7b0 0x7f472c0aebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:11.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.211+0000 7f4730a00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f472c15e7d0 con 0x7f472c0ae070 2024-05-26T22:57:11.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.211+0000 7f4730a00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f472c0af120 con 0x7f472c0ae7b0 2024-05-26T22:57:11.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.211+0000 7f4730a00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f472c162300 con 0x7f472c1478d0 2024-05-26T22:57:11.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.211+0000 7f472b200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f472c0ae7b0 0x7f472c0aebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:11.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.211+0000 7f472b200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f472c0ae7b0 0x7f472c0aebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57406/0 (socket says 172.21.15.131:57406) 2024-05-26T22:57:11.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.212+0000 7f472b200640 1 -- 172.21.15.131:0/178865230 learned_addr learned my addr 172.21.15.131:0/178865230 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:57:11.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.212+0000 7f472b200640 1 -- 172.21.15.131:0/178865230 >> v2:172.21.15.187:3300/0 conn(0x7f472c1478d0 msgr2=0x7f472c161dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:57:11.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.212+0000 7f472b200640 1 --2- 172.21.15.131:0/178865230 >> v2:172.21.15.187:3300/0 conn(0x7f472c1478d0 0x7f472c161dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:11.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.212+0000 7f472b200640 1 -- 172.21.15.131:0/178865230 >> v1:172.21.15.187:6789/0 conn(0x7f472c0ae070 legacy=0x7f472c15ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:57:11.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.212+0000 7f472b200640 1 -- 172.21.15.131:0/178865230 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f472c1625a0 con 0x7f472c0ae7b0 2024-05-26T22:57:11.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.213+0000 7f472b200640 1 --2- 172.21.15.131:0/178865230 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f472c0ae7b0 0x7f472c0aebe0 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f472c0b2c50 tx=0x7f472402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f13a2075f96d8daf server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:57:11.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.213+0000 7f4729400640 1 -- 172.21.15.131:0/178865230 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4724037070 con 0x7f472c0ae7b0 2024-05-26T22:57:11.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.213+0000 7f4729400640 1 -- 172.21.15.131:0/178865230 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4724032440 con 0x7f472c0ae7b0 2024-05-26T22:57:11.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.213+0000 7f4729400640 1 -- 172.21.15.131:0/178865230 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4724032810 con 0x7f472c0ae7b0 2024-05-26T22:57:11.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.214+0000 7f4730a00640 1 -- 172.21.15.131:0/178865230 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f472c0ae7b0 msgr2=0x7f472c0aebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:11.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.214+0000 7f4730a00640 1 --2- 172.21.15.131:0/178865230 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f472c0ae7b0 0x7f472c0aebe0 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f472c0b2c50 tx=0x7f472402a230 comp rx=0 tx=0).stop 2024-05-26T22:57:11.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.214+0000 7f4730a00640 1 -- 172.21.15.131:0/178865230 shutdown_connections 2024-05-26T22:57:11.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.214+0000 7f4730a00640 1 --2- 172.21.15.131:0/178865230 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f472c0ae7b0 0x7f472c0aebe0 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:11.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.214+0000 7f4730a00640 1 --2- 172.21.15.131:0/178865230 >> v2:172.21.15.187:3300/0 conn(0x7f472c1478d0 0x7f472c161dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:11.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.214+0000 7f4730a00640 1 -- 172.21.15.131:0/178865230 >> 172.21.15.131:0/178865230 conn(0x7f472c0b5410 msgr2=0x7f472c0b57f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:57:11.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.214+0000 7f4730a00640 1 -- 172.21.15.131:0/178865230 shutdown_connections 2024-05-26T22:57:11.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.214+0000 7f4730a00640 1 -- 172.21.15.131:0/178865230 wait complete. 2024-05-26T22:57:11.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.216+0000 7f4730a00640 1 Processor -- start 2024-05-26T22:57:11.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.216+0000 7f4730a00640 1 -- start start 2024-05-26T22:57:11.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.217+0000 7f4730a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f472c0ae070 0x7f472c0b1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:11.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.217+0000 7f4730a00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f472c0ae7b0 0x7f472c0c4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:11.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.217+0000 7f4730a00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f472c1478d0 0x7f472c1f44f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:11.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.217+0000 7f4730a00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f472c15ea50 con 0x7f472c0ae070 2024-05-26T22:57:11.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.217+0000 7f4730a00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f472c162b60 con 0x7f472c1478d0 2024-05-26T22:57:11.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.217+0000 7f4730a00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f472c147cb0 con 0x7f472c0ae7b0 2024-05-26T22:57:11.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.217+0000 7f472a800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f472c0ae070 0x7f472c0b1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:11.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.217+0000 7f472a800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f472c0ae070 0x7f472c0b1ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57422/0 (socket says 172.21.15.131:57422) 2024-05-26T22:57:11.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.217+0000 7f472a800640 1 -- 172.21.15.131:0/2502171099 learned_addr learned my addr 172.21.15.131:0/2502171099 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:57:11.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.217+0000 7f472a800640 1 -- 172.21.15.131:0/2502171099 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f472c0ae7b0 msgr2=0x7f472c0c4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:11.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.217+0000 7f472a800640 1 --2- 172.21.15.131:0/2502171099 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f472c0ae7b0 0x7f472c0c4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:11.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.217+0000 7f472a800640 1 -- 172.21.15.131:0/2502171099 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f472c1478d0 msgr2=0x7f472c1f44f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:11.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.217+0000 7f472a800640 1 --2- 172.21.15.131:0/2502171099 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f472c1478d0 0x7f472c1f44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:11.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.217+0000 7f472a800640 1 -- 172.21.15.131:0/2502171099 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f472c0cc510 con 0x7f472c0ae070 2024-05-26T22:57:11.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.218+0000 7f472a800640 1 --2- 172.21.15.131:0/2502171099 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f472c0ae070 0x7f472c0b1ad0 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7f471400b4d0 tx=0x7f471400b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:57:11.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.218+0000 7f4722a00640 1 -- 172.21.15.131:0/2502171099 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4714014070 con 0x7f472c0ae070 2024-05-26T22:57:11.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.218+0000 7f4722a00640 1 -- 172.21.15.131:0/2502171099 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4714010610 con 0x7f472c0ae070 2024-05-26T22:57:11.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.218+0000 7f4722a00640 1 -- 172.21.15.131:0/2502171099 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4714010a00 con 0x7f472c0ae070 2024-05-26T22:57:11.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.218+0000 7f4730a00640 1 -- 172.21.15.131:0/2502171099 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f472c0c55d0 con 0x7f472c0ae070 2024-05-26T22:57:11.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.218+0000 7f4730a00640 1 -- 172.21.15.131:0/2502171099 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f472c0c5b90 con 0x7f472c0ae070 2024-05-26T22:57:11.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.219+0000 7f4730a00640 1 -- 172.21.15.131:0/2502171099 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f472c0694e0 con 0x7f472c0ae070 2024-05-26T22:57:11.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.220+0000 7f4722a00640 1 -- 172.21.15.131:0/2502171099 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4714017070 con 0x7f472c0ae070 2024-05-26T22:57:11.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.221+0000 7f4722a00640 1 --2- 172.21.15.131:0/2502171099 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f470806be30 0x7f470806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:11.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.221+0000 7f4722a00640 1 -- 172.21.15.131:0/2502171099 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5115+0+0 (secure 0 0 0) 0x7f471408a570 con 0x7f472c0ae070 2024-05-26T22:57:11.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.222+0000 7f4722a00640 1 --2- 172.21.15.131:0/2502171099 >> [v2:172.21.15.187:6824/1220065622,v1:172.21.15.187:6825/1220065622] conn(0x7f4708071e30 0x7f4708074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:11.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.222+0000 7f4729e00640 1 --2- 172.21.15.131:0/2502171099 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f470806be30 0x7f470806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:11.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.222+0000 7f4722a00640 1 -- 172.21.15.131:0/2502171099 --> [v2:172.21.15.187:6824/1220065622,v1:172.21.15.187:6825/1220065622] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4714021e50 con 0x7f4708071e30 2024-05-26T22:57:11.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.222+0000 7f4722a00640 1 -- 172.21.15.131:0/2502171099 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_get_version_reply(handle=1 version=110) ==== 24+0+0 (secure 0 0 0) 0x7f4714091050 con 0x7f472c0ae070 2024-05-26T22:57:11.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.222+0000 7f472b200640 1 --2- 172.21.15.131:0/2502171099 >> [v2:172.21.15.187:6824/1220065622,v1:172.21.15.187:6825/1220065622] conn(0x7f4708071e30 0x7f4708074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:11.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.222+0000 7f4729e00640 1 --2- 172.21.15.131:0/2502171099 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f470806be30 0x7f470806e2f0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f472c0b1560 tx=0x7f471c009250 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:57:11.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.223+0000 7f472b200640 1 --2- 172.21.15.131:0/2502171099 >> [v2:172.21.15.187:6824/1220065622,v1:172.21.15.187:6825/1220065622] conn(0x7f4708071e30 0x7f4708074250 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:57:11.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.225+0000 7f4722a00640 1 -- 172.21.15.131:0/2502171099 <== osd.7 v2:172.21.15.187:6824/1220065622 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4714021e50 con 0x7f4708071e30 2024-05-26T22:57:11.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.240+0000 7f4730a00640 1 -- 172.21.15.131:0/2502171099 --> [v2:172.21.15.187:6824/1220065622,v1:172.21.15.187:6825/1220065622] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f472c0b2f30 con 0x7f4708071e30 2024-05-26T22:57:11.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.241+0000 7f4722a00640 1 -- 172.21.15.131:0/2502171099 <== osd.7 v2:172.21.15.187:6824/1220065622 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f472c0b2f30 con 0x7f4708071e30 2024-05-26T22:57:11.241 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-05-26T22:57:11.241 INFO:teuthology.orchestra.run.smithi131.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-26T22:57:11.241 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-05-26T22:57:11.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.241+0000 7f4730a00640 1 -- 172.21.15.131:0/2502171099 >> [v2:172.21.15.187:6824/1220065622,v1:172.21.15.187:6825/1220065622] conn(0x7f4708071e30 msgr2=0x7f4708074250 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:11.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.241+0000 7f4730a00640 1 --2- 172.21.15.131:0/2502171099 >> [v2:172.21.15.187:6824/1220065622,v1:172.21.15.187:6825/1220065622] conn(0x7f4708071e30 0x7f4708074250 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:11.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.242+0000 7f4730a00640 1 -- 172.21.15.131:0/2502171099 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f470806be30 msgr2=0x7f470806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:11.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.242+0000 7f4730a00640 1 --2- 172.21.15.131:0/2502171099 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f470806be30 0x7f470806e2f0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f472c0b1560 tx=0x7f471c009250 comp rx=0 tx=0).stop 2024-05-26T22:57:11.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.242+0000 7f4730a00640 1 -- 172.21.15.131:0/2502171099 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f472c0ae070 msgr2=0x7f472c0b1ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:11.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.242+0000 7f4730a00640 1 --2- 172.21.15.131:0/2502171099 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f472c0ae070 0x7f472c0b1ad0 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7f471400b4d0 tx=0x7f471400b990 comp rx=0 tx=0).stop 2024-05-26T22:57:11.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.242+0000 7f472b200640 1 -- 172.21.15.131:0/2502171099 reap_dead start 2024-05-26T22:57:11.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.242+0000 7f4730a00640 1 -- 172.21.15.131:0/2502171099 shutdown_connections 2024-05-26T22:57:11.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.242+0000 7f4730a00640 1 -- 172.21.15.131:0/2502171099 >> 172.21.15.131:0/2502171099 conn(0x7f472c0b5410 msgr2=0x7f472c0ce9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:57:11.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.242+0000 7f4730a00640 1 -- 172.21.15.131:0/2502171099 shutdown_connections 2024-05-26T22:57:11.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.242+0000 7f4730a00640 1 -- 172.21.15.131:0/2502171099 wait complete. 2024-05-26T22:57:11.259 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-05-26T22:57:11.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.454+0000 7fa443e00640 1 Processor -- start 2024-05-26T22:57:11.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.454+0000 7fa443e00640 1 -- start start 2024-05-26T22:57:11.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.456+0000 7fa443e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa44415bfa0 0x7fa444071500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:11.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.456+0000 7fa443e00640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fa44406d960 0x7fa444071a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:11.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.456+0000 7fa443e00640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4441552d0 con 0x7fa44415dc40 2024-05-26T22:57:11.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.456+0000 7fa443e00640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fa4440b3f30 con 0x7fa44406d960 2024-05-26T22:57:11.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.456+0000 7fa443e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa444071f80 con 0x7fa44415bfa0 2024-05-26T22:57:11.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.456+0000 7fa442000640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa44415bfa0 0x7fa444071500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:11.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.456+0000 7fa443400640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fa44406d960 0x7fa444071a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:11.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.456+0000 7fa442a00640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7fa44415dc40 0x7fa44406d250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.131:38558/0 (socket says 172.21.15.131:38558) 2024-05-26T22:57:11.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.456+0000 7fa442a00640 1 -- 172.21.15.131:0/3025424097 learned_addr learned my addr 172.21.15.131:0/3025424097 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:57:11.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.457+0000 7fa443400640 1 -- 172.21.15.131:0/3025424097 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa44415bfa0 msgr2=0x7fa444071500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:57:11.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.457+0000 7fa443400640 1 --2- 172.21.15.131:0/3025424097 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa44415bfa0 0x7fa444071500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:11.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.457+0000 7fa443400640 1 -- 172.21.15.131:0/3025424097 >> v1:172.21.15.187:6789/0 conn(0x7fa44415dc40 legacy=0x7fa44406d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:11.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.457+0000 7fa443400640 1 -- 172.21.15.131:0/3025424097 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa444072230 con 0x7fa44406d960 2024-05-26T22:57:11.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.458+0000 7fa443400640 1 --2- 172.21.15.131:0/3025424097 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa44406d960 0x7fa444071a40 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fa43c009d90 tx=0x7fa43c02a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3fccc4ea3a2a420 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:57:11.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.459+0000 7fa441600640 1 -- 172.21.15.131:0/3025424097 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa43c007740 con 0x7fa44406d960 2024-05-26T22:57:11.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.459+0000 7fa441600640 1 -- 172.21.15.131:0/3025424097 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa43c0078e0 con 0x7fa44406d960 2024-05-26T22:57:11.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.459+0000 7fa441600640 1 -- 172.21.15.131:0/3025424097 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa43c007cb0 con 0x7fa44406d960 2024-05-26T22:57:11.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.460+0000 7fa443e00640 1 -- 172.21.15.131:0/3025424097 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa44406d960 msgr2=0x7fa444071a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:11.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.460+0000 7fa443e00640 1 --2- 172.21.15.131:0/3025424097 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa44406d960 0x7fa444071a40 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fa43c009d90 tx=0x7fa43c02a660 comp rx=0 tx=0).stop 2024-05-26T22:57:11.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.460+0000 7fa443e00640 1 -- 172.21.15.131:0/3025424097 shutdown_connections 2024-05-26T22:57:11.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.460+0000 7fa443e00640 1 --2- 172.21.15.131:0/3025424097 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa44406d960 0x7fa444071a40 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:11.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.460+0000 7fa443e00640 1 --2- 172.21.15.131:0/3025424097 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa44415bfa0 0x7fa444071500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:11.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.460+0000 7fa443e00640 1 -- 172.21.15.131:0/3025424097 >> 172.21.15.131:0/3025424097 conn(0x7fa4440b5660 msgr2=0x7fa4440b5a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:57:11.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.460+0000 7fa443e00640 1 -- 172.21.15.131:0/3025424097 shutdown_connections 2024-05-26T22:57:11.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.460+0000 7fa443e00640 1 -- 172.21.15.131:0/3025424097 wait complete. 2024-05-26T22:57:11.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.462+0000 7fa443e00640 1 Processor -- start 2024-05-26T22:57:11.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.462+0000 7fa443e00640 1 -- start start 2024-05-26T22:57:11.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.463+0000 7fa443e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa44406d960 0x7fa44414e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:11.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.463+0000 7fa443e00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa44415bfa0 0x7fa444152870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:11.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.463+0000 7fa443e00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa44415dc40 0x7fa4440cefa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:11.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.463+0000 7fa443e00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa44415e020 con 0x7fa44406d960 2024-05-26T22:57:11.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.463+0000 7fa443e00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa4440b3bc0 con 0x7fa44415dc40 2024-05-26T22:57:11.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.463+0000 7fa443e00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa44415e1a0 con 0x7fa44415bfa0 2024-05-26T22:57:11.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.464+0000 7fa442a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa44406d960 0x7fa44414e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:11.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.464+0000 7fa442a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa44406d960 0x7fa44414e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57430/0 (socket says 172.21.15.131:57430) 2024-05-26T22:57:11.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.464+0000 7fa442a00640 1 -- 172.21.15.131:0/2794016256 learned_addr learned my addr 172.21.15.131:0/2794016256 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:57:11.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.464+0000 7fa442000640 1 --2- 172.21.15.131:0/2794016256 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa44415bfa0 0x7fa444152870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:11.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.464+0000 7fa443400640 1 --2- 172.21.15.131:0/2794016256 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa44415dc40 0x7fa4440cefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:11.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.464+0000 7fa442a00640 1 -- 172.21.15.131:0/2794016256 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa44415bfa0 msgr2=0x7fa444152870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:11.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.464+0000 7fa442a00640 1 --2- 172.21.15.131:0/2794016256 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa44415bfa0 0x7fa444152870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:11.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.464+0000 7fa442a00640 1 -- 172.21.15.131:0/2794016256 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa44415dc40 msgr2=0x7fa4440cefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:11.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.464+0000 7fa442a00640 1 --2- 172.21.15.131:0/2794016256 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa44415dc40 0x7fa4440cefa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:11.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.464+0000 7fa442a00640 1 -- 172.21.15.131:0/2794016256 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4441531f0 con 0x7fa44406d960 2024-05-26T22:57:11.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.464+0000 7fa442000640 1 --2- 172.21.15.131:0/2794016256 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa44415bfa0 0x7fa444152870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:57:11.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.465+0000 7fa442a00640 1 --2- 172.21.15.131:0/2794016256 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa44406d960 0x7fa44414e930 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7fa43400c4b0 tx=0x7fa43400c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:57:11.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.465+0000 7fa43aa00640 1 -- 172.21.15.131:0/2794016256 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa434017070 con 0x7fa44406d960 2024-05-26T22:57:11.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.465+0000 7fa43aa00640 1 -- 172.21.15.131:0/2794016256 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa434005ce0 con 0x7fa44406d960 2024-05-26T22:57:11.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.465+0000 7fa43aa00640 1 -- 172.21.15.131:0/2794016256 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa434007270 con 0x7fa44406d960 2024-05-26T22:57:11.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.465+0000 7fa443e00640 1 -- 172.21.15.131:0/2794016256 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4441534d0 con 0x7fa44406d960 2024-05-26T22:57:11.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.465+0000 7fa443e00640 1 -- 172.21.15.131:0/2794016256 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa444153a90 con 0x7fa44406d960 2024-05-26T22:57:11.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.466+0000 7fa443e00640 1 -- 172.21.15.131:0/2794016256 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4440694e0 con 0x7fa44406d960 2024-05-26T22:57:11.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.467+0000 7fa43aa00640 1 -- 172.21.15.131:0/2794016256 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa43401a070 con 0x7fa44406d960 2024-05-26T22:57:11.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.468+0000 7fa43aa00640 1 --2- 172.21.15.131:0/2794016256 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa42406be30 0x7fa42406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:11.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.468+0000 7fa43aa00640 1 -- 172.21.15.131:0/2794016256 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5115+0+0 (secure 0 0 0) 0x7fa434089ea0 con 0x7fa44406d960 2024-05-26T22:57:11.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.469+0000 7fa442000640 1 --2- 172.21.15.131:0/2794016256 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa42406be30 0x7fa42406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:11.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.469+0000 7fa442000640 1 --2- 172.21.15.131:0/2794016256 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa42406be30 0x7fa42406e2f0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fa44415b8a0 tx=0x7fa430008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:57:11.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.473+0000 7fa43aa00640 1 -- 172.21.15.131:0/2794016256 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fa434058440 con 0x7fa44406d960 2024-05-26T22:57:11.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.670+0000 7fa443e00640 1 -- 172.21.15.131:0/2794016256 --> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7fa4440719c0 con 0x7fa42406be30 2024-05-26T22:57:11.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.671+0000 7fa43aa00640 1 -- 172.21.15.131:0/2794016256 <== mgr.4104 v2:172.21.15.131:6832/3484832947 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fa4440719c0 con 0x7fa42406be30 2024-05-26T22:57:11.671 INFO:teuthology.orchestra.run.smithi131.stderr:instructed osd(s) 7 to deep-scrub 2024-05-26T22:57:11.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.673+0000 7fa443e00640 1 -- 172.21.15.131:0/2794016256 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa42406be30 msgr2=0x7fa42406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:11.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.673+0000 7fa443e00640 1 --2- 172.21.15.131:0/2794016256 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa42406be30 0x7fa42406e2f0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fa44415b8a0 tx=0x7fa430008040 comp rx=0 tx=0).stop 2024-05-26T22:57:11.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.673+0000 7fa443e00640 1 -- 172.21.15.131:0/2794016256 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa44406d960 msgr2=0x7fa44414e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:11.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.673+0000 7fa443e00640 1 --2- 172.21.15.131:0/2794016256 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa44406d960 0x7fa44414e930 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7fa43400c4b0 tx=0x7fa43400c970 comp rx=0 tx=0).stop 2024-05-26T22:57:11.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.674+0000 7fa443e00640 1 -- 172.21.15.131:0/2794016256 shutdown_connections 2024-05-26T22:57:11.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.674+0000 7fa443e00640 1 --2- 172.21.15.131:0/2794016256 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa44415bfa0 0x7fa444152870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:11.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.674+0000 7fa443e00640 1 --2- 172.21.15.131:0/2794016256 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa42406be30 0x7fa42406e2f0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:11.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.674+0000 7fa443e00640 1 --2- 172.21.15.131:0/2794016256 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa44415dc40 0x7fa4440cefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:11.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.674+0000 7fa443e00640 1 --2- 172.21.15.131:0/2794016256 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa44406d960 0x7fa44414e930 unknown :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:11.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.674+0000 7fa443e00640 1 -- 172.21.15.131:0/2794016256 >> 172.21.15.131:0/2794016256 conn(0x7fa4440b5660 msgr2=0x7fa44415c880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:57:11.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.674+0000 7fa443e00640 1 -- 172.21.15.131:0/2794016256 shutdown_connections 2024-05-26T22:57:11.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.674+0000 7fa443e00640 1 -- 172.21.15.131:0/2794016256 wait complete. 2024-05-26T22:57:11.692 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-26T22:57:11.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.895+0000 7f3d94800640 1 Processor -- start 2024-05-26T22:57:11.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.896+0000 7f3d94800640 1 -- start start 2024-05-26T22:57:11.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.897+0000 7f3d94800640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f3d8c06ee20 0x7f3d8c0b3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:11.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.897+0000 7f3d94800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3d8c1569d0 0x7f3d8c0af790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:11.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.897+0000 7f3d94800640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d8c15cb20 con 0x7f3d8c0ad250 2024-05-26T22:57:11.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.897+0000 7f3d94800640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f3d8c15ce00 con 0x7f3d8c06ee20 2024-05-26T22:57:11.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.897+0000 7f3d94800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3d8c15d0e0 con 0x7f3d8c1569d0 2024-05-26T22:57:11.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.898+0000 7f3d92800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3d8c1569d0 0x7f3d8c0af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:11.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.898+0000 7f3d92800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3d8c1569d0 0x7f3d8c0af790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.131:47866/0 (socket says 172.21.15.131:47866) 2024-05-26T22:57:11.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.898+0000 7f3d92800640 1 -- 172.21.15.131:0/2811649983 learned_addr learned my addr 172.21.15.131:0/2811649983 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:57:11.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.898+0000 7f3d91e00640 1 --2- 172.21.15.131:0/2811649983 >> v2:172.21.15.187:3300/0 conn(0x7f3d8c06ee20 0x7f3d8c0b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:11.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.899+0000 7f3d92800640 1 -- 172.21.15.131:0/2811649983 >> v2:172.21.15.187:3300/0 conn(0x7f3d8c06ee20 msgr2=0x7f3d8c0b3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:57:11.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.899+0000 7f3d92800640 1 --2- 172.21.15.131:0/2811649983 >> v2:172.21.15.187:3300/0 conn(0x7f3d8c06ee20 0x7f3d8c0b3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:11.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.899+0000 7f3d92800640 1 -- 172.21.15.131:0/2811649983 >> v1:172.21.15.187:6789/0 conn(0x7f3d8c0ad250 legacy=0x7f3d8c1562c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:11.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.899+0000 7f3d92800640 1 -- 172.21.15.131:0/2811649983 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d8c15d3c0 con 0x7f3d8c1569d0 2024-05-26T22:57:11.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.900+0000 7f3d92800640 1 --2- 172.21.15.131:0/2811649983 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3d8c1569d0 0x7f3d8c0af790 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3d88009870 tx=0x7f3d8802a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9f5674fd4d19fa8 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:57:11.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.900+0000 7f3d90a00640 1 -- 172.21.15.131:0/2811649983 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d88037070 con 0x7f3d8c1569d0 2024-05-26T22:57:11.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.901+0000 7f3d90a00640 1 -- 172.21.15.131:0/2811649983 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d88032440 con 0x7f3d8c1569d0 2024-05-26T22:57:11.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.901+0000 7f3d90a00640 1 -- 172.21.15.131:0/2811649983 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d88032810 con 0x7f3d8c1569d0 2024-05-26T22:57:11.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.901+0000 7f3d94800640 1 -- 172.21.15.131:0/2811649983 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3d8c1569d0 msgr2=0x7f3d8c0af790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:11.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.901+0000 7f3d94800640 1 --2- 172.21.15.131:0/2811649983 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3d8c1569d0 0x7f3d8c0af790 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3d88009870 tx=0x7f3d8802a250 comp rx=0 tx=0).stop 2024-05-26T22:57:11.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.901+0000 7f3d94800640 1 -- 172.21.15.131:0/2811649983 shutdown_connections 2024-05-26T22:57:11.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.901+0000 7f3d94800640 1 --2- 172.21.15.131:0/2811649983 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3d8c1569d0 0x7f3d8c0af790 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:11.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.901+0000 7f3d94800640 1 --2- 172.21.15.131:0/2811649983 >> v2:172.21.15.187:3300/0 conn(0x7f3d8c06ee20 0x7f3d8c0b3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:11.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.901+0000 7f3d94800640 1 -- 172.21.15.131:0/2811649983 >> 172.21.15.131:0/2811649983 conn(0x7f3d8c0b1e90 msgr2=0x7f3d8c0b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:57:11.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.901+0000 7f3d94800640 1 -- 172.21.15.131:0/2811649983 shutdown_connections 2024-05-26T22:57:11.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.901+0000 7f3d94800640 1 -- 172.21.15.131:0/2811649983 wait complete. 2024-05-26T22:57:11.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.903+0000 7f3d94800640 1 Processor -- start 2024-05-26T22:57:11.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.903+0000 7f3d94800640 1 -- start start 2024-05-26T22:57:11.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.904+0000 7f3d94800640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3d8c06ee20 0x7f3d8c1f40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:11.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.904+0000 7f3d94800640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3d8c0ad250 0x7f3d8c09cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:11.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.904+0000 7f3d94800640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3d8c1569d0 0x7f3d8c09d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:11.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.904+0000 7f3d94800640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f3d8c15c900 con 0x7f3d8c0ad250 2024-05-26T22:57:11.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.904+0000 7f3d94800640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3d8c15c780 con 0x7f3d8c06ee20 2024-05-26T22:57:11.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.904+0000 7f3d94800640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3d8c06f200 con 0x7f3d8c1569d0 2024-05-26T22:57:11.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.904+0000 7f3d91400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3d8c0ad250 0x7f3d8c09cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:11.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.904+0000 7f3d91400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3d8c0ad250 0x7f3d8c09cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57444/0 (socket says 172.21.15.131:57444) 2024-05-26T22:57:11.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.904+0000 7f3d91400640 1 -- 172.21.15.131:0/1851583299 learned_addr learned my addr 172.21.15.131:0/1851583299 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:57:11.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.905+0000 7f3d91400640 1 -- 172.21.15.131:0/1851583299 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3d8c1569d0 msgr2=0x7f3d8c09d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:11.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.905+0000 7f3d91400640 1 --2- 172.21.15.131:0/1851583299 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3d8c1569d0 0x7f3d8c09d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:11.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.905+0000 7f3d91400640 1 -- 172.21.15.131:0/1851583299 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3d8c06ee20 msgr2=0x7f3d8c1f40e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:11.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.905+0000 7f3d91400640 1 --2- 172.21.15.131:0/1851583299 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3d8c06ee20 0x7f3d8c1f40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:11.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.905+0000 7f3d91400640 1 -- 172.21.15.131:0/1851583299 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d8c1440f0 con 0x7f3d8c0ad250 2024-05-26T22:57:11.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.905+0000 7f3d91400640 1 --2- 172.21.15.131:0/1851583299 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3d8c0ad250 0x7f3d8c09cdd0 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c00c440 tx=0x7f3d7c001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:57:11.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.905+0000 7f3d82000640 1 -- 172.21.15.131:0/1851583299 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d7c0052d0 con 0x7f3d8c0ad250 2024-05-26T22:57:11.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.905+0000 7f3d82000640 1 -- 172.21.15.131:0/1851583299 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d7c005470 con 0x7f3d8c0ad250 2024-05-26T22:57:11.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.905+0000 7f3d82000640 1 -- 172.21.15.131:0/1851583299 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d7c007260 con 0x7f3d8c0ad250 2024-05-26T22:57:11.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.905+0000 7f3d94800640 1 -- 172.21.15.131:0/1851583299 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d8c1443d0 con 0x7f3d8c0ad250 2024-05-26T22:57:11.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.906+0000 7f3d94800640 1 -- 172.21.15.131:0/1851583299 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d8c1f4620 con 0x7f3d8c0ad250 2024-05-26T22:57:11.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.907+0000 7f3d94800640 1 -- 172.21.15.131:0/1851583299 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d8c06dbc0 con 0x7f3d8c0ad250 2024-05-26T22:57:11.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.908+0000 7f3d82000640 1 -- 172.21.15.131:0/1851583299 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3d7c007400 con 0x7f3d8c0ad250 2024-05-26T22:57:11.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.908+0000 7f3d82000640 1 --2- 172.21.15.131:0/1851583299 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f3d7006be30 0x7f3d7006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:11.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.909+0000 7f3d82000640 1 -- 172.21.15.131:0/1851583299 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5115+0+0 (secure 0 0 0) 0x7f3d7c08a0c0 con 0x7f3d8c0ad250 2024-05-26T22:57:11.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.909+0000 7f3d91e00640 1 --2- 172.21.15.131:0/1851583299 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f3d7006be30 0x7f3d7006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:11.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.910+0000 7f3d91e00640 1 --2- 172.21.15.131:0/1851583299 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f3d7006be30 0x7f3d7006e2f0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f3d840070b0 tx=0x7f3d84007040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:57:11.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:11.913+0000 7f3d82000640 1 -- 172.21.15.131:0/1851583299 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f3d8c06dbc0 con 0x7f3d8c0ad250 2024-05-26T22:57:12.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:12.106+0000 7f3d94800640 1 -- 172.21.15.131:0/1851583299 --> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3d8c0ae2d0 con 0x7f3d7006be30 2024-05-26T22:57:12.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:12.108+0000 7f3d82000640 1 -- 172.21.15.131:0/1851583299 <== mgr.4104 v2:172.21.15.131:6832/3484832947 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13370 (secure 0 0 0) 0x7f3d8c0ae2d0 con 0x7f3d7006be30 2024-05-26T22:57:12.108 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:57:12.108 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-05-26T22:57:12.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:12.111+0000 7f3d94800640 1 -- 172.21.15.131:0/1851583299 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f3d7006be30 msgr2=0x7f3d7006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:12.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:12.111+0000 7f3d94800640 1 --2- 172.21.15.131:0/1851583299 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f3d7006be30 0x7f3d7006e2f0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f3d840070b0 tx=0x7f3d84007040 comp rx=0 tx=0).stop 2024-05-26T22:57:12.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:12.111+0000 7f3d94800640 1 -- 172.21.15.131:0/1851583299 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3d8c0ad250 msgr2=0x7f3d8c09cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:12.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:12.111+0000 7f3d94800640 1 --2- 172.21.15.131:0/1851583299 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3d8c0ad250 0x7f3d8c09cdd0 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c00c440 tx=0x7f3d7c001e70 comp rx=0 tx=0).stop 2024-05-26T22:57:12.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:12.111+0000 7f3d94800640 1 -- 172.21.15.131:0/1851583299 shutdown_connections 2024-05-26T22:57:12.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:12.111+0000 7f3d94800640 1 --2- 172.21.15.131:0/1851583299 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3d8c1569d0 0x7f3d8c09d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:12.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:12.111+0000 7f3d94800640 1 --2- 172.21.15.131:0/1851583299 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f3d7006be30 0x7f3d7006e2f0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:12.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:12.111+0000 7f3d94800640 1 --2- 172.21.15.131:0/1851583299 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3d8c0ad250 0x7f3d8c09cdd0 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:12.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:12.111+0000 7f3d94800640 1 --2- 172.21.15.131:0/1851583299 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3d8c06ee20 0x7f3d8c1f40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:12.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:12.111+0000 7f3d94800640 1 -- 172.21.15.131:0/1851583299 >> 172.21.15.131:0/1851583299 conn(0x7f3d8c0b1e90 msgr2=0x7f3d8c06d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:57:12.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:12.111+0000 7f3d94800640 1 -- 172.21.15.131:0/1851583299 shutdown_connections 2024-05-26T22:57:12.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:12.111+0000 7f3d94800640 1 -- 172.21.15.131:0/1851583299 wait complete. 2024-05-26T22:57:12.130 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":373,"stamp":"2024-05-26T22:57:10.561961+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":290496,"kb_used_data":11912,"kb_used_omap":12,"kb_used_meta":278451,"kb_avail":754684224,"statfs":{"total":773094113280,"available":772796645376,"internally_reserved":0,"allocated":12197888,"data_stored":5972184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":285134385},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.761933"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":338,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-05-26T22:56:59.023584+0000","last_change":"2024-05-26T22:48:12.068917+0000","last_active":"2024-05-26T22:56:59.023584+0000","last_peered":"2024-05-26T22:56:59.023584+0000","last_clean":"2024-05-26T22:56:59.023584+0000","last_became_active":"2024-05-26T22:48:12.065934+0000","last_became_peered":"2024-05-26T22:48:12.065934+0000","last_unstale":"2024-05-26T22:56:59.023584+0000","last_undegraded":"2024-05-26T22:56:59.023584+0000","last_fullsized":"2024-05-26T22:56:59.023584+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-26T22:48:10.628122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-26T22:48:10.628122+0000","last_clean_scrub_stamp":"2024-05-26T22:48:10.628122+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T00:14:58.895911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574959,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37268,"kb_used_data":1924,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334572,"statfs":{"total":96636764160,"available":96598601728,"internally_reserved":0,"allocated":1970176,"data_stored":1189299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574959,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36688,"kb_used_data":1344,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335152,"statfs":{"total":96636764160,"available":96599195648,"internally_reserved":0,"allocated":1376256,"data_stored":598931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574960,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32528,"kb_used_data":1344,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339312,"statfs":{"total":96636764160,"available":96603455488,"internally_reserved":0,"allocated":1376256,"data_stored":598931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574959,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36684,"kb_used_data":1344,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335156,"statfs":{"total":96636764160,"available":96599199744,"internally_reserved":0,"allocated":1376256,"data_stored":598931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574959,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36684,"kb_used_data":1344,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335156,"statfs":{"total":96636764160,"available":96599199744,"internally_reserved":0,"allocated":1376256,"data_stored":598931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574959,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36684,"kb_used_data":1344,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335156,"statfs":{"total":96636764160,"available":96599199744,"internally_reserved":0,"allocated":1376256,"data_stored":598931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574959,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36692,"kb_used_data":1344,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335148,"statfs":{"total":96636764160,"available":96599191552,"internally_reserved":0,"allocated":1376256,"data_stored":598931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574959,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37268,"kb_used_data":1924,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334572,"statfs":{"total":96636764160,"available":96598601728,"internally_reserved":0,"allocated":1970176,"data_stored":1189299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-26T22:57:12.133 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-05-26T22:48:10.628122+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=26, tm_hour=22, tm_min=48, tm_sec=10, tm_wday=6, tm_yday=147, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=26, tm_hour=22, tm_min=57, tm_sec=5, tm_wday=6, tm_yday=147, tm_isdst=0) 2024-05-26T22:57:12.133 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-05-26T22:57:16.243 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-26T22:57:16.252 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-26T22:57:32.134 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-26T22:57:32.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.345+0000 7fb99b200640 1 Processor -- start 2024-05-26T22:57:32.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.345+0000 7fb99b200640 1 -- start start 2024-05-26T22:57:32.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.347+0000 7fb99b200640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fb99c0cc7f0 0x7fb99c0ccc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:32.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.347+0000 7fb99b200640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb99c160620 con 0x7fb99c156750 2024-05-26T22:57:32.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.347+0000 7fb99b200640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb99c160900 con 0x7fb99c06e8c0 2024-05-26T22:57:32.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.347+0000 7fb99b200640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fb99c15d770 con 0x7fb99c0cc7f0 2024-05-26T22:57:32.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.347+0000 7fb999e00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fb99c156750 0x7fb99c15fd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:58630/0 (socket says 172.21.15.131:58630) 2024-05-26T22:57:32.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.347+0000 7fb999e00640 1 -- 172.21.15.131:0/653219199 learned_addr learned my addr 172.21.15.131:0/653219199 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:57:32.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.347+0000 7fb99a800640 1 --2- 172.21.15.131:0/653219199 >> v2:172.21.15.131:3300/0 conn(0x7fb99c0cc7f0 0x7fb99c0ccc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:32.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.348+0000 7fb999400640 1 -- 172.21.15.131:0/653219199 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1202608119 0 0) 0x7fb99c160620 con 0x7fb99c156750 2024-05-26T22:57:32.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.348+0000 7fb999400640 1 -- 172.21.15.131:0/653219199 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb980003520 con 0x7fb99c156750 2024-05-26T22:57:32.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.348+0000 7fb99a800640 1 -- 172.21.15.131:0/653219199 >> v1:172.21.15.187:6789/0 conn(0x7fb99c06e8c0 legacy=0x7fb99c15d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:32.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.348+0000 7fb99a800640 1 -- 172.21.15.131:0/653219199 >> v1:172.21.15.131:6789/0 conn(0x7fb99c156750 legacy=0x7fb99c15fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:32.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.348+0000 7fb99a800640 1 -- 172.21.15.131:0/653219199 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb99c160be0 con 0x7fb99c0cc7f0 2024-05-26T22:57:32.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.349+0000 7fb99a800640 1 --2- 172.21.15.131:0/653219199 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb99c0cc7f0 0x7fb99c0ccc60 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7fb994009d90 tx=0x7fb99402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ec983eb84c205ce server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:57:32.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.349+0000 7fb999400640 1 -- 172.21.15.131:0/653219199 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb994007740 con 0x7fb99c0cc7f0 2024-05-26T22:57:32.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.349+0000 7fb999400640 1 -- 172.21.15.131:0/653219199 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9940078e0 con 0x7fb99c0cc7f0 2024-05-26T22:57:32.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.349+0000 7fb999400640 1 -- 172.21.15.131:0/653219199 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb994007c90 con 0x7fb99c0cc7f0 2024-05-26T22:57:32.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.350+0000 7fb99b200640 1 -- 172.21.15.131:0/653219199 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb99c0cc7f0 msgr2=0x7fb99c0ccc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:32.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.350+0000 7fb99b200640 1 --2- 172.21.15.131:0/653219199 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb99c0cc7f0 0x7fb99c0ccc60 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7fb994009d90 tx=0x7fb99402a6f0 comp rx=0 tx=0).stop 2024-05-26T22:57:32.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.350+0000 7fb99b200640 1 -- 172.21.15.131:0/653219199 shutdown_connections 2024-05-26T22:57:32.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.350+0000 7fb99b200640 1 --2- 172.21.15.131:0/653219199 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb99c0cc7f0 0x7fb99c0ccc60 unknown :-1 s=CLOSED pgs=1693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:32.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.350+0000 7fb99b200640 1 -- 172.21.15.131:0/653219199 >> 172.21.15.131:0/653219199 conn(0x7fb99c0cff40 msgr2=0x7fb99c0d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:57:32.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.350+0000 7fb99b200640 1 -- 172.21.15.131:0/653219199 shutdown_connections 2024-05-26T22:57:32.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.350+0000 7fb99b200640 1 -- 172.21.15.131:0/653219199 wait complete. 2024-05-26T22:57:32.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.352+0000 7fb99b200640 1 Processor -- start 2024-05-26T22:57:32.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.352+0000 7fb99b200640 1 -- start start 2024-05-26T22:57:32.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.353+0000 7fb99b200640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb99c06e8c0 0x7fb99c131fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:32.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.353+0000 7fb99b200640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb99c0cc7f0 0x7fb99c155680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:32.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.353+0000 7fb99b200640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb99c156750 0x7fb99c130570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:32.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.353+0000 7fb99b200640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fb99c15cbd0 con 0x7fb99c06e8c0 2024-05-26T22:57:32.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.353+0000 7fb99b200640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb99c0cd1a0 con 0x7fb99c156750 2024-05-26T22:57:32.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.353+0000 7fb99b200640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb99c156b30 con 0x7fb99c0cc7f0 2024-05-26T22:57:32.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.353+0000 7fb999e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb99c06e8c0 0x7fb99c131fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:32.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.353+0000 7fb999e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb99c06e8c0 0x7fb99c131fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57564/0 (socket says 172.21.15.131:57564) 2024-05-26T22:57:32.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.353+0000 7fb999e00640 1 -- 172.21.15.131:0/3845594737 learned_addr learned my addr 172.21.15.131:0/3845594737 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:57:32.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.354+0000 7fb999e00640 1 -- 172.21.15.131:0/3845594737 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb99c0cc7f0 msgr2=0x7fb99c155680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:32.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.354+0000 7fb999e00640 1 --2- 172.21.15.131:0/3845594737 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb99c0cc7f0 0x7fb99c155680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:32.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.354+0000 7fb999e00640 1 -- 172.21.15.131:0/3845594737 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb99c156750 msgr2=0x7fb99c130570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:32.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.354+0000 7fb999e00640 1 --2- 172.21.15.131:0/3845594737 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb99c156750 0x7fb99c130570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:32.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.354+0000 7fb999e00640 1 -- 172.21.15.131:0/3845594737 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb99c130f10 con 0x7fb99c06e8c0 2024-05-26T22:57:32.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.354+0000 7fb999e00640 1 --2- 172.21.15.131:0/3845594737 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb99c06e8c0 0x7fb99c131fb0 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7fb98c00c410 tx=0x7fb98c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:57:32.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.354+0000 7fb992a00640 1 -- 172.21.15.131:0/3845594737 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb98c017070 con 0x7fb99c06e8c0 2024-05-26T22:57:32.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.354+0000 7fb992a00640 1 -- 172.21.15.131:0/3845594737 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb98c005ce0 con 0x7fb99c06e8c0 2024-05-26T22:57:32.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.354+0000 7fb992a00640 1 -- 172.21.15.131:0/3845594737 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb98c0072f0 con 0x7fb99c06e8c0 2024-05-26T22:57:32.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.354+0000 7fb99b200640 1 -- 172.21.15.131:0/3845594737 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb99c1311f0 con 0x7fb99c06e8c0 2024-05-26T22:57:32.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.354+0000 7fb99b200640 1 -- 172.21.15.131:0/3845594737 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb99c1317b0 con 0x7fb99c06e8c0 2024-05-26T22:57:32.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.356+0000 7fb99b200640 1 -- 172.21.15.131:0/3845594737 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb99c0694e0 con 0x7fb99c06e8c0 2024-05-26T22:57:32.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.356+0000 7fb992a00640 1 -- 172.21.15.131:0/3845594737 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb98c01a070 con 0x7fb99c06e8c0 2024-05-26T22:57:32.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.357+0000 7fb992a00640 1 --2- 172.21.15.131:0/3845594737 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb97806be30 0x7fb97806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:32.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.357+0000 7fb992a00640 1 -- 172.21.15.131:0/3845594737 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5115+0+0 (secure 0 0 0) 0x7fb98c08a0f0 con 0x7fb99c06e8c0 2024-05-26T22:57:32.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.358+0000 7fb991400640 1 --2- 172.21.15.131:0/3845594737 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb97806be30 0x7fb97806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:32.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.358+0000 7fb991400640 1 --2- 172.21.15.131:0/3845594737 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb97806be30 0x7fb97806e2f0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fb984000f60 tx=0x7fb9840079d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:57:32.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.362+0000 7fb992a00640 1 -- 172.21.15.131:0/3845594737 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fb98c058d10 con 0x7fb99c06e8c0 2024-05-26T22:57:32.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.557+0000 7fb99b200640 1 -- 172.21.15.131:0/3845594737 --> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb99c0ce070 con 0x7fb97806be30 2024-05-26T22:57:32.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.558+0000 7fb992a00640 1 -- 172.21.15.131:0/3845594737 <== mgr.4104 v2:172.21.15.131:6832/3484832947 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13379 (secure 0 0 0) 0x7fb99c0ce070 con 0x7fb97806be30 2024-05-26T22:57:32.559 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:57:32.559 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-05-26T22:57:32.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.561+0000 7fb99b200640 1 -- 172.21.15.131:0/3845594737 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb97806be30 msgr2=0x7fb97806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:32.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.561+0000 7fb99b200640 1 --2- 172.21.15.131:0/3845594737 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb97806be30 0x7fb97806e2f0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fb984000f60 tx=0x7fb9840079d0 comp rx=0 tx=0).stop 2024-05-26T22:57:32.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.561+0000 7fb99b200640 1 -- 172.21.15.131:0/3845594737 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb99c06e8c0 msgr2=0x7fb99c131fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:32.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.561+0000 7fb99b200640 1 --2- 172.21.15.131:0/3845594737 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb99c06e8c0 0x7fb99c131fb0 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7fb98c00c410 tx=0x7fb98c00c8d0 comp rx=0 tx=0).stop 2024-05-26T22:57:32.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.561+0000 7fb99b200640 1 -- 172.21.15.131:0/3845594737 shutdown_connections 2024-05-26T22:57:32.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.561+0000 7fb99b200640 1 --2- 172.21.15.131:0/3845594737 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb99c0cc7f0 0x7fb99c155680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:32.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.561+0000 7fb99b200640 1 --2- 172.21.15.131:0/3845594737 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fb97806be30 0x7fb97806e2f0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:32.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.562+0000 7fb99b200640 1 --2- 172.21.15.131:0/3845594737 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb99c156750 0x7fb99c130570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:32.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.562+0000 7fb99b200640 1 --2- 172.21.15.131:0/3845594737 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb99c06e8c0 0x7fb99c131fb0 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:32.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.562+0000 7fb99b200640 1 -- 172.21.15.131:0/3845594737 >> 172.21.15.131:0/3845594737 conn(0x7fb99c0cff40 msgr2=0x7fb99c15e7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:57:32.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.562+0000 7fb99b200640 1 -- 172.21.15.131:0/3845594737 shutdown_connections 2024-05-26T22:57:32.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.562+0000 7fb99b200640 1 -- 172.21.15.131:0/3845594737 wait complete. 2024-05-26T22:57:32.581 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":383,"stamp":"2024-05-26T22:57:30.567485+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":290480,"kb_used_data":11912,"kb_used_omap":12,"kb_used_meta":278451,"kb_avail":754684240,"statfs":{"total":773094113280,"available":772796661760,"internally_reserved":0,"allocated":12197888,"data_stored":5972184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":285134385},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003381"},"pg_stats":[{"pgid":"1.0","version":"110'119","reported_seq":349,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-05-26T22:57:11.853470+0000","last_change":"2024-05-26T22:57:11.853545+0000","last_active":"2024-05-26T22:57:11.853470+0000","last_peered":"2024-05-26T22:57:11.853470+0000","last_clean":"2024-05-26T22:57:11.853470+0000","last_became_active":"2024-05-26T22:48:12.065934+0000","last_became_peered":"2024-05-26T22:48:12.065934+0000","last_unstale":"2024-05-26T22:57:11.853470+0000","last_undegraded":"2024-05-26T22:57:11.853470+0000","last_fullsized":"2024-05-26T22:57:11.853470+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'119","last_scrub_stamp":"2024-05-26T22:57:11.853279+0000","last_deep_scrub":"110'119","last_deep_scrub_stamp":"2024-05-26T22:57:11.853279+0000","last_clean_scrub_stamp":"2024-05-26T22:57:11.853279+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:52:25.635559+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574963,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37260,"kb_used_data":1924,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334580,"statfs":{"total":96636764160,"available":96598609920,"internally_reserved":0,"allocated":1970176,"data_stored":1189299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574963,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36688,"kb_used_data":1344,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335152,"statfs":{"total":96636764160,"available":96599195648,"internally_reserved":0,"allocated":1376256,"data_stored":598931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574964,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32528,"kb_used_data":1344,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339312,"statfs":{"total":96636764160,"available":96603455488,"internally_reserved":0,"allocated":1376256,"data_stored":598931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574963,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36684,"kb_used_data":1344,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335156,"statfs":{"total":96636764160,"available":96599199744,"internally_reserved":0,"allocated":1376256,"data_stored":598931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574963,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36684,"kb_used_data":1344,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335156,"statfs":{"total":96636764160,"available":96599199744,"internally_reserved":0,"allocated":1376256,"data_stored":598931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574963,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36684,"kb_used_data":1344,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335156,"statfs":{"total":96636764160,"available":96599199744,"internally_reserved":0,"allocated":1376256,"data_stored":598931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574963,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36692,"kb_used_data":1344,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335148,"statfs":{"total":96636764160,"available":96599191552,"internally_reserved":0,"allocated":1376256,"data_stored":598931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574963,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37260,"kb_used_data":1924,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334580,"statfs":{"total":96636764160,"available":96598609920,"internally_reserved":0,"allocated":1970176,"data_stored":1189299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-26T22:57:32.582 DEBUG:teuthology.orchestra.run.smithi131:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-05-26T22:57:32.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.776+0000 7f5f92400640 1 Processor -- start 2024-05-26T22:57:32.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.776+0000 7f5f92400640 1 -- start start 2024-05-26T22:57:32.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.778+0000 7f5f92400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7f5f8c072e90 0x7f5f8c072540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:32.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.778+0000 7f5f92400640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f8c0af8e0 con 0x7f5f8c06f180 2024-05-26T22:57:32.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.778+0000 7f5f92400640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f8c0cf200 con 0x7f5f8c0d0560 2024-05-26T22:57:32.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.778+0000 7f5f92400640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7f5f8c0cfb30 con 0x7f5f8c072e90 2024-05-26T22:57:32.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.778+0000 7f5f8b400640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f5f8c06f180 0x7f5f8c0cfe50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:58646/0 (socket says 172.21.15.131:58646) 2024-05-26T22:57:32.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.778+0000 7f5f8b400640 1 -- 172.21.15.131:0/4294478513 learned_addr learned my addr 172.21.15.131:0/4294478513 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:57:32.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.778+0000 7f5f8be00640 1 --2- 172.21.15.131:0/4294478513 >> v2:172.21.15.131:3300/0 conn(0x7f5f8c072e90 0x7f5f8c072540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:32.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.779+0000 7f5f8a000640 1 -- 172.21.15.131:0/4294478513 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1906539014 0 0) 0x7f5f8c0af8e0 con 0x7f5f8c06f180 2024-05-26T22:57:32.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.779+0000 7f5f8a000640 1 -- 172.21.15.131:0/4294478513 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5f70003520 con 0x7f5f8c06f180 2024-05-26T22:57:32.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.779+0000 7f5f8be00640 1 -- 172.21.15.131:0/4294478513 >> v1:172.21.15.187:6789/0 conn(0x7f5f8c0d0560 legacy=0x7f5f8c0715a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:32.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.779+0000 7f5f8be00640 1 -- 172.21.15.131:0/4294478513 >> v1:172.21.15.131:6789/0 conn(0x7f5f8c06f180 legacy=0x7f5f8c0cfe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:32.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.779+0000 7f5f8a000640 1 -- 172.21.15.131:0/4294478513 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 638465641 0 0) 0x7f5f70003520 con 0x7f5f8c06f180 2024-05-26T22:57:32.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.779+0000 7f5f8be00640 1 -- 172.21.15.131:0/4294478513 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f8c0cf4e0 con 0x7f5f8c072e90 2024-05-26T22:57:32.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.780+0000 7f5f8be00640 1 --2- 172.21.15.131:0/4294478513 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5f8c072e90 0x7f5f8c072540 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f5f84009d90 tx=0x7f5f8402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c768228d6dcbc26f server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:57:32.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.781+0000 7f5f8a000640 1 -- 172.21.15.131:0/4294478513 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f7c001670 con 0x7f5f8c072e90 2024-05-26T22:57:32.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.781+0000 7f5f8a000640 1 -- 172.21.15.131:0/4294478513 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f84007540 con 0x7f5f8c072e90 2024-05-26T22:57:32.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.781+0000 7f5f8a000640 1 -- 172.21.15.131:0/4294478513 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f84007910 con 0x7f5f8c072e90 2024-05-26T22:57:32.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.781+0000 7f5f92400640 1 -- 172.21.15.131:0/4294478513 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5f8c072e90 msgr2=0x7f5f8c072540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:32.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.781+0000 7f5f92400640 1 --2- 172.21.15.131:0/4294478513 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5f8c072e90 0x7f5f8c072540 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f5f84009d90 tx=0x7f5f8402a680 comp rx=0 tx=0).stop 2024-05-26T22:57:32.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.781+0000 7f5f92400640 1 -- 172.21.15.131:0/4294478513 shutdown_connections 2024-05-26T22:57:32.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.781+0000 7f5f92400640 1 --2- 172.21.15.131:0/4294478513 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5f8c072e90 0x7f5f8c072540 unknown :-1 s=CLOSED pgs=1696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:32.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.781+0000 7f5f92400640 1 -- 172.21.15.131:0/4294478513 >> 172.21.15.131:0/4294478513 conn(0x7f5f8c06bc90 msgr2=0x7f5f8c0b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:57:32.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.781+0000 7f5f92400640 1 -- 172.21.15.131:0/4294478513 shutdown_connections 2024-05-26T22:57:32.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.782+0000 7f5f92400640 1 -- 172.21.15.131:0/4294478513 wait complete. 2024-05-26T22:57:32.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.783+0000 7f5f92400640 1 Processor -- start 2024-05-26T22:57:32.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.784+0000 7f5f92400640 1 -- start start 2024-05-26T22:57:32.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.784+0000 7f5f92400640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5f8c06f180 0x7f5f8c09be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:32.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.784+0000 7f5f92400640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5f8c072e90 0x7f5f8c0ce900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:32.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.784+0000 7f5f92400640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5f8c0d0560 0x7f5f8c0b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:32.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.784+0000 7f5f92400640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f5f8c06f6e0 con 0x7f5f8c0d0560 2024-05-26T22:57:32.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.784+0000 7f5f92400640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5f8c06f560 con 0x7f5f8c072e90 2024-05-26T22:57:32.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.784+0000 7f5f92400640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f5f8c161ff0 con 0x7f5f8c06f180 2024-05-26T22:57:32.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.785+0000 7f5f8be00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5f8c0d0560 0x7f5f8c0b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:32.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.785+0000 7f5f8be00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5f8c0d0560 0x7f5f8c0b51b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57592/0 (socket says 172.21.15.131:57592) 2024-05-26T22:57:32.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.785+0000 7f5f8be00640 1 -- 172.21.15.131:0/3071265858 learned_addr learned my addr 172.21.15.131:0/3071265858 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:57:32.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.785+0000 7f5f8aa00640 1 --2- 172.21.15.131:0/3071265858 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5f8c072e90 0x7f5f8c0ce900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:32.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.785+0000 7f5f8b400640 1 --2- 172.21.15.131:0/3071265858 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5f8c06f180 0x7f5f8c09be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:32.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.785+0000 7f5f8be00640 1 -- 172.21.15.131:0/3071265858 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5f8c06f180 msgr2=0x7f5f8c09be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:32.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.785+0000 7f5f8be00640 1 --2- 172.21.15.131:0/3071265858 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5f8c06f180 0x7f5f8c09be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:32.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.785+0000 7f5f8be00640 1 -- 172.21.15.131:0/3071265858 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5f8c072e90 msgr2=0x7f5f8c0ce900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:32.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.785+0000 7f5f8be00640 1 --2- 172.21.15.131:0/3071265858 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5f8c072e90 0x7f5f8c0ce900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:32.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.785+0000 7f5f8be00640 1 -- 172.21.15.131:0/3071265858 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f8c09adc0 con 0x7f5f8c0d0560 2024-05-26T22:57:32.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.785+0000 7f5f8b400640 1 --2- 172.21.15.131:0/3071265858 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5f8c06f180 0x7f5f8c09be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-26T22:57:32.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.786+0000 7f5f8be00640 1 --2- 172.21.15.131:0/3071265858 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5f8c0d0560 0x7f5f8c0b51b0 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f5f84009eb0 tx=0x7f5f84004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:57:32.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.786+0000 7f5f83400640 1 -- 172.21.15.131:0/3071265858 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f84040070 con 0x7f5f8c0d0560 2024-05-26T22:57:32.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.786+0000 7f5f83400640 1 -- 172.21.15.131:0/3071265858 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f84031bc0 con 0x7f5f8c0d0560 2024-05-26T22:57:32.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.786+0000 7f5f83400640 1 -- 172.21.15.131:0/3071265858 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f84009450 con 0x7f5f8c0d0560 2024-05-26T22:57:32.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.786+0000 7f5f92400640 1 -- 172.21.15.131:0/3071265858 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f8c09b0a0 con 0x7f5f8c0d0560 2024-05-26T22:57:32.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.786+0000 7f5f92400640 1 -- 172.21.15.131:0/3071265858 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f8c09b940 con 0x7f5f8c0d0560 2024-05-26T22:57:32.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.787+0000 7f5f92400640 1 -- 172.21.15.131:0/3071265858 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f8c09b270 con 0x7f5f8c0d0560 2024-05-26T22:57:32.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.788+0000 7f5f83400640 1 -- 172.21.15.131:0/3071265858 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5f84039040 con 0x7f5f8c0d0560 2024-05-26T22:57:32.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.789+0000 7f5f83400640 1 --2- 172.21.15.131:0/3071265858 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f5f6c06be30 0x7f5f6c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:32.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.789+0000 7f5f83400640 1 -- 172.21.15.131:0/3071265858 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5115+0+0 (secure 0 0 0) 0x7f5f840ac130 con 0x7f5f8c0d0560 2024-05-26T22:57:32.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.789+0000 7f5f8b400640 1 --2- 172.21.15.131:0/3071265858 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f5f6c06be30 0x7f5f6c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:32.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.790+0000 7f5f8b400640 1 --2- 172.21.15.131:0/3071265858 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f5f6c06be30 0x7f5f6c06e2f0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c001380 tx=0x7f5f7c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:57:32.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.793+0000 7f5f83400640 1 -- 172.21.15.131:0/3071265858 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7f5f84038350 con 0x7f5f8c0d0560 2024-05-26T22:57:32.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.991+0000 7f5f92400640 1 -- 172.21.15.131:0/3071265858 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7f5f8c074e90 con 0x7f5f8c0d0560 2024-05-26T22:57:32.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:32.998+0000 7f5f83400640 1 -- 172.21.15.131:0/3071265858 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v67)=0 v67) ==== 123+0+0 (secure 0 0 0) 0x7f5f8407ac90 con 0x7f5f8c0d0560 2024-05-26T22:57:32.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:33.000+0000 7f5f83400640 1 -- 172.21.15.131:0/3071265858 <== mon.0 v2:172.21.15.131:3300/0 8 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f5f8407a590 con 0x7f5f8c0d0560 2024-05-26T22:57:33.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:33.001+0000 7f5f92400640 1 -- 172.21.15.131:0/3071265858 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f5f6c06be30 msgr2=0x7f5f6c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:33.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:33.001+0000 7f5f92400640 1 --2- 172.21.15.131:0/3071265858 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f5f6c06be30 0x7f5f6c06e2f0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c001380 tx=0x7f5f7c009040 comp rx=0 tx=0).stop 2024-05-26T22:57:33.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:33.001+0000 7f5f92400640 1 -- 172.21.15.131:0/3071265858 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5f8c0d0560 msgr2=0x7f5f8c0b51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:33.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:33.001+0000 7f5f92400640 1 --2- 172.21.15.131:0/3071265858 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5f8c0d0560 0x7f5f8c0b51b0 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f5f84009eb0 tx=0x7f5f84004a40 comp rx=0 tx=0).stop 2024-05-26T22:57:33.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:33.001+0000 7f5f92400640 1 -- 172.21.15.131:0/3071265858 shutdown_connections 2024-05-26T22:57:33.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:33.001+0000 7f5f92400640 1 --2- 172.21.15.131:0/3071265858 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7f5f6c06be30 0x7f5f6c06e2f0 secure :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c001380 tx=0x7f5f7c009040 comp rx=0 tx=0).stop 2024-05-26T22:57:33.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:33.001+0000 7f5f92400640 1 --2- 172.21.15.131:0/3071265858 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5f8c0d0560 0x7f5f8c0b51b0 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:33.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:33.001+0000 7f5f92400640 1 --2- 172.21.15.131:0/3071265858 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5f8c072e90 0x7f5f8c0ce900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:33.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:33.001+0000 7f5f92400640 1 --2- 172.21.15.131:0/3071265858 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5f8c06f180 0x7f5f8c09be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:33.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:33.001+0000 7f5f92400640 1 -- 172.21.15.131:0/3071265858 >> 172.21.15.131:0/3071265858 conn(0x7f5f8c06bc90 msgr2=0x7f5f8c160410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:57:33.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:33.002+0000 7f5f92400640 1 -- 172.21.15.131:0/3071265858 shutdown_connections 2024-05-26T22:57:33.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:33.002+0000 7f5f92400640 1 -- 172.21.15.131:0/3071265858 wait complete. 2024-05-26T22:57:33.022 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-26T22:57:33.023 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-26T22:57:33.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:33.225+0000 7fa88be00640 1 Processor -- start 2024-05-26T22:57:33.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:33.225+0000 7fa88be00640 1 -- start start 2024-05-26T22:57:33.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:33.226+0000 7fa88be00640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fa884155840 0x7fa884071930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:33.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:33.227+0000 7fa88be00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa88415c3d0 0x7fa88406ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:33.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:33.227+0000 7fa88be00640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8840722a0 con 0x7fa88406f490 2024-05-26T22:57:33.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:33.227+0000 7fa88be00640 1 -- --> v2:172.21.15.131:3300/0 -- mon_getmap magic: 0 -- 0x7fa884072580 con 0x7fa884155840 2024-05-26T22:57:33.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:33.227+0000 7fa88be00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa884072860 con 0x7fa88415c3d0 2024-05-26T22:57:33.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:33.227+0000 7fa889e00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fa88406f490 0x7fa88415ec80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:58648/0 (socket says 172.21.15.131:58648) 2024-05-26T22:57:33.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:33.227+0000 7fa889400640 1 --2- >> v2:172.21.15.131:3300/0 conn(0x7fa884155840 0x7fa884071930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:33.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:33.227+0000 7fa889e00640 1 -- 172.21.15.131:0/1497237537 learned_addr learned my addr 172.21.15.131:0/1497237537 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-05-26T22:57:33.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:33.227+0000 7fa888a00640 1 --2- 172.21.15.131:0/1497237537 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa88415c3d0 0x7fa88406ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:33.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:33.227+0000 7fa883e00640 1 -- 172.21.15.131:0/1497237537 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 545796240 0 0) 0x7fa8840722a0 con 0x7fa88406f490 2024-05-26T22:57:33.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:33.227+0000 7fa883e00640 1 -- 172.21.15.131:0/1497237537 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa86c003520 con 0x7fa88406f490 2024-05-26T22:57:33.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:33.227+0000 7fa889400640 1 -- 172.21.15.131:0/1497237537 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa88415c3d0 msgr2=0x7fa88406ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-26T22:57:33.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:33.227+0000 7fa889400640 1 --2- 172.21.15.131:0/1497237537 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa88415c3d0 0x7fa88406ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:33.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:33.228+0000 7fa889400640 1 -- 172.21.15.131:0/1497237537 >> v1:172.21.15.131:6789/0 conn(0x7fa88406f490 legacy=0x7fa88415ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:33.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:33.228+0000 7fa889400640 1 -- 172.21.15.131:0/1497237537 --> v2:172.21.15.131:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa884072b40 con 0x7fa884155840 2024-05-26T22:57:33.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:33.229+0000 7fa889400640 1 --2- 172.21.15.131:0/1497237537 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa884155840 0x7fa884071930 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7fa8780097a0 tx=0x7fa87802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4ba4e6813fdaf3a server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:57:33.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:33.229+0000 7fa883e00640 1 -- 172.21.15.131:0/1497237537 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa878037070 con 0x7fa884155840 2024-05-26T22:57:33.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:33.229+0000 7fa883e00640 1 -- 172.21.15.131:0/1497237537 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fa878032440 con 0x7fa884155840 2024-05-26T22:57:33.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:33.229+0000 7fa883e00640 1 -- 172.21.15.131:0/1497237537 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa878032830 con 0x7fa884155840 2024-05-26T22:57:33.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:33.229+0000 7fa88be00640 1 -- 172.21.15.131:0/1497237537 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa884155840 msgr2=0x7fa884071930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:33.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:33.229+0000 7fa88be00640 1 --2- 172.21.15.131:0/1497237537 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa884155840 0x7fa884071930 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7fa8780097a0 tx=0x7fa87802a180 comp rx=0 tx=0).stop 2024-05-26T22:57:33.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:33.230+0000 7fa88be00640 1 -- 172.21.15.131:0/1497237537 shutdown_connections 2024-05-26T22:57:33.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:33.230+0000 7fa88be00640 1 --2- 172.21.15.131:0/1497237537 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa88415c3d0 0x7fa88406ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:33.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:33.230+0000 7fa88be00640 1 --2- 172.21.15.131:0/1497237537 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa884155840 0x7fa884071930 unknown :-1 s=CLOSED pgs=1699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:33.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:33.230+0000 7fa88be00640 1 -- 172.21.15.131:0/1497237537 >> 172.21.15.131:0/1497237537 conn(0x7fa8840b1e90 msgr2=0x7fa8840b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:57:33.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:33.230+0000 7fa88be00640 1 -- 172.21.15.131:0/1497237537 shutdown_connections 2024-05-26T22:57:33.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:33.230+0000 7fa88be00640 1 -- 172.21.15.131:0/1497237537 wait complete. 2024-05-26T22:57:33.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:33.231+0000 7fa88be00640 1 Processor -- start 2024-05-26T22:57:33.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:33.231+0000 7fa88be00640 1 -- start start 2024-05-26T22:57:33.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:33.232+0000 7fa88be00640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa88406f490 0x7fa884148570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:33.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:33.232+0000 7fa88be00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa884155840 0x7fa8841592b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:33.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:33.232+0000 7fa88be00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa88415c3d0 0x7fa8841505f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:33.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:33.232+0000 7fa88be00640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa884071ff0 con 0x7fa88415c3d0 2024-05-26T22:57:33.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:33.232+0000 7fa88be00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa884071e70 con 0x7fa884155840 2024-05-26T22:57:33.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:33.232+0000 7fa88be00640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa8840aee50 con 0x7fa88406f490 2024-05-26T22:57:33.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:33.233+0000 7fa889e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa88415c3d0 0x7fa8841505f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:33.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:33.233+0000 7fa889e00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa88415c3d0 0x7fa8841505f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57606/0 (socket says 172.21.15.131:57606) 2024-05-26T22:57:33.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:33.233+0000 7fa889e00640 1 -- 172.21.15.131:0/3593715778 learned_addr learned my addr 172.21.15.131:0/3593715778 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-05-26T22:57:33.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:33.233+0000 7fa889400640 1 --2- 172.21.15.131:0/3593715778 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa88406f490 0x7fa884148570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:33.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:33.233+0000 7fa888a00640 1 --2- 172.21.15.131:0/3593715778 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa884155840 0x7fa8841592b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:33.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:33.233+0000 7fa889e00640 1 -- 172.21.15.131:0/3593715778 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa88406f490 msgr2=0x7fa884148570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:33.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:33.233+0000 7fa889e00640 1 --2- 172.21.15.131:0/3593715778 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa88406f490 0x7fa884148570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:33.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:33.233+0000 7fa889e00640 1 -- 172.21.15.131:0/3593715778 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa884155840 msgr2=0x7fa8841592b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:33.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:33.233+0000 7fa889e00640 1 --2- 172.21.15.131:0/3593715778 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa884155840 0x7fa8841592b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:33.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:33.233+0000 7fa889e00640 1 -- 172.21.15.131:0/3593715778 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa884153580 con 0x7fa88415c3d0 2024-05-26T22:57:33.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:33.234+0000 7fa889e00640 1 --2- 172.21.15.131:0/3593715778 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa88415c3d0 0x7fa8841505f0 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7fa87c00cb70 tx=0x7fa87c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:57:33.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:33.234+0000 7fa881600640 1 -- 172.21.15.131:0/3593715778 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa87c017070 con 0x7fa88415c3d0 2024-05-26T22:57:33.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:33.234+0000 7fa881600640 1 -- 172.21.15.131:0/3593715778 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fa87c00ad40 con 0x7fa88415c3d0 2024-05-26T22:57:33.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:33.234+0000 7fa881600640 1 -- 172.21.15.131:0/3593715778 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa87c012640 con 0x7fa88415c3d0 2024-05-26T22:57:33.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:33.234+0000 7fa88be00640 1 -- 172.21.15.131:0/3593715778 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa884153860 con 0x7fa88415c3d0 2024-05-26T22:57:33.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:33.234+0000 7fa88be00640 1 -- 172.21.15.131:0/3593715778 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa884153e20 con 0x7fa88415c3d0 2024-05-26T22:57:33.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:33.235+0000 7fa88be00640 1 -- 172.21.15.131:0/3593715778 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8840694e0 con 0x7fa88415c3d0 2024-05-26T22:57:33.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:33.236+0000 7fa881600640 1 -- 172.21.15.131:0/3593715778 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa87c01a070 con 0x7fa88415c3d0 2024-05-26T22:57:33.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:33.237+0000 7fa881600640 1 --2- 172.21.15.131:0/3593715778 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa86406be30 0x7fa86406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-26T22:57:33.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:33.237+0000 7fa881600640 1 -- 172.21.15.131:0/3593715778 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5115+0+0 (secure 0 0 0) 0x7fa87c08abe0 con 0x7fa88415c3d0 2024-05-26T22:57:33.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:33.237+0000 7fa889400640 1 --2- 172.21.15.131:0/3593715778 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa86406be30 0x7fa86406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-26T22:57:33.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:33.238+0000 7fa889400640 1 --2- 172.21.15.131:0/3593715778 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa86406be30 0x7fa86406e2f0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fa878034000 tx=0x7fa87802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-26T22:57:33.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:33.241+0000 7fa881600640 1 -- 172.21.15.131:0/3593715778 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185294 (secure 0 0 0) 0x7fa87c0596b0 con 0x7fa88415c3d0 2024-05-26T22:57:33.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:33.481+0000 7fa88be00640 1 -- 172.21.15.131:0/3593715778 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa88414e820 con 0x7fa88415c3d0 2024-05-26T22:57:33.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:33.482+0000 7fa881600640 1 -- 172.21.15.131:0/3593715778 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 120 v120) ==== 95+0+3883 (secure 0 0 0) 0x7fa87c058fb0 con 0x7fa88415c3d0 2024-05-26T22:57:33.482 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-05-26T22:57:33.483 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":120,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5959,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/2949813640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":2949813640},{"type":"v1","addr":"172.21.15.187:6833","nonce":2949813640}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104},{"gid":6151,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/331414687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":331414687},{"type":"v1","addr":"172.21.15.187:6835","nonce":331414687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":120},{"gid":8655,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6836/2528436689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":2528436689},{"type":"v1","addr":"172.21.15.131:6836","nonce":2528436689}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":120},{"gid":8658,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/3181544993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6835","nonce":3181544993},{"type":"v1","addr":"172.21.15.131:6837","nonce":3181544993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":120}],"filesystems":[]} 2024-05-26T22:57:33.483 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 120 2024-05-26T22:57:33.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:33.485+0000 7fa88be00640 1 -- 172.21.15.131:0/3593715778 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa86406be30 msgr2=0x7fa86406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:33.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:33.485+0000 7fa88be00640 1 --2- 172.21.15.131:0/3593715778 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa86406be30 0x7fa86406e2f0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fa878034000 tx=0x7fa87802e000 comp rx=0 tx=0).stop 2024-05-26T22:57:33.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:33.485+0000 7fa88be00640 1 -- 172.21.15.131:0/3593715778 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa88415c3d0 msgr2=0x7fa8841505f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-26T22:57:33.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:33.485+0000 7fa88be00640 1 --2- 172.21.15.131:0/3593715778 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa88415c3d0 0x7fa8841505f0 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7fa87c00cb70 tx=0x7fa87c00a430 comp rx=0 tx=0).stop 2024-05-26T22:57:33.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:33.485+0000 7fa88be00640 1 -- 172.21.15.131:0/3593715778 shutdown_connections 2024-05-26T22:57:33.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:33.485+0000 7fa88be00640 1 --2- 172.21.15.131:0/3593715778 >> [v2:172.21.15.131:6832/3484832947,v1:172.21.15.131:6833/3484832947] conn(0x7fa86406be30 0x7fa86406e2f0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:33.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:33.485+0000 7fa88be00640 1 --2- 172.21.15.131:0/3593715778 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa88415c3d0 0x7fa8841505f0 unknown :-1 s=CLOSED pgs=1700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:33.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:33.485+0000 7fa88be00640 1 --2- 172.21.15.131:0/3593715778 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa884155840 0x7fa8841592b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:33.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:33.485+0000 7fa88be00640 1 --2- 172.21.15.131:0/3593715778 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa88406f490 0x7fa884148570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-26T22:57:33.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:33.485+0000 7fa88be00640 1 -- 172.21.15.131:0/3593715778 >> 172.21.15.131:0/3593715778 conn(0x7fa8840b1e90 msgr2=0x7fa88415dcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-26T22:57:33.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:33.486+0000 7fa88be00640 1 -- 172.21.15.131:0/3593715778 shutdown_connections 2024-05-26T22:57:33.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-05-26T22:57:33.486+0000 7fa88be00640 1 -- 172.21.15.131:0/3593715778 wait complete. 2024-05-26T22:57:33.504 DEBUG:tasks.cephfs.filesystem:already dead... 2024-05-26T22:57:33.504 INFO:teuthology.misc:Shutting down mds daemons... 2024-05-26T22:57:33.505 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-26T22:57:33.505 INFO:teuthology.orchestra.run:waiting for 300 2024-05-26T22:57:33.525 INFO:tasks.ceph.mds.a:Stopped 2024-05-26T22:57:33.525 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-26T22:57:33.525 INFO:teuthology.orchestra.run:waiting for 300 2024-05-26T22:57:33.592 INFO:tasks.ceph.mds.c:Stopped 2024-05-26T22:57:33.592 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-26T22:57:33.592 INFO:teuthology.orchestra.run:waiting for 300 2024-05-26T22:57:33.620 INFO:tasks.ceph.mds.b:Stopped 2024-05-26T22:57:33.621 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-26T22:57:33.621 INFO:teuthology.orchestra.run:waiting for 300 2024-05-26T22:57:33.685 INFO:tasks.ceph.mds.d:Stopped 2024-05-26T22:57:33.685 INFO:teuthology.misc:Shutting down osd daemons... 2024-05-26T22:57:33.686 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-05-26T22:57:33.686 INFO:teuthology.orchestra.run:waiting for 300 2024-05-26T22:57:33.754 INFO:tasks.ceph.osd.0:Stopped 2024-05-26T22:57:33.755 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-05-26T22:57:33.755 INFO:teuthology.orchestra.run:waiting for 300 2024-05-26T22:57:33.820 INFO:tasks.ceph.osd.1:Stopped 2024-05-26T22:57:33.820 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-05-26T22:57:33.820 INFO:teuthology.orchestra.run:waiting for 300 2024-05-26T22:57:33.891 INFO:tasks.ceph.osd.2:Stopped 2024-05-26T22:57:33.891 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-05-26T22:57:33.892 INFO:teuthology.orchestra.run:waiting for 300 2024-05-26T22:57:33.958 INFO:tasks.ceph.osd.3:Stopped 2024-05-26T22:57:33.958 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-05-26T22:57:33.959 INFO:teuthology.orchestra.run:waiting for 300 2024-05-26T22:57:34.024 INFO:tasks.ceph.osd.4:Stopped 2024-05-26T22:57:34.025 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-05-26T22:57:34.025 INFO:teuthology.orchestra.run:waiting for 300 2024-05-26T22:57:34.094 INFO:tasks.ceph.osd.5:Stopped 2024-05-26T22:57:34.095 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-05-26T22:57:34.095 INFO:teuthology.orchestra.run:waiting for 300 2024-05-26T22:57:34.168 INFO:tasks.ceph.osd.6:Stopped 2024-05-26T22:57:34.169 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-05-26T22:57:34.169 INFO:teuthology.orchestra.run:waiting for 300 2024-05-26T22:57:34.249 INFO:tasks.ceph.osd.7:Stopped 2024-05-26T22:57:34.249 INFO:teuthology.misc:Shutting down mgr daemons... 2024-05-26T22:57:34.250 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-05-26T22:57:34.250 INFO:teuthology.orchestra.run:waiting for 300 2024-05-26T22:57:34.332 INFO:tasks.ceph.mgr.y:Stopped 2024-05-26T22:57:34.332 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-05-26T22:57:34.332 INFO:teuthology.orchestra.run:waiting for 300 2024-05-26T22:57:38.016 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~0s 2024-05-26T22:57:40.434 INFO:tasks.ceph.mgr.x:Stopped 2024-05-26T22:57:40.435 INFO:teuthology.misc:Shutting down mon daemons... 2024-05-26T22:57:40.435 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-05-26T22:57:40.435 INFO:teuthology.orchestra.run:waiting for 300 2024-05-26T22:57:40.482 INFO:tasks.ceph.mon.a:Stopped 2024-05-26T22:57:40.483 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-05-26T22:57:40.483 INFO:teuthology.orchestra.run:waiting for 300 2024-05-26T22:57:40.516 INFO:tasks.ceph.mon.b:Stopped 2024-05-26T22:57:40.516 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-05-26T22:57:40.517 INFO:teuthology.orchestra.run:waiting for 300 2024-05-26T22:57:40.582 INFO:tasks.ceph.mon.c:Stopped 2024-05-26T22:57:40.582 INFO:tasks.ceph:Checking cluster log for badness... 2024-05-26T22:57:40.582 DEBUG:teuthology.orchestra.run.smithi131:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Replacing daemon mds' | head -n 1 2024-05-26T22:57:40.687 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi131.front.sepia.ceph.com 2024-05-26T22:57:40.688 DEBUG:teuthology.orchestra.run.smithi131:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-05-26T22:57:40.990 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi131.front.sepia.ceph.com 2024-05-26T22:57:40.990 DEBUG:teuthology.orchestra.run.smithi131:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-05-26T22:57:41.145 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi131.front.sepia.ceph.com 2024-05-26T22:57:41.146 DEBUG:teuthology.orchestra.run.smithi131:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-05-26T22:57:41.292 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi131.front.sepia.ceph.com 2024-05-26T22:57:41.292 DEBUG:teuthology.orchestra.run.smithi131:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-05-26T22:57:41.445 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi187.front.sepia.ceph.com 2024-05-26T22:57:41.445 DEBUG:teuthology.orchestra.run.smithi187:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-05-26T22:57:41.755 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi187.front.sepia.ceph.com 2024-05-26T22:57:41.755 DEBUG:teuthology.orchestra.run.smithi187:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-05-26T22:57:41.888 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi187.front.sepia.ceph.com 2024-05-26T22:57:41.889 DEBUG:teuthology.orchestra.run.smithi187:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-05-26T22:57:42.033 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi187.front.sepia.ceph.com 2024-05-26T22:57:42.034 DEBUG:teuthology.orchestra.run.smithi187:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-05-26T22:57:42.176 INFO:tasks.ceph:Archiving mon data... 2024-05-26T22:57:42.178 DEBUG:teuthology.misc:Transferring archived files from smithi131:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2024-05-26_21:24:02-fs-squid-distro-default-smithi/7727073/data/mon.a.tgz 2024-05-26T22:57:42.178 DEBUG:teuthology.orchestra.run.smithi131:> mktemp 2024-05-26T22:57:42.236 INFO:teuthology.orchestra.run.smithi131.stdout:/tmp/tmp.yA75XJRiCA 2024-05-26T22:57:42.236 DEBUG:teuthology.orchestra.run.smithi131:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.yA75XJRiCA 2024-05-26T22:57:42.432 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod 0666 /tmp/tmp.yA75XJRiCA 2024-05-26T22:57:42.570 DEBUG:teuthology.orchestra.remote:smithi131:/tmp/tmp.yA75XJRiCA is 460KB 2024-05-26T22:57:42.677 DEBUG:teuthology.orchestra.run.smithi131:> rm -fr /tmp/tmp.yA75XJRiCA 2024-05-26T22:57:42.734 DEBUG:teuthology.misc:Transferring archived files from smithi187:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/teuthology-2024-05-26_21:24:02-fs-squid-distro-default-smithi/7727073/data/mon.b.tgz 2024-05-26T22:57:42.734 DEBUG:teuthology.orchestra.run.smithi187:> mktemp 2024-05-26T22:57:42.791 INFO:teuthology.orchestra.run.smithi187.stdout:/tmp/tmp.zWtngP0vsV 2024-05-26T22:57:42.791 DEBUG:teuthology.orchestra.run.smithi187:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.zWtngP0vsV 2024-05-26T22:57:42.988 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod 0666 /tmp/tmp.zWtngP0vsV 2024-05-26T22:57:43.017 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x has been restored 2024-05-26T22:57:43.125 DEBUG:teuthology.orchestra.remote:smithi187:/tmp/tmp.zWtngP0vsV is 464KB 2024-05-26T22:57:43.233 DEBUG:teuthology.orchestra.run.smithi187:> rm -fr /tmp/tmp.zWtngP0vsV 2024-05-26T22:57:43.290 DEBUG:teuthology.misc:Transferring archived files from smithi187:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/teuthology-2024-05-26_21:24:02-fs-squid-distro-default-smithi/7727073/data/mon.c.tgz 2024-05-26T22:57:43.290 DEBUG:teuthology.orchestra.run.smithi187:> mktemp 2024-05-26T22:57:43.343 INFO:teuthology.orchestra.run.smithi187.stdout:/tmp/tmp.sgzZVdymRh 2024-05-26T22:57:43.343 DEBUG:teuthology.orchestra.run.smithi187:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.sgzZVdymRh 2024-05-26T22:57:43.524 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod 0666 /tmp/tmp.sgzZVdymRh 2024-05-26T22:57:43.648 DEBUG:teuthology.orchestra.remote:smithi187:/tmp/tmp.sgzZVdymRh is 464KB 2024-05-26T22:57:43.755 DEBUG:teuthology.orchestra.run.smithi187:> rm -fr /tmp/tmp.sgzZVdymRh 2024-05-26T22:57:43.812 INFO:tasks.ceph:Cleaning ceph cluster... 2024-05-26T22:57:43.812 DEBUG:teuthology.orchestra.run.smithi131:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-05-26T22:57:43.820 DEBUG:teuthology.orchestra.run.smithi187:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-05-26T22:57:43.962 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-05-26T22:57:44.020 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-05-26T22:57:44.021 INFO:tasks.ceph:Archiving crash dumps... 2024-05-26T22:57:44.023 DEBUG:teuthology.misc:Transferring archived files from smithi131:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-05-26_21:24:02-fs-squid-distro-default-smithi/7727073/remote/smithi131/crash 2024-05-26T22:57:44.024 DEBUG:teuthology.orchestra.run.smithi131:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-05-26T22:57:44.105 DEBUG:teuthology.misc:Transferring archived files from smithi187:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-05-26_21:24:02-fs-squid-distro-default-smithi/7727073/remote/smithi187/crash 2024-05-26T22:57:44.106 DEBUG:teuthology.orchestra.run.smithi187:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-05-26T22:57:44.192 INFO:tasks.ceph:Shutting down logrotate 2024-05-26T22:57:44.193 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-26T22:57:44.200 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-26T22:57:44.315 DEBUG:teuthology.orchestra.run.smithi131:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-05-26T22:57:44.401 DEBUG:teuthology.orchestra.run.smithi187:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-05-26T22:57:44.490 INFO:tasks.ceph:Compressing logs... 2024-05-26T22:57:44.490 DEBUG:teuthology.orchestra.run.smithi131:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-26T22:57:44.498 DEBUG:teuthology.orchestra.run.smithi187:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-26T22:57:44.576 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63221.log 2024-05-26T22:57:44.577 INFO:teuthology.orchestra.run.smithi187.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36983.log 2024-05-26T22:57:44.578 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72817.log 2024-05-26T22:57:44.578 INFO:teuthology.orchestra.run.smithi187.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-05-26T22:57:44.579 INFO:teuthology.orchestra.run.smithi187.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-05-26T22:57:44.579 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.63221.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50484.log 2024-05-26T22:57:44.581 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-client.admin.36983.log: /var/log/ceph/ceph.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.36983.log.gz 2024-05-26T22:57:44.581 INFO:teuthology.orchestra.run.smithi187.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37023.log 2024-05-26T22:57:44.581 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.72817.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.63221.log.gz 2024-05-26T22:57:44.581 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- 87.9% /var/log/ceph/ceph-client.admin.52271.log -- replaced with /var/log/ceph/ceph-client.admin.72817.log.gz 2024-05-26T22:57:44.581 INFO:teuthology.orchestra.run.smithi131.stderr: 2024-05-26T22:57:44.583 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.42933.log 2024-05-26T22:57:44.583 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-client.admin.37023.log: 92.7% -- replaced with /var/log/ceph/ceph.log.gz 2024-05-26T22:57:44.583 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.50484.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42609.log 2024-05-26T22:57:44.583 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.52271.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.50484.log.gz 2024-05-26T22:57:44.584 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71898.log 2024-05-26T22:57:44.584 INFO:teuthology.orchestra.run.smithi131.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52271.log.gz 2024-05-26T22:57:44.585 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.42609.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77001.log 2024-05-26T22:57:44.586 INFO:teuthology.orchestra.run.smithi131.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42609.log.gz 2024-05-26T22:57:44.586 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.71898.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.71898.log.gz 2024-05-26T22:57:44.586 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42945.log 2024-05-26T22:57:44.586 INFO:teuthology.orchestra.run.smithi187.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.37023.log.gz 2024-05-26T22:57:44.587 INFO:teuthology.orchestra.run.smithi187.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37063.log 2024-05-26T22:57:44.587 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-client.1.42933.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.42933.log.gz 2024-05-26T22:57:44.587 INFO:teuthology.orchestra.run.smithi187.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-05-26T22:57:44.587 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-client.admin.37063.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.37063.log.gz 2024-05-26T22:57:44.587 INFO:teuthology.orchestra.run.smithi187.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-05-26T22:57:44.589 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-osd.7.log: 93.8% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-05-26T22:57:44.589 INFO:teuthology.orchestra.run.smithi187.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-05-26T22:57:44.589 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36854.log 2024-05-26T22:57:44.590 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.77001.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.77001.log.gz 2024-05-26T22:57:44.590 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46964.log 2024-05-26T22:57:44.590 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.42945.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51490.log 2024-05-26T22:57:44.590 INFO:teuthology.orchestra.run.smithi131.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42945.log.gz 2024-05-26T22:57:44.591 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46964.log: gzip 89.5% -5 -- replaced with /var/log/ceph/ceph-client.admin.46964.log.gz --verbose 2024-05-26T22:57:44.591 INFO:teuthology.orchestra.run.smithi131.stderr: -- /var/log/ceph/ceph-client.admin.44751.log 2024-05-26T22:57:44.591 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.51490.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71131.log 2024-05-26T22:57:44.591 INFO:teuthology.orchestra.run.smithi131.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51490.log.gz 2024-05-26T22:57:44.593 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-05-26T22:57:44.593 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-client.admin.36854.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.36854.log.gz 2024-05-26T22:57:44.593 INFO:teuthology.orchestra.run.smithi187.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36903.log 2024-05-26T22:57:44.593 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-05-26T22:57:44.593 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.44751.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76875.log 2024-05-26T22:57:44.594 INFO:teuthology.orchestra.run.smithi131.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.44751.log.gz 2024-05-26T22:57:44.594 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.71131.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55117.log 2024-05-26T22:57:44.594 INFO:teuthology.orchestra.run.smithi131.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.71131.log.gz 2024-05-26T22:57:44.596 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-client.admin.36903.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-05-26T22:57:44.596 INFO:teuthology.orchestra.run.smithi187.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.36903.log.gz 2024-05-26T22:57:44.596 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-05-26T22:57:44.596 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.76875.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.76875.log.gz 2024-05-26T22:57:44.597 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57786.log 2024-05-26T22:57:44.597 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.55117.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.55117.log.gz 2024-05-26T22:57:44.597 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44457.log 2024-05-26T22:57:44.597 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.57786.log: gzip 88.1% -- replaced with /var/log/ceph/ceph-client.admin.57786.log.gz 2024-05-26T22:57:44.597 INFO:teuthology.orchestra.run.smithi131.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.61921.log 2024-05-26T22:57:44.598 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36772.log 2024-05-26T22:57:44.599 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.44457.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.30877.log 2024-05-26T22:57:44.599 INFO:teuthology.orchestra.run.smithi131.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44457.log.gz 2024-05-26T22:57:44.599 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.61921.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.61921.log.gz 2024-05-26T22:57:44.599 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48080.log 2024-05-26T22:57:44.599 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph.tmp-client.admin.30877.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.30877.log.gz 2024-05-26T22:57:44.601 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60636.log 2024-05-26T22:57:44.601 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48080.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.58226.log 2024-05-26T22:57:44.601 INFO:teuthology.orchestra.run.smithi131.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.48080.log.gz 2024-05-26T22:57:44.602 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-osd.6.log: /var/log/ceph/ceph-client.admin.36772.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36943.log 2024-05-26T22:57:44.603 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.60636.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.60636.log.gz 2024-05-26T22:57:44.604 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57828.log 2024-05-26T22:57:44.604 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.0.58226.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.58226.log.gz 2024-05-26T22:57:44.604 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52176.log 2024-05-26T22:57:44.604 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-client.admin.36943.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.36772.log.gz 2024-05-26T22:57:44.604 INFO:teuthology.orchestra.run.smithi187.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.43348.log 2024-05-26T22:57:44.606 INFO:teuthology.orchestra.run.smithi187.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 94.1% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-05-26T22:57:44.607 INFO:teuthology.orchestra.run.smithi187.stderr: 2024-05-26T22:57:44.607 INFO:teuthology.orchestra.run.smithi187.stderr: 88.1%gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-05-26T22:57:44.607 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.57828.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61238.log 2024-05-26T22:57:44.607 INFO:teuthology.orchestra.run.smithi131.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.57828.log.gz 2024-05-26T22:57:44.608 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.52176.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.52176.log.gz 2024-05-26T22:57:44.608 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47306.log 2024-05-26T22:57:44.608 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.61238.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.61238.log.gz 2024-05-26T22:57:44.608 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76335.log 2024-05-26T22:57:44.608 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47306.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42437.log 2024-05-26T22:57:44.608 INFO:teuthology.orchestra.run.smithi131.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47306.log.gz 2024-05-26T22:57:44.610 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.76335.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56811.log 2024-05-26T22:57:44.610 INFO:teuthology.orchestra.run.smithi131.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.76335.log.gz 2024-05-26T22:57:44.610 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.42437.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75531.log 2024-05-26T22:57:44.610 INFO:teuthology.orchestra.run.smithi131.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.42437.log.gz 2024-05-26T22:57:44.611 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-client.3.43348.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36943.log.gz 2024-05-26T22:57:44.611 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-mds.d.log: -- replaced with /var/log/ceph/ceph-client.3.43348.log.gz 2024-05-26T22:57:44.612 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36812.log 2024-05-26T22:57:44.613 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.56811.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.70258.log 88.9% 2024-05-26T22:57:44.613 INFO:teuthology.orchestra.run.smithi131.stderr: -- replaced with /var/log/ceph/ceph-client.admin.56811.log.gz 2024-05-26T22:57:44.613 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.75531.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64597.log 2024-05-26T22:57:44.613 INFO:teuthology.orchestra.run.smithi131.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.75531.log.gz 2024-05-26T22:57:44.613 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.0.70258.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.70258.log.gz 2024-05-26T22:57:44.613 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61010.log 2024-05-26T22:57:44.614 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-client.admin.36812.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.36812.log.gz 2024-05-26T22:57:44.615 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.64597.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56643.log 2024-05-26T22:57:44.615 INFO:teuthology.orchestra.run.smithi131.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.64597.log.gz 2024-05-26T22:57:44.615 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.61010.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.61010.log.gz 2024-05-26T22:57:44.615 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64391.log 2024-05-26T22:57:44.616 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.56643.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77253.log 2024-05-26T22:57:44.616 INFO:teuthology.orchestra.run.smithi131.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.56643.log.gz 2024-05-26T22:57:44.617 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.64391.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67159.log 2024-05-26T22:57:44.617 INFO:teuthology.orchestra.run.smithi131.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.64391.log.gz 2024-05-26T22:57:44.618 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.77253.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47912.log 2024-05-26T22:57:44.618 INFO:teuthology.orchestra.run.smithi131.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.77253.log.gz 2024-05-26T22:57:44.619 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.67159.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70379.log 2024-05-26T22:57:44.620 INFO:teuthology.orchestra.run.smithi131.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.67159.log.gz 2024-05-26T22:57:44.620 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47912.log: gzip -5 87.7% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.47912.log.gz /var/log/ceph/ceph-client.admin.46475.log 2024-05-26T22:57:44.621 INFO:teuthology.orchestra.run.smithi131.stderr: 2024-05-26T22:57:44.622 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.70379.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.70379.log.gz 2024-05-26T22:57:44.622 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53707.log 2024-05-26T22:57:44.623 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46475.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46475.log.gz 2024-05-26T22:57:44.623 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64181.log 2024-05-26T22:57:44.624 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.53707.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69440.log 2024-05-26T22:57:44.625 INFO:teuthology.orchestra.run.smithi131.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.53707.log.gz 2024-05-26T22:57:44.626 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.64181.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.64181.log.gz 2024-05-26T22:57:44.626 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46811.log 2024-05-26T22:57:44.626 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.69440.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.69440.log.gz 2024-05-26T22:57:44.627 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55636.log 2024-05-26T22:57:44.628 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46811.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68973.log 2024-05-26T22:57:44.628 INFO:teuthology.orchestra.run.smithi131.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.46811.log.gz 2024-05-26T22:57:44.629 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.55636.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77547.log 2024-05-26T22:57:44.629 INFO:teuthology.orchestra.run.smithi131.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.55636.log.gz 2024-05-26T22:57:44.630 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.68973.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.68973.log.gz 2024-05-26T22:57:44.630 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46391.log 2024-05-26T22:57:44.631 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.77547.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42521.log 2024-05-26T22:57:44.632 INFO:teuthology.orchestra.run.smithi131.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.77547.log.gz 2024-05-26T22:57:44.633 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46391.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.46391.log.gz 2024-05-26T22:57:44.633 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42735.log 2024-05-26T22:57:44.634 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.42521.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50550.log 2024-05-26T22:57:44.634 INFO:teuthology.orchestra.run.smithi131.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42521.log.gz 2024-05-26T22:57:44.635 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.42735.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42735.log.gz 2024-05-26T22:57:44.635 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61621.log 2024-05-26T22:57:44.636 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.50550.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57362.log 2024-05-26T22:57:44.636 INFO:teuthology.orchestra.run.smithi131.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.50550.log.gz 2024-05-26T22:57:44.637 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.61621.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.61621.log.gz 2024-05-26T22:57:44.638 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76377.log 2024-05-26T22:57:44.639 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.57362.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63835.log 2024-05-26T22:57:44.639 INFO:teuthology.orchestra.run.smithi131.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.57362.log.gz 2024-05-26T22:57:44.640 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.76377.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.76377.log.gz 2024-05-26T22:57:44.640 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64639.log 2024-05-26T22:57:44.641 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.63835.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.63835.log.gz 2024-05-26T22:57:44.641 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61536.log 2024-05-26T22:57:44.642 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.64639.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70337.log 2024-05-26T22:57:44.642 INFO:teuthology.orchestra.run.smithi131.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.64639.log.gz 2024-05-26T22:57:44.643 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.61536.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.61536.log.gz 2024-05-26T22:57:44.643 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.53586.log 2024-05-26T22:57:44.644 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.70337.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.70337.log.gz 2024-05-26T22:57:44.644 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53206.log 2024-05-26T22:57:44.645 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.0.53586.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56353.log 0.0% 2024-05-26T22:57:44.645 INFO:teuthology.orchestra.run.smithi131.stderr: -- replaced with /var/log/ceph/ceph-client.0.53586.log.gz 2024-05-26T22:57:44.646 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.53206.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51060.log 2024-05-26T22:57:44.647 INFO:teuthology.orchestra.run.smithi131.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.53206.log.gz 2024-05-26T22:57:44.647 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.56353.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43900.log 2024-05-26T22:57:44.647 INFO:teuthology.orchestra.run.smithi131.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.56353.log.gz 2024-05-26T22:57:44.648 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.51060.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51060.log.gz 2024-05-26T22:57:44.649 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47564.log 2024-05-26T22:57:44.650 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.43900.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67225.log 2024-05-26T22:57:44.650 INFO:teuthology.orchestra.run.smithi131.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43900.log.gz 2024-05-26T22:57:44.651 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47564.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47438.log 2024-05-26T22:57:44.651 INFO:teuthology.orchestra.run.smithi131.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47564.log.gz 2024-05-26T22:57:44.652 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.67225.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73721.log 2024-05-26T22:57:44.652 INFO:teuthology.orchestra.run.smithi131.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.67225.log.gz 2024-05-26T22:57:44.653 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47438.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57320.log 2024-05-26T22:57:44.653 INFO:teuthology.orchestra.run.smithi131.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.47438.log.gz 2024-05-26T22:57:44.654 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.73721.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65207.log 2024-05-26T22:57:44.655 INFO:teuthology.orchestra.run.smithi131.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.73721.log.gz 2024-05-26T22:57:44.655 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.57320.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73595.log 2024-05-26T22:57:44.655 INFO:teuthology.orchestra.run.smithi131.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.57320.log.gz 2024-05-26T22:57:44.657 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.65207.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.65207.log.gz 2024-05-26T22:57:44.657 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51364.log 2024-05-26T22:57:44.658 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.73595.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43879.log 2024-05-26T22:57:44.658 INFO:teuthology.orchestra.run.smithi131.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.73595.log.gz 2024-05-26T22:57:44.659 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.51364.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43924.log 2024-05-26T22:57:44.659 INFO:teuthology.orchestra.run.smithi131.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51364.log.gz 2024-05-26T22:57:44.663 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.43879.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43879.log.gz 2024-05-26T22:57:44.663 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53332.log 2024-05-26T22:57:44.663 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.43924.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73847.log 2024-05-26T22:57:44.663 INFO:teuthology.orchestra.run.smithi131.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43924.log.gz 2024-05-26T22:57:44.663 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.53332.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46013.log 2024-05-26T22:57:44.663 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.73847.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.53332.log.gz 2024-05-26T22:57:44.663 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52517.log 2024-05-26T22:57:44.664 INFO:teuthology.orchestra.run.smithi131.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.73847.log.gz 2024-05-26T22:57:44.665 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46013.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64904.log 2024-05-26T22:57:44.665 INFO:teuthology.orchestra.run.smithi131.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46013.log.gz 2024-05-26T22:57:44.665 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.52517.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53080.log 2024-05-26T22:57:44.665 INFO:teuthology.orchestra.run.smithi131.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.52517.log.gz 2024-05-26T22:57:44.666 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.64904.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.64904.log.gz 2024-05-26T22:57:44.666 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42310.log 2024-05-26T22:57:44.667 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.53080.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56979.log 2024-05-26T22:57:44.667 INFO:teuthology.orchestra.run.smithi131.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.53080.log.gz 2024-05-26T22:57:44.668 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.42310.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42310.log.gz 2024-05-26T22:57:44.668 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-05-26T22:57:44.669 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.56979.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.56979.log.gz 2024-05-26T22:57:44.669 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62257.log 2024-05-26T22:57:44.670 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42819.log 2024-05-26T22:57:44.671 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.62257.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47606.log 2024-05-26T22:57:44.672 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.42819.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.62257.log.gz 2024-05-26T22:57:44.672 INFO:teuthology.orchestra.run.smithi131.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42819.log.gz 2024-05-26T22:57:44.672 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55312.log 2024-05-26T22:57:44.673 INFO:teuthology.orchestra.run.smithi131.stderr: 86.8% -- replaced with /var/log/ceph/ceph.log.gz 2024-05-26T22:57:44.673 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47606.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65046.log 2024-05-26T22:57:44.674 INFO:teuthology.orchestra.run.smithi131.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.47606.log.gz 2024-05-26T22:57:44.675 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.55312.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62591.log 2024-05-26T22:57:44.675 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.65046.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.55312.log.gz 2024-05-26T22:57:44.676 INFO:teuthology.orchestra.run.smithi131.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.65046.log.gz 2024-05-26T22:57:44.676 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72276.log 2024-05-26T22:57:44.677 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.62591.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68054.log 2024-05-26T22:57:44.677 INFO:teuthology.orchestra.run.smithi131.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.62591.log.gz 2024-05-26T22:57:44.678 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.72276.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73553.log 2024-05-26T22:57:44.678 INFO:teuthology.orchestra.run.smithi131.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.72276.log.gz 2024-05-26T22:57:44.679 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.68054.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48374.log 2024-05-26T22:57:44.679 INFO:teuthology.orchestra.run.smithi131.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.68054.log.gz 2024-05-26T22:57:44.680 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.73553.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65897.log 2024-05-26T22:57:44.680 INFO:teuthology.orchestra.run.smithi131.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.73553.log.gz 2024-05-26T22:57:44.681 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48374.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65464.log 2024-05-26T22:57:44.681 INFO:teuthology.orchestra.run.smithi131.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48374.log.gz 2024-05-26T22:57:44.682 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.65897.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55923.log 2024-05-26T22:57:44.682 INFO:teuthology.orchestra.run.smithi131.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.65897.log.gz 2024-05-26T22:57:44.683 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.65464.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57404.log 2024-05-26T22:57:44.683 INFO:teuthology.orchestra.run.smithi131.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.65464.log.gz 2024-05-26T22:57:44.684 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.55923.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-05-26T22:57:44.684 INFO:teuthology.orchestra.run.smithi131.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.55923.log.gz 2024-05-26T22:57:44.685 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.57404.log: gzip -5 --verbose -- 87.3% -- replaced with /var/log/ceph/ceph-client.admin.57404.log.gz 2024-05-26T22:57:44.685 INFO:teuthology.orchestra.run.smithi131.stderr: /var/log/ceph/ceph-client.admin.52559.log 2024-05-26T22:57:44.686 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61280.log 2024-05-26T22:57:44.687 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.52559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64513.log 2024-05-26T22:57:44.688 INFO:teuthology.orchestra.run.smithi131.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52559.log.gz 2024-05-26T22:57:44.688 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.61280.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.61280.log.gz 2024-05-26T22:57:44.689 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77505.log 2024-05-26T22:57:44.689 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.64513.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46643.log 2024-05-26T22:57:44.690 INFO:teuthology.orchestra.run.smithi131.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.64513.log.gz 2024-05-26T22:57:44.691 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.77505.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.77505.log.gz 2024-05-26T22:57:44.691 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69015.log 2024-05-26T22:57:44.692 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46643.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49519.log 2024-05-26T22:57:44.692 INFO:teuthology.orchestra.run.smithi131.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46643.log.gz 2024-05-26T22:57:44.693 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.69015.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75919.log 2024-05-26T22:57:44.693 INFO:teuthology.orchestra.run.smithi131.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.69015.log.gz 2024-05-26T22:57:44.694 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.49519.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76293.log 2024-05-26T22:57:44.694 INFO:teuthology.orchestra.run.smithi131.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49519.log.gz 2024-05-26T22:57:44.695 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.75919.log: gzip -5 --verbose -- 88.1% /var/log/ceph/ceph-client.admin.51280.log 2024-05-26T22:57:44.695 INFO:teuthology.orchestra.run.smithi131.stderr: -- replaced with /var/log/ceph/ceph-client.admin.75919.log.gz 2024-05-26T22:57:44.696 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.76293.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65515.log 2024-05-26T22:57:44.696 INFO:teuthology.orchestra.run.smithi131.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.76293.log.gz 2024-05-26T22:57:44.697 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.51280.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55797.log 2024-05-26T22:57:44.698 INFO:teuthology.orchestra.run.smithi131.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.51280.log.gz 2024-05-26T22:57:44.698 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.65515.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.65515.log.gz 2024-05-26T22:57:44.699 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68805.log 2024-05-26T22:57:44.699 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.55797.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63152.log 2024-05-26T22:57:44.700 INFO:teuthology.orchestra.run.smithi131.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.55797.log.gz 2024-05-26T22:57:44.700 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.68805.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74180.log 2024-05-26T22:57:44.701 INFO:teuthology.orchestra.run.smithi131.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.68805.log.gz 2024-05-26T22:57:44.702 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.63152.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56685.log 2024-05-26T22:57:44.702 INFO:teuthology.orchestra.run.smithi131.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.63152.log.gz 2024-05-26T22:57:44.703 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.74180.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57702.log 2024-05-26T22:57:44.703 INFO:teuthology.orchestra.run.smithi131.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.74180.log.gz 2024-05-26T22:57:44.704 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.56685.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60884.log 2024-05-26T22:57:44.704 INFO:teuthology.orchestra.run.smithi131.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.56685.log.gz 2024-05-26T22:57:44.705 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.57702.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49084.log 2024-05-26T22:57:44.705 INFO:teuthology.orchestra.run.smithi131.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.57702.log.gz 2024-05-26T22:57:44.706 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.60884.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75877.log 2024-05-26T22:57:44.706 INFO:teuthology.orchestra.run.smithi131.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.60884.log.gz 2024-05-26T22:57:44.707 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.49084.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67834.log 2024-05-26T22:57:44.707 INFO:teuthology.orchestra.run.smithi131.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.49084.log.gz 2024-05-26T22:57:44.708 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.75877.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56269.log 2024-05-26T22:57:44.708 INFO:teuthology.orchestra.run.smithi131.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.75877.log.gz 2024-05-26T22:57:44.709 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.67834.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44583.log 2024-05-26T22:57:44.710 INFO:teuthology.orchestra.run.smithi131.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.67834.log.gz 2024-05-26T22:57:44.711 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.56269.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.56269.log.gz 2024-05-26T22:57:44.711 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42693.log 2024-05-26T22:57:44.711 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.44583.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46223.log 2024-05-26T22:57:44.712 INFO:teuthology.orchestra.run.smithi131.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44583.log.gz 2024-05-26T22:57:44.712 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.42693.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76917.log 2024-05-26T22:57:44.713 INFO:teuthology.orchestra.run.smithi131.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42693.log.gz 2024-05-26T22:57:44.714 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46223.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49603.log 2024-05-26T22:57:44.714 INFO:teuthology.orchestra.run.smithi131.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46223.log.gz 2024-05-26T22:57:44.715 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.76917.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77463.log 2024-05-26T22:57:44.715 INFO:teuthology.orchestra.run.smithi131.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.76917.log.gz 2024-05-26T22:57:44.715 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.49603.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63877.log 2024-05-26T22:57:44.716 INFO:teuthology.orchestra.run.smithi131.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.49603.log.gz 2024-05-26T22:57:44.717 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.77463.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72360.log 2024-05-26T22:57:44.717 INFO:teuthology.orchestra.run.smithi131.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.77463.log.gz 2024-05-26T22:57:44.718 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.63877.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71047.log 2024-05-26T22:57:44.718 INFO:teuthology.orchestra.run.smithi131.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.63877.log.gz 2024-05-26T22:57:44.719 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.72360.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71263.log 2024-05-26T22:57:44.719 INFO:teuthology.orchestra.run.smithi131.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.72360.log.gz 2024-05-26T22:57:44.720 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.71047.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76461.log 2024-05-26T22:57:44.720 INFO:teuthology.orchestra.run.smithi131.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.71047.log.gz 2024-05-26T22:57:44.721 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.71263.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58389.log 2024-05-26T22:57:44.722 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.76461.log: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.71263.log.gz 2024-05-26T22:57:44.722 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55246.log 2024-05-26T22:57:44.723 INFO:teuthology.orchestra.run.smithi131.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.76461.log.gz 2024-05-26T22:57:44.724 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.58389.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72463.log 2024-05-26T22:57:44.724 INFO:teuthology.orchestra.run.smithi131.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.58389.log.gz 2024-05-26T22:57:44.724 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.55246.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64681.log 2024-05-26T22:57:44.726 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.72463.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.55246.log.gz 2024-05-26T22:57:44.726 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46181.log 2024-05-26T22:57:44.726 INFO:teuthology.orchestra.run.smithi131.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.72463.log.gz 2024-05-26T22:57:44.727 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.64681.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42987.log 2024-05-26T22:57:44.727 INFO:teuthology.orchestra.run.smithi131.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.64681.log.gz 2024-05-26T22:57:44.728 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46181.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73763.log 2024-05-26T22:57:44.728 INFO:teuthology.orchestra.run.smithi131.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.46181.log.gz 2024-05-26T22:57:44.729 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.42987.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44415.log 2024-05-26T22:57:44.729 INFO:teuthology.orchestra.run.smithi131.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42987.log.gz 2024-05-26T22:57:44.730 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.73763.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73328.log 2024-05-26T22:57:44.730 INFO:teuthology.orchestra.run.smithi131.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.73763.log.gz 2024-05-26T22:57:44.731 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.44415.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44415.log.gz 2024-05-26T22:57:44.731 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62131.log 2024-05-26T22:57:44.732 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.73328.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37334.log 2024-05-26T22:57:44.733 INFO:teuthology.orchestra.run.smithi131.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.73328.log.gz 2024-05-26T22:57:44.733 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.62131.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51882.log 2024-05-26T22:57:44.733 INFO:teuthology.orchestra.run.smithi131.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.62131.log.gz 2024-05-26T22:57:44.734 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.37334.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67051.log 2024-05-26T22:57:44.734 INFO:teuthology.orchestra.run.smithi131.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.37334.log.gz 2024-05-26T22:57:44.735 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.51882.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71197.log 2024-05-26T22:57:44.736 INFO:teuthology.orchestra.run.smithi131.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51882.log.gz 2024-05-26T22:57:44.736 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.67051.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-05-26T22:57:44.736 INFO:teuthology.orchestra.run.smithi131.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.67051.log.gz 2024-05-26T22:57:44.738 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.71197.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68096.log 2024-05-26T22:57:44.738 INFO:teuthology.orchestra.run.smithi131.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.71197.log.gz 2024-05-26T22:57:44.739 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76503.log 2024-05-26T22:57:44.740 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.68096.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60426.log 2024-05-26T22:57:44.740 INFO:teuthology.orchestra.run.smithi131.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.68096.log.gz 2024-05-26T22:57:44.741 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.76503.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73074.log 2024-05-26T22:57:44.741 INFO:teuthology.orchestra.run.smithi131.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.76503.log.gz 2024-05-26T22:57:44.742 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.60426.log: 89.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-05-26T22:57:44.742 INFO:teuthology.orchestra.run.smithi131.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.60426.log.gz 2024-05-26T22:57:44.742 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48998.log 2024-05-26T22:57:44.743 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.73074.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59362.log 2024-05-26T22:57:44.744 INFO:teuthology.orchestra.run.smithi131.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.73074.log.gz 2024-05-26T22:57:44.744 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48998.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51738.log 2024-05-26T22:57:44.745 INFO:teuthology.orchestra.run.smithi131.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48998.log.gz 2024-05-26T22:57:44.745 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.59362.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68222.log 2024-05-26T22:57:44.746 INFO:teuthology.orchestra.run.smithi131.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.59362.log.gz 2024-05-26T22:57:44.747 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.51738.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43248.log 2024-05-26T22:57:44.747 INFO:teuthology.orchestra.run.smithi131.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51738.log.gz 2024-05-26T22:57:44.747 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.68222.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49309.log 2024-05-26T22:57:44.748 INFO:teuthology.orchestra.run.smithi131.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.68222.log.gz 2024-05-26T22:57:44.749 INFO:teuthology.orchestra.run.smithi187.stderr: 92.8% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-05-26T22:57:44.749 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.43248.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46559.log 2024-05-26T22:57:44.749 INFO:teuthology.orchestra.run.smithi131.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43248.log.gz 2024-05-26T22:57:44.750 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.49309.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54998.log 2024-05-26T22:57:44.750 INFO:teuthology.orchestra.run.smithi131.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49309.log.gz 2024-05-26T22:57:44.751 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43584.log 2024-05-26T22:57:44.751 INFO:teuthology.orchestra.run.smithi131.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46559.log.gz 2024-05-26T22:57:44.752 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.54998.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63107.log 2024-05-26T22:57:44.752 INFO:teuthology.orchestra.run.smithi131.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.54998.log.gz 2024-05-26T22:57:44.753 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.43584.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49561.log 2024-05-26T22:57:44.753 INFO:teuthology.orchestra.run.smithi131.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43584.log.gz 2024-05-26T22:57:44.754 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.63107.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48290.log 2024-05-26T22:57:44.755 INFO:teuthology.orchestra.run.smithi131.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.63107.log.gz 2024-05-26T22:57:44.755 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.49561.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65771.log 2024-05-26T22:57:44.755 INFO:teuthology.orchestra.run.smithi131.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49561.log.gz 2024-05-26T22:57:44.756 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48290.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48290.log.gz 2024-05-26T22:57:44.757 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43581.log 2024-05-26T22:57:44.758 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.65771.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68306.log 2024-05-26T22:57:44.758 INFO:teuthology.orchestra.run.smithi131.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.65771.log.gz 2024-05-26T22:57:44.759 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.43581.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65292.log 2024-05-26T22:57:44.759 INFO:teuthology.orchestra.run.smithi131.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43581.log.gz 2024-05-26T22:57:44.760 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.68306.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66356.log 2024-05-26T22:57:44.760 INFO:teuthology.orchestra.run.smithi131.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.68306.log.gz 2024-05-26T22:57:44.761 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.65292.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.65292.log.gz 2024-05-26T22:57:44.761 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56101.log 2024-05-26T22:57:44.762 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.66356.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71762.log 2024-05-26T22:57:44.762 INFO:teuthology.orchestra.run.smithi131.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.66356.log.gz 2024-05-26T22:57:44.763 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.56101.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77379.log 2024-05-26T22:57:44.763 INFO:teuthology.orchestra.run.smithi131.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.56101.log.gz 2024-05-26T22:57:44.764 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.71762.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.71762.log.gz 2024-05-26T22:57:44.764 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51238.log 2024-05-26T22:57:44.765 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.77379.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50976.log 2024-05-26T22:57:44.766 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.51238.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.77379.log.gz 2024-05-26T22:57:44.767 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62683.log 2024-05-26T22:57:44.767 INFO:teuthology.orchestra.run.smithi131.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51238.log.gz 2024-05-26T22:57:44.767 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.50976.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52870.log 2024-05-26T22:57:44.768 INFO:teuthology.orchestra.run.smithi131.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.50976.log.gz 2024-05-26T22:57:44.769 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.62683.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.62683.log.gz 2024-05-26T22:57:44.769 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68474.log 2024-05-26T22:57:44.770 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.52870.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54424.log 2024-05-26T22:57:44.771 INFO:teuthology.orchestra.run.smithi131.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52870.log.gz 2024-05-26T22:57:44.771 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.68474.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52740.log 2024-05-26T22:57:44.771 INFO:teuthology.orchestra.run.smithi131.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.68474.log.gz 2024-05-26T22:57:44.772 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.54424.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69836.log 2024-05-26T22:57:44.772 INFO:teuthology.orchestra.run.smithi131.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.54424.log.gz 2024-05-26T22:57:44.773 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.52740.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52740.log.gz 2024-05-26T22:57:44.773 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65334.log 2024-05-26T22:57:44.774 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.69836.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62089.log 2024-05-26T22:57:44.775 INFO:teuthology.orchestra.run.smithi131.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.69836.log.gz 2024-05-26T22:57:44.775 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.65334.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57618.log 2024-05-26T22:57:44.776 INFO:teuthology.orchestra.run.smithi131.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.65334.log.gz 2024-05-26T22:57:44.776 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.62089.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.62089.log.gz 2024-05-26T22:57:44.777 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65687.log 2024-05-26T22:57:44.778 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.57618.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.57618.log.gz 2024-05-26T22:57:44.778 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47684.log 2024-05-26T22:57:44.779 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.65687.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51798.log 2024-05-26T22:57:44.779 INFO:teuthology.orchestra.run.smithi131.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.65687.log.gz 2024-05-26T22:57:44.780 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47684.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61879.log 2024-05-26T22:57:44.780 INFO:teuthology.orchestra.run.smithi131.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47684.log.gz 2024-05-26T22:57:44.781 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.51798.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.51798.log.gz 2024-05-26T22:57:44.781 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42777.log 2024-05-26T22:57:44.781 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.61879.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-05-26T22:57:44.782 INFO:teuthology.orchestra.run.smithi131.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.61879.log.gz 2024-05-26T22:57:44.783 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.42777.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42565.log 2024-05-26T22:57:44.783 INFO:teuthology.orchestra.run.smithi131.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42777.log.gz 2024-05-26T22:57:44.783 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76419.log 2024-05-26T22:57:44.785 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.42565.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52996.log 2024-05-26T22:57:44.785 INFO:teuthology.orchestra.run.smithi131.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42565.log.gz 2024-05-26T22:57:44.786 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.76419.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.76419.log.gz 2024-05-26T22:57:44.786 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56185.log 2024-05-26T22:57:44.787 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.52996.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64951.log 2024-05-26T22:57:44.788 INFO:teuthology.orchestra.run.smithi131.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.52996.log.gz 2024-05-26T22:57:44.789 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.56185.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.56185.log.gz 2024-05-26T22:57:44.789 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59644.log 2024-05-26T22:57:44.791 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.64951.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43235.log 2024-05-26T22:57:44.791 INFO:teuthology.orchestra.run.smithi131.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.64951.log.gz 2024-05-26T22:57:44.792 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.59644.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52782.log 2024-05-26T22:57:44.792 INFO:teuthology.orchestra.run.smithi131.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.59644.log.gz 2024-05-26T22:57:44.793 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.43235.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56937.log 2024-05-26T22:57:44.793 INFO:teuthology.orchestra.run.smithi131.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43235.log.gz 2024-05-26T22:57:44.793 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.52782.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.52782.log.gz 2024-05-26T22:57:44.794 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47172.log 2024-05-26T22:57:44.794 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.56937.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45929.log 2024-05-26T22:57:44.795 INFO:teuthology.orchestra.run.smithi131.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.56937.log.gz 2024-05-26T22:57:44.796 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47172.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47172.log.gz 2024-05-26T22:57:44.796 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69709.log 2024-05-26T22:57:44.797 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.45929.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42229.log 2024-05-26T22:57:44.797 INFO:teuthology.orchestra.run.smithi131.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.45929.log.gz 2024-05-26T22:57:44.798 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.69709.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.69709.log.gz 2024-05-26T22:57:44.798 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52912.log 2024-05-26T22:57:44.799 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.42229.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64265.log 2024-05-26T22:57:44.799 INFO:teuthology.orchestra.run.smithi131.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42229.log.gz 2024-05-26T22:57:44.800 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.52912.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56601.log 2024-05-26T22:57:44.801 INFO:teuthology.orchestra.run.smithi131.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52912.log.gz 2024-05-26T22:57:44.802 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.64265.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73116.log 2024-05-26T22:57:44.802 INFO:teuthology.orchestra.run.smithi131.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.64265.log.gz 2024-05-26T22:57:44.803 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.56601.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55839.log 2024-05-26T22:57:44.803 INFO:teuthology.orchestra.run.smithi131.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.56601.log.gz 2024-05-26T22:57:44.804 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.73116.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57534.log 2024-05-26T22:57:44.804 INFO:teuthology.orchestra.run.smithi131.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.73116.log.gz 2024-05-26T22:57:44.805 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.55839.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47768.log 2024-05-26T22:57:44.806 INFO:teuthology.orchestra.run.smithi131.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.55839.log.gz 2024-05-26T22:57:44.807 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.57534.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44667.log 2024-05-26T22:57:44.807 INFO:teuthology.orchestra.run.smithi131.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.57534.log.gz 2024-05-26T22:57:44.808 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47768.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44121.log 2024-05-26T22:57:44.808 INFO:teuthology.orchestra.run.smithi131.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.47768.log.gz 2024-05-26T22:57:44.809 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.44667.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77421.log 2024-05-26T22:57:44.809 INFO:teuthology.orchestra.run.smithi131.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.44667.log.gz 2024-05-26T22:57:44.810 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.44121.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68534.log 2024-05-26T22:57:44.811 INFO:teuthology.orchestra.run.smithi131.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44121.log.gz 2024-05-26T22:57:44.812 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.77421.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68721.log 2024-05-26T22:57:44.812 INFO:teuthology.orchestra.run.smithi131.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.77421.log.gz 2024-05-26T22:57:44.813 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.68534.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46727.log 2024-05-26T22:57:44.813 INFO:teuthology.orchestra.run.smithi131.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.68534.log.gz 2024-05-26T22:57:44.814 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.68721.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51840.log 2024-05-26T22:57:44.814 INFO:teuthology.orchestra.run.smithi131.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.68721.log.gz 2024-05-26T22:57:44.815 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46727.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74900.log 2024-05-26T22:57:44.816 INFO:teuthology.orchestra.run.smithi131.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46727.log.gz 2024-05-26T22:57:44.817 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.51840.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-05-26T22:57:44.817 INFO:teuthology.orchestra.run.smithi131.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51840.log.gz 2024-05-26T22:57:44.818 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.74900.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48830.log 2024-05-26T22:57:44.818 INFO:teuthology.orchestra.run.smithi131.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.74900.log.gz 2024-05-26T22:57:44.819 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61750.log 2024-05-26T22:57:44.821 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48830.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.48830.log.gz 2024-05-26T22:57:44.821 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54824.log 2024-05-26T22:57:44.822 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61795.log 2024-05-26T22:57:44.823 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.61750.log: /var/log/ceph/ceph-client.admin.54824.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.61750.log.gz 2024-05-26T22:57:44.824 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69190.log 2024-05-26T22:57:44.824 INFO:teuthology.orchestra.run.smithi131.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.54824.log.gz 2024-05-26T22:57:44.825 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.61795.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76003.log 2024-05-26T22:57:44.826 INFO:teuthology.orchestra.run.smithi131.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.61795.log.gz 2024-05-26T22:57:44.826 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.69190.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49435.log 2024-05-26T22:57:44.827 INFO:teuthology.orchestra.run.smithi131.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.69190.log.gz 2024-05-26T22:57:44.828 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.76003.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57074.log 2024-05-26T22:57:44.828 INFO:teuthology.orchestra.run.smithi131.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.76003.log.gz 2024-05-26T22:57:44.830 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.49435.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49978.log 2024-05-26T22:57:44.830 INFO:teuthology.orchestra.run.smithi131.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.49435.log.gz 2024-05-26T22:57:44.831 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.57074.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64013.log 2024-05-26T22:57:44.831 INFO:teuthology.orchestra.run.smithi131.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.57074.log.gz 2024-05-26T22:57:44.832 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.49978.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72733.log 2024-05-26T22:57:44.832 INFO:teuthology.orchestra.run.smithi131.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49978.log.gz 2024-05-26T22:57:44.834 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.64013.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76780.log 2024-05-26T22:57:44.834 INFO:teuthology.orchestra.run.smithi131.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.64013.log.gz 2024-05-26T22:57:44.835 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.72733.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44373.log 2024-05-26T22:57:44.836 INFO:teuthology.orchestra.run.smithi131.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.72733.log.gz 2024-05-26T22:57:44.837 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.76780.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51532.log 2024-05-26T22:57:44.837 INFO:teuthology.orchestra.run.smithi131.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.76780.log.gz 2024-05-26T22:57:44.838 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.44373.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70421.log 2024-05-26T22:57:44.838 INFO:teuthology.orchestra.run.smithi131.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.44373.log.gz 2024-05-26T22:57:44.839 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.51532.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69313.log 2024-05-26T22:57:44.840 INFO:teuthology.orchestra.run.smithi131.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.51532.log.gz 2024-05-26T22:57:44.841 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.70421.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.74101.log 2024-05-26T22:57:44.841 INFO:teuthology.orchestra.run.smithi131.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.70421.log.gz 2024-05-26T22:57:44.842 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.69313.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71940.log 2024-05-26T22:57:44.842 INFO:teuthology.orchestra.run.smithi131.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.69313.log.gz 2024-05-26T22:57:44.843 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.0.74101.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-05-26T22:57:44.843 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.74101.log.gz 2024-05-26T22:57:44.845 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.71940.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44499.log 2024-05-26T22:57:44.845 INFO:teuthology.orchestra.run.smithi131.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.71940.log.gz 2024-05-26T22:57:44.846 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60384.log 2024-05-26T22:57:44.847 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.44499.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47090.log 2024-05-26T22:57:44.848 INFO:teuthology.orchestra.run.smithi131.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44499.log.gz 2024-05-26T22:57:44.849 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.60384.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-05-26T22:57:44.849 INFO:teuthology.orchestra.run.smithi131.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.60384.log.gz 2024-05-26T22:57:44.850 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47090.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48038.log 2024-05-26T22:57:44.850 INFO:teuthology.orchestra.run.smithi131.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47090.log.gz 2024-05-26T22:57:44.851 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48248.log 2024-05-26T22:57:44.853 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48038.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69484.log 2024-05-26T22:57:44.853 INFO:teuthology.orchestra.run.smithi131.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48038.log.gz 2024-05-26T22:57:44.854 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48248.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60122.log 2024-05-26T22:57:44.854 INFO:teuthology.orchestra.run.smithi131.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.48248.log.gz 2024-05-26T22:57:44.855 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.69484.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.69484.log.gz 2024-05-26T22:57:44.856 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67792.log 2024-05-26T22:57:44.856 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.60122.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.60122.log.gz 2024-05-26T22:57:44.857 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77690.log 2024-05-26T22:57:44.859 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.67792.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67960.log 2024-05-26T22:57:44.859 INFO:teuthology.orchestra.run.smithi131.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.67792.log.gz 2024-05-26T22:57:44.860 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.77690.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36950.log 2024-05-26T22:57:44.860 INFO:teuthology.orchestra.run.smithi131.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.77690.log.gz 2024-05-26T22:57:44.862 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.67960.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57930.log 2024-05-26T22:57:44.862 INFO:teuthology.orchestra.run.smithi131.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.67960.log.gz 2024-05-26T22:57:44.865 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.36950.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69878.log 2024-05-26T22:57:44.866 INFO:teuthology.orchestra.run.smithi131.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.36950.log.gz 2024-05-26T22:57:44.866 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.57930.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46349.log 2024-05-26T22:57:44.866 INFO:teuthology.orchestra.run.smithi131.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.57930.log.gz 2024-05-26T22:57:44.866 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.69878.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72108.log 2024-05-26T22:57:44.866 INFO:teuthology.orchestra.run.smithi131.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.69878.log.gz 2024-05-26T22:57:44.867 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46349.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51018.log 2024-05-26T22:57:44.867 INFO:teuthology.orchestra.run.smithi131.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46349.log.gz 2024-05-26T22:57:44.868 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.72108.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55570.log 2024-05-26T22:57:44.869 INFO:teuthology.orchestra.run.smithi131.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.72108.log.gz 2024-05-26T22:57:44.870 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.51018.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71005.log 2024-05-26T22:57:44.870 INFO:teuthology.orchestra.run.smithi131.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51018.log.gz 2024-05-26T22:57:44.871 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.55570.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60552.log 2024-05-26T22:57:44.872 INFO:teuthology.orchestra.run.smithi131.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.55570.log.gz/var/log/ceph/ceph-client.admin.71005.log: 2024-05-26T22:57:44.872 INFO:teuthology.orchestra.run.smithi131.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.71005.log.gz 2024-05-26T22:57:44.873 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54866.log 2024-05-26T22:57:44.874 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.60552.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.60552.log.gz 2024-05-26T22:57:44.874 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56437.log 2024-05-26T22:57:44.876 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.54866.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-05-26T22:57:44.876 INFO:teuthology.orchestra.run.smithi131.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.54866.log.gz 2024-05-26T22:57:44.877 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.56437.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72150.log 2024-05-26T22:57:44.877 INFO:teuthology.orchestra.run.smithi131.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.56437.log.gz 2024-05-26T22:57:44.878 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74222.log 2024-05-26T22:57:44.880 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.72150.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.72150.log.gz 2024-05-26T22:57:44.882 INFO:teuthology.orchestra.run.smithi131.stderr:gzip/var/log/ceph/ceph-client.admin.74222.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.51924.log 2024-05-26T22:57:44.882 INFO:teuthology.orchestra.run.smithi131.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.74222.log.gz 2024-05-26T22:57:44.883 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76129.log 2024-05-26T22:57:44.884 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.51924.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.51924.log.gz 2024-05-26T22:57:44.885 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.76129.log: /var/log/ceph/ceph-client.admin.59413.log 2024-05-26T22:57:44.886 INFO:teuthology.orchestra.run.smithi131.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.76129.log.gz 2024-05-26T22:57:44.887 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43253.log 2024-05-26T22:57:44.887 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.59413.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.59413.log.gz 2024-05-26T22:57:44.888 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55504.log 2024-05-26T22:57:44.889 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.43253.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43253.log.gz 2024-05-26T22:57:44.891 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.55504.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77127.log 2024-05-26T22:57:44.892 INFO:teuthology.orchestra.run.smithi131.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.55504.log.gz 2024-05-26T22:57:44.893 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73805.log 2024-05-26T22:57:44.894 INFO:teuthology.orchestra.run.smithi187.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-05-26T22:57:44.894 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.77127.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.77127.log.gz 2024-05-26T22:57:44.895 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.73805.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.73805.log.gz 2024-05-26T22:57:44.896 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57448.log 2024-05-26T22:57:44.898 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65855.log 2024-05-26T22:57:44.899 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.57448.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.57448.log.gz 2024-05-26T22:57:44.900 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.65855.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.65855.log.gz 2024-05-26T22:57:44.901 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47263.log 2024-05-26T22:57:44.902 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51676.log 2024-05-26T22:57:44.904 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47263.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47263.log.gz 2024-05-26T22:57:44.905 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.51676.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.51676.log.gz 2024-05-26T22:57:44.906 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52050.log 2024-05-26T22:57:44.907 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52689.log 2024-05-26T22:57:44.909 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.52050.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.52050.log.gz 2024-05-26T22:57:44.910 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.52689.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.52689.log.gz 2024-05-26T22:57:44.911 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73200.log 2024-05-26T22:57:44.912 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64223.log 2024-05-26T22:57:44.914 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.73200.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.73200.log.gz 2024-05-26T22:57:44.915 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.64223.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.64223.log.gz 2024-05-26T22:57:44.916 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56507.log 2024-05-26T22:57:44.917 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57154.log 2024-05-26T22:57:44.918 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.56507.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.56507.log.gz 2024-05-26T22:57:44.919 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.57154.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.57154.log.gz 2024-05-26T22:57:44.920 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69794.log 2024-05-26T22:57:44.922 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48788.log 2024-05-26T22:57:44.923 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.69794.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.69794.log.gz 2024-05-26T22:57:44.924 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48788.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48788.log.gz 2024-05-26T22:57:44.925 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60926.log 2024-05-26T22:57:44.926 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75961.log 2024-05-26T22:57:44.928 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.60926.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.60926.log.gz 2024-05-26T22:57:44.929 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.75961.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.75961.log.gz 2024-05-26T22:57:44.930 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72631.log 2024-05-26T22:57:44.931 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56395.log 2024-05-26T22:57:44.932 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.72631.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.72631.log.gz 2024-05-26T22:57:44.933 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5/var/log/ceph/ceph-client.admin.56395.log: --verbose -- /var/log/ceph/ceph-client.admin.52092.log 2024-05-26T22:57:44.934 INFO:teuthology.orchestra.run.smithi131.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.56395.log.gz 2024-05-26T22:57:44.935 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50801.log 2024-05-26T22:57:44.936 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.52092.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47006.log 2024-05-26T22:57:44.937 INFO:teuthology.orchestra.run.smithi131.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52092.log.gz 2024-05-26T22:57:44.938 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.50801.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50801.log.gz 2024-05-26T22:57:44.939 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58305.log 2024-05-26T22:57:44.939 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47006.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47006.log.gz 2024-05-26T22:57:44.941 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74812.log 2024-05-26T22:57:44.941 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.58305.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.58305.log.gz 2024-05-26T22:57:44.943 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74858.log 2024-05-26T22:57:44.943 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.74812.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.74812.log.gz 2024-05-26T22:57:44.944 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68577.log 2024-05-26T22:57:44.945 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.74858.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.74858.log.gz 2024-05-26T22:57:44.946 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54466.log 2024-05-26T22:57:44.947 INFO:teuthology.orchestra.run.smithi187.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-05-26T22:57:44.948 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.68577.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.68577.log.gz 2024-05-26T22:57:44.949 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60468.log 2024-05-26T22:57:44.950 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.54466.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.54466.log.gz 2024-05-26T22:57:44.951 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5/var/log/ceph/ceph-client.admin.60468.log: --verbose -- /var/log/ceph/ceph-client.admin.62005.log 2024-05-26T22:57:44.952 INFO:teuthology.orchestra.run.smithi131.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.60468.log.gz 2024-05-26T22:57:44.953 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46685.log 2024-05-26T22:57:44.955 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.62005.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.62005.log.gz 2024-05-26T22:57:44.955 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63053.log 2024-05-26T22:57:44.955 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46685.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46685.log.gz 2024-05-26T22:57:44.956 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76959.log 2024-05-26T22:57:44.957 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.63053.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.63053.log.gz 2024-05-26T22:57:44.958 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44889.log 2024-05-26T22:57:44.960 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.76959.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.76959.log.gz 2024-05-26T22:57:44.960 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69920.log 2024-05-26T22:57:44.961 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.44889.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44889.log.gz 2024-05-26T22:57:44.962 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60038.log 2024-05-26T22:57:44.962 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.69920.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.69920.log.gz 2024-05-26T22:57:44.964 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.60038.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.60038.log.gz 2024-05-26T22:57:44.965 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52601.log 2024-05-26T22:57:44.966 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68264.log 2024-05-26T22:57:44.968 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.52601.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52601.log.gz 2024-05-26T22:57:44.968 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.66193.log 2024-05-26T22:57:44.969 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.68264.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.68264.log.gz 2024-05-26T22:57:44.970 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76187.log 2024-05-26T22:57:44.971 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.0.66193.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.66193.log.gz 2024-05-26T22:57:44.972 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49221.log 2024-05-26T22:57:44.973 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.76187.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.76187.log.gz 2024-05-26T22:57:44.974 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73158.log 2024-05-26T22:57:44.975 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.49221.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.49221.log.gz 2024-05-26T22:57:44.975 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48746.log 2024-05-26T22:57:44.978 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.73158.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.73158.log.gz 2024-05-26T22:57:44.978 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66272.log 2024-05-26T22:57:44.979 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48746.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48746.log.gz 2024-05-26T22:57:44.979 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.45236.log 2024-05-26T22:57:44.980 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.66272.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.66272.log.gz 2024-05-26T22:57:44.981 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77211.log 2024-05-26T22:57:44.982 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.0.45236.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.45236.log.gz 2024-05-26T22:57:44.983 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43029.log 2024-05-26T22:57:44.983 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.77211.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.77211.log.gz 2024-05-26T22:57:44.984 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72505.log 2024-05-26T22:57:44.986 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.43029.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43029.log.gz 2024-05-26T22:57:44.987 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60164.log 2024-05-26T22:57:44.989 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.72505.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.72505.log.gz 2024-05-26T22:57:44.989 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60300.log 2024-05-26T22:57:44.990 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.60164.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.60164.log.gz 2024-05-26T22:57:44.990 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.49857.log 2024-05-26T22:57:44.992 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.60300.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.60300.log.gz 2024-05-26T22:57:44.992 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43877.log 2024-05-26T22:57:44.994 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.0.49857.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.49857.log.gz 2024-05-26T22:57:44.994 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53398.log 2024-05-26T22:57:44.996 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.43877.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43877.log.gz 2024-05-26T22:57:44.996 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65249.log 2024-05-26T22:57:44.997 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.53398.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.53398.log.gz 2024-05-26T22:57:44.999 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.65249.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.65249.log.gz 2024-05-26T22:57:44.999 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50020.log 2024-05-26T22:57:45.001 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- 91.6% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-05-26T22:57:45.001 INFO:teuthology.orchestra.run.smithi131.stderr: /var/log/ceph/ceph-client.admin.49040.log 2024-05-26T22:57:45.002 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.50020.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50020.log.gz 2024-05-26T22:57:45.003 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-05-26T22:57:45.003 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.49040.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49040.log.gz 2024-05-26T22:57:45.004 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62173.log 2024-05-26T22:57:45.005 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68889.log 2024-05-26T22:57:45.007 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.62173.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.62173.log.gz 2024-05-26T22:57:45.008 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.68889.log: -- /var/log/ceph/ceph-client.admin.67876.log 2024-05-26T22:57:45.008 INFO:teuthology.orchestra.run.smithi131.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.68889.log.gz 2024-05-26T22:57:45.010 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60796.log 2024-05-26T22:57:45.010 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.67876.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.67876.log.gz 2024-05-26T22:57:45.011 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69110.log 2024-05-26T22:57:45.012 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.60796.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.60796.log.gz 2024-05-26T22:57:45.014 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57744.log/var/log/ceph/ceph-client.admin.69110.log: 2024-05-26T22:57:45.014 INFO:teuthology.orchestra.run.smithi131.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.69110.log.gz 2024-05-26T22:57:45.015 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77043.log 2024-05-26T22:57:45.016 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.57744.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.57744.log.gz 2024-05-26T22:57:45.017 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73423.log 2024-05-26T22:57:45.018 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.77043.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.77043.log.gz 2024-05-26T22:57:45.019 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61963.log 2024-05-26T22:57:45.021 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.73423.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.73423.log.gz 2024-05-26T22:57:45.021 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73372.log/var/log/ceph/ceph-client.admin.61963.log: 2024-05-26T22:57:45.021 INFO:teuthology.orchestra.run.smithi131.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.61963.log.gz 2024-05-26T22:57:45.023 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77337.log 2024-05-26T22:57:45.024 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.73372.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.73372.log.gz 2024-05-26T22:57:45.025 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5/var/log/ceph/ceph-client.admin.77337.log: --verbose -- /var/log/ceph/ceph-client.admin.68847.log 2024-05-26T22:57:45.026 INFO:teuthology.orchestra.run.smithi131.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.77337.log.gz 2024-05-26T22:57:45.026 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74743.log 2024-05-26T22:57:45.028 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.68847.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.68847.log.gz 2024-05-26T22:57:45.028 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46601.log 2024-05-26T22:57:45.030 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.74743.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.74743.log.gz 2024-05-26T22:57:45.031 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75751.log 2024-05-26T22:57:45.032 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46601.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46601.log.gz 2024-05-26T22:57:45.032 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43589.log/var/log/ceph/ceph-client.admin.75751.log: 2024-05-26T22:57:45.033 INFO:teuthology.orchestra.run.smithi131.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.75751.log.gz 2024-05-26T22:57:45.034 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57576.log 2024-05-26T22:57:45.035 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.43589.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43589.log.gz 2024-05-26T22:57:45.036 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.57576.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.57576.log.gz 2024-05-26T22:57:45.037 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45971.log 2024-05-26T22:57:45.038 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51196.log 2024-05-26T22:57:45.040 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.45971.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45971.log.gz 2024-05-26T22:57:45.041 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.51196.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51196.log.gz 2024-05-26T22:57:45.042 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60206.log 2024-05-26T22:57:45.043 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48956.log 2024-05-26T22:57:45.044 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.60206.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.60206.log.gz 2024-05-26T22:57:45.045 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68138.log 2024-05-26T22:57:45.046 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48956.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48956.log.gz 2024-05-26T22:57:45.047 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56769.log 2024-05-26T22:57:45.048 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.68138.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.68138.log.gz 2024-05-26T22:57:45.049 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72421.log 2024-05-26T22:57:45.051 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.56769.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.56769.log.gz 2024-05-26T22:57:45.051 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47221.log 2024-05-26T22:57:45.052 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.72421.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.72421.log.gz 2024-05-26T22:57:45.053 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47221.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47221.log.gz 2024-05-26T22:57:45.054 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48458.log 2024-05-26T22:57:45.055 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69271.log 2024-05-26T22:57:45.057 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48458.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48458.log.gz 2024-05-26T22:57:45.057 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64807.log 2024-05-26T22:57:45.058 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.69271.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.69271.log.gz 2024-05-26T22:57:45.059 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42903.log 2024-05-26T22:57:45.061 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.64807.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.64807.log.gz 2024-05-26T22:57:45.061 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66967.log 2024-05-26T22:57:45.063 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.42903.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.42903.log.gz 2024-05-26T22:57:45.064 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62633.log 2024-05-26T22:57:45.065 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.66967.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.66967.log.gz 2024-05-26T22:57:45.065 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73913.log 2024-05-26T22:57:45.070 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.62633.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.62633.log.gz 2024-05-26T22:57:45.070 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48872.log 2024-05-26T22:57:45.072 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.73913.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.73913.log.gz 2024-05-26T22:57:45.073 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53290.log 2024-05-26T22:57:45.073 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48872.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48872.log.gz 2024-05-26T22:57:45.073 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44709.log 2024-05-26T22:57:45.073 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.53290.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.53290.log.gz 2024-05-26T22:57:45.073 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43071.log 2024-05-26T22:57:45.075 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.44709.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44709.log.gz 2024-05-26T22:57:45.075 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65645.log 2024-05-26T22:57:45.076 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.43071.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43071.log.gz 2024-05-26T22:57:45.076 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57870.log 2024-05-26T22:57:45.078 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.65645.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.65645.log.gz 2024-05-26T22:57:45.078 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52824.log 2024-05-26T22:57:45.079 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.57870.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.57870.log.gz 2024-05-26T22:57:45.080 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63287.log 2024-05-26T22:57:45.081 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.52824.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52824.log.gz 2024-05-26T22:57:45.082 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72066.log 2024-05-26T22:57:45.083 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.63287.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.63287.log.gz 2024-05-26T22:57:45.084 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.72066.log: -- /var/log/ceph/ceph-client.admin.71678.log 2024-05-26T22:57:45.085 INFO:teuthology.orchestra.run.smithi131.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.72066.log.gz 2024-05-26T22:57:45.085 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61196.log 2024-05-26T22:57:45.087 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.71678.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.71678.log.gz 2024-05-26T22:57:45.088 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.61196.log: /var/log/ceph/ceph-client.admin.56017.log 2024-05-26T22:57:45.088 INFO:teuthology.orchestra.run.smithi131.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.61196.log.gz 2024-05-26T22:57:45.089 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69579.log 2024-05-26T22:57:45.091 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.56017.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.56017.log.gz 2024-05-26T22:57:45.092 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48704.log 2024-05-26T22:57:45.092 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.69579.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.69579.log.gz 2024-05-26T22:57:45.093 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44247.log 2024-05-26T22:57:45.094 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48704.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48704.log.gz 2024-05-26T22:57:45.096 INFO:teuthology.orchestra.run.smithi131.stderr:gzip/var/log/ceph/ceph-client.admin.44247.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.59467.log 2024-05-26T22:57:45.096 INFO:teuthology.orchestra.run.smithi131.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44247.log.gz 2024-05-26T22:57:45.097 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48164.log 2024-05-26T22:57:45.099 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.59467.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.59467.log.gz 2024-05-26T22:57:45.099 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57492.log 2024-05-26T22:57:45.101 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48164.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48164.log.gz 2024-05-26T22:57:45.102 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57972.log 2024-05-26T22:57:45.102 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.57492.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.57492.log.gz 2024-05-26T22:57:45.103 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73637.log 2024-05-26T22:57:45.104 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.57972.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.57972.log.gz 2024-05-26T22:57:45.105 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.73637.log: /var/log/ceph/ceph-client.admin.70004.log 2024-05-26T22:57:45.106 INFO:teuthology.orchestra.run.smithi131.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.73637.log.gz 2024-05-26T22:57:45.107 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59512.log 2024-05-26T22:57:45.109 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.70004.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.70004.log.gz 2024-05-26T22:57:45.109 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.59512.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43252.log 2024-05-26T22:57:45.110 INFO:teuthology.orchestra.run.smithi131.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.59512.log.gz 2024-05-26T22:57:45.111 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63751.log 2024-05-26T22:57:45.112 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.43252.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43252.log.gz 2024-05-26T22:57:45.113 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69663.log 2024-05-26T22:57:45.114 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.63751.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.63751.log.gz 2024-05-26T22:57:45.116 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.69663.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.69663.log.gz 2024-05-26T22:57:45.116 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72589.log 2024-05-26T22:57:45.118 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43588.log 2024-05-26T22:57:45.119 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.72589.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.72589.log.gz 2024-05-26T22:57:45.120 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42651.log 2024-05-26T22:57:45.121 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.43588.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43588.log.gz 2024-05-26T22:57:45.121 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69356.log 2024-05-26T22:57:45.123 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.42651.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42651.log.gz 2024-05-26T22:57:45.124 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43570.log 2024-05-26T22:57:45.125 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.69356.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.69356.log.gz 2024-05-26T22:57:45.125 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77295.log 2024-05-26T22:57:45.126 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.43570.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43570.log.gz 2024-05-26T22:57:45.127 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50616.log 2024-05-26T22:57:45.128 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.77295.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.77295.log.gz 2024-05-26T22:57:45.129 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56227.log 2024-05-26T22:57:45.131 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.50616.log: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.50616.log.gz 2024-05-26T22:57:45.131 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.56227.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61375.log 2024-05-26T22:57:45.132 INFO:teuthology.orchestra.run.smithi131.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.56227.log.gz 2024-05-26T22:57:45.133 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54932.log 2024-05-26T22:57:45.134 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.61375.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.61375.log.gz 2024-05-26T22:57:45.135 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64055.log 2024-05-26T22:57:45.136 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.54932.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.54932.log.gz 2024-05-26T22:57:45.137 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72775.log 2024-05-26T22:57:45.139 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.64055.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.64055.log.gz 2024-05-26T22:57:45.139 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68390.log 2024-05-26T22:57:45.140 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.72775.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.72775.log.gz 2024-05-26T22:57:45.142 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.68390.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72234.log 2024-05-26T22:57:45.142 INFO:teuthology.orchestra.run.smithi131.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.68390.log.gz 2024-05-26T22:57:45.143 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60842.log 2024-05-26T22:57:45.145 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.72234.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.72234.log.gz 2024-05-26T22:57:45.145 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52645.log 2024-05-26T22:57:45.146 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.60842.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.60842.log.gz 2024-05-26T22:57:45.147 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46922.log 2024-05-26T22:57:45.148 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.52645.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52645.log.gz 2024-05-26T22:57:45.149 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44625.log 2024-05-26T22:57:45.150 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46922.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46922.log.gz 2024-05-26T22:57:45.151 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.44625.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44625.log.gz 2024-05-26T22:57:45.152 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68679.log 2024-05-26T22:57:45.153 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64555.log 2024-05-26T22:57:45.155 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.68679.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.68679.log.gz 2024-05-26T22:57:45.156 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43925.log 2024-05-26T22:57:45.157 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.64555.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.64555.log.gz 2024-05-26T22:57:45.157 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75447.log 2024-05-26T22:57:45.159 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.43925.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43925.log.gz 2024-05-26T22:57:45.160 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5/var/log/ceph/ceph-client.admin.75447.log: --verbose -- /var/log/ceph/ceph-client.admin.72318.log 2024-05-26T22:57:45.160 INFO:teuthology.orchestra.run.smithi131.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.75447.log.gz 2024-05-26T22:57:45.161 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76825.log 2024-05-26T22:57:45.162 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.72318.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.72318.log.gz 2024-05-26T22:57:45.163 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46517.log/var/log/ceph/ceph-client.admin.76825.log: 2024-05-26T22:57:45.165 INFO:teuthology.orchestra.run.smithi131.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.76825.log.gz 2024-05-26T22:57:45.165 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73679.log 2024-05-26T22:57:45.166 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46517.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46517.log.gz 2024-05-26T22:57:45.167 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56059.log 2024-05-26T22:57:45.168 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.73679.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.73679.log.gz 2024-05-26T22:57:45.169 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60720.log 2024-05-26T22:57:45.171 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.56059.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.56059.log.gz 2024-05-26T22:57:45.171 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47996.log 2024-05-26T22:57:45.173 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.60720.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.60720.log.gz 2024-05-26T22:57:45.173 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49936.log 2024-05-26T22:57:45.174 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47996.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.47996.log.gz 2024-05-26T22:57:45.175 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.49936.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.49936.log.gz 2024-05-26T22:57:45.176 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48661.log 2024-05-26T22:57:45.177 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71804.log 2024-05-26T22:57:45.179 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48661.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48661.log.gz 2024-05-26T22:57:45.179 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66314.log 2024-05-26T22:57:45.180 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.71804.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.71804.log.gz 2024-05-26T22:57:45.182 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.66314.log: /var/log/ceph/ceph-client.admin.54382.log 2024-05-26T22:57:45.182 INFO:teuthology.orchestra.run.smithi131.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.66314.log.gz 2024-05-26T22:57:45.183 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72673.log 2024-05-26T22:57:45.184 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.54382.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.54382.log.gz 2024-05-26T22:57:45.185 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57660.log/var/log/ceph/ceph-client.admin.72673.log: 2024-05-26T22:57:45.187 INFO:teuthology.orchestra.run.smithi131.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.72673.log.gz 2024-05-26T22:57:45.187 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49267.log 2024-05-26T22:57:45.188 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.57660.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.57660.log.gz 2024-05-26T22:57:45.189 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.49267.log: /var/log/ceph/ceph-client.admin.56727.log 2024-05-26T22:57:45.190 INFO:teuthology.orchestra.run.smithi131.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.49267.log.gz 2024-05-26T22:57:45.191 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.56727.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.56727.log.gz 2024-05-26T22:57:45.192 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58038.log 2024-05-26T22:57:45.193 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46139.log 2024-05-26T22:57:45.195 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.58038.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.58038.log.gz 2024-05-26T22:57:45.195 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77085.log 2024-05-26T22:57:45.197 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46139.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46139.log.gz 2024-05-26T22:57:45.197 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43242.log 2024-05-26T22:57:45.198 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.77085.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.77085.log.gz 2024-05-26T22:57:45.199 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71982.log 2024-05-26T22:57:45.200 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.43242.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43242.log.gz 2024-05-26T22:57:45.201 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65126.log 2024-05-26T22:57:45.202 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.71982.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.71982.log.gz 2024-05-26T22:57:45.203 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.65126.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.65126.log.gz 2024-05-26T22:57:45.204 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61705.log 2024-05-26T22:57:45.206 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48416.log 2024-05-26T22:57:45.207 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.61705.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.61705.log.gz 2024-05-26T22:57:45.207 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52134.log 2024-05-26T22:57:45.209 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48416.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48416.log.gz 2024-05-26T22:57:45.210 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53164.log 2024-05-26T22:57:45.211 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.52134.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52134.log.gz 2024-05-26T22:57:45.211 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61578.log 2024-05-26T22:57:45.212 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.53164.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53164.log.gz 2024-05-26T22:57:45.213 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71089.log 2024-05-26T22:57:45.214 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.61578.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.61578.log.gz 2024-05-26T22:57:45.216 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.71089.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.71089.log.gz 2024-05-26T22:57:45.216 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64765.log 2024-05-26T22:57:45.218 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64441.log 2024-05-26T22:57:45.219 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.64765.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.64765.log.gz 2024-05-26T22:57:45.219 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60510.log 2024-05-26T22:57:45.221 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.64441.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.64441.log.gz 2024-05-26T22:57:45.222 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47810.log/var/log/ceph/ceph-client.admin.60510.log: 2024-05-26T22:57:45.222 INFO:teuthology.orchestra.run.smithi131.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.60510.log.gz 2024-05-26T22:57:45.224 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47810.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72954.log 2024-05-26T22:57:45.224 INFO:teuthology.orchestra.run.smithi131.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47810.log.gz 2024-05-26T22:57:45.225 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61455.log 2024-05-26T22:57:45.227 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.72954.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.72954.log.gz 2024-05-26T22:57:45.227 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75793.log 2024-05-26T22:57:45.228 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.61455.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.61455.log.gz 2024-05-26T22:57:45.229 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.62511.log 2024-05-26T22:57:45.230 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.75793.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.75793.log.gz 2024-05-26T22:57:45.231 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5/var/log/ceph/ceph-client.0.62511.log: --verbose -- /var/log/ceph/ceph-client.admin.46769.log 2024-05-26T22:57:45.231 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.62511.log.gz 2024-05-26T22:57:45.232 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64349.log 2024-05-26T22:57:45.234 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46769.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46769.log.gz 2024-05-26T22:57:45.237 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.64349.log: /var/log/ceph/ceph-client.admin.50439.log 2024-05-26T22:57:45.237 INFO:teuthology.orchestra.run.smithi131.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.64349.log.gz 2024-05-26T22:57:45.237 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42133.log 2024-05-26T22:57:45.237 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.50439.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50439.log.gz 2024-05-26T22:57:45.238 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52432.log 2024-05-26T22:57:45.239 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.42133.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42133.log.gz 2024-05-26T22:57:45.240 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65557.log 2024-05-26T22:57:45.241 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.52432.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.52432.log.gz 2024-05-26T22:57:45.242 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.65557.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.65557.log.gz 2024-05-26T22:57:45.243 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51102.log 2024-05-26T22:57:45.244 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53665.log 2024-05-26T22:57:45.245 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.51102.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51102.log.gz 2024-05-26T22:57:45.246 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76251.log 2024-05-26T22:57:45.247 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.53665.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53665.log.gz 2024-05-26T22:57:45.248 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42395.log 2024-05-26T22:57:45.249 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.76251.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.76251.log.gz 2024-05-26T22:57:45.250 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50328.log 2024-05-26T22:57:45.250 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.42395.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.42395.log.gz 2024-05-26T22:57:45.251 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61663.log 2024-05-26T22:57:45.253 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.50328.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.50328.log.gz 2024-05-26T22:57:45.254 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.61663.log: /var/log/ceph/ceph-client.admin.45407.log 2024-05-26T22:57:45.254 INFO:teuthology.orchestra.run.smithi131.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.61663.log.gz 2024-05-26T22:57:45.255 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55378.log 2024-05-26T22:57:45.256 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.45407.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45407.log.gz 2024-05-26T22:57:45.257 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.55378.log: /var/log/ceph/ceph-client.admin.64723.log 2024-05-26T22:57:45.258 INFO:teuthology.orchestra.run.smithi131.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.55378.log.gz 2024-05-26T22:57:45.259 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54532.log 2024-05-26T22:57:45.260 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.64723.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.64723.log.gz 2024-05-26T22:57:45.261 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61154.log 2024-05-26T22:57:45.262 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.54532.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.54532.log.gz 2024-05-26T22:57:45.263 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75032.log/var/log/ceph/ceph-client.admin.61154.log: 2024-05-26T22:57:45.263 INFO:teuthology.orchestra.run.smithi131.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.61154.log.gz 2024-05-26T22:57:45.264 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72547.log 2024-05-26T22:57:45.266 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.75032.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.75032.log.gz 2024-05-26T22:57:45.267 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.72547.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64307.log 2024-05-26T22:57:45.267 INFO:teuthology.orchestra.run.smithi131.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.72547.log.gz 2024-05-26T22:57:45.268 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63793.log 2024-05-26T22:57:45.271 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.64307.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.64307.log.gz 2024-05-26T22:57:45.271 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72024.log 2024-05-26T22:57:45.274 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.63793.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.63793.log.gz 2024-05-26T22:57:45.274 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.72024.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.72024.log.gz 2024-05-26T22:57:45.274 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73465.log 2024-05-26T22:57:45.275 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46433.log 2024-05-26T22:57:45.276 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.73465.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.73465.log.gz 2024-05-26T22:57:45.277 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52954.log 2024-05-26T22:57:45.278 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46433.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46433.log.gz 2024-05-26T22:57:45.278 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55201.log 2024-05-26T22:57:45.281 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.52954.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52954.log.gz 2024-05-26T22:57:45.281 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5/var/log/ceph/ceph-client.admin.55201.log: --verbose -- /var/log/ceph/ceph-client.admin.75835.log 2024-05-26T22:57:45.281 INFO:teuthology.orchestra.run.smithi131.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.55201.log.gz 2024-05-26T22:57:45.283 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75489.log 2024-05-26T22:57:45.284 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.75835.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.75835.log.gz 2024-05-26T22:57:45.285 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.75489.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48332.log 2024-05-26T22:57:45.285 INFO:teuthology.orchestra.run.smithi131.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.75489.log.gz 2024-05-26T22:57:45.286 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57277.log 2024-05-26T22:57:45.288 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48332.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48332.log.gz 2024-05-26T22:57:45.288 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65939.log 2024-05-26T22:57:45.289 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.57277.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.57277.log.gz 2024-05-26T22:57:45.290 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65376.log 2024-05-26T22:57:45.292 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.65939.log: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.65939.log.gz 2024-05-26T22:57:45.292 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.65376.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64139.log 2024-05-26T22:57:45.292 INFO:teuthology.orchestra.run.smithi131.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.65376.log.gz 2024-05-26T22:57:45.294 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69398.log 2024-05-26T22:57:45.295 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.64139.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.64139.log.gz 2024-05-26T22:57:45.296 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76087.log 2024-05-26T22:57:45.297 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.69398.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.69398.log.gz 2024-05-26T22:57:45.298 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61094.log 2024-05-26T22:57:45.299 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.76087.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.76087.log.gz 2024-05-26T22:57:45.300 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69528.log/var/log/ceph/ceph-client.admin.61094.log: 2024-05-26T22:57:45.301 INFO:teuthology.orchestra.run.smithi131.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.61094.log.gz 2024-05-26T22:57:45.301 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43586.log 2024-05-26T22:57:45.303 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.69528.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.69528.log.gz 2024-05-26T22:57:45.303 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47954.log 2024-05-26T22:57:45.304 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.43586.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43586.log.gz 2024-05-26T22:57:45.305 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45887.log 2024-05-26T22:57:45.306 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47954.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47954.log.gz 2024-05-26T22:57:45.307 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76737.log 2024-05-26T22:57:45.308 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.45887.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45887.log.gz 2024-05-26T22:57:45.309 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.76737.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.76737.log.gz 2024-05-26T22:57:45.310 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43573.log 2024-05-26T22:57:45.311 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76045.log 2024-05-26T22:57:45.312 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.43573.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43573.log.gz 2024-05-26T22:57:45.313 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60342.log 2024-05-26T22:57:45.315 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.76045.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.76045.log.gz 2024-05-26T22:57:45.315 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49351.log 2024-05-26T22:57:45.316 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.60342.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.60342.log.gz 2024-05-26T22:57:45.317 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.49351.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49351.log.gz 2024-05-26T22:57:45.318 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42479.log 2024-05-26T22:57:45.319 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72192.log 2024-05-26T22:57:45.320 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.42479.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42479.log.gz 2024-05-26T22:57:45.321 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56853.log 2024-05-26T22:57:45.323 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.72192.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.72192.log.gz 2024-05-26T22:57:45.323 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60678.log 2024-05-26T22:57:45.324 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.56853.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.56853.log.gz 2024-05-26T22:57:45.325 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.60678.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.60678.log.gz 2024-05-26T22:57:45.326 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60968.log 2024-05-26T22:57:45.327 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60080.log 2024-05-26T22:57:45.328 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.60968.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.60968.log.gz 2024-05-26T22:57:45.329 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55462.log 2024-05-26T22:57:45.330 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.60080.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.60080.log.gz 2024-05-26T22:57:45.331 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70070.log 2024-05-26T22:57:45.332 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.55462.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.55462.log.gz 2024-05-26T22:57:45.333 INFO:teuthology.orchestra.run.smithi187.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-05-26T22:57:45.334 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.70070.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.70070.log.gz 2024-05-26T22:57:45.334 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50388.log 2024-05-26T22:57:45.335 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65813.log 2024-05-26T22:57:45.336 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.50388.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.50388.log.gz 2024-05-26T22:57:45.337 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75573.log 2024-05-26T22:57:45.338 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.65813.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.65813.log.gz 2024-05-26T22:57:45.339 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64097.log 2024-05-26T22:57:45.340 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.75573.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.75573.log.gz 2024-05-26T22:57:45.341 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64849.log 2024-05-26T22:57:45.341 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.64097.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.64097.log.gz 2024-05-26T22:57:45.343 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.45711.log 2024-05-26T22:57:45.344 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.64849.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.64849.log.gz 2024-05-26T22:57:45.344 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46097.log/var/log/ceph/ceph-client.2.45711.log: 2024-05-26T22:57:45.345 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.2.45711.log.gz 2024-05-26T22:57:45.346 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69962.log 2024-05-26T22:57:45.347 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46097.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46097.log.gz 2024-05-26T22:57:45.348 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65420.log 2024-05-26T22:57:45.349 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.69962.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.69962.log.gz 2024-05-26T22:57:45.349 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68432.log 2024-05-26T22:57:45.351 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.65420.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.65420.log.gz 2024-05-26T22:57:45.352 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50934.log 2024-05-26T22:57:45.353 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.68432.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.68432.log.gz 2024-05-26T22:57:45.354 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43243.log 2024-05-26T22:57:45.356 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.50934.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.50934.log.gz 2024-05-26T22:57:45.357 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.43243.log: -- /var/log/ceph/ceph-client.admin.61837.log 2024-05-26T22:57:45.357 INFO:teuthology.orchestra.run.smithi131.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43243.log.gz 2024-05-26T22:57:45.358 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77169.log 2024-05-26T22:57:45.358 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.61837.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.61837.log.gz 2024-05-26T22:57:45.358 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57235.log 2024-05-26T22:57:45.359 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.77169.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.77169.log.gz 2024-05-26T22:57:45.360 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48619.log 2024-05-26T22:57:45.361 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.57235.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.57235.log.gz 2024-05-26T22:57:45.362 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56559.log 2024-05-26T22:57:45.363 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48619.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48619.log.gz 2024-05-26T22:57:45.364 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68619.log 2024-05-26T22:57:45.365 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.56559.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.56559.log.gz 2024-05-26T22:57:45.366 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.68619.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.68619.log.gz 2024-05-26T22:57:45.367 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62323.log 2024-05-26T22:57:45.368 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46307.log 2024-05-26T22:57:45.369 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.62323.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.62323.log.gz 2024-05-26T22:57:45.370 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68180.log 2024-05-26T22:57:45.371 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46307.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46307.log.gz 2024-05-26T22:57:45.371 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49477.log 2024-05-26T22:57:45.373 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.68180.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.68180.log.gz 2024-05-26T22:57:45.374 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.49477.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49477.log.gz 2024-05-26T22:57:45.374 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43583.log 2024-05-26T22:57:45.376 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55159.log 2024-05-26T22:57:45.377 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.43583.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60594.log 2024-05-26T22:57:45.378 INFO:teuthology.orchestra.run.smithi131.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43583.log.gz 2024-05-26T22:57:45.380 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.55159.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.55159.log.gz 2024-05-26T22:57:45.380 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.60594.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51448.log 2024-05-26T22:57:45.380 INFO:teuthology.orchestra.run.smithi131.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.60594.log.gz 2024-05-26T22:57:45.381 INFO:teuthology.orchestra.run.smithi187.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-05-26T22:57:45.382 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68931.log 2024-05-26T22:57:45.383 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.51448.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51448.log.gz 2024-05-26T22:57:45.384 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.68931.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.68931.log.gz 2024-05-26T22:57:45.385 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51966.log 2024-05-26T22:57:45.386 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68348.log 2024-05-26T22:57:45.388 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.51966.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51966.log.gz 2024-05-26T22:57:45.388 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53038.log 2024-05-26T22:57:45.390 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.68348.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.68348.log.gz 2024-05-26T22:57:45.390 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42861.log 2024-05-26T22:57:45.392 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.53038.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53038.log.gz 2024-05-26T22:57:45.392 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.42861.log: /var/log/ceph/ceph-client.admin.56311.log 2024-05-26T22:57:45.393 INFO:teuthology.orchestra.run.smithi131.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.42861.log.gz 2024-05-26T22:57:45.394 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43249.log 2024-05-26T22:57:45.395 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.56311.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.56311.log.gz 2024-05-26T22:57:45.395 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47522.log 2024-05-26T22:57:45.396 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.43249.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43249.log.gz 2024-05-26T22:57:45.397 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47852.log 2024-05-26T22:57:45.398 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47522.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.47522.log.gz 2024-05-26T22:57:45.400 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5/var/log/ceph/ceph-client.admin.47852.log: --verbose -- /var/log/ceph/ceph-client.admin.52474.log 2024-05-26T22:57:45.400 INFO:teuthology.orchestra.run.smithi131.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47852.log.gz 2024-05-26T22:57:45.401 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49179.log 2024-05-26T22:57:45.402 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.52474.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52474.log.gz 2024-05-26T22:57:45.403 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67093.log 2024-05-26T22:57:45.404 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.49179.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49179.log.gz 2024-05-26T22:57:45.404 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65729.log 2024-05-26T22:57:45.406 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.67093.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.67093.log.gz 2024-05-26T22:57:45.407 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.65729.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-05-26T22:57:45.407 INFO:teuthology.orchestra.run.smithi131.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.65729.log.gz 2024-05-26T22:57:45.408 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67918.log 2024-05-26T22:57:45.411 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-mds.a.log: /var/log/ceph/ceph-client.admin.67918.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.67918.log.gz 2024-05-26T22:57:45.411 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44331.log 2024-05-26T22:57:45.414 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.44331.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62047.log 2024-05-26T22:57:45.415 INFO:teuthology.orchestra.run.smithi131.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.44331.log.gz 2024-05-26T22:57:45.416 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69621.log 2024-05-26T22:57:45.419 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.62047.log: 88.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56143.log 2024-05-26T22:57:45.419 INFO:teuthology.orchestra.run.smithi131.stderr: -- replaced with /var/log/ceph/ceph-client.admin.62047.log.gz 2024-05-26T22:57:45.421 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.69621.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.69621.log.gz 2024-05-26T22:57:45.422 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55064.log 2024-05-26T22:57:45.425 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.56143.log: 89.0%/var/log/ceph/ceph-client.admin.55064.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61052.log 2024-05-26T22:57:45.427 INFO:teuthology.orchestra.run.smithi131.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.55064.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53122.log 2024-05-26T22:57:45.427 INFO:teuthology.orchestra.run.smithi131.stderr: 2024-05-26T22:57:45.427 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.61052.log: -- replaced with /var/log/ceph/ceph-client.admin.56143.log.gz 2024-05-26T22:57:45.428 INFO:teuthology.orchestra.run.smithi131.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.61052.log.gz 2024-05-26T22:57:45.431 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.53122.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53122.log.gz 2024-05-26T22:57:45.431 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69752.log 2024-05-26T22:57:45.434 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.69752.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44541.log 2024-05-26T22:57:45.435 INFO:teuthology.orchestra.run.smithi131.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.69752.log.gz 2024-05-26T22:57:45.437 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44163.log 2024-05-26T22:57:45.441 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.44541.log: /var/log/ceph/ceph-client.admin.44163.log: gzip -5 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44541.log.gz 2024-05-26T22:57:45.442 INFO:teuthology.orchestra.run.smithi131.stderr: --verbose -- /var/log/ceph/ceph-client.admin.67009.log 2024-05-26T22:57:45.442 INFO:teuthology.orchestra.run.smithi131.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44163.log.gz 2024-05-26T22:57:45.443 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42352.log 2024-05-26T22:57:45.447 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.67009.log: /var/log/ceph/ceph-client.admin.42352.log: 87.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47348.log 2024-05-26T22:57:45.448 INFO:teuthology.orchestra.run.smithi131.stderr: -- replaced with /var/log/ceph/ceph-client.admin.67009.log.gz 2024-05-26T22:57:45.450 INFO:teuthology.orchestra.run.smithi131.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.42352.log.gz 2024-05-26T22:57:45.450 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72859.log 2024-05-26T22:57:45.454 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47348.log: /var/log/ceph/ceph-client.admin.72859.log: 89.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75709.log 2024-05-26T22:57:45.455 INFO:teuthology.orchestra.run.smithi131.stderr: -- replaced with /var/log/ceph/ceph-client.admin.47348.log.gz 2024-05-26T22:57:45.456 INFO:teuthology.orchestra.run.smithi131.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.72859.log.gz 2024-05-26T22:57:45.457 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74264.log 2024-05-26T22:57:45.462 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.75709.log: /var/log/ceph/ceph-client.admin.74264.log: 88.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49669.log 2024-05-26T22:57:45.462 INFO:teuthology.orchestra.run.smithi131.stderr: -- replaced with /var/log/ceph/ceph-client.admin.75709.log.gz 2024-05-26T22:57:45.463 INFO:teuthology.orchestra.run.smithi131.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.74264.log.gz 2024-05-26T22:57:45.465 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5/var/log/ceph/ceph-client.admin.49669.log: --verbose -- /var/log/ceph/ceph-client.admin.73507.log 2024-05-26T22:57:45.469 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.73507.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.73507.log.gz 2024-05-26T22:57:45.470 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 89.0% --verbose -- /var/log/ceph/ceph-client.admin.53248.log 2024-05-26T22:57:45.472 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.53248.log: -- replaced with /var/log/ceph/ceph-client.admin.49669.log.gz 2024-05-26T22:57:45.473 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46265.log 2024-05-26T22:57:45.474 INFO:teuthology.orchestra.run.smithi131.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.53248.log.gz 2024-05-26T22:57:45.474 INFO:teuthology.orchestra.run.smithi131.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-05-26T22:57:45.475 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48538.log 2024-05-26T22:57:45.477 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46265.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46265.log.gz 2024-05-26T22:57:45.478 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48538.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62215.log 2024-05-26T22:57:45.478 INFO:teuthology.orchestra.run.smithi131.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48538.log.gz 2024-05-26T22:57:45.479 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56895.log 2024-05-26T22:57:45.483 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.62215.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.62215.log.gz 2024-05-26T22:57:45.484 INFO:teuthology.orchestra.run.smithi131.stderr:gzip/var/log/ceph/ceph-client.admin.56895.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.66005.log 2024-05-26T22:57:45.484 INFO:teuthology.orchestra.run.smithi131.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.56895.log.gz 2024-05-26T22:57:45.484 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37217.log 2024-05-26T22:57:45.487 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.66005.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.66005.log.gz 2024-05-26T22:57:45.487 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.37217.log: -- /var/log/ceph/ceph-client.admin.51406.log 2024-05-26T22:57:45.487 INFO:teuthology.orchestra.run.smithi131.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.37217.log.gz 2024-05-26T22:57:45.487 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58347.log 2024-05-26T22:57:45.488 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.51406.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.51406.log.gz 2024-05-26T22:57:45.488 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43843.log 2024-05-26T22:57:45.489 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.58347.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.58347.log.gz 2024-05-26T22:57:45.490 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55881.log 2024-05-26T22:57:45.491 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.43843.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.43843.log.gz 2024-05-26T22:57:45.492 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76695.log 2024-05-26T22:57:45.493 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.55881.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55881.log.gz 2024-05-26T22:57:45.494 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77608.log 2024-05-26T22:57:45.495 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.76695.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.76695.log.gz 2024-05-26T22:57:45.496 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47048.log 2024-05-26T22:57:45.498 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.77608.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.77608.log.gz 2024-05-26T22:57:45.498 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42023.log 2024-05-26T22:57:45.499 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47048.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.47048.log.gz 2024-05-26T22:57:45.499 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63971.log 2024-05-26T22:57:45.501 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.42023.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42023.log.gz 2024-05-26T22:57:45.502 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71720.log 2024-05-26T22:57:45.502 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.63971.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.63971.log.gz 2024-05-26T22:57:45.504 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43923.log 2024-05-26T22:57:45.505 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.71720.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.71720.log.gz 2024-05-26T22:57:45.505 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52008.log 2024-05-26T22:57:45.506 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.43923.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43923.log.gz 2024-05-26T22:57:45.507 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47480.log 2024-05-26T22:57:45.508 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.52008.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.52008.log.gz 2024-05-26T22:57:45.509 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63002.log 2024-05-26T22:57:45.510 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47480.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47480.log.gz 2024-05-26T22:57:45.511 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73242.log 2024-05-26T22:57:45.512 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.63002.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.63002.log.gz 2024-05-26T22:57:45.513 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71636.log 2024-05-26T22:57:45.514 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.73242.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.73242.log.gz 2024-05-26T22:57:45.514 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54598.log 2024-05-26T22:57:45.516 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.71636.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.71636.log.gz 2024-05-26T22:57:45.516 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68763.log 2024-05-26T22:57:45.518 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.54598.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.54598.log.gz 2024-05-26T22:57:45.519 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46055.log 2024-05-26T22:57:45.519 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.68763.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.68763.log.gz 2024-05-26T22:57:45.520 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65599.log 2024-05-26T22:57:45.521 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46055.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46055.log.gz 2024-05-26T22:57:45.522 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63709.log 2024-05-26T22:57:45.523 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.65599.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.65599.log.gz 2024-05-26T22:57:45.524 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51322.log 2024-05-26T22:57:45.525 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.63709.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.63709.log.gz 2024-05-26T22:57:45.526 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52351.log 2024-05-26T22:57:45.527 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.51322.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.51322.log.gz 2024-05-26T22:57:45.528 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51574.log 2024-05-26T22:57:45.529 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.52351.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.52351.log.gz 2024-05-26T22:57:45.530 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54340.log 2024-05-26T22:57:45.531 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.51574.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.51574.log.gz 2024-05-26T22:57:45.532 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47726.log 2024-05-26T22:57:45.535 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.54340.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.54340.log.gz 2024-05-26T22:57:45.536 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45845.log/var/log/ceph/ceph-client.admin.47726.log: 2024-05-26T22:57:45.536 INFO:teuthology.orchestra.run.smithi131.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47726.log.gz 2024-05-26T22:57:45.536 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73284.log 2024-05-26T22:57:45.537 INFO:teuthology.orchestra.run.smithi187.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-05-26T22:57:45.540 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.45845.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.45845.log.gz 2024-05-26T22:57:45.540 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49393.log 2024-05-26T22:57:45.540 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.73284.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.73284.log.gz 2024-05-26T22:57:45.540 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48206.log 2024-05-26T22:57:45.541 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.49393.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49393.log.gz 2024-05-26T22:57:45.541 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48914.log 2024-05-26T22:57:45.542 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48206.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48206.log.gz 2024-05-26T22:57:45.543 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43246.log 2024-05-26T22:57:45.544 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48914.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48914.log.gz 2024-05-26T22:57:45.545 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44205.log 2024-05-26T22:57:45.545 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.43246.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43246.log.gz 2024-05-26T22:57:45.546 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77648.log 2024-05-26T22:57:45.548 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.44205.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44205.log.gz 2024-05-26T22:57:45.548 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51616.log 2024-05-26T22:57:45.549 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.77648.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.77648.log.gz 2024-05-26T22:57:45.550 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74966.log 2024-05-26T22:57:45.551 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.51616.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51616.log.gz 2024-05-26T22:57:45.552 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75615.log 2024-05-26T22:57:45.554 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.74966.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.74966.log.gz 2024-05-26T22:57:45.554 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55755.log 2024-05-26T22:57:45.555 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.75615.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.75615.log.gz 2024-05-26T22:57:45.556 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44289.log 2024-05-26T22:57:45.557 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.55755.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.55755.log.gz 2024-05-26T22:57:45.558 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48122.log 2024-05-26T22:57:45.559 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.44289.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.44289.log.gz 2024-05-26T22:57:45.559 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49128.log 2024-05-26T22:57:45.561 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48122.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48122.log.gz 2024-05-26T22:57:45.562 INFO:teuthology.orchestra.run.smithi131.stderr:gzip/var/log/ceph/ceph-client.admin.49128.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.59578.log 2024-05-26T22:57:45.563 INFO:teuthology.orchestra.run.smithi131.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49128.log.gz 2024-05-26T22:57:45.563 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53749.log 2024-05-26T22:57:45.565 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.59578.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.59578.log.gz 2024-05-26T22:57:45.566 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.53749.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43899.log 2024-05-26T22:57:45.566 INFO:teuthology.orchestra.run.smithi131.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.53749.log.gz 2024-05-26T22:57:45.568 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.43899.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43899.log.gz 2024-05-26T22:57:45.569 INFO:teuthology.orchestra.run.smithi187.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-05-26T22:57:45.809 INFO:teuthology.orchestra.run.smithi131.stderr: 90.5% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-05-26T22:57:45.827 INFO:teuthology.orchestra.run.smithi131.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-05-26T22:57:45.854 INFO:teuthology.orchestra.run.smithi131.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-05-26T22:57:45.975 INFO:teuthology.orchestra.run.smithi131.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-05-26T22:57:47.806 INFO:teuthology.orchestra.run.smithi187.stderr: 93.6% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-05-26T22:57:47.810 INFO:teuthology.orchestra.run.smithi187.stderr: 2024-05-26T22:57:47.810 INFO:teuthology.orchestra.run.smithi187.stderr:real 0m3.261s 2024-05-26T22:57:47.811 INFO:teuthology.orchestra.run.smithi187.stderr:user 0m6.930s 2024-05-26T22:57:47.811 INFO:teuthology.orchestra.run.smithi187.stderr:sys 0m0.645s 2024-05-26T22:57:48.285 INFO:teuthology.orchestra.run.smithi131.stderr: 93.8% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-05-26T22:57:48.385 INFO:teuthology.orchestra.run.smithi131.stderr: 93.9% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-05-26T22:57:48.391 INFO:teuthology.orchestra.run.smithi131.stderr: 2024-05-26T22:57:48.391 INFO:teuthology.orchestra.run.smithi131.stderr:real 0m3.845s 2024-05-26T22:57:48.391 INFO:teuthology.orchestra.run.smithi131.stderr:user 0m10.744s 2024-05-26T22:57:48.391 INFO:teuthology.orchestra.run.smithi131.stderr:sys 0m2.462s 2024-05-26T22:57:48.392 INFO:tasks.ceph:Archiving logs... 2024-05-26T22:57:48.392 DEBUG:teuthology.misc:Transferring archived files from smithi131:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-05-26_21:24:02-fs-squid-distro-default-smithi/7727073/remote/smithi131/log 2024-05-26T22:57:48.393 DEBUG:teuthology.orchestra.run.smithi131:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-26T22:57:50.205 DEBUG:teuthology.misc:Transferring archived files from smithi187:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-05-26_21:24:02-fs-squid-distro-default-smithi/7727073/remote/smithi187/log 2024-05-26T22:57:50.205 DEBUG:teuthology.orchestra.run.smithi187:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-26T22:57:51.125 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-05-26T22:57:51.135 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-05-26T22:57:51.135 DEBUG:teuthology.orchestra.run.smithi131:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-26T22:57:51.143 DEBUG:teuthology.orchestra.run.smithi187:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-26T22:57:51.291 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-05-26T22:57:51.301 INFO:teuthology.task.clock:Checking final clock skew... 2024-05-26T22:57:51.301 DEBUG:teuthology.orchestra.run.smithi131:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-26T22:57:51.309 DEBUG:teuthology.orchestra.run.smithi187:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-26T22:57:51.357 INFO:teuthology.orchestra.run.smithi131.stderr:bash: line 1: ntpq: command not found 2024-05-26T22:57:51.365 INFO:teuthology.orchestra.run.smithi187.stderr:bash: line 1: ntpq: command not found 2024-05-26T22:57:51.367 INFO:teuthology.orchestra.run.smithi131.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-26T22:57:51.367 INFO:teuthology.orchestra.run.smithi131.stdout:=============================================================================== 2024-05-26T22:57:51.367 INFO:teuthology.orchestra.run.smithi131.stdout:^* hv01.front.sepia.ceph.com 3 6 377 54 +571us[ +570us] +/- 62ms 2024-05-26T22:57:51.367 INFO:teuthology.orchestra.run.smithi131.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 56 -695us[ -696us] +/- 63ms 2024-05-26T22:57:51.367 INFO:teuthology.orchestra.run.smithi131.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 55 +108us[ +107us] +/- 69ms 2024-05-26T22:57:51.368 INFO:teuthology.orchestra.run.smithi131.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-05-26T22:57:51.375 INFO:teuthology.orchestra.run.smithi187.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-26T22:57:51.375 INFO:teuthology.orchestra.run.smithi187.stdout:=============================================================================== 2024-05-26T22:57:51.375 INFO:teuthology.orchestra.run.smithi187.stdout:^* hv01.front.sepia.ceph.com 3 6 377 55 +569us[ +573us] +/- 62ms 2024-05-26T22:57:51.375 INFO:teuthology.orchestra.run.smithi187.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 55 -676us[ -672us] +/- 63ms 2024-05-26T22:57:51.375 INFO:teuthology.orchestra.run.smithi187.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 55 +130us[ +134us] +/- 69ms 2024-05-26T22:57:51.375 INFO:teuthology.orchestra.run.smithi187.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-05-26T22:57:51.376 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-05-26T22:57:51.392 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-05-26T22:57:51.392 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-05-26T22:57:51.415 DEBUG:teuthology.orchestra.run.smithi131:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-26T22:57:51.637 DEBUG:teuthology.orchestra.run.smithi187:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-26T22:57:51.829 DEBUG:teuthology.orchestra.run.smithi131:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-26T22:57:51.926 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-26T22:57:51.926 DEBUG:teuthology.orchestra.run.smithi187:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-26T22:57:52.016 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-26T22:57:52.016 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-05-26T22:57:52.027 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-05-26T22:57:52.069 INFO:teuthology.task.internal:Duration was 1170.903810 seconds 2024-05-26T22:57:52.070 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-05-26T22:57:52.093 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-05-26T22:57:52.094 DEBUG:teuthology.orchestra.run.smithi131:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-26T22:57:52.102 DEBUG:teuthology.orchestra.run.smithi187:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-26T22:57:52.234 INFO:teuthology.orchestra.run.smithi187.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-26T22:57:52.249 INFO:teuthology.orchestra.run.smithi131.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-26T22:57:52.510 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-05-26T22:57:52.510 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi131.front.sepia.ceph.com 2024-05-26T22:57:52.511 DEBUG:teuthology.orchestra.run.smithi131:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-26T22:57:52.572 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi187.front.sepia.ceph.com 2024-05-26T22:57:52.572 DEBUG:teuthology.orchestra.run.smithi187:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-26T22:57:52.655 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-05-26T22:57:52.655 DEBUG:teuthology.orchestra.run.smithi131:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-26T22:57:52.663 DEBUG:teuthology.orchestra.run.smithi187:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-26T22:57:52.745 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-05-26T22:57:52.746 DEBUG:teuthology.orchestra.run.smithi131:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-26T22:57:52.751 DEBUG:teuthology.orchestra.run.smithi187:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-26T22:57:52.939 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-05-26T22:57:52.950 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-05-26T22:57:52.951 DEBUG:teuthology.orchestra.run.smithi131:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-26T22:57:53.026 DEBUG:teuthology.orchestra.run.smithi187:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-26T22:57:53.114 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-05-26T22:57:53.125 DEBUG:teuthology.orchestra.run.smithi131:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-26T22:57:53.133 DEBUG:teuthology.orchestra.run.smithi187:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-26T22:57:53.228 INFO:teuthology.orchestra.run.smithi131.stdout:kernel.core_pattern = core 2024-05-26T22:57:53.232 INFO:teuthology.orchestra.run.smithi187.stdout:kernel.core_pattern = core 2024-05-26T22:57:53.273 DEBUG:teuthology.orchestra.run.smithi131:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-26T22:57:53.321 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-26T22:57:53.322 DEBUG:teuthology.orchestra.run.smithi187:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-26T22:57:53.378 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-26T22:57:53.378 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-05-26T22:57:53.389 INFO:teuthology.task.internal:Transferring archived files... 2024-05-26T22:57:53.389 DEBUG:teuthology.misc:Transferring archived files from smithi131:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-05-26_21:24:02-fs-squid-distro-default-smithi/7727073/remote/smithi131 2024-05-26T22:57:53.389 DEBUG:teuthology.orchestra.run.smithi131:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-26T22:57:53.481 DEBUG:teuthology.misc:Transferring archived files from smithi187:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-05-26_21:24:02-fs-squid-distro-default-smithi/7727073/remote/smithi187 2024-05-26T22:57:53.481 DEBUG:teuthology.orchestra.run.smithi187:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-26T22:57:53.576 INFO:teuthology.task.internal:Removing archive directory... 2024-05-26T22:57:53.576 DEBUG:teuthology.orchestra.run.smithi131:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-26T22:57:53.584 DEBUG:teuthology.orchestra.run.smithi187:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-26T22:57:53.637 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-05-26T22:57:53.648 INFO:teuthology.task.internal:Not uploading archives. 2024-05-26T22:57:53.648 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-05-26T22:57:53.678 INFO:teuthology.task.internal:Tidying up after the test... 2024-05-26T22:57:53.678 DEBUG:teuthology.orchestra.run.smithi131:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-26T22:57:53.685 DEBUG:teuthology.orchestra.run.smithi187:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-26T22:57:53.736 INFO:teuthology.orchestra.run.smithi131.stdout: 393249 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 26 22:57 /home/ubuntu/cephtest 2024-05-26T22:57:53.743 INFO:teuthology.orchestra.run.smithi187.stdout: 393940 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 26 22:57 /home/ubuntu/cephtest 2024-05-26T22:57:53.746 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-05-26T22:57:53.757 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-05-26T22:57:53.824 INFO:teuthology.run:Summary data: client.0-kernel-sha1: b9459a26b0888785d05c47deb35abc9d0354ab93 client.1-kernel-sha1: b9459a26b0888785d05c47deb35abc9d0354ab93 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/exports} duration: 1170.9038100242615 failure_reason: 'Test failure: test_ephemeral_pin_dist_override_pin (tasks.cephfs.test_exports.TestEphemeralPins)' flavor: default owner: scheduled_teuthology@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=d548d81f515d49148ccecd462d3a7664 status: fail success: false 2024-05-26T22:57:53.824 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-26T22:57:53.898 INFO:teuthology.run:FAIL